2023-10-18T10:14:08.642 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-18T10:14:08.643 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --archive /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772 --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/orig.config.yaml 2023-10-18T10:14:08.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T10:14:08.777 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772 branch: wip-batrick-testing-20231016.203825 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: pdonnell@redhat.com first_in_suite: false job_id: '7430772' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20231016.203825 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231016.203825 sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7703 sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb sleep_before_teardown: 0 subset: 153/256 suite: fs suite_branch: wip-batrick-testing-20231016.203825 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi125.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgPjGOGMEwwnQSgDNGZ8Aebsniz3u1sRO9Cd/9KcNxkMp6n9mP36ADoPQ1fAXDMOtBc/tTwQAGL4sR1kzdS10w= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbVFAJYQJyVQjPaD1xZQAnsbNlp7gdhA4zpfnjzKe/rNxup6kcxf7nwkxc83xyMAEE+KyWlyHHVeaQnT9zM0VA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_23:59:58 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-18T10:14:08.777 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa; will attempt to use it 2023-10-18T10:14:08.778 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks 2023-10-18T10:14:08.778 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-18T10:14:08.780 INFO:teuthology.task.internal:Checking packages... 2023-10-18T10:14:08.804 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f7420d7f8e55ceea786dcbf536c338ce920c21eb' 2023-10-18T10:14:08.804 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-18T10:14:08.804 INFO:teuthology.packaging:ref: None 2023-10-18T10:14:08.805 INFO:teuthology.packaging:tag: None 2023-10-18T10:14:08.805 INFO:teuthology.packaging:branch: wip-batrick-testing-20231016.203825 2023-10-18T10:14:08.805 INFO:teuthology.packaging:sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T10:14:08.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20231016.203825 2023-10-18T10:14:08.947 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6786.gf7420d7f 2023-10-18T10:14:08.948 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-18T10:14:08.954 INFO:teuthology.task.internal:no buildpackages task found 2023-10-18T10:14:08.954 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-18T10:14:08.965 INFO:teuthology.task.internal:Saving configuration 2023-10-18T10:14:08.983 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-18T10:14:08.989 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T10:14:09.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 10:09:09.842624', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgPjGOGMEwwnQSgDNGZ8Aebsniz3u1sRO9Cd/9KcNxkMp6n9mP36ADoPQ1fAXDMOtBc/tTwQAGL4sR1kzdS10w='} 2023-10-18T10:14:09.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 10:09:09.841463', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbVFAJYQJyVQjPaD1xZQAnsbNlp7gdhA4zpfnjzKe/rNxup6kcxf7nwkxc83xyMAEE+KyWlyHHVeaQnT9zM0VA='} 2023-10-18T10:14:09.034 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-18T10:14:09.041 INFO:teuthology.task.internal:roles: ubuntu@smithi125.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-18T10:14:09.041 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-18T10:14:09.041 INFO:teuthology.run_tasks:Running task console_log... 2023-10-18T10:14:09.109 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5a8f081b80>, signals=[15]) 2023-10-18T10:14:09.111 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-18T10:14:09.117 INFO:teuthology.task.internal:Opening connections... 2023-10-18T10:14:09.118 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:14:09.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:14:09.247 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:14:09.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:14:09.330 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-18T10:14:09.340 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2023-10-18T10:14:09.357 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2023-10-18T10:14:09.358 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-10-18T10:14:09.414 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2023-10-18T10:14:09.414 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="9" 2023-10-18T10:14:09.414 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2023-10-18T10:14:09.414 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2023-10-18T10:14:09.415 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="9" 2023-10-18T10:14:09.415 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el9" 2023-10-18T10:14:09.415 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-18T10:14:09.415 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-10-18T10:14:09.415 INFO:teuthology.orchestra.run.smithi125.stdout:LOGO="fedora-logo-icon" 2023-10-18T10:14:09.416 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-18T10:14:09.416 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2023-10-18T10:14:09.416 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T10:14:09.416 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-18T10:14:09.416 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-18T10:14:09.417 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-10-18T10:14:09.442 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2023-10-18T10:14:09.459 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2023-10-18T10:14:09.460 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2023-10-18T10:14:09.515 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2023-10-18T10:14:09.515 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2023-10-18T10:14:09.515 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2023-10-18T10:14:09.516 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2023-10-18T10:14:09.516 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2023-10-18T10:14:09.516 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2023-10-18T10:14:09.516 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-18T10:14:09.517 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2023-10-18T10:14:09.517 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2023-10-18T10:14:09.517 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-18T10:14:09.517 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2023-10-18T10:14:09.517 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T10:14:09.518 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-18T10:14:09.518 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-18T10:14:09.518 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2023-10-18T10:14:09.542 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-18T10:14:09.551 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-18T10:14:09.621 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-18T10:14:09.622 DEBUG:teuthology.orchestra.run.smithi125:> test '!' -e /home/ubuntu/cephtest 2023-10-18T10:14:09.625 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2023-10-18T10:14:09.641 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-18T10:14:09.647 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-18T10:14:09.648 DEBUG:teuthology.orchestra.run.smithi125:> test -z $(ls -A /var/lib/ceph) 2023-10-18T10:14:09.681 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2023-10-18T10:14:09.779 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-18T10:14:09.841 INFO:teuthology.run_tasks:Running task kernel... 2023-10-18T10:14:09.855 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-18T10:14:09.856 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-18T10:14:09.857 DEBUG:teuthology.orchestra.run.smithi125:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T10:14:09.857 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T10:14:09.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:14:09.874 DEBUG:teuthology.orchestra.run.smithi125:> uname -r 2023-10-18T10:14:09.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:14:09.875 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2023-10-18T10:14:09.930 INFO:teuthology.orchestra.run.smithi125.stdout:5.14.0-361.el9.x86_64 2023-10-18T10:14:09.931 INFO:teuthology.task.kernel:Running kernel on smithi125: 5.14.0-361.el9.x86_64 2023-10-18T10:14:09.932 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum install -y kernel 2023-10-18T10:14:09.933 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-361.el9.x86_64 2023-10-18T10:14:09.934 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-361.el9.x86_64 2023-10-18T10:14:09.934 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2023-10-18T10:14:12.255 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.3 MB 00:00 2023-10-18T10:14:12.350 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS-9 - AppStream 7.4 MB/s | 6.3 MB 00:00 2023-10-18T10:14:13.613 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2023-10-18T10:14:13.667 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2023-10-18T10:14:14.404 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 7.9 MB 00:00 2023-10-18T10:14:16.849 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 7.9 MB 00:02 2023-10-18T10:14:17.350 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 18 MB 00:01 2023-10-18T10:14:19.359 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 12 MB/s | 18 MB 00:01 2023-10-18T10:14:21.522 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 5.7 MB 00:00 2023-10-18T10:14:23.123 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 14 kB 00:00 2023-10-18T10:14:24.383 INFO:teuthology.orchestra.run.smithi125.stdout:Extra Packages for Enterprise Linux 16 MB/s | 19 MB 00:01 2023-10-18T10:14:24.626 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 5.7 MB 00:02 2023-10-18T10:14:26.093 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 14 kB 00:00 2023-10-18T10:14:26.794 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 32 MB/s | 19 MB 00:00 2023-10-18T10:14:28.907 INFO:teuthology.orchestra.run.smithi125.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-10-18T10:14:30.922 INFO:teuthology.orchestra.run.smithi125.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-18T10:14:30.923 INFO:teuthology.orchestra.run.smithi125.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-18T10:14:31.025 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-18T10:14:31.036 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-18T10:14:31.036 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-18T10:14:31.037 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-18T10:14:31.037 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-18T10:14:31.037 INFO:teuthology.orchestra.run.smithi125.stdout: kernel x86_64 5.14.0-376.el9 CentOS-BaseOS 5.3 M 2023-10-18T10:14:31.037 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-18T10:14:31.037 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-core x86_64 5.14.0-376.el9 CentOS-BaseOS 20 M 2023-10-18T10:14:31.038 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules x86_64 5.14.0-376.el9 CentOS-BaseOS 38 M 2023-10-18T10:14:31.038 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-core x86_64 5.14.0-376.el9 CentOS-BaseOS 32 M 2023-10-18T10:14:31.038 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:14:31.038 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-18T10:14:31.038 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2023-10-18T10:14:31.039 INFO:teuthology.orchestra.run.smithi125.stdout:Install 4 Packages 2023-10-18T10:14:31.039 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:14:31.040 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 95 M 2023-10-18T10:14:31.040 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 120 M 2023-10-18T10:14:31.040 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-18T10:14:31.286 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2023-10-18T10:14:31.784 INFO:teuthology.orchestra.run.smithi125.stdout:(1/4): kernel-5.14.0-376.el9.x86_64.rpm 7.1 MB/s | 5.3 MB 00:00 2023-10-18T10:14:33.300 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-18T10:14:33.301 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-18T10:14:33.388 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2023-10-18T10:14:33.400 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2023-10-18T10:14:33.400 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2023-10-18T10:14:33.400 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2023-10-18T10:14:33.401 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2023-10-18T10:14:33.401 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-376.el9 CentOS-BaseOS 5.3 M 2023-10-18T10:14:33.401 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2023-10-18T10:14:33.401 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core x86_64 5.14.0-376.el9 CentOS-BaseOS 20 M 2023-10-18T10:14:33.401 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules x86_64 5.14.0-376.el9 CentOS-BaseOS 38 M 2023-10-18T10:14:33.402 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core x86_64 5.14.0-376.el9 CentOS-BaseOS 32 M 2023-10-18T10:14:33.402 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:14:33.402 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2023-10-18T10:14:33.402 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2023-10-18T10:14:33.403 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2023-10-18T10:14:33.403 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:14:33.403 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 95 M 2023-10-18T10:14:33.404 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 120 M 2023-10-18T10:14:33.404 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2023-10-18T10:14:33.659 INFO:teuthology.orchestra.run.smithi125.stdout:(2/4): kernel-core-5.14.0-376.el9.x86_64.rpm 7.6 MB/s | 20 MB 00:02 2023-10-18T10:14:33.895 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): kernel-5.14.0-376.el9.x86_64.rpm 11 MB/s | 5.3 MB 00:00 2023-10-18T10:14:34.504 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): kernel-core-5.14.0-376.el9.x86_64.rpm 18 MB/s | 20 MB 00:01 2023-10-18T10:14:34.784 INFO:teuthology.orchestra.run.smithi125.stdout:(3/4): kernel-modules-core-5.14.0-376.el9.x86_6 11 MB/s | 32 MB 00:02 2023-10-18T10:14:35.321 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): kernel-modules-5.14.0-376.el9.x86_64.rpm 20 MB/s | 38 MB 00:01 2023-10-18T10:14:35.384 INFO:teuthology.orchestra.run.smithi125.stdout:(4/4): kernel-modules-5.14.0-376.el9.x86_64.rpm 8.6 MB/s | 38 MB 00:04 2023-10-18T10:14:35.387 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-18T10:14:35.388 INFO:teuthology.orchestra.run.smithi125.stdout:Total 22 MB/s | 95 MB 00:04 2023-10-18T10:14:35.388 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-18T10:14:35.704 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): kernel-modules-core-5.14.0-376.el9.x86_6 18 MB/s | 32 MB 00:01 2023-10-18T10:14:35.707 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2023-10-18T10:14:35.707 INFO:teuthology.orchestra.run.smithi139.stdout:Total 41 MB/s | 95 MB 00:02 2023-10-18T10:14:35.708 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2023-10-18T10:14:35.756 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-18T10:14:35.756 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-18T10:14:36.045 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2023-10-18T10:14:36.045 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2023-10-18T10:14:36.734 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-18T10:14:36.734 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-18T10:14:37.021 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2023-10-18T10:14:37.022 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2023-10-18T10:14:39.089 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-18T10:14:39.376 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2023-10-18T10:14:39.709 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-modules-core-5.14.0-376.el9.x86_64 1/4 2023-10-18T10:14:39.825 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-18T10:14:40.002 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-core-5.14.0-376.el9.x86_64 1/4 2023-10-18T10:14:40.120 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-18T10:14:42.548 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-18T10:14:42.743 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-18T10:14:42.879 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-18T10:14:43.045 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-18T10:14:47.368 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-18T10:14:47.578 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:14:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:14:52.151 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-core-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:14:52.308 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-core-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:20.816 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-core-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:20.858 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:21.123 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:21.166 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:21.267 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:21.267 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-5.14.0-376.el9.x86_64 1/4 2023-10-18T10:15:21.268 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-18T10:15:21.268 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-18T10:15:21.550 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:21.550 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-376.el9.x86_64 1/4 2023-10-18T10:15:21.550 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-core-5.14.0-376.el9.x86_64 2/4 2023-10-18T10:15:21.551 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-5.14.0-376.el9.x86_64 3/4 2023-10-18T10:15:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-modules-core-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:21.708 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:15:21.708 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-18T10:15:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-5.14.0-376.el9.x86_64 2023-10-18T10:15:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-core-5.14.0-376.el9.x86_64 2023-10-18T10:15:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-5.14.0-376.el9.x86_64 2023-10-18T10:15:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-core-5.14.0-376.el9.x86_64 2023-10-18T10:15:21.709 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:15:21.710 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-18T10:15:21.955 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-core-5.14.0-376.el9.x86_64 4/4 2023-10-18T10:15:21.956 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:15:21.956 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2023-10-18T10:15:21.956 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-376.el9.x86_64 2023-10-18T10:15:21.956 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core-5.14.0-376.el9.x86_64 2023-10-18T10:15:21.956 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-5.14.0-376.el9.x86_64 2023-10-18T10:15:21.957 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core-5.14.0-376.el9.x86_64 2023-10-18T10:15:21.957 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:15:21.957 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2023-10-18T10:15:22.208 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q kernel | sort -rV | head -n 1 2023-10-18T10:15:22.377 INFO:teuthology.orchestra.run.smithi125.stdout:kernel-5.14.0-376.el9.x86_64 2023-10-18T10:15:22.378 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-376.el9.x86_64 2023-10-18T10:15:22.378 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-376.el9.x86_64 2023-10-18T10:15:22.378 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-18T10:15:22.379 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-18T10:15:22.379 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-18T10:15:22.379 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi125.front.sepia.ceph.com, path=None, version=distro) 2023-10-18T10:15:22.379 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q kernel | sort -rV | head -n 1 2023-10-18T10:15:22.435 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2023-10-18T10:15:22.483 INFO:teuthology.orchestra.run.smithi125.stdout:kernel-5.14.0-376.el9.x86_64 2023-10-18T10:15:22.484 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-376.el9.x86_64 2023-10-18T10:15:22.485 DEBUG:teuthology.orchestra.run.smithi125:> sudo rpm -qi grub2-tools 2023-10-18T10:15:22.518 INFO:teuthology.orchestra.run.smithi125.stdout:Name : grub2-tools 2023-10-18T10:15:22.518 INFO:teuthology.orchestra.run.smithi125.stdout:Epoch : 1 2023-10-18T10:15:22.519 INFO:teuthology.orchestra.run.smithi125.stdout:Version : 2.06 2023-10-18T10:15:22.521 INFO:teuthology.orchestra.run.smithi125.stdout:Release : 68.el9 2023-10-18T10:15:22.521 INFO:teuthology.orchestra.run.smithi125.stdout:Architecture: x86_64 2023-10-18T10:15:22.522 INFO:teuthology.orchestra.run.smithi125.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-18T10:15:22.522 INFO:teuthology.orchestra.run.smithi125.stdout:Group : Unspecified 2023-10-18T10:15:22.522 INFO:teuthology.orchestra.run.smithi125.stdout:Size : 8264254 2023-10-18T10:15:22.522 INFO:teuthology.orchestra.run.smithi125.stdout:License : GPLv3+ 2023-10-18T10:15:22.523 INFO:teuthology.orchestra.run.smithi125.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-18T10:15:22.523 INFO:teuthology.orchestra.run.smithi125.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-18T10:15:22.523 INFO:teuthology.orchestra.run.smithi125.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-18T10:15:22.523 INFO:teuthology.orchestra.run.smithi125.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-18T10:15:22.523 INFO:teuthology.orchestra.run.smithi125.stdout:Packager : builder@centos.org 2023-10-18T10:15:22.524 INFO:teuthology.orchestra.run.smithi125.stdout:Vendor : CentOS 2023-10-18T10:15:22.524 INFO:teuthology.orchestra.run.smithi125.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-18T10:15:22.524 INFO:teuthology.orchestra.run.smithi125.stdout:Summary : Support tools for GRUB. 2023-10-18T10:15:22.524 INFO:teuthology.orchestra.run.smithi125.stdout:Description : 2023-10-18T10:15:22.524 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:15:22.525 INFO:teuthology.orchestra.run.smithi125.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-18T10:15:22.525 INFO:teuthology.orchestra.run.smithi125.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-18T10:15:22.525 INFO:teuthology.orchestra.run.smithi125.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-18T10:15:22.525 INFO:teuthology.orchestra.run.smithi125.stdout:hardware devices. 2023-10-18T10:15:22.525 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:15:22.525 INFO:teuthology.orchestra.run.smithi125.stdout:This subpackage provides tools for support of all platforms. 2023-10-18T10:15:22.526 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-18T10:15:22.527 INFO:teuthology.task.kernel:Updating grub on smithi125 to boot 5.14.0-376.el9.x86_64 2023-10-18T10:15:22.527 DEBUG:teuthology.orchestra.run.smithi125:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-18T10:15:22.574 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-376.el9.x86_64 2023-10-18T10:15:22.574 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-376.el9.x86_64 2023-10-18T10:15:22.574 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-376.el9.x86_64 2023-10-18T10:15:22.575 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-18T10:15:22.575 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-18T10:15:22.575 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-18T10:15:22.575 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2023-10-18T10:15:22.575 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2023-10-18T10:15:22.681 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-376.el9.x86_64 2023-10-18T10:15:22.681 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-376.el9.x86_64 2023-10-18T10:15:22.681 DEBUG:teuthology.orchestra.run.smithi139:> sudo rpm -qi grub2-tools 2023-10-18T10:15:22.723 INFO:teuthology.orchestra.run.smithi139.stdout:Name : grub2-tools 2023-10-18T10:15:22.723 INFO:teuthology.orchestra.run.smithi139.stdout:Epoch : 1 2023-10-18T10:15:22.723 INFO:teuthology.orchestra.run.smithi139.stdout:Version : 2.06 2023-10-18T10:15:22.724 INFO:teuthology.orchestra.run.smithi139.stdout:Release : 68.el9 2023-10-18T10:15:22.724 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: x86_64 2023-10-18T10:15:22.724 INFO:teuthology.orchestra.run.smithi139.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-18T10:15:22.724 INFO:teuthology.orchestra.run.smithi139.stdout:Group : Unspecified 2023-10-18T10:15:22.725 INFO:teuthology.orchestra.run.smithi139.stdout:Size : 8264254 2023-10-18T10:15:22.725 INFO:teuthology.orchestra.run.smithi139.stdout:License : GPLv3+ 2023-10-18T10:15:22.725 INFO:teuthology.orchestra.run.smithi139.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-18T10:15:22.725 INFO:teuthology.orchestra.run.smithi139.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-18T10:15:22.725 INFO:teuthology.orchestra.run.smithi139.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-18T10:15:22.726 INFO:teuthology.orchestra.run.smithi139.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-18T10:15:22.726 INFO:teuthology.orchestra.run.smithi139.stdout:Packager : builder@centos.org 2023-10-18T10:15:22.726 INFO:teuthology.orchestra.run.smithi139.stdout:Vendor : CentOS 2023-10-18T10:15:22.726 INFO:teuthology.orchestra.run.smithi139.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-18T10:15:22.727 INFO:teuthology.orchestra.run.smithi139.stdout:Summary : Support tools for GRUB. 2023-10-18T10:15:22.727 INFO:teuthology.orchestra.run.smithi139.stdout:Description : 2023-10-18T10:15:22.727 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:15:22.727 INFO:teuthology.orchestra.run.smithi139.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-18T10:15:22.727 INFO:teuthology.orchestra.run.smithi139.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-18T10:15:22.727 INFO:teuthology.orchestra.run.smithi139.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-18T10:15:22.728 INFO:teuthology.orchestra.run.smithi139.stdout:hardware devices. 2023-10-18T10:15:22.730 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:15:22.730 INFO:teuthology.orchestra.run.smithi139.stdout:This subpackage provides tools for support of all platforms. 2023-10-18T10:15:22.731 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-18T10:15:22.731 INFO:teuthology.task.kernel:Updating grub on smithi139 to boot 5.14.0-376.el9.x86_64 2023-10-18T10:15:22.732 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-18T10:15:23.163 INFO:teuthology.orchestra.run.smithi125.stderr:Generating grub configuration file ... 2023-10-18T10:15:23.322 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2023-10-18T10:15:24.592 INFO:teuthology.orchestra.run.smithi125.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-18T10:15:24.618 INFO:teuthology.orchestra.run.smithi125.stderr:done 2023-10-18T10:15:24.625 DEBUG:teuthology.orchestra.run.smithi125:> mktemp 2023-10-18T10:15:24.641 INFO:teuthology.orchestra.run.smithi125.stdout:/tmp/tmp.8hQEqPRC77 2023-10-18T10:15:24.641 DEBUG:teuthology.orchestra.run.smithi125:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8hQEqPRC77 2023-10-18T10:15:24.655 INFO:teuthology.orchestra.run.smithi139.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-18T10:15:24.693 INFO:teuthology.orchestra.run.smithi139.stderr:done 2023-10-18T10:15:24.695 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2023-10-18T10:15:24.709 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0666 /tmp/tmp.8hQEqPRC77 2023-10-18T10:15:24.711 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.lkDGZTkt7w 2023-10-18T10:15:24.711 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lkDGZTkt7w 2023-10-18T10:15:24.780 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.lkDGZTkt7w 2023-10-18T10:15:24.845 DEBUG:teuthology.orchestra.remote:smithi125:/tmp/tmp.8hQEqPRC77 is 6KB 2023-10-18T10:15:24.894 DEBUG:teuthology.orchestra.run.smithi125:> rm -fr /tmp/tmp.8hQEqPRC77 2023-10-18T10:15:24.909 DEBUG:teuthology.orchestra.run.smithi125:> sudo /bin/ls /boot/loader/entries || true 2023-10-18T10:15:24.915 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.lkDGZTkt7w is 6KB 2023-10-18T10:15:24.962 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.lkDGZTkt7w 2023-10-18T10:15:24.977 DEBUG:teuthology.orchestra.run.smithi139:> sudo /bin/ls /boot/loader/entries || true 2023-10-18T10:15:24.979 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-18T10:15:24.979 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-18T10:15:24.979 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-18T10:15:24.979 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-376.el9.x86_64.conf 2023-10-18T10:15:24.980 DEBUG:teuthology.orchestra.run.smithi125:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-376.el9.x86_64 2023-10-18T10:15:25.041 INFO:teuthology.orchestra.run.smithi139.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-18T10:15:25.041 INFO:teuthology.orchestra.run.smithi139.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-18T10:15:25.042 INFO:teuthology.orchestra.run.smithi139.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-18T10:15:25.042 INFO:teuthology.orchestra.run.smithi139.stdout:15949e3c88704300882482eddfd7baae-5.14.0-376.el9.x86_64.conf 2023-10-18T10:15:25.043 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-376.el9.x86_64 2023-10-18T10:15:25.133 DEBUG:teuthology.orchestra.run.smithi125:> sudo shutdown -r now 2023-10-18T10:15:25.200 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2023-10-18T10:15:55.162 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-18T10:15:55.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-18T10:15:55.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:15:55.204 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-18T10:15:55.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-10-18T10:15:55.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:16:55.227 DEBUG:teuthology.orchestra.remote:timed out 2023-10-18T10:16:55.228 DEBUG:teuthology.orchestra.remote:timed out 2023-10-18T10:17:04.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-18T10:17:04.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:17:04.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-10-18T10:17:04.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:17:04.651 DEBUG:teuthology.orchestra.run.smithi125:> true 2023-10-18T10:17:04.766 DEBUG:teuthology.orchestra.run.smithi139:> true 2023-10-18T10:17:05.154 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-18T10:17:05.154 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-376.el9.x86_64"... 2023-10-18T10:17:05.155 DEBUG:teuthology.orchestra.run.smithi125:> uname -r 2023-10-18T10:17:05.169 INFO:teuthology.orchestra.run.smithi125.stdout:5.14.0-376.el9.x86_64 2023-10-18T10:17:05.169 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-376.el9.x86_64 vs 5.14.0-376.el9.x86_64 2023-10-18T10:17:05.169 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T10:17:05.170 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-18T10:17:05.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-10-18T10:17:05.171 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-376.el9.x86_64"... 2023-10-18T10:17:05.171 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2023-10-18T10:17:05.186 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-376.el9.x86_64 2023-10-18T10:17:05.186 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-376.el9.x86_64 vs 5.14.0-376.el9.x86_64 2023-10-18T10:17:05.187 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T10:17:05.187 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-18T10:17:06.171 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-18T10:17:06.172 DEBUG:teuthology.orchestra.run.smithi125:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-18T10:17:06.188 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-18T10:17:06.189 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-18T10:17:06.238 INFO:teuthology.orchestra.run.smithi125.stdout:ttyS1 2023-10-18T10:17:06.238 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2023-10-18T10:17:06.276 DEBUG:teuthology.parallel:result is None 2023-10-18T10:17:06.277 DEBUG:teuthology.parallel:result is None 2023-10-18T10:17:06.278 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-18T10:17:06.286 INFO:teuthology.task.internal:Creating test directory... 2023-10-18T10:17:06.286 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T10:17:06.289 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T10:17:06.303 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-18T10:17:06.312 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-18T10:17:06.326 INFO:teuthology.task.internal:Creating archive directory... 2023-10-18T10:17:06.327 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T10:17:06.343 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T10:17:06.384 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-18T10:17:06.393 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-18T10:17:06.393 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T10:17:06.422 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T10:17:06.452 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T10:17:06.453 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T10:17:06.465 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T10:17:06.466 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T10:17:06.468 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-18T10:17:06.475 INFO:teuthology.task.internal:Configuring sudo... 2023-10-18T10:17:06.475 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T10:17:06.511 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T10:17:06.539 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-18T10:17:06.549 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-18T10:17:06.549 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T10:17:06.580 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T10:17:06.594 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T10:17:06.675 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T10:17:06.727 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T10:17:06.812 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T10:17:06.853 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:17:06.854 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T10:17:06.924 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T10:17:06.963 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T10:17:07.012 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T10:17:07.093 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T10:17:07.139 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:17:07.139 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T10:17:07.205 DEBUG:teuthology.orchestra.run.smithi125:> sudo service rsyslog restart 2023-10-18T10:17:07.209 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2023-10-18T10:17:07.255 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T10:17:07.289 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T10:17:07.702 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-18T10:17:07.747 INFO:teuthology.task.internal:Starting timer... 2023-10-18T10:17:07.748 INFO:teuthology.run_tasks:Running task pcp... 2023-10-18T10:17:07.757 INFO:teuthology.run_tasks:Running task selinux... 2023-10-18T10:17:07.765 DEBUG:teuthology.orchestra.run.smithi125:> sudo service auditd rotate 2023-10-18T10:17:07.816 INFO:teuthology.orchestra.run.smithi125.stdout:Rotating logs: 2023-10-18T10:17:07.818 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2023-10-18T10:17:07.887 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2023-10-18T10:17:07.889 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-18T10:17:07.890 DEBUG:teuthology.orchestra.run.smithi125:> /usr/sbin/getenforce 2023-10-18T10:17:07.907 INFO:teuthology.orchestra.run.smithi125.stdout:Permissive 2023-10-18T10:17:07.907 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2023-10-18T10:17:07.932 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2023-10-18T10:17:07.933 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi125.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive'} 2023-10-18T10:17:07.933 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T10:17:07.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:17:07.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T10:17:08.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:17:08.000 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-18T10:17:08.001 DEBUG:teuthology.orchestra.run.smithi125:> sudo /usr/sbin/setenforce permissive 2023-10-18T10:17:08.076 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2023-10-18T10:17:08.118 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-18T10:17:08.132 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-18T10:17:08.138 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-18T10:17:08.248 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-18T10:17:08.266 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-18T10:17:08.268 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi125.front.sepia.ceph.com,smithi139.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-18T10:22:29.597 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi125.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com')] 2023-10-18T10:22:29.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-18T10:22:29.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:22:29.685 DEBUG:teuthology.orchestra.run.smithi125:> true 2023-10-18T10:22:29.752 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-18T10:22:29.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-10-18T10:22:29.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T10:22:29.836 DEBUG:teuthology.orchestra.run.smithi139:> true 2023-10-18T10:22:29.905 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-10-18T10:22:29.906 INFO:teuthology.run_tasks:Running task clock... 2023-10-18T10:22:29.924 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-18T10:22:29.925 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T10:22:29.926 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T10:22:29.930 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T10:22:29.930 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T10:22:29.956 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T10:22:29.973 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T10:22:29.976 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T10:22:29.993 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T10:22:30.001 INFO:teuthology.orchestra.run.smithi125.stderr:sudo: ntpd: command not found 2023-10-18T10:22:30.014 INFO:teuthology.orchestra.run.smithi125.stdout:506 Cannot talk to daemon 2023-10-18T10:22:30.022 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2023-10-18T10:22:30.031 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T10:22:30.035 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2023-10-18T10:22:30.047 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T10:22:30.052 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T10:22:30.068 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T10:22:30.090 INFO:teuthology.orchestra.run.smithi125.stderr:bash: line 1: ntpq: command not found 2023-10-18T10:22:30.093 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T10:22:30.094 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2023-10-18T10:22:30.094 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T10:22:30.094 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T10:22:30.094 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T10:22:30.095 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T10:22:30.111 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2023-10-18T10:22:30.114 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T10:22:30.115 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2023-10-18T10:22:30.115 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T10:22:30.115 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T10:22:30.115 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T10:22:30.115 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T10:22:30.116 INFO:teuthology.run_tasks:Running task install... 2023-10-18T10:22:30.124 DEBUG:teuthology.task.install:project ceph 2023-10-18T10:22:30.125 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}} 2023-10-18T10:22:30.125 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'} 2023-10-18T10:22:30.126 INFO:teuthology.task.install:Using flavor: default 2023-10-18T10:22:30.132 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-18T10:22:30.132 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-18T10:22:30.133 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T10:22:30.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T10:22:30.135 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T10:22:30.136 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T10:22:30.358 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/ 2023-10-18T10:22:30.359 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T10:22:30.375 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/ 2023-10-18T10:22:30.375 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T10:22:30.569 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T10:22:30.569 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:22:30.570 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T10:22:30.576 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T10:22:30.577 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:22:30.577 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T10:22:30.606 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T10:22:30.607 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T10:22:30.617 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T10:22:30.617 DEBUG:teuthology.orchestra.run.smithi125:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T10:22:30.685 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T10:22:30.692 DEBUG:teuthology.orchestra.run.smithi125:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T10:22:30.773 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T10:22:30.778 DEBUG:teuthology.orchestra.run.smithi125:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T10:22:30.852 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2023-10-18T10:22:30.854 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2023-10-18T10:22:30.858 INFO:teuthology.orchestra.run.smithi125.stdout:check_obsoletes = 1 2023-10-18T10:22:30.860 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum clean all 2023-10-18T10:22:31.105 INFO:teuthology.orchestra.run.smithi139.stdout:72 files removed 2023-10-18T10:22:31.117 INFO:teuthology.orchestra.run.smithi125.stdout:72 files removed 2023-10-18T10:22:31.133 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T10:22:31.145 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T10:22:31.804 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 224 kB/s | 78 kB 00:00 2023-10-18T10:22:31.899 INFO:teuthology.orchestra.run.smithi125.stdout:ceph packages for x86_64 179 kB/s | 78 kB 00:00 2023-10-18T10:22:32.204 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-10-18T10:22:32.366 INFO:teuthology.orchestra.run.smithi125.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-10-18T10:22:32.529 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2023-10-18T10:22:32.717 INFO:teuthology.orchestra.run.smithi125.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2023-10-18T10:22:32.990 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS-9 - AppStream 15 MB/s | 6.3 MB 00:00 2023-10-18T10:22:33.155 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - AppStream 16 MB/s | 6.3 MB 00:00 2023-10-18T10:22:34.357 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2023-10-18T10:22:34.480 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2023-10-18T10:22:35.164 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 7.9 MB 00:00 2023-10-18T10:22:36.256 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 7.9 MB 00:01 2023-10-18T10:22:37.723 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 12 MB/s | 18 MB 00:01 2023-10-18T10:22:39.546 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 18 MB 00:02 2023-10-18T10:22:41.424 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 14 MB/s | 5.7 MB 00:00 2023-10-18T10:22:42.967 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 14 kB 00:00 2023-10-18T10:22:44.043 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 20 MB/s | 19 MB 00:00 2023-10-18T10:22:44.447 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 5.7 MB 00:01 2023-10-18T10:22:45.781 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 14 kB 00:00 2023-10-18T10:22:48.082 INFO:teuthology.orchestra.run.smithi125.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 19 MB 00:02 2023-10-18T10:22:48.568 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2023-10-18T10:22:50.839 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-18T10:22:50.840 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-18T10:22:50.852 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-18T10:22:50.853 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-18T10:22:50.854 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-18T10:22:50.855 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-18T10:22:50.857 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-18T10:22:50.860 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-18T10:22:50.861 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-18T10:22:50.929 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2023-10-18T10:22:50.958 INFO:teuthology.orchestra.run.smithi139.stdout:============================================================================================ 2023-10-18T10:22:50.958 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2023-10-18T10:22:50.958 INFO:teuthology.orchestra.run.smithi139.stdout:============================================================================================ 2023-10-18T10:22:50.958 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2023-10-18T10:22:50.959 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-18T10:22:50.959 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 6.5 k 2023-10-18T10:22:50.959 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 5.2 M 2023-10-18T10:22:50.959 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 857 k 2023-10-18T10:22:50.959 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 142 k 2023-10-18T10:22:50.960 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 1.5 M 2023-10-18T10:22:50.960 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 132 k 2023-10-18T10:22:50.960 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 1.8 M 2023-10-18T10:22:50.960 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 7.4 M 2023-10-18T10:22:50.960 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 47 k 2023-10-18T10:22:50.960 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 8.6 M 2023-10-18T10:22:50.961 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 41 M 2023-10-18T10:22:50.961 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 262 k 2023-10-18T10:22:50.961 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 270 k 2023-10-18T10:22:50.961 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 218 k 2023-10-18T10:22:50.961 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 18 k 2023-10-18T10:22:50.962 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-18T10:22:50.962 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-18T10:22:50.962 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-18T10:22:50.962 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 31 k 2023-10-18T10:22:50.962 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 716 k 2023-10-18T10:22:50.963 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 127 k 2023-10-18T10:22:50.963 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-18T10:22:50.963 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-18T10:22:50.963 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-18T10:22:50.963 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-18T10:22:50.964 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-18T10:22:50.964 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-18T10:22:50.964 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 162 k 2023-10-18T10:22:50.964 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 321 k 2023-10-18T10:22:50.964 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 297 k 2023-10-18T10:22:50.965 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 99 k 2023-10-18T10:22:50.965 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 86 k 2023-10-18T10:22:50.965 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.0 M 2023-10-18T10:22:50.965 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 169 k 2023-10-18T10:22:50.965 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-18T10:22:50.965 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2023-10-18T10:22:50.966 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-18T10:22:50.966 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-18T10:22:50.966 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-18T10:22:50.966 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-18T10:22:50.966 INFO:teuthology.orchestra.run.smithi139.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-18T10:22:50.967 INFO:teuthology.orchestra.run.smithi139.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-18T10:22:50.967 INFO:teuthology.orchestra.run.smithi139.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-18T10:22:50.967 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.3 M 2023-10-18T10:22:50.967 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.0 M 2023-10-18T10:22:50.967 INFO:teuthology.orchestra.run.smithi139.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-18T10:22:50.968 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2023-10-18T10:22:50.968 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-18T10:22:50.968 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-18T10:22:50.968 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 19 M 2023-10-18T10:22:50.968 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 23 k 2023-10-18T10:22:50.969 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 2.1 M 2023-10-18T10:22:50.969 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 242 k 2023-10-18T10:22:50.969 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 4.4 M 2023-10-18T10:22:50.969 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 18 M 2023-10-18T10:22:50.969 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 15 k 2023-10-18T10:22:50.970 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 24 k 2023-10-18T10:22:50.970 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-18T10:22:50.970 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-18T10:22:50.970 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-18T10:22:50.970 INFO:teuthology.orchestra.run.smithi139.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-18T10:22:50.971 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-18T10:22:50.971 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-18T10:22:50.971 INFO:teuthology.orchestra.run.smithi139.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-18T10:22:50.971 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-18T10:22:50.971 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-18T10:22:50.971 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 164 k 2023-10-18T10:22:50.972 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-18T10:22:50.972 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-18T10:22:50.972 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-18T10:22:50.972 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-18T10:22:50.972 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-18T10:22:50.973 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 472 k 2023-10-18T10:22:50.973 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-18T10:22:50.973 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 4.8 M 2023-10-18T10:22:50.973 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-18T10:22:50.973 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-18T10:22:50.974 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-18T10:22:50.974 INFO:teuthology.orchestra.run.smithi139.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-18T10:22:50.974 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-18T10:22:50.974 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-18T10:22:50.974 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-18T10:22:50.975 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-18T10:22:50.975 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-18T10:22:50.975 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-18T10:22:50.975 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-18T10:22:50.975 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-18T10:22:50.976 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 45 k 2023-10-18T10:22:50.976 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 127 k 2023-10-18T10:22:50.976 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-18T10:22:50.976 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-18T10:22:50.976 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-18T10:22:50.977 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-18T10:22:50.977 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-18T10:22:50.977 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-18T10:22:50.977 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-18T10:22:50.977 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-18T10:22:50.977 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-18T10:22:50.978 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-18T10:22:50.978 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-18T10:22:50.978 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-18T10:22:50.978 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-18T10:22:50.978 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-18T10:22:50.979 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-18T10:22:50.979 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-18T10:22:50.979 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-18T10:22:50.979 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-18T10:22:50.979 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-18T10:22:50.980 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-18T10:22:50.980 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-18T10:22:50.980 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-18T10:22:50.980 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-18T10:22:50.980 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-18T10:22:50.981 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-18T10:22:50.981 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-18T10:22:50.981 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-18T10:22:50.981 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-18T10:22:50.981 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-18T10:22:50.982 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-18T10:22:50.982 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-18T10:22:50.982 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-18T10:22:50.982 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-18T10:22:50.982 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-18T10:22:50.983 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-18T10:22:50.983 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-18T10:22:50.983 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-18T10:22:50.983 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-18T10:22:50.983 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-18T10:22:50.983 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-18T10:22:50.984 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-18T10:22:50.984 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-18T10:22:50.984 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-18T10:22:50.984 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-18T10:22:50.984 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-18T10:22:50.985 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-18T10:22:50.985 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-18T10:22:50.985 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-18T10:22:50.985 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-18T10:22:50.985 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-18T10:22:50.985 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-18T10:22:50.986 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-18T10:22:50.986 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-18T10:22:50.986 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-18T10:22:50.986 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2023-10-18T10:22:50.986 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-18T10:22:50.987 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:22:50.987 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2023-10-18T10:22:50.987 INFO:teuthology.orchestra.run.smithi139.stdout:============================================================================================ 2023-10-18T10:22:50.987 INFO:teuthology.orchestra.run.smithi139.stdout:Install 133 Packages 2023-10-18T10:22:50.987 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 10 Packages 2023-10-18T10:22:50.988 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:22:50.988 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 200 M 2023-10-18T10:22:50.988 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2023-10-18T10:22:51.561 INFO:teuthology.orchestra.run.smithi139.stdout:(1/143): ceph-18.0.0-6786.gf7420d7f.el9.x86_64. 38 kB/s | 6.5 kB 00:00 2023-10-18T10:22:51.853 INFO:teuthology.orchestra.run.smithi139.stdout:(2/143): ceph-fuse-18.0.0-6786.gf7420d7f.el9.x8 2.9 MB/s | 857 kB 00:00 2023-10-18T10:22:51.920 INFO:teuthology.orchestra.run.smithi139.stdout:(3/143): ceph-immutable-object-cache-18.0.0-678 2.1 MB/s | 142 kB 00:00 2023-10-18T10:22:52.228 INFO:teuthology.orchestra.run.smithi139.stdout:(4/143): ceph-mds-18.0.0-6786.gf7420d7f.el9.x86 6.8 MB/s | 2.1 MB 00:00 2023-10-18T10:22:52.362 INFO:teuthology.orchestra.run.smithi139.stdout:(5/143): ceph-base-18.0.0-6786.gf7420d7f.el9.x8 5.4 MB/s | 5.2 MB 00:00 2023-10-18T10:22:52.487 INFO:teuthology.orchestra.run.smithi139.stdout:(6/143): ceph-mgr-18.0.0-6786.gf7420d7f.el9.x86 5.7 MB/s | 1.5 MB 00:00 2023-10-18T10:22:52.574 INFO:teuthology.orchestra.run.smithi125.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2023-10-18T10:22:52.754 INFO:teuthology.orchestra.run.smithi139.stdout:(7/143): ceph-mon-18.0.0-6786.gf7420d7f.el9.x86 11 MB/s | 4.4 MB 00:00 2023-10-18T10:22:53.479 INFO:teuthology.orchestra.run.smithi139.stdout:(8/143): ceph-radosgw-18.0.0-6786.gf7420d7f.el9 12 MB/s | 8.6 MB 00:00 2023-10-18T10:22:53.546 INFO:teuthology.orchestra.run.smithi139.stdout:(9/143): ceph-selinux-18.0.0-6786.gf7420d7f.el9 361 kB/s | 24 kB 00:00 2023-10-18T10:22:54.298 INFO:teuthology.orchestra.run.smithi139.stdout:(10/143): ceph-osd-18.0.0-6786.gf7420d7f.el9.x8 9.8 MB/s | 18 MB 00:01 2023-10-18T10:22:54.380 INFO:teuthology.orchestra.run.smithi139.stdout:(11/143): cephfs-mirror-18.0.0-6786.gf7420d7f.e 2.6 MB/s | 218 kB 00:00 2023-10-18T10:22:54.456 INFO:teuthology.orchestra.run.smithi139.stdout:(12/143): libcephfs-devel-18.0.0-6786.gf7420d7f 414 kB/s | 31 kB 00:00 2023-10-18T10:22:54.614 INFO:teuthology.orchestra.run.smithi139.stdout:(13/143): libcephfs2-18.0.0-6786.gf7420d7f.el9. 4.4 MB/s | 716 kB 00:00 2023-10-18T10:22:54.842 INFO:teuthology.orchestra.run.smithi125.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-18T10:22:54.843 INFO:teuthology.orchestra.run.smithi125.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-18T10:22:54.847 INFO:teuthology.orchestra.run.smithi139.stdout:(14/143): ceph-common-18.0.0-6786.gf7420d7f.el9 5.5 MB/s | 19 MB 00:03 2023-10-18T10:22:54.854 INFO:teuthology.orchestra.run.smithi125.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-18T10:22:54.855 INFO:teuthology.orchestra.run.smithi125.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-18T10:22:54.856 INFO:teuthology.orchestra.run.smithi125.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-18T10:22:54.856 INFO:teuthology.orchestra.run.smithi125.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-18T10:22:54.858 INFO:teuthology.orchestra.run.smithi125.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-18T10:22:54.860 INFO:teuthology.orchestra.run.smithi125.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-18T10:22:54.861 INFO:teuthology.orchestra.run.smithi125.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-18T10:22:54.873 INFO:teuthology.orchestra.run.smithi139.stdout:(15/143): libcephsqlite-18.0.0-6786.gf7420d7f.e 636 kB/s | 164 kB 00:00 2023-10-18T10:22:54.932 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2023-10-18T10:22:54.942 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2023-10-18T10:22:54.942 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2023-10-18T10:22:54.942 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2023-10-18T10:22:54.943 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2023-10-18T10:22:54.943 INFO:teuthology.orchestra.run.smithi125.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-18T10:22:54.943 INFO:teuthology.orchestra.run.smithi125.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 6.5 k 2023-10-18T10:22:54.943 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 5.2 M 2023-10-18T10:22:54.943 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 857 k 2023-10-18T10:22:54.944 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 142 k 2023-10-18T10:22:54.944 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 1.5 M 2023-10-18T10:22:54.944 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 132 k 2023-10-18T10:22:54.944 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 1.8 M 2023-10-18T10:22:54.944 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 7.4 M 2023-10-18T10:22:54.945 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 47 k 2023-10-18T10:22:54.945 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 8.6 M 2023-10-18T10:22:54.945 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 41 M 2023-10-18T10:22:54.945 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 262 k 2023-10-18T10:22:54.945 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 270 k 2023-10-18T10:22:54.946 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 218 k 2023-10-18T10:22:54.946 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 18 k 2023-10-18T10:22:54.946 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-18T10:22:54.946 INFO:teuthology.orchestra.run.smithi125.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-18T10:22:54.946 INFO:teuthology.orchestra.run.smithi125.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-18T10:22:54.947 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 31 k 2023-10-18T10:22:54.947 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 716 k 2023-10-18T10:22:54.947 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 127 k 2023-10-18T10:22:54.947 INFO:teuthology.orchestra.run.smithi125.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-18T10:22:54.948 INFO:teuthology.orchestra.run.smithi125.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-18T10:22:54.948 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-18T10:22:54.948 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-18T10:22:54.948 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-18T10:22:54.948 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-18T10:22:54.949 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 162 k 2023-10-18T10:22:54.949 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 321 k 2023-10-18T10:22:54.949 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 297 k 2023-10-18T10:22:54.949 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 99 k 2023-10-18T10:22:54.949 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 86 k 2023-10-18T10:22:54.950 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.0 M 2023-10-18T10:22:54.950 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 169 k 2023-10-18T10:22:54.950 INFO:teuthology.orchestra.run.smithi125.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-18T10:22:54.950 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrading: 2023-10-18T10:22:54.950 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-18T10:22:54.951 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-18T10:22:54.951 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-18T10:22:54.951 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-18T10:22:54.951 INFO:teuthology.orchestra.run.smithi125.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-18T10:22:54.951 INFO:teuthology.orchestra.run.smithi125.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-18T10:22:54.951 INFO:teuthology.orchestra.run.smithi125.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-18T10:22:54.952 INFO:teuthology.orchestra.run.smithi125.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.3 M 2023-10-18T10:22:54.952 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.0 M 2023-10-18T10:22:54.952 INFO:teuthology.orchestra.run.smithi125.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-18T10:22:54.952 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2023-10-18T10:22:54.952 INFO:teuthology.orchestra.run.smithi125.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-18T10:22:54.953 INFO:teuthology.orchestra.run.smithi125.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-18T10:22:54.953 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 19 M 2023-10-18T10:22:54.953 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 23 k 2023-10-18T10:22:54.953 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 2.1 M 2023-10-18T10:22:54.953 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 242 k 2023-10-18T10:22:54.954 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 4.4 M 2023-10-18T10:22:54.954 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 18 M 2023-10-18T10:22:54.954 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 15 k 2023-10-18T10:22:54.954 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 24 k 2023-10-18T10:22:54.954 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-18T10:22:54.955 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-18T10:22:54.955 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-18T10:22:54.955 INFO:teuthology.orchestra.run.smithi125.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-18T10:22:54.955 INFO:teuthology.orchestra.run.smithi125.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-18T10:22:54.955 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-18T10:22:54.956 INFO:teuthology.orchestra.run.smithi125.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-18T10:22:54.956 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-18T10:22:54.956 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-18T10:22:54.956 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 164 k 2023-10-18T10:22:54.956 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-18T10:22:54.957 INFO:teuthology.orchestra.run.smithi125.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-18T10:22:54.957 INFO:teuthology.orchestra.run.smithi125.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-18T10:22:54.957 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-18T10:22:54.957 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-18T10:22:54.957 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 472 k 2023-10-18T10:22:54.957 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-18T10:22:54.958 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 4.8 M 2023-10-18T10:22:54.958 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-18T10:22:54.958 INFO:teuthology.orchestra.run.smithi125.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-18T10:22:54.958 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-18T10:22:54.959 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-18T10:22:54.959 INFO:teuthology.orchestra.run.smithi125.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-18T10:22:54.959 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-18T10:22:54.959 INFO:teuthology.orchestra.run.smithi125.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-18T10:22:54.959 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-18T10:22:54.959 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-18T10:22:54.960 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-18T10:22:54.960 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-18T10:22:54.960 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-18T10:22:54.960 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 45 k 2023-10-18T10:22:54.960 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 127 k 2023-10-18T10:22:54.961 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-18T10:22:54.961 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-18T10:22:54.961 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-18T10:22:54.961 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-18T10:22:54.961 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-18T10:22:54.962 INFO:teuthology.orchestra.run.smithi125.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-18T10:22:54.962 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-18T10:22:54.962 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-18T10:22:54.962 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-18T10:22:54.962 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-18T10:22:54.963 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-18T10:22:54.963 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-18T10:22:54.963 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-18T10:22:54.963 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-18T10:22:54.963 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-18T10:22:54.964 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-18T10:22:54.964 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-18T10:22:54.964 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-18T10:22:54.964 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-18T10:22:54.964 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-18T10:22:54.964 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-18T10:22:54.965 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-18T10:22:54.965 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-18T10:22:54.965 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-18T10:22:54.965 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-18T10:22:54.965 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-18T10:22:54.966 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-18T10:22:54.966 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-18T10:22:54.966 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-18T10:22:54.966 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-18T10:22:54.966 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-18T10:22:54.966 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-18T10:22:54.967 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-18T10:22:54.967 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-18T10:22:54.967 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-18T10:22:54.967 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-18T10:22:54.967 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-18T10:22:54.967 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-18T10:22:54.968 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-18T10:22:54.968 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-18T10:22:54.968 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-18T10:22:54.968 INFO:teuthology.orchestra.run.smithi125.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-18T10:22:54.968 INFO:teuthology.orchestra.run.smithi125.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-18T10:22:54.968 INFO:teuthology.orchestra.run.smithi125.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-18T10:22:54.969 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-18T10:22:54.969 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-18T10:22:54.969 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-18T10:22:54.969 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-18T10:22:54.969 INFO:teuthology.orchestra.run.smithi125.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-18T10:22:54.970 INFO:teuthology.orchestra.run.smithi125.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-18T10:22:54.970 INFO:teuthology.orchestra.run.smithi125.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-18T10:22:54.970 INFO:teuthology.orchestra.run.smithi125.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-18T10:22:54.970 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-18T10:22:54.970 INFO:teuthology.orchestra.run.smithi125.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-18T10:22:54.971 INFO:teuthology.orchestra.run.smithi125.stdout:Installing weak dependencies: 2023-10-18T10:22:54.971 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-18T10:22:54.971 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:22:54.971 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2023-10-18T10:22:54.971 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2023-10-18T10:22:54.972 INFO:teuthology.orchestra.run.smithi125.stdout:Install 133 Packages 2023-10-18T10:22:54.972 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrade 10 Packages 2023-10-18T10:22:54.972 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:22:54.973 INFO:teuthology.orchestra.run.smithi139.stdout:(16/143): librados-devel-18.0.0-6786.gf7420d7f. 1.2 MB/s | 127 kB 00:00 2023-10-18T10:22:54.973 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 200 M 2023-10-18T10:22:54.973 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2023-10-18T10:22:54.974 INFO:teuthology.orchestra.run.smithi139.stdout:(17/143): libradosstriper1-18.0.0-6786.gf7420d7 4.6 MB/s | 472 kB 00:00 2023-10-18T10:22:55.065 INFO:teuthology.orchestra.run.smithi139.stdout:(18/143): python3-ceph-argparse-18.0.0-6786.gf7 492 kB/s | 45 kB 00:00 2023-10-18T10:22:55.140 INFO:teuthology.orchestra.run.smithi139.stdout:(19/143): python3-ceph-common-18.0.0-6786.gf742 1.7 MB/s | 127 kB 00:00 2023-10-18T10:22:55.216 INFO:teuthology.orchestra.run.smithi139.stdout:(20/143): python3-cephfs-18.0.0-6786.gf7420d7f. 2.1 MB/s | 162 kB 00:00 2023-10-18T10:22:55.316 INFO:teuthology.orchestra.run.smithi139.stdout:(21/143): python3-rados-18.0.0-6786.gf7420d7f.e 3.1 MB/s | 321 kB 00:00 2023-10-18T10:22:55.375 INFO:teuthology.orchestra.run.smithi139.stdout:(22/143): librgw2-18.0.0-6786.gf7420d7f.el9.x86 11 MB/s | 4.8 MB 00:00 2023-10-18T10:22:55.400 INFO:teuthology.orchestra.run.smithi139.stdout:(23/143): python3-rbd-18.0.0-6786.gf7420d7f.el9 3.5 MB/s | 297 kB 00:00 2023-10-18T10:22:55.450 INFO:teuthology.orchestra.run.smithi139.stdout:(24/143): python3-rgw-18.0.0-6786.gf7420d7f.el9 1.3 MB/s | 99 kB 00:00 2023-10-18T10:22:55.475 INFO:teuthology.orchestra.run.smithi139.stdout:(25/143): rbd-fuse-18.0.0-6786.gf7420d7f.el9.x8 1.1 MB/s | 86 kB 00:00 2023-10-18T10:22:55.576 INFO:teuthology.orchestra.run.smithi139.stdout:(26/143): rbd-nbd-18.0.0-6786.gf7420d7f.el9.x86 1.7 MB/s | 169 kB 00:00 2023-10-18T10:22:55.651 INFO:teuthology.orchestra.run.smithi139.stdout:(27/143): ceph-grafana-dashboards-18.0.0-6786.g 309 kB/s | 23 kB 00:00 2023-10-18T10:22:55.700 INFO:teuthology.orchestra.run.smithi125.stdout:(1/143): ceph-18.0.0-6786.gf7420d7f.el9.x86_64. 23 kB/s | 6.5 kB 00:00 2023-10-18T10:22:55.718 INFO:teuthology.orchestra.run.smithi139.stdout:(28/143): rbd-mirror-18.0.0-6786.gf7420d7f.el9. 11 MB/s | 3.0 MB 00:00 2023-10-18T10:22:55.743 INFO:teuthology.orchestra.run.smithi139.stdout:(29/143): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 1.4 MB/s | 132 kB 00:00 2023-10-18T10:22:55.894 INFO:teuthology.orchestra.run.smithi139.stdout:(30/143): ceph-mgr-dashboard-18.0.0-6786.gf7420 10 MB/s | 1.8 MB 00:00 2023-10-18T10:22:55.977 INFO:teuthology.orchestra.run.smithi139.stdout:(31/143): ceph-mgr-modules-core-18.0.0-6786.gf7 2.8 MB/s | 242 kB 00:00 2023-10-18T10:22:56.052 INFO:teuthology.orchestra.run.smithi139.stdout:(32/143): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 634 kB/s | 47 kB 00:00 2023-10-18T10:22:56.119 INFO:teuthology.orchestra.run.smithi139.stdout:(33/143): ceph-prometheus-alerts-18.0.0-6786.gf 226 kB/s | 15 kB 00:00 2023-10-18T10:22:56.203 INFO:teuthology.orchestra.run.smithi139.stdout:(34/143): ceph-volume-18.0.0-6786.gf7420d7f.el9 3.1 MB/s | 262 kB 00:00 2023-10-18T10:22:56.250 INFO:teuthology.orchestra.run.smithi125.stdout:(2/143): ceph-fuse-18.0.0-6786.gf7420d7f.el9.x8 1.5 MB/s | 857 kB 00:00 2023-10-18T10:22:56.287 INFO:teuthology.orchestra.run.smithi139.stdout:(35/143): cephadm-18.0.0-6786.gf7420d7f.el9.noa 3.2 MB/s | 270 kB 00:00 2023-10-18T10:22:56.334 INFO:teuthology.orchestra.run.smithi125.stdout:(3/143): ceph-immutable-object-cache-18.0.0-678 1.7 MB/s | 142 kB 00:00 2023-10-18T10:22:56.353 INFO:teuthology.orchestra.run.smithi139.stdout:(36/143): cephfs-top-18.0.0-6786.gf7420d7f.el9. 274 kB/s | 18 kB 00:00 2023-10-18T10:22:56.771 INFO:teuthology.orchestra.run.smithi139.stdout:(37/143): ceph-test-18.0.0-6786.gf7420d7f.el9.x 13 MB/s | 41 MB 00:03 2023-10-18T10:22:56.775 INFO:teuthology.orchestra.run.smithi125.stdout:(4/143): ceph-mds-18.0.0-6786.gf7420d7f.el9.x86 4.7 MB/s | 2.1 MB 00:00 2023-10-18T10:22:56.812 INFO:teuthology.orchestra.run.smithi139.stdout:(38/143): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 920 kB 00:00 2023-10-18T10:22:56.837 INFO:teuthology.orchestra.run.smithi139.stdout:(39/143): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2023-10-18T10:22:56.851 INFO:teuthology.orchestra.run.smithi125.stdout:(5/143): ceph-base-18.0.0-6786.gf7420d7f.el9.x8 3.6 MB/s | 5.2 MB 00:01 2023-10-18T10:22:56.888 INFO:teuthology.orchestra.run.smithi139.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2023-10-18T10:22:56.922 INFO:teuthology.orchestra.run.smithi139.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 377 kB/s | 32 kB 00:00 2023-10-18T10:22:57.101 INFO:teuthology.orchestra.run.smithi125.stdout:(6/143): ceph-mgr-18.0.0-6786.gf7420d7f.el9.x86 4.5 MB/s | 1.5 MB 00:00 2023-10-18T10:22:57.139 INFO:teuthology.orchestra.run.smithi139.stdout:(42/143): ceph-mgr-diskprediction-local-18.0.0- 5.3 MB/s | 7.4 MB 00:01 2023-10-18T10:22:57.198 INFO:teuthology.orchestra.run.smithi139.stdout:(43/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2023-10-18T10:22:57.222 INFO:teuthology.orchestra.run.smithi139.stdout:(44/143): flexiblas-openblas-openmp-3.0.4-8.el9 57 kB/s | 17 kB 00:00 2023-10-18T10:22:57.248 INFO:teuthology.orchestra.run.smithi139.stdout:(45/143): libaio-devel-0.3.111-13.el9.x86_64.rp 94 kB/s | 10 kB 00:00 2023-10-18T10:22:57.273 INFO:teuthology.orchestra.run.smithi139.stdout:(46/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-10-18T10:22:57.306 INFO:teuthology.orchestra.run.smithi139.stdout:(47/143): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2023-10-18T10:22:57.340 INFO:teuthology.orchestra.run.smithi139.stdout:(48/143): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2023-10-18T10:22:57.365 INFO:teuthology.orchestra.run.smithi139.stdout:(49/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 509 kB/s | 46 kB 00:00 2023-10-18T10:22:57.384 INFO:teuthology.orchestra.run.smithi125.stdout:(7/143): ceph-mon-18.0.0-6786.gf7420d7f.el9.x86 8.2 MB/s | 4.4 MB 00:00 2023-10-18T10:22:57.391 INFO:teuthology.orchestra.run.smithi139.stdout:(50/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2023-10-18T10:22:57.424 INFO:teuthology.orchestra.run.smithi139.stdout:(51/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.7 MB/s | 217 kB 00:00 2023-10-18T10:22:57.449 INFO:teuthology.orchestra.run.smithi139.stdout:(52/143): openblas-0.3.21-2.el9.x86_64.rpm 412 kB/s | 34 kB 00:00 2023-10-18T10:22:57.633 INFO:teuthology.orchestra.run.smithi139.stdout:(53/143): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2023-10-18T10:22:57.675 INFO:teuthology.orchestra.run.smithi139.stdout:(54/143): postgresql-13.11-1.el9.x86_64.rpm 6.1 MB/s | 1.5 MB 00:00 2023-10-18T10:22:57.700 INFO:teuthology.orchestra.run.smithi139.stdout:(55/143): postgresql-contrib-13.11-1.el9.x86_64 3.2 MB/s | 811 kB 00:00 2023-10-18T10:22:57.734 INFO:teuthology.orchestra.run.smithi139.stdout:(56/143): postgresql-private-libs-13.11-1.el9.x 1.4 MB/s | 134 kB 00:00 2023-10-18T10:22:58.101 INFO:teuthology.orchestra.run.smithi139.stdout:(57/143): postgresql-server-13.11-1.el9.x86_64. 14 MB/s | 5.7 MB 00:00 2023-10-18T10:22:58.101 INFO:teuthology.orchestra.run.smithi125.stdout:(8/143): ceph-common-18.0.0-6786.gf7420d7f.el9. 7.1 MB/s | 19 MB 00:02 2023-10-18T10:22:58.126 INFO:teuthology.orchestra.run.smithi139.stdout:(58/143): python3-devel-3.9.17-2.el9.x86_64.rpm 523 kB/s | 205 kB 00:00 2023-10-18T10:22:58.193 INFO:teuthology.orchestra.run.smithi139.stdout:(59/143): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 5.8 MB 00:00 2023-10-18T10:22:58.193 INFO:teuthology.orchestra.run.smithi125.stdout:(9/143): ceph-selinux-18.0.0-6786.gf7420d7f.el9 263 kB/s | 24 kB 00:00 2023-10-18T10:22:58.218 INFO:teuthology.orchestra.run.smithi139.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 1.9 MB/s | 228 kB 00:00 2023-10-18T10:22:58.235 INFO:teuthology.orchestra.run.smithi139.stdout:(61/143): python3-jsonpatch-1.21-16.el9.noarch. 227 kB/s | 24 kB 00:00 2023-10-18T10:22:58.251 INFO:teuthology.orchestra.run.smithi139.stdout:(62/143): python3-jsonpointer-2.0-4.el9.noarch. 311 kB/s | 18 kB 00:00 2023-10-18T10:22:58.277 INFO:teuthology.orchestra.run.smithi139.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2023-10-18T10:22:58.285 INFO:teuthology.orchestra.run.smithi125.stdout:(10/143): ceph-radosgw-18.0.0-6786.gf7420d7f.el 9.6 MB/s | 8.6 MB 00:00 2023-10-18T10:22:58.294 INFO:teuthology.orchestra.run.smithi139.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 541 kB/s | 32 kB 00:00 2023-10-18T10:22:58.385 INFO:teuthology.orchestra.run.smithi125.stdout:(11/143): cephfs-mirror-18.0.0-6786.gf7420d7f.e 2.1 MB/s | 218 kB 00:00 2023-10-18T10:22:58.453 INFO:teuthology.orchestra.run.smithi139.stdout:(65/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2023-10-18T10:22:58.478 INFO:teuthology.orchestra.run.smithi139.stdout:(66/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2023-10-18T10:22:58.486 INFO:teuthology.orchestra.run.smithi125.stdout:(12/143): libcephfs-devel-18.0.0-6786.gf7420d7f 311 kB/s | 31 kB 00:00 2023-10-18T10:22:58.503 INFO:teuthology.orchestra.run.smithi139.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 846 kB/s | 174 kB 00:00 2023-10-18T10:22:58.520 INFO:teuthology.orchestra.run.smithi139.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 616 kB/s | 41 kB 00:00 2023-10-18T10:22:58.553 INFO:teuthology.orchestra.run.smithi139.stdout:(69/143): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2023-10-18T10:22:58.578 INFO:teuthology.orchestra.run.smithi139.stdout:(70/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-10-18T10:22:58.595 INFO:teuthology.orchestra.run.smithi139.stdout:(71/143): python3-requests-oauthlib-1.3.0-12.el 572 kB/s | 43 kB 00:00 2023-10-18T10:22:58.837 INFO:teuthology.orchestra.run.smithi139.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 162 kB/s | 39 kB 00:00 2023-10-18T10:22:58.862 INFO:teuthology.orchestra.run.smithi139.stdout:(73/143): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2023-10-18T10:22:59.069 INFO:teuthology.orchestra.run.smithi125.stdout:(13/143): ceph-osd-18.0.0-6786.gf7420d7f.el9.x8 9.0 MB/s | 18 MB 00:01 2023-10-18T10:22:59.071 INFO:teuthology.orchestra.run.smithi139.stdout:(74/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2023-10-18T10:22:59.096 INFO:teuthology.orchestra.run.smithi139.stdout:(75/143): uuid-1.6.2-55.el9.x86_64.rpm 218 kB/s | 56 kB 00:00 2023-10-18T10:22:59.113 INFO:teuthology.orchestra.run.smithi139.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 252 kB/s | 63 kB 00:00 2023-10-18T10:22:59.119 INFO:teuthology.orchestra.run.smithi125.stdout:(14/143): libcephfs2-18.0.0-6786.gf7420d7f.el9. 1.1 MB/s | 716 kB 00:00 2023-10-18T10:22:59.138 INFO:teuthology.orchestra.run.smithi139.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2023-10-18T10:22:59.144 INFO:teuthology.orchestra.run.smithi125.stdout:(15/143): libcephsqlite-18.0.0-6786.gf7420d7f.e 2.1 MB/s | 164 kB 00:00 2023-10-18T10:22:59.164 INFO:teuthology.orchestra.run.smithi139.stdout:(78/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2023-10-18T10:22:59.195 INFO:teuthology.orchestra.run.smithi125.stdout:(16/143): librados-devel-18.0.0-6786.gf7420d7f. 1.6 MB/s | 127 kB 00:00 2023-10-18T10:22:59.198 INFO:teuthology.orchestra.run.smithi139.stdout:(79/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2023-10-18T10:22:59.223 INFO:teuthology.orchestra.run.smithi139.stdout:(80/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2023-10-18T10:22:59.228 INFO:teuthology.orchestra.run.smithi125.stdout:(17/143): libradosstriper1-18.0.0-6786.gf7420d7 5.5 MB/s | 472 kB 00:00 2023-10-18T10:22:59.248 INFO:teuthology.orchestra.run.smithi139.stdout:(81/143): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2023-10-18T10:22:59.273 INFO:teuthology.orchestra.run.smithi139.stdout:(82/143): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2023-10-18T10:22:59.298 INFO:teuthology.orchestra.run.smithi139.stdout:(83/143): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-10-18T10:22:59.303 INFO:teuthology.orchestra.run.smithi125.stdout:(18/143): python3-ceph-argparse-18.0.0-6786.gf7 602 kB/s | 45 kB 00:00 2023-10-18T10:22:59.324 INFO:teuthology.orchestra.run.smithi139.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 2.4 MB/s | 188 kB 00:00 2023-10-18T10:22:59.387 INFO:teuthology.orchestra.run.smithi125.stdout:(19/143): python3-ceph-common-18.0.0-6786.gf742 1.5 MB/s | 127 kB 00:00 2023-10-18T10:22:59.479 INFO:teuthology.orchestra.run.smithi125.stdout:(20/143): python3-cephfs-18.0.0-6786.gf7420d7f. 1.7 MB/s | 162 kB 00:00 2023-10-18T10:22:59.571 INFO:teuthology.orchestra.run.smithi125.stdout:(21/143): python3-rados-18.0.0-6786.gf7420d7f.e 3.4 MB/s | 321 kB 00:00 2023-10-18T10:22:59.663 INFO:teuthology.orchestra.run.smithi125.stdout:(22/143): python3-rbd-18.0.0-6786.gf7420d7f.el9 3.2 MB/s | 297 kB 00:00 2023-10-18T10:22:59.730 INFO:teuthology.orchestra.run.smithi125.stdout:(23/143): librgw2-18.0.0-6786.gf7420d7f.el9.x86 8.9 MB/s | 4.8 MB 00:00 2023-10-18T10:22:59.756 INFO:teuthology.orchestra.run.smithi125.stdout:(24/143): python3-rgw-18.0.0-6786.gf7420d7f.el9 1.1 MB/s | 99 kB 00:00 2023-10-18T10:22:59.789 INFO:teuthology.orchestra.run.smithi125.stdout:(25/143): rbd-fuse-18.0.0-6786.gf7420d7f.el9.x8 1.4 MB/s | 86 kB 00:00 2023-10-18T10:22:59.881 INFO:teuthology.orchestra.run.smithi125.stdout:(26/143): rbd-nbd-18.0.0-6786.gf7420d7f.el9.x86 1.8 MB/s | 169 kB 00:00 2023-10-18T10:22:59.965 INFO:teuthology.orchestra.run.smithi125.stdout:(27/143): ceph-grafana-dashboards-18.0.0-6786.g 279 kB/s | 23 kB 00:00 2023-10-18T10:23:00.058 INFO:teuthology.orchestra.run.smithi125.stdout:(28/143): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 1.4 MB/s | 132 kB 00:00 2023-10-18T10:23:00.116 INFO:teuthology.orchestra.run.smithi125.stdout:(29/143): rbd-mirror-18.0.0-6786.gf7420d7f.el9. 8.4 MB/s | 3.0 MB 00:00 2023-10-18T10:23:00.341 INFO:teuthology.orchestra.run.smithi125.stdout:(30/143): ceph-mgr-dashboard-18.0.0-6786.gf7420 6.2 MB/s | 1.8 MB 00:00 2023-10-18T10:23:00.433 INFO:teuthology.orchestra.run.smithi125.stdout:(31/143): ceph-mgr-modules-core-18.0.0-6786.gf7 2.6 MB/s | 242 kB 00:00 2023-10-18T10:23:00.449 INFO:teuthology.orchestra.run.smithi139.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 20 kB/s | 22 kB 00:01 2023-10-18T10:23:00.508 INFO:teuthology.orchestra.run.smithi125.stdout:(32/143): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 636 kB/s | 47 kB 00:00 2023-10-18T10:23:00.541 INFO:teuthology.orchestra.run.smithi139.stdout:(86/143): automake-1.16.2-8.el9.noarch.rpm 555 kB/s | 689 kB 00:01 2023-10-18T10:23:00.641 INFO:teuthology.orchestra.run.smithi139.stdout:(87/143): jq-1.6-15.el9.x86_64.rpm 976 kB/s | 187 kB 00:00 2023-10-18T10:23:00.666 INFO:teuthology.orchestra.run.smithi139.stdout:(88/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 365 kB/s | 45 kB 00:00 2023-10-18T10:23:00.816 INFO:teuthology.orchestra.run.smithi139.stdout:(89/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 146 kB/s | 26 kB 00:00 2023-10-18T10:23:00.866 INFO:teuthology.orchestra.run.smithi125.stdout:(33/143): ceph-test-18.0.0-6786.gf7420d7f.el9.x 15 MB/s | 41 MB 00:02 2023-10-18T10:23:00.883 INFO:teuthology.orchestra.run.smithi125.stdout:(34/143): ceph-prometheus-alerts-18.0.0-6786.gf 40 kB/s | 15 kB 00:00 2023-10-18T10:23:00.942 INFO:teuthology.orchestra.run.smithi139.stdout:(90/143): python3-pytz-2021.1-5.el9.noarch.rpm 409 kB/s | 51 kB 00:00 2023-10-18T10:23:00.942 INFO:teuthology.orchestra.run.smithi125.stdout:(35/143): cephadm-18.0.0-6786.gf7420d7f.el9.noa 4.5 MB/s | 270 kB 00:00 2023-10-18T10:23:00.993 INFO:teuthology.orchestra.run.smithi125.stdout:(36/143): ceph-volume-18.0.0-6786.gf7420d7f.el9 2.0 MB/s | 262 kB 00:00 2023-10-18T10:23:01.018 INFO:teuthology.orchestra.run.smithi125.stdout:(37/143): cephfs-top-18.0.0-6786.gf7420d7f.el9. 243 kB/s | 18 kB 00:00 2023-10-18T10:23:01.042 INFO:teuthology.orchestra.run.smithi139.stdout:(91/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 866 kB/s | 86 kB 00:00 2023-10-18T10:23:01.100 INFO:teuthology.orchestra.run.smithi139.stdout:(92/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 10 MB/s | 4.1 MB 00:00 2023-10-18T10:23:01.109 INFO:teuthology.orchestra.run.smithi125.stdout:(38/143): ceph-mgr-diskprediction-local-18.0.0- 7.4 MB/s | 7.4 MB 00:00 2023-10-18T10:23:01.187 INFO:teuthology.orchestra.run.smithi139.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 311 kB/s | 45 kB 00:00 2023-10-18T10:23:01.253 INFO:teuthology.orchestra.run.smithi125.stdout:(39/143): boost-program-options-1.75.0-8.el9.x8 454 kB/s | 106 kB 00:00 2023-10-18T10:23:01.293 INFO:teuthology.orchestra.run.smithi125.stdout:(40/143): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 920 kB 00:00 2023-10-18T10:23:01.318 INFO:teuthology.orchestra.run.smithi125.stdout:(41/143): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 307 kB 00:00 2023-10-18T10:23:01.344 INFO:teuthology.orchestra.run.smithi125.stdout:(42/143): flexiblas-3.0.4-8.el9.x86_64.rpm 347 kB/s | 32 kB 00:00 2023-10-18T10:23:01.396 INFO:teuthology.orchestra.run.smithi139.stdout:(94/143): python3-cryptography-36.0.1-4.el9.x86 599 kB/s | 1.2 MB 00:02 2023-10-18T10:23:01.427 INFO:teuthology.orchestra.run.smithi125.stdout:(43/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2023-10-18T10:23:01.453 INFO:teuthology.orchestra.run.smithi125.stdout:(44/143): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 10 kB 00:00 2023-10-18T10:23:01.478 INFO:teuthology.orchestra.run.smithi125.stdout:(45/143): flexiblas-openblas-openmp-3.0.4-8.el9 130 kB/s | 17 kB 00:00 2023-10-18T10:23:01.503 INFO:teuthology.orchestra.run.smithi125.stdout:(46/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-10-18T10:23:01.504 INFO:teuthology.orchestra.run.smithi139.stdout:(95/143): fmt-8.1.1-5.el9.x86_64.rpm 291 kB/s | 111 kB 00:00 2023-10-18T10:23:01.528 INFO:teuthology.orchestra.run.smithi125.stdout:(47/143): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2023-10-18T10:23:01.562 INFO:teuthology.orchestra.run.smithi125.stdout:(48/143): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2023-10-18T10:23:01.580 INFO:teuthology.orchestra.run.smithi139.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 323 kB/s | 24 kB 00:00 2023-10-18T10:23:01.587 INFO:teuthology.orchestra.run.smithi125.stdout:(49/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 561 kB/s | 46 kB 00:00 2023-10-18T10:23:01.612 INFO:teuthology.orchestra.run.smithi125.stdout:(50/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2023-10-18T10:23:01.637 INFO:teuthology.orchestra.run.smithi125.stdout:(51/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2023-10-18T10:23:01.639 INFO:teuthology.orchestra.run.smithi139.stdout:(97/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 721 kB/s | 309 kB 00:00 2023-10-18T10:23:01.662 INFO:teuthology.orchestra.run.smithi125.stdout:(52/143): openblas-0.3.21-2.el9.x86_64.rpm 457 kB/s | 34 kB 00:00 2023-10-18T10:23:01.664 INFO:teuthology.orchestra.run.smithi139.stdout:(98/143): liboath-2.6.7-2.el9.x86_64.rpm 578 kB/s | 49 kB 00:00 2023-10-18T10:23:01.723 INFO:teuthology.orchestra.run.smithi139.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2023-10-18T10:23:01.746 INFO:teuthology.orchestra.run.smithi125.stdout:(53/143): postgresql-contrib-13.11-1.el9.x86_64 9.8 MB/s | 811 kB 00:00 2023-10-18T10:23:01.821 INFO:teuthology.orchestra.run.smithi125.stdout:(54/143): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2023-10-18T10:23:01.863 INFO:teuthology.orchestra.run.smithi125.stdout:(55/143): postgresql-13.11-1.el9.x86_64.rpm 7.7 MB/s | 1.5 MB 00:00 2023-10-18T10:23:01.864 INFO:teuthology.orchestra.run.smithi139.stdout:(100/143): python3-asyncssh-2.13.2-3.el9.noarch 3.8 MB/s | 548 kB 00:00 2023-10-18T10:23:01.897 INFO:teuthology.orchestra.run.smithi125.stdout:(56/143): postgresql-private-libs-13.11-1.el9.x 893 kB/s | 134 kB 00:00 2023-10-18T10:23:01.907 INFO:teuthology.orchestra.run.smithi139.stdout:(101/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.4 MB/s | 837 kB 00:00 2023-10-18T10:23:01.957 INFO:teuthology.orchestra.run.smithi139.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2023-10-18T10:23:02.049 INFO:teuthology.orchestra.run.smithi139.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 226 kB/s | 32 kB 00:00 2023-10-18T10:23:02.066 INFO:teuthology.orchestra.run.smithi139.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2023-10-18T10:23:02.147 INFO:teuthology.orchestra.run.smithi125.stdout:(57/143): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2023-10-18T10:23:02.149 INFO:teuthology.orchestra.run.smithi139.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2023-10-18T10:23:02.217 INFO:teuthology.orchestra.run.smithi139.stdout:(106/143): libarrow-9.0.0-9.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2023-10-18T10:23:02.222 INFO:teuthology.orchestra.run.smithi125.stdout:(58/143): postgresql-server-13.11-1.el9.x86_64. 16 MB/s | 5.7 MB 00:00 2023-10-18T10:23:02.241 INFO:teuthology.orchestra.run.smithi139.stdout:(107/143): python3-google-auth-2.23.2-1.el9.noa 2.1 MB/s | 196 kB 00:00 2023-10-18T10:23:02.247 INFO:teuthology.orchestra.run.smithi125.stdout:(59/143): python3-devel-3.9.17-2.el9.x86_64.rpm 588 kB/s | 205 kB 00:00 2023-10-18T10:23:02.267 INFO:teuthology.orchestra.run.smithi139.stdout:(108/143): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2023-10-18T10:23:02.273 INFO:teuthology.orchestra.run.smithi125.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 1.8 MB/s | 228 kB 00:00 2023-10-18T10:23:02.290 INFO:teuthology.orchestra.run.smithi125.stdout:(61/143): python3-jsonpointer-2.0-4.el9.noarch. 436 kB/s | 18 kB 00:00 2023-10-18T10:23:02.292 INFO:teuthology.orchestra.run.smithi139.stdout:(109/143): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2023-10-18T10:23:02.307 INFO:teuthology.orchestra.run.smithi125.stdout:(62/143): python3-jsonpatch-1.21-16.el9.noarch. 418 kB/s | 24 kB 00:00 2023-10-18T10:23:02.318 INFO:teuthology.orchestra.run.smithi139.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 233 kB/s | 18 kB 00:00 2023-10-18T10:23:02.332 INFO:teuthology.orchestra.run.smithi125.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2023-10-18T10:23:02.343 INFO:teuthology.orchestra.run.smithi139.stdout:(111/143): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2023-10-18T10:23:02.349 INFO:teuthology.orchestra.run.smithi125.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 540 kB/s | 32 kB 00:00 2023-10-18T10:23:02.368 INFO:teuthology.orchestra.run.smithi139.stdout:(112/143): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2023-10-18T10:23:02.374 INFO:teuthology.orchestra.run.smithi125.stdout:(65/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.1 MB/s | 217 kB 00:00 2023-10-18T10:23:02.410 INFO:teuthology.orchestra.run.smithi139.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2023-10-18T10:23:02.452 INFO:teuthology.orchestra.run.smithi139.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 83 kB/s | 9.0 kB 00:00 2023-10-18T10:23:02.477 INFO:teuthology.orchestra.run.smithi139.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 376 kB/s | 41 kB 00:00 2023-10-18T10:23:02.524 INFO:teuthology.orchestra.run.smithi125.stdout:(66/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-10-18T10:23:02.544 INFO:teuthology.orchestra.run.smithi139.stdout:(116/143): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2023-10-18T10:23:02.549 INFO:teuthology.orchestra.run.smithi125.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 872 kB/s | 174 kB 00:00 2023-10-18T10:23:02.566 INFO:teuthology.orchestra.run.smithi125.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 216 kB/s | 41 kB 00:00 2023-10-18T10:23:02.569 INFO:teuthology.orchestra.run.smithi139.stdout:(117/143): python3-more-itertools-8.12.0-2.el9. 859 kB/s | 79 kB 00:00 2023-10-18T10:23:02.592 INFO:teuthology.orchestra.run.smithi125.stdout:(69/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2023-10-18T10:23:02.603 INFO:teuthology.orchestra.run.smithi139.stdout:(118/143): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2023-10-18T10:23:02.608 INFO:teuthology.orchestra.run.smithi125.stdout:(70/143): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2023-10-18T10:23:02.634 INFO:teuthology.orchestra.run.smithi125.stdout:(71/143): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2023-10-18T10:23:02.636 INFO:teuthology.orchestra.run.smithi139.stdout:(119/143): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2023-10-18T10:23:02.670 INFO:teuthology.orchestra.run.smithi139.stdout:(120/143): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2023-10-18T10:23:02.704 INFO:teuthology.orchestra.run.smithi139.stdout:(121/143): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2023-10-18T10:23:02.746 INFO:teuthology.orchestra.run.smithi139.stdout:(122/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2023-10-18T10:23:02.779 INFO:teuthology.orchestra.run.smithi139.stdout:(123/143): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2023-10-18T10:23:02.821 INFO:teuthology.orchestra.run.smithi139.stdout:(124/143): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2023-10-18T10:23:02.863 INFO:teuthology.orchestra.run.smithi139.stdout:(125/143): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2023-10-18T10:23:02.880 INFO:teuthology.orchestra.run.smithi139.stdout:(126/143): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2023-10-18T10:23:02.914 INFO:teuthology.orchestra.run.smithi139.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2023-10-18T10:23:02.951 INFO:teuthology.orchestra.run.smithi125.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 113 kB/s | 39 kB 00:00 2023-10-18T10:23:02.972 INFO:teuthology.orchestra.run.smithi139.stdout:(128/143): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2023-10-18T10:23:02.998 INFO:teuthology.orchestra.run.smithi139.stdout:(129/143): python3-websocket-client-1.2.3-2.el9 765 kB/s | 90 kB 00:00 2023-10-18T10:23:03.031 INFO:teuthology.orchestra.run.smithi139.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2023-10-18T10:23:03.048 INFO:teuthology.orchestra.run.smithi139.stdout:(131/143): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2023-10-18T10:23:03.073 INFO:teuthology.orchestra.run.smithi139.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2023-10-18T10:23:03.159 INFO:teuthology.orchestra.run.smithi125.stdout:(73/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2023-10-18T10:23:03.182 INFO:teuthology.orchestra.run.smithi139.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2023-10-18T10:23:03.184 INFO:teuthology.orchestra.run.smithi125.stdout:(74/143): socat-1.7.4.1-5.el9.x86_64.rpm 546 kB/s | 300 kB 00:00 2023-10-18T10:23:03.201 INFO:teuthology.orchestra.run.smithi125.stdout:(75/143): uuid-1.6.2-55.el9.x86_64.rpm 226 kB/s | 56 kB 00:00 2023-10-18T10:23:03.218 INFO:teuthology.orchestra.run.smithi125.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2023-10-18T10:23:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-10-18T10:23:03.244 INFO:teuthology.orchestra.run.smithi125.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2023-10-18T10:23:03.285 INFO:teuthology.orchestra.run.smithi125.stdout:(78/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2023-10-18T10:23:03.311 INFO:teuthology.orchestra.run.smithi125.stdout:(79/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2023-10-18T10:23:03.336 INFO:teuthology.orchestra.run.smithi125.stdout:(80/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-10-18T10:23:03.349 INFO:teuthology.orchestra.run.smithi139.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 18 MB/s | 2.2 MB 00:00 2023-10-18T10:23:03.361 INFO:teuthology.orchestra.run.smithi125.stdout:(81/143): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2023-10-18T10:23:03.386 INFO:teuthology.orchestra.run.smithi125.stdout:(82/143): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2023-10-18T10:23:03.412 INFO:teuthology.orchestra.run.smithi125.stdout:(83/143): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-10-18T10:23:03.437 INFO:teuthology.orchestra.run.smithi125.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 2.4 MB/s | 188 kB 00:00 2023-10-18T10:23:03.466 INFO:teuthology.orchestra.run.smithi139.stdout:(136/143): librbd1-18.0.0-6786.gf7420d7f.el9.x8 7.7 MB/s | 3.0 MB 00:00 2023-10-18T10:23:03.508 INFO:teuthology.orchestra.run.smithi139.stdout:(137/143): NetworkManager-libnm-1.45.4-1.el9.x8 11 MB/s | 1.8 MB 00:00 2023-10-18T10:23:03.525 INFO:teuthology.orchestra.run.smithi139.stdout:(138/143): NetworkManager-team-1.45.4-1.el9.x86 709 kB/s | 41 kB 00:00 2023-10-18T10:23:03.545 INFO:teuthology.orchestra.run.smithi125.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 206 kB/s | 22 kB 00:00 2023-10-18T10:23:03.575 INFO:teuthology.orchestra.run.smithi139.stdout:(139/143): librados2-18.0.0-6786.gf7420d7f.el9. 6.3 MB/s | 3.3 MB 00:00 2023-10-18T10:23:03.600 INFO:teuthology.orchestra.run.smithi139.stdout:(140/143): NetworkManager-tui-1.45.4-1.el9.x86_ 2.6 MB/s | 244 kB 00:00 2023-10-18T10:23:03.626 INFO:teuthology.orchestra.run.smithi139.stdout:(141/143): acl-2.3.1-4.el9.x86_64.rpm 685 kB/s | 69 kB 00:00 2023-10-18T10:23:03.642 INFO:teuthology.orchestra.run.smithi139.stdout:(142/143): libacl-2.3.1-4.el9.x86_64.rpm 329 kB/s | 22 kB 00:00 2023-10-18T10:23:03.684 INFO:teuthology.orchestra.run.smithi139.stdout:(143/143): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2023-10-18T10:23:03.687 INFO:teuthology.orchestra.run.smithi125.stdout:(86/143): jq-1.6-15.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2023-10-18T10:23:03.691 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2023-10-18T10:23:03.691 INFO:teuthology.orchestra.run.smithi139.stdout:Total 16 MB/s | 200 MB 00:12 2023-10-18T10:23:03.829 INFO:teuthology.orchestra.run.smithi125.stdout:(87/143): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2023-10-18T10:23:03.833 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2023-10-18T10:23:03.888 INFO:teuthology.orchestra.run.smithi125.stdout:(88/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 438 kB/s | 26 kB 00:00 2023-10-18T10:23:03.957 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2023-10-18T10:23:03.958 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2023-10-18T10:23:03.971 INFO:teuthology.orchestra.run.smithi125.stdout:(89/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 159 kB/s | 45 kB 00:00 2023-10-18T10:23:04.147 INFO:teuthology.orchestra.run.smithi125.stdout:(90/143): python3-pytz-2021.1-5.el9.noarch.rpm 291 kB/s | 51 kB 00:00 2023-10-18T10:23:04.372 INFO:teuthology.orchestra.run.smithi125.stdout:(91/143): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:00 2023-10-18T10:23:04.397 INFO:teuthology.orchestra.run.smithi125.stdout:(92/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 344 kB/s | 86 kB 00:00 2023-10-18T10:23:04.497 INFO:teuthology.orchestra.run.smithi125.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2023-10-18T10:23:04.849 INFO:teuthology.orchestra.run.smithi125.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 245 kB/s | 111 kB 00:00 2023-10-18T10:23:04.982 INFO:teuthology.orchestra.run.smithi125.stdout:(95/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 3.8 MB/s | 4.1 MB 00:01 2023-10-18T10:23:05.057 INFO:teuthology.orchestra.run.smithi125.stdout:(96/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 552 kB/s | 309 kB 00:00 2023-10-18T10:23:05.158 INFO:teuthology.orchestra.run.smithi125.stdout:(97/143): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2023-10-18T10:23:05.275 INFO:teuthology.orchestra.run.smithi125.stdout:(98/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 83 kB/s | 24 kB 00:00 2023-10-18T10:23:05.301 INFO:teuthology.orchestra.run.smithi125.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 473 kB/s | 67 kB 00:00 2023-10-18T10:23:05.451 INFO:teuthology.orchestra.run.smithi125.stdout:(100/143): libarrow-9.0.0-9.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2023-10-18T10:23:05.501 INFO:teuthology.orchestra.run.smithi125.stdout:(101/143): python3-asyncssh-2.13.2-3.el9.noarch 2.7 MB/s | 548 kB 00:00 2023-10-18T10:23:05.543 INFO:teuthology.orchestra.run.smithi125.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 474 kB/s | 43 kB 00:00 2023-10-18T10:23:05.593 INFO:teuthology.orchestra.run.smithi125.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2023-10-18T10:23:05.635 INFO:teuthology.orchestra.run.smithi125.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2023-10-18T10:23:05.702 INFO:teuthology.orchestra.run.smithi125.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2023-10-18T10:23:05.706 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2023-10-18T10:23:05.707 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2023-10-18T10:23:05.735 INFO:teuthology.orchestra.run.smithi125.stdout:(106/143): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2023-10-18T10:23:05.777 INFO:teuthology.orchestra.run.smithi125.stdout:(107/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.6 MB/s | 837 kB 00:00 2023-10-18T10:23:05.811 INFO:teuthology.orchestra.run.smithi125.stdout:(108/143): python3-google-auth-2.23.2-1.el9.noa 1.8 MB/s | 196 kB 00:00 2023-10-18T10:23:05.828 INFO:teuthology.orchestra.run.smithi125.stdout:(109/143): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2023-10-18T10:23:05.870 INFO:teuthology.orchestra.run.smithi125.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2023-10-18T10:23:05.903 INFO:teuthology.orchestra.run.smithi125.stdout:(111/143): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2023-10-18T10:23:05.920 INFO:teuthology.orchestra.run.smithi125.stdout:(112/143): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2023-10-18T10:23:05.962 INFO:teuthology.orchestra.run.smithi125.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2023-10-18T10:23:05.996 INFO:teuthology.orchestra.run.smithi125.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 98 kB/s | 9.0 kB 00:00 2023-10-18T10:23:06.021 INFO:teuthology.orchestra.run.smithi125.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 406 kB/s | 41 kB 00:00 2023-10-18T10:23:06.096 INFO:teuthology.orchestra.run.smithi125.stdout:(116/143): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2023-10-18T10:23:06.122 INFO:teuthology.orchestra.run.smithi125.stdout:(117/143): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2023-10-18T10:23:06.164 INFO:teuthology.orchestra.run.smithi125.stdout:(118/143): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2023-10-18T10:23:06.197 INFO:teuthology.orchestra.run.smithi125.stdout:(119/143): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2023-10-18T10:23:06.239 INFO:teuthology.orchestra.run.smithi125.stdout:(120/143): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2023-10-18T10:23:06.264 INFO:teuthology.orchestra.run.smithi125.stdout:(121/143): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2023-10-18T10:23:06.290 INFO:teuthology.orchestra.run.smithi125.stdout:(122/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 975 kB/s | 90 kB 00:00 2023-10-18T10:23:06.340 INFO:teuthology.orchestra.run.smithi125.stdout:(123/143): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2023-10-18T10:23:06.382 INFO:teuthology.orchestra.run.smithi125.stdout:(124/143): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2023-10-18T10:23:06.407 INFO:teuthology.orchestra.run.smithi125.stdout:(125/143): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2023-10-18T10:23:06.441 INFO:teuthology.orchestra.run.smithi125.stdout:(126/143): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2023-10-18T10:23:06.483 INFO:teuthology.orchestra.run.smithi125.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2023-10-18T10:23:06.508 INFO:teuthology.orchestra.run.smithi125.stdout:(128/143): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2023-10-18T10:23:06.541 INFO:teuthology.orchestra.run.smithi125.stdout:(129/143): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2023-10-18T10:23:06.592 INFO:teuthology.orchestra.run.smithi125.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2023-10-18T10:23:06.617 INFO:teuthology.orchestra.run.smithi125.stdout:(131/143): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2023-10-18T10:23:06.651 INFO:teuthology.orchestra.run.smithi125.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2023-10-18T10:23:06.784 INFO:teuthology.orchestra.run.smithi125.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2023-10-18T10:23:06.819 INFO:teuthology.orchestra.run.smithi125.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-10-18T10:23:06.902 INFO:teuthology.orchestra.run.smithi125.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 27 MB/s | 2.2 MB 00:00 2023-10-18T10:23:06.977 INFO:teuthology.orchestra.run.smithi125.stdout:(136/143): NetworkManager-libnm-1.45.4-1.el9.x8 24 MB/s | 1.8 MB 00:00 2023-10-18T10:23:07.027 INFO:teuthology.orchestra.run.smithi125.stdout:(137/143): librados2-18.0.0-6786.gf7420d7f.el9. 8.1 MB/s | 3.3 MB 00:00 2023-10-18T10:23:07.052 INFO:teuthology.orchestra.run.smithi125.stdout:(138/143): NetworkManager-team-1.45.4-1.el9.x86 551 kB/s | 41 kB 00:00 2023-10-18T10:23:07.078 INFO:teuthology.orchestra.run.smithi125.stdout:(139/143): NetworkManager-tui-1.45.4-1.el9.x86_ 4.8 MB/s | 244 kB 00:00 2023-10-18T10:23:07.103 INFO:teuthology.orchestra.run.smithi125.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2023-10-18T10:23:07.128 INFO:teuthology.orchestra.run.smithi125.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 439 kB/s | 22 kB 00:00 2023-10-18T10:23:07.153 INFO:teuthology.orchestra.run.smithi125.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2023-10-18T10:23:07.212 INFO:teuthology.orchestra.run.smithi125.stdout:(143/143): librbd1-18.0.0-6786.gf7420d7f.el9.x8 5.4 MB/s | 3.0 MB 00:00 2023-10-18T10:23:07.218 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2023-10-18T10:23:07.218 INFO:teuthology.orchestra.run.smithi125.stdout:Total 16 MB/s | 200 MB 00:12 2023-10-18T10:23:07.361 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2023-10-18T10:23:07.486 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2023-10-18T10:23:07.487 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2023-10-18T10:23:09.220 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2023-10-18T10:23:09.221 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2023-10-18T10:23:11.471 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2023-10-18T10:23:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-18T10:23:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-18T10:23:11.614 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-18T10:23:11.661 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-18T10:23:11.986 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-18T10:23:12.039 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T10:23:12.141 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T10:23:12.173 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T10:23:12.251 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T10:23:12.298 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 8/153 2023-10-18T10:23:12.334 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-18T10:23:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-18T10:23:12.550 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-18T10:23:12.578 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T10:23:12.679 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T10:23:12.704 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 13/153 2023-10-18T10:23:12.765 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 13/153 2023-10-18T10:23:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-18T10:23:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-18T10:23:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-18T10:23:13.246 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T10:23:13.303 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T10:23:13.580 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T10:23:13.624 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-18T10:23:13.658 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-18T10:23:13.729 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-18T10:23:13.781 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-18T10:23:13.811 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-18T10:23:13.856 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-18T10:23:13.928 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-18T10:23:13.971 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-18T10:23:13.998 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-18T10:23:14.060 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-18T10:23:14.095 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9. 27/153 2023-10-18T10:23:14.139 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 28/153 2023-10-18T10:23:14.212 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_6 29/153 2023-10-18T10:23:14.347 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-18T10:23:14.398 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-18T10:23:14.805 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2023-10-18T10:23:14.872 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-18T10:23:14.897 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-18T10:23:14.923 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-18T10:23:14.932 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-18T10:23:14.981 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-18T10:23:15.003 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-18T10:23:15.040 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-18T10:23:15.086 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_6 35/153 2023-10-18T10:23:15.117 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-18T10:23:15.223 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-18T10:23:15.253 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-18T10:23:15.309 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-18T10:23:15.321 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-18T10:23:15.381 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-18T10:23:15.382 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T10:23:15.412 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-18T10:23:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-18T10:23:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-18T10:23:15.501 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T10:23:15.533 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T10:23:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-18T10:23:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-18T10:23:15.611 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T10:23:15.660 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 8/153 2023-10-18T10:23:15.694 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-18T10:23:15.718 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-18T10:23:15.760 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-18T10:23:15.801 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-18T10:23:15.849 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-18T10:23:15.895 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-18T10:23:15.911 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-18T10:23:15.943 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 50/153 2023-10-18T10:23:15.984 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-18T10:23:16.017 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-18T10:23:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-18T10:23:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-18T10:23:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-18T10:23:16.121 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T10:23:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-18T10:23:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-18T10:23:16.222 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T10:23:16.247 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 13/153 2023-10-18T10:23:16.270 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-18T10:23:16.308 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 13/153 2023-10-18T10:23:16.326 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-18T10:23:16.502 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-18T10:23:16.514 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-18T10:23:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-18T10:23:16.609 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-18T10:23:16.653 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-18T10:23:16.705 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-18T10:23:16.744 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-18T10:23:16.749 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T10:23:16.773 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-18T10:23:16.804 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T10:23:16.807 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-18T10:23:16.840 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-18T10:23:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-18T10:23:17.019 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-18T10:23:17.073 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T10:23:17.108 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-18T10:23:17.143 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-18T10:23:17.212 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-18T10:23:17.263 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-18T10:23:17.273 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-18T10:23:17.304 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-18T10:23:17.304 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-18T10:23:17.343 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-18T10:23:17.348 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-18T10:23:17.419 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-18T10:23:17.425 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-18T10:23:17.461 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-18T10:23:17.462 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-18T10:23:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-18T10:23:17.489 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-18T10:23:17.528 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-18T10:23:17.559 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-18T10:23:17.562 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-18T10:23:17.594 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9. 27/153 2023-10-18T10:23:17.638 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 28/153 2023-10-18T10:23:17.655 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-18T10:23:17.697 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-18T10:23:17.710 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_6 29/153 2023-10-18T10:23:17.801 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-18T10:23:17.848 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-18T10:23:17.851 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-18T10:23:17.906 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-18T10:23:18.429 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-18T10:23:18.501 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-18T10:23:18.547 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-18T10:23:18.585 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_6 35/153 2023-10-18T10:23:18.586 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-18T10:23:18.615 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-18T10:23:18.636 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-18T10:23:18.670 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-18T10:23:18.721 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-18T10:23:18.760 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-18T10:23:18.770 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-18T10:23:18.812 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-18T10:23:18.815 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-18T10:23:18.890 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-18T10:23:18.919 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-18T10:23:18.955 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-18T10:23:18.988 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-18T10:23:19.003 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-18T10:23:19.018 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-18T10:23:19.048 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-18T10:23:19.090 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-18T10:23:19.096 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-18T10:23:19.225 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-18T10:23:19.267 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-18T10:23:19.308 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-18T10:23:19.401 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-18T10:23:19.442 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 50/153 2023-10-18T10:23:19.483 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-18T10:23:19.515 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-18T10:23:19.558 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-18T10:23:19.584 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-18T10:23:19.612 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-18T10:23:19.698 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-18T10:23:19.736 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-18T10:23:19.743 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-18T10:23:19.785 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-18T10:23:19.841 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-18T10:23:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-18T10:23:20.035 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-18T10:23:20.220 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-18T10:23:20.356 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-18T10:23:20.393 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-18T10:23:20.422 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-18T10:23:20.456 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-18T10:23:20.494 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-18T10:23:20.530 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-18T10:23:20.579 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-18T10:23:20.624 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-18T10:23:20.659 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-18T10:23:20.690 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-18T10:23:20.766 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-18T10:23:20.800 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-18T10:23:20.833 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-18T10:23:20.875 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-18T10:23:20.909 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-18T10:23:20.999 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-18T10:23:21.028 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-18T10:23:21.106 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-18T10:23:21.131 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-18T10:23:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-18T10:23:21.187 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-18T10:23:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-18T10:23:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-18T10:23:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-18T10:23:21.573 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-18T10:23:21.602 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 97/153 2023-10-18T10:23:21.693 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 97/153 2023-10-18T10:23:21.723 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 98/153 2023-10-18T10:23:21.759 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-18T10:23:21.919 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-18T10:23:21.959 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-18T10:23:21.993 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-18T10:23:22.093 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-18T10:23:22.127 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-18T10:23:22.184 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-18T10:23:22.213 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 101/153 2023-10-18T10:23:22.267 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 101/153 2023-10-18T10:23:22.298 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 102/153 2023-10-18T10:23:22.302 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-18T10:23:22.333 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-18T10:23:22.411 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-18T10:23:23.069 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-18T10:23:23.192 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-18T10:23:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T10:23:24.436 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T10:23:24.447 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-18T10:23:24.543 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-18T10:23:24.585 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-18T10:23:24.614 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-18T10:23:24.641 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-18T10:23:24.943 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-18T10:23:24.963 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 97/153 2023-10-18T10:23:24.979 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T10:23:25.014 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 104/153 2023-10-18T10:23:25.053 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 97/153 2023-10-18T10:23:25.084 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 98/153 2023-10-18T10:23:25.087 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 104/153 2023-10-18T10:23:25.087 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-18T10:23:25.087 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-18T10:23:25.087 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:25.124 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 105/153 2023-10-18T10:23:25.229 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-18T10:23:25.715 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-18T10:23:25.749 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 101/153 2023-10-18T10:23:25.803 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 101/153 2023-10-18T10:23:25.825 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 102/153 2023-10-18T10:23:27.903 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T10:23:27.939 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T10:23:28.490 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T10:23:28.542 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 104/153 2023-10-18T10:23:28.618 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 104/153 2023-10-18T10:23:28.618 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-18T10:23:28.618 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-18T10:23:28.619 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:28.662 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 105/153 2023-10-18T10:23:36.550 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 105/153 2023-10-18T10:23:36.550 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2023-10-18T10:23:36.551 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2023-10-18T10:23:36.551 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2023-10-18T10:23:36.551 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2023-10-18T10:23:36.551 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2023-10-18T10:23:36.552 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2023-10-18T10:23:36.552 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2023-10-18T10:23:36.552 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:36.721 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 106/153 2023-10-18T10:23:36.733 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 106/153 2023-10-18T10:23:36.733 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:36.734 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-18T10:23:36.734 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:38.294 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 107/153 2023-10-18T10:23:38.320 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 107/153 2023-10-18T10:23:38.320 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:38.320 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-18T10:23:38.320 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-18T10:23:38.321 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-18T10:23:38.321 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:38.520 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 108/153 2023-10-18T10:23:38.542 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 108/153 2023-10-18T10:23:38.543 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:38.543 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-18T10:23:38.543 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-18T10:23:38.543 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-18T10:23:38.544 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 109/153 2023-10-18T10:23:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 109/153 2023-10-18T10:23:39.019 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:39.019 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-18T10:23:39.020 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-18T10:23:39.020 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-18T10:23:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 110/153 2023-10-18T10:23:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 110/153 2023-10-18T10:23:39.506 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 111/153 2023-10-18T10:23:39.569 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 112/153 2023-10-18T10:23:39.593 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 113/153 2023-10-18T10:23:39.815 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 113/153 2023-10-18T10:23:39.852 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 114/153 2023-10-18T10:23:40.131 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 105/153 2023-10-18T10:23:40.132 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /sys 2023-10-18T10:23:40.132 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /proc 2023-10-18T10:23:40.132 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /mnt 2023-10-18T10:23:40.132 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /var/tmp 2023-10-18T10:23:40.133 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /home 2023-10-18T10:23:40.133 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /root 2023-10-18T10:23:40.133 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /tmp 2023-10-18T10:23:40.133 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:40.296 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 106/153 2023-10-18T10:23:40.315 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 106/153 2023-10-18T10:23:40.316 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:40.316 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-18T10:23:40.316 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:40.805 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 114/153 2023-10-18T10:23:40.835 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 115/153 2023-10-18T10:23:40.933 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 115/153 2023-10-18T10:23:41.083 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 116/153 2023-10-18T10:23:41.112 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 117/153 2023-10-18T10:23:41.135 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 117/153 2023-10-18T10:23:41.136 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:41.136 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-18T10:23:41.136 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-18T10:23:41.136 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-18T10:23:41.136 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:41.180 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 118/153 2023-10-18T10:23:41.190 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 118/153 2023-10-18T10:23:41.874 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 107/153 2023-10-18T10:23:41.899 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 107/153 2023-10-18T10:23:41.899 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:41.899 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-18T10:23:41.900 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-18T10:23:41.900 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-18T10:23:41.900 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 119/153 2023-10-18T10:23:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 120/153 2023-10-18T10:23:42.036 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 120/153 2023-10-18T10:23:42.037 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:42.037 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-18T10:23:42.037 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-18T10:23:42.037 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-18T10:23:42.038 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:42.081 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 121/153 2023-10-18T10:23:42.099 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 108/153 2023-10-18T10:23:42.104 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 121/153 2023-10-18T10:23:42.104 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:42.104 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-18T10:23:42.104 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:42.122 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 108/153 2023-10-18T10:23:42.122 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:42.122 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-18T10:23:42.122 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-18T10:23:42.123 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-18T10:23:42.123 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:42.148 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 122/153 2023-10-18T10:23:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 122/153 2023-10-18T10:23:42.171 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:42.171 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-18T10:23:42.172 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-18T10:23:42.172 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-18T10:23:42.172 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:42.474 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 123/153 2023-10-18T10:23:42.496 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 123/153 2023-10-18T10:23:42.497 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:42.497 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-18T10:23:42.497 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-18T10:23:42.498 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-18T10:23:42.498 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:42.575 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 109/153 2023-10-18T10:23:42.597 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 109/153 2023-10-18T10:23:42.597 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:42.597 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-18T10:23:42.598 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-18T10:23:42.598 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-18T10:23:42.598 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:43.032 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 110/153 2023-10-18T10:23:43.057 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 110/153 2023-10-18T10:23:43.085 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 111/153 2023-10-18T10:23:43.148 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 112/153 2023-10-18T10:23:43.172 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 113/153 2023-10-18T10:23:43.390 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 113/153 2023-10-18T10:23:43.423 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 114/153 2023-10-18T10:23:44.374 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 114/153 2023-10-18T10:23:44.398 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 115/153 2023-10-18T10:23:44.496 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 115/153 2023-10-18T10:23:44.662 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 116/153 2023-10-18T10:23:44.700 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 117/153 2023-10-18T10:23:44.722 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 117/153 2023-10-18T10:23:44.723 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:44.723 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-18T10:23:44.723 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-18T10:23:44.723 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-18T10:23:44.724 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:44.767 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 118/153 2023-10-18T10:23:44.778 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 118/153 2023-10-18T10:23:45.560 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 119/153 2023-10-18T10:23:45.593 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 120/153 2023-10-18T10:23:45.616 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 120/153 2023-10-18T10:23:45.616 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:45.617 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-18T10:23:45.617 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-18T10:23:45.617 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-18T10:23:45.617 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:45.660 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 121/153 2023-10-18T10:23:45.682 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 121/153 2023-10-18T10:23:45.683 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:45.683 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-18T10:23:45.683 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:45.728 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 122/153 2023-10-18T10:23:45.750 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 122/153 2023-10-18T10:23:45.751 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:45.751 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-18T10:23:45.751 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-18T10:23:45.752 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-18T10:23:45.752 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:46.045 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 123/153 2023-10-18T10:23:46.068 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 123/153 2023-10-18T10:23:46.068 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T10:23:46.069 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-18T10:23:46.069 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-18T10:23:46.069 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-18T10:23:46.069 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:46.277 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 124/153 2023-10-18T10:23:46.318 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 125/153 2023-10-18T10:23:46.348 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 126/153 2023-10-18T10:23:46.520 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-18T10:23:46.629 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-18T10:23:46.695 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 129/153 2023-10-18T10:23:46.740 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-18T10:23:46.773 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-18T10:23:46.804 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_ 132/153 2023-10-18T10:23:46.833 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-18T10:23:46.910 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-18T10:23:46.979 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-18T10:23:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-18T10:23:47.586 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T10:23:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T10:23:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T10:23:47.733 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 138/153 2023-10-18T10:23:48.555 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-18T10:23:48.635 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-18T10:23:48.675 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-18T10:23:48.773 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-18T10:23:48.804 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-18T10:23:48.830 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-18T10:23:48.831 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-18T10:23:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-18T10:23:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-18T10:23:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-18T10:23:48.990 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-18T10:23:48.996 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T10:23:48.996 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T10:23:49.176 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T10:23:49.176 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-18T10:23:49.259 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-18T10:23:49.283 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-18T10:23:49.284 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-18T10:23:49.369 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-18T10:23:49.433 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T10:23:49.856 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 124/153 2023-10-18T10:23:49.897 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 125/153 2023-10-18T10:23:49.928 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 126/153 2023-10-18T10:23:50.101 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-18T10:23:50.210 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-18T10:23:50.265 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 129/153 2023-10-18T10:23:50.304 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-18T10:23:50.337 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-18T10:23:50.367 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_ 132/153 2023-10-18T10:23:50.397 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-18T10:23:50.463 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-18T10:23:50.538 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-18T10:23:50.570 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-18T10:23:51.156 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T10:23:51.194 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T10:23:51.214 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T10:23:51.276 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 138/153 2023-10-18T10:23:52.108 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-18T10:23:52.191 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-18T10:23:52.231 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-18T10:23:52.345 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-18T10:23:52.377 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-18T10:23:52.403 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-18T10:23:52.403 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-18T10:23:52.471 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-18T10:23:52.495 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-18T10:23:52.521 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-18T10:23:52.546 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-18T10:23:52.552 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T10:23:52.553 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T10:23:52.732 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T10:23:52.732 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-18T10:23:52.815 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-18T10:23:52.840 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-18T10:23:52.840 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-18T10:23:52.916 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-18T10:23:52.981 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T10:23:53.626 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T10:23:53.626 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 1/153 2023-10-18T10:23:53.627 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 2/153 2023-10-18T10:23:53.627 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 3/153 2023-10-18T10:23:53.627 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 4/153 2023-10-18T10:23:53.627 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/153 2023-10-18T10:23:53.627 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T10:23:53.628 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T10:23:53.628 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 8/153 2023-10-18T10:23:53.628 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 9/153 2023-10-18T10:23:53.628 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 10/153 2023-10-18T10:23:53.628 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 11/153 2023-10-18T10:23:53.628 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T10:23:53.629 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 13/153 2023-10-18T10:23:53.629 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_ 14/153 2023-10-18T10:23:53.629 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 15/153 2023-10-18T10:23:53.629 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 16/153 2023-10-18T10:23:53.629 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_6 17/153 2023-10-18T10:23:53.630 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 18/153 2023-10-18T10:23:53.630 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 19/153 2023-10-18T10:23:53.630 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/153 2023-10-18T10:23:53.630 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9. 21/153 2023-10-18T10:23:53.630 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_6 22/153 2023-10-18T10:23:53.631 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 23/153 2023-10-18T10:23:53.631 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 24/153 2023-10-18T10:23:53.631 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 25/153 2023-10-18T10:23:53.631 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 26/153 2023-10-18T10:23:53.631 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 27/153 2023-10-18T10:23:53.643 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 28/153 2023-10-18T10:23:53.643 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/153 2023-10-18T10:23:53.643 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 30/153 2023-10-18T10:23:53.643 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 31/153 2023-10-18T10:23:53.643 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/153 2023-10-18T10:23:53.644 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/153 2023-10-18T10:23:53.644 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 34/153 2023-10-18T10:23:53.644 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/153 2023-10-18T10:23:53.644 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 36/153 2023-10-18T10:23:53.644 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 37/153 2023-10-18T10:23:53.645 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 38/153 2023-10-18T10:23:53.645 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-18T10:23:53.645 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-18T10:23:53.645 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-18T10:23:53.645 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-18T10:23:53.646 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-18T10:23:53.646 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-18T10:23:53.646 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-18T10:23:53.646 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-18T10:23:53.646 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-18T10:23:53.647 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-18T10:23:53.647 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-18T10:23:53.647 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-18T10:23:53.647 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-18T10:23:53.647 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-18T10:23:53.648 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-18T10:23:53.648 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-18T10:23:53.648 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-18T10:23:53.648 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-18T10:23:53.648 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-18T10:23:53.649 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-18T10:23:53.649 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-18T10:23:53.649 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-18T10:23:53.649 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-18T10:23:53.649 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-18T10:23:53.649 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-18T10:23:53.650 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-18T10:23:53.650 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-18T10:23:53.650 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-18T10:23:53.650 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-18T10:23:53.651 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-18T10:23:53.651 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-18T10:23:53.651 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-18T10:23:53.651 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-18T10:23:53.651 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-18T10:23:53.651 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-18T10:23:53.652 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-18T10:23:53.652 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-18T10:23:53.652 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-18T10:23:53.652 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-18T10:23:53.653 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-18T10:23:53.653 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-18T10:23:53.653 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-18T10:23:53.653 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-18T10:23:53.653 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-18T10:23:53.654 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-18T10:23:53.654 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-18T10:23:53.654 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-18T10:23:53.654 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-18T10:23:53.654 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-18T10:23:53.655 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-18T10:23:53.655 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-18T10:23:53.655 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-18T10:23:53.655 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-18T10:23:53.655 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-18T10:23:53.656 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-18T10:23:53.656 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-18T10:23:53.656 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-18T10:23:53.656 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-18T10:23:53.656 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-18T10:23:53.657 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-18T10:23:53.657 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-18T10:23:53.657 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-18T10:23:53.657 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-18T10:23:53.657 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-18T10:23:53.657 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-18T10:23:53.658 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-18T10:23:53.658 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-18T10:23:53.658 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-18T10:23:53.658 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-18T10:23:53.658 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 108/153 2023-10-18T10:23:53.659 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-18T10:23:53.659 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-18T10:23:53.659 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-18T10:23:53.659 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-18T10:23:53.659 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-18T10:23:53.659 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-18T10:23:53.660 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-18T10:23:53.660 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-18T10:23:53.660 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-18T10:23:53.660 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-18T10:23:53.660 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-18T10:23:53.660 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-18T10:23:53.661 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-18T10:23:53.661 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-18T10:23:53.661 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-18T10:23:53.661 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-18T10:23:53.661 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-18T10:23:53.661 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-18T10:23:53.662 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-18T10:23:53.662 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-18T10:23:53.662 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-18T10:23:53.662 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-18T10:23:53.662 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-18T10:23:53.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-18T10:23:53.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-18T10:23:53.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 134/153 2023-10-18T10:23:53.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-18T10:23:53.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 136/153 2023-10-18T10:23:53.663 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-18T10:23:53.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-18T10:23:53.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-18T10:23:53.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-18T10:23:53.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-18T10:23:53.664 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-18T10:23:53.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-18T10:23:53.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-18T10:23:53.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-18T10:23:53.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-18T10:23:53.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-18T10:23:53.665 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-18T10:23:53.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-18T10:23:53.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-18T10:23:53.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-18T10:23:53.666 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-18T10:23:56.414 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T10:23:56.414 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:56.414 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2023-10-18T10:23:56.415 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-18T10:23:56.415 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-18T10:23:56.415 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-18T10:23:56.415 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-18T10:23:56.415 INFO:teuthology.orchestra.run.smithi139.stdout: acl-2.3.1-4.el9.x86_64 2023-10-18T10:23:56.416 INFO:teuthology.orchestra.run.smithi139.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-18T10:23:56.416 INFO:teuthology.orchestra.run.smithi139.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-18T10:23:56.416 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.416 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.416 INFO:teuthology.orchestra.run.smithi139.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-18T10:23:56.417 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2023-10-18T10:23:56.417 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2023-10-18T10:23:56.417 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2023-10-18T10:23:56.417 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-18T10:23:56.417 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.417 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.418 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.418 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.418 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.418 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.418 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.419 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.419 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.419 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.419 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.419 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.420 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.420 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.420 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.420 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.420 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.421 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.421 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.421 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.421 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.421 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.421 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:56.422 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-18T10:23:56.422 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2023-10-18T10:23:56.422 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-18T10:23:56.422 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-18T10:23:56.422 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-18T10:23:56.423 INFO:teuthology.orchestra.run.smithi139.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-18T10:23:56.423 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-18T10:23:56.423 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-18T10:23:56.423 INFO:teuthology.orchestra.run.smithi139.stdout: jq-1.6-15.el9.x86_64 2023-10-18T10:23:56.423 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-18T10:23:56.424 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-18T10:23:56.424 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-18T10:23:56.424 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.424 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.424 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.424 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-18T10:23:56.425 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-18T10:23:56.425 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-18T10:23:56.425 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-18T10:23:56.425 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-18T10:23:56.425 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.426 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.426 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-18T10:23:56.426 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.426 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-18T10:23:56.426 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-18T10:23:56.427 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-18T10:23:56.427 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-18T10:23:56.427 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-18T10:23:56.427 INFO:teuthology.orchestra.run.smithi139.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-18T10:23:56.427 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-18T10:23:56.427 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-18T10:23:56.428 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-18T10:23:56.428 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-18T10:23:56.428 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.11-1.el9.x86_64 2023-10-18T10:23:56.428 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-18T10:23:56.428 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-18T10:23:56.429 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-18T10:23:56.429 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-18T10:23:56.429 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-18T10:23:56.429 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-18T10:23:56.429 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-18T10:23:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-18T10:23:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-18T10:23:56.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-18T10:23:56.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-18T10:23:56.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-18T10:23:56.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-18T10:23:56.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-18T10:23:56.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-18T10:23:56.432 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-18T10:23:56.432 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-18T10:23:56.432 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-18T10:23:56.432 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-18T10:23:56.432 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-18T10:23:56.433 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-18T10:23:56.433 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-18T10:23:56.433 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-18T10:23:56.433 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-18T10:23:56.433 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-18T10:23:56.434 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-18T10:23:56.434 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-18T10:23:56.434 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-18T10:23:56.434 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-18T10:23:56.434 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-18T10:23:56.435 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-18T10:23:56.435 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-18T10:23:56.435 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-18T10:23:56.435 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-18T10:23:56.436 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2023-10-18T10:23:56.436 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-18T10:23:56.436 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-18T10:23:56.436 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-18T10:23:56.436 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-18T10:23:56.436 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-18T10:23:56.437 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-18T10:23:56.437 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-18T10:23:56.437 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.437 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.437 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-18T10:23:56.438 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-18T10:23:56.438 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-18T10:23:56.438 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.438 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-18T10:23:56.439 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-18T10:23:56.439 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-18T10:23:56.439 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-18T10:23:56.439 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-18T10:23:56.439 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-18T10:23:56.440 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-18T10:23:56.440 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-18T10:23:56.440 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-18T10:23:56.440 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-18T10:23:56.440 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-18T10:23:56.441 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.441 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.441 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:56.441 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-3.el9.x86_64 2023-10-18T10:23:56.441 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-18T10:23:56.441 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-18T10:23:56.442 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-18T10:23:56.442 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-18T10:23:56.442 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-18T10:23:56.443 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-18T10:23:56.443 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:23:56.443 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2023-10-18T10:23:56.853 DEBUG:teuthology.parallel:result is None 2023-10-18T10:23:57.261 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T10:23:57.261 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 1/153 2023-10-18T10:23:57.261 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 2/153 2023-10-18T10:23:57.261 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 3/153 2023-10-18T10:23:57.262 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 4/153 2023-10-18T10:23:57.262 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/153 2023-10-18T10:23:57.262 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T10:23:57.262 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T10:23:57.262 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 8/153 2023-10-18T10:23:57.263 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 9/153 2023-10-18T10:23:57.263 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 10/153 2023-10-18T10:23:57.263 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 11/153 2023-10-18T10:23:57.263 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T10:23:57.263 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 13/153 2023-10-18T10:23:57.263 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_ 14/153 2023-10-18T10:23:57.264 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 15/153 2023-10-18T10:23:57.264 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 16/153 2023-10-18T10:23:57.264 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_6 17/153 2023-10-18T10:23:57.264 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 18/153 2023-10-18T10:23:57.264 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 19/153 2023-10-18T10:23:57.265 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/153 2023-10-18T10:23:57.265 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9. 21/153 2023-10-18T10:23:57.265 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_6 22/153 2023-10-18T10:23:57.265 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 23/153 2023-10-18T10:23:57.265 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 24/153 2023-10-18T10:23:57.266 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 25/153 2023-10-18T10:23:57.266 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 26/153 2023-10-18T10:23:57.266 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 27/153 2023-10-18T10:23:57.266 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 28/153 2023-10-18T10:23:57.266 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/153 2023-10-18T10:23:57.267 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 30/153 2023-10-18T10:23:57.277 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 31/153 2023-10-18T10:23:57.277 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/153 2023-10-18T10:23:57.277 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/153 2023-10-18T10:23:57.278 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 34/153 2023-10-18T10:23:57.278 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/153 2023-10-18T10:23:57.278 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 36/153 2023-10-18T10:23:57.278 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 37/153 2023-10-18T10:23:57.278 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 38/153 2023-10-18T10:23:57.279 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-18T10:23:57.279 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-18T10:23:57.279 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-18T10:23:57.279 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-18T10:23:57.279 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-18T10:23:57.279 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-18T10:23:57.280 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-18T10:23:57.280 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-18T10:23:57.280 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-18T10:23:57.280 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-18T10:23:57.280 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-18T10:23:57.281 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-18T10:23:57.281 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-18T10:23:57.281 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-18T10:23:57.281 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-18T10:23:57.281 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-18T10:23:57.282 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-18T10:23:57.282 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-18T10:23:57.282 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-18T10:23:57.282 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-18T10:23:57.282 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-18T10:23:57.283 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-18T10:23:57.283 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-18T10:23:57.283 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-18T10:23:57.283 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-18T10:23:57.283 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-18T10:23:57.283 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-18T10:23:57.284 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-18T10:23:57.284 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-18T10:23:57.284 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-18T10:23:57.284 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-18T10:23:57.284 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-18T10:23:57.285 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-18T10:23:57.285 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-18T10:23:57.285 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-18T10:23:57.285 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-18T10:23:57.285 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-18T10:23:57.286 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-18T10:23:57.286 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-18T10:23:57.286 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-18T10:23:57.286 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-18T10:23:57.286 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-18T10:23:57.286 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-18T10:23:57.287 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-18T10:23:57.287 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-18T10:23:57.287 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-18T10:23:57.287 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-18T10:23:57.287 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-18T10:23:57.288 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-18T10:23:57.288 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-18T10:23:57.288 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-18T10:23:57.288 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-18T10:23:57.288 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-18T10:23:57.289 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-18T10:23:57.289 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-18T10:23:57.289 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-18T10:23:57.289 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-18T10:23:57.289 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-18T10:23:57.289 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-18T10:23:57.290 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-18T10:23:57.290 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-18T10:23:57.290 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-18T10:23:57.290 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-18T10:23:57.290 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-18T10:23:57.291 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-18T10:23:57.291 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-18T10:23:57.291 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-18T10:23:57.291 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-18T10:23:57.291 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-18T10:23:57.292 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 108/153 2023-10-18T10:23:57.292 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-18T10:23:57.292 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-18T10:23:57.292 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-18T10:23:57.292 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-18T10:23:57.292 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-18T10:23:57.293 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-18T10:23:57.293 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-18T10:23:57.293 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-18T10:23:57.293 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-18T10:23:57.293 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-18T10:23:57.294 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-18T10:23:57.294 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-18T10:23:57.294 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-18T10:23:57.294 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-18T10:23:57.294 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-18T10:23:57.295 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-18T10:23:57.295 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-18T10:23:57.295 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-18T10:23:57.295 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-18T10:23:57.295 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-18T10:23:57.295 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-18T10:23:57.296 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-18T10:23:57.296 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-18T10:23:57.296 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-18T10:23:57.296 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-18T10:23:57.296 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 134/153 2023-10-18T10:23:57.297 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-18T10:23:57.297 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 136/153 2023-10-18T10:23:57.297 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-18T10:23:57.297 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-18T10:23:57.297 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-18T10:23:57.297 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-18T10:23:57.298 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-18T10:23:57.298 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-18T10:23:57.298 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-18T10:23:57.298 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-18T10:23:57.298 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-18T10:23:57.299 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-18T10:23:57.299 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-18T10:23:57.299 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-18T10:23:57.299 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-18T10:23:57.299 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-18T10:23:57.300 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-18T10:23:57.300 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-18T10:23:59.882 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T10:23:59.883 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:59.883 INFO:teuthology.orchestra.run.smithi125.stdout:Upgraded: 2023-10-18T10:23:59.883 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-18T10:23:59.883 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-18T10:23:59.883 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-18T10:23:59.884 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-18T10:23:59.884 INFO:teuthology.orchestra.run.smithi125.stdout: acl-2.3.1-4.el9.x86_64 2023-10-18T10:23:59.884 INFO:teuthology.orchestra.run.smithi125.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-18T10:23:59.884 INFO:teuthology.orchestra.run.smithi125.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-18T10:23:59.884 INFO:teuthology.orchestra.run.smithi125.stdout: librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.884 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.885 INFO:teuthology.orchestra.run.smithi125.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-18T10:23:59.885 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2023-10-18T10:23:59.885 INFO:teuthology.orchestra.run.smithi125.stdout: automake-1.16.2-8.el9.noarch 2023-10-18T10:23:59.885 INFO:teuthology.orchestra.run.smithi125.stdout: bison-3.7.4-5.el9.x86_64 2023-10-18T10:23:59.885 INFO:teuthology.orchestra.run.smithi125.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-18T10:23:59.885 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.886 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.886 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.886 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.886 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.886 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.887 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.887 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.887 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.887 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.887 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.887 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.888 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.888 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.888 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.888 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.888 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.889 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.889 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.889 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.889 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.889 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.890 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T10:23:59.890 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-18T10:23:59.890 INFO:teuthology.orchestra.run.smithi125.stdout: flex-2.6.4-9.el9.x86_64 2023-10-18T10:23:59.890 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-18T10:23:59.890 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-18T10:23:59.891 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-18T10:23:59.891 INFO:teuthology.orchestra.run.smithi125.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-18T10:23:59.891 INFO:teuthology.orchestra.run.smithi125.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-18T10:23:59.891 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-18T10:23:59.891 INFO:teuthology.orchestra.run.smithi125.stdout: jq-1.6-15.el9.x86_64 2023-10-18T10:23:59.891 INFO:teuthology.orchestra.run.smithi125.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-18T10:23:59.892 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-18T10:23:59.892 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-18T10:23:59.892 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.892 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.892 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.893 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-18T10:23:59.893 INFO:teuthology.orchestra.run.smithi125.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-18T10:23:59.893 INFO:teuthology.orchestra.run.smithi125.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-18T10:23:59.893 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-18T10:23:59.893 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-18T10:23:59.893 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.894 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.894 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-18T10:23:59.894 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.894 INFO:teuthology.orchestra.run.smithi125.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-18T10:23:59.894 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-18T10:23:59.894 INFO:teuthology.orchestra.run.smithi125.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-18T10:23:59.895 INFO:teuthology.orchestra.run.smithi125.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-18T10:23:59.895 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-18T10:23:59.895 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-18T10:23:59.895 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-18T10:23:59.895 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-18T10:23:59.896 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-18T10:23:59.896 INFO:teuthology.orchestra.run.smithi125.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-18T10:23:59.896 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-13.11-1.el9.x86_64 2023-10-18T10:23:59.896 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-18T10:23:59.896 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-18T10:23:59.896 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-18T10:23:59.897 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-18T10:23:59.897 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-18T10:23:59.897 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-18T10:23:59.897 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-18T10:23:59.897 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.897 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.898 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.898 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-18T10:23:59.898 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-18T10:23:59.898 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-18T10:23:59.898 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-18T10:23:59.899 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-18T10:23:59.899 INFO:teuthology.orchestra.run.smithi125.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-18T10:23:59.899 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-18T10:23:59.899 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-18T10:23:59.899 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-18T10:23:59.900 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-18T10:23:59.900 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-18T10:23:59.900 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-18T10:23:59.900 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-18T10:23:59.900 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-18T10:23:59.901 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-18T10:23:59.901 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-18T10:23:59.901 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-18T10:23:59.901 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-18T10:23:59.901 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-18T10:23:59.901 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-18T10:23:59.902 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-18T10:23:59.902 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-18T10:23:59.902 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-18T10:23:59.902 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-18T10:23:59.902 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-18T10:23:59.903 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-18T10:23:59.903 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-18T10:23:59.903 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ply-3.11-14.el9.noarch 2023-10-18T10:23:59.903 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-18T10:23:59.903 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-18T10:23:59.904 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-18T10:23:59.904 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-18T10:23:59.904 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-18T10:23:59.904 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-18T10:23:59.904 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-18T10:23:59.905 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.905 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.905 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-18T10:23:59.905 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-18T10:23:59.905 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-18T10:23:59.905 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.906 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-18T10:23:59.906 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-18T10:23:59.906 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-18T10:23:59.906 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-18T10:23:59.906 INFO:teuthology.orchestra.run.smithi125.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-18T10:23:59.907 INFO:teuthology.orchestra.run.smithi125.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-18T10:23:59.907 INFO:teuthology.orchestra.run.smithi125.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-18T10:23:59.907 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-18T10:23:59.907 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-18T10:23:59.907 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-18T10:23:59.908 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-18T10:23:59.908 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.908 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.908 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T10:23:59.908 INFO:teuthology.orchestra.run.smithi125.stdout: re2-1:20211101-3.el9.x86_64 2023-10-18T10:23:59.908 INFO:teuthology.orchestra.run.smithi125.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-18T10:23:59.909 INFO:teuthology.orchestra.run.smithi125.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-18T10:23:59.909 INFO:teuthology.orchestra.run.smithi125.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-18T10:23:59.909 INFO:teuthology.orchestra.run.smithi125.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-18T10:23:59.909 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-18T10:23:59.910 INFO:teuthology.orchestra.run.smithi125.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-18T10:23:59.910 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:23:59.910 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2023-10-18T10:24:00.339 DEBUG:teuthology.parallel:result is None 2023-10-18T10:24:00.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T10:24:00.483 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T10:24:00.504 INFO:teuthology.orchestra.run.smithi125.stdout:18.0.0-6786.gf7420d7f.el9 2023-10-18T10:24:00.504 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el9 2023-10-18T10:24:00.505 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T10:24:00.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T10:24:00.643 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T10:24:00.668 INFO:teuthology.orchestra.run.smithi139.stdout:18.0.0-6786.gf7420d7f.el9 2023-10-18T10:24:00.669 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el9 2023-10-18T10:24:00.669 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T10:24:00.669 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-18T10:24:00.670 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:00.670 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T10:24:00.701 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:00.702 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T10:24:00.741 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-18T10:24:00.741 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:00.741 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T10:24:00.769 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T10:24:00.836 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:00.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T10:24:00.865 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T10:24:00.934 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-18T10:24:00.934 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:00.935 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T10:24:00.961 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T10:24:01.028 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:01.028 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T10:24:01.056 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T10:24:01.124 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-18T10:24:01.125 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:01.125 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T10:24:01.152 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T10:24:01.218 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:01.218 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T10:24:01.245 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T10:24:01.312 INFO:teuthology.run_tasks:Running task ceph... 2023-10-18T10:24:01.385 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-18T10:24:01.386 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 777 /var/log/ceph 2023-10-18T10:24:01.388 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /var/log/ceph 2023-10-18T10:24:01.415 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-18T10:24:01.415 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T10:24:01.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T10:24:01.481 INFO:tasks.ceph:Creating extra log directories... 2023-10-18T10:24:01.482 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T10:24:01.521 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T10:24:01.554 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-18T10:24:01.554 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-18T10:24:01.554 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-18T10:24:01.555 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:01.555 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-18T10:24:01.600 DEBUG:teuthology.orchestra.run.smithi125:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-18T10:24:01.718 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-18T10:24:01.745 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:01.745 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-18T10:24:01.760 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-18T10:24:01.876 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-18T10:24:01.907 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-18T10:24:01.908 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-18T10:24:01.908 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772', 'branch': 'wip-batrick-testing-20231016.203825', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7430772', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20231016.203825'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20231016.203825', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7703, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'sleep_before_teardown': 0, 'subset': '153/256', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20231016.203825', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi125.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgPjGOGMEwwnQSgDNGZ8Aebsniz3u1sRO9Cd/9KcNxkMp6n9mP36ADoPQ1fAXDMOtBc/tTwQAGL4sR1kzdS10w=', 'smithi139.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbVFAJYQJyVQjPaD1xZQAnsbNlp7gdhA4zpfnjzKe/rNxup6kcxf7nwkxc83xyMAEE+KyWlyHHVeaQnT9zM0VA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-16_23:59:58', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-18T10:24:01.909 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T10:24:01.911 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T10:24:01.957 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T10:24:01.966 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T10:24:02.022 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:02.023 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T10:24:02.045 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T10:24:02.045 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_1 2023-10-18T10:24:02.098 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T10:24:02.099 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:24:02.099 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 662 Links: 1 2023-10-18T10:24:02.099 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:24:02.099 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T10:24:02.099 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-10-18 10:23:53.344894574 +0000 2023-10-18T10:24:02.100 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-10-18 10:22:05.477819110 +0000 2023-10-18T10:24:02.100 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-10-18 10:22:05.477819110 +0000 2023-10-18T10:24:02.100 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2023-10-18 10:22:05.477819110 +0000 2023-10-18T10:24:02.100 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T10:24:02.163 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-10-18T10:24:02.164 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-10-18T10:24:02.164 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000238609 s, 2.1 MB/s 2023-10-18T10:24:02.165 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T10:24:02.220 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_2 2023-10-18T10:24:02.274 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T10:24:02.275 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:24:02.275 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 684 Links: 1 2023-10-18T10:24:02.275 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:24:02.275 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T10:24:02.275 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-10-18 10:23:53.344894574 +0000 2023-10-18T10:24:02.275 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-10-18 10:22:05.975811265 +0000 2023-10-18T10:24:02.276 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-10-18 10:22:05.975811265 +0000 2023-10-18T10:24:02.276 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2023-10-18 10:22:05.975811265 +0000 2023-10-18T10:24:02.276 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T10:24:02.339 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-10-18T10:24:02.339 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-10-18T10:24:02.339 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000181557 s, 2.8 MB/s 2023-10-18T10:24:02.340 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T10:24:02.395 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_3 2023-10-18T10:24:02.449 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T10:24:02.450 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:24:02.450 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 701 Links: 1 2023-10-18T10:24:02.450 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:24:02.451 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T10:24:02.451 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-10-18 10:23:53.345894558 +0000 2023-10-18T10:24:02.451 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-10-18 10:22:06.477803357 +0000 2023-10-18T10:24:02.451 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-10-18 10:22:06.477803357 +0000 2023-10-18T10:24:02.451 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2023-10-18 10:22:06.477803357 +0000 2023-10-18T10:24:02.452 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T10:24:02.514 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-10-18T10:24:02.515 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-10-18T10:24:02.515 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000193325 s, 2.6 MB/s 2023-10-18T10:24:02.516 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T10:24:02.571 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_4 2023-10-18T10:24:02.626 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T10:24:02.626 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:24:02.626 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 719 Links: 1 2023-10-18T10:24:02.626 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:24:02.627 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T10:24:02.627 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2023-10-18 10:23:53.345894558 +0000 2023-10-18T10:24:02.627 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2023-10-18 10:22:06.966795653 +0000 2023-10-18T10:24:02.627 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2023-10-18 10:22:06.966795653 +0000 2023-10-18T10:24:02.627 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2023-10-18 10:22:06.966795653 +0000 2023-10-18T10:24:02.628 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T10:24:02.691 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2023-10-18T10:24:02.692 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2023-10-18T10:24:02.692 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000220014 s, 2.3 MB/s 2023-10-18T10:24:02.693 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T10:24:02.748 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:02.749 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:02.749 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T10:24:02.765 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T10:24:02.765 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2023-10-18T10:24:02.821 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T10:24:02.821 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:24:02.821 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 665 Links: 1 2023-10-18T10:24:02.821 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:24:02.821 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T10:24:02.822 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2023-10-18 10:23:49.784587387 +0000 2023-10-18T10:24:02.822 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2023-10-18 10:22:05.485567031 +0000 2023-10-18T10:24:02.822 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2023-10-18 10:22:05.485567031 +0000 2023-10-18T10:24:02.822 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2023-10-18 10:22:05.485567031 +0000 2023-10-18T10:24:02.822 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T10:24:02.888 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2023-10-18T10:24:02.888 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2023-10-18T10:24:02.889 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000204742 s, 2.5 MB/s 2023-10-18T10:24:02.890 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T10:24:02.947 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2023-10-18T10:24:03.003 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T10:24:03.003 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:24:03.003 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 683 Links: 1 2023-10-18T10:24:03.004 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:24:03.004 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T10:24:03.004 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2023-10-18 10:23:49.784587387 +0000 2023-10-18T10:24:03.004 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2023-10-18 10:22:05.974557369 +0000 2023-10-18T10:24:03.004 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2023-10-18 10:22:05.974557369 +0000 2023-10-18T10:24:03.005 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2023-10-18 10:22:05.974557369 +0000 2023-10-18T10:24:03.005 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T10:24:03.070 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2023-10-18T10:24:03.071 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2023-10-18T10:24:03.071 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000166694 s, 3.1 MB/s 2023-10-18T10:24:03.072 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T10:24:03.128 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2023-10-18T10:24:03.184 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T10:24:03.184 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:24:03.184 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 700 Links: 1 2023-10-18T10:24:03.184 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:24:03.184 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T10:24:03.185 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2023-10-18 10:23:49.785587367 +0000 2023-10-18T10:24:03.185 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2023-10-18 10:22:06.482547330 +0000 2023-10-18T10:24:03.185 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2023-10-18 10:22:06.482547330 +0000 2023-10-18T10:24:03.185 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2023-10-18 10:22:06.482547330 +0000 2023-10-18T10:24:03.186 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T10:24:03.249 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2023-10-18T10:24:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2023-10-18T10:24:03.250 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000192365 s, 2.7 MB/s 2023-10-18T10:24:03.251 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T10:24:03.307 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2023-10-18T10:24:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T10:24:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T10:24:03.363 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 721 Links: 1 2023-10-18T10:24:03.363 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T10:24:03.363 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T10:24:03.364 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2023-10-18 10:23:49.785587367 +0000 2023-10-18T10:24:03.364 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2023-10-18 10:22:06.958537924 +0000 2023-10-18T10:24:03.364 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2023-10-18 10:22:06.958537924 +0000 2023-10-18T10:24:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2023-10-18 10:22:06.958537924 +0000 2023-10-18T10:24:03.364 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T10:24:03.428 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2023-10-18T10:24:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2023-10-18T10:24:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000188851 s, 2.7 MB/s 2023-10-18T10:24:03.431 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T10:24:03.485 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:03.486 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi125.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-18T10:24:03.486 INFO:tasks.ceph:Generating config... 2023-10-18T10:24:03.488 INFO:tasks.ceph:[client] pid file = 2023-10-18T10:24:03.488 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-18T10:24:03.488 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-18T10:24:03.488 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-18T10:24:03.489 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-18T10:24:03.489 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-18T10:24:03.489 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-18T10:24:03.489 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-18T10:24:03.490 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-18T10:24:03.490 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-18T10:24:03.490 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-18T10:24:03.491 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-18T10:24:03.491 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-18T10:24:03.491 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-18T10:24:03.491 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-18T10:24:03.492 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-18T10:24:03.492 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-18T10:24:03.492 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-18T10:24:03.492 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-18T10:24:03.493 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-18T10:24:03.493 INFO:tasks.ceph:Setting up mon.a... 2023-10-18T10:24:03.493 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-18T10:24:03.551 INFO:teuthology.orchestra.run.smithi125.stdout:creating /etc/ceph/ceph.keyring 2023-10-18T10:24:03.554 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-18T10:24:03.605 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T10:24:03.637 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.125'), ('mon.b', '172.21.15.139'), ('mon.c', '[v2:172.21.15.139:3301,v1:172.21.15.139:6790]')] 2023-10-18T10:24:03.638 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.125,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2023-10-18T10:24:03.638 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:03.641 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T10:24:03.695 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.125 --add b 172.21.15.139 --addv c '[v2:172.21.15.139:3301,v1:172.21.15.139:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:03.788 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:03.788 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: generated fsid 9708ef69-1ff8-449e-8a32-79d14c57ee5d 2023-10-18T10:24:03.789 INFO:teuthology.orchestra.run.smithi125.stdout:setting min_mon_release = pacific 2023-10-18T10:24:03.789 INFO:teuthology.orchestra.run.smithi125.stdout:epoch 0 2023-10-18T10:24:03.789 INFO:teuthology.orchestra.run.smithi125.stdout:fsid 9708ef69-1ff8-449e-8a32-79d14c57ee5d 2023-10-18T10:24:03.789 INFO:teuthology.orchestra.run.smithi125.stdout:last_changed 2023-10-18T10:24:03.787721+0000 2023-10-18T10:24:03.789 INFO:teuthology.orchestra.run.smithi125.stdout:created 2023-10-18T10:24:03.787721+0000 2023-10-18T10:24:03.789 INFO:teuthology.orchestra.run.smithi125.stdout:min_mon_release 16 (pacific) 2023-10-18T10:24:03.790 INFO:teuthology.orchestra.run.smithi125.stdout:election_strategy: 1 2023-10-18T10:24:03.790 INFO:teuthology.orchestra.run.smithi125.stdout:0: [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] mon.a 2023-10-18T10:24:03.790 INFO:teuthology.orchestra.run.smithi125.stdout:1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2023-10-18T10:24:03.790 INFO:teuthology.orchestra.run.smithi125.stdout:2: [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] mon.c 2023-10-18T10:24:03.790 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-18T10:24:03.792 DEBUG:teuthology.orchestra.run.smithi125:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T10:24:03.809 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9708ef69-1ff8-449e-8a32-79d14c57ee5d... 2023-10-18T10:24:03.810 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T10:24:03.852 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T10:24:03.904 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2023-10-18T10:24:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2023-10-18T10:24:03.904 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T10:24:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2023-10-18T10:24:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2023-10-18T10:24:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.905 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2023-10-18T10:24:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2023-10-18T10:24:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2023-10-18T10:24:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.906 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2023-10-18T10:24:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2023-10-18T10:24:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T10:24:03.907 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2023-10-18T10:24:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T10:24:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T10:24:03.908 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2023-10-18T10:24:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2023-10-18T10:24:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2023-10-18T10:24:03.909 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2023-10-18T10:24:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2023-10-18T10:24:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2023-10-18T10:24:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2023-10-18T10:24:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2023-10-18T10:24:03.910 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T10:24:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T10:24:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2023-10-18T10:24:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.911 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-18T10:24:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2023-10-18T10:24:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2023-10-18T10:24:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.912 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2023-10-18T10:24:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2023-10-18T10:24:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2023-10-18T10:24:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.125,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]" 2023-10-18T10:24:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 9708ef69-1ff8-449e-8a32-79d14c57ee5d 2023-10-18T10:24:03.913 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.914 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2023-10-18T10:24:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2023-10-18T10:24:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2023-10-18T10:24:03.914 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2023-10-18T10:24:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2023-10-18T10:24:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2023-10-18T10:24:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2023-10-18T10:24:03.915 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2023-10-18T10:24:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2023-10-18T10:24:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2023-10-18T10:24:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2023-10-18T10:24:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2023-10-18T10:24:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2023-10-18T10:24:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.917 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2023-10-18T10:24:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2023-10-18T10:24:03.918 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2023-10-18T10:24:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2023-10-18T10:24:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.919 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2023-10-18T10:24:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2023-10-18T10:24:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2023-10-18T10:24:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2023-10-18T10:24:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 20 2023-10-18T10:24:03.920 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 20 2023-10-18T10:24:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2023-10-18T10:24:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2023-10-18T10:24:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 10 2023-10-18T10:24:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T10:24:03.921 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2023-10-18T10:24:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2023-10-18T10:24:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2023-10-18T10:24:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2023-10-18T10:24:03.922 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.922 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2023-10-18T10:24:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2023-10-18T10:24:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2023-10-18T10:24:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2023-10-18T10:24:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2023-10-18T10:24:03.923 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2023-10-18T10:24:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2023-10-18T10:24:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2023-10-18T10:24:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2023-10-18T10:24:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: debug finisher = 20 2023-10-18T10:24:03.924 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.925 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2023-10-18T10:24:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2023-10-18T10:24:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2023-10-18T10:24:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2023-10-18T10:24:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2023-10-18T10:24:03.925 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2023-10-18T10:24:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2023-10-18T10:24:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2023-10-18T10:24:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2023-10-18T10:24:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2023-10-18T10:24:03.926 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2023-10-18T10:24:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T10:24:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T10:24:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T10:24:03.927 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T10:24:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T10:24:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T10:24:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.928 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2023-10-18T10:24:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2023-10-18T10:24:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2023-10-18T10:24:03.929 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-10-18T10:24:03.929 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2023-10-18T10:24:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2023-10-18T10:24:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2023-10-18T10:24:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2023-10-18T10:24:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T10:24:03.930 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T10:24:03.931 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = "" 2023-10-18T10:24:03.931 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2023-10-18T10:24:03.931 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2023-10-18T10:24:03.931 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2023-10-18T10:24:03.931 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2023-10-18T10:24:03.932 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2023-10-18T10:24:03.932 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2023-10-18T10:24:03.932 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2023-10-18T10:24:03.934 INFO:teuthology.orchestra.run.smithi125.stdout:[global] 2023-10-18T10:24:03.934 INFO:teuthology.orchestra.run.smithi125.stdout: chdir = "" 2023-10-18T10:24:03.934 INFO:teuthology.orchestra.run.smithi125.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T10:24:03.934 INFO:teuthology.orchestra.run.smithi125.stdout: auth supported = cephx 2023-10-18T10:24:03.935 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.935 INFO:teuthology.orchestra.run.smithi125.stdout: filestore xattr use omap = true 2023-10-18T10:24:03.935 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.935 INFO:teuthology.orchestra.run.smithi125.stdout: mon clock drift allowed = 1.000 2023-10-18T10:24:03.935 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.936 INFO:teuthology.orchestra.run.smithi125.stdout: osd crush chooseleaf type = 0 2023-10-18T10:24:03.936 INFO:teuthology.orchestra.run.smithi125.stdout: auth debug = true 2023-10-18T10:24:03.936 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.936 INFO:teuthology.orchestra.run.smithi125.stdout: ms die on old message = true 2023-10-18T10:24:03.936 INFO:teuthology.orchestra.run.smithi125.stdout: ms die on bug = true 2023-10-18T10:24:03.937 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.937 INFO:teuthology.orchestra.run.smithi125.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T10:24:03.937 INFO:teuthology.orchestra.run.smithi125.stdout: mon pg warn max object skew = 0 2023-10-18T10:24:03.937 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.937 INFO:teuthology.orchestra.run.smithi125.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T10:24:03.938 INFO:teuthology.orchestra.run.smithi125.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T10:24:03.938 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.938 INFO:teuthology.orchestra.run.smithi125.stdout: osd pool default size = 2 2023-10-18T10:24:03.938 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.938 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd allow primary affinity = true 2023-10-18T10:24:03.939 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd allow pg remap = true 2023-10-18T10:24:03.939 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on legacy crush tunables = false 2023-10-18T10:24:03.939 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on crush straw calc version zero = false 2023-10-18T10:24:03.939 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on no sortbitwise = false 2023-10-18T10:24:03.939 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on osd down out interval zero = false 2023-10-18T10:24:03.940 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on too few osds = false 2023-10-18T10:24:03.940 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T10:24:03.940 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T10:24:03.940 INFO:teuthology.orchestra.run.smithi125.stdout: mon_allow_pool_size_one = true 2023-10-18T10:24:03.940 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.940 INFO:teuthology.orchestra.run.smithi125.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-18T10:24:03.941 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.941 INFO:teuthology.orchestra.run.smithi125.stdout: osd default data pool replay window = 5 2023-10-18T10:24:03.941 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.941 INFO:teuthology.orchestra.run.smithi125.stdout: mon allow pool delete = true 2023-10-18T10:24:03.941 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.942 INFO:teuthology.orchestra.run.smithi125.stdout: mon cluster log file level = debug 2023-10-18T10:24:03.942 INFO:teuthology.orchestra.run.smithi125.stdout: debug asserts on shutdown = true 2023-10-18T10:24:03.942 INFO:teuthology.orchestra.run.smithi125.stdout: mon health detail to clog = false 2023-10-18T10:24:03.942 INFO:teuthology.orchestra.run.smithi125.stdout: mon host = "172.21.15.125,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]" 2023-10-18T10:24:03.942 INFO:teuthology.orchestra.run.smithi125.stdout: fsid = 9708ef69-1ff8-449e-8a32-79d14c57ee5d 2023-10-18T10:24:03.943 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.943 INFO:teuthology.orchestra.run.smithi125.stdout:[osd] 2023-10-18T10:24:03.943 INFO:teuthology.orchestra.run.smithi125.stdout: osd journal size = 100 2023-10-18T10:24:03.943 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.943 INFO:teuthology.orchestra.run.smithi125.stdout: osd scrub load threshold = 5.0 2023-10-18T10:24:03.944 INFO:teuthology.orchestra.run.smithi125.stdout: osd scrub max interval = 600 2023-10-18T10:24:03.944 INFO:teuthology.orchestra.run.smithi125.stdout: osd mclock profile = high_recovery_ops 2023-10-18T10:24:03.944 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.944 INFO:teuthology.orchestra.run.smithi125.stdout: osd recover clone overlap = true 2023-10-18T10:24:03.944 INFO:teuthology.orchestra.run.smithi125.stdout: osd recovery max chunk = 1048576 2023-10-18T10:24:03.945 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.945 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug shutdown = true 2023-10-18T10:24:03.945 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug op order = true 2023-10-18T10:24:03.945 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug verify stray on activate = true 2023-10-18T10:24:03.945 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.945 INFO:teuthology.orchestra.run.smithi125.stdout: osd open classes on start = true 2023-10-18T10:24:03.946 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug pg log writeout = true 2023-10-18T10:24:03.946 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.946 INFO:teuthology.orchestra.run.smithi125.stdout: osd deep scrub update digest min age = 30 2023-10-18T10:24:03.946 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.946 INFO:teuthology.orchestra.run.smithi125.stdout: osd map max advance = 10 2023-10-18T10:24:03.947 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.947 INFO:teuthology.orchestra.run.smithi125.stdout: journal zero on create = true 2023-10-18T10:24:03.947 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.947 INFO:teuthology.orchestra.run.smithi125.stdout: filestore ondisk finisher threads = 3 2023-10-18T10:24:03.947 INFO:teuthology.orchestra.run.smithi125.stdout: filestore apply finisher threads = 3 2023-10-18T10:24:03.948 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.948 INFO:teuthology.orchestra.run.smithi125.stdout: bdev debug aio = true 2023-10-18T10:24:03.948 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug misdirected ops = true 2023-10-18T10:24:03.948 INFO:teuthology.orchestra.run.smithi125.stdout: bluestore block size = 96636764160 2023-10-18T10:24:03.948 INFO:teuthology.orchestra.run.smithi125.stdout: bluestore fsck on mount = True 2023-10-18T10:24:03.949 INFO:teuthology.orchestra.run.smithi125.stdout: debug bluefs = 20 2023-10-18T10:24:03.949 INFO:teuthology.orchestra.run.smithi125.stdout: debug bluestore = 20 2023-10-18T10:24:03.949 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2023-10-18T10:24:03.949 INFO:teuthology.orchestra.run.smithi125.stdout: debug osd = 20 2023-10-18T10:24:03.949 INFO:teuthology.orchestra.run.smithi125.stdout: debug rocksdb = 10 2023-10-18T10:24:03.950 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T10:24:03.950 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd full ratio = 0.9 2023-10-18T10:24:03.950 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd nearfull ratio = 0.8 2023-10-18T10:24:03.950 INFO:teuthology.orchestra.run.smithi125.stdout: osd failsafe full ratio = 0.95 2023-10-18T10:24:03.951 INFO:teuthology.orchestra.run.smithi125.stdout: osd objectstore = bluestore 2023-10-18T10:24:03.951 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.951 INFO:teuthology.orchestra.run.smithi125.stdout:[mgr] 2023-10-18T10:24:03.951 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2023-10-18T10:24:03.951 INFO:teuthology.orchestra.run.smithi125.stdout: debug mgr = 20 2023-10-18T10:24:03.952 INFO:teuthology.orchestra.run.smithi125.stdout: debug mon = 20 2023-10-18T10:24:03.952 INFO:teuthology.orchestra.run.smithi125.stdout: debug auth = 20 2023-10-18T10:24:03.952 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min pgs per osd = 4 2023-10-18T10:24:03.952 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min bytes per osd = 10 2023-10-18T10:24:03.952 INFO:teuthology.orchestra.run.smithi125.stdout: mgr/telemetry/nag = false 2023-10-18T10:24:03.953 INFO:teuthology.orchestra.run.smithi125.stdout: debug client = 20 2023-10-18T10:24:03.953 INFO:teuthology.orchestra.run.smithi125.stdout: debug finisher = 20 2023-10-18T10:24:03.953 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.953 INFO:teuthology.orchestra.run.smithi125.stdout:[mon] 2023-10-18T10:24:03.953 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2023-10-18T10:24:03.954 INFO:teuthology.orchestra.run.smithi125.stdout: debug mon = 20 2023-10-18T10:24:03.954 INFO:teuthology.orchestra.run.smithi125.stdout: debug paxos = 20 2023-10-18T10:24:03.954 INFO:teuthology.orchestra.run.smithi125.stdout: debug auth = 20 2023-10-18T10:24:03.954 INFO:teuthology.orchestra.run.smithi125.stdout: mon data avail warn = 5 2023-10-18T10:24:03.955 INFO:teuthology.orchestra.run.smithi125.stdout: mon mgr mkfs grace = 240 2023-10-18T10:24:03.955 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min pgs per osd = 4 2023-10-18T10:24:03.955 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd reporter subtree level = osd 2023-10-18T10:24:03.955 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd prime pg temp = true 2023-10-18T10:24:03.955 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min bytes per osd = 10 2023-10-18T10:24:03.956 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.956 INFO:teuthology.orchestra.run.smithi125.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T10:24:03.956 INFO:teuthology.orchestra.run.smithi125.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T10:24:03.956 INFO:teuthology.orchestra.run.smithi125.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T10:24:03.956 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.957 INFO:teuthology.orchestra.run.smithi125.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T10:24:03.957 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T10:24:03.957 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T10:24:03.957 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.957 INFO:teuthology.orchestra.run.smithi125.stdout: # 1m isn't quite enough 2023-10-18T10:24:03.957 INFO:teuthology.orchestra.run.smithi125.stdout: mon_down_mkfs_grace = 2m 2023-10-18T10:24:03.958 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.958 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_filestore_osds = false 2023-10-18T10:24:03.958 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:03.958 INFO:teuthology.orchestra.run.smithi125.stdout:[client] 2023-10-18T10:24:03.958 INFO:teuthology.orchestra.run.smithi125.stdout: rgw cache enabled = true 2023-10-18T10:24:03.959 INFO:teuthology.orchestra.run.smithi125.stdout: rgw enable ops log = true 2023-10-18T10:24:03.959 INFO:teuthology.orchestra.run.smithi125.stdout: rgw enable usage log = true 2023-10-18T10:24:03.959 INFO:teuthology.orchestra.run.smithi125.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T10:24:03.959 INFO:teuthology.orchestra.run.smithi125.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T10:24:03.960 INFO:teuthology.orchestra.run.smithi125.stdout: pid file = "" 2023-10-18T10:24:03.960 INFO:teuthology.orchestra.run.smithi125.stdout:[mon.a] 2023-10-18T10:24:03.960 INFO:teuthology.orchestra.run.smithi125.stdout:[mon.b] 2023-10-18T10:24:03.960 INFO:teuthology.orchestra.run.smithi125.stdout:[mon.c] 2023-10-18T10:24:03.961 INFO:teuthology.orchestra.run.smithi125.stdout:[mds.a] 2023-10-18T10:24:03.961 INFO:teuthology.orchestra.run.smithi125.stdout:[mds.c] 2023-10-18T10:24:03.961 INFO:teuthology.orchestra.run.smithi125.stdout:[mds.b] 2023-10-18T10:24:03.961 INFO:teuthology.orchestra.run.smithi125.stdout:[mds.d] 2023-10-18T10:24:03.961 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-18T10:24:03.962 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-18T10:24:04.020 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-18T10:24:04.020 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:04.021 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-18T10:24:04.036 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:04.036 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-18T10:24:04.090 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:24:04.090 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:04.091 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T10:24:04.091 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T10:24:04.169 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:04.170 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:04.224 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:24:04.225 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:04.225 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T10:24:04.225 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T10:24:04.266 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:04.267 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:04.322 INFO:tasks.ceph:Setting up mon nodes... 2023-10-18T10:24:04.322 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-18T10:24:04.323 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-18T10:24:04.377 INFO:teuthology.orchestra.run.smithi125.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-18T10:24:04.380 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T10:24:04.458 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T10:24:04.461 INFO:tasks.ceph:Setting up mds nodes... 2023-10-18T10:24:04.461 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-18T10:24:04.516 INFO:teuthology.orchestra.run.smithi125.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-18T10:24:04.518 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-18T10:24:04.545 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-18T10:24:04.639 INFO:teuthology.orchestra.run.smithi125.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-18T10:24:04.641 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-18T10:24:04.668 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-18T10:24:04.723 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-18T10:24:04.726 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-18T10:24:04.753 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-18T10:24:04.846 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-18T10:24:04.849 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-18T10:24:04.875 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-18T10:24:04.876 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-18T10:24:04.918 INFO:teuthology.orchestra.run.smithi125.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-18T10:24:04.933 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-18T10:24:05.014 INFO:teuthology.orchestra.run.smithi125.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-18T10:24:05.029 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-18T10:24:05.070 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-18T10:24:05.085 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-18T10:24:05.167 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-18T10:24:05.182 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-18T10:24:05.182 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi125.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-18T10:24:05.183 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-18T10:24:05.209 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:05.210 INFO:tasks.ceph:role: osd.0 2023-10-18T10:24:05.210 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:24:05.210 DEBUG:teuthology.orchestra.run.smithi125:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T10:24:05.283 INFO:teuthology.orchestra.run.smithi125.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:24:05.283 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:24:05.283 INFO:teuthology.orchestra.run.smithi125.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:24:05.284 INFO:teuthology.orchestra.run.smithi125.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T10:24:05.284 INFO:teuthology.orchestra.run.smithi125.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:24:05.284 INFO:teuthology.orchestra.run.smithi125.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:24:05.284 INFO:teuthology.orchestra.run.smithi125.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:24:05.284 INFO:teuthology.orchestra.run.smithi125.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T10:24:05.285 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:24:05.285 INFO:teuthology.orchestra.run.smithi125.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:24:05.809 INFO:teuthology.orchestra.run.smithi125.stdout:Discarding blocks...Done. 2023-10-18T10:24:05.811 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi125.front.sepia.ceph.com -o noatime 2023-10-18T10:24:05.812 DEBUG:teuthology.orchestra.run.smithi125:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-18T10:24:05.946 DEBUG:teuthology.orchestra.run.smithi125:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-18T10:24:05.984 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-18T10:24:06.052 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:06.053 INFO:tasks.ceph:role: osd.1 2023-10-18T10:24:06.053 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:24:06.054 DEBUG:teuthology.orchestra.run.smithi125:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T10:24:06.128 INFO:teuthology.orchestra.run.smithi125.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:24:06.128 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:24:06.129 INFO:teuthology.orchestra.run.smithi125.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:24:06.129 INFO:teuthology.orchestra.run.smithi125.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T10:24:06.129 INFO:teuthology.orchestra.run.smithi125.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:24:06.129 INFO:teuthology.orchestra.run.smithi125.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:24:06.129 INFO:teuthology.orchestra.run.smithi125.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:24:06.130 INFO:teuthology.orchestra.run.smithi125.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T10:24:06.130 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:24:06.130 INFO:teuthology.orchestra.run.smithi125.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:24:06.617 INFO:teuthology.orchestra.run.smithi125.stdout:Discarding blocks...Done. 2023-10-18T10:24:06.619 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi125.front.sepia.ceph.com -o noatime 2023-10-18T10:24:06.620 DEBUG:teuthology.orchestra.run.smithi125:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-18T10:24:06.719 DEBUG:teuthology.orchestra.run.smithi125:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-18T10:24:06.755 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-18T10:24:06.826 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:06.827 INFO:tasks.ceph:role: osd.2 2023-10-18T10:24:06.827 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:24:06.828 DEBUG:teuthology.orchestra.run.smithi125:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T10:24:06.903 INFO:teuthology.orchestra.run.smithi125.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:24:06.904 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:24:06.904 INFO:teuthology.orchestra.run.smithi125.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:24:06.904 INFO:teuthology.orchestra.run.smithi125.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T10:24:06.904 INFO:teuthology.orchestra.run.smithi125.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:24:06.905 INFO:teuthology.orchestra.run.smithi125.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:24:06.905 INFO:teuthology.orchestra.run.smithi125.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:24:06.905 INFO:teuthology.orchestra.run.smithi125.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T10:24:06.905 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:24:06.905 INFO:teuthology.orchestra.run.smithi125.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:24:07.423 INFO:teuthology.orchestra.run.smithi125.stdout:Discarding blocks...Done. 2023-10-18T10:24:07.425 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi125.front.sepia.ceph.com -o noatime 2023-10-18T10:24:07.425 DEBUG:teuthology.orchestra.run.smithi125:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-18T10:24:07.549 DEBUG:teuthology.orchestra.run.smithi125:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-18T10:24:07.587 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-18T10:24:07.657 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:07.658 INFO:tasks.ceph:role: osd.3 2023-10-18T10:24:07.658 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:24:07.658 DEBUG:teuthology.orchestra.run.smithi125:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-18T10:24:07.734 INFO:teuthology.orchestra.run.smithi125.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:24:07.735 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:24:07.735 INFO:teuthology.orchestra.run.smithi125.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:24:07.735 INFO:teuthology.orchestra.run.smithi125.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T10:24:07.736 INFO:teuthology.orchestra.run.smithi125.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:24:07.736 INFO:teuthology.orchestra.run.smithi125.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:24:07.736 INFO:teuthology.orchestra.run.smithi125.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:24:07.736 INFO:teuthology.orchestra.run.smithi125.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T10:24:07.736 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:24:07.736 INFO:teuthology.orchestra.run.smithi125.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:24:08.235 INFO:teuthology.orchestra.run.smithi125.stdout:Discarding blocks...Done. 2023-10-18T10:24:08.237 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi125.front.sepia.ceph.com -o noatime 2023-10-18T10:24:08.238 DEBUG:teuthology.orchestra.run.smithi125:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-18T10:24:08.352 DEBUG:teuthology.orchestra.run.smithi125:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-18T10:24:08.389 DEBUG:teuthology.orchestra.run.smithi125:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:08.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:08.494+0000 7f5bf5d46500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-18T10:24:08.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:08.495+0000 7f5bf5d46500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-18T10:24:08.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:08.495+0000 7f5bf5d46500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T10:24:08.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:08.495+0000 7f5bf5d46500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T10:24:08.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:08.495+0000 7f5bf5d46500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T10:24:08.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:08.495+0000 7f5bf5d46500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-18T10:24:11.357 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-18T10:24:11.389 DEBUG:teuthology.orchestra.run.smithi125:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:11.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:11.480+0000 7f1483002500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-18T10:24:11.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:11.480+0000 7f1483002500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-18T10:24:11.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:11.481+0000 7f1483002500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T10:24:11.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:11.481+0000 7f1483002500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T10:24:11.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:11.481+0000 7f1483002500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T10:24:11.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:11.481+0000 7f1483002500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-18T10:24:14.328 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-18T10:24:14.360 DEBUG:teuthology.orchestra.run.smithi125:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:14.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:14.450+0000 7f3a04727500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-18T10:24:14.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:14.451+0000 7f3a04727500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-18T10:24:14.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:14.451+0000 7f3a04727500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T10:24:14.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:14.451+0000 7f3a04727500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T10:24:14.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:14.451+0000 7f3a04727500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T10:24:14.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:14.451+0000 7f3a04727500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-18T10:24:17.308 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-18T10:24:17.341 DEBUG:teuthology.orchestra.run.smithi125:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:17.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:17.432+0000 7f59e61e2500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-18T10:24:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:17.432+0000 7f59e61e2500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-18T10:24:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:17.432+0000 7f59e61e2500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T10:24:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:17.432+0000 7f59e61e2500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T10:24:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:17.432+0000 7f59e61e2500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T10:24:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:17.433+0000 7f59e61e2500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-18T10:24:20.281 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-18T10:24:20.315 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-18T10:24:20.347 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:20.347 INFO:tasks.ceph:role: osd.4 2023-10-18T10:24:20.347 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:24:20.348 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T10:24:20.425 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:24:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:24:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:24:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T10:24:20.426 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:24:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:24:20.426 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:24:20.426 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T10:24:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:24:20.427 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:24:20.920 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2023-10-18T10:24:20.922 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2023-10-18T10:24:20.922 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-18T10:24:21.034 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-18T10:24:21.067 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-18T10:24:21.135 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:21.135 INFO:tasks.ceph:role: osd.5 2023-10-18T10:24:21.135 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:24:21.136 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T10:24:21.212 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:24:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:24:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:24:21.213 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T10:24:21.213 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:24:21.213 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:24:21.213 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:24:21.214 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T10:24:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:24:21.214 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:24:21.680 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2023-10-18T10:24:21.683 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2023-10-18T10:24:21.683 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-18T10:24:21.768 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-18T10:24:21.798 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-18T10:24:21.864 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:21.865 INFO:tasks.ceph:role: osd.6 2023-10-18T10:24:21.865 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:24:21.866 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T10:24:21.938 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:24:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:24:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:24:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T10:24:21.939 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:24:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:24:21.940 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:24:21.940 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T10:24:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:24:21.940 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:24:22.434 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2023-10-18T10:24:22.436 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2023-10-18T10:24:22.437 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-18T10:24:22.570 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-18T10:24:22.604 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-18T10:24:22.672 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T10:24:22.672 INFO:tasks.ceph:role: osd.7 2023-10-18T10:24:22.673 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:24:22.673 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-18T10:24:22.747 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T10:24:22.748 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T10:24:22.748 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T10:24:22.748 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T10:24:22.749 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T10:24:22.749 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2023-10-18T10:24:22.749 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T10:24:22.749 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T10:24:22.750 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T10:24:22.750 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T10:24:23.269 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2023-10-18T10:24:23.271 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2023-10-18T10:24:23.272 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-18T10:24:23.419 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-18T10:24:23.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:23.557 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:23.555+0000 7f4223bad500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-18T10:24:23.557 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:23.555+0000 7f4223bad500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-18T10:24:23.558 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:23.555+0000 7f4223bad500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T10:24:23.558 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:23.555+0000 7f4223bad500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T10:24:23.558 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:23.555+0000 7f4223bad500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T10:24:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:23.556+0000 7f4223bad500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-18T10:24:26.414 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-18T10:24:26.445 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:26.536 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:26.534+0000 7f43f9d48500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-18T10:24:26.536 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:26.534+0000 7f43f9d48500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-18T10:24:26.537 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:26.534+0000 7f43f9d48500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T10:24:26.537 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:26.534+0000 7f43f9d48500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T10:24:26.537 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:26.534+0000 7f43f9d48500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T10:24:26.538 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:26.535+0000 7f43f9d48500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-18T10:24:29.384 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-18T10:24:29.415 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:29.507 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:29.504+0000 7fcbea612500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-18T10:24:29.507 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:29.504+0000 7fcbea612500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-18T10:24:29.507 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:29.504+0000 7fcbea612500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T10:24:29.507 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:29.505+0000 7fcbea612500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T10:24:29.508 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:29.505+0000 7fcbea612500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T10:24:29.508 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:29.505+0000 7fcbea612500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-18T10:24:31.910 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:24:31.913 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:24:32.359 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-18T10:24:32.387 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:32.474 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:32.472+0000 7f2913426500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-18T10:24:32.474 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:32.472+0000 7f2913426500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-18T10:24:32.474 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:32.472+0000 7f2913426500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T10:24:32.475 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:32.472+0000 7f2913426500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T10:24:32.475 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:32.472+0000 7f2913426500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T10:24:32.475 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:32.473+0000 7f2913426500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-18T10:24:35.331 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-18T10:24:35.360 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-18T10:24:35.361 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:35.361 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-18T10:24:35.393 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:35.393 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-18T10:24:35.464 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:35.464 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-18T10:24:35.536 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:35.537 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-18T10:24:35.606 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:35.607 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-18T10:24:35.676 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:35.676 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-18T10:24:35.746 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:35.746 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-18T10:24:35.815 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:35.815 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-18T10:24:35.847 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:35.847 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-18T10:24:35.917 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:35.918 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-18T10:24:35.986 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:35.987 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-18T10:24:36.055 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:36.056 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-18T10:24:36.123 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:36.124 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-18T10:24:36.191 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:36.191 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-18T10:24:36.259 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:36.259 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-18T10:24:36.277 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:36.277 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-18T10:24:36.334 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:36.335 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-18T10:24:36.350 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:36.351 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-18T10:24:36.405 INFO:tasks.ceph:Adding keys to all mons... 2023-10-18T10:24:36.405 DEBUG:teuthology.orchestra.run.smithi125:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T10:24:36.408 DEBUG:teuthology.orchestra.run.smithi139:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T10:24:36.449 INFO:teuthology.orchestra.run.smithi125.stdout:[mgr.y] 2023-10-18T10:24:36.449 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw== 2023-10-18T10:24:36.449 INFO:teuthology.orchestra.run.smithi125.stdout:[mds.a] 2023-10-18T10:24:36.450 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g== 2023-10-18T10:24:36.450 INFO:teuthology.orchestra.run.smithi125.stdout:[mds.c] 2023-10-18T10:24:36.450 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g== 2023-10-18T10:24:36.450 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.0] 2023-10-18T10:24:36.451 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A== 2023-10-18T10:24:36.451 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.1] 2023-10-18T10:24:36.451 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg== 2023-10-18T10:24:36.451 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.2] 2023-10-18T10:24:36.452 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA== 2023-10-18T10:24:36.452 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.3] 2023-10-18T10:24:36.452 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw== 2023-10-18T10:24:36.452 INFO:teuthology.orchestra.run.smithi125.stdout:[mgr.x] 2023-10-18T10:24:36.453 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw== 2023-10-18T10:24:36.453 INFO:teuthology.orchestra.run.smithi125.stdout:[mds.b] 2023-10-18T10:24:36.453 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ== 2023-10-18T10:24:36.453 INFO:teuthology.orchestra.run.smithi125.stdout:[mds.d] 2023-10-18T10:24:36.453 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw== 2023-10-18T10:24:36.454 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.4] 2023-10-18T10:24:36.454 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww== 2023-10-18T10:24:36.454 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.5] 2023-10-18T10:24:36.454 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q== 2023-10-18T10:24:36.455 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.6] 2023-10-18T10:24:36.455 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw== 2023-10-18T10:24:36.455 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.7] 2023-10-18T10:24:36.455 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw== 2023-10-18T10:24:36.455 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:24:36.456 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:24:36.456 INFO:teuthology.orchestra.run.smithi125.stdout:[client.2] 2023-10-18T10:24:36.456 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg== 2023-10-18T10:24:36.456 INFO:teuthology.orchestra.run.smithi125.stdout:[client.1] 2023-10-18T10:24:36.456 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg== 2023-10-18T10:24:36.457 INFO:teuthology.orchestra.run.smithi125.stdout:[client.3] 2023-10-18T10:24:36.457 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g== 2023-10-18T10:24:36.471 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.y] 2023-10-18T10:24:36.472 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw== 2023-10-18T10:24:36.472 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2023-10-18T10:24:36.472 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g== 2023-10-18T10:24:36.472 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2023-10-18T10:24:36.472 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g== 2023-10-18T10:24:36.473 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.0] 2023-10-18T10:24:36.473 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A== 2023-10-18T10:24:36.473 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.1] 2023-10-18T10:24:36.473 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg== 2023-10-18T10:24:36.474 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.2] 2023-10-18T10:24:36.474 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA== 2023-10-18T10:24:36.474 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.3] 2023-10-18T10:24:36.474 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw== 2023-10-18T10:24:36.475 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.x] 2023-10-18T10:24:36.475 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw== 2023-10-18T10:24:36.475 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2023-10-18T10:24:36.475 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ== 2023-10-18T10:24:36.475 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2023-10-18T10:24:36.476 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw== 2023-10-18T10:24:36.476 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.4] 2023-10-18T10:24:36.476 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww== 2023-10-18T10:24:36.476 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.5] 2023-10-18T10:24:36.476 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q== 2023-10-18T10:24:36.477 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.6] 2023-10-18T10:24:36.477 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw== 2023-10-18T10:24:36.477 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.7] 2023-10-18T10:24:36.477 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw== 2023-10-18T10:24:36.478 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2023-10-18T10:24:36.478 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:24:36.478 INFO:teuthology.orchestra.run.smithi139.stdout:[client.2] 2023-10-18T10:24:36.478 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg== 2023-10-18T10:24:36.478 INFO:teuthology.orchestra.run.smithi139.stdout:[client.1] 2023-10-18T10:24:36.479 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg== 2023-10-18T10:24:36.479 INFO:teuthology.orchestra.run.smithi139.stdout:[client.3] 2023-10-18T10:24:36.479 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g== 2023-10-18T10:24:36.480 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T10:24:36.501 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T10:24:36.573 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:24:36.575 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:24:36.631 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:24:36.633 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:24:36.687 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:36.689 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:36.742 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:36.744 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:36.797 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:36.799 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:36.850 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:36.853 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:36.905 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T10:24:36.908 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T10:24:36.959 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:24:36.962 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:24:37.015 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:24:37.018 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T10:24:37.072 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:37.075 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:37.125 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:37.127 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:37.179 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:37.181 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:37.232 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:37.234 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T10:24:37.286 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:24:37.288 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:24:37.342 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:24:37.345 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:24:37.399 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:24:37.402 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:24:37.455 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:24:37.458 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T10:24:37.513 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-18T10:24:37.513 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-18T10:24:37.539 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T10:24:37.636 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-18T10:24:37.662 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-18T10:24:37.689 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T10:24:37.785 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-18T10:24:37.812 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-18T10:24:37.879 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T10:24:37.975 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-18T10:24:38.003 DEBUG:teuthology.orchestra.run.smithi125:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:38.005 DEBUG:teuthology.orchestra.run.smithi139:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T10:24:38.056 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-18T10:24:38.057 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-18T10:24:38.058 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-18T10:24:38.060 INFO:tasks.ceph.mon.a:Started 2023-10-18T10:24:38.060 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-18T10:24:38.061 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-18T10:24:38.098 INFO:tasks.ceph.mon.b:Started 2023-10-18T10:24:38.099 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-18T10:24:38.099 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-18T10:24:38.101 INFO:tasks.ceph.mon.c:Started 2023-10-18T10:24:38.101 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-18T10:24:38.102 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-18T10:24:38.102 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-18T10:24:38.104 INFO:tasks.ceph.mgr.y:Started 2023-10-18T10:24:38.105 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-18T10:24:38.105 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-18T10:24:38.107 INFO:tasks.ceph.mgr.x:Started 2023-10-18T10:24:38.107 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2023-10-18T10:24:38.108 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2023-10-18T10:24:38.110 INFO:tasks.ceph:setting config [client] debug_client = 20 2023-10-18T10:24:38.110 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set client debug_client 20 2023-10-18T10:24:38.153 INFO:tasks.ceph:setting config [client] debug_ms = 1 2023-10-18T10:24:38.153 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set client debug_ms 1 2023-10-18T10:24:38.196 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2023-10-18T10:24:38.197 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2023-10-18T10:24:38.240 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2023-10-18T10:24:38.241 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2023-10-18T10:24:38.283 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2023-10-18T10:24:38.283 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds debug_mds 20 2023-10-18T10:24:38.327 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2023-10-18T10:24:38.328 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2023-10-18T10:24:38.371 INFO:tasks.ceph:setting config [mds] debug_ms = 1 2023-10-18T10:24:38.371 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds debug_ms 1 2023-10-18T10:24:38.414 INFO:tasks.ceph:setting config [mds] mds_bal_fragment_size_max = 10000 2023-10-18T10:24:38.415 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds mds_bal_fragment_size_max 10000 2023-10-18T10:24:38.460 INFO:tasks.ceph:setting config [mds] mds_bal_merge_size = 5 2023-10-18T10:24:38.461 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds mds_bal_merge_size 5 2023-10-18T10:24:38.504 INFO:tasks.ceph:setting config [mds] mds_bal_split_bits = 3 2023-10-18T10:24:38.505 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds mds_bal_split_bits 3 2023-10-18T10:24:38.547 INFO:tasks.ceph:setting config [mds] mds_bal_split_size = 100 2023-10-18T10:24:38.548 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds mds_bal_split_size 100 2023-10-18T10:24:38.590 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2023-10-18T10:24:38.591 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2023-10-18T10:24:38.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.622+0000 7f0bacd0f640 1 -- 172.21.15.125:0/178330560 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) v1 -- 0x7f0b800050a0 con 0x7f0ba814a8f0 2023-10-18T10:24:38.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.622+0000 7f0fa5d76640 1 -- 172.21.15.125:0/2074633937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fa00c4c60 msgr2=0x7f0fa0156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.622+0000 7f0fa5d76640 1 --2- 172.21.15.125:0/2074633937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fa00c4c60 0x7f0fa0156c10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f94005d80 tx=0x7f0f94003dd0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.622+0000 7f0fa5d76640 1 -- 172.21.15.125:0/2074633937 shutdown_connections 2023-10-18T10:24:38.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.622+0000 7f0fa5d76640 1 --2- 172.21.15.125:0/2074633937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0fa00c50e0 0x7f0fa0157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.622+0000 7f0fa5d76640 1 --2- 172.21.15.125:0/2074633937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fa00c4c60 0x7f0fa0156c10 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f94005d80 tx=0x7f0f94003dd0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.622+0000 7f0fa5d76640 1 --2- 172.21.15.125:0/2074633937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f88005180 0x7f0fa01563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.622+0000 7f0fa5d76640 1 -- 172.21.15.125:0/2074633937 >> 172.21.15.125:0/2074633937 conn(0x7f0fa00b0670 msgr2=0x7f0fa00ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.626 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2023-10-18T10:24:38.627 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2023-10-18T10:24:38.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.623+0000 7f2b694b4640 1 -- 172.21.15.125:0/4095851915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b6415ab30 msgr2=0x7f2b6406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.623+0000 7f2b694b4640 1 --2- 172.21.15.125:0/4095851915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b6415ab30 0x7f2b6406f6b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005fd0 tx=0x7f2b5c02c260 comp rx=0 tx=0).stop 2023-10-18T10:24:38.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.624+0000 7f0fa5d76640 1 -- 172.21.15.125:0/2074633937 shutdown_connections 2023-10-18T10:24:38.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.624+0000 7f0fa5d76640 1 -- 172.21.15.125:0/2074633937 wait complete. 2023-10-18T10:24:38.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.624+0000 7f2b694b4640 1 -- 172.21.15.125:0/4095851915 shutdown_connections 2023-10-18T10:24:38.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.624+0000 7f2b694b4640 1 --2- 172.21.15.125:0/4095851915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b6415ab30 0x7f2b6406f6b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.624+0000 7f2b694b4640 1 --2- 172.21.15.125:0/4095851915 >> v2:172.21.15.125:3300/0 conn(0x7f2b640b0750 0x7f2b640cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.624+0000 7f2b694b4640 1 -- 172.21.15.125:0/4095851915 >> 172.21.15.125:0/4095851915 conn(0x7f2b640ae720 msgr2=0x7f2b640aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.624+0000 7f2b694b4640 1 -- 172.21.15.125:0/4095851915 shutdown_connections 2023-10-18T10:24:38.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2e2d432640 1 -- 172.21.15.125:0/1418946259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280888c0 msgr2=0x7f2e280a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.624+0000 7f2b694b4640 1 -- 172.21.15.125:0/4095851915 wait complete. 2023-10-18T10:24:38.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2e2d432640 1 --2- 172.21.15.125:0/1418946259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280888c0 0x7f2e280a6720 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c8d0 tx=0x7f2e2000cd90 comp rx=0 tx=0).stop 2023-10-18T10:24:38.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.626+0000 7f2b694b4640 1 Processor -- start 2023-10-18T10:24:38.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.626+0000 7f2b694b4640 1 -- start start 2023-10-18T10:24:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.627+0000 7f2b694b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b640b0750 0x7f2b640a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.627+0000 7f2b694b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b6415ab30 0x7f2b640a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.627+0000 7f2b694b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b6415b430 0x7f2b64157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.627+0000 7f2b694b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64152b90 con 0x7f2b640b0750 2023-10-18T10:24:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.627+0000 7f2b694b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64159430 con 0x7f2b6415b430 2023-10-18T10:24:38.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.627+0000 7f2b694b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64158b90 con 0x7f2b6415ab30 2023-10-18T10:24:38.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b627fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b6415ab30 0x7f2b640a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b6415b430 0x7f2b64157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b6415b430 0x7f2b64157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:45820/0 (socket says 172.21.15.125:45820) 2023-10-18T10:24:38.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b637fe640 1 -- 172.21.15.125:0/2159490734 learned_addr learned my addr 172.21.15.125:0/2159490734 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2e2d432640 1 -- 172.21.15.125:0/1418946259 shutdown_connections 2023-10-18T10:24:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2e2d432640 1 --2- 172.21.15.125:0/1418946259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e28167bf0 0x7f2e2820da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2e2d432640 1 --2- 172.21.15.125:0/1418946259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e280b0900 0x7f2e2820d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2e2d432640 1 --2- 172.21.15.125:0/1418946259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280888c0 0x7f2e280a6720 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b637fe640 1 -- 172.21.15.125:0/2159490734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b6415ab30 msgr2=0x7f2b640a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b637fe640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b6415ab30 0x7f2b640a73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2e2d432640 1 -- 172.21.15.125:0/1418946259 >> 172.21.15.125:0/1418946259 conn(0x7f2e280ae720 msgr2=0x7f2e2806d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b637fe640 1 -- 172.21.15.125:0/2159490734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b640b0750 msgr2=0x7f2b640a6ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:24:38.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b62ffd640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b640b0750 0x7f2b640a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b637fe640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b640b0750 0x7f2b640a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b637fe640 1 -- 172.21.15.125:0/2159490734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c009900 con 0x7f2b6415b430 2023-10-18T10:24:38.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.628+0000 7f2b62ffd640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b640b0750 0x7f2b640a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:24:38.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.629+0000 7f2b637fe640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b6415b430 0x7f2b64157b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c02c870 tx=0x7f2b5c034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:38.640 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2023-10-18T10:24:38.640 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2023-10-18T10:24:38.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.630+0000 7f0b96ffd640 1 -- 172.21.15.125:0/178330560 <== mon.1 v2:172.21.15.139:3300/0 10 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f0ba0011550 con 0x7f0ba814a8f0 2023-10-18T10:24:38.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.630+0000 7f0b96ffd640 1 -- 172.21.15.125:0/178330560 <== mon.1 v2:172.21.15.139:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v6)=0 v6) v1 ==== 114+0+0 (secure 0 0 0) 0x7f0ba0029020 con 0x7f0ba814a8f0 2023-10-18T10:24:38.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.631+0000 7f8cc17fa640 1 -- 172.21.15.125:0/417710032 <== mon.1 v2:172.21.15.139:3300/0 10 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f8cb00037d0 con 0x7f8cc40ac300 2023-10-18T10:24:38.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.635+0000 7f0bacd0f640 1 -- 172.21.15.125:0/178330560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ba814a8f0 msgr2=0x7f0ba81f47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.635+0000 7f0bacd0f640 1 --2- 172.21.15.125:0/178330560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ba814a8f0 0x7f0ba81f47b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ba806e3a0 tx=0x7f0ba0010f00 comp rx=0 tx=0).stop 2023-10-18T10:24:38.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f0bacd0f640 1 -- 172.21.15.125:0/178330560 shutdown_connections 2023-10-18T10:24:38.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.631+0000 7fe3cdffb640 1 -- 172.21.15.125:0/1029781778 <== mon.1 v2:172.21.15.139:3300/0 10 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7fe3c803f710 con 0x7fe3d815b260 2023-10-18T10:24:38.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.635+0000 7f2e2d432640 1 -- 172.21.15.125:0/1418946259 shutdown_connections 2023-10-18T10:24:38.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f0bacd0f640 1 --2- 172.21.15.125:0/178330560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ba814a8f0 0x7f0ba81f47b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f0bacd0f640 1 --2- 172.21.15.125:0/178330560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ba814a470 0x7f0ba81f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f0bacd0f640 1 --2- 172.21.15.125:0/178330560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0b90003b30 0x7f0ba80c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f0bacd0f640 1 -- 172.21.15.125:0/178330560 >> 172.21.15.125:0/178330560 conn(0x7f0ba80b2040 msgr2=0x7f0ba8157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f2b4b7fe640 1 -- 172.21.15.125:0/2159490734 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c004450 con 0x7f2b6415b430 2023-10-18T10:24:38.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f2e2d432640 1 -- 172.21.15.125:0/1418946259 wait complete. 2023-10-18T10:24:38.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f0bacd0f640 1 -- 172.21.15.125:0/178330560 shutdown_connections 2023-10-18T10:24:38.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f2b694b4640 1 -- 172.21.15.125:0/2159490734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b64158e70 con 0x7f2b6415b430 2023-10-18T10:24:38.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.636+0000 7f2b694b4640 1 -- 172.21.15.125:0/2159490734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b64156a50 con 0x7f2b6415b430 2023-10-18T10:24:38.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.637+0000 7f2b4b7fe640 1 -- 172.21.15.125:0/2159490734 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f2b5c004680 con 0x7f2b6415b430 2023-10-18T10:24:38.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.637+0000 7f2b4b7fe640 1 -- 172.21.15.125:0/2159490734 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c03e410 con 0x7f2b6415b430 2023-10-18T10:24:38.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.637+0000 7f2b4b7fe640 1 -- 172.21.15.125:0/2159490734 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2b5c03b050 con 0x7f2b6415b430 2023-10-18T10:24:38.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.638+0000 7f2b4b7fe640 1 -- 172.21.15.125:0/2159490734 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2b5c049790 con 0x7f2b6415b430 2023-10-18T10:24:38.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.639+0000 7f0bacd0f640 1 -- 172.21.15.125:0/178330560 wait complete. 2023-10-18T10:24:38.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.641+0000 7f2b694b4640 1 -- 172.21.15.125:0/2159490734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b44005040 con 0x7f2b6415b430 2023-10-18T10:24:38.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.651+0000 7f2b4b7fe640 1 -- 172.21.15.125:0/2159490734 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2b5c068b40 con 0x7f2b6415b430 2023-10-18T10:24:38.653 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2023-10-18T10:24:38.654 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2023-10-18T10:24:38.660 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2023-10-18T10:24:38.660 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2023-10-18T10:24:38.728 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2023-10-18T10:24:38.729 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2023-10-18T10:24:38.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.746+0000 7f541da07640 1 -- 172.21.15.125:0/466015282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f541806fdc0 msgr2=0x7f54180af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.746+0000 7f541da07640 1 --2- 172.21.15.125:0/466015282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f541806fdc0 0x7f54180af2e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f54180b2750 tx=0x7f540802d040 comp rx=0 tx=0).stop 2023-10-18T10:24:38.749 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2023-10-18T10:24:38.750 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2023-10-18T10:24:38.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.750+0000 7f541da07640 1 -- 172.21.15.125:0/466015282 shutdown_connections 2023-10-18T10:24:38.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.750+0000 7f541da07640 1 --2- 172.21.15.125:0/466015282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f541806fdc0 0x7f54180af2e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.750+0000 7f541da07640 1 --2- 172.21.15.125:0/466015282 >> v2:172.21.15.139:3300/0 conn(0x7f54181573f0 0x7f54180aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.750+0000 7f541da07640 1 -- 172.21.15.125:0/466015282 >> 172.21.15.125:0/466015282 conn(0x7f54180b0cd0 msgr2=0x7f54180b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.750+0000 7f541da07640 1 -- 172.21.15.125:0/466015282 shutdown_connections 2023-10-18T10:24:38.753 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2023-10-18T10:24:38.753 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2023-10-18T10:24:38.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.750+0000 7f541da07640 1 -- 172.21.15.125:0/466015282 wait complete. 2023-10-18T10:24:38.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.752+0000 7f541da07640 1 Processor -- start 2023-10-18T10:24:38.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.752+0000 7f541da07640 1 -- start start 2023-10-18T10:24:38.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f541da07640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f541806ab60 0x7f541808abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f541da07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f541806fdc0 0x7f5418159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f541da07640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54181573f0 0x7f54181445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f541da07640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541808a740 con 0x7f54181573f0 2023-10-18T10:24:38.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f541da07640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541808a410 con 0x7f541806fdc0 2023-10-18T10:24:38.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f541da07640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5418157d80 con 0x7f541806ab60 2023-10-18T10:24:38.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f54177fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54181573f0 0x7f54181445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f54177fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54181573f0 0x7f54181445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35812/0 (socket says 172.21.15.125:35812) 2023-10-18T10:24:38.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.753+0000 7f54177fe640 1 -- 172.21.15.125:0/3752440021 learned_addr learned my addr 172.21.15.125:0/3752440021 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:38.758 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2023-10-18T10:24:38.760 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2023-10-18T10:24:38.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.755+0000 7f54177fe640 1 -- 172.21.15.125:0/3752440021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f541806ab60 msgr2=0x7f541808abd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:24:38.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.755+0000 7f54177fe640 1 --2- 172.21.15.125:0/3752440021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f541806ab60 0x7f541808abd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.755+0000 7f54177fe640 1 -- 172.21.15.125:0/3752440021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f541806fdc0 msgr2=0x7f5418159f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:24:38.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.755+0000 7f54177fe640 1 --2- 172.21.15.125:0/3752440021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f541806fdc0 0x7f5418159f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.755+0000 7f54177fe640 1 -- 172.21.15.125:0/3752440021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540800a5a0 con 0x7f54181573f0 2023-10-18T10:24:38.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.755+0000 7f54177fe640 1 --2- 172.21.15.125:0/3752440021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54181573f0 0x7f54181445f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c480 tx=0x7f540c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:38.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.756+0000 7f53fffff640 1 -- 172.21.15.125:0/3752440021 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c005480 con 0x7f54181573f0 2023-10-18T10:24:38.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.756+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5418158060 con 0x7f54181573f0 2023-10-18T10:24:38.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.756+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541808b110 con 0x7f54181573f0 2023-10-18T10:24:38.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.758+0000 7f53fffff640 1 -- 172.21.15.125:0/3752440021 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f540c0036c0 con 0x7f54181573f0 2023-10-18T10:24:38.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.759+0000 7f53fffff640 1 -- 172.21.15.125:0/3752440021 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c003ab0 con 0x7f54181573f0 2023-10-18T10:24:38.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.758+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/6340418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc15b550 msgr2=0x7f8dcc069b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.758+0000 7f8dd0bc5640 1 --2- 172.21.15.125:0/6340418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc15b550 0x7f8dcc069b90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0005fd0 tx=0x7f8dc003ce70 comp rx=0 tx=0).stop 2023-10-18T10:24:38.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.760+0000 7f53fffff640 1 -- 172.21.15.125:0/3752440021 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f540c01b400 con 0x7f54181573f0 2023-10-18T10:24:38.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.760+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/6340418 shutdown_connections 2023-10-18T10:24:38.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.760+0000 7f8dd0bc5640 1 --2- 172.21.15.125:0/6340418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc15b550 0x7f8dcc069b90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.760+0000 7f8dd0bc5640 1 --2- 172.21.15.125:0/6340418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8dcc06e0b0 0x7f8dcc06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.760+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/6340418 >> 172.21.15.125:0/6340418 conn(0x7f8dcc0af730 msgr2=0x7f8dcc0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.766 DEBUG:tasks.ceph:set 21 configs 2023-10-18T10:24:38.766 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.767 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.767 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.767 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.763+0000 7f53fffff640 1 -- 172.21.15.125:0/3752440021 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f540c012910 con 0x7f54181573f0 2023-10-18T10:24:38.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.763+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f8005040 con 0x7f54181573f0 2023-10-18T10:24:38.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.763+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/6340418 shutdown_connections 2023-10-18T10:24:38.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.764+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/6340418 wait complete. 2023-10-18T10:24:38.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.766+0000 7f8dd0bc5640 1 Processor -- start 2023-10-18T10:24:38.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.766+0000 7f8dd0bc5640 1 -- start start 2023-10-18T10:24:38.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.767+0000 7f53fffff640 1 -- 172.21.15.125:0/3752440021 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f540c01b890 con 0x7f54181573f0 2023-10-18T10:24:38.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.767+0000 7f8dd0bc5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dcc06e0b0 0x7f8dcc1ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.772+0000 7f8dd0bc5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc0b12a0 0x7f8dcc144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.772+0000 7f8dca59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dcc06e0b0 0x7f8dcc1ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.772+0000 7f8dca59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dcc06e0b0 0x7f8dcc1ecfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:45854/0 (socket says 172.21.15.125:45854) 2023-10-18T10:24:38.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.772+0000 7f8dc9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc0b12a0 0x7f8dcc144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.773+0000 7f8dc9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc0b12a0 0x7f8dcc144500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35826/0 (socket says 172.21.15.125:35826) 2023-10-18T10:24:38.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.773+0000 7f8dca59c640 1 -- 172.21.15.125:0/2211238072 learned_addr learned my addr 172.21.15.125:0/2211238072 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:38.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.773+0000 7f8dd0bc5640 1 --2- 172.21.15.125:0/2211238072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8dcc15b550 0x7f8dcc0cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.773+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/2211238072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc144a40 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.773+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/2211238072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0cd430 con 0x7f8dcc06e0b0 2023-10-18T10:24:38.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.773+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/2211238072 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0ccb90 con 0x7f8dcc15b550 2023-10-18T10:24:38.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.774+0000 7f8dc9d9b640 1 -- 172.21.15.125:0/2211238072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8dcc15b550 msgr2=0x7f8dcc0cc140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:24:38.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.774+0000 7f8dc9d9b640 1 --2- 172.21.15.125:0/2211238072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8dcc15b550 0x7f8dcc0cc140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.774+0000 7f8dc9d9b640 1 -- 172.21.15.125:0/2211238072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dcc06e0b0 msgr2=0x7f8dcc1ecfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.774+0000 7f8dc9d9b640 1 --2- 172.21.15.125:0/2211238072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dcc06e0b0 0x7f8dcc1ecfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.774+0000 7f8dc9d9b640 1 -- 172.21.15.125:0/2211238072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0009560 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.774+0000 7f8dc9d9b640 1 --2- 172.21.15.125:0/2211238072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc0b12a0 0x7f8dcc144500 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0044000 tx=0x7f8dc0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:38.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.775+0000 7f8dbaffd640 1 -- 172.21.15.125:0/2211238072 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc003ef00 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.775+0000 7f8dbaffd640 1 -- 172.21.15.125:0/2211238072 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f8dc0003ae0 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.775+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/2211238072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc0cce70 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.776+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/2211238072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc0a9210 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.777+0000 7fe3c37fe640 1 -- 172.21.15.125:0/1029781778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7fe3b00050c0 con 0x7fe3d815b260 2023-10-18T10:24:38.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.777+0000 7f8dbaffd640 1 -- 172.21.15.125:0/2211238072 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0045440 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.780+0000 7f8dbaffd640 1 -- 172.21.15.125:0/2211238072 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8dc0045920 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.785+0000 7f8cc17fa640 1 -- 172.21.15.125:0/417710032 <== mon.1 v2:172.21.15.139:3300/0 11 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8cb00174a0 con 0x7f8cc40ac300 2023-10-18T10:24:38.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.787+0000 7fe3cdffb640 1 -- 172.21.15.125:0/1029781778 <== mon.1 v2:172.21.15.139:3300/0 11 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3c8005020 con 0x7fe3d815b260 2023-10-18T10:24:38.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.787+0000 7fe3cdffb640 1 -- 172.21.15.125:0/1029781778 <== mon.1 v2:172.21.15.139:3300/0 12 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v7)=0 v7) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe3c8057020 con 0x7fe3d815b260 2023-10-18T10:24:38.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.787+0000 7f53fffff640 1 -- 172.21.15.125:0/3752440021 <== mon.0 v2:172.21.15.125:3300/0 7 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f540c01ba30 con 0x7f54181573f0 2023-10-18T10:24:38.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.788+0000 7f8dbaffd640 1 -- 172.21.15.125:0/2211238072 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8dc0058d70 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.788+0000 7f8dbaffd640 1 -- 172.21.15.125:0/2211238072 <== mon.0 v2:172.21.15.125:3300/0 6 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8dc004f510 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.789+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/2211238072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da4005250 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.790+0000 7f2b4b7fe640 1 -- 172.21.15.125:0/2159490734 <== mon.1 v2:172.21.15.139:3300/0 7 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2b5c0349c0 con 0x7f2b6415b430 2023-10-18T10:24:38.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.790+0000 7fe3c37fe640 1 -- 172.21.15.125:0/1029781778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d815b260 msgr2=0x7fe3d815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.790+0000 7fe3c37fe640 1 --2- 172.21.15.125:0/1029781778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d815b260 0x7fe3d815daf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8034000 tx=0x7fe3c80046c0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.793+0000 7fe3c37fe640 1 -- 172.21.15.125:0/1029781778 shutdown_connections 2023-10-18T10:24:38.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.793+0000 7fe3c37fe640 1 --2- 172.21.15.125:0/1029781778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d815b260 0x7fe3d815daf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.793+0000 7fe3c37fe640 1 --2- 172.21.15.125:0/1029781778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d815b640 0x7fe3d815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.793+0000 7fe3c37fe640 1 --2- 172.21.15.125:0/1029781778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3d80b0b10 0x7fe3d81f03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.794+0000 7fe3c37fe640 1 -- 172.21.15.125:0/1029781778 >> 172.21.15.125:0/1029781778 conn(0x7fe3d809a030 msgr2=0x7fe3d80af210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.793+0000 7f8ccabb0640 1 -- 172.21.15.125:0/417710032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7f8c9c0052b0 con 0x7f8cc40ac300 2023-10-18T10:24:38.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.794+0000 7f8dbaffd640 1 -- 172.21.15.125:0/2211238072 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f8dc0077b40 con 0x7f8dcc0b12a0 2023-10-18T10:24:38.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.796+0000 7fe3c37fe640 1 -- 172.21.15.125:0/1029781778 shutdown_connections 2023-10-18T10:24:38.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.796+0000 7fe3c37fe640 1 -- 172.21.15.125:0/1029781778 wait complete. 2023-10-18T10:24:38.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.802+0000 7f8cc17fa640 1 -- 172.21.15.125:0/417710032 <== mon.1 v2:172.21.15.139:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) v1 ==== 117+0+0 (secure 0 0 0) 0x7f8cb0017850 con 0x7f8cc40ac300 2023-10-18T10:24:38.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.804+0000 7f8caeffd640 1 -- 172.21.15.125:0/417710032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cc40ac300 msgr2=0x7f8cc4149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.804+0000 7f8caeffd640 1 --2- 172.21.15.125:0/417710032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cc40ac300 0x7f8cc4149720 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000c760 tx=0x7f8cb000cc20 comp rx=0 tx=0).stop 2023-10-18T10:24:38.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.804+0000 7f8caeffd640 1 -- 172.21.15.125:0/417710032 shutdown_connections 2023-10-18T10:24:38.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.804+0000 7f8caeffd640 1 --2- 172.21.15.125:0/417710032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cc40ac300 0x7f8cc4149720 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.804+0000 7f8caeffd640 1 --2- 172.21.15.125:0/417710032 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cc40ac6e0 0x7f8cc4149c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.804+0000 7f8caeffd640 1 --2- 172.21.15.125:0/417710032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8cc4069720 0x7f8cc4141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.804+0000 7f8caeffd640 1 -- 172.21.15.125:0/417710032 >> 172.21.15.125:0/417710032 conn(0x7f8cc40b0cd0 msgr2=0x7f8cc414c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.806+0000 7f3065573640 1 -- 172.21.15.125:0/2573589828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015bd90 msgr2=0x7f306015c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.806+0000 7f3065573640 1 --2- 172.21.15.125:0/2573589828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015bd90 0x7f306015c270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30600b1de0 tx=0x7f305002bdb0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.806+0000 7f3065573640 1 -- 172.21.15.125:0/2573589828 shutdown_connections 2023-10-18T10:24:38.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.806+0000 7f3065573640 1 --2- 172.21.15.125:0/2573589828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015bd90 0x7f306015c270 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30600b1de0 tx=0x7f305002bdb0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.806+0000 7f3065573640 1 --2- 172.21.15.125:0/2573589828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f306015c7b0 0x7f306006bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.806+0000 7f3065573640 1 --2- 172.21.15.125:0/2573589828 >> v2:172.21.15.139:3300/0 conn(0x7f306006cf20 0x7f306015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.806+0000 7f3065573640 1 -- 172.21.15.125:0/2573589828 >> 172.21.15.125:0/2573589828 conn(0x7f30600ae3f0 msgr2=0x7f30600ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.809+0000 7f8caeffd640 1 -- 172.21.15.125:0/417710032 shutdown_connections 2023-10-18T10:24:38.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.809+0000 7f8caeffd640 1 -- 172.21.15.125:0/417710032 wait complete. 2023-10-18T10:24:38.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.813+0000 7f3065573640 1 -- 172.21.15.125:0/2573589828 shutdown_connections 2023-10-18T10:24:38.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.813+0000 7f3065573640 1 -- 172.21.15.125:0/2573589828 wait complete. 2023-10-18T10:24:38.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.815+0000 7f3065573640 1 Processor -- start 2023-10-18T10:24:38.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.815+0000 7f3065573640 1 -- start start 2023-10-18T10:24:38.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.817+0000 7f3065573640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015c7b0 0x7f3060156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.818+0000 7f305effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015c7b0 0x7f3060156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.818+0000 7f305effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015c7b0 0x7f3060156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35848/0 (socket says 172.21.15.125:35848) 2023-10-18T10:24:38.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.819+0000 7f305effd640 1 -- 172.21.15.125:0/1444374969 learned_addr learned my addr 172.21.15.125:0/1444374969 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:38.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.819+0000 7f3065573640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30600ab030 0x7f30600a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.819+0000 7f3065573640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cb610 0x7f30600cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.819+0000 7f3065573640 1 -- 172.21.15.125:0/1444374969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600c9150 con 0x7f306015c7b0 2023-10-18T10:24:38.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.819+0000 7f3065573640 1 -- 172.21.15.125:0/1444374969 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600cbfd0 con 0x7f30600ab030 2023-10-18T10:24:38.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.819+0000 7f305f7fe640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cb610 0x7f30600cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.819+0000 7f3065573640 1 -- 172.21.15.125:0/1444374969 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600a6d00 con 0x7f30600cb610 2023-10-18T10:24:38.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.821+0000 7f305effd640 1 -- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cb610 msgr2=0x7f30600cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.821+0000 7f305effd640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cb610 0x7f30600cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.821+0000 7f305effd640 1 -- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30600ab030 msgr2=0x7f30600a64b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:24:38.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.821+0000 7f305effd640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30600ab030 0x7f30600a64b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.821+0000 7f305effd640 1 -- 172.21.15.125:0/1444374969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3050009560 con 0x7f306015c7b0 2023-10-18T10:24:38.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.821+0000 7f305effd640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015c7b0 0x7f3060156370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f305400ca80 tx=0x7f305400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:38.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.822+0000 7f30477fe640 1 -- 172.21.15.125:0/1444374969 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054004220 con 0x7f306015c7b0 2023-10-18T10:24:38.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.822+0000 7f3065573640 1 -- 172.21.15.125:0/1444374969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30600a6fe0 con 0x7f306015c7b0 2023-10-18T10:24:38.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.823+0000 7f3065573640 1 -- 172.21.15.125:0/1444374969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30600c9360 con 0x7f306015c7b0 2023-10-18T10:24:38.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.824+0000 7f30477fe640 1 -- 172.21.15.125:0/1444374969 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3054004470 con 0x7f306015c7b0 2023-10-18T10:24:38.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.824+0000 7f30477fe640 1 -- 172.21.15.125:0/1444374969 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054007b60 con 0x7f306015c7b0 2023-10-18T10:24:38.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.825+0000 7f30477fe640 1 -- 172.21.15.125:0/1444374969 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3054007d40 con 0x7f306015c7b0 2023-10-18T10:24:38.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.825+0000 7f30477fe640 1 -- 172.21.15.125:0/1444374969 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f305400f660 con 0x7f306015c7b0 2023-10-18T10:24:38.832 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.832 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.833 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.830+0000 7f3065573640 1 -- 172.21.15.125:0/1444374969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3040005250 con 0x7f306015c7b0 2023-10-18T10:24:38.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.843+0000 7f30477fe640 1 -- 172.21.15.125:0/1444374969 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f305401d060 con 0x7f306015c7b0 2023-10-18T10:24:38.851 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.881+0000 7f6b7601a640 1 -- 172.21.15.125:0/1492189698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e02a0 msgr2=0x7f6b680e3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.881+0000 7f6b7601a640 1 --2- 172.21.15.125:0/1492189698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e02a0 0x7f6b680e3b90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6b680e3ad0 tx=0x7f6b6002be60 comp rx=0 tx=0).stop 2023-10-18T10:24:38.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.887+0000 7f6b7601a640 1 -- 172.21.15.125:0/1492189698 shutdown_connections 2023-10-18T10:24:38.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.887+0000 7f6b7601a640 1 --2- 172.21.15.125:0/1492189698 >> v2:172.21.15.139:3300/0 conn(0x7f6b680e40d0 0x7f6b680539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.887+0000 7f6b7601a640 1 --2- 172.21.15.125:0/1492189698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e02a0 0x7f6b680e3b90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.887+0000 7f6b7601a640 1 -- 172.21.15.125:0/1492189698 >> 172.21.15.125:0/1492189698 conn(0x7f6b6801ef10 msgr2=0x7f6b68055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.890+0000 7f2b694b4640 1 -- 172.21.15.125:0/2159490734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7f2b440052f0 con 0x7f2b6415b430 2023-10-18T10:24:38.892 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:38.891+0000 7f304f80f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:24:38.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.890+0000 7f6b7601a640 1 -- 172.21.15.125:0/1492189698 shutdown_connections 2023-10-18T10:24:38.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.890+0000 7f6b7601a640 1 -- 172.21.15.125:0/1492189698 wait complete. 2023-10-18T10:24:38.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.892+0000 7f6b7601a640 1 Processor -- start 2023-10-18T10:24:38.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.892+0000 7f6b7601a640 1 -- start start 2023-10-18T10:24:38.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.894+0000 7f6b7601a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680dfec0 0x7f6b680e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.894+0000 7f6b7601a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b680e40d0 0x7f6b680ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.894+0000 7f6b7601a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e0690 0x7f6b68034bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.894+0000 7f6b75018640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680dfec0 0x7f6b680e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.894+0000 7f6b75018640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680dfec0 0x7f6b680e3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:45888/0 (socket says 172.21.15.125:45888) 2023-10-18T10:24:38.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.894+0000 7f6b75819640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e0690 0x7f6b68034bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.895+0000 7f6b75819640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e0690 0x7f6b68034bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35854/0 (socket says 172.21.15.125:35854) 2023-10-18T10:24:38.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.895+0000 7f6b75018640 1 -- 172.21.15.125:0/2098834788 learned_addr learned my addr 172.21.15.125:0/2098834788 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:38.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.895+0000 7f6b74817640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b680e40d0 0x7f6b680ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.895+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680de400 con 0x7f6b680e0690 2023-10-18T10:24:38.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.895+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680e1610 con 0x7f6b680dfec0 2023-10-18T10:24:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.895+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68050000 con 0x7f6b680e40d0 2023-10-18T10:24:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.896+0000 7f6b75819640 1 -- 172.21.15.125:0/2098834788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b680e40d0 msgr2=0x7f6b680ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.896+0000 7f6b75819640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b680e40d0 0x7f6b680ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.896+0000 7f6b75819640 1 -- 172.21.15.125:0/2098834788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680dfec0 msgr2=0x7f6b680e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.896+0000 7f6b75819640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680dfec0 0x7f6b680e3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.896+0000 7f6b75819640 1 -- 172.21.15.125:0/2098834788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60009560 con 0x7f6b680e0690 2023-10-18T10:24:38.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.896+0000 7f6b75819640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e0690 0x7f6b68034bf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b700771b0 tx=0x7f6b70069570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:38.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.897+0000 7f6b74817640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b680e40d0 0x7f6b680ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:24:38.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.897+0000 7f6b65ffb640 1 -- 172.21.15.125:0/2098834788 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70066d30 con 0x7f6b680e0690 2023-10-18T10:24:38.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.897+0000 7f6b65ffb640 1 -- 172.21.15.125:0/2098834788 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b70066f80 con 0x7f6b680e0690 2023-10-18T10:24:38.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.897+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b680e1330 con 0x7f6b680e0690 2023-10-18T10:24:38.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.898+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6804fd20 con 0x7f6b680e0690 2023-10-18T10:24:38.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.898+0000 7f6b65ffb640 1 -- 172.21.15.125:0/2098834788 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70071fe0 con 0x7f6b680e0690 2023-10-18T10:24:38.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.899+0000 7f6b65ffb640 1 -- 172.21.15.125:0/2098834788 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6b700727b0 con 0x7f6b680e0690 2023-10-18T10:24:38.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.900+0000 7f6b65ffb640 1 -- 172.21.15.125:0/2098834788 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6b70069d70 con 0x7f6b680e0690 2023-10-18T10:24:38.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.900+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b50005250 con 0x7f6b680e0690 2023-10-18T10:24:38.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.906+0000 7f6b65ffb640 1 -- 172.21.15.125:0/2098834788 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f6b7008e030 con 0x7f6b680e0690 2023-10-18T10:24:38.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.913+0000 7f2b4b7fe640 1 -- 172.21.15.125:0/2159490734 <== mon.1 v2:172.21.15.139:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) v1 ==== 106+0+0 (secure 0 0 0) 0x7f2b5c057070 con 0x7f2b6415b430 2023-10-18T10:24:38.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 -- 172.21.15.125:0/2159490734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b6415b430 msgr2=0x7f2b64157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b6415b430 0x7f2b64157b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c02c870 tx=0x7f2b5c034670 comp rx=0 tx=0).stop 2023-10-18T10:24:38.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 -- 172.21.15.125:0/2159490734 shutdown_connections 2023-10-18T10:24:38.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b6415b430 0x7f2b64157b10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b6415ab30 0x7f2b640a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 --2- 172.21.15.125:0/2159490734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b640b0750 0x7f2b640a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 -- 172.21.15.125:0/2159490734 >> 172.21.15.125:0/2159490734 conn(0x7f2b640ae720 msgr2=0x7f2b640aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 -- 172.21.15.125:0/2159490734 shutdown_connections 2023-10-18T10:24:38.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.914+0000 7f2b497fa640 1 -- 172.21.15.125:0/2159490734 wait complete. 2023-10-18T10:24:38.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.924+0000 7ffa3ea26640 1 -- 172.21.15.125:0/4011574411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3004f250 msgr2=0x7ffa30006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.924+0000 7ffa3ea26640 1 --2- 172.21.15.125:0/4011574411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3004f250 0x7ffa30006e60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffa30053420 tx=0x7ffa2402be20 comp rx=0 tx=0).stop 2023-10-18T10:24:38.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.924+0000 7fec66cc2640 1 -- 172.21.15.125:0/1416604224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec480047c0 msgr2=0x7fec48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.924+0000 7fec66cc2640 1 --2- 172.21.15.125:0/1416604224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec480047c0 0x7fec48004c40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec540017a0 tx=0x7fec480057f0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.930+0000 7fec6525f640 1 -- 172.21.15.125:0/1416604224 reap_dead start 2023-10-18T10:24:38.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.930+0000 7fec66cc2640 1 -- 172.21.15.125:0/1416604224 shutdown_connections 2023-10-18T10:24:38.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.930+0000 7fec66cc2640 1 -- 172.21.15.125:0/1416604224 >> 172.21.15.125:0/1416604224 conn(0x7fec600ae720 msgr2=0x7fec600aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.930+0000 7ffa3ea26640 1 -- 172.21.15.125:0/4011574411 shutdown_connections 2023-10-18T10:24:38.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.930+0000 7ffa3ea26640 1 --2- 172.21.15.125:0/4011574411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa300073a0 0x7ffa30037be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.930+0000 7ffa3ea26640 1 --2- 172.21.15.125:0/4011574411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3004f250 0x7ffa30006e60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.930+0000 7ffa3ea26640 1 -- 172.21.15.125:0/4011574411 >> 172.21.15.125:0/4011574411 conn(0x7ffa30009f70 msgr2=0x7ffa3000a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.931+0000 7ffa3ea26640 1 -- 172.21.15.125:0/4011574411 shutdown_connections 2023-10-18T10:24:38.938 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi125') 2023-10-18T10:24:38.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.931+0000 7ffa3ea26640 1 -- 172.21.15.125:0/4011574411 wait complete. 2023-10-18T10:24:38.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.933+0000 7ffa3ea26640 1 Processor -- start 2023-10-18T10:24:38.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.933+0000 7ffa3ea26640 1 -- start start 2023-10-18T10:24:38.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.934+0000 7ffa3ea26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa300073a0 0x7ffa3002fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.934+0000 7ffa3ea26640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa3004f250 0x7ffa300deed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.934+0000 7ffa3ea26640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3019f2c0 0x7ffa301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.935+0000 7ffa3ea26640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa30181c90 con 0x7ffa3019f2c0 2023-10-18T10:24:38.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.935+0000 7ffa3ea26640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300df410 con 0x7ffa300073a0 2023-10-18T10:24:38.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.935+0000 7ffa3ea26640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300df670 con 0x7ffa3004f250 2023-10-18T10:24:38.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3019f2c0 0x7ffa301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3019f2c0 0x7ffa301989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35882/0 (socket says 172.21.15.125:35882) 2023-10-18T10:24:38.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 -- 172.21.15.125:0/2814458480 learned_addr learned my addr 172.21.15.125:0/2814458480 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:38.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 -- 172.21.15.125:0/2814458480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa3004f250 msgr2=0x7ffa300deed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:24:38.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 --2- 172.21.15.125:0/2814458480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa3004f250 0x7ffa300deed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 -- 172.21.15.125:0/2814458480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa300073a0 msgr2=0x7ffa3002fd50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:24:38.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 --2- 172.21.15.125:0/2814458480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa300073a0 0x7ffa3002fd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 -- 172.21.15.125:0/2814458480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa24009560 con 0x7ffa3019f2c0 2023-10-18T10:24:38.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.937+0000 7ffa3cfc3640 1 --2- 172.21.15.125:0/2814458480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3019f2c0 0x7ffa301989f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000b60 tx=0x7ffa380694b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:38.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.936+0000 7fec66cc2640 1 -- 172.21.15.125:0/1416604224 shutdown_connections 2023-10-18T10:24:38.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.936+0000 7fec66cc2640 1 -- 172.21.15.125:0/1416604224 wait complete. 2023-10-18T10:24:38.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.938+0000 7fec66cc2640 1 Processor -- start 2023-10-18T10:24:38.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.938+0000 7fec66cc2640 1 -- start start 2023-10-18T10:24:38.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7ffa34ff9640 1 -- 172.21.15.125:0/2814458480 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38066ee0 con 0x7ffa3019f2c0 2023-10-18T10:24:38.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7fec66cc2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60157b80 0x7fec60164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7fec66cc2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec6015ebf0 0x7fec60162610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7fec66cc2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec60162b50 0x7fec60201a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7fec66cc2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60163000 con 0x7fec60162b50 2023-10-18T10:24:38.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7fec66cc2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec601692e0 con 0x7fec60157b80 2023-10-18T10:24:38.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7fec66cc2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6015f160 con 0x7fec6015ebf0 2023-10-18T10:24:38.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.940+0000 7fec6525f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec60162b50 0x7fec60201a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.940+0000 7fec6525f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec60162b50 0x7fec60201a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35890/0 (socket says 172.21.15.125:35890) 2023-10-18T10:24:38.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.940+0000 7fec6525f640 1 -- 172.21.15.125:0/1116456385 learned_addr learned my addr 172.21.15.125:0/1116456385 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:38.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.940+0000 7fec64a5e640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60157b80 0x7fec60164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa300df950 con 0x7ffa3019f2c0 2023-10-18T10:24:38.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.939+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa30031970 con 0x7ffa3019f2c0 2023-10-18T10:24:38.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.940+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa30008990 con 0x7ffa3019f2c0 2023-10-18T10:24:38.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.941+0000 7ffa34ff9640 1 -- 172.21.15.125:0/2814458480 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa38071fe0 con 0x7ffa3019f2c0 2023-10-18T10:24:38.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.941+0000 7ffa34ff9640 1 -- 172.21.15.125:0/2814458480 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38072370 con 0x7ffa3019f2c0 2023-10-18T10:24:38.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.942+0000 7ffa34ff9640 1 -- 172.21.15.125:0/2814458480 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffa3806a8a0 con 0x7ffa3019f2c0 2023-10-18T10:24:38.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.942+0000 7ffa34ff9640 1 -- 172.21.15.125:0/2814458480 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffa38072bb0 con 0x7ffa3019f2c0 2023-10-18T10:24:38.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.943+0000 7f206df73640 1 -- 172.21.15.125:0/1364062082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f20680882e0 msgr2=0x7f2068088770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.943+0000 7f206df73640 1 --2- 172.21.15.125:0/1364062082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f20680882e0 0x7f2068088770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2060005fd0 tx=0x7f206002c2f0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.943+0000 7f206df73640 1 -- 172.21.15.125:0/1364062082 shutdown_connections 2023-10-18T10:24:38.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.943+0000 7f206df73640 1 --2- 172.21.15.125:0/1364062082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20680b02a0 0x7f20680b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.943+0000 7f206df73640 1 --2- 172.21.15.125:0/1364062082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f20680882e0 0x7f2068088770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.943+0000 7f206df73640 1 -- 172.21.15.125:0/1364062082 >> 172.21.15.125:0/1364062082 conn(0x7f20680ae720 msgr2=0x7f20680aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:38.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.945+0000 7ffa34ff9640 1 -- 172.21.15.125:0/2814458480 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ffa3808e060 con 0x7ffa3019f2c0 2023-10-18T10:24:38.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.941+0000 7fec6525f640 1 -- 172.21.15.125:0/1116456385 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec6015ebf0 msgr2=0x7fec60162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.949+0000 7f206df73640 1 -- 172.21.15.125:0/1364062082 shutdown_connections 2023-10-18T10:24:38.953 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:38.945+0000 7f304f80f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:24:38.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.941+0000 7fec6525f640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec6015ebf0 0x7fec60162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.941+0000 7fec6525f640 1 -- 172.21.15.125:0/1116456385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60157b80 msgr2=0x7fec60164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:38.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.941+0000 7fec6525f640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60157b80 0x7fec60164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.941+0000 7fec6525f640 1 -- 172.21.15.125:0/1116456385 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6015f440 con 0x7fec60162b50 2023-10-18T10:24:38.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.941+0000 7fec6525f640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec60162b50 0x7fec60201a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec600b1ed0 tx=0x7fec5800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:38.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.949+0000 7fec64a5e640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60157b80 0x7fec60164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:24:38.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.949+0000 7fec5d7fa640 1 -- 172.21.15.125:0/1116456385 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5800ac00 con 0x7fec60162b50 2023-10-18T10:24:38.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.949+0000 7fec5d7fa640 1 -- 172.21.15.125:0/1116456385 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec5800ae50 con 0x7fec60162b50 2023-10-18T10:24:38.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.950+0000 7fec66cc2640 1 -- 172.21.15.125:0/1116456385 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6015f720 con 0x7fec60162b50 2023-10-18T10:24:38.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.950+0000 7fec66cc2640 1 -- 172.21.15.125:0/1116456385 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec60164cb0 con 0x7fec60162b50 2023-10-18T10:24:38.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.951+0000 7fec5d7fa640 1 -- 172.21.15.125:0/1116456385 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec580072d0 con 0x7fec60162b50 2023-10-18T10:24:38.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.952+0000 7fec5d7fa640 1 -- 172.21.15.125:0/1116456385 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fec580077b0 con 0x7fec60162b50 2023-10-18T10:24:38.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.952+0000 7fec5d7fa640 1 -- 172.21.15.125:0/1116456385 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fec58012740 con 0x7fec60162b50 2023-10-18T10:24:38.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.955+0000 7f206df73640 1 -- 172.21.15.125:0/1364062082 wait complete. 2023-10-18T10:24:38.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.956+0000 7f206df73640 1 Processor -- start 2023-10-18T10:24:38.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.957+0000 7fec46ffd640 1 -- 172.21.15.125:0/1116456385 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec40005250 con 0x7fec60162b50 2023-10-18T10:24:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.958+0000 7f206df73640 1 -- start start 2023-10-18T10:24:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.959+0000 7f206df73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f206806f920 0x7f206816a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.959+0000 7f206df73640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20680882e0 0x7f20680ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.959+0000 7f206df73640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20680b02a0 0x7f20680acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.959+0000 7f206df73640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2068163590 con 0x7f206806f920 2023-10-18T10:24:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.959+0000 7f206df73640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20680871e0 con 0x7f20680882e0 2023-10-18T10:24:38.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.959+0000 7f206df73640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20680874c0 con 0x7f20680b02a0 2023-10-18T10:24:38.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.960+0000 7f20677fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f206806f920 0x7f206816a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:38.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.961+0000 7f20677fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f206806f920 0x7f206816a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35894/0 (socket says 172.21.15.125:35894) 2023-10-18T10:24:38.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.961+0000 7f20677fe640 1 -- 172.21.15.125:0/2974615588 learned_addr learned my addr 172.21.15.125:0/2974615588 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:38.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.961+0000 7fec5d7fa640 1 -- 172.21.15.125:0/1116456385 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fec58022060 con 0x7fec60162b50 2023-10-18T10:24:38.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.965+0000 7f20677fe640 1 -- 172.21.15.125:0/2974615588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20680b02a0 msgr2=0x7f20680acad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:24:38.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.965+0000 7f20677fe640 1 --2- 172.21.15.125:0/2974615588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20680b02a0 0x7f20680acad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.965+0000 7f20677fe640 1 -- 172.21.15.125:0/2974615588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20680882e0 msgr2=0x7f20680ac590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:24:38.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.965+0000 7f20677fe640 1 --2- 172.21.15.125:0/2974615588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20680882e0 0x7f20680ac590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:38.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.965+0000 7f20677fe640 1 -- 172.21.15.125:0/2974615588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2060009900 con 0x7f206806f920 2023-10-18T10:24:38.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.966+0000 7f20677fe640 1 --2- 172.21.15.125:0/2974615588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f206806f920 0x7f206816a480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f205800c430 tx=0x7f205800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:38.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.966+0000 7f204ffff640 1 -- 172.21.15.125:0/2974615588 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058011790 con 0x7f206806f920 2023-10-18T10:24:38.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.966+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206806ff10 con 0x7f206806f920 2023-10-18T10:24:38.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.966+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2068162ed0 con 0x7f206806f920 2023-10-18T10:24:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.967+0000 7f204ffff640 1 -- 172.21.15.125:0/2974615588 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f20580119e0 con 0x7f206806f920 2023-10-18T10:24:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.969+0000 7f204ffff640 1 -- 172.21.15.125:0/2974615588 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058011e30 con 0x7f206806f920 2023-10-18T10:24:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.969+0000 7f204ffff640 1 -- 172.21.15.125:0/2974615588 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f205801c760 con 0x7f206806f920 2023-10-18T10:24:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.969+0000 7f204ffff640 1 -- 172.21.15.125:0/2974615588 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f20580129a0 con 0x7f206806f920 2023-10-18T10:24:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.970+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2048005250 con 0x7f206806f920 2023-10-18T10:24:38.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:38.974+0000 7f204ffff640 1 -- 172.21.15.125:0/2974615588 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2058012420 con 0x7f206806f920 2023-10-18T10:24:39.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.001+0000 7f948d3c5640 1 -- 172.21.15.125:0/410447606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9488165660 msgr2=0x7f948806f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.001+0000 7f948d3c5640 1 --2- 172.21.15.125:0/410447606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9488165660 0x7f948806f920 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9478009830 tx=0x7f947802baa0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.001+0000 7f948d3c5640 1 -- 172.21.15.125:0/410447606 shutdown_connections 2023-10-18T10:24:39.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.001+0000 7f948d3c5640 1 --2- 172.21.15.125:0/410447606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9488165660 0x7f948806f920 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.001+0000 7f948d3c5640 1 -- 172.21.15.125:0/410447606 >> 172.21.15.125:0/410447606 conn(0x7f94880ae720 msgr2=0x7f94880aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.002+0000 7f8dd0bc5640 1 -- 172.21.15.125:0/2211238072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f8da40050c0 con 0x7f8dcc0b12a0 2023-10-18T10:24:39.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.002+0000 7f948d3c5640 1 -- 172.21.15.125:0/410447606 shutdown_connections 2023-10-18T10:24:39.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.002+0000 7f948d3c5640 1 -- 172.21.15.125:0/410447606 wait complete. 2023-10-18T10:24:39.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.003+0000 7f948d3c5640 1 Processor -- start 2023-10-18T10:24:39.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.004+0000 7f948d3c5640 1 -- start start 2023-10-18T10:24:39.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f948d3c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94880882e0 0x7f9488177bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f948d3c5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94880b09b0 0x7f9488178130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f948d3c5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9488178d00 0x7f9488178670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f948d3c5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948806d190 con 0x7f94880b09b0 2023-10-18T10:24:39.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f948d3c5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9488088100 con 0x7f94880882e0 2023-10-18T10:24:39.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f948d3c5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94880c8fb0 con 0x7f9488178d00 2023-10-18T10:24:39.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f94867fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94880b09b0 0x7f9488178130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f94867fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94880b09b0 0x7f9488178130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35898/0 (socket says 172.21.15.125:35898) 2023-10-18T10:24:39.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.005+0000 7f94867fc640 1 -- 172.21.15.125:0/2666595352 learned_addr learned my addr 172.21.15.125:0/2666595352 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:39.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.006+0000 7f94877fe640 1 --2- 172.21.15.125:0/2666595352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9488178d00 0x7f9488178670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.006+0000 7f94867fc640 1 -- 172.21.15.125:0/2666595352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9488178d00 msgr2=0x7f9488178670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.006+0000 7f94867fc640 1 --2- 172.21.15.125:0/2666595352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9488178d00 0x7f9488178670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.006+0000 7f94867fc640 1 -- 172.21.15.125:0/2666595352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94880882e0 msgr2=0x7f9488177bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.006+0000 7f94867fc640 1 --2- 172.21.15.125:0/2666595352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94880882e0 0x7f9488177bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.006+0000 7f94867fc640 1 -- 172.21.15.125:0/2666595352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9478009490 con 0x7f94880b09b0 2023-10-18T10:24:39.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.009+0000 7f8dbaffd640 1 -- 172.21.15.125:0/2211238072 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v10)=0 v10) v1 ==== 116+0+0 (secure 0 0 0) 0x7f8dc0066020 con 0x7f8dcc0b12a0 2023-10-18T10:24:39.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.009+0000 7f94867fc640 1 --2- 172.21.15.125:0/2666595352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94880b09b0 0x7f9488178130 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f948000cb20 tx=0x7f9480002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:39.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.010+0000 7f946f7fe640 1 -- 172.21.15.125:0/2666595352 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480003da0 con 0x7f94880b09b0 2023-10-18T10:24:39.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.010+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948806cb40 con 0x7f94880b09b0 2023-10-18T10:24:39.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.010+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948806c820 con 0x7f94880b09b0 2023-10-18T10:24:39.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.011+0000 7f8db8ff9640 1 -- 172.21.15.125:0/2211238072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc0b12a0 msgr2=0x7f8dcc144500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.011+0000 7f8db8ff9640 1 --2- 172.21.15.125:0/2211238072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc0b12a0 0x7f8dcc144500 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0044000 tx=0x7f8dc0003820 comp rx=0 tx=0).stop 2023-10-18T10:24:39.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.012+0000 7f946f7fe640 1 -- 172.21.15.125:0/2666595352 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f948000a760 con 0x7f94880b09b0 2023-10-18T10:24:39.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.012+0000 7f946f7fe640 1 -- 172.21.15.125:0/2666595352 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000ab70 con 0x7f94880b09b0 2023-10-18T10:24:39.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.012+0000 7f946f7fe640 1 -- 172.21.15.125:0/2666595352 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9480005370 con 0x7f94880b09b0 2023-10-18T10:24:39.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.012+0000 7f946f7fe640 1 -- 172.21.15.125:0/2666595352 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f948001ccf0 con 0x7f94880b09b0 2023-10-18T10:24:39.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.017+0000 7f8db8ff9640 1 -- 172.21.15.125:0/2211238072 shutdown_connections 2023-10-18T10:24:39.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.017+0000 7f8db8ff9640 1 --2- 172.21.15.125:0/2211238072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8dcc15b550 0x7f8dcc0cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.017+0000 7f8db8ff9640 1 --2- 172.21.15.125:0/2211238072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8dcc0b12a0 0x7f8dcc144500 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.017+0000 7f8db8ff9640 1 --2- 172.21.15.125:0/2211238072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dcc06e0b0 0x7f8dcc1ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.017+0000 7f8db8ff9640 1 -- 172.21.15.125:0/2211238072 >> 172.21.15.125:0/2211238072 conn(0x7f8dcc0af730 msgr2=0x7f8dcc06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.017+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9468005250 con 0x7f94880b09b0 2023-10-18T10:24:39.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.018+0000 7f4cf28ea640 1 -- 172.21.15.125:0/2779090178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce4050170 msgr2=0x7f4ce4038050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.018+0000 7f4cf28ea640 1 --2- 172.21.15.125:0/2779090178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce4050170 0x7f4ce4038050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4cec068940 tx=0x7f4cec0a4e30 comp rx=0 tx=0).stop 2023-10-18T10:24:39.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.017+0000 7f8db8ff9640 1 -- 172.21.15.125:0/2211238072 shutdown_connections 2023-10-18T10:24:39.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.017+0000 7f8db8ff9640 1 -- 172.21.15.125:0/2211238072 wait complete. 2023-10-18T10:24:39.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.019+0000 7f4cf28ea640 1 -- 172.21.15.125:0/2779090178 shutdown_connections 2023-10-18T10:24:39.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.019+0000 7f4cf28ea640 1 --2- 172.21.15.125:0/2779090178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ce4053c40 0x7f4ce40069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.019+0000 7f4cf28ea640 1 --2- 172.21.15.125:0/2779090178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce4050170 0x7f4ce4038050 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4cec068940 tx=0x7f4cec0a4e30 comp rx=0 tx=0).stop 2023-10-18T10:24:39.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.019+0000 7f4cf28ea640 1 -- 172.21.15.125:0/2779090178 >> 172.21.15.125:0/2779090178 conn(0x7f4ce4009f80 msgr2=0x7f4ce400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.021+0000 7f4cf28ea640 1 -- 172.21.15.125:0/2779090178 shutdown_connections 2023-10-18T10:24:39.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.021+0000 7f946f7fe640 1 -- 172.21.15.125:0/2666595352 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9480017070 con 0x7f94880b09b0 2023-10-18T10:24:39.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.028+0000 7f4cf28ea640 1 -- 172.21.15.125:0/2779090178 wait complete. 2023-10-18T10:24:39.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.029+0000 7f4cf28ea640 1 Processor -- start 2023-10-18T10:24:39.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.029+0000 7f4cf28ea640 1 -- start start 2023-10-18T10:24:39.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.031+0000 7f4cf28ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ce4053c40 0x7f4ce40e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.031+0000 7f4cf28ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce40df1f0 0x7f4ce4181420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.031+0000 7f4cf28ea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ce4182030 0x7f4ce4181960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.031+0000 7f4cf28ea640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40cf780 con 0x7f4ce4053c40 2023-10-18T10:24:39.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.031+0000 7f4cf28ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40e1390 con 0x7f4ce40df1f0 2023-10-18T10:24:39.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.031+0000 7f4cf28ea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40e10b0 con 0x7f4ce4182030 2023-10-18T10:24:39.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.033+0000 7f4cf10e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce40df1f0 0x7f4ce4181420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.033+0000 7f4cf10e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce40df1f0 0x7f4ce4181420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:45968/0 (socket says 172.21.15.125:45968) 2023-10-18T10:24:39.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.033+0000 7f4cf10e7640 1 -- 172.21.15.125:0/2340598859 learned_addr learned my addr 172.21.15.125:0/2340598859 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:39.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.037+0000 7f4cf10e7640 1 -- 172.21.15.125:0/2340598859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ce4182030 msgr2=0x7f4ce4181960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:24:39.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.037+0000 7f4cf10e7640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ce4182030 0x7f4ce4181960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.037+0000 7f4cf10e7640 1 -- 172.21.15.125:0/2340598859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ce4053c40 msgr2=0x7f4ce40e1e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:24:39.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.040+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f53f80052f0 con 0x7f54181573f0 2023-10-18T10:24:39.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.041+0000 7f4cf18e8640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ce4053c40 0x7f4ce40e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.042+0000 7f4cf10e7640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ce4053c40 0x7f4ce40e1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.042+0000 7f4cf10e7640 1 -- 172.21.15.125:0/2340598859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec04e960 con 0x7f4ce40df1f0 2023-10-18T10:24:39.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.042+0000 7f4cf18e8640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ce4053c40 0x7f4ce40e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:24:39.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.042+0000 7f4cf10e7640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce40df1f0 0x7f4ce4181420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4cec067d70 tx=0x7f4cec0ade20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:39.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.042+0000 7f4ce27fc640 1 -- 172.21.15.125:0/2340598859 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec0bc070 con 0x7f4ce40df1f0 2023-10-18T10:24:39.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.042+0000 7f4ce27fc640 1 -- 172.21.15.125:0/2340598859 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4cec0b6630 con 0x7f4ce40df1f0 2023-10-18T10:24:39.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.042+0000 7f4cf28ea640 1 -- 172.21.15.125:0/2340598859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce404bc90 con 0x7f4ce40df1f0 2023-10-18T10:24:39.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.042+0000 7f4cf28ea640 1 -- 172.21.15.125:0/2340598859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce404c250 con 0x7f4ce40df1f0 2023-10-18T10:24:39.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.047+0000 7f4cd7fff640 1 -- 172.21.15.125:0/2340598859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce404c470 con 0x7f4ce40df1f0 2023-10-18T10:24:39.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.049+0000 7f4ce27fc640 1 -- 172.21.15.125:0/2340598859 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec0b6a60 con 0x7f4ce40df1f0 2023-10-18T10:24:39.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.049+0000 7f4ce27fc640 1 -- 172.21.15.125:0/2340598859 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4cec0b4070 con 0x7f4ce40df1f0 2023-10-18T10:24:39.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.049+0000 7f4ce27fc640 1 -- 172.21.15.125:0/2340598859 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4cec0c1410 con 0x7f4ce40df1f0 2023-10-18T10:24:39.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.054+0000 7faa47890640 1 -- 172.21.15.125:0/3876538558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2c005180 msgr2=0x7faa2c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.054+0000 7faa47890640 1 --2- 172.21.15.125:0/3876538558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2c005180 0x7faa2c00b980 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa3c003f80 tx=0x7faa2c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.054+0000 7faa45e2d640 1 -- 172.21.15.125:0/3876538558 reap_dead start 2023-10-18T10:24:39.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.054+0000 7faa47890640 1 -- 172.21.15.125:0/3876538558 shutdown_connections 2023-10-18T10:24:39.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.054+0000 7faa47890640 1 -- 172.21.15.125:0/3876538558 >> 172.21.15.125:0/3876538558 conn(0x7faa38009f70 msgr2=0x7faa3800a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.054+0000 7f4ce27fc640 1 -- 172.21.15.125:0/2340598859 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4cec0b2070 con 0x7f4ce40df1f0 2023-10-18T10:24:39.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.054+0000 7f7fca2da640 1 -- 172.21.15.125:0/3113524161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc415a7f0 msgr2=0x7f7fc4151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.055+0000 7f7fca2da640 1 --2- 172.21.15.125:0/3113524161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc415a7f0 0x7f7fc4151200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7fc404eb00 tx=0x7f7fb402bd80 comp rx=0 tx=0).stop 2023-10-18T10:24:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.055+0000 7f7fca2da640 1 -- 172.21.15.125:0/3113524161 shutdown_connections 2023-10-18T10:24:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.055+0000 7f7fca2da640 1 --2- 172.21.15.125:0/3113524161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc415a7f0 0x7f7fc4151200 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7fc404eb00 tx=0x7f7fb402bd80 comp rx=0 tx=0).stop 2023-10-18T10:24:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.055+0000 7f7fca2da640 1 --2- 172.21.15.125:0/3113524161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fc41578c0 0x7f7fc4157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.055+0000 7f7fca2da640 1 --2- 172.21.15.125:0/3113524161 >> v2:172.21.15.139:3300/0 conn(0x7f7fc406ce60 0x7f7fc40af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.055+0000 7f7fca2da640 1 -- 172.21.15.125:0/3113524161 >> 172.21.15.125:0/3113524161 conn(0x7f7fc40ae460 msgr2=0x7f7fc40ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.057+0000 7f7fca2da640 1 -- 172.21.15.125:0/3113524161 shutdown_connections 2023-10-18T10:24:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.054+0000 7faa47890640 1 -- 172.21.15.125:0/3876538558 shutdown_connections 2023-10-18T10:24:39.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.055+0000 7f53fffff640 1 -- 172.21.15.125:0/3752440021 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v11)=0 v11) v1 ==== 127+0+0 (secure 0 0 0) 0x7f540c01bba0 con 0x7f54181573f0 2023-10-18T10:24:39.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.057+0000 7f7fca2da640 1 -- 172.21.15.125:0/3113524161 wait complete. 2023-10-18T10:24:39.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.058+0000 7f7fca2da640 1 Processor -- start 2023-10-18T10:24:39.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.059+0000 7f7fca2da640 1 -- start start 2023-10-18T10:24:39.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fca2da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc406ce60 0x7f7fc4147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fca2da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fc41578c0 0x7f7fc413dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fca2da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fc413e2a0 0x7f7fc40cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fca2da640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40cabe0 con 0x7f7fc406ce60 2023-10-18T10:24:39.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fca2da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc41481e0 con 0x7f7fc41578c0 2023-10-18T10:24:39.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fca2da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc41484c0 con 0x7f7fc413e2a0 2023-10-18T10:24:39.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fc9ad9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fc413e2a0 0x7f7fc40cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54181573f0 msgr2=0x7f54181445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f541da07640 1 --2- 172.21.15.125:0/3752440021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54181573f0 0x7f54181445f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c480 tx=0x7f540c00c940 comp rx=0 tx=0).stop 2023-10-18T10:24:39.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fc92d8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc406ce60 0x7f7fc4147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fc92d8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc406ce60 0x7f7fc4147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35950/0 (socket says 172.21.15.125:35950) 2023-10-18T10:24:39.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.060+0000 7f7fc92d8640 1 -- 172.21.15.125:0/3270117166 learned_addr learned my addr 172.21.15.125:0/3270117166 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:39.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.061+0000 7f7fc92d8640 1 -- 172.21.15.125:0/3270117166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fc413e2a0 msgr2=0x7f7fc40cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.061+0000 7f7fc92d8640 1 --2- 172.21.15.125:0/3270117166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fc413e2a0 0x7f7fc40cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.061+0000 7f7fc92d8640 1 -- 172.21.15.125:0/3270117166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fc41578c0 msgr2=0x7f7fc413dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.061+0000 7f7fc92d8640 1 --2- 172.21.15.125:0/3270117166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fc41578c0 0x7f7fc413dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.061+0000 7f7fc92d8640 1 -- 172.21.15.125:0/3270117166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4009490 con 0x7f7fc406ce60 2023-10-18T10:24:39.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.061+0000 7f7fc92d8640 1 --2- 172.21.15.125:0/3270117166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc406ce60 0x7f7fc4147880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d9d0 tx=0x7f7fc000de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:39.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.061+0000 7f7fb9ffb640 1 -- 172.21.15.125:0/3270117166 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0009b40 con 0x7f7fc406ce60 2023-10-18T10:24:39.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.061+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc41487a0 con 0x7f7fc406ce60 2023-10-18T10:24:39.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc4147dc0 con 0x7f7fc406ce60 2023-10-18T10:24:39.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f7fb9ffb640 1 -- 172.21.15.125:0/3270117166 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7fc0009d90 con 0x7f7fc406ce60 2023-10-18T10:24:39.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 shutdown_connections 2023-10-18T10:24:39.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f541da07640 1 --2- 172.21.15.125:0/3752440021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f541806fdc0 0x7f5418159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f541da07640 1 --2- 172.21.15.125:0/3752440021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54181573f0 0x7f54181445f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f541da07640 1 --2- 172.21.15.125:0/3752440021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f541806ab60 0x7f541808abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 >> 172.21.15.125:0/3752440021 conn(0x7f54180b0cd0 msgr2=0x7f5418147ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.062+0000 7f7fb9ffb640 1 -- 172.21.15.125:0/3270117166 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00106c0 con 0x7f7fc406ce60 2023-10-18T10:24:39.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.063+0000 7f7fb9ffb640 1 -- 172.21.15.125:0/3270117166 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7fc0010b20 con 0x7f7fc406ce60 2023-10-18T10:24:39.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.063+0000 7f7fb9ffb640 1 -- 172.21.15.125:0/3270117166 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7fc000f3c0 con 0x7f7fc406ce60 2023-10-18T10:24:39.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.063+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 shutdown_connections 2023-10-18T10:24:39.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.055+0000 7faa47890640 1 -- 172.21.15.125:0/3876538558 wait complete. 2023-10-18T10:24:39.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.064+0000 7f541da07640 1 -- 172.21.15.125:0/3752440021 wait complete. 2023-10-18T10:24:39.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.056+0000 7faa47890640 1 Processor -- start 2023-10-18T10:24:39.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.064+0000 7faa47890640 1 -- start start 2023-10-18T10:24:39.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.065+0000 7faa47890640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa2c005180 0x7faa380d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.066+0000 7faa47890640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa3804e2f0 0x7faa380dde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.067+0000 7faa47890640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa38186030 0x7faa380355e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.067+0000 7faa44e2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa3804e2f0 0x7faa380dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.067+0000 7faa44e2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa3804e2f0 0x7faa380dde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:45994/0 (socket says 172.21.15.125:45994) 2023-10-18T10:24:39.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.067+0000 7faa44e2b640 1 -- 172.21.15.125:0/207528982 learned_addr learned my addr 172.21.15.125:0/207528982 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:39.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.067+0000 7faa45e2d640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa38186030 0x7faa380355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.067+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa380d2250 con 0x7faa38186030 2023-10-18T10:24:39.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.067+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3804ead0 con 0x7faa3804e2f0 2023-10-18T10:24:39.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.067+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa380d24f0 con 0x7faa2c005180 2023-10-18T10:24:39.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.074+0000 7faa4562c640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa2c005180 0x7faa380d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.074+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa4005250 con 0x7f7fc406ce60 2023-10-18T10:24:39.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.076+0000 7faa44e2b640 1 -- 172.21.15.125:0/207528982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa2c005180 msgr2=0x7faa380d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.076+0000 7faa44e2b640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa2c005180 0x7faa380d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.076+0000 7faa44e2b640 1 -- 172.21.15.125:0/207528982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa38186030 msgr2=0x7faa380355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.078+0000 7f7fb9ffb640 1 -- 172.21.15.125:0/3270117166 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f7fc0021060 con 0x7f7fc406ce60 2023-10-18T10:24:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.082+0000 7faa44e2b640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa38186030 0x7faa380355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.082+0000 7faa44e2b640 1 -- 172.21.15.125:0/207528982 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380d27d0 con 0x7faa3804e2f0 2023-10-18T10:24:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.082+0000 7faa45e2d640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa38186030 0x7faa380355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.082+0000 7faa44e2b640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa3804e2f0 0x7faa380dde00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa400682a0 tx=0x7faa400694b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:39.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.083+0000 7f89ac99e640 1 -- 172.21.15.125:0/587567691 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a80abdf0 msgr2=0x7f89a815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.083+0000 7f89ac99e640 1 --2- 172.21.15.125:0/587567691 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a80abdf0 0x7f89a815bee0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f899c009830 tx=0x7f899c02bb40 comp rx=0 tx=0).stop 2023-10-18T10:24:39.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.083+0000 7f89ac99e640 1 -- 172.21.15.125:0/587567691 shutdown_connections 2023-10-18T10:24:39.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.083+0000 7f89ac99e640 1 --2- 172.21.15.125:0/587567691 >> v2:172.21.15.139:3300/0 conn(0x7f89a806f5b0 0x7f89a8070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.083+0000 7f89ac99e640 1 --2- 172.21.15.125:0/587567691 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a80abdf0 0x7f89a815bee0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.083+0000 7f89ac99e640 1 -- 172.21.15.125:0/587567691 >> 172.21.15.125:0/587567691 conn(0x7f89a80ae720 msgr2=0x7f89a80aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.084+0000 7f3065573640 1 -- 172.21.15.125:0/1444374969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7f30400050c0 con 0x7f306015c7b0 2023-10-18T10:24:39.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.084+0000 7f89ac99e640 1 -- 172.21.15.125:0/587567691 shutdown_connections 2023-10-18T10:24:39.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.084+0000 7f89ac99e640 1 -- 172.21.15.125:0/587567691 wait complete. 2023-10-18T10:24:39.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.085+0000 7faa35ffb640 1 -- 172.21.15.125:0/207528982 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4008d070 con 0x7faa3804e2f0 2023-10-18T10:24:39.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.085+0000 7faa35ffb640 1 -- 172.21.15.125:0/207528982 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa40066d60 con 0x7faa3804e2f0 2023-10-18T10:24:39.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.085+0000 7faa35ffb640 1 -- 172.21.15.125:0/207528982 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40071fe0 con 0x7faa3804e2f0 2023-10-18T10:24:39.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.086+0000 7f89ac99e640 1 Processor -- start 2023-10-18T10:24:39.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.086+0000 7f89ac99e640 1 -- start start 2023-10-18T10:24:39.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.087+0000 7f89ac99e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a806f5b0 0x7f89a8205f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.087+0000 7f89ac99e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89a80abdf0 0x7f89a81591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.087+0000 7f89ac99e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89a815f700 0x7f89a81596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.087+0000 7f89ac99e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8209d50 con 0x7f89a806f5b0 2023-10-18T10:24:39.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.087+0000 7f89ac99e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8209fa0 con 0x7f89a80abdf0 2023-10-18T10:24:39.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.087+0000 7f89ac99e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a820a280 con 0x7f89a815f700 2023-10-18T10:24:39.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.088+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa380d2ab0 con 0x7faa3804e2f0 2023-10-18T10:24:39.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.089+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3804e770 con 0x7faa3804e2f0 2023-10-18T10:24:39.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.089+0000 7faa35ffb640 1 -- 172.21.15.125:0/207528982 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faa40072440 con 0x7faa3804e2f0 2023-10-18T10:24:39.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.089+0000 7faa35ffb640 1 -- 172.21.15.125:0/207528982 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faa40069a60 con 0x7faa3804e2f0 2023-10-18T10:24:39.095 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.096 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.091+0000 7f25477ee640 1 -- 172.21.15.125:0/1415055703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400acd60 msgr2=0x7f254015e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.091+0000 7f25477ee640 1 --2- 172.21.15.125:0/1415055703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400acd60 0x7f254015e960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f253c009860 tx=0x7f253c03ccf0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.091+0000 7f25477ee640 1 -- 172.21.15.125:0/1415055703 shutdown_connections 2023-10-18T10:24:39.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.091+0000 7f25477ee640 1 --2- 172.21.15.125:0/1415055703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400acd60 0x7f254015e960 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.091+0000 7f25477ee640 1 -- 172.21.15.125:0/1415055703 >> 172.21.15.125:0/1415055703 conn(0x7f25400ca480 msgr2=0x7f25400ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.091+0000 7f25477ee640 1 -- 172.21.15.125:0/1415055703 shutdown_connections 2023-10-18T10:24:39.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.091+0000 7f25477ee640 1 -- 172.21.15.125:0/1415055703 wait complete. 2023-10-18T10:24:39.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.092+0000 7f89a659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a806f5b0 0x7f89a8205f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.092+0000 7f89a659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a806f5b0 0x7f89a8205f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35984/0 (socket says 172.21.15.125:35984) 2023-10-18T10:24:39.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.092+0000 7f89a659c640 1 -- 172.21.15.125:0/2057796327 learned_addr learned my addr 172.21.15.125:0/2057796327 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:39.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.092+0000 7f30477fe640 1 -- 172.21.15.125:0/1444374969 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v12)=0 v12) v1 ==== 116+0+0 (secure 0 0 0) 0x7f305400f820 con 0x7f306015c7b0 2023-10-18T10:24:39.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f89a5d9b640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89a80abdf0 0x7f89a81591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f89a6d9d640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89a815f700 0x7f89a81596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa380d2fd0 con 0x7faa3804e2f0 2023-10-18T10:24:39.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f89a5d9b640 1 -- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89a815f700 msgr2=0x7f89a81596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f89a5d9b640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89a815f700 0x7f89a81596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f89a5d9b640 1 -- 172.21.15.125:0/2057796327 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a806f5b0 msgr2=0x7f89a8205f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f89a5d9b640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a806f5b0 0x7f89a8205f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f89a5d9b640 1 -- 172.21.15.125:0/2057796327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c009490 con 0x7f89a80abdf0 2023-10-18T10:24:39.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f89a5d9b640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89a80abdf0 0x7f89a81591a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f89a000c890 tx=0x7f89a000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:39.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f25477ee640 1 Processor -- start 2023-10-18T10:24:39.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.093+0000 7f25477ee640 1 -- start start 2023-10-18T10:24:39.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.094+0000 7f30457fa640 1 -- 172.21.15.125:0/1444374969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015c7b0 msgr2=0x7f3060156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.094+0000 7f30457fa640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015c7b0 0x7f3060156370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f305400ca80 tx=0x7f305400cf40 comp rx=0 tx=0).stop 2023-10-18T10:24:39.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.094+0000 7f30457fa640 1 -- 172.21.15.125:0/1444374969 shutdown_connections 2023-10-18T10:24:39.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.094+0000 7f30457fa640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30600ab030 0x7f30600a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.094+0000 7f30457fa640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cb610 0x7f30600cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.094+0000 7f30457fa640 1 --2- 172.21.15.125:0/1444374969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015c7b0 0x7f3060156370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.094+0000 7f30457fa640 1 -- 172.21.15.125:0/1444374969 >> 172.21.15.125:0/1444374969 conn(0x7f30600ae3f0 msgr2=0x7f306015de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.098+0000 7f30457fa640 1 -- 172.21.15.125:0/1444374969 shutdown_connections 2023-10-18T10:24:39.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.098+0000 7f30457fa640 1 -- 172.21.15.125:0/1444374969 wait complete. 2023-10-18T10:24:39.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.098+0000 7f25477ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25400acd60 0x7f25401559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7faa35ffb640 1 -- 172.21.15.125:0/207528982 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7faa380d2fd0 con 0x7faa3804e2f0 2023-10-18T10:24:39.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f8996ffd640 1 -- 172.21.15.125:0/2057796327 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0003da0 con 0x7f89a80abdf0 2023-10-18T10:24:39.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f8996ffd640 1 -- 172.21.15.125:0/2057796327 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f89a00050f0 con 0x7f89a80abdf0 2023-10-18T10:24:39.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f8996ffd640 1 -- 172.21.15.125:0/2057796327 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0005500 con 0x7f89a80abdf0 2023-10-18T10:24:39.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.103+0000 7f89ac99e640 1 -- 172.21.15.125:0/2057796327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a820a560 con 0x7f89a80abdf0 2023-10-18T10:24:39.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.103+0000 7f89ac99e640 1 -- 172.21.15.125:0/2057796327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a8088cb0 con 0x7f89a80abdf0 2023-10-18T10:24:39.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f25477ee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400ad140 0x7f254015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f25477ee640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f254015eea0 0x7f254014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f254558a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25400acd60 0x7f25401559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f254558a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25400acd60 0x7f25401559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:46020/0 (socket says 172.21.15.125:46020) 2023-10-18T10:24:39.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f2544d89640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400ad140 0x7f254015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f2544d89640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400ad140 0x7f254015dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:58662/0 (socket says 172.21.15.125:58662) 2023-10-18T10:24:39.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f2545d8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f254015eea0 0x7f254014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f2545d8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f254015eea0 0x7f254014f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36000/0 (socket says 172.21.15.125:36000) 2023-10-18T10:24:39.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f254558a640 1 -- 172.21.15.125:0/4033024982 learned_addr learned my addr 172.21.15.125:0/4033024982 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:39.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25401ed590 con 0x7f254015eea0 2023-10-18T10:24:39.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2540155f20 con 0x7f25400acd60 2023-10-18T10:24:39.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.099+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25401f1010 con 0x7f25400ad140 2023-10-18T10:24:39.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.100+0000 7f2545d8b640 1 -- 172.21.15.125:0/4033024982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400ad140 msgr2=0x7f254015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.100+0000 7f2545d8b640 1 --2- 172.21.15.125:0/4033024982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400ad140 0x7f254015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.100+0000 7f2545d8b640 1 -- 172.21.15.125:0/4033024982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25400acd60 msgr2=0x7f25401559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.100+0000 7f2545d8b640 1 --2- 172.21.15.125:0/4033024982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25400acd60 0x7f25401559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.100+0000 7f2545d8b640 1 -- 172.21.15.125:0/4033024982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c009490 con 0x7f254015eea0 2023-10-18T10:24:39.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.100+0000 7f2545d8b640 1 --2- 172.21.15.125:0/4033024982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f254015eea0 0x7f254014f120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f253800cbe0 tx=0x7f253800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:39.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.102+0000 7f2535ffb640 1 -- 172.21.15.125:0/4033024982 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800ab40 con 0x7f254015eea0 2023-10-18T10:24:39.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.102+0000 7f2535ffb640 1 -- 172.21.15.125:0/4033024982 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f253800ad90 con 0x7f254015eea0 2023-10-18T10:24:39.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.102+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25401f12f0 con 0x7f254015eea0 2023-10-18T10:24:39.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.102+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25401f18b0 con 0x7f254015eea0 2023-10-18T10:24:39.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.102+0000 7f2535ffb640 1 -- 172.21.15.125:0/4033024982 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538003bd0 con 0x7f254015eea0 2023-10-18T10:24:39.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.103+0000 7f2535ffb640 1 -- 172.21.15.125:0/4033024982 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2538004330 con 0x7f254015eea0 2023-10-18T10:24:39.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.103+0000 7f2535ffb640 1 -- 172.21.15.125:0/4033024982 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2538012c10 con 0x7f254015eea0 2023-10-18T10:24:39.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.110+0000 7f8996ffd640 1 -- 172.21.15.125:0/2057796327 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f89a000a640 con 0x7f89a80abdf0 2023-10-18T10:24:39.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.110+0000 7f8996ffd640 1 -- 172.21.15.125:0/2057796327 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f89a001cb20 con 0x7f89a80abdf0 2023-10-18T10:24:39.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.110+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2520005250 con 0x7f254015eea0 2023-10-18T10:24:39.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.114+0000 7f2535ffb640 1 -- 172.21.15.125:0/4033024982 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2538012420 con 0x7f254015eea0 2023-10-18T10:24:39.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.111+0000 7f8994ff9640 1 -- 172.21.15.125:0/2057796327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a80690e0 con 0x7f89a80abdf0 2023-10-18T10:24:39.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.120+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f6b500055a0 con 0x7f6b680e0690 2023-10-18T10:24:39.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.120+0000 7f8996ffd640 1 -- 172.21.15.125:0/2057796327 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f89a0021060 con 0x7f89a80abdf0 2023-10-18T10:24:39.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.125+0000 7f6b65ffb640 1 -- 172.21.15.125:0/2098834788 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v13)=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6b70072a60 con 0x7f6b680e0690 2023-10-18T10:24:39.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e0690 msgr2=0x7f6b68034bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e0690 0x7f6b68034bf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b700771b0 tx=0x7f6b70069570 comp rx=0 tx=0).stop 2023-10-18T10:24:39.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 shutdown_connections 2023-10-18T10:24:39.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b680e0690 0x7f6b68034bf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b680e40d0 0x7f6b680ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 --2- 172.21.15.125:0/2098834788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680dfec0 0x7f6b680e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 >> 172.21.15.125:0/2098834788 conn(0x7f6b6801ef10 msgr2=0x7f6b680df070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 shutdown_connections 2023-10-18T10:24:39.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.131+0000 7f6b7601a640 1 -- 172.21.15.125:0/2098834788 wait complete. 2023-10-18T10:24:39.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.142+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7ffa30031b60 con 0x7ffa3019f2c0 2023-10-18T10:24:39.147 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.147+0000 7ffa34ff9640 1 -- 172.21.15.125:0/2814458480 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v14)=0 v14) v1 ==== 119+0+0 (secure 0 0 0) 0x7ffa3806ac30 con 0x7ffa3019f2c0 2023-10-18T10:24:39.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3019f2c0 msgr2=0x7ffa301989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 --2- 172.21.15.125:0/2814458480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3019f2c0 0x7ffa301989f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000b60 tx=0x7ffa380694b0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 shutdown_connections 2023-10-18T10:24:39.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 --2- 172.21.15.125:0/2814458480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa3019f2c0 0x7ffa301989f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 --2- 172.21.15.125:0/2814458480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa3004f250 0x7ffa300deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 --2- 172.21.15.125:0/2814458480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa300073a0 0x7ffa3002fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 >> 172.21.15.125:0/2814458480 conn(0x7ffa30009f70 msgr2=0x7ffa30004bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 shutdown_connections 2023-10-18T10:24:39.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.152+0000 7ffa3ea26640 1 -- 172.21.15.125:0/2814458480 wait complete. 2023-10-18T10:24:39.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.156+0000 7fec46ffd640 1 -- 172.21.15.125:0/1116456385 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7fec400050c0 con 0x7fec60162b50 2023-10-18T10:24:39.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.162+0000 7fec5d7fa640 1 -- 172.21.15.125:0/1116456385 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v15)=0 v15) v1 ==== 115+0+0 (secure 0 0 0) 0x7fec58018070 con 0x7fec60162b50 2023-10-18T10:24:39.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.166+0000 7fec46ffd640 1 -- 172.21.15.125:0/1116456385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec60162b50 msgr2=0x7fec60201a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.166+0000 7fec46ffd640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec60162b50 0x7fec60201a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec600b1ed0 tx=0x7fec5800cf80 comp rx=0 tx=0).stop 2023-10-18T10:24:39.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.167+0000 7fec46ffd640 1 -- 172.21.15.125:0/1116456385 shutdown_connections 2023-10-18T10:24:39.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.167+0000 7fec46ffd640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec60162b50 0x7fec60201a30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.167+0000 7fec46ffd640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec6015ebf0 0x7fec60162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.167+0000 7fec46ffd640 1 --2- 172.21.15.125:0/1116456385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec60157b80 0x7fec60164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.167+0000 7fec46ffd640 1 -- 172.21.15.125:0/1116456385 >> 172.21.15.125:0/1116456385 conn(0x7fec600ae720 msgr2=0x7fec600b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.171+0000 7fec46ffd640 1 -- 172.21.15.125:0/1116456385 shutdown_connections 2023-10-18T10:24:39.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.171+0000 7fec46ffd640 1 -- 172.21.15.125:0/1116456385 wait complete. 2023-10-18T10:24:39.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.178+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7f20480057d0 con 0x7f206806f920 2023-10-18T10:24:39.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.184+0000 7f204ffff640 1 -- 172.21.15.125:0/2974615588 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f205802b0c0 con 0x7f206806f920 2023-10-18T10:24:39.187 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.185+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f206806f920 msgr2=0x7f206816a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.185+0000 7f206df73640 1 --2- 172.21.15.125:0/2974615588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f206806f920 0x7f206816a480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f205800c430 tx=0x7f205800c8f0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.186+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 shutdown_connections 2023-10-18T10:24:39.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.186+0000 7f206df73640 1 --2- 172.21.15.125:0/2974615588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20680882e0 0x7f20680ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.186+0000 7f206df73640 1 --2- 172.21.15.125:0/2974615588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20680b02a0 0x7f20680acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.186+0000 7f206df73640 1 --2- 172.21.15.125:0/2974615588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f206806f920 0x7f206816a480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.186+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 >> 172.21.15.125:0/2974615588 conn(0x7f20680ae720 msgr2=0x7f20680c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.187+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 shutdown_connections 2023-10-18T10:24:39.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.187+0000 7f206df73640 1 -- 172.21.15.125:0/2974615588 wait complete. 2023-10-18T10:24:39.205 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.213+0000 7f4cd7fff640 1 -- 172.21.15.125:0/2340598859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f4ce4005360 con 0x7f4ce40df1f0 2023-10-18T10:24:39.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.219+0000 7f4ce27fc640 1 -- 172.21.15.125:0/2340598859 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v17)=0 v17) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4cec0ad440 con 0x7f4ce40df1f0 2023-10-18T10:24:39.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.220+0000 7f4cd7fff640 1 -- 172.21.15.125:0/2340598859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce40df1f0 msgr2=0x7f4ce4181420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.220+0000 7f4cd7fff640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce40df1f0 0x7f4ce4181420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4cec067d70 tx=0x7f4cec0ade20 comp rx=0 tx=0).stop 2023-10-18T10:24:39.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.221+0000 7f4cd7fff640 1 -- 172.21.15.125:0/2340598859 shutdown_connections 2023-10-18T10:24:39.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.221+0000 7f4cd7fff640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ce40df1f0 0x7f4ce4181420 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.221+0000 7f4cd7fff640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ce4182030 0x7f4ce4181960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.221+0000 7f4cd7fff640 1 --2- 172.21.15.125:0/2340598859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ce4053c40 0x7f4ce40e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.221+0000 7f4cd7fff640 1 -- 172.21.15.125:0/2340598859 >> 172.21.15.125:0/2340598859 conn(0x7f4ce4009f80 msgr2=0x7f4ce40dd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.221+0000 7f4cd7fff640 1 -- 172.21.15.125:0/2340598859 shutdown_connections 2023-10-18T10:24:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.221+0000 7f4cd7fff640 1 -- 172.21.15.125:0/2340598859 wait complete. 2023-10-18T10:24:39.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.237+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f94680050c0 con 0x7f94880b09b0 2023-10-18T10:24:39.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.241+0000 7f946f7fe640 1 -- 172.21.15.125:0/2666595352 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v18)=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9480005800 con 0x7f94880b09b0 2023-10-18T10:24:39.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.242+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7faa38005350 con 0x7faa3804e2f0 2023-10-18T10:24:39.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94880b09b0 msgr2=0x7f9488178130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 --2- 172.21.15.125:0/2666595352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94880b09b0 0x7f9488178130 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f948000cb20 tx=0x7f9480002130 comp rx=0 tx=0).stop 2023-10-18T10:24:39.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 shutdown_connections 2023-10-18T10:24:39.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 --2- 172.21.15.125:0/2666595352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9488178d00 0x7f9488178670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 --2- 172.21.15.125:0/2666595352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94880b09b0 0x7f9488178130 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f7fa40050c0 con 0x7f7fc406ce60 2023-10-18T10:24:39.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 --2- 172.21.15.125:0/2666595352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94880882e0 0x7f9488177bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 >> 172.21.15.125:0/2666595352 conn(0x7f94880ae720 msgr2=0x7f94881534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 shutdown_connections 2023-10-18T10:24:39.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.243+0000 7f948d3c5640 1 -- 172.21.15.125:0/2666595352 wait complete. 2023-10-18T10:24:39.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.248+0000 7faa35ffb640 1 -- 172.21.15.125:0/207528982 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v19)=0 v19) v1 ==== 120+0+0 (secure 0 0 0) 0x7faa40097060 con 0x7faa3804e2f0 2023-10-18T10:24:39.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa3804e2f0 msgr2=0x7faa380dde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa3804e2f0 0x7faa380dde00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa400682a0 tx=0x7faa400694b0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 shutdown_connections 2023-10-18T10:24:39.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa3804e2f0 0x7faa380dde00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa38186030 0x7faa380355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 --2- 172.21.15.125:0/207528982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa2c005180 0x7faa380d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 >> 172.21.15.125:0/207528982 conn(0x7faa38009f70 msgr2=0x7faa380d7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 shutdown_connections 2023-10-18T10:24:39.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.249+0000 7faa47890640 1 -- 172.21.15.125:0/207528982 wait complete. 2023-10-18T10:24:39.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.253+0000 7f7fb9ffb640 1 -- 172.21.15.125:0/3270117166 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v20)=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7fc0017070 con 0x7f7fc406ce60 2023-10-18T10:24:39.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.255+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc406ce60 msgr2=0x7f7fc4147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.255+0000 7f7fca2da640 1 --2- 172.21.15.125:0/3270117166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc406ce60 0x7f7fc4147880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d9d0 tx=0x7f7fc000de90 comp rx=0 tx=0).stop 2023-10-18T10:24:39.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.256+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 shutdown_connections 2023-10-18T10:24:39.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.256+0000 7f7fca2da640 1 --2- 172.21.15.125:0/3270117166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fc41578c0 0x7f7fc413dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.256+0000 7f7fca2da640 1 --2- 172.21.15.125:0/3270117166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fc413e2a0 0x7f7fc40cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.256+0000 7f7fca2da640 1 --2- 172.21.15.125:0/3270117166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fc406ce60 0x7f7fc4147880 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.256+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 >> 172.21.15.125:0/3270117166 conn(0x7f7fc40ae460 msgr2=0x7f7fc406ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.256+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 shutdown_connections 2023-10-18T10:24:39.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.256+0000 7f7fca2da640 1 -- 172.21.15.125:0/3270117166 wait complete. 2023-10-18T10:24:39.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.257+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f2520005870 con 0x7f254015eea0 2023-10-18T10:24:39.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.257+0000 7f8994ff9640 1 -- 172.21.15.125:0/2057796327 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f89a80cb050 con 0x7f89a80abdf0 2023-10-18T10:24:39.261 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.261 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.261+0000 7f2535ffb640 1 -- 172.21.15.125:0/4033024982 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f25380047d0 con 0x7f254015eea0 2023-10-18T10:24:39.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f254015eea0 msgr2=0x7f254014f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 --2- 172.21.15.125:0/4033024982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f254015eea0 0x7f254014f120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f253800cbe0 tx=0x7f253800a430 comp rx=0 tx=0).stop 2023-10-18T10:24:39.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 shutdown_connections 2023-10-18T10:24:39.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 --2- 172.21.15.125:0/4033024982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f254015eea0 0x7f254014f120 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 --2- 172.21.15.125:0/4033024982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25400ad140 0x7f254015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 --2- 172.21.15.125:0/4033024982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25400acd60 0x7f25401559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 >> 172.21.15.125:0/4033024982 conn(0x7f25400ca480 msgr2=0x7f25400c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 shutdown_connections 2023-10-18T10:24:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.262+0000 7f25477ee640 1 -- 172.21.15.125:0/4033024982 wait complete. 2023-10-18T10:24:39.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.267+0000 7f8996ffd640 1 -- 172.21.15.125:0/2057796327 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v22)=0 v22) v1 ==== 121+0+0 (secure 0 0 0) 0x7f89a0017070 con 0x7f89a80abdf0 2023-10-18T10:24:39.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 -- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89a80abdf0 msgr2=0x7f89a81591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89a80abdf0 0x7f89a81591a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f89a000c890 tx=0x7f89a000cd50 comp rx=0 tx=0).stop 2023-10-18T10:24:39.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 -- 172.21.15.125:0/2057796327 shutdown_connections 2023-10-18T10:24:39.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89a80abdf0 0x7f89a81591a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89a815f700 0x7f89a81596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 --2- 172.21.15.125:0/2057796327 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89a806f5b0 0x7f89a8205f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 -- 172.21.15.125:0/2057796327 >> 172.21.15.125:0/2057796327 conn(0x7f89a80ae720 msgr2=0x7f89a806fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 -- 172.21.15.125:0/2057796327 shutdown_connections 2023-10-18T10:24:39.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.268+0000 7f8994ff9640 1 -- 172.21.15.125:0/2057796327 wait complete. 2023-10-18T10:24:39.272 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.273 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.273 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.281 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi125') 2023-10-18T10:24:39.281 INFO:tasks.ceph:Setting crush tunables to default 2023-10-18T10:24:39.282 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd crush tunables default 2023-10-18T10:24:39.305 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:39.304+0000 7f022840d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:24:39.359 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:39.358+0000 7f022840d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:24:39.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.407+0000 7f11a2333640 1 -- 172.21.15.125:0/4076945837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f119c0cb960 msgr2=0x7f119c0701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.407+0000 7f11a2333640 1 --2- 172.21.15.125:0/4076945837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f119c0cb960 0x7f119c0701b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f118c009830 tx=0x7f118c02bd60 comp rx=0 tx=0).stop 2023-10-18T10:24:39.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.407+0000 7f11a2333640 1 -- 172.21.15.125:0/4076945837 shutdown_connections 2023-10-18T10:24:39.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.407+0000 7f11a2333640 1 --2- 172.21.15.125:0/4076945837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f119c0cb960 0x7f119c0701b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.407+0000 7f11a2333640 1 --2- 172.21.15.125:0/4076945837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f119c141460 0x7f119c158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.407+0000 7f11a2333640 1 -- 172.21.15.125:0/4076945837 >> 172.21.15.125:0/4076945837 conn(0x7f119c0af810 msgr2=0x7f119c0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.407+0000 7f11a2333640 1 -- 172.21.15.125:0/4076945837 shutdown_connections 2023-10-18T10:24:39.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.407+0000 7f11a2333640 1 -- 172.21.15.125:0/4076945837 wait complete. 2023-10-18T10:24:39.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.408+0000 7f11a2333640 1 Processor -- start 2023-10-18T10:24:39.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.408+0000 7f11a2333640 1 -- start start 2023-10-18T10:24:39.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a2333640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f119c0706f0 0x7f119c0a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a2333640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f119c0cb960 0x7f119c139690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a2333640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f119c141460 0x7f119c0acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:39.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a2333640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0ad1c0 con 0x7f119c141460 2023-10-18T10:24:39.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a2333640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0a6650 con 0x7f119c0cb960 2023-10-18T10:24:39.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a2333640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0a6930 con 0x7f119c0706f0 2023-10-18T10:24:39.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f119c141460 0x7f119c0acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f119c141460 0x7f119c0acc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36016/0 (socket says 172.21.15.125:36016) 2023-10-18T10:24:39.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 -- 172.21.15.125:0/915742599 learned_addr learned my addr 172.21.15.125:0/915742599 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:39.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f119b7fe640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f119c0cb960 0x7f119c139690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f119bfff640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f119c0706f0 0x7f119c0a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:39.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 -- 172.21.15.125:0/915742599 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f119c0706f0 msgr2=0x7f119c0a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f119c0706f0 0x7f119c0a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 -- 172.21.15.125:0/915742599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f119c0cb960 msgr2=0x7f119c139690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f119c0cb960 0x7f119c139690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 -- 172.21.15.125:0/915742599 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190009490 con 0x7f119c141460 2023-10-18T10:24:39.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.409+0000 7f11a08d0640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f119c141460 0x7f119c0acc80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f119400cbc0 tx=0x7f119400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:39.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.410+0000 7f1198ff9640 1 -- 172.21.15.125:0/915742599 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400ace0 con 0x7f119c141460 2023-10-18T10:24:39.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.410+0000 7f1198ff9640 1 -- 172.21.15.125:0/915742599 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1194012420 con 0x7f119c141460 2023-10-18T10:24:39.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.410+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c009490 con 0x7f119c141460 2023-10-18T10:24:39.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.410+0000 7f1198ff9640 1 -- 172.21.15.125:0/915742599 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11940127f0 con 0x7f119c141460 2023-10-18T10:24:39.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.410+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c0a6ef0 con 0x7f119c141460 2023-10-18T10:24:39.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.410+0000 7f1198ff9640 1 -- 172.21.15.125:0/915742599 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1194012c50 con 0x7f119c141460 2023-10-18T10:24:39.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.410+0000 7f1198ff9640 1 -- 172.21.15.125:0/915742599 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f119401bde0 con 0x7f119c141460 2023-10-18T10:24:39.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.411+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c0675f0 con 0x7f119c141460 2023-10-18T10:24:39.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.412+0000 7f1198ff9640 1 -- 172.21.15.125:0/915742599 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f119403ab40 con 0x7f119c141460 2023-10-18T10:24:39.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.477+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f119c140a50 con 0x7f119c141460 2023-10-18T10:24:39.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.478+0000 7f1198ff9640 1 -- 172.21.15.125:0/915742599 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1194029030 con 0x7f119c141460 2023-10-18T10:24:39.480 INFO:teuthology.orchestra.run.smithi125.stderr:adjusted tunables profile to default 2023-10-18T10:24:39.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f119c141460 msgr2=0x7f119c0acc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:39.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f119c141460 0x7f119c0acc80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f119400cbc0 tx=0x7f119400a430 comp rx=0 tx=0).stop 2023-10-18T10:24:39.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 shutdown_connections 2023-10-18T10:24:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f119c0cb960 0x7f119c139690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f119c141460 0x7f119c0acc80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 --2- 172.21.15.125:0/915742599 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f119c0706f0 0x7f119c0a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 >> 172.21.15.125:0/915742599 conn(0x7f119c0af810 msgr2=0x7f119c1595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:39.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 shutdown_connections 2023-10-18T10:24:39.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:39.479+0000 7f11a2333640 1 -- 172.21.15.125:0/915742599 wait complete. 2023-10-18T10:24:39.491 INFO:tasks.ceph:check_enable_crimson: False 2023-10-18T10:24:39.491 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-18T10:24:39.492 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:39.492 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-18T10:24:39.493 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.491+0000 7f304f80f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:24:39.519 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:39.520 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-18T10:24:39.545 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.543+0000 7f304f80f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:24:39.585 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:39.586 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-18T10:24:39.589 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.587+0000 7f304f80f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:24:39.642 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.640+0000 7f304f80f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:24:39.652 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:24:39.652 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-18T10:24:39.683 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.682+0000 7f304f80f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:24:39.718 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:39.719 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-18T10:24:39.745 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:39.745 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-18T10:24:39.786 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.784+0000 7f304f80f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:24:39.812 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:39.812 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-18T10:24:39.858 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.857+0000 7f304f80f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:24:39.879 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:24:39.880 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-18T10:24:39.905 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:39.904+0000 7f022840d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:24:39.906 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.905+0000 7f304f80f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:24:39.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 96f5dda2-779b-493a-a915-3c2243c90fea 0 2023-10-18T10:24:39.956 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:39.954+0000 7f022840d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:24:39.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:39.959+0000 7f304f80f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:24:40.000 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:39.998+0000 7f022840d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:24:40.009 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:40.008+0000 7f304f80f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:24:40.052 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.050+0000 7f022840d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:24:40.063 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.062+0000 7f72b4094640 1 -- 172.21.15.139:0/2202431034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac0cdb30 msgr2=0x7f72ac0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.064 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.062+0000 7f72b4094640 1 --2- 172.21.15.139:0/2202431034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac0cdb30 0x7f72ac0cdfb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f729c005fd0 tx=0x7f729c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:24:40.064 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.062+0000 7f72b4094640 1 -- 172.21.15.139:0/2202431034 shutdown_connections 2023-10-18T10:24:40.064 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.062+0000 7f72b4094640 1 --2- 172.21.15.139:0/2202431034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac0cdb30 0x7f72ac0cdfb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.064 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.062+0000 7f72b4094640 1 --2- 172.21.15.139:0/2202431034 >> v2:172.21.15.125:3300/0 conn(0x7f72ac159060 0x7f72ac06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.064 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.062+0000 7f72b4094640 1 -- 172.21.15.139:0/2202431034 >> 172.21.15.139:0/2202431034 conn(0x7f72ac0b0670 msgr2=0x7f72ac0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.065 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.062+0000 7f72b4094640 1 -- 172.21.15.139:0/2202431034 shutdown_connections 2023-10-18T10:24:40.065 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.062+0000 7f72b4094640 1 -- 172.21.15.139:0/2202431034 wait complete. 2023-10-18T10:24:40.067 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.063+0000 7f72b4094640 1 Processor -- start 2023-10-18T10:24:40.068 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.063+0000 7f72b4094640 1 -- start start 2023-10-18T10:24:40.068 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.063+0000 7f72b4094640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72ac0cdb30 0x7f72ac155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.068 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.063+0000 7f72b4094640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ac159060 0x7f72ac154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.068 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.063+0000 7f72b4094640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac15ccf0 0x7f72ac0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.068 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.063+0000 7f72b4094640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0c6dc0 con 0x7f72ac0cdb30 2023-10-18T10:24:40.068 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.063+0000 7f72b4094640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0c6140 con 0x7f72ac159060 2023-10-18T10:24:40.069 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.063+0000 7f72b4094640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0c6420 con 0x7f72ac15ccf0 2023-10-18T10:24:40.069 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ac159060 0x7f72ac154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.069 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b2631640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac15ccf0 0x7f72ac0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.069 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ac159060 0x7f72ac154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36280/0 (socket says 172.21.15.139:36280) 2023-10-18T10:24:40.069 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 -- 172.21.15.139:0/3701897483 learned_addr learned my addr 172.21.15.139:0/3701897483 (peer_addr_for_me v2:172.21.15.139:0/0) 2023-10-18T10:24:40.070 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b1e30640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72ac0cdb30 0x7f72ac155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.070 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 -- 172.21.15.139:0/3701897483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac15ccf0 msgr2=0x7f72ac0c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.070 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac15ccf0 0x7f72ac0c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.070 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 -- 172.21.15.139:0/3701897483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72ac0cdb30 msgr2=0x7f72ac155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.070 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72ac0cdb30 0x7f72ac155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.071 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 -- 172.21.15.139:0/3701897483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c009560 con 0x7f72ac159060 2023-10-18T10:24:40.071 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b2631640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac15ccf0 0x7f72ac0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:40.071 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b162f640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ac159060 0x7f72ac154df0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f729c006010 tx=0x7f729c02d9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:40.071 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72a27fc640 1 -- 172.21.15.139:0/3701897483 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c02ddf0 con 0x7f72ac159060 2023-10-18T10:24:40.071 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72a27fc640 1 -- 172.21.15.139:0/3701897483 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f729c0036e0 con 0x7f72ac159060 2023-10-18T10:24:40.072 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ac0c6700 con 0x7f72ac159060 2023-10-18T10:24:40.072 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72a27fc640 1 -- 172.21.15.139:0/3701897483 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c003af0 con 0x7f72ac159060 2023-10-18T10:24:40.072 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac155330 con 0x7f72ac159060 2023-10-18T10:24:40.072 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72a27fc640 1 -- 172.21.15.139:0/3701897483 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f729c03e450 con 0x7f72ac159060 2023-10-18T10:24:40.072 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.064+0000 7f72a27fc640 1 -- 172.21.15.139:0/3701897483 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f729c034440 con 0x7f72ac159060 2023-10-18T10:24:40.073 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.065+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac0c6f00 con 0x7f72ac159060 2023-10-18T10:24:40.073 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.067+0000 7f72a27fc640 1 -- 172.21.15.139:0/3701897483 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f72ac0c6f00 con 0x7f72ac159060 2023-10-18T10:24:40.093 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.092+0000 7f022840d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:24:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.139+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "96f5dda2-779b-493a-a915-3c2243c90fea", "id": 0} v 0) v1 -- 0x7f72ac0cc4d0 con 0x7f72ac159060 2023-10-18T10:24:40.143 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.142+0000 7f72a27fc640 1 -- 172.21.15.139:0/3701897483 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "96f5dda2-779b-493a-a915-3c2243c90fea", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f729c043020 con 0x7f72ac159060 2023-10-18T10:24:40.144 INFO:teuthology.orchestra.run.smithi139.stdout:0 2023-10-18T10:24:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ac159060 msgr2=0x7f72ac154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ac159060 0x7f72ac154df0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f729c006010 tx=0x7f729c02d9f0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 shutdown_connections 2023-10-18T10:24:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72ac159060 0x7f72ac154df0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72ac15ccf0 0x7f72ac0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 --2- 172.21.15.139:0/3701897483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72ac0cdb30 0x7f72ac155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 >> 172.21.15.139:0/3701897483 conn(0x7f72ac0b0670 msgr2=0x7f72ac0b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 shutdown_connections 2023-10-18T10:24:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.143+0000 7f72b4094640 1 -- 172.21.15.139:0/3701897483 wait complete. 2023-10-18T10:24:40.154 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new e2cb000b-290a-481f-a916-397d23e8561b 1 2023-10-18T10:24:40.192 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.191+0000 7f022840d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:24:40.258 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.256+0000 7fa34f305640 1 -- 172.21.15.139:0/1478453272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 msgr2=0x7fa3480704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.258 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.256+0000 7fa34f305640 1 --2- 172.21.15.139:0/1478453272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 0x7fa3480704e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa3480b06d0 tx=0x7fa33802bd70 comp rx=0 tx=0).stop 2023-10-18T10:24:40.258 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.256+0000 7fa34f305640 1 -- 172.21.15.139:0/1478453272 shutdown_connections 2023-10-18T10:24:40.258 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.256+0000 7fa34f305640 1 --2- 172.21.15.139:0/1478453272 >> v2:172.21.15.125:3300/0 conn(0x7fa3480cc200 0x7fa3480cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.259 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.256+0000 7fa34f305640 1 --2- 172.21.15.139:0/1478453272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 0x7fa3480704e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.259 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.256+0000 7fa34f305640 1 -- 172.21.15.139:0/1478453272 >> 172.21.15.139:0/1478453272 conn(0x7fa3480af730 msgr2=0x7fa3480afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.259 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.256+0000 7fa34f305640 1 -- 172.21.15.139:0/1478453272 shutdown_connections 2023-10-18T10:24:40.259 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.256+0000 7fa34f305640 1 -- 172.21.15.139:0/1478453272 wait complete. 2023-10-18T10:24:40.262 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.257+0000 7fa34f305640 1 Processor -- start 2023-10-18T10:24:40.262 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.257+0000 7fa34f305640 1 -- start start 2023-10-18T10:24:40.262 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34f305640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 0x7fa348134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.262 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34f305640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3480cc200 0x7fa3481520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.262 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34f305640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3481416f0 0x7fa348157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.263 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34f305640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa348150e00 con 0x7fa3481416f0 2023-10-18T10:24:40.263 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34f305640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481510e0 con 0x7fa348070080 2023-10-18T10:24:40.263 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34f305640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481513c0 con 0x7fa3480cc200 2023-10-18T10:24:40.263 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34d0a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 0x7fa348134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.263 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34c8a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3480cc200 0x7fa3481520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.264 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34d0a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 0x7fa348134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36304/0 (socket says 172.21.15.139:36304) 2023-10-18T10:24:40.264 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34d0a1640 1 -- 172.21.15.139:0/1456427792 learned_addr learned my addr 172.21.15.139:0/1456427792 (peer_addr_for_me v2:172.21.15.139:0/0) 2023-10-18T10:24:40.264 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34d8a2640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3481416f0 0x7fa348157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.264 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34c8a0640 1 -- 172.21.15.139:0/1456427792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 msgr2=0x7fa348134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.264 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34c8a0640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 0x7fa348134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.265 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34c8a0640 1 -- 172.21.15.139:0/1456427792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3481416f0 msgr2=0x7fa348157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.265 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34c8a0640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3481416f0 0x7fa348157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.265 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34c8a0640 1 -- 172.21.15.139:0/1456427792 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa338009490 con 0x7fa3480cc200 2023-10-18T10:24:40.265 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34d8a2640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3481416f0 0x7fa348157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:40.265 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34d0a1640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 0x7fa348134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:40.266 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34c8a0640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3480cc200 0x7fa3481520c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa338033000 tx=0x7fa338004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:40.266 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa33dffb640 1 -- 172.21.15.139:0/1456427792 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338003ba0 con 0x7fa3480cc200 2023-10-18T10:24:40.266 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa33dffb640 1 -- 172.21.15.139:0/1456427792 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa338041070 con 0x7fa3480cc200 2023-10-18T10:24:40.266 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3480b1cd0 con 0x7fa3480cc200 2023-10-18T10:24:40.266 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.258+0000 7fa33dffb640 1 -- 172.21.15.139:0/1456427792 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338005020 con 0x7fa3480cc200 2023-10-18T10:24:40.267 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.259+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34813f140 con 0x7fa3480cc200 2023-10-18T10:24:40.267 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.259+0000 7fa33dffb640 1 -- 172.21.15.139:0/1456427792 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa338043030 con 0x7fa3480cc200 2023-10-18T10:24:40.267 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.259+0000 7fa33dffb640 1 -- 172.21.15.139:0/1456427792 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa338004d80 con 0x7fa3480cc200 2023-10-18T10:24:40.267 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.260+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3481515e0 con 0x7fa3480cc200 2023-10-18T10:24:40.268 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.259+0000 7f022840d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:24:40.268 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.262+0000 7fa33dffb640 1 -- 172.21.15.139:0/1456427792 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa338065b40 con 0x7fa3480cc200 2023-10-18T10:24:40.305 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.304+0000 7f022840d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:24:40.335 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.334+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e2cb000b-290a-481f-a916-397d23e8561b", "id": 1} v 0) v1 -- 0x7fa3480704e0 con 0x7fa3480cc200 2023-10-18T10:24:40.339 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.337+0000 7fa33dffb640 1 -- 172.21.15.139:0/1456427792 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2cb000b-290a-481f-a916-397d23e8561b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa338054020 con 0x7fa3480cc200 2023-10-18T10:24:40.339 INFO:teuthology.orchestra.run.smithi139.stdout:1 2023-10-18T10:24:40.340 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3480cc200 msgr2=0x7fa3481520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.340 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3480cc200 0x7fa3481520c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa338033000 tx=0x7fa338004570 comp rx=0 tx=0).stop 2023-10-18T10:24:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 shutdown_connections 2023-10-18T10:24:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3481416f0 0x7fa348157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3480cc200 0x7fa3481520c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 --2- 172.21.15.139:0/1456427792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa348070080 0x7fa348134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 >> 172.21.15.139:0/1456427792 conn(0x7fa3480af730 msgr2=0x7fa34806c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 shutdown_connections 2023-10-18T10:24:40.342 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.338+0000 7fa34f305640 1 -- 172.21.15.139:0/1456427792 wait complete. 2023-10-18T10:24:40.350 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new df7960de-984d-4b38-95ec-b0ee55f0e268 2 2023-10-18T10:24:40.357 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.355+0000 7f022840d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:24:40.403 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.401+0000 7f022840d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:24:40.409 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:40.407+0000 7f304f80f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:24:40.454 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.452+0000 7f57588d7640 1 -- 172.21.15.139:0/605848128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f575415b2a0 msgr2=0x7f57540d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.455 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.452+0000 7f57588d7640 1 --2- 172.21.15.139:0/605848128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f575415b2a0 0x7f57540d1910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5744009830 tx=0x7f574402bd60 comp rx=0 tx=0).stop 2023-10-18T10:24:40.455 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.452+0000 7f57588d7640 1 -- 172.21.15.139:0/605848128 shutdown_connections 2023-10-18T10:24:40.455 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.452+0000 7f57588d7640 1 --2- 172.21.15.139:0/605848128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f575415b2a0 0x7f57540d1910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.455 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.452+0000 7f57588d7640 1 -- 172.21.15.139:0/605848128 >> 172.21.15.139:0/605848128 conn(0x7f57540d0d60 msgr2=0x7f57540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.455 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.452+0000 7f57588d7640 1 -- 172.21.15.139:0/605848128 shutdown_connections 2023-10-18T10:24:40.456 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.452+0000 7f57588d7640 1 -- 172.21.15.139:0/605848128 wait complete. 2023-10-18T10:24:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.454+0000 7f57588d7640 1 Processor -- start 2023-10-18T10:24:40.458 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.454+0000 7f57588d7640 1 -- start start 2023-10-18T10:24:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.454+0000 7f57588d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57540ceb40 0x7f5754159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.454+0000 7f57588d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57540cf340 0x7f57540c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.454+0000 7f57588d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f575415b2a0 0x7f57540c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.454+0000 7f57588d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57540b50d0 con 0x7f57540ceb40 2023-10-18T10:24:40.459 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.454+0000 7f57588d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57540b4470 con 0x7f575415b2a0 2023-10-18T10:24:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.454+0000 7f57588d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57540b4730 con 0x7f57540cf340 2023-10-18T10:24:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5751d9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57540cf340 0x7f57540c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5752d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f575415b2a0 0x7f57540c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5751d9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57540cf340 0x7f57540c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:60944/0 (socket says 172.21.15.139:60944) 2023-10-18T10:24:40.460 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5752d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f575415b2a0 0x7f57540c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36310/0 (socket says 172.21.15.139:36310) 2023-10-18T10:24:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5751d9b640 1 -- 172.21.15.139:0/15891905 learned_addr learned my addr 172.21.15.139:0/15891905 (peer_addr_for_me v2:172.21.15.139:0/0) 2023-10-18T10:24:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f575259c640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57540ceb40 0x7f5754159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5752d9d640 1 -- 172.21.15.139:0/15891905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57540cf340 msgr2=0x7f57540c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5752d9d640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57540cf340 0x7f57540c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.461 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5752d9d640 1 -- 172.21.15.139:0/15891905 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57540ceb40 msgr2=0x7f5754159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5752d9d640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57540ceb40 0x7f5754159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5752d9d640 1 -- 172.21.15.139:0/15891905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744009490 con 0x7f575415b2a0 2023-10-18T10:24:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5751d9b640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57540cf340 0x7f57540c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5752d9d640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f575415b2a0 0x7f57540c8370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f574c00caa0 tx=0x7f574c00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:40.462 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5742ffd640 1 -- 172.21.15.139:0/15891905 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c00ac00 con 0x7f575415b2a0 2023-10-18T10:24:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5742ffd640 1 -- 172.21.15.139:0/15891905 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f574c00ae50 con 0x7f575415b2a0 2023-10-18T10:24:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57540b4a10 con 0x7f575415b2a0 2023-10-18T10:24:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5742ffd640 1 -- 172.21.15.139:0/15891905 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c012680 con 0x7f575415b2a0 2023-10-18T10:24:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5754159950 con 0x7f575415b2a0 2023-10-18T10:24:40.463 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5742ffd640 1 -- 172.21.15.139:0/15891905 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f574c012ae0 con 0x7f575415b2a0 2023-10-18T10:24:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.455+0000 7f5742ffd640 1 -- 172.21.15.139:0/15891905 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f574c01bcc0 con 0x7f575415b2a0 2023-10-18T10:24:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.456+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57540b5210 con 0x7f575415b2a0 2023-10-18T10:24:40.464 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.458+0000 7f5742ffd640 1 -- 172.21.15.139:0/15891905 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f574c039b40 con 0x7f575415b2a0 2023-10-18T10:24:40.549 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.547+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df7960de-984d-4b38-95ec-b0ee55f0e268", "id": 2} v 0) v1 -- 0x7f57540690e0 con 0x7f575415b2a0 2023-10-18T10:24:40.552 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.551+0000 7f5742ffd640 1 -- 172.21.15.139:0/15891905 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df7960de-984d-4b38-95ec-b0ee55f0e268", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f574c028020 con 0x7f575415b2a0 2023-10-18T10:24:40.552 INFO:teuthology.orchestra.run.smithi139.stdout:2 2023-10-18T10:24:40.553 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.551+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f575415b2a0 msgr2=0x7f57540c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.554 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.551+0000 7f57588d7640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f575415b2a0 0x7f57540c8370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f574c00caa0 tx=0x7f574c00cf60 comp rx=0 tx=0).stop 2023-10-18T10:24:40.554 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.551+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 shutdown_connections 2023-10-18T10:24:40.554 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.551+0000 7f57588d7640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f575415b2a0 0x7f57540c8370 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.554 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.551+0000 7f57588d7640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57540cf340 0x7f57540c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.551+0000 7f57588d7640 1 --2- 172.21.15.139:0/15891905 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57540ceb40 0x7f5754159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.552+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 >> 172.21.15.139:0/15891905 conn(0x7f57540d0d60 msgr2=0x7f57541e5880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.552+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 shutdown_connections 2023-10-18T10:24:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.552+0000 7f57588d7640 1 -- 172.21.15.139:0/15891905 wait complete. 2023-10-18T10:24:40.562 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:40.561+0000 7f304f80f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:24:40.563 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new e37e6670-26f5-4525-9bfe-488052e330f6 3 2023-10-18T10:24:40.667 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.664+0000 7f9989c44640 1 -- 172.21.15.139:0/1413451608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 msgr2=0x7f998406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.667 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.664+0000 7f9989c44640 1 --2- 172.21.15.139:0/1413451608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 0x7f998406d260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f996c0097b0 tx=0x7f996c02ba10 comp rx=0 tx=0).stop 2023-10-18T10:24:40.667 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.665+0000 7f9989c44640 1 -- 172.21.15.139:0/1413451608 shutdown_connections 2023-10-18T10:24:40.667 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.665+0000 7f9989c44640 1 --2- 172.21.15.139:0/1413451608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 0x7f998406d260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.668 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.665+0000 7f9989c44640 1 -- 172.21.15.139:0/1413451608 >> 172.21.15.139:0/1413451608 conn(0x7f99840b0670 msgr2=0x7f99840b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.668 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.665+0000 7f9989c44640 1 -- 172.21.15.139:0/1413451608 shutdown_connections 2023-10-18T10:24:40.668 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.665+0000 7f9989c44640 1 -- 172.21.15.139:0/1413451608 wait complete. 2023-10-18T10:24:40.671 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.666+0000 7f9989c44640 1 Processor -- start 2023-10-18T10:24:40.671 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.666+0000 7f9989c44640 1 -- start start 2023-10-18T10:24:40.671 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.666+0000 7f9989c44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 0x7f998406da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.671 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.666+0000 7f9989c44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9984159060 0x7f99841e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.671 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.666+0000 7f9989c44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f998415ccf0 0x7f99841e3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.666+0000 7f9989c44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99841e3640 con 0x7f99840ce240 2023-10-18T10:24:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.666+0000 7f9989c44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998406a970 con 0x7f9984159060 2023-10-18T10:24:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.666+0000 7f9989c44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99840c4e40 con 0x7f998415ccf0 2023-10-18T10:24:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9984159060 0x7f99841e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9984159060 0x7f99841e2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36316/0 (socket says 172.21.15.139:36316) 2023-10-18T10:24:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9983fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f998415ccf0 0x7f99841e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.673 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 -- 172.21.15.139:0/2392027465 learned_addr learned my addr 172.21.15.139:0/2392027465 (peer_addr_for_me v2:172.21.15.139:0/0) 2023-10-18T10:24:40.673 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f99837fe640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 0x7f998406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.673 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 -- 172.21.15.139:0/2392027465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f998415ccf0 msgr2=0x7f99841e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.673 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f998415ccf0 0x7f99841e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.673 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 -- 172.21.15.139:0/2392027465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 msgr2=0x7f998406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.674 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 0x7f998406da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.674 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 -- 172.21.15.139:0/2392027465 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c009490 con 0x7f9984159060 2023-10-18T10:24:40.674 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9983fff640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f998415ccf0 0x7f99841e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:40.674 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f99837fe640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 0x7f998406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:40.674 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f997bfff640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9984159060 0x7f99841e2af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f996c0098d0 tx=0x7f996c031c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:40.675 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9980ff9640 1 -- 172.21.15.139:0/2392027465 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c03f070 con 0x7f9984159060 2023-10-18T10:24:40.675 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9980ff9640 1 -- 172.21.15.139:0/2392027465 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f996c005020 con 0x7f9984159060 2023-10-18T10:24:40.675 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998406abd0 con 0x7f9984159060 2023-10-18T10:24:40.675 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9980ff9640 1 -- 172.21.15.139:0/2392027465 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c005410 con 0x7f9984159060 2023-10-18T10:24:40.675 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998406b190 con 0x7f9984159060 2023-10-18T10:24:40.676 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9980ff9640 1 -- 172.21.15.139:0/2392027465 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f996c02e070 con 0x7f9984159060 2023-10-18T10:24:40.676 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.667+0000 7f9980ff9640 1 -- 172.21.15.139:0/2392027465 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f996c003b20 con 0x7f9984159060 2023-10-18T10:24:40.676 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.668+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99840675f0 con 0x7f9984159060 2023-10-18T10:24:40.676 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.670+0000 7f9980ff9640 1 -- 172.21.15.139:0/2392027465 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f996c051060 con 0x7f9984159060 2023-10-18T10:24:40.692 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:40.690+0000 7f304f80f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:24:40.738 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:40.737+0000 7f304f80f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:24:40.742 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.741+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e37e6670-26f5-4525-9bfe-488052e330f6", "id": 3} v 0) v1 -- 0x7f99840cdad0 con 0x7f9984159060 2023-10-18T10:24:40.746 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.744+0000 7f9980ff9640 1 -- 172.21.15.139:0/2392027465 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e37e6670-26f5-4525-9bfe-488052e330f6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f996c034070 con 0x7f9984159060 2023-10-18T10:24:40.746 INFO:teuthology.orchestra.run.smithi139.stdout:3 2023-10-18T10:24:40.747 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9984159060 msgr2=0x7f99841e2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.748 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9984159060 0x7f99841e2af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f996c0098d0 tx=0x7f996c031c50 comp rx=0 tx=0).stop 2023-10-18T10:24:40.748 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 shutdown_connections 2023-10-18T10:24:40.748 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9984159060 0x7f99841e2af0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.748 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f998415ccf0 0x7f99841e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.749 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 --2- 172.21.15.139:0/2392027465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99840ce240 0x7f998406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.749 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 >> 172.21.15.139:0/2392027465 conn(0x7f99840b0670 msgr2=0x7f99840cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.749 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 shutdown_connections 2023-10-18T10:24:40.749 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.745+0000 7f9989c44640 1 -- 172.21.15.139:0/2392027465 wait complete. 2023-10-18T10:24:40.757 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new ff2a9e1a-8966-4124-a0ea-d32a9c494f50 4 2023-10-18T10:24:40.781 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.779+0000 7f022840d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:24:40.841 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:40.840+0000 7f304f80f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:24:40.860 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.858+0000 7f1eb4490640 1 -- 172.21.15.139:0/303730892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 msgr2=0x7f1eac06e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.860 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.858+0000 7f1eb4490640 1 --2- 172.21.15.139:0/303730892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 0x7f1eac06e3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1eac066fe0 tx=0x7f1ea402c1d0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.861 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.858+0000 7f1eb4490640 1 -- 172.21.15.139:0/303730892 shutdown_connections 2023-10-18T10:24:40.861 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.858+0000 7f1eb4490640 1 --2- 172.21.15.139:0/303730892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 0x7f1eac06e3d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.861 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.858+0000 7f1eb4490640 1 --2- 172.21.15.139:0/303730892 >> v2:172.21.15.125:3300/0 conn(0x7f1eac06d9c0 0x7f1eac06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.861 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.858+0000 7f1eb4490640 1 -- 172.21.15.139:0/303730892 >> 172.21.15.139:0/303730892 conn(0x7f1eac0b06e0 msgr2=0x7f1eac0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.862 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.858+0000 7f1eb4490640 1 -- 172.21.15.139:0/303730892 shutdown_connections 2023-10-18T10:24:40.862 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.858+0000 7f1eb4490640 1 -- 172.21.15.139:0/303730892 wait complete. 2023-10-18T10:24:40.864 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.859+0000 7f1eb4490640 1 Processor -- start 2023-10-18T10:24:40.864 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.859+0000 7f1eb4490640 1 -- start start 2023-10-18T10:24:40.865 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb4490640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eac06d9c0 0x7f1eac158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.865 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb4490640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 0x7f1eac0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.865 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb4490640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1eac15c5b0 0x7f1eac158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:40.865 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb4490640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac158980 con 0x7f1eac06d9c0 2023-10-18T10:24:40.865 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb4490640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac06b790 con 0x7f1eac0c9490 2023-10-18T10:24:40.866 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb4490640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac0b1620 con 0x7f1eac15c5b0 2023-10-18T10:24:40.866 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb1a2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 0x7f1eac0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.866 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb2a2d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1eac15c5b0 0x7f1eac158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.866 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb1a2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 0x7f1eac0b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36332/0 (socket says 172.21.15.139:36332) 2023-10-18T10:24:40.866 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb1a2b640 1 -- 172.21.15.139:0/3768059675 learned_addr learned my addr 172.21.15.139:0/3768059675 (peer_addr_for_me v2:172.21.15.139:0/0) 2023-10-18T10:24:40.867 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb2a2d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1eac15c5b0 0x7f1eac158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:60972/0 (socket says 172.21.15.139:60972) 2023-10-18T10:24:40.867 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb222c640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eac06d9c0 0x7f1eac158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:40.867 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb1a2b640 1 -- 172.21.15.139:0/3768059675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1eac15c5b0 msgr2=0x7f1eac158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.867 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.860+0000 7f1eb1a2b640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1eac15c5b0 0x7f1eac158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.867 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1eb1a2b640 1 -- 172.21.15.139:0/3768059675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eac06d9c0 msgr2=0x7f1eac158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.868 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1eb1a2b640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eac06d9c0 0x7f1eac158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.868 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1eb1a2b640 1 -- 172.21.15.139:0/3768059675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea4009900 con 0x7f1eac0c9490 2023-10-18T10:24:40.868 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1eb2a2d640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1eac15c5b0 0x7f1eac158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:40.868 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1eb1a2b640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 0x7f1eac0b2630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800d950 tx=0x7f1ea800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:40.868 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1ea2ffd640 1 -- 172.21.15.139:0/3768059675 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8009c30 con 0x7f1eac0c9490 2023-10-18T10:24:40.869 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1ea2ffd640 1 -- 172.21.15.139:0/3768059675 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1ea8009e80 con 0x7f1eac0c9490 2023-10-18T10:24:40.869 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eac0b1870 con 0x7f1eac0c9490 2023-10-18T10:24:40.869 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1ea2ffd640 1 -- 172.21.15.139:0/3768059675 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80106a0 con 0x7f1eac0c9490 2023-10-18T10:24:40.869 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eac0b1e30 con 0x7f1eac0c9490 2023-10-18T10:24:40.869 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1ea2ffd640 1 -- 172.21.15.139:0/3768059675 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1ea8010b00 con 0x7f1eac0c9490 2023-10-18T10:24:40.870 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.861+0000 7f1ea2ffd640 1 -- 172.21.15.139:0/3768059675 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f1ea800f3c0 con 0x7f1eac0c9490 2023-10-18T10:24:40.870 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.862+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eac0cc510 con 0x7f1eac0c9490 2023-10-18T10:24:40.870 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.864+0000 7f1ea2ffd640 1 -- 172.21.15.139:0/3768059675 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1ea803bb40 con 0x7f1eac0c9490 2023-10-18T10:24:40.914 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:40.913+0000 7f022840d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:24:40.937 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.935+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ff2a9e1a-8966-4124-a0ea-d32a9c494f50", "id": 4} v 0) v1 -- 0x7f1eac0690e0 con 0x7f1eac0c9490 2023-10-18T10:24:40.940 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.938+0000 7f1ea2ffd640 1 -- 172.21.15.139:0/3768059675 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ff2a9e1a-8966-4124-a0ea-d32a9c494f50", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ea802a030 con 0x7f1eac0c9490 2023-10-18T10:24:40.940 INFO:teuthology.orchestra.run.smithi139.stdout:4 2023-10-18T10:24:40.941 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f304f80f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:24:40.942 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 msgr2=0x7f1eac0b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:40.942 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 0x7f1eac0b2630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800d950 tx=0x7f1ea800de10 comp rx=0 tx=0).stop 2023-10-18T10:24:40.942 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 shutdown_connections 2023-10-18T10:24:40.942 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eac0c9490 0x7f1eac0b2630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.943 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1eac15c5b0 0x7f1eac158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.943 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 --2- 172.21.15.139:0/3768059675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eac06d9c0 0x7f1eac158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:40.943 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 >> 172.21.15.139:0/3768059675 conn(0x7f1eac0b06e0 msgr2=0x7f1eac0acb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:40.943 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 shutdown_connections 2023-10-18T10:24:40.943 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:40.939+0000 7f1eb4490640 1 -- 172.21.15.139:0/3768059675 wait complete. 2023-10-18T10:24:40.951 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 7edb5f0f-6e32-4285-8345-e719307b0955 5 2023-10-18T10:24:41.037 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:41.036+0000 7f022840d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:24:41.054 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.052+0000 7f94207e7640 1 -- 172.21.15.139:0/457240041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f941815ccf0 msgr2=0x7f941815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.054 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.052+0000 7f94207e7640 1 --2- 172.21.15.139:0/457240041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f941815ccf0 0x7f941815d120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9410005fd0 tx=0x7f941002c240 comp rx=0 tx=0).stop 2023-10-18T10:24:41.054 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.052+0000 7f94207e7640 1 -- 172.21.15.139:0/457240041 shutdown_connections 2023-10-18T10:24:41.054 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.052+0000 7f94207e7640 1 --2- 172.21.15.139:0/457240041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f941815ccf0 0x7f941815d120 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.055 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.052+0000 7f94207e7640 1 --2- 172.21.15.139:0/457240041 >> v2:172.21.15.139:3300/0 conn(0x7f94180ce240 0x7f941806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.055 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.052+0000 7f94207e7640 1 -- 172.21.15.139:0/457240041 >> 172.21.15.139:0/457240041 conn(0x7f94180b0670 msgr2=0x7f94180b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:41.055 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.052+0000 7f94207e7640 1 -- 172.21.15.139:0/457240041 shutdown_connections 2023-10-18T10:24:41.055 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.052+0000 7f94207e7640 1 -- 172.21.15.139:0/457240041 wait complete. 2023-10-18T10:24:41.057 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.053+0000 7f94207e7640 1 Processor -- start 2023-10-18T10:24:41.058 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.053+0000 7f94207e7640 1 -- start start 2023-10-18T10:24:41.058 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f94207e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94180ce240 0x7f94181f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.058 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f94207e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9418159060 0x7f94180c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.058 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f94207e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f941815ccf0 0x7f94180c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.059 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f94207e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941814e580 con 0x7f941815ccf0 2023-10-18T10:24:41.059 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f94207e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180c36c0 con 0x7f94180ce240 2023-10-18T10:24:41.059 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f94207e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94180c2e20 con 0x7f9418159060 2023-10-18T10:24:41.059 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9418159060 0x7f94180c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.059 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941e583640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94180ce240 0x7f94181f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.059 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9418159060 0x7f94180c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:60992/0 (socket says 172.21.15.139:60992) 2023-10-18T10:24:41.060 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 -- 172.21.15.139:0/1747967524 learned_addr learned my addr 172.21.15.139:0/1747967524 (peer_addr_for_me v2:172.21.15.139:0/0) 2023-10-18T10:24:41.060 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941ed84640 1 --2- 172.21.15.139:0/1747967524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f941815ccf0 0x7f94180c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.060 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 -- 172.21.15.139:0/1747967524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94180ce240 msgr2=0x7f94181f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.060 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 --2- 172.21.15.139:0/1747967524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94180ce240 0x7f94181f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.061 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 -- 172.21.15.139:0/1747967524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f941815ccf0 msgr2=0x7f94180c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.061 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 --2- 172.21.15.139:0/1747967524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f941815ccf0 0x7f94180c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.061 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 -- 172.21.15.139:0/1747967524 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410009900 con 0x7f9418159060 2023-10-18T10:24:41.061 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f941dd82640 1 --2- 172.21.15.139:0/1747967524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9418159060 0x7f94180c25d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f941400e9f0 tx=0x7f941400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:41.061 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f940effd640 1 -- 172.21.15.139:0/1747967524 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414009700 con 0x7f9418159060 2023-10-18T10:24:41.062 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f940effd640 1 -- 172.21.15.139:0/1747967524 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f941400ce70 con 0x7f9418159060 2023-10-18T10:24:41.062 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94180c3100 con 0x7f9418159060 2023-10-18T10:24:41.062 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f940effd640 1 -- 172.21.15.139:0/1747967524 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414010670 con 0x7f9418159060 2023-10-18T10:24:41.062 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.054+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94180c2b10 con 0x7f9418159060 2023-10-18T10:24:41.062 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.055+0000 7f940effd640 1 -- 172.21.15.139:0/1747967524 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f94140107d0 con 0x7f9418159060 2023-10-18T10:24:41.063 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.055+0000 7f940effd640 1 -- 172.21.15.139:0/1747967524 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9414014070 con 0x7f9418159060 2023-10-18T10:24:41.063 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.055+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94180af600 con 0x7f9418159060 2023-10-18T10:24:41.063 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.057+0000 7f940effd640 1 -- 172.21.15.139:0/1747967524 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9414026060 con 0x7f9418159060 2023-10-18T10:24:41.082 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:41.081+0000 7f022840d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:24:41.089 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:41.087+0000 7f304f80f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:24:41.130 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.128+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7edb5f0f-6e32-4285-8345-e719307b0955", "id": 5} v 0) v1 -- 0x7f94180675f0 con 0x7f9418159060 2023-10-18T10:24:41.132 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:41.131+0000 7f304f80f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:24:41.133 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.132+0000 7f940effd640 1 -- 172.21.15.139:0/1747967524 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7edb5f0f-6e32-4285-8345-e719307b0955", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9414010a80 con 0x7f9418159060 2023-10-18T10:24:41.134 INFO:teuthology.orchestra.run.smithi139.stdout:5 2023-10-18T10:24:41.135 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9418159060 msgr2=0x7f94180c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.135 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 --2- 172.21.15.139:0/1747967524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9418159060 0x7f94180c25d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f941400e9f0 tx=0x7f941400eeb0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.135 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 shutdown_connections 2023-10-18T10:24:41.135 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 --2- 172.21.15.139:0/1747967524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f941815ccf0 0x7f94180c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.136 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 --2- 172.21.15.139:0/1747967524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9418159060 0x7f94180c25d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.136 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 --2- 172.21.15.139:0/1747967524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94180ce240 0x7f94181f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.136 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 >> 172.21.15.139:0/1747967524 conn(0x7f94180b0670 msgr2=0x7f94180aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:41.136 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 shutdown_connections 2023-10-18T10:24:41.136 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.133+0000 7f94207e7640 1 -- 172.21.15.139:0/1747967524 wait complete. 2023-10-18T10:24:41.144 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 54361b74-6765-46c0-b15b-37fdb24f950b 6 2023-10-18T10:24:41.179 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:41.177+0000 7f022840d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:24:41.247 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.245+0000 7ff4cae4f640 1 -- 172.21.15.139:0/281379801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c415ccf0 msgr2=0x7ff4c415d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.248 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.245+0000 7ff4cae4f640 1 --2- 172.21.15.139:0/281379801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c415ccf0 0x7ff4c415d100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4071150 tx=0x7ff4ac02bdb0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.248 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.245+0000 7ff4cae4f640 1 -- 172.21.15.139:0/281379801 shutdown_connections 2023-10-18T10:24:41.248 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.245+0000 7ff4cae4f640 1 --2- 172.21.15.139:0/281379801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c415ccf0 0x7ff4c415d100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.249 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.245+0000 7ff4cae4f640 1 --2- 172.21.15.139:0/281379801 >> v2:172.21.15.125:3300/0 conn(0x7ff4c40ce240 0x7ff4c406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.249 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.245+0000 7ff4cae4f640 1 -- 172.21.15.139:0/281379801 >> 172.21.15.139:0/281379801 conn(0x7ff4c40b0670 msgr2=0x7ff4c40b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:41.249 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.245+0000 7ff4cae4f640 1 -- 172.21.15.139:0/281379801 shutdown_connections 2023-10-18T10:24:41.249 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.246+0000 7ff4cae4f640 1 -- 172.21.15.139:0/281379801 wait complete. 2023-10-18T10:24:41.252 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.247+0000 7ff4cae4f640 1 Processor -- start 2023-10-18T10:24:41.252 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.247+0000 7ff4cae4f640 1 -- start start 2023-10-18T10:24:41.252 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.247+0000 7ff4cae4f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40ce240 0x7ff4c40b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.252 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.247+0000 7ff4cae4f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c4159060 0x7ff4c41e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.253 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4cae4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415ccf0 0x7ff4c414ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.253 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4cae4f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40b2170 con 0x7ff4c415ccf0 2023-10-18T10:24:41.253 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4cae4f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40b4360 con 0x7ff4c4159060 2023-10-18T10:24:41.253 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4cae4f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c40b4640 con 0x7ff4c40ce240 2023-10-18T10:24:41.253 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c4159060 0x7ff4c41e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.254 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c8beb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40ce240 0x7ff4c40b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.254 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c4159060 0x7ff4c41e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36374/0 (socket says 172.21.15.139:36374) 2023-10-18T10:24:41.254 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c3fff640 1 -- 172.21.15.139:0/844672834 learned_addr learned my addr 172.21.15.139:0/844672834 (peer_addr_for_me v2:172.21.15.139:0/0) 2023-10-18T10:24:41.254 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c93ec640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415ccf0 0x7ff4c414ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.254 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c8beb640 1 -- 172.21.15.139:0/844672834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c4159060 msgr2=0x7ff4c41e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.255 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c8beb640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c4159060 0x7ff4c41e4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.255 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c8beb640 1 -- 172.21.15.139:0/844672834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415ccf0 msgr2=0x7ff4c414ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.255 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c8beb640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415ccf0 0x7ff4c414ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.255 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c8beb640 1 -- 172.21.15.139:0/844672834 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac009560 con 0x7ff4c40ce240 2023-10-18T10:24:41.255 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c3fff640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c4159060 0x7ff4c41e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:41.256 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c8beb640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40ce240 0x7ff4c40b2c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c390 tx=0x7ff4bc00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:41.256 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c17fa640 1 -- 172.21.15.139:0/844672834 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0052d0 con 0x7ff4c40ce240 2023-10-18T10:24:41.256 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c17fa640 1 -- 172.21.15.139:0/844672834 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4bc0035f0 con 0x7ff4c40ce240 2023-10-18T10:24:41.256 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c40b2430 con 0x7ff4c40ce240 2023-10-18T10:24:41.256 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4c17fa640 1 -- 172.21.15.139:0/844672834 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc003a00 con 0x7ff4c40ce240 2023-10-18T10:24:41.257 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.248+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c41586d0 con 0x7ff4c40ce240 2023-10-18T10:24:41.257 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.249+0000 7ff4c17fa640 1 -- 172.21.15.139:0/844672834 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff4bc011590 con 0x7ff4c40ce240 2023-10-18T10:24:41.257 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.249+0000 7ff4c17fa640 1 -- 172.21.15.139:0/844672834 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff4bc012a10 con 0x7ff4c40ce240 2023-10-18T10:24:41.257 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.249+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c40675f0 con 0x7ff4c40ce240 2023-10-18T10:24:41.258 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.253+0000 7ff4c17fa640 1 -- 172.21.15.139:0/844672834 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff4bc017070 con 0x7ff4c40ce240 2023-10-18T10:24:41.273 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:41.271+0000 7f022840d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:24:41.283 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:41.281+0000 7f304f80f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:24:41.326 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.325+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "54361b74-6765-46c0-b15b-37fdb24f950b", "id": 6} v 0) v1 -- 0x7ff4c40cde40 con 0x7ff4c40ce240 2023-10-18T10:24:41.330 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.328+0000 7ff4c17fa640 1 -- 172.21.15.139:0/844672834 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "54361b74-6765-46c0-b15b-37fdb24f950b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff4bc012420 con 0x7ff4c40ce240 2023-10-18T10:24:41.330 INFO:teuthology.orchestra.run.smithi139.stdout:6 2023-10-18T10:24:41.331 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40ce240 msgr2=0x7ff4c40b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.331 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40ce240 0x7ff4c40b2c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c390 tx=0x7ff4bc00c850 comp rx=0 tx=0).stop 2023-10-18T10:24:41.332 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 shutdown_connections 2023-10-18T10:24:41.332 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c4159060 0x7ff4c41e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.332 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415ccf0 0x7ff4c414ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.332 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 --2- 172.21.15.139:0/844672834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40ce240 0x7ff4c40b2c30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.332 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 >> 172.21.15.139:0/844672834 conn(0x7ff4c40b0670 msgr2=0x7ff4c40cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:41.333 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 shutdown_connections 2023-10-18T10:24:41.333 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.329+0000 7ff4cae4f640 1 -- 172.21.15.139:0/844672834 wait complete. 2023-10-18T10:24:41.341 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new ac3ffb65-f724-44ce-a9cf-d06ce7966b4a 7 2023-10-18T10:24:41.413 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:41.412+0000 7f022840d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:24:41.446 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.444+0000 7f38984b4640 1 -- 172.21.15.139:0/1789725091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38900cf340 msgr2=0x7f38900d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.446 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.444+0000 7f38984b4640 1 --2- 172.21.15.139:0/1789725091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38900cf340 0x7f38900d1910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3880009830 tx=0x7f388002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.446 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.444+0000 7f38984b4640 1 -- 172.21.15.139:0/1789725091 shutdown_connections 2023-10-18T10:24:41.447 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.444+0000 7f38984b4640 1 --2- 172.21.15.139:0/1789725091 >> v2:172.21.15.125:3300/0 conn(0x7f38900ceb40 0x7f38900b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.447 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.444+0000 7f38984b4640 1 --2- 172.21.15.139:0/1789725091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38900cf340 0x7f38900d1910 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.447 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.444+0000 7f38984b4640 1 -- 172.21.15.139:0/1789725091 >> 172.21.15.139:0/1789725091 conn(0x7f38900d0d60 msgr2=0x7f38900d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:41.447 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.444+0000 7f38984b4640 1 -- 172.21.15.139:0/1789725091 shutdown_connections 2023-10-18T10:24:41.447 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.444+0000 7f38984b4640 1 -- 172.21.15.139:0/1789725091 wait complete. 2023-10-18T10:24:41.450 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.445+0000 7f38984b4640 1 Processor -- start 2023-10-18T10:24:41.450 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.445+0000 7f38984b4640 1 -- start start 2023-10-18T10:24:41.450 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f38984b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38900ceb40 0x7f38900b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.450 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f38984b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38900cf340 0x7f38901510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.451 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f38984b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f389015b2a0 0x7f389014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.451 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f38984b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900b5230 con 0x7f38900cf340 2023-10-18T10:24:41.451 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f38984b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38901f14f0 con 0x7f38900ceb40 2023-10-18T10:24:41.451 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f38984b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38900b4620 con 0x7f389015b2a0 2023-10-18T10:24:41.451 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38900ceb40 0x7f38900b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.452 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896a51640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f389015b2a0 0x7f389014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.452 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38900ceb40 0x7f38900b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36388/0 (socket says 172.21.15.139:36388) 2023-10-18T10:24:41.452 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 -- 172.21.15.139:0/1533156480 learned_addr learned my addr 172.21.15.139:0/1533156480 (peer_addr_for_me v2:172.21.15.139:0/0) 2023-10-18T10:24:41.452 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3895a4f640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38900cf340 0x7f38901510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.452 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 -- 172.21.15.139:0/1533156480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f389015b2a0 msgr2=0x7f389014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.453 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f389015b2a0 0x7f389014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.453 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 -- 172.21.15.139:0/1533156480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38900cf340 msgr2=0x7f38901510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.453 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38900cf340 0x7f38901510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.453 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 -- 172.21.15.139:0/1533156480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880009490 con 0x7f38900ceb40 2023-10-18T10:24:41.453 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896a51640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f389015b2a0 0x7f389014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:24:41.454 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3896250640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38900ceb40 0x7f38900b3f90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3880009780 tx=0x7f38800038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:41.454 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3886ffd640 1 -- 172.21.15.139:0/1533156480 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880003ea0 con 0x7f38900ceb40 2023-10-18T10:24:41.454 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f3886ffd640 1 -- 172.21.15.139:0/1533156480 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3880034440 con 0x7f38900ceb40 2023-10-18T10:24:41.454 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.446+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38900b4900 con 0x7f38900ceb40 2023-10-18T10:24:41.454 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.447+0000 7f3886ffd640 1 -- 172.21.15.139:0/1533156480 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880034850 con 0x7f38900ceb40 2023-10-18T10:24:41.454 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.447+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38900b4ec0 con 0x7f38900ceb40 2023-10-18T10:24:41.455 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.447+0000 7f3886ffd640 1 -- 172.21.15.139:0/1533156480 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f388003b040 con 0x7f38900ceb40 2023-10-18T10:24:41.455 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.447+0000 7f3886ffd640 1 -- 172.21.15.139:0/1533156480 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f3880046400 con 0x7f38900ceb40 2023-10-18T10:24:41.455 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.447+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38900690e0 con 0x7f38900ceb40 2023-10-18T10:24:41.456 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:41.454+0000 7f022840d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:24:41.456 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.449+0000 7f3886ffd640 1 -- 172.21.15.139:0/1533156480 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3880041020 con 0x7f38900ceb40 2023-10-18T10:24:41.522 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.521+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ac3ffb65-f724-44ce-a9cf-d06ce7966b4a", "id": 7} v 0) v1 -- 0x7f38900cba00 con 0x7f38900ceb40 2023-10-18T10:24:41.525 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.524+0000 7f3886ffd640 1 -- 172.21.15.139:0/1533156480 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac3ffb65-f724-44ce-a9cf-d06ce7966b4a", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3880039070 con 0x7f38900ceb40 2023-10-18T10:24:41.526 INFO:teuthology.orchestra.run.smithi139.stdout:7 2023-10-18T10:24:41.527 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38900ceb40 msgr2=0x7f38900b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.527 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38900ceb40 0x7f38900b3f90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3880009780 tx=0x7f38800038a0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.527 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 shutdown_connections 2023-10-18T10:24:41.527 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f389015b2a0 0x7f389014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.527 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38900cf340 0x7f38901510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.528 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 --2- 172.21.15.139:0/1533156480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38900ceb40 0x7f38900b3f90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.528 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 >> 172.21.15.139:0/1533156480 conn(0x7f38900d0d60 msgr2=0x7f389015a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:41.528 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 shutdown_connections 2023-10-18T10:24:41.528 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:24:41.525+0000 7f38984b4640 1 -- 172.21.15.139:0/1533156480 wait complete. 2023-10-18T10:24:41.536 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-18T10:24:41.537 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-18T10:24:41.539 INFO:tasks.ceph.osd.0:Started 2023-10-18T10:24:41.540 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-18T10:24:41.540 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-18T10:24:41.542 INFO:tasks.ceph.osd.1:Started 2023-10-18T10:24:41.542 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-18T10:24:41.543 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-18T10:24:41.545 INFO:tasks.ceph.osd.2:Started 2023-10-18T10:24:41.545 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-18T10:24:41.546 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-18T10:24:41.548 INFO:tasks.ceph.osd.3:Started 2023-10-18T10:24:41.548 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-18T10:24:41.548 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-18T10:24:41.551 INFO:tasks.ceph.osd.4:Started 2023-10-18T10:24:41.551 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-18T10:24:41.551 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-18T10:24:41.554 INFO:tasks.ceph.osd.5:Started 2023-10-18T10:24:41.554 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-18T10:24:41.554 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-18T10:24:41.557 INFO:tasks.ceph.osd.6:Started 2023-10-18T10:24:41.557 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-18T10:24:41.557 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-18T10:24:41.559 INFO:tasks.ceph.osd.7:Started 2023-10-18T10:24:41.560 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:24:41.630 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:41.628+0000 7f022840d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:24:41.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.678+0000 7f54482da640 1 -- 172.21.15.125:0/3661267354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54400cdb30 msgr2=0x7f54400cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.679+0000 7f54482da640 1 --2- 172.21.15.125:0/3661267354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54400cdb30 0x7f54400cdfb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f543c009830 tx=0x7f543c02bcd0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.679+0000 7f54482da640 1 -- 172.21.15.125:0/3661267354 shutdown_connections 2023-10-18T10:24:41.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.679+0000 7f54482da640 1 --2- 172.21.15.125:0/3661267354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54400cdb30 0x7f54400cdfb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.679+0000 7f54482da640 1 --2- 172.21.15.125:0/3661267354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5440159060 0x7f544006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.679+0000 7f54482da640 1 -- 172.21.15.125:0/3661267354 >> 172.21.15.125:0/3661267354 conn(0x7f54400b0670 msgr2=0x7f54400b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:41.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.679+0000 7f54482da640 1 -- 172.21.15.125:0/3661267354 shutdown_connections 2023-10-18T10:24:41.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.679+0000 7f54482da640 1 -- 172.21.15.125:0/3661267354 wait complete. 2023-10-18T10:24:41.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.680+0000 7f54482da640 1 Processor -- start 2023-10-18T10:24:41.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.680+0000 7f54482da640 1 -- start start 2023-10-18T10:24:41.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.681+0000 7f54482da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54400cdb30 0x7f54400c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.681+0000 7f54482da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5440159060 0x7f54400ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.681+0000 7f54482da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f544015ccf0 0x7f54400ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:41.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.681+0000 7f54482da640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54400b2110 con 0x7f5440159060 2023-10-18T10:24:41.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.681+0000 7f5445875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5440159060 0x7f54400ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.681+0000 7f5445875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5440159060 0x7f54400ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36050/0 (socket says 172.21.15.125:36050) 2023-10-18T10:24:41.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.681+0000 7f5445875640 1 -- 172.21.15.125:0/2850241120 learned_addr learned my addr 172.21.15.125:0/2850241120 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:41.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.681+0000 7f5446076640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54400cdb30 0x7f54400c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5446877640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f544015ccf0 0x7f54400ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:41.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54400b14d0 con 0x7f54400cdb30 2023-10-18T10:24:41.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54400c8000 con 0x7f544015ccf0 2023-10-18T10:24:41.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5445875640 1 -- 172.21.15.125:0/2850241120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f544015ccf0 msgr2=0x7f54400ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5445875640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f544015ccf0 0x7f54400ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5445875640 1 -- 172.21.15.125:0/2850241120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54400cdb30 msgr2=0x7f54400c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5445875640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54400cdb30 0x7f54400c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5445875640 1 -- 172.21.15.125:0/2850241120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c009490 con 0x7f5440159060 2023-10-18T10:24:41.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5445875640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5440159060 0x7f54400ae380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f543c009780 tx=0x7f543c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:41.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5436ffd640 1 -- 172.21.15.125:0/2850241120 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c003ea0 con 0x7f5440159060 2023-10-18T10:24:41.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5436ffd640 1 -- 172.21.15.125:0/2850241120 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f543c034440 con 0x7f5440159060 2023-10-18T10:24:41.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f5436ffd640 1 -- 172.21.15.125:0/2850241120 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c034850 con 0x7f5440159060 2023-10-18T10:24:41.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54400b1770 con 0x7f5440159060 2023-10-18T10:24:41.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.682+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54400b1d30 con 0x7f5440159060 2023-10-18T10:24:41.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.683+0000 7f5436ffd640 1 -- 172.21.15.125:0/2850241120 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f543c03b040 con 0x7f5440159060 2023-10-18T10:24:41.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.683+0000 7f5436ffd640 1 -- 172.21.15.125:0/2850241120 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f543c046ad0 con 0x7f5440159060 2023-10-18T10:24:41.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.684+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54400b1ec0 con 0x7f5440159060 2023-10-18T10:24:41.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.685+0000 7f5436ffd640 1 -- 172.21.15.125:0/2850241120 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f543c041020 con 0x7f5440159060 2023-10-18T10:24:41.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.746+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54400b2250 con 0x7f5440159060 2023-10-18T10:24:41.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.746+0000 7f5436ffd640 1 -- 172.21.15.125:0/2850241120 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f54400b2250 con 0x7f5440159060 2023-10-18T10:24:41.748 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:41.749 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":10,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:24:41.522717+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:24:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5440159060 msgr2=0x7f54400ae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5440159060 0x7f54400ae380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f543c009780 tx=0x7f543c0038a0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 shutdown_connections 2023-10-18T10:24:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f544015ccf0 0x7f54400ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5440159060 0x7f54400ae380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 --2- 172.21.15.125:0/2850241120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54400cdb30 0x7f54400c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 >> 172.21.15.125:0/2850241120 conn(0x7f54400b0670 msgr2=0x7f54400af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 shutdown_connections 2023-10-18T10:24:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:41.747+0000 7f54482da640 1 -- 172.21.15.125:0/2850241120 wait complete. 2023-10-18T10:24:41.759 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-18T10:24:41.759 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-18T10:24:42.118 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:42.116+0000 7f304f80f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:24:42.137 INFO:tasks.ceph.osd.0.smithi125.stderr:2023-10-18T10:24:42.135+0000 7f8b9b0da500 -1 Falling back to public interface 2023-10-18T10:24:42.153 INFO:tasks.ceph.osd.4.smithi139.stderr:2023-10-18T10:24:42.151+0000 7f4852aad500 -1 Falling back to public interface 2023-10-18T10:24:42.158 INFO:tasks.ceph.osd.6.smithi139.stderr:2023-10-18T10:24:42.156+0000 7f5b60014500 -1 Falling back to public interface 2023-10-18T10:24:42.170 INFO:tasks.ceph.osd.5.smithi139.stderr:2023-10-18T10:24:42.168+0000 7fc857416500 -1 Falling back to public interface 2023-10-18T10:24:42.170 INFO:tasks.ceph.osd.3.smithi125.stderr:2023-10-18T10:24:42.168+0000 7f12ae2b4500 -1 Falling back to public interface 2023-10-18T10:24:42.171 INFO:tasks.ceph.osd.1.smithi125.stderr:2023-10-18T10:24:42.169+0000 7f4829ca9500 -1 Falling back to public interface 2023-10-18T10:24:42.174 INFO:tasks.ceph.osd.2.smithi125.stderr:2023-10-18T10:24:42.172+0000 7fb0cfe0d500 -1 Falling back to public interface 2023-10-18T10:24:42.175 INFO:tasks.ceph.osd.7.smithi139.stderr:2023-10-18T10:24:42.173+0000 7f6cf7211500 -1 Falling back to public interface 2023-10-18T10:24:42.388 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:42.387+0000 7f304f80f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:24:42.441 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:42.439+0000 7f304f80f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:24:42.448 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:42.446+0000 7f022840d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:24:42.541 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:42.539+0000 7f304f80f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:24:42.565 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T10:24:42.659 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.656+0000 7f2c3244b640 1 -- 172.21.15.125:0/2545430578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0cf340 msgr2=0x7f2c2c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:42.659 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.656+0000 7f2c3244b640 1 --2- 172.21.15.125:0/2545430578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0cf340 0x7f2c2c0d1910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005fd0 tx=0x7f2c1c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:24:42.659 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.656+0000 7f2c3244b640 1 -- 172.21.15.125:0/2545430578 shutdown_connections 2023-10-18T10:24:42.660 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.656+0000 7f2c3244b640 1 --2- 172.21.15.125:0/2545430578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c2c0ceb40 0x7f2c2c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:42.660 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.656+0000 7f2c3244b640 1 --2- 172.21.15.125:0/2545430578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0cf340 0x7f2c2c0d1910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:42.660 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.656+0000 7f2c3244b640 1 -- 172.21.15.125:0/2545430578 >> 172.21.15.125:0/2545430578 conn(0x7f2c2c0d0d60 msgr2=0x7f2c2c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:42.660 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.656+0000 7f2c3244b640 1 -- 172.21.15.125:0/2545430578 shutdown_connections 2023-10-18T10:24:42.660 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.656+0000 7f2c3244b640 1 -- 172.21.15.125:0/2545430578 wait complete. 2023-10-18T10:24:42.663 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.657+0000 7f2c3244b640 1 Processor -- start 2023-10-18T10:24:42.663 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c3244b640 1 -- start start 2023-10-18T10:24:42.663 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c3244b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0ceb40 0x7f2c2c1f1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:42.663 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c3244b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c0cf340 0x7f2c2c0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:42.663 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c3244b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c2c15b2a0 0x7f2c2c0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:42.664 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c3244b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15e520 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.664 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c3244b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15db60 con 0x7f2c2c0cf340 2023-10-18T10:24:42.664 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c3244b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0c9780 con 0x7f2c2c15b2a0 2023-10-18T10:24:42.664 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0ceb40 0x7f2c2c1f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:42.664 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0ceb40 0x7f2c2c1f1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36072/0 (socket says 172.21.15.125:36072) 2023-10-18T10:24:42.665 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.658+0000 7f2c2bfff640 1 -- 172.21.15.125:0/1033256874 learned_addr learned my addr 172.21.15.125:0/1033256874 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:42.665 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c309e8640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c2c15b2a0 0x7f2c2c0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:42.665 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c2b7fe640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c0cf340 0x7f2c2c0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:42.665 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c2bfff640 1 -- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c2c15b2a0 msgr2=0x7f2c2c0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:42.665 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c2bfff640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c2c15b2a0 0x7f2c2c0b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:42.666 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c2bfff640 1 -- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c0cf340 msgr2=0x7f2c2c0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:42.666 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c2bfff640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c0cf340 0x7f2c2c0c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:42.666 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c2bfff640 1 -- 172.21.15.125:0/1033256874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c009560 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.666 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c309e8640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c2c15b2a0 0x7f2c2c0b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:24:42.666 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c2bfff640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0ceb40 0x7f2c2c1f1310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009850 tx=0x7f2c1c02d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:42.667 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c28ff9640 1 -- 172.21.15.125:0/1033256874 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c02ddd0 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.667 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c28ff9640 1 -- 172.21.15.125:0/1033256874 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2c1c0036e0 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.667 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c0b0d50 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.667 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c28ff9640 1 -- 172.21.15.125:0/1033256874 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c003a90 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.667 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.659+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c1f1bf0 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.668 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.660+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c0690e0 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.668 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.660+0000 7f2c28ff9640 1 -- 172.21.15.125:0/1033256874 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 2) v1 ==== 44566+0+0 (secure 0 0 0) 0x7f2c1c003bf0 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.668 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.660+0000 7f2c28ff9640 1 -- 172.21.15.125:0/1033256874 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f2c1c06c110 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.668 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.662+0000 7f2c28ff9640 1 -- 172.21.15.125:0/1033256874 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2c1c047d80 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.724 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.722+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c2c0d0b40 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.725 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.723+0000 7f2c28ff9640 1 -- 172.21.15.125:0/1033256874 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2c1c043030 con 0x7f2c2c0ceb40 2023-10-18T10:24:42.725 INFO:teuthology.misc.health.smithi125.stdout: 2023-10-18T10:24:42.726 INFO:teuthology.misc.health.smithi125.stdout:{"epoch":10,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:24:41.522717+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:24:42.727 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0ceb40 msgr2=0x7f2c2c1f1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:42.727 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0ceb40 0x7f2c2c1f1310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009850 tx=0x7f2c1c02d9d0 comp rx=0 tx=0).stop 2023-10-18T10:24:42.727 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 shutdown_connections 2023-10-18T10:24:42.727 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c0cf340 0x7f2c2c0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:42.728 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c2c15b2a0 0x7f2c2c0b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:42.728 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 --2- 172.21.15.125:0/1033256874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c2c0ceb40 0x7f2c2c1f1310 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:42.728 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 >> 172.21.15.125:0/1033256874 conn(0x7f2c2c0d0d60 msgr2=0x7f2c2c15a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:42.728 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 shutdown_connections 2023-10-18T10:24:42.728 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:42.724+0000 7f2c3244b640 1 -- 172.21.15.125:0/1033256874 wait complete. 2023-10-18T10:24:42.735 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-18T10:24:42.736 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:42.733+0000 7f022840d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:24:42.787 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:42.785+0000 7f022840d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:24:42.887 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:24:42.886+0000 7f022840d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:24:43.570 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:43.568+0000 7f2ff08d7640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-18T10:24:43.570 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:24:43.568+0000 7f2ff08d7640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-18T10:24:45.584 INFO:tasks.ceph.osd.0.smithi125.stderr:2023-10-18T10:24:45.582+0000 7f8b9b0da500 -1 osd.0 0 log_to_monitors true 2023-10-18T10:24:45.590 INFO:tasks.ceph.osd.4.smithi139.stderr:2023-10-18T10:24:45.589+0000 7f4852aad500 -1 osd.4 0 log_to_monitors true 2023-10-18T10:24:45.615 INFO:tasks.ceph.osd.3.smithi125.stderr:2023-10-18T10:24:45.613+0000 7f12ae2b4500 -1 osd.3 0 log_to_monitors true 2023-10-18T10:24:45.646 INFO:tasks.ceph.osd.6.smithi139.stderr:2023-10-18T10:24:45.645+0000 7f5b60014500 -1 osd.6 0 log_to_monitors true 2023-10-18T10:24:45.654 INFO:tasks.ceph.osd.1.smithi125.stderr:2023-10-18T10:24:45.652+0000 7f4829ca9500 -1 osd.1 0 log_to_monitors true 2023-10-18T10:24:45.657 INFO:tasks.ceph.osd.5.smithi139.stderr:2023-10-18T10:24:45.656+0000 7fc857416500 -1 osd.5 0 log_to_monitors true 2023-10-18T10:24:45.666 INFO:tasks.ceph.osd.7.smithi139.stderr:2023-10-18T10:24:45.665+0000 7f6cf7211500 -1 osd.7 0 log_to_monitors true 2023-10-18T10:24:45.688 INFO:tasks.ceph.osd.2.smithi125.stderr:2023-10-18T10:24:45.686+0000 7fb0cfe0d500 -1 osd.2 0 log_to_monitors true 2023-10-18T10:24:49.542 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T10:24:49.607 INFO:tasks.ceph.osd.1.smithi125.stderr:2023-10-18T10:24:49.605+0000 7f4825c35640 -1 osd.1 0 waiting for initial osdmap 2023-10-18T10:24:49.623 INFO:tasks.ceph.osd.1.smithi125.stderr:2023-10-18T10:24:49.621+0000 7f481da55640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:24:49.682 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.680+0000 7ff49f4bb640 1 -- 172.21.15.125:0/2823314428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498158300 msgr2=0x7ff4980c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:49.682 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.680+0000 7ff49f4bb640 1 --2- 172.21.15.125:0/2823314428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498158300 0x7ff4980c9730 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff494009830 tx=0x7ff49402bd60 comp rx=0 tx=0).stop 2023-10-18T10:24:49.682 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.680+0000 7ff49f4bb640 1 -- 172.21.15.125:0/2823314428 shutdown_connections 2023-10-18T10:24:49.683 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.680+0000 7ff49f4bb640 1 --2- 172.21.15.125:0/2823314428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498158300 0x7ff4980c9730 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.683 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.680+0000 7ff49f4bb640 1 --2- 172.21.15.125:0/2823314428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4980afa20 0x7ff4980cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.683 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.680+0000 7ff49f4bb640 1 -- 172.21.15.125:0/2823314428 >> 172.21.15.125:0/2823314428 conn(0x7ff498069930 msgr2=0x7ff498069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:49.683 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.680+0000 7ff49f4bb640 1 -- 172.21.15.125:0/2823314428 shutdown_connections 2023-10-18T10:24:49.683 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.680+0000 7ff49f4bb640 1 -- 172.21.15.125:0/2823314428 wait complete. 2023-10-18T10:24:49.684 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.682+0000 7ff49f4bb640 1 Processor -- start 2023-10-18T10:24:49.684 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.682+0000 7ff49f4bb640 1 -- start start 2023-10-18T10:24:49.686 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.683+0000 7ff49f4bb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4980705a0 0x7ff49813b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:49.686 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.683+0000 7ff49f4bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4980afa20 0x7ff498152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:49.686 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.683+0000 7ff49f4bb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498140ad0 0x7ff4981f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:49.686 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.683+0000 7ff49da58640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498140ad0 0x7ff4981f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:49.686 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.683+0000 7ff49da58640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498140ad0 0x7ff4981f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37964/0 (socket says 172.21.15.125:37964) 2023-10-18T10:24:49.687 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.683+0000 7ff49f4bb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498154190 con 0x7ff498140ad0 2023-10-18T10:24:49.687 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.683+0000 7ff49f4bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498154620 con 0x7ff4980afa20 2023-10-18T10:24:49.687 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.683+0000 7ff49f4bb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff498153b80 con 0x7ff4980705a0 2023-10-18T10:24:49.687 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.684+0000 7ff49ca56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4980afa20 0x7ff498152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:49.687 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.684+0000 7ff49ca56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4980afa20 0x7ff498152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:43522/0 (socket says 172.21.15.125:43522) 2023-10-18T10:24:49.688 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.684+0000 7ff49ca56640 1 -- 172.21.15.125:0/1107275417 learned_addr learned my addr 172.21.15.125:0/1107275417 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:49.688 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.684+0000 7ff49d257640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4980705a0 0x7ff49813b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:49.689 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.685+0000 7ff49ca56640 1 -- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4980705a0 msgr2=0x7ff49813b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:49.689 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.685+0000 7ff49ca56640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4980705a0 0x7ff49813b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.689 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.685+0000 7ff49ca56640 1 -- 172.21.15.125:0/1107275417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498140ad0 msgr2=0x7ff4981f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:49.690 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.685+0000 7ff49ca56640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498140ad0 0x7ff4981f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.690 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.685+0000 7ff49ca56640 1 -- 172.21.15.125:0/1107275417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009490 con 0x7ff4980afa20 2023-10-18T10:24:49.690 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.685+0000 7ff49ca56640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4980afa20 0x7ff498152120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff494000c00 tx=0x7ff494003ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:49.690 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.686+0000 7ff48dffb640 1 -- 172.21.15.125:0/1107275417 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49402c900 con 0x7ff4980afa20 2023-10-18T10:24:49.690 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.686+0000 7ff48dffb640 1 -- 172.21.15.125:0/1107275417 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff494005020 con 0x7ff4980afa20 2023-10-18T10:24:49.691 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.687+0000 7ff48dffb640 1 -- 172.21.15.125:0/1107275417 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494005410 con 0x7ff4980afa20 2023-10-18T10:24:49.691 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.687+0000 7ff49f4bb640 1 -- 172.21.15.125:0/1107275417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff498140fd0 con 0x7ff4980afa20 2023-10-18T10:24:49.691 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.687+0000 7ff49f4bb640 1 -- 172.21.15.125:0/1107275417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4981412b0 con 0x7ff4980afa20 2023-10-18T10:24:49.692 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.688+0000 7ff47b7fe640 1 -- 172.21.15.125:0/1107275417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff474005250 con 0x7ff4980afa20 2023-10-18T10:24:49.692 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.690+0000 7ff48dffb640 1 -- 172.21.15.125:0/1107275417 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7ff4940047a0 con 0x7ff4980afa20 2023-10-18T10:24:49.692 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.690+0000 7ff48dffb640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff47c06adb0 0x7ff47c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:49.693 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.690+0000 7ff48dffb640 1 -- 172.21.15.125:0/1107275417 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7ff4940aad80 con 0x7ff4980afa20 2023-10-18T10:24:49.693 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.690+0000 7ff49d257640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff47c06adb0 0x7ff47c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:49.693 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.691+0000 7ff49d257640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff47c06adb0 0x7ff47c06d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff488009ac0 tx=0x7ff488005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:49.701 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.699+0000 7ff48dffb640 1 -- 172.21.15.125:0/1107275417 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff494076d30 con 0x7ff4980afa20 2023-10-18T10:24:49.704 INFO:tasks.ceph.osd.4.smithi139.stderr:2023-10-18T10:24:49.702+0000 7f484ea39640 -1 osd.4 0 waiting for initial osdmap 2023-10-18T10:24:49.709 INFO:tasks.ceph.osd.5.smithi139.stderr:2023-10-18T10:24:49.708+0000 7fc852d7c640 -1 osd.5 0 waiting for initial osdmap 2023-10-18T10:24:49.712 INFO:tasks.ceph.osd.6.smithi139.stderr:2023-10-18T10:24:49.710+0000 7f5b5bfa2640 -1 osd.6 0 waiting for initial osdmap 2023-10-18T10:24:49.714 INFO:tasks.ceph.osd.7.smithi139.stderr:2023-10-18T10:24:49.713+0000 7f6cf2bdc640 -1 osd.7 0 waiting for initial osdmap 2023-10-18T10:24:49.720 INFO:tasks.ceph.osd.4.smithi139.stderr:2023-10-18T10:24:49.719+0000 7f4846859640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:24:49.724 INFO:tasks.ceph.osd.5.smithi139.stderr:2023-10-18T10:24:49.723+0000 7fc84ab9c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:24:49.725 INFO:tasks.ceph.osd.6.smithi139.stderr:2023-10-18T10:24:49.723+0000 7f5b53dc2640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:24:49.728 INFO:tasks.ceph.osd.7.smithi139.stderr:2023-10-18T10:24:49.727+0000 7f6cea9fc640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:24:49.784 INFO:tasks.ceph.osd.0.smithi125.stderr:2023-10-18T10:24:49.782+0000 7f8b97068640 -1 osd.0 0 waiting for initial osdmap 2023-10-18T10:24:49.792 INFO:tasks.ceph.osd.0.smithi125.stderr:2023-10-18T10:24:49.791+0000 7f8b8ee88640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:24:49.799 INFO:tasks.ceph.osd.3.smithi125.stderr:2023-10-18T10:24:49.797+0000 7f12aaa56640 -1 osd.3 0 waiting for initial osdmap 2023-10-18T10:24:49.812 INFO:tasks.ceph.osd.3.smithi125.stderr:2023-10-18T10:24:49.811+0000 7f12a2060640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:24:49.815 INFO:tasks.ceph.osd.2.smithi125.stderr:2023-10-18T10:24:49.814+0000 7fb0cb725640 -1 osd.2 0 waiting for initial osdmap 2023-10-18T10:24:49.831 INFO:tasks.ceph.osd.2.smithi125.stderr:2023-10-18T10:24:49.830+0000 7fb0c3545640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T10:24:49.921 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.920+0000 7ff47b7fe640 1 -- 172.21.15.125:0/1107275417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4740050c0 con 0x7ff4980afa20 2023-10-18T10:24:49.922 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.920+0000 7ff48dffb640 1 -- 172.21.15.125:0/1107275417 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff494042020 con 0x7ff4980afa20 2023-10-18T10:24:49.922 INFO:teuthology.misc.health.smithi125.stdout: 2023-10-18T10:24:49.923 INFO:teuthology.misc.health.smithi125.stdout:{"epoch":12,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:24:47.566517+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:24:49.924 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.922+0000 7ff47b7fe640 1 -- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff47c06adb0 msgr2=0x7ff47c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:49.925 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff47c06adb0 0x7ff47c06d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff488009ac0 tx=0x7ff488005f90 comp rx=0 tx=0).stop 2023-10-18T10:24:49.925 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 -- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4980afa20 msgr2=0x7ff498152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:49.925 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4980afa20 0x7ff498152120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff494000c00 tx=0x7ff494003ae0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.925 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 -- 172.21.15.125:0/1107275417 shutdown_connections 2023-10-18T10:24:49.926 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff47c06adb0 0x7ff47c06d270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.926 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4980afa20 0x7ff498152120 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.926 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff498140ad0 0x7ff4981f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.926 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 --2- 172.21.15.125:0/1107275417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4980705a0 0x7ff49813b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:49.926 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 -- 172.21.15.125:0/1107275417 >> 172.21.15.125:0/1107275417 conn(0x7ff498069930 msgr2=0x7ff4980ac590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:49.927 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 -- 172.21.15.125:0/1107275417 shutdown_connections 2023-10-18T10:24:49.927 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:49.923+0000 7ff47b7fe640 1 -- 172.21.15.125:0/1107275417 wait complete. 2023-10-18T10:24:49.933 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-18T10:24:56.740 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T10:24:56.840 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.837+0000 7f2313256640 1 -- 172.21.15.125:0/1359812960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c06cf80 msgr2=0x7f230c0ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:56.840 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.837+0000 7f2313256640 1 --2- 172.21.15.125:0/1359812960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c06cf80 0x7f230c0ca870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005fd0 tx=0x7f22f802be10 comp rx=0 tx=0).stop 2023-10-18T10:24:56.840 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.837+0000 7f2313256640 1 -- 172.21.15.125:0/1359812960 shutdown_connections 2023-10-18T10:24:56.840 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.837+0000 7f2313256640 1 --2- 172.21.15.125:0/1359812960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c06cf80 0x7f230c0ca870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:56.841 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.837+0000 7f2313256640 1 --2- 172.21.15.125:0/1359812960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f230c1519d0 0x7f230c158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:56.841 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.837+0000 7f2313256640 1 -- 172.21.15.125:0/1359812960 >> 172.21.15.125:0/1359812960 conn(0x7f230c069930 msgr2=0x7f230c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:56.841 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.837+0000 7f2313256640 1 -- 172.21.15.125:0/1359812960 shutdown_connections 2023-10-18T10:24:56.841 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.837+0000 7f2313256640 1 -- 172.21.15.125:0/1359812960 wait complete. 2023-10-18T10:24:56.844 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.839+0000 7f2313256640 1 Processor -- start 2023-10-18T10:24:56.844 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.839+0000 7f2313256640 1 -- start start 2023-10-18T10:24:56.844 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2313256640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f230c06cf80 0x7f230c13f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:56.844 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2313256640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f230c1519d0 0x7f230c0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:56.844 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2313256640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c0cbbf0 0x7f230c149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:56.845 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2313256640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c0cc0a0 con 0x7f230c0cbbf0 2023-10-18T10:24:56.845 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2313256640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c14a660 con 0x7f230c1519d0 2023-10-18T10:24:56.845 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2313256640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c14a380 con 0x7f230c06cf80 2023-10-18T10:24:56.845 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c0cbbf0 0x7f230c149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:56.845 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c0cbbf0 0x7f230c149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37988/0 (socket says 172.21.15.125:37988) 2023-10-18T10:24:56.846 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 -- 172.21.15.125:0/66421874 learned_addr learned my addr 172.21.15.125:0/66421874 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:56.846 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2310ff2640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f230c06cf80 0x7f230c13f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:56.846 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f230bfff640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f230c1519d0 0x7f230c0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:56.846 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 -- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f230c06cf80 msgr2=0x7f230c13f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:56.846 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f230c06cf80 0x7f230c13f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:56.847 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 -- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f230c1519d0 msgr2=0x7f230c0cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:56.847 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f230c1519d0 0x7f230c0cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:56.847 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 -- 172.21.15.125:0/66421874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009560 con 0x7f230c0cbbf0 2023-10-18T10:24:56.847 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23117f3640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c0cbbf0 0x7f230c149700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f230400c7a0 tx=0x7f230400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:56.847 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23097fa640 1 -- 172.21.15.125:0/66421874 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304003b70 con 0x7f230c0cbbf0 2023-10-18T10:24:56.848 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23097fa640 1 -- 172.21.15.125:0/66421874 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2304003cf0 con 0x7f230c0cbbf0 2023-10-18T10:24:56.848 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c1432e0 con 0x7f230c0cbbf0 2023-10-18T10:24:56.848 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f23097fa640 1 -- 172.21.15.125:0/66421874 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400aab0 con 0x7f230c0cbbf0 2023-10-18T10:24:56.848 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.840+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c1438a0 con 0x7f230c0cbbf0 2023-10-18T10:24:56.848 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.841+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c0b1380 con 0x7f230c0cbbf0 2023-10-18T10:24:56.849 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.842+0000 7f23097fa640 1 -- 172.21.15.125:0/66421874 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f230400ac10 con 0x7f230c0cbbf0 2023-10-18T10:24:56.849 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.842+0000 7f23097fa640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22f006ae40 0x7f22f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:56.849 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.842+0000 7f23097fa640 1 -- 172.21.15.125:0/66421874 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2304089160 con 0x7f230c0cbbf0 2023-10-18T10:24:56.849 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.842+0000 7f2310ff2640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22f006ae40 0x7f22f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:56.850 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.843+0000 7f2310ff2640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22f006ae40 0x7f22f006d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f230000a490 tx=0x7f230000a420 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:56.850 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:56.847+0000 7f23097fa640 1 -- 172.21.15.125:0/66421874 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2304054610 con 0x7f230c0cbbf0 2023-10-18T10:24:57.019 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.016+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f230c06bf20 con 0x7f230c0cbbf0 2023-10-18T10:24:57.019 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.017+0000 7f23097fa640 1 -- 172.21.15.125:0/66421874 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12898 (secure 0 0 0) 0x7f2304057c30 con 0x7f230c0cbbf0 2023-10-18T10:24:57.020 INFO:teuthology.misc.health.smithi125.stdout: 2023-10-18T10:24:57.020 INFO:teuthology.misc.health.smithi125.stdout:{"epoch":17,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:24:54.628448+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:24:57.021 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22f006ae40 msgr2=0x7f22f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.022 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22f006ae40 0x7f22f006d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f230000a490 tx=0x7f230000a420 comp rx=0 tx=0).stop 2023-10-18T10:24:57.022 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c0cbbf0 msgr2=0x7f230c149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.022 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c0cbbf0 0x7f230c149700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f230400c7a0 tx=0x7f230400cc60 comp rx=0 tx=0).stop 2023-10-18T10:24:57.022 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 shutdown_connections 2023-10-18T10:24:57.022 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22f006ae40 0x7f22f006d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.023 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f230c1519d0 0x7f230c0cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.023 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f230c0cbbf0 0x7f230c149700 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.023 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 --2- 172.21.15.125:0/66421874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f230c06cf80 0x7f230c13f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.023 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 >> 172.21.15.125:0/66421874 conn(0x7f230c069930 msgr2=0x7f230c0ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:57.023 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 shutdown_connections 2023-10-18T10:24:57.024 INFO:teuthology.misc.health.smithi125.stderr:2023-10-18T10:24:57.019+0000 7f2313256640 1 -- 172.21.15.125:0/66421874 wait complete. 2023-10-18T10:24:57.030 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-18T10:24:57.031 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-18T10:24:57.032 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:24:57.032 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:24:57.035 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:24:57.035 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:24:57.035 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:24:57.038 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:24:57.038 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:24:57.038 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:24:57.041 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:24:57.041 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:24:57.042 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:24:57.044 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:24:57.045 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-18T10:24:57.045 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:24:57.109 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:24:57.110 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:24:57.124 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:24:57.125 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:24:57.125 INFO:tasks.ceph.mds.b.smithi139.stderr:2023-10-18T10:24:57.124+0000 7fb667f89640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-18T10:24:57.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.151+0000 7f76bfe32640 1 -- 172.21.15.125:0/12744045 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80ceb40 msgr2=0x7f76b80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.151+0000 7f76bfe32640 1 --2- 172.21.15.125:0/12744045 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80ceb40 0x7f76b80b0650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f76b808a810 tx=0x7f76b002c260 comp rx=0 tx=0).stop 2023-10-18T10:24:57.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.151+0000 7f76bfe32640 1 -- 172.21.15.125:0/12744045 shutdown_connections 2023-10-18T10:24:57.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.151+0000 7f76bfe32640 1 --2- 172.21.15.125:0/12744045 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80ceb40 0x7f76b80b0650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.151+0000 7f76bfe32640 1 --2- 172.21.15.125:0/12744045 >> v2:172.21.15.139:3300/0 conn(0x7f76b80cf340 0x7f76b80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.151+0000 7f76bfe32640 1 -- 172.21.15.125:0/12744045 >> 172.21.15.125:0/12744045 conn(0x7f76b80d0d60 msgr2=0x7f76b80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:57.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.151+0000 7f76bfe32640 1 -- 172.21.15.125:0/12744045 shutdown_connections 2023-10-18T10:24:57.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.151+0000 7f76bfe32640 1 -- 172.21.15.125:0/12744045 wait complete. 2023-10-18T10:24:57.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.152+0000 7f76bfe32640 1 Processor -- start 2023-10-18T10:24:57.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.152+0000 7f76bfe32640 1 -- start start 2023-10-18T10:24:57.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bfe32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76b80ceb40 0x7f76b815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bfe32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80cf340 0x7f76b8158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bfe32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76b815b2a0 0x7f76b80c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bfe32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b809ab70 con 0x7f76b80cf340 2023-10-18T10:24:57.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bfe32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b815d620 con 0x7f76b80ceb40 2023-10-18T10:24:57.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bfe32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80af2d0 con 0x7f76b815b2a0 2023-10-18T10:24:57.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80cf340 0x7f76b8158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80cf340 0x7f76b8158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38038/0 (socket says 172.21.15.125:38038) 2023-10-18T10:24:57.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 -- 172.21.15.125:0/3591107585 learned_addr learned my addr 172.21.15.125:0/3591107585 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:57.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bdbce640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76b80ceb40 0x7f76b815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76be3cf640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76b815b2a0 0x7f76b80c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 -- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76b815b2a0 msgr2=0x7f76b80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76b815b2a0 0x7f76b80c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 -- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76b80ceb40 msgr2=0x7f76b815d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76b80ceb40 0x7f76b815d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 -- 172.21.15.125:0/3591107585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0009900 con 0x7f76b80cf340 2023-10-18T10:24:57.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bdbce640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76b80ceb40 0x7f76b815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:24:57.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.153+0000 7f76bd3cd640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80cf340 0x7f76b8158140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00c3c0 tx=0x7f76ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:57.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.154+0000 7f76aa7fc640 1 -- 172.21.15.125:0/3591107585 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac0052d0 con 0x7f76b80cf340 2023-10-18T10:24:57.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.154+0000 7f76aa7fc640 1 -- 172.21.15.125:0/3591107585 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76ac007020 con 0x7f76b80cf340 2023-10-18T10:24:57.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.154+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b8158680 con 0x7f76b80cf340 2023-10-18T10:24:57.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.154+0000 7f76aa7fc640 1 -- 172.21.15.125:0/3591107585 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac007470 con 0x7f76b80cf340 2023-10-18T10:24:57.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.154+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b80af800 con 0x7f76b80cf340 2023-10-18T10:24:57.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.155+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b80690e0 con 0x7f76b80cf340 2023-10-18T10:24:57.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.155+0000 7f76aa7fc640 1 -- 172.21.15.125:0/3591107585 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f76ac0067d0 con 0x7f76b80cf340 2023-10-18T10:24:57.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.155+0000 7f76aa7fc640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f769c06ae40 0x7f769c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.156+0000 7f76aa7fc640 1 -- 172.21.15.125:0/3591107585 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76ac088970 con 0x7f76b80cf340 2023-10-18T10:24:57.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.156+0000 7f76bdbce640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f769c06ae40 0x7f769c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.156+0000 7f76bdbce640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f769c06ae40 0x7f769c06d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009b30 tx=0x7f76b4009530 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:57.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.160+0000 7f76aa7fc640 1 -- 172.21.15.125:0/3591107585 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76ac053ce0 con 0x7f76b80cf340 2023-10-18T10:24:57.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.325+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76b80aedd0 con 0x7f76b80cf340 2023-10-18T10:24:57.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.326+0000 7f76aa7fc640 1 -- 172.21.15.125:0/3591107585 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12898 (secure 0 0 0) 0x7f76ac057300 con 0x7f76b80cf340 2023-10-18T10:24:57.328 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:57.328 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":17,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:24:54.628448+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:24:57.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f769c06ae40 msgr2=0x7f769c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f769c06ae40 0x7f769c06d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009b30 tx=0x7f76b4009530 comp rx=0 tx=0).stop 2023-10-18T10:24:57.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80cf340 msgr2=0x7f76b8158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80cf340 0x7f76b8158140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00c3c0 tx=0x7f76ac00c880 comp rx=0 tx=0).stop 2023-10-18T10:24:57.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 shutdown_connections 2023-10-18T10:24:57.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f769c06ae40 0x7f769c06d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76b815b2a0 0x7f76b80c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76b80cf340 0x7f76b8158140 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 --2- 172.21.15.125:0/3591107585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76b80ceb40 0x7f76b815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 >> 172.21.15.125:0/3591107585 conn(0x7f76b80d0d60 msgr2=0x7f76b815a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:57.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 shutdown_connections 2023-10-18T10:24:57.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.328+0000 7f76bfe32640 1 -- 172.21.15.125:0/3591107585 wait complete. 2023-10-18T10:24:57.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:24:57.341 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:24:57.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.445+0000 7ff2baf39640 1 -- 172.21.15.125:0/3543000588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b40ab5b0 msgr2=0x7ff2b41595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.445+0000 7ff2baf39640 1 --2- 172.21.15.125:0/3543000588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b40ab5b0 0x7ff2b41595f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009be0 tx=0x7ff2ac02c020 comp rx=0 tx=0).stop 2023-10-18T10:24:57.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.445+0000 7ff2baf39640 1 -- 172.21.15.125:0/3543000588 shutdown_connections 2023-10-18T10:24:57.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.445+0000 7ff2baf39640 1 --2- 172.21.15.125:0/3543000588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b40ab5b0 0x7ff2b41595f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.445+0000 7ff2baf39640 1 -- 172.21.15.125:0/3543000588 >> 172.21.15.125:0/3543000588 conn(0x7ff2b40ae9c0 msgr2=0x7ff2b40aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:57.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.445+0000 7ff2baf39640 1 -- 172.21.15.125:0/3543000588 shutdown_connections 2023-10-18T10:24:57.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.445+0000 7ff2baf39640 1 -- 172.21.15.125:0/3543000588 wait complete. 2023-10-18T10:24:57.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.446+0000 7ff2baf39640 1 Processor -- start 2023-10-18T10:24:57.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.446+0000 7ff2baf39640 1 -- start start 2023-10-18T10:24:57.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2baf39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2b40ab5b0 0x7ff2b41f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2baf39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b4152180 0x7ff2b41f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2baf39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b4159170 0x7ff2b40c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2baf39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b41f1890 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2baf39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40c8e70 con 0x7ff2b4159170 2023-10-18T10:24:57.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2baf39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40c88b0 con 0x7ff2b4152180 2023-10-18T10:24:57.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2b8cd5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2b40ab5b0 0x7ff2b41f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2b8cd5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2b40ab5b0 0x7ff2b41f1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52016/0 (socket says 172.21.15.125:52016) 2023-10-18T10:24:57.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.447+0000 7ff2b8cd5640 1 -- 172.21.15.125:0/1644260185 learned_addr learned my addr 172.21.15.125:0/1644260185 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:57.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b94d6640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b4159170 0x7ff2b40c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b3fff640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b4152180 0x7ff2b41f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b8cd5640 1 -- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b4152180 msgr2=0x7ff2b41f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b8cd5640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b4152180 0x7ff2b41f0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b8cd5640 1 -- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b4159170 msgr2=0x7ff2b40c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b8cd5640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b4159170 0x7ff2b40c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b8cd5640 1 -- 172.21.15.125:0/1644260185 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac009840 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b94d6640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b4159170 0x7ff2b40c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:24:57.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b3fff640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b4152180 0x7ff2b41f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:24:57.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b8cd5640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2b40ab5b0 0x7ff2b41f1cb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c490 tx=0x7ff2a400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:57.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b17fa640 1 -- 172.21.15.125:0/1644260185 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40052d0 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b17fa640 1 -- 172.21.15.125:0/1644260185 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff2a4005430 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b40c8b90 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2b17fa640 1 -- 172.21.15.125:0/1644260185 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4012890 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.448+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b41f21f0 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.449+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b4066fa0 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.449+0000 7ff2b17fa640 1 -- 172.21.15.125:0/1644260185 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff2a40129f0 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.450+0000 7ff2b17fa640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2a006ae40 0x7ff2a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.450+0000 7ff2b17fa640 1 -- 172.21.15.125:0/1644260185 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff2a4089000 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.450+0000 7ff2b3fff640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2a006ae40 0x7ff2a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.450+0000 7ff2b3fff640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2a006ae40 0x7ff2a006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40ab430 tx=0x7ff2a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:57.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.454+0000 7ff2b17fa640 1 -- 172.21.15.125:0/1644260185 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2a4054400 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.619+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2b4159890 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.619+0000 7ff2b17fa640 1 -- 172.21.15.125:0/1644260185 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2a4057a20 con 0x7ff2b40ab5b0 2023-10-18T10:24:57.621 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:24:57.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2a006ae40 msgr2=0x7ff2a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2a006ae40 0x7ff2a006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40ab430 tx=0x7ff2a8009040 comp rx=0 tx=0).stop 2023-10-18T10:24:57.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2b40ab5b0 msgr2=0x7ff2b41f1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2b40ab5b0 0x7ff2b41f1cb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c490 tx=0x7ff2a400c950 comp rx=0 tx=0).stop 2023-10-18T10:24:57.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 shutdown_connections 2023-10-18T10:24:57.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2a006ae40 0x7ff2a006d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b4159170 0x7ff2b40c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b4152180 0x7ff2b41f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 --2- 172.21.15.125:0/1644260185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2b40ab5b0 0x7ff2b41f1cb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 >> 172.21.15.125:0/1644260185 conn(0x7ff2b40ae9c0 msgr2=0x7ff2b415acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:57.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 shutdown_connections 2023-10-18T10:24:57.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.621+0000 7ff2baf39640 1 -- 172.21.15.125:0/1644260185 wait complete. 2023-10-18T10:24:57.633 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:24:57.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.734+0000 7f77dcba9640 1 -- 172.21.15.125:0/997667895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c4004d00 msgr2=0x7f77c4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.734+0000 7f77dcba9640 1 --2- 172.21.15.125:0/997667895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c4004d00 0x7f77c4005180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009f50 tx=0x7f77c40057f0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.734+0000 7f77d6d9d640 1 -- 172.21.15.125:0/997667895 reap_dead start 2023-10-18T10:24:57.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.734+0000 7f77dcba9640 1 -- 172.21.15.125:0/997667895 shutdown_connections 2023-10-18T10:24:57.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.734+0000 7f77dcba9640 1 -- 172.21.15.125:0/997667895 >> 172.21.15.125:0/997667895 conn(0x7f77d806fae0 msgr2=0x7f77d806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:57.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.734+0000 7f77dcba9640 1 -- 172.21.15.125:0/997667895 shutdown_connections 2023-10-18T10:24:57.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.734+0000 7f77dcba9640 1 -- 172.21.15.125:0/997667895 wait complete. 2023-10-18T10:24:57.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.735+0000 7f77dcba9640 1 Processor -- start 2023-10-18T10:24:57.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.735+0000 7f77dcba9640 1 -- start start 2023-10-18T10:24:57.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77dcba9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d8148080 0x7f77d81e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77dcba9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77d81e3f00 0x7f77d81546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77dcba9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d8148460 0x7f77d8154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77dcba9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81e5020 con 0x7f77d8148460 2023-10-18T10:24:57.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77dcba9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81e43a0 con 0x7f77d8148080 2023-10-18T10:24:57.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77dcba9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81e4680 con 0x7f77d81e3f00 2023-10-18T10:24:57.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d8148460 0x7f77d8154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d8148460 0x7f77d8154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52040/0 (socket says 172.21.15.125:52040) 2023-10-18T10:24:57.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 -- 172.21.15.125:0/3150549574 learned_addr learned my addr 172.21.15.125:0/3150549574 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:24:57.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d659c640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d8148080 0x7f77d81e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 -- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77d81e3f00 msgr2=0x7f77d81546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77d81e3f00 0x7f77d81546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 -- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d8148080 msgr2=0x7f77d81e39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d8148080 0x7f77d81e39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 -- 172.21.15.125:0/3150549574 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80b2330 con 0x7f77d8148460 2023-10-18T10:24:57.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.736+0000 7f77d6d9d640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d8148460 0x7f77d8154be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77d0038040 tx=0x7f77d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:57.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.737+0000 7f77ceffd640 1 -- 172.21.15.125:0/3150549574 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0004570 con 0x7f77d8148460 2023-10-18T10:24:57.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.737+0000 7f77ceffd640 1 -- 172.21.15.125:0/3150549574 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f77d00047c0 con 0x7f77d8148460 2023-10-18T10:24:57.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.737+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d81e4960 con 0x7f77d8148460 2023-10-18T10:24:57.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.737+0000 7f77ceffd640 1 -- 172.21.15.125:0/3150549574 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0004d80 con 0x7f77d8148460 2023-10-18T10:24:57.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.737+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d81e5440 con 0x7f77d8148460 2023-10-18T10:24:57.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.737+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d81e5160 con 0x7f77d8148460 2023-10-18T10:24:57.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.738+0000 7f77ceffd640 1 -- 172.21.15.125:0/3150549574 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f77d003c070 con 0x7f77d8148460 2023-10-18T10:24:57.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.738+0000 7f77ceffd640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f77b806ae40 0x7f77b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:24:57.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.739+0000 7f77ceffd640 1 -- 172.21.15.125:0/3150549574 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f77d002ce40 con 0x7f77d8148460 2023-10-18T10:24:57.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.739+0000 7f77d659c640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f77b806ae40 0x7f77b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:24:57.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.739+0000 7f77d659c640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f77b806ae40 0x7f77b806d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77c8003670 tx=0x7f77c80067a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:24:57.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.743+0000 7f77ceffd640 1 -- 172.21.15.125:0/3150549574 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77d81e5160 con 0x7f77d8148460 2023-10-18T10:24:57.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.944+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77d81e5160 con 0x7f77d8148460 2023-10-18T10:24:57.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.945+0000 7f77ceffd640 1 -- 172.21.15.125:0/3150549574 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f77d81e5160 con 0x7f77d8148460 2023-10-18T10:24:57.947 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:24:57.947 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4401,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-18T10:24:57.949 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 2 2023-10-18T10:24:57.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.946+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f77b806ae40 msgr2=0x7f77b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.946+0000 7f77dcba9640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f77b806ae40 0x7f77b806d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77c8003670 tx=0x7f77c80067a0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d8148460 msgr2=0x7f77d8154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:24:57.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d8148460 0x7f77d8154be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77d0038040 tx=0x7f77d0004270 comp rx=0 tx=0).stop 2023-10-18T10:24:57.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 shutdown_connections 2023-10-18T10:24:57.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f77b806ae40 0x7f77b806d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d8148460 0x7f77d8154be0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77d81e3f00 0x7f77d81546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 --2- 172.21.15.125:0/3150549574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d8148080 0x7f77d81e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:24:57.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 >> 172.21.15.125:0/3150549574 conn(0x7f77d806fae0 msgr2=0x7f77d80b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:24:57.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 shutdown_connections 2023-10-18T10:24:57.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:24:57.947+0000 7f77dcba9640 1 -- 172.21.15.125:0/3150549574 wait complete. 2023-10-18T10:24:59.960 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:25:00.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.079+0000 7ff396607640 1 -- 172.21.15.125:0/2578295453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff39015b5e0 msgr2=0x7ff3900cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.079+0000 7ff396607640 1 --2- 172.21.15.125:0/2578295453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff39015b5e0 0x7ff3900cd4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff39008a0c0 tx=0x7ff38402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.079+0000 7ff396607640 1 -- 172.21.15.125:0/2578295453 shutdown_connections 2023-10-18T10:25:00.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.079+0000 7ff396607640 1 --2- 172.21.15.125:0/2578295453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff39015b5e0 0x7ff3900cd4a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.079+0000 7ff396607640 1 --2- 172.21.15.125:0/2578295453 >> v2:172.21.15.139:3300/0 conn(0x7ff39015b200 0x7ff3900b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.079+0000 7ff396607640 1 -- 172.21.15.125:0/2578295453 >> 172.21.15.125:0/2578295453 conn(0x7ff3900d0cc0 msgr2=0x7ff3900d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:00.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.079+0000 7ff396607640 1 -- 172.21.15.125:0/2578295453 shutdown_connections 2023-10-18T10:25:00.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.079+0000 7ff396607640 1 -- 172.21.15.125:0/2578295453 wait complete. 2023-10-18T10:25:00.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.081+0000 7ff396607640 1 Processor -- start 2023-10-18T10:25:00.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.081+0000 7ff396607640 1 -- start start 2023-10-18T10:25:00.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff396607640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900b1e10 0x7ff3900b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff396607640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff39015b200 0x7ff3900b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff396607640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff39015b5e0 0x7ff39015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff396607640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390146980 con 0x7ff3900b1e10 2023-10-18T10:25:00.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff396607640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900b6630 con 0x7ff39015b5e0 2023-10-18T10:25:00.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff396607640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900b5a20 con 0x7ff39015b200 2023-10-18T10:25:00.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900b1e10 0x7ff3900b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900b1e10 0x7ff3900b4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52054/0 (socket says 172.21.15.125:52054) 2023-10-18T10:25:00.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 -- 172.21.15.125:0/2090379022 learned_addr learned my addr 172.21.15.125:0/2090379022 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:00.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38f7fe640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff39015b200 0x7ff3900b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff394ba4640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff39015b5e0 0x7ff39015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 -- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff39015b200 msgr2=0x7ff3900b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff39015b200 0x7ff3900b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 -- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff39015b5e0 msgr2=0x7ff39015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff39015b5e0 0x7ff39015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 -- 172.21.15.125:0/2090379022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff384009490 con 0x7ff3900b1e10 2023-10-18T10:25:00.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38f7fe640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff39015b200 0x7ff3900b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:00.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38ffff640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900b1e10 0x7ff3900b4ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e9f0 tx=0x7ff38000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:00.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38cff9640 1 -- 172.21.15.125:0/2090379022 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380009700 con 0x7ff3900b1e10 2023-10-18T10:25:00.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38cff9640 1 -- 172.21.15.125:0/2090379022 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff38000ce70 con 0x7ff3900b1e10 2023-10-18T10:25:00.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3900b5d00 con 0x7ff3900b1e10 2023-10-18T10:25:00.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff38cff9640 1 -- 172.21.15.125:0/2090379022 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380010670 con 0x7ff3900b1e10 2023-10-18T10:25:00.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.082+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3900b62c0 con 0x7ff3900b1e10 2023-10-18T10:25:00.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.083+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3900b6450 con 0x7ff3900b1e10 2023-10-18T10:25:00.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.084+0000 7ff38cff9640 1 -- 172.21.15.125:0/2090379022 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff3800107d0 con 0x7ff3900b1e10 2023-10-18T10:25:00.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.084+0000 7ff38cff9640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff37c06ae40 0x7ff37c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.084+0000 7ff38cff9640 1 -- 172.21.15.125:0/2090379022 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff380014070 con 0x7ff3900b1e10 2023-10-18T10:25:00.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.084+0000 7ff38f7fe640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff37c06ae40 0x7ff37c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.085+0000 7ff38f7fe640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff37c06ae40 0x7ff37c06d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff384036000 tx=0x7ff384004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:00.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.089+0000 7ff38cff9640 1 -- 172.21.15.125:0/2090379022 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff380051ca0 con 0x7ff3900b1e10 2023-10-18T10:25:00.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.290+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff390066930 con 0x7ff3900b1e10 2023-10-18T10:25:00.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.291+0000 7ff38cff9640 1 -- 172.21.15.125:0/2090379022 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7ff380058e40 con 0x7ff3900b1e10 2023-10-18T10:25:00.293 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:00.293 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4401,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-18T10:25:00.294 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 3 2023-10-18T10:25:00.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff37c06ae40 msgr2=0x7ff37c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff37c06ae40 0x7ff37c06d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff384036000 tx=0x7ff384004730 comp rx=0 tx=0).stop 2023-10-18T10:25:00.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900b1e10 msgr2=0x7ff3900b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900b1e10 0x7ff3900b4ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e9f0 tx=0x7ff38000eeb0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 shutdown_connections 2023-10-18T10:25:00.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff37c06ae40 0x7ff37c06d300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff39015b5e0 0x7ff39015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff39015b200 0x7ff3900b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 --2- 172.21.15.125:0/2090379022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900b1e10 0x7ff3900b4ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 >> 172.21.15.125:0/2090379022 conn(0x7ff3900d0cc0 msgr2=0x7ff3900ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:00.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 shutdown_connections 2023-10-18T10:25:00.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.293+0000 7ff396607640 1 -- 172.21.15.125:0/2090379022 wait complete. 2023-10-18T10:25:00.305 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:00.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.406+0000 7fab46c44640 1 -- 172.21.15.125:0/4087368437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab30005180 msgr2=0x7fab3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.406+0000 7fab46c44640 1 --2- 172.21.15.125:0/4087368437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab30005180 0x7fab3000b980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab40067b80 tx=0x7fab300057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.406+0000 7fab46c44640 1 -- 172.21.15.125:0/4087368437 shutdown_connections 2023-10-18T10:25:00.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.406+0000 7fab46c44640 1 --2- 172.21.15.125:0/4087368437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab30005180 0x7fab3000b980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.406+0000 7fab46c44640 1 -- 172.21.15.125:0/4087368437 >> 172.21.15.125:0/4087368437 conn(0x7fab4006fae0 msgr2=0x7fab4006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:00.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.406+0000 7fab46c44640 1 -- 172.21.15.125:0/4087368437 shutdown_connections 2023-10-18T10:25:00.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.406+0000 7fab46c44640 1 -- 172.21.15.125:0/4087368437 wait complete. 2023-10-18T10:25:00.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.407+0000 7fab46c44640 1 Processor -- start 2023-10-18T10:25:00.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.407+0000 7fab46c44640 1 -- start start 2023-10-18T10:25:00.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab46c44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab30005180 0x7fab40147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab46c44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab40147210 0x7fab4014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab46c44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab4014b3b0 0x7fab4014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab46c44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4014aa50 con 0x7fab4014b3b0 2023-10-18T10:25:00.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab46c44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40147d10 con 0x7fab40147210 2023-10-18T10:25:00.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab46c44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab40149e80 con 0x7fab30005180 2023-10-18T10:25:00.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab4014b3b0 0x7fab4014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab4014b3b0 0x7fab4014f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52062/0 (socket says 172.21.15.125:52062) 2023-10-18T10:25:00.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 -- 172.21.15.125:0/1603682988 learned_addr learned my addr 172.21.15.125:0/1603682988 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:00.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab3ffff640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab40147210 0x7fab4014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab449e0640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab30005180 0x7fab40147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 -- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab30005180 msgr2=0x7fab40147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab30005180 0x7fab40147f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 -- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab40147210 msgr2=0x7fab4014ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab40147210 0x7fab4014ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 -- 172.21.15.125:0/1603682988 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400b2330 con 0x7fab4014b3b0 2023-10-18T10:25:00.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab451e1640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab4014b3b0 0x7fab4014f440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab38002230 tx=0x7fab380019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:00.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.408+0000 7fab3d7fa640 1 -- 172.21.15.125:0/1603682988 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3803e070 con 0x7fab4014b3b0 2023-10-18T10:25:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.409+0000 7fab3d7fa640 1 -- 172.21.15.125:0/1603682988 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fab38001250 con 0x7fab4014b3b0 2023-10-18T10:25:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.409+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab40155460 con 0x7fab4014b3b0 2023-10-18T10:25:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.409+0000 7fab3d7fa640 1 -- 172.21.15.125:0/1603682988 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38003de0 con 0x7fab4014b3b0 2023-10-18T10:25:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.409+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4014a390 con 0x7fab4014b3b0 2023-10-18T10:25:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.409+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4014ab90 con 0x7fab4014b3b0 2023-10-18T10:25:00.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.410+0000 7fab3d7fa640 1 -- 172.21.15.125:0/1603682988 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fab38035020 con 0x7fab4014b3b0 2023-10-18T10:25:00.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.410+0000 7fab3d7fa640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.410+0000 7fab3d7fa640 1 -- 172.21.15.125:0/1603682988 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fab380adea0 con 0x7fab4014b3b0 2023-10-18T10:25:00.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.411+0000 7fab449e0640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.411+0000 7fab449e0640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab2406ae40 0x7fab2406d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab34009a10 tx=0x7fab34005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:00.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.414+0000 7fab3d7fa640 1 -- 172.21.15.125:0/1603682988 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab4014ab90 con 0x7fab4014b3b0 2023-10-18T10:25:00.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.581+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab4014ab90 con 0x7fab4014b3b0 2023-10-18T10:25:00.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.581+0000 7fab3d7fa640 1 -- 172.21.15.125:0/1603682988 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12898 (secure 0 0 0) 0x7fab4014ab90 con 0x7fab4014b3b0 2023-10-18T10:25:00.583 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:00.584 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":17,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:24:54.628448+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:00.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab2406ae40 msgr2=0x7fab2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab2406ae40 0x7fab2406d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab34009a10 tx=0x7fab34005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:00.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab4014b3b0 msgr2=0x7fab4014f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab4014b3b0 0x7fab4014f440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab38002230 tx=0x7fab380019e0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 shutdown_connections 2023-10-18T10:25:00.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab40147210 0x7fab4014ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab4014b3b0 0x7fab4014f440 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 --2- 172.21.15.125:0/1603682988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab30005180 0x7fab40147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 >> 172.21.15.125:0/1603682988 conn(0x7fab4006fae0 msgr2=0x7fab401536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:00.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 shutdown_connections 2023-10-18T10:25:00.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.583+0000 7fab46c44640 1 -- 172.21.15.125:0/1603682988 wait complete. 2023-10-18T10:25:00.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:25:00.596 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:00.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.700+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/1655468661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 msgr2=0x7f8a24003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.700+0000 7f8a3d3b6640 1 --2- 172.21.15.125:0/1655468661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 0x7f8a24003d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c260 tx=0x7f8a240057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.700+0000 7f8a377fe640 1 -- 172.21.15.125:0/1655468661 reap_dead start 2023-10-18T10:25:00.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.700+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/1655468661 shutdown_connections 2023-10-18T10:25:00.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.700+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/1655468661 >> 172.21.15.125:0/1655468661 conn(0x7f8a3808a650 msgr2=0x7f8a38099f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:00.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.700+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/1655468661 shutdown_connections 2023-10-18T10:25:00.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.700+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/1655468661 wait complete. 2023-10-18T10:25:00.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.701+0000 7f8a3d3b6640 1 Processor -- start 2023-10-18T10:25:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.701+0000 7f8a3d3b6640 1 -- start start 2023-10-18T10:25:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a3d3b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 0x7f8a380cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a3d3b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a380ccea0 0x7f8a380ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a3d3b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a381f2190 0x7f8a38150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a3d3b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a380ce1e0 con 0x7f8a24003970 2023-10-18T10:25:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a3d3b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a380cc0e0 con 0x7f8a381f2190 2023-10-18T10:25:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a3d3b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a380cc3c0 con 0x7f8a380ccea0 2023-10-18T10:25:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 0x7f8a380cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 0x7f8a380cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52076/0 (socket says 172.21.15.125:52076) 2023-10-18T10:25:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 -- 172.21.15.125:0/3103228348 learned_addr learned my addr 172.21.15.125:0/3103228348 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:00.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a377fe640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a381f2190 0x7f8a38150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a2ffff640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a380ccea0 0x7f8a380ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 -- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a380ccea0 msgr2=0x7f8a380ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a380ccea0 0x7f8a380ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 -- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a381f2190 msgr2=0x7f8a38150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a381f2190 0x7f8a38150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 -- 172.21.15.125:0/3103228348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a380cc6a0 con 0x7f8a24003970 2023-10-18T10:25:00.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a377fe640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a381f2190 0x7f8a38150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:00.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a2ffff640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a380ccea0 0x7f8a380ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:00.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.702+0000 7f8a36ffd640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 0x7f8a380cdbe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a28000c00 tx=0x7f8a280015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.703+0000 7f8a2f7fe640 1 -- 172.21.15.125:0/3103228348 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28006310 con 0x7f8a24003970 2023-10-18T10:25:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.703+0000 7f8a2f7fe640 1 -- 172.21.15.125:0/3103228348 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a28006470 con 0x7f8a24003970 2023-10-18T10:25:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.703+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a380cc980 con 0x7f8a24003970 2023-10-18T10:25:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.703+0000 7f8a2f7fe640 1 -- 172.21.15.125:0/3103228348 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28003850 con 0x7f8a24003970 2023-10-18T10:25:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.703+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a38150fb0 con 0x7f8a24003970 2023-10-18T10:25:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.704+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a38066930 con 0x7f8a24003970 2023-10-18T10:25:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.704+0000 7f8a2f7fe640 1 -- 172.21.15.125:0/3103228348 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a28031030 con 0x7f8a24003970 2023-10-18T10:25:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.704+0000 7f8a2f7fe640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1806ae70 0x7f8a1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.704+0000 7f8a2f7fe640 1 -- 172.21.15.125:0/3103228348 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8a280aa050 con 0x7f8a24003970 2023-10-18T10:25:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.704+0000 7f8a2ffff640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1806ae70 0x7f8a1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.705+0000 7f8a2ffff640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1806ae70 0x7f8a1806d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a3809ebb0 tx=0x7f8a20005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:00.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.708+0000 7f8a2f7fe640 1 -- 172.21.15.125:0/3103228348 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a28078f70 con 0x7f8a24003970 2023-10-18T10:25:00.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.872+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a3804dae0 con 0x7f8a24003970 2023-10-18T10:25:00.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.872+0000 7f8a2f7fe640 1 -- 172.21.15.125:0/3103228348 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a280440a0 con 0x7f8a24003970 2023-10-18T10:25:00.874 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:00.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1806ae70 msgr2=0x7f8a1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1806ae70 0x7f8a1806d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a3809ebb0 tx=0x7f8a20005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:00.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 msgr2=0x7f8a380cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 0x7f8a380cdbe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a28000c00 tx=0x7f8a280015f0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 shutdown_connections 2023-10-18T10:25:00.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1806ae70 0x7f8a1806d330 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a3809ebb0 tx=0x7f8a20005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:00.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a381f2190 0x7f8a38150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a380ccea0 0x7f8a380ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 --2- 172.21.15.125:0/3103228348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a24003970 0x7f8a380cdbe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 >> 172.21.15.125:0/3103228348 conn(0x7f8a3808a650 msgr2=0x7f8a380b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:00.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 shutdown_connections 2023-10-18T10:25:00.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.874+0000 7f8a3d3b6640 1 -- 172.21.15.125:0/3103228348 wait complete. 2023-10-18T10:25:00.886 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:25:00.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.987+0000 7f8750b6e640 1 -- 172.21.15.125:0/480839853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87380047c0 msgr2=0x7f8738004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.987+0000 7f8750b6e640 1 --2- 172.21.15.125:0/480839853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87380047c0 0x7f8738004c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8734006fa0 tx=0x7f87380057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.987+0000 7f874ad9d640 1 -- 172.21.15.125:0/480839853 reap_dead start 2023-10-18T10:25:00.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.987+0000 7f8750b6e640 1 -- 172.21.15.125:0/480839853 shutdown_connections 2023-10-18T10:25:00.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.987+0000 7f8750b6e640 1 -- 172.21.15.125:0/480839853 >> 172.21.15.125:0/480839853 conn(0x7f874c069930 msgr2=0x7f874c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:00.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.987+0000 7f8750b6e640 1 -- 172.21.15.125:0/480839853 shutdown_connections 2023-10-18T10:25:00.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.987+0000 7f8750b6e640 1 -- 172.21.15.125:0/480839853 wait complete. 2023-10-18T10:25:00.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.988+0000 7f8750b6e640 1 Processor -- start 2023-10-18T10:25:00.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.988+0000 7f8750b6e640 1 -- start start 2023-10-18T10:25:00.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f8750b6e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87380047c0 0x7f874c0a8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f8750b6e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f874c15a0a0 0x7f874c13cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f8750b6e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c13d040 0x7f874c0aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f8750b6e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0a8590 con 0x7f874c13d040 2023-10-18T10:25:00.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f8750b6e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0a7bc0 con 0x7f87380047c0 2023-10-18T10:25:00.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f8750b6e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0a6ff0 con 0x7f874c15a0a0 2023-10-18T10:25:00.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c13d040 0x7f874c0aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c13d040 0x7f874c0aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52102/0 (socket says 172.21.15.125:52102) 2023-10-18T10:25:00.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 -- 172.21.15.125:0/3160672669 learned_addr learned my addr 172.21.15.125:0/3160672669 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:00.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 -- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f874c15a0a0 msgr2=0x7f874c13cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f874c15a0a0 0x7f874c13cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 -- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87380047c0 msgr2=0x7f874c0a8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:00.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874a59c640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87380047c0 0x7f874c0a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87380047c0 0x7f874c0a8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:00.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 -- 172.21.15.125:0/3160672669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c0a7290 con 0x7f874c13d040 2023-10-18T10:25:00.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.989+0000 7f874ad9d640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c13d040 0x7f874c0aff50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f874c0cbac0 tx=0x7f874400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:00.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.990+0000 7f8742ffd640 1 -- 172.21.15.125:0/3160672669 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744015070 con 0x7f874c13d040 2023-10-18T10:25:00.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.990+0000 7f8742ffd640 1 -- 172.21.15.125:0/3160672669 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f874400ced0 con 0x7f874c13d040 2023-10-18T10:25:00.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.990+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c0a7570 con 0x7f874c13d040 2023-10-18T10:25:00.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.990+0000 7f8742ffd640 1 -- 172.21.15.125:0/3160672669 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744004d80 con 0x7f874c13d040 2023-10-18T10:25:00.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.990+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c15a820 con 0x7f874c13d040 2023-10-18T10:25:00.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.991+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c0690e0 con 0x7f874c13d040 2023-10-18T10:25:00.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.991+0000 7f8742ffd640 1 -- 172.21.15.125:0/3160672669 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8744003970 con 0x7f874c13d040 2023-10-18T10:25:00.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.991+0000 7f8742ffd640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f872c06ae40 0x7f872c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:00.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.992+0000 7f8742ffd640 1 -- 172.21.15.125:0/3160672669 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f874408b630 con 0x7f874c13d040 2023-10-18T10:25:00.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.992+0000 7f874a59c640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f872c06ae40 0x7f872c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:00.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.992+0000 7f874a59c640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f872c06ae40 0x7f872c06d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f873c006540 tx=0x7f873c006b30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:00.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:00.996+0000 7f8742ffd640 1 -- 172.21.15.125:0/3160672669 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87440569b0 con 0x7f874c13d040 2023-10-18T10:25:01.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.165+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f874c15ba50 con 0x7f874c13d040 2023-10-18T10:25:01.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.165+0000 7f8742ffd640 1 -- 172.21.15.125:0/3160672669 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8744059fd0 con 0x7f874c13d040 2023-10-18T10:25:01.168 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:01.168 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:25:01.168 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:25:01.168 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:25:01.169 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:25:01.169 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:25:01.169 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:25:01.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f872c06ae40 msgr2=0x7f872c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:01.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f872c06ae40 0x7f872c06d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f873c006540 tx=0x7f873c006b30 comp rx=0 tx=0).stop 2023-10-18T10:25:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c13d040 msgr2=0x7f874c0aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c13d040 0x7f874c0aff50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f874c0cbac0 tx=0x7f874400c6a0 comp rx=0 tx=0).stop 2023-10-18T10:25:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 shutdown_connections 2023-10-18T10:25:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f872c06ae40 0x7f872c06d300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:01.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c13d040 0x7f874c0aff50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:01.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f874c15a0a0 0x7f874c13cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:01.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 --2- 172.21.15.125:0/3160672669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87380047c0 0x7f874c0a8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:01.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 >> 172.21.15.125:0/3160672669 conn(0x7f874c069930 msgr2=0x7f874c0703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:01.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.167+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 shutdown_connections 2023-10-18T10:25:01.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.168+0000 7f8750b6e640 1 -- 172.21.15.125:0/3160672669 wait complete. 2023-10-18T10:25:01.179 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:25:01.180 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:25:01.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.284+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/533926060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f20005460 msgr2=0x7f3f20005930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:01.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.284+0000 7f3f3e0ec640 1 --2- 172.21.15.125:0/533926060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f20005460 0x7f3f20005930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f28006f70 tx=0x7f3f20006900 comp rx=0 tx=0).stop 2023-10-18T10:25:01.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.284+0000 7f3f37fff640 1 -- 172.21.15.125:0/533926060 reap_dead start 2023-10-18T10:25:01.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.284+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/533926060 shutdown_connections 2023-10-18T10:25:01.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.284+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/533926060 >> 172.21.15.125:0/533926060 conn(0x7f3f38069930 msgr2=0x7f3f38069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:01.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.284+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/533926060 shutdown_connections 2023-10-18T10:25:01.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.284+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/533926060 wait complete. 2023-10-18T10:25:01.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.285+0000 7f3f3e0ec640 1 Processor -- start 2023-10-18T10:25:01.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.285+0000 7f3f3e0ec640 1 -- start start 2023-10-18T10:25:01.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f3e0ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f20005460 0x7f3f3813a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:01.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f3e0ec640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f38146e40 0x7f3f381562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:01.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f3e0ec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f38156800 0x7f3f381491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:01.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f3e0ec640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3813b2e0 con 0x7f3f38156800 2023-10-18T10:25:01.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f3e0ec640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3814e9c0 con 0x7f3f20005460 2023-10-18T10:25:01.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f3e0ec640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f38156c90 con 0x7f3f38146e40 2023-10-18T10:25:01.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f38156800 0x7f3f381491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:01.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f38156800 0x7f3f381491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52112/0 (socket says 172.21.15.125:52112) 2023-10-18T10:25:01.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 -- 172.21.15.125:0/3949263932 learned_addr learned my addr 172.21.15.125:0/3949263932 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:01.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f377fe640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f20005460 0x7f3f3813a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:01.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f36ffd640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f38146e40 0x7f3f381562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:01.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 -- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f38146e40 msgr2=0x7f3f381562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:01.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f38146e40 0x7f3f381562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:01.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 -- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f20005460 msgr2=0x7f3f3813a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:01.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f20005460 0x7f3f3813a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:01.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 -- 172.21.15.125:0/3949263932 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f38149df0 con 0x7f3f38156800 2023-10-18T10:25:01.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f37fff640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f38156800 0x7f3f381491c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3f30005fd0 tx=0x7f3f3000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:01.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.286+0000 7f3f1ffff640 1 -- 172.21.15.125:0/3949263932 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30004b20 con 0x7f3f38156800 2023-10-18T10:25:01.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.287+0000 7f3f1ffff640 1 -- 172.21.15.125:0/3949263932 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f30004c80 con 0x7f3f38156800 2023-10-18T10:25:01.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.287+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3813a940 con 0x7f3f38156800 2023-10-18T10:25:01.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.287+0000 7f3f1ffff640 1 -- 172.21.15.125:0/3949263932 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30003bd0 con 0x7f3f38156800 2023-10-18T10:25:01.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.287+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3813af00 con 0x7f3f38156800 2023-10-18T10:25:01.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.288+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3813b420 con 0x7f3f38156800 2023-10-18T10:25:01.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.288+0000 7f3f1ffff640 1 -- 172.21.15.125:0/3949263932 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3f30003d30 con 0x7f3f38156800 2023-10-18T10:25:01.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.288+0000 7f3f1ffff640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3f2406ae40 0x7f3f2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:01.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.288+0000 7f3f1ffff640 1 -- 172.21.15.125:0/3949263932 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3f30089a50 con 0x7f3f38156800 2023-10-18T10:25:01.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.289+0000 7f3f377fe640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3f2406ae40 0x7f3f2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:01.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.289+0000 7f3f377fe640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3f2406ae40 0x7f3f2406d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f380c8d00 tx=0x7f3f2c006b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:01.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.292+0000 7f3f1ffff640 1 -- 172.21.15.125:0/3949263932 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f3813b420 con 0x7f3f38156800 2023-10-18T10:25:01.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:01.458+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3f38149df0 con 0x7f3f38156800 2023-10-18T10:25:01.993 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:25:01.996 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:25:02.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.184+0000 7f3f1ffff640 1 -- 172.21.15.125:0/3949263932 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3f38149df0 con 0x7f3f38156800 2023-10-18T10:25:02.187 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:25:02.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3f2406ae40 msgr2=0x7f3f2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:02.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3f2406ae40 0x7f3f2406d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f380c8d00 tx=0x7f3f2c006b70 comp rx=0 tx=0).stop 2023-10-18T10:25:02.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f38156800 msgr2=0x7f3f381491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:02.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f38156800 0x7f3f381491c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3f30005fd0 tx=0x7f3f3000f3c0 comp rx=0 tx=0).stop 2023-10-18T10:25:02.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 shutdown_connections 2023-10-18T10:25:02.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3f2406ae40 0x7f3f2406d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:02.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f38156800 0x7f3f381491c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:02.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f38146e40 0x7f3f381562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:02.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 --2- 172.21.15.125:0/3949263932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f20005460 0x7f3f3813a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:02.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 >> 172.21.15.125:0/3949263932 conn(0x7f3f38069930 msgr2=0x7f3f3806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:02.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 shutdown_connections 2023-10-18T10:25:02.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.187+0000 7f3f3e0ec640 1 -- 172.21.15.125:0/3949263932 wait complete. 2023-10-18T10:25:02.206 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-18T10:25:02.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.320+0000 7fe9e5a39640 1 -- 172.21.15.125:0/1303175668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9c8004d00 msgr2=0x7fe9c8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:02.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.320+0000 7fe9e5a39640 1 --2- 172.21.15.125:0/1303175668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9c8004d00 0x7fe9c8005180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006f70 tx=0x7fe9c80057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:02.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.320+0000 7fe9df7fe640 1 -- 172.21.15.125:0/1303175668 reap_dead start 2023-10-18T10:25:02.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.320+0000 7fe9e5a39640 1 -- 172.21.15.125:0/1303175668 shutdown_connections 2023-10-18T10:25:02.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.320+0000 7fe9e5a39640 1 -- 172.21.15.125:0/1303175668 >> 172.21.15.125:0/1303175668 conn(0x7fe9e00d0d60 msgr2=0x7fe9e00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:02.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.320+0000 7fe9e5a39640 1 -- 172.21.15.125:0/1303175668 shutdown_connections 2023-10-18T10:25:02.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.320+0000 7fe9e5a39640 1 -- 172.21.15.125:0/1303175668 wait complete. 2023-10-18T10:25:02.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.321+0000 7fe9e5a39640 1 Processor -- start 2023-10-18T10:25:02.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.321+0000 7fe9e5a39640 1 -- start start 2023-10-18T10:25:02.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9e5a39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8004d00 0x7fe9e006b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:02.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9e5a39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9e01e54f0 0x7fe9e01e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:02.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9e5a39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e01560e0 0x7fe9e0156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:02.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9e5a39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00b5480 con 0x7fe9e01e54f0 2023-10-18T10:25:02.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9e5a39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01e5eb0 con 0x7fe9e01560e0 2023-10-18T10:25:02.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9e5a39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00b4580 con 0x7fe9c8004d00 2023-10-18T10:25:02.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9e01e54f0 0x7fe9e01e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:02.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9e01e54f0 0x7fe9e01e5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52130/0 (socket says 172.21.15.125:52130) 2023-10-18T10:25:02.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 -- 172.21.15.125:0/395586049 learned_addr learned my addr 172.21.15.125:0/395586049 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:02.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9deffd640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8004d00 0x7fe9e006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:02.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9df7fe640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e01560e0 0x7fe9e0156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:02.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 -- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8004d00 msgr2=0x7fe9e006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:02.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8004d00 0x7fe9e006b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:02.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 -- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e01560e0 msgr2=0x7fe9e0156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:02.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e01560e0 0x7fe9e0156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:02.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 -- 172.21.15.125:0/395586049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e0156a60 con 0x7fe9e01e54f0 2023-10-18T10:25:02.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9deffd640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8004d00 0x7fe9e006b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:02.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9df7fe640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e01560e0 0x7fe9e0156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:02.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9de7fc640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9e01e54f0 0x7fe9e01e5970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4003ec0 tx=0x7fe9d4001760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:02.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9c7fff640 1 -- 172.21.15.125:0/395586049 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4011900 con 0x7fe9e01e54f0 2023-10-18T10:25:02.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9c7fff640 1 -- 172.21.15.125:0/395586049 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe9d4011a60 con 0x7fe9e01e54f0 2023-10-18T10:25:02.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e00b4800 con 0x7fe9e01e54f0 2023-10-18T10:25:02.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.322+0000 7fe9c7fff640 1 -- 172.21.15.125:0/395586049 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4011e70 con 0x7fe9e01e54f0 2023-10-18T10:25:02.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.323+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e00b4dc0 con 0x7fe9e01e54f0 2023-10-18T10:25:02.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.323+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e00b55c0 con 0x7fe9e01e54f0 2023-10-18T10:25:02.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.324+0000 7fe9c7fff640 1 -- 172.21.15.125:0/395586049 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe9d40103c0 con 0x7fe9e01e54f0 2023-10-18T10:25:02.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.324+0000 7fe9c7fff640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9cc06ae40 0x7fe9cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:02.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.324+0000 7fe9c7fff640 1 -- 172.21.15.125:0/395586049 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe9d408a3e0 con 0x7fe9e01e54f0 2023-10-18T10:25:02.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.324+0000 7fe9deffd640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9cc06ae40 0x7fe9cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:02.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.325+0000 7fe9deffd640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9cc06ae40 0x7fe9cc06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0005fd0 tx=0x7fe9d0009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:02.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.329+0000 7fe9c7fff640 1 -- 172.21.15.125:0/395586049 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9e00b55c0 con 0x7fe9e01e54f0 2023-10-18T10:25:02.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:02.496+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fe9e00b55c0 con 0x7fe9e01e54f0 2023-10-18T10:25:03.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.188+0000 7fe9c7fff640 1 -- 172.21.15.125:0/395586049 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fe9e00b55c0 con 0x7fe9e01e54f0 2023-10-18T10:25:03.191 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:25:03.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9cc06ae40 msgr2=0x7fe9cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:03.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9cc06ae40 0x7fe9cc06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0005fd0 tx=0x7fe9d0009500 comp rx=0 tx=0).stop 2023-10-18T10:25:03.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9e01e54f0 msgr2=0x7fe9e01e5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:03.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9e01e54f0 0x7fe9e01e5970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4003ec0 tx=0x7fe9d4001760 comp rx=0 tx=0).stop 2023-10-18T10:25:03.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 shutdown_connections 2023-10-18T10:25:03.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9cc06ae40 0x7fe9cc06d300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:03.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9e01560e0 0x7fe9e0156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:03.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9e01e54f0 0x7fe9e01e5970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:03.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 --2- 172.21.15.125:0/395586049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8004d00 0x7fe9e006b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:03.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 >> 172.21.15.125:0/395586049 conn(0x7fe9e00d0d60 msgr2=0x7fe9e00b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:03.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 shutdown_connections 2023-10-18T10:25:03.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.191+0000 7fe9e5a39640 1 -- 172.21.15.125:0/395586049 wait complete. 2023-10-18T10:25:03.216 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:25:03.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.323+0000 7f92c8061640 1 -- 172.21.15.125:0/2079173637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c00ac720 msgr2=0x7f92c015a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.323+0000 7f92c8061640 1 --2- 172.21.15.125:0/2079173637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c00ac720 0x7f92c015a420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92c0067b40 tx=0x7f92bc02bd30 comp rx=0 tx=0).stop 2023-10-18T10:25:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.323+0000 7f92c8061640 1 -- 172.21.15.125:0/2079173637 shutdown_connections 2023-10-18T10:25:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.323+0000 7f92c8061640 1 --2- 172.21.15.125:0/2079173637 >> v2:172.21.15.139:3300/0 conn(0x7f92c0159530 0x7f92c00acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.323+0000 7f92c8061640 1 --2- 172.21.15.125:0/2079173637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c00ac720 0x7f92c015a420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:03.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.323+0000 7f92c8061640 1 -- 172.21.15.125:0/2079173637 >> 172.21.15.125:0/2079173637 conn(0x7f92c00cd940 msgr2=0x7f92c00cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:03.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.323+0000 7f92c8061640 1 -- 172.21.15.125:0/2079173637 shutdown_connections 2023-10-18T10:25:03.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.324+0000 7f92c8061640 1 -- 172.21.15.125:0/2079173637 wait complete. 2023-10-18T10:25:03.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c8061640 1 Processor -- start 2023-10-18T10:25:03.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c8061640 1 -- start start 2023-10-18T10:25:03.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c8061640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92c00ac720 0x7f92c01462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:03.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c8061640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c0157de0 0x7f92c014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:03.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c8061640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92c0159530 0x7f92c01552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:03.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c8061640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c01f15c0 con 0x7f92c0157de0 2023-10-18T10:25:03.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c8061640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c01f5020 con 0x7f92c00ac720 2023-10-18T10:25:03.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c8061640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c01f5300 con 0x7f92c0159530 2023-10-18T10:25:03.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c55fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c0157de0 0x7f92c014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:03.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c55fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c0157de0 0x7f92c014c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52148/0 (socket says 172.21.15.125:52148) 2023-10-18T10:25:03.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.325+0000 7f92c55fc640 1 -- 172.21.15.125:0/754136402 learned_addr learned my addr 172.21.15.125:0/754136402 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:03.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c65fe640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92c0159530 0x7f92c01552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:03.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c5dfd640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92c00ac720 0x7f92c01462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:03.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c55fc640 1 -- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92c0159530 msgr2=0x7f92c01552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:03.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c55fc640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92c0159530 0x7f92c01552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:03.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c55fc640 1 -- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92c00ac720 msgr2=0x7f92c01462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:03.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c55fc640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92c00ac720 0x7f92c01462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:03.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c55fc640 1 -- 172.21.15.125:0/754136402 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc009560 con 0x7f92c0157de0 2023-10-18T10:25:03.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c5dfd640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92c00ac720 0x7f92c01462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:03.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c55fc640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c0157de0 0x7f92c014c820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c970 tx=0x7f92b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92b67fc640 1 -- 172.21.15.125:0/754136402 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0007bb0 con 0x7f92c0157de0 2023-10-18T10:25:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92b67fc640 1 -- 172.21.15.125:0/754136402 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f92b0007e00 con 0x7f92c0157de0 2023-10-18T10:25:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c01f55e0 con 0x7f92c0157de0 2023-10-18T10:25:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92b67fc640 1 -- 172.21.15.125:0/754136402 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0005280 con 0x7f92c0157de0 2023-10-18T10:25:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.326+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c0146800 con 0x7f92c0157de0 2023-10-18T10:25:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.327+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c01f1730 con 0x7f92c0157de0 2023-10-18T10:25:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.327+0000 7f92b67fc640 1 -- 172.21.15.125:0/754136402 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f92b00053e0 con 0x7f92c0157de0 2023-10-18T10:25:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.328+0000 7f92b67fc640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f92a406ae40 0x7f92a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.328+0000 7f92b67fc640 1 -- 172.21.15.125:0/754136402 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f92b0089990 con 0x7f92c0157de0 2023-10-18T10:25:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.328+0000 7f92c5dfd640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f92a406ae40 0x7f92a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:03.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.329+0000 7f92c5dfd640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f92a406ae40 0x7f92a406d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f92bc035000 tx=0x7f92bc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:03.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.332+0000 7f92b67fc640 1 -- 172.21.15.125:0/754136402 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92c01f1730 con 0x7f92c0157de0 2023-10-18T10:25:03.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:03.533+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f92c01f1700 con 0x7f92c0157de0 2023-10-18T10:25:04.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.219+0000 7f92b67fc640 1 -- 172.21.15.125:0/754136402 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f92b001d070 con 0x7f92c0157de0 2023-10-18T10:25:04.228 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 2 and data pool 3 2023-10-18T10:25:04.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.232+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f92a406ae40 msgr2=0x7f92a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:04.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.232+0000 7f92c8061640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f92a406ae40 0x7f92a406d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f92bc035000 tx=0x7f92bc004730 comp rx=0 tx=0).stop 2023-10-18T10:25:04.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.232+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c0157de0 msgr2=0x7f92c014c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:04.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.232+0000 7f92c8061640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c0157de0 0x7f92c014c820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c970 tx=0x7f92b000ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:04.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.235+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 shutdown_connections 2023-10-18T10:25:04.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.235+0000 7f92c8061640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f92a406ae40 0x7f92a406d300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:04.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.235+0000 7f92c8061640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92c0159530 0x7f92c01552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:04.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.235+0000 7f92c8061640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92c0157de0 0x7f92c014c820 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:04.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.235+0000 7f92c8061640 1 --2- 172.21.15.125:0/754136402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92c00ac720 0x7f92c01462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:04.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.235+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 >> 172.21.15.125:0/754136402 conn(0x7f92c00cd940 msgr2=0x7f92c00ad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:04.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.236+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 shutdown_connections 2023-10-18T10:25:04.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.236+0000 7f92c8061640 1 -- 172.21.15.125:0/754136402 wait complete. 2023-10-18T10:25:04.256 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-18T10:25:04.257 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-18T10:25:04.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.365+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2960942484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd9e0 msgr2=0x7f3a1c0cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:04.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.365+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2960942484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd9e0 0x7f3a1c0cde50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c073460 tx=0x7f3a1402bd70 comp rx=0 tx=0).stop 2023-10-18T10:25:04.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.365+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2960942484 shutdown_connections 2023-10-18T10:25:04.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.365+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2960942484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd9e0 0x7f3a1c0cde50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:04.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.365+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2960942484 >> v2:172.21.15.139:3300/0 conn(0x7f3a1c158910 0x7f3a1c158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:04.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.365+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2960942484 >> 172.21.15.125:0/2960942484 conn(0x7f3a1c0af7a0 msgr2=0x7f3a1c0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:04.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.365+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2960942484 shutdown_connections 2023-10-18T10:25:04.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.365+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2960942484 wait complete. 2023-10-18T10:25:04.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.366+0000 7f3a20cf2640 1 Processor -- start 2023-10-18T10:25:04.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.366+0000 7f3a20cf2640 1 -- start start 2023-10-18T10:25:04.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a20cf2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd1c0 0x7f3a1c1f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:04.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a20cf2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a1c0cd9e0 0x7f3a1c14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:04.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a20cf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a1c158910 0x7f3a1c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:04.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a20cf2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c154af0 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a20cf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c153ee0 con 0x7f3a1c158910 2023-10-18T10:25:04.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a20cf2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1541c0 con 0x7f3a1c0cd9e0 2023-10-18T10:25:04.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd1c0 0x7f3a1c1f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:04.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd1c0 0x7f3a1c1f1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52180/0 (socket says 172.21.15.125:52180) 2023-10-18T10:25:04.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 -- 172.21.15.125:0/2334911692 learned_addr learned my addr 172.21.15.125:0/2334911692 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:04.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a19d9b640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a1c0cd9e0 0x7f3a1c14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:04.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1ad9d640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a1c158910 0x7f3a1c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:04.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 -- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a1c0cd9e0 msgr2=0x7f3a1c14f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:04.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a1c0cd9e0 0x7f3a1c14f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:04.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 -- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a1c158910 msgr2=0x7f3a1c155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:04.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a1c158910 0x7f3a1c155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:04.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 -- 172.21.15.125:0/2334911692 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14009490 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.367+0000 7f3a1a59c640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd1c0 0x7f3a1c1f1610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00ea20 tx=0x7f3a0c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:04.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.368+0000 7f3a0affd640 1 -- 172.21.15.125:0/2334911692 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00cde0 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.368+0000 7f3a0affd640 1 -- 172.21.15.125:0/2334911692 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a0c011560 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.368+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c1544a0 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.368+0000 7f3a0affd640 1 -- 172.21.15.125:0/2334911692 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011950 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.368+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c155580 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.369+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c1557a0 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.369+0000 7f3a0affd640 1 -- 172.21.15.125:0/2334911692 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3a0c011ab0 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.369+0000 7f3a0affd640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39fc06ae40 0x7f39fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:04.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.370+0000 7f3a0affd640 1 -- 172.21.15.125:0/2334911692 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a0c08a240 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.370+0000 7f3a19d9b640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39fc06ae40 0x7f39fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:04.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.370+0000 7f3a19d9b640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39fc06ae40 0x7f39fc06d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c070e80 tx=0x7f3a10009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:04.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.374+0000 7f3a0affd640 1 -- 172.21.15.125:0/2334911692 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a1c1557a0 con 0x7f3a1c0cd1c0 2023-10-18T10:25:04.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:04.549+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f3a1c1557a0 con 0x7f3a1c0cd1c0 2023-10-18T10:25:05.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.227+0000 7f3a0affd640 1 -- 172.21.15.125:0/2334911692 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f3a1c1557a0 con 0x7f3a1c0cd1c0 2023-10-18T10:25:05.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39fc06ae40 msgr2=0x7f39fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:05.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39fc06ae40 0x7f39fc06d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c070e80 tx=0x7f3a10009040 comp rx=0 tx=0).stop 2023-10-18T10:25:05.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd1c0 msgr2=0x7f3a1c1f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:05.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd1c0 0x7f3a1c1f1610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00ea20 tx=0x7f3a0c00eee0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 shutdown_connections 2023-10-18T10:25:05.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39fc06ae40 0x7f39fc06d300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a1c158910 0x7f3a1c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a1c0cd9e0 0x7f3a1c14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 --2- 172.21.15.125:0/2334911692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a1c0cd1c0 0x7f3a1c1f1610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 >> 172.21.15.125:0/2334911692 conn(0x7f3a1c0af7a0 msgr2=0x7f3a1c1521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:05.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 shutdown_connections 2023-10-18T10:25:05.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.230+0000 7f3a20cf2640 1 -- 172.21.15.125:0/2334911692 wait complete. 2023-10-18T10:25:05.246 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:25:05.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.361+0000 7f115b70f640 1 -- 172.21.15.125:0/2138831723 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11540cd9f0 msgr2=0x7f1154162910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:05.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.361+0000 7f115b70f640 1 --2- 172.21.15.125:0/2138831723 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11540cd9f0 0x7f1154162910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f114c005fd0 tx=0x7f114c02c0e0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.361+0000 7f115b70f640 1 -- 172.21.15.125:0/2138831723 shutdown_connections 2023-10-18T10:25:05.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.361+0000 7f115b70f640 1 --2- 172.21.15.125:0/2138831723 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11540cd9f0 0x7f1154162910 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.361+0000 7f115b70f640 1 --2- 172.21.15.125:0/2138831723 >> v2:172.21.15.139:3300/0 conn(0x7f11541593e0 0x7f115406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.361+0000 7f115b70f640 1 -- 172.21.15.125:0/2138831723 >> 172.21.15.125:0/2138831723 conn(0x7f11540b07f0 msgr2=0x7f11540b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:05.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.361+0000 7f115b70f640 1 -- 172.21.15.125:0/2138831723 shutdown_connections 2023-10-18T10:25:05.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.361+0000 7f115b70f640 1 -- 172.21.15.125:0/2138831723 wait complete. 2023-10-18T10:25:05.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.362+0000 7f115b70f640 1 Processor -- start 2023-10-18T10:25:05.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.362+0000 7f115b70f640 1 -- start start 2023-10-18T10:25:05.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.362+0000 7f115b70f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11540cd9f0 0x7f1154145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:05.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.362+0000 7f115b70f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11541593e0 0x7f1154145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:05.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.362+0000 7f115b70f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11541597c0 0x7f115406c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:05.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.362+0000 7f115b70f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115406c830 con 0x7f11541597c0 2023-10-18T10:25:05.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.362+0000 7f115b70f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115414f3b0 con 0x7f11540cd9f0 2023-10-18T10:25:05.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.362+0000 7f115b70f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115414edf0 con 0x7f11541593e0 2023-10-18T10:25:05.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11541597c0 0x7f115406c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:05.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11541597c0 0x7f115406c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52196/0 (socket says 172.21.15.125:52196) 2023-10-18T10:25:05.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 -- 172.21.15.125:0/1288272330 learned_addr learned my addr 172.21.15.125:0/1288272330 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:05.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f11594ab640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11540cd9f0 0x7f1154145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:05.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1158caa640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11541593e0 0x7f1154145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:05.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 -- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11541593e0 msgr2=0x7f1154145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:05.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11541593e0 0x7f1154145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 -- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11540cd9f0 msgr2=0x7f1154145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:05.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11540cd9f0 0x7f1154145ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:05.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 -- 172.21.15.125:0/1288272330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150009560 con 0x7f11541597c0 2023-10-18T10:25:05.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1159cac640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11541597c0 0x7f115406c2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f114c036040 tx=0x7f114c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:05.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1149ffb640 1 -- 172.21.15.125:0/1288272330 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c003e60 con 0x7f11541597c0 2023-10-18T10:25:05.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1149ffb640 1 -- 172.21.15.125:0/1288272330 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f114c004790 con 0x7f11541597c0 2023-10-18T10:25:05.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c009840 con 0x7f11541597c0 2023-10-18T10:25:05.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f1149ffb640 1 -- 172.21.15.125:0/1288272330 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c034800 con 0x7f11541597c0 2023-10-18T10:25:05.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.363+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115406baf0 con 0x7f11541597c0 2023-10-18T10:25:05.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.364+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11540675f0 con 0x7f11541597c0 2023-10-18T10:25:05.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.365+0000 7f1149ffb640 1 -- 172.21.15.125:0/1288272330 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f114c034960 con 0x7f11541597c0 2023-10-18T10:25:05.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.365+0000 7f1149ffb640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f113806ae40 0x7f113806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:05.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.365+0000 7f1149ffb640 1 -- 172.21.15.125:0/1288272330 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f114c0ac5c0 con 0x7f11541597c0 2023-10-18T10:25:05.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.365+0000 7f11594ab640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f113806ae40 0x7f113806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:05.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.366+0000 7f11594ab640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f113806ae40 0x7f113806d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1150009530 tx=0x7f1150009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:05.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.369+0000 7f1149ffb640 1 -- 172.21.15.125:0/1288272330 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f114c077640 con 0x7f11541597c0 2023-10-18T10:25:05.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:05.538+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f115414c800 con 0x7f11541597c0 2023-10-18T10:25:07.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.246+0000 7f1149ffb640 1 -- 172.21.15.125:0/1288272330 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f114c07ac60 con 0x7f11541597c0 2023-10-18T10:25:07.249 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data_ec' created 2023-10-18T10:25:07.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.249+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f113806ae40 msgr2=0x7f113806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:07.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.249+0000 7f115b70f640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f113806ae40 0x7f113806d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1150009530 tx=0x7f1150009260 comp rx=0 tx=0).stop 2023-10-18T10:25:07.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.249+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11541597c0 msgr2=0x7f115406c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:07.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.249+0000 7f115b70f640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11541597c0 0x7f115406c2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f114c036040 tx=0x7f114c005c80 comp rx=0 tx=0).stop 2023-10-18T10:25:07.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.249+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 shutdown_connections 2023-10-18T10:25:07.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.249+0000 7f115b70f640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f113806ae40 0x7f113806d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:07.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.249+0000 7f115b70f640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11541597c0 0x7f115406c2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:07.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.250+0000 7f115b70f640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11541593e0 0x7f1154145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:07.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.250+0000 7f115b70f640 1 --2- 172.21.15.125:0/1288272330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11540cd9f0 0x7f1154145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:07.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.250+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 >> 172.21.15.125:0/1288272330 conn(0x7f11540b07f0 msgr2=0x7f115414d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:07.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.250+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 shutdown_connections 2023-10-18T10:25:07.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.250+0000 7f115b70f640 1 -- 172.21.15.125:0/1288272330 wait complete. 2023-10-18T10:25:07.269 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-18T10:25:07.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.378+0000 7f86f44d7640 1 -- 172.21.15.125:0/3759197485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86ec0ceb40 msgr2=0x7f86ec0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:07.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.378+0000 7f86f44d7640 1 --2- 172.21.15.125:0/3759197485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86ec0ceb40 0x7f86ec0b0650 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f86e4005fd0 tx=0x7f86e402bd90 comp rx=0 tx=0).stop 2023-10-18T10:25:07.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.378+0000 7f86f44d7640 1 -- 172.21.15.125:0/3759197485 shutdown_connections 2023-10-18T10:25:07.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.378+0000 7f86f44d7640 1 --2- 172.21.15.125:0/3759197485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86ec0ceb40 0x7f86ec0b0650 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:07.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.378+0000 7f86f44d7640 1 --2- 172.21.15.125:0/3759197485 >> v2:172.21.15.139:3300/0 conn(0x7f86ec15b2a0 0x7f86ec0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:07.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.379+0000 7f86f44d7640 1 -- 172.21.15.125:0/3759197485 >> 172.21.15.125:0/3759197485 conn(0x7f86ec0d0d60 msgr2=0x7f86ec0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:07.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.379+0000 7f86f44d7640 1 -- 172.21.15.125:0/3759197485 shutdown_connections 2023-10-18T10:25:07.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.379+0000 7f86f44d7640 1 -- 172.21.15.125:0/3759197485 wait complete. 2023-10-18T10:25:07.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f44d7640 1 Processor -- start 2023-10-18T10:25:07.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f44d7640 1 -- start start 2023-10-18T10:25:07.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f44d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86ec0ceb40 0x7f86ec157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:07.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f44d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86ec0cf340 0x7f86ec1f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:07.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f44d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86ec15b2a0 0x7f86ec1f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:07.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f44d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec09e410 con 0x7f86ec15b2a0 2023-10-18T10:25:07.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f44d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec09dee0 con 0x7f86ec0ceb40 2023-10-18T10:25:07.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f44d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec09e130 con 0x7f86ec0cf340 2023-10-18T10:25:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.380+0000 7f86f2a74640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86ec15b2a0 0x7f86ec1f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2a74640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86ec15b2a0 0x7f86ec1f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52198/0 (socket says 172.21.15.125:52198) 2023-10-18T10:25:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2a74640 1 -- 172.21.15.125:0/1152178770 learned_addr learned my addr 172.21.15.125:0/1152178770 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f1a72640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86ec0cf340 0x7f86ec1f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2273640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86ec0ceb40 0x7f86ec157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2a74640 1 -- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86ec0cf340 msgr2=0x7f86ec1f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2a74640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86ec0cf340 0x7f86ec1f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2a74640 1 -- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86ec0ceb40 msgr2=0x7f86ec157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2a74640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86ec0ceb40 0x7f86ec157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2a74640 1 -- 172.21.15.125:0/1152178770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e4009560 con 0x7f86ec15b2a0 2023-10-18T10:25:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f2a74640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86ec15b2a0 0x7f86ec1f5250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86e4006010 tx=0x7f86e402d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86e2ffd640 1 -- 172.21.15.125:0/1152178770 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e402ddf0 con 0x7f86ec15b2a0 2023-10-18T10:25:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86e2ffd640 1 -- 172.21.15.125:0/1152178770 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f86e40035f0 con 0x7f86ec15b2a0 2023-10-18T10:25:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec0b0d50 con 0x7f86ec15b2a0 2023-10-18T10:25:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86e2ffd640 1 -- 172.21.15.125:0/1152178770 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4003a00 con 0x7f86ec15b2a0 2023-10-18T10:25:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.381+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec15dd10 con 0x7f86ec15b2a0 2023-10-18T10:25:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.382+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec15d860 con 0x7f86ec15b2a0 2023-10-18T10:25:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.382+0000 7f86e2ffd640 1 -- 172.21.15.125:0/1152178770 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f86e4003b60 con 0x7f86ec15b2a0 2023-10-18T10:25:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.383+0000 7f86e2ffd640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f86d006ae40 0x7f86d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.383+0000 7f86e2ffd640 1 -- 172.21.15.125:0/1152178770 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f86e40ac110 con 0x7f86ec15b2a0 2023-10-18T10:25:07.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.383+0000 7f86f2273640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f86d006ae40 0x7f86d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:07.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.384+0000 7f86f2273640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f86d006ae40 0x7f86d006d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86e8001ea0 tx=0x7f86e8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:07.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.387+0000 7f86e2ffd640 1 -- 172.21.15.125:0/1152178770 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86e4077030 con 0x7f86ec15b2a0 2023-10-18T10:25:07.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:07.554+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f86ec0690e0 con 0x7f86ec15b2a0 2023-10-18T10:25:08.132 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T10:25:08.130+0000 7feca2c77640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:25:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.248+0000 7f86e2ffd640 1 -- 172.21.15.125:0/1152178770 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f86e407a650 con 0x7f86ec15b2a0 2023-10-18T10:25:08.250 INFO:teuthology.orchestra.run.smithi125.stderr:set pool 4 allow_ec_overwrites to true 2023-10-18T10:25:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.250+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f86d006ae40 msgr2=0x7f86d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.250+0000 7f86f44d7640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f86d006ae40 0x7f86d006d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86e8001ea0 tx=0x7f86e8009040 comp rx=0 tx=0).stop 2023-10-18T10:25:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.250+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86ec15b2a0 msgr2=0x7f86ec1f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.250+0000 7f86f44d7640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86ec15b2a0 0x7f86ec1f5250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86e4006010 tx=0x7f86e402d9f0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.251+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 shutdown_connections 2023-10-18T10:25:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.251+0000 7f86f44d7640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f86d006ae40 0x7f86d006d300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.251+0000 7f86f44d7640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86ec15b2a0 0x7f86ec1f5250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.251+0000 7f86f44d7640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86ec0cf340 0x7f86ec1f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.251+0000 7f86f44d7640 1 --2- 172.21.15.125:0/1152178770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86ec0ceb40 0x7f86ec157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.251+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 >> 172.21.15.125:0/1152178770 conn(0x7f86ec0d0d60 msgr2=0x7f86ec0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.251+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 shutdown_connections 2023-10-18T10:25:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.251+0000 7f86f44d7640 1 -- 172.21.15.125:0/1152178770 wait complete. 2023-10-18T10:25:08.281 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-18T10:25:08.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.388+0000 7ff6034da640 1 -- 172.21.15.125:0/1998095631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 msgr2=0x7ff5fc15d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:08.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.388+0000 7ff6034da640 1 --2- 172.21.15.125:0/1998095631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 0x7ff5fc15d160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4005fd0 tx=0x7ff5f402c050 comp rx=0 tx=0).stop 2023-10-18T10:25:08.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.388+0000 7ff6034da640 1 -- 172.21.15.125:0/1998095631 shutdown_connections 2023-10-18T10:25:08.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.388+0000 7ff6034da640 1 --2- 172.21.15.125:0/1998095631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 0x7ff5fc15d160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.388+0000 7ff6034da640 1 --2- 172.21.15.125:0/1998095631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5fc0ce240 0x7ff5fc06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.388+0000 7ff6034da640 1 -- 172.21.15.125:0/1998095631 >> 172.21.15.125:0/1998095631 conn(0x7ff5fc0b0670 msgr2=0x7ff5fc0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:08.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.388+0000 7ff6034da640 1 -- 172.21.15.125:0/1998095631 shutdown_connections 2023-10-18T10:25:08.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.388+0000 7ff6034da640 1 -- 172.21.15.125:0/1998095631 wait complete. 2023-10-18T10:25:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.389+0000 7ff6034da640 1 Processor -- start 2023-10-18T10:25:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.389+0000 7ff6034da640 1 -- start start 2023-10-18T10:25:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff6034da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5fc0ce240 0x7ff5fc0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff6034da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5fc159060 0x7ff5fc0b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:08.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff6034da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 0x7ff5fc155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:08.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff6034da640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0b2850 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff6034da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0b40c0 con 0x7ff5fc159060 2023-10-18T10:25:08.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff6034da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0c6910 con 0x7ff5fc0ce240 2023-10-18T10:25:08.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601a77640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 0x7ff5fc155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:08.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601a77640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 0x7ff5fc155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40088/0 (socket says 172.21.15.125:40088) 2023-10-18T10:25:08.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601a77640 1 -- 172.21.15.125:0/2894079083 learned_addr learned my addr 172.21.15.125:0/2894079083 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:08.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601a77640 1 -- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5fc0ce240 msgr2=0x7ff5fc0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:08.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff600a75640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5fc159060 0x7ff5fc0b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:08.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601276640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5fc0ce240 0x7ff5fc0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:08.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601a77640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5fc0ce240 0x7ff5fc0b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601a77640 1 -- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5fc159060 msgr2=0x7ff5fc0b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:08.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601a77640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5fc159060 0x7ff5fc0b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:08.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601a77640 1 -- 172.21.15.125:0/2894079083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4009840 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff600a75640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5fc159060 0x7ff5fc0b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:08.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.390+0000 7ff601276640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5fc0ce240 0x7ff5fc0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:08.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.391+0000 7ff601a77640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 0x7ff5fc155ef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009cd0 tx=0x7ff5f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:08.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.391+0000 7ff5f1ffb640 1 -- 172.21.15.125:0/2894079083 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f403e460 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.391+0000 7ff5f1ffb640 1 -- 172.21.15.125:0/2894079083 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5f403e6b0 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.391+0000 7ff5f1ffb640 1 -- 172.21.15.125:0/2894079083 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f403ea80 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.391+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc0b1fb0 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.391+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc0b2570 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.392+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc0675f0 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.392+0000 7ff5f1ffb640 1 -- 172.21.15.125:0/2894079083 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff5f403ebe0 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.393+0000 7ff5f1ffb640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff5e006ae40 0x7ff5e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:08.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.393+0000 7ff5f1ffb640 1 -- 172.21.15.125:0/2894079083 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7ff5f40acc00 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.393+0000 7ff601276640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff5e006ae40 0x7ff5e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:08.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.393+0000 7ff601276640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff5e006ae40 0x7ff5e006d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0011e0 tx=0x7ff5ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:08.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.397+0000 7ff5f1ffb640 1 -- 172.21.15.125:0/2894079083 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5f40b3080 con 0x7ff5fc15ccf0 2023-10-18T10:25:08.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:08.600+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7ff5fc0b4aa0 con 0x7ff5fc15ccf0 2023-10-18T10:25:09.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.258+0000 7ff5f1ffb640 1 -- 172.21.15.125:0/2894079083 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7ff5f4042020 con 0x7ff5fc15ccf0 2023-10-18T10:25:09.260 INFO:teuthology.orchestra.run.smithi125.stderr:added data pool 4 to fsmap 2023-10-18T10:25:09.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff5e006ae40 msgr2=0x7ff5e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff5e006ae40 0x7ff5e006d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0011e0 tx=0x7ff5ec009040 comp rx=0 tx=0).stop 2023-10-18T10:25:09.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 msgr2=0x7ff5fc155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 0x7ff5fc155ef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009cd0 tx=0x7ff5f40042f0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 shutdown_connections 2023-10-18T10:25:09.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff5e006ae40 0x7ff5e006d300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5fc159060 0x7ff5fc0b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5fc15ccf0 0x7ff5fc155ef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 --2- 172.21.15.125:0/2894079083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5fc0ce240 0x7ff5fc0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 >> 172.21.15.125:0/2894079083 conn(0x7ff5fc0b0670 msgr2=0x7ff5fc0ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:09.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 shutdown_connections 2023-10-18T10:25:09.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.262+0000 7ff6034da640 1 -- 172.21.15.125:0/2894079083 wait complete. 2023-10-18T10:25:09.277 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:25:09.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.383+0000 7f30662d7640 1 -- 172.21.15.125:0/1706375756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3048005380 msgr2=0x7f304800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.383+0000 7f30662d7640 1 --2- 172.21.15.125:0/1706375756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3048005380 0x7f304800b980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f305800c640 tx=0x7f30480057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.383+0000 7f30662d7640 1 -- 172.21.15.125:0/1706375756 shutdown_connections 2023-10-18T10:25:09.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.383+0000 7f30662d7640 1 --2- 172.21.15.125:0/1706375756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3048005380 0x7f304800b980 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.383+0000 7f30662d7640 1 -- 172.21.15.125:0/1706375756 >> 172.21.15.125:0/1706375756 conn(0x7f3060069930 msgr2=0x7f3060069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:09.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.383+0000 7f30662d7640 1 -- 172.21.15.125:0/1706375756 shutdown_connections 2023-10-18T10:25:09.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.383+0000 7f30662d7640 1 -- 172.21.15.125:0/1706375756 wait complete. 2023-10-18T10:25:09.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.384+0000 7f30662d7640 1 Processor -- start 2023-10-18T10:25:09.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.384+0000 7f30662d7640 1 -- start start 2023-10-18T10:25:09.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f30662d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3048005380 0x7f306013b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:09.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f30662d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3060152f60 0x7f306006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:09.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f30662d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30600aa8c0 0x7f30600aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:09.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f30662d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600ab290 con 0x7f30600aa8c0 2023-10-18T10:25:09.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f30662d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060070220 con 0x7f3060152f60 2023-10-18T10:25:09.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f30662d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3060153840 con 0x7f3048005380 2023-10-18T10:25:09.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f3064874640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30600aa8c0 0x7f30600aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:09.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f3064874640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30600aa8c0 0x7f30600aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40102/0 (socket says 172.21.15.125:40102) 2023-10-18T10:25:09.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f3064874640 1 -- 172.21.15.125:0/830996197 learned_addr learned my addr 172.21.15.125:0/830996197 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:09.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f305ffff640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3048005380 0x7f306013b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:09.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f305f7fe640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3060152f60 0x7f306006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:09.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f3064874640 1 -- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3048005380 msgr2=0x7f306013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f3064874640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3048005380 0x7f306013b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f3064874640 1 -- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3060152f60 msgr2=0x7f306006f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f3064874640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3060152f60 0x7f306006f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.385+0000 7f3064874640 1 -- 172.21.15.125:0/830996197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060157810 con 0x7f30600aa8c0 2023-10-18T10:25:09.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.386+0000 7f3064874640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30600aa8c0 0x7f30600aad50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f305802cd40 tx=0x7f3058004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:09.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.386+0000 7f305cff9640 1 -- 172.21.15.125:0/830996197 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580017b0 con 0x7f30600aa8c0 2023-10-18T10:25:09.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.386+0000 7f305cff9640 1 -- 172.21.15.125:0/830996197 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3058001910 con 0x7f30600aa8c0 2023-10-18T10:25:09.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.386+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3060157a90 con 0x7f30600aa8c0 2023-10-18T10:25:09.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.386+0000 7f305cff9640 1 -- 172.21.15.125:0/830996197 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058006030 con 0x7f30600aa8c0 2023-10-18T10:25:09.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.386+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306006fc60 con 0x7f30600aa8c0 2023-10-18T10:25:09.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.387+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3060157c50 con 0x7f30600aa8c0 2023-10-18T10:25:09.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.387+0000 7f305cff9640 1 -- 172.21.15.125:0/830996197 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3058031030 con 0x7f30600aa8c0 2023-10-18T10:25:09.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.388+0000 7f305cff9640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:09.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.388+0000 7f305cff9640 1 -- 172.21.15.125:0/830996197 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f30580aa910 con 0x7f30600aa8c0 2023-10-18T10:25:09.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.388+0000 7f305ffff640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:09.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.388+0000 7f305ffff640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f304c06ae40 0x7f304c06d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f30600cb390 tx=0x7f3050005d40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:09.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.392+0000 7f305cff9640 1 -- 172.21.15.125:0/830996197 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30580756d0 con 0x7f30600aa8c0 2023-10-18T10:25:09.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.592+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30600690e0 con 0x7f30600aa8c0 2023-10-18T10:25:09.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.592+0000 7f305cff9640 1 -- 172.21.15.125:0/830996197 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f3058078cf0 con 0x7f30600aa8c0 2023-10-18T10:25:09.595 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:09.595 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:08.602257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-18T10:25:09.596 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 7 2023-10-18T10:25:09.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f304c06ae40 msgr2=0x7f304c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f304c06ae40 0x7f304c06d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f30600cb390 tx=0x7f3050005d40 comp rx=0 tx=0).stop 2023-10-18T10:25:09.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30600aa8c0 msgr2=0x7f30600aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30600aa8c0 0x7f30600aad50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f305802cd40 tx=0x7f3058004a00 comp rx=0 tx=0).stop 2023-10-18T10:25:09.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 shutdown_connections 2023-10-18T10:25:09.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3060152f60 0x7f306006f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30600aa8c0 0x7f30600aad50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 --2- 172.21.15.125:0/830996197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3048005380 0x7f306013b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 >> 172.21.15.125:0/830996197 conn(0x7f3060069930 msgr2=0x7f306015b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:09.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 shutdown_connections 2023-10-18T10:25:09.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.594+0000 7f30662d7640 1 -- 172.21.15.125:0/830996197 wait complete. 2023-10-18T10:25:09.606 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:09.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.706+0000 7ff1342bb640 1 -- 172.21.15.125:0/2564586467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c1579f0 msgr2=0x7ff12c0b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.706+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2564586467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c1579f0 0x7ff12c0b0e90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff12c06b930 tx=0x7ff12802be40 comp rx=0 tx=0).stop 2023-10-18T10:25:09.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.706+0000 7ff1342bb640 1 -- 172.21.15.125:0/2564586467 shutdown_connections 2023-10-18T10:25:09.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.706+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2564586467 >> v2:172.21.15.139:3300/0 conn(0x7ff12c15c1f0 0x7ff12c06a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.706+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2564586467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c1579f0 0x7ff12c0b0e90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.706+0000 7ff1342bb640 1 -- 172.21.15.125:0/2564586467 >> 172.21.15.125:0/2564586467 conn(0x7ff12c06fae0 msgr2=0x7ff12c06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:09.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.706+0000 7ff1342bb640 1 -- 172.21.15.125:0/2564586467 shutdown_connections 2023-10-18T10:25:09.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.706+0000 7ff1342bb640 1 -- 172.21.15.125:0/2564586467 wait complete. 2023-10-18T10:25:09.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff1342bb640 1 Processor -- start 2023-10-18T10:25:09.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff1342bb640 1 -- start start 2023-10-18T10:25:09.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff1342bb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c06ac70 0x7ff12c1482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:09.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff1342bb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff12c1579f0 0x7ff12c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:09.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff1342bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff12c15c1f0 0x7ff12c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:09.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff1342bb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1487f0 con 0x7ff12c06ac70 2023-10-18T10:25:09.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff1342bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c154990 con 0x7ff12c15c1f0 2023-10-18T10:25:09.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff1342bb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c154c70 con 0x7ff12c1579f0 2023-10-18T10:25:09.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff132057640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c06ac70 0x7ff12c1482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:09.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff132057640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c06ac70 0x7ff12c1482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40120/0 (socket says 172.21.15.125:40120) 2023-10-18T10:25:09.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.708+0000 7ff132057640 1 -- 172.21.15.125:0/2921602369 learned_addr learned my addr 172.21.15.125:0/2921602369 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:09.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff132057640 1 -- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff12c1579f0 msgr2=0x7ff12c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff132057640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff12c1579f0 0x7ff12c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff132057640 1 -- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff12c15c1f0 msgr2=0x7ff12c1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff132057640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff12c15c1f0 0x7ff12c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff132057640 1 -- 172.21.15.125:0/2921602369 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128009560 con 0x7ff12c06ac70 2023-10-18T10:25:09.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff132057640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c06ac70 0x7ff12c1482b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff128036040 tx=0x7ff12802db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:09.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff122ffd640 1 -- 172.21.15.125:0/2921602369 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128003800 con 0x7ff12c06ac70 2023-10-18T10:25:09.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff122ffd640 1 -- 172.21.15.125:0/2921602369 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff128003a50 con 0x7ff12c06ac70 2023-10-18T10:25:09.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff122ffd640 1 -- 172.21.15.125:0/2921602369 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128003de0 con 0x7ff12c06ac70 2023-10-18T10:25:09.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c154f50 con 0x7ff12c06ac70 2023-10-18T10:25:09.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.709+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c155510 con 0x7ff12c06ac70 2023-10-18T10:25:09.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.710+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c0cc120 con 0x7ff12c06ac70 2023-10-18T10:25:09.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.710+0000 7ff122ffd640 1 -- 172.21.15.125:0/2921602369 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff12802dd50 con 0x7ff12c06ac70 2023-10-18T10:25:09.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.711+0000 7ff122ffd640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff11006ae40 0x7ff11006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:09.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.711+0000 7ff122ffd640 1 -- 172.21.15.125:0/2921602369 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff1280ac100 con 0x7ff12c06ac70 2023-10-18T10:25:09.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.711+0000 7ff131856640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff11006ae40 0x7ff11006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:09.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.711+0000 7ff131856640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff11006ae40 0x7ff11006d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0058c0 tx=0x7ff11c005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:09.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.715+0000 7ff122ffd640 1 -- 172.21.15.125:0/2921602369 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff128077070 con 0x7ff12c06ac70 2023-10-18T10:25:09.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.881+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff12c0690e0 con 0x7ff12c06ac70 2023-10-18T10:25:09.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.882+0000 7ff122ffd640 1 -- 172.21.15.125:0/2921602369 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7ff12807a690 con 0x7ff12c06ac70 2023-10-18T10:25:09.884 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:09.885 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:09.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff11006ae40 msgr2=0x7ff11006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff11006ae40 0x7ff11006d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0058c0 tx=0x7ff11c005850 comp rx=0 tx=0).stop 2023-10-18T10:25:09.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c06ac70 msgr2=0x7ff12c1482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c06ac70 0x7ff12c1482b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff128036040 tx=0x7ff12802db60 comp rx=0 tx=0).stop 2023-10-18T10:25:09.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 shutdown_connections 2023-10-18T10:25:09.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff11006ae40 0x7ff11006d300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff12c15c1f0 0x7ff12c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff12c1579f0 0x7ff12c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 --2- 172.21.15.125:0/2921602369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff12c06ac70 0x7ff12c1482b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:09.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 >> 172.21.15.125:0/2921602369 conn(0x7ff12c06fae0 msgr2=0x7ff12c14a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:09.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 shutdown_connections 2023-10-18T10:25:09.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.884+0000 7ff1342bb640 1 -- 172.21.15.125:0/2921602369 wait complete. 2023-10-18T10:25:09.897 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:09.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.997+0000 7fccbb607640 1 -- 172.21.15.125:0/1164204768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 msgr2=0x7fccb40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:09.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.997+0000 7fccbb607640 1 --2- 172.21.15.125:0/1164204768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 0x7fccb40b0650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fccac009830 tx=0x7fccac02bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.997+0000 7fccbb607640 1 -- 172.21.15.125:0/1164204768 shutdown_connections 2023-10-18T10:25:10.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.997+0000 7fccbb607640 1 --2- 172.21.15.125:0/1164204768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 0x7fccb40b0650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.997+0000 7fccbb607640 1 --2- 172.21.15.125:0/1164204768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccb415b2a0 0x7fccb40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.997+0000 7fccbb607640 1 -- 172.21.15.125:0/1164204768 >> 172.21.15.125:0/1164204768 conn(0x7fccb40d0d60 msgr2=0x7fccb40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:10.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.997+0000 7fccbb607640 1 -- 172.21.15.125:0/1164204768 shutdown_connections 2023-10-18T10:25:10.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.997+0000 7fccbb607640 1 -- 172.21.15.125:0/1164204768 wait complete. 2023-10-18T10:25:10.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.998+0000 7fccbb607640 1 Processor -- start 2023-10-18T10:25:10.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.998+0000 7fccbb607640 1 -- start start 2023-10-18T10:25:10.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccbb607640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 0x7fccb40ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccbb607640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccb40cf340 0x7fccb4157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccbb607640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccb415b2a0 0x7fccb41e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccbb607640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4158480 con 0x7fccb40ceb40 2023-10-18T10:25:10.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccbb607640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb41e3fd0 con 0x7fccb415b2a0 2023-10-18T10:25:10.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccbb607640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb41e5120 con 0x7fccb40cf340 2023-10-18T10:25:10.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 0x7fccb40ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 0x7fccb40ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40132/0 (socket says 172.21.15.125:40132) 2023-10-18T10:25:10.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 -- 172.21.15.125:0/2726403775 learned_addr learned my addr 172.21.15.125:0/2726403775 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:10.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb8ba2640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccb40cf340 0x7fccb4157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 -- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccb40cf340 msgr2=0x7fccb4157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccb40cf340 0x7fccb4157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 -- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccb415b2a0 msgr2=0x7fccb41e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccb415b2a0 0x7fccb41e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 -- 172.21.15.125:0/2726403775 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac009490 con 0x7fccb40ceb40 2023-10-18T10:25:10.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccb93a3640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 0x7fccb40ca960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fccb000cbc0 tx=0x7fccb000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:10.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fcca9ffb640 1 -- 172.21.15.125:0/2726403775 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb000ace0 con 0x7fccb40ceb40 2023-10-18T10:25:10.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fcca9ffb640 1 -- 172.21.15.125:0/2726403775 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fccb0012420 con 0x7fccb40ceb40 2023-10-18T10:25:10.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb41e5380 con 0x7fccb40ceb40 2023-10-18T10:25:10.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:09.999+0000 7fcca9ffb640 1 -- 172.21.15.125:0/2726403775 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0012830 con 0x7fccb40ceb40 2023-10-18T10:25:10.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.000+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb41e3980 con 0x7fccb40ceb40 2023-10-18T10:25:10.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.001+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb40690e0 con 0x7fccb40ceb40 2023-10-18T10:25:10.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.001+0000 7fcca9ffb640 1 -- 172.21.15.125:0/2726403775 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fccb001a3b0 con 0x7fccb40ceb40 2023-10-18T10:25:10.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.001+0000 7fcca9ffb640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcc9806ae70 0x7fcc9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.001+0000 7fcca9ffb640 1 -- 172.21.15.125:0/2726403775 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fccb008a120 con 0x7fccb40ceb40 2023-10-18T10:25:10.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.001+0000 7fccb8ba2640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcc9806ae70 0x7fcc9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.002+0000 7fccb8ba2640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcc9806ae70 0x7fcc9806d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcca40058c0 tx=0x7fcca4005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:10.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.005+0000 7fcca9ffb640 1 -- 172.21.15.125:0/2726403775 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fccb0054ee0 con 0x7fccb40ceb40 2023-10-18T10:25:10.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.172+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccb40cb980 con 0x7fccb40ceb40 2023-10-18T10:25:10.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.173+0000 7fcca9ffb640 1 -- 172.21.15.125:0/2726403775 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7fccb0058500 con 0x7fccb40ceb40 2023-10-18T10:25:10.175 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:10.176 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:10.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcc9806ae70 msgr2=0x7fcc9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcc9806ae70 0x7fcc9806d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcca40058c0 tx=0x7fcca4005850 comp rx=0 tx=0).stop 2023-10-18T10:25:10.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 msgr2=0x7fccb40ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 0x7fccb40ca960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fccb000cbc0 tx=0x7fccb000a430 comp rx=0 tx=0).stop 2023-10-18T10:25:10.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 shutdown_connections 2023-10-18T10:25:10.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcc9806ae70 0x7fcc9806d330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccb415b2a0 0x7fccb41e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccb40cf340 0x7fccb4157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 --2- 172.21.15.125:0/2726403775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccb40ceb40 0x7fccb40ca960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 >> 172.21.15.125:0/2726403775 conn(0x7fccb40d0d60 msgr2=0x7fccb40ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:10.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 shutdown_connections 2023-10-18T10:25:10.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.175+0000 7fccbb607640 1 -- 172.21.15.125:0/2726403775 wait complete. 2023-10-18T10:25:10.201 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:25:10.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 -- 172.21.15.125:0/3563404243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f466806b030 msgr2=0x7f466806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 --2- 172.21.15.125:0/3563404243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f466806b030 0x7f466806b410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f46680cb9b0 tx=0x7f466402bd10 comp rx=0 tx=0).stop 2023-10-18T10:25:10.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 -- 172.21.15.125:0/3563404243 shutdown_connections 2023-10-18T10:25:10.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 --2- 172.21.15.125:0/3563404243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f466815ccb0 0x7f46680c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 --2- 172.21.15.125:0/3563404243 >> v2:172.21.15.139:3300/0 conn(0x7f4668159bd0 0x7f466815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 --2- 172.21.15.125:0/3563404243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f466806b030 0x7f466806b410 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 -- 172.21.15.125:0/3563404243 >> 172.21.15.125:0/3563404243 conn(0x7f4668069930 msgr2=0x7f4668069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:10.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 -- 172.21.15.125:0/3563404243 shutdown_connections 2023-10-18T10:25:10.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.301+0000 7f466f22b640 1 -- 172.21.15.125:0/3563404243 wait complete. 2023-10-18T10:25:10.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.302+0000 7f466f22b640 1 Processor -- start 2023-10-18T10:25:10.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.303+0000 7f466f22b640 1 -- start start 2023-10-18T10:25:10.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.303+0000 7f466f22b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f466806b030 0x7f46681eec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.303+0000 7f466f22b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668159bd0 0x7f46680ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.303+0000 7f466f22b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f466815ccb0 0x7f46680cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.303+0000 7f466f22b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46680ab390 con 0x7f4668159bd0 2023-10-18T10:25:10.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.303+0000 7f466f22b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46680aa470 con 0x7f466815ccb0 2023-10-18T10:25:10.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.303+0000 7f466f22b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46680a96b0 con 0x7f466806b030 2023-10-18T10:25:10.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668159bd0 0x7f46680ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668159bd0 0x7f46680ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40158/0 (socket says 172.21.15.125:40158) 2023-10-18T10:25:10.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 -- 172.21.15.125:0/3933126896 learned_addr learned my addr 172.21.15.125:0/3933126896 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:10.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f466cfc7640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f466806b030 0x7f46681eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f466d7c8640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f466815ccb0 0x7f46680cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 -- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f466806b030 msgr2=0x7f46681eec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f466806b030 0x7f46681eec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 -- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f466815ccb0 msgr2=0x7f46680cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f466815ccb0 0x7f46680cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 -- 172.21.15.125:0/3933126896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4664009560 con 0x7f4668159bd0 2023-10-18T10:25:10.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f466cfc7640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f466806b030 0x7f46681eec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:10.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465ffff640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668159bd0 0x7f46680ca870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f465800c920 tx=0x7f465800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:10.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465d7fa640 1 -- 172.21.15.125:0/3933126896 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658007c70 con 0x7f4668159bd0 2023-10-18T10:25:10.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465d7fa640 1 -- 172.21.15.125:0/3933126896 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f465800f450 con 0x7f4668159bd0 2023-10-18T10:25:10.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46680a9990 con 0x7f4668159bd0 2023-10-18T10:25:10.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f465d7fa640 1 -- 172.21.15.125:0/3933126896 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465800f860 con 0x7f4668159bd0 2023-10-18T10:25:10.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.304+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46680a9f50 con 0x7f4668159bd0 2023-10-18T10:25:10.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.305+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46680690e0 con 0x7f4668159bd0 2023-10-18T10:25:10.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.306+0000 7f465d7fa640 1 -- 172.21.15.125:0/3933126896 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f465800f9c0 con 0x7f4668159bd0 2023-10-18T10:25:10.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.306+0000 7f465d7fa640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f465406ae40 0x7f465406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.307+0000 7f465d7fa640 1 -- 172.21.15.125:0/3933126896 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f465808a200 con 0x7f4668159bd0 2023-10-18T10:25:10.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.307+0000 7f466cfc7640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f465406ae40 0x7f465406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.308+0000 7f466cfc7640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f465406ae40 0x7f465406d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4664035000 tx=0x7f4664004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:10.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.311+0000 7f465d7fa640 1 -- 172.21.15.125:0/3933126896 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4658055050 con 0x7f4668159bd0 2023-10-18T10:25:10.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.512+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46680cc460 con 0x7f4668159bd0 2023-10-18T10:25:10.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.513+0000 7f465d7fa640 1 -- 172.21.15.125:0/3933126896 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f4658058670 con 0x7f4668159bd0 2023-10-18T10:25:10.515 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:10.515 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:08.602257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-18T10:25:10.518 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 7 2023-10-18T10:25:10.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.514+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f465406ae40 msgr2=0x7f465406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.514+0000 7f466f22b640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f465406ae40 0x7f465406d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4664035000 tx=0x7f4664004730 comp rx=0 tx=0).stop 2023-10-18T10:25:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.514+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668159bd0 msgr2=0x7f46680ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.514+0000 7f466f22b640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668159bd0 0x7f46680ca870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f465800c920 tx=0x7f465800cde0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.515+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 shutdown_connections 2023-10-18T10:25:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.515+0000 7f466f22b640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f465406ae40 0x7f465406d300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.515+0000 7f466f22b640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f466815ccb0 0x7f46680cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.515+0000 7f466f22b640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668159bd0 0x7f46680ca870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.515+0000 7f466f22b640 1 --2- 172.21.15.125:0/3933126896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f466806b030 0x7f46681eec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.515+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 >> 172.21.15.125:0/3933126896 conn(0x7f4668069930 msgr2=0x7f4668069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:10.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.515+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 shutdown_connections 2023-10-18T10:25:10.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.515+0000 7f466f22b640 1 -- 172.21.15.125:0/3933126896 wait complete. 2023-10-18T10:25:10.527 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:10.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 -- 172.21.15.125:0/2074450918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 msgr2=0x7f22ec0b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 --2- 172.21.15.125:0/2074450918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 0x7f22ec0b0e90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0acd30 tx=0x7f22dc02bda0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 -- 172.21.15.125:0/2074450918 shutdown_connections 2023-10-18T10:25:10.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 --2- 172.21.15.125:0/2074450918 >> v2:172.21.15.139:3300/0 conn(0x7f22ec06ac70 0x7f22ec06ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 --2- 172.21.15.125:0/2074450918 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec15c1f0 0x7f22ec06a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 --2- 172.21.15.125:0/2074450918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 0x7f22ec0b0e90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 -- 172.21.15.125:0/2074450918 >> 172.21.15.125:0/2074450918 conn(0x7f22ec06fae0 msgr2=0x7f22ec06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:10.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 -- 172.21.15.125:0/2074450918 shutdown_connections 2023-10-18T10:25:10.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.626+0000 7f22f25ce640 1 -- 172.21.15.125:0/2074450918 wait complete. 2023-10-18T10:25:10.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.627+0000 7f22f25ce640 1 Processor -- start 2023-10-18T10:25:10.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22f25ce640 1 -- start start 2023-10-18T10:25:10.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22f25ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06ac70 0x7f22ec1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22f25ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 0x7f22ec14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22f25ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec15c1f0 0x7f22ec151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22f25ce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec14d930 con 0x7f22ec1579f0 2023-10-18T10:25:10.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22f25ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec14ccf0 con 0x7f22ec15c1f0 2023-10-18T10:25:10.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22f25ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec14cf90 con 0x7f22ec06ac70 2023-10-18T10:25:10.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22eb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 0x7f22ec14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22eb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 0x7f22ec14dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40172/0 (socket says 172.21.15.125:40172) 2023-10-18T10:25:10.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22eb7fe640 1 -- 172.21.15.125:0/3114502551 learned_addr learned my addr 172.21.15.125:0/3114502551 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:10.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22ebfff640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06ac70 0x7f22ec1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.628+0000 7f22f0b6b640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec15c1f0 0x7f22ec151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22eb7fe640 1 -- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06ac70 msgr2=0x7f22ec1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22eb7fe640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06ac70 0x7f22ec1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22eb7fe640 1 -- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec15c1f0 msgr2=0x7f22ec151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22eb7fe640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec15c1f0 0x7f22ec151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22eb7fe640 1 -- 172.21.15.125:0/3114502551 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc009560 con 0x7f22ec1579f0 2023-10-18T10:25:10.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22f0b6b640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec15c1f0 0x7f22ec151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:10.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22eb7fe640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 0x7f22ec14dd50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22e000d750 tx=0x7f22e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:10.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22e8ff9640 1 -- 172.21.15.125:0/3114502551 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0009a30 con 0x7f22ec1579f0 2023-10-18T10:25:10.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22e8ff9640 1 -- 172.21.15.125:0/3114502551 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f22e0009c80 con 0x7f22ec1579f0 2023-10-18T10:25:10.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22e8ff9640 1 -- 172.21.15.125:0/3114502551 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e00106c0 con 0x7f22ec1579f0 2023-10-18T10:25:10.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec0b2330 con 0x7f22ec1579f0 2023-10-18T10:25:10.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.629+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec14d550 con 0x7f22ec1579f0 2023-10-18T10:25:10.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.630+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec0690e0 con 0x7f22ec1579f0 2023-10-18T10:25:10.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.630+0000 7f22e8ff9640 1 -- 172.21.15.125:0/3114502551 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f22e0010820 con 0x7f22ec1579f0 2023-10-18T10:25:10.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.631+0000 7f22e8ff9640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22d806ae40 0x7f22d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:10.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.631+0000 7f22e8ff9640 1 -- 172.21.15.125:0/3114502551 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f22e0089c20 con 0x7f22ec1579f0 2023-10-18T10:25:10.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.631+0000 7f22ebfff640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22d806ae40 0x7f22d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:10.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.632+0000 7f22ebfff640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22d806ae40 0x7f22d806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22dc035000 tx=0x7f22dc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:10.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.635+0000 7f22e8ff9640 1 -- 172.21.15.125:0/3114502551 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22e0054b20 con 0x7f22ec1579f0 2023-10-18T10:25:10.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.802+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22ec0b1d80 con 0x7f22ec1579f0 2023-10-18T10:25:10.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.803+0000 7f22e8ff9640 1 -- 172.21.15.125:0/3114502551 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f22e0021090 con 0x7f22ec1579f0 2023-10-18T10:25:10.805 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:10.806 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:10.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22d806ae40 msgr2=0x7f22d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22d806ae40 0x7f22d806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22dc035000 tx=0x7f22dc004730 comp rx=0 tx=0).stop 2023-10-18T10:25:10.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 msgr2=0x7f22ec14dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:10.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 0x7f22ec14dd50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22e000d750 tx=0x7f22e000dc10 comp rx=0 tx=0).stop 2023-10-18T10:25:10.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 shutdown_connections 2023-10-18T10:25:10.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f22d806ae40 0x7f22d806d300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec15c1f0 0x7f22ec151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec1579f0 0x7f22ec14dd50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 --2- 172.21.15.125:0/3114502551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06ac70 0x7f22ec1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:10.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 >> 172.21.15.125:0/3114502551 conn(0x7f22ec06fae0 msgr2=0x7f22ec0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:10.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 shutdown_connections 2023-10-18T10:25:10.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:10.805+0000 7f22f25ce640 1 -- 172.21.15.125:0/3114502551 wait complete. 2023-10-18T10:25:11.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:25:03.534893+0000', 'modified': '2023-10-18T10:25:08.602257+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4401}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4401': {'gid': 4401, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6834/4043797924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6832', 'nonce': 4043797924}, {'type': 'v1', 'addr': '172.21.15.125:6834', 'nonce': 4043797924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-18T10:25:11.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:25:11.221 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:25:11.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.325+0000 7fdaf144b640 1 -- 172.21.15.125:0/1162832708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec0cdb30 msgr2=0x7fdaec0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.325+0000 7fdaf144b640 1 --2- 172.21.15.125:0/1162832708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec0cdb30 0x7fdaec0cdfb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0ceab0 tx=0x7fdae002bd10 comp rx=0 tx=0).stop 2023-10-18T10:25:11.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.326+0000 7fdaf144b640 1 -- 172.21.15.125:0/1162832708 shutdown_connections 2023-10-18T10:25:11.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.326+0000 7fdaf144b640 1 --2- 172.21.15.125:0/1162832708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec0cdb30 0x7fdaec0cdfb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.326+0000 7fdaf144b640 1 --2- 172.21.15.125:0/1162832708 >> v2:172.21.15.139:3300/0 conn(0x7fdaec15ccf0 0x7fdaec15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.326+0000 7fdaf144b640 1 --2- 172.21.15.125:0/1162832708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaec159060 0x7fdaec06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.326+0000 7fdaf144b640 1 -- 172.21.15.125:0/1162832708 >> 172.21.15.125:0/1162832708 conn(0x7fdaec0b0670 msgr2=0x7fdaec0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:11.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.326+0000 7fdaf144b640 1 -- 172.21.15.125:0/1162832708 shutdown_connections 2023-10-18T10:25:11.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.326+0000 7fdaf144b640 1 -- 172.21.15.125:0/1162832708 wait complete. 2023-10-18T10:25:11.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.327+0000 7fdaf144b640 1 Processor -- start 2023-10-18T10:25:11.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.327+0000 7fdaf144b640 1 -- start start 2023-10-18T10:25:11.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.327+0000 7fdaf144b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaec0cdb30 0x7fdaec156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaf144b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec159060 0x7fdaec0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaf144b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec15ccf0 0x7fdaec0b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaf144b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec0c2610 con 0x7fdaec15ccf0 2023-10-18T10:25:11.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaf144b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec06c1f0 con 0x7fdaec159060 2023-10-18T10:25:11.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaf144b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec0b3860 con 0x7fdaec0cdb30 2023-10-18T10:25:11.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec15ccf0 0x7fdaec0b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec15ccf0 0x7fdaec0b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40196/0 (socket says 172.21.15.125:40196) 2023-10-18T10:25:11.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 -- 172.21.15.125:0/3121648943 learned_addr learned my addr 172.21.15.125:0/3121648943 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:11.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaec0cdb30 0x7fdaec156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec159060 0x7fdaec0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 -- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaec0cdb30 msgr2=0x7fdaec156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaec0cdb30 0x7fdaec156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 -- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec159060 msgr2=0x7fdaec0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec159060 0x7fdaec0b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 -- 172.21.15.125:0/3121648943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae0009560 con 0x7fdaec15ccf0 2023-10-18T10:25:11.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaeb7fe640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec15ccf0 0x7fdaec0b3250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00c8f0 tx=0x7fdadc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:11.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdad37fe640 1 -- 172.21.15.125:0/3121648943 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc007bb0 con 0x7fdaec15ccf0 2023-10-18T10:25:11.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdad37fe640 1 -- 172.21.15.125:0/3121648943 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdadc007e00 con 0x7fdaec15ccf0 2023-10-18T10:25:11.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaf144b640 1 -- 172.21.15.125:0/3121648943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec0b3b40 con 0x7fdaec15ccf0 2023-10-18T10:25:11.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdad37fe640 1 -- 172.21.15.125:0/3121648943 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc005280 con 0x7fdaec15ccf0 2023-10-18T10:25:11.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.328+0000 7fdaf144b640 1 -- 172.21.15.125:0/3121648943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec0b4100 con 0x7fdaec15ccf0 2023-10-18T10:25:11.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.330+0000 7fdad37fe640 1 -- 172.21.15.125:0/3121648943 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdadc0053e0 con 0x7fdaec15ccf0 2023-10-18T10:25:11.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.330+0000 7fdad37fe640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdad806ae40 0x7fdad806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.330+0000 7fdad37fe640 1 -- 172.21.15.125:0/3121648943 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdadc089ce0 con 0x7fdaec15ccf0 2023-10-18T10:25:11.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.330+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdad806ae40 0x7fdad806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.331+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdad806ae40 0x7fdad806d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005fd0 tx=0x7fdae4009680 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:11.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.331+0000 7fdaf3fda740 1 Processor -- start 2023-10-18T10:25:11.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.331+0000 7fdaf3fda740 1 -- start start 2023-10-18T10:25:11.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a812eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a815e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a812f020 0x5620a815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a8040730 con 0x5620a815bf20 2023-10-18T10:25:11.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a80a1780 con 0x5620a815bb40 2023-10-18T10:25:11.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a809ee70 con 0x5620a812f020 2023-10-18T10:25:11.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a815e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a815e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40202/0 (socket says 172.21.15.125:40202) 2023-10-18T10:25:11.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1084825934 learned_addr learned my addr 172.21.15.125:0/1084825934 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:11.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a812eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaeb7fe640 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a812f020 0x5620a815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1084825934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a812f020 msgr2=0x5620a815ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:11.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a812f020 0x5620a815ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1084825934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 msgr2=0x5620a812eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:11.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a812eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.332+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1084825934 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a809f3d0 con 0x5620a815bf20 2023-10-18T10:25:11.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a815e8d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdae0036040 tx=0x7fdae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d056c62ca42206 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:11.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdad0ff9640 1 -- 172.21.15.125:0/1084825934 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0004280 con 0x5620a815bf20 2023-10-18T10:25:11.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdad0ff9640 1 -- 172.21.15.125:0/1084825934 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdae00044f0 con 0x5620a815bf20 2023-10-18T10:25:11.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdad0ff9640 1 -- 172.21.15.125:0/1084825934 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae002c8b0 con 0x5620a815bf20 2023-10-18T10:25:11.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1084825934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 msgr2=0x5620a815e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a815e8d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdae0036040 tx=0x7fdae00046c0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1084825934 shutdown_connections 2023-10-18T10:25:11.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a812f020 0x5620a815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a815e8d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1084825934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a812eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.333+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1084825934 >> 172.21.15.125:0/1084825934 conn(0x5620a8130d50 msgr2=0x5620a81798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:11.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.334+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1084825934 shutdown_connections 2023-10-18T10:25:11.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.334+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1084825934 wait complete. 2023-10-18T10:25:11.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.334+0000 7fdaf3fda740 1 -- start start 2023-10-18T10:25:11.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.334+0000 7fdaf3fda740 1 Processor -- start 2023-10-18T10:25:11.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bb40 0x5620a818a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a815bf20 0x5620a814fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a818af70 0x5620a8174b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a809a760 con 0x5620a815bb40 2023-10-18T10:25:11.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a80558f0 con 0x5620a818af70 2023-10-18T10:25:11.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a8055590 con 0x5620a815bf20 2023-10-18T10:25:11.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bb40 0x5620a818a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaf3fda740 10 client.0 _ll_register_callbacks cb 0x5620a8091ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:25:11.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bb40 0x5620a818a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40216/0 (socket says 172.21.15.125:40216) 2023-10-18T10:25:11.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 -- 172.21.15.125:0/1829582886 learned_addr learned my addr 172.21.15.125:0/1829582886 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:11.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a815bf20 0x5620a814fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeb7fe640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a818af70 0x5620a8174b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 -- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a815bf20 msgr2=0x5620a814fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:11.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a815bf20 0x5620a814fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 -- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a818af70 msgr2=0x5620a8174b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:11.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a818af70 0x5620a8174b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 -- 172.21.15.125:0/1829582886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec0c9980 con 0x5620a815bb40 2023-10-18T10:25:11.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a815bf20 0x5620a814fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:11.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeb7fe640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a818af70 0x5620a8174b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:11.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bb40 0x5620a818a520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdae40098b0 tx=0x7fdae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6008734e1817b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:11.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:25:11.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1829582886 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4029670 con 0x5620a815bb40 2023-10-18T10:25:11.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1829582886 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdae40297d0 con 0x5620a815bb40 2023-10-18T10:25:11.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.335+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1829582886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620a8093740 con 0x5620a815bb40 2023-10-18T10:25:11.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1829582886 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdae4029990 con 0x5620a815bb40 2023-10-18T10:25:11.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaf3fda740 10 client.4545 fetch_fsmap learned FSMap version 7 2023-10-18T10:25:11.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1829582886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5620a8098de0 con 0x5620a815bb40 2023-10-18T10:25:11.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1829582886 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5620a8093740 con 0x5620a815bb40 2023-10-18T10:25:11.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1829582886 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fdae4029af0 con 0x5620a815bb40 2023-10-18T10:25:11.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaf3fda740 10 client.4545 fetch_fsmap finished waiting for FSMap version 7 2023-10-18T10:25:11.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaf3fda740 10 client.4545 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:25:11.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaf3fda740 20 client.4545 populate_metadata read hostname 'smithi125' 2023-10-18T10:25:11.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x5620a818d0a0 0x5620a818d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaf3fda740 4 client.4545 mds_command: new command op to 4401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:25:11.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1829582886 --> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620a8093460 con 0x5620a818d0a0 2023-10-18T10:25:11.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x5620a818d0a0 0x5620a818d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdac9ffb640 10 client.4545 ms_handle_connect on v2:172.21.15.125:6832/4043797924 2023-10-18T10:25:11.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.336+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x5620a818d0a0 0x5620a818d480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c485b9a73ee6d3 server_cookie=1d701163db4b92c1 in_seq=0 out_seq=0 2023-10-18T10:25:11.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1829582886 <== mds.0 v2:172.21.15.125:6832/4043797924 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5620a8093460 con 0x5620a818d0a0 2023-10-18T10:25:11.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdac9ffb640 10 client.4545 handle_command_reply: tid=0 2023-10-18T10:25:11.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 client.4545 shutdown 2023-10-18T10:25:11.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1829582886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bb40 msgr2=0x5620a818a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bb40 0x5620a818a520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdae40098b0 tx=0x7fdae4004a40 comp rx=0 tx=0).stop 2023-10-18T10:25:11.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1829582886 shutdown_connections 2023-10-18T10:25:11.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x5620a818d0a0 0x5620a818d480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a818af70 0x5620a8174b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a815bf20 0x5620a814fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdac9ffb640 0 client.4545 ms_handle_reset on v2:172.21.15.125:6832/4043797924 2023-10-18T10:25:11.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1829582886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bb40 0x5620a818a520 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1829582886 >> 172.21.15.125:0/1829582886 conn(0x5620a812f020 msgr2=0x5620a81329c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:11.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1829582886 shutdown_connections 2023-10-18T10:25:11.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1829582886 wait complete. 2023-10-18T10:25:11.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.337+0000 7fdaf3fda740 20 client.4545 trim_cache size 0 max 16384 2023-10-18T10:25:11.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.352+0000 7fdaf3fda740 1 Processor -- start 2023-10-18T10:25:11.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaf3fda740 1 -- start start 2023-10-18T10:25:11.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a812dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a812e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 0x5620a81342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a801b270 con 0x5620a815bf20 2023-10-18T10:25:11.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a8193820 con 0x5620a815bb40 2023-10-18T10:25:11.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a8173990 con 0x5620a818af70 2023-10-18T10:25:11.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a812e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a812e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40218/0 (socket says 172.21.15.125:40218) 2023-10-18T10:25:11.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1079100936 learned_addr learned my addr 172.21.15.125:0/1079100936 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:11.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.353+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a812dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaeb7fe640 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 0x5620a81342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1079100936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 msgr2=0x5620a81342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:11.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 0x5620a81342e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1079100936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 msgr2=0x5620a812dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:11.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a812dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1079100936 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a8037d00 con 0x5620a815bf20 2023-10-18T10:25:11.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a812e020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdae00099c0 tx=0x7fdae002bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f35284138f78919 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:11.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdad17fa640 1 -- 172.21.15.125:0/1079100936 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae004d040 con 0x5620a815bf20 2023-10-18T10:25:11.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdad17fa640 1 -- 172.21.15.125:0/1079100936 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdae004f4c0 con 0x5620a815bf20 2023-10-18T10:25:11.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdad17fa640 1 -- 172.21.15.125:0/1079100936 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae002c6b0 con 0x5620a815bf20 2023-10-18T10:25:11.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1079100936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 msgr2=0x5620a812e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a812e020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdae00099c0 tx=0x7fdae002bd10 comp rx=0 tx=0).stop 2023-10-18T10:25:11.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1079100936 shutdown_connections 2023-10-18T10:25:11.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 0x5620a81342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a812e020 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1079100936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a812dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1079100936 >> 172.21.15.125:0/1079100936 conn(0x5620a812f020 msgr2=0x5620a817c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:11.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1079100936 shutdown_connections 2023-10-18T10:25:11.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.354+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1079100936 wait complete. 2023-10-18T10:25:11.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.355+0000 7fdaf3fda740 1 -- start start 2023-10-18T10:25:11.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.355+0000 7fdaf3fda740 1 Processor -- start 2023-10-18T10:25:11.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a81e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaf3fda740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 0x5620a812dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a81e2340 con 0x5620a815bf20 2023-10-18T10:25:11.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a819c770 con 0x5620a815bb40 2023-10-18T10:25:11.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaf3fda740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a81e68f0 con 0x5620a818af70 2023-10-18T10:25:11.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaf3fda740 10 client.0 _ll_register_callbacks cb 0x7fdae0005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:25:11.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a81e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a81e1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40232/0 (socket says 172.21.15.125:40232) 2023-10-18T10:25:11.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1822714168 learned_addr learned my addr 172.21.15.125:0/1822714168 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:11.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaeb7fe640 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 0x5620a812dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1822714168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 msgr2=0x5620a812dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:11.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 0x5620a812dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1822714168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 msgr2=0x5620a814e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:11.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a814e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 -- 172.21.15.125:0/1822714168 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620a809f3d0 con 0x5620a815bf20 2023-10-18T10:25:11.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaea7fc640 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a81e1e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdae0036040 tx=0x7fdae0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d961d20b64c6094b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:11.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:25:11.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1822714168 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae004d040 con 0x5620a815bf20 2023-10-18T10:25:11.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1822714168 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdae002cda0 con 0x5620a815bf20 2023-10-18T10:25:11.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1822714168 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdae00044f0 con 0x5620a815bf20 2023-10-18T10:25:11.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.356+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1822714168 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdae004f630 con 0x5620a815bf20 2023-10-18T10:25:11.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 10 client.4551 fetch_fsmap learned FSMap version 7 2023-10-18T10:25:11.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1822714168 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5620a805d4f0 con 0x5620a815bf20 2023-10-18T10:25:11.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1822714168 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdae00044f0 con 0x5620a815bf20 2023-10-18T10:25:11.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1822714168 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fdae0057070 con 0x5620a815bf20 2023-10-18T10:25:11.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 10 client.4551 fetch_fsmap finished waiting for FSMap version 7 2023-10-18T10:25:11.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 10 client.4551 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:25:11.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 20 client.4551 populate_metadata read hostname 'smithi125' 2023-10-18T10:25:11.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x5620a81e5a10 0x5620a81e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 4 client.4551 mds_command: new command op to 4401 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:25:11.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1822714168 --> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5620a7d9c6c0 con 0x5620a81e5a10 2023-10-18T10:25:11.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x5620a81e5a10 0x5620a81e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaeaffd640 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x5620a81e5a10 0x5620a81e5df0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14bd073eb980e86 server_cookie=f4c40f2e1474ba80 in_seq=0 out_seq=0 2023-10-18T10:25:11.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdac9ffb640 10 client.4551 ms_handle_connect on v2:172.21.15.125:6832/4043797924 2023-10-18T10:25:11.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdac9ffb640 1 -- 172.21.15.125:0/1822714168 <== mds.0 v2:172.21.15.125:6832/4043797924 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5620a7d9c6c0 con 0x5620a81e5a10 2023-10-18T10:25:11.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdac9ffb640 10 client.4551 handle_command_reply: tid=0 2023-10-18T10:25:11.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 client.4551 shutdown 2023-10-18T10:25:11.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1822714168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 msgr2=0x5620a81e1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a81e1e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdae0036040 tx=0x7fdae0005850 comp rx=0 tx=0).stop 2023-10-18T10:25:11.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1822714168 shutdown_connections 2023-10-18T10:25:11.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5620a818af70 0x5620a812dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5620a815bf20 0x5620a81e1e00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x5620a81e5a10 0x5620a81e5df0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 --2- 172.21.15.125:0/1822714168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620a815bb40 0x5620a814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdac9ffb640 0 client.4551 ms_handle_reset on v2:172.21.15.125:6832/4043797924 2023-10-18T10:25:11.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.357+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1822714168 >> 172.21.15.125:0/1822714168 conn(0x5620a812f020 msgr2=0x5620a812a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:11.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1822714168 shutdown_connections 2023-10-18T10:25:11.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf3fda740 1 -- 172.21.15.125:0/1822714168 wait complete. 2023-10-18T10:25:11.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf3fda740 20 client.4551 trim_cache size 0 max 16384 2023-10-18T10:25:11.392 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:25:11.392 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:25:11.392 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:25:11.392 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 4401, 2023-10-18T10:25:11.392 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:25:11.393 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:25:11.393 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:25:11.393 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 7.1369427940000003, 2023-10-18T10:25:11.393 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 7, 2023-10-18T10:25:11.393 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 25, 2023-10-18T10:25:11.394 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 24, 2023-10-18T10:25:11.394 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 14.247883653000001 2023-10-18T10:25:11.394 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:25:11.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 -- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdad806ae40 msgr2=0x7fdad806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdad806ae40 0x7fdad806d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005fd0 tx=0x7fdae4009680 comp rx=0 tx=0).stop 2023-10-18T10:25:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 -- 172.21.15.125:0/3121648943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec15ccf0 msgr2=0x7fdaec0b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec15ccf0 0x7fdaec0b3250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00c8f0 tx=0x7fdadc00cdb0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 -- 172.21.15.125:0/3121648943 shutdown_connections 2023-10-18T10:25:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdad806ae40 0x7fdad806d300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec159060 0x7fdaec0b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdaec15ccf0 0x7fdaec0b3250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 --2- 172.21.15.125:0/3121648943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdaec0cdb30 0x7fdaec156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 -- 172.21.15.125:0/3121648943 >> 172.21.15.125:0/3121648943 conn(0x7fdaec0b0670 msgr2=0x7fdaec0cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:11.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 -- 172.21.15.125:0/3121648943 shutdown_connections 2023-10-18T10:25:11.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.358+0000 7fdaf144b640 1 -- 172.21.15.125:0/3121648943 wait complete. 2023-10-18T10:25:11.398 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:25:11.399 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:25:11.399 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:25:11.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 -- 172.21.15.125:0/1465327318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 msgr2=0x7f3fbc0cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 --2- 172.21.15.125:0/1465327318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 0x7f3fbc0cad00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0ca400 tx=0x7f3fb402d0d0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 -- 172.21.15.125:0/1465327318 shutdown_connections 2023-10-18T10:25:11.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 --2- 172.21.15.125:0/1465327318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 0x7f3fbc0cad00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 --2- 172.21.15.125:0/1465327318 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fbc140660 0x7f3fbc15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 --2- 172.21.15.125:0/1465327318 >> v2:172.21.15.139:3300/0 conn(0x7f3fbc06d170 0x7f3fbc13d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 -- 172.21.15.125:0/1465327318 >> 172.21.15.125:0/1465327318 conn(0x7f3fbc069930 msgr2=0x7f3fbc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:11.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 -- 172.21.15.125:0/1465327318 shutdown_connections 2023-10-18T10:25:11.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.499+0000 7f3fc364f640 1 -- 172.21.15.125:0/1465327318 wait complete. 2023-10-18T10:25:11.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.500+0000 7f3fc364f640 1 Processor -- start 2023-10-18T10:25:11.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.500+0000 7f3fc364f640 1 -- start start 2023-10-18T10:25:11.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc364f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fbc06d170 0x7f3fbc133550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc364f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 0x7f3fbc13ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc364f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fbc140660 0x7f3fbc13f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc364f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc133a90 con 0x7f3fbc0ca870 2023-10-18T10:25:11.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc364f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc134df0 con 0x7f3fbc06d170 2023-10-18T10:25:11.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc364f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc134b10 con 0x7f3fbc140660 2023-10-18T10:25:11.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc0bea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 0x7f3fbc13ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc0bea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 0x7f3fbc13ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40246/0 (socket says 172.21.15.125:40246) 2023-10-18T10:25:11.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc0bea640 1 -- 172.21.15.125:0/3117302292 learned_addr learned my addr 172.21.15.125:0/3117302292 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:11.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc13eb640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fbc06d170 0x7f3fbc133550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc1bec640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fbc140660 0x7f3fbc13f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc0bea640 1 -- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fbc140660 msgr2=0x7f3fbc13f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc0bea640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fbc140660 0x7f3fbc13f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc0bea640 1 -- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fbc06d170 msgr2=0x7f3fbc133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc0bea640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fbc06d170 0x7f3fbc133550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc0bea640 1 -- 172.21.15.125:0/3117302292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb400a670 con 0x7f3fbc0ca870 2023-10-18T10:25:11.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc13eb640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fbc06d170 0x7f3fbc133550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:11.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.501+0000 7f3fc1bec640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fbc140660 0x7f3fbc13f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:11.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.502+0000 7f3fc0bea640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 0x7f3fbc13ed80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c9a0 tx=0x7f3fac00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:11.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.502+0000 7f3fb1ffb640 1 -- 172.21.15.125:0/3117302292 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac007bb0 con 0x7f3fbc0ca870 2023-10-18T10:25:11.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.502+0000 7f3fb1ffb640 1 -- 172.21.15.125:0/3117302292 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3fac007e20 con 0x7f3fbc0ca870 2023-10-18T10:25:11.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.502+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1350d0 con 0x7f3fbc0ca870 2023-10-18T10:25:11.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.502+0000 7f3fb1ffb640 1 -- 172.21.15.125:0/3117302292 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac005280 con 0x7f3fbc0ca870 2023-10-18T10:25:11.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.502+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc135690 con 0x7f3fbc0ca870 2023-10-18T10:25:11.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.503+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc067930 con 0x7f3fbc0ca870 2023-10-18T10:25:11.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.503+0000 7f3fb1ffb640 1 -- 172.21.15.125:0/3117302292 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3fac0053e0 con 0x7f3fbc0ca870 2023-10-18T10:25:11.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.503+0000 7f3fb1ffb640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3fa006ae40 0x7f3fa006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:11.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.503+0000 7f3fb1ffb640 1 -- 172.21.15.125:0/3117302292 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3fac089da0 con 0x7f3fbc0ca870 2023-10-18T10:25:11.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.504+0000 7f3fc13eb640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3fa006ae40 0x7f3fa006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:11.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.504+0000 7f3fc13eb640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3fa006ae40 0x7f3fa006d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8005c40 tx=0x7f3fb800a430 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:11.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.507+0000 7f3fb1ffb640 1 -- 172.21.15.125:0/3117302292 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fac054b60 con 0x7f3fbc0ca870 2023-10-18T10:25:11.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.711+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3fbc15b040 con 0x7f3fbc0ca870 2023-10-18T10:25:11.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.711+0000 7f3fb1ffb640 1 -- 172.21.15.125:0/3117302292 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3fac01d070 con 0x7f3fbc0ca870 2023-10-18T10:25:11.714 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:25:11.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3fa006ae40 msgr2=0x7f3fa006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3fa006ae40 0x7f3fa006d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8005c40 tx=0x7f3fb800a430 comp rx=0 tx=0).stop 2023-10-18T10:25:11.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 msgr2=0x7f3fbc13ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:11.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 0x7f3fbc13ed80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c9a0 tx=0x7f3fac00ce60 comp rx=0 tx=0).stop 2023-10-18T10:25:11.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 shutdown_connections 2023-10-18T10:25:11.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3fa006ae40 0x7f3fa006d300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fbc140660 0x7f3fbc13f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fbc0ca870 0x7f3fbc13ed80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 --2- 172.21.15.125:0/3117302292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fbc06d170 0x7f3fbc133550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:11.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 >> 172.21.15.125:0/3117302292 conn(0x7f3fbc069930 msgr2=0x7f3fbc157470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:11.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 shutdown_connections 2023-10-18T10:25:11.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:11.713+0000 7f3fc364f640 1 -- 172.21.15.125:0/3117302292 wait complete. 2023-10-18T10:25:11.726 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T10:25:11.726 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:25:11.726 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:25:11.727 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-18T10:25:11.727 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:25:11.727 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T10:25:11.728 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:25:11.728 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:25:11.728 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-18T10:25:11.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:11.729 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:25:11.744 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:11.744 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:11.744 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:11.745 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:11.745 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:11.745 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:11.745 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:11.745 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:11.746 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:25:11.746 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:11.746 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:11.746 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:25:11.747 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:11.747 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:11.747 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:25:11.747 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:11.747 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:25:11.748 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 42712sec preferred_lft 42712sec 2023-10-18T10:25:11.748 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:25:11.748 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:11.748 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T10:25:11.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:11.749 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:11.750 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:11.750 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link add name ceph-brx type bridge 2023-10-18T10:25:11.750 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip addr flush dev ceph-brx 2023-10-18T10:25:11.751 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set ceph-brx up 2023-10-18T10:25:11.751 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T10:25:11.751 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:11.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:11.905 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:25:11.932 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:25:11.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:11.934 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:25:11.997 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:25:11.997 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:25:11.997 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:25:11.997 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:25:11.998 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:25:11.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:11.999 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:11.999 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:11.999 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:25:12.000 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:25:12.000 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:25:12.000 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:12.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:12.259 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:25:12.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:12.274 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list-id 2023-10-18T10:25:12.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:12.329 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:12.329 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:12.329 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T10:25:12.330 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-18T10:25:12.330 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:12.412 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-18T10:25:12.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:12.413 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:12.413 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:12.414 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-18T10:25:12.414 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T10:25:12.414 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-18T10:25:12.415 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-18T10:25:12.415 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-18T10:25:12.415 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:12.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:12.578 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:12.578 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:12.578 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:25:12.579 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T10:25:12.579 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:12.620 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-18T10:25:12.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:12.621 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:25:12.674 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-18T10:25:12.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:12.675 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:25:12.728 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:25:12.796 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:12.851 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:25:12.851 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:25:12.851 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:25:12.852 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:25:12.852 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:25:12.852 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:25:12.852 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:25:12.852 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:25:12.853 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:25:12.853 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:25:12.853 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:25:12.853 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:25:12.853 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:12.853 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:25:12.854 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:25:12.854 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:25:12.854 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:25:12.854 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:12.854 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:12.855 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:12.855 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:25:12.855 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:12.855 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:12.855 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:25:12.856 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:25:12.856 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:25:12.856 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:25:12.856 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:25:12.856 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:25:12.856 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:25:12.857 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T10:25:12.857 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T10:25:12.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:12.858 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:25:12.911 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:25:12.912 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-18T10:25:12.954 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:25:12.980 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:13.035 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:25:13.035 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:25:13.036 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:25:13.036 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:25:13.036 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:25:13.036 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:25:13.036 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:25:13.037 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:25:13.037 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:25:13.037 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:25:13.037 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:25:13.037 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:25:13.038 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:13.038 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:25:13.038 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:25:13.038 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:25:13.039 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:25:13.039 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:13.039 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:13.039 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:13.039 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:25:13.040 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:13.040 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:13.040 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:25:13.040 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:25:13.040 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:25:13.040 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:25:13.041 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:25:13.041 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:25:13.041 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:25:13.041 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T10:25:13.041 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T10:25:13.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:13.042 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:25:13.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi125.stderr:2023-10-18T10:25:13.046+0000 7f541500f480 -1 init, newargv = 0x55f29feecc80 newargc=15 2023-10-18T10:25:13.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi125.stderr:ceph-fuse[96110]: starting ceph client 2023-10-18T10:25:13.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi125.stderr:ceph-fuse[96110]: starting fuse 2023-10-18T10:25:13.062 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:25:13.063 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:25:13.063 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:25:13.064 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:25:13.064 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:25:13.064 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:25:13.064 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:25:13.065 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:13.065 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:25:13.065 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-18T10:25:13.066 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:25:13.066 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-18T10:25:13.067 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:13.067 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:25:13.067 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:25:13.068 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:25:13.068 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:25:13.068 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:13.069 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:25:13.069 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:25:13.069 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:25:13.070 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:25:13.070 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:25:13.070 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:25:13.071 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:25:13.071 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:25:13.071 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:13.072 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.admin")) 2023-10-18T10:25:13.072 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:25:13.183 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.admin.96110.asok 2023-10-18T10:25:13.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:25:13.195 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.96110.asok 2023-10-18T10:25:13.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:13.195 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.96110.asok status 2023-10-18T10:25:13.329 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:25:13.329 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:25:13.329 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:25:13.330 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:25:13.330 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "admin", 2023-10-18T10:25:13.330 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:25:13.330 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-18T10:25:13.331 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "96110", 2023-10-18T10:25:13.331 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:25:13.331 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:13.331 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:25:13.331 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:25:13.331 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 4563, 2023-10-18T10:25:13.332 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:25:13.332 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:25:13.332 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:25:13.332 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 4563 2023-10-18T10:25:13.332 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:13.333 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:25:13.333 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:25:13.333 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:25:13.333 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 668939553 2023-10-18T10:25:13.333 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:25:13.334 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:13.334 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:25:13.334 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:25:13.334 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:25:13.334 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 668939553 2023-10-18T10:25:13.335 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:13.335 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.4563 192.168.0.1:0/668939553", 2023-10-18T10:25:13.335 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/668939553", 2023-10-18T10:25:13.335 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:25:13.335 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 7, 2023-10-18T10:25:13.336 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 25, 2023-10-18T10:25:13.336 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:25:13.336 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:25:13.336 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:25:13.336 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:25:13.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:13.338 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:25:13.338 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:25:13.353 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:25:13.353 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:25:13.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:13.354 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:25:13.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:13.421 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-18T10:25:13.492 INFO:teuthology.orchestra.run.smithi125.stdout:# file: . 2023-10-18T10:25:13.493 INFO:teuthology.orchestra.run.smithi125.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-18T10:25:13.493 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:13.493 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:25:13.494 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:13.547 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:13.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.688+0000 7f4d95f8a640 1 -- 172.21.15.125:0/4039587621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d7c003ef0 msgr2=0x7f4d7c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:13.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.688+0000 7f4d95f8a640 1 --2- 172.21.15.125:0/4039587621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d7c003ef0 0x7f4d7c0047c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009b50 tx=0x7f4d7c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.688+0000 7f4d8ffff640 1 -- 172.21.15.125:0/4039587621 reap_dead start 2023-10-18T10:25:13.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.688+0000 7f4d95f8a640 1 -- 172.21.15.125:0/4039587621 shutdown_connections 2023-10-18T10:25:13.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.688+0000 7f4d95f8a640 1 -- 172.21.15.125:0/4039587621 >> 172.21.15.125:0/4039587621 conn(0x7f4d90069930 msgr2=0x7f4d90069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:13.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.688+0000 7f4d95f8a640 1 -- 172.21.15.125:0/4039587621 shutdown_connections 2023-10-18T10:25:13.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.689+0000 7f4d95f8a640 1 -- 172.21.15.125:0/4039587621 wait complete. 2023-10-18T10:25:13.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.690+0000 7f4d95f8a640 1 Processor -- start 2023-10-18T10:25:13.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.690+0000 7f4d95f8a640 1 -- start start 2023-10-18T10:25:13.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.690+0000 7f4d95f8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d7c003ef0 0x7f4d90134260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:13.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.690+0000 7f4d95f8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d9015cbc0 0x7f4d9015d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:13.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.690+0000 7f4d95f8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d90134e00 0x7f4d901352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:13.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.690+0000 7f4d95f8a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9015d580 con 0x7f4d90134e00 2023-10-18T10:25:13.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.690+0000 7f4d95f8a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9013ccb0 con 0x7f4d9015cbc0 2023-10-18T10:25:13.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.690+0000 7f4d95f8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9013cf90 con 0x7f4d7c003ef0 2023-10-18T10:25:13.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d90134e00 0x7f4d901352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:13.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d90134e00 0x7f4d901352b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40274/0 (socket says 172.21.15.125:40274) 2023-10-18T10:25:13.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 -- 172.21.15.125:0/1838096110 learned_addr learned my addr 172.21.15.125:0/1838096110 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:13.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d86ffd640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d9015cbc0 0x7f4d9015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:13.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8f7fe640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d7c003ef0 0x7f4d90134260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:13.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 -- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d7c003ef0 msgr2=0x7f4d90134260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:13.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d7c003ef0 0x7f4d90134260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 -- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d9015cbc0 msgr2=0x7f4d9015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:13.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d9015cbc0 0x7f4d9015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 -- 172.21.15.125:0/1838096110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9013d270 con 0x7f4d90134e00 2023-10-18T10:25:13.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8f7fe640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d7c003ef0 0x7f4d90134260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:13.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8ffff640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d90134e00 0x7f4d901352b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4d900cc150 tx=0x7f4d88011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:13.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8cff9640 1 -- 172.21.15.125:0/1838096110 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800fd40 con 0x7f4d90134e00 2023-10-18T10:25:13.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8cff9640 1 -- 172.21.15.125:0/1838096110 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d8800fea0 con 0x7f4d90134e00 2023-10-18T10:25:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9013d550 con 0x7f4d90134e00 2023-10-18T10:25:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d8cff9640 1 -- 172.21.15.125:0/1838096110 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88004d80 con 0x7f4d90134e00 2023-10-18T10:25:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.691+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d901347a0 con 0x7f4d90134e00 2023-10-18T10:25:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.692+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d900690e0 con 0x7f4d90134e00 2023-10-18T10:25:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.693+0000 7f4d8cff9640 1 -- 172.21.15.125:0/1838096110 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4d88019410 con 0x7f4d90134e00 2023-10-18T10:25:13.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.693+0000 7f4d8cff9640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d7006ae70 0x7f4d7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:13.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.693+0000 7f4d8cff9640 1 -- 172.21.15.125:0/1838096110 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4d8808a130 con 0x7f4d90134e00 2023-10-18T10:25:13.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.693+0000 7f4d8f7fe640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d7006ae70 0x7f4d7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:13.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.694+0000 7f4d8f7fe640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d7006ae70 0x7f4d7006d330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d78000fb0 tx=0x7f4d780067a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:13.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.697+0000 7f4d8cff9640 1 -- 172.21.15.125:0/1838096110 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d88054ef0 con 0x7f4d90134e00 2023-10-18T10:25:13.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.864+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d900cc420 con 0x7f4d90134e00 2023-10-18T10:25:13.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.865+0000 7f4d8cff9640 1 -- 172.21.15.125:0/1838096110 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f4d88058510 con 0x7f4d90134e00 2023-10-18T10:25:13.867 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:13.868 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:13.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d7006ae70 msgr2=0x7f4d7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:13.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d7006ae70 0x7f4d7006d330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d78000fb0 tx=0x7f4d780067a0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d90134e00 msgr2=0x7f4d901352b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:13.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d90134e00 0x7f4d901352b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4d900cc150 tx=0x7f4d88011f90 comp rx=0 tx=0).stop 2023-10-18T10:25:13.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 shutdown_connections 2023-10-18T10:25:13.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d7006ae70 0x7f4d7006d330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d9015cbc0 0x7f4d9015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d90134e00 0x7f4d901352b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 --2- 172.21.15.125:0/1838096110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d7c003ef0 0x7f4d90134260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 >> 172.21.15.125:0/1838096110 conn(0x7f4d90069930 msgr2=0x7f4d90152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:13.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 shutdown_connections 2023-10-18T10:25:13.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.867+0000 7f4d95f8a640 1 -- 172.21.15.125:0/1838096110 wait complete. 2023-10-18T10:25:13.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:13.886 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:13.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.990+0000 7efe7d22b640 1 -- 172.21.15.125:0/2205267458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 msgr2=0x7efe780b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:13.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.990+0000 7efe7d22b640 1 --2- 172.21.15.125:0/2205267458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 0x7efe780b0e90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe7806b930 tx=0x7efe6802be40 comp rx=0 tx=0).stop 2023-10-18T10:25:13.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.990+0000 7efe7d22b640 1 -- 172.21.15.125:0/2205267458 shutdown_connections 2023-10-18T10:25:13.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.990+0000 7efe7d22b640 1 --2- 172.21.15.125:0/2205267458 >> v2:172.21.15.139:3300/0 conn(0x7efe7815c1f0 0x7efe7806a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.990+0000 7efe7d22b640 1 --2- 172.21.15.125:0/2205267458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 0x7efe780b0e90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:13.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.990+0000 7efe7d22b640 1 -- 172.21.15.125:0/2205267458 >> 172.21.15.125:0/2205267458 conn(0x7efe7806fae0 msgr2=0x7efe7806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:13.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.991+0000 7efe7d22b640 1 -- 172.21.15.125:0/2205267458 shutdown_connections 2023-10-18T10:25:13.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.991+0000 7efe7d22b640 1 -- 172.21.15.125:0/2205267458 wait complete. 2023-10-18T10:25:13.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.992+0000 7efe7d22b640 1 Processor -- start 2023-10-18T10:25:13.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.992+0000 7efe7d22b640 1 -- start start 2023-10-18T10:25:13.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.992+0000 7efe7d22b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe7806ac70 0x7efe781edff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:13.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe7d22b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 0x7efe78146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:13.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe7d22b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7815c1f0 0x7efe781ef200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:13.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe7d22b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7814d6d0 con 0x7efe781579f0 2023-10-18T10:25:13.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe7d22b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe781ee5e0 con 0x7efe7815c1f0 2023-10-18T10:25:13.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe7d22b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe781ee8c0 con 0x7efe7806ac70 2023-10-18T10:25:13.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 0x7efe78146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 0x7efe78146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40290/0 (socket says 172.21.15.125:40290) 2023-10-18T10:25:14.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 -- 172.21.15.125:0/3376399822 learned_addr learned my addr 172.21.15.125:0/3376399822 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:14.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe76d9d640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe7806ac70 0x7efe781edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe7759e640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7815c1f0 0x7efe781ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 -- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe7806ac70 msgr2=0x7efe781edff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe7806ac70 0x7efe781edff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 -- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7815c1f0 msgr2=0x7efe781ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7815c1f0 0x7efe781ef200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 -- 172.21.15.125:0/3376399822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68009560 con 0x7efe781579f0 2023-10-18T10:25:14.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6ffff640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 0x7efe78146590 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe6000e790 tx=0x7efe6000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:14.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6f7fe640 1 -- 172.21.15.125:0/3376399822 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60009580 con 0x7efe781579f0 2023-10-18T10:25:14.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6f7fe640 1 -- 172.21.15.125:0/3376399822 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efe6000cdb0 con 0x7efe781579f0 2023-10-18T10:25:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe781eeba0 con 0x7efe781579f0 2023-10-18T10:25:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe6f7fe640 1 -- 172.21.15.125:0/3376399822 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60010690 con 0x7efe781579f0 2023-10-18T10:25:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.993+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe781ef740 con 0x7efe781579f0 2023-10-18T10:25:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.994+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe781e2f90 con 0x7efe781579f0 2023-10-18T10:25:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.995+0000 7efe6f7fe640 1 -- 172.21.15.125:0/3376399822 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efe600107f0 con 0x7efe781579f0 2023-10-18T10:25:14.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.995+0000 7efe6f7fe640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efe5806ae40 0x7efe5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.995+0000 7efe6f7fe640 1 -- 172.21.15.125:0/3376399822 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efe60014070 con 0x7efe781579f0 2023-10-18T10:25:14.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.995+0000 7efe76d9d640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efe5806ae40 0x7efe5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.996+0000 7efe76d9d640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efe5806ae40 0x7efe5806d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe68035000 tx=0x7efe68004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:14.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:13.999+0000 7efe6f7fe640 1 -- 172.21.15.125:0/3376399822 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe60051ca0 con 0x7efe781579f0 2023-10-18T10:25:14.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.165+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe78070d80 con 0x7efe781579f0 2023-10-18T10:25:14.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.165+0000 7efe6f7fe640 1 -- 172.21.15.125:0/3376399822 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe60055130 con 0x7efe781579f0 2023-10-18T10:25:14.167 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:14.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efe5806ae40 msgr2=0x7efe5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efe5806ae40 0x7efe5806d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe68035000 tx=0x7efe68004730 comp rx=0 tx=0).stop 2023-10-18T10:25:14.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 msgr2=0x7efe78146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 0x7efe78146590 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe6000e790 tx=0x7efe6000ec50 comp rx=0 tx=0).stop 2023-10-18T10:25:14.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 shutdown_connections 2023-10-18T10:25:14.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efe5806ae40 0x7efe5806d300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7815c1f0 0x7efe781ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe781579f0 0x7efe78146590 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 --2- 172.21.15.125:0/3376399822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe7806ac70 0x7efe781edff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 >> 172.21.15.125:0/3376399822 conn(0x7efe7806fae0 msgr2=0x7efe7815a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:14.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 shutdown_connections 2023-10-18T10:25:14.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.167+0000 7efe7d22b640 1 -- 172.21.15.125:0/3376399822 wait complete. 2023-10-18T10:25:14.179 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:14.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.282+0000 7f6d9587c640 1 -- 172.21.15.125:0/3480318976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d9015d400 msgr2=0x7f6d900cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.282+0000 7f6d9587c640 1 --2- 172.21.15.125:0/3480318976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d9015d400 0x7f6d900cdfe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005fd0 tx=0x7f6d8802bd90 comp rx=0 tx=0).stop 2023-10-18T10:25:14.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.282+0000 7f6d9587c640 1 -- 172.21.15.125:0/3480318976 shutdown_connections 2023-10-18T10:25:14.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.282+0000 7f6d9587c640 1 --2- 172.21.15.125:0/3480318976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d9015d400 0x7f6d900cdfe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.282+0000 7f6d9587c640 1 --2- 172.21.15.125:0/3480318976 >> v2:172.21.15.139:3300/0 conn(0x7f6d90159060 0x7f6d9006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.282+0000 7f6d9587c640 1 -- 172.21.15.125:0/3480318976 >> 172.21.15.125:0/3480318976 conn(0x7f6d900b0670 msgr2=0x7f6d900b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:14.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.282+0000 7f6d9587c640 1 -- 172.21.15.125:0/3480318976 shutdown_connections 2023-10-18T10:25:14.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.282+0000 7f6d9587c640 1 -- 172.21.15.125:0/3480318976 wait complete. 2023-10-18T10:25:14.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.283+0000 7f6d9587c640 1 Processor -- start 2023-10-18T10:25:14.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.283+0000 7f6d9587c640 1 -- start start 2023-10-18T10:25:14.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d9587c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d900cdb30 0x7f6d900c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d9587c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d90159060 0x7f6d90150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d9587c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d9015d400 0x7f6d900cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d9587c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d901512f0 con 0x7f6d90159060 2023-10-18T10:25:14.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d9587c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900b1800 con 0x7f6d9015d400 2023-10-18T10:25:14.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d9587c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900c5100 con 0x7f6d900cdb30 2023-10-18T10:25:14.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d90159060 0x7f6d90150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d90159060 0x7f6d90150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40304/0 (socket says 172.21.15.125:40304) 2023-10-18T10:25:14.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 -- 172.21.15.125:0/241125823 learned_addr learned my addr 172.21.15.125:0/241125823 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:14.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8f7fe640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d9015d400 0x7f6d900cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8effd640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d900cdb30 0x7f6d900c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 -- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d900cdb30 msgr2=0x7f6d900c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d900cdb30 0x7f6d900c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 -- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d9015d400 msgr2=0x7f6d900cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d9015d400 0x7f6d900cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 -- 172.21.15.125:0/241125823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88009560 con 0x7f6d90159060 2023-10-18T10:25:14.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8f7fe640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d9015d400 0x7f6d900cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:14.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.284+0000 7f6d8e7fc640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d90159060 0x7f6d90150db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c460 tx=0x7f6d8400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:14.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.285+0000 7f6d77fff640 1 -- 172.21.15.125:0/241125823 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84005480 con 0x7f6d90159060 2023-10-18T10:25:14.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.285+0000 7f6d77fff640 1 -- 172.21.15.125:0/241125823 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d840035f0 con 0x7f6d90159060 2023-10-18T10:25:14.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.285+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d900b1aa0 con 0x7f6d90159060 2023-10-18T10:25:14.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.285+0000 7f6d77fff640 1 -- 172.21.15.125:0/241125823 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84003a00 con 0x7f6d90159060 2023-10-18T10:25:14.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.285+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d900c53e0 con 0x7f6d90159060 2023-10-18T10:25:14.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.286+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d900b1fc0 con 0x7f6d90159060 2023-10-18T10:25:14.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.286+0000 7f6d77fff640 1 -- 172.21.15.125:0/241125823 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6d84003b60 con 0x7f6d90159060 2023-10-18T10:25:14.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.286+0000 7f6d77fff640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6d7c06ae40 0x7f6d7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.287+0000 7f6d77fff640 1 -- 172.21.15.125:0/241125823 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6d84089590 con 0x7f6d90159060 2023-10-18T10:25:14.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.287+0000 7f6d8effd640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6d7c06ae40 0x7f6d7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.287+0000 7f6d8effd640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6d7c06ae40 0x7f6d7c06d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d80005c40 tx=0x7f6d8000a780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:14.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.291+0000 7f6d77fff640 1 -- 172.21.15.125:0/241125823 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d900b1fc0 con 0x7f6d90159060 2023-10-18T10:25:14.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.455+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d900b1fc0 con 0x7f6d90159060 2023-10-18T10:25:14.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.455+0000 7f6d77fff640 1 -- 172.21.15.125:0/241125823 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d900b1fc0 con 0x7f6d90159060 2023-10-18T10:25:14.457 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:14.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6d7c06ae40 msgr2=0x7f6d7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6d7c06ae40 0x7f6d7c06d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d80005c40 tx=0x7f6d8000a780 comp rx=0 tx=0).stop 2023-10-18T10:25:14.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d90159060 msgr2=0x7f6d90150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d90159060 0x7f6d90150db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c460 tx=0x7f6d8400c920 comp rx=0 tx=0).stop 2023-10-18T10:25:14.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 shutdown_connections 2023-10-18T10:25:14.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6d7c06ae40 0x7f6d7c06d300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d9015d400 0x7f6d900cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d90159060 0x7f6d90150db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 --2- 172.21.15.125:0/241125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d900cdb30 0x7f6d900c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 >> 172.21.15.125:0/241125823 conn(0x7f6d900b0670 msgr2=0x7f6d900ab9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:14.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 shutdown_connections 2023-10-18T10:25:14.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.457+0000 7f6d9587c640 1 -- 172.21.15.125:0/241125823 wait complete. 2023-10-18T10:25:14.469 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:14.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.573+0000 7fbef9c6f640 1 -- 172.21.15.125:0/2941029726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbee0003970 msgr2=0x7fbee0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.573+0000 7fbef9c6f640 1 --2- 172.21.15.125:0/2941029726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbee0003970 0x7fbee0003d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009b50 tx=0x7fbee00057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.573+0000 7fbef3fff640 1 -- 172.21.15.125:0/2941029726 reap_dead start 2023-10-18T10:25:14.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.573+0000 7fbef9c6f640 1 -- 172.21.15.125:0/2941029726 shutdown_connections 2023-10-18T10:25:14.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.573+0000 7fbef9c6f640 1 -- 172.21.15.125:0/2941029726 >> 172.21.15.125:0/2941029726 conn(0x7fbef40b0670 msgr2=0x7fbef40b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:14.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.573+0000 7fbef9c6f640 1 -- 172.21.15.125:0/2941029726 shutdown_connections 2023-10-18T10:25:14.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.573+0000 7fbef9c6f640 1 -- 172.21.15.125:0/2941029726 wait complete. 2023-10-18T10:25:14.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef9c6f640 1 Processor -- start 2023-10-18T10:25:14.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef9c6f640 1 -- start start 2023-10-18T10:25:14.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef9c6f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbee0003970 0x7fbef41504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef9c6f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef4150c40 0x7fbef40ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef9c6f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef40c7080 0x7fbef40acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef9c6f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415a680 con 0x7fbef40c7080 2023-10-18T10:25:14.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef9c6f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef41509f0 con 0x7fbef4150c40 2023-10-18T10:25:14.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef9c6f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4159a70 con 0x7fbee0003970 2023-10-18T10:25:14.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef40c7080 0x7fbef40acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef40c7080 0x7fbef40acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40312/0 (socket says 172.21.15.125:40312) 2023-10-18T10:25:14.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.575+0000 7fbef3fff640 1 -- 172.21.15.125:0/1357502641 learned_addr learned my addr 172.21.15.125:0/1357502641 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:14.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbeebfff640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef4150c40 0x7fbef40ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef3fff640 1 -- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbee0003970 msgr2=0x7fbef41504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef37fe640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbee0003970 0x7fbef41504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef3fff640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbee0003970 0x7fbef41504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef3fff640 1 -- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef4150c40 msgr2=0x7fbef40ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef3fff640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef4150c40 0x7fbef40ac9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef3fff640 1 -- 172.21.15.125:0/1357502641 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4159d50 con 0x7fbef40c7080 2023-10-18T10:25:14.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef3fff640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef40c7080 0x7fbef40acee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef415d7e0 tx=0x7fbeec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:14.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef0ff9640 1 -- 172.21.15.125:0/1357502641 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004d20 con 0x7fbef40c7080 2023-10-18T10:25:14.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef0ff9640 1 -- 172.21.15.125:0/1357502641 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbeec004e80 con 0x7fbef40c7080 2023-10-18T10:25:14.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef415a030 con 0x7fbef40c7080 2023-10-18T10:25:14.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef0ff9640 1 -- 172.21.15.125:0/1357502641 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec005260 con 0x7fbef40c7080 2023-10-18T10:25:14.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.576+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef4159720 con 0x7fbef40c7080 2023-10-18T10:25:14.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.577+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef415ab20 con 0x7fbef40c7080 2023-10-18T10:25:14.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.577+0000 7fbef0ff9640 1 -- 172.21.15.125:0/1357502641 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbeec0053c0 con 0x7fbef40c7080 2023-10-18T10:25:14.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.578+0000 7fbef0ff9640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbed406ae70 0x7fbed406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.578+0000 7fbef0ff9640 1 -- 172.21.15.125:0/1357502641 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbeec08a5f0 con 0x7fbef40c7080 2023-10-18T10:25:14.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.578+0000 7fbef37fe640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbed406ae70 0x7fbed406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.578+0000 7fbef37fe640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbed406ae70 0x7fbed406d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009b50 tx=0x7fbee4035040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:14.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.582+0000 7fbef0ff9640 1 -- 172.21.15.125:0/1357502641 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbeec055440 con 0x7fbef40c7080 2023-10-18T10:25:14.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.748+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbef40cc4e0 con 0x7fbef40c7080 2023-10-18T10:25:14.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.749+0000 7fbef0ff9640 1 -- 172.21.15.125:0/1357502641 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbeec00eea0 con 0x7fbef40c7080 2023-10-18T10:25:14.751 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:14.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbed406ae70 msgr2=0x7fbed406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbed406ae70 0x7fbed406d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009b50 tx=0x7fbee4035040 comp rx=0 tx=0).stop 2023-10-18T10:25:14.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef40c7080 msgr2=0x7fbef40acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef40c7080 0x7fbef40acee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef415d7e0 tx=0x7fbeec00e720 comp rx=0 tx=0).stop 2023-10-18T10:25:14.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 shutdown_connections 2023-10-18T10:25:14.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbed406ae70 0x7fbed406d330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef4150c40 0x7fbef40ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbef40c7080 0x7fbef40acee0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 --2- 172.21.15.125:0/1357502641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbee0003970 0x7fbef41504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.750+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 >> 172.21.15.125:0/1357502641 conn(0x7fbef40b0670 msgr2=0x7fbef40ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:14.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.751+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 shutdown_connections 2023-10-18T10:25:14.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.751+0000 7fbef9c6f640 1 -- 172.21.15.125:0/1357502641 wait complete. 2023-10-18T10:25:14.763 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:14.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.865+0000 7fe51b639640 1 -- 172.21.15.125:0/1981121838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 msgr2=0x7fe5140ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.865+0000 7fe51b639640 1 --2- 172.21.15.125:0/1981121838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 0x7fe5140ab9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe50c005fd0 tx=0x7fe50c02c2e0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.866+0000 7fe51b639640 1 -- 172.21.15.125:0/1981121838 shutdown_connections 2023-10-18T10:25:14.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.866+0000 7fe51b639640 1 --2- 172.21.15.125:0/1981121838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 0x7fe5140ab9b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.866+0000 7fe51b639640 1 --2- 172.21.15.125:0/1981121838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5140c9dc0 0x7fe51406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.866+0000 7fe51b639640 1 -- 172.21.15.125:0/1981121838 >> 172.21.15.125:0/1981121838 conn(0x7fe514069930 msgr2=0x7fe514069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:14.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.866+0000 7fe51b639640 1 -- 172.21.15.125:0/1981121838 shutdown_connections 2023-10-18T10:25:14.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.866+0000 7fe51b639640 1 -- 172.21.15.125:0/1981121838 wait complete. 2023-10-18T10:25:14.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.867+0000 7fe51b639640 1 Processor -- start 2023-10-18T10:25:14.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.867+0000 7fe51b639640 1 -- start start 2023-10-18T10:25:14.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.867+0000 7fe51b639640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe51406af00 0x7fe5141f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.867+0000 7fe51b639640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 0x7fe51413a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.867+0000 7fe51b639640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5140c9dc0 0x7fe5141f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.867+0000 7fe51b639640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5141f21e0 con 0x7fe5140ab500 2023-10-18T10:25:14.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.867+0000 7fe51b639640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe514158620 con 0x7fe51406af00 2023-10-18T10:25:14.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.867+0000 7fe51b639640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140aedf0 con 0x7fe5140c9dc0 2023-10-18T10:25:14.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 0x7fe51413a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 0x7fe51413a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40324/0 (socket says 172.21.15.125:40324) 2023-10-18T10:25:14.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 -- 172.21.15.125:0/2953920871 learned_addr learned my addr 172.21.15.125:0/2953920871 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:14.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe519bd6640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5140c9dc0 0x7fe5141f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe5193d5640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe51406af00 0x7fe5141f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 -- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5140c9dc0 msgr2=0x7fe5141f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5140c9dc0 0x7fe5141f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 -- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe51406af00 msgr2=0x7fe5141f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:14.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe51406af00 0x7fe5141f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:14.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 -- 172.21.15.125:0/2953920871 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c009900 con 0x7fe5140ab500 2023-10-18T10:25:14.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe519bd6640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5140c9dc0 0x7fe5141f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:14.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe5193d5640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe51406af00 0x7fe5141f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:14.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe518bd4640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 0x7fe51413a2c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe50400c970 tx=0x7fe50400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:14.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe509ffb640 1 -- 172.21.15.125:0/2953920871 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504007bb0 con 0x7fe5140ab500 2023-10-18T10:25:14.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe509ffb640 1 -- 172.21.15.125:0/2953920871 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe504007e00 con 0x7fe5140ab500 2023-10-18T10:25:14.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51413a800 con 0x7fe5140ab500 2023-10-18T10:25:14.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe509ffb640 1 -- 172.21.15.125:0/2953920871 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504005280 con 0x7fe5140ab500 2023-10-18T10:25:14.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.868+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5140aeb10 con 0x7fe5140ab500 2023-10-18T10:25:14.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.869+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51406c500 con 0x7fe5140ab500 2023-10-18T10:25:14.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.870+0000 7fe509ffb640 1 -- 172.21.15.125:0/2953920871 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe5040053e0 con 0x7fe5140ab500 2023-10-18T10:25:14.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.870+0000 7fe509ffb640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe4f806ae40 0x7fe4f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:14.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.870+0000 7fe509ffb640 1 -- 172.21.15.125:0/2953920871 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe504089d10 con 0x7fe5140ab500 2023-10-18T10:25:14.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.870+0000 7fe5193d5640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe4f806ae40 0x7fe4f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:14.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.871+0000 7fe5193d5640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe4f806ae40 0x7fe4f806d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe510000f60 tx=0x7fe510009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:14.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:14.874+0000 7fe509ffb640 1 -- 172.21.15.125:0/2953920871 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe504054c20 con 0x7fe5140ab500 2023-10-18T10:25:15.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.038+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe5140690e0 con 0x7fe5140ab500 2023-10-18T10:25:15.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.038+0000 7fe509ffb640 1 -- 172.21.15.125:0/2953920871 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe50401d070 con 0x7fe5140ab500 2023-10-18T10:25:15.041 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:15.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe4f806ae40 msgr2=0x7fe4f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe4f806ae40 0x7fe4f806d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe510000f60 tx=0x7fe510009040 comp rx=0 tx=0).stop 2023-10-18T10:25:15.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 msgr2=0x7fe51413a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 0x7fe51413a2c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe50400c970 tx=0x7fe50400ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:15.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 shutdown_connections 2023-10-18T10:25:15.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe4f806ae40 0x7fe4f806d300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5140c9dc0 0x7fe5141f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5140ab500 0x7fe51413a2c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 --2- 172.21.15.125:0/2953920871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe51406af00 0x7fe5141f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 >> 172.21.15.125:0/2953920871 conn(0x7fe514069930 msgr2=0x7fe5140cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:15.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 shutdown_connections 2023-10-18T10:25:15.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.040+0000 7fe51b639640 1 -- 172.21.15.125:0/2953920871 wait complete. 2023-10-18T10:25:15.052 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:25:15.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.154+0000 7f607d925640 1 -- 172.21.15.125:0/3364887108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60780704e0 msgr2=0x7f60780cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.154+0000 7f607d925640 1 --2- 172.21.15.125:0/3364887108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60780704e0 0x7f60780cc910 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6070009be0 tx=0x7f607002c160 comp rx=0 tx=0).stop 2023-10-18T10:25:15.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.154+0000 7f607d925640 1 -- 172.21.15.125:0/3364887108 shutdown_connections 2023-10-18T10:25:15.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.154+0000 7f607d925640 1 --2- 172.21.15.125:0/3364887108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60780704e0 0x7f60780cc910 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.154+0000 7f607d925640 1 -- 172.21.15.125:0/3364887108 >> 172.21.15.125:0/3364887108 conn(0x7f60780af730 msgr2=0x7f60780afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:15.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.154+0000 7f607d925640 1 -- 172.21.15.125:0/3364887108 shutdown_connections 2023-10-18T10:25:15.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.154+0000 7f607d925640 1 -- 172.21.15.125:0/3364887108 wait complete. 2023-10-18T10:25:15.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.155+0000 7f607d925640 1 Processor -- start 2023-10-18T10:25:15.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.155+0000 7f607d925640 1 -- start start 2023-10-18T10:25:15.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f607d925640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078070080 0x7f60781f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f607d925640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780704e0 0x7f6078158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f607d925640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60781416f0 0x7f6078069720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f607d925640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6078158840 con 0x7f60781416f0 2023-10-18T10:25:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f607d925640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781f5660 con 0x7f60780704e0 2023-10-18T10:25:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f607d925640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60781f5940 con 0x7f6078070080 2023-10-18T10:25:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f60777fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60781416f0 0x7f6078069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f60777fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60781416f0 0x7f6078069720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40326/0 (socket says 172.21.15.125:40326) 2023-10-18T10:25:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.156+0000 7f60777fe640 1 -- 172.21.15.125:0/2326672585 learned_addr learned my addr 172.21.15.125:0/2326672585 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f6076ffd640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078070080 0x7f60781f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f60767fc640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780704e0 0x7f6078158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f60777fe640 1 -- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078070080 msgr2=0x7f60781f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f60777fe640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078070080 0x7f60781f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f60777fe640 1 -- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780704e0 msgr2=0x7f6078158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f60777fe640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780704e0 0x7f6078158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f60777fe640 1 -- 172.21.15.125:0/2326672585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070009840 con 0x7f60781416f0 2023-10-18T10:25:15.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f6076ffd640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078070080 0x7f60781f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:15.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f60767fc640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780704e0 0x7f6078158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:15.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f60777fe640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60781416f0 0x7f6078069720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6070036040 tx=0x7f6070005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:15.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f605ffff640 1 -- 172.21.15.125:0/2326672585 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070003e60 con 0x7f60781416f0 2023-10-18T10:25:15.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f605ffff640 1 -- 172.21.15.125:0/2326672585 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6070004790 con 0x7f60781416f0 2023-10-18T10:25:15.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60780b1cd0 con 0x7f60781416f0 2023-10-18T10:25:15.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f605ffff640 1 -- 172.21.15.125:0/2326672585 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070034800 con 0x7f60781416f0 2023-10-18T10:25:15.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.157+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60781584d0 con 0x7f60781416f0 2023-10-18T10:25:15.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.158+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60781589f0 con 0x7f60781416f0 2023-10-18T10:25:15.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.158+0000 7f605ffff640 1 -- 172.21.15.125:0/2326672585 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6070034960 con 0x7f60781416f0 2023-10-18T10:25:15.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.159+0000 7f605ffff640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f606406ae70 0x7f606406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.159+0000 7f605ffff640 1 -- 172.21.15.125:0/2326672585 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f60700ac9e0 con 0x7f60781416f0 2023-10-18T10:25:15.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.159+0000 7f6076ffd640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f606406ae70 0x7f606406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.159+0000 7f6076ffd640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f606406ae70 0x7f606406d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f606c0011e0 tx=0x7f606c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:15.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.163+0000 7f605ffff640 1 -- 172.21.15.125:0/2326672585 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60700777b0 con 0x7f60781416f0 2023-10-18T10:25:15.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.341+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f60780675f0 con 0x7f60781416f0 2023-10-18T10:25:15.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.342+0000 7f605ffff640 1 -- 172.21.15.125:0/2326672585 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f607007add0 con 0x7f60781416f0 2023-10-18T10:25:15.344 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:25:15.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f606406ae70 msgr2=0x7f606406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f606406ae70 0x7f606406d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f606c0011e0 tx=0x7f606c009040 comp rx=0 tx=0).stop 2023-10-18T10:25:15.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60781416f0 msgr2=0x7f6078069720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60781416f0 0x7f6078069720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6070036040 tx=0x7f6070005c80 comp rx=0 tx=0).stop 2023-10-18T10:25:15.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 shutdown_connections 2023-10-18T10:25:15.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f606406ae70 0x7f606406d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780704e0 0x7f6078158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60781416f0 0x7f6078069720 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 --2- 172.21.15.125:0/2326672585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078070080 0x7f60781f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.343+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 >> 172.21.15.125:0/2326672585 conn(0x7f60780af730 msgr2=0x7f60780adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:15.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.344+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 shutdown_connections 2023-10-18T10:25:15.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.344+0000 7f607d925640 1 -- 172.21.15.125:0/2326672585 wait complete. 2023-10-18T10:25:15.355 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:25:15.356 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:15.371 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:15.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.513+0000 7fd8353a9640 1 -- 172.21.15.125:0/2836864785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83015b2a0 msgr2=0x7fd8300cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.513+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2836864785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83015b2a0 0x7fd8300cf720 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd818005fd0 tx=0x7fd81802bd90 comp rx=0 tx=0).stop 2023-10-18T10:25:15.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.513+0000 7fd8353a9640 1 -- 172.21.15.125:0/2836864785 shutdown_connections 2023-10-18T10:25:15.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.513+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2836864785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd83015b2a0 0x7fd8300cf720 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.513+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2836864785 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8300cf340 0x7fd8300d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.513+0000 7fd8353a9640 1 -- 172.21.15.125:0/2836864785 >> 172.21.15.125:0/2836864785 conn(0x7fd8300d0d60 msgr2=0x7fd8300d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:15.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.513+0000 7fd8353a9640 1 -- 172.21.15.125:0/2836864785 shutdown_connections 2023-10-18T10:25:15.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.513+0000 7fd8353a9640 1 -- 172.21.15.125:0/2836864785 wait complete. 2023-10-18T10:25:15.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.514+0000 7fd8353a9640 1 Processor -- start 2023-10-18T10:25:15.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.514+0000 7fd8353a9640 1 -- start start 2023-10-18T10:25:15.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd8353a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8300ceb40 0x7fd830158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd8353a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8300cf340 0x7fd83006a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd8353a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd83015b2a0 0x7fd83006af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd8353a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83006b440 con 0x7fd8300ceb40 2023-10-18T10:25:15.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd8353a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300b5bf0 con 0x7fd8300cf340 2023-10-18T10:25:15.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd8353a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd830158a40 con 0x7fd83015b2a0 2023-10-18T10:25:15.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8300ceb40 0x7fd830158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8300ceb40 0x7fd830158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40338/0 (socket says 172.21.15.125:40338) 2023-10-18T10:25:15.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 -- 172.21.15.125:0/2872507292 learned_addr learned my addr 172.21.15.125:0/2872507292 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:15.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82f7fe640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd83015b2a0 0x7fd83006af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd827fff640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8300cf340 0x7fd83006a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 -- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd83015b2a0 msgr2=0x7fd83006af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd83015b2a0 0x7fd83006af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 -- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8300cf340 msgr2=0x7fd83006a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8300cf340 0x7fd83006a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 -- 172.21.15.125:0/2872507292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd818009560 con 0x7fd8300ceb40 2023-10-18T10:25:15.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd827fff640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8300cf340 0x7fd83006a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:15.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.515+0000 7fd82effd640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8300ceb40 0x7fd830158500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd82000e9f0 tx=0x7fd82000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:15.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.516+0000 7fd8277fe640 1 -- 172.21.15.125:0/2872507292 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820009700 con 0x7fd8300ceb40 2023-10-18T10:25:15.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.516+0000 7fd8277fe640 1 -- 172.21.15.125:0/2872507292 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd82000ce70 con 0x7fd8300ceb40 2023-10-18T10:25:15.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.516+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8300b5e40 con 0x7fd8300ceb40 2023-10-18T10:25:15.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.516+0000 7fd8277fe640 1 -- 172.21.15.125:0/2872507292 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820010670 con 0x7fd8300ceb40 2023-10-18T10:25:15.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.516+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8300b6400 con 0x7fd8300ceb40 2023-10-18T10:25:15.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.517+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8300cadb0 con 0x7fd8300ceb40 2023-10-18T10:25:15.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.517+0000 7fd8277fe640 1 -- 172.21.15.125:0/2872507292 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd8200107d0 con 0x7fd8300ceb40 2023-10-18T10:25:15.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.517+0000 7fd8277fe640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81006ae40 0x7fd81006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.517+0000 7fd8277fe640 1 -- 172.21.15.125:0/2872507292 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd820014070 con 0x7fd8300ceb40 2023-10-18T10:25:15.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.518+0000 7fd827fff640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81006ae40 0x7fd81006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.518+0000 7fd827fff640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81006ae40 0x7fd81006d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd818035000 tx=0x7fd818004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:15.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.522+0000 7fd8277fe640 1 -- 172.21.15.125:0/2872507292 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd820051ca0 con 0x7fd8300ceb40 2023-10-18T10:25:15.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.687+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8300690e0 con 0x7fd8300ceb40 2023-10-18T10:25:15.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.688+0000 7fd8277fe640 1 -- 172.21.15.125:0/2872507292 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7fd820058e40 con 0x7fd8300ceb40 2023-10-18T10:25:15.691 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:15.691 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:15.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81006ae40 msgr2=0x7fd81006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81006ae40 0x7fd81006d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd818035000 tx=0x7fd818004730 comp rx=0 tx=0).stop 2023-10-18T10:25:15.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8300ceb40 msgr2=0x7fd830158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8300ceb40 0x7fd830158500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd82000e9f0 tx=0x7fd82000eeb0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 shutdown_connections 2023-10-18T10:25:15.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81006ae40 0x7fd81006d300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8300cf340 0x7fd83006a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd83015b2a0 0x7fd83006af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 --2- 172.21.15.125:0/2872507292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8300ceb40 0x7fd830158500 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 >> 172.21.15.125:0/2872507292 conn(0x7fd8300d0d60 msgr2=0x7fd830159650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:15.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 shutdown_connections 2023-10-18T10:25:15.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.690+0000 7fd8353a9640 1 -- 172.21.15.125:0/2872507292 wait complete. 2023-10-18T10:25:15.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:15.704 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:15.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.806+0000 7f4a075ce640 1 -- 172.21.15.125:0/3585408190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a0015b2a0 msgr2=0x7f4a000d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.806+0000 7f4a075ce640 1 --2- 172.21.15.125:0/3585408190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a0015b2a0 0x7f4a000d1910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a0015b680 tx=0x7f49f002bd30 comp rx=0 tx=0).stop 2023-10-18T10:25:15.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.806+0000 7f4a075ce640 1 -- 172.21.15.125:0/3585408190 shutdown_connections 2023-10-18T10:25:15.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.806+0000 7f4a075ce640 1 --2- 172.21.15.125:0/3585408190 >> v2:172.21.15.139:3300/0 conn(0x7f4a000ceb40 0x7f4a000b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.806+0000 7f4a075ce640 1 --2- 172.21.15.125:0/3585408190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a0015b2a0 0x7f4a000d1910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.806+0000 7f4a075ce640 1 -- 172.21.15.125:0/3585408190 >> 172.21.15.125:0/3585408190 conn(0x7f4a000d0d60 msgr2=0x7f4a000d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:15.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.806+0000 7f4a075ce640 1 -- 172.21.15.125:0/3585408190 shutdown_connections 2023-10-18T10:25:15.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.806+0000 7f4a075ce640 1 -- 172.21.15.125:0/3585408190 wait complete. 2023-10-18T10:25:15.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.807+0000 7f4a075ce640 1 Processor -- start 2023-10-18T10:25:15.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.807+0000 7f4a075ce640 1 -- start start 2023-10-18T10:25:15.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a075ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a000ceb40 0x7f4a0015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a075ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a000cf340 0x7f4a000c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a075ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0015b2a0 0x7f4a001514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a075ce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0015cdb0 con 0x7f4a000cf340 2023-10-18T10:25:15.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a075ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c9570 con 0x7f4a0015b2a0 2023-10-18T10:25:15.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a075ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001519f0 con 0x7f4a000ceb40 2023-10-18T10:25:15.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a04b69640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a000cf340 0x7f4a000c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a04b69640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a000cf340 0x7f4a000c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40362/0 (socket says 172.21.15.125:40362) 2023-10-18T10:25:15.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a04b69640 1 -- 172.21.15.125:0/1118466015 learned_addr learned my addr 172.21.15.125:0/1118466015 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:15.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a0536a640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a000ceb40 0x7f4a0015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a05b6b640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0015b2a0 0x7f4a001514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a04b69640 1 -- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a000ceb40 msgr2=0x7f4a0015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a04b69640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a000ceb40 0x7f4a0015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a04b69640 1 -- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0015b2a0 msgr2=0x7f4a001514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a04b69640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0015b2a0 0x7f4a001514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a04b69640 1 -- 172.21.15.125:0/1118466015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0009560 con 0x7f4a000cf340 2023-10-18T10:25:15.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.808+0000 7f4a05b6b640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0015b2a0 0x7f4a001514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:15.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.809+0000 7f4a04b69640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a000cf340 0x7f4a000c9030 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49f00099c0 tx=0x7f49f002dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:15.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.809+0000 7f49f5ffb640 1 -- 172.21.15.125:0/1118466015 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0005670 con 0x7f4a000cf340 2023-10-18T10:25:15.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.809+0000 7f49f5ffb640 1 -- 172.21.15.125:0/1118466015 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f49f00038f0 con 0x7f4a000cf340 2023-10-18T10:25:15.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.809+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0015cfe0 con 0x7f4a000cf340 2023-10-18T10:25:15.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.809+0000 7f49f5ffb640 1 -- 172.21.15.125:0/1118466015 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0003ce0 con 0x7f4a000cf340 2023-10-18T10:25:15.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.809+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0015d5a0 con 0x7f4a000cf340 2023-10-18T10:25:15.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.810+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a000690e0 con 0x7f4a000cf340 2023-10-18T10:25:15.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.810+0000 7f49f5ffb640 1 -- 172.21.15.125:0/1118466015 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f49f003e450 con 0x7f4a000cf340 2023-10-18T10:25:15.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.811+0000 7f49f5ffb640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f49e406ae40 0x7f49e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:15.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.811+0000 7f49f5ffb640 1 -- 172.21.15.125:0/1118466015 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f49f00ac270 con 0x7f4a000cf340 2023-10-18T10:25:15.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.811+0000 7f4a0536a640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f49e406ae40 0x7f49e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:15.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.812+0000 7f4a0536a640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f49e406ae40 0x7f49e406d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49fc005660 tx=0x7f49fc003d90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:15.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.815+0000 7f49f5ffb640 1 -- 172.21.15.125:0/1118466015 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49f0077150 con 0x7f4a000cf340 2023-10-18T10:25:15.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.979+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a000b0d00 con 0x7f4a000cf340 2023-10-18T10:25:15.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.980+0000 7f49f5ffb640 1 -- 172.21.15.125:0/1118466015 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49f007a770 con 0x7f4a000cf340 2023-10-18T10:25:15.982 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:15.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f49e406ae40 msgr2=0x7f49e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f49e406ae40 0x7f49e406d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49fc005660 tx=0x7f49fc003d90 comp rx=0 tx=0).stop 2023-10-18T10:25:15.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a000cf340 msgr2=0x7f4a000c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:15.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a000cf340 0x7f4a000c9030 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49f00099c0 tx=0x7f49f002dd00 comp rx=0 tx=0).stop 2023-10-18T10:25:15.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 shutdown_connections 2023-10-18T10:25:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f49e406ae40 0x7f49e406d300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0015b2a0 0x7f4a001514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a000cf340 0x7f4a000c9030 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 --2- 172.21.15.125:0/1118466015 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a000ceb40 0x7f4a0015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 >> 172.21.15.125:0/1118466015 conn(0x7f4a000d0d60 msgr2=0x7f4a0015a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 shutdown_connections 2023-10-18T10:25:15.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:15.982+0000 7f4a075ce640 1 -- 172.21.15.125:0/1118466015 wait complete. 2023-10-18T10:25:15.994 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:16.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.097+0000 7f579d58e640 1 -- 172.21.15.125:0/1839346971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158cf0 msgr2=0x7f579815c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.097+0000 7f579d58e640 1 --2- 172.21.15.125:0/1839346971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158cf0 0x7f579815c0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f578c005fd0 tx=0x7f578c02bed0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.097+0000 7f579d58e640 1 -- 172.21.15.125:0/1839346971 shutdown_connections 2023-10-18T10:25:16.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.097+0000 7f579d58e640 1 --2- 172.21.15.125:0/1839346971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158cf0 0x7f579815c0e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.097+0000 7f579d58e640 1 -- 172.21.15.125:0/1839346971 >> 172.21.15.125:0/1839346971 conn(0x7f57980af7a0 msgr2=0x7f57980afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:16.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.097+0000 7f579d58e640 1 -- 172.21.15.125:0/1839346971 shutdown_connections 2023-10-18T10:25:16.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.097+0000 7f579d58e640 1 -- 172.21.15.125:0/1839346971 wait complete. 2023-10-18T10:25:16.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.098+0000 7f579d58e640 1 Processor -- start 2023-10-18T10:25:16.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.098+0000 7f579d58e640 1 -- start start 2023-10-18T10:25:16.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f579d58e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57980cd9e0 0x7f5798151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f579d58e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158910 0x7f579814cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f579d58e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5798158cf0 0x7f579814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f579d58e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579814fac0 con 0x7f5798158910 2023-10-18T10:25:16.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f579d58e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579814d800 con 0x7f5798158cf0 2023-10-18T10:25:16.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f579d58e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579814ec50 con 0x7f57980cd9e0 2023-10-18T10:25:16.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158910 0x7f579814cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158910 0x7f579814cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40368/0 (socket says 172.21.15.125:40368) 2023-10-18T10:25:16.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 -- 172.21.15.125:0/2876027997 learned_addr learned my addr 172.21.15.125:0/2876027997 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:16.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57977fe640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5798158cf0 0x7f579814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 -- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57980cd9e0 msgr2=0x7f5798151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57980cd9e0 0x7f5798151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 -- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5798158cf0 msgr2=0x7f579814d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5798158cf0 0x7f579814d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 -- 172.21.15.125:0/2876027997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009560 con 0x7f5798158910 2023-10-18T10:25:16.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.099+0000 7f57967fc640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158910 0x7f579814cd80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f578c0099f0 tx=0x7f578c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.100+0000 7f577f7fe640 1 -- 172.21.15.125:0/2876027997 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c003d60 con 0x7f5798158910 2023-10-18T10:25:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.100+0000 7f577f7fe640 1 -- 172.21.15.125:0/2876027997 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f578c005020 con 0x7f5798158910 2023-10-18T10:25:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.100+0000 7f577f7fe640 1 -- 172.21.15.125:0/2876027997 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c005430 con 0x7f5798158910 2023-10-18T10:25:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.100+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579814eed0 con 0x7f5798158910 2023-10-18T10:25:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.100+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579814f470 con 0x7f5798158910 2023-10-18T10:25:16.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.101+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57980af5c0 con 0x7f5798158910 2023-10-18T10:25:16.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.101+0000 7f577f7fe640 1 -- 172.21.15.125:0/2876027997 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f578c005590 con 0x7f5798158910 2023-10-18T10:25:16.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.101+0000 7f577f7fe640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f578406ae40 0x7f578406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.102+0000 7f577f7fe640 1 -- 172.21.15.125:0/2876027997 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f578c0ac6e0 con 0x7f5798158910 2023-10-18T10:25:16.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.102+0000 7f5796ffd640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f578406ae40 0x7f578406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.102+0000 7f5796ffd640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f578406ae40 0x7f578406d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57880012e0 tx=0x7f5788007c70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:16.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.106+0000 7f577f7fe640 1 -- 172.21.15.125:0/2876027997 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f578c077530 con 0x7f5798158910 2023-10-18T10:25:16.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.273+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f579806d5f0 con 0x7f5798158910 2023-10-18T10:25:16.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.273+0000 7f577f7fe640 1 -- 172.21.15.125:0/2876027997 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f578c07ab50 con 0x7f5798158910 2023-10-18T10:25:16.276 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:16.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f578406ae40 msgr2=0x7f578406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f578406ae40 0x7f578406d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57880012e0 tx=0x7f5788007c70 comp rx=0 tx=0).stop 2023-10-18T10:25:16.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158910 msgr2=0x7f579814cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158910 0x7f579814cd80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f578c0099f0 tx=0x7f578c0038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 shutdown_connections 2023-10-18T10:25:16.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f578406ae40 0x7f578406d300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5798158cf0 0x7f579814d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5798158910 0x7f579814cd80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 --2- 172.21.15.125:0/2876027997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57980cd9e0 0x7f5798151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 >> 172.21.15.125:0/2876027997 conn(0x7f57980af7a0 msgr2=0x7f5798069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:16.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 shutdown_connections 2023-10-18T10:25:16.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.275+0000 7f579d58e640 1 -- 172.21.15.125:0/2876027997 wait complete. 2023-10-18T10:25:16.288 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:16.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.392+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/4159311660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d48141690 msgr2=0x7f3d480703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.392+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/4159311660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d48141690 0x7f3d480703e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c005fd0 tx=0x7f3d3c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:25:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.392+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/4159311660 shutdown_connections 2023-10-18T10:25:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.392+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/4159311660 >> v2:172.21.15.139:3300/0 conn(0x7f3d48070920 0x7f3d480affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.392+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/4159311660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d48141690 0x7f3d480703e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.392+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/4159311660 >> 172.21.15.125:0/4159311660 conn(0x7f3d48069930 msgr2=0x7f3d48069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.392+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/4159311660 shutdown_connections 2023-10-18T10:25:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.392+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/4159311660 wait complete. 2023-10-18T10:25:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.393+0000 7f3d4e1c0640 1 Processor -- start 2023-10-18T10:25:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.393+0000 7f3d4e1c0640 1 -- start start 2023-10-18T10:25:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d4e1c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d48070920 0x7f3d480ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d4e1c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d4813e520 0x7f3d480a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d4e1c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d48141690 0x7f3d480caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d4e1c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d481f4750 con 0x7f3d48141690 2023-10-18T10:25:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d4e1c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480c6b50 con 0x7f3d4813e520 2023-10-18T10:25:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d4e1c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480c5da0 con 0x7f3d48070920 2023-10-18T10:25:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d47fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d48141690 0x7f3d480caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d47fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d48141690 0x7f3d480caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40378/0 (socket says 172.21.15.125:40378) 2023-10-18T10:25:16.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d477fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d48070920 0x7f3d480ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d47fff640 1 -- 172.21.15.125:0/2225054952 learned_addr learned my addr 172.21.15.125:0/2225054952 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:16.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d46ffd640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d4813e520 0x7f3d480a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.394+0000 7f3d47fff640 1 -- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d48070920 msgr2=0x7f3d480ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d47fff640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d48070920 0x7f3d480ab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d47fff640 1 -- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d4813e520 msgr2=0x7f3d480a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d47fff640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d4813e520 0x7f3d480a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d47fff640 1 -- 172.21.15.125:0/2225054952 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c009560 con 0x7f3d48141690 2023-10-18T10:25:16.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d47fff640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d48141690 0x7f3d480caae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000eb10 tx=0x7f3d4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:16.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2225054952 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40004b20 con 0x7f3d48141690 2023-10-18T10:25:16.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2225054952 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d40004ca0 con 0x7f3d48141690 2023-10-18T10:25:16.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d480c6080 con 0x7f3d48141690 2023-10-18T10:25:16.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2225054952 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40005280 con 0x7f3d48141690 2023-10-18T10:25:16.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.395+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d481e1f90 con 0x7f3d48141690 2023-10-18T10:25:16.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.396+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d481f48c0 con 0x7f3d48141690 2023-10-18T10:25:16.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.396+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2225054952 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3d400053e0 con 0x7f3d48141690 2023-10-18T10:25:16.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.397+0000 7f3d2ffff640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d3406ae40 0x7f3d3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.397+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2225054952 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d40014070 con 0x7f3d48141690 2023-10-18T10:25:16.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.397+0000 7f3d477fe640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d3406ae40 0x7f3d3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.397+0000 7f3d477fe640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d3406ae40 0x7f3d3406d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3d38001780 tx=0x7f3d38009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:16.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.401+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2225054952 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d400553d0 con 0x7f3d48141690 2023-10-18T10:25:16.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.569+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d480690e0 con 0x7f3d48141690 2023-10-18T10:25:16.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.569+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2225054952 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d400589f0 con 0x7f3d48141690 2023-10-18T10:25:16.571 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:16.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d3406ae40 msgr2=0x7f3d3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d3406ae40 0x7f3d3406d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3d38001780 tx=0x7f3d38009040 comp rx=0 tx=0).stop 2023-10-18T10:25:16.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d48141690 msgr2=0x7f3d480caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d48141690 0x7f3d480caae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000eb10 tx=0x7f3d4000c2d0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 shutdown_connections 2023-10-18T10:25:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d3406ae40 0x7f3d3406d300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d4813e520 0x7f3d480a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d48141690 0x7f3d480caae0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 --2- 172.21.15.125:0/2225054952 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d48070920 0x7f3d480ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 >> 172.21.15.125:0/2225054952 conn(0x7f3d48069930 msgr2=0x7f3d480af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:16.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 shutdown_connections 2023-10-18T10:25:16.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.571+0000 7f3d4e1c0640 1 -- 172.21.15.125:0/2225054952 wait complete. 2023-10-18T10:25:16.583 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:16.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.687+0000 7f486dcc2640 1 -- 172.21.15.125:0/472340505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 msgr2=0x7f486806a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.687+0000 7f486dcc2640 1 --2- 172.21.15.125:0/472340505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 0x7f486806a730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f485c005fd0 tx=0x7f485c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:25:16.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.687+0000 7f486dcc2640 1 -- 172.21.15.125:0/472340505 shutdown_connections 2023-10-18T10:25:16.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.687+0000 7f486dcc2640 1 --2- 172.21.15.125:0/472340505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 0x7f486806a730 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.687+0000 7f486dcc2640 1 --2- 172.21.15.125:0/472340505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48681579f0 0x7f48680b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.687+0000 7f486dcc2640 1 -- 172.21.15.125:0/472340505 >> 172.21.15.125:0/472340505 conn(0x7f486806fae0 msgr2=0x7f486806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:16.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.687+0000 7f486dcc2640 1 -- 172.21.15.125:0/472340505 shutdown_connections 2023-10-18T10:25:16.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.687+0000 7f486dcc2640 1 -- 172.21.15.125:0/472340505 wait complete. 2023-10-18T10:25:16.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.688+0000 7f486dcc2640 1 Processor -- start 2023-10-18T10:25:16.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.688+0000 7f486dcc2640 1 -- start start 2023-10-18T10:25:16.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f486dcc2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486806ac70 0x7f4868156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f486dcc2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48681579f0 0x7f4868143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f486dcc2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 0x7f4868153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f486dcc2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4868155e60 con 0x7f486815c1f0 2023-10-18T10:25:16.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f486dcc2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486806c560 con 0x7f48681579f0 2023-10-18T10:25:16.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f486dcc2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4868156090 con 0x7f486806ac70 2023-10-18T10:25:16.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 0x7f4868153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 0x7f4868153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40402/0 (socket says 172.21.15.125:40402) 2023-10-18T10:25:16.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 -- 172.21.15.125:0/2234982485 learned_addr learned my addr 172.21.15.125:0/2234982485 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:16.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f48677fe640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486806ac70 0x7f4868156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4866ffd640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48681579f0 0x7f4868143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 -- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486806ac70 msgr2=0x7f4868156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486806ac70 0x7f4868156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 -- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48681579f0 msgr2=0x7f4868143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48681579f0 0x7f4868143d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 -- 172.21.15.125:0/2234982485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c009560 con 0x7f486815c1f0 2023-10-18T10:25:16.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f4867fff640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 0x7f4868153eb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f486000c780 tx=0x7f486000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:16.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f484ffff640 1 -- 172.21.15.125:0/2234982485 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860004b60 con 0x7f486815c1f0 2023-10-18T10:25:16.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f484ffff640 1 -- 172.21.15.125:0/2234982485 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4860004dd0 con 0x7f486815c1f0 2023-10-18T10:25:16.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f484ffff640 1 -- 172.21.15.125:0/2234982485 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860005290 con 0x7f486815c1f0 2023-10-18T10:25:16.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4868156370 con 0x7f486815c1f0 2023-10-18T10:25:16.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.689+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4868156930 con 0x7f486815c1f0 2023-10-18T10:25:16.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.690+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48680690e0 con 0x7f486815c1f0 2023-10-18T10:25:16.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.691+0000 7f484ffff640 1 -- 172.21.15.125:0/2234982485 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f48600053f0 con 0x7f486815c1f0 2023-10-18T10:25:16.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.691+0000 7f484ffff640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f485406ae40 0x7f485406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.691+0000 7f484ffff640 1 -- 172.21.15.125:0/2234982485 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4860089bb0 con 0x7f486815c1f0 2023-10-18T10:25:16.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.691+0000 7f48677fe640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f485406ae40 0x7f485406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.692+0000 7f48677fe640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f485406ae40 0x7f485406d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f485800abe0 tx=0x7f485800a2d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:16.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.695+0000 7f484ffff640 1 -- 172.21.15.125:0/2234982485 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4860054970 con 0x7f486815c1f0 2023-10-18T10:25:16.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.860+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4868156b20 con 0x7f486815c1f0 2023-10-18T10:25:16.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.860+0000 7f484ffff640 1 -- 172.21.15.125:0/2234982485 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4860057f90 con 0x7f486815c1f0 2023-10-18T10:25:16.863 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:16.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f485406ae40 msgr2=0x7f485406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f485406ae40 0x7f485406d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f485800abe0 tx=0x7f485800a2d0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 msgr2=0x7f4868153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 0x7f4868153eb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f486000c780 tx=0x7f486000cb80 comp rx=0 tx=0).stop 2023-10-18T10:25:16.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 shutdown_connections 2023-10-18T10:25:16.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f485406ae40 0x7f485406d300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48681579f0 0x7f4868143d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f486815c1f0 0x7f4868153eb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 --2- 172.21.15.125:0/2234982485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f486806ac70 0x7f4868156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 >> 172.21.15.125:0/2234982485 conn(0x7f486806fae0 msgr2=0x7f4868069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 shutdown_connections 2023-10-18T10:25:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.862+0000 7f486dcc2640 1 -- 172.21.15.125:0/2234982485 wait complete. 2023-10-18T10:25:16.875 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:25:16.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.976+0000 7fa4770a9640 1 -- 172.21.15.125:0/3272103128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b3070 msgr2=0x7fa4701590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.976+0000 7fa4770a9640 1 --2- 172.21.15.125:0/3272103128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b3070 0x7fa4701590b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa47006fd80 tx=0x7fa46402bdb0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.976+0000 7fa4770a9640 1 -- 172.21.15.125:0/3272103128 shutdown_connections 2023-10-18T10:25:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.976+0000 7fa4770a9640 1 --2- 172.21.15.125:0/3272103128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b3070 0x7fa4701590b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.976+0000 7fa4770a9640 1 --2- 172.21.15.125:0/3272103128 >> v2:172.21.15.139:3300/0 conn(0x7fa4700b24d0 0x7fa4700b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.976+0000 7fa4770a9640 1 -- 172.21.15.125:0/3272103128 >> 172.21.15.125:0/3272103128 conn(0x7fa4700b3f00 msgr2=0x7fa4700b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.976+0000 7fa4770a9640 1 -- 172.21.15.125:0/3272103128 shutdown_connections 2023-10-18T10:25:16.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.976+0000 7fa4770a9640 1 -- 172.21.15.125:0/3272103128 wait complete. 2023-10-18T10:25:16.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.977+0000 7fa4770a9640 1 Processor -- start 2023-10-18T10:25:16.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.977+0000 7fa4770a9640 1 -- start start 2023-10-18T10:25:16.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa4770a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4700acd40 0x7fa4701512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa4770a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b24d0 0x7fa47006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa4770a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700b3070 0x7fa4700c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa4770a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700c9050 con 0x7fa4700b24d0 2023-10-18T10:25:16.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa4770a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700b2000 con 0x7fa4700acd40 2023-10-18T10:25:16.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa4770a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700c7d60 con 0x7fa4700b3070 2023-10-18T10:25:16.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b24d0 0x7fa47006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b24d0 0x7fa47006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40414/0 (socket says 172.21.15.125:40414) 2023-10-18T10:25:16.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 -- 172.21.15.125:0/1667747409 learned_addr learned my addr 172.21.15.125:0/1667747409 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:16.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa474e45640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4700acd40 0x7fa4701512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 -- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700b3070 msgr2=0x7fa4700c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa475646640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700b3070 0x7fa4700c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700b3070 0x7fa4700c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 -- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4700acd40 msgr2=0x7fa4701512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:16.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4700acd40 0x7fa4701512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:16.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 -- 172.21.15.125:0/1667747409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa464009560 con 0x7fa4700b24d0 2023-10-18T10:25:16.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa474e45640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4700acd40 0x7fa4701512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:16.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.978+0000 7fa46ffff640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b24d0 0x7fa47006beb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa464009850 tx=0x7fa46402da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:16.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.979+0000 7fa46d7fa640 1 -- 172.21.15.125:0/1667747409 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464003800 con 0x7fa4700b24d0 2023-10-18T10:25:16.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.979+0000 7fa46d7fa640 1 -- 172.21.15.125:0/1667747409 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa464003a50 con 0x7fa4700b24d0 2023-10-18T10:25:16.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.979+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4700c7fb0 con 0x7fa4700b24d0 2023-10-18T10:25:16.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.979+0000 7fa46d7fa640 1 -- 172.21.15.125:0/1667747409 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464003e20 con 0x7fa4700b24d0 2023-10-18T10:25:16.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.979+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4700c8570 con 0x7fa4700b24d0 2023-10-18T10:25:16.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.980+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4700690e0 con 0x7fa4700b24d0 2023-10-18T10:25:16.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.980+0000 7fa46d7fa640 1 -- 172.21.15.125:0/1667747409 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa46403e450 con 0x7fa4700b24d0 2023-10-18T10:25:16.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.980+0000 7fa46d7fa640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa45c06ae70 0x7fa45c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:16.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.980+0000 7fa46d7fa640 1 -- 172.21.15.125:0/1667747409 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa4640ac3b0 con 0x7fa4700b24d0 2023-10-18T10:25:16.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.981+0000 7fa474e45640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa45c06ae70 0x7fa45c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:16.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.981+0000 7fa474e45640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa45c06ae70 0x7fa45c06d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa4700b1520 tx=0x7fa460005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:16.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:16.985+0000 7fa46d7fa640 1 -- 172.21.15.125:0/1667747409 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa464077290 con 0x7fa4700b24d0 2023-10-18T10:25:17.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.163+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa4700675f0 con 0x7fa4700b24d0 2023-10-18T10:25:17.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.163+0000 7fa46d7fa640 1 -- 172.21.15.125:0/1667747409 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa46407a8b0 con 0x7fa4700b24d0 2023-10-18T10:25:17.165 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:25:17.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa45c06ae70 msgr2=0x7fa45c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa45c06ae70 0x7fa45c06d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa4700b1520 tx=0x7fa460005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:17.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b24d0 msgr2=0x7fa47006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b24d0 0x7fa47006beb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa464009850 tx=0x7fa46402da80 comp rx=0 tx=0).stop 2023-10-18T10:25:17.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 shutdown_connections 2023-10-18T10:25:17.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa45c06ae70 0x7fa45c06d330 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700b3070 0x7fa4700c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700b24d0 0x7fa47006beb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 --2- 172.21.15.125:0/1667747409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4700acd40 0x7fa4701512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 >> 172.21.15.125:0/1667747409 conn(0x7fa4700b3f00 msgr2=0x7fa4701559b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:17.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 shutdown_connections 2023-10-18T10:25:17.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.165+0000 7fa4770a9640 1 -- 172.21.15.125:0/1667747409 wait complete. 2023-10-18T10:25:17.177 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:25:17.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:17.178 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-18T10:25:17.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi125.stderr:ceph-fuse[96110]: fuse finished with error 0 and tester_r 0 2023-10-18T10:25:17.205 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:25:17.240 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:25:17.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:17.240 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-18T10:25:17.305 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:17.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.444+0000 7ff167a40640 1 -- 172.21.15.125:0/575082317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff16013d360 msgr2=0x7ff16015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.444+0000 7ff167a40640 1 --2- 172.21.15.125:0/575082317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff16013d360 0x7ff16015a9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff15c005fd0 tx=0x7ff15c02c0e0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.444+0000 7ff167a40640 1 -- 172.21.15.125:0/575082317 shutdown_connections 2023-10-18T10:25:17.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.444+0000 7ff167a40640 1 --2- 172.21.15.125:0/575082317 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1600c9ed0 0x7ff16006b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.444+0000 7ff167a40640 1 --2- 172.21.15.125:0/575082317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff16013d360 0x7ff16015a9f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.444+0000 7ff167a40640 1 -- 172.21.15.125:0/575082317 >> 172.21.15.125:0/575082317 conn(0x7ff160069930 msgr2=0x7ff160069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:17.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.444+0000 7ff167a40640 1 -- 172.21.15.125:0/575082317 shutdown_connections 2023-10-18T10:25:17.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.444+0000 7ff167a40640 1 -- 172.21.15.125:0/575082317 wait complete. 2023-10-18T10:25:17.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.445+0000 7ff167a40640 1 Processor -- start 2023-10-18T10:25:17.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.445+0000 7ff167a40640 1 -- start start 2023-10-18T10:25:17.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff167a40640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1600c9ed0 0x7ff1601375b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:17.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff167a40640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff16013d360 0x7ff160145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:17.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff167a40640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff160140660 0x7ff160149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:17.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff167a40640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff160146320 con 0x7ff160140660 2023-10-18T10:25:17.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff167a40640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16014d920 con 0x7ff1600c9ed0 2023-10-18T10:25:17.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff167a40640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16014a880 con 0x7ff16013d360 2023-10-18T10:25:17.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff160140660 0x7ff160149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:17.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff160140660 0x7ff160149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49398/0 (socket says 172.21.15.125:49398) 2023-10-18T10:25:17.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 -- 172.21.15.125:0/4014501677 learned_addr learned my addr 172.21.15.125:0/4014501677 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:17.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff164fdb640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff16013d360 0x7ff160145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:17.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 -- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff16013d360 msgr2=0x7ff160145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff16013d360 0x7ff160145de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 -- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1600c9ed0 msgr2=0x7ff1601375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1600c9ed0 0x7ff1601375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 -- 172.21.15.125:0/4014501677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c009840 con 0x7ff160140660 2023-10-18T10:25:17.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.446+0000 7ff165fdd640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff160140660 0x7ff160149f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c970 tx=0x7ff15800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:17.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.447+0000 7ff155ffb640 1 -- 172.21.15.125:0/4014501677 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158007bb0 con 0x7ff160140660 2023-10-18T10:25:17.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.447+0000 7ff155ffb640 1 -- 172.21.15.125:0/4014501677 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff158007e00 con 0x7ff160140660 2023-10-18T10:25:17.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.447+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16014d2d0 con 0x7ff160140660 2023-10-18T10:25:17.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.447+0000 7ff155ffb640 1 -- 172.21.15.125:0/4014501677 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158005280 con 0x7ff160140660 2023-10-18T10:25:17.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.447+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16014ab60 con 0x7ff160140660 2023-10-18T10:25:17.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.448+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1600690e0 con 0x7ff160140660 2023-10-18T10:25:17.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.448+0000 7ff155ffb640 1 -- 172.21.15.125:0/4014501677 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff1580053e0 con 0x7ff160140660 2023-10-18T10:25:17.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.448+0000 7ff155ffb640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff14406ae40 0x7ff14406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:17.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.449+0000 7ff155ffb640 1 -- 172.21.15.125:0/4014501677 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff158089e00 con 0x7ff160140660 2023-10-18T10:25:17.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.449+0000 7ff1657dc640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff14406ae40 0x7ff14406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:17.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.449+0000 7ff1657dc640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff14406ae40 0x7ff14406d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff15c035000 tx=0x7ff15c004a40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:17.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.453+0000 7ff155ffb640 1 -- 172.21.15.125:0/4014501677 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff158054bc0 con 0x7ff160140660 2023-10-18T10:25:17.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.622+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff16014da60 con 0x7ff160140660 2023-10-18T10:25:17.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.623+0000 7ff155ffb640 1 -- 172.21.15.125:0/4014501677 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7ff15801d070 con 0x7ff160140660 2023-10-18T10:25:17.626 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:17.626 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:17.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff14406ae40 msgr2=0x7ff14406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff14406ae40 0x7ff14406d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff15c035000 tx=0x7ff15c004a40 comp rx=0 tx=0).stop 2023-10-18T10:25:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff160140660 msgr2=0x7ff160149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff160140660 0x7ff160149f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c970 tx=0x7ff15800ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 shutdown_connections 2023-10-18T10:25:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff14406ae40 0x7ff14406d300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff160140660 0x7ff160149f50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff16013d360 0x7ff160145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 --2- 172.21.15.125:0/4014501677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1600c9ed0 0x7ff1601375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.625+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 >> 172.21.15.125:0/4014501677 conn(0x7ff160069930 msgr2=0x7ff1600cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.626+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 shutdown_connections 2023-10-18T10:25:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.626+0000 7ff167a40640 1 -- 172.21.15.125:0/4014501677 wait complete. 2023-10-18T10:25:17.638 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:17.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.738+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/1701012842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e5415b260 msgr2=0x7f5e540cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.738+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/1701012842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e5415b260 0x7f5e540cf310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009830 tx=0x7f5e5002bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:17.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.738+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/1701012842 shutdown_connections 2023-10-18T10:25:17.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.738+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/1701012842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e540cf850 0x7f5e540cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.738+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/1701012842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e5415b260 0x7f5e540cf310 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.738+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/1701012842 >> 172.21.15.125:0/1701012842 conn(0x7f5e5409a030 msgr2=0x7f5e540d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:17.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.738+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/1701012842 shutdown_connections 2023-10-18T10:25:17.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.738+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/1701012842 wait complete. 2023-10-18T10:25:17.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5c6e7640 1 Processor -- start 2023-10-18T10:25:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5c6e7640 1 -- start start 2023-10-18T10:25:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5c6e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e540cf850 0x7f5e540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5c6e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e5415b260 0x7f5e5415df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5c6e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e5415b640 0x7f5e5415e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5c6e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5415ba20 con 0x7f5e5415b260 2023-10-18T10:25:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5c6e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540b3fd0 con 0x7f5e540cf850 2023-10-18T10:25:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5c6e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540b42b0 con 0x7f5e5415b640 2023-10-18T10:25:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e59c82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e5415b260 0x7f5e5415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e59c82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e5415b260 0x7f5e5415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49414/0 (socket says 172.21.15.125:49414) 2023-10-18T10:25:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e59c82640 1 -- 172.21.15.125:0/484559509 learned_addr learned my addr 172.21.15.125:0/484559509 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5a483640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e540cf850 0x7f5e540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.740+0000 7f5e5ac84640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e5415b640 0x7f5e5415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e59c82640 1 -- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e5415b640 msgr2=0x7f5e5415e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e59c82640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e5415b640 0x7f5e5415e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e59c82640 1 -- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e540cf850 msgr2=0x7f5e540b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e59c82640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e540cf850 0x7f5e540b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e59c82640 1 -- 172.21.15.125:0/484559509 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c009900 con 0x7f5e5415b260 2023-10-18T10:25:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e59c82640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e5415b260 0x7f5e5415df30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400c400 tx=0x7f5e4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e4affd640 1 -- 172.21.15.125:0/484559509 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440052d0 con 0x7f5e5415b260 2023-10-18T10:25:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e50009490 con 0x7f5e5415b260 2023-10-18T10:25:17.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e4affd640 1 -- 172.21.15.125:0/484559509 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e440035f0 con 0x7f5e5415b260 2023-10-18T10:25:17.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e4affd640 1 -- 172.21.15.125:0/484559509 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44003a00 con 0x7f5e5415b260 2023-10-18T10:25:17.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.741+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e540b4870 con 0x7f5e5415b260 2023-10-18T10:25:17.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.742+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5415bb90 con 0x7f5e5415b260 2023-10-18T10:25:17.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.742+0000 7f5e4affd640 1 -- 172.21.15.125:0/484559509 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5e44003b60 con 0x7f5e5415b260 2023-10-18T10:25:17.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.743+0000 7f5e4affd640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5e3806ae70 0x7f5e3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:17.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.743+0000 7f5e4affd640 1 -- 172.21.15.125:0/484559509 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5e44089030 con 0x7f5e5415b260 2023-10-18T10:25:17.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.743+0000 7f5e5a483640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5e3806ae70 0x7f5e3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:17.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.743+0000 7f5e5a483640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5e3806ae70 0x7f5e3806d330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5e50036000 tx=0x7f5e50004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:17.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.747+0000 7f5e4affd640 1 -- 172.21.15.125:0/484559509 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e44053eb0 con 0x7f5e5415b260 2023-10-18T10:25:17.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.913+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e540cd870 con 0x7f5e5415b260 2023-10-18T10:25:17.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.915+0000 7f5e4affd640 1 -- 172.21.15.125:0/484559509 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f5e440574d0 con 0x7f5e5415b260 2023-10-18T10:25:17.917 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:17.917 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:17.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.916+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5e3806ae70 msgr2=0x7f5e3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.916+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5e3806ae70 0x7f5e3806d330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5e50036000 tx=0x7f5e50004730 comp rx=0 tx=0).stop 2023-10-18T10:25:17.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.916+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e5415b260 msgr2=0x7f5e5415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:17.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.916+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e5415b260 0x7f5e5415df30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400c400 tx=0x7f5e4400c8c0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.917+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 shutdown_connections 2023-10-18T10:25:17.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.917+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5e3806ae70 0x7f5e3806d330 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.917+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e5415b640 0x7f5e5415e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.917+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e5415b260 0x7f5e5415df30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.917+0000 7f5e5c6e7640 1 --2- 172.21.15.125:0/484559509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e540cf850 0x7f5e540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:17.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.917+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 >> 172.21.15.125:0/484559509 conn(0x7f5e5409a030 msgr2=0x7f5e540b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:17.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.917+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 shutdown_connections 2023-10-18T10:25:17.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:17.917+0000 7f5e5c6e7640 1 -- 172.21.15.125:0/484559509 wait complete. 2023-10-18T10:25:17.929 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:25:18.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.032+0000 7f513ebe2640 1 -- 172.21.15.125:0/1980570961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 msgr2=0x7f513806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:18.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.032+0000 7f513ebe2640 1 --2- 172.21.15.125:0/1980570961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 0x7f513806bb60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f512c009830 tx=0x7f512c02bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:18.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.032+0000 7f513ebe2640 1 -- 172.21.15.125:0/1980570961 shutdown_connections 2023-10-18T10:25:18.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.032+0000 7f513ebe2640 1 --2- 172.21.15.125:0/1980570961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 0x7f513806bb60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:18.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.032+0000 7f513ebe2640 1 -- 172.21.15.125:0/1980570961 >> 172.21.15.125:0/1980570961 conn(0x7f5138069930 msgr2=0x7f5138069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:18.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.032+0000 7f513ebe2640 1 -- 172.21.15.125:0/1980570961 shutdown_connections 2023-10-18T10:25:18.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.032+0000 7f513ebe2640 1 -- 172.21.15.125:0/1980570961 wait complete. 2023-10-18T10:25:18.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.033+0000 7f513ebe2640 1 Processor -- start 2023-10-18T10:25:18.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.033+0000 7f513ebe2640 1 -- start start 2023-10-18T10:25:18.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f513ebe2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5138070af0 0x7f513813abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:18.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f513ebe2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 0x7f5138154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:18.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f513ebe2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5138141690 0x7f51380ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:18.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f513ebe2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5138154d70 con 0x7f513813e520 2023-10-18T10:25:18.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f513ebe2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5138155050 con 0x7f5138141690 2023-10-18T10:25:18.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f513ebe2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5138155330 con 0x7f5138070af0 2023-10-18T10:25:18.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f5137fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 0x7f5138154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:18.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f5137fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 0x7f5138154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49430/0 (socket says 172.21.15.125:49430) 2023-10-18T10:25:18.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.034+0000 7f5137fff640 1 -- 172.21.15.125:0/3827786012 learned_addr learned my addr 172.21.15.125:0/3827786012 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:18.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f513d17f640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5138141690 0x7f51380ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:18.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f513c97e640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5138070af0 0x7f513813abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:18.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f5137fff640 1 -- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5138070af0 msgr2=0x7f513813abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:18.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f5137fff640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5138070af0 0x7f513813abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:18.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f5137fff640 1 -- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5138141690 msgr2=0x7f51380ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:18.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f5137fff640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5138141690 0x7f51380ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:18.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f5137fff640 1 -- 172.21.15.125:0/3827786012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c009490 con 0x7f513813e520 2023-10-18T10:25:18.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f513d17f640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5138141690 0x7f51380ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:18.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f5137fff640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 0x7f5138154370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f512400c3c0 tx=0x7f512400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:18.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f51357fa640 1 -- 172.21.15.125:0/3827786012 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011790 con 0x7f513813e520 2023-10-18T10:25:18.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f51357fa640 1 -- 172.21.15.125:0/3827786012 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f51240119e0 con 0x7f513813e520 2023-10-18T10:25:18.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5138155610 con 0x7f513813e520 2023-10-18T10:25:18.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f51357fa640 1 -- 172.21.15.125:0/3827786012 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011e10 con 0x7f513813e520 2023-10-18T10:25:18.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.035+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51381548b0 con 0x7f513813e520 2023-10-18T10:25:18.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.036+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5138154b30 con 0x7f513813e520 2023-10-18T10:25:18.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.036+0000 7f51357fa640 1 -- 172.21.15.125:0/3827786012 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f512401c460 con 0x7f513813e520 2023-10-18T10:25:18.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.037+0000 7f51357fa640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f511c06ae40 0x7f511c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:18.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.037+0000 7f51357fa640 1 -- 172.21.15.125:0/3827786012 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5124089950 con 0x7f513813e520 2023-10-18T10:25:18.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.037+0000 7f513c97e640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f511c06ae40 0x7f511c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:18.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.038+0000 7f513c97e640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f511c06ae40 0x7f511c06d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f512c036000 tx=0x7f512c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:18.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.041+0000 7f51357fa640 1 -- 172.21.15.125:0/3827786012 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5138154b30 con 0x7f513813e520 2023-10-18T10:25:18.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:18.242+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5138154b30 con 0x7f513813e520 2023-10-18T10:25:19.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.181+0000 7f51357fa640 1 -- 172.21.15.125:0/3827786012 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5138154b30 con 0x7f513813e520 2023-10-18T10:25:19.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.183+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f511c06ae40 msgr2=0x7f511c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:19.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f511c06ae40 0x7f511c06d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f512c036000 tx=0x7f512c004730 comp rx=0 tx=0).stop 2023-10-18T10:25:19.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 msgr2=0x7f5138154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:19.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 0x7f5138154370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f512400c3c0 tx=0x7f512400c880 comp rx=0 tx=0).stop 2023-10-18T10:25:19.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 shutdown_connections 2023-10-18T10:25:19.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f511c06ae40 0x7f511c06d300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:19.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5138141690 0x7f51380ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:19.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513813e520 0x7f5138154370 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:19.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 --2- 172.21.15.125:0/3827786012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5138070af0 0x7f513813abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:19.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 >> 172.21.15.125:0/3827786012 conn(0x7f5138069930 msgr2=0x7f5138069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:19.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 shutdown_connections 2023-10-18T10:25:19.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.184+0000 7f513ebe2640 1 -- 172.21.15.125:0/3827786012 wait complete. 2023-10-18T10:25:19.199 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-10-18T10:25:19.199 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:25:19.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.309+0000 7fce2621a640 1 -- 172.21.15.125:0/1143251484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce20157c00 msgr2=0x7fce200c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:19.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.309+0000 7fce2621a640 1 --2- 172.21.15.125:0/1143251484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce20157c00 0x7fce200c98b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fce10005fd0 tx=0x7fce1002bc60 comp rx=0 tx=0).stop 2023-10-18T10:25:19.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.309+0000 7fce2621a640 1 -- 172.21.15.125:0/1143251484 shutdown_connections 2023-10-18T10:25:19.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.309+0000 7fce2621a640 1 --2- 172.21.15.125:0/1143251484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce20070c30 0x7fce2006bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:19.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.309+0000 7fce2621a640 1 --2- 172.21.15.125:0/1143251484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce20157c00 0x7fce200c98b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:19.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.309+0000 7fce2621a640 1 -- 172.21.15.125:0/1143251484 >> 172.21.15.125:0/1143251484 conn(0x7fce200aec40 msgr2=0x7fce200af050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:19.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.309+0000 7fce2621a640 1 -- 172.21.15.125:0/1143251484 shutdown_connections 2023-10-18T10:25:19.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.309+0000 7fce2621a640 1 -- 172.21.15.125:0/1143251484 wait complete. 2023-10-18T10:25:19.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.310+0000 7fce2621a640 1 Processor -- start 2023-10-18T10:25:19.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.310+0000 7fce2621a640 1 -- start start 2023-10-18T10:25:19.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce2621a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce20070c30 0x7fce2014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:19.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce2621a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce200cb2f0 0x7fce20142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:19.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce2621a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce20157c00 0x7fce20142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:19.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce2621a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20146c30 con 0x7fce200cb2f0 2023-10-18T10:25:19.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce2621a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20147400 con 0x7fce20157c00 2023-10-18T10:25:19.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce2621a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce20142da0 con 0x7fce20070c30 2023-10-18T10:25:19.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce200cb2f0 0x7fce20142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:19.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce200cb2f0 0x7fce20142860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49450/0 (socket says 172.21.15.125:49450) 2023-10-18T10:25:19.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 -- 172.21.15.125:0/2876114904 learned_addr learned my addr 172.21.15.125:0/2876114904 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:19.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1f7fe640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce20070c30 0x7fce2014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:19.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1ffff640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce20157c00 0x7fce20142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:19.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 -- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce20070c30 msgr2=0x7fce2014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:19.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce20070c30 0x7fce2014e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:19.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 -- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce20157c00 msgr2=0x7fce20142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:19.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce20157c00 0x7fce20142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:19.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 -- 172.21.15.125:0/2876114904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce10009560 con 0x7fce200cb2f0 2023-10-18T10:25:19.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce1effd640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce200cb2f0 0x7fce20142860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fce1400c460 tx=0x7fce1400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:19.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce07fff640 1 -- 172.21.15.125:0/2876114904 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce14012830 con 0x7fce200cb2f0 2023-10-18T10:25:19.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce07fff640 1 -- 172.21.15.125:0/2876114904 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce14012ac0 con 0x7fce200cb2f0 2023-10-18T10:25:19.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce07fff640 1 -- 172.21.15.125:0/2876114904 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce14011380 con 0x7fce200cb2f0 2023-10-18T10:25:19.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.311+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2014be60 con 0x7fce200cb2f0 2023-10-18T10:25:19.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.312+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce20147120 con 0x7fce200cb2f0 2023-10-18T10:25:19.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.312+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2014bcc0 con 0x7fce200cb2f0 2023-10-18T10:25:19.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.313+0000 7fce07fff640 1 -- 172.21.15.125:0/2876114904 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fce140114e0 con 0x7fce200cb2f0 2023-10-18T10:25:19.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.313+0000 7fce07fff640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fce0c06ae40 0x7fce0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:19.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.313+0000 7fce07fff640 1 -- 172.21.15.125:0/2876114904 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fce14089620 con 0x7fce200cb2f0 2023-10-18T10:25:19.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.313+0000 7fce1f7fe640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fce0c06ae40 0x7fce0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:19.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.314+0000 7fce1f7fe640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fce0c06ae40 0x7fce0c06d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fce10035000 tx=0x7fce10004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:19.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.317+0000 7fce07fff640 1 -- 172.21.15.125:0/2876114904 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce2014bcc0 con 0x7fce200cb2f0 2023-10-18T10:25:19.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:19.517+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fce2014bcc0 con 0x7fce200cb2f0 2023-10-18T10:25:20.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.292+0000 7fce07fff640 1 -- 172.21.15.125:0/2876114904 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fce2014bcc0 con 0x7fce200cb2f0 2023-10-18T10:25:20.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fce0c06ae40 msgr2=0x7fce0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fce0c06ae40 0x7fce0c06d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fce10035000 tx=0x7fce10004730 comp rx=0 tx=0).stop 2023-10-18T10:25:20.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce200cb2f0 msgr2=0x7fce20142860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce200cb2f0 0x7fce20142860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fce1400c460 tx=0x7fce1400c920 comp rx=0 tx=0).stop 2023-10-18T10:25:20.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 shutdown_connections 2023-10-18T10:25:20.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fce0c06ae40 0x7fce0c06d300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce20157c00 0x7fce20142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce200cb2f0 0x7fce20142860 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 --2- 172.21.15.125:0/2876114904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce20070c30 0x7fce2014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 >> 172.21.15.125:0/2876114904 conn(0x7fce200aec40 msgr2=0x7fce2006f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:20.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 shutdown_connections 2023-10-18T10:25:20.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.295+0000 7fce2621a640 1 -- 172.21.15.125:0/2876114904 wait complete. 2023-10-18T10:25:20.307 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:25:20.308 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:25:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.410+0000 7fe3a28a9640 1 -- 172.21.15.125:0/4216137539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c13e520 msgr2=0x7fe39c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.410+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/4216137539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c13e520 0x7fe39c06bb60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe38c005fd0 tx=0x7fe38c02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.410+0000 7fe3a28a9640 1 -- 172.21.15.125:0/4216137539 shutdown_connections 2023-10-18T10:25:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.410+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/4216137539 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe39c070af0 0x7fe39c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.410+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/4216137539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c13e520 0x7fe39c06bb60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.410+0000 7fe3a28a9640 1 -- 172.21.15.125:0/4216137539 >> 172.21.15.125:0/4216137539 conn(0x7fe39c069930 msgr2=0x7fe39c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.410+0000 7fe3a28a9640 1 -- 172.21.15.125:0/4216137539 shutdown_connections 2023-10-18T10:25:20.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.410+0000 7fe3a28a9640 1 -- 172.21.15.125:0/4216137539 wait complete. 2023-10-18T10:25:20.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.411+0000 7fe3a28a9640 1 Processor -- start 2023-10-18T10:25:20.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.411+0000 7fe3a28a9640 1 -- start start 2023-10-18T10:25:20.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a28a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39c070af0 0x7fe39c1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:20.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a28a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe39c13e520 0x7fe39c158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:20.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a28a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c141690 0x7fe39c158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:20.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a28a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c13bf70 con 0x7fe39c141690 2023-10-18T10:25:20.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a28a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c159b00 con 0x7fe39c070af0 2023-10-18T10:25:20.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a28a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c0a7060 con 0x7fe39c13e520 2023-10-18T10:25:20.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c141690 0x7fe39c158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:20.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c141690 0x7fe39c158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49472/0 (socket says 172.21.15.125:49472) 2023-10-18T10:25:20.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 -- 172.21.15.125:0/1089645246 learned_addr learned my addr 172.21.15.125:0/1089645246 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:20.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe39bfff640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39c070af0 0x7fe39c1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:20.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe39b7fe640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe39c13e520 0x7fe39c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:20.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 -- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe39c13e520 msgr2=0x7fe39c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe39c13e520 0x7fe39c158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 -- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39c070af0 msgr2=0x7fe39c1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39c070af0 0x7fe39c1580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 -- 172.21.15.125:0/1089645246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c009560 con 0x7fe39c141690 2023-10-18T10:25:20.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a0e46640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c141690 0x7fe39c158b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe39400cad0 tx=0x7fe39400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:20.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe398ff9640 1 -- 172.21.15.125:0/1089645246 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400ae20 con 0x7fe39c141690 2023-10-18T10:25:20.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe398ff9640 1 -- 172.21.15.125:0/1089645246 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe394003aa0 con 0x7fe39c141690 2023-10-18T10:25:20.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c0a74f0 con 0x7fe39c141690 2023-10-18T10:25:20.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe398ff9640 1 -- 172.21.15.125:0/1089645246 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394004030 con 0x7fe39c141690 2023-10-18T10:25:20.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.412+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c1391d0 con 0x7fe39c141690 2023-10-18T10:25:20.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.413+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c0a6e30 con 0x7fe39c141690 2023-10-18T10:25:20.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.414+0000 7fe398ff9640 1 -- 172.21.15.125:0/1089645246 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe394004190 con 0x7fe39c141690 2023-10-18T10:25:20.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.414+0000 7fe398ff9640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe38806ae40 0x7fe38806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:20.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.414+0000 7fe398ff9640 1 -- 172.21.15.125:0/1089645246 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe39408a050 con 0x7fe39c141690 2023-10-18T10:25:20.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.414+0000 7fe39bfff640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe38806ae40 0x7fe38806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:20.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.415+0000 7fe39bfff640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe38806ae40 0x7fe38806d300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe38c035000 tx=0x7fe38c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:20.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.419+0000 7fe398ff9640 1 -- 172.21.15.125:0/1089645246 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe39c0a6e30 con 0x7fe39c141690 2023-10-18T10:25:20.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.621+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe39c0a6e30 con 0x7fe39c141690 2023-10-18T10:25:20.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.621+0000 7fe398ff9640 1 -- 172.21.15.125:0/1089645246 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fe39c0a6e30 con 0x7fe39c141690 2023-10-18T10:25:20.624 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:20.624 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:19.519174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:25:20.624 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 9 2023-10-18T10:25:20.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe38806ae40 msgr2=0x7fe38806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe38806ae40 0x7fe38806d300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe38c035000 tx=0x7fe38c004730 comp rx=0 tx=0).stop 2023-10-18T10:25:20.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c141690 msgr2=0x7fe39c158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c141690 0x7fe39c158b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe39400cad0 tx=0x7fe39400cf90 comp rx=0 tx=0).stop 2023-10-18T10:25:20.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 shutdown_connections 2023-10-18T10:25:20.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe38806ae40 0x7fe38806d300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe39c141690 0x7fe39c158b40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe39c13e520 0x7fe39c158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 --2- 172.21.15.125:0/1089645246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39c070af0 0x7fe39c1580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.623+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 >> 172.21.15.125:0/1089645246 conn(0x7fe39c069930 msgr2=0x7fe39c06f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:20.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.624+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 shutdown_connections 2023-10-18T10:25:20.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.624+0000 7fe3a28a9640 1 -- 172.21.15.125:0/1089645246 wait complete. 2023-10-18T10:25:20.636 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:20.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.736+0000 7f93799a5640 1 -- 172.21.15.125:0/66005783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 msgr2=0x7f93740cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.736+0000 7f93799a5640 1 --2- 172.21.15.125:0/66005783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 0x7f93740cc200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f93740711c0 tx=0x7f936c02bd70 comp rx=0 tx=0).stop 2023-10-18T10:25:20.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.736+0000 7f93799a5640 1 -- 172.21.15.125:0/66005783 shutdown_connections 2023-10-18T10:25:20.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.736+0000 7f93799a5640 1 --2- 172.21.15.125:0/66005783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 0x7f93740cc200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.736+0000 7f93799a5640 1 --2- 172.21.15.125:0/66005783 >> v2:172.21.15.139:3300/0 conn(0x7f93741416f0 0x7f93740b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.736+0000 7f93799a5640 1 -- 172.21.15.125:0/66005783 >> 172.21.15.125:0/66005783 conn(0x7f93740af730 msgr2=0x7f93740afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:20.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.736+0000 7f93799a5640 1 -- 172.21.15.125:0/66005783 shutdown_connections 2023-10-18T10:25:20.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.736+0000 7f93799a5640 1 -- 172.21.15.125:0/66005783 wait complete. 2023-10-18T10:25:20.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.737+0000 7f93799a5640 1 Processor -- start 2023-10-18T10:25:20.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.737+0000 7f93799a5640 1 -- start start 2023-10-18T10:25:20.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f93799a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 0x7f9374134300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:20.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f93799a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93740b1980 0x7f937414d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:20.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f93799a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93741416f0 0x7f937414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:20.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f93799a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937413b470 con 0x7f9374070790 2023-10-18T10:25:20.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f93799a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9374157a10 con 0x7f93741416f0 2023-10-18T10:25:20.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f93799a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9374070ed0 con 0x7f93740b1980 2023-10-18T10:25:20.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 0x7f9374134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:20.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 0x7f9374134300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49492/0 (socket says 172.21.15.125:49492) 2023-10-18T10:25:20.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 -- 172.21.15.125:0/852161270 learned_addr learned my addr 172.21.15.125:0/852161270 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:20.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 -- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93740b1980 msgr2=0x7f937414d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93740b1980 0x7f937414d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 -- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93741416f0 msgr2=0x7f937414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93741416f0 0x7f937414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 -- 172.21.15.125:0/852161270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c009490 con 0x7f9374070790 2023-10-18T10:25:20.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f9372ffd640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 0x7f9374134300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f936400ea20 tx=0x7f936400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:20.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.738+0000 7f935bfff640 1 -- 172.21.15.125:0/852161270 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400cde0 con 0x7f9374070790 2023-10-18T10:25:20.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.739+0000 7f935bfff640 1 -- 172.21.15.125:0/852161270 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9364011560 con 0x7f9374070790 2023-10-18T10:25:20.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.739+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937413b090 con 0x7f9374070790 2023-10-18T10:25:20.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.739+0000 7f935bfff640 1 -- 172.21.15.125:0/852161270 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364011950 con 0x7f9374070790 2023-10-18T10:25:20.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.739+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9374134840 con 0x7f9374070790 2023-10-18T10:25:20.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.739+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93740675f0 con 0x7f9374070790 2023-10-18T10:25:20.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.740+0000 7f935bfff640 1 -- 172.21.15.125:0/852161270 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9364011ab0 con 0x7f9374070790 2023-10-18T10:25:20.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.740+0000 7f935bfff640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f936006ae40 0x7f936006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:20.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.740+0000 7f935bfff640 1 -- 172.21.15.125:0/852161270 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f936408a560 con 0x7f9374070790 2023-10-18T10:25:20.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.741+0000 7f93727fc640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f936006ae40 0x7f936006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:20.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.741+0000 7f93727fc640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f936006ae40 0x7f936006d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9368000f60 tx=0x7f9368007a30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:20.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.745+0000 7f935bfff640 1 -- 172.21.15.125:0/852161270 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93640553b0 con 0x7f9374070790 2023-10-18T10:25:20.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.911+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93740b2150 con 0x7f9374070790 2023-10-18T10:25:20.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.913+0000 7f935bfff640 1 -- 172.21.15.125:0/852161270 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f93640589d0 con 0x7f9374070790 2023-10-18T10:25:20.915 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:20.915 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:20.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f936006ae40 msgr2=0x7f936006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f936006ae40 0x7f936006d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9368000f60 tx=0x7f9368007a30 comp rx=0 tx=0).stop 2023-10-18T10:25:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 msgr2=0x7f9374134300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 0x7f9374134300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f936400ea20 tx=0x7f936400eee0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 shutdown_connections 2023-10-18T10:25:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f936006ae40 0x7f936006d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93741416f0 0x7f937414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93740b1980 0x7f937414d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.914+0000 7f93799a5640 1 --2- 172.21.15.125:0/852161270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9374070790 0x7f9374134300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.915+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 >> 172.21.15.125:0/852161270 conn(0x7f93740af730 msgr2=0x7f937406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.915+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 shutdown_connections 2023-10-18T10:25:20.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:20.915+0000 7f93799a5640 1 -- 172.21.15.125:0/852161270 wait complete. 2023-10-18T10:25:20.927 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:25:20.927 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:25:21.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.028+0000 7fa4995e9640 1 -- 172.21.15.125:0/3230205531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa47c004d00 msgr2=0x7fa47c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.028+0000 7fa4995e9640 1 --2- 172.21.15.125:0/3230205531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa47c004d00 0x7fa47c005180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009b50 tx=0x7fa47c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.028+0000 7fa4937fe640 1 -- 172.21.15.125:0/3230205531 reap_dead start 2023-10-18T10:25:21.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.028+0000 7fa4995e9640 1 -- 172.21.15.125:0/3230205531 shutdown_connections 2023-10-18T10:25:21.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.028+0000 7fa4995e9640 1 -- 172.21.15.125:0/3230205531 >> 172.21.15.125:0/3230205531 conn(0x7fa4940b0670 msgr2=0x7fa4940b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:21.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.028+0000 7fa4995e9640 1 -- 172.21.15.125:0/3230205531 shutdown_connections 2023-10-18T10:25:21.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.028+0000 7fa4995e9640 1 -- 172.21.15.125:0/3230205531 wait complete. 2023-10-18T10:25:21.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.029+0000 7fa4995e9640 1 Processor -- start 2023-10-18T10:25:21.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.029+0000 7fa4995e9640 1 -- start start 2023-10-18T10:25:21.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4995e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa494153ce0 0x7fa4940c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4995e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4941540c0 0x7fa4941e4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4995e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4941f3720 0x7fa4941e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4995e9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941e4770 con 0x7fa4941f3720 2023-10-18T10:25:21.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4995e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940c54f0 con 0x7fa4941540c0 2023-10-18T10:25:21.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4995e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941e41b0 con 0x7fa494153ce0 2023-10-18T10:25:21.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4941f3720 0x7fa4941e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4941f3720 0x7fa4941e4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49520/0 (socket says 172.21.15.125:49520) 2023-10-18T10:25:21.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 -- 172.21.15.125:0/3641648769 learned_addr learned my addr 172.21.15.125:0/3641648769 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:21.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa492ffd640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa494153ce0 0x7fa4940c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4927fc640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4941540c0 0x7fa4941e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 -- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa494153ce0 msgr2=0x7fa4940c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa494153ce0 0x7fa4940c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 -- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4941540c0 msgr2=0x7fa4941e4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4941540c0 0x7fa4941e4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 -- 172.21.15.125:0/3641648769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4941e4490 con 0x7fa4941f3720 2023-10-18T10:25:21.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4937fe640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4941f3720 0x7fa4941e4e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48c02cb70 tx=0x7fa48c034c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa47b7fe640 1 -- 172.21.15.125:0/3641648769 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c004230 con 0x7fa4941f3720 2023-10-18T10:25:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa47b7fe640 1 -- 172.21.15.125:0/3641648769 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa48c004390 con 0x7fa4941f3720 2023-10-18T10:25:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4940c2730 con 0x7fa4941f3720 2023-10-18T10:25:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.030+0000 7fa47b7fe640 1 -- 172.21.15.125:0/3641648769 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c003850 con 0x7fa4941f3720 2023-10-18T10:25:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.031+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4941e2cd0 con 0x7fa4941f3720 2023-10-18T10:25:21.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.032+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4940c2590 con 0x7fa4941f3720 2023-10-18T10:25:21.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.032+0000 7fa47b7fe640 1 -- 172.21.15.125:0/3641648769 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa48c0039b0 con 0x7fa4941f3720 2023-10-18T10:25:21.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.032+0000 7fa47b7fe640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa48006ae70 0x7fa48006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.032+0000 7fa47b7fe640 1 -- 172.21.15.125:0/3641648769 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa48c000e40 con 0x7fa4941f3720 2023-10-18T10:25:21.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.032+0000 7fa492ffd640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa48006ae70 0x7fa48006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.033+0000 7fa492ffd640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa48006ae70 0x7fa48006d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa484009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:21.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.036+0000 7fa47b7fe640 1 -- 172.21.15.125:0/3641648769 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa4940c2590 con 0x7fa4941f3720 2023-10-18T10:25:21.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.200+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa494159650 con 0x7fa48006ae70 2023-10-18T10:25:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.209+0000 7fa47b7fe640 1 -- 172.21.15.125:0/3641648769 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345262 (secure 0 0 0) 0x7fa484009c70 con 0x7fa48006ae70 2023-10-18T10:25:21.215 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:21.217 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:25:21.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa48006ae70 msgr2=0x7fa48006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa48006ae70 0x7fa48006d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa484009500 comp rx=0 tx=0).stop 2023-10-18T10:25:21.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4941f3720 msgr2=0x7fa4941e4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4941f3720 0x7fa4941e4e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48c02cb70 tx=0x7fa48c034c00 comp rx=0 tx=0).stop 2023-10-18T10:25:21.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 shutdown_connections 2023-10-18T10:25:21.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa48006ae70 0x7fa48006d330 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4941540c0 0x7fa4941e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4941f3720 0x7fa4941e4e40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 --2- 172.21.15.125:0/3641648769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa494153ce0 0x7fa4940c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 >> 172.21.15.125:0/3641648769 conn(0x7fa4940b0670 msgr2=0x7fa4940caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:21.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 shutdown_connections 2023-10-18T10:25:21.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.213+0000 7fa4995e9640 1 -- 172.21.15.125:0/3641648769 wait complete. 2023-10-18T10:25:21.225 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-18T10:25:20.570262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219484,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755236,"statfs":{"total":773094113280,"available":772869361664,"internally_reserved":0,"allocated":3383296,"data_stored":2025340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.308924"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254754+0000","last_change":"2023-10-18T10:25:03.211565+0000","last_active":"2023-10-18T10:25:08.254754+0000","last_peered":"2023-10-18T10:25:08.254754+0000","last_clean":"2023-10-18T10:25:08.254754+0000","last_became_active":"2023-10-18T10:25:03.211312+0000","last_became_peered":"2023-10-18T10:25:03.211312+0000","last_unstale":"2023-10-18T10:25:08.254754+0000","last_undegraded":"2023-10-18T10:25:08.254754+0000","last_fullsized":"2023-10-18T10:25:08.254754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254543+0000","last_change":"2023-10-18T10:25:04.232540+0000","last_active":"2023-10-18T10:25:08.254543+0000","last_peered":"2023-10-18T10:25:08.254543+0000","last_clean":"2023-10-18T10:25:08.254543+0000","last_became_active":"2023-10-18T10:25:04.232270+0000","last_became_peered":"2023-10-18T10:25:04.232270+0000","last_unstale":"2023-10-18T10:25:08.254543+0000","last_undegraded":"2023-10-18T10:25:08.254543+0000","last_fullsized":"2023-10-18T10:25:08.254543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920838+0000","last_change":"2023-10-18T10:25:08.375121+0000","last_active":"2023-10-18T10:25:09.920838+0000","last_peered":"2023-10-18T10:25:09.920838+0000","last_clean":"2023-10-18T10:25:09.920838+0000","last_became_active":"2023-10-18T10:25:08.374754+0000","last_became_peered":"2023-10-18T10:25:08.374754+0000","last_unstale":"2023-10-18T10:25:09.920838+0000","last_undegraded":"2023-10-18T10:25:09.920838+0000","last_fullsized":"2023-10-18T10:25:09.920838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:26.275317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254691+0000","last_change":"2023-10-18T10:25:03.209283+0000","last_active":"2023-10-18T10:25:08.254691+0000","last_peered":"2023-10-18T10:25:08.254691+0000","last_clean":"2023-10-18T10:25:08.254691+0000","last_became_active":"2023-10-18T10:25:03.208718+0000","last_became_peered":"2023-10-18T10:25:03.208718+0000","last_unstale":"2023-10-18T10:25:08.254691+0000","last_undegraded":"2023-10-18T10:25:08.254691+0000","last_fullsized":"2023-10-18T10:25:08.254691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:43.110925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254524+0000","last_change":"2023-10-18T10:25:04.239316+0000","last_active":"2023-10-18T10:25:08.254524+0000","last_peered":"2023-10-18T10:25:08.254524+0000","last_clean":"2023-10-18T10:25:08.254524+0000","last_became_active":"2023-10-18T10:25:04.235952+0000","last_became_peered":"2023-10-18T10:25:04.235952+0000","last_unstale":"2023-10-18T10:25:08.254524+0000","last_undegraded":"2023-10-18T10:25:08.254524+0000","last_fullsized":"2023-10-18T10:25:08.254524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:09.100769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263035+0000","last_change":"2023-10-18T10:25:08.267055+0000","last_active":"2023-10-18T10:25:09.263035+0000","last_peered":"2023-10-18T10:25:09.263035+0000","last_clean":"2023-10-18T10:25:09.263035+0000","last_became_active":"2023-10-18T10:25:08.266533+0000","last_became_peered":"2023-10-18T10:25:08.266533+0000","last_unstale":"2023-10-18T10:25:09.263035+0000","last_undegraded":"2023-10-18T10:25:09.263035+0000","last_fullsized":"2023-10-18T10:25:09.263035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:31.219679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370926+0000","last_change":"2023-10-18T10:25:03.212550+0000","last_active":"2023-10-18T10:25:08.370926+0000","last_peered":"2023-10-18T10:25:08.370926+0000","last_clean":"2023-10-18T10:25:08.370926+0000","last_became_active":"2023-10-18T10:25:03.211719+0000","last_became_peered":"2023-10-18T10:25:03.211719+0000","last_unstale":"2023-10-18T10:25:08.370926+0000","last_undegraded":"2023-10-18T10:25:08.370926+0000","last_fullsized":"2023-10-18T10:25:08.370926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257891+0000","last_change":"2023-10-18T10:25:04.235362+0000","last_active":"2023-10-18T10:25:08.257891+0000","last_peered":"2023-10-18T10:25:08.257891+0000","last_clean":"2023-10-18T10:25:08.257891+0000","last_became_active":"2023-10-18T10:25:04.235036+0000","last_became_peered":"2023-10-18T10:25:04.235036+0000","last_unstale":"2023-10-18T10:25:08.257891+0000","last_undegraded":"2023-10-18T10:25:08.257891+0000","last_fullsized":"2023-10-18T10:25:08.257891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:26.705420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263812+0000","last_change":"2023-10-18T10:25:08.270146+0000","last_active":"2023-10-18T10:25:09.263812+0000","last_peered":"2023-10-18T10:25:09.263812+0000","last_clean":"2023-10-18T10:25:09.263812+0000","last_became_active":"2023-10-18T10:25:08.269580+0000","last_became_peered":"2023-10-18T10:25:08.269580+0000","last_unstale":"2023-10-18T10:25:09.263812+0000","last_undegraded":"2023-10-18T10:25:09.263812+0000","last_fullsized":"2023-10-18T10:25:09.263812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370610+0000","last_change":"2023-10-18T10:25:03.216100+0000","last_active":"2023-10-18T10:25:08.370610+0000","last_peered":"2023-10-18T10:25:08.370610+0000","last_clean":"2023-10-18T10:25:08.370610+0000","last_became_active":"2023-10-18T10:25:03.215819+0000","last_became_peered":"2023-10-18T10:25:03.215819+0000","last_unstale":"2023-10-18T10:25:08.370610+0000","last_undegraded":"2023-10-18T10:25:08.370610+0000","last_fullsized":"2023-10-18T10:25:08.370610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:43.110925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256236+0000","last_change":"2023-10-18T10:25:04.236788+0000","last_active":"2023-10-18T10:25:08.256236+0000","last_peered":"2023-10-18T10:25:08.256236+0000","last_clean":"2023-10-18T10:25:08.256236+0000","last_became_active":"2023-10-18T10:25:04.236291+0000","last_became_peered":"2023-10-18T10:25:04.236291+0000","last_unstale":"2023-10-18T10:25:08.256236+0000","last_undegraded":"2023-10-18T10:25:08.256236+0000","last_fullsized":"2023-10-18T10:25:08.256236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:39.174524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263153+0000","last_change":"2023-10-18T10:25:08.265913+0000","last_active":"2023-10-18T10:25:09.263153+0000","last_peered":"2023-10-18T10:25:09.263153+0000","last_clean":"2023-10-18T10:25:09.263153+0000","last_became_active":"2023-10-18T10:25:08.264755+0000","last_became_peered":"2023-10-18T10:25:08.264755+0000","last_unstale":"2023-10-18T10:25:09.263153+0000","last_undegraded":"2023-10-18T10:25:09.263153+0000","last_fullsized":"2023-10-18T10:25:09.263153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:22.912468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265054+0000","last_change":"2023-10-18T10:25:04.230696+0000","last_active":"2023-10-18T10:25:08.265054+0000","last_peered":"2023-10-18T10:25:08.265054+0000","last_clean":"2023-10-18T10:25:08.265054+0000","last_became_active":"2023-10-18T10:25:04.230402+0000","last_became_peered":"2023-10-18T10:25:04.230402+0000","last_unstale":"2023-10-18T10:25:08.265054+0000","last_undegraded":"2023-10-18T10:25:08.265054+0000","last_fullsized":"2023-10-18T10:25:08.265054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:34.587493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370834+0000","last_change":"2023-10-18T10:25:03.210831+0000","last_active":"2023-10-18T10:25:08.370834+0000","last_peered":"2023-10-18T10:25:08.370834+0000","last_clean":"2023-10-18T10:25:08.370834+0000","last_became_active":"2023-10-18T10:25:03.210586+0000","last_became_peered":"2023-10-18T10:25:03.210586+0000","last_unstale":"2023-10-18T10:25:08.370834+0000","last_undegraded":"2023-10-18T10:25:08.370834+0000","last_fullsized":"2023-10-18T10:25:08.370834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263925+0000","last_change":"2023-10-18T10:25:08.270065+0000","last_active":"2023-10-18T10:25:09.263925+0000","last_peered":"2023-10-18T10:25:09.263925+0000","last_clean":"2023-10-18T10:25:09.263925+0000","last_became_active":"2023-10-18T10:25:08.269544+0000","last_became_peered":"2023-10-18T10:25:08.269544+0000","last_unstale":"2023-10-18T10:25:09.263925+0000","last_undegraded":"2023-10-18T10:25:09.263925+0000","last_fullsized":"2023-10-18T10:25:09.263925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253560+0000","last_change":"2023-10-18T10:25:04.237642+0000","last_active":"2023-10-18T10:25:08.253560+0000","last_peered":"2023-10-18T10:25:08.253560+0000","last_clean":"2023-10-18T10:25:08.253560+0000","last_became_active":"2023-10-18T10:25:04.237321+0000","last_became_peered":"2023-10-18T10:25:04.237321+0000","last_unstale":"2023-10-18T10:25:08.253560+0000","last_undegraded":"2023-10-18T10:25:08.253560+0000","last_fullsized":"2023-10-18T10:25:08.253560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254793+0000","last_change":"2023-10-18T10:25:03.207491+0000","last_active":"2023-10-18T10:25:08.254793+0000","last_peered":"2023-10-18T10:25:08.254793+0000","last_clean":"2023-10-18T10:25:08.254793+0000","last_became_active":"2023-10-18T10:25:03.207227+0000","last_became_peered":"2023-10-18T10:25:03.207227+0000","last_unstale":"2023-10-18T10:25:08.254793+0000","last_undegraded":"2023-10-18T10:25:08.254793+0000","last_fullsized":"2023-10-18T10:25:08.254793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263777+0000","last_change":"2023-10-18T10:25:08.270542+0000","last_active":"2023-10-18T10:25:09.263777+0000","last_peered":"2023-10-18T10:25:09.263777+0000","last_clean":"2023-10-18T10:25:09.263777+0000","last_became_active":"2023-10-18T10:25:08.270153+0000","last_became_peered":"2023-10-18T10:25:08.270153+0000","last_unstale":"2023-10-18T10:25:09.263777+0000","last_undegraded":"2023-10-18T10:25:09.263777+0000","last_fullsized":"2023-10-18T10:25:09.263777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254454+0000","last_change":"2023-10-18T10:25:04.230256+0000","last_active":"2023-10-18T10:25:08.254454+0000","last_peered":"2023-10-18T10:25:08.254454+0000","last_clean":"2023-10-18T10:25:08.254454+0000","last_became_active":"2023-10-18T10:25:04.230007+0000","last_became_peered":"2023-10-18T10:25:04.230007+0000","last_unstale":"2023-10-18T10:25:08.254454+0000","last_undegraded":"2023-10-18T10:25:08.254454+0000","last_fullsized":"2023-10-18T10:25:08.254454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253450+0000","last_change":"2023-10-18T10:25:03.203658+0000","last_active":"2023-10-18T10:25:08.253450+0000","last_peered":"2023-10-18T10:25:08.253450+0000","last_clean":"2023-10-18T10:25:08.253450+0000","last_became_active":"2023-10-18T10:25:03.203419+0000","last_became_peered":"2023-10-18T10:25:03.203419+0000","last_unstale":"2023-10-18T10:25:08.253450+0000","last_undegraded":"2023-10-18T10:25:08.253450+0000","last_fullsized":"2023-10-18T10:25:08.253450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262990+0000","last_change":"2023-10-18T10:25:08.373967+0000","last_active":"2023-10-18T10:25:09.262990+0000","last_peered":"2023-10-18T10:25:09.262990+0000","last_clean":"2023-10-18T10:25:09.262990+0000","last_became_active":"2023-10-18T10:25:08.373624+0000","last_became_peered":"2023-10-18T10:25:08.373624+0000","last_unstale":"2023-10-18T10:25:09.262990+0000","last_undegraded":"2023-10-18T10:25:09.262990+0000","last_fullsized":"2023-10-18T10:25:09.262990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:47.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257377+0000","last_change":"2023-10-18T10:25:04.227308+0000","last_active":"2023-10-18T10:25:08.257377+0000","last_peered":"2023-10-18T10:25:08.257377+0000","last_clean":"2023-10-18T10:25:08.257377+0000","last_became_active":"2023-10-18T10:25:04.227010+0000","last_became_peered":"2023-10-18T10:25:04.227010+0000","last_unstale":"2023-10-18T10:25:08.257377+0000","last_undegraded":"2023-10-18T10:25:08.257377+0000","last_fullsized":"2023-10-18T10:25:08.257377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:03.438820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369608+0000","last_change":"2023-10-18T10:25:03.210132+0000","last_active":"2023-10-18T10:25:08.369608+0000","last_peered":"2023-10-18T10:25:08.369608+0000","last_clean":"2023-10-18T10:25:08.369608+0000","last_became_active":"2023-10-18T10:25:03.209850+0000","last_became_peered":"2023-10-18T10:25:03.209850+0000","last_unstale":"2023-10-18T10:25:08.369608+0000","last_undegraded":"2023-10-18T10:25:08.369608+0000","last_fullsized":"2023-10-18T10:25:08.369608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263730+0000","last_change":"2023-10-18T10:25:08.270370+0000","last_active":"2023-10-18T10:25:09.263730+0000","last_peered":"2023-10-18T10:25:09.263730+0000","last_clean":"2023-10-18T10:25:09.263730+0000","last_became_active":"2023-10-18T10:25:08.269625+0000","last_became_peered":"2023-10-18T10:25:08.269625+0000","last_unstale":"2023-10-18T10:25:09.263730+0000","last_undegraded":"2023-10-18T10:25:09.263730+0000","last_fullsized":"2023-10-18T10:25:09.263730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:45.178929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257699+0000","last_change":"2023-10-18T10:25:04.239309+0000","last_active":"2023-10-18T10:25:08.257699+0000","last_peered":"2023-10-18T10:25:08.257699+0000","last_clean":"2023-10-18T10:25:08.257699+0000","last_became_active":"2023-10-18T10:25:04.236322+0000","last_became_peered":"2023-10-18T10:25:04.236322+0000","last_unstale":"2023-10-18T10:25:08.257699+0000","last_undegraded":"2023-10-18T10:25:08.257699+0000","last_fullsized":"2023-10-18T10:25:08.257699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264329+0000","last_change":"2023-10-18T10:25:03.211765+0000","last_active":"2023-10-18T10:25:08.264329+0000","last_peered":"2023-10-18T10:25:08.264329+0000","last_clean":"2023-10-18T10:25:08.264329+0000","last_became_active":"2023-10-18T10:25:03.211504+0000","last_became_peered":"2023-10-18T10:25:03.211504+0000","last_unstale":"2023-10-18T10:25:08.264329+0000","last_undegraded":"2023-10-18T10:25:08.264329+0000","last_fullsized":"2023-10-18T10:25:08.264329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:41.039503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263210+0000","last_change":"2023-10-18T10:25:08.374129+0000","last_active":"2023-10-18T10:25:09.263210+0000","last_peered":"2023-10-18T10:25:09.263210+0000","last_clean":"2023-10-18T10:25:09.263210+0000","last_became_active":"2023-10-18T10:25:08.373675+0000","last_became_peered":"2023-10-18T10:25:08.373675+0000","last_unstale":"2023-10-18T10:25:09.263210+0000","last_undegraded":"2023-10-18T10:25:09.263210+0000","last_fullsized":"2023-10-18T10:25:09.263210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:12.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255528+0000","last_change":"2023-10-18T10:25:04.232333+0000","last_active":"2023-10-18T10:25:08.255528+0000","last_peered":"2023-10-18T10:25:08.255528+0000","last_clean":"2023-10-18T10:25:08.255528+0000","last_became_active":"2023-10-18T10:25:04.232075+0000","last_became_peered":"2023-10-18T10:25:04.232075+0000","last_unstale":"2023-10-18T10:25:08.255528+0000","last_undegraded":"2023-10-18T10:25:08.255528+0000","last_fullsized":"2023-10-18T10:25:08.255528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:39.174524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255192+0000","last_change":"2023-10-18T10:25:03.209779+0000","last_active":"2023-10-18T10:25:08.255192+0000","last_peered":"2023-10-18T10:25:08.255192+0000","last_clean":"2023-10-18T10:25:08.255192+0000","last_became_active":"2023-10-18T10:25:03.209508+0000","last_became_peered":"2023-10-18T10:25:03.209508+0000","last_unstale":"2023-10-18T10:25:08.255192+0000","last_undegraded":"2023-10-18T10:25:08.255192+0000","last_fullsized":"2023-10-18T10:25:08.255192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:33.585439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920818+0000","last_change":"2023-10-18T10:25:08.372735+0000","last_active":"2023-10-18T10:25:09.920818+0000","last_peered":"2023-10-18T10:25:09.920818+0000","last_clean":"2023-10-18T10:25:09.920818+0000","last_became_active":"2023-10-18T10:25:08.372386+0000","last_became_peered":"2023-10-18T10:25:08.372386+0000","last_unstale":"2023-10-18T10:25:09.920818+0000","last_undegraded":"2023-10-18T10:25:09.920818+0000","last_fullsized":"2023-10-18T10:25:09.920818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:53.362022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255868+0000","last_change":"2023-10-18T10:25:04.240297+0000","last_active":"2023-10-18T10:25:08.255868+0000","last_peered":"2023-10-18T10:25:08.255868+0000","last_clean":"2023-10-18T10:25:08.255868+0000","last_became_active":"2023-10-18T10:25:04.239997+0000","last_became_peered":"2023-10-18T10:25:04.239997+0000","last_unstale":"2023-10-18T10:25:08.255868+0000","last_undegraded":"2023-10-18T10:25:08.255868+0000","last_fullsized":"2023-10-18T10:25:08.255868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371087+0000","last_change":"2023-10-18T10:25:03.213798+0000","last_active":"2023-10-18T10:25:08.371087+0000","last_peered":"2023-10-18T10:25:08.371087+0000","last_clean":"2023-10-18T10:25:08.371087+0000","last_became_active":"2023-10-18T10:25:03.213557+0000","last_became_peered":"2023-10-18T10:25:03.213557+0000","last_unstale":"2023-10-18T10:25:08.371087+0000","last_undegraded":"2023-10-18T10:25:08.371087+0000","last_fullsized":"2023-10-18T10:25:08.371087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:08.120358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307135+0000","last_change":"2023-10-18T10:25:08.370813+0000","last_active":"2023-10-18T10:25:10.307135+0000","last_peered":"2023-10-18T10:25:10.307135+0000","last_clean":"2023-10-18T10:25:10.307135+0000","last_became_active":"2023-10-18T10:25:08.370441+0000","last_became_peered":"2023-10-18T10:25:08.370441+0000","last_unstale":"2023-10-18T10:25:10.307135+0000","last_undegraded":"2023-10-18T10:25:10.307135+0000","last_fullsized":"2023-10-18T10:25:10.307135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256321+0000","last_change":"2023-10-18T10:25:04.242148+0000","last_active":"2023-10-18T10:25:08.256321+0000","last_peered":"2023-10-18T10:25:08.256321+0000","last_clean":"2023-10-18T10:25:08.256321+0000","last_became_active":"2023-10-18T10:25:04.239749+0000","last_became_peered":"2023-10-18T10:25:04.239749+0000","last_unstale":"2023-10-18T10:25:08.256321+0000","last_undegraded":"2023-10-18T10:25:08.256321+0000","last_fullsized":"2023-10-18T10:25:08.256321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255943+0000","last_change":"2023-10-18T10:25:03.205435+0000","last_active":"2023-10-18T10:25:08.255943+0000","last_peered":"2023-10-18T10:25:08.255943+0000","last_clean":"2023-10-18T10:25:08.255943+0000","last_became_active":"2023-10-18T10:25:03.205166+0000","last_became_peered":"2023-10-18T10:25:03.205166+0000","last_unstale":"2023-10-18T10:25:08.255943+0000","last_undegraded":"2023-10-18T10:25:08.255943+0000","last_fullsized":"2023-10-18T10:25:08.255943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:41.039503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262304+0000","last_change":"2023-10-18T10:25:08.267310+0000","last_active":"2023-10-18T10:25:09.262304+0000","last_peered":"2023-10-18T10:25:09.262304+0000","last_clean":"2023-10-18T10:25:09.262304+0000","last_became_active":"2023-10-18T10:25:08.266919+0000","last_became_peered":"2023-10-18T10:25:08.266919+0000","last_unstale":"2023-10-18T10:25:09.262304+0000","last_undegraded":"2023-10-18T10:25:09.262304+0000","last_fullsized":"2023-10-18T10:25:09.262304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257274+0000","last_change":"2023-10-18T10:25:04.234960+0000","last_active":"2023-10-18T10:25:08.257274+0000","last_peered":"2023-10-18T10:25:08.257274+0000","last_clean":"2023-10-18T10:25:08.257274+0000","last_became_active":"2023-10-18T10:25:04.234695+0000","last_became_peered":"2023-10-18T10:25:04.234695+0000","last_unstale":"2023-10-18T10:25:08.257274+0000","last_undegraded":"2023-10-18T10:25:08.257274+0000","last_fullsized":"2023-10-18T10:25:08.257274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:44.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254998+0000","last_change":"2023-10-18T10:25:03.217364+0000","last_active":"2023-10-18T10:25:08.254998+0000","last_peered":"2023-10-18T10:25:08.254998+0000","last_clean":"2023-10-18T10:25:08.254998+0000","last_became_active":"2023-10-18T10:25:03.217105+0000","last_became_peered":"2023-10-18T10:25:03.217105+0000","last_unstale":"2023-10-18T10:25:08.254998+0000","last_undegraded":"2023-10-18T10:25:08.254998+0000","last_fullsized":"2023-10-18T10:25:08.254998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262547+0000","last_change":"2023-10-18T10:25:08.268370+0000","last_active":"2023-10-18T10:25:09.262547+0000","last_peered":"2023-10-18T10:25:09.262547+0000","last_clean":"2023-10-18T10:25:09.262547+0000","last_became_active":"2023-10-18T10:25:08.267649+0000","last_became_peered":"2023-10-18T10:25:08.267649+0000","last_unstale":"2023-10-18T10:25:09.262547+0000","last_undegraded":"2023-10-18T10:25:09.262547+0000","last_fullsized":"2023-10-18T10:25:09.262547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253484+0000","last_change":"2023-10-18T10:25:04.229878+0000","last_active":"2023-10-18T10:25:08.253484+0000","last_peered":"2023-10-18T10:25:08.253484+0000","last_clean":"2023-10-18T10:25:08.253484+0000","last_became_active":"2023-10-18T10:25:04.227272+0000","last_became_peered":"2023-10-18T10:25:04.227272+0000","last_unstale":"2023-10-18T10:25:08.253484+0000","last_undegraded":"2023-10-18T10:25:08.253484+0000","last_fullsized":"2023-10-18T10:25:08.253484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250602+0000","last_change":"2023-10-18T10:25:03.209065+0000","last_active":"2023-10-18T10:25:08.250602+0000","last_peered":"2023-10-18T10:25:08.250602+0000","last_clean":"2023-10-18T10:25:08.250602+0000","last_became_active":"2023-10-18T10:25:03.208772+0000","last_became_peered":"2023-10-18T10:25:03.208772+0000","last_unstale":"2023-10-18T10:25:08.250602+0000","last_undegraded":"2023-10-18T10:25:08.250602+0000","last_fullsized":"2023-10-18T10:25:08.250602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:34.438877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262993+0000","last_change":"2023-10-18T10:25:08.266888+0000","last_active":"2023-10-18T10:25:09.262993+0000","last_peered":"2023-10-18T10:25:09.262993+0000","last_clean":"2023-10-18T10:25:09.262993+0000","last_became_active":"2023-10-18T10:25:08.266538+0000","last_became_peered":"2023-10-18T10:25:08.266538+0000","last_unstale":"2023-10-18T10:25:09.262993+0000","last_undegraded":"2023-10-18T10:25:09.262993+0000","last_fullsized":"2023-10-18T10:25:09.262993+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:53.048432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253849+0000","last_change":"2023-10-18T10:25:04.233140+0000","last_active":"2023-10-18T10:25:08.253849+0000","last_peered":"2023-10-18T10:25:08.253849+0000","last_clean":"2023-10-18T10:25:08.253849+0000","last_became_active":"2023-10-18T10:25:04.232753+0000","last_became_peered":"2023-10-18T10:25:04.232753+0000","last_unstale":"2023-10-18T10:25:08.253849+0000","last_undegraded":"2023-10-18T10:25:08.253849+0000","last_fullsized":"2023-10-18T10:25:08.253849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:27.915879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369629+0000","last_change":"2023-10-18T10:25:03.217333+0000","last_active":"2023-10-18T10:25:08.369629+0000","last_peered":"2023-10-18T10:25:08.369629+0000","last_clean":"2023-10-18T10:25:08.369629+0000","last_became_active":"2023-10-18T10:25:03.217060+0000","last_became_peered":"2023-10-18T10:25:03.217060+0000","last_unstale":"2023-10-18T10:25:08.369629+0000","last_undegraded":"2023-10-18T10:25:08.369629+0000","last_fullsized":"2023-10-18T10:25:08.369629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:02.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307375+0000","last_change":"2023-10-18T10:25:08.373111+0000","last_active":"2023-10-18T10:25:10.307375+0000","last_peered":"2023-10-18T10:25:10.307375+0000","last_clean":"2023-10-18T10:25:10.307375+0000","last_became_active":"2023-10-18T10:25:08.372684+0000","last_became_peered":"2023-10-18T10:25:08.372684+0000","last_unstale":"2023-10-18T10:25:10.307375+0000","last_undegraded":"2023-10-18T10:25:10.307375+0000","last_fullsized":"2023-10-18T10:25:10.307375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:53.048432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253864+0000","last_change":"2023-10-18T10:25:04.238187+0000","last_active":"2023-10-18T10:25:08.253864+0000","last_peered":"2023-10-18T10:25:08.253864+0000","last_clean":"2023-10-18T10:25:08.253864+0000","last_became_active":"2023-10-18T10:25:04.237921+0000","last_became_peered":"2023-10-18T10:25:04.237921+0000","last_unstale":"2023-10-18T10:25:08.253864+0000","last_undegraded":"2023-10-18T10:25:08.253864+0000","last_fullsized":"2023-10-18T10:25:08.253864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253911+0000","last_change":"2023-10-18T10:25:03.220362+0000","last_active":"2023-10-18T10:25:08.253911+0000","last_peered":"2023-10-18T10:25:08.253911+0000","last_clean":"2023-10-18T10:25:08.253911+0000","last_became_active":"2023-10-18T10:25:03.220150+0000","last_became_peered":"2023-10-18T10:25:03.220150+0000","last_unstale":"2023-10-18T10:25:08.253911+0000","last_undegraded":"2023-10-18T10:25:08.253911+0000","last_fullsized":"2023-10-18T10:25:08.253911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369864+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:25:08.369864+0000","last_peered":"2023-10-18T10:25:08.369864+0000","last_clean":"2023-10-18T10:25:08.369864+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:25:08.369864+0000","last_undegraded":"2023-10-18T10:25:08.369864+0000","last_fullsized":"2023-10-18T10:25:08.369864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307435+0000","last_change":"2023-10-18T10:25:08.374370+0000","last_active":"2023-10-18T10:25:10.307435+0000","last_peered":"2023-10-18T10:25:10.307435+0000","last_clean":"2023-10-18T10:25:10.307435+0000","last_became_active":"2023-10-18T10:25:08.372029+0000","last_became_peered":"2023-10-18T10:25:08.372029+0000","last_unstale":"2023-10-18T10:25:10.307435+0000","last_undegraded":"2023-10-18T10:25:10.307435+0000","last_fullsized":"2023-10-18T10:25:10.307435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:45.178929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370575+0000","last_change":"2023-10-18T10:25:04.239066+0000","last_active":"2023-10-18T10:25:08.370575+0000","last_peered":"2023-10-18T10:25:08.370575+0000","last_clean":"2023-10-18T10:25:08.370575+0000","last_became_active":"2023-10-18T10:25:04.238531+0000","last_became_peered":"2023-10-18T10:25:04.238531+0000","last_unstale":"2023-10-18T10:25:08.370575+0000","last_undegraded":"2023-10-18T10:25:08.370575+0000","last_fullsized":"2023-10-18T10:25:08.370575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250745+0000","last_change":"2023-10-18T10:25:03.213157+0000","last_active":"2023-10-18T10:25:08.250745+0000","last_peered":"2023-10-18T10:25:08.250745+0000","last_clean":"2023-10-18T10:25:08.250745+0000","last_became_active":"2023-10-18T10:25:03.212900+0000","last_became_peered":"2023-10-18T10:25:03.212900+0000","last_unstale":"2023-10-18T10:25:08.250745+0000","last_undegraded":"2023-10-18T10:25:08.250745+0000","last_fullsized":"2023-10-18T10:25:08.250745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:29:02.497235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307254+0000","last_change":"2023-10-18T10:25:08.370854+0000","last_active":"2023-10-18T10:25:10.307254+0000","last_peered":"2023-10-18T10:25:10.307254+0000","last_clean":"2023-10-18T10:25:10.307254+0000","last_became_active":"2023-10-18T10:25:08.370423+0000","last_became_peered":"2023-10-18T10:25:08.370423+0000","last_unstale":"2023-10-18T10:25:10.307254+0000","last_undegraded":"2023-10-18T10:25:10.307254+0000","last_fullsized":"2023-10-18T10:25:10.307254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:57.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257993+0000","last_change":"2023-10-18T10:25:04.239837+0000","last_active":"2023-10-18T10:25:08.257993+0000","last_peered":"2023-10-18T10:25:08.257993+0000","last_clean":"2023-10-18T10:25:08.257993+0000","last_became_active":"2023-10-18T10:25:04.239576+0000","last_became_peered":"2023-10-18T10:25:04.239576+0000","last_unstale":"2023-10-18T10:25:08.257993+0000","last_undegraded":"2023-10-18T10:25:08.257993+0000","last_fullsized":"2023-10-18T10:25:08.257993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:43.022445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253804+0000","last_change":"2023-10-18T10:25:03.204821+0000","last_active":"2023-10-18T10:25:08.253804+0000","last_peered":"2023-10-18T10:25:08.253804+0000","last_clean":"2023-10-18T10:25:08.253804+0000","last_became_active":"2023-10-18T10:25:03.204545+0000","last_became_peered":"2023-10-18T10:25:03.204545+0000","last_unstale":"2023-10-18T10:25:08.253804+0000","last_undegraded":"2023-10-18T10:25:08.253804+0000","last_fullsized":"2023-10-18T10:25:08.253804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307183+0000","last_change":"2023-10-18T10:25:08.265973+0000","last_active":"2023-10-18T10:25:10.307183+0000","last_peered":"2023-10-18T10:25:10.307183+0000","last_clean":"2023-10-18T10:25:10.307183+0000","last_became_active":"2023-10-18T10:25:08.265607+0000","last_became_peered":"2023-10-18T10:25:08.265607+0000","last_unstale":"2023-10-18T10:25:10.307183+0000","last_undegraded":"2023-10-18T10:25:10.307183+0000","last_fullsized":"2023-10-18T10:25:10.307183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371147+0000","last_change":"2023-10-18T10:25:04.245074+0000","last_active":"2023-10-18T10:25:08.371147+0000","last_peered":"2023-10-18T10:25:08.371147+0000","last_clean":"2023-10-18T10:25:08.371147+0000","last_became_active":"2023-10-18T10:25:04.239593+0000","last_became_peered":"2023-10-18T10:25:04.239593+0000","last_unstale":"2023-10-18T10:25:08.371147+0000","last_undegraded":"2023-10-18T10:25:08.371147+0000","last_fullsized":"2023-10-18T10:25:08.371147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:29.983102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250759+0000","last_change":"2023-10-18T10:25:03.206351+0000","last_active":"2023-10-18T10:25:08.250759+0000","last_peered":"2023-10-18T10:25:08.250759+0000","last_clean":"2023-10-18T10:25:08.250759+0000","last_became_active":"2023-10-18T10:25:03.206082+0000","last_became_peered":"2023-10-18T10:25:03.206082+0000","last_unstale":"2023-10-18T10:25:08.250759+0000","last_undegraded":"2023-10-18T10:25:08.250759+0000","last_fullsized":"2023-10-18T10:25:08.250759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307655+0000","last_change":"2023-10-18T10:25:08.372691+0000","last_active":"2023-10-18T10:25:10.307655+0000","last_peered":"2023-10-18T10:25:10.307655+0000","last_clean":"2023-10-18T10:25:10.307655+0000","last_became_active":"2023-10-18T10:25:08.372252+0000","last_became_peered":"2023-10-18T10:25:08.372252+0000","last_unstale":"2023-10-18T10:25:10.307655+0000","last_undegraded":"2023-10-18T10:25:10.307655+0000","last_fullsized":"2023-10-18T10:25:10.307655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:53.362022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370821+0000","last_change":"2023-10-18T10:25:04.237483+0000","last_active":"2023-10-18T10:25:08.370821+0000","last_peered":"2023-10-18T10:25:08.370821+0000","last_clean":"2023-10-18T10:25:08.370821+0000","last_became_active":"2023-10-18T10:25:04.237196+0000","last_became_peered":"2023-10-18T10:25:04.237196+0000","last_unstale":"2023-10-18T10:25:08.370821+0000","last_undegraded":"2023-10-18T10:25:08.370821+0000","last_fullsized":"2023-10-18T10:25:08.370821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:41.123375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254019+0000","last_change":"2023-10-18T10:25:03.202658+0000","last_active":"2023-10-18T10:25:08.254019+0000","last_peered":"2023-10-18T10:25:08.254019+0000","last_clean":"2023-10-18T10:25:08.254019+0000","last_became_active":"2023-10-18T10:25:03.202313+0000","last_became_peered":"2023-10-18T10:25:03.202313+0000","last_unstale":"2023-10-18T10:25:08.254019+0000","last_undegraded":"2023-10-18T10:25:08.254019+0000","last_fullsized":"2023-10-18T10:25:08.254019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:18:46.417045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262979+0000","last_change":"2023-10-18T10:25:08.273431+0000","last_active":"2023-10-18T10:25:09.262979+0000","last_peered":"2023-10-18T10:25:09.262979+0000","last_clean":"2023-10-18T10:25:09.262979+0000","last_became_active":"2023-10-18T10:25:08.273002+0000","last_became_peered":"2023-10-18T10:25:08.273002+0000","last_unstale":"2023-10-18T10:25:09.262979+0000","last_undegraded":"2023-10-18T10:25:09.262979+0000","last_fullsized":"2023-10-18T10:25:09.262979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:43.230078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371654+0000","last_change":"2023-10-18T10:25:04.237817+0000","last_active":"2023-10-18T10:25:08.371654+0000","last_peered":"2023-10-18T10:25:08.371654+0000","last_clean":"2023-10-18T10:25:08.371654+0000","last_became_active":"2023-10-18T10:25:04.237459+0000","last_became_peered":"2023-10-18T10:25:04.237459+0000","last_unstale":"2023-10-18T10:25:08.371654+0000","last_undegraded":"2023-10-18T10:25:08.371654+0000","last_fullsized":"2023-10-18T10:25:08.371654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250534+0000","last_change":"2023-10-18T10:25:03.204327+0000","last_active":"2023-10-18T10:25:08.250534+0000","last_peered":"2023-10-18T10:25:08.250534+0000","last_clean":"2023-10-18T10:25:08.250534+0000","last_became_active":"2023-10-18T10:25:03.204079+0000","last_became_peered":"2023-10-18T10:25:03.204079+0000","last_unstale":"2023-10-18T10:25:08.250534+0000","last_undegraded":"2023-10-18T10:25:08.250534+0000","last_fullsized":"2023-10-18T10:25:08.250534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:05.461286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307373+0000","last_change":"2023-10-18T10:25:08.268324+0000","last_active":"2023-10-18T10:25:10.307373+0000","last_peered":"2023-10-18T10:25:10.307373+0000","last_clean":"2023-10-18T10:25:10.307373+0000","last_became_active":"2023-10-18T10:25:08.267959+0000","last_became_peered":"2023-10-18T10:25:08.267959+0000","last_unstale":"2023-10-18T10:25:10.307373+0000","last_undegraded":"2023-10-18T10:25:10.307373+0000","last_fullsized":"2023-10-18T10:25:10.307373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:47.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263268+0000","last_change":"2023-10-18T10:25:04.234809+0000","last_active":"2023-10-18T10:25:08.263268+0000","last_peered":"2023-10-18T10:25:08.263268+0000","last_clean":"2023-10-18T10:25:08.263268+0000","last_became_active":"2023-10-18T10:25:04.234527+0000","last_became_peered":"2023-10-18T10:25:04.234527+0000","last_unstale":"2023-10-18T10:25:08.263268+0000","last_undegraded":"2023-10-18T10:25:08.263268+0000","last_fullsized":"2023-10-18T10:25:08.263268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263204+0000","last_change":"2023-10-18T10:25:03.207281+0000","last_active":"2023-10-18T10:25:08.263204+0000","last_peered":"2023-10-18T10:25:08.263204+0000","last_clean":"2023-10-18T10:25:08.263204+0000","last_became_active":"2023-10-18T10:25:03.206828+0000","last_became_peered":"2023-10-18T10:25:03.206828+0000","last_unstale":"2023-10-18T10:25:08.263204+0000","last_undegraded":"2023-10-18T10:25:08.263204+0000","last_fullsized":"2023-10-18T10:25:08.263204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921617+0000","last_change":"2023-10-18T10:25:08.372503+0000","last_active":"2023-10-18T10:25:09.921617+0000","last_peered":"2023-10-18T10:25:09.921617+0000","last_clean":"2023-10-18T10:25:09.921617+0000","last_became_active":"2023-10-18T10:25:08.371994+0000","last_became_peered":"2023-10-18T10:25:08.371994+0000","last_unstale":"2023-10-18T10:25:09.921617+0000","last_undegraded":"2023-10-18T10:25:09.921617+0000","last_fullsized":"2023-10-18T10:25:09.921617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.553280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370729+0000","last_change":"2023-10-18T10:25:04.237797+0000","last_active":"2023-10-18T10:25:08.370729+0000","last_peered":"2023-10-18T10:25:08.370729+0000","last_clean":"2023-10-18T10:25:08.370729+0000","last_became_active":"2023-10-18T10:25:04.237473+0000","last_became_peered":"2023-10-18T10:25:04.237473+0000","last_unstale":"2023-10-18T10:25:08.370729+0000","last_undegraded":"2023-10-18T10:25:08.370729+0000","last_fullsized":"2023-10-18T10:25:08.370729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:48.992878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254912+0000","last_change":"2023-10-18T10:25:03.213281+0000","last_active":"2023-10-18T10:25:08.254912+0000","last_peered":"2023-10-18T10:25:08.254912+0000","last_clean":"2023-10-18T10:25:08.254912+0000","last_became_active":"2023-10-18T10:25:03.213019+0000","last_became_peered":"2023-10-18T10:25:03.213019+0000","last_unstale":"2023-10-18T10:25:08.254912+0000","last_undegraded":"2023-10-18T10:25:08.254912+0000","last_fullsized":"2023-10-18T10:25:08.254912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:07.702798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921380+0000","last_change":"2023-10-18T10:25:08.370771+0000","last_active":"2023-10-18T10:25:09.921380+0000","last_peered":"2023-10-18T10:25:09.921380+0000","last_clean":"2023-10-18T10:25:09.921380+0000","last_became_active":"2023-10-18T10:25:08.370438+0000","last_became_peered":"2023-10-18T10:25:08.370438+0000","last_unstale":"2023-10-18T10:25:09.921380+0000","last_undegraded":"2023-10-18T10:25:09.921380+0000","last_fullsized":"2023-10-18T10:25:09.921380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:31.794489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254101+0000","last_change":"2023-10-18T10:25:04.232734+0000","last_active":"2023-10-18T10:25:08.254101+0000","last_peered":"2023-10-18T10:25:08.254101+0000","last_clean":"2023-10-18T10:25:08.254101+0000","last_became_active":"2023-10-18T10:25:04.232474+0000","last_became_peered":"2023-10-18T10:25:04.232474+0000","last_unstale":"2023-10-18T10:25:08.254101+0000","last_undegraded":"2023-10-18T10:25:08.254101+0000","last_fullsized":"2023-10-18T10:25:08.254101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257928+0000","last_change":"2023-10-18T10:25:03.202060+0000","last_active":"2023-10-18T10:25:08.257928+0000","last_peered":"2023-10-18T10:25:08.257928+0000","last_clean":"2023-10-18T10:25:08.257928+0000","last_became_active":"2023-10-18T10:25:03.201719+0000","last_became_peered":"2023-10-18T10:25:03.201719+0000","last_unstale":"2023-10-18T10:25:08.257928+0000","last_undegraded":"2023-10-18T10:25:08.257928+0000","last_fullsized":"2023-10-18T10:25:08.257928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.308982+0000","last_change":"2023-10-18T10:25:08.370830+0000","last_active":"2023-10-18T10:25:10.308982+0000","last_peered":"2023-10-18T10:25:10.308982+0000","last_clean":"2023-10-18T10:25:10.308982+0000","last_became_active":"2023-10-18T10:25:08.370387+0000","last_became_peered":"2023-10-18T10:25:08.370387+0000","last_unstale":"2023-10-18T10:25:10.308982+0000","last_undegraded":"2023-10-18T10:25:10.308982+0000","last_fullsized":"2023-10-18T10:25:10.308982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:26.275317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370740+0000","last_change":"2023-10-18T10:25:04.241881+0000","last_active":"2023-10-18T10:25:08.370740+0000","last_peered":"2023-10-18T10:25:08.370740+0000","last_clean":"2023-10-18T10:25:08.370740+0000","last_became_active":"2023-10-18T10:25:04.241615+0000","last_became_peered":"2023-10-18T10:25:04.241615+0000","last_unstale":"2023-10-18T10:25:08.370740+0000","last_undegraded":"2023-10-18T10:25:08.370740+0000","last_fullsized":"2023-10-18T10:25:08.370740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:13.052474+0000","last_change":"2023-10-18T10:25:03.211150+0000","last_active":"2023-10-18T10:25:13.052474+0000","last_peered":"2023-10-18T10:25:13.052474+0000","last_clean":"2023-10-18T10:25:13.052474+0000","last_became_active":"2023-10-18T10:25:03.210879+0000","last_became_peered":"2023-10-18T10:25:03.210879+0000","last_unstale":"2023-10-18T10:25:13.052474+0000","last_undegraded":"2023-10-18T10:25:13.052474+0000","last_fullsized":"2023-10-18T10:25:13.052474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262397+0000","last_change":"2023-10-18T10:25:08.268389+0000","last_active":"2023-10-18T10:25:09.262397+0000","last_peered":"2023-10-18T10:25:09.262397+0000","last_clean":"2023-10-18T10:25:09.262397+0000","last_became_active":"2023-10-18T10:25:08.268025+0000","last_became_peered":"2023-10-18T10:25:08.268025+0000","last_unstale":"2023-10-18T10:25:09.262397+0000","last_undegraded":"2023-10-18T10:25:09.262397+0000","last_fullsized":"2023-10-18T10:25:09.262397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256542+0000","last_change":"2023-10-18T10:25:04.229742+0000","last_active":"2023-10-18T10:25:08.256542+0000","last_peered":"2023-10-18T10:25:08.256542+0000","last_clean":"2023-10-18T10:25:08.256542+0000","last_became_active":"2023-10-18T10:25:04.229281+0000","last_became_peered":"2023-10-18T10:25:04.229281+0000","last_unstale":"2023-10-18T10:25:08.256542+0000","last_undegraded":"2023-10-18T10:25:08.256542+0000","last_fullsized":"2023-10-18T10:25:08.256542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:08.704852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253721+0000","last_change":"2023-10-18T10:25:03.213327+0000","last_active":"2023-10-18T10:25:08.253721+0000","last_peered":"2023-10-18T10:25:08.253721+0000","last_clean":"2023-10-18T10:25:08.253721+0000","last_became_active":"2023-10-18T10:25:03.212993+0000","last_became_peered":"2023-10-18T10:25:03.212993+0000","last_unstale":"2023-10-18T10:25:08.253721+0000","last_undegraded":"2023-10-18T10:25:08.253721+0000","last_fullsized":"2023-10-18T10:25:08.253721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263300+0000","last_change":"2023-10-18T10:25:08.271366+0000","last_active":"2023-10-18T10:25:09.263300+0000","last_peered":"2023-10-18T10:25:09.263300+0000","last_clean":"2023-10-18T10:25:09.263300+0000","last_became_active":"2023-10-18T10:25:08.270581+0000","last_became_peered":"2023-10-18T10:25:08.270581+0000","last_unstale":"2023-10-18T10:25:09.263300+0000","last_undegraded":"2023-10-18T10:25:09.263300+0000","last_fullsized":"2023-10-18T10:25:09.263300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:57.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254604+0000","last_change":"2023-10-18T10:25:04.234658+0000","last_active":"2023-10-18T10:25:08.254604+0000","last_peered":"2023-10-18T10:25:08.254604+0000","last_clean":"2023-10-18T10:25:08.254604+0000","last_became_active":"2023-10-18T10:25:04.234334+0000","last_became_peered":"2023-10-18T10:25:04.234334+0000","last_unstale":"2023-10-18T10:25:08.254604+0000","last_undegraded":"2023-10-18T10:25:08.254604+0000","last_fullsized":"2023-10-18T10:25:08.254604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254170+0000","last_change":"2023-10-18T10:25:03.213955+0000","last_active":"2023-10-18T10:25:08.254170+0000","last_peered":"2023-10-18T10:25:08.254170+0000","last_clean":"2023-10-18T10:25:08.254170+0000","last_became_active":"2023-10-18T10:25:03.213692+0000","last_became_peered":"2023-10-18T10:25:03.213692+0000","last_unstale":"2023-10-18T10:25:08.254170+0000","last_undegraded":"2023-10-18T10:25:08.254170+0000","last_fullsized":"2023-10-18T10:25:08.254170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:21.099526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920943+0000","last_change":"2023-10-18T10:25:08.271334+0000","last_active":"2023-10-18T10:25:09.920943+0000","last_peered":"2023-10-18T10:25:09.920943+0000","last_clean":"2023-10-18T10:25:09.920943+0000","last_became_active":"2023-10-18T10:25:08.270900+0000","last_became_peered":"2023-10-18T10:25:08.270900+0000","last_unstale":"2023-10-18T10:25:09.920943+0000","last_undegraded":"2023-10-18T10:25:09.920943+0000","last_fullsized":"2023-10-18T10:25:09.920943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:53:25.617273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254094+0000","last_change":"2023-10-18T10:25:04.238611+0000","last_active":"2023-10-18T10:25:08.254094+0000","last_peered":"2023-10-18T10:25:08.254094+0000","last_clean":"2023-10-18T10:25:08.254094+0000","last_became_active":"2023-10-18T10:25:04.238304+0000","last_became_peered":"2023-10-18T10:25:04.238304+0000","last_unstale":"2023-10-18T10:25:08.254094+0000","last_undegraded":"2023-10-18T10:25:08.254094+0000","last_fullsized":"2023-10-18T10:25:08.254094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264215+0000","last_change":"2023-10-18T10:25:03.213189+0000","last_active":"2023-10-18T10:25:08.264215+0000","last_peered":"2023-10-18T10:25:08.264215+0000","last_clean":"2023-10-18T10:25:08.264215+0000","last_became_active":"2023-10-18T10:25:03.212915+0000","last_became_peered":"2023-10-18T10:25:03.212915+0000","last_unstale":"2023-10-18T10:25:08.264215+0000","last_undegraded":"2023-10-18T10:25:08.264215+0000","last_fullsized":"2023-10-18T10:25:08.264215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920935+0000","last_change":"2023-10-18T10:25:08.374903+0000","last_active":"2023-10-18T10:25:09.920935+0000","last_peered":"2023-10-18T10:25:09.920935+0000","last_clean":"2023-10-18T10:25:09.920935+0000","last_became_active":"2023-10-18T10:25:08.374525+0000","last_became_peered":"2023-10-18T10:25:08.374525+0000","last_unstale":"2023-10-18T10:25:09.920935+0000","last_undegraded":"2023-10-18T10:25:09.920935+0000","last_fullsized":"2023-10-18T10:25:09.920935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:38.455325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307246+0000","last_change":"2023-10-18T10:25:08.372484+0000","last_active":"2023-10-18T10:25:10.307246+0000","last_peered":"2023-10-18T10:25:10.307246+0000","last_clean":"2023-10-18T10:25:10.307246+0000","last_became_active":"2023-10-18T10:25:08.372132+0000","last_became_peered":"2023-10-18T10:25:08.372132+0000","last_unstale":"2023-10-18T10:25:10.307246+0000","last_undegraded":"2023-10-18T10:25:10.307246+0000","last_fullsized":"2023-10-18T10:25:10.307246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253285+0000","last_change":"2023-10-18T10:25:04.239197+0000","last_active":"2023-10-18T10:25:08.253285+0000","last_peered":"2023-10-18T10:25:08.253285+0000","last_clean":"2023-10-18T10:25:08.253285+0000","last_became_active":"2023-10-18T10:25:04.238917+0000","last_became_peered":"2023-10-18T10:25:04.238917+0000","last_unstale":"2023-10-18T10:25:08.253285+0000","last_undegraded":"2023-10-18T10:25:08.253285+0000","last_fullsized":"2023-10-18T10:25:08.253285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254371+0000","last_change":"2023-10-18T10:25:03.207877+0000","last_active":"2023-10-18T10:25:08.254371+0000","last_peered":"2023-10-18T10:25:08.254371+0000","last_clean":"2023-10-18T10:25:08.254371+0000","last_became_active":"2023-10-18T10:25:03.207621+0000","last_became_peered":"2023-10-18T10:25:03.207621+0000","last_unstale":"2023-10-18T10:25:08.254371+0000","last_undegraded":"2023-10-18T10:25:08.254371+0000","last_fullsized":"2023-10-18T10:25:08.254371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:52.476561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.308246+0000","last_change":"2023-10-18T10:25:08.272814+0000","last_active":"2023-10-18T10:25:10.308246+0000","last_peered":"2023-10-18T10:25:10.308246+0000","last_clean":"2023-10-18T10:25:10.308246+0000","last_became_active":"2023-10-18T10:25:08.272372+0000","last_became_peered":"2023-10-18T10:25:08.272372+0000","last_unstale":"2023-10-18T10:25:10.308246+0000","last_undegraded":"2023-10-18T10:25:10.308246+0000","last_fullsized":"2023-10-18T10:25:10.308246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254375+0000","last_change":"2023-10-18T10:25:04.237028+0000","last_active":"2023-10-18T10:25:08.254375+0000","last_peered":"2023-10-18T10:25:08.254375+0000","last_clean":"2023-10-18T10:25:08.254375+0000","last_became_active":"2023-10-18T10:25:04.236620+0000","last_became_peered":"2023-10-18T10:25:04.236620+0000","last_unstale":"2023-10-18T10:25:08.254375+0000","last_undegraded":"2023-10-18T10:25:08.254375+0000","last_fullsized":"2023-10-18T10:25:08.254375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255920+0000","last_change":"2023-10-18T10:25:03.212737+0000","last_active":"2023-10-18T10:25:08.255920+0000","last_peered":"2023-10-18T10:25:08.255920+0000","last_clean":"2023-10-18T10:25:08.255920+0000","last_became_active":"2023-10-18T10:25:03.212444+0000","last_became_peered":"2023-10-18T10:25:03.212444+0000","last_unstale":"2023-10-18T10:25:08.255920+0000","last_undegraded":"2023-10-18T10:25:08.255920+0000","last_fullsized":"2023-10-18T10:25:08.255920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:53.307830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265148+0000","last_change":"2023-10-18T10:25:03.208047+0000","last_active":"2023-10-18T10:25:08.265148+0000","last_peered":"2023-10-18T10:25:08.265148+0000","last_clean":"2023-10-18T10:25:08.265148+0000","last_became_active":"2023-10-18T10:25:03.207745+0000","last_became_peered":"2023-10-18T10:25:03.207745+0000","last_unstale":"2023-10-18T10:25:08.265148+0000","last_undegraded":"2023-10-18T10:25:08.265148+0000","last_fullsized":"2023-10-18T10:25:08.265148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255246+0000","last_change":"2023-10-18T10:25:04.240334+0000","last_active":"2023-10-18T10:25:08.255246+0000","last_peered":"2023-10-18T10:25:08.255246+0000","last_clean":"2023-10-18T10:25:08.255246+0000","last_became_active":"2023-10-18T10:25:04.239964+0000","last_became_peered":"2023-10-18T10:25:04.239964+0000","last_unstale":"2023-10-18T10:25:08.255246+0000","last_undegraded":"2023-10-18T10:25:08.255246+0000","last_fullsized":"2023-10-18T10:25:08.255246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.294555+0000","last_change":"2023-10-18T10:25:08.264225+0000","last_active":"2023-10-18T10:25:09.294555+0000","last_peered":"2023-10-18T10:25:09.294555+0000","last_clean":"2023-10-18T10:25:09.294555+0000","last_became_active":"2023-10-18T10:25:08.263712+0000","last_became_peered":"2023-10-18T10:25:08.263712+0000","last_unstale":"2023-10-18T10:25:09.294555+0000","last_undegraded":"2023-10-18T10:25:09.294555+0000","last_fullsized":"2023-10-18T10:25:09.294555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:13.156323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253645+0000","last_change":"2023-10-18T10:25:03.211608+0000","last_active":"2023-10-18T10:25:08.253645+0000","last_peered":"2023-10-18T10:25:08.253645+0000","last_clean":"2023-10-18T10:25:08.253645+0000","last_became_active":"2023-10-18T10:25:03.211346+0000","last_became_peered":"2023-10-18T10:25:03.211346+0000","last_unstale":"2023-10-18T10:25:08.253645+0000","last_undegraded":"2023-10-18T10:25:08.253645+0000","last_fullsized":"2023-10-18T10:25:08.253645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255576+0000","last_change":"2023-10-18T10:25:04.242544+0000","last_active":"2023-10-18T10:25:08.255576+0000","last_peered":"2023-10-18T10:25:08.255576+0000","last_clean":"2023-10-18T10:25:08.255576+0000","last_became_active":"2023-10-18T10:25:04.242258+0000","last_became_peered":"2023-10-18T10:25:04.242258+0000","last_unstale":"2023-10-18T10:25:08.255576+0000","last_undegraded":"2023-10-18T10:25:08.255576+0000","last_fullsized":"2023-10-18T10:25:08.255576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:55.422361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921493+0000","last_change":"2023-10-18T10:25:08.273772+0000","last_active":"2023-10-18T10:25:09.921493+0000","last_peered":"2023-10-18T10:25:09.921493+0000","last_clean":"2023-10-18T10:25:09.921493+0000","last_became_active":"2023-10-18T10:25:08.273410+0000","last_became_peered":"2023-10-18T10:25:08.273410+0000","last_unstale":"2023-10-18T10:25:09.921493+0000","last_undegraded":"2023-10-18T10:25:09.921493+0000","last_fullsized":"2023-10-18T10:25:09.921493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:19.382604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263176+0000","last_change":"2023-10-18T10:25:04.235479+0000","last_active":"2023-10-18T10:25:08.263176+0000","last_peered":"2023-10-18T10:25:08.263176+0000","last_clean":"2023-10-18T10:25:08.263176+0000","last_became_active":"2023-10-18T10:25:04.235205+0000","last_became_peered":"2023-10-18T10:25:04.235205+0000","last_unstale":"2023-10-18T10:25:08.263176+0000","last_undegraded":"2023-10-18T10:25:08.263176+0000","last_fullsized":"2023-10-18T10:25:08.263176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250534+0000","last_change":"2023-10-18T10:25:04.236754+0000","last_active":"2023-10-18T10:25:08.250534+0000","last_peered":"2023-10-18T10:25:08.250534+0000","last_clean":"2023-10-18T10:25:08.250534+0000","last_became_active":"2023-10-18T10:25:04.236482+0000","last_became_peered":"2023-10-18T10:25:04.236482+0000","last_unstale":"2023-10-18T10:25:08.250534+0000","last_undegraded":"2023-10-18T10:25:08.250534+0000","last_fullsized":"2023-10-18T10:25:08.250534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:08.704852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254828+0000","last_change":"2023-10-18T10:25:04.239429+0000","last_active":"2023-10-18T10:25:08.254828+0000","last_peered":"2023-10-18T10:25:08.254828+0000","last_clean":"2023-10-18T10:25:08.254828+0000","last_became_active":"2023-10-18T10:25:04.239185+0000","last_became_peered":"2023-10-18T10:25:04.239185+0000","last_unstale":"2023-10-18T10:25:08.254828+0000","last_undegraded":"2023-10-18T10:25:08.254828+0000","last_fullsized":"2023-10-18T10:25:08.254828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:41.123375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255164+0000","last_change":"2023-10-18T10:25:04.235523+0000","last_active":"2023-10-18T10:25:08.255164+0000","last_peered":"2023-10-18T10:25:08.255164+0000","last_clean":"2023-10-18T10:25:08.255164+0000","last_became_active":"2023-10-18T10:25:04.235266+0000","last_became_peered":"2023-10-18T10:25:04.235266+0000","last_unstale":"2023-10-18T10:25:08.255164+0000","last_undegraded":"2023-10-18T10:25:08.255164+0000","last_fullsized":"2023-10-18T10:25:08.255164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254545+0000","last_change":"2023-10-18T10:25:04.225083+0000","last_active":"2023-10-18T10:25:08.254545+0000","last_peered":"2023-10-18T10:25:08.254545+0000","last_clean":"2023-10-18T10:25:08.254545+0000","last_became_active":"2023-10-18T10:25:04.224243+0000","last_became_peered":"2023-10-18T10:25:04.224243+0000","last_unstale":"2023-10-18T10:25:08.254545+0000","last_undegraded":"2023-10-18T10:25:08.254545+0000","last_fullsized":"2023-10-18T10:25:08.254545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253638+0000","last_change":"2023-10-18T10:25:04.239380+0000","last_active":"2023-10-18T10:25:08.253638+0000","last_peered":"2023-10-18T10:25:08.253638+0000","last_clean":"2023-10-18T10:25:08.253638+0000","last_became_active":"2023-10-18T10:25:04.239121+0000","last_became_peered":"2023-10-18T10:25:04.239121+0000","last_unstale":"2023-10-18T10:25:08.253638+0000","last_undegraded":"2023-10-18T10:25:08.253638+0000","last_fullsized":"2023-10-18T10:25:08.253638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265703+0000","last_change":"2023-10-18T10:25:04.231489+0000","last_active":"2023-10-18T10:25:08.265703+0000","last_peered":"2023-10-18T10:25:08.265703+0000","last_clean":"2023-10-18T10:25:08.265703+0000","last_became_active":"2023-10-18T10:25:04.231228+0000","last_became_peered":"2023-10-18T10:25:04.231228+0000","last_unstale":"2023-10-18T10:25:08.265703+0000","last_undegraded":"2023-10-18T10:25:08.265703+0000","last_fullsized":"2023-10-18T10:25:08.265703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255438+0000","last_change":"2023-10-18T10:25:04.226121+0000","last_active":"2023-10-18T10:25:08.255438+0000","last_peered":"2023-10-18T10:25:08.255438+0000","last_clean":"2023-10-18T10:25:08.255438+0000","last_became_active":"2023-10-18T10:25:04.225472+0000","last_became_peered":"2023-10-18T10:25:04.225472+0000","last_unstale":"2023-10-18T10:25:08.255438+0000","last_undegraded":"2023-10-18T10:25:08.255438+0000","last_fullsized":"2023-10-18T10:25:08.255438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254296+0000","last_change":"2023-10-18T10:25:04.239623+0000","last_active":"2023-10-18T10:25:08.254296+0000","last_peered":"2023-10-18T10:25:08.254296+0000","last_clean":"2023-10-18T10:25:08.254296+0000","last_became_active":"2023-10-18T10:25:04.239376+0000","last_became_peered":"2023-10-18T10:25:04.239376+0000","last_unstale":"2023-10-18T10:25:08.254296+0000","last_undegraded":"2023-10-18T10:25:08.254296+0000","last_fullsized":"2023-10-18T10:25:08.254296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.789045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255778+0000","last_change":"2023-10-18T10:25:04.224851+0000","last_active":"2023-10-18T10:25:08.255778+0000","last_peered":"2023-10-18T10:25:08.255778+0000","last_clean":"2023-10-18T10:25:08.255778+0000","last_became_active":"2023-10-18T10:25:04.224579+0000","last_became_peered":"2023-10-18T10:25:04.224579+0000","last_unstale":"2023-10-18T10:25:08.255778+0000","last_undegraded":"2023-10-18T10:25:08.255778+0000","last_fullsized":"2023-10-18T10:25:08.255778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254745+0000","last_change":"2023-10-18T10:25:04.233040+0000","last_active":"2023-10-18T10:25:08.254745+0000","last_peered":"2023-10-18T10:25:08.254745+0000","last_clean":"2023-10-18T10:25:08.254745+0000","last_became_active":"2023-10-18T10:25:04.232782+0000","last_became_peered":"2023-10-18T10:25:04.232782+0000","last_unstale":"2023-10-18T10:25:08.254745+0000","last_undegraded":"2023-10-18T10:25:08.254745+0000","last_fullsized":"2023-10-18T10:25:08.254745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255053+0000","last_change":"2023-10-18T10:25:04.235058+0000","last_active":"2023-10-18T10:25:08.255053+0000","last_peered":"2023-10-18T10:25:08.255053+0000","last_clean":"2023-10-18T10:25:08.255053+0000","last_became_active":"2023-10-18T10:25:04.234763+0000","last_became_peered":"2023-10-18T10:25:04.234763+0000","last_unstale":"2023-10-18T10:25:08.255053+0000","last_undegraded":"2023-10-18T10:25:08.255053+0000","last_fullsized":"2023-10-18T10:25:08.255053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263580+0000","last_change":"2023-10-18T10:25:04.231136+0000","last_active":"2023-10-18T10:25:08.263580+0000","last_peered":"2023-10-18T10:25:08.263580+0000","last_clean":"2023-10-18T10:25:08.263580+0000","last_became_active":"2023-10-18T10:25:04.230835+0000","last_became_peered":"2023-10-18T10:25:04.230835+0000","last_unstale":"2023-10-18T10:25:08.263580+0000","last_undegraded":"2023-10-18T10:25:08.263580+0000","last_fullsized":"2023-10-18T10:25:08.263580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370824+0000","last_change":"2023-10-18T10:25:04.236843+0000","last_active":"2023-10-18T10:25:08.370824+0000","last_peered":"2023-10-18T10:25:08.370824+0000","last_clean":"2023-10-18T10:25:08.370824+0000","last_became_active":"2023-10-18T10:25:04.236484+0000","last_became_peered":"2023-10-18T10:25:04.236484+0000","last_unstale":"2023-10-18T10:25:08.370824+0000","last_undegraded":"2023-10-18T10:25:08.370824+0000","last_fullsized":"2023-10-18T10:25:08.370824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.789045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250697+0000","last_change":"2023-10-18T10:25:04.230906+0000","last_active":"2023-10-18T10:25:08.250697+0000","last_peered":"2023-10-18T10:25:08.250697+0000","last_clean":"2023-10-18T10:25:08.250697+0000","last_became_active":"2023-10-18T10:25:04.230641+0000","last_became_peered":"2023-10-18T10:25:04.230641+0000","last_unstale":"2023-10-18T10:25:08.250697+0000","last_undegraded":"2023-10-18T10:25:08.250697+0000","last_fullsized":"2023-10-18T10:25:08.250697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:44.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254323+0000","last_change":"2023-10-18T10:25:04.235839+0000","last_active":"2023-10-18T10:25:08.254323+0000","last_peered":"2023-10-18T10:25:08.254323+0000","last_clean":"2023-10-18T10:25:08.254323+0000","last_became_active":"2023-10-18T10:25:04.235602+0000","last_became_peered":"2023-10-18T10:25:04.235602+0000","last_unstale":"2023-10-18T10:25:08.254323+0000","last_undegraded":"2023-10-18T10:25:08.254323+0000","last_fullsized":"2023-10-18T10:25:08.254323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254219+0000","last_change":"2023-10-18T10:25:04.235249+0000","last_active":"2023-10-18T10:25:08.254219+0000","last_peered":"2023-10-18T10:25:08.254219+0000","last_clean":"2023-10-18T10:25:08.254219+0000","last_became_active":"2023-10-18T10:25:04.234967+0000","last_became_peered":"2023-10-18T10:25:04.234967+0000","last_unstale":"2023-10-18T10:25:08.254219+0000","last_undegraded":"2023-10-18T10:25:08.254219+0000","last_fullsized":"2023-10-18T10:25:08.254219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253370+0000","last_change":"2023-10-18T10:25:04.232541+0000","last_active":"2023-10-18T10:25:08.253370+0000","last_peered":"2023-10-18T10:25:08.253370+0000","last_clean":"2023-10-18T10:25:08.253370+0000","last_became_active":"2023-10-18T10:25:04.231594+0000","last_became_peered":"2023-10-18T10:25:04.231594+0000","last_unstale":"2023-10-18T10:25:08.253370+0000","last_undegraded":"2023-10-18T10:25:08.253370+0000","last_fullsized":"2023-10-18T10:25:08.253370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254018+0000","last_change":"2023-10-18T10:25:04.239452+0000","last_active":"2023-10-18T10:25:08.254018+0000","last_peered":"2023-10-18T10:25:08.254018+0000","last_clean":"2023-10-18T10:25:08.254018+0000","last_became_active":"2023-10-18T10:25:04.239175+0000","last_became_peered":"2023-10-18T10:25:04.239175+0000","last_unstale":"2023-10-18T10:25:08.254018+0000","last_undegraded":"2023-10-18T10:25:08.254018+0000","last_fullsized":"2023-10-18T10:25:08.254018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:42.041557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253482+0000","last_change":"2023-10-18T10:25:04.225873+0000","last_active":"2023-10-18T10:25:08.253482+0000","last_peered":"2023-10-18T10:25:08.253482+0000","last_clean":"2023-10-18T10:25:08.253482+0000","last_became_active":"2023-10-18T10:25:04.225624+0000","last_became_peered":"2023-10-18T10:25:04.225624+0000","last_unstale":"2023-10-18T10:25:08.253482+0000","last_undegraded":"2023-10-18T10:25:08.253482+0000","last_fullsized":"2023-10-18T10:25:08.253482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256440+0000","last_change":"2023-10-18T10:25:04.226107+0000","last_active":"2023-10-18T10:25:08.256440+0000","last_peered":"2023-10-18T10:25:08.256440+0000","last_clean":"2023-10-18T10:25:08.256440+0000","last_became_active":"2023-10-18T10:25:04.225823+0000","last_became_peered":"2023-10-18T10:25:04.225823+0000","last_unstale":"2023-10-18T10:25:08.256440+0000","last_undegraded":"2023-10-18T10:25:08.256440+0000","last_fullsized":"2023-10-18T10:25:08.256440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:35.440931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263514+0000","last_change":"2023-10-18T10:25:04.237730+0000","last_active":"2023-10-18T10:25:08.263514+0000","last_peered":"2023-10-18T10:25:08.263514+0000","last_clean":"2023-10-18T10:25:08.263514+0000","last_became_active":"2023-10-18T10:25:04.237436+0000","last_became_peered":"2023-10-18T10:25:04.237436+0000","last_unstale":"2023-10-18T10:25:08.263514+0000","last_undegraded":"2023-10-18T10:25:08.263514+0000","last_fullsized":"2023-10-18T10:25:08.263514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264090+0000","last_change":"2023-10-18T10:25:04.242105+0000","last_active":"2023-10-18T10:25:08.264090+0000","last_peered":"2023-10-18T10:25:08.264090+0000","last_clean":"2023-10-18T10:25:08.264090+0000","last_became_active":"2023-10-18T10:25:04.241762+0000","last_became_peered":"2023-10-18T10:25:04.241762+0000","last_unstale":"2023-10-18T10:25:08.264090+0000","last_undegraded":"2023-10-18T10:25:08.264090+0000","last_fullsized":"2023-10-18T10:25:08.264090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371062+0000","last_change":"2023-10-18T10:25:04.239055+0000","last_active":"2023-10-18T10:25:08.371062+0000","last_peered":"2023-10-18T10:25:08.371062+0000","last_clean":"2023-10-18T10:25:08.371062+0000","last_became_active":"2023-10-18T10:25:04.238566+0000","last_became_peered":"2023-10-18T10:25:04.238566+0000","last_unstale":"2023-10-18T10:25:08.371062+0000","last_undegraded":"2023-10-18T10:25:08.371062+0000","last_fullsized":"2023-10-18T10:25:08.371062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:43.022445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254270+0000","last_change":"2023-10-18T10:25:04.238819+0000","last_active":"2023-10-18T10:25:08.254270+0000","last_peered":"2023-10-18T10:25:08.254270+0000","last_clean":"2023-10-18T10:25:08.254270+0000","last_became_active":"2023-10-18T10:25:04.238546+0000","last_became_peered":"2023-10-18T10:25:04.238546+0000","last_unstale":"2023-10-18T10:25:08.254270+0000","last_undegraded":"2023-10-18T10:25:08.254270+0000","last_fullsized":"2023-10-18T10:25:08.254270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371571+0000","last_change":"2023-10-18T10:25:04.234955+0000","last_active":"2023-10-18T10:25:08.371571+0000","last_peered":"2023-10-18T10:25:08.371571+0000","last_clean":"2023-10-18T10:25:08.371571+0000","last_became_active":"2023-10-18T10:25:04.234687+0000","last_became_peered":"2023-10-18T10:25:04.234687+0000","last_unstale":"2023-10-18T10:25:08.371571+0000","last_undegraded":"2023-10-18T10:25:08.371571+0000","last_fullsized":"2023-10-18T10:25:08.371571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254664+0000","last_change":"2023-10-18T10:25:04.235078+0000","last_active":"2023-10-18T10:25:08.254664+0000","last_peered":"2023-10-18T10:25:08.254664+0000","last_clean":"2023-10-18T10:25:08.254664+0000","last_became_active":"2023-10-18T10:25:04.234817+0000","last_became_peered":"2023-10-18T10:25:04.234817+0000","last_unstale":"2023-10-18T10:25:08.254664+0000","last_undegraded":"2023-10-18T10:25:08.254664+0000","last_fullsized":"2023-10-18T10:25:08.254664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370554+0000","last_change":"2023-10-18T10:25:04.228077+0000","last_active":"2023-10-18T10:25:08.370554+0000","last_peered":"2023-10-18T10:25:08.370554+0000","last_clean":"2023-10-18T10:25:08.370554+0000","last_became_active":"2023-10-18T10:25:04.227740+0000","last_became_peered":"2023-10-18T10:25:04.227740+0000","last_unstale":"2023-10-18T10:25:08.370554+0000","last_undegraded":"2023-10-18T10:25:08.370554+0000","last_fullsized":"2023-10-18T10:25:08.370554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:26.705420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.258398+0000","last_change":"2023-10-18T10:25:04.240451+0000","last_active":"2023-10-18T10:25:08.258398+0000","last_peered":"2023-10-18T10:25:08.258398+0000","last_clean":"2023-10-18T10:25:08.258398+0000","last_became_active":"2023-10-18T10:25:04.240190+0000","last_became_peered":"2023-10-18T10:25:04.240190+0000","last_unstale":"2023-10-18T10:25:08.258398+0000","last_undegraded":"2023-10-18T10:25:08.258398+0000","last_fullsized":"2023-10-18T10:25:08.258398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:01.626703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254434+0000","last_change":"2023-10-18T10:25:04.233621+0000","last_active":"2023-10-18T10:25:08.254434+0000","last_peered":"2023-10-18T10:25:08.254434+0000","last_clean":"2023-10-18T10:25:08.254434+0000","last_became_active":"2023-10-18T10:25:04.233376+0000","last_became_peered":"2023-10-18T10:25:04.233376+0000","last_unstale":"2023-10-18T10:25:08.254434+0000","last_undegraded":"2023-10-18T10:25:08.254434+0000","last_fullsized":"2023-10-18T10:25:08.254434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:34.587493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255094+0000","last_change":"2023-10-18T10:25:04.234775+0000","last_active":"2023-10-18T10:25:08.255094+0000","last_peered":"2023-10-18T10:25:08.255094+0000","last_clean":"2023-10-18T10:25:08.255094+0000","last_became_active":"2023-10-18T10:25:04.234527+0000","last_became_peered":"2023-10-18T10:25:04.234527+0000","last_unstale":"2023-10-18T10:25:08.255094+0000","last_undegraded":"2023-10-18T10:25:08.255094+0000","last_fullsized":"2023-10-18T10:25:08.255094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370952+0000","last_change":"2023-10-18T10:25:04.236702+0000","last_active":"2023-10-18T10:25:08.370952+0000","last_peered":"2023-10-18T10:25:08.370952+0000","last_clean":"2023-10-18T10:25:08.370952+0000","last_became_active":"2023-10-18T10:25:04.236396+0000","last_became_peered":"2023-10-18T10:25:04.236396+0000","last_unstale":"2023-10-18T10:25:08.370952+0000","last_undegraded":"2023-10-18T10:25:08.370952+0000","last_fullsized":"2023-10-18T10:25:08.370952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:01.626703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:25:21.233 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:25:21.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.367+0000 7f2e1641a640 1 -- 172.21.15.125:0/1045523244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e100c96b0 msgr2=0x7f2e100c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.367+0000 7f2e1641a640 1 --2- 172.21.15.125:0/1045523244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e100c96b0 0x7f2e100c9b30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009900 tx=0x7f2e0402be00 comp rx=0 tx=0).stop 2023-10-18T10:25:21.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.367+0000 7f2e1641a640 1 -- 172.21.15.125:0/1045523244 shutdown_connections 2023-10-18T10:25:21.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.367+0000 7f2e1641a640 1 --2- 172.21.15.125:0/1045523244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e100c96b0 0x7f2e100c9b30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.367+0000 7f2e1641a640 1 --2- 172.21.15.125:0/1045523244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e1006af00 0x7f2e1006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.367+0000 7f2e1641a640 1 -- 172.21.15.125:0/1045523244 >> 172.21.15.125:0/1045523244 conn(0x7f2e10069930 msgr2=0x7f2e10069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:21.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.367+0000 7f2e1641a640 1 -- 172.21.15.125:0/1045523244 shutdown_connections 2023-10-18T10:25:21.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.367+0000 7f2e1641a640 1 -- 172.21.15.125:0/1045523244 wait complete. 2023-10-18T10:25:21.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.368+0000 7f2e1641a640 1 Processor -- start 2023-10-18T10:25:21.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.368+0000 7f2e1641a640 1 -- start start 2023-10-18T10:25:21.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e1641a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e1006af00 0x7f2e101372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e1641a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e100ab500 0x7f2e1015c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e1641a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100c96b0 0x7f2e1015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e1641a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e100b0300 con 0x7f2e1006af00 2023-10-18T10:25:21.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e1641a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e10137820 con 0x7f2e100ab500 2023-10-18T10:25:21.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e1641a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e100af7f0 con 0x7f2e100c96b0 2023-10-18T10:25:21.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e1006af00 0x7f2e101372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e1006af00 0x7f2e101372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49544/0 (socket says 172.21.15.125:49544) 2023-10-18T10:25:21.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 -- 172.21.15.125:0/764029768 learned_addr learned my addr 172.21.15.125:0/764029768 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:21.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e149b7640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100c96b0 0x7f2e1015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0f7fe640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e100ab500 0x7f2e1015c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 -- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100c96b0 msgr2=0x7f2e1015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100c96b0 0x7f2e1015cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 -- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e100ab500 msgr2=0x7f2e1015c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e100ab500 0x7f2e1015c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 -- 172.21.15.125:0/764029768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e04009560 con 0x7f2e1006af00 2023-10-18T10:25:21.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0ffff640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e1006af00 0x7f2e101372e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000d770 tx=0x7f2e0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:21.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0cff9640 1 -- 172.21.15.125:0/764029768 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00009c30 con 0x7f2e1006af00 2023-10-18T10:25:21.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0cff9640 1 -- 172.21.15.125:0/764029768 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2e0000de80 con 0x7f2e1006af00 2023-10-18T10:25:21.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e100afad0 con 0x7f2e1006af00 2023-10-18T10:25:21.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e0cff9640 1 -- 172.21.15.125:0/764029768 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00005280 con 0x7f2e1006af00 2023-10-18T10:25:21.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.369+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e10158470 con 0x7f2e1006af00 2023-10-18T10:25:21.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.370+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e10067930 con 0x7f2e1006af00 2023-10-18T10:25:21.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.371+0000 7f2e0cff9640 1 -- 172.21.15.125:0/764029768 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2e000053e0 con 0x7f2e1006af00 2023-10-18T10:25:21.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.371+0000 7f2e0cff9640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2dfc06ae40 0x7f2dfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.371+0000 7f2e0cff9640 1 -- 172.21.15.125:0/764029768 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2e00089ab0 con 0x7f2e1006af00 2023-10-18T10:25:21.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.371+0000 7f2e0f7fe640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2dfc06ae40 0x7f2dfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.372+0000 7f2e0f7fe640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2dfc06ae40 0x7f2dfc06d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2e04035000 tx=0x7f2e04004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:21.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.375+0000 7f2e0cff9640 1 -- 172.21.15.125:0/764029768 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e00054920 con 0x7f2e1006af00 2023-10-18T10:25:21.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.538+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e10159c50 con 0x7f2dfc06ae40 2023-10-18T10:25:21.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.547+0000 7f2e0cff9640 1 -- 172.21.15.125:0/764029768 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345262 (secure 0 0 0) 0x7f2e040042c0 con 0x7f2dfc06ae40 2023-10-18T10:25:21.552 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:21.554 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:25:21.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2dfc06ae40 msgr2=0x7f2dfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2dfc06ae40 0x7f2dfc06d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2e04035000 tx=0x7f2e04004730 comp rx=0 tx=0).stop 2023-10-18T10:25:21.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e1006af00 msgr2=0x7f2e101372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e1006af00 0x7f2e101372e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000d770 tx=0x7f2e0000dc30 comp rx=0 tx=0).stop 2023-10-18T10:25:21.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 shutdown_connections 2023-10-18T10:25:21.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2dfc06ae40 0x7f2dfc06d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e100ab500 0x7f2e1015c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100c96b0 0x7f2e1015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 --2- 172.21.15.125:0/764029768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e1006af00 0x7f2e101372e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 >> 172.21.15.125:0/764029768 conn(0x7f2e10069930 msgr2=0x7f2e1015bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:21.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 shutdown_connections 2023-10-18T10:25:21.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.550+0000 7f2e1641a640 1 -- 172.21.15.125:0/764029768 wait complete. 2023-10-18T10:25:21.563 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-18T10:25:20.570262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219484,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755236,"statfs":{"total":773094113280,"available":772869361664,"internally_reserved":0,"allocated":3383296,"data_stored":2025340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.308924"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254754+0000","last_change":"2023-10-18T10:25:03.211565+0000","last_active":"2023-10-18T10:25:08.254754+0000","last_peered":"2023-10-18T10:25:08.254754+0000","last_clean":"2023-10-18T10:25:08.254754+0000","last_became_active":"2023-10-18T10:25:03.211312+0000","last_became_peered":"2023-10-18T10:25:03.211312+0000","last_unstale":"2023-10-18T10:25:08.254754+0000","last_undegraded":"2023-10-18T10:25:08.254754+0000","last_fullsized":"2023-10-18T10:25:08.254754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254543+0000","last_change":"2023-10-18T10:25:04.232540+0000","last_active":"2023-10-18T10:25:08.254543+0000","last_peered":"2023-10-18T10:25:08.254543+0000","last_clean":"2023-10-18T10:25:08.254543+0000","last_became_active":"2023-10-18T10:25:04.232270+0000","last_became_peered":"2023-10-18T10:25:04.232270+0000","last_unstale":"2023-10-18T10:25:08.254543+0000","last_undegraded":"2023-10-18T10:25:08.254543+0000","last_fullsized":"2023-10-18T10:25:08.254543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920838+0000","last_change":"2023-10-18T10:25:08.375121+0000","last_active":"2023-10-18T10:25:09.920838+0000","last_peered":"2023-10-18T10:25:09.920838+0000","last_clean":"2023-10-18T10:25:09.920838+0000","last_became_active":"2023-10-18T10:25:08.374754+0000","last_became_peered":"2023-10-18T10:25:08.374754+0000","last_unstale":"2023-10-18T10:25:09.920838+0000","last_undegraded":"2023-10-18T10:25:09.920838+0000","last_fullsized":"2023-10-18T10:25:09.920838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:26.275317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254691+0000","last_change":"2023-10-18T10:25:03.209283+0000","last_active":"2023-10-18T10:25:08.254691+0000","last_peered":"2023-10-18T10:25:08.254691+0000","last_clean":"2023-10-18T10:25:08.254691+0000","last_became_active":"2023-10-18T10:25:03.208718+0000","last_became_peered":"2023-10-18T10:25:03.208718+0000","last_unstale":"2023-10-18T10:25:08.254691+0000","last_undegraded":"2023-10-18T10:25:08.254691+0000","last_fullsized":"2023-10-18T10:25:08.254691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:43.110925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254524+0000","last_change":"2023-10-18T10:25:04.239316+0000","last_active":"2023-10-18T10:25:08.254524+0000","last_peered":"2023-10-18T10:25:08.254524+0000","last_clean":"2023-10-18T10:25:08.254524+0000","last_became_active":"2023-10-18T10:25:04.235952+0000","last_became_peered":"2023-10-18T10:25:04.235952+0000","last_unstale":"2023-10-18T10:25:08.254524+0000","last_undegraded":"2023-10-18T10:25:08.254524+0000","last_fullsized":"2023-10-18T10:25:08.254524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:09.100769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263035+0000","last_change":"2023-10-18T10:25:08.267055+0000","last_active":"2023-10-18T10:25:09.263035+0000","last_peered":"2023-10-18T10:25:09.263035+0000","last_clean":"2023-10-18T10:25:09.263035+0000","last_became_active":"2023-10-18T10:25:08.266533+0000","last_became_peered":"2023-10-18T10:25:08.266533+0000","last_unstale":"2023-10-18T10:25:09.263035+0000","last_undegraded":"2023-10-18T10:25:09.263035+0000","last_fullsized":"2023-10-18T10:25:09.263035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:31.219679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370926+0000","last_change":"2023-10-18T10:25:03.212550+0000","last_active":"2023-10-18T10:25:08.370926+0000","last_peered":"2023-10-18T10:25:08.370926+0000","last_clean":"2023-10-18T10:25:08.370926+0000","last_became_active":"2023-10-18T10:25:03.211719+0000","last_became_peered":"2023-10-18T10:25:03.211719+0000","last_unstale":"2023-10-18T10:25:08.370926+0000","last_undegraded":"2023-10-18T10:25:08.370926+0000","last_fullsized":"2023-10-18T10:25:08.370926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257891+0000","last_change":"2023-10-18T10:25:04.235362+0000","last_active":"2023-10-18T10:25:08.257891+0000","last_peered":"2023-10-18T10:25:08.257891+0000","last_clean":"2023-10-18T10:25:08.257891+0000","last_became_active":"2023-10-18T10:25:04.235036+0000","last_became_peered":"2023-10-18T10:25:04.235036+0000","last_unstale":"2023-10-18T10:25:08.257891+0000","last_undegraded":"2023-10-18T10:25:08.257891+0000","last_fullsized":"2023-10-18T10:25:08.257891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:26.705420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263812+0000","last_change":"2023-10-18T10:25:08.270146+0000","last_active":"2023-10-18T10:25:09.263812+0000","last_peered":"2023-10-18T10:25:09.263812+0000","last_clean":"2023-10-18T10:25:09.263812+0000","last_became_active":"2023-10-18T10:25:08.269580+0000","last_became_peered":"2023-10-18T10:25:08.269580+0000","last_unstale":"2023-10-18T10:25:09.263812+0000","last_undegraded":"2023-10-18T10:25:09.263812+0000","last_fullsized":"2023-10-18T10:25:09.263812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370610+0000","last_change":"2023-10-18T10:25:03.216100+0000","last_active":"2023-10-18T10:25:08.370610+0000","last_peered":"2023-10-18T10:25:08.370610+0000","last_clean":"2023-10-18T10:25:08.370610+0000","last_became_active":"2023-10-18T10:25:03.215819+0000","last_became_peered":"2023-10-18T10:25:03.215819+0000","last_unstale":"2023-10-18T10:25:08.370610+0000","last_undegraded":"2023-10-18T10:25:08.370610+0000","last_fullsized":"2023-10-18T10:25:08.370610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:43.110925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256236+0000","last_change":"2023-10-18T10:25:04.236788+0000","last_active":"2023-10-18T10:25:08.256236+0000","last_peered":"2023-10-18T10:25:08.256236+0000","last_clean":"2023-10-18T10:25:08.256236+0000","last_became_active":"2023-10-18T10:25:04.236291+0000","last_became_peered":"2023-10-18T10:25:04.236291+0000","last_unstale":"2023-10-18T10:25:08.256236+0000","last_undegraded":"2023-10-18T10:25:08.256236+0000","last_fullsized":"2023-10-18T10:25:08.256236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:39.174524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263153+0000","last_change":"2023-10-18T10:25:08.265913+0000","last_active":"2023-10-18T10:25:09.263153+0000","last_peered":"2023-10-18T10:25:09.263153+0000","last_clean":"2023-10-18T10:25:09.263153+0000","last_became_active":"2023-10-18T10:25:08.264755+0000","last_became_peered":"2023-10-18T10:25:08.264755+0000","last_unstale":"2023-10-18T10:25:09.263153+0000","last_undegraded":"2023-10-18T10:25:09.263153+0000","last_fullsized":"2023-10-18T10:25:09.263153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:22.912468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265054+0000","last_change":"2023-10-18T10:25:04.230696+0000","last_active":"2023-10-18T10:25:08.265054+0000","last_peered":"2023-10-18T10:25:08.265054+0000","last_clean":"2023-10-18T10:25:08.265054+0000","last_became_active":"2023-10-18T10:25:04.230402+0000","last_became_peered":"2023-10-18T10:25:04.230402+0000","last_unstale":"2023-10-18T10:25:08.265054+0000","last_undegraded":"2023-10-18T10:25:08.265054+0000","last_fullsized":"2023-10-18T10:25:08.265054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:34.587493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370834+0000","last_change":"2023-10-18T10:25:03.210831+0000","last_active":"2023-10-18T10:25:08.370834+0000","last_peered":"2023-10-18T10:25:08.370834+0000","last_clean":"2023-10-18T10:25:08.370834+0000","last_became_active":"2023-10-18T10:25:03.210586+0000","last_became_peered":"2023-10-18T10:25:03.210586+0000","last_unstale":"2023-10-18T10:25:08.370834+0000","last_undegraded":"2023-10-18T10:25:08.370834+0000","last_fullsized":"2023-10-18T10:25:08.370834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263925+0000","last_change":"2023-10-18T10:25:08.270065+0000","last_active":"2023-10-18T10:25:09.263925+0000","last_peered":"2023-10-18T10:25:09.263925+0000","last_clean":"2023-10-18T10:25:09.263925+0000","last_became_active":"2023-10-18T10:25:08.269544+0000","last_became_peered":"2023-10-18T10:25:08.269544+0000","last_unstale":"2023-10-18T10:25:09.263925+0000","last_undegraded":"2023-10-18T10:25:09.263925+0000","last_fullsized":"2023-10-18T10:25:09.263925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253560+0000","last_change":"2023-10-18T10:25:04.237642+0000","last_active":"2023-10-18T10:25:08.253560+0000","last_peered":"2023-10-18T10:25:08.253560+0000","last_clean":"2023-10-18T10:25:08.253560+0000","last_became_active":"2023-10-18T10:25:04.237321+0000","last_became_peered":"2023-10-18T10:25:04.237321+0000","last_unstale":"2023-10-18T10:25:08.253560+0000","last_undegraded":"2023-10-18T10:25:08.253560+0000","last_fullsized":"2023-10-18T10:25:08.253560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254793+0000","last_change":"2023-10-18T10:25:03.207491+0000","last_active":"2023-10-18T10:25:08.254793+0000","last_peered":"2023-10-18T10:25:08.254793+0000","last_clean":"2023-10-18T10:25:08.254793+0000","last_became_active":"2023-10-18T10:25:03.207227+0000","last_became_peered":"2023-10-18T10:25:03.207227+0000","last_unstale":"2023-10-18T10:25:08.254793+0000","last_undegraded":"2023-10-18T10:25:08.254793+0000","last_fullsized":"2023-10-18T10:25:08.254793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263777+0000","last_change":"2023-10-18T10:25:08.270542+0000","last_active":"2023-10-18T10:25:09.263777+0000","last_peered":"2023-10-18T10:25:09.263777+0000","last_clean":"2023-10-18T10:25:09.263777+0000","last_became_active":"2023-10-18T10:25:08.270153+0000","last_became_peered":"2023-10-18T10:25:08.270153+0000","last_unstale":"2023-10-18T10:25:09.263777+0000","last_undegraded":"2023-10-18T10:25:09.263777+0000","last_fullsized":"2023-10-18T10:25:09.263777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254454+0000","last_change":"2023-10-18T10:25:04.230256+0000","last_active":"2023-10-18T10:25:08.254454+0000","last_peered":"2023-10-18T10:25:08.254454+0000","last_clean":"2023-10-18T10:25:08.254454+0000","last_became_active":"2023-10-18T10:25:04.230007+0000","last_became_peered":"2023-10-18T10:25:04.230007+0000","last_unstale":"2023-10-18T10:25:08.254454+0000","last_undegraded":"2023-10-18T10:25:08.254454+0000","last_fullsized":"2023-10-18T10:25:08.254454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253450+0000","last_change":"2023-10-18T10:25:03.203658+0000","last_active":"2023-10-18T10:25:08.253450+0000","last_peered":"2023-10-18T10:25:08.253450+0000","last_clean":"2023-10-18T10:25:08.253450+0000","last_became_active":"2023-10-18T10:25:03.203419+0000","last_became_peered":"2023-10-18T10:25:03.203419+0000","last_unstale":"2023-10-18T10:25:08.253450+0000","last_undegraded":"2023-10-18T10:25:08.253450+0000","last_fullsized":"2023-10-18T10:25:08.253450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262990+0000","last_change":"2023-10-18T10:25:08.373967+0000","last_active":"2023-10-18T10:25:09.262990+0000","last_peered":"2023-10-18T10:25:09.262990+0000","last_clean":"2023-10-18T10:25:09.262990+0000","last_became_active":"2023-10-18T10:25:08.373624+0000","last_became_peered":"2023-10-18T10:25:08.373624+0000","last_unstale":"2023-10-18T10:25:09.262990+0000","last_undegraded":"2023-10-18T10:25:09.262990+0000","last_fullsized":"2023-10-18T10:25:09.262990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:47.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257377+0000","last_change":"2023-10-18T10:25:04.227308+0000","last_active":"2023-10-18T10:25:08.257377+0000","last_peered":"2023-10-18T10:25:08.257377+0000","last_clean":"2023-10-18T10:25:08.257377+0000","last_became_active":"2023-10-18T10:25:04.227010+0000","last_became_peered":"2023-10-18T10:25:04.227010+0000","last_unstale":"2023-10-18T10:25:08.257377+0000","last_undegraded":"2023-10-18T10:25:08.257377+0000","last_fullsized":"2023-10-18T10:25:08.257377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:03.438820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369608+0000","last_change":"2023-10-18T10:25:03.210132+0000","last_active":"2023-10-18T10:25:08.369608+0000","last_peered":"2023-10-18T10:25:08.369608+0000","last_clean":"2023-10-18T10:25:08.369608+0000","last_became_active":"2023-10-18T10:25:03.209850+0000","last_became_peered":"2023-10-18T10:25:03.209850+0000","last_unstale":"2023-10-18T10:25:08.369608+0000","last_undegraded":"2023-10-18T10:25:08.369608+0000","last_fullsized":"2023-10-18T10:25:08.369608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263730+0000","last_change":"2023-10-18T10:25:08.270370+0000","last_active":"2023-10-18T10:25:09.263730+0000","last_peered":"2023-10-18T10:25:09.263730+0000","last_clean":"2023-10-18T10:25:09.263730+0000","last_became_active":"2023-10-18T10:25:08.269625+0000","last_became_peered":"2023-10-18T10:25:08.269625+0000","last_unstale":"2023-10-18T10:25:09.263730+0000","last_undegraded":"2023-10-18T10:25:09.263730+0000","last_fullsized":"2023-10-18T10:25:09.263730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:45.178929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257699+0000","last_change":"2023-10-18T10:25:04.239309+0000","last_active":"2023-10-18T10:25:08.257699+0000","last_peered":"2023-10-18T10:25:08.257699+0000","last_clean":"2023-10-18T10:25:08.257699+0000","last_became_active":"2023-10-18T10:25:04.236322+0000","last_became_peered":"2023-10-18T10:25:04.236322+0000","last_unstale":"2023-10-18T10:25:08.257699+0000","last_undegraded":"2023-10-18T10:25:08.257699+0000","last_fullsized":"2023-10-18T10:25:08.257699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264329+0000","last_change":"2023-10-18T10:25:03.211765+0000","last_active":"2023-10-18T10:25:08.264329+0000","last_peered":"2023-10-18T10:25:08.264329+0000","last_clean":"2023-10-18T10:25:08.264329+0000","last_became_active":"2023-10-18T10:25:03.211504+0000","last_became_peered":"2023-10-18T10:25:03.211504+0000","last_unstale":"2023-10-18T10:25:08.264329+0000","last_undegraded":"2023-10-18T10:25:08.264329+0000","last_fullsized":"2023-10-18T10:25:08.264329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:41.039503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263210+0000","last_change":"2023-10-18T10:25:08.374129+0000","last_active":"2023-10-18T10:25:09.263210+0000","last_peered":"2023-10-18T10:25:09.263210+0000","last_clean":"2023-10-18T10:25:09.263210+0000","last_became_active":"2023-10-18T10:25:08.373675+0000","last_became_peered":"2023-10-18T10:25:08.373675+0000","last_unstale":"2023-10-18T10:25:09.263210+0000","last_undegraded":"2023-10-18T10:25:09.263210+0000","last_fullsized":"2023-10-18T10:25:09.263210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:12.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255528+0000","last_change":"2023-10-18T10:25:04.232333+0000","last_active":"2023-10-18T10:25:08.255528+0000","last_peered":"2023-10-18T10:25:08.255528+0000","last_clean":"2023-10-18T10:25:08.255528+0000","last_became_active":"2023-10-18T10:25:04.232075+0000","last_became_peered":"2023-10-18T10:25:04.232075+0000","last_unstale":"2023-10-18T10:25:08.255528+0000","last_undegraded":"2023-10-18T10:25:08.255528+0000","last_fullsized":"2023-10-18T10:25:08.255528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:39.174524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255192+0000","last_change":"2023-10-18T10:25:03.209779+0000","last_active":"2023-10-18T10:25:08.255192+0000","last_peered":"2023-10-18T10:25:08.255192+0000","last_clean":"2023-10-18T10:25:08.255192+0000","last_became_active":"2023-10-18T10:25:03.209508+0000","last_became_peered":"2023-10-18T10:25:03.209508+0000","last_unstale":"2023-10-18T10:25:08.255192+0000","last_undegraded":"2023-10-18T10:25:08.255192+0000","last_fullsized":"2023-10-18T10:25:08.255192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:33.585439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920818+0000","last_change":"2023-10-18T10:25:08.372735+0000","last_active":"2023-10-18T10:25:09.920818+0000","last_peered":"2023-10-18T10:25:09.920818+0000","last_clean":"2023-10-18T10:25:09.920818+0000","last_became_active":"2023-10-18T10:25:08.372386+0000","last_became_peered":"2023-10-18T10:25:08.372386+0000","last_unstale":"2023-10-18T10:25:09.920818+0000","last_undegraded":"2023-10-18T10:25:09.920818+0000","last_fullsized":"2023-10-18T10:25:09.920818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:53.362022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255868+0000","last_change":"2023-10-18T10:25:04.240297+0000","last_active":"2023-10-18T10:25:08.255868+0000","last_peered":"2023-10-18T10:25:08.255868+0000","last_clean":"2023-10-18T10:25:08.255868+0000","last_became_active":"2023-10-18T10:25:04.239997+0000","last_became_peered":"2023-10-18T10:25:04.239997+0000","last_unstale":"2023-10-18T10:25:08.255868+0000","last_undegraded":"2023-10-18T10:25:08.255868+0000","last_fullsized":"2023-10-18T10:25:08.255868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371087+0000","last_change":"2023-10-18T10:25:03.213798+0000","last_active":"2023-10-18T10:25:08.371087+0000","last_peered":"2023-10-18T10:25:08.371087+0000","last_clean":"2023-10-18T10:25:08.371087+0000","last_became_active":"2023-10-18T10:25:03.213557+0000","last_became_peered":"2023-10-18T10:25:03.213557+0000","last_unstale":"2023-10-18T10:25:08.371087+0000","last_undegraded":"2023-10-18T10:25:08.371087+0000","last_fullsized":"2023-10-18T10:25:08.371087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:08.120358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307135+0000","last_change":"2023-10-18T10:25:08.370813+0000","last_active":"2023-10-18T10:25:10.307135+0000","last_peered":"2023-10-18T10:25:10.307135+0000","last_clean":"2023-10-18T10:25:10.307135+0000","last_became_active":"2023-10-18T10:25:08.370441+0000","last_became_peered":"2023-10-18T10:25:08.370441+0000","last_unstale":"2023-10-18T10:25:10.307135+0000","last_undegraded":"2023-10-18T10:25:10.307135+0000","last_fullsized":"2023-10-18T10:25:10.307135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256321+0000","last_change":"2023-10-18T10:25:04.242148+0000","last_active":"2023-10-18T10:25:08.256321+0000","last_peered":"2023-10-18T10:25:08.256321+0000","last_clean":"2023-10-18T10:25:08.256321+0000","last_became_active":"2023-10-18T10:25:04.239749+0000","last_became_peered":"2023-10-18T10:25:04.239749+0000","last_unstale":"2023-10-18T10:25:08.256321+0000","last_undegraded":"2023-10-18T10:25:08.256321+0000","last_fullsized":"2023-10-18T10:25:08.256321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255943+0000","last_change":"2023-10-18T10:25:03.205435+0000","last_active":"2023-10-18T10:25:08.255943+0000","last_peered":"2023-10-18T10:25:08.255943+0000","last_clean":"2023-10-18T10:25:08.255943+0000","last_became_active":"2023-10-18T10:25:03.205166+0000","last_became_peered":"2023-10-18T10:25:03.205166+0000","last_unstale":"2023-10-18T10:25:08.255943+0000","last_undegraded":"2023-10-18T10:25:08.255943+0000","last_fullsized":"2023-10-18T10:25:08.255943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:41.039503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262304+0000","last_change":"2023-10-18T10:25:08.267310+0000","last_active":"2023-10-18T10:25:09.262304+0000","last_peered":"2023-10-18T10:25:09.262304+0000","last_clean":"2023-10-18T10:25:09.262304+0000","last_became_active":"2023-10-18T10:25:08.266919+0000","last_became_peered":"2023-10-18T10:25:08.266919+0000","last_unstale":"2023-10-18T10:25:09.262304+0000","last_undegraded":"2023-10-18T10:25:09.262304+0000","last_fullsized":"2023-10-18T10:25:09.262304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257274+0000","last_change":"2023-10-18T10:25:04.234960+0000","last_active":"2023-10-18T10:25:08.257274+0000","last_peered":"2023-10-18T10:25:08.257274+0000","last_clean":"2023-10-18T10:25:08.257274+0000","last_became_active":"2023-10-18T10:25:04.234695+0000","last_became_peered":"2023-10-18T10:25:04.234695+0000","last_unstale":"2023-10-18T10:25:08.257274+0000","last_undegraded":"2023-10-18T10:25:08.257274+0000","last_fullsized":"2023-10-18T10:25:08.257274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:44.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254998+0000","last_change":"2023-10-18T10:25:03.217364+0000","last_active":"2023-10-18T10:25:08.254998+0000","last_peered":"2023-10-18T10:25:08.254998+0000","last_clean":"2023-10-18T10:25:08.254998+0000","last_became_active":"2023-10-18T10:25:03.217105+0000","last_became_peered":"2023-10-18T10:25:03.217105+0000","last_unstale":"2023-10-18T10:25:08.254998+0000","last_undegraded":"2023-10-18T10:25:08.254998+0000","last_fullsized":"2023-10-18T10:25:08.254998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262547+0000","last_change":"2023-10-18T10:25:08.268370+0000","last_active":"2023-10-18T10:25:09.262547+0000","last_peered":"2023-10-18T10:25:09.262547+0000","last_clean":"2023-10-18T10:25:09.262547+0000","last_became_active":"2023-10-18T10:25:08.267649+0000","last_became_peered":"2023-10-18T10:25:08.267649+0000","last_unstale":"2023-10-18T10:25:09.262547+0000","last_undegraded":"2023-10-18T10:25:09.262547+0000","last_fullsized":"2023-10-18T10:25:09.262547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253484+0000","last_change":"2023-10-18T10:25:04.229878+0000","last_active":"2023-10-18T10:25:08.253484+0000","last_peered":"2023-10-18T10:25:08.253484+0000","last_clean":"2023-10-18T10:25:08.253484+0000","last_became_active":"2023-10-18T10:25:04.227272+0000","last_became_peered":"2023-10-18T10:25:04.227272+0000","last_unstale":"2023-10-18T10:25:08.253484+0000","last_undegraded":"2023-10-18T10:25:08.253484+0000","last_fullsized":"2023-10-18T10:25:08.253484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250602+0000","last_change":"2023-10-18T10:25:03.209065+0000","last_active":"2023-10-18T10:25:08.250602+0000","last_peered":"2023-10-18T10:25:08.250602+0000","last_clean":"2023-10-18T10:25:08.250602+0000","last_became_active":"2023-10-18T10:25:03.208772+0000","last_became_peered":"2023-10-18T10:25:03.208772+0000","last_unstale":"2023-10-18T10:25:08.250602+0000","last_undegraded":"2023-10-18T10:25:08.250602+0000","last_fullsized":"2023-10-18T10:25:08.250602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:34.438877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262993+0000","last_change":"2023-10-18T10:25:08.266888+0000","last_active":"2023-10-18T10:25:09.262993+0000","last_peered":"2023-10-18T10:25:09.262993+0000","last_clean":"2023-10-18T10:25:09.262993+0000","last_became_active":"2023-10-18T10:25:08.266538+0000","last_became_peered":"2023-10-18T10:25:08.266538+0000","last_unstale":"2023-10-18T10:25:09.262993+0000","last_undegraded":"2023-10-18T10:25:09.262993+0000","last_fullsized":"2023-10-18T10:25:09.262993+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:53.048432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253849+0000","last_change":"2023-10-18T10:25:04.233140+0000","last_active":"2023-10-18T10:25:08.253849+0000","last_peered":"2023-10-18T10:25:08.253849+0000","last_clean":"2023-10-18T10:25:08.253849+0000","last_became_active":"2023-10-18T10:25:04.232753+0000","last_became_peered":"2023-10-18T10:25:04.232753+0000","last_unstale":"2023-10-18T10:25:08.253849+0000","last_undegraded":"2023-10-18T10:25:08.253849+0000","last_fullsized":"2023-10-18T10:25:08.253849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:27.915879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369629+0000","last_change":"2023-10-18T10:25:03.217333+0000","last_active":"2023-10-18T10:25:08.369629+0000","last_peered":"2023-10-18T10:25:08.369629+0000","last_clean":"2023-10-18T10:25:08.369629+0000","last_became_active":"2023-10-18T10:25:03.217060+0000","last_became_peered":"2023-10-18T10:25:03.217060+0000","last_unstale":"2023-10-18T10:25:08.369629+0000","last_undegraded":"2023-10-18T10:25:08.369629+0000","last_fullsized":"2023-10-18T10:25:08.369629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:02.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307375+0000","last_change":"2023-10-18T10:25:08.373111+0000","last_active":"2023-10-18T10:25:10.307375+0000","last_peered":"2023-10-18T10:25:10.307375+0000","last_clean":"2023-10-18T10:25:10.307375+0000","last_became_active":"2023-10-18T10:25:08.372684+0000","last_became_peered":"2023-10-18T10:25:08.372684+0000","last_unstale":"2023-10-18T10:25:10.307375+0000","last_undegraded":"2023-10-18T10:25:10.307375+0000","last_fullsized":"2023-10-18T10:25:10.307375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:53.048432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253864+0000","last_change":"2023-10-18T10:25:04.238187+0000","last_active":"2023-10-18T10:25:08.253864+0000","last_peered":"2023-10-18T10:25:08.253864+0000","last_clean":"2023-10-18T10:25:08.253864+0000","last_became_active":"2023-10-18T10:25:04.237921+0000","last_became_peered":"2023-10-18T10:25:04.237921+0000","last_unstale":"2023-10-18T10:25:08.253864+0000","last_undegraded":"2023-10-18T10:25:08.253864+0000","last_fullsized":"2023-10-18T10:25:08.253864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253911+0000","last_change":"2023-10-18T10:25:03.220362+0000","last_active":"2023-10-18T10:25:08.253911+0000","last_peered":"2023-10-18T10:25:08.253911+0000","last_clean":"2023-10-18T10:25:08.253911+0000","last_became_active":"2023-10-18T10:25:03.220150+0000","last_became_peered":"2023-10-18T10:25:03.220150+0000","last_unstale":"2023-10-18T10:25:08.253911+0000","last_undegraded":"2023-10-18T10:25:08.253911+0000","last_fullsized":"2023-10-18T10:25:08.253911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369864+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:25:08.369864+0000","last_peered":"2023-10-18T10:25:08.369864+0000","last_clean":"2023-10-18T10:25:08.369864+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:25:08.369864+0000","last_undegraded":"2023-10-18T10:25:08.369864+0000","last_fullsized":"2023-10-18T10:25:08.369864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307435+0000","last_change":"2023-10-18T10:25:08.374370+0000","last_active":"2023-10-18T10:25:10.307435+0000","last_peered":"2023-10-18T10:25:10.307435+0000","last_clean":"2023-10-18T10:25:10.307435+0000","last_became_active":"2023-10-18T10:25:08.372029+0000","last_became_peered":"2023-10-18T10:25:08.372029+0000","last_unstale":"2023-10-18T10:25:10.307435+0000","last_undegraded":"2023-10-18T10:25:10.307435+0000","last_fullsized":"2023-10-18T10:25:10.307435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:45.178929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370575+0000","last_change":"2023-10-18T10:25:04.239066+0000","last_active":"2023-10-18T10:25:08.370575+0000","last_peered":"2023-10-18T10:25:08.370575+0000","last_clean":"2023-10-18T10:25:08.370575+0000","last_became_active":"2023-10-18T10:25:04.238531+0000","last_became_peered":"2023-10-18T10:25:04.238531+0000","last_unstale":"2023-10-18T10:25:08.370575+0000","last_undegraded":"2023-10-18T10:25:08.370575+0000","last_fullsized":"2023-10-18T10:25:08.370575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250745+0000","last_change":"2023-10-18T10:25:03.213157+0000","last_active":"2023-10-18T10:25:08.250745+0000","last_peered":"2023-10-18T10:25:08.250745+0000","last_clean":"2023-10-18T10:25:08.250745+0000","last_became_active":"2023-10-18T10:25:03.212900+0000","last_became_peered":"2023-10-18T10:25:03.212900+0000","last_unstale":"2023-10-18T10:25:08.250745+0000","last_undegraded":"2023-10-18T10:25:08.250745+0000","last_fullsized":"2023-10-18T10:25:08.250745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:29:02.497235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307254+0000","last_change":"2023-10-18T10:25:08.370854+0000","last_active":"2023-10-18T10:25:10.307254+0000","last_peered":"2023-10-18T10:25:10.307254+0000","last_clean":"2023-10-18T10:25:10.307254+0000","last_became_active":"2023-10-18T10:25:08.370423+0000","last_became_peered":"2023-10-18T10:25:08.370423+0000","last_unstale":"2023-10-18T10:25:10.307254+0000","last_undegraded":"2023-10-18T10:25:10.307254+0000","last_fullsized":"2023-10-18T10:25:10.307254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:57.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257993+0000","last_change":"2023-10-18T10:25:04.239837+0000","last_active":"2023-10-18T10:25:08.257993+0000","last_peered":"2023-10-18T10:25:08.257993+0000","last_clean":"2023-10-18T10:25:08.257993+0000","last_became_active":"2023-10-18T10:25:04.239576+0000","last_became_peered":"2023-10-18T10:25:04.239576+0000","last_unstale":"2023-10-18T10:25:08.257993+0000","last_undegraded":"2023-10-18T10:25:08.257993+0000","last_fullsized":"2023-10-18T10:25:08.257993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:43.022445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253804+0000","last_change":"2023-10-18T10:25:03.204821+0000","last_active":"2023-10-18T10:25:08.253804+0000","last_peered":"2023-10-18T10:25:08.253804+0000","last_clean":"2023-10-18T10:25:08.253804+0000","last_became_active":"2023-10-18T10:25:03.204545+0000","last_became_peered":"2023-10-18T10:25:03.204545+0000","last_unstale":"2023-10-18T10:25:08.253804+0000","last_undegraded":"2023-10-18T10:25:08.253804+0000","last_fullsized":"2023-10-18T10:25:08.253804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307183+0000","last_change":"2023-10-18T10:25:08.265973+0000","last_active":"2023-10-18T10:25:10.307183+0000","last_peered":"2023-10-18T10:25:10.307183+0000","last_clean":"2023-10-18T10:25:10.307183+0000","last_became_active":"2023-10-18T10:25:08.265607+0000","last_became_peered":"2023-10-18T10:25:08.265607+0000","last_unstale":"2023-10-18T10:25:10.307183+0000","last_undegraded":"2023-10-18T10:25:10.307183+0000","last_fullsized":"2023-10-18T10:25:10.307183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371147+0000","last_change":"2023-10-18T10:25:04.245074+0000","last_active":"2023-10-18T10:25:08.371147+0000","last_peered":"2023-10-18T10:25:08.371147+0000","last_clean":"2023-10-18T10:25:08.371147+0000","last_became_active":"2023-10-18T10:25:04.239593+0000","last_became_peered":"2023-10-18T10:25:04.239593+0000","last_unstale":"2023-10-18T10:25:08.371147+0000","last_undegraded":"2023-10-18T10:25:08.371147+0000","last_fullsized":"2023-10-18T10:25:08.371147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:29.983102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250759+0000","last_change":"2023-10-18T10:25:03.206351+0000","last_active":"2023-10-18T10:25:08.250759+0000","last_peered":"2023-10-18T10:25:08.250759+0000","last_clean":"2023-10-18T10:25:08.250759+0000","last_became_active":"2023-10-18T10:25:03.206082+0000","last_became_peered":"2023-10-18T10:25:03.206082+0000","last_unstale":"2023-10-18T10:25:08.250759+0000","last_undegraded":"2023-10-18T10:25:08.250759+0000","last_fullsized":"2023-10-18T10:25:08.250759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307655+0000","last_change":"2023-10-18T10:25:08.372691+0000","last_active":"2023-10-18T10:25:10.307655+0000","last_peered":"2023-10-18T10:25:10.307655+0000","last_clean":"2023-10-18T10:25:10.307655+0000","last_became_active":"2023-10-18T10:25:08.372252+0000","last_became_peered":"2023-10-18T10:25:08.372252+0000","last_unstale":"2023-10-18T10:25:10.307655+0000","last_undegraded":"2023-10-18T10:25:10.307655+0000","last_fullsized":"2023-10-18T10:25:10.307655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:53.362022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370821+0000","last_change":"2023-10-18T10:25:04.237483+0000","last_active":"2023-10-18T10:25:08.370821+0000","last_peered":"2023-10-18T10:25:08.370821+0000","last_clean":"2023-10-18T10:25:08.370821+0000","last_became_active":"2023-10-18T10:25:04.237196+0000","last_became_peered":"2023-10-18T10:25:04.237196+0000","last_unstale":"2023-10-18T10:25:08.370821+0000","last_undegraded":"2023-10-18T10:25:08.370821+0000","last_fullsized":"2023-10-18T10:25:08.370821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:41.123375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254019+0000","last_change":"2023-10-18T10:25:03.202658+0000","last_active":"2023-10-18T10:25:08.254019+0000","last_peered":"2023-10-18T10:25:08.254019+0000","last_clean":"2023-10-18T10:25:08.254019+0000","last_became_active":"2023-10-18T10:25:03.202313+0000","last_became_peered":"2023-10-18T10:25:03.202313+0000","last_unstale":"2023-10-18T10:25:08.254019+0000","last_undegraded":"2023-10-18T10:25:08.254019+0000","last_fullsized":"2023-10-18T10:25:08.254019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:18:46.417045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262979+0000","last_change":"2023-10-18T10:25:08.273431+0000","last_active":"2023-10-18T10:25:09.262979+0000","last_peered":"2023-10-18T10:25:09.262979+0000","last_clean":"2023-10-18T10:25:09.262979+0000","last_became_active":"2023-10-18T10:25:08.273002+0000","last_became_peered":"2023-10-18T10:25:08.273002+0000","last_unstale":"2023-10-18T10:25:09.262979+0000","last_undegraded":"2023-10-18T10:25:09.262979+0000","last_fullsized":"2023-10-18T10:25:09.262979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:43.230078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371654+0000","last_change":"2023-10-18T10:25:04.237817+0000","last_active":"2023-10-18T10:25:08.371654+0000","last_peered":"2023-10-18T10:25:08.371654+0000","last_clean":"2023-10-18T10:25:08.371654+0000","last_became_active":"2023-10-18T10:25:04.237459+0000","last_became_peered":"2023-10-18T10:25:04.237459+0000","last_unstale":"2023-10-18T10:25:08.371654+0000","last_undegraded":"2023-10-18T10:25:08.371654+0000","last_fullsized":"2023-10-18T10:25:08.371654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250534+0000","last_change":"2023-10-18T10:25:03.204327+0000","last_active":"2023-10-18T10:25:08.250534+0000","last_peered":"2023-10-18T10:25:08.250534+0000","last_clean":"2023-10-18T10:25:08.250534+0000","last_became_active":"2023-10-18T10:25:03.204079+0000","last_became_peered":"2023-10-18T10:25:03.204079+0000","last_unstale":"2023-10-18T10:25:08.250534+0000","last_undegraded":"2023-10-18T10:25:08.250534+0000","last_fullsized":"2023-10-18T10:25:08.250534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:05.461286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307373+0000","last_change":"2023-10-18T10:25:08.268324+0000","last_active":"2023-10-18T10:25:10.307373+0000","last_peered":"2023-10-18T10:25:10.307373+0000","last_clean":"2023-10-18T10:25:10.307373+0000","last_became_active":"2023-10-18T10:25:08.267959+0000","last_became_peered":"2023-10-18T10:25:08.267959+0000","last_unstale":"2023-10-18T10:25:10.307373+0000","last_undegraded":"2023-10-18T10:25:10.307373+0000","last_fullsized":"2023-10-18T10:25:10.307373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:47.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263268+0000","last_change":"2023-10-18T10:25:04.234809+0000","last_active":"2023-10-18T10:25:08.263268+0000","last_peered":"2023-10-18T10:25:08.263268+0000","last_clean":"2023-10-18T10:25:08.263268+0000","last_became_active":"2023-10-18T10:25:04.234527+0000","last_became_peered":"2023-10-18T10:25:04.234527+0000","last_unstale":"2023-10-18T10:25:08.263268+0000","last_undegraded":"2023-10-18T10:25:08.263268+0000","last_fullsized":"2023-10-18T10:25:08.263268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263204+0000","last_change":"2023-10-18T10:25:03.207281+0000","last_active":"2023-10-18T10:25:08.263204+0000","last_peered":"2023-10-18T10:25:08.263204+0000","last_clean":"2023-10-18T10:25:08.263204+0000","last_became_active":"2023-10-18T10:25:03.206828+0000","last_became_peered":"2023-10-18T10:25:03.206828+0000","last_unstale":"2023-10-18T10:25:08.263204+0000","last_undegraded":"2023-10-18T10:25:08.263204+0000","last_fullsized":"2023-10-18T10:25:08.263204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921617+0000","last_change":"2023-10-18T10:25:08.372503+0000","last_active":"2023-10-18T10:25:09.921617+0000","last_peered":"2023-10-18T10:25:09.921617+0000","last_clean":"2023-10-18T10:25:09.921617+0000","last_became_active":"2023-10-18T10:25:08.371994+0000","last_became_peered":"2023-10-18T10:25:08.371994+0000","last_unstale":"2023-10-18T10:25:09.921617+0000","last_undegraded":"2023-10-18T10:25:09.921617+0000","last_fullsized":"2023-10-18T10:25:09.921617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.553280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370729+0000","last_change":"2023-10-18T10:25:04.237797+0000","last_active":"2023-10-18T10:25:08.370729+0000","last_peered":"2023-10-18T10:25:08.370729+0000","last_clean":"2023-10-18T10:25:08.370729+0000","last_became_active":"2023-10-18T10:25:04.237473+0000","last_became_peered":"2023-10-18T10:25:04.237473+0000","last_unstale":"2023-10-18T10:25:08.370729+0000","last_undegraded":"2023-10-18T10:25:08.370729+0000","last_fullsized":"2023-10-18T10:25:08.370729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:48.992878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254912+0000","last_change":"2023-10-18T10:25:03.213281+0000","last_active":"2023-10-18T10:25:08.254912+0000","last_peered":"2023-10-18T10:25:08.254912+0000","last_clean":"2023-10-18T10:25:08.254912+0000","last_became_active":"2023-10-18T10:25:03.213019+0000","last_became_peered":"2023-10-18T10:25:03.213019+0000","last_unstale":"2023-10-18T10:25:08.254912+0000","last_undegraded":"2023-10-18T10:25:08.254912+0000","last_fullsized":"2023-10-18T10:25:08.254912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:07.702798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921380+0000","last_change":"2023-10-18T10:25:08.370771+0000","last_active":"2023-10-18T10:25:09.921380+0000","last_peered":"2023-10-18T10:25:09.921380+0000","last_clean":"2023-10-18T10:25:09.921380+0000","last_became_active":"2023-10-18T10:25:08.370438+0000","last_became_peered":"2023-10-18T10:25:08.370438+0000","last_unstale":"2023-10-18T10:25:09.921380+0000","last_undegraded":"2023-10-18T10:25:09.921380+0000","last_fullsized":"2023-10-18T10:25:09.921380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:31.794489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254101+0000","last_change":"2023-10-18T10:25:04.232734+0000","last_active":"2023-10-18T10:25:08.254101+0000","last_peered":"2023-10-18T10:25:08.254101+0000","last_clean":"2023-10-18T10:25:08.254101+0000","last_became_active":"2023-10-18T10:25:04.232474+0000","last_became_peered":"2023-10-18T10:25:04.232474+0000","last_unstale":"2023-10-18T10:25:08.254101+0000","last_undegraded":"2023-10-18T10:25:08.254101+0000","last_fullsized":"2023-10-18T10:25:08.254101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257928+0000","last_change":"2023-10-18T10:25:03.202060+0000","last_active":"2023-10-18T10:25:08.257928+0000","last_peered":"2023-10-18T10:25:08.257928+0000","last_clean":"2023-10-18T10:25:08.257928+0000","last_became_active":"2023-10-18T10:25:03.201719+0000","last_became_peered":"2023-10-18T10:25:03.201719+0000","last_unstale":"2023-10-18T10:25:08.257928+0000","last_undegraded":"2023-10-18T10:25:08.257928+0000","last_fullsized":"2023-10-18T10:25:08.257928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.308982+0000","last_change":"2023-10-18T10:25:08.370830+0000","last_active":"2023-10-18T10:25:10.308982+0000","last_peered":"2023-10-18T10:25:10.308982+0000","last_clean":"2023-10-18T10:25:10.308982+0000","last_became_active":"2023-10-18T10:25:08.370387+0000","last_became_peered":"2023-10-18T10:25:08.370387+0000","last_unstale":"2023-10-18T10:25:10.308982+0000","last_undegraded":"2023-10-18T10:25:10.308982+0000","last_fullsized":"2023-10-18T10:25:10.308982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:26.275317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370740+0000","last_change":"2023-10-18T10:25:04.241881+0000","last_active":"2023-10-18T10:25:08.370740+0000","last_peered":"2023-10-18T10:25:08.370740+0000","last_clean":"2023-10-18T10:25:08.370740+0000","last_became_active":"2023-10-18T10:25:04.241615+0000","last_became_peered":"2023-10-18T10:25:04.241615+0000","last_unstale":"2023-10-18T10:25:08.370740+0000","last_undegraded":"2023-10-18T10:25:08.370740+0000","last_fullsized":"2023-10-18T10:25:08.370740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:13.052474+0000","last_change":"2023-10-18T10:25:03.211150+0000","last_active":"2023-10-18T10:25:13.052474+0000","last_peered":"2023-10-18T10:25:13.052474+0000","last_clean":"2023-10-18T10:25:13.052474+0000","last_became_active":"2023-10-18T10:25:03.210879+0000","last_became_peered":"2023-10-18T10:25:03.210879+0000","last_unstale":"2023-10-18T10:25:13.052474+0000","last_undegraded":"2023-10-18T10:25:13.052474+0000","last_fullsized":"2023-10-18T10:25:13.052474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262397+0000","last_change":"2023-10-18T10:25:08.268389+0000","last_active":"2023-10-18T10:25:09.262397+0000","last_peered":"2023-10-18T10:25:09.262397+0000","last_clean":"2023-10-18T10:25:09.262397+0000","last_became_active":"2023-10-18T10:25:08.268025+0000","last_became_peered":"2023-10-18T10:25:08.268025+0000","last_unstale":"2023-10-18T10:25:09.262397+0000","last_undegraded":"2023-10-18T10:25:09.262397+0000","last_fullsized":"2023-10-18T10:25:09.262397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256542+0000","last_change":"2023-10-18T10:25:04.229742+0000","last_active":"2023-10-18T10:25:08.256542+0000","last_peered":"2023-10-18T10:25:08.256542+0000","last_clean":"2023-10-18T10:25:08.256542+0000","last_became_active":"2023-10-18T10:25:04.229281+0000","last_became_peered":"2023-10-18T10:25:04.229281+0000","last_unstale":"2023-10-18T10:25:08.256542+0000","last_undegraded":"2023-10-18T10:25:08.256542+0000","last_fullsized":"2023-10-18T10:25:08.256542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:08.704852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253721+0000","last_change":"2023-10-18T10:25:03.213327+0000","last_active":"2023-10-18T10:25:08.253721+0000","last_peered":"2023-10-18T10:25:08.253721+0000","last_clean":"2023-10-18T10:25:08.253721+0000","last_became_active":"2023-10-18T10:25:03.212993+0000","last_became_peered":"2023-10-18T10:25:03.212993+0000","last_unstale":"2023-10-18T10:25:08.253721+0000","last_undegraded":"2023-10-18T10:25:08.253721+0000","last_fullsized":"2023-10-18T10:25:08.253721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263300+0000","last_change":"2023-10-18T10:25:08.271366+0000","last_active":"2023-10-18T10:25:09.263300+0000","last_peered":"2023-10-18T10:25:09.263300+0000","last_clean":"2023-10-18T10:25:09.263300+0000","last_became_active":"2023-10-18T10:25:08.270581+0000","last_became_peered":"2023-10-18T10:25:08.270581+0000","last_unstale":"2023-10-18T10:25:09.263300+0000","last_undegraded":"2023-10-18T10:25:09.263300+0000","last_fullsized":"2023-10-18T10:25:09.263300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:57.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254604+0000","last_change":"2023-10-18T10:25:04.234658+0000","last_active":"2023-10-18T10:25:08.254604+0000","last_peered":"2023-10-18T10:25:08.254604+0000","last_clean":"2023-10-18T10:25:08.254604+0000","last_became_active":"2023-10-18T10:25:04.234334+0000","last_became_peered":"2023-10-18T10:25:04.234334+0000","last_unstale":"2023-10-18T10:25:08.254604+0000","last_undegraded":"2023-10-18T10:25:08.254604+0000","last_fullsized":"2023-10-18T10:25:08.254604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254170+0000","last_change":"2023-10-18T10:25:03.213955+0000","last_active":"2023-10-18T10:25:08.254170+0000","last_peered":"2023-10-18T10:25:08.254170+0000","last_clean":"2023-10-18T10:25:08.254170+0000","last_became_active":"2023-10-18T10:25:03.213692+0000","last_became_peered":"2023-10-18T10:25:03.213692+0000","last_unstale":"2023-10-18T10:25:08.254170+0000","last_undegraded":"2023-10-18T10:25:08.254170+0000","last_fullsized":"2023-10-18T10:25:08.254170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:21.099526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920943+0000","last_change":"2023-10-18T10:25:08.271334+0000","last_active":"2023-10-18T10:25:09.920943+0000","last_peered":"2023-10-18T10:25:09.920943+0000","last_clean":"2023-10-18T10:25:09.920943+0000","last_became_active":"2023-10-18T10:25:08.270900+0000","last_became_peered":"2023-10-18T10:25:08.270900+0000","last_unstale":"2023-10-18T10:25:09.920943+0000","last_undegraded":"2023-10-18T10:25:09.920943+0000","last_fullsized":"2023-10-18T10:25:09.920943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:53:25.617273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254094+0000","last_change":"2023-10-18T10:25:04.238611+0000","last_active":"2023-10-18T10:25:08.254094+0000","last_peered":"2023-10-18T10:25:08.254094+0000","last_clean":"2023-10-18T10:25:08.254094+0000","last_became_active":"2023-10-18T10:25:04.238304+0000","last_became_peered":"2023-10-18T10:25:04.238304+0000","last_unstale":"2023-10-18T10:25:08.254094+0000","last_undegraded":"2023-10-18T10:25:08.254094+0000","last_fullsized":"2023-10-18T10:25:08.254094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264215+0000","last_change":"2023-10-18T10:25:03.213189+0000","last_active":"2023-10-18T10:25:08.264215+0000","last_peered":"2023-10-18T10:25:08.264215+0000","last_clean":"2023-10-18T10:25:08.264215+0000","last_became_active":"2023-10-18T10:25:03.212915+0000","last_became_peered":"2023-10-18T10:25:03.212915+0000","last_unstale":"2023-10-18T10:25:08.264215+0000","last_undegraded":"2023-10-18T10:25:08.264215+0000","last_fullsized":"2023-10-18T10:25:08.264215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920935+0000","last_change":"2023-10-18T10:25:08.374903+0000","last_active":"2023-10-18T10:25:09.920935+0000","last_peered":"2023-10-18T10:25:09.920935+0000","last_clean":"2023-10-18T10:25:09.920935+0000","last_became_active":"2023-10-18T10:25:08.374525+0000","last_became_peered":"2023-10-18T10:25:08.374525+0000","last_unstale":"2023-10-18T10:25:09.920935+0000","last_undegraded":"2023-10-18T10:25:09.920935+0000","last_fullsized":"2023-10-18T10:25:09.920935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:38.455325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307246+0000","last_change":"2023-10-18T10:25:08.372484+0000","last_active":"2023-10-18T10:25:10.307246+0000","last_peered":"2023-10-18T10:25:10.307246+0000","last_clean":"2023-10-18T10:25:10.307246+0000","last_became_active":"2023-10-18T10:25:08.372132+0000","last_became_peered":"2023-10-18T10:25:08.372132+0000","last_unstale":"2023-10-18T10:25:10.307246+0000","last_undegraded":"2023-10-18T10:25:10.307246+0000","last_fullsized":"2023-10-18T10:25:10.307246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253285+0000","last_change":"2023-10-18T10:25:04.239197+0000","last_active":"2023-10-18T10:25:08.253285+0000","last_peered":"2023-10-18T10:25:08.253285+0000","last_clean":"2023-10-18T10:25:08.253285+0000","last_became_active":"2023-10-18T10:25:04.238917+0000","last_became_peered":"2023-10-18T10:25:04.238917+0000","last_unstale":"2023-10-18T10:25:08.253285+0000","last_undegraded":"2023-10-18T10:25:08.253285+0000","last_fullsized":"2023-10-18T10:25:08.253285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254371+0000","last_change":"2023-10-18T10:25:03.207877+0000","last_active":"2023-10-18T10:25:08.254371+0000","last_peered":"2023-10-18T10:25:08.254371+0000","last_clean":"2023-10-18T10:25:08.254371+0000","last_became_active":"2023-10-18T10:25:03.207621+0000","last_became_peered":"2023-10-18T10:25:03.207621+0000","last_unstale":"2023-10-18T10:25:08.254371+0000","last_undegraded":"2023-10-18T10:25:08.254371+0000","last_fullsized":"2023-10-18T10:25:08.254371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:52.476561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.308246+0000","last_change":"2023-10-18T10:25:08.272814+0000","last_active":"2023-10-18T10:25:10.308246+0000","last_peered":"2023-10-18T10:25:10.308246+0000","last_clean":"2023-10-18T10:25:10.308246+0000","last_became_active":"2023-10-18T10:25:08.272372+0000","last_became_peered":"2023-10-18T10:25:08.272372+0000","last_unstale":"2023-10-18T10:25:10.308246+0000","last_undegraded":"2023-10-18T10:25:10.308246+0000","last_fullsized":"2023-10-18T10:25:10.308246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254375+0000","last_change":"2023-10-18T10:25:04.237028+0000","last_active":"2023-10-18T10:25:08.254375+0000","last_peered":"2023-10-18T10:25:08.254375+0000","last_clean":"2023-10-18T10:25:08.254375+0000","last_became_active":"2023-10-18T10:25:04.236620+0000","last_became_peered":"2023-10-18T10:25:04.236620+0000","last_unstale":"2023-10-18T10:25:08.254375+0000","last_undegraded":"2023-10-18T10:25:08.254375+0000","last_fullsized":"2023-10-18T10:25:08.254375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255920+0000","last_change":"2023-10-18T10:25:03.212737+0000","last_active":"2023-10-18T10:25:08.255920+0000","last_peered":"2023-10-18T10:25:08.255920+0000","last_clean":"2023-10-18T10:25:08.255920+0000","last_became_active":"2023-10-18T10:25:03.212444+0000","last_became_peered":"2023-10-18T10:25:03.212444+0000","last_unstale":"2023-10-18T10:25:08.255920+0000","last_undegraded":"2023-10-18T10:25:08.255920+0000","last_fullsized":"2023-10-18T10:25:08.255920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:53.307830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265148+0000","last_change":"2023-10-18T10:25:03.208047+0000","last_active":"2023-10-18T10:25:08.265148+0000","last_peered":"2023-10-18T10:25:08.265148+0000","last_clean":"2023-10-18T10:25:08.265148+0000","last_became_active":"2023-10-18T10:25:03.207745+0000","last_became_peered":"2023-10-18T10:25:03.207745+0000","last_unstale":"2023-10-18T10:25:08.265148+0000","last_undegraded":"2023-10-18T10:25:08.265148+0000","last_fullsized":"2023-10-18T10:25:08.265148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255246+0000","last_change":"2023-10-18T10:25:04.240334+0000","last_active":"2023-10-18T10:25:08.255246+0000","last_peered":"2023-10-18T10:25:08.255246+0000","last_clean":"2023-10-18T10:25:08.255246+0000","last_became_active":"2023-10-18T10:25:04.239964+0000","last_became_peered":"2023-10-18T10:25:04.239964+0000","last_unstale":"2023-10-18T10:25:08.255246+0000","last_undegraded":"2023-10-18T10:25:08.255246+0000","last_fullsized":"2023-10-18T10:25:08.255246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.294555+0000","last_change":"2023-10-18T10:25:08.264225+0000","last_active":"2023-10-18T10:25:09.294555+0000","last_peered":"2023-10-18T10:25:09.294555+0000","last_clean":"2023-10-18T10:25:09.294555+0000","last_became_active":"2023-10-18T10:25:08.263712+0000","last_became_peered":"2023-10-18T10:25:08.263712+0000","last_unstale":"2023-10-18T10:25:09.294555+0000","last_undegraded":"2023-10-18T10:25:09.294555+0000","last_fullsized":"2023-10-18T10:25:09.294555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:13.156323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253645+0000","last_change":"2023-10-18T10:25:03.211608+0000","last_active":"2023-10-18T10:25:08.253645+0000","last_peered":"2023-10-18T10:25:08.253645+0000","last_clean":"2023-10-18T10:25:08.253645+0000","last_became_active":"2023-10-18T10:25:03.211346+0000","last_became_peered":"2023-10-18T10:25:03.211346+0000","last_unstale":"2023-10-18T10:25:08.253645+0000","last_undegraded":"2023-10-18T10:25:08.253645+0000","last_fullsized":"2023-10-18T10:25:08.253645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255576+0000","last_change":"2023-10-18T10:25:04.242544+0000","last_active":"2023-10-18T10:25:08.255576+0000","last_peered":"2023-10-18T10:25:08.255576+0000","last_clean":"2023-10-18T10:25:08.255576+0000","last_became_active":"2023-10-18T10:25:04.242258+0000","last_became_peered":"2023-10-18T10:25:04.242258+0000","last_unstale":"2023-10-18T10:25:08.255576+0000","last_undegraded":"2023-10-18T10:25:08.255576+0000","last_fullsized":"2023-10-18T10:25:08.255576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:55.422361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921493+0000","last_change":"2023-10-18T10:25:08.273772+0000","last_active":"2023-10-18T10:25:09.921493+0000","last_peered":"2023-10-18T10:25:09.921493+0000","last_clean":"2023-10-18T10:25:09.921493+0000","last_became_active":"2023-10-18T10:25:08.273410+0000","last_became_peered":"2023-10-18T10:25:08.273410+0000","last_unstale":"2023-10-18T10:25:09.921493+0000","last_undegraded":"2023-10-18T10:25:09.921493+0000","last_fullsized":"2023-10-18T10:25:09.921493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:19.382604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263176+0000","last_change":"2023-10-18T10:25:04.235479+0000","last_active":"2023-10-18T10:25:08.263176+0000","last_peered":"2023-10-18T10:25:08.263176+0000","last_clean":"2023-10-18T10:25:08.263176+0000","last_became_active":"2023-10-18T10:25:04.235205+0000","last_became_peered":"2023-10-18T10:25:04.235205+0000","last_unstale":"2023-10-18T10:25:08.263176+0000","last_undegraded":"2023-10-18T10:25:08.263176+0000","last_fullsized":"2023-10-18T10:25:08.263176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250534+0000","last_change":"2023-10-18T10:25:04.236754+0000","last_active":"2023-10-18T10:25:08.250534+0000","last_peered":"2023-10-18T10:25:08.250534+0000","last_clean":"2023-10-18T10:25:08.250534+0000","last_became_active":"2023-10-18T10:25:04.236482+0000","last_became_peered":"2023-10-18T10:25:04.236482+0000","last_unstale":"2023-10-18T10:25:08.250534+0000","last_undegraded":"2023-10-18T10:25:08.250534+0000","last_fullsized":"2023-10-18T10:25:08.250534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:08.704852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254828+0000","last_change":"2023-10-18T10:25:04.239429+0000","last_active":"2023-10-18T10:25:08.254828+0000","last_peered":"2023-10-18T10:25:08.254828+0000","last_clean":"2023-10-18T10:25:08.254828+0000","last_became_active":"2023-10-18T10:25:04.239185+0000","last_became_peered":"2023-10-18T10:25:04.239185+0000","last_unstale":"2023-10-18T10:25:08.254828+0000","last_undegraded":"2023-10-18T10:25:08.254828+0000","last_fullsized":"2023-10-18T10:25:08.254828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:41.123375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255164+0000","last_change":"2023-10-18T10:25:04.235523+0000","last_active":"2023-10-18T10:25:08.255164+0000","last_peered":"2023-10-18T10:25:08.255164+0000","last_clean":"2023-10-18T10:25:08.255164+0000","last_became_active":"2023-10-18T10:25:04.235266+0000","last_became_peered":"2023-10-18T10:25:04.235266+0000","last_unstale":"2023-10-18T10:25:08.255164+0000","last_undegraded":"2023-10-18T10:25:08.255164+0000","last_fullsized":"2023-10-18T10:25:08.255164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254545+0000","last_change":"2023-10-18T10:25:04.225083+0000","last_active":"2023-10-18T10:25:08.254545+0000","last_peered":"2023-10-18T10:25:08.254545+0000","last_clean":"2023-10-18T10:25:08.254545+0000","last_became_active":"2023-10-18T10:25:04.224243+0000","last_became_peered":"2023-10-18T10:25:04.224243+0000","last_unstale":"2023-10-18T10:25:08.254545+0000","last_undegraded":"2023-10-18T10:25:08.254545+0000","last_fullsized":"2023-10-18T10:25:08.254545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253638+0000","last_change":"2023-10-18T10:25:04.239380+0000","last_active":"2023-10-18T10:25:08.253638+0000","last_peered":"2023-10-18T10:25:08.253638+0000","last_clean":"2023-10-18T10:25:08.253638+0000","last_became_active":"2023-10-18T10:25:04.239121+0000","last_became_peered":"2023-10-18T10:25:04.239121+0000","last_unstale":"2023-10-18T10:25:08.253638+0000","last_undegraded":"2023-10-18T10:25:08.253638+0000","last_fullsized":"2023-10-18T10:25:08.253638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265703+0000","last_change":"2023-10-18T10:25:04.231489+0000","last_active":"2023-10-18T10:25:08.265703+0000","last_peered":"2023-10-18T10:25:08.265703+0000","last_clean":"2023-10-18T10:25:08.265703+0000","last_became_active":"2023-10-18T10:25:04.231228+0000","last_became_peered":"2023-10-18T10:25:04.231228+0000","last_unstale":"2023-10-18T10:25:08.265703+0000","last_undegraded":"2023-10-18T10:25:08.265703+0000","last_fullsized":"2023-10-18T10:25:08.265703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255438+0000","last_change":"2023-10-18T10:25:04.226121+0000","last_active":"2023-10-18T10:25:08.255438+0000","last_peered":"2023-10-18T10:25:08.255438+0000","last_clean":"2023-10-18T10:25:08.255438+0000","last_became_active":"2023-10-18T10:25:04.225472+0000","last_became_peered":"2023-10-18T10:25:04.225472+0000","last_unstale":"2023-10-18T10:25:08.255438+0000","last_undegraded":"2023-10-18T10:25:08.255438+0000","last_fullsized":"2023-10-18T10:25:08.255438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254296+0000","last_change":"2023-10-18T10:25:04.239623+0000","last_active":"2023-10-18T10:25:08.254296+0000","last_peered":"2023-10-18T10:25:08.254296+0000","last_clean":"2023-10-18T10:25:08.254296+0000","last_became_active":"2023-10-18T10:25:04.239376+0000","last_became_peered":"2023-10-18T10:25:04.239376+0000","last_unstale":"2023-10-18T10:25:08.254296+0000","last_undegraded":"2023-10-18T10:25:08.254296+0000","last_fullsized":"2023-10-18T10:25:08.254296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.789045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255778+0000","last_change":"2023-10-18T10:25:04.224851+0000","last_active":"2023-10-18T10:25:08.255778+0000","last_peered":"2023-10-18T10:25:08.255778+0000","last_clean":"2023-10-18T10:25:08.255778+0000","last_became_active":"2023-10-18T10:25:04.224579+0000","last_became_peered":"2023-10-18T10:25:04.224579+0000","last_unstale":"2023-10-18T10:25:08.255778+0000","last_undegraded":"2023-10-18T10:25:08.255778+0000","last_fullsized":"2023-10-18T10:25:08.255778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254745+0000","last_change":"2023-10-18T10:25:04.233040+0000","last_active":"2023-10-18T10:25:08.254745+0000","last_peered":"2023-10-18T10:25:08.254745+0000","last_clean":"2023-10-18T10:25:08.254745+0000","last_became_active":"2023-10-18T10:25:04.232782+0000","last_became_peered":"2023-10-18T10:25:04.232782+0000","last_unstale":"2023-10-18T10:25:08.254745+0000","last_undegraded":"2023-10-18T10:25:08.254745+0000","last_fullsized":"2023-10-18T10:25:08.254745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255053+0000","last_change":"2023-10-18T10:25:04.235058+0000","last_active":"2023-10-18T10:25:08.255053+0000","last_peered":"2023-10-18T10:25:08.255053+0000","last_clean":"2023-10-18T10:25:08.255053+0000","last_became_active":"2023-10-18T10:25:04.234763+0000","last_became_peered":"2023-10-18T10:25:04.234763+0000","last_unstale":"2023-10-18T10:25:08.255053+0000","last_undegraded":"2023-10-18T10:25:08.255053+0000","last_fullsized":"2023-10-18T10:25:08.255053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263580+0000","last_change":"2023-10-18T10:25:04.231136+0000","last_active":"2023-10-18T10:25:08.263580+0000","last_peered":"2023-10-18T10:25:08.263580+0000","last_clean":"2023-10-18T10:25:08.263580+0000","last_became_active":"2023-10-18T10:25:04.230835+0000","last_became_peered":"2023-10-18T10:25:04.230835+0000","last_unstale":"2023-10-18T10:25:08.263580+0000","last_undegraded":"2023-10-18T10:25:08.263580+0000","last_fullsized":"2023-10-18T10:25:08.263580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370824+0000","last_change":"2023-10-18T10:25:04.236843+0000","last_active":"2023-10-18T10:25:08.370824+0000","last_peered":"2023-10-18T10:25:08.370824+0000","last_clean":"2023-10-18T10:25:08.370824+0000","last_became_active":"2023-10-18T10:25:04.236484+0000","last_became_peered":"2023-10-18T10:25:04.236484+0000","last_unstale":"2023-10-18T10:25:08.370824+0000","last_undegraded":"2023-10-18T10:25:08.370824+0000","last_fullsized":"2023-10-18T10:25:08.370824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.789045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250697+0000","last_change":"2023-10-18T10:25:04.230906+0000","last_active":"2023-10-18T10:25:08.250697+0000","last_peered":"2023-10-18T10:25:08.250697+0000","last_clean":"2023-10-18T10:25:08.250697+0000","last_became_active":"2023-10-18T10:25:04.230641+0000","last_became_peered":"2023-10-18T10:25:04.230641+0000","last_unstale":"2023-10-18T10:25:08.250697+0000","last_undegraded":"2023-10-18T10:25:08.250697+0000","last_fullsized":"2023-10-18T10:25:08.250697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:44.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254323+0000","last_change":"2023-10-18T10:25:04.235839+0000","last_active":"2023-10-18T10:25:08.254323+0000","last_peered":"2023-10-18T10:25:08.254323+0000","last_clean":"2023-10-18T10:25:08.254323+0000","last_became_active":"2023-10-18T10:25:04.235602+0000","last_became_peered":"2023-10-18T10:25:04.235602+0000","last_unstale":"2023-10-18T10:25:08.254323+0000","last_undegraded":"2023-10-18T10:25:08.254323+0000","last_fullsized":"2023-10-18T10:25:08.254323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254219+0000","last_change":"2023-10-18T10:25:04.235249+0000","last_active":"2023-10-18T10:25:08.254219+0000","last_peered":"2023-10-18T10:25:08.254219+0000","last_clean":"2023-10-18T10:25:08.254219+0000","last_became_active":"2023-10-18T10:25:04.234967+0000","last_became_peered":"2023-10-18T10:25:04.234967+0000","last_unstale":"2023-10-18T10:25:08.254219+0000","last_undegraded":"2023-10-18T10:25:08.254219+0000","last_fullsized":"2023-10-18T10:25:08.254219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253370+0000","last_change":"2023-10-18T10:25:04.232541+0000","last_active":"2023-10-18T10:25:08.253370+0000","last_peered":"2023-10-18T10:25:08.253370+0000","last_clean":"2023-10-18T10:25:08.253370+0000","last_became_active":"2023-10-18T10:25:04.231594+0000","last_became_peered":"2023-10-18T10:25:04.231594+0000","last_unstale":"2023-10-18T10:25:08.253370+0000","last_undegraded":"2023-10-18T10:25:08.253370+0000","last_fullsized":"2023-10-18T10:25:08.253370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254018+0000","last_change":"2023-10-18T10:25:04.239452+0000","last_active":"2023-10-18T10:25:08.254018+0000","last_peered":"2023-10-18T10:25:08.254018+0000","last_clean":"2023-10-18T10:25:08.254018+0000","last_became_active":"2023-10-18T10:25:04.239175+0000","last_became_peered":"2023-10-18T10:25:04.239175+0000","last_unstale":"2023-10-18T10:25:08.254018+0000","last_undegraded":"2023-10-18T10:25:08.254018+0000","last_fullsized":"2023-10-18T10:25:08.254018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:42.041557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253482+0000","last_change":"2023-10-18T10:25:04.225873+0000","last_active":"2023-10-18T10:25:08.253482+0000","last_peered":"2023-10-18T10:25:08.253482+0000","last_clean":"2023-10-18T10:25:08.253482+0000","last_became_active":"2023-10-18T10:25:04.225624+0000","last_became_peered":"2023-10-18T10:25:04.225624+0000","last_unstale":"2023-10-18T10:25:08.253482+0000","last_undegraded":"2023-10-18T10:25:08.253482+0000","last_fullsized":"2023-10-18T10:25:08.253482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256440+0000","last_change":"2023-10-18T10:25:04.226107+0000","last_active":"2023-10-18T10:25:08.256440+0000","last_peered":"2023-10-18T10:25:08.256440+0000","last_clean":"2023-10-18T10:25:08.256440+0000","last_became_active":"2023-10-18T10:25:04.225823+0000","last_became_peered":"2023-10-18T10:25:04.225823+0000","last_unstale":"2023-10-18T10:25:08.256440+0000","last_undegraded":"2023-10-18T10:25:08.256440+0000","last_fullsized":"2023-10-18T10:25:08.256440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:35.440931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263514+0000","last_change":"2023-10-18T10:25:04.237730+0000","last_active":"2023-10-18T10:25:08.263514+0000","last_peered":"2023-10-18T10:25:08.263514+0000","last_clean":"2023-10-18T10:25:08.263514+0000","last_became_active":"2023-10-18T10:25:04.237436+0000","last_became_peered":"2023-10-18T10:25:04.237436+0000","last_unstale":"2023-10-18T10:25:08.263514+0000","last_undegraded":"2023-10-18T10:25:08.263514+0000","last_fullsized":"2023-10-18T10:25:08.263514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264090+0000","last_change":"2023-10-18T10:25:04.242105+0000","last_active":"2023-10-18T10:25:08.264090+0000","last_peered":"2023-10-18T10:25:08.264090+0000","last_clean":"2023-10-18T10:25:08.264090+0000","last_became_active":"2023-10-18T10:25:04.241762+0000","last_became_peered":"2023-10-18T10:25:04.241762+0000","last_unstale":"2023-10-18T10:25:08.264090+0000","last_undegraded":"2023-10-18T10:25:08.264090+0000","last_fullsized":"2023-10-18T10:25:08.264090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371062+0000","last_change":"2023-10-18T10:25:04.239055+0000","last_active":"2023-10-18T10:25:08.371062+0000","last_peered":"2023-10-18T10:25:08.371062+0000","last_clean":"2023-10-18T10:25:08.371062+0000","last_became_active":"2023-10-18T10:25:04.238566+0000","last_became_peered":"2023-10-18T10:25:04.238566+0000","last_unstale":"2023-10-18T10:25:08.371062+0000","last_undegraded":"2023-10-18T10:25:08.371062+0000","last_fullsized":"2023-10-18T10:25:08.371062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:43.022445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254270+0000","last_change":"2023-10-18T10:25:04.238819+0000","last_active":"2023-10-18T10:25:08.254270+0000","last_peered":"2023-10-18T10:25:08.254270+0000","last_clean":"2023-10-18T10:25:08.254270+0000","last_became_active":"2023-10-18T10:25:04.238546+0000","last_became_peered":"2023-10-18T10:25:04.238546+0000","last_unstale":"2023-10-18T10:25:08.254270+0000","last_undegraded":"2023-10-18T10:25:08.254270+0000","last_fullsized":"2023-10-18T10:25:08.254270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371571+0000","last_change":"2023-10-18T10:25:04.234955+0000","last_active":"2023-10-18T10:25:08.371571+0000","last_peered":"2023-10-18T10:25:08.371571+0000","last_clean":"2023-10-18T10:25:08.371571+0000","last_became_active":"2023-10-18T10:25:04.234687+0000","last_became_peered":"2023-10-18T10:25:04.234687+0000","last_unstale":"2023-10-18T10:25:08.371571+0000","last_undegraded":"2023-10-18T10:25:08.371571+0000","last_fullsized":"2023-10-18T10:25:08.371571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254664+0000","last_change":"2023-10-18T10:25:04.235078+0000","last_active":"2023-10-18T10:25:08.254664+0000","last_peered":"2023-10-18T10:25:08.254664+0000","last_clean":"2023-10-18T10:25:08.254664+0000","last_became_active":"2023-10-18T10:25:04.234817+0000","last_became_peered":"2023-10-18T10:25:04.234817+0000","last_unstale":"2023-10-18T10:25:08.254664+0000","last_undegraded":"2023-10-18T10:25:08.254664+0000","last_fullsized":"2023-10-18T10:25:08.254664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370554+0000","last_change":"2023-10-18T10:25:04.228077+0000","last_active":"2023-10-18T10:25:08.370554+0000","last_peered":"2023-10-18T10:25:08.370554+0000","last_clean":"2023-10-18T10:25:08.370554+0000","last_became_active":"2023-10-18T10:25:04.227740+0000","last_became_peered":"2023-10-18T10:25:04.227740+0000","last_unstale":"2023-10-18T10:25:08.370554+0000","last_undegraded":"2023-10-18T10:25:08.370554+0000","last_fullsized":"2023-10-18T10:25:08.370554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:26.705420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.258398+0000","last_change":"2023-10-18T10:25:04.240451+0000","last_active":"2023-10-18T10:25:08.258398+0000","last_peered":"2023-10-18T10:25:08.258398+0000","last_clean":"2023-10-18T10:25:08.258398+0000","last_became_active":"2023-10-18T10:25:04.240190+0000","last_became_peered":"2023-10-18T10:25:04.240190+0000","last_unstale":"2023-10-18T10:25:08.258398+0000","last_undegraded":"2023-10-18T10:25:08.258398+0000","last_fullsized":"2023-10-18T10:25:08.258398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:01.626703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254434+0000","last_change":"2023-10-18T10:25:04.233621+0000","last_active":"2023-10-18T10:25:08.254434+0000","last_peered":"2023-10-18T10:25:08.254434+0000","last_clean":"2023-10-18T10:25:08.254434+0000","last_became_active":"2023-10-18T10:25:04.233376+0000","last_became_peered":"2023-10-18T10:25:04.233376+0000","last_unstale":"2023-10-18T10:25:08.254434+0000","last_undegraded":"2023-10-18T10:25:08.254434+0000","last_fullsized":"2023-10-18T10:25:08.254434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:34.587493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255094+0000","last_change":"2023-10-18T10:25:04.234775+0000","last_active":"2023-10-18T10:25:08.255094+0000","last_peered":"2023-10-18T10:25:08.255094+0000","last_clean":"2023-10-18T10:25:08.255094+0000","last_became_active":"2023-10-18T10:25:04.234527+0000","last_became_peered":"2023-10-18T10:25:04.234527+0000","last_unstale":"2023-10-18T10:25:08.255094+0000","last_undegraded":"2023-10-18T10:25:08.255094+0000","last_fullsized":"2023-10-18T10:25:08.255094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370952+0000","last_change":"2023-10-18T10:25:04.236702+0000","last_active":"2023-10-18T10:25:08.370952+0000","last_peered":"2023-10-18T10:25:08.370952+0000","last_clean":"2023-10-18T10:25:08.370952+0000","last_became_active":"2023-10-18T10:25:04.236396+0000","last_became_peered":"2023-10-18T10:25:04.236396+0000","last_unstale":"2023-10-18T10:25:08.370952+0000","last_undegraded":"2023-10-18T10:25:08.370952+0000","last_fullsized":"2023-10-18T10:25:08.370952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:01.626703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:25:21.569 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:25:21.570 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2023-10-18T10:25:21.571 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-18T10:25:21.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.704+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/2842852157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 msgr2=0x7f3cb0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.704+0000 7f3ccb5b6640 1 --2- 172.21.15.125:0/2842852157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 0x7f3cb0003d70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00043b0 tx=0x7f3cb00057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.704+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/2842852157 shutdown_connections 2023-10-18T10:25:21.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.704+0000 7f3ccb5b6640 1 --2- 172.21.15.125:0/2842852157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 0x7f3cb0003d70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.704+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/2842852157 >> 172.21.15.125:0/2842852157 conn(0x7f3cc406fae0 msgr2=0x7f3cc406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:21.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.705+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/2842852157 shutdown_connections 2023-10-18T10:25:21.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.705+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/2842852157 wait complete. 2023-10-18T10:25:21.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3ccb5b6640 1 Processor -- start 2023-10-18T10:25:21.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3ccb5b6640 1 -- start start 2023-10-18T10:25:21.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3ccb5b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 0x7f3cc4153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3ccb5b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cc413e600 0x7f3cc413ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3ccb5b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cc413efc0 0x7f3cc415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3ccb5b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc41429e0 con 0x7f3cb0003970 2023-10-18T10:25:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3ccb5b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4154490 con 0x7f3cc413e600 2023-10-18T10:25:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3ccb5b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4154770 con 0x7f3cc413efc0 2023-10-18T10:25:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3cc9352640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 0x7f3cc4153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3cc9352640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 0x7f3cc4153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49564/0 (socket says 172.21.15.125:49564) 2023-10-18T10:25:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.706+0000 7f3cc9352640 1 -- 172.21.15.125:0/3669513444 learned_addr learned my addr 172.21.15.125:0/3669513444 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cc9b53640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cc413efc0 0x7f3cc415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cc9352640 1 -- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cc413efc0 msgr2=0x7f3cc415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cc9352640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cc413efc0 0x7f3cc415baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cc9352640 1 -- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cc413e600 msgr2=0x7f3cc413ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cc9352640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cc413e600 0x7f3cc413ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cc9352640 1 -- 172.21.15.125:0/3669513444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4154a50 con 0x7f3cb0003970 2023-10-18T10:25:21.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cc9352640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 0x7f3cc4153e40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0004300 tx=0x7f3cc0004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:21.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cb9ffb640 1 -- 172.21.15.125:0/3669513444 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0006460 con 0x7f3cb0003970 2023-10-18T10:25:21.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cb9ffb640 1 -- 172.21.15.125:0/3669513444 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3cc00066d0 con 0x7f3cb0003970 2023-10-18T10:25:21.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc4154d30 con 0x7f3cb0003970 2023-10-18T10:25:21.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3cb9ffb640 1 -- 172.21.15.125:0/3669513444 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00356d0 con 0x7f3cb0003970 2023-10-18T10:25:21.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.707+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc413f440 con 0x7f3cb0003970 2023-10-18T10:25:21.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.708+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc40690e0 con 0x7f3cb0003970 2023-10-18T10:25:21.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.709+0000 7f3cb9ffb640 1 -- 172.21.15.125:0/3669513444 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3cc0032070 con 0x7f3cb0003970 2023-10-18T10:25:21.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.709+0000 7f3cb9ffb640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3ca806ae70 0x7f3ca806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:21.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.709+0000 7f3cb9ffb640 1 -- 172.21.15.125:0/3669513444 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3cc00ab520 con 0x7f3cb0003970 2023-10-18T10:25:21.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.709+0000 7f3cc8b51640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3ca806ae70 0x7f3ca806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:21.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.710+0000 7f3cc8b51640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3ca806ae70 0x7f3ca806d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4005bc0 tx=0x7f3cb4005a70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:21.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.713+0000 7f3cb9ffb640 1 -- 172.21.15.125:0/3669513444 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3cc0076260 con 0x7f3cb0003970 2023-10-18T10:25:21.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.879+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7f3cc40707b0 con 0x7f3cb0003970 2023-10-18T10:25:21.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.879+0000 7f3cb9ffb640 1 -- 172.21.15.125:0/3669513444 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v30) v1 ==== 86+0+160 (secure 0 0 0) 0x7f3cc0006ca0 con 0x7f3cb0003970 2023-10-18T10:25:21.882 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:21.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3ca806ae70 msgr2=0x7f3ca806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3ca806ae70 0x7f3ca806d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4005bc0 tx=0x7f3cb4005a70 comp rx=0 tx=0).stop 2023-10-18T10:25:21.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 msgr2=0x7f3cc4153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:21.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 0x7f3cc4153e40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0004300 tx=0x7f3cc0004a20 comp rx=0 tx=0).stop 2023-10-18T10:25:21.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 shutdown_connections 2023-10-18T10:25:21.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3ca806ae70 0x7f3ca806d330 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cc413e600 0x7f3cc413ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cc413efc0 0x7f3cc415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 --2- 172.21.15.125:0/3669513444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0003970 0x7f3cc4153e40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:21.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 >> 172.21.15.125:0/3669513444 conn(0x7f3cc406fae0 msgr2=0x7f3cc406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:21.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 shutdown_connections 2023-10-18T10:25:21.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.881+0000 7f3ccb5b6640 1 -- 172.21.15.125:0/3669513444 wait complete. 2023-10-18T10:25:21.893 INFO:teuthology.orchestra.run.smithi125.stdout:[{"version":30,"timestamp":"2023-10-18T10:24:49.815298+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"4899.127456"}]}] 2023-10-18T10:25:21.893 INFO:tasks.ceph_manager:config epoch is 30 2023-10-18T10:25:21.894 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-18T10:25:21.894 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-18T10:25:21.895 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-18T10:25:21.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.997+0000 7f3de7933640 1 -- 172.21.15.125:0/3905125555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00cf340 msgr2=0x7f3de00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.997+0000 7f3de7933640 1 --2- 172.21.15.125:0/3905125555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00cf340 0x7f3de00d1910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8005fd0 tx=0x7f3dd802c050 comp rx=0 tx=0).stop 2023-10-18T10:25:22.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.997+0000 7f3de7933640 1 -- 172.21.15.125:0/3905125555 shutdown_connections 2023-10-18T10:25:22.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.997+0000 7f3de7933640 1 --2- 172.21.15.125:0/3905125555 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de00ceb40 0x7f3de00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.997+0000 7f3de7933640 1 --2- 172.21.15.125:0/3905125555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00cf340 0x7f3de00d1910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.997+0000 7f3de7933640 1 -- 172.21.15.125:0/3905125555 >> 172.21.15.125:0/3905125555 conn(0x7f3de00d0d60 msgr2=0x7f3de00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:22.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.997+0000 7f3de7933640 1 -- 172.21.15.125:0/3905125555 shutdown_connections 2023-10-18T10:25:22.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.997+0000 7f3de7933640 1 -- 172.21.15.125:0/3905125555 wait complete. 2023-10-18T10:25:22.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.998+0000 7f3de7933640 1 Processor -- start 2023-10-18T10:25:22.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.998+0000 7f3de7933640 1 -- start start 2023-10-18T10:25:22.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de7933640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00ceb40 0x7f3de00b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de7933640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de00cf340 0x7f3de00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de7933640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de015b2a0 0x7f3de00ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de7933640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de009b220 con 0x7f3de00ceb40 2023-10-18T10:25:22.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de7933640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00b69e0 con 0x7f3de015b2a0 2023-10-18T10:25:22.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de7933640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de01f2b40 con 0x7f3de00cf340 2023-10-18T10:25:22.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00ceb40 0x7f3de00b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00ceb40 0x7f3de00b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49578/0 (socket says 172.21.15.125:49578) 2023-10-18T10:25:22.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 -- 172.21.15.125:0/4146827409 learned_addr learned my addr 172.21.15.125:0/4146827409 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:22.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de4ece640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de00cf340 0x7f3de00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de5ed0640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de015b2a0 0x7f3de00ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 -- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de00cf340 msgr2=0x7f3de00c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de00cf340 0x7f3de00c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 -- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de015b2a0 msgr2=0x7f3de00ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de015b2a0 0x7f3de00ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 -- 172.21.15.125:0/4146827409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8009840 con 0x7f3de00ceb40 2023-10-18T10:25:22.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:21.999+0000 7f3de56cf640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00ceb40 0x7f3de00b38e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009b30 tx=0x7f3dd8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:22.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.000+0000 7f3dd5ffb640 1 -- 172.21.15.125:0/4146827409 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd803e660 con 0x7f3de00ceb40 2023-10-18T10:25:22.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.000+0000 7f3dd5ffb640 1 -- 172.21.15.125:0/4146827409 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3dd803e8b0 con 0x7f3de00ceb40 2023-10-18T10:25:22.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.000+0000 7f3dd5ffb640 1 -- 172.21.15.125:0/4146827409 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd803eca0 con 0x7f3de00ceb40 2023-10-18T10:25:22.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.000+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de00b0d50 con 0x7f3de00ceb40 2023-10-18T10:25:22.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.000+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de00c7f60 con 0x7f3de00ceb40 2023-10-18T10:25:22.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.001+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de00cca00 con 0x7f3de00ceb40 2023-10-18T10:25:22.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.001+0000 7f3dd5ffb640 1 -- 172.21.15.125:0/4146827409 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3dd8034430 con 0x7f3de00ceb40 2023-10-18T10:25:22.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.002+0000 7f3dd5ffb640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3dc406ae40 0x7f3dc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.002+0000 7f3dd5ffb640 1 -- 172.21.15.125:0/4146827409 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3dd80acaf0 con 0x7f3de00ceb40 2023-10-18T10:25:22.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.002+0000 7f3de4ece640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3dc406ae40 0x7f3dc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.002+0000 7f3de4ece640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3dc406ae40 0x7f3dc406d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0007fd0 tx=0x7f3dd0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:22.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.006+0000 7f3dd5ffb640 1 -- 172.21.15.125:0/4146827409 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dd80778c0 con 0x7f3de00ceb40 2023-10-18T10:25:22.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.216+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3de00690e0 con 0x7f3de00ceb40 2023-10-18T10:25:22.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.219+0000 7f3dd5ffb640 1 -- 172.21.15.125:0/4146827409 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171036 (secure 0 0 0) 0x7f3dd807aee0 con 0x7f3de00ceb40 2023-10-18T10:25:22.224 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:22.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.221+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3dc406ae40 msgr2=0x7f3dc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.221+0000 7f3de7933640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3dc406ae40 0x7f3dc406d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0007fd0 tx=0x7f3dd0009040 comp rx=0 tx=0).stop 2023-10-18T10:25:22.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.221+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00ceb40 msgr2=0x7f3de00b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.221+0000 7f3de7933640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00ceb40 0x7f3de00b38e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009b30 tx=0x7f3dd8004360 comp rx=0 tx=0).stop 2023-10-18T10:25:22.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.222+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 shutdown_connections 2023-10-18T10:25:22.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.222+0000 7f3de7933640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3dc406ae40 0x7f3dc406d300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.222+0000 7f3de7933640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de015b2a0 0x7f3de00ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.222+0000 7f3de7933640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de00cf340 0x7f3de00c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.222+0000 7f3de7933640 1 --2- 172.21.15.125:0/4146827409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de00ceb40 0x7f3de00b38e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.222+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 >> 172.21.15.125:0/4146827409 conn(0x7f3de00d0d60 msgr2=0x7f3de00cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:22.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.222+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 shutdown_connections 2023-10-18T10:25:22.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.222+0000 7f3de7933640 1 -- 172.21.15.125:0/4146827409 wait complete. 2023-10-18T10:25:22.233 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":5,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2976686360},{"type":"v1","addr":"172.21.15.139:6833","nonce":2976686360}]},"active_addr":"172.21.15.139:6833/2976686360","active_change":"2023-10-18T10:24:42.554444+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4123,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1401100783}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":3880096480}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":557166225}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":2631710418}]}]} 2023-10-18T10:25:22.238 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-18T10:25:22.239 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-18T10:25:22.239 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:22.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.341+0000 7fe72932c640 1 -- 172.21.15.125:0/2072210157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 msgr2=0x7fe72406ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.341+0000 7fe72932c640 1 --2- 172.21.15.125:0/2072210157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 0x7fe72406ddc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe7240c9200 tx=0x7fe71802bd70 comp rx=0 tx=0).stop 2023-10-18T10:25:22.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.341+0000 7fe72932c640 1 -- 172.21.15.125:0/2072210157 shutdown_connections 2023-10-18T10:25:22.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.341+0000 7fe72932c640 1 --2- 172.21.15.125:0/2072210157 >> v2:172.21.15.139:3300/0 conn(0x7fe7240ad370 0x7fe7240ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.341+0000 7fe72932c640 1 --2- 172.21.15.125:0/2072210157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 0x7fe72406ddc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.341+0000 7fe72932c640 1 -- 172.21.15.125:0/2072210157 >> 172.21.15.125:0/2072210157 conn(0x7fe7240b0670 msgr2=0x7fe7240b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:22.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.341+0000 7fe72932c640 1 -- 172.21.15.125:0/2072210157 shutdown_connections 2023-10-18T10:25:22.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.341+0000 7fe72932c640 1 -- 172.21.15.125:0/2072210157 wait complete. 2023-10-18T10:25:22.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.342+0000 7fe72932c640 1 Processor -- start 2023-10-18T10:25:22.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.342+0000 7fe72932c640 1 -- start start 2023-10-18T10:25:22.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe72932c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72406d2d0 0x7fe72414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe72932c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 0x7fe7240c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe72932c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7240ad370 0x7fe7240cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe72932c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c9fe0 con 0x7fe72406f0a0 2023-10-18T10:25:22.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe72932c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c96b0 con 0x7fe7240ad370 2023-10-18T10:25:22.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe72932c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240c9990 con 0x7fe72406d2d0 2023-10-18T10:25:22.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 0x7fe7240c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 0x7fe7240c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49598/0 (socket says 172.21.15.125:49598) 2023-10-18T10:25:22.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 -- 172.21.15.125:0/3773839334 learned_addr learned my addr 172.21.15.125:0/3773839334 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:22.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe722ffd640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72406d2d0 0x7fe72414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7237fe640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7240ad370 0x7fe7240cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 -- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72406d2d0 msgr2=0x7fe72414ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72406d2d0 0x7fe72414ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 -- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7240ad370 msgr2=0x7fe7240cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7240ad370 0x7fe7240cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 -- 172.21.15.125:0/3773839334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718009490 con 0x7fe72406f0a0 2023-10-18T10:25:22.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe722ffd640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72406d2d0 0x7fe72414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:22.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7237fe640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7240ad370 0x7fe7240cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:22.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe7227fc640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 0x7fe7240c67e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe718000c00 tx=0x7fe7180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:22.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe70b7fe640 1 -- 172.21.15.125:0/3773839334 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718003ea0 con 0x7fe72406f0a0 2023-10-18T10:25:22.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe70b7fe640 1 -- 172.21.15.125:0/3773839334 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe718034550 con 0x7fe72406f0a0 2023-10-18T10:25:22.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7240c9c70 con 0x7fe72406f0a0 2023-10-18T10:25:22.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.343+0000 7fe70b7fe640 1 -- 172.21.15.125:0/3773839334 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718034960 con 0x7fe72406f0a0 2023-10-18T10:25:22.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.344+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7240b4010 con 0x7fe72406f0a0 2023-10-18T10:25:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.344+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7240c9e00 con 0x7fe72406f0a0 2023-10-18T10:25:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.345+0000 7fe70b7fe640 1 -- 172.21.15.125:0/3773839334 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe71803b040 con 0x7fe72406f0a0 2023-10-18T10:25:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.345+0000 7fe70b7fe640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe71006ae40 0x7fe71006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.345+0000 7fe70b7fe640 1 -- 172.21.15.125:0/3773839334 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe7180abe40 con 0x7fe72406f0a0 2023-10-18T10:25:22.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.345+0000 7fe722ffd640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe71006ae40 0x7fe71006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.346+0000 7fe722ffd640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe71006ae40 0x7fe71006d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe7140046b0 tx=0x7fe714009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:22.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.349+0000 7fe70b7fe640 1 -- 172.21.15.125:0/3773839334 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe718076c90 con 0x7fe72406f0a0 2023-10-18T10:25:22.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.519+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7240ce1d0 con 0x7fe72406f0a0 2023-10-18T10:25:22.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.520+0000 7fe70b7fe640 1 -- 172.21.15.125:0/3773839334 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7fe718039090 con 0x7fe72406f0a0 2023-10-18T10:25:22.522 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:22.523 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:22.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe71006ae40 msgr2=0x7fe71006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe71006ae40 0x7fe71006d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe7140046b0 tx=0x7fe714009040 comp rx=0 tx=0).stop 2023-10-18T10:25:22.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 msgr2=0x7fe7240c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 0x7fe7240c67e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe718000c00 tx=0x7fe7180038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 shutdown_connections 2023-10-18T10:25:22.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe71006ae40 0x7fe71006d300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7240ad370 0x7fe7240cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe72406f0a0 0x7fe7240c67e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 --2- 172.21.15.125:0/3773839334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe72406d2d0 0x7fe72414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 >> 172.21.15.125:0/3773839334 conn(0x7fe7240b0670 msgr2=0x7fe724158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:22.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 shutdown_connections 2023-10-18T10:25:22.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.522+0000 7fe72932c640 1 -- 172.21.15.125:0/3773839334 wait complete. 2023-10-18T10:25:22.535 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-18T10:25:22.536 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:22.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.636+0000 7fb736fb6640 1 -- 172.21.15.125:0/1545802301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb71c003f20 msgr2=0x7fb71c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.636+0000 7fb736fb6640 1 --2- 172.21.15.125:0/1545802301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb71c003f20 0x7fb71c0047c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb718009b50 tx=0x7fb71c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.636+0000 7fb735553640 1 -- 172.21.15.125:0/1545802301 reap_dead start 2023-10-18T10:25:22.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.636+0000 7fb736fb6640 1 -- 172.21.15.125:0/1545802301 shutdown_connections 2023-10-18T10:25:22.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.636+0000 7fb736fb6640 1 -- 172.21.15.125:0/1545802301 >> 172.21.15.125:0/1545802301 conn(0x7fb730069930 msgr2=0x7fb730069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:22.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.636+0000 7fb736fb6640 1 -- 172.21.15.125:0/1545802301 shutdown_connections 2023-10-18T10:25:22.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.636+0000 7fb736fb6640 1 -- 172.21.15.125:0/1545802301 wait complete. 2023-10-18T10:25:22.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.637+0000 7fb736fb6640 1 Processor -- start 2023-10-18T10:25:22.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.637+0000 7fb736fb6640 1 -- start start 2023-10-18T10:25:22.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb736fb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c003f20 0x7fb7301eff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb736fb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7300a64b0 0x7fb7300ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb736fb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7300ca870 0x7fb7300cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb736fb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300cb230 con 0x7fb7300ca870 2023-10-18T10:25:22.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb736fb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300ab670 con 0x7fb71c003f20 2023-10-18T10:25:22.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb736fb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300a6c50 con 0x7fb7300a64b0 2023-10-18T10:25:22.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7300ca870 0x7fb7300cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7300ca870 0x7fb7300cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49626/0 (socket says 172.21.15.125:49626) 2023-10-18T10:25:22.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 -- 172.21.15.125:0/2401862164 learned_addr learned my addr 172.21.15.125:0/2401862164 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:22.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 -- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7300a64b0 msgr2=0x7fb7300ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7300a64b0 0x7fb7300ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 -- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c003f20 msgr2=0x7fb7301eff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c003f20 0x7fb7301eff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 -- 172.21.15.125:0/2401862164 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300a6f30 con 0x7fb7300ca870 2023-10-18T10:25:22.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb735553640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7300ca870 0x7fb7300cacf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb720000c00 tx=0x7fb720001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:22.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.638+0000 7fb72d7fa640 1 -- 172.21.15.125:0/2401862164 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720011630 con 0x7fb7300ca870 2023-10-18T10:25:22.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.639+0000 7fb72d7fa640 1 -- 172.21.15.125:0/2401862164 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb7200117b0 con 0x7fb7300ca870 2023-10-18T10:25:22.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.639+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7300a7210 con 0x7fb7300ca870 2023-10-18T10:25:22.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.639+0000 7fb72d7fa640 1 -- 172.21.15.125:0/2401862164 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720011bc0 con 0x7fb7300ca870 2023-10-18T10:25:22.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.639+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7301f0660 con 0x7fb7300ca870 2023-10-18T10:25:22.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.639+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7300b1bd0 con 0x7fb7300ca870 2023-10-18T10:25:22.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.640+0000 7fb72d7fa640 1 -- 172.21.15.125:0/2401862164 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb720011d20 con 0x7fb7300ca870 2023-10-18T10:25:22.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.640+0000 7fb72d7fa640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb71006ae70 0x7fb71006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:22.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.640+0000 7fb72d7fa640 1 -- 172.21.15.125:0/2401862164 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb72008a160 con 0x7fb7300ca870 2023-10-18T10:25:22.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.640+0000 7fb734d52640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb71006ae70 0x7fb71006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:22.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.641+0000 7fb734d52640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb71006ae70 0x7fb71006d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb728006fd0 tx=0x7fb728001f30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:22.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.645+0000 7fb72d7fa640 1 -- 172.21.15.125:0/2401862164 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb720054f20 con 0x7fb7300ca870 2023-10-18T10:25:22.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.816+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7300690e0 con 0x7fb7300ca870 2023-10-18T10:25:22.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.817+0000 7fb72d7fa640 1 -- 172.21.15.125:0/2401862164 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7fb720058540 con 0x7fb7300ca870 2023-10-18T10:25:22.820 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:22.820 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:22.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.819+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb71006ae70 msgr2=0x7fb71006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.819+0000 7fb736fb6640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb71006ae70 0x7fb71006d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb728006fd0 tx=0x7fb728001f30 comp rx=0 tx=0).stop 2023-10-18T10:25:22.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.819+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7300ca870 msgr2=0x7fb7300cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:22.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.819+0000 7fb736fb6640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7300ca870 0x7fb7300cacf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb720000c00 tx=0x7fb720001630 comp rx=0 tx=0).stop 2023-10-18T10:25:22.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.820+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 shutdown_connections 2023-10-18T10:25:22.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.820+0000 7fb736fb6640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb71006ae70 0x7fb71006d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.820+0000 7fb736fb6640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7300ca870 0x7fb7300cacf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.820+0000 7fb736fb6640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7300a64b0 0x7fb7300ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.820+0000 7fb736fb6640 1 --2- 172.21.15.125:0/2401862164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c003f20 0x7fb7301eff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:22.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.820+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 >> 172.21.15.125:0/2401862164 conn(0x7fb730069930 msgr2=0x7fb730140260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:22.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.820+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 shutdown_connections 2023-10-18T10:25:22.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:22.820+0000 7fb736fb6640 1 -- 172.21.15.125:0/2401862164 wait complete. 2023-10-18T10:25:22.832 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-18T10:25:22.833 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-18T10:25:22.833 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-18T10:25:22.834 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-18T10:25:22.834 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-18T10:25:22.835 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-18T10:25:22.835 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-18T10:25:22.836 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-18T10:25:23.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.003+0000 7f993f59e640 1 -- 172.21.15.125:0/3495087592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938034e50 msgr2=0x7f9938050930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.003+0000 7f993f59e640 1 --2- 172.21.15.125:0/3495087592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938034e50 0x7f9938050930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99400771b0 tx=0x7f99400a3d20 comp rx=0 tx=0).stop 2023-10-18T10:25:23.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.004+0000 7f993f59e640 1 -- 172.21.15.125:0/3495087592 shutdown_connections 2023-10-18T10:25:23.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.004+0000 7f993f59e640 1 --2- 172.21.15.125:0/3495087592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938034e50 0x7f9938050930 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.004+0000 7f993f59e640 1 --2- 172.21.15.125:0/3495087592 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9938053760 0x7f99380503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.004+0000 7f993f59e640 1 --2- 172.21.15.125:0/3495087592 >> v2:172.21.15.139:3300/0 conn(0x7f9938006d20 0x7f9938054d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.004+0000 7f993f59e640 1 -- 172.21.15.125:0/3495087592 >> 172.21.15.125:0/3495087592 conn(0x7f9938055b90 msgr2=0x7f9938055fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.005+0000 7f993f59e640 1 -- 172.21.15.125:0/3495087592 shutdown_connections 2023-10-18T10:25:23.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.005+0000 7f993f59e640 1 -- 172.21.15.125:0/3495087592 wait complete. 2023-10-18T10:25:23.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fdee3fff640 1 -- 172.21.15.125:0/3397627226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 msgr2=0x7fdee40ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fdee3fff640 1 --2- 172.21.15.125:0/3397627226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 0x7fdee40ac2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdee4073570 tx=0x7fdedc02c320 comp rx=0 tx=0).stop 2023-10-18T10:25:23.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f993f59e640 1 Processor -- start 2023-10-18T10:25:23.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f65c7fff640 1 -- 172.21.15.125:0/2143831415 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80062d0 con 0x7f65bc004f00 2023-10-18T10:25:23.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f65d7639640 1 -- 172.21.15.125:0/2143831415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65bc004f00 msgr2=0x7f65bc00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f65d7639640 1 --2- 172.21.15.125:0/2143831415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65bc004f00 0x7f65bc00db90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65c800c530 tx=0x7f65bc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f65d5bd6640 1 -- 172.21.15.125:0/2143831415 reap_dead start 2023-10-18T10:25:23.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f65d7639640 1 -- 172.21.15.125:0/2143831415 shutdown_connections 2023-10-18T10:25:23.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f65d7639640 1 -- 172.21.15.125:0/2143831415 >> 172.21.15.125:0/2143831415 conn(0x7f65d0069900 msgr2=0x7f65d0069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f993f59e640 1 -- start start 2023-10-18T10:25:23.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7f993f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938006d20 0x7f993804a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7fa885da9640 1 -- 172.21.15.125:0/3525473210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa880156a50 msgr2=0x7fa8800b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7fa885da9640 1 --2- 172.21.15.125:0/3525473210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa880156a50 0x7fa8800b17e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa870005fd0 tx=0x7fa87002e250 comp rx=0 tx=0).stop 2023-10-18T10:25:23.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7fa885da9640 1 -- 172.21.15.125:0/3525473210 shutdown_connections 2023-10-18T10:25:23.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7fa885da9640 1 --2- 172.21.15.125:0/3525473210 >> v2:172.21.15.125:3300/0 conn(0x7fa8800cd1a0 0x7fa88006f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fa885da9640 1 --2- 172.21.15.125:0/3525473210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa880156a50 0x7fa8800b17e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fa885da9640 1 -- 172.21.15.125:0/3525473210 >> 172.21.15.125:0/3525473210 conn(0x7fa880069900 msgr2=0x7fa880069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fa885da9640 1 -- 172.21.15.125:0/3525473210 shutdown_connections 2023-10-18T10:25:23.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fa885da9640 1 -- 172.21.15.125:0/3525473210 wait complete. 2023-10-18T10:25:23.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7f993f59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938034e50 0x7f99380c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7f993f59e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9938053760 0x7f9938037730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7f993f59e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938036690 con 0x7f9938006d20 2023-10-18T10:25:23.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fdee3fff640 1 -- 172.21.15.125:0/3397627226 shutdown_connections 2023-10-18T10:25:23.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fdee3fff640 1 --2- 172.21.15.125:0/3397627226 >> v2:172.21.15.125:3300/0 conn(0x7fdee40aa4b0 0x7fdee40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fdee3fff640 1 --2- 172.21.15.125:0/3397627226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 0x7fdee40ac2d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fdee3fff640 1 -- 172.21.15.125:0/3397627226 >> 172.21.15.125:0/3397627226 conn(0x7fdee40ccc70 msgr2=0x7fdee40cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fc61bda4640 1 -- 172.21.15.125:0/344920556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6000047c0 msgr2=0x7fc600004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fc61bda4640 1 --2- 172.21.15.125:0/344920556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6000047c0 0x7fc600004c40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc6040099d0 tx=0x7fc6000057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f65d7639640 1 -- 172.21.15.125:0/2143831415 shutdown_connections 2023-10-18T10:25:23.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.007+0000 7f65d7639640 1 -- 172.21.15.125:0/2143831415 wait complete. 2023-10-18T10:25:23.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fdee3fff640 1 -- 172.21.15.125:0/3397627226 shutdown_connections 2023-10-18T10:25:23.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fdee3fff640 1 -- 172.21.15.125:0/3397627226 wait complete. 2023-10-18T10:25:23.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7f993e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938006d20 0x7f993804a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fc61bda4640 1 -- 172.21.15.125:0/344920556 shutdown_connections 2023-10-18T10:25:23.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fc61bda4640 1 --2- 172.21.15.125:0/344920556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6000047c0 0x7fc600004c40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7fc61bda4640 1 -- 172.21.15.125:0/344920556 >> 172.21.15.125:0/344920556 conn(0x7fc60c054ed0 msgr2=0x7fc60c0552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.009+0000 7fa885da9640 1 Processor -- start 2023-10-18T10:25:23.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7fa885da9640 1 -- start start 2023-10-18T10:25:23.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f1e8c70f640 1 -- 172.21.15.125:0/1348984865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e840cbea0 msgr2=0x7f1e84069720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f1e8c70f640 1 --2- 172.21.15.125:0/1348984865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e840cbea0 0x7f1e84069720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e8406b880 tx=0x7f1e8002be20 comp rx=0 tx=0).stop 2023-10-18T10:25:23.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7f993e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938006d20 0x7f993804a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49688/0 (socket says 172.21.15.125:49688) 2023-10-18T10:25:23.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.008+0000 7f993e59c640 1 -- 172.21.15.125:0/3739551355 learned_addr learned my addr 172.21.15.125:0/3739551355 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.009+0000 7f993dd9b640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938034e50 0x7f99380c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938036970 con 0x7f9938034e50 2023-10-18T10:25:23.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9938036c50 con 0x7f9938053760 2023-10-18T10:25:23.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.009+0000 7fc61bda4640 1 -- 172.21.15.125:0/344920556 shutdown_connections 2023-10-18T10:25:23.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.009+0000 7fc61bda4640 1 -- 172.21.15.125:0/344920556 wait complete. 2023-10-18T10:25:23.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f1e8c70f640 1 -- 172.21.15.125:0/1348984865 shutdown_connections 2023-10-18T10:25:23.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f1e8c70f640 1 --2- 172.21.15.125:0/1348984865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e840cbea0 0x7f1e84069720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f1e8c70f640 1 --2- 172.21.15.125:0/1348984865 >> v2:172.21.15.139:3300/0 conn(0x7f1e8415ea60 0x7f1e840cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f1e8c70f640 1 --2- 172.21.15.125:0/1348984865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e84152040 0x7f1e840c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f1e8c70f640 1 -- 172.21.15.125:0/1348984865 >> 172.21.15.125:0/1348984865 conn(0x7f1e840af810 msgr2=0x7f1e840afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.009+0000 7f65d7639640 1 Processor -- start 2023-10-18T10:25:23.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.009+0000 7f65d7639640 1 -- start start 2023-10-18T10:25:23.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7fdee3fff640 1 Processor -- start 2023-10-18T10:25:23.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7f993ed9d640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9938053760 0x7f9938037730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.010+0000 7fdee3fff640 1 -- start start 2023-10-18T10:25:23.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fc61bda4640 1 Processor -- start 2023-10-18T10:25:23.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fc61bda4640 1 -- start start 2023-10-18T10:25:23.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f65d7639640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d0159bd0 0x7f65d01fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f65d7639640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d0159fb0 0x7f65d01fd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f65d7639640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d014fbc0 0x7f65d0150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f65d7639640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d01fd9c0 con 0x7f65d014fbc0 2023-10-18T10:25:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f65d7639640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0201f10 con 0x7f65d0159bd0 2023-10-18T10:25:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f65d7639640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0201190 con 0x7f65d0159fb0 2023-10-18T10:25:23.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f993e59c640 1 -- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9938053760 msgr2=0x7f9938037730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f993e59c640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9938053760 0x7f9938037730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fa885da9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8800cd1a0 0x7fa88014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f993e59c640 1 -- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938034e50 msgr2=0x7f99380c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fa885da9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa880156a50 0x7fa88015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f993e59c640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938034e50 0x7f99380c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f993e59c640 1 -- 172.21.15.125:0/3739551355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994004e960 con 0x7f9938006d20 2023-10-18T10:25:23.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f993e59c640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938006d20 0x7f993804a6d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9f0 tx=0x7f993400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fa885da9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa880156e30 0x7fa88015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fa885da9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880158480 con 0x7fa880156a50 2023-10-18T10:25:23.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f65d53d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d0159bd0 0x7f65d01fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fa87effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa880156a50 0x7fa88015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fa87effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa880156a50 0x7fa88015c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49710/0 (socket says 172.21.15.125:49710) 2023-10-18T10:25:23.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7fa87effd640 1 -- 172.21.15.125:0/3050177322 learned_addr learned my addr 172.21.15.125:0/3050177322 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.011+0000 7f65d53d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d0159bd0 0x7f65d01fcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:60864/0 (socket says 172.21.15.125:60864) 2023-10-18T10:25:23.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d53d5640 1 -- 172.21.15.125:0/1184763181 learned_addr learned my addr 172.21.15.125:0/1184763181 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fa885da9640 1 -- 172.21.15.125:0/3050177322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88014a8d0 con 0x7fa880156e30 2023-10-18T10:25:23.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fa885da9640 1 -- 172.21.15.125:0/3050177322 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa880157800 con 0x7fa8800cd1a0 2023-10-18T10:25:23.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993400ccc0 con 0x7f9938006d20 2023-10-18T10:25:23.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 0x7fdee4153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9934010430 con 0x7f9938006d20 2023-10-18T10:25:23.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee40aa4b0 0x7fdee40993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9938036f30 con 0x7f9938006d20 2023-10-18T10:25:23.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993804ac10 con 0x7f9938006d20 2023-10-18T10:25:23.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d4bd4640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d0159fb0 0x7f65d01fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdee409a390 0x7fdee41f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee3fff640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee41f9dd0 con 0x7fdee409a390 2023-10-18T10:25:23.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee3fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee41f9810 con 0x7fdee40aa0d0 2023-10-18T10:25:23.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee3fff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee41f9af0 con 0x7fdee40aa4b0 2023-10-18T10:25:23.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d5bd6640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d014fbc0 0x7f65d0150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 0x7fdee4153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 0x7fdee4153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:60902/0 (socket says 172.21.15.125:60902) 2023-10-18T10:25:23.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee2ffd640 1 -- 172.21.15.125:0/3648318434 learned_addr learned my addr 172.21.15.125:0/3648318434 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7fdee27fc640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee40aa4b0 0x7fdee40993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f1e8c70f640 1 -- 172.21.15.125:0/1348984865 shutdown_connections 2023-10-18T10:25:23.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d53d5640 1 -- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d0159fb0 msgr2=0x7f65d01fd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d53d5640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d0159fb0 0x7f65d01fd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d53d5640 1 -- 172.21.15.125:0/1184763181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d014fbc0 msgr2=0x7f65d0150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d53d5640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d014fbc0 0x7f65d0150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d53d5640 1 -- 172.21.15.125:0/1184763181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d0201430 con 0x7f65d0159bd0 2023-10-18T10:25:23.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f65d4bd4640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d0159fb0 0x7f65d01fd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:23.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdee2ffd640 1 -- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee40aa4b0 msgr2=0x7fdee40993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdee2ffd640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdee40aa4b0 0x7fdee40993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdee2ffd640 1 -- 172.21.15.125:0/3648318434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdee409a390 msgr2=0x7fdee41f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdee2ffd640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdee409a390 0x7fdee41f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdee2ffd640 1 -- 172.21.15.125:0/3648318434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc009900 con 0x7fdee40aa0d0 2023-10-18T10:25:23.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fa87effd640 1 -- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8800cd1a0 msgr2=0x7fa88014a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fa87effd640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8800cd1a0 0x7fa88014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fa87effd640 1 -- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa880156e30 msgr2=0x7fa88015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fa87effd640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa880156e30 0x7fa88015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fa87effd640 1 -- 172.21.15.125:0/3050177322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009840 con 0x7fa880156a50 2023-10-18T10:25:23.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7f65d53d5640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d0159bd0 0x7f65d01fcf40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c970 tx=0x7f65cc00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdee2ffd640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 0x7fdee4153ff0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000f80 tx=0x7fdedc0325f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc007bb0 con 0x7f65d0159bd0 2023-10-18T10:25:23.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f65cc007e00 con 0x7f65d0159bd0 2023-10-18T10:25:23.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc005280 con 0x7f65d0159bd0 2023-10-18T10:25:23.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.012+0000 7f1e8c70f640 1 -- 172.21.15.125:0/1348984865 wait complete. 2023-10-18T10:25:23.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdecb7fe640 1 -- 172.21.15.125:0/3648318434 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc032bf0 con 0x7fdee40aa0d0 2023-10-18T10:25:23.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fc61bda4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6000047c0 0x7fc60c1807a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fc61bda4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60c0d7e50 0x7fc60c0d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7f65d7639640 1 -- 172.21.15.125:0/1184763181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d0201710 con 0x7f65d0159bd0 2023-10-18T10:25:23.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fa87effd640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa880156a50 0x7fa88015c550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b3f0 tx=0x7fa87400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fc61bda4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc60c0e2720 0x7fc60c0e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fc61bda4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c180ce0 con 0x7fc60c0e2720 2023-10-18T10:25:23.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fc61bda4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0d8810 con 0x7fc60c0d7e50 2023-10-18T10:25:23.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fc61bda4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0e34a0 con 0x7fc6000047c0 2023-10-18T10:25:23.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdee3fff640 1 -- 172.21.15.125:0/3648318434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee4098bd0 con 0x7fdee40aa0d0 2023-10-18T10:25:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdecb7fe640 1 -- 172.21.15.125:0/3648318434 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdedc032e40 con 0x7fdee40aa0d0 2023-10-18T10:25:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fdee3fff640 1 -- 172.21.15.125:0/3648318434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee4154530 con 0x7fdee40aa0d0 2023-10-18T10:25:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7fc61933f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60c0d7e50 0x7fc60c0d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc60c0e2720 0x7fc60c0e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc60c0e2720 0x7fc60c0e2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49742/0 (socket says 172.21.15.125:49742) 2023-10-18T10:25:23.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 -- 172.21.15.125:0/2766062201 learned_addr learned my addr 172.21.15.125:0/2766062201 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.013+0000 7f65d7639640 1 -- 172.21.15.125:0/1184763181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d01505b0 con 0x7f65d0159bd0 2023-10-18T10:25:23.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fdecb7fe640 1 -- 172.21.15.125:0/3648318434 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc03d690 con 0x7fdee40aa0d0 2023-10-18T10:25:23.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7f1e8c70f640 1 Processor -- start 2023-10-18T10:25:23.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 -- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6000047c0 msgr2=0x7fc60c1807a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:25:23.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6000047c0 0x7fc60c1807a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 -- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60c0d7e50 msgr2=0x7fc60c0d82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60c0d7e50 0x7fc60c0d82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 -- 172.21.15.125:0/2766062201 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0e3780 con 0x7fc60c0e2720 2023-10-18T10:25:23.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61933f640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60c0d7e50 0x7fc60c0d82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:25:23.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fc61a341640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc60c0e2720 0x7fc60c0e2ba0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00e180 tx=0x7fc610001780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7f68cdffb640 1 -- 172.21.15.125:0/3615251652 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c0005280 con 0x7f68c80e02a0 2023-10-18T10:25:23.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874011830 con 0x7fa880156a50 2023-10-18T10:25:23.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7fa885da9640 1 -- 172.21.15.125:0/3050177322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa880157ae0 con 0x7fa880156a50 2023-10-18T10:25:23.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65a8000fc0 con 0x7f65d0159bd0 2023-10-18T10:25:23.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7f1e8c70f640 1 -- start start 2023-10-18T10:25:23.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7fa885da9640 1 -- 172.21.15.125:0/3050177322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8801580a0 con 0x7fa880156a50 2023-10-18T10:25:23.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa874011990 con 0x7fa880156a50 2023-10-18T10:25:23.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f68cffff640 1 -- 172.21.15.125:0/3615251652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e02a0 msgr2=0x7f68c80e3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f68cffff640 1 --2- 172.21.15.125:0/3615251652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e02a0 0x7f68c80e3b90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009900 tx=0x7f68c002bea0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f1e8c70f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e840cbea0 0x7f1e84153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f1e8c70f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e84152040 0x7f1e8415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f1e8c70f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e8415ea60 0x7f1e841582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f1e8c70f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84153c40 con 0x7f1e84152040 2023-10-18T10:25:23.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f1e8c70f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e841f8b80 con 0x7f1e8415ea60 2023-10-18T10:25:23.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7f1e8c70f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e841f8e60 con 0x7f1e840cbea0 2023-10-18T10:25:23.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.015+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874011d20 con 0x7fa880156a50 2023-10-18T10:25:23.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.014+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934010800 con 0x7f9938006d20 2023-10-18T10:25:23.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7f1e8a4ab640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e840cbea0 0x7f1e84153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7f1e8a4ab640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e840cbea0 0x7f1e84153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:52950/0 (socket says 172.21.15.125:52950) 2023-10-18T10:25:23.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7fa885da9640 1 -- 172.21.15.125:0/3050177322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa88004da70 con 0x7fa880156a50 2023-10-18T10:25:23.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7f1e8a4ab640 1 -- 172.21.15.125:0/4243586373 learned_addr learned my addr 172.21.15.125:0/4243586373 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f65cc0053e0 con 0x7f65d0159bd0 2023-10-18T10:25:23.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7fdecb7fe640 1 -- 172.21.15.125:0/3648318434 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdedc03b040 con 0x7fdee40aa0d0 2023-10-18T10:25:23.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7fdecb7fe640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fded006af10 0x7fded006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7fdee27fc640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fded006af10 0x7fded006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7f1e89caa640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e84152040 0x7f1e8415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7f65c5ffb640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b406ae80 0x7f65b406d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f65cc08a070 con 0x7f65d0159bd0 2023-10-18T10:25:23.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fdecb7fe640 1 -- 172.21.15.125:0/3648318434 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdedc0ac290 con 0x7fdee40aa0d0 2023-10-18T10:25:23.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f65c5ffb640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7f65b40744c0 0x7f65b4076900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 --> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65b4076fd0 con 0x7f65b40744c0 2023-10-18T10:25:23.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65cc08a510 con 0x7f65d0159bd0 2023-10-18T10:25:23.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.016+0000 7f992cff9640 1 -- 172.21.15.125:0/3739551355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9924000fc0 con 0x7f9938006d20 2023-10-18T10:25:23.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9934010960 con 0x7f9938006d20 2023-10-18T10:25:23.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f0233fff640 1 -- 172.21.15.125:0/636820043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021c004e90 msgr2=0x7f021c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f0233fff640 1 --2- 172.21.15.125:0/636820043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021c004e90 0x7f021c00b980 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f022c009eb0 tx=0x7f021c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f02337fe640 1 -- 172.21.15.125:0/636820043 reap_dead start 2023-10-18T10:25:23.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f1e8a4ab640 1 -- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e8415ea60 msgr2=0x7f1e841582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f1e8a4ab640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e8415ea60 0x7f1e841582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f1e8a4ab640 1 -- 172.21.15.125:0/4243586373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e84152040 msgr2=0x7f1e8415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f1e8a4ab640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e84152040 0x7f1e8415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f1e8a4ab640 1 -- 172.21.15.125:0/4243586373 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74009490 con 0x7f1e840cbea0 2023-10-18T10:25:23.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f65d4bd4640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b406ae80 0x7f65b406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f0233fff640 1 -- 172.21.15.125:0/636820043 shutdown_connections 2023-10-18T10:25:23.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f0233fff640 1 -- 172.21.15.125:0/636820043 >> 172.21.15.125:0/636820043 conn(0x7f02340ae460 msgr2=0x7f02340ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f68cffff640 1 -- 172.21.15.125:0/3615251652 shutdown_connections 2023-10-18T10:25:23.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f68cffff640 1 --2- 172.21.15.125:0/3615251652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e02a0 0x7f68c80e3b90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f68cffff640 1 -- 172.21.15.125:0/3615251652 >> 172.21.15.125:0/3615251652 conn(0x7f68c801ef10 msgr2=0x7f68c8055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fdee3fff640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7fdec40015e0 0x7fdec4003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fdee27fc640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fded006af10 0x7fded006d3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdee41562e0 tx=0x7fded400b390 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fdee3fff640 1 -- 172.21.15.125:0/3648318434 --> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdec4006b40 con 0x7fdec40015e0 2023-10-18T10:25:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fdee37fe640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7fdec40015e0 0x7fdec4003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7fdee37fe640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7fdec40015e0 0x7fdec4003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa87401a400 con 0x7fa880156a50 2023-10-18T10:25:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7fa867fff640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa86c06ae40 0x7fa86c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa874058d80 con 0x7fa880156a50 2023-10-18T10:25:23.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7fa867fff640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7fa86c075050 0x7fa86c0774f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 --> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa86c077bc0 con 0x7fa86c075050 2023-10-18T10:25:23.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f1e8a4ab640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e840cbea0 0x7f1e84153700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400ea60 tx=0x7f1e7400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400cde0 con 0x7f1e840cbea0 2023-10-18T10:25:23.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1e74004020 con 0x7f1e840cbea0 2023-10-18T10:25:23.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74004410 con 0x7f1e840cbea0 2023-10-18T10:25:23.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f1e8c70f640 1 -- 172.21.15.125:0/4243586373 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e80009560 con 0x7f1e840cbea0 2023-10-18T10:25:23.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f1e8c70f640 1 -- 172.21.15.125:0/4243586373 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e841f9700 con 0x7f1e840cbea0 2023-10-18T10:25:23.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610003e50 con 0x7fc60c0e2720 2023-10-18T10:25:23.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc610011640 con 0x7fc60c0e2720 2023-10-18T10:25:23.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610011a50 con 0x7fc60c0e2720 2023-10-18T10:25:23.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c0e3a60 con 0x7fc60c0e2720 2023-10-18T10:25:23.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c0e4020 con 0x7fc60c0e2720 2023-10-18T10:25:23.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f65d5bd6640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7f65b40744c0 0x7f65b4076900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f65d4bd4640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b406ae80 0x7f65b406d340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f65c0006fd0 tx=0x7f65c0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f65d5bd6640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7f65b40744c0 0x7f65b4076900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f0233fff640 1 -- 172.21.15.125:0/636820043 shutdown_connections 2023-10-18T10:25:23.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f0233fff640 1 -- 172.21.15.125:0/636820043 wait complete. 2023-10-18T10:25:23.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7f0233fff640 1 Processor -- start 2023-10-18T10:25:23.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f68cffff640 1 -- 172.21.15.125:0/3615251652 shutdown_connections 2023-10-18T10:25:23.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f68cffff640 1 -- 172.21.15.125:0/3615251652 wait complete. 2023-10-18T10:25:23.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7fa87f7fe640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa86c06ae40 0x7fa86c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7fdecb7fe640 1 -- 172.21.15.125:0/3648318434 <== osd.0 v2:172.21.15.125:6800/3876811193 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fdec4006b40 con 0x7fdec40015e0 2023-10-18T10:25:23.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7fa87ffff640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7fa86c075050 0x7fa86c0774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7f68cffff640 1 Processor -- start 2023-10-18T10:25:23.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f992effd640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992006af10 0x7f992006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.017+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9934014070 con 0x7f9938006d20 2023-10-18T10:25:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f992effd640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f9920074550 0x7f9920076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 --> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9920077060 con 0x7f9920074550 2023-10-18T10:25:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f993408c840 con 0x7f9938006d20 2023-10-18T10:25:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.018+0000 7f993dd9b640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992006af10 0x7f992006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7f993dd9b640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992006af10 0x7f992006d3d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9938180d20 tx=0x7f9930005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7f1e8c70f640 1 -- 172.21.15.125:0/4243586373 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e841f9260 con 0x7f1e840cbea0 2023-10-18T10:25:23.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc60c0066f0 con 0x7fc60c0e2720 2023-10-18T10:25:23.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7f68cffff640 1 -- start start 2023-10-18T10:25:23.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 <== osd.5 v2:172.21.15.139:6816/1786373778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f65b4076fd0 con 0x7f65b40744c0 2023-10-18T10:25:23.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7fa87ffff640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7fa86c075050 0x7fa86c0774f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7fa87f7fe640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa86c06ae40 0x7fa86c06d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa870037000 tx=0x7fa870004a40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80dfec0 0x7f68c800b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c80e40d0 0x7f68c800bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e2ea0 0x7f68c80d6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cffff640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80df630 con 0x7f68c80dfec0 2023-10-18T10:25:23.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cffff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80e3380 con 0x7f68c80e40d0 2023-10-18T10:25:23.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cffff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80df070 con 0x7f68c80e2ea0 2023-10-18T10:25:23.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.019+0000 7f0233fff640 1 -- start start 2023-10-18T10:25:23.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1e740035f0 con 0x7f1e840cbea0 2023-10-18T10:25:23.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cf7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e2ea0 0x7f68c80d6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68ce7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c80e40d0 0x7f68c800bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cf7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e2ea0 0x7f68c80d6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:52964/0 (socket says 172.21.15.125:52964) 2023-10-18T10:25:23.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68ce7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c80e40d0 0x7f68c800bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:60922/0 (socket says 172.21.15.125:60922) 2023-10-18T10:25:23.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f68cf7fe640 1 -- 172.21.15.125:0/78058646 learned_addr learned my addr 172.21.15.125:0/78058646 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f993ed9d640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f9920074550 0x7f9920076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.020+0000 7f0233fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02340cb3a0 0x7f0234158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f0233fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f023415a5a0 0x7f02341585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f0233fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02341fefc0 0x7f0234201b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f0233fff640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023415a110 con 0x7f02340cb3a0 2023-10-18T10:25:23.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f0233fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02341ff530 con 0x7f02341fefc0 2023-10-18T10:25:23.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f0233fff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02341597e0 con 0x7f023415a5a0 2023-10-18T10:25:23.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa874058450 con 0x7fa880156a50 2023-10-18T10:25:23.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 <== osd.1 v2:172.21.15.125:6814/817623589 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fa86c077bc0 con 0x7fa86c075050 2023-10-18T10:25:23.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f02337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02341fefc0 0x7f0234201b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f68cf7fe640 1 -- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c80e40d0 msgr2=0x7f68c800bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f68cf7fe640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68c80e40d0 0x7f68c800bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f68cf7fe640 1 -- 172.21.15.125:0/78058646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80dfec0 msgr2=0x7f68c800b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f68cf7fe640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80dfec0 0x7f68c800b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f68cf7fe640 1 -- 172.21.15.125:0/78058646 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0009560 con 0x7f68c80e2ea0 2023-10-18T10:25:23.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f993ed9d640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f9920074550 0x7f9920076990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f1e7affd640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1e6806af10 0x7f1e6806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1e74014070 con 0x7f1e840cbea0 2023-10-18T10:25:23.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc610011bb0 con 0x7fc60c0e2720 2023-10-18T10:25:23.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f1e7affd640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f1e68074550 0x7f1e68076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 --> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e68077060 con 0x7f1e68074550 2023-10-18T10:25:23.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e7408a970 con 0x7f1e840cbea0 2023-10-18T10:25:23.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f02327fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f023415a5a0 0x7f02341585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f02327fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f023415a5a0 0x7f02341585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:52976/0 (socket says 172.21.15.125:52976) 2023-10-18T10:25:23.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.021+0000 7f02327fc640 1 -- 172.21.15.125:0/510323687 learned_addr learned my addr 172.21.15.125:0/510323687 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f0232ffd640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02340cb3a0 0x7f0234158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f02327fc640 1 -- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02341fefc0 msgr2=0x7f0234201b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f02327fc640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02341fefc0 0x7f0234201b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f02327fc640 1 -- 172.21.15.125:0/510323687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02340cb3a0 msgr2=0x7f0234158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f02327fc640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02340cb3a0 0x7f0234158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f02327fc640 1 -- 172.21.15.125:0/510323687 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234159ac0 con 0x7f023415a5a0 2023-10-18T10:25:23.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f68cf7fe640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e2ea0 0x7f68c80d6130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f68c400b410 tx=0x7f68c400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7fc60a7fc640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5f806ae40 0x7fc5f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc61008a190 con 0x7fc60c0e2720 2023-10-18T10:25:23.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f1e89caa640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1e6806af10 0x7f1e6806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f1e8acac640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f1e68074550 0x7f1e68076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7fc60a7fc640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fc5f8074480 0x7fc5f80768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f02327fc640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f023415a5a0 0x7f02341585a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f022800e740 tx=0x7f022800ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022800cbd0 con 0x7f023415a5a0 2023-10-18T10:25:23.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f022800ce40 con 0x7f023415a5a0 2023-10-18T10:25:23.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4004020 con 0x7f68c80e2ea0 2023-10-18T10:25:23.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f68c4004270 con 0x7f68c80e2ea0 2023-10-18T10:25:23.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 <== osd.3 v2:172.21.15.125:6808/3581645100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f9920077060 con 0x7f9920074550 2023-10-18T10:25:23.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.023+0000 7f1e8acac640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f1e68074550 0x7f1e68076990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 --> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f8076f90 con 0x7fc5f8074480 2023-10-18T10:25:23.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.022+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc61008c7b0 con 0x7fc60c0e2720 2023-10-18T10:25:23.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.023+0000 7fc61933f640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fc5f8074480 0x7fc5f80768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.023+0000 7f68cffff640 1 -- 172.21.15.125:0/78058646 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c80df350 con 0x7f68c80e2ea0 2023-10-18T10:25:23.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.023+0000 7f68cffff640 1 -- 172.21.15.125:0/78058646 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c80e1700 con 0x7f68c80e2ea0 2023-10-18T10:25:23.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.023+0000 7fc61933f640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fc5f8074480 0x7fc5f80768c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.023+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4011630 con 0x7f68c80e2ea0 2023-10-18T10:25:23.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.024+0000 7fc619b40640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5f806ae40 0x7fc5f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.023+0000 7f1e89caa640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1e6806af10 0x7f1e6806d3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c005c40 tx=0x7f1e7c005bd0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.023+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0228004280 con 0x7f023415a5a0 2023-10-18T10:25:23.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.024+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0234159da0 con 0x7f023415a5a0 2023-10-18T10:25:23.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.024+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02340cb780 con 0x7f023415a5a0 2023-10-18T10:25:23.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.024+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 <== osd.2 v2:172.21.15.125:6824/3658119855 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f1e68077060 con 0x7f1e68074550 2023-10-18T10:25:23.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.024+0000 7f68cffff640 1 -- 172.21.15.125:0/78058646 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68c8006400 con 0x7f68c80e2ea0 2023-10-18T10:25:23.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.024+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 <== osd.4 v2:172.21.15.139:6800/3552682439 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fc5f8076f90 con 0x7fc5f8074480 2023-10-18T10:25:23.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.024+0000 7fc619b40640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5f806ae40 0x7fc5f806d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc61404f250 tx=0x7fc614067c40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.026+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f68c4011790 con 0x7f68c80e2ea0 2023-10-18T10:25:23.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.026+0000 7f68b77fe640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f68bc06af40 0x7f68bc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.026+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f68c408a070 con 0x7f68c80e2ea0 2023-10-18T10:25:23.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f68b77fe640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7f68bc074410 0x7f68bc076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 --> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68bc076f20 con 0x7f68bc074410 2023-10-18T10:25:23.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68c408a420 con 0x7f68c80e2ea0 2023-10-18T10:25:23.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f68ceffd640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f68bc06af40 0x7f68bc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.025+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0234158f00 con 0x7f023415a5a0 2023-10-18T10:25:23.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.026+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f02280043e0 con 0x7f023415a5a0 2023-10-18T10:25:23.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f021bfff640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f022006af40 0x7f022006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f022808a220 con 0x7f023415a5a0 2023-10-18T10:25:23.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f021bfff640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7f0220074580 0x7f02200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 --> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0220077090 con 0x7f0220074580 2023-10-18T10:25:23.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f022808a5d0 con 0x7f023415a5a0 2023-10-18T10:25:23.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.027+0000 7f02337fe640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7f0220074580 0x7f02200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.028+0000 7f68ce7fc640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7f68bc074410 0x7f68bc076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.028+0000 7f0232ffd640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f022006af40 0x7f022006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.028+0000 7f68ceffd640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f68bc06af40 0x7f68bc06d400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68d00685a0 tx=0x7f68d00687d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.028+0000 7f68ce7fc640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7f68bc074410 0x7f68bc076850 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.028+0000 7f02337fe640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7f0220074580 0x7f02200769c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.029+0000 7f0232ffd640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f022006af40 0x7f022006d400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02340aab60 tx=0x7f0224005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.029+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 <== osd.7 v2:172.21.15.139:6824/2679488483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f68bc076f20 con 0x7f68bc074410 2023-10-18T10:25:23.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.032+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 <== osd.6 v2:172.21.15.139:6808/1564178764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f0220077090 con 0x7f0220074580 2023-10-18T10:25:23.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.047+0000 7fa885da9640 1 -- 172.21.15.125:0/3050177322 --> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8801585c0 con 0x7fa86c075050 2023-10-18T10:25:23.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.048+0000 7fa867fff640 1 -- 172.21.15.125:0/3050177322 <== osd.1 v2:172.21.15.125:6814/817623589 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8801585c0 con 0x7fa86c075050 2023-10-18T10:25:23.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.048+0000 7fa865ffb640 1 -- 172.21.15.125:0/3050177322 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7fa86c075050 msgr2=0x7fa86c0774f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.048+0000 7fa865ffb640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7fa86c075050 0x7fa86c0774f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa865ffb640 1 -- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa86c06ae40 msgr2=0x7fa86c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa865ffb640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa86c06ae40 0x7fa86c06d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa870037000 tx=0x7fa870004a40 comp rx=0 tx=0).stop 2023-10-18T10:25:23.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa865ffb640 1 -- 172.21.15.125:0/3050177322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa880156a50 msgr2=0x7fa88015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa865ffb640 1 --2- 172.21.15.125:0/3050177322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa880156a50 0x7fa88015c550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b3f0 tx=0x7fa87400b8b0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa87ffff640 1 -- 172.21.15.125:0/3050177322 reap_dead start 2023-10-18T10:25:23.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa865ffb640 1 -- 172.21.15.125:0/3050177322 shutdown_connections 2023-10-18T10:25:23.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa865ffb640 1 -- 172.21.15.125:0/3050177322 >> 172.21.15.125:0/3050177322 conn(0x7fa880069900 msgr2=0x7fa880152330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa865ffb640 1 -- 172.21.15.125:0/3050177322 shutdown_connections 2023-10-18T10:25:23.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.049+0000 7fa865ffb640 1 -- 172.21.15.125:0/3050177322 wait complete. 2023-10-18T10:25:23.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.051+0000 7fdee3fff640 1 -- 172.21.15.125:0/3648318434 --> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdec4005c20 con 0x7fdec40015e0 2023-10-18T10:25:23.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.051+0000 7fdecb7fe640 1 -- 172.21.15.125:0/3648318434 <== osd.0 v2:172.21.15.125:6800/3876811193 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdec4005c20 con 0x7fdec40015e0 2023-10-18T10:25:23.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.052+0000 7fdec97fa640 1 -- 172.21.15.125:0/3648318434 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7fdec40015e0 msgr2=0x7fdec4003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.052+0000 7fdec97fa640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7fdec40015e0 0x7fdec4003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdec97fa640 1 -- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fded006af10 msgr2=0x7fded006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdec97fa640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fded006af10 0x7fded006d3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdee41562e0 tx=0x7fded400b390 comp rx=0 tx=0).stop 2023-10-18T10:25:23.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 --> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65a8002cb0 con 0x7f65b40744c0 2023-10-18T10:25:23.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdec97fa640 1 -- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 msgr2=0x7fdee4153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdec97fa640 1 --2- 172.21.15.125:0/3648318434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdee40aa0d0 0x7fdee4153ff0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000f80 tx=0x7fdedc0325f0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdee37fe640 1 -- 172.21.15.125:0/3648318434 reap_dead start 2023-10-18T10:25:23.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdec97fa640 1 -- 172.21.15.125:0/3648318434 shutdown_connections 2023-10-18T10:25:23.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdec97fa640 1 -- 172.21.15.125:0/3648318434 >> 172.21.15.125:0/3648318434 conn(0x7fdee40ccc70 msgr2=0x7fdee40ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdec97fa640 1 -- 172.21.15.125:0/3648318434 shutdown_connections 2023-10-18T10:25:23.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7fdec97fa640 1 -- 172.21.15.125:0/3648318434 wait complete. 2023-10-18T10:25:23.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.053+0000 7f65c5ffb640 1 -- 172.21.15.125:0/1184763181 <== osd.5 v2:172.21.15.139:6816/1786373778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f65a8002cb0 con 0x7f65b40744c0 2023-10-18T10:25:23.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7f65b40744c0 msgr2=0x7f65b4076900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65bb7fe640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7f65b40744c0 0x7f65b4076900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 --> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc60c0d1c60 con 0x7fc5f8074480 2023-10-18T10:25:23.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b406ae80 msgr2=0x7f65b406d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65bb7fe640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b406ae80 0x7f65b406d340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f65c0006fd0 tx=0x7f65c0008040 comp rx=0 tx=0).stop 2023-10-18T10:25:23.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d0159bd0 msgr2=0x7f65d01fcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65bb7fe640 1 --2- 172.21.15.125:0/1184763181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d0159bd0 0x7f65d01fcf40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c970 tx=0x7f65cc00ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:23.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65d5bd6640 1 -- 172.21.15.125:0/1184763181 reap_dead start 2023-10-18T10:25:23.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f992cff9640 1 -- 172.21.15.125:0/3739551355 --> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9924002cb0 con 0x7f9920074550 2023-10-18T10:25:23.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 shutdown_connections 2023-10-18T10:25:23.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.054+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 >> 172.21.15.125:0/1184763181 conn(0x7f65d0069900 msgr2=0x7f65d00aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.055+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 shutdown_connections 2023-10-18T10:25:23.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.055+0000 7f65bb7fe640 1 -- 172.21.15.125:0/1184763181 wait complete. 2023-10-18T10:25:23.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.055+0000 7fc60a7fc640 1 -- 172.21.15.125:0/2766062201 <== osd.4 v2:172.21.15.139:6800/3552682439 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc60c0d1c60 con 0x7fc5f8074480 2023-10-18T10:25:23.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.055+0000 7f992effd640 1 -- 172.21.15.125:0/3739551355 <== osd.3 v2:172.21.15.125:6808/3581645100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9924002cb0 con 0x7f9920074550 2023-10-18T10:25:23.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.055+0000 7f1e8c70f640 1 -- 172.21.15.125:0/4243586373 --> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e841f9260 con 0x7f1e68074550 2023-10-18T10:25:23.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.055+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fc5f8074480 msgr2=0x7fc5f80768c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.055+0000 7fc61bda4640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fc5f8074480 0x7fc5f80768c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f1e7affd640 1 -- 172.21.15.125:0/4243586373 <== osd.2 v2:172.21.15.125:6824/3658119855 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e841f9260 con 0x7f1e68074550 2023-10-18T10:25:23.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f9920074550 msgr2=0x7f9920076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993f59e640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f9920074550 0x7f9920076990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5f806ae40 msgr2=0x7fc5f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7fc61bda4640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5f806ae40 0x7fc5f806d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc61404f250 tx=0x7fc614067c40 comp rx=0 tx=0).stop 2023-10-18T10:25:23.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc60c0e2720 msgr2=0x7fc60c0e2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7fc61bda4640 1 --2- 172.21.15.125:0/2766062201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc60c0e2720 0x7fc60c0e2ba0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00e180 tx=0x7fc610001780 comp rx=0 tx=0).stop 2023-10-18T10:25:23.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992006af10 msgr2=0x7f992006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993f59e640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992006af10 0x7f992006d3d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9938180d20 tx=0x7f9930005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:23.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938006d20 msgr2=0x7f993804a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993f59e640 1 --2- 172.21.15.125:0/3739551355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938006d20 0x7f993804a6d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9f0 tx=0x7f993400eeb0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993ed9d640 1 -- 172.21.15.125:0/3739551355 reap_dead start 2023-10-18T10:25:23.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 shutdown_connections 2023-10-18T10:25:23.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 >> 172.21.15.125:0/3739551355 conn(0x7f9938055b90 msgr2=0x7f993804ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7fc61a341640 1 -- 172.21.15.125:0/2766062201 reap_dead start 2023-10-18T10:25:23.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.056+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 shutdown_connections 2023-10-18T10:25:23.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 >> 172.21.15.125:0/2766062201 conn(0x7fc60c054ed0 msgr2=0x7fc60c006c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 -- 172.21.15.125:0/4243586373 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f1e68074550 msgr2=0x7f1e68076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f1e68074550 0x7f1e68076990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 shutdown_connections 2023-10-18T10:25:23.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 shutdown_connections 2023-10-18T10:25:23.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 -- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1e6806af10 msgr2=0x7f1e6806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1e6806af10 0x7f1e6806d3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c005c40 tx=0x7f1e7c005bd0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 -- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e840cbea0 msgr2=0x7f1e84153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 --2- 172.21.15.125:0/4243586373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1e840cbea0 0x7f1e84153700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400ea60 tx=0x7f1e7400ef20 comp rx=0 tx=0).stop 2023-10-18T10:25:23.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e8acac640 1 -- 172.21.15.125:0/4243586373 reap_dead start 2023-10-18T10:25:23.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f993f59e640 1 -- 172.21.15.125:0/3739551355 wait complete. 2023-10-18T10:25:23.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7fc61bda4640 1 -- 172.21.15.125:0/2766062201 wait complete. 2023-10-18T10:25:23.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 -- 172.21.15.125:0/4243586373 shutdown_connections 2023-10-18T10:25:23.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 -- 172.21.15.125:0/4243586373 >> 172.21.15.125:0/4243586373 conn(0x7f1e840af810 msgr2=0x7f1e8415a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.057+0000 7f1e78ff9640 1 -- 172.21.15.125:0/4243586373 shutdown_connections 2023-10-18T10:25:23.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.058+0000 7f1e78ff9640 1 -- 172.21.15.125:0/4243586373 wait complete. 2023-10-18T10:25:23.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.060+0000 7f68cffff640 1 -- 172.21.15.125:0/78058646 --> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f68c804a840 con 0x7f68bc074410 2023-10-18T10:25:23.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.061+0000 7f68b77fe640 1 -- 172.21.15.125:0/78058646 <== osd.7 v2:172.21.15.139:6824/2679488483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f68c804a840 con 0x7f68bc074410 2023-10-18T10:25:23.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.062+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 --> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f023404da70 con 0x7f0220074580 2023-10-18T10:25:23.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.062+0000 7f68b57fa640 1 -- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7f68bc074410 msgr2=0x7f68bc076850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.062+0000 7f68b57fa640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7f68bc074410 0x7f68bc076850 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.062+0000 7f68b57fa640 1 -- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f68bc06af40 msgr2=0x7f68bc06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.062+0000 7f68b57fa640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f68bc06af40 0x7f68bc06d400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68d00685a0 tx=0x7f68d00687d0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.062+0000 7f68b57fa640 1 -- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e2ea0 msgr2=0x7f68c80d6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.062+0000 7f68b57fa640 1 --2- 172.21.15.125:0/78058646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c80e2ea0 0x7f68c80d6130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f68c400b410 tx=0x7f68c400b8d0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.062+0000 7f68cf7fe640 1 -- 172.21.15.125:0/78058646 reap_dead start 2023-10-18T10:25:23.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.063+0000 7f68b57fa640 1 -- 172.21.15.125:0/78058646 shutdown_connections 2023-10-18T10:25:23.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.063+0000 7f68b57fa640 1 -- 172.21.15.125:0/78058646 >> 172.21.15.125:0/78058646 conn(0x7f68c801ef10 msgr2=0x7f68c80539c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.063+0000 7f68b57fa640 1 -- 172.21.15.125:0/78058646 shutdown_connections 2023-10-18T10:25:23.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.063+0000 7f021bfff640 1 -- 172.21.15.125:0/510323687 <== osd.6 v2:172.21.15.139:6808/1564178764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f023404da70 con 0x7f0220074580 2023-10-18T10:25:23.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.063+0000 7f68b57fa640 1 -- 172.21.15.125:0/78058646 wait complete. 2023-10-18T10:25:23.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.063+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7f0220074580 msgr2=0x7f02200769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.064+0000 7f0233fff640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7f0220074580 0x7f02200769c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.064+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f022006af40 msgr2=0x7f022006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.064+0000 7f0233fff640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f022006af40 0x7f022006d400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02340aab60 tx=0x7f0224005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:23.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.064+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f023415a5a0 msgr2=0x7f02341585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.064+0000 7f0233fff640 1 --2- 172.21.15.125:0/510323687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f023415a5a0 0x7f02341585a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f022800e740 tx=0x7f022800ec00 comp rx=0 tx=0).stop 2023-10-18T10:25:23.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.064+0000 7f02337fe640 1 -- 172.21.15.125:0/510323687 reap_dead start 2023-10-18T10:25:23.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.064+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 shutdown_connections 2023-10-18T10:25:23.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.064+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 >> 172.21.15.125:0/510323687 conn(0x7f02340ae460 msgr2=0x7f023406ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.065+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 shutdown_connections 2023-10-18T10:25:23.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.065+0000 7f0233fff640 1 -- 172.21.15.125:0/510323687 wait complete. 2023-10-18T10:25:23.171 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:23.172 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-18T10:25:23.173 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:23.173 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T10:25:23.174 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:23.174 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-18T10:25:23.175 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:23.176 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-18T10:25:23.177 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:23.177 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-18T10:25:23.178 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:23.178 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-18T10:25:23.179 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:23.180 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-18T10:25:23.181 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:23.182 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-18T10:25:23.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.390+0000 7f6af0b33640 1 -- 172.21.15.125:0/2538162817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aec0c9ed0 msgr2=0x7f6aec06b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.390+0000 7f6af0b33640 1 --2- 172.21.15.125:0/2538162817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aec0c9ed0 0x7f6aec06b910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009830 tx=0x7f6ae002bd70 comp rx=0 tx=0).stop 2023-10-18T10:25:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.390+0000 7f6af0b33640 1 -- 172.21.15.125:0/2538162817 shutdown_connections 2023-10-18T10:25:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.390+0000 7f6af0b33640 1 --2- 172.21.15.125:0/2538162817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aec0c9ed0 0x7f6aec06b910 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.390+0000 7f6af0b33640 1 --2- 172.21.15.125:0/2538162817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6aec13d360 0x7f6aec15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.390+0000 7f6af0b33640 1 -- 172.21.15.125:0/2538162817 >> 172.21.15.125:0/2538162817 conn(0x7f6aec069930 msgr2=0x7f6aec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.390+0000 7f6af0b33640 1 -- 172.21.15.125:0/2538162817 shutdown_connections 2023-10-18T10:25:23.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.391+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/3992637064 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0069b0 msgr2=0x7fa0dc006e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7ff6bd626640 1 -- 172.21.15.125:0/132483108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 msgr2=0x7ff6b80c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7ff6bd626640 1 --2- 172.21.15.125:0/132483108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 0x7ff6b80c97c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8007350 tx=0x7ff6a802c9d0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.391+0000 7fa0e8a9a640 1 --2- 172.21.15.125:0/3992637064 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0069b0 0x7fa0dc006e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009860 tx=0x7fa0d003cdf0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.391+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/3992637064 shutdown_connections 2023-10-18T10:25:23.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.391+0000 7fa0e8a9a640 1 --2- 172.21.15.125:0/3992637064 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0069b0 0x7fa0dc006e40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.391+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/3992637064 >> 172.21.15.125:0/3992637064 conn(0x7fa0dc009f80 msgr2=0x7fa0dc00a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7f6af0b33640 1 -- 172.21.15.125:0/2538162817 wait complete. 2023-10-18T10:25:23.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7ff6bd626640 1 -- 172.21.15.125:0/132483108 shutdown_connections 2023-10-18T10:25:23.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7ff6bd626640 1 --2- 172.21.15.125:0/132483108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b80c9d00 0x7ff6b806b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7ff6bd626640 1 --2- 172.21.15.125:0/132483108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 0x7ff6b80c97c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7ff6bd626640 1 -- 172.21.15.125:0/132483108 >> 172.21.15.125:0/132483108 conn(0x7ff6b8069930 msgr2=0x7ff6b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7ff6bd626640 1 -- 172.21.15.125:0/132483108 shutdown_connections 2023-10-18T10:25:23.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/3992637064 shutdown_connections 2023-10-18T10:25:23.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/3992637064 wait complete. 2023-10-18T10:25:23.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.394+0000 7f6af0b33640 1 Processor -- start 2023-10-18T10:25:23.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.392+0000 7ff6bd626640 1 -- 172.21.15.125:0/132483108 wait complete. 2023-10-18T10:25:23.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.394+0000 7fa0e8a9a640 1 Processor -- start 2023-10-18T10:25:23.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.394+0000 7ff6bd626640 1 Processor -- start 2023-10-18T10:25:23.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f8655c60640 1 -- 172.21.15.125:0/3771510842 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86400042d0 msgr2=0x7f86400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f8655c60640 1 --2- 172.21.15.125:0/3771510842 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86400042d0 0x7f86400047c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f863c009880 tx=0x7f86400057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.394+0000 7ff6bd626640 1 -- start start 2023-10-18T10:25:23.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f865545f640 1 -- 172.21.15.125:0/3771510842 reap_dead start 2023-10-18T10:25:23.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f8655c60640 1 -- 172.21.15.125:0/3771510842 shutdown_connections 2023-10-18T10:25:23.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f8655c60640 1 -- 172.21.15.125:0/3771510842 >> 172.21.15.125:0/3771510842 conn(0x7f8648054ee0 msgr2=0x7f86480552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.394+0000 7f6af0b33640 1 -- start start 2023-10-18T10:25:23.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f6af0b33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6aec0c9ed0 0x7f6aec159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f6af0b33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aec13d360 0x7f6aec131580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f8655c60640 1 -- 172.21.15.125:0/3771510842 shutdown_connections 2023-10-18T10:25:23.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f6af0b33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aec140660 0x7f6aec1344f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f8655c60640 1 -- 172.21.15.125:0/3771510842 wait complete. 2023-10-18T10:25:23.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f6af0b33640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec131ac0 con 0x7f6aec13d360 2023-10-18T10:25:23.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f6af0b33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec159210 con 0x7f6aec140660 2023-10-18T10:25:23.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f6ae9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aec13d360 0x7f6aec131580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f6aea59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6aec0c9ed0 0x7f6aec159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.395+0000 7f6ae9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aec13d360 0x7f6aec131580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49794/0 (socket says 172.21.15.125:49794) 2023-10-18T10:25:23.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.394+0000 7fa0e8a9a640 1 -- start start 2023-10-18T10:25:23.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7f6ae9d9b640 1 -- 172.21.15.125:0/1265095994 learned_addr learned my addr 172.21.15.125:0/1265095994 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7f6aead9d640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aec140660 0x7f6aec1344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7f6af0b33640 1 -- 172.21.15.125:0/1265095994 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec1329b0 con 0x7f6aec0c9ed0 2023-10-18T10:25:23.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7fa0e8a9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc005e60 0x7fa0dc0e03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7fa0e8a9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0069b0 0x7fa0dc0dc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7fa0e8a9a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0eacd0 0x7fa0dc04ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6bd626640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b80c9d00 0x7ff6b812f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6bd626640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b813d360 0x7ff6b8200af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6bd626640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 0x7ff6b8131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6bd626640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8130f90 con 0x7ff6b813d360 2023-10-18T10:25:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6bd626640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81303c0 con 0x7ff6b8140660 2023-10-18T10:25:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6bd626640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8130660 con 0x7ff6b80c9d00 2023-10-18T10:25:23.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7fa0e8a9a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc04deb0 con 0x7fa0dc005e60 2023-10-18T10:25:23.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7fa0e8a9a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc04d2a0 con 0x7fa0dc0069b0 2023-10-18T10:25:23.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7fa0e8a9a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc04d580 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6b6ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b80c9d00 0x7ff6b812f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6b6ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b80c9d00 0x7ff6b812f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53042/0 (socket says 172.21.15.125:53042) 2023-10-18T10:25:23.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6b6ffd640 1 -- 172.21.15.125:0/2135734219 learned_addr learned my addr 172.21.15.125:0/2135734219 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7f6ae9d9b640 1 -- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6aec0c9ed0 msgr2=0x7f6aec159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7ff6b67fc640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b813d360 0x7ff6b8200af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.396+0000 7f6ae9d9b640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6aec0c9ed0 0x7f6aec159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6ae9d9b640 1 -- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aec140660 msgr2=0x7f6aec1344f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6ae9d9b640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aec140660 0x7f6aec1344f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6ae9d9b640 1 -- 172.21.15.125:0/1265095994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae0009490 con 0x7f6aec13d360 2023-10-18T10:25:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7ff6b6ffd640 1 -- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 msgr2=0x7ff6b8131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7ff6b77fe640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 0x7ff6b8131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6ae9d9b640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aec13d360 0x7f6aec131580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400c780 tx=0x7f6ae400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7ff6b6ffd640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 0x7ff6b8131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f8655c60640 1 Processor -- start 2023-10-18T10:25:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7ff6b6ffd640 1 -- 172.21.15.125:0/2135734219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b813d360 msgr2=0x7ff6b8200af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f8655c60640 1 -- start start 2023-10-18T10:25:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6adaffd640 1 -- 172.21.15.125:0/1265095994 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4003da0 con 0x7f6aec13d360 2023-10-18T10:25:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6adaffd640 1 -- 172.21.15.125:0/1265095994 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ae400ce90 con 0x7f6aec13d360 2023-10-18T10:25:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6adaffd640 1 -- 172.21.15.125:0/1265095994 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40052d0 con 0x7f6aec13d360 2023-10-18T10:25:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6af0b33640 1 -- 172.21.15.125:0/1265095994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec1594b0 con 0x7f6aec13d360 2023-10-18T10:25:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7f6af0b33640 1 -- 172.21.15.125:0/1265095994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec131f30 con 0x7f6aec13d360 2023-10-18T10:25:23.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7ff6b6ffd640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b813d360 0x7ff6b8200af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7ff6b6ffd640 1 -- 172.21.15.125:0/2135734219 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a800ae30 con 0x7ff6b80c9d00 2023-10-18T10:25:23.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7ff6b77fe640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 0x7ff6b8131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fe88884f640 1 -- 172.21.15.125:0/526738080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8840cb950 msgr2=0x7fe8840cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fe88884f640 1 --2- 172.21.15.125:0/526738080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8840cb950 0x7fe8840cdd50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe88406f4b0 tx=0x7fe87402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fe88884f640 1 -- 172.21.15.125:0/526738080 shutdown_connections 2023-10-18T10:25:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fe88884f640 1 --2- 172.21.15.125:0/526738080 >> v2:172.21.15.125:3300/0 conn(0x7fe8840b1190 0x7fe88415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fe88884f640 1 --2- 172.21.15.125:0/526738080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8840cb950 0x7fe8840cdd50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fe88884f640 1 -- 172.21.15.125:0/526738080 >> 172.21.15.125:0/526738080 conn(0x7fe8840b0670 msgr2=0x7fe8840b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7f8655c60640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86400042d0 0x7f864817f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7f8655c60640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86480d77c0 0x7f86480e2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7f8655c60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86480e28b0 0x7f86480368f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7f8655c60640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864804eca0 con 0x7f86480d77c0 2023-10-18T10:25:23.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7f8655c60640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864817f7b0 con 0x7f86480e28b0 2023-10-18T10:25:23.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7f8655c60640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864804c050 con 0x7f86400042d0 2023-10-18T10:25:23.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f864ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86480d77c0 0x7f86480e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f864ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86480d77c0 0x7f86480e2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49820/0 (socket says 172.21.15.125:49820) 2023-10-18T10:25:23.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f864ffff640 1 -- 172.21.15.125:0/3001836662 learned_addr learned my addr 172.21.15.125:0/3001836662 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7ff6b67fc640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b813d360 0x7ff6b8200af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:25:23.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f8654c5e640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86400042d0 0x7f864817f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0069b0 0x7fa0dc0dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0eacd0 0x7fa0dc04ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0eacd0 0x7fa0dc04ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53058/0 (socket says 172.21.15.125:53058) 2023-10-18T10:25:23.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e3fff640 1 -- 172.21.15.125:0/1899955103 learned_addr learned my addr 172.21.15.125:0/1899955103 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e3fff640 1 -- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0069b0 msgr2=0x7fa0dc0dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e3fff640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0069b0 0x7fa0dc0dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e3fff640 1 -- 172.21.15.125:0/1899955103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc005e60 msgr2=0x7fa0dc0e03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e3fff640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc005e60 0x7fa0dc0e03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.397+0000 7fa0e3fff640 1 -- 172.21.15.125:0/1899955103 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d0009490 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fa0e3fff640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0eacd0 0x7fa0dc04ca50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009980 tx=0x7fa0d0004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fa0cffff640 1 -- 172.21.15.125:0/1899955103 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0046b80 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fa0cffff640 1 -- 172.21.15.125:0/1899955103 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa0d0046ce0 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.398+0000 7fa0cffff640 1 -- 172.21.15.125:0/1899955103 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0003850 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7fe88884f640 1 -- 172.21.15.125:0/526738080 shutdown_connections 2023-10-18T10:25:23.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7fe88884f640 1 -- 172.21.15.125:0/526738080 wait complete. 2023-10-18T10:25:23.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f865545f640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86480e28b0 0x7f86480368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f864ffff640 1 -- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86400042d0 msgr2=0x7f864817f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7ff6b6ffd640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b80c9d00 0x7ff6b812f300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000cbc0 tx=0x7ff6b000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.400+0000 7f6af0b33640 1 -- 172.21.15.125:0/1265095994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec1599d0 con 0x7f6aec13d360 2023-10-18T10:25:23.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.400+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/1899955103 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc04d860 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.400+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/1899955103 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc04cf90 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f864ffff640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86400042d0 0x7f864817f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f864ffff640 1 -- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86480e28b0 msgr2=0x7f86480368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f864ffff640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86480e28b0 0x7f86480368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f864ffff640 1 -- 172.21.15.125:0/3001836662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864804c2f0 con 0x7f86480d77c0 2023-10-18T10:25:23.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.399+0000 7f865545f640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86480e28b0 0x7f86480368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:23.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.400+0000 7f864ffff640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86480d77c0 0x7f86480e2370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f863c0099a0 tx=0x7f863c02dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.401+0000 7f6adaffd640 1 -- 172.21.15.125:0/1265095994 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6ae4005430 con 0x7f6aec13d360 2023-10-18T10:25:23.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.401+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/1899955103 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc005360 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.401+0000 7ff49f7fe640 1 -- 172.21.15.125:0/2778619442 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494004700 con 0x7ff488003970 2023-10-18T10:25:23.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.401+0000 7fe88884f640 1 Processor -- start 2023-10-18T10:25:23.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.401+0000 7fe88884f640 1 -- start start 2023-10-18T10:25:23.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2778619442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff488003970 msgr2=0x7ff488003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7ff4a70d7640 1 --2- 172.21.15.125:0/2778619442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff488003970 0x7ff488003d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff49401d320 tx=0x7ff4880057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7f6adaffd640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6acc06af10 0x7f6acc06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.401+0000 7ff69f7fe640 1 -- 172.21.15.125:0/2135734219 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000ad40 con 0x7ff6b80c9d00 2023-10-18T10:25:23.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.401+0000 7ff69f7fe640 1 -- 172.21.15.125:0/2135734219 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff6b0012420 con 0x7ff6b80c9d00 2023-10-18T10:25:23.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.401+0000 7ff69f7fe640 1 -- 172.21.15.125:0/2135734219 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0012830 con 0x7ff6b80c9d00 2023-10-18T10:25:23.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7f6adaffd640 1 -- 172.21.15.125:0/1265095994 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6ae4089fd0 con 0x7f6aec13d360 2023-10-18T10:25:23.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7fa0cffff640 1 -- 172.21.15.125:0/1899955103 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa0d004c040 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7f6aea59c640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6acc06af10 0x7f6acc06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b8130940 con 0x7ff6b80c9d00 2023-10-18T10:25:23.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b8201030 con 0x7ff6b80c9d00 2023-10-18T10:25:23.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7ff4a5674640 1 -- 172.21.15.125:0/2778619442 reap_dead start 2023-10-18T10:25:23.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2778619442 shutdown_connections 2023-10-18T10:25:23.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2778619442 >> 172.21.15.125:0/2778619442 conn(0x7ff4a0069930 msgr2=0x7ff4a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7fa0cffff640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa0c806ae40 0x7fa0c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.402+0000 7fa0cffff640 1 -- 172.21.15.125:0/1899955103 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa0d00bc9c0 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fa0e37fe640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa0c806ae40 0x7fa0c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2778619442 shutdown_connections 2023-10-18T10:25:23.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7f6aea59c640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6acc06af10 0x7f6acc06d3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00a7c0 tx=0x7f6adc00c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2778619442 wait complete. 2023-10-18T10:25:23.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88884f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8840b1190 0x7fe8840c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88884f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8840cb950 0x7fe8841570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88884f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88415f3c0 0x7fe884157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88884f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8841513c0 con 0x7fe8840cb950 2023-10-18T10:25:23.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88884f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840c5310 con 0x7fe88415f3c0 2023-10-18T10:25:23.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88884f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840ab9a0 con 0x7fe8840b1190 2023-10-18T10:25:23.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88259c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8840b1190 0x7fe8840c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88259c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8840b1190 0x7fe8840c2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53122/0 (socket says 172.21.15.125:53122) 2023-10-18T10:25:23.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88259c640 1 -- 172.21.15.125:0/2447234878 learned_addr learned my addr 172.21.15.125:0/2447234878 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe881d9b640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8840cb950 0x7fe8841570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe882d9d640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88415f3c0 0x7fe884157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88259c640 1 -- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88415f3c0 msgr2=0x7fe884157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88259c640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88415f3c0 0x7fe884157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88259c640 1 -- 172.21.15.125:0/2447234878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8840cb950 msgr2=0x7fe8841570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88259c640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8840cb950 0x7fe8841570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.403+0000 7fe88259c640 1 -- 172.21.15.125:0/2447234878 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe874009490 con 0x7fe8840b1190 2023-10-18T10:25:23.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.404+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b80690e0 con 0x7ff6b80c9d00 2023-10-18T10:25:23.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.404+0000 7fe88259c640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8840b1190 0x7fe8840c2980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe874009780 tx=0x7fe874003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.404+0000 7fa0e37fe640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa0c806ae40 0x7fa0c806d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc04c9e0 tx=0x7fa0d80079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.404+0000 7ff4a70d7640 1 Processor -- start 2023-10-18T10:25:23.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.405+0000 7ff4a70d7640 1 -- start start 2023-10-18T10:25:23.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.405+0000 7ff4a70d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4a0156260 0x7ff4a0159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.405+0000 7ff4a70d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4a0156640 0x7ff4a00ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.405+0000 7ff4a70d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a015a3a0 0x7ff4a00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.405+0000 7ff4a70d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a0158190 con 0x7ff4a0156260 2023-10-18T10:25:23.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.405+0000 7ff4a70d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a0157510 con 0x7ff4a015a3a0 2023-10-18T10:25:23.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.405+0000 7ff4a70d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a01577f0 con 0x7ff4a0156640 2023-10-18T10:25:23.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4a0156260 0x7ff4a0159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4a0156260 0x7ff4a0159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49846/0 (socket says 172.21.15.125:49846) 2023-10-18T10:25:23.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 -- 172.21.15.125:0/2587100411 learned_addr learned my addr 172.21.15.125:0/2587100411 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a5674640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a015a3a0 0x7ff4a00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff49ffff640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4a0156640 0x7ff4a00ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 -- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4a0156640 msgr2=0x7ff4a00ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4a0156640 0x7ff4a00ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 -- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a015a3a0 msgr2=0x7ff4a00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a015a3a0 0x7ff4a00adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 -- 172.21.15.125:0/2587100411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a0157ad0 con 0x7ff4a0156260 2023-10-18T10:25:23.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a5674640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a015a3a0 0x7ff4a00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:25:23.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.406+0000 7ff4a4e73640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4a0156260 0x7ff4a0159e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff49401d440 tx=0x7ff494001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7f864d7fa640 1 -- 172.21.15.125:0/3001836662 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c02e620 con 0x7f86480d77c0 2023-10-18T10:25:23.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7f6adaffd640 1 -- 172.21.15.125:0/1265095994 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ae4058940 con 0x7f6aec13d360 2023-10-18T10:25:23.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7ff49d7fa640 1 -- 172.21.15.125:0/2587100411 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494006630 con 0x7ff4a0156260 2023-10-18T10:25:23.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864804c5d0 con 0x7f86480d77c0 2023-10-18T10:25:23.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864804cb90 con 0x7f86480d77c0 2023-10-18T10:25:23.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7f864d7fa640 1 -- 172.21.15.125:0/3001836662 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f863c03c040 con 0x7f86480d77c0 2023-10-18T10:25:23.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7f864d7fa640 1 -- 172.21.15.125:0/3001836662 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c02ea20 con 0x7f86480d77c0 2023-10-18T10:25:23.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a0157db0 con 0x7ff4a0156260 2023-10-18T10:25:23.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a0136950 con 0x7ff4a0156260 2023-10-18T10:25:23.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.407+0000 7ff49d7fa640 1 -- 172.21.15.125:0/2587100411 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff494006630 con 0x7ff4a0156260 2023-10-18T10:25:23.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.408+0000 7fe87affd640 1 -- 172.21.15.125:0/2447234878 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874003c20 con 0x7fe8840b1190 2023-10-18T10:25:23.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.408+0000 7fe87affd640 1 -- 172.21.15.125:0/2447234878 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe874003e70 con 0x7fe8840b1190 2023-10-18T10:25:23.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.408+0000 7fe87affd640 1 -- 172.21.15.125:0/2447234878 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874005280 con 0x7fe8840b1190 2023-10-18T10:25:23.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.408+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8840abc80 con 0x7fe8840b1190 2023-10-18T10:25:23.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.408+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8840ac240 con 0x7fe8840b1190 2023-10-18T10:25:23.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.408+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8630005250 con 0x7f86480d77c0 2023-10-18T10:25:23.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.408+0000 7fa0cffff640 1 -- 172.21.15.125:0/1899955103 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0d0087940 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.408+0000 7ff49d7fa640 1 -- 172.21.15.125:0/2587100411 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940036d0 con 0x7ff4a0156260 2023-10-18T10:25:23.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.409+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff480005250 con 0x7ff4a0156260 2023-10-18T10:25:23.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.409+0000 7f864d7fa640 1 -- 172.21.15.125:0/3001836662 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f863c037400 con 0x7f86480d77c0 2023-10-18T10:25:23.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.410+0000 7f864d7fa640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f863406ae40 0x7f863406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.410+0000 7f864d7fa640 1 -- 172.21.15.125:0/3001836662 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f863c0ab7d0 con 0x7f86480d77c0 2023-10-18T10:25:23.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.411+0000 7f8654c5e640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f863406ae40 0x7f863406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.411+0000 7ff69f7fe640 1 -- 172.21.15.125:0/2135734219 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff6b001b400 con 0x7ff6b80c9d00 2023-10-18T10:25:23.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.410+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe868005250 con 0x7fe8840b1190 2023-10-18T10:25:23.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.411+0000 7ff69f7fe640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff6a406ae90 0x7ff6a406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.411+0000 7f8654c5e640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f863406ae40 0x7f863406d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f865004f250 tx=0x7f865008a040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.411+0000 7ff49d7fa640 1 -- 172.21.15.125:0/2587100411 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff494042030 con 0x7ff4a0156260 2023-10-18T10:25:23.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.412+0000 7ff6b67fc640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff6a406ae90 0x7ff6a406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.412+0000 7ff49d7fa640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff48c06aea0 0x7ff48c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.412+0000 7ff69f7fe640 1 -- 172.21.15.125:0/2135734219 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff6b008a9e0 con 0x7ff6b80c9d00 2023-10-18T10:25:23.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.412+0000 7ff49ffff640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff48c06aea0 0x7ff48c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.412+0000 7ff49d7fa640 1 -- 172.21.15.125:0/2587100411 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff49408bab0 con 0x7ff4a0156260 2023-10-18T10:25:23.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.412+0000 7ff6b67fc640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff6a406ae90 0x7ff6a406d350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8036000 tx=0x7ff6a8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.413+0000 7ff49ffff640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff48c06aea0 0x7ff48c06d360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00c9170 tx=0x7ff49000b260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.413+0000 7ff69f7fe640 1 -- 172.21.15.125:0/2135734219 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6b00556f0 con 0x7ff6b80c9d00 2023-10-18T10:25:23.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.414+0000 7f9450a9a640 1 -- 172.21.15.125:0/1838875703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f944c0cf120 msgr2=0x7f944c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.414+0000 7f9450a9a640 1 --2- 172.21.15.125:0/1838875703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f944c0cf120 0x7f944c0cf590 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94400097b0 tx=0x7f944002bc60 comp rx=0 tx=0).stop 2023-10-18T10:25:23.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.414+0000 7f9450a9a640 1 -- 172.21.15.125:0/1838875703 shutdown_connections 2023-10-18T10:25:23.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.414+0000 7f9450a9a640 1 --2- 172.21.15.125:0/1838875703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f944c0cf120 0x7f944c0cf590 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.414+0000 7f9450a9a640 1 -- 172.21.15.125:0/1838875703 >> 172.21.15.125:0/1838875703 conn(0x7f944c099ee0 msgr2=0x7f944c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.415+0000 7fe87affd640 1 -- 172.21.15.125:0/2447234878 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe8740053e0 con 0x7fe8840b1190 2023-10-18T10:25:23.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.414+0000 7f9450a9a640 1 -- 172.21.15.125:0/1838875703 shutdown_connections 2023-10-18T10:25:23.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.414+0000 7f9450a9a640 1 -- 172.21.15.125:0/1838875703 wait complete. 2023-10-18T10:25:23.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.415+0000 7fe87affd640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe86406af10 0x7fe86406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.415+0000 7fe87affd640 1 -- 172.21.15.125:0/2447234878 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe8740abef0 con 0x7fe8840b1190 2023-10-18T10:25:23.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.415+0000 7fe881d9b640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe86406af10 0x7fe86406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.416+0000 7f9450a9a640 1 Processor -- start 2023-10-18T10:25:23.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.416+0000 7fe881d9b640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe86406af10 0x7fe86406d3d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe88406f4b0 tx=0x7fe86c00a2d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.416+0000 7f864d7fa640 1 -- 172.21.15.125:0/3001836662 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f863c076510 con 0x7f86480d77c0 2023-10-18T10:25:23.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.416+0000 7f9450a9a640 1 -- start start 2023-10-18T10:25:23.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.417+0000 7f9450a9a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f944c0afbf0 0x7f944c0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f9450a9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f944c0cf120 0x7f944c0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f9450a9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f944c15af50 0x7f944c0c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f944affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f944c0cf120 0x7f944c0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f944affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f944c0cf120 0x7f944c0c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49862/0 (socket says 172.21.15.125:49862) 2023-10-18T10:25:23.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f944b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f944c0afbf0 0x7f944c0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f944b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f944c0afbf0 0x7f944c0b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53154/0 (socket says 172.21.15.125:53154) 2023-10-18T10:25:23.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f944affd640 1 -- 172.21.15.125:0/3239052381 learned_addr learned my addr 172.21.15.125:0/3239052381 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7fe87affd640 1 -- 172.21.15.125:0/2447234878 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe874076d20 con 0x7fe8840b1190 2023-10-18T10:25:23.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c14fad0 con 0x7f944c0cf120 2023-10-18T10:25:23.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0b3520 con 0x7f944c15af50 2023-10-18T10:25:23.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f944bfff640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f944c15af50 0x7f944c0c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.418+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c14ee50 con 0x7f944c0afbf0 2023-10-18T10:25:23.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.419+0000 7f944b7fe640 1 -- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f944c15af50 msgr2=0x7f944c0c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.419+0000 7f944b7fe640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f944c15af50 0x7f944c0c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.419+0000 7f944b7fe640 1 -- 172.21.15.125:0/3239052381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f944c0cf120 msgr2=0x7f944c0c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.419+0000 7f944b7fe640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f944c0cf120 0x7f944c0c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.419+0000 7f944b7fe640 1 -- 172.21.15.125:0/3239052381 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440009490 con 0x7f944c0afbf0 2023-10-18T10:25:23.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.419+0000 7f944b7fe640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f944c0afbf0 0x7f944c0b2fe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c3c0 tx=0x7f943c00c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.419+0000 7f9433fff640 1 -- 172.21.15.125:0/3239052381 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c011790 con 0x7f944c0afbf0 2023-10-18T10:25:23.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.419+0000 7f9433fff640 1 -- 172.21.15.125:0/3239052381 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f943c0119e0 con 0x7f944c0afbf0 2023-10-18T10:25:23.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.420+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c14f130 con 0x7f944c0afbf0 2023-10-18T10:25:23.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.420+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c14f6f0 con 0x7f944c0afbf0 2023-10-18T10:25:23.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.420+0000 7f9433fff640 1 -- 172.21.15.125:0/3239052381 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c011790 con 0x7f944c0afbf0 2023-10-18T10:25:23.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.421+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944c066960 con 0x7f944c0afbf0 2023-10-18T10:25:23.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.422+0000 7f9433fff640 1 -- 172.21.15.125:0/3239052381 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f943c011b50 con 0x7f944c0afbf0 2023-10-18T10:25:23.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7f9433fff640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f943806af00 0x7f943806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7fcb1509f640 1 -- 172.21.15.125:0/1277602441 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800b830 con 0x7fcb100c9d00 2023-10-18T10:25:23.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7f9433fff640 1 -- 172.21.15.125:0/3239052381 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f943c089250 con 0x7f944c0afbf0 2023-10-18T10:25:23.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.422+0000 7ff49d7fa640 1 -- 172.21.15.125:0/2587100411 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff494088580 con 0x7ff4a0156260 2023-10-18T10:25:23.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7fcb18305640 1 -- 172.21.15.125:0/1277602441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb100c9d00 msgr2=0x7fcb1006b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7fcb18305640 1 --2- 172.21.15.125:0/1277602441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb100c9d00 0x7fcb1006b910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb08005fd0 tx=0x7fcb08030070 comp rx=0 tx=0).stop 2023-10-18T10:25:23.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7f944affd640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f943806af00 0x7f943806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7fcb18305640 1 -- 172.21.15.125:0/1277602441 shutdown_connections 2023-10-18T10:25:23.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7fcb18305640 1 --2- 172.21.15.125:0/1277602441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb100c9d00 0x7fcb1006b910 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7fcb18305640 1 --2- 172.21.15.125:0/1277602441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb10140660 0x7fcb100c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.423+0000 7fcb18305640 1 -- 172.21.15.125:0/1277602441 >> 172.21.15.125:0/1277602441 conn(0x7fcb10069930 msgr2=0x7fcb10069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.424+0000 7fcb18305640 1 -- 172.21.15.125:0/1277602441 shutdown_connections 2023-10-18T10:25:23.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.424+0000 7fcb18305640 1 -- 172.21.15.125:0/1277602441 wait complete. 2023-10-18T10:25:23.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.424+0000 7f944affd640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f943806af00 0x7f943806d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f944c0cdaf0 tx=0x7f9444009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.426+0000 7fcb18305640 1 Processor -- start 2023-10-18T10:25:23.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.426+0000 7fcb18305640 1 -- start start 2023-10-18T10:25:23.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.427+0000 7fcb18305640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb100c9d00 0x7fcb100a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.428+0000 7fcb18305640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1013d360 0x7fcb100b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.428+0000 7fcb18305640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb10140660 0x7fcb1013b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.428+0000 7fcb160a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb100c9d00 0x7fcb100a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.428+0000 7fcb160a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb100c9d00 0x7fcb100a6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:32830/0 (socket says 172.21.15.125:32830) 2023-10-18T10:25:23.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.428+0000 7f9433fff640 1 -- 172.21.15.125:0/3239052381 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f943c0540d0 con 0x7f944c0afbf0 2023-10-18T10:25:23.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.428+0000 7fcb168a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb10140660 0x7fcb1013b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.428+0000 7fcb168a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb10140660 0x7fcb1013b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49872/0 (socket says 172.21.15.125:49872) 2023-10-18T10:25:23.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.428+0000 7fcb158a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1013d360 0x7fcb100b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.429+0000 7fcb18305640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100b0740 con 0x7fcb10140660 2023-10-18T10:25:23.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.429+0000 7fcb18305640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100aef10 con 0x7fcb100c9d00 2023-10-18T10:25:23.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.429+0000 7fcb18305640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100af160 con 0x7fcb1013d360 2023-10-18T10:25:23.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.429+0000 7fcb158a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1013d360 0x7fcb100b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53156/0 (socket says 172.21.15.125:53156) 2023-10-18T10:25:23.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.429+0000 7fcb158a0640 1 -- 172.21.15.125:0/3200805958 learned_addr learned my addr 172.21.15.125:0/3200805958 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:23.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.430+0000 7fcb158a0640 1 -- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb100c9d00 msgr2=0x7fcb100a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.430+0000 7fcb158a0640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb100c9d00 0x7fcb100a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.430+0000 7fcb158a0640 1 -- 172.21.15.125:0/3200805958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb10140660 msgr2=0x7fcb1013b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.430+0000 7fcb158a0640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb10140660 0x7fcb1013b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.430+0000 7fcb158a0640 1 -- 172.21.15.125:0/3200805958 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0800dd10 con 0x7fcb1013d360 2023-10-18T10:25:23.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.430+0000 7fcb158a0640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1013d360 0x7fcb100b0200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e7f0 tx=0x7fcb0c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.430+0000 7fcb06ffd640 1 -- 172.21.15.125:0/3200805958 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c009580 con 0x7fcb1013d360 2023-10-18T10:25:23.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.430+0000 7fcb06ffd640 1 -- 172.21.15.125:0/3200805958 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb0c00cdb0 con 0x7fcb1013d360 2023-10-18T10:25:23.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.431+0000 7fcb06ffd640 1 -- 172.21.15.125:0/3200805958 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c009580 con 0x7fcb1013d360 2023-10-18T10:25:23.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.431+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb100af440 con 0x7fcb1013d360 2023-10-18T10:25:23.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.432+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb100afa00 con 0x7fcb1013d360 2023-10-18T10:25:23.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.433+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1006ccd0 con 0x7fcb1013d360 2023-10-18T10:25:23.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.434+0000 7fcb06ffd640 1 -- 172.21.15.125:0/3200805958 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb0c004030 con 0x7fcb1013d360 2023-10-18T10:25:23.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.435+0000 7fcb06ffd640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcaf406af00 0x7fcaf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:23.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.435+0000 7fcb06ffd640 1 -- 172.21.15.125:0/3200805958 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcb0c014070 con 0x7fcb1013d360 2023-10-18T10:25:23.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.435+0000 7fcb160a1640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcaf406af00 0x7fcaf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:23.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.436+0000 7fcb160a1640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcaf406af00 0x7fcaf406d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcb00004f60 tx=0x7fcb00007e30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:23.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.438+0000 7fcb06ffd640 1 -- 172.21.15.125:0/3200805958 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb0c054d90 con 0x7fcb1013d360 2023-10-18T10:25:23.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.804+0000 7f6af0b33640 1 -- 172.21.15.125:0/1265095994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6aec1599d0 con 0x7f6aec13d360 2023-10-18T10:25:23.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.804+0000 7fa0e8a9a640 1 -- 172.21.15.125:0/1899955103 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa0dc04d6c0 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.804+0000 7f6adaffd640 1 -- 172.21.15.125:0/1265095994 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6aec1599d0 con 0x7f6aec13d360 2023-10-18T10:25:23.807 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:23.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.805+0000 7fa0cffff640 1 -- 172.21.15.125:0/1899955103 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa0d004a020 con 0x7fa0dc0eacd0 2023-10-18T10:25:23.808 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:23.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.805+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff4800050c0 con 0x7ff4a0156260 2023-10-18T10:25:23.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.806+0000 7ff49d7fa640 1 -- 172.21.15.125:0/2587100411 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff494088580 con 0x7ff4a0156260 2023-10-18T10:25:23.810 INFO:teuthology.orchestra.run.smithi125.stdout:55834574856 2023-10-18T10:25:23.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.807+0000 7f6ad8ff9640 1 -- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6acc06af10 msgr2=0x7f6acc06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6acc06af10 0x7f6acc06d3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00a7c0 tx=0x7f6adc00c040 comp rx=0 tx=0).stop 2023-10-18T10:25:23.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 -- 172.21.15.125:0/1265095994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aec13d360 msgr2=0x7f6aec131580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aec13d360 0x7f6aec131580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400c780 tx=0x7f6ae400cc40 comp rx=0 tx=0).stop 2023-10-18T10:25:23.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 -- 172.21.15.125:0/1265095994 shutdown_connections 2023-10-18T10:25:23.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6acc06af10 0x7f6acc06d3d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aec140660 0x7f6aec1344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 -- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa0c806ae40 msgr2=0x7fa0c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6aec13d360 0x7f6aec131580 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa0c806ae40 0x7fa0c806d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc04c9e0 tx=0x7fa0d80079b0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 --2- 172.21.15.125:0/1265095994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6aec0c9ed0 0x7f6aec159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 -- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0eacd0 msgr2=0x7fa0dc04ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7f6ad8ff9640 1 -- 172.21.15.125:0/1265095994 >> 172.21.15.125:0/1265095994 conn(0x7f6aec069930 msgr2=0x7f6aec13ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0eacd0 0x7fa0dc04ca50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009980 tx=0x7fa0d0004530 comp rx=0 tx=0).stop 2023-10-18T10:25:23.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 -- 172.21.15.125:0/1899955103 shutdown_connections 2023-10-18T10:25:23.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa0c806ae40 0x7fa0c806d300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0069b0 0x7fa0dc0dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0eacd0 0x7fa0dc04ca50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 --2- 172.21.15.125:0/1899955103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc005e60 0x7fa0dc0e03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.808+0000 7fa0cdffb640 1 -- 172.21.15.125:0/1899955103 >> 172.21.15.125:0/1899955103 conn(0x7fa0dc009f80 msgr2=0x7fa0dc038e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.809+0000 7fa0cdffb640 1 -- 172.21.15.125:0/1899955103 shutdown_connections 2023-10-18T10:25:23.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.809+0000 7fa0cdffb640 1 -- 172.21.15.125:0/1899955103 wait complete. 2023-10-18T10:25:23.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.809+0000 7f6ad8ff9640 1 -- 172.21.15.125:0/1265095994 shutdown_connections 2023-10-18T10:25:23.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7f6ad8ff9640 1 -- 172.21.15.125:0/1265095994 wait complete. 2023-10-18T10:25:23.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff48c06aea0 msgr2=0x7ff48c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff48c06aea0 0x7ff48c06d360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00c9170 tx=0x7ff49000b260 comp rx=0 tx=0).stop 2023-10-18T10:25:23.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4a0156260 msgr2=0x7ff4a0159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4a0156260 0x7ff4a0159e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff49401d440 tx=0x7ff494001680 comp rx=0 tx=0).stop 2023-10-18T10:25:23.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 shutdown_connections 2023-10-18T10:25:23.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff48c06aea0 0x7ff48c06d360 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00c9170 tx=0x7ff49000b260 comp rx=0 tx=0).stop 2023-10-18T10:25:23.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4a015a3a0 0x7ff4a00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4a0156640 0x7ff4a00ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 --2- 172.21.15.125:0/2587100411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4a0156260 0x7ff4a0159e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.810+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 >> 172.21.15.125:0/2587100411 conn(0x7ff4a0069930 msgr2=0x7ff4a00ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.811+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 shutdown_connections 2023-10-18T10:25:23.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.811+0000 7ff4a70d7640 1 -- 172.21.15.125:0/2587100411 wait complete. 2023-10-18T10:25:23.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.820+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe8680057d0 con 0x7fe8840b1190 2023-10-18T10:25:23.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.820+0000 7fe87affd640 1 -- 172.21.15.125:0/2447234878 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe874042090 con 0x7fe8840b1190 2023-10-18T10:25:23.825 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:23.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.821+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff6b80cbed0 con 0x7ff6b80c9d00 2023-10-18T10:25:23.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.821+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86300050c0 con 0x7f86480d77c0 2023-10-18T10:25:23.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.821+0000 7ff69f7fe640 1 -- 172.21.15.125:0/2135734219 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff6b0058d10 con 0x7ff6b80c9d00 2023-10-18T10:25:23.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.821+0000 7f864d7fa640 1 -- 172.21.15.125:0/3001836662 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f863c079b30 con 0x7f86480d77c0 2023-10-18T10:25:23.827 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:23.828 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:23.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.822+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f944c04da70 con 0x7f944c0afbf0 2023-10-18T10:25:23.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.823+0000 7f9433fff640 1 -- 172.21.15.125:0/3239052381 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f943c0540d0 con 0x7f944c0afbf0 2023-10-18T10:25:23.829 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:23.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.824+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe86406af10 msgr2=0x7fe86406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.824+0000 7fe88884f640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe86406af10 0x7fe86406d3d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe88406f4b0 tx=0x7fe86c00a2d0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.824+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8840b1190 msgr2=0x7fe8840c2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.824+0000 7fe88884f640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8840b1190 0x7fe8840c2980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe874009780 tx=0x7fe874003820 comp rx=0 tx=0).stop 2023-10-18T10:25:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.824+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 shutdown_connections 2023-10-18T10:25:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.824+0000 7fe88884f640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe86406af10 0x7fe86406d3d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7fe88884f640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe88415f3c0 0x7fe884157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7fe88884f640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8840cb950 0x7fe8841570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7fe88884f640 1 --2- 172.21.15.125:0/2447234878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8840b1190 0x7fe8840c2980 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 >> 172.21.15.125:0/2447234878 conn(0x7fe8840b0670 msgr2=0x7fe8840ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f863406ae40 msgr2=0x7f863406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f863406ae40 0x7f863406d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f865004f250 tx=0x7f865008a040 comp rx=0 tx=0).stop 2023-10-18T10:25:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86480d77c0 msgr2=0x7f86480e2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86480d77c0 0x7f86480e2370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f863c0099a0 tx=0x7f863c02dc00 comp rx=0 tx=0).stop 2023-10-18T10:25:23.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 shutdown_connections 2023-10-18T10:25:23.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f863406ae40 0x7f863406d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86480e28b0 0x7f86480368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86480d77c0 0x7f86480e2370 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 --2- 172.21.15.125:0/3001836662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86400042d0 0x7f864817f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 >> 172.21.15.125:0/3001836662 conn(0x7f8648054ee0 msgr2=0x7f864804b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 shutdown_connections 2023-10-18T10:25:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7f8655c60640 1 -- 172.21.15.125:0/3001836662 wait complete. 2023-10-18T10:25:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff6a406ae90 msgr2=0x7ff6a406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7ff6bd626640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff6a406ae90 0x7ff6a406d350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8036000 tx=0x7ff6a8004730 comp rx=0 tx=0).stop 2023-10-18T10:25:23.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b80c9d00 msgr2=0x7ff6b812f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.825+0000 7ff6bd626640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b80c9d00 0x7ff6b812f300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000cbc0 tx=0x7ff6b000a430 comp rx=0 tx=0).stop 2023-10-18T10:25:23.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 shutdown_connections 2023-10-18T10:25:23.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7ff6bd626640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff6a406ae90 0x7ff6a406d350 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7ff6bd626640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b8140660 0x7ff6b8131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7ff6bd626640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b813d360 0x7ff6b8200af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7ff6bd626640 1 --2- 172.21.15.125:0/2135734219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b80c9d00 0x7ff6b812f300 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000cbc0 tx=0x7ff6b000a430 comp rx=0 tx=0).stop 2023-10-18T10:25:23.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 >> 172.21.15.125:0/2135734219 conn(0x7ff6b8069930 msgr2=0x7ff6b813c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 shutdown_connections 2023-10-18T10:25:23.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7fe88884f640 1 -- 172.21.15.125:0/2447234878 wait complete. 2023-10-18T10:25:23.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 shutdown_connections 2023-10-18T10:25:23.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7ff6bd626640 1 -- 172.21.15.125:0/2135734219 wait complete. 2023-10-18T10:25:23.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2023-10-18T10:25:23.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f943806af00 msgr2=0x7f943806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f943806af00 0x7f943806d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f944c0cdaf0 tx=0x7f9444009040 comp rx=0 tx=0).stop 2023-10-18T10:25:23.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f944c0afbf0 msgr2=0x7f944c0b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f944c0afbf0 0x7f944c0b2fe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c3c0 tx=0x7f943c00c7c0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 shutdown_connections 2023-10-18T10:25:23.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f943806af00 0x7f943806d3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f944c15af50 0x7f944c0c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f944c0cf120 0x7f944c0c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 --2- 172.21.15.125:0/3239052381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f944c0afbf0 0x7f944c0b2fe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.826+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 >> 172.21.15.125:0/3239052381 conn(0x7f944c099ee0 msgr2=0x7f944c15bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.827+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 shutdown_connections 2023-10-18T10:25:23.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.827+0000 7f9450a9a640 1 -- 172.21.15.125:0/3239052381 wait complete. 2023-10-18T10:25:23.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.827+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcb10067930 con 0x7fcb1013d360 2023-10-18T10:25:23.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.828+0000 7fcb06ffd640 1 -- 172.21.15.125:0/3200805958 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb0c0583b0 con 0x7fcb1013d360 2023-10-18T10:25:23.848 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:23.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2023-10-18T10:25:23.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-10-18T10:25:23.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcaf406af00 msgr2=0x7fcaf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcaf406af00 0x7fcaf406d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcb00004f60 tx=0x7fcb00007e30 comp rx=0 tx=0).stop 2023-10-18T10:25:23.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1013d360 msgr2=0x7fcb100b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:23.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1013d360 0x7fcb100b0200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e7f0 tx=0x7fcb0c00ecb0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 shutdown_connections 2023-10-18T10:25:23.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcaf406af00 0x7fcaf406d3c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb10140660 0x7fcb1013b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1013d360 0x7fcb100b0200 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 --2- 172.21.15.125:0/3200805958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb100c9d00 0x7fcb100a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:23.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.831+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 >> 172.21.15.125:0/3200805958 conn(0x7fcb10069930 msgr2=0x7fcb1006c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:23.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.832+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 shutdown_connections 2023-10-18T10:25:23.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:23.832+0000 7fcb18305640 1 -- 172.21.15.125:0/3200805958 wait complete. 2023-10-18T10:25:23.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-10-18T10:25:23.855 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2023-10-18T10:25:23.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2023-10-18T10:25:23.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2023-10-18T10:25:23.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2023-10-18T10:25:24.841 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-18T10:25:24.851 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-18T10:25:24.851 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T10:25:24.855 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-18T10:25:24.857 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-18T10:25:24.857 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-18T10:25:24.858 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-18T10:25:24.858 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-18T10:25:25.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.004+0000 7f100571b640 1 -- 172.21.15.125:0/1459812065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fe80047c0 msgr2=0x7f0fe8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.004+0000 7f100571b640 1 --2- 172.21.15.125:0/1459812065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fe80047c0 0x7f0fe8004c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0006e90 tx=0x7f0fe80057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.004+0000 7f1004f1a640 1 -- 172.21.15.125:0/1459812065 reap_dead start 2023-10-18T10:25:25.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.005+0000 7f100571b640 1 -- 172.21.15.125:0/1459812065 shutdown_connections 2023-10-18T10:25:25.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.005+0000 7f100571b640 1 -- 172.21.15.125:0/1459812065 >> 172.21.15.125:0/1459812065 conn(0x7f1000069720 msgr2=0x7f1000069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.005+0000 7f100571b640 1 -- 172.21.15.125:0/1459812065 shutdown_connections 2023-10-18T10:25:25.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.005+0000 7f100571b640 1 -- 172.21.15.125:0/1459812065 wait complete. 2023-10-18T10:25:25.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.007+0000 7f100571b640 1 Processor -- start 2023-10-18T10:25:25.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.007+0000 7f100571b640 1 -- start start 2023-10-18T10:25:25.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.008+0000 7f100571b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0fe80047c0 0x7f10000aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.008+0000 7f100571b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10001f5600 0x7f100014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.008+0000 7f100571b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f100014ee80 0x7f100014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f1004f1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f100014ee80 0x7f100014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f100571b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10001526f0 con 0x7f100014ee80 2023-10-18T10:25:25.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f1004f1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f100014ee80 0x7f100014f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49890/0 (socket says 172.21.15.125:49890) 2023-10-18T10:25:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f1004f1a640 1 -- 172.21.15.125:0/3117514528 learned_addr learned my addr 172.21.15.125:0/3117514528 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f0fff7fe640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10001f5600 0x7f100014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f0ffffff640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0fe80047c0 0x7f10000aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f100571b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1000151790 con 0x7f10001f5600 2023-10-18T10:25:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f100571b640 1 -- 172.21.15.125:0/3117514528 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1000151a70 con 0x7f0fe80047c0 2023-10-18T10:25:25.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f1004f1a640 1 -- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0fe80047c0 msgr2=0x7f10000aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f1004f1a640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0fe80047c0 0x7f10000aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f1004f1a640 1 -- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10001f5600 msgr2=0x7f100014e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f1004f1a640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10001f5600 0x7f100014e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.009+0000 7f1004f1a640 1 -- 172.21.15.125:0/3117514528 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000151d50 con 0x7f100014ee80 2023-10-18T10:25:25.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.010+0000 7f1004f1a640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f100014ee80 0x7f100014f3e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10000b06b0 tx=0x7f0ff8006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.011+0000 7f0ffcff9640 1 -- 172.21.15.125:0/3117514528 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8011900 con 0x7f100014ee80 2023-10-18T10:25:25.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.011+0000 7f100571b640 1 -- 172.21.15.125:0/3117514528 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1000152030 con 0x7f100014ee80 2023-10-18T10:25:25.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.011+0000 7f100571b640 1 -- 172.21.15.125:0/3117514528 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10001f5a80 con 0x7f100014ee80 2023-10-18T10:25:25.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.015+0000 7f0ffcff9640 1 -- 172.21.15.125:0/3117514528 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ff8011a60 con 0x7f100014ee80 2023-10-18T10:25:25.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.015+0000 7f0ffcff9640 1 -- 172.21.15.125:0/3117514528 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8011e70 con 0x7f100014ee80 2023-10-18T10:25:25.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.015+0000 7f0ffcff9640 1 -- 172.21.15.125:0/3117514528 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0ff801d460 con 0x7f100014ee80 2023-10-18T10:25:25.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.016+0000 7f0ffcff9640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0fec06afc0 0x7f0fec06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.016+0000 7f0ffcff9640 1 -- 172.21.15.125:0/3117514528 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0ff808a670 con 0x7f100014ee80 2023-10-18T10:25:25.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.017+0000 7f0ffffff640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0fec06afc0 0x7f0fec06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.018+0000 7f100571b640 1 -- 172.21.15.125:0/3117514528 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe0005250 con 0x7f100014ee80 2023-10-18T10:25:25.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.027+0000 7f0ffcff9640 1 -- 172.21.15.125:0/3117514528 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ff8055380 con 0x7f100014ee80 2023-10-18T10:25:25.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.030+0000 7f0ffffff640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0fec06afc0 0x7f0fec06d480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009460 tx=0x7f0ff4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.059+0000 7fd0d7fff640 1 -- 172.21.15.125:0/1618380242 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc02dcc0 con 0x7fd0cc0047c0 2023-10-18T10:25:25.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.060+0000 7fd0e6059640 1 -- 172.21.15.125:0/1618380242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0cc0047c0 msgr2=0x7fd0cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.060+0000 7fd0e6059640 1 --2- 172.21.15.125:0/1618380242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0cc0047c0 0x7fd0cc004c40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009870 tx=0x7fd0cc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.060+0000 7fd0e5858640 1 -- 172.21.15.125:0/1618380242 reap_dead start 2023-10-18T10:25:25.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.060+0000 7fd0e6059640 1 -- 172.21.15.125:0/1618380242 shutdown_connections 2023-10-18T10:25:25.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.061+0000 7fd0e6059640 1 -- 172.21.15.125:0/1618380242 >> 172.21.15.125:0/1618380242 conn(0x7fd0e00ae460 msgr2=0x7fd0e00ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.062+0000 7fd0e6059640 1 -- 172.21.15.125:0/1618380242 shutdown_connections 2023-10-18T10:25:25.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.062+0000 7f90bad14640 1 -- 172.21.15.125:0/3991588306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90a00047c0 msgr2=0x7f90a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.062+0000 7f90bad14640 1 --2- 172.21.15.125:0/3991588306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90a00047c0 0x7f90a0004c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90b404ece0 tx=0x7f90a00057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.062+0000 7f90bad14640 1 -- 172.21.15.125:0/3991588306 shutdown_connections 2023-10-18T10:25:25.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.062+0000 7f90bad14640 1 --2- 172.21.15.125:0/3991588306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90a00047c0 0x7f90a0004c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.062+0000 7f90bad14640 1 -- 172.21.15.125:0/3991588306 >> 172.21.15.125:0/3991588306 conn(0x7f90ac01ef10 msgr2=0x7f90ac055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.062+0000 7fd0e6059640 1 -- 172.21.15.125:0/1618380242 wait complete. 2023-10-18T10:25:25.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.063+0000 7f90bad14640 1 -- 172.21.15.125:0/3991588306 shutdown_connections 2023-10-18T10:25:25.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.064+0000 7f90bad14640 1 -- 172.21.15.125:0/3991588306 wait complete. 2023-10-18T10:25:25.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.064+0000 7fd0e6059640 1 Processor -- start 2023-10-18T10:25:25.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.064+0000 7f07ff59e640 1 -- 172.21.15.125:0/2205625963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07e8004340 msgr2=0x7f07e8004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.065+0000 7f07fd59a640 1 -- 172.21.15.125:0/2205625963 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000abb00 con 0x7f07e8004340 2023-10-18T10:25:25.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.065+0000 7f07ff59e640 1 --2- 172.21.15.125:0/2205625963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07e8004340 0x7f07e8004820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0800066af0 tx=0x7f07e80057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.065+0000 7f07fed9d640 1 -- 172.21.15.125:0/2205625963 reap_dead start 2023-10-18T10:25:25.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.064+0000 7fdb26bf5640 1 -- 172.21.15.125:0/4167792286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb201550a0 msgr2=0x7fdb200aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.064+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/4167792286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb201550a0 0x7fdb200aa8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009860 tx=0x7fdb1402bdd0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.065+0000 7fdb26bf5640 1 -- 172.21.15.125:0/4167792286 shutdown_connections 2023-10-18T10:25:25.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.065+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/4167792286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb201550a0 0x7fdb200aa8c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.065+0000 7fdb26bf5640 1 -- 172.21.15.125:0/4167792286 >> 172.21.15.125:0/4167792286 conn(0x7fdb200ae3f0 msgr2=0x7fdb200ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.065+0000 7f90bad14640 1 Processor -- start 2023-10-18T10:25:25.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.064+0000 7fd0e6059640 1 -- start start 2023-10-18T10:25:25.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fd0e6059640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0cc0047c0 0x7fd0e01f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fd0e6059640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0e0154000 0x7fd0e0156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fd0e6059640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e0154480 0x7fd0e0156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fdb26bf5640 1 -- 172.21.15.125:0/4167792286 shutdown_connections 2023-10-18T10:25:25.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fdb26bf5640 1 -- 172.21.15.125:0/4167792286 wait complete. 2023-10-18T10:25:25.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f07ff59e640 1 -- 172.21.15.125:0/2205625963 shutdown_connections 2023-10-18T10:25:25.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f07ff59e640 1 -- 172.21.15.125:0/2205625963 >> 172.21.15.125:0/2205625963 conn(0x7f07f80559b0 msgr2=0x7f07f8055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f07ff59e640 1 -- 172.21.15.125:0/2205625963 shutdown_connections 2023-10-18T10:25:25.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fd0e5858640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e0154480 0x7fd0e0156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fd0e5858640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e0154480 0x7fd0e0156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49964/0 (socket says 172.21.15.125:49964) 2023-10-18T10:25:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.065+0000 7f431a7fc640 1 -- 172.21.15.125:0/3410428241 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314034d00 con 0x7f431c15c0c0 2023-10-18T10:25:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f4320c47640 1 -- 172.21.15.125:0/3410428241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c15c0c0 msgr2=0x7f431c070850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f4320c47640 1 --2- 172.21.15.125:0/3410428241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c15c0c0 0x7f431c070850 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4314005fd0 tx=0x7f431402c2d0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fd0e4856640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0e0154000 0x7fd0e0156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f4320c47640 1 -- 172.21.15.125:0/3410428241 shutdown_connections 2023-10-18T10:25:25.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f4320c47640 1 --2- 172.21.15.125:0/3410428241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c15c0c0 0x7f431c070850 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f4320c47640 1 --2- 172.21.15.125:0/3410428241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f431c1411e0 0x7f431c06d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f4320c47640 1 -- 172.21.15.125:0/3410428241 >> 172.21.15.125:0/3410428241 conn(0x7f431c0ae5d0 msgr2=0x7f431c0ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7fd0e4856640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0e0154000 0x7fd0e0156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:32904/0 (socket says 172.21.15.125:32904) 2023-10-18T10:25:25.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f07ff59e640 1 -- 172.21.15.125:0/2205625963 wait complete. 2023-10-18T10:25:25.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f4320c47640 1 -- 172.21.15.125:0/3410428241 shutdown_connections 2023-10-18T10:25:25.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f90bad14640 1 -- start start 2023-10-18T10:25:25.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90bad14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90a00047c0 0x7f90ac0df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90bad14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90ac052e90 0x7f90ac033fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90bad14640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90ac034510 0x7f90ac04eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90bad14640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac051df0 con 0x7f90ac034510 2023-10-18T10:25:25.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90bad14640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac0520d0 con 0x7f90ac052e90 2023-10-18T10:25:25.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90bad14640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac0523b0 con 0x7f90a00047c0 2023-10-18T10:25:25.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90b9511640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90ac052e90 0x7f90ac033fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7fdb26bf5640 1 Processor -- start 2023-10-18T10:25:25.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90b9511640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90ac052e90 0x7f90ac033fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:32910/0 (socket says 172.21.15.125:32910) 2023-10-18T10:25:25.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90b9511640 1 -- 172.21.15.125:0/2008695496 learned_addr learned my addr 172.21.15.125:0/2008695496 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7f90ba513640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90ac034510 0x7f90ac04eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.067+0000 7fdb26bf5640 1 -- start start 2023-10-18T10:25:25.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.066+0000 7f4320c47640 1 -- 172.21.15.125:0/3410428241 wait complete. 2023-10-18T10:25:25.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7fd0e5858640 1 -- 172.21.15.125:0/1756368349 learned_addr learned my addr 172.21.15.125:0/1756368349 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0154ae0 con 0x7fd0e0154480 2023-10-18T10:25:25.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e0154dc0 con 0x7fd0e0154000 2023-10-18T10:25:25.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e01550a0 con 0x7fd0cc0047c0 2023-10-18T10:25:25.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7fd0e5057640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0cc0047c0 0x7fd0e01f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f06bf59e640 1 -- 172.21.15.125:0/3750582058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06a80039c0 msgr2=0x7f06a8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f06bf59e640 1 --2- 172.21.15.125:0/3750582058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06a80039c0 0x7f06a8003dc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009850 tx=0x7f06a80057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f90ba513640 1 -- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90a00047c0 msgr2=0x7f90ac0df0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:25:25.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f07ff59e640 1 Processor -- start 2023-10-18T10:25:25.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f07ff59e640 1 -- start start 2023-10-18T10:25:25.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f90ba513640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90a00047c0 0x7f90ac0df0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f90ba513640 1 -- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90ac052e90 msgr2=0x7f90ac033fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f4320c47640 1 Processor -- start 2023-10-18T10:25:25.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f4320c47640 1 -- start start 2023-10-18T10:25:25.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f06bf59e640 1 -- 172.21.15.125:0/3750582058 shutdown_connections 2023-10-18T10:25:25.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f06bf59e640 1 --2- 172.21.15.125:0/3750582058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06a80039c0 0x7f06a8003dc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f06bf59e640 1 -- 172.21.15.125:0/3750582058 >> 172.21.15.125:0/3750582058 conn(0x7f06b801ef10 msgr2=0x7f06b8055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.068+0000 7f06bf59e640 1 -- 172.21.15.125:0/3750582058 shutdown_connections 2023-10-18T10:25:25.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f06bf59e640 1 -- 172.21.15.125:0/3750582058 wait complete. 2023-10-18T10:25:25.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fdb26bf5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb200aae00 0x7fdb201f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07ff59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07e8004340 0x7f07f804f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07ff59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f800c1f0 0x7f07f800c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07ff59e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80e7ef0 0x7f07f80e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07ff59e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81865d0 con 0x7f07e8004340 2023-10-18T10:25:25.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07ff59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f800cb90 con 0x7f07f800c1f0 2023-10-18T10:25:25.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07ff59e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f800ce40 con 0x7f07f80e7ef0 2023-10-18T10:25:25.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fdb26bf5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb201550a0 0x7fdb201f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fd0e5858640 1 -- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0cc0047c0 msgr2=0x7fd0e01f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fd0e5858640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0cc0047c0 0x7fd0e01f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fd0e5858640 1 -- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0e0154000 msgr2=0x7fd0e0156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fd0e5858640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0e0154000 0x7fd0e0156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fd0e5858640 1 -- 172.21.15.125:0/1756368349 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0155380 con 0x7fd0e0154480 2023-10-18T10:25:25.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07fed9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80e7ef0 0x7f07f80e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07fed9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80e7ef0 0x7f07f80e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53330/0 (socket says 172.21.15.125:53330) 2023-10-18T10:25:25.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f07fed9d640 1 -- 172.21.15.125:0/1795019145 learned_addr learned my addr 172.21.15.125:0/1795019145 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fdb24991640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb200aae00 0x7fdb201f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fdb24991640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb200aae00 0x7fdb201f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53314/0 (socket says 172.21.15.125:53314) 2023-10-18T10:25:25.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fdb1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb201550a0 0x7fdb201f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fdb1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb201550a0 0x7fdb201f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:32920/0 (socket says 172.21.15.125:32920) 2023-10-18T10:25:25.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f90ba513640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90ac052e90 0x7f90ac033fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f90ba513640 1 -- 172.21.15.125:0/2008695496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac052690 con 0x7f90ac034510 2023-10-18T10:25:25.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7f90b9511640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90ac052e90 0x7f90ac033fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:25:25.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb24991640 1 -- 172.21.15.125:0/3947478912 learned_addr learned my addr 172.21.15.125:0/3947478912 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f07fed9d640 1 -- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f800c1f0 msgr2=0x7f07f800c650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T10:25:25.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb2015b950 0x7fdb200c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.069+0000 7fd0e5858640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e0154480 0x7fd0e0156f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0057a0 tx=0x7fd0d800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb201f3b40 con 0x7fdb2015b950 2023-10-18T10:25:25.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb201f2ec0 con 0x7fdb201550a0 2023-10-18T10:25:25.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb201f31a0 con 0x7fdb200aae00 2023-10-18T10:25:25.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f07fdd9b640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f800c1f0 0x7f07f800c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fd0e5057640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0cc0047c0 0x7fd0e01f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:25:25.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f07fed9d640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f800c1f0 0x7f07f800c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fd0d5ffb640 1 -- 172.21.15.125:0/1756368349 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d800a5b0 con 0x7fd0e0154480 2023-10-18T10:25:25.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f07fed9d640 1 -- 172.21.15.125:0/1795019145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07e8004340 msgr2=0x7f07f804f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f07fed9d640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07e8004340 0x7f07f804f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f07fed9d640 1 -- 172.21.15.125:0/1795019145 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f800d120 con 0x7f07f80e7ef0 2023-10-18T10:25:25.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f4320c47640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c0b09e0 0x7f431c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f4320c47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f431c1411e0 0x7f431c13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb25192640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb2015b950 0x7fdb200c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f90ba513640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90ac034510 0x7f90ac04eee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0375f0 tx=0x7f90a4011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f4320c47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f431c15c0c0 0x7f431c1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e0155660 con 0x7fd0e0154480 2023-10-18T10:25:25.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e0155e60 con 0x7fd0e0154480 2023-10-18T10:25:25.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb24991640 1 -- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb201550a0 msgr2=0x7fdb201f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f431affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f431c1411e0 0x7f431c13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f431affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f431c1411e0 0x7f431c13f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53344/0 (socket says 172.21.15.125:53344) 2023-10-18T10:25:25.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f431affd640 1 -- 172.21.15.125:0/2195461214 learned_addr learned my addr 172.21.15.125:0/2195461214 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb24991640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb201550a0 0x7fdb201f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f431bfff640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f431c15c0c0 0x7f431c1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f07fed9d640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80e7ef0 0x7f07f80e83a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f080004ece0 tx=0x7f07f000c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f4320c47640 1 -- 172.21.15.125:0/2195461214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1f4030 con 0x7f431c0b09e0 2023-10-18T10:25:25.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f4320c47640 1 -- 172.21.15.125:0/2195461214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1f3660 con 0x7f431c15c0c0 2023-10-18T10:25:25.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f4320c47640 1 -- 172.21.15.125:0/2195461214 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1f2a50 con 0x7f431c1411e0 2023-10-18T10:25:25.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb24991640 1 -- 172.21.15.125:0/3947478912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb2015b950 msgr2=0x7fdb200c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7f90aa7fc640 1 -- 172.21.15.125:0/2008695496 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a400faf0 con 0x7f90ac034510 2023-10-18T10:25:25.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f90aa7fc640 1 -- 172.21.15.125:0/2008695496 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f90a400fd40 con 0x7f90ac034510 2023-10-18T10:25:25.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f90aa7fc640 1 -- 172.21.15.125:0/2008695496 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40126b0 con 0x7f90ac034510 2023-10-18T10:25:25.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f431b7fe640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c0b09e0 0x7f431c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb24991640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb2015b950 0x7fdb200c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.070+0000 7fdb24991640 1 -- 172.21.15.125:0/3947478912 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb14009490 con 0x7fdb200aae00 2023-10-18T10:25:25.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f07eeffd640 1 -- 172.21.15.125:0/1795019145 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0004220 con 0x7f07f80e7ef0 2023-10-18T10:25:25.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac052970 con 0x7f90ac034510 2023-10-18T10:25:25.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac033560 con 0x7f90ac034510 2023-10-18T10:25:25.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f07eeffd640 1 -- 172.21.15.125:0/1795019145 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07f00043a0 con 0x7f07f80e7ef0 2023-10-18T10:25:25.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f06bf59e640 1 Processor -- start 2023-10-18T10:25:25.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f06bf59e640 1 -- start start 2023-10-18T10:25:25.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f07ff59e640 1 -- 172.21.15.125:0/1795019145 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f800d400 con 0x7f07f80e7ef0 2023-10-18T10:25:25.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f07ff59e640 1 -- 172.21.15.125:0/1795019145 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f800d9c0 con 0x7f07f80e7ef0 2023-10-18T10:25:25.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f197356e640 1 -- 172.21.15.125:0/795263295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19580039a0 msgr2=0x7f1958003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f197356e640 1 --2- 172.21.15.125:0/795263295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19580039a0 0x7f1958003da0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f196400c610 tx=0x7f19580057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f1971b0b640 1 -- 172.21.15.125:0/795263295 reap_dead start 2023-10-18T10:25:25.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7fd0d5ffb640 1 -- 172.21.15.125:0/1756368349 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0d800a820 con 0x7fd0e0154480 2023-10-18T10:25:25.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a80039c0 0x7f06b804bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bf59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b80ccde0 0x7f06b8049810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bf59e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06b8178ec0 0x7f06b8179370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bdd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b80ccde0 0x7f06b8049810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bf59e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b8049d50 con 0x7f06b80ccde0 2023-10-18T10:25:25.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bdd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b80ccde0 0x7f06b8049810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49994/0 (socket says 172.21.15.125:49994) 2023-10-18T10:25:25.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bf59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80d92c0 con 0x7f06a80039c0 2023-10-18T10:25:25.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bdd9b640 1 -- 172.21.15.125:0/2286562312 learned_addr learned my addr 172.21.15.125:0/2286562312 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b804c3c0 con 0x7f06b8178ec0 2023-10-18T10:25:25.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f197356e640 1 -- 172.21.15.125:0/795263295 shutdown_connections 2023-10-18T10:25:25.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7fd0d5ffb640 1 -- 172.21.15.125:0/1756368349 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d800ac30 con 0x7fd0e0154480 2023-10-18T10:25:25.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06bed9d640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06b8178ec0 0x7f06b8179370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f06be59c640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a80039c0 0x7f06b804bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f197356e640 1 -- 172.21.15.125:0/795263295 >> 172.21.15.125:0/795263295 conn(0x7f196c0ae3f0 msgr2=0x7f196c0ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f07eeffd640 1 -- 172.21.15.125:0/1795019145 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0007330 con 0x7f07f80e7ef0 2023-10-18T10:25:25.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f197356e640 1 -- 172.21.15.125:0/795263295 shutdown_connections 2023-10-18T10:25:25.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7f90aa7fc640 1 -- 172.21.15.125:0/2008695496 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f90a4012810 con 0x7f90ac034510 2023-10-18T10:25:25.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7f90aa7fc640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f909806af10 0x7f909806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7fdb24991640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb200aae00 0x7fdb201f2550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009980 tx=0x7fdb140038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7fdb1d7fa640 1 -- 172.21.15.125:0/3947478912 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14003ea0 con 0x7fdb200aae00 2023-10-18T10:25:25.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7fdb1d7fa640 1 -- 172.21.15.125:0/3947478912 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb14034530 con 0x7fdb200aae00 2023-10-18T10:25:25.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7fdb1d7fa640 1 -- 172.21.15.125:0/3947478912 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14034920 con 0x7fdb200aae00 2023-10-18T10:25:25.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f431b7fe640 1 -- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f431c1411e0 msgr2=0x7f431c13f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f431b7fe640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f431c1411e0 0x7f431c13f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f431b7fe640 1 -- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f431c15c0c0 msgr2=0x7f431c1f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f431b7fe640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f431c15c0c0 0x7f431c1f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.071+0000 7f431b7fe640 1 -- 172.21.15.125:0/2195461214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009900 con 0x7f431c0b09e0 2023-10-18T10:25:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.072+0000 7f431b7fe640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c0b09e0 0x7f431c15a0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c970 tx=0x7f430c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f4303fff640 1 -- 172.21.15.125:0/2195461214 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c007bb0 con 0x7f431c0b09e0 2023-10-18T10:25:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f4320c47640 1 -- 172.21.15.125:0/2195461214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c1f2d30 con 0x7f431c0b09e0 2023-10-18T10:25:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f4320c47640 1 -- 172.21.15.125:0/2195461214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1f32f0 con 0x7f431c0b09e0 2023-10-18T10:25:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7f4303fff640 1 -- 172.21.15.125:0/2195461214 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f430c007e00 con 0x7f431c0b09e0 2023-10-18T10:25:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7f4303fff640 1 -- 172.21.15.125:0/2195461214 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c007bb0 con 0x7f431c0b09e0 2023-10-18T10:25:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f06bed9d640 1 -- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a80039c0 msgr2=0x7f06b804bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f06bed9d640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a80039c0 0x7f06b804bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f06bed9d640 1 -- 172.21.15.125:0/2286562312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b80ccde0 msgr2=0x7f06b8049810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f06bed9d640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b80ccde0 0x7f06b8049810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f06bed9d640 1 -- 172.21.15.125:0/2286562312 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b804c6a0 con 0x7f06b8178ec0 2023-10-18T10:25:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7fd0d5ffb640 1 -- 172.21.15.125:0/1756368349 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd0d800ad90 con 0x7fd0e0154480 2023-10-18T10:25:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e004da70 con 0x7fd0e0154480 2023-10-18T10:25:25.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7fd0d5ffb640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd0c406af10 0x7fd0c406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7fd0d5ffb640 1 -- 172.21.15.125:0/1756368349 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd0d8089290 con 0x7fd0e0154480 2023-10-18T10:25:25.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7f06bed9d640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06b8178ec0 0x7f06b8179370 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06c00685a0 tx=0x7f06c00694b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.073+0000 7f197356e640 1 -- 172.21.15.125:0/795263295 wait complete. 2023-10-18T10:25:25.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f90b9d12640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f909806af10 0x7f909806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7f197356e640 1 Processor -- start 2023-10-18T10:25:25.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac0057b0 con 0x7f90ac034510 2023-10-18T10:25:25.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f197356e640 1 -- start start 2023-10-18T10:25:25.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f07eeffd640 1 -- 172.21.15.125:0/1795019145 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f07f0007510 con 0x7f07f80e7ef0 2023-10-18T10:25:25.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f90aa7fc640 1 -- 172.21.15.125:0/2008695496 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f90a408a530 con 0x7f90ac034510 2023-10-18T10:25:25.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f4301ffb640 1 -- 172.21.15.125:0/2195461214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42fc005250 con 0x7f431c0b09e0 2023-10-18T10:25:25.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f07ecff9640 1 -- 172.21.15.125:0/1795019145 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e4005250 con 0x7f07f80e7ef0 2023-10-18T10:25:25.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f06aeffd640 1 -- 172.21.15.125:0/2286562312 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c008d070 con 0x7f06b8178ec0 2023-10-18T10:25:25.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f06aeffd640 1 -- 172.21.15.125:0/2286562312 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f06c0066d60 con 0x7f06b8178ec0 2023-10-18T10:25:25.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f06aeffd640 1 -- 172.21.15.125:0/2286562312 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0071fe0 con 0x7f06b8178ec0 2023-10-18T10:25:25.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f90b9d12640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f909806af10 0x7f909806d3d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f90b0001050 tx=0x7f90b0001e10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b80d8c70 con 0x7f06b8178ec0 2023-10-18T10:25:25.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f197356e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f196c0cb3a0 0x7f196c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f07eeffd640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07e006af40 0x7f07e006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f07eeffd640 1 -- 172.21.15.125:0/1795019145 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f07f0088b50 con 0x7f07f80e7ef0 2023-10-18T10:25:25.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f07fe59c640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07e006af40 0x7f07e006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.075+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b817ced0 con 0x7f06b8178ec0 2023-10-18T10:25:25.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f197356e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f196c206e80 0x7f196c202be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f197356e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f196c06f6f0 0x7f196c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f197130a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f196c0cb3a0 0x7f196c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f07fe59c640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07e006af40 0x7f07e006d400 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f07f4005d00 tx=0x7f07f4005c50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb201f3480 con 0x7fdb200aae00 2023-10-18T10:25:25.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.074+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb20153870 con 0x7fdb200aae00 2023-10-18T10:25:25.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb201f3c80 con 0x7fdb200aae00 2023-10-18T10:25:25.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f197130a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f196c0cb3a0 0x7f196c0cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53364/0 (socket says 172.21.15.125:53364) 2023-10-18T10:25:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f1971b0b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f196c06f6f0 0x7f196c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f1971b0b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f196c06f6f0 0x7f196c06fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50004/0 (socket says 172.21.15.125:50004) 2023-10-18T10:25:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f1970b09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f196c206e80 0x7f196c202be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.076+0000 7f1970b09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f196c206e80 0x7f196c202be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:32934/0 (socket says 172.21.15.125:32934) 2023-10-18T10:25:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197130a640 1 -- 172.21.15.125:0/4074944349 learned_addr learned my addr 172.21.15.125:0/4074944349 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0cbcc0 con 0x7f196c06f6f0 2023-10-18T10:25:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c203550 con 0x7f196c206e80 2023-10-18T10:25:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c203830 con 0x7f196c0cb3a0 2023-10-18T10:25:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b80d9430 con 0x7f06b8178ec0 2023-10-18T10:25:25.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7fd0e5057640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd0c406af10 0x7fd0c406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197130a640 1 -- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f196c206e80 msgr2=0x7f196c202be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197130a640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f196c206e80 0x7f196c202be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197130a640 1 -- 172.21.15.125:0/4074944349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f196c06f6f0 msgr2=0x7f196c06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197130a640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f196c06f6f0 0x7f196c06fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197130a640 1 -- 172.21.15.125:0/4074944349 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c203b10 con 0x7f196c0cb3a0 2023-10-18T10:25:25.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f1971b0b640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f196c06f6f0 0x7f196c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:25:25.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7f197130a640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f196c0cb3a0 0x7f196c0cb780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f196402ccc0 tx=0x7f19640017b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.077+0000 7fd0e5057640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd0c406af10 0x7fd0c406d3d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd0e015ebe0 tx=0x7fd0d0006040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.078+0000 7fdb1d7fa640 1 -- 172.21.15.125:0/3947478912 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdb1403b040 con 0x7fdb200aae00 2023-10-18T10:25:25.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.078+0000 7fdb1d7fa640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdb0406af40 0x7fdb0406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.078+0000 7fdb1d7fa640 1 -- 172.21.15.125:0/3947478912 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdb140abf00 con 0x7fdb200aae00 2023-10-18T10:25:25.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.079+0000 7fdb1ffff640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdb0406af40 0x7fdb0406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.079+0000 7fdb1ffff640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdb0406af40 0x7fdb0406d400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdb200c7870 tx=0x7fdb0c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.080+0000 7f07eeffd640 1 -- 172.21.15.125:0/1795019145 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07f0053860 con 0x7f07f80e7ef0 2023-10-18T10:25:25.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.081+0000 7f1961ffb640 1 -- 172.21.15.125:0/4074944349 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964003e60 con 0x7f196c0cb3a0 2023-10-18T10:25:25.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.081+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c14df60 con 0x7f196c0cb3a0 2023-10-18T10:25:25.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.081+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c203120 con 0x7f196c0cb3a0 2023-10-18T10:25:25.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.082+0000 7f06aeffd640 1 -- 172.21.15.125:0/2286562312 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f06c0072140 con 0x7f06b8178ec0 2023-10-18T10:25:25.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.082+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c14ddc0 con 0x7f196c0cb3a0 2023-10-18T10:25:25.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.082+0000 7f06aeffd640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f06a006aec0 0x7f06a006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.083+0000 7f06be59c640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f06a006aec0 0x7f06a006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.083+0000 7f06be59c640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f06a006aec0 0x7f06a006d380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06b4036000 tx=0x7f06b4031040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.083+0000 7fd0d5ffb640 1 -- 172.21.15.125:0/1756368349 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0d8054170 con 0x7fd0e0154480 2023-10-18T10:25:25.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.085+0000 7f06aeffd640 1 -- 172.21.15.125:0/2286562312 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f06c00d0420 con 0x7f06b8178ec0 2023-10-18T10:25:25.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.085+0000 7fdb1d7fa640 1 -- 172.21.15.125:0/3947478912 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb14076e50 con 0x7fdb200aae00 2023-10-18T10:25:25.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.085+0000 7f1961ffb640 1 -- 172.21.15.125:0/4074944349 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1964006070 con 0x7f196c0cb3a0 2023-10-18T10:25:25.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.085+0000 7f1961ffb640 1 -- 172.21.15.125:0/4074944349 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964006420 con 0x7f196c0cb3a0 2023-10-18T10:25:25.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.085+0000 7f4303fff640 1 -- 172.21.15.125:0/2195461214 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f430c005020 con 0x7f431c0b09e0 2023-10-18T10:25:25.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.086+0000 7f4303fff640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4308073710 0x7f4308075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.086+0000 7f431affd640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4308073710 0x7f4308075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.087+0000 7f431affd640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4308073710 0x7f4308075bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f431c0cb980 tx=0x7f4314004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.090+0000 7f06aeffd640 1 -- 172.21.15.125:0/2286562312 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06c00cc060 con 0x7f06b8178ec0 2023-10-18T10:25:25.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.091+0000 7f1961ffb640 1 -- 172.21.15.125:0/4074944349 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1964031030 con 0x7f196c0cb3a0 2023-10-18T10:25:25.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.092+0000 7f1961ffb640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f195006ae40 0x7f195006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.092+0000 7f1961ffb640 1 -- 172.21.15.125:0/4074944349 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f19640ab320 con 0x7f196c0cb3a0 2023-10-18T10:25:25.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.093+0000 7f1970b09640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f195006ae40 0x7f195006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.093+0000 7f4303fff640 1 -- 172.21.15.125:0/2195461214 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f430c058f50 con 0x7f431c0b09e0 2023-10-18T10:25:25.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.093+0000 7f90aa7fc640 1 -- 172.21.15.125:0/2008695496 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f90a4055380 con 0x7f90ac034510 2023-10-18T10:25:25.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.094+0000 7f1970b09640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f195006ae40 0x7f195006d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f196c207280 tx=0x7f195c005d80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.095+0000 7f1961ffb640 1 -- 172.21.15.125:0/4074944349 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1964076170 con 0x7f196c0cb3a0 2023-10-18T10:25:25.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.102+0000 7f4303fff640 1 -- 172.21.15.125:0/2195461214 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f430c054b90 con 0x7f431c0b09e0 2023-10-18T10:25:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.414+0000 7f100571b640 1 -- 172.21.15.125:0/3117514528 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0fe0005500 con 0x7f100014ee80 2023-10-18T10:25:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.415+0000 7f0ffcff9640 1 -- 172.21.15.125:0/3117514528 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ff80589a0 con 0x7f100014ee80 2023-10-18T10:25:25.417 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:25.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.417+0000 7f0fe67fc640 1 -- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0fec06afc0 msgr2=0x7f0fec06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0fec06afc0 0x7f0fec06d480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009460 tx=0x7f0ff4005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:25.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 -- 172.21.15.125:0/3117514528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f100014ee80 msgr2=0x7f100014f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f100014ee80 0x7f100014f3e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10000b06b0 tx=0x7f0ff8006430 comp rx=0 tx=0).stop 2023-10-18T10:25:25.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 -- 172.21.15.125:0/3117514528 shutdown_connections 2023-10-18T10:25:25.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0fec06afc0 0x7f0fec06d480 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009460 tx=0x7f0ff4005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:25.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10001f5600 0x7f100014e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f100014ee80 0x7f100014f3e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 --2- 172.21.15.125:0/3117514528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0fe80047c0 0x7f10000aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.418+0000 7f0fe67fc640 1 -- 172.21.15.125:0/3117514528 >> 172.21.15.125:0/3117514528 conn(0x7f1000069720 msgr2=0x7f100015df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.419+0000 7f0fe67fc640 1 -- 172.21.15.125:0/3117514528 shutdown_connections 2023-10-18T10:25:25.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.419+0000 7f0fe67fc640 1 -- 172.21.15.125:0/3117514528 wait complete. 2023-10-18T10:25:25.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2023-10-18T10:25:25.437 DEBUG:teuthology.parallel:result is None 2023-10-18T10:25:25.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.463+0000 7f4301ffb640 1 -- 172.21.15.125:0/2195461214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f42fc0050c0 con 0x7f431c0b09e0 2023-10-18T10:25:25.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.464+0000 7f4303fff640 1 -- 172.21.15.125:0/2195461214 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f430c01d030 con 0x7f431c0b09e0 2023-10-18T10:25:25.466 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:25.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 -- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4308073710 msgr2=0x7f4308075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4308073710 0x7f4308075bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f431c0cb980 tx=0x7f4314004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 -- 172.21.15.125:0/2195461214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c0b09e0 msgr2=0x7f431c15a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c0b09e0 0x7f431c15a0c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c970 tx=0x7f430c00ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:25.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 -- 172.21.15.125:0/2195461214 shutdown_connections 2023-10-18T10:25:25.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4308073710 0x7f4308075bd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f431c15c0c0 0x7f431c1f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f431c1411e0 0x7f431c13f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 --2- 172.21.15.125:0/2195461214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f431c0b09e0 0x7f431c15a0c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 -- 172.21.15.125:0/2195461214 >> 172.21.15.125:0/2195461214 conn(0x7f431c0ae5d0 msgr2=0x7f431c0aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 -- 172.21.15.125:0/2195461214 shutdown_connections 2023-10-18T10:25:25.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.466+0000 7f4301ffb640 1 -- 172.21.15.125:0/2195461214 wait complete. 2023-10-18T10:25:25.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.474+0000 7f07ecff9640 1 -- 172.21.15.125:0/1795019145 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f07e4005500 con 0x7f07f80e7ef0 2023-10-18T10:25:25.477 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:25.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.475+0000 7f07eeffd640 1 -- 172.21.15.125:0/1795019145 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f07f0056e80 con 0x7f07f80e7ef0 2023-10-18T10:25:25.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2023-10-18T10:25:25.479 DEBUG:teuthology.parallel:result is None 2023-10-18T10:25:25.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 -- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07e006af40 msgr2=0x7f07e006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07e006af40 0x7f07e006d400 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f07f4005d00 tx=0x7f07f4005c50 comp rx=0 tx=0).stop 2023-10-18T10:25:25.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 -- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80e7ef0 msgr2=0x7f07f80e83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80e7ef0 0x7f07f80e83a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f080004ece0 tx=0x7f07f000c820 comp rx=0 tx=0).stop 2023-10-18T10:25:25.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 -- 172.21.15.125:0/1795019145 shutdown_connections 2023-10-18T10:25:25.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07e006af40 0x7f07e006d400 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f800c1f0 0x7f07f800c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80e7ef0 0x7f07f80e83a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 --2- 172.21.15.125:0/1795019145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07e8004340 0x7f07f804f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 -- 172.21.15.125:0/1795019145 >> 172.21.15.125:0/1795019145 conn(0x7f07f80559b0 msgr2=0x7f07f80e6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 -- 172.21.15.125:0/1795019145 shutdown_connections 2023-10-18T10:25:25.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.478+0000 7f07ecff9640 1 -- 172.21.15.125:0/1795019145 wait complete. 2023-10-18T10:25:25.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.481+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd0e00b0300 con 0x7fd0e0154480 2023-10-18T10:25:25.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.481+0000 7fd0d5ffb640 1 -- 172.21.15.125:0/1756368349 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd0d8057790 con 0x7fd0e0154480 2023-10-18T10:25:25.484 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:25.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdb201f3c80 con 0x7fdb200aae00 2023-10-18T10:25:25.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd0c406af10 msgr2=0x7fd0c406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd0c406af10 0x7fd0c406d3d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd0e015ebe0 tx=0x7fd0d0006040 comp rx=0 tx=0).stop 2023-10-18T10:25:25.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e0154480 msgr2=0x7fd0e0156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e0154480 0x7fd0e0156f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0057a0 tx=0x7fd0d800cb90 comp rx=0 tx=0).stop 2023-10-18T10:25:25.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 shutdown_connections 2023-10-18T10:25:25.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd0c406af10 0x7fd0c406d3d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0e0154000 0x7fd0e0156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e0154480 0x7fd0e0156f60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 --2- 172.21.15.125:0/1756368349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0cc0047c0 0x7fd0e01f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.485+0000 7fdb1d7fa640 1 -- 172.21.15.125:0/3947478912 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb201f3c80 con 0x7fdb200aae00 2023-10-18T10:25:25.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.484+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 >> 172.21.15.125:0/1756368349 conn(0x7fd0e00ae460 msgr2=0x7fd0e00ac2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.485+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 shutdown_connections 2023-10-18T10:25:25.490 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:25.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.485+0000 7fd0e6059640 1 -- 172.21.15.125:0/1756368349 wait complete. 2023-10-18T10:25:25.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdb0406af40 msgr2=0x7fdb0406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdb0406af40 0x7fdb0406d400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdb200c7870 tx=0x7fdb0c009040 comp rx=0 tx=0).stop 2023-10-18T10:25:25.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb200aae00 msgr2=0x7fdb201f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb200aae00 0x7fdb201f2550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009980 tx=0x7fdb140038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 shutdown_connections 2023-10-18T10:25:25.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdb0406af40 0x7fdb0406d400 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb201550a0 0x7fdb201f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb2015b950 0x7fdb200c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 --2- 172.21.15.125:0/3947478912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb200aae00 0x7fdb201f2550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 >> 172.21.15.125:0/3947478912 conn(0x7fdb200ae3f0 msgr2=0x7fdb200b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 shutdown_connections 2023-10-18T10:25:25.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.488+0000 7fdb26bf5640 1 -- 172.21.15.125:0/3947478912 wait complete. 2023-10-18T10:25:25.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.491+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f06b80057b0 con 0x7f06b8178ec0 2023-10-18T10:25:25.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.491+0000 7f06aeffd640 1 -- 172.21.15.125:0/2286562312 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f06c00cf680 con 0x7f06b8178ec0 2023-10-18T10:25:25.494 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:25.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.493+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f90ac00f390 con 0x7f90ac034510 2023-10-18T10:25:25.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.493+0000 7f90aa7fc640 1 -- 172.21.15.125:0/2008695496 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f90a401f020 con 0x7f90ac034510 2023-10-18T10:25:25.496 INFO:teuthology.orchestra.run.smithi125.stdout:55834574858 2023-10-18T10:25:25.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.494+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f196c14ddc0 con 0x7f196c0cb3a0 2023-10-18T10:25:25.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.494+0000 7f1961ffb640 1 -- 172.21.15.125:0/4074944349 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f196c14ddc0 con 0x7f196c0cb3a0 2023-10-18T10:25:25.497 INFO:teuthology.orchestra.run.smithi125.stdout:55834574857 2023-10-18T10:25:25.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2023-10-18T10:25:25.498 DEBUG:teuthology.parallel:result is None 2023-10-18T10:25:25.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f06a006aec0 msgr2=0x7f06a006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f06a006aec0 0x7f06a006d380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06b4036000 tx=0x7f06b4031040 comp rx=0 tx=0).stop 2023-10-18T10:25:25.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06b8178ec0 msgr2=0x7f06b8179370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06b8178ec0 0x7f06b8179370 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06c00685a0 tx=0x7f06c00694b0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 shutdown_connections 2023-10-18T10:25:25.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f06a006aec0 0x7f06a006d380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06b8178ec0 0x7f06b8179370 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b80ccde0 0x7f06b8049810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.495+0000 7f06bf59e640 1 --2- 172.21.15.125:0/2286562312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a80039c0 0x7f06b804bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 >> 172.21.15.125:0/2286562312 conn(0x7f06b801ef10 msgr2=0x7f06b8055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 shutdown_connections 2023-10-18T10:25:25.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f06bf59e640 1 -- 172.21.15.125:0/2286562312 wait complete. 2023-10-18T10:25:25.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f909806af10 msgr2=0x7f909806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f909806af10 0x7f909806d3d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f90b0001050 tx=0x7f90b0001e10 comp rx=0 tx=0).stop 2023-10-18T10:25:25.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90ac034510 msgr2=0x7f90ac04eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90ac034510 0x7f90ac04eee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0375f0 tx=0x7f90a4011e10 comp rx=0 tx=0).stop 2023-10-18T10:25:25.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 shutdown_connections 2023-10-18T10:25:25.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f909806af10 0x7f909806d3d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90ac052e90 0x7f90ac033fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90ac034510 0x7f90ac04eee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 --2- 172.21.15.125:0/2008695496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90a00047c0 0x7f90ac0df0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 >> 172.21.15.125:0/2008695496 conn(0x7f90ac01ef10 msgr2=0x7f90ac03aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 shutdown_connections 2023-10-18T10:25:25.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.496+0000 7f90bad14640 1 -- 172.21.15.125:0/2008695496 wait complete. 2023-10-18T10:25:25.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f195006ae40 msgr2=0x7f195006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f195006ae40 0x7f195006d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f196c207280 tx=0x7f195c005d80 comp rx=0 tx=0).stop 2023-10-18T10:25:25.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f196c0cb3a0 msgr2=0x7f196c0cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f196c0cb3a0 0x7f196c0cb780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f196402ccc0 tx=0x7f19640017b0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 shutdown_connections 2023-10-18T10:25:25.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f195006ae40 0x7f195006d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f196c206e80 0x7f196c202be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f196c06f6f0 0x7f196c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 --2- 172.21.15.125:0/4074944349 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f196c0cb3a0 0x7f196c0cb780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 >> 172.21.15.125:0/4074944349 conn(0x7f196c0ae3f0 msgr2=0x7f196c152660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 shutdown_connections 2023-10-18T10:25:25.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.498+0000 7f197356e640 1 -- 172.21.15.125:0/4074944349 wait complete. 2023-10-18T10:25:25.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2023-10-18T10:25:25.508 DEBUG:teuthology.parallel:result is None 2023-10-18T10:25:25.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2023-10-18T10:25:25.509 DEBUG:teuthology.parallel:result is None 2023-10-18T10:25:25.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2023-10-18T10:25:25.513 DEBUG:teuthology.parallel:result is None 2023-10-18T10:25:25.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2023-10-18T10:25:25.514 DEBUG:teuthology.parallel:result is None 2023-10-18T10:25:25.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-10-18T10:25:25.515 DEBUG:teuthology.parallel:result is None 2023-10-18T10:25:25.515 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T10:25:25.515 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:25:25.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.653+0000 7f25f55e2640 1 -- 172.21.15.125:0/2161285548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00cbd20 msgr2=0x7f25f0158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.653+0000 7f25f55e2640 1 --2- 172.21.15.125:0/2161285548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00cbd20 0x7f25f0158300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f25e0005fd0 tx=0x7f25e002c0e0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.653+0000 7f25f55e2640 1 -- 172.21.15.125:0/2161285548 shutdown_connections 2023-10-18T10:25:25.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.653+0000 7f25f55e2640 1 --2- 172.21.15.125:0/2161285548 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25f00b0530 0x7f25f00b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.653+0000 7f25f55e2640 1 --2- 172.21.15.125:0/2161285548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00cbd20 0x7f25f0158300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.653+0000 7f25f55e2640 1 -- 172.21.15.125:0/2161285548 >> 172.21.15.125:0/2161285548 conn(0x7f25f0069930 msgr2=0x7f25f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.653+0000 7f25f55e2640 1 -- 172.21.15.125:0/2161285548 shutdown_connections 2023-10-18T10:25:25.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.653+0000 7f25f55e2640 1 -- 172.21.15.125:0/2161285548 wait complete. 2023-10-18T10:25:25.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.654+0000 7f25f55e2640 1 Processor -- start 2023-10-18T10:25:25.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.654+0000 7f25f55e2640 1 -- start start 2023-10-18T10:25:25.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25f55e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25f00afa20 0x7f25f0166fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25f55e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00b0530 0x7f25f0156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25f55e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25f00cbd20 0x7f25f00a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25f55e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f01466c0 con 0x7f25f00b0530 2023-10-18T10:25:25.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25f55e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f00a77f0 con 0x7f25f00afa20 2023-10-18T10:25:25.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25f55e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f01664e0 con 0x7f25f00cbd20 2023-10-18T10:25:25.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ee7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00b0530 0x7f25f0156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ee7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00b0530 0x7f25f0156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50030/0 (socket says 172.21.15.125:50030) 2023-10-18T10:25:25.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ee7fc640 1 -- 172.21.15.125:0/1208543029 learned_addr learned my addr 172.21.15.125:0/1208543029 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ef7fe640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25f00cbd20 0x7f25f00a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25eeffd640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25f00afa20 0x7f25f0166fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ee7fc640 1 -- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25f00cbd20 msgr2=0x7f25f00a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ee7fc640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25f00cbd20 0x7f25f00a6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ee7fc640 1 -- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25f00afa20 msgr2=0x7f25f0166fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ee7fc640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25f00afa20 0x7f25f0166fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ee7fc640 1 -- 172.21.15.125:0/1208543029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0009840 con 0x7f25f00b0530 2023-10-18T10:25:25.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25ef7fe640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25f00cbd20 0x7f25f00a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:25.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.655+0000 7f25eeffd640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25f00afa20 0x7f25f0166fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:25.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.656+0000 7f25ee7fc640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00b0530 0x7f25f0156530 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f25e400c4b0 tx=0x7f25e400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.656+0000 7f25d77fe640 1 -- 172.21.15.125:0/1208543029 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4005480 con 0x7f25f00b0530 2023-10-18T10:25:25.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.656+0000 7f25d77fe640 1 -- 172.21.15.125:0/1208543029 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25e40036e0 con 0x7f25f00b0530 2023-10-18T10:25:25.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.656+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f01667c0 con 0x7f25f00b0530 2023-10-18T10:25:25.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.656+0000 7f25d77fe640 1 -- 172.21.15.125:0/1208543029 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4003b30 con 0x7f25f00b0530 2023-10-18T10:25:25.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.656+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f0156a70 con 0x7f25f00b0530 2023-10-18T10:25:25.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.657+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f0166ce0 con 0x7f25f00b0530 2023-10-18T10:25:25.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.657+0000 7f25d77fe640 1 -- 172.21.15.125:0/1208543029 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25e4003c90 con 0x7f25f00b0530 2023-10-18T10:25:25.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.657+0000 7f25d77fe640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f25dc06ae40 0x7f25dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.658+0000 7f25d77fe640 1 -- 172.21.15.125:0/1208543029 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25e4089110 con 0x7f25f00b0530 2023-10-18T10:25:25.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.658+0000 7f25eeffd640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f25dc06ae40 0x7f25dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.658+0000 7f25eeffd640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f25dc06ae40 0x7f25dc06d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f25e0035000 tx=0x7f25e0004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.662+0000 7f25d77fe640 1 -- 172.21.15.125:0/1208543029 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25f0166ce0 con 0x7f25f00b0530 2023-10-18T10:25:25.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.824+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25f00cc100 con 0x7f25dc06ae40 2023-10-18T10:25:25.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.833+0000 7f25d77fe640 1 -- 172.21.15.125:0/1208543029 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345206 (secure 0 0 0) 0x7f25e0003970 con 0x7f25dc06ae40 2023-10-18T10:25:25.838 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:25.840 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:25:25.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f25dc06ae40 msgr2=0x7f25dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f25dc06ae40 0x7f25dc06d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f25e0035000 tx=0x7f25e0004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00b0530 msgr2=0x7f25f0156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00b0530 0x7f25f0156530 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f25e400c4b0 tx=0x7f25e400c970 comp rx=0 tx=0).stop 2023-10-18T10:25:25.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 shutdown_connections 2023-10-18T10:25:25.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f25dc06ae40 0x7f25dc06d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25f00cbd20 0x7f25f00a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25f00b0530 0x7f25f0156530 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 --2- 172.21.15.125:0/1208543029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25f00afa20 0x7f25f0166fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.836+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 >> 172.21.15.125:0/1208543029 conn(0x7f25f0069930 msgr2=0x7f25f0146b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.837+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 shutdown_connections 2023-10-18T10:25:25.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.837+0000 7f25f55e2640 1 -- 172.21.15.125:0/1208543029 wait complete. 2023-10-18T10:25:25.849 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-18T10:25:24.571501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219484,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755236,"statfs":{"total":773094113280,"available":772869361664,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003425"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254754+0000","last_change":"2023-10-18T10:25:03.211565+0000","last_active":"2023-10-18T10:25:08.254754+0000","last_peered":"2023-10-18T10:25:08.254754+0000","last_clean":"2023-10-18T10:25:08.254754+0000","last_became_active":"2023-10-18T10:25:03.211312+0000","last_became_peered":"2023-10-18T10:25:03.211312+0000","last_unstale":"2023-10-18T10:25:08.254754+0000","last_undegraded":"2023-10-18T10:25:08.254754+0000","last_fullsized":"2023-10-18T10:25:08.254754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254543+0000","last_change":"2023-10-18T10:25:04.232540+0000","last_active":"2023-10-18T10:25:08.254543+0000","last_peered":"2023-10-18T10:25:08.254543+0000","last_clean":"2023-10-18T10:25:08.254543+0000","last_became_active":"2023-10-18T10:25:04.232270+0000","last_became_peered":"2023-10-18T10:25:04.232270+0000","last_unstale":"2023-10-18T10:25:08.254543+0000","last_undegraded":"2023-10-18T10:25:08.254543+0000","last_fullsized":"2023-10-18T10:25:08.254543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920838+0000","last_change":"2023-10-18T10:25:08.375121+0000","last_active":"2023-10-18T10:25:09.920838+0000","last_peered":"2023-10-18T10:25:09.920838+0000","last_clean":"2023-10-18T10:25:09.920838+0000","last_became_active":"2023-10-18T10:25:08.374754+0000","last_became_peered":"2023-10-18T10:25:08.374754+0000","last_unstale":"2023-10-18T10:25:09.920838+0000","last_undegraded":"2023-10-18T10:25:09.920838+0000","last_fullsized":"2023-10-18T10:25:09.920838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:26.275317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254691+0000","last_change":"2023-10-18T10:25:03.209283+0000","last_active":"2023-10-18T10:25:08.254691+0000","last_peered":"2023-10-18T10:25:08.254691+0000","last_clean":"2023-10-18T10:25:08.254691+0000","last_became_active":"2023-10-18T10:25:03.208718+0000","last_became_peered":"2023-10-18T10:25:03.208718+0000","last_unstale":"2023-10-18T10:25:08.254691+0000","last_undegraded":"2023-10-18T10:25:08.254691+0000","last_fullsized":"2023-10-18T10:25:08.254691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:43.110925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254524+0000","last_change":"2023-10-18T10:25:04.239316+0000","last_active":"2023-10-18T10:25:08.254524+0000","last_peered":"2023-10-18T10:25:08.254524+0000","last_clean":"2023-10-18T10:25:08.254524+0000","last_became_active":"2023-10-18T10:25:04.235952+0000","last_became_peered":"2023-10-18T10:25:04.235952+0000","last_unstale":"2023-10-18T10:25:08.254524+0000","last_undegraded":"2023-10-18T10:25:08.254524+0000","last_fullsized":"2023-10-18T10:25:08.254524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:09.100769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263035+0000","last_change":"2023-10-18T10:25:08.267055+0000","last_active":"2023-10-18T10:25:09.263035+0000","last_peered":"2023-10-18T10:25:09.263035+0000","last_clean":"2023-10-18T10:25:09.263035+0000","last_became_active":"2023-10-18T10:25:08.266533+0000","last_became_peered":"2023-10-18T10:25:08.266533+0000","last_unstale":"2023-10-18T10:25:09.263035+0000","last_undegraded":"2023-10-18T10:25:09.263035+0000","last_fullsized":"2023-10-18T10:25:09.263035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:31.219679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370926+0000","last_change":"2023-10-18T10:25:03.212550+0000","last_active":"2023-10-18T10:25:08.370926+0000","last_peered":"2023-10-18T10:25:08.370926+0000","last_clean":"2023-10-18T10:25:08.370926+0000","last_became_active":"2023-10-18T10:25:03.211719+0000","last_became_peered":"2023-10-18T10:25:03.211719+0000","last_unstale":"2023-10-18T10:25:08.370926+0000","last_undegraded":"2023-10-18T10:25:08.370926+0000","last_fullsized":"2023-10-18T10:25:08.370926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257891+0000","last_change":"2023-10-18T10:25:04.235362+0000","last_active":"2023-10-18T10:25:08.257891+0000","last_peered":"2023-10-18T10:25:08.257891+0000","last_clean":"2023-10-18T10:25:08.257891+0000","last_became_active":"2023-10-18T10:25:04.235036+0000","last_became_peered":"2023-10-18T10:25:04.235036+0000","last_unstale":"2023-10-18T10:25:08.257891+0000","last_undegraded":"2023-10-18T10:25:08.257891+0000","last_fullsized":"2023-10-18T10:25:08.257891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:26.705420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263812+0000","last_change":"2023-10-18T10:25:08.270146+0000","last_active":"2023-10-18T10:25:09.263812+0000","last_peered":"2023-10-18T10:25:09.263812+0000","last_clean":"2023-10-18T10:25:09.263812+0000","last_became_active":"2023-10-18T10:25:08.269580+0000","last_became_peered":"2023-10-18T10:25:08.269580+0000","last_unstale":"2023-10-18T10:25:09.263812+0000","last_undegraded":"2023-10-18T10:25:09.263812+0000","last_fullsized":"2023-10-18T10:25:09.263812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370610+0000","last_change":"2023-10-18T10:25:03.216100+0000","last_active":"2023-10-18T10:25:08.370610+0000","last_peered":"2023-10-18T10:25:08.370610+0000","last_clean":"2023-10-18T10:25:08.370610+0000","last_became_active":"2023-10-18T10:25:03.215819+0000","last_became_peered":"2023-10-18T10:25:03.215819+0000","last_unstale":"2023-10-18T10:25:08.370610+0000","last_undegraded":"2023-10-18T10:25:08.370610+0000","last_fullsized":"2023-10-18T10:25:08.370610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:43.110925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256236+0000","last_change":"2023-10-18T10:25:04.236788+0000","last_active":"2023-10-18T10:25:08.256236+0000","last_peered":"2023-10-18T10:25:08.256236+0000","last_clean":"2023-10-18T10:25:08.256236+0000","last_became_active":"2023-10-18T10:25:04.236291+0000","last_became_peered":"2023-10-18T10:25:04.236291+0000","last_unstale":"2023-10-18T10:25:08.256236+0000","last_undegraded":"2023-10-18T10:25:08.256236+0000","last_fullsized":"2023-10-18T10:25:08.256236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:39.174524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263153+0000","last_change":"2023-10-18T10:25:08.265913+0000","last_active":"2023-10-18T10:25:09.263153+0000","last_peered":"2023-10-18T10:25:09.263153+0000","last_clean":"2023-10-18T10:25:09.263153+0000","last_became_active":"2023-10-18T10:25:08.264755+0000","last_became_peered":"2023-10-18T10:25:08.264755+0000","last_unstale":"2023-10-18T10:25:09.263153+0000","last_undegraded":"2023-10-18T10:25:09.263153+0000","last_fullsized":"2023-10-18T10:25:09.263153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:22.912468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265054+0000","last_change":"2023-10-18T10:25:04.230696+0000","last_active":"2023-10-18T10:25:08.265054+0000","last_peered":"2023-10-18T10:25:08.265054+0000","last_clean":"2023-10-18T10:25:08.265054+0000","last_became_active":"2023-10-18T10:25:04.230402+0000","last_became_peered":"2023-10-18T10:25:04.230402+0000","last_unstale":"2023-10-18T10:25:08.265054+0000","last_undegraded":"2023-10-18T10:25:08.265054+0000","last_fullsized":"2023-10-18T10:25:08.265054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:34.587493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370834+0000","last_change":"2023-10-18T10:25:03.210831+0000","last_active":"2023-10-18T10:25:08.370834+0000","last_peered":"2023-10-18T10:25:08.370834+0000","last_clean":"2023-10-18T10:25:08.370834+0000","last_became_active":"2023-10-18T10:25:03.210586+0000","last_became_peered":"2023-10-18T10:25:03.210586+0000","last_unstale":"2023-10-18T10:25:08.370834+0000","last_undegraded":"2023-10-18T10:25:08.370834+0000","last_fullsized":"2023-10-18T10:25:08.370834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263925+0000","last_change":"2023-10-18T10:25:08.270065+0000","last_active":"2023-10-18T10:25:09.263925+0000","last_peered":"2023-10-18T10:25:09.263925+0000","last_clean":"2023-10-18T10:25:09.263925+0000","last_became_active":"2023-10-18T10:25:08.269544+0000","last_became_peered":"2023-10-18T10:25:08.269544+0000","last_unstale":"2023-10-18T10:25:09.263925+0000","last_undegraded":"2023-10-18T10:25:09.263925+0000","last_fullsized":"2023-10-18T10:25:09.263925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253560+0000","last_change":"2023-10-18T10:25:04.237642+0000","last_active":"2023-10-18T10:25:08.253560+0000","last_peered":"2023-10-18T10:25:08.253560+0000","last_clean":"2023-10-18T10:25:08.253560+0000","last_became_active":"2023-10-18T10:25:04.237321+0000","last_became_peered":"2023-10-18T10:25:04.237321+0000","last_unstale":"2023-10-18T10:25:08.253560+0000","last_undegraded":"2023-10-18T10:25:08.253560+0000","last_fullsized":"2023-10-18T10:25:08.253560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254793+0000","last_change":"2023-10-18T10:25:03.207491+0000","last_active":"2023-10-18T10:25:08.254793+0000","last_peered":"2023-10-18T10:25:08.254793+0000","last_clean":"2023-10-18T10:25:08.254793+0000","last_became_active":"2023-10-18T10:25:03.207227+0000","last_became_peered":"2023-10-18T10:25:03.207227+0000","last_unstale":"2023-10-18T10:25:08.254793+0000","last_undegraded":"2023-10-18T10:25:08.254793+0000","last_fullsized":"2023-10-18T10:25:08.254793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263777+0000","last_change":"2023-10-18T10:25:08.270542+0000","last_active":"2023-10-18T10:25:09.263777+0000","last_peered":"2023-10-18T10:25:09.263777+0000","last_clean":"2023-10-18T10:25:09.263777+0000","last_became_active":"2023-10-18T10:25:08.270153+0000","last_became_peered":"2023-10-18T10:25:08.270153+0000","last_unstale":"2023-10-18T10:25:09.263777+0000","last_undegraded":"2023-10-18T10:25:09.263777+0000","last_fullsized":"2023-10-18T10:25:09.263777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254454+0000","last_change":"2023-10-18T10:25:04.230256+0000","last_active":"2023-10-18T10:25:08.254454+0000","last_peered":"2023-10-18T10:25:08.254454+0000","last_clean":"2023-10-18T10:25:08.254454+0000","last_became_active":"2023-10-18T10:25:04.230007+0000","last_became_peered":"2023-10-18T10:25:04.230007+0000","last_unstale":"2023-10-18T10:25:08.254454+0000","last_undegraded":"2023-10-18T10:25:08.254454+0000","last_fullsized":"2023-10-18T10:25:08.254454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253450+0000","last_change":"2023-10-18T10:25:03.203658+0000","last_active":"2023-10-18T10:25:08.253450+0000","last_peered":"2023-10-18T10:25:08.253450+0000","last_clean":"2023-10-18T10:25:08.253450+0000","last_became_active":"2023-10-18T10:25:03.203419+0000","last_became_peered":"2023-10-18T10:25:03.203419+0000","last_unstale":"2023-10-18T10:25:08.253450+0000","last_undegraded":"2023-10-18T10:25:08.253450+0000","last_fullsized":"2023-10-18T10:25:08.253450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262990+0000","last_change":"2023-10-18T10:25:08.373967+0000","last_active":"2023-10-18T10:25:09.262990+0000","last_peered":"2023-10-18T10:25:09.262990+0000","last_clean":"2023-10-18T10:25:09.262990+0000","last_became_active":"2023-10-18T10:25:08.373624+0000","last_became_peered":"2023-10-18T10:25:08.373624+0000","last_unstale":"2023-10-18T10:25:09.262990+0000","last_undegraded":"2023-10-18T10:25:09.262990+0000","last_fullsized":"2023-10-18T10:25:09.262990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:47.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257377+0000","last_change":"2023-10-18T10:25:04.227308+0000","last_active":"2023-10-18T10:25:08.257377+0000","last_peered":"2023-10-18T10:25:08.257377+0000","last_clean":"2023-10-18T10:25:08.257377+0000","last_became_active":"2023-10-18T10:25:04.227010+0000","last_became_peered":"2023-10-18T10:25:04.227010+0000","last_unstale":"2023-10-18T10:25:08.257377+0000","last_undegraded":"2023-10-18T10:25:08.257377+0000","last_fullsized":"2023-10-18T10:25:08.257377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:03.438820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369608+0000","last_change":"2023-10-18T10:25:03.210132+0000","last_active":"2023-10-18T10:25:08.369608+0000","last_peered":"2023-10-18T10:25:08.369608+0000","last_clean":"2023-10-18T10:25:08.369608+0000","last_became_active":"2023-10-18T10:25:03.209850+0000","last_became_peered":"2023-10-18T10:25:03.209850+0000","last_unstale":"2023-10-18T10:25:08.369608+0000","last_undegraded":"2023-10-18T10:25:08.369608+0000","last_fullsized":"2023-10-18T10:25:08.369608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263730+0000","last_change":"2023-10-18T10:25:08.270370+0000","last_active":"2023-10-18T10:25:09.263730+0000","last_peered":"2023-10-18T10:25:09.263730+0000","last_clean":"2023-10-18T10:25:09.263730+0000","last_became_active":"2023-10-18T10:25:08.269625+0000","last_became_peered":"2023-10-18T10:25:08.269625+0000","last_unstale":"2023-10-18T10:25:09.263730+0000","last_undegraded":"2023-10-18T10:25:09.263730+0000","last_fullsized":"2023-10-18T10:25:09.263730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:45.178929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257699+0000","last_change":"2023-10-18T10:25:04.239309+0000","last_active":"2023-10-18T10:25:08.257699+0000","last_peered":"2023-10-18T10:25:08.257699+0000","last_clean":"2023-10-18T10:25:08.257699+0000","last_became_active":"2023-10-18T10:25:04.236322+0000","last_became_peered":"2023-10-18T10:25:04.236322+0000","last_unstale":"2023-10-18T10:25:08.257699+0000","last_undegraded":"2023-10-18T10:25:08.257699+0000","last_fullsized":"2023-10-18T10:25:08.257699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264329+0000","last_change":"2023-10-18T10:25:03.211765+0000","last_active":"2023-10-18T10:25:08.264329+0000","last_peered":"2023-10-18T10:25:08.264329+0000","last_clean":"2023-10-18T10:25:08.264329+0000","last_became_active":"2023-10-18T10:25:03.211504+0000","last_became_peered":"2023-10-18T10:25:03.211504+0000","last_unstale":"2023-10-18T10:25:08.264329+0000","last_undegraded":"2023-10-18T10:25:08.264329+0000","last_fullsized":"2023-10-18T10:25:08.264329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:41.039503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263210+0000","last_change":"2023-10-18T10:25:08.374129+0000","last_active":"2023-10-18T10:25:09.263210+0000","last_peered":"2023-10-18T10:25:09.263210+0000","last_clean":"2023-10-18T10:25:09.263210+0000","last_became_active":"2023-10-18T10:25:08.373675+0000","last_became_peered":"2023-10-18T10:25:08.373675+0000","last_unstale":"2023-10-18T10:25:09.263210+0000","last_undegraded":"2023-10-18T10:25:09.263210+0000","last_fullsized":"2023-10-18T10:25:09.263210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:12.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255528+0000","last_change":"2023-10-18T10:25:04.232333+0000","last_active":"2023-10-18T10:25:08.255528+0000","last_peered":"2023-10-18T10:25:08.255528+0000","last_clean":"2023-10-18T10:25:08.255528+0000","last_became_active":"2023-10-18T10:25:04.232075+0000","last_became_peered":"2023-10-18T10:25:04.232075+0000","last_unstale":"2023-10-18T10:25:08.255528+0000","last_undegraded":"2023-10-18T10:25:08.255528+0000","last_fullsized":"2023-10-18T10:25:08.255528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:39.174524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255192+0000","last_change":"2023-10-18T10:25:03.209779+0000","last_active":"2023-10-18T10:25:08.255192+0000","last_peered":"2023-10-18T10:25:08.255192+0000","last_clean":"2023-10-18T10:25:08.255192+0000","last_became_active":"2023-10-18T10:25:03.209508+0000","last_became_peered":"2023-10-18T10:25:03.209508+0000","last_unstale":"2023-10-18T10:25:08.255192+0000","last_undegraded":"2023-10-18T10:25:08.255192+0000","last_fullsized":"2023-10-18T10:25:08.255192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:33.585439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920818+0000","last_change":"2023-10-18T10:25:08.372735+0000","last_active":"2023-10-18T10:25:09.920818+0000","last_peered":"2023-10-18T10:25:09.920818+0000","last_clean":"2023-10-18T10:25:09.920818+0000","last_became_active":"2023-10-18T10:25:08.372386+0000","last_became_peered":"2023-10-18T10:25:08.372386+0000","last_unstale":"2023-10-18T10:25:09.920818+0000","last_undegraded":"2023-10-18T10:25:09.920818+0000","last_fullsized":"2023-10-18T10:25:09.920818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:53.362022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255868+0000","last_change":"2023-10-18T10:25:04.240297+0000","last_active":"2023-10-18T10:25:08.255868+0000","last_peered":"2023-10-18T10:25:08.255868+0000","last_clean":"2023-10-18T10:25:08.255868+0000","last_became_active":"2023-10-18T10:25:04.239997+0000","last_became_peered":"2023-10-18T10:25:04.239997+0000","last_unstale":"2023-10-18T10:25:08.255868+0000","last_undegraded":"2023-10-18T10:25:08.255868+0000","last_fullsized":"2023-10-18T10:25:08.255868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371087+0000","last_change":"2023-10-18T10:25:03.213798+0000","last_active":"2023-10-18T10:25:08.371087+0000","last_peered":"2023-10-18T10:25:08.371087+0000","last_clean":"2023-10-18T10:25:08.371087+0000","last_became_active":"2023-10-18T10:25:03.213557+0000","last_became_peered":"2023-10-18T10:25:03.213557+0000","last_unstale":"2023-10-18T10:25:08.371087+0000","last_undegraded":"2023-10-18T10:25:08.371087+0000","last_fullsized":"2023-10-18T10:25:08.371087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:08.120358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307135+0000","last_change":"2023-10-18T10:25:08.370813+0000","last_active":"2023-10-18T10:25:10.307135+0000","last_peered":"2023-10-18T10:25:10.307135+0000","last_clean":"2023-10-18T10:25:10.307135+0000","last_became_active":"2023-10-18T10:25:08.370441+0000","last_became_peered":"2023-10-18T10:25:08.370441+0000","last_unstale":"2023-10-18T10:25:10.307135+0000","last_undegraded":"2023-10-18T10:25:10.307135+0000","last_fullsized":"2023-10-18T10:25:10.307135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256321+0000","last_change":"2023-10-18T10:25:04.242148+0000","last_active":"2023-10-18T10:25:08.256321+0000","last_peered":"2023-10-18T10:25:08.256321+0000","last_clean":"2023-10-18T10:25:08.256321+0000","last_became_active":"2023-10-18T10:25:04.239749+0000","last_became_peered":"2023-10-18T10:25:04.239749+0000","last_unstale":"2023-10-18T10:25:08.256321+0000","last_undegraded":"2023-10-18T10:25:08.256321+0000","last_fullsized":"2023-10-18T10:25:08.256321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255943+0000","last_change":"2023-10-18T10:25:03.205435+0000","last_active":"2023-10-18T10:25:08.255943+0000","last_peered":"2023-10-18T10:25:08.255943+0000","last_clean":"2023-10-18T10:25:08.255943+0000","last_became_active":"2023-10-18T10:25:03.205166+0000","last_became_peered":"2023-10-18T10:25:03.205166+0000","last_unstale":"2023-10-18T10:25:08.255943+0000","last_undegraded":"2023-10-18T10:25:08.255943+0000","last_fullsized":"2023-10-18T10:25:08.255943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:41.039503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262304+0000","last_change":"2023-10-18T10:25:08.267310+0000","last_active":"2023-10-18T10:25:09.262304+0000","last_peered":"2023-10-18T10:25:09.262304+0000","last_clean":"2023-10-18T10:25:09.262304+0000","last_became_active":"2023-10-18T10:25:08.266919+0000","last_became_peered":"2023-10-18T10:25:08.266919+0000","last_unstale":"2023-10-18T10:25:09.262304+0000","last_undegraded":"2023-10-18T10:25:09.262304+0000","last_fullsized":"2023-10-18T10:25:09.262304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257274+0000","last_change":"2023-10-18T10:25:04.234960+0000","last_active":"2023-10-18T10:25:08.257274+0000","last_peered":"2023-10-18T10:25:08.257274+0000","last_clean":"2023-10-18T10:25:08.257274+0000","last_became_active":"2023-10-18T10:25:04.234695+0000","last_became_peered":"2023-10-18T10:25:04.234695+0000","last_unstale":"2023-10-18T10:25:08.257274+0000","last_undegraded":"2023-10-18T10:25:08.257274+0000","last_fullsized":"2023-10-18T10:25:08.257274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:44.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254998+0000","last_change":"2023-10-18T10:25:03.217364+0000","last_active":"2023-10-18T10:25:08.254998+0000","last_peered":"2023-10-18T10:25:08.254998+0000","last_clean":"2023-10-18T10:25:08.254998+0000","last_became_active":"2023-10-18T10:25:03.217105+0000","last_became_peered":"2023-10-18T10:25:03.217105+0000","last_unstale":"2023-10-18T10:25:08.254998+0000","last_undegraded":"2023-10-18T10:25:08.254998+0000","last_fullsized":"2023-10-18T10:25:08.254998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262547+0000","last_change":"2023-10-18T10:25:08.268370+0000","last_active":"2023-10-18T10:25:09.262547+0000","last_peered":"2023-10-18T10:25:09.262547+0000","last_clean":"2023-10-18T10:25:09.262547+0000","last_became_active":"2023-10-18T10:25:08.267649+0000","last_became_peered":"2023-10-18T10:25:08.267649+0000","last_unstale":"2023-10-18T10:25:09.262547+0000","last_undegraded":"2023-10-18T10:25:09.262547+0000","last_fullsized":"2023-10-18T10:25:09.262547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253484+0000","last_change":"2023-10-18T10:25:04.229878+0000","last_active":"2023-10-18T10:25:08.253484+0000","last_peered":"2023-10-18T10:25:08.253484+0000","last_clean":"2023-10-18T10:25:08.253484+0000","last_became_active":"2023-10-18T10:25:04.227272+0000","last_became_peered":"2023-10-18T10:25:04.227272+0000","last_unstale":"2023-10-18T10:25:08.253484+0000","last_undegraded":"2023-10-18T10:25:08.253484+0000","last_fullsized":"2023-10-18T10:25:08.253484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250602+0000","last_change":"2023-10-18T10:25:03.209065+0000","last_active":"2023-10-18T10:25:08.250602+0000","last_peered":"2023-10-18T10:25:08.250602+0000","last_clean":"2023-10-18T10:25:08.250602+0000","last_became_active":"2023-10-18T10:25:03.208772+0000","last_became_peered":"2023-10-18T10:25:03.208772+0000","last_unstale":"2023-10-18T10:25:08.250602+0000","last_undegraded":"2023-10-18T10:25:08.250602+0000","last_fullsized":"2023-10-18T10:25:08.250602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:34.438877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262993+0000","last_change":"2023-10-18T10:25:08.266888+0000","last_active":"2023-10-18T10:25:09.262993+0000","last_peered":"2023-10-18T10:25:09.262993+0000","last_clean":"2023-10-18T10:25:09.262993+0000","last_became_active":"2023-10-18T10:25:08.266538+0000","last_became_peered":"2023-10-18T10:25:08.266538+0000","last_unstale":"2023-10-18T10:25:09.262993+0000","last_undegraded":"2023-10-18T10:25:09.262993+0000","last_fullsized":"2023-10-18T10:25:09.262993+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:53.048432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253849+0000","last_change":"2023-10-18T10:25:04.233140+0000","last_active":"2023-10-18T10:25:08.253849+0000","last_peered":"2023-10-18T10:25:08.253849+0000","last_clean":"2023-10-18T10:25:08.253849+0000","last_became_active":"2023-10-18T10:25:04.232753+0000","last_became_peered":"2023-10-18T10:25:04.232753+0000","last_unstale":"2023-10-18T10:25:08.253849+0000","last_undegraded":"2023-10-18T10:25:08.253849+0000","last_fullsized":"2023-10-18T10:25:08.253849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:27.915879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369629+0000","last_change":"2023-10-18T10:25:03.217333+0000","last_active":"2023-10-18T10:25:08.369629+0000","last_peered":"2023-10-18T10:25:08.369629+0000","last_clean":"2023-10-18T10:25:08.369629+0000","last_became_active":"2023-10-18T10:25:03.217060+0000","last_became_peered":"2023-10-18T10:25:03.217060+0000","last_unstale":"2023-10-18T10:25:08.369629+0000","last_undegraded":"2023-10-18T10:25:08.369629+0000","last_fullsized":"2023-10-18T10:25:08.369629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:02.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307375+0000","last_change":"2023-10-18T10:25:08.373111+0000","last_active":"2023-10-18T10:25:10.307375+0000","last_peered":"2023-10-18T10:25:10.307375+0000","last_clean":"2023-10-18T10:25:10.307375+0000","last_became_active":"2023-10-18T10:25:08.372684+0000","last_became_peered":"2023-10-18T10:25:08.372684+0000","last_unstale":"2023-10-18T10:25:10.307375+0000","last_undegraded":"2023-10-18T10:25:10.307375+0000","last_fullsized":"2023-10-18T10:25:10.307375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:53.048432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253864+0000","last_change":"2023-10-18T10:25:04.238187+0000","last_active":"2023-10-18T10:25:08.253864+0000","last_peered":"2023-10-18T10:25:08.253864+0000","last_clean":"2023-10-18T10:25:08.253864+0000","last_became_active":"2023-10-18T10:25:04.237921+0000","last_became_peered":"2023-10-18T10:25:04.237921+0000","last_unstale":"2023-10-18T10:25:08.253864+0000","last_undegraded":"2023-10-18T10:25:08.253864+0000","last_fullsized":"2023-10-18T10:25:08.253864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253911+0000","last_change":"2023-10-18T10:25:03.220362+0000","last_active":"2023-10-18T10:25:08.253911+0000","last_peered":"2023-10-18T10:25:08.253911+0000","last_clean":"2023-10-18T10:25:08.253911+0000","last_became_active":"2023-10-18T10:25:03.220150+0000","last_became_peered":"2023-10-18T10:25:03.220150+0000","last_unstale":"2023-10-18T10:25:08.253911+0000","last_undegraded":"2023-10-18T10:25:08.253911+0000","last_fullsized":"2023-10-18T10:25:08.253911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369864+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:25:08.369864+0000","last_peered":"2023-10-18T10:25:08.369864+0000","last_clean":"2023-10-18T10:25:08.369864+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:25:08.369864+0000","last_undegraded":"2023-10-18T10:25:08.369864+0000","last_fullsized":"2023-10-18T10:25:08.369864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307435+0000","last_change":"2023-10-18T10:25:08.374370+0000","last_active":"2023-10-18T10:25:10.307435+0000","last_peered":"2023-10-18T10:25:10.307435+0000","last_clean":"2023-10-18T10:25:10.307435+0000","last_became_active":"2023-10-18T10:25:08.372029+0000","last_became_peered":"2023-10-18T10:25:08.372029+0000","last_unstale":"2023-10-18T10:25:10.307435+0000","last_undegraded":"2023-10-18T10:25:10.307435+0000","last_fullsized":"2023-10-18T10:25:10.307435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:45.178929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370575+0000","last_change":"2023-10-18T10:25:04.239066+0000","last_active":"2023-10-18T10:25:08.370575+0000","last_peered":"2023-10-18T10:25:08.370575+0000","last_clean":"2023-10-18T10:25:08.370575+0000","last_became_active":"2023-10-18T10:25:04.238531+0000","last_became_peered":"2023-10-18T10:25:04.238531+0000","last_unstale":"2023-10-18T10:25:08.370575+0000","last_undegraded":"2023-10-18T10:25:08.370575+0000","last_fullsized":"2023-10-18T10:25:08.370575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250745+0000","last_change":"2023-10-18T10:25:03.213157+0000","last_active":"2023-10-18T10:25:08.250745+0000","last_peered":"2023-10-18T10:25:08.250745+0000","last_clean":"2023-10-18T10:25:08.250745+0000","last_became_active":"2023-10-18T10:25:03.212900+0000","last_became_peered":"2023-10-18T10:25:03.212900+0000","last_unstale":"2023-10-18T10:25:08.250745+0000","last_undegraded":"2023-10-18T10:25:08.250745+0000","last_fullsized":"2023-10-18T10:25:08.250745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:29:02.497235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307254+0000","last_change":"2023-10-18T10:25:08.370854+0000","last_active":"2023-10-18T10:25:10.307254+0000","last_peered":"2023-10-18T10:25:10.307254+0000","last_clean":"2023-10-18T10:25:10.307254+0000","last_became_active":"2023-10-18T10:25:08.370423+0000","last_became_peered":"2023-10-18T10:25:08.370423+0000","last_unstale":"2023-10-18T10:25:10.307254+0000","last_undegraded":"2023-10-18T10:25:10.307254+0000","last_fullsized":"2023-10-18T10:25:10.307254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:57.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257993+0000","last_change":"2023-10-18T10:25:04.239837+0000","last_active":"2023-10-18T10:25:08.257993+0000","last_peered":"2023-10-18T10:25:08.257993+0000","last_clean":"2023-10-18T10:25:08.257993+0000","last_became_active":"2023-10-18T10:25:04.239576+0000","last_became_peered":"2023-10-18T10:25:04.239576+0000","last_unstale":"2023-10-18T10:25:08.257993+0000","last_undegraded":"2023-10-18T10:25:08.257993+0000","last_fullsized":"2023-10-18T10:25:08.257993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:43.022445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253804+0000","last_change":"2023-10-18T10:25:03.204821+0000","last_active":"2023-10-18T10:25:08.253804+0000","last_peered":"2023-10-18T10:25:08.253804+0000","last_clean":"2023-10-18T10:25:08.253804+0000","last_became_active":"2023-10-18T10:25:03.204545+0000","last_became_peered":"2023-10-18T10:25:03.204545+0000","last_unstale":"2023-10-18T10:25:08.253804+0000","last_undegraded":"2023-10-18T10:25:08.253804+0000","last_fullsized":"2023-10-18T10:25:08.253804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307183+0000","last_change":"2023-10-18T10:25:08.265973+0000","last_active":"2023-10-18T10:25:10.307183+0000","last_peered":"2023-10-18T10:25:10.307183+0000","last_clean":"2023-10-18T10:25:10.307183+0000","last_became_active":"2023-10-18T10:25:08.265607+0000","last_became_peered":"2023-10-18T10:25:08.265607+0000","last_unstale":"2023-10-18T10:25:10.307183+0000","last_undegraded":"2023-10-18T10:25:10.307183+0000","last_fullsized":"2023-10-18T10:25:10.307183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371147+0000","last_change":"2023-10-18T10:25:04.245074+0000","last_active":"2023-10-18T10:25:08.371147+0000","last_peered":"2023-10-18T10:25:08.371147+0000","last_clean":"2023-10-18T10:25:08.371147+0000","last_became_active":"2023-10-18T10:25:04.239593+0000","last_became_peered":"2023-10-18T10:25:04.239593+0000","last_unstale":"2023-10-18T10:25:08.371147+0000","last_undegraded":"2023-10-18T10:25:08.371147+0000","last_fullsized":"2023-10-18T10:25:08.371147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:29.983102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250759+0000","last_change":"2023-10-18T10:25:03.206351+0000","last_active":"2023-10-18T10:25:08.250759+0000","last_peered":"2023-10-18T10:25:08.250759+0000","last_clean":"2023-10-18T10:25:08.250759+0000","last_became_active":"2023-10-18T10:25:03.206082+0000","last_became_peered":"2023-10-18T10:25:03.206082+0000","last_unstale":"2023-10-18T10:25:08.250759+0000","last_undegraded":"2023-10-18T10:25:08.250759+0000","last_fullsized":"2023-10-18T10:25:08.250759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307655+0000","last_change":"2023-10-18T10:25:08.372691+0000","last_active":"2023-10-18T10:25:10.307655+0000","last_peered":"2023-10-18T10:25:10.307655+0000","last_clean":"2023-10-18T10:25:10.307655+0000","last_became_active":"2023-10-18T10:25:08.372252+0000","last_became_peered":"2023-10-18T10:25:08.372252+0000","last_unstale":"2023-10-18T10:25:10.307655+0000","last_undegraded":"2023-10-18T10:25:10.307655+0000","last_fullsized":"2023-10-18T10:25:10.307655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:53.362022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370821+0000","last_change":"2023-10-18T10:25:04.237483+0000","last_active":"2023-10-18T10:25:08.370821+0000","last_peered":"2023-10-18T10:25:08.370821+0000","last_clean":"2023-10-18T10:25:08.370821+0000","last_became_active":"2023-10-18T10:25:04.237196+0000","last_became_peered":"2023-10-18T10:25:04.237196+0000","last_unstale":"2023-10-18T10:25:08.370821+0000","last_undegraded":"2023-10-18T10:25:08.370821+0000","last_fullsized":"2023-10-18T10:25:08.370821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:41.123375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254019+0000","last_change":"2023-10-18T10:25:03.202658+0000","last_active":"2023-10-18T10:25:08.254019+0000","last_peered":"2023-10-18T10:25:08.254019+0000","last_clean":"2023-10-18T10:25:08.254019+0000","last_became_active":"2023-10-18T10:25:03.202313+0000","last_became_peered":"2023-10-18T10:25:03.202313+0000","last_unstale":"2023-10-18T10:25:08.254019+0000","last_undegraded":"2023-10-18T10:25:08.254019+0000","last_fullsized":"2023-10-18T10:25:08.254019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:18:46.417045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262979+0000","last_change":"2023-10-18T10:25:08.273431+0000","last_active":"2023-10-18T10:25:09.262979+0000","last_peered":"2023-10-18T10:25:09.262979+0000","last_clean":"2023-10-18T10:25:09.262979+0000","last_became_active":"2023-10-18T10:25:08.273002+0000","last_became_peered":"2023-10-18T10:25:08.273002+0000","last_unstale":"2023-10-18T10:25:09.262979+0000","last_undegraded":"2023-10-18T10:25:09.262979+0000","last_fullsized":"2023-10-18T10:25:09.262979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:43.230078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371654+0000","last_change":"2023-10-18T10:25:04.237817+0000","last_active":"2023-10-18T10:25:08.371654+0000","last_peered":"2023-10-18T10:25:08.371654+0000","last_clean":"2023-10-18T10:25:08.371654+0000","last_became_active":"2023-10-18T10:25:04.237459+0000","last_became_peered":"2023-10-18T10:25:04.237459+0000","last_unstale":"2023-10-18T10:25:08.371654+0000","last_undegraded":"2023-10-18T10:25:08.371654+0000","last_fullsized":"2023-10-18T10:25:08.371654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250534+0000","last_change":"2023-10-18T10:25:03.204327+0000","last_active":"2023-10-18T10:25:08.250534+0000","last_peered":"2023-10-18T10:25:08.250534+0000","last_clean":"2023-10-18T10:25:08.250534+0000","last_became_active":"2023-10-18T10:25:03.204079+0000","last_became_peered":"2023-10-18T10:25:03.204079+0000","last_unstale":"2023-10-18T10:25:08.250534+0000","last_undegraded":"2023-10-18T10:25:08.250534+0000","last_fullsized":"2023-10-18T10:25:08.250534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:05.461286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307373+0000","last_change":"2023-10-18T10:25:08.268324+0000","last_active":"2023-10-18T10:25:10.307373+0000","last_peered":"2023-10-18T10:25:10.307373+0000","last_clean":"2023-10-18T10:25:10.307373+0000","last_became_active":"2023-10-18T10:25:08.267959+0000","last_became_peered":"2023-10-18T10:25:08.267959+0000","last_unstale":"2023-10-18T10:25:10.307373+0000","last_undegraded":"2023-10-18T10:25:10.307373+0000","last_fullsized":"2023-10-18T10:25:10.307373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:47.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263268+0000","last_change":"2023-10-18T10:25:04.234809+0000","last_active":"2023-10-18T10:25:08.263268+0000","last_peered":"2023-10-18T10:25:08.263268+0000","last_clean":"2023-10-18T10:25:08.263268+0000","last_became_active":"2023-10-18T10:25:04.234527+0000","last_became_peered":"2023-10-18T10:25:04.234527+0000","last_unstale":"2023-10-18T10:25:08.263268+0000","last_undegraded":"2023-10-18T10:25:08.263268+0000","last_fullsized":"2023-10-18T10:25:08.263268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263204+0000","last_change":"2023-10-18T10:25:03.207281+0000","last_active":"2023-10-18T10:25:08.263204+0000","last_peered":"2023-10-18T10:25:08.263204+0000","last_clean":"2023-10-18T10:25:08.263204+0000","last_became_active":"2023-10-18T10:25:03.206828+0000","last_became_peered":"2023-10-18T10:25:03.206828+0000","last_unstale":"2023-10-18T10:25:08.263204+0000","last_undegraded":"2023-10-18T10:25:08.263204+0000","last_fullsized":"2023-10-18T10:25:08.263204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921617+0000","last_change":"2023-10-18T10:25:08.372503+0000","last_active":"2023-10-18T10:25:09.921617+0000","last_peered":"2023-10-18T10:25:09.921617+0000","last_clean":"2023-10-18T10:25:09.921617+0000","last_became_active":"2023-10-18T10:25:08.371994+0000","last_became_peered":"2023-10-18T10:25:08.371994+0000","last_unstale":"2023-10-18T10:25:09.921617+0000","last_undegraded":"2023-10-18T10:25:09.921617+0000","last_fullsized":"2023-10-18T10:25:09.921617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.553280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370729+0000","last_change":"2023-10-18T10:25:04.237797+0000","last_active":"2023-10-18T10:25:08.370729+0000","last_peered":"2023-10-18T10:25:08.370729+0000","last_clean":"2023-10-18T10:25:08.370729+0000","last_became_active":"2023-10-18T10:25:04.237473+0000","last_became_peered":"2023-10-18T10:25:04.237473+0000","last_unstale":"2023-10-18T10:25:08.370729+0000","last_undegraded":"2023-10-18T10:25:08.370729+0000","last_fullsized":"2023-10-18T10:25:08.370729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:48.992878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254912+0000","last_change":"2023-10-18T10:25:03.213281+0000","last_active":"2023-10-18T10:25:08.254912+0000","last_peered":"2023-10-18T10:25:08.254912+0000","last_clean":"2023-10-18T10:25:08.254912+0000","last_became_active":"2023-10-18T10:25:03.213019+0000","last_became_peered":"2023-10-18T10:25:03.213019+0000","last_unstale":"2023-10-18T10:25:08.254912+0000","last_undegraded":"2023-10-18T10:25:08.254912+0000","last_fullsized":"2023-10-18T10:25:08.254912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:07.702798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921380+0000","last_change":"2023-10-18T10:25:08.370771+0000","last_active":"2023-10-18T10:25:09.921380+0000","last_peered":"2023-10-18T10:25:09.921380+0000","last_clean":"2023-10-18T10:25:09.921380+0000","last_became_active":"2023-10-18T10:25:08.370438+0000","last_became_peered":"2023-10-18T10:25:08.370438+0000","last_unstale":"2023-10-18T10:25:09.921380+0000","last_undegraded":"2023-10-18T10:25:09.921380+0000","last_fullsized":"2023-10-18T10:25:09.921380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:31.794489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254101+0000","last_change":"2023-10-18T10:25:04.232734+0000","last_active":"2023-10-18T10:25:08.254101+0000","last_peered":"2023-10-18T10:25:08.254101+0000","last_clean":"2023-10-18T10:25:08.254101+0000","last_became_active":"2023-10-18T10:25:04.232474+0000","last_became_peered":"2023-10-18T10:25:04.232474+0000","last_unstale":"2023-10-18T10:25:08.254101+0000","last_undegraded":"2023-10-18T10:25:08.254101+0000","last_fullsized":"2023-10-18T10:25:08.254101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257928+0000","last_change":"2023-10-18T10:25:03.202060+0000","last_active":"2023-10-18T10:25:08.257928+0000","last_peered":"2023-10-18T10:25:08.257928+0000","last_clean":"2023-10-18T10:25:08.257928+0000","last_became_active":"2023-10-18T10:25:03.201719+0000","last_became_peered":"2023-10-18T10:25:03.201719+0000","last_unstale":"2023-10-18T10:25:08.257928+0000","last_undegraded":"2023-10-18T10:25:08.257928+0000","last_fullsized":"2023-10-18T10:25:08.257928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.308982+0000","last_change":"2023-10-18T10:25:08.370830+0000","last_active":"2023-10-18T10:25:10.308982+0000","last_peered":"2023-10-18T10:25:10.308982+0000","last_clean":"2023-10-18T10:25:10.308982+0000","last_became_active":"2023-10-18T10:25:08.370387+0000","last_became_peered":"2023-10-18T10:25:08.370387+0000","last_unstale":"2023-10-18T10:25:10.308982+0000","last_undegraded":"2023-10-18T10:25:10.308982+0000","last_fullsized":"2023-10-18T10:25:10.308982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:26.275317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370740+0000","last_change":"2023-10-18T10:25:04.241881+0000","last_active":"2023-10-18T10:25:08.370740+0000","last_peered":"2023-10-18T10:25:08.370740+0000","last_clean":"2023-10-18T10:25:08.370740+0000","last_became_active":"2023-10-18T10:25:04.241615+0000","last_became_peered":"2023-10-18T10:25:04.241615+0000","last_unstale":"2023-10-18T10:25:08.370740+0000","last_undegraded":"2023-10-18T10:25:08.370740+0000","last_fullsized":"2023-10-18T10:25:08.370740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:13.490096+0000","last_change":"2023-10-18T10:25:03.211150+0000","last_active":"2023-10-18T10:25:13.490096+0000","last_peered":"2023-10-18T10:25:13.490096+0000","last_clean":"2023-10-18T10:25:13.490096+0000","last_became_active":"2023-10-18T10:25:03.210879+0000","last_became_peered":"2023-10-18T10:25:03.210879+0000","last_unstale":"2023-10-18T10:25:13.490096+0000","last_undegraded":"2023-10-18T10:25:13.490096+0000","last_fullsized":"2023-10-18T10:25:13.490096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262397+0000","last_change":"2023-10-18T10:25:08.268389+0000","last_active":"2023-10-18T10:25:09.262397+0000","last_peered":"2023-10-18T10:25:09.262397+0000","last_clean":"2023-10-18T10:25:09.262397+0000","last_became_active":"2023-10-18T10:25:08.268025+0000","last_became_peered":"2023-10-18T10:25:08.268025+0000","last_unstale":"2023-10-18T10:25:09.262397+0000","last_undegraded":"2023-10-18T10:25:09.262397+0000","last_fullsized":"2023-10-18T10:25:09.262397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256542+0000","last_change":"2023-10-18T10:25:04.229742+0000","last_active":"2023-10-18T10:25:08.256542+0000","last_peered":"2023-10-18T10:25:08.256542+0000","last_clean":"2023-10-18T10:25:08.256542+0000","last_became_active":"2023-10-18T10:25:04.229281+0000","last_became_peered":"2023-10-18T10:25:04.229281+0000","last_unstale":"2023-10-18T10:25:08.256542+0000","last_undegraded":"2023-10-18T10:25:08.256542+0000","last_fullsized":"2023-10-18T10:25:08.256542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:08.704852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253721+0000","last_change":"2023-10-18T10:25:03.213327+0000","last_active":"2023-10-18T10:25:08.253721+0000","last_peered":"2023-10-18T10:25:08.253721+0000","last_clean":"2023-10-18T10:25:08.253721+0000","last_became_active":"2023-10-18T10:25:03.212993+0000","last_became_peered":"2023-10-18T10:25:03.212993+0000","last_unstale":"2023-10-18T10:25:08.253721+0000","last_undegraded":"2023-10-18T10:25:08.253721+0000","last_fullsized":"2023-10-18T10:25:08.253721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263300+0000","last_change":"2023-10-18T10:25:08.271366+0000","last_active":"2023-10-18T10:25:09.263300+0000","last_peered":"2023-10-18T10:25:09.263300+0000","last_clean":"2023-10-18T10:25:09.263300+0000","last_became_active":"2023-10-18T10:25:08.270581+0000","last_became_peered":"2023-10-18T10:25:08.270581+0000","last_unstale":"2023-10-18T10:25:09.263300+0000","last_undegraded":"2023-10-18T10:25:09.263300+0000","last_fullsized":"2023-10-18T10:25:09.263300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:57.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254604+0000","last_change":"2023-10-18T10:25:04.234658+0000","last_active":"2023-10-18T10:25:08.254604+0000","last_peered":"2023-10-18T10:25:08.254604+0000","last_clean":"2023-10-18T10:25:08.254604+0000","last_became_active":"2023-10-18T10:25:04.234334+0000","last_became_peered":"2023-10-18T10:25:04.234334+0000","last_unstale":"2023-10-18T10:25:08.254604+0000","last_undegraded":"2023-10-18T10:25:08.254604+0000","last_fullsized":"2023-10-18T10:25:08.254604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254170+0000","last_change":"2023-10-18T10:25:03.213955+0000","last_active":"2023-10-18T10:25:08.254170+0000","last_peered":"2023-10-18T10:25:08.254170+0000","last_clean":"2023-10-18T10:25:08.254170+0000","last_became_active":"2023-10-18T10:25:03.213692+0000","last_became_peered":"2023-10-18T10:25:03.213692+0000","last_unstale":"2023-10-18T10:25:08.254170+0000","last_undegraded":"2023-10-18T10:25:08.254170+0000","last_fullsized":"2023-10-18T10:25:08.254170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:21.099526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920943+0000","last_change":"2023-10-18T10:25:08.271334+0000","last_active":"2023-10-18T10:25:09.920943+0000","last_peered":"2023-10-18T10:25:09.920943+0000","last_clean":"2023-10-18T10:25:09.920943+0000","last_became_active":"2023-10-18T10:25:08.270900+0000","last_became_peered":"2023-10-18T10:25:08.270900+0000","last_unstale":"2023-10-18T10:25:09.920943+0000","last_undegraded":"2023-10-18T10:25:09.920943+0000","last_fullsized":"2023-10-18T10:25:09.920943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:53:25.617273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254094+0000","last_change":"2023-10-18T10:25:04.238611+0000","last_active":"2023-10-18T10:25:08.254094+0000","last_peered":"2023-10-18T10:25:08.254094+0000","last_clean":"2023-10-18T10:25:08.254094+0000","last_became_active":"2023-10-18T10:25:04.238304+0000","last_became_peered":"2023-10-18T10:25:04.238304+0000","last_unstale":"2023-10-18T10:25:08.254094+0000","last_undegraded":"2023-10-18T10:25:08.254094+0000","last_fullsized":"2023-10-18T10:25:08.254094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264215+0000","last_change":"2023-10-18T10:25:03.213189+0000","last_active":"2023-10-18T10:25:08.264215+0000","last_peered":"2023-10-18T10:25:08.264215+0000","last_clean":"2023-10-18T10:25:08.264215+0000","last_became_active":"2023-10-18T10:25:03.212915+0000","last_became_peered":"2023-10-18T10:25:03.212915+0000","last_unstale":"2023-10-18T10:25:08.264215+0000","last_undegraded":"2023-10-18T10:25:08.264215+0000","last_fullsized":"2023-10-18T10:25:08.264215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920935+0000","last_change":"2023-10-18T10:25:08.374903+0000","last_active":"2023-10-18T10:25:09.920935+0000","last_peered":"2023-10-18T10:25:09.920935+0000","last_clean":"2023-10-18T10:25:09.920935+0000","last_became_active":"2023-10-18T10:25:08.374525+0000","last_became_peered":"2023-10-18T10:25:08.374525+0000","last_unstale":"2023-10-18T10:25:09.920935+0000","last_undegraded":"2023-10-18T10:25:09.920935+0000","last_fullsized":"2023-10-18T10:25:09.920935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:38.455325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307246+0000","last_change":"2023-10-18T10:25:08.372484+0000","last_active":"2023-10-18T10:25:10.307246+0000","last_peered":"2023-10-18T10:25:10.307246+0000","last_clean":"2023-10-18T10:25:10.307246+0000","last_became_active":"2023-10-18T10:25:08.372132+0000","last_became_peered":"2023-10-18T10:25:08.372132+0000","last_unstale":"2023-10-18T10:25:10.307246+0000","last_undegraded":"2023-10-18T10:25:10.307246+0000","last_fullsized":"2023-10-18T10:25:10.307246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253285+0000","last_change":"2023-10-18T10:25:04.239197+0000","last_active":"2023-10-18T10:25:08.253285+0000","last_peered":"2023-10-18T10:25:08.253285+0000","last_clean":"2023-10-18T10:25:08.253285+0000","last_became_active":"2023-10-18T10:25:04.238917+0000","last_became_peered":"2023-10-18T10:25:04.238917+0000","last_unstale":"2023-10-18T10:25:08.253285+0000","last_undegraded":"2023-10-18T10:25:08.253285+0000","last_fullsized":"2023-10-18T10:25:08.253285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254371+0000","last_change":"2023-10-18T10:25:03.207877+0000","last_active":"2023-10-18T10:25:08.254371+0000","last_peered":"2023-10-18T10:25:08.254371+0000","last_clean":"2023-10-18T10:25:08.254371+0000","last_became_active":"2023-10-18T10:25:03.207621+0000","last_became_peered":"2023-10-18T10:25:03.207621+0000","last_unstale":"2023-10-18T10:25:08.254371+0000","last_undegraded":"2023-10-18T10:25:08.254371+0000","last_fullsized":"2023-10-18T10:25:08.254371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:52.476561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.308246+0000","last_change":"2023-10-18T10:25:08.272814+0000","last_active":"2023-10-18T10:25:10.308246+0000","last_peered":"2023-10-18T10:25:10.308246+0000","last_clean":"2023-10-18T10:25:10.308246+0000","last_became_active":"2023-10-18T10:25:08.272372+0000","last_became_peered":"2023-10-18T10:25:08.272372+0000","last_unstale":"2023-10-18T10:25:10.308246+0000","last_undegraded":"2023-10-18T10:25:10.308246+0000","last_fullsized":"2023-10-18T10:25:10.308246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254375+0000","last_change":"2023-10-18T10:25:04.237028+0000","last_active":"2023-10-18T10:25:08.254375+0000","last_peered":"2023-10-18T10:25:08.254375+0000","last_clean":"2023-10-18T10:25:08.254375+0000","last_became_active":"2023-10-18T10:25:04.236620+0000","last_became_peered":"2023-10-18T10:25:04.236620+0000","last_unstale":"2023-10-18T10:25:08.254375+0000","last_undegraded":"2023-10-18T10:25:08.254375+0000","last_fullsized":"2023-10-18T10:25:08.254375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255920+0000","last_change":"2023-10-18T10:25:03.212737+0000","last_active":"2023-10-18T10:25:08.255920+0000","last_peered":"2023-10-18T10:25:08.255920+0000","last_clean":"2023-10-18T10:25:08.255920+0000","last_became_active":"2023-10-18T10:25:03.212444+0000","last_became_peered":"2023-10-18T10:25:03.212444+0000","last_unstale":"2023-10-18T10:25:08.255920+0000","last_undegraded":"2023-10-18T10:25:08.255920+0000","last_fullsized":"2023-10-18T10:25:08.255920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:53.307830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265148+0000","last_change":"2023-10-18T10:25:03.208047+0000","last_active":"2023-10-18T10:25:08.265148+0000","last_peered":"2023-10-18T10:25:08.265148+0000","last_clean":"2023-10-18T10:25:08.265148+0000","last_became_active":"2023-10-18T10:25:03.207745+0000","last_became_peered":"2023-10-18T10:25:03.207745+0000","last_unstale":"2023-10-18T10:25:08.265148+0000","last_undegraded":"2023-10-18T10:25:08.265148+0000","last_fullsized":"2023-10-18T10:25:08.265148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255246+0000","last_change":"2023-10-18T10:25:04.240334+0000","last_active":"2023-10-18T10:25:08.255246+0000","last_peered":"2023-10-18T10:25:08.255246+0000","last_clean":"2023-10-18T10:25:08.255246+0000","last_became_active":"2023-10-18T10:25:04.239964+0000","last_became_peered":"2023-10-18T10:25:04.239964+0000","last_unstale":"2023-10-18T10:25:08.255246+0000","last_undegraded":"2023-10-18T10:25:08.255246+0000","last_fullsized":"2023-10-18T10:25:08.255246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.294555+0000","last_change":"2023-10-18T10:25:08.264225+0000","last_active":"2023-10-18T10:25:09.294555+0000","last_peered":"2023-10-18T10:25:09.294555+0000","last_clean":"2023-10-18T10:25:09.294555+0000","last_became_active":"2023-10-18T10:25:08.263712+0000","last_became_peered":"2023-10-18T10:25:08.263712+0000","last_unstale":"2023-10-18T10:25:09.294555+0000","last_undegraded":"2023-10-18T10:25:09.294555+0000","last_fullsized":"2023-10-18T10:25:09.294555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:13.156323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253645+0000","last_change":"2023-10-18T10:25:03.211608+0000","last_active":"2023-10-18T10:25:08.253645+0000","last_peered":"2023-10-18T10:25:08.253645+0000","last_clean":"2023-10-18T10:25:08.253645+0000","last_became_active":"2023-10-18T10:25:03.211346+0000","last_became_peered":"2023-10-18T10:25:03.211346+0000","last_unstale":"2023-10-18T10:25:08.253645+0000","last_undegraded":"2023-10-18T10:25:08.253645+0000","last_fullsized":"2023-10-18T10:25:08.253645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255576+0000","last_change":"2023-10-18T10:25:04.242544+0000","last_active":"2023-10-18T10:25:08.255576+0000","last_peered":"2023-10-18T10:25:08.255576+0000","last_clean":"2023-10-18T10:25:08.255576+0000","last_became_active":"2023-10-18T10:25:04.242258+0000","last_became_peered":"2023-10-18T10:25:04.242258+0000","last_unstale":"2023-10-18T10:25:08.255576+0000","last_undegraded":"2023-10-18T10:25:08.255576+0000","last_fullsized":"2023-10-18T10:25:08.255576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:55.422361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921493+0000","last_change":"2023-10-18T10:25:08.273772+0000","last_active":"2023-10-18T10:25:09.921493+0000","last_peered":"2023-10-18T10:25:09.921493+0000","last_clean":"2023-10-18T10:25:09.921493+0000","last_became_active":"2023-10-18T10:25:08.273410+0000","last_became_peered":"2023-10-18T10:25:08.273410+0000","last_unstale":"2023-10-18T10:25:09.921493+0000","last_undegraded":"2023-10-18T10:25:09.921493+0000","last_fullsized":"2023-10-18T10:25:09.921493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:19.382604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263176+0000","last_change":"2023-10-18T10:25:04.235479+0000","last_active":"2023-10-18T10:25:08.263176+0000","last_peered":"2023-10-18T10:25:08.263176+0000","last_clean":"2023-10-18T10:25:08.263176+0000","last_became_active":"2023-10-18T10:25:04.235205+0000","last_became_peered":"2023-10-18T10:25:04.235205+0000","last_unstale":"2023-10-18T10:25:08.263176+0000","last_undegraded":"2023-10-18T10:25:08.263176+0000","last_fullsized":"2023-10-18T10:25:08.263176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250534+0000","last_change":"2023-10-18T10:25:04.236754+0000","last_active":"2023-10-18T10:25:08.250534+0000","last_peered":"2023-10-18T10:25:08.250534+0000","last_clean":"2023-10-18T10:25:08.250534+0000","last_became_active":"2023-10-18T10:25:04.236482+0000","last_became_peered":"2023-10-18T10:25:04.236482+0000","last_unstale":"2023-10-18T10:25:08.250534+0000","last_undegraded":"2023-10-18T10:25:08.250534+0000","last_fullsized":"2023-10-18T10:25:08.250534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:08.704852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254828+0000","last_change":"2023-10-18T10:25:04.239429+0000","last_active":"2023-10-18T10:25:08.254828+0000","last_peered":"2023-10-18T10:25:08.254828+0000","last_clean":"2023-10-18T10:25:08.254828+0000","last_became_active":"2023-10-18T10:25:04.239185+0000","last_became_peered":"2023-10-18T10:25:04.239185+0000","last_unstale":"2023-10-18T10:25:08.254828+0000","last_undegraded":"2023-10-18T10:25:08.254828+0000","last_fullsized":"2023-10-18T10:25:08.254828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:41.123375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255164+0000","last_change":"2023-10-18T10:25:04.235523+0000","last_active":"2023-10-18T10:25:08.255164+0000","last_peered":"2023-10-18T10:25:08.255164+0000","last_clean":"2023-10-18T10:25:08.255164+0000","last_became_active":"2023-10-18T10:25:04.235266+0000","last_became_peered":"2023-10-18T10:25:04.235266+0000","last_unstale":"2023-10-18T10:25:08.255164+0000","last_undegraded":"2023-10-18T10:25:08.255164+0000","last_fullsized":"2023-10-18T10:25:08.255164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254545+0000","last_change":"2023-10-18T10:25:04.225083+0000","last_active":"2023-10-18T10:25:08.254545+0000","last_peered":"2023-10-18T10:25:08.254545+0000","last_clean":"2023-10-18T10:25:08.254545+0000","last_became_active":"2023-10-18T10:25:04.224243+0000","last_became_peered":"2023-10-18T10:25:04.224243+0000","last_unstale":"2023-10-18T10:25:08.254545+0000","last_undegraded":"2023-10-18T10:25:08.254545+0000","last_fullsized":"2023-10-18T10:25:08.254545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253638+0000","last_change":"2023-10-18T10:25:04.239380+0000","last_active":"2023-10-18T10:25:08.253638+0000","last_peered":"2023-10-18T10:25:08.253638+0000","last_clean":"2023-10-18T10:25:08.253638+0000","last_became_active":"2023-10-18T10:25:04.239121+0000","last_became_peered":"2023-10-18T10:25:04.239121+0000","last_unstale":"2023-10-18T10:25:08.253638+0000","last_undegraded":"2023-10-18T10:25:08.253638+0000","last_fullsized":"2023-10-18T10:25:08.253638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265703+0000","last_change":"2023-10-18T10:25:04.231489+0000","last_active":"2023-10-18T10:25:08.265703+0000","last_peered":"2023-10-18T10:25:08.265703+0000","last_clean":"2023-10-18T10:25:08.265703+0000","last_became_active":"2023-10-18T10:25:04.231228+0000","last_became_peered":"2023-10-18T10:25:04.231228+0000","last_unstale":"2023-10-18T10:25:08.265703+0000","last_undegraded":"2023-10-18T10:25:08.265703+0000","last_fullsized":"2023-10-18T10:25:08.265703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255438+0000","last_change":"2023-10-18T10:25:04.226121+0000","last_active":"2023-10-18T10:25:08.255438+0000","last_peered":"2023-10-18T10:25:08.255438+0000","last_clean":"2023-10-18T10:25:08.255438+0000","last_became_active":"2023-10-18T10:25:04.225472+0000","last_became_peered":"2023-10-18T10:25:04.225472+0000","last_unstale":"2023-10-18T10:25:08.255438+0000","last_undegraded":"2023-10-18T10:25:08.255438+0000","last_fullsized":"2023-10-18T10:25:08.255438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254296+0000","last_change":"2023-10-18T10:25:04.239623+0000","last_active":"2023-10-18T10:25:08.254296+0000","last_peered":"2023-10-18T10:25:08.254296+0000","last_clean":"2023-10-18T10:25:08.254296+0000","last_became_active":"2023-10-18T10:25:04.239376+0000","last_became_peered":"2023-10-18T10:25:04.239376+0000","last_unstale":"2023-10-18T10:25:08.254296+0000","last_undegraded":"2023-10-18T10:25:08.254296+0000","last_fullsized":"2023-10-18T10:25:08.254296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.789045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255778+0000","last_change":"2023-10-18T10:25:04.224851+0000","last_active":"2023-10-18T10:25:08.255778+0000","last_peered":"2023-10-18T10:25:08.255778+0000","last_clean":"2023-10-18T10:25:08.255778+0000","last_became_active":"2023-10-18T10:25:04.224579+0000","last_became_peered":"2023-10-18T10:25:04.224579+0000","last_unstale":"2023-10-18T10:25:08.255778+0000","last_undegraded":"2023-10-18T10:25:08.255778+0000","last_fullsized":"2023-10-18T10:25:08.255778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254745+0000","last_change":"2023-10-18T10:25:04.233040+0000","last_active":"2023-10-18T10:25:08.254745+0000","last_peered":"2023-10-18T10:25:08.254745+0000","last_clean":"2023-10-18T10:25:08.254745+0000","last_became_active":"2023-10-18T10:25:04.232782+0000","last_became_peered":"2023-10-18T10:25:04.232782+0000","last_unstale":"2023-10-18T10:25:08.254745+0000","last_undegraded":"2023-10-18T10:25:08.254745+0000","last_fullsized":"2023-10-18T10:25:08.254745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255053+0000","last_change":"2023-10-18T10:25:04.235058+0000","last_active":"2023-10-18T10:25:08.255053+0000","last_peered":"2023-10-18T10:25:08.255053+0000","last_clean":"2023-10-18T10:25:08.255053+0000","last_became_active":"2023-10-18T10:25:04.234763+0000","last_became_peered":"2023-10-18T10:25:04.234763+0000","last_unstale":"2023-10-18T10:25:08.255053+0000","last_undegraded":"2023-10-18T10:25:08.255053+0000","last_fullsized":"2023-10-18T10:25:08.255053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263580+0000","last_change":"2023-10-18T10:25:04.231136+0000","last_active":"2023-10-18T10:25:08.263580+0000","last_peered":"2023-10-18T10:25:08.263580+0000","last_clean":"2023-10-18T10:25:08.263580+0000","last_became_active":"2023-10-18T10:25:04.230835+0000","last_became_peered":"2023-10-18T10:25:04.230835+0000","last_unstale":"2023-10-18T10:25:08.263580+0000","last_undegraded":"2023-10-18T10:25:08.263580+0000","last_fullsized":"2023-10-18T10:25:08.263580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370824+0000","last_change":"2023-10-18T10:25:04.236843+0000","last_active":"2023-10-18T10:25:08.370824+0000","last_peered":"2023-10-18T10:25:08.370824+0000","last_clean":"2023-10-18T10:25:08.370824+0000","last_became_active":"2023-10-18T10:25:04.236484+0000","last_became_peered":"2023-10-18T10:25:04.236484+0000","last_unstale":"2023-10-18T10:25:08.370824+0000","last_undegraded":"2023-10-18T10:25:08.370824+0000","last_fullsized":"2023-10-18T10:25:08.370824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.789045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250697+0000","last_change":"2023-10-18T10:25:04.230906+0000","last_active":"2023-10-18T10:25:08.250697+0000","last_peered":"2023-10-18T10:25:08.250697+0000","last_clean":"2023-10-18T10:25:08.250697+0000","last_became_active":"2023-10-18T10:25:04.230641+0000","last_became_peered":"2023-10-18T10:25:04.230641+0000","last_unstale":"2023-10-18T10:25:08.250697+0000","last_undegraded":"2023-10-18T10:25:08.250697+0000","last_fullsized":"2023-10-18T10:25:08.250697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:44.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254323+0000","last_change":"2023-10-18T10:25:04.235839+0000","last_active":"2023-10-18T10:25:08.254323+0000","last_peered":"2023-10-18T10:25:08.254323+0000","last_clean":"2023-10-18T10:25:08.254323+0000","last_became_active":"2023-10-18T10:25:04.235602+0000","last_became_peered":"2023-10-18T10:25:04.235602+0000","last_unstale":"2023-10-18T10:25:08.254323+0000","last_undegraded":"2023-10-18T10:25:08.254323+0000","last_fullsized":"2023-10-18T10:25:08.254323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254219+0000","last_change":"2023-10-18T10:25:04.235249+0000","last_active":"2023-10-18T10:25:08.254219+0000","last_peered":"2023-10-18T10:25:08.254219+0000","last_clean":"2023-10-18T10:25:08.254219+0000","last_became_active":"2023-10-18T10:25:04.234967+0000","last_became_peered":"2023-10-18T10:25:04.234967+0000","last_unstale":"2023-10-18T10:25:08.254219+0000","last_undegraded":"2023-10-18T10:25:08.254219+0000","last_fullsized":"2023-10-18T10:25:08.254219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253370+0000","last_change":"2023-10-18T10:25:04.232541+0000","last_active":"2023-10-18T10:25:08.253370+0000","last_peered":"2023-10-18T10:25:08.253370+0000","last_clean":"2023-10-18T10:25:08.253370+0000","last_became_active":"2023-10-18T10:25:04.231594+0000","last_became_peered":"2023-10-18T10:25:04.231594+0000","last_unstale":"2023-10-18T10:25:08.253370+0000","last_undegraded":"2023-10-18T10:25:08.253370+0000","last_fullsized":"2023-10-18T10:25:08.253370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254018+0000","last_change":"2023-10-18T10:25:04.239452+0000","last_active":"2023-10-18T10:25:08.254018+0000","last_peered":"2023-10-18T10:25:08.254018+0000","last_clean":"2023-10-18T10:25:08.254018+0000","last_became_active":"2023-10-18T10:25:04.239175+0000","last_became_peered":"2023-10-18T10:25:04.239175+0000","last_unstale":"2023-10-18T10:25:08.254018+0000","last_undegraded":"2023-10-18T10:25:08.254018+0000","last_fullsized":"2023-10-18T10:25:08.254018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:42.041557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253482+0000","last_change":"2023-10-18T10:25:04.225873+0000","last_active":"2023-10-18T10:25:08.253482+0000","last_peered":"2023-10-18T10:25:08.253482+0000","last_clean":"2023-10-18T10:25:08.253482+0000","last_became_active":"2023-10-18T10:25:04.225624+0000","last_became_peered":"2023-10-18T10:25:04.225624+0000","last_unstale":"2023-10-18T10:25:08.253482+0000","last_undegraded":"2023-10-18T10:25:08.253482+0000","last_fullsized":"2023-10-18T10:25:08.253482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256440+0000","last_change":"2023-10-18T10:25:04.226107+0000","last_active":"2023-10-18T10:25:08.256440+0000","last_peered":"2023-10-18T10:25:08.256440+0000","last_clean":"2023-10-18T10:25:08.256440+0000","last_became_active":"2023-10-18T10:25:04.225823+0000","last_became_peered":"2023-10-18T10:25:04.225823+0000","last_unstale":"2023-10-18T10:25:08.256440+0000","last_undegraded":"2023-10-18T10:25:08.256440+0000","last_fullsized":"2023-10-18T10:25:08.256440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:35.440931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263514+0000","last_change":"2023-10-18T10:25:04.237730+0000","last_active":"2023-10-18T10:25:08.263514+0000","last_peered":"2023-10-18T10:25:08.263514+0000","last_clean":"2023-10-18T10:25:08.263514+0000","last_became_active":"2023-10-18T10:25:04.237436+0000","last_became_peered":"2023-10-18T10:25:04.237436+0000","last_unstale":"2023-10-18T10:25:08.263514+0000","last_undegraded":"2023-10-18T10:25:08.263514+0000","last_fullsized":"2023-10-18T10:25:08.263514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264090+0000","last_change":"2023-10-18T10:25:04.242105+0000","last_active":"2023-10-18T10:25:08.264090+0000","last_peered":"2023-10-18T10:25:08.264090+0000","last_clean":"2023-10-18T10:25:08.264090+0000","last_became_active":"2023-10-18T10:25:04.241762+0000","last_became_peered":"2023-10-18T10:25:04.241762+0000","last_unstale":"2023-10-18T10:25:08.264090+0000","last_undegraded":"2023-10-18T10:25:08.264090+0000","last_fullsized":"2023-10-18T10:25:08.264090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371062+0000","last_change":"2023-10-18T10:25:04.239055+0000","last_active":"2023-10-18T10:25:08.371062+0000","last_peered":"2023-10-18T10:25:08.371062+0000","last_clean":"2023-10-18T10:25:08.371062+0000","last_became_active":"2023-10-18T10:25:04.238566+0000","last_became_peered":"2023-10-18T10:25:04.238566+0000","last_unstale":"2023-10-18T10:25:08.371062+0000","last_undegraded":"2023-10-18T10:25:08.371062+0000","last_fullsized":"2023-10-18T10:25:08.371062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:43.022445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254270+0000","last_change":"2023-10-18T10:25:04.238819+0000","last_active":"2023-10-18T10:25:08.254270+0000","last_peered":"2023-10-18T10:25:08.254270+0000","last_clean":"2023-10-18T10:25:08.254270+0000","last_became_active":"2023-10-18T10:25:04.238546+0000","last_became_peered":"2023-10-18T10:25:04.238546+0000","last_unstale":"2023-10-18T10:25:08.254270+0000","last_undegraded":"2023-10-18T10:25:08.254270+0000","last_fullsized":"2023-10-18T10:25:08.254270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371571+0000","last_change":"2023-10-18T10:25:04.234955+0000","last_active":"2023-10-18T10:25:08.371571+0000","last_peered":"2023-10-18T10:25:08.371571+0000","last_clean":"2023-10-18T10:25:08.371571+0000","last_became_active":"2023-10-18T10:25:04.234687+0000","last_became_peered":"2023-10-18T10:25:04.234687+0000","last_unstale":"2023-10-18T10:25:08.371571+0000","last_undegraded":"2023-10-18T10:25:08.371571+0000","last_fullsized":"2023-10-18T10:25:08.371571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254664+0000","last_change":"2023-10-18T10:25:04.235078+0000","last_active":"2023-10-18T10:25:08.254664+0000","last_peered":"2023-10-18T10:25:08.254664+0000","last_clean":"2023-10-18T10:25:08.254664+0000","last_became_active":"2023-10-18T10:25:04.234817+0000","last_became_peered":"2023-10-18T10:25:04.234817+0000","last_unstale":"2023-10-18T10:25:08.254664+0000","last_undegraded":"2023-10-18T10:25:08.254664+0000","last_fullsized":"2023-10-18T10:25:08.254664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370554+0000","last_change":"2023-10-18T10:25:04.228077+0000","last_active":"2023-10-18T10:25:08.370554+0000","last_peered":"2023-10-18T10:25:08.370554+0000","last_clean":"2023-10-18T10:25:08.370554+0000","last_became_active":"2023-10-18T10:25:04.227740+0000","last_became_peered":"2023-10-18T10:25:04.227740+0000","last_unstale":"2023-10-18T10:25:08.370554+0000","last_undegraded":"2023-10-18T10:25:08.370554+0000","last_fullsized":"2023-10-18T10:25:08.370554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:26.705420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.258398+0000","last_change":"2023-10-18T10:25:04.240451+0000","last_active":"2023-10-18T10:25:08.258398+0000","last_peered":"2023-10-18T10:25:08.258398+0000","last_clean":"2023-10-18T10:25:08.258398+0000","last_became_active":"2023-10-18T10:25:04.240190+0000","last_became_peered":"2023-10-18T10:25:04.240190+0000","last_unstale":"2023-10-18T10:25:08.258398+0000","last_undegraded":"2023-10-18T10:25:08.258398+0000","last_fullsized":"2023-10-18T10:25:08.258398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:01.626703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254434+0000","last_change":"2023-10-18T10:25:04.233621+0000","last_active":"2023-10-18T10:25:08.254434+0000","last_peered":"2023-10-18T10:25:08.254434+0000","last_clean":"2023-10-18T10:25:08.254434+0000","last_became_active":"2023-10-18T10:25:04.233376+0000","last_became_peered":"2023-10-18T10:25:04.233376+0000","last_unstale":"2023-10-18T10:25:08.254434+0000","last_undegraded":"2023-10-18T10:25:08.254434+0000","last_fullsized":"2023-10-18T10:25:08.254434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:34.587493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255094+0000","last_change":"2023-10-18T10:25:04.234775+0000","last_active":"2023-10-18T10:25:08.255094+0000","last_peered":"2023-10-18T10:25:08.255094+0000","last_clean":"2023-10-18T10:25:08.255094+0000","last_became_active":"2023-10-18T10:25:04.234527+0000","last_became_peered":"2023-10-18T10:25:04.234527+0000","last_unstale":"2023-10-18T10:25:08.255094+0000","last_undegraded":"2023-10-18T10:25:08.255094+0000","last_fullsized":"2023-10-18T10:25:08.255094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370952+0000","last_change":"2023-10-18T10:25:04.236702+0000","last_active":"2023-10-18T10:25:08.370952+0000","last_peered":"2023-10-18T10:25:08.370952+0000","last_clean":"2023-10-18T10:25:08.370952+0000","last_became_active":"2023-10-18T10:25:04.236396+0000","last_became_peered":"2023-10-18T10:25:04.236396+0000","last_unstale":"2023-10-18T10:25:08.370952+0000","last_undegraded":"2023-10-18T10:25:08.370952+0000","last_fullsized":"2023-10-18T10:25:08.370952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:01.626703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:25:25.855 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:25:25.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.989+0000 7f6446275640 1 -- 172.21.15.125:0/2760040643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 msgr2=0x7f6440152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.989+0000 7f6446275640 1 --2- 172.21.15.125:0/2760040643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 0x7f6440152c90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6434009830 tx=0x7f643402bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:25.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.989+0000 7f6446275640 1 -- 172.21.15.125:0/2760040643 shutdown_connections 2023-10-18T10:25:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.989+0000 7f6446275640 1 --2- 172.21.15.125:0/2760040643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64400abc10 0x7f64401531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.989+0000 7f6446275640 1 --2- 172.21.15.125:0/2760040643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 0x7f6440152c90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.989+0000 7f6446275640 1 -- 172.21.15.125:0/2760040643 >> 172.21.15.125:0/2760040643 conn(0x7f6440069930 msgr2=0x7f6440069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.989+0000 7f6446275640 1 -- 172.21.15.125:0/2760040643 shutdown_connections 2023-10-18T10:25:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.989+0000 7f6446275640 1 -- 172.21.15.125:0/2760040643 wait complete. 2023-10-18T10:25:25.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.990+0000 7f6446275640 1 Processor -- start 2023-10-18T10:25:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f6446275640 1 -- start start 2023-10-18T10:25:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f6446275640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64400abc10 0x7f64400b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f6446275640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 0x7f64400aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f6446275640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64400c9dc0 0x7f64400aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f6446275640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64401f5bd0 con 0x7f64400c99e0 2023-10-18T10:25:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f6446275640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64400b01f0 con 0x7f64400abc10 2023-10-18T10:25:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f6446275640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64401f58f0 con 0x7f64400c9dc0 2023-10-18T10:25:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 0x7f64400aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 0x7f64400aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50048/0 (socket says 172.21.15.125:50048) 2023-10-18T10:25:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.991+0000 7f643f7fe640 1 -- 172.21.15.125:0/3556184781 learned_addr learned my addr 172.21.15.125:0/3556184781 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643ffff640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64400abc10 0x7f64400b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f6444812640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64400c9dc0 0x7f64400aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:25.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643f7fe640 1 -- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64400c9dc0 msgr2=0x7f64400aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643f7fe640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64400c9dc0 0x7f64400aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643f7fe640 1 -- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64400abc10 msgr2=0x7f64400b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:25.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643f7fe640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64400abc10 0x7f64400b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:25.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643f7fe640 1 -- 172.21.15.125:0/3556184781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434009490 con 0x7f64400c99e0 2023-10-18T10:25:25.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643f7fe640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 0x7f64400aa8c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f643000c390 tx=0x7f643000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:25.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643cff9640 1 -- 172.21.15.125:0/3556184781 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430011790 con 0x7f64400c99e0 2023-10-18T10:25:25.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643cff9640 1 -- 172.21.15.125:0/3556184781 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64300119e0 con 0x7f64400c99e0 2023-10-18T10:25:25.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f643cff9640 1 -- 172.21.15.125:0/3556184781 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430011db0 con 0x7f64400c99e0 2023-10-18T10:25:25.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64400b0470 con 0x7f64400c99e0 2023-10-18T10:25:25.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.992+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64400b0a30 con 0x7f64400c99e0 2023-10-18T10:25:25.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.993+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64400aea70 con 0x7f64400c99e0 2023-10-18T10:25:26.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.993+0000 7f643cff9640 1 -- 172.21.15.125:0/3556184781 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f643001c460 con 0x7f64400c99e0 2023-10-18T10:25:26.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.994+0000 7f643cff9640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f642c06ae40 0x7f642c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.994+0000 7f643cff9640 1 -- 172.21.15.125:0/3556184781 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6430089980 con 0x7f64400c99e0 2023-10-18T10:25:26.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.994+0000 7f643ffff640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f642c06ae40 0x7f642c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.994+0000 7f643ffff640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f642c06ae40 0x7f642c06d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6434036000 tx=0x7f6434004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:26.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:25.998+0000 7f643cff9640 1 -- 172.21.15.125:0/3556184781 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64400aea70 con 0x7f64400c99e0 2023-10-18T10:25:26.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.159+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f644015d270 con 0x7f642c06ae40 2023-10-18T10:25:26.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.168+0000 7f643cff9640 1 -- 172.21.15.125:0/3556184781 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345206 (secure 0 0 0) 0x7f64340040c0 con 0x7f642c06ae40 2023-10-18T10:25:26.173 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:26.175 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:25:26.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f642c06ae40 msgr2=0x7f642c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f642c06ae40 0x7f642c06d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6434036000 tx=0x7f6434004730 comp rx=0 tx=0).stop 2023-10-18T10:25:26.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 msgr2=0x7f64400aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 0x7f64400aa8c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f643000c390 tx=0x7f643000c850 comp rx=0 tx=0).stop 2023-10-18T10:25:26.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 shutdown_connections 2023-10-18T10:25:26.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f642c06ae40 0x7f642c06d300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64400c9dc0 0x7f64400aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64400c99e0 0x7f64400aa8c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 --2- 172.21.15.125:0/3556184781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64400abc10 0x7f64400b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 >> 172.21.15.125:0/3556184781 conn(0x7f6440069930 msgr2=0x7f64400ac630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:26.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 shutdown_connections 2023-10-18T10:25:26.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.172+0000 7f6446275640 1 -- 172.21.15.125:0/3556184781 wait complete. 2023-10-18T10:25:26.184 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-18T10:25:24.571501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219484,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755236,"statfs":{"total":773094113280,"available":772869361664,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003425"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254754+0000","last_change":"2023-10-18T10:25:03.211565+0000","last_active":"2023-10-18T10:25:08.254754+0000","last_peered":"2023-10-18T10:25:08.254754+0000","last_clean":"2023-10-18T10:25:08.254754+0000","last_became_active":"2023-10-18T10:25:03.211312+0000","last_became_peered":"2023-10-18T10:25:03.211312+0000","last_unstale":"2023-10-18T10:25:08.254754+0000","last_undegraded":"2023-10-18T10:25:08.254754+0000","last_fullsized":"2023-10-18T10:25:08.254754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254543+0000","last_change":"2023-10-18T10:25:04.232540+0000","last_active":"2023-10-18T10:25:08.254543+0000","last_peered":"2023-10-18T10:25:08.254543+0000","last_clean":"2023-10-18T10:25:08.254543+0000","last_became_active":"2023-10-18T10:25:04.232270+0000","last_became_peered":"2023-10-18T10:25:04.232270+0000","last_unstale":"2023-10-18T10:25:08.254543+0000","last_undegraded":"2023-10-18T10:25:08.254543+0000","last_fullsized":"2023-10-18T10:25:08.254543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920838+0000","last_change":"2023-10-18T10:25:08.375121+0000","last_active":"2023-10-18T10:25:09.920838+0000","last_peered":"2023-10-18T10:25:09.920838+0000","last_clean":"2023-10-18T10:25:09.920838+0000","last_became_active":"2023-10-18T10:25:08.374754+0000","last_became_peered":"2023-10-18T10:25:08.374754+0000","last_unstale":"2023-10-18T10:25:09.920838+0000","last_undegraded":"2023-10-18T10:25:09.920838+0000","last_fullsized":"2023-10-18T10:25:09.920838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:26.275317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254691+0000","last_change":"2023-10-18T10:25:03.209283+0000","last_active":"2023-10-18T10:25:08.254691+0000","last_peered":"2023-10-18T10:25:08.254691+0000","last_clean":"2023-10-18T10:25:08.254691+0000","last_became_active":"2023-10-18T10:25:03.208718+0000","last_became_peered":"2023-10-18T10:25:03.208718+0000","last_unstale":"2023-10-18T10:25:08.254691+0000","last_undegraded":"2023-10-18T10:25:08.254691+0000","last_fullsized":"2023-10-18T10:25:08.254691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:43.110925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254524+0000","last_change":"2023-10-18T10:25:04.239316+0000","last_active":"2023-10-18T10:25:08.254524+0000","last_peered":"2023-10-18T10:25:08.254524+0000","last_clean":"2023-10-18T10:25:08.254524+0000","last_became_active":"2023-10-18T10:25:04.235952+0000","last_became_peered":"2023-10-18T10:25:04.235952+0000","last_unstale":"2023-10-18T10:25:08.254524+0000","last_undegraded":"2023-10-18T10:25:08.254524+0000","last_fullsized":"2023-10-18T10:25:08.254524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:09.100769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263035+0000","last_change":"2023-10-18T10:25:08.267055+0000","last_active":"2023-10-18T10:25:09.263035+0000","last_peered":"2023-10-18T10:25:09.263035+0000","last_clean":"2023-10-18T10:25:09.263035+0000","last_became_active":"2023-10-18T10:25:08.266533+0000","last_became_peered":"2023-10-18T10:25:08.266533+0000","last_unstale":"2023-10-18T10:25:09.263035+0000","last_undegraded":"2023-10-18T10:25:09.263035+0000","last_fullsized":"2023-10-18T10:25:09.263035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:31.219679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370926+0000","last_change":"2023-10-18T10:25:03.212550+0000","last_active":"2023-10-18T10:25:08.370926+0000","last_peered":"2023-10-18T10:25:08.370926+0000","last_clean":"2023-10-18T10:25:08.370926+0000","last_became_active":"2023-10-18T10:25:03.211719+0000","last_became_peered":"2023-10-18T10:25:03.211719+0000","last_unstale":"2023-10-18T10:25:08.370926+0000","last_undegraded":"2023-10-18T10:25:08.370926+0000","last_fullsized":"2023-10-18T10:25:08.370926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257891+0000","last_change":"2023-10-18T10:25:04.235362+0000","last_active":"2023-10-18T10:25:08.257891+0000","last_peered":"2023-10-18T10:25:08.257891+0000","last_clean":"2023-10-18T10:25:08.257891+0000","last_became_active":"2023-10-18T10:25:04.235036+0000","last_became_peered":"2023-10-18T10:25:04.235036+0000","last_unstale":"2023-10-18T10:25:08.257891+0000","last_undegraded":"2023-10-18T10:25:08.257891+0000","last_fullsized":"2023-10-18T10:25:08.257891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:26.705420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263812+0000","last_change":"2023-10-18T10:25:08.270146+0000","last_active":"2023-10-18T10:25:09.263812+0000","last_peered":"2023-10-18T10:25:09.263812+0000","last_clean":"2023-10-18T10:25:09.263812+0000","last_became_active":"2023-10-18T10:25:08.269580+0000","last_became_peered":"2023-10-18T10:25:08.269580+0000","last_unstale":"2023-10-18T10:25:09.263812+0000","last_undegraded":"2023-10-18T10:25:09.263812+0000","last_fullsized":"2023-10-18T10:25:09.263812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370610+0000","last_change":"2023-10-18T10:25:03.216100+0000","last_active":"2023-10-18T10:25:08.370610+0000","last_peered":"2023-10-18T10:25:08.370610+0000","last_clean":"2023-10-18T10:25:08.370610+0000","last_became_active":"2023-10-18T10:25:03.215819+0000","last_became_peered":"2023-10-18T10:25:03.215819+0000","last_unstale":"2023-10-18T10:25:08.370610+0000","last_undegraded":"2023-10-18T10:25:08.370610+0000","last_fullsized":"2023-10-18T10:25:08.370610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:43.110925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256236+0000","last_change":"2023-10-18T10:25:04.236788+0000","last_active":"2023-10-18T10:25:08.256236+0000","last_peered":"2023-10-18T10:25:08.256236+0000","last_clean":"2023-10-18T10:25:08.256236+0000","last_became_active":"2023-10-18T10:25:04.236291+0000","last_became_peered":"2023-10-18T10:25:04.236291+0000","last_unstale":"2023-10-18T10:25:08.256236+0000","last_undegraded":"2023-10-18T10:25:08.256236+0000","last_fullsized":"2023-10-18T10:25:08.256236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:39.174524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263153+0000","last_change":"2023-10-18T10:25:08.265913+0000","last_active":"2023-10-18T10:25:09.263153+0000","last_peered":"2023-10-18T10:25:09.263153+0000","last_clean":"2023-10-18T10:25:09.263153+0000","last_became_active":"2023-10-18T10:25:08.264755+0000","last_became_peered":"2023-10-18T10:25:08.264755+0000","last_unstale":"2023-10-18T10:25:09.263153+0000","last_undegraded":"2023-10-18T10:25:09.263153+0000","last_fullsized":"2023-10-18T10:25:09.263153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:22.912468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265054+0000","last_change":"2023-10-18T10:25:04.230696+0000","last_active":"2023-10-18T10:25:08.265054+0000","last_peered":"2023-10-18T10:25:08.265054+0000","last_clean":"2023-10-18T10:25:08.265054+0000","last_became_active":"2023-10-18T10:25:04.230402+0000","last_became_peered":"2023-10-18T10:25:04.230402+0000","last_unstale":"2023-10-18T10:25:08.265054+0000","last_undegraded":"2023-10-18T10:25:08.265054+0000","last_fullsized":"2023-10-18T10:25:08.265054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:34.587493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370834+0000","last_change":"2023-10-18T10:25:03.210831+0000","last_active":"2023-10-18T10:25:08.370834+0000","last_peered":"2023-10-18T10:25:08.370834+0000","last_clean":"2023-10-18T10:25:08.370834+0000","last_became_active":"2023-10-18T10:25:03.210586+0000","last_became_peered":"2023-10-18T10:25:03.210586+0000","last_unstale":"2023-10-18T10:25:08.370834+0000","last_undegraded":"2023-10-18T10:25:08.370834+0000","last_fullsized":"2023-10-18T10:25:08.370834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263925+0000","last_change":"2023-10-18T10:25:08.270065+0000","last_active":"2023-10-18T10:25:09.263925+0000","last_peered":"2023-10-18T10:25:09.263925+0000","last_clean":"2023-10-18T10:25:09.263925+0000","last_became_active":"2023-10-18T10:25:08.269544+0000","last_became_peered":"2023-10-18T10:25:08.269544+0000","last_unstale":"2023-10-18T10:25:09.263925+0000","last_undegraded":"2023-10-18T10:25:09.263925+0000","last_fullsized":"2023-10-18T10:25:09.263925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253560+0000","last_change":"2023-10-18T10:25:04.237642+0000","last_active":"2023-10-18T10:25:08.253560+0000","last_peered":"2023-10-18T10:25:08.253560+0000","last_clean":"2023-10-18T10:25:08.253560+0000","last_became_active":"2023-10-18T10:25:04.237321+0000","last_became_peered":"2023-10-18T10:25:04.237321+0000","last_unstale":"2023-10-18T10:25:08.253560+0000","last_undegraded":"2023-10-18T10:25:08.253560+0000","last_fullsized":"2023-10-18T10:25:08.253560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254793+0000","last_change":"2023-10-18T10:25:03.207491+0000","last_active":"2023-10-18T10:25:08.254793+0000","last_peered":"2023-10-18T10:25:08.254793+0000","last_clean":"2023-10-18T10:25:08.254793+0000","last_became_active":"2023-10-18T10:25:03.207227+0000","last_became_peered":"2023-10-18T10:25:03.207227+0000","last_unstale":"2023-10-18T10:25:08.254793+0000","last_undegraded":"2023-10-18T10:25:08.254793+0000","last_fullsized":"2023-10-18T10:25:08.254793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263777+0000","last_change":"2023-10-18T10:25:08.270542+0000","last_active":"2023-10-18T10:25:09.263777+0000","last_peered":"2023-10-18T10:25:09.263777+0000","last_clean":"2023-10-18T10:25:09.263777+0000","last_became_active":"2023-10-18T10:25:08.270153+0000","last_became_peered":"2023-10-18T10:25:08.270153+0000","last_unstale":"2023-10-18T10:25:09.263777+0000","last_undegraded":"2023-10-18T10:25:09.263777+0000","last_fullsized":"2023-10-18T10:25:09.263777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254454+0000","last_change":"2023-10-18T10:25:04.230256+0000","last_active":"2023-10-18T10:25:08.254454+0000","last_peered":"2023-10-18T10:25:08.254454+0000","last_clean":"2023-10-18T10:25:08.254454+0000","last_became_active":"2023-10-18T10:25:04.230007+0000","last_became_peered":"2023-10-18T10:25:04.230007+0000","last_unstale":"2023-10-18T10:25:08.254454+0000","last_undegraded":"2023-10-18T10:25:08.254454+0000","last_fullsized":"2023-10-18T10:25:08.254454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253450+0000","last_change":"2023-10-18T10:25:03.203658+0000","last_active":"2023-10-18T10:25:08.253450+0000","last_peered":"2023-10-18T10:25:08.253450+0000","last_clean":"2023-10-18T10:25:08.253450+0000","last_became_active":"2023-10-18T10:25:03.203419+0000","last_became_peered":"2023-10-18T10:25:03.203419+0000","last_unstale":"2023-10-18T10:25:08.253450+0000","last_undegraded":"2023-10-18T10:25:08.253450+0000","last_fullsized":"2023-10-18T10:25:08.253450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262990+0000","last_change":"2023-10-18T10:25:08.373967+0000","last_active":"2023-10-18T10:25:09.262990+0000","last_peered":"2023-10-18T10:25:09.262990+0000","last_clean":"2023-10-18T10:25:09.262990+0000","last_became_active":"2023-10-18T10:25:08.373624+0000","last_became_peered":"2023-10-18T10:25:08.373624+0000","last_unstale":"2023-10-18T10:25:09.262990+0000","last_undegraded":"2023-10-18T10:25:09.262990+0000","last_fullsized":"2023-10-18T10:25:09.262990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:47.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257377+0000","last_change":"2023-10-18T10:25:04.227308+0000","last_active":"2023-10-18T10:25:08.257377+0000","last_peered":"2023-10-18T10:25:08.257377+0000","last_clean":"2023-10-18T10:25:08.257377+0000","last_became_active":"2023-10-18T10:25:04.227010+0000","last_became_peered":"2023-10-18T10:25:04.227010+0000","last_unstale":"2023-10-18T10:25:08.257377+0000","last_undegraded":"2023-10-18T10:25:08.257377+0000","last_fullsized":"2023-10-18T10:25:08.257377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:03.438820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369608+0000","last_change":"2023-10-18T10:25:03.210132+0000","last_active":"2023-10-18T10:25:08.369608+0000","last_peered":"2023-10-18T10:25:08.369608+0000","last_clean":"2023-10-18T10:25:08.369608+0000","last_became_active":"2023-10-18T10:25:03.209850+0000","last_became_peered":"2023-10-18T10:25:03.209850+0000","last_unstale":"2023-10-18T10:25:08.369608+0000","last_undegraded":"2023-10-18T10:25:08.369608+0000","last_fullsized":"2023-10-18T10:25:08.369608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263730+0000","last_change":"2023-10-18T10:25:08.270370+0000","last_active":"2023-10-18T10:25:09.263730+0000","last_peered":"2023-10-18T10:25:09.263730+0000","last_clean":"2023-10-18T10:25:09.263730+0000","last_became_active":"2023-10-18T10:25:08.269625+0000","last_became_peered":"2023-10-18T10:25:08.269625+0000","last_unstale":"2023-10-18T10:25:09.263730+0000","last_undegraded":"2023-10-18T10:25:09.263730+0000","last_fullsized":"2023-10-18T10:25:09.263730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:45.178929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257699+0000","last_change":"2023-10-18T10:25:04.239309+0000","last_active":"2023-10-18T10:25:08.257699+0000","last_peered":"2023-10-18T10:25:08.257699+0000","last_clean":"2023-10-18T10:25:08.257699+0000","last_became_active":"2023-10-18T10:25:04.236322+0000","last_became_peered":"2023-10-18T10:25:04.236322+0000","last_unstale":"2023-10-18T10:25:08.257699+0000","last_undegraded":"2023-10-18T10:25:08.257699+0000","last_fullsized":"2023-10-18T10:25:08.257699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264329+0000","last_change":"2023-10-18T10:25:03.211765+0000","last_active":"2023-10-18T10:25:08.264329+0000","last_peered":"2023-10-18T10:25:08.264329+0000","last_clean":"2023-10-18T10:25:08.264329+0000","last_became_active":"2023-10-18T10:25:03.211504+0000","last_became_peered":"2023-10-18T10:25:03.211504+0000","last_unstale":"2023-10-18T10:25:08.264329+0000","last_undegraded":"2023-10-18T10:25:08.264329+0000","last_fullsized":"2023-10-18T10:25:08.264329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:41.039503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263210+0000","last_change":"2023-10-18T10:25:08.374129+0000","last_active":"2023-10-18T10:25:09.263210+0000","last_peered":"2023-10-18T10:25:09.263210+0000","last_clean":"2023-10-18T10:25:09.263210+0000","last_became_active":"2023-10-18T10:25:08.373675+0000","last_became_peered":"2023-10-18T10:25:08.373675+0000","last_unstale":"2023-10-18T10:25:09.263210+0000","last_undegraded":"2023-10-18T10:25:09.263210+0000","last_fullsized":"2023-10-18T10:25:09.263210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:12.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255528+0000","last_change":"2023-10-18T10:25:04.232333+0000","last_active":"2023-10-18T10:25:08.255528+0000","last_peered":"2023-10-18T10:25:08.255528+0000","last_clean":"2023-10-18T10:25:08.255528+0000","last_became_active":"2023-10-18T10:25:04.232075+0000","last_became_peered":"2023-10-18T10:25:04.232075+0000","last_unstale":"2023-10-18T10:25:08.255528+0000","last_undegraded":"2023-10-18T10:25:08.255528+0000","last_fullsized":"2023-10-18T10:25:08.255528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:39.174524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255192+0000","last_change":"2023-10-18T10:25:03.209779+0000","last_active":"2023-10-18T10:25:08.255192+0000","last_peered":"2023-10-18T10:25:08.255192+0000","last_clean":"2023-10-18T10:25:08.255192+0000","last_became_active":"2023-10-18T10:25:03.209508+0000","last_became_peered":"2023-10-18T10:25:03.209508+0000","last_unstale":"2023-10-18T10:25:08.255192+0000","last_undegraded":"2023-10-18T10:25:08.255192+0000","last_fullsized":"2023-10-18T10:25:08.255192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:33.585439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920818+0000","last_change":"2023-10-18T10:25:08.372735+0000","last_active":"2023-10-18T10:25:09.920818+0000","last_peered":"2023-10-18T10:25:09.920818+0000","last_clean":"2023-10-18T10:25:09.920818+0000","last_became_active":"2023-10-18T10:25:08.372386+0000","last_became_peered":"2023-10-18T10:25:08.372386+0000","last_unstale":"2023-10-18T10:25:09.920818+0000","last_undegraded":"2023-10-18T10:25:09.920818+0000","last_fullsized":"2023-10-18T10:25:09.920818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:53.362022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255868+0000","last_change":"2023-10-18T10:25:04.240297+0000","last_active":"2023-10-18T10:25:08.255868+0000","last_peered":"2023-10-18T10:25:08.255868+0000","last_clean":"2023-10-18T10:25:08.255868+0000","last_became_active":"2023-10-18T10:25:04.239997+0000","last_became_peered":"2023-10-18T10:25:04.239997+0000","last_unstale":"2023-10-18T10:25:08.255868+0000","last_undegraded":"2023-10-18T10:25:08.255868+0000","last_fullsized":"2023-10-18T10:25:08.255868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371087+0000","last_change":"2023-10-18T10:25:03.213798+0000","last_active":"2023-10-18T10:25:08.371087+0000","last_peered":"2023-10-18T10:25:08.371087+0000","last_clean":"2023-10-18T10:25:08.371087+0000","last_became_active":"2023-10-18T10:25:03.213557+0000","last_became_peered":"2023-10-18T10:25:03.213557+0000","last_unstale":"2023-10-18T10:25:08.371087+0000","last_undegraded":"2023-10-18T10:25:08.371087+0000","last_fullsized":"2023-10-18T10:25:08.371087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:08.120358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307135+0000","last_change":"2023-10-18T10:25:08.370813+0000","last_active":"2023-10-18T10:25:10.307135+0000","last_peered":"2023-10-18T10:25:10.307135+0000","last_clean":"2023-10-18T10:25:10.307135+0000","last_became_active":"2023-10-18T10:25:08.370441+0000","last_became_peered":"2023-10-18T10:25:08.370441+0000","last_unstale":"2023-10-18T10:25:10.307135+0000","last_undegraded":"2023-10-18T10:25:10.307135+0000","last_fullsized":"2023-10-18T10:25:10.307135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256321+0000","last_change":"2023-10-18T10:25:04.242148+0000","last_active":"2023-10-18T10:25:08.256321+0000","last_peered":"2023-10-18T10:25:08.256321+0000","last_clean":"2023-10-18T10:25:08.256321+0000","last_became_active":"2023-10-18T10:25:04.239749+0000","last_became_peered":"2023-10-18T10:25:04.239749+0000","last_unstale":"2023-10-18T10:25:08.256321+0000","last_undegraded":"2023-10-18T10:25:08.256321+0000","last_fullsized":"2023-10-18T10:25:08.256321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255943+0000","last_change":"2023-10-18T10:25:03.205435+0000","last_active":"2023-10-18T10:25:08.255943+0000","last_peered":"2023-10-18T10:25:08.255943+0000","last_clean":"2023-10-18T10:25:08.255943+0000","last_became_active":"2023-10-18T10:25:03.205166+0000","last_became_peered":"2023-10-18T10:25:03.205166+0000","last_unstale":"2023-10-18T10:25:08.255943+0000","last_undegraded":"2023-10-18T10:25:08.255943+0000","last_fullsized":"2023-10-18T10:25:08.255943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:41.039503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262304+0000","last_change":"2023-10-18T10:25:08.267310+0000","last_active":"2023-10-18T10:25:09.262304+0000","last_peered":"2023-10-18T10:25:09.262304+0000","last_clean":"2023-10-18T10:25:09.262304+0000","last_became_active":"2023-10-18T10:25:08.266919+0000","last_became_peered":"2023-10-18T10:25:08.266919+0000","last_unstale":"2023-10-18T10:25:09.262304+0000","last_undegraded":"2023-10-18T10:25:09.262304+0000","last_fullsized":"2023-10-18T10:25:09.262304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257274+0000","last_change":"2023-10-18T10:25:04.234960+0000","last_active":"2023-10-18T10:25:08.257274+0000","last_peered":"2023-10-18T10:25:08.257274+0000","last_clean":"2023-10-18T10:25:08.257274+0000","last_became_active":"2023-10-18T10:25:04.234695+0000","last_became_peered":"2023-10-18T10:25:04.234695+0000","last_unstale":"2023-10-18T10:25:08.257274+0000","last_undegraded":"2023-10-18T10:25:08.257274+0000","last_fullsized":"2023-10-18T10:25:08.257274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:44.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254998+0000","last_change":"2023-10-18T10:25:03.217364+0000","last_active":"2023-10-18T10:25:08.254998+0000","last_peered":"2023-10-18T10:25:08.254998+0000","last_clean":"2023-10-18T10:25:08.254998+0000","last_became_active":"2023-10-18T10:25:03.217105+0000","last_became_peered":"2023-10-18T10:25:03.217105+0000","last_unstale":"2023-10-18T10:25:08.254998+0000","last_undegraded":"2023-10-18T10:25:08.254998+0000","last_fullsized":"2023-10-18T10:25:08.254998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262547+0000","last_change":"2023-10-18T10:25:08.268370+0000","last_active":"2023-10-18T10:25:09.262547+0000","last_peered":"2023-10-18T10:25:09.262547+0000","last_clean":"2023-10-18T10:25:09.262547+0000","last_became_active":"2023-10-18T10:25:08.267649+0000","last_became_peered":"2023-10-18T10:25:08.267649+0000","last_unstale":"2023-10-18T10:25:09.262547+0000","last_undegraded":"2023-10-18T10:25:09.262547+0000","last_fullsized":"2023-10-18T10:25:09.262547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253484+0000","last_change":"2023-10-18T10:25:04.229878+0000","last_active":"2023-10-18T10:25:08.253484+0000","last_peered":"2023-10-18T10:25:08.253484+0000","last_clean":"2023-10-18T10:25:08.253484+0000","last_became_active":"2023-10-18T10:25:04.227272+0000","last_became_peered":"2023-10-18T10:25:04.227272+0000","last_unstale":"2023-10-18T10:25:08.253484+0000","last_undegraded":"2023-10-18T10:25:08.253484+0000","last_fullsized":"2023-10-18T10:25:08.253484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250602+0000","last_change":"2023-10-18T10:25:03.209065+0000","last_active":"2023-10-18T10:25:08.250602+0000","last_peered":"2023-10-18T10:25:08.250602+0000","last_clean":"2023-10-18T10:25:08.250602+0000","last_became_active":"2023-10-18T10:25:03.208772+0000","last_became_peered":"2023-10-18T10:25:03.208772+0000","last_unstale":"2023-10-18T10:25:08.250602+0000","last_undegraded":"2023-10-18T10:25:08.250602+0000","last_fullsized":"2023-10-18T10:25:08.250602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:34.438877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262993+0000","last_change":"2023-10-18T10:25:08.266888+0000","last_active":"2023-10-18T10:25:09.262993+0000","last_peered":"2023-10-18T10:25:09.262993+0000","last_clean":"2023-10-18T10:25:09.262993+0000","last_became_active":"2023-10-18T10:25:08.266538+0000","last_became_peered":"2023-10-18T10:25:08.266538+0000","last_unstale":"2023-10-18T10:25:09.262993+0000","last_undegraded":"2023-10-18T10:25:09.262993+0000","last_fullsized":"2023-10-18T10:25:09.262993+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:53.048432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253849+0000","last_change":"2023-10-18T10:25:04.233140+0000","last_active":"2023-10-18T10:25:08.253849+0000","last_peered":"2023-10-18T10:25:08.253849+0000","last_clean":"2023-10-18T10:25:08.253849+0000","last_became_active":"2023-10-18T10:25:04.232753+0000","last_became_peered":"2023-10-18T10:25:04.232753+0000","last_unstale":"2023-10-18T10:25:08.253849+0000","last_undegraded":"2023-10-18T10:25:08.253849+0000","last_fullsized":"2023-10-18T10:25:08.253849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:27.915879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369629+0000","last_change":"2023-10-18T10:25:03.217333+0000","last_active":"2023-10-18T10:25:08.369629+0000","last_peered":"2023-10-18T10:25:08.369629+0000","last_clean":"2023-10-18T10:25:08.369629+0000","last_became_active":"2023-10-18T10:25:03.217060+0000","last_became_peered":"2023-10-18T10:25:03.217060+0000","last_unstale":"2023-10-18T10:25:08.369629+0000","last_undegraded":"2023-10-18T10:25:08.369629+0000","last_fullsized":"2023-10-18T10:25:08.369629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:02.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307375+0000","last_change":"2023-10-18T10:25:08.373111+0000","last_active":"2023-10-18T10:25:10.307375+0000","last_peered":"2023-10-18T10:25:10.307375+0000","last_clean":"2023-10-18T10:25:10.307375+0000","last_became_active":"2023-10-18T10:25:08.372684+0000","last_became_peered":"2023-10-18T10:25:08.372684+0000","last_unstale":"2023-10-18T10:25:10.307375+0000","last_undegraded":"2023-10-18T10:25:10.307375+0000","last_fullsized":"2023-10-18T10:25:10.307375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:53.048432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253864+0000","last_change":"2023-10-18T10:25:04.238187+0000","last_active":"2023-10-18T10:25:08.253864+0000","last_peered":"2023-10-18T10:25:08.253864+0000","last_clean":"2023-10-18T10:25:08.253864+0000","last_became_active":"2023-10-18T10:25:04.237921+0000","last_became_peered":"2023-10-18T10:25:04.237921+0000","last_unstale":"2023-10-18T10:25:08.253864+0000","last_undegraded":"2023-10-18T10:25:08.253864+0000","last_fullsized":"2023-10-18T10:25:08.253864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253911+0000","last_change":"2023-10-18T10:25:03.220362+0000","last_active":"2023-10-18T10:25:08.253911+0000","last_peered":"2023-10-18T10:25:08.253911+0000","last_clean":"2023-10-18T10:25:08.253911+0000","last_became_active":"2023-10-18T10:25:03.220150+0000","last_became_peered":"2023-10-18T10:25:03.220150+0000","last_unstale":"2023-10-18T10:25:08.253911+0000","last_undegraded":"2023-10-18T10:25:08.253911+0000","last_fullsized":"2023-10-18T10:25:08.253911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.369864+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:25:08.369864+0000","last_peered":"2023-10-18T10:25:08.369864+0000","last_clean":"2023-10-18T10:25:08.369864+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:25:08.369864+0000","last_undegraded":"2023-10-18T10:25:08.369864+0000","last_fullsized":"2023-10-18T10:25:08.369864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307435+0000","last_change":"2023-10-18T10:25:08.374370+0000","last_active":"2023-10-18T10:25:10.307435+0000","last_peered":"2023-10-18T10:25:10.307435+0000","last_clean":"2023-10-18T10:25:10.307435+0000","last_became_active":"2023-10-18T10:25:08.372029+0000","last_became_peered":"2023-10-18T10:25:08.372029+0000","last_unstale":"2023-10-18T10:25:10.307435+0000","last_undegraded":"2023-10-18T10:25:10.307435+0000","last_fullsized":"2023-10-18T10:25:10.307435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:45.178929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370575+0000","last_change":"2023-10-18T10:25:04.239066+0000","last_active":"2023-10-18T10:25:08.370575+0000","last_peered":"2023-10-18T10:25:08.370575+0000","last_clean":"2023-10-18T10:25:08.370575+0000","last_became_active":"2023-10-18T10:25:04.238531+0000","last_became_peered":"2023-10-18T10:25:04.238531+0000","last_unstale":"2023-10-18T10:25:08.370575+0000","last_undegraded":"2023-10-18T10:25:08.370575+0000","last_fullsized":"2023-10-18T10:25:08.370575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250745+0000","last_change":"2023-10-18T10:25:03.213157+0000","last_active":"2023-10-18T10:25:08.250745+0000","last_peered":"2023-10-18T10:25:08.250745+0000","last_clean":"2023-10-18T10:25:08.250745+0000","last_became_active":"2023-10-18T10:25:03.212900+0000","last_became_peered":"2023-10-18T10:25:03.212900+0000","last_unstale":"2023-10-18T10:25:08.250745+0000","last_undegraded":"2023-10-18T10:25:08.250745+0000","last_fullsized":"2023-10-18T10:25:08.250745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:29:02.497235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307254+0000","last_change":"2023-10-18T10:25:08.370854+0000","last_active":"2023-10-18T10:25:10.307254+0000","last_peered":"2023-10-18T10:25:10.307254+0000","last_clean":"2023-10-18T10:25:10.307254+0000","last_became_active":"2023-10-18T10:25:08.370423+0000","last_became_peered":"2023-10-18T10:25:08.370423+0000","last_unstale":"2023-10-18T10:25:10.307254+0000","last_undegraded":"2023-10-18T10:25:10.307254+0000","last_fullsized":"2023-10-18T10:25:10.307254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:57.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257993+0000","last_change":"2023-10-18T10:25:04.239837+0000","last_active":"2023-10-18T10:25:08.257993+0000","last_peered":"2023-10-18T10:25:08.257993+0000","last_clean":"2023-10-18T10:25:08.257993+0000","last_became_active":"2023-10-18T10:25:04.239576+0000","last_became_peered":"2023-10-18T10:25:04.239576+0000","last_unstale":"2023-10-18T10:25:08.257993+0000","last_undegraded":"2023-10-18T10:25:08.257993+0000","last_fullsized":"2023-10-18T10:25:08.257993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:43.022445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253804+0000","last_change":"2023-10-18T10:25:03.204821+0000","last_active":"2023-10-18T10:25:08.253804+0000","last_peered":"2023-10-18T10:25:08.253804+0000","last_clean":"2023-10-18T10:25:08.253804+0000","last_became_active":"2023-10-18T10:25:03.204545+0000","last_became_peered":"2023-10-18T10:25:03.204545+0000","last_unstale":"2023-10-18T10:25:08.253804+0000","last_undegraded":"2023-10-18T10:25:08.253804+0000","last_fullsized":"2023-10-18T10:25:08.253804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307183+0000","last_change":"2023-10-18T10:25:08.265973+0000","last_active":"2023-10-18T10:25:10.307183+0000","last_peered":"2023-10-18T10:25:10.307183+0000","last_clean":"2023-10-18T10:25:10.307183+0000","last_became_active":"2023-10-18T10:25:08.265607+0000","last_became_peered":"2023-10-18T10:25:08.265607+0000","last_unstale":"2023-10-18T10:25:10.307183+0000","last_undegraded":"2023-10-18T10:25:10.307183+0000","last_fullsized":"2023-10-18T10:25:10.307183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371147+0000","last_change":"2023-10-18T10:25:04.245074+0000","last_active":"2023-10-18T10:25:08.371147+0000","last_peered":"2023-10-18T10:25:08.371147+0000","last_clean":"2023-10-18T10:25:08.371147+0000","last_became_active":"2023-10-18T10:25:04.239593+0000","last_became_peered":"2023-10-18T10:25:04.239593+0000","last_unstale":"2023-10-18T10:25:08.371147+0000","last_undegraded":"2023-10-18T10:25:08.371147+0000","last_fullsized":"2023-10-18T10:25:08.371147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:29.983102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250759+0000","last_change":"2023-10-18T10:25:03.206351+0000","last_active":"2023-10-18T10:25:08.250759+0000","last_peered":"2023-10-18T10:25:08.250759+0000","last_clean":"2023-10-18T10:25:08.250759+0000","last_became_active":"2023-10-18T10:25:03.206082+0000","last_became_peered":"2023-10-18T10:25:03.206082+0000","last_unstale":"2023-10-18T10:25:08.250759+0000","last_undegraded":"2023-10-18T10:25:08.250759+0000","last_fullsized":"2023-10-18T10:25:08.250759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:54.420307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307655+0000","last_change":"2023-10-18T10:25:08.372691+0000","last_active":"2023-10-18T10:25:10.307655+0000","last_peered":"2023-10-18T10:25:10.307655+0000","last_clean":"2023-10-18T10:25:10.307655+0000","last_became_active":"2023-10-18T10:25:08.372252+0000","last_became_peered":"2023-10-18T10:25:08.372252+0000","last_unstale":"2023-10-18T10:25:10.307655+0000","last_undegraded":"2023-10-18T10:25:10.307655+0000","last_fullsized":"2023-10-18T10:25:10.307655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:53.362022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370821+0000","last_change":"2023-10-18T10:25:04.237483+0000","last_active":"2023-10-18T10:25:08.370821+0000","last_peered":"2023-10-18T10:25:08.370821+0000","last_clean":"2023-10-18T10:25:08.370821+0000","last_became_active":"2023-10-18T10:25:04.237196+0000","last_became_peered":"2023-10-18T10:25:04.237196+0000","last_unstale":"2023-10-18T10:25:08.370821+0000","last_undegraded":"2023-10-18T10:25:08.370821+0000","last_fullsized":"2023-10-18T10:25:08.370821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:41.123375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254019+0000","last_change":"2023-10-18T10:25:03.202658+0000","last_active":"2023-10-18T10:25:08.254019+0000","last_peered":"2023-10-18T10:25:08.254019+0000","last_clean":"2023-10-18T10:25:08.254019+0000","last_became_active":"2023-10-18T10:25:03.202313+0000","last_became_peered":"2023-10-18T10:25:03.202313+0000","last_unstale":"2023-10-18T10:25:08.254019+0000","last_undegraded":"2023-10-18T10:25:08.254019+0000","last_fullsized":"2023-10-18T10:25:08.254019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:18:46.417045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262979+0000","last_change":"2023-10-18T10:25:08.273431+0000","last_active":"2023-10-18T10:25:09.262979+0000","last_peered":"2023-10-18T10:25:09.262979+0000","last_clean":"2023-10-18T10:25:09.262979+0000","last_became_active":"2023-10-18T10:25:08.273002+0000","last_became_peered":"2023-10-18T10:25:08.273002+0000","last_unstale":"2023-10-18T10:25:09.262979+0000","last_undegraded":"2023-10-18T10:25:09.262979+0000","last_fullsized":"2023-10-18T10:25:09.262979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:43.230078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371654+0000","last_change":"2023-10-18T10:25:04.237817+0000","last_active":"2023-10-18T10:25:08.371654+0000","last_peered":"2023-10-18T10:25:08.371654+0000","last_clean":"2023-10-18T10:25:08.371654+0000","last_became_active":"2023-10-18T10:25:04.237459+0000","last_became_peered":"2023-10-18T10:25:04.237459+0000","last_unstale":"2023-10-18T10:25:08.371654+0000","last_undegraded":"2023-10-18T10:25:08.371654+0000","last_fullsized":"2023-10-18T10:25:08.371654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250534+0000","last_change":"2023-10-18T10:25:03.204327+0000","last_active":"2023-10-18T10:25:08.250534+0000","last_peered":"2023-10-18T10:25:08.250534+0000","last_clean":"2023-10-18T10:25:08.250534+0000","last_became_active":"2023-10-18T10:25:03.204079+0000","last_became_peered":"2023-10-18T10:25:03.204079+0000","last_unstale":"2023-10-18T10:25:08.250534+0000","last_undegraded":"2023-10-18T10:25:08.250534+0000","last_fullsized":"2023-10-18T10:25:08.250534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:05.461286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307373+0000","last_change":"2023-10-18T10:25:08.268324+0000","last_active":"2023-10-18T10:25:10.307373+0000","last_peered":"2023-10-18T10:25:10.307373+0000","last_clean":"2023-10-18T10:25:10.307373+0000","last_became_active":"2023-10-18T10:25:08.267959+0000","last_became_peered":"2023-10-18T10:25:08.267959+0000","last_unstale":"2023-10-18T10:25:10.307373+0000","last_undegraded":"2023-10-18T10:25:10.307373+0000","last_fullsized":"2023-10-18T10:25:10.307373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:47.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263268+0000","last_change":"2023-10-18T10:25:04.234809+0000","last_active":"2023-10-18T10:25:08.263268+0000","last_peered":"2023-10-18T10:25:08.263268+0000","last_clean":"2023-10-18T10:25:08.263268+0000","last_became_active":"2023-10-18T10:25:04.234527+0000","last_became_peered":"2023-10-18T10:25:04.234527+0000","last_unstale":"2023-10-18T10:25:08.263268+0000","last_undegraded":"2023-10-18T10:25:08.263268+0000","last_fullsized":"2023-10-18T10:25:08.263268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263204+0000","last_change":"2023-10-18T10:25:03.207281+0000","last_active":"2023-10-18T10:25:08.263204+0000","last_peered":"2023-10-18T10:25:08.263204+0000","last_clean":"2023-10-18T10:25:08.263204+0000","last_became_active":"2023-10-18T10:25:03.206828+0000","last_became_peered":"2023-10-18T10:25:03.206828+0000","last_unstale":"2023-10-18T10:25:08.263204+0000","last_undegraded":"2023-10-18T10:25:08.263204+0000","last_fullsized":"2023-10-18T10:25:08.263204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.098715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921617+0000","last_change":"2023-10-18T10:25:08.372503+0000","last_active":"2023-10-18T10:25:09.921617+0000","last_peered":"2023-10-18T10:25:09.921617+0000","last_clean":"2023-10-18T10:25:09.921617+0000","last_became_active":"2023-10-18T10:25:08.371994+0000","last_became_peered":"2023-10-18T10:25:08.371994+0000","last_unstale":"2023-10-18T10:25:09.921617+0000","last_undegraded":"2023-10-18T10:25:09.921617+0000","last_fullsized":"2023-10-18T10:25:09.921617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.553280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370729+0000","last_change":"2023-10-18T10:25:04.237797+0000","last_active":"2023-10-18T10:25:08.370729+0000","last_peered":"2023-10-18T10:25:08.370729+0000","last_clean":"2023-10-18T10:25:08.370729+0000","last_became_active":"2023-10-18T10:25:04.237473+0000","last_became_peered":"2023-10-18T10:25:04.237473+0000","last_unstale":"2023-10-18T10:25:08.370729+0000","last_undegraded":"2023-10-18T10:25:08.370729+0000","last_fullsized":"2023-10-18T10:25:08.370729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:48.992878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254912+0000","last_change":"2023-10-18T10:25:03.213281+0000","last_active":"2023-10-18T10:25:08.254912+0000","last_peered":"2023-10-18T10:25:08.254912+0000","last_clean":"2023-10-18T10:25:08.254912+0000","last_became_active":"2023-10-18T10:25:03.213019+0000","last_became_peered":"2023-10-18T10:25:03.213019+0000","last_unstale":"2023-10-18T10:25:08.254912+0000","last_undegraded":"2023-10-18T10:25:08.254912+0000","last_fullsized":"2023-10-18T10:25:08.254912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:07.702798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921380+0000","last_change":"2023-10-18T10:25:08.370771+0000","last_active":"2023-10-18T10:25:09.921380+0000","last_peered":"2023-10-18T10:25:09.921380+0000","last_clean":"2023-10-18T10:25:09.921380+0000","last_became_active":"2023-10-18T10:25:08.370438+0000","last_became_peered":"2023-10-18T10:25:08.370438+0000","last_unstale":"2023-10-18T10:25:09.921380+0000","last_undegraded":"2023-10-18T10:25:09.921380+0000","last_fullsized":"2023-10-18T10:25:09.921380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:31.794489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254101+0000","last_change":"2023-10-18T10:25:04.232734+0000","last_active":"2023-10-18T10:25:08.254101+0000","last_peered":"2023-10-18T10:25:08.254101+0000","last_clean":"2023-10-18T10:25:08.254101+0000","last_became_active":"2023-10-18T10:25:04.232474+0000","last_became_peered":"2023-10-18T10:25:04.232474+0000","last_unstale":"2023-10-18T10:25:08.254101+0000","last_undegraded":"2023-10-18T10:25:08.254101+0000","last_fullsized":"2023-10-18T10:25:08.254101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:09.122412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.257928+0000","last_change":"2023-10-18T10:25:03.202060+0000","last_active":"2023-10-18T10:25:08.257928+0000","last_peered":"2023-10-18T10:25:08.257928+0000","last_clean":"2023-10-18T10:25:08.257928+0000","last_became_active":"2023-10-18T10:25:03.201719+0000","last_became_peered":"2023-10-18T10:25:03.201719+0000","last_unstale":"2023-10-18T10:25:08.257928+0000","last_undegraded":"2023-10-18T10:25:08.257928+0000","last_fullsized":"2023-10-18T10:25:08.257928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.308982+0000","last_change":"2023-10-18T10:25:08.370830+0000","last_active":"2023-10-18T10:25:10.308982+0000","last_peered":"2023-10-18T10:25:10.308982+0000","last_clean":"2023-10-18T10:25:10.308982+0000","last_became_active":"2023-10-18T10:25:08.370387+0000","last_became_peered":"2023-10-18T10:25:08.370387+0000","last_unstale":"2023-10-18T10:25:10.308982+0000","last_undegraded":"2023-10-18T10:25:10.308982+0000","last_fullsized":"2023-10-18T10:25:10.308982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:26.275317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370740+0000","last_change":"2023-10-18T10:25:04.241881+0000","last_active":"2023-10-18T10:25:08.370740+0000","last_peered":"2023-10-18T10:25:08.370740+0000","last_clean":"2023-10-18T10:25:08.370740+0000","last_became_active":"2023-10-18T10:25:04.241615+0000","last_became_peered":"2023-10-18T10:25:04.241615+0000","last_unstale":"2023-10-18T10:25:08.370740+0000","last_undegraded":"2023-10-18T10:25:08.370740+0000","last_fullsized":"2023-10-18T10:25:08.370740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:08.360111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:13.490096+0000","last_change":"2023-10-18T10:25:03.211150+0000","last_active":"2023-10-18T10:25:13.490096+0000","last_peered":"2023-10-18T10:25:13.490096+0000","last_clean":"2023-10-18T10:25:13.490096+0000","last_became_active":"2023-10-18T10:25:03.210879+0000","last_became_peered":"2023-10-18T10:25:03.210879+0000","last_unstale":"2023-10-18T10:25:13.490096+0000","last_undegraded":"2023-10-18T10:25:13.490096+0000","last_fullsized":"2023-10-18T10:25:13.490096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:47.119632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.262397+0000","last_change":"2023-10-18T10:25:08.268389+0000","last_active":"2023-10-18T10:25:09.262397+0000","last_peered":"2023-10-18T10:25:09.262397+0000","last_clean":"2023-10-18T10:25:09.262397+0000","last_became_active":"2023-10-18T10:25:08.268025+0000","last_became_peered":"2023-10-18T10:25:08.268025+0000","last_unstale":"2023-10-18T10:25:09.262397+0000","last_undegraded":"2023-10-18T10:25:09.262397+0000","last_fullsized":"2023-10-18T10:25:09.262397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:05.682257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256542+0000","last_change":"2023-10-18T10:25:04.229742+0000","last_active":"2023-10-18T10:25:08.256542+0000","last_peered":"2023-10-18T10:25:08.256542+0000","last_clean":"2023-10-18T10:25:08.256542+0000","last_became_active":"2023-10-18T10:25:04.229281+0000","last_became_peered":"2023-10-18T10:25:04.229281+0000","last_unstale":"2023-10-18T10:25:08.256542+0000","last_undegraded":"2023-10-18T10:25:08.256542+0000","last_fullsized":"2023-10-18T10:25:08.256542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:08.704852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253721+0000","last_change":"2023-10-18T10:25:03.213327+0000","last_active":"2023-10-18T10:25:08.253721+0000","last_peered":"2023-10-18T10:25:08.253721+0000","last_clean":"2023-10-18T10:25:08.253721+0000","last_became_active":"2023-10-18T10:25:03.212993+0000","last_became_peered":"2023-10-18T10:25:03.212993+0000","last_unstale":"2023-10-18T10:25:08.253721+0000","last_undegraded":"2023-10-18T10:25:08.253721+0000","last_fullsized":"2023-10-18T10:25:08.253721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.263300+0000","last_change":"2023-10-18T10:25:08.271366+0000","last_active":"2023-10-18T10:25:09.263300+0000","last_peered":"2023-10-18T10:25:09.263300+0000","last_clean":"2023-10-18T10:25:09.263300+0000","last_became_active":"2023-10-18T10:25:08.270581+0000","last_became_peered":"2023-10-18T10:25:08.270581+0000","last_unstale":"2023-10-18T10:25:09.263300+0000","last_undegraded":"2023-10-18T10:25:09.263300+0000","last_fullsized":"2023-10-18T10:25:09.263300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:57.769840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254604+0000","last_change":"2023-10-18T10:25:04.234658+0000","last_active":"2023-10-18T10:25:08.254604+0000","last_peered":"2023-10-18T10:25:08.254604+0000","last_clean":"2023-10-18T10:25:08.254604+0000","last_became_active":"2023-10-18T10:25:04.234334+0000","last_became_peered":"2023-10-18T10:25:04.234334+0000","last_unstale":"2023-10-18T10:25:08.254604+0000","last_undegraded":"2023-10-18T10:25:08.254604+0000","last_fullsized":"2023-10-18T10:25:08.254604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254170+0000","last_change":"2023-10-18T10:25:03.213955+0000","last_active":"2023-10-18T10:25:08.254170+0000","last_peered":"2023-10-18T10:25:08.254170+0000","last_clean":"2023-10-18T10:25:08.254170+0000","last_became_active":"2023-10-18T10:25:03.213692+0000","last_became_peered":"2023-10-18T10:25:03.213692+0000","last_unstale":"2023-10-18T10:25:08.254170+0000","last_undegraded":"2023-10-18T10:25:08.254170+0000","last_fullsized":"2023-10-18T10:25:08.254170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:21.099526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920943+0000","last_change":"2023-10-18T10:25:08.271334+0000","last_active":"2023-10-18T10:25:09.920943+0000","last_peered":"2023-10-18T10:25:09.920943+0000","last_clean":"2023-10-18T10:25:09.920943+0000","last_became_active":"2023-10-18T10:25:08.270900+0000","last_became_peered":"2023-10-18T10:25:08.270900+0000","last_unstale":"2023-10-18T10:25:09.920943+0000","last_undegraded":"2023-10-18T10:25:09.920943+0000","last_fullsized":"2023-10-18T10:25:09.920943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:53:25.617273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254094+0000","last_change":"2023-10-18T10:25:04.238611+0000","last_active":"2023-10-18T10:25:08.254094+0000","last_peered":"2023-10-18T10:25:08.254094+0000","last_clean":"2023-10-18T10:25:08.254094+0000","last_became_active":"2023-10-18T10:25:04.238304+0000","last_became_peered":"2023-10-18T10:25:04.238304+0000","last_unstale":"2023-10-18T10:25:08.254094+0000","last_undegraded":"2023-10-18T10:25:08.254094+0000","last_fullsized":"2023-10-18T10:25:08.254094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264215+0000","last_change":"2023-10-18T10:25:03.213189+0000","last_active":"2023-10-18T10:25:08.264215+0000","last_peered":"2023-10-18T10:25:08.264215+0000","last_clean":"2023-10-18T10:25:08.264215+0000","last_became_active":"2023-10-18T10:25:03.212915+0000","last_became_peered":"2023-10-18T10:25:03.212915+0000","last_unstale":"2023-10-18T10:25:08.264215+0000","last_undegraded":"2023-10-18T10:25:08.264215+0000","last_fullsized":"2023-10-18T10:25:08.264215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.920935+0000","last_change":"2023-10-18T10:25:08.374903+0000","last_active":"2023-10-18T10:25:09.920935+0000","last_peered":"2023-10-18T10:25:09.920935+0000","last_clean":"2023-10-18T10:25:09.920935+0000","last_became_active":"2023-10-18T10:25:08.374525+0000","last_became_peered":"2023-10-18T10:25:08.374525+0000","last_unstale":"2023-10-18T10:25:09.920935+0000","last_undegraded":"2023-10-18T10:25:09.920935+0000","last_fullsized":"2023-10-18T10:25:09.920935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:38.455325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.307246+0000","last_change":"2023-10-18T10:25:08.372484+0000","last_active":"2023-10-18T10:25:10.307246+0000","last_peered":"2023-10-18T10:25:10.307246+0000","last_clean":"2023-10-18T10:25:10.307246+0000","last_became_active":"2023-10-18T10:25:08.372132+0000","last_became_peered":"2023-10-18T10:25:08.372132+0000","last_unstale":"2023-10-18T10:25:10.307246+0000","last_undegraded":"2023-10-18T10:25:10.307246+0000","last_fullsized":"2023-10-18T10:25:10.307246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:30.760975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253285+0000","last_change":"2023-10-18T10:25:04.239197+0000","last_active":"2023-10-18T10:25:08.253285+0000","last_peered":"2023-10-18T10:25:08.253285+0000","last_clean":"2023-10-18T10:25:08.253285+0000","last_became_active":"2023-10-18T10:25:04.238917+0000","last_became_peered":"2023-10-18T10:25:04.238917+0000","last_unstale":"2023-10-18T10:25:08.253285+0000","last_undegraded":"2023-10-18T10:25:08.253285+0000","last_fullsized":"2023-10-18T10:25:08.253285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254371+0000","last_change":"2023-10-18T10:25:03.207877+0000","last_active":"2023-10-18T10:25:08.254371+0000","last_peered":"2023-10-18T10:25:08.254371+0000","last_clean":"2023-10-18T10:25:08.254371+0000","last_became_active":"2023-10-18T10:25:03.207621+0000","last_became_peered":"2023-10-18T10:25:03.207621+0000","last_unstale":"2023-10-18T10:25:08.254371+0000","last_undegraded":"2023-10-18T10:25:08.254371+0000","last_fullsized":"2023-10-18T10:25:08.254371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:52.476561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:10.308246+0000","last_change":"2023-10-18T10:25:08.272814+0000","last_active":"2023-10-18T10:25:10.308246+0000","last_peered":"2023-10-18T10:25:10.308246+0000","last_clean":"2023-10-18T10:25:10.308246+0000","last_became_active":"2023-10-18T10:25:08.272372+0000","last_became_peered":"2023-10-18T10:25:08.272372+0000","last_unstale":"2023-10-18T10:25:10.308246+0000","last_undegraded":"2023-10-18T10:25:10.308246+0000","last_fullsized":"2023-10-18T10:25:10.308246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:02.844599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254375+0000","last_change":"2023-10-18T10:25:04.237028+0000","last_active":"2023-10-18T10:25:08.254375+0000","last_peered":"2023-10-18T10:25:08.254375+0000","last_clean":"2023-10-18T10:25:08.254375+0000","last_became_active":"2023-10-18T10:25:04.236620+0000","last_became_peered":"2023-10-18T10:25:04.236620+0000","last_unstale":"2023-10-18T10:25:08.254375+0000","last_undegraded":"2023-10-18T10:25:08.254375+0000","last_fullsized":"2023-10-18T10:25:08.254375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255920+0000","last_change":"2023-10-18T10:25:03.212737+0000","last_active":"2023-10-18T10:25:08.255920+0000","last_peered":"2023-10-18T10:25:08.255920+0000","last_clean":"2023-10-18T10:25:08.255920+0000","last_became_active":"2023-10-18T10:25:03.212444+0000","last_became_peered":"2023-10-18T10:25:03.212444+0000","last_unstale":"2023-10-18T10:25:08.255920+0000","last_undegraded":"2023-10-18T10:25:08.255920+0000","last_fullsized":"2023-10-18T10:25:08.255920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:53.307830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265148+0000","last_change":"2023-10-18T10:25:03.208047+0000","last_active":"2023-10-18T10:25:08.265148+0000","last_peered":"2023-10-18T10:25:08.265148+0000","last_clean":"2023-10-18T10:25:08.265148+0000","last_became_active":"2023-10-18T10:25:03.207745+0000","last_became_peered":"2023-10-18T10:25:03.207745+0000","last_unstale":"2023-10-18T10:25:08.265148+0000","last_undegraded":"2023-10-18T10:25:08.265148+0000","last_fullsized":"2023-10-18T10:25:08.265148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:38:48.623033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255246+0000","last_change":"2023-10-18T10:25:04.240334+0000","last_active":"2023-10-18T10:25:08.255246+0000","last_peered":"2023-10-18T10:25:08.255246+0000","last_clean":"2023-10-18T10:25:08.255246+0000","last_became_active":"2023-10-18T10:25:04.239964+0000","last_became_peered":"2023-10-18T10:25:04.239964+0000","last_unstale":"2023-10-18T10:25:08.255246+0000","last_undegraded":"2023-10-18T10:25:08.255246+0000","last_fullsized":"2023-10-18T10:25:08.255246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.294555+0000","last_change":"2023-10-18T10:25:08.264225+0000","last_active":"2023-10-18T10:25:09.294555+0000","last_peered":"2023-10-18T10:25:09.294555+0000","last_clean":"2023-10-18T10:25:09.294555+0000","last_became_active":"2023-10-18T10:25:08.263712+0000","last_became_peered":"2023-10-18T10:25:08.263712+0000","last_unstale":"2023-10-18T10:25:09.294555+0000","last_undegraded":"2023-10-18T10:25:09.294555+0000","last_fullsized":"2023-10-18T10:25:09.294555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:13.156323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253645+0000","last_change":"2023-10-18T10:25:03.211608+0000","last_active":"2023-10-18T10:25:08.253645+0000","last_peered":"2023-10-18T10:25:08.253645+0000","last_clean":"2023-10-18T10:25:08.253645+0000","last_became_active":"2023-10-18T10:25:03.211346+0000","last_became_peered":"2023-10-18T10:25:03.211346+0000","last_unstale":"2023-10-18T10:25:08.253645+0000","last_undegraded":"2023-10-18T10:25:08.253645+0000","last_fullsized":"2023-10-18T10:25:08.253645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:02.183625+0000","last_clean_scrub_stamp":"2023-10-18T10:25:02.183625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:26.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255576+0000","last_change":"2023-10-18T10:25:04.242544+0000","last_active":"2023-10-18T10:25:08.255576+0000","last_peered":"2023-10-18T10:25:08.255576+0000","last_clean":"2023-10-18T10:25:08.255576+0000","last_became_active":"2023-10-18T10:25:04.242258+0000","last_became_peered":"2023-10-18T10:25:04.242258+0000","last_unstale":"2023-10-18T10:25:08.255576+0000","last_undegraded":"2023-10-18T10:25:08.255576+0000","last_fullsized":"2023-10-18T10:25:08.255576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:55.422361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-18T10:25:09.921493+0000","last_change":"2023-10-18T10:25:08.273772+0000","last_active":"2023-10-18T10:25:09.921493+0000","last_peered":"2023-10-18T10:25:09.921493+0000","last_clean":"2023-10-18T10:25:09.921493+0000","last_became_active":"2023-10-18T10:25:08.273410+0000","last_became_peered":"2023-10-18T10:25:08.273410+0000","last_unstale":"2023-10-18T10:25:09.921493+0000","last_undegraded":"2023-10-18T10:25:09.921493+0000","last_fullsized":"2023-10-18T10:25:09.921493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:07.241233+0000","last_clean_scrub_stamp":"2023-10-18T10:25:07.241233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:19.382604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263176+0000","last_change":"2023-10-18T10:25:04.235479+0000","last_active":"2023-10-18T10:25:08.263176+0000","last_peered":"2023-10-18T10:25:08.263176+0000","last_clean":"2023-10-18T10:25:08.263176+0000","last_became_active":"2023-10-18T10:25:04.235205+0000","last_became_peered":"2023-10-18T10:25:04.235205+0000","last_unstale":"2023-10-18T10:25:08.263176+0000","last_undegraded":"2023-10-18T10:25:08.263176+0000","last_fullsized":"2023-10-18T10:25:08.263176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250534+0000","last_change":"2023-10-18T10:25:04.236754+0000","last_active":"2023-10-18T10:25:08.250534+0000","last_peered":"2023-10-18T10:25:08.250534+0000","last_clean":"2023-10-18T10:25:08.250534+0000","last_became_active":"2023-10-18T10:25:04.236482+0000","last_became_peered":"2023-10-18T10:25:04.236482+0000","last_unstale":"2023-10-18T10:25:08.250534+0000","last_undegraded":"2023-10-18T10:25:08.250534+0000","last_fullsized":"2023-10-18T10:25:08.250534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:29:08.704852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254828+0000","last_change":"2023-10-18T10:25:04.239429+0000","last_active":"2023-10-18T10:25:08.254828+0000","last_peered":"2023-10-18T10:25:08.254828+0000","last_clean":"2023-10-18T10:25:08.254828+0000","last_became_active":"2023-10-18T10:25:04.239185+0000","last_became_peered":"2023-10-18T10:25:04.239185+0000","last_unstale":"2023-10-18T10:25:08.254828+0000","last_undegraded":"2023-10-18T10:25:08.254828+0000","last_fullsized":"2023-10-18T10:25:08.254828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:41.123375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255164+0000","last_change":"2023-10-18T10:25:04.235523+0000","last_active":"2023-10-18T10:25:08.255164+0000","last_peered":"2023-10-18T10:25:08.255164+0000","last_clean":"2023-10-18T10:25:08.255164+0000","last_became_active":"2023-10-18T10:25:04.235266+0000","last_became_peered":"2023-10-18T10:25:04.235266+0000","last_unstale":"2023-10-18T10:25:08.255164+0000","last_undegraded":"2023-10-18T10:25:08.255164+0000","last_fullsized":"2023-10-18T10:25:08.255164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254545+0000","last_change":"2023-10-18T10:25:04.225083+0000","last_active":"2023-10-18T10:25:08.254545+0000","last_peered":"2023-10-18T10:25:08.254545+0000","last_clean":"2023-10-18T10:25:08.254545+0000","last_became_active":"2023-10-18T10:25:04.224243+0000","last_became_peered":"2023-10-18T10:25:04.224243+0000","last_unstale":"2023-10-18T10:25:08.254545+0000","last_undegraded":"2023-10-18T10:25:08.254545+0000","last_fullsized":"2023-10-18T10:25:08.254545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253638+0000","last_change":"2023-10-18T10:25:04.239380+0000","last_active":"2023-10-18T10:25:08.253638+0000","last_peered":"2023-10-18T10:25:08.253638+0000","last_clean":"2023-10-18T10:25:08.253638+0000","last_became_active":"2023-10-18T10:25:04.239121+0000","last_became_peered":"2023-10-18T10:25:04.239121+0000","last_unstale":"2023-10-18T10:25:08.253638+0000","last_undegraded":"2023-10-18T10:25:08.253638+0000","last_fullsized":"2023-10-18T10:25:08.253638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.265703+0000","last_change":"2023-10-18T10:25:04.231489+0000","last_active":"2023-10-18T10:25:08.265703+0000","last_peered":"2023-10-18T10:25:08.265703+0000","last_clean":"2023-10-18T10:25:08.265703+0000","last_became_active":"2023-10-18T10:25:04.231228+0000","last_became_peered":"2023-10-18T10:25:04.231228+0000","last_unstale":"2023-10-18T10:25:08.265703+0000","last_undegraded":"2023-10-18T10:25:08.265703+0000","last_fullsized":"2023-10-18T10:25:08.265703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:53.478615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255438+0000","last_change":"2023-10-18T10:25:04.226121+0000","last_active":"2023-10-18T10:25:08.255438+0000","last_peered":"2023-10-18T10:25:08.255438+0000","last_clean":"2023-10-18T10:25:08.255438+0000","last_became_active":"2023-10-18T10:25:04.225472+0000","last_became_peered":"2023-10-18T10:25:04.225472+0000","last_unstale":"2023-10-18T10:25:08.255438+0000","last_undegraded":"2023-10-18T10:25:08.255438+0000","last_fullsized":"2023-10-18T10:25:08.255438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254296+0000","last_change":"2023-10-18T10:25:04.239623+0000","last_active":"2023-10-18T10:25:08.254296+0000","last_peered":"2023-10-18T10:25:08.254296+0000","last_clean":"2023-10-18T10:25:08.254296+0000","last_became_active":"2023-10-18T10:25:04.239376+0000","last_became_peered":"2023-10-18T10:25:04.239376+0000","last_unstale":"2023-10-18T10:25:08.254296+0000","last_undegraded":"2023-10-18T10:25:08.254296+0000","last_fullsized":"2023-10-18T10:25:08.254296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.789045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255778+0000","last_change":"2023-10-18T10:25:04.224851+0000","last_active":"2023-10-18T10:25:08.255778+0000","last_peered":"2023-10-18T10:25:08.255778+0000","last_clean":"2023-10-18T10:25:08.255778+0000","last_became_active":"2023-10-18T10:25:04.224579+0000","last_became_peered":"2023-10-18T10:25:04.224579+0000","last_unstale":"2023-10-18T10:25:08.255778+0000","last_undegraded":"2023-10-18T10:25:08.255778+0000","last_fullsized":"2023-10-18T10:25:08.255778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254745+0000","last_change":"2023-10-18T10:25:04.233040+0000","last_active":"2023-10-18T10:25:08.254745+0000","last_peered":"2023-10-18T10:25:08.254745+0000","last_clean":"2023-10-18T10:25:08.254745+0000","last_became_active":"2023-10-18T10:25:04.232782+0000","last_became_peered":"2023-10-18T10:25:04.232782+0000","last_unstale":"2023-10-18T10:25:08.254745+0000","last_undegraded":"2023-10-18T10:25:08.254745+0000","last_fullsized":"2023-10-18T10:25:08.254745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:53.714286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255053+0000","last_change":"2023-10-18T10:25:04.235058+0000","last_active":"2023-10-18T10:25:08.255053+0000","last_peered":"2023-10-18T10:25:08.255053+0000","last_clean":"2023-10-18T10:25:08.255053+0000","last_became_active":"2023-10-18T10:25:04.234763+0000","last_became_peered":"2023-10-18T10:25:04.234763+0000","last_unstale":"2023-10-18T10:25:08.255053+0000","last_undegraded":"2023-10-18T10:25:08.255053+0000","last_fullsized":"2023-10-18T10:25:08.255053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263580+0000","last_change":"2023-10-18T10:25:04.231136+0000","last_active":"2023-10-18T10:25:08.263580+0000","last_peered":"2023-10-18T10:25:08.263580+0000","last_clean":"2023-10-18T10:25:08.263580+0000","last_became_active":"2023-10-18T10:25:04.230835+0000","last_became_peered":"2023-10-18T10:25:04.230835+0000","last_unstale":"2023-10-18T10:25:08.263580+0000","last_undegraded":"2023-10-18T10:25:08.263580+0000","last_fullsized":"2023-10-18T10:25:08.263580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370824+0000","last_change":"2023-10-18T10:25:04.236843+0000","last_active":"2023-10-18T10:25:08.370824+0000","last_peered":"2023-10-18T10:25:08.370824+0000","last_clean":"2023-10-18T10:25:08.370824+0000","last_became_active":"2023-10-18T10:25:04.236484+0000","last_became_peered":"2023-10-18T10:25:04.236484+0000","last_unstale":"2023-10-18T10:25:08.370824+0000","last_undegraded":"2023-10-18T10:25:08.370824+0000","last_fullsized":"2023-10-18T10:25:08.370824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.789045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.250697+0000","last_change":"2023-10-18T10:25:04.230906+0000","last_active":"2023-10-18T10:25:08.250697+0000","last_peered":"2023-10-18T10:25:08.250697+0000","last_clean":"2023-10-18T10:25:08.250697+0000","last_became_active":"2023-10-18T10:25:04.230641+0000","last_became_peered":"2023-10-18T10:25:04.230641+0000","last_unstale":"2023-10-18T10:25:08.250697+0000","last_undegraded":"2023-10-18T10:25:08.250697+0000","last_fullsized":"2023-10-18T10:25:08.250697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:44.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254323+0000","last_change":"2023-10-18T10:25:04.235839+0000","last_active":"2023-10-18T10:25:08.254323+0000","last_peered":"2023-10-18T10:25:08.254323+0000","last_clean":"2023-10-18T10:25:08.254323+0000","last_became_active":"2023-10-18T10:25:04.235602+0000","last_became_peered":"2023-10-18T10:25:04.235602+0000","last_unstale":"2023-10-18T10:25:08.254323+0000","last_undegraded":"2023-10-18T10:25:08.254323+0000","last_fullsized":"2023-10-18T10:25:08.254323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:27.164125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254219+0000","last_change":"2023-10-18T10:25:04.235249+0000","last_active":"2023-10-18T10:25:08.254219+0000","last_peered":"2023-10-18T10:25:08.254219+0000","last_clean":"2023-10-18T10:25:08.254219+0000","last_became_active":"2023-10-18T10:25:04.234967+0000","last_became_peered":"2023-10-18T10:25:04.234967+0000","last_unstale":"2023-10-18T10:25:08.254219+0000","last_undegraded":"2023-10-18T10:25:08.254219+0000","last_fullsized":"2023-10-18T10:25:08.254219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253370+0000","last_change":"2023-10-18T10:25:04.232541+0000","last_active":"2023-10-18T10:25:08.253370+0000","last_peered":"2023-10-18T10:25:08.253370+0000","last_clean":"2023-10-18T10:25:08.253370+0000","last_became_active":"2023-10-18T10:25:04.231594+0000","last_became_peered":"2023-10-18T10:25:04.231594+0000","last_unstale":"2023-10-18T10:25:08.253370+0000","last_undegraded":"2023-10-18T10:25:08.253370+0000","last_fullsized":"2023-10-18T10:25:08.253370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:50.930409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254018+0000","last_change":"2023-10-18T10:25:04.239452+0000","last_active":"2023-10-18T10:25:08.254018+0000","last_peered":"2023-10-18T10:25:08.254018+0000","last_clean":"2023-10-18T10:25:08.254018+0000","last_became_active":"2023-10-18T10:25:04.239175+0000","last_became_peered":"2023-10-18T10:25:04.239175+0000","last_unstale":"2023-10-18T10:25:08.254018+0000","last_undegraded":"2023-10-18T10:25:08.254018+0000","last_fullsized":"2023-10-18T10:25:08.254018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:42.041557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.253482+0000","last_change":"2023-10-18T10:25:04.225873+0000","last_active":"2023-10-18T10:25:08.253482+0000","last_peered":"2023-10-18T10:25:08.253482+0000","last_clean":"2023-10-18T10:25:08.253482+0000","last_became_active":"2023-10-18T10:25:04.225624+0000","last_became_peered":"2023-10-18T10:25:04.225624+0000","last_unstale":"2023-10-18T10:25:08.253482+0000","last_undegraded":"2023-10-18T10:25:08.253482+0000","last_fullsized":"2023-10-18T10:25:08.253482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.256440+0000","last_change":"2023-10-18T10:25:04.226107+0000","last_active":"2023-10-18T10:25:08.256440+0000","last_peered":"2023-10-18T10:25:08.256440+0000","last_clean":"2023-10-18T10:25:08.256440+0000","last_became_active":"2023-10-18T10:25:04.225823+0000","last_became_peered":"2023-10-18T10:25:04.225823+0000","last_unstale":"2023-10-18T10:25:08.256440+0000","last_undegraded":"2023-10-18T10:25:08.256440+0000","last_fullsized":"2023-10-18T10:25:08.256440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:35.440931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.263514+0000","last_change":"2023-10-18T10:25:04.237730+0000","last_active":"2023-10-18T10:25:08.263514+0000","last_peered":"2023-10-18T10:25:08.263514+0000","last_clean":"2023-10-18T10:25:08.263514+0000","last_became_active":"2023-10-18T10:25:04.237436+0000","last_became_peered":"2023-10-18T10:25:04.237436+0000","last_unstale":"2023-10-18T10:25:08.263514+0000","last_undegraded":"2023-10-18T10:25:08.263514+0000","last_fullsized":"2023-10-18T10:25:08.263514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:22.101580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.264090+0000","last_change":"2023-10-18T10:25:04.242105+0000","last_active":"2023-10-18T10:25:08.264090+0000","last_peered":"2023-10-18T10:25:08.264090+0000","last_clean":"2023-10-18T10:25:08.264090+0000","last_became_active":"2023-10-18T10:25:04.241762+0000","last_became_peered":"2023-10-18T10:25:04.241762+0000","last_unstale":"2023-10-18T10:25:08.264090+0000","last_undegraded":"2023-10-18T10:25:08.264090+0000","last_fullsized":"2023-10-18T10:25:08.264090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:55.818373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371062+0000","last_change":"2023-10-18T10:25:04.239055+0000","last_active":"2023-10-18T10:25:08.371062+0000","last_peered":"2023-10-18T10:25:08.371062+0000","last_clean":"2023-10-18T10:25:08.371062+0000","last_became_active":"2023-10-18T10:25:04.238566+0000","last_became_peered":"2023-10-18T10:25:04.238566+0000","last_unstale":"2023-10-18T10:25:08.371062+0000","last_undegraded":"2023-10-18T10:25:08.371062+0000","last_fullsized":"2023-10-18T10:25:08.371062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:43.022445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254270+0000","last_change":"2023-10-18T10:25:04.238819+0000","last_active":"2023-10-18T10:25:08.254270+0000","last_peered":"2023-10-18T10:25:08.254270+0000","last_clean":"2023-10-18T10:25:08.254270+0000","last_became_active":"2023-10-18T10:25:04.238546+0000","last_became_peered":"2023-10-18T10:25:04.238546+0000","last_unstale":"2023-10-18T10:25:08.254270+0000","last_undegraded":"2023-10-18T10:25:08.254270+0000","last_fullsized":"2023-10-18T10:25:08.254270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.371571+0000","last_change":"2023-10-18T10:25:04.234955+0000","last_active":"2023-10-18T10:25:08.371571+0000","last_peered":"2023-10-18T10:25:08.371571+0000","last_clean":"2023-10-18T10:25:08.371571+0000","last_became_active":"2023-10-18T10:25:04.234687+0000","last_became_peered":"2023-10-18T10:25:04.234687+0000","last_unstale":"2023-10-18T10:25:08.371571+0000","last_undegraded":"2023-10-18T10:25:08.371571+0000","last_fullsized":"2023-10-18T10:25:08.371571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:18.856914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254664+0000","last_change":"2023-10-18T10:25:04.235078+0000","last_active":"2023-10-18T10:25:08.254664+0000","last_peered":"2023-10-18T10:25:08.254664+0000","last_clean":"2023-10-18T10:25:08.254664+0000","last_became_active":"2023-10-18T10:25:04.234817+0000","last_became_peered":"2023-10-18T10:25:04.234817+0000","last_unstale":"2023-10-18T10:25:08.254664+0000","last_undegraded":"2023-10-18T10:25:08.254664+0000","last_fullsized":"2023-10-18T10:25:08.254664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370554+0000","last_change":"2023-10-18T10:25:04.228077+0000","last_active":"2023-10-18T10:25:08.370554+0000","last_peered":"2023-10-18T10:25:08.370554+0000","last_clean":"2023-10-18T10:25:08.370554+0000","last_became_active":"2023-10-18T10:25:04.227740+0000","last_became_peered":"2023-10-18T10:25:04.227740+0000","last_unstale":"2023-10-18T10:25:08.370554+0000","last_undegraded":"2023-10-18T10:25:08.370554+0000","last_fullsized":"2023-10-18T10:25:08.370554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:26.705420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.258398+0000","last_change":"2023-10-18T10:25:04.240451+0000","last_active":"2023-10-18T10:25:08.258398+0000","last_peered":"2023-10-18T10:25:08.258398+0000","last_clean":"2023-10-18T10:25:08.258398+0000","last_became_active":"2023-10-18T10:25:04.240190+0000","last_became_peered":"2023-10-18T10:25:04.240190+0000","last_unstale":"2023-10-18T10:25:08.258398+0000","last_undegraded":"2023-10-18T10:25:08.258398+0000","last_fullsized":"2023-10-18T10:25:08.258398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:01.626703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.254434+0000","last_change":"2023-10-18T10:25:04.233621+0000","last_active":"2023-10-18T10:25:08.254434+0000","last_peered":"2023-10-18T10:25:08.254434+0000","last_clean":"2023-10-18T10:25:08.254434+0000","last_became_active":"2023-10-18T10:25:04.233376+0000","last_became_peered":"2023-10-18T10:25:04.233376+0000","last_unstale":"2023-10-18T10:25:08.254434+0000","last_undegraded":"2023-10-18T10:25:08.254434+0000","last_fullsized":"2023-10-18T10:25:08.254434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:34.587493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.255094+0000","last_change":"2023-10-18T10:25:04.234775+0000","last_active":"2023-10-18T10:25:08.255094+0000","last_peered":"2023-10-18T10:25:08.255094+0000","last_clean":"2023-10-18T10:25:08.255094+0000","last_became_active":"2023-10-18T10:25:04.234527+0000","last_became_peered":"2023-10-18T10:25:04.234527+0000","last_unstale":"2023-10-18T10:25:08.255094+0000","last_undegraded":"2023-10-18T10:25:08.255094+0000","last_fullsized":"2023-10-18T10:25:08.255094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:14.950717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-18T10:25:08.370952+0000","last_change":"2023-10-18T10:25:04.236702+0000","last_active":"2023-10-18T10:25:08.370952+0000","last_peered":"2023-10-18T10:25:08.370952+0000","last_clean":"2023-10-18T10:25:08.370952+0000","last_became_active":"2023-10-18T10:25:04.236396+0000","last_became_peered":"2023-10-18T10:25:04.236396+0000","last_unstale":"2023-10-18T10:25:08.370952+0000","last_undegraded":"2023-10-18T10:25:08.370952+0000","last_fullsized":"2023-10-18T10:25:08.370952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:25:03.185679+0000","last_clean_scrub_stamp":"2023-10-18T10:25:03.185679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:01.626703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:25:26.190 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T10:25:26.190 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-18T10:25:26.191 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-18T10:25:26.191 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T10:25:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.324+0000 7f15bb15e640 1 -- 172.21.15.125:0/2013794148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40b3070 msgr2=0x7f15b41590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.324+0000 7f15bb15e640 1 --2- 172.21.15.125:0/2013794148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40b3070 0x7f15b41590b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f15a0005fd0 tx=0x7f15a002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.324+0000 7f15bb15e640 1 -- 172.21.15.125:0/2013794148 shutdown_connections 2023-10-18T10:25:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.324+0000 7f15bb15e640 1 --2- 172.21.15.125:0/2013794148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40b3070 0x7f15b41590b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.324+0000 7f15bb15e640 1 --2- 172.21.15.125:0/2013794148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b40b24d0 0x7f15b40b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.324+0000 7f15bb15e640 1 -- 172.21.15.125:0/2013794148 >> 172.21.15.125:0/2013794148 conn(0x7f15b40b3f00 msgr2=0x7f15b40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.325+0000 7f15bb15e640 1 -- 172.21.15.125:0/2013794148 shutdown_connections 2023-10-18T10:25:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.325+0000 7f15bb15e640 1 -- 172.21.15.125:0/2013794148 wait complete. 2023-10-18T10:25:26.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.326+0000 7f15bb15e640 1 Processor -- start 2023-10-18T10:25:26.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.326+0000 7f15bb15e640 1 -- start start 2023-10-18T10:25:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.326+0000 7f15bb15e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40acd40 0x7f15b414a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15bb15e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40b24d0 0x7f15b40c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15bb15e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b40b3070 0x7f15b40c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15bb15e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b406c770 con 0x7f15b40acd40 2023-10-18T10:25:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15bb15e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b406c2e0 con 0x7f15b40b24d0 2023-10-18T10:25:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15bb15e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b406b6d0 con 0x7f15b40b3070 2023-10-18T10:25:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40acd40 0x7f15b414a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40acd40 0x7f15b414a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50072/0 (socket says 172.21.15.125:50072) 2023-10-18T10:25:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 -- 172.21.15.125:0/3173893232 learned_addr learned my addr 172.21.15.125:0/3173893232 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b3fff640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40b24d0 0x7f15b40c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b96fb640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b40b3070 0x7f15b40c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 -- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b40b3070 msgr2=0x7f15b40c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b40b3070 0x7f15b40c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 -- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40b24d0 msgr2=0x7f15b40c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40b24d0 0x7f15b40c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 -- 172.21.15.125:0/3173893232 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0009560 con 0x7f15b40acd40 2023-10-18T10:25:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b8efa640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40acd40 0x7f15b414a490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f15a800c970 tx=0x7f15a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b17fa640 1 -- 172.21.15.125:0/3173893232 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8007bb0 con 0x7f15b40acd40 2023-10-18T10:25:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b17fa640 1 -- 172.21.15.125:0/3173893232 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f15a8007e20 con 0x7f15b40acd40 2023-10-18T10:25:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b406b9b0 con 0x7f15b40acd40 2023-10-18T10:25:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15b17fa640 1 -- 172.21.15.125:0/3173893232 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8005280 con 0x7f15b40acd40 2023-10-18T10:25:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.327+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b406bf70 con 0x7f15b40acd40 2023-10-18T10:25:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.328+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b40690e0 con 0x7f15b40acd40 2023-10-18T10:25:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.329+0000 7f15b17fa640 1 -- 172.21.15.125:0/3173893232 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f15a80053e0 con 0x7f15b40acd40 2023-10-18T10:25:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.329+0000 7f15b17fa640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f159806ae40 0x7f159806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.329+0000 7f15b17fa640 1 -- 172.21.15.125:0/3173893232 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f15a8089ee0 con 0x7f15b40acd40 2023-10-18T10:25:26.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.329+0000 7f15b3fff640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f159806ae40 0x7f159806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.330+0000 7f15b3fff640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f159806ae40 0x7f159806d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15a0035000 tx=0x7f15a0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:26.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.333+0000 7f15b17fa640 1 -- 172.21.15.125:0/3173893232 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15a8054cc0 con 0x7f15b40acd40 2023-10-18T10:25:26.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.535+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f15b41561b0 con 0x7f15b40acd40 2023-10-18T10:25:26.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.535+0000 7f15b17fa640 1 -- 172.21.15.125:0/3173893232 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f15a801d070 con 0x7f15b40acd40 2023-10-18T10:25:26.538 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:26.538 INFO:teuthology.orchestra.run.smithi125.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-18T10:25:26.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f159806ae40 msgr2=0x7f159806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f159806ae40 0x7f159806d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15a0035000 tx=0x7f15a0004730 comp rx=0 tx=0).stop 2023-10-18T10:25:26.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40acd40 msgr2=0x7f15b414a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40acd40 0x7f15b414a490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f15a800c970 tx=0x7f15a800ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:26.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 shutdown_connections 2023-10-18T10:25:26.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f159806ae40 0x7f159806d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40b24d0 0x7f15b40c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b40b3070 0x7f15b40c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 --2- 172.21.15.125:0/3173893232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15b40acd40 0x7f15b414a490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 >> 172.21.15.125:0/3173893232 conn(0x7f15b40b3f00 msgr2=0x7f15b4153620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:26.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 shutdown_connections 2023-10-18T10:25:26.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.537+0000 7f15bb15e640 1 -- 172.21.15.125:0/3173893232 wait complete. 2023-10-18T10:25:26.550 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-18T10:25:26.550 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:26.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.651+0000 7f472d52c640 1 -- 172.21.15.125:0/3874187216 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472815b260 msgr2=0x7f47280cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.651+0000 7f472d52c640 1 --2- 172.21.15.125:0/3874187216 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472815b260 0x7f47280cf310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4718005fd0 tx=0x7f471802c0e0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.651+0000 7f472d52c640 1 -- 172.21.15.125:0/3874187216 shutdown_connections 2023-10-18T10:25:26.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.651+0000 7f472d52c640 1 --2- 172.21.15.125:0/3874187216 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472815b260 0x7f47280cf310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.651+0000 7f472d52c640 1 -- 172.21.15.125:0/3874187216 >> 172.21.15.125:0/3874187216 conn(0x7f472809a030 msgr2=0x7f47280d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:26.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.651+0000 7f472d52c640 1 -- 172.21.15.125:0/3874187216 shutdown_connections 2023-10-18T10:25:26.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.651+0000 7f472d52c640 1 -- 172.21.15.125:0/3874187216 wait complete. 2023-10-18T10:25:26.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.652+0000 7f472d52c640 1 Processor -- start 2023-10-18T10:25:26.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.652+0000 7f472d52c640 1 -- start start 2023-10-18T10:25:26.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f472d52c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47280cf850 0x7f472814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f472d52c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f472815b260 0x7f47280b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f472d52c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f472815b640 0x7f47280b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f472d52c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472814f060 con 0x7f47280cf850 2023-10-18T10:25:26.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f472d52c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47281f13b0 con 0x7f472815b260 2023-10-18T10:25:26.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f472d52c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280b63a0 con 0x7f472815b640 2023-10-18T10:25:26.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47280cf850 0x7f472814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47280cf850 0x7f472814eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50078/0 (socket says 172.21.15.125:50078) 2023-10-18T10:25:26.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 -- 172.21.15.125:0/1476526446 learned_addr learned my addr 172.21.15.125:0/1476526446 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:26.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f47267fc640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f472815b260 0x7f47280b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f47277fe640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f472815b640 0x7f47280b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 -- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f472815b640 msgr2=0x7f47280b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f472815b640 0x7f47280b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 -- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f472815b260 msgr2=0x7f47280b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f472815b260 0x7f47280b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 -- 172.21.15.125:0/1476526446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718009840 con 0x7f47280cf850 2023-10-18T10:25:26.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f47267fc640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f472815b260 0x7f47280b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:26.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.653+0000 7f4726ffd640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47280cf850 0x7f472814eb20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4718005fd0 tx=0x7f4718004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:26.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.654+0000 7f470f7fe640 1 -- 172.21.15.125:0/1476526446 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718004760 con 0x7f47280cf850 2023-10-18T10:25:26.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.654+0000 7f470f7fe640 1 -- 172.21.15.125:0/1476526446 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47180345a0 con 0x7f47280cf850 2023-10-18T10:25:26.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.654+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47280b5b00 con 0x7f47280cf850 2023-10-18T10:25:26.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.654+0000 7f470f7fe640 1 -- 172.21.15.125:0/1476526446 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718034970 con 0x7f47280cf850 2023-10-18T10:25:26.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.654+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47280b60c0 con 0x7f47280cf850 2023-10-18T10:25:26.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.655+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47280675f0 con 0x7f47280cf850 2023-10-18T10:25:26.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.655+0000 7f470f7fe640 1 -- 172.21.15.125:0/1476526446 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4718034ad0 con 0x7f47280cf850 2023-10-18T10:25:26.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.655+0000 7f470f7fe640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f471406ae40 0x7f471406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.655+0000 7f470f7fe640 1 -- 172.21.15.125:0/1476526446 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f47180acbd0 con 0x7f47280cf850 2023-10-18T10:25:26.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.656+0000 7f47267fc640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f471406ae40 0x7f471406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.656+0000 7f47267fc640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f471406ae40 0x7f471406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f47280b4920 tx=0x7f471c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:26.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.660+0000 7f470f7fe640 1 -- 172.21.15.125:0/1476526446 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4718077a40 con 0x7f47280cf850 2023-10-18T10:25:26.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.826+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47280b0f40 con 0x7f47280cf850 2023-10-18T10:25:26.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.828+0000 7f470f7fe640 1 -- 172.21.15.125:0/1476526446 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f4718043090 con 0x7f47280cf850 2023-10-18T10:25:26.830 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:26.830 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:26.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f471406ae40 msgr2=0x7f471406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f471406ae40 0x7f471406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f47280b4920 tx=0x7f471c008040 comp rx=0 tx=0).stop 2023-10-18T10:25:26.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47280cf850 msgr2=0x7f472814eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47280cf850 0x7f472814eb20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4718005fd0 tx=0x7f4718004360 comp rx=0 tx=0).stop 2023-10-18T10:25:26.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 shutdown_connections 2023-10-18T10:25:26.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f471406ae40 0x7f471406d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f472815b260 0x7f47280b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f472815b640 0x7f47280b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 --2- 172.21.15.125:0/1476526446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47280cf850 0x7f472814eb20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.829+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 >> 172.21.15.125:0/1476526446 conn(0x7f472809a030 msgr2=0x7f472815a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:26.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.830+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 shutdown_connections 2023-10-18T10:25:26.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.830+0000 7f472d52c640 1 -- 172.21.15.125:0/1476526446 wait complete. 2023-10-18T10:25:26.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:26.843 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:26.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.946+0000 7f0acd887640 1 -- 172.21.15.125:0/1856471267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac815d400 msgr2=0x7f0ac80cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.946+0000 7f0acd887640 1 --2- 172.21.15.125:0/1856471267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac815d400 0x7f0ac80cdfb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0005fd0 tx=0x7f0ac002c1b0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.946+0000 7f0acd887640 1 -- 172.21.15.125:0/1856471267 shutdown_connections 2023-10-18T10:25:26.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.946+0000 7f0acd887640 1 --2- 172.21.15.125:0/1856471267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac815d400 0x7f0ac80cdfb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.946+0000 7f0acd887640 1 --2- 172.21.15.125:0/1856471267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac8159060 0x7f0ac806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.946+0000 7f0acd887640 1 -- 172.21.15.125:0/1856471267 >> 172.21.15.125:0/1856471267 conn(0x7f0ac80b0670 msgr2=0x7f0ac80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:26.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.946+0000 7f0acd887640 1 -- 172.21.15.125:0/1856471267 shutdown_connections 2023-10-18T10:25:26.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.946+0000 7f0acd887640 1 -- 172.21.15.125:0/1856471267 wait complete. 2023-10-18T10:25:26.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.947+0000 7f0acd887640 1 Processor -- start 2023-10-18T10:25:26.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.947+0000 7f0acd887640 1 -- start start 2023-10-18T10:25:26.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0acd887640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac80cdb30 0x7f0ac80ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0acd887640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac8159060 0x7f0ac815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0acd887640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac815d400 0x7f0ac815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0acd887640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80c3340 con 0x7f0ac80cdb30 2023-10-18T10:25:26.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0acd887640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac815a670 con 0x7f0ac815d400 2023-10-18T10:25:26.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0acd887640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac815a950 con 0x7f0ac8159060 2023-10-18T10:25:26.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac80cdb30 0x7f0ac80ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac80cdb30 0x7f0ac80ced70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50100/0 (socket says 172.21.15.125:50100) 2023-10-18T10:25:26.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac6ffd640 1 -- 172.21.15.125:0/1128049407 learned_addr learned my addr 172.21.15.125:0/1128049407 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:26.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac77fe640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac815d400 0x7f0ac815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac6ffd640 1 -- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac8159060 msgr2=0x7f0ac815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0abe7fc640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac8159060 0x7f0ac815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac6ffd640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac8159060 0x7f0ac815bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac6ffd640 1 -- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac815d400 msgr2=0x7f0ac815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:26.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac6ffd640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac815d400 0x7f0ac815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:26.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac6ffd640 1 -- 172.21.15.125:0/1128049407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac0009900 con 0x7f0ac80cdb30 2023-10-18T10:25:26.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.948+0000 7f0ac77fe640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac815d400 0x7f0ac815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:26.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.949+0000 7f0ac6ffd640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac80cdb30 0x7f0ac80ced70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c970 tx=0x7f0ab800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:26.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.949+0000 7f0ac4ff9640 1 -- 172.21.15.125:0/1128049407 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8007bb0 con 0x7f0ac80cdb30 2023-10-18T10:25:26.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.949+0000 7f0ac4ff9640 1 -- 172.21.15.125:0/1128049407 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ab8007e20 con 0x7f0ac80cdb30 2023-10-18T10:25:26.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.949+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac815ac30 con 0x7f0ac80cdb30 2023-10-18T10:25:26.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.949+0000 7f0ac4ff9640 1 -- 172.21.15.125:0/1128049407 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8005280 con 0x7f0ac80cdb30 2023-10-18T10:25:26.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.949+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac80cf2b0 con 0x7f0ac80cdb30 2023-10-18T10:25:26.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.950+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac806eea0 con 0x7f0ac80cdb30 2023-10-18T10:25:26.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.950+0000 7f0ac4ff9640 1 -- 172.21.15.125:0/1128049407 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0ab80053e0 con 0x7f0ac80cdb30 2023-10-18T10:25:26.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.950+0000 7f0ac4ff9640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0aa806ae40 0x7f0aa806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:26.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.951+0000 7f0ac4ff9640 1 -- 172.21.15.125:0/1128049407 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0ab8089ea0 con 0x7f0ac80cdb30 2023-10-18T10:25:26.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.951+0000 7f0abe7fc640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0aa806ae40 0x7f0aa806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:26.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.951+0000 7f0abe7fc640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0aa806ae40 0x7f0aa806d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ac81f27f0 tx=0x7f0ab0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:26.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:26.955+0000 7f0ac4ff9640 1 -- 172.21.15.125:0/1128049407 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ab8054c60 con 0x7f0ac80cdb30 2023-10-18T10:25:27.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.120+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ac80675f0 con 0x7f0ac80cdb30 2023-10-18T10:25:27.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.120+0000 7f0ac4ff9640 1 -- 172.21.15.125:0/1128049407 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ab801d070 con 0x7f0ac80cdb30 2023-10-18T10:25:27.122 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:27.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0aa806ae40 msgr2=0x7f0aa806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0aa806ae40 0x7f0aa806d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ac81f27f0 tx=0x7f0ab0009040 comp rx=0 tx=0).stop 2023-10-18T10:25:27.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac80cdb30 msgr2=0x7f0ac80ced70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac80cdb30 0x7f0ac80ced70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c970 tx=0x7f0ab800ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:27.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 shutdown_connections 2023-10-18T10:25:27.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0aa806ae40 0x7f0aa806d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac815d400 0x7f0ac815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac8159060 0x7f0ac815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 --2- 172.21.15.125:0/1128049407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ac80cdb30 0x7f0ac80ced70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 >> 172.21.15.125:0/1128049407 conn(0x7f0ac80b0670 msgr2=0x7f0ac80b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:27.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 shutdown_connections 2023-10-18T10:25:27.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.122+0000 7f0acd887640 1 -- 172.21.15.125:0/1128049407 wait complete. 2023-10-18T10:25:27.134 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:27.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.239+0000 7fe4072c2640 1 -- 172.21.15.125:0/2044357564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000d1910 msgr2=0x7fe4000b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.239+0000 7fe4072c2640 1 --2- 172.21.15.125:0/2044357564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000d1910 0x7fe4000b0650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005fd0 tx=0x7fe3f802c0e0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.239+0000 7fe4072c2640 1 -- 172.21.15.125:0/2044357564 shutdown_connections 2023-10-18T10:25:27.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.239+0000 7fe4072c2640 1 --2- 172.21.15.125:0/2044357564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000d1910 0x7fe4000b0650 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.239+0000 7fe4072c2640 1 -- 172.21.15.125:0/2044357564 >> 172.21.15.125:0/2044357564 conn(0x7fe4000d0d60 msgr2=0x7fe4000d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:27.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.239+0000 7fe4072c2640 1 -- 172.21.15.125:0/2044357564 shutdown_connections 2023-10-18T10:25:27.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.239+0000 7fe4072c2640 1 -- 172.21.15.125:0/2044357564 wait complete. 2023-10-18T10:25:27.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.240+0000 7fe4072c2640 1 Processor -- start 2023-10-18T10:25:27.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.240+0000 7fe4072c2640 1 -- start start 2023-10-18T10:25:27.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe4072c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000cf340 0x7fe4000b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe4072c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4000d1910 0x7fe400157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe4072c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe40015b2a0 0x7fe4001f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe4072c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001f20d0 con 0x7fe4000cf340 2023-10-18T10:25:27.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe4072c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001f23b0 con 0x7fe4000d1910 2023-10-18T10:25:27.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe4072c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b3ad0 con 0x7fe40015b2a0 2023-10-18T10:25:27.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe40505e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000cf340 0x7fe4000b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe40505e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000cf340 0x7fe4000b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50116/0 (socket says 172.21.15.125:50116) 2023-10-18T10:25:27.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe40505e640 1 -- 172.21.15.125:0/2731642247 learned_addr learned my addr 172.21.15.125:0/2731642247 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:27.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe40485d640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4000d1910 0x7fe400157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe40585f640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe40015b2a0 0x7fe4001f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe40505e640 1 -- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe40015b2a0 msgr2=0x7fe4001f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe40505e640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe40015b2a0 0x7fe4001f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.241+0000 7fe40505e640 1 -- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4000d1910 msgr2=0x7fe400157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe40505e640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4000d1910 0x7fe400157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe40505e640 1 -- 172.21.15.125:0/2731642247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8009840 con 0x7fe4000cf340 2023-10-18T10:25:27.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe40585f640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe40015b2a0 0x7fe4001f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:27.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe40505e640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000cf340 0x7fe4000b42d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c450 tx=0x7fe3f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:27.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe3f5ffb640 1 -- 172.21.15.125:0/2731642247 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00052d0 con 0x7fe4000cf340 2023-10-18T10:25:27.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe3f5ffb640 1 -- 172.21.15.125:0/2731642247 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3f0005430 con 0x7fe4000cf340 2023-10-18T10:25:27.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4000b3db0 con 0x7fe4000cf340 2023-10-18T10:25:27.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe3f5ffb640 1 -- 172.21.15.125:0/2731642247 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0012890 con 0x7fe4000cf340 2023-10-18T10:25:27.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.242+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4000caa10 con 0x7fe4000cf340 2023-10-18T10:25:27.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.243+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4001f25d0 con 0x7fe4000cf340 2023-10-18T10:25:27.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.243+0000 7fe3f5ffb640 1 -- 172.21.15.125:0/2731642247 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe3f00035f0 con 0x7fe4000cf340 2023-10-18T10:25:27.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.244+0000 7fe3f5ffb640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3e406ae40 0x7fe3e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.244+0000 7fe3f5ffb640 1 -- 172.21.15.125:0/2731642247 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe3f0089510 con 0x7fe4000cf340 2023-10-18T10:25:27.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.244+0000 7fe40485d640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3e406ae40 0x7fe3e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.244+0000 7fe40485d640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3e406ae40 0x7fe3e406d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc001150 tx=0x7fe3fc0010e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:27.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.248+0000 7fe3f5ffb640 1 -- 172.21.15.125:0/2731642247 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3f00542d0 con 0x7fe4000cf340 2023-10-18T10:25:27.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.413+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4000d04f0 con 0x7fe4000cf340 2023-10-18T10:25:27.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.413+0000 7fe3f5ffb640 1 -- 172.21.15.125:0/2731642247 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3f00578f0 con 0x7fe4000cf340 2023-10-18T10:25:27.416 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:27.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3e406ae40 msgr2=0x7fe3e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3e406ae40 0x7fe3e406d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc001150 tx=0x7fe3fc0010e0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000cf340 msgr2=0x7fe4000b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000cf340 0x7fe4000b42d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c450 tx=0x7fe3f000c910 comp rx=0 tx=0).stop 2023-10-18T10:25:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 shutdown_connections 2023-10-18T10:25:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3e406ae40 0x7fe3e406d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4000d1910 0x7fe400157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe40015b2a0 0x7fe4001f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 --2- 172.21.15.125:0/2731642247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4000cf340 0x7fe4000b42d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 >> 172.21.15.125:0/2731642247 conn(0x7fe4000d0d60 msgr2=0x7fe4000ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 shutdown_connections 2023-10-18T10:25:27.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.415+0000 7fe4072c2640 1 -- 172.21.15.125:0/2731642247 wait complete. 2023-10-18T10:25:27.427 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:27.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.530+0000 7f9f6904f640 1 -- 172.21.15.125:0/2547632838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f6415b2a0 msgr2=0x7f9f640d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.530+0000 7f9f6904f640 1 --2- 172.21.15.125:0/2547632838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f6415b2a0 0x7f9f640d1910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009830 tx=0x7f9f5802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.530+0000 7f9f6904f640 1 -- 172.21.15.125:0/2547632838 shutdown_connections 2023-10-18T10:25:27.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.530+0000 7f9f6904f640 1 --2- 172.21.15.125:0/2547632838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f640ceb40 0x7f9f640b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.530+0000 7f9f6904f640 1 --2- 172.21.15.125:0/2547632838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f6415b2a0 0x7f9f640d1910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.530+0000 7f9f6904f640 1 -- 172.21.15.125:0/2547632838 >> 172.21.15.125:0/2547632838 conn(0x7f9f640d0d60 msgr2=0x7f9f640d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:27.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.530+0000 7f9f6904f640 1 -- 172.21.15.125:0/2547632838 shutdown_connections 2023-10-18T10:25:27.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.530+0000 7f9f6904f640 1 -- 172.21.15.125:0/2547632838 wait complete. 2023-10-18T10:25:27.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.531+0000 7f9f6904f640 1 Processor -- start 2023-10-18T10:25:27.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6904f640 1 -- start start 2023-10-18T10:25:27.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6904f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f640ceb40 0x7f9f640c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6904f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f640cf340 0x7f9f6414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6904f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f6415b2a0 0x7f9f6415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6904f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640c8410 con 0x7f9f640cf340 2023-10-18T10:25:27.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6904f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640c86f0 con 0x7f9f6415b2a0 2023-10-18T10:25:27.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6904f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640c89d0 con 0x7f9f640ceb40 2023-10-18T10:25:27.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f640cf340 0x7f9f6414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f640cf340 0x7f9f6414fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48486/0 (socket says 172.21.15.125:48486) 2023-10-18T10:25:27.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6259c640 1 -- 172.21.15.125:0/123602853 learned_addr learned my addr 172.21.15.125:0/123602853 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:27.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f62d9d640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f640ceb40 0x7f9f640c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.532+0000 7f9f6359e640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f6415b2a0 0x7f9f6415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f6259c640 1 -- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f640ceb40 msgr2=0x7f9f640c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f6259c640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f640ceb40 0x7f9f640c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f6259c640 1 -- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f6415b2a0 msgr2=0x7f9f6415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f6259c640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f6415b2a0 0x7f9f6415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f6259c640 1 -- 172.21.15.125:0/123602853 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f58009490 con 0x7f9f640cf340 2023-10-18T10:25:27.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f6259c640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f640cf340 0x7f9f6414fda0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009780 tx=0x7f9f580038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:27.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f4b7fe640 1 -- 172.21.15.125:0/123602853 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58003ea0 con 0x7f9f640cf340 2023-10-18T10:25:27.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f4b7fe640 1 -- 172.21.15.125:0/123602853 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9f58034550 con 0x7f9f640cf340 2023-10-18T10:25:27.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f640c8cb0 con 0x7f9f640cf340 2023-10-18T10:25:27.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f4b7fe640 1 -- 172.21.15.125:0/123602853 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58034960 con 0x7f9f640cf340 2023-10-18T10:25:27.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.533+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6415d830 con 0x7f9f640cf340 2023-10-18T10:25:27.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.534+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f640d07d0 con 0x7f9f640cf340 2023-10-18T10:25:27.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.534+0000 7f9f4b7fe640 1 -- 172.21.15.125:0/123602853 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9f5803b040 con 0x7f9f640cf340 2023-10-18T10:25:27.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.535+0000 7f9f4b7fe640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9f5006ae40 0x7f9f5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.535+0000 7f9f4b7fe640 1 -- 172.21.15.125:0/123602853 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9f580abcf0 con 0x7f9f640cf340 2023-10-18T10:25:27.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.535+0000 7f9f62d9d640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9f5006ae40 0x7f9f5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.535+0000 7f9f62d9d640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9f5006ae40 0x7f9f5006d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f540012e0 tx=0x7f9f54009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:27.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.539+0000 7f9f4b7fe640 1 -- 172.21.15.125:0/123602853 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f580b0070 con 0x7f9f640cf340 2023-10-18T10:25:27.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.704+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f640b0d30 con 0x7f9f640cf340 2023-10-18T10:25:27.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.704+0000 7f9f4b7fe640 1 -- 172.21.15.125:0/123602853 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f580390a0 con 0x7f9f640cf340 2023-10-18T10:25:27.707 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:27.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9f5006ae40 msgr2=0x7f9f5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9f5006ae40 0x7f9f5006d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f540012e0 tx=0x7f9f54009040 comp rx=0 tx=0).stop 2023-10-18T10:25:27.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f640cf340 msgr2=0x7f9f6414fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f640cf340 0x7f9f6414fda0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009780 tx=0x7f9f580038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 shutdown_connections 2023-10-18T10:25:27.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9f5006ae40 0x7f9f5006d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f6415b2a0 0x7f9f6415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f640cf340 0x7f9f6414fda0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 --2- 172.21.15.125:0/123602853 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f640ceb40 0x7f9f640c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 >> 172.21.15.125:0/123602853 conn(0x7f9f640d0d60 msgr2=0x7f9f640d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:27.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 shutdown_connections 2023-10-18T10:25:27.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.706+0000 7f9f6904f640 1 -- 172.21.15.125:0/123602853 wait complete. 2023-10-18T10:25:27.719 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:27.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.822+0000 7f8ac0896640 1 -- 172.21.15.125:0/2761329984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc06af00 msgr2=0x7f8abc06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.822+0000 7f8ac0896640 1 --2- 172.21.15.125:0/2761329984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc06af00 0x7f8abc06b2e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009be0 tx=0x7f8ab402c0e0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.822+0000 7f8ac0896640 1 -- 172.21.15.125:0/2761329984 shutdown_connections 2023-10-18T10:25:27.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.822+0000 7f8ac0896640 1 --2- 172.21.15.125:0/2761329984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc06af00 0x7f8abc06b2e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.822+0000 7f8ac0896640 1 -- 172.21.15.125:0/2761329984 >> 172.21.15.125:0/2761329984 conn(0x7f8abc069930 msgr2=0x7f8abc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:27.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.822+0000 7f8ac0896640 1 -- 172.21.15.125:0/2761329984 shutdown_connections 2023-10-18T10:25:27.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.822+0000 7f8ac0896640 1 -- 172.21.15.125:0/2761329984 wait complete. 2023-10-18T10:25:27.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.823+0000 7f8ac0896640 1 Processor -- start 2023-10-18T10:25:27.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.823+0000 7f8ac0896640 1 -- start start 2023-10-18T10:25:27.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ac0896640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8abc06af00 0x7f8abc13a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ac0896640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc0abc10 0x7f8abc0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ac0896640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8abc0c96b0 0x7f8abc0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ac0896640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0b08e0 con 0x7f8abc0abc10 2023-10-18T10:25:27.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ac0896640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc158fa0 con 0x7f8abc0c96b0 2023-10-18T10:25:27.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ac0896640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc13a900 con 0x7f8abc06af00 2023-10-18T10:25:27.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc0abc10 0x7f8abc0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc0abc10 0x7f8abc0af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48514/0 (socket says 172.21.15.125:48514) 2023-10-18T10:25:27.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 -- 172.21.15.125:0/1484067388 learned_addr learned my addr 172.21.15.125:0/1484067388 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:27.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8aba59c640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8abc06af00 0x7f8abc13a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 -- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8abc06af00 msgr2=0x7f8abc13a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8abc06af00 0x7f8abc13a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8abad9d640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8abc0c96b0 0x7f8abc0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 -- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8abc0c96b0 msgr2=0x7f8abc0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:27.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8abc0c96b0 0x7f8abc0af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:27.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 -- 172.21.15.125:0/1484067388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009840 con 0x7f8abc0abc10 2023-10-18T10:25:27.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8aba59c640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8abc06af00 0x7f8abc13a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:27.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8abad9d640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8abc0c96b0 0x7f8abc0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:27.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.824+0000 7f8ab9d9b640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc0abc10 0x7f8abc0af110 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000c3c0 tx=0x7f8ab000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:27.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.825+0000 7f8aaaffd640 1 -- 172.21.15.125:0/1484067388 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00035f0 con 0x7f8abc0abc10 2023-10-18T10:25:27.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.825+0000 7f8aaaffd640 1 -- 172.21.15.125:0/1484067388 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ab0003770 con 0x7f8abc0abc10 2023-10-18T10:25:27.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.825+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc0afd10 con 0x7f8abc0abc10 2023-10-18T10:25:27.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.825+0000 7f8aaaffd640 1 -- 172.21.15.125:0/1484067388 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0003b80 con 0x7f8abc0abc10 2023-10-18T10:25:27.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.825+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc0b0290 con 0x7f8abc0abc10 2023-10-18T10:25:27.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.826+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc0b0a50 con 0x7f8abc0abc10 2023-10-18T10:25:27.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.826+0000 7f8aaaffd640 1 -- 172.21.15.125:0/1484067388 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8ab0003ce0 con 0x7f8abc0abc10 2023-10-18T10:25:27.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.826+0000 7f8aaaffd640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a9c06ae40 0x7f8a9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:27.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.827+0000 7f8aaaffd640 1 -- 172.21.15.125:0/1484067388 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8ab0088f50 con 0x7f8abc0abc10 2023-10-18T10:25:27.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.827+0000 7f8aba59c640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a9c06ae40 0x7f8a9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:27.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.827+0000 7f8aba59c640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a9c06ae40 0x7f8a9c06d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4035000 tx=0x7f8ab4004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:27.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:27.831+0000 7f8aaaffd640 1 -- 172.21.15.125:0/1484067388 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ab0053da0 con 0x7f8abc0abc10 2023-10-18T10:25:28.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.000+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8abc0690e0 con 0x7f8abc0abc10 2023-10-18T10:25:28.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.000+0000 7f8aaaffd640 1 -- 172.21.15.125:0/1484067388 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8ab0017020 con 0x7f8abc0abc10 2023-10-18T10:25:28.003 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:28.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.002+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a9c06ae40 msgr2=0x7f8a9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.002+0000 7f8ac0896640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a9c06ae40 0x7f8a9c06d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4035000 tx=0x7f8ab4004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc0abc10 msgr2=0x7f8abc0af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc0abc10 0x7f8abc0af110 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000c3c0 tx=0x7f8ab000c880 comp rx=0 tx=0).stop 2023-10-18T10:25:28.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 shutdown_connections 2023-10-18T10:25:28.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a9c06ae40 0x7f8a9c06d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8abc0c96b0 0x7f8abc0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8abc0abc10 0x7f8abc0af110 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 --2- 172.21.15.125:0/1484067388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8abc06af00 0x7f8abc13a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 >> 172.21.15.125:0/1484067388 conn(0x7f8abc069930 msgr2=0x7f8abc0cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:28.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 shutdown_connections 2023-10-18T10:25:28.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.003+0000 7f8ac0896640 1 -- 172.21.15.125:0/1484067388 wait complete. 2023-10-18T10:25:28.015 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:25:28.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.116+0000 7fc5d90e1640 1 -- 172.21.15.125:0/3336502954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5d413e520 msgr2=0x7fc5d406bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.116+0000 7fc5d90e1640 1 --2- 172.21.15.125:0/3336502954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5d413e520 0x7fc5d406bb60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4005fd0 tx=0x7fc5c402bfc0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.116+0000 7fc5d90e1640 1 -- 172.21.15.125:0/3336502954 shutdown_connections 2023-10-18T10:25:28.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.116+0000 7fc5d90e1640 1 --2- 172.21.15.125:0/3336502954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5d413e520 0x7fc5d406bb60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.116+0000 7fc5d90e1640 1 -- 172.21.15.125:0/3336502954 >> 172.21.15.125:0/3336502954 conn(0x7fc5d4069930 msgr2=0x7fc5d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:28.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.116+0000 7fc5d90e1640 1 -- 172.21.15.125:0/3336502954 shutdown_connections 2023-10-18T10:25:28.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.116+0000 7fc5d90e1640 1 -- 172.21.15.125:0/3336502954 wait complete. 2023-10-18T10:25:28.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.117+0000 7fc5d90e1640 1 Processor -- start 2023-10-18T10:25:28.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.117+0000 7fc5d90e1640 1 -- start start 2023-10-18T10:25:28.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d90e1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5d4070af0 0x7fc5d41555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d90e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5d413e520 0x7fc5d40add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d90e1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5d4141690 0x7fc5d40ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d90e1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d41e1fd0 con 0x7fc5d4070af0 2023-10-18T10:25:28.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d90e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d41e22b0 con 0x7fc5d413e520 2023-10-18T10:25:28.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d90e1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d41e2590 con 0x7fc5d4141690 2023-10-18T10:25:28.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5d4070af0 0x7fc5d41555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5d4070af0 0x7fc5d41555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48530/0 (socket says 172.21.15.125:48530) 2023-10-18T10:25:28.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d2d9d640 1 -- 172.21.15.125:0/633581044 learned_addr learned my addr 172.21.15.125:0/633581044 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:28.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d259c640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5d413e520 0x7fc5d40add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d359e640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5d4141690 0x7fc5d40ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d2d9d640 1 -- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5d4141690 msgr2=0x7fc5d40ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d2d9d640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5d4141690 0x7fc5d40ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d2d9d640 1 -- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5d413e520 msgr2=0x7fc5d40add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d2d9d640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5d413e520 0x7fc5d40add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.118+0000 7fc5d2d9d640 1 -- 172.21.15.125:0/633581044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c4009840 con 0x7fc5d4070af0 2023-10-18T10:25:28.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.119+0000 7fc5d2d9d640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5d4070af0 0x7fc5d41555c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006010 tx=0x7fc5c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:28.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.119+0000 7fc5bb7fe640 1 -- 172.21.15.125:0/633581044 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4003e20 con 0x7fc5d4070af0 2023-10-18T10:25:28.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.119+0000 7fc5bb7fe640 1 -- 172.21.15.125:0/633581044 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc5c40047e0 con 0x7fc5d4070af0 2023-10-18T10:25:28.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.119+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d41e2870 con 0x7fc5d4070af0 2023-10-18T10:25:28.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.119+0000 7fc5bb7fe640 1 -- 172.21.15.125:0/633581044 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c403e6b0 con 0x7fc5d4070af0 2023-10-18T10:25:28.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.119+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d4155b00 con 0x7fc5d4070af0 2023-10-18T10:25:28.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.120+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d40690e0 con 0x7fc5d4070af0 2023-10-18T10:25:28.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.120+0000 7fc5bb7fe640 1 -- 172.21.15.125:0/633581044 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc5c403e810 con 0x7fc5d4070af0 2023-10-18T10:25:28.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.120+0000 7fc5bb7fe640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5c006ae40 0x7fc5c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.121+0000 7fc5bb7fe640 1 -- 172.21.15.125:0/633581044 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc5c40ac790 con 0x7fc5d4070af0 2023-10-18T10:25:28.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.121+0000 7fc5d259c640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5c006ae40 0x7fc5c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.121+0000 7fc5d259c640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5c006ae40 0x7fc5c006d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc5d413b220 tx=0x7fc5c8001d20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:28.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.125+0000 7fc5bb7fe640 1 -- 172.21.15.125:0/633581044 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5c4077600 con 0x7fc5d4070af0 2023-10-18T10:25:28.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.334+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5d4070480 con 0x7fc5d4070af0 2023-10-18T10:25:28.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.334+0000 7fc5bb7fe640 1 -- 172.21.15.125:0/633581044 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fc5c407ac20 con 0x7fc5d4070af0 2023-10-18T10:25:28.336 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:28.337 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:19.519174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:25:28.337 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 9 2023-10-18T10:25:28.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5c006ae40 msgr2=0x7fc5c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5c006ae40 0x7fc5c006d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc5d413b220 tx=0x7fc5c8001d20 comp rx=0 tx=0).stop 2023-10-18T10:25:28.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5d4070af0 msgr2=0x7fc5d41555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5d4070af0 0x7fc5d41555c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006010 tx=0x7fc5c40042f0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 shutdown_connections 2023-10-18T10:25:28.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc5c006ae40 0x7fc5c006d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5d413e520 0x7fc5d40add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5d4141690 0x7fc5d40ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 --2- 172.21.15.125:0/633581044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5d4070af0 0x7fc5d41555c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 >> 172.21.15.125:0/633581044 conn(0x7fc5d4069930 msgr2=0x7fc5d4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:28.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 shutdown_connections 2023-10-18T10:25:28.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.336+0000 7fc5d90e1640 1 -- 172.21.15.125:0/633581044 wait complete. 2023-10-18T10:25:28.349 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:28.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.449+0000 7f844344b640 1 -- 172.21.15.125:0/1550710768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84280047c0 msgr2=0x7f8428004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.449+0000 7f844344b640 1 --2- 172.21.15.125:0/1550710768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84280047c0 0x7f8428004c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f842c006fa0 tx=0x7f84280057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.450+0000 7f844344b640 1 -- 172.21.15.125:0/1550710768 shutdown_connections 2023-10-18T10:25:28.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.450+0000 7f844344b640 1 --2- 172.21.15.125:0/1550710768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84280047c0 0x7f8428004c40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.450+0000 7f844344b640 1 -- 172.21.15.125:0/1550710768 >> 172.21.15.125:0/1550710768 conn(0x7f843c0aec40 msgr2=0x7f843c0af050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:28.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.450+0000 7f844344b640 1 -- 172.21.15.125:0/1550710768 shutdown_connections 2023-10-18T10:25:28.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.450+0000 7f844344b640 1 -- 172.21.15.125:0/1550710768 wait complete. 2023-10-18T10:25:28.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.451+0000 7f844344b640 1 Processor -- start 2023-10-18T10:25:28.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.451+0000 7f844344b640 1 -- start start 2023-10-18T10:25:28.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.451+0000 7f844344b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84280047c0 0x7f843c1478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.451+0000 7f844344b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f843c158090 0x7f843c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.451+0000 7f844344b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f843c15a440 0x7f843c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.451+0000 7f844344b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c15ae30 con 0x7f843c158090 2023-10-18T10:25:28.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.451+0000 7f844344b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c155870 con 0x7f84280047c0 2023-10-18T10:25:28.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.451+0000 7f844344b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c158c00 con 0x7f843c15a440 2023-10-18T10:25:28.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f843c158090 0x7f843c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f843c158090 0x7f843c158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48544/0 (socket says 172.21.15.125:48544) 2023-10-18T10:25:28.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 -- 172.21.15.125:0/2093231291 learned_addr learned my addr 172.21.15.125:0/2093231291 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:28.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84411e7640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84280047c0 0x7f843c1478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84419e8640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f843c15a440 0x7f843c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 -- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f843c15a440 msgr2=0x7f843c15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f843c15a440 0x7f843c15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 -- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84280047c0 msgr2=0x7f843c1478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84280047c0 0x7f843c1478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 -- 172.21.15.125:0/2093231291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c158ee0 con 0x7f843c158090 2023-10-18T10:25:28.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f84409e6640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f843c158090 0x7f843c158510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f842c02c8a0 tx=0x7f842c02dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:28.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f8431ffb640 1 -- 172.21.15.125:0/2093231291 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c03c040 con 0x7f843c158090 2023-10-18T10:25:28.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f8431ffb640 1 -- 172.21.15.125:0/2093231291 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f842c0035f0 con 0x7f843c158090 2023-10-18T10:25:28.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c1591c0 con 0x7f843c158090 2023-10-18T10:25:28.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f8431ffb640 1 -- 172.21.15.125:0/2093231291 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0039e0 con 0x7f843c158090 2023-10-18T10:25:28.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.452+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c159780 con 0x7f843c158090 2023-10-18T10:25:28.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.453+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c0690e0 con 0x7f843c158090 2023-10-18T10:25:28.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.454+0000 7f8431ffb640 1 -- 172.21.15.125:0/2093231291 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f842c032070 con 0x7f843c158090 2023-10-18T10:25:28.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.454+0000 7f8431ffb640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f842006ae40 0x7f842006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.454+0000 7f8431ffb640 1 -- 172.21.15.125:0/2093231291 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f842c0aad30 con 0x7f843c158090 2023-10-18T10:25:28.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.454+0000 7f84411e7640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f842006ae40 0x7f842006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.455+0000 7f84411e7640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f842006ae40 0x7f842006d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8438009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:28.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.458+0000 7f8431ffb640 1 -- 172.21.15.125:0/2093231291 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f842c075c30 con 0x7f843c158090 2023-10-18T10:25:28.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.625+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f843c071440 con 0x7f843c158090 2023-10-18T10:25:28.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.626+0000 7f8431ffb640 1 -- 172.21.15.125:0/2093231291 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f842c046090 con 0x7f843c158090 2023-10-18T10:25:28.628 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:28.628 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:28.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f842006ae40 msgr2=0x7f842006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f842006ae40 0x7f842006d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8438009500 comp rx=0 tx=0).stop 2023-10-18T10:25:28.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f843c158090 msgr2=0x7f843c158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f843c158090 0x7f843c158510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f842c02c8a0 tx=0x7f842c02dfc0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 shutdown_connections 2023-10-18T10:25:28.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f842006ae40 0x7f842006d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f843c15a440 0x7f843c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f843c158090 0x7f843c158510 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 --2- 172.21.15.125:0/2093231291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84280047c0 0x7f843c1478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 >> 172.21.15.125:0/2093231291 conn(0x7f843c0aec40 msgr2=0x7f843c148370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:28.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 shutdown_connections 2023-10-18T10:25:28.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.628+0000 7f844344b640 1 -- 172.21.15.125:0/2093231291 wait complete. 2023-10-18T10:25:28.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:28.641 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:28.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.745+0000 7fc0223a9640 1 -- 172.21.15.125:0/4250384285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 msgr2=0x7fc004003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.745+0000 7fc0223a9640 1 --2- 172.21.15.125:0/4250384285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 0x7fc004003dc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009b50 tx=0x7fc0040057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.745+0000 7fc020946640 1 -- 172.21.15.125:0/4250384285 reap_dead start 2023-10-18T10:25:28.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.746+0000 7fc0223a9640 1 -- 172.21.15.125:0/4250384285 shutdown_connections 2023-10-18T10:25:28.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.746+0000 7fc0223a9640 1 -- 172.21.15.125:0/4250384285 >> 172.21.15.125:0/4250384285 conn(0x7fc01c0b3f00 msgr2=0x7fc01c0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:28.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.746+0000 7fc0223a9640 1 -- 172.21.15.125:0/4250384285 shutdown_connections 2023-10-18T10:25:28.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.746+0000 7fc0223a9640 1 -- 172.21.15.125:0/4250384285 wait complete. 2023-10-18T10:25:28.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.747+0000 7fc0223a9640 1 Processor -- start 2023-10-18T10:25:28.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.747+0000 7fc0223a9640 1 -- start start 2023-10-18T10:25:28.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc0223a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 0x7fc01c1f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc0223a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc01c142420 0x7fc01c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc0223a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01c14d590 0x7fc01c1f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc0223a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c1504f0 con 0x7fc0040039a0 2023-10-18T10:25:28.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc0223a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c1f1200 con 0x7fc01c14d590 2023-10-18T10:25:28.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc0223a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c150750 con 0x7fc01c142420 2023-10-18T10:25:28.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 0x7fc01c1f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 0x7fc01c1f54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48572/0 (socket says 172.21.15.125:48572) 2023-10-18T10:25:28.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 -- 172.21.15.125:0/1832479945 learned_addr learned my addr 172.21.15.125:0/1832479945 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:28.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01b7fe640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc01c142420 0x7fc01c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc020946640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01c14d590 0x7fc01c1f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 -- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc01c142420 msgr2=0x7fc01c14ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc01c142420 0x7fc01c14ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 -- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01c14d590 msgr2=0x7fc01c1f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01c14d590 0x7fc01c1f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 -- 172.21.15.125:0/1832479945 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c06a960 con 0x7fc0040039a0 2023-10-18T10:25:28.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc01bfff640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 0x7fc01c1f54c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009a70 tx=0x7fc00c033af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc018ff9640 1 -- 172.21.15.125:0/1832479945 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c004220 con 0x7fc0040039a0 2023-10-18T10:25:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc018ff9640 1 -- 172.21.15.125:0/1832479945 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc00c004380 con 0x7fc0040039a0 2023-10-18T10:25:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c1504f0 con 0x7fc0040039a0 2023-10-18T10:25:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc018ff9640 1 -- 172.21.15.125:0/1832479945 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c03d940 con 0x7fc0040039a0 2023-10-18T10:25:28.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.748+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c06aee0 con 0x7fc0040039a0 2023-10-18T10:25:28.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.749+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c0690e0 con 0x7fc0040039a0 2023-10-18T10:25:28.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.750+0000 7fc018ff9640 1 -- 172.21.15.125:0/1832479945 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc00c03daa0 con 0x7fc0040039a0 2023-10-18T10:25:28.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.750+0000 7fc018ff9640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc00806ae40 0x7fc00806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:28.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.750+0000 7fc018ff9640 1 -- 172.21.15.125:0/1832479945 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc00c0ac510 con 0x7fc0040039a0 2023-10-18T10:25:28.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.750+0000 7fc01b7fe640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc00806ae40 0x7fc00806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:28.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.751+0000 7fc01b7fe640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc00806ae40 0x7fc00806d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0ae090 tx=0x7fc010000f80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:28.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.754+0000 7fc018ff9640 1 -- 172.21.15.125:0/1832479945 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc00c077410 con 0x7fc0040039a0 2023-10-18T10:25:28.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.918+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc01c154640 con 0x7fc0040039a0 2023-10-18T10:25:28.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.919+0000 7fc018ff9640 1 -- 172.21.15.125:0/1832479945 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc00c04a020 con 0x7fc0040039a0 2023-10-18T10:25:28.921 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:28.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc00806ae40 msgr2=0x7fc00806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc00806ae40 0x7fc00806d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0ae090 tx=0x7fc010000f80 comp rx=0 tx=0).stop 2023-10-18T10:25:28.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 msgr2=0x7fc01c1f54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:28.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 0x7fc01c1f54c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009a70 tx=0x7fc00c033af0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 shutdown_connections 2023-10-18T10:25:28.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc00806ae40 0x7fc00806d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01c14d590 0x7fc01c1f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc01c142420 0x7fc01c14ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 --2- 172.21.15.125:0/1832479945 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0040039a0 0x7fc01c1f54c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:28.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 >> 172.21.15.125:0/1832479945 conn(0x7fc01c0b3f00 msgr2=0x7fc01c0cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:28.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 shutdown_connections 2023-10-18T10:25:28.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:28.921+0000 7fc0223a9640 1 -- 172.21.15.125:0/1832479945 wait complete. 2023-10-18T10:25:28.933 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:29.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.037+0000 7fc230c4b640 1 -- 172.21.15.125:0/4188125449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2140047c0 msgr2=0x7fc214004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.037+0000 7fc230c4b640 1 --2- 172.21.15.125:0/4188125449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc220006fa0 tx=0x7fc2140057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.037+0000 7fc230c4b640 1 -- 172.21.15.125:0/4188125449 shutdown_connections 2023-10-18T10:25:29.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.037+0000 7fc230c4b640 1 --2- 172.21.15.125:0/4188125449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2140047c0 0x7fc214004c40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.037+0000 7fc230c4b640 1 -- 172.21.15.125:0/4188125449 >> 172.21.15.125:0/4188125449 conn(0x7fc22c09a030 msgr2=0x7fc22c0d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:29.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.038+0000 7fc230c4b640 1 -- 172.21.15.125:0/4188125449 shutdown_connections 2023-10-18T10:25:29.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.038+0000 7fc230c4b640 1 -- 172.21.15.125:0/4188125449 wait complete. 2023-10-18T10:25:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.039+0000 7fc230c4b640 1 Processor -- start 2023-10-18T10:25:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.039+0000 7fc230c4b640 1 -- start start 2023-10-18T10:25:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.039+0000 7fc230c4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2140047c0 0x7fc22c157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.039+0000 7fc230c4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc22c0c4850 0x7fc22c0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.039+0000 7fc230c4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc22c0c67d0 0x7fc22c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.039+0000 7fc230c4b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c1f5730 con 0x7fc22c0c4850 2023-10-18T10:25:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.039+0000 7fc230c4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c1f4bb0 con 0x7fc2140047c0 2023-10-18T10:25:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.039+0000 7fc230c4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c1f4e90 con 0x7fc22c0c67d0 2023-10-18T10:25:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc22c0c4850 0x7fc22c0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc22c0c4850 0x7fc22c0c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48582/0 (socket says 172.21.15.125:48582) 2023-10-18T10:25:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 -- 172.21.15.125:0/2138090445 learned_addr learned my addr 172.21.15.125:0/2138090445 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc22ad9d640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc22c0c67d0 0x7fc22c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc22a59c640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2140047c0 0x7fc22c157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 -- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc22c0c67d0 msgr2=0x7fc22c0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc22c0c67d0 0x7fc22c0ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 -- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2140047c0 msgr2=0x7fc22c157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2140047c0 0x7fc22c157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 -- 172.21.15.125:0/2138090445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c1f5170 con 0x7fc22c0c4850 2023-10-18T10:25:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc229d9b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc22c0c4850 0x7fc22c0c6290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc22002c8c0 tx=0x7fc220001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc21affd640 1 -- 172.21.15.125:0/2138090445 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220006480 con 0x7fc22c0c4850 2023-10-18T10:25:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc21affd640 1 -- 172.21.15.125:0/2138090445 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc2200035f0 con 0x7fc22c0c4850 2023-10-18T10:25:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c1f5450 con 0x7fc22c0c4850 2023-10-18T10:25:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc21affd640 1 -- 172.21.15.125:0/2138090445 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2200039e0 con 0x7fc22c0c4850 2023-10-18T10:25:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.040+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c1f47d0 con 0x7fc22c0c4850 2023-10-18T10:25:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.041+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c0675f0 con 0x7fc22c0c4850 2023-10-18T10:25:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.042+0000 7fc21affd640 1 -- 172.21.15.125:0/2138090445 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc220031030 con 0x7fc22c0c4850 2023-10-18T10:25:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.042+0000 7fc21affd640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc20c06ae40 0x7fc20c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.042+0000 7fc21affd640 1 -- 172.21.15.125:0/2138090445 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc2200aab90 con 0x7fc22c0c4850 2023-10-18T10:25:29.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.042+0000 7fc22a59c640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc20c06ae40 0x7fc20c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.043+0000 7fc22a59c640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc20c06ae40 0x7fc20c06d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc21c000f80 tx=0x7fc21c001580 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:29.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.046+0000 7fc21affd640 1 -- 172.21.15.125:0/2138090445 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc220075a00 con 0x7fc22c0c4850 2023-10-18T10:25:29.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.213+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc22c0b5720 con 0x7fc22c0c4850 2023-10-18T10:25:29.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.213+0000 7fc21affd640 1 -- 172.21.15.125:0/2138090445 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc220045020 con 0x7fc22c0c4850 2023-10-18T10:25:29.215 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:29.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc20c06ae40 msgr2=0x7fc20c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc20c06ae40 0x7fc20c06d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc21c000f80 tx=0x7fc21c001580 comp rx=0 tx=0).stop 2023-10-18T10:25:29.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc22c0c4850 msgr2=0x7fc22c0c6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc22c0c4850 0x7fc22c0c6290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc22002c8c0 tx=0x7fc220001700 comp rx=0 tx=0).stop 2023-10-18T10:25:29.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 shutdown_connections 2023-10-18T10:25:29.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc20c06ae40 0x7fc20c06d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc22c0c67d0 0x7fc22c0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc22c0c4850 0x7fc22c0c6290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 --2- 172.21.15.125:0/2138090445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2140047c0 0x7fc22c157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 >> 172.21.15.125:0/2138090445 conn(0x7fc22c09a030 msgr2=0x7fc22c0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:29.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 shutdown_connections 2023-10-18T10:25:29.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.215+0000 7fc230c4b640 1 -- 172.21.15.125:0/2138090445 wait complete. 2023-10-18T10:25:29.228 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:29.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.334+0000 7f2f50287640 1 -- 172.21.15.125:0/2964766776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f480cd9e0 msgr2=0x7f2f480cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.334+0000 7f2f50287640 1 --2- 172.21.15.125:0/2964766776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f480cd9e0 0x7f2f480cde70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005fd0 tx=0x7f2f4002c2e0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.334+0000 7f2f50287640 1 -- 172.21.15.125:0/2964766776 shutdown_connections 2023-10-18T10:25:29.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.334+0000 7f2f50287640 1 --2- 172.21.15.125:0/2964766776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f480cd9e0 0x7f2f480cde70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.334+0000 7f2f50287640 1 --2- 172.21.15.125:0/2964766776 >> v2:172.21.15.139:3300/0 conn(0x7f2f48158910 0x7f2f48158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.334+0000 7f2f50287640 1 -- 172.21.15.125:0/2964766776 >> 172.21.15.125:0/2964766776 conn(0x7f2f480af7a0 msgr2=0x7f2f480afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:29.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.334+0000 7f2f50287640 1 -- 172.21.15.125:0/2964766776 shutdown_connections 2023-10-18T10:25:29.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.334+0000 7f2f50287640 1 -- 172.21.15.125:0/2964766776 wait complete. 2023-10-18T10:25:29.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.335+0000 7f2f50287640 1 Processor -- start 2023-10-18T10:25:29.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.335+0000 7f2f50287640 1 -- start start 2023-10-18T10:25:29.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f50287640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f480cd1c0 0x7f2f4814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f50287640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f480cd9e0 0x7f2f481e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f50287640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f48158910 0x7f2f481e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f50287640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48153f50 con 0x7f2f480cd1c0 2023-10-18T10:25:29.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f50287640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f481ee0e0 con 0x7f2f480cd9e0 2023-10-18T10:25:29.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f50287640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f481536b0 con 0x7f2f48158910 2023-10-18T10:25:29.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f480cd1c0 0x7f2f4814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f480cd1c0 0x7f2f4814edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48598/0 (socket says 172.21.15.125:48598) 2023-10-18T10:25:29.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 -- 172.21.15.125:0/3604615886 learned_addr learned my addr 172.21.15.125:0/3604615886 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:29.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e824640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f48158910 0x7f2f481e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4d822640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f480cd9e0 0x7f2f481e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 -- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f48158910 msgr2=0x7f2f481e2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f48158910 0x7f2f481e2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 -- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f480cd9e0 msgr2=0x7f2f481e26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f480cd9e0 0x7f2f481e26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 -- 172.21.15.125:0/3604615886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009900 con 0x7f2f480cd1c0 2023-10-18T10:25:29.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e824640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f48158910 0x7f2f481e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:29.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f4e023640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f480cd1c0 0x7f2f4814edd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400e790 tx=0x7f2f4400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:29.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f3effd640 1 -- 172.21.15.125:0/3604615886 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44009580 con 0x7f2f480cd1c0 2023-10-18T10:25:29.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f3effd640 1 -- 172.21.15.125:0/3604615886 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2f4400cdb0 con 0x7f2f480cd1c0 2023-10-18T10:25:29.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f48153990 con 0x7f2f480cd1c0 2023-10-18T10:25:29.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.336+0000 7f2f3effd640 1 -- 172.21.15.125:0/3604615886 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44010690 con 0x7f2f480cd1c0 2023-10-18T10:25:29.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.337+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f481ee330 con 0x7f2f480cd1c0 2023-10-18T10:25:29.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.338+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f480690e0 con 0x7f2f480cd1c0 2023-10-18T10:25:29.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.338+0000 7f2f3effd640 1 -- 172.21.15.125:0/3604615886 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2f440107f0 con 0x7f2f480cd1c0 2023-10-18T10:25:29.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.338+0000 7f2f3effd640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2f2c06ae40 0x7f2f2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.338+0000 7f2f3effd640 1 -- 172.21.15.125:0/3604615886 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2f44014070 con 0x7f2f480cd1c0 2023-10-18T10:25:29.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.338+0000 7f2f4d822640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2f2c06ae40 0x7f2f2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.339+0000 7f2f4d822640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2f2c06ae40 0x7f2f2c06d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f48067a90 tx=0x7f2f38009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:29.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.342+0000 7f2f3effd640 1 -- 172.21.15.125:0/3604615886 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f44051ca0 con 0x7f2f480cd1c0 2023-10-18T10:25:29.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.509+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f480ce5e0 con 0x7f2f480cd1c0 2023-10-18T10:25:29.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.509+0000 7f2f3effd640 1 -- 172.21.15.125:0/3604615886 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f44055140 con 0x7f2f480cd1c0 2023-10-18T10:25:29.512 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:29.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2f2c06ae40 msgr2=0x7f2f2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2f2c06ae40 0x7f2f2c06d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f48067a90 tx=0x7f2f38009040 comp rx=0 tx=0).stop 2023-10-18T10:25:29.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f480cd1c0 msgr2=0x7f2f4814edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f480cd1c0 0x7f2f4814edd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400e790 tx=0x7f2f4400ec50 comp rx=0 tx=0).stop 2023-10-18T10:25:29.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 shutdown_connections 2023-10-18T10:25:29.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2f2c06ae40 0x7f2f2c06d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f480cd9e0 0x7f2f481e26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f48158910 0x7f2f481e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 --2- 172.21.15.125:0/3604615886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f480cd1c0 0x7f2f4814edd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 >> 172.21.15.125:0/3604615886 conn(0x7f2f480af7a0 msgr2=0x7f2f48070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:29.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 shutdown_connections 2023-10-18T10:25:29.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.511+0000 7f2f50287640 1 -- 172.21.15.125:0/3604615886 wait complete. 2023-10-18T10:25:29.524 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:29.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.627+0000 7fe0ea724640 1 -- 172.21.15.125:0/3855122044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 msgr2=0x7fe0e40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.627+0000 7fe0ea724640 1 --2- 172.21.15.125:0/3855122044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 0x7fe0e40705a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009830 tx=0x7fe0d802bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:29.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.627+0000 7fe0ea724640 1 -- 172.21.15.125:0/3855122044 shutdown_connections 2023-10-18T10:25:29.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.627+0000 7fe0ea724640 1 --2- 172.21.15.125:0/3855122044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 0x7fe0e40705a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.627+0000 7fe0ea724640 1 --2- 172.21.15.125:0/3855122044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cbd20 0x7fe0e4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.627+0000 7fe0ea724640 1 -- 172.21.15.125:0/3855122044 >> 172.21.15.125:0/3855122044 conn(0x7fe0e4069930 msgr2=0x7fe0e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:29.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.627+0000 7fe0ea724640 1 -- 172.21.15.125:0/3855122044 shutdown_connections 2023-10-18T10:25:29.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.627+0000 7fe0ea724640 1 -- 172.21.15.125:0/3855122044 wait complete. 2023-10-18T10:25:29.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.628+0000 7fe0ea724640 1 Processor -- start 2023-10-18T10:25:29.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.628+0000 7fe0ea724640 1 -- start start 2023-10-18T10:25:29.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0ea724640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 0x7fe0e413dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0ea724640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40b0530 0x7fe0e40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0ea724640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cbd20 0x7fe0e4148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0ea724640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e413e6a0 con 0x7fe0e40afa20 2023-10-18T10:25:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0ea724640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e413e980 con 0x7fe0e40b0530 2023-10-18T10:25:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0ea724640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e4139760 con 0x7fe0e40cbd20 2023-10-18T10:25:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 0x7fe0e413dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 0x7fe0e413dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48606/0 (socket says 172.21.15.125:48606) 2023-10-18T10:25:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 -- 172.21.15.125:0/146264912 learned_addr learned my addr 172.21.15.125:0/146264912 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e8cc1640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cbd20 0x7fe0e4148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 -- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cbd20 msgr2=0x7fe0e4148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cbd20 0x7fe0e4148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 -- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40b0530 msgr2=0x7fe0e40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40b0530 0x7fe0e40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 -- 172.21.15.125:0/146264912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d8009490 con 0x7fe0e40afa20 2023-10-18T10:25:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e3fff640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 0x7fe0e413dfc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400c970 tx=0x7fe0d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.629+0000 7fe0e0ff9640 1 -- 172.21.15.125:0/146264912 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4007bb0 con 0x7fe0e40afa20 2023-10-18T10:25:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.630+0000 7fe0e0ff9640 1 -- 172.21.15.125:0/146264912 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0d4007e00 con 0x7fe0e40afa20 2023-10-18T10:25:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.630+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e4070cf0 con 0x7fe0e40afa20 2023-10-18T10:25:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.630+0000 7fe0e0ff9640 1 -- 172.21.15.125:0/146264912 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4005280 con 0x7fe0e40afa20 2023-10-18T10:25:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.630+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e413b1c0 con 0x7fe0e40afa20 2023-10-18T10:25:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.631+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e413eba0 con 0x7fe0e40afa20 2023-10-18T10:25:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.631+0000 7fe0e0ff9640 1 -- 172.21.15.125:0/146264912 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe0d40053e0 con 0x7fe0e40afa20 2023-10-18T10:25:29.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.631+0000 7fe0e0ff9640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe0d006ae40 0x7fe0d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.631+0000 7fe0e0ff9640 1 -- 172.21.15.125:0/146264912 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe0d4089d90 con 0x7fe0e40afa20 2023-10-18T10:25:29.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.632+0000 7fe0e37fe640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe0d006ae40 0x7fe0d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.632+0000 7fe0e37fe640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe0d006ae40 0x7fe0d006d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8036000 tx=0x7fe0d80046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:29.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.636+0000 7fe0e0ff9640 1 -- 172.21.15.125:0/146264912 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe0d4054c10 con 0x7fe0e40afa20 2023-10-18T10:25:29.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.802+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe0e40690e0 con 0x7fe0e40afa20 2023-10-18T10:25:29.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.802+0000 7fe0e0ff9640 1 -- 172.21.15.125:0/146264912 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe0d401d070 con 0x7fe0e40afa20 2023-10-18T10:25:29.804 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:29.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe0d006ae40 msgr2=0x7fe0d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe0d006ae40 0x7fe0d006d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8036000 tx=0x7fe0d80046c0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 msgr2=0x7fe0e413dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 0x7fe0e413dfc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400c970 tx=0x7fe0d400ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:29.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 shutdown_connections 2023-10-18T10:25:29.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe0d006ae40 0x7fe0d006d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40b0530 0x7fe0e40caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cbd20 0x7fe0e4148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 --2- 172.21.15.125:0/146264912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40afa20 0x7fe0e413dfc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 >> 172.21.15.125:0/146264912 conn(0x7fe0e4069930 msgr2=0x7fe0e40ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:29.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 shutdown_connections 2023-10-18T10:25:29.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.804+0000 7fe0ea724640 1 -- 172.21.15.125:0/146264912 wait complete. 2023-10-18T10:25:29.816 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:25:29.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.916+0000 7fba6edc7640 1 -- 172.21.15.125:0/1811534319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680ceb40 msgr2=0x7fba680b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.916+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1811534319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680ceb40 0x7fba680b0650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fba60005fd0 tx=0x7fba6002c050 comp rx=0 tx=0).stop 2023-10-18T10:25:29.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.916+0000 7fba6edc7640 1 -- 172.21.15.125:0/1811534319 shutdown_connections 2023-10-18T10:25:29.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.916+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1811534319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680ceb40 0x7fba680b0650 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.916+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1811534319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba6815b2a0 0x7fba680d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.916+0000 7fba6edc7640 1 -- 172.21.15.125:0/1811534319 >> 172.21.15.125:0/1811534319 conn(0x7fba680d0d60 msgr2=0x7fba680d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:29.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.916+0000 7fba6edc7640 1 -- 172.21.15.125:0/1811534319 shutdown_connections 2023-10-18T10:25:29.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.916+0000 7fba6edc7640 1 -- 172.21.15.125:0/1811534319 wait complete. 2023-10-18T10:25:29.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.917+0000 7fba6edc7640 1 Processor -- start 2023-10-18T10:25:29.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.917+0000 7fba6edc7640 1 -- start start 2023-10-18T10:25:29.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba6edc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba680ceb40 0x7fba68154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba6edc7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680cf340 0x7fba681e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba6edc7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba6815b2a0 0x7fba681e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba6edc7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba680ca9c0 con 0x7fba680cf340 2023-10-18T10:25:29.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba6edc7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba680c9db0 con 0x7fba680ceb40 2023-10-18T10:25:29.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba6edc7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba680ca090 con 0x7fba6815b2a0 2023-10-18T10:25:29.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba67fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680cf340 0x7fba681e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba67fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680cf340 0x7fba681e5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48622/0 (socket says 172.21.15.125:48622) 2023-10-18T10:25:29.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba67fff640 1 -- 172.21.15.125:0/1893210060 learned_addr learned my addr 172.21.15.125:0/1893210060 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:29.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba6d364640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba6815b2a0 0x7fba681e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.918+0000 7fba6cb63640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba680ceb40 0x7fba68154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba67fff640 1 -- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba6815b2a0 msgr2=0x7fba681e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba67fff640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba6815b2a0 0x7fba681e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba67fff640 1 -- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba680ceb40 msgr2=0x7fba68154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:29.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba67fff640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba680ceb40 0x7fba68154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:29.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba67fff640 1 -- 172.21.15.125:0/1893210060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba60009840 con 0x7fba680cf340 2023-10-18T10:25:29.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba6d364640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba6815b2a0 0x7fba681e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:29.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba6cb63640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba680ceb40 0x7fba68154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:29.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba67fff640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680cf340 0x7fba681e5880 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00e790 tx=0x7fba5c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:29.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba657fa640 1 -- 172.21.15.125:0/1893210060 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c009580 con 0x7fba680cf340 2023-10-18T10:25:29.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba657fa640 1 -- 172.21.15.125:0/1893210060 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fba5c00cdb0 con 0x7fba680cf340 2023-10-18T10:25:29.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba680b0d50 con 0x7fba680cf340 2023-10-18T10:25:29.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba657fa640 1 -- 172.21.15.125:0/1893210060 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c010690 con 0x7fba680cf340 2023-10-18T10:25:29.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.919+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba680ca650 con 0x7fba680cf340 2023-10-18T10:25:29.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.920+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba680cab70 con 0x7fba680cf340 2023-10-18T10:25:29.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.920+0000 7fba657fa640 1 -- 172.21.15.125:0/1893210060 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fba5c0107f0 con 0x7fba680cf340 2023-10-18T10:25:29.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.920+0000 7fba657fa640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fba5406ae70 0x7fba5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:29.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.921+0000 7fba657fa640 1 -- 172.21.15.125:0/1893210060 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fba5c014070 con 0x7fba680cf340 2023-10-18T10:25:29.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.921+0000 7fba6cb63640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fba5406ae70 0x7fba5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:29.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.921+0000 7fba6cb63640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fba5406ae70 0x7fba5406d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba580046b0 tx=0x7fba58009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:29.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:29.925+0000 7fba657fa640 1 -- 172.21.15.125:0/1893210060 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba5c051ca0 con 0x7fba680cf340 2023-10-18T10:25:30.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.126+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba680690e0 con 0x7fba680cf340 2023-10-18T10:25:30.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.126+0000 7fba657fa640 1 -- 172.21.15.125:0/1893210060 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fba680cab70 con 0x7fba680cf340 2023-10-18T10:25:30.129 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:30.129 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:19.519174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:25:30.129 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 9 2023-10-18T10:25:30.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fba5406ae70 msgr2=0x7fba5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fba5406ae70 0x7fba5406d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba580046b0 tx=0x7fba58009040 comp rx=0 tx=0).stop 2023-10-18T10:25:30.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680cf340 msgr2=0x7fba681e5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680cf340 0x7fba681e5880 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00e790 tx=0x7fba5c00ec50 comp rx=0 tx=0).stop 2023-10-18T10:25:30.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 shutdown_connections 2023-10-18T10:25:30.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fba5406ae70 0x7fba5406d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba6815b2a0 0x7fba681e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba680cf340 0x7fba681e5880 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 --2- 172.21.15.125:0/1893210060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba680ceb40 0x7fba68154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 >> 172.21.15.125:0/1893210060 conn(0x7fba680d0d60 msgr2=0x7fba6815a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:30.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.128+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 shutdown_connections 2023-10-18T10:25:30.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.129+0000 7fba6edc7640 1 -- 172.21.15.125:0/1893210060 wait complete. 2023-10-18T10:25:30.143 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:30.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.244+0000 7f3d09675640 1 -- 172.21.15.125:0/2109765825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b000 msgr2=0x7f3d040d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.244+0000 7f3d09675640 1 --2- 172.21.15.125:0/2109765825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b000 0x7f3d040d1810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3d040b37f0 tx=0x7f3cf402c260 comp rx=0 tx=0).stop 2023-10-18T10:25:30.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.244+0000 7f3d09675640 1 -- 172.21.15.125:0/2109765825 shutdown_connections 2023-10-18T10:25:30.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.244+0000 7f3d09675640 1 --2- 172.21.15.125:0/2109765825 >> v2:172.21.15.139:3300/0 conn(0x7f3d0415b3e0 0x7f3d040cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.244+0000 7f3d09675640 1 --2- 172.21.15.125:0/2109765825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b000 0x7f3d040d1810 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.244+0000 7f3d09675640 1 -- 172.21.15.125:0/2109765825 >> 172.21.15.125:0/2109765825 conn(0x7f3d040d0cf0 msgr2=0x7f3d040d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:30.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.244+0000 7f3d09675640 1 -- 172.21.15.125:0/2109765825 shutdown_connections 2023-10-18T10:25:30.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.244+0000 7f3d09675640 1 -- 172.21.15.125:0/2109765825 wait complete. 2023-10-18T10:25:30.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.245+0000 7f3d09675640 1 Processor -- start 2023-10-18T10:25:30.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.245+0000 7f3d09675640 1 -- start start 2023-10-18T10:25:30.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d09675640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d040afdc0 0x7f3d04159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:30.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d09675640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d0415b000 0x7f3d040c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:30.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d09675640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b3e0 0x7f3d040ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:30.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d09675640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040ca890 con 0x7f3d0415b3e0 2023-10-18T10:25:30.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d09675640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040c5230 con 0x7f3d0415b000 2023-10-18T10:25:30.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d09675640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040c5510 con 0x7f3d040afdc0 2023-10-18T10:25:30.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b3e0 0x7f3d040ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:30.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b3e0 0x7f3d040ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48642/0 (socket says 172.21.15.125:48642) 2023-10-18T10:25:30.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d037fe640 1 -- 172.21.15.125:0/3529467854 learned_addr learned my addr 172.21.15.125:0/3529467854 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:30.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d027fc640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d0415b000 0x7f3d040c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:30.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d02ffd640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d040afdc0 0x7f3d04159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:30.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d037fe640 1 -- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d040afdc0 msgr2=0x7f3d04159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d037fe640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d040afdc0 0x7f3d04159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d037fe640 1 -- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d0415b000 msgr2=0x7f3d040c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d037fe640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d0415b000 0x7f3d040c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.246+0000 7f3d037fe640 1 -- 172.21.15.125:0/3529467854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf4009900 con 0x7f3d0415b3e0 2023-10-18T10:25:30.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.247+0000 7f3d037fe640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b3e0 0x7f3d040ae1f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00c410 tx=0x7f3cfc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:30.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.247+0000 7f3ceb7fe640 1 -- 172.21.15.125:0/3529467854 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc011790 con 0x7f3d0415b3e0 2023-10-18T10:25:30.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.247+0000 7f3ceb7fe640 1 -- 172.21.15.125:0/3529467854 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3cfc0119e0 con 0x7f3d0415b3e0 2023-10-18T10:25:30.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.247+0000 7f3ceb7fe640 1 -- 172.21.15.125:0/3529467854 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc011e30 con 0x7f3d0415b3e0 2023-10-18T10:25:30.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.247+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d040c57f0 con 0x7f3d0415b3e0 2023-10-18T10:25:30.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.247+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d040c5db0 con 0x7f3d0415b3e0 2023-10-18T10:25:30.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.248+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d040690e0 con 0x7f3d0415b3e0 2023-10-18T10:25:30.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.248+0000 7f3ceb7fe640 1 -- 172.21.15.125:0/3529467854 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3cfc01c460 con 0x7f3d0415b3e0 2023-10-18T10:25:30.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.248+0000 7f3ceb7fe640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3cf006ae40 0x7f3cf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:30.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.249+0000 7f3ceb7fe640 1 -- 172.21.15.125:0/3529467854 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3cfc089680 con 0x7f3d0415b3e0 2023-10-18T10:25:30.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.249+0000 7f3d02ffd640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3cf006ae40 0x7f3cf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:30.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.249+0000 7f3d02ffd640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3cf006ae40 0x7f3cf006d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4036000 tx=0x7f3cf4004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:30.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.253+0000 7f3ceb7fe640 1 -- 172.21.15.125:0/3529467854 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3cfc054580 con 0x7f3d0415b3e0 2023-10-18T10:25:30.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.422+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d040d0250 con 0x7f3d0415b3e0 2023-10-18T10:25:30.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.424+0000 7f3ceb7fe640 1 -- 172.21.15.125:0/3529467854 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f3cfc057ba0 con 0x7f3d0415b3e0 2023-10-18T10:25:30.426 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:30.426 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:30.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.425+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3cf006ae40 msgr2=0x7f3cf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3cf006ae40 0x7f3cf006d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4036000 tx=0x7f3cf4004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b3e0 msgr2=0x7f3d040ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b3e0 0x7f3d040ae1f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00c410 tx=0x7f3cfc00c8d0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 shutdown_connections 2023-10-18T10:25:30.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3cf006ae40 0x7f3cf006d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d0415b000 0x7f3d040c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d0415b3e0 0x7f3d040ae1f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 --2- 172.21.15.125:0/3529467854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d040afdc0 0x7f3d04159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 >> 172.21.15.125:0/3529467854 conn(0x7f3d040d0cf0 msgr2=0x7f3d0406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:30.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 shutdown_connections 2023-10-18T10:25:30.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.426+0000 7f3d09675640 1 -- 172.21.15.125:0/3529467854 wait complete. 2023-10-18T10:25:30.438 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:25:30.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.539+0000 7f8f84256640 1 -- 172.21.15.125:0/3954849723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0cbd20 msgr2=0x7f8f7c158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.539+0000 7f8f84256640 1 --2- 172.21.15.125:0/3954849723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0cbd20 0x7f8f7c158300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005fd0 tx=0x7f8f7402c170 comp rx=0 tx=0).stop 2023-10-18T10:25:30.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.539+0000 7f8f84256640 1 -- 172.21.15.125:0/3954849723 shutdown_connections 2023-10-18T10:25:30.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.539+0000 7f8f84256640 1 --2- 172.21.15.125:0/3954849723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0cbd20 0x7f8f7c158300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.539+0000 7f8f84256640 1 -- 172.21.15.125:0/3954849723 >> 172.21.15.125:0/3954849723 conn(0x7f8f7c069930 msgr2=0x7f8f7c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:30.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.539+0000 7f8f84256640 1 -- 172.21.15.125:0/3954849723 shutdown_connections 2023-10-18T10:25:30.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.539+0000 7f8f84256640 1 -- 172.21.15.125:0/3954849723 wait complete. 2023-10-18T10:25:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f84256640 1 Processor -- start 2023-10-18T10:25:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f84256640 1 -- start start 2023-10-18T10:25:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f84256640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0afa20 0x7f8f7c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f84256640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f7c0b0530 0x7f8f7c1405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f84256640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f7c0cbd20 0x7f8f7c13f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f84256640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c140b20 con 0x7f8f7c0afa20 2023-10-18T10:25:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f84256640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c15a5b0 con 0x7f8f7c0cbd20 2023-10-18T10:25:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f84256640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c15a890 con 0x7f8f7c0b0530 2023-10-18T10:25:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f81ff2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0afa20 0x7f8f7c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f81ff2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0afa20 0x7f8f7c159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48674/0 (socket says 172.21.15.125:48674) 2023-10-18T10:25:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.541+0000 7f8f81ff2640 1 -- 172.21.15.125:0/291006225 learned_addr learned my addr 172.21.15.125:0/291006225 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f827f3640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f7c0cbd20 0x7f8f7c13f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f817f1640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f7c0b0530 0x7f8f7c1405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:30.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f81ff2640 1 -- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f7c0b0530 msgr2=0x7f8f7c1405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f81ff2640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f7c0b0530 0x7f8f7c1405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f81ff2640 1 -- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f7c0cbd20 msgr2=0x7f8f7c13f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f81ff2640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f7c0cbd20 0x7f8f7c13f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f81ff2640 1 -- 172.21.15.125:0/291006225 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74009840 con 0x7f8f7c0afa20 2023-10-18T10:25:30.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f827f3640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f7c0cbd20 0x7f8f7c13f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:30.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f817f1640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f7c0b0530 0x7f8f7c1405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:30.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f81ff2640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0afa20 0x7f8f7c159ed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009b30 tx=0x7f8f740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:30.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f727fc640 1 -- 172.21.15.125:0/291006225 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f740047b0 con 0x7f8f7c0afa20 2023-10-18T10:25:30.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f727fc640 1 -- 172.21.15.125:0/291006225 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8f740345a0 con 0x7f8f7c0afa20 2023-10-18T10:25:30.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f727fc640 1 -- 172.21.15.125:0/291006225 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74034990 con 0x7f8f7c0afa20 2023-10-18T10:25:30.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c147f70 con 0x7f8f7c0afa20 2023-10-18T10:25:30.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.542+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c148530 con 0x7f8f7c0afa20 2023-10-18T10:25:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.543+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7c148750 con 0x7f8f7c0afa20 2023-10-18T10:25:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.543+0000 7f8f727fc640 1 -- 172.21.15.125:0/291006225 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8f74034af0 con 0x7f8f7c0afa20 2023-10-18T10:25:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.544+0000 7f8f727fc640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8f6006ae40 0x7f8f6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.544+0000 7f8f727fc640 1 -- 172.21.15.125:0/291006225 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8f740acc20 con 0x7f8f7c0afa20 2023-10-18T10:25:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.544+0000 7f8f817f1640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8f6006ae40 0x7f8f6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.545+0000 7f8f817f1640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8f6006ae40 0x7f8f6006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c156ed0 tx=0x7f8f6c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.548+0000 7f8f727fc640 1 -- 172.21.15.125:0/291006225 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f74077b20 con 0x7f8f7c0afa20 2023-10-18T10:25:30.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.750+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f7c165150 con 0x7f8f7c0afa20 2023-10-18T10:25:30.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.750+0000 7f8f727fc640 1 -- 172.21.15.125:0/291006225 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f8f74043090 con 0x7f8f7c0afa20 2023-10-18T10:25:30.752 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:30.753 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:19.519174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:25:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 9 2023-10-18T10:25:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8f6006ae40 msgr2=0x7f8f6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8f6006ae40 0x7f8f6006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c156ed0 tx=0x7f8f6c009040 comp rx=0 tx=0).stop 2023-10-18T10:25:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0afa20 msgr2=0x7f8f7c159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0afa20 0x7f8f7c159ed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009b30 tx=0x7f8f740042f0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 shutdown_connections 2023-10-18T10:25:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8f6006ae40 0x7f8f6006d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f7c0cbd20 0x7f8f7c13f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f7c0b0530 0x7f8f7c1405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 --2- 172.21.15.125:0/291006225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f7c0afa20 0x7f8f7c159ed0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 >> 172.21.15.125:0/291006225 conn(0x7f8f7c069930 msgr2=0x7f8f7c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 shutdown_connections 2023-10-18T10:25:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:30.752+0000 7f8f84256640 1 -- 172.21.15.125:0/291006225 wait complete. 2023-10-18T10:25:31.167 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:25:03.534893+0000', 'modified': '2023-10-18T10:25:19.519174+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4401}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4401': {'gid': 4401, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6834/4043797924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6832', 'nonce': 4043797924}, {'type': 'v1', 'addr': '172.21.15.125:6834', 'nonce': 4043797924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:25:31.167 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:25:31.167 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:25:31.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.272+0000 7efffefd0640 1 -- 172.21.15.125:0/1210447389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe0004d00 msgr2=0x7effe0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.272+0000 7efffefd0640 1 --2- 172.21.15.125:0/1210447389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe0004d00 0x7effe0005180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efff0009f00 tx=0x7effe00057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.272+0000 7efffd56d640 1 -- 172.21.15.125:0/1210447389 reap_dead start 2023-10-18T10:25:31.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.272+0000 7efffefd0640 1 -- 172.21.15.125:0/1210447389 shutdown_connections 2023-10-18T10:25:31.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.272+0000 7efffefd0640 1 -- 172.21.15.125:0/1210447389 >> 172.21.15.125:0/1210447389 conn(0x7efff80d0d60 msgr2=0x7efff80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:31.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.272+0000 7efffefd0640 1 -- 172.21.15.125:0/1210447389 shutdown_connections 2023-10-18T10:25:31.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.272+0000 7efffefd0640 1 -- 172.21.15.125:0/1210447389 wait complete. 2023-10-18T10:25:31.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.273+0000 7efffefd0640 1 Processor -- start 2023-10-18T10:25:31.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.273+0000 7efffefd0640 1 -- start start 2023-10-18T10:25:31.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffefd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff80b55f0 0x7efff80b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffefd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efff80b4830 0x7efff80b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffefd0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efff80b5f10 0x7efff80c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffefd0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff80b5420 con 0x7efff80b55f0 2023-10-18T10:25:31.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffefd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff81f1160 con 0x7efff80b4830 2023-10-18T10:25:31.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffefd0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff80b4e60 con 0x7efff80b5f10 2023-10-18T10:25:31.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff80b55f0 0x7efff80b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff80b55f0 0x7efff80b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48684/0 (socket says 172.21.15.125:48684) 2023-10-18T10:25:31.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 -- 172.21.15.125:0/3395309310 learned_addr learned my addr 172.21.15.125:0/3395309310 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:31.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efff7fff640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efff80b4830 0x7efff80b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffd56d640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efff80b5f10 0x7efff80c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 -- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efff80b5f10 msgr2=0x7efff80c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efff80b5f10 0x7efff80c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 -- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efff80b4830 msgr2=0x7efff80b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efff80b4830 0x7efff80b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 -- 172.21.15.125:0/3395309310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff80b5140 con 0x7efff80b55f0 2023-10-18T10:25:31.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efff7fff640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efff80b4830 0x7efff80b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:31.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffcd6c640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff80b55f0 0x7efff80b42f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7effe8011a50 tx=0x7effe8011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:31.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efff57fa640 1 -- 172.21.15.125:0/3395309310 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800fd90 con 0x7efff80b55f0 2023-10-18T10:25:31.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efff57fa640 1 -- 172.21.15.125:0/3395309310 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7effe8012450 con 0x7efff80b55f0 2023-10-18T10:25:31.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffefd0640 1 -- 172.21.15.125:0/3395309310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff80c7fd0 con 0x7efff80b55f0 2023-10-18T10:25:31.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efff57fa640 1 -- 172.21.15.125:0/3395309310 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8012800 con 0x7efff80b55f0 2023-10-18T10:25:31.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.274+0000 7efffefd0640 1 -- 172.21.15.125:0/3395309310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff80b68a0 con 0x7efff80b55f0 2023-10-18T10:25:31.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.276+0000 7efff57fa640 1 -- 172.21.15.125:0/3395309310 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7effe8012960 con 0x7efff80b55f0 2023-10-18T10:25:31.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.276+0000 7efff57fa640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7effe406ae40 0x7effe406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.276+0000 7efff57fa640 1 -- 172.21.15.125:0/3395309310 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7effe808a800 con 0x7efff80b55f0 2023-10-18T10:25:31.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.276+0000 7efff7fff640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7effe406ae40 0x7effe406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.277+0000 7efff7fff640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7effe406ae40 0x7effe406d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efff81f4850 tx=0x7effec0067a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:31.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.277+0000 7f0001c33740 1 Processor -- start 2023-10-18T10:25:31.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.277+0000 7f0001c33740 1 -- start start 2023-10-18T10:25:31.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7f0001c33740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3cbfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7f0001c33740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7f0001c33740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cc0020 0x55f4d3cefe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7f0001c33740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3bd1730 con 0x55f4d3cecf20 2023-10-18T10:25:31.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7f0001c33740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3c32780 con 0x55f4d3cecb40 2023-10-18T10:25:31.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7f0001c33740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3c2fe70 con 0x55f4d3cc0020 2023-10-18T10:25:31.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48686/0 (socket says 172.21.15.125:48686) 2023-10-18T10:25:31.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 -- 172.21.15.125:0/901399574 learned_addr learned my addr 172.21.15.125:0/901399574 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:31.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efffcd6c640 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3cbfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efffd56d640 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cc0020 0x55f4d3cefe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 -- 172.21.15.125:0/901399574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cc0020 msgr2=0x55f4d3cefe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:31.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cc0020 0x55f4d3cefe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 -- 172.21.15.125:0/901399574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 msgr2=0x55f4d3cbfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:31.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3cbfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 -- 172.21.15.125:0/901399574 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4d3c303d0 con 0x55f4d3cecf20 2023-10-18T10:25:31.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.278+0000 7efff7fff640 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cef8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7effec006f10 tx=0x7effec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7460bf3125cf16c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:31.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7effde7fc640 1 -- 172.21.15.125:0/901399574 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0129c0 con 0x55f4d3cecf20 2023-10-18T10:25:31.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7effde7fc640 1 -- 172.21.15.125:0/901399574 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7effec016070 con 0x55f4d3cecf20 2023-10-18T10:25:31.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7effde7fc640 1 -- 172.21.15.125:0/901399574 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec012d20 con 0x55f4d3cecf20 2023-10-18T10:25:31.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 -- 172.21.15.125:0/901399574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 msgr2=0x55f4d3cef8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cef8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7effec006f10 tx=0x7effec014000 comp rx=0 tx=0).stop 2023-10-18T10:25:31.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 -- 172.21.15.125:0/901399574 shutdown_connections 2023-10-18T10:25:31.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cc0020 0x55f4d3cefe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cef8d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 --2- 172.21.15.125:0/901399574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3cbfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 -- 172.21.15.125:0/901399574 >> 172.21.15.125:0/901399574 conn(0x55f4d3cc1d50 msgr2=0x55f4d3d0a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:31.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 -- 172.21.15.125:0/901399574 shutdown_connections 2023-10-18T10:25:31.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.279+0000 7f0001c33740 1 -- 172.21.15.125:0/901399574 wait complete. 2023-10-18T10:25:31.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.280+0000 7f0001c33740 1 -- start start 2023-10-18T10:25:31.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.280+0000 7f0001c33740 1 Processor -- start 2023-10-18T10:25:31.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7f0001c33740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3d03c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7f0001c33740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cecf20 0x55f4d3d013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7f0001c33740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3ce0970 0x55f4d3ce0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7f0001c33740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3c2b760 con 0x55f4d3ce0970 2023-10-18T10:25:31.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7f0001c33740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3be68f0 con 0x55f4d3cecb40 2023-10-18T10:25:31.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7f0001c33740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3be6590 con 0x55f4d3cecf20 2023-10-18T10:25:31.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7f0001c33740 10 client.0 _ll_register_callbacks cb 0x55f4d3c22ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:25:31.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3ce0970 0x55f4d3ce0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3ce0970 0x55f4d3ce0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48696/0 (socket says 172.21.15.125:48696) 2023-10-18T10:25:31.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 -- 172.21.15.125:0/2206593251 learned_addr learned my addr 172.21.15.125:0/2206593251 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:31.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efff7fff640 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cecf20 0x55f4d3d013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffcd6c640 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3d03c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 -- 172.21.15.125:0/2206593251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cecf20 msgr2=0x55f4d3d013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:31.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cecf20 0x55f4d3d013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 -- 172.21.15.125:0/2206593251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 msgr2=0x55f4d3d03c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:31.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3d03c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 -- 172.21.15.125:0/2206593251 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff80cbc50 con 0x55f4d3ce0970 2023-10-18T10:25:31.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7efffd56d640 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3ce0970 0x55f4d3ce0df0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efff0009f00 tx=0x7efff00098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39f00f94978ba0f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:31.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7effd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:25:31.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.281+0000 7effd77fe640 1 -- 172.21.15.125:0/2206593251 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0040040 con 0x55f4d3ce0970 2023-10-18T10:25:31.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 1 -- 172.21.15.125:0/2206593251 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4d3c24740 con 0x55f4d3ce0970 2023-10-18T10:25:31.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7effd77fe640 1 -- 172.21.15.125:0/2206593251 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efff0003b70 con 0x55f4d3ce0970 2023-10-18T10:25:31.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7effd77fe640 1 -- 172.21.15.125:0/2206593251 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff0003d30 con 0x55f4d3ce0970 2023-10-18T10:25:31.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 10 client.4983 fetch_fsmap learned FSMap version 9 2023-10-18T10:25:31.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 1 -- 172.21.15.125:0/2206593251 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55f4d3c29de0 con 0x55f4d3ce0970 2023-10-18T10:25:31.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7effd77fe640 1 -- 172.21.15.125:0/2206593251 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55f4d3c24740 con 0x55f4d3ce0970 2023-10-18T10:25:31.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7effd77fe640 1 -- 172.21.15.125:0/2206593251 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7efff0043070 con 0x55f4d3ce0970 2023-10-18T10:25:31.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 10 client.4983 fetch_fsmap finished waiting for FSMap version 9 2023-10-18T10:25:31.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 10 client.4983 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:25:31.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 20 client.4983 populate_metadata read hostname 'smithi125' 2023-10-18T10:25:31.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x55f4d3cf1de0 0x55f4d3cf21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 4 client.4983 mds_command: new command op to 4401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:25:31.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7f0001c33740 1 -- 172.21.15.125:0/2206593251 --> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4d3c24460 con 0x55f4d3cf1de0 2023-10-18T10:25:31.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7efffcd6c640 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x55f4d3cf1de0 0x55f4d3cf21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7effd77fe640 10 client.4983 ms_handle_connect on v2:172.21.15.125:6832/4043797924 2023-10-18T10:25:31.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.282+0000 7efffcd6c640 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x55f4d3cf1de0 0x55f4d3cf21c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf9ca7cd2d53dc1 server_cookie=20eaba1ba1168383 in_seq=0 out_seq=0 2023-10-18T10:25:31.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7effd77fe640 1 -- 172.21.15.125:0/2206593251 <== mds.0 v2:172.21.15.125:6832/4043797924 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f4d3c24460 con 0x55f4d3cf1de0 2023-10-18T10:25:31.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7effd77fe640 10 client.4983 handle_command_reply: tid=0 2023-10-18T10:25:31.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 client.4983 shutdown 2023-10-18T10:25:31.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 -- 172.21.15.125:0/2206593251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3ce0970 msgr2=0x55f4d3ce0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3ce0970 0x55f4d3ce0df0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efff0009f00 tx=0x7efff00098e0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 -- 172.21.15.125:0/2206593251 shutdown_connections 2023-10-18T10:25:31.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3ce0970 0x55f4d3ce0df0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3cecf20 0x55f4d3d013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x55f4d3cf1de0 0x55f4d3cf21c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 --2- 172.21.15.125:0/2206593251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3d03c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 -- 172.21.15.125:0/2206593251 >> 172.21.15.125:0/2206593251 conn(0x55f4d3cc0020 msgr2=0x55f4d3d0dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:31.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7effd77fe640 0 client.4983 ms_handle_reset on v2:172.21.15.125:6832/4043797924 2023-10-18T10:25:31.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 -- 172.21.15.125:0/2206593251 shutdown_connections 2023-10-18T10:25:31.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 1 -- 172.21.15.125:0/2206593251 wait complete. 2023-10-18T10:25:31.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.283+0000 7f0001c33740 20 client.4983 trim_cache size 0 max 16384 2023-10-18T10:25:31.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.298+0000 7f0001c33740 1 Processor -- start 2023-10-18T10:25:31.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.298+0000 7f0001c33740 1 -- start start 2023-10-18T10:25:31.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7f0001c33740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d0afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7f0001c33740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecb40 0x55f4d3d0d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7f0001c33740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7f0001c33740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3bac270 con 0x55f4d3cecb40 2023-10-18T10:25:31.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7f0001c33740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3d1a960 con 0x55f4d3cecf20 2023-10-18T10:25:31.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7f0001c33740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3d0db60 con 0x55f4d3ce0970 2023-10-18T10:25:31.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecb40 0x55f4d3d0d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecb40 0x55f4d3d0d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48710/0 (socket says 172.21.15.125:48710) 2023-10-18T10:25:31.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 -- 172.21.15.125:0/3954824991 learned_addr learned my addr 172.21.15.125:0/3954824991 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:31.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efffcd6c640 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d0afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efffd56d640 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 -- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 msgr2=0x55f4d3d0afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:31.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d0afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 -- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecf20 msgr2=0x55f4d3cbb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:31.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 -- 172.21.15.125:0/3954824991 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4d3bc8d00 con 0x55f4d3cecb40 2023-10-18T10:25:31.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efffd56d640 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:31.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7efff7fff640 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecb40 0x55f4d3d0d620 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7effec000fb0 tx=0x7effec000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125d1a1473425bea server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:31.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.299+0000 7effdeffd640 1 -- 172.21.15.125:0/3954824991 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec02d040 con 0x55f4d3cecb40 2023-10-18T10:25:31.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7effdeffd640 1 -- 172.21.15.125:0/3954824991 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7effec012ce0 con 0x55f4d3cecb40 2023-10-18T10:25:31.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7effdeffd640 1 -- 172.21.15.125:0/3954824991 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec032690 con 0x55f4d3cecb40 2023-10-18T10:25:31.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 -- 172.21.15.125:0/3954824991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecb40 msgr2=0x55f4d3d0d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecb40 0x55f4d3d0d620 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7effec000fb0 tx=0x7effec000fe0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 -- 172.21.15.125:0/3954824991 shutdown_connections 2023-10-18T10:25:31.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecb40 0x55f4d3d0d620 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 --2- 172.21.15.125:0/3954824991 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 -- 172.21.15.125:0/3954824991 >> 172.21.15.125:0/3954824991 conn(0x55f4d3cc0020 msgr2=0x55f4d3cef8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:31.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 -- 172.21.15.125:0/3954824991 shutdown_connections 2023-10-18T10:25:31.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.300+0000 7f0001c33740 1 -- 172.21.15.125:0/3954824991 wait complete. 2023-10-18T10:25:31.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.301+0000 7f0001c33740 1 -- start start 2023-10-18T10:25:31.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.301+0000 7f0001c33740 1 Processor -- start 2023-10-18T10:25:31.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.301+0000 7f0001c33740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d23ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3d043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3d245b0 con 0x55f4d3cecf20 2023-10-18T10:25:31.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3d0d9b0 con 0x55f4d3cecb40 2023-10-18T10:25:31.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4d3d728d0 con 0x55f4d3ce0970 2023-10-18T10:25:31.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 10 client.0 _ll_register_callbacks cb 0x7effec006630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:25:31.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffd56d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffcd6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d23ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffcd6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d23ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:55522/0 (socket says 172.21.15.125:55522) 2023-10-18T10:25:31.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffd56d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48722/0 (socket says 172.21.15.125:48722) 2023-10-18T10:25:31.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffcd6c640 1 -- 172.21.15.125:0/1096226309 learned_addr learned my addr 172.21.15.125:0/1096226309 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:31.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efff7fff640 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3d043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffd56d640 1 -- 172.21.15.125:0/1096226309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 msgr2=0x55f4d3d23ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:31.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffd56d640 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d23ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffd56d640 1 -- 172.21.15.125:0/1096226309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 msgr2=0x55f4d3d043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:25:31.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffd56d640 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3d043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffd56d640 1 -- 172.21.15.125:0/1096226309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4d3c303d0 con 0x55f4d3cecf20 2023-10-18T10:25:31.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffcd6c640 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d23ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:25:31.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7efffd56d640 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efff0009d20 tx=0x7efff0005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22560bba74e03848 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:31.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7effd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:25:31.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7effd77fe640 1 -- 172.21.15.125:0/1096226309 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0004cb0 con 0x55f4d3cecf20 2023-10-18T10:25:31.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7effd77fe640 1 -- 172.21.15.125:0/1096226309 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efff0004e10 con 0x55f4d3cecf20 2023-10-18T10:25:31.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 1 -- 172.21.15.125:0/1096226309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effec016070 con 0x55f4d3cecf20 2023-10-18T10:25:31.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7effd77fe640 1 -- 172.21.15.125:0/1096226309 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff0003b80 con 0x55f4d3cecf20 2023-10-18T10:25:31.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 10 client.4989 fetch_fsmap learned FSMap version 9 2023-10-18T10:25:31.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.302+0000 7f0001c33740 1 -- 172.21.15.125:0/1096226309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55f4d3bee4f0 con 0x55f4d3cecf20 2023-10-18T10:25:31.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7effd77fe640 1 -- 172.21.15.125:0/1096226309 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7effec016070 con 0x55f4d3cecf20 2023-10-18T10:25:31.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7effd77fe640 1 -- 172.21.15.125:0/1096226309 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7efff0043040 con 0x55f4d3cecf20 2023-10-18T10:25:31.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 10 client.4989 fetch_fsmap finished waiting for FSMap version 9 2023-10-18T10:25:31.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 10 client.4989 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:25:31.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 20 client.4989 populate_metadata read hostname 'smithi125' 2023-10-18T10:25:31.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x55f4d3cefe60 0x55f4d3cbb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 4 client.4989 mds_command: new command op to 4401 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:25:31.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 -- 172.21.15.125:0/1096226309 --> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f4d392d6c0 con 0x55f4d3cefe60 2023-10-18T10:25:31.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7efffcd6c640 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x55f4d3cefe60 0x55f4d3cbb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7effd77fe640 10 client.4989 ms_handle_connect on v2:172.21.15.125:6832/4043797924 2023-10-18T10:25:31.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7efffcd6c640 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x55f4d3cefe60 0x55f4d3cbb740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=658124e3ed370645 server_cookie=2f41afc5c5efd502 in_seq=0 out_seq=0 2023-10-18T10:25:31.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7effd77fe640 1 -- 172.21.15.125:0/1096226309 <== mds.0 v2:172.21.15.125:6832/4043797924 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55f4d392d6c0 con 0x55f4d3cefe60 2023-10-18T10:25:31.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7effd77fe640 10 client.4989 handle_command_reply: tid=0 2023-10-18T10:25:31.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 client.4989 shutdown 2023-10-18T10:25:31.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 -- 172.21.15.125:0/1096226309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 msgr2=0x55f4d3cbb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efff0009d20 tx=0x7efff0005f50 comp rx=0 tx=0).stop 2023-10-18T10:25:31.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 -- 172.21.15.125:0/1096226309 shutdown_connections 2023-10-18T10:25:31.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.125:6832/4043797924,v1:172.21.15.125:6834/4043797924] conn(0x55f4d3cefe60 0x55f4d3cbb740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4d3cecb40 0x55f4d3d043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4d3cecf20 0x55f4d3cbb200 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 --2- 172.21.15.125:0/1096226309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f4d3ce0970 0x55f4d3d23ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7effd77fe640 0 client.4989 ms_handle_reset on v2:172.21.15.125:6832/4043797924 2023-10-18T10:25:31.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.303+0000 7f0001c33740 1 -- 172.21.15.125:0/1096226309 >> 172.21.15.125:0/1096226309 conn(0x55f4d3cc0020 msgr2=0x55f4d3cdfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:31.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7f0001c33740 1 -- 172.21.15.125:0/1096226309 shutdown_connections 2023-10-18T10:25:31.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7f0001c33740 1 -- 172.21.15.125:0/1096226309 wait complete. 2023-10-18T10:25:31.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7f0001c33740 20 client.4989 trim_cache size 0 max 16384 2023-10-18T10:25:31.327 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:25:31.327 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:25:31.327 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:25:31.328 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 4401, 2023-10-18T10:25:31.328 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:25:31.328 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:25:31.328 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:25:31.328 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 27.082954658999999, 2023-10-18T10:25:31.329 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 9, 2023-10-18T10:25:31.329 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 25, 2023-10-18T10:25:31.329 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 25, 2023-10-18T10:25:31.329 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 34.193897372999999 2023-10-18T10:25:31.329 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:25:31.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 -- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7effe406ae40 msgr2=0x7effe406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7effe406ae40 0x7effe406d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efff81f4850 tx=0x7effec0067a0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 -- 172.21.15.125:0/3395309310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff80b55f0 msgr2=0x7efff80b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff80b55f0 0x7efff80b42f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7effe8011a50 tx=0x7effe8011f10 comp rx=0 tx=0).stop 2023-10-18T10:25:31.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 -- 172.21.15.125:0/3395309310 shutdown_connections 2023-10-18T10:25:31.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7effe406ae40 0x7effe406d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efff80b4830 0x7efff80b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efff80b5f10 0x7efff80c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 --2- 172.21.15.125:0/3395309310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efff80b55f0 0x7efff80b42f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 -- 172.21.15.125:0/3395309310 >> 172.21.15.125:0/3395309310 conn(0x7efff80d0d60 msgr2=0x7efff80b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:31.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 -- 172.21.15.125:0/3395309310 shutdown_connections 2023-10-18T10:25:31.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.304+0000 7efffefd0640 1 -- 172.21.15.125:0/3395309310 wait complete. 2023-10-18T10:25:31.333 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-18T10:25:31.344 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-18T10:25:31.344 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-18T10:25:31.345 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-18T10:25:31.345 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:25:31.345 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:31.345 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-18T10:25:31.346 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:25:31.346 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-18T10:25:31.346 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-18T10:25:31.346 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:25:31.347 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:31.347 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-18T10:25:31.347 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T10:25:31.347 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-18T10:25:31.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:31.348 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:25:31.363 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-18T10:25:31.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:31.364 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T10:25:31.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:31.430 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link delete ceph-brx 2023-10-18T10:25:31.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:31.533 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2023-10-18T10:25:31.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:31.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2023-10-18T10:25:31.625 INFO:teuthology.orchestra.run.smithi139.stderr:Cannot find device "ceph-brx" 2023-10-18T10:25:31.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:25:31.627 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-18T10:25:31.627 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:25:31.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.730+0000 7f57f421a640 1 -- 172.21.15.125:0/4095966750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec13e520 msgr2=0x7f57ec06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.730+0000 7f57f421a640 1 --2- 172.21.15.125:0/4095966750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec13e520 0x7f57ec06bb60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005fd0 tx=0x7f57e802bfc0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.730+0000 7f57f421a640 1 -- 172.21.15.125:0/4095966750 shutdown_connections 2023-10-18T10:25:31.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.730+0000 7f57f421a640 1 --2- 172.21.15.125:0/4095966750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57ec141690 0x7f57ec0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.730+0000 7f57f421a640 1 --2- 172.21.15.125:0/4095966750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec13e520 0x7f57ec06bb60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.730+0000 7f57f421a640 1 -- 172.21.15.125:0/4095966750 >> 172.21.15.125:0/4095966750 conn(0x7f57ec069930 msgr2=0x7f57ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:31.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.730+0000 7f57f421a640 1 -- 172.21.15.125:0/4095966750 shutdown_connections 2023-10-18T10:25:31.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.730+0000 7f57f421a640 1 -- 172.21.15.125:0/4095966750 wait complete. 2023-10-18T10:25:31.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.731+0000 7f57f421a640 1 Processor -- start 2023-10-18T10:25:31.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.731+0000 7f57f421a640 1 -- start start 2023-10-18T10:25:31.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f421a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec06c0a0 0x7f57ec0ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f421a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57ec13e520 0x7f57ec1d4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f421a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57ec141690 0x7f57ec1d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f421a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0a8f40 con 0x7f57ec06c0a0 2023-10-18T10:25:31.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f421a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0a7c60 con 0x7f57ec141690 2023-10-18T10:25:31.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f421a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0a88f0 con 0x7f57ec13e520 2023-10-18T10:25:31.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec06c0a0 0x7f57ec0ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec06c0a0 0x7f57ec0ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48730/0 (socket says 172.21.15.125:48730) 2023-10-18T10:25:31.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 -- 172.21.15.125:0/2858943835 learned_addr learned my addr 172.21.15.125:0/2858943835 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:31.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f17b5640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57ec13e520 0x7f57ec1d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 -- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57ec13e520 msgr2=0x7f57ec1d4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f27b7640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57ec141690 0x7f57ec1d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57ec13e520 0x7f57ec1d4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 -- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57ec141690 msgr2=0x7f57ec1d45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57ec141690 0x7f57ec1d45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 -- 172.21.15.125:0/2858943835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8009840 con 0x7f57ec06c0a0 2023-10-18T10:25:31.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f27b7640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57ec141690 0x7f57ec1d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:31.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f17b5640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57ec13e520 0x7f57ec1d4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:31.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57f1fb6640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec06c0a0 0x7f57ec0ab400 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005fd0 tx=0x7f57e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:31.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.732+0000 7f57e27fc640 1 -- 172.21.15.125:0/2858943835 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8045440 con 0x7f57ec06c0a0 2023-10-18T10:25:31.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.733+0000 7f57e27fc640 1 -- 172.21.15.125:0/2858943835 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57e80455a0 con 0x7f57ec06c0a0 2023-10-18T10:25:31.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.733+0000 7f57e27fc640 1 -- 172.21.15.125:0/2858943835 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8045990 con 0x7f57ec06c0a0 2023-10-18T10:25:31.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.733+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec0a8bd0 con 0x7f57ec06c0a0 2023-10-18T10:25:31.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.733+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec0abb20 con 0x7f57ec06c0a0 2023-10-18T10:25:31.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.733+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec06d360 con 0x7f57ec06c0a0 2023-10-18T10:25:31.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.734+0000 7f57e27fc640 1 -- 172.21.15.125:0/2858943835 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f57e804e440 con 0x7f57ec06c0a0 2023-10-18T10:25:31.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.734+0000 7f57e27fc640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f57d006ae40 0x7f57d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:31.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.734+0000 7f57e27fc640 1 -- 172.21.15.125:0/2858943835 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f57e80ae380 con 0x7f57ec06c0a0 2023-10-18T10:25:31.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.734+0000 7f57f17b5640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f57d006ae40 0x7f57d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:31.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.735+0000 7f57f17b5640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f57d006ae40 0x7f57d006d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ae620 tx=0x7f57dc005db0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:31.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.738+0000 7f57e27fc640 1 -- 172.21.15.125:0/2858943835 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57e80791f0 con 0x7f57ec06c0a0 2023-10-18T10:25:31.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.940+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57ec0abb20 con 0x7f57ec06c0a0 2023-10-18T10:25:31.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.940+0000 7f57e27fc640 1 -- 172.21.15.125:0/2858943835 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f57ec0abb20 con 0x7f57ec06c0a0 2023-10-18T10:25:31.942 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:25:31.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f57d006ae40 msgr2=0x7f57d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f57d006ae40 0x7f57d006d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ae620 tx=0x7f57dc005db0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec06c0a0 msgr2=0x7f57ec0ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:31.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec06c0a0 0x7f57ec0ab400 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005fd0 tx=0x7f57e80042f0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 shutdown_connections 2023-10-18T10:25:31.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f57d006ae40 0x7f57d006d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57ec141690 0x7f57ec1d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57ec13e520 0x7f57ec1d4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 --2- 172.21.15.125:0/2858943835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57ec06c0a0 0x7f57ec0ab400 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:31.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 >> 172.21.15.125:0/2858943835 conn(0x7f57ec069930 msgr2=0x7f57ec0c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:31.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 shutdown_connections 2023-10-18T10:25:31.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:31.942+0000 7f57f421a640 1 -- 172.21.15.125:0/2858943835 wait complete. 2023-10-18T10:25:31.954 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:25:31.954 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:25:31.955 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:25:31.955 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:25:31.955 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:31.955 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:25:31.956 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:25:31.956 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:25:31.956 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:25:31.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:31.957 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:25:31.971 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:31.972 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:31.972 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:31.972 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:31.972 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:31.972 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:31.972 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:31.973 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:31.973 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:25:31.973 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:31.973 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:31.973 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:25:31.973 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:31.974 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:31.974 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:25:31.974 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:31.974 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:25:31.974 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 42692sec preferred_lft 42692sec 2023-10-18T10:25:31.975 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:25:31.975 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:31.975 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T10:25:31.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:31.976 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:31.976 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:31.976 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link add name ceph-brx type bridge 2023-10-18T10:25:31.976 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip addr flush dev ceph-brx 2023-10-18T10:25:31.977 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set ceph-brx up 2023-10-18T10:25:31.977 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T10:25:31.977 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:32.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.079 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:25:32.084 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:25:32.104 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:25:32.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.105 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:25:32.106 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:25:32.121 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:25:32.121 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:25:32.122 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:25:32.122 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:25:32.122 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:25:32.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.123 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:32.123 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:32.123 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:25:32.124 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:25:32.124 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:25:32.124 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:32.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.216 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:25:32.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.231 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list-id 2023-10-18T10:25:32.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.285 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:32.286 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:32.286 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:32.286 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-18T10:25:32.286 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:32.367 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-18T10:25:32.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.368 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:32.368 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:32.368 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-18T10:25:32.369 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T10:25:32.369 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-18T10:25:32.369 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-18T10:25:32.370 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-18T10:25:32.370 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:32.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.492 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:32.492 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:32.492 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:25:32.493 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T10:25:32.493 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:32.534 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:25:32.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:32.534 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:32.589 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:25:32.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:32.590 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:32.644 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:25:32.710 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:32.768 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:25:32.768 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:25:32.769 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:25:32.769 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:25:32.769 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:25:32.769 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:25:32.769 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:25:32.770 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:25:32.770 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:25:32.770 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:25:32.770 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:25:32.770 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:25:32.771 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:32.771 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:25:32.771 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:25:32.771 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:25:32.771 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:25:32.772 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:32.772 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:32.772 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:32.772 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:25:32.772 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:32.772 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:32.773 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:25:32.773 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:25:32.773 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:25:32.773 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:25:32.773 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:25:32.773 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:25:32.774 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:25:32.774 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:32.774 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:32.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.775 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:25:32.822 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:25:32.823 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:25:32.865 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:25:32.891 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:32.947 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:25:32.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:25:32.947 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:25:32.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:25:32.948 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:25:32.948 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:25:32.948 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:25:32.948 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:25:32.948 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:25:32.949 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:25:32.949 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:25:32.949 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:25:32.949 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:32.949 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:25:32.950 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:25:32.950 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:25:32.950 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:25:32.950 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:32.950 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:32.951 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:32.951 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:25:32.951 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:32.951 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:32.951 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:25:32.951 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:25:32.952 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:25:32.952 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:25:32.952 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:25:32.952 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:25:32.952 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:25:32.953 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:32.953 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:32.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:32.953 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:25:32.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:25:32.947+0000 7f69b463b480 -1 init, newargv = 0x5620c472eaf0 newargc=15 2023-10-18T10:25:32.955 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[99610]: starting ceph client 2023-10-18T10:25:32.955 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[99610]: starting fuse 2023-10-18T10:25:32.968 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:25:32.968 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:25:32.969 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:25:32.969 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:25:32.969 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:25:32.970 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:25:32.970 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:25:32.970 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:32.970 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:25:32.971 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:25:32.971 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:25:32.971 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:25:32.972 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:32.972 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:25:32.972 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:25:32.972 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:25:32.973 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:25:32.973 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:32.973 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:25:32.973 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:25:32.974 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:25:32.974 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:25:32.974 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:25:32.975 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:25:32.975 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:25:32.975 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:25:32.975 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:32.976 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:25:32.976 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:25:33.086 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.99610.asok 2023-10-18T10:25:33.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:25:33.097 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.99610.asok 2023-10-18T10:25:33.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.097 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.99610.asok status 2023-10-18T10:25:33.232 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:25:33.233 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:25:33.233 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:25:33.233 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:25:33.233 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:25:33.234 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:25:33.234 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:25:33.234 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "99610", 2023-10-18T10:25:33.234 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:25:33.234 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:33.234 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:25:33.235 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:25:33.235 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 5007, 2023-10-18T10:25:33.235 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:25:33.235 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:25:33.235 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:25:33.236 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 5007 2023-10-18T10:25:33.236 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:33.236 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:25:33.236 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:25:33.236 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:25:33.236 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 2802659462 2023-10-18T10:25:33.237 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:25:33.237 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:33.237 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:25:33.237 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:25:33.237 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:25:33.237 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 2802659462 2023-10-18T10:25:33.238 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:33.238 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.5007 192.168.0.1:0/2802659462", 2023-10-18T10:25:33.238 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/2802659462", 2023-10-18T10:25:33.238 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:25:33.238 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 9, 2023-10-18T10:25:33.239 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 25, 2023-10-18T10:25:33.239 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:25:33.239 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:25:33.239 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:25:33.239 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:25:33.241 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:25:33.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 -- 172.21.15.125:0/719873620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 msgr2=0x7fd5f006b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:33.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 --2- 172.21.15.125:0/719873620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 0x7fd5f006b410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00cb9b0 tx=0x7fd5e002bd10 comp rx=0 tx=0).stop 2023-10-18T10:25:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 -- 172.21.15.125:0/719873620 shutdown_connections 2023-10-18T10:25:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 --2- 172.21.15.125:0/719873620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5f015ccb0 0x7fd5f00c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 --2- 172.21.15.125:0/719873620 >> v2:172.21.15.139:3300/0 conn(0x7fd5f0159bd0 0x7fd5f015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 --2- 172.21.15.125:0/719873620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 0x7fd5f006b410 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 -- 172.21.15.125:0/719873620 >> 172.21.15.125:0/719873620 conn(0x7fd5f0069930 msgr2=0x7fd5f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:33.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 -- 172.21.15.125:0/719873620 shutdown_connections 2023-10-18T10:25:33.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.340+0000 7fd5f727c640 1 -- 172.21.15.125:0/719873620 wait complete. 2023-10-18T10:25:33.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.341+0000 7fd5f727c640 1 Processor -- start 2023-10-18T10:25:33.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f727c640 1 -- start start 2023-10-18T10:25:33.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f727c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 0x7fd5f0152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:33.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f727c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5f0159bd0 0x7fd5f00ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:33.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f727c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5f015ccb0 0x7fd5f00a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:33.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f727c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00aab90 con 0x7fd5f006b030 2023-10-18T10:25:33.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f727c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00a9f80 con 0x7fd5f015ccb0 2023-10-18T10:25:33.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f727c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00aa260 con 0x7fd5f0159bd0 2023-10-18T10:25:33.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f5018640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 0x7fd5f0152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:33.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f5018640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 0x7fd5f0152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48738/0 (socket says 172.21.15.125:48738) 2023-10-18T10:25:33.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f5018640 1 -- 172.21.15.125:0/2150340675 learned_addr learned my addr 172.21.15.125:0/2150340675 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:33.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f4817640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5f0159bd0 0x7fd5f00ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:33.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.342+0000 7fd5f5819640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5f015ccb0 0x7fd5f00a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:33.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5f5018640 1 -- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5f0159bd0 msgr2=0x7fd5f00ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:33.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5f5018640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5f0159bd0 0x7fd5f00ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5f5018640 1 -- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5f015ccb0 msgr2=0x7fd5f00a6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:33.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5f5018640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5f015ccb0 0x7fd5f00a6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5f5018640 1 -- 172.21.15.125:0/2150340675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0009560 con 0x7fd5f006b030 2023-10-18T10:25:33.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5f5018640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 0x7fd5f0152b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0036040 tx=0x7fd5e002dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:33.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5e5ffb640 1 -- 172.21.15.125:0/2150340675 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0005d50 con 0x7fd5f006b030 2023-10-18T10:25:33.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5e5ffb640 1 -- 172.21.15.125:0/2150340675 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd5e0003910 con 0x7fd5f006b030 2023-10-18T10:25:33.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f00aa540 con 0x7fd5f006b030 2023-10-18T10:25:33.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5e5ffb640 1 -- 172.21.15.125:0/2150340675 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0003d20 con 0x7fd5f006b030 2023-10-18T10:25:33.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.343+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f0153090 con 0x7fd5f006b030 2023-10-18T10:25:33.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.344+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f00c9a30 con 0x7fd5f006b030 2023-10-18T10:25:33.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.344+0000 7fd5e5ffb640 1 -- 172.21.15.125:0/2150340675 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd5e003e450 con 0x7fd5f006b030 2023-10-18T10:25:33.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.345+0000 7fd5e5ffb640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd5d406ae40 0x7fd5d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:33.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.345+0000 7fd5e5ffb640 1 -- 172.21.15.125:0/2150340675 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd5e00ac310 con 0x7fd5f006b030 2023-10-18T10:25:33.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.345+0000 7fd5f4817640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd5d406ae40 0x7fd5d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:33.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.345+0000 7fd5f4817640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd5d406ae40 0x7fd5d406d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec005d80 tx=0x7fd5ec005c50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:33.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.349+0000 7fd5e5ffb640 1 -- 172.21.15.125:0/2150340675 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5e00771f0 con 0x7fd5f006b030 2023-10-18T10:25:33.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.552+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd5f00690e0 con 0x7fd5f006b030 2023-10-18T10:25:33.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.553+0000 7fd5e5ffb640 1 -- 172.21.15.125:0/2150340675 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd5e007a810 con 0x7fd5f006b030 2023-10-18T10:25:33.555 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:25:33.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd5d406ae40 msgr2=0x7fd5d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:33.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd5d406ae40 0x7fd5d406d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec005d80 tx=0x7fd5ec005c50 comp rx=0 tx=0).stop 2023-10-18T10:25:33.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 msgr2=0x7fd5f0152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:33.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 0x7fd5f0152b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0036040 tx=0x7fd5e002dce0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 shutdown_connections 2023-10-18T10:25:33.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd5d406ae40 0x7fd5d406d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5f015ccb0 0x7fd5f00a6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5f0159bd0 0x7fd5f00ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 --2- 172.21.15.125:0/2150340675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f006b030 0x7fd5f0152b50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:33.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.554+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 >> 172.21.15.125:0/2150340675 conn(0x7fd5f0069930 msgr2=0x7fd5f0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:33.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.555+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 shutdown_connections 2023-10-18T10:25:33.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:33.555+0000 7fd5f727c640 1 -- 172.21.15.125:0/2150340675 wait complete. 2023-10-18T10:25:33.566 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:25:33.567 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:25:33.567 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:25:33.567 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-18T10:25:33.568 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-18T10:25:33.568 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:25:33.568 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:25:33.568 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:25:33.568 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-18T10:25:33.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.569 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:25:33.584 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:33.584 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:33.585 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:33.585 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:33.585 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:33.585 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:33.585 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:33.586 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:33.586 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:25:33.586 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:33.586 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:33.586 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:25:33.587 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:33.587 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:33.587 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:25:33.587 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:33.587 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:25:33.588 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 42690sec preferred_lft 42690sec 2023-10-18T10:25:33.588 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:25:33.588 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:33.588 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:25:33.588 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:33.589 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:25:33.589 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:33.589 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link tentative 2023-10-18T10:25:33.589 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:33.589 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:25:33.590 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:33.590 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link tentative 2023-10-18T10:25:33.590 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:33.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.591 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:25:33.650 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:25:33.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.651 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:25:33.707 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:25:33.708 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:25:33.708 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:25:33.708 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:25:33.708 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:25:33.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.709 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:33.709 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:33.710 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:25:33.710 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:25:33.710 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:25:33.711 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:33.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.803 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:25:33.817 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:25:33.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.818 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list-id 2023-10-18T10:25:33.872 INFO:teuthology.orchestra.run.smithi125.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-18T10:25:33.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.873 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:33.873 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:33.873 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:25:33.874 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-18T10:25:33.874 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:33.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:33.954 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-18T10:25:34.020 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:34.020 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:34.021 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:34.021 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:34.021 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:34.021 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:34.021 INFO:teuthology.orchestra.run.smithi125.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:25:34.022 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 72:19:f1:1a:47:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T10:25:34.022 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T10:25:34.022 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:34.022 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::7019:f1ff:fe1a:4778/64 scope link tentative 2023-10-18T10:25:34.022 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:34.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:34.024 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-18T10:25:34.090 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:34.090 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:34.090 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:34.090 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:34.091 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:34.091 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:34.091 INFO:teuthology.orchestra.run.smithi125.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:25:34.091 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 72:19:f1:1a:47:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T10:25:34.091 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T10:25:34.092 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:34.092 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::7019:f1ff:fe1a:4778/64 scope link tentative 2023-10-18T10:25:34.092 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:34.093 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-18T10:25:34.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:34.094 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:34.094 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:34.095 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-18T10:25:34.095 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-18T10:25:34.095 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-18T10:25:34.095 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-18T10:25:34.096 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-18T10:25:34.096 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:34.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:34.217 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:25:34.218 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:25:34.218 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.1 up 2023-10-18T10:25:34.218 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set dev brx.1 master ceph-brx 2023-10-18T10:25:34.219 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:25:34.260 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-18T10:25:34.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:34.261 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-18T10:25:34.315 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-18T10:25:34.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:34.316 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-18T10:25:34.371 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:25:34.446 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:34.503 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:25:34.503 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:25:34.503 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:25:34.503 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:25:34.504 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:25:34.504 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:25:34.504 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:25:34.504 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:25:34.504 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:25:34.505 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:25:34.505 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:25:34.505 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:25:34.505 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:34.505 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:25:34.505 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:25:34.506 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:25:34.506 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:25:34.506 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:34.506 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:34.506 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:34.507 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:25:34.507 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:34.507 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:34.507 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:25:34.507 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:25:34.507 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:25:34.508 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:25:34.508 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:25:34.508 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:25:34.508 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:25:34.508 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:34.509 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:34.509 INFO:teuthology.orchestra.run.smithi125.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:34.509 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:25:34.509 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:25:34.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:34.510 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:25:34.559 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:25:34.560 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-10-18T10:25:34.560 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-18T10:25:34.603 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:25:34.629 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:34.685 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:25:34.686 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:25:34.686 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:25:34.686 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:25:34.686 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:25:34.687 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:25:34.687 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:25:34.687 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:25:34.687 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:25:34.687 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:25:34.688 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:25:34.688 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:25:34.688 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:34.688 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:25:34.688 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:25:34.689 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:25:34.689 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:25:34.689 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:34.689 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:34.689 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:34.690 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:25:34.690 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:34.690 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:34.690 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:25:34.690 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:25:34.691 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:25:34.691 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:25:34.691 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:25:34.691 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:25:34.691 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:25:34.692 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:34.692 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:25:34.692 INFO:teuthology.orchestra.run.smithi125.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:34.692 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:25:34.692 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:25:34.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:34.694 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:25:34.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi125.stderr:2023-10-18T10:25:34.687+0000 7f19b8eda480 -1 init, newargv = 0x5633481e5b30 newargc=15 2023-10-18T10:25:34.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi125.stderr:ceph-fuse[100104]: starting ceph client 2023-10-18T10:25:34.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi125.stderr:ceph-fuse[100104]: starting fuse 2023-10-18T10:25:34.748 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:25:34.748 INFO:teuthology.orchestra.run.smithi125.stdout:44 2023-10-18T10:25:34.749 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-10-18T10:25:34.749 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:25:34.749 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:25:34.750 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:25:34.750 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:25:34.750 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:25:34.751 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:34.751 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:25:34.751 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-18T10:25:34.752 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:25:34.752 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-18T10:25:34.752 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:34.753 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:25:34.753 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:25:34.753 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:25:34.754 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:25:34.754 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:34.754 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:25:34.755 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:25:34.755 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:25:34.755 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:25:34.756 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:25:34.756 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:25:34.756 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:25:34.756 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:25:34.757 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:25:34.757 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.2")) 2023-10-18T10:25:34.757 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:25:34.867 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.2.100104.asok 2023-10-18T10:25:34.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:25:34.877 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.100104.asok 2023-10-18T10:25:34.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:34.878 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.100104.asok status 2023-10-18T10:25:35.012 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:25:35.012 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:25:35.012 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:25:35.013 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:25:35.013 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "2", 2023-10-18T10:25:35.013 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:25:35.013 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-18T10:25:35.013 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "100104", 2023-10-18T10:25:35.014 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:25:35.014 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:35.014 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:25:35.014 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:25:35.015 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 5019, 2023-10-18T10:25:35.015 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:25:35.015 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:25:35.015 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:25:35.015 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 5019 2023-10-18T10:25:35.015 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:35.016 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:25:35.016 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:25:35.016 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.2:0", 2023-10-18T10:25:35.016 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 520348241 2023-10-18T10:25:35.016 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:25:35.017 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:35.017 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:25:35.017 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:25:35.017 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.2:0", 2023-10-18T10:25:35.017 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 520348241 2023-10-18T10:25:35.018 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:25:35.018 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.5019 192.168.0.2:0/520348241", 2023-10-18T10:25:35.018 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.2:0/520348241", 2023-10-18T10:25:35.018 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:25:35.018 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 9, 2023-10-18T10:25:35.019 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 25, 2023-10-18T10:25:35.019 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:25:35.019 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:25:35.019 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:25:35.019 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:25:35.021 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:25:35.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.121+0000 7fb3024bb640 1 -- 172.21.15.125:0/416983733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e8004d00 msgr2=0x7fb2e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:35.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.121+0000 7fb3024bb640 1 --2- 172.21.15.125:0/416983733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e8004d00 0x7fb2e8005180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400e3d0 tx=0x7fb2e80057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:35.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.121+0000 7fb300a58640 1 -- 172.21.15.125:0/416983733 reap_dead start 2023-10-18T10:25:35.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.121+0000 7fb3024bb640 1 -- 172.21.15.125:0/416983733 shutdown_connections 2023-10-18T10:25:35.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.121+0000 7fb3024bb640 1 -- 172.21.15.125:0/416983733 >> 172.21.15.125:0/416983733 conn(0x7fb2fc069930 msgr2=0x7fb2fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:35.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.121+0000 7fb3024bb640 1 -- 172.21.15.125:0/416983733 shutdown_connections 2023-10-18T10:25:35.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.121+0000 7fb3024bb640 1 -- 172.21.15.125:0/416983733 wait complete. 2023-10-18T10:25:35.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.122+0000 7fb3024bb640 1 Processor -- start 2023-10-18T10:25:35.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.122+0000 7fb3024bb640 1 -- start start 2023-10-18T10:25:35.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb3024bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e8004d00 0x7fb2fc139480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:35.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb3024bb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2fc140ac0 0x7fb2fc1eea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:35.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb3024bb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2fc158360 0x7fb2fc1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:35.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb3024bb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc158d20 con 0x7fb2fc140ac0 2023-10-18T10:25:35.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb3024bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc13e2d0 con 0x7fb2e8004d00 2023-10-18T10:25:35.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb3024bb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc13e890 con 0x7fb2fc158360 2023-10-18T10:25:35.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2fc140ac0 0x7fb2fc1eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:35.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2fc140ac0 0x7fb2fc1eea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48752/0 (socket says 172.21.15.125:48752) 2023-10-18T10:25:35.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 -- 172.21.15.125:0/2936682802 learned_addr learned my addr 172.21.15.125:0/2936682802 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:35.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fbfff640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e8004d00 0x7fb2fc139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:35.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb300a58640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2fc158360 0x7fb2fc1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:35.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 -- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2fc158360 msgr2=0x7fb2fc1587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:35.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2fc158360 0x7fb2fc1587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:35.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 -- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e8004d00 msgr2=0x7fb2fc139480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:35.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e8004d00 0x7fb2fc139480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:35.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 -- 172.21.15.125:0/2936682802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc139cd0 con 0x7fb2fc140ac0 2023-10-18T10:25:35.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fbfff640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e8004d00 0x7fb2fc139480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:35.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2fb7fe640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2fc140ac0 0x7fb2fc1eea40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec006680 tx=0x7fb2ec0066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:35.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2f8ff9640 1 -- 172.21.15.125:0/2936682802 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec011900 con 0x7fb2fc140ac0 2023-10-18T10:25:35.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.123+0000 7fb2f8ff9640 1 -- 172.21.15.125:0/2936682802 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2ec011a60 con 0x7fb2fc140ac0 2023-10-18T10:25:35.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.124+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc139fb0 con 0x7fb2fc140ac0 2023-10-18T10:25:35.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.124+0000 7fb2f8ff9640 1 -- 172.21.15.125:0/2936682802 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec011e50 con 0x7fb2fc140ac0 2023-10-18T10:25:35.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.124+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc1399c0 con 0x7fb2fc140ac0 2023-10-18T10:25:35.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.125+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc0690e0 con 0x7fb2fc140ac0 2023-10-18T10:25:35.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.125+0000 7fb2f8ff9640 1 -- 172.21.15.125:0/2936682802 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb2ec010420 con 0x7fb2fc140ac0 2023-10-18T10:25:35.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.125+0000 7fb2f8ff9640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb2dc06ae70 0x7fb2dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:35.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.125+0000 7fb2f8ff9640 1 -- 172.21.15.125:0/2936682802 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb2ec08a5c0 con 0x7fb2fc140ac0 2023-10-18T10:25:35.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.126+0000 7fb2fbfff640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb2dc06ae70 0x7fb2dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:35.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.126+0000 7fb2fbfff640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb2dc06ae70 0x7fb2dc06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009c50 tx=0x7fb2e4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:35.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.130+0000 7fb2f8ff9640 1 -- 172.21.15.125:0/2936682802 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2ec0554c0 con 0x7fb2fc140ac0 2023-10-18T10:25:35.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.335+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb2fc13db50 con 0x7fb2fc140ac0 2023-10-18T10:25:35.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.335+0000 7fb2f8ff9640 1 -- 172.21.15.125:0/2936682802 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb2ec058ae0 con 0x7fb2fc140ac0 2023-10-18T10:25:35.337 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:25:35.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb2dc06ae70 msgr2=0x7fb2dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:35.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb2dc06ae70 0x7fb2dc06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009c50 tx=0x7fb2e4005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:35.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2fc140ac0 msgr2=0x7fb2fc1eea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:35.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2fc140ac0 0x7fb2fc1eea40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec006680 tx=0x7fb2ec0066b0 comp rx=0 tx=0).stop 2023-10-18T10:25:35.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 shutdown_connections 2023-10-18T10:25:35.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb2dc06ae70 0x7fb2dc06d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:35.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2fc158360 0x7fb2fc1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:35.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2fc140ac0 0x7fb2fc1eea40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:35.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 --2- 172.21.15.125:0/2936682802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e8004d00 0x7fb2fc139480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:35.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 >> 172.21.15.125:0/2936682802 conn(0x7fb2fc069930 msgr2=0x7fb2fc0ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:35.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 shutdown_connections 2023-10-18T10:25:35.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:35.337+0000 7fb3024bb640 1 -- 172.21.15.125:0/2936682802 wait complete. 2023-10-18T10:25:35.349 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:25:35.349 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:25:35.349 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2023-10-18T10:25:35.350 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T10:25:35.350 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:35.350 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:25:35.350 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:25:35.351 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:25:35.351 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T10:25:35.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:35.352 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2023-10-18T10:25:35.370 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:35.370 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:35.370 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:35.370 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:35.371 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:35.371 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:35.371 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:35.371 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:35.371 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2023-10-18T10:25:35.372 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:35.372 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:35.372 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2023-10-18T10:25:35.372 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:35.372 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:35.372 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:25:35.373 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:35.373 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:25:35.373 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42688sec preferred_lft 42688sec 2023-10-18T10:25:35.373 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2023-10-18T10:25:35.373 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:35.374 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T10:25:35.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:35.374 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:35.375 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:35.375 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add name ceph-brx type bridge 2023-10-18T10:25:35.375 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr flush dev ceph-brx 2023-10-18T10:25:35.375 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set ceph-brx up 2023-10-18T10:25:35.376 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T10:25:35.376 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:35.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:35.624 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:25:35.651 INFO:teuthology.orchestra.run.smithi139.stdout:1 2023-10-18T10:25:35.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:35.653 DEBUG:teuthology.orchestra.run.smithi139:> route 2023-10-18T10:25:35.720 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2023-10-18T10:25:35.720 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:25:35.720 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:25:35.720 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:25:35.721 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:25:35.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:35.721 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:35.722 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:35.722 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:25:35.722 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:25:35.722 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:25:35.723 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:35.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:35.991 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2023-10-18T10:25:36.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:36.005 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2023-10-18T10:25:36.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:36.060 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:36.061 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:36.061 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:36.061 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-18T10:25:36.061 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:36.145 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-18T10:25:36.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:36.146 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:36.146 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:36.147 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-18T10:25:36.147 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T10:25:36.147 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-18T10:25:36.147 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-18T10:25:36.148 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-18T10:25:36.148 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:36.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:36.298 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:36.298 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:36.299 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2023-10-18T10:25:36.299 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T10:25:36.299 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:36.340 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-18T10:25:36.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:36.341 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:36.395 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T10:25:36.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:36.396 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:36.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2023-10-18T10:25:36.518 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:36.572 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2023-10-18T10:25:36.573 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2023-10-18T10:25:36.573 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2023-10-18T10:25:36.573 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2023-10-18T10:25:36.573 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2023-10-18T10:25:36.573 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2023-10-18T10:25:36.574 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2023-10-18T10:25:36.574 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2023-10-18T10:25:36.574 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2023-10-18T10:25:36.574 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2023-10-18T10:25:36.574 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2023-10-18T10:25:36.574 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2023-10-18T10:25:36.575 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:36.575 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2023-10-18T10:25:36.575 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2023-10-18T10:25:36.575 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2023-10-18T10:25:36.575 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2023-10-18T10:25:36.575 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:36.576 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2023-10-18T10:25:36.576 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:36.576 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:36.576 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:36.576 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:36.577 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2023-10-18T10:25:36.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2023-10-18T10:25:36.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2023-10-18T10:25:36.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2023-10-18T10:25:36.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2023-10-18T10:25:36.578 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2023-10-18T10:25:36.578 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2023-10-18T10:25:36.578 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:36.578 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:36.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:36.579 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2023-10-18T10:25:36.626 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:25:36.627 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-18T10:25:36.669 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2023-10-18T10:25:36.694 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:36.749 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2023-10-18T10:25:36.749 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2023-10-18T10:25:36.750 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2023-10-18T10:25:36.750 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2023-10-18T10:25:36.750 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2023-10-18T10:25:36.750 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2023-10-18T10:25:36.750 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2023-10-18T10:25:36.751 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2023-10-18T10:25:36.751 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2023-10-18T10:25:36.751 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2023-10-18T10:25:36.751 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2023-10-18T10:25:36.751 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2023-10-18T10:25:36.751 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:36.752 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2023-10-18T10:25:36.752 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2023-10-18T10:25:36.752 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2023-10-18T10:25:36.752 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2023-10-18T10:25:36.752 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:36.752 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2023-10-18T10:25:36.753 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:36.753 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:36.753 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:36.753 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:36.753 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2023-10-18T10:25:36.753 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2023-10-18T10:25:36.754 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2023-10-18T10:25:36.754 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2023-10-18T10:25:36.754 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2023-10-18T10:25:36.754 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2023-10-18T10:25:36.754 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2023-10-18T10:25:36.755 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:36.755 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:36.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:36.755 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2023-10-18T10:25:36.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2023-10-18T10:25:36.761+0000 7f2f65b6a480 -1 init, newargv = 0x561ff88fecf0 newargc=15 2023-10-18T10:25:36.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[94437]: starting ceph client 2023-10-18T10:25:36.768 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[94437]: starting fuse 2023-10-18T10:25:36.777 INFO:teuthology.orchestra.run.smithi139.stdout:41 2023-10-18T10:25:36.778 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:25:36.778 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:25:36.778 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2023-10-18T10:25:36.779 DEBUG:teuthology.orchestra.run.smithi139:> import re 2023-10-18T10:25:36.779 DEBUG:teuthology.orchestra.run.smithi139:> import os 2023-10-18T10:25:36.779 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2023-10-18T10:25:36.780 DEBUG:teuthology.orchestra.run.smithi139:> 2023-10-18T10:25:36.780 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2023-10-18T10:25:36.780 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-18T10:25:36.780 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2023-10-18T10:25:36.781 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-18T10:25:36.781 DEBUG:teuthology.orchestra.run.smithi139:> 2023-10-18T10:25:36.781 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2023-10-18T10:25:36.782 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2023-10-18T10:25:36.782 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2023-10-18T10:25:36.782 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2023-10-18T10:25:36.783 DEBUG:teuthology.orchestra.run.smithi139:> 2023-10-18T10:25:36.783 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2023-10-18T10:25:36.783 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:25:36.783 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:25:36.784 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:25:36.784 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2023-10-18T10:25:36.784 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2023-10-18T10:25:36.784 DEBUG:teuthology.orchestra.run.smithi139:> return f 2023-10-18T10:25:36.785 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:25:36.785 DEBUG:teuthology.orchestra.run.smithi139:> 2023-10-18T10:25:36.785 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2023-10-18T10:25:36.786 DEBUG:teuthology.orchestra.run.smithi139:> ' 2023-10-18T10:25:36.896 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.94437.asok 2023-10-18T10:25:36.900 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:25:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:25:36.907 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94437.asok 2023-10-18T10:25:36.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:36.907 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94437.asok status 2023-10-18T10:25:37.044 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2023-10-18T10:25:37.045 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2023-10-18T10:25:37.045 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:25:37.045 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:25:37.045 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2023-10-18T10:25:37.045 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2023-10-18T10:25:37.046 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-18T10:25:37.046 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "94437", 2023-10-18T10:25:37.046 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2023-10-18T10:25:37.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2023-10-18T10:25:37.046 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2023-10-18T10:25:37.047 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2023-10-18T10:25:37.047 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4643, 2023-10-18T10:25:37.047 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2023-10-18T10:25:37.047 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2023-10-18T10:25:37.047 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2023-10-18T10:25:37.047 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 4643 2023-10-18T10:25:37.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2023-10-18T10:25:37.048 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2023-10-18T10:25:37.048 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2023-10-18T10:25:37.048 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:25:37.048 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 178241534 2023-10-18T10:25:37.049 INFO:teuthology.orchestra.run.smithi139.stdout: } 2023-10-18T10:25:37.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2023-10-18T10:25:37.049 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2023-10-18T10:25:37.049 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2023-10-18T10:25:37.049 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:25:37.050 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 178241534 2023-10-18T10:25:37.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2023-10-18T10:25:37.050 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.4643 192.168.0.1:0/178241534", 2023-10-18T10:25:37.050 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/178241534", 2023-10-18T10:25:37.050 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2023-10-18T10:25:37.051 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 9, 2023-10-18T10:25:37.051 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 25, 2023-10-18T10:25:37.051 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:25:37.051 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2023-10-18T10:25:37.051 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2023-10-18T10:25:37.051 INFO:teuthology.orchestra.run.smithi139.stdout:} 2023-10-18T10:25:37.053 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:25:37.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.166+0000 7f64cd36c640 1 -- 172.21.15.125:0/169109420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 msgr2=0x7f64b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:37.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.166+0000 7f64cd36c640 1 --2- 172.21.15.125:0/169109420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 0x7f64b0003d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001d80 tx=0x7f64b00057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.166+0000 7f64cd36c640 1 -- 172.21.15.125:0/169109420 shutdown_connections 2023-10-18T10:25:37.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.166+0000 7f64cd36c640 1 --2- 172.21.15.125:0/169109420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 0x7f64b0003d90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.166+0000 7f64cd36c640 1 -- 172.21.15.125:0/169109420 >> 172.21.15.125:0/169109420 conn(0x7f64c80d0d60 msgr2=0x7f64c80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:37.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.166+0000 7f64cd36c640 1 -- 172.21.15.125:0/169109420 shutdown_connections 2023-10-18T10:25:37.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.166+0000 7f64cd36c640 1 -- 172.21.15.125:0/169109420 wait complete. 2023-10-18T10:25:37.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.167+0000 7f64cd36c640 1 Processor -- start 2023-10-18T10:25:37.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.167+0000 7f64cd36c640 1 -- start start 2023-10-18T10:25:37.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64cd36c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 0x7f64c81f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:37.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64cd36c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c8158800 0x7f64c81f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:37.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64cd36c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c81ef250 0x7f64c806aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:37.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64cd36c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8158c80 con 0x7f64b0003970 2023-10-18T10:25:37.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64cd36c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c81f25d0 con 0x7f64c8158800 2023-10-18T10:25:37.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64cd36c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c81f2dd0 con 0x7f64c81ef250 2023-10-18T10:25:37.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64c6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 0x7f64c81f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:37.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64c6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 0x7f64c81f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48776/0 (socket says 172.21.15.125:48776) 2023-10-18T10:25:37.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64c6ffd640 1 -- 172.21.15.125:0/2639819989 learned_addr learned my addr 172.21.15.125:0/2639819989 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:37.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64c67fc640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c8158800 0x7f64c81f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:37.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.168+0000 7f64c77fe640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c81ef250 0x7f64c806aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:37.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64c6ffd640 1 -- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c81ef250 msgr2=0x7f64c806aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:37.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64c6ffd640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c81ef250 0x7f64c806aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64c6ffd640 1 -- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c8158800 msgr2=0x7f64c81f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:37.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64c6ffd640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c8158800 0x7f64c81f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64c6ffd640 1 -- 172.21.15.125:0/2639819989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c81f30b0 con 0x7f64b0003970 2023-10-18T10:25:37.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64c6ffd640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 0x7f64c81f27d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f64b8000f80 tx=0x7f64b802c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:37.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64af7fe640 1 -- 172.21.15.125:0/2639819989 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b802cea0 con 0x7f64b0003970 2023-10-18T10:25:37.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64af7fe640 1 -- 172.21.15.125:0/2639819989 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64b8006140 con 0x7f64b0003970 2023-10-18T10:25:37.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c81f3390 con 0x7f64b0003970 2023-10-18T10:25:37.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64af7fe640 1 -- 172.21.15.125:0/2639819989 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8006550 con 0x7f64b0003970 2023-10-18T10:25:37.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.169+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c81f3950 con 0x7f64b0003970 2023-10-18T10:25:37.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.170+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c8158be0 con 0x7f64b0003970 2023-10-18T10:25:37.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.170+0000 7f64af7fe640 1 -- 172.21.15.125:0/2639819989 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f64b80066b0 con 0x7f64b0003970 2023-10-18T10:25:37.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.171+0000 7f64af7fe640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f64b406ae40 0x7f64b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:37.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.171+0000 7f64af7fe640 1 -- 172.21.15.125:0/2639819989 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f64b80aacc0 con 0x7f64b0003970 2023-10-18T10:25:37.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.171+0000 7f64c67fc640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f64b406ae40 0x7f64b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:37.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.171+0000 7f64c67fc640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f64b406ae40 0x7f64b406d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64c80c9ce0 tx=0x7f64bc005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:37.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.175+0000 7f64af7fe640 1 -- 172.21.15.125:0/2639819989 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64b8075b40 con 0x7f64b0003970 2023-10-18T10:25:37.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.379+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64c80690e0 con 0x7f64b0003970 2023-10-18T10:25:37.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.379+0000 7f64af7fe640 1 -- 172.21.15.125:0/2639819989 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f64b8079160 con 0x7f64b0003970 2023-10-18T10:25:37.381 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T10:25:37.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f64b406ae40 msgr2=0x7f64b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:37.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f64b406ae40 0x7f64b406d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64c80c9ce0 tx=0x7f64bc005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:37.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 msgr2=0x7f64c81f27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:37.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 0x7f64c81f27d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f64b8000f80 tx=0x7f64b802c8a0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 shutdown_connections 2023-10-18T10:25:37.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f64b406ae40 0x7f64b406d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c8158800 0x7f64c81f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c81ef250 0x7f64c806aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 --2- 172.21.15.125:0/2639819989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b0003970 0x7f64c81f27d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:37.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 >> 172.21.15.125:0/2639819989 conn(0x7f64c80d0d60 msgr2=0x7f64c80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:37.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 shutdown_connections 2023-10-18T10:25:37.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:37.381+0000 7f64cd36c640 1 -- 172.21.15.125:0/2639819989 wait complete. 2023-10-18T10:25:37.393 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:25:37.394 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:25:37.394 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2023-10-18T10:25:37.394 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-18T10:25:37.394 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-18T10:25:37.395 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:25:37.395 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:25:37.395 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:25:37.395 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-18T10:25:37.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.396 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2023-10-18T10:25:37.411 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:37.411 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:37.411 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:37.411 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.412 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:37.412 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.412 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:37.412 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:37.412 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2023-10-18T10:25:37.412 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:37.413 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:37.413 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2023-10-18T10:25:37.413 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:25:37.413 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:37.413 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:25:37.414 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:37.414 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:25:37.414 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42686sec preferred_lft 42686sec 2023-10-18T10:25:37.414 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2023-10-18T10:25:37.414 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.415 INFO:teuthology.orchestra.run.smithi139.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:25:37.415 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether d6:56:1b:48:d3:33 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:25:37.415 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:25:37.415 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.415 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::3461:f0ff:fe96:5fb0/64 scope link 2023-10-18T10:25:37.416 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.416 INFO:teuthology.orchestra.run.smithi139.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:25:37.416 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether d6:56:1b:48:d3:33 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:37.416 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::d456:1bff:fe48:d333/64 scope link tentative 2023-10-18T10:25:37.416 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.417 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:25:37.477 INFO:teuthology.orchestra.run.smithi139.stdout:1 2023-10-18T10:25:37.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.479 DEBUG:teuthology.orchestra.run.smithi139:> route 2023-10-18T10:25:37.533 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2023-10-18T10:25:37.534 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:25:37.534 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:25:37.534 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:25:37.534 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:25:37.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.535 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:37.535 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:37.535 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:25:37.536 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:25:37.536 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:25:37.536 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:37.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.629 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2023-10-18T10:25:37.643 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T10:25:37.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.644 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2023-10-18T10:25:37.698 INFO:teuthology.orchestra.run.smithi139.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-18T10:25:37.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.698 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:37.699 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:37.699 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T10:25:37.699 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-18T10:25:37.700 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:37.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.780 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-18T10:25:37.845 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:37.845 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:37.846 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:37.846 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.846 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:37.846 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.846 INFO:teuthology.orchestra.run.smithi139.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:25:37.847 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 3e:dd:51:ef:8d:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T10:25:37.847 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T10:25:37.847 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.847 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::3cdd:51ff:feef:8d7f/64 scope link 2023-10-18T10:25:37.847 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.848 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-18T10:25:37.914 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:25:37.915 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:25:37.915 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:25:37.915 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.915 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2023-10-18T10:25:37.915 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.915 INFO:teuthology.orchestra.run.smithi139.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:25:37.916 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 3e:dd:51:ef:8d:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T10:25:37.916 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T10:25:37.916 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.916 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::3cdd:51ff:feef:8d7f/64 scope link 2023-10-18T10:25:37.916 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:25:37.917 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-18T10:25:37.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:37.918 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:37.919 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:37.919 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-18T10:25:37.919 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-18T10:25:37.919 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-18T10:25:37.920 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-18T10:25:37.920 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-18T10:25:37.920 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:38.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:38.042 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2023-10-18T10:25:38.042 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2023-10-18T10:25:38.042 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.1 up 2023-10-18T10:25:38.043 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.1 master ceph-brx 2023-10-18T10:25:38.043 DEBUG:teuthology.orchestra.run.smithi139:> ') 2023-10-18T10:25:38.084 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-18T10:25:38.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:38.084 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-18T10:25:38.139 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-18T10:25:38.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:25:38.139 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-18T10:25:38.194 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2023-10-18T10:25:38.260 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:38.315 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2023-10-18T10:25:38.316 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2023-10-18T10:25:38.316 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2023-10-18T10:25:38.316 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2023-10-18T10:25:38.316 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2023-10-18T10:25:38.316 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2023-10-18T10:25:38.317 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2023-10-18T10:25:38.317 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2023-10-18T10:25:38.317 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2023-10-18T10:25:38.317 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2023-10-18T10:25:38.317 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2023-10-18T10:25:38.317 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2023-10-18T10:25:38.318 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:38.318 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2023-10-18T10:25:38.318 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2023-10-18T10:25:38.318 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2023-10-18T10:25:38.319 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2023-10-18T10:25:38.319 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:38.319 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2023-10-18T10:25:38.319 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:38.319 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:38.319 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:38.320 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:38.320 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2023-10-18T10:25:38.320 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2023-10-18T10:25:38.320 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2023-10-18T10:25:38.320 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2023-10-18T10:25:38.321 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2023-10-18T10:25:38.321 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2023-10-18T10:25:38.321 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2023-10-18T10:25:38.321 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:38.321 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:38.321 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:38.322 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T10:25:38.322 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T10:25:38.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:38.322 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2023-10-18T10:25:38.370 INFO:teuthology.orchestra.run.smithi139.stdout:41 2023-10-18T10:25:38.370 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-10-18T10:25:38.370 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-18T10:25:38.412 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2023-10-18T10:25:38.438 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:25:38.494 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2023-10-18T10:25:38.494 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2023-10-18T10:25:38.494 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2023-10-18T10:25:38.495 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2023-10-18T10:25:38.495 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2023-10-18T10:25:38.495 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2023-10-18T10:25:38.495 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2023-10-18T10:25:38.495 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2023-10-18T10:25:38.496 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2023-10-18T10:25:38.496 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2023-10-18T10:25:38.496 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2023-10-18T10:25:38.496 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2023-10-18T10:25:38.496 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:25:38.497 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2023-10-18T10:25:38.497 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2023-10-18T10:25:38.497 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2023-10-18T10:25:38.497 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2023-10-18T10:25:38.497 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2023-10-18T10:25:38.497 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2023-10-18T10:25:38.498 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:25:38.498 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:25:38.498 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:25:38.498 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:25:38.498 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2023-10-18T10:25:38.499 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2023-10-18T10:25:38.499 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2023-10-18T10:25:38.499 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2023-10-18T10:25:38.499 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2023-10-18T10:25:38.499 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2023-10-18T10:25:38.500 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2023-10-18T10:25:38.500 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:38.500 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T10:25:38.500 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:38.500 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T10:25:38.501 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T10:25:38.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:38.501 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2023-10-18T10:25:38.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi139.stderr:2023-10-18T10:25:38.495+0000 7f211360c480 -1 init, newargv = 0x560545cf6cf0 newargc=15 2023-10-18T10:25:38.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi139.stderr:ceph-fuse[94882]: starting ceph client 2023-10-18T10:25:38.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi139.stderr:ceph-fuse[94882]: starting fuse 2023-10-18T10:25:38.516 INFO:teuthology.orchestra.run.smithi139.stdout:41 2023-10-18T10:25:38.516 INFO:teuthology.orchestra.run.smithi139.stdout:44 2023-10-18T10:25:38.517 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-10-18T10:25:38.517 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:25:38.517 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2023-10-18T10:25:38.518 DEBUG:teuthology.orchestra.run.smithi139:> import re 2023-10-18T10:25:38.518 DEBUG:teuthology.orchestra.run.smithi139:> import os 2023-10-18T10:25:38.518 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2023-10-18T10:25:38.519 DEBUG:teuthology.orchestra.run.smithi139:> 2023-10-18T10:25:38.519 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2023-10-18T10:25:38.519 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-18T10:25:38.520 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2023-10-18T10:25:38.520 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-18T10:25:38.520 DEBUG:teuthology.orchestra.run.smithi139:> 2023-10-18T10:25:38.521 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2023-10-18T10:25:38.521 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2023-10-18T10:25:38.521 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2023-10-18T10:25:38.521 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2023-10-18T10:25:38.522 DEBUG:teuthology.orchestra.run.smithi139:> 2023-10-18T10:25:38.522 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2023-10-18T10:25:38.522 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:25:38.523 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:25:38.523 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:25:38.523 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2023-10-18T10:25:38.523 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2023-10-18T10:25:38.524 DEBUG:teuthology.orchestra.run.smithi139:> return f 2023-10-18T10:25:38.524 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:25:38.524 DEBUG:teuthology.orchestra.run.smithi139:> 2023-10-18T10:25:38.524 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.3")) 2023-10-18T10:25:38.525 DEBUG:teuthology.orchestra.run.smithi139:> ' 2023-10-18T10:25:38.635 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.3.94882.asok 2023-10-18T10:25:38.638 INFO:teuthology.orchestra.run.smithi139.stderr:2023-10-18T10:25:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:25:38.646 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.94882.asok 2023-10-18T10:25:38.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:38.647 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.94882.asok status 2023-10-18T10:25:38.781 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2023-10-18T10:25:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2023-10-18T10:25:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:25:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:25:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "3", 2023-10-18T10:25:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2023-10-18T10:25:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-18T10:25:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "94882", 2023-10-18T10:25:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2023-10-18T10:25:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2023-10-18T10:25:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2023-10-18T10:25:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2023-10-18T10:25:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4652, 2023-10-18T10:25:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2023-10-18T10:25:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2023-10-18T10:25:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2023-10-18T10:25:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 4652 2023-10-18T10:25:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2023-10-18T10:25:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2023-10-18T10:25:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2023-10-18T10:25:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.2:0", 2023-10-18T10:25:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3985545258 2023-10-18T10:25:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: } 2023-10-18T10:25:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2023-10-18T10:25:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2023-10-18T10:25:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2023-10-18T10:25:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.2:0", 2023-10-18T10:25:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3985545258 2023-10-18T10:25:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2023-10-18T10:25:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.4652 192.168.0.2:0/3985545258", 2023-10-18T10:25:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.2:0/3985545258", 2023-10-18T10:25:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2023-10-18T10:25:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 9, 2023-10-18T10:25:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 25, 2023-10-18T10:25:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:25:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2023-10-18T10:25:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2023-10-18T10:25:38.788 INFO:teuthology.orchestra.run.smithi139.stdout:} 2023-10-18T10:25:38.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:38.790 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:25:38.790 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:38.809 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:25:38.809 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:38.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:38.810 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:38.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:38.881 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:25:38.881 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-18T10:25:38.937 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:25:38.938 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-18T10:25:38.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:38.938 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-18T10:25:39.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:39.007 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2023-10-18T10:25:39.007 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:39.022 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2023-10-18T10:25:39.023 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:39.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:39.023 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:39.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:39.089 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2023-10-18T10:25:39.090 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-18T10:25:39.144 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2023-10-18T10:25:39.144 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-18T10:25:39.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:39.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-18T10:25:39.210 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-18T10:25:39.226 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:39.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.333+0000 7f183aeb4640 1 -- 172.21.15.125:0/534725474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f183413e520 msgr2=0x7f183406bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.333+0000 7f183aeb4640 1 --2- 172.21.15.125:0/534725474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f183413e520 0x7f183406bb60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1828009830 tx=0x7f182802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.334+0000 7f183aeb4640 1 -- 172.21.15.125:0/534725474 shutdown_connections 2023-10-18T10:25:39.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.334+0000 7f183aeb4640 1 --2- 172.21.15.125:0/534725474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1834070af0 0x7f18340affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.334+0000 7f183aeb4640 1 --2- 172.21.15.125:0/534725474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f183413e520 0x7f183406bb60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.334+0000 7f183aeb4640 1 -- 172.21.15.125:0/534725474 >> 172.21.15.125:0/534725474 conn(0x7f1834069930 msgr2=0x7f1834069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:39.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.334+0000 7f183aeb4640 1 -- 172.21.15.125:0/534725474 shutdown_connections 2023-10-18T10:25:39.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.334+0000 7f183aeb4640 1 -- 172.21.15.125:0/534725474 wait complete. 2023-10-18T10:25:39.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.335+0000 7f183aeb4640 1 Processor -- start 2023-10-18T10:25:39.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.335+0000 7f183aeb4640 1 -- start start 2023-10-18T10:25:39.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f183aeb4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1834070af0 0x7f18340cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f183aeb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183413e520 0x7f183413b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f183aeb4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1834141690 0x7f18340ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f183aeb4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340a6f80 con 0x7f1834070af0 2023-10-18T10:25:39.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f183aeb4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183413b810 con 0x7f183413e520 2023-10-18T10:25:39.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f183aeb4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18340a8ff0 con 0x7f1834141690 2023-10-18T10:25:39.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1834070af0 0x7f18340cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1834070af0 0x7f18340cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58156/0 (socket says 172.21.15.125:58156) 2023-10-18T10:25:39.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 -- 172.21.15.125:0/1636355888 learned_addr learned my addr 172.21.15.125:0/1636355888 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:39.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1839451640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1834141690 0x7f18340ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1833fff640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183413e520 0x7f183413b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 -- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1834141690 msgr2=0x7f18340ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1834141690 0x7f18340ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 -- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183413e520 msgr2=0x7f183413b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183413e520 0x7f183413b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 -- 172.21.15.125:0/1636355888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828009490 con 0x7f1834070af0 2023-10-18T10:25:39.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f1838c50640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1834070af0 0x7f18340cae10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1828009950 tx=0x7f18280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:39.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f18317fa640 1 -- 172.21.15.125:0/1636355888 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828003ea0 con 0x7f1834070af0 2023-10-18T10:25:39.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f18317fa640 1 -- 172.21.15.125:0/1636355888 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1828034530 con 0x7f1834070af0 2023-10-18T10:25:39.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18340a65e0 con 0x7f1834070af0 2023-10-18T10:25:39.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f18317fa640 1 -- 172.21.15.125:0/1636355888 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828034940 con 0x7f1834070af0 2023-10-18T10:25:39.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.336+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18340a6ba0 con 0x7f1834070af0 2023-10-18T10:25:39.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.337+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18340b1000 con 0x7f1834070af0 2023-10-18T10:25:39.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.338+0000 7f18317fa640 1 -- 172.21.15.125:0/1636355888 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1828034aa0 con 0x7f1834070af0 2023-10-18T10:25:39.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.338+0000 7f18317fa640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f181806ae70 0x7f181806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.338+0000 7f18317fa640 1 -- 172.21.15.125:0/1636355888 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f18280ac130 con 0x7f1834070af0 2023-10-18T10:25:39.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.338+0000 7f1833fff640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f181806ae70 0x7f181806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.339+0000 7f1833fff640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f181806ae70 0x7f181806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f18341f29f0 tx=0x7f1820007e50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:39.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.343+0000 7f18317fa640 1 -- 172.21.15.125:0/1636355888 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1828076fa0 con 0x7f1834070af0 2023-10-18T10:25:39.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.511+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18340690e0 con 0x7f1834070af0 2023-10-18T10:25:39.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.512+0000 7f18317fa640 1 -- 172.21.15.125:0/1636355888 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f182807a5c0 con 0x7f1834070af0 2023-10-18T10:25:39.515 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:39.517 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:39.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.514+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f181806ae70 msgr2=0x7f181806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.514+0000 7f183aeb4640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f181806ae70 0x7f181806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f18341f29f0 tx=0x7f1820007e50 comp rx=0 tx=0).stop 2023-10-18T10:25:39.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.514+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1834070af0 msgr2=0x7f18340cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.514+0000 7f183aeb4640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1834070af0 0x7f18340cae10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1828009950 tx=0x7f18280038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.515+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 shutdown_connections 2023-10-18T10:25:39.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.515+0000 7f183aeb4640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f181806ae70 0x7f181806d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.515+0000 7f183aeb4640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183413e520 0x7f183413b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.515+0000 7f183aeb4640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1834141690 0x7f18340ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.515+0000 7f183aeb4640 1 --2- 172.21.15.125:0/1636355888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1834070af0 0x7f18340cae10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.515+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 >> 172.21.15.125:0/1636355888 conn(0x7f1834069930 msgr2=0x7f1834140380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:39.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.515+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 shutdown_connections 2023-10-18T10:25:39.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.515+0000 7f183aeb4640 1 -- 172.21.15.125:0/1636355888 wait complete. 2023-10-18T10:25:39.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:39.528 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:39.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.633+0000 7fb28bbd7640 1 -- 172.21.15.125:0/3872889052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2840cd9e0 msgr2=0x7fb2840cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.633+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/3872889052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2840cd9e0 0x7fb2840cde70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009830 tx=0x7fb27c02bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:39.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.633+0000 7fb28bbd7640 1 -- 172.21.15.125:0/3872889052 shutdown_connections 2023-10-18T10:25:39.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.633+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/3872889052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2840cd9e0 0x7fb2840cde70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.633+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/3872889052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb284158cf0 0x7fb28415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.633+0000 7fb28bbd7640 1 -- 172.21.15.125:0/3872889052 >> 172.21.15.125:0/3872889052 conn(0x7fb2840af7a0 msgr2=0x7fb2840afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:39.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.633+0000 7fb28bbd7640 1 -- 172.21.15.125:0/3872889052 shutdown_connections 2023-10-18T10:25:39.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.633+0000 7fb28bbd7640 1 -- 172.21.15.125:0/3872889052 wait complete. 2023-10-18T10:25:39.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.634+0000 7fb28bbd7640 1 Processor -- start 2023-10-18T10:25:39.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.634+0000 7fb28bbd7640 1 -- start start 2023-10-18T10:25:39.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28bbd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2840cd9e0 0x7fb284150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28bbd7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb284158910 0x7fb284140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28bbd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb284158cf0 0x7fb284069720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28bbd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb284151e80 con 0x7fb284158cf0 2023-10-18T10:25:39.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28bbd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28413ddb0 con 0x7fb2840cd9e0 2023-10-18T10:25:39.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28bbd7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb284151270 con 0x7fb284158910 2023-10-18T10:25:39.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28a174640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb284158cf0 0x7fb284069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28a174640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb284158cf0 0x7fb284069720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58164/0 (socket says 172.21.15.125:58164) 2023-10-18T10:25:39.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb28a174640 1 -- 172.21.15.125:0/4069079788 learned_addr learned my addr 172.21.15.125:0/4069079788 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:39.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb289172640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb284158910 0x7fb284140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.635+0000 7fb289973640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2840cd9e0 0x7fb284150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb28a174640 1 -- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb284158910 msgr2=0x7fb284140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb28a174640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb284158910 0x7fb284140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb28a174640 1 -- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2840cd9e0 msgr2=0x7fb284150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb28a174640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2840cd9e0 0x7fb284150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb28a174640 1 -- 172.21.15.125:0/4069079788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274009490 con 0x7fb284158cf0 2023-10-18T10:25:39.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb28a174640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb284158cf0 0x7fb284069720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009950 tx=0x7fb27c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:39.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb27a7fc640 1 -- 172.21.15.125:0/4069079788 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c003ea0 con 0x7fb284158cf0 2023-10-18T10:25:39.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb27a7fc640 1 -- 172.21.15.125:0/4069079788 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb27c005020 con 0x7fb284158cf0 2023-10-18T10:25:39.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c009490 con 0x7fb284158cf0 2023-10-18T10:25:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb27a7fc640 1 -- 172.21.15.125:0/4069079788 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c005410 con 0x7fb284158cf0 2023-10-18T10:25:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.636+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb284151830 con 0x7fb284158cf0 2023-10-18T10:25:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.637+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2840690e0 con 0x7fb284158cf0 2023-10-18T10:25:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.637+0000 7fb27a7fc640 1 -- 172.21.15.125:0/4069079788 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb27c0047a0 con 0x7fb284158cf0 2023-10-18T10:25:39.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.637+0000 7fb27a7fc640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb26806ae70 0x7fb26806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.638+0000 7fb27a7fc640 1 -- 172.21.15.125:0/4069079788 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb27c0abbc0 con 0x7fb284158cf0 2023-10-18T10:25:39.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.638+0000 7fb289973640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb26806ae70 0x7fb26806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.638+0000 7fb289973640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb26806ae70 0x7fb26806d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2841f4a70 tx=0x7fb280001a70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:39.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.642+0000 7fb27a7fc640 1 -- 172.21.15.125:0/4069079788 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb27c0769b0 con 0x7fb284158cf0 2023-10-18T10:25:39.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.808+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2840ce5e0 con 0x7fb284158cf0 2023-10-18T10:25:39.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.809+0000 7fb27a7fc640 1 -- 172.21.15.125:0/4069079788 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb27c079fd0 con 0x7fb284158cf0 2023-10-18T10:25:39.811 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:39.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb26806ae70 msgr2=0x7fb26806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb26806ae70 0x7fb26806d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2841f4a70 tx=0x7fb280001a70 comp rx=0 tx=0).stop 2023-10-18T10:25:39.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb284158cf0 msgr2=0x7fb284069720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb284158cf0 0x7fb284069720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009950 tx=0x7fb27c0038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 shutdown_connections 2023-10-18T10:25:39.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb26806ae70 0x7fb26806d330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb284158cf0 0x7fb284069720 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb284158910 0x7fb284140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 --2- 172.21.15.125:0/4069079788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2840cd9e0 0x7fb284150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 >> 172.21.15.125:0/4069079788 conn(0x7fb2840af7a0 msgr2=0x7fb28414b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:39.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 shutdown_connections 2023-10-18T10:25:39.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.811+0000 7fb28bbd7640 1 -- 172.21.15.125:0/4069079788 wait complete. 2023-10-18T10:25:39.824 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:39.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.935+0000 7fab720c2640 1 -- 172.21.15.125:0/4074952496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c159060 msgr2=0x7fab6c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.935+0000 7fab720c2640 1 --2- 172.21.15.125:0/4074952496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c159060 0x7fab6c06d260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009860 tx=0x7fab5c02bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:39.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.935+0000 7fab720c2640 1 -- 172.21.15.125:0/4074952496 shutdown_connections 2023-10-18T10:25:39.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.935+0000 7fab720c2640 1 --2- 172.21.15.125:0/4074952496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c159060 0x7fab6c06d260 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.935+0000 7fab720c2640 1 -- 172.21.15.125:0/4074952496 >> 172.21.15.125:0/4074952496 conn(0x7fab6c0b0670 msgr2=0x7fab6c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:39.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.935+0000 7fab720c2640 1 -- 172.21.15.125:0/4074952496 shutdown_connections 2023-10-18T10:25:39.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.935+0000 7fab720c2640 1 -- 172.21.15.125:0/4074952496 wait complete. 2023-10-18T10:25:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.936+0000 7fab720c2640 1 Processor -- start 2023-10-18T10:25:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.936+0000 7fab720c2640 1 -- start start 2023-10-18T10:25:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab720c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c0cdb30 0x7fab6c153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab720c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6c159060 0x7fab6c1f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab720c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab6c15d400 0x7fab6c1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab720c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c154360 con 0x7fab6c0cdb30 2023-10-18T10:25:39.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab720c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1f4cf0 con 0x7fab6c15d400 2023-10-18T10:25:39.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab720c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c34a0 con 0x7fab6c159060 2023-10-18T10:25:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab6b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c0cdb30 0x7fab6c153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab6b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c0cdb30 0x7fab6c153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58174/0 (socket says 172.21.15.125:58174) 2023-10-18T10:25:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.937+0000 7fab6b7fe640 1 -- 172.21.15.125:0/2964166523 learned_addr learned my addr 172.21.15.125:0/2964166523 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab6bfff640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab6c15d400 0x7fab6c1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab62ffd640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6c159060 0x7fab6c1f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab6b7fe640 1 -- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6c159060 msgr2=0x7fab6c1f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab6b7fe640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6c159060 0x7fab6c1f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab6b7fe640 1 -- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab6c15d400 msgr2=0x7fab6c1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab6b7fe640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab6c15d400 0x7fab6c1547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab6b7fe640 1 -- 172.21.15.125:0/2964166523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c009490 con 0x7fab6c0cdb30 2023-10-18T10:25:39.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab6bfff640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab6c15d400 0x7fab6c1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:39.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab62ffd640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6c159060 0x7fab6c1f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:39.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab6b7fe640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c0cdb30 0x7fab6c153e20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009780 tx=0x7fab5c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:39.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab68ff9640 1 -- 172.21.15.125:0/2964166523 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c003ea0 con 0x7fab6c0cdb30 2023-10-18T10:25:39.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab68ff9640 1 -- 172.21.15.125:0/2964166523 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fab5c005110 con 0x7fab6c0cdb30 2023-10-18T10:25:39.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c0c2c00 con 0x7fab6c0cdb30 2023-10-18T10:25:39.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab68ff9640 1 -- 172.21.15.125:0/2964166523 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c005520 con 0x7fab6c0cdb30 2023-10-18T10:25:39.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.938+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c0c31c0 con 0x7fab6c0cdb30 2023-10-18T10:25:39.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.939+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c1f4b50 con 0x7fab6c0cdb30 2023-10-18T10:25:39.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.939+0000 7fab68ff9640 1 -- 172.21.15.125:0/2964166523 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fab5c0047a0 con 0x7fab6c0cdb30 2023-10-18T10:25:39.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.940+0000 7fab68ff9640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab4c06ae40 0x7fab4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:39.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.940+0000 7fab68ff9640 1 -- 172.21.15.125:0/2964166523 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fab5c0abef0 con 0x7fab6c0cdb30 2023-10-18T10:25:39.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.940+0000 7fab62ffd640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab4c06ae40 0x7fab4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:39.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.940+0000 7fab62ffd640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab4c06ae40 0x7fab4c06d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fab6c070db0 tx=0x7fab54009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:39.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:39.944+0000 7fab68ff9640 1 -- 172.21.15.125:0/2964166523 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab6c1f4b50 con 0x7fab6c0cdb30 2023-10-18T10:25:40.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.108+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab6c1f4b50 con 0x7fab6c0cdb30 2023-10-18T10:25:40.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.109+0000 7fab68ff9640 1 -- 172.21.15.125:0/2964166523 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab5c042020 con 0x7fab6c0cdb30 2023-10-18T10:25:40.111 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:40.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab4c06ae40 msgr2=0x7fab4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab4c06ae40 0x7fab4c06d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fab6c070db0 tx=0x7fab54009040 comp rx=0 tx=0).stop 2023-10-18T10:25:40.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c0cdb30 msgr2=0x7fab6c153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c0cdb30 0x7fab6c153e20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009780 tx=0x7fab5c0038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 shutdown_connections 2023-10-18T10:25:40.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab4c06ae40 0x7fab4c06d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab6c15d400 0x7fab6c1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab6c159060 0x7fab6c1f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 --2- 172.21.15.125:0/2964166523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab6c0cdb30 0x7fab6c153e20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 >> 172.21.15.125:0/2964166523 conn(0x7fab6c0b0670 msgr2=0x7fab6c1578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:40.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 shutdown_connections 2023-10-18T10:25:40.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.111+0000 7fab720c2640 1 -- 172.21.15.125:0/2964166523 wait complete. 2023-10-18T10:25:40.123 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:40.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.233+0000 7f8947c15640 1 -- 172.21.15.125:0/3106348302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400b0530 msgr2=0x7f89400b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.233+0000 7f8947c15640 1 --2- 172.21.15.125:0/3106348302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400b0530 0x7f89400b09a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f89400cc7e0 tx=0x7f893802d0e0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.233+0000 7f8947c15640 1 -- 172.21.15.125:0/3106348302 shutdown_connections 2023-10-18T10:25:40.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.233+0000 7f8947c15640 1 --2- 172.21.15.125:0/3106348302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400b0530 0x7f89400b09a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.233+0000 7f8947c15640 1 --2- 172.21.15.125:0/3106348302 >> v2:172.21.15.139:3300/0 conn(0x7f89400cbd20 0x7f8940158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.233+0000 7f8947c15640 1 -- 172.21.15.125:0/3106348302 >> 172.21.15.125:0/3106348302 conn(0x7f8940069930 msgr2=0x7f8940069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:40.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.233+0000 7f8947c15640 1 -- 172.21.15.125:0/3106348302 shutdown_connections 2023-10-18T10:25:40.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.233+0000 7f8947c15640 1 -- 172.21.15.125:0/3106348302 wait complete. 2023-10-18T10:25:40.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.234+0000 7f8947c15640 1 Processor -- start 2023-10-18T10:25:40.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.234+0000 7f8947c15640 1 -- start start 2023-10-18T10:25:40.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f8947c15640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400afa20 0x7f8940148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f8947c15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89400b0530 0x7f8940151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f8947c15640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89400cbd20 0x7f89401523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f8947c15640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401e12a0 con 0x7f89400afa20 2023-10-18T10:25:40.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f8947c15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940153ad0 con 0x7f89400b0530 2023-10-18T10:25:40.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f8947c15640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8940148f70 con 0x7f89400cbd20 2023-10-18T10:25:40.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400afa20 0x7f8940148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400afa20 0x7f8940148a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58194/0 (socket says 172.21.15.125:58194) 2023-10-18T10:25:40.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 -- 172.21.15.125:0/1203198774 learned_addr learned my addr 172.21.15.125:0/1203198774 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:40.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89451b0640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89400b0530 0x7f8940151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89461b2640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89400cbd20 0x7f89401523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 -- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89400cbd20 msgr2=0x7f89401523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89400cbd20 0x7f89401523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 -- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89400b0530 msgr2=0x7f8940151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89400b0530 0x7f8940151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 -- 172.21.15.125:0/1203198774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893800a670 con 0x7f89400afa20 2023-10-18T10:25:40.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89461b2640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89400cbd20 0x7f89401523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:40.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.235+0000 7f89459b1640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400afa20 0x7f8940148a30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f893c00cce0 tx=0x7f893c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:40.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.236+0000 7f89367fc640 1 -- 172.21.15.125:0/1203198774 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c00f450 con 0x7f89400afa20 2023-10-18T10:25:40.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.236+0000 7f89367fc640 1 -- 172.21.15.125:0/1203198774 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f893c00f6a0 con 0x7f89400afa20 2023-10-18T10:25:40.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.236+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894014c700 con 0x7f89400afa20 2023-10-18T10:25:40.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.236+0000 7f89367fc640 1 -- 172.21.15.125:0/1203198774 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c00fab0 con 0x7f89400afa20 2023-10-18T10:25:40.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.236+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8940156900 con 0x7f89400afa20 2023-10-18T10:25:40.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.237+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89400690e0 con 0x7f89400afa20 2023-10-18T10:25:40.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.237+0000 7f89367fc640 1 -- 172.21.15.125:0/1203198774 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f893c00fc10 con 0x7f89400afa20 2023-10-18T10:25:40.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.237+0000 7f89367fc640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f892406ae40 0x7f892406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.237+0000 7f89367fc640 1 -- 172.21.15.125:0/1203198774 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f893c08a2d0 con 0x7f89400afa20 2023-10-18T10:25:40.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.238+0000 7f89451b0640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f892406ae40 0x7f892406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.238+0000 7f89451b0640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f892406ae40 0x7f892406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f894014fbc0 tx=0x7f8930007e30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:40.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.241+0000 7f89367fc640 1 -- 172.21.15.125:0/1203198774 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f893c055120 con 0x7f89400afa20 2023-10-18T10:25:40.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.414+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f894006f700 con 0x7f89400afa20 2023-10-18T10:25:40.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.414+0000 7f89367fc640 1 -- 172.21.15.125:0/1203198774 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f893c058740 con 0x7f89400afa20 2023-10-18T10:25:40.416 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:40.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f892406ae40 msgr2=0x7f892406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f892406ae40 0x7f892406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f894014fbc0 tx=0x7f8930007e30 comp rx=0 tx=0).stop 2023-10-18T10:25:40.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400afa20 msgr2=0x7f8940148a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400afa20 0x7f8940148a30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f893c00cce0 tx=0x7f893c00a620 comp rx=0 tx=0).stop 2023-10-18T10:25:40.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 shutdown_connections 2023-10-18T10:25:40.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f892406ae40 0x7f892406d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89400b0530 0x7f8940151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89400cbd20 0x7f89401523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 --2- 172.21.15.125:0/1203198774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89400afa20 0x7f8940148a30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 >> 172.21.15.125:0/1203198774 conn(0x7f8940069930 msgr2=0x7f894006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:40.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 shutdown_connections 2023-10-18T10:25:40.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.416+0000 7f8947c15640 1 -- 172.21.15.125:0/1203198774 wait complete. 2023-10-18T10:25:40.429 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:40.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.536+0000 7f2c568e7640 1 -- 172.21.15.125:0/4153456960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c380047c0 msgr2=0x7f2c38004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.536+0000 7f2c568e7640 1 --2- 172.21.15.125:0/4153456960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c380047c0 0x7f2c38004c40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2c500acac0 tx=0x7f2c380057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.536+0000 7f2c54e84640 1 -- 172.21.15.125:0/4153456960 reap_dead start 2023-10-18T10:25:40.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.536+0000 7f2c568e7640 1 -- 172.21.15.125:0/4153456960 shutdown_connections 2023-10-18T10:25:40.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.536+0000 7f2c568e7640 1 -- 172.21.15.125:0/4153456960 >> 172.21.15.125:0/4153456960 conn(0x7f2c5006fae0 msgr2=0x7f2c5006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:40.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.536+0000 7f2c568e7640 1 -- 172.21.15.125:0/4153456960 shutdown_connections 2023-10-18T10:25:40.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.536+0000 7f2c568e7640 1 -- 172.21.15.125:0/4153456960 wait complete. 2023-10-18T10:25:40.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.537+0000 7f2c568e7640 1 Processor -- start 2023-10-18T10:25:40.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.537+0000 7f2c568e7640 1 -- start start 2023-10-18T10:25:40.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c568e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380047c0 0x7f2c500ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c568e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c501458f0 0x7f2c5014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c568e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c5006cf90 0x7f2c5006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c568e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5006d950 con 0x7f2c5006cf90 2023-10-18T10:25:40.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c568e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500ae610 con 0x7f2c380047c0 2023-10-18T10:25:40.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c568e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500aeaa0 con 0x7f2c501458f0 2023-10-18T10:25:40.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c54e84640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c5006cf90 0x7f2c5006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c54e84640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c5006cf90 0x7f2c5006d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58216/0 (socket says 172.21.15.125:58216) 2023-10-18T10:25:40.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.538+0000 7f2c54e84640 1 -- 172.21.15.125:0/943090332 learned_addr learned my addr 172.21.15.125:0/943090332 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:40.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c4ffff640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380047c0 0x7f2c500ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c4f7fe640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c501458f0 0x7f2c5014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c54e84640 1 -- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c501458f0 msgr2=0x7f2c5014e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c54e84640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c501458f0 0x7f2c5014e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c54e84640 1 -- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380047c0 msgr2=0x7f2c500ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c54e84640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380047c0 0x7f2c500ad300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c54e84640 1 -- 172.21.15.125:0/943090332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500b2330 con 0x7f2c5006cf90 2023-10-18T10:25:40.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c4ffff640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380047c0 0x7f2c500ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:40.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c54e84640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c5006cf90 0x7f2c5006d410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2c5006b8c0 tx=0x7f2c48006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:40.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c4cff9640 1 -- 172.21.15.125:0/943090332 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48017030 con 0x7f2c5006cf90 2023-10-18T10:25:40.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c4cff9640 1 -- 172.21.15.125:0/943090332 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2c4800fa70 con 0x7f2c5006cf90 2023-10-18T10:25:40.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c4cff9640 1 -- 172.21.15.125:0/943090332 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800fe80 con 0x7f2c5006cf90 2023-10-18T10:25:40.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c500ada00 con 0x7f2c5006cf90 2023-10-18T10:25:40.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.539+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c500adfc0 con 0x7f2c5006cf90 2023-10-18T10:25:40.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.540+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c500690e0 con 0x7f2c5006cf90 2023-10-18T10:25:40.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.540+0000 7f2c4cff9640 1 -- 172.21.15.125:0/943090332 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c4801a070 con 0x7f2c5006cf90 2023-10-18T10:25:40.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.541+0000 7f2c4cff9640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2c3c06ae40 0x7f2c3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.541+0000 7f2c4cff9640 1 -- 172.21.15.125:0/943090332 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c48018070 con 0x7f2c5006cf90 2023-10-18T10:25:40.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.541+0000 7f2c4ffff640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2c3c06ae40 0x7f2c3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.541+0000 7f2c4ffff640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2c3c06ae40 0x7f2c3c06d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c40006fd0 tx=0x7f2c40006f60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:40.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.545+0000 7f2c4cff9640 1 -- 172.21.15.125:0/943090332 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c48054d50 con 0x7f2c5006cf90 2023-10-18T10:25:40.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.712+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2c500ae750 con 0x7f2c5006cf90 2023-10-18T10:25:40.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.712+0000 7f2c4cff9640 1 -- 172.21.15.125:0/943090332 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2c48058370 con 0x7f2c5006cf90 2023-10-18T10:25:40.715 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:40.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2c3c06ae40 msgr2=0x7f2c3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2c3c06ae40 0x7f2c3c06d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c40006fd0 tx=0x7f2c40006f60 comp rx=0 tx=0).stop 2023-10-18T10:25:40.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c5006cf90 msgr2=0x7f2c5006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c5006cf90 0x7f2c5006d410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2c5006b8c0 tx=0x7f2c48006720 comp rx=0 tx=0).stop 2023-10-18T10:25:40.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 shutdown_connections 2023-10-18T10:25:40.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2c3c06ae40 0x7f2c3c06d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2c5006cf90 0x7f2c5006d410 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c501458f0 0x7f2c5014e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 --2- 172.21.15.125:0/943090332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380047c0 0x7f2c500ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 >> 172.21.15.125:0/943090332 conn(0x7f2c5006fae0 msgr2=0x7f2c50150ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:40.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 shutdown_connections 2023-10-18T10:25:40.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.714+0000 7f2c568e7640 1 -- 172.21.15.125:0/943090332 wait complete. 2023-10-18T10:25:40.727 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:40.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.829+0000 7f58be664640 1 -- 172.21.15.125:0/2979128227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58b80afa20 msgr2=0x7f58b8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.829+0000 7f58be664640 1 --2- 172.21.15.125:0/2979128227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58b80afa20 0x7f58b8158300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009830 tx=0x7f58a802bdf0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.829+0000 7f58be664640 1 -- 172.21.15.125:0/2979128227 shutdown_connections 2023-10-18T10:25:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.829+0000 7f58be664640 1 --2- 172.21.15.125:0/2979128227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58b80afa20 0x7f58b8158300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.829+0000 7f58be664640 1 -- 172.21.15.125:0/2979128227 >> 172.21.15.125:0/2979128227 conn(0x7f58b8069930 msgr2=0x7f58b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:40.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.829+0000 7f58be664640 1 -- 172.21.15.125:0/2979128227 shutdown_connections 2023-10-18T10:25:40.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.829+0000 7f58be664640 1 -- 172.21.15.125:0/2979128227 wait complete. 2023-10-18T10:25:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.830+0000 7f58be664640 1 Processor -- start 2023-10-18T10:25:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.830+0000 7f58be664640 1 -- start start 2023-10-18T10:25:40.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58be664640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b80afa20 0x7f58b8141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58be664640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58b80b0530 0x7f58b8153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58be664640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58b80cbd20 0x7f58b8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58be664640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81532f0 con 0x7f58b80afa20 2023-10-18T10:25:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58be664640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81417c0 con 0x7f58b80cbd20 2023-10-18T10:25:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58be664640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8152250 con 0x7f58b80b0530 2023-10-18T10:25:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b80afa20 0x7f58b8141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b80afa20 0x7f58b8141280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58230/0 (socket says 172.21.15.125:58230) 2023-10-18T10:25:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b7fff640 1 -- 172.21.15.125:0/3865968354 learned_addr learned my addr 172.21.15.125:0/3865968354 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b77fe640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58b80b0530 0x7f58b8153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b7fff640 1 -- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58b80b0530 msgr2=0x7f58b8153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b7fff640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58b80b0530 0x7f58b8153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b7fff640 1 -- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58b80cbd20 msgr2=0x7f58b8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58bcc01640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58b80cbd20 0x7f58b8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b7fff640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58b80cbd20 0x7f58b8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.831+0000 7f58b7fff640 1 -- 172.21.15.125:0/3865968354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a8009490 con 0x7f58b80afa20 2023-10-18T10:25:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.832+0000 7f58b7fff640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b80afa20 0x7f58b8141280 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58a000c3c0 tx=0x7f58a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:40.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.832+0000 7f58b4ff9640 1 -- 172.21.15.125:0/3865968354 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00052d0 con 0x7f58b80afa20 2023-10-18T10:25:40.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.832+0000 7f58b4ff9640 1 -- 172.21.15.125:0/3865968354 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f58a0007020 con 0x7f58b80afa20 2023-10-18T10:25:40.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.832+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b8152530 con 0x7f58b80afa20 2023-10-18T10:25:40.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.832+0000 7f58b4ff9640 1 -- 172.21.15.125:0/3865968354 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0007470 con 0x7f58b80afa20 2023-10-18T10:25:40.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.832+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b8152af0 con 0x7f58b80afa20 2023-10-18T10:25:40.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.833+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b80690e0 con 0x7f58b80afa20 2023-10-18T10:25:40.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.833+0000 7f58b4ff9640 1 -- 172.21.15.125:0/3865968354 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f58a00067d0 con 0x7f58b80afa20 2023-10-18T10:25:40.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.833+0000 7f58b4ff9640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f589806ae40 0x7f589806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:40.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.834+0000 7f58b4ff9640 1 -- 172.21.15.125:0/3865968354 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f58a0088fa0 con 0x7f58b80afa20 2023-10-18T10:25:40.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.834+0000 7f58b77fe640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f589806ae40 0x7f589806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:40.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.834+0000 7f58b77fe640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f589806ae40 0x7f589806d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58a8036000 tx=0x7f58a8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:40.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:40.838+0000 7f58b4ff9640 1 -- 172.21.15.125:0/3865968354 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58a00bbca0 con 0x7f58b80afa20 2023-10-18T10:25:41.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.006+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58b80ae490 con 0x7f58b80afa20 2023-10-18T10:25:41.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.007+0000 7f58b4ff9640 1 -- 172.21.15.125:0/3865968354 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f58a0053ce0 con 0x7f58b80afa20 2023-10-18T10:25:41.009 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:41.010 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:41.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f589806ae40 msgr2=0x7f589806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f589806ae40 0x7f589806d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58a8036000 tx=0x7f58a8004730 comp rx=0 tx=0).stop 2023-10-18T10:25:41.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b80afa20 msgr2=0x7f58b8141280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b80afa20 0x7f58b8141280 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58a000c3c0 tx=0x7f58a000c880 comp rx=0 tx=0).stop 2023-10-18T10:25:41.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 shutdown_connections 2023-10-18T10:25:41.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f589806ae40 0x7f589806d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58b80cbd20 0x7f58b8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58b80b0530 0x7f58b8153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 --2- 172.21.15.125:0/3865968354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b80afa20 0x7f58b8141280 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 >> 172.21.15.125:0/3865968354 conn(0x7f58b8069930 msgr2=0x7f58b8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:41.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 shutdown_connections 2023-10-18T10:25:41.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.009+0000 7f58be664640 1 -- 172.21.15.125:0/3865968354 wait complete. 2023-10-18T10:25:41.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:41.023 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:41.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.127+0000 7f538276c640 1 -- 172.21.15.125:0/2724681881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53640052a0 msgr2=0x7f536400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.127+0000 7f538276c640 1 --2- 172.21.15.125:0/2724681881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53640052a0 0x7f536400b980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f537400c610 tx=0x7f53640057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.127+0000 7f538276c640 1 -- 172.21.15.125:0/2724681881 shutdown_connections 2023-10-18T10:25:41.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.127+0000 7f538276c640 1 --2- 172.21.15.125:0/2724681881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53640052a0 0x7f536400b980 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.127+0000 7f538276c640 1 -- 172.21.15.125:0/2724681881 >> 172.21.15.125:0/2724681881 conn(0x7f537c0d0d60 msgr2=0x7f537c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:41.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.128+0000 7f538276c640 1 -- 172.21.15.125:0/2724681881 shutdown_connections 2023-10-18T10:25:41.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.128+0000 7f538276c640 1 -- 172.21.15.125:0/2724681881 wait complete. 2023-10-18T10:25:41.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.129+0000 7f538276c640 1 Processor -- start 2023-10-18T10:25:41.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.129+0000 7f538276c640 1 -- start start 2023-10-18T10:25:41.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f538276c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640052a0 0x7f537c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f538276c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f537c1e5770 0x7f537c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f538276c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f537c15c5d0 0x7f537c158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f538276c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c15d6d0 con 0x7f537c1e5770 2023-10-18T10:25:41.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f538276c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c15ca50 con 0x7f53640052a0 2023-10-18T10:25:41.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f538276c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c15cd30 con 0x7f537c15c5d0 2023-10-18T10:25:41.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f537c1e5770 0x7f537c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f537c1e5770 0x7f537c15c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58242/0 (socket says 172.21.15.125:58242) 2023-10-18T10:25:41.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 -- 172.21.15.125:0/130488709 learned_addr learned my addr 172.21.15.125:0/130488709 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:41.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537bfff640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640052a0 0x7f537c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 -- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f537c15c5d0 msgr2=0x7f537c158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f5380d09640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f537c15c5d0 0x7f537c158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f537c15c5d0 0x7f537c158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 -- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640052a0 msgr2=0x7f537c1584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640052a0 0x7f537c1584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 -- 172.21.15.125:0/130488709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c15d010 con 0x7f537c1e5770 2023-10-18T10:25:41.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537bfff640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640052a0 0x7f537c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:41.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f537b7fe640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f537c1e5770 0x7f537c15c090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f537000f7d0 tx=0x7f537000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:41.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f5378ff9640 1 -- 172.21.15.125:0/130488709 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537000fee0 con 0x7f537c1e5770 2023-10-18T10:25:41.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f5378ff9640 1 -- 172.21.15.125:0/130488709 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f537000bda0 con 0x7f537c1e5770 2023-10-18T10:25:41.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c15d2f0 con 0x7f537c1e5770 2023-10-18T10:25:41.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f5378ff9640 1 -- 172.21.15.125:0/130488709 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370005280 con 0x7f537c1e5770 2023-10-18T10:25:41.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.130+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c1594a0 con 0x7f537c1e5770 2023-10-18T10:25:41.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.131+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c15d9f0 con 0x7f537c1e5770 2023-10-18T10:25:41.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.132+0000 7f5378ff9640 1 -- 172.21.15.125:0/130488709 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53700053e0 con 0x7f537c1e5770 2023-10-18T10:25:41.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.132+0000 7f5378ff9640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f536806ae70 0x7f536806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.132+0000 7f5378ff9640 1 -- 172.21.15.125:0/130488709 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f537008a210 con 0x7f537c1e5770 2023-10-18T10:25:41.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.132+0000 7f537bfff640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f536806ae70 0x7f536806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.133+0000 7f537bfff640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f536806ae70 0x7f536806d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f537c0cb060 tx=0x7f536c006210 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:41.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.136+0000 7f5378ff9640 1 -- 172.21.15.125:0/130488709 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5370055110 con 0x7f537c1e5770 2023-10-18T10:25:41.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.302+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f537c0690e0 con 0x7f537c1e5770 2023-10-18T10:25:41.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.302+0000 7f5378ff9640 1 -- 172.21.15.125:0/130488709 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5370058730 con 0x7f537c1e5770 2023-10-18T10:25:41.304 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f536806ae70 msgr2=0x7f536806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f536806ae70 0x7f536806d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f537c0cb060 tx=0x7f536c006210 comp rx=0 tx=0).stop 2023-10-18T10:25:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f537c1e5770 msgr2=0x7f537c15c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f537c1e5770 0x7f537c15c090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f537000f7d0 tx=0x7f537000fc90 comp rx=0 tx=0).stop 2023-10-18T10:25:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 shutdown_connections 2023-10-18T10:25:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f536806ae70 0x7f536806d330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f537c15c5d0 0x7f537c158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f537c1e5770 0x7f537c15c090 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 --2- 172.21.15.125:0/130488709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640052a0 0x7f537c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 >> 172.21.15.125:0/130488709 conn(0x7f537c0d0d60 msgr2=0x7f537c15a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 shutdown_connections 2023-10-18T10:25:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.304+0000 7f538276c640 1 -- 172.21.15.125:0/130488709 wait complete. 2023-10-18T10:25:41.316 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:41.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1775075181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40b0530 msgr2=0x7f73e40b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1775075181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40b0530 0x7f73e40b09c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f73e40cc4c0 tx=0x7f73dc02c250 comp rx=0 tx=0).stop 2023-10-18T10:25:41.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1775075181 shutdown_connections 2023-10-18T10:25:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1775075181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40b0530 0x7f73e40b09c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1775075181 >> v2:172.21.15.139:3300/0 conn(0x7f73e40afa20 0x7f73e40705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1775075181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e40cbd20 0x7f73e4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1775075181 >> 172.21.15.125:0/1775075181 conn(0x7f73e4069930 msgr2=0x7f73e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1775075181 shutdown_connections 2023-10-18T10:25:41.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.420+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1775075181 wait complete. 2023-10-18T10:25:41.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.421+0000 7f73e9ae5640 1 Processor -- start 2023-10-18T10:25:41.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.421+0000 7f73e9ae5640 1 -- start start 2023-10-18T10:25:41.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e9ae5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e40afa20 0x7f73e413fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e9ae5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e40b0530 0x7f73e413dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e9ae5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40cbd20 0x7f73e413e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e9ae5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e4141690 con 0x7f73e40cbd20 2023-10-18T10:25:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e9ae5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e413e660 con 0x7f73e40afa20 2023-10-18T10:25:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e9ae5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e413e940 con 0x7f73e40b0530 2023-10-18T10:25:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40cbd20 0x7f73e413e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40cbd20 0x7f73e413e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58258/0 (socket says 172.21.15.125:58258) 2023-10-18T10:25:41.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 -- 172.21.15.125:0/1130177682 learned_addr learned my addr 172.21.15.125:0/1130177682 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:41.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e37fe640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e40afa20 0x7f73e413fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e2ffd640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e40b0530 0x7f73e413dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 -- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e40b0530 msgr2=0x7f73e413dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e40b0530 0x7f73e413dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 -- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e40afa20 msgr2=0x7f73e413fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e40afa20 0x7f73e413fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 -- 172.21.15.125:0/1130177682 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc009900 con 0x7f73e40cbd20 2023-10-18T10:25:41.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.422+0000 7f73e3fff640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40cbd20 0x7f73e413e010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73dc037040 tx=0x7f73dc035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:41.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.423+0000 7f73cbfff640 1 -- 172.21.15.125:0/1130177682 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc034440 con 0x7f73e40cbd20 2023-10-18T10:25:41.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.423+0000 7f73cbfff640 1 -- 172.21.15.125:0/1130177682 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f73dc004250 con 0x7f73e40cbd20 2023-10-18T10:25:41.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.423+0000 7f73cbfff640 1 -- 172.21.15.125:0/1130177682 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc004640 con 0x7f73e40cbd20 2023-10-18T10:25:41.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.423+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e413ec20 con 0x7f73e40cbd20 2023-10-18T10:25:41.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.423+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e41401b0 con 0x7f73e40cbd20 2023-10-18T10:25:41.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.424+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e4158980 con 0x7f73e40cbd20 2023-10-18T10:25:41.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.424+0000 7f73cbfff640 1 -- 172.21.15.125:0/1130177682 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f73dc03b040 con 0x7f73e40cbd20 2023-10-18T10:25:41.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.424+0000 7f73cbfff640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f73d006ae40 0x7f73d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.425+0000 7f73cbfff640 1 -- 172.21.15.125:0/1130177682 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f73dc0ac5b0 con 0x7f73e40cbd20 2023-10-18T10:25:41.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.425+0000 7f73e37fe640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f73d006ae40 0x7f73d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.425+0000 7f73e37fe640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f73d006ae40 0x7f73d006d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73d4005d70 tx=0x7f73d4005d00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:41.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.429+0000 7f73cbfff640 1 -- 172.21.15.125:0/1130177682 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73e4158980 con 0x7f73e40cbd20 2023-10-18T10:25:41.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.596+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73e4158980 con 0x7f73e40cbd20 2023-10-18T10:25:41.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.596+0000 7f73cbfff640 1 -- 172.21.15.125:0/1130177682 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73e4158980 con 0x7f73e40cbd20 2023-10-18T10:25:41.598 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:41.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f73d006ae40 msgr2=0x7f73d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f73d006ae40 0x7f73d006d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73d4005d70 tx=0x7f73d4005d00 comp rx=0 tx=0).stop 2023-10-18T10:25:41.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40cbd20 msgr2=0x7f73e413e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40cbd20 0x7f73e413e010 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73dc037040 tx=0x7f73dc035db0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 shutdown_connections 2023-10-18T10:25:41.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f73d006ae40 0x7f73d006d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73e40cbd20 0x7f73e413e010 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e40b0530 0x7f73e413dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 --2- 172.21.15.125:0/1130177682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e40afa20 0x7f73e413fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 >> 172.21.15.125:0/1130177682 conn(0x7f73e4069930 msgr2=0x7f73e4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 shutdown_connections 2023-10-18T10:25:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.598+0000 7f73e9ae5640 1 -- 172.21.15.125:0/1130177682 wait complete. 2023-10-18T10:25:41.610 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:41.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.714+0000 7fc98f883640 1 -- 172.21.15.125:0/1000647926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 msgr2=0x7fc9880d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.714+0000 7fc98f883640 1 --2- 172.21.15.125:0/1000647926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 0x7fc9880d1910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc984005fd0 tx=0x7fc98402c050 comp rx=0 tx=0).stop 2023-10-18T10:25:41.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.714+0000 7fc98f883640 1 -- 172.21.15.125:0/1000647926 shutdown_connections 2023-10-18T10:25:41.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.714+0000 7fc98f883640 1 --2- 172.21.15.125:0/1000647926 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98815b2a0 0x7fc9880cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.714+0000 7fc98f883640 1 --2- 172.21.15.125:0/1000647926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 0x7fc9880d1910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.714+0000 7fc98f883640 1 -- 172.21.15.125:0/1000647926 >> 172.21.15.125:0/1000647926 conn(0x7fc9880d0d60 msgr2=0x7fc9880d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:41.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.714+0000 7fc98f883640 1 -- 172.21.15.125:0/1000647926 shutdown_connections 2023-10-18T10:25:41.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.714+0000 7fc98f883640 1 -- 172.21.15.125:0/1000647926 wait complete. 2023-10-18T10:25:41.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.715+0000 7fc98f883640 1 Processor -- start 2023-10-18T10:25:41.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98f883640 1 -- start start 2023-10-18T10:25:41.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98f883640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9880ceb40 0x7fc9880ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98f883640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 0x7fc988157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98f883640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98815b2a0 0x7fc9880b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98f883640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98815bb70 con 0x7fc9880cf340 2023-10-18T10:25:41.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98f883640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98815be50 con 0x7fc9880ceb40 2023-10-18T10:25:41.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98f883640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98815c130 con 0x7fc98815b2a0 2023-10-18T10:25:41.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98ce1e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 0x7fc988157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98ce1e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 0x7fc988157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58268/0 (socket says 172.21.15.125:58268) 2023-10-18T10:25:41.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.716+0000 7fc98ce1e640 1 -- 172.21.15.125:0/2614267623 learned_addr learned my addr 172.21.15.125:0/2614267623 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:41.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98ce1e640 1 -- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98815b2a0 msgr2=0x7fc9880b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98ce1e640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98815b2a0 0x7fc9880b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98d61f640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9880ceb40 0x7fc9880ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98ce1e640 1 -- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9880ceb40 msgr2=0x7fc9880ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98ce1e640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9880ceb40 0x7fc9880ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98ce1e640 1 -- 172.21.15.125:0/2614267623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc984009840 con 0x7fc9880cf340 2023-10-18T10:25:41.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98d61f640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9880ceb40 0x7fc9880ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:41.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98ce1e640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 0x7fc988157f20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b410 tx=0x7fc97800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:41.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc97dffb640 1 -- 172.21.15.125:0/2614267623 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978011830 con 0x7fc9880cf340 2023-10-18T10:25:41.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc97dffb640 1 -- 172.21.15.125:0/2614267623 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc978011aa0 con 0x7fc9880cf340 2023-10-18T10:25:41.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98815c410 con 0x7fc9880cf340 2023-10-18T10:25:41.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc97dffb640 1 -- 172.21.15.125:0/2614267623 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978010380 con 0x7fc9880cf340 2023-10-18T10:25:41.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.717+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9880cabf0 con 0x7fc9880cf340 2023-10-18T10:25:41.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.718+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9880690e0 con 0x7fc9880cf340 2023-10-18T10:25:41.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.719+0000 7fc97dffb640 1 -- 172.21.15.125:0/2614267623 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc9780104e0 con 0x7fc9880cf340 2023-10-18T10:25:41.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.719+0000 7fc97dffb640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc96c06ae70 0x7fc96c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:41.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.719+0000 7fc97dffb640 1 -- 172.21.15.125:0/2614267623 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc978089be0 con 0x7fc9880cf340 2023-10-18T10:25:41.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.719+0000 7fc98d61f640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc96c06ae70 0x7fc96c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:41.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.720+0000 7fc98d61f640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc96c06ae70 0x7fc96c06d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc984035000 tx=0x7fc984004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:41.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.723+0000 7fc97dffb640 1 -- 172.21.15.125:0/2614267623 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc978054a30 con 0x7fc9880cf340 2023-10-18T10:25:41.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.892+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9880d0b40 con 0x7fc9880cf340 2023-10-18T10:25:41.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.892+0000 7fc97dffb640 1 -- 172.21.15.125:0/2614267623 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9780150a0 con 0x7fc9880cf340 2023-10-18T10:25:41.895 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc96c06ae70 msgr2=0x7fc96c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc96c06ae70 0x7fc96c06d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc984035000 tx=0x7fc984004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 msgr2=0x7fc988157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 0x7fc988157f20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b410 tx=0x7fc97800b8d0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 shutdown_connections 2023-10-18T10:25:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc96c06ae70 0x7fc96c06d330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98815b2a0 0x7fc9880b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9880cf340 0x7fc988157f20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 --2- 172.21.15.125:0/2614267623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9880ceb40 0x7fc9880ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 >> 172.21.15.125:0/2614267623 conn(0x7fc9880d0d60 msgr2=0x7fc98815a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 shutdown_connections 2023-10-18T10:25:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:41.895+0000 7fc98f883640 1 -- 172.21.15.125:0/2614267623 wait complete. 2023-10-18T10:25:41.907 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:42.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.010+0000 7f399a6c2640 1 -- 172.21.15.125:0/2617249262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399415b260 msgr2=0x7f39940cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.010+0000 7f399a6c2640 1 --2- 172.21.15.125:0/2617249262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399415b260 0x7f39940cf310 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3984009830 tx=0x7f398402bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:42.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.010+0000 7f399a6c2640 1 -- 172.21.15.125:0/2617249262 shutdown_connections 2023-10-18T10:25:42.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.010+0000 7f399a6c2640 1 --2- 172.21.15.125:0/2617249262 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f399415b640 0x7f39940cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.010+0000 7f399a6c2640 1 --2- 172.21.15.125:0/2617249262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f399415b260 0x7f39940cf310 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.010+0000 7f399a6c2640 1 -- 172.21.15.125:0/2617249262 >> 172.21.15.125:0/2617249262 conn(0x7f399409a030 msgr2=0x7f39940d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:42.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.010+0000 7f399a6c2640 1 -- 172.21.15.125:0/2617249262 shutdown_connections 2023-10-18T10:25:42.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.010+0000 7f399a6c2640 1 -- 172.21.15.125:0/2617249262 wait complete. 2023-10-18T10:25:42.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.011+0000 7f399a6c2640 1 Processor -- start 2023-10-18T10:25:42.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.011+0000 7f399a6c2640 1 -- start start 2023-10-18T10:25:42.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f399a6c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39940b0b10 0x7f399414fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f399a6c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f399415b260 0x7f3994150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f399a6c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f399415b640 0x7f3994150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f399a6c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b3940 con 0x7f39940b0b10 2023-10-18T10:25:42.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f399a6c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39941f1ad0 con 0x7f399415b640 2023-10-18T10:25:42.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f399a6c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399415d5c0 con 0x7f399415b260 2023-10-18T10:25:42.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3993fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39940b0b10 0x7f399414fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3993fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39940b0b10 0x7f399414fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58276/0 (socket says 172.21.15.125:58276) 2023-10-18T10:25:42.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3993fff640 1 -- 172.21.15.125:0/1693371999 learned_addr learned my addr 172.21.15.125:0/1693371999 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:42.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3993fff640 1 -- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f399415b260 msgr2=0x7f3994150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f39937fe640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f399415b260 0x7f3994150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3993fff640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f399415b260 0x7f3994150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3998c5f640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f399415b640 0x7f3994150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3993fff640 1 -- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f399415b640 msgr2=0x7f3994150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3993fff640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f399415b640 0x7f3994150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3993fff640 1 -- 172.21.15.125:0/1693371999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984009490 con 0x7f39940b0b10 2023-10-18T10:25:42.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f39937fe640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f399415b260 0x7f3994150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:42.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.012+0000 7f3998c5f640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f399415b640 0x7f3994150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:42.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.013+0000 7f3993fff640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39940b0b10 0x7f399414fd60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3984009950 tx=0x7f39840038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:42.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.013+0000 7f3990ff9640 1 -- 172.21.15.125:0/1693371999 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984003ea0 con 0x7f39940b0b10 2023-10-18T10:25:42.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.013+0000 7f3990ff9640 1 -- 172.21.15.125:0/1693371999 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3984034550 con 0x7f39940b0b10 2023-10-18T10:25:42.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.013+0000 7f3990ff9640 1 -- 172.21.15.125:0/1693371999 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984034960 con 0x7f39940b0b10 2023-10-18T10:25:42.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.013+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3994151080 con 0x7f39940b0b10 2023-10-18T10:25:42.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.013+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3994151640 con 0x7f39940b0b10 2023-10-18T10:25:42.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.014+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39940d02f0 con 0x7f39940b0b10 2023-10-18T10:25:42.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.014+0000 7f3990ff9640 1 -- 172.21.15.125:0/1693371999 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f398403b040 con 0x7f39940b0b10 2023-10-18T10:25:42.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.014+0000 7f3990ff9640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f398006ae40 0x7f398006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.015+0000 7f3990ff9640 1 -- 172.21.15.125:0/1693371999 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f39840ac010 con 0x7f39940b0b10 2023-10-18T10:25:42.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.015+0000 7f39937fe640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f398006ae40 0x7f398006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.015+0000 7f39937fe640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f398006ae40 0x7f398006d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39940b6c70 tx=0x7f3988008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:42.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.019+0000 7f3990ff9640 1 -- 172.21.15.125:0/1693371999 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3984076d40 con 0x7f39940b0b10 2023-10-18T10:25:42.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.187+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f39940675f0 con 0x7f39940b0b10 2023-10-18T10:25:42.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.188+0000 7f3990ff9640 1 -- 172.21.15.125:0/1693371999 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3984039020 con 0x7f39940b0b10 2023-10-18T10:25:42.190 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:42.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.189+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f398006ae40 msgr2=0x7f398006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.189+0000 7f399a6c2640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f398006ae40 0x7f398006d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39940b6c70 tx=0x7f3988008040 comp rx=0 tx=0).stop 2023-10-18T10:25:42.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.189+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39940b0b10 msgr2=0x7f399414fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.189+0000 7f399a6c2640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39940b0b10 0x7f399414fd60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3984009950 tx=0x7f39840038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.190+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 shutdown_connections 2023-10-18T10:25:42.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.190+0000 7f399a6c2640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f398006ae40 0x7f398006d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.190+0000 7f399a6c2640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f399415b640 0x7f3994150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.190+0000 7f399a6c2640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f399415b260 0x7f3994150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.190+0000 7f399a6c2640 1 --2- 172.21.15.125:0/1693371999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39940b0b10 0x7f399414fd60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.190+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 >> 172.21.15.125:0/1693371999 conn(0x7f399409a030 msgr2=0x7f399415a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:42.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.190+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 shutdown_connections 2023-10-18T10:25:42.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.190+0000 7f399a6c2640 1 -- 172.21.15.125:0/1693371999 wait complete. 2023-10-18T10:25:42.202 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:42.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.302+0000 7f03d34e5640 1 -- 172.21.15.125:0/3611263421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 msgr2=0x7f03cc0affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.302+0000 7f03d34e5640 1 --2- 172.21.15.125:0/3611263421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 0x7f03cc0affe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0cbd00 tx=0x7f03c402c080 comp rx=0 tx=0).stop 2023-10-18T10:25:42.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.302+0000 7f03d34e5640 1 -- 172.21.15.125:0/3611263421 shutdown_connections 2023-10-18T10:25:42.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.302+0000 7f03d34e5640 1 --2- 172.21.15.125:0/3611263421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 0x7f03cc0affe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.302+0000 7f03d34e5640 1 --2- 172.21.15.125:0/3611263421 >> v2:172.21.15.139:3300/0 conn(0x7f03cc13e520 0x7f03cc06bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.302+0000 7f03d34e5640 1 -- 172.21.15.125:0/3611263421 >> 172.21.15.125:0/3611263421 conn(0x7f03cc069930 msgr2=0x7f03cc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:42.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.302+0000 7f03d34e5640 1 -- 172.21.15.125:0/3611263421 shutdown_connections 2023-10-18T10:25:42.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.302+0000 7f03d34e5640 1 -- 172.21.15.125:0/3611263421 wait complete. 2023-10-18T10:25:42.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.303+0000 7f03d34e5640 1 Processor -- start 2023-10-18T10:25:42.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d34e5640 1 -- start start 2023-10-18T10:25:42.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d34e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 0x7f03cc155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d34e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03cc13e520 0x7f03cc13abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d34e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03cc141690 0x7f03cc0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d34e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc13b100 con 0x7f03cc070af0 2023-10-18T10:25:42.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d34e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc153710 con 0x7f03cc141690 2023-10-18T10:25:42.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d34e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc156d10 con 0x7f03cc13e520 2023-10-18T10:25:42.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d1281640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 0x7f03cc155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d1281640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 0x7f03cc155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58288/0 (socket says 172.21.15.125:58288) 2023-10-18T10:25:42.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.304+0000 7f03d1281640 1 -- 172.21.15.125:0/2624678120 learned_addr learned my addr 172.21.15.125:0/2624678120 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:42.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03d1281640 1 -- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03cc13e520 msgr2=0x7f03cc13abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03d1281640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03cc13e520 0x7f03cc13abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03d1281640 1 -- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03cc141690 msgr2=0x7f03cc0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03d1281640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03cc141690 0x7f03cc0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03d1281640 1 -- 172.21.15.125:0/2624678120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c4009840 con 0x7f03cc070af0 2023-10-18T10:25:42.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03d1281640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 0x7f03cc155970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00d950 tx=0x7f03bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:42.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03c1ffb640 1 -- 172.21.15.125:0/2624678120 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc009c30 con 0x7f03cc070af0 2023-10-18T10:25:42.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03c1ffb640 1 -- 172.21.15.125:0/2624678120 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03bc009e80 con 0x7f03cc070af0 2023-10-18T10:25:42.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc13b580 con 0x7f03cc070af0 2023-10-18T10:25:42.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03c1ffb640 1 -- 172.21.15.125:0/2624678120 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0106c0 con 0x7f03cc070af0 2023-10-18T10:25:42.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.305+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc154e90 con 0x7f03cc070af0 2023-10-18T10:25:42.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.306+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc0b1bd0 con 0x7f03cc070af0 2023-10-18T10:25:42.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.306+0000 7f03c1ffb640 1 -- 172.21.15.125:0/2624678120 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f03bc010820 con 0x7f03cc070af0 2023-10-18T10:25:42.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.306+0000 7f03c1ffb640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f03b006ae40 0x7f03b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.307+0000 7f03c1ffb640 1 -- 172.21.15.125:0/2624678120 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f03bc089d30 con 0x7f03cc070af0 2023-10-18T10:25:42.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.307+0000 7f03d0a80640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f03b006ae40 0x7f03b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.307+0000 7f03d0a80640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f03b006ae40 0x7f03b006d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0cb410 tx=0x7f03c8007930 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:42.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.311+0000 7f03c1ffb640 1 -- 172.21.15.125:0/2624678120 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03bc054b80 con 0x7f03cc070af0 2023-10-18T10:25:42.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.482+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03cc0690e0 con 0x7f03cc070af0 2023-10-18T10:25:42.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.483+0000 7f03c1ffb640 1 -- 172.21.15.125:0/2624678120 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f03bc020090 con 0x7f03cc070af0 2023-10-18T10:25:42.485 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:42.486 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:08.602106+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:42.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f03b006ae40 msgr2=0x7f03b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f03b006ae40 0x7f03b006d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0cb410 tx=0x7f03c8007930 comp rx=0 tx=0).stop 2023-10-18T10:25:42.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 msgr2=0x7f03cc155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 0x7f03cc155970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00d950 tx=0x7f03bc00de10 comp rx=0 tx=0).stop 2023-10-18T10:25:42.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 shutdown_connections 2023-10-18T10:25:42.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f03b006ae40 0x7f03b006d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03cc141690 0x7f03cc0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03cc13e520 0x7f03cc13abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 --2- 172.21.15.125:0/2624678120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03cc070af0 0x7f03cc155970 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 >> 172.21.15.125:0/2624678120 conn(0x7f03cc069930 msgr2=0x7f03cc13d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:42.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 shutdown_connections 2023-10-18T10:25:42.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.485+0000 7f03d34e5640 1 -- 172.21.15.125:0/2624678120 wait complete. 2023-10-18T10:25:42.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:42.498 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:42.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.602+0000 7ff308ce5640 1 -- 172.21.15.125:0/2170132355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 msgr2=0x7ff3040d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.602+0000 7ff308ce5640 1 --2- 172.21.15.125:0/2170132355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 0x7ff3040d1910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009830 tx=0x7ff2f402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.602+0000 7ff308ce5640 1 -- 172.21.15.125:0/2170132355 shutdown_connections 2023-10-18T10:25:42.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.602+0000 7ff308ce5640 1 --2- 172.21.15.125:0/2170132355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff30415b2a0 0x7ff3040cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.602+0000 7ff308ce5640 1 --2- 172.21.15.125:0/2170132355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 0x7ff3040d1910 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.602+0000 7ff308ce5640 1 -- 172.21.15.125:0/2170132355 >> 172.21.15.125:0/2170132355 conn(0x7ff3040d0d60 msgr2=0x7ff3040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:42.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.602+0000 7ff308ce5640 1 -- 172.21.15.125:0/2170132355 shutdown_connections 2023-10-18T10:25:42.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.602+0000 7ff308ce5640 1 -- 172.21.15.125:0/2170132355 wait complete. 2023-10-18T10:25:42.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff308ce5640 1 Processor -- start 2023-10-18T10:25:42.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff308ce5640 1 -- start start 2023-10-18T10:25:42.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff308ce5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3040ceb40 0x7ff3041582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff308ce5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 0x7ff30406b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff308ce5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff30415b2a0 0x7ff30406b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff308ce5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c8340 con 0x7ff3040cf340 2023-10-18T10:25:42.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff308ce5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c84d0 con 0x7ff3040ceb40 2023-10-18T10:25:42.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff308ce5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041e5c70 con 0x7ff30415b2a0 2023-10-18T10:25:42.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 0x7ff30406b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 0x7ff30406b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58304/0 (socket says 172.21.15.125:58304) 2023-10-18T10:25:42.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff2fbfff640 1 -- 172.21.15.125:0/1657877335 learned_addr learned my addr 172.21.15.125:0/1657877335 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:42.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.604+0000 7ff30259c640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3040ceb40 0x7ff3041582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff302d9d640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff30415b2a0 0x7ff30406b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2fbfff640 1 -- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff30415b2a0 msgr2=0x7ff30406b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2fbfff640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff30415b2a0 0x7ff30406b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2fbfff640 1 -- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3040ceb40 msgr2=0x7ff3041582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2fbfff640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3040ceb40 0x7ff3041582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2fbfff640 1 -- 172.21.15.125:0/1657877335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4009490 con 0x7ff3040cf340 2023-10-18T10:25:42.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff302d9d640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff30415b2a0 0x7ff30406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:42.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2fbfff640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 0x7ff30406b130 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00d750 tx=0x7ff2ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:42.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2faffd640 1 -- 172.21.15.125:0/1657877335 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004220 con 0x7ff3040cf340 2023-10-18T10:25:42.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2faffd640 1 -- 172.21.15.125:0/1657877335 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff2ec004470 con 0x7ff3040cf340 2023-10-18T10:25:42.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff2faffd640 1 -- 172.21.15.125:0/1657877335 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec009c30 con 0x7ff3040cf340 2023-10-18T10:25:42.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3040b4f40 con 0x7ff3040cf340 2023-10-18T10:25:42.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.605+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3040b5500 con 0x7ff3040cf340 2023-10-18T10:25:42.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.606+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3040c8640 con 0x7ff3040cf340 2023-10-18T10:25:42.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.606+0000 7ff2faffd640 1 -- 172.21.15.125:0/1657877335 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff2ec005020 con 0x7ff3040cf340 2023-10-18T10:25:42.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.607+0000 7ff2faffd640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2e406ae40 0x7ff2e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.607+0000 7ff2faffd640 1 -- 172.21.15.125:0/1657877335 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff2ec089cf0 con 0x7ff3040cf340 2023-10-18T10:25:42.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.607+0000 7ff30259c640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2e406ae40 0x7ff2e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.608+0000 7ff30259c640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2e406ae40 0x7ff2e406d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4036000 tx=0x7ff2f4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:42.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.611+0000 7ff2faffd640 1 -- 172.21.15.125:0/1657877335 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3040c8640 con 0x7ff3040cf340 2023-10-18T10:25:42.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.778+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3040c8640 con 0x7ff3040cf340 2023-10-18T10:25:42.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.779+0000 7ff2faffd640 1 -- 172.21.15.125:0/1657877335 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3040c8640 con 0x7ff3040cf340 2023-10-18T10:25:42.781 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:42.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.780+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2e406ae40 msgr2=0x7ff2e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.780+0000 7ff308ce5640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2e406ae40 0x7ff2e406d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4036000 tx=0x7ff2f4004730 comp rx=0 tx=0).stop 2023-10-18T10:25:42.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 msgr2=0x7ff30406b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 0x7ff30406b130 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00d750 tx=0x7ff2ec00dc10 comp rx=0 tx=0).stop 2023-10-18T10:25:42.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 shutdown_connections 2023-10-18T10:25:42.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff2e406ae40 0x7ff2e406d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff30415b2a0 0x7ff30406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3040cf340 0x7ff30406b130 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 --2- 172.21.15.125:0/1657877335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3040ceb40 0x7ff3041582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 >> 172.21.15.125:0/1657877335 conn(0x7ff3040d0d60 msgr2=0x7ff30415a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:42.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 shutdown_connections 2023-10-18T10:25:42.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.781+0000 7ff308ce5640 1 -- 172.21.15.125:0/1657877335 wait complete. 2023-10-18T10:25:42.793 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:42.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.896+0000 7fab8c432640 1 -- 172.21.15.125:0/1919980134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 msgr2=0x7fab8406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.896+0000 7fab8c432640 1 --2- 172.21.15.125:0/1919980134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 0x7fab8406b2e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fab80009830 tx=0x7fab8002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.896+0000 7fab8c432640 1 -- 172.21.15.125:0/1919980134 shutdown_connections 2023-10-18T10:25:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.896+0000 7fab8c432640 1 --2- 172.21.15.125:0/1919980134 >> v2:172.21.15.139:3300/0 conn(0x7fab840abc10 0x7fab840c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.896+0000 7fab8c432640 1 --2- 172.21.15.125:0/1919980134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 0x7fab8406b2e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.896+0000 7fab8c432640 1 -- 172.21.15.125:0/1919980134 >> 172.21.15.125:0/1919980134 conn(0x7fab84069930 msgr2=0x7fab84069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.896+0000 7fab8c432640 1 -- 172.21.15.125:0/1919980134 shutdown_connections 2023-10-18T10:25:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.896+0000 7fab8c432640 1 -- 172.21.15.125:0/1919980134 wait complete. 2023-10-18T10:25:42.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.897+0000 7fab8c432640 1 Processor -- start 2023-10-18T10:25:42.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.897+0000 7fab8c432640 1 -- start start 2023-10-18T10:25:42.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8c432640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 0x7fab841380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8c432640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab840abc10 0x7fab841f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8c432640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab840c96b0 0x7fab840b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8c432640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab84139850 con 0x7fab8406af00 2023-10-18T10:25:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8c432640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab840b1450 con 0x7fab840c96b0 2023-10-18T10:25:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8c432640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab841386d0 con 0x7fab840abc10 2023-10-18T10:25:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 0x7fab841380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 0x7fab841380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58322/0 (socket says 172.21.15.125:58322) 2023-10-18T10:25:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 -- 172.21.15.125:0/2679893003 learned_addr learned my addr 172.21.15.125:0/2679893003 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a9cf640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab840c96b0 0x7fab840b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab899cd640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab840abc10 0x7fab841f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 -- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab840abc10 msgr2=0x7fab841f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab840abc10 0x7fab841f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 -- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab840c96b0 msgr2=0x7fab840b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab840c96b0 0x7fab840b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 -- 172.21.15.125:0/2679893003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab80009490 con 0x7fab8406af00 2023-10-18T10:25:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a9cf640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab840c96b0 0x7fab840b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab8a1ce640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 0x7fab841380c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab80009950 tx=0x7fab800038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.898+0000 7fab7affd640 1 -- 172.21.15.125:0/2679893003 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80003ea0 con 0x7fab8406af00 2023-10-18T10:25:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.899+0000 7fab7affd640 1 -- 172.21.15.125:0/2679893003 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fab80034550 con 0x7fab8406af00 2023-10-18T10:25:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.899+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab841389b0 con 0x7fab8406af00 2023-10-18T10:25:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.899+0000 7fab7affd640 1 -- 172.21.15.125:0/2679893003 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80034960 con 0x7fab8406af00 2023-10-18T10:25:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.899+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab84138f70 con 0x7fab8406af00 2023-10-18T10:25:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.900+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab840690e0 con 0x7fab8406af00 2023-10-18T10:25:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.900+0000 7fab7affd640 1 -- 172.21.15.125:0/2679893003 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fab8003b040 con 0x7fab8406af00 2023-10-18T10:25:42.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.900+0000 7fab7affd640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab6806ae70 0x7fab6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:42.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.900+0000 7fab7affd640 1 -- 172.21.15.125:0/2679893003 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fab800abec0 con 0x7fab8406af00 2023-10-18T10:25:42.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.901+0000 7fab899cd640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab6806ae70 0x7fab6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:42.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.901+0000 7fab899cd640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab6806ae70 0x7fab6806d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab74007fd0 tx=0x7fab74009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:42.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:42.905+0000 7fab7affd640 1 -- 172.21.15.125:0/2679893003 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab80076d10 con 0x7fab8406af00 2023-10-18T10:25:43.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.070+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab840c8d30 con 0x7fab8406af00 2023-10-18T10:25:43.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.070+0000 7fab7affd640 1 -- 172.21.15.125:0/2679893003 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab80039020 con 0x7fab8406af00 2023-10-18T10:25:43.073 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab6806ae70 msgr2=0x7fab6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab6806ae70 0x7fab6806d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab74007fd0 tx=0x7fab74009040 comp rx=0 tx=0).stop 2023-10-18T10:25:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 msgr2=0x7fab841380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 0x7fab841380c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab80009950 tx=0x7fab800038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 shutdown_connections 2023-10-18T10:25:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fab6806ae70 0x7fab6806d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab840c96b0 0x7fab840b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab840abc10 0x7fab841f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 --2- 172.21.15.125:0/2679893003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab8406af00 0x7fab841380c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 >> 172.21.15.125:0/2679893003 conn(0x7fab84069930 msgr2=0x7fab840cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:43.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 shutdown_connections 2023-10-18T10:25:43.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.072+0000 7fab8c432640 1 -- 172.21.15.125:0/2679893003 wait complete. 2023-10-18T10:25:43.085 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:43.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 -- 172.21.15.125:0/3892604848 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a80cdb30 msgr2=0x7fc4a80cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/3892604848 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a80cdb30 0x7fc4a80cdfb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80ceab0 tx=0x7fc49c02bd10 comp rx=0 tx=0).stop 2023-10-18T10:25:43.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 -- 172.21.15.125:0/3892604848 shutdown_connections 2023-10-18T10:25:43.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/3892604848 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a80cdb30 0x7fc4a80cdfb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/3892604848 >> v2:172.21.15.139:3300/0 conn(0x7fc4a815ccf0 0x7fc4a815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/3892604848 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4a8159060 0x7fc4a806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 -- 172.21.15.125:0/3892604848 >> 172.21.15.125:0/3892604848 conn(0x7fc4a80b0670 msgr2=0x7fc4a80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:43.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 -- 172.21.15.125:0/3892604848 shutdown_connections 2023-10-18T10:25:43.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.189+0000 7fc4adfb6640 1 -- 172.21.15.125:0/3892604848 wait complete. 2023-10-18T10:25:43.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.190+0000 7fc4adfb6640 1 Processor -- start 2023-10-18T10:25:43.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.190+0000 7fc4adfb6640 1 -- start start 2023-10-18T10:25:43.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4adfb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4a80cdb30 0x7fc4a81f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4adfb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a8159060 0x7fc4a815a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4adfb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a815ccf0 0x7fc4a815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4adfb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a806a8c0 con 0x7fc4a815ccf0 2023-10-18T10:25:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4adfb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80c3b70 con 0x7fc4a8159060 2023-10-18T10:25:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4adfb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80c3d80 con 0x7fc4a80cdb30 2023-10-18T10:25:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a815ccf0 0x7fc4a815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a815ccf0 0x7fc4a815ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58338/0 (socket says 172.21.15.125:58338) 2023-10-18T10:25:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a7fff640 1 -- 172.21.15.125:0/2280093097 learned_addr learned my addr 172.21.15.125:0/2280093097 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a77fe640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4a80cdb30 0x7fc4a81f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a6ffd640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a8159060 0x7fc4a815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a7fff640 1 -- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4a80cdb30 msgr2=0x7fc4a81f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a7fff640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4a80cdb30 0x7fc4a81f1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a7fff640 1 -- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a8159060 msgr2=0x7fc4a815a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a7fff640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a8159060 0x7fc4a815a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a7fff640 1 -- 172.21.15.125:0/2280093097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c009560 con 0x7fc4a815ccf0 2023-10-18T10:25:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.191+0000 7fc4a6ffd640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a8159060 0x7fc4a815a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.192+0000 7fc4a7fff640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a815ccf0 0x7fc4a815ab90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000ea80 tx=0x7fc4a000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.192+0000 7fc48ffff640 1 -- 172.21.15.125:0/2280093097 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000cea0 con 0x7fc4a815ccf0 2023-10-18T10:25:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.192+0000 7fc48ffff640 1 -- 172.21.15.125:0/2280093097 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4a0004b20 con 0x7fc4a815ccf0 2023-10-18T10:25:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.192+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a80c4060 con 0x7fc4a815ccf0 2023-10-18T10:25:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.192+0000 7fc48ffff640 1 -- 172.21.15.125:0/2280093097 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0005020 con 0x7fc4a815ccf0 2023-10-18T10:25:43.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.192+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a80c4620 con 0x7fc4a815ccf0 2023-10-18T10:25:43.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.193+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a806aa30 con 0x7fc4a815ccf0 2023-10-18T10:25:43.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.193+0000 7fc48ffff640 1 -- 172.21.15.125:0/2280093097 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc4a0005180 con 0x7fc4a815ccf0 2023-10-18T10:25:43.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.193+0000 7fc48ffff640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc49406ae40 0x7fc49406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.193+0000 7fc48ffff640 1 -- 172.21.15.125:0/2280093097 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc4a0014070 con 0x7fc4a815ccf0 2023-10-18T10:25:43.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.194+0000 7fc4a77fe640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc49406ae40 0x7fc49406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.194+0000 7fc4a77fe640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc49406ae40 0x7fc49406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc498005d80 tx=0x7fc498005c70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:43.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.198+0000 7fc48ffff640 1 -- 172.21.15.125:0/2280093097 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4a806aa30 con 0x7fc4a815ccf0 2023-10-18T10:25:43.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.366+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4a806aa30 con 0x7fc4a815ccf0 2023-10-18T10:25:43.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.367+0000 7fc48ffff640 1 -- 172.21.15.125:0/2280093097 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4a806aa30 con 0x7fc4a815ccf0 2023-10-18T10:25:43.370 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc49406ae40 msgr2=0x7fc49406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc49406ae40 0x7fc49406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc498005d80 tx=0x7fc498005c70 comp rx=0 tx=0).stop 2023-10-18T10:25:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a815ccf0 msgr2=0x7fc4a815ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a815ccf0 0x7fc4a815ab90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000ea80 tx=0x7fc4a000ef40 comp rx=0 tx=0).stop 2023-10-18T10:25:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 shutdown_connections 2023-10-18T10:25:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc49406ae40 0x7fc49406d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4a8159060 0x7fc4a815a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4a815ccf0 0x7fc4a815ab90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.372+0000 7fc4adfb6640 1 --2- 172.21.15.125:0/2280093097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4a80cdb30 0x7fc4a81f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.373+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 >> 172.21.15.125:0/2280093097 conn(0x7fc4a80b0670 msgr2=0x7fc4a80b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:43.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.373+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 shutdown_connections 2023-10-18T10:25:43.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.373+0000 7fc4adfb6640 1 -- 172.21.15.125:0/2280093097 wait complete. 2023-10-18T10:25:43.385 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:43.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.488+0000 7f72772b4640 1 -- 172.21.15.125:0/1112932547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700cbd20 msgr2=0x7f7270158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.488+0000 7f72772b4640 1 --2- 172.21.15.125:0/1112932547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700cbd20 0x7f7270158300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f72700cc490 tx=0x7f726c02be40 comp rx=0 tx=0).stop 2023-10-18T10:25:43.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.489+0000 7f72772b4640 1 -- 172.21.15.125:0/1112932547 shutdown_connections 2023-10-18T10:25:43.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.489+0000 7f72772b4640 1 --2- 172.21.15.125:0/1112932547 >> v2:172.21.15.139:3300/0 conn(0x7f72700afa20 0x7f72700705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.489+0000 7f72772b4640 1 --2- 172.21.15.125:0/1112932547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700cbd20 0x7f7270158300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.489+0000 7f72772b4640 1 -- 172.21.15.125:0/1112932547 >> 172.21.15.125:0/1112932547 conn(0x7f7270069930 msgr2=0x7f7270069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:43.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.489+0000 7f72772b4640 1 -- 172.21.15.125:0/1112932547 shutdown_connections 2023-10-18T10:25:43.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.489+0000 7f72772b4640 1 -- 172.21.15.125:0/1112932547 wait complete. 2023-10-18T10:25:43.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f72772b4640 1 Processor -- start 2023-10-18T10:25:43.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f72772b4640 1 -- start start 2023-10-18T10:25:43.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f72772b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72700afa20 0x7f7270141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f72772b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700b0530 0x7f727013af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f72772b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72700cbd20 0x7f7270140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f72772b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7270140de0 con 0x7f72700b0530 2023-10-18T10:25:43.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f72772b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72701410c0 con 0x7f72700afa20 2023-10-18T10:25:43.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f72772b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72701413a0 con 0x7f72700cbd20 2023-10-18T10:25:43.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.490+0000 7f727484f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700b0530 0x7f727013af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f727484f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700b0530 0x7f727013af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58360/0 (socket says 172.21.15.125:58360) 2023-10-18T10:25:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f727484f640 1 -- 172.21.15.125:0/956951828 learned_addr learned my addr 172.21.15.125:0/956951828 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f7275050640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72700afa20 0x7f7270141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f7275851640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72700cbd20 0x7f7270140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f727484f640 1 -- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72700cbd20 msgr2=0x7f7270140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f727484f640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72700cbd20 0x7f7270140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f727484f640 1 -- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72700afa20 msgr2=0x7f7270141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f727484f640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72700afa20 0x7f7270141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f727484f640 1 -- 172.21.15.125:0/956951828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c009560 con 0x7f72700b0530 2023-10-18T10:25:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f7275851640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72700cbd20 0x7f7270140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f727484f640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700b0530 0x7f727013af60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f726000d950 tx=0x7f726000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f7265ffb640 1 -- 172.21.15.125:0/956951828 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260009c30 con 0x7f72700b0530 2023-10-18T10:25:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f7265ffb640 1 -- 172.21.15.125:0/956951828 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7260009e80 con 0x7f72700b0530 2023-10-18T10:25:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7270141680 con 0x7f72700b0530 2023-10-18T10:25:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f7265ffb640 1 -- 172.21.15.125:0/956951828 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600106c0 con 0x7f72700b0530 2023-10-18T10:25:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.491+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727013b4a0 con 0x7f72700b0530 2023-10-18T10:25:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.492+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7270069d10 con 0x7f72700b0530 2023-10-18T10:25:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.492+0000 7f7265ffb640 1 -- 172.21.15.125:0/956951828 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7260010820 con 0x7f72700b0530 2023-10-18T10:25:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.493+0000 7f7265ffb640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f725406ae40 0x7f725406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.493+0000 7f7265ffb640 1 -- 172.21.15.125:0/956951828 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7260089e10 con 0x7f72700b0530 2023-10-18T10:25:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.493+0000 7f7275050640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f725406ae40 0x7f725406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.494+0000 7f7275050640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f725406ae40 0x7f725406d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f726c035000 tx=0x7f726c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.497+0000 7f7265ffb640 1 -- 172.21.15.125:0/956951828 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7260054bb0 con 0x7f72700b0530 2023-10-18T10:25:43.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.663+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f72700690e0 con 0x7f72700b0530 2023-10-18T10:25:43.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.663+0000 7f7265ffb640 1 -- 172.21.15.125:0/956951828 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f72600170a0 con 0x7f72700b0530 2023-10-18T10:25:43.665 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:43.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f725406ae40 msgr2=0x7f725406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f725406ae40 0x7f725406d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f726c035000 tx=0x7f726c004730 comp rx=0 tx=0).stop 2023-10-18T10:25:43.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700b0530 msgr2=0x7f727013af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700b0530 0x7f727013af60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f726000d950 tx=0x7f726000de10 comp rx=0 tx=0).stop 2023-10-18T10:25:43.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 shutdown_connections 2023-10-18T10:25:43.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f725406ae40 0x7f725406d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72700cbd20 0x7f7270140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72700b0530 0x7f727013af60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 --2- 172.21.15.125:0/956951828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72700afa20 0x7f7270141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 >> 172.21.15.125:0/956951828 conn(0x7f7270069930 msgr2=0x7f7270157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:43.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 shutdown_connections 2023-10-18T10:25:43.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.665+0000 7f72772b4640 1 -- 172.21.15.125:0/956951828 wait complete. 2023-10-18T10:25:43.678 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:43.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.779+0000 7fb270807640 1 -- 172.21.15.125:0/821141672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb26c0cf340 msgr2=0x7fb26c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.779+0000 7fb270807640 1 --2- 172.21.15.125:0/821141672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb26c0cf340 0x7fb26c0d1910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005fd0 tx=0x7fb25c02c050 comp rx=0 tx=0).stop 2023-10-18T10:25:43.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.779+0000 7fb270807640 1 -- 172.21.15.125:0/821141672 shutdown_connections 2023-10-18T10:25:43.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.779+0000 7fb270807640 1 --2- 172.21.15.125:0/821141672 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb26c15b2a0 0x7fb26c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.779+0000 7fb270807640 1 --2- 172.21.15.125:0/821141672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb26c0cf340 0x7fb26c0d1910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.779+0000 7fb270807640 1 -- 172.21.15.125:0/821141672 >> 172.21.15.125:0/821141672 conn(0x7fb26c0d0d60 msgr2=0x7fb26c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:43.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.779+0000 7fb270807640 1 -- 172.21.15.125:0/821141672 shutdown_connections 2023-10-18T10:25:43.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.779+0000 7fb270807640 1 -- 172.21.15.125:0/821141672 wait complete. 2023-10-18T10:25:43.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb270807640 1 Processor -- start 2023-10-18T10:25:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb270807640 1 -- start start 2023-10-18T10:25:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb270807640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb26c0ceb40 0x7fb26c0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb270807640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb26c0cf340 0x7fb26c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb270807640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb26c15b2a0 0x7fb26c0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb270807640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c1e3870 con 0x7fb26c15b2a0 2023-10-18T10:25:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb270807640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c1e4110 con 0x7fb26c0cf340 2023-10-18T10:25:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb270807640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0c7540 con 0x7fb26c0ceb40 2023-10-18T10:25:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.781+0000 7fb26ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb26c15b2a0 0x7fb26c0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb26c15b2a0 0x7fb26c0af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58362/0 (socket says 172.21.15.125:58362) 2023-10-18T10:25:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26ad9d640 1 -- 172.21.15.125:0/2374464865 learned_addr learned my addr 172.21.15.125:0/2374464865 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb269d9b640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb26c0cf340 0x7fb26c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26a59c640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb26c0ceb40 0x7fb26c0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26ad9d640 1 -- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb26c0ceb40 msgr2=0x7fb26c0c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26ad9d640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb26c0ceb40 0x7fb26c0c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26ad9d640 1 -- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb26c0cf340 msgr2=0x7fb26c0aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26ad9d640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb26c0cf340 0x7fb26c0aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26ad9d640 1 -- 172.21.15.125:0/2374464865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c009840 con 0x7fb26c15b2a0 2023-10-18T10:25:43.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26a59c640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb26c0ceb40 0x7fb26c0c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:43.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb26ad9d640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb26c15b2a0 0x7fb26c0af1d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb26400c460 tx=0x7fb26400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:43.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb25affd640 1 -- 172.21.15.125:0/2374464865 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264012830 con 0x7fb26c15b2a0 2023-10-18T10:25:43.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c15e670 con 0x7fb26c15b2a0 2023-10-18T10:25:43.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb25affd640 1 -- 172.21.15.125:0/2374464865 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb264012a80 con 0x7fb26c15b2a0 2023-10-18T10:25:43.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb25affd640 1 -- 172.21.15.125:0/2374464865 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264012e70 con 0x7fb26c15b2a0 2023-10-18T10:25:43.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.782+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c1e3da0 con 0x7fb26c15b2a0 2023-10-18T10:25:43.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.783+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c1e4280 con 0x7fb26c15b2a0 2023-10-18T10:25:43.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.784+0000 7fb25affd640 1 -- 172.21.15.125:0/2374464865 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb26401b400 con 0x7fb26c15b2a0 2023-10-18T10:25:43.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.784+0000 7fb25affd640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb24c06ae40 0x7fb24c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:43.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.784+0000 7fb25affd640 1 -- 172.21.15.125:0/2374464865 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb264089ae0 con 0x7fb26c15b2a0 2023-10-18T10:25:43.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.784+0000 7fb26a59c640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb24c06ae40 0x7fb24c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:43.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.785+0000 7fb26a59c640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb24c06ae40 0x7fb24c06d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb25c035000 tx=0x7fb25c004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:43.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.788+0000 7fb25affd640 1 -- 172.21.15.125:0/2374464865 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2640547e0 con 0x7fb26c15b2a0 2023-10-18T10:25:43.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.955+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb26c0690e0 con 0x7fb26c15b2a0 2023-10-18T10:25:43.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.957+0000 7fb25affd640 1 -- 172.21.15.125:0/2374464865 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fb264057e00 con 0x7fb26c15b2a0 2023-10-18T10:25:43.959 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:43.959 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":26,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:43.351724+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:43.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.958+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb24c06ae40 msgr2=0x7fb24c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.958+0000 7fb270807640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb24c06ae40 0x7fb24c06d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb25c035000 tx=0x7fb25c004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb26c15b2a0 msgr2=0x7fb26c0af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:43.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb26c15b2a0 0x7fb26c0af1d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb26400c460 tx=0x7fb26400c920 comp rx=0 tx=0).stop 2023-10-18T10:25:43.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 shutdown_connections 2023-10-18T10:25:43.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb24c06ae40 0x7fb24c06d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb26c0cf340 0x7fb26c0aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb26c15b2a0 0x7fb26c0af1d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 --2- 172.21.15.125:0/2374464865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb26c0ceb40 0x7fb26c0c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:43.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 >> 172.21.15.125:0/2374464865 conn(0x7fb26c0d0d60 msgr2=0x7fb26c0ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:43.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 shutdown_connections 2023-10-18T10:25:43.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:43.959+0000 7fb270807640 1 -- 172.21.15.125:0/2374464865 wait complete. 2023-10-18T10:25:43.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:43.972 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:44.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.075+0000 7f8417694640 1 -- 172.21.15.125:0/782296816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f841015b2a0 msgr2=0x7f84100cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.075+0000 7f8417694640 1 --2- 172.21.15.125:0/782296816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f841015b2a0 0x7f84100cf720 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8400005fd0 tx=0x7f840002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.075+0000 7f8417694640 1 -- 172.21.15.125:0/782296816 shutdown_connections 2023-10-18T10:25:44.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.075+0000 7f8417694640 1 --2- 172.21.15.125:0/782296816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f841015b2a0 0x7f84100cf720 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.075+0000 7f8417694640 1 --2- 172.21.15.125:0/782296816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84100cf340 0x7f84100d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.075+0000 7f8417694640 1 -- 172.21.15.125:0/782296816 >> 172.21.15.125:0/782296816 conn(0x7f84100d0d60 msgr2=0x7f84100d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:44.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.075+0000 7f8417694640 1 -- 172.21.15.125:0/782296816 shutdown_connections 2023-10-18T10:25:44.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.075+0000 7f8417694640 1 -- 172.21.15.125:0/782296816 wait complete. 2023-10-18T10:25:44.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.076+0000 7f8417694640 1 Processor -- start 2023-10-18T10:25:44.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.076+0000 7f8417694640 1 -- start start 2023-10-18T10:25:44.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8417694640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84100ceb40 0x7f8410157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8417694640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84100cf340 0x7f841006a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8417694640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841015b2a0 0x7f841006ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8417694640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c8160 con 0x7f84100ceb40 2023-10-18T10:25:44.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8417694640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c8f50 con 0x7f841015b2a0 2023-10-18T10:25:44.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8417694640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c8340 con 0x7f84100cf340 2023-10-18T10:25:44.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84100ceb40 0x7f8410157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84100ceb40 0x7f8410157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58378/0 (socket says 172.21.15.125:58378) 2023-10-18T10:25:44.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 -- 172.21.15.125:0/2993957457 learned_addr learned my addr 172.21.15.125:0/2993957457 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:44.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8414c2f640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84100cf340 0x7f841006a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415c31640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841015b2a0 0x7f841006ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 -- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84100cf340 msgr2=0x7f841006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84100cf340 0x7f841006a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 -- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841015b2a0 msgr2=0x7f841006ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841015b2a0 0x7f841006ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 -- 172.21.15.125:0/2993957457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400009560 con 0x7f84100ceb40 2023-10-18T10:25:44.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415c31640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841015b2a0 0x7f841006ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:44.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.077+0000 7f8415430640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84100ceb40 0x7f8410157720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f840c00ea80 tx=0x7f840c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:44.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.078+0000 7f8405ffb640 1 -- 172.21.15.125:0/2993957457 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00ce40 con 0x7f84100ceb40 2023-10-18T10:25:44.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.078+0000 7f8405ffb640 1 -- 172.21.15.125:0/2993957457 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f840c004c10 con 0x7f84100ceb40 2023-10-18T10:25:44.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.078+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84100c8620 con 0x7f84100ceb40 2023-10-18T10:25:44.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.078+0000 7f8405ffb640 1 -- 172.21.15.125:0/2993957457 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c005020 con 0x7f84100ceb40 2023-10-18T10:25:44.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.078+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84100c8be0 con 0x7f84100ceb40 2023-10-18T10:25:44.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.079+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84100690e0 con 0x7f84100ceb40 2023-10-18T10:25:44.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.079+0000 7f8405ffb640 1 -- 172.21.15.125:0/2993957457 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f840c005180 con 0x7f84100ceb40 2023-10-18T10:25:44.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.079+0000 7f8405ffb640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f83f406ae70 0x7f83f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.079+0000 7f8405ffb640 1 -- 172.21.15.125:0/2993957457 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f840c08a4b0 con 0x7f84100ceb40 2023-10-18T10:25:44.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.080+0000 7f8414c2f640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f83f406ae70 0x7f83f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.080+0000 7f8414c2f640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f83f406ae70 0x7f83f406d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8400035000 tx=0x7f8400004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:44.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.083+0000 7f8405ffb640 1 -- 172.21.15.125:0/2993957457 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f840c0551a0 con 0x7f84100ceb40 2023-10-18T10:25:44.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.251+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84100d0b40 con 0x7f84100ceb40 2023-10-18T10:25:44.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.251+0000 7f8405ffb640 1 -- 172.21.15.125:0/2993957457 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f840c01f020 con 0x7f84100ceb40 2023-10-18T10:25:44.254 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:44.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.253+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f83f406ae70 msgr2=0x7f83f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.253+0000 7f8417694640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f83f406ae70 0x7f83f406d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8400035000 tx=0x7f8400004730 comp rx=0 tx=0).stop 2023-10-18T10:25:44.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.253+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84100ceb40 msgr2=0x7f8410157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.253+0000 7f8417694640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84100ceb40 0x7f8410157720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f840c00ea80 tx=0x7f840c00ef40 comp rx=0 tx=0).stop 2023-10-18T10:25:44.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.253+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 shutdown_connections 2023-10-18T10:25:44.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.253+0000 7f8417694640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f83f406ae70 0x7f83f406d330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.253+0000 7f8417694640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f841015b2a0 0x7f841006ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.254+0000 7f8417694640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84100cf340 0x7f841006a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.254+0000 7f8417694640 1 --2- 172.21.15.125:0/2993957457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84100ceb40 0x7f8410157720 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.254+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 >> 172.21.15.125:0/2993957457 conn(0x7f84100d0d60 msgr2=0x7f84100ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:44.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.254+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 shutdown_connections 2023-10-18T10:25:44.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.254+0000 7f8417694640 1 -- 172.21.15.125:0/2993957457 wait complete. 2023-10-18T10:25:44.266 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:44.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/1557996214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d240abf90 msgr2=0x7f4d2415bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/1557996214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d240abf90 0x7f4d2415bf10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4d240af7e0 tx=0x7f4d1c02c1d0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/1557996214 shutdown_connections 2023-10-18T10:25:44.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/1557996214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d2406f5b0 0x7f4d24070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/1557996214 >> v2:172.21.15.139:3300/0 conn(0x7f4d2415f720 0x7f4d2406fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/1557996214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d240abf90 0x7f4d2415bf10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/1557996214 >> 172.21.15.125:0/1557996214 conn(0x7f4d240ae720 msgr2=0x7f4d240aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:44.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/1557996214 shutdown_connections 2023-10-18T10:25:44.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.373+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/1557996214 wait complete. 2023-10-18T10:25:44.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.374+0000 7f4d2a9a5640 1 Processor -- start 2023-10-18T10:25:44.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.375+0000 7f4d2a9a5640 1 -- start start 2023-10-18T10:25:44.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.375+0000 7f4d2a9a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2406f5b0 0x7f4d241f1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.375+0000 7f4d2a9a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d240abf90 0x7f4d2414cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.375+0000 7f4d2a9a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2415f720 0x7f4d240c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d2a9a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d241f1640 con 0x7f4d2406f5b0 2023-10-18T10:25:44.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d2a9a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2414d010 con 0x7f4d2415f720 2023-10-18T10:25:44.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d2a9a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2414bd10 con 0x7f4d240abf90 2023-10-18T10:25:44.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2406f5b0 0x7f4d241f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2406f5b0 0x7f4d241f1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58398/0 (socket says 172.21.15.125:58398) 2023-10-18T10:25:44.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 -- 172.21.15.125:0/800275844 learned_addr learned my addr 172.21.15.125:0/800275844 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:44.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d237fe640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d240abf90 0x7f4d2414cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d28f42640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2415f720 0x7f4d240c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 -- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d240abf90 msgr2=0x7f4d2414cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d240abf90 0x7f4d2414cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 -- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2415f720 msgr2=0x7f4d240c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2415f720 0x7f4d240c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 -- 172.21.15.125:0/800275844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c009900 con 0x7f4d2406f5b0 2023-10-18T10:25:44.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d23fff640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2406f5b0 0x7f4d241f1100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c000f80 tx=0x7f4d1c035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:44.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d20ff9640 1 -- 172.21.15.125:0/800275844 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c034440 con 0x7f4d2406f5b0 2023-10-18T10:25:44.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d20ff9640 1 -- 172.21.15.125:0/800275844 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d1c004540 con 0x7f4d2406f5b0 2023-10-18T10:25:44.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d20ff9640 1 -- 172.21.15.125:0/800275844 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c003e60 con 0x7f4d2406f5b0 2023-10-18T10:25:44.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2414bff0 con 0x7f4d2406f5b0 2023-10-18T10:25:44.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.376+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2414c5b0 con 0x7f4d2406f5b0 2023-10-18T10:25:44.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.377+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d240b2d40 con 0x7f4d2406f5b0 2023-10-18T10:25:44.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.378+0000 7f4d20ff9640 1 -- 172.21.15.125:0/800275844 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4d1c03b040 con 0x7f4d2406f5b0 2023-10-18T10:25:44.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.378+0000 7f4d20ff9640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d1006ae40 0x7f4d1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.378+0000 7f4d20ff9640 1 -- 172.21.15.125:0/800275844 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4d1c0ac690 con 0x7f4d2406f5b0 2023-10-18T10:25:44.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.378+0000 7f4d237fe640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d1006ae40 0x7f4d1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.379+0000 7f4d237fe640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d1006ae40 0x7f4d1006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4d2406cdf0 tx=0x7f4d1800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:44.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.382+0000 7f4d20ff9640 1 -- 172.21.15.125:0/800275844 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d1c077300 con 0x7f4d2406f5b0 2023-10-18T10:25:44.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.548+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d240690e0 con 0x7f4d2406f5b0 2023-10-18T10:25:44.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.548+0000 7f4d20ff9640 1 -- 172.21.15.125:0/800275844 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d1c07a920 con 0x7f4d2406f5b0 2023-10-18T10:25:44.550 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:44.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d1006ae40 msgr2=0x7f4d1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d1006ae40 0x7f4d1006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4d2406cdf0 tx=0x7f4d1800b2e0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2406f5b0 msgr2=0x7f4d241f1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2406f5b0 0x7f4d241f1100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c000f80 tx=0x7f4d1c035e30 comp rx=0 tx=0).stop 2023-10-18T10:25:44.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 shutdown_connections 2023-10-18T10:25:44.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4d1006ae40 0x7f4d1006d300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2415f720 0x7f4d240c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d240abf90 0x7f4d2414cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 --2- 172.21.15.125:0/800275844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2406f5b0 0x7f4d241f1100 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 >> 172.21.15.125:0/800275844 conn(0x7f4d240ae720 msgr2=0x7f4d2415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:44.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 shutdown_connections 2023-10-18T10:25:44.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.550+0000 7f4d2a9a5640 1 -- 172.21.15.125:0/800275844 wait complete. 2023-10-18T10:25:44.563 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:44.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.667+0000 7fa7b06fc640 1 -- 172.21.15.125:0/2850967926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa794003970 msgr2=0x7fa794003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.667+0000 7fa7b06fc640 1 --2- 172.21.15.125:0/2850967926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa794003970 0x7fa794003d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c2c0 tx=0x7fa7940057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.667+0000 7fa7aec99640 1 -- 172.21.15.125:0/2850967926 reap_dead start 2023-10-18T10:25:44.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.667+0000 7fa7b06fc640 1 -- 172.21.15.125:0/2850967926 shutdown_connections 2023-10-18T10:25:44.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.667+0000 7fa7b06fc640 1 -- 172.21.15.125:0/2850967926 >> 172.21.15.125:0/2850967926 conn(0x7fa7a8069930 msgr2=0x7fa7a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:44.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.667+0000 7fa7b06fc640 1 -- 172.21.15.125:0/2850967926 shutdown_connections 2023-10-18T10:25:44.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.667+0000 7fa7b06fc640 1 -- 172.21.15.125:0/2850967926 wait complete. 2023-10-18T10:25:44.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.668+0000 7fa7b06fc640 1 Processor -- start 2023-10-18T10:25:44.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.668+0000 7fa7b06fc640 1 -- start start 2023-10-18T10:25:44.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7b06fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa794003970 0x7fa7a8150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7b06fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a81428a0 0x7fa7a812df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7b06fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a8130f90 0x7fa7a8146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7b06fc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8130820 con 0x7fa7a8130f90 2023-10-18T10:25:44.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7b06fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8143140 con 0x7fa794003970 2023-10-18T10:25:44.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7b06fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a812e4d0 con 0x7fa7a81428a0 2023-10-18T10:25:44.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a8130f90 0x7fa7a8146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a8130f90 0x7fa7a8146aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58426/0 (socket says 172.21.15.125:58426) 2023-10-18T10:25:44.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 -- 172.21.15.125:0/4206610843 learned_addr learned my addr 172.21.15.125:0/4206610843 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:44.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 -- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a81428a0 msgr2=0x7fa7a812df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a81428a0 0x7fa7a812df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 -- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa794003970 msgr2=0x7fa7a8150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa794003970 0x7fa7a8150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 -- 172.21.15.125:0/4206610843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a8147520 con 0x7fa7a8130f90 2023-10-18T10:25:44.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.669+0000 7fa7aec99640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a8130f90 0x7fa7a8146aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806f410 tx=0x7fa7a0009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:44.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.670+0000 7fa79effd640 1 -- 172.21.15.125:0/4206610843 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0004d20 con 0x7fa7a8130f90 2023-10-18T10:25:44.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.670+0000 7fa79effd640 1 -- 172.21.15.125:0/4206610843 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa7a0009e80 con 0x7fa7a8130f90 2023-10-18T10:25:44.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.670+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a8150ef0 con 0x7fa7a8130f90 2023-10-18T10:25:44.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.670+0000 7fa79effd640 1 -- 172.21.15.125:0/4206610843 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000faf0 con 0x7fa7a8130f90 2023-10-18T10:25:44.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.670+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81514b0 con 0x7fa7a8130f90 2023-10-18T10:25:44.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.671+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a80690e0 con 0x7fa7a8130f90 2023-10-18T10:25:44.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.671+0000 7fa79effd640 1 -- 172.21.15.125:0/4206610843 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa7a000fc50 con 0x7fa7a8130f90 2023-10-18T10:25:44.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.671+0000 7fa79effd640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa78c06ae70 0x7fa78c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.672+0000 7fa79effd640 1 -- 172.21.15.125:0/4206610843 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa7a008a240 con 0x7fa7a8130f90 2023-10-18T10:25:44.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.672+0000 7fa7ae498640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa78c06ae70 0x7fa78c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.672+0000 7fa7ae498640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa78c06ae70 0x7fa78c06d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4034000 tx=0x7fa7a4001a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:44.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.676+0000 7fa79effd640 1 -- 172.21.15.125:0/4206610843 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7a0054fc0 con 0x7fa7a8130f90 2023-10-18T10:25:44.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.841+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa7a81598d0 con 0x7fa7a8130f90 2023-10-18T10:25:44.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.841+0000 7fa79effd640 1 -- 172.21.15.125:0/4206610843 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7a00585e0 con 0x7fa7a8130f90 2023-10-18T10:25:44.843 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:44.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa78c06ae70 msgr2=0x7fa78c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa78c06ae70 0x7fa78c06d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4034000 tx=0x7fa7a4001a00 comp rx=0 tx=0).stop 2023-10-18T10:25:44.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a8130f90 msgr2=0x7fa7a8146aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a8130f90 0x7fa7a8146aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa7a806f410 tx=0x7fa7a0009b40 comp rx=0 tx=0).stop 2023-10-18T10:25:44.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 shutdown_connections 2023-10-18T10:25:44.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa78c06ae70 0x7fa78c06d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a8130f90 0x7fa7a8146aa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a81428a0 0x7fa7a812df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 --2- 172.21.15.125:0/4206610843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa794003970 0x7fa7a8150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 >> 172.21.15.125:0/4206610843 conn(0x7fa7a8069930 msgr2=0x7fa7a8158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:44.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 shutdown_connections 2023-10-18T10:25:44.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.843+0000 7fa7b06fc640 1 -- 172.21.15.125:0/4206610843 wait complete. 2023-10-18T10:25:44.855 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:44.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.959+0000 7f5aa052c640 1 -- 172.21.15.125:0/938373263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a84003970 msgr2=0x7f5a84003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.959+0000 7f5aa052c640 1 --2- 172.21.15.125:0/938373263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a84003970 0x7f5a84003d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c610 tx=0x7f5a840057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.960+0000 7f5aa052c640 1 -- 172.21.15.125:0/938373263 shutdown_connections 2023-10-18T10:25:44.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.960+0000 7f5aa052c640 1 --2- 172.21.15.125:0/938373263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a84003970 0x7f5a84003d70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.960+0000 7f5aa052c640 1 -- 172.21.15.125:0/938373263 >> 172.21.15.125:0/938373263 conn(0x7f5a980d0d60 msgr2=0x7f5a980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:44.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.960+0000 7f5aa052c640 1 -- 172.21.15.125:0/938373263 shutdown_connections 2023-10-18T10:25:44.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.960+0000 7f5aa052c640 1 -- 172.21.15.125:0/938373263 wait complete. 2023-10-18T10:25:44.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.961+0000 7f5aa052c640 1 Processor -- start 2023-10-18T10:25:44.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.961+0000 7f5aa052c640 1 -- start start 2023-10-18T10:25:44.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.961+0000 7f5aa052c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a84003970 0x7f5a9814a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.961+0000 7f5aa052c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a980ca730 0x7f5a980b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.961+0000 7f5aa052c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a980c94c0 0x7f5a980b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.961+0000 7f5aa052c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980caea0 con 0x7f5a980c94c0 2023-10-18T10:25:44.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.961+0000 7f5aa052c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980c87f0 con 0x7f5a980ca730 2023-10-18T10:25:44.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.961+0000 7f5aa052c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980c9030 con 0x7f5a84003970 2023-10-18T10:25:44.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a980c94c0 0x7f5a980b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a980c94c0 0x7f5a980b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58440/0 (socket says 172.21.15.125:58440) 2023-10-18T10:25:44.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 -- 172.21.15.125:0/418088702 learned_addr learned my addr 172.21.15.125:0/418088702 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:44.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9dac7640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a980ca730 0x7f5a980b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9e2c8640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a84003970 0x7f5a9814a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 -- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a84003970 msgr2=0x7f5a9814a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a84003970 0x7f5a9814a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 -- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a980ca730 msgr2=0x7f5a980b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:44.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a980ca730 0x7f5a980b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:44.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 -- 172.21.15.125:0/418088702 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a980cabc0 con 0x7f5a980c94c0 2023-10-18T10:25:44.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9dac7640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a980ca730 0x7f5a980b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:44.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a9eac9640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a980c94c0 0x7f5a980b5fb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a980b6e10 tx=0x7f5a9000d6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:44.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a8effd640 1 -- 172.21.15.125:0/418088702 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90010660 con 0x7f5a980c94c0 2023-10-18T10:25:44.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a8effd640 1 -- 172.21.15.125:0/418088702 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a900108b0 con 0x7f5a980c94c0 2023-10-18T10:25:44.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a980c89e0 con 0x7f5a980c94c0 2023-10-18T10:25:44.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5a8effd640 1 -- 172.21.15.125:0/418088702 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90010cc0 con 0x7f5a980c94c0 2023-10-18T10:25:44.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.962+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9814a5a0 con 0x7f5a980c94c0 2023-10-18T10:25:44.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.963+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a980c91e0 con 0x7f5a980c94c0 2023-10-18T10:25:44.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.964+0000 7f5a8effd640 1 -- 172.21.15.125:0/418088702 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5a9001c460 con 0x7f5a980c94c0 2023-10-18T10:25:44.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.964+0000 7f5a8effd640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5a7c06ae40 0x7f5a7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:44.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.964+0000 7f5a8effd640 1 -- 172.21.15.125:0/418088702 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5a90089a30 con 0x7f5a980c94c0 2023-10-18T10:25:44.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.964+0000 7f5a9e2c8640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5a7c06ae40 0x7f5a7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:44.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.965+0000 7f5a9e2c8640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5a7c06ae40 0x7f5a7c06d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a94034000 tx=0x7f5a9402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:44.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:44.968+0000 7f5a8effd640 1 -- 172.21.15.125:0/418088702 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a90054720 con 0x7f5a980c94c0 2023-10-18T10:25:45.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.138+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5a980690e0 con 0x7f5a980c94c0 2023-10-18T10:25:45.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.138+0000 7f5a8effd640 1 -- 172.21.15.125:0/418088702 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5a90057d40 con 0x7f5a980c94c0 2023-10-18T10:25:45.140 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:45.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5a7c06ae40 msgr2=0x7f5a7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5a7c06ae40 0x7f5a7c06d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a94034000 tx=0x7f5a9402e000 comp rx=0 tx=0).stop 2023-10-18T10:25:45.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a980c94c0 msgr2=0x7f5a980b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a980c94c0 0x7f5a980b5fb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a980b6e10 tx=0x7f5a9000d6a0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 shutdown_connections 2023-10-18T10:25:45.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f5a7c06ae40 0x7f5a7c06d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a980ca730 0x7f5a980b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a980c94c0 0x7f5a980b5fb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 --2- 172.21.15.125:0/418088702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a84003970 0x7f5a9814a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 >> 172.21.15.125:0/418088702 conn(0x7f5a980d0d60 msgr2=0x7f5a9806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:45.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 shutdown_connections 2023-10-18T10:25:45.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.140+0000 7f5aa052c640 1 -- 172.21.15.125:0/418088702 wait complete. 2023-10-18T10:25:45.183 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-10-18T10:25:45.184 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T10:25:45.184 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:45.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.286+0000 7fe3b339e640 1 -- 172.21.15.125:0/2392421469 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 msgr2=0x7fe3ac0ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.286+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2392421469 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 0x7fe3ac0ab990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0713a0 tx=0x7fe3a402c250 comp rx=0 tx=0).stop 2023-10-18T10:25:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.287+0000 7fe3b339e640 1 -- 172.21.15.125:0/2392421469 shutdown_connections 2023-10-18T10:25:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.287+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2392421469 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 0x7fe3ac0ab990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.287+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2392421469 >> v2:172.21.15.139:3300/0 conn(0x7fe3ac0c96b0 0x7fe3ac0c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.287+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2392421469 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3ac06af00 0x7fe3ac06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.287+0000 7fe3b339e640 1 -- 172.21.15.125:0/2392421469 >> 172.21.15.125:0/2392421469 conn(0x7fe3ac069930 msgr2=0x7fe3ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:45.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.287+0000 7fe3b339e640 1 -- 172.21.15.125:0/2392421469 shutdown_connections 2023-10-18T10:25:45.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.287+0000 7fe3b339e640 1 -- 172.21.15.125:0/2392421469 wait complete. 2023-10-18T10:25:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.288+0000 7fe3b339e640 1 Processor -- start 2023-10-18T10:25:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.288+0000 7fe3b339e640 1 -- start start 2023-10-18T10:25:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.288+0000 7fe3b339e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3ac06af00 0x7fe3ac12e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.288+0000 7fe3b339e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 0x7fe3ac1e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.288+0000 7fe3b339e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3ac0c96b0 0x7fe3ac1e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.288+0000 7fe3b339e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac1e62a0 con 0x7fe3ac0ab500 2023-10-18T10:25:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.288+0000 7fe3b339e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac1e7b20 con 0x7fe3ac0c96b0 2023-10-18T10:25:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.288+0000 7fe3b339e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac1e5620 con 0x7fe3ac06af00 2023-10-18T10:25:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 0x7fe3ac1e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 0x7fe3ac1e4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58464/0 (socket says 172.21.15.125:58464) 2023-10-18T10:25:45.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 -- 172.21.15.125:0/2859557417 learned_addr learned my addr 172.21.15.125:0/2859557417 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:45.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b193b640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3ac0c96b0 0x7fe3ac1e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b113a640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3ac06af00 0x7fe3ac12e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 -- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3ac06af00 msgr2=0x7fe3ac12e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3ac06af00 0x7fe3ac12e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 -- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3ac0c96b0 msgr2=0x7fe3ac1e75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3ac0c96b0 0x7fe3ac1e75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 -- 172.21.15.125:0/2859557417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a4009900 con 0x7fe3ac0ab500 2023-10-18T10:25:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b193b640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3ac0c96b0 0x7fe3ac1e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b0939640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 0x7fe3ac1e4f20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00d770 tx=0x7fe39c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3a1ffb640 1 -- 172.21.15.125:0/2859557417 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c009c30 con 0x7fe3ac0ab500 2023-10-18T10:25:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3a1ffb640 1 -- 172.21.15.125:0/2859557417 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe39c009e80 con 0x7fe3ac0ab500 2023-10-18T10:25:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac1e5900 con 0x7fe3ac0ab500 2023-10-18T10:25:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3a1ffb640 1 -- 172.21.15.125:0/2859557417 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0106c0 con 0x7fe3ac0ab500 2023-10-18T10:25:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.289+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac1e5ec0 con 0x7fe3ac0ab500 2023-10-18T10:25:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.290+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac0690e0 con 0x7fe3ac0ab500 2023-10-18T10:25:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.291+0000 7fe3a1ffb640 1 -- 172.21.15.125:0/2859557417 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe39c010820 con 0x7fe3ac0ab500 2023-10-18T10:25:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.291+0000 7fe3a1ffb640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe39006ae40 0x7fe39006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.291+0000 7fe3a1ffb640 1 -- 172.21.15.125:0/2859557417 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe39c089db0 con 0x7fe3ac0ab500 2023-10-18T10:25:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.292+0000 7fe3b113a640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe39006ae40 0x7fe39006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.292+0000 7fe3b113a640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe39006ae40 0x7fe39006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800ba40 tx=0x7fe3a8005cc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:45.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.295+0000 7fe3a1ffb640 1 -- 172.21.15.125:0/2859557417 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe39c054b30 con 0x7fe3ac0ab500 2023-10-18T10:25:45.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.466+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3ac158f90 con 0x7fe3ac0ab500 2023-10-18T10:25:45.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.467+0000 7fe3a1ffb640 1 -- 172.21.15.125:0/2859557417 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fe39c021090 con 0x7fe3ac0ab500 2023-10-18T10:25:45.469 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:45.469 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:45.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe39006ae40 msgr2=0x7fe39006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe39006ae40 0x7fe39006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800ba40 tx=0x7fe3a8005cc0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 msgr2=0x7fe3ac1e4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 0x7fe3ac1e4f20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00d770 tx=0x7fe39c00dc30 comp rx=0 tx=0).stop 2023-10-18T10:25:45.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 shutdown_connections 2023-10-18T10:25:45.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe39006ae40 0x7fe39006d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3ac0c96b0 0x7fe3ac1e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3ac0ab500 0x7fe3ac1e4f20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 --2- 172.21.15.125:0/2859557417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3ac06af00 0x7fe3ac12e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 >> 172.21.15.125:0/2859557417 conn(0x7fe3ac069930 msgr2=0x7fe3ac0cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:45.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 shutdown_connections 2023-10-18T10:25:45.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.469+0000 7fe3b339e640 1 -- 172.21.15.125:0/2859557417 wait complete. 2023-10-18T10:25:45.481 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:45.482 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:45.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.585+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3724687220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0c9dc0 msgr2=0x7fd7bc154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.585+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3724687220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0c9dc0 0x7fd7bc154af0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0ca460 tx=0x7fd7b402c110 comp rx=0 tx=0).stop 2023-10-18T10:25:45.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.585+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3724687220 shutdown_connections 2023-10-18T10:25:45.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.585+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3724687220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0c9dc0 0x7fd7bc154af0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.585+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3724687220 >> v2:172.21.15.139:3300/0 conn(0x7fd7bc0b1920 0x7fd7bc06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.585+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3724687220 >> 172.21.15.125:0/3724687220 conn(0x7fd7bc069930 msgr2=0x7fd7bc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:45.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.585+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3724687220 shutdown_connections 2023-10-18T10:25:45.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.585+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3724687220 wait complete. 2023-10-18T10:25:45.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.586+0000 7fd7c0c90640 1 Processor -- start 2023-10-18T10:25:45.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.586+0000 7fd7c0c90640 1 -- start start 2023-10-18T10:25:45.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7c0c90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7bc06ffb0 0x7fd7bc149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7c0c90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0b1920 0x7fd7bc155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7c0c90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7bc0c9dc0 0x7fd7bc157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7c0c90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0ac7c0 con 0x7fd7bc0b1920 2023-10-18T10:25:45.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7c0c90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc156cd0 con 0x7fd7bc0c9dc0 2023-10-18T10:25:45.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7c0c90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1497d0 con 0x7fd7bc06ffb0 2023-10-18T10:25:45.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0b1920 0x7fd7bc155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0b1920 0x7fd7bc155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58468/0 (socket says 172.21.15.125:58468) 2023-10-18T10:25:45.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 -- 172.21.15.125:0/3076510781 learned_addr learned my addr 172.21.15.125:0/3076510781 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:45.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7bad9d640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7bc0c9dc0 0x7fd7bc157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 -- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7bc06ffb0 msgr2=0x7fd7bc149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7ba59c640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7bc06ffb0 0x7fd7bc149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7bc06ffb0 0x7fd7bc149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 -- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7bc0c9dc0 msgr2=0x7fd7bc157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7bc0c9dc0 0x7fd7bc157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 -- 172.21.15.125:0/3076510781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4009840 con 0x7fd7bc0b1920 2023-10-18T10:25:45.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7bad9d640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7bc0c9dc0 0x7fd7bc157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:45.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7ba59c640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7bc06ffb0 0x7fd7bc149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:45.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7b9d9b640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0b1920 0x7fd7bc155f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000c4d0 tx=0x7fd7b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:45.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7aaffd640 1 -- 172.21.15.125:0/3076510781 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0012830 con 0x7fd7bc0b1920 2023-10-18T10:25:45.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7aaffd640 1 -- 172.21.15.125:0/3076510781 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd7b0012a80 con 0x7fd7bc0b1920 2023-10-18T10:25:45.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc1569f0 con 0x7fd7bc0b1920 2023-10-18T10:25:45.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7aaffd640 1 -- 172.21.15.125:0/3076510781 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0012e30 con 0x7fd7bc0b1920 2023-10-18T10:25:45.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.587+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc0ac3e0 con 0x7fd7bc0b1920 2023-10-18T10:25:45.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.589+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc0ac900 con 0x7fd7bc0b1920 2023-10-18T10:25:45.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.589+0000 7fd7aaffd640 1 -- 172.21.15.125:0/3076510781 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd7b001a3b0 con 0x7fd7bc0b1920 2023-10-18T10:25:45.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.589+0000 7fd7aaffd640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd79c06ae70 0x7fd79c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.589+0000 7fd7aaffd640 1 -- 172.21.15.125:0/3076510781 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd7b0089800 con 0x7fd7bc0b1920 2023-10-18T10:25:45.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.589+0000 7fd7ba59c640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd79c06ae70 0x7fd79c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.590+0000 7fd7ba59c640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd79c06ae70 0x7fd79c06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0058c0 tx=0x7fd7ac005d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:45.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.593+0000 7fd7aaffd640 1 -- 172.21.15.125:0/3076510781 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd7bc0ac900 con 0x7fd7bc0b1920 2023-10-18T10:25:45.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.758+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7bc0ac900 con 0x7fd7bc0b1920 2023-10-18T10:25:45.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.759+0000 7fd7aaffd640 1 -- 172.21.15.125:0/3076510781 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7bc0ac900 con 0x7fd7bc0b1920 2023-10-18T10:25:45.761 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:45.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.760+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd79c06ae70 msgr2=0x7fd79c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.760+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd79c06ae70 0x7fd79c06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0058c0 tx=0x7fd7ac005d30 comp rx=0 tx=0).stop 2023-10-18T10:25:45.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.760+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0b1920 msgr2=0x7fd7bc155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.760+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0b1920 0x7fd7bc155f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000c4d0 tx=0x7fd7b000c990 comp rx=0 tx=0).stop 2023-10-18T10:25:45.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.761+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 shutdown_connections 2023-10-18T10:25:45.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.761+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd79c06ae70 0x7fd79c06d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.761+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7bc0c9dc0 0x7fd7bc157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.761+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7bc0b1920 0x7fd7bc155f30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.761+0000 7fd7c0c90640 1 --2- 172.21.15.125:0/3076510781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7bc06ffb0 0x7fd7bc149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.761+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 >> 172.21.15.125:0/3076510781 conn(0x7fd7bc069930 msgr2=0x7fd7bc0c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:45.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.761+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 shutdown_connections 2023-10-18T10:25:45.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.761+0000 7fd7c0c90640 1 -- 172.21.15.125:0/3076510781 wait complete. 2023-10-18T10:25:45.773 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:45.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.876+0000 7fe581e7c640 1 -- 172.21.15.125:0/3653776737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 msgr2=0x7fe57c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.876+0000 7fe581e7c640 1 --2- 172.21.15.125:0/3653776737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 0x7fe57c06d260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe56c005fd0 tx=0x7fe56c02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.876+0000 7fe581e7c640 1 -- 172.21.15.125:0/3653776737 shutdown_connections 2023-10-18T10:25:45.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.876+0000 7fe581e7c640 1 --2- 172.21.15.125:0/3653776737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57c0cdb30 0x7fe57c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.876+0000 7fe581e7c640 1 --2- 172.21.15.125:0/3653776737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 0x7fe57c06d260 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.876+0000 7fe581e7c640 1 -- 172.21.15.125:0/3653776737 >> 172.21.15.125:0/3653776737 conn(0x7fe57c0b0670 msgr2=0x7fe57c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:45.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.876+0000 7fe581e7c640 1 -- 172.21.15.125:0/3653776737 shutdown_connections 2023-10-18T10:25:45.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.876+0000 7fe581e7c640 1 -- 172.21.15.125:0/3653776737 wait complete. 2023-10-18T10:25:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.877+0000 7fe581e7c640 1 Processor -- start 2023-10-18T10:25:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.877+0000 7fe581e7c640 1 -- start start 2023-10-18T10:25:45.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe581e7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57c0cdb30 0x7fe57c0ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe581e7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 0x7fe57c0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe581e7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57c15ccf0 0x7fe57c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe581e7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0b31d0 con 0x7fe57c159060 2023-10-18T10:25:45.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe581e7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0b3430 con 0x7fe57c15ccf0 2023-10-18T10:25:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe581e7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0b3710 con 0x7fe57c0cdb30 2023-10-18T10:25:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 0x7fe57c0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 0x7fe57c0b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58476/0 (socket says 172.21.15.125:58476) 2023-10-18T10:25:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57affd640 1 -- 172.21.15.125:0/2840661769 learned_addr learned my addr 172.21.15.125:0/2840661769 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57b7fe640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57c0cdb30 0x7fe57c0ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57bfff640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57c15ccf0 0x7fe57c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57affd640 1 -- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57c0cdb30 msgr2=0x7fe57c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57affd640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57c0cdb30 0x7fe57c0ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57affd640 1 -- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57c15ccf0 msgr2=0x7fe57c0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57affd640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57c15ccf0 0x7fe57c0c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57affd640 1 -- 172.21.15.125:0/2840661769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c009560 con 0x7fe57c159060 2023-10-18T10:25:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57b7fe640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57c0cdb30 0x7fe57c0ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.878+0000 7fe57bfff640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57c15ccf0 0x7fe57c0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.879+0000 7fe57affd640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 0x7fe57c0b2c90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c890 tx=0x7fe57000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.879+0000 7fe563fff640 1 -- 172.21.15.125:0/2840661769 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570004220 con 0x7fe57c159060 2023-10-18T10:25:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.879+0000 7fe563fff640 1 -- 172.21.15.125:0/2840661769 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe570004470 con 0x7fe57c159060 2023-10-18T10:25:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.879+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c0b1760 con 0x7fe57c159060 2023-10-18T10:25:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.879+0000 7fe563fff640 1 -- 172.21.15.125:0/2840661769 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570007c10 con 0x7fe57c159060 2023-10-18T10:25:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.879+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c0b1d20 con 0x7fe57c159060 2023-10-18T10:25:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.880+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c0675f0 con 0x7fe57c159060 2023-10-18T10:25:45.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.880+0000 7fe563fff640 1 -- 172.21.15.125:0/2840661769 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe570005020 con 0x7fe57c159060 2023-10-18T10:25:45.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.881+0000 7fe563fff640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe56806ae40 0x7fe56806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:45.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.881+0000 7fe563fff640 1 -- 172.21.15.125:0/2840661769 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe570089f60 con 0x7fe57c159060 2023-10-18T10:25:45.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.881+0000 7fe57b7fe640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe56806ae40 0x7fe56806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:45.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.881+0000 7fe57b7fe640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe56806ae40 0x7fe56806d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe56c035000 tx=0x7fe56c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:45.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:45.885+0000 7fe563fff640 1 -- 172.21.15.125:0/2840661769 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe570054c50 con 0x7fe57c159060 2023-10-18T10:25:46.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.054+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe57c0cc4d0 con 0x7fe57c159060 2023-10-18T10:25:46.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.055+0000 7fe563fff640 1 -- 172.21.15.125:0/2840661769 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe57001d0a0 con 0x7fe57c159060 2023-10-18T10:25:46.057 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:46.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe56806ae40 msgr2=0x7fe56806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe56806ae40 0x7fe56806d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe56c035000 tx=0x7fe56c004730 comp rx=0 tx=0).stop 2023-10-18T10:25:46.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 msgr2=0x7fe57c0b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 0x7fe57c0b2c90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c890 tx=0x7fe57000cd50 comp rx=0 tx=0).stop 2023-10-18T10:25:46.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 shutdown_connections 2023-10-18T10:25:46.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe56806ae40 0x7fe56806d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57c15ccf0 0x7fe57c0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe57c159060 0x7fe57c0b2c90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 --2- 172.21.15.125:0/2840661769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57c0cdb30 0x7fe57c0ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 >> 172.21.15.125:0/2840661769 conn(0x7fe57c0b0670 msgr2=0x7fe57c0ac5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 shutdown_connections 2023-10-18T10:25:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.056+0000 7fe581e7c640 1 -- 172.21.15.125:0/2840661769 wait complete. 2023-10-18T10:25:46.069 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:46.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 -- 172.21.15.125:0/504138552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc39006ffb0 msgr2=0x7fc3900c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 --2- 172.21.15.125:0/504138552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc39006ffb0 0x7fc3900c96b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc3900ca460 tx=0x7fc38402bda0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 -- 172.21.15.125:0/504138552 shutdown_connections 2023-10-18T10:25:46.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 --2- 172.21.15.125:0/504138552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc39006ffb0 0x7fc3900c96b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 --2- 172.21.15.125:0/504138552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3900c9bf0 0x7fc390154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 --2- 172.21.15.125:0/504138552 >> v2:172.21.15.139:3300/0 conn(0x7fc3900b1920 0x7fc39006fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 -- 172.21.15.125:0/504138552 >> 172.21.15.125:0/504138552 conn(0x7fc390069930 msgr2=0x7fc390069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:46.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 -- 172.21.15.125:0/504138552 shutdown_connections 2023-10-18T10:25:46.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.173+0000 7fc3949ce640 1 -- 172.21.15.125:0/504138552 wait complete. 2023-10-18T10:25:46.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.174+0000 7fc3949ce640 1 Processor -- start 2023-10-18T10:25:46.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.174+0000 7fc3949ce640 1 -- start start 2023-10-18T10:25:46.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc3949ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc39006ffb0 0x7fc3901f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc3949ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3900b1920 0x7fc3900ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc3949ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3900c9bf0 0x7fc39015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc3949ce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900addb0 con 0x7fc3900c9bf0 2023-10-18T10:25:46.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc3949ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900ae240 con 0x7fc3900b1920 2023-10-18T10:25:46.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc3949ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900ad1a0 con 0x7fc39006ffb0 2023-10-18T10:25:46.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc38ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3900c9bf0 0x7fc39015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc38ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3900c9bf0 0x7fc39015c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58500/0 (socket says 172.21.15.125:58500) 2023-10-18T10:25:46.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc38ed9d640 1 -- 172.21.15.125:0/2608010686 learned_addr learned my addr 172.21.15.125:0/2608010686 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:46.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc38dd9b640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3900b1920 0x7fc3900ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.175+0000 7fc38e59c640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc39006ffb0 0x7fc3901f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc38ed9d640 1 -- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc39006ffb0 msgr2=0x7fc3901f0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc38ed9d640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc39006ffb0 0x7fc3901f0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc38ed9d640 1 -- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3900b1920 msgr2=0x7fc3900ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc38ed9d640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3900b1920 0x7fc3900ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc38ed9d640 1 -- 172.21.15.125:0/2608010686 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc384009560 con 0x7fc3900c9bf0 2023-10-18T10:25:46.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc38dd9b640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3900b1920 0x7fc3900ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:46.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc38ed9d640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3900c9bf0 0x7fc39015c790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c8f0 tx=0x7fc38800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:46.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc37effd640 1 -- 172.21.15.125:0/2608010686 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388007bb0 con 0x7fc3900c9bf0 2023-10-18T10:25:46.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc37effd640 1 -- 172.21.15.125:0/2608010686 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc388007e00 con 0x7fc3900c9bf0 2023-10-18T10:25:46.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc37effd640 1 -- 172.21.15.125:0/2608010686 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388005280 con 0x7fc3900c9bf0 2023-10-18T10:25:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3900ad480 con 0x7fc3900c9bf0 2023-10-18T10:25:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.176+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3900ada40 con 0x7fc3900c9bf0 2023-10-18T10:25:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.177+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3900adf20 con 0x7fc3900c9bf0 2023-10-18T10:25:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.177+0000 7fc37effd640 1 -- 172.21.15.125:0/2608010686 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc3880053e0 con 0x7fc3900c9bf0 2023-10-18T10:25:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.178+0000 7fc37effd640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc37006ae40 0x7fc37006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.178+0000 7fc37effd640 1 -- 172.21.15.125:0/2608010686 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc38808a1e0 con 0x7fc3900c9bf0 2023-10-18T10:25:46.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.178+0000 7fc38e59c640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc37006ae40 0x7fc37006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.178+0000 7fc38e59c640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc37006ae40 0x7fc37006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc380005c40 tx=0x7fc38000a780 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.182+0000 7fc37effd640 1 -- 172.21.15.125:0/2608010686 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc388054e30 con 0x7fc3900c9bf0 2023-10-18T10:25:46.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.346+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3900b1460 con 0x7fc3900c9bf0 2023-10-18T10:25:46.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.346+0000 7fc37effd640 1 -- 172.21.15.125:0/2608010686 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc38801d0a0 con 0x7fc3900c9bf0 2023-10-18T10:25:46.348 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:46.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc37006ae40 msgr2=0x7fc37006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc37006ae40 0x7fc37006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc380005c40 tx=0x7fc38000a780 comp rx=0 tx=0).stop 2023-10-18T10:25:46.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3900c9bf0 msgr2=0x7fc39015c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3900c9bf0 0x7fc39015c790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c8f0 tx=0x7fc38800cdb0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 shutdown_connections 2023-10-18T10:25:46.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc37006ae40 0x7fc37006d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3900b1920 0x7fc3900ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3900c9bf0 0x7fc39015c790 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 --2- 172.21.15.125:0/2608010686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc39006ffb0 0x7fc3901f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 >> 172.21.15.125:0/2608010686 conn(0x7fc390069930 msgr2=0x7fc390153f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:46.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 shutdown_connections 2023-10-18T10:25:46.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.348+0000 7fc3949ce640 1 -- 172.21.15.125:0/2608010686 wait complete. 2023-10-18T10:25:46.360 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:46.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.464+0000 7f83bc044640 1 -- 172.21.15.125:0/2291672457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40afa20 msgr2=0x7f83b40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.464+0000 7f83bc044640 1 --2- 172.21.15.125:0/2291672457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40afa20 0x7f83b40705a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f83a4005fd0 tx=0x7f83a402be10 comp rx=0 tx=0).stop 2023-10-18T10:25:46.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.465+0000 7f83bc044640 1 -- 172.21.15.125:0/2291672457 shutdown_connections 2023-10-18T10:25:46.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.465+0000 7f83bc044640 1 --2- 172.21.15.125:0/2291672457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40afa20 0x7f83b40705a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.465+0000 7f83bc044640 1 --2- 172.21.15.125:0/2291672457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83b40cbd20 0x7f83b4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.465+0000 7f83bc044640 1 -- 172.21.15.125:0/2291672457 >> 172.21.15.125:0/2291672457 conn(0x7f83b4069930 msgr2=0x7f83b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:46.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.465+0000 7f83bc044640 1 -- 172.21.15.125:0/2291672457 shutdown_connections 2023-10-18T10:25:46.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.465+0000 7f83bc044640 1 -- 172.21.15.125:0/2291672457 wait complete. 2023-10-18T10:25:46.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.466+0000 7f83bc044640 1 Processor -- start 2023-10-18T10:25:46.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.466+0000 7f83bc044640 1 -- start start 2023-10-18T10:25:46.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.466+0000 7f83bc044640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b40afa20 0x7f83b413fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.466+0000 7f83bc044640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40b0530 0x7f83b414b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.466+0000 7f83bc044640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83b40cbd20 0x7f83b41ef410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.466+0000 7f83bc044640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b41421c0 con 0x7f83b40b0530 2023-10-18T10:25:46.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.466+0000 7f83bc044640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b41424a0 con 0x7f83b40afa20 2023-10-18T10:25:46.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.466+0000 7f83bc044640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4142780 con 0x7f83b40cbd20 2023-10-18T10:25:46.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40b0530 0x7f83b414b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40b0530 0x7f83b414b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58524/0 (socket says 172.21.15.125:58524) 2023-10-18T10:25:46.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 -- 172.21.15.125:0/4230870210 learned_addr learned my addr 172.21.15.125:0/4230870210 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:46.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b9de0640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b40afa20 0x7f83b413fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83ba5e1640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83b40cbd20 0x7f83b41ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 -- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83b40cbd20 msgr2=0x7f83b41ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83b40cbd20 0x7f83b41ef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 -- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b40afa20 msgr2=0x7f83b413fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b40afa20 0x7f83b413fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 -- 172.21.15.125:0/4230870210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4009560 con 0x7f83b40b0530 2023-10-18T10:25:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83b95df640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40b0530 0x7f83b414b4b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f83a4006010 tx=0x7f83a402dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83aa7fc640 1 -- 172.21.15.125:0/4230870210 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a40035f0 con 0x7f83b40b0530 2023-10-18T10:25:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83aa7fc640 1 -- 172.21.15.125:0/4230870210 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83a4003840 con 0x7f83b40b0530 2023-10-18T10:25:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83aa7fc640 1 -- 172.21.15.125:0/4230870210 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4003c50 con 0x7f83b40b0530 2023-10-18T10:25:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b4070cf0 con 0x7f83b40b0530 2023-10-18T10:25:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.467+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b4142d40 con 0x7f83b40b0530 2023-10-18T10:25:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.468+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b40690e0 con 0x7f83b40b0530 2023-10-18T10:25:46.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.468+0000 7f83aa7fc640 1 -- 172.21.15.125:0/4230870210 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f83a403e450 con 0x7f83b40b0530 2023-10-18T10:25:46.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.469+0000 7f83aa7fc640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f839806ae40 0x7f839806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.469+0000 7f83aa7fc640 1 -- 172.21.15.125:0/4230870210 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f83a40ac630 con 0x7f83b40b0530 2023-10-18T10:25:46.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.469+0000 7f83b9de0640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f839806ae40 0x7f839806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.470+0000 7f83b9de0640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f839806ae40 0x7f839806d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005fd0 tx=0x7f83b0005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:46.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.473+0000 7f83aa7fc640 1 -- 172.21.15.125:0/4230870210 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83a40774f0 con 0x7f83b40b0530 2023-10-18T10:25:46.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.639+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f83b40cc1e0 con 0x7f83b40b0530 2023-10-18T10:25:46.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.640+0000 7f83aa7fc640 1 -- 172.21.15.125:0/4230870210 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f83a4003db0 con 0x7f83b40b0530 2023-10-18T10:25:46.642 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:46.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f839806ae40 msgr2=0x7f839806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f839806ae40 0x7f839806d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005fd0 tx=0x7f83b0005850 comp rx=0 tx=0).stop 2023-10-18T10:25:46.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40b0530 msgr2=0x7f83b414b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40b0530 0x7f83b414b4b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f83a4006010 tx=0x7f83a402dbb0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 shutdown_connections 2023-10-18T10:25:46.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f839806ae40 0x7f839806d300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83b40cbd20 0x7f83b41ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83b40b0530 0x7f83b414b4b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 --2- 172.21.15.125:0/4230870210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b40afa20 0x7f83b413fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 >> 172.21.15.125:0/4230870210 conn(0x7f83b4069930 msgr2=0x7f83b40ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:46.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 shutdown_connections 2023-10-18T10:25:46.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.642+0000 7f83bc044640 1 -- 172.21.15.125:0/4230870210 wait complete. 2023-10-18T10:25:46.654 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-18T10:25:46.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 -- 172.21.15.125:0/2204743047 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 msgr2=0x7fdf140b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 --2- 172.21.15.125:0/2204743047 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 0x7fdf140b0650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdf1408a810 tx=0x7fdf0c02c1d0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 -- 172.21.15.125:0/2204743047 shutdown_connections 2023-10-18T10:25:46.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 --2- 172.21.15.125:0/2204743047 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 0x7fdf140b0650 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 --2- 172.21.15.125:0/2204743047 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf1415b2a0 0x7fdf140cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 --2- 172.21.15.125:0/2204743047 >> v2:172.21.15.139:3300/0 conn(0x7fdf140cf340 0x7fdf140d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 -- 172.21.15.125:0/2204743047 >> 172.21.15.125:0/2204743047 conn(0x7fdf140d0d60 msgr2=0x7fdf140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:46.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 -- 172.21.15.125:0/2204743047 shutdown_connections 2023-10-18T10:25:46.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.755+0000 7fdf19e10640 1 -- 172.21.15.125:0/2204743047 wait complete. 2023-10-18T10:25:46.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.756+0000 7fdf19e10640 1 Processor -- start 2023-10-18T10:25:46.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.756+0000 7fdf19e10640 1 -- start start 2023-10-18T10:25:46.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf19e10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 0x7fdf14159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf19e10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf140cf340 0x7fdf141e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf19e10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1415b2a0 0x7fdf140cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf19e10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14159950 con 0x7fdf140ceb40 2023-10-18T10:25:46.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf19e10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140b5270 con 0x7fdf1415b2a0 2023-10-18T10:25:46.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf19e10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf141f5490 con 0x7fdf140cf340 2023-10-18T10:25:46.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 0x7fdf14159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 0x7fdf14159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:58538/0 (socket says 172.21.15.125:58538) 2023-10-18T10:25:46.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf137fe640 1 -- 172.21.15.125:0/110841203 learned_addr learned my addr 172.21.15.125:0/110841203 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:46.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf137fe640 1 -- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf140cf340 msgr2=0x7fdf141e5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf137fe640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf140cf340 0x7fdf141e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf137fe640 1 -- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1415b2a0 msgr2=0x7fdf140cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:46.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf137fe640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1415b2a0 0x7fdf140cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:46.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.757+0000 7fdf137fe640 1 -- 172.21.15.125:0/110841203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c009900 con 0x7fdf140ceb40 2023-10-18T10:25:46.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.758+0000 7fdf137fe640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 0x7fdf14159410 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c890 tx=0x7fdf0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:46.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.758+0000 7fdefbfff640 1 -- 172.21.15.125:0/110841203 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004220 con 0x7fdf140ceb40 2023-10-18T10:25:46.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.758+0000 7fdefbfff640 1 -- 172.21.15.125:0/110841203 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf04004470 con 0x7fdf140ceb40 2023-10-18T10:25:46.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.758+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf141f5770 con 0x7fdf140ceb40 2023-10-18T10:25:46.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.758+0000 7fdefbfff640 1 -- 172.21.15.125:0/110841203 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04007bf0 con 0x7fdf140ceb40 2023-10-18T10:25:46.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.758+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf141f5d30 con 0x7fdf140ceb40 2023-10-18T10:25:46.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.759+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf140690e0 con 0x7fdf140ceb40 2023-10-18T10:25:46.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.759+0000 7fdefbfff640 1 -- 172.21.15.125:0/110841203 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf04007d50 con 0x7fdf140ceb40 2023-10-18T10:25:46.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.759+0000 7fdefbfff640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdf0006ae40 0x7fdf0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:46.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.760+0000 7fdefbfff640 1 -- 172.21.15.125:0/110841203 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdf04089f50 con 0x7fdf140ceb40 2023-10-18T10:25:46.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.760+0000 7fdf12ffd640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdf0006ae40 0x7fdf0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:46.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.760+0000 7fdf12ffd640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdf0006ae40 0x7fdf0006d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdf141e5b60 tx=0x7fdf08006c10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:46.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.764+0000 7fdefbfff640 1 -- 172.21.15.125:0/110841203 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf04054cd0 con 0x7fdf140ceb40 2023-10-18T10:25:46.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:46.993+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fdf140d01d0 con 0x7fdf140ceb40 2023-10-18T10:25:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.381+0000 7fdefbfff640 1 -- 172.21.15.125:0/110841203 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdf0401d020 con 0x7fdf140ceb40 2023-10-18T10:25:47.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdf0006ae40 msgr2=0x7fdf0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdf0006ae40 0x7fdf0006d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdf141e5b60 tx=0x7fdf08006c10 comp rx=0 tx=0).stop 2023-10-18T10:25:47.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 msgr2=0x7fdf14159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 0x7fdf14159410 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c890 tx=0x7fdf0400cd50 comp rx=0 tx=0).stop 2023-10-18T10:25:47.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 shutdown_connections 2023-10-18T10:25:47.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdf0006ae40 0x7fdf0006d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf1415b2a0 0x7fdf140cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf140cf340 0x7fdf141e5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 --2- 172.21.15.125:0/110841203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf140ceb40 0x7fdf14159410 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.383+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 >> 172.21.15.125:0/110841203 conn(0x7fdf140d0d60 msgr2=0x7fdf1415a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.384+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 shutdown_connections 2023-10-18T10:25:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.384+0000 7fdf19e10640 1 -- 172.21.15.125:0/110841203 wait complete. 2023-10-18T10:25:47.397 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T10:25:47.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.498+0000 7f6fb9133640 1 -- 172.21.15.125:0/1544997029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb40ce240 msgr2=0x7f6fb406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.498+0000 7f6fb9133640 1 --2- 172.21.15.125:0/1544997029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb40ce240 0x7f6fb406d260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009830 tx=0x7f6fa802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.498+0000 7f6fb9133640 1 -- 172.21.15.125:0/1544997029 shutdown_connections 2023-10-18T10:25:47.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.498+0000 7f6fb9133640 1 --2- 172.21.15.125:0/1544997029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fb415ccf0 0x7f6fb415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.498+0000 7f6fb9133640 1 --2- 172.21.15.125:0/1544997029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb40ce240 0x7f6fb406d260 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.498+0000 7f6fb9133640 1 -- 172.21.15.125:0/1544997029 >> 172.21.15.125:0/1544997029 conn(0x7f6fb40b0670 msgr2=0x7f6fb40b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:47.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.498+0000 7f6fb9133640 1 -- 172.21.15.125:0/1544997029 shutdown_connections 2023-10-18T10:25:47.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.498+0000 7f6fb9133640 1 -- 172.21.15.125:0/1544997029 wait complete. 2023-10-18T10:25:47.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.499+0000 7f6fb9133640 1 Processor -- start 2023-10-18T10:25:47.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.499+0000 7f6fb9133640 1 -- start start 2023-10-18T10:25:47.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb9133640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fb40ce240 0x7f6fb40c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb9133640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb4159060 0x7f6fb40c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb9133640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fb415ccf0 0x7f6fb4156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb9133640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41574f0 con 0x7f6fb4159060 2023-10-18T10:25:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb9133640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40c6260 con 0x7f6fb40ce240 2023-10-18T10:25:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb9133640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40c6540 con 0x7f6fb415ccf0 2023-10-18T10:25:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb4159060 0x7f6fb40c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb4159060 0x7f6fb40c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37626/0 (socket says 172.21.15.125:37626) 2023-10-18T10:25:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.500+0000 7f6fb259c640 1 -- 172.21.15.125:0/4294019389 learned_addr learned my addr 172.21.15.125:0/4294019389 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb259c640 1 -- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fb415ccf0 msgr2=0x7f6fb4156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb359e640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fb415ccf0 0x7f6fb4156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb2d9d640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fb40ce240 0x7f6fb40c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb259c640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fb415ccf0 0x7f6fb4156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb259c640 1 -- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fb40ce240 msgr2=0x7f6fb40c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb259c640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fb40ce240 0x7f6fb40c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb259c640 1 -- 172.21.15.125:0/4294019389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8009490 con 0x7f6fb4159060 2023-10-18T10:25:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb2d9d640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fb40ce240 0x7f6fb40c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb359e640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fb415ccf0 0x7f6fb4156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb259c640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb4159060 0x7f6fb40c7300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009780 tx=0x7f6fa80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6f9b7fe640 1 -- 172.21.15.125:0/4294019389 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8003ea0 con 0x7f6fb4159060 2023-10-18T10:25:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6f9b7fe640 1 -- 172.21.15.125:0/4294019389 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6fa8034550 con 0x7f6fb4159060 2023-10-18T10:25:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6f9b7fe640 1 -- 172.21.15.125:0/4294019389 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8034960 con 0x7f6fb4159060 2023-10-18T10:25:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb40c6820 con 0x7f6fb4159060 2023-10-18T10:25:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.501+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb40c6de0 con 0x7f6fb4159060 2023-10-18T10:25:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.502+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb40c6fd0 con 0x7f6fb4159060 2023-10-18T10:25:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.502+0000 7f6f9b7fe640 1 -- 172.21.15.125:0/4294019389 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6fa803b040 con 0x7f6fb4159060 2023-10-18T10:25:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.503+0000 7f6f9b7fe640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6fa006ae40 0x7f6fa006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.503+0000 7f6f9b7fe640 1 -- 172.21.15.125:0/4294019389 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6fa80abfc0 con 0x7f6fb4159060 2023-10-18T10:25:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.503+0000 7f6fb2d9d640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6fa006ae40 0x7f6fa006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.503+0000 7f6fb2d9d640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6fa006ae40 0x7f6fa006d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fac004a30 tx=0x7f6fac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:47.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.507+0000 7f6f9b7fe640 1 -- 172.21.15.125:0/4294019389 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6fa80deca0 con 0x7f6fb4159060 2023-10-18T10:25:47.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.674+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f6fb40675f0 con 0x7f6fb4159060 2023-10-18T10:25:47.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.675+0000 7f6f9b7fe640 1 -- 172.21.15.125:0/4294019389 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f6fa8039080 con 0x7f6fb4159060 2023-10-18T10:25:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T10:25:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T10:25:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T10:25:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T10:25:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T10:25:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T10:25:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T10:25:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T10:25:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T10:25:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T10:25:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T10:25:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T10:25:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T10:25:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T10:25:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T10:25:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T10:25:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T10:25:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T10:25:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T10:25:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T10:25:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T10:25:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T10:25:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T10:25:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T10:25:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T10:25:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T10:25:47.682 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T10:25:47.682 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T10:25:47.682 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T10:25:47.682 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T10:25:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6fa006ae40 msgr2=0x7f6fa006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6fa006ae40 0x7f6fa006d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fac004a30 tx=0x7f6fac009040 comp rx=0 tx=0).stop 2023-10-18T10:25:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb4159060 msgr2=0x7f6fb40c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb4159060 0x7f6fb40c7300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009780 tx=0x7f6fa80038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 shutdown_connections 2023-10-18T10:25:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6fa006ae40 0x7f6fa006d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fb415ccf0 0x7f6fb4156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fb4159060 0x7f6fb40c7300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 --2- 172.21.15.125:0/4294019389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fb40ce240 0x7f6fb40c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 >> 172.21.15.125:0/4294019389 conn(0x7f6fb40b0670 msgr2=0x7f6fb4158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:47.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 shutdown_connections 2023-10-18T10:25:47.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.677+0000 7f6fb9133640 1 -- 172.21.15.125:0/4294019389 wait complete. 2023-10-18T10:25:47.689 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:25:47.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 -- 172.21.15.125:0/3688119319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440ab500 msgr2=0x7fa2440ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 --2- 172.21.15.125:0/3688119319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440ab500 0x7fa2440ab990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa2440713a0 tx=0x7fa23c02bda0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 -- 172.21.15.125:0/3688119319 shutdown_connections 2023-10-18T10:25:47.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 --2- 172.21.15.125:0/3688119319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440ab500 0x7fa2440ab990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 --2- 172.21.15.125:0/3688119319 >> v2:172.21.15.139:3300/0 conn(0x7fa2440c96b0 0x7fa2440c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 --2- 172.21.15.125:0/3688119319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa24406af00 0x7fa24406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 -- 172.21.15.125:0/3688119319 >> 172.21.15.125:0/3688119319 conn(0x7fa244069930 msgr2=0x7fa244069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:47.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 -- 172.21.15.125:0/3688119319 shutdown_connections 2023-10-18T10:25:47.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.792+0000 7fa2491d7640 1 -- 172.21.15.125:0/3688119319 wait complete. 2023-10-18T10:25:47.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.793+0000 7fa2491d7640 1 Processor -- start 2023-10-18T10:25:47.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.793+0000 7fa2491d7640 1 -- start start 2023-10-18T10:25:47.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa2491d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa24406af00 0x7fa2440aa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:47.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa2491d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2440ab500 0x7fa24413a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:47.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa2491d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440c96b0 0x7fa2440a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:47.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa2491d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440ab350 con 0x7fa2440c96b0 2023-10-18T10:25:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa2491d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440aaab0 con 0x7fa24406af00 2023-10-18T10:25:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa2491d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440aad90 con 0x7fa2440ab500 2023-10-18T10:25:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa24359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440c96b0 0x7fa2440a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa24359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440c96b0 0x7fa2440a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37642/0 (socket says 172.21.15.125:37642) 2023-10-18T10:25:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa24359e640 1 -- 172.21.15.125:0/2227980527 learned_addr learned my addr 172.21.15.125:0/2227980527 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa242d9d640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa24406af00 0x7fa2440aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa24359e640 1 -- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2440ab500 msgr2=0x7fa24413a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa24359e640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2440ab500 0x7fa24413a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa24359e640 1 -- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa24406af00 msgr2=0x7fa2440aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa24359e640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa24406af00 0x7fa2440aa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.794+0000 7fa24359e640 1 -- 172.21.15.125:0/2227980527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c009560 con 0x7fa2440c96b0 2023-10-18T10:25:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.795+0000 7fa24359e640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440c96b0 0x7fa2440a6b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa23c036040 tx=0x7fa23c02dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.795+0000 7fa22b7fe640 1 -- 172.21.15.125:0/2227980527 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c005d50 con 0x7fa2440c96b0 2023-10-18T10:25:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.795+0000 7fa22b7fe640 1 -- 172.21.15.125:0/2227980527 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa23c0038f0 con 0x7fa2440c96b0 2023-10-18T10:25:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.795+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2440ab070 con 0x7fa2440c96b0 2023-10-18T10:25:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.795+0000 7fa22b7fe640 1 -- 172.21.15.125:0/2227980527 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c003d00 con 0x7fa2440c96b0 2023-10-18T10:25:47.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.795+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2440aa680 con 0x7fa2440c96b0 2023-10-18T10:25:47.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.796+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa244159020 con 0x7fa2440c96b0 2023-10-18T10:25:47.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.796+0000 7fa22b7fe640 1 -- 172.21.15.125:0/2227980527 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa23c03e450 con 0x7fa2440c96b0 2023-10-18T10:25:47.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.797+0000 7fa22b7fe640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa23006ae40 0x7fa23006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:47.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.797+0000 7fa22b7fe640 1 -- 172.21.15.125:0/2227980527 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa23c0ac140 con 0x7fa2440c96b0 2023-10-18T10:25:47.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.797+0000 7fa242d9d640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa23006ae40 0x7fa23006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:47.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.798+0000 7fa242d9d640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa23006ae40 0x7fa23006d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa234005c40 tx=0x7fa234005bd0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:47.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.801+0000 7fa22b7fe640 1 -- 172.21.15.125:0/2227980527 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa23c076ec0 con 0x7fa2440c96b0 2023-10-18T10:25:47.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.965+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa2440a6820 con 0x7fa2440c96b0 2023-10-18T10:25:47.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.969+0000 7fa22b7fe640 1 -- 172.21.15.125:0/2227980527 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa23c003e60 con 0x7fa2440c96b0 2023-10-18T10:25:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa23006ae40 msgr2=0x7fa23006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa23006ae40 0x7fa23006d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa234005c40 tx=0x7fa234005bd0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440c96b0 msgr2=0x7fa2440a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440c96b0 0x7fa2440a6b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa23c036040 tx=0x7fa23c02dcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 shutdown_connections 2023-10-18T10:25:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa23006ae40 0x7fa23006d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2440c96b0 0x7fa2440a6b00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2440ab500 0x7fa24413a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 --2- 172.21.15.125:0/2227980527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa24406af00 0x7fa2440aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 >> 172.21.15.125:0/2227980527 conn(0x7fa244069930 msgr2=0x7fa24413ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 shutdown_connections 2023-10-18T10:25:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:47.971+0000 7fa2491d7640 1 -- 172.21.15.125:0/2227980527 wait complete. 2023-10-18T10:25:47.983 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:25:47.984 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:25:48.021 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:25:48.072 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:25:48.072 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:25:48.073 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:48.088 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:25:48.088 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:48.143 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:48.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.287+0000 7fdcd9c61640 1 -- 172.21.15.125:0/3814942616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 msgr2=0x7fdcd4158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.287+0000 7fdcd9c61640 1 --2- 172.21.15.125:0/3814942616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 0x7fdcd4158b20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4005fd0 tx=0x7fdcc402bfc0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.287+0000 7fdcd9c61640 1 -- 172.21.15.125:0/3814942616 shutdown_connections 2023-10-18T10:25:48.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.287+0000 7fdcd9c61640 1 --2- 172.21.15.125:0/3814942616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 0x7fdcd4158b20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.287+0000 7fdcd9c61640 1 -- 172.21.15.125:0/3814942616 >> 172.21.15.125:0/3814942616 conn(0x7fdcd40af830 msgr2=0x7fdcd40afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:48.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.287+0000 7fdcd9c61640 1 -- 172.21.15.125:0/3814942616 shutdown_connections 2023-10-18T10:25:48.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.287+0000 7fdcd9c61640 1 -- 172.21.15.125:0/3814942616 wait complete. 2023-10-18T10:25:48.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.288+0000 7fdcd9c61640 1 Processor -- start 2023-10-18T10:25:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.288+0000 7fdcd9c61640 1 -- start start 2023-10-18T10:25:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd9c61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcd406d9a0 0x7fdcd41f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd9c61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcd4070130 0x7fdcd4150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd9c61640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 0x7fdcd4146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd9c61640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd4150770 con 0x7fdcd40b1870 2023-10-18T10:25:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd9c61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd4156540 con 0x7fdcd406d9a0 2023-10-18T10:25:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd9c61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd4150980 con 0x7fdcd4070130 2023-10-18T10:25:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 0x7fdcd4146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 0x7fdcd4146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37660/0 (socket says 172.21.15.125:37660) 2023-10-18T10:25:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd3fff640 1 -- 172.21.15.125:0/2687418930 learned_addr learned my addr 172.21.15.125:0/2687418930 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd37fe640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcd406d9a0 0x7fdcd41f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.289+0000 7fdcd2ffd640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcd4070130 0x7fdcd4150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd3fff640 1 -- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcd4070130 msgr2=0x7fdcd4150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd3fff640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcd4070130 0x7fdcd4150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd3fff640 1 -- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcd406d9a0 msgr2=0x7fdcd41f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd3fff640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcd406d9a0 0x7fdcd41f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd3fff640 1 -- 172.21.15.125:0/2687418930 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4009840 con 0x7fdcd40b1870 2023-10-18T10:25:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd37fe640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcd406d9a0 0x7fdcd41f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd3fff640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 0x7fdcd4146350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c390 tx=0x7fdccc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcbbfff640 1 -- 172.21.15.125:0/2687418930 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0035f0 con 0x7fdcd40b1870 2023-10-18T10:25:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcbbfff640 1 -- 172.21.15.125:0/2687418930 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdccc003750 con 0x7fdcd40b1870 2023-10-18T10:25:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd4155ef0 con 0x7fdcd40b1870 2023-10-18T10:25:48.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcbbfff640 1 -- 172.21.15.125:0/2687418930 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc003b60 con 0x7fdcd40b1870 2023-10-18T10:25:48.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.290+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd4150c60 con 0x7fdcd40b1870 2023-10-18T10:25:48.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.291+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd41566b0 con 0x7fdcd40b1870 2023-10-18T10:25:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.291+0000 7fdcbbfff640 1 -- 172.21.15.125:0/2687418930 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdccc003cc0 con 0x7fdcd40b1870 2023-10-18T10:25:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.292+0000 7fdcbbfff640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdcc006ae40 0x7fdcc006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.292+0000 7fdcbbfff640 1 -- 172.21.15.125:0/2687418930 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdccc0890f0 con 0x7fdcd40b1870 2023-10-18T10:25:48.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.292+0000 7fdcd37fe640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdcc006ae40 0x7fdcc006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.292+0000 7fdcd37fe640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdcc006ae40 0x7fdcc006d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4035000 tx=0x7fdcc4004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:48.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.296+0000 7fdcbbfff640 1 -- 172.21.15.125:0/2687418930 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdccc053e70 con 0x7fdcd40b1870 2023-10-18T10:25:48.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.466+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcd40690e0 con 0x7fdcd40b1870 2023-10-18T10:25:48.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.467+0000 7fdcbbfff640 1 -- 172.21.15.125:0/2687418930 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fdccc017090 con 0x7fdcd40b1870 2023-10-18T10:25:48.469 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:48.469 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:48.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdcc006ae40 msgr2=0x7fdcc006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdcc006ae40 0x7fdcc006d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4035000 tx=0x7fdcc4004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 msgr2=0x7fdcd4146350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 0x7fdcd4146350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c390 tx=0x7fdccc00c850 comp rx=0 tx=0).stop 2023-10-18T10:25:48.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 shutdown_connections 2023-10-18T10:25:48.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdcc006ae40 0x7fdcc006d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcd40b1870 0x7fdcd4146350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcd4070130 0x7fdcd4150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 --2- 172.21.15.125:0/2687418930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcd406d9a0 0x7fdcd41f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 >> 172.21.15.125:0/2687418930 conn(0x7fdcd40af830 msgr2=0x7fdcd40cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:48.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 shutdown_connections 2023-10-18T10:25:48.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.469+0000 7fdcd9c61640 1 -- 172.21.15.125:0/2687418930 wait complete. 2023-10-18T10:25:48.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:48.483 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:48.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.586+0000 7feb65b3c640 1 -- 172.21.15.125:0/1292937036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600c9dc0 msgr2=0x7feb6006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.586+0000 7feb65b3c640 1 --2- 172.21.15.125:0/1292937036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600c9dc0 0x7feb6006b2e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feb54009830 tx=0x7feb5402bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:48.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.586+0000 7feb65b3c640 1 -- 172.21.15.125:0/1292937036 shutdown_connections 2023-10-18T10:25:48.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.586+0000 7feb65b3c640 1 --2- 172.21.15.125:0/1292937036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600c9dc0 0x7feb6006b2e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.586+0000 7feb65b3c640 1 -- 172.21.15.125:0/1292937036 >> 172.21.15.125:0/1292937036 conn(0x7feb60069930 msgr2=0x7feb60069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:48.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.586+0000 7feb65b3c640 1 -- 172.21.15.125:0/1292937036 shutdown_connections 2023-10-18T10:25:48.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.586+0000 7feb65b3c640 1 -- 172.21.15.125:0/1292937036 wait complete. 2023-10-18T10:25:48.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.587+0000 7feb65b3c640 1 Processor -- start 2023-10-18T10:25:48.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.587+0000 7feb65b3c640 1 -- start start 2023-10-18T10:25:48.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb65b3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006af00 0x7feb6013d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb65b3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600ab500 0x7feb600a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb65b3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600c9dc0 0x7feb6014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb65b3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600aaad0 con 0x7feb600ab500 2023-10-18T10:25:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb65b3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600aa640 con 0x7feb6006af00 2023-10-18T10:25:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb65b3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600a9a30 con 0x7feb600c9dc0 2023-10-18T10:25:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600ab500 0x7feb600a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600ab500 0x7feb600a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37690/0 (socket says 172.21.15.125:37690) 2023-10-18T10:25:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 -- 172.21.15.125:0/758089073 learned_addr learned my addr 172.21.15.125:0/758089073 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5f7fe640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006af00 0x7feb6013d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5ffff640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600c9dc0 0x7feb6014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 -- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600c9dc0 msgr2=0x7feb6014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600c9dc0 0x7feb6014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 -- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006af00 msgr2=0x7feb6013d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006af00 0x7feb6013d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 -- 172.21.15.125:0/758089073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54009490 con 0x7feb600ab500 2023-10-18T10:25:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5f7fe640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006af00 0x7feb6013d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.588+0000 7feb5effd640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600ab500 0x7feb600a6720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feb54000c00 tx=0x7feb540038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.589+0000 7feb47fff640 1 -- 172.21.15.125:0/758089073 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54003ea0 con 0x7feb600ab500 2023-10-18T10:25:48.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.589+0000 7feb47fff640 1 -- 172.21.15.125:0/758089073 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb54034440 con 0x7feb600ab500 2023-10-18T10:25:48.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.589+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb60159a20 con 0x7feb600ab500 2023-10-18T10:25:48.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.589+0000 7feb47fff640 1 -- 172.21.15.125:0/758089073 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54034850 con 0x7feb600ab500 2023-10-18T10:25:48.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.589+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb600a9ff0 con 0x7feb600ab500 2023-10-18T10:25:48.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.590+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb600aa7d0 con 0x7feb600ab500 2023-10-18T10:25:48.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.590+0000 7feb47fff640 1 -- 172.21.15.125:0/758089073 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feb540349b0 con 0x7feb600ab500 2023-10-18T10:25:48.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.590+0000 7feb47fff640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7feb4c06ae70 0x7feb4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.591+0000 7feb47fff640 1 -- 172.21.15.125:0/758089073 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feb540ac1e0 con 0x7feb600ab500 2023-10-18T10:25:48.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.591+0000 7feb5f7fe640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7feb4c06ae70 0x7feb4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.591+0000 7feb5f7fe640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7feb4c06ae70 0x7feb4c06d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feb6006b620 tx=0x7feb50009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:48.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.595+0000 7feb47fff640 1 -- 172.21.15.125:0/758089073 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb54076f60 con 0x7feb600ab500 2023-10-18T10:25:48.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.759+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb600690e0 con 0x7feb600ab500 2023-10-18T10:25:48.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.759+0000 7feb47fff640 1 -- 172.21.15.125:0/758089073 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb5407a580 con 0x7feb600ab500 2023-10-18T10:25:48.762 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:48.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7feb4c06ae70 msgr2=0x7feb4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7feb4c06ae70 0x7feb4c06d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feb6006b620 tx=0x7feb50009040 comp rx=0 tx=0).stop 2023-10-18T10:25:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600ab500 msgr2=0x7feb600a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600ab500 0x7feb600a6720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feb54000c00 tx=0x7feb540038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 shutdown_connections 2023-10-18T10:25:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7feb4c06ae70 0x7feb4c06d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600c9dc0 0x7feb6014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb600ab500 0x7feb600a6720 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 --2- 172.21.15.125:0/758089073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006af00 0x7feb6013d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 >> 172.21.15.125:0/758089073 conn(0x7feb60069930 msgr2=0x7feb600ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 shutdown_connections 2023-10-18T10:25:48.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.761+0000 7feb65b3c640 1 -- 172.21.15.125:0/758089073 wait complete. 2023-10-18T10:25:48.774 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:48.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.877+0000 7f3d94210640 1 -- 172.21.15.125:0/3999757813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0cf340 msgr2=0x7f3d8c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.877+0000 7f3d94210640 1 --2- 172.21.15.125:0/3999757813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0cf340 0x7f3d8c0d1910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3d84005fd0 tx=0x7f3d8402bd90 comp rx=0 tx=0).stop 2023-10-18T10:25:48.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.877+0000 7f3d94210640 1 -- 172.21.15.125:0/3999757813 shutdown_connections 2023-10-18T10:25:48.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.877+0000 7f3d94210640 1 --2- 172.21.15.125:0/3999757813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8c15b2a0 0x7f3d8c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.877+0000 7f3d94210640 1 --2- 172.21.15.125:0/3999757813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0cf340 0x7f3d8c0d1910 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.877+0000 7f3d94210640 1 -- 172.21.15.125:0/3999757813 >> 172.21.15.125:0/3999757813 conn(0x7f3d8c0d0d60 msgr2=0x7f3d8c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:48.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.877+0000 7f3d94210640 1 -- 172.21.15.125:0/3999757813 shutdown_connections 2023-10-18T10:25:48.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.877+0000 7f3d94210640 1 -- 172.21.15.125:0/3999757813 wait complete. 2023-10-18T10:25:48.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.878+0000 7f3d94210640 1 Processor -- start 2023-10-18T10:25:48.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.878+0000 7f3d94210640 1 -- start start 2023-10-18T10:25:48.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d94210640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0ceb40 0x7f3d8c1e5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d94210640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8c0cf340 0x7f3d8c0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d94210640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8c15b2a0 0x7f3d8c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d94210640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c1e67f0 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d94210640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c1e39a0 con 0x7f3d8c0cf340 2023-10-18T10:25:48.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d94210640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c1f4df0 con 0x7f3d8c15b2a0 2023-10-18T10:25:48.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d91fac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0ceb40 0x7f3d8c1e5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d91fac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0ceb40 0x7f3d8c1e5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37704/0 (socket says 172.21.15.125:37704) 2023-10-18T10:25:48.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d91fac640 1 -- 172.21.15.125:0/4233861026 learned_addr learned my addr 172.21.15.125:0/4233861026 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:48.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d91fac640 1 -- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8c15b2a0 msgr2=0x7f3d8c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d91fac640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8c15b2a0 0x7f3d8c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d91fac640 1 -- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8c0cf340 msgr2=0x7f3d8c0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:48.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d91fac640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8c0cf340 0x7f3d8c0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:48.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.879+0000 7f3d91fac640 1 -- 172.21.15.125:0/4233861026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84009560 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.880+0000 7f3d91fac640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0ceb40 0x7f3d8c1e5390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3d840099f0 tx=0x7f3d8402db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:48.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.880+0000 7f3d827fc640 1 -- 172.21.15.125:0/4233861026 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84003800 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.880+0000 7f3d827fc640 1 -- 172.21.15.125:0/4233861026 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d84003a50 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.880+0000 7f3d827fc640 1 -- 172.21.15.125:0/4233861026 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84003e00 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.880+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c1f50d0 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.880+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c1f5690 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.881+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8c0690e0 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.881+0000 7f3d827fc640 1 -- 172.21.15.125:0/4233861026 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3d8403e450 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.882+0000 7f3d827fc640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d7006ae70 0x7f3d7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:48.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.882+0000 7f3d827fc640 1 -- 172.21.15.125:0/4233861026 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3d840ac420 con 0x7f3d8c0ceb40 2023-10-18T10:25:48.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.882+0000 7f3d917ab640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d7006ae70 0x7f3d7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:48.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.883+0000 7f3d917ab640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d7006ae70 0x7f3d7006d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0c7890 tx=0x7f3d7c005ce0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:48.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:48.886+0000 7f3d827fc640 1 -- 172.21.15.125:0/4233861026 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d840771a0 con 0x7f3d8c0ceb40 2023-10-18T10:25:49.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.052+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d8c0d0b40 con 0x7f3d8c0ceb40 2023-10-18T10:25:49.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.052+0000 7f3d827fc640 1 -- 172.21.15.125:0/4233861026 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d8407a7c0 con 0x7f3d8c0ceb40 2023-10-18T10:25:49.054 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:49.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d7006ae70 msgr2=0x7f3d7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d7006ae70 0x7f3d7006d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0c7890 tx=0x7f3d7c005ce0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0ceb40 msgr2=0x7f3d8c1e5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0ceb40 0x7f3d8c1e5390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3d840099f0 tx=0x7f3d8402db30 comp rx=0 tx=0).stop 2023-10-18T10:25:49.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 shutdown_connections 2023-10-18T10:25:49.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3d7006ae70 0x7f3d7006d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8c0cf340 0x7f3d8c0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8c15b2a0 0x7f3d8c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 --2- 172.21.15.125:0/4233861026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d8c0ceb40 0x7f3d8c1e5390 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 >> 172.21.15.125:0/4233861026 conn(0x7f3d8c0d0d60 msgr2=0x7f3d8c15a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:49.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 shutdown_connections 2023-10-18T10:25:49.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.054+0000 7f3d94210640 1 -- 172.21.15.125:0/4233861026 wait complete. 2023-10-18T10:25:49.067 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:49.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.171+0000 7f3a14f61640 1 -- 172.21.15.125:0/1825035299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 msgr2=0x7f39f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.171+0000 7f3a14f61640 1 --2- 172.21.15.125:0/1825035299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 0x7f39f8003d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009e20 tx=0x7f39f80057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.171+0000 7f3a0ed9d640 1 -- 172.21.15.125:0/1825035299 reap_dead start 2023-10-18T10:25:49.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.171+0000 7f3a14f61640 1 -- 172.21.15.125:0/1825035299 shutdown_connections 2023-10-18T10:25:49.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.171+0000 7f3a14f61640 1 -- 172.21.15.125:0/1825035299 >> 172.21.15.125:0/1825035299 conn(0x7f3a100d0d60 msgr2=0x7f3a100d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:49.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.171+0000 7f3a14f61640 1 -- 172.21.15.125:0/1825035299 shutdown_connections 2023-10-18T10:25:49.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.171+0000 7f3a14f61640 1 -- 172.21.15.125:0/1825035299 wait complete. 2023-10-18T10:25:49.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.172+0000 7f3a14f61640 1 Processor -- start 2023-10-18T10:25:49.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.172+0000 7f3a14f61640 1 -- start start 2023-10-18T10:25:49.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a14f61640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 0x7f3a100b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a14f61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a10154eb0 0x7f3a10155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a14f61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a100c9b80 0x7f3a100ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a14f61640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100c8b50 con 0x7f39f8003970 2023-10-18T10:25:49.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a14f61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10155870 con 0x7f3a10154eb0 2023-10-18T10:25:49.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a14f61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100c7ed0 con 0x7f3a100c9b80 2023-10-18T10:25:49.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 0x7f3a100b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 0x7f3a100b6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37714/0 (socket says 172.21.15.125:37714) 2023-10-18T10:25:49.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 -- 172.21.15.125:0/3450991543 learned_addr learned my addr 172.21.15.125:0/3450991543 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:49.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0dd9b640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a10154eb0 0x7f3a10155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 -- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a100c9b80 msgr2=0x7f3a100ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0ed9d640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a100c9b80 0x7f3a100ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a100c9b80 0x7f3a100ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 -- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a10154eb0 msgr2=0x7f3a10155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a10154eb0 0x7f3a10155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 -- 172.21.15.125:0/3450991543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100c81b0 con 0x7f39f8003970 2023-10-18T10:25:49.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a0e59c640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 0x7f3a100b6140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3a0002ca50 tx=0x7f3a00033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:49.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f39feffd640 1 -- 172.21.15.125:0/3450991543 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00003b70 con 0x7f39f8003970 2023-10-18T10:25:49.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f39feffd640 1 -- 172.21.15.125:0/3450991543 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a00003cd0 con 0x7f39f8003970 2023-10-18T10:25:49.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a14f61640 1 -- 172.21.15.125:0/3450991543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a100c8490 con 0x7f39f8003970 2023-10-18T10:25:49.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f39feffd640 1 -- 172.21.15.125:0/3450991543 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00004270 con 0x7f39f8003970 2023-10-18T10:25:49.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.173+0000 7f3a14f61640 1 -- 172.21.15.125:0/3450991543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a100b6680 con 0x7f39f8003970 2023-10-18T10:25:49.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.174+0000 7f39fcff9640 1 -- 172.21.15.125:0/3450991543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e8005250 con 0x7f39f8003970 2023-10-18T10:25:49.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.175+0000 7f39feffd640 1 -- 172.21.15.125:0/3450991543 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3a000043d0 con 0x7f39f8003970 2023-10-18T10:25:49.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.175+0000 7f39feffd640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.175+0000 7f39feffd640 1 -- 172.21.15.125:0/3450991543 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3a000ac940 con 0x7f39f8003970 2023-10-18T10:25:49.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.175+0000 7f3a0dd9b640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.176+0000 7f3a0dd9b640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39f006ae40 0x7f39f006d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a04001eb0 tx=0x7f3a04000bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:49.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.180+0000 7f39feffd640 1 -- 172.21.15.125:0/3450991543 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a0007b1f0 con 0x7f39f8003970 2023-10-18T10:25:49.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.345+0000 7f39fcff9640 1 -- 172.21.15.125:0/3450991543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39e8005a80 con 0x7f39f8003970 2023-10-18T10:25:49.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.346+0000 7f39feffd640 1 -- 172.21.15.125:0/3450991543 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a0004a020 con 0x7f39f8003970 2023-10-18T10:25:49.348 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:49.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.347+0000 7f39fcff9640 1 -- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39f006ae40 msgr2=0x7f39f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.347+0000 7f39fcff9640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39f006ae40 0x7f39f006d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a04001eb0 tx=0x7f3a04000bc0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 -- 172.21.15.125:0/3450991543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 msgr2=0x7f3a100b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 0x7f3a100b6140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3a0002ca50 tx=0x7f3a00033c70 comp rx=0 tx=0).stop 2023-10-18T10:25:49.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 -- 172.21.15.125:0/3450991543 shutdown_connections 2023-10-18T10:25:49.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a10154eb0 0x7f3a10155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a100c9b80 0x7f3a100ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 --2- 172.21.15.125:0/3450991543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39f8003970 0x7f3a100b6140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 -- 172.21.15.125:0/3450991543 >> 172.21.15.125:0/3450991543 conn(0x7f3a100d0d60 msgr2=0x7f3a100ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:49.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 -- 172.21.15.125:0/3450991543 shutdown_connections 2023-10-18T10:25:49.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.348+0000 7f39fcff9640 1 -- 172.21.15.125:0/3450991543 wait complete. 2023-10-18T10:25:49.360 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:49.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.463+0000 7f6e46db6640 1 -- 172.21.15.125:0/3558309776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e400ceb40 msgr2=0x7f6e400b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.463+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3558309776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e400ceb40 0x7f6e400b0650 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6e4008a810 tx=0x7f6e3802c260 comp rx=0 tx=0).stop 2023-10-18T10:25:49.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.463+0000 7f6e46db6640 1 -- 172.21.15.125:0/3558309776 shutdown_connections 2023-10-18T10:25:49.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.463+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3558309776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e400ceb40 0x7f6e400b0650 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.463+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3558309776 >> v2:172.21.15.139:3300/0 conn(0x7f6e400cf340 0x7f6e400d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.463+0000 7f6e46db6640 1 -- 172.21.15.125:0/3558309776 >> 172.21.15.125:0/3558309776 conn(0x7f6e400d0d60 msgr2=0x7f6e400d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:49.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.463+0000 7f6e46db6640 1 -- 172.21.15.125:0/3558309776 shutdown_connections 2023-10-18T10:25:49.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.463+0000 7f6e46db6640 1 -- 172.21.15.125:0/3558309776 wait complete. 2023-10-18T10:25:49.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.464+0000 7f6e46db6640 1 Processor -- start 2023-10-18T10:25:49.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.464+0000 7f6e46db6640 1 -- start start 2023-10-18T10:25:49.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e46db6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e400ceb40 0x7f6e40158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e46db6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e400cf340 0x7f6e400b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e46db6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4015b2a0 0x7f6e400b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e46db6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40158b40 con 0x7f6e4015b2a0 2023-10-18T10:25:49.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e46db6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400b3ea0 con 0x7f6e400cf340 2023-10-18T10:25:49.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e46db6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400c96e0 con 0x7f6e400ceb40 2023-10-18T10:25:49.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e45353640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4015b2a0 0x7f6e400b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e45353640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4015b2a0 0x7f6e400b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37736/0 (socket says 172.21.15.125:37736) 2023-10-18T10:25:49.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.465+0000 7f6e45353640 1 -- 172.21.15.125:0/3215643184 learned_addr learned my addr 172.21.15.125:0/3215643184 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:49.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e45353640 1 -- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e400ceb40 msgr2=0x7f6e40158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e45353640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e400ceb40 0x7f6e40158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e45353640 1 -- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e400cf340 msgr2=0x7f6e400b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e45353640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e400cf340 0x7f6e400b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e45353640 1 -- 172.21.15.125:0/3215643184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38009900 con 0x7f6e4015b2a0 2023-10-18T10:25:49.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e45353640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4015b2a0 0x7f6e400b1420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009d60 tx=0x7f6e380093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:49.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e3d7fa640 1 -- 172.21.15.125:0/3215643184 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38004240 con 0x7f6e4015b2a0 2023-10-18T10:25:49.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e3d7fa640 1 -- 172.21.15.125:0/3215643184 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e38004490 con 0x7f6e4015b2a0 2023-10-18T10:25:49.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e3d7fa640 1 -- 172.21.15.125:0/3215643184 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380048a0 con 0x7f6e4015b2a0 2023-10-18T10:25:49.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e400c99c0 con 0x7f6e4015b2a0 2023-10-18T10:25:49.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.466+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e400c9f80 con 0x7f6e4015b2a0 2023-10-18T10:25:49.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.467+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e400690e0 con 0x7f6e4015b2a0 2023-10-18T10:25:49.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.467+0000 7f6e3d7fa640 1 -- 172.21.15.125:0/3215643184 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e3800a040 con 0x7f6e4015b2a0 2023-10-18T10:25:49.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.468+0000 7f6e3d7fa640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e2c06ae40 0x7f6e2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.468+0000 7f6e3d7fa640 1 -- 172.21.15.125:0/3215643184 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6e380ace80 con 0x7f6e4015b2a0 2023-10-18T10:25:49.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.468+0000 7f6e44b52640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e2c06ae40 0x7f6e2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.469+0000 7f6e44b52640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e2c06ae40 0x7f6e2c06d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6e401e5fe0 tx=0x7f6e30006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:49.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.472+0000 7f6e3d7fa640 1 -- 172.21.15.125:0/3215643184 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e38077c90 con 0x7f6e4015b2a0 2023-10-18T10:25:49.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.639+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6e400b4010 con 0x7f6e4015b2a0 2023-10-18T10:25:49.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.639+0000 7f6e3d7fa640 1 -- 172.21.15.125:0/3215643184 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6e380440a0 con 0x7f6e4015b2a0 2023-10-18T10:25:49.641 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:49.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e2c06ae40 msgr2=0x7f6e2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e2c06ae40 0x7f6e2c06d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6e401e5fe0 tx=0x7f6e30006c90 comp rx=0 tx=0).stop 2023-10-18T10:25:49.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4015b2a0 msgr2=0x7f6e400b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4015b2a0 0x7f6e400b1420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009d60 tx=0x7f6e380093f0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 shutdown_connections 2023-10-18T10:25:49.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e2c06ae40 0x7f6e2c06d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e400cf340 0x7f6e400b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4015b2a0 0x7f6e400b1420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 --2- 172.21.15.125:0/3215643184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e400ceb40 0x7f6e40158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 >> 172.21.15.125:0/3215643184 conn(0x7f6e400d0d60 msgr2=0x7f6e4015a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:49.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 shutdown_connections 2023-10-18T10:25:49.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.641+0000 7f6e46db6640 1 -- 172.21.15.125:0/3215643184 wait complete. 2023-10-18T10:25:49.654 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:25:49.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 -- 172.21.15.125:0/917666127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c156790 msgr2=0x7fca9c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/917666127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c156790 0x7fca9c0cc990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca9c065ed0 tx=0x7fca9802c250 comp rx=0 tx=0).stop 2023-10-18T10:25:49.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 -- 172.21.15.125:0/917666127 shutdown_connections 2023-10-18T10:25:49.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/917666127 >> v2:172.21.15.139:3300/0 conn(0x7fca9c06bec0 0x7fca9c06a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/917666127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca9c0ac870 0x7fca9c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/917666127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c156790 0x7fca9c0cc990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 -- 172.21.15.125:0/917666127 >> 172.21.15.125:0/917666127 conn(0x7fca9c06fae0 msgr2=0x7fca9c06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:49.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 -- 172.21.15.125:0/917666127 shutdown_connections 2023-10-18T10:25:49.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.755+0000 7fcaa44b4640 1 -- 172.21.15.125:0/917666127 wait complete. 2023-10-18T10:25:49.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.756+0000 7fcaa44b4640 1 Processor -- start 2023-10-18T10:25:49.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.756+0000 7fcaa44b4640 1 -- start start 2023-10-18T10:25:49.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa44b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca9c06bec0 0x7fca9c1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa44b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c0ac870 0x7fca9c14eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa44b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca9c156790 0x7fca9c140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa44b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c15b840 con 0x7fca9c0ac870 2023-10-18T10:25:49.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa44b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c15bb20 con 0x7fca9c156790 2023-10-18T10:25:49.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa44b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c15be00 con 0x7fca9c06bec0 2023-10-18T10:25:49.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c0ac870 0x7fca9c14eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c0ac870 0x7fca9c14eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37758/0 (socket says 172.21.15.125:37758) 2023-10-18T10:25:49.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 -- 172.21.15.125:0/3777113173 learned_addr learned my addr 172.21.15.125:0/3777113173 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:49.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa2a51640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca9c156790 0x7fca9c140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa2250640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca9c06bec0 0x7fca9c1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 -- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca9c06bec0 msgr2=0x7fca9c1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca9c06bec0 0x7fca9c1578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 -- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca9c156790 msgr2=0x7fca9c140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca9c156790 0x7fca9c140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 -- 172.21.15.125:0/3777113173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98009900 con 0x7fca9c0ac870 2023-10-18T10:25:49.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.757+0000 7fcaa1a4f640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c0ac870 0x7fca9c14eca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d950 tx=0x7fca8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:49.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.758+0000 7fca92ffd640 1 -- 172.21.15.125:0/3777113173 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c009c30 con 0x7fca9c0ac870 2023-10-18T10:25:49.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.758+0000 7fca92ffd640 1 -- 172.21.15.125:0/3777113173 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fca8c009e80 con 0x7fca9c0ac870 2023-10-18T10:25:49.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.758+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c15c0e0 con 0x7fca9c0ac870 2023-10-18T10:25:49.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.758+0000 7fca92ffd640 1 -- 172.21.15.125:0/3777113173 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0106c0 con 0x7fca9c0ac870 2023-10-18T10:25:49.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.758+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c15c8e0 con 0x7fca9c0ac870 2023-10-18T10:25:49.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.759+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c0690e0 con 0x7fca9c0ac870 2023-10-18T10:25:49.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.759+0000 7fca92ffd640 1 -- 172.21.15.125:0/3777113173 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fca8c010820 con 0x7fca9c0ac870 2023-10-18T10:25:49.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.759+0000 7fca92ffd640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fca8006ae40 0x7fca8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:49.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.759+0000 7fca92ffd640 1 -- 172.21.15.125:0/3777113173 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fca8c089e10 con 0x7fca9c0ac870 2023-10-18T10:25:49.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.760+0000 7fcaa2250640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fca8006ae40 0x7fca8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:49.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.760+0000 7fcaa2250640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fca8006ae40 0x7fca8006d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca98036000 tx=0x7fca98004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:49.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.764+0000 7fca92ffd640 1 -- 172.21.15.125:0/3777113173 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca8c054bb0 con 0x7fca9c0ac870 2023-10-18T10:25:49.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.946+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fca9c06a870 con 0x7fca9c0ac870 2023-10-18T10:25:49.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.947+0000 7fca92ffd640 1 -- 172.21.15.125:0/3777113173 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fca8c017080 con 0x7fca9c0ac870 2023-10-18T10:25:49.949 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:25:49.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.948+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fca8006ae40 msgr2=0x7fca8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.948+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fca8006ae40 0x7fca8006d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca98036000 tx=0x7fca98004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.948+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c0ac870 msgr2=0x7fca9c14eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:49.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.948+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c0ac870 0x7fca9c14eca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d950 tx=0x7fca8c00de10 comp rx=0 tx=0).stop 2023-10-18T10:25:49.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.949+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 shutdown_connections 2023-10-18T10:25:49.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.949+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fca8006ae40 0x7fca8006d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.949+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca9c156790 0x7fca9c140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.949+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca9c0ac870 0x7fca9c14eca0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.949+0000 7fcaa44b4640 1 --2- 172.21.15.125:0/3777113173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca9c06bec0 0x7fca9c1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:49.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.949+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 >> 172.21.15.125:0/3777113173 conn(0x7fca9c06fae0 msgr2=0x7fca9c158350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:49.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.949+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 shutdown_connections 2023-10-18T10:25:49.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:49.949+0000 7fcaa44b4640 1 -- 172.21.15.125:0/3777113173 wait complete. 2023-10-18T10:25:49.961 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:25:49.961 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:49.977 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:50.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.117+0000 7f8d4175e640 1 -- 172.21.15.125:0/113417176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d24004d00 msgr2=0x7f8d24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.117+0000 7f8d4175e640 1 --2- 172.21.15.125:0/113417176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d24004d00 0x7f8d24005180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009b20 tx=0x7f8d240057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.117+0000 7f8d3b7fe640 1 -- 172.21.15.125:0/113417176 reap_dead start 2023-10-18T10:25:50.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.117+0000 7f8d4175e640 1 -- 172.21.15.125:0/113417176 shutdown_connections 2023-10-18T10:25:50.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.117+0000 7f8d4175e640 1 -- 172.21.15.125:0/113417176 >> 172.21.15.125:0/113417176 conn(0x7f8d3c069930 msgr2=0x7f8d3c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:50.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.117+0000 7f8d4175e640 1 -- 172.21.15.125:0/113417176 shutdown_connections 2023-10-18T10:25:50.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.117+0000 7f8d4175e640 1 -- 172.21.15.125:0/113417176 wait complete. 2023-10-18T10:25:50.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d4175e640 1 Processor -- start 2023-10-18T10:25:50.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d4175e640 1 -- start start 2023-10-18T10:25:50.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d4175e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d3c154b70 0x7f8d3c12a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d4175e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d3c0ca870 0x7f8d3c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d4175e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d3c13f9e0 0x7f8d3c13fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d4175e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0acf30 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d4175e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cb230 con 0x7f8d3c0ca870 2023-10-18T10:25:50.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d4175e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c155300 con 0x7f8d3c154b70 2023-10-18T10:25:50.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d3b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d3c13f9e0 0x7f8d3c13fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d3b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d3c13f9e0 0x7f8d3c13fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37774/0 (socket says 172.21.15.125:37774) 2023-10-18T10:25:50.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.119+0000 7f8d3b7fe640 1 -- 172.21.15.125:0/505638524 learned_addr learned my addr 172.21.15.125:0/505638524 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:50.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3a7fc640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d3c0ca870 0x7f8d3c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3affd640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d3c154b70 0x7f8d3c12a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3b7fe640 1 -- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d3c154b70 msgr2=0x7f8d3c12a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3b7fe640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d3c154b70 0x7f8d3c12a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3b7fe640 1 -- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d3c0ca870 msgr2=0x7f8d3c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3b7fe640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d3c0ca870 0x7f8d3c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3b7fe640 1 -- 172.21.15.125:0/505638524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c12ad90 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3a7fc640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d3c0ca870 0x7f8d3c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:50.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d3b7fe640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d3c13f9e0 0x7f8d3c13fe40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009c40 tx=0x7f8d34034be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:50.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d237fe640 1 -- 172.21.15.125:0/505638524 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34034e60 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d237fe640 1 -- 172.21.15.125:0/505638524 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d3403e420 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c1555a0 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d237fe640 1 -- 172.21.15.125:0/505638524 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3403e810 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.120+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c0acb50 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.121+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c0690e0 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.122+0000 7f8d237fe640 1 -- 172.21.15.125:0/505638524 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8d3403e970 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.122+0000 7f8d237fe640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8d2806ae40 0x7f8d2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.122+0000 7f8d237fe640 1 -- 172.21.15.125:0/505638524 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8d34000e40 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.122+0000 7f8d3affd640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8d2806ae40 0x7f8d2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.123+0000 7f8d3affd640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8d2806ae40 0x7f8d2806d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c006fd0 tx=0x7f8d2c006f20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:50.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.126+0000 7f8d237fe640 1 -- 172.21.15.125:0/505638524 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d3407ade0 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.295+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d3c13dd20 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.297+0000 7f8d237fe640 1 -- 172.21.15.125:0/505638524 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f8d3404aed0 con 0x7f8d3c13f9e0 2023-10-18T10:25:50.299 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:50.299 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:50.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.298+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8d2806ae40 msgr2=0x7f8d2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.298+0000 7f8d4175e640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8d2806ae40 0x7f8d2806d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c006fd0 tx=0x7f8d2c006f20 comp rx=0 tx=0).stop 2023-10-18T10:25:50.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.298+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d3c13f9e0 msgr2=0x7f8d3c13fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.298+0000 7f8d4175e640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d3c13f9e0 0x7f8d3c13fe40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009c40 tx=0x7f8d34034be0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.299+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 shutdown_connections 2023-10-18T10:25:50.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.299+0000 7f8d4175e640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8d2806ae40 0x7f8d2806d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.299+0000 7f8d4175e640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d3c0ca870 0x7f8d3c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.299+0000 7f8d4175e640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d3c13f9e0 0x7f8d3c13fe40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.299+0000 7f8d4175e640 1 --2- 172.21.15.125:0/505638524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d3c154b70 0x7f8d3c12a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.299+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 >> 172.21.15.125:0/505638524 conn(0x7f8d3c069930 msgr2=0x7f8d3c160100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:50.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.299+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 shutdown_connections 2023-10-18T10:25:50.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.299+0000 7f8d4175e640 1 -- 172.21.15.125:0/505638524 wait complete. 2023-10-18T10:25:50.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:50.312 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:50.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.417+0000 7fa1f2296640 1 -- 172.21.15.125:0/2239443588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1d40039a0 msgr2=0x7fa1d4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.417+0000 7fa1f2296640 1 --2- 172.21.15.125:0/2239443588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1d40039a0 0x7fa1d4003dc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009b50 tx=0x7fa1d40057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.417+0000 7fa1f0833640 1 -- 172.21.15.125:0/2239443588 reap_dead start 2023-10-18T10:25:50.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.417+0000 7fa1f2296640 1 -- 172.21.15.125:0/2239443588 shutdown_connections 2023-10-18T10:25:50.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.417+0000 7fa1f2296640 1 -- 172.21.15.125:0/2239443588 >> 172.21.15.125:0/2239443588 conn(0x7fa1ec0d0d60 msgr2=0x7fa1ec0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:50.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.417+0000 7fa1f2296640 1 -- 172.21.15.125:0/2239443588 shutdown_connections 2023-10-18T10:25:50.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.417+0000 7fa1f2296640 1 -- 172.21.15.125:0/2239443588 wait complete. 2023-10-18T10:25:50.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.418+0000 7fa1f2296640 1 Processor -- start 2023-10-18T10:25:50.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.418+0000 7fa1f2296640 1 -- start start 2023-10-18T10:25:50.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f2296640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1d40039a0 0x7fa1ec1f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f2296640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1ec0b61b0 0x7fa1ec1e5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f2296640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ec1e5930 0x7fa1ec1e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f2296640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0b6630 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f2296640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec1e57a0 con 0x7fa1d40039a0 2023-10-18T10:25:50.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f2296640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec1e62f0 con 0x7fa1ec1e5930 2023-10-18T10:25:50.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1ec0b61b0 0x7fa1ec1e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1ec0b61b0 0x7fa1ec1e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37786/0 (socket says 172.21.15.125:37786) 2023-10-18T10:25:50.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 -- 172.21.15.125:0/3251851625 learned_addr learned my addr 172.21.15.125:0/3251851625 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:50.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f0833640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ec1e5930 0x7fa1ec1e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1ebfff640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1d40039a0 0x7fa1ec1f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 -- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ec1e5930 msgr2=0x7fa1ec1e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ec1e5930 0x7fa1ec1e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 -- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1d40039a0 msgr2=0x7fa1ec1f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1d40039a0 0x7fa1ec1f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 -- 172.21.15.125:0/3251851625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec1e4c30 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1ebfff640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1d40039a0 0x7fa1ec1f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:50.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1eb7fe640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1ec0b61b0 0x7fa1ec1e5260 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0001200 tx=0x7fa1e000ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:50.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1e8ff9640 1 -- 172.21.15.125:0/3251851625 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0010890 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1e8ff9640 1 -- 172.21.15.125:0/3251851625 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1e0010ae0 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec0b6840 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1e8ff9640 1 -- 172.21.15.125:0/3251851625 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0011460 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.419+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec1e45e0 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.420+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec15c650 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.421+0000 7fa1e8ff9640 1 -- 172.21.15.125:0/3251851625 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa1e00115c0 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.421+0000 7fa1e8ff9640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa1d806ae70 0x7fa1d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.421+0000 7fa1e8ff9640 1 -- 172.21.15.125:0/3251851625 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa1e008a0b0 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.422+0000 7fa1ebfff640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa1d806ae70 0x7fa1d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.422+0000 7fa1ebfff640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa1d806ae70 0x7fa1d806d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc036000 tx=0x7fa1dc0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:50.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.425+0000 7fa1e8ff9640 1 -- 172.21.15.125:0/3251851625 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1e0054da0 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.593+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1ec0690e0 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.593+0000 7fa1e8ff9640 1 -- 172.21.15.125:0/3251851625 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1e001f0a0 con 0x7fa1ec0b61b0 2023-10-18T10:25:50.595 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:50.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa1d806ae70 msgr2=0x7fa1d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa1d806ae70 0x7fa1d806d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc036000 tx=0x7fa1dc0046c0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1ec0b61b0 msgr2=0x7fa1ec1e5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1ec0b61b0 0x7fa1ec1e5260 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0001200 tx=0x7fa1e000ea00 comp rx=0 tx=0).stop 2023-10-18T10:25:50.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 shutdown_connections 2023-10-18T10:25:50.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa1d806ae70 0x7fa1d806d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1ec1e5930 0x7fa1ec1e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1ec0b61b0 0x7fa1ec1e5260 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 --2- 172.21.15.125:0/3251851625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1d40039a0 0x7fa1ec1f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 >> 172.21.15.125:0/3251851625 conn(0x7fa1ec0d0d60 msgr2=0x7fa1ec06a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:50.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 shutdown_connections 2023-10-18T10:25:50.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.595+0000 7fa1f2296640 1 -- 172.21.15.125:0/3251851625 wait complete. 2023-10-18T10:25:50.608 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:50.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.712+0000 7f6275626640 1 -- 172.21.15.125:0/714191658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f625c004d00 msgr2=0x7f625c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.712+0000 7f6275626640 1 --2- 172.21.15.125:0/714191658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f625c004d00 0x7f625c005180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f626800c640 tx=0x7f625c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.712+0000 7f626f7fe640 1 -- 172.21.15.125:0/714191658 reap_dead start 2023-10-18T10:25:50.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.712+0000 7f6275626640 1 -- 172.21.15.125:0/714191658 shutdown_connections 2023-10-18T10:25:50.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.712+0000 7f6275626640 1 -- 172.21.15.125:0/714191658 >> 172.21.15.125:0/714191658 conn(0x7f62700d0d70 msgr2=0x7f62700d1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:50.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.712+0000 7f6275626640 1 -- 172.21.15.125:0/714191658 shutdown_connections 2023-10-18T10:25:50.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.712+0000 7f6275626640 1 -- 172.21.15.125:0/714191658 wait complete. 2023-10-18T10:25:50.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.713+0000 7f6275626640 1 Processor -- start 2023-10-18T10:25:50.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.713+0000 7f6275626640 1 -- start start 2023-10-18T10:25:50.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f6275626640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f625c004d00 0x7f6270151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f6275626640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62700c52b0 0x7f62700c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f6275626640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6270155740 0x7f6270155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f6275626640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6270152430 con 0x7f6270155740 2023-10-18T10:25:50.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f6275626640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62700c5c50 con 0x7f625c004d00 2023-10-18T10:25:50.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f6275626640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62700c6aa0 con 0x7f62700c52b0 2023-10-18T10:25:50.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6270155740 0x7f6270155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6270155740 0x7f6270155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37800/0 (socket says 172.21.15.125:37800) 2023-10-18T10:25:50.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 -- 172.21.15.125:0/2783228487 learned_addr learned my addr 172.21.15.125:0/2783228487 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:50.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 -- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62700c52b0 msgr2=0x7f62700c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62700c52b0 0x7f62700c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 -- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f625c004d00 msgr2=0x7f6270151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f625c004d00 0x7f6270151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 -- 172.21.15.125:0/2783228487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62700cee50 con 0x7f6270155740 2023-10-18T10:25:50.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f626f7fe640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6270155740 0x7f6270155bc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f626802ccc0 tx=0x7f62680012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:50.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.714+0000 7f62677fe640 1 -- 172.21.15.125:0/2783228487 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268001af0 con 0x7f6270155740 2023-10-18T10:25:50.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.715+0000 7f62677fe640 1 -- 172.21.15.125:0/2783228487 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6268003970 con 0x7f6270155740 2023-10-18T10:25:50.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.715+0000 7f62677fe640 1 -- 172.21.15.125:0/2783228487 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268003d40 con 0x7f6270155740 2023-10-18T10:25:50.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.715+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62700b5c20 con 0x7f6270155740 2023-10-18T10:25:50.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.715+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62700c6170 con 0x7f6270155740 2023-10-18T10:25:50.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.716+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62700690e0 con 0x7f6270155740 2023-10-18T10:25:50.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.716+0000 7f62677fe640 1 -- 172.21.15.125:0/2783228487 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6268031030 con 0x7f6270155740 2023-10-18T10:25:50.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.716+0000 7f62677fe640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f625006ae40 0x7f625006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:50.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.716+0000 7f62677fe640 1 -- 172.21.15.125:0/2783228487 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f62680aa5e0 con 0x7f6270155740 2023-10-18T10:25:50.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.717+0000 7f626effd640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f625006ae40 0x7f625006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:50.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.717+0000 7f626effd640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f625006ae40 0x7f625006d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6260006fd0 tx=0x7f6260006960 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:50.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.721+0000 7f62677fe640 1 -- 172.21.15.125:0/2783228487 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6268075240 con 0x7f6270155740 2023-10-18T10:25:50.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.886+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62700c6be0 con 0x7f6270155740 2023-10-18T10:25:50.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.886+0000 7f62677fe640 1 -- 172.21.15.125:0/2783228487 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6268078860 con 0x7f6270155740 2023-10-18T10:25:50.888 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:50.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f625006ae40 msgr2=0x7f625006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f625006ae40 0x7f625006d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6260006fd0 tx=0x7f6260006960 comp rx=0 tx=0).stop 2023-10-18T10:25:50.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6270155740 msgr2=0x7f6270155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:50.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6270155740 0x7f6270155bc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f626802ccc0 tx=0x7f62680012e0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 shutdown_connections 2023-10-18T10:25:50.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f625006ae40 0x7f625006d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6270155740 0x7f6270155bc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62700c52b0 0x7f62700c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 --2- 172.21.15.125:0/2783228487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f625c004d00 0x7f6270151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:50.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 >> 172.21.15.125:0/2783228487 conn(0x7f62700d0d70 msgr2=0x7f6270154050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:50.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 shutdown_connections 2023-10-18T10:25:50.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:50.888+0000 7f6275626640 1 -- 172.21.15.125:0/2783228487 wait complete. 2023-10-18T10:25:50.900 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:51.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.003+0000 7efed44d7640 1 -- 172.21.15.125:0/833564488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efeb80039a0 msgr2=0x7efeb8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.003+0000 7efed44d7640 1 --2- 172.21.15.125:0/833564488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efeb80039a0 0x7efeb8003dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efec8001d80 tx=0x7efeb80057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.003+0000 7efed2a74640 1 -- 172.21.15.125:0/833564488 reap_dead start 2023-10-18T10:25:51.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.004+0000 7efed44d7640 1 -- 172.21.15.125:0/833564488 shutdown_connections 2023-10-18T10:25:51.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.004+0000 7efed44d7640 1 -- 172.21.15.125:0/833564488 >> 172.21.15.125:0/833564488 conn(0x7efecc069930 msgr2=0x7efecc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:51.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.004+0000 7efed44d7640 1 -- 172.21.15.125:0/833564488 shutdown_connections 2023-10-18T10:25:51.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.004+0000 7efed44d7640 1 -- 172.21.15.125:0/833564488 wait complete. 2023-10-18T10:25:51.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.005+0000 7efed44d7640 1 Processor -- start 2023-10-18T10:25:51.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.005+0000 7efed44d7640 1 -- start start 2023-10-18T10:25:51.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.005+0000 7efed44d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeb80039a0 0x7efecc0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.005+0000 7efed44d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efecc15ad50 0x7efecc15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.005+0000 7efed44d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efecc12fbe0 0x7efecc130060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.005+0000 7efed44d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc15b710 con 0x7efecc12fbe0 2023-10-18T10:25:51.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.005+0000 7efed44d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1ef8b0 con 0x7efeb80039a0 2023-10-18T10:25:51.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.005+0000 7efed44d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc130bd0 con 0x7efecc15ad50 2023-10-18T10:25:51.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efecc12fbe0 0x7efecc130060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efecc12fbe0 0x7efecc130060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37816/0 (socket says 172.21.15.125:37816) 2023-10-18T10:25:51.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 -- 172.21.15.125:0/3656539139 learned_addr learned my addr 172.21.15.125:0/3656539139 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:51.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2273640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeb80039a0 0x7efecc0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed1a72640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efecc15ad50 0x7efecc15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 -- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efecc15ad50 msgr2=0x7efecc15b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efecc15ad50 0x7efecc15b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 -- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeb80039a0 msgr2=0x7efecc0b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeb80039a0 0x7efecc0b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 -- 172.21.15.125:0/3656539139 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc159d00 con 0x7efecc12fbe0 2023-10-18T10:25:51.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed2a74640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efecc12fbe0 0x7efecc130060 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efec4000c00 tx=0x7efec40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:51.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efec2ffd640 1 -- 172.21.15.125:0/3656539139 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4011840 con 0x7efecc12fbe0 2023-10-18T10:25:51.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efec2ffd640 1 -- 172.21.15.125:0/3656539139 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efec40119a0 con 0x7efecc12fbe0 2023-10-18T10:25:51.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc1308f0 con 0x7efecc12fbe0 2023-10-18T10:25:51.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efec2ffd640 1 -- 172.21.15.125:0/3656539139 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4011db0 con 0x7efecc12fbe0 2023-10-18T10:25:51.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.006+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc1ef540 con 0x7efecc12fbe0 2023-10-18T10:25:51.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.007+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc1ef0c0 con 0x7efecc12fbe0 2023-10-18T10:25:51.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.007+0000 7efec2ffd640 1 -- 172.21.15.125:0/3656539139 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efec401d460 con 0x7efecc12fbe0 2023-10-18T10:25:51.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.008+0000 7efec2ffd640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efeb006ae70 0x7efeb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.008+0000 7efec2ffd640 1 -- 172.21.15.125:0/3656539139 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efec408a2e0 con 0x7efecc12fbe0 2023-10-18T10:25:51.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.008+0000 7efed2273640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efeb006ae70 0x7efeb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.009+0000 7efed2273640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efeb006ae70 0x7efeb006d330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efec8001d80 tx=0x7efec8001510 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:51.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.012+0000 7efec2ffd640 1 -- 172.21.15.125:0/3656539139 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efecc1ef0c0 con 0x7efecc12fbe0 2023-10-18T10:25:51.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.178+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efecc1ef0c0 con 0x7efecc12fbe0 2023-10-18T10:25:51.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.178+0000 7efec2ffd640 1 -- 172.21.15.125:0/3656539139 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7efecc1ef0c0 con 0x7efecc12fbe0 2023-10-18T10:25:51.181 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:51.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efeb006ae70 msgr2=0x7efeb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efeb006ae70 0x7efeb006d330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efec8001d80 tx=0x7efec8001510 comp rx=0 tx=0).stop 2023-10-18T10:25:51.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efecc12fbe0 msgr2=0x7efecc130060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efecc12fbe0 0x7efecc130060 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efec4000c00 tx=0x7efec40039c0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 shutdown_connections 2023-10-18T10:25:51.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7efeb006ae70 0x7efeb006d330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efecc12fbe0 0x7efecc130060 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efecc15ad50 0x7efecc15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 --2- 172.21.15.125:0/3656539139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeb80039a0 0x7efecc0b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 >> 172.21.15.125:0/3656539139 conn(0x7efecc069930 msgr2=0x7efecc13ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:51.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 shutdown_connections 2023-10-18T10:25:51.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.180+0000 7efed44d7640 1 -- 172.21.15.125:0/3656539139 wait complete. 2023-10-18T10:25:51.193 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:51.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.297+0000 7f2f00f25640 1 -- 172.21.15.125:0/3640018927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc0abc10 msgr2=0x7f2efc0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.297+0000 7f2f00f25640 1 --2- 172.21.15.125:0/3640018927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc0abc10 0x7f2efc0c9b30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009ca0 tx=0x7f2ef402c2e0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.297+0000 7f2f00f25640 1 -- 172.21.15.125:0/3640018927 shutdown_connections 2023-10-18T10:25:51.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.297+0000 7f2f00f25640 1 --2- 172.21.15.125:0/3640018927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc0abc10 0x7f2efc0c9b30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.297+0000 7f2f00f25640 1 --2- 172.21.15.125:0/3640018927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2efc06af00 0x7f2efc06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.297+0000 7f2f00f25640 1 -- 172.21.15.125:0/3640018927 >> 172.21.15.125:0/3640018927 conn(0x7f2efc069930 msgr2=0x7f2efc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:51.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.297+0000 7f2f00f25640 1 -- 172.21.15.125:0/3640018927 shutdown_connections 2023-10-18T10:25:51.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.297+0000 7f2f00f25640 1 -- 172.21.15.125:0/3640018927 wait complete. 2023-10-18T10:25:51.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.298+0000 7f2f00f25640 1 Processor -- start 2023-10-18T10:25:51.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.298+0000 7f2f00f25640 1 -- start start 2023-10-18T10:25:51.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2f00f25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc06af00 0x7f2efc1eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2f00f25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2efc0abc10 0x7f2efc134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2f00f25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2efc0c96b0 0x7f2efc0ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2f00f25640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc1ef330 con 0x7f2efc06af00 2023-10-18T10:25:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2f00f25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc139290 con 0x7f2efc0abc10 2023-10-18T10:25:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2f00f25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc139570 con 0x7f2efc0c96b0 2023-10-18T10:25:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efa59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc06af00 0x7f2efc1eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efa59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc06af00 0x7f2efc1eedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37838/0 (socket says 172.21.15.125:37838) 2023-10-18T10:25:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efa59c640 1 -- 172.21.15.125:0/574444911 learned_addr learned my addr 172.21.15.125:0/574444911 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2ef1d9b640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2efc0abc10 0x7f2efc134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efad9d640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2efc0c96b0 0x7f2efc0ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efa59c640 1 -- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2efc0c96b0 msgr2=0x7f2efc0ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efa59c640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2efc0c96b0 0x7f2efc0ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efa59c640 1 -- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2efc0abc10 msgr2=0x7f2efc134fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efa59c640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2efc0abc10 0x7f2efc134fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.299+0000 7f2efa59c640 1 -- 172.21.15.125:0/574444911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef4009900 con 0x7f2efc06af00 2023-10-18T10:25:51.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.300+0000 7f2efa59c640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc06af00 0x7f2efc1eedf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00d770 tx=0x7f2eec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:51.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.300+0000 7f2ef37fe640 1 -- 172.21.15.125:0/574444911 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec009c30 con 0x7f2efc06af00 2023-10-18T10:25:51.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.300+0000 7f2ef37fe640 1 -- 172.21.15.125:0/574444911 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2eec00de80 con 0x7f2efc06af00 2023-10-18T10:25:51.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.300+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc1368f0 con 0x7f2efc06af00 2023-10-18T10:25:51.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.300+0000 7f2ef37fe640 1 -- 172.21.15.125:0/574444911 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec005280 con 0x7f2efc06af00 2023-10-18T10:25:51.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.300+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc139850 con 0x7f2efc06af00 2023-10-18T10:25:51.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.301+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2efc136e10 con 0x7f2efc06af00 2023-10-18T10:25:51.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.301+0000 7f2ef37fe640 1 -- 172.21.15.125:0/574444911 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2eec0053e0 con 0x7f2efc06af00 2023-10-18T10:25:51.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.301+0000 7f2ef37fe640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2edc06ae70 0x7f2edc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.302+0000 7f2ef37fe640 1 -- 172.21.15.125:0/574444911 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2eec089bd0 con 0x7f2efc06af00 2023-10-18T10:25:51.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.302+0000 7f2ef1d9b640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2edc06ae70 0x7f2edc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.302+0000 7f2ef1d9b640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2edc06ae70 0x7f2edc06d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0b14e0 tx=0x7f2ee4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:51.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.306+0000 7f2ef37fe640 1 -- 172.21.15.125:0/574444911 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2efc136e10 con 0x7f2efc06af00 2023-10-18T10:25:51.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.472+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2efc1399e0 con 0x7f2efc06af00 2023-10-18T10:25:51.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.473+0000 7f2ef37fe640 1 -- 172.21.15.125:0/574444911 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2eec017020 con 0x7f2efc06af00 2023-10-18T10:25:51.475 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:51.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2edc06ae70 msgr2=0x7f2edc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2edc06ae70 0x7f2edc06d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0b14e0 tx=0x7f2ee4009040 comp rx=0 tx=0).stop 2023-10-18T10:25:51.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc06af00 msgr2=0x7f2efc1eedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc06af00 0x7f2efc1eedf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00d770 tx=0x7f2eec00dc30 comp rx=0 tx=0).stop 2023-10-18T10:25:51.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 shutdown_connections 2023-10-18T10:25:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2edc06ae70 0x7f2edc06d330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2efc0abc10 0x7f2efc134fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2efc0c96b0 0x7f2efc0ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 --2- 172.21.15.125:0/574444911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2efc06af00 0x7f2efc1eedf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 >> 172.21.15.125:0/574444911 conn(0x7f2efc069930 msgr2=0x7f2efc069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:51.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 shutdown_connections 2023-10-18T10:25:51.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.475+0000 7f2f00f25640 1 -- 172.21.15.125:0/574444911 wait complete. 2023-10-18T10:25:51.488 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:25:51.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.591+0000 7f11282fc640 1 -- 172.21.15.125:0/636540278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 msgr2=0x7f110c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.591+0000 7f11282fc640 1 --2- 172.21.15.125:0/636540278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 0x7f110c0047c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1110009b80 tx=0x7f110c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.591+0000 7f1126899640 1 -- 172.21.15.125:0/636540278 reap_dead start 2023-10-18T10:25:51.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.591+0000 7f11282fc640 1 -- 172.21.15.125:0/636540278 shutdown_connections 2023-10-18T10:25:51.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.591+0000 7f11282fc640 1 -- 172.21.15.125:0/636540278 >> 172.21.15.125:0/636540278 conn(0x7f11200b0670 msgr2=0x7f11200b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:51.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.591+0000 7f11282fc640 1 -- 172.21.15.125:0/636540278 shutdown_connections 2023-10-18T10:25:51.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.591+0000 7f11282fc640 1 -- 172.21.15.125:0/636540278 wait complete. 2023-10-18T10:25:51.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.592+0000 7f11282fc640 1 Processor -- start 2023-10-18T10:25:51.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.592+0000 7f11282fc640 1 -- start start 2023-10-18T10:25:51.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f11282fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 0x7f1120154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f11282fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112014f790 0x7f11200c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f11282fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11200b1c50 0x7f11200c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f11282fc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11200c4e90 con 0x7f110c003ef0 2023-10-18T10:25:51.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f11282fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11200b2210 con 0x7f11200b1c50 2023-10-18T10:25:51.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f11282fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11200b24f0 con 0x7f112014f790 2023-10-18T10:25:51.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 0x7f1120154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 0x7f1120154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37858/0 (socket says 172.21.15.125:37858) 2023-10-18T10:25:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 -- 172.21.15.125:0/199073235 learned_addr learned my addr 172.21.15.125:0/199073235 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1125897640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112014f790 0x7f11200c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 -- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112014f790 msgr2=0x7f11200c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126899640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11200b1c50 0x7f11200c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112014f790 0x7f11200c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 -- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11200b1c50 msgr2=0x7f11200c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11200b1c50 0x7f11200c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 -- 172.21.15.125:0/199073235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11200b27d0 con 0x7f110c003ef0 2023-10-18T10:25:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1125897640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112014f790 0x7f11200c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1126098640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 0x7f1120154460 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f111c001330 tx=0x7f111c006810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1116ffd640 1 -- 172.21.15.125:0/199073235 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c00fd10 con 0x7f110c003ef0 2023-10-18T10:25:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1116ffd640 1 -- 172.21.15.125:0/199073235 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f111c00fe70 con 0x7f110c003ef0 2023-10-18T10:25:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11200b2ab0 con 0x7f110c003ef0 2023-10-18T10:25:51.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f1116ffd640 1 -- 172.21.15.125:0/199073235 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011670 con 0x7f110c003ef0 2023-10-18T10:25:51.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.593+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11201549a0 con 0x7f110c003ef0 2023-10-18T10:25:51.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.594+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11200675f0 con 0x7f110c003ef0 2023-10-18T10:25:51.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.595+0000 7f1116ffd640 1 -- 172.21.15.125:0/199073235 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f111c0117d0 con 0x7f110c003ef0 2023-10-18T10:25:51.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.595+0000 7f1116ffd640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f110406ae70 0x7f110406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:51.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.595+0000 7f1116ffd640 1 -- 172.21.15.125:0/199073235 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f111c08abe0 con 0x7f110c003ef0 2023-10-18T10:25:51.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.595+0000 7f1125897640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f110406ae70 0x7f110406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:51.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.596+0000 7f1125897640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f110406ae70 0x7f110406d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1110036000 tx=0x7f1110035040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:51.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.599+0000 7f1116ffd640 1 -- 172.21.15.125:0/199073235 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f111c0558d0 con 0x7f110c003ef0 2023-10-18T10:25:51.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.779+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f11200b4aa0 con 0x7f110c003ef0 2023-10-18T10:25:51.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.779+0000 7f1116ffd640 1 -- 172.21.15.125:0/199073235 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f111c020020 con 0x7f110c003ef0 2023-10-18T10:25:51.782 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:25:51.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f110406ae70 msgr2=0x7f110406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f110406ae70 0x7f110406d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1110036000 tx=0x7f1110035040 comp rx=0 tx=0).stop 2023-10-18T10:25:51.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 msgr2=0x7f1120154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:51.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 0x7f1120154460 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f111c001330 tx=0x7f111c006810 comp rx=0 tx=0).stop 2023-10-18T10:25:51.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 shutdown_connections 2023-10-18T10:25:51.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f110406ae70 0x7f110406d330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11200b1c50 0x7f11200c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112014f790 0x7f11200c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 --2- 172.21.15.125:0/199073235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f110c003ef0 0x7f1120154460 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:51.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 >> 172.21.15.125:0/199073235 conn(0x7f11200b0670 msgr2=0x7f11200acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:51.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 shutdown_connections 2023-10-18T10:25:51.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:51.781+0000 7f11282fc640 1 -- 172.21.15.125:0/199073235 wait complete. 2023-10-18T10:25:51.794 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:25:51.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:51.794 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T10:25:51.821 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:25:51.822 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:25:51.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:51.823 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:25:51.876 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:25:51.877 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:51.896 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:25:51.896 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:51.955 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:52.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 -- 172.21.15.125:0/2153048322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 msgr2=0x7f214c06a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 --2- 172.21.15.125:0/2153048322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 0x7f214c06a730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f214c0712e0 tx=0x7f213c02bda0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 -- 172.21.15.125:0/2153048322 shutdown_connections 2023-10-18T10:25:52.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 --2- 172.21.15.125:0/2153048322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 0x7f214c06a730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 --2- 172.21.15.125:0/2153048322 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f214c06ac70 0x7f214c15e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 --2- 172.21.15.125:0/2153048322 >> v2:172.21.15.139:3300/0 conn(0x7f214c155580 0x7f214c0b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 -- 172.21.15.125:0/2153048322 >> 172.21.15.125:0/2153048322 conn(0x7f214c06fae0 msgr2=0x7f214c06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:52.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 -- 172.21.15.125:0/2153048322 shutdown_connections 2023-10-18T10:25:52.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.055+0000 7f2153300640 1 -- 172.21.15.125:0/2153048322 wait complete. 2023-10-18T10:25:52.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.056+0000 7f2153300640 1 Processor -- start 2023-10-18T10:25:52.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.056+0000 7f2153300640 1 -- start start 2023-10-18T10:25:52.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f2153300640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c06ac70 0x7f214c0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f2153300640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 0x7f214c147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f2153300640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f214c155580 0x7f214c0ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f2153300640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c15b0f0 con 0x7f214c0b12b0 2023-10-18T10:25:52.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f2153300640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0abe30 con 0x7f214c06ac70 2023-10-18T10:25:52.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f2153300640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0ac110 con 0x7f214c155580 2023-10-18T10:25:52.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215089b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 0x7f214c147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215089b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 0x7f214c147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37878/0 (socket says 172.21.15.125:37878) 2023-10-18T10:25:52.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215089b640 1 -- 172.21.15.125:0/3676933992 learned_addr learned my addr 172.21.15.125:0/3676933992 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:52.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215089b640 1 -- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f214c155580 msgr2=0x7f214c0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215189d640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f214c155580 0x7f214c0ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215109c640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c06ac70 0x7f214c0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215089b640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f214c155580 0x7f214c0ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215089b640 1 -- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c06ac70 msgr2=0x7f214c0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215089b640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c06ac70 0x7f214c0cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215089b640 1 -- 172.21.15.125:0/3676933992 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c009560 con 0x7f214c0b12b0 2023-10-18T10:25:52.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215189d640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f214c155580 0x7f214c0ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:52.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.057+0000 7f215109c640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c06ac70 0x7f214c0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:52.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.058+0000 7f215089b640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 0x7f214c147880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f213c000c00 tx=0x7f213c02dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:52.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.058+0000 7f2141ffb640 1 -- 172.21.15.125:0/3676933992 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c005670 con 0x7f214c0b12b0 2023-10-18T10:25:52.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.058+0000 7f2141ffb640 1 -- 172.21.15.125:0/3676933992 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f213c0038f0 con 0x7f214c0b12b0 2023-10-18T10:25:52.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.058+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c0ac3f0 con 0x7f214c0b12b0 2023-10-18T10:25:52.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.058+0000 7f2141ffb640 1 -- 172.21.15.125:0/3676933992 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c003ce0 con 0x7f214c0b12b0 2023-10-18T10:25:52.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.058+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c0ac9b0 con 0x7f214c0b12b0 2023-10-18T10:25:52.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.059+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c0690e0 con 0x7f214c0b12b0 2023-10-18T10:25:52.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.059+0000 7f2141ffb640 1 -- 172.21.15.125:0/3676933992 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f213c03e450 con 0x7f214c0b12b0 2023-10-18T10:25:52.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.060+0000 7f2141ffb640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f213006ae40 0x7f213006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.060+0000 7f2141ffb640 1 -- 172.21.15.125:0/3676933992 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f213c0ac420 con 0x7f214c0b12b0 2023-10-18T10:25:52.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.060+0000 7f215109c640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f213006ae40 0x7f213006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.060+0000 7f215109c640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f213006ae40 0x7f213006d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2148005c40 tx=0x7f2148005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:52.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.064+0000 7f2141ffb640 1 -- 172.21.15.125:0/3676933992 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f213c077090 con 0x7f214c0b12b0 2023-10-18T10:25:52.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.230+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f214c0707b0 con 0x7f214c0b12b0 2023-10-18T10:25:52.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.231+0000 7f2141ffb640 1 -- 172.21.15.125:0/3676933992 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f213c07a6b0 con 0x7f214c0b12b0 2023-10-18T10:25:52.233 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:52.233 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:52.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f213006ae40 msgr2=0x7f213006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f213006ae40 0x7f213006d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2148005c40 tx=0x7f2148005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:52.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 msgr2=0x7f214c147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 0x7f214c147880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f213c000c00 tx=0x7f213c02dd70 comp rx=0 tx=0).stop 2023-10-18T10:25:52.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 shutdown_connections 2023-10-18T10:25:52.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f213006ae40 0x7f213006d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f214c155580 0x7f214c0ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f214c0b12b0 0x7f214c147880 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 --2- 172.21.15.125:0/3676933992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f214c06ac70 0x7f214c0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 >> 172.21.15.125:0/3676933992 conn(0x7f214c06fae0 msgr2=0x7f214c15b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:52.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 shutdown_connections 2023-10-18T10:25:52.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.233+0000 7f2153300640 1 -- 172.21.15.125:0/3676933992 wait complete. 2023-10-18T10:25:52.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:52.247 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:52.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.351+0000 7ff367b3e640 1 -- 172.21.15.125:0/3838547507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 msgr2=0x7ff3600cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.351+0000 7ff367b3e640 1 --2- 172.21.15.125:0/3838547507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 0x7ff3600cf310 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff35c005fd0 tx=0x7ff35c02c0e0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.351+0000 7ff367b3e640 1 -- 172.21.15.125:0/3838547507 shutdown_connections 2023-10-18T10:25:52.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.351+0000 7ff367b3e640 1 --2- 172.21.15.125:0/3838547507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3600cf850 0x7ff3600cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.351+0000 7ff367b3e640 1 --2- 172.21.15.125:0/3838547507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 0x7ff3600cf310 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.351+0000 7ff367b3e640 1 -- 172.21.15.125:0/3838547507 >> 172.21.15.125:0/3838547507 conn(0x7ff36009a030 msgr2=0x7ff3600d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:52.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.351+0000 7ff367b3e640 1 -- 172.21.15.125:0/3838547507 shutdown_connections 2023-10-18T10:25:52.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.351+0000 7ff367b3e640 1 -- 172.21.15.125:0/3838547507 wait complete. 2023-10-18T10:25:52.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.352+0000 7ff367b3e640 1 Processor -- start 2023-10-18T10:25:52.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.352+0000 7ff367b3e640 1 -- start start 2023-10-18T10:25:52.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff367b3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3600cf850 0x7ff3600c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff367b3e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 0x7ff3600b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff367b3e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff36015b640 0x7ff3600c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff367b3e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3600b49e0 con 0x7ff36015b260 2023-10-18T10:25:52.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff367b3e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3600ca280 con 0x7ff3600cf850 2023-10-18T10:25:52.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff367b3e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36015bac0 con 0x7ff36015b640 2023-10-18T10:25:52.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3650d9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 0x7ff3600b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3650d9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 0x7ff3600b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37898/0 (socket says 172.21.15.125:37898) 2023-10-18T10:25:52.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3650d9640 1 -- 172.21.15.125:0/1070390266 learned_addr learned my addr 172.21.15.125:0/1070390266 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:52.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3660db640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff36015b640 0x7ff3600c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3658da640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3600cf850 0x7ff3600c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3650d9640 1 -- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff36015b640 msgr2=0x7ff3600c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3650d9640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff36015b640 0x7ff3600c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3650d9640 1 -- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3600cf850 msgr2=0x7ff3600c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3650d9640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3600cf850 0x7ff3600c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3650d9640 1 -- 172.21.15.125:0/1070390266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c009840 con 0x7ff36015b260 2023-10-18T10:25:52.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.353+0000 7ff3658da640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3600cf850 0x7ff3600c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:52.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.354+0000 7ff3650d9640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 0x7ff3600b44a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff35000c390 tx=0x7ff35000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:52.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.354+0000 7ff3567fc640 1 -- 172.21.15.125:0/1070390266 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350004230 con 0x7ff36015b260 2023-10-18T10:25:52.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.354+0000 7ff3567fc640 1 -- 172.21.15.125:0/1070390266 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff3500044a0 con 0x7ff36015b260 2023-10-18T10:25:52.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.354+0000 7ff3567fc640 1 -- 172.21.15.125:0/1070390266 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350003850 con 0x7ff36015b260 2023-10-18T10:25:52.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.354+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36015bda0 con 0x7ff36015b260 2023-10-18T10:25:52.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.354+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36015c360 con 0x7ff36015b260 2023-10-18T10:25:52.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.355+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36015c580 con 0x7ff36015b260 2023-10-18T10:25:52.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.355+0000 7ff3567fc640 1 -- 172.21.15.125:0/1070390266 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff3500039f0 con 0x7ff36015b260 2023-10-18T10:25:52.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.355+0000 7ff3567fc640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff34406ae40 0x7ff34406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.356+0000 7ff3567fc640 1 -- 172.21.15.125:0/1070390266 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff3500891f0 con 0x7ff36015b260 2023-10-18T10:25:52.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.356+0000 7ff3658da640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff34406ae40 0x7ff34406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.356+0000 7ff3658da640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff34406ae40 0x7ff34406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff35c035000 tx=0x7ff35c004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:52.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.360+0000 7ff3567fc640 1 -- 172.21.15.125:0/1070390266 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff350053fa0 con 0x7ff36015b260 2023-10-18T10:25:52.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.528+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3600cd8c0 con 0x7ff36015b260 2023-10-18T10:25:52.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.529+0000 7ff3567fc640 1 -- 172.21.15.125:0/1070390266 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3500575c0 con 0x7ff36015b260 2023-10-18T10:25:52.531 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:52.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff34406ae40 msgr2=0x7ff34406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff34406ae40 0x7ff34406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff35c035000 tx=0x7ff35c004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 msgr2=0x7ff3600b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 0x7ff3600b44a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff35000c390 tx=0x7ff35000c850 comp rx=0 tx=0).stop 2023-10-18T10:25:52.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 shutdown_connections 2023-10-18T10:25:52.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff34406ae40 0x7ff34406d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff36015b640 0x7ff3600c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff36015b260 0x7ff3600b44a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 --2- 172.21.15.125:0/1070390266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3600cf850 0x7ff3600c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 >> 172.21.15.125:0/1070390266 conn(0x7ff36009a030 msgr2=0x7ff360069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:52.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 shutdown_connections 2023-10-18T10:25:52.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.531+0000 7ff367b3e640 1 -- 172.21.15.125:0/1070390266 wait complete. 2023-10-18T10:25:52.543 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:52.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.646+0000 7f2222626640 1 -- 172.21.15.125:0/3006554027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c15b2a0 msgr2=0x7f221c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.646+0000 7f2222626640 1 --2- 172.21.15.125:0/3006554027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c15b2a0 0x7f221c0cf720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f221c08a810 tx=0x7f221002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.646+0000 7f2222626640 1 -- 172.21.15.125:0/3006554027 shutdown_connections 2023-10-18T10:25:52.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.646+0000 7f2222626640 1 --2- 172.21.15.125:0/3006554027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c15b2a0 0x7f221c0cf720 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.646+0000 7f2222626640 1 --2- 172.21.15.125:0/3006554027 >> v2:172.21.15.139:3300/0 conn(0x7f221c0cf340 0x7f221c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.646+0000 7f2222626640 1 -- 172.21.15.125:0/3006554027 >> 172.21.15.125:0/3006554027 conn(0x7f221c0d0d60 msgr2=0x7f221c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:52.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.646+0000 7f2222626640 1 -- 172.21.15.125:0/3006554027 shutdown_connections 2023-10-18T10:25:52.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.646+0000 7f2222626640 1 -- 172.21.15.125:0/3006554027 wait complete. 2023-10-18T10:25:52.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.647+0000 7f2222626640 1 Processor -- start 2023-10-18T10:25:52.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.647+0000 7f2222626640 1 -- start start 2023-10-18T10:25:52.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f2222626640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f221c0ceb40 0x7f221c0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f2222626640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c0cf340 0x7f221c0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f2222626640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f221c15b2a0 0x7f221c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f2222626640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c14fd50 con 0x7f221c0cf340 2023-10-18T10:25:52.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f2222626640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c0c9540 con 0x7f221c0ceb40 2023-10-18T10:25:52.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f2222626640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c0c7f60 con 0x7f221c15b2a0 2023-10-18T10:25:52.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c0cf340 0x7f221c0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c0cf340 0x7f221c0b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37916/0 (socket says 172.21.15.125:37916) 2023-10-18T10:25:52.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 -- 172.21.15.125:0/2578796652 learned_addr learned my addr 172.21.15.125:0/2578796652 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:52.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221bfff640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f221c0ceb40 0x7f221c0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f2220bc3640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f221c15b2a0 0x7f221c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 -- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f221c15b2a0 msgr2=0x7f221c0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f221c15b2a0 0x7f221c0b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 -- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f221c0ceb40 msgr2=0x7f221c0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f221c0ceb40 0x7f221c0c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 -- 172.21.15.125:0/2578796652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2210009490 con 0x7f221c0cf340 2023-10-18T10:25:52.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f2220bc3640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f221c15b2a0 0x7f221c0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:52.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.648+0000 7f221b7fe640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c0cf340 0x7f221c0b58f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f221002c300 tx=0x7f22100038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:52.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.649+0000 7f2218ff9640 1 -- 172.21.15.125:0/2578796652 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210003ea0 con 0x7f221c0cf340 2023-10-18T10:25:52.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.649+0000 7f2218ff9640 1 -- 172.21.15.125:0/2578796652 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2210034440 con 0x7f221c0cf340 2023-10-18T10:25:52.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.649+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221c0c8240 con 0x7f221c0cf340 2023-10-18T10:25:52.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.649+0000 7f2218ff9640 1 -- 172.21.15.125:0/2578796652 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210034810 con 0x7f221c0cf340 2023-10-18T10:25:52.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.649+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221c0c8800 con 0x7f221c0cf340 2023-10-18T10:25:52.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.650+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221c0d04f0 con 0x7f221c0cf340 2023-10-18T10:25:52.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.650+0000 7f2218ff9640 1 -- 172.21.15.125:0/2578796652 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2210034970 con 0x7f221c0cf340 2023-10-18T10:25:52.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.650+0000 7f2218ff9640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f220806ae40 0x7f220806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.651+0000 7f2218ff9640 1 -- 172.21.15.125:0/2578796652 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f22100ac700 con 0x7f221c0cf340 2023-10-18T10:25:52.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.651+0000 7f221bfff640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f220806ae40 0x7f220806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.651+0000 7f221bfff640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f220806ae40 0x7f220806d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f220c005fd0 tx=0x7f220c005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:52.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.655+0000 7f2218ff9640 1 -- 172.21.15.125:0/2578796652 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22100773f0 con 0x7f221c0cf340 2023-10-18T10:25:52.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.822+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f221c0cb980 con 0x7f221c0cf340 2023-10-18T10:25:52.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.822+0000 7f2218ff9640 1 -- 172.21.15.125:0/2578796652 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f221007aa10 con 0x7f221c0cf340 2023-10-18T10:25:52.824 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:52.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f220806ae40 msgr2=0x7f220806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f220806ae40 0x7f220806d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f220c005fd0 tx=0x7f220c005850 comp rx=0 tx=0).stop 2023-10-18T10:25:52.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c0cf340 msgr2=0x7f221c0b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c0cf340 0x7f221c0b58f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f221002c300 tx=0x7f22100038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 shutdown_connections 2023-10-18T10:25:52.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f220806ae40 0x7f220806d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f221c15b2a0 0x7f221c0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f221c0cf340 0x7f221c0b58f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 --2- 172.21.15.125:0/2578796652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f221c0ceb40 0x7f221c0c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 >> 172.21.15.125:0/2578796652 conn(0x7f221c0d0d60 msgr2=0x7f221c069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:52.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 shutdown_connections 2023-10-18T10:25:52.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.824+0000 7f2222626640 1 -- 172.21.15.125:0/2578796652 wait complete. 2023-10-18T10:25:52.836 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:52.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.940+0000 7f89a29d0640 1 -- 172.21.15.125:0/1483151468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0ceb40 msgr2=0x7f899c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.940+0000 7f89a29d0640 1 --2- 172.21.15.125:0/1483151468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0ceb40 0x7f899c0b0650 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f899c08a120 tx=0x7f899402c080 comp rx=0 tx=0).stop 2023-10-18T10:25:52.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.940+0000 7f89a29d0640 1 -- 172.21.15.125:0/1483151468 shutdown_connections 2023-10-18T10:25:52.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.940+0000 7f89a29d0640 1 --2- 172.21.15.125:0/1483151468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0ceb40 0x7f899c0b0650 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.940+0000 7f89a29d0640 1 --2- 172.21.15.125:0/1483151468 >> v2:172.21.15.139:3300/0 conn(0x7f899c15b2a0 0x7f899c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.940+0000 7f89a29d0640 1 -- 172.21.15.125:0/1483151468 >> 172.21.15.125:0/1483151468 conn(0x7f899c0d0d60 msgr2=0x7f899c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:52.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.940+0000 7f89a29d0640 1 -- 172.21.15.125:0/1483151468 shutdown_connections 2023-10-18T10:25:52.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.940+0000 7f89a29d0640 1 -- 172.21.15.125:0/1483151468 wait complete. 2023-10-18T10:25:52.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.941+0000 7f89a29d0640 1 Processor -- start 2023-10-18T10:25:52.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.941+0000 7f89a29d0640 1 -- start start 2023-10-18T10:25:52.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f89a29d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899c0ceb40 0x7f899c156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f89a29d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0cf340 0x7f899c09e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f89a29d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899c15b2a0 0x7f899c0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f89a29d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0b6ba0 con 0x7f899c0cf340 2023-10-18T10:25:52.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f89a29d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c157120 con 0x7f899c15b2a0 2023-10-18T10:25:52.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f89a29d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c09eab0 con 0x7f899c0ceb40 2023-10-18T10:25:52.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0cf340 0x7f899c09e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0cf340 0x7f899c09e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37934/0 (socket says 172.21.15.125:37934) 2023-10-18T10:25:52.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899b7fe640 1 -- 172.21.15.125:0/3889999088 learned_addr learned my addr 172.21.15.125:0/3889999088 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:52.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f89a0f6d640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899c15b2a0 0x7f899c0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899bfff640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899c0ceb40 0x7f899c156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899b7fe640 1 -- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899c0ceb40 msgr2=0x7f899c156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899b7fe640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899c0ceb40 0x7f899c156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899b7fe640 1 -- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899c15b2a0 msgr2=0x7f899c0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:52.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899b7fe640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899c15b2a0 0x7f899c0b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:52.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899b7fe640 1 -- 172.21.15.125:0/3889999088 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8994009840 con 0x7f899c0cf340 2023-10-18T10:25:52.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.942+0000 7f899bfff640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899c0ceb40 0x7f899c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:52.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.943+0000 7f89a0f6d640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899c15b2a0 0x7f899c0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:52.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.943+0000 7f899b7fe640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0cf340 0x7f899c09e570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f899000c970 tx=0x7f899000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:52.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.943+0000 7f8998ff9640 1 -- 172.21.15.125:0/3889999088 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990007bb0 con 0x7f899c0cf340 2023-10-18T10:25:52.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.943+0000 7f8998ff9640 1 -- 172.21.15.125:0/3889999088 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8990007e00 con 0x7f899c0cf340 2023-10-18T10:25:52.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.943+0000 7f8998ff9640 1 -- 172.21.15.125:0/3889999088 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990005280 con 0x7f899c0cf340 2023-10-18T10:25:52.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.943+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c0c8340 con 0x7f899c0cf340 2023-10-18T10:25:52.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.943+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c0b6390 con 0x7f899c0cf340 2023-10-18T10:25:52.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.944+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c0c84d0 con 0x7f899c0cf340 2023-10-18T10:25:52.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.944+0000 7f8998ff9640 1 -- 172.21.15.125:0/3889999088 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f89900053e0 con 0x7f899c0cf340 2023-10-18T10:25:52.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.945+0000 7f8998ff9640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f898806ae40 0x7f898806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:52.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.945+0000 7f8998ff9640 1 -- 172.21.15.125:0/3889999088 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8990089ea0 con 0x7f899c0cf340 2023-10-18T10:25:52.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.945+0000 7f899bfff640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f898806ae40 0x7f898806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:52.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.946+0000 7f899bfff640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f898806ae40 0x7f898806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f898c000f60 tx=0x7f898c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:52.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:52.949+0000 7f8998ff9640 1 -- 172.21.15.125:0/3889999088 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8990054c50 con 0x7f899c0cf340 2023-10-18T10:25:53.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.116+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f899c0b0d00 con 0x7f899c0cf340 2023-10-18T10:25:53.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.116+0000 7f8998ff9640 1 -- 172.21.15.125:0/3889999088 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f899001d070 con 0x7f899c0cf340 2023-10-18T10:25:53.119 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:53.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f898806ae40 msgr2=0x7f898806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f898806ae40 0x7f898806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f898c000f60 tx=0x7f898c009040 comp rx=0 tx=0).stop 2023-10-18T10:25:53.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0cf340 msgr2=0x7f899c09e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0cf340 0x7f899c09e570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f899000c970 tx=0x7f899000ce30 comp rx=0 tx=0).stop 2023-10-18T10:25:53.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 shutdown_connections 2023-10-18T10:25:53.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f898806ae40 0x7f898806d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899c15b2a0 0x7f899c0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f899c0cf340 0x7f899c09e570 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 --2- 172.21.15.125:0/3889999088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899c0ceb40 0x7f899c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 >> 172.21.15.125:0/3889999088 conn(0x7f899c0d0d60 msgr2=0x7f899c15a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:53.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 shutdown_connections 2023-10-18T10:25:53.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.118+0000 7f89a29d0640 1 -- 172.21.15.125:0/3889999088 wait complete. 2023-10-18T10:25:53.131 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:53.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.234+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/686419094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd9815b2a0 msgr2=0x7fdd980d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.234+0000 7fdd9d7c0640 1 --2- 172.21.15.125:0/686419094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd9815b2a0 0x7fdd980d1910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009830 tx=0x7fdd8002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.234+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/686419094 shutdown_connections 2023-10-18T10:25:53.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.234+0000 7fdd9d7c0640 1 --2- 172.21.15.125:0/686419094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd9815b2a0 0x7fdd980d1910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.234+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/686419094 >> 172.21.15.125:0/686419094 conn(0x7fdd980d0d60 msgr2=0x7fdd980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:53.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.234+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/686419094 shutdown_connections 2023-10-18T10:25:53.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.234+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/686419094 wait complete. 2023-10-18T10:25:53.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.235+0000 7fdd9d7c0640 1 Processor -- start 2023-10-18T10:25:53.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd9d7c0640 1 -- start start 2023-10-18T10:25:53.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd9d7c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd980ceb40 0x7fdd980b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd9d7c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd980cf340 0x7fdd980b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd9d7c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9815b2a0 0x7fdd9814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd9d7c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980b5d30 con 0x7fdd980cf340 2023-10-18T10:25:53.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd9d7c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980b6b30 con 0x7fdd980ceb40 2023-10-18T10:25:53.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd9d7c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd98157b00 con 0x7fdd9815b2a0 2023-10-18T10:25:53.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd8ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd980cf340 0x7fdd980b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd8ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd980cf340 0x7fdd980b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37950/0 (socket says 172.21.15.125:37950) 2023-10-18T10:25:53.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.236+0000 7fdd8ffff640 1 -- 172.21.15.125:0/3445212732 learned_addr learned my addr 172.21.15.125:0/3445212732 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:53.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd977fe640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9815b2a0 0x7fdd9814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd96ffd640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd980ceb40 0x7fdd980b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8ffff640 1 -- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9815b2a0 msgr2=0x7fdd9814b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8ffff640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9815b2a0 0x7fdd9814b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8ffff640 1 -- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd980ceb40 msgr2=0x7fdd980b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8ffff640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd980ceb40 0x7fdd980b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8ffff640 1 -- 172.21.15.125:0/3445212732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80009490 con 0x7fdd980cf340 2023-10-18T10:25:53.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd977fe640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9815b2a0 0x7fdd9814b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:53.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd96ffd640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd980ceb40 0x7fdd980b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:53.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8ffff640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd980cf340 0x7fdd980b57f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000c00 tx=0x7fdd800038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:53.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8f7fe640 1 -- 172.21.15.125:0/3445212732 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80003ea0 con 0x7fdd980cf340 2023-10-18T10:25:53.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8f7fe640 1 -- 172.21.15.125:0/3445212732 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd80034530 con 0x7fdd980cf340 2023-10-18T10:25:53.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9814b6e0 con 0x7fdd980cf340 2023-10-18T10:25:53.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd8f7fe640 1 -- 172.21.15.125:0/3445212732 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80034940 con 0x7fdd980cf340 2023-10-18T10:25:53.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.237+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd980b6200 con 0x7fdd980cf340 2023-10-18T10:25:53.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.238+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd980c91d0 con 0x7fdd980cf340 2023-10-18T10:25:53.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.238+0000 7fdd8f7fe640 1 -- 172.21.15.125:0/3445212732 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdd80034aa0 con 0x7fdd980cf340 2023-10-18T10:25:53.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.239+0000 7fdd8f7fe640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdd7806ae40 0x7fdd7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.239+0000 7fdd8f7fe640 1 -- 172.21.15.125:0/3445212732 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdd800ac420 con 0x7fdd980cf340 2023-10-18T10:25:53.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.239+0000 7fdd96ffd640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdd7806ae40 0x7fdd7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.239+0000 7fdd96ffd640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdd7806ae40 0x7fdd7806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd88001680 tx=0x7fdd88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:53.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.243+0000 7fdd8f7fe640 1 -- 172.21.15.125:0/3445212732 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd80077080 con 0x7fdd980cf340 2023-10-18T10:25:53.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.412+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdd980690e0 con 0x7fdd980cf340 2023-10-18T10:25:53.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.412+0000 7fdd8f7fe640 1 -- 172.21.15.125:0/3445212732 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdd8007a6a0 con 0x7fdd980cf340 2023-10-18T10:25:53.415 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:53.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdd7806ae40 msgr2=0x7fdd7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdd7806ae40 0x7fdd7806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd88001680 tx=0x7fdd88009040 comp rx=0 tx=0).stop 2023-10-18T10:25:53.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd980cf340 msgr2=0x7fdd980b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd980cf340 0x7fdd980b57f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000c00 tx=0x7fdd800038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 shutdown_connections 2023-10-18T10:25:53.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fdd7806ae40 0x7fdd7806d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9815b2a0 0x7fdd9814b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd980cf340 0x7fdd980b57f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 --2- 172.21.15.125:0/3445212732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd980ceb40 0x7fdd980b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.414+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 >> 172.21.15.125:0/3445212732 conn(0x7fdd980d0d60 msgr2=0x7fdd9815a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:53.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.415+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 shutdown_connections 2023-10-18T10:25:53.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.415+0000 7fdd9d7c0640 1 -- 172.21.15.125:0/3445212732 wait complete. 2023-10-18T10:25:53.427 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:25:53.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.528+0000 7fdea26e5640 1 -- 172.21.15.125:0/1360023270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0b0530 msgr2=0x7fde9c0b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.528+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1360023270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0b0530 0x7fde9c0b09c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde94005fd0 tx=0x7fde9402c2e0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.528+0000 7fdea26e5640 1 -- 172.21.15.125:0/1360023270 shutdown_connections 2023-10-18T10:25:53.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.528+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1360023270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0b0530 0x7fde9c0b09c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.528+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1360023270 >> v2:172.21.15.139:3300/0 conn(0x7fde9c0afa20 0x7fde9c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.528+0000 7fdea26e5640 1 -- 172.21.15.125:0/1360023270 >> 172.21.15.125:0/1360023270 conn(0x7fde9c069930 msgr2=0x7fde9c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:53.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.528+0000 7fdea26e5640 1 -- 172.21.15.125:0/1360023270 shutdown_connections 2023-10-18T10:25:53.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.528+0000 7fdea26e5640 1 -- 172.21.15.125:0/1360023270 wait complete. 2023-10-18T10:25:53.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.529+0000 7fdea26e5640 1 Processor -- start 2023-10-18T10:25:53.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.529+0000 7fdea26e5640 1 -- start start 2023-10-18T10:25:53.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fdea26e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0afa20 0x7fde9c145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fdea26e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde9c0b0530 0x7fde9c146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fdea26e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde9c0cbd20 0x7fde9c14f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fdea26e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c146b40 con 0x7fde9c0b0530 2023-10-18T10:25:53.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fdea26e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c14c890 con 0x7fde9c0cbd20 2023-10-18T10:25:53.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fdea26e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c145f50 con 0x7fde9c0afa20 2023-10-18T10:25:53.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde9c0b0530 0x7fde9c146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde9c0b0530 0x7fde9c146f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37952/0 (socket says 172.21.15.125:37952) 2023-10-18T10:25:53.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.530+0000 7fde9b7fe640 1 -- 172.21.15.125:0/1738246558 learned_addr learned my addr 172.21.15.125:0/1738246558 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:53.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fdea0c82640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde9c0cbd20 0x7fde9c14f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde9bfff640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0afa20 0x7fde9c145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde9b7fe640 1 -- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0afa20 msgr2=0x7fde9c145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde9b7fe640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0afa20 0x7fde9c145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde9b7fe640 1 -- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde9c0cbd20 msgr2=0x7fde9c14f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde9b7fe640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde9c0cbd20 0x7fde9c14f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde9b7fe640 1 -- 172.21.15.125:0/1738246558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90009490 con 0x7fde9c0b0530 2023-10-18T10:25:53.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde9bfff640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0afa20 0x7fde9c145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:53.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fdea0c82640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde9c0cbd20 0x7fde9c14f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:53.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde9b7fe640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde9c0b0530 0x7fde9c146f60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fde9000ea60 tx=0x7fde9000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:53.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde98ff9640 1 -- 172.21.15.125:0/1738246558 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9000cde0 con 0x7fde9c0b0530 2023-10-18T10:25:53.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde98ff9640 1 -- 172.21.15.125:0/1738246558 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde90004020 con 0x7fde9c0b0530 2023-10-18T10:25:53.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde94009900 con 0x7fde9c0b0530 2023-10-18T10:25:53.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fde98ff9640 1 -- 172.21.15.125:0/1738246558 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90004410 con 0x7fde9c0b0530 2023-10-18T10:25:53.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.531+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c146480 con 0x7fde9c0b0530 2023-10-18T10:25:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.532+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c0690e0 con 0x7fde9c0b0530 2023-10-18T10:25:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.532+0000 7fde98ff9640 1 -- 172.21.15.125:0/1738246558 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde900035f0 con 0x7fde9c0b0530 2023-10-18T10:25:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.533+0000 7fde98ff9640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fde8806ae70 0x7fde8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.533+0000 7fde98ff9640 1 -- 172.21.15.125:0/1738246558 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fde90014070 con 0x7fde9c0b0530 2023-10-18T10:25:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.533+0000 7fde9bfff640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fde8806ae70 0x7fde8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.533+0000 7fde9bfff640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fde8806ae70 0x7fde8806d330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fde8c000f60 tx=0x7fde8c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:53.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.537+0000 7fde98ff9640 1 -- 172.21.15.125:0/1738246558 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde90055320 con 0x7fde9c0b0530 2023-10-18T10:25:53.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.714+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fde9c0700d0 con 0x7fde9c0b0530 2023-10-18T10:25:53.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.714+0000 7fde98ff9640 1 -- 172.21.15.125:0/1738246558 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fde90058940 con 0x7fde9c0b0530 2023-10-18T10:25:53.717 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:25:53.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fde8806ae70 msgr2=0x7fde8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fde8806ae70 0x7fde8806d330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fde8c000f60 tx=0x7fde8c009040 comp rx=0 tx=0).stop 2023-10-18T10:25:53.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde9c0b0530 msgr2=0x7fde9c146f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde9c0b0530 0x7fde9c146f60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fde9000ea60 tx=0x7fde9000ef20 comp rx=0 tx=0).stop 2023-10-18T10:25:53.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 shutdown_connections 2023-10-18T10:25:53.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fde8806ae70 0x7fde8806d330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde9c0cbd20 0x7fde9c14f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde9c0b0530 0x7fde9c146f60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 --2- 172.21.15.125:0/1738246558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde9c0afa20 0x7fde9c145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 >> 172.21.15.125:0/1738246558 conn(0x7fde9c069930 msgr2=0x7fde9c150520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:53.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 shutdown_connections 2023-10-18T10:25:53.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.716+0000 7fdea26e5640 1 -- 172.21.15.125:0/1738246558 wait complete. 2023-10-18T10:25:53.729 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:25:53.730 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:53.749 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:53.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.849+0000 7fe998dce640 1 -- 172.21.15.125:0/4049605679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe97c0047c0 msgr2=0x7fe97c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.849+0000 7fe998dce640 1 --2- 172.21.15.125:0/4049605679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe97c0047c0 0x7fe97c004c40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe988009b80 tx=0x7fe97c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.849+0000 7fe992d9d640 1 -- 172.21.15.125:0/4049605679 reap_dead start 2023-10-18T10:25:53.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.849+0000 7fe998dce640 1 -- 172.21.15.125:0/4049605679 shutdown_connections 2023-10-18T10:25:53.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.849+0000 7fe998dce640 1 -- 172.21.15.125:0/4049605679 >> 172.21.15.125:0/4049605679 conn(0x7fe9940d0d60 msgr2=0x7fe9940d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:53.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.849+0000 7fe998dce640 1 -- 172.21.15.125:0/4049605679 shutdown_connections 2023-10-18T10:25:53.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.849+0000 7fe998dce640 1 -- 172.21.15.125:0/4049605679 wait complete. 2023-10-18T10:25:53.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.850+0000 7fe998dce640 1 Processor -- start 2023-10-18T10:25:53.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.850+0000 7fe998dce640 1 -- start start 2023-10-18T10:25:53.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe998dce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe97c0047c0 0x7fe994155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe998dce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9940ca0b0 0x7fe99415d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe998dce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe994156650 0x7fe99415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe998dce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99415de20 con 0x7fe994156650 2023-10-18T10:25:53.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe998dce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99415c460 con 0x7fe97c0047c0 2023-10-18T10:25:53.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe998dce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9940c7fd0 con 0x7fe9940ca0b0 2023-10-18T10:25:53.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe994156650 0x7fe99415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe994156650 0x7fe99415b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37958/0 (socket says 172.21.15.125:37958) 2023-10-18T10:25:53.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 -- 172.21.15.125:0/3178471391 learned_addr learned my addr 172.21.15.125:0/3178471391 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:53.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe991d9b640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9940ca0b0 0x7fe99415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 -- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9940ca0b0 msgr2=0x7fe99415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe99259c640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe97c0047c0 0x7fe994155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9940ca0b0 0x7fe99415d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 -- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe97c0047c0 msgr2=0x7fe994155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:53.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe97c0047c0 0x7fe994155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:53.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 -- 172.21.15.125:0/3178471391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9940c82b0 con 0x7fe994156650 2023-10-18T10:25:53.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.851+0000 7fe992d9d640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe994156650 0x7fe99415b680 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9940b6e10 tx=0x7fe98400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:53.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.852+0000 7fe982ffd640 1 -- 172.21.15.125:0/3178471391 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984004b20 con 0x7fe994156650 2023-10-18T10:25:53.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.852+0000 7fe982ffd640 1 -- 172.21.15.125:0/3178471391 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe984004c80 con 0x7fe994156650 2023-10-18T10:25:53.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.852+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9940c8590 con 0x7fe994156650 2023-10-18T10:25:53.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.852+0000 7fe982ffd640 1 -- 172.21.15.125:0/3178471391 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984003bd0 con 0x7fe994156650 2023-10-18T10:25:53.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.852+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe994156ad0 con 0x7fe994156650 2023-10-18T10:25:53.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.853+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9940690e0 con 0x7fe994156650 2023-10-18T10:25:53.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.853+0000 7fe982ffd640 1 -- 172.21.15.125:0/3178471391 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe984003d30 con 0x7fe994156650 2023-10-18T10:25:53.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.853+0000 7fe982ffd640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe97406ae40 0x7fe97406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:53.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.853+0000 7fe982ffd640 1 -- 172.21.15.125:0/3178471391 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe98408a2d0 con 0x7fe994156650 2023-10-18T10:25:53.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.854+0000 7fe99259c640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe97406ae40 0x7fe97406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:53.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.854+0000 7fe99259c640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe97406ae40 0x7fe97406d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe98c001860 tx=0x7fe98c000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:53.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:53.858+0000 7fe982ffd640 1 -- 172.21.15.125:0/3178471391 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe984055050 con 0x7fe994156650 2023-10-18T10:25:54.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.026+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9940d0b40 con 0x7fe994156650 2023-10-18T10:25:54.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.028+0000 7fe982ffd640 1 -- 172.21.15.125:0/3178471391 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fe984058670 con 0x7fe994156650 2023-10-18T10:25:54.031 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:54.031 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:54.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe97406ae40 msgr2=0x7fe97406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe97406ae40 0x7fe97406d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe98c001860 tx=0x7fe98c000ed0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe994156650 msgr2=0x7fe99415b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe994156650 0x7fe99415b680 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9940b6e10 tx=0x7fe98400f3c0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 shutdown_connections 2023-10-18T10:25:54.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe97406ae40 0x7fe97406d300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe994156650 0x7fe99415b680 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9940ca0b0 0x7fe99415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 --2- 172.21.15.125:0/3178471391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe97c0047c0 0x7fe994155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.029+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 >> 172.21.15.125:0/3178471391 conn(0x7fe9940d0d60 msgr2=0x7fe9940ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:54.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.030+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 shutdown_connections 2023-10-18T10:25:54.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.030+0000 7fe998dce640 1 -- 172.21.15.125:0/3178471391 wait complete. 2023-10-18T10:25:54.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:54.044 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:54.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.146+0000 7fd7d0e10640 1 -- 172.21.15.125:0/713473543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc0cf340 msgr2=0x7fd7cc0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.146+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/713473543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc0cf340 0x7fd7cc0d1910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4005fd0 tx=0x7fd7c402c240 comp rx=0 tx=0).stop 2023-10-18T10:25:54.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.146+0000 7fd7d0e10640 1 -- 172.21.15.125:0/713473543 shutdown_connections 2023-10-18T10:25:54.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.146+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/713473543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7cc15b2a0 0x7fd7cc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.146+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/713473543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc0cf340 0x7fd7cc0d1910 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.146+0000 7fd7d0e10640 1 -- 172.21.15.125:0/713473543 >> 172.21.15.125:0/713473543 conn(0x7fd7cc0d0d60 msgr2=0x7fd7cc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:54.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.146+0000 7fd7d0e10640 1 -- 172.21.15.125:0/713473543 shutdown_connections 2023-10-18T10:25:54.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.146+0000 7fd7d0e10640 1 -- 172.21.15.125:0/713473543 wait complete. 2023-10-18T10:25:54.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7d0e10640 1 Processor -- start 2023-10-18T10:25:54.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7d0e10640 1 -- start start 2023-10-18T10:25:54.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7d0e10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7cc0ceb40 0x7fd7cc06b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7d0e10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7cc0cf340 0x7fd7cc15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7d0e10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc15b2a0 0x7fd7cc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7d0e10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0b3980 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7d0e10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc06bc30 con 0x7fd7cc0ceb40 2023-10-18T10:25:54.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7d0e10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc1f1cf0 con 0x7fd7cc0cf340 2023-10-18T10:25:54.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7cad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc15b2a0 0x7fd7cc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7cad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc15b2a0 0x7fd7cc0b6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37976/0 (socket says 172.21.15.125:37976) 2023-10-18T10:25:54.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.148+0000 7fd7cad9d640 1 -- 172.21.15.125:0/3394187184 learned_addr learned my addr 172.21.15.125:0/3394187184 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:54.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7ca59c640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7cc0ceb40 0x7fd7cc06b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7c9d9b640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7cc0cf340 0x7fd7cc15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7cad9d640 1 -- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7cc0cf340 msgr2=0x7fd7cc15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7cad9d640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7cc0cf340 0x7fd7cc15d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7cad9d640 1 -- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7cc0ceb40 msgr2=0x7fd7cc06b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7cad9d640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7cc0ceb40 0x7fd7cc06b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7cad9d640 1 -- 172.21.15.125:0/3394187184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4009900 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7cad9d640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc15b2a0 0x7fd7cc0b6280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c3c0 tx=0x7fd7bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:54.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7c2ffd640 1 -- 172.21.15.125:0/3394187184 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0052d0 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7c2ffd640 1 -- 172.21.15.125:0/3394187184 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd7bc007020 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc1f1fd0 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7c2ffd640 1 -- 172.21.15.125:0/3394187184 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc007430 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.149+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc1f2590 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.150+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc0b3b50 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.150+0000 7fd7c2ffd640 1 -- 172.21.15.125:0/3394187184 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd7bc0067d0 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.151+0000 7fd7c2ffd640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd7ac06ae40 0x7fd7ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.151+0000 7fd7c2ffd640 1 -- 172.21.15.125:0/3394187184 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd7bc088f00 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.151+0000 7fd7ca59c640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd7ac06ae40 0x7fd7ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.151+0000 7fd7ca59c640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd7ac06ae40 0x7fd7ac06d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4036000 tx=0x7fd7c4004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:54.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.156+0000 7fd7c2ffd640 1 -- 172.21.15.125:0/3394187184 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd7cc0b3b50 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.320+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7cc0b3b50 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.321+0000 7fd7c2ffd640 1 -- 172.21.15.125:0/3394187184 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7cc0b3b50 con 0x7fd7cc15b2a0 2023-10-18T10:25:54.324 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:54.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd7ac06ae40 msgr2=0x7fd7ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd7ac06ae40 0x7fd7ac06d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4036000 tx=0x7fd7c4004ab0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc15b2a0 msgr2=0x7fd7cc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc15b2a0 0x7fd7cc0b6280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c3c0 tx=0x7fd7bc00c880 comp rx=0 tx=0).stop 2023-10-18T10:25:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 shutdown_connections 2023-10-18T10:25:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd7ac06ae40 0x7fd7ac06d300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7cc15b2a0 0x7fd7cc0b6280 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7cc0cf340 0x7fd7cc15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 --2- 172.21.15.125:0/3394187184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7cc0ceb40 0x7fd7cc06b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 >> 172.21.15.125:0/3394187184 conn(0x7fd7cc0d0d60 msgr2=0x7fd7cc0c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:54.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 shutdown_connections 2023-10-18T10:25:54.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.323+0000 7fd7d0e10640 1 -- 172.21.15.125:0/3394187184 wait complete. 2023-10-18T10:25:54.336 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:54.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.442+0000 7fb8fe221640 1 -- 172.21.15.125:0/2268613324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 msgr2=0x7fb8f80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.442+0000 7fb8fe221640 1 --2- 172.21.15.125:0/2268613324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 0x7fb8f80d1910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb8f815b680 tx=0x7fb8ec02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.442+0000 7fb8fe221640 1 -- 172.21.15.125:0/2268613324 shutdown_connections 2023-10-18T10:25:54.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.442+0000 7fb8fe221640 1 --2- 172.21.15.125:0/2268613324 >> v2:172.21.15.139:3300/0 conn(0x7fb8f80ceb40 0x7fb8f80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.442+0000 7fb8fe221640 1 --2- 172.21.15.125:0/2268613324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 0x7fb8f80d1910 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.442+0000 7fb8fe221640 1 -- 172.21.15.125:0/2268613324 >> 172.21.15.125:0/2268613324 conn(0x7fb8f80d0d60 msgr2=0x7fb8f80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:54.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.442+0000 7fb8fe221640 1 -- 172.21.15.125:0/2268613324 shutdown_connections 2023-10-18T10:25:54.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.442+0000 7fb8fe221640 1 -- 172.21.15.125:0/2268613324 wait complete. 2023-10-18T10:25:54.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.443+0000 7fb8fe221640 1 Processor -- start 2023-10-18T10:25:54.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8fe221640 1 -- start start 2023-10-18T10:25:54.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8fe221640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8f80ceb40 0x7fb8f80c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8fe221640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f80cf340 0x7fb8f80c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8fe221640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 0x7fb8f81f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8fe221640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f81e4690 con 0x7fb8f815b2a0 2023-10-18T10:25:54.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8fe221640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80ca530 con 0x7fb8f80cf340 2023-10-18T10:25:54.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8fe221640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80ca810 con 0x7fb8f80ceb40 2023-10-18T10:25:54.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 0x7fb8f81f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 0x7fb8f81f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37986/0 (socket says 172.21.15.125:37986) 2023-10-18T10:25:54.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.444+0000 7fb8f7fff640 1 -- 172.21.15.125:0/607491933 learned_addr learned my addr 172.21.15.125:0/607491933 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:54.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8f7fff640 1 -- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8f80ceb40 msgr2=0x7fb8f80c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8f7fff640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8f80ceb40 0x7fb8f80c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8f7fff640 1 -- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f80cf340 msgr2=0x7fb8f80c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8f7fff640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f80cf340 0x7fb8f80c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8f7fff640 1 -- 172.21.15.125:0/607491933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec009490 con 0x7fb8f815b2a0 2023-10-18T10:25:54.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8f7fff640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 0x7fb8f81f0d30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000eaa0 tx=0x7fb8f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:54.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8dffff640 1 -- 172.21.15.125:0/607491933 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f000cdb0 con 0x7fb8f815b2a0 2023-10-18T10:25:54.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8dffff640 1 -- 172.21.15.125:0/607491933 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb8f0004b20 con 0x7fb8f815b2a0 2023-10-18T10:25:54.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f80caaf0 con 0x7fb8f815b2a0 2023-10-18T10:25:54.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8dffff640 1 -- 172.21.15.125:0/607491933 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0005020 con 0x7fb8f815b2a0 2023-10-18T10:25:54.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.445+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f80ca240 con 0x7fb8f815b2a0 2023-10-18T10:25:54.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.446+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f80677b0 con 0x7fb8f815b2a0 2023-10-18T10:25:54.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.447+0000 7fb8dffff640 1 -- 172.21.15.125:0/607491933 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb8f0005180 con 0x7fb8f815b2a0 2023-10-18T10:25:54.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.447+0000 7fb8dffff640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb8e406ae40 0x7fb8e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.447+0000 7fb8dffff640 1 -- 172.21.15.125:0/607491933 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb8f008a430 con 0x7fb8f815b2a0 2023-10-18T10:25:54.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.447+0000 7fb8f77fe640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb8e406ae40 0x7fb8e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.448+0000 7fb8f77fe640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb8e406ae40 0x7fb8e406d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8001600 tx=0x7fb8e8007910 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:54.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.451+0000 7fb8dffff640 1 -- 172.21.15.125:0/607491933 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8f00550a0 con 0x7fb8f815b2a0 2023-10-18T10:25:54.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.624+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8f80690e0 con 0x7fb8f815b2a0 2023-10-18T10:25:54.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.625+0000 7fb8dffff640 1 -- 172.21.15.125:0/607491933 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8f00586c0 con 0x7fb8f815b2a0 2023-10-18T10:25:54.628 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:54.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb8e406ae40 msgr2=0x7fb8e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb8e406ae40 0x7fb8e406d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8001600 tx=0x7fb8e8007910 comp rx=0 tx=0).stop 2023-10-18T10:25:54.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 msgr2=0x7fb8f81f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 0x7fb8f81f0d30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000eaa0 tx=0x7fb8f000ef60 comp rx=0 tx=0).stop 2023-10-18T10:25:54.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 shutdown_connections 2023-10-18T10:25:54.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb8e406ae40 0x7fb8e406d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f80cf340 0x7fb8f80c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8f815b2a0 0x7fb8f81f0d30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 --2- 172.21.15.125:0/607491933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8f80ceb40 0x7fb8f80c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 >> 172.21.15.125:0/607491933 conn(0x7fb8f80d0d60 msgr2=0x7fb8f806b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:54.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 shutdown_connections 2023-10-18T10:25:54.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.627+0000 7fb8fe221640 1 -- 172.21.15.125:0/607491933 wait complete. 2023-10-18T10:25:54.640 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:54.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.747+0000 7f2506abb640 1 -- 172.21.15.125:0/2416123192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25000cf340 msgr2=0x7f25000d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.747+0000 7f2506abb640 1 --2- 172.21.15.125:0/2416123192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25000cf340 0x7f25000d1910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009830 tx=0x7f24f402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.747+0000 7f2506abb640 1 -- 172.21.15.125:0/2416123192 shutdown_connections 2023-10-18T10:25:54.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.747+0000 7f2506abb640 1 --2- 172.21.15.125:0/2416123192 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f250015b2a0 0x7f25000cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.747+0000 7f2506abb640 1 --2- 172.21.15.125:0/2416123192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25000cf340 0x7f25000d1910 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.747+0000 7f2506abb640 1 -- 172.21.15.125:0/2416123192 >> 172.21.15.125:0/2416123192 conn(0x7f25000d0d60 msgr2=0x7f25000d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:54.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.747+0000 7f2506abb640 1 -- 172.21.15.125:0/2416123192 shutdown_connections 2023-10-18T10:25:54.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.747+0000 7f2506abb640 1 -- 172.21.15.125:0/2416123192 wait complete. 2023-10-18T10:25:54.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.748+0000 7f2506abb640 1 Processor -- start 2023-10-18T10:25:54.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.748+0000 7f2506abb640 1 -- start start 2023-10-18T10:25:54.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2506abb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25000ceb40 0x7f250015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2506abb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25000cf340 0x7f250009dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2506abb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f250015b2a0 0x7f25000b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2506abb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250015e0d0 con 0x7f250015b2a0 2023-10-18T10:25:54.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2506abb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250009e0f0 con 0x7f25000ceb40 2023-10-18T10:25:54.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2506abb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250015d830 con 0x7f25000cf340 2023-10-18T10:25:54.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2505058640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f250015b2a0 0x7f25000b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2505058640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f250015b2a0 0x7f25000b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38008/0 (socket says 172.21.15.125:38008) 2023-10-18T10:25:54.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2505058640 1 -- 172.21.15.125:0/1427679081 learned_addr learned my addr 172.21.15.125:0/1427679081 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:54.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2504857640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25000ceb40 0x7f250015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f24fffff640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25000cf340 0x7f250009dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2505058640 1 -- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25000cf340 msgr2=0x7f250009dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2505058640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25000cf340 0x7f250009dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2505058640 1 -- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25000ceb40 msgr2=0x7f250015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2505058640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25000ceb40 0x7f250015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.749+0000 7f2505058640 1 -- 172.21.15.125:0/1427679081 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4009490 con 0x7f250015b2a0 2023-10-18T10:25:54.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.750+0000 7f2505058640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f250015b2a0 0x7f25000b6670 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24f800cb10 tx=0x7f24f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:54.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.750+0000 7f24fd7fa640 1 -- 172.21.15.125:0/1427679081 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f800ac70 con 0x7f250015b2a0 2023-10-18T10:25:54.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.750+0000 7f24fd7fa640 1 -- 172.21.15.125:0/1427679081 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24f8003970 con 0x7f250015b2a0 2023-10-18T10:25:54.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.750+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250015db10 con 0x7f250015b2a0 2023-10-18T10:25:54.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.750+0000 7f24fd7fa640 1 -- 172.21.15.125:0/1427679081 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8003d80 con 0x7f250015b2a0 2023-10-18T10:25:54.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.750+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25000b4550 con 0x7f250015b2a0 2023-10-18T10:25:54.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.751+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25000b43b0 con 0x7f250015b2a0 2023-10-18T10:25:54.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.751+0000 7f24fd7fa640 1 -- 172.21.15.125:0/1427679081 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f24f8004030 con 0x7f250015b2a0 2023-10-18T10:25:54.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.752+0000 7f24fd7fa640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f24e406ae40 0x7f24e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:54.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.752+0000 7f24fd7fa640 1 -- 172.21.15.125:0/1427679081 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f24f808a300 con 0x7f250015b2a0 2023-10-18T10:25:54.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.752+0000 7f2504857640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f24e406ae40 0x7f24e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:54.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.752+0000 7f2504857640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f24e406ae40 0x7f24e406d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f24f4036000 tx=0x7f24f4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:54.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.757+0000 7f24fd7fa640 1 -- 172.21.15.125:0/1427679081 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25000b43b0 con 0x7f250015b2a0 2023-10-18T10:25:54.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.930+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f25000b43b0 con 0x7f250015b2a0 2023-10-18T10:25:54.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.930+0000 7f24fd7fa640 1 -- 172.21.15.125:0/1427679081 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f25000b43b0 con 0x7f250015b2a0 2023-10-18T10:25:54.934 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:54.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.932+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f24e406ae40 msgr2=0x7f24e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.932+0000 7f2506abb640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f24e406ae40 0x7f24e406d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f24f4036000 tx=0x7f24f4004730 comp rx=0 tx=0).stop 2023-10-18T10:25:54.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.932+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f250015b2a0 msgr2=0x7f25000b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:54.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.932+0000 7f2506abb640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f250015b2a0 0x7f25000b6670 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24f800cb10 tx=0x7f24f800a060 comp rx=0 tx=0).stop 2023-10-18T10:25:54.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.932+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 shutdown_connections 2023-10-18T10:25:54.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.932+0000 7f2506abb640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f24e406ae40 0x7f24e406d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.933+0000 7f2506abb640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f250015b2a0 0x7f25000b6670 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.933+0000 7f2506abb640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25000cf340 0x7f250009dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.933+0000 7f2506abb640 1 --2- 172.21.15.125:0/1427679081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25000ceb40 0x7f250015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:54.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.933+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 >> 172.21.15.125:0/1427679081 conn(0x7f25000d0d60 msgr2=0x7f250006b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:54.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.933+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 shutdown_connections 2023-10-18T10:25:54.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:54.933+0000 7f2506abb640 1 -- 172.21.15.125:0/1427679081 wait complete. 2023-10-18T10:25:54.946 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:55.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.051+0000 7f79fc353640 1 -- 172.21.15.125:0/4216256708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f40ab500 msgr2=0x7f79f40ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.051+0000 7f79fc353640 1 --2- 172.21.15.125:0/4216256708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f40ab500 0x7f79f40ab990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f79f40ca260 tx=0x7f79f002bd70 comp rx=0 tx=0).stop 2023-10-18T10:25:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.051+0000 7f79fc353640 1 -- 172.21.15.125:0/4216256708 shutdown_connections 2023-10-18T10:25:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.051+0000 7f79fc353640 1 --2- 172.21.15.125:0/4216256708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f40ab500 0x7f79f40ab990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.051+0000 7f79fc353640 1 --2- 172.21.15.125:0/4216256708 >> v2:172.21.15.139:3300/0 conn(0x7f79f40c9dc0 0x7f79f406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.051+0000 7f79fc353640 1 -- 172.21.15.125:0/4216256708 >> 172.21.15.125:0/4216256708 conn(0x7f79f4069930 msgr2=0x7f79f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.051+0000 7f79fc353640 1 -- 172.21.15.125:0/4216256708 shutdown_connections 2023-10-18T10:25:55.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.051+0000 7f79fc353640 1 -- 172.21.15.125:0/4216256708 wait complete. 2023-10-18T10:25:55.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.052+0000 7f79fc353640 1 Processor -- start 2023-10-18T10:25:55.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.052+0000 7f79fc353640 1 -- start start 2023-10-18T10:25:55.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fc353640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f406af00 0x7f79f40af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fc353640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79f40ab500 0x7f79f41527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fc353640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40c9dc0 0x7f79f413a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fc353640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f406f080 con 0x7f79f406af00 2023-10-18T10:25:55.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fc353640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f4152d30 con 0x7f79f40c9dc0 2023-10-18T10:25:55.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fc353640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f4152fd0 con 0x7f79f40ab500 2023-10-18T10:25:55.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa0ef640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f406af00 0x7f79f40af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa0ef640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f406af00 0x7f79f40af5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38030/0 (socket says 172.21.15.125:38030) 2023-10-18T10:25:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa0ef640 1 -- 172.21.15.125:0/2025904560 learned_addr learned my addr 172.21.15.125:0/2025904560 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79f98ee640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79f40ab500 0x7f79f41527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa8f0640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40c9dc0 0x7f79f413a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa0ef640 1 -- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79f40ab500 msgr2=0x7f79f41527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa0ef640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79f40ab500 0x7f79f41527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa0ef640 1 -- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40c9dc0 msgr2=0x7f79f413a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa0ef640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40c9dc0 0x7f79f413a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa0ef640 1 -- 172.21.15.125:0/2025904560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009490 con 0x7f79f406af00 2023-10-18T10:25:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79f98ee640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79f40ab500 0x7f79f41527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.053+0000 7f79fa8f0640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40c9dc0 0x7f79f413a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.054+0000 7f79fa0ef640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f406af00 0x7f79f40af5a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00cbc0 tx=0x7f79ec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.054+0000 7f79eaffd640 1 -- 172.21.15.125:0/2025904560 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00ace0 con 0x7f79f406af00 2023-10-18T10:25:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.054+0000 7f79eaffd640 1 -- 172.21.15.125:0/2025904560 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79ec003970 con 0x7f79f406af00 2023-10-18T10:25:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.054+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f4159a20 con 0x7f79f406af00 2023-10-18T10:25:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.054+0000 7f79eaffd640 1 -- 172.21.15.125:0/2025904560 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec003da0 con 0x7f79f406af00 2023-10-18T10:25:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.054+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f406f570 con 0x7f79f406af00 2023-10-18T10:25:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.055+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f40690e0 con 0x7f79f406af00 2023-10-18T10:25:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.055+0000 7f79eaffd640 1 -- 172.21.15.125:0/2025904560 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f79ec004030 con 0x7f79f406af00 2023-10-18T10:25:55.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.056+0000 7f79eaffd640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f79d806ae40 0x7f79d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.056+0000 7f79eaffd640 1 -- 172.21.15.125:0/2025904560 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f79ec08a3a0 con 0x7f79f406af00 2023-10-18T10:25:55.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.056+0000 7f79f98ee640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f79d806ae40 0x7f79d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.056+0000 7f79f98ee640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f79d806ae40 0x7f79d806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79e400ba60 tx=0x7f79e4005cf0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:55.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.060+0000 7f79eaffd640 1 -- 172.21.15.125:0/2025904560 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79ec055120 con 0x7f79f406af00 2023-10-18T10:25:55.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.232+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f79f406bf00 con 0x7f79f406af00 2023-10-18T10:25:55.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.233+0000 7f79eaffd640 1 -- 172.21.15.125:0/2025904560 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f79ec058740 con 0x7f79f406af00 2023-10-18T10:25:55.235 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:55.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.234+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f79d806ae40 msgr2=0x7f79d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f79d806ae40 0x7f79d806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79e400ba60 tx=0x7f79e4005cf0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f406af00 msgr2=0x7f79f40af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f406af00 0x7f79f40af5a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00cbc0 tx=0x7f79ec00a430 comp rx=0 tx=0).stop 2023-10-18T10:25:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 shutdown_connections 2023-10-18T10:25:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f79d806ae40 0x7f79d806d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40c9dc0 0x7f79f413a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79f40ab500 0x7f79f41527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 --2- 172.21.15.125:0/2025904560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79f406af00 0x7f79f40af5a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 >> 172.21.15.125:0/2025904560 conn(0x7f79f4069930 msgr2=0x7f79f4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:55.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 shutdown_connections 2023-10-18T10:25:55.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.235+0000 7f79fc353640 1 -- 172.21.15.125:0/2025904560 wait complete. 2023-10-18T10:25:55.247 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:25:55.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.349+0000 7f3003a39640 1 -- 172.21.15.125:0/330241386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 msgr2=0x7f2ffc0cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.349+0000 7f3003a39640 1 --2- 172.21.15.125:0/330241386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 0x7f2ffc0cbf20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0895b0 tx=0x7f2ff402c110 comp rx=0 tx=0).stop 2023-10-18T10:25:55.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.349+0000 7f3003a39640 1 -- 172.21.15.125:0/330241386 shutdown_connections 2023-10-18T10:25:55.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.349+0000 7f3003a39640 1 --2- 172.21.15.125:0/330241386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 0x7f2ffc0cbf20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.349+0000 7f3003a39640 1 --2- 172.21.15.125:0/330241386 >> v2:172.21.15.139:3300/0 conn(0x7f2ffc0cfa20 0x7f2ffc0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.349+0000 7f3003a39640 1 -- 172.21.15.125:0/330241386 >> 172.21.15.125:0/330241386 conn(0x7f2ffc09a030 msgr2=0x7f2ffc0d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:55.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.349+0000 7f3003a39640 1 -- 172.21.15.125:0/330241386 shutdown_connections 2023-10-18T10:25:55.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.349+0000 7f3003a39640 1 -- 172.21.15.125:0/330241386 wait complete. 2023-10-18T10:25:55.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3003a39640 1 Processor -- start 2023-10-18T10:25:55.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3003a39640 1 -- start start 2023-10-18T10:25:55.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3003a39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc0cfa20 0x7f2ffc151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3003a39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ffc15b260 0x7f2ffc1e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3003a39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 0x7f2ffc1f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3003a39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc1521e0 con 0x7f2ffc15b640 2023-10-18T10:25:55.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3003a39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0c8eb0 con 0x7f2ffc0cfa20 2023-10-18T10:25:55.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3003a39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc1523f0 con 0x7f2ffc15b260 2023-10-18T10:25:55.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3001fd6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 0x7f2ffc1f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3001fd6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 0x7f2ffc1f0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38042/0 (socket says 172.21.15.125:38042) 2023-10-18T10:25:55.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.351+0000 7f3001fd6640 1 -- 172.21.15.125:0/2774956523 learned_addr learned my addr 172.21.15.125:0/2774956523 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:55.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f30017d5640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc0cfa20 0x7f2ffc151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3000fd4640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ffc15b260 0x7f2ffc1e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3001fd6640 1 -- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ffc15b260 msgr2=0x7f2ffc1e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3001fd6640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ffc15b260 0x7f2ffc1e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3001fd6640 1 -- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc0cfa20 msgr2=0x7f2ffc151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3001fd6640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc0cfa20 0x7f2ffc151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3001fd6640 1 -- 172.21.15.125:0/2774956523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4009840 con 0x7f2ffc15b640 2023-10-18T10:25:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f30017d5640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc0cfa20 0x7f2ffc151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3001fd6640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 0x7f2ffc1f0c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009ca0 tx=0x7f2ff4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f2ff1ffb640 1 -- 172.21.15.125:0/2774956523 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4004670 con 0x7f2ffc15b640 2023-10-18T10:25:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f2ff1ffb640 1 -- 172.21.15.125:0/2774956523 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ff40047f0 con 0x7f2ffc15b640 2023-10-18T10:25:55.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f2ff1ffb640 1 -- 172.21.15.125:0/2774956523 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4034800 con 0x7f2ffc15b640 2023-10-18T10:25:55.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc15cf40 con 0x7f2ffc15b640 2023-10-18T10:25:55.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.352+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc1e4f00 con 0x7f2ffc15b640 2023-10-18T10:25:55.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.354+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc0c8c80 con 0x7f2ffc15b640 2023-10-18T10:25:55.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.354+0000 7f2ff1ffb640 1 -- 172.21.15.125:0/2774956523 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2ff4034960 con 0x7f2ffc15b640 2023-10-18T10:25:55.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.355+0000 7f2ff1ffb640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2fe006ae40 0x7f2fe006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.355+0000 7f2ff1ffb640 1 -- 172.21.15.125:0/2774956523 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2ff40acb00 con 0x7f2ffc15b640 2023-10-18T10:25:55.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.355+0000 7f30017d5640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2fe006ae40 0x7f2fe006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.356+0000 7f30017d5640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2fe006ae40 0x7f2fe006d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2fec000f60 tx=0x7f2fec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.359+0000 7f2ff1ffb640 1 -- 172.21.15.125:0/2774956523 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ffc0c8c80 con 0x7f2ffc15b640 2023-10-18T10:25:55.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.586+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2ffc0c8c80 con 0x7f2ffc15b640 2023-10-18T10:25:55.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.587+0000 7f2ff1ffb640 1 -- 172.21.15.125:0/2774956523 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ffc0c8c80 con 0x7f2ffc15b640 2023-10-18T10:25:55.589 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:25:55.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2fe006ae40 msgr2=0x7f2fe006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2fe006ae40 0x7f2fe006d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2fec000f60 tx=0x7f2fec009040 comp rx=0 tx=0).stop 2023-10-18T10:25:55.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 msgr2=0x7f2ffc1f0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 0x7f2ffc1f0c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009ca0 tx=0x7f2ff4004270 comp rx=0 tx=0).stop 2023-10-18T10:25:55.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 shutdown_connections 2023-10-18T10:25:55.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f2fe006ae40 0x7f2fe006d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ffc15b640 0x7f2ffc1f0c90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ffc15b260 0x7f2ffc1e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 --2- 172.21.15.125:0/2774956523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ffc0cfa20 0x7f2ffc151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 >> 172.21.15.125:0/2774956523 conn(0x7f2ffc09a030 msgr2=0x7f2ffc06a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:55.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 shutdown_connections 2023-10-18T10:25:55.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.589+0000 7f3003a39640 1 -- 172.21.15.125:0/2774956523 wait complete. 2023-10-18T10:25:55.601 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2023-10-18T10:25:55.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:55.602 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-18T10:25:55.635 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:25:55.635 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:25:55.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:55.636 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:25:55.721 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:25:55.721 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:55.737 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:25:55.737 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:55.791 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:55.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.931+0000 7fcb21773640 1 -- 172.21.15.125:0/2750438337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1c0acd40 msgr2=0x7fcb1c1597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.931+0000 7fcb21773640 1 --2- 172.21.15.125:0/2750438337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1c0acd40 0x7fcb1c1597c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009830 tx=0x7fcb1402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.931+0000 7fcb21773640 1 -- 172.21.15.125:0/2750438337 shutdown_connections 2023-10-18T10:25:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.931+0000 7fcb21773640 1 --2- 172.21.15.125:0/2750438337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1c0acd40 0x7fcb1c1597c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.931+0000 7fcb21773640 1 --2- 172.21.15.125:0/2750438337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1c0b24d0 0x7fcb1c0b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.931+0000 7fcb21773640 1 -- 172.21.15.125:0/2750438337 >> 172.21.15.125:0/2750438337 conn(0x7fcb1c0b3f00 msgr2=0x7fcb1c0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:55.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.932+0000 7fcb21773640 1 -- 172.21.15.125:0/2750438337 shutdown_connections 2023-10-18T10:25:55.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.932+0000 7fcb21773640 1 -- 172.21.15.125:0/2750438337 wait complete. 2023-10-18T10:25:55.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.933+0000 7fcb21773640 1 Processor -- start 2023-10-18T10:25:55.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.933+0000 7fcb21773640 1 -- start start 2023-10-18T10:25:55.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb21773640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb1c0acd40 0x7fcb1c152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb21773640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1c0b24d0 0x7fcb1c1f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb21773640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1c0b3070 0x7fcb1c1f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb21773640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c09c9d0 con 0x7fcb1c0acd40 2023-10-18T10:25:55.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb21773640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c09bd50 con 0x7fcb1c0b24d0 2023-10-18T10:25:55.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb21773640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c09c030 con 0x7fcb1c0b3070 2023-10-18T10:25:55.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb1c0acd40 0x7fcb1c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb1c0acd40 0x7fcb1c152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38056/0 (socket says 172.21.15.125:38056) 2023-10-18T10:25:55.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 -- 172.21.15.125:0/3469412921 learned_addr learned my addr 172.21.15.125:0/3469412921 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:55.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1a7fc640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1c0b24d0 0x7fcb1c1f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1b7fe640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1c0b3070 0x7fcb1c1f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 -- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1c0b3070 msgr2=0x7fcb1c1f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1c0b3070 0x7fcb1c1f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 -- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1c0b24d0 msgr2=0x7fcb1c1f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:55.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1c0b24d0 0x7fcb1c1f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:55.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 -- 172.21.15.125:0/3469412921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c009900 con 0x7fcb1c0acd40 2023-10-18T10:25:55.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb1affd640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb1c0acd40 0x7fcb1c152860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c420 tx=0x7fcb0c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:55.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb037fe640 1 -- 172.21.15.125:0/3469412921 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c003970 con 0x7fcb1c0acd40 2023-10-18T10:25:55.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb037fe640 1 -- 172.21.15.125:0/3469412921 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb0c003bc0 con 0x7fcb1c0acd40 2023-10-18T10:25:55.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb14009490 con 0x7fcb1c0acd40 2023-10-18T10:25:55.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb037fe640 1 -- 172.21.15.125:0/3469412921 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c004020 con 0x7fcb1c0acd40 2023-10-18T10:25:55.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.934+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c09c5f0 con 0x7fcb1c0acd40 2023-10-18T10:25:55.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.935+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c0cb610 con 0x7fcb1c0acd40 2023-10-18T10:25:55.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.936+0000 7fcb037fe640 1 -- 172.21.15.125:0/3469412921 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb0c00a040 con 0x7fcb1c0acd40 2023-10-18T10:25:55.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.936+0000 7fcb037fe640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb0806ae40 0x7fcb0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:55.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.936+0000 7fcb037fe640 1 -- 172.21.15.125:0/3469412921 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcb0c08a200 con 0x7fcb1c0acd40 2023-10-18T10:25:55.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.936+0000 7fcb1a7fc640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb0806ae40 0x7fcb0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:55.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.937+0000 7fcb1a7fc640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb0806ae40 0x7fcb0806d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0cf4f0 tx=0x7fcb10009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:55.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:55.940+0000 7fcb037fe640 1 -- 172.21.15.125:0/3469412921 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb0c054fa0 con 0x7fcb1c0acd40 2023-10-18T10:25:56.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.108+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb1c159e70 con 0x7fcb1c0acd40 2023-10-18T10:25:56.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.110+0000 7fcb037fe640 1 -- 172.21.15.125:0/3469412921 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fcb0c0585c0 con 0x7fcb1c0acd40 2023-10-18T10:25:56.112 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:56.112 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:56.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb0806ae40 msgr2=0x7fcb0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb0806ae40 0x7fcb0806d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0cf4f0 tx=0x7fcb10009040 comp rx=0 tx=0).stop 2023-10-18T10:25:56.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb1c0acd40 msgr2=0x7fcb1c152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb1c0acd40 0x7fcb1c152860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c420 tx=0x7fcb0c009f90 comp rx=0 tx=0).stop 2023-10-18T10:25:56.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 shutdown_connections 2023-10-18T10:25:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb0806ae40 0x7fcb0806d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1c0b24d0 0x7fcb1c1f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb1c0b3070 0x7fcb1c1f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 --2- 172.21.15.125:0/3469412921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb1c0acd40 0x7fcb1c152860 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 >> 172.21.15.125:0/3469412921 conn(0x7fcb1c0b3f00 msgr2=0x7fcb1c156580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 shutdown_connections 2023-10-18T10:25:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.112+0000 7fcb21773640 1 -- 172.21.15.125:0/3469412921 wait complete. 2023-10-18T10:25:56.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:56.125 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:56.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.228+0000 7f71dee75640 1 -- 172.21.15.125:0/3254819530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80cbd20 msgr2=0x7f71d8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.228+0000 7f71dee75640 1 --2- 172.21.15.125:0/3254819530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80cbd20 0x7f71d8158300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009830 tx=0x7f71cc02bd60 comp rx=0 tx=0).stop 2023-10-18T10:25:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.228+0000 7f71dee75640 1 -- 172.21.15.125:0/3254819530 shutdown_connections 2023-10-18T10:25:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.228+0000 7f71dee75640 1 --2- 172.21.15.125:0/3254819530 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71d80b0530 0x7f71d80b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.228+0000 7f71dee75640 1 --2- 172.21.15.125:0/3254819530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80cbd20 0x7f71d8158300 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.228+0000 7f71dee75640 1 -- 172.21.15.125:0/3254819530 >> 172.21.15.125:0/3254819530 conn(0x7f71d8069930 msgr2=0x7f71d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:56.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.228+0000 7f71dee75640 1 -- 172.21.15.125:0/3254819530 shutdown_connections 2023-10-18T10:25:56.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.228+0000 7f71dee75640 1 -- 172.21.15.125:0/3254819530 wait complete. 2023-10-18T10:25:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.229+0000 7f71dee75640 1 Processor -- start 2023-10-18T10:25:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71dee75640 1 -- start start 2023-10-18T10:25:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71dee75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71d80afa20 0x7f71d81408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71dee75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80b0530 0x7f71d813a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71dee75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71d80cbd20 0x7f71d8146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71dee75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d813aa50 con 0x7f71d80b0530 2023-10-18T10:25:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71dee75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d8140e30 con 0x7f71d80cbd20 2023-10-18T10:25:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71dee75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d813aca0 con 0x7f71d80afa20 2023-10-18T10:25:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80b0530 0x7f71d813a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.230+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80b0530 0x7f71d813a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38064/0 (socket says 172.21.15.125:38064) 2023-10-18T10:25:56.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d7fff640 1 -- 172.21.15.125:0/2975382950 learned_addr learned my addr 172.21.15.125:0/2975382950 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:56.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71dcc11640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71d80afa20 0x7f71d81408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71dd412640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71d80cbd20 0x7f71d8146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d7fff640 1 -- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71d80afa20 msgr2=0x7f71d81408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d7fff640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71d80afa20 0x7f71d81408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d7fff640 1 -- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71d80cbd20 msgr2=0x7f71d8146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d7fff640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71d80cbd20 0x7f71d8146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d7fff640 1 -- 172.21.15.125:0/2975382950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc009490 con 0x7f71d80b0530 2023-10-18T10:25:56.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d7fff640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80b0530 0x7f71d813a1c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f71c400c500 tx=0x7f71c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:56.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d57fa640 1 -- 172.21.15.125:0/2975382950 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4012830 con 0x7f71d80b0530 2023-10-18T10:25:56.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d57fa640 1 -- 172.21.15.125:0/2975382950 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f71c4012a80 con 0x7f71d80b0530 2023-10-18T10:25:56.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d813af80 con 0x7f71d80b0530 2023-10-18T10:25:56.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71d57fa640 1 -- 172.21.15.125:0/2975382950 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4012e90 con 0x7f71d80b0530 2023-10-18T10:25:56.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.231+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d813b540 con 0x7f71d80b0530 2023-10-18T10:25:56.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.232+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d80690e0 con 0x7f71d80b0530 2023-10-18T10:25:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.232+0000 7f71d57fa640 1 -- 172.21.15.125:0/2975382950 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f71c401b400 con 0x7f71d80b0530 2023-10-18T10:25:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.233+0000 7f71d57fa640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f71bc06ae40 0x7f71bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.233+0000 7f71d57fa640 1 -- 172.21.15.125:0/2975382950 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f71c4089b80 con 0x7f71d80b0530 2023-10-18T10:25:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.233+0000 7f71dcc11640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f71bc06ae40 0x7f71bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.234+0000 7f71dcc11640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f71bc06ae40 0x7f71bc06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71cc036000 tx=0x7f71cc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.237+0000 7f71d57fa640 1 -- 172.21.15.125:0/2975382950 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71c4054880 con 0x7f71d80b0530 2023-10-18T10:25:56.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.408+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71d80700d0 con 0x7f71d80b0530 2023-10-18T10:25:56.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.409+0000 7f71d57fa640 1 -- 172.21.15.125:0/2975382950 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71c40160a0 con 0x7f71d80b0530 2023-10-18T10:25:56.411 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:56.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f71bc06ae40 msgr2=0x7f71bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f71bc06ae40 0x7f71bc06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71cc036000 tx=0x7f71cc004730 comp rx=0 tx=0).stop 2023-10-18T10:25:56.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80b0530 msgr2=0x7f71d813a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80b0530 0x7f71d813a1c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f71c400c500 tx=0x7f71c400c9c0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 shutdown_connections 2023-10-18T10:25:56.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f71bc06ae40 0x7f71bc06d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71d80cbd20 0x7f71d8146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71d80b0530 0x7f71d813a1c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 --2- 172.21.15.125:0/2975382950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71d80afa20 0x7f71d81408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 >> 172.21.15.125:0/2975382950 conn(0x7f71d8069930 msgr2=0x7f71d814f400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:56.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 shutdown_connections 2023-10-18T10:25:56.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.411+0000 7f71dee75640 1 -- 172.21.15.125:0/2975382950 wait complete. 2023-10-18T10:25:56.423 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:56.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.526+0000 7fa532cf2640 1 -- 172.21.15.125:0/1278336969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0c96b0 msgr2=0x7fa52c0c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.526+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1278336969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0c96b0 0x7fa52c0c9b60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa518009830 tx=0x7fa51802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.526+0000 7fa532cf2640 1 -- 172.21.15.125:0/1278336969 shutdown_connections 2023-10-18T10:25:56.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.526+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1278336969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0c96b0 0x7fa52c0c9b60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.526+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1278336969 >> v2:172.21.15.139:3300/0 conn(0x7fa52c06af00 0x7fa52c06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.526+0000 7fa532cf2640 1 -- 172.21.15.125:0/1278336969 >> 172.21.15.125:0/1278336969 conn(0x7fa52c069930 msgr2=0x7fa52c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:56.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.527+0000 7fa532cf2640 1 -- 172.21.15.125:0/1278336969 shutdown_connections 2023-10-18T10:25:56.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.527+0000 7fa532cf2640 1 -- 172.21.15.125:0/1278336969 wait complete. 2023-10-18T10:25:56.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.528+0000 7fa532cf2640 1 Processor -- start 2023-10-18T10:25:56.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.528+0000 7fa532cf2640 1 -- start start 2023-10-18T10:25:56.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa532cf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa52c06af00 0x7fa52c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa532cf2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0ab500 0x7fa52c153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa532cf2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa52c0c96b0 0x7fa52c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa532cf2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0aef50 con 0x7fa52c0ab500 2023-10-18T10:25:56.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa532cf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0af230 con 0x7fa52c06af00 2023-10-18T10:25:56.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa532cf2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0af510 con 0x7fa52c0c96b0 2023-10-18T10:25:56.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0ab500 0x7fa52c153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0ab500 0x7fa52c153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38086/0 (socket says 172.21.15.125:38086) 2023-10-18T10:25:56.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 -- 172.21.15.125:0/1721069141 learned_addr learned my addr 172.21.15.125:0/1721069141 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:56.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa530a8e640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa52c06af00 0x7fa52c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa53128f640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa52c0c96b0 0x7fa52c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 -- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa52c0c96b0 msgr2=0x7fa52c151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa52c0c96b0 0x7fa52c151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 -- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa52c06af00 msgr2=0x7fa52c0ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa52c06af00 0x7fa52c0ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 -- 172.21.15.125:0/1721069141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa518009490 con 0x7fa52c0ab500 2023-10-18T10:25:56.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa53128f640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa52c0c96b0 0x7fa52c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:56.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa52bfff640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0ab500 0x7fa52c153b40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa5180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:56.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa5297fa640 1 -- 172.21.15.125:0/1721069141 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518003ea0 con 0x7fa52c0ab500 2023-10-18T10:25:56.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa5297fa640 1 -- 172.21.15.125:0/1721069141 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa518034440 con 0x7fa52c0ab500 2023-10-18T10:25:56.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa5297fa640 1 -- 172.21.15.125:0/1721069141 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518034850 con 0x7fa52c0ab500 2023-10-18T10:25:56.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c0af7f0 con 0x7fa52c0ab500 2023-10-18T10:25:56.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.529+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c0aec40 con 0x7fa52c0ab500 2023-10-18T10:25:56.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.530+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c0690e0 con 0x7fa52c0ab500 2023-10-18T10:25:56.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.531+0000 7fa5297fa640 1 -- 172.21.15.125:0/1721069141 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa5180349b0 con 0x7fa52c0ab500 2023-10-18T10:25:56.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.531+0000 7fa5297fa640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa51006ae40 0x7fa51006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.531+0000 7fa5297fa640 1 -- 172.21.15.125:0/1721069141 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa5180ac200 con 0x7fa52c0ab500 2023-10-18T10:25:56.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.531+0000 7fa530a8e640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa51006ae40 0x7fa51006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.532+0000 7fa530a8e640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa51006ae40 0x7fa51006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa5200058c0 tx=0x7fa52000a2d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:56.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.535+0000 7fa5297fa640 1 -- 172.21.15.125:0/1721069141 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa518076ef0 con 0x7fa52c0ab500 2023-10-18T10:25:56.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.701+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa52c158f90 con 0x7fa52c0ab500 2023-10-18T10:25:56.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.701+0000 7fa5297fa640 1 -- 172.21.15.125:0/1721069141 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa51807a510 con 0x7fa52c0ab500 2023-10-18T10:25:56.704 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:56.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa51006ae40 msgr2=0x7fa51006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa51006ae40 0x7fa51006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa5200058c0 tx=0x7fa52000a2d0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0ab500 msgr2=0x7fa52c153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0ab500 0x7fa52c153b40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa5180038a0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 shutdown_connections 2023-10-18T10:25:56.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa51006ae40 0x7fa51006d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa52c0c96b0 0x7fa52c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa52c0ab500 0x7fa52c153b40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 --2- 172.21.15.125:0/1721069141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa52c06af00 0x7fa52c0ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 >> 172.21.15.125:0/1721069141 conn(0x7fa52c069930 msgr2=0x7fa52c0cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:56.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 shutdown_connections 2023-10-18T10:25:56.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.703+0000 7fa532cf2640 1 -- 172.21.15.125:0/1721069141 wait complete. 2023-10-18T10:25:56.716 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:56.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.819+0000 7eff715c0640 1 -- 172.21.15.125:0/3354979188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 msgr2=0x7eff58004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.819+0000 7eff715c0640 1 --2- 172.21.15.125:0/3354979188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 0x7eff58004ea0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009b50 tx=0x7eff58006900 comp rx=0 tx=0).stop 2023-10-18T10:25:56.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.819+0000 7eff715c0640 1 -- 172.21.15.125:0/3354979188 shutdown_connections 2023-10-18T10:25:56.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.819+0000 7eff715c0640 1 --2- 172.21.15.125:0/3354979188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 0x7eff58004ea0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.819+0000 7eff715c0640 1 -- 172.21.15.125:0/3354979188 >> 172.21.15.125:0/3354979188 conn(0x7eff6c069930 msgr2=0x7eff6c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:56.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.819+0000 7eff715c0640 1 -- 172.21.15.125:0/3354979188 shutdown_connections 2023-10-18T10:25:56.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.819+0000 7eff715c0640 1 -- 172.21.15.125:0/3354979188 wait complete. 2023-10-18T10:25:56.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.820+0000 7eff715c0640 1 Processor -- start 2023-10-18T10:25:56.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.820+0000 7eff715c0640 1 -- start start 2023-10-18T10:25:56.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff715c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 0x7eff6c143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff715c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff6c0ad8c0 0x7eff6c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff715c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff6c1ef130 0x7eff6c1ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff715c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0ae280 con 0x7eff58004a80 2023-10-18T10:25:56.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff715c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c154960 con 0x7eff6c0ad8c0 2023-10-18T10:25:56.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff715c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c1538c0 con 0x7eff6c1ef130 2023-10-18T10:25:56.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 0x7eff6c143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 0x7eff6c143420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38112/0 (socket says 172.21.15.125:38112) 2023-10-18T10:25:56.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 -- 172.21.15.125:0/36794352 learned_addr learned my addr 172.21.15.125:0/36794352 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:56.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6b7fe640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff6c1ef130 0x7eff6c1ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6a7fc640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff6c0ad8c0 0x7eff6c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 -- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff6c1ef130 msgr2=0x7eff6c1ef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff6c1ef130 0x7eff6c1ef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 -- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff6c0ad8c0 msgr2=0x7eff6c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff6c0ad8c0 0x7eff6c0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 -- 172.21.15.125:0/36794352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c153ba0 con 0x7eff58004a80 2023-10-18T10:25:56.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6b7fe640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff6c1ef130 0x7eff6c1ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:56.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff6affd640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 0x7eff6c143420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009a70 tx=0x7eff5c033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:56.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff637fe640 1 -- 172.21.15.125:0/36794352 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004220 con 0x7eff58004a80 2023-10-18T10:25:56.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff637fe640 1 -- 172.21.15.125:0/36794352 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff5c004380 con 0x7eff58004a80 2023-10-18T10:25:56.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff637fe640 1 -- 172.21.15.125:0/36794352 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c03e950 con 0x7eff58004a80 2023-10-18T10:25:56.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c153e80 con 0x7eff58004a80 2023-10-18T10:25:56.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.821+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c154440 con 0x7eff58004a80 2023-10-18T10:25:56.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.822+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c1545d0 con 0x7eff58004a80 2023-10-18T10:25:56.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.823+0000 7eff637fe640 1 -- 172.21.15.125:0/36794352 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7eff5c03eab0 con 0x7eff58004a80 2023-10-18T10:25:56.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.823+0000 7eff637fe640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7eff4c06ae70 0x7eff4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:56.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.823+0000 7eff637fe640 1 -- 172.21.15.125:0/36794352 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7eff5c0ac7d0 con 0x7eff58004a80 2023-10-18T10:25:56.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.823+0000 7eff6a7fc640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7eff4c06ae70 0x7eff4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:56.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.824+0000 7eff6a7fc640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7eff4c06ae70 0x7eff4c06d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff6c14ba00 tx=0x7eff54001430 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:56.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.827+0000 7eff637fe640 1 -- 172.21.15.125:0/36794352 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7eff5c0774c0 con 0x7eff58004a80 2023-10-18T10:25:56.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.993+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff6c0690e0 con 0x7eff58004a80 2023-10-18T10:25:56.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.993+0000 7eff637fe640 1 -- 172.21.15.125:0/36794352 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff5c04b020 con 0x7eff58004a80 2023-10-18T10:25:56.995 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:56.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7eff4c06ae70 msgr2=0x7eff4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7eff4c06ae70 0x7eff4c06d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff6c14ba00 tx=0x7eff54001430 comp rx=0 tx=0).stop 2023-10-18T10:25:56.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 msgr2=0x7eff6c143420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:56.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 0x7eff6c143420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009a70 tx=0x7eff5c033c70 comp rx=0 tx=0).stop 2023-10-18T10:25:56.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 shutdown_connections 2023-10-18T10:25:56.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7eff4c06ae70 0x7eff4c06d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff6c0ad8c0 0x7eff6c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff6c1ef130 0x7eff6c1ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 --2- 172.21.15.125:0/36794352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff58004a80 0x7eff6c143420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:56.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 >> 172.21.15.125:0/36794352 conn(0x7eff6c069930 msgr2=0x7eff6c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:57.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 shutdown_connections 2023-10-18T10:25:57.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:56.995+0000 7eff715c0640 1 -- 172.21.15.125:0/36794352 wait complete. 2023-10-18T10:25:57.008 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:57.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.111+0000 7f80b967c640 1 -- 172.21.15.125:0/2961284994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b415c1f0 msgr2=0x7f80b406a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.111+0000 7f80b967c640 1 --2- 172.21.15.125:0/2961284994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b415c1f0 0x7f80b406a730 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f809c005fd0 tx=0x7f809c02be10 comp rx=0 tx=0).stop 2023-10-18T10:25:57.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.111+0000 7f80b967c640 1 -- 172.21.15.125:0/2961284994 shutdown_connections 2023-10-18T10:25:57.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.111+0000 7f80b967c640 1 --2- 172.21.15.125:0/2961284994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b415c1f0 0x7f80b406a730 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.111+0000 7f80b967c640 1 --2- 172.21.15.125:0/2961284994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b41579f0 0x7f80b40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.111+0000 7f80b967c640 1 -- 172.21.15.125:0/2961284994 >> 172.21.15.125:0/2961284994 conn(0x7f80b406fae0 msgr2=0x7f80b406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:57.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.111+0000 7f80b967c640 1 -- 172.21.15.125:0/2961284994 shutdown_connections 2023-10-18T10:25:57.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.111+0000 7f80b967c640 1 -- 172.21.15.125:0/2961284994 wait complete. 2023-10-18T10:25:57.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80b967c640 1 Processor -- start 2023-10-18T10:25:57.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80b967c640 1 -- start start 2023-10-18T10:25:57.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80b967c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b406ac70 0x7f80b41e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80b967c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b41579f0 0x7f80b41e1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80b967c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b415c1f0 0x7f80b41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80b967c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b413db30 con 0x7f80b41579f0 2023-10-18T10:25:57.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80b967c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b413cf20 con 0x7f80b406ac70 2023-10-18T10:25:57.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80b967c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b413d200 con 0x7f80b415c1f0 2023-10-18T10:25:57.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80abfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b41579f0 0x7f80b41e1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80abfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b41579f0 0x7f80b41e1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38128/0 (socket says 172.21.15.125:38128) 2023-10-18T10:25:57.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.113+0000 7f80abfff640 1 -- 172.21.15.125:0/4108140266 learned_addr learned my addr 172.21.15.125:0/4108140266 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:57.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80b2ffd640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b406ac70 0x7f80b41e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80b37fe640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b415c1f0 0x7f80b41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80abfff640 1 -- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b415c1f0 msgr2=0x7f80b41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80abfff640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b415c1f0 0x7f80b41561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80abfff640 1 -- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b406ac70 msgr2=0x7f80b41e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80abfff640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b406ac70 0x7f80b41e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80abfff640 1 -- 172.21.15.125:0/4108140266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c009560 con 0x7f80b41579f0 2023-10-18T10:25:57.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80b2ffd640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b406ac70 0x7f80b41e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:57.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80abfff640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b41579f0 0x7f80b41e1460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f809c036040 tx=0x7f809c02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:57.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80ab7fe640 1 -- 172.21.15.125:0/4108140266 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c0035f0 con 0x7f80b41579f0 2023-10-18T10:25:57.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80ab7fe640 1 -- 172.21.15.125:0/4108140266 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f809c003840 con 0x7f80b41579f0 2023-10-18T10:25:57.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b413d4e0 con 0x7f80b41579f0 2023-10-18T10:25:57.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80ab7fe640 1 -- 172.21.15.125:0/4108140266 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c003c10 con 0x7f80b41579f0 2023-10-18T10:25:57.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.114+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b413cb60 con 0x7f80b41579f0 2023-10-18T10:25:57.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.115+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b413d340 con 0x7f80b41579f0 2023-10-18T10:25:57.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.115+0000 7f80ab7fe640 1 -- 172.21.15.125:0/4108140266 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f809c03e450 con 0x7f80b41579f0 2023-10-18T10:25:57.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.116+0000 7f80ab7fe640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f809406ae40 0x7f809406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.116+0000 7f80ab7fe640 1 -- 172.21.15.125:0/4108140266 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f809c0ac6c0 con 0x7f80b41579f0 2023-10-18T10:25:57.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.116+0000 7f80b2ffd640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f809406ae40 0x7f809406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.117+0000 7f80b2ffd640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f809406ae40 0x7f809406d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009b50 tx=0x7f80a4005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:57.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.120+0000 7f80ab7fe640 1 -- 172.21.15.125:0/4108140266 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80b413d340 con 0x7f80b41579f0 2023-10-18T10:25:57.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.285+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f80b413d340 con 0x7f80b41579f0 2023-10-18T10:25:57.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.286+0000 7f80ab7fe640 1 -- 172.21.15.125:0/4108140266 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f80b413d340 con 0x7f80b41579f0 2023-10-18T10:25:57.288 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:57.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.287+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f809406ae40 msgr2=0x7f809406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.287+0000 7f80b967c640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f809406ae40 0x7f809406d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009b50 tx=0x7f80a4005f90 comp rx=0 tx=0).stop 2023-10-18T10:25:57.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.287+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b41579f0 msgr2=0x7f80b41e1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b41579f0 0x7f80b41e1460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f809c036040 tx=0x7f809c02dae0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 shutdown_connections 2023-10-18T10:25:57.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f809406ae40 0x7f809406d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b415c1f0 0x7f80b41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80b41579f0 0x7f80b41e1460 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 --2- 172.21.15.125:0/4108140266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b406ac70 0x7f80b41e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 >> 172.21.15.125:0/4108140266 conn(0x7f80b406fae0 msgr2=0x7f80b40cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:57.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 shutdown_connections 2023-10-18T10:25:57.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.288+0000 7f80b967c640 1 -- 172.21.15.125:0/4108140266 wait complete. 2023-10-18T10:25:57.300 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:25:57.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.403+0000 7f07f3839640 1 -- 172.21.15.125:0/2649925783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d80039a0 msgr2=0x7f07d8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.403+0000 7f07f3839640 1 --2- 172.21.15.125:0/2649925783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d80039a0 0x7f07d8003dc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f07e8001d80 tx=0x7f07d80057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.403+0000 7f07f1dd6640 1 -- 172.21.15.125:0/2649925783 reap_dead start 2023-10-18T10:25:57.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.403+0000 7f07f3839640 1 -- 172.21.15.125:0/2649925783 shutdown_connections 2023-10-18T10:25:57.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.403+0000 7f07f3839640 1 -- 172.21.15.125:0/2649925783 >> 172.21.15.125:0/2649925783 conn(0x7f07ec0b0670 msgr2=0x7f07ec0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:57.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.403+0000 7f07f3839640 1 -- 172.21.15.125:0/2649925783 shutdown_connections 2023-10-18T10:25:57.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.403+0000 7f07f3839640 1 -- 172.21.15.125:0/2649925783 wait complete. 2023-10-18T10:25:57.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.404+0000 7f07f3839640 1 Processor -- start 2023-10-18T10:25:57.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.404+0000 7f07f3839640 1 -- start start 2023-10-18T10:25:57.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f3839640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07d80039a0 0x7f07ec153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f3839640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07ec0b2170 0x7f07ec0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f3839640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ec0b25f0 0x7f07ec0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f3839640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec156830 con 0x7f07ec0b2170 2023-10-18T10:25:57.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f3839640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0b46f0 con 0x7f07ec0b25f0 2023-10-18T10:25:57.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f3839640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec154290 con 0x7f07d80039a0 2023-10-18T10:25:57.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07ec0b2170 0x7f07ec0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07ec0b2170 0x7f07ec0c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38142/0 (socket says 172.21.15.125:38142) 2023-10-18T10:25:57.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 -- 172.21.15.125:0/1647986358 learned_addr learned my addr 172.21.15.125:0/1647986358 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:57.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f15d5640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07d80039a0 0x7f07ec153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f1dd6640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ec0b25f0 0x7f07ec0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 -- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07d80039a0 msgr2=0x7f07ec153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07d80039a0 0x7f07ec153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 -- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ec0b25f0 msgr2=0x7f07ec0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ec0b25f0 0x7f07ec0c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 -- 172.21.15.125:0/1647986358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec0b4130 con 0x7f07ec0b2170 2023-10-18T10:25:57.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f0dd4640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07ec0b2170 0x7f07ec0c7dc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f07dc006630 tx=0x7f07dc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:57.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07e1ffb640 1 -- 172.21.15.125:0/1647986358 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc011900 con 0x7f07ec0b2170 2023-10-18T10:25:57.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07e1ffb640 1 -- 172.21.15.125:0/1647986358 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07dc011a80 con 0x7f07ec0b2170 2023-10-18T10:25:57.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec0b4410 con 0x7f07ec0b2170 2023-10-18T10:25:57.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.405+0000 7f07e1ffb640 1 -- 172.21.15.125:0/1647986358 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc011e90 con 0x7f07ec0b2170 2023-10-18T10:25:57.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.406+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ec156310 con 0x7f07ec0b2170 2023-10-18T10:25:57.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.407+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ec155e90 con 0x7f07ec0b2170 2023-10-18T10:25:57.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.407+0000 7f07e1ffb640 1 -- 172.21.15.125:0/1647986358 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f07dc0103c0 con 0x7f07ec0b2170 2023-10-18T10:25:57.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.407+0000 7f07e1ffb640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07d006ae70 0x7f07d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.408+0000 7f07e1ffb640 1 -- 172.21.15.125:0/1647986358 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f07dc08aa10 con 0x7f07ec0b2170 2023-10-18T10:25:57.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.408+0000 7f07f15d5640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07d006ae70 0x7f07d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.408+0000 7f07f15d5640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07d006ae70 0x7f07d006d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f07e8034000 tx=0x7f07e8001510 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:57.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.412+0000 7f07e1ffb640 1 -- 172.21.15.125:0/1647986358 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07ec155e90 con 0x7f07ec0b2170 2023-10-18T10:25:57.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.592+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f07ec155e90 con 0x7f07ec0b2170 2023-10-18T10:25:57.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.593+0000 7f07e1ffb640 1 -- 172.21.15.125:0/1647986358 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f07ec155e90 con 0x7f07ec0b2170 2023-10-18T10:25:57.595 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:25:57.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07d006ae70 msgr2=0x7f07d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07d006ae70 0x7f07d006d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f07e8034000 tx=0x7f07e8001510 comp rx=0 tx=0).stop 2023-10-18T10:25:57.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07ec0b2170 msgr2=0x7f07ec0c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07ec0b2170 0x7f07ec0c7dc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f07dc006630 tx=0x7f07dc0039c0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 shutdown_connections 2023-10-18T10:25:57.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f07d006ae70 0x7f07d006d330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ec0b25f0 0x7f07ec0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07ec0b2170 0x7f07ec0c7dc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 --2- 172.21.15.125:0/1647986358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07d80039a0 0x7f07ec153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 >> 172.21.15.125:0/1647986358 conn(0x7f07ec0b0670 msgr2=0x7f07ec157490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:57.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 shutdown_connections 2023-10-18T10:25:57.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.595+0000 7f07f3839640 1 -- 172.21.15.125:0/1647986358 wait complete. 2023-10-18T10:25:57.607 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:25:57.607 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:57.623 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:57.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.764+0000 7f1b7a875640 1 -- 172.21.15.125:0/1430954274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 msgr2=0x7f1b60005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.764+0000 7f1b7a875640 1 --2- 172.21.15.125:0/1430954274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 0x7f1b60005180 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c670 tx=0x7f1b600057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.764+0000 7f1b78e12640 1 -- 172.21.15.125:0/1430954274 reap_dead start 2023-10-18T10:25:57.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.764+0000 7f1b7a875640 1 -- 172.21.15.125:0/1430954274 shutdown_connections 2023-10-18T10:25:57.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.764+0000 7f1b7a875640 1 -- 172.21.15.125:0/1430954274 >> 172.21.15.125:0/1430954274 conn(0x7f1b740d0d60 msgr2=0x7f1b740d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:57.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.764+0000 7f1b7a875640 1 -- 172.21.15.125:0/1430954274 shutdown_connections 2023-10-18T10:25:57.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.764+0000 7f1b7a875640 1 -- 172.21.15.125:0/1430954274 wait complete. 2023-10-18T10:25:57.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.765+0000 7f1b7a875640 1 Processor -- start 2023-10-18T10:25:57.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.765+0000 7f1b7a875640 1 -- start start 2023-10-18T10:25:57.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b7a875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 0x7f1b740c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b7a875640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b740c9fe0 0x7f1b740b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b7a875640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b741f4090 0x7f1b740b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b7a875640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740ca510 con 0x7f1b60004d00 2023-10-18T10:25:57.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b7a875640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740ca7f0 con 0x7f1b740c9fe0 2023-10-18T10:25:57.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b7a875640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740caad0 con 0x7f1b741f4090 2023-10-18T10:25:57.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 0x7f1b740c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 0x7f1b740c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36370/0 (socket says 172.21.15.125:36370) 2023-10-18T10:25:57.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 -- 172.21.15.125:0/3690446207 learned_addr learned my addr 172.21.15.125:0/3690446207 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:57.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 -- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b741f4090 msgr2=0x7f1b740b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b741f4090 0x7f1b740b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 -- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b740c9fe0 msgr2=0x7f1b740b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b740c9fe0 0x7f1b740b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 -- 172.21.15.125:0/3690446207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b740cadb0 con 0x7f1b60004d00 2023-10-18T10:25:57.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b73fff640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 0x7f1b740c9aa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c011a30 tx=0x7f1b5c011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:57.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.766+0000 7f1b70ff9640 1 -- 172.21.15.125:0/3690446207 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00fe70 con 0x7f1b60004d00 2023-10-18T10:25:57.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.767+0000 7f1b70ff9640 1 -- 172.21.15.125:0/3690446207 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b5c019430 con 0x7f1b60004d00 2023-10-18T10:25:57.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.767+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b740c8340 con 0x7f1b60004d00 2023-10-18T10:25:57.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.767+0000 7f1b70ff9640 1 -- 172.21.15.125:0/3690446207 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c019840 con 0x7f1b60004d00 2023-10-18T10:25:57.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.767+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7414a7b0 con 0x7f1b60004d00 2023-10-18T10:25:57.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.767+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b740c8670 con 0x7f1b60004d00 2023-10-18T10:25:57.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.768+0000 7f1b70ff9640 1 -- 172.21.15.125:0/3690446207 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1b5c0199a0 con 0x7f1b60004d00 2023-10-18T10:25:57.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.768+0000 7f1b70ff9640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1b5406ae40 0x7f1b5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:57.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.768+0000 7f1b70ff9640 1 -- 172.21.15.125:0/3690446207 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1b5c08a750 con 0x7f1b60004d00 2023-10-18T10:25:57.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.769+0000 7f1b737fe640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1b5406ae40 0x7f1b5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:57.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.769+0000 7f1b737fe640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1b5406ae40 0x7f1b5406d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b741575b0 tx=0x7f1b64009660 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:57.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.773+0000 7f1b70ff9640 1 -- 172.21.15.125:0/3690446207 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b5c0554f0 con 0x7f1b60004d00 2023-10-18T10:25:57.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.938+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b740cb980 con 0x7f1b60004d00 2023-10-18T10:25:57.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.939+0000 7f1b70ff9640 1 -- 172.21.15.125:0/3690446207 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f1b5c058b10 con 0x7f1b60004d00 2023-10-18T10:25:57.942 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:57.942 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:57.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1b5406ae40 msgr2=0x7f1b5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1b5406ae40 0x7f1b5406d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b741575b0 tx=0x7f1b64009660 comp rx=0 tx=0).stop 2023-10-18T10:25:57.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 msgr2=0x7f1b740c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:57.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 0x7f1b740c9aa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c011a30 tx=0x7f1b5c011ef0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 shutdown_connections 2023-10-18T10:25:57.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1b5406ae40 0x7f1b5406d300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b740c9fe0 0x7f1b740b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b741f4090 0x7f1b740b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 --2- 172.21.15.125:0/3690446207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b60004d00 0x7f1b740c9aa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:57.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 >> 172.21.15.125:0/3690446207 conn(0x7f1b740d0d60 msgr2=0x7f1b7406b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:57.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 shutdown_connections 2023-10-18T10:25:57.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:57.941+0000 7f1b7a875640 1 -- 172.21.15.125:0/3690446207 wait complete. 2023-10-18T10:25:57.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:57.955 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:58.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.057+0000 7f49a127c640 1 -- 172.21.15.125:0/1283248906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b3e0 msgr2=0x7f499c0cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.057+0000 7f49a127c640 1 --2- 172.21.15.125:0/1283248906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b3e0 0x7f499c0cf2d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f499c089b90 tx=0x7f499002bdb0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.057+0000 7f49a127c640 1 -- 172.21.15.125:0/1283248906 shutdown_connections 2023-10-18T10:25:58.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.057+0000 7f49a127c640 1 --2- 172.21.15.125:0/1283248906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b3e0 0x7f499c0cf2d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.057+0000 7f49a127c640 1 --2- 172.21.15.125:0/1283248906 >> v2:172.21.15.139:3300/0 conn(0x7f499c15b000 0x7f499c0d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.057+0000 7f49a127c640 1 -- 172.21.15.125:0/1283248906 >> 172.21.15.125:0/1283248906 conn(0x7f499c0d0cf0 msgr2=0x7f499c0d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:58.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.057+0000 7f49a127c640 1 -- 172.21.15.125:0/1283248906 shutdown_connections 2023-10-18T10:25:58.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.057+0000 7f49a127c640 1 -- 172.21.15.125:0/1283248906 wait complete. 2023-10-18T10:25:58.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.059+0000 7f49a127c640 1 Processor -- start 2023-10-18T10:25:58.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.059+0000 7f49a127c640 1 -- start start 2023-10-18T10:25:58.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.059+0000 7f49a127c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f499c0afdc0 0x7f499c1e6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.059+0000 7f49a127c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b000 0x7f499c0c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.059+0000 7f49a127c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f499c15b3e0 0x7f499c0c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.059+0000 7f49a127c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0c91b0 con 0x7f499c15b000 2023-10-18T10:25:58.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.059+0000 7f49a127c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0c6950 con 0x7f499c0afdc0 2023-10-18T10:25:58.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.059+0000 7f49a127c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0c6bf0 con 0x7f499c15b3e0 2023-10-18T10:25:58.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b000 0x7f499c0c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b000 0x7f499c0c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36388/0 (socket says 172.21.15.125:36388) 2023-10-18T10:25:58.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 -- 172.21.15.125:0/1161621097 learned_addr learned my addr 172.21.15.125:0/1161621097 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:58.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499affd640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f499c0afdc0 0x7f499c1e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499b7fe640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f499c15b3e0 0x7f499c0c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 -- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f499c15b3e0 msgr2=0x7f499c0c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f499c15b3e0 0x7f499c0c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 -- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f499c0afdc0 msgr2=0x7f499c1e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f499c0afdc0 0x7f499c1e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 -- 172.21.15.125:0/1161621097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990009560 con 0x7f499c15b000 2023-10-18T10:25:58.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f499a7fc640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b000 0x7f499c0c6410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4990000c00 tx=0x7f499002dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:58.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f49837fe640 1 -- 172.21.15.125:0/1161621097 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990005670 con 0x7f499c15b000 2023-10-18T10:25:58.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f49837fe640 1 -- 172.21.15.125:0/1161621097 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f49900038f0 con 0x7f499c15b000 2023-10-18T10:25:58.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c0c6ed0 con 0x7f499c15b000 2023-10-18T10:25:58.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f49837fe640 1 -- 172.21.15.125:0/1161621097 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990003ce0 con 0x7f499c15b000 2023-10-18T10:25:58.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.060+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c0c4c20 con 0x7f499c15b000 2023-10-18T10:25:58.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.061+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c0c73f0 con 0x7f499c15b000 2023-10-18T10:25:58.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.062+0000 7f49837fe640 1 -- 172.21.15.125:0/1161621097 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f499003e450 con 0x7f499c15b000 2023-10-18T10:25:58.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.062+0000 7f49837fe640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f498806ae40 0x7f498806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.062+0000 7f49837fe640 1 -- 172.21.15.125:0/1161621097 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f49900ac300 con 0x7f499c15b000 2023-10-18T10:25:58.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.062+0000 7f499affd640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f498806ae40 0x7f498806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.063+0000 7f499affd640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f498806ae40 0x7f498806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f498c005fd0 tx=0x7f498c005e20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:58.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.066+0000 7f49837fe640 1 -- 172.21.15.125:0/1161621097 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f499c0c73f0 con 0x7f499c15b000 2023-10-18T10:25:58.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.234+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f499c0c73f0 con 0x7f499c15b000 2023-10-18T10:25:58.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.234+0000 7f49837fe640 1 -- 172.21.15.125:0/1161621097 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f499c0c73f0 con 0x7f499c15b000 2023-10-18T10:25:58.236 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:25:58.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f498806ae40 msgr2=0x7f498806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f498806ae40 0x7f498806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f498c005fd0 tx=0x7f498c005e20 comp rx=0 tx=0).stop 2023-10-18T10:25:58.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b000 msgr2=0x7f499c0c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b000 0x7f499c0c6410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4990000c00 tx=0x7f499002dd80 comp rx=0 tx=0).stop 2023-10-18T10:25:58.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 shutdown_connections 2023-10-18T10:25:58.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f498806ae40 0x7f498806d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f499c15b3e0 0x7f499c0c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f499c15b000 0x7f499c0c6410 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 --2- 172.21.15.125:0/1161621097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f499c0afdc0 0x7f499c1e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 >> 172.21.15.125:0/1161621097 conn(0x7f499c0d0cf0 msgr2=0x7f499c0d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:58.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 shutdown_connections 2023-10-18T10:25:58.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.236+0000 7f49a127c640 1 -- 172.21.15.125:0/1161621097 wait complete. 2023-10-18T10:25:58.248 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:25:58.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.352+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/32002390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640abc10 msgr2=0x7f3c640c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.352+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/32002390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640abc10 0x7f3c640c9b60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c005fd0 tx=0x7f3c5c02c1d0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.352+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/32002390 shutdown_connections 2023-10-18T10:25:58.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.352+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/32002390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640abc10 0x7f3c640c9b60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.352+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/32002390 >> v2:172.21.15.139:3300/0 conn(0x7f3c6406af00 0x7f3c6406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.352+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/32002390 >> 172.21.15.125:0/32002390 conn(0x7f3c64069930 msgr2=0x7f3c64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:58.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.352+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/32002390 shutdown_connections 2023-10-18T10:25:58.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.352+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/32002390 wait complete. 2023-10-18T10:25:58.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.353+0000 7f3c6b0b4640 1 Processor -- start 2023-10-18T10:25:58.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.353+0000 7f3c6b0b4640 1 -- start start 2023-10-18T10:25:58.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c6b0b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c6406af00 0x7f3c641eece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c6b0b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c640abc10 0x7f3c641563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c6b0b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640c96b0 0x7f3c64156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c6b0b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640ae000 con 0x7f3c640c96b0 2023-10-18T10:25:58.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c6b0b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640add20 con 0x7f3c6406af00 2023-10-18T10:25:58.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c6b0b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c64153b00 con 0x7f3c640abc10 2023-10-18T10:25:58.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640c96b0 0x7f3c64156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640c96b0 0x7f3c64156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36398/0 (socket says 172.21.15.125:36398) 2023-10-18T10:25:58.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 -- 172.21.15.125:0/3497148355 learned_addr learned my addr 172.21.15.125:0/3497148355 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:58.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c68e50640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c6406af00 0x7f3c641eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c63fff640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c640abc10 0x7f3c641563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 -- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c640abc10 msgr2=0x7f3c641563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c640abc10 0x7f3c641563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 -- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c6406af00 msgr2=0x7f3c641eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c6406af00 0x7f3c641eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 -- 172.21.15.125:0/3497148355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c009900 con 0x7f3c640c96b0 2023-10-18T10:25:58.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c63fff640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c640abc10 0x7f3c641563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:25:58.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c69651640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640c96b0 0x7f3c64156900 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009bf0 tx=0x7f3c5c035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:58.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c617fa640 1 -- 172.21.15.125:0/3497148355 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c034440 con 0x7f3c640c96b0 2023-10-18T10:25:58.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c617fa640 1 -- 172.21.15.125:0/3497148355 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c5c004540 con 0x7f3c640c96b0 2023-10-18T10:25:58.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c64153de0 con 0x7f3c640c96b0 2023-10-18T10:25:58.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c617fa640 1 -- 172.21.15.125:0/3497148355 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c040450 con 0x7f3c640c96b0 2023-10-18T10:25:58.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.354+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c641ef220 con 0x7f3c640c96b0 2023-10-18T10:25:58.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.355+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c64154300 con 0x7f3c640c96b0 2023-10-18T10:25:58.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.356+0000 7f3c617fa640 1 -- 172.21.15.125:0/3497148355 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3c5c00a040 con 0x7f3c640c96b0 2023-10-18T10:25:58.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.356+0000 7f3c617fa640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3c4806ae70 0x7f3c4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.356+0000 7f3c617fa640 1 -- 172.21.15.125:0/3497148355 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3c5c0aca70 con 0x7f3c640c96b0 2023-10-18T10:25:58.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.356+0000 7f3c68e50640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3c4806ae70 0x7f3c4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.357+0000 7f3c68e50640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3c4806ae70 0x7f3c4806d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800abe0 tx=0x7f3c5800a2d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:58.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.360+0000 7f3c617fa640 1 -- 172.21.15.125:0/3497148355 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c64154300 con 0x7f3c640c96b0 2023-10-18T10:25:58.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.528+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c64154300 con 0x7f3c640c96b0 2023-10-18T10:25:58.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.528+0000 7f3c617fa640 1 -- 172.21.15.125:0/3497148355 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c64154300 con 0x7f3c640c96b0 2023-10-18T10:25:58.531 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:58.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.530+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3c4806ae70 msgr2=0x7f3c4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.530+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3c4806ae70 0x7f3c4806d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800abe0 tx=0x7f3c5800a2d0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.530+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640c96b0 msgr2=0x7f3c64156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.530+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640c96b0 0x7f3c64156900 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009bf0 tx=0x7f3c5c035e30 comp rx=0 tx=0).stop 2023-10-18T10:25:58.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.530+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 shutdown_connections 2023-10-18T10:25:58.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.530+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3c4806ae70 0x7f3c4806d330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.530+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c640c96b0 0x7f3c64156900 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.531+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c640abc10 0x7f3c641563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.531+0000 7f3c6b0b4640 1 --2- 172.21.15.125:0/3497148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c6406af00 0x7f3c641eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.531+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 >> 172.21.15.125:0/3497148355 conn(0x7f3c64069930 msgr2=0x7f3c64069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:58.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.531+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 shutdown_connections 2023-10-18T10:25:58.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.531+0000 7f3c6b0b4640 1 -- 172.21.15.125:0/3497148355 wait complete. 2023-10-18T10:25:58.543 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:25:58.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.646+0000 7f4e2d716640 1 -- 172.21.15.125:0/1983418832 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 msgr2=0x7f4e280d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.646+0000 7f4e2d716640 1 --2- 172.21.15.125:0/1983418832 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 0x7f4e280d1910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4e2808a810 tx=0x7f4e2002bd10 comp rx=0 tx=0).stop 2023-10-18T10:25:58.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.646+0000 7f4e2d716640 1 -- 172.21.15.125:0/1983418832 shutdown_connections 2023-10-18T10:25:58.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.646+0000 7f4e2d716640 1 --2- 172.21.15.125:0/1983418832 >> v2:172.21.15.139:3300/0 conn(0x7f4e2815b2a0 0x7f4e280cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.646+0000 7f4e2d716640 1 --2- 172.21.15.125:0/1983418832 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 0x7f4e280d1910 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.646+0000 7f4e2d716640 1 -- 172.21.15.125:0/1983418832 >> 172.21.15.125:0/1983418832 conn(0x7f4e280d0d60 msgr2=0x7f4e280d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:58.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.646+0000 7f4e2d716640 1 -- 172.21.15.125:0/1983418832 shutdown_connections 2023-10-18T10:25:58.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.646+0000 7f4e2d716640 1 -- 172.21.15.125:0/1983418832 wait complete. 2023-10-18T10:25:58.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.647+0000 7f4e2d716640 1 Processor -- start 2023-10-18T10:25:58.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.647+0000 7f4e2d716640 1 -- start start 2023-10-18T10:25:58.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e2d716640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e280ceb40 0x7f4e2814c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e2d716640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 0x7f4e2815e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e2d716640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2815b2a0 0x7f4e280c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e2d716640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280c8840 con 0x7f4e280cf340 2023-10-18T10:25:58.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e2d716640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2814cbb0 con 0x7f4e2815b2a0 2023-10-18T10:25:58.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e2d716640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280c8a50 con 0x7f4e280ceb40 2023-10-18T10:25:58.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e267fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 0x7f4e2815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e267fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 0x7f4e2815e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36406/0 (socket says 172.21.15.125:36406) 2023-10-18T10:25:58.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e267fc640 1 -- 172.21.15.125:0/2865259690 learned_addr learned my addr 172.21.15.125:0/2865259690 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:58.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e277fe640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2815b2a0 0x7f4e280c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e26ffd640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e280ceb40 0x7f4e2814c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e267fc640 1 -- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e280ceb40 msgr2=0x7f4e2814c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e267fc640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e280ceb40 0x7f4e2814c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e267fc640 1 -- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2815b2a0 msgr2=0x7f4e280c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e267fc640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2815b2a0 0x7f4e280c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e267fc640 1 -- 172.21.15.125:0/2865259690 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e20009560 con 0x7f4e280cf340 2023-10-18T10:25:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e277fe640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2815b2a0 0x7f4e280c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.648+0000 7f4e26ffd640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e280ceb40 0x7f4e2814c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.649+0000 7f4e267fc640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 0x7f4e2815e230 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00b410 tx=0x7f4e1c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.649+0000 7f4e0f7fe640 1 -- 172.21.15.125:0/2865259690 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c011830 con 0x7f4e280cf340 2023-10-18T10:25:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.649+0000 7f4e0f7fe640 1 -- 172.21.15.125:0/2865259690 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e1c011aa0 con 0x7f4e280cf340 2023-10-18T10:25:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.649+0000 7f4e0f7fe640 1 -- 172.21.15.125:0/2865259690 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c010380 con 0x7f4e280cf340 2023-10-18T10:25:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.649+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e280c8d30 con 0x7f4e280cf340 2023-10-18T10:25:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.649+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e280c92f0 con 0x7f4e280cf340 2023-10-18T10:25:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.650+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e280677b0 con 0x7f4e280cf340 2023-10-18T10:25:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.650+0000 7f4e0f7fe640 1 -- 172.21.15.125:0/2865259690 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4e1c0104e0 con 0x7f4e280cf340 2023-10-18T10:25:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.650+0000 7f4e0f7fe640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4e1406ae40 0x7f4e1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.651+0000 7f4e0f7fe640 1 -- 172.21.15.125:0/2865259690 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4e1c089d70 con 0x7f4e280cf340 2023-10-18T10:25:58.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.651+0000 7f4e26ffd640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4e1406ae40 0x7f4e1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.651+0000 7f4e26ffd640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4e1406ae40 0x7f4e1406d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e20035000 tx=0x7f4e20004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:58.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.655+0000 7f4e0f7fe640 1 -- 172.21.15.125:0/2865259690 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e1c054a60 con 0x7f4e280cf340 2023-10-18T10:25:58.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.819+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e280cb9d0 con 0x7f4e280cf340 2023-10-18T10:25:58.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.819+0000 7f4e0f7fe640 1 -- 172.21.15.125:0/2865259690 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e1c0150a0 con 0x7f4e280cf340 2023-10-18T10:25:58.822 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:25:58.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4e1406ae40 msgr2=0x7f4e1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4e1406ae40 0x7f4e1406d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e20035000 tx=0x7f4e20004730 comp rx=0 tx=0).stop 2023-10-18T10:25:58.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 msgr2=0x7f4e2815e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 0x7f4e2815e230 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00b410 tx=0x7f4e1c00b8d0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 shutdown_connections 2023-10-18T10:25:58.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4e1406ae40 0x7f4e1406d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2815b2a0 0x7f4e280c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e280cf340 0x7f4e2815e230 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 --2- 172.21.15.125:0/2865259690 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e280ceb40 0x7f4e2814c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 >> 172.21.15.125:0/2865259690 conn(0x7f4e280d0d60 msgr2=0x7f4e2806b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:58.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 shutdown_connections 2023-10-18T10:25:58.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.821+0000 7f4e2d716640 1 -- 172.21.15.125:0/2865259690 wait complete. 2023-10-18T10:25:58.834 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:25:58.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.937+0000 7fc5adebb640 1 -- 172.21.15.125:0/58076311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80cbd20 msgr2=0x7fc5a8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.937+0000 7fc5adebb640 1 --2- 172.21.15.125:0/58076311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80cbd20 0x7fc5a8158300 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc59800a940 tx=0x7fc59802d090 comp rx=0 tx=0).stop 2023-10-18T10:25:58.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.937+0000 7fc5adebb640 1 -- 172.21.15.125:0/58076311 shutdown_connections 2023-10-18T10:25:58.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.937+0000 7fc5adebb640 1 --2- 172.21.15.125:0/58076311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80cbd20 0x7fc5a8158300 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.937+0000 7fc5adebb640 1 -- 172.21.15.125:0/58076311 >> 172.21.15.125:0/58076311 conn(0x7fc5a8069930 msgr2=0x7fc5a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:58.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.937+0000 7fc5adebb640 1 -- 172.21.15.125:0/58076311 shutdown_connections 2023-10-18T10:25:58.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.937+0000 7fc5adebb640 1 -- 172.21.15.125:0/58076311 wait complete. 2023-10-18T10:25:58.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.938+0000 7fc5adebb640 1 Processor -- start 2023-10-18T10:25:58.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.938+0000 7fc5adebb640 1 -- start start 2023-10-18T10:25:58.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5adebb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5a80afa20 0x7fc5a813e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5adebb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80b0530 0x7fc5a814cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5adebb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a80cbd20 0x7fc5a80aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5adebb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a813e820 con 0x7fc5a80b0530 2023-10-18T10:25:58.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5adebb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a814dad0 con 0x7fc5a80cbd20 2023-10-18T10:25:58.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5adebb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a814ddb0 con 0x7fc5a80afa20 2023-10-18T10:25:58.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80b0530 0x7fc5a814cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80b0530 0x7fc5a814cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36432/0 (socket says 172.21.15.125:36432) 2023-10-18T10:25:58.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 -- 172.21.15.125:0/2520809323 learned_addr learned my addr 172.21.15.125:0/2520809323 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:58.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a77fe640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5a80afa20 0x7fc5a813e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 -- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5a80afa20 msgr2=0x7fc5a813e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5a80afa20 0x7fc5a813e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 -- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a80cbd20 msgr2=0x7fc5a80aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:58.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a7fff640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a80cbd20 0x7fc5a80aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a80cbd20 0x7fc5a80aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:58.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 -- 172.21.15.125:0/2520809323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59800a5a0 con 0x7fc5a80b0530 2023-10-18T10:25:58.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a77fe640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5a80afa20 0x7fc5a813e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:58.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5a6ffd640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80b0530 0x7fc5a814cfe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00c4b0 tx=0x7fc59c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:58.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc58ffff640 1 -- 172.21.15.125:0/2520809323 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c005480 con 0x7fc5a80b0530 2023-10-18T10:25:58.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc58ffff640 1 -- 172.21.15.125:0/2520809323 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc59c0036e0 con 0x7fc5a80b0530 2023-10-18T10:25:58.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a814e090 con 0x7fc5a80b0530 2023-10-18T10:25:58.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.939+0000 7fc58ffff640 1 -- 172.21.15.125:0/2520809323 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c003b30 con 0x7fc5a80b0530 2023-10-18T10:25:58.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.940+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a814e650 con 0x7fc5a80b0530 2023-10-18T10:25:58.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.941+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a814e870 con 0x7fc5a80b0530 2023-10-18T10:25:58.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.941+0000 7fc58ffff640 1 -- 172.21.15.125:0/2520809323 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc59c003c90 con 0x7fc5a80b0530 2023-10-18T10:25:58.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.941+0000 7fc58ffff640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc59406ae40 0x7fc59406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:58.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.941+0000 7fc58ffff640 1 -- 172.21.15.125:0/2520809323 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc59c089250 con 0x7fc5a80b0530 2023-10-18T10:25:58.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.941+0000 7fc5a77fe640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc59406ae40 0x7fc59406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:58.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.942+0000 7fc5a77fe640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc59406ae40 0x7fc59406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc598037000 tx=0x7fc598004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:58.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:58.946+0000 7fc58ffff640 1 -- 172.21.15.125:0/2520809323 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc59c053eb0 con 0x7fc5a80b0530 2023-10-18T10:25:59.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.110+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc5a80690e0 con 0x7fc5a80b0530 2023-10-18T10:25:59.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.110+0000 7fc58ffff640 1 -- 172.21.15.125:0/2520809323 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc59c017020 con 0x7fc5a80b0530 2023-10-18T10:25:59.113 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:25:59.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc59406ae40 msgr2=0x7fc59406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc59406ae40 0x7fc59406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc598037000 tx=0x7fc598004730 comp rx=0 tx=0).stop 2023-10-18T10:25:59.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80b0530 msgr2=0x7fc5a814cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80b0530 0x7fc5a814cfe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00c4b0 tx=0x7fc59c00c970 comp rx=0 tx=0).stop 2023-10-18T10:25:59.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 shutdown_connections 2023-10-18T10:25:59.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc59406ae40 0x7fc59406d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a80cbd20 0x7fc5a80aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc5a80b0530 0x7fc5a814cfe0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 --2- 172.21.15.125:0/2520809323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5a80afa20 0x7fc5a813e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 >> 172.21.15.125:0/2520809323 conn(0x7fc5a8069930 msgr2=0x7fc5a81562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:59.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 shutdown_connections 2023-10-18T10:25:59.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.112+0000 7fc5adebb640 1 -- 172.21.15.125:0/2520809323 wait complete. 2023-10-18T10:25:59.125 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:25:59.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.228+0000 7fe9de3b6640 1 -- 172.21.15.125:0/830479613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9c0003970 msgr2=0x7fe9c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.228+0000 7fe9de3b6640 1 --2- 172.21.15.125:0/830479613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9c0003970 0x7fe9c0003d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009b50 tx=0x7fe9c00057f0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.228+0000 7fe9dc953640 1 -- 172.21.15.125:0/830479613 reap_dead start 2023-10-18T10:25:59.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.228+0000 7fe9de3b6640 1 -- 172.21.15.125:0/830479613 shutdown_connections 2023-10-18T10:25:59.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.228+0000 7fe9de3b6640 1 -- 172.21.15.125:0/830479613 >> 172.21.15.125:0/830479613 conn(0x7fe9d806fae0 msgr2=0x7fe9d806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:59.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.228+0000 7fe9de3b6640 1 -- 172.21.15.125:0/830479613 shutdown_connections 2023-10-18T10:25:59.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.228+0000 7fe9de3b6640 1 -- 172.21.15.125:0/830479613 wait complete. 2023-10-18T10:25:59.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.229+0000 7fe9de3b6640 1 Processor -- start 2023-10-18T10:25:59.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.229+0000 7fe9de3b6640 1 -- start start 2023-10-18T10:25:59.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9de3b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c0003970 0x7fe9d813c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9de3b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9d81ed9c0 0x7fe9d80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9de3b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9d80cc9c0 0x7fe9d81549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9de3b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80cce40 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9de3b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8150d20 con 0x7fe9c0003970 2023-10-18T10:25:59.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9de3b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8150f90 con 0x7fe9d80cc9c0 2023-10-18T10:25:59.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9d81ed9c0 0x7fe9d80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9d81ed9c0 0x7fe9d80cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36446/0 (socket says 172.21.15.125:36446) 2023-10-18T10:25:59.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d77fe640 1 -- 172.21.15.125:0/2101950550 learned_addr learned my addr 172.21.15.125:0/2101950550 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:59.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9dc953640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9d80cc9c0 0x7fe9d81549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d7fff640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c0003970 0x7fe9d813c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d77fe640 1 -- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9d80cc9c0 msgr2=0x7fe9d81549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d77fe640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9d80cc9c0 0x7fe9d81549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d77fe640 1 -- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c0003970 msgr2=0x7fe9d813c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d77fe640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c0003970 0x7fe9d813c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.230+0000 7fe9d77fe640 1 -- 172.21.15.125:0/2101950550 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d8151270 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.231+0000 7fe9d77fe640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9d81ed9c0 0x7fe9d80cc480 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800e6d0 tx=0x7fe9c80017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:59.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.231+0000 7fe9d4ff9640 1 -- 172.21.15.125:0/2101950550 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8013070 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.231+0000 7fe9d4ff9640 1 -- 172.21.15.125:0/2101950550 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe9c800a7c0 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.231+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d8151550 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.231+0000 7fe9d4ff9640 1 -- 172.21.15.125:0/2101950550 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800abb0 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.231+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d81ede20 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.232+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d80690e0 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.232+0000 7fe9d4ff9640 1 -- 172.21.15.125:0/2101950550 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe9c800ad10 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.233+0000 7fe9d4ff9640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9c406ae70 0x7fe9c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.233+0000 7fe9d4ff9640 1 -- 172.21.15.125:0/2101950550 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe9c808b330 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.233+0000 7fe9d7fff640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9c406ae70 0x7fe9c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.233+0000 7fe9d7fff640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9c406ae70 0x7fe9c406d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc036000 tx=0x7fe9cc035040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:59.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.237+0000 7fe9d4ff9640 1 -- 172.21.15.125:0/2101950550 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9c8055f90 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.414+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe9d8158820 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.414+0000 7fe9d4ff9640 1 -- 172.21.15.125:0/2101950550 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe9c801c020 con 0x7fe9d81ed9c0 2023-10-18T10:25:59.417 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:25:59.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9c406ae70 msgr2=0x7fe9c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9c406ae70 0x7fe9c406d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc036000 tx=0x7fe9cc035040 comp rx=0 tx=0).stop 2023-10-18T10:25:59.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9d81ed9c0 msgr2=0x7fe9d80cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9d81ed9c0 0x7fe9d80cc480 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800e6d0 tx=0x7fe9c80017e0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 shutdown_connections 2023-10-18T10:25:59.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe9c406ae70 0x7fe9c406d330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9d80cc9c0 0x7fe9d81549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9d81ed9c0 0x7fe9d80cc480 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 --2- 172.21.15.125:0/2101950550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c0003970 0x7fe9d813c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 >> 172.21.15.125:0/2101950550 conn(0x7fe9d806fae0 msgr2=0x7fe9d814fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:59.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 shutdown_connections 2023-10-18T10:25:59.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.416+0000 7fe9de3b6640 1 -- 172.21.15.125:0/2101950550 wait complete. 2023-10-18T10:25:59.429 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:25:59.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:59.430 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-18T10:25:59.456 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:25:59.457 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:25:59.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:25:59.458 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:25:59.511 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:25:59.512 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:59.531 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:25:59.531 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:25:59.589 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:25:59.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.689+0000 7f947c9b7640 1 -- 172.21.15.125:0/1322194207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478070af0 msgr2=0x7f94780affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.689+0000 7f947c9b7640 1 --2- 172.21.15.125:0/1322194207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478070af0 0x7f94780affe0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f94780cbd00 tx=0x7f947002c260 comp rx=0 tx=0).stop 2023-10-18T10:25:59.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.689+0000 7f947c9b7640 1 -- 172.21.15.125:0/1322194207 shutdown_connections 2023-10-18T10:25:59.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.689+0000 7f947c9b7640 1 --2- 172.21.15.125:0/1322194207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478070af0 0x7f94780affe0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.689+0000 7f947c9b7640 1 --2- 172.21.15.125:0/1322194207 >> v2:172.21.15.139:3300/0 conn(0x7f947813e520 0x7f947806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.689+0000 7f947c9b7640 1 -- 172.21.15.125:0/1322194207 >> 172.21.15.125:0/1322194207 conn(0x7f9478069930 msgr2=0x7f9478069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:59.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.689+0000 7f947c9b7640 1 -- 172.21.15.125:0/1322194207 shutdown_connections 2023-10-18T10:25:59.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.690+0000 7f947c9b7640 1 -- 172.21.15.125:0/1322194207 wait complete. 2023-10-18T10:25:59.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.691+0000 7f947c9b7640 1 Processor -- start 2023-10-18T10:25:59.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.691+0000 7f947c9b7640 1 -- start start 2023-10-18T10:25:59.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.691+0000 7f947c9b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9478070af0 0x7f94781e1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.691+0000 7f947c9b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947813e520 0x7f94780a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.691+0000 7f947c9b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478141690 0x7f94780a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.691+0000 7f947c9b7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781e1ad0 con 0x7f9478141690 2023-10-18T10:25:59.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.691+0000 7f947c9b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947813dbd0 con 0x7f947813e520 2023-10-18T10:25:59.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.691+0000 7f947c9b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94781f04f0 con 0x7f9478070af0 2023-10-18T10:25:59.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478141690 0x7f94780a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478141690 0x7f94780a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36462/0 (socket says 172.21.15.125:36462) 2023-10-18T10:25:59.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 -- 172.21.15.125:0/2209838279 learned_addr learned my addr 172.21.15.125:0/2209838279 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:59.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f946ffff640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947813e520 0x7f94780a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f947659c640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9478070af0 0x7f94781e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 -- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9478070af0 msgr2=0x7f94781e1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9478070af0 0x7f94781e1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 -- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947813e520 msgr2=0x7f94780a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947813e520 0x7f94780a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 -- 172.21.15.125:0/2209838279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470009900 con 0x7f9478141690 2023-10-18T10:25:59.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f946ffff640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947813e520 0x7f94780a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:59.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f947659c640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9478070af0 0x7f94781e1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:25:59.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f9476d9d640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478141690 0x7f94780a9c50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9470037040 tx=0x7f9470003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:59.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f946effd640 1 -- 172.21.15.125:0/2209838279 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470004240 con 0x7f9478141690 2023-10-18T10:25:59.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f946effd640 1 -- 172.21.15.125:0/2209838279 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9470004490 con 0x7f9478141690 2023-10-18T10:25:59.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f946effd640 1 -- 172.21.15.125:0/2209838279 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470004880 con 0x7f9478141690 2023-10-18T10:25:59.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94781f07d0 con 0x7f9478141690 2023-10-18T10:25:59.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.692+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94781f01f0 con 0x7f9478141690 2023-10-18T10:25:59.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.693+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94781f0cf0 con 0x7f9478141690 2023-10-18T10:25:59.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.694+0000 7f946effd640 1 -- 172.21.15.125:0/2209838279 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f947003b040 con 0x7f9478141690 2023-10-18T10:25:59.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.694+0000 7f946effd640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f945806ae70 0x7f945806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.694+0000 7f946effd640 1 -- 172.21.15.125:0/2209838279 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f94700ac750 con 0x7f9478141690 2023-10-18T10:25:59.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.694+0000 7f947659c640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f945806ae70 0x7f945806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.695+0000 7f947659c640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f945806ae70 0x7f945806d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94780ae370 tx=0x7f9468005e70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:59.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.698+0000 7f946effd640 1 -- 172.21.15.125:0/2209838279 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94781f0cf0 con 0x7f9478141690 2023-10-18T10:25:59.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.865+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94781f0cf0 con 0x7f9478141690 2023-10-18T10:25:59.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.867+0000 7f946effd640 1 -- 172.21.15.125:0/2209838279 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f94781f0cf0 con 0x7f9478141690 2023-10-18T10:25:59.869 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:25:59.869 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:25:59.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f945806ae70 msgr2=0x7f945806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f945806ae70 0x7f945806d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94780ae370 tx=0x7f9468005e70 comp rx=0 tx=0).stop 2023-10-18T10:25:59.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478141690 msgr2=0x7f94780a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478141690 0x7f94780a9c50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9470037040 tx=0x7f9470003ba0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 shutdown_connections 2023-10-18T10:25:59.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f945806ae70 0x7f945806d330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947813e520 0x7f94780a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9478141690 0x7f94780a9c50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 --2- 172.21.15.125:0/2209838279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9478070af0 0x7f94781e1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.868+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 >> 172.21.15.125:0/2209838279 conn(0x7f9478069930 msgr2=0x7f94780ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:59.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.869+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 shutdown_connections 2023-10-18T10:25:59.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.869+0000 7f947c9b7640 1 -- 172.21.15.125:0/2209838279 wait complete. 2023-10-18T10:25:59.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:25:59.882 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:25:59.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.987+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3442058399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 msgr2=0x7f3e98009b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.987+0000 7f3eb3cc2640 1 --2- 172.21.15.125:0/3442058399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 0x7f3e98009b40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009b50 tx=0x7f3e980037d0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.987+0000 7f3eb225f640 1 -- 172.21.15.125:0/3442058399 reap_dead start 2023-10-18T10:25:59.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.987+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3442058399 shutdown_connections 2023-10-18T10:25:59.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.987+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3442058399 >> 172.21.15.125:0/3442058399 conn(0x7f3eac069930 msgr2=0x7f3eac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:25:59.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.988+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3442058399 shutdown_connections 2023-10-18T10:25:59.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.988+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3442058399 wait complete. 2023-10-18T10:25:59.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb3cc2640 1 Processor -- start 2023-10-18T10:25:59.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb3cc2640 1 -- start start 2023-10-18T10:25:59.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb3cc2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 0x7f3eac1f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb3cc2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3eac155e10 0x7f3eac1e4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb3cc2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3eac1e5180 0x7f3eac138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb3cc2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0ad160 con 0x7f3e98007710 2023-10-18T10:25:59.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb3cc2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1e5630 con 0x7f3eac155e10 2023-10-18T10:25:59.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb3cc2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac138e40 con 0x7f3eac1e5180 2023-10-18T10:25:59.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb1a5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 0x7f3eac1f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.989+0000 7f3eb1a5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 0x7f3eac1f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36492/0 (socket says 172.21.15.125:36492) 2023-10-18T10:25:59.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb1a5e640 1 -- 172.21.15.125:0/3975663925 learned_addr learned my addr 172.21.15.125:0/3975663925 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:25:59.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb225f640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3eac1e5180 0x7f3eac138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb125d640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3eac155e10 0x7f3eac1e4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:25:59.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb1a5e640 1 -- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3eac1e5180 msgr2=0x7f3eac138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb1a5e640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3eac1e5180 0x7f3eac138110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb1a5e640 1 -- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3eac155e10 msgr2=0x7f3eac1e4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:25:59.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb1a5e640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3eac155e10 0x7f3eac1e4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:25:59.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb1a5e640 1 -- 172.21.15.125:0/3975663925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac0ac7c0 con 0x7f3e98007710 2023-10-18T10:25:59.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb1a5e640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 0x7f3eac1f0f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8010bf0 tx=0x7f3ea800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:25:59.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3ea27fc640 1 -- 172.21.15.125:0/3975663925 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8003b70 con 0x7f3e98007710 2023-10-18T10:25:59.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3ea27fc640 1 -- 172.21.15.125:0/3975663925 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ea8003cd0 con 0x7f3e98007710 2023-10-18T10:25:59.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac0acaa0 con 0x7f3e98007710 2023-10-18T10:25:59.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3ea27fc640 1 -- 172.21.15.125:0/3975663925 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80042b0 con 0x7f3e98007710 2023-10-18T10:25:59.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.990+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac1f1480 con 0x7f3e98007710 2023-10-18T10:25:59.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.991+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac0ad3d0 con 0x7f3e98007710 2023-10-18T10:25:59.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.991+0000 7f3ea27fc640 1 -- 172.21.15.125:0/3975663925 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ea8004410 con 0x7f3e98007710 2023-10-18T10:25:59.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.992+0000 7f3ea27fc640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3e9006ae70 0x7f3e9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:25:59.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.992+0000 7f3ea27fc640 1 -- 172.21.15.125:0/3975663925 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3ea808ab90 con 0x7f3e98007710 2023-10-18T10:25:59.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.992+0000 7f3eb125d640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3e9006ae70 0x7f3e9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.993+0000 7f3eb125d640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3e9006ae70 0x7f3e9006d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c036000 tx=0x7f3e9c0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:00.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:25:59.996+0000 7f3ea27fc640 1 -- 172.21.15.125:0/3975663925 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ea8055880 con 0x7f3e98007710 2023-10-18T10:26:00.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.160+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3eac06d140 con 0x7f3e98007710 2023-10-18T10:26:00.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.161+0000 7f3ea27fc640 1 -- 172.21.15.125:0/3975663925 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ea8020070 con 0x7f3e98007710 2023-10-18T10:26:00.163 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:26:00.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3e9006ae70 msgr2=0x7f3e9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3e9006ae70 0x7f3e9006d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c036000 tx=0x7f3e9c0046c0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 msgr2=0x7f3eac1f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 0x7f3eac1f0f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8010bf0 tx=0x7f3ea800e6c0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 shutdown_connections 2023-10-18T10:26:00.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f3e9006ae70 0x7f3e9006d330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3eac155e10 0x7f3eac1e4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3eac1e5180 0x7f3eac138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 --2- 172.21.15.125:0/3975663925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e98007710 0x7f3eac1f0f40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 >> 172.21.15.125:0/3975663925 conn(0x7f3eac069930 msgr2=0x7f3eac0af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:00.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 shutdown_connections 2023-10-18T10:26:00.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.163+0000 7f3eb3cc2640 1 -- 172.21.15.125:0/3975663925 wait complete. 2023-10-18T10:26:00.175 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:26:00.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.280+0000 7f6d13df5640 1 -- 172.21.15.125:0/3671146038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c06af00 msgr2=0x7f6d0c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.280+0000 7f6d13df5640 1 --2- 172.21.15.125:0/3671146038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c06af00 0x7f6d0c06b2e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009860 tx=0x7f6d0802bd60 comp rx=0 tx=0).stop 2023-10-18T10:26:00.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.280+0000 7f6d13df5640 1 -- 172.21.15.125:0/3671146038 shutdown_connections 2023-10-18T10:26:00.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.280+0000 7f6d13df5640 1 --2- 172.21.15.125:0/3671146038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c06af00 0x7f6d0c06b2e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.280+0000 7f6d13df5640 1 -- 172.21.15.125:0/3671146038 >> 172.21.15.125:0/3671146038 conn(0x7f6d0c069930 msgr2=0x7f6d0c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:00.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.280+0000 7f6d13df5640 1 -- 172.21.15.125:0/3671146038 shutdown_connections 2023-10-18T10:26:00.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.280+0000 7f6d13df5640 1 -- 172.21.15.125:0/3671146038 wait complete. 2023-10-18T10:26:00.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.281+0000 7f6d13df5640 1 Processor -- start 2023-10-18T10:26:00.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d13df5640 1 -- start start 2023-10-18T10:26:00.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d13df5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d0c06af00 0x7f6d0c0af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d13df5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d0c0abc10 0x7f6d0c1efb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d13df5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c0c96b0 0x7f6d0c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d13df5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0b0990 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d13df5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c1f0080 con 0x7f6d0c06af00 2023-10-18T10:26:00.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d13df5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c1f0210 con 0x7f6d0c0abc10 2023-10-18T10:26:00.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d12392640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c0c96b0 0x7f6d0c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d12392640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c0c96b0 0x7f6d0c151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36496/0 (socket says 172.21.15.125:36496) 2023-10-18T10:26:00.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d12392640 1 -- 172.21.15.125:0/1194755006 learned_addr learned my addr 172.21.15.125:0/1194755006 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:00.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d11b91640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d0c06af00 0x7f6d0c0af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.282+0000 7f6d11390640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d0c0abc10 0x7f6d0c1efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d12392640 1 -- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d0c0abc10 msgr2=0x7f6d0c1efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d12392640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d0c0abc10 0x7f6d0c1efb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d12392640 1 -- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d0c06af00 msgr2=0x7f6d0c0af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d12392640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d0c06af00 0x7f6d0c0af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d12392640 1 -- 172.21.15.125:0/1194755006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d08009490 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d11390640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d0c0abc10 0x7f6d0c1efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:00.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d12392640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c0c96b0 0x7f6d0c151de0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400c7a0 tx=0x7f6d0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:00.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6cfe7fc640 1 -- 172.21.15.125:0/1194755006 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04003b70 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6cfe7fc640 1 -- 172.21.15.125:0/1194755006 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d04003cf0 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c0afe10 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6cfe7fc640 1 -- 172.21.15.125:0/1194755006 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400aab0 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.283+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0c0b03d0 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.284+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0c0690e0 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.284+0000 7f6cfe7fc640 1 -- 172.21.15.125:0/1194755006 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6d0400ac10 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.285+0000 7f6cfe7fc640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6cf006ae40 0x7f6cf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.285+0000 7f6cfe7fc640 1 -- 172.21.15.125:0/1194755006 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6d04089910 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.285+0000 7f6d11b91640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6cf006ae40 0x7f6cf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.286+0000 7f6d11b91640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6cf006ae40 0x7f6cf006d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d08036000 tx=0x7f6d08004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:00.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.289+0000 7f6cfe7fc640 1 -- 172.21.15.125:0/1194755006 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d04054600 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.456+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d0c0b1950 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.456+0000 7f6cfe7fc640 1 -- 172.21.15.125:0/1194755006 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d04057c20 con 0x7f6d0c0c96b0 2023-10-18T10:26:00.458 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:26:00.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6cf006ae40 msgr2=0x7f6cf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6cf006ae40 0x7f6cf006d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d08036000 tx=0x7f6d08004730 comp rx=0 tx=0).stop 2023-10-18T10:26:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c0c96b0 msgr2=0x7f6d0c151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c0c96b0 0x7f6d0c151de0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400c7a0 tx=0x7f6d0400cc60 comp rx=0 tx=0).stop 2023-10-18T10:26:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 shutdown_connections 2023-10-18T10:26:00.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6cf006ae40 0x7f6cf006d300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d0c0c96b0 0x7f6d0c151de0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d0c0abc10 0x7f6d0c1efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 --2- 172.21.15.125:0/1194755006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d0c06af00 0x7f6d0c0af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 >> 172.21.15.125:0/1194755006 conn(0x7f6d0c069930 msgr2=0x7f6d0c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:00.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 shutdown_connections 2023-10-18T10:26:00.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.458+0000 7f6d13df5640 1 -- 172.21.15.125:0/1194755006 wait complete. 2023-10-18T10:26:00.471 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:26:00.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.574+0000 7f99458a2640 1 -- 172.21.15.125:0/1414020743 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f994015e5e0 msgr2=0x7f99400ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.574+0000 7f99458a2640 1 --2- 172.21.15.125:0/1414020743 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f994015e5e0 0x7f99400ce9a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9930005fd0 tx=0x7f993002c240 comp rx=0 tx=0).stop 2023-10-18T10:26:00.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.574+0000 7f99458a2640 1 -- 172.21.15.125:0/1414020743 shutdown_connections 2023-10-18T10:26:00.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.574+0000 7f99458a2640 1 --2- 172.21.15.125:0/1414020743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99400b1ca0 0x7f99400cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.574+0000 7f99458a2640 1 --2- 172.21.15.125:0/1414020743 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f994015e5e0 0x7f99400ce9a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.574+0000 7f99458a2640 1 -- 172.21.15.125:0/1414020743 >> 172.21.15.125:0/1414020743 conn(0x7f99400d0c50 msgr2=0x7f99400d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:00.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.574+0000 7f99458a2640 1 -- 172.21.15.125:0/1414020743 shutdown_connections 2023-10-18T10:26:00.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.574+0000 7f99458a2640 1 -- 172.21.15.125:0/1414020743 wait complete. 2023-10-18T10:26:00.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.575+0000 7f99458a2640 1 Processor -- start 2023-10-18T10:26:00.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.575+0000 7f99458a2640 1 -- start start 2023-10-18T10:26:00.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f99458a2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99400b1ca0 0x7f99400ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f99458a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99400cf6e0 0x7f994015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f99458a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994015e5e0 0x7f99400ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f99458a2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400cb020 con 0x7f99400cf6e0 2023-10-18T10:26:00.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f99458a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400c8740 con 0x7f994015e5e0 2023-10-18T10:26:00.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f99458a2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99400c8a20 con 0x7f99400b1ca0 2023-10-18T10:26:00.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99400cf6e0 0x7f994015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99400cf6e0 0x7f994015aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36520/0 (socket says 172.21.15.125:36520) 2023-10-18T10:26:00.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 -- 172.21.15.125:0/1932959391 learned_addr learned my addr 172.21.15.125:0/1932959391 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:00.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993effd640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99400b1ca0 0x7f99400ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993f7fe640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994015e5e0 0x7f99400ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 -- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99400b1ca0 msgr2=0x7f99400ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99400b1ca0 0x7f99400ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 -- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994015e5e0 msgr2=0x7f99400ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994015e5e0 0x7f99400ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 -- 172.21.15.125:0/1932959391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930009900 con 0x7f99400cf6e0 2023-10-18T10:26:00.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993f7fe640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994015e5e0 0x7f99400ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:00.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f993e7fc640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99400cf6e0 0x7f994015aef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f993400c920 tx=0x7f993400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:00.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f9927fff640 1 -- 172.21.15.125:0/1932959391 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934007c70 con 0x7f99400cf6e0 2023-10-18T10:26:00.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f9927fff640 1 -- 172.21.15.125:0/1932959391 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f993400f450 con 0x7f99400cf6e0 2023-10-18T10:26:00.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99400c8d00 con 0x7f99400cf6e0 2023-10-18T10:26:00.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f9927fff640 1 -- 172.21.15.125:0/1932959391 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993400f860 con 0x7f99400cf6e0 2023-10-18T10:26:00.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.576+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99400ae5a0 con 0x7f99400cf6e0 2023-10-18T10:26:00.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.577+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99400c9220 con 0x7f99400cf6e0 2023-10-18T10:26:00.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.578+0000 7f9927fff640 1 -- 172.21.15.125:0/1932959391 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f993400f9c0 con 0x7f99400cf6e0 2023-10-18T10:26:00.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.578+0000 7f9927fff640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae40 0x7f992c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.578+0000 7f9927fff640 1 -- 172.21.15.125:0/1932959391 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f993408a2f0 con 0x7f99400cf6e0 2023-10-18T10:26:00.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.579+0000 7f993effd640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae40 0x7f992c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.579+0000 7f993effd640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae40 0x7f992c06d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9930036000 tx=0x7f9930004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:00.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.582+0000 7f9927fff640 1 -- 172.21.15.125:0/1932959391 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99400c9220 con 0x7f99400cf6e0 2023-10-18T10:26:00.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.749+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99400c9220 con 0x7f99400cf6e0 2023-10-18T10:26:00.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.749+0000 7f9927fff640 1 -- 172.21.15.125:0/1932959391 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99400c9220 con 0x7f99400cf6e0 2023-10-18T10:26:00.751 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:26:00.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae40 msgr2=0x7f992c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae40 0x7f992c06d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9930036000 tx=0x7f9930004ab0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99400cf6e0 msgr2=0x7f994015aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99400cf6e0 0x7f994015aef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f993400c920 tx=0x7f993400cde0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 shutdown_connections 2023-10-18T10:26:00.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae40 0x7f992c06d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994015e5e0 0x7f99400ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99400cf6e0 0x7f994015aef0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 --2- 172.21.15.125:0/1932959391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99400b1ca0 0x7f99400ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.751+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 >> 172.21.15.125:0/1932959391 conn(0x7f99400d0c50 msgr2=0x7f994006acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:00.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.752+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 shutdown_connections 2023-10-18T10:26:00.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.752+0000 7f99458a2640 1 -- 172.21.15.125:0/1932959391 wait complete. 2023-10-18T10:26:00.764 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:26:00.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 -- 172.21.15.125:0/3200034722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00afa20 msgr2=0x7f76f00705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 --2- 172.21.15.125:0/3200034722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00afa20 0x7f76f00705a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f76f00b1450 tx=0x7f76e402bd90 comp rx=0 tx=0).stop 2023-10-18T10:26:00.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 -- 172.21.15.125:0/3200034722 shutdown_connections 2023-10-18T10:26:00.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 --2- 172.21.15.125:0/3200034722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00afa20 0x7f76f00705a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 --2- 172.21.15.125:0/3200034722 >> v2:172.21.15.139:3300/0 conn(0x7f76f00b0530 0x7f76f00b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 --2- 172.21.15.125:0/3200034722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00cbd20 0x7f76f0158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 -- 172.21.15.125:0/3200034722 >> 172.21.15.125:0/3200034722 conn(0x7f76f0069930 msgr2=0x7f76f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:00.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 -- 172.21.15.125:0/3200034722 shutdown_connections 2023-10-18T10:26:00.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.866+0000 7f76f6426640 1 -- 172.21.15.125:0/3200034722 wait complete. 2023-10-18T10:26:00.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.867+0000 7f76f6426640 1 Processor -- start 2023-10-18T10:26:00.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.868+0000 7f76f6426640 1 -- start start 2023-10-18T10:26:00.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.868+0000 7f76f6426640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00afa20 0x7f76f00aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.868+0000 7f76f6426640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00b0530 0x7f76f00af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.868+0000 7f76f6426640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00cbd20 0x7f76f01468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.868+0000 7f76f6426640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f014c810 con 0x7f76f00cbd20 2023-10-18T10:26:00.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.868+0000 7f76f6426640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f014bbc0 con 0x7f76f00afa20 2023-10-18T10:26:00.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.868+0000 7f76f6426640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f014b690 con 0x7f76f00b0530 2023-10-18T10:26:00.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.868+0000 7f76f49c3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00cbd20 0x7f76f01468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f49c3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00cbd20 0x7f76f01468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36538/0 (socket says 172.21.15.125:36538) 2023-10-18T10:26:00.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f49c3640 1 -- 172.21.15.125:0/1231859055 learned_addr learned my addr 172.21.15.125:0/1231859055 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:00.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76effff640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00afa20 0x7f76f00aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76ef7fe640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00b0530 0x7f76f00af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f49c3640 1 -- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00b0530 msgr2=0x7f76f00af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f49c3640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00b0530 0x7f76f00af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f49c3640 1 -- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00afa20 msgr2=0x7f76f00aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:00.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f49c3640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00afa20 0x7f76f00aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:00.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f49c3640 1 -- 172.21.15.125:0/1231859055 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e4009560 con 0x7f76f00cbd20 2023-10-18T10:26:00.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f49c3640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00cbd20 0x7f76f01468d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f76e800cd10 tx=0x7f76e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:00.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76ecff9640 1 -- 172.21.15.125:0/1231859055 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e800f450 con 0x7f76f00cbd20 2023-10-18T10:26:00.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76ecff9640 1 -- 172.21.15.125:0/1231859055 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76e800f6c0 con 0x7f76f00cbd20 2023-10-18T10:26:00.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76ecff9640 1 -- 172.21.15.125:0/1231859055 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e800fad0 con 0x7f76f00cbd20 2023-10-18T10:26:00.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f014c470 con 0x7f76f00cbd20 2023-10-18T10:26:00.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.869+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f0156330 con 0x7f76f00cbd20 2023-10-18T10:26:00.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.870+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f014c2d0 con 0x7f76f00cbd20 2023-10-18T10:26:00.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.871+0000 7f76ecff9640 1 -- 172.21.15.125:0/1231859055 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f76e800fc30 con 0x7f76f00cbd20 2023-10-18T10:26:00.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.871+0000 7f76ecff9640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f76dc06ae40 0x7f76dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:00.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.871+0000 7f76ecff9640 1 -- 172.21.15.125:0/1231859055 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f76e808a4a0 con 0x7f76f00cbd20 2023-10-18T10:26:00.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.871+0000 7f76effff640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f76dc06ae40 0x7f76dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:00.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.872+0000 7f76effff640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f76dc06ae40 0x7f76dc06d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f76e0005d70 tx=0x7f76e0005d00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:00.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:00.875+0000 7f76ecff9640 1 -- 172.21.15.125:0/1231859055 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76f014c2d0 con 0x7f76f00cbd20 2023-10-18T10:26:01.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.042+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f76f014c2d0 con 0x7f76f00cbd20 2023-10-18T10:26:01.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.042+0000 7f76ecff9640 1 -- 172.21.15.125:0/1231859055 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f76f014c2d0 con 0x7f76f00cbd20 2023-10-18T10:26:01.045 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:26:01.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f76dc06ae40 msgr2=0x7f76dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f76dc06ae40 0x7f76dc06d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f76e0005d70 tx=0x7f76e0005d00 comp rx=0 tx=0).stop 2023-10-18T10:26:01.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00cbd20 msgr2=0x7f76f01468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00cbd20 0x7f76f01468d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f76e800cd10 tx=0x7f76e800a620 comp rx=0 tx=0).stop 2023-10-18T10:26:01.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 shutdown_connections 2023-10-18T10:26:01.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f76dc06ae40 0x7f76dc06d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f00cbd20 0x7f76f01468d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00b0530 0x7f76f00af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 --2- 172.21.15.125:0/1231859055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00afa20 0x7f76f00aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 >> 172.21.15.125:0/1231859055 conn(0x7f76f0069930 msgr2=0x7f76f0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:01.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 shutdown_connections 2023-10-18T10:26:01.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.044+0000 7f76f6426640 1 -- 172.21.15.125:0/1231859055 wait complete. 2023-10-18T10:26:01.057 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:26:01.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 -- 172.21.15.125:0/1300089922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0cf340 msgr2=0x7f26ac0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 --2- 172.21.15.125:0/1300089922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0cf340 0x7f26ac0d1910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f26ac08a120 tx=0x7f26a402c1d0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 -- 172.21.15.125:0/1300089922 shutdown_connections 2023-10-18T10:26:01.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 --2- 172.21.15.125:0/1300089922 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26ac0ceb40 0x7f26ac0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 --2- 172.21.15.125:0/1300089922 >> v2:172.21.15.139:3300/0 conn(0x7f26ac15b2a0 0x7f26ac0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 --2- 172.21.15.125:0/1300089922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0cf340 0x7f26ac0d1910 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 -- 172.21.15.125:0/1300089922 >> 172.21.15.125:0/1300089922 conn(0x7f26ac0d0d60 msgr2=0x7f26ac0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:01.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 -- 172.21.15.125:0/1300089922 shutdown_connections 2023-10-18T10:26:01.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.158+0000 7f26b406f640 1 -- 172.21.15.125:0/1300089922 wait complete. 2023-10-18T10:26:01.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.159+0000 7f26b406f640 1 Processor -- start 2023-10-18T10:26:01.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.159+0000 7f26b406f640 1 -- start start 2023-10-18T10:26:01.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b406f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0ceb40 0x7f26ac1476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b406f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26ac0cf340 0x7f26ac15da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b406f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26ac15b2a0 0x7f26ac15c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b406f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0c7fd0 con 0x7f26ac0ceb40 2023-10-18T10:26:01.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b406f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0c8270 con 0x7f26ac15b2a0 2023-10-18T10:26:01.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b406f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac15d260 con 0x7f26ac0cf340 2023-10-18T10:26:01.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b1e0b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0ceb40 0x7f26ac1476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b1e0b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0ceb40 0x7f26ac1476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36556/0 (socket says 172.21.15.125:36556) 2023-10-18T10:26:01.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b1e0b640 1 -- 172.21.15.125:0/4046106104 learned_addr learned my addr 172.21.15.125:0/4046106104 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:01.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b160a640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26ac0cf340 0x7f26ac15da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b260c640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26ac15b2a0 0x7f26ac15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b1e0b640 1 -- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26ac0cf340 msgr2=0x7f26ac15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b1e0b640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26ac0cf340 0x7f26ac15da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b1e0b640 1 -- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26ac15b2a0 msgr2=0x7f26ac15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b1e0b640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26ac15b2a0 0x7f26ac15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b1e0b640 1 -- 172.21.15.125:0/4046106104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4009900 con 0x7f26ac0ceb40 2023-10-18T10:26:01.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b160a640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26ac0cf340 0x7f26ac15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:01.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.160+0000 7f26b260c640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26ac15b2a0 0x7f26ac15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:01.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.161+0000 7f26b1e0b640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0ceb40 0x7f26ac1476b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f26a4000f80 tx=0x7f26a4035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:01.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.161+0000 7f26aa7fc640 1 -- 172.21.15.125:0/4046106104 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4034440 con 0x7f26ac0ceb40 2023-10-18T10:26:01.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.161+0000 7f26aa7fc640 1 -- 172.21.15.125:0/4046106104 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f26a4004450 con 0x7f26ac0ceb40 2023-10-18T10:26:01.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.161+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac15d540 con 0x7f26ac0ceb40 2023-10-18T10:26:01.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.161+0000 7f26aa7fc640 1 -- 172.21.15.125:0/4046106104 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4004820 con 0x7f26ac0ceb40 2023-10-18T10:26:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.161+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac15dfa0 con 0x7f26ac0ceb40 2023-10-18T10:26:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.162+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac0677b0 con 0x7f26ac0ceb40 2023-10-18T10:26:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.162+0000 7f26aa7fc640 1 -- 172.21.15.125:0/4046106104 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f26a403b040 con 0x7f26ac0ceb40 2023-10-18T10:26:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.163+0000 7f26aa7fc640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f269806ae40 0x7f269806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.163+0000 7f26aa7fc640 1 -- 172.21.15.125:0/4046106104 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f26a40ac8d0 con 0x7f26ac0ceb40 2023-10-18T10:26:01.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.163+0000 7f26b160a640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f269806ae40 0x7f269806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.163+0000 7f26b160a640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f269806ae40 0x7f269806d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f26ac156ad0 tx=0x7f26a0005c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:01.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.167+0000 7f26aa7fc640 1 -- 172.21.15.125:0/4046106104 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26a40775d0 con 0x7f26ac0ceb40 2023-10-18T10:26:01.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.344+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26ac0690e0 con 0x7f26ac0ceb40 2023-10-18T10:26:01.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.344+0000 7f26aa7fc640 1 -- 172.21.15.125:0/4046106104 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f26a40aae60 con 0x7f26ac0ceb40 2023-10-18T10:26:01.347 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:26:01.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f269806ae40 msgr2=0x7f269806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f269806ae40 0x7f269806d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f26ac156ad0 tx=0x7f26a0005c90 comp rx=0 tx=0).stop 2023-10-18T10:26:01.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0ceb40 msgr2=0x7f26ac1476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0ceb40 0x7f26ac1476b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f26a4000f80 tx=0x7f26a4035db0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 shutdown_connections 2023-10-18T10:26:01.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f269806ae40 0x7f269806d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26ac15b2a0 0x7f26ac15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26ac0cf340 0x7f26ac15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 --2- 172.21.15.125:0/4046106104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26ac0ceb40 0x7f26ac1476b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 >> 172.21.15.125:0/4046106104 conn(0x7f26ac0d0d60 msgr2=0x7f26ac0aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:01.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 shutdown_connections 2023-10-18T10:26:01.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.346+0000 7f26b406f640 1 -- 172.21.15.125:0/4046106104 wait complete. 2023-10-18T10:26:01.359 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:26:01.359 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:26:01.378 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:26:01.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.479+0000 7fb17e739640 1 -- 172.21.15.125:0/3744185980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780cbd20 msgr2=0x7fb178158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.479+0000 7fb17e739640 1 --2- 172.21.15.125:0/3744185980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780cbd20 0x7fb178158300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb1780b1450 tx=0x7fb16802bda0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.480+0000 7fb17e739640 1 -- 172.21.15.125:0/3744185980 shutdown_connections 2023-10-18T10:26:01.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.480+0000 7fb17e739640 1 --2- 172.21.15.125:0/3744185980 >> v2:172.21.15.139:3300/0 conn(0x7fb1780b0530 0x7fb1780b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.480+0000 7fb17e739640 1 --2- 172.21.15.125:0/3744185980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1780afa20 0x7fb1780705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.480+0000 7fb17e739640 1 --2- 172.21.15.125:0/3744185980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780cbd20 0x7fb178158300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.480+0000 7fb17e739640 1 -- 172.21.15.125:0/3744185980 >> 172.21.15.125:0/3744185980 conn(0x7fb178069930 msgr2=0x7fb178069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:01.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.480+0000 7fb17e739640 1 -- 172.21.15.125:0/3744185980 shutdown_connections 2023-10-18T10:26:01.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.480+0000 7fb17e739640 1 -- 172.21.15.125:0/3744185980 wait complete. 2023-10-18T10:26:01.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.481+0000 7fb17e739640 1 Processor -- start 2023-10-18T10:26:01.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.481+0000 7fb17e739640 1 -- start start 2023-10-18T10:26:01.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17e739640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780afa20 0x7fb1781f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17e739640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1780b0530 0x7fb17814dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17e739640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1780cbd20 0x7fb17814e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17e739640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb178159330 con 0x7fb1780afa20 2023-10-18T10:26:01.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17e739640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb178159b40 con 0x7fb1780cbd20 2023-10-18T10:26:01.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17e739640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb178159580 con 0x7fb1780b0530 2023-10-18T10:26:01.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780afa20 0x7fb1781f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780afa20 0x7fb1781f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36568/0 (socket says 172.21.15.125:36568) 2023-10-18T10:26:01.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 -- 172.21.15.125:0/1473906254 learned_addr learned my addr 172.21.15.125:0/1473906254 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:01.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17ccd6640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1780cbd20 0x7fb17814e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb1777fe640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1780b0530 0x7fb17814dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 -- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1780b0530 msgr2=0x7fb17814dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1780b0530 0x7fb17814dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 -- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1780cbd20 msgr2=0x7fb17814e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1780cbd20 0x7fb17814e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 -- 172.21.15.125:0/1473906254 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb168009560 con 0x7fb1780afa20 2023-10-18T10:26:01.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17ccd6640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1780cbd20 0x7fb17814e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:01.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb1777fe640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1780b0530 0x7fb17814dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:01.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb177fff640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780afa20 0x7fb1781f1d30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb168009850 tx=0x7fb16802dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:01.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb174ff9640 1 -- 172.21.15.125:0/1473906254 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168003800 con 0x7fb1780afa20 2023-10-18T10:26:01.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb174ff9640 1 -- 172.21.15.125:0/1473906254 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb168003a50 con 0x7fb1780afa20 2023-10-18T10:26:01.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb178159860 con 0x7fb1780afa20 2023-10-18T10:26:01.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.482+0000 7fb174ff9640 1 -- 172.21.15.125:0/1473906254 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168003e20 con 0x7fb1780afa20 2023-10-18T10:26:01.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.483+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1780a68c0 con 0x7fb1780afa20 2023-10-18T10:26:01.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.483+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1780ac400 con 0x7fb1780afa20 2023-10-18T10:26:01.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.484+0000 7fb174ff9640 1 -- 172.21.15.125:0/1473906254 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb16803e450 con 0x7fb1780afa20 2023-10-18T10:26:01.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.484+0000 7fb174ff9640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb16406ae40 0x7fb16406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.484+0000 7fb174ff9640 1 -- 172.21.15.125:0/1473906254 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb1680ac4f0 con 0x7fb1780afa20 2023-10-18T10:26:01.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.485+0000 7fb1777fe640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb16406ae40 0x7fb16406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.485+0000 7fb1777fe640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb16406ae40 0x7fb16406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb1781592a0 tx=0x7fb16c005ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:01.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.489+0000 7fb174ff9640 1 -- 172.21.15.125:0/1473906254 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1680771f0 con 0x7fb1780afa20 2023-10-18T10:26:01.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.656+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1780c96b0 con 0x7fb1780afa20 2023-10-18T10:26:01.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.657+0000 7fb174ff9640 1 -- 172.21.15.125:0/1473906254 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fb16807a810 con 0x7fb1780afa20 2023-10-18T10:26:01.660 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:01.660 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:26:01.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb16406ae40 msgr2=0x7fb16406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb16406ae40 0x7fb16406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb1781592a0 tx=0x7fb16c005ef0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780afa20 msgr2=0x7fb1781f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780afa20 0x7fb1781f1d30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb168009850 tx=0x7fb16802dad0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 shutdown_connections 2023-10-18T10:26:01.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb16406ae40 0x7fb16406d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1780cbd20 0x7fb17814e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1780b0530 0x7fb17814dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 --2- 172.21.15.125:0/1473906254 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1780afa20 0x7fb1781f1d30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 >> 172.21.15.125:0/1473906254 conn(0x7fb178069930 msgr2=0x7fb17806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:01.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 shutdown_connections 2023-10-18T10:26:01.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.659+0000 7fb17e739640 1 -- 172.21.15.125:0/1473906254 wait complete. 2023-10-18T10:26:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:26:01.673 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:26:01.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.777+0000 7f73bbec9640 1 -- 172.21.15.125:0/3511017151 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b413d360 msgr2=0x7f73b415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.777+0000 7f73bbec9640 1 --2- 172.21.15.125:0/3511017151 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b413d360 0x7f73b415a9f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f73b000a940 tx=0x7f73b002d090 comp rx=0 tx=0).stop 2023-10-18T10:26:01.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.777+0000 7f73bbec9640 1 -- 172.21.15.125:0/3511017151 shutdown_connections 2023-10-18T10:26:01.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.777+0000 7f73bbec9640 1 --2- 172.21.15.125:0/3511017151 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b413d360 0x7f73b415a9f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.777+0000 7f73bbec9640 1 -- 172.21.15.125:0/3511017151 >> 172.21.15.125:0/3511017151 conn(0x7f73b4069930 msgr2=0x7f73b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:01.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.777+0000 7f73bbec9640 1 -- 172.21.15.125:0/3511017151 shutdown_connections 2023-10-18T10:26:01.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.777+0000 7f73bbec9640 1 -- 172.21.15.125:0/3511017151 wait complete. 2023-10-18T10:26:01.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.778+0000 7f73bbec9640 1 Processor -- start 2023-10-18T10:26:01.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.778+0000 7f73bbec9640 1 -- start start 2023-10-18T10:26:01.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.778+0000 7f73bbec9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73b40c9ed0 0x7f73b412f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.778+0000 7f73bbec9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73b413d360 0x7f73b41e1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.778+0000 7f73bbec9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b4140660 0x7f73b40a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.778+0000 7f73bbec9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b41e22c0 con 0x7f73b4140660 2023-10-18T10:26:01.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.778+0000 7f73bbec9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b41e3ab0 con 0x7f73b413d360 2023-10-18T10:26:01.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.778+0000 7f73bbec9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b412f5c0 con 0x7f73b40c9ed0 2023-10-18T10:26:01.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b4140660 0x7f73b40a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b4140660 0x7f73b40a64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36588/0 (socket says 172.21.15.125:36588) 2023-10-18T10:26:01.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 -- 172.21.15.125:0/744804596 learned_addr learned my addr 172.21.15.125:0/744804596 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:01.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73b9464640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73b413d360 0x7f73b41e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73b9c65640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73b40c9ed0 0x7f73b412f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 -- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73b40c9ed0 msgr2=0x7f73b412f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73b40c9ed0 0x7f73b412f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 -- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73b413d360 msgr2=0x7f73b41e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73b413d360 0x7f73b41e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 -- 172.21.15.125:0/744804596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b000a5a0 con 0x7f73b4140660 2023-10-18T10:26:01.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73b9c65640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73b40c9ed0 0x7f73b412f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:01.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73ba466640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b4140660 0x7f73b40a64b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f73ac001560 tx=0x7f73ac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:01.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73aa7fc640 1 -- 172.21.15.125:0/744804596 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac003da0 con 0x7f73b4140660 2023-10-18T10:26:01.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73aa7fc640 1 -- 172.21.15.125:0/744804596 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f73ac005070 con 0x7f73b4140660 2023-10-18T10:26:01.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b41e1d00 con 0x7f73b4140660 2023-10-18T10:26:01.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73aa7fc640 1 -- 172.21.15.125:0/744804596 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac005460 con 0x7f73b4140660 2023-10-18T10:26:01.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.779+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b41e3740 con 0x7f73b4140660 2023-10-18T10:26:01.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.780+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b41e35a0 con 0x7f73b4140660 2023-10-18T10:26:01.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.781+0000 7f73aa7fc640 1 -- 172.21.15.125:0/744804596 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f73ac0055c0 con 0x7f73b4140660 2023-10-18T10:26:01.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.781+0000 7f73aa7fc640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f739806ae40 0x7f739806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:01.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.781+0000 7f73aa7fc640 1 -- 172.21.15.125:0/744804596 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f73ac089470 con 0x7f73b4140660 2023-10-18T10:26:01.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.781+0000 7f73b9c65640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f739806ae40 0x7f739806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:01.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.782+0000 7f73b9c65640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f739806ae40 0x7f739806d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f73b0037000 tx=0x7f73b0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:01.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.785+0000 7f73aa7fc640 1 -- 172.21.15.125:0/744804596 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73b41e35a0 con 0x7f73b4140660 2023-10-18T10:26:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.950+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73b41e35a0 con 0x7f73b4140660 2023-10-18T10:26:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.951+0000 7f73aa7fc640 1 -- 172.21.15.125:0/744804596 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73ac017020 con 0x7f73b4140660 2023-10-18T10:26:01.953 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:26:01.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f739806ae40 msgr2=0x7f739806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f739806ae40 0x7f739806d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f73b0037000 tx=0x7f73b0004730 comp rx=0 tx=0).stop 2023-10-18T10:26:01.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b4140660 msgr2=0x7f73b40a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:01.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b4140660 0x7f73b40a64b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f73ac001560 tx=0x7f73ac00cc70 comp rx=0 tx=0).stop 2023-10-18T10:26:01.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 shutdown_connections 2023-10-18T10:26:01.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f739806ae40 0x7f739806d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73b413d360 0x7f73b41e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73b4140660 0x7f73b40a64b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 --2- 172.21.15.125:0/744804596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73b40c9ed0 0x7f73b412f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:01.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 >> 172.21.15.125:0/744804596 conn(0x7f73b4069930 msgr2=0x7f73b413f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:01.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 shutdown_connections 2023-10-18T10:26:01.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:01.953+0000 7f73bbec9640 1 -- 172.21.15.125:0/744804596 wait complete. 2023-10-18T10:26:01.965 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:26:02.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.068+0000 7f65d4a7c640 1 -- 172.21.15.125:0/1278240032 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 msgr2=0x7f65d00b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.068+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/1278240032 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 0x7f65d00b0e90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f65c0005fd0 tx=0x7f65c002be10 comp rx=0 tx=0).stop 2023-10-18T10:26:02.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.068+0000 7f65d4a7c640 1 -- 172.21.15.125:0/1278240032 shutdown_connections 2023-10-18T10:26:02.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.068+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/1278240032 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d015c1f0 0x7f65d006a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.068+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/1278240032 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 0x7f65d00b0e90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.069+0000 7f65d4a7c640 1 -- 172.21.15.125:0/1278240032 >> 172.21.15.125:0/1278240032 conn(0x7f65d006fae0 msgr2=0x7f65d006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:02.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.069+0000 7f65d4a7c640 1 -- 172.21.15.125:0/1278240032 shutdown_connections 2023-10-18T10:26:02.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.069+0000 7f65d4a7c640 1 -- 172.21.15.125:0/1278240032 wait complete. 2023-10-18T10:26:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.070+0000 7f65d4a7c640 1 Processor -- start 2023-10-18T10:26:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.070+0000 7f65d4a7c640 1 -- start start 2023-10-18T10:26:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65d4a7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d006ac70 0x7f65d01503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65d4a7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 0x7f65d00cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65d4a7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d015c1f0 0x7f65d00cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65d4a7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00c73b0 con 0x7f65d01579f0 2023-10-18T10:26:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65d4a7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00c7690 con 0x7f65d006ac70 2023-10-18T10:26:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65d4a7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00c7970 con 0x7f65d015c1f0 2023-10-18T10:26:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 0x7f65d00cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 0x7f65d00cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36598/0 (socket says 172.21.15.125:36598) 2023-10-18T10:26:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 -- 172.21.15.125:0/3291551935 learned_addr learned my addr 172.21.15.125:0/3291551935 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65ced9d640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d015c1f0 0x7f65d00cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65ce59c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d006ac70 0x7f65d01503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 -- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d015c1f0 msgr2=0x7f65d00cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d015c1f0 0x7f65d00cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 -- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d006ac70 msgr2=0x7f65d01503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d006ac70 0x7f65d01503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 -- 172.21.15.125:0/3291551935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0009560 con 0x7f65d01579f0 2023-10-18T10:26:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c7fff640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 0x7f65d00cca40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d750 tx=0x7f65b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c6ffd640 1 -- 172.21.15.125:0/3291551935 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8009c30 con 0x7f65d01579f0 2023-10-18T10:26:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c6ffd640 1 -- 172.21.15.125:0/3291551935 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f65b8009e80 con 0x7f65d01579f0 2023-10-18T10:26:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00c7c50 con 0x7f65d01579f0 2023-10-18T10:26:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65c6ffd640 1 -- 172.21.15.125:0/3291551935 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8004280 con 0x7f65d01579f0 2023-10-18T10:26:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.071+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d00ccf80 con 0x7f65d01579f0 2023-10-18T10:26:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.072+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d00690e0 con 0x7f65d01579f0 2023-10-18T10:26:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.073+0000 7f65c6ffd640 1 -- 172.21.15.125:0/3291551935 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f65b80035f0 con 0x7f65d01579f0 2023-10-18T10:26:02.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.073+0000 7f65c6ffd640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b006ae40 0x7f65b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.073+0000 7f65c6ffd640 1 -- 172.21.15.125:0/3291551935 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f65b8089a90 con 0x7f65d01579f0 2023-10-18T10:26:02.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.074+0000 7f65ce59c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b006ae40 0x7f65b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.074+0000 7f65ce59c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b006ae40 0x7f65b006d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f65c0035000 tx=0x7f65c0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:02.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.077+0000 7f65c6ffd640 1 -- 172.21.15.125:0/3291551935 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65b8054810 con 0x7f65d01579f0 2023-10-18T10:26:02.175 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:26:02.177 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:26:02.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.247+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65d006d170 con 0x7f65d01579f0 2023-10-18T10:26:02.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.248+0000 7f65c6ffd640 1 -- 172.21.15.125:0/3291551935 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65b8017020 con 0x7f65d01579f0 2023-10-18T10:26:02.250 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:26:02.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b006ae40 msgr2=0x7f65b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b006ae40 0x7f65b006d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f65c0035000 tx=0x7f65c0004730 comp rx=0 tx=0).stop 2023-10-18T10:26:02.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 msgr2=0x7f65d00cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 0x7f65d00cca40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d750 tx=0x7f65b800dc10 comp rx=0 tx=0).stop 2023-10-18T10:26:02.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 shutdown_connections 2023-10-18T10:26:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f65b006ae40 0x7f65b006d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65d015c1f0 0x7f65d00cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65d01579f0 0x7f65d00cca40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 --2- 172.21.15.125:0/3291551935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65d006ac70 0x7f65d01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 >> 172.21.15.125:0/3291551935 conn(0x7f65d006fae0 msgr2=0x7f65d0151fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:02.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 shutdown_connections 2023-10-18T10:26:02.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.250+0000 7f65d4a7c640 1 -- 172.21.15.125:0/3291551935 wait complete. 2023-10-18T10:26:02.262 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:26:02.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.407+0000 7fefe5c10640 1 -- 172.21.15.125:0/2115461794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefc8005e10 msgr2=0x7fefc8006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.407+0000 7fefe5c10640 1 --2- 172.21.15.125:0/2115461794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefc8005e10 0x7fefc8006290 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fefd800c610 tx=0x7fefc8006900 comp rx=0 tx=0).stop 2023-10-18T10:26:02.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.407+0000 7fefdffff640 1 -- 172.21.15.125:0/2115461794 reap_dead start 2023-10-18T10:26:02.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.407+0000 7fefe5c10640 1 -- 172.21.15.125:0/2115461794 shutdown_connections 2023-10-18T10:26:02.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.407+0000 7fefe5c10640 1 -- 172.21.15.125:0/2115461794 >> 172.21.15.125:0/2115461794 conn(0x7fefe0069930 msgr2=0x7fefe0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:02.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.407+0000 7fefe5c10640 1 -- 172.21.15.125:0/2115461794 shutdown_connections 2023-10-18T10:26:02.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.407+0000 7fefe5c10640 1 -- 172.21.15.125:0/2115461794 wait complete. 2023-10-18T10:26:02.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.408+0000 7fefe5c10640 1 Processor -- start 2023-10-18T10:26:02.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefe5c10640 1 -- start start 2023-10-18T10:26:02.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefe5c10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefc8005e10 0x7fefe0156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefe5c10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefe0139e00 0x7fefe013a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefe5c10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefe014c250 0x7fefe014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefe5c10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe013a7c0 con 0x7fefe0139e00 2023-10-18T10:26:02.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefe5c10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01e1380 con 0x7fefe014c250 2023-10-18T10:26:02.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefe5c10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe014d2d0 con 0x7fefc8005e10 2023-10-18T10:26:02.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefdeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefe0139e00 0x7fefe013a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefdeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefe0139e00 0x7fefe013a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36624/0 (socket says 172.21.15.125:36624) 2023-10-18T10:26:02.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.409+0000 7fefdeffd640 1 -- 172.21.15.125:0/4156330830 learned_addr learned my addr 172.21.15.125:0/4156330830 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:02.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdf7fe640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefc8005e10 0x7fefe0156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdffff640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefe014c250 0x7fefe014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdeffd640 1 -- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefc8005e10 msgr2=0x7fefe0156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdeffd640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefc8005e10 0x7fefe0156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdeffd640 1 -- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefe014c250 msgr2=0x7fefe014c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdeffd640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefe014c250 0x7fefe014c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdeffd640 1 -- 172.21.15.125:0/4156330830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe014cff0 con 0x7fefe0139e00 2023-10-18T10:26:02.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdffff640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefe014c250 0x7fefe014c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:02.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefdeffd640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefe0139e00 0x7fefe013a280 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fefd4011a50 tx=0x7fefd4011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:02.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefc7fff640 1 -- 172.21.15.125:0/4156330830 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd400fdf0 con 0x7fefe0139e00 2023-10-18T10:26:02.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefc7fff640 1 -- 172.21.15.125:0/4156330830 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fefd4012450 con 0x7fefe0139e00 2023-10-18T10:26:02.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe01e0d30 con 0x7fefe0139e00 2023-10-18T10:26:02.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefc7fff640 1 -- 172.21.15.125:0/4156330830 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4012840 con 0x7fefe0139e00 2023-10-18T10:26:02.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.410+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe014cc40 con 0x7fefe0139e00 2023-10-18T10:26:02.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.411+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe01e1530 con 0x7fefe0139e00 2023-10-18T10:26:02.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.412+0000 7fefc7fff640 1 -- 172.21.15.125:0/4156330830 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fefd40129a0 con 0x7fefe0139e00 2023-10-18T10:26:02.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.412+0000 7fefc7fff640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fefcc06ae40 0x7fefcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.412+0000 7fefc7fff640 1 -- 172.21.15.125:0/4156330830 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fefd408a9d0 con 0x7fefe0139e00 2023-10-18T10:26:02.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.412+0000 7fefdf7fe640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fefcc06ae40 0x7fefcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.413+0000 7fefdf7fe640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fefcc06ae40 0x7fefcc06d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fefe0151130 tx=0x7fefd0005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:02.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.416+0000 7fefc7fff640 1 -- 172.21.15.125:0/4156330830 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fefe01e1530 con 0x7fefe0139e00 2023-10-18T10:26:02.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.585+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fefe0151c50 con 0x7fefe0139e00 2023-10-18T10:26:02.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.585+0000 7fefc7fff640 1 -- 172.21.15.125:0/4156330830 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fefd401f0a0 con 0x7fefe0139e00 2023-10-18T10:26:02.588 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:26:02.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fefcc06ae40 msgr2=0x7fefcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fefcc06ae40 0x7fefcc06d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fefe0151130 tx=0x7fefd0005f90 comp rx=0 tx=0).stop 2023-10-18T10:26:02.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefe0139e00 msgr2=0x7fefe013a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefe0139e00 0x7fefe013a280 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fefd4011a50 tx=0x7fefd4011f10 comp rx=0 tx=0).stop 2023-10-18T10:26:02.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 shutdown_connections 2023-10-18T10:26:02.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fefcc06ae40 0x7fefcc06d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefe014c250 0x7fefe014c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefe0139e00 0x7fefe013a280 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 --2- 172.21.15.125:0/4156330830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefc8005e10 0x7fefe0156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 >> 172.21.15.125:0/4156330830 conn(0x7fefe0069930 msgr2=0x7fefe014fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:02.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 shutdown_connections 2023-10-18T10:26:02.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.587+0000 7fefe5c10640 1 -- 172.21.15.125:0/4156330830 wait complete. 2023-10-18T10:26:02.600 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:26:02.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.703+0000 7fb235897640 1 -- 172.21.15.125:0/808845938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2300ce240 msgr2=0x7fb23006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.703+0000 7fb235897640 1 --2- 172.21.15.125:0/808845938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2300ce240 0x7fb23006d260 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb218009830 tx=0x7fb21802bd60 comp rx=0 tx=0).stop 2023-10-18T10:26:02.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.703+0000 7fb235897640 1 -- 172.21.15.125:0/808845938 shutdown_connections 2023-10-18T10:26:02.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.703+0000 7fb235897640 1 --2- 172.21.15.125:0/808845938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2300ce240 0x7fb23006d260 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.703+0000 7fb235897640 1 -- 172.21.15.125:0/808845938 >> 172.21.15.125:0/808845938 conn(0x7fb2300b0670 msgr2=0x7fb2300b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:02.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.703+0000 7fb235897640 1 -- 172.21.15.125:0/808845938 shutdown_connections 2023-10-18T10:26:02.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.703+0000 7fb235897640 1 -- 172.21.15.125:0/808845938 wait complete. 2023-10-18T10:26:02.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.704+0000 7fb235897640 1 Processor -- start 2023-10-18T10:26:02.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.704+0000 7fb235897640 1 -- start start 2023-10-18T10:26:02.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb235897640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2300ce240 0x7fb230157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb235897640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb230159060 0x7fb2301f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb235897640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb23015ccf0 0x7fb2301f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb235897640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c3980 con 0x7fb230159060 2023-10-18T10:26:02.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb235897640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2301582d0 con 0x7fb2300ce240 2023-10-18T10:26:02.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb235897640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c33c0 con 0x7fb23015ccf0 2023-10-18T10:26:02.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb230159060 0x7fb2301f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb230159060 0x7fb2301f2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36642/0 (socket says 172.21.15.125:36642) 2023-10-18T10:26:02.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 -- 172.21.15.125:0/3936398834 learned_addr learned my addr 172.21.15.125:0/3936398834 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:02.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb22f7fe640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb23015ccf0 0x7fb2301f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb22effd640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2300ce240 0x7fb230157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 -- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb23015ccf0 msgr2=0x7fb2301f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb23015ccf0 0x7fb2301f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 -- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2300ce240 msgr2=0x7fb230157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2300ce240 0x7fb230157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 -- 172.21.15.125:0/3936398834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009490 con 0x7fb230159060 2023-10-18T10:26:02.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.705+0000 7fb227fff640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb230159060 0x7fb2301f2e60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb218000c00 tx=0x7fb2180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:02.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.706+0000 7fb22cff9640 1 -- 172.21.15.125:0/3936398834 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218003ea0 con 0x7fb230159060 2023-10-18T10:26:02.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.706+0000 7fb22cff9640 1 -- 172.21.15.125:0/3936398834 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb218034530 con 0x7fb230159060 2023-10-18T10:26:02.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.706+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300c36a0 con 0x7fb230159060 2023-10-18T10:26:02.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.706+0000 7fb22cff9640 1 -- 172.21.15.125:0/3936398834 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218034940 con 0x7fb230159060 2023-10-18T10:26:02.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.706+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb230159ad0 con 0x7fb230159060 2023-10-18T10:26:02.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.707+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230159650 con 0x7fb230159060 2023-10-18T10:26:02.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.707+0000 7fb22cff9640 1 -- 172.21.15.125:0/3936398834 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb218034aa0 con 0x7fb230159060 2023-10-18T10:26:02.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.708+0000 7fb22cff9640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb21006ae70 0x7fb21006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.708+0000 7fb22cff9640 1 -- 172.21.15.125:0/3936398834 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb2180ac210 con 0x7fb230159060 2023-10-18T10:26:02.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.708+0000 7fb22effd640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb21006ae70 0x7fb21006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:02.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.708+0000 7fb22effd640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb21006ae70 0x7fb21006d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2200046b0 tx=0x7fb220007eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:02.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.712+0000 7fb22cff9640 1 -- 172.21.15.125:0/3936398834 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb230159650 con 0x7fb230159060 2023-10-18T10:26:02.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.876+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb230159650 con 0x7fb230159060 2023-10-18T10:26:02.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.876+0000 7fb22cff9640 1 -- 172.21.15.125:0/3936398834 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb230159650 con 0x7fb230159060 2023-10-18T10:26:02.879 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:26:02.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb21006ae70 msgr2=0x7fb21006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb21006ae70 0x7fb21006d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2200046b0 tx=0x7fb220007eb0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb230159060 msgr2=0x7fb2301f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb230159060 0x7fb2301f2e60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb218000c00 tx=0x7fb2180038a0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 shutdown_connections 2023-10-18T10:26:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb21006ae70 0x7fb21006d330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb23015ccf0 0x7fb2301f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb230159060 0x7fb2301f2e60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 --2- 172.21.15.125:0/3936398834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2300ce240 0x7fb230157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 >> 172.21.15.125:0/3936398834 conn(0x7fb2300b0670 msgr2=0x7fb2300afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 shutdown_connections 2023-10-18T10:26:02.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.878+0000 7fb235897640 1 -- 172.21.15.125:0/3936398834 wait complete. 2023-10-18T10:26:02.891 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:26:02.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 -- 172.21.15.125:0/3656589317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c9dc0 msgr2=0x7fa6641531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:02.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 --2- 172.21.15.125:0/3656589317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c9dc0 0x7fa6641531d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa6640cc060 tx=0x7fa65402bda0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 -- 172.21.15.125:0/3656589317 shutdown_connections 2023-10-18T10:26:02.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 --2- 172.21.15.125:0/3656589317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c9dc0 0x7fa6641531d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 --2- 172.21.15.125:0/3656589317 >> v2:172.21.15.139:3300/0 conn(0x7fa6640ab500 0x7fa6640ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 --2- 172.21.15.125:0/3656589317 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6640c99e0 0x7fa664152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:02.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 -- 172.21.15.125:0/3656589317 >> 172.21.15.125:0/3656589317 conn(0x7fa664069930 msgr2=0x7fa664069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:02.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 -- 172.21.15.125:0/3656589317 shutdown_connections 2023-10-18T10:26:02.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.992+0000 7fa66aba9640 1 -- 172.21.15.125:0/3656589317 wait complete. 2023-10-18T10:26:02.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.993+0000 7fa66aba9640 1 Processor -- start 2023-10-18T10:26:02.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.993+0000 7fa66aba9640 1 -- start start 2023-10-18T10:26:02.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa66aba9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6640ab500 0x7fa6640aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa66aba9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c99e0 0x7fa66412d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa66aba9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6640c9dc0 0x7fa6640a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:02.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa66aba9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66412eca0 con 0x7fa6640c99e0 2023-10-18T10:26:02.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa66aba9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa664152a80 con 0x7fa6640c9dc0 2023-10-18T10:26:02.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa66aba9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66412e020 con 0x7fa6640ab500 2023-10-18T10:26:02.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa65bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c99e0 0x7fa66412d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa65bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c99e0 0x7fa66412d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36666/0 (socket says 172.21.15.125:36666) 2023-10-18T10:26:03.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa65bfff640 1 -- 172.21.15.125:0/1588892071 learned_addr learned my addr 172.21.15.125:0/1588892071 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:03.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa668945640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6640ab500 0x7fa6640aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa669146640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6640c9dc0 0x7fa6640a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa65bfff640 1 -- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6640ab500 msgr2=0x7fa6640aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa65bfff640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6640ab500 0x7fa6640aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa65bfff640 1 -- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6640c9dc0 msgr2=0x7fa6640a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa65bfff640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6640c9dc0 0x7fa6640a7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa65bfff640 1 -- 172.21.15.125:0/1588892071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654009560 con 0x7fa6640c99e0 2023-10-18T10:26:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.994+0000 7fa668945640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6640ab500 0x7fa6640aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:26:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.995+0000 7fa65bfff640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c99e0 0x7fa66412d6b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa654009850 tx=0x7fa65402da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.995+0000 7fa6597fa640 1 -- 172.21.15.125:0/1588892071 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654003800 con 0x7fa6640c99e0 2023-10-18T10:26:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.995+0000 7fa6597fa640 1 -- 172.21.15.125:0/1588892071 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa654003a50 con 0x7fa6640c99e0 2023-10-18T10:26:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.995+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6640abff0 con 0x7fa6640c99e0 2023-10-18T10:26:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.995+0000 7fa6597fa640 1 -- 172.21.15.125:0/1588892071 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654003e20 con 0x7fa6640c99e0 2023-10-18T10:26:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.995+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66412e5e0 con 0x7fa6640c99e0 2023-10-18T10:26:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.996+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66412ede0 con 0x7fa6640c99e0 2023-10-18T10:26:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.996+0000 7fa6597fa640 1 -- 172.21.15.125:0/1588892071 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa65403e450 con 0x7fa6640c99e0 2023-10-18T10:26:03.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.997+0000 7fa6597fa640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa65006ae70 0x7fa65006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.997+0000 7fa6597fa640 1 -- 172.21.15.125:0/1588892071 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa6540ac3e0 con 0x7fa6640c99e0 2023-10-18T10:26:03.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.997+0000 7fa668945640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa65006ae70 0x7fa65006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:02.997+0000 7fa668945640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa65006ae70 0x7fa65006d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa660009f40 tx=0x7fa660005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:03.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.001+0000 7fa6597fa640 1 -- 172.21.15.125:0/1588892071 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa66412ede0 con 0x7fa6640c99e0 2023-10-18T10:26:03.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.179+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa66412ede0 con 0x7fa6640c99e0 2023-10-18T10:26:03.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.180+0000 7fa6597fa640 1 -- 172.21.15.125:0/1588892071 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa66412ede0 con 0x7fa6640c99e0 2023-10-18T10:26:03.182 INFO:teuthology.orchestra.run.smithi125.stderr:listed 0 entries 2023-10-18T10:26:03.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa65006ae70 msgr2=0x7fa65006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa65006ae70 0x7fa65006d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa660009f40 tx=0x7fa660005f90 comp rx=0 tx=0).stop 2023-10-18T10:26:03.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c99e0 msgr2=0x7fa66412d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c99e0 0x7fa66412d6b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa654009850 tx=0x7fa65402da70 comp rx=0 tx=0).stop 2023-10-18T10:26:03.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 shutdown_connections 2023-10-18T10:26:03.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fa65006ae70 0x7fa65006d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6640c9dc0 0x7fa6640a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6640c99e0 0x7fa66412d6b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.181+0000 7fa66aba9640 1 --2- 172.21.15.125:0/1588892071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6640ab500 0x7fa6640aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.182+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 >> 172.21.15.125:0/1588892071 conn(0x7fa664069930 msgr2=0x7fa664157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:03.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.182+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 shutdown_connections 2023-10-18T10:26:03.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.182+0000 7fa66aba9640 1 -- 172.21.15.125:0/1588892071 wait complete. 2023-10-18T10:26:03.194 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2023-10-18T10:26:03.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:03.195 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-18T10:26:03.222 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:26:03.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:26:03.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:03.223 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:26:03.277 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:26:03.277 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:26:03.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.377+0000 7f12843f0640 1 -- 172.21.15.125:0/809553962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c141690 msgr2=0x7f127c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.377+0000 7f12843f0640 1 --2- 172.21.15.125:0/809553962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c141690 0x7f127c0703e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f127c0b20f0 tx=0x7f126c02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.377+0000 7f12843f0640 1 -- 172.21.15.125:0/809553962 shutdown_connections 2023-10-18T10:26:03.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.377+0000 7f12843f0640 1 --2- 172.21.15.125:0/809553962 >> v2:172.21.15.139:3300/0 conn(0x7f127c070920 0x7f127c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.377+0000 7f12843f0640 1 --2- 172.21.15.125:0/809553962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c141690 0x7f127c0703e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.377+0000 7f12843f0640 1 -- 172.21.15.125:0/809553962 >> 172.21.15.125:0/809553962 conn(0x7f127c069930 msgr2=0x7f127c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:03.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.378+0000 7f12843f0640 1 -- 172.21.15.125:0/809553962 shutdown_connections 2023-10-18T10:26:03.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.378+0000 7f12843f0640 1 -- 172.21.15.125:0/809553962 wait complete. 2023-10-18T10:26:03.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f12843f0640 1 Processor -- start 2023-10-18T10:26:03.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f12843f0640 1 -- start start 2023-10-18T10:26:03.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f12843f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c070920 0x7f127c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f12843f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f127c13e520 0x7f127c1f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f12843f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f127c141690 0x7f127c1f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f12843f0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0aafb0 con 0x7f127c070920 2023-10-18T10:26:03.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f12843f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c1d1a40 con 0x7f127c141690 2023-10-18T10:26:03.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f12843f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c1d1760 con 0x7f127c13e520 2023-10-18T10:26:03.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f128218c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c070920 0x7f127c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f128218c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c070920 0x7f127c0caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36682/0 (socket says 172.21.15.125:36682) 2023-10-18T10:26:03.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.379+0000 7f128218c640 1 -- 172.21.15.125:0/2151861146 learned_addr learned my addr 172.21.15.125:0/2151861146 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:03.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f128298d640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f127c141690 0x7f127c1f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f128198b640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f127c13e520 0x7f127c1f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f128218c640 1 -- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f127c13e520 msgr2=0x7f127c1f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f128218c640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f127c13e520 0x7f127c1f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f128218c640 1 -- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f127c141690 msgr2=0x7f127c1f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f128218c640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f127c141690 0x7f127c1f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f128218c640 1 -- 172.21.15.125:0/2151861146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c009490 con 0x7f127c070920 2023-10-18T10:26:03.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f128218c640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c070920 0x7f127c0caae0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f127800c4b0 tx=0x7f127800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:03.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f1272ffd640 1 -- 172.21.15.125:0/2151861146 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278005480 con 0x7f127c070920 2023-10-18T10:26:03.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f1272ffd640 1 -- 172.21.15.125:0/2151861146 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f12780036e0 con 0x7f127c070920 2023-10-18T10:26:03.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c0aa960 con 0x7f127c070920 2023-10-18T10:26:03.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f1272ffd640 1 -- 172.21.15.125:0/2151861146 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278003b30 con 0x7f127c070920 2023-10-18T10:26:03.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.380+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c0cb020 con 0x7f127c070920 2023-10-18T10:26:03.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.381+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c0cb210 con 0x7f127c070920 2023-10-18T10:26:03.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.382+0000 7f1272ffd640 1 -- 172.21.15.125:0/2151861146 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1278003c90 con 0x7f127c070920 2023-10-18T10:26:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.382+0000 7f1272ffd640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f126006ae40 0x7f126006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.382+0000 7f1272ffd640 1 -- 172.21.15.125:0/2151861146 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f12780891b0 con 0x7f127c070920 2023-10-18T10:26:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.382+0000 7f128198b640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f126006ae40 0x7f126006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.383+0000 7f128198b640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f126006ae40 0x7f126006d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f126c036000 tx=0x7f126c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:03.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.386+0000 7f1272ffd640 1 -- 172.21.15.125:0/2151861146 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1278053f50 con 0x7f127c070920 2023-10-18T10:26:03.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.588+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f127c067930 con 0x7f127c070920 2023-10-18T10:26:03.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.588+0000 7f1272ffd640 1 -- 172.21.15.125:0/2151861146 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f1278057570 con 0x7f127c070920 2023-10-18T10:26:03.591 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:03.591 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:19.519174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:26:03.593 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 9 2023-10-18T10:26:03.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f126006ae40 msgr2=0x7f126006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f126006ae40 0x7f126006d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f126c036000 tx=0x7f126c004730 comp rx=0 tx=0).stop 2023-10-18T10:26:03.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c070920 msgr2=0x7f127c0caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c070920 0x7f127c0caae0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f127800c4b0 tx=0x7f127800c970 comp rx=0 tx=0).stop 2023-10-18T10:26:03.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 shutdown_connections 2023-10-18T10:26:03.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f126006ae40 0x7f126006d300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f127c141690 0x7f127c1f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f127c13e520 0x7f127c1f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 --2- 172.21.15.125:0/2151861146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f127c070920 0x7f127c0caae0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 >> 172.21.15.125:0/2151861146 conn(0x7f127c069930 msgr2=0x7f127c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:03.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 shutdown_connections 2023-10-18T10:26:03.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.590+0000 7f12843f0640 1 -- 172.21.15.125:0/2151861146 wait complete. 2023-10-18T10:26:03.602 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:26:03.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 -- 172.21.15.125:0/950694235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 msgr2=0x7fb354070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/950694235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 0x7fb354070920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb35406a780 tx=0x7fb34c02c140 comp rx=0 tx=0).stop 2023-10-18T10:26:03.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 -- 172.21.15.125:0/950694235 shutdown_connections 2023-10-18T10:26:03.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/950694235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 0x7fb354070920 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/950694235 >> v2:172.21.15.139:3300/0 conn(0x7fb354141690 0x7fb3540703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/950694235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb35413e520 0x7fb35406bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 -- 172.21.15.125:0/950694235 >> 172.21.15.125:0/950694235 conn(0x7fb354069930 msgr2=0x7fb354069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:03.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 -- 172.21.15.125:0/950694235 shutdown_connections 2023-10-18T10:26:03.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.702+0000 7fb35a1d0640 1 -- 172.21.15.125:0/950694235 wait complete. 2023-10-18T10:26:03.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.703+0000 7fb35a1d0640 1 Processor -- start 2023-10-18T10:26:03.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb35a1d0640 1 -- start start 2023-10-18T10:26:03.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb35a1d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 0x7fb3541db750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb35a1d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb35413e520 0x7fb3541e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb35a1d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb354141690 0x7fb3541e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb35a1d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3541e1940 con 0x7fb35406c0a0 2023-10-18T10:26:03.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb35a1d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540aac10 con 0x7fb35413e520 2023-10-18T10:26:03.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb35a1d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3541f1470 con 0x7fb354141690 2023-10-18T10:26:03.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb3537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 0x7fb3541db750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb3537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 0x7fb3541db750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36700/0 (socket says 172.21.15.125:36700) 2023-10-18T10:26:03.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb3537fe640 1 -- 172.21.15.125:0/1442804846 learned_addr learned my addr 172.21.15.125:0/1442804846 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:03.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb352ffd640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb35413e520 0x7fb3541e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.704+0000 7fb353fff640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb354141690 0x7fb3541e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb3537fe640 1 -- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb354141690 msgr2=0x7fb3541e0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb3537fe640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb354141690 0x7fb3541e0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb3537fe640 1 -- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb35413e520 msgr2=0x7fb3541e07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb3537fe640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb35413e520 0x7fb3541e07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb3537fe640 1 -- 172.21.15.125:0/1442804846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c009900 con 0x7fb35406c0a0 2023-10-18T10:26:03.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb352ffd640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb35413e520 0x7fb3541e07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:03.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb353fff640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb354141690 0x7fb3541e0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:03.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb3537fe640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 0x7fb3541db750 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb34400d750 tx=0x7fb34400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:03.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb33bfff640 1 -- 172.21.15.125:0/1442804846 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344009c30 con 0x7fb35406c0a0 2023-10-18T10:26:03.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb33bfff640 1 -- 172.21.15.125:0/1442804846 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb344009e80 con 0x7fb35406c0a0 2023-10-18T10:26:03.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3541e1380 con 0x7fb35406c0a0 2023-10-18T10:26:03.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb33bfff640 1 -- 172.21.15.125:0/1442804846 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344004260 con 0x7fb35406c0a0 2023-10-18T10:26:03.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.705+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35413bd80 con 0x7fb35406c0a0 2023-10-18T10:26:03.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.706+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35413bbe0 con 0x7fb35406c0a0 2023-10-18T10:26:03.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.706+0000 7fb33bfff640 1 -- 172.21.15.125:0/1442804846 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb344005020 con 0x7fb35406c0a0 2023-10-18T10:26:03.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.707+0000 7fb33bfff640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb34006ae40 0x7fb34006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:03.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.707+0000 7fb33bfff640 1 -- 172.21.15.125:0/1442804846 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb344089e40 con 0x7fb35406c0a0 2023-10-18T10:26:03.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.707+0000 7fb352ffd640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb34006ae40 0x7fb34006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:03.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.707+0000 7fb352ffd640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb34006ae40 0x7fb34006d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb35406d3c0 tx=0x7fb348005ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:03.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.711+0000 7fb33bfff640 1 -- 172.21.15.125:0/1442804846 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb35413bbe0 con 0x7fb35406c0a0 2023-10-18T10:26:03.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.878+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb35413bbe0 con 0x7fb35406c0a0 2023-10-18T10:26:03.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.879+0000 7fb33bfff640 1 -- 172.21.15.125:0/1442804846 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fb35413bbe0 con 0x7fb35406c0a0 2023-10-18T10:26:03.881 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:03.881 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:26:03.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb34006ae40 msgr2=0x7fb34006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb34006ae40 0x7fb34006d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb35406d3c0 tx=0x7fb348005ef0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 msgr2=0x7fb3541db750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:03.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 0x7fb3541db750 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb34400d750 tx=0x7fb34400dc10 comp rx=0 tx=0).stop 2023-10-18T10:26:03.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 shutdown_connections 2023-10-18T10:26:03.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb34006ae40 0x7fb34006d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb35413e520 0x7fb3541e07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb354141690 0x7fb3541e0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 --2- 172.21.15.125:0/1442804846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb35406c0a0 0x7fb3541db750 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:03.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 >> 172.21.15.125:0/1442804846 conn(0x7fb354069930 msgr2=0x7fb3541de380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:03.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 shutdown_connections 2023-10-18T10:26:03.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.881+0000 7fb35a1d0640 1 -- 172.21.15.125:0/1442804846 wait complete. 2023-10-18T10:26:03.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:25:01.477046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:25:02.515145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T10:25:06.263035+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T10:26:03.894 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:26:04.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.998+0000 7f086d125640 1 -- 172.21.15.125:0/992404798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0854004fd0 msgr2=0x7f085400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.998+0000 7f086d125640 1 --2- 172.21.15.125:0/992404798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0854004fd0 0x7f085400b980 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0860001e90 tx=0x7f08540057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.998+0000 7f086759e640 1 -- 172.21.15.125:0/992404798 reap_dead start 2023-10-18T10:26:04.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.998+0000 7f086d125640 1 -- 172.21.15.125:0/992404798 shutdown_connections 2023-10-18T10:26:04.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.998+0000 7f086d125640 1 -- 172.21.15.125:0/992404798 >> 172.21.15.125:0/992404798 conn(0x7f086806fae0 msgr2=0x7f086806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:04.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.998+0000 7f086d125640 1 -- 172.21.15.125:0/992404798 shutdown_connections 2023-10-18T10:26:04.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.998+0000 7f086d125640 1 -- 172.21.15.125:0/992404798 wait complete. 2023-10-18T10:26:04.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.999+0000 7f086d125640 1 Processor -- start 2023-10-18T10:26:04.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:03.999+0000 7f086d125640 1 -- start start 2023-10-18T10:26:04.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f086d125640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08681eea20 0x7f086814f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f086d125640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868146880 0x7f086814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f086d125640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086814fc40 0x7f086814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f086d125640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0868149880 con 0x7f08681eea20 2023-10-18T10:26:04.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f086d125640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086814b590 con 0x7f0868146880 2023-10-18T10:26:04.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f086d125640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086814b870 con 0x7f086814fc40 2023-10-18T10:26:04.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08681eea20 0x7f086814f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08681eea20 0x7f086814f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36718/0 (socket says 172.21.15.125:36718) 2023-10-18T10:26:04.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 -- 172.21.15.125:0/4070922862 learned_addr learned my addr 172.21.15.125:0/4070922862 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:04.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 -- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086814fc40 msgr2=0x7f086814c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086814fc40 0x7f086814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 -- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868146880 msgr2=0x7f086814f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868146880 0x7f086814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 -- 172.21.15.125:0/4070922862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680b2330 con 0x7f08681eea20 2023-10-18T10:26:04.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.000+0000 7f0866d9d640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08681eea20 0x7f086814f1c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0858006600 tx=0x7f0858003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:04.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.001+0000 7f085f7fe640 1 -- 172.21.15.125:0/4070922862 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858011900 con 0x7f08681eea20 2023-10-18T10:26:04.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.001+0000 7f085f7fe640 1 -- 172.21.15.125:0/4070922862 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0858011a80 con 0x7f08681eea20 2023-10-18T10:26:04.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.001+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086814bb50 con 0x7f08681eea20 2023-10-18T10:26:04.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.001+0000 7f085f7fe640 1 -- 172.21.15.125:0/4070922862 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858011e90 con 0x7f08681eea20 2023-10-18T10:26:04.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.001+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08681eefe0 con 0x7f08681eea20 2023-10-18T10:26:04.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.002+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086814b3f0 con 0x7f08681eea20 2023-10-18T10:26:04.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.002+0000 7f085f7fe640 1 -- 172.21.15.125:0/4070922862 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f085800f430 con 0x7f08681eea20 2023-10-18T10:26:04.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.002+0000 7f085f7fe640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f084806ae70 0x7f084806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.002+0000 7f085f7fe640 1 -- 172.21.15.125:0/4070922862 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f085808a840 con 0x7f08681eea20 2023-10-18T10:26:04.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.003+0000 7f085ffff640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f084806ae70 0x7f084806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.003+0000 7f085ffff640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f084806ae70 0x7f084806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0868148060 tx=0x7f0850009820 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:04.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.007+0000 7f085f7fe640 1 -- 172.21.15.125:0/4070922862 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f086814b3f0 con 0x7f08681eea20 2023-10-18T10:26:04.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.170+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f086814b3f0 con 0x7f08681eea20 2023-10-18T10:26:04.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.171+0000 7f085f7fe640 1 -- 172.21.15.125:0/4070922862 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f086814b3f0 con 0x7f08681eea20 2023-10-18T10:26:04.173 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:26:04.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f084806ae70 msgr2=0x7f084806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f084806ae70 0x7f084806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0868148060 tx=0x7f0850009820 comp rx=0 tx=0).stop 2023-10-18T10:26:04.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08681eea20 msgr2=0x7f086814f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08681eea20 0x7f086814f1c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0858006600 tx=0x7f0858003aa0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 shutdown_connections 2023-10-18T10:26:04.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f084806ae70 0x7f084806d330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0868146880 0x7f086814f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086814fc40 0x7f086814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 --2- 172.21.15.125:0/4070922862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08681eea20 0x7f086814f1c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 >> 172.21.15.125:0/4070922862 conn(0x7f086806fae0 msgr2=0x7f086814a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:04.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 shutdown_connections 2023-10-18T10:26:04.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.173+0000 7f086d125640 1 -- 172.21.15.125:0/4070922862 wait complete. 2023-10-18T10:26:04.185 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:26:04.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.290+0000 7f23e2fee640 1 -- 172.21.15.125:0/3236719827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 msgr2=0x7f23dc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.290+0000 7f23e2fee640 1 --2- 172.21.15.125:0/3236719827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 0x7f23dc0705a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f23c8005fd0 tx=0x7f23c802be10 comp rx=0 tx=0).stop 2023-10-18T10:26:04.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.290+0000 7f23e2fee640 1 -- 172.21.15.125:0/3236719827 shutdown_connections 2023-10-18T10:26:04.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.290+0000 7f23e2fee640 1 --2- 172.21.15.125:0/3236719827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 0x7f23dc0705a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.290+0000 7f23e2fee640 1 --2- 172.21.15.125:0/3236719827 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23dc0cbd20 0x7f23dc158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.290+0000 7f23e2fee640 1 -- 172.21.15.125:0/3236719827 >> 172.21.15.125:0/3236719827 conn(0x7f23dc069930 msgr2=0x7f23dc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:04.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.290+0000 7f23e2fee640 1 -- 172.21.15.125:0/3236719827 shutdown_connections 2023-10-18T10:26:04.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.290+0000 7f23e2fee640 1 -- 172.21.15.125:0/3236719827 wait complete. 2023-10-18T10:26:04.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.291+0000 7f23e2fee640 1 Processor -- start 2023-10-18T10:26:04.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.291+0000 7f23e2fee640 1 -- start start 2023-10-18T10:26:04.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e2fee640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 0x7f23dc146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e2fee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23dc0b0530 0x7f23dc159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e2fee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23dc0cbd20 0x7f23dc145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e2fee640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc1481f0 con 0x7f23dc0afa20 2023-10-18T10:26:04.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e2fee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc148680 con 0x7f23dc0cbd20 2023-10-18T10:26:04.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e2fee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc1475e0 con 0x7f23dc0b0530 2023-10-18T10:26:04.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e0d8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 0x7f23dc146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e0d8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 0x7f23dc146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36734/0 (socket says 172.21.15.125:36734) 2023-10-18T10:26:04.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.292+0000 7f23e0d8a640 1 -- 172.21.15.125:0/1531082661 learned_addr learned my addr 172.21.15.125:0/1531082661 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:04.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23dbfff640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23dc0b0530 0x7f23dc159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e158b640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23dc0cbd20 0x7f23dc145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e0d8a640 1 -- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23dc0b0530 msgr2=0x7f23dc159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e0d8a640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23dc0b0530 0x7f23dc159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e0d8a640 1 -- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23dc0cbd20 msgr2=0x7f23dc145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e0d8a640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23dc0cbd20 0x7f23dc145710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e0d8a640 1 -- 172.21.15.125:0/1531082661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c8009560 con 0x7f23dc0afa20 2023-10-18T10:26:04.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e158b640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23dc0cbd20 0x7f23dc145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:04.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e0d8a640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 0x7f23dc146f20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f23d000e790 tx=0x7f23d000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:04.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23d97fa640 1 -- 172.21.15.125:0/1531082661 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0009580 con 0x7f23dc0afa20 2023-10-18T10:26:04.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23d97fa640 1 -- 172.21.15.125:0/1531082661 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f23d000cdb0 con 0x7f23dc0afa20 2023-10-18T10:26:04.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc070cf0 con 0x7f23dc0afa20 2023-10-18T10:26:04.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23d97fa640 1 -- 172.21.15.125:0/1531082661 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0010690 con 0x7f23dc0afa20 2023-10-18T10:26:04.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.293+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc147ba0 con 0x7f23dc0afa20 2023-10-18T10:26:04.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.294+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc0690e0 con 0x7f23dc0afa20 2023-10-18T10:26:04.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.294+0000 7f23d97fa640 1 -- 172.21.15.125:0/1531082661 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f23d00107f0 con 0x7f23dc0afa20 2023-10-18T10:26:04.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.295+0000 7f23d97fa640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f23c006ae40 0x7f23c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.295+0000 7f23d97fa640 1 -- 172.21.15.125:0/1531082661 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f23d0014070 con 0x7f23dc0afa20 2023-10-18T10:26:04.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.295+0000 7f23dbfff640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f23c006ae40 0x7f23c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.296+0000 7f23dbfff640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f23c006ae40 0x7f23c006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f23c8035000 tx=0x7f23c8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:04.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.299+0000 7f23d97fa640 1 -- 172.21.15.125:0/1531082661 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23d0051ca0 con 0x7f23dc0afa20 2023-10-18T10:26:04.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.470+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23dc0cc1e0 con 0x7f23dc0afa20 2023-10-18T10:26:04.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.470+0000 7f23d97fa640 1 -- 172.21.15.125:0/1531082661 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23d0055130 con 0x7f23dc0afa20 2023-10-18T10:26:04.472 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:26:04.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f23c006ae40 msgr2=0x7f23c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f23c006ae40 0x7f23c006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f23c8035000 tx=0x7f23c8004730 comp rx=0 tx=0).stop 2023-10-18T10:26:04.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 msgr2=0x7f23dc146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 0x7f23dc146f20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f23d000e790 tx=0x7f23d000ec50 comp rx=0 tx=0).stop 2023-10-18T10:26:04.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 shutdown_connections 2023-10-18T10:26:04.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f23c006ae40 0x7f23c006d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23dc0cbd20 0x7f23dc145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23dc0b0530 0x7f23dc159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 --2- 172.21.15.125:0/1531082661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23dc0afa20 0x7f23dc146f20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 >> 172.21.15.125:0/1531082661 conn(0x7f23dc069930 msgr2=0x7f23dc0c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:04.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 shutdown_connections 2023-10-18T10:26:04.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.472+0000 7f23e2fee640 1 -- 172.21.15.125:0/1531082661 wait complete. 2023-10-18T10:26:04.484 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:26:04.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.588+0000 7f083471d640 1 -- 172.21.15.125:0/2181021506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0818003970 msgr2=0x7f0818003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.588+0000 7f083471d640 1 --2- 172.21.15.125:0/2181021506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0818003970 0x7f0818003d70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f082c0b00b0 tx=0x7f08180057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.588+0000 7f083471d640 1 -- 172.21.15.125:0/2181021506 shutdown_connections 2023-10-18T10:26:04.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.588+0000 7f083471d640 1 --2- 172.21.15.125:0/2181021506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0818003970 0x7f0818003d70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.588+0000 7f083471d640 1 -- 172.21.15.125:0/2181021506 >> 172.21.15.125:0/2181021506 conn(0x7f082c0d0d60 msgr2=0x7f082c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:04.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.588+0000 7f083471d640 1 -- 172.21.15.125:0/2181021506 shutdown_connections 2023-10-18T10:26:04.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.588+0000 7f083471d640 1 -- 172.21.15.125:0/2181021506 wait complete. 2023-10-18T10:26:04.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.589+0000 7f083471d640 1 Processor -- start 2023-10-18T10:26:04.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.589+0000 7f083471d640 1 -- start start 2023-10-18T10:26:04.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f083471d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0818003970 0x7f082c1e3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f083471d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f082c1e3a50 0x7f082c1e5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f083471d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f082c1e58c0 0x7f082c1e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f083471d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c1e3ed0 con 0x7f082c1e3a50 2023-10-18T10:26:04.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f083471d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c15bc60 con 0x7f082c1e58c0 2023-10-18T10:26:04.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f083471d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c1e6280 con 0x7f0818003970 2023-10-18T10:26:04.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f082c1e3a50 0x7f082c1e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f082c1e3a50 0x7f082c1e5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36766/0 (socket says 172.21.15.125:36766) 2023-10-18T10:26:04.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 -- 172.21.15.125:0/2623784058 learned_addr learned my addr 172.21.15.125:0/2623784058 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:04.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 -- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0818003970 msgr2=0x7f082c1e3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0818003970 0x7f082c1e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 -- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f082c1e58c0 msgr2=0x7f082c1e5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f082c1e58c0 0x7f082c1e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 -- 172.21.15.125:0/2623784058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c15bf00 con 0x7f082c1e3a50 2023-10-18T10:26:04.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.590+0000 7f0831cb8640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f082c1e3a50 0x7f082c1e5380 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f081c00d5d0 tx=0x7f081c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:04.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.591+0000 7f0822ffd640 1 -- 172.21.15.125:0/2623784058 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c00f880 con 0x7f082c1e3a50 2023-10-18T10:26:04.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.591+0000 7f0822ffd640 1 -- 172.21.15.125:0/2623784058 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f081c00faf0 con 0x7f082c1e3a50 2023-10-18T10:26:04.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.591+0000 7f0822ffd640 1 -- 172.21.15.125:0/2623784058 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c010450 con 0x7f082c1e3a50 2023-10-18T10:26:04.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.591+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c15c1e0 con 0x7f082c1e3a50 2023-10-18T10:26:04.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.591+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c15c7a0 con 0x7f082c1e3a50 2023-10-18T10:26:04.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.592+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c0690e0 con 0x7f082c1e3a50 2023-10-18T10:26:04.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.592+0000 7f0822ffd640 1 -- 172.21.15.125:0/2623784058 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f081c0105b0 con 0x7f082c1e3a50 2023-10-18T10:26:04.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.592+0000 7f0822ffd640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f081006ae40 0x7f081006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.592+0000 7f0822ffd640 1 -- 172.21.15.125:0/2623784058 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f081c089ff0 con 0x7f082c1e3a50 2023-10-18T10:26:04.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.593+0000 7f08324b9640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f081006ae40 0x7f081006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.593+0000 7f08324b9640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f081006ae40 0x7f081006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0828034000 tx=0x7f0828039040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:04.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.597+0000 7f0822ffd640 1 -- 172.21.15.125:0/2623784058 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f081c054d70 con 0x7f082c1e3a50 2023-10-18T10:26:04.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.763+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f082c0d0b40 con 0x7f082c1e3a50 2023-10-18T10:26:04.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.763+0000 7f0822ffd640 1 -- 172.21.15.125:0/2623784058 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f081c01e0a0 con 0x7f082c1e3a50 2023-10-18T10:26:04.765 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:26:04.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f081006ae40 msgr2=0x7f081006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f081006ae40 0x7f081006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0828034000 tx=0x7f0828039040 comp rx=0 tx=0).stop 2023-10-18T10:26:04.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f082c1e3a50 msgr2=0x7f082c1e5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f082c1e3a50 0x7f082c1e5380 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f081c00d5d0 tx=0x7f081c00da90 comp rx=0 tx=0).stop 2023-10-18T10:26:04.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 shutdown_connections 2023-10-18T10:26:04.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f081006ae40 0x7f081006d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f082c1e58c0 0x7f082c1e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f082c1e3a50 0x7f082c1e5380 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 --2- 172.21.15.125:0/2623784058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0818003970 0x7f082c1e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 >> 172.21.15.125:0/2623784058 conn(0x7f082c0d0d60 msgr2=0x7f082c1e67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:04.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 shutdown_connections 2023-10-18T10:26:04.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.765+0000 7f083471d640 1 -- 172.21.15.125:0/2623784058 wait complete. 2023-10-18T10:26:04.777 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T10:26:04.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.881+0000 7fbaedc10640 1 -- 172.21.15.125:0/1106065711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbad00058d0 msgr2=0x7fbad0005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.881+0000 7fbaedc10640 1 --2- 172.21.15.125:0/1106065711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbad00058d0 0x7fbad0005d50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbadc006fd0 tx=0x7fbad0006900 comp rx=0 tx=0).stop 2023-10-18T10:26:04.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.881+0000 7fbaedc10640 1 -- 172.21.15.125:0/1106065711 shutdown_connections 2023-10-18T10:26:04.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.881+0000 7fbaedc10640 1 --2- 172.21.15.125:0/1106065711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbad00058d0 0x7fbad0005d50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.881+0000 7fbaedc10640 1 -- 172.21.15.125:0/1106065711 >> 172.21.15.125:0/1106065711 conn(0x7fbae8069930 msgr2=0x7fbae8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:04.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.881+0000 7fbaedc10640 1 -- 172.21.15.125:0/1106065711 shutdown_connections 2023-10-18T10:26:04.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.881+0000 7fbaedc10640 1 -- 172.21.15.125:0/1106065711 wait complete. 2023-10-18T10:26:04.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.882+0000 7fbaedc10640 1 Processor -- start 2023-10-18T10:26:04.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbaedc10640 1 -- start start 2023-10-18T10:26:04.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbaedc10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad00058d0 0x7fbae81e5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbaedc10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae8159360 0x7fbae81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbaedc10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbae8152d20 0x7fbae813fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbaedc10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8159800 con 0x7fbae8152d20 2023-10-18T10:26:04.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbaedc10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8159ae0 con 0x7fbad00058d0 2023-10-18T10:26:04.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbaedc10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8159dc0 con 0x7fbae8159360 2023-10-18T10:26:04.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbae7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbae8152d20 0x7fbae813fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbae7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbae8152d20 0x7fbae813fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36788/0 (socket says 172.21.15.125:36788) 2023-10-18T10:26:04.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.883+0000 7fbae7fff640 1 -- 172.21.15.125:0/1277653300 learned_addr learned my addr 172.21.15.125:0/1277653300 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:04.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbae6ffd640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae8159360 0x7fbae81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbae77fe640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad00058d0 0x7fbae81e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbae7fff640 1 -- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae8159360 msgr2=0x7fbae81527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbae7fff640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae8159360 0x7fbae81527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbae7fff640 1 -- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad00058d0 msgr2=0x7fbae81e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:04.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbae7fff640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad00058d0 0x7fbae81e5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:04.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbae7fff640 1 -- 172.21.15.125:0/1277653300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae815a0a0 con 0x7fbae8152d20 2023-10-18T10:26:04.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbae7fff640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbae8152d20 0x7fbae813fe60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbae806f6d0 tx=0x7fbae0001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:04.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbacffff640 1 -- 172.21.15.125:0/1277653300 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011840 con 0x7fbae8152d20 2023-10-18T10:26:04.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbacffff640 1 -- 172.21.15.125:0/1277653300 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbae00119a0 con 0x7fbae8152d20 2023-10-18T10:26:04.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae815a380 con 0x7fbae8152d20 2023-10-18T10:26:04.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbacffff640 1 -- 172.21.15.125:0/1277653300 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011db0 con 0x7fbae8152d20 2023-10-18T10:26:04.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.884+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae81e5850 con 0x7fbae8152d20 2023-10-18T10:26:04.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.885+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae81e59e0 con 0x7fbae8152d20 2023-10-18T10:26:04.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.885+0000 7fbacffff640 1 -- 172.21.15.125:0/1277653300 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbae001d460 con 0x7fbae8152d20 2023-10-18T10:26:04.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.886+0000 7fbacffff640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbad406ae40 0x7fbad406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:04.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.886+0000 7fbacffff640 1 -- 172.21.15.125:0/1277653300 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbae008a3f0 con 0x7fbae8152d20 2023-10-18T10:26:04.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.886+0000 7fbae77fe640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbad406ae40 0x7fbad406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:04.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.887+0000 7fbae77fe640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbad406ae40 0x7fbad406d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009a90 tx=0x7fbad8005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:04.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:04.890+0000 7fbacffff640 1 -- 172.21.15.125:0/1277653300 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbae00550e0 con 0x7fbae8152d20 2023-10-18T10:26:05.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.059+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbae80690e0 con 0x7fbae8152d20 2023-10-18T10:26:05.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.059+0000 7fbacffff640 1 -- 172.21.15.125:0/1277653300 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbae0058700 con 0x7fbae8152d20 2023-10-18T10:26:05.061 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:26:05.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.061+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbad406ae40 msgr2=0x7fbad406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.061+0000 7fbaedc10640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbad406ae40 0x7fbad406d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009a90 tx=0x7fbad8005f90 comp rx=0 tx=0).stop 2023-10-18T10:26:05.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.061+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbae8152d20 msgr2=0x7fbae813fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.061+0000 7fbaedc10640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbae8152d20 0x7fbae813fe60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbae806f6d0 tx=0x7fbae0001c30 comp rx=0 tx=0).stop 2023-10-18T10:26:05.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.062+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 shutdown_connections 2023-10-18T10:26:05.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.062+0000 7fbaedc10640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbad406ae40 0x7fbad406d300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.062+0000 7fbaedc10640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbae8152d20 0x7fbae813fe60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.062+0000 7fbaedc10640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae8159360 0x7fbae81527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.062+0000 7fbaedc10640 1 --2- 172.21.15.125:0/1277653300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad00058d0 0x7fbae81e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.062+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 >> 172.21.15.125:0/1277653300 conn(0x7fbae8069930 msgr2=0x7fbae81578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:05.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.062+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 shutdown_connections 2023-10-18T10:26:05.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.062+0000 7fbaedc10640 1 -- 172.21.15.125:0/1277653300 wait complete. 2023-10-18T10:26:05.076 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:26:05.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.178+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/3036791091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a10005180 msgr2=0x7f8a1000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.178+0000 7f8a2d7e2640 1 --2- 172.21.15.125:0/3036791091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a10005180 0x7f8a1000b980 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8a20001680 tx=0x7f8a100057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.179+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/3036791091 shutdown_connections 2023-10-18T10:26:05.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.179+0000 7f8a2d7e2640 1 --2- 172.21.15.125:0/3036791091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a10005180 0x7f8a1000b980 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.179+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/3036791091 >> 172.21.15.125:0/3036791091 conn(0x7f8a2809a030 msgr2=0x7f8a280d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:05.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.179+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/3036791091 shutdown_connections 2023-10-18T10:26:05.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.179+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/3036791091 wait complete. 2023-10-18T10:26:05.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.180+0000 7f8a2d7e2640 1 Processor -- start 2023-10-18T10:26:05.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.180+0000 7f8a2d7e2640 1 -- start start 2023-10-18T10:26:05.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a2d7e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a10005180 0x7f8a28158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a2d7e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a280b38e0 0x7f8a281f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a2d7e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a2806b230 0x7f8a2806b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a2d7e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a28158ca0 con 0x7f8a280b38e0 2023-10-18T10:26:05.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a2d7e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2806bbf0 con 0x7f8a10005180 2023-10-18T10:26:05.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a2d7e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a281f27e0 con 0x7f8a2806b230 2023-10-18T10:26:05.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a280b38e0 0x7f8a281f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a280b38e0 0x7f8a281f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36816/0 (socket says 172.21.15.125:36816) 2023-10-18T10:26:05.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 -- 172.21.15.125:0/1343640974 learned_addr learned my addr 172.21.15.125:0/1343640974 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:05.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a26ffd640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a10005180 0x7f8a28158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a277fe640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a2806b230 0x7f8a2806b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 -- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a2806b230 msgr2=0x7f8a2806b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a2806b230 0x7f8a2806b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 -- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a10005180 msgr2=0x7f8a28158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a10005180 0x7f8a28158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 -- 172.21.15.125:0/1343640974 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a281f25b0 con 0x7f8a280b38e0 2023-10-18T10:26:05.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a277fe640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a2806b230 0x7f8a2806b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:05.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a26ffd640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a10005180 0x7f8a28158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:05.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a267fc640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a280b38e0 0x7f8a281f2070 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00e570 tx=0x7f8a1c00ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:05.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a0f7fe640 1 -- 172.21.15.125:0/1343640974 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00c8f0 con 0x7f8a280b38e0 2023-10-18T10:26:05.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a0f7fe640 1 -- 172.21.15.125:0/1343640974 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a1c00cb40 con 0x7f8a280b38e0 2023-10-18T10:26:05.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a281f2a80 con 0x7f8a280b38e0 2023-10-18T10:26:05.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a0f7fe640 1 -- 172.21.15.125:0/1343640974 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c011460 con 0x7f8a280b38e0 2023-10-18T10:26:05.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.181+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a281f3040 con 0x7f8a280b38e0 2023-10-18T10:26:05.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.182+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2808a400 con 0x7f8a280b38e0 2023-10-18T10:26:05.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.183+0000 7f8a0f7fe640 1 -- 172.21.15.125:0/1343640974 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a1c0115c0 con 0x7f8a280b38e0 2023-10-18T10:26:05.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.183+0000 7f8a0f7fe640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1406ae70 0x7f8a1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.183+0000 7f8a0f7fe640 1 -- 172.21.15.125:0/1343640974 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8a1c08a100 con 0x7f8a280b38e0 2023-10-18T10:26:05.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.183+0000 7f8a26ffd640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1406ae70 0x7f8a1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.184+0000 7f8a26ffd640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1406ae70 0x7f8a1406d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009810 tx=0x7f8a18009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:05.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.187+0000 7f8a0f7fe640 1 -- 172.21.15.125:0/1343640974 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a1c054e80 con 0x7f8a280b38e0 2023-10-18T10:26:05.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.387+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a280675f0 con 0x7f8a280b38e0 2023-10-18T10:26:05.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.387+0000 7f8a0f7fe640 1 -- 172.21.15.125:0/1343640974 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f8a1c015090 con 0x7f8a280b38e0 2023-10-18T10:26:05.390 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:05.390 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:19.519174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:26:05.391 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 9 2023-10-18T10:26:05.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1406ae70 msgr2=0x7f8a1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1406ae70 0x7f8a1406d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009810 tx=0x7f8a18009500 comp rx=0 tx=0).stop 2023-10-18T10:26:05.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a280b38e0 msgr2=0x7f8a281f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a280b38e0 0x7f8a281f2070 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00e570 tx=0x7f8a1c00ea30 comp rx=0 tx=0).stop 2023-10-18T10:26:05.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 shutdown_connections 2023-10-18T10:26:05.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f8a1406ae70 0x7f8a1406d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a2806b230 0x7f8a2806b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a280b38e0 0x7f8a281f2070 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 --2- 172.21.15.125:0/1343640974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a10005180 0x7f8a28158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 >> 172.21.15.125:0/1343640974 conn(0x7f8a2809a030 msgr2=0x7f8a2815a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:05.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 shutdown_connections 2023-10-18T10:26:05.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.389+0000 7f8a2d7e2640 1 -- 172.21.15.125:0/1343640974 wait complete. 2023-10-18T10:26:05.402 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:26:05.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.505+0000 7f7e2334c640 1 -- 172.21.15.125:0/1377454955 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 msgr2=0x7f7e1c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.505+0000 7f7e2334c640 1 --2- 172.21.15.125:0/1377454955 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 0x7f7e1c0b0650 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005fd0 tx=0x7f7e1402c0e0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.505+0000 7f7e2334c640 1 -- 172.21.15.125:0/1377454955 shutdown_connections 2023-10-18T10:26:05.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.505+0000 7f7e2334c640 1 --2- 172.21.15.125:0/1377454955 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 0x7f7e1c0b0650 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.505+0000 7f7e2334c640 1 -- 172.21.15.125:0/1377454955 >> 172.21.15.125:0/1377454955 conn(0x7f7e1c0d0d60 msgr2=0x7f7e1c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:05.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.505+0000 7f7e2334c640 1 -- 172.21.15.125:0/1377454955 shutdown_connections 2023-10-18T10:26:05.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.505+0000 7f7e2334c640 1 -- 172.21.15.125:0/1377454955 wait complete. 2023-10-18T10:26:05.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.506+0000 7f7e2334c640 1 Processor -- start 2023-10-18T10:26:05.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.506+0000 7f7e2334c640 1 -- start start 2023-10-18T10:26:05.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e2334c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e1c0cf340 0x7f7e1c14b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e2334c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 0x7f7e1c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e2334c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e1c15b2a0 0x7f7e1c15c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e2334c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1f22d0 con 0x7f7e1c0d1910 2023-10-18T10:26:05.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e2334c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c14be70 con 0x7f7e1c0cf340 2023-10-18T10:26:05.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e2334c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1f1750 con 0x7f7e1c15b2a0 2023-10-18T10:26:05.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e208e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 0x7f7e1c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e208e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 0x7f7e1c0b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36820/0 (socket says 172.21.15.125:36820) 2023-10-18T10:26:05.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e208e7640 1 -- 172.21.15.125:0/3324923302 learned_addr learned my addr 172.21.15.125:0/3324923302 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:05.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e218e9640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e1c15b2a0 0x7f7e1c15c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e210e8640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e1c0cf340 0x7f7e1c14b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e208e7640 1 -- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e1c15b2a0 msgr2=0x7f7e1c15c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e208e7640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e1c15b2a0 0x7f7e1c15c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e208e7640 1 -- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e1c0cf340 msgr2=0x7f7e1c14b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e208e7640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e1c0cf340 0x7f7e1c14b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e208e7640 1 -- 172.21.15.125:0/3324923302 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e14009840 con 0x7f7e1c0d1910 2023-10-18T10:26:05.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e210e8640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e1c0cf340 0x7f7e1c14b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:05.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.507+0000 7f7e218e9640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e1c15b2a0 0x7f7e1c15c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:05.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.508+0000 7f7e208e7640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 0x7f7e1c0b50b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000c430 tx=0x7f7e1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:05.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.508+0000 7f7e19ffb640 1 -- 172.21.15.125:0/3324923302 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e100052d0 con 0x7f7e1c0d1910 2023-10-18T10:26:05.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.508+0000 7f7e19ffb640 1 -- 172.21.15.125:0/3324923302 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7e100035f0 con 0x7f7e1c0d1910 2023-10-18T10:26:05.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.508+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1c1f1a30 con 0x7f7e1c0d1910 2023-10-18T10:26:05.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.508+0000 7f7e19ffb640 1 -- 172.21.15.125:0/3324923302 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10003a00 con 0x7f7e1c0d1910 2023-10-18T10:26:05.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.508+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1c1f1ff0 con 0x7f7e1c0d1910 2023-10-18T10:26:05.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.509+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e1c0677b0 con 0x7f7e1c0d1910 2023-10-18T10:26:05.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.509+0000 7f7e19ffb640 1 -- 172.21.15.125:0/3324923302 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e10003b60 con 0x7f7e1c0d1910 2023-10-18T10:26:05.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.510+0000 7f7e19ffb640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e0806ae40 0x7f7e0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.510+0000 7f7e19ffb640 1 -- 172.21.15.125:0/3324923302 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7e10089140 con 0x7f7e1c0d1910 2023-10-18T10:26:05.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.510+0000 7f7e210e8640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e0806ae40 0x7f7e0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.511+0000 7f7e210e8640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e0806ae40 0x7f7e0806d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c007a90 tx=0x7f7e0c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:05.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.514+0000 7f7e19ffb640 1 -- 172.21.15.125:0/3324923302 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e10053ec0 con 0x7f7e1c0d1910 2023-10-18T10:26:05.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.684+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e1c0690e0 con 0x7f7e1c0d1910 2023-10-18T10:26:05.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.686+0000 7f7e19ffb640 1 -- 172.21.15.125:0/3324923302 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f7e100574e0 con 0x7f7e1c0d1910 2023-10-18T10:26:05.688 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:05.688 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:26:05.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e0806ae40 msgr2=0x7f7e0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e0806ae40 0x7f7e0806d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c007a90 tx=0x7f7e0c009040 comp rx=0 tx=0).stop 2023-10-18T10:26:05.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 msgr2=0x7f7e1c0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 0x7f7e1c0b50b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000c430 tx=0x7f7e1000c8f0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 shutdown_connections 2023-10-18T10:26:05.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e0806ae40 0x7f7e0806d300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e1c15b2a0 0x7f7e1c15c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1c0d1910 0x7f7e1c0b50b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 --2- 172.21.15.125:0/3324923302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e1c0cf340 0x7f7e1c14b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 >> 172.21.15.125:0/3324923302 conn(0x7f7e1c0d0d60 msgr2=0x7f7e1c0b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:05.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 shutdown_connections 2023-10-18T10:26:05.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.688+0000 7f7e2334c640 1 -- 172.21.15.125:0/3324923302 wait complete. 2023-10-18T10:26:05.701 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:26:05.701 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:26:05.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.802+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/187410779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 msgr2=0x7fc8d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.802+0000 7fc8eaa7c640 1 --2- 172.21.15.125:0/187410779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 0x7fc8d4004c40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009b20 tx=0x7fc8d40057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.802+0000 7fc8e9019640 1 -- 172.21.15.125:0/187410779 reap_dead start 2023-10-18T10:26:05.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.802+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/187410779 shutdown_connections 2023-10-18T10:26:05.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.802+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/187410779 >> 172.21.15.125:0/187410779 conn(0x7fc8e40af730 msgr2=0x7fc8e40afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:05.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.802+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/187410779 shutdown_connections 2023-10-18T10:26:05.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.802+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/187410779 wait complete. 2023-10-18T10:26:05.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.803+0000 7fc8eaa7c640 1 Processor -- start 2023-10-18T10:26:05.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.803+0000 7fc8eaa7c640 1 -- start start 2023-10-18T10:26:05.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8eaa7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 0x7fc8e41e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8eaa7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8e4135ac0 0x7fc8e4136e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8eaa7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8e4135f40 0x7fc8e4137340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8eaa7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41e1770 con 0x7fc8d40047c0 2023-10-18T10:26:05.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8eaa7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41e1a50 con 0x7fc8e4135f40 2023-10-18T10:26:05.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8eaa7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e41e1d30 con 0x7fc8e4135ac0 2023-10-18T10:26:05.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8e8818640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 0x7fc8e41e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8e8818640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 0x7fc8e41e2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36842/0 (socket says 172.21.15.125:36842) 2023-10-18T10:26:05.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8e8818640 1 -- 172.21.15.125:0/2729646197 learned_addr learned my addr 172.21.15.125:0/2729646197 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:05.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8e9019640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8e4135f40 0x7fc8e4137340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.804+0000 7fc8e3fff640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8e4135ac0 0x7fc8e4136e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e8818640 1 -- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8e4135ac0 msgr2=0x7fc8e4136e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e8818640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8e4135ac0 0x7fc8e4136e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e8818640 1 -- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8e4135f40 msgr2=0x7fc8e4137340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:05.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e8818640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8e4135f40 0x7fc8e4137340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:05.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e8818640 1 -- 172.21.15.125:0/2729646197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40b1cd0 con 0x7fc8d40047c0 2023-10-18T10:26:05.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e8818640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 0x7fc8e41e2af0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8000fb0 tx=0x7fc8d800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:05.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e17fa640 1 -- 172.21.15.125:0/2729646197 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800c910 con 0x7fc8d40047c0 2023-10-18T10:26:05.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e17fa640 1 -- 172.21.15.125:0/2729646197 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc8d800cb60 con 0x7fc8d40047c0 2023-10-18T10:26:05.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e41e2010 con 0x7fc8d40047c0 2023-10-18T10:26:05.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8e17fa640 1 -- 172.21.15.125:0/2729646197 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8011460 con 0x7fc8d40047c0 2023-10-18T10:26:05.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.805+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e41e25d0 con 0x7fc8d40047c0 2023-10-18T10:26:05.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.806+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e40b1ad0 con 0x7fc8d40047c0 2023-10-18T10:26:05.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.806+0000 7fc8e17fa640 1 -- 172.21.15.125:0/2729646197 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc8d80115c0 con 0x7fc8d40047c0 2023-10-18T10:26:05.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.807+0000 7fc8e17fa640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc8c806ae70 0x7fc8c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:05.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.807+0000 7fc8e17fa640 1 -- 172.21.15.125:0/2729646197 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc8d808a100 con 0x7fc8d40047c0 2023-10-18T10:26:05.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.807+0000 7fc8e3fff640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc8c806ae70 0x7fc8c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:05.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.807+0000 7fc8e3fff640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc8c806ae70 0x7fc8c806d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0036000 tx=0x7fc8d00046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:05.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:05.811+0000 7fc8e17fa640 1 -- 172.21.15.125:0/2729646197 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8d8054e80 con 0x7fc8d40047c0 2023-10-18T10:26:06.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.009+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8e40675f0 con 0x7fc8d40047c0 2023-10-18T10:26:06.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.010+0000 7fc8e17fa640 1 -- 172.21.15.125:0/2729646197 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fc8d8015090 con 0x7fc8d40047c0 2023-10-18T10:26:06.012 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:06.012 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:19.519174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:26:06.013 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 9 2023-10-18T10:26:06.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc8c806ae70 msgr2=0x7fc8c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc8c806ae70 0x7fc8c806d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0036000 tx=0x7fc8d00046c0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 msgr2=0x7fc8e41e2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 0x7fc8e41e2af0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8000fb0 tx=0x7fc8d800ea70 comp rx=0 tx=0).stop 2023-10-18T10:26:06.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 shutdown_connections 2023-10-18T10:26:06.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc8c806ae70 0x7fc8c806d330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8e4135f40 0x7fc8e4137340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8e4135ac0 0x7fc8e4136e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 --2- 172.21.15.125:0/2729646197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8d40047c0 0x7fc8e41e2af0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 >> 172.21.15.125:0/2729646197 conn(0x7fc8e40af730 msgr2=0x7fc8e40c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:06.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 shutdown_connections 2023-10-18T10:26:06.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.012+0000 7fc8eaa7c640 1 -- 172.21.15.125:0/2729646197 wait complete. 2023-10-18T10:26:06.024 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:26:06.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.126+0000 7fb1b6b76640 1 -- 172.21.15.125:0/2021568633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1a00047c0 msgr2=0x7fb1a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.126+0000 7fb1b6b76640 1 --2- 172.21.15.125:0/2021568633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1a00047c0 0x7fb1a0004c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00baf0 tx=0x7fb1a00057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.126+0000 7fb1b5113640 1 -- 172.21.15.125:0/2021568633 reap_dead start 2023-10-18T10:26:06.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.126+0000 7fb1b6b76640 1 -- 172.21.15.125:0/2021568633 shutdown_connections 2023-10-18T10:26:06.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.126+0000 7fb1b6b76640 1 -- 172.21.15.125:0/2021568633 >> 172.21.15.125:0/2021568633 conn(0x7fb1b0069930 msgr2=0x7fb1b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:06.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.126+0000 7fb1b6b76640 1 -- 172.21.15.125:0/2021568633 shutdown_connections 2023-10-18T10:26:06.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.126+0000 7fb1b6b76640 1 -- 172.21.15.125:0/2021568633 wait complete. 2023-10-18T10:26:06.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.128+0000 7fb1b6b76640 1 Processor -- start 2023-10-18T10:26:06.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.128+0000 7fb1b6b76640 1 -- start start 2023-10-18T10:26:06.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.128+0000 7fb1b6b76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1a00047c0 0x7fb1b00ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.128+0000 7fb1b6b76640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b00aa9f0 0x7fb1b01526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.128+0000 7fb1b6b76640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0152c20 0x7fb1b00a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.128+0000 7fb1b6b76640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b01530d0 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.128+0000 7fb1b6b76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b00a95b0 con 0x7fb1a00047c0 2023-10-18T10:26:06.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.128+0000 7fb1b6b76640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b00adb90 con 0x7fb1b0152c20 2023-10-18T10:26:06.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b00aa9f0 0x7fb1b01526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b00aa9f0 0x7fb1b01526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36856/0 (socket says 172.21.15.125:36856) 2023-10-18T10:26:06.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 -- 172.21.15.125:0/4173114893 learned_addr learned my addr 172.21.15.125:0/4173114893 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:06.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1b5113640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0152c20 0x7fb1b00a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1b4912640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1a00047c0 0x7fb1b00ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 -- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0152c20 msgr2=0x7fb1b00a7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0152c20 0x7fb1b00a7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 -- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1a00047c0 msgr2=0x7fb1b00ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1a00047c0 0x7fb1b00ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 -- 172.21.15.125:0/4173114893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00a89a0 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1b4912640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1a00047c0 0x7fb1b00ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:06.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1b5113640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0152c20 0x7fb1b00a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:06.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1affff640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b00aa9f0 0x7fb1b01526e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00baf0 tx=0x7fb19c036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:06.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1ad7fa640 1 -- 172.21.15.125:0/4173114893 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c036d20 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1ad7fa640 1 -- 172.21.15.125:0/4173114893 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb19c004030 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b00a8c80 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1ad7fa640 1 -- 172.21.15.125:0/4173114893 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c004400 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.129+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b00a9240 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.130+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b00a93d0 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.131+0000 7fb1ad7fa640 1 -- 172.21.15.125:0/4173114893 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb19c0035f0 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.131+0000 7fb1ad7fa640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb19406ae70 0x7fb19406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.131+0000 7fb1ad7fa640 1 -- 172.21.15.125:0/4173114893 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb19c0abb60 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.131+0000 7fb1b4912640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb19406ae70 0x7fb19406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.132+0000 7fb1b4912640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb19406ae70 0x7fb19406d330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4004500 tx=0x7fb1a400b260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:06.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.135+0000 7fb1ad7fa640 1 -- 172.21.15.125:0/4173114893 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb19c0767d0 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.336+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1b00690e0 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.336+0000 7fb1ad7fa640 1 -- 172.21.15.125:0/4173114893 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fb19c048090 con 0x7fb1b00aa9f0 2023-10-18T10:26:06.339 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:06.339 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1993468497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1993468497},{"type":"v1","addr":"172.21.15.139:6836","nonce":1993468497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4360,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/577330753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":577330753},{"type":"v1","addr":"172.21.15.139:6837","nonce":577330753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3783162976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3783162976},{"type":"v1","addr":"172.21.15.125:6835","nonce":3783162976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:25:03.534893+0000","modified":"2023-10-18T10:25:19.519174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/4043797924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":4043797924},{"type":"v1","addr":"172.21.15.125:6834","nonce":4043797924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T10:26:06.340 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 9 2023-10-18T10:26:06.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb19406ae70 msgr2=0x7fb19406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb19406ae70 0x7fb19406d330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4004500 tx=0x7fb1a400b260 comp rx=0 tx=0).stop 2023-10-18T10:26:06.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b00aa9f0 msgr2=0x7fb1b01526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b00aa9f0 0x7fb1b01526e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00baf0 tx=0x7fb19c036720 comp rx=0 tx=0).stop 2023-10-18T10:26:06.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 shutdown_connections 2023-10-18T10:26:06.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fb19406ae70 0x7fb19406d330 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0152c20 0x7fb1b00a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b00aa9f0 0x7fb1b01526e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 --2- 172.21.15.125:0/4173114893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1a00047c0 0x7fb1b00ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 >> 172.21.15.125:0/4173114893 conn(0x7fb1b0069930 msgr2=0x7fb1b00b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:06.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 shutdown_connections 2023-10-18T10:26:06.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.338+0000 7fb1b6b76640 1 -- 172.21.15.125:0/4173114893 wait complete. 2023-10-18T10:26:06.351 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:26:06.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.451+0000 7f458284f640 1 -- 172.21.15.125:0/3563865717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c152120 msgr2=0x7f457c0b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.451+0000 7f458284f640 1 --2- 172.21.15.125:0/3563865717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c152120 0x7f457c0b12a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f456c005fd0 tx=0x7f456c02be10 comp rx=0 tx=0).stop 2023-10-18T10:26:06.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.451+0000 7f458284f640 1 -- 172.21.15.125:0/3563865717 shutdown_connections 2023-10-18T10:26:06.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.451+0000 7f458284f640 1 --2- 172.21.15.125:0/3563865717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f457c06e7c0 0x7f457c069de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.451+0000 7f458284f640 1 --2- 172.21.15.125:0/3563865717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c152120 0x7f457c0b12a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.451+0000 7f458284f640 1 -- 172.21.15.125:0/3563865717 >> 172.21.15.125:0/3563865717 conn(0x7f457c0af730 msgr2=0x7f457c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:06.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.451+0000 7f458284f640 1 -- 172.21.15.125:0/3563865717 shutdown_connections 2023-10-18T10:26:06.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.451+0000 7f458284f640 1 -- 172.21.15.125:0/3563865717 wait complete. 2023-10-18T10:26:06.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.452+0000 7f458284f640 1 Processor -- start 2023-10-18T10:26:06.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f458284f640 1 -- start start 2023-10-18T10:26:06.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f458284f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f457c069950 0x7f457c142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f458284f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c06e7c0 0x7f457c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f458284f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457c152120 0x7f457c1f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f458284f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c15d160 con 0x7f457c06e7c0 2023-10-18T10:26:06.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f458284f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c15c850 con 0x7f457c152120 2023-10-18T10:26:06.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f458284f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457c142b40 con 0x7f457c069950 2023-10-18T10:26:06.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f457b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c06e7c0 0x7f457c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f457b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c06e7c0 0x7f457c0caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36886/0 (socket says 172.21.15.125:36886) 2023-10-18T10:26:06.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.453+0000 7f457b7fe640 1 -- 172.21.15.125:0/2455680975 learned_addr learned my addr 172.21.15.125:0/2455680975 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:06.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f457b7fe640 1 -- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f457c069950 msgr2=0x7f457c142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f457b7fe640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f457c069950 0x7f457c142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f457b7fe640 1 -- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457c152120 msgr2=0x7f457c1f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f457b7fe640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457c152120 0x7f457c1f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f457b7fe640 1 -- 172.21.15.125:0/2455680975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c009560 con 0x7f457c06e7c0 2023-10-18T10:26:06.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f457b7fe640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c06e7c0 0x7f457c0caa00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f457000c480 tx=0x7f457000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:06.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f4578ff9640 1 -- 172.21.15.125:0/2455680975 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45700052d0 con 0x7f457c06e7c0 2023-10-18T10:26:06.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f4578ff9640 1 -- 172.21.15.125:0/2455680975 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4570005430 con 0x7f457c06e7c0 2023-10-18T10:26:06.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f4578ff9640 1 -- 172.21.15.125:0/2455680975 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570012890 con 0x7f457c06e7c0 2023-10-18T10:26:06.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457c0cb3a0 con 0x7f457c06e7c0 2023-10-18T10:26:06.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.454+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457c15cd80 con 0x7f457c06e7c0 2023-10-18T10:26:06.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.455+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457c0675f0 con 0x7f457c06e7c0 2023-10-18T10:26:06.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.455+0000 7f4578ff9640 1 -- 172.21.15.125:0/2455680975 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f45700035f0 con 0x7f457c06e7c0 2023-10-18T10:26:06.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.456+0000 7f4578ff9640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f456806ae40 0x7f456806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.456+0000 7f4578ff9640 1 -- 172.21.15.125:0/2455680975 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4570089670 con 0x7f457c06e7c0 2023-10-18T10:26:06.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.456+0000 7f457bfff640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f456806ae40 0x7f456806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.456+0000 7f457bfff640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f456806ae40 0x7f456806d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f456c035000 tx=0x7f456c0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:06.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.460+0000 7f4578ff9640 1 -- 172.21.15.125:0/2455680975 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45700543f0 con 0x7f457c06e7c0 2023-10-18T10:26:06.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.628+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f457c15cf10 con 0x7f457c06e7c0 2023-10-18T10:26:06.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.629+0000 7f4578ff9640 1 -- 172.21.15.125:0/2455680975 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f4570057a10 con 0x7f457c06e7c0 2023-10-18T10:26:06.631 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:06.631 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":27,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:25:44.360081+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:25:01.477046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T10:25:02.515145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T10:25:06.263035+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:26:06.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f456806ae40 msgr2=0x7f456806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f456806ae40 0x7f456806d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f456c035000 tx=0x7f456c0046c0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c06e7c0 msgr2=0x7f457c0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c06e7c0 0x7f457c0caa00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f457000c480 tx=0x7f457000c940 comp rx=0 tx=0).stop 2023-10-18T10:26:06.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 shutdown_connections 2023-10-18T10:26:06.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f456806ae40 0x7f456806d300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457c152120 0x7f457c1f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f457c06e7c0 0x7f457c0caa00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 --2- 172.21.15.125:0/2455680975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f457c069950 0x7f457c142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 >> 172.21.15.125:0/2455680975 conn(0x7f457c0af730 msgr2=0x7f457c0aad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:06.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 shutdown_connections 2023-10-18T10:26:06.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.631+0000 7f458284f640 1 -- 172.21.15.125:0/2455680975 wait complete. 2023-10-18T10:26:06.644 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:26:06.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.746+0000 7fcb8231d640 1 -- 172.21.15.125:0/3066144964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb7c070920 msgr2=0x7fcb7c0affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.746+0000 7fcb8231d640 1 --2- 172.21.15.125:0/3066144964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb7c070920 0x7fcb7c0affe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005fd0 tx=0x7fcb7402c240 comp rx=0 tx=0).stop 2023-10-18T10:26:06.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.746+0000 7fcb8231d640 1 -- 172.21.15.125:0/3066144964 shutdown_connections 2023-10-18T10:26:06.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.746+0000 7fcb8231d640 1 --2- 172.21.15.125:0/3066144964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb7c070920 0x7fcb7c0affe0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.746+0000 7fcb8231d640 1 --2- 172.21.15.125:0/3066144964 >> v2:172.21.15.139:3300/0 conn(0x7fcb7c141690 0x7fcb7c0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.746+0000 7fcb8231d640 1 -- 172.21.15.125:0/3066144964 >> 172.21.15.125:0/3066144964 conn(0x7fcb7c069930 msgr2=0x7fcb7c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:06.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.746+0000 7fcb8231d640 1 -- 172.21.15.125:0/3066144964 shutdown_connections 2023-10-18T10:26:06.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.746+0000 7fcb8231d640 1 -- 172.21.15.125:0/3066144964 wait complete. 2023-10-18T10:26:06.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.747+0000 7fcb8231d640 1 Processor -- start 2023-10-18T10:26:06.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.747+0000 7fcb8231d640 1 -- start start 2023-10-18T10:26:06.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb8231d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb7c070920 0x7fcb7c1de400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb8231d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb7c13e520 0x7fcb7c0a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb8231d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb7c141690 0x7fcb7c13f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb8231d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c0a7710 con 0x7fcb7c141690 2023-10-18T10:26:06.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb8231d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c0a79f0 con 0x7fcb7c13e520 2023-10-18T10:26:06.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb8231d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c0a7cd0 con 0x7fcb7c070920 2023-10-18T10:26:06.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb7c141690 0x7fcb7c13f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb7c141690 0x7fcb7c13f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36896/0 (socket says 172.21.15.125:36896) 2023-10-18T10:26:06.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 -- 172.21.15.125:0/1109763132 learned_addr learned my addr 172.21.15.125:0/1109763132 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:06.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb7b7fe640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb7c13e520 0x7fcb7c0a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb7bfff640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb7c070920 0x7fcb7c1de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 -- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb7c070920 msgr2=0x7fcb7c1de400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb7c070920 0x7fcb7c1de400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 -- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb7c13e520 msgr2=0x7fcb7c0a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb7c13e520 0x7fcb7c0a6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 -- 172.21.15.125:0/1109763132 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74009900 con 0x7fcb7c141690 2023-10-18T10:26:06.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.748+0000 7fcb808ba640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb7c141690 0x7fcb7c13f9e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009bf0 tx=0x7fcb74035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:06.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.749+0000 7fcb78ff9640 1 -- 172.21.15.125:0/1109763132 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74034440 con 0x7fcb7c141690 2023-10-18T10:26:06.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.749+0000 7fcb78ff9640 1 -- 172.21.15.125:0/1109763132 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb74003e60 con 0x7fcb7c141690 2023-10-18T10:26:06.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.749+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c0a7fb0 con 0x7fcb7c141690 2023-10-18T10:26:06.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.749+0000 7fcb78ff9640 1 -- 172.21.15.125:0/1109763132 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb740044b0 con 0x7fcb7c141690 2023-10-18T10:26:06.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.749+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c0a73b0 con 0x7fcb7c141690 2023-10-18T10:26:06.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.750+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c0a98a0 con 0x7fcb7c141690 2023-10-18T10:26:06.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.750+0000 7fcb78ff9640 1 -- 172.21.15.125:0/1109763132 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb7400a040 con 0x7fcb7c141690 2023-10-18T10:26:06.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.751+0000 7fcb78ff9640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb6806ae40 0x7fcb6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:06.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.751+0000 7fcb78ff9640 1 -- 172.21.15.125:0/1109763132 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcb740ac650 con 0x7fcb7c141690 2023-10-18T10:26:06.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.751+0000 7fcb7bfff640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb6806ae40 0x7fcb6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:06.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.751+0000 7fcb7bfff640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb6806ae40 0x7fcb6806d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c007fd0 tx=0x7fcb6c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:06.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.755+0000 7fcb78ff9640 1 -- 172.21.15.125:0/1109763132 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb74077370 con 0x7fcb7c141690 2023-10-18T10:26:06.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.955+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcb7c0a7850 con 0x7fcb7c141690 2023-10-18T10:26:06.958 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T10:26:06.956+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:26:06.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.957+0000 7fcb78ff9640 1 -- 172.21.15.125:0/1109763132 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcb7c0a7850 con 0x7fcb7c141690 2023-10-18T10:26:06.959 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:26:06.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb6806ae40 msgr2=0x7fcb6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb6806ae40 0x7fcb6806d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c007fd0 tx=0x7fcb6c009040 comp rx=0 tx=0).stop 2023-10-18T10:26:06.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb7c141690 msgr2=0x7fcb7c13f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:06.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb7c141690 0x7fcb7c13f9e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009bf0 tx=0x7fcb74035d80 comp rx=0 tx=0).stop 2023-10-18T10:26:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 shutdown_connections 2023-10-18T10:26:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fcb6806ae40 0x7fcb6806d300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb7c13e520 0x7fcb7c0a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb7c141690 0x7fcb7c13f9e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 --2- 172.21.15.125:0/1109763132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb7c070920 0x7fcb7c1de400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 >> 172.21.15.125:0/1109763132 conn(0x7fcb7c069930 msgr2=0x7fcb7c06f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:06.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 shutdown_connections 2023-10-18T10:26:06.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:06.960+0000 7fcb8231d640 1 -- 172.21.15.125:0/1109763132 wait complete. 2023-10-18T10:26:06.964 INFO:tasks.ceph.mds.a.smithi125.stderr: -3952> 2023-10-18T10:25:08.130+0000 7feca2c77640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:26:06.976 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:26:06.995 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:26:07.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.079+0000 7f474e497640 1 -- 172.21.15.125:0/1385422191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47300047c0 msgr2=0x7f4730004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.079+0000 7f474e497640 1 --2- 172.21.15.125:0/1385422191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47300047c0 0x7f4730004c40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f473c00ac60 tx=0x7f47300057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.079+0000 7f474e497640 1 -- 172.21.15.125:0/1385422191 shutdown_connections 2023-10-18T10:26:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.079+0000 7f474e497640 1 --2- 172.21.15.125:0/1385422191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47300047c0 0x7f4730004c40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.079+0000 7f474e497640 1 -- 172.21.15.125:0/1385422191 >> 172.21.15.125:0/1385422191 conn(0x7f4748069930 msgr2=0x7f4748069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.079+0000 7f474e497640 1 -- 172.21.15.125:0/1385422191 shutdown_connections 2023-10-18T10:26:07.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.079+0000 7f474e497640 1 -- 172.21.15.125:0/1385422191 wait complete. 2023-10-18T10:26:07.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.080+0000 7f474e497640 1 Processor -- start 2023-10-18T10:26:07.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f474e497640 1 -- start start 2023-10-18T10:26:07.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f474e497640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47300047c0 0x7f4748147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:07.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f474e497640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47481f11c0 0x7f474814b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:07.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f474e497640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474814bae0 0x7f4748156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:07.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f474e497640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4748158980 con 0x7f47481f11c0 2023-10-18T10:26:07.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f474e497640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4748158c20 con 0x7f474814bae0 2023-10-18T10:26:07.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f474e497640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4748158f00 con 0x7f47300047c0 2023-10-18T10:26:07.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f47477fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47481f11c0 0x7f474814b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:07.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f47477fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47481f11c0 0x7f474814b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36920/0 (socket says 172.21.15.125:36920) 2023-10-18T10:26:07.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.081+0000 7f47477fe640 1 -- 172.21.15.125:0/3591932766 learned_addr learned my addr 172.21.15.125:0/3591932766 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:07.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f474ca34640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474814bae0 0x7f4748156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:07.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f47477fe640 1 -- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47300047c0 msgr2=0x7f4748147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:07.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f4747fff640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47300047c0 0x7f4748147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:07.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f47477fe640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47300047c0 0x7f4748147d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:07.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f47477fe640 1 -- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474814bae0 msgr2=0x7f4748156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:07.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f47477fe640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474814bae0 0x7f4748156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:07.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f47477fe640 1 -- 172.21.15.125:0/3591932766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47481591e0 con 0x7f47481f11c0 2023-10-18T10:26:07.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f474ca34640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474814bae0 0x7f4748156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:07.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f47477fe640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47481f11c0 0x7f474814b5a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f473c036040 tx=0x7f473c03d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:07.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f4744ff9640 1 -- 172.21.15.125:0/3591932766 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c03d990 con 0x7f47481f11c0 2023-10-18T10:26:07.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f4744ff9640 1 -- 172.21.15.125:0/3591932766 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f473c042070 con 0x7f47481f11c0 2023-10-18T10:26:07.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47481594c0 con 0x7f47481f11c0 2023-10-18T10:26:07.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f4744ff9640 1 -- 172.21.15.125:0/3591932766 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c03de60 con 0x7f47481f11c0 2023-10-18T10:26:07.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.082+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4748148250 con 0x7f47481f11c0 2023-10-18T10:26:07.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.083+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474814bef0 con 0x7f47481f11c0 2023-10-18T10:26:07.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.083+0000 7f4744ff9640 1 -- 172.21.15.125:0/3591932766 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f473c03e400 con 0x7f47481f11c0 2023-10-18T10:26:07.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.084+0000 7f4744ff9640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f473406ae40 0x7f473406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:07.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.084+0000 7f4744ff9640 1 -- 172.21.15.125:0/3591932766 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f473c044070 con 0x7f47481f11c0 2023-10-18T10:26:07.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.084+0000 7f4747fff640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f473406ae40 0x7f473406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:07.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.084+0000 7f4747fff640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f473406ae40 0x7f473406d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4738000c00 tx=0x7f4738006b30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:07.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.088+0000 7f4744ff9640 1 -- 172.21.15.125:0/3591932766 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f473c0dfca0 con 0x7f47481f11c0 2023-10-18T10:26:07.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.293+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4748159040 con 0x7f47481f11c0 2023-10-18T10:26:07.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.961+0000 7f4744ff9640 1 -- 172.21.15.125:0/3591932766 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4748159040 con 0x7f47481f11c0 2023-10-18T10:26:07.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f473406ae40 msgr2=0x7f473406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:07.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f473406ae40 0x7f473406d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4738000c00 tx=0x7f4738006b30 comp rx=0 tx=0).stop 2023-10-18T10:26:07.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47481f11c0 msgr2=0x7f474814b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47481f11c0 0x7f474814b5a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f473c036040 tx=0x7f473c03d390 comp rx=0 tx=0).stop 2023-10-18T10:26:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 shutdown_connections 2023-10-18T10:26:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f473406ae40 0x7f473406d300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474814bae0 0x7f4748156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47481f11c0 0x7f474814b5a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:07.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 --2- 172.21.15.125:0/3591932766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47300047c0 0x7f4748147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:07.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 >> 172.21.15.125:0/3591932766 conn(0x7f4748069930 msgr2=0x7f47480ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:07.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 shutdown_connections 2023-10-18T10:26:07.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:07.964+0000 7f474e497640 1 -- 172.21.15.125:0/3591932766 wait complete. 2023-10-18T10:26:07.979 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:26:08.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.129+0000 7f7e3c090640 1 -- 172.21.15.125:0/3494159550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3413d360 msgr2=0x7f7e3415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:08.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.129+0000 7f7e3c090640 1 --2- 172.21.15.125:0/3494159550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3413d360 0x7f7e3415a9f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7e3406f010 tx=0x7f7e3002c260 comp rx=0 tx=0).stop 2023-10-18T10:26:08.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.129+0000 7f7e3c090640 1 -- 172.21.15.125:0/3494159550 shutdown_connections 2023-10-18T10:26:08.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.129+0000 7f7e3c090640 1 --2- 172.21.15.125:0/3494159550 >> v2:172.21.15.139:3300/0 conn(0x7f7e340c9ed0 0x7f7e3406b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:08.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.129+0000 7f7e3c090640 1 --2- 172.21.15.125:0/3494159550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3413d360 0x7f7e3415a9f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.129+0000 7f7e3c090640 1 -- 172.21.15.125:0/3494159550 >> 172.21.15.125:0/3494159550 conn(0x7f7e34069930 msgr2=0x7f7e34069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.130+0000 7f7e3c090640 1 -- 172.21.15.125:0/3494159550 shutdown_connections 2023-10-18T10:26:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.130+0000 7f7e3c090640 1 -- 172.21.15.125:0/3494159550 wait complete. 2023-10-18T10:26:08.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e3c090640 1 Processor -- start 2023-10-18T10:26:08.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e3c090640 1 -- start start 2023-10-18T10:26:08.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e3c090640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e340c9ed0 0x7f7e340b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:08.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e3c090640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3413d360 0x7f7e34136740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:08.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e3c090640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e34140660 0x7f7e34158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:08.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e3c090640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3413bce0 con 0x7f7e340c9ed0 2023-10-18T10:26:08.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e3c090640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e34156b80 con 0x7f7e3413d360 2023-10-18T10:26:08.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e3c090640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3413c820 con 0x7f7e34140660 2023-10-18T10:26:08.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.131+0000 7f7e39e2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e340c9ed0 0x7f7e340b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:08.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e39e2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e340c9ed0 0x7f7e340b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33260/0 (socket says 172.21.15.125:33260) 2023-10-18T10:26:08.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e39e2c640 1 -- 172.21.15.125:0/2720741041 learned_addr learned my addr 172.21.15.125:0/2720741041 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:08.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e3a62d640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e34140660 0x7f7e34158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:08.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e3962b640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3413d360 0x7f7e34136740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:08.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e39e2c640 1 -- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e34140660 msgr2=0x7f7e34158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:08.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e39e2c640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e34140660 0x7f7e34158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:08.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e39e2c640 1 -- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3413d360 msgr2=0x7f7e34136740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:08.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e39e2c640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3413d360 0x7f7e34136740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:08.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e39e2c640 1 -- 172.21.15.125:0/2720741041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30009900 con 0x7f7e340c9ed0 2023-10-18T10:26:08.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e3a62d640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e34140660 0x7f7e34158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:08.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e3962b640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3413d360 0x7f7e34136740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:08.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e39e2c640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e340c9ed0 0x7f7e340b0e70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009bf0 tx=0x7f7e30003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:08.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e2a7fc640 1 -- 172.21.15.125:0/2720741041 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30004450 con 0x7f7e340c9ed0 2023-10-18T10:26:08.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e2a7fc640 1 -- 172.21.15.125:0/2720741041 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7e300046a0 con 0x7f7e340c9ed0 2023-10-18T10:26:08.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3413bef0 con 0x7f7e340c9ed0 2023-10-18T10:26:08.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e2a7fc640 1 -- 172.21.15.125:0/2720741041 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3003d650 con 0x7f7e340c9ed0 2023-10-18T10:26:08.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.132+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3413c4b0 con 0x7f7e340c9ed0 2023-10-18T10:26:08.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.133+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e340690e0 con 0x7f7e340c9ed0 2023-10-18T10:26:08.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.134+0000 7f7e2a7fc640 1 -- 172.21.15.125:0/2720741041 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e3003b040 con 0x7f7e340c9ed0 2023-10-18T10:26:08.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.134+0000 7f7e2a7fc640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e1806ae40 0x7f7e1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:08.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.134+0000 7f7e2a7fc640 1 -- 172.21.15.125:0/2720741041 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f7e300ac780 con 0x7f7e340c9ed0 2023-10-18T10:26:08.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.134+0000 7f7e3962b640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e1806ae40 0x7f7e1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:08.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.135+0000 7f7e3962b640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e1806ae40 0x7f7e1806d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fd0 tx=0x7f7e24008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:08.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.138+0000 7f7e2a7fc640 1 -- 172.21.15.125:0/2720741041 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e30077310 con 0x7f7e340c9ed0 2023-10-18T10:26:08.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:08.309+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7e340b0be0 con 0x7f7e340c9ed0 2023-10-18T10:26:09.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.192+0000 7f7e2a7fc640 1 -- 172.21.15.125:0/2720741041 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7e3007a930 con 0x7f7e340c9ed0 2023-10-18T10:26:09.195 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:26:09.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e1806ae40 msgr2=0x7f7e1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:09.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e1806ae40 0x7f7e1806d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fd0 tx=0x7f7e24008040 comp rx=0 tx=0).stop 2023-10-18T10:26:09.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e340c9ed0 msgr2=0x7f7e340b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:09.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e340c9ed0 0x7f7e340b0e70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009bf0 tx=0x7f7e30003c20 comp rx=0 tx=0).stop 2023-10-18T10:26:09.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 shutdown_connections 2023-10-18T10:26:09.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f7e1806ae40 0x7f7e1806d300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:09.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e3413d360 0x7f7e34136740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:09.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e34140660 0x7f7e34158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:09.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 --2- 172.21.15.125:0/2720741041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e340c9ed0 0x7f7e340b0e70 secure :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009bf0 tx=0x7f7e30003c20 comp rx=0 tx=0).stop 2023-10-18T10:26:09.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 >> 172.21.15.125:0/2720741041 conn(0x7f7e34069930 msgr2=0x7f7e34159840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:09.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 shutdown_connections 2023-10-18T10:26:09.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.196+0000 7f7e3c090640 1 -- 172.21.15.125:0/2720741041 wait complete. 2023-10-18T10:26:09.212 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:26:09.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.322+0000 7fac274c2640 1 -- 172.21.15.125:0/2789704367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200b1920 msgr2=0x7fac2006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:09.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.322+0000 7fac274c2640 1 --2- 172.21.15.125:0/2789704367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200b1920 0x7fac2006fa70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fac1c005fd0 tx=0x7fac1c02bda0 comp rx=0 tx=0).stop 2023-10-18T10:26:09.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.322+0000 7fac274c2640 1 -- 172.21.15.125:0/2789704367 shutdown_connections 2023-10-18T10:26:09.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.322+0000 7fac274c2640 1 --2- 172.21.15.125:0/2789704367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac200c9dc0 0x7fac20154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:09.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.322+0000 7fac274c2640 1 --2- 172.21.15.125:0/2789704367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200b1920 0x7fac2006fa70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:09.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.322+0000 7fac274c2640 1 -- 172.21.15.125:0/2789704367 >> 172.21.15.125:0/2789704367 conn(0x7fac20069930 msgr2=0x7fac20069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:09.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.322+0000 7fac274c2640 1 -- 172.21.15.125:0/2789704367 shutdown_connections 2023-10-18T10:26:09.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.322+0000 7fac274c2640 1 -- 172.21.15.125:0/2789704367 wait complete. 2023-10-18T10:26:09.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.323+0000 7fac274c2640 1 Processor -- start 2023-10-18T10:26:09.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.323+0000 7fac274c2640 1 -- start start 2023-10-18T10:26:09.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac274c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac2006ffb0 0x7fac201f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:09.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac274c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac200b1920 0x7fac20156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:09.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac274c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200c9dc0 0x7fac20141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:09.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac274c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200ac590 con 0x7fac200c9dc0 2023-10-18T10:26:09.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac274c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200ac830 con 0x7fac2006ffb0 2023-10-18T10:26:09.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac274c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac200acb10 con 0x7fac200b1920 2023-10-18T10:26:09.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200c9dc0 0x7fac20141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:09.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200c9dc0 0x7fac20141ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33282/0 (socket says 172.21.15.125:33282) 2023-10-18T10:26:09.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 -- 172.21.15.125:0/2219434953 learned_addr learned my addr 172.21.15.125:0/2219434953 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:09.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac2525e640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac2006ffb0 0x7fac201f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:09.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac24a5d640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac200b1920 0x7fac20156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:09.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 -- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac200b1920 msgr2=0x7fac20156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:09.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac200b1920 0x7fac20156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:09.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 -- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac2006ffb0 msgr2=0x7fac201f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:09.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac2006ffb0 0x7fac201f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:09.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 -- 172.21.15.125:0/2219434953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009560 con 0x7fac200c9dc0 2023-10-18T10:26:09.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac25a5f640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200c9dc0 0x7fac20141ac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fac1800b9e0 tx=0x7fac1800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:09.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac15ffb640 1 -- 172.21.15.125:0/2219434953 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1800cb20 con 0x7fac200c9dc0 2023-10-18T10:26:09.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac15ffb640 1 -- 172.21.15.125:0/2219434953 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac1800cd90 con 0x7fac200c9dc0 2023-10-18T10:26:09.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac200acdf0 con 0x7fac200c9dc0 2023-10-18T10:26:09.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.324+0000 7fac15ffb640 1 -- 172.21.15.125:0/2219434953 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180146a0 con 0x7fac200c9dc0 2023-10-18T10:26:09.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.325+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201f13f0 con 0x7fac200c9dc0 2023-10-18T10:26:09.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.325+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac200ad310 con 0x7fac200c9dc0 2023-10-18T10:26:09.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.326+0000 7fac15ffb640 1 -- 172.21.15.125:0/2219434953 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac18014800 con 0x7fac200c9dc0 2023-10-18T10:26:09.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.326+0000 7fac15ffb640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fac0406ae40 0x7fac0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:09.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.326+0000 7fac15ffb640 1 -- 172.21.15.125:0/2219434953 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fac18089840 con 0x7fac200c9dc0 2023-10-18T10:26:09.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.326+0000 7fac2525e640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fac0406ae40 0x7fac0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:09.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.327+0000 7fac2525e640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fac0406ae40 0x7fac0406d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac1c035000 tx=0x7fac1c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:09.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.330+0000 7fac15ffb640 1 -- 172.21.15.125:0/2219434953 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac200ad310 con 0x7fac200c9dc0 2023-10-18T10:26:09.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:09.500+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fac200ad310 con 0x7fac200c9dc0 2023-10-18T10:26:10.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.459+0000 7fac15ffb640 1 -- 172.21.15.125:0/2219434953 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fac200ad310 con 0x7fac200c9dc0 2023-10-18T10:26:10.461 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T10:26:10.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fac0406ae40 msgr2=0x7fac0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:10.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fac0406ae40 0x7fac0406d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac1c035000 tx=0x7fac1c004730 comp rx=0 tx=0).stop 2023-10-18T10:26:10.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200c9dc0 msgr2=0x7fac20141ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:10.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200c9dc0 0x7fac20141ac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fac1800b9e0 tx=0x7fac1800bea0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 shutdown_connections 2023-10-18T10:26:10.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fac0406ae40 0x7fac0406d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac200c9dc0 0x7fac20141ac0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac200b1920 0x7fac20156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 --2- 172.21.15.125:0/2219434953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac2006ffb0 0x7fac201f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 >> 172.21.15.125:0/2219434953 conn(0x7fac20069930 msgr2=0x7fac20159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:10.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.461+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 shutdown_connections 2023-10-18T10:26:10.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.462+0000 7fac274c2640 1 -- 172.21.15.125:0/2219434953 wait complete. 2023-10-18T10:26:10.486 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-18T10:26:10.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.596+0000 7ff4d17a5640 1 -- 172.21.15.125:0/992630719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc15b3e0 msgr2=0x7ff4cc0cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:10.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.596+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/992630719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc15b3e0 0x7ff4cc0cf2d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009830 tx=0x7ff4b402bd60 comp rx=0 tx=0).stop 2023-10-18T10:26:10.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.597+0000 7ff4d17a5640 1 -- 172.21.15.125:0/992630719 shutdown_connections 2023-10-18T10:26:10.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.597+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/992630719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc15b3e0 0x7ff4cc0cf2d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.597+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/992630719 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4cc15b000 0x7ff4cc0d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.597+0000 7ff4d17a5640 1 -- 172.21.15.125:0/992630719 >> 172.21.15.125:0/992630719 conn(0x7ff4cc0d0cf0 msgr2=0x7ff4cc0d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:10.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.597+0000 7ff4d17a5640 1 -- 172.21.15.125:0/992630719 shutdown_connections 2023-10-18T10:26:10.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.597+0000 7ff4d17a5640 1 -- 172.21.15.125:0/992630719 wait complete. 2023-10-18T10:26:10.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4d17a5640 1 Processor -- start 2023-10-18T10:26:10.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4d17a5640 1 -- start start 2023-10-18T10:26:10.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4d17a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc0afdc0 0x7ff4cc0af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:10.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4d17a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4cc15b000 0x7ff4cc1e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:10.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4d17a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4cc15b3e0 0x7ff4cc1e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:10.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4d17a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0ca350 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4d17a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1e1690 con 0x7ff4cc15b3e0 2023-10-18T10:26:10.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4d17a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1e1970 con 0x7ff4cc15b000 2023-10-18T10:26:10.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4caffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc0afdc0 0x7ff4cc0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:10.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4caffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc0afdc0 0x7ff4cc0af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33296/0 (socket says 172.21.15.125:33296) 2023-10-18T10:26:10.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.598+0000 7ff4caffd640 1 -- 172.21.15.125:0/2112837285 learned_addr learned my addr 172.21.15.125:0/2112837285 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:10.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4caffd640 1 -- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4cc15b000 msgr2=0x7ff4cc1e5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:10.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4caffd640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4cc15b000 0x7ff4cc1e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4caffd640 1 -- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4cc15b3e0 msgr2=0x7ff4cc1e0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:10.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4caffd640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4cc15b3e0 0x7ff4cc1e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:10.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4caffd640 1 -- 172.21.15.125:0/2112837285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4009490 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4caffd640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc0afdc0 0x7ff4cc0af780 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00d950 tx=0x7ff4bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:10.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4c37fe640 1 -- 172.21.15.125:0/2112837285 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc009c30 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4c37fe640 1 -- 172.21.15.125:0/2112837285 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4bc009e80 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc0c9c90 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4c37fe640 1 -- 172.21.15.125:0/2112837285 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0106c0 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.599+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc159090 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.600+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc0cbfd0 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.600+0000 7ff4c37fe640 1 -- 172.21.15.125:0/2112837285 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff4bc010820 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.601+0000 7ff4c37fe640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff4ac06ae70 0x7ff4ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:10.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.601+0000 7ff4c37fe640 1 -- 172.21.15.125:0/2112837285 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7ff4bc089aa0 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.601+0000 7ff4c3fff640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff4ac06ae70 0x7ff4ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:10.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.601+0000 7ff4c3fff640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff4ac06ae70 0x7ff4ac06d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4036000 tx=0x7ff4b40046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:10.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.605+0000 7ff4c37fe640 1 -- 172.21.15.125:0/2112837285 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4bc054b90 con 0x7ff4cc0afdc0 2023-10-18T10:26:10.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:10.770+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff4cc0690e0 con 0x7ff4cc0afdc0 2023-10-18T10:26:11.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.464+0000 7ff4c37fe640 1 -- 172.21.15.125:0/2112837285 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff4bc0170d0 con 0x7ff4cc0afdc0 2023-10-18T10:26:11.466 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data_ec' removed 2023-10-18T10:26:11.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff4ac06ae70 msgr2=0x7ff4ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:11.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff4ac06ae70 0x7ff4ac06d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4036000 tx=0x7ff4b40046c0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc0afdc0 msgr2=0x7ff4cc0af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:11.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc0afdc0 0x7ff4cc0af780 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00d950 tx=0x7ff4bc00de10 comp rx=0 tx=0).stop 2023-10-18T10:26:11.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 shutdown_connections 2023-10-18T10:26:11.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff4ac06ae70 0x7ff4ac06d330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4cc15b3e0 0x7ff4cc1e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4cc15b000 0x7ff4cc1e5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 --2- 172.21.15.125:0/2112837285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4cc0afdc0 0x7ff4cc0af780 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 >> 172.21.15.125:0/2112837285 conn(0x7ff4cc0d0cf0 msgr2=0x7ff4cc0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:11.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 shutdown_connections 2023-10-18T10:26:11.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.466+0000 7ff4d17a5640 1 -- 172.21.15.125:0/2112837285 wait complete. 2023-10-18T10:26:11.481 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:26:11.482 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:26:11.482 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:26:11.482 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:26:11.483 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:26:11.483 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:26:11.483 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:26:11.483 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:26:11.484 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:26:11.484 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:26:11.484 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:26:11.484 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:26:11.485 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:26:11.485 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:26:11.485 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:26:11.486 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:26:11.493 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:26:11.493 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:26:11.494 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:26:11.494 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:26:11.496 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:26:11.496 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:26:11.497 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:26:11.497 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:26:11.536 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:26:11.536 DEBUG:teuthology.parallel:result is None 2023-10-18T10:26:11.538 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:26:11.538 DEBUG:teuthology.parallel:result is None 2023-10-18T10:26:11.539 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:26:11.539 DEBUG:teuthology.parallel:result is None 2023-10-18T10:26:11.540 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:26:11.540 DEBUG:teuthology.parallel:result is None 2023-10-18T10:26:11.541 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:26:11.616 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:26:11.617 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:26:11.620 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:26:11.621 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:26:11.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.652+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/2658454472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f54070080 msgr2=0x7f4f540704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:11.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.652+0000 7f4f5bfb6640 1 --2- 172.21.15.125:0/2658454472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f54070080 0x7f4f540704e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009830 tx=0x7f4f4402bdf0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.652+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/2658454472 shutdown_connections 2023-10-18T10:26:11.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.652+0000 7f4f5bfb6640 1 --2- 172.21.15.125:0/2658454472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f54070080 0x7f4f540704e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.652+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/2658454472 >> 172.21.15.125:0/2658454472 conn(0x7f4f540af730 msgr2=0x7f4f540afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:11.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.652+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/2658454472 shutdown_connections 2023-10-18T10:26:11.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.652+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/2658454472 wait complete. 2023-10-18T10:26:11.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.653+0000 7f4f5bfb6640 1 Processor -- start 2023-10-18T10:26:11.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.653+0000 7f4f5bfb6640 1 -- start start 2023-10-18T10:26:11.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f5bfb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4f54070080 0x7f4f540aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:11.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f5bfb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f540cc200 0x7f4f5413d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:11.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f5bfb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f541416f0 0x7f4f54069720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:11.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f5bfb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5406d6e0 con 0x7f4f54070080 2023-10-18T10:26:11.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f5bfb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5406cd40 con 0x7f4f541416f0 2023-10-18T10:26:11.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f5bfb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5406d020 con 0x7f4f540cc200 2023-10-18T10:26:11.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59d52640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4f54070080 0x7f4f540aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:11.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59d52640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4f54070080 0x7f4f540aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33350/0 (socket says 172.21.15.125:33350) 2023-10-18T10:26:11.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59d52640 1 -- 172.21.15.125:0/741260449 learned_addr learned my addr 172.21.15.125:0/741260449 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:11.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f5a553640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f541416f0 0x7f4f54069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:11.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59551640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f540cc200 0x7f4f5413d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:11.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59d52640 1 -- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f540cc200 msgr2=0x7f4f5413d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:11.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59d52640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f540cc200 0x7f4f5413d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59d52640 1 -- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f541416f0 msgr2=0x7f4f54069720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:11.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59d52640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f541416f0 0x7f4f54069720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:11.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f59d52640 1 -- 172.21.15.125:0/741260449 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44009490 con 0x7f4f54070080 2023-10-18T10:26:11.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.654+0000 7f4f5a553640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f541416f0 0x7f4f54069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:11.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.655+0000 7f4f59d52640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4f54070080 0x7f4f540aa8c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000c7a0 tx=0x7f4f5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:11.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.655+0000 7f4f4a7fc640 1 -- 172.21.15.125:0/741260449 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50003b70 con 0x7f4f54070080 2023-10-18T10:26:11.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.655+0000 7f4f4a7fc640 1 -- 172.21.15.125:0/741260449 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4f50003cf0 con 0x7f4f54070080 2023-10-18T10:26:11.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.655+0000 7f4f4a7fc640 1 -- 172.21.15.125:0/741260449 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50003ea0 con 0x7f4f54070080 2023-10-18T10:26:11.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.655+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f540b1cd0 con 0x7f4f54070080 2023-10-18T10:26:11.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.655+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5406db00 con 0x7f4f54070080 2023-10-18T10:26:11.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.656+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f540b1ad0 con 0x7f4f54070080 2023-10-18T10:26:11.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.656+0000 7f4f4a7fc640 1 -- 172.21.15.125:0/741260449 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4f5000aa50 con 0x7f4f54070080 2023-10-18T10:26:11.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.657+0000 7f4f4a7fc640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4f3806ae40 0x7f4f3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:11.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.657+0000 7f4f4a7fc640 1 -- 172.21.15.125:0/741260449 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4f50089430 con 0x7f4f54070080 2023-10-18T10:26:11.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.657+0000 7f4f59551640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4f3806ae40 0x7f4f3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:11.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.657+0000 7f4f59551640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4f3806ae40 0x7f4f3806d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4f44036000 tx=0x7f4f44004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:11.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.661+0000 7f4f4a7fc640 1 -- 172.21.15.125:0/741260449 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f500546b0 con 0x7f4f54070080 2023-10-18T10:26:11.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:11.841+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4f540675f0 con 0x7f4f54070080 2023-10-18T10:26:12.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.474+0000 7f4f4a7fc640 1 -- 172.21.15.125:0/741260449 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4f50057cd0 con 0x7f4f54070080 2023-10-18T10:26:12.476 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T10:26:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4f3806ae40 msgr2=0x7f4f3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4f3806ae40 0x7f4f3806d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4f44036000 tx=0x7f4f44004730 comp rx=0 tx=0).stop 2023-10-18T10:26:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4f54070080 msgr2=0x7f4f540aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4f54070080 0x7f4f540aa8c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000c7a0 tx=0x7f4f5000cc60 comp rx=0 tx=0).stop 2023-10-18T10:26:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 shutdown_connections 2023-10-18T10:26:12.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f4f3806ae40 0x7f4f3806d300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f541416f0 0x7f4f54069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f540cc200 0x7f4f5413d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 --2- 172.21.15.125:0/741260449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4f54070080 0x7f4f540aa8c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 >> 172.21.15.125:0/741260449 conn(0x7f4f540af730 msgr2=0x7f4f54156740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 shutdown_connections 2023-10-18T10:26:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.478+0000 7f4f5bfb6640 1 -- 172.21.15.125:0/741260449 wait complete. 2023-10-18T10:26:12.497 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:26:12.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.607+0000 7f17be5b0640 1 -- 172.21.15.125:0/359967856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80cc440 msgr2=0x7f17b80b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:12.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.607+0000 7f17be5b0640 1 --2- 172.21.15.125:0/359967856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80cc440 0x7f17b80b0320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009830 tx=0x7f17ac02bde0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.607+0000 7f17be5b0640 1 -- 172.21.15.125:0/359967856 shutdown_connections 2023-10-18T10:26:12.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.607+0000 7f17be5b0640 1 --2- 172.21.15.125:0/359967856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80cc440 0x7f17b80b0320 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.607+0000 7f17be5b0640 1 --2- 172.21.15.125:0/359967856 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b8157aa0 0x7f17b806fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.607+0000 7f17be5b0640 1 -- 172.21.15.125:0/359967856 >> 172.21.15.125:0/359967856 conn(0x7f17b80cc970 msgr2=0x7f17b80ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:12.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.607+0000 7f17be5b0640 1 -- 172.21.15.125:0/359967856 shutdown_connections 2023-10-18T10:26:12.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.607+0000 7f17be5b0640 1 -- 172.21.15.125:0/359967856 wait complete. 2023-10-18T10:26:12.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.608+0000 7f17be5b0640 1 Processor -- start 2023-10-18T10:26:12.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.608+0000 7f17be5b0640 1 -- start start 2023-10-18T10:26:12.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17be5b0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80b0860 0x7f17b8149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:12.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17be5b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b80cc440 0x7f17b80aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:12.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17be5b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b8157aa0 0x7f17b80af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:12.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17be5b0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8151450 con 0x7f17b80b0860 2023-10-18T10:26:12.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17be5b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b814f410 con 0x7f17b80cc440 2023-10-18T10:26:12.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17be5b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8150e00 con 0x7f17b8157aa0 2023-10-18T10:26:12.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80b0860 0x7f17b8149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:12.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80b0860 0x7f17b8149390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33366/0 (socket says 172.21.15.125:33366) 2023-10-18T10:26:12.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 -- 172.21.15.125:0/285115117 learned_addr learned my addr 172.21.15.125:0/285115117 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:12.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b77fe640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b80cc440 0x7f17b80aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:12.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17bcb4d640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b8157aa0 0x7f17b80af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:12.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 -- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b8157aa0 msgr2=0x7f17b80af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:12.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b8157aa0 0x7f17b80af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 -- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b80cc440 msgr2=0x7f17b80aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:12.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b80cc440 0x7f17b80aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 -- 172.21.15.125:0/285115117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac009490 con 0x7f17b80b0860 2023-10-18T10:26:12.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17bcb4d640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b8157aa0 0x7f17b80af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:12.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b7fff640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80b0860 0x7f17b8149390 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f17a800d6d0 tx=0x7f17a800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:12.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b4ff9640 1 -- 172.21.15.125:0/285115117 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8005220 con 0x7f17b80b0860 2023-10-18T10:26:12.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b4ff9640 1 -- 172.21.15.125:0/285115117 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f17a8005490 con 0x7f17b80b0860 2023-10-18T10:26:12.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b8070100 con 0x7f17b80b0860 2023-10-18T10:26:12.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17b4ff9640 1 -- 172.21.15.125:0/285115117 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8009a80 con 0x7f17b80b0860 2023-10-18T10:26:12.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.609+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b81498d0 con 0x7f17b80b0860 2023-10-18T10:26:12.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.610+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b804dae0 con 0x7f17b80b0860 2023-10-18T10:26:12.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.611+0000 7f17b4ff9640 1 -- 172.21.15.125:0/285115117 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f17a80047a0 con 0x7f17b80b0860 2023-10-18T10:26:12.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.611+0000 7f17b4ff9640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f17a406ae70 0x7f17a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:12.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.611+0000 7f17b4ff9640 1 -- 172.21.15.125:0/285115117 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f17a8089760 con 0x7f17b80b0860 2023-10-18T10:26:12.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.612+0000 7f17b77fe640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f17a406ae70 0x7f17a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:12.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.612+0000 7f17b77fe640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f17a406ae70 0x7f17a406d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17ac036000 tx=0x7f17ac004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:12.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.616+0000 7f17b4ff9640 1 -- 172.21.15.125:0/285115117 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17a8054a20 con 0x7f17b80b0860 2023-10-18T10:26:12.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.844+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f17b80afbf0 con 0x7f17b80b0860 2023-10-18T10:26:12.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.845+0000 7f17b4ff9640 1 -- 172.21.15.125:0/285115117 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f17a8017090 con 0x7f17b80b0860 2023-10-18T10:26:12.847 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:12.847 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:26:12.847 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T10:26:12.848 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.848 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T10:26:12.848 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T10:26:12.848 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.848 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:26:12.849 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:26:12.849 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:26:12.849 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.849 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.849 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.850 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.850 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T10:26:12.850 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T10:26:12.850 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.850 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:26:12.851 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:26:12.851 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:26:12.851 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.851 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.851 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.852 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.852 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T10:26:12.852 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T10:26:12.852 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.852 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:26:12.852 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:26:12.853 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:26:12.853 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.853 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.853 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.853 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.854 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T10:26:12.854 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T10:26:12.854 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.854 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:26:12.854 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:26:12.855 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:26:12.855 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.855 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.855 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.855 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.856 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T10:26:12.856 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T10:26:12.856 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.856 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:26:12.856 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:26:12.856 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.857 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.857 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.857 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.857 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T10:26:12.857 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T10:26:12.858 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.858 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:26:12.858 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:26:12.858 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.858 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.859 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.859 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.859 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T10:26:12.859 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T10:26:12.859 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.860 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:26:12.860 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:26:12.860 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.860 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.860 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.861 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.861 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T10:26:12.861 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T10:26:12.861 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.861 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:26:12.862 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:26:12.862 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.862 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.862 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.862 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.862 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T10:26:12.863 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T10:26:12.863 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.863 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:26:12.863 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:26:12.863 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.863 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.864 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.864 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.864 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T10:26:12.864 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T10:26:12.864 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.865 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:26:12.865 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:26:12.865 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.865 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.865 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.865 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.866 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T10:26:12.866 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T10:26:12.866 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.866 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:26:12.866 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:26:12.867 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.867 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.867 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.867 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.867 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T10:26:12.867 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T10:26:12.868 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.868 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:26:12.868 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:26:12.868 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.868 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.869 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.869 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.869 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T10:26:12.869 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T10:26:12.869 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:26:12.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:26:12.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:26:12.870 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:26:12.870 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.870 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.871 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.871 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T10:26:12.871 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T10:26:12.871 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.871 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:26:12.872 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:26:12.872 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:26:12.872 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:26:12.872 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.872 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.872 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.873 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T10:26:12.873 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T10:26:12.873 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.873 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:26:12.873 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:26:12.873 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:26:12.874 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:26:12.874 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.874 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.874 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.874 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T10:26:12.875 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T10:26:12.875 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.875 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:26:12.875 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:26:12.875 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:26:12.875 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:26:12.876 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.876 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.876 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.876 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T10:26:12.876 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T10:26:12.877 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.877 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:26:12.877 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:26:12.877 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:26:12.877 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.878 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.878 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.878 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.878 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.bootstrap-mds", 2023-10-18T10:26:12.878 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBmsi9lntQ3FxAAQgmIJNvqoaSsQxejeOD6lg==", 2023-10-18T10:26:12.879 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.879 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile bootstrap-mds" 2023-10-18T10:26:12.879 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.879 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.879 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.880 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.bootstrap-mgr", 2023-10-18T10:26:12.880 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBmsi9lxd03FxAAykmW9ie69hynwTm59fPt3Q==", 2023-10-18T10:26:12.880 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.880 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-18T10:26:12.880 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.880 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.881 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.881 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.bootstrap-osd", 2023-10-18T10:26:12.881 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBmsi9l0OQ3FxAAc4NMwZQywKkNtPR9AoJCzQ==", 2023-10-18T10:26:12.881 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.881 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile bootstrap-osd" 2023-10-18T10:26:12.882 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.882 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.882 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.882 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.bootstrap-rbd", 2023-10-18T10:26:12.882 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBmsi9lrus3FxAASIl6Q1qNyDZJXxL7ll5Aeg==", 2023-10-18T10:26:12.883 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.883 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-18T10:26:12.883 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.883 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.883 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.883 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-18T10:26:12.884 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBmsi9lovI3FxAAU9mYvTkW7H191PA7aT1zOw==", 2023-10-18T10:26:12.884 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.884 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-18T10:26:12.884 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.884 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.885 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.885 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.bootstrap-rgw", 2023-10-18T10:26:12.885 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBmsi9lCvo3FxAA2mivxCKj4VvPAYaMFw+NAQ==", 2023-10-18T10:26:12.885 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-18T10:26:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T10:26:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T10:26:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:26:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:26:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.888 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:12.888 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:12.888 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T10:26:12.888 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T10:26:12.888 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:26:12.889 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:26:12.889 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:26:12.889 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:26:12.889 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.889 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:12.889 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T10:26:12.890 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:26:12.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f17a406ae70 msgr2=0x7f17a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:12.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f17a406ae70 0x7f17a406d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17ac036000 tx=0x7f17ac004730 comp rx=0 tx=0).stop 2023-10-18T10:26:12.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80b0860 msgr2=0x7f17b8149390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:12.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80b0860 0x7f17b8149390 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f17a800d6d0 tx=0x7f17a800db90 comp rx=0 tx=0).stop 2023-10-18T10:26:12.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 shutdown_connections 2023-10-18T10:26:12.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f17a406ae70 0x7f17a406d330 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b80cc440 0x7f17b80aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b8157aa0 0x7f17b80af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.846+0000 7f17be5b0640 1 --2- 172.21.15.125:0/285115117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17b80b0860 0x7f17b8149390 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.847+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 >> 172.21.15.125:0/285115117 conn(0x7f17b80cc970 msgr2=0x7f17b8158920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:12.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.847+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 shutdown_connections 2023-10-18T10:26:12.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.847+0000 7f17be5b0640 1 -- 172.21.15.125:0/285115117 wait complete. 2023-10-18T10:26:12.894 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-18T10:26:12.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.995+0000 7f4771c75640 1 -- 172.21.15.125:0/1666365277 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0afa20 msgr2=0x7f476c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:12.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.995+0000 7f4771c75640 1 --2- 172.21.15.125:0/1666365277 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0afa20 0x7f476c0705a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cc7e0 tx=0x7f475402d0a0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.995+0000 7f4771c75640 1 -- 172.21.15.125:0/1666365277 shutdown_connections 2023-10-18T10:26:12.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.995+0000 7f4771c75640 1 --2- 172.21.15.125:0/1666365277 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0afa20 0x7f476c0705a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.995+0000 7f4771c75640 1 --2- 172.21.15.125:0/1666365277 >> v2:172.21.15.139:3300/0 conn(0x7f476c0cbd20 0x7f476c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:12.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.995+0000 7f4771c75640 1 -- 172.21.15.125:0/1666365277 >> 172.21.15.125:0/1666365277 conn(0x7f476c069930 msgr2=0x7f476c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:12.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.995+0000 7f4771c75640 1 -- 172.21.15.125:0/1666365277 shutdown_connections 2023-10-18T10:26:12.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.995+0000 7f4771c75640 1 -- 172.21.15.125:0/1666365277 wait complete. 2023-10-18T10:26:13.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.996+0000 7f4771c75640 1 Processor -- start 2023-10-18T10:26:13.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.996+0000 7f4771c75640 1 -- start start 2023-10-18T10:26:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f4771c75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f476c0afa20 0x7f476c1451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f4771c75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f476c0b0530 0x7f476c0a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f4771c75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0cbd20 0x7f476c0a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f4771c75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c14ba80 con 0x7f476c0cbd20 2023-10-18T10:26:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f4771c75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1f26d0 con 0x7f476c0afa20 2023-10-18T10:26:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f4771c75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c1f1ca0 con 0x7f476c0b0530 2023-10-18T10:26:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0cbd20 0x7f476c0a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0cbd20 0x7f476c0a69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33374/0 (socket says 172.21.15.125:33374) 2023-10-18T10:26:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 -- 172.21.15.125:0/3244826648 learned_addr learned my addr 172.21.15.125:0/3244826648 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f4762ffd640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f476c0b0530 0x7f476c0a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476b7fe640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f476c0afa20 0x7f476c1451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 -- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f476c0b0530 msgr2=0x7f476c0a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f476c0b0530 0x7f476c0a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 -- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f476c0afa20 msgr2=0x7f476c1451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f476c0afa20 0x7f476c1451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 -- 172.21.15.125:0/3244826648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475400a5a0 con 0x7f476c0cbd20 2023-10-18T10:26:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.997+0000 7f476bfff640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0cbd20 0x7f476c0a69f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4764001560 tx=0x7f476400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.998+0000 7f4768ff9640 1 -- 172.21.15.125:0/3244826648 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764003da0 con 0x7f476c0cbd20 2023-10-18T10:26:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.998+0000 7f4768ff9640 1 -- 172.21.15.125:0/3244826648 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4764005070 con 0x7f476c0cbd20 2023-10-18T10:26:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.998+0000 7f4768ff9640 1 -- 172.21.15.125:0/3244826648 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764005460 con 0x7f476c0cbd20 2023-10-18T10:26:13.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.998+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c0ad950 con 0x7f476c0cbd20 2023-10-18T10:26:13.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.998+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c1f2210 con 0x7f476c0cbd20 2023-10-18T10:26:13.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.999+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c1f2840 con 0x7f476c0cbd20 2023-10-18T10:26:13.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.999+0000 7f4768ff9640 1 -- 172.21.15.125:0/3244826648 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47640055c0 con 0x7f476c0cbd20 2023-10-18T10:26:13.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:12.999+0000 7f4768ff9640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f475006ae40 0x7f475006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.000+0000 7f4768ff9640 1 -- 172.21.15.125:0/3244826648 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4764088ea0 con 0x7f476c0cbd20 2023-10-18T10:26:13.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.000+0000 7f476b7fe640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f475006ae40 0x7f475006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.000+0000 7f476b7fe640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f475006ae40 0x7f475006d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f475c0058c0 tx=0x7f475c005d80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:13.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.004+0000 7f4768ff9640 1 -- 172.21.15.125:0/3244826648 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47640541f0 con 0x7f476c0cbd20 2023-10-18T10:26:13.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.235+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f476c0690e0 con 0x7f476c0cbd20 2023-10-18T10:26:13.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.236+0000 7f4768ff9640 1 -- 172.21.15.125:0/3244826648 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4764017020 con 0x7f476c0cbd20 2023-10-18T10:26:13.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f475006ae40 msgr2=0x7f475006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f475006ae40 0x7f475006d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f475c0058c0 tx=0x7f475c005d80 comp rx=0 tx=0).stop 2023-10-18T10:26:13.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0cbd20 msgr2=0x7f476c0a69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0cbd20 0x7f476c0a69f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4764001560 tx=0x7f476400cc70 comp rx=0 tx=0).stop 2023-10-18T10:26:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 shutdown_connections 2023-10-18T10:26:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f475006ae40 0x7f475006d300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f476c0cbd20 0x7f476c0a69f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f476c0b0530 0x7f476c0a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 --2- 172.21.15.125:0/3244826648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f476c0afa20 0x7f476c1451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.238+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 >> 172.21.15.125:0/3244826648 conn(0x7f476c069930 msgr2=0x7f476c1502a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:13.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.239+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 shutdown_connections 2023-10-18T10:26:13.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.239+0000 7f4771c75640 1 -- 172.21.15.125:0/3244826648 wait complete. 2023-10-18T10:26:13.251 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-18T10:26:13.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.352+0000 7f6e52f55640 1 -- 172.21.15.125:0/1988271041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c13e520 msgr2=0x7f6e4c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.352+0000 7f6e52f55640 1 --2- 172.21.15.125:0/1988271041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c13e520 0x7f6e4c06bb60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005fd0 tx=0x7f6e4002bc60 comp rx=0 tx=0).stop 2023-10-18T10:26:13.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.352+0000 7f6e52f55640 1 -- 172.21.15.125:0/1988271041 shutdown_connections 2023-10-18T10:26:13.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.352+0000 7f6e52f55640 1 --2- 172.21.15.125:0/1988271041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e4c141690 0x7f6e4c0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.352+0000 7f6e52f55640 1 --2- 172.21.15.125:0/1988271041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c13e520 0x7f6e4c06bb60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.352+0000 7f6e52f55640 1 -- 172.21.15.125:0/1988271041 >> 172.21.15.125:0/1988271041 conn(0x7f6e4c069930 msgr2=0x7f6e4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:13.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.352+0000 7f6e52f55640 1 -- 172.21.15.125:0/1988271041 shutdown_connections 2023-10-18T10:26:13.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.353+0000 7f6e52f55640 1 -- 172.21.15.125:0/1988271041 wait complete. 2023-10-18T10:26:13.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.354+0000 7f6e52f55640 1 Processor -- start 2023-10-18T10:26:13.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.354+0000 7f6e52f55640 1 -- start start 2023-10-18T10:26:13.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.354+0000 7f6e52f55640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c06c0a0 0x7f6e4c1305e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.354+0000 7f6e52f55640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e4c13e520 0x7f6e4c13b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.354+0000 7f6e52f55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e4c141690 0x7f6e4c1e2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.354+0000 7f6e52f55640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c13c3b0 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.354+0000 7f6e52f55640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c13c690 con 0x7f6e4c141690 2023-10-18T10:26:13.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.354+0000 7f6e52f55640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c13c970 con 0x7f6e4c13e520 2023-10-18T10:26:13.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c06c0a0 0x7f6e4c1305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c06c0a0 0x7f6e4c1305e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33392/0 (socket says 172.21.15.125:33392) 2023-10-18T10:26:13.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 -- 172.21.15.125:0/502941920 learned_addr learned my addr 172.21.15.125:0/502941920 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:13.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e4bfff640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e4c13e520 0x7f6e4c13b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e514f2640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e4c141690 0x7f6e4c1e2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 -- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e4c13e520 msgr2=0x7f6e4c13b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e4c13e520 0x7f6e4c13b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 -- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e4c141690 msgr2=0x7f6e4c1e2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e4c141690 0x7f6e4c1e2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 -- 172.21.15.125:0/502941920 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40009560 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e50cf1640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c06c0a0 0x7f6e4c1305e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009850 tx=0x7f6e4002dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:13.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e497fa640 1 -- 172.21.15.125:0/502941920 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40005690 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e497fa640 1 -- 172.21.15.125:0/502941920 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e40003800 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e497fa640 1 -- 172.21.15.125:0/502941920 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40003b90 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c13cc50 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.355+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c13d450 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.356+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c13cab0 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.357+0000 7f6e497fa640 1 -- 172.21.15.125:0/502941920 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e40003cf0 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.357+0000 7f6e497fa640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e3006ae40 0x7f6e3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.357+0000 7f6e497fa640 1 -- 172.21.15.125:0/502941920 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6e400abfd0 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.357+0000 7f6e4bfff640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e3006ae40 0x7f6e3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.358+0000 7f6e4bfff640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e3006ae40 0x7f6e3006d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0adf40 tx=0x7f6e38009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:13.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.361+0000 7f6e497fa640 1 -- 172.21.15.125:0/502941920 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e4c13cab0 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.595+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f6e4c13cab0 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.597+0000 7f6e497fa640 1 -- 172.21.15.125:0/502941920 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6e4c13cab0 con 0x7f6e4c06c0a0 2023-10-18T10:26:13.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e3006ae40 msgr2=0x7f6e3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e3006ae40 0x7f6e3006d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0adf40 tx=0x7f6e38009060 comp rx=0 tx=0).stop 2023-10-18T10:26:13.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c06c0a0 msgr2=0x7f6e4c1305e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c06c0a0 0x7f6e4c1305e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009850 tx=0x7f6e4002dc30 comp rx=0 tx=0).stop 2023-10-18T10:26:13.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 shutdown_connections 2023-10-18T10:26:13.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e3006ae40 0x7f6e3006d300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e4c141690 0x7f6e4c1e2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e4c13e520 0x7f6e4c13b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 --2- 172.21.15.125:0/502941920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e4c06c0a0 0x7f6e4c1305e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 >> 172.21.15.125:0/502941920 conn(0x7f6e4c069930 msgr2=0x7f6e4c0aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:13.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 shutdown_connections 2023-10-18T10:26:13.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.599+0000 7f6e52f55640 1 -- 172.21.15.125:0/502941920 wait complete. 2023-10-18T10:26:13.611 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-18T10:26:13.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.713+0000 7f0514044640 1 -- 172.21.15.125:0/117873836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0d1910 msgr2=0x7f050c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.713+0000 7f0514044640 1 --2- 172.21.15.125:0/117873836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0d1910 0x7f050c0b0650 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0504005fd0 tx=0x7f050402c050 comp rx=0 tx=0).stop 2023-10-18T10:26:13.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.713+0000 7f0514044640 1 -- 172.21.15.125:0/117873836 shutdown_connections 2023-10-18T10:26:13.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.713+0000 7f0514044640 1 --2- 172.21.15.125:0/117873836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0d1910 0x7f050c0b0650 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.713+0000 7f0514044640 1 -- 172.21.15.125:0/117873836 >> 172.21.15.125:0/117873836 conn(0x7f050c0d0d60 msgr2=0x7f050c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:13.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.713+0000 7f0514044640 1 -- 172.21.15.125:0/117873836 shutdown_connections 2023-10-18T10:26:13.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.713+0000 7f0514044640 1 -- 172.21.15.125:0/117873836 wait complete. 2023-10-18T10:26:13.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.714+0000 7f0514044640 1 Processor -- start 2023-10-18T10:26:13.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.714+0000 7f0514044640 1 -- start start 2023-10-18T10:26:13.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.714+0000 7f0514044640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0cf340 0x7f050c14f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0514044640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f050c0d1910 0x7f050c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0514044640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f050c15b2a0 0x7f050c0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0514044640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c15c8a0 con 0x7f050c0cf340 2023-10-18T10:26:13.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0514044640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c15bc20 con 0x7f050c0d1910 2023-10-18T10:26:13.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0514044640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c15bf00 con 0x7f050c15b2a0 2023-10-18T10:26:13.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0cf340 0x7f050c14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0cf340 0x7f050c14f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33408/0 (socket says 172.21.15.125:33408) 2023-10-18T10:26:13.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 -- 172.21.15.125:0/1095546892 learned_addr learned my addr 172.21.15.125:0/1095546892 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:13.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f05115df640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f050c0d1910 0x7f050c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f05125e1640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f050c15b2a0 0x7f050c0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 -- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f050c15b2a0 msgr2=0x7f050c0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f050c15b2a0 0x7f050c0ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 -- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f050c0d1910 msgr2=0x7f050c0c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f050c0d1910 0x7f050c0c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 -- 172.21.15.125:0/1095546892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0504009840 con 0x7f050c0cf340 2023-10-18T10:26:13.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0511de0640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0cf340 0x7f050c14f130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f050800c4a0 tx=0x7f050800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:13.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f05027fc640 1 -- 172.21.15.125:0/1095546892 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508012830 con 0x7f050c0cf340 2023-10-18T10:26:13.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f05027fc640 1 -- 172.21.15.125:0/1095546892 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0508012a80 con 0x7f050c0cf340 2023-10-18T10:26:13.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c15c1e0 con 0x7f050c0cf340 2023-10-18T10:26:13.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f05027fc640 1 -- 172.21.15.125:0/1095546892 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508012e90 con 0x7f050c0cf340 2023-10-18T10:26:13.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.715+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c14f670 con 0x7f050c0cf340 2023-10-18T10:26:13.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.716+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c0690e0 con 0x7f050c0cf340 2023-10-18T10:26:13.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.716+0000 7f05027fc640 1 -- 172.21.15.125:0/1095546892 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f050801c460 con 0x7f050c0cf340 2023-10-18T10:26:13.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.717+0000 7f05027fc640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f04f006ae40 0x7f04f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:13.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.717+0000 7f05027fc640 1 -- 172.21.15.125:0/1095546892 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0508089450 con 0x7f050c0cf340 2023-10-18T10:26:13.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.717+0000 7f05115df640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f04f006ae40 0x7f04f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:13.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.718+0000 7f05115df640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f04f006ae40 0x7f04f006d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f04fc006fd0 tx=0x7f04fc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:13.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.721+0000 7f05027fc640 1 -- 172.21.15.125:0/1095546892 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05080547a0 con 0x7f050c0cf340 2023-10-18T10:26:13.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.949+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f050c0d0b40 con 0x7f050c0cf340 2023-10-18T10:26:13.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.951+0000 7f05027fc640 1 -- 172.21.15.125:0/1095546892 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v13) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0508057dc0 con 0x7f050c0cf340 2023-10-18T10:26:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.953+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f04f006ae40 msgr2=0x7f04f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.953+0000 7f0514044640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f04f006ae40 0x7f04f006d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f04fc006fd0 tx=0x7f04fc008040 comp rx=0 tx=0).stop 2023-10-18T10:26:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.953+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0cf340 msgr2=0x7f050c14f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.953+0000 7f0514044640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0cf340 0x7f050c14f130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f050800c4a0 tx=0x7f050800c960 comp rx=0 tx=0).stop 2023-10-18T10:26:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.954+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 shutdown_connections 2023-10-18T10:26:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.954+0000 7f0514044640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f04f006ae40 0x7f04f006d300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.954+0000 7f0514044640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f050c0d1910 0x7f050c0c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.954+0000 7f0514044640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f050c15b2a0 0x7f050c0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.954+0000 7f0514044640 1 --2- 172.21.15.125:0/1095546892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f050c0cf340 0x7f050c14f130 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:13.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.954+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 >> 172.21.15.125:0/1095546892 conn(0x7f050c0d0d60 msgr2=0x7f050c0b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:13.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.954+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 shutdown_connections 2023-10-18T10:26:13.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:13.954+0000 7f0514044640 1 -- 172.21.15.125:0/1095546892 wait complete. 2023-10-18T10:26:13.966 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-18T10:26:14.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.066+0000 7f8267ef2640 1 -- 172.21.15.125:0/2511810592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8260070920 msgr2=0x7f82600affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.066+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2511810592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8260070920 0x7f82600affe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f82600cbdc0 tx=0x7f825802c260 comp rx=0 tx=0).stop 2023-10-18T10:26:14.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.066+0000 7f8267ef2640 1 -- 172.21.15.125:0/2511810592 shutdown_connections 2023-10-18T10:26:14.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.066+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2511810592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8260070920 0x7f82600affe0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.066+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2511810592 >> v2:172.21.15.139:3300/0 conn(0x7f8260141690 0x7f82600703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.066+0000 7f8267ef2640 1 -- 172.21.15.125:0/2511810592 >> 172.21.15.125:0/2511810592 conn(0x7f8260069930 msgr2=0x7f8260069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:14.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.066+0000 7f8267ef2640 1 -- 172.21.15.125:0/2511810592 shutdown_connections 2023-10-18T10:26:14.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.066+0000 7f8267ef2640 1 -- 172.21.15.125:0/2511810592 wait complete. 2023-10-18T10:26:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.067+0000 7f8267ef2640 1 Processor -- start 2023-10-18T10:26:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f8267ef2640 1 -- start start 2023-10-18T10:26:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f8267ef2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8260070920 0x7f82601567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f8267ef2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f826013e520 0x7f82600ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f8267ef2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8260141690 0x7f82600cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f8267ef2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8260133100 con 0x7f826013e520 2023-10-18T10:26:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f8267ef2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8260156d10 con 0x7f8260141690 2023-10-18T10:26:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f8267ef2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8260132480 con 0x7f8260070920 2023-10-18T10:26:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f826548d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f826013e520 0x7f82600ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f826548d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f826013e520 0x7f82600ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33414/0 (socket says 172.21.15.125:33414) 2023-10-18T10:26:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.068+0000 7f826548d640 1 -- 172.21.15.125:0/2372977867 learned_addr learned my addr 172.21.15.125:0/2372977867 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f826648f640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8260141690 0x7f82600cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f8265c8e640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8260070920 0x7f82601567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f826548d640 1 -- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8260070920 msgr2=0x7f82601567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f826548d640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8260070920 0x7f82601567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f826548d640 1 -- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8260141690 msgr2=0x7f82600cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f826548d640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8260141690 0x7f82600cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f826548d640 1 -- 172.21.15.125:0/2372977867 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009900 con 0x7f826013e520 2023-10-18T10:26:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f826548d640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f826013e520 0x7f82600ca870 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f825000d740 tx=0x7f825000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f82567fc640 1 -- 172.21.15.125:0/2372977867 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250009c30 con 0x7f826013e520 2023-10-18T10:26:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f82567fc640 1 -- 172.21.15.125:0/2372977867 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8250009e80 con 0x7f826013e520 2023-10-18T10:26:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8260132760 con 0x7f826013e520 2023-10-18T10:26:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f82567fc640 1 -- 172.21.15.125:0/2372977867 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500106c0 con 0x7f826013e520 2023-10-18T10:26:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.069+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8260132d20 con 0x7f826013e520 2023-10-18T10:26:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.070+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8260133240 con 0x7f826013e520 2023-10-18T10:26:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.071+0000 7f82567fc640 1 -- 172.21.15.125:0/2372977867 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8250010820 con 0x7f826013e520 2023-10-18T10:26:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.071+0000 7f82567fc640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.071+0000 7f82567fc640 1 -- 172.21.15.125:0/2372977867 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8250089960 con 0x7f826013e520 2023-10-18T10:26:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.071+0000 7f8265c8e640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.072+0000 7f8265c8e640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f824406ae70 0x7f824406d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f825c001640 tx=0x7f825c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:14.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.075+0000 7f82567fc640 1 -- 172.21.15.125:0/2372977867 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8260133240 con 0x7f826013e520 2023-10-18T10:26:14.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.305+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8260133240 con 0x7f826013e520 2023-10-18T10:26:14.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.307+0000 7f82567fc640 1 -- 172.21.15.125:0/2372977867 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8260133240 con 0x7f826013e520 2023-10-18T10:26:14.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f824406ae70 msgr2=0x7f824406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f824406ae70 0x7f824406d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f825c001640 tx=0x7f825c009040 comp rx=0 tx=0).stop 2023-10-18T10:26:14.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f826013e520 msgr2=0x7f82600ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f826013e520 0x7f82600ca870 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f825000d740 tx=0x7f825000dc00 comp rx=0 tx=0).stop 2023-10-18T10:26:14.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 shutdown_connections 2023-10-18T10:26:14.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8260141690 0x7f82600cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f826013e520 0x7f82600ca870 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 --2- 172.21.15.125:0/2372977867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8260070920 0x7f82601567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 >> 172.21.15.125:0/2372977867 conn(0x7f8260069930 msgr2=0x7f82601585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:14.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 shutdown_connections 2023-10-18T10:26:14.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.309+0000 7f8267ef2640 1 -- 172.21.15.125:0/2372977867 wait complete. 2023-10-18T10:26:14.321 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-18T10:26:14.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.422+0000 7f29010a9640 1 -- 172.21.15.125:0/4046252397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc0c9d00 msgr2=0x7f28fc06b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.422+0000 7f29010a9640 1 --2- 172.21.15.125:0/4046252397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc0c9d00 0x7f28fc06b910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f28fc140c40 tx=0x7f28f402c110 comp rx=0 tx=0).stop 2023-10-18T10:26:14.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.423+0000 7f29010a9640 1 -- 172.21.15.125:0/4046252397 shutdown_connections 2023-10-18T10:26:14.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.423+0000 7f29010a9640 1 --2- 172.21.15.125:0/4046252397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc0c9d00 0x7f28fc06b910 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.423+0000 7f29010a9640 1 --2- 172.21.15.125:0/4046252397 >> v2:172.21.15.139:3300/0 conn(0x7f28fc140660 0x7f28fc0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.423+0000 7f29010a9640 1 -- 172.21.15.125:0/4046252397 >> 172.21.15.125:0/4046252397 conn(0x7f28fc069930 msgr2=0x7f28fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:14.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.423+0000 7f29010a9640 1 -- 172.21.15.125:0/4046252397 shutdown_connections 2023-10-18T10:26:14.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.423+0000 7f29010a9640 1 -- 172.21.15.125:0/4046252397 wait complete. 2023-10-18T10:26:14.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.424+0000 7f29010a9640 1 Processor -- start 2023-10-18T10:26:14.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.424+0000 7f29010a9640 1 -- start start 2023-10-18T10:26:14.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.424+0000 7f29010a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28fc0c9d00 0x7f28fc1ee890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f29010a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc13d360 0x7f28fc1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f29010a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28fc140660 0x7f28fc1f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f29010a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc137720 con 0x7f28fc13d360 2023-10-18T10:26:14.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f29010a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc137a00 con 0x7f28fc0c9d00 2023-10-18T10:26:14.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f29010a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc131ba0 con 0x7f28fc140660 2023-10-18T10:26:14.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc13d360 0x7f28fc1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc13d360 0x7f28fc1f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33426/0 (socket says 172.21.15.125:33426) 2023-10-18T10:26:14.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 -- 172.21.15.125:0/1827067356 learned_addr learned my addr 172.21.15.125:0/1827067356 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:14.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28fad9d640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28fc0c9d00 0x7f28fc1ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28fb59e640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28fc140660 0x7f28fc1f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 -- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28fc140660 msgr2=0x7f28fc1f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28fc140660 0x7f28fc1f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 -- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28fc0c9d00 msgr2=0x7f28fc1ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28fc0c9d00 0x7f28fc1ee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 -- 172.21.15.125:0/1827067356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4009840 con 0x7f28fc13d360 2023-10-18T10:26:14.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28fb59e640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28fc140660 0x7f28fc1f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:14.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f3fff640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc13d360 0x7f28fc1f2700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f28e400b410 tx=0x7f28e400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:14.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f37fe640 1 -- 172.21.15.125:0/1827067356 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4011830 con 0x7f28fc13d360 2023-10-18T10:26:14.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f37fe640 1 -- 172.21.15.125:0/1827067356 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f28e4011aa0 con 0x7f28fc13d360 2023-10-18T10:26:14.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc131e80 con 0x7f28fc13d360 2023-10-18T10:26:14.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f28f37fe640 1 -- 172.21.15.125:0/1827067356 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4010380 con 0x7f28fc13d360 2023-10-18T10:26:14.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.425+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc1eedd0 con 0x7f28fc13d360 2023-10-18T10:26:14.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.426+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc1323a0 con 0x7f28fc13d360 2023-10-18T10:26:14.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.427+0000 7f28f37fe640 1 -- 172.21.15.125:0/1827067356 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f28e40104e0 con 0x7f28fc13d360 2023-10-18T10:26:14.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.427+0000 7f28f37fe640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f28dc06ae70 0x7f28dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.427+0000 7f28f37fe640 1 -- 172.21.15.125:0/1827067356 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f28e4089620 con 0x7f28fc13d360 2023-10-18T10:26:14.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.427+0000 7f28fad9d640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f28dc06ae70 0x7f28dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.428+0000 7f28fad9d640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f28dc06ae70 0x7f28dc06d330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000f60 tx=0x7f28ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:14.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.431+0000 7f28f37fe640 1 -- 172.21.15.125:0/1827067356 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28fc1323a0 con 0x7f28fc13d360 2023-10-18T10:26:14.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.662+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f28fc1323a0 con 0x7f28fc13d360 2023-10-18T10:26:14.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.664+0000 7f28f37fe640 1 -- 172.21.15.125:0/1827067356 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v15) v1 ==== 97+0+0 (secure 0 0 0) 0x7f28fc1323a0 con 0x7f28fc13d360 2023-10-18T10:26:14.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f28dc06ae70 msgr2=0x7f28dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f28dc06ae70 0x7f28dc06d330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000f60 tx=0x7f28ec009040 comp rx=0 tx=0).stop 2023-10-18T10:26:14.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc13d360 msgr2=0x7f28fc1f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc13d360 0x7f28fc1f2700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f28e400b410 tx=0x7f28e400b8d0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 shutdown_connections 2023-10-18T10:26:14.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f28dc06ae70 0x7f28dc06d330 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28fc140660 0x7f28fc1f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28fc13d360 0x7f28fc1f2700 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 --2- 172.21.15.125:0/1827067356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28fc0c9d00 0x7f28fc1ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 >> 172.21.15.125:0/1827067356 conn(0x7f28fc069930 msgr2=0x7f28fc13bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:14.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 shutdown_connections 2023-10-18T10:26:14.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.666+0000 7f29010a9640 1 -- 172.21.15.125:0/1827067356 wait complete. 2023-10-18T10:26:14.678 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-18T10:26:14.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.780+0000 7ff0be39e640 1 -- 172.21.15.125:0/3407759204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0a0003f20 msgr2=0x7ff0a00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.780+0000 7ff0be39e640 1 --2- 172.21.15.125:0/3407759204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0a0003f20 0x7ff0a00047c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8001680 tx=0x7ff0a00057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.780+0000 7ff0bc93b640 1 -- 172.21.15.125:0/3407759204 reap_dead start 2023-10-18T10:26:14.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.780+0000 7ff0be39e640 1 -- 172.21.15.125:0/3407759204 shutdown_connections 2023-10-18T10:26:14.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.780+0000 7ff0be39e640 1 -- 172.21.15.125:0/3407759204 >> 172.21.15.125:0/3407759204 conn(0x7ff0b8069930 msgr2=0x7ff0b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:14.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.780+0000 7ff0be39e640 1 -- 172.21.15.125:0/3407759204 shutdown_connections 2023-10-18T10:26:14.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.780+0000 7ff0be39e640 1 -- 172.21.15.125:0/3407759204 wait complete. 2023-10-18T10:26:14.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.781+0000 7ff0be39e640 1 Processor -- start 2023-10-18T10:26:14.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.781+0000 7ff0be39e640 1 -- start start 2023-10-18T10:26:14.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0be39e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a0003f20 0x7ff0b8151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0be39e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0b8156930 0x7ff0b80ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0be39e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0b8156db0 0x7ff0b80aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0be39e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8150b80 con 0x7ff0b8156db0 2023-10-18T10:26:14.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0be39e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80ae4e0 con 0x7ff0a0003f20 2023-10-18T10:26:14.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0be39e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8150d90 con 0x7ff0b8156930 2023-10-18T10:26:14.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0b8156db0 0x7ff0b80aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0b8156db0 0x7ff0b80aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33450/0 (socket says 172.21.15.125:33450) 2023-10-18T10:26:14.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 -- 172.21.15.125:0/1552706815 learned_addr learned my addr 172.21.15.125:0/1552706815 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:14.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0b77fe640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0b8156930 0x7ff0b80ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0b7fff640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a0003f20 0x7ff0b8151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 -- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0b8156930 msgr2=0x7ff0b80ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0b8156930 0x7ff0b80ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 -- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a0003f20 msgr2=0x7ff0b8151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:14.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a0003f20 0x7ff0b8151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 -- 172.21.15.125:0/1552706815 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8151070 con 0x7ff0b8156db0 2023-10-18T10:26:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0bc93b640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0b8156db0 0x7ff0b80aded0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff0b806d510 tx=0x7ff0b000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0b4ff9640 1 -- 172.21.15.125:0/1552706815 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0015070 con 0x7ff0b8156db0 2023-10-18T10:26:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0b4ff9640 1 -- 172.21.15.125:0/1552706815 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff0b0004b20 con 0x7ff0b8156db0 2023-10-18T10:26:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b8151350 con 0x7ff0b8156db0 2023-10-18T10:26:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0b4ff9640 1 -- 172.21.15.125:0/1552706815 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0003970 con 0x7ff0b8156db0 2023-10-18T10:26:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.782+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b80b1710 con 0x7ff0b8156db0 2023-10-18T10:26:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.783+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b8067930 con 0x7ff0b8156db0 2023-10-18T10:26:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.784+0000 7ff0b4ff9640 1 -- 172.21.15.125:0/1552706815 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff0b0003ad0 con 0x7ff0b8156db0 2023-10-18T10:26:14.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.784+0000 7ff0b4ff9640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff0a406ae40 0x7ff0a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:14.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.784+0000 7ff0b4ff9640 1 -- 172.21.15.125:0/1552706815 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff0b008b550 con 0x7ff0b8156db0 2023-10-18T10:26:14.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.784+0000 7ff0b7fff640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff0a406ae40 0x7ff0a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:14.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.785+0000 7ff0b7fff640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff0a406ae40 0x7ff0a406d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac001ce0 tx=0x7ff0ac006040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:14.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:14.788+0000 7ff0b4ff9640 1 -- 172.21.15.125:0/1552706815 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0b00568c0 con 0x7ff0b8156db0 2023-10-18T10:26:15.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.016+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7ff0b80690e0 con 0x7ff0b8156db0 2023-10-18T10:26:15.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.019+0000 7ff0b4ff9640 1 -- 172.21.15.125:0/1552706815 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v16) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff0b0059ee0 con 0x7ff0b8156db0 2023-10-18T10:26:15.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.020+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff0a406ae40 msgr2=0x7ff0a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.020+0000 7ff0be39e640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff0a406ae40 0x7ff0a406d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac001ce0 tx=0x7ff0ac006040 comp rx=0 tx=0).stop 2023-10-18T10:26:15.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.020+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0b8156db0 msgr2=0x7ff0b80aded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.020+0000 7ff0be39e640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0b8156db0 0x7ff0b80aded0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff0b806d510 tx=0x7ff0b000c6a0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.021+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 shutdown_connections 2023-10-18T10:26:15.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.021+0000 7ff0be39e640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff0a406ae40 0x7ff0a406d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.021+0000 7ff0be39e640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0b8156db0 0x7ff0b80aded0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.021+0000 7ff0be39e640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0b8156930 0x7ff0b80ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.021+0000 7ff0be39e640 1 --2- 172.21.15.125:0/1552706815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a0003f20 0x7ff0b8151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.021+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 >> 172.21.15.125:0/1552706815 conn(0x7ff0b8069930 msgr2=0x7ff0b8158800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:15.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.021+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 shutdown_connections 2023-10-18T10:26:15.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.021+0000 7ff0be39e640 1 -- 172.21.15.125:0/1552706815 wait complete. 2023-10-18T10:26:15.033 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:26:15.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.135+0000 7f6e35007640 1 -- 172.21.15.125:0/4119285127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e1c003970 msgr2=0x7f6e1c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.135+0000 7f6e35007640 1 --2- 172.21.15.125:0/4119285127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e1c003970 0x7f6e1c003d90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6e20001680 tx=0x7f6e1c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.135+0000 7f6e35007640 1 -- 172.21.15.125:0/4119285127 shutdown_connections 2023-10-18T10:26:15.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.135+0000 7f6e35007640 1 --2- 172.21.15.125:0/4119285127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e1c003970 0x7f6e1c003d90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.135+0000 7f6e35007640 1 -- 172.21.15.125:0/4119285127 >> 172.21.15.125:0/4119285127 conn(0x7f6e30069930 msgr2=0x7f6e30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:15.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.135+0000 7f6e35007640 1 -- 172.21.15.125:0/4119285127 shutdown_connections 2023-10-18T10:26:15.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.135+0000 7f6e35007640 1 -- 172.21.15.125:0/4119285127 wait complete. 2023-10-18T10:26:15.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.136+0000 7f6e35007640 1 Processor -- start 2023-10-18T10:26:15.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.136+0000 7f6e35007640 1 -- start start 2023-10-18T10:26:15.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e35007640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c003970 0x7f6e30151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e35007640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e30148080 0x7f6e30156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e35007640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e3013bc70 0x7f6e300aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e35007640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30139d10 con 0x7f6e3013bc70 2023-10-18T10:26:15.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e35007640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3013c1f0 con 0x7f6e1c003970 2023-10-18T10:26:15.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e35007640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e301523d0 con 0x7f6e30148080 2023-10-18T10:26:15.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e2f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e3013bc70 0x7f6e300aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e2f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e3013bc70 0x7f6e300aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33458/0 (socket says 172.21.15.125:33458) 2023-10-18T10:26:15.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e2f59e640 1 -- 172.21.15.125:0/2863084050 learned_addr learned my addr 172.21.15.125:0/2863084050 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:15.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e2ed9d640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c003970 0x7f6e30151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e27fff640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e30148080 0x7f6e30156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e2f59e640 1 -- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e30148080 msgr2=0x7f6e30156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.137+0000 7f6e2f59e640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e30148080 0x7f6e30156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e2f59e640 1 -- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c003970 msgr2=0x7f6e30151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e2f59e640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c003970 0x7f6e30151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e2f59e640 1 -- 172.21.15.125:0/2863084050 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30139370 con 0x7f6e3013bc70 2023-10-18T10:26:15.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e2f59e640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e3013bc70 0x7f6e300aeb90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6e300ca740 tx=0x7f6e28011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:15.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e277fe640 1 -- 172.21.15.125:0/2863084050 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28004b20 con 0x7f6e3013bc70 2023-10-18T10:26:15.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e277fe640 1 -- 172.21.15.125:0/2863084050 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e28004c80 con 0x7f6e3013bc70 2023-10-18T10:26:15.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e30139650 con 0x7f6e3013bc70 2023-10-18T10:26:15.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e277fe640 1 -- 172.21.15.125:0/2863084050 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28003bd0 con 0x7f6e3013bc70 2023-10-18T10:26:15.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.138+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30157290 con 0x7f6e3013bc70 2023-10-18T10:26:15.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.139+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e30139e50 con 0x7f6e3013bc70 2023-10-18T10:26:15.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.139+0000 7f6e277fe640 1 -- 172.21.15.125:0/2863084050 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e28003d30 con 0x7f6e3013bc70 2023-10-18T10:26:15.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.140+0000 7f6e277fe640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e1006ae70 0x7f6e1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.140+0000 7f6e277fe640 1 -- 172.21.15.125:0/2863084050 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6e28089fd0 con 0x7f6e3013bc70 2023-10-18T10:26:15.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.140+0000 7f6e2ed9d640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e1006ae70 0x7f6e1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.140+0000 7f6e2ed9d640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e1006ae70 0x7f6e1006d330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e20034000 tx=0x7f6e20001190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:15.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.144+0000 7f6e277fe640 1 -- 172.21.15.125:0/2863084050 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e280553d0 con 0x7f6e3013bc70 2023-10-18T10:26:15.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.311+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e300b0780 con 0x7f6e3013bc70 2023-10-18T10:26:15.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.312+0000 7f6e277fe640 1 -- 172.21.15.125:0/2863084050 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13110 (secure 0 0 0) 0x7f6e280589f0 con 0x7f6e3013bc70 2023-10-18T10:26:15.314 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:15.314 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":35,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:26:12.483105+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:26:15.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.313+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e1006ae70 msgr2=0x7f6e1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.313+0000 7f6e35007640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e1006ae70 0x7f6e1006d330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e20034000 tx=0x7f6e20001190 comp rx=0 tx=0).stop 2023-10-18T10:26:15.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.313+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e3013bc70 msgr2=0x7f6e300aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.313+0000 7f6e35007640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e3013bc70 0x7f6e300aeb90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6e300ca740 tx=0x7f6e28011f40 comp rx=0 tx=0).stop 2023-10-18T10:26:15.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.314+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 shutdown_connections 2023-10-18T10:26:15.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.314+0000 7f6e35007640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6e1006ae70 0x7f6e1006d330 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.314+0000 7f6e35007640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e3013bc70 0x7f6e300aeb90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.314+0000 7f6e35007640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e30148080 0x7f6e30156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.314+0000 7f6e35007640 1 --2- 172.21.15.125:0/2863084050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1c003970 0x7f6e30151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.314+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 >> 172.21.15.125:0/2863084050 conn(0x7f6e30069930 msgr2=0x7f6e300ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:15.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.314+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 shutdown_connections 2023-10-18T10:26:15.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.314+0000 7f6e35007640 1 -- 172.21.15.125:0/2863084050 wait complete. 2023-10-18T10:26:15.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:26:15.327 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:26:15.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.430+0000 7f0a8be02640 1 -- 172.21.15.125:0/2026484780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 msgr2=0x7f0a840d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.430+0000 7f0a8be02640 1 --2- 172.21.15.125:0/2026484780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 0x7f0a840d1910 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005fd0 tx=0x7f0a7402bd90 comp rx=0 tx=0).stop 2023-10-18T10:26:15.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.430+0000 7f0a8be02640 1 -- 172.21.15.125:0/2026484780 shutdown_connections 2023-10-18T10:26:15.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.430+0000 7f0a8be02640 1 --2- 172.21.15.125:0/2026484780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a840ceb40 0x7f0a840b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.430+0000 7f0a8be02640 1 --2- 172.21.15.125:0/2026484780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 0x7f0a840d1910 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.430+0000 7f0a8be02640 1 -- 172.21.15.125:0/2026484780 >> 172.21.15.125:0/2026484780 conn(0x7f0a840d0d60 msgr2=0x7f0a840d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:15.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.430+0000 7f0a8be02640 1 -- 172.21.15.125:0/2026484780 shutdown_connections 2023-10-18T10:26:15.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.430+0000 7f0a8be02640 1 -- 172.21.15.125:0/2026484780 wait complete. 2023-10-18T10:26:15.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.431+0000 7f0a8be02640 1 Processor -- start 2023-10-18T10:26:15.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.431+0000 7f0a8be02640 1 -- start start 2023-10-18T10:26:15.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8be02640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a840ceb40 0x7f0a84155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8be02640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a840cf340 0x7f0a8415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8be02640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 0x7f0a8415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8be02640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841f1730 con 0x7f0a8415b2a0 2023-10-18T10:26:15.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8be02640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841570c0 con 0x7f0a840cf340 2023-10-18T10:26:15.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8be02640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a84157550 con 0x7f0a840ceb40 2023-10-18T10:26:15.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8a39f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 0x7f0a8415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8a39f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 0x7f0a8415dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33478/0 (socket says 172.21.15.125:33478) 2023-10-18T10:26:15.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8a39f640 1 -- 172.21.15.125:0/1773808119 learned_addr learned my addr 172.21.15.125:0/1773808119 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:15.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8939d640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a840cf340 0x7f0a8415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a89b9e640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a840ceb40 0x7f0a84155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.432+0000 7f0a8a39f640 1 -- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a840ceb40 msgr2=0x7f0a84155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a8a39f640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a840ceb40 0x7f0a84155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a8a39f640 1 -- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a840cf340 msgr2=0x7f0a8415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a8a39f640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a840cf340 0x7f0a8415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a8a39f640 1 -- 172.21.15.125:0/1773808119 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74009560 con 0x7f0a8415b2a0 2023-10-18T10:26:15.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a8939d640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a840cf340 0x7f0a8415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:15.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a89b9e640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a840ceb40 0x7f0a84155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:15.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a8a39f640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 0x7f0a8415dc20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00daa0 tx=0x7f0a7c00df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:15.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a7a7fc640 1 -- 172.21.15.125:0/1773808119 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00be00 con 0x7f0a8415b2a0 2023-10-18T10:26:15.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a7a7fc640 1 -- 172.21.15.125:0/1773808119 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a7c003a60 con 0x7f0a8415b2a0 2023-10-18T10:26:15.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a840b0d50 con 0x7f0a8415b2a0 2023-10-18T10:26:15.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a7a7fc640 1 -- 172.21.15.125:0/1773808119 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c003e70 con 0x7f0a8415b2a0 2023-10-18T10:26:15.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.433+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a841564b0 con 0x7f0a8415b2a0 2023-10-18T10:26:15.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.434+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a84157250 con 0x7f0a8415b2a0 2023-10-18T10:26:15.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.434+0000 7f0a7a7fc640 1 -- 172.21.15.125:0/1773808119 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0a7c003fd0 con 0x7f0a8415b2a0 2023-10-18T10:26:15.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.434+0000 7f0a7a7fc640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a6806ae40 0x7f0a6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.435+0000 7f0a7a7fc640 1 -- 172.21.15.125:0/1773808119 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0a7c089980 con 0x7f0a8415b2a0 2023-10-18T10:26:15.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.435+0000 7f0a89b9e640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a6806ae40 0x7f0a6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.435+0000 7f0a89b9e640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a6806ae40 0x7f0a6806d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000f60 tx=0x7f0a80009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:15.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.439+0000 7f0a7a7fc640 1 -- 172.21.15.125:0/1773808119 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a7c054c40 con 0x7f0a8415b2a0 2023-10-18T10:26:15.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.604+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a840690e0 con 0x7f0a8415b2a0 2023-10-18T10:26:15.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.604+0000 7f0a7a7fc640 1 -- 172.21.15.125:0/1773808119 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a7c0170a0 con 0x7f0a8415b2a0 2023-10-18T10:26:15.606 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:26:15.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a6806ae40 msgr2=0x7f0a6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a6806ae40 0x7f0a6806d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000f60 tx=0x7f0a80009040 comp rx=0 tx=0).stop 2023-10-18T10:26:15.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 msgr2=0x7f0a8415dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 0x7f0a8415dc20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00daa0 tx=0x7f0a7c00df60 comp rx=0 tx=0).stop 2023-10-18T10:26:15.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 shutdown_connections 2023-10-18T10:26:15.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a6806ae40 0x7f0a6806d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a840cf340 0x7f0a8415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a8415b2a0 0x7f0a8415dc20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 --2- 172.21.15.125:0/1773808119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a840ceb40 0x7f0a84155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 >> 172.21.15.125:0/1773808119 conn(0x7f0a840d0d60 msgr2=0x7f0a8406b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:15.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 shutdown_connections 2023-10-18T10:26:15.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.606+0000 7f0a8be02640 1 -- 172.21.15.125:0/1773808119 wait complete. 2023-10-18T10:26:15.618 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:26:15.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.723+0000 7fd83d21a640 1 -- 172.21.15.125:0/405828995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 msgr2=0x7fd824004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.723+0000 7fd83d21a640 1 --2- 172.21.15.125:0/405828995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 0x7fd824004c40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd820006fc0 tx=0x7fd8240057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.723+0000 7fd83d21a640 1 -- 172.21.15.125:0/405828995 shutdown_connections 2023-10-18T10:26:15.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.723+0000 7fd83d21a640 1 --2- 172.21.15.125:0/405828995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 0x7fd824004c40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.723+0000 7fd83d21a640 1 -- 172.21.15.125:0/405828995 >> 172.21.15.125:0/405828995 conn(0x7fd8380d0d60 msgr2=0x7fd8380d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:15.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.723+0000 7fd83d21a640 1 -- 172.21.15.125:0/405828995 shutdown_connections 2023-10-18T10:26:15.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.723+0000 7fd83d21a640 1 -- 172.21.15.125:0/405828995 wait complete. 2023-10-18T10:26:15.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.724+0000 7fd83d21a640 1 Processor -- start 2023-10-18T10:26:15.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.724+0000 7fd83d21a640 1 -- start start 2023-10-18T10:26:15.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd83d21a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 0x7fd838158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd83d21a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8380c7e30 0x7fd83806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd83d21a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8380c8d10 0x7fd83806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd83d21a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd838158a40 con 0x7fd8240047c0 2023-10-18T10:26:15.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd83d21a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380c92c0 con 0x7fd8380c8d10 2023-10-18T10:26:15.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd83d21a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380c95a0 con 0x7fd8380c7e30 2023-10-18T10:26:15.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 0x7fd838158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 0x7fd838158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33502/0 (socket says 172.21.15.125:33502) 2023-10-18T10:26:15.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 -- 172.21.15.125:0/3747433964 learned_addr learned my addr 172.21.15.125:0/3747433964 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:15.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 -- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8380c7e30 msgr2=0x7fd83806a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8380c7e30 0x7fd83806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 -- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8380c8d10 msgr2=0x7fd83806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8380c8d10 0x7fd83806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 -- 172.21.15.125:0/3747433964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380c9880 con 0x7fd8240047c0 2023-10-18T10:26:15.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd836d9d640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 0x7fd838158500 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd828004cb0 tx=0x7fd828001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:15.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd82f7fe640 1 -- 172.21.15.125:0/3747433964 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828011660 con 0x7fd8240047c0 2023-10-18T10:26:15.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd82f7fe640 1 -- 172.21.15.125:0/3747433964 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd8280118b0 con 0x7fd8240047c0 2023-10-18T10:26:15.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd82f7fe640 1 -- 172.21.15.125:0/3747433964 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828011c80 con 0x7fd8240047c0 2023-10-18T10:26:15.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.725+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83815c810 con 0x7fd8240047c0 2023-10-18T10:26:15.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.726+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8380b22e0 con 0x7fd8240047c0 2023-10-18T10:26:15.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.726+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8380690e0 con 0x7fd8240047c0 2023-10-18T10:26:15.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.727+0000 7fd82f7fe640 1 -- 172.21.15.125:0/3747433964 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd82801a400 con 0x7fd8240047c0 2023-10-18T10:26:15.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.727+0000 7fd82f7fe640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81806ae70 0x7fd81806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:15.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.727+0000 7fd82f7fe640 1 -- 172.21.15.125:0/3747433964 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd828089ff0 con 0x7fd8240047c0 2023-10-18T10:26:15.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.727+0000 7fd82ffff640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81806ae70 0x7fd81806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:15.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.728+0000 7fd82ffff640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81806ae70 0x7fd81806d330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd820001a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:15.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.731+0000 7fd82f7fe640 1 -- 172.21.15.125:0/3747433964 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd828055340 con 0x7fd8240047c0 2023-10-18T10:26:15.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.900+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd8380d0b40 con 0x7fd8240047c0 2023-10-18T10:26:15.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.901+0000 7fd82f7fe640 1 -- 172.21.15.125:0/3747433964 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd828058960 con 0x7fd8240047c0 2023-10-18T10:26:15.903 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:15.904 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:26:15.904 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:15.904 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:26:15.904 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:26:15.905 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:15.905 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:26:15.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.902+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81806ae70 msgr2=0x7fd81806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.902+0000 7fd83d21a640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81806ae70 0x7fd81806d330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd820001a20 comp rx=0 tx=0).stop 2023-10-18T10:26:15.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.902+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 msgr2=0x7fd838158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:15.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.902+0000 7fd83d21a640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 0x7fd838158500 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd828004cb0 tx=0x7fd828001630 comp rx=0 tx=0).stop 2023-10-18T10:26:15.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.903+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 shutdown_connections 2023-10-18T10:26:15.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.903+0000 7fd83d21a640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd81806ae70 0x7fd81806d330 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.903+0000 7fd83d21a640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8380c8d10 0x7fd83806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.903+0000 7fd83d21a640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8380c7e30 0x7fd83806a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.903+0000 7fd83d21a640 1 --2- 172.21.15.125:0/3747433964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8240047c0 0x7fd838158500 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:15.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.903+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 >> 172.21.15.125:0/3747433964 conn(0x7fd8380d0d60 msgr2=0x7fd8380b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:15.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.903+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 shutdown_connections 2023-10-18T10:26:15.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:15.903+0000 7fd83d21a640 1 -- 172.21.15.125:0/3747433964 wait complete. 2023-10-18T10:26:15.915 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:26:15.915 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:26:16.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.020+0000 7fd9eadf5640 1 -- 172.21.15.125:0/233313173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4004d00 msgr2=0x7fd9d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:16.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.020+0000 7fd9eadf5640 1 --2- 172.21.15.125:0/233313173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4004d00 0x7fd9d4005180 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40b0180 tx=0x7fd9d40057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.020+0000 7fd9e9392640 1 -- 172.21.15.125:0/233313173 reap_dead start 2023-10-18T10:26:16.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.020+0000 7fd9eadf5640 1 -- 172.21.15.125:0/233313173 shutdown_connections 2023-10-18T10:26:16.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.020+0000 7fd9eadf5640 1 -- 172.21.15.125:0/233313173 >> 172.21.15.125:0/233313173 conn(0x7fd9e406fae0 msgr2=0x7fd9e406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:16.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.020+0000 7fd9eadf5640 1 -- 172.21.15.125:0/233313173 shutdown_connections 2023-10-18T10:26:16.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.020+0000 7fd9eadf5640 1 -- 172.21.15.125:0/233313173 wait complete. 2023-10-18T10:26:16.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.021+0000 7fd9eadf5640 1 Processor -- start 2023-10-18T10:26:16.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.021+0000 7fd9eadf5640 1 -- start start 2023-10-18T10:26:16.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9eadf5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e4145050 0x7fd9e406b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:16.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9eadf5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e4145430 0x7fd9e406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:16.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9eadf5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e4156750 0x7fd9e4156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:16.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9eadf5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e4157140 con 0x7fd9e4145430 2023-10-18T10:26:16.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9eadf5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e41f22f0 con 0x7fd9e4156750 2023-10-18T10:26:16.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9eadf5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e41f19c0 con 0x7fd9e4145050 2023-10-18T10:26:16.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e4145430 0x7fd9e406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:16.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e4145430 0x7fd9e406bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33520/0 (socket says 172.21.15.125:33520) 2023-10-18T10:26:16.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e3fff640 1 -- 172.21.15.125:0/1505294365 learned_addr learned my addr 172.21.15.125:0/1505294365 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:16.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e9392640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e4156750 0x7fd9e4156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e3fff640 1 -- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e4145050 msgr2=0x7fd9e406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e3fff640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e4145050 0x7fd9e406b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e3fff640 1 -- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e4156750 msgr2=0x7fd9e4156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e3fff640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e4156750 0x7fd9e4156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.022+0000 7fd9e3fff640 1 -- 172.21.15.125:0/1505294365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40b2330 con 0x7fd9e4145430 2023-10-18T10:26:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.023+0000 7fd9e3fff640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e4145430 0x7fd9e406bec0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000e590 tx=0x7fd9d000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.023+0000 7fd9e17fa640 1 -- 172.21.15.125:0/1505294365 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0010890 con 0x7fd9e4145430 2023-10-18T10:26:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.023+0000 7fd9e17fa640 1 -- 172.21.15.125:0/1505294365 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9d0010ae0 con 0x7fd9e4145430 2023-10-18T10:26:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.023+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e41f1ca0 con 0x7fd9e4145430 2023-10-18T10:26:16.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.023+0000 7fd9e17fa640 1 -- 172.21.15.125:0/1505294365 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0011460 con 0x7fd9e4145430 2023-10-18T10:26:16.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.023+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e41f1640 con 0x7fd9e4145430 2023-10-18T10:26:16.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.024+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e40690e0 con 0x7fd9e4145430 2023-10-18T10:26:16.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.024+0000 7fd9e17fa640 1 -- 172.21.15.125:0/1505294365 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd9d00115c0 con 0x7fd9e4145430 2023-10-18T10:26:16.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.025+0000 7fd9e17fa640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd9c806ae70 0x7fd9c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:16.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.025+0000 7fd9e17fa640 1 -- 172.21.15.125:0/1505294365 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd9d0089ba0 con 0x7fd9e4145430 2023-10-18T10:26:16.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.025+0000 7fd9e8b91640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd9c806ae70 0x7fd9c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:16.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.025+0000 7fd9e8b91640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd9c806ae70 0x7fd9c806d330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009ab0 tx=0x7fd9d8009550 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:16.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.029+0000 7fd9e17fa640 1 -- 172.21.15.125:0/1505294365 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9d0054dd0 con 0x7fd9e4145430 2023-10-18T10:26:16.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.194+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9e40ac840 con 0x7fd9e4145430 2023-10-18T10:26:16.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.482+0000 7fd9e17fa640 1 -- 172.21.15.125:0/1505294365 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd9d001f070 con 0x7fd9e4145430 2023-10-18T10:26:16.485 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:26:16.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd9c806ae70 msgr2=0x7fd9c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:16.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd9c806ae70 0x7fd9c806d330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009ab0 tx=0x7fd9d8009550 comp rx=0 tx=0).stop 2023-10-18T10:26:16.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e4145430 msgr2=0x7fd9e406bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:16.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e4145430 0x7fd9e406bec0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000e590 tx=0x7fd9d000ea50 comp rx=0 tx=0).stop 2023-10-18T10:26:16.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 shutdown_connections 2023-10-18T10:26:16.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fd9c806ae70 0x7fd9c806d330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e4156750 0x7fd9e4156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e4145430 0x7fd9e406bec0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 --2- 172.21.15.125:0/1505294365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e4145050 0x7fd9e406b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 >> 172.21.15.125:0/1505294365 conn(0x7fd9e406fae0 msgr2=0x7fd9e415a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:16.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 shutdown_connections 2023-10-18T10:26:16.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.485+0000 7fd9eadf5640 1 -- 172.21.15.125:0/1505294365 wait complete. 2023-10-18T10:26:16.501 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:26:16.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.609+0000 7f31b2f33640 1 -- 172.21.15.125:0/3369074225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 msgr2=0x7f31ac06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:16.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.609+0000 7f31b2f33640 1 --2- 172.21.15.125:0/3369074225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 0x7f31ac06bb60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009830 tx=0x7f31a002bc30 comp rx=0 tx=0).stop 2023-10-18T10:26:16.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.609+0000 7f31b2f33640 1 -- 172.21.15.125:0/3369074225 shutdown_connections 2023-10-18T10:26:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.609+0000 7f31b2f33640 1 --2- 172.21.15.125:0/3369074225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31ac141690 0x7f31ac0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.609+0000 7f31b2f33640 1 --2- 172.21.15.125:0/3369074225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 0x7f31ac06bb60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.609+0000 7f31b2f33640 1 -- 172.21.15.125:0/3369074225 >> 172.21.15.125:0/3369074225 conn(0x7f31ac069930 msgr2=0x7f31ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.609+0000 7f31b2f33640 1 -- 172.21.15.125:0/3369074225 shutdown_connections 2023-10-18T10:26:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.609+0000 7f31b2f33640 1 -- 172.21.15.125:0/3369074225 wait complete. 2023-10-18T10:26:16.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.610+0000 7f31b2f33640 1 Processor -- start 2023-10-18T10:26:16.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.610+0000 7f31b2f33640 1 -- start start 2023-10-18T10:26:16.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31b2f33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31ac06c0a0 0x7f31ac13ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:16.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31b2f33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 0x7f31ac1e2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:16.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31b2f33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31ac141690 0x7f31ac1e2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:16.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31b2f33640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac1dc220 con 0x7f31ac13e520 2023-10-18T10:26:16.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31b2f33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac1d9cf0 con 0x7f31ac141690 2023-10-18T10:26:16.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31b2f33640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac1e0160 con 0x7f31ac06c0a0 2023-10-18T10:26:16.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 0x7f31ac1e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:16.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 0x7f31ac1e2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33538/0 (socket says 172.21.15.125:33538) 2023-10-18T10:26:16.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 -- 172.21.15.125:0/4227210595 learned_addr learned my addr 172.21.15.125:0/4227210595 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:16.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31b14d0640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31ac141690 0x7f31ac1e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:16.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31b0ccf640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31ac06c0a0 0x7f31ac13ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:16.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 -- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31ac06c0a0 msgr2=0x7f31ac13ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:16.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31ac06c0a0 0x7f31ac13ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 -- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31ac141690 msgr2=0x7f31ac1e2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:16.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31ac141690 0x7f31ac1e2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:16.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 -- 172.21.15.125:0/4227210595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a0009490 con 0x7f31ac13e520 2023-10-18T10:26:16.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.611+0000 7f31abfff640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 0x7f31ac1e2420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f319800b410 tx=0x7f319800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:16.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.612+0000 7f31a97fa640 1 -- 172.21.15.125:0/4227210595 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198011830 con 0x7f31ac13e520 2023-10-18T10:26:16.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.612+0000 7f31a97fa640 1 -- 172.21.15.125:0/4227210595 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3198011aa0 con 0x7f31ac13e520 2023-10-18T10:26:16.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.612+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ac1d6b50 con 0x7f31ac13e520 2023-10-18T10:26:16.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.612+0000 7f31a97fa640 1 -- 172.21.15.125:0/4227210595 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198010380 con 0x7f31ac13e520 2023-10-18T10:26:16.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.612+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ac1dbf40 con 0x7f31ac13e520 2023-10-18T10:26:16.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.612+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ac1db9e0 con 0x7f31ac13e520 2023-10-18T10:26:16.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.613+0000 7f31a97fa640 1 -- 172.21.15.125:0/4227210595 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f31980104e0 con 0x7f31ac13e520 2023-10-18T10:26:16.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.613+0000 7f31a97fa640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f319006ae40 0x7f319006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:16.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.614+0000 7f31a97fa640 1 -- 172.21.15.125:0/4227210595 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f3198089790 con 0x7f31ac13e520 2023-10-18T10:26:16.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.614+0000 7f31b0ccf640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f319006ae40 0x7f319006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:16.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.614+0000 7f31b0ccf640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f319006ae40 0x7f319006d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f31a0036000 tx=0x7f31a0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:16.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.618+0000 7f31a97fa640 1 -- 172.21.15.125:0/4227210595 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31980549a0 con 0x7f31ac13e520 2023-10-18T10:26:16.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:16.783+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f31ac0690e0 con 0x7f31ac13e520 2023-10-18T10:26:17.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.485+0000 7f31a97fa640 1 -- 172.21.15.125:0/4227210595 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3198057fc0 con 0x7f31ac13e520 2023-10-18T10:26:17.487 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:26:17.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.487+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f319006ae40 msgr2=0x7f319006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:17.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.487+0000 7f31b2f33640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f319006ae40 0x7f319006d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f31a0036000 tx=0x7f31a0004730 comp rx=0 tx=0).stop 2023-10-18T10:26:17.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 msgr2=0x7f31ac1e2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:17.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 0x7f31ac1e2420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f319800b410 tx=0x7f319800b8d0 comp rx=0 tx=0).stop 2023-10-18T10:26:17.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 shutdown_connections 2023-10-18T10:26:17.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f319006ae40 0x7f319006d300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:17.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31ac141690 0x7f31ac1e2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:17.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31ac13e520 0x7f31ac1e2420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:17.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 --2- 172.21.15.125:0/4227210595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31ac06c0a0 0x7f31ac13ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:17.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 >> 172.21.15.125:0/4227210595 conn(0x7f31ac069930 msgr2=0x7f31ac1df970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:17.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 shutdown_connections 2023-10-18T10:26:17.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.488+0000 7f31b2f33640 1 -- 172.21.15.125:0/4227210595 wait complete. 2023-10-18T10:26:17.515 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:26:17.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.620+0000 7f6b8a78a640 1 -- 172.21.15.125:0/1187840163 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6c0039a0 msgr2=0x7f6b6c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.620+0000 7f6b8a78a640 1 --2- 172.21.15.125:0/1187840163 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6c0039a0 0x7f6b6c003dc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6b74001650 tx=0x7f6b6c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.620+0000 7f6b88d27640 1 -- 172.21.15.125:0/1187840163 reap_dead start 2023-10-18T10:26:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.620+0000 7f6b8a78a640 1 -- 172.21.15.125:0/1187840163 shutdown_connections 2023-10-18T10:26:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.620+0000 7f6b8a78a640 1 -- 172.21.15.125:0/1187840163 >> 172.21.15.125:0/1187840163 conn(0x7f6b84069930 msgr2=0x7f6b84069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:17.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.620+0000 7f6b8a78a640 1 -- 172.21.15.125:0/1187840163 shutdown_connections 2023-10-18T10:26:17.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.620+0000 7f6b8a78a640 1 -- 172.21.15.125:0/1187840163 wait complete. 2023-10-18T10:26:17.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.621+0000 7f6b8a78a640 1 Processor -- start 2023-10-18T10:26:17.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.622+0000 7f6b8a78a640 1 -- start start 2023-10-18T10:26:17.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.622+0000 7f6b8a78a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b6c0039a0 0x7f6b841e2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:17.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.622+0000 7f6b8a78a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b84135a70 0x7f6b84156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:17.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.622+0000 7f6b8a78a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b841573d0 0x7f6b8413e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.622+0000 7f6b8a78a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840ca870 con 0x7f6b841573d0 2023-10-18T10:26:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.622+0000 7f6b8a78a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b84157810 con 0x7f6b84135a70 2023-10-18T10:26:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.622+0000 7f6b8a78a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840caac0 con 0x7f6b6c0039a0 2023-10-18T10:26:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b841573d0 0x7f6b8413e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b841573d0 0x7f6b8413e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43862/0 (socket says 172.21.15.125:43862) 2023-10-18T10:26:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 -- 172.21.15.125:0/3368553899 learned_addr learned my addr 172.21.15.125:0/3368553899 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b83fff640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b6c0039a0 0x7f6b841e2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b837fe640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b84135a70 0x7f6b84156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 -- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b6c0039a0 msgr2=0x7f6b841e2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b6c0039a0 0x7f6b841e2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 -- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b84135a70 msgr2=0x7f6b84156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b84135a70 0x7f6b84156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 -- 172.21.15.125:0/3368553899 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b840cada0 con 0x7f6b841573d0 2023-10-18T10:26:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b88d27640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b841573d0 0x7f6b8413e8b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c000f80 tx=0x7f6b7c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b80ff9640 1 -- 172.21.15.125:0/3368553899 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00cc20 con 0x7f6b841573d0 2023-10-18T10:26:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b80ff9640 1 -- 172.21.15.125:0/3368553899 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b7c00ce90 con 0x7f6b841573d0 2023-10-18T10:26:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b840cb080 con 0x7f6b841573d0 2023-10-18T10:26:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b80ff9640 1 -- 172.21.15.125:0/3368553899 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0116c0 con 0x7f6b841573d0 2023-10-18T10:26:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.623+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b841e2d70 con 0x7f6b841573d0 2023-10-18T10:26:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.624+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b840cb5a0 con 0x7f6b841573d0 2023-10-18T10:26:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.625+0000 7f6b80ff9640 1 -- 172.21.15.125:0/3368553899 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b7c011820 con 0x7f6b841573d0 2023-10-18T10:26:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.625+0000 7f6b80ff9640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6b7006ae40 0x7f6b7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.625+0000 7f6b80ff9640 1 -- 172.21.15.125:0/3368553899 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f6b7c089d20 con 0x7f6b841573d0 2023-10-18T10:26:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.625+0000 7f6b83fff640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6b7006ae40 0x7f6b7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:17.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.626+0000 7f6b83fff640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6b7006ae40 0x7f6b7006d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b74034000 tx=0x7f6b74001190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:17.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.629+0000 7f6b80ff9640 1 -- 172.21.15.125:0/3368553899 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b840cb5a0 con 0x7f6b841573d0 2023-10-18T10:26:17.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:17.829+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6b841e2f00 con 0x7f6b841573d0 2023-10-18T10:26:18.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.514+0000 7f6b80ff9640 1 -- 172.21.15.125:0/3368553899 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6b7c054d20 con 0x7f6b841573d0 2023-10-18T10:26:18.517 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 5 and data pool 6 2023-10-18T10:26:18.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6b7006ae40 msgr2=0x7f6b7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6b7006ae40 0x7f6b7006d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b74034000 tx=0x7f6b74001190 comp rx=0 tx=0).stop 2023-10-18T10:26:18.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b841573d0 msgr2=0x7f6b8413e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b841573d0 0x7f6b8413e8b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c000f80 tx=0x7f6b7c00ed90 comp rx=0 tx=0).stop 2023-10-18T10:26:18.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 shutdown_connections 2023-10-18T10:26:18.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6b7006ae40 0x7f6b7006d300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b84135a70 0x7f6b84156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b841573d0 0x7f6b8413e8b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 --2- 172.21.15.125:0/3368553899 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b6c0039a0 0x7f6b841e2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.517+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 >> 172.21.15.125:0/3368553899 conn(0x7f6b84069930 msgr2=0x7f6b8413c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:18.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.518+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 shutdown_connections 2023-10-18T10:26:18.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.518+0000 7f6b8a78a640 1 -- 172.21.15.125:0/3368553899 wait complete. 2023-10-18T10:26:18.533 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:26:18.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.634+0000 7fee17a75640 1 -- 172.21.15.125:0/3642554641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc005e10 msgr2=0x7fedfc006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.634+0000 7fee17a75640 1 --2- 172.21.15.125:0/3642554641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedfc005e10 0x7fedfc006290 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fee080043e0 tx=0x7fedfc006900 comp rx=0 tx=0).stop 2023-10-18T10:26:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.634+0000 7fee16012640 1 -- 172.21.15.125:0/3642554641 reap_dead start 2023-10-18T10:26:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.634+0000 7fee17a75640 1 -- 172.21.15.125:0/3642554641 shutdown_connections 2023-10-18T10:26:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.634+0000 7fee17a75640 1 -- 172.21.15.125:0/3642554641 >> 172.21.15.125:0/3642554641 conn(0x7fee10069930 msgr2=0x7fee10069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.634+0000 7fee17a75640 1 -- 172.21.15.125:0/3642554641 shutdown_connections 2023-10-18T10:26:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.634+0000 7fee17a75640 1 -- 172.21.15.125:0/3642554641 wait complete. 2023-10-18T10:26:18.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.635+0000 7fee17a75640 1 Processor -- start 2023-10-18T10:26:18.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.635+0000 7fee17a75640 1 -- start start 2023-10-18T10:26:18.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee17a75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee10142c40 0x7fee10152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:18.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee17a75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee10143020 0x7fee100ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:18.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee17a75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee1015ae00 0x7fee1015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:18.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee17a75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1015b7d0 con 0x7fee10142c40 2023-10-18T10:26:18.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee17a75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10143520 con 0x7fee10143020 2023-10-18T10:26:18.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee17a75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee10143770 con 0x7fee1015ae00 2023-10-18T10:26:18.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15811640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee10142c40 0x7fee10152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:18.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15811640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee10142c40 0x7fee10152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43884/0 (socket says 172.21.15.125:43884) 2023-10-18T10:26:18.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15811640 1 -- 172.21.15.125:0/3457758166 learned_addr learned my addr 172.21.15.125:0/3457758166 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:18.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15010640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee10143020 0x7fee100ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:18.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee16012640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee1015ae00 0x7fee1015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:18.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15811640 1 -- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee1015ae00 msgr2=0x7fee1015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15811640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee1015ae00 0x7fee1015b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15811640 1 -- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee10143020 msgr2=0x7fee100ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15811640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee10143020 0x7fee100ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.636+0000 7fee15811640 1 -- 172.21.15.125:0/3457758166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee10143a50 con 0x7fee10142c40 2023-10-18T10:26:18.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.637+0000 7fee15811640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee10142c40 0x7fee10152fb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fee0c004650 tx=0x7fee0c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:18.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.637+0000 7fee067fc640 1 -- 172.21.15.125:0/3457758166 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0041d0 con 0x7fee10142c40 2023-10-18T10:26:18.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.637+0000 7fee067fc640 1 -- 172.21.15.125:0/3457758166 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee0c015070 con 0x7fee10142c40 2023-10-18T10:26:18.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.637+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee10143d30 con 0x7fee10142c40 2023-10-18T10:26:18.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.637+0000 7fee067fc640 1 -- 172.21.15.125:0/3457758166 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c010ad0 con 0x7fee10142c40 2023-10-18T10:26:18.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.637+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee101442f0 con 0x7fee10142c40 2023-10-18T10:26:18.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.638+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee101444b0 con 0x7fee10142c40 2023-10-18T10:26:18.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.638+0000 7fee067fc640 1 -- 172.21.15.125:0/3457758166 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fee0c017030 con 0x7fee10142c40 2023-10-18T10:26:18.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.639+0000 7fee067fc640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fedf406ae40 0x7fedf406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:18.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.639+0000 7fee067fc640 1 -- 172.21.15.125:0/3457758166 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fee0c08a120 con 0x7fee10142c40 2023-10-18T10:26:18.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.639+0000 7fee15010640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fedf406ae40 0x7fedf406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:18.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.639+0000 7fee15010640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fedf406ae40 0x7fedf406d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fee000017a0 tx=0x7fee00001df0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:18.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.643+0000 7fee067fc640 1 -- 172.21.15.125:0/3457758166 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee0c058b30 con 0x7fee10142c40 2023-10-18T10:26:18.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.811+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee10070920 con 0x7fee10142c40 2023-10-18T10:26:18.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.812+0000 7fee067fc640 1 -- 172.21.15.125:0/3457758166 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16835 (secure 0 0 0) 0x7fee0c0586c0 con 0x7fee10142c40 2023-10-18T10:26:18.814 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:18.815 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":38,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:26:17.830729+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:26:16.213313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-18T10:26:16.801795+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:26:18.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fedf406ae40 msgr2=0x7fedf406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fedf406ae40 0x7fedf406d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fee000017a0 tx=0x7fee00001df0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee10142c40 msgr2=0x7fee10152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee10142c40 0x7fee10152fb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fee0c004650 tx=0x7fee0c00ba40 comp rx=0 tx=0).stop 2023-10-18T10:26:18.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 shutdown_connections 2023-10-18T10:26:18.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fedf406ae40 0x7fedf406d300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee10143020 0x7fee100ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee1015ae00 0x7fee1015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 --2- 172.21.15.125:0/3457758166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee10142c40 0x7fee10152fb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 >> 172.21.15.125:0/3457758166 conn(0x7fee10069930 msgr2=0x7fee101578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:18.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 shutdown_connections 2023-10-18T10:26:18.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.814+0000 7fee17a75640 1 -- 172.21.15.125:0/3457758166 wait complete. 2023-10-18T10:26:18.827 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:26:18.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.927+0000 7f2241eaf640 1 -- 172.21.15.125:0/902108315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c13e520 msgr2=0x7f223c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.927+0000 7f2241eaf640 1 --2- 172.21.15.125:0/902108315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c13e520 0x7f223c06bb60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f223c0cbdc0 tx=0x7f222c02bd30 comp rx=0 tx=0).stop 2023-10-18T10:26:18.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.927+0000 7f2241eaf640 1 -- 172.21.15.125:0/902108315 shutdown_connections 2023-10-18T10:26:18.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.927+0000 7f2241eaf640 1 --2- 172.21.15.125:0/902108315 >> v2:172.21.15.139:3300/0 conn(0x7f223c141690 0x7f223c0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.927+0000 7f2241eaf640 1 --2- 172.21.15.125:0/902108315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c13e520 0x7f223c06bb60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.927+0000 7f2241eaf640 1 -- 172.21.15.125:0/902108315 >> 172.21.15.125:0/902108315 conn(0x7f223c069930 msgr2=0x7f223c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:18.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.927+0000 7f2241eaf640 1 -- 172.21.15.125:0/902108315 shutdown_connections 2023-10-18T10:26:18.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.927+0000 7f2241eaf640 1 -- 172.21.15.125:0/902108315 wait complete. 2023-10-18T10:26:18.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.928+0000 7f2241eaf640 1 Processor -- start 2023-10-18T10:26:18.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.928+0000 7f2241eaf640 1 -- start start 2023-10-18T10:26:18.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f2241eaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f223c06c0a0 0x7f223c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:18.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f2241eaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f223c13e520 0x7f223c13a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:18.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f2241eaf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c141690 0x7f223c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:18.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f2241eaf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c151ce0 con 0x7f223c141690 2023-10-18T10:26:18.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f2241eaf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c13d4c0 con 0x7f223c06c0a0 2023-10-18T10:26:18.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f2241eaf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c1e0640 con 0x7f223c13e520 2023-10-18T10:26:18.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c141690 0x7f223c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:18.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c141690 0x7f223c15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43910/0 (socket says 172.21.15.125:43910) 2023-10-18T10:26:18.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 -- 172.21.15.125:0/547153516 learned_addr learned my addr 172.21.15.125:0/547153516 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:18.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223affd640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f223c13e520 0x7f223c13a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:18.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223b7fe640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f223c06c0a0 0x7f223c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:18.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 -- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f223c13e520 msgr2=0x7f223c13a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f223c13e520 0x7f223c13a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 -- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f223c06c0a0 msgr2=0x7f223c1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:18.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f223c06c0a0 0x7f223c1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:18.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 -- 172.21.15.125:0/547153516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c009560 con 0x7f223c141690 2023-10-18T10:26:18.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f223bfff640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c141690 0x7f223c15aa20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f223400c780 tx=0x7f223400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:18.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.929+0000 7f2223fff640 1 -- 172.21.15.125:0/547153516 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234003d80 con 0x7f223c141690 2023-10-18T10:26:18.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.930+0000 7f2223fff640 1 -- 172.21.15.125:0/547153516 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f223400ce90 con 0x7f223c141690 2023-10-18T10:26:18.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.930+0000 7f2223fff640 1 -- 172.21.15.125:0/547153516 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340052d0 con 0x7f223c141690 2023-10-18T10:26:18.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.930+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c1e08e0 con 0x7f223c141690 2023-10-18T10:26:18.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.930+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c1e0ea0 con 0x7f223c141690 2023-10-18T10:26:18.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.931+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c0690e0 con 0x7f223c141690 2023-10-18T10:26:18.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.931+0000 7f2223fff640 1 -- 172.21.15.125:0/547153516 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2234005430 con 0x7f223c141690 2023-10-18T10:26:18.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.931+0000 7f2223fff640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f222806ae40 0x7f222806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:18.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.931+0000 7f2223fff640 1 -- 172.21.15.125:0/547153516 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2234088d90 con 0x7f223c141690 2023-10-18T10:26:18.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.932+0000 7f223b7fe640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f222806ae40 0x7f222806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:18.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.932+0000 7f223b7fe640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f222806ae40 0x7f222806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f222c035000 tx=0x7f222c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:18.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:18.936+0000 7f2223fff640 1 -- 172.21.15.125:0/547153516 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2234053dc0 con 0x7f223c141690 2023-10-18T10:26:19.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.102+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f223c06a090 con 0x7f223c141690 2023-10-18T10:26:19.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.103+0000 7f2223fff640 1 -- 172.21.15.125:0/547153516 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16835 (secure 0 0 0) 0x7f22340573e0 con 0x7f223c141690 2023-10-18T10:26:19.105 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:19.105 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":38,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:26:17.830729+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:26:16.213313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-18T10:26:16.801795+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:26:19.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.104+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f222806ae40 msgr2=0x7f222806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.104+0000 7f2241eaf640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f222806ae40 0x7f222806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f222c035000 tx=0x7f222c004730 comp rx=0 tx=0).stop 2023-10-18T10:26:19.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.104+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c141690 msgr2=0x7f223c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.104+0000 7f2241eaf640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c141690 0x7f223c15aa20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f223400c780 tx=0x7f223400cc40 comp rx=0 tx=0).stop 2023-10-18T10:26:19.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.105+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 shutdown_connections 2023-10-18T10:26:19.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.105+0000 7f2241eaf640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f222806ae40 0x7f222806d300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.105+0000 7f2241eaf640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f223c141690 0x7f223c15aa20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.105+0000 7f2241eaf640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f223c13e520 0x7f223c13a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.105+0000 7f2241eaf640 1 --2- 172.21.15.125:0/547153516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f223c06c0a0 0x7f223c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.105+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 >> 172.21.15.125:0/547153516 conn(0x7f223c069930 msgr2=0x7f223c0af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:19.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.105+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 shutdown_connections 2023-10-18T10:26:19.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.105+0000 7f2241eaf640 1 -- 172.21.15.125:0/547153516 wait complete. 2023-10-18T10:26:19.117 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:26:19.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.222+0000 7fc9c2d16640 1 -- 172.21.15.125:0/1286488387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b260 msgr2=0x7fc9bc0cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.222+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/1286488387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b260 0x7fc9bc0cf310 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0895b0 tx=0x7fc9ac02be40 comp rx=0 tx=0).stop 2023-10-18T10:26:19.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.222+0000 7fc9c2d16640 1 -- 172.21.15.125:0/1286488387 shutdown_connections 2023-10-18T10:26:19.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.222+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/1286488387 >> v2:172.21.15.139:3300/0 conn(0x7fc9bc15b640 0x7fc9bc0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.222+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/1286488387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b260 0x7fc9bc0cf310 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.222+0000 7fc9c2d16640 1 -- 172.21.15.125:0/1286488387 >> 172.21.15.125:0/1286488387 conn(0x7fc9bc09a030 msgr2=0x7fc9bc0d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:19.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.222+0000 7fc9c2d16640 1 -- 172.21.15.125:0/1286488387 shutdown_connections 2023-10-18T10:26:19.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.222+0000 7fc9c2d16640 1 -- 172.21.15.125:0/1286488387 wait complete. 2023-10-18T10:26:19.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.223+0000 7fc9c2d16640 1 Processor -- start 2023-10-18T10:26:19.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.223+0000 7fc9c2d16640 1 -- start start 2023-10-18T10:26:19.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c2d16640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9bc0b0b10 0x7fc9bc15e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:19.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c2d16640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9bc15b260 0x7fc9bc151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:19.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c2d16640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b640 0x7fc9bc0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:19.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c2d16640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc15dfb0 con 0x7fc9bc15b640 2023-10-18T10:26:19.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c2d16640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc15d3a0 con 0x7fc9bc0b0b10 2023-10-18T10:26:19.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c2d16640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc15d680 con 0x7fc9bc15b260 2023-10-18T10:26:19.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c12b3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b640 0x7fc9bc0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:19.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c12b3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b640 0x7fc9bc0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43930/0 (socket says 172.21.15.125:43930) 2023-10-18T10:26:19.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c12b3640 1 -- 172.21.15.125:0/3491103318 learned_addr learned my addr 172.21.15.125:0/3491103318 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:19.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c12b3640 1 -- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9bc15b260 msgr2=0x7fc9bc151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c12b3640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9bc15b260 0x7fc9bc151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c12b3640 1 -- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9bc0b0b10 msgr2=0x7fc9bc15e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c12b3640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9bc0b0b10 0x7fc9bc15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.224+0000 7fc9c12b3640 1 -- 172.21.15.125:0/3491103318 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac009560 con 0x7fc9bc15b640 2023-10-18T10:26:19.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.225+0000 7fc9c12b3640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b640 0x7fc9bc0b4670 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c7a0 tx=0x7fc9b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:19.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.225+0000 7fc9b97fa640 1 -- 172.21.15.125:0/3491103318 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4003b70 con 0x7fc9bc15b640 2023-10-18T10:26:19.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.225+0000 7fc9b97fa640 1 -- 172.21.15.125:0/3491103318 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc9b4003cf0 con 0x7fc9bc15b640 2023-10-18T10:26:19.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.225+0000 7fc9b97fa640 1 -- 172.21.15.125:0/3491103318 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400aab0 con 0x7fc9bc15b640 2023-10-18T10:26:19.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.225+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc15d960 con 0x7fc9bc15b640 2023-10-18T10:26:19.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.225+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc0b4bb0 con 0x7fc9bc15b640 2023-10-18T10:26:19.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.226+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc0675f0 con 0x7fc9bc15b640 2023-10-18T10:26:19.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.226+0000 7fc9b97fa640 1 -- 172.21.15.125:0/3491103318 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc9b400ac10 con 0x7fc9bc15b640 2023-10-18T10:26:19.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.226+0000 7fc9b97fa640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc9a806ae40 0x7fc9a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:19.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.227+0000 7fc9b97fa640 1 -- 172.21.15.125:0/3491103318 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc9b4089800 con 0x7fc9bc15b640 2023-10-18T10:26:19.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.227+0000 7fc9c0ab2640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc9a806ae40 0x7fc9a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:19.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.227+0000 7fc9c0ab2640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc9a806ae40 0x7fc9a806d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac035000 tx=0x7fc9ac0046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:19.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.231+0000 7fc9b97fa640 1 -- 172.21.15.125:0/3491103318 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9b40546e0 con 0x7fc9bc15b640 2023-10-18T10:26:19.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.430+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc9bc15e0f0 con 0x7fc9bc15b640 2023-10-18T10:26:19.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.521+0000 7fc9b97fa640 1 -- 172.21.15.125:0/3491103318 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc9b4057d00 con 0x7fc9bc15b640 2023-10-18T10:26:19.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc9a806ae40 msgr2=0x7fc9a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc9a806ae40 0x7fc9a806d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac035000 tx=0x7fc9ac0046c0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b640 msgr2=0x7fc9bc0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b640 0x7fc9bc0b4670 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c7a0 tx=0x7fc9b400cc60 comp rx=0 tx=0).stop 2023-10-18T10:26:19.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 shutdown_connections 2023-10-18T10:26:19.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc9a806ae40 0x7fc9a806d300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9bc15b640 0x7fc9bc0b4670 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9bc15b260 0x7fc9bc151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 --2- 172.21.15.125:0/3491103318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9bc0b0b10 0x7fc9bc15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 >> 172.21.15.125:0/3491103318 conn(0x7fc9bc09a030 msgr2=0x7fc9bc0c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:19.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 shutdown_connections 2023-10-18T10:26:19.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.523+0000 7fc9c2d16640 1 -- 172.21.15.125:0/3491103318 wait complete. 2023-10-18T10:26:19.535 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:26:19.536 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:26:19.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.639+0000 7fc00520e640 1 -- 172.21.15.125:0/1954657001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 msgr2=0x7fc00006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.639+0000 7fc00520e640 1 --2- 172.21.15.125:0/1954657001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 0x7fc00006fa70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005fd0 tx=0x7fbff402bd90 comp rx=0 tx=0).stop 2023-10-18T10:26:19.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.639+0000 7fc00520e640 1 -- 172.21.15.125:0/1954657001 shutdown_connections 2023-10-18T10:26:19.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.639+0000 7fc00520e640 1 --2- 172.21.15.125:0/1954657001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc00006ffb0 0x7fc0000c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.639+0000 7fc00520e640 1 --2- 172.21.15.125:0/1954657001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 0x7fc00006fa70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.639+0000 7fc00520e640 1 -- 172.21.15.125:0/1954657001 >> 172.21.15.125:0/1954657001 conn(0x7fc000069930 msgr2=0x7fc000069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.639+0000 7fc00520e640 1 -- 172.21.15.125:0/1954657001 shutdown_connections 2023-10-18T10:26:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.639+0000 7fc00520e640 1 -- 172.21.15.125:0/1954657001 wait complete. 2023-10-18T10:26:19.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.640+0000 7fc00520e640 1 Processor -- start 2023-10-18T10:26:19.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.640+0000 7fc00520e640 1 -- start start 2023-10-18T10:26:19.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fc00520e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00006ffb0 0x7fc0001595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:19.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fc00520e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 0x7fc0000ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:19.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fc00520e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0000c9bf0 0x7fc0000ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:19.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fc00520e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001478e0 con 0x7fc0000b1920 2023-10-18T10:26:19.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fc00520e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001467f0 con 0x7fc00006ffb0 2023-10-18T10:26:19.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fc00520e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc000147af0 con 0x7fc0000c9bf0 2023-10-18T10:26:19.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fbffe59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 0x7fc0000ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:19.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fbffe59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 0x7fc0000ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43948/0 (socket says 172.21.15.125:43948) 2023-10-18T10:26:19.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fbffe59c640 1 -- 172.21.15.125:0/1095585480 learned_addr learned my addr 172.21.15.125:0/1095585480 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:19.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fbffed9d640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00006ffb0 0x7fc0001595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:19.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.641+0000 7fbfff59e640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0000c9bf0 0x7fc0000ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:19.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbffe59c640 1 -- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0000c9bf0 msgr2=0x7fc0000ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbffe59c640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0000c9bf0 0x7fc0000ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbffe59c640 1 -- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00006ffb0 msgr2=0x7fc0001595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:19.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbffe59c640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00006ffb0 0x7fc0001595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:19.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbffe59c640 1 -- 172.21.15.125:0/1095585480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4009560 con 0x7fc0000b1920 2023-10-18T10:26:19.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbffed9d640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00006ffb0 0x7fc0001595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:19.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbffe59c640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 0x7fc0000ad480 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbff000c970 tx=0x7fbff000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:19.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbfe77fe640 1 -- 172.21.15.125:0/1095585480 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0007bb0 con 0x7fc0000b1920 2023-10-18T10:26:19.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbfe77fe640 1 -- 172.21.15.125:0/1095585480 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbff0007e00 con 0x7fc0000b1920 2023-10-18T10:26:19.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fbfe77fe640 1 -- 172.21.15.125:0/1095585480 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000f6b0 con 0x7fc0000b1920 2023-10-18T10:26:19.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc000147dd0 con 0x7fc0000b1920 2023-10-18T10:26:19.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.642+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc000149a80 con 0x7fc0000b1920 2023-10-18T10:26:19.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.643+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000154960 con 0x7fc0000b1920 2023-10-18T10:26:19.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.643+0000 7fbfe77fe640 1 -- 172.21.15.125:0/1095585480 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbff000f810 con 0x7fc0000b1920 2023-10-18T10:26:19.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.644+0000 7fbfe77fe640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbfec06ae40 0x7fbfec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:19.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.644+0000 7fbfe77fe640 1 -- 172.21.15.125:0/1095585480 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbff0089fc0 con 0x7fc0000b1920 2023-10-18T10:26:19.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.644+0000 7fbffed9d640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbfec06ae40 0x7fbfec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:19.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.644+0000 7fbffed9d640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbfec06ae40 0x7fbfec06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbff4035000 tx=0x7fbff4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:19.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.648+0000 7fbfe77fe640 1 -- 172.21.15.125:0/1095585480 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff00bcca0 con 0x7fc0000b1920 2023-10-18T10:26:19.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:19.848+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc0001482f0 con 0x7fc0000b1920 2023-10-18T10:26:20.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.528+0000 7fbfe77fe640 1 -- 172.21.15.125:0/1095585480 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc0001482f0 con 0x7fc0000b1920 2023-10-18T10:26:20.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbfec06ae40 msgr2=0x7fbfec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbfec06ae40 0x7fbfec06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbff4035000 tx=0x7fbff4004730 comp rx=0 tx=0).stop 2023-10-18T10:26:20.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 msgr2=0x7fc0000ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 0x7fc0000ad480 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbff000c970 tx=0x7fbff000ce30 comp rx=0 tx=0).stop 2023-10-18T10:26:20.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 shutdown_connections 2023-10-18T10:26:20.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbfec06ae40 0x7fbfec06d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0000c9bf0 0x7fc0000ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0000b1920 0x7fc0000ad480 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 --2- 172.21.15.125:0/1095585480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00006ffb0 0x7fc0001595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.530+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 >> 172.21.15.125:0/1095585480 conn(0x7fc000069930 msgr2=0x7fc0000cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:20.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.531+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 shutdown_connections 2023-10-18T10:26:20.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.531+0000 7fc00520e640 1 -- 172.21.15.125:0/1095585480 wait complete. 2023-10-18T10:26:20.544 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:26:20.544 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:26:20.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.648+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/400231631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d00cf340 msgr2=0x7fe3d00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.648+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/400231631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d00cf340 0x7fe3d00d1910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009830 tx=0x7fe3c002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.648+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/400231631 shutdown_connections 2023-10-18T10:26:20.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.648+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/400231631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d00ceb40 0x7fe3d00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.648+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/400231631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d00cf340 0x7fe3d00d1910 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.648+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/400231631 >> 172.21.15.125:0/400231631 conn(0x7fe3d00d0d60 msgr2=0x7fe3d00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:20.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.648+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/400231631 shutdown_connections 2023-10-18T10:26:20.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.648+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/400231631 wait complete. 2023-10-18T10:26:20.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.650+0000 7fe3d5b4c640 1 Processor -- start 2023-10-18T10:26:20.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.650+0000 7fe3d5b4c640 1 -- start start 2023-10-18T10:26:20.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.650+0000 7fe3d5b4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d00ceb40 0x7fe3d00b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:20.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.650+0000 7fe3d5b4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3d00cf340 0x7fe3d01f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:20.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.650+0000 7fe3d5b4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d015b2a0 0x7fe3d01f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:20.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.650+0000 7fe3d5b4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00b4040 con 0x7fe3d00cf340 2023-10-18T10:26:20.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.650+0000 7fe3d5b4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00c9810 con 0x7fe3d00ceb40 2023-10-18T10:26:20.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.650+0000 7fe3d5b4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01f5810 con 0x7fe3d015b2a0 2023-10-18T10:26:20.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3d00cf340 0x7fe3d01f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:20.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3d00cf340 0x7fe3d01f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43958/0 (socket says 172.21.15.125:43958) 2023-10-18T10:26:20.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 -- 172.21.15.125:0/2525941744 learned_addr learned my addr 172.21.15.125:0/2525941744 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:20.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3cffff640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d015b2a0 0x7fe3d01f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:20.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3cf7fe640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d00ceb40 0x7fe3d00b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:20.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 -- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d015b2a0 msgr2=0x7fe3d01f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d015b2a0 0x7fe3d01f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 -- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d00ceb40 msgr2=0x7fe3d00b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d00ceb40 0x7fe3d00b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 -- 172.21.15.125:0/2525941744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0009490 con 0x7fe3d00cf340 2023-10-18T10:26:20.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3ceffd640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3d00cf340 0x7fe3d01f49c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c3c0 tx=0x7fe3c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:20.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3b7fff640 1 -- 172.21.15.125:0/2525941744 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011790 con 0x7fe3d00cf340 2023-10-18T10:26:20.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3b7fff640 1 -- 172.21.15.125:0/2525941744 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3c40119e0 con 0x7fe3d00cf340 2023-10-18T10:26:20.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d01f5530 con 0x7fe3d00cf340 2023-10-18T10:26:20.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3b7fff640 1 -- 172.21.15.125:0/2525941744 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011e10 con 0x7fe3d00cf340 2023-10-18T10:26:20.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.651+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d00b3c60 con 0x7fe3d00cf340 2023-10-18T10:26:20.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.652+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d00690e0 con 0x7fe3d00cf340 2023-10-18T10:26:20.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.653+0000 7fe3b7fff640 1 -- 172.21.15.125:0/2525941744 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe3c401c460 con 0x7fe3d00cf340 2023-10-18T10:26:20.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.653+0000 7fe3b7fff640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3bc06ae40 0x7fe3bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:20.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.653+0000 7fe3b7fff640 1 -- 172.21.15.125:0/2525941744 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe3c4089670 con 0x7fe3d00cf340 2023-10-18T10:26:20.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.653+0000 7fe3cf7fe640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3bc06ae40 0x7fe3bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:20.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.654+0000 7fe3cf7fe640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3bc06ae40 0x7fe3bc06d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0036000 tx=0x7fe3c0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:20.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.657+0000 7fe3b7fff640 1 -- 172.21.15.125:0/2525941744 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3c40546a0 con 0x7fe3d00cf340 2023-10-18T10:26:20.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.856+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3d00b0cd0 con 0x7fe3d00cf340 2023-10-18T10:26:20.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.857+0000 7fe3b7fff640 1 -- 172.21.15.125:0/2525941744 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fe3c4057cc0 con 0x7fe3d00cf340 2023-10-18T10:26:20.859 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:20.859 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3511459076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3511459076},{"type":"v1","addr":"172.21.15.139:6836","nonce":3511459076}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/984169411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":984169411},{"type":"v1","addr":"172.21.15.139:6837","nonce":984169411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2755714237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2755714237},{"type":"v1","addr":"172.21.15.125:6834","nonce":2755714237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:26:17.831071+0000","modified":"2023-10-18T10:26:19.850357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1934222457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1934222457},{"type":"v1","addr":"172.21.15.125:6835","nonce":1934222457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:26:20.860 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 16 2023-10-18T10:26:20.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3bc06ae40 msgr2=0x7fe3bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3bc06ae40 0x7fe3bc06d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0036000 tx=0x7fe3c0004730 comp rx=0 tx=0).stop 2023-10-18T10:26:20.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3d00cf340 msgr2=0x7fe3d01f49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3d00cf340 0x7fe3d01f49c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c3c0 tx=0x7fe3c400c880 comp rx=0 tx=0).stop 2023-10-18T10:26:20.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 shutdown_connections 2023-10-18T10:26:20.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fe3bc06ae40 0x7fe3bc06d300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d015b2a0 0x7fe3d01f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3d00cf340 0x7fe3d01f49c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.858+0000 7fe3d5b4c640 1 --2- 172.21.15.125:0/2525941744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d00ceb40 0x7fe3d00b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.859+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 >> 172.21.15.125:0/2525941744 conn(0x7fe3d00d0d60 msgr2=0x7fe3d00cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:20.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.859+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 shutdown_connections 2023-10-18T10:26:20.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.859+0000 7fe3d5b4c640 1 -- 172.21.15.125:0/2525941744 wait complete. 2023-10-18T10:26:20.871 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:26:20.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.972+0000 7fc27ea61640 1 -- 172.21.15.125:0/1745032957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc27815c8b0 msgr2=0x7fc27815cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.972+0000 7fc27ea61640 1 --2- 172.21.15.125:0/1745032957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc27815c8b0 0x7fc27815cd40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc270005fd0 tx=0x7fc27002c120 comp rx=0 tx=0).stop 2023-10-18T10:26:20.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.972+0000 7fc27ea61640 1 -- 172.21.15.125:0/1745032957 shutdown_connections 2023-10-18T10:26:20.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.972+0000 7fc27ea61640 1 --2- 172.21.15.125:0/1745032957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc27815c8b0 0x7fc27815cd40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.972+0000 7fc27ea61640 1 --2- 172.21.15.125:0/1745032957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2780b12e0 0x7fc2780cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.972+0000 7fc27ea61640 1 -- 172.21.15.125:0/1745032957 >> 172.21.15.125:0/1745032957 conn(0x7fc2780b0670 msgr2=0x7fc2780b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:20.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.972+0000 7fc27ea61640 1 -- 172.21.15.125:0/1745032957 shutdown_connections 2023-10-18T10:26:20.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.972+0000 7fc27ea61640 1 -- 172.21.15.125:0/1745032957 wait complete. 2023-10-18T10:26:20.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.973+0000 7fc27ea61640 1 Processor -- start 2023-10-18T10:26:20.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.973+0000 7fc27ea61640 1 -- start start 2023-10-18T10:26:20.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27ea61640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2780b12e0 0x7fc2781efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:20.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27ea61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc278158e50 0x7fc27815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:20.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27ea61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc27815c8b0 0x7fc27815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:20.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27ea61640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2780b29e0 con 0x7fc2780b12e0 2023-10-18T10:26:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27ea61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2780b48c0 con 0x7fc27815c8b0 2023-10-18T10:26:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27ea61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781f0230 con 0x7fc278158e50 2023-10-18T10:26:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2780b12e0 0x7fc2781efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2780b12e0 0x7fc2781efcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43970/0 (socket says 172.21.15.125:43970) 2023-10-18T10:26:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 -- 172.21.15.125:0/4207631802 learned_addr learned my addr 172.21.15.125:0/4207631802 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc2777fe640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc278158e50 0x7fc27815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27cffe640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc27815c8b0 0x7fc27815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 -- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc278158e50 msgr2=0x7fc27815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc278158e50 0x7fc27815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 -- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc27815c8b0 msgr2=0x7fc27815bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc27815c8b0 0x7fc27815bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 -- 172.21.15.125:0/4207631802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009900 con 0x7fc2780b12e0 2023-10-18T10:26:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc2777fe640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc278158e50 0x7fc27815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27cffe640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc27815c8b0 0x7fc27815bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:26:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc277fff640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2780b12e0 0x7fc2781efcf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c460 tx=0x7fc26800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc274ff9640 1 -- 172.21.15.125:0/4207631802 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268011790 con 0x7fc2780b12e0 2023-10-18T10:26:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc274ff9640 1 -- 172.21.15.125:0/4207631802 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc268011a20 con 0x7fc2780b12e0 2023-10-18T10:26:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc274ff9640 1 -- 172.21.15.125:0/4207631802 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268011e30 con 0x7fc2780b12e0 2023-10-18T10:26:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27815b0b0 con 0x7fc2780b12e0 2023-10-18T10:26:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.974+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2781f2d20 con 0x7fc2780b12e0 2023-10-18T10:26:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.975+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2780cdd10 con 0x7fc2780b12e0 2023-10-18T10:26:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.976+0000 7fc274ff9640 1 -- 172.21.15.125:0/4207631802 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc26801c460 con 0x7fc2780b12e0 2023-10-18T10:26:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.976+0000 7fc274ff9640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc26406ae40 0x7fc26406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.976+0000 7fc274ff9640 1 -- 172.21.15.125:0/4207631802 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc2680898a0 con 0x7fc2780b12e0 2023-10-18T10:26:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.976+0000 7fc2777fe640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc26406ae40 0x7fc26406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.977+0000 7fc2777fe640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc26406ae40 0x7fc26406d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc278157220 tx=0x7fc26c005d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:20.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:20.980+0000 7fc274ff9640 1 -- 172.21.15.125:0/4207631802 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2680548a0 con 0x7fc2780b12e0 2023-10-18T10:26:21.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.146+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2780cc530 con 0x7fc2780b12e0 2023-10-18T10:26:21.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.147+0000 7fc274ff9640 1 -- 172.21.15.125:0/4207631802 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16822 (secure 0 0 0) 0x7fc268057ec0 con 0x7fc2780b12e0 2023-10-18T10:26:21.149 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:21.150 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":39,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:26:19.191532+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:26:16.213313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-18T10:26:16.801795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:26:21.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc26406ae40 msgr2=0x7fc26406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc26406ae40 0x7fc26406d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc278157220 tx=0x7fc26c005d50 comp rx=0 tx=0).stop 2023-10-18T10:26:21.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2780b12e0 msgr2=0x7fc2781efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2780b12e0 0x7fc2781efcf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c460 tx=0x7fc26800c920 comp rx=0 tx=0).stop 2023-10-18T10:26:21.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 shutdown_connections 2023-10-18T10:26:21.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc26406ae40 0x7fc26406d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc27815c8b0 0x7fc27815bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc278158e50 0x7fc27815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 --2- 172.21.15.125:0/4207631802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2780b12e0 0x7fc2781efcf0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 >> 172.21.15.125:0/4207631802 conn(0x7fc2780b0670 msgr2=0x7fc2780cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:21.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 shutdown_connections 2023-10-18T10:26:21.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.149+0000 7fc27ea61640 1 -- 172.21.15.125:0/4207631802 wait complete. 2023-10-18T10:26:21.162 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:26:21.162 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:26:21.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.262+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/2446947248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d3806af00 msgr2=0x7f0d3806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.262+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/2446947248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d3806af00 0x7f0d3806b2e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0d28005fd0 tx=0x7f0d2802bd10 comp rx=0 tx=0).stop 2023-10-18T10:26:21.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.262+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/2446947248 shutdown_connections 2023-10-18T10:26:21.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.262+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/2446947248 >> v2:172.21.15.139:3300/0 conn(0x7f0d380c96b0 0x7f0d380c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.262+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/2446947248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d3806af00 0x7f0d3806b2e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.262+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/2446947248 >> 172.21.15.125:0/2446947248 conn(0x7f0d38069930 msgr2=0x7f0d38069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:21.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.263+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/2446947248 shutdown_connections 2023-10-18T10:26:21.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.263+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/2446947248 wait complete. 2023-10-18T10:26:21.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.264+0000 7f0d3e7ce640 1 Processor -- start 2023-10-18T10:26:21.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.264+0000 7f0d3e7ce640 1 -- start start 2023-10-18T10:26:21.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.264+0000 7f0d3e7ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d3806af00 0x7f0d3813dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.264+0000 7f0d3e7ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d380ab500 0x7f0d381542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.264+0000 7f0d3e7ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d380c96b0 0x7f0d3813a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.264+0000 7f0d3e7ce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38156820 con 0x7f0d380ab500 2023-10-18T10:26:21.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.264+0000 7f0d3e7ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d381547f0 con 0x7f0d380c96b0 2023-10-18T10:26:21.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.264+0000 7f0d3e7ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38154ac0 con 0x7f0d3806af00 2023-10-18T10:26:21.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d380ab500 0x7f0d381542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d380ab500 0x7f0d381542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44000/0 (socket says 172.21.15.125:44000) 2023-10-18T10:26:21.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 -- 172.21.15.125:0/971680332 learned_addr learned my addr 172.21.15.125:0/971680332 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:21.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d3cd6b640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d380c96b0 0x7f0d3813a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d37fff640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d3806af00 0x7f0d3813dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 -- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d3806af00 msgr2=0x7f0d3813dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d3806af00 0x7f0d3813dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 -- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d380c96b0 msgr2=0x7f0d3813a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d380c96b0 0x7f0d3813a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 -- 172.21.15.125:0/971680332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009560 con 0x7f0d380ab500 2023-10-18T10:26:21.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d3cd6b640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d380c96b0 0x7f0d3813a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:21.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d377fe640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d380ab500 0x7f0d381542b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c970 tx=0x7f0d2c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:21.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d34ff9640 1 -- 172.21.15.125:0/971680332 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c007bb0 con 0x7f0d380ab500 2023-10-18T10:26:21.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d34ff9640 1 -- 172.21.15.125:0/971680332 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d2c007e00 con 0x7f0d380ab500 2023-10-18T10:26:21.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d38154da0 con 0x7f0d380ab500 2023-10-18T10:26:21.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d34ff9640 1 -- 172.21.15.125:0/971680332 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c005280 con 0x7f0d380ab500 2023-10-18T10:26:21.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.265+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d38155360 con 0x7f0d380ab500 2023-10-18T10:26:21.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.266+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d380ac320 con 0x7f0d380ab500 2023-10-18T10:26:21.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.267+0000 7f0d34ff9640 1 -- 172.21.15.125:0/971680332 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0d2c0053e0 con 0x7f0d380ab500 2023-10-18T10:26:21.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.267+0000 7f0d34ff9640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0d2406ae40 0x7f0d2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.267+0000 7f0d34ff9640 1 -- 172.21.15.125:0/971680332 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0d2c089c70 con 0x7f0d380ab500 2023-10-18T10:26:21.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.267+0000 7f0d37fff640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0d2406ae40 0x7f0d2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.268+0000 7f0d37fff640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0d2406ae40 0x7f0d2406d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0d28035000 tx=0x7f0d28004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:21.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.271+0000 7f0d34ff9640 1 -- 172.21.15.125:0/971680332 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d2c054ca0 con 0x7f0d380ab500 2023-10-18T10:26:21.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.435+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d38159c50 con 0x7f0d2406ae40 2023-10-18T10:26:21.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.443+0000 7f0d34ff9640 1 -- 172.21.15.125:0/971680332 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277739 (secure 0 0 0) 0x7f0d280055e0 con 0x7f0d2406ae40 2023-10-18T10:26:21.448 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:21.449 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:26:21.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0d2406ae40 msgr2=0x7f0d2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0d2406ae40 0x7f0d2406d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0d28035000 tx=0x7f0d28004730 comp rx=0 tx=0).stop 2023-10-18T10:26:21.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d380ab500 msgr2=0x7f0d381542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d380ab500 0x7f0d381542b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c970 tx=0x7f0d2c00ce30 comp rx=0 tx=0).stop 2023-10-18T10:26:21.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 shutdown_connections 2023-10-18T10:26:21.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0d2406ae40 0x7f0d2406d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d380c96b0 0x7f0d3813a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d380ab500 0x7f0d381542b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 --2- 172.21.15.125:0/971680332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d3806af00 0x7f0d3813dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.446+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 >> 172.21.15.125:0/971680332 conn(0x7f0d38069930 msgr2=0x7f0d3815a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:21.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.447+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 shutdown_connections 2023-10-18T10:26:21.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.447+0000 7f0d3e7ce640 1 -- 172.21.15.125:0/971680332 wait complete. 2023-10-18T10:26:21.459 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-10-18T10:26:20.585476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220512,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4399104,"data_stored":2519936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.100481"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486777+0000","last_change":"2023-10-18T10:26:16.486777+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486777+0000","last_undegraded":"2023-10-18T10:26:16.486777+0000","last_fullsized":"2023-10-18T10:26:16.486777+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488641+0000","last_change":"2023-10-18T10:26:16.488641+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488641+0000","last_undegraded":"2023-10-18T10:26:16.488641+0000","last_fullsized":"2023-10-18T10:26:16.488641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490397+0000","last_change":"2023-10-18T10:26:16.490397+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490397+0000","last_undegraded":"2023-10-18T10:26:16.490397+0000","last_fullsized":"2023-10-18T10:26:16.490397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490798+0000","last_change":"2023-10-18T10:26:16.490798+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490798+0000","last_undegraded":"2023-10-18T10:26:16.490798+0000","last_fullsized":"2023-10-18T10:26:16.490798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490228+0000","last_change":"2023-10-18T10:26:16.490228+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490228+0000","last_undegraded":"2023-10-18T10:26:16.490228+0000","last_fullsized":"2023-10-18T10:26:16.490228+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490752+0000","last_change":"2023-10-18T10:26:16.490752+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490752+0000","last_undegraded":"2023-10-18T10:26:16.490752+0000","last_fullsized":"2023-10-18T10:26:16.490752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490407+0000","last_change":"2023-10-18T10:26:16.490407+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490407+0000","last_undegraded":"2023-10-18T10:26:16.490407+0000","last_fullsized":"2023-10-18T10:26:16.490407+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490582+0000","last_change":"2023-10-18T10:26:16.490582+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490582+0000","last_undegraded":"2023-10-18T10:26:16.490582+0000","last_fullsized":"2023-10-18T10:26:16.490582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493329+0000","last_change":"2023-10-18T10:26:16.493329+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493329+0000","last_undegraded":"2023-10-18T10:26:16.493329+0000","last_fullsized":"2023-10-18T10:26:16.493329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490790+0000","last_change":"2023-10-18T10:26:16.490790+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490790+0000","last_undegraded":"2023-10-18T10:26:16.490790+0000","last_fullsized":"2023-10-18T10:26:16.490790+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490410+0000","last_change":"2023-10-18T10:26:16.490410+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490410+0000","last_undegraded":"2023-10-18T10:26:16.490410+0000","last_fullsized":"2023-10-18T10:26:16.490410+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493298+0000","last_change":"2023-10-18T10:26:16.493298+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493298+0000","last_undegraded":"2023-10-18T10:26:16.493298+0000","last_fullsized":"2023-10-18T10:26:16.493298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493307+0000","last_change":"2023-10-18T10:26:16.493307+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493307+0000","last_undegraded":"2023-10-18T10:26:16.493307+0000","last_fullsized":"2023-10-18T10:26:16.493307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490407+0000","last_change":"2023-10-18T10:26:16.490407+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490407+0000","last_undegraded":"2023-10-18T10:26:16.490407+0000","last_fullsized":"2023-10-18T10:26:16.490407+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486718+0000","last_change":"2023-10-18T10:26:16.486718+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486718+0000","last_undegraded":"2023-10-18T10:26:16.486718+0000","last_fullsized":"2023-10-18T10:26:16.486718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490886+0000","last_change":"2023-10-18T10:26:16.490886+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490886+0000","last_undegraded":"2023-10-18T10:26:16.490886+0000","last_fullsized":"2023-10-18T10:26:16.490886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.491320+0000","last_change":"2023-10-18T10:26:16.491320+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.491320+0000","last_undegraded":"2023-10-18T10:26:16.491320+0000","last_fullsized":"2023-10-18T10:26:16.491320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490204+0000","last_change":"2023-10-18T10:26:16.490204+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490204+0000","last_undegraded":"2023-10-18T10:26:16.490204+0000","last_fullsized":"2023-10-18T10:26:16.490204+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487893+0000","last_change":"2023-10-18T10:26:16.487893+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487893+0000","last_undegraded":"2023-10-18T10:26:16.487893+0000","last_fullsized":"2023-10-18T10:26:16.487893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490418+0000","last_change":"2023-10-18T10:26:16.490418+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490418+0000","last_undegraded":"2023-10-18T10:26:16.490418+0000","last_fullsized":"2023-10-18T10:26:16.490418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487567+0000","last_change":"2023-10-18T10:26:16.487567+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487567+0000","last_undegraded":"2023-10-18T10:26:16.487567+0000","last_fullsized":"2023-10-18T10:26:16.487567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487423+0000","last_change":"2023-10-18T10:26:16.487423+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487423+0000","last_undegraded":"2023-10-18T10:26:16.487423+0000","last_fullsized":"2023-10-18T10:26:16.487423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490274+0000","last_change":"2023-10-18T10:26:16.490274+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490274+0000","last_undegraded":"2023-10-18T10:26:16.490274+0000","last_fullsized":"2023-10-18T10:26:16.490274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487464+0000","last_change":"2023-10-18T10:26:16.487464+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487464+0000","last_undegraded":"2023-10-18T10:26:16.487464+0000","last_fullsized":"2023-10-18T10:26:16.487464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490900+0000","last_change":"2023-10-18T10:26:16.490900+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490900+0000","last_undegraded":"2023-10-18T10:26:16.490900+0000","last_fullsized":"2023-10-18T10:26:16.490900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487428+0000","last_change":"2023-10-18T10:26:16.487428+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487428+0000","last_undegraded":"2023-10-18T10:26:16.487428+0000","last_fullsized":"2023-10-18T10:26:16.487428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488653+0000","last_change":"2023-10-18T10:26:16.488653+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488653+0000","last_undegraded":"2023-10-18T10:26:16.488653+0000","last_fullsized":"2023-10-18T10:26:16.488653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488668+0000","last_change":"2023-10-18T10:26:16.488668+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488668+0000","last_undegraded":"2023-10-18T10:26:16.488668+0000","last_fullsized":"2023-10-18T10:26:16.488668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486716+0000","last_change":"2023-10-18T10:26:16.486716+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486716+0000","last_undegraded":"2023-10-18T10:26:16.486716+0000","last_fullsized":"2023-10-18T10:26:16.486716+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490101+0000","last_change":"2023-10-18T10:26:16.490101+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490101+0000","last_undegraded":"2023-10-18T10:26:16.490101+0000","last_fullsized":"2023-10-18T10:26:16.490101+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493277+0000","last_change":"2023-10-18T10:26:16.493277+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493277+0000","last_undegraded":"2023-10-18T10:26:16.493277+0000","last_fullsized":"2023-10-18T10:26:16.493277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.491169+0000","last_change":"2023-10-18T10:26:16.491169+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.491169+0000","last_undegraded":"2023-10-18T10:26:16.491169+0000","last_fullsized":"2023-10-18T10:26:16.491169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-18T10:26:12.781678+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:26:12.781678+0000","last_peered":"2023-10-18T10:26:12.781678+0000","last_clean":"2023-10-18T10:26:12.781678+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:26:12.781678+0000","last_undegraded":"2023-10-18T10:26:12.781678+0000","last_fullsized":"2023-10-18T10:26:12.781678+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":995328,"data_stored":757768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.435},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.38200000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59099999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.629}]},{"osd":5,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.5},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.41299999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.62}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.438}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.496},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.44900000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.434}]},{"osd":3,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.75}]},{"osd":6,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.73799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27700000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.29599999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.375}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52600000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.62},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.254},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.316},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.627}]},{"osd":5,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.44}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.43099999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.78700000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.72499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":995328,"data_stored":757768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.32700000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.254},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.24299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.35699999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.374},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:26:21.465 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:26:21.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.601+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/3123285492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 msgr2=0x7f6ac00cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.601+0000 7f6ac5d2c640 1 --2- 172.21.15.125:0/3123285492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 0x7f6ac00cf310 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009830 tx=0x7f6ab002bd60 comp rx=0 tx=0).stop 2023-10-18T10:26:21.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.601+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/3123285492 shutdown_connections 2023-10-18T10:26:21.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.601+0000 7f6ac5d2c640 1 --2- 172.21.15.125:0/3123285492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 0x7f6ac00cf310 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.601+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/3123285492 >> 172.21.15.125:0/3123285492 conn(0x7f6ac009a030 msgr2=0x7f6ac00d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:21.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.601+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/3123285492 shutdown_connections 2023-10-18T10:26:21.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.601+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/3123285492 wait complete. 2023-10-18T10:26:21.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.602+0000 7f6ac5d2c640 1 Processor -- start 2023-10-18T10:26:21.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.602+0000 7f6ac5d2c640 1 -- start start 2023-10-18T10:26:21.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6ac5d2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ac00cf850 0x7f6ac0156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6ac5d2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 0x7f6ac015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6ac5d2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015b640 0x7f6ac00b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6ac5d2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac015e450 con 0x7f6ac015b260 2023-10-18T10:26:21.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6ac5d2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac015c980 con 0x7f6ac015b640 2023-10-18T10:26:21.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6ac5d2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac0156870 con 0x7f6ac00cf850 2023-10-18T10:26:21.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6abeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 0x7f6ac015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6abeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 0x7f6ac015ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44004/0 (socket says 172.21.15.125:44004) 2023-10-18T10:26:21.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6abeffd640 1 -- 172.21.15.125:0/196019734 learned_addr learned my addr 172.21.15.125:0/196019734 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:21.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6abf7fe640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ac00cf850 0x7f6ac0156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.603+0000 7f6abffff640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015b640 0x7f6ac00b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6abeffd640 1 -- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ac00cf850 msgr2=0x7f6ac0156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6abeffd640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ac00cf850 0x7f6ac0156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6abeffd640 1 -- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015b640 msgr2=0x7f6ac00b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6abeffd640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015b640 0x7f6ac00b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6abeffd640 1 -- 172.21.15.125:0/196019734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab0009490 con 0x7f6ac015b260 2023-10-18T10:26:21.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6abeffd640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 0x7f6ac015ce10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400b3f0 tx=0x7f6ab400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:21.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6aa7fff640 1 -- 172.21.15.125:0/196019734 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4011830 con 0x7f6ac015b260 2023-10-18T10:26:21.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6aa7fff640 1 -- 172.21.15.125:0/196019734 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ab4011a80 con 0x7f6ac015b260 2023-10-18T10:26:21.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac015e6a0 con 0x7f6ac015b260 2023-10-18T10:26:21.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6aa7fff640 1 -- 172.21.15.125:0/196019734 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4011e90 con 0x7f6ac015b260 2023-10-18T10:26:21.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.604+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac015d350 con 0x7f6ac015b260 2023-10-18T10:26:21.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.605+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac015c440 con 0x7f6ac015b260 2023-10-18T10:26:21.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.605+0000 7f6aa7fff640 1 -- 172.21.15.125:0/196019734 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6ab401a400 con 0x7f6ac015b260 2023-10-18T10:26:21.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.606+0000 7f6aa7fff640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6aac06ae40 0x7f6aac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.606+0000 7f6aa7fff640 1 -- 172.21.15.125:0/196019734 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ab40895c0 con 0x7f6ac015b260 2023-10-18T10:26:21.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.606+0000 7f6abf7fe640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6aac06ae40 0x7f6aac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.606+0000 7f6abf7fe640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6aac06ae40 0x7f6aac06d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0036000 tx=0x7f6ab0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:21.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.610+0000 7f6aa7fff640 1 -- 172.21.15.125:0/196019734 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ab4054670 con 0x7f6ac015b260 2023-10-18T10:26:21.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.773+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ac00cecd0 con 0x7f6aac06ae40 2023-10-18T10:26:21.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.781+0000 7f6aa7fff640 1 -- 172.21.15.125:0/196019734 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277739 (secure 0 0 0) 0x7f6ac00cecd0 con 0x7f6aac06ae40 2023-10-18T10:26:21.786 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:21.787 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:26:21.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6aac06ae40 msgr2=0x7f6aac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6aac06ae40 0x7f6aac06d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0036000 tx=0x7f6ab0004730 comp rx=0 tx=0).stop 2023-10-18T10:26:21.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 msgr2=0x7f6ac015ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 0x7f6ac015ce10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400b3f0 tx=0x7f6ab400b8b0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 shutdown_connections 2023-10-18T10:26:21.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6aac06ae40 0x7f6aac06d300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015b640 0x7f6ac00b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ac015b260 0x7f6ac015ce10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 --2- 172.21.15.125:0/196019734 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ac00cf850 0x7f6ac0156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 >> 172.21.15.125:0/196019734 conn(0x7f6ac009a030 msgr2=0x7f6ac0159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:21.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 shutdown_connections 2023-10-18T10:26:21.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.784+0000 7f6ac5d2c640 1 -- 172.21.15.125:0/196019734 wait complete. 2023-10-18T10:26:21.796 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-10-18T10:26:20.585476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220512,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4399104,"data_stored":2519936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.100481"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486777+0000","last_change":"2023-10-18T10:26:16.486777+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486777+0000","last_undegraded":"2023-10-18T10:26:16.486777+0000","last_fullsized":"2023-10-18T10:26:16.486777+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488641+0000","last_change":"2023-10-18T10:26:16.488641+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488641+0000","last_undegraded":"2023-10-18T10:26:16.488641+0000","last_fullsized":"2023-10-18T10:26:16.488641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490397+0000","last_change":"2023-10-18T10:26:16.490397+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490397+0000","last_undegraded":"2023-10-18T10:26:16.490397+0000","last_fullsized":"2023-10-18T10:26:16.490397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490798+0000","last_change":"2023-10-18T10:26:16.490798+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490798+0000","last_undegraded":"2023-10-18T10:26:16.490798+0000","last_fullsized":"2023-10-18T10:26:16.490798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490228+0000","last_change":"2023-10-18T10:26:16.490228+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490228+0000","last_undegraded":"2023-10-18T10:26:16.490228+0000","last_fullsized":"2023-10-18T10:26:16.490228+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490752+0000","last_change":"2023-10-18T10:26:16.490752+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490752+0000","last_undegraded":"2023-10-18T10:26:16.490752+0000","last_fullsized":"2023-10-18T10:26:16.490752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490407+0000","last_change":"2023-10-18T10:26:16.490407+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490407+0000","last_undegraded":"2023-10-18T10:26:16.490407+0000","last_fullsized":"2023-10-18T10:26:16.490407+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490582+0000","last_change":"2023-10-18T10:26:16.490582+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490582+0000","last_undegraded":"2023-10-18T10:26:16.490582+0000","last_fullsized":"2023-10-18T10:26:16.490582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493329+0000","last_change":"2023-10-18T10:26:16.493329+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493329+0000","last_undegraded":"2023-10-18T10:26:16.493329+0000","last_fullsized":"2023-10-18T10:26:16.493329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490790+0000","last_change":"2023-10-18T10:26:16.490790+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490790+0000","last_undegraded":"2023-10-18T10:26:16.490790+0000","last_fullsized":"2023-10-18T10:26:16.490790+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490410+0000","last_change":"2023-10-18T10:26:16.490410+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490410+0000","last_undegraded":"2023-10-18T10:26:16.490410+0000","last_fullsized":"2023-10-18T10:26:16.490410+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493298+0000","last_change":"2023-10-18T10:26:16.493298+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493298+0000","last_undegraded":"2023-10-18T10:26:16.493298+0000","last_fullsized":"2023-10-18T10:26:16.493298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493307+0000","last_change":"2023-10-18T10:26:16.493307+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493307+0000","last_undegraded":"2023-10-18T10:26:16.493307+0000","last_fullsized":"2023-10-18T10:26:16.493307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490407+0000","last_change":"2023-10-18T10:26:16.490407+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490407+0000","last_undegraded":"2023-10-18T10:26:16.490407+0000","last_fullsized":"2023-10-18T10:26:16.490407+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486718+0000","last_change":"2023-10-18T10:26:16.486718+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486718+0000","last_undegraded":"2023-10-18T10:26:16.486718+0000","last_fullsized":"2023-10-18T10:26:16.486718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490886+0000","last_change":"2023-10-18T10:26:16.490886+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490886+0000","last_undegraded":"2023-10-18T10:26:16.490886+0000","last_fullsized":"2023-10-18T10:26:16.490886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.491320+0000","last_change":"2023-10-18T10:26:16.491320+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.491320+0000","last_undegraded":"2023-10-18T10:26:16.491320+0000","last_fullsized":"2023-10-18T10:26:16.491320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490204+0000","last_change":"2023-10-18T10:26:16.490204+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490204+0000","last_undegraded":"2023-10-18T10:26:16.490204+0000","last_fullsized":"2023-10-18T10:26:16.490204+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487893+0000","last_change":"2023-10-18T10:26:16.487893+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487893+0000","last_undegraded":"2023-10-18T10:26:16.487893+0000","last_fullsized":"2023-10-18T10:26:16.487893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490418+0000","last_change":"2023-10-18T10:26:16.490418+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490418+0000","last_undegraded":"2023-10-18T10:26:16.490418+0000","last_fullsized":"2023-10-18T10:26:16.490418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487567+0000","last_change":"2023-10-18T10:26:16.487567+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487567+0000","last_undegraded":"2023-10-18T10:26:16.487567+0000","last_fullsized":"2023-10-18T10:26:16.487567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487423+0000","last_change":"2023-10-18T10:26:16.487423+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487423+0000","last_undegraded":"2023-10-18T10:26:16.487423+0000","last_fullsized":"2023-10-18T10:26:16.487423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490274+0000","last_change":"2023-10-18T10:26:16.490274+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490274+0000","last_undegraded":"2023-10-18T10:26:16.490274+0000","last_fullsized":"2023-10-18T10:26:16.490274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487464+0000","last_change":"2023-10-18T10:26:16.487464+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487464+0000","last_undegraded":"2023-10-18T10:26:16.487464+0000","last_fullsized":"2023-10-18T10:26:16.487464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490900+0000","last_change":"2023-10-18T10:26:16.490900+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490900+0000","last_undegraded":"2023-10-18T10:26:16.490900+0000","last_fullsized":"2023-10-18T10:26:16.490900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487428+0000","last_change":"2023-10-18T10:26:16.487428+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487428+0000","last_undegraded":"2023-10-18T10:26:16.487428+0000","last_fullsized":"2023-10-18T10:26:16.487428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488653+0000","last_change":"2023-10-18T10:26:16.488653+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488653+0000","last_undegraded":"2023-10-18T10:26:16.488653+0000","last_fullsized":"2023-10-18T10:26:16.488653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488668+0000","last_change":"2023-10-18T10:26:16.488668+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488668+0000","last_undegraded":"2023-10-18T10:26:16.488668+0000","last_fullsized":"2023-10-18T10:26:16.488668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486716+0000","last_change":"2023-10-18T10:26:16.486716+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486716+0000","last_undegraded":"2023-10-18T10:26:16.486716+0000","last_fullsized":"2023-10-18T10:26:16.486716+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490101+0000","last_change":"2023-10-18T10:26:16.490101+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490101+0000","last_undegraded":"2023-10-18T10:26:16.490101+0000","last_fullsized":"2023-10-18T10:26:16.490101+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493277+0000","last_change":"2023-10-18T10:26:16.493277+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493277+0000","last_undegraded":"2023-10-18T10:26:16.493277+0000","last_fullsized":"2023-10-18T10:26:16.493277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.491169+0000","last_change":"2023-10-18T10:26:16.491169+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.491169+0000","last_undegraded":"2023-10-18T10:26:16.491169+0000","last_fullsized":"2023-10-18T10:26:16.491169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-18T10:26:12.781678+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:26:12.781678+0000","last_peered":"2023-10-18T10:26:12.781678+0000","last_clean":"2023-10-18T10:26:12.781678+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:26:12.781678+0000","last_undegraded":"2023-10-18T10:26:12.781678+0000","last_fullsized":"2023-10-18T10:26:12.781678+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":995328,"data_stored":757768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.435},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.38200000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59099999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.629}]},{"osd":5,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.5},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.41299999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.62}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.438}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.496},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.44900000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.434}]},{"osd":3,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.75}]},{"osd":6,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.73799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27700000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.29599999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.375}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52600000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.62},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.254},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.316},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.627}]},{"osd":5,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.44}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.43099999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.78700000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.72499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":995328,"data_stored":757768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.32700000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.254},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.24299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.35699999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.374},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:26:21.802 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:26:21.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.937+0000 7f47cbccf640 1 -- 172.21.15.125:0/1694752625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b0005180 msgr2=0x7f47b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.937+0000 7f47cbccf640 1 --2- 172.21.15.125:0/1694752625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b0005180 0x7f47b000b980 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009ed0 tx=0x7f47b00057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.937+0000 7f47cbccf640 1 -- 172.21.15.125:0/1694752625 shutdown_connections 2023-10-18T10:26:21.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.937+0000 7f47cbccf640 1 --2- 172.21.15.125:0/1694752625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b0005180 0x7f47b000b980 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.937+0000 7f47cbccf640 1 -- 172.21.15.125:0/1694752625 >> 172.21.15.125:0/1694752625 conn(0x7f47c4069930 msgr2=0x7f47c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:21.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.937+0000 7f47cbccf640 1 -- 172.21.15.125:0/1694752625 shutdown_connections 2023-10-18T10:26:21.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.937+0000 7f47cbccf640 1 -- 172.21.15.125:0/1694752625 wait complete. 2023-10-18T10:26:21.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.938+0000 7f47cbccf640 1 Processor -- start 2023-10-18T10:26:21.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.938+0000 7f47cbccf640 1 -- start start 2023-10-18T10:26:21.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47cbccf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b0005180 0x7f47c40af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47cbccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c4154690 0x7f47c4151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47cbccf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47c4156130 0x7f47c413a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47cbccf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4154b10 con 0x7f47c4156130 2023-10-18T10:26:21.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47cbccf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40afb50 con 0x7f47c4154690 2023-10-18T10:26:21.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47cbccf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40afdf0 con 0x7f47b0005180 2023-10-18T10:26:21.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47c4156130 0x7f47c413a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47c4156130 0x7f47c413a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44026/0 (socket says 172.21.15.125:44026) 2023-10-18T10:26:21.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 -- 172.21.15.125:0/4126751835 learned_addr learned my addr 172.21.15.125:0/4126751835 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:21.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47c9a6b640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b0005180 0x7f47c40af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47c926a640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c4154690 0x7f47c4151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 -- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b0005180 msgr2=0x7f47c40af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b0005180 0x7f47c40af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 -- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c4154690 msgr2=0x7f47c4151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:21.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c4154690 0x7f47c4151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:21.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 -- 172.21.15.125:0/4126751835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40b00d0 con 0x7f47c4156130 2023-10-18T10:26:21.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47c926a640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c4154690 0x7f47c4151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:21.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47c9a6b640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b0005180 0x7f47c40af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:21.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.939+0000 7f47ca26c640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47c4156130 0x7f47c413a6a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009ed0 tx=0x7f47bc003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:21.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.940+0000 7f47ba7fc640 1 -- 172.21.15.125:0/4126751835 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc03e070 con 0x7f47c4156130 2023-10-18T10:26:21.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.940+0000 7f47ba7fc640 1 -- 172.21.15.125:0/4126751835 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47bc0042b0 con 0x7f47c4156130 2023-10-18T10:26:21.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.940+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c40b03b0 con 0x7f47c4156130 2023-10-18T10:26:21.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.940+0000 7f47ba7fc640 1 -- 172.21.15.125:0/4126751835 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc004660 con 0x7f47c4156130 2023-10-18T10:26:21.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.940+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c4152040 con 0x7f47c4156130 2023-10-18T10:26:21.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.941+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c40b05f0 con 0x7f47c4156130 2023-10-18T10:26:21.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.941+0000 7f47ba7fc640 1 -- 172.21.15.125:0/4126751835 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47bc0304c0 con 0x7f47c4156130 2023-10-18T10:26:21.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.941+0000 7f47ba7fc640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f47a806ae40 0x7f47a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:21.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.942+0000 7f47ba7fc640 1 -- 172.21.15.125:0/4126751835 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f47bc0ab520 con 0x7f47c4156130 2023-10-18T10:26:21.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.942+0000 7f47c9a6b640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f47a806ae40 0x7f47a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:21.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.942+0000 7f47c9a6b640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f47a806ae40 0x7f47a806d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f47c41f2540 tx=0x7f47c0006b00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:21.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:21.946+0000 7f47ba7fc640 1 -- 172.21.15.125:0/4126751835 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47c40b05f0 con 0x7f47c4156130 2023-10-18T10:26:22.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.106+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47c4152320 con 0x7f47a806ae40 2023-10-18T10:26:22.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.113+0000 7f47ba7fc640 1 -- 172.21.15.125:0/4126751835 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277739 (secure 0 0 0) 0x7f47c0012500 con 0x7f47a806ae40 2023-10-18T10:26:22.118 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:22.119 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:26:22.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f47a806ae40 msgr2=0x7f47a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f47a806ae40 0x7f47a806d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f47c41f2540 tx=0x7f47c0006b00 comp rx=0 tx=0).stop 2023-10-18T10:26:22.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47c4156130 msgr2=0x7f47c413a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47c4156130 0x7f47c413a6a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009ed0 tx=0x7f47bc003b80 comp rx=0 tx=0).stop 2023-10-18T10:26:22.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 shutdown_connections 2023-10-18T10:26:22.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f47a806ae40 0x7f47a806d300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c4154690 0x7f47c4151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47c4156130 0x7f47c413a6a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 --2- 172.21.15.125:0/4126751835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b0005180 0x7f47c40af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 >> 172.21.15.125:0/4126751835 conn(0x7f47c4069930 msgr2=0x7f47c415b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:22.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 shutdown_connections 2023-10-18T10:26:22.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.116+0000 7f47cbccf640 1 -- 172.21.15.125:0/4126751835 wait complete. 2023-10-18T10:26:22.128 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-10-18T10:26:20.585476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220512,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4399104,"data_stored":2519936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.100481"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486777+0000","last_change":"2023-10-18T10:26:16.486777+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486777+0000","last_undegraded":"2023-10-18T10:26:16.486777+0000","last_fullsized":"2023-10-18T10:26:16.486777+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488641+0000","last_change":"2023-10-18T10:26:16.488641+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488641+0000","last_undegraded":"2023-10-18T10:26:16.488641+0000","last_fullsized":"2023-10-18T10:26:16.488641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490397+0000","last_change":"2023-10-18T10:26:16.490397+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490397+0000","last_undegraded":"2023-10-18T10:26:16.490397+0000","last_fullsized":"2023-10-18T10:26:16.490397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490798+0000","last_change":"2023-10-18T10:26:16.490798+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490798+0000","last_undegraded":"2023-10-18T10:26:16.490798+0000","last_fullsized":"2023-10-18T10:26:16.490798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490228+0000","last_change":"2023-10-18T10:26:16.490228+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490228+0000","last_undegraded":"2023-10-18T10:26:16.490228+0000","last_fullsized":"2023-10-18T10:26:16.490228+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490752+0000","last_change":"2023-10-18T10:26:16.490752+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490752+0000","last_undegraded":"2023-10-18T10:26:16.490752+0000","last_fullsized":"2023-10-18T10:26:16.490752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490407+0000","last_change":"2023-10-18T10:26:16.490407+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490407+0000","last_undegraded":"2023-10-18T10:26:16.490407+0000","last_fullsized":"2023-10-18T10:26:16.490407+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490582+0000","last_change":"2023-10-18T10:26:16.490582+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490582+0000","last_undegraded":"2023-10-18T10:26:16.490582+0000","last_fullsized":"2023-10-18T10:26:16.490582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493329+0000","last_change":"2023-10-18T10:26:16.493329+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493329+0000","last_undegraded":"2023-10-18T10:26:16.493329+0000","last_fullsized":"2023-10-18T10:26:16.493329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490790+0000","last_change":"2023-10-18T10:26:16.490790+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490790+0000","last_undegraded":"2023-10-18T10:26:16.490790+0000","last_fullsized":"2023-10-18T10:26:16.490790+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490410+0000","last_change":"2023-10-18T10:26:16.490410+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490410+0000","last_undegraded":"2023-10-18T10:26:16.490410+0000","last_fullsized":"2023-10-18T10:26:16.490410+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493298+0000","last_change":"2023-10-18T10:26:16.493298+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493298+0000","last_undegraded":"2023-10-18T10:26:16.493298+0000","last_fullsized":"2023-10-18T10:26:16.493298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493307+0000","last_change":"2023-10-18T10:26:16.493307+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493307+0000","last_undegraded":"2023-10-18T10:26:16.493307+0000","last_fullsized":"2023-10-18T10:26:16.493307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490407+0000","last_change":"2023-10-18T10:26:16.490407+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490407+0000","last_undegraded":"2023-10-18T10:26:16.490407+0000","last_fullsized":"2023-10-18T10:26:16.490407+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486718+0000","last_change":"2023-10-18T10:26:16.486718+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486718+0000","last_undegraded":"2023-10-18T10:26:16.486718+0000","last_fullsized":"2023-10-18T10:26:16.486718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490886+0000","last_change":"2023-10-18T10:26:16.490886+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490886+0000","last_undegraded":"2023-10-18T10:26:16.490886+0000","last_fullsized":"2023-10-18T10:26:16.490886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.491320+0000","last_change":"2023-10-18T10:26:16.491320+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.491320+0000","last_undegraded":"2023-10-18T10:26:16.491320+0000","last_fullsized":"2023-10-18T10:26:16.491320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490204+0000","last_change":"2023-10-18T10:26:16.490204+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490204+0000","last_undegraded":"2023-10-18T10:26:16.490204+0000","last_fullsized":"2023-10-18T10:26:16.490204+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487893+0000","last_change":"2023-10-18T10:26:16.487893+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487893+0000","last_undegraded":"2023-10-18T10:26:16.487893+0000","last_fullsized":"2023-10-18T10:26:16.487893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490418+0000","last_change":"2023-10-18T10:26:16.490418+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490418+0000","last_undegraded":"2023-10-18T10:26:16.490418+0000","last_fullsized":"2023-10-18T10:26:16.490418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487567+0000","last_change":"2023-10-18T10:26:16.487567+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487567+0000","last_undegraded":"2023-10-18T10:26:16.487567+0000","last_fullsized":"2023-10-18T10:26:16.487567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487423+0000","last_change":"2023-10-18T10:26:16.487423+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487423+0000","last_undegraded":"2023-10-18T10:26:16.487423+0000","last_fullsized":"2023-10-18T10:26:16.487423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490274+0000","last_change":"2023-10-18T10:26:16.490274+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490274+0000","last_undegraded":"2023-10-18T10:26:16.490274+0000","last_fullsized":"2023-10-18T10:26:16.490274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487464+0000","last_change":"2023-10-18T10:26:16.487464+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487464+0000","last_undegraded":"2023-10-18T10:26:16.487464+0000","last_fullsized":"2023-10-18T10:26:16.487464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490900+0000","last_change":"2023-10-18T10:26:16.490900+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490900+0000","last_undegraded":"2023-10-18T10:26:16.490900+0000","last_fullsized":"2023-10-18T10:26:16.490900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.487428+0000","last_change":"2023-10-18T10:26:16.487428+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.487428+0000","last_undegraded":"2023-10-18T10:26:16.487428+0000","last_fullsized":"2023-10-18T10:26:16.487428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488653+0000","last_change":"2023-10-18T10:26:16.488653+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488653+0000","last_undegraded":"2023-10-18T10:26:16.488653+0000","last_fullsized":"2023-10-18T10:26:16.488653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.488668+0000","last_change":"2023-10-18T10:26:16.488668+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.488668+0000","last_undegraded":"2023-10-18T10:26:16.488668+0000","last_fullsized":"2023-10-18T10:26:16.488668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.486716+0000","last_change":"2023-10-18T10:26:16.486716+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.486716+0000","last_undegraded":"2023-10-18T10:26:16.486716+0000","last_fullsized":"2023-10-18T10:26:16.486716+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.490101+0000","last_change":"2023-10-18T10:26:16.490101+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.490101+0000","last_undegraded":"2023-10-18T10:26:16.490101+0000","last_fullsized":"2023-10-18T10:26:16.490101+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.493277+0000","last_change":"2023-10-18T10:26:16.493277+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.493277+0000","last_undegraded":"2023-10-18T10:26:16.493277+0000","last_fullsized":"2023-10-18T10:26:16.493277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:26:17.483887+0000","last_change":"2023-10-18T10:26:17.483887+0000","last_active":"2023-10-18T10:26:17.483887+0000","last_peered":"2023-10-18T10:26:17.483887+0000","last_clean":"2023-10-18T10:26:17.483887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:17.483887+0000","last_undegraded":"2023-10-18T10:26:17.483887+0000","last_fullsized":"2023-10-18T10:26:17.483887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-10-18T10:26:16.491169+0000","last_change":"2023-10-18T10:26:16.491169+0000","last_active":"2023-10-18T10:26:16.482091+0000","last_peered":"2023-10-18T10:26:16.482091+0000","last_clean":"2023-10-18T10:26:16.482091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:26:16.491169+0000","last_undegraded":"2023-10-18T10:26:16.491169+0000","last_fullsized":"2023-10-18T10:26:16.491169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-18T10:26:12.781678+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:26:12.781678+0000","last_peered":"2023-10-18T10:26:12.781678+0000","last_clean":"2023-10-18T10:26:12.781678+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:26:12.781678+0000","last_undegraded":"2023-10-18T10:26:12.781678+0000","last_fullsized":"2023-10-18T10:26:12.781678+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":995328,"data_stored":757768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.435},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.38200000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59099999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.629}]},{"osd":5,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.5},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.41299999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.62}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.438}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.496},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.44900000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.56200000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.434}]},{"osd":3,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.75}]},{"osd":6,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.73799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27700000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.29599999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.375}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52600000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.62},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.254},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.316},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.627}]},{"osd":5,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":167400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.44}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.43099999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.78700000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.72499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":995328,"data_stored":757768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.32700000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.254},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.24299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.35699999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.374},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:26:22.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2023-10-18T10:26:22.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2023-10-18T10:26:22.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-10-18T10:26:22.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2023-10-18T10:26:22.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2023-10-18T10:26:22.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-10-18T10:26:22.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-10-18T10:26:22.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2023-10-18T10:26:22.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-10-18T10:26:22.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-10-18T10:26:22.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-10-18T10:26:22.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-10-18T10:26:22.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-10-18T10:26:22.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-10-18T10:26:22.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-10-18T10:26:22.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2023-10-18T10:26:22.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-10-18T10:26:22.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2023-10-18T10:26:22.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2023-10-18T10:26:22.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2023-10-18T10:26:22.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-10-18T10:26:22.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-10-18T10:26:22.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-10-18T10:26:22.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2023-10-18T10:26:22.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2023-10-18T10:26:22.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-10-18T10:26:22.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2023-10-18T10:26:22.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-10-18T10:26:22.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2023-10-18T10:26:22.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2023-10-18T10:26:22.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2023-10-18T10:26:22.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-10-18T10:26:22.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-10-18T10:26:22.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-10-18T10:26:22.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.486777+0000', 'last_change': '2023-10-18T10:26:16.486777+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.486777+0000', 'last_undegraded': '2023-10-18T10:26:16.486777+0000', 'last_fullsized': '2023-10-18T10:26:16.486777+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T10:26:22.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-10-18T10:26:22.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-10-18T10:26:22.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.488641+0000', 'last_change': '2023-10-18T10:26:16.488641+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.488641+0000', 'last_undegraded': '2023-10-18T10:26:16.488641+0000', 'last_fullsized': '2023-10-18T10:26:16.488641+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T10:26:22.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-10-18T10:26:22.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-10-18T10:26:22.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490397+0000', 'last_change': '2023-10-18T10:26:16.490397+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490397+0000', 'last_undegraded': '2023-10-18T10:26:16.490397+0000', 'last_fullsized': '2023-10-18T10:26:16.490397+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T10:26:22.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-10-18T10:26:22.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-10-18T10:26:22.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490798+0000', 'last_change': '2023-10-18T10:26:16.490798+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490798+0000', 'last_undegraded': '2023-10-18T10:26:16.490798+0000', 'last_fullsized': '2023-10-18T10:26:16.490798+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T10:26:22.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-10-18T10:26:22.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-10-18T10:26:22.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490228+0000', 'last_change': '2023-10-18T10:26:16.490228+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490228+0000', 'last_undegraded': '2023-10-18T10:26:16.490228+0000', 'last_fullsized': '2023-10-18T10:26:16.490228+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T10:26:22.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-10-18T10:26:22.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-10-18T10:26:22.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490752+0000', 'last_change': '2023-10-18T10:26:16.490752+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490752+0000', 'last_undegraded': '2023-10-18T10:26:16.490752+0000', 'last_fullsized': '2023-10-18T10:26:16.490752+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T10:26:22.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-10-18T10:26:22.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-10-18T10:26:22.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490407+0000', 'last_change': '2023-10-18T10:26:16.490407+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490407+0000', 'last_undegraded': '2023-10-18T10:26:16.490407+0000', 'last_fullsized': '2023-10-18T10:26:16.490407+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T10:26:22.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-10-18T10:26:22.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-10-18T10:26:22.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490582+0000', 'last_change': '2023-10-18T10:26:16.490582+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490582+0000', 'last_undegraded': '2023-10-18T10:26:16.490582+0000', 'last_fullsized': '2023-10-18T10:26:16.490582+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T10:26:22.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-10-18T10:26:22.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-10-18T10:26:22.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.493329+0000', 'last_change': '2023-10-18T10:26:16.493329+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.493329+0000', 'last_undegraded': '2023-10-18T10:26:16.493329+0000', 'last_fullsized': '2023-10-18T10:26:16.493329+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T10:26:22.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-10-18T10:26:22.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-10-18T10:26:22.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490790+0000', 'last_change': '2023-10-18T10:26:16.490790+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490790+0000', 'last_undegraded': '2023-10-18T10:26:16.490790+0000', 'last_fullsized': '2023-10-18T10:26:16.490790+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T10:26:22.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-10-18T10:26:22.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-10-18T10:26:22.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490410+0000', 'last_change': '2023-10-18T10:26:16.490410+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490410+0000', 'last_undegraded': '2023-10-18T10:26:16.490410+0000', 'last_fullsized': '2023-10-18T10:26:16.490410+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T10:26:22.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-10-18T10:26:22.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-10-18T10:26:22.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.493298+0000', 'last_change': '2023-10-18T10:26:16.493298+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.493298+0000', 'last_undegraded': '2023-10-18T10:26:16.493298+0000', 'last_fullsized': '2023-10-18T10:26:16.493298+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T10:26:22.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-10-18T10:26:22.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-10-18T10:26:22.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.493307+0000', 'last_change': '2023-10-18T10:26:16.493307+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.493307+0000', 'last_undegraded': '2023-10-18T10:26:16.493307+0000', 'last_fullsized': '2023-10-18T10:26:16.493307+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T10:26:22.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-10-18T10:26:22.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-10-18T10:26:22.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490407+0000', 'last_change': '2023-10-18T10:26:16.490407+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490407+0000', 'last_undegraded': '2023-10-18T10:26:16.490407+0000', 'last_fullsized': '2023-10-18T10:26:16.490407+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T10:26:22.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-10-18T10:26:22.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-10-18T10:26:22.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.486718+0000', 'last_change': '2023-10-18T10:26:16.486718+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.486718+0000', 'last_undegraded': '2023-10-18T10:26:16.486718+0000', 'last_fullsized': '2023-10-18T10:26:16.486718+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T10:26:22.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-10-18T10:26:22.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-10-18T10:26:22.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490886+0000', 'last_change': '2023-10-18T10:26:16.490886+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490886+0000', 'last_undegraded': '2023-10-18T10:26:16.490886+0000', 'last_fullsized': '2023-10-18T10:26:16.490886+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T10:26:22.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-10-18T10:26:22.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-10-18T10:26:22.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.491320+0000', 'last_change': '2023-10-18T10:26:16.491320+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.491320+0000', 'last_undegraded': '2023-10-18T10:26:16.491320+0000', 'last_fullsized': '2023-10-18T10:26:16.491320+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T10:26:22.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-10-18T10:26:22.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-10-18T10:26:22.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490204+0000', 'last_change': '2023-10-18T10:26:16.490204+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490204+0000', 'last_undegraded': '2023-10-18T10:26:16.490204+0000', 'last_fullsized': '2023-10-18T10:26:16.490204+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T10:26:22.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-10-18T10:26:22.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-10-18T10:26:22.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.487893+0000', 'last_change': '2023-10-18T10:26:16.487893+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.487893+0000', 'last_undegraded': '2023-10-18T10:26:16.487893+0000', 'last_fullsized': '2023-10-18T10:26:16.487893+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T10:26:22.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-10-18T10:26:22.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-10-18T10:26:22.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490418+0000', 'last_change': '2023-10-18T10:26:16.490418+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490418+0000', 'last_undegraded': '2023-10-18T10:26:16.490418+0000', 'last_fullsized': '2023-10-18T10:26:16.490418+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T10:26:22.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-10-18T10:26:22.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-10-18T10:26:22.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.487567+0000', 'last_change': '2023-10-18T10:26:16.487567+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.487567+0000', 'last_undegraded': '2023-10-18T10:26:16.487567+0000', 'last_fullsized': '2023-10-18T10:26:16.487567+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T10:26:22.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-10-18T10:26:22.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-10-18T10:26:22.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.487423+0000', 'last_change': '2023-10-18T10:26:16.487423+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.487423+0000', 'last_undegraded': '2023-10-18T10:26:16.487423+0000', 'last_fullsized': '2023-10-18T10:26:16.487423+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T10:26:22.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-10-18T10:26:22.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-10-18T10:26:22.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490274+0000', 'last_change': '2023-10-18T10:26:16.490274+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490274+0000', 'last_undegraded': '2023-10-18T10:26:16.490274+0000', 'last_fullsized': '2023-10-18T10:26:16.490274+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T10:26:22.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-10-18T10:26:22.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-10-18T10:26:22.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.487464+0000', 'last_change': '2023-10-18T10:26:16.487464+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.487464+0000', 'last_undegraded': '2023-10-18T10:26:16.487464+0000', 'last_fullsized': '2023-10-18T10:26:16.487464+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T10:26:22.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-10-18T10:26:22.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-10-18T10:26:22.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490900+0000', 'last_change': '2023-10-18T10:26:16.490900+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490900+0000', 'last_undegraded': '2023-10-18T10:26:16.490900+0000', 'last_fullsized': '2023-10-18T10:26:16.490900+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T10:26:22.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-10-18T10:26:22.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-10-18T10:26:22.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.487428+0000', 'last_change': '2023-10-18T10:26:16.487428+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.487428+0000', 'last_undegraded': '2023-10-18T10:26:16.487428+0000', 'last_fullsized': '2023-10-18T10:26:16.487428+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T10:26:22.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-10-18T10:26:22.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-10-18T10:26:22.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.488653+0000', 'last_change': '2023-10-18T10:26:16.488653+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.488653+0000', 'last_undegraded': '2023-10-18T10:26:16.488653+0000', 'last_fullsized': '2023-10-18T10:26:16.488653+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T10:26:22.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-10-18T10:26:22.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-10-18T10:26:22.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.488668+0000', 'last_change': '2023-10-18T10:26:16.488668+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.488668+0000', 'last_undegraded': '2023-10-18T10:26:16.488668+0000', 'last_fullsized': '2023-10-18T10:26:16.488668+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T10:26:22.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-10-18T10:26:22.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-10-18T10:26:22.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.486716+0000', 'last_change': '2023-10-18T10:26:16.486716+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.486716+0000', 'last_undegraded': '2023-10-18T10:26:16.486716+0000', 'last_fullsized': '2023-10-18T10:26:16.486716+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T10:26:22.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-10-18T10:26:22.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-10-18T10:26:22.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.490101+0000', 'last_change': '2023-10-18T10:26:16.490101+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.490101+0000', 'last_undegraded': '2023-10-18T10:26:16.490101+0000', 'last_fullsized': '2023-10-18T10:26:16.490101+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T10:26:22.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-10-18T10:26:22.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-10-18T10:26:22.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.493277+0000', 'last_change': '2023-10-18T10:26:16.493277+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.493277+0000', 'last_undegraded': '2023-10-18T10:26:16.493277+0000', 'last_fullsized': '2023-10-18T10:26:16.493277+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T10:26:22.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-10-18T10:26:22.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:26:17.483887+0000', 'last_change': '2023-10-18T10:26:17.483887+0000', 'last_active': '2023-10-18T10:26:17.483887+0000', 'last_peered': '2023-10-18T10:26:17.483887+0000', 'last_clean': '2023-10-18T10:26:17.483887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:17.483887+0000', 'last_undegraded': '2023-10-18T10:26:17.483887+0000', 'last_fullsized': '2023-10-18T10:26:17.483887+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:17.483887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:26:22.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-10-18T10:26:22.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-10-18T10:26:16.491169+0000', 'last_change': '2023-10-18T10:26:16.491169+0000', 'last_active': '2023-10-18T10:26:16.482091+0000', 'last_peered': '2023-10-18T10:26:16.482091+0000', 'last_clean': '2023-10-18T10:26:16.482091+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:26:16.491169+0000', 'last_undegraded': '2023-10-18T10:26:16.491169+0000', 'last_fullsized': '2023-10-18T10:26:16.491169+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'last_clean_scrub_stamp': '2023-10-18T10:26:16.482091+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T10:26:22.213 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T10:26:22.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.313+0000 7f0a28ada640 1 -- 172.21.15.125:0/507672486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240afa20 msgr2=0x7f0a240705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.313+0000 7f0a28ada640 1 --2- 172.21.15.125:0/507672486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240afa20 0x7f0a240705a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009830 tx=0x7f0a0c02bd60 comp rx=0 tx=0).stop 2023-10-18T10:26:22.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.313+0000 7f0a28ada640 1 -- 172.21.15.125:0/507672486 shutdown_connections 2023-10-18T10:26:22.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.313+0000 7f0a28ada640 1 --2- 172.21.15.125:0/507672486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240afa20 0x7f0a240705a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.313+0000 7f0a28ada640 1 --2- 172.21.15.125:0/507672486 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a240cbd20 0x7f0a24158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.313+0000 7f0a28ada640 1 -- 172.21.15.125:0/507672486 >> 172.21.15.125:0/507672486 conn(0x7f0a24069930 msgr2=0x7f0a24069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:22.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.313+0000 7f0a28ada640 1 -- 172.21.15.125:0/507672486 shutdown_connections 2023-10-18T10:26:22.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.313+0000 7f0a28ada640 1 -- 172.21.15.125:0/507672486 wait complete. 2023-10-18T10:26:22.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.314+0000 7f0a28ada640 1 Processor -- start 2023-10-18T10:26:22.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.314+0000 7f0a28ada640 1 -- start start 2023-10-18T10:26:22.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a28ada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240afa20 0x7f0a24141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:22.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a28ada640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a240b0530 0x7f0a241562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:22.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a28ada640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a240cbd20 0x7f0a24156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:22.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a28ada640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2414cd00 con 0x7f0a240b0530 2023-10-18T10:26:22.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a28ada640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2414cf10 con 0x7f0a240afa20 2023-10-18T10:26:22.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a28ada640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2414d1f0 con 0x7f0a240cbd20 2023-10-18T10:26:22.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a240b0530 0x7f0a241562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:22.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a240b0530 0x7f0a241562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44040/0 (socket says 172.21.15.125:44040) 2023-10-18T10:26:22.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.315+0000 7f0a1bfff640 1 -- 172.21.15.125:0/1760832581 learned_addr learned my addr 172.21.15.125:0/1760832581 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:22.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a2259c640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240afa20 0x7f0a24141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:22.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a22d9d640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a240cbd20 0x7f0a24156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:22.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1bfff640 1 -- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a240cbd20 msgr2=0x7f0a24156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1bfff640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a240cbd20 0x7f0a24156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1bfff640 1 -- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240afa20 msgr2=0x7f0a24141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1bfff640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240afa20 0x7f0a24141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1bfff640 1 -- 172.21.15.125:0/1760832581 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a14009490 con 0x7f0a240b0530 2023-10-18T10:26:22.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1bfff640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a240b0530 0x7f0a241562c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c000c00 tx=0x7f0a0c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:22.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1affd640 1 -- 172.21.15.125:0/1760832581 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c003ea0 con 0x7f0a240b0530 2023-10-18T10:26:22.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1affd640 1 -- 172.21.15.125:0/1760832581 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a0c034440 con 0x7f0a240b0530 2023-10-18T10:26:22.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c009490 con 0x7f0a240b0530 2023-10-18T10:26:22.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a1affd640 1 -- 172.21.15.125:0/1760832581 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c034850 con 0x7f0a240b0530 2023-10-18T10:26:22.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.316+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a24070cf0 con 0x7f0a240b0530 2023-10-18T10:26:22.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.317+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2406a290 con 0x7f0a240b0530 2023-10-18T10:26:22.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.317+0000 7f0a1affd640 1 -- 172.21.15.125:0/1760832581 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0a0c0349b0 con 0x7f0a240b0530 2023-10-18T10:26:22.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.318+0000 7f0a1affd640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a0406ae40 0x7f0a0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:22.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.318+0000 7f0a1affd640 1 -- 172.21.15.125:0/1760832581 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0a0c0ac000 con 0x7f0a240b0530 2023-10-18T10:26:22.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.318+0000 7f0a2259c640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a0406ae40 0x7f0a0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:22.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.318+0000 7f0a2259c640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a0406ae40 0x7f0a0406d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009b70 tx=0x7f0a14005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:22.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.322+0000 7f0a1affd640 1 -- 172.21.15.125:0/1760832581 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a0c076ef0 con 0x7f0a240b0530 2023-10-18T10:26:22.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.553+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0a240cc1e0 con 0x7f0a240b0530 2023-10-18T10:26:22.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.553+0000 7f0a1affd640 1 -- 172.21.15.125:0/1760832581 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1269 (secure 0 0 0) 0x7f0a0c07a510 con 0x7f0a240b0530 2023-10-18T10:26:22.555 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:22.556 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":104,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225804288,"bytes_avail":772868308992,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5697}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-18T10:25:58.580766+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-18T10:26:22.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a0406ae40 msgr2=0x7f0a0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a0406ae40 0x7f0a0406d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009b70 tx=0x7f0a14005f90 comp rx=0 tx=0).stop 2023-10-18T10:26:22.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a240b0530 msgr2=0x7f0a241562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a240b0530 0x7f0a241562c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c000c00 tx=0x7f0a0c0038a0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 shutdown_connections 2023-10-18T10:26:22.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f0a0406ae40 0x7f0a0406d300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a240cbd20 0x7f0a24156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a240b0530 0x7f0a241562c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 --2- 172.21.15.125:0/1760832581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240afa20 0x7f0a24141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 >> 172.21.15.125:0/1760832581 conn(0x7f0a24069930 msgr2=0x7f0a240c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:22.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 shutdown_connections 2023-10-18T10:26:22.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.555+0000 7f0a28ada640 1 -- 172.21.15.125:0/1760832581 wait complete. 2023-10-18T10:26:22.567 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T10:26:22.568 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:26:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.669+0000 7f1be5dd7640 1 -- 172.21.15.125:0/44649924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00afa20 msgr2=0x7f1be00705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.669+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/44649924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00afa20 0x7f1be00705a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1be00cc7e0 tx=0x7f1bd402bdb0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.669+0000 7f1be5dd7640 1 -- 172.21.15.125:0/44649924 shutdown_connections 2023-10-18T10:26:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.669+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/44649924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00afa20 0x7f1be00705a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.669+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/44649924 >> v2:172.21.15.139:3300/0 conn(0x7f1be00cbd20 0x7f1be0158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.669+0000 7f1be5dd7640 1 -- 172.21.15.125:0/44649924 >> 172.21.15.125:0/44649924 conn(0x7f1be0069930 msgr2=0x7f1be0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.669+0000 7f1be5dd7640 1 -- 172.21.15.125:0/44649924 shutdown_connections 2023-10-18T10:26:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.669+0000 7f1be5dd7640 1 -- 172.21.15.125:0/44649924 wait complete. 2023-10-18T10:26:22.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.670+0000 7f1be5dd7640 1 Processor -- start 2023-10-18T10:26:22.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.670+0000 7f1be5dd7640 1 -- start start 2023-10-18T10:26:22.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1be5dd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00afa20 0x7f1be01433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:22.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1be5dd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00b0530 0x7f1be01567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:22.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1be5dd7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be00cbd20 0x7f1be0156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:22.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1be5dd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be014ab00 con 0x7f1be00b0530 2023-10-18T10:26:22.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1be5dd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0143930 con 0x7f1be00afa20 2023-10-18T10:26:22.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1be5dd7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0149a60 con 0x7f1be00cbd20 2023-10-18T10:26:22.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00b0530 0x7f1be01567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:22.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00b0530 0x7f1be01567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44058/0 (socket says 172.21.15.125:44058) 2023-10-18T10:26:22.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 -- 172.21.15.125:0/3599981072 learned_addr learned my addr 172.21.15.125:0/3599981072 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:22.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdffff640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be00cbd20 0x7f1be0156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:22.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdf7fe640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00afa20 0x7f1be01433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:22.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 -- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be00cbd20 msgr2=0x7f1be0156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be00cbd20 0x7f1be0156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 -- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00afa20 msgr2=0x7f1be01433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00afa20 0x7f1be01433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 -- 172.21.15.125:0/3599981072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd4009560 con 0x7f1be00b0530 2023-10-18T10:26:22.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bdeffd640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00b0530 0x7f1be01567e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009850 tx=0x7f1bd402dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:22.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bc7fff640 1 -- 172.21.15.125:0/3599981072 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4005650 con 0x7f1be00b0530 2023-10-18T10:26:22.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bc7fff640 1 -- 172.21.15.125:0/3599981072 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bd40038f0 con 0x7f1be00b0530 2023-10-18T10:26:22.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be0149d40 con 0x7f1be00b0530 2023-10-18T10:26:22.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1bc7fff640 1 -- 172.21.15.125:0/3599981072 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4003ce0 con 0x7f1be00b0530 2023-10-18T10:26:22.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.671+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be014a300 con 0x7f1be00b0530 2023-10-18T10:26:22.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.672+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be00690e0 con 0x7f1be00b0530 2023-10-18T10:26:22.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.673+0000 7f1bc7fff640 1 -- 172.21.15.125:0/3599981072 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1bd403e450 con 0x7f1be00b0530 2023-10-18T10:26:22.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.673+0000 7f1bc7fff640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1bcc06ae40 0x7f1bcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:22.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.673+0000 7f1bc7fff640 1 -- 172.21.15.125:0/3599981072 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1bd40abee0 con 0x7f1be00b0530 2023-10-18T10:26:22.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.674+0000 7f1bdf7fe640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1bcc06ae40 0x7f1bcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:22.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.674+0000 7f1bdf7fe640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1bcc06ae40 0x7f1bcc06d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00058c0 tx=0x7f1bd0005f60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:22.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.678+0000 7f1bc7fff640 1 -- 172.21.15.125:0/3599981072 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1bd40770b0 con 0x7f1be00b0530 2023-10-18T10:26:22.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.840+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1be006b910 con 0x7f1bcc06ae40 2023-10-18T10:26:22.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.847+0000 7f1bc7fff640 1 -- 172.21.15.125:0/3599981072 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289640 (secure 0 0 0) 0x7f1bd0014070 con 0x7f1bcc06ae40 2023-10-18T10:26:22.852 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:22.853 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:26:22.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.850+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1bcc06ae40 msgr2=0x7f1bcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.850+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1bcc06ae40 0x7f1bcc06d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00058c0 tx=0x7f1bd0005f60 comp rx=0 tx=0).stop 2023-10-18T10:26:22.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.850+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00b0530 msgr2=0x7f1be01567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:22.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.850+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00b0530 0x7f1be01567e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009850 tx=0x7f1bd402dd80 comp rx=0 tx=0).stop 2023-10-18T10:26:22.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.851+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 shutdown_connections 2023-10-18T10:26:22.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.851+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1bcc06ae40 0x7f1bcc06d300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.851+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be00cbd20 0x7f1be0156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.851+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1be00b0530 0x7f1be01567e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.851+0000 7f1be5dd7640 1 --2- 172.21.15.125:0/3599981072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00afa20 0x7f1be01433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:22.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.851+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 >> 172.21.15.125:0/3599981072 conn(0x7f1be0069930 msgr2=0x7f1be0148330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:22.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.851+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 shutdown_connections 2023-10-18T10:26:22.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:22.851+0000 7f1be5dd7640 1 -- 172.21.15.125:0/3599981072 wait complete. 2023-10-18T10:26:22.863 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-10-18T10:26:22.585819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220912,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753808,"statfs":{"total":773094113280,"available":772867899392,"internally_reserved":0,"allocated":4857856,"data_stored":2736332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001108"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518358+0000","last_change":"2023-10-18T10:26:18.205457+0000","last_active":"2023-10-18T10:26:19.518358+0000","last_peered":"2023-10-18T10:26:19.518358+0000","last_clean":"2023-10-18T10:26:19.518358+0000","last_became_active":"2023-10-18T10:26:18.205208+0000","last_became_peered":"2023-10-18T10:26:18.205208+0000","last_unstale":"2023-10-18T10:26:19.518358+0000","last_undegraded":"2023-10-18T10:26:19.518358+0000","last_fullsized":"2023-10-18T10:26:19.518358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:33.118950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518029+0000","last_change":"2023-10-18T10:26:18.204439+0000","last_active":"2023-10-18T10:26:19.518029+0000","last_peered":"2023-10-18T10:26:19.518029+0000","last_clean":"2023-10-18T10:26:19.518029+0000","last_became_active":"2023-10-18T10:26:18.204188+0000","last_became_peered":"2023-10-18T10:26:18.204188+0000","last_unstale":"2023-10-18T10:26:19.518029+0000","last_undegraded":"2023-10-18T10:26:19.518029+0000","last_fullsized":"2023-10-18T10:26:19.518029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:43:19.114729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199504+0000","last_change":"2023-10-18T10:26:18.209384+0000","last_active":"2023-10-18T10:26:19.199504+0000","last_peered":"2023-10-18T10:26:19.199504+0000","last_clean":"2023-10-18T10:26:19.199504+0000","last_became_active":"2023-10-18T10:26:18.209121+0000","last_became_peered":"2023-10-18T10:26:18.209121+0000","last_unstale":"2023-10-18T10:26:19.199504+0000","last_undegraded":"2023-10-18T10:26:19.199504+0000","last_fullsized":"2023-10-18T10:26:19.199504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:39.135581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518110+0000","last_change":"2023-10-18T10:26:18.210322+0000","last_active":"2023-10-18T10:26:19.518110+0000","last_peered":"2023-10-18T10:26:19.518110+0000","last_clean":"2023-10-18T10:26:19.518110+0000","last_became_active":"2023-10-18T10:26:18.210070+0000","last_became_peered":"2023-10-18T10:26:18.210070+0000","last_unstale":"2023-10-18T10:26:19.518110+0000","last_undegraded":"2023-10-18T10:26:19.518110+0000","last_fullsized":"2023-10-18T10:26:19.518110+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:12.484868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517551+0000","last_change":"2023-10-18T10:26:18.205374+0000","last_active":"2023-10-18T10:26:19.517551+0000","last_peered":"2023-10-18T10:26:19.517551+0000","last_clean":"2023-10-18T10:26:19.517551+0000","last_became_active":"2023-10-18T10:26:18.205127+0000","last_became_peered":"2023-10-18T10:26:18.205127+0000","last_unstale":"2023-10-18T10:26:19.517551+0000","last_undegraded":"2023-10-18T10:26:19.517551+0000","last_fullsized":"2023-10-18T10:26:19.517551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:32.648102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196770+0000","last_change":"2023-10-18T10:26:18.208709+0000","last_active":"2023-10-18T10:26:19.196770+0000","last_peered":"2023-10-18T10:26:19.196770+0000","last_clean":"2023-10-18T10:26:19.196770+0000","last_became_active":"2023-10-18T10:26:18.208448+0000","last_became_peered":"2023-10-18T10:26:18.208448+0000","last_unstale":"2023-10-18T10:26:19.196770+0000","last_undegraded":"2023-10-18T10:26:19.196770+0000","last_fullsized":"2023-10-18T10:26:19.196770+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:11.946296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518568+0000","last_change":"2023-10-18T10:26:18.208759+0000","last_active":"2023-10-18T10:26:19.518568+0000","last_peered":"2023-10-18T10:26:19.518568+0000","last_clean":"2023-10-18T10:26:19.518568+0000","last_became_active":"2023-10-18T10:26:18.208466+0000","last_became_peered":"2023-10-18T10:26:18.208466+0000","last_unstale":"2023-10-18T10:26:19.518568+0000","last_undegraded":"2023-10-18T10:26:19.518568+0000","last_fullsized":"2023-10-18T10:26:19.518568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:53.789576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517719+0000","last_change":"2023-10-18T10:26:18.203793+0000","last_active":"2023-10-18T10:26:19.517719+0000","last_peered":"2023-10-18T10:26:19.517719+0000","last_clean":"2023-10-18T10:26:19.517719+0000","last_became_active":"2023-10-18T10:26:18.203052+0000","last_became_peered":"2023-10-18T10:26:18.203052+0000","last_unstale":"2023-10-18T10:26:19.517719+0000","last_undegraded":"2023-10-18T10:26:19.517719+0000","last_fullsized":"2023-10-18T10:26:19.517719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:26.845326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199599+0000","last_change":"2023-10-18T10:26:18.210375+0000","last_active":"2023-10-18T10:26:19.199599+0000","last_peered":"2023-10-18T10:26:19.199599+0000","last_clean":"2023-10-18T10:26:19.199599+0000","last_became_active":"2023-10-18T10:26:18.210111+0000","last_became_peered":"2023-10-18T10:26:18.210111+0000","last_unstale":"2023-10-18T10:26:19.199599+0000","last_undegraded":"2023-10-18T10:26:19.199599+0000","last_fullsized":"2023-10-18T10:26:19.199599+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:53.789576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519421+0000","last_change":"2023-10-18T10:26:18.215225+0000","last_active":"2023-10-18T10:26:19.519421+0000","last_peered":"2023-10-18T10:26:19.519421+0000","last_clean":"2023-10-18T10:26:19.519421+0000","last_became_active":"2023-10-18T10:26:18.214830+0000","last_became_peered":"2023-10-18T10:26:18.214830+0000","last_unstale":"2023-10-18T10:26:19.519421+0000","last_undegraded":"2023-10-18T10:26:19.519421+0000","last_fullsized":"2023-10-18T10:26:19.519421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:45.278732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519401+0000","last_change":"2023-10-18T10:26:18.205857+0000","last_active":"2023-10-18T10:26:19.519401+0000","last_peered":"2023-10-18T10:26:19.519401+0000","last_clean":"2023-10-18T10:26:19.519401+0000","last_became_active":"2023-10-18T10:26:18.205629+0000","last_became_peered":"2023-10-18T10:26:18.205629+0000","last_unstale":"2023-10-18T10:26:19.519401+0000","last_undegraded":"2023-10-18T10:26:19.519401+0000","last_fullsized":"2023-10-18T10:26:19.519401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:26.845326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199463+0000","last_change":"2023-10-18T10:26:18.208843+0000","last_active":"2023-10-18T10:26:19.199463+0000","last_peered":"2023-10-18T10:26:19.199463+0000","last_clean":"2023-10-18T10:26:19.199463+0000","last_became_active":"2023-10-18T10:26:18.208589+0000","last_became_peered":"2023-10-18T10:26:18.208589+0000","last_unstale":"2023-10-18T10:26:19.199463+0000","last_undegraded":"2023-10-18T10:26:19.199463+0000","last_fullsized":"2023-10-18T10:26:19.199463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:30:51.605675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519049+0000","last_change":"2023-10-18T10:26:18.218554+0000","last_active":"2023-10-18T10:26:19.519049+0000","last_peered":"2023-10-18T10:26:19.519049+0000","last_clean":"2023-10-18T10:26:19.519049+0000","last_became_active":"2023-10-18T10:26:18.218237+0000","last_became_peered":"2023-10-18T10:26:18.218237+0000","last_unstale":"2023-10-18T10:26:19.519049+0000","last_undegraded":"2023-10-18T10:26:19.519049+0000","last_fullsized":"2023-10-18T10:26:19.519049+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:12.715480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195327+0000","last_change":"2023-10-18T10:26:18.214467+0000","last_active":"2023-10-18T10:26:19.195327+0000","last_peered":"2023-10-18T10:26:19.195327+0000","last_clean":"2023-10-18T10:26:19.195327+0000","last_became_active":"2023-10-18T10:26:18.214198+0000","last_became_peered":"2023-10-18T10:26:18.214198+0000","last_unstale":"2023-10-18T10:26:19.195327+0000","last_undegraded":"2023-10-18T10:26:19.195327+0000","last_fullsized":"2023-10-18T10:26:19.195327+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:18.021127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518806+0000","last_change":"2023-10-18T10:26:18.205469+0000","last_active":"2023-10-18T10:26:19.518806+0000","last_peered":"2023-10-18T10:26:19.518806+0000","last_clean":"2023-10-18T10:26:19.518806+0000","last_became_active":"2023-10-18T10:26:18.205150+0000","last_became_peered":"2023-10-18T10:26:18.205150+0000","last_unstale":"2023-10-18T10:26:19.518806+0000","last_undegraded":"2023-10-18T10:26:19.518806+0000","last_fullsized":"2023-10-18T10:26:19.518806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:18.089923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484032+0000","last_change":"2023-10-18T10:26:18.219014+0000","last_active":"2023-10-18T10:26:19.484032+0000","last_peered":"2023-10-18T10:26:19.484032+0000","last_clean":"2023-10-18T10:26:19.484032+0000","last_became_active":"2023-10-18T10:26:18.218751+0000","last_became_peered":"2023-10-18T10:26:18.218751+0000","last_unstale":"2023-10-18T10:26:19.484032+0000","last_undegraded":"2023-10-18T10:26:19.484032+0000","last_fullsized":"2023-10-18T10:26:19.484032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:56.504440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484433+0000","last_change":"2023-10-18T10:26:18.220123+0000","last_active":"2023-10-18T10:26:19.484433+0000","last_peered":"2023-10-18T10:26:19.484433+0000","last_clean":"2023-10-18T10:26:19.484433+0000","last_became_active":"2023-10-18T10:26:18.219811+0000","last_became_peered":"2023-10-18T10:26:18.219811+0000","last_unstale":"2023-10-18T10:26:19.484433+0000","last_undegraded":"2023-10-18T10:26:19.484433+0000","last_fullsized":"2023-10-18T10:26:19.484433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:15.053138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518857+0000","last_change":"2023-10-18T10:26:18.212334+0000","last_active":"2023-10-18T10:26:19.518857+0000","last_peered":"2023-10-18T10:26:19.518857+0000","last_clean":"2023-10-18T10:26:19.518857+0000","last_became_active":"2023-10-18T10:26:18.212076+0000","last_became_peered":"2023-10-18T10:26:18.212076+0000","last_unstale":"2023-10-18T10:26:19.518857+0000","last_undegraded":"2023-10-18T10:26:19.518857+0000","last_fullsized":"2023-10-18T10:26:19.518857+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:26.801320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518011+0000","last_change":"2023-10-18T10:26:18.211737+0000","last_active":"2023-10-18T10:26:19.518011+0000","last_peered":"2023-10-18T10:26:19.518011+0000","last_clean":"2023-10-18T10:26:19.518011+0000","last_became_active":"2023-10-18T10:26:18.211483+0000","last_became_peered":"2023-10-18T10:26:18.211483+0000","last_unstale":"2023-10-18T10:26:19.518011+0000","last_undegraded":"2023-10-18T10:26:19.518011+0000","last_fullsized":"2023-10-18T10:26:19.518011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:13.566881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518572+0000","last_change":"2023-10-18T10:26:18.213694+0000","last_active":"2023-10-18T10:26:19.518572+0000","last_peered":"2023-10-18T10:26:19.518572+0000","last_clean":"2023-10-18T10:26:19.518572+0000","last_became_active":"2023-10-18T10:26:18.213400+0000","last_became_peered":"2023-10-18T10:26:18.213400+0000","last_unstale":"2023-10-18T10:26:19.518572+0000","last_undegraded":"2023-10-18T10:26:19.518572+0000","last_fullsized":"2023-10-18T10:26:19.518572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:18.021127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519104+0000","last_change":"2023-10-18T10:26:18.209470+0000","last_active":"2023-10-18T10:26:19.519104+0000","last_peered":"2023-10-18T10:26:19.519104+0000","last_clean":"2023-10-18T10:26:19.519104+0000","last_became_active":"2023-10-18T10:26:18.209154+0000","last_became_peered":"2023-10-18T10:26:18.209154+0000","last_unstale":"2023-10-18T10:26:19.519104+0000","last_undegraded":"2023-10-18T10:26:19.519104+0000","last_fullsized":"2023-10-18T10:26:19.519104+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:03.950514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195587+0000","last_change":"2023-10-18T10:26:18.206634+0000","last_active":"2023-10-18T10:26:19.195587+0000","last_peered":"2023-10-18T10:26:19.195587+0000","last_clean":"2023-10-18T10:26:19.195587+0000","last_became_active":"2023-10-18T10:26:18.206354+0000","last_became_peered":"2023-10-18T10:26:18.206354+0000","last_unstale":"2023-10-18T10:26:19.195587+0000","last_undegraded":"2023-10-18T10:26:19.195587+0000","last_fullsized":"2023-10-18T10:26:19.195587+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:21.333154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484112+0000","last_change":"2023-10-18T10:26:18.218218+0000","last_active":"2023-10-18T10:26:19.484112+0000","last_peered":"2023-10-18T10:26:19.484112+0000","last_clean":"2023-10-18T10:26:19.484112+0000","last_became_active":"2023-10-18T10:26:18.217962+0000","last_became_peered":"2023-10-18T10:26:18.217962+0000","last_unstale":"2023-10-18T10:26:19.484112+0000","last_undegraded":"2023-10-18T10:26:19.484112+0000","last_fullsized":"2023-10-18T10:26:19.484112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:30.003248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517498+0000","last_change":"2023-10-18T10:26:18.215015+0000","last_active":"2023-10-18T10:26:19.517498+0000","last_peered":"2023-10-18T10:26:19.517498+0000","last_clean":"2023-10-18T10:26:19.517498+0000","last_became_active":"2023-10-18T10:26:18.214756+0000","last_became_peered":"2023-10-18T10:26:18.214756+0000","last_unstale":"2023-10-18T10:26:19.517498+0000","last_undegraded":"2023-10-18T10:26:19.517498+0000","last_fullsized":"2023-10-18T10:26:19.517498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:50.047140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519206+0000","last_change":"2023-10-18T10:26:18.209245+0000","last_active":"2023-10-18T10:26:19.519206+0000","last_peered":"2023-10-18T10:26:19.519206+0000","last_clean":"2023-10-18T10:26:19.519206+0000","last_became_active":"2023-10-18T10:26:18.208964+0000","last_became_peered":"2023-10-18T10:26:18.208964+0000","last_unstale":"2023-10-18T10:26:19.519206+0000","last_undegraded":"2023-10-18T10:26:19.519206+0000","last_fullsized":"2023-10-18T10:26:19.519206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:21.333154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196982+0000","last_change":"2023-10-18T10:26:18.214786+0000","last_active":"2023-10-18T10:26:19.196982+0000","last_peered":"2023-10-18T10:26:19.196982+0000","last_clean":"2023-10-18T10:26:19.196982+0000","last_became_active":"2023-10-18T10:26:18.214535+0000","last_became_peered":"2023-10-18T10:26:18.214535+0000","last_unstale":"2023-10-18T10:26:19.196982+0000","last_undegraded":"2023-10-18T10:26:19.196982+0000","last_fullsized":"2023-10-18T10:26:19.196982+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:39.135581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518802+0000","last_change":"2023-10-18T10:26:18.207616+0000","last_active":"2023-10-18T10:26:19.518802+0000","last_peered":"2023-10-18T10:26:19.518802+0000","last_clean":"2023-10-18T10:26:19.518802+0000","last_became_active":"2023-10-18T10:26:18.207264+0000","last_became_peered":"2023-10-18T10:26:18.207264+0000","last_unstale":"2023-10-18T10:26:19.518802+0000","last_undegraded":"2023-10-18T10:26:19.518802+0000","last_fullsized":"2023-10-18T10:26:19.518802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:23.951622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519474+0000","last_change":"2023-10-18T10:26:18.208906+0000","last_active":"2023-10-18T10:26:19.519474+0000","last_peered":"2023-10-18T10:26:19.519474+0000","last_clean":"2023-10-18T10:26:19.519474+0000","last_became_active":"2023-10-18T10:26:18.208593+0000","last_became_peered":"2023-10-18T10:26:18.208593+0000","last_unstale":"2023-10-18T10:26:19.519474+0000","last_undegraded":"2023-10-18T10:26:19.519474+0000","last_fullsized":"2023-10-18T10:26:19.519474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:43:19.114729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518491+0000","last_change":"2023-10-18T10:26:18.209524+0000","last_active":"2023-10-18T10:26:19.518491+0000","last_peered":"2023-10-18T10:26:19.518491+0000","last_clean":"2023-10-18T10:26:19.518491+0000","last_became_active":"2023-10-18T10:26:18.209158+0000","last_became_peered":"2023-10-18T10:26:18.209158+0000","last_unstale":"2023-10-18T10:26:19.518491+0000","last_undegraded":"2023-10-18T10:26:19.518491+0000","last_fullsized":"2023-10-18T10:26:19.518491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:14.659949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518422+0000","last_change":"2023-10-18T10:26:18.207877+0000","last_active":"2023-10-18T10:26:19.518422+0000","last_peered":"2023-10-18T10:26:19.518422+0000","last_clean":"2023-10-18T10:26:19.518422+0000","last_became_active":"2023-10-18T10:26:18.207513+0000","last_became_peered":"2023-10-18T10:26:18.207513+0000","last_unstale":"2023-10-18T10:26:19.518422+0000","last_undegraded":"2023-10-18T10:26:19.518422+0000","last_fullsized":"2023-10-18T10:26:19.518422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:59.398367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484243+0000","last_change":"2023-10-18T10:26:18.218659+0000","last_active":"2023-10-18T10:26:19.484243+0000","last_peered":"2023-10-18T10:26:19.484243+0000","last_clean":"2023-10-18T10:26:19.484243+0000","last_became_active":"2023-10-18T10:26:18.217587+0000","last_became_peered":"2023-10-18T10:26:18.217587+0000","last_unstale":"2023-10-18T10:26:19.484243+0000","last_undegraded":"2023-10-18T10:26:19.484243+0000","last_fullsized":"2023-10-18T10:26:19.484243+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:18.089923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196898+0000","last_change":"2023-10-18T10:26:18.208494+0000","last_active":"2023-10-18T10:26:19.196898+0000","last_peered":"2023-10-18T10:26:19.196898+0000","last_clean":"2023-10-18T10:26:19.196898+0000","last_became_active":"2023-10-18T10:26:18.208232+0000","last_became_peered":"2023-10-18T10:26:18.208232+0000","last_unstale":"2023-10-18T10:26:19.196898+0000","last_undegraded":"2023-10-18T10:26:19.196898+0000","last_fullsized":"2023-10-18T10:26:19.196898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:56.504440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484548+0000","last_change":"2023-10-18T10:26:18.219843+0000","last_active":"2023-10-18T10:26:19.484548+0000","last_peered":"2023-10-18T10:26:19.484548+0000","last_clean":"2023-10-18T10:26:19.484548+0000","last_became_active":"2023-10-18T10:26:18.219471+0000","last_became_peered":"2023-10-18T10:26:18.219471+0000","last_unstale":"2023-10-18T10:26:19.484548+0000","last_undegraded":"2023-10-18T10:26:19.484548+0000","last_fullsized":"2023-10-18T10:26:19.484548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:26.845326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529465+0000","last_change":"2023-10-18T10:26:17.507234+0000","last_active":"2023-10-18T10:26:18.529465+0000","last_peered":"2023-10-18T10:26:18.529465+0000","last_clean":"2023-10-18T10:26:18.529465+0000","last_became_active":"2023-10-18T10:26:17.506966+0000","last_became_peered":"2023-10-18T10:26:17.506966+0000","last_unstale":"2023-10-18T10:26:18.529465+0000","last_undegraded":"2023-10-18T10:26:18.529465+0000","last_fullsized":"2023-10-18T10:26:18.529465+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:42.444062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.197093+0000","last_change":"2023-10-18T10:26:18.209900+0000","last_active":"2023-10-18T10:26:19.197093+0000","last_peered":"2023-10-18T10:26:19.197093+0000","last_clean":"2023-10-18T10:26:19.197093+0000","last_became_active":"2023-10-18T10:26:18.209641+0000","last_became_peered":"2023-10-18T10:26:18.209641+0000","last_unstale":"2023-10-18T10:26:19.197093+0000","last_undegraded":"2023-10-18T10:26:19.197093+0000","last_fullsized":"2023-10-18T10:26:19.197093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:30.003248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.526367+0000","last_change":"2023-10-18T10:26:17.506492+0000","last_active":"2023-10-18T10:26:18.526367+0000","last_peered":"2023-10-18T10:26:18.526367+0000","last_clean":"2023-10-18T10:26:18.526367+0000","last_became_active":"2023-10-18T10:26:17.506187+0000","last_became_peered":"2023-10-18T10:26:17.506187+0000","last_unstale":"2023-10-18T10:26:18.526367+0000","last_undegraded":"2023-10-18T10:26:18.526367+0000","last_fullsized":"2023-10-18T10:26:18.526367+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:58.396571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195774+0000","last_change":"2023-10-18T10:26:18.215771+0000","last_active":"2023-10-18T10:26:19.195774+0000","last_peered":"2023-10-18T10:26:19.195774+0000","last_clean":"2023-10-18T10:26:19.195774+0000","last_became_active":"2023-10-18T10:26:18.215516+0000","last_became_peered":"2023-10-18T10:26:18.215516+0000","last_unstale":"2023-10-18T10:26:19.195774+0000","last_undegraded":"2023-10-18T10:26:19.195774+0000","last_fullsized":"2023-10-18T10:26:19.195774+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:29:03.501058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.526260+0000","last_change":"2023-10-18T10:26:17.503565+0000","last_active":"2023-10-18T10:26:18.526260+0000","last_peered":"2023-10-18T10:26:18.526260+0000","last_clean":"2023-10-18T10:26:18.526260+0000","last_became_active":"2023-10-18T10:26:17.503338+0000","last_became_peered":"2023-10-18T10:26:17.503338+0000","last_unstale":"2023-10-18T10:26:18.526260+0000","last_undegraded":"2023-10-18T10:26:18.526260+0000","last_fullsized":"2023-10-18T10:26:18.526260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:33.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518354+0000","last_change":"2023-10-18T10:26:18.209007+0000","last_active":"2023-10-18T10:26:19.518354+0000","last_peered":"2023-10-18T10:26:19.518354+0000","last_clean":"2023-10-18T10:26:19.518354+0000","last_became_active":"2023-10-18T10:26:18.208626+0000","last_became_peered":"2023-10-18T10:26:18.208626+0000","last_unstale":"2023-10-18T10:26:19.518354+0000","last_undegraded":"2023-10-18T10:26:19.518354+0000","last_fullsized":"2023-10-18T10:26:19.518354+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:14.115689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.525197+0000","last_change":"2023-10-18T10:26:17.503503+0000","last_active":"2023-10-18T10:26:18.525197+0000","last_peered":"2023-10-18T10:26:18.525197+0000","last_clean":"2023-10-18T10:26:18.525197+0000","last_became_active":"2023-10-18T10:26:17.503216+0000","last_became_peered":"2023-10-18T10:26:17.503216+0000","last_unstale":"2023-10-18T10:26:18.525197+0000","last_undegraded":"2023-10-18T10:26:18.525197+0000","last_fullsized":"2023-10-18T10:26:18.525197+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:13.658154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199332+0000","last_change":"2023-10-18T10:26:18.210290+0000","last_active":"2023-10-18T10:26:19.199332+0000","last_peered":"2023-10-18T10:26:19.199332+0000","last_clean":"2023-10-18T10:26:19.199332+0000","last_became_active":"2023-10-18T10:26:18.210047+0000","last_became_peered":"2023-10-18T10:26:18.210047+0000","last_unstale":"2023-10-18T10:26:19.199332+0000","last_undegraded":"2023-10-18T10:26:19.199332+0000","last_fullsized":"2023-10-18T10:26:19.199332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:51.851998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529189+0000","last_change":"2023-10-18T10:26:17.500080+0000","last_active":"2023-10-18T10:26:18.529189+0000","last_peered":"2023-10-18T10:26:18.529189+0000","last_clean":"2023-10-18T10:26:18.529189+0000","last_became_active":"2023-10-18T10:26:17.499772+0000","last_became_peered":"2023-10-18T10:26:17.499772+0000","last_unstale":"2023-10-18T10:26:18.529189+0000","last_undegraded":"2023-10-18T10:26:18.529189+0000","last_fullsized":"2023-10-18T10:26:18.529189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:38.133785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518957+0000","last_change":"2023-10-18T10:26:18.210166+0000","last_active":"2023-10-18T10:26:19.518957+0000","last_peered":"2023-10-18T10:26:19.518957+0000","last_clean":"2023-10-18T10:26:19.518957+0000","last_became_active":"2023-10-18T10:26:18.209251+0000","last_became_peered":"2023-10-18T10:26:18.209251+0000","last_unstale":"2023-10-18T10:26:19.518957+0000","last_undegraded":"2023-10-18T10:26:19.518957+0000","last_fullsized":"2023-10-18T10:26:19.518957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:44.389666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.524073+0000","last_change":"2023-10-18T10:26:17.505717+0000","last_active":"2023-10-18T10:26:18.524073+0000","last_peered":"2023-10-18T10:26:18.524073+0000","last_clean":"2023-10-18T10:26:18.524073+0000","last_became_active":"2023-10-18T10:26:17.505479+0000","last_became_peered":"2023-10-18T10:26:17.505479+0000","last_unstale":"2023-10-18T10:26:18.524073+0000","last_undegraded":"2023-10-18T10:26:18.524073+0000","last_fullsized":"2023-10-18T10:26:18.524073+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:58.396571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195625+0000","last_change":"2023-10-18T10:26:18.210194+0000","last_active":"2023-10-18T10:26:19.195625+0000","last_peered":"2023-10-18T10:26:19.195625+0000","last_clean":"2023-10-18T10:26:19.195625+0000","last_became_active":"2023-10-18T10:26:18.209925+0000","last_became_peered":"2023-10-18T10:26:18.209925+0000","last_unstale":"2023-10-18T10:26:19.195625+0000","last_undegraded":"2023-10-18T10:26:19.195625+0000","last_fullsized":"2023-10-18T10:26:19.195625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:34.944286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.202498+0000","last_change":"2023-10-18T10:26:17.501283+0000","last_active":"2023-10-18T10:26:18.202498+0000","last_peered":"2023-10-18T10:26:18.202498+0000","last_clean":"2023-10-18T10:26:18.202498+0000","last_became_active":"2023-10-18T10:26:17.501020+0000","last_became_peered":"2023-10-18T10:26:17.501020+0000","last_unstale":"2023-10-18T10:26:18.202498+0000","last_undegraded":"2023-10-18T10:26:18.202498+0000","last_fullsized":"2023-10-18T10:26:18.202498+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:22.949827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517554+0000","last_change":"2023-10-18T10:26:18.213883+0000","last_active":"2023-10-18T10:26:19.517554+0000","last_peered":"2023-10-18T10:26:19.517554+0000","last_clean":"2023-10-18T10:26:19.517554+0000","last_became_active":"2023-10-18T10:26:18.213619+0000","last_became_peered":"2023-10-18T10:26:18.213619+0000","last_unstale":"2023-10-18T10:26:19.517554+0000","last_undegraded":"2023-10-18T10:26:19.517554+0000","last_fullsized":"2023-10-18T10:26:19.517554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:15.053138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.204963+0000","last_change":"2023-10-18T10:26:17.507546+0000","last_active":"2023-10-18T10:26:18.204963+0000","last_peered":"2023-10-18T10:26:18.204963+0000","last_clean":"2023-10-18T10:26:18.204963+0000","last_became_active":"2023-10-18T10:26:17.507288+0000","last_became_peered":"2023-10-18T10:26:17.507288+0000","last_unstale":"2023-10-18T10:26:18.204963+0000","last_undegraded":"2023-10-18T10:26:18.204963+0000","last_fullsized":"2023-10-18T10:26:18.204963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:42.444062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517926+0000","last_change":"2023-10-18T10:26:18.205946+0000","last_active":"2023-10-18T10:26:19.517926+0000","last_peered":"2023-10-18T10:26:19.517926+0000","last_clean":"2023-10-18T10:26:19.517926+0000","last_became_active":"2023-10-18T10:26:18.205690+0000","last_became_peered":"2023-10-18T10:26:18.205690+0000","last_unstale":"2023-10-18T10:26:19.517926+0000","last_undegraded":"2023-10-18T10:26:19.517926+0000","last_fullsized":"2023-10-18T10:26:19.517926+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:42:27.717120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.198721+0000","last_change":"2023-10-18T10:26:17.500469+0000","last_active":"2023-10-18T10:26:18.198721+0000","last_peered":"2023-10-18T10:26:18.198721+0000","last_clean":"2023-10-18T10:26:18.198721+0000","last_became_active":"2023-10-18T10:26:17.500200+0000","last_became_peered":"2023-10-18T10:26:17.500200+0000","last_unstale":"2023-10-18T10:26:18.198721+0000","last_undegraded":"2023-10-18T10:26:18.198721+0000","last_fullsized":"2023-10-18T10:26:18.198721+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:43.387870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484387+0000","last_change":"2023-10-18T10:26:18.221851+0000","last_active":"2023-10-18T10:26:19.484387+0000","last_peered":"2023-10-18T10:26:19.484387+0000","last_clean":"2023-10-18T10:26:19.484387+0000","last_became_active":"2023-10-18T10:26:18.221575+0000","last_became_peered":"2023-10-18T10:26:18.221575+0000","last_unstale":"2023-10-18T10:26:19.484387+0000","last_undegraded":"2023-10-18T10:26:19.484387+0000","last_fullsized":"2023-10-18T10:26:19.484387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:44.389666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.200370+0000","last_change":"2023-10-18T10:26:17.501638+0000","last_active":"2023-10-18T10:26:18.200370+0000","last_peered":"2023-10-18T10:26:18.200370+0000","last_clean":"2023-10-18T10:26:18.200370+0000","last_became_active":"2023-10-18T10:26:17.501382+0000","last_became_peered":"2023-10-18T10:26:17.501382+0000","last_unstale":"2023-10-18T10:26:18.200370+0000","last_undegraded":"2023-10-18T10:26:18.200370+0000","last_fullsized":"2023-10-18T10:26:18.200370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:29:02.499262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199464+0000","last_change":"2023-10-18T10:26:18.215325+0000","last_active":"2023-10-18T10:26:19.199464+0000","last_peered":"2023-10-18T10:26:19.199464+0000","last_clean":"2023-10-18T10:26:19.199464+0000","last_became_active":"2023-10-18T10:26:18.215146+0000","last_became_peered":"2023-10-18T10:26:18.215146+0000","last_unstale":"2023-10-18T10:26:19.199464+0000","last_undegraded":"2023-10-18T10:26:19.199464+0000","last_fullsized":"2023-10-18T10:26:19.199464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:56.504440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529279+0000","last_change":"2023-10-18T10:26:17.504701+0000","last_active":"2023-10-18T10:26:18.529279+0000","last_peered":"2023-10-18T10:26:18.529279+0000","last_clean":"2023-10-18T10:26:18.529279+0000","last_became_active":"2023-10-18T10:26:17.504469+0000","last_became_peered":"2023-10-18T10:26:17.504469+0000","last_unstale":"2023-10-18T10:26:18.529279+0000","last_undegraded":"2023-10-18T10:26:18.529279+0000","last_fullsized":"2023-10-18T10:26:18.529279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:30:50.603880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195569+0000","last_change":"2023-10-18T10:26:18.206715+0000","last_active":"2023-10-18T10:26:19.195569+0000","last_peered":"2023-10-18T10:26:19.195569+0000","last_clean":"2023-10-18T10:26:19.195569+0000","last_became_active":"2023-10-18T10:26:18.206445+0000","last_became_peered":"2023-10-18T10:26:18.206445+0000","last_unstale":"2023-10-18T10:26:19.195569+0000","last_undegraded":"2023-10-18T10:26:19.195569+0000","last_fullsized":"2023-10-18T10:26:19.195569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:26.801320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.531922+0000","last_change":"2023-10-18T10:26:17.506418+0000","last_active":"2023-10-18T10:26:18.531922+0000","last_peered":"2023-10-18T10:26:18.531922+0000","last_clean":"2023-10-18T10:26:18.531922+0000","last_became_active":"2023-10-18T10:26:17.506173+0000","last_became_peered":"2023-10-18T10:26:17.506173+0000","last_unstale":"2023-10-18T10:26:18.531922+0000","last_undegraded":"2023-10-18T10:26:18.531922+0000","last_fullsized":"2023-10-18T10:26:18.531922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:38.133785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199344+0000","last_change":"2023-10-18T10:26:18.209406+0000","last_active":"2023-10-18T10:26:19.199344+0000","last_peered":"2023-10-18T10:26:19.199344+0000","last_clean":"2023-10-18T10:26:19.199344+0000","last_became_active":"2023-10-18T10:26:18.209098+0000","last_became_peered":"2023-10-18T10:26:18.209098+0000","last_unstale":"2023-10-18T10:26:19.199344+0000","last_undegraded":"2023-10-18T10:26:19.199344+0000","last_fullsized":"2023-10-18T10:26:19.199344+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:11.946296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.524585+0000","last_change":"2023-10-18T10:26:17.505146+0000","last_active":"2023-10-18T10:26:18.524585+0000","last_peered":"2023-10-18T10:26:18.524585+0000","last_clean":"2023-10-18T10:26:18.524585+0000","last_became_active":"2023-10-18T10:26:17.504876+0000","last_became_peered":"2023-10-18T10:26:17.504876+0000","last_unstale":"2023-10-18T10:26:18.524585+0000","last_undegraded":"2023-10-18T10:26:18.524585+0000","last_fullsized":"2023-10-18T10:26:18.524585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:17.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517649+0000","last_change":"2023-10-18T10:26:18.205349+0000","last_active":"2023-10-18T10:26:19.517649+0000","last_peered":"2023-10-18T10:26:19.517649+0000","last_clean":"2023-10-18T10:26:19.517649+0000","last_became_active":"2023-10-18T10:26:18.205098+0000","last_became_peered":"2023-10-18T10:26:18.205098+0000","last_unstale":"2023-10-18T10:26:19.517649+0000","last_undegraded":"2023-10-18T10:26:19.517649+0000","last_fullsized":"2023-10-18T10:26:19.517649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:12.715480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.202906+0000","last_change":"2023-10-18T10:26:17.502227+0000","last_active":"2023-10-18T10:26:18.202906+0000","last_peered":"2023-10-18T10:26:18.202906+0000","last_clean":"2023-10-18T10:26:18.202906+0000","last_became_active":"2023-10-18T10:26:17.501980+0000","last_became_peered":"2023-10-18T10:26:17.501980+0000","last_unstale":"2023-10-18T10:26:18.202906+0000","last_undegraded":"2023-10-18T10:26:18.202906+0000","last_fullsized":"2023-10-18T10:26:18.202906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:20.331358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484100+0000","last_change":"2023-10-18T10:26:18.217805+0000","last_active":"2023-10-18T10:26:19.484100+0000","last_peered":"2023-10-18T10:26:19.484100+0000","last_clean":"2023-10-18T10:26:19.484100+0000","last_became_active":"2023-10-18T10:26:18.217501+0000","last_became_peered":"2023-10-18T10:26:18.217501+0000","last_unstale":"2023-10-18T10:26:19.484100+0000","last_undegraded":"2023-10-18T10:26:19.484100+0000","last_fullsized":"2023-10-18T10:26:19.484100+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:39.135581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.201827+0000","last_change":"2023-10-18T10:26:17.507336+0000","last_active":"2023-10-18T10:26:18.201827+0000","last_peered":"2023-10-18T10:26:18.201827+0000","last_clean":"2023-10-18T10:26:18.201827+0000","last_became_active":"2023-10-18T10:26:17.507076+0000","last_became_peered":"2023-10-18T10:26:17.507076+0000","last_unstale":"2023-10-18T10:26:18.201827+0000","last_undegraded":"2023-10-18T10:26:18.201827+0000","last_fullsized":"2023-10-18T10:26:18.201827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:39.362487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517674+0000","last_change":"2023-10-18T10:26:18.205381+0000","last_active":"2023-10-18T10:26:19.517674+0000","last_peered":"2023-10-18T10:26:19.517674+0000","last_clean":"2023-10-18T10:26:19.517674+0000","last_became_active":"2023-10-18T10:26:18.205143+0000","last_became_peered":"2023-10-18T10:26:18.205143+0000","last_unstale":"2023-10-18T10:26:19.517674+0000","last_undegraded":"2023-10-18T10:26:19.517674+0000","last_fullsized":"2023-10-18T10:26:19.517674+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:03.950514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529817+0000","last_change":"2023-10-18T10:26:17.505711+0000","last_active":"2023-10-18T10:26:18.529817+0000","last_peered":"2023-10-18T10:26:18.529817+0000","last_clean":"2023-10-18T10:26:18.529817+0000","last_became_active":"2023-10-18T10:26:17.504986+0000","last_became_peered":"2023-10-18T10:26:17.504986+0000","last_unstale":"2023-10-18T10:26:18.529817+0000","last_undegraded":"2023-10-18T10:26:18.529817+0000","last_fullsized":"2023-10-18T10:26:18.529817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:25.799525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518215+0000","last_change":"2023-10-18T10:26:18.207872+0000","last_active":"2023-10-18T10:26:19.518215+0000","last_peered":"2023-10-18T10:26:19.518215+0000","last_clean":"2023-10-18T10:26:19.518215+0000","last_became_active":"2023-10-18T10:26:18.207613+0000","last_became_peered":"2023-10-18T10:26:18.207613+0000","last_unstale":"2023-10-18T10:26:19.518215+0000","last_undegraded":"2023-10-18T10:26:19.518215+0000","last_fullsized":"2023-10-18T10:26:19.518215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:03:41.361726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529477+0000","last_change":"2023-10-18T10:26:17.500144+0000","last_active":"2023-10-18T10:26:18.529477+0000","last_peered":"2023-10-18T10:26:18.529477+0000","last_clean":"2023-10-18T10:26:18.529477+0000","last_became_active":"2023-10-18T10:26:17.499871+0000","last_became_peered":"2023-10-18T10:26:17.499871+0000","last_unstale":"2023-10-18T10:26:18.529477+0000","last_undegraded":"2023-10-18T10:26:18.529477+0000","last_fullsized":"2023-10-18T10:26:18.529477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:33.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519631+0000","last_change":"2023-10-18T10:26:18.210224+0000","last_active":"2023-10-18T10:26:19.519631+0000","last_peered":"2023-10-18T10:26:19.519631+0000","last_clean":"2023-10-18T10:26:19.519631+0000","last_became_active":"2023-10-18T10:26:18.209603+0000","last_became_peered":"2023-10-18T10:26:18.209603+0000","last_unstale":"2023-10-18T10:26:19.519631+0000","last_undegraded":"2023-10-18T10:26:19.519631+0000","last_fullsized":"2023-10-18T10:26:19.519631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:03:41.361726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.525009+0000","last_change":"2023-10-18T10:26:17.502150+0000","last_active":"2023-10-18T10:26:18.525009+0000","last_peered":"2023-10-18T10:26:18.525009+0000","last_clean":"2023-10-18T10:26:18.525009+0000","last_became_active":"2023-10-18T10:26:17.501873+0000","last_became_peered":"2023-10-18T10:26:17.501873+0000","last_unstale":"2023-10-18T10:26:18.525009+0000","last_undegraded":"2023-10-18T10:26:18.525009+0000","last_fullsized":"2023-10-18T10:26:18.525009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:58.643291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195747+0000","last_change":"2023-10-18T10:26:18.211454+0000","last_active":"2023-10-18T10:26:19.195747+0000","last_peered":"2023-10-18T10:26:19.195747+0000","last_clean":"2023-10-18T10:26:19.195747+0000","last_became_active":"2023-10-18T10:26:18.210821+0000","last_became_peered":"2023-10-18T10:26:18.210821+0000","last_unstale":"2023-10-18T10:26:19.195747+0000","last_undegraded":"2023-10-18T10:26:19.195747+0000","last_fullsized":"2023-10-18T10:26:19.195747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:03.672912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.523245+0000","last_change":"2023-10-18T10:26:17.504022+0000","last_active":"2023-10-18T10:26:18.523245+0000","last_peered":"2023-10-18T10:26:18.523245+0000","last_clean":"2023-10-18T10:26:18.523245+0000","last_became_active":"2023-10-18T10:26:17.503753+0000","last_became_peered":"2023-10-18T10:26:17.503753+0000","last_unstale":"2023-10-18T10:26:18.523245+0000","last_undegraded":"2023-10-18T10:26:18.523245+0000","last_fullsized":"2023-10-18T10:26:18.523245+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:13.767227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518152+0000","last_change":"2023-10-18T10:26:18.205486+0000","last_active":"2023-10-18T10:26:19.518152+0000","last_peered":"2023-10-18T10:26:19.518152+0000","last_clean":"2023-10-18T10:26:19.518152+0000","last_became_active":"2023-10-18T10:26:18.205239+0000","last_became_peered":"2023-10-18T10:26:18.205239+0000","last_unstale":"2023-10-18T10:26:19.518152+0000","last_undegraded":"2023-10-18T10:26:19.518152+0000","last_fullsized":"2023-10-18T10:26:19.518152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:13.524299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.202680+0000","last_change":"2023-10-18T10:26:17.504149+0000","last_active":"2023-10-18T10:26:18.202680+0000","last_peered":"2023-10-18T10:26:18.202680+0000","last_clean":"2023-10-18T10:26:18.202680+0000","last_became_active":"2023-10-18T10:26:17.503861+0000","last_became_peered":"2023-10-18T10:26:17.503861+0000","last_unstale":"2023-10-18T10:26:18.202680+0000","last_undegraded":"2023-10-18T10:26:18.202680+0000","last_fullsized":"2023-10-18T10:26:18.202680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:25.799525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484238+0000","last_change":"2023-10-18T10:26:18.218061+0000","last_active":"2023-10-18T10:26:19.484238+0000","last_peered":"2023-10-18T10:26:19.484238+0000","last_clean":"2023-10-18T10:26:19.484238+0000","last_became_active":"2023-10-18T10:26:18.217548+0000","last_became_peered":"2023-10-18T10:26:18.217548+0000","last_unstale":"2023-10-18T10:26:19.484238+0000","last_undegraded":"2023-10-18T10:26:19.484238+0000","last_fullsized":"2023-10-18T10:26:19.484238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:53.789576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.522468+0000","last_change":"2023-10-18T10:26:17.498026+0000","last_active":"2023-10-18T10:26:18.522468+0000","last_peered":"2023-10-18T10:26:18.522468+0000","last_clean":"2023-10-18T10:26:18.522468+0000","last_became_active":"2023-10-18T10:26:17.497718+0000","last_became_peered":"2023-10-18T10:26:17.497718+0000","last_unstale":"2023-10-18T10:26:18.522468+0000","last_undegraded":"2023-10-18T10:26:18.522468+0000","last_fullsized":"2023-10-18T10:26:18.522468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:04.586729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199545+0000","last_change":"2023-10-18T10:26:18.213944+0000","last_active":"2023-10-18T10:26:19.199545+0000","last_peered":"2023-10-18T10:26:19.199545+0000","last_clean":"2023-10-18T10:26:19.199545+0000","last_became_active":"2023-10-18T10:26:18.213649+0000","last_became_peered":"2023-10-18T10:26:18.213649+0000","last_unstale":"2023-10-18T10:26:19.199545+0000","last_undegraded":"2023-10-18T10:26:19.199545+0000","last_fullsized":"2023-10-18T10:26:19.199545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:30.003248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.195535+0000","last_change":"2023-10-18T10:26:17.500915+0000","last_active":"2023-10-18T10:26:18.195535+0000","last_peered":"2023-10-18T10:26:18.195535+0000","last_clean":"2023-10-18T10:26:18.195535+0000","last_became_active":"2023-10-18T10:26:17.500634+0000","last_became_peered":"2023-10-18T10:26:17.500634+0000","last_unstale":"2023-10-18T10:26:18.195535+0000","last_undegraded":"2023-10-18T10:26:18.195535+0000","last_fullsized":"2023-10-18T10:26:18.195535+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:42:24.755476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518989+0000","last_change":"2023-10-18T10:26:18.209478+0000","last_active":"2023-10-18T10:26:19.518989+0000","last_peered":"2023-10-18T10:26:19.518989+0000","last_clean":"2023-10-18T10:26:19.518989+0000","last_became_active":"2023-10-18T10:26:18.209081+0000","last_became_peered":"2023-10-18T10:26:18.209081+0000","last_unstale":"2023-10-18T10:26:19.518989+0000","last_undegraded":"2023-10-18T10:26:19.518989+0000","last_fullsized":"2023-10-18T10:26:19.518989+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:15.053138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.212448+0000","last_change":"2023-10-18T10:26:17.511097+0000","last_active":"2023-10-18T10:26:18.212448+0000","last_peered":"2023-10-18T10:26:18.212448+0000","last_clean":"2023-10-18T10:26:18.212448+0000","last_became_active":"2023-10-18T10:26:17.510804+0000","last_became_peered":"2023-10-18T10:26:17.510804+0000","last_unstale":"2023-10-18T10:26:18.212448+0000","last_undegraded":"2023-10-18T10:26:18.212448+0000","last_fullsized":"2023-10-18T10:26:18.212448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:20.331358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517893+0000","last_change":"2023-10-18T10:26:18.212152+0000","last_active":"2023-10-18T10:26:19.517893+0000","last_peered":"2023-10-18T10:26:19.517893+0000","last_clean":"2023-10-18T10:26:19.517893+0000","last_became_active":"2023-10-18T10:26:18.211890+0000","last_became_peered":"2023-10-18T10:26:18.211890+0000","last_unstale":"2023-10-18T10:26:19.517893+0000","last_undegraded":"2023-10-18T10:26:19.517893+0000","last_fullsized":"2023-10-18T10:26:19.517893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:45.278732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.195526+0000","last_change":"2023-10-18T10:26:17.497412+0000","last_active":"2023-10-18T10:26:18.195526+0000","last_peered":"2023-10-18T10:26:18.195526+0000","last_clean":"2023-10-18T10:26:18.195526+0000","last_became_active":"2023-10-18T10:26:17.497133+0000","last_became_peered":"2023-10-18T10:26:17.497133+0000","last_unstale":"2023-10-18T10:26:18.195526+0000","last_undegraded":"2023-10-18T10:26:18.195526+0000","last_fullsized":"2023-10-18T10:26:18.195526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:33.389824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196890+0000","last_change":"2023-10-18T10:26:18.214525+0000","last_active":"2023-10-18T10:26:19.196890+0000","last_peered":"2023-10-18T10:26:19.196890+0000","last_clean":"2023-10-18T10:26:19.196890+0000","last_became_active":"2023-10-18T10:26:18.214253+0000","last_became_peered":"2023-10-18T10:26:18.214253+0000","last_unstale":"2023-10-18T10:26:19.196890+0000","last_undegraded":"2023-10-18T10:26:19.196890+0000","last_fullsized":"2023-10-18T10:26:19.196890+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:30:51.605675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529267+0000","last_change":"2023-10-18T10:26:17.501120+0000","last_active":"2023-10-18T10:26:18.529267+0000","last_peered":"2023-10-18T10:26:18.529267+0000","last_clean":"2023-10-18T10:26:18.529267+0000","last_became_active":"2023-10-18T10:26:17.500845+0000","last_became_peered":"2023-10-18T10:26:17.500845+0000","last_unstale":"2023-10-18T10:26:18.529267+0000","last_undegraded":"2023-10-18T10:26:18.529267+0000","last_fullsized":"2023-10-18T10:26:18.529267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:02.671116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484515+0000","last_change":"2023-10-18T10:26:18.219294+0000","last_active":"2023-10-18T10:26:19.484515+0000","last_peered":"2023-10-18T10:26:19.484515+0000","last_clean":"2023-10-18T10:26:19.484515+0000","last_became_active":"2023-10-18T10:26:18.219026+0000","last_became_peered":"2023-10-18T10:26:18.219026+0000","last_unstale":"2023-10-18T10:26:19.484515+0000","last_undegraded":"2023-10-18T10:26:19.484515+0000","last_fullsized":"2023-10-18T10:26:19.484515+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:03.950514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.195603+0000","last_change":"2023-10-18T10:26:17.504817+0000","last_active":"2023-10-18T10:26:18.195603+0000","last_peered":"2023-10-18T10:26:18.195603+0000","last_clean":"2023-10-18T10:26:18.195603+0000","last_became_active":"2023-10-18T10:26:17.504563+0000","last_became_peered":"2023-10-18T10:26:17.504563+0000","last_unstale":"2023-10-18T10:26:18.195603+0000","last_undegraded":"2023-10-18T10:26:18.195603+0000","last_fullsized":"2023-10-18T10:26:18.195603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:21.134554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518134+0000","last_change":"2023-10-18T10:26:18.209905+0000","last_active":"2023-10-18T10:26:19.518134+0000","last_peered":"2023-10-18T10:26:19.518134+0000","last_clean":"2023-10-18T10:26:19.518134+0000","last_became_active":"2023-10-18T10:26:18.209661+0000","last_became_peered":"2023-10-18T10:26:18.209661+0000","last_unstale":"2023-10-18T10:26:19.518134+0000","last_undegraded":"2023-10-18T10:26:19.518134+0000","last_fullsized":"2023-10-18T10:26:19.518134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:37.078539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.201878+0000","last_change":"2023-10-18T10:26:17.500442+0000","last_active":"2023-10-18T10:26:18.201878+0000","last_peered":"2023-10-18T10:26:18.201878+0000","last_clean":"2023-10-18T10:26:18.201878+0000","last_became_active":"2023-10-18T10:26:17.500175+0000","last_became_peered":"2023-10-18T10:26:17.500175+0000","last_unstale":"2023-10-18T10:26:18.201878+0000","last_undegraded":"2023-10-18T10:26:18.201878+0000","last_fullsized":"2023-10-18T10:26:18.201878+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:33.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199598+0000","last_change":"2023-10-18T10:26:18.209802+0000","last_active":"2023-10-18T10:26:19.199598+0000","last_peered":"2023-10-18T10:26:19.199598+0000","last_clean":"2023-10-18T10:26:19.199598+0000","last_became_active":"2023-10-18T10:26:18.209572+0000","last_became_peered":"2023-10-18T10:26:18.209572+0000","last_unstale":"2023-10-18T10:26:19.199598+0000","last_undegraded":"2023-10-18T10:26:19.199598+0000","last_fullsized":"2023-10-18T10:26:19.199598+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:18.089923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.201824+0000","last_change":"2023-10-18T10:26:17.507447+0000","last_active":"2023-10-18T10:26:18.201824+0000","last_peered":"2023-10-18T10:26:18.201824+0000","last_clean":"2023-10-18T10:26:18.201824+0000","last_became_active":"2023-10-18T10:26:17.507215+0000","last_became_peered":"2023-10-18T10:26:17.507215+0000","last_unstale":"2023-10-18T10:26:18.201824+0000","last_undegraded":"2023-10-18T10:26:18.201824+0000","last_fullsized":"2023-10-18T10:26:18.201824+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:20.331358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196770+0000","last_change":"2023-10-18T10:26:18.214539+0000","last_active":"2023-10-18T10:26:19.196770+0000","last_peered":"2023-10-18T10:26:19.196770+0000","last_clean":"2023-10-18T10:26:19.196770+0000","last_became_active":"2023-10-18T10:26:18.214295+0000","last_became_peered":"2023-10-18T10:26:18.214295+0000","last_unstale":"2023-10-18T10:26:19.196770+0000","last_undegraded":"2023-10-18T10:26:19.196770+0000","last_fullsized":"2023-10-18T10:26:19.196770+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:51.851998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.201589+0000","last_change":"2023-10-18T10:26:17.503268+0000","last_active":"2023-10-18T10:26:18.201589+0000","last_peered":"2023-10-18T10:26:18.201589+0000","last_clean":"2023-10-18T10:26:18.201589+0000","last_became_active":"2023-10-18T10:26:17.502999+0000","last_became_peered":"2023-10-18T10:26:17.502999+0000","last_unstale":"2023-10-18T10:26:18.201589+0000","last_undegraded":"2023-10-18T10:26:18.201589+0000","last_fullsized":"2023-10-18T10:26:18.201589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:12.522503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195438+0000","last_change":"2023-10-18T10:26:18.209179+0000","last_active":"2023-10-18T10:26:19.195438+0000","last_peered":"2023-10-18T10:26:19.195438+0000","last_clean":"2023-10-18T10:26:19.195438+0000","last_became_active":"2023-10-18T10:26:18.208923+0000","last_became_peered":"2023-10-18T10:26:18.208923+0000","last_unstale":"2023-10-18T10:26:19.195438+0000","last_undegraded":"2023-10-18T10:26:19.195438+0000","last_fullsized":"2023-10-18T10:26:19.195438+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:23.951622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.205301+0000","last_change":"2023-10-18T10:26:17.501090+0000","last_active":"2023-10-18T10:26:18.205301+0000","last_peered":"2023-10-18T10:26:18.205301+0000","last_clean":"2023-10-18T10:26:18.205301+0000","last_became_active":"2023-10-18T10:26:17.500760+0000","last_became_peered":"2023-10-18T10:26:17.500760+0000","last_unstale":"2023-10-18T10:26:18.205301+0000","last_undegraded":"2023-10-18T10:26:18.205301+0000","last_fullsized":"2023-10-18T10:26:18.205301+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:58.396571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484436+0000","last_change":"2023-10-18T10:26:18.221838+0000","last_active":"2023-10-18T10:26:19.484436+0000","last_peered":"2023-10-18T10:26:19.484436+0000","last_clean":"2023-10-18T10:26:19.484436+0000","last_became_active":"2023-10-18T10:26:18.221565+0000","last_became_peered":"2023-10-18T10:26:18.221565+0000","last_unstale":"2023-10-18T10:26:19.484436+0000","last_undegraded":"2023-10-18T10:26:19.484436+0000","last_fullsized":"2023-10-18T10:26:19.484436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:12.715480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.198054+0000","last_change":"2023-10-18T10:26:17.499940+0000","last_active":"2023-10-18T10:26:18.198054+0000","last_peered":"2023-10-18T10:26:18.198054+0000","last_clean":"2023-10-18T10:26:18.198054+0000","last_became_active":"2023-10-18T10:26:17.499697+0000","last_became_peered":"2023-10-18T10:26:17.499697+0000","last_unstale":"2023-10-18T10:26:18.198054+0000","last_undegraded":"2023-10-18T10:26:18.198054+0000","last_fullsized":"2023-10-18T10:26:18.198054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:30:50.603880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519543+0000","last_change":"2023-10-18T10:26:18.208638+0000","last_active":"2023-10-18T10:26:19.519543+0000","last_peered":"2023-10-18T10:26:19.519543+0000","last_clean":"2023-10-18T10:26:19.519543+0000","last_became_active":"2023-10-18T10:26:18.208358+0000","last_became_peered":"2023-10-18T10:26:18.208358+0000","last_unstale":"2023-10-18T10:26:19.519543+0000","last_undegraded":"2023-10-18T10:26:19.519543+0000","last_fullsized":"2023-10-18T10:26:19.519543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:37.078539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.212635+0000","last_change":"2023-10-18T10:26:17.506877+0000","last_active":"2023-10-18T10:26:18.212635+0000","last_peered":"2023-10-18T10:26:18.212635+0000","last_clean":"2023-10-18T10:26:18.212635+0000","last_became_active":"2023-10-18T10:26:17.506627+0000","last_became_peered":"2023-10-18T10:26:17.506627+0000","last_unstale":"2023-10-18T10:26:18.212635+0000","last_undegraded":"2023-10-18T10:26:18.212635+0000","last_fullsized":"2023-10-18T10:26:18.212635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:33.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":181,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.212076+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:26:18.212076+0000","last_peered":"2023-10-18T10:26:18.212076+0000","last_clean":"2023-10-18T10:26:18.212076+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:26:18.212076+0000","last_undegraded":"2023-10-18T10:26:18.212076+0000","last_fullsized":"2023-10-18T10:26:18.212076+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":785897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.755},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.505}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68500000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.442}]},{"osd":3,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.373}]},{"osd":4,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.499},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.32800000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":193959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.63}]},{"osd":2,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.36099999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.54200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38200000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.63600000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.69599999999999995}]},{"osd":6,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.755},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.89700000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.879}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":195493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.316}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.26200000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53900000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.438}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":193917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.245},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.375},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.38700000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33300000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":193937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.317},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33200000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.622}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.79600000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.754},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":784249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.76900000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.752}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:26:23.482 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T10:26:23.480+0000 7f94bddfa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:26:25.869 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:26:25.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 -- 172.21.15.125:0/2827923580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 msgr2=0x7f84d80ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:25.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 --2- 172.21.15.125:0/2827923580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 0x7f84d80ca870 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f84d806a780 tx=0x7f84d002c250 comp rx=0 tx=0).stop 2023-10-18T10:26:25.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 -- 172.21.15.125:0/2827923580 shutdown_connections 2023-10-18T10:26:25.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 --2- 172.21.15.125:0/2827923580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 0x7f84d80ca870 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:25.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 --2- 172.21.15.125:0/2827923580 >> v2:172.21.15.139:3300/0 conn(0x7f84d8140660 0x7f84d80c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:25.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 --2- 172.21.15.125:0/2827923580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84d813d360 0x7f84d815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:25.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 -- 172.21.15.125:0/2827923580 >> 172.21.15.125:0/2827923580 conn(0x7f84d8069930 msgr2=0x7f84d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:25.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 -- 172.21.15.125:0/2827923580 shutdown_connections 2023-10-18T10:26:25.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.982+0000 7f84df7b6640 1 -- 172.21.15.125:0/2827923580 wait complete. 2023-10-18T10:26:25.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.983+0000 7f84df7b6640 1 Processor -- start 2023-10-18T10:26:25.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.983+0000 7f84df7b6640 1 -- start start 2023-10-18T10:26:25.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84df7b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 0x7f84d80b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:25.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84df7b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84d813d360 0x7f84d81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:25.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84df7b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84d8140660 0x7f84d81e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:25.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84df7b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81efff0 con 0x7f84d80c9d00 2023-10-18T10:26:25.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84df7b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81e4e40 con 0x7f84d813d360 2023-10-18T10:26:25.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84df7b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80afe40 con 0x7f84d8140660 2023-10-18T10:26:25.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 0x7f84d80b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:25.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 0x7f84d80b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44078/0 (socket says 172.21.15.125:44078) 2023-10-18T10:26:25.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 -- 172.21.15.125:0/542837924 learned_addr learned my addr 172.21.15.125:0/542837924 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:25.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84ddd53640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84d8140660 0x7f84d81e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:25.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dcd51640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84d813d360 0x7f84d81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:25.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 -- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84d8140660 msgr2=0x7f84d81e42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:25.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84d8140660 0x7f84d81e42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:25.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 -- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84d813d360 msgr2=0x7f84d81562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:25.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84d813d360 0x7f84d81562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:25.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 -- 172.21.15.125:0/542837924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0009900 con 0x7f84d80c9d00 2023-10-18T10:26:25.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84dd552640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 0x7f84d80b0920 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f84d400d770 tx=0x7f84d400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.984+0000 7f84cdffb640 1 -- 172.21.15.125:0/542837924 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4005220 con 0x7f84d80c9d00 2023-10-18T10:26:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.985+0000 7f84cdffb640 1 -- 172.21.15.125:0/542837924 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84d40054b0 con 0x7f84d80c9d00 2023-10-18T10:26:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.985+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d80b0120 con 0x7f84d80c9d00 2023-10-18T10:26:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.985+0000 7f84cdffb640 1 -- 172.21.15.125:0/542837924 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4009a80 con 0x7f84d80c9d00 2023-10-18T10:26:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.985+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d80b0e60 con 0x7f84d80c9d00 2023-10-18T10:26:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.986+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d80b0640 con 0x7f84d80c9d00 2023-10-18T10:26:25.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.986+0000 7f84cdffb640 1 -- 172.21.15.125:0/542837924 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f84d4009be0 con 0x7f84d80c9d00 2023-10-18T10:26:25.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.987+0000 7f84cdffb640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f84bc06ae40 0x7f84bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:25.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.987+0000 7f84cdffb640 1 -- 172.21.15.125:0/542837924 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f84d4089880 con 0x7f84d80c9d00 2023-10-18T10:26:25.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.987+0000 7f84dcd51640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f84bc06ae40 0x7f84bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:25.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.987+0000 7f84dcd51640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f84bc06ae40 0x7f84bc06d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f84d81e30f0 tx=0x7f84c800a2d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:25.991+0000 7f84cdffb640 1 -- 172.21.15.125:0/542837924 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84d80b0640 con 0x7f84d80c9d00 2023-10-18T10:26:26.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.154+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84d806d170 con 0x7f84bc06ae40 2023-10-18T10:26:26.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.161+0000 7f84cdffb640 1 -- 172.21.15.125:0/542837924 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289640 (secure 0 0 0) 0x7f84c8014070 con 0x7f84bc06ae40 2023-10-18T10:26:26.166 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:26.168 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:26:26.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f84bc06ae40 msgr2=0x7f84bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f84bc06ae40 0x7f84bc06d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f84d81e30f0 tx=0x7f84c800a2d0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 msgr2=0x7f84d80b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 0x7f84d80b0920 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f84d400d770 tx=0x7f84d400db70 comp rx=0 tx=0).stop 2023-10-18T10:26:26.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 shutdown_connections 2023-10-18T10:26:26.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f84bc06ae40 0x7f84bc06d300 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84d813d360 0x7f84d81562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84d8140660 0x7f84d81e42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 --2- 172.21.15.125:0/542837924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84d80c9d00 0x7f84d80b0920 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.164+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 >> 172.21.15.125:0/542837924 conn(0x7f84d8069930 msgr2=0x7f84d813b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:26.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.165+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 shutdown_connections 2023-10-18T10:26:26.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.165+0000 7f84df7b6640 1 -- 172.21.15.125:0/542837924 wait complete. 2023-10-18T10:26:26.177 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-10-18T10:26:24.586560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220912,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753808,"statfs":{"total":773094113280,"available":772867899392,"internally_reserved":0,"allocated":4857856,"data_stored":2736332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.098178"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518358+0000","last_change":"2023-10-18T10:26:18.205457+0000","last_active":"2023-10-18T10:26:19.518358+0000","last_peered":"2023-10-18T10:26:19.518358+0000","last_clean":"2023-10-18T10:26:19.518358+0000","last_became_active":"2023-10-18T10:26:18.205208+0000","last_became_peered":"2023-10-18T10:26:18.205208+0000","last_unstale":"2023-10-18T10:26:19.518358+0000","last_undegraded":"2023-10-18T10:26:19.518358+0000","last_fullsized":"2023-10-18T10:26:19.518358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:33.118950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518029+0000","last_change":"2023-10-18T10:26:18.204439+0000","last_active":"2023-10-18T10:26:19.518029+0000","last_peered":"2023-10-18T10:26:19.518029+0000","last_clean":"2023-10-18T10:26:19.518029+0000","last_became_active":"2023-10-18T10:26:18.204188+0000","last_became_peered":"2023-10-18T10:26:18.204188+0000","last_unstale":"2023-10-18T10:26:19.518029+0000","last_undegraded":"2023-10-18T10:26:19.518029+0000","last_fullsized":"2023-10-18T10:26:19.518029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:43:19.114729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199504+0000","last_change":"2023-10-18T10:26:18.209384+0000","last_active":"2023-10-18T10:26:19.199504+0000","last_peered":"2023-10-18T10:26:19.199504+0000","last_clean":"2023-10-18T10:26:19.199504+0000","last_became_active":"2023-10-18T10:26:18.209121+0000","last_became_peered":"2023-10-18T10:26:18.209121+0000","last_unstale":"2023-10-18T10:26:19.199504+0000","last_undegraded":"2023-10-18T10:26:19.199504+0000","last_fullsized":"2023-10-18T10:26:19.199504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:39.135581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518110+0000","last_change":"2023-10-18T10:26:18.210322+0000","last_active":"2023-10-18T10:26:19.518110+0000","last_peered":"2023-10-18T10:26:19.518110+0000","last_clean":"2023-10-18T10:26:19.518110+0000","last_became_active":"2023-10-18T10:26:18.210070+0000","last_became_peered":"2023-10-18T10:26:18.210070+0000","last_unstale":"2023-10-18T10:26:19.518110+0000","last_undegraded":"2023-10-18T10:26:19.518110+0000","last_fullsized":"2023-10-18T10:26:19.518110+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:12.484868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517551+0000","last_change":"2023-10-18T10:26:18.205374+0000","last_active":"2023-10-18T10:26:19.517551+0000","last_peered":"2023-10-18T10:26:19.517551+0000","last_clean":"2023-10-18T10:26:19.517551+0000","last_became_active":"2023-10-18T10:26:18.205127+0000","last_became_peered":"2023-10-18T10:26:18.205127+0000","last_unstale":"2023-10-18T10:26:19.517551+0000","last_undegraded":"2023-10-18T10:26:19.517551+0000","last_fullsized":"2023-10-18T10:26:19.517551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:32.648102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196770+0000","last_change":"2023-10-18T10:26:18.208709+0000","last_active":"2023-10-18T10:26:19.196770+0000","last_peered":"2023-10-18T10:26:19.196770+0000","last_clean":"2023-10-18T10:26:19.196770+0000","last_became_active":"2023-10-18T10:26:18.208448+0000","last_became_peered":"2023-10-18T10:26:18.208448+0000","last_unstale":"2023-10-18T10:26:19.196770+0000","last_undegraded":"2023-10-18T10:26:19.196770+0000","last_fullsized":"2023-10-18T10:26:19.196770+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:11.946296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518568+0000","last_change":"2023-10-18T10:26:18.208759+0000","last_active":"2023-10-18T10:26:19.518568+0000","last_peered":"2023-10-18T10:26:19.518568+0000","last_clean":"2023-10-18T10:26:19.518568+0000","last_became_active":"2023-10-18T10:26:18.208466+0000","last_became_peered":"2023-10-18T10:26:18.208466+0000","last_unstale":"2023-10-18T10:26:19.518568+0000","last_undegraded":"2023-10-18T10:26:19.518568+0000","last_fullsized":"2023-10-18T10:26:19.518568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:53.789576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517719+0000","last_change":"2023-10-18T10:26:18.203793+0000","last_active":"2023-10-18T10:26:19.517719+0000","last_peered":"2023-10-18T10:26:19.517719+0000","last_clean":"2023-10-18T10:26:19.517719+0000","last_became_active":"2023-10-18T10:26:18.203052+0000","last_became_peered":"2023-10-18T10:26:18.203052+0000","last_unstale":"2023-10-18T10:26:19.517719+0000","last_undegraded":"2023-10-18T10:26:19.517719+0000","last_fullsized":"2023-10-18T10:26:19.517719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:26.845326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199599+0000","last_change":"2023-10-18T10:26:18.210375+0000","last_active":"2023-10-18T10:26:19.199599+0000","last_peered":"2023-10-18T10:26:19.199599+0000","last_clean":"2023-10-18T10:26:19.199599+0000","last_became_active":"2023-10-18T10:26:18.210111+0000","last_became_peered":"2023-10-18T10:26:18.210111+0000","last_unstale":"2023-10-18T10:26:19.199599+0000","last_undegraded":"2023-10-18T10:26:19.199599+0000","last_fullsized":"2023-10-18T10:26:19.199599+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:53.789576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519421+0000","last_change":"2023-10-18T10:26:18.215225+0000","last_active":"2023-10-18T10:26:19.519421+0000","last_peered":"2023-10-18T10:26:19.519421+0000","last_clean":"2023-10-18T10:26:19.519421+0000","last_became_active":"2023-10-18T10:26:18.214830+0000","last_became_peered":"2023-10-18T10:26:18.214830+0000","last_unstale":"2023-10-18T10:26:19.519421+0000","last_undegraded":"2023-10-18T10:26:19.519421+0000","last_fullsized":"2023-10-18T10:26:19.519421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:45.278732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519401+0000","last_change":"2023-10-18T10:26:18.205857+0000","last_active":"2023-10-18T10:26:19.519401+0000","last_peered":"2023-10-18T10:26:19.519401+0000","last_clean":"2023-10-18T10:26:19.519401+0000","last_became_active":"2023-10-18T10:26:18.205629+0000","last_became_peered":"2023-10-18T10:26:18.205629+0000","last_unstale":"2023-10-18T10:26:19.519401+0000","last_undegraded":"2023-10-18T10:26:19.519401+0000","last_fullsized":"2023-10-18T10:26:19.519401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:26.845326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199463+0000","last_change":"2023-10-18T10:26:18.208843+0000","last_active":"2023-10-18T10:26:19.199463+0000","last_peered":"2023-10-18T10:26:19.199463+0000","last_clean":"2023-10-18T10:26:19.199463+0000","last_became_active":"2023-10-18T10:26:18.208589+0000","last_became_peered":"2023-10-18T10:26:18.208589+0000","last_unstale":"2023-10-18T10:26:19.199463+0000","last_undegraded":"2023-10-18T10:26:19.199463+0000","last_fullsized":"2023-10-18T10:26:19.199463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:30:51.605675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519049+0000","last_change":"2023-10-18T10:26:18.218554+0000","last_active":"2023-10-18T10:26:19.519049+0000","last_peered":"2023-10-18T10:26:19.519049+0000","last_clean":"2023-10-18T10:26:19.519049+0000","last_became_active":"2023-10-18T10:26:18.218237+0000","last_became_peered":"2023-10-18T10:26:18.218237+0000","last_unstale":"2023-10-18T10:26:19.519049+0000","last_undegraded":"2023-10-18T10:26:19.519049+0000","last_fullsized":"2023-10-18T10:26:19.519049+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:12.715480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195327+0000","last_change":"2023-10-18T10:26:18.214467+0000","last_active":"2023-10-18T10:26:19.195327+0000","last_peered":"2023-10-18T10:26:19.195327+0000","last_clean":"2023-10-18T10:26:19.195327+0000","last_became_active":"2023-10-18T10:26:18.214198+0000","last_became_peered":"2023-10-18T10:26:18.214198+0000","last_unstale":"2023-10-18T10:26:19.195327+0000","last_undegraded":"2023-10-18T10:26:19.195327+0000","last_fullsized":"2023-10-18T10:26:19.195327+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:18.021127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518806+0000","last_change":"2023-10-18T10:26:18.205469+0000","last_active":"2023-10-18T10:26:19.518806+0000","last_peered":"2023-10-18T10:26:19.518806+0000","last_clean":"2023-10-18T10:26:19.518806+0000","last_became_active":"2023-10-18T10:26:18.205150+0000","last_became_peered":"2023-10-18T10:26:18.205150+0000","last_unstale":"2023-10-18T10:26:19.518806+0000","last_undegraded":"2023-10-18T10:26:19.518806+0000","last_fullsized":"2023-10-18T10:26:19.518806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:18.089923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484032+0000","last_change":"2023-10-18T10:26:18.219014+0000","last_active":"2023-10-18T10:26:19.484032+0000","last_peered":"2023-10-18T10:26:19.484032+0000","last_clean":"2023-10-18T10:26:19.484032+0000","last_became_active":"2023-10-18T10:26:18.218751+0000","last_became_peered":"2023-10-18T10:26:18.218751+0000","last_unstale":"2023-10-18T10:26:19.484032+0000","last_undegraded":"2023-10-18T10:26:19.484032+0000","last_fullsized":"2023-10-18T10:26:19.484032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:56.504440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484433+0000","last_change":"2023-10-18T10:26:18.220123+0000","last_active":"2023-10-18T10:26:19.484433+0000","last_peered":"2023-10-18T10:26:19.484433+0000","last_clean":"2023-10-18T10:26:19.484433+0000","last_became_active":"2023-10-18T10:26:18.219811+0000","last_became_peered":"2023-10-18T10:26:18.219811+0000","last_unstale":"2023-10-18T10:26:19.484433+0000","last_undegraded":"2023-10-18T10:26:19.484433+0000","last_fullsized":"2023-10-18T10:26:19.484433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:15.053138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518857+0000","last_change":"2023-10-18T10:26:18.212334+0000","last_active":"2023-10-18T10:26:19.518857+0000","last_peered":"2023-10-18T10:26:19.518857+0000","last_clean":"2023-10-18T10:26:19.518857+0000","last_became_active":"2023-10-18T10:26:18.212076+0000","last_became_peered":"2023-10-18T10:26:18.212076+0000","last_unstale":"2023-10-18T10:26:19.518857+0000","last_undegraded":"2023-10-18T10:26:19.518857+0000","last_fullsized":"2023-10-18T10:26:19.518857+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:26.801320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518011+0000","last_change":"2023-10-18T10:26:18.211737+0000","last_active":"2023-10-18T10:26:19.518011+0000","last_peered":"2023-10-18T10:26:19.518011+0000","last_clean":"2023-10-18T10:26:19.518011+0000","last_became_active":"2023-10-18T10:26:18.211483+0000","last_became_peered":"2023-10-18T10:26:18.211483+0000","last_unstale":"2023-10-18T10:26:19.518011+0000","last_undegraded":"2023-10-18T10:26:19.518011+0000","last_fullsized":"2023-10-18T10:26:19.518011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:13.566881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518572+0000","last_change":"2023-10-18T10:26:18.213694+0000","last_active":"2023-10-18T10:26:19.518572+0000","last_peered":"2023-10-18T10:26:19.518572+0000","last_clean":"2023-10-18T10:26:19.518572+0000","last_became_active":"2023-10-18T10:26:18.213400+0000","last_became_peered":"2023-10-18T10:26:18.213400+0000","last_unstale":"2023-10-18T10:26:19.518572+0000","last_undegraded":"2023-10-18T10:26:19.518572+0000","last_fullsized":"2023-10-18T10:26:19.518572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:18.021127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519104+0000","last_change":"2023-10-18T10:26:18.209470+0000","last_active":"2023-10-18T10:26:19.519104+0000","last_peered":"2023-10-18T10:26:19.519104+0000","last_clean":"2023-10-18T10:26:19.519104+0000","last_became_active":"2023-10-18T10:26:18.209154+0000","last_became_peered":"2023-10-18T10:26:18.209154+0000","last_unstale":"2023-10-18T10:26:19.519104+0000","last_undegraded":"2023-10-18T10:26:19.519104+0000","last_fullsized":"2023-10-18T10:26:19.519104+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:03.950514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195587+0000","last_change":"2023-10-18T10:26:18.206634+0000","last_active":"2023-10-18T10:26:19.195587+0000","last_peered":"2023-10-18T10:26:19.195587+0000","last_clean":"2023-10-18T10:26:19.195587+0000","last_became_active":"2023-10-18T10:26:18.206354+0000","last_became_peered":"2023-10-18T10:26:18.206354+0000","last_unstale":"2023-10-18T10:26:19.195587+0000","last_undegraded":"2023-10-18T10:26:19.195587+0000","last_fullsized":"2023-10-18T10:26:19.195587+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:21.333154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484112+0000","last_change":"2023-10-18T10:26:18.218218+0000","last_active":"2023-10-18T10:26:19.484112+0000","last_peered":"2023-10-18T10:26:19.484112+0000","last_clean":"2023-10-18T10:26:19.484112+0000","last_became_active":"2023-10-18T10:26:18.217962+0000","last_became_peered":"2023-10-18T10:26:18.217962+0000","last_unstale":"2023-10-18T10:26:19.484112+0000","last_undegraded":"2023-10-18T10:26:19.484112+0000","last_fullsized":"2023-10-18T10:26:19.484112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:30.003248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517498+0000","last_change":"2023-10-18T10:26:18.215015+0000","last_active":"2023-10-18T10:26:19.517498+0000","last_peered":"2023-10-18T10:26:19.517498+0000","last_clean":"2023-10-18T10:26:19.517498+0000","last_became_active":"2023-10-18T10:26:18.214756+0000","last_became_peered":"2023-10-18T10:26:18.214756+0000","last_unstale":"2023-10-18T10:26:19.517498+0000","last_undegraded":"2023-10-18T10:26:19.517498+0000","last_fullsized":"2023-10-18T10:26:19.517498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:50.047140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519206+0000","last_change":"2023-10-18T10:26:18.209245+0000","last_active":"2023-10-18T10:26:19.519206+0000","last_peered":"2023-10-18T10:26:19.519206+0000","last_clean":"2023-10-18T10:26:19.519206+0000","last_became_active":"2023-10-18T10:26:18.208964+0000","last_became_peered":"2023-10-18T10:26:18.208964+0000","last_unstale":"2023-10-18T10:26:19.519206+0000","last_undegraded":"2023-10-18T10:26:19.519206+0000","last_fullsized":"2023-10-18T10:26:19.519206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:21.333154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196982+0000","last_change":"2023-10-18T10:26:18.214786+0000","last_active":"2023-10-18T10:26:19.196982+0000","last_peered":"2023-10-18T10:26:19.196982+0000","last_clean":"2023-10-18T10:26:19.196982+0000","last_became_active":"2023-10-18T10:26:18.214535+0000","last_became_peered":"2023-10-18T10:26:18.214535+0000","last_unstale":"2023-10-18T10:26:19.196982+0000","last_undegraded":"2023-10-18T10:26:19.196982+0000","last_fullsized":"2023-10-18T10:26:19.196982+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:39.135581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518802+0000","last_change":"2023-10-18T10:26:18.207616+0000","last_active":"2023-10-18T10:26:19.518802+0000","last_peered":"2023-10-18T10:26:19.518802+0000","last_clean":"2023-10-18T10:26:19.518802+0000","last_became_active":"2023-10-18T10:26:18.207264+0000","last_became_peered":"2023-10-18T10:26:18.207264+0000","last_unstale":"2023-10-18T10:26:19.518802+0000","last_undegraded":"2023-10-18T10:26:19.518802+0000","last_fullsized":"2023-10-18T10:26:19.518802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:23.951622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519474+0000","last_change":"2023-10-18T10:26:18.208906+0000","last_active":"2023-10-18T10:26:19.519474+0000","last_peered":"2023-10-18T10:26:19.519474+0000","last_clean":"2023-10-18T10:26:19.519474+0000","last_became_active":"2023-10-18T10:26:18.208593+0000","last_became_peered":"2023-10-18T10:26:18.208593+0000","last_unstale":"2023-10-18T10:26:19.519474+0000","last_undegraded":"2023-10-18T10:26:19.519474+0000","last_fullsized":"2023-10-18T10:26:19.519474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:43:19.114729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518491+0000","last_change":"2023-10-18T10:26:18.209524+0000","last_active":"2023-10-18T10:26:19.518491+0000","last_peered":"2023-10-18T10:26:19.518491+0000","last_clean":"2023-10-18T10:26:19.518491+0000","last_became_active":"2023-10-18T10:26:18.209158+0000","last_became_peered":"2023-10-18T10:26:18.209158+0000","last_unstale":"2023-10-18T10:26:19.518491+0000","last_undegraded":"2023-10-18T10:26:19.518491+0000","last_fullsized":"2023-10-18T10:26:19.518491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:14.659949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518422+0000","last_change":"2023-10-18T10:26:18.207877+0000","last_active":"2023-10-18T10:26:19.518422+0000","last_peered":"2023-10-18T10:26:19.518422+0000","last_clean":"2023-10-18T10:26:19.518422+0000","last_became_active":"2023-10-18T10:26:18.207513+0000","last_became_peered":"2023-10-18T10:26:18.207513+0000","last_unstale":"2023-10-18T10:26:19.518422+0000","last_undegraded":"2023-10-18T10:26:19.518422+0000","last_fullsized":"2023-10-18T10:26:19.518422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:59.398367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484243+0000","last_change":"2023-10-18T10:26:18.218659+0000","last_active":"2023-10-18T10:26:19.484243+0000","last_peered":"2023-10-18T10:26:19.484243+0000","last_clean":"2023-10-18T10:26:19.484243+0000","last_became_active":"2023-10-18T10:26:18.217587+0000","last_became_peered":"2023-10-18T10:26:18.217587+0000","last_unstale":"2023-10-18T10:26:19.484243+0000","last_undegraded":"2023-10-18T10:26:19.484243+0000","last_fullsized":"2023-10-18T10:26:19.484243+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:18.089923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196898+0000","last_change":"2023-10-18T10:26:18.208494+0000","last_active":"2023-10-18T10:26:19.196898+0000","last_peered":"2023-10-18T10:26:19.196898+0000","last_clean":"2023-10-18T10:26:19.196898+0000","last_became_active":"2023-10-18T10:26:18.208232+0000","last_became_peered":"2023-10-18T10:26:18.208232+0000","last_unstale":"2023-10-18T10:26:19.196898+0000","last_undegraded":"2023-10-18T10:26:19.196898+0000","last_fullsized":"2023-10-18T10:26:19.196898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:56.504440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484548+0000","last_change":"2023-10-18T10:26:18.219843+0000","last_active":"2023-10-18T10:26:19.484548+0000","last_peered":"2023-10-18T10:26:19.484548+0000","last_clean":"2023-10-18T10:26:19.484548+0000","last_became_active":"2023-10-18T10:26:18.219471+0000","last_became_peered":"2023-10-18T10:26:18.219471+0000","last_unstale":"2023-10-18T10:26:19.484548+0000","last_undegraded":"2023-10-18T10:26:19.484548+0000","last_fullsized":"2023-10-18T10:26:19.484548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:26.845326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529465+0000","last_change":"2023-10-18T10:26:17.507234+0000","last_active":"2023-10-18T10:26:18.529465+0000","last_peered":"2023-10-18T10:26:18.529465+0000","last_clean":"2023-10-18T10:26:18.529465+0000","last_became_active":"2023-10-18T10:26:17.506966+0000","last_became_peered":"2023-10-18T10:26:17.506966+0000","last_unstale":"2023-10-18T10:26:18.529465+0000","last_undegraded":"2023-10-18T10:26:18.529465+0000","last_fullsized":"2023-10-18T10:26:18.529465+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:42.444062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.197093+0000","last_change":"2023-10-18T10:26:18.209900+0000","last_active":"2023-10-18T10:26:19.197093+0000","last_peered":"2023-10-18T10:26:19.197093+0000","last_clean":"2023-10-18T10:26:19.197093+0000","last_became_active":"2023-10-18T10:26:18.209641+0000","last_became_peered":"2023-10-18T10:26:18.209641+0000","last_unstale":"2023-10-18T10:26:19.197093+0000","last_undegraded":"2023-10-18T10:26:19.197093+0000","last_fullsized":"2023-10-18T10:26:19.197093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:30.003248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.526367+0000","last_change":"2023-10-18T10:26:17.506492+0000","last_active":"2023-10-18T10:26:18.526367+0000","last_peered":"2023-10-18T10:26:18.526367+0000","last_clean":"2023-10-18T10:26:18.526367+0000","last_became_active":"2023-10-18T10:26:17.506187+0000","last_became_peered":"2023-10-18T10:26:17.506187+0000","last_unstale":"2023-10-18T10:26:18.526367+0000","last_undegraded":"2023-10-18T10:26:18.526367+0000","last_fullsized":"2023-10-18T10:26:18.526367+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:58.396571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195774+0000","last_change":"2023-10-18T10:26:18.215771+0000","last_active":"2023-10-18T10:26:19.195774+0000","last_peered":"2023-10-18T10:26:19.195774+0000","last_clean":"2023-10-18T10:26:19.195774+0000","last_became_active":"2023-10-18T10:26:18.215516+0000","last_became_peered":"2023-10-18T10:26:18.215516+0000","last_unstale":"2023-10-18T10:26:19.195774+0000","last_undegraded":"2023-10-18T10:26:19.195774+0000","last_fullsized":"2023-10-18T10:26:19.195774+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:29:03.501058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.526260+0000","last_change":"2023-10-18T10:26:17.503565+0000","last_active":"2023-10-18T10:26:18.526260+0000","last_peered":"2023-10-18T10:26:18.526260+0000","last_clean":"2023-10-18T10:26:18.526260+0000","last_became_active":"2023-10-18T10:26:17.503338+0000","last_became_peered":"2023-10-18T10:26:17.503338+0000","last_unstale":"2023-10-18T10:26:18.526260+0000","last_undegraded":"2023-10-18T10:26:18.526260+0000","last_fullsized":"2023-10-18T10:26:18.526260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:33.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518354+0000","last_change":"2023-10-18T10:26:18.209007+0000","last_active":"2023-10-18T10:26:19.518354+0000","last_peered":"2023-10-18T10:26:19.518354+0000","last_clean":"2023-10-18T10:26:19.518354+0000","last_became_active":"2023-10-18T10:26:18.208626+0000","last_became_peered":"2023-10-18T10:26:18.208626+0000","last_unstale":"2023-10-18T10:26:19.518354+0000","last_undegraded":"2023-10-18T10:26:19.518354+0000","last_fullsized":"2023-10-18T10:26:19.518354+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:14.115689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.525197+0000","last_change":"2023-10-18T10:26:17.503503+0000","last_active":"2023-10-18T10:26:18.525197+0000","last_peered":"2023-10-18T10:26:18.525197+0000","last_clean":"2023-10-18T10:26:18.525197+0000","last_became_active":"2023-10-18T10:26:17.503216+0000","last_became_peered":"2023-10-18T10:26:17.503216+0000","last_unstale":"2023-10-18T10:26:18.525197+0000","last_undegraded":"2023-10-18T10:26:18.525197+0000","last_fullsized":"2023-10-18T10:26:18.525197+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:13.658154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199332+0000","last_change":"2023-10-18T10:26:18.210290+0000","last_active":"2023-10-18T10:26:19.199332+0000","last_peered":"2023-10-18T10:26:19.199332+0000","last_clean":"2023-10-18T10:26:19.199332+0000","last_became_active":"2023-10-18T10:26:18.210047+0000","last_became_peered":"2023-10-18T10:26:18.210047+0000","last_unstale":"2023-10-18T10:26:19.199332+0000","last_undegraded":"2023-10-18T10:26:19.199332+0000","last_fullsized":"2023-10-18T10:26:19.199332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:51.851998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529189+0000","last_change":"2023-10-18T10:26:17.500080+0000","last_active":"2023-10-18T10:26:18.529189+0000","last_peered":"2023-10-18T10:26:18.529189+0000","last_clean":"2023-10-18T10:26:18.529189+0000","last_became_active":"2023-10-18T10:26:17.499772+0000","last_became_peered":"2023-10-18T10:26:17.499772+0000","last_unstale":"2023-10-18T10:26:18.529189+0000","last_undegraded":"2023-10-18T10:26:18.529189+0000","last_fullsized":"2023-10-18T10:26:18.529189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:38.133785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518957+0000","last_change":"2023-10-18T10:26:18.210166+0000","last_active":"2023-10-18T10:26:19.518957+0000","last_peered":"2023-10-18T10:26:19.518957+0000","last_clean":"2023-10-18T10:26:19.518957+0000","last_became_active":"2023-10-18T10:26:18.209251+0000","last_became_peered":"2023-10-18T10:26:18.209251+0000","last_unstale":"2023-10-18T10:26:19.518957+0000","last_undegraded":"2023-10-18T10:26:19.518957+0000","last_fullsized":"2023-10-18T10:26:19.518957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:44.389666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.524073+0000","last_change":"2023-10-18T10:26:17.505717+0000","last_active":"2023-10-18T10:26:18.524073+0000","last_peered":"2023-10-18T10:26:18.524073+0000","last_clean":"2023-10-18T10:26:18.524073+0000","last_became_active":"2023-10-18T10:26:17.505479+0000","last_became_peered":"2023-10-18T10:26:17.505479+0000","last_unstale":"2023-10-18T10:26:18.524073+0000","last_undegraded":"2023-10-18T10:26:18.524073+0000","last_fullsized":"2023-10-18T10:26:18.524073+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:58.396571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195625+0000","last_change":"2023-10-18T10:26:18.210194+0000","last_active":"2023-10-18T10:26:19.195625+0000","last_peered":"2023-10-18T10:26:19.195625+0000","last_clean":"2023-10-18T10:26:19.195625+0000","last_became_active":"2023-10-18T10:26:18.209925+0000","last_became_peered":"2023-10-18T10:26:18.209925+0000","last_unstale":"2023-10-18T10:26:19.195625+0000","last_undegraded":"2023-10-18T10:26:19.195625+0000","last_fullsized":"2023-10-18T10:26:19.195625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:34.944286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.202498+0000","last_change":"2023-10-18T10:26:17.501283+0000","last_active":"2023-10-18T10:26:18.202498+0000","last_peered":"2023-10-18T10:26:18.202498+0000","last_clean":"2023-10-18T10:26:18.202498+0000","last_became_active":"2023-10-18T10:26:17.501020+0000","last_became_peered":"2023-10-18T10:26:17.501020+0000","last_unstale":"2023-10-18T10:26:18.202498+0000","last_undegraded":"2023-10-18T10:26:18.202498+0000","last_fullsized":"2023-10-18T10:26:18.202498+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:22.949827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517554+0000","last_change":"2023-10-18T10:26:18.213883+0000","last_active":"2023-10-18T10:26:19.517554+0000","last_peered":"2023-10-18T10:26:19.517554+0000","last_clean":"2023-10-18T10:26:19.517554+0000","last_became_active":"2023-10-18T10:26:18.213619+0000","last_became_peered":"2023-10-18T10:26:18.213619+0000","last_unstale":"2023-10-18T10:26:19.517554+0000","last_undegraded":"2023-10-18T10:26:19.517554+0000","last_fullsized":"2023-10-18T10:26:19.517554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:15.053138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.204963+0000","last_change":"2023-10-18T10:26:17.507546+0000","last_active":"2023-10-18T10:26:18.204963+0000","last_peered":"2023-10-18T10:26:18.204963+0000","last_clean":"2023-10-18T10:26:18.204963+0000","last_became_active":"2023-10-18T10:26:17.507288+0000","last_became_peered":"2023-10-18T10:26:17.507288+0000","last_unstale":"2023-10-18T10:26:18.204963+0000","last_undegraded":"2023-10-18T10:26:18.204963+0000","last_fullsized":"2023-10-18T10:26:18.204963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:42.444062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517926+0000","last_change":"2023-10-18T10:26:18.205946+0000","last_active":"2023-10-18T10:26:19.517926+0000","last_peered":"2023-10-18T10:26:19.517926+0000","last_clean":"2023-10-18T10:26:19.517926+0000","last_became_active":"2023-10-18T10:26:18.205690+0000","last_became_peered":"2023-10-18T10:26:18.205690+0000","last_unstale":"2023-10-18T10:26:19.517926+0000","last_undegraded":"2023-10-18T10:26:19.517926+0000","last_fullsized":"2023-10-18T10:26:19.517926+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:42:27.717120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.198721+0000","last_change":"2023-10-18T10:26:17.500469+0000","last_active":"2023-10-18T10:26:18.198721+0000","last_peered":"2023-10-18T10:26:18.198721+0000","last_clean":"2023-10-18T10:26:18.198721+0000","last_became_active":"2023-10-18T10:26:17.500200+0000","last_became_peered":"2023-10-18T10:26:17.500200+0000","last_unstale":"2023-10-18T10:26:18.198721+0000","last_undegraded":"2023-10-18T10:26:18.198721+0000","last_fullsized":"2023-10-18T10:26:18.198721+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:43.387870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484387+0000","last_change":"2023-10-18T10:26:18.221851+0000","last_active":"2023-10-18T10:26:19.484387+0000","last_peered":"2023-10-18T10:26:19.484387+0000","last_clean":"2023-10-18T10:26:19.484387+0000","last_became_active":"2023-10-18T10:26:18.221575+0000","last_became_peered":"2023-10-18T10:26:18.221575+0000","last_unstale":"2023-10-18T10:26:19.484387+0000","last_undegraded":"2023-10-18T10:26:19.484387+0000","last_fullsized":"2023-10-18T10:26:19.484387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:44.389666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.200370+0000","last_change":"2023-10-18T10:26:17.501638+0000","last_active":"2023-10-18T10:26:18.200370+0000","last_peered":"2023-10-18T10:26:18.200370+0000","last_clean":"2023-10-18T10:26:18.200370+0000","last_became_active":"2023-10-18T10:26:17.501382+0000","last_became_peered":"2023-10-18T10:26:17.501382+0000","last_unstale":"2023-10-18T10:26:18.200370+0000","last_undegraded":"2023-10-18T10:26:18.200370+0000","last_fullsized":"2023-10-18T10:26:18.200370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:29:02.499262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199464+0000","last_change":"2023-10-18T10:26:18.215325+0000","last_active":"2023-10-18T10:26:19.199464+0000","last_peered":"2023-10-18T10:26:19.199464+0000","last_clean":"2023-10-18T10:26:19.199464+0000","last_became_active":"2023-10-18T10:26:18.215146+0000","last_became_peered":"2023-10-18T10:26:18.215146+0000","last_unstale":"2023-10-18T10:26:19.199464+0000","last_undegraded":"2023-10-18T10:26:19.199464+0000","last_fullsized":"2023-10-18T10:26:19.199464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:56.504440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529279+0000","last_change":"2023-10-18T10:26:17.504701+0000","last_active":"2023-10-18T10:26:18.529279+0000","last_peered":"2023-10-18T10:26:18.529279+0000","last_clean":"2023-10-18T10:26:18.529279+0000","last_became_active":"2023-10-18T10:26:17.504469+0000","last_became_peered":"2023-10-18T10:26:17.504469+0000","last_unstale":"2023-10-18T10:26:18.529279+0000","last_undegraded":"2023-10-18T10:26:18.529279+0000","last_fullsized":"2023-10-18T10:26:18.529279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:30:50.603880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195569+0000","last_change":"2023-10-18T10:26:18.206715+0000","last_active":"2023-10-18T10:26:19.195569+0000","last_peered":"2023-10-18T10:26:19.195569+0000","last_clean":"2023-10-18T10:26:19.195569+0000","last_became_active":"2023-10-18T10:26:18.206445+0000","last_became_peered":"2023-10-18T10:26:18.206445+0000","last_unstale":"2023-10-18T10:26:19.195569+0000","last_undegraded":"2023-10-18T10:26:19.195569+0000","last_fullsized":"2023-10-18T10:26:19.195569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:26.801320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.531922+0000","last_change":"2023-10-18T10:26:17.506418+0000","last_active":"2023-10-18T10:26:18.531922+0000","last_peered":"2023-10-18T10:26:18.531922+0000","last_clean":"2023-10-18T10:26:18.531922+0000","last_became_active":"2023-10-18T10:26:17.506173+0000","last_became_peered":"2023-10-18T10:26:17.506173+0000","last_unstale":"2023-10-18T10:26:18.531922+0000","last_undegraded":"2023-10-18T10:26:18.531922+0000","last_fullsized":"2023-10-18T10:26:18.531922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:38.133785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199344+0000","last_change":"2023-10-18T10:26:18.209406+0000","last_active":"2023-10-18T10:26:19.199344+0000","last_peered":"2023-10-18T10:26:19.199344+0000","last_clean":"2023-10-18T10:26:19.199344+0000","last_became_active":"2023-10-18T10:26:18.209098+0000","last_became_peered":"2023-10-18T10:26:18.209098+0000","last_unstale":"2023-10-18T10:26:19.199344+0000","last_undegraded":"2023-10-18T10:26:19.199344+0000","last_fullsized":"2023-10-18T10:26:19.199344+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:11.946296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.524585+0000","last_change":"2023-10-18T10:26:17.505146+0000","last_active":"2023-10-18T10:26:18.524585+0000","last_peered":"2023-10-18T10:26:18.524585+0000","last_clean":"2023-10-18T10:26:18.524585+0000","last_became_active":"2023-10-18T10:26:17.504876+0000","last_became_peered":"2023-10-18T10:26:17.504876+0000","last_unstale":"2023-10-18T10:26:18.524585+0000","last_undegraded":"2023-10-18T10:26:18.524585+0000","last_fullsized":"2023-10-18T10:26:18.524585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:17.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517649+0000","last_change":"2023-10-18T10:26:18.205349+0000","last_active":"2023-10-18T10:26:19.517649+0000","last_peered":"2023-10-18T10:26:19.517649+0000","last_clean":"2023-10-18T10:26:19.517649+0000","last_became_active":"2023-10-18T10:26:18.205098+0000","last_became_peered":"2023-10-18T10:26:18.205098+0000","last_unstale":"2023-10-18T10:26:19.517649+0000","last_undegraded":"2023-10-18T10:26:19.517649+0000","last_fullsized":"2023-10-18T10:26:19.517649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:12.715480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.202906+0000","last_change":"2023-10-18T10:26:17.502227+0000","last_active":"2023-10-18T10:26:18.202906+0000","last_peered":"2023-10-18T10:26:18.202906+0000","last_clean":"2023-10-18T10:26:18.202906+0000","last_became_active":"2023-10-18T10:26:17.501980+0000","last_became_peered":"2023-10-18T10:26:17.501980+0000","last_unstale":"2023-10-18T10:26:18.202906+0000","last_undegraded":"2023-10-18T10:26:18.202906+0000","last_fullsized":"2023-10-18T10:26:18.202906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:20.331358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484100+0000","last_change":"2023-10-18T10:26:18.217805+0000","last_active":"2023-10-18T10:26:19.484100+0000","last_peered":"2023-10-18T10:26:19.484100+0000","last_clean":"2023-10-18T10:26:19.484100+0000","last_became_active":"2023-10-18T10:26:18.217501+0000","last_became_peered":"2023-10-18T10:26:18.217501+0000","last_unstale":"2023-10-18T10:26:19.484100+0000","last_undegraded":"2023-10-18T10:26:19.484100+0000","last_fullsized":"2023-10-18T10:26:19.484100+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:39.135581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.201827+0000","last_change":"2023-10-18T10:26:17.507336+0000","last_active":"2023-10-18T10:26:18.201827+0000","last_peered":"2023-10-18T10:26:18.201827+0000","last_clean":"2023-10-18T10:26:18.201827+0000","last_became_active":"2023-10-18T10:26:17.507076+0000","last_became_peered":"2023-10-18T10:26:17.507076+0000","last_unstale":"2023-10-18T10:26:18.201827+0000","last_undegraded":"2023-10-18T10:26:18.201827+0000","last_fullsized":"2023-10-18T10:26:18.201827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:39.362487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517674+0000","last_change":"2023-10-18T10:26:18.205381+0000","last_active":"2023-10-18T10:26:19.517674+0000","last_peered":"2023-10-18T10:26:19.517674+0000","last_clean":"2023-10-18T10:26:19.517674+0000","last_became_active":"2023-10-18T10:26:18.205143+0000","last_became_peered":"2023-10-18T10:26:18.205143+0000","last_unstale":"2023-10-18T10:26:19.517674+0000","last_undegraded":"2023-10-18T10:26:19.517674+0000","last_fullsized":"2023-10-18T10:26:19.517674+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:03.950514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529817+0000","last_change":"2023-10-18T10:26:17.505711+0000","last_active":"2023-10-18T10:26:18.529817+0000","last_peered":"2023-10-18T10:26:18.529817+0000","last_clean":"2023-10-18T10:26:18.529817+0000","last_became_active":"2023-10-18T10:26:17.504986+0000","last_became_peered":"2023-10-18T10:26:17.504986+0000","last_unstale":"2023-10-18T10:26:18.529817+0000","last_undegraded":"2023-10-18T10:26:18.529817+0000","last_fullsized":"2023-10-18T10:26:18.529817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:25.799525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518215+0000","last_change":"2023-10-18T10:26:18.207872+0000","last_active":"2023-10-18T10:26:19.518215+0000","last_peered":"2023-10-18T10:26:19.518215+0000","last_clean":"2023-10-18T10:26:19.518215+0000","last_became_active":"2023-10-18T10:26:18.207613+0000","last_became_peered":"2023-10-18T10:26:18.207613+0000","last_unstale":"2023-10-18T10:26:19.518215+0000","last_undegraded":"2023-10-18T10:26:19.518215+0000","last_fullsized":"2023-10-18T10:26:19.518215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:03:41.361726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529477+0000","last_change":"2023-10-18T10:26:17.500144+0000","last_active":"2023-10-18T10:26:18.529477+0000","last_peered":"2023-10-18T10:26:18.529477+0000","last_clean":"2023-10-18T10:26:18.529477+0000","last_became_active":"2023-10-18T10:26:17.499871+0000","last_became_peered":"2023-10-18T10:26:17.499871+0000","last_unstale":"2023-10-18T10:26:18.529477+0000","last_undegraded":"2023-10-18T10:26:18.529477+0000","last_fullsized":"2023-10-18T10:26:18.529477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:33.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519631+0000","last_change":"2023-10-18T10:26:18.210224+0000","last_active":"2023-10-18T10:26:19.519631+0000","last_peered":"2023-10-18T10:26:19.519631+0000","last_clean":"2023-10-18T10:26:19.519631+0000","last_became_active":"2023-10-18T10:26:18.209603+0000","last_became_peered":"2023-10-18T10:26:18.209603+0000","last_unstale":"2023-10-18T10:26:19.519631+0000","last_undegraded":"2023-10-18T10:26:19.519631+0000","last_fullsized":"2023-10-18T10:26:19.519631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:03:41.361726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.525009+0000","last_change":"2023-10-18T10:26:17.502150+0000","last_active":"2023-10-18T10:26:18.525009+0000","last_peered":"2023-10-18T10:26:18.525009+0000","last_clean":"2023-10-18T10:26:18.525009+0000","last_became_active":"2023-10-18T10:26:17.501873+0000","last_became_peered":"2023-10-18T10:26:17.501873+0000","last_unstale":"2023-10-18T10:26:18.525009+0000","last_undegraded":"2023-10-18T10:26:18.525009+0000","last_fullsized":"2023-10-18T10:26:18.525009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:58.643291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195747+0000","last_change":"2023-10-18T10:26:18.211454+0000","last_active":"2023-10-18T10:26:19.195747+0000","last_peered":"2023-10-18T10:26:19.195747+0000","last_clean":"2023-10-18T10:26:19.195747+0000","last_became_active":"2023-10-18T10:26:18.210821+0000","last_became_peered":"2023-10-18T10:26:18.210821+0000","last_unstale":"2023-10-18T10:26:19.195747+0000","last_undegraded":"2023-10-18T10:26:19.195747+0000","last_fullsized":"2023-10-18T10:26:19.195747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:03.672912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.523245+0000","last_change":"2023-10-18T10:26:17.504022+0000","last_active":"2023-10-18T10:26:18.523245+0000","last_peered":"2023-10-18T10:26:18.523245+0000","last_clean":"2023-10-18T10:26:18.523245+0000","last_became_active":"2023-10-18T10:26:17.503753+0000","last_became_peered":"2023-10-18T10:26:17.503753+0000","last_unstale":"2023-10-18T10:26:18.523245+0000","last_undegraded":"2023-10-18T10:26:18.523245+0000","last_fullsized":"2023-10-18T10:26:18.523245+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:13.767227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518152+0000","last_change":"2023-10-18T10:26:18.205486+0000","last_active":"2023-10-18T10:26:19.518152+0000","last_peered":"2023-10-18T10:26:19.518152+0000","last_clean":"2023-10-18T10:26:19.518152+0000","last_became_active":"2023-10-18T10:26:18.205239+0000","last_became_peered":"2023-10-18T10:26:18.205239+0000","last_unstale":"2023-10-18T10:26:19.518152+0000","last_undegraded":"2023-10-18T10:26:19.518152+0000","last_fullsized":"2023-10-18T10:26:19.518152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:13.524299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.202680+0000","last_change":"2023-10-18T10:26:17.504149+0000","last_active":"2023-10-18T10:26:18.202680+0000","last_peered":"2023-10-18T10:26:18.202680+0000","last_clean":"2023-10-18T10:26:18.202680+0000","last_became_active":"2023-10-18T10:26:17.503861+0000","last_became_peered":"2023-10-18T10:26:17.503861+0000","last_unstale":"2023-10-18T10:26:18.202680+0000","last_undegraded":"2023-10-18T10:26:18.202680+0000","last_fullsized":"2023-10-18T10:26:18.202680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:25.799525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484238+0000","last_change":"2023-10-18T10:26:18.218061+0000","last_active":"2023-10-18T10:26:19.484238+0000","last_peered":"2023-10-18T10:26:19.484238+0000","last_clean":"2023-10-18T10:26:19.484238+0000","last_became_active":"2023-10-18T10:26:18.217548+0000","last_became_peered":"2023-10-18T10:26:18.217548+0000","last_unstale":"2023-10-18T10:26:19.484238+0000","last_undegraded":"2023-10-18T10:26:19.484238+0000","last_fullsized":"2023-10-18T10:26:19.484238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:53.789576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.522468+0000","last_change":"2023-10-18T10:26:17.498026+0000","last_active":"2023-10-18T10:26:18.522468+0000","last_peered":"2023-10-18T10:26:18.522468+0000","last_clean":"2023-10-18T10:26:18.522468+0000","last_became_active":"2023-10-18T10:26:17.497718+0000","last_became_peered":"2023-10-18T10:26:17.497718+0000","last_unstale":"2023-10-18T10:26:18.522468+0000","last_undegraded":"2023-10-18T10:26:18.522468+0000","last_fullsized":"2023-10-18T10:26:18.522468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:04.586729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199545+0000","last_change":"2023-10-18T10:26:18.213944+0000","last_active":"2023-10-18T10:26:19.199545+0000","last_peered":"2023-10-18T10:26:19.199545+0000","last_clean":"2023-10-18T10:26:19.199545+0000","last_became_active":"2023-10-18T10:26:18.213649+0000","last_became_peered":"2023-10-18T10:26:18.213649+0000","last_unstale":"2023-10-18T10:26:19.199545+0000","last_undegraded":"2023-10-18T10:26:19.199545+0000","last_fullsized":"2023-10-18T10:26:19.199545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:30.003248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.195535+0000","last_change":"2023-10-18T10:26:17.500915+0000","last_active":"2023-10-18T10:26:18.195535+0000","last_peered":"2023-10-18T10:26:18.195535+0000","last_clean":"2023-10-18T10:26:18.195535+0000","last_became_active":"2023-10-18T10:26:17.500634+0000","last_became_peered":"2023-10-18T10:26:17.500634+0000","last_unstale":"2023-10-18T10:26:18.195535+0000","last_undegraded":"2023-10-18T10:26:18.195535+0000","last_fullsized":"2023-10-18T10:26:18.195535+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:42:24.755476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518989+0000","last_change":"2023-10-18T10:26:18.209478+0000","last_active":"2023-10-18T10:26:19.518989+0000","last_peered":"2023-10-18T10:26:19.518989+0000","last_clean":"2023-10-18T10:26:19.518989+0000","last_became_active":"2023-10-18T10:26:18.209081+0000","last_became_peered":"2023-10-18T10:26:18.209081+0000","last_unstale":"2023-10-18T10:26:19.518989+0000","last_undegraded":"2023-10-18T10:26:19.518989+0000","last_fullsized":"2023-10-18T10:26:19.518989+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:15.053138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.212448+0000","last_change":"2023-10-18T10:26:17.511097+0000","last_active":"2023-10-18T10:26:18.212448+0000","last_peered":"2023-10-18T10:26:18.212448+0000","last_clean":"2023-10-18T10:26:18.212448+0000","last_became_active":"2023-10-18T10:26:17.510804+0000","last_became_peered":"2023-10-18T10:26:17.510804+0000","last_unstale":"2023-10-18T10:26:18.212448+0000","last_undegraded":"2023-10-18T10:26:18.212448+0000","last_fullsized":"2023-10-18T10:26:18.212448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:20.331358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.517893+0000","last_change":"2023-10-18T10:26:18.212152+0000","last_active":"2023-10-18T10:26:19.517893+0000","last_peered":"2023-10-18T10:26:19.517893+0000","last_clean":"2023-10-18T10:26:19.517893+0000","last_became_active":"2023-10-18T10:26:18.211890+0000","last_became_peered":"2023-10-18T10:26:18.211890+0000","last_unstale":"2023-10-18T10:26:19.517893+0000","last_undegraded":"2023-10-18T10:26:19.517893+0000","last_fullsized":"2023-10-18T10:26:19.517893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:45.278732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.195526+0000","last_change":"2023-10-18T10:26:17.497412+0000","last_active":"2023-10-18T10:26:18.195526+0000","last_peered":"2023-10-18T10:26:18.195526+0000","last_clean":"2023-10-18T10:26:18.195526+0000","last_became_active":"2023-10-18T10:26:17.497133+0000","last_became_peered":"2023-10-18T10:26:17.497133+0000","last_unstale":"2023-10-18T10:26:18.195526+0000","last_undegraded":"2023-10-18T10:26:18.195526+0000","last_fullsized":"2023-10-18T10:26:18.195526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:33.389824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196890+0000","last_change":"2023-10-18T10:26:18.214525+0000","last_active":"2023-10-18T10:26:19.196890+0000","last_peered":"2023-10-18T10:26:19.196890+0000","last_clean":"2023-10-18T10:26:19.196890+0000","last_became_active":"2023-10-18T10:26:18.214253+0000","last_became_peered":"2023-10-18T10:26:18.214253+0000","last_unstale":"2023-10-18T10:26:19.196890+0000","last_undegraded":"2023-10-18T10:26:19.196890+0000","last_fullsized":"2023-10-18T10:26:19.196890+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:30:51.605675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.529267+0000","last_change":"2023-10-18T10:26:17.501120+0000","last_active":"2023-10-18T10:26:18.529267+0000","last_peered":"2023-10-18T10:26:18.529267+0000","last_clean":"2023-10-18T10:26:18.529267+0000","last_became_active":"2023-10-18T10:26:17.500845+0000","last_became_peered":"2023-10-18T10:26:17.500845+0000","last_unstale":"2023-10-18T10:26:18.529267+0000","last_undegraded":"2023-10-18T10:26:18.529267+0000","last_fullsized":"2023-10-18T10:26:18.529267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:02.671116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484515+0000","last_change":"2023-10-18T10:26:18.219294+0000","last_active":"2023-10-18T10:26:19.484515+0000","last_peered":"2023-10-18T10:26:19.484515+0000","last_clean":"2023-10-18T10:26:19.484515+0000","last_became_active":"2023-10-18T10:26:18.219026+0000","last_became_peered":"2023-10-18T10:26:18.219026+0000","last_unstale":"2023-10-18T10:26:19.484515+0000","last_undegraded":"2023-10-18T10:26:19.484515+0000","last_fullsized":"2023-10-18T10:26:19.484515+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:03.950514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.195603+0000","last_change":"2023-10-18T10:26:17.504817+0000","last_active":"2023-10-18T10:26:18.195603+0000","last_peered":"2023-10-18T10:26:18.195603+0000","last_clean":"2023-10-18T10:26:18.195603+0000","last_became_active":"2023-10-18T10:26:17.504563+0000","last_became_peered":"2023-10-18T10:26:17.504563+0000","last_unstale":"2023-10-18T10:26:18.195603+0000","last_undegraded":"2023-10-18T10:26:18.195603+0000","last_fullsized":"2023-10-18T10:26:18.195603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:21.134554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.518134+0000","last_change":"2023-10-18T10:26:18.209905+0000","last_active":"2023-10-18T10:26:19.518134+0000","last_peered":"2023-10-18T10:26:19.518134+0000","last_clean":"2023-10-18T10:26:19.518134+0000","last_became_active":"2023-10-18T10:26:18.209661+0000","last_became_peered":"2023-10-18T10:26:18.209661+0000","last_unstale":"2023-10-18T10:26:19.518134+0000","last_undegraded":"2023-10-18T10:26:19.518134+0000","last_fullsized":"2023-10-18T10:26:19.518134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:37.078539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.201878+0000","last_change":"2023-10-18T10:26:17.500442+0000","last_active":"2023-10-18T10:26:18.201878+0000","last_peered":"2023-10-18T10:26:18.201878+0000","last_clean":"2023-10-18T10:26:18.201878+0000","last_became_active":"2023-10-18T10:26:17.500175+0000","last_became_peered":"2023-10-18T10:26:17.500175+0000","last_unstale":"2023-10-18T10:26:18.201878+0000","last_undegraded":"2023-10-18T10:26:18.201878+0000","last_fullsized":"2023-10-18T10:26:18.201878+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:33.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.199598+0000","last_change":"2023-10-18T10:26:18.209802+0000","last_active":"2023-10-18T10:26:19.199598+0000","last_peered":"2023-10-18T10:26:19.199598+0000","last_clean":"2023-10-18T10:26:19.199598+0000","last_became_active":"2023-10-18T10:26:18.209572+0000","last_became_peered":"2023-10-18T10:26:18.209572+0000","last_unstale":"2023-10-18T10:26:19.199598+0000","last_undegraded":"2023-10-18T10:26:19.199598+0000","last_fullsized":"2023-10-18T10:26:19.199598+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:18.089923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.201824+0000","last_change":"2023-10-18T10:26:17.507447+0000","last_active":"2023-10-18T10:26:18.201824+0000","last_peered":"2023-10-18T10:26:18.201824+0000","last_clean":"2023-10-18T10:26:18.201824+0000","last_became_active":"2023-10-18T10:26:17.507215+0000","last_became_peered":"2023-10-18T10:26:17.507215+0000","last_unstale":"2023-10-18T10:26:18.201824+0000","last_undegraded":"2023-10-18T10:26:18.201824+0000","last_fullsized":"2023-10-18T10:26:18.201824+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:20.331358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.196770+0000","last_change":"2023-10-18T10:26:18.214539+0000","last_active":"2023-10-18T10:26:19.196770+0000","last_peered":"2023-10-18T10:26:19.196770+0000","last_clean":"2023-10-18T10:26:19.196770+0000","last_became_active":"2023-10-18T10:26:18.214295+0000","last_became_peered":"2023-10-18T10:26:18.214295+0000","last_unstale":"2023-10-18T10:26:19.196770+0000","last_undegraded":"2023-10-18T10:26:19.196770+0000","last_fullsized":"2023-10-18T10:26:19.196770+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:51.851998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.201589+0000","last_change":"2023-10-18T10:26:17.503268+0000","last_active":"2023-10-18T10:26:18.201589+0000","last_peered":"2023-10-18T10:26:18.201589+0000","last_clean":"2023-10-18T10:26:18.201589+0000","last_became_active":"2023-10-18T10:26:17.502999+0000","last_became_peered":"2023-10-18T10:26:17.502999+0000","last_unstale":"2023-10-18T10:26:18.201589+0000","last_undegraded":"2023-10-18T10:26:18.201589+0000","last_fullsized":"2023-10-18T10:26:18.201589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:12.522503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.195438+0000","last_change":"2023-10-18T10:26:18.209179+0000","last_active":"2023-10-18T10:26:19.195438+0000","last_peered":"2023-10-18T10:26:19.195438+0000","last_clean":"2023-10-18T10:26:19.195438+0000","last_became_active":"2023-10-18T10:26:18.208923+0000","last_became_peered":"2023-10-18T10:26:18.208923+0000","last_unstale":"2023-10-18T10:26:19.195438+0000","last_undegraded":"2023-10-18T10:26:19.195438+0000","last_fullsized":"2023-10-18T10:26:19.195438+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:23.951622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.205301+0000","last_change":"2023-10-18T10:26:17.501090+0000","last_active":"2023-10-18T10:26:18.205301+0000","last_peered":"2023-10-18T10:26:18.205301+0000","last_clean":"2023-10-18T10:26:18.205301+0000","last_became_active":"2023-10-18T10:26:17.500760+0000","last_became_peered":"2023-10-18T10:26:17.500760+0000","last_unstale":"2023-10-18T10:26:18.205301+0000","last_undegraded":"2023-10-18T10:26:18.205301+0000","last_fullsized":"2023-10-18T10:26:18.205301+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:58.396571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.484436+0000","last_change":"2023-10-18T10:26:18.221838+0000","last_active":"2023-10-18T10:26:19.484436+0000","last_peered":"2023-10-18T10:26:19.484436+0000","last_clean":"2023-10-18T10:26:19.484436+0000","last_became_active":"2023-10-18T10:26:18.221565+0000","last_became_peered":"2023-10-18T10:26:18.221565+0000","last_unstale":"2023-10-18T10:26:19.484436+0000","last_undegraded":"2023-10-18T10:26:19.484436+0000","last_fullsized":"2023-10-18T10:26:19.484436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:12.715480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.198054+0000","last_change":"2023-10-18T10:26:17.499940+0000","last_active":"2023-10-18T10:26:18.198054+0000","last_peered":"2023-10-18T10:26:18.198054+0000","last_clean":"2023-10-18T10:26:18.198054+0000","last_became_active":"2023-10-18T10:26:17.499697+0000","last_became_peered":"2023-10-18T10:26:17.499697+0000","last_unstale":"2023-10-18T10:26:18.198054+0000","last_undegraded":"2023-10-18T10:26:18.198054+0000","last_fullsized":"2023-10-18T10:26:18.198054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:30:50.603880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-18T10:26:19.519543+0000","last_change":"2023-10-18T10:26:18.208638+0000","last_active":"2023-10-18T10:26:19.519543+0000","last_peered":"2023-10-18T10:26:19.519543+0000","last_clean":"2023-10-18T10:26:19.519543+0000","last_became_active":"2023-10-18T10:26:18.208358+0000","last_became_peered":"2023-10-18T10:26:18.208358+0000","last_unstale":"2023-10-18T10:26:19.519543+0000","last_undegraded":"2023-10-18T10:26:19.519543+0000","last_fullsized":"2023-10-18T10:26:19.519543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:17.483887+0000","last_clean_scrub_stamp":"2023-10-18T10:26:17.483887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:37.078539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.212635+0000","last_change":"2023-10-18T10:26:17.506877+0000","last_active":"2023-10-18T10:26:18.212635+0000","last_peered":"2023-10-18T10:26:18.212635+0000","last_clean":"2023-10-18T10:26:18.212635+0000","last_became_active":"2023-10-18T10:26:17.506627+0000","last_became_peered":"2023-10-18T10:26:17.506627+0000","last_unstale":"2023-10-18T10:26:18.212635+0000","last_undegraded":"2023-10-18T10:26:18.212635+0000","last_fullsized":"2023-10-18T10:26:18.212635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:26:16.482091+0000","last_clean_scrub_stamp":"2023-10-18T10:26:16.482091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:33.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":181,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-18T10:26:18.212076+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:26:18.212076+0000","last_peered":"2023-10-18T10:26:18.212076+0000","last_clean":"2023-10-18T10:26:18.212076+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:26:18.212076+0000","last_undegraded":"2023-10-18T10:26:18.212076+0000","last_fullsized":"2023-10-18T10:26:18.212076+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":785897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.755},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.505}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68500000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.442}]},{"osd":3,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.373}]},{"osd":4,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.499},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.32800000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":193959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.63}]},{"osd":2,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.36099999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.54200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38200000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.63600000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.69599999999999995}]},{"osd":6,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.755},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.89700000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.879}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":195493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.316}]},{"osd":1,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.26200000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53900000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.438}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":193917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.245},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.375},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.38700000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.33300000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:25:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":193937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.317},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33200000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.622}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.79600000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.754},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":784249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.76900000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77700000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.752}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:26:26.183 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:26:26.183 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:26:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.325+0000 7fbe572fc640 1 -- 172.21.15.125:0/2146023707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 msgr2=0x7fbe3c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.325+0000 7fbe572fc640 1 --2- 172.21.15.125:0/2146023707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 0x7fbe3c004c40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbe40001800 tx=0x7fbe3c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.325+0000 7fbe572fc640 1 -- 172.21.15.125:0/2146023707 shutdown_connections 2023-10-18T10:26:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.325+0000 7fbe572fc640 1 --2- 172.21.15.125:0/2146023707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 0x7fbe3c004c40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.325+0000 7fbe572fc640 1 -- 172.21.15.125:0/2146023707 >> 172.21.15.125:0/2146023707 conn(0x7fbe50069930 msgr2=0x7fbe50069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.325+0000 7fbe572fc640 1 -- 172.21.15.125:0/2146023707 shutdown_connections 2023-10-18T10:26:26.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.325+0000 7fbe572fc640 1 -- 172.21.15.125:0/2146023707 wait complete. 2023-10-18T10:26:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.326+0000 7fbe572fc640 1 Processor -- start 2023-10-18T10:26:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.326+0000 7fbe572fc640 1 -- start start 2023-10-18T10:26:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe572fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 0x7fbe500b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe572fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe500a81e0 0x7fbe500a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe572fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe501f18a0 0x7fbe501f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe572fc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500a9890 con 0x7fbe3c0047c0 2023-10-18T10:26:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe572fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501f2260 con 0x7fbe500a81e0 2023-10-18T10:26:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe572fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500a8c80 con 0x7fbe501f18a0 2023-10-18T10:26:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 0x7fbe500b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 0x7fbe500b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44100/0 (socket says 172.21.15.125:44100) 2023-10-18T10:26:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 -- 172.21.15.125:0/4267995059 learned_addr learned my addr 172.21.15.125:0/4267995059 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55899640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe501f18a0 0x7fbe501f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe54897640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe500a81e0 0x7fbe500a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 -- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe501f18a0 msgr2=0x7fbe501f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe501f18a0 0x7fbe501f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 -- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe500a81e0 msgr2=0x7fbe500a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe500a81e0 0x7fbe500a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 -- 172.21.15.125:0/4267995059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500a8f60 con 0x7fbe3c0047c0 2023-10-18T10:26:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe55098640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 0x7fbe500b0b40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c011780 tx=0x7fbe4c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe45ffb640 1 -- 172.21.15.125:0/4267995059 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00fd50 con 0x7fbe3c0047c0 2023-10-18T10:26:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.327+0000 7fbe45ffb640 1 -- 172.21.15.125:0/4267995059 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe4c011e90 con 0x7fbe3c0047c0 2023-10-18T10:26:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.328+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe500a9240 con 0x7fbe3c0047c0 2023-10-18T10:26:26.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.328+0000 7fbe45ffb640 1 -- 172.21.15.125:0/4267995059 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c004250 con 0x7fbe3c0047c0 2023-10-18T10:26:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.328+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe500aa360 con 0x7fbe3c0047c0 2023-10-18T10:26:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.329+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe501eef70 con 0x7fbe3c0047c0 2023-10-18T10:26:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.329+0000 7fbe45ffb640 1 -- 172.21.15.125:0/4267995059 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbe4c0043b0 con 0x7fbe3c0047c0 2023-10-18T10:26:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.330+0000 7fbe45ffb640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbe3406ae70 0x7fbe3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.330+0000 7fbe45ffb640 1 -- 172.21.15.125:0/4267995059 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbe4c08a4b0 con 0x7fbe3c0047c0 2023-10-18T10:26:26.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.330+0000 7fbe54897640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbe3406ae70 0x7fbe3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:26.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.330+0000 7fbe54897640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbe3406ae70 0x7fbe3406d330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe40034000 tx=0x7fbe40039040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:26.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.334+0000 7fbe45ffb640 1 -- 172.21.15.125:0/4267995059 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe4c055390 con 0x7fbe3c0047c0 2023-10-18T10:26:26.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.563+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbe500690e0 con 0x7fbe3c0047c0 2023-10-18T10:26:26.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.565+0000 7fbe45ffb640 1 -- 172.21.15.125:0/4267995059 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbe4c0589b0 con 0x7fbe3c0047c0 2023-10-18T10:26:26.567 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:26:26.567 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:26:26.567 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T10:26:26.568 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T10:26:26.568 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:26:26.568 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T10:26:26.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbe3406ae70 msgr2=0x7fbe3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbe3406ae70 0x7fbe3406d330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe40034000 tx=0x7fbe40039040 comp rx=0 tx=0).stop 2023-10-18T10:26:26.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 msgr2=0x7fbe500b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 0x7fbe500b0b40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c011780 tx=0x7fbe4c011c40 comp rx=0 tx=0).stop 2023-10-18T10:26:26.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 shutdown_connections 2023-10-18T10:26:26.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fbe3406ae70 0x7fbe3406d330 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe500a81e0 0x7fbe500a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe501f18a0 0x7fbe501f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 --2- 172.21.15.125:0/4267995059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe3c0047c0 0x7fbe500b0b40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 >> 172.21.15.125:0/4267995059 conn(0x7fbe50069930 msgr2=0x7fbe50152800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:26.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 shutdown_connections 2023-10-18T10:26:26.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.567+0000 7fbe572fc640 1 -- 172.21.15.125:0/4267995059 wait complete. 2023-10-18T10:26:26.579 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:26:26.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.681+0000 7f6f127f5640 1 -- 172.21.15.125:0/617775225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 msgr2=0x7f6f0c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.681+0000 7f6f127f5640 1 --2- 172.21.15.125:0/617775225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 0x7f6f0c0703e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6f00005fd0 tx=0x7f6f0002bd90 comp rx=0 tx=0).stop 2023-10-18T10:26:26.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.681+0000 7f6f127f5640 1 -- 172.21.15.125:0/617775225 shutdown_connections 2023-10-18T10:26:26.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.681+0000 7f6f127f5640 1 --2- 172.21.15.125:0/617775225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f0c070920 0x7f6f0c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.681+0000 7f6f127f5640 1 --2- 172.21.15.125:0/617775225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 0x7f6f0c0703e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.681+0000 7f6f127f5640 1 -- 172.21.15.125:0/617775225 >> 172.21.15.125:0/617775225 conn(0x7f6f0c069930 msgr2=0x7f6f0c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:26.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.681+0000 7f6f127f5640 1 -- 172.21.15.125:0/617775225 shutdown_connections 2023-10-18T10:26:26.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.681+0000 7f6f127f5640 1 -- 172.21.15.125:0/617775225 wait complete. 2023-10-18T10:26:26.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.682+0000 7f6f127f5640 1 Processor -- start 2023-10-18T10:26:26.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f127f5640 1 -- start start 2023-10-18T10:26:26.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f127f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f0c070920 0x7f6f0c0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:26.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f127f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f0c13e520 0x7f6f0c1cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:26.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f127f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 0x7f6f0c1d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:26.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f127f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0c8f30 con 0x7f6f0c141690 2023-10-18T10:26:26.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f127f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1cfab0 con 0x7f6f0c13e520 2023-10-18T10:26:26.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f127f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1cfd90 con 0x7f6f0c070920 2023-10-18T10:26:26.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f10d92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 0x7f6f0c1d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:26.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f10d92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 0x7f6f0c1d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44124/0 (socket says 172.21.15.125:44124) 2023-10-18T10:26:26.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.683+0000 7f6f10d92640 1 -- 172.21.15.125:0/755398074 learned_addr learned my addr 172.21.15.125:0/755398074 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:26.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f0b7fe640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f0c13e520 0x7f6f0c1cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:26.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f0bfff640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f0c070920 0x7f6f0c0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:26.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f10d92640 1 -- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f0c070920 msgr2=0x7f6f0c0c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f10d92640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f0c070920 0x7f6f0c0c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f10d92640 1 -- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f0c13e520 msgr2=0x7f6f0c1cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f10d92640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f0c13e520 0x7f6f0c1cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f10d92640 1 -- 172.21.15.125:0/755398074 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f00009560 con 0x7f6f0c141690 2023-10-18T10:26:26.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f10d92640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 0x7f6f0c1d10f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400eaa0 tx=0x7f6f0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:26.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f08ff9640 1 -- 172.21.15.125:0/755398074 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400cdb0 con 0x7f6f0c141690 2023-10-18T10:26:26.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f08ff9640 1 -- 172.21.15.125:0/755398074 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6f04004b20 con 0x7f6f0c141690 2023-10-18T10:26:26.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c1d0070 con 0x7f6f0c141690 2023-10-18T10:26:26.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f08ff9640 1 -- 172.21.15.125:0/755398074 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04005020 con 0x7f6f0c141690 2023-10-18T10:26:26.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.684+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c1d0630 con 0x7f6f0c141690 2023-10-18T10:26:26.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.685+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c0cc5d0 con 0x7f6f0c141690 2023-10-18T10:26:26.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.685+0000 7f6f08ff9640 1 -- 172.21.15.125:0/755398074 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6f04005180 con 0x7f6f0c141690 2023-10-18T10:26:26.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.686+0000 7f6f08ff9640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6ef806ae40 0x7f6ef806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:26.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.686+0000 7f6f08ff9640 1 -- 172.21.15.125:0/755398074 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6f0408a150 con 0x7f6f0c141690 2023-10-18T10:26:26.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.686+0000 7f6f0bfff640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6ef806ae40 0x7f6ef806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:26.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.686+0000 7f6f0bfff640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6ef806ae40 0x7f6ef806d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6efc000f60 tx=0x7f6efc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:26.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.690+0000 7f6f08ff9640 1 -- 172.21.15.125:0/755398074 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f04055150 con 0x7f6f0c141690 2023-10-18T10:26:26.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.888+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f0c13e0d0 con 0x7f6f0c141690 2023-10-18T10:26:26.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.888+0000 7f6f08ff9640 1 -- 172.21.15.125:0/755398074 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f6f04058770 con 0x7f6f0c141690 2023-10-18T10:26:26.891 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:26:26.891 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3511459076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3511459076},{"type":"v1","addr":"172.21.15.139:6836","nonce":3511459076}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/984169411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":984169411},{"type":"v1","addr":"172.21.15.139:6837","nonce":984169411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2755714237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2755714237},{"type":"v1","addr":"172.21.15.125:6834","nonce":2755714237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:26:17.831071+0000","modified":"2023-10-18T10:26:19.850357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1934222457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1934222457},{"type":"v1","addr":"172.21.15.125:6835","nonce":1934222457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:26:26.891 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 16 2023-10-18T10:26:26.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6ef806ae40 msgr2=0x7f6ef806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6ef806ae40 0x7f6ef806d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6efc000f60 tx=0x7f6efc009040 comp rx=0 tx=0).stop 2023-10-18T10:26:26.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 msgr2=0x7f6f0c1d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:26.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 0x7f6f0c1d10f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400eaa0 tx=0x7f6f0400ef60 comp rx=0 tx=0).stop 2023-10-18T10:26:26.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 shutdown_connections 2023-10-18T10:26:26.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f6ef806ae40 0x7f6ef806d300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f0c13e520 0x7f6f0c1cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f0c141690 0x7f6f0c1d10f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 --2- 172.21.15.125:0/755398074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f0c070920 0x7f6f0c0c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:26.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 >> 172.21.15.125:0/755398074 conn(0x7f6f0c069930 msgr2=0x7f6f0c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:26.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 shutdown_connections 2023-10-18T10:26:26.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:26.890+0000 7f6f127f5640 1 -- 172.21.15.125:0/755398074 wait complete. 2023-10-18T10:26:27.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:26:17.831071+0000', 'modified': '2023-10-18T10:26:19.850357+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5697}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5697': {'gid': 5697, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/1934222457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 1934222457}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 1934222457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:26:27.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:26:27.306 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:26:27.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.408+0000 7ff1b8da9640 1 -- 172.21.15.125:0/1276896175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b4159bd0 msgr2=0x7ff1b415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.408+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/1276896175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b4159bd0 0x7ff1b415c770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8005fd0 tx=0x7ff1a802bcf0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.408+0000 7ff1b8da9640 1 -- 172.21.15.125:0/1276896175 shutdown_connections 2023-10-18T10:26:27.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.408+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/1276896175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b4159bd0 0x7ff1b415c770 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.408+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/1276896175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b406b030 0x7ff1b406b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.408+0000 7ff1b8da9640 1 -- 172.21.15.125:0/1276896175 >> 172.21.15.125:0/1276896175 conn(0x7ff1b4069930 msgr2=0x7ff1b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:27.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.408+0000 7ff1b8da9640 1 -- 172.21.15.125:0/1276896175 shutdown_connections 2023-10-18T10:26:27.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.408+0000 7ff1b8da9640 1 -- 172.21.15.125:0/1276896175 wait complete. 2023-10-18T10:26:27.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b8da9640 1 Processor -- start 2023-10-18T10:26:27.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b8da9640 1 -- start start 2023-10-18T10:26:27.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b8da9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b406b030 0x7ff1b40ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b8da9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b4159bd0 0x7ff1b414f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b8da9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b415ccb0 0x7ff1b4135b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b8da9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4152bd0 con 0x7ff1b406b030 2023-10-18T10:26:27.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b8da9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4151d00 con 0x7ff1b4159bd0 2023-10-18T10:26:27.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b8da9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4151f50 con 0x7ff1b415ccb0 2023-10-18T10:26:27.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b406b030 0x7ff1b40ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b406b030 0x7ff1b40ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44140/0 (socket says 172.21.15.125:44140) 2023-10-18T10:26:27.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.410+0000 7ff1b259c640 1 -- 172.21.15.125:0/4075089332 learned_addr learned my addr 172.21.15.125:0/4075089332 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:27.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b259c640 1 -- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b415ccb0 msgr2=0x7ff1b4135b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b2d9d640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b415ccb0 0x7ff1b4135b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b4159bd0 0x7ff1b414f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b259c640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b415ccb0 0x7ff1b4135b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b259c640 1 -- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b4159bd0 msgr2=0x7ff1b414f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b259c640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b4159bd0 0x7ff1b414f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b259c640 1 -- 172.21.15.125:0/4075089332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8009560 con 0x7ff1b406b030 2023-10-18T10:26:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b259c640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b406b030 0x7ff1b40ada60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400ba50 tx=0x7ff1a400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1a2ffd640 1 -- 172.21.15.125:0/4075089332 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400cbe0 con 0x7ff1b406b030 2023-10-18T10:26:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1a2ffd640 1 -- 172.21.15.125:0/4075089332 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1a400ce50 con 0x7ff1b406b030 2023-10-18T10:26:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b8da9640 1 -- 172.21.15.125:0/4075089332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b4152230 con 0x7ff1b406b030 2023-10-18T10:26:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1a2ffd640 1 -- 172.21.15.125:0/4075089332 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4014680 con 0x7ff1b406b030 2023-10-18T10:26:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.411+0000 7ff1b8da9640 1 -- 172.21.15.125:0/4075089332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b41527f0 con 0x7ff1b406b030 2023-10-18T10:26:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.412+0000 7ff1a2ffd640 1 -- 172.21.15.125:0/4075089332 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff1a401d3b0 con 0x7ff1b406b030 2023-10-18T10:26:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.413+0000 7ff1a2ffd640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff19406ae40 0x7ff19406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.413+0000 7ff1a2ffd640 1 -- 172.21.15.125:0/4075089332 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff1a40899b0 con 0x7ff1b406b030 2023-10-18T10:26:27.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.413+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff19406ae40 0x7ff19406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.413+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff19406ae40 0x7ff19406d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8035000 tx=0x7ff1a8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:27.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.414+0000 7ff1bba1c740 1 Processor -- start 2023-10-18T10:26:27.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.414+0000 7ff1bba1c740 1 -- start start 2023-10-18T10:26:27.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.414+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd4bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.414+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 0x55c7bbd7b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.414+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd4c020 0x55c7bbd7be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.414+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbc5d730 con 0x55c7bbd78b40 2023-10-18T10:26:27.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.414+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbcbe780 con 0x55c7bbd4c020 2023-10-18T10:26:27.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.414+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbcbbe70 con 0x55c7bbd78f20 2023-10-18T10:26:27.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd4bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd4bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44148/0 (socket says 172.21.15.125:44148) 2023-10-18T10:26:27.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 -- 172.21.15.125:0/2170902997 learned_addr learned my addr 172.21.15.125:0/2170902997 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:27.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 0x55c7bbd7b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b2d9d640 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd4c020 0x55c7bbd7be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 -- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 msgr2=0x55c7bbd7b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:26:27.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 0x55c7bbd7b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 -- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd4c020 msgr2=0x55c7bbd7be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:26:27.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd4c020 0x55c7bbd7be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 -- 172.21.15.125:0/2170902997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7bbcbc3d0 con 0x55c7bbd78b40 2023-10-18T10:26:27.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 0x55c7bbd7b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:27.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b2d9d640 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd4c020 0x55c7bbd7be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:27.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff1b259c640 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd4bae0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40603b0 tx=0x7ff1a4058720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca54b8c3d71cfaf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:27.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff193fff640 1 -- 172.21.15.125:0/2170902997 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4056980 con 0x55c7bbd78b40 2023-10-18T10:26:27.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff193fff640 1 -- 172.21.15.125:0/2170902997 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1a4054cf0 con 0x55c7bbd78b40 2023-10-18T10:26:27.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.415+0000 7ff193fff640 1 -- 172.21.15.125:0/2170902997 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4057d30 con 0x55c7bbd78b40 2023-10-18T10:26:27.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 -- 172.21.15.125:0/2170902997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 msgr2=0x55c7bbd4bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd4bae0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40603b0 tx=0x7ff1a4058720 comp rx=0 tx=0).stop 2023-10-18T10:26:27.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 -- 172.21.15.125:0/2170902997 shutdown_connections 2023-10-18T10:26:27.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd4c020 0x55c7bbd7be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 0x55c7bbd7b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/2170902997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd4bae0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 -- 172.21.15.125:0/2170902997 >> 172.21.15.125:0/2170902997 conn(0x55c7bbd4dd50 msgr2=0x55c7bbd968d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:27.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 -- 172.21.15.125:0/2170902997 shutdown_connections 2023-10-18T10:26:27.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 -- 172.21.15.125:0/2170902997 wait complete. 2023-10-18T10:26:27.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 -- start start 2023-10-18T10:26:27.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.416+0000 7ff1bba1c740 1 Processor -- start 2023-10-18T10:26:27.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78b40 0x55c7bbd8f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 0x55c7bbd8e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbcb7760 con 0x55c7bbd78f20 2023-10-18T10:26:27.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbc728f0 con 0x55c7bbda6390 2023-10-18T10:26:27.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbc72590 con 0x55c7bbd78b40 2023-10-18T10:26:27.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1bba1c740 10 client.0 _ll_register_callbacks cb 0x55c7bbcaeee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:26:27.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1b1d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1b1d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44152/0 (socket says 172.21.15.125:44152) 2023-10-18T10:26:27.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.417+0000 7ff1b1d9b640 1 -- 172.21.15.125:0/4131487412 learned_addr learned my addr 172.21.15.125:0/4131487412 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:27.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1b2d9d640 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 0x55c7bbd8e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1b259c640 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78b40 0x55c7bbd8f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1b1d9b640 1 -- 172.21.15.125:0/4131487412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78b40 msgr2=0x55c7bbd8f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:26:27.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78b40 0x55c7bbd8f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1b1d9b640 1 -- 172.21.15.125:0/4131487412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 msgr2=0x55c7bbd8e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:26:27.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 0x55c7bbd8e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1b1d9b640 1 -- 172.21.15.125:0/4131487412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b41527f0 con 0x55c7bbd78f20 2023-10-18T10:26:27.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f830 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff1a802c300 tx=0x7ff1a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe68adaa9b4b02d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:27.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1917fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:26:27.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1917fa640 1 -- 172.21.15.125:0/4131487412 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8003b40 con 0x55c7bbd78f20 2023-10-18T10:26:27.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1917fa640 1 -- 172.21.15.125:0/4131487412 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff1a8003ca0 con 0x55c7bbd78f20 2023-10-18T10:26:27.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 1 -- 172.21.15.125:0/4131487412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7bbcb0740 con 0x55c7bbd78f20 2023-10-18T10:26:27.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1917fa640 1 -- 172.21.15.125:0/4131487412 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1a8003e60 con 0x55c7bbd78f20 2023-10-18T10:26:27.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 10 client.5901 fetch_fsmap learned FSMap version 16 2023-10-18T10:26:27.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 1 -- 172.21.15.125:0/4131487412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55c7bbcb5de0 con 0x55c7bbd78f20 2023-10-18T10:26:27.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1917fa640 1 -- 172.21.15.125:0/4131487412 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55c7bbcb0740 con 0x55c7bbd78f20 2023-10-18T10:26:27.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1917fa640 1 -- 172.21.15.125:0/4131487412 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1a803c040 con 0x55c7bbd78f20 2023-10-18T10:26:27.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 10 client.5901 fetch_fsmap finished waiting for FSMap version 16 2023-10-18T10:26:27.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 10 client.5901 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:26:27.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 20 client.5901 populate_metadata read hostname 'smithi125' 2023-10-18T10:26:27.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] conn(0x55c7bbd7da40 0x55c7bbdb3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 4 client.5901 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:26:27.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.418+0000 7ff1bba1c740 1 -- 172.21.15.125:0/4131487412 --> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7bbcb0460 con 0x55c7bbd7da40 2023-10-18T10:26:27.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.419+0000 7ff1b259c640 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] conn(0x55c7bbd7da40 0x55c7bbdb3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.419+0000 7ff1917fa640 10 client.5901 ms_handle_connect on v2:172.21.15.125:6833/1934222457 2023-10-18T10:26:27.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.419+0000 7ff1b259c640 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] conn(0x55c7bbd7da40 0x55c7bbdb3c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34c67d48d3b19be server_cookie=3c0f04d90bd332a1 in_seq=0 out_seq=0 2023-10-18T10:26:27.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.419+0000 7ff1917fa640 1 -- 172.21.15.125:0/4131487412 <== mds.0 v2:172.21.15.125:6833/1934222457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c7bbcb0460 con 0x55c7bbd7da40 2023-10-18T10:26:27.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.419+0000 7ff1917fa640 10 client.5901 handle_command_reply: tid=0 2023-10-18T10:26:27.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.419+0000 7ff1bba1c740 1 client.5901 shutdown 2023-10-18T10:26:27.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.419+0000 7ff1bba1c740 1 -- 172.21.15.125:0/4131487412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78f20 msgr2=0x55c7bbd8f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.419+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f830 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff1a802c300 tx=0x7ff1a8009260 comp rx=0 tx=0).stop 2023-10-18T10:26:27.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 1 -- 172.21.15.125:0/4131487412 shutdown_connections 2023-10-18T10:26:27.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] conn(0x55c7bbd7da40 0x55c7bbdb3c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 0x55c7bbd8e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f830 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1917fa640 0 client.5901 ms_handle_reset on v2:172.21.15.125:6833/1934222457 2023-10-18T10:26:27.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/4131487412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78b40 0x55c7bbd8f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 1 -- 172.21.15.125:0/4131487412 >> 172.21.15.125:0/4131487412 conn(0x55c7bbd4c020 msgr2=0x55c7bbd4f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:27.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 1 -- 172.21.15.125:0/4131487412 shutdown_connections 2023-10-18T10:26:27.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 1 -- 172.21.15.125:0/4131487412 wait complete. 2023-10-18T10:26:27.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.420+0000 7ff1bba1c740 20 client.5901 trim_cache size 0 max 16384 2023-10-18T10:26:27.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.435+0000 7ff1bba1c740 1 Processor -- start 2023-10-18T10:26:27.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.435+0000 7ff1bba1c740 1 -- start start 2023-10-18T10:26:27.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 0x55c7bbd75430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 0x55c7bbd99fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbc38270 con 0x55c7bbd78b40 2023-10-18T10:26:27.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbdb7ee0 con 0x55c7bbda6390 2023-10-18T10:26:27.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbd478c0 con 0x55c7bbd78f20 2023-10-18T10:26:27.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44164/0 (socket says 172.21.15.125:44164) 2023-10-18T10:26:27.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 -- 172.21.15.125:0/1621144337 learned_addr learned my addr 172.21.15.125:0/1621144337 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:27.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 -- 172.21.15.125:0/1621144337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 msgr2=0x55c7bbd75430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:26:27.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 --2- 172.21.15.125:0/1621144337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 0x55c7bbd75430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 -- 172.21.15.125:0/1621144337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 msgr2=0x55c7bbd99fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:26:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 --2- 172.21.15.125:0/1621144337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 0x55c7bbd99fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 -- 172.21.15.125:0/1621144337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7bbc54d00 con 0x55c7bbd78b40 2023-10-18T10:26:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1b259c640 1 --2- 172.21.15.125:0/1621144337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8fb70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4060aa0 tx=0x7ff1a4058720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc85038711948b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1a0ff9640 1 -- 172.21.15.125:0/1621144337 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4075340 con 0x55c7bbd78b40 2023-10-18T10:26:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1a0ff9640 1 -- 172.21.15.125:0/1621144337 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1a40754a0 con 0x55c7bbd78b40 2023-10-18T10:26:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.436+0000 7ff1a0ff9640 1 -- 172.21.15.125:0/1621144337 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4057d30 con 0x55c7bbd78b40 2023-10-18T10:26:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 -- 172.21.15.125:0/1621144337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 msgr2=0x55c7bbd8fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/1621144337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8fb70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4060aa0 tx=0x7ff1a4058720 comp rx=0 tx=0).stop 2023-10-18T10:26:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 -- 172.21.15.125:0/1621144337 shutdown_connections 2023-10-18T10:26:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/1621144337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbda6390 0x55c7bbd99fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/1621144337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbd78f20 0x55c7bbd75430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/1621144337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8fb70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 -- 172.21.15.125:0/1621144337 >> 172.21.15.125:0/1621144337 conn(0x55c7bbd4c020 msgr2=0x55c7bbd4f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:27.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 -- 172.21.15.125:0/1621144337 shutdown_connections 2023-10-18T10:26:27.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.437+0000 7ff1bba1c740 1 -- 172.21.15.125:0/1621144337 wait complete. 2023-10-18T10:26:27.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.438+0000 7ff1bba1c740 1 -- start start 2023-10-18T10:26:27.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.438+0000 7ff1bba1c740 1 Processor -- start 2023-10-18T10:26:27.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.438+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.438+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.438+0000 7ff1bba1c740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbda6390 0x55c7bbdf4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbd76b90 con 0x55c7bbd78b40 2023-10-18T10:26:27.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbd78320 con 0x55c7bbd78f20 2023-10-18T10:26:27.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7bbd7c4f0 con 0x55c7bbda6390 2023-10-18T10:26:27.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 10 client.0 _ll_register_callbacks cb 0x7ff1a4058120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:26:27.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44166/0 (socket says 172.21.15.125:44166) 2023-10-18T10:26:27.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 -- 172.21.15.125:0/3805083292 learned_addr learned my addr 172.21.15.125:0/3805083292 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:27.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b2d9d640 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbda6390 0x55c7bbdf4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 -- 172.21.15.125:0/3805083292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbda6390 msgr2=0x55c7bbdf4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:26:27.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbda6390 0x55c7bbdf4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 -- 172.21.15.125:0/3805083292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd78f20 msgr2=0x55c7bbd8f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:26:27.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 -- 172.21.15.125:0/3805083292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7bbcbc3d0 con 0x55c7bbd78b40 2023-10-18T10:26:27.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1b259c640 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8f220 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40582e0 tx=0x7ff1a4058050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68068d016c4183a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:27.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1917fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:26:27.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1917fa640 1 -- 172.21.15.125:0/3805083292 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4057cd0 con 0x55c7bbd78b40 2023-10-18T10:26:27.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1917fa640 1 -- 172.21.15.125:0/3805083292 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff1a4086070 con 0x55c7bbd78b40 2023-10-18T10:26:27.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 1 -- 172.21.15.125:0/3805083292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1a4054cf0 con 0x55c7bbd78b40 2023-10-18T10:26:27.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1917fa640 1 -- 172.21.15.125:0/3805083292 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1a4060450 con 0x55c7bbd78b40 2023-10-18T10:26:27.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 10 client.5907 fetch_fsmap learned FSMap version 16 2023-10-18T10:26:27.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 1 -- 172.21.15.125:0/3805083292 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55c7bbc7a4f0 con 0x55c7bbd78b40 2023-10-18T10:26:27.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1917fa640 1 -- 172.21.15.125:0/3805083292 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1a4054cf0 con 0x55c7bbd78b40 2023-10-18T10:26:27.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1917fa640 1 -- 172.21.15.125:0/3805083292 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1a4060b30 con 0x55c7bbd78b40 2023-10-18T10:26:27.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 10 client.5907 fetch_fsmap finished waiting for FSMap version 16 2023-10-18T10:26:27.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.439+0000 7ff1bba1c740 10 client.5907 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:26:27.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 20 client.5907 populate_metadata read hostname 'smithi125' 2023-10-18T10:26:27.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] conn(0x55c7bbda5040 0x55c7bbda5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 4 client.5907 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:26:27.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 -- 172.21.15.125:0/3805083292 --> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c7bb9b96c0 con 0x55c7bbda5040 2023-10-18T10:26:27.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] conn(0x55c7bbda5040 0x55c7bbda5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1917fa640 10 client.5907 ms_handle_connect on v2:172.21.15.125:6833/1934222457 2023-10-18T10:26:27.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1b1d9b640 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] conn(0x55c7bbda5040 0x55c7bbda5420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3899cfd51450e3 server_cookie=e53506f9e3e96ddd in_seq=0 out_seq=0 2023-10-18T10:26:27.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1917fa640 1 -- 172.21.15.125:0/3805083292 <== mds.0 v2:172.21.15.125:6833/1934222457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c7bb9b96c0 con 0x55c7bbda5040 2023-10-18T10:26:27.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1917fa640 10 client.5907 handle_command_reply: tid=0 2023-10-18T10:26:27.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 client.5907 shutdown 2023-10-18T10:26:27.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 -- 172.21.15.125:0/3805083292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 msgr2=0x55c7bbd8f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8f220 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40582e0 tx=0x7ff1a4058050 comp rx=0 tx=0).stop 2023-10-18T10:26:27.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 -- 172.21.15.125:0/3805083292 shutdown_connections 2023-10-18T10:26:27.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.125:6833/1934222457,v1:172.21.15.125:6835/1934222457] conn(0x55c7bbda5040 0x55c7bbda5420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c7bbd78f20 0x55c7bbd8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c7bbda6390 0x55c7bbdf4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 --2- 172.21.15.125:0/3805083292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7bbd78b40 0x55c7bbd8f220 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1917fa640 0 client.5907 ms_handle_reset on v2:172.21.15.125:6833/1934222457 2023-10-18T10:26:27.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 -- 172.21.15.125:0/3805083292 >> 172.21.15.125:0/3805083292 conn(0x55c7bbd4c020 msgr2=0x55c7bbdb7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:27.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 -- 172.21.15.125:0/3805083292 shutdown_connections 2023-10-18T10:26:27.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 1 -- 172.21.15.125:0/3805083292 wait complete. 2023-10-18T10:26:27.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.440+0000 7ff1bba1c740 20 client.5907 trim_cache size 0 max 16384 2023-10-18T10:26:27.464 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:26:27.464 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:26:27.465 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:26:27.465 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 5697, 2023-10-18T10:26:27.465 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:26:27.465 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:26:27.465 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:26:27.465 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.9252437330000003, 2023-10-18T10:26:27.466 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 16, 2023-10-18T10:26:27.466 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 39, 2023-10-18T10:26:27.466 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 39, 2023-10-18T10:26:27.466 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 15.823210574999999 2023-10-18T10:26:27.466 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:26:27.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 -- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff19406ae40 msgr2=0x7ff19406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff19406ae40 0x7ff19406d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8035000 tx=0x7ff1a8004730 comp rx=0 tx=0).stop 2023-10-18T10:26:27.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 -- 172.21.15.125:0/4075089332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b406b030 msgr2=0x7ff1b40ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b406b030 0x7ff1b40ada60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400ba50 tx=0x7ff1a400bf10 comp rx=0 tx=0).stop 2023-10-18T10:26:27.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 -- 172.21.15.125:0/4075089332 shutdown_connections 2023-10-18T10:26:27.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7ff19406ae40 0x7ff19406d300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1b4159bd0 0x7ff1b414f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1b415ccb0 0x7ff1b4135b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 --2- 172.21.15.125:0/4075089332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1b406b030 0x7ff1b40ada60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 -- 172.21.15.125:0/4075089332 >> 172.21.15.125:0/4075089332 conn(0x7ff1b4069930 msgr2=0x7ff1b415c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:27.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 -- 172.21.15.125:0/4075089332 shutdown_connections 2023-10-18T10:26:27.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.441+0000 7ff1b8da9640 1 -- 172.21.15.125:0/4075089332 wait complete. 2023-10-18T10:26:27.471 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:26:27.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.572+0000 7f9946c75640 1 -- 172.21.15.125:0/1054610509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940070af0 msgr2=0x7f99400affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.572+0000 7f9946c75640 1 --2- 172.21.15.125:0/1054610509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940070af0 0x7f99400affe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9930005fd0 tx=0x7f993002bd90 comp rx=0 tx=0).stop 2023-10-18T10:26:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.572+0000 7f9946c75640 1 -- 172.21.15.125:0/1054610509 shutdown_connections 2023-10-18T10:26:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.572+0000 7f9946c75640 1 --2- 172.21.15.125:0/1054610509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940070af0 0x7f99400affe0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.572+0000 7f9946c75640 1 --2- 172.21.15.125:0/1054610509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f994013e520 0x7f994006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.572+0000 7f9946c75640 1 -- 172.21.15.125:0/1054610509 >> 172.21.15.125:0/1054610509 conn(0x7f9940069930 msgr2=0x7f9940069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.572+0000 7f9946c75640 1 -- 172.21.15.125:0/1054610509 shutdown_connections 2023-10-18T10:26:27.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.572+0000 7f9946c75640 1 -- 172.21.15.125:0/1054610509 wait complete. 2023-10-18T10:26:27.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.573+0000 7f9946c75640 1 Processor -- start 2023-10-18T10:26:27.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.573+0000 7f9946c75640 1 -- start start 2023-10-18T10:26:27.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9946c75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9940070af0 0x7f99401d4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9946c75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994013e520 0x7f99401da360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9946c75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940141690 0x7f99401da8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9946c75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401e0a40 con 0x7f9940141690 2023-10-18T10:26:27.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9946c75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401e12e0 con 0x7f994013e520 2023-10-18T10:26:27.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9946c75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99401dade0 con 0x7f9940070af0 2023-10-18T10:26:27.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9945212640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940141690 0x7f99401da8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9945212640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940141690 0x7f99401da8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43088/0 (socket says 172.21.15.125:43088) 2023-10-18T10:26:27.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9945212640 1 -- 172.21.15.125:0/1857011755 learned_addr learned my addr 172.21.15.125:0/1857011755 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:27.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9945212640 1 -- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9940070af0 msgr2=0x7f99401d4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9944a11640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9940070af0 0x7f99401d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f993ffff640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994013e520 0x7f99401da360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9945212640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9940070af0 0x7f99401d4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9945212640 1 -- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994013e520 msgr2=0x7f99401da360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9945212640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994013e520 0x7f99401da360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.574+0000 7f9945212640 1 -- 172.21.15.125:0/1857011755 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930009560 con 0x7f9940141690 2023-10-18T10:26:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.575+0000 7f9945212640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940141690 0x7f99401da8a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9930036040 tx=0x7f993002eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.575+0000 7f993d7fa640 1 -- 172.21.15.125:0/1857011755 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300035f0 con 0x7f9940141690 2023-10-18T10:26:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.575+0000 7f993d7fa640 1 -- 172.21.15.125:0/1857011755 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9930003860 con 0x7f9940141690 2023-10-18T10:26:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.575+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99401d4d70 con 0x7f9940141690 2023-10-18T10:26:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.575+0000 7f993d7fa640 1 -- 172.21.15.125:0/1857011755 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930003c70 con 0x7f9940141690 2023-10-18T10:26:27.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.575+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99401e0c20 con 0x7f9940141690 2023-10-18T10:26:27.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.576+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99401e1420 con 0x7f9940141690 2023-10-18T10:26:27.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.576+0000 7f993d7fa640 1 -- 172.21.15.125:0/1857011755 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9930034440 con 0x7f9940141690 2023-10-18T10:26:27.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.576+0000 7f993d7fa640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae70 0x7f992c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:27.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.577+0000 7f993d7fa640 1 -- 172.21.15.125:0/1857011755 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f99300ae2d0 con 0x7f9940141690 2023-10-18T10:26:27.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.577+0000 7f9944a11640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae70 0x7f992c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:27.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.577+0000 7f9944a11640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae70 0x7f992c06d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9938005fd0 tx=0x7f9938009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:27.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.581+0000 7f993d7fa640 1 -- 172.21.15.125:0/1857011755 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99401e1420 con 0x7f9940141690 2023-10-18T10:26:27.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.785+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99401e1420 con 0x7f9940141690 2023-10-18T10:26:27.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.785+0000 7f993d7fa640 1 -- 172.21.15.125:0/1857011755 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f99401e1420 con 0x7f9940141690 2023-10-18T10:26:27.788 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:26:27.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae70 msgr2=0x7f992c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae70 0x7f992c06d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9938005fd0 tx=0x7f9938009500 comp rx=0 tx=0).stop 2023-10-18T10:26:27.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940141690 msgr2=0x7f99401da8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:27.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940141690 0x7f99401da8a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9930036040 tx=0x7f993002eb70 comp rx=0 tx=0).stop 2023-10-18T10:26:27.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 shutdown_connections 2023-10-18T10:26:27.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f992c06ae70 0x7f992c06d330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f994013e520 0x7f99401da360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9940141690 0x7f99401da8a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 --2- 172.21.15.125:0/1857011755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9940070af0 0x7f99401d4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:27.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.787+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 >> 172.21.15.125:0/1857011755 conn(0x7f9940069930 msgr2=0x7f99400b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:27.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.788+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 shutdown_connections 2023-10-18T10:26:27.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:27.788+0000 7f9946c75640 1 -- 172.21.15.125:0/1857011755 wait complete. 2023-10-18T10:26:27.800 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:26:27.800 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:26:27.800 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:26:27.801 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:26:27.801 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:26:27.801 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:26:27.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:26:27.802 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:26:27.802 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:26:27.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:27.803 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:26:27.818 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:26:27.819 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:26:27.819 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:26:27.819 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:26:27.819 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:26:27.819 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:26:27.820 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:26:27.820 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:26:27.820 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:26:27.820 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:26:27.820 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:26:27.821 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:26:27.821 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:26:27.821 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:26:27.821 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:26:27.821 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:26:27.821 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:26:27.822 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 42636sec preferred_lft 42636sec 2023-10-18T10:26:27.822 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:26:27.822 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:26:27.822 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:26:27.822 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:26:27.823 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:26:27.823 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:26:27.823 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T10:26:27.823 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:26:27.823 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:26:27.824 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:26:27.824 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T10:26:27.824 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:26:27.824 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:26:27.824 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:26:27.824 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T10:26:27.825 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:26:27.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:27.825 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:26:27.884 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:26:27.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:27.886 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:26:27.941 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:26:27.941 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:26:27.941 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:26:27.942 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:26:27.942 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:26:27.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:27.943 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:26:27.943 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:26:27.943 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:26:27.944 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:26:27.944 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:26:27.944 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:26:28.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:28.037 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:26:28.052 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:26:28.052 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:26:28.053 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:26:28.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:28.054 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:26:28.119 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:26:28.120 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:26:28.120 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:26:28.174 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:26:28.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:26:28.175 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:26:28.229 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:26:28.295 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:26:28.351 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:26:28.351 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:26:28.351 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:26:28.351 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:26:28.352 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:26:28.352 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:26:28.352 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:26:28.352 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:26:28.353 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:26:28.353 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:26:28.353 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:26:28.353 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:26:28.353 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:26:28.354 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:26:28.354 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:26:28.354 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:26:28.354 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:26:28.355 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:26:28.355 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:26:28.355 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:26:28.355 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:26:28.355 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:26:28.356 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:26:28.356 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:26:28.356 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:26:28.356 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:26:28.356 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:26:28.357 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:26:28.357 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:26:28.357 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:26:28.357 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:26:28.357 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:26:28.357 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:26:28.358 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:26:28.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:28.358 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:26:28.405 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:26:28.406 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:26:28.448 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:26:28.474 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:26:28.529 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:26:28.529 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:26:28.529 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:26:28.530 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:26:28.530 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:26:28.530 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:26:28.530 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:26:28.530 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:26:28.531 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:26:28.531 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:26:28.531 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:26:28.531 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:26:28.531 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:26:28.532 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:26:28.532 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:26:28.532 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:26:28.532 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:26:28.532 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:26:28.533 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:26:28.533 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:26:28.533 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:26:28.533 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:26:28.533 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:26:28.534 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:26:28.534 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:26:28.534 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:26:28.534 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:26:28.534 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:26:28.535 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:26:28.535 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:26:28.535 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:26:28.535 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:26:28.535 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:26:28.535 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:26:28.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:28.536 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:26:28.537 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:26:28.531+0000 7f7658585480 -1 init, newargv = 0x5599c6087be0 newargc=15 2023-10-18T10:26:28.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[106073]: starting ceph client 2023-10-18T10:26:28.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[106073]: starting fuse 2023-10-18T10:26:28.551 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:26:28.552 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:26:28.552 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:26:28.552 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:26:28.553 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:26:28.553 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:26:28.554 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:26:28.554 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:26:28.554 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:26:28.555 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:26:28.555 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:26:28.556 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:26:28.556 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:26:28.556 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:26:28.557 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:26:28.557 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:26:28.557 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:26:28.558 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:26:28.558 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:26:28.559 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:26:28.559 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:26:28.559 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:26:28.560 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:26:28.560 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:26:28.560 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:26:28.561 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:26:28.561 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:26:28.561 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:26:28.562 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:26:28.672 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.106073.asok 2023-10-18T10:26:28.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:26:28.682 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106073.asok 2023-10-18T10:26:28.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:28.683 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106073.asok status 2023-10-18T10:26:28.817 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:26:28.818 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:26:28.818 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:26:28.818 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:26:28.818 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:26:28.818 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:26:28.819 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:26:28.819 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "106073", 2023-10-18T10:26:28.819 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:26:28.819 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:28.819 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:26:28.820 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:26:28.820 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 5916, 2023-10-18T10:26:28.820 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:26:28.820 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:26:28.820 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:26:28.821 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 5916 2023-10-18T10:26:28.821 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:28.821 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:26:28.821 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:26:28.821 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:26:28.822 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 672558083 2023-10-18T10:26:28.822 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:28.822 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:28.822 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:26:28.822 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:26:28.823 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:26:28.823 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 672558083 2023-10-18T10:26:28.823 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:26:28.823 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.5916 192.168.0.1:0/672558083", 2023-10-18T10:26:28.823 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/672558083", 2023-10-18T10:26:28.823 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:26:28.824 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 16, 2023-10-18T10:26:28.824 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 39, 2023-10-18T10:26:28.824 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:26:28.824 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:26:28.824 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:26:28.825 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:26:28.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:28.826 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:26:28.827 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:26:28.841 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:26:28.842 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:26:28.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:28.842 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:26:28.909 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T10:26:29.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.050+0000 7fc439bd0640 1 -- 172.21.15.125:0/2793852555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc41c0039a0 msgr2=0x7fc41c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.050+0000 7fc439bd0640 1 --2- 172.21.15.125:0/2793852555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc41c0039a0 0x7fc41c003dc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc42400bb20 tx=0x7fc41c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.050+0000 7fc433fff640 1 -- 172.21.15.125:0/2793852555 reap_dead start 2023-10-18T10:26:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.050+0000 7fc439bd0640 1 -- 172.21.15.125:0/2793852555 shutdown_connections 2023-10-18T10:26:29.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.050+0000 7fc439bd0640 1 -- 172.21.15.125:0/2793852555 >> 172.21.15.125:0/2793852555 conn(0x7fc4340d0d60 msgr2=0x7fc4340d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:29.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.050+0000 7fc439bd0640 1 -- 172.21.15.125:0/2793852555 shutdown_connections 2023-10-18T10:26:29.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.050+0000 7fc439bd0640 1 -- 172.21.15.125:0/2793852555 wait complete. 2023-10-18T10:26:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.051+0000 7fc439bd0640 1 Processor -- start 2023-10-18T10:26:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc439bd0640 1 -- start start 2023-10-18T10:26:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc439bd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41c0039a0 0x7fc434154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc439bd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc43415dde0 0x7fc43415e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc439bd0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4341f54c0 0x7fc4341f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:29.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc439bd0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4341559d0 con 0x7fc43415dde0 2023-10-18T10:26:29.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc439bd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43414f840 con 0x7fc41c0039a0 2023-10-18T10:26:29.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc439bd0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc434155410 con 0x7fc4341f54c0 2023-10-18T10:26:29.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc432ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc43415dde0 0x7fc43415e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:29.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc432ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc43415dde0 0x7fc43415e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43112/0 (socket says 172.21.15.125:43112) 2023-10-18T10:26:29.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.052+0000 7fc432ffd640 1 -- 172.21.15.125:0/1578207012 learned_addr learned my addr 172.21.15.125:0/1578207012 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:29.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc4337fe640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41c0039a0 0x7fc434154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:29.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc433fff640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4341f54c0 0x7fc4341f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:29.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc432ffd640 1 -- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4341f54c0 msgr2=0x7fc4341f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:29.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc432ffd640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4341f54c0 0x7fc4341f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc432ffd640 1 -- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41c0039a0 msgr2=0x7fc434154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:29.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc432ffd640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41c0039a0 0x7fc434154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc432ffd640 1 -- 172.21.15.125:0/1578207012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4341556f0 con 0x7fc43415dde0 2023-10-18T10:26:29.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc432ffd640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc43415dde0 0x7fc43415e260 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc428004d60 tx=0x7fc428001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:29.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc41bfff640 1 -- 172.21.15.125:0/1578207012 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280107d0 con 0x7fc43415dde0 2023-10-18T10:26:29.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc41bfff640 1 -- 172.21.15.125:0/1578207012 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc428010950 con 0x7fc43415dde0 2023-10-18T10:26:29.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4340b3a80 con 0x7fc43415dde0 2023-10-18T10:26:29.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc41bfff640 1 -- 172.21.15.125:0/1578207012 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc428010d60 con 0x7fc43415dde0 2023-10-18T10:26:29.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.053+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43415e7a0 con 0x7fc43415dde0 2023-10-18T10:26:29.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.054+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4340690e0 con 0x7fc43415dde0 2023-10-18T10:26:29.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.054+0000 7fc41bfff640 1 -- 172.21.15.125:0/1578207012 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc42801b460 con 0x7fc43415dde0 2023-10-18T10:26:29.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.055+0000 7fc41bfff640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc42006ae40 0x7fc42006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:29.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.055+0000 7fc41bfff640 1 -- 172.21.15.125:0/1578207012 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc428089f90 con 0x7fc43415dde0 2023-10-18T10:26:29.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.055+0000 7fc4337fe640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc42006ae40 0x7fc42006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:29.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.056+0000 7fc4337fe640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc42006ae40 0x7fc42006d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc424037000 tx=0x7fc42402e040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:29.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.059+0000 7fc41bfff640 1 -- 172.21.15.125:0/1578207012 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4280bcca0 con 0x7fc43415dde0 2023-10-18T10:26:29.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.246+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4340af230 con 0x7fc42006ae40 2023-10-18T10:26:29.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.248+0000 7fc41bfff640 1 -- 172.21.15.125:0/1578207012 <== mgr.4109 v2:172.21.15.139:6832/2976686360 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc4340af230 con 0x7fc42006ae40 2023-10-18T10:26:29.250 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:26:29.250 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:26:29.250 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T10:26:29.250 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:26:29.251 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:26:29.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.249+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc42006ae40 msgr2=0x7fc42006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:29.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.249+0000 7fc439bd0640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc42006ae40 0x7fc42006d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc424037000 tx=0x7fc42402e040 comp rx=0 tx=0).stop 2023-10-18T10:26:29.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.249+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc43415dde0 msgr2=0x7fc43415e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:29.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.249+0000 7fc439bd0640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc43415dde0 0x7fc43415e260 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc428004d60 tx=0x7fc428001630 comp rx=0 tx=0).stop 2023-10-18T10:26:29.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.250+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 shutdown_connections 2023-10-18T10:26:29.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.250+0000 7fc439bd0640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc42006ae40 0x7fc42006d300 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.250+0000 7fc439bd0640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4341f54c0 0x7fc4341f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.250+0000 7fc439bd0640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc43415dde0 0x7fc43415e260 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.250+0000 7fc439bd0640 1 --2- 172.21.15.125:0/1578207012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41c0039a0 0x7fc434154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.250+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 >> 172.21.15.125:0/1578207012 conn(0x7fc4340d0d60 msgr2=0x7fc43415a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:29.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.250+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 shutdown_connections 2023-10-18T10:26:29.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.250+0000 7fc439bd0640 1 -- 172.21.15.125:0/1578207012 wait complete. 2023-10-18T10:26:29.262 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T10:26:29.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.364+0000 7faf5052c640 1 -- 172.21.15.125:0/4236458833 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf480cf340 msgr2=0x7faf480d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:29.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.364+0000 7faf5052c640 1 --2- 172.21.15.125:0/4236458833 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf480cf340 0x7faf480d1910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faf40005fd0 tx=0x7faf4002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.364+0000 7faf5052c640 1 -- 172.21.15.125:0/4236458833 shutdown_connections 2023-10-18T10:26:29.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.364+0000 7faf5052c640 1 --2- 172.21.15.125:0/4236458833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf4815b2a0 0x7faf480cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.364+0000 7faf5052c640 1 --2- 172.21.15.125:0/4236458833 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf480cf340 0x7faf480d1910 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.364+0000 7faf5052c640 1 -- 172.21.15.125:0/4236458833 >> 172.21.15.125:0/4236458833 conn(0x7faf480d0d60 msgr2=0x7faf480d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:29.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.364+0000 7faf5052c640 1 -- 172.21.15.125:0/4236458833 shutdown_connections 2023-10-18T10:26:29.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.364+0000 7faf5052c640 1 -- 172.21.15.125:0/4236458833 wait complete. 2023-10-18T10:26:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.365+0000 7faf5052c640 1 Processor -- start 2023-10-18T10:26:29.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.365+0000 7faf5052c640 1 -- start start 2023-10-18T10:26:29.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf5052c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf480ceb40 0x7faf481588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:29.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf5052c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cf340 0x7faf480ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:29.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf5052c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4815b2a0 0x7faf4806a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:29.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf5052c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf480b4ef0 con 0x7faf4815b2a0 2023-10-18T10:26:29.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf5052c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf480b4a60 con 0x7faf480cf340 2023-10-18T10:26:29.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf5052c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf480b3e50 con 0x7faf480ceb40 2023-10-18T10:26:29.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf4eac9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4815b2a0 0x7faf4806a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:29.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf4eac9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4815b2a0 0x7faf4806a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43124/0 (socket says 172.21.15.125:43124) 2023-10-18T10:26:29.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf4eac9640 1 -- 172.21.15.125:0/302536709 learned_addr learned my addr 172.21.15.125:0/302536709 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf4e2c8640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf480ceb40 0x7faf481588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.366+0000 7faf4dac7640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cf340 0x7faf480ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf4eac9640 1 -- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf480ceb40 msgr2=0x7faf481588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf4eac9640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf480ceb40 0x7faf481588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf4eac9640 1 -- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cf340 msgr2=0x7faf480ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:29.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf4eac9640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cf340 0x7faf480ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:29.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf4eac9640 1 -- 172.21.15.125:0/302536709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40009560 con 0x7faf4815b2a0 2023-10-18T10:26:29.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf4e2c8640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf480ceb40 0x7faf481588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf4eac9640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4815b2a0 0x7faf4806a9c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c780 tx=0x7faf4400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf3effd640 1 -- 172.21.15.125:0/302536709 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44004b60 con 0x7faf4815b2a0 2023-10-18T10:26:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf3effd640 1 -- 172.21.15.125:0/302536709 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf4400ce90 con 0x7faf4815b2a0 2023-10-18T10:26:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf480b4130 con 0x7faf4815b2a0 2023-10-18T10:26:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf3effd640 1 -- 172.21.15.125:0/302536709 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400aab0 con 0x7faf4815b2a0 2023-10-18T10:26:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.367+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf480b46f0 con 0x7faf4815b2a0 2023-10-18T10:26:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.368+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf480690e0 con 0x7faf4815b2a0 2023-10-18T10:26:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.368+0000 7faf3effd640 1 -- 172.21.15.125:0/302536709 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faf4400ac10 con 0x7faf4815b2a0 2023-10-18T10:26:29.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.369+0000 7faf3effd640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7faf2c06ae40 0x7faf2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:29.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.369+0000 7faf3effd640 1 -- 172.21.15.125:0/302536709 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faf44089bb0 con 0x7faf4815b2a0 2023-10-18T10:26:29.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.369+0000 7faf4e2c8640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7faf2c06ae40 0x7faf2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:29.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.369+0000 7faf4e2c8640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7faf2c06ae40 0x7faf2c06d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf40035000 tx=0x7faf40004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:29.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.373+0000 7faf3effd640 1 -- 172.21.15.125:0/302536709 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf44054b20 con 0x7faf4815b2a0 2023-10-18T10:26:29.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:29.587+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7faf480d0b40 con 0x7faf4815b2a0 2023-10-18T10:26:30.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.196+0000 7faf3effd640 1 -- 172.21.15.125:0/302536709 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7faf44058140 con 0x7faf4815b2a0 2023-10-18T10:26:30.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7faf2c06ae40 msgr2=0x7faf2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7faf2c06ae40 0x7faf2c06d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf40035000 tx=0x7faf40004730 comp rx=0 tx=0).stop 2023-10-18T10:26:30.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4815b2a0 msgr2=0x7faf4806a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4815b2a0 0x7faf4806a9c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c780 tx=0x7faf4400cc40 comp rx=0 tx=0).stop 2023-10-18T10:26:30.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 shutdown_connections 2023-10-18T10:26:30.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7faf2c06ae40 0x7faf2c06d300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cf340 0x7faf480ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4815b2a0 0x7faf4806a9c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 --2- 172.21.15.125:0/302536709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf480ceb40 0x7faf481588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 >> 172.21.15.125:0/302536709 conn(0x7faf480d0d60 msgr2=0x7faf4806b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:30.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 shutdown_connections 2023-10-18T10:26:30.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.199+0000 7faf5052c640 1 -- 172.21.15.125:0/302536709 wait complete. 2023-10-18T10:26:30.216 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T10:26:30.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.327+0000 7fc66a290640 1 -- 172.21.15.125:0/3237090539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6640ce240 msgr2=0x7fc66406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.327+0000 7fc66a290640 1 --2- 172.21.15.125:0/3237090539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6640ce240 0x7fc66406d260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc654009830 tx=0x7fc65402bd60 comp rx=0 tx=0).stop 2023-10-18T10:26:30.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.327+0000 7fc66a290640 1 -- 172.21.15.125:0/3237090539 shutdown_connections 2023-10-18T10:26:30.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.327+0000 7fc66a290640 1 --2- 172.21.15.125:0/3237090539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6640ce240 0x7fc66406d260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.327+0000 7fc66a290640 1 -- 172.21.15.125:0/3237090539 >> 172.21.15.125:0/3237090539 conn(0x7fc6640b0670 msgr2=0x7fc6640b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:30.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.327+0000 7fc66a290640 1 -- 172.21.15.125:0/3237090539 shutdown_connections 2023-10-18T10:26:30.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.327+0000 7fc66a290640 1 -- 172.21.15.125:0/3237090539 wait complete. 2023-10-18T10:26:30.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.329+0000 7fc66a290640 1 Processor -- start 2023-10-18T10:26:30.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.329+0000 7fc66a290640 1 -- start start 2023-10-18T10:26:30.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.329+0000 7fc66a290640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6640ce240 0x7fc6640c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:30.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.329+0000 7fc66a290640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664159060 0x7fc6640c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:30.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.329+0000 7fc66a290640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66415ccf0 0x7fc6640c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:30.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.329+0000 7fc66a290640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640b3fa0 con 0x7fc664159060 2023-10-18T10:26:30.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.329+0000 7fc66a290640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c60f0 con 0x7fc66415ccf0 2023-10-18T10:26:30.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.329+0000 7fc66a290640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c63d0 con 0x7fc6640ce240 2023-10-18T10:26:30.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664159060 0x7fc6640c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:30.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664159060 0x7fc6640c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43152/0 (socket says 172.21.15.125:43152) 2023-10-18T10:26:30.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 -- 172.21.15.125:0/872633240 learned_addr learned my addr 172.21.15.125:0/872633240 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:30.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc663fff640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6640ce240 0x7fc6640c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:30.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc66882d640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66415ccf0 0x7fc6640c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:30.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 -- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6640ce240 msgr2=0x7fc6640c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6640ce240 0x7fc6640c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 -- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66415ccf0 msgr2=0x7fc6640c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66415ccf0 0x7fc6640c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 -- 172.21.15.125:0/872633240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654009490 con 0x7fc664159060 2023-10-18T10:26:30.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc6637fe640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664159060 0x7fc6640c8540 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc654009950 tx=0x7fc6540038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:30.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc660ff9640 1 -- 172.21.15.125:0/872633240 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654003ea0 con 0x7fc664159060 2023-10-18T10:26:30.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc660ff9640 1 -- 172.21.15.125:0/872633240 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc654005c90 con 0x7fc664159060 2023-10-18T10:26:30.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6640c66b0 con 0x7fc664159060 2023-10-18T10:26:30.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc660ff9640 1 -- 172.21.15.125:0/872633240 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654005280 con 0x7fc664159060 2023-10-18T10:26:30.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.330+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6640c4780 con 0x7fc664159060 2023-10-18T10:26:30.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.332+0000 7fc660ff9640 1 -- 172.21.15.125:0/872633240 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7fc6540053e0 con 0x7fc664159060 2023-10-18T10:26:30.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.332+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6640675f0 con 0x7fc664159060 2023-10-18T10:26:30.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.332+0000 7fc660ff9640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc65006ae90 0x7fc65006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:30.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.332+0000 7fc663fff640 1 -- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc65006ae90 msgr2=0x7fc65006d350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2976686360 2023-10-18T10:26:30.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.332+0000 7fc660ff9640 1 -- 172.21.15.125:0/872633240 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc6540ac240 con 0x7fc664159060 2023-10-18T10:26:30.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.332+0000 7fc663fff640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc65006ae90 0x7fc65006d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:26:30.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.336+0000 7fc660ff9640 1 -- 172.21.15.125:0/872633240 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6540772f0 con 0x7fc664159060 2023-10-18T10:26:30.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.524+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc6640b4aa0 con 0x7fc664159060 2023-10-18T10:26:30.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.527+0000 7fc660ff9640 1 -- 172.21.15.125:0/872633240 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc65407a910 con 0x7fc664159060 2023-10-18T10:26:30.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc65006ae90 msgr2=0x7fc65006d350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:26:30.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc65006ae90 0x7fc65006d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664159060 msgr2=0x7fc6640c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664159060 0x7fc6640c8540 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc654009950 tx=0x7fc6540038a0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 shutdown_connections 2023-10-18T10:26:30.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7fc65006ae90 0x7fc65006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66415ccf0 0x7fc6640c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664159060 0x7fc6640c8540 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 --2- 172.21.15.125:0/872633240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6640ce240 0x7fc6640c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 >> 172.21.15.125:0/872633240 conn(0x7fc6640b0670 msgr2=0x7fc66415d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:30.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 shutdown_connections 2023-10-18T10:26:30.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.530+0000 7fc66a290640 1 -- 172.21.15.125:0/872633240 wait complete. 2023-10-18T10:26:30.543 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T10:26:30.579 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:30.578+0000 7f29d964c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:26:30.609 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:30.607+0000 7f6ec58b2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:26:30.623 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:30.622+0000 7f29d964c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:26:30.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.647+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1924064136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa40cf340 msgr2=0x7f1fa40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.647+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1924064136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa40cf340 0x7f1fa40d1910 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1fa408a120 tx=0x7f1f9402bdb0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.647+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1924064136 shutdown_connections 2023-10-18T10:26:30.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.647+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1924064136 >> v2:172.21.15.139:3300/0 conn(0x7f1fa415b2a0 0x7f1fa40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.647+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1924064136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa40cf340 0x7f1fa40d1910 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.647+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1924064136 >> 172.21.15.125:0/1924064136 conn(0x7f1fa40d0d60 msgr2=0x7f1fa40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:30.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.648+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1924064136 shutdown_connections 2023-10-18T10:26:30.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.648+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1924064136 wait complete. 2023-10-18T10:26:30.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.649+0000 7f1fa9ec9640 1 Processor -- start 2023-10-18T10:26:30.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.649+0000 7f1fa9ec9640 1 -- start start 2023-10-18T10:26:30.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa9ec9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fa40ceb40 0x7f1fa41f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:30.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa9ec9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1fa40cf340 0x7f1fa40cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:30.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa9ec9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa415b2a0 0x7f1fa40ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:30.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa9ec9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41e5fe0 con 0x7f1fa415b2a0 2023-10-18T10:26:30.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa9ec9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40b3950 con 0x7f1fa40ceb40 2023-10-18T10:26:30.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa9ec9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41e5740 con 0x7f1fa40cf340 2023-10-18T10:26:30.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa415b2a0 0x7f1fa40ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:30.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa415b2a0 0x7f1fa40ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43180/0 (socket says 172.21.15.125:43180) 2023-10-18T10:26:30.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 -- 172.21.15.125:0/1873800387 learned_addr learned my addr 172.21.15.125:0/1873800387 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:30.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fa40ceb40 0x7f1fa41f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:30.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa2ffd640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1fa40cf340 0x7f1fa40cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:30.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 -- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1fa40cf340 msgr2=0x7f1fa40cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1fa40cf340 0x7f1fa40cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 -- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fa40ceb40 msgr2=0x7f1fa41f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fa40ceb40 0x7f1fa41f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 -- 172.21.15.125:0/1873800387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f94009560 con 0x7f1fa415b2a0 2023-10-18T10:26:30.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa3fff640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa415b2a0 0x7f1fa40ccd40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c730 tx=0x7f1f9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:30.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1f8bfff640 1 -- 172.21.15.125:0/1873800387 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c003b90 con 0x7f1fa415b2a0 2023-10-18T10:26:30.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1f8bfff640 1 -- 172.21.15.125:0/1873800387 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f9c003cf0 con 0x7f1fa415b2a0 2023-10-18T10:26:30.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa41e5a20 con 0x7f1fa415b2a0 2023-10-18T10:26:30.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1f8bfff640 1 -- 172.21.15.125:0/1873800387 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00aab0 con 0x7f1fa415b2a0 2023-10-18T10:26:30.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.650+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4157210 con 0x7f1fa415b2a0 2023-10-18T10:26:30.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.652+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa4157430 con 0x7f1fa415b2a0 2023-10-18T10:26:30.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.652+0000 7f1f8bfff640 1 -- 172.21.15.125:0/1873800387 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f1f9c00ac10 con 0x7f1fa415b2a0 2023-10-18T10:26:30.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.652+0000 7f1f8bfff640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1f9006ae90 0x7f1f9006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:30.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.652+0000 7f1f8bfff640 1 -- 172.21.15.125:0/1873800387 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1f9c0895d0 con 0x7f1fa415b2a0 2023-10-18T10:26:30.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.652+0000 7f1fa37fe640 1 -- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1f9006ae90 msgr2=0x7f1f9006d350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2976686360 2023-10-18T10:26:30.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.652+0000 7f1fa37fe640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1f9006ae90 0x7f1f9006d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:26:30.660 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:30.656+0000 7f6ec58b2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:26:30.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.656+0000 7f1f8bfff640 1 -- 172.21.15.125:0/1873800387 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f9c0545f0 con 0x7f1fa415b2a0 2023-10-18T10:26:30.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.827+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f1fa40cb9d0 con 0x7f1fa415b2a0 2023-10-18T10:26:30.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.831+0000 7f1f8bfff640 1 -- 172.21.15.125:0/1873800387 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1f9c057c10 con 0x7f1fa415b2a0 2023-10-18T10:26:30.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1f9006ae90 msgr2=0x7f1f9006d350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:26:30.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1f9006ae90 0x7f1f9006d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa415b2a0 msgr2=0x7f1fa40ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:30.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa415b2a0 0x7f1fa40ccd40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c730 tx=0x7f1f9c00cbf0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 shutdown_connections 2023-10-18T10:26:30.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f1f9006ae90 0x7f1f9006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fa415b2a0 0x7f1fa40ccd40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1fa40cf340 0x7f1fa40cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 --2- 172.21.15.125:0/1873800387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1fa40ceb40 0x7f1fa41f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:30.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 >> 172.21.15.125:0/1873800387 conn(0x7f1fa40d0d60 msgr2=0x7f1fa415a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:30.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 shutdown_connections 2023-10-18T10:26:30.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:30.833+0000 7f1fa9ec9640 1 -- 172.21.15.125:0/1873800387 wait complete. 2023-10-18T10:26:30.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:30.847 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T10:26:30.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:30.867 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-10-18T10:26:30.923 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1m 2023-10-18T10:26:31.010 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.009+0000 7f29d964c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:26:31.053 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.052+0000 7f29d964c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:26:31.064 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.063+0000 7f6ec58b2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:26:31.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.068+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2160189150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8159060 msgr2=0x7f9ee806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:31.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.068+0000 7f9eee7a9640 1 --2- 172.21.15.125:0/2160189150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8159060 0x7f9ee806d260 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009830 tx=0x7f9edc02bcd0 comp rx=0 tx=0).stop 2023-10-18T10:26:31.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.068+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2160189150 shutdown_connections 2023-10-18T10:26:31.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.068+0000 7f9eee7a9640 1 --2- 172.21.15.125:0/2160189150 >> v2:172.21.15.139:3300/0 conn(0x7f9ee80cdb30 0x7f9ee80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:31.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.068+0000 7f9eee7a9640 1 --2- 172.21.15.125:0/2160189150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8159060 0x7f9ee806d260 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:31.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.068+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2160189150 >> 172.21.15.125:0/2160189150 conn(0x7f9ee80b0670 msgr2=0x7f9ee80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:31.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.068+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2160189150 shutdown_connections 2023-10-18T10:26:31.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.068+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2160189150 wait complete. 2023-10-18T10:26:31.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.069+0000 7f9eee7a9640 1 Processor -- start 2023-10-18T10:26:31.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9eee7a9640 1 -- start start 2023-10-18T10:26:31.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9eee7a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee80cdb30 0x7f9ee8151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:31.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9eee7a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee8159060 0x7f9ee806c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:31.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9eee7a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ee815ccf0 0x7f9ee80af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:31.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9eee7a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80c2840 con 0x7f9ee80cdb30 2023-10-18T10:26:31.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9eee7a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80b3c80 con 0x7f9ee815ccf0 2023-10-18T10:26:31.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9eee7a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80b33e0 con 0x7f9ee8159060 2023-10-18T10:26:31.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee80cdb30 0x7f9ee8151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:31.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee80cdb30 0x7f9ee8151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43200/0 (socket says 172.21.15.125:43200) 2023-10-18T10:26:31.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.070+0000 7f9ee7fff640 1 -- 172.21.15.125:0/2430761953 learned_addr learned my addr 172.21.15.125:0/2430761953 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:31.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee77fe640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee8159060 0x7f9ee806c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:31.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9eecd46640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ee815ccf0 0x7f9ee80af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:31.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee7fff640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee8159060 msgr2=0x7f9ee806c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:31.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee7fff640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ee8159060 0x7f9ee806c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:31.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee7fff640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ee815ccf0 msgr2=0x7f9ee80af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:31.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee7fff640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ee815ccf0 0x7f9ee80af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:31.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee7fff640 1 -- 172.21.15.125:0/2430761953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc009490 con 0x7f9ee80cdb30 2023-10-18T10:26:31.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9eecd46640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ee815ccf0 0x7f9ee80af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:31.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee7fff640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee80cdb30 0x7f9ee8151200 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9edc02c2e0 tx=0x7f9edc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:31.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc003ea0 con 0x7f9ee80cdb30 2023-10-18T10:26:31.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9edc005c90 con 0x7f9ee80cdb30 2023-10-18T10:26:31.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee80b36c0 con 0x7f9ee80cdb30 2023-10-18T10:26:31.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc005280 con 0x7f9ee80cdb30 2023-10-18T10:26:31.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.071+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee806c650 con 0x7f9ee80cdb30 2023-10-18T10:26:31.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.072+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee80675f0 con 0x7f9ee80cdb30 2023-10-18T10:26:31.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.072+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f9edc0053e0 con 0x7f9ee80cdb30 2023-10-18T10:26:31.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.073+0000 7f9ee4ff9640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 0x7f9ed406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:31.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.073+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9edc0ac290 con 0x7f9ee80cdb30 2023-10-18T10:26:31.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.073+0000 7f9ee77fe640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 msgr2=0x7f9ed406d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2976686360 2023-10-18T10:26:31.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.073+0000 7f9ee77fe640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 0x7f9ed406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:26:31.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.077+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9edc077200 con 0x7f9ee80cdb30 2023-10-18T10:26:31.096 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.094+0000 7f29d964c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:26:31.111 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.110+0000 7f6ec58b2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:26:31.147 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.146+0000 7f29d964c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:26:31.156 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.154+0000 7f6ec58b2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:26:31.188 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.186+0000 7f29d964c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:26:31.210 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.209+0000 7f6ec58b2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:26:31.254 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.253+0000 7f6ec58b2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:26:31.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.271+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 --> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ee80c9e60 con 0x7f9ed406aec0 2023-10-18T10:26:31.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.274+0000 7f9ee77fe640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 msgr2=0x7f9ed406d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2976686360 2023-10-18T10:26:31.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.274+0000 7f9ee77fe640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 0x7f9ed406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:26:31.286 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.284+0000 7f29d964c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:26:31.338 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.336+0000 7f29d964c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:26:31.352 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.350+0000 7f6ec58b2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:26:31.379 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.378+0000 7f29d964c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:26:31.405 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.403+0000 7f6ec58b2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:26:31.428 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.426+0000 7f29d964c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:26:31.446 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.444+0000 7f6ec58b2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:26:31.470 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.468+0000 7f29d964c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:26:31.494 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.492+0000 7f6ec58b2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:26:31.536 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.534+0000 7f6ec58b2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:26:31.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.675+0000 7f9ee77fe640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 msgr2=0x7f9ed406d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2976686360 2023-10-18T10:26:31.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:31.675+0000 7f9ee77fe640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 0x7f9ed406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:26:31.787 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.785+0000 7f29d964c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:26:31.851 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.849+0000 7f6ec58b2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:26:31.918 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:31.916+0000 7f29d964c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:26:31.980 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:31.979+0000 7f6ec58b2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:26:32.008 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:32.006+0000 7f29d964c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:26:32.049 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:32.048+0000 7f29d964c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:26:32.071 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:32.069+0000 7f6ec58b2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:26:32.113 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:32.111+0000 7f6ec58b2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:26:32.143 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:32.141+0000 7f29d964c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:26:32.207 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:32.205+0000 7f6ec58b2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:26:32.232 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:32.231+0000 7f29d964c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:26:32.239 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:26:32.241 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:26:32.300 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:32.298+0000 7f6ec58b2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:26:32.373 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:32.371+0000 7f29d964c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:26:32.414 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:32.413+0000 7f29d964c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:26:32.438 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:32.436+0000 7f6ec58b2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:26:32.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:32.476+0000 7f9ee77fe640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 msgr2=0x7f9ed406d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2976686360 2023-10-18T10:26:32.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:32.476+0000 7f9ee77fe640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 0x7f9ed406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:26:32.479 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:32.478+0000 7f6ec58b2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:26:32.518 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:32.517+0000 7f29d964c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:26:32.586 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:32.584+0000 7f6ec58b2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:26:33.126 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.124+0000 7f29d964c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:26:33.193 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:33.192+0000 7f6ec58b2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:26:33.389 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.387+0000 7f29d964c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:26:33.437 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.435+0000 7f29d964c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:26:33.455 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:33.453+0000 7f6ec58b2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:26:33.504 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:33.502+0000 7f6ec58b2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:26:33.538 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.537+0000 7f29d964c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:26:33.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:33.560+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 7) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f9edc0775d0 con 0x7f9ee80cdb30 2023-10-18T10:26:33.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:33.560+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 msgr2=0x7f9ed406d380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:26:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:33.560+0000 7f9ee4ff9640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/2976686360,v1:172.21.15.139:6833/2976686360] conn(0x7f9ed406aec0 0x7f9ed406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:33.612 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:26:33.610+0000 7f6ec58b2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:26:33.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.667+0000 7f2956179640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.667+0000 7f2956179640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.667+0000 7f2956179640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.667+0000 7f2956179640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.671 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.667+0000 7f2956179640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.671 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.668+0000 7f295917f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.671 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.668+0000 7f295917f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.671 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.668+0000 7f295917f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.671 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.668+0000 7f295917f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:33.672 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:26:33.668+0000 7f295917f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:26:34.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.564+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f9edc07fd50 con 0x7f9ee80cdb30 2023-10-18T10:26:34.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.564+0000 7f9ee4ff9640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ed4035ae0 0x7f9ed4075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:34.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.564+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ed4035ec0 con 0x7f9ed4035ae0 2023-10-18T10:26:34.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.567+0000 7f9ee77fe640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ed4035ae0 0x7f9ed4075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:34.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.568+0000 7f9ee77fe640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ed4035ae0 0x7f9ed4075290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80aff90 tx=0x7f9ed8005f90 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:34.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.907+0000 7f9ee4ff9640 1 -- 172.21.15.125:0/2430761953 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9ed4035ec0 con 0x7f9ed4035ae0 2023-10-18T10:26:34.909 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir0 2023-10-18T10:26:34.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.909+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ed4035ae0 msgr2=0x7f9ed4075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:34.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.909+0000 7f9eee7a9640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ed4035ae0 0x7f9ed4075290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80aff90 tx=0x7f9ed8005f90 comp rx=0 tx=0).stop 2023-10-18T10:26:34.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.909+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee80cdb30 msgr2=0x7f9ee8151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:34.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.909+0000 7f9eee7a9640 1 --2- 172.21.15.125:0/2430761953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee80cdb30 0x7f9ee8151200 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9edc02c2e0 tx=0x7f9edc0038a0 comp rx=0 tx=0).stop 2023-10-18T10:26:34.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.909+0000 7f9eecd46640 1 -- 172.21.15.125:0/2430761953 reap_dead start 2023-10-18T10:26:34.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.909+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 shutdown_connections 2023-10-18T10:26:34.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.909+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 >> 172.21.15.125:0/2430761953 conn(0x7f9ee80b0670 msgr2=0x7f9ee80ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:34.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.910+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 shutdown_connections 2023-10-18T10:26:34.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:34.910+0000 7f9eee7a9640 1 -- 172.21.15.125:0/2430761953 wait complete. 2023-10-18T10:26:34.923 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-10-18T10:26:34.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:34.924 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-10-18T10:26:34.942 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1m 2023-10-18T10:26:35.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.090+0000 7f5a3b826640 1 -- 172.21.15.125:0/82560814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 msgr2=0x7f5a340cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.090+0000 7f5a3b826640 1 --2- 172.21.15.125:0/82560814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 0x7f5a340cf720 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5a3408a810 tx=0x7f5a2402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.090+0000 7f5a3b826640 1 -- 172.21.15.125:0/82560814 shutdown_connections 2023-10-18T10:26:35.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.090+0000 7f5a3b826640 1 --2- 172.21.15.125:0/82560814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 0x7f5a340cf720 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.090+0000 7f5a3b826640 1 --2- 172.21.15.125:0/82560814 >> v2:172.21.15.139:3300/0 conn(0x7f5a340cf340 0x7f5a340d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.090+0000 7f5a3b826640 1 -- 172.21.15.125:0/82560814 >> 172.21.15.125:0/82560814 conn(0x7f5a340d0d60 msgr2=0x7f5a340d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:35.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.090+0000 7f5a3b826640 1 -- 172.21.15.125:0/82560814 shutdown_connections 2023-10-18T10:26:35.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.090+0000 7f5a3b826640 1 -- 172.21.15.125:0/82560814 wait complete. 2023-10-18T10:26:35.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.091+0000 7f5a3b826640 1 Processor -- start 2023-10-18T10:26:35.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.091+0000 7f5a3b826640 1 -- start start 2023-10-18T10:26:35.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a3b826640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a340ceb40 0x7f5a340b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a3b826640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a340cf340 0x7f5a34150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a3b826640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 0x7f5a3415da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a3b826640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3415bfc0 con 0x7f5a3415b2a0 2023-10-18T10:26:35.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a3b826640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a340b67f0 con 0x7f5a340cf340 2023-10-18T10:26:35.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a3b826640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3415c260 con 0x7f5a340ceb40 2023-10-18T10:26:35.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a39dc3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 0x7f5a3415da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:35.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a39dc3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 0x7f5a3415da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43216/0 (socket says 172.21.15.125:43216) 2023-10-18T10:26:35.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a39dc3640 1 -- 172.21.15.125:0/1423150975 learned_addr learned my addr 172.21.15.125:0/1423150975 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:35.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a39dc3640 1 -- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a340ceb40 msgr2=0x7f5a340b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.092+0000 7f5a39dc3640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a340ceb40 0x7f5a340b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a39dc3640 1 -- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a340cf340 msgr2=0x7f5a34150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a39dc3640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a340cf340 0x7f5a34150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a39dc3640 1 -- 172.21.15.125:0/1423150975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24009490 con 0x7f5a3415b2a0 2023-10-18T10:26:35.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a39dc3640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 0x7f5a3415da90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00cc10 tx=0x7f5a2c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:35.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a29ffb640 1 -- 172.21.15.125:0/1423150975 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00ad40 con 0x7f5a3415b2a0 2023-10-18T10:26:35.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a29ffb640 1 -- 172.21.15.125:0/1423150975 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a2c012420 con 0x7f5a3415b2a0 2023-10-18T10:26:35.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a34154bf0 con 0x7f5a3415b2a0 2023-10-18T10:26:35.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a29ffb640 1 -- 172.21.15.125:0/1423150975 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c012830 con 0x7f5a3415b2a0 2023-10-18T10:26:35.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.093+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3415c540 con 0x7f5a3415b2a0 2023-10-18T10:26:35.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.094+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a340690e0 con 0x7f5a3415b2a0 2023-10-18T10:26:35.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.094+0000 7f5a29ffb640 1 -- 172.21.15.125:0/1423150975 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f5a2c01a3b0 con 0x7f5a3415b2a0 2023-10-18T10:26:35.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.095+0000 7f5a29ffb640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1806ace0 0x7f5a1806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.095+0000 7f5a29ffb640 1 -- 172.21.15.125:0/1423150975 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f5a2c089d60 con 0x7f5a3415b2a0 2023-10-18T10:26:35.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.095+0000 7f5a395c2640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1806ace0 0x7f5a1806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:35.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.095+0000 7f5a395c2640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1806ace0 0x7f5a1806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5a341592d0 tx=0x7f5a30005c70 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:35.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.099+0000 7f5a29ffb640 1 -- 172.21.15.125:0/1423150975 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a2c0589a0 con 0x7f5a3415b2a0 2023-10-18T10:26:35.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.285+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a340d0b40 con 0x7f5a1806ace0 2023-10-18T10:26:35.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.338+0000 7f5a29ffb640 1 -- 172.21.15.125:0/1423150975 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5a340d0b40 con 0x7f5a1806ace0 2023-10-18T10:26:35.340 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir1 2023-10-18T10:26:35.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1806ace0 msgr2=0x7f5a1806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1806ace0 0x7f5a1806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5a341592d0 tx=0x7f5a30005c70 comp rx=0 tx=0).stop 2023-10-18T10:26:35.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 msgr2=0x7f5a3415da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 0x7f5a3415da90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00cc10 tx=0x7f5a2c00a430 comp rx=0 tx=0).stop 2023-10-18T10:26:35.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 shutdown_connections 2023-10-18T10:26:35.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1806ace0 0x7f5a1806d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a340cf340 0x7f5a34150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3415b2a0 0x7f5a3415da90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 --2- 172.21.15.125:0/1423150975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a340ceb40 0x7f5a340b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 >> 172.21.15.125:0/1423150975 conn(0x7f5a340d0d60 msgr2=0x7f5a34159540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:35.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 shutdown_connections 2023-10-18T10:26:35.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.340+0000 7f5a3b826640 1 -- 172.21.15.125:0/1423150975 wait complete. 2023-10-18T10:26:35.352 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-10-18T10:26:35.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:35.353 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-10-18T10:26:35.369 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1m 2023-10-18T10:26:35.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.511+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/2292668132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340cf340 msgr2=0x7f2d340d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.511+0000 7f2d3b9e9640 1 --2- 172.21.15.125:0/2292668132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340cf340 0x7f2d340d1910 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009830 tx=0x7f2d2402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.511+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/2292668132 shutdown_connections 2023-10-18T10:26:35.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.511+0000 7f2d3b9e9640 1 --2- 172.21.15.125:0/2292668132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340cf340 0x7f2d340d1910 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.511+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/2292668132 >> 172.21.15.125:0/2292668132 conn(0x7f2d340d0d60 msgr2=0x7f2d340d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:35.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.511+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/2292668132 shutdown_connections 2023-10-18T10:26:35.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.511+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/2292668132 wait complete. 2023-10-18T10:26:35.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.512+0000 7f2d3b9e9640 1 Processor -- start 2023-10-18T10:26:35.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.512+0000 7f2d3b9e9640 1 -- start start 2023-10-18T10:26:35.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d3b9e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340ceb40 0x7f2d3406b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d3b9e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d340cf340 0x7f2d34156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d3b9e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d3415b2a0 0x7f2d3415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d3b9e9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340b67f0 con 0x7f2d340ceb40 2023-10-18T10:26:35.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d3b9e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340b6c10 con 0x7f2d3415b2a0 2023-10-18T10:26:35.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d3b9e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340b5b70 con 0x7f2d340cf340 2023-10-18T10:26:35.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d39785640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340ceb40 0x7f2d3406b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:35.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d39785640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340ceb40 0x7f2d3406b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43228/0 (socket says 172.21.15.125:43228) 2023-10-18T10:26:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d39785640 1 -- 172.21.15.125:0/642804150 learned_addr learned my addr 172.21.15.125:0/642804150 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d38f84640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d340cf340 0x7f2d34156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.513+0000 7f2d39f86640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d3415b2a0 0x7f2d3415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d39785640 1 -- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d340cf340 msgr2=0x7f2d34156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d39785640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d340cf340 0x7f2d34156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d39785640 1 -- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d3415b2a0 msgr2=0x7f2d3415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d39785640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d3415b2a0 0x7f2d3415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d39785640 1 -- 172.21.15.125:0/642804150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d24009490 con 0x7f2d340ceb40 2023-10-18T10:26:35.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d39785640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340ceb40 0x7f2d3406b4b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009950 tx=0x7f2d240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:35.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d29ffb640 1 -- 172.21.15.125:0/642804150 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24003ea0 con 0x7f2d340ceb40 2023-10-18T10:26:35.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d29ffb640 1 -- 172.21.15.125:0/642804150 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2d24005020 con 0x7f2d340ceb40 2023-10-18T10:26:35.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d29ffb640 1 -- 172.21.15.125:0/642804150 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24005410 con 0x7f2d340ceb40 2023-10-18T10:26:35.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d340b5e50 con 0x7f2d340ceb40 2023-10-18T10:26:35.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.514+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d340b6410 con 0x7f2d340ceb40 2023-10-18T10:26:35.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.515+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d340b6930 con 0x7f2d340ceb40 2023-10-18T10:26:35.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.516+0000 7f2d29ffb640 1 -- 172.21.15.125:0/642804150 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2d240047a0 con 0x7f2d340ceb40 2023-10-18T10:26:35.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.516+0000 7f2d29ffb640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f2d1806ace0 0x7f2d1806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.516+0000 7f2d29ffb640 1 -- 172.21.15.125:0/642804150 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f2d240ac210 con 0x7f2d340ceb40 2023-10-18T10:26:35.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.516+0000 7f2d38f84640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f2d1806ace0 0x7f2d1806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:35.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.517+0000 7f2d38f84640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f2d1806ace0 0x7f2d1806d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2d30006fd0 tx=0x7f2d30008040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:35.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.520+0000 7f2d29ffb640 1 -- 172.21.15.125:0/642804150 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d340b6930 con 0x7f2d340ceb40 2023-10-18T10:26:35.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.713+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d340690e0 con 0x7f2d1806ace0 2023-10-18T10:26:35.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.774+0000 7f2d29ffb640 1 -- 172.21.15.125:0/642804150 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2d340690e0 con 0x7f2d1806ace0 2023-10-18T10:26:35.776 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir2 2023-10-18T10:26:35.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f2d1806ace0 msgr2=0x7f2d1806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f2d1806ace0 0x7f2d1806d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2d30006fd0 tx=0x7f2d30008040 comp rx=0 tx=0).stop 2023-10-18T10:26:35.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340ceb40 msgr2=0x7f2d3406b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340ceb40 0x7f2d3406b4b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009950 tx=0x7f2d240038a0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 shutdown_connections 2023-10-18T10:26:35.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f2d1806ace0 0x7f2d1806d1a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d3415b2a0 0x7f2d3415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d340cf340 0x7f2d34156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 --2- 172.21.15.125:0/642804150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d340ceb40 0x7f2d3406b4b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 >> 172.21.15.125:0/642804150 conn(0x7f2d340d0d60 msgr2=0x7f2d340ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:35.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 shutdown_connections 2023-10-18T10:26:35.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.776+0000 7f2d3b9e9640 1 -- 172.21.15.125:0/642804150 wait complete. 2023-10-18T10:26:35.789 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-10-18T10:26:35.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:35.789 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-10-18T10:26:35.806 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1m 2023-10-18T10:26:35.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.951+0000 7f3e3f56c640 1 -- 172.21.15.125:0/2695298410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 msgr2=0x7f3e24004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.951+0000 7f3e3f56c640 1 --2- 172.21.15.125:0/2695298410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 0x7f3e24004c40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009b50 tx=0x7f3e240057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.951+0000 7f3e3f56c640 1 -- 172.21.15.125:0/2695298410 shutdown_connections 2023-10-18T10:26:35.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.951+0000 7f3e3f56c640 1 --2- 172.21.15.125:0/2695298410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 0x7f3e24004c40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.951+0000 7f3e3f56c640 1 -- 172.21.15.125:0/2695298410 >> 172.21.15.125:0/2695298410 conn(0x7f3e380d0d60 msgr2=0x7f3e380d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:35.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.951+0000 7f3e3f56c640 1 -- 172.21.15.125:0/2695298410 shutdown_connections 2023-10-18T10:26:35.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.951+0000 7f3e3f56c640 1 -- 172.21.15.125:0/2695298410 wait complete. 2023-10-18T10:26:35.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.952+0000 7f3e3f56c640 1 Processor -- start 2023-10-18T10:26:35.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.952+0000 7f3e3f56c640 1 -- start start 2023-10-18T10:26:35.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3f56c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 0x7f3e380b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3f56c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e380c9db0 0x7f3e380c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3f56c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38147d40 0x7f3e381481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3f56c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381486e0 con 0x7f3e240047c0 2023-10-18T10:26:35.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3f56c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380ca600 con 0x7f3e380c9db0 2023-10-18T10:26:35.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3f56c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380ca8e0 con 0x7f3e38147d40 2023-10-18T10:26:35.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 0x7f3e380b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:35.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 0x7f3e380b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43236/0 (socket says 172.21.15.125:43236) 2023-10-18T10:26:35.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 -- 172.21.15.125:0/355904480 learned_addr learned my addr 172.21.15.125:0/355904480 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:35.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 -- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38147d40 msgr2=0x7f3e381481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38147d40 0x7f3e381481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 -- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e380c9db0 msgr2=0x7f3e380c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:35.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e380c9db0 0x7f3e380c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:35.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 -- 172.21.15.125:0/355904480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e380cabc0 con 0x7f3e240047c0 2023-10-18T10:26:35.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3d308640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 0x7f3e380b2280 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006a30 tx=0x7f3e34001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:35.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e2dffb640 1 -- 172.21.15.125:0/355904480 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34010890 con 0x7f3e240047c0 2023-10-18T10:26:35.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e2dffb640 1 -- 172.21.15.125:0/355904480 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3e34010ae0 con 0x7f3e240047c0 2023-10-18T10:26:35.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.953+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e380caea0 con 0x7f3e240047c0 2023-10-18T10:26:35.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.954+0000 7f3e2dffb640 1 -- 172.21.15.125:0/355904480 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34010e70 con 0x7f3e240047c0 2023-10-18T10:26:35.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.954+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e380c85e0 con 0x7f3e240047c0 2023-10-18T10:26:35.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.955+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e380690e0 con 0x7f3e240047c0 2023-10-18T10:26:35.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.955+0000 7f3e2dffb640 1 -- 172.21.15.125:0/355904480 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3e34011460 con 0x7f3e240047c0 2023-10-18T10:26:35.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.955+0000 7f3e2dffb640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3e1c06ace0 0x7f3e1c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:35.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.955+0000 7f3e2dffb640 1 -- 172.21.15.125:0/355904480 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f3e3408a140 con 0x7f3e240047c0 2023-10-18T10:26:35.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.955+0000 7f3e3cb07640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3e1c06ace0 0x7f3e1c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:35.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.956+0000 7f3e3cb07640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3e1c06ace0 0x7f3e1c06d1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e28036000 tx=0x7f3e28035040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:35.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:35.960+0000 7f3e2dffb640 1 -- 172.21.15.125:0/355904480 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e340552f0 con 0x7f3e240047c0 2023-10-18T10:26:36.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.143+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e380d0b40 con 0x7f3e1c06ace0 2023-10-18T10:26:36.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.206+0000 7f3e2dffb640 1 -- 172.21.15.125:0/355904480 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3e380d0b40 con 0x7f3e1c06ace0 2023-10-18T10:26:36.208 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir3 2023-10-18T10:26:36.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3e1c06ace0 msgr2=0x7f3e1c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3e1c06ace0 0x7f3e1c06d1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e28036000 tx=0x7f3e28035040 comp rx=0 tx=0).stop 2023-10-18T10:26:36.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 msgr2=0x7f3e380b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 0x7f3e380b2280 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006a30 tx=0x7f3e34001c30 comp rx=0 tx=0).stop 2023-10-18T10:26:36.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 shutdown_connections 2023-10-18T10:26:36.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3e1c06ace0 0x7f3e1c06d1a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e380c9db0 0x7f3e380c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3e38147d40 0x7f3e381481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 --2- 172.21.15.125:0/355904480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e240047c0 0x7f3e380b2280 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 >> 172.21.15.125:0/355904480 conn(0x7f3e380d0d60 msgr2=0x7f3e380ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:36.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 shutdown_connections 2023-10-18T10:26:36.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.208+0000 7f3e3f56c640 1 -- 172.21.15.125:0/355904480 wait complete. 2023-10-18T10:26:36.220 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-10-18T10:26:36.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:36.221 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-10-18T10:26:36.238 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1m 2023-10-18T10:26:36.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.381+0000 7f15bc658640 1 -- 172.21.15.125:0/764700154 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 msgr2=0x7f15a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.381+0000 7f15bc658640 1 --2- 172.21.15.125:0/764700154 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 0x7f15a0005180 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f15ac001f00 tx=0x7f15a00057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.381+0000 7f15babf5640 1 -- 172.21.15.125:0/764700154 reap_dead start 2023-10-18T10:26:36.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.381+0000 7f15bc658640 1 -- 172.21.15.125:0/764700154 shutdown_connections 2023-10-18T10:26:36.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.381+0000 7f15bc658640 1 -- 172.21.15.125:0/764700154 >> 172.21.15.125:0/764700154 conn(0x7f15b4069930 msgr2=0x7f15b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:36.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.381+0000 7f15bc658640 1 -- 172.21.15.125:0/764700154 shutdown_connections 2023-10-18T10:26:36.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.381+0000 7f15bc658640 1 -- 172.21.15.125:0/764700154 wait complete. 2023-10-18T10:26:36.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.382+0000 7f15bc658640 1 Processor -- start 2023-10-18T10:26:36.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.382+0000 7f15bc658640 1 -- start start 2023-10-18T10:26:36.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15bc658640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 0x7f15b413b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:36.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15bc658640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b4070da0 0x7f15b4150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:36.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15bc658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40aa8c0 0x7f15b40aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:36.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15bc658640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40ab280 con 0x7f15a0004d00 2023-10-18T10:26:36.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15bc658640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b41513e0 con 0x7f15b40aa8c0 2023-10-18T10:26:36.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15bc658640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b41516c0 con 0x7f15b4070da0 2023-10-18T10:26:36.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15ba3f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 0x7f15b413b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:36.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15ba3f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 0x7f15b413b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43266/0 (socket says 172.21.15.125:43266) 2023-10-18T10:26:36.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15ba3f4640 1 -- 172.21.15.125:0/3229239473 learned_addr learned my addr 172.21.15.125:0/3229239473 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:36.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15b9bf3640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b4070da0 0x7f15b4150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:36.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.383+0000 7f15babf5640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40aa8c0 0x7f15b40aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:36.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15ba3f4640 1 -- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b4070da0 msgr2=0x7f15b4150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15ba3f4640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b4070da0 0x7f15b4150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15ba3f4640 1 -- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40aa8c0 msgr2=0x7f15b40aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15ba3f4640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40aa8c0 0x7f15b40aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15ba3f4640 1 -- 172.21.15.125:0/3229239473 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b41519a0 con 0x7f15a0004d00 2023-10-18T10:26:36.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15babf5640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40aa8c0 0x7f15b40aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:36.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15ba3f4640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 0x7f15b413b3b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f15b0011ae0 tx=0x7f15b0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:36.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15aaffd640 1 -- 172.21.15.125:0/3229239473 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b000fe70 con 0x7f15a0004d00 2023-10-18T10:26:36.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15aaffd640 1 -- 172.21.15.125:0/3229239473 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f15b0004050 con 0x7f15a0004d00 2023-10-18T10:26:36.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15aaffd640 1 -- 172.21.15.125:0/3229239473 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0004460 con 0x7f15a0004d00 2023-10-18T10:26:36.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b4151c80 con 0x7f15a0004d00 2023-10-18T10:26:36.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.384+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b4151040 con 0x7f15a0004d00 2023-10-18T10:26:36.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.385+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b40690e0 con 0x7f15a0004d00 2023-10-18T10:26:36.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.385+0000 7f15aaffd640 1 -- 172.21.15.125:0/3229239473 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f15b00035f0 con 0x7f15a0004d00 2023-10-18T10:26:36.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.386+0000 7f15aaffd640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f159806ad10 0x7f159806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:36.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.386+0000 7f15aaffd640 1 -- 172.21.15.125:0/3229239473 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f15b008a290 con 0x7f15a0004d00 2023-10-18T10:26:36.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.386+0000 7f15b9bf3640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f159806ad10 0x7f159806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:36.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.386+0000 7f15b9bf3640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f159806ad10 0x7f159806d1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f15a4000c00 tx=0x7f15a4006040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:36.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.390+0000 7f15aaffd640 1 -- 172.21.15.125:0/3229239473 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15b00555b0 con 0x7f15a0004d00 2023-10-18T10:26:36.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.567+0000 7f15aaffd640 1 -- 172.21.15.125:0/3229239473 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f15b0055980 con 0x7f15a0004d00 2023-10-18T10:26:36.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.580+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f15b415cf30 con 0x7f159806ad10 2023-10-18T10:26:36.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.633+0000 7f15aaffd640 1 -- 172.21.15.125:0/3229239473 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f15a4004740 con 0x7f159806ad10 2023-10-18T10:26:36.635 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir4 2023-10-18T10:26:36.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f159806ad10 msgr2=0x7f159806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f159806ad10 0x7f159806d1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f15a4000c00 tx=0x7f15a4006040 comp rx=0 tx=0).stop 2023-10-18T10:26:36.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 msgr2=0x7f15b413b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 0x7f15b413b3b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f15b0011ae0 tx=0x7f15b0009e90 comp rx=0 tx=0).stop 2023-10-18T10:26:36.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 shutdown_connections 2023-10-18T10:26:36.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f159806ad10 0x7f159806d1d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15b40aa8c0 0x7f15b40aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15b4070da0 0x7f15b4150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 --2- 172.21.15.125:0/3229239473 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15a0004d00 0x7f15b413b3b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 >> 172.21.15.125:0/3229239473 conn(0x7f15b4069930 msgr2=0x7f15b40ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 shutdown_connections 2023-10-18T10:26:36.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.635+0000 7f15bc658640 1 -- 172.21.15.125:0/3229239473 wait complete. 2023-10-18T10:26:36.647 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-10-18T10:26:36.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:36.648 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-10-18T10:26:36.664 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1m 2023-10-18T10:26:36.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.807+0000 7f1c85075640 1 -- 172.21.15.125:0/2804594372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c8015b2a0 msgr2=0x7f1c800d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.807+0000 7f1c85075640 1 --2- 172.21.15.125:0/2804594372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c8015b2a0 0x7f1c800d1910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009830 tx=0x7f1c7402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.807+0000 7f1c85075640 1 -- 172.21.15.125:0/2804594372 shutdown_connections 2023-10-18T10:26:36.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.807+0000 7f1c85075640 1 --2- 172.21.15.125:0/2804594372 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c800ceb40 0x7f1c800b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.807+0000 7f1c85075640 1 --2- 172.21.15.125:0/2804594372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c8015b2a0 0x7f1c800d1910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.807+0000 7f1c85075640 1 -- 172.21.15.125:0/2804594372 >> 172.21.15.125:0/2804594372 conn(0x7f1c800d0d60 msgr2=0x7f1c800d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:36.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.807+0000 7f1c85075640 1 -- 172.21.15.125:0/2804594372 shutdown_connections 2023-10-18T10:26:36.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.807+0000 7f1c85075640 1 -- 172.21.15.125:0/2804594372 wait complete. 2023-10-18T10:26:36.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c85075640 1 Processor -- start 2023-10-18T10:26:36.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c85075640 1 -- start start 2023-10-18T10:26:36.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c85075640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1c800ceb40 0x7f1c800c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:36.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c85075640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c800cf340 0x7f1c801f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:36.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c85075640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c8015b2a0 0x7f1c800aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:36.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c85075640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c801e5450 con 0x7f1c800ceb40 2023-10-18T10:26:36.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c85075640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c801e5730 con 0x7f1c800cf340 2023-10-18T10:26:36.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c85075640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c801e47d0 con 0x7f1c8015b2a0 2023-10-18T10:26:36.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.809+0000 7f1c7ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1c800ceb40 0x7f1c800c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:36.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1c800ceb40 0x7f1c800c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43268/0 (socket says 172.21.15.125:43268) 2023-10-18T10:26:36.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7ed9d640 1 -- 172.21.15.125:0/2519892366 learned_addr learned my addr 172.21.15.125:0/2519892366 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:36.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7e59c640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c800cf340 0x7f1c801f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:36.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7f59e640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c8015b2a0 0x7f1c800aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:36.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7ed9d640 1 -- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c8015b2a0 msgr2=0x7f1c800aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7ed9d640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c8015b2a0 0x7f1c800aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7ed9d640 1 -- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c800cf340 msgr2=0x7f1c801f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:36.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7ed9d640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c800cf340 0x7f1c801f4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:36.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7ed9d640 1 -- 172.21.15.125:0/2519892366 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c78009490 con 0x7f1c800ceb40 2023-10-18T10:26:36.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c7ed9d640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1c800ceb40 0x7f1c800c99b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000cbf0 tx=0x7f1c7000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:36.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c677fe640 1 -- 172.21.15.125:0/2519892366 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000ac40 con 0x7f1c800ceb40 2023-10-18T10:26:36.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c677fe640 1 -- 172.21.15.125:0/2519892366 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1c7000ae90 con 0x7f1c800ceb40 2023-10-18T10:26:36.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c74009490 con 0x7f1c800ceb40 2023-10-18T10:26:36.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c677fe640 1 -- 172.21.15.125:0/2519892366 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70003bd0 con 0x7f1c800ceb40 2023-10-18T10:26:36.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.810+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c801e4d90 con 0x7f1c800ceb40 2023-10-18T10:26:36.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.811+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c801e5590 con 0x7f1c800ceb40 2023-10-18T10:26:36.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.812+0000 7f1c677fe640 1 -- 172.21.15.125:0/2519892366 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f1c70003d30 con 0x7f1c800ceb40 2023-10-18T10:26:36.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.812+0000 7f1c677fe640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1c6c06afb0 0x7f1c6c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:36.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.812+0000 7f1c677fe640 1 -- 172.21.15.125:0/2519892366 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f1c7008a300 con 0x7f1c800ceb40 2023-10-18T10:26:36.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.812+0000 7f1c7e59c640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1c6c06afb0 0x7f1c6c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:36.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.813+0000 7f1c7e59c640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1c6c06afb0 0x7f1c6c06d470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1c74036000 tx=0x7f1c74004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:36.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:36.816+0000 7f1c677fe640 1 -- 172.21.15.125:0/2519892366 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c801e5590 con 0x7f1c800ceb40 2023-10-18T10:26:37.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.001+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c801e5590 con 0x7f1c6c06afb0 2023-10-18T10:26:37.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.058+0000 7f1c677fe640 1 -- 172.21.15.125:0/2519892366 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1c801e5590 con 0x7f1c6c06afb0 2023-10-18T10:26:37.060 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir5 2023-10-18T10:26:37.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1c6c06afb0 msgr2=0x7f1c6c06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1c6c06afb0 0x7f1c6c06d470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1c74036000 tx=0x7f1c74004730 comp rx=0 tx=0).stop 2023-10-18T10:26:37.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1c800ceb40 msgr2=0x7f1c800c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1c800ceb40 0x7f1c800c99b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000cbf0 tx=0x7f1c7000a060 comp rx=0 tx=0).stop 2023-10-18T10:26:37.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 shutdown_connections 2023-10-18T10:26:37.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1c6c06afb0 0x7f1c6c06d470 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c800cf340 0x7f1c801f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c8015b2a0 0x7f1c800aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 --2- 172.21.15.125:0/2519892366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1c800ceb40 0x7f1c800c99b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 >> 172.21.15.125:0/2519892366 conn(0x7f1c800d0d60 msgr2=0x7f1c80069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:37.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 shutdown_connections 2023-10-18T10:26:37.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.060+0000 7f1c85075640 1 -- 172.21.15.125:0/2519892366 wait complete. 2023-10-18T10:26:37.072 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-10-18T10:26:37.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:37.073 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-10-18T10:26:37.089 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1m 2023-10-18T10:26:37.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.232+0000 7f440538e640 1 -- 172.21.15.125:0/3078531352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43e80047c0 msgr2=0x7f43e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.232+0000 7f440538e640 1 --2- 172.21.15.125:0/3078531352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43e80047c0 0x7f43e8004c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f43f400baf0 tx=0x7f43e80057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.232+0000 7f43ff7fe640 1 -- 172.21.15.125:0/3078531352 reap_dead start 2023-10-18T10:26:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.232+0000 7f440538e640 1 -- 172.21.15.125:0/3078531352 shutdown_connections 2023-10-18T10:26:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.232+0000 7f440538e640 1 -- 172.21.15.125:0/3078531352 >> 172.21.15.125:0/3078531352 conn(0x7f440006fae0 msgr2=0x7f440006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.232+0000 7f440538e640 1 -- 172.21.15.125:0/3078531352 shutdown_connections 2023-10-18T10:26:37.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.232+0000 7f440538e640 1 -- 172.21.15.125:0/3078531352 wait complete. 2023-10-18T10:26:37.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f440538e640 1 Processor -- start 2023-10-18T10:26:37.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f440538e640 1 -- start start 2023-10-18T10:26:37.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f440538e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43e80047c0 0x7f4400153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f440538e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440014d040 0x7f4400157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f440538e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440014dd10 0x7f440014e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f440538e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440014e6d0 con 0x7f440014d040 2023-10-18T10:26:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f440538e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4400154570 con 0x7f440014dd10 2023-10-18T10:26:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f440538e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4400154850 con 0x7f43e80047c0 2023-10-18T10:26:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f43fe7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440014d040 0x7f4400157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f43fe7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440014d040 0x7f4400157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43288/0 (socket says 172.21.15.125:43288) 2023-10-18T10:26:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.234+0000 7f43fe7fc640 1 -- 172.21.15.125:0/1049382783 learned_addr learned my addr 172.21.15.125:0/1049382783 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43fe7fc640 1 -- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43e80047c0 msgr2=0x7f4400153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43feffd640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43e80047c0 0x7f4400153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43ff7fe640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440014dd10 0x7f440014e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43fe7fc640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43e80047c0 0x7f4400153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43fe7fc640 1 -- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440014dd10 msgr2=0x7f440014e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43fe7fc640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440014dd10 0x7f440014e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43fe7fc640 1 -- 172.21.15.125:0/1049382783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400154b30 con 0x7f440014d040 2023-10-18T10:26:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43fe7fc640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440014d040 0x7f4400157e00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43f400baf0 tx=0x7f43f4036780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43e77fe640 1 -- 172.21.15.125:0/1049382783 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4036e40 con 0x7f440014d040 2023-10-18T10:26:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43e77fe640 1 -- 172.21.15.125:0/1049382783 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f43f4004120 con 0x7f440014d040 2023-10-18T10:26:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4400154e10 con 0x7f440014d040 2023-10-18T10:26:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f43e77fe640 1 -- 172.21.15.125:0/1049382783 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40035f0 con 0x7f440014d040 2023-10-18T10:26:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.235+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440014d4a0 con 0x7f440014d040 2023-10-18T10:26:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.236+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44000690e0 con 0x7f440014d040 2023-10-18T10:26:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.237+0000 7f43e77fe640 1 -- 172.21.15.125:0/1049382783 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f43f4003750 con 0x7f440014d040 2023-10-18T10:26:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.237+0000 7f43e77fe640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f43ec06afe0 0x7f43ec06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.237+0000 7f43e77fe640 1 -- 172.21.15.125:0/1049382783 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f43f40abb70 con 0x7f440014d040 2023-10-18T10:26:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.237+0000 7f43feffd640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f43ec06afe0 0x7f43ec06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.238+0000 7f43feffd640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f43ec06afe0 0x7f43ec06d4a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43f0006fd0 tx=0x7f43f0001ea0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:37.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.241+0000 7f43e77fe640 1 -- 172.21.15.125:0/1049382783 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43f4076840 con 0x7f440014d040 2023-10-18T10:26:37.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.425+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f44000ab610 con 0x7f43ec06afe0 2023-10-18T10:26:37.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.479+0000 7f43e77fe640 1 -- 172.21.15.125:0/1049382783 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f44000ab610 con 0x7f43ec06afe0 2023-10-18T10:26:37.481 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir6 2023-10-18T10:26:37.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f43ec06afe0 msgr2=0x7f43ec06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f43ec06afe0 0x7f43ec06d4a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43f0006fd0 tx=0x7f43f0001ea0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440014d040 msgr2=0x7f4400157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440014d040 0x7f4400157e00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43f400baf0 tx=0x7f43f4036780 comp rx=0 tx=0).stop 2023-10-18T10:26:37.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 shutdown_connections 2023-10-18T10:26:37.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f43ec06afe0 0x7f43ec06d4a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f440014dd10 0x7f440014e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440014d040 0x7f4400157e00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 --2- 172.21.15.125:0/1049382783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43e80047c0 0x7f4400153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 >> 172.21.15.125:0/1049382783 conn(0x7f440006fae0 msgr2=0x7f440014ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:37.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 shutdown_connections 2023-10-18T10:26:37.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.481+0000 7f440538e640 1 -- 172.21.15.125:0/1049382783 wait complete. 2023-10-18T10:26:37.493 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-10-18T10:26:37.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:37.494 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-10-18T10:26:37.511 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1m 2023-10-18T10:26:37.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.657+0000 7fd50f5ee640 1 -- 172.21.15.125:0/838846776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5080b0a30 msgr2=0x7fd508157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.657+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/838846776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5080b0a30 0x7fd508157e80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd5080b1ef0 tx=0x7fd50002d420 comp rx=0 tx=0).stop 2023-10-18T10:26:37.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.657+0000 7fd50f5ee640 1 -- 172.21.15.125:0/838846776 shutdown_connections 2023-10-18T10:26:37.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.657+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/838846776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5080b0a30 0x7fd508157e80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.657+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/838846776 >> v2:172.21.15.139:3300/0 conn(0x7fd508157aa0 0x7fd50806fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.657+0000 7fd50f5ee640 1 -- 172.21.15.125:0/838846776 >> 172.21.15.125:0/838846776 conn(0x7fd5080cc970 msgr2=0x7fd5080ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:37.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.657+0000 7fd50f5ee640 1 -- 172.21.15.125:0/838846776 shutdown_connections 2023-10-18T10:26:37.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.657+0000 7fd50f5ee640 1 -- 172.21.15.125:0/838846776 wait complete. 2023-10-18T10:26:37.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.658+0000 7fd50f5ee640 1 Processor -- start 2023-10-18T10:26:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.658+0000 7fd50f5ee640 1 -- start start 2023-10-18T10:26:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50f5ee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5080b0a30 0x7fd5081ef970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50f5ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5080cc440 0x7fd5080caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50f5ee640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd508157aa0 0x7fd50806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50f5ee640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081ed400 con 0x7fd508157aa0 2023-10-18T10:26:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50f5ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081f0330 con 0x7fd5080cc440 2023-10-18T10:26:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50f5ee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081f0610 con 0x7fd5080b0a30 2023-10-18T10:26:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd508157aa0 0x7fd50806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd508157aa0 0x7fd50806c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41108/0 (socket says 172.21.15.125:41108) 2023-10-18T10:26:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 -- 172.21.15.125:0/1494948288 learned_addr learned my addr 172.21.15.125:0/1494948288 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50cb89640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5080cc440 0x7fd5080caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50d38a640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5080b0a30 0x7fd5081ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 -- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5080b0a30 msgr2=0x7fd5081ef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5080b0a30 0x7fd5081ef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 -- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5080cc440 msgr2=0x7fd5080caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5080cc440 0x7fd5080caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 -- 172.21.15.125:0/1494948288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50000a950 con 0x7fd508157aa0 2023-10-18T10:26:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50d38a640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5080b0a30 0x7fd5081ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd50db8b640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd508157aa0 0x7fd50806c710 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd500037040 tx=0x7fd500032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.659+0000 7fd4fdffb640 1 -- 172.21.15.125:0/1494948288 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500032f10 con 0x7fd508157aa0 2023-10-18T10:26:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.660+0000 7fd4fdffb640 1 -- 172.21.15.125:0/1494948288 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd500004330 con 0x7fd508157aa0 2023-10-18T10:26:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.660+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5081f08f0 con 0x7fd508157aa0 2023-10-18T10:26:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.660+0000 7fd4fdffb640 1 -- 172.21.15.125:0/1494948288 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500004720 con 0x7fd508157aa0 2023-10-18T10:26:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.660+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5080ca9f0 con 0x7fd508157aa0 2023-10-18T10:26:37.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.661+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd508154e90 con 0x7fd508157aa0 2023-10-18T10:26:37.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.661+0000 7fd4fdffb640 1 -- 172.21.15.125:0/1494948288 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fd500034440 con 0x7fd508157aa0 2023-10-18T10:26:37.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.661+0000 7fd4fdffb640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd4ec06b040 0x7fd4ec06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:37.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.661+0000 7fd4fdffb640 1 -- 172.21.15.125:0/1494948288 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fd5000ad3a0 con 0x7fd508157aa0 2023-10-18T10:26:37.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.662+0000 7fd50d38a640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd4ec06b040 0x7fd4ec06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:37.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.662+0000 7fd50d38a640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd4ec06b040 0x7fd4ec06d500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005fd0 tx=0x7fd4f800f360 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:37.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.666+0000 7fd4fdffb640 1 -- 172.21.15.125:0/1494948288 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd500077f30 con 0x7fd508157aa0 2023-10-18T10:26:37.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.851+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd50806afc0 con 0x7fd4ec06b040 2023-10-18T10:26:37.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.899+0000 7fd4fdffb640 1 -- 172.21.15.125:0/1494948288 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd4f800b890 con 0x7fd4ec06b040 2023-10-18T10:26:37.901 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir7 2023-10-18T10:26:37.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd4ec06b040 msgr2=0x7fd4ec06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd4ec06b040 0x7fd4ec06d500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005fd0 tx=0x7fd4f800f360 comp rx=0 tx=0).stop 2023-10-18T10:26:37.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd508157aa0 msgr2=0x7fd50806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:37.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd508157aa0 0x7fd50806c710 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd500037040 tx=0x7fd500032b90 comp rx=0 tx=0).stop 2023-10-18T10:26:37.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 shutdown_connections 2023-10-18T10:26:37.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd4ec06b040 0x7fd4ec06d500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5080cc440 0x7fd5080caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd508157aa0 0x7fd50806c710 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.901+0000 7fd50f5ee640 1 --2- 172.21.15.125:0/1494948288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd5080b0a30 0x7fd5081ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:37.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.902+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 >> 172.21.15.125:0/1494948288 conn(0x7fd5080cc970 msgr2=0x7fd5081567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:37.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.902+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 shutdown_connections 2023-10-18T10:26:37.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:37.902+0000 7fd50f5ee640 1 -- 172.21.15.125:0/1494948288 wait complete. 2023-10-18T10:26:37.914 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-10-18T10:26:37.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:37.915 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-10-18T10:26:37.931 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1m 2023-10-18T10:26:38.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.074+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/3683961274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c40afa20 msgr2=0x7fb3c40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.074+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/3683961274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c40afa20 0x7fb3c40705a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009830 tx=0x7fb3b802bd60 comp rx=0 tx=0).stop 2023-10-18T10:26:38.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.074+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/3683961274 shutdown_connections 2023-10-18T10:26:38.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.074+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/3683961274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c40afa20 0x7fb3c40705a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.074+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/3683961274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3c40cbd20 0x7fb3c4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.074+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/3683961274 >> 172.21.15.125:0/3683961274 conn(0x7fb3c4069930 msgr2=0x7fb3c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:38.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.074+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/3683961274 shutdown_connections 2023-10-18T10:26:38.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.074+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/3683961274 wait complete. 2023-10-18T10:26:38.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.075+0000 7fb3c9ee7640 1 Processor -- start 2023-10-18T10:26:38.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.075+0000 7fb3c9ee7640 1 -- start start 2023-10-18T10:26:38.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c9ee7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3c40afa20 0x7fb3c413d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c9ee7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c40b0530 0x7fb3c4153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c9ee7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3c40cbd20 0x7fb3c414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c9ee7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c41523b0 con 0x7fb3c40afa20 2023-10-18T10:26:38.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c9ee7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4152690 con 0x7fb3c40b0530 2023-10-18T10:26:38.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c9ee7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4152970 con 0x7fb3c40cbd20 2023-10-18T10:26:38.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3c40afa20 0x7fb3c413d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3c40afa20 0x7fb3c413d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41110/0 (socket says 172.21.15.125:41110) 2023-10-18T10:26:38.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c37fe640 1 -- 172.21.15.125:0/2277868670 learned_addr learned my addr 172.21.15.125:0/2277868670 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:38.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c2ffd640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c40b0530 0x7fb3c4153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c3fff640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3c40cbd20 0x7fb3c414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c37fe640 1 -- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3c40cbd20 msgr2=0x7fb3c414dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c37fe640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3c40cbd20 0x7fb3c414dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c37fe640 1 -- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c40b0530 msgr2=0x7fb3c4153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c37fe640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c40b0530 0x7fb3c4153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.076+0000 7fb3c37fe640 1 -- 172.21.15.125:0/2277868670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b400a5a0 con 0x7fb3c40afa20 2023-10-18T10:26:38.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.077+0000 7fb3c37fe640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3c40afa20 0x7fb3c413d030 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400ea70 tx=0x7fb3b400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:38.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.077+0000 7fb3abfff640 1 -- 172.21.15.125:0/2277868670 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400cd40 con 0x7fb3c40afa20 2023-10-18T10:26:38.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.077+0000 7fb3abfff640 1 -- 172.21.15.125:0/2277868670 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3b4004020 con 0x7fb3c40afa20 2023-10-18T10:26:38.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.077+0000 7fb3abfff640 1 -- 172.21.15.125:0/2277868670 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004430 con 0x7fb3c40afa20 2023-10-18T10:26:38.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.077+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b8009490 con 0x7fb3c40afa20 2023-10-18T10:26:38.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.077+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c4070cf0 con 0x7fb3c40afa20 2023-10-18T10:26:38.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.078+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c40690e0 con 0x7fb3c40afa20 2023-10-18T10:26:38.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.078+0000 7fb3abfff640 1 -- 172.21.15.125:0/2277868670 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fb3b40035f0 con 0x7fb3c40afa20 2023-10-18T10:26:38.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.079+0000 7fb3abfff640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb3b006b070 0x7fb3b006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.079+0000 7fb3abfff640 1 -- 172.21.15.125:0/2277868670 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fb3b408a1d0 con 0x7fb3c40afa20 2023-10-18T10:26:38.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.079+0000 7fb3c2ffd640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb3b006b070 0x7fb3b006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.079+0000 7fb3c2ffd640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb3b006b070 0x7fb3b006d530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8036000 tx=0x7fb3b8004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:38.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.083+0000 7fb3abfff640 1 -- 172.21.15.125:0/2277868670 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3b4054d60 con 0x7fb3c40afa20 2023-10-18T10:26:38.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.270+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3c413d760 con 0x7fb3b006b070 2023-10-18T10:26:38.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.320+0000 7fb3abfff640 1 -- 172.21.15.125:0/2277868670 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb3c413d760 con 0x7fb3b006b070 2023-10-18T10:26:38.323 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir8 2023-10-18T10:26:38.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb3b006b070 msgr2=0x7fb3b006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb3b006b070 0x7fb3b006d530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8036000 tx=0x7fb3b8004730 comp rx=0 tx=0).stop 2023-10-18T10:26:38.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3c40afa20 msgr2=0x7fb3c413d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3c40afa20 0x7fb3c413d030 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400ea70 tx=0x7fb3b400ef30 comp rx=0 tx=0).stop 2023-10-18T10:26:38.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 shutdown_connections 2023-10-18T10:26:38.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb3b006b070 0x7fb3b006d530 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3c40b0530 0x7fb3c4153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3c40cbd20 0x7fb3c414dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 --2- 172.21.15.125:0/2277868670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3c40afa20 0x7fb3c413d030 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.322+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 >> 172.21.15.125:0/2277868670 conn(0x7fb3c4069930 msgr2=0x7fb3c40acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.323+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 shutdown_connections 2023-10-18T10:26:38.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.323+0000 7fb3c9ee7640 1 -- 172.21.15.125:0/2277868670 wait complete. 2023-10-18T10:26:38.335 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-10-18T10:26:38.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:26:38.335 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-10-18T10:26:38.352 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1m 2023-10-18T10:26:38.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.495+0000 7f979e7d0640 1 -- 172.21.15.125:0/2930864275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9780004d00 msgr2=0x7f9780005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.495+0000 7f979e7d0640 1 --2- 172.21.15.125:0/2930864275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9780004d00 0x7f9780005180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9790005fd0 tx=0x7f97800057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.495+0000 7f979cd6d640 1 -- 172.21.15.125:0/2930864275 reap_dead start 2023-10-18T10:26:38.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.495+0000 7f979e7d0640 1 -- 172.21.15.125:0/2930864275 shutdown_connections 2023-10-18T10:26:38.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.495+0000 7f979e7d0640 1 -- 172.21.15.125:0/2930864275 >> 172.21.15.125:0/2930864275 conn(0x7f9798069930 msgr2=0x7f9798069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:38.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.495+0000 7f979e7d0640 1 -- 172.21.15.125:0/2930864275 shutdown_connections 2023-10-18T10:26:38.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.496+0000 7f979e7d0640 1 -- 172.21.15.125:0/2930864275 wait complete. 2023-10-18T10:26:38.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.497+0000 7f979e7d0640 1 Processor -- start 2023-10-18T10:26:38.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.497+0000 7f979e7d0640 1 -- start start 2023-10-18T10:26:38.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.497+0000 7f979e7d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780004d00 0x7f9798153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.497+0000 7f979e7d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9798153d80 0x7f979812d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.497+0000 7f979e7d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f979812d630 0x7f979812dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.497+0000 7f979e7d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979812b060 con 0x7f979812d630 2023-10-18T10:26:38.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.497+0000 7f979e7d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9798154200 con 0x7f9780004d00 2023-10-18T10:26:38.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.497+0000 7f979e7d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97980a8b30 con 0x7f9798153d80 2023-10-18T10:26:38.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f979812d630 0x7f979812dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f979812d630 0x7f979812dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41126/0 (socket says 172.21.15.125:41126) 2023-10-18T10:26:38.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 -- 172.21.15.125:0/1142569145 learned_addr learned my addr 172.21.15.125:0/1142569145 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:38.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f97977fe640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9798153d80 0x7f979812d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f9797fff640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780004d00 0x7f9798153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 -- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9798153d80 msgr2=0x7f979812d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9798153d80 0x7f979812d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 -- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780004d00 msgr2=0x7f9798153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780004d00 0x7f9798153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 -- 172.21.15.125:0/1142569145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980a7e20 con 0x7f979812d630 2023-10-18T10:26:38.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979cd6d640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f979812d630 0x7f979812dab0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9790005fd0 tx=0x7f9790033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:38.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f9794ff9640 1 -- 172.21.15.125:0/1142569145 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790004b20 con 0x7f979812d630 2023-10-18T10:26:38.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f9794ff9640 1 -- 172.21.15.125:0/1142569145 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9790004c80 con 0x7f979812d630 2023-10-18T10:26:38.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979812b270 con 0x7f979812d630 2023-10-18T10:26:38.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f9794ff9640 1 -- 172.21.15.125:0/1142569145 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790005020 con 0x7f979812d630 2023-10-18T10:26:38.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.498+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979812b830 con 0x7f979812d630 2023-10-18T10:26:38.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.499+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979806c850 con 0x7f979812d630 2023-10-18T10:26:38.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.499+0000 7f9794ff9640 1 -- 172.21.15.125:0/1142569145 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f9790005180 con 0x7f979812d630 2023-10-18T10:26:38.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.500+0000 7f9794ff9640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f978406b040 0x7f978406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.500+0000 7f9794ff9640 1 -- 172.21.15.125:0/1142569145 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f97900ac590 con 0x7f979812d630 2023-10-18T10:26:38.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.500+0000 7f9797fff640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f978406b040 0x7f978406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.500+0000 7f9797fff640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f978406b040 0x7f978406d500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9788001eb0 tx=0x7f97880067a0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:38.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.504+0000 7f9794ff9640 1 -- 172.21.15.125:0/1142569145 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9790077190 con 0x7f979812d630 2023-10-18T10:26:38.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.691+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f979806c290 con 0x7f978406b040 2023-10-18T10:26:38.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.753+0000 7f9794ff9640 1 -- 172.21.15.125:0/1142569145 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f979806c290 con 0x7f978406b040 2023-10-18T10:26:38.755 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/dir9 2023-10-18T10:26:38.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f978406b040 msgr2=0x7f978406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f978406b040 0x7f978406d500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9788001eb0 tx=0x7f97880067a0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f979812d630 msgr2=0x7f979812dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f979812d630 0x7f979812dab0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9790005fd0 tx=0x7f9790033bf0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 shutdown_connections 2023-10-18T10:26:38.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f978406b040 0x7f978406d500 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f979812d630 0x7f979812dab0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9798153d80 0x7f979812d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 --2- 172.21.15.125:0/1142569145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9780004d00 0x7f9798153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 >> 172.21.15.125:0/1142569145 conn(0x7f9798069930 msgr2=0x7f9798152290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:38.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 shutdown_connections 2023-10-18T10:26:38.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.755+0000 7f979e7d0640 1 -- 172.21.15.125:0/1142569145 wait complete. 2023-10-18T10:26:38.767 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-10-18T10:26:38.768 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1m 2023-10-18T10:26:38.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.871+0000 7f5a3864f640 1 -- 172.21.15.125:0/2378023971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a1c003970 msgr2=0x7f5a1c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.871+0000 7f5a3864f640 1 --2- 172.21.15.125:0/2378023971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a1c003970 0x7f5a1c003d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009ed0 tx=0x7f5a1c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.871+0000 7f5a36bec640 1 -- 172.21.15.125:0/2378023971 reap_dead start 2023-10-18T10:26:38.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.872+0000 7f5a3864f640 1 -- 172.21.15.125:0/2378023971 shutdown_connections 2023-10-18T10:26:38.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.872+0000 7f5a3864f640 1 -- 172.21.15.125:0/2378023971 >> 172.21.15.125:0/2378023971 conn(0x7f5a30069930 msgr2=0x7f5a30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:38.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.872+0000 7f5a3864f640 1 -- 172.21.15.125:0/2378023971 shutdown_connections 2023-10-18T10:26:38.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.872+0000 7f5a3864f640 1 -- 172.21.15.125:0/2378023971 wait complete. 2023-10-18T10:26:38.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.873+0000 7f5a3864f640 1 Processor -- start 2023-10-18T10:26:38.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.873+0000 7f5a3864f640 1 -- start start 2023-10-18T10:26:38.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.873+0000 7f5a3864f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1c003970 0x7f5a3015e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a3864f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a301483a0 0x7f5a3015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a3864f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a30144d40 0x7f5a301451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a3864f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301456f0 con 0x7f5a30144d40 2023-10-18T10:26:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a3864f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015d240 con 0x7f5a301483a0 2023-10-18T10:26:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a3864f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015d520 con 0x7f5a1c003970 2023-10-18T10:26:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a30144d40 0x7f5a301451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a30144d40 0x7f5a301451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41140/0 (socket says 172.21.15.125:41140) 2023-10-18T10:26:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 -- 172.21.15.125:0/3137232059 learned_addr learned my addr 172.21.15.125:0/3137232059 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a363eb640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1c003970 0x7f5a3015e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a35bea640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a301483a0 0x7f5a3015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 -- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1c003970 msgr2=0x7f5a3015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1c003970 0x7f5a3015e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 -- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a301483a0 msgr2=0x7f5a3015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a301483a0 0x7f5a3015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 -- 172.21.15.125:0/3137232059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3015d800 con 0x7f5a30144d40 2023-10-18T10:26:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a36bec640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a30144d40 0x7f5a301451b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5a300cac00 tx=0x7f5a28011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a26ffd640 1 -- 172.21.15.125:0/3137232059 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28004230 con 0x7f5a30144d40 2023-10-18T10:26:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a26ffd640 1 -- 172.21.15.125:0/3137232059 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a28004390 con 0x7f5a30144d40 2023-10-18T10:26:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3015dae0 con 0x7f5a30144d40 2023-10-18T10:26:38.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a26ffd640 1 -- 172.21.15.125:0/3137232059 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2800fa90 con 0x7f5a30144d40 2023-10-18T10:26:38.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.874+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3015a9b0 con 0x7f5a30144d40 2023-10-18T10:26:38.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.875+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3015e000 con 0x7f5a30144d40 2023-10-18T10:26:38.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.876+0000 7f5a26ffd640 1 -- 172.21.15.125:0/3137232059 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f5a28018070 con 0x7f5a30144d40 2023-10-18T10:26:38.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.876+0000 7f5a26ffd640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1406b040 0x7f5a1406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:38.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.876+0000 7f5a26ffd640 1 -- 172.21.15.125:0/3137232059 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f5a2808a540 con 0x7f5a30144d40 2023-10-18T10:26:38.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.876+0000 7f5a363eb640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1406b040 0x7f5a1406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:38.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.877+0000 7f5a363eb640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1406b040 0x7f5a1406d500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c037000 tx=0x7f5a2c004a40 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:38.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:38.880+0000 7f5a26ffd640 1 -- 172.21.15.125:0/3137232059 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a3015e000 con 0x7f5a30144d40 2023-10-18T10:26:39.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.065+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a3006fa70 con 0x7f5a1406b040 2023-10-18T10:26:39.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.117+0000 7f5a26ffd640 1 -- 172.21.15.125:0/3137232059 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5a2c040040 con 0x7f5a1406b040 2023-10-18T10:26:39.119 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-10-18T10:26:39.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1406b040 msgr2=0x7f5a1406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1406b040 0x7f5a1406d500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c037000 tx=0x7f5a2c004a40 comp rx=0 tx=0).stop 2023-10-18T10:26:39.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a30144d40 msgr2=0x7f5a301451b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a30144d40 0x7f5a301451b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5a300cac00 tx=0x7f5a28011b10 comp rx=0 tx=0).stop 2023-10-18T10:26:39.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 shutdown_connections 2023-10-18T10:26:39.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5a1406b040 0x7f5a1406d500 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a301483a0 0x7f5a3015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a30144d40 0x7f5a301451b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 --2- 172.21.15.125:0/3137232059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1c003970 0x7f5a3015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 >> 172.21.15.125:0/3137232059 conn(0x7f5a30069930 msgr2=0x7f5a30157210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:39.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 shutdown_connections 2023-10-18T10:26:39.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.119+0000 7f5a3864f640 1 -- 172.21.15.125:0/3137232059 wait complete. 2023-10-18T10:26:39.131 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-10-18T10:26:39.131 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1m 2023-10-18T10:26:39.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.234+0000 7fed6d533640 1 -- 172.21.15.125:0/2124407841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680cbd20 msgr2=0x7fed68158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.234+0000 7fed6d533640 1 --2- 172.21.15.125:0/2124407841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680cbd20 0x7fed68158300 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fed58005fd0 tx=0x7fed5802c0e0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.234+0000 7fed6d533640 1 -- 172.21.15.125:0/2124407841 shutdown_connections 2023-10-18T10:26:39.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.234+0000 7fed6d533640 1 --2- 172.21.15.125:0/2124407841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed680afa20 0x7fed680705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.234+0000 7fed6d533640 1 --2- 172.21.15.125:0/2124407841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680cbd20 0x7fed68158300 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.234+0000 7fed6d533640 1 -- 172.21.15.125:0/2124407841 >> 172.21.15.125:0/2124407841 conn(0x7fed68069930 msgr2=0x7fed68069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:39.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.234+0000 7fed6d533640 1 -- 172.21.15.125:0/2124407841 shutdown_connections 2023-10-18T10:26:39.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.234+0000 7fed6d533640 1 -- 172.21.15.125:0/2124407841 wait complete. 2023-10-18T10:26:39.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.235+0000 7fed6d533640 1 Processor -- start 2023-10-18T10:26:39.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.235+0000 7fed6d533640 1 -- start start 2023-10-18T10:26:39.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed6d533640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680afa20 0x7fed6815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed6d533640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed680b0530 0x7fed6813c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed6d533640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed680cbd20 0x7fed680a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed6d533640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68159000 con 0x7fed680afa20 2023-10-18T10:26:39.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed6d533640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681592e0 con 0x7fed680b0530 2023-10-18T10:26:39.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed6d533640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed681595c0 con 0x7fed680cbd20 2023-10-18T10:26:39.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed66ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680afa20 0x7fed6815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed66ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680afa20 0x7fed6815a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41146/0 (socket says 172.21.15.125:41146) 2023-10-18T10:26:39.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed66ffd640 1 -- 172.21.15.125:0/3519356307 learned_addr learned my addr 172.21.15.125:0/3519356307 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:39.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed66ffd640 1 -- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed680cbd20 msgr2=0x7fed680a7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed66ffd640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed680cbd20 0x7fed680a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed66ffd640 1 -- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed680b0530 msgr2=0x7fed6813c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed66ffd640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed680b0530 0x7fed6813c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.236+0000 7fed66ffd640 1 -- 172.21.15.125:0/3519356307 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed58009840 con 0x7fed680afa20 2023-10-18T10:26:39.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.237+0000 7fed66ffd640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680afa20 0x7fed6815a0a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fed58005fd0 tx=0x7fed58004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:39.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.237+0000 7fed4f7fe640 1 -- 172.21.15.125:0/3519356307 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58004760 con 0x7fed680afa20 2023-10-18T10:26:39.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.237+0000 7fed4f7fe640 1 -- 172.21.15.125:0/3519356307 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed580345a0 con 0x7fed680afa20 2023-10-18T10:26:39.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.237+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed68070cf0 con 0x7fed680afa20 2023-10-18T10:26:39.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.237+0000 7fed4f7fe640 1 -- 172.21.15.125:0/3519356307 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58034950 con 0x7fed680afa20 2023-10-18T10:26:39.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.237+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed68159b80 con 0x7fed680afa20 2023-10-18T10:26:39.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.238+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed680690e0 con 0x7fed680afa20 2023-10-18T10:26:39.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.238+0000 7fed4f7fe640 1 -- 172.21.15.125:0/3519356307 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fed58034ab0 con 0x7fed680afa20 2023-10-18T10:26:39.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.239+0000 7fed4f7fe640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fed5406b040 0x7fed5406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.239+0000 7fed4f7fe640 1 -- 172.21.15.125:0/3519356307 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fed580acfc0 con 0x7fed680afa20 2023-10-18T10:26:39.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.239+0000 7fed667fc640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fed5406b040 0x7fed5406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.239+0000 7fed667fc640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fed5406b040 0x7fed5406d500 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fed680ac4e0 tx=0x7fed5c005c70 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:39.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.243+0000 7fed4f7fe640 1 -- 172.21.15.125:0/3519356307 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed580dfca0 con 0x7fed680afa20 2023-10-18T10:26:39.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.429+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fed68159450 con 0x7fed5406b040 2023-10-18T10:26:39.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.470+0000 7fed4f7fe640 1 -- 172.21.15.125:0/3519356307 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fed68159450 con 0x7fed5406b040 2023-10-18T10:26:39.472 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-10-18T10:26:39.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fed5406b040 msgr2=0x7fed5406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fed5406b040 0x7fed5406d500 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fed680ac4e0 tx=0x7fed5c005c70 comp rx=0 tx=0).stop 2023-10-18T10:26:39.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680afa20 msgr2=0x7fed6815a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680afa20 0x7fed6815a0a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fed58005fd0 tx=0x7fed58004360 comp rx=0 tx=0).stop 2023-10-18T10:26:39.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 shutdown_connections 2023-10-18T10:26:39.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fed5406b040 0x7fed5406d500 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed680b0530 0x7fed6813c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed680cbd20 0x7fed680a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 --2- 172.21.15.125:0/3519356307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed680afa20 0x7fed6815a0a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 >> 172.21.15.125:0/3519356307 conn(0x7fed68069930 msgr2=0x7fed681580e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:39.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.472+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 shutdown_connections 2023-10-18T10:26:39.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.473+0000 7fed6d533640 1 -- 172.21.15.125:0/3519356307 wait complete. 2023-10-18T10:26:39.485 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-10-18T10:26:39.485 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1m 2023-10-18T10:26:39.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.592+0000 7f7207887640 1 -- 172.21.15.125:0/3847563754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 msgr2=0x7f72000aa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.592+0000 7f7207887640 1 --2- 172.21.15.125:0/3847563754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 0x7f72000aa830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005fd0 tx=0x7f71f802c0e0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.592+0000 7f7207887640 1 -- 172.21.15.125:0/3847563754 shutdown_connections 2023-10-18T10:26:39.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.592+0000 7f7207887640 1 --2- 172.21.15.125:0/3847563754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 0x7f72000aa830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.592+0000 7f7207887640 1 --2- 172.21.15.125:0/3847563754 >> v2:172.21.15.139:3300/0 conn(0x7f72000af4b0 0x7f72000aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.592+0000 7f7207887640 1 -- 172.21.15.125:0/3847563754 >> 172.21.15.125:0/3847563754 conn(0x7f72000b0cd0 msgr2=0x7f72000b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:39.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.592+0000 7f7207887640 1 -- 172.21.15.125:0/3847563754 shutdown_connections 2023-10-18T10:26:39.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.592+0000 7f7207887640 1 -- 172.21.15.125:0/3847563754 wait complete. 2023-10-18T10:26:39.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.593+0000 7f7207887640 1 Processor -- start 2023-10-18T10:26:39.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.593+0000 7f7207887640 1 -- start start 2023-10-18T10:26:39.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7207887640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 0x7f7200155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7207887640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72000af4b0 0x7f72001597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7207887640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72001573f0 0x7f72001522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7207887640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72001f25e0 con 0x7f72000af4b0 2023-10-18T10:26:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7207887640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7200159ff0 con 0x7f72001573f0 2023-10-18T10:26:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7207887640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720015a2d0 con 0x7f720006fdc0 2023-10-18T10:26:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72000af4b0 0x7f72001597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72000af4b0 0x7f72001597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41162/0 (socket says 172.21.15.125:41162) 2023-10-18T10:26:39.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 -- 172.21.15.125:0/3737234875 learned_addr learned my addr 172.21.15.125:0/3737234875 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:39.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7205e24640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72001573f0 0x7f72001522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7205623640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 0x7f7200155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 -- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 msgr2=0x7f7200155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 0x7f7200155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 -- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72001573f0 msgr2=0x7f72001522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72001573f0 0x7f72001522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 -- 172.21.15.125:0/3737234875 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8009840 con 0x7f72000af4b0 2023-10-18T10:26:39.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7205623640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 0x7f7200155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7204e22640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72000af4b0 0x7f72001597c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f71f000e9c0 tx=0x7f71f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f71f5ffb640 1 -- 172.21.15.125:0/3737234875 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0009700 con 0x7f72000af4b0 2023-10-18T10:26:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f71f5ffb640 1 -- 172.21.15.125:0/3737234875 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f71f000cd80 con 0x7f72000af4b0 2023-10-18T10:26:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72001f28a0 con 0x7f72000af4b0 2023-10-18T10:26:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.594+0000 7f71f5ffb640 1 -- 172.21.15.125:0/3737234875 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0005280 con 0x7f72000af4b0 2023-10-18T10:26:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.595+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7200159d00 con 0x7f72000af4b0 2023-10-18T10:26:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.596+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7200070210 con 0x7f72000af4b0 2023-10-18T10:26:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.596+0000 7f71f5ffb640 1 -- 172.21.15.125:0/3737234875 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f71f00053e0 con 0x7f72000af4b0 2023-10-18T10:26:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.596+0000 7f71f5ffb640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f71e406b040 0x7f71e406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.597+0000 7f71f5ffb640 1 -- 172.21.15.125:0/3737234875 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f71f0014070 con 0x7f72000af4b0 2023-10-18T10:26:39.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.597+0000 7f7205623640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f71e406b040 0x7f71e406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.597+0000 7f7205623640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f71e406b040 0x7f71e406d500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0046b0 tx=0x7f71fc009040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:39.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.601+0000 7f71f5ffb640 1 -- 172.21.15.125:0/3737234875 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71f0055210 con 0x7f72000af4b0 2023-10-18T10:26:39.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.788+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f720006ad70 con 0x7f71e406b040 2023-10-18T10:26:39.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.831+0000 7f71f5ffb640 1 -- 172.21.15.125:0/3737234875 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f71fc001c30 con 0x7f71e406b040 2023-10-18T10:26:39.833 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-10-18T10:26:39.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.832+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f71e406b040 msgr2=0x7f71e406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.832+0000 7f7207887640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f71e406b040 0x7f71e406d500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0046b0 tx=0x7f71fc009040 comp rx=0 tx=0).stop 2023-10-18T10:26:39.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.832+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72000af4b0 msgr2=0x7f72001597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.832+0000 7f7207887640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72000af4b0 0x7f72001597c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f71f000e9c0 tx=0x7f71f000ee80 comp rx=0 tx=0).stop 2023-10-18T10:26:39.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.833+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 shutdown_connections 2023-10-18T10:26:39.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.833+0000 7f7207887640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f71e406b040 0x7f71e406d500 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.833+0000 7f7207887640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72001573f0 0x7f72001522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.833+0000 7f7207887640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72000af4b0 0x7f72001597c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.833+0000 7f7207887640 1 --2- 172.21.15.125:0/3737234875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f720006fdc0 0x7f7200155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.833+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 >> 172.21.15.125:0/3737234875 conn(0x7f72000b0cd0 msgr2=0x7f720015ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:39.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.833+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 shutdown_connections 2023-10-18T10:26:39.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.833+0000 7f7207887640 1 -- 172.21.15.125:0/3737234875 wait complete. 2023-10-18T10:26:39.845 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-10-18T10:26:39.845 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1m 2023-10-18T10:26:39.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.948+0000 7f05a4d24640 1 -- 172.21.15.125:0/1061603140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05a015b2a0 msgr2=0x7f05a00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.948+0000 7f05a4d24640 1 --2- 172.21.15.125:0/1061603140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05a015b2a0 0x7f05a00d1910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0594009830 tx=0x7f059402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.949+0000 7f05a4d24640 1 -- 172.21.15.125:0/1061603140 shutdown_connections 2023-10-18T10:26:39.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.949+0000 7f05a4d24640 1 --2- 172.21.15.125:0/1061603140 >> v2:172.21.15.139:3300/0 conn(0x7f05a00ceb40 0x7f05a00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.949+0000 7f05a4d24640 1 --2- 172.21.15.125:0/1061603140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05a015b2a0 0x7f05a00d1910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.949+0000 7f05a4d24640 1 -- 172.21.15.125:0/1061603140 >> 172.21.15.125:0/1061603140 conn(0x7f05a00d0d60 msgr2=0x7f05a00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:39.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.949+0000 7f05a4d24640 1 -- 172.21.15.125:0/1061603140 shutdown_connections 2023-10-18T10:26:39.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.949+0000 7f05a4d24640 1 -- 172.21.15.125:0/1061603140 wait complete. 2023-10-18T10:26:39.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.950+0000 7f05a4d24640 1 Processor -- start 2023-10-18T10:26:39.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.950+0000 7f05a4d24640 1 -- start start 2023-10-18T10:26:39.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f05a4d24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05a00ceb40 0x7f05a015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f05a4d24640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05a00cf340 0x7f05a015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f05a4d24640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a015b2a0 0x7f05a00ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f05a4d24640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a015e970 con 0x7f05a015b2a0 2023-10-18T10:26:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f05a4d24640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a015d6b0 con 0x7f05a00ceb40 2023-10-18T10:26:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f05a4d24640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a015b800 con 0x7f05a00cf340 2023-10-18T10:26:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a015b2a0 0x7f05a00ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a015b2a0 0x7f05a00ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41174/0 (socket says 172.21.15.125:41174) 2023-10-18T10:26:39.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 -- 172.21.15.125:0/2285962968 learned_addr learned my addr 172.21.15.125:0/2285962968 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:39.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059dd9b640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05a00cf340 0x7f05a015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059e59c640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05a00ceb40 0x7f05a015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 -- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05a00cf340 msgr2=0x7f05a015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05a00cf340 0x7f05a015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 -- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05a00ceb40 msgr2=0x7f05a015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:39.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05a00ceb40 0x7f05a015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:39.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 -- 172.21.15.125:0/2285962968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009490 con 0x7f05a015b2a0 2023-10-18T10:26:39.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f059ed9d640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a015b2a0 0x7f05a00ae3a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f059800e9b0 tx=0x7f059800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f058effd640 1 -- 172.21.15.125:0/2285962968 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800cd80 con 0x7f05a015b2a0 2023-10-18T10:26:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f058effd640 1 -- 172.21.15.125:0/2285962968 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0598004020 con 0x7f05a015b2a0 2023-10-18T10:26:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0594009490 con 0x7f05a015b2a0 2023-10-18T10:26:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f058effd640 1 -- 172.21.15.125:0/2285962968 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800cee0 con 0x7f05a015b2a0 2023-10-18T10:26:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.951+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a015bd80 con 0x7f05a015b2a0 2023-10-18T10:26:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.952+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a00690e0 con 0x7f05a015b2a0 2023-10-18T10:26:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.953+0000 7f058effd640 1 -- 172.21.15.125:0/2285962968 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f05980035f0 con 0x7f05a015b2a0 2023-10-18T10:26:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.953+0000 7f058effd640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f058006b040 0x7f058006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.953+0000 7f058effd640 1 -- 172.21.15.125:0/2285962968 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f0598014070 con 0x7f05a015b2a0 2023-10-18T10:26:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.953+0000 7f059e59c640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f058006b040 0x7f058006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:39.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.954+0000 7f059e59c640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f058006b040 0x7f058006d500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f059000f290 tx=0x7f0590001a70 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:39.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:39.957+0000 7f058effd640 1 -- 172.21.15.125:0/2285962968 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0598055360 con 0x7f05a015b2a0 2023-10-18T10:26:40.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.144+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f05a00cc170 con 0x7f058006b040 2023-10-18T10:26:40.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.194+0000 7f058effd640 1 -- 172.21.15.125:0/2285962968 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f059000fe90 con 0x7f058006b040 2023-10-18T10:26:40.196 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-10-18T10:26:40.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f058006b040 msgr2=0x7f058006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f058006b040 0x7f058006d500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f059000f290 tx=0x7f0590001a70 comp rx=0 tx=0).stop 2023-10-18T10:26:40.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a015b2a0 msgr2=0x7f05a00ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a015b2a0 0x7f05a00ae3a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f059800e9b0 tx=0x7f059800ee70 comp rx=0 tx=0).stop 2023-10-18T10:26:40.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 shutdown_connections 2023-10-18T10:26:40.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f058006b040 0x7f058006d500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a015b2a0 0x7f05a00ae3a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05a00cf340 0x7f05a015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 --2- 172.21.15.125:0/2285962968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05a00ceb40 0x7f05a015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 >> 172.21.15.125:0/2285962968 conn(0x7f05a00d0d60 msgr2=0x7f05a00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:40.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 shutdown_connections 2023-10-18T10:26:40.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.196+0000 7f05a4d24640 1 -- 172.21.15.125:0/2285962968 wait complete. 2023-10-18T10:26:40.208 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-10-18T10:26:40.209 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1m 2023-10-18T10:26:40.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.312+0000 7f0d397a0640 1 -- 172.21.15.125:0/4226588000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 msgr2=0x7f0d1c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.312+0000 7f0d397a0640 1 --2- 172.21.15.125:0/4226588000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 0x7f0d1c003d90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0d24006fd0 tx=0x7f0d1c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.312+0000 7f0d397a0640 1 -- 172.21.15.125:0/4226588000 shutdown_connections 2023-10-18T10:26:40.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.312+0000 7f0d397a0640 1 --2- 172.21.15.125:0/4226588000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 0x7f0d1c003d90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.312+0000 7f0d397a0640 1 -- 172.21.15.125:0/4226588000 >> 172.21.15.125:0/4226588000 conn(0x7f0d3406fdc0 msgr2=0x7f0d340b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:40.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.312+0000 7f0d397a0640 1 -- 172.21.15.125:0/4226588000 shutdown_connections 2023-10-18T10:26:40.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.312+0000 7f0d397a0640 1 -- 172.21.15.125:0/4226588000 wait complete. 2023-10-18T10:26:40.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.314+0000 7f0d397a0640 1 Processor -- start 2023-10-18T10:26:40.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.314+0000 7f0d397a0640 1 -- start start 2023-10-18T10:26:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.314+0000 7f0d397a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 0x7f0d3415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.314+0000 7f0d397a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d341598b0 0x7f0d341580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.314+0000 7f0d397a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d34148330 0x7f0d3414e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.314+0000 7f0d397a0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d341585e0 con 0x7f0d1c003970 2023-10-18T10:26:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.314+0000 7f0d397a0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34158810 con 0x7f0d341598b0 2023-10-18T10:26:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.314+0000 7f0d397a0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34158af0 con 0x7f0d34148330 2023-10-18T10:26:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 0x7f0d3415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 0x7f0d3415af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41196/0 (socket says 172.21.15.125:41196) 2023-10-18T10:26:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 -- 172.21.15.125:0/411417639 learned_addr learned my addr 172.21.15.125:0/411417639 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d337fe640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d341598b0 0x7f0d341580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:40.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d38f9f640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d34148330 0x7f0d3414e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:40.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 -- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d34148330 msgr2=0x7f0d3414e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d34148330 0x7f0d3414e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 -- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d341598b0 msgr2=0x7f0d341580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d341598b0 0x7f0d341580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 -- 172.21.15.125:0/411417639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d34158dd0 con 0x7f0d1c003970 2023-10-18T10:26:40.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d33fff640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 0x7f0d3415af10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0d24001930 tx=0x7f0d240014f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:40.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d30ff9640 1 -- 172.21.15.125:0/411417639 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24006250 con 0x7f0d1c003970 2023-10-18T10:26:40.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d30ff9640 1 -- 172.21.15.125:0/411417639 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d240063b0 con 0x7f0d1c003970 2023-10-18T10:26:40.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d341590b0 con 0x7f0d1c003970 2023-10-18T10:26:40.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d30ff9640 1 -- 172.21.15.125:0/411417639 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240035f0 con 0x7f0d1c003970 2023-10-18T10:26:40.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.315+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d341487b0 con 0x7f0d1c003970 2023-10-18T10:26:40.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.316+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d340ae150 con 0x7f0d1c003970 2023-10-18T10:26:40.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.317+0000 7f0d30ff9640 1 -- 172.21.15.125:0/411417639 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f0d24031030 con 0x7f0d1c003970 2023-10-18T10:26:40.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.317+0000 7f0d30ff9640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f0d2006b070 0x7f0d2006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:40.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.317+0000 7f0d30ff9640 1 -- 172.21.15.125:0/411417639 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f0d240aaeb0 con 0x7f0d1c003970 2023-10-18T10:26:40.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.317+0000 7f0d337fe640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f0d2006b070 0x7f0d2006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:40.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.318+0000 7f0d337fe640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f0d2006b070 0x7f0d2006d530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d28005fd0 tx=0x7f0d28005c50 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:40.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.321+0000 7f0d30ff9640 1 -- 172.21.15.125:0/411417639 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d24075a20 con 0x7f0d1c003970 2023-10-18T10:26:40.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.510+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d34065f70 con 0x7f0d2006b070 2023-10-18T10:26:40.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.557+0000 7f0d30ff9640 1 -- 172.21.15.125:0/411417639 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0d28016070 con 0x7f0d2006b070 2023-10-18T10:26:40.560 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-10-18T10:26:40.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f0d2006b070 msgr2=0x7f0d2006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f0d2006b070 0x7f0d2006d530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d28005fd0 tx=0x7f0d28005c50 comp rx=0 tx=0).stop 2023-10-18T10:26:40.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 msgr2=0x7f0d3415af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 0x7f0d3415af10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0d24001930 tx=0x7f0d240014f0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 shutdown_connections 2023-10-18T10:26:40.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f0d2006b070 0x7f0d2006d530 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d341598b0 0x7f0d341580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d34148330 0x7f0d3414e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 --2- 172.21.15.125:0/411417639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d1c003970 0x7f0d3415af10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.559+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 >> 172.21.15.125:0/411417639 conn(0x7f0d3406fdc0 msgr2=0x7f0d34155d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:40.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.560+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 shutdown_connections 2023-10-18T10:26:40.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.560+0000 7f0d397a0640 1 -- 172.21.15.125:0/411417639 wait complete. 2023-10-18T10:26:40.572 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-10-18T10:26:40.572 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1m 2023-10-18T10:26:40.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.676+0000 7fb17a9f0640 1 -- 172.21.15.125:0/2173616126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb174142850 msgr2=0x7fb174159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.676+0000 7fb17a9f0640 1 --2- 172.21.15.125:0/2173616126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb174142850 0x7fb174159440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb164009860 tx=0x7fb16402bc20 comp rx=0 tx=0).stop 2023-10-18T10:26:40.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.676+0000 7fb17a9f0640 1 -- 172.21.15.125:0/2173616126 shutdown_connections 2023-10-18T10:26:40.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.676+0000 7fb17a9f0640 1 --2- 172.21.15.125:0/2173616126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb174142850 0x7fb174159440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.676+0000 7fb17a9f0640 1 -- 172.21.15.125:0/2173616126 >> 172.21.15.125:0/2173616126 conn(0x7fb1740af7a0 msgr2=0x7fb1740afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:40.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.676+0000 7fb17a9f0640 1 -- 172.21.15.125:0/2173616126 shutdown_connections 2023-10-18T10:26:40.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.676+0000 7fb17a9f0640 1 -- 172.21.15.125:0/2173616126 wait complete. 2023-10-18T10:26:40.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.677+0000 7fb17a9f0640 1 Processor -- start 2023-10-18T10:26:40.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.677+0000 7fb17a9f0640 1 -- start start 2023-10-18T10:26:40.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb17a9f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb174069e30 0x7fb17413a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:40.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb17a9f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1740cd6d0 0x7fb174137630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:40.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb17a9f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb174142850 0x7fb1741de960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:40.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb17a9f0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb174139280 con 0x7fb1740cd6d0 2023-10-18T10:26:40.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb17a9f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb174139560 con 0x7fb174142850 2023-10-18T10:26:40.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb17a9f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb174139840 con 0x7fb174069e30 2023-10-18T10:26:40.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1740cd6d0 0x7fb174137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:40.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1740cd6d0 0x7fb174137630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41206/0 (socket says 172.21.15.125:41206) 2023-10-18T10:26:40.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 -- 172.21.15.125:0/396898365 learned_addr learned my addr 172.21.15.125:0/396898365 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:40.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb173fff640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb174069e30 0x7fb17413a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:40.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb178f8d640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb174142850 0x7fb1741de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:40.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 -- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb174069e30 msgr2=0x7fb17413a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb174069e30 0x7fb17413a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 -- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb174142850 msgr2=0x7fb1741de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb174142850 0x7fb1741de960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 -- 172.21.15.125:0/396898365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb164009490 con 0x7fb1740cd6d0 2023-10-18T10:26:40.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.678+0000 7fb1737fe640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1740cd6d0 0x7fb174137630 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb1680010e0 tx=0x7fb16800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:40.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.679+0000 7fb170ff9640 1 -- 172.21.15.125:0/396898365 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1680052d0 con 0x7fb1740cd6d0 2023-10-18T10:26:40.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.679+0000 7fb170ff9640 1 -- 172.21.15.125:0/396898365 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb168005450 con 0x7fb1740cd6d0 2023-10-18T10:26:40.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.679+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb174139b20 con 0x7fb1740cd6d0 2023-10-18T10:26:40.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.679+0000 7fb170ff9640 1 -- 172.21.15.125:0/396898365 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168012890 con 0x7fb1740cd6d0 2023-10-18T10:26:40.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.679+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17413a860 con 0x7fb1740cd6d0 2023-10-18T10:26:40.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.680+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1741df760 con 0x7fb1740cd6d0 2023-10-18T10:26:40.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.680+0000 7fb170ff9640 1 -- 172.21.15.125:0/396898365 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fb1680035f0 con 0x7fb1740cd6d0 2023-10-18T10:26:40.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.680+0000 7fb170ff9640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb16006b040 0x7fb16006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:40.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.680+0000 7fb170ff9640 1 -- 172.21.15.125:0/396898365 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fb1680898d0 con 0x7fb1740cd6d0 2023-10-18T10:26:40.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.681+0000 7fb173fff640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb16006b040 0x7fb16006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:40.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.681+0000 7fb173fff640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb16006b040 0x7fb16006d500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb164036000 tx=0x7fb164004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:40.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.685+0000 7fb170ff9640 1 -- 172.21.15.125:0/396898365 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1741df760 con 0x7fb1740cd6d0 2023-10-18T10:26:40.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.870+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb17413a040 con 0x7fb16006b040 2023-10-18T10:26:40.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.919+0000 7fb170ff9640 1 -- 172.21.15.125:0/396898365 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb17413a040 con 0x7fb16006b040 2023-10-18T10:26:40.921 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-10-18T10:26:40.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb16006b040 msgr2=0x7fb16006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb16006b040 0x7fb16006d500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb164036000 tx=0x7fb164004730 comp rx=0 tx=0).stop 2023-10-18T10:26:40.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1740cd6d0 msgr2=0x7fb174137630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:40.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1740cd6d0 0x7fb174137630 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb1680010e0 tx=0x7fb16800c980 comp rx=0 tx=0).stop 2023-10-18T10:26:40.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 shutdown_connections 2023-10-18T10:26:40.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb16006b040 0x7fb16006d500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb174142850 0x7fb1741de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1740cd6d0 0x7fb174137630 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 --2- 172.21.15.125:0/396898365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb174069e30 0x7fb17413a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:40.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 >> 172.21.15.125:0/396898365 conn(0x7fb1740af7a0 msgr2=0x7fb174141670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:40.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 shutdown_connections 2023-10-18T10:26:40.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:40.921+0000 7fb17a9f0640 1 -- 172.21.15.125:0/396898365 wait complete. 2023-10-18T10:26:40.933 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-10-18T10:26:40.933 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1m 2023-10-18T10:26:41.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.036+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/722030837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa00cf340 msgr2=0x7fcaa00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.036+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/722030837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa00cf340 0x7fcaa00d1910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca98005fd0 tx=0x7fca9802c050 comp rx=0 tx=0).stop 2023-10-18T10:26:41.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.036+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/722030837 shutdown_connections 2023-10-18T10:26:41.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.036+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/722030837 >> v2:172.21.15.139:3300/0 conn(0x7fcaa015b2a0 0x7fcaa00cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.036+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/722030837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa00cf340 0x7fcaa00d1910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.036+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/722030837 >> 172.21.15.125:0/722030837 conn(0x7fcaa00d0d60 msgr2=0x7fcaa00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:41.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.036+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/722030837 shutdown_connections 2023-10-18T10:26:41.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.036+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/722030837 wait complete. 2023-10-18T10:26:41.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa7e7c640 1 Processor -- start 2023-10-18T10:26:41.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa7e7c640 1 -- start start 2023-10-18T10:26:41.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa7e7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaa00ceb40 0x7fcaa0156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa7e7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaa00cf340 0x7fcaa01e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa7e7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa015b2a0 0x7fcaa01580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa7e7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa00b6cc0 con 0x7fcaa00ceb40 2023-10-18T10:26:41.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa7e7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa01e5280 con 0x7fcaa00cf340 2023-10-18T10:26:41.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa7e7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa01e3ca0 con 0x7fcaa015b2a0 2023-10-18T10:26:41.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa5c18640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaa00ceb40 0x7fcaa0156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa5c18640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaa00ceb40 0x7fcaa0156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41220/0 (socket says 172.21.15.125:41220) 2023-10-18T10:26:41.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.038+0000 7fcaa5c18640 1 -- 172.21.15.125:0/4207604552 learned_addr learned my addr 172.21.15.125:0/4207604552 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:41.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa5417640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaa00cf340 0x7fcaa01e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa6419640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa015b2a0 0x7fcaa01580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa5c18640 1 -- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa015b2a0 msgr2=0x7fcaa01580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa5c18640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa015b2a0 0x7fcaa01580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa5c18640 1 -- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaa00cf340 msgr2=0x7fcaa01e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa5c18640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaa00cf340 0x7fcaa01e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa5c18640 1 -- 172.21.15.125:0/4207604552 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98009840 con 0x7fcaa00ceb40 2023-10-18T10:26:41.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa6419640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa015b2a0 0x7fcaa01580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:26:41.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa5c18640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaa00ceb40 0x7fcaa0156e10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fca98036040 tx=0x7fca980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:41.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fca967fc640 1 -- 172.21.15.125:0/4207604552 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9803e460 con 0x7fcaa00ceb40 2023-10-18T10:26:41.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fca967fc640 1 -- 172.21.15.125:0/4207604552 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fca9803e6f0 con 0x7fcaa00ceb40 2023-10-18T10:26:41.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fca967fc640 1 -- 172.21.15.125:0/4207604552 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9803eb00 con 0x7fcaa00ceb40 2023-10-18T10:26:41.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa01e3f80 con 0x7fcaa00ceb40 2023-10-18T10:26:41.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.039+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa01e4540 con 0x7fcaa00ceb40 2023-10-18T10:26:41.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.040+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa00690e0 con 0x7fcaa00ceb40 2023-10-18T10:26:41.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.040+0000 7fca967fc640 1 -- 172.21.15.125:0/4207604552 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fca9803ec60 con 0x7fcaa00ceb40 2023-10-18T10:26:41.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.041+0000 7fca967fc640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fca8406b040 0x7fca8406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.041+0000 7fca967fc640 1 -- 172.21.15.125:0/4207604552 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fca980acc30 con 0x7fcaa00ceb40 2023-10-18T10:26:41.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.041+0000 7fcaa5417640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fca8406b040 0x7fca8406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.041+0000 7fcaa5417640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fca8406b040 0x7fca8406d500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00c74e0 tx=0x7fca90005f90 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:41.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.045+0000 7fca967fc640 1 -- 172.21.15.125:0/4207604552 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca98077940 con 0x7fcaa00ceb40 2023-10-18T10:26:41.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.231+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaa00cc170 con 0x7fca8406b040 2023-10-18T10:26:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.276+0000 7fca967fc640 1 -- 172.21.15.125:0/4207604552 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fca90016070 con 0x7fca8406b040 2023-10-18T10:26:41.278 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-10-18T10:26:41.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fca8406b040 msgr2=0x7fca8406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fca8406b040 0x7fca8406d500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00c74e0 tx=0x7fca90005f90 comp rx=0 tx=0).stop 2023-10-18T10:26:41.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaa00ceb40 msgr2=0x7fcaa0156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaa00ceb40 0x7fcaa0156e10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fca98036040 tx=0x7fca980042f0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 shutdown_connections 2023-10-18T10:26:41.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fca8406b040 0x7fca8406d500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaa00cf340 0x7fcaa01e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcaa015b2a0 0x7fcaa01580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 --2- 172.21.15.125:0/4207604552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaa00ceb40 0x7fcaa0156e10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 >> 172.21.15.125:0/4207604552 conn(0x7fcaa00d0d60 msgr2=0x7fcaa015a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:41.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 shutdown_connections 2023-10-18T10:26:41.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.278+0000 7fcaa7e7c640 1 -- 172.21.15.125:0/4207604552 wait complete. 2023-10-18T10:26:41.290 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-10-18T10:26:41.290 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1m 2023-10-18T10:26:41.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.393+0000 7ffa116a9640 1 -- 172.21.15.125:0/1607788543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 msgr2=0x7ffa0c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.393+0000 7ffa116a9640 1 --2- 172.21.15.125:0/1607788543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 0x7ffa0c0d1910 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009830 tx=0x7ff9f402bd60 comp rx=0 tx=0).stop 2023-10-18T10:26:41.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.393+0000 7ffa116a9640 1 -- 172.21.15.125:0/1607788543 shutdown_connections 2023-10-18T10:26:41.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.393+0000 7ffa116a9640 1 --2- 172.21.15.125:0/1607788543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 0x7ffa0c0d1910 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.393+0000 7ffa116a9640 1 -- 172.21.15.125:0/1607788543 >> 172.21.15.125:0/1607788543 conn(0x7ffa0c0d0d60 msgr2=0x7ffa0c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:41.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.393+0000 7ffa116a9640 1 -- 172.21.15.125:0/1607788543 shutdown_connections 2023-10-18T10:26:41.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.393+0000 7ffa116a9640 1 -- 172.21.15.125:0/1607788543 wait complete. 2023-10-18T10:26:41.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.394+0000 7ffa116a9640 1 Processor -- start 2023-10-18T10:26:41.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.394+0000 7ffa116a9640 1 -- start start 2023-10-18T10:26:41.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.394+0000 7ffa116a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa0c0ceb40 0x7ffa0c0b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.394+0000 7ffa116a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa0c0cf340 0x7ffa0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa116a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 0x7ffa0c1f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa116a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0b6970 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa116a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0b3cc0 con 0x7ffa0c0ceb40 2023-10-18T10:26:41.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa116a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0b3fa0 con 0x7ffa0c0cf340 2023-10-18T10:26:41.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 0x7ffa0c1f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa0c0ceb40 0x7ffa0c0b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 0x7ffa0c1f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41236/0 (socket says 172.21.15.125:41236) 2023-10-18T10:26:41.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 -- 172.21.15.125:0/3170810133 learned_addr learned my addr 172.21.15.125:0/3170810133 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:41.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa03fff640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa0c0cf340 0x7ffa0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 -- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa0c0cf340 msgr2=0x7ffa0c0b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa0c0cf340 0x7ffa0c0b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 -- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa0c0ceb40 msgr2=0x7ffa0c0b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa0c0ceb40 0x7ffa0c0b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 -- 172.21.15.125:0/3170810133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f4009490 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa0b7fe640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 0x7ffa0c1f8d00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c760 tx=0x7ffa0400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:41.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa037fe640 1 -- 172.21.15.125:0/3170810133 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04003da0 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa037fe640 1 -- 172.21.15.125:0/3170810133 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa0400ce70 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c0b4280 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa037fe640 1 -- 172.21.15.125:0/3170810133 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040052b0 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.395+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c0b4840 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.396+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c0690e0 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.397+0000 7ffa037fe640 1 -- 172.21.15.125:0/3170810133 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ffa04005410 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.397+0000 7ffa037fe640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff9ec06b040 0x7ff9ec06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.397+0000 7ffa037fe640 1 -- 172.21.15.125:0/3170810133 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7ffa04089a90 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.397+0000 7ffa0affd640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff9ec06b040 0x7ff9ec06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.398+0000 7ffa0affd640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff9ec06b040 0x7ff9ec06d500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000f60 tx=0x7ff9fc009040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:41.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.401+0000 7ffa037fe640 1 -- 172.21.15.125:0/3170810133 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa04054730 con 0x7ffa0c15b2a0 2023-10-18T10:26:41.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.591+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa0c0d0b40 con 0x7ff9ec06b040 2023-10-18T10:26:41.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.631+0000 7ffa037fe640 1 -- 172.21.15.125:0/3170810133 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ffa0c0d0b40 con 0x7ff9ec06b040 2023-10-18T10:26:41.633 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-10-18T10:26:41.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff9ec06b040 msgr2=0x7ff9ec06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff9ec06b040 0x7ff9ec06d500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000f60 tx=0x7ff9fc009040 comp rx=0 tx=0).stop 2023-10-18T10:26:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 msgr2=0x7ffa0c1f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 0x7ffa0c1f8d00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c760 tx=0x7ffa0400cc20 comp rx=0 tx=0).stop 2023-10-18T10:26:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 shutdown_connections 2023-10-18T10:26:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff9ec06b040 0x7ff9ec06d500 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa0c15b2a0 0x7ffa0c1f8d00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa0c0cf340 0x7ffa0c0b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 --2- 172.21.15.125:0/3170810133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa0c0ceb40 0x7ffa0c0b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 >> 172.21.15.125:0/3170810133 conn(0x7ffa0c0d0d60 msgr2=0x7ffa0c0cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 shutdown_connections 2023-10-18T10:26:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.633+0000 7ffa116a9640 1 -- 172.21.15.125:0/3170810133 wait complete. 2023-10-18T10:26:41.646 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-10-18T10:26:41.646 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1m 2023-10-18T10:26:41.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.750+0000 7f635e7a9640 1 -- 172.21.15.125:0/2288531706 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63580ceb40 msgr2=0x7f63580b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.750+0000 7f635e7a9640 1 --2- 172.21.15.125:0/2288531706 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63580ceb40 0x7f63580b0650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6350005fd0 tx=0x7f635002c050 comp rx=0 tx=0).stop 2023-10-18T10:26:41.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.750+0000 7f635e7a9640 1 -- 172.21.15.125:0/2288531706 shutdown_connections 2023-10-18T10:26:41.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.750+0000 7f635e7a9640 1 --2- 172.21.15.125:0/2288531706 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63580ceb40 0x7f63580b0650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.750+0000 7f635e7a9640 1 --2- 172.21.15.125:0/2288531706 >> v2:172.21.15.139:3300/0 conn(0x7f63580cf340 0x7f63580d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.750+0000 7f635e7a9640 1 -- 172.21.15.125:0/2288531706 >> 172.21.15.125:0/2288531706 conn(0x7f63580d0d60 msgr2=0x7f63580d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:41.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.750+0000 7f635e7a9640 1 -- 172.21.15.125:0/2288531706 shutdown_connections 2023-10-18T10:26:41.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.750+0000 7f635e7a9640 1 -- 172.21.15.125:0/2288531706 wait complete. 2023-10-18T10:26:41.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.751+0000 7f635e7a9640 1 Processor -- start 2023-10-18T10:26:41.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.752+0000 7f635e7a9640 1 -- start start 2023-10-18T10:26:41.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.752+0000 7f635e7a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63580ceb40 0x7f63581f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.752+0000 7f635e7a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63580cf340 0x7f63580b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.752+0000 7f635e7a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f635815b2a0 0x7f63580aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.752+0000 7f635e7a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63580c85e0 con 0x7f635815b2a0 2023-10-18T10:26:41.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.752+0000 7f635e7a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63580c7fd0 con 0x7f63580cf340 2023-10-18T10:26:41.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.752+0000 7f635e7a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63580b54e0 con 0x7f63580ceb40 2023-10-18T10:26:41.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f635815b2a0 0x7f63580aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f6357fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63580ceb40 0x7f63581f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f635815b2a0 0x7f63580aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41242/0 (socket says 172.21.15.125:41242) 2023-10-18T10:26:41.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f63577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63580cf340 0x7f63580b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 -- 172.21.15.125:0/1190711852 learned_addr learned my addr 172.21.15.125:0/1190711852 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:41.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 -- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63580ceb40 msgr2=0x7f63581f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63580ceb40 0x7f63581f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 -- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63580cf340 msgr2=0x7f63580b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63580cf340 0x7f63580b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 -- 172.21.15.125:0/1190711852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348009490 con 0x7f635815b2a0 2023-10-18T10:26:41.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635cd46640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f635815b2a0 0x7f63580aec60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6350009b30 tx=0x7f6350004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:41.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f6354ff9640 1 -- 172.21.15.125:0/1190711852 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350004870 con 0x7f635815b2a0 2023-10-18T10:26:41.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f6354ff9640 1 -- 172.21.15.125:0/1190711852 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f635003e550 con 0x7f635815b2a0 2023-10-18T10:26:41.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f6354ff9640 1 -- 172.21.15.125:0/1190711852 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635003e960 con 0x7f635815b2a0 2023-10-18T10:26:41.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6350009840 con 0x7f635815b2a0 2023-10-18T10:26:41.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.753+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63580b0d50 con 0x7f635815b2a0 2023-10-18T10:26:41.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.754+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63580c7e30 con 0x7f635815b2a0 2023-10-18T10:26:41.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.755+0000 7f6354ff9640 1 -- 172.21.15.125:0/1190711852 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f635003eac0 con 0x7f635815b2a0 2023-10-18T10:26:41.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.755+0000 7f6354ff9640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f634406b040 0x7f634406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:41.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.755+0000 7f6354ff9640 1 -- 172.21.15.125:0/1190711852 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f63500ac9d0 con 0x7f635815b2a0 2023-10-18T10:26:41.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.755+0000 7f6357fff640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f634406b040 0x7f634406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:41.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.756+0000 7f6357fff640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f634406b040 0x7f634406d500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6348009d60 tx=0x7f6348005f90 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:41.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.759+0000 7f6354ff9640 1 -- 172.21.15.125:0/1190711852 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63580c7e30 con 0x7f635815b2a0 2023-10-18T10:26:41.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.945+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f63580c7e30 con 0x7f634406b040 2023-10-18T10:26:41.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.986+0000 7f6354ff9640 1 -- 172.21.15.125:0/1190711852 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f63580c7e30 con 0x7f634406b040 2023-10-18T10:26:41.988 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-10-18T10:26:41.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f634406b040 msgr2=0x7f634406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f634406b040 0x7f634406d500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6348009d60 tx=0x7f6348005f90 comp rx=0 tx=0).stop 2023-10-18T10:26:41.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f635815b2a0 msgr2=0x7f63580aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:41.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f635815b2a0 0x7f63580aec60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6350009b30 tx=0x7f6350004270 comp rx=0 tx=0).stop 2023-10-18T10:26:41.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 shutdown_connections 2023-10-18T10:26:41.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f634406b040 0x7f634406d500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63580cf340 0x7f63580b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f635815b2a0 0x7f63580aec60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 --2- 172.21.15.125:0/1190711852 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63580ceb40 0x7f63581f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:41.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 >> 172.21.15.125:0/1190711852 conn(0x7f63580d0d60 msgr2=0x7f63580ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:41.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 shutdown_connections 2023-10-18T10:26:41.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:41.988+0000 7f635e7a9640 1 -- 172.21.15.125:0/1190711852 wait complete. 2023-10-18T10:26:42.000 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-10-18T10:26:42.001 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1m 2023-10-18T10:26:42.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.104+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2125250223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40b0530 msgr2=0x7f76e40b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:42.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.104+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2125250223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40b0530 0x7f76e40b09a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f76e40cc490 tx=0x7f76dc02c260 comp rx=0 tx=0).stop 2023-10-18T10:26:42.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.104+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2125250223 shutdown_connections 2023-10-18T10:26:42.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.104+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2125250223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40b0530 0x7f76e40b09a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.104+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2125250223 >> v2:172.21.15.139:3300/0 conn(0x7f76e40afa20 0x7f76e4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.104+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2125250223 >> 172.21.15.125:0/2125250223 conn(0x7f76e4069930 msgr2=0x7f76e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:42.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.104+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2125250223 shutdown_connections 2023-10-18T10:26:42.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.104+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2125250223 wait complete. 2023-10-18T10:26:42.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.105+0000 7f76ea3bb640 1 Processor -- start 2023-10-18T10:26:42.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.105+0000 7f76ea3bb640 1 -- start start 2023-10-18T10:26:42.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76ea3bb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40afa20 0x7f76e41f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:42.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76ea3bb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e40b0530 0x7f76e41422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:42.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76ea3bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e40cbd20 0x7f76e41434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:42.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76ea3bb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e41f32b0 con 0x7f76e40afa20 2023-10-18T10:26:42.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76ea3bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e414a600 con 0x7f76e40cbd20 2023-10-18T10:26:42.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76ea3bb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e414a8e0 con 0x7f76e40b0530 2023-10-18T10:26:42.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40afa20 0x7f76e41f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:42.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40afa20 0x7f76e41f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41266/0 (socket says 172.21.15.125:41266) 2023-10-18T10:26:42.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 -- 172.21.15.125:0/2112631229 learned_addr learned my addr 172.21.15.125:0/2112631229 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:26:42.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e37fe640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e40b0530 0x7f76e41422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:42.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e8958640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e40cbd20 0x7f76e41434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:42.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 -- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e40b0530 msgr2=0x7f76e41422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:42.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e40b0530 0x7f76e41422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 -- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e40cbd20 msgr2=0x7f76e41434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:42.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e40cbd20 0x7f76e41434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 -- 172.21.15.125:0/2112631229 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc009900 con 0x7f76e40afa20 2023-10-18T10:26:42.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e3fff640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40afa20 0x7f76e41f3740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f76d400b3f0 tx=0x7f76d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:42.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.106+0000 7f76e0ff9640 1 -- 172.21.15.125:0/2112631229 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4011830 con 0x7f76e40afa20 2023-10-18T10:26:42.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.107+0000 7f76e0ff9640 1 -- 172.21.15.125:0/2112631229 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76d4011a80 con 0x7f76e40afa20 2023-10-18T10:26:42.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.107+0000 7f76e0ff9640 1 -- 172.21.15.125:0/2112631229 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4011e90 con 0x7f76e40afa20 2023-10-18T10:26:42.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.107+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e41f2c60 con 0x7f76e40afa20 2023-10-18T10:26:42.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.107+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e41427f0 con 0x7f76e40afa20 2023-10-18T10:26:42.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.108+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e40690e0 con 0x7f76e40afa20 2023-10-18T10:26:42.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.108+0000 7f76e0ff9640 1 -- 172.21.15.125:0/2112631229 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f76d401a400 con 0x7f76e40afa20 2023-10-18T10:26:42.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.108+0000 7f76e0ff9640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f76d006b040 0x7f76d006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:26:42.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.108+0000 7f76e0ff9640 1 -- 172.21.15.125:0/2112631229 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f76d4089c90 con 0x7f76e40afa20 2023-10-18T10:26:42.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.109+0000 7f76e37fe640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f76d006b040 0x7f76d006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:26:42.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.109+0000 7f76e37fe640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f76d006b040 0x7f76d006d500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76d8005fd0 tx=0x7f76d8005850 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:26:42.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.113+0000 7f76e0ff9640 1 -- 172.21.15.125:0/2112631229 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76d4054880 con 0x7f76e40afa20 2023-10-18T10:26:42.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.302+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f76e40cc100 con 0x7f76d006b040 2023-10-18T10:26:42.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.343+0000 7f76e0ff9640 1 -- 172.21.15.125:0/2112631229 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f76d800fb30 con 0x7f76d006b040 2023-10-18T10:26:42.345 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-10-18T10:26:42.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f76d006b040 msgr2=0x7f76d006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:42.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f76d006b040 0x7f76d006d500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76d8005fd0 tx=0x7f76d8005850 comp rx=0 tx=0).stop 2023-10-18T10:26:42.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40afa20 msgr2=0x7f76e41f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:26:42.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40afa20 0x7f76e41f3740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f76d400b3f0 tx=0x7f76d400b8b0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 shutdown_connections 2023-10-18T10:26:42.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f76d006b040 0x7f76d006d500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76e40cbd20 0x7f76e41434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e40b0530 0x7f76e41422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 --2- 172.21.15.125:0/2112631229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e40afa20 0x7f76e41f3740 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:26:42.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.345+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 >> 172.21.15.125:0/2112631229 conn(0x7f76e4069930 msgr2=0x7f76e40aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:26:42.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.346+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 shutdown_connections 2023-10-18T10:26:42.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:26:42.346+0000 7f76ea3bb640 1 -- 172.21.15.125:0/2112631229 wait complete. 2023-10-18T10:26:42.358 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-10-18T10:27:00.129 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:27:00.127+0000 7f293f88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:27:00.130 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:27:00.127+0000 7f293f88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:27:00.130 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:27:00.127+0000 7f293f88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:27:00.130 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:27:00.127+0000 7f293f88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:27:00.131 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:27:00.127+0000 7f293f88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:27:02.359 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:27:02.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:27:32.582 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:27:32.586 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:28:02.746 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:28:02.750 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:28:32.901 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:28:32.905 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:29:02.970 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:29:02.974 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:29:33.108 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:29:33.112 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:30:00.372 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:30:00.370+0000 7f2942892640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:30:00.372 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:30:00.370+0000 7f2942892640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:30:00.373 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:30:00.370+0000 7f2942892640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:30:00.373 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:30:00.370+0000 7f2942892640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:30:00.373 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:30:00.370+0000 7f2942892640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:30:03.196 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:30:03.199 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:30:33.346 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:30:33.350 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:31:03.484 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:31:03.488 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:31:33.567 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:31:33.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:32:00.291 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:32:00.289+0000 7f2943093640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:32:00.291 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:32:00.289+0000 7f2943093640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:32:00.291 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:32:00.289+0000 7f2943093640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:32:00.292 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:32:00.289+0000 7f2943093640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:32:00.292 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:32:00.289+0000 7f2943093640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:32:03.685 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:32:03.688 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:32:33.760 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:32:33.764 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:33:03.830 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:33:03.833 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:33:33.977 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:33:33.982 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:34:04.240 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:34:04.244 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:34:34.352 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:34:34.356 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:35:04.515 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:35:04.522 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:35:34.669 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:35:34.673 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:36:04.732 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:36:04.736 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:36:34.879 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:36:34.882 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:37:04.974 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:37:04.978 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:37:19.359 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1m 2023-10-18T10:37:19.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.474+0000 7fe5915a4640 1 -- 172.21.15.125:0/2401891064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0cbd20 msgr2=0x7fe58c158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:19.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.474+0000 7fe5915a4640 1 --2- 172.21.15.125:0/2401891064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0cbd20 0x7fe58c158300 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0b10f0 tx=0x7fe57c02bdb0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.474+0000 7fe5915a4640 1 -- 172.21.15.125:0/2401891064 shutdown_connections 2023-10-18T10:37:19.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.474+0000 7fe5915a4640 1 --2- 172.21.15.125:0/2401891064 >> v2:172.21.15.139:3300/0 conn(0x7fe58c0b0530 0x7fe58c0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.474+0000 7fe5915a4640 1 --2- 172.21.15.125:0/2401891064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0cbd20 0x7fe58c158300 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.474+0000 7fe5915a4640 1 -- 172.21.15.125:0/2401891064 >> 172.21.15.125:0/2401891064 conn(0x7fe58c069930 msgr2=0x7fe58c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:19.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.474+0000 7fe5915a4640 1 -- 172.21.15.125:0/2401891064 shutdown_connections 2023-10-18T10:37:19.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.474+0000 7fe5915a4640 1 -- 172.21.15.125:0/2401891064 wait complete. 2023-10-18T10:37:19.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.476+0000 7fe5915a4640 1 Processor -- start 2023-10-18T10:37:19.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.476+0000 7fe5915a4640 1 -- start start 2023-10-18T10:37:19.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5915a4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe58c0afa20 0x7fe58c14b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:19.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5915a4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0b0530 0x7fe58c143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:19.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5915a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe58c0cbd20 0x7fe58c130860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:19.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5915a4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c14cd60 con 0x7fe58c0b0530 2023-10-18T10:37:19.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5915a4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c14d040 con 0x7fe58c0cbd20 2023-10-18T10:37:19.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5915a4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c14d320 con 0x7fe58c0afa20 2023-10-18T10:37:19.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0b0530 0x7fe58c143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:19.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0b0530 0x7fe58c143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59394/0 (socket says 172.21.15.125:59394) 2023-10-18T10:37:19.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 -- 172.21.15.125:0/4202649861 learned_addr learned my addr 172.21.15.125:0/4202649861 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:19.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe58affd640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe58c0afa20 0x7fe58c14b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:19.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe58b7fe640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe58c0cbd20 0x7fe58c130860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:19.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 -- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe58c0afa20 msgr2=0x7fe58c14b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:19.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe58c0afa20 0x7fe58c14b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 -- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe58c0cbd20 msgr2=0x7fe58c130860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:19.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe58c0cbd20 0x7fe58c130860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 -- 172.21.15.125:0/4202649861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009560 con 0x7fe58c0b0530 2023-10-18T10:37:19.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.477+0000 7fe5827fc640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0b0530 0x7fe58c143d40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c410 tx=0x7fe57400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:19.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.478+0000 7fe583fff640 1 -- 172.21.15.125:0/4202649861 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5740035f0 con 0x7fe58c0b0530 2023-10-18T10:37:19.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.478+0000 7fe583fff640 1 -- 172.21.15.125:0/4202649861 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe574003750 con 0x7fe58c0b0530 2023-10-18T10:37:19.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.478+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c14d600 con 0x7fe58c0b0530 2023-10-18T10:37:19.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.478+0000 7fe583fff640 1 -- 172.21.15.125:0/4202649861 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574003ba0 con 0x7fe58c0b0530 2023-10-18T10:37:19.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.478+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c130da0 con 0x7fe58c0b0530 2023-10-18T10:37:19.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.479+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58c14d820 con 0x7fe58c0b0530 2023-10-18T10:37:19.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.479+0000 7fe583fff640 1 -- 172.21.15.125:0/4202649861 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fe574003d00 con 0x7fe58c0b0530 2023-10-18T10:37:19.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.479+0000 7fe583fff640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe56c06b070 0x7fe56c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:19.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.480+0000 7fe583fff640 1 -- 172.21.15.125:0/4202649861 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fe574088f90 con 0x7fe58c0b0530 2023-10-18T10:37:19.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.480+0000 7fe58affd640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe56c06b070 0x7fe56c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:19.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.480+0000 7fe58affd640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe56c06b070 0x7fe56c06d530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe57c035000 tx=0x7fe57c004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:19.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.484+0000 7fe583fff640 1 -- 172.21.15.125:0/4202649861 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5740bbca0 con 0x7fe58c0b0530 2023-10-18T10:37:19.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.673+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe58c0cc100 con 0x7fe56c06b070 2023-10-18T10:37:19.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.737+0000 7fe583fff640 1 -- 172.21.15.125:0/4202649861 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe57c004390 con 0x7fe56c06b070 2023-10-18T10:37:19.739 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-10-18T10:37:19.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe56c06b070 msgr2=0x7fe56c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:19.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe56c06b070 0x7fe56c06d530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe57c035000 tx=0x7fe57c004730 comp rx=0 tx=0).stop 2023-10-18T10:37:19.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0b0530 msgr2=0x7fe58c143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:19.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0b0530 0x7fe58c143d40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c410 tx=0x7fe57400c8d0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 shutdown_connections 2023-10-18T10:37:19.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe56c06b070 0x7fe56c06d530 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe58c0cbd20 0x7fe58c130860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe58c0b0530 0x7fe58c143d40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 --2- 172.21.15.125:0/4202649861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe58c0afa20 0x7fe58c14b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 >> 172.21.15.125:0/4202649861 conn(0x7fe58c069930 msgr2=0x7fe58c14a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:19.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 shutdown_connections 2023-10-18T10:37:19.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.739+0000 7fe5915a4640 1 -- 172.21.15.125:0/4202649861 wait complete. 2023-10-18T10:37:19.751 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-10-18T10:37:19.751 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1m 2023-10-18T10:37:19.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.855+0000 7f9ee4621640 1 -- 172.21.15.125:0/1598599268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 msgr2=0x7f9edc154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:19.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.855+0000 7f9ee4621640 1 --2- 172.21.15.125:0/1598599268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 0x7f9edc154af0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0ca460 tx=0x7f9ed402bd70 comp rx=0 tx=0).stop 2023-10-18T10:37:19.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.855+0000 7f9ee4621640 1 -- 172.21.15.125:0/1598599268 shutdown_connections 2023-10-18T10:37:19.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.855+0000 7f9ee4621640 1 --2- 172.21.15.125:0/1598599268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 0x7f9edc154af0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.855+0000 7f9ee4621640 1 --2- 172.21.15.125:0/1598599268 >> v2:172.21.15.139:3300/0 conn(0x7f9edc0b1920 0x7f9edc06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.855+0000 7f9ee4621640 1 -- 172.21.15.125:0/1598599268 >> 172.21.15.125:0/1598599268 conn(0x7f9edc069930 msgr2=0x7f9edc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:19.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.855+0000 7f9ee4621640 1 -- 172.21.15.125:0/1598599268 shutdown_connections 2023-10-18T10:37:19.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.855+0000 7f9ee4621640 1 -- 172.21.15.125:0/1598599268 wait complete. 2023-10-18T10:37:19.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.856+0000 7f9ee4621640 1 Processor -- start 2023-10-18T10:37:19.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.856+0000 7f9ee4621640 1 -- start start 2023-10-18T10:37:19.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee4621640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9edc06ffb0 0x7f9edc145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:19.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee4621640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9edc0b1920 0x7f9edc155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:19.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee4621640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 0x7f9edc158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:19.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee4621640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc155c90 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee4621640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc152d60 con 0x7f9edc06ffb0 2023-10-18T10:37:19.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee4621640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc1585b0 con 0x7f9edc0b1920 2023-10-18T10:37:19.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee2bbe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 0x7f9edc158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:19.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee2bbe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 0x7f9edc158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59424/0 (socket says 172.21.15.125:59424) 2023-10-18T10:37:19.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee2bbe640 1 -- 172.21.15.125:0/3583392189 learned_addr learned my addr 172.21.15.125:0/3583392189 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:19.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee2bbe640 1 -- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9edc0b1920 msgr2=0x7f9edc155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:19.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee2bbe640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9edc0b1920 0x7f9edc155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee2bbe640 1 -- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9edc06ffb0 msgr2=0x7f9edc145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:19.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee23bd640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9edc06ffb0 0x7f9edc145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:19.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee2bbe640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9edc06ffb0 0x7f9edc145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:19.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee2bbe640 1 -- 172.21.15.125:0/3583392189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4009490 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.857+0000 7f9ee23bd640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9edc06ffb0 0x7f9edc145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:19.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.858+0000 7f9ee2bbe640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 0x7f9edc158070 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009920 tx=0x7f9ed40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:19.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.858+0000 7f9ed2ffd640 1 -- 172.21.15.125:0/3583392189 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4003ea0 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.858+0000 7f9ed2ffd640 1 -- 172.21.15.125:0/3583392189 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ed4005020 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.858+0000 7f9ed2ffd640 1 -- 172.21.15.125:0/3583392189 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4005430 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.858+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc0ac7f0 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.858+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc0acdb0 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.859+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc0690e0 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.859+0000 7f9ed2ffd640 1 -- 172.21.15.125:0/3583392189 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f9ed40047a0 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.859+0000 7f9ed2ffd640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ec006b070 0x7f9ec006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:19.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.860+0000 7f9ed2ffd640 1 -- 172.21.15.125:0/3583392189 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f9ed40ac110 con 0x7f9edc0c9dc0 2023-10-18T10:37:19.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.860+0000 7f9ee23bd640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ec006b070 0x7f9ec006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:19.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.860+0000 7f9ee23bd640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ec006b070 0x7f9ec006d530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8007eb0 tx=0x7f9ed80073c0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:19.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:19.864+0000 7f9ed2ffd640 1 -- 172.21.15.125:0/3583392189 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ed4076c50 con 0x7f9edc0c9dc0 2023-10-18T10:37:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.049+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9edc155030 con 0x7f9ec006b070 2023-10-18T10:37:20.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.109+0000 7f9ed2ffd640 1 -- 172.21.15.125:0/3583392189 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9ed800cdd0 con 0x7f9ec006b070 2023-10-18T10:37:20.111 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-10-18T10:37:20.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ec006b070 msgr2=0x7f9ec006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ec006b070 0x7f9ec006d530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8007eb0 tx=0x7f9ed80073c0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 msgr2=0x7f9edc158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 0x7f9edc158070 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009920 tx=0x7f9ed40038a0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 shutdown_connections 2023-10-18T10:37:20.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9ec006b070 0x7f9ec006d530 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9edc0c9dc0 0x7f9edc158070 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9edc0b1920 0x7f9edc155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 --2- 172.21.15.125:0/3583392189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9edc06ffb0 0x7f9edc145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 >> 172.21.15.125:0/3583392189 conn(0x7f9edc069930 msgr2=0x7f9edc1533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:20.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 shutdown_connections 2023-10-18T10:37:20.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.111+0000 7f9ee4621640 1 -- 172.21.15.125:0/3583392189 wait complete. 2023-10-18T10:37:20.122 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-10-18T10:37:20.123 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1m 2023-10-18T10:37:20.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.227+0000 7f3b0107c640 1 -- 172.21.15.125:0/3106538533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 msgr2=0x7f3ae4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.227+0000 7f3b0107c640 1 --2- 172.21.15.125:0/3106538533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 0x7f3ae4005180 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c260 tx=0x7f3ae40057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.227+0000 7f3afb59e640 1 -- 172.21.15.125:0/3106538533 reap_dead start 2023-10-18T10:37:20.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.228+0000 7f3b0107c640 1 -- 172.21.15.125:0/3106538533 shutdown_connections 2023-10-18T10:37:20.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.228+0000 7f3b0107c640 1 -- 172.21.15.125:0/3106538533 >> 172.21.15.125:0/3106538533 conn(0x7f3afc0af7a0 msgr2=0x7f3afc0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:20.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.228+0000 7f3b0107c640 1 -- 172.21.15.125:0/3106538533 shutdown_connections 2023-10-18T10:37:20.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.228+0000 7f3b0107c640 1 -- 172.21.15.125:0/3106538533 wait complete. 2023-10-18T10:37:20.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.229+0000 7f3b0107c640 1 Processor -- start 2023-10-18T10:37:20.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.229+0000 7f3b0107c640 1 -- start start 2023-10-18T10:37:20.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3b0107c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 0x7f3afc14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3b0107c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3afc1f2a50 0x7f3afc14b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3b0107c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3afc150190 0x7f3afc0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3b0107c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1ef410 con 0x7f3ae4004d00 2023-10-18T10:37:20.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3b0107c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1f2fc0 con 0x7f3afc1f2a50 2023-10-18T10:37:20.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3b0107c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1f32a0 con 0x7f3afc150190 2023-10-18T10:37:20.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 0x7f3afc14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 0x7f3afc14d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59442/0 (socket says 172.21.15.125:59442) 2023-10-18T10:37:20.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 -- 172.21.15.125:0/320735973 learned_addr learned my addr 172.21.15.125:0/320735973 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:20.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afa59c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3afc1f2a50 0x7f3afc14b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 -- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3afc150190 msgr2=0x7f3afc0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3afc150190 0x7f3afc0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 -- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3afc1f2a50 msgr2=0x7f3afc14b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3afc1f2a50 0x7f3afc14b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 -- 172.21.15.125:0/320735973 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc1f3580 con 0x7f3ae4004d00 2023-10-18T10:37:20.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3afad9d640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 0x7f3afc14d930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3aec011d60 tx=0x7f3aec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:20.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3ae37fe640 1 -- 172.21.15.125:0/320735973 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00fee0 con 0x7f3ae4004d00 2023-10-18T10:37:20.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3ae37fe640 1 -- 172.21.15.125:0/320735973 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3aec003d80 con 0x7f3ae4004d00 2023-10-18T10:37:20.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3ae37fe640 1 -- 172.21.15.125:0/320735973 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004d60 con 0x7f3ae4004d00 2023-10-18T10:37:20.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc1f3860 con 0x7f3ae4004d00 2023-10-18T10:37:20.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.230+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc14ba20 con 0x7f3ae4004d00 2023-10-18T10:37:20.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.231+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc0690e0 con 0x7f3ae4004d00 2023-10-18T10:37:20.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.232+0000 7f3ae37fe640 1 -- 172.21.15.125:0/320735973 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f3aec004050 con 0x7f3ae4004d00 2023-10-18T10:37:20.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.232+0000 7f3ae37fe640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3ae806b070 0x7f3ae806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.232+0000 7f3ae37fe640 1 -- 172.21.15.125:0/320735973 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f3aec08ad90 con 0x7f3ae4004d00 2023-10-18T10:37:20.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.232+0000 7f3afa59c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3ae806b070 0x7f3ae806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.233+0000 7f3afa59c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3ae806b070 0x7f3ae806d530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3afc15b3a0 tx=0x7f3af0005f90 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:20.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.236+0000 7f3ae37fe640 1 -- 172.21.15.125:0/320735973 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3aec055850 con 0x7f3ae4004d00 2023-10-18T10:37:20.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.425+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3afc1f3410 con 0x7f3ae806b070 2023-10-18T10:37:20.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.483+0000 7f3ae37fe640 1 -- 172.21.15.125:0/320735973 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f3afc1f3410 con 0x7f3ae806b070 2023-10-18T10:37:20.485 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-10-18T10:37:20.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3ae806b070 msgr2=0x7f3ae806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3ae806b070 0x7f3ae806d530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3afc15b3a0 tx=0x7f3af0005f90 comp rx=0 tx=0).stop 2023-10-18T10:37:20.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 msgr2=0x7f3afc14d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 0x7f3afc14d930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3aec011d60 tx=0x7f3aec00f790 comp rx=0 tx=0).stop 2023-10-18T10:37:20.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 shutdown_connections 2023-10-18T10:37:20.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f3ae806b070 0x7f3ae806d530 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3afc1f2a50 0x7f3afc14b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3afc150190 0x7f3afc0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 --2- 172.21.15.125:0/320735973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ae4004d00 0x7f3afc14d930 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 >> 172.21.15.125:0/320735973 conn(0x7f3afc0af7a0 msgr2=0x7f3afc14f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:20.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 shutdown_connections 2023-10-18T10:37:20.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.485+0000 7f3b0107c640 1 -- 172.21.15.125:0/320735973 wait complete. 2023-10-18T10:37:20.497 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-10-18T10:37:20.497 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1m 2023-10-18T10:37:20.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.600+0000 7fdae4f97640 1 -- 172.21.15.125:0/3820811514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00cf340 msgr2=0x7fdae00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.600+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3820811514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00cf340 0x7fdae00d1910 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fdad0005fd0 tx=0x7fdad002c050 comp rx=0 tx=0).stop 2023-10-18T10:37:20.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.600+0000 7fdae4f97640 1 -- 172.21.15.125:0/3820811514 shutdown_connections 2023-10-18T10:37:20.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.600+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3820811514 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae015b2a0 0x7fdae00cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.600+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3820811514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00cf340 0x7fdae00d1910 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.600+0000 7fdae4f97640 1 -- 172.21.15.125:0/3820811514 >> 172.21.15.125:0/3820811514 conn(0x7fdae00d0d60 msgr2=0x7fdae00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:20.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.600+0000 7fdae4f97640 1 -- 172.21.15.125:0/3820811514 shutdown_connections 2023-10-18T10:37:20.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.600+0000 7fdae4f97640 1 -- 172.21.15.125:0/3820811514 wait complete. 2023-10-18T10:37:20.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.601+0000 7fdae4f97640 1 Processor -- start 2023-10-18T10:37:20.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdae4f97640 1 -- start start 2023-10-18T10:37:20.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdae4f97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00ceb40 0x7fdae0155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdae4f97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae00cf340 0x7fdae00b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdae4f97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae015b2a0 0x7fdae00b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdae4f97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00c7ec0 con 0x7fdae00ceb40 2023-10-18T10:37:20.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdae4f97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00b4280 con 0x7fdae00cf340 2023-10-18T10:37:20.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdae4f97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00b4560 con 0x7fdae015b2a0 2023-10-18T10:37:20.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdade59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00ceb40 0x7fdae0155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdade59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00ceb40 0x7fdae0155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59462/0 (socket says 172.21.15.125:59462) 2023-10-18T10:37:20.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.602+0000 7fdade59c640 1 -- 172.21.15.125:0/3754623640 learned_addr learned my addr 172.21.15.125:0/3754623640 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:20.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdaded9d640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae015b2a0 0x7fdae00b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdaddd9b640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae00cf340 0x7fdae00b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdade59c640 1 -- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae015b2a0 msgr2=0x7fdae00b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdade59c640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae015b2a0 0x7fdae00b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdade59c640 1 -- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae00cf340 msgr2=0x7fdae00b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdade59c640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae00cf340 0x7fdae00b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdade59c640 1 -- 172.21.15.125:0/3754623640 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad0009840 con 0x7fdae00ceb40 2023-10-18T10:37:20.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdaddd9b640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae00cf340 0x7fdae00b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:20.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdade59c640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00ceb40 0x7fdae0155f20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009cd0 tx=0x7fdad0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:20.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdaceffd640 1 -- 172.21.15.125:0/3754623640 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad003e460 con 0x7fdae00ceb40 2023-10-18T10:37:20.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdaceffd640 1 -- 172.21.15.125:0/3754623640 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdad003e6b0 con 0x7fdae00ceb40 2023-10-18T10:37:20.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae00b4840 con 0x7fdae00ceb40 2023-10-18T10:37:20.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdaceffd640 1 -- 172.21.15.125:0/3754623640 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad003ea80 con 0x7fdae00ceb40 2023-10-18T10:37:20.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.603+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae00ca980 con 0x7fdae00ceb40 2023-10-18T10:37:20.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.604+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae00caba0 con 0x7fdae00ceb40 2023-10-18T10:37:20.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.604+0000 7fdaceffd640 1 -- 172.21.15.125:0/3754623640 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fdad003ebe0 con 0x7fdae00ceb40 2023-10-18T10:37:20.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.605+0000 7fdaceffd640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdac006b040 0x7fdac006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.605+0000 7fdaceffd640 1 -- 172.21.15.125:0/3754623640 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fdad00ace10 con 0x7fdae00ceb40 2023-10-18T10:37:20.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.605+0000 7fdaddd9b640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdac006b040 0x7fdac006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.605+0000 7fdaddd9b640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdac006b040 0x7fdac006d500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdae00ca7f0 tx=0x7fdad4008040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:20.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.609+0000 7fdaceffd640 1 -- 172.21.15.125:0/3754623640 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdae00caba0 con 0x7fdae00ceb40 2023-10-18T10:37:20.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.797+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fdae00cc170 con 0x7fdac006b040 2023-10-18T10:37:20.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.855+0000 7fdaceffd640 1 -- 172.21.15.125:0/3754623640 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fdad400a460 con 0x7fdac006b040 2023-10-18T10:37:20.857 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-10-18T10:37:20.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdac006b040 msgr2=0x7fdac006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdac006b040 0x7fdac006d500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdae00ca7f0 tx=0x7fdad4008040 comp rx=0 tx=0).stop 2023-10-18T10:37:20.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00ceb40 msgr2=0x7fdae0155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00ceb40 0x7fdae0155f20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009cd0 tx=0x7fdad0004360 comp rx=0 tx=0).stop 2023-10-18T10:37:20.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 shutdown_connections 2023-10-18T10:37:20.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdac006b040 0x7fdac006d500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdae00cf340 0x7fdae00b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdae015b2a0 0x7fdae00b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 --2- 172.21.15.125:0/3754623640 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae00ceb40 0x7fdae0155f20 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 >> 172.21.15.125:0/3754623640 conn(0x7fdae00d0d60 msgr2=0x7fdae015a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:20.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 shutdown_connections 2023-10-18T10:37:20.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.857+0000 7fdae4f97640 1 -- 172.21.15.125:0/3754623640 wait complete. 2023-10-18T10:37:20.869 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-10-18T10:37:20.870 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1m 2023-10-18T10:37:20.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.973+0000 7f22046bb640 1 -- 172.21.15.125:0/2914937402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 msgr2=0x7f21fc157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.973+0000 7f22046bb640 1 --2- 172.21.15.125:0/2914937402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 0x7f21fc157dd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f21f4005fd0 tx=0x7f21f402c0e0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.973+0000 7f22046bb640 1 -- 172.21.15.125:0/2914937402 shutdown_connections 2023-10-18T10:37:20.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.973+0000 7f22046bb640 1 --2- 172.21.15.125:0/2914937402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 0x7f21fc157dd0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.973+0000 7f22046bb640 1 --2- 172.21.15.125:0/2914937402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21fc1579f0 0x7f21fc0b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.973+0000 7f22046bb640 1 -- 172.21.15.125:0/2914937402 >> 172.21.15.125:0/2914937402 conn(0x7f21fc06fae0 msgr2=0x7f21fc06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:20.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.973+0000 7f22046bb640 1 -- 172.21.15.125:0/2914937402 shutdown_connections 2023-10-18T10:37:20.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.973+0000 7f22046bb640 1 -- 172.21.15.125:0/2914937402 wait complete. 2023-10-18T10:37:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.974+0000 7f22046bb640 1 Processor -- start 2023-10-18T10:37:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.974+0000 7f22046bb640 1 -- start start 2023-10-18T10:37:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f22046bb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 0x7f21fc1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f22046bb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21fc1579f0 0x7f21fc14e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f22046bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21fc15c1f0 0x7f21fc14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f22046bb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc1572f0 con 0x7f21fc06ae40 2023-10-18T10:37:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f22046bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc14e9e0 con 0x7f21fc15c1f0 2023-10-18T10:37:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f22046bb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc156770 con 0x7f21fc1579f0 2023-10-18T10:37:20.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 0x7f21fc1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 0x7f21fc1560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59480/0 (socket says 172.21.15.125:59480) 2023-10-18T10:37:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 -- 172.21.15.125:0/3279907274 learned_addr learned my addr 172.21.15.125:0/3279907274 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2201c56640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21fc1579f0 0x7f21fc14e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 -- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21fc1579f0 msgr2=0x7f21fc14e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21fc1579f0 0x7f21fc14e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 -- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21fc15c1f0 msgr2=0x7f21fc14f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202c58640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21fc15c1f0 0x7f21fc14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21fc15c1f0 0x7f21fc14f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:20.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 -- 172.21.15.125:0/3279907274 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f4009840 con 0x7f21fc06ae40 2023-10-18T10:37:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2201c56640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21fc1579f0 0x7f21fc14e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f2202457640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 0x7f21fc1560d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f21f800d770 tx=0x7f21f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f21f2ffd640 1 -- 172.21.15.125:0/3279907274 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8009c30 con 0x7f21fc06ae40 2023-10-18T10:37:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f21f2ffd640 1 -- 172.21.15.125:0/3279907274 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f21f800de80 con 0x7f21fc06ae40 2023-10-18T10:37:20.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f21f2ffd640 1 -- 172.21.15.125:0/3279907274 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8005280 con 0x7f21fc06ae40 2023-10-18T10:37:20.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc0b2330 con 0x7f21fc06ae40 2023-10-18T10:37:20.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.975+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc156d30 con 0x7f21fc06ae40 2023-10-18T10:37:20.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.976+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc0690e0 con 0x7f21fc06ae40 2023-10-18T10:37:20.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.977+0000 7f21f2ffd640 1 -- 172.21.15.125:0/3279907274 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f21f80053e0 con 0x7f21fc06ae40 2023-10-18T10:37:20.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.977+0000 7f21f2ffd640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f21e006b040 0x7f21e006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:20.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.977+0000 7f21f2ffd640 1 -- 172.21.15.125:0/3279907274 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f21f8089f70 con 0x7f21fc06ae40 2023-10-18T10:37:20.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.978+0000 7f2201c56640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f21e006b040 0x7f21e006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:20.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.978+0000 7f2201c56640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f21e006b040 0x7f21e006d500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0ac650 tx=0x7f21ec009040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:20.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:20.982+0000 7f21f2ffd640 1 -- 172.21.15.125:0/3279907274 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21f80bcca0 con 0x7f21fc06ae40 2023-10-18T10:37:21.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.166+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f21fc0b1a50 con 0x7f21e006b040 2023-10-18T10:37:21.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.225+0000 7f21f2ffd640 1 -- 172.21.15.125:0/3279907274 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f21fc0b1a50 con 0x7f21e006b040 2023-10-18T10:37:21.228 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-10-18T10:37:21.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.227+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f21e006b040 msgr2=0x7f21e006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.227+0000 7f22046bb640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f21e006b040 0x7f21e006d500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0ac650 tx=0x7f21ec009040 comp rx=0 tx=0).stop 2023-10-18T10:37:21.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.227+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 msgr2=0x7f21fc1560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.227+0000 7f22046bb640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 0x7f21fc1560d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f21f800d770 tx=0x7f21f800dc30 comp rx=0 tx=0).stop 2023-10-18T10:37:21.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.228+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 shutdown_connections 2023-10-18T10:37:21.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.228+0000 7f22046bb640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f21e006b040 0x7f21e006d500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.228+0000 7f22046bb640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21fc15c1f0 0x7f21fc14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.228+0000 7f22046bb640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21fc1579f0 0x7f21fc14e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.228+0000 7f22046bb640 1 --2- 172.21.15.125:0/3279907274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21fc06ae40 0x7f21fc1560d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.228+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 >> 172.21.15.125:0/3279907274 conn(0x7f21fc06fae0 msgr2=0x7f21fc14ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:21.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.228+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 shutdown_connections 2023-10-18T10:37:21.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.228+0000 7f22046bb640 1 -- 172.21.15.125:0/3279907274 wait complete. 2023-10-18T10:37:21.242 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-10-18T10:37:21.242 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1m 2023-10-18T10:37:21.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.347+0000 7f79d48b4640 1 -- 172.21.15.125:0/1741206658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 msgr2=0x7f79d00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.347+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1741206658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 0x7f79d00d1910 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f79d015b680 tx=0x7f79c402bdb0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.347+0000 7f79d48b4640 1 -- 172.21.15.125:0/1741206658 shutdown_connections 2023-10-18T10:37:21.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.347+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1741206658 >> v2:172.21.15.139:3300/0 conn(0x7f79d00ceb40 0x7f79d00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.347+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1741206658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 0x7f79d00d1910 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.347+0000 7f79d48b4640 1 -- 172.21.15.125:0/1741206658 >> 172.21.15.125:0/1741206658 conn(0x7f79d00d0d60 msgr2=0x7f79d00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:21.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.347+0000 7f79d48b4640 1 -- 172.21.15.125:0/1741206658 shutdown_connections 2023-10-18T10:37:21.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.347+0000 7f79d48b4640 1 -- 172.21.15.125:0/1741206658 wait complete. 2023-10-18T10:37:21.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.348+0000 7f79d48b4640 1 Processor -- start 2023-10-18T10:37:21.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.348+0000 7f79d48b4640 1 -- start start 2023-10-18T10:37:21.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79d48b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79d00ceb40 0x7f79d0150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:21.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79d48b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79d00cf340 0x7f79d00b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:21.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79d48b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 0x7f79d00ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:21.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79d48b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d01514a0 con 0x7f79d015b2a0 2023-10-18T10:37:21.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79d48b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d00c93f0 con 0x7f79d00cf340 2023-10-18T10:37:21.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79d48b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d00c96d0 con 0x7f79d00ceb40 2023-10-18T10:37:21.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 0x7f79d00ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:21.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 0x7f79d00ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59500/0 (socket says 172.21.15.125:59500) 2023-10-18T10:37:21.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 -- 172.21.15.125:0/1610037896 learned_addr learned my addr 172.21.15.125:0/1610037896 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:21.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79cdd9b640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79d00cf340 0x7f79d00b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:21.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 -- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79d00ceb40 msgr2=0x7f79d0150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ce59c640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79d00ceb40 0x7f79d0150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:21.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79d00ceb40 0x7f79d0150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 -- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79d00cf340 msgr2=0x7f79d00b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79d00cf340 0x7f79d00b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 -- 172.21.15.125:0/1610037896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c4009560 con 0x7f79d015b2a0 2023-10-18T10:37:21.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.349+0000 7f79ced9d640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 0x7f79d00ca490 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f79c000eaa0 tx=0x7f79c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:21.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.350+0000 7f79beffd640 1 -- 172.21.15.125:0/1610037896 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000cdb0 con 0x7f79d015b2a0 2023-10-18T10:37:21.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.350+0000 7f79beffd640 1 -- 172.21.15.125:0/1610037896 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79c0004b20 con 0x7f79d015b2a0 2023-10-18T10:37:21.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.350+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d00c99b0 con 0x7f79d015b2a0 2023-10-18T10:37:21.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.350+0000 7f79beffd640 1 -- 172.21.15.125:0/1610037896 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0005020 con 0x7f79d015b2a0 2023-10-18T10:37:21.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.350+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d00c9f70 con 0x7f79d015b2a0 2023-10-18T10:37:21.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.351+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d00ca190 con 0x7f79d015b2a0 2023-10-18T10:37:21.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.351+0000 7f79beffd640 1 -- 172.21.15.125:0/1610037896 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f79c0005180 con 0x7f79d015b2a0 2023-10-18T10:37:21.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.351+0000 7f79beffd640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f79b006b070 0x7f79b006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:21.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.352+0000 7f79beffd640 1 -- 172.21.15.125:0/1610037896 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f79c008a6d0 con 0x7f79d015b2a0 2023-10-18T10:37:21.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.352+0000 7f79ce59c640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f79b006b070 0x7f79b006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:21.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.352+0000 7f79ce59c640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f79b006b070 0x7f79b006d530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f79c80015a0 tx=0x7f79c8009040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:21.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.356+0000 7f79beffd640 1 -- 172.21.15.125:0/1610037896 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79c0055190 con 0x7f79d015b2a0 2023-10-18T10:37:21.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.544+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f79d00b0d00 con 0x7f79b006b070 2023-10-18T10:37:21.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.601+0000 7f79beffd640 1 -- 172.21.15.125:0/1610037896 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f79d00b0d00 con 0x7f79b006b070 2023-10-18T10:37:21.603 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-10-18T10:37:21.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f79b006b070 msgr2=0x7f79b006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f79b006b070 0x7f79b006d530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f79c80015a0 tx=0x7f79c8009040 comp rx=0 tx=0).stop 2023-10-18T10:37:21.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 msgr2=0x7f79d00ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 0x7f79d00ca490 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f79c000eaa0 tx=0x7f79c000ef60 comp rx=0 tx=0).stop 2023-10-18T10:37:21.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 shutdown_connections 2023-10-18T10:37:21.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f79b006b070 0x7f79b006d530 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79d00cf340 0x7f79d00b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79d015b2a0 0x7f79d00ca490 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 --2- 172.21.15.125:0/1610037896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79d00ceb40 0x7f79d0150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 >> 172.21.15.125:0/1610037896 conn(0x7f79d00d0d60 msgr2=0x7f79d01588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:21.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 shutdown_connections 2023-10-18T10:37:21.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.603+0000 7f79d48b4640 1 -- 172.21.15.125:0/1610037896 wait complete. 2023-10-18T10:37:21.615 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-10-18T10:37:21.616 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1m 2023-10-18T10:37:21.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.720+0000 7fc13f890640 1 -- 172.21.15.125:0/4265112877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc13813d360 msgr2=0x7fc13815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.720+0000 7fc13f890640 1 --2- 172.21.15.125:0/4265112877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc13813d360 0x7fc13815a9f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc134005fd0 tx=0x7fc13402c7a0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.720+0000 7fc13f890640 1 -- 172.21.15.125:0/4265112877 shutdown_connections 2023-10-18T10:37:21.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.720+0000 7fc13f890640 1 --2- 172.21.15.125:0/4265112877 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380c9ed0 0x7fc13806b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.720+0000 7fc13f890640 1 --2- 172.21.15.125:0/4265112877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc13813d360 0x7fc13815a9f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.720+0000 7fc13f890640 1 -- 172.21.15.125:0/4265112877 >> 172.21.15.125:0/4265112877 conn(0x7fc138069930 msgr2=0x7fc138069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:21.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.720+0000 7fc13f890640 1 -- 172.21.15.125:0/4265112877 shutdown_connections 2023-10-18T10:37:21.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.720+0000 7fc13f890640 1 -- 172.21.15.125:0/4265112877 wait complete. 2023-10-18T10:37:21.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.721+0000 7fc13f890640 1 Processor -- start 2023-10-18T10:37:21.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.721+0000 7fc13f890640 1 -- start start 2023-10-18T10:37:21.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13f890640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380c9ed0 0x7fc138156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:21.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13f890640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc13813d360 0x7fc13813ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:21.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13f890640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc138140660 0x7fc13813f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:21.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13f890640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1380abe80 con 0x7fc138140660 2023-10-18T10:37:21.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13f890640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1380af7b0 con 0x7fc13813d360 2023-10-18T10:37:21.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13f890640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1380afa90 con 0x7fc1380c9ed0 2023-10-18T10:37:21.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc138140660 0x7fc13813f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:21.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc138140660 0x7fc13813f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59518/0 (socket says 172.21.15.125:59518) 2023-10-18T10:37:21.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 -- 172.21.15.125:0/3438870448 learned_addr learned my addr 172.21.15.125:0/3438870448 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:21.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 -- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380c9ed0 msgr2=0x7fc138156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380c9ed0 0x7fc138156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 -- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc13813d360 msgr2=0x7fc13813ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc13813d360 0x7fc13813ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 -- 172.21.15.125:0/3438870448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13400c4a0 con 0x7fc138140660 2023-10-18T10:37:21.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13de2d640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc138140660 0x7fc13813f290 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c970 tx=0x7fc13000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:21.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc12dffb640 1 -- 172.21.15.125:0/3438870448 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130007bb0 con 0x7fc138140660 2023-10-18T10:37:21.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc12dffb640 1 -- 172.21.15.125:0/3438870448 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc130007e20 con 0x7fc138140660 2023-10-18T10:37:21.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc12dffb640 1 -- 172.21.15.125:0/3438870448 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13000f6b0 con 0x7fc138140660 2023-10-18T10:37:21.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.722+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1380afd70 con 0x7fc138140660 2023-10-18T10:37:21.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.723+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1380b0330 con 0x7fc138140660 2023-10-18T10:37:21.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.724+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1380b0550 con 0x7fc138140660 2023-10-18T10:37:21.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.724+0000 7fc12dffb640 1 -- 172.21.15.125:0/3438870448 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fc13000f810 con 0x7fc138140660 2023-10-18T10:37:21.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.724+0000 7fc12dffb640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc11c06b040 0x7fc11c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:21.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.725+0000 7fc12dffb640 1 -- 172.21.15.125:0/3438870448 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fc13008a640 con 0x7fc138140660 2023-10-18T10:37:21.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.725+0000 7fc13d62c640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc11c06b040 0x7fc11c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:21.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.725+0000 7fc13d62c640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc11c06b040 0x7fc11c06d500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc134037000 tx=0x7fc134004a40 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:21.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.729+0000 7fc12dffb640 1 -- 172.21.15.125:0/3438870448 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc130054fd0 con 0x7fc138140660 2023-10-18T10:37:21.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.917+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc13806be50 con 0x7fc11c06b040 2023-10-18T10:37:21.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.979+0000 7fc12dffb640 1 -- 172.21.15.125:0/3438870448 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc134032500 con 0x7fc11c06b040 2023-10-18T10:37:21.981 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-10-18T10:37:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc11c06b040 msgr2=0x7fc11c06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc11c06b040 0x7fc11c06d500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc134037000 tx=0x7fc134004a40 comp rx=0 tx=0).stop 2023-10-18T10:37:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc138140660 msgr2=0x7fc13813f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc138140660 0x7fc13813f290 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c970 tx=0x7fc13000ce30 comp rx=0 tx=0).stop 2023-10-18T10:37:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 shutdown_connections 2023-10-18T10:37:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc11c06b040 0x7fc11c06d500 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc13813d360 0x7fc13813ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc138140660 0x7fc13813f290 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 --2- 172.21.15.125:0/3438870448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380c9ed0 0x7fc138156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:21.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 >> 172.21.15.125:0/3438870448 conn(0x7fc138069930 msgr2=0x7fc13813c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:21.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 shutdown_connections 2023-10-18T10:37:21.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:21.981+0000 7fc13f890640 1 -- 172.21.15.125:0/3438870448 wait complete. 2023-10-18T10:37:21.993 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-10-18T10:37:21.993 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1m 2023-10-18T10:37:22.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.097+0000 7fdb63ad7640 1 -- 172.21.15.125:0/2049201186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c1413a0 msgr2=0x7fdb5c15b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.097+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/2049201186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c1413a0 0x7fdb5c15b760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005fd0 tx=0x7fdb5802be10 comp rx=0 tx=0).stop 2023-10-18T10:37:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.097+0000 7fdb63ad7640 1 -- 172.21.15.125:0/2049201186 shutdown_connections 2023-10-18T10:37:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.097+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/2049201186 >> v2:172.21.15.139:3300/0 conn(0x7fdb5c1582d0 0x7fdb5c06c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.097+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/2049201186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c1413a0 0x7fdb5c15b760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.097+0000 7fdb63ad7640 1 -- 172.21.15.125:0/2049201186 >> 172.21.15.125:0/2049201186 conn(0x7fdb5c0cc970 msgr2=0x7fdb5c0ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:22.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.097+0000 7fdb63ad7640 1 -- 172.21.15.125:0/2049201186 shutdown_connections 2023-10-18T10:37:22.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.097+0000 7fdb63ad7640 1 -- 172.21.15.125:0/2049201186 wait complete. 2023-10-18T10:37:22.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.098+0000 7fdb63ad7640 1 Processor -- start 2023-10-18T10:37:22.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.098+0000 7fdb63ad7640 1 -- start start 2023-10-18T10:37:22.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb63ad7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c06cea0 0x7fdb5c0aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb63ad7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c1413a0 0x7fdb5c12eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb63ad7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c1582d0 0x7fdb5c130a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb63ad7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0af0d0 con 0x7fdb5c06cea0 2023-10-18T10:37:22.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb63ad7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c12e6b0 con 0x7fdb5c1582d0 2023-10-18T10:37:22.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb63ad7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c12daa0 con 0x7fdb5c1413a0 2023-10-18T10:37:22.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb61873640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c06cea0 0x7fdb5c0aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb61873640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c06cea0 0x7fdb5c0aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59534/0 (socket says 172.21.15.125:59534) 2023-10-18T10:37:22.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb61873640 1 -- 172.21.15.125:0/127500620 learned_addr learned my addr 172.21.15.125:0/127500620 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:22.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb61072640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c1413a0 0x7fdb5c12eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb62074640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c1582d0 0x7fdb5c130a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb61873640 1 -- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c1413a0 msgr2=0x7fdb5c12eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb61873640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c1413a0 0x7fdb5c12eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb61873640 1 -- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c1582d0 msgr2=0x7fdb5c130a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.099+0000 7fdb61873640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c1582d0 0x7fdb5c130a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.100+0000 7fdb61873640 1 -- 172.21.15.125:0/127500620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58009560 con 0x7fdb5c06cea0 2023-10-18T10:37:22.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.100+0000 7fdb61873640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c06cea0 0x7fdb5c0aeb90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdb58006010 tx=0x7fdb5802da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:22.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.100+0000 7fdb527fc640 1 -- 172.21.15.125:0/127500620 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5802de70 con 0x7fdb5c06cea0 2023-10-18T10:37:22.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.100+0000 7fdb527fc640 1 -- 172.21.15.125:0/127500620 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb580036e0 con 0x7fdb5c06cea0 2023-10-18T10:37:22.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.100+0000 7fdb527fc640 1 -- 172.21.15.125:0/127500620 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58003af0 con 0x7fdb5c06cea0 2023-10-18T10:37:22.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.100+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c12dd80 con 0x7fdb5c06cea0 2023-10-18T10:37:22.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.100+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c12e340 con 0x7fdb5c06cea0 2023-10-18T10:37:22.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.101+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c04dae0 con 0x7fdb5c06cea0 2023-10-18T10:37:22.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.101+0000 7fdb527fc640 1 -- 172.21.15.125:0/127500620 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fdb58003c50 con 0x7fdb5c06cea0 2023-10-18T10:37:22.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.102+0000 7fdb527fc640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdb4006b070 0x7fdb4006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.102+0000 7fdb527fc640 1 -- 172.21.15.125:0/127500620 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fdb580ac6c0 con 0x7fdb5c06cea0 2023-10-18T10:37:22.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.102+0000 7fdb61072640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdb4006b070 0x7fdb4006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.102+0000 7fdb61072640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdb4006b070 0x7fdb4006d530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c06c740 tx=0x7fdb4c005f90 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:22.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.106+0000 7fdb527fc640 1 -- 172.21.15.125:0/127500620 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb580773d0 con 0x7fdb5c06cea0 2023-10-18T10:37:22.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.296+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb5c1418c0 con 0x7fdb4006b070 2023-10-18T10:37:22.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.356+0000 7fdb527fc640 1 -- 172.21.15.125:0/127500620 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fdb5c1418c0 con 0x7fdb4006b070 2023-10-18T10:37:22.359 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-10-18T10:37:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdb4006b070 msgr2=0x7fdb4006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdb4006b070 0x7fdb4006d530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c06c740 tx=0x7fdb4c005f90 comp rx=0 tx=0).stop 2023-10-18T10:37:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c06cea0 msgr2=0x7fdb5c0aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c06cea0 0x7fdb5c0aeb90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdb58006010 tx=0x7fdb5802da70 comp rx=0 tx=0).stop 2023-10-18T10:37:22.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 shutdown_connections 2023-10-18T10:37:22.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fdb4006b070 0x7fdb4006d530 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c1582d0 0x7fdb5c130a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c1413a0 0x7fdb5c12eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 --2- 172.21.15.125:0/127500620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c06cea0 0x7fdb5c0aeb90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 >> 172.21.15.125:0/127500620 conn(0x7fdb5c0cc970 msgr2=0x7fdb5c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:22.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 shutdown_connections 2023-10-18T10:37:22.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.359+0000 7fdb63ad7640 1 -- 172.21.15.125:0/127500620 wait complete. 2023-10-18T10:37:22.371 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-10-18T10:37:22.371 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1m 2023-10-18T10:37:22.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.474+0000 7f0376305640 1 -- 172.21.15.125:0/9591300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700d1910 msgr2=0x7f03700b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.474+0000 7f0376305640 1 --2- 172.21.15.125:0/9591300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700d1910 0x7f03700b0650 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0368009be0 tx=0x7f036802c0c0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.474+0000 7f0376305640 1 -- 172.21.15.125:0/9591300 shutdown_connections 2023-10-18T10:37:22.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.474+0000 7f0376305640 1 --2- 172.21.15.125:0/9591300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700d1910 0x7f03700b0650 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.474+0000 7f0376305640 1 -- 172.21.15.125:0/9591300 >> 172.21.15.125:0/9591300 conn(0x7f03700d0d60 msgr2=0x7f03700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:22.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.474+0000 7f0376305640 1 -- 172.21.15.125:0/9591300 shutdown_connections 2023-10-18T10:37:22.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.474+0000 7f0376305640 1 -- 172.21.15.125:0/9591300 wait complete. 2023-10-18T10:37:22.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.476+0000 7f0376305640 1 Processor -- start 2023-10-18T10:37:22.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.476+0000 7f0376305640 1 -- start start 2023-10-18T10:37:22.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.476+0000 7f0376305640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700cf340 0x7f037009d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.476+0000 7f0376305640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03700d1910 0x7f037009b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.476+0000 7f0376305640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f037015b2a0 0x7f037009dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.476+0000 7f0376305640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037015e670 con 0x7f03700d1910 2023-10-18T10:37:22.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.476+0000 7f0376305640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037009dc20 con 0x7f037015b2a0 2023-10-18T10:37:22.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.476+0000 7f0376305640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037009bc60 con 0x7f03700cf340 2023-10-18T10:37:22.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03700d1910 0x7f037009b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03700d1910 0x7f037009b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59540/0 (socket says 172.21.15.125:59540) 2023-10-18T10:37:22.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 -- 172.21.15.125:0/3737384239 learned_addr learned my addr 172.21.15.125:0/3737384239 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:22.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f03748a2640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f037015b2a0 0x7f037009dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036ffff640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700cf340 0x7f037009d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 -- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700cf340 msgr2=0x7f037009d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700cf340 0x7f037009d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 -- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f037015b2a0 msgr2=0x7f037009dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f037015b2a0 0x7f037009dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 -- 172.21.15.125:0/3737384239 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0368009840 con 0x7f03700d1910 2023-10-18T10:37:22.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036ffff640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700cf340 0x7f037009d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:22.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036f7fe640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03700d1910 0x7f037009b720 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f035800c430 tx=0x7f035800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:22.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036cff9640 1 -- 172.21.15.125:0/3737384239 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03580052d0 con 0x7f03700d1910 2023-10-18T10:37:22.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037015e390 con 0x7f03700d1910 2023-10-18T10:37:22.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036cff9640 1 -- 172.21.15.125:0/3737384239 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03580035f0 con 0x7f03700d1910 2023-10-18T10:37:22.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f036cff9640 1 -- 172.21.15.125:0/3737384239 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358003a00 con 0x7f03700d1910 2023-10-18T10:37:22.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.477+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037015bb50 con 0x7f03700d1910 2023-10-18T10:37:22.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.478+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037015b6d0 con 0x7f03700d1910 2023-10-18T10:37:22.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.479+0000 7f036cff9640 1 -- 172.21.15.125:0/3737384239 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f0358003b60 con 0x7f03700d1910 2023-10-18T10:37:22.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.479+0000 7f036cff9640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f035006b040 0x7f035006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.479+0000 7f036cff9640 1 -- 172.21.15.125:0/3737384239 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f0358089460 con 0x7f03700d1910 2023-10-18T10:37:22.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.479+0000 7f036ffff640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f035006b040 0x7f035006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.480+0000 7f036ffff640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f035006b040 0x7f035006d500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03600011e0 tx=0x7f0360009040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:22.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.483+0000 7f036cff9640 1 -- 172.21.15.125:0/3737384239 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f037015b6d0 con 0x7f03700d1910 2023-10-18T10:37:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.669+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f037015b6b0 con 0x7f035006b040 2023-10-18T10:37:22.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.725+0000 7f036cff9640 1 -- 172.21.15.125:0/3737384239 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f037015b6b0 con 0x7f035006b040 2023-10-18T10:37:22.727 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-10-18T10:37:22.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f035006b040 msgr2=0x7f035006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f035006b040 0x7f035006d500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03600011e0 tx=0x7f0360009040 comp rx=0 tx=0).stop 2023-10-18T10:37:22.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03700d1910 msgr2=0x7f037009b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03700d1910 0x7f037009b720 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f035800c430 tx=0x7f035800c8f0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 shutdown_connections 2023-10-18T10:37:22.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f035006b040 0x7f035006d500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f037015b2a0 0x7f037009dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03700d1910 0x7f037009b720 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 --2- 172.21.15.125:0/3737384239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03700cf340 0x7f037009d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 >> 172.21.15.125:0/3737384239 conn(0x7f03700d0d60 msgr2=0x7f037006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:22.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 shutdown_connections 2023-10-18T10:37:22.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.727+0000 7f0376305640 1 -- 172.21.15.125:0/3737384239 wait complete. 2023-10-18T10:37:22.739 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-10-18T10:37:22.740 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1m 2023-10-18T10:37:22.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.844+0000 7ff2958d7640 1 -- 172.21.15.125:0/1921609341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff27c003ee0 msgr2=0x7ff27c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.844+0000 7ff2958d7640 1 --2- 172.21.15.125:0/1921609341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff27c003ee0 0x7ff27c0047c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c290 tx=0x7ff27c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.844+0000 7ff28f7fe640 1 -- 172.21.15.125:0/1921609341 reap_dead start 2023-10-18T10:37:22.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.844+0000 7ff2958d7640 1 -- 172.21.15.125:0/1921609341 shutdown_connections 2023-10-18T10:37:22.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.844+0000 7ff2958d7640 1 -- 172.21.15.125:0/1921609341 >> 172.21.15.125:0/1921609341 conn(0x7ff2900d0c50 msgr2=0x7ff2900d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:22.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.844+0000 7ff2958d7640 1 -- 172.21.15.125:0/1921609341 shutdown_connections 2023-10-18T10:37:22.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.844+0000 7ff2958d7640 1 -- 172.21.15.125:0/1921609341 wait complete. 2023-10-18T10:37:22.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.845+0000 7ff2958d7640 1 Processor -- start 2023-10-18T10:37:22.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.845+0000 7ff2958d7640 1 -- start start 2023-10-18T10:37:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff2958d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff29014fc00 0x7ff29015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff2958d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff29014ffe0 0x7ff29015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff2958d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2900ade30 0x7ff2900afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff2958d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900c83b0 con 0x7ff29014ffe0 2023-10-18T10:37:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff2958d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900c7d30 con 0x7ff29014fc00 2023-10-18T10:37:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff2958d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900ae3a0 con 0x7ff2900ade30 2023-10-18T10:37:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff29014ffe0 0x7ff29015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff29014ffe0 0x7ff29015d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59560/0 (socket says 172.21.15.125:59560) 2023-10-18T10:37:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 -- 172.21.15.125:0/3089890900 learned_addr learned my addr 172.21.15.125:0/3089890900 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 -- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2900ade30 msgr2=0x7ff2900afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2900ade30 0x7ff2900afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 -- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff29014fc00 msgr2=0x7ff29015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:22.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff29014fc00 0x7ff29015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:22.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 -- 172.21.15.125:0/3089890900 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2900c7fd0 con 0x7ff29014ffe0 2023-10-18T10:37:22.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.846+0000 7ff287fff640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff29014ffe0 0x7ff29015d800 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff27802c850 tx=0x7ff278006570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:22.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.847+0000 7ff28cff9640 1 -- 172.21.15.125:0/3089890900 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27803c040 con 0x7ff29014ffe0 2023-10-18T10:37:22.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.847+0000 7ff28cff9640 1 -- 172.21.15.125:0/3089890900 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff2780036d0 con 0x7ff29014ffe0 2023-10-18T10:37:22.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.847+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2900ae680 con 0x7ff29014ffe0 2023-10-18T10:37:22.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.847+0000 7ff28cff9640 1 -- 172.21.15.125:0/3089890900 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278003b00 con 0x7ff29014ffe0 2023-10-18T10:37:22.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.847+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29015c7f0 con 0x7ff29014ffe0 2023-10-18T10:37:22.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.847+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2900c8620 con 0x7ff29014ffe0 2023-10-18T10:37:22.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.848+0000 7ff28cff9640 1 -- 172.21.15.125:0/3089890900 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff278032070 con 0x7ff29014ffe0 2023-10-18T10:37:22.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.848+0000 7ff28cff9640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff27006b040 0x7ff27006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:22.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.848+0000 7ff28cff9640 1 -- 172.21.15.125:0/3089890900 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7ff2780ab060 con 0x7ff29014ffe0 2023-10-18T10:37:22.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.848+0000 7ff28effd640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff27006b040 0x7ff27006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:22.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.849+0000 7ff28effd640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff27006b040 0x7ff27006d500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff280006fd0 tx=0x7ff280006f20 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:22.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:22.852+0000 7ff28cff9640 1 -- 172.21.15.125:0/3089890900 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff278075e20 con 0x7ff29014ffe0 2023-10-18T10:37:23.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.040+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff29015c7f0 con 0x7ff27006b040 2023-10-18T10:37:23.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.092+0000 7ff28cff9640 1 -- 172.21.15.125:0/3089890900 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff29015c7f0 con 0x7ff27006b040 2023-10-18T10:37:23.095 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-10-18T10:37:23.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff27006b040 msgr2=0x7ff27006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff27006b040 0x7ff27006d500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff280006fd0 tx=0x7ff280006f20 comp rx=0 tx=0).stop 2023-10-18T10:37:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff29014ffe0 msgr2=0x7ff29015d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff29014ffe0 0x7ff29015d800 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff27802c850 tx=0x7ff278006570 comp rx=0 tx=0).stop 2023-10-18T10:37:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 shutdown_connections 2023-10-18T10:37:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7ff27006b040 0x7ff27006d500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2900ade30 0x7ff2900afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff29014ffe0 0x7ff29015d800 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 --2- 172.21.15.125:0/3089890900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff29014fc00 0x7ff29015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 >> 172.21.15.125:0/3089890900 conn(0x7ff2900d0c50 msgr2=0x7ff2900d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:23.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 shutdown_connections 2023-10-18T10:37:23.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.094+0000 7ff2958d7640 1 -- 172.21.15.125:0/3089890900 wait complete. 2023-10-18T10:37:23.106 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-10-18T10:37:23.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:23.107 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-18T10:37:23.127 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:23.127 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:23.127 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:23.127 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:23.128 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:23.128 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:23.128 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:23.128 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:23.128 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:23.128 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:23.129 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:23.129 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:23.129 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2023-10-18T10:37:23.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.269+0000 7f60048f2640 1 -- 172.21.15.125:0/3645985603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fec004fe0 msgr2=0x7f5fec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.269+0000 7f60048f2640 1 --2- 172.21.15.125:0/3645985603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fec004fe0 0x7f5fec00b980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800c640 tx=0x7f5fec0057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.269+0000 7f5ffed9d640 1 -- 172.21.15.125:0/3645985603 reap_dead start 2023-10-18T10:37:23.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.269+0000 7f60048f2640 1 -- 172.21.15.125:0/3645985603 shutdown_connections 2023-10-18T10:37:23.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.269+0000 7f60048f2640 1 -- 172.21.15.125:0/3645985603 >> 172.21.15.125:0/3645985603 conn(0x7f6000069930 msgr2=0x7f6000069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:23.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.269+0000 7f60048f2640 1 -- 172.21.15.125:0/3645985603 shutdown_connections 2023-10-18T10:37:23.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.269+0000 7f60048f2640 1 -- 172.21.15.125:0/3645985603 wait complete. 2023-10-18T10:37:23.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.270+0000 7f60048f2640 1 Processor -- start 2023-10-18T10:37:23.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.270+0000 7f60048f2640 1 -- start start 2023-10-18T10:37:23.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f60048f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60001f1010 0x7f60000abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:23.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f60048f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6000136160 0x7f60000af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:23.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f60048f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6000156e50 0x7f60001572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:23.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f60048f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000136f70 con 0x7f6000156e50 2023-10-18T10:37:23.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f60048f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000157810 con 0x7f6000136160 2023-10-18T10:37:23.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f60048f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6000134b10 con 0x7f60001f1010 2023-10-18T10:37:23.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6000156e50 0x7f60001572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:23.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6000156e50 0x7f60001572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59586/0 (socket says 172.21.15.125:59586) 2023-10-18T10:37:23.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffed9d640 1 -- 172.21.15.125:0/2638022444 learned_addr learned my addr 172.21.15.125:0/2638022444 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:23.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ff7fff640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6000136160 0x7f60000af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:23.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffe59c640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60001f1010 0x7f60000abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:23.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffed9d640 1 -- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60001f1010 msgr2=0x7f60000abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffed9d640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60001f1010 0x7f60000abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffed9d640 1 -- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6000136160 msgr2=0x7f60000af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffed9d640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6000136160 0x7f60000af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.271+0000 7f5ffed9d640 1 -- 172.21.15.125:0/2638022444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600013b460 con 0x7f6000156e50 2023-10-18T10:37:23.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.272+0000 7f5ffed9d640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6000156e50 0x7f60001572d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8000f80 tx=0x7f5ff80012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:23.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.272+0000 7f5ff6ffd640 1 -- 172.21.15.125:0/2638022444 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8001a70 con 0x7f6000156e50 2023-10-18T10:37:23.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.272+0000 7f5ff6ffd640 1 -- 172.21.15.125:0/2638022444 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ff8003970 con 0x7f6000156e50 2023-10-18T10:37:23.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.272+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60001369b0 con 0x7f6000156e50 2023-10-18T10:37:23.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.272+0000 7f5ff6ffd640 1 -- 172.21.15.125:0/2638022444 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8003d40 con 0x7f6000156e50 2023-10-18T10:37:23.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.272+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600013b0f0 con 0x7f6000156e50 2023-10-18T10:37:23.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.273+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600013af50 con 0x7f6000156e50 2023-10-18T10:37:23.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.273+0000 7f5ff6ffd640 1 -- 172.21.15.125:0/2638022444 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f5ff8031030 con 0x7f6000156e50 2023-10-18T10:37:23.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.274+0000 7f5ff6ffd640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5fe006b040 0x7f5fe006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:23.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.274+0000 7f5ff6ffd640 1 -- 172.21.15.125:0/2638022444 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f5ff80aa8b0 con 0x7f6000156e50 2023-10-18T10:37:23.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.274+0000 7f5ffe59c640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5fe006b040 0x7f5fe006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:23.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.274+0000 7f5ffe59c640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5fe006b040 0x7f5fe006d500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0006600 tx=0x7f5ff0006b30 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:23.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.278+0000 7f5ff6ffd640 1 -- 172.21.15.125:0/2638022444 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f600013af50 con 0x7f6000156e50 2023-10-18T10:37:23.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.464+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60000cbd60 con 0x7f5fe006b040 2023-10-18T10:37:23.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.472+0000 7f5ff6ffd640 1 -- 172.21.15.125:0/2638022444 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f5ff00062a0 con 0x7f5fe006b040 2023-10-18T10:37:23.474 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:23.474 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:34", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:23.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5fe006b040 msgr2=0x7f5fe006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5fe006b040 0x7f5fe006d500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0006600 tx=0x7f5ff0006b30 comp rx=0 tx=0).stop 2023-10-18T10:37:23.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6000156e50 msgr2=0x7f60001572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6000156e50 0x7f60001572d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8000f80 tx=0x7f5ff80012e0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 shutdown_connections 2023-10-18T10:37:23.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f5fe006b040 0x7f5fe006d500 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6000136160 0x7f60000af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6000156e50 0x7f60001572d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 --2- 172.21.15.125:0/2638022444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60001f1010 0x7f60000abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 >> 172.21.15.125:0/2638022444 conn(0x7f6000069930 msgr2=0x7f600006c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:23.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 shutdown_connections 2023-10-18T10:37:23.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.474+0000 7f60048f2640 1 -- 172.21.15.125:0/2638022444 wait complete. 2023-10-18T10:37:23.486 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:34", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:23.486 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:34', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:23.487 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:23.487 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:23.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:23.488 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-18T10:37:23.508 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:23.508 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:23.509 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:23.509 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:23.509 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:23.509 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:23.510 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:23.511 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:23.511 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:23.511 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:23.511 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:23.512 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:23.512 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2023-10-18T10:37:23.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.650+0000 7fd65ad16640 1 -- 172.21.15.125:0/3773260829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd65415b2a0 msgr2=0x7fd6540d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.650+0000 7fd65ad16640 1 --2- 172.21.15.125:0/3773260829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd65415b2a0 0x7fd6540d1910 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd644009830 tx=0x7fd64402bd60 comp rx=0 tx=0).stop 2023-10-18T10:37:23.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.650+0000 7fd65ad16640 1 -- 172.21.15.125:0/3773260829 shutdown_connections 2023-10-18T10:37:23.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.650+0000 7fd65ad16640 1 --2- 172.21.15.125:0/3773260829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd65415b2a0 0x7fd6540d1910 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.650+0000 7fd65ad16640 1 -- 172.21.15.125:0/3773260829 >> 172.21.15.125:0/3773260829 conn(0x7fd6540d0d60 msgr2=0x7fd6540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:23.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.650+0000 7fd65ad16640 1 -- 172.21.15.125:0/3773260829 shutdown_connections 2023-10-18T10:37:23.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.650+0000 7fd65ad16640 1 -- 172.21.15.125:0/3773260829 wait complete. 2023-10-18T10:37:23.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.651+0000 7fd65ad16640 1 Processor -- start 2023-10-18T10:37:23.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.651+0000 7fd65ad16640 1 -- start start 2023-10-18T10:37:23.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd65ad16640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6540ceb40 0x7fd6540c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:23.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd65ad16640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6540cf340 0x7fd6540ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:23.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd65ad16640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd65415b2a0 0x7fd65414fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:23.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd65ad16640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540c7e30 con 0x7fd6540ceb40 2023-10-18T10:37:23.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd65ad16640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540c8110 con 0x7fd6540cf340 2023-10-18T10:37:23.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd65ad16640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540c83f0 con 0x7fd65415b2a0 2023-10-18T10:37:23.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd658ab2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6540ceb40 0x7fd6540c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:23.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd658ab2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6540ceb40 0x7fd6540c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59606/0 (socket says 172.21.15.125:59606) 2023-10-18T10:37:23.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.652+0000 7fd658ab2640 1 -- 172.21.15.125:0/859040464 learned_addr learned my addr 172.21.15.125:0/859040464 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:23.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd658ab2640 1 -- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd65415b2a0 msgr2=0x7fd65414fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd658ab2640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd65415b2a0 0x7fd65414fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd658ab2640 1 -- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6540cf340 msgr2=0x7fd6540ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd658ab2640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6540cf340 0x7fd6540ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd658ab2640 1 -- 172.21.15.125:0/859040464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd644009490 con 0x7fd6540ceb40 2023-10-18T10:37:23.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd658ab2640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6540ceb40 0x7fd6540c91b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00c400 tx=0x7fd63c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:23.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd6517fa640 1 -- 172.21.15.125:0/859040464 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0052d0 con 0x7fd6540ceb40 2023-10-18T10:37:23.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd6517fa640 1 -- 172.21.15.125:0/859040464 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd63c007020 con 0x7fd6540ceb40 2023-10-18T10:37:23.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6540c86d0 con 0x7fd6540ceb40 2023-10-18T10:37:23.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd6517fa640 1 -- 172.21.15.125:0/859040464 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c007470 con 0x7fd6540ceb40 2023-10-18T10:37:23.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.653+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6540c8c90 con 0x7fd6540ceb40 2023-10-18T10:37:23.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.654+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6540d04f0 con 0x7fd6540ceb40 2023-10-18T10:37:23.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.654+0000 7fd6517fa640 1 -- 172.21.15.125:0/859040464 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd63c0067d0 con 0x7fd6540ceb40 2023-10-18T10:37:23.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.655+0000 7fd6517fa640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd63406b040 0x7fd63406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:23.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.655+0000 7fd6517fa640 1 -- 172.21.15.125:0/859040464 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fd63c089290 con 0x7fd6540ceb40 2023-10-18T10:37:23.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.655+0000 7fd653fff640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd63406b040 0x7fd63406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:23.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.655+0000 7fd653fff640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd63406b040 0x7fd63406d500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd644036000 tx=0x7fd6440046c0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:23.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.659+0000 7fd6517fa640 1 -- 172.21.15.125:0/859040464 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd63c053e60 con 0x7fd6540ceb40 2023-10-18T10:37:23.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.844+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6540690e0 con 0x7fd63406b040 2023-10-18T10:37:23.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.853+0000 7fd6517fa640 1 -- 172.21.15.125:0/859040464 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd6540690e0 con 0x7fd63406b040 2023-10-18T10:37:23.855 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:23.855 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:35", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:23.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd63406b040 msgr2=0x7fd63406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd63406b040 0x7fd63406d500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd644036000 tx=0x7fd6440046c0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6540ceb40 msgr2=0x7fd6540c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:23.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6540ceb40 0x7fd6540c91b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00c400 tx=0x7fd63c00c8c0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 shutdown_connections 2023-10-18T10:37:23.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd63406b040 0x7fd63406d500 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6540cf340 0x7fd6540ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd65415b2a0 0x7fd65414fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 --2- 172.21.15.125:0/859040464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6540ceb40 0x7fd6540c91b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:23.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 >> 172.21.15.125:0/859040464 conn(0x7fd6540d0d60 msgr2=0x7fd65415a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:23.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 shutdown_connections 2023-10-18T10:37:23.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:23.855+0000 7fd65ad16640 1 -- 172.21.15.125:0/859040464 wait complete. 2023-10-18T10:37:23.867 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:35", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:23.868 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:35', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:23.868 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:23.868 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:23.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:23.869 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-18T10:37:23.887 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:23.888 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:23.888 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:23.888 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:23.888 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:23.889 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:23.889 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:23.889 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:23.889 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:23.889 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:23.890 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:23.890 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:23.890 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2023-10-18T10:37:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.029+0000 7f1d8f10f640 1 -- 172.21.15.125:0/3250357620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 msgr2=0x7f1d880b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.029+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/3250357620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 0x7f1d880b09c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1d880cc7e0 tx=0x7f1d8002c260 comp rx=0 tx=0).stop 2023-10-18T10:37:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.030+0000 7f1d8f10f640 1 -- 172.21.15.125:0/3250357620 shutdown_connections 2023-10-18T10:37:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.030+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/3250357620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 0x7f1d880b09c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.030+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/3250357620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d880afa20 0x7f1d880705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.030+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/3250357620 >> v2:172.21.15.139:3300/0 conn(0x7f1d880cbd20 0x7f1d88158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.030+0000 7f1d8f10f640 1 -- 172.21.15.125:0/3250357620 >> 172.21.15.125:0/3250357620 conn(0x7f1d88069930 msgr2=0x7f1d88069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.030+0000 7f1d8f10f640 1 -- 172.21.15.125:0/3250357620 shutdown_connections 2023-10-18T10:37:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.030+0000 7f1d8f10f640 1 -- 172.21.15.125:0/3250357620 wait complete. 2023-10-18T10:37:24.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.031+0000 7f1d8f10f640 1 Processor -- start 2023-10-18T10:37:24.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.031+0000 7f1d8f10f640 1 -- start start 2023-10-18T10:37:24.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8f10f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d880afa20 0x7f1d8814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8f10f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 0x7f1d88150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8f10f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d880cbd20 0x7f1d8814b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8f10f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88148880 con 0x7f1d880b0530 2023-10-18T10:37:24.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8f10f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88150020 con 0x7f1d880cbd20 2023-10-18T10:37:24.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8f10f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8814f650 con 0x7f1d880afa20 2023-10-18T10:37:24.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 0x7f1d88150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 0x7f1d88150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59632/0 (socket says 172.21.15.125:59632) 2023-10-18T10:37:24.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 -- 172.21.15.125:0/2953008080 learned_addr learned my addr 172.21.15.125:0/2953008080 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:24.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8ceab640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d880afa20 0x7f1d8814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8d6ac640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d880cbd20 0x7f1d8814b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 -- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d880afa20 msgr2=0x7f1d8814fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d880afa20 0x7f1d8814fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 -- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d880cbd20 msgr2=0x7f1d8814b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d880cbd20 0x7f1d8814b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 -- 172.21.15.125:0/2953008080 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80009900 con 0x7f1d880b0530 2023-10-18T10:37:24.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8d6ac640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d880cbd20 0x7f1d8814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:24.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d87fff640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 0x7f1d88150250 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1d74005940 tx=0x7f1d7400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:24.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d857fa640 1 -- 172.21.15.125:0/2953008080 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74007b50 con 0x7f1d880b0530 2023-10-18T10:37:24.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d857fa640 1 -- 172.21.15.125:0/2953008080 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d74007da0 con 0x7f1d880b0530 2023-10-18T10:37:24.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d857fa640 1 -- 172.21.15.125:0/2953008080 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7400f6b0 con 0x7f1d880b0530 2023-10-18T10:37:24.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d88150790 con 0x7f1d880b0530 2023-10-18T10:37:24.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.032+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8814f000 con 0x7f1d880b0530 2023-10-18T10:37:24.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.033+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d880c96b0 con 0x7f1d880b0530 2023-10-18T10:37:24.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.034+0000 7f1d857fa640 1 -- 172.21.15.125:0/2953008080 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f1d7400f810 con 0x7f1d880b0530 2023-10-18T10:37:24.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.034+0000 7f1d857fa640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1d6c06b040 0x7f1d6c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.034+0000 7f1d857fa640 1 -- 172.21.15.125:0/2953008080 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f1d7408a5c0 con 0x7f1d880b0530 2023-10-18T10:37:24.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.034+0000 7f1d8ceab640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1d6c06b040 0x7f1d6c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.035+0000 7f1d8ceab640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1d6c06b040 0x7f1d6c06d500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005d70 tx=0x7f1d7c005d00 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:24.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.038+0000 7f1d857fa640 1 -- 172.21.15.125:0/2953008080 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d74055000 con 0x7f1d880b0530 2023-10-18T10:37:24.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.222+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d880690e0 con 0x7f1d6c06b040 2023-10-18T10:37:24.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.230+0000 7f1d857fa640 1 -- 172.21.15.125:0/2953008080 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f1d880690e0 con 0x7f1d6c06b040 2023-10-18T10:37:24.232 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:24.232 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:35", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:24.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1d6c06b040 msgr2=0x7f1d6c06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1d6c06b040 0x7f1d6c06d500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005d70 tx=0x7f1d7c005d00 comp rx=0 tx=0).stop 2023-10-18T10:37:24.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 msgr2=0x7f1d88150250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 0x7f1d88150250 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1d74005940 tx=0x7f1d7400cd60 comp rx=0 tx=0).stop 2023-10-18T10:37:24.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 shutdown_connections 2023-10-18T10:37:24.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1d6c06b040 0x7f1d6c06d500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d880cbd20 0x7f1d8814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d880b0530 0x7f1d88150250 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 --2- 172.21.15.125:0/2953008080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d880afa20 0x7f1d8814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.232+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 >> 172.21.15.125:0/2953008080 conn(0x7f1d88069930 msgr2=0x7f1d8806bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:24.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.233+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 shutdown_connections 2023-10-18T10:37:24.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.233+0000 7f1d8f10f640 1 -- 172.21.15.125:0/2953008080 wait complete. 2023-10-18T10:37:24.244 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:35", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:24.245 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:35', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:24.245 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:24.245 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:24.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:24.246 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-18T10:37:24.265 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:24.265 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:24.265 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:24.265 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:24.265 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:24.266 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:24.266 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:24.266 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:24.266 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:24.266 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:24.267 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:24.267 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:24.267 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2023-10-18T10:37:24.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.409+0000 7f1befa44640 1 -- 172.21.15.125:0/1296627626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 msgr2=0x7f1bd40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.409+0000 7f1befa44640 1 --2- 172.21.15.125:0/1296627626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 0x7f1bd40047c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800bb20 tx=0x7f1bd40057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.409+0000 7f1bedfe1640 1 -- 172.21.15.125:0/1296627626 reap_dead start 2023-10-18T10:37:24.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.409+0000 7f1befa44640 1 -- 172.21.15.125:0/1296627626 shutdown_connections 2023-10-18T10:37:24.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.409+0000 7f1befa44640 1 -- 172.21.15.125:0/1296627626 >> 172.21.15.125:0/1296627626 conn(0x7f1be806fae0 msgr2=0x7f1be806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:24.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.409+0000 7f1befa44640 1 -- 172.21.15.125:0/1296627626 shutdown_connections 2023-10-18T10:37:24.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.409+0000 7f1befa44640 1 -- 172.21.15.125:0/1296627626 wait complete. 2023-10-18T10:37:24.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.410+0000 7f1befa44640 1 Processor -- start 2023-10-18T10:37:24.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.410+0000 7f1befa44640 1 -- start start 2023-10-18T10:37:24.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1befa44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 0x7f1be8141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1befa44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be80acd80 0x7f1be813de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1befa44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be8159120 0x7f1be81595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1befa44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80addc0 con 0x7f1bd4003fa0 2023-10-18T10:37:24.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1befa44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8159ae0 con 0x7f1be80acd80 2023-10-18T10:37:24.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1befa44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be813e9d0 con 0x7f1be8159120 2023-10-18T10:37:24.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1bed7e0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 0x7f1be8141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1bed7e0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 0x7f1be8141340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59660/0 (socket says 172.21.15.125:59660) 2023-10-18T10:37:24.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1bed7e0640 1 -- 172.21.15.125:0/3863435836 learned_addr learned my addr 172.21.15.125:0/3863435836 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:24.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1becfdf640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be80acd80 0x7f1be813de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.411+0000 7f1bedfe1640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be8159120 0x7f1be81595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bed7e0640 1 -- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be8159120 msgr2=0x7f1be81595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bed7e0640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be8159120 0x7f1be81595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bed7e0640 1 -- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be80acd80 msgr2=0x7f1be813de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bed7e0640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be80acd80 0x7f1be813de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bed7e0640 1 -- 172.21.15.125:0/3863435836 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be80b2330 con 0x7f1bd4003fa0 2023-10-18T10:37:24.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bedfe1640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be8159120 0x7f1be81595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:24.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1becfdf640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be80acd80 0x7f1be813de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:24.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bed7e0640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 0x7f1be8141340 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1be4001d10 tx=0x7f1be4003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:24.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bddffb640 1 -- 172.21.15.125:0/3863435836 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4006e50 con 0x7f1bd4003fa0 2023-10-18T10:37:24.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bddffb640 1 -- 172.21.15.125:0/3863435836 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1be4011b60 con 0x7f1bd4003fa0 2023-10-18T10:37:24.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be80ad520 con 0x7f1bd4003fa0 2023-10-18T10:37:24.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1bddffb640 1 -- 172.21.15.125:0/3863435836 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4013450 con 0x7f1bd4003fa0 2023-10-18T10:37:24.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.412+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be80adae0 con 0x7f1bd4003fa0 2023-10-18T10:37:24.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.413+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be813e7a0 con 0x7f1bd4003fa0 2023-10-18T10:37:24.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.413+0000 7f1bddffb640 1 -- 172.21.15.125:0/3863435836 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f1be40135b0 con 0x7f1bd4003fa0 2023-10-18T10:37:24.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.414+0000 7f1bddffb640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1bcc06b040 0x7f1bcc06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.414+0000 7f1bddffb640 1 -- 172.21.15.125:0/3863435836 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f1be408d460 con 0x7f1bd4003fa0 2023-10-18T10:37:24.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.414+0000 7f1becfdf640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1bcc06b040 0x7f1bcc06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.415+0000 7f1becfdf640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1bcc06b040 0x7f1bcc06d500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8037000 tx=0x7f1bd802e040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:24.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.418+0000 7f1bddffb640 1 -- 172.21.15.125:0/3863435836 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1be813e7a0 con 0x7f1bd4003fa0 2023-10-18T10:37:24.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.604+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1be80b1dd0 con 0x7f1bcc06b040 2023-10-18T10:37:24.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.613+0000 7f1bddffb640 1 -- 172.21.15.125:0/3863435836 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f1bd802ce10 con 0x7f1bcc06b040 2023-10-18T10:37:24.615 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:24.615 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:36", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:24.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1bcc06b040 msgr2=0x7f1bcc06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1bcc06b040 0x7f1bcc06d500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8037000 tx=0x7f1bd802e040 comp rx=0 tx=0).stop 2023-10-18T10:37:24.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 msgr2=0x7f1be8141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 0x7f1be8141340 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1be4001d10 tx=0x7f1be4003d30 comp rx=0 tx=0).stop 2023-10-18T10:37:24.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 shutdown_connections 2023-10-18T10:37:24.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f1bcc06b040 0x7f1bcc06d500 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be80acd80 0x7f1be813de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1be8159120 0x7f1be81595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 --2- 172.21.15.125:0/3863435836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bd4003fa0 0x7f1be8141340 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 >> 172.21.15.125:0/3863435836 conn(0x7f1be806fae0 msgr2=0x7f1be814fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:24.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 shutdown_connections 2023-10-18T10:37:24.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.615+0000 7f1befa44640 1 -- 172.21.15.125:0/3863435836 wait complete. 2023-10-18T10:37:24.627 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:36", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:24.627 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:36', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:24.627 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:24.628 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:24.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:24.628 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-18T10:37:24.648 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:24.648 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:24.648 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:24.649 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:24.649 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:24.649 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:24.649 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:24.650 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:24.650 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:24.650 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:24.650 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:24.650 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:24.651 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2023-10-18T10:37:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.793+0000 7f884ba32640 1 -- 172.21.15.125:0/3733135559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88440cf340 msgr2=0x7f88440d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.793+0000 7f884ba32640 1 --2- 172.21.15.125:0/3733135559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88440cf340 0x7f88440d1910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8840009830 tx=0x7f884002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.793+0000 7f884ba32640 1 -- 172.21.15.125:0/3733135559 shutdown_connections 2023-10-18T10:37:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.793+0000 7f884ba32640 1 --2- 172.21.15.125:0/3733135559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88440ceb40 0x7f88440b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.793+0000 7f884ba32640 1 --2- 172.21.15.125:0/3733135559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88440cf340 0x7f88440d1910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.793+0000 7f884ba32640 1 -- 172.21.15.125:0/3733135559 >> 172.21.15.125:0/3733135559 conn(0x7f88440d0d60 msgr2=0x7f88440d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.793+0000 7f884ba32640 1 -- 172.21.15.125:0/3733135559 shutdown_connections 2023-10-18T10:37:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.793+0000 7f884ba32640 1 -- 172.21.15.125:0/3733135559 wait complete. 2023-10-18T10:37:24.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.794+0000 7f884ba32640 1 Processor -- start 2023-10-18T10:37:24.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.794+0000 7f884ba32640 1 -- start start 2023-10-18T10:37:24.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f884ba32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88440ceb40 0x7f8844158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f884ba32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88440cf340 0x7f88440c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f884ba32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f884415b2a0 0x7f88440cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f884ba32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884415cc70 con 0x7f88440cf340 2023-10-18T10:37:24.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f884ba32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884415cf50 con 0x7f884415b2a0 2023-10-18T10:37:24.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f884ba32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884415d230 con 0x7f88440ceb40 2023-10-18T10:37:24.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88440cf340 0x7f88440c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88440cf340 0x7f88440c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59664/0 (socket says 172.21.15.125:59664) 2023-10-18T10:37:24.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 -- 172.21.15.125:0/3364448082 learned_addr learned my addr 172.21.15.125:0/3364448082 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:24.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f88497ce640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88440ceb40 0x7f8844158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8849fcf640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f884415b2a0 0x7f88440cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 -- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88440ceb40 msgr2=0x7f8844158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88440ceb40 0x7f8844158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 -- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f884415b2a0 msgr2=0x7f88440cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:24.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f884415b2a0 0x7f88440cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:24.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 -- 172.21.15.125:0/3364448082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c009900 con 0x7f88440cf340 2023-10-18T10:37:24.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8848fcd640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88440cf340 0x7f88440c9ea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f883400c3c0 tx=0x7f883400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:24.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8839ffb640 1 -- 172.21.15.125:0/3364448082 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88340052d0 con 0x7f88440cf340 2023-10-18T10:37:24.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8839ffb640 1 -- 172.21.15.125:0/3364448082 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8834007020 con 0x7f88440cf340 2023-10-18T10:37:24.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.795+0000 7f8839ffb640 1 -- 172.21.15.125:0/3364448082 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834007470 con 0x7f88440cf340 2023-10-18T10:37:24.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.796+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8840009490 con 0x7f88440cf340 2023-10-18T10:37:24.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.796+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88441f1d70 con 0x7f88440cf340 2023-10-18T10:37:24.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.797+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88440690e0 con 0x7f88440cf340 2023-10-18T10:37:24.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.797+0000 7f8839ffb640 1 -- 172.21.15.125:0/3364448082 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f88340067d0 con 0x7f88440cf340 2023-10-18T10:37:24.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.797+0000 7f8839ffb640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f882806b040 0x7f882806d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:24.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.797+0000 7f8839ffb640 1 -- 172.21.15.125:0/3364448082 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f88340891c0 con 0x7f88440cf340 2023-10-18T10:37:24.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.798+0000 7f88497ce640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f882806b040 0x7f882806d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:24.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.798+0000 7f88497ce640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f882806b040 0x7f882806d500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8840036000 tx=0x7f8840004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:24.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.802+0000 7f8839ffb640 1 -- 172.21.15.125:0/3364448082 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8834053d90 con 0x7f88440cf340 2023-10-18T10:37:24.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.988+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f884415d0c0 con 0x7f882806b040 2023-10-18T10:37:24.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.996+0000 7f8839ffb640 1 -- 172.21.15.125:0/3364448082 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f884415d0c0 con 0x7f882806b040 2023-10-18T10:37:24.998 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:24.999 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:36", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:25.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f882806b040 msgr2=0x7f882806d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f882806b040 0x7f882806d500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8840036000 tx=0x7f8840004730 comp rx=0 tx=0).stop 2023-10-18T10:37:25.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88440cf340 msgr2=0x7f88440c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88440cf340 0x7f88440c9ea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f883400c3c0 tx=0x7f883400c880 comp rx=0 tx=0).stop 2023-10-18T10:37:25.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 shutdown_connections 2023-10-18T10:37:25.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f882806b040 0x7f882806d500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f884415b2a0 0x7f88440cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88440cf340 0x7f88440c9ea0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 --2- 172.21.15.125:0/3364448082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88440ceb40 0x7f8844158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 >> 172.21.15.125:0/3364448082 conn(0x7f88440d0d60 msgr2=0x7f884406b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:25.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 shutdown_connections 2023-10-18T10:37:25.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:24.999+0000 7f884ba32640 1 -- 172.21.15.125:0/3364448082 wait complete. 2023-10-18T10:37:25.011 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:36", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:25.011 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:36', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:25.011 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:25.012 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:25.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:25.012 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-18T10:37:25.031 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:25.032 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:25.032 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:25.032 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:25.032 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:25.033 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:25.033 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:25.033 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:25.033 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:25.033 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:25.034 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:25.034 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:25.034 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2023-10-18T10:37:25.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.173+0000 7f4277e90640 1 -- 172.21.15.125:0/3576881974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700afa20 msgr2=0x7f4270158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.173+0000 7f4277e90640 1 --2- 172.21.15.125:0/3576881974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700afa20 0x7f4270158300 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f42700b10f0 tx=0x7f426002be40 comp rx=0 tx=0).stop 2023-10-18T10:37:25.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.173+0000 7f4277e90640 1 -- 172.21.15.125:0/3576881974 shutdown_connections 2023-10-18T10:37:25.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.173+0000 7f4277e90640 1 --2- 172.21.15.125:0/3576881974 >> v2:172.21.15.139:3300/0 conn(0x7f42700b0530 0x7f42700b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.173+0000 7f4277e90640 1 --2- 172.21.15.125:0/3576881974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700afa20 0x7f4270158300 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.173+0000 7f4277e90640 1 -- 172.21.15.125:0/3576881974 >> 172.21.15.125:0/3576881974 conn(0x7f4270069930 msgr2=0x7f4270069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:25.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.174+0000 7f4277e90640 1 -- 172.21.15.125:0/3576881974 shutdown_connections 2023-10-18T10:37:25.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.174+0000 7f4277e90640 1 -- 172.21.15.125:0/3576881974 wait complete. 2023-10-18T10:37:25.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f4277e90640 1 Processor -- start 2023-10-18T10:37:25.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f4277e90640 1 -- start start 2023-10-18T10:37:25.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f4277e90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42700afa20 0x7f4270140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f4277e90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700b0530 0x7f4270144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f4277e90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42700cbd20 0x7f42700aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f4277e90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4270143670 con 0x7f42700b0530 2023-10-18T10:37:25.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f4277e90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42701429f0 con 0x7f42700afa20 2023-10-18T10:37:25.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f4277e90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4270142cd0 con 0x7f42700cbd20 2023-10-18T10:37:25.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f427542b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700b0530 0x7f4270144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f427542b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700b0530 0x7f4270144890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59688/0 (socket says 172.21.15.125:59688) 2023-10-18T10:37:25.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.175+0000 7f427542b640 1 -- 172.21.15.125:0/228101248 learned_addr learned my addr 172.21.15.125:0/228101248 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:25.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f427642d640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42700cbd20 0x7f42700aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f4275c2c640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42700afa20 0x7f4270140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f427542b640 1 -- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42700cbd20 msgr2=0x7f42700aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f427542b640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42700cbd20 0x7f42700aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f427542b640 1 -- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42700afa20 msgr2=0x7f4270140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f427542b640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42700afa20 0x7f4270140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f427542b640 1 -- 172.21.15.125:0/228101248 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260009560 con 0x7f42700b0530 2023-10-18T10:37:25.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f427542b640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700b0530 0x7f4270144890 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4260036040 tx=0x7f426002de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:25.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f42667fc640 1 -- 172.21.15.125:0/228101248 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260003800 con 0x7f42700b0530 2023-10-18T10:37:25.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f42667fc640 1 -- 172.21.15.125:0/228101248 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4260003a50 con 0x7f42700b0530 2023-10-18T10:37:25.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f42667fc640 1 -- 172.21.15.125:0/228101248 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260003de0 con 0x7f42700b0530 2023-10-18T10:37:25.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4270142fb0 con 0x7f42700b0530 2023-10-18T10:37:25.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.176+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4270141240 con 0x7f42700b0530 2023-10-18T10:37:25.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.177+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42701437b0 con 0x7f42700b0530 2023-10-18T10:37:25.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.177+0000 7f42667fc640 1 -- 172.21.15.125:0/228101248 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f426003e450 con 0x7f42700b0530 2023-10-18T10:37:25.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.178+0000 7f42667fc640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f425406b070 0x7f425406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.178+0000 7f42667fc640 1 -- 172.21.15.125:0/228101248 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f42600ac700 con 0x7f42700b0530 2023-10-18T10:37:25.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.178+0000 7f4275c2c640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f425406b070 0x7f425406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.178+0000 7f4275c2c640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f425406b070 0x7f425406d530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f426c000f60 tx=0x7f426c009040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:25.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.182+0000 7f42667fc640 1 -- 172.21.15.125:0/228101248 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42701437b0 con 0x7f42700b0530 2023-10-18T10:37:25.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.367+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42701413d0 con 0x7f425406b070 2023-10-18T10:37:25.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.376+0000 7f42667fc640 1 -- 172.21.15.125:0/228101248 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f42701413d0 con 0x7f425406b070 2023-10-18T10:37:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:25.379 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:37", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:25.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f425406b070 msgr2=0x7f425406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f425406b070 0x7f425406d530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f426c000f60 tx=0x7f426c009040 comp rx=0 tx=0).stop 2023-10-18T10:37:25.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700b0530 msgr2=0x7f4270144890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700b0530 0x7f4270144890 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4260036040 tx=0x7f426002de10 comp rx=0 tx=0).stop 2023-10-18T10:37:25.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 shutdown_connections 2023-10-18T10:37:25.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f425406b070 0x7f425406d530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42700cbd20 0x7f42700aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42700b0530 0x7f4270144890 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 --2- 172.21.15.125:0/228101248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42700afa20 0x7f4270140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 >> 172.21.15.125:0/228101248 conn(0x7f4270069930 msgr2=0x7f427014bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:25.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 shutdown_connections 2023-10-18T10:37:25.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.378+0000 7f4277e90640 1 -- 172.21.15.125:0/228101248 wait complete. 2023-10-18T10:37:25.390 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:37", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:25.391 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:37', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:25.391 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:25.391 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:25.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:25.392 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-18T10:37:25.411 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:25.411 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:25.412 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:25.412 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:25.412 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:25.412 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:25.412 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:25.413 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:25.413 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:25.413 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:25.413 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:25.414 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:25.414 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2023-10-18T10:37:25.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 -- 172.21.15.125:0/4040356550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc00cdb30 msgr2=0x7f7cc00cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/4040356550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc00cdb30 0x7f7cc00cdfe0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4005fd0 tx=0x7f7cb402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 -- 172.21.15.125:0/4040356550 shutdown_connections 2023-10-18T10:37:25.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/4040356550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc00cdb30 0x7f7cc00cdfe0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/4040356550 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc015ccf0 0x7f7cc015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/4040356550 >> v2:172.21.15.139:3300/0 conn(0x7f7cc0159060 0x7f7cc006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 -- 172.21.15.125:0/4040356550 >> 172.21.15.125:0/4040356550 conn(0x7f7cc00b0670 msgr2=0x7f7cc00b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:25.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 -- 172.21.15.125:0/4040356550 shutdown_connections 2023-10-18T10:37:25.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.554+0000 7f7cc58e5640 1 -- 172.21.15.125:0/4040356550 wait complete. 2023-10-18T10:37:25.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.555+0000 7f7cc58e5640 1 Processor -- start 2023-10-18T10:37:25.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.555+0000 7f7cc58e5640 1 -- start start 2023-10-18T10:37:25.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cc58e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc00cdb30 0x7f7cc0159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cc58e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc0159060 0x7f7cc00af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cc58e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc015ccf0 0x7f7cc014bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cc58e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0159c80 con 0x7f7cc015ccf0 2023-10-18T10:37:25.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cc58e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc015a6b0 con 0x7f7cc00cdb30 2023-10-18T10:37:25.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cc58e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc015a0f0 con 0x7f7cc0159060 2023-10-18T10:37:25.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cbf7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc015ccf0 0x7f7cc014bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cbf7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc015ccf0 0x7f7cc014bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59714/0 (socket says 172.21.15.125:59714) 2023-10-18T10:37:25.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cbf7fe640 1 -- 172.21.15.125:0/3700418398 learned_addr learned my addr 172.21.15.125:0/3700418398 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:25.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cbf7fe640 1 -- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc0159060 msgr2=0x7f7cc00af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cbf7fe640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc0159060 0x7f7cc00af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cbf7fe640 1 -- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc00cdb30 msgr2=0x7f7cc0159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cbf7fe640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc00cdb30 0x7f7cc0159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.556+0000 7f7cbf7fe640 1 -- 172.21.15.125:0/3700418398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb4009560 con 0x7f7cc015ccf0 2023-10-18T10:37:25.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.557+0000 7f7cbf7fe640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc015ccf0 0x7f7cc014bc40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b410 tx=0x7f7cb800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:25.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.557+0000 7f7ca7fff640 1 -- 172.21.15.125:0/3700418398 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8011830 con 0x7f7cc015ccf0 2023-10-18T10:37:25.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.557+0000 7f7ca7fff640 1 -- 172.21.15.125:0/3700418398 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7cb8011aa0 con 0x7f7cc015ccf0 2023-10-18T10:37:25.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.557+0000 7f7ca7fff640 1 -- 172.21.15.125:0/3700418398 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8010380 con 0x7f7cc015ccf0 2023-10-18T10:37:25.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.557+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc015a3d0 con 0x7f7cc015ccf0 2023-10-18T10:37:25.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.557+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc00af210 con 0x7f7cc015ccf0 2023-10-18T10:37:25.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.558+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc00aed90 con 0x7f7cc015ccf0 2023-10-18T10:37:25.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.558+0000 7f7ca7fff640 1 -- 172.21.15.125:0/3700418398 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f7cb80104e0 con 0x7f7cc015ccf0 2023-10-18T10:37:25.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.559+0000 7f7ca7fff640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f7cac06b040 0x7f7cac06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.559+0000 7f7ca7fff640 1 -- 172.21.15.125:0/3700418398 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f7cb8089f40 con 0x7f7cc015ccf0 2023-10-18T10:37:25.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.559+0000 7f7cbeffd640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f7cac06b040 0x7f7cac06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.559+0000 7f7cbeffd640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f7cac06b040 0x7f7cac06d500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0005fd0 tx=0x7f7cb00071c0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:25.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.563+0000 7f7ca7fff640 1 -- 172.21.15.125:0/3700418398 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7cc00aed90 con 0x7f7cc015ccf0 2023-10-18T10:37:25.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.750+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7cc00c9e60 con 0x7f7cac06b040 2023-10-18T10:37:25.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.759+0000 7f7ca7fff640 1 -- 172.21.15.125:0/3700418398 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7cb000cea0 con 0x7f7cac06b040 2023-10-18T10:37:25.761 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:25.761 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:37", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:25.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f7cac06b040 msgr2=0x7f7cac06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f7cac06b040 0x7f7cac06d500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0005fd0 tx=0x7f7cb00071c0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc015ccf0 msgr2=0x7f7cc014bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc015ccf0 0x7f7cc014bc40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b410 tx=0x7f7cb800b8d0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 shutdown_connections 2023-10-18T10:37:25.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f7cac06b040 0x7f7cac06d500 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc015ccf0 0x7f7cc014bc40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc0159060 0x7f7cc00af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 --2- 172.21.15.125:0/3700418398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc00cdb30 0x7f7cc0159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 >> 172.21.15.125:0/3700418398 conn(0x7f7cc00b0670 msgr2=0x7f7cc00b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:25.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 shutdown_connections 2023-10-18T10:37:25.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.761+0000 7f7cc58e5640 1 -- 172.21.15.125:0/3700418398 wait complete. 2023-10-18T10:37:25.773 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:37", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:25.773 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:37', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:25.774 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:25.774 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:25.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:25.775 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-18T10:37:25.794 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:25.794 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:25.795 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:25.795 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:25.795 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:25.795 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:25.796 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:25.796 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_01_UTC 2023-10-18T10:37:25.796 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:25.796 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:25.796 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:25.797 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_01_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:25.797 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2023-10-18T10:37:25.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.935+0000 7f41baaa9640 1 -- 172.21.15.125:0/3482729846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b40c96b0 msgr2=0x7f41b40c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.935+0000 7f41baaa9640 1 --2- 172.21.15.125:0/3482729846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b40c96b0 0x7f41b40c9b30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f419c009900 tx=0x7f419c02bd70 comp rx=0 tx=0).stop 2023-10-18T10:37:25.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.935+0000 7f41baaa9640 1 -- 172.21.15.125:0/3482729846 shutdown_connections 2023-10-18T10:37:25.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.935+0000 7f41baaa9640 1 --2- 172.21.15.125:0/3482729846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b40c96b0 0x7f41b40c9b30 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.935+0000 7f41baaa9640 1 --2- 172.21.15.125:0/3482729846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41b406af00 0x7f41b406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.935+0000 7f41baaa9640 1 -- 172.21.15.125:0/3482729846 >> 172.21.15.125:0/3482729846 conn(0x7f41b4069930 msgr2=0x7f41b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:25.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.935+0000 7f41baaa9640 1 -- 172.21.15.125:0/3482729846 shutdown_connections 2023-10-18T10:37:25.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.935+0000 7f41baaa9640 1 -- 172.21.15.125:0/3482729846 wait complete. 2023-10-18T10:37:25.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.936+0000 7f41baaa9640 1 Processor -- start 2023-10-18T10:37:25.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41baaa9640 1 -- start start 2023-10-18T10:37:25.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41baaa9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b406af00 0x7f41b415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41baaa9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41b40ab500 0x7f41b40a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41baaa9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41b40c96b0 0x7f41b4150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41baaa9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b414f4e0 con 0x7f41b406af00 2023-10-18T10:37:25.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41baaa9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b40a90e0 con 0x7f41b40c96b0 2023-10-18T10:37:25.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41baaa9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b414f770 con 0x7f41b40ab500 2023-10-18T10:37:25.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41b8845640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b406af00 0x7f41b415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41b8845640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b406af00 0x7f41b415a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59732/0 (socket says 172.21.15.125:59732) 2023-10-18T10:37:25.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.937+0000 7f41b8845640 1 -- 172.21.15.125:0/2396272211 learned_addr learned my addr 172.21.15.125:0/2396272211 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:25.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b3fff640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41b40ab500 0x7f41b40a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b8845640 1 -- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41b40ab500 msgr2=0x7f41b40a8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b8845640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41b40ab500 0x7f41b40a8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b8845640 1 -- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41b40c96b0 msgr2=0x7f41b4150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:25.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b8845640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41b40c96b0 0x7f41b4150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:25.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b8845640 1 -- 172.21.15.125:0/2396272211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c009560 con 0x7f41b406af00 2023-10-18T10:37:25.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b8845640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b406af00 0x7f41b415a9c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00cce0 tx=0x7f41ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:25.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b17fa640 1 -- 172.21.15.125:0/2396272211 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac004b20 con 0x7f41b406af00 2023-10-18T10:37:25.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b17fa640 1 -- 172.21.15.125:0/2396272211 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f41ac004d70 con 0x7f41b406af00 2023-10-18T10:37:25.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b414fa50 con 0x7f41b406af00 2023-10-18T10:37:25.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41b17fa640 1 -- 172.21.15.125:0/2396272211 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac005280 con 0x7f41b406af00 2023-10-18T10:37:25.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.938+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b4150010 con 0x7f41b406af00 2023-10-18T10:37:25.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.939+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b4150200 con 0x7f41b406af00 2023-10-18T10:37:25.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.939+0000 7f41b17fa640 1 -- 172.21.15.125:0/2396272211 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f41ac0053e0 con 0x7f41b406af00 2023-10-18T10:37:25.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.940+0000 7f41b17fa640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f419406b070 0x7f419406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:25.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.940+0000 7f41b17fa640 1 -- 172.21.15.125:0/2396272211 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f41ac08a5d0 con 0x7f41b406af00 2023-10-18T10:37:25.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.940+0000 7f41b3fff640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f419406b070 0x7f419406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:25.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.940+0000 7f41b3fff640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f419406b070 0x7f419406d530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f419c035000 tx=0x7f419c004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:25.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:25.944+0000 7f41b17fa640 1 -- 172.21.15.125:0/2396272211 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41ac055090 con 0x7f41b406af00 2023-10-18T10:37:26.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.133+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41b4159c50 con 0x7f419406b070 2023-10-18T10:37:26.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.142+0000 7f41b17fa640 1 -- 172.21.15.125:0/2396272211 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f419c004260 con 0x7f419406b070 2023-10-18T10:37:26.144 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:26.144 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:37", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:26.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f419406b070 msgr2=0x7f419406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f419406b070 0x7f419406d530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f419c035000 tx=0x7f419c004730 comp rx=0 tx=0).stop 2023-10-18T10:37:26.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b406af00 msgr2=0x7f41b415a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b406af00 0x7f41b415a9c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00cce0 tx=0x7f41ac00a620 comp rx=0 tx=0).stop 2023-10-18T10:37:26.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 shutdown_connections 2023-10-18T10:37:26.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f419406b070 0x7f419406d530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41b40c96b0 0x7f41b4150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41b40ab500 0x7f41b40a8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 --2- 172.21.15.125:0/2396272211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41b406af00 0x7f41b415a9c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 >> 172.21.15.125:0/2396272211 conn(0x7f41b4069930 msgr2=0x7f41b415bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:26.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 shutdown_connections 2023-10-18T10:37:26.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.144+0000 7f41baaa9640 1 -- 172.21.15.125:0/2396272211 wait complete. 2023-10-18T10:37:26.156 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:37", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:26.156 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:37', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:26.156 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:26.157 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:26.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:26.158 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-18T10:37:26.177 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:26.177 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:26.177 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:26.177 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:26.178 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:26.178 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:26.178 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:26.178 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:26.178 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:26.178 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:26.179 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:26.179 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:26.179 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2023-10-18T10:37:26.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.320+0000 7fd8878d7640 1 -- 172.21.15.125:0/4081640227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 msgr2=0x7fd880158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.320+0000 7fd8878d7640 1 --2- 172.21.15.125:0/4081640227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 0x7fd880158310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd87c005fd0 tx=0x7fd87c02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.320+0000 7fd8878d7640 1 -- 172.21.15.125:0/4081640227 shutdown_connections 2023-10-18T10:37:26.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.320+0000 7fd8878d7640 1 --2- 172.21.15.125:0/4081640227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8800b0430 0x7fd8800b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.320+0000 7fd8878d7640 1 --2- 172.21.15.125:0/4081640227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 0x7fd880158310 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.320+0000 7fd8878d7640 1 -- 172.21.15.125:0/4081640227 >> 172.21.15.125:0/4081640227 conn(0x7fd880069930 msgr2=0x7fd880069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:26.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.320+0000 7fd8878d7640 1 -- 172.21.15.125:0/4081640227 shutdown_connections 2023-10-18T10:37:26.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.320+0000 7fd8878d7640 1 -- 172.21.15.125:0/4081640227 wait complete. 2023-10-18T10:37:26.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.321+0000 7fd8878d7640 1 Processor -- start 2023-10-18T10:37:26.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.321+0000 7fd8878d7640 1 -- start start 2023-10-18T10:37:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.321+0000 7fd8878d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd88006d020 0x7fd880147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.321+0000 7fd8878d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8800b0430 0x7fd8800acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.321+0000 7fd8878d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 0x7fd880145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.321+0000 7fd8878d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880148120 con 0x7fd8800cbc80 2023-10-18T10:37:26.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.321+0000 7fd8878d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8800ad4e0 con 0x7fd8800b0430 2023-10-18T10:37:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.321+0000 7fd8878d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8800ae320 con 0x7fd88006d020 2023-10-18T10:37:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 0x7fd880145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 0x7fd880145de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59754/0 (socket says 172.21.15.125:59754) 2023-10-18T10:37:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 -- 172.21.15.125:0/1062757484 learned_addr learned my addr 172.21.15.125:0/1062757484 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:26.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885673640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd88006d020 0x7fd880147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:26.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd884e72640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8800b0430 0x7fd8800acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:26.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 -- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd88006d020 msgr2=0x7fd880147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd88006d020 0x7fd880147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 -- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8800b0430 msgr2=0x7fd8800acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8800b0430 0x7fd8800acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 -- 172.21.15.125:0/1062757484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c009560 con 0x7fd8800cbc80 2023-10-18T10:37:26.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885673640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd88006d020 0x7fd880147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:26.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd884e72640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8800b0430 0x7fd8800acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:26.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd885e74640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 0x7fd880145de0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd87800ead0 tx=0x7fd87800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd875ffb640 1 -- 172.21.15.125:0/1062757484 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87800cde0 con 0x7fd8800cbc80 2023-10-18T10:37:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd875ffb640 1 -- 172.21.15.125:0/1062757484 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd878004c10 con 0x7fd8800cbc80 2023-10-18T10:37:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8800ad710 con 0x7fd8800cbc80 2023-10-18T10:37:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd875ffb640 1 -- 172.21.15.125:0/1062757484 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878005020 con 0x7fd8800cbc80 2023-10-18T10:37:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.322+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8800adcd0 con 0x7fd8800cbc80 2023-10-18T10:37:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.323+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8800690e0 con 0x7fd8800cbc80 2023-10-18T10:37:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.324+0000 7fd875ffb640 1 -- 172.21.15.125:0/1062757484 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd878005180 con 0x7fd8800cbc80 2023-10-18T10:37:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.324+0000 7fd875ffb640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd86406b040 0x7fd86406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.324+0000 7fd875ffb640 1 -- 172.21.15.125:0/1062757484 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fd87808a6c0 con 0x7fd8800cbc80 2023-10-18T10:37:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.324+0000 7fd885673640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd86406b040 0x7fd86406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.325+0000 7fd885673640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd86406b040 0x7fd86406d500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd87c035000 tx=0x7fd87c004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.328+0000 7fd875ffb640 1 -- 172.21.15.125:0/1062757484 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd878055210 con 0x7fd8800cbc80 2023-10-18T10:37:26.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.514+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8800c96f0 con 0x7fd86406b040 2023-10-18T10:37:26.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.522+0000 7fd875ffb640 1 -- 172.21.15.125:0/1062757484 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd87c004260 con 0x7fd86406b040 2023-10-18T10:37:26.525 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:26.525 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:38", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:26.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd86406b040 msgr2=0x7fd86406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd86406b040 0x7fd86406d500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd87c035000 tx=0x7fd87c004730 comp rx=0 tx=0).stop 2023-10-18T10:37:26.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 msgr2=0x7fd880145de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 0x7fd880145de0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd87800ead0 tx=0x7fd87800ef90 comp rx=0 tx=0).stop 2023-10-18T10:37:26.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 shutdown_connections 2023-10-18T10:37:26.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd86406b040 0x7fd86406d500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8800b0430 0x7fd8800acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8800cbc80 0x7fd880145de0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 --2- 172.21.15.125:0/1062757484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd88006d020 0x7fd880147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 >> 172.21.15.125:0/1062757484 conn(0x7fd880069930 msgr2=0x7fd88015a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:26.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 shutdown_connections 2023-10-18T10:37:26.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.524+0000 7fd8878d7640 1 -- 172.21.15.125:0/1062757484 wait complete. 2023-10-18T10:37:26.536 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:38", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:26.536 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:38', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:26.537 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:26.537 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:26.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:26.538 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-18T10:37:26.558 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:26.558 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:26.558 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:26.559 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:26.559 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:26.559 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:26.559 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:26.559 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:26.560 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:26.560 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:26.560 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:26.560 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_27_00_UTC', 'scheduled-2023-10-18-10_28_00_UTC', 'scheduled-2023-10-18-10_29_00_UTC', 'scheduled-2023-10-18-10_30_00_UTC', 'scheduled-2023-10-18-10_31_00_UTC', 'scheduled-2023-10-18-10_32_00_UTC', 'scheduled-2023-10-18-10_33_00_UTC', 'scheduled-2023-10-18-10_34_00_UTC', 'scheduled-2023-10-18-10_35_00_UTC', 'scheduled-2023-10-18-10_36_00_UTC', 'scheduled-2023-10-18-10_37_00_UTC'] 2023-10-18T10:37:26.561 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2023-10-18T10:37:26.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.700+0000 7f57c41b7640 1 -- 172.21.15.125:0/3412850435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc15b2a0 msgr2=0x7f57bc0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.700+0000 7f57c41b7640 1 --2- 172.21.15.125:0/3412850435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc15b2a0 0x7f57bc0d1910 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f57bc15b680 tx=0x7f57ac02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.700+0000 7f57c41b7640 1 -- 172.21.15.125:0/3412850435 shutdown_connections 2023-10-18T10:37:26.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.700+0000 7f57c41b7640 1 --2- 172.21.15.125:0/3412850435 >> v2:172.21.15.139:3300/0 conn(0x7f57bc0ceb40 0x7f57bc0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.700+0000 7f57c41b7640 1 --2- 172.21.15.125:0/3412850435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc15b2a0 0x7f57bc0d1910 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.700+0000 7f57c41b7640 1 -- 172.21.15.125:0/3412850435 >> 172.21.15.125:0/3412850435 conn(0x7f57bc0d0d60 msgr2=0x7f57bc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:26.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.700+0000 7f57c41b7640 1 -- 172.21.15.125:0/3412850435 shutdown_connections 2023-10-18T10:37:26.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.700+0000 7f57c41b7640 1 -- 172.21.15.125:0/3412850435 wait complete. 2023-10-18T10:37:26.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.701+0000 7f57c41b7640 1 Processor -- start 2023-10-18T10:37:26.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.701+0000 7f57c41b7640 1 -- start start 2023-10-18T10:37:26.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c41b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc0ceb40 0x7f57bc158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:26.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c41b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57bc0cf340 0x7f57bc15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:26.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c41b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57bc15b2a0 0x7f57bc15de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:26.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c41b7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc158bd0 con 0x7f57bc0ceb40 2023-10-18T10:37:26.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c41b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0cacf0 con 0x7f57bc15b2a0 2023-10-18T10:37:26.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c41b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc1f1410 con 0x7f57bc0cf340 2023-10-18T10:37:26.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c1f53640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc0ceb40 0x7f57bc158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:26.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c1f53640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc0ceb40 0x7f57bc158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59766/0 (socket says 172.21.15.125:59766) 2023-10-18T10:37:26.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c1f53640 1 -- 172.21.15.125:0/542018537 learned_addr learned my addr 172.21.15.125:0/542018537 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:26.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c1f53640 1 -- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57bc0cf340 msgr2=0x7f57bc15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c1f53640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57bc0cf340 0x7f57bc15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c1f53640 1 -- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57bc15b2a0 msgr2=0x7f57bc15de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.702+0000 7f57c1f53640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57bc15b2a0 0x7f57bc15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.703+0000 7f57c1f53640 1 -- 172.21.15.125:0/542018537 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac009490 con 0x7f57bc0ceb40 2023-10-18T10:37:26.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.703+0000 7f57c1f53640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc0ceb40 0x7f57bc158690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f57b800c500 tx=0x7f57b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:26.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.703+0000 7f57b27fc640 1 -- 172.21.15.125:0/542018537 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8012830 con 0x7f57bc0ceb40 2023-10-18T10:37:26.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.703+0000 7f57b27fc640 1 -- 172.21.15.125:0/542018537 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57b8012a80 con 0x7f57bc0ceb40 2023-10-18T10:37:26.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.703+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57bc1f16f0 con 0x7f57bc0ceb40 2023-10-18T10:37:26.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.703+0000 7f57b27fc640 1 -- 172.21.15.125:0/542018537 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8012e90 con 0x7f57bc0ceb40 2023-10-18T10:37:26.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.703+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57bc1f1cb0 con 0x7f57bc0ceb40 2023-10-18T10:37:26.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.704+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57bc1f1ed0 con 0x7f57bc0ceb40 2023-10-18T10:37:26.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.704+0000 7f57b27fc640 1 -- 172.21.15.125:0/542018537 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f57b801a3b0 con 0x7f57bc0ceb40 2023-10-18T10:37:26.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.705+0000 7f57b27fc640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f57a006b040 0x7f57a006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:26.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.705+0000 7f57b27fc640 1 -- 172.21.15.125:0/542018537 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f57b8089d70 con 0x7f57bc0ceb40 2023-10-18T10:37:26.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.705+0000 7f57c1752640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f57a006b040 0x7f57a006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:26.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.705+0000 7f57c1752640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f57a006b040 0x7f57a006d500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f57ac036000 tx=0x7f57ac0046c0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:26.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.709+0000 7f57b27fc640 1 -- 172.21.15.125:0/542018537 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57b8054950 con 0x7f57bc0ceb40 2023-10-18T10:37:26.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.892+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57bc0b0d00 con 0x7f57a006b040 2023-10-18T10:37:26.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.900+0000 7f57b27fc640 1 -- 172.21.15.125:0/542018537 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f57bc0b0d00 con 0x7f57a006b040 2023-10-18T10:37:26.902 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:26.902 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:38", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:26.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f57a006b040 msgr2=0x7f57a006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f57a006b040 0x7f57a006d500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f57ac036000 tx=0x7f57ac0046c0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc0ceb40 msgr2=0x7f57bc158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:26.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc0ceb40 0x7f57bc158690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f57b800c500 tx=0x7f57b800c9c0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 shutdown_connections 2023-10-18T10:37:26.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f57a006b040 0x7f57a006d500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57bc15b2a0 0x7f57bc15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57bc0cf340 0x7f57bc15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 --2- 172.21.15.125:0/542018537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57bc0ceb40 0x7f57bc158690 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:26.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 >> 172.21.15.125:0/542018537 conn(0x7f57bc0d0d60 msgr2=0x7f57bc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:26.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 shutdown_connections 2023-10-18T10:37:26.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:26.902+0000 7f57c41b7640 1 -- 172.21.15.125:0/542018537 wait complete. 2023-10-18T10:37:26.914 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:26:38", "first": "2023-10-18T10:27:00", "last": "2023-10-18T10:37:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-18T10:37:26.915 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:26:38', 'first': '2023-10-18T10:27:00', 'last': '2023-10-18T10:37:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-18T10:37:26.915 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T10:37:26.915 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T10:37:26.916 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1m 2023-10-18T10:37:27.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.019+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2294808633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec070920 msgr2=0x7fd6ec0affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:27.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.019+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2294808633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec070920 0x7fd6ec0affe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0cbdc0 tx=0x7fd6e402c260 comp rx=0 tx=0).stop 2023-10-18T10:37:27.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.019+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2294808633 shutdown_connections 2023-10-18T10:37:27.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.019+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2294808633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec070920 0x7fd6ec0affe0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:27.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.019+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2294808633 >> v2:172.21.15.139:3300/0 conn(0x7fd6ec141690 0x7fd6ec0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:27.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.019+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2294808633 >> 172.21.15.125:0/2294808633 conn(0x7fd6ec069930 msgr2=0x7fd6ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:27.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.019+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2294808633 shutdown_connections 2023-10-18T10:37:27.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.019+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2294808633 wait complete. 2023-10-18T10:37:27.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.020+0000 7fd6f1b73640 1 Processor -- start 2023-10-18T10:37:27.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.020+0000 7fd6f1b73640 1 -- start start 2023-10-18T10:37:27.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6f1b73640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6ec070920 0x7fd6ec1e0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:27.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6f1b73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec13e520 0x7fd6ec0a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:27.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6f1b73640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6ec141690 0x7fd6ec1e2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:27.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6f1b73640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1e0a50 con 0x7fd6ec13e520 2023-10-18T10:37:27.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6f1b73640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0a8eb0 con 0x7fd6ec141690 2023-10-18T10:37:27.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6f1b73640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1e0c80 con 0x7fd6ec070920 2023-10-18T10:37:27.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6eaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec13e520 0x7fd6ec0a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:27.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6eaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec13e520 0x7fd6ec0a8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59790/0 (socket says 172.21.15.125:59790) 2023-10-18T10:37:27.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6eaffd640 1 -- 172.21.15.125:0/2153277819 learned_addr learned my addr 172.21.15.125:0/2153277819 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:27.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6ebfff640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6ec141690 0x7fd6ec1e2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:27.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6eaffd640 1 -- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6ec070920 msgr2=0x7fd6ec1e0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:27.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.021+0000 7fd6eb7fe640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6ec070920 0x7fd6ec1e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:27.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6eaffd640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6ec070920 0x7fd6ec1e0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:27.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6eaffd640 1 -- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6ec141690 msgr2=0x7fd6ec1e2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:27.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6eaffd640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6ec141690 0x7fd6ec1e2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:27.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6eaffd640 1 -- 172.21.15.125:0/2153277819 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e4009900 con 0x7fd6ec13e520 2023-10-18T10:37:27.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6ebfff640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6ec141690 0x7fd6ec1e2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:27.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6eb7fe640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6ec070920 0x7fd6ec1e0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:27.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6eaffd640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec13e520 0x7fd6ec0a8970 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000d6d0 tx=0x7fd6e000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:27.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6d3fff640 1 -- 172.21.15.125:0/2153277819 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0005220 con 0x7fd6ec13e520 2023-10-18T10:37:27.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6d3fff640 1 -- 172.21.15.125:0/2153277819 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd6e0005490 con 0x7fd6ec13e520 2023-10-18T10:37:27.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec1e0f60 con 0x7fd6ec13e520 2023-10-18T10:37:27.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6d3fff640 1 -- 172.21.15.125:0/2153277819 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0009a80 con 0x7fd6ec13e520 2023-10-18T10:37:27.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.022+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec1e1520 con 0x7fd6ec13e520 2023-10-18T10:37:27.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.023+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec1e10f0 con 0x7fd6ec13e520 2023-10-18T10:37:27.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.023+0000 7fd6d3fff640 1 -- 172.21.15.125:0/2153277819 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd6e0009be0 con 0x7fd6ec13e520 2023-10-18T10:37:27.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.024+0000 7fd6d3fff640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd6d806b040 0x7fd6d806d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:27.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.024+0000 7fd6d3fff640 1 -- 172.21.15.125:0/2153277819 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fd6e0089fc0 con 0x7fd6ec13e520 2023-10-18T10:37:27.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.024+0000 7fd6eb7fe640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd6d806b040 0x7fd6d806d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:27.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.024+0000 7fd6eb7fe640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd6d806b040 0x7fd6d806d500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc000f60 tx=0x7fd6dc009040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:27.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.028+0000 7fd6d3fff640 1 -- 172.21.15.125:0/2153277819 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6e0054970 con 0x7fd6ec13e520 2023-10-18T10:37:27.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.214+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ec0690e0 con 0x7fd6d806b040 2023-10-18T10:37:27.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.284+0000 7fd6d3fff640 1 -- 172.21.15.125:0/2153277819 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd6ec0690e0 con 0x7fd6d806b040 2023-10-18T10:37:27.286 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-10-18T10:37:27.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd6d806b040 msgr2=0x7fd6d806d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:27.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd6d806b040 0x7fd6d806d500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc000f60 tx=0x7fd6dc009040 comp rx=0 tx=0).stop 2023-10-18T10:37:27.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec13e520 msgr2=0x7fd6ec0a8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:27.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec13e520 0x7fd6ec0a8970 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000d6d0 tx=0x7fd6e000db90 comp rx=0 tx=0).stop 2023-10-18T10:37:27.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 shutdown_connections 2023-10-18T10:37:27.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fd6d806b040 0x7fd6d806d500 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:27.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6ec141690 0x7fd6ec1e2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:27.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ec13e520 0x7fd6ec0a8970 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:27.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 --2- 172.21.15.125:0/2153277819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6ec070920 0x7fd6ec1e0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:27.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 >> 172.21.15.125:0/2153277819 conn(0x7fd6ec069930 msgr2=0x7fd6ec0c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:27.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 shutdown_connections 2023-10-18T10:37:27.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:27.286+0000 7fd6f1b73640 1 -- 172.21.15.125:0/2153277819 wait complete. 2023-10-18T10:37:27.298 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-10-18T10:37:27.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.299 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-18T10:37:27.317 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:27.318 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:27.318 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:27.318 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:27.318 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:27.318 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:27.319 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:27.319 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:27.319 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:27.319 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:27.319 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:27.320 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:27.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.321 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:27.383 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:27.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.384 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:27.447 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:27.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.448 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:27.510 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:27.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.511 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:27.575 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:27.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.576 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:27.637 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:27.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.638 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:27.701 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:27.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.702 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:27.764 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:27.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.765 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:27.831 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:27.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.832 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:27.892 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:27.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.893 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:27.956 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:27.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:27.957 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:28.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.018 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-10-18T10:37:28.076 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1m 2023-10-18T10:37:28.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.219+0000 7f8161165640 1 -- 172.21.15.125:0/3036523850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f815c0ceb40 msgr2=0x7f815c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:28.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.219+0000 7f8161165640 1 --2- 172.21.15.125:0/3036523850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f815c0ceb40 0x7f815c0b0650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8154005fd0 tx=0x7f815402bd90 comp rx=0 tx=0).stop 2023-10-18T10:37:28.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.219+0000 7f8161165640 1 -- 172.21.15.125:0/3036523850 shutdown_connections 2023-10-18T10:37:28.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.219+0000 7f8161165640 1 --2- 172.21.15.125:0/3036523850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f815c0ceb40 0x7f815c0b0650 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:28.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.219+0000 7f8161165640 1 --2- 172.21.15.125:0/3036523850 >> v2:172.21.15.139:3300/0 conn(0x7f815c0cf340 0x7f815c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:28.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.219+0000 7f8161165640 1 -- 172.21.15.125:0/3036523850 >> 172.21.15.125:0/3036523850 conn(0x7f815c0d0d60 msgr2=0x7f815c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:28.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.219+0000 7f8161165640 1 -- 172.21.15.125:0/3036523850 shutdown_connections 2023-10-18T10:37:28.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.219+0000 7f8161165640 1 -- 172.21.15.125:0/3036523850 wait complete. 2023-10-18T10:37:28.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.221+0000 7f8161165640 1 Processor -- start 2023-10-18T10:37:28.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.221+0000 7f8161165640 1 -- start start 2023-10-18T10:37:28.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.221+0000 7f8161165640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815c0ceb40 0x7f815c1f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:28.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.221+0000 7f8161165640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f815c0cf340 0x7f815c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:28.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.221+0000 7f8161165640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f815c15b2a0 0x7f815c1ee920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:28.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f8161165640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c1f2960 con 0x7f815c15b2a0 2023-10-18T10:37:28.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f8161165640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0b6000 con 0x7f815c0ceb40 2023-10-18T10:37:28.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f8161165640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c15bbb0 con 0x7f815c0cf340 2023-10-18T10:37:28.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f815c15b2a0 0x7f815c1ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:28.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f815c15b2a0 0x7f815c1ee920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59160/0 (socket says 172.21.15.125:59160) 2023-10-18T10:37:28.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 -- 172.21.15.125:0/4267716096 learned_addr learned my addr 172.21.15.125:0/4267716096 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:28.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815a59c640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f815c0cf340 0x7f815c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:28.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815ad9d640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815c0ceb40 0x7f815c1f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:28.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 -- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f815c0cf340 msgr2=0x7f815c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:28.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f815c0cf340 0x7f815c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:28.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 -- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815c0ceb40 msgr2=0x7f815c1f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:28.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815c0ceb40 0x7f815c1f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:28.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 -- 172.21.15.125:0/4267716096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c009840 con 0x7f815c15b2a0 2023-10-18T10:37:28.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f815b59e640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f815c15b2a0 0x7f815c1ee920 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8154036040 tx=0x7f815402d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:28.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f81437fe640 1 -- 172.21.15.125:0/4267716096 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815402dd70 con 0x7f815c15b2a0 2023-10-18T10:37:28.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f81437fe640 1 -- 172.21.15.125:0/4267716096 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f81540035f0 con 0x7f815c15b2a0 2023-10-18T10:37:28.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8154009560 con 0x7f815c15b2a0 2023-10-18T10:37:28.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f81437fe640 1 -- 172.21.15.125:0/4267716096 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154003a20 con 0x7f815c15b2a0 2023-10-18T10:37:28.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.222+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c0b0d50 con 0x7f815c15b2a0 2023-10-18T10:37:28.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.223+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815c0b38e0 con 0x7f815c15b2a0 2023-10-18T10:37:28.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.224+0000 7f81437fe640 1 -- 172.21.15.125:0/4267716096 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f8154003b80 con 0x7f815c15b2a0 2023-10-18T10:37:28.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.224+0000 7f81437fe640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f814806b070 0x7f814806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:28.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.224+0000 7f81437fe640 1 -- 172.21.15.125:0/4267716096 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f81540ac4a0 con 0x7f815c15b2a0 2023-10-18T10:37:28.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.224+0000 7f815ad9d640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f814806b070 0x7f814806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:28.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.225+0000 7f815ad9d640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f814806b070 0x7f814806d530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f815c0b0830 tx=0x7f814c009500 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:28.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.228+0000 7f81437fe640 1 -- 172.21.15.125:0/4267716096 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81540772d0 con 0x7f815c15b2a0 2023-10-18T10:37:28.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.416+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f815c0690e0 con 0x7f814806b070 2023-10-18T10:37:28.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.485+0000 7f81437fe640 1 -- 172.21.15.125:0/4267716096 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f815c0690e0 con 0x7f814806b070 2023-10-18T10:37:28.487 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-10-18T10:37:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f814806b070 msgr2=0x7f814806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f814806b070 0x7f814806d530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f815c0b0830 tx=0x7f814c009500 comp rx=0 tx=0).stop 2023-10-18T10:37:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f815c15b2a0 msgr2=0x7f815c1ee920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f815c15b2a0 0x7f815c1ee920 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8154036040 tx=0x7f815402d970 comp rx=0 tx=0).stop 2023-10-18T10:37:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 shutdown_connections 2023-10-18T10:37:28.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f814806b070 0x7f814806d530 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:28.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f815c15b2a0 0x7f815c1ee920 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:28.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f815c0cf340 0x7f815c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:28.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 --2- 172.21.15.125:0/4267716096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815c0ceb40 0x7f815c1f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:28.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 >> 172.21.15.125:0/4267716096 conn(0x7f815c0d0d60 msgr2=0x7f815c06b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:28.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 shutdown_connections 2023-10-18T10:37:28.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:28.487+0000 7f8161165640 1 -- 172.21.15.125:0/4267716096 wait complete. 2023-10-18T10:37:28.499 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-10-18T10:37:28.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.499 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-18T10:37:28.521 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:28.521 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:28.521 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:28.522 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:28.522 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:28.522 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:28.522 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:28.523 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:28.523 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:28.523 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:28.523 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:28.523 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:28.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.524 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:28.585 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:28.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.586 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:28.646 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:28.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.647 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:28.708 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:28.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.709 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:28.770 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:28.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.771 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:28.831 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:28.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.832 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:28.893 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:28.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.894 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:28.955 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:28.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:28.956 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:29.017 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:29.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.018 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:29.079 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:29.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.080 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:29.141 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:29.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.142 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:29.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.203 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-10-18T10:37:29.261 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1m 2023-10-18T10:37:29.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.404+0000 7fc4944a9640 1 -- 172.21.15.125:0/2379455267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4780039a0 msgr2=0x7fc478003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:29.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.404+0000 7fc4944a9640 1 --2- 172.21.15.125:0/2379455267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4780039a0 0x7fc478003dc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc488009b50 tx=0x7fc4780057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:29.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.404+0000 7fc492a46640 1 -- 172.21.15.125:0/2379455267 reap_dead start 2023-10-18T10:37:29.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.404+0000 7fc4944a9640 1 -- 172.21.15.125:0/2379455267 shutdown_connections 2023-10-18T10:37:29.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.404+0000 7fc4944a9640 1 -- 172.21.15.125:0/2379455267 >> 172.21.15.125:0/2379455267 conn(0x7fc48c069930 msgr2=0x7fc48c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:29.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.404+0000 7fc4944a9640 1 -- 172.21.15.125:0/2379455267 shutdown_connections 2023-10-18T10:37:29.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.404+0000 7fc4944a9640 1 -- 172.21.15.125:0/2379455267 wait complete. 2023-10-18T10:37:29.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.405+0000 7fc4944a9640 1 Processor -- start 2023-10-18T10:37:29.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.405+0000 7fc4944a9640 1 -- start start 2023-10-18T10:37:29.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc4944a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780039a0 0x7fc48c139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:29.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc4944a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc48c0aefc0 0x7fc48c1d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:29.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc4944a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc48c1da090 0x7fc48c1da510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:29.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc4944a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c13a0f0 con 0x7fc48c0aefc0 2023-10-18T10:37:29.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc4944a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c1daa50 con 0x7fc4780039a0 2023-10-18T10:37:29.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc4944a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c1eefe0 con 0x7fc48c1da090 2023-10-18T10:37:29.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc48c0aefc0 0x7fc48c1d8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:29.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc48c0aefc0 0x7fc48c1d8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59186/0 (socket says 172.21.15.125:59186) 2023-10-18T10:37:29.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 -- 172.21.15.125:0/818119460 learned_addr learned my addr 172.21.15.125:0/818119460 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:29.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 -- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc48c1da090 msgr2=0x7fc48c1da510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:29.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc492a46640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc48c1da090 0x7fc48c1da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:29.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc492245640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780039a0 0x7fc48c139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:29.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc48c1da090 0x7fc48c1da510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:29.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 -- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780039a0 msgr2=0x7fc48c139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:29.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780039a0 0x7fc48c139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:29.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 -- 172.21.15.125:0/818119460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c0adf20 con 0x7fc48c0aefc0 2023-10-18T10:37:29.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.406+0000 7fc491a44640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc48c0aefc0 0x7fc48c1d8fe0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004e20 tx=0x7fc47c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:29.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.407+0000 7fc482ffd640 1 -- 172.21.15.125:0/818119460 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0107d0 con 0x7fc48c0aefc0 2023-10-18T10:37:29.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.407+0000 7fc482ffd640 1 -- 172.21.15.125:0/818119460 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc47c010a40 con 0x7fc48c0aefc0 2023-10-18T10:37:29.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.407+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c0ae200 con 0x7fc48c0aefc0 2023-10-18T10:37:29.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.407+0000 7fc482ffd640 1 -- 172.21.15.125:0/818119460 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c010e50 con 0x7fc48c0aefc0 2023-10-18T10:37:29.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.407+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c0ae7c0 con 0x7fc48c0aefc0 2023-10-18T10:37:29.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.408+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c0690e0 con 0x7fc48c0aefc0 2023-10-18T10:37:29.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.408+0000 7fc482ffd640 1 -- 172.21.15.125:0/818119460 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fc47c01b460 con 0x7fc48c0aefc0 2023-10-18T10:37:29.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.408+0000 7fc482ffd640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc47006b040 0x7fc47006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:29.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.408+0000 7fc482ffd640 1 -- 172.21.15.125:0/818119460 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fc47c08a810 con 0x7fc48c0aefc0 2023-10-18T10:37:29.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.409+0000 7fc492245640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc47006b040 0x7fc47006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:29.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.409+0000 7fc492245640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc47006b040 0x7fc47006d500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc488036000 tx=0x7fc488035040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:29.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.413+0000 7fc482ffd640 1 -- 172.21.15.125:0/818119460 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc47c055240 con 0x7fc48c0aefc0 2023-10-18T10:37:29.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.601+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc48c0b06f0 con 0x7fc47006b040 2023-10-18T10:37:29.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.669+0000 7fc482ffd640 1 -- 172.21.15.125:0/818119460 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc488033700 con 0x7fc47006b040 2023-10-18T10:37:29.672 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-10-18T10:37:29.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc47006b040 msgr2=0x7fc47006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:29.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc47006b040 0x7fc47006d500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc488036000 tx=0x7fc488035040 comp rx=0 tx=0).stop 2023-10-18T10:37:29.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc48c0aefc0 msgr2=0x7fc48c1d8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:29.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc48c0aefc0 0x7fc48c1d8fe0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004e20 tx=0x7fc47c001630 comp rx=0 tx=0).stop 2023-10-18T10:37:29.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 shutdown_connections 2023-10-18T10:37:29.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc47006b040 0x7fc47006d500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:29.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc48c1da090 0x7fc48c1da510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:29.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc48c0aefc0 0x7fc48c1d8fe0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:29.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 --2- 172.21.15.125:0/818119460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780039a0 0x7fc48c139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:29.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 >> 172.21.15.125:0/818119460 conn(0x7fc48c069930 msgr2=0x7fc48c0affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:29.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 shutdown_connections 2023-10-18T10:37:29.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:29.671+0000 7fc4944a9640 1 -- 172.21.15.125:0/818119460 wait complete. 2023-10-18T10:37:29.684 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-10-18T10:37:29.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.684 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-18T10:37:29.707 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:29.708 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:29.708 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:29.708 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:29.708 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:29.708 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:29.709 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:29.709 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:29.709 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:29.709 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:29.709 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:29.710 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:29.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.711 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:29.770 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:29.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.771 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:29.833 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:29.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.834 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:29.896 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:29.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.897 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:29.959 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:29.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:29.960 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:30.021 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:30.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.022 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:30.083 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:30.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.084 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:30.146 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:30.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.147 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:30.209 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:30.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.211 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:30.272 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:30.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.273 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:30.335 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:30.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.336 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:30.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.398 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-10-18T10:37:30.455 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1m 2023-10-18T10:37:30.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 -- 172.21.15.125:0/1825052117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 msgr2=0x7f41a0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:30.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/1825052117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 0x7f41a0070920 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f41a006a780 tx=0x7f419802c140 comp rx=0 tx=0).stop 2023-10-18T10:37:30.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 -- 172.21.15.125:0/1825052117 shutdown_connections 2023-10-18T10:37:30.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/1825052117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 0x7f41a0070920 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/1825052117 >> v2:172.21.15.139:3300/0 conn(0x7f41a0141690 0x7f41a00703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/1825052117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41a013e520 0x7f41a006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 -- 172.21.15.125:0/1825052117 >> 172.21.15.125:0/1825052117 conn(0x7f41a0069930 msgr2=0x7f41a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:30.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 -- 172.21.15.125:0/1825052117 shutdown_connections 2023-10-18T10:37:30.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.598+0000 7f41a6ee1640 1 -- 172.21.15.125:0/1825052117 wait complete. 2023-10-18T10:37:30.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.599+0000 7f41a6ee1640 1 Processor -- start 2023-10-18T10:37:30.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.599+0000 7f41a6ee1640 1 -- start start 2023-10-18T10:37:30.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a6ee1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 0x7f41a01df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:30.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a6ee1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41a013e520 0x7f41a01e0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:30.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a6ee1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41a0141690 0x7f41a01e1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:30.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a6ee1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a01e3200 con 0x7f41a006c0a0 2023-10-18T10:37:30.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a6ee1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a01e1990 con 0x7f41a013e520 2023-10-18T10:37:30.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a6ee1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a01e16b0 con 0x7f41a0141690 2023-10-18T10:37:30.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 0x7f41a01df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:30.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 0x7f41a01df8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59204/0 (socket says 172.21.15.125:59204) 2023-10-18T10:37:30.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 -- 172.21.15.125:0/762100168 learned_addr learned my addr 172.21.15.125:0/762100168 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:30.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 -- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41a0141690 msgr2=0x7f41a01e1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:30.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41a0141690 0x7f41a01e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 -- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41a013e520 msgr2=0x7f41a01e0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:30.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41a013e520 0x7f41a01e0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 -- 172.21.15.125:0/762100168 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4198009900 con 0x7f41a006c0a0 2023-10-18T10:37:30.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.600+0000 7f41a4c7d640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 0x7f41a01df8b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f419000b4a0 tx=0x7f419000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:30.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.601+0000 7f419d7fa640 1 -- 172.21.15.125:0/762100168 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190010610 con 0x7f41a006c0a0 2023-10-18T10:37:30.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.601+0000 7f419d7fa640 1 -- 172.21.15.125:0/762100168 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4190010860 con 0x7f41a006c0a0 2023-10-18T10:37:30.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.601+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a01dbcf0 con 0x7f41a006c0a0 2023-10-18T10:37:30.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.601+0000 7f419d7fa640 1 -- 172.21.15.125:0/762100168 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190010c70 con 0x7f41a006c0a0 2023-10-18T10:37:30.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.601+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a01dc2b0 con 0x7f41a006c0a0 2023-10-18T10:37:30.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.602+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a00690e0 con 0x7f41a006c0a0 2023-10-18T10:37:30.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.602+0000 7f419d7fa640 1 -- 172.21.15.125:0/762100168 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f4190017360 con 0x7f41a006c0a0 2023-10-18T10:37:30.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.602+0000 7f419d7fa640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418c06b040 0x7f418c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:30.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.602+0000 7f419d7fa640 1 -- 172.21.15.125:0/762100168 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f4190089b70 con 0x7f41a006c0a0 2023-10-18T10:37:30.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.602+0000 7f419ffff640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418c06b040 0x7f418c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:30.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.603+0000 7f419ffff640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418c06b040 0x7f418c06d500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41a00ae6e0 tx=0x7f4194006bf0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:30.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.606+0000 7f419d7fa640 1 -- 172.21.15.125:0/762100168 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4190054630 con 0x7f41a006c0a0 2023-10-18T10:37:30.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.791+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f41a006be80 con 0x7f418c06b040 2023-10-18T10:37:30.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.861+0000 7f419d7fa640 1 -- 172.21.15.125:0/762100168 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f41a006be80 con 0x7f418c06b040 2023-10-18T10:37:30.863 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-10-18T10:37:30.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418c06b040 msgr2=0x7f418c06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:30.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418c06b040 0x7f418c06d500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41a00ae6e0 tx=0x7f4194006bf0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 msgr2=0x7f41a01df8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:30.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 0x7f41a01df8b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f419000b4a0 tx=0x7f419000b960 comp rx=0 tx=0).stop 2023-10-18T10:37:30.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 shutdown_connections 2023-10-18T10:37:30.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418c06b040 0x7f418c06d500 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41a013e520 0x7f41a01e0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41a0141690 0x7f41a01e1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 --2- 172.21.15.125:0/762100168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41a006c0a0 0x7f41a01df8b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 >> 172.21.15.125:0/762100168 conn(0x7f41a0069930 msgr2=0x7f41a013d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 shutdown_connections 2023-10-18T10:37:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:30.863+0000 7f41a6ee1640 1 -- 172.21.15.125:0/762100168 wait complete. 2023-10-18T10:37:30.875 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-10-18T10:37:30.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.876 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-18T10:37:30.898 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:30.898 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:30.899 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:30.899 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:30.899 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:30.899 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:30.899 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:30.900 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:30.900 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:30.900 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:30.900 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:30.900 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:30.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.901 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:30.960 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:30.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:30.961 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:31.024 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:31.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.025 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:31.086 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:31.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.087 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:31.148 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:31.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.149 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:31.210 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:31.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.211 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:31.273 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:31.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.274 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:31.336 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:31.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.337 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:31.398 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:31.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.399 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:31.460 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:31.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.461 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:31.523 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:31.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.524 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:31.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:31.585 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-10-18T10:37:31.643 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1m 2023-10-18T10:37:31.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.788+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/283496165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b413d350 msgr2=0x7fb3b40cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:31.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.788+0000 7fb3bc1b0640 1 --2- 172.21.15.125:0/283496165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b413d350 0x7fb3b40cbd60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4005fd0 tx=0x7fb3a402be20 comp rx=0 tx=0).stop 2023-10-18T10:37:31.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.788+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/283496165 shutdown_connections 2023-10-18T10:37:31.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.788+0000 7fb3bc1b0640 1 --2- 172.21.15.125:0/283496165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b413d350 0x7fb3b40cbd60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.788+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/283496165 >> 172.21.15.125:0/283496165 conn(0x7fb3b4069930 msgr2=0x7fb3b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.788+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/283496165 shutdown_connections 2023-10-18T10:37:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.788+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/283496165 wait complete. 2023-10-18T10:37:31.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.789+0000 7fb3bc1b0640 1 Processor -- start 2023-10-18T10:37:31.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.789+0000 7fb3bc1b0640 1 -- start start 2023-10-18T10:37:31.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.789+0000 7fb3bc1b0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b4070350 0x7fb3b40aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:31.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.789+0000 7fb3bc1b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b40cb980 0x7fb3b40adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:31.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.789+0000 7fb3bc1b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b40aae00 0x7fb3b40ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:31.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.789+0000 7fb3bc1b0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4137000 con 0x7fb3b4070350 2023-10-18T10:37:31.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3bc1b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b413b260 con 0x7fb3b40cb980 2023-10-18T10:37:31.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3bc1b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40acd80 con 0x7fb3b40aae00 2023-10-18T10:37:31.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b4070350 0x7fb3b40aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:31.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b4070350 0x7fb3b40aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59230/0 (socket says 172.21.15.125:59230) 2023-10-18T10:37:31.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 -- 172.21.15.125:0/634438407 learned_addr learned my addr 172.21.15.125:0/634438407 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:31.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3ba74d640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b40aae00 0x7fb3b40ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:31.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b974b640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b40cb980 0x7fb3b40adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:31.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 -- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b40aae00 msgr2=0x7fb3b40ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:31.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b40aae00 0x7fb3b40ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:31.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 -- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b40cb980 msgr2=0x7fb3b40adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:31.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b40cb980 0x7fb3b40adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:31.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 -- 172.21.15.125:0/634438407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a4009560 con 0x7fb3b4070350 2023-10-18T10:37:31.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3ba74d640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b40aae00 0x7fb3b40ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:31.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b974b640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b40cb980 0x7fb3b40adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:31.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3b9f4c640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b4070350 0x7fb3b40aa8c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c430 tx=0x7fb3b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:31.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3aa7fc640 1 -- 172.21.15.125:0/634438407 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0011790 con 0x7fb3b4070350 2023-10-18T10:37:31.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3aa7fc640 1 -- 172.21.15.125:0/634438407 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3b00119e0 con 0x7fb3b4070350 2023-10-18T10:37:31.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b40ad060 con 0x7fb3b4070350 2023-10-18T10:37:31.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3aa7fc640 1 -- 172.21.15.125:0/634438407 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0011df0 con 0x7fb3b4070350 2023-10-18T10:37:31.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.790+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b40ad620 con 0x7fb3b4070350 2023-10-18T10:37:31.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.791+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b40b1940 con 0x7fb3b4070350 2023-10-18T10:37:31.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.792+0000 7fb3aa7fc640 1 -- 172.21.15.125:0/634438407 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fb3b001c460 con 0x7fb3b4070350 2023-10-18T10:37:31.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.792+0000 7fb3aa7fc640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb39806b070 0x7fb39806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:31.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.792+0000 7fb3aa7fc640 1 -- 172.21.15.125:0/634438407 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fb3b0089e20 con 0x7fb3b4070350 2023-10-18T10:37:31.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.792+0000 7fb3b974b640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb39806b070 0x7fb39806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:31.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.793+0000 7fb3b974b640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb39806b070 0x7fb39806d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4035000 tx=0x7fb3a4004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:31.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.796+0000 7fb3aa7fc640 1 -- 172.21.15.125:0/634438407 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3b00549f0 con 0x7fb3b4070350 2023-10-18T10:37:31.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:31.980+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3b4070e90 con 0x7fb39806b070 2023-10-18T10:37:32.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.049+0000 7fb3aa7fc640 1 -- 172.21.15.125:0/634438407 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb3a40043c0 con 0x7fb39806b070 2023-10-18T10:37:32.051 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-10-18T10:37:32.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb39806b070 msgr2=0x7fb39806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:32.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb39806b070 0x7fb39806d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4035000 tx=0x7fb3a4004730 comp rx=0 tx=0).stop 2023-10-18T10:37:32.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b4070350 msgr2=0x7fb3b40aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:32.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b4070350 0x7fb3b40aa8c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c430 tx=0x7fb3b000c8f0 comp rx=0 tx=0).stop 2023-10-18T10:37:32.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 shutdown_connections 2023-10-18T10:37:32.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fb39806b070 0x7fb39806d530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:32.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b40cb980 0x7fb3b40adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:32.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b40aae00 0x7fb3b40ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:32.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 --2- 172.21.15.125:0/634438407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3b4070350 0x7fb3b40aa8c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:32.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 >> 172.21.15.125:0/634438407 conn(0x7fb3b4069930 msgr2=0x7fb3b4070780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:32.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 shutdown_connections 2023-10-18T10:37:32.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:32.051+0000 7fb3bc1b0640 1 -- 172.21.15.125:0/634438407 wait complete. 2023-10-18T10:37:32.063 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-10-18T10:37:32.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.064 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-18T10:37:32.085 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:32.087 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:32.087 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:32.087 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:32.087 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:32.087 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:32.088 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:32.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.089 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:32.150 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:32.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.151 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:32.212 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:32.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.213 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:32.273 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:32.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.274 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:32.335 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:32.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.336 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:32.437 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:32.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.438 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:32.473 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:32.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.474 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:32.542 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:32.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.543 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:32.606 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:32.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.607 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:32.684 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:32.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.685 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:32.757 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:32.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.758 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:32.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:32.820 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-10-18T10:37:32.877 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1m 2023-10-18T10:37:33.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.052+0000 7fc259475640 1 -- 172.21.15.125:0/1165530646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2541424c0 msgr2=0x7fc25406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:33.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.052+0000 7fc259475640 1 --2- 172.21.15.125:0/1165530646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2541424c0 0x7fc25406c630 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc244009830 tx=0x7fc24402bdf0 comp rx=0 tx=0).stop 2023-10-18T10:37:33.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.052+0000 7fc259475640 1 -- 172.21.15.125:0/1165530646 shutdown_connections 2023-10-18T10:37:33.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.052+0000 7fc259475640 1 --2- 172.21.15.125:0/1165530646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2541424c0 0x7fc25406c630 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:33.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.052+0000 7fc259475640 1 -- 172.21.15.125:0/1165530646 >> 172.21.15.125:0/1165530646 conn(0x7fc2540af730 msgr2=0x7fc2540afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.053+0000 7fc259475640 1 -- 172.21.15.125:0/1165530646 shutdown_connections 2023-10-18T10:37:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.053+0000 7fc259475640 1 -- 172.21.15.125:0/1165530646 wait complete. 2023-10-18T10:37:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.054+0000 7fc259475640 1 Processor -- start 2023-10-18T10:37:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.054+0000 7fc259475640 1 -- start start 2023-10-18T10:37:33.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc259475640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc254069e30 0x7fc2541e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:33.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc259475640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc254070080 0x7fc2540acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:33.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc259475640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2541424c0 0x7fc2541e1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:33.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc259475640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc254154a00 con 0x7fc254070080 2023-10-18T10:37:33.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc259475640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2541e44f0 con 0x7fc2541424c0 2023-10-18T10:37:33.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc259475640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2541ef310 con 0x7fc254069e30 2023-10-18T10:37:33.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc2527fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc254070080 0x7fc2540acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:33.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc2527fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc254070080 0x7fc2540acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59242/0 (socket says 172.21.15.125:59242) 2023-10-18T10:37:33.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.055+0000 7fc2527fc640 1 -- 172.21.15.125:0/1415772191 learned_addr learned my addr 172.21.15.125:0/1415772191 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:33.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.056+0000 7fc2537fe640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2541424c0 0x7fc2541e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:33.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.056+0000 7fc252ffd640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc254069e30 0x7fc2541e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:33.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.056+0000 7fc2527fc640 1 -- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc254069e30 msgr2=0x7fc2541e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:33.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.056+0000 7fc2527fc640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc254069e30 0x7fc2541e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:33.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.056+0000 7fc2527fc640 1 -- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2541424c0 msgr2=0x7fc2541e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:33.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.056+0000 7fc2527fc640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2541424c0 0x7fc2541e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:33.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.056+0000 7fc2527fc640 1 -- 172.21.15.125:0/1415772191 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc244009490 con 0x7fc254070080 2023-10-18T10:37:33.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.056+0000 7fc2527fc640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc254070080 0x7fc2540acda0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c490 tx=0x7fc24800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:33.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.057+0000 7fc23b7fe640 1 -- 172.21.15.125:0/1415772191 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2480035f0 con 0x7fc254070080 2023-10-18T10:37:33.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.057+0000 7fc23b7fe640 1 -- 172.21.15.125:0/1415772191 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc248003770 con 0x7fc254070080 2023-10-18T10:37:33.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.057+0000 7fc23b7fe640 1 -- 172.21.15.125:0/1415772191 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248003b80 con 0x7fc254070080 2023-10-18T10:37:33.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.057+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc254154440 con 0x7fc254070080 2023-10-18T10:37:33.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.057+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2540a80a0 con 0x7fc254070080 2023-10-18T10:37:33.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.058+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2540a7f00 con 0x7fc254070080 2023-10-18T10:37:33.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.059+0000 7fc23b7fe640 1 -- 172.21.15.125:0/1415772191 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fc248003ce0 con 0x7fc254070080 2023-10-18T10:37:33.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.060+0000 7fc23b7fe640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc24006b090 0x7fc24006d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:33.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.060+0000 7fc23b7fe640 1 -- 172.21.15.125:0/1415772191 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 7364+0+0 (secure 0 0 0) 0x7fc248089980 con 0x7fc254070080 2023-10-18T10:37:33.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.060+0000 7fc252ffd640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc24006b090 0x7fc24006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:33.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.061+0000 7fc252ffd640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc24006b090 0x7fc24006d550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc244036000 tx=0x7fc244040040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:33.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.064+0000 7fc23b7fe640 1 -- 172.21.15.125:0/1415772191 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc248054270 con 0x7fc254070080 2023-10-18T10:37:33.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.274+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2540cccd0 con 0x7fc24006b090 2023-10-18T10:37:33.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.338+0000 7fc23b7fe640 1 -- 172.21.15.125:0/1415772191 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc24402c8b0 con 0x7fc24006b090 2023-10-18T10:37:33.340 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-10-18T10:37:33.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc24006b090 msgr2=0x7fc24006d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:33.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc24006b090 0x7fc24006d550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc244036000 tx=0x7fc244040040 comp rx=0 tx=0).stop 2023-10-18T10:37:33.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc254070080 msgr2=0x7fc2540acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:33.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc254070080 0x7fc2540acda0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c490 tx=0x7fc24800c950 comp rx=0 tx=0).stop 2023-10-18T10:37:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 shutdown_connections 2023-10-18T10:37:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc24006b090 0x7fc24006d550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2541424c0 0x7fc2541e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc254070080 0x7fc2540acda0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:33.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 --2- 172.21.15.125:0/1415772191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc254069e30 0x7fc2541e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:33.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 >> 172.21.15.125:0/1415772191 conn(0x7fc2540af730 msgr2=0x7fc254156610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:33.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 shutdown_connections 2023-10-18T10:37:33.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:33.340+0000 7fc259475640 1 -- 172.21.15.125:0/1415772191 wait complete. 2023-10-18T10:37:33.353 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-10-18T10:37:33.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.354 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-18T10:37:33.380 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:33.380 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:33.381 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:33.381 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:33.381 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:33.381 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:33.381 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:33.382 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:33.382 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:33.382 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:33.382 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:33.383 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:33.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.383 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:33.443 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:33.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.444 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:33.505 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:33.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.506 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:33.567 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:33.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.568 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:33.630 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:33.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.631 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:33.692 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:33.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.693 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:33.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:33.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.755 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:33.816 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:33.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.817 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:33.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:33.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.881 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:33.942 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:33.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:33.943 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:34.003 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:34.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.004 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:34.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.065 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-10-18T10:37:34.126 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1m 2023-10-18T10:37:34.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.270+0000 7fe3e2da9640 1 -- 172.21.15.125:0/835752232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3c8004d00 msgr2=0x7fe3c8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:34.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.270+0000 7fe3e2da9640 1 --2- 172.21.15.125:0/835752232 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3c8004d00 0x7fe3c8005180 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00bb20 tx=0x7fe3c80057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.270+0000 7fe3e1346640 1 -- 172.21.15.125:0/835752232 reap_dead start 2023-10-18T10:37:34.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.270+0000 7fe3e2da9640 1 -- 172.21.15.125:0/835752232 shutdown_connections 2023-10-18T10:37:34.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.270+0000 7fe3e2da9640 1 -- 172.21.15.125:0/835752232 >> 172.21.15.125:0/835752232 conn(0x7fe3dc0d0d60 msgr2=0x7fe3dc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:34.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.270+0000 7fe3e2da9640 1 -- 172.21.15.125:0/835752232 shutdown_connections 2023-10-18T10:37:34.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.270+0000 7fe3e2da9640 1 -- 172.21.15.125:0/835752232 wait complete. 2023-10-18T10:37:34.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.271+0000 7fe3e2da9640 1 Processor -- start 2023-10-18T10:37:34.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.271+0000 7fe3e2da9640 1 -- start start 2023-10-18T10:37:34.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e2da9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3c8004d00 0x7fe3dc158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:34.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e2da9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3dc0aeb90 0x7fe3dc0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:34.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e2da9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3dc0b2d90 0x7fe3dc1e4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:34.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e2da9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0b3240 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e2da9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc15ba20 con 0x7fe3c8004d00 2023-10-18T10:37:34.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e2da9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc15bd00 con 0x7fe3dc0aeb90 2023-10-18T10:37:34.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3dc0b2d90 0x7fe3dc1e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:34.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3dc0b2d90 0x7fe3dc1e4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59252/0 (socket says 172.21.15.125:59252) 2023-10-18T10:37:34.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 -- 172.21.15.125:0/2151663145 learned_addr learned my addr 172.21.15.125:0/2151663145 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:34.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 -- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3dc0aeb90 msgr2=0x7fe3dc0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:34.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3dc0aeb90 0x7fe3dc0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 -- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3c8004d00 msgr2=0x7fe3dc158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:34.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e0b45640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3c8004d00 0x7fe3dc158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:34.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3c8004d00 0x7fe3dc158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 -- 172.21.15.125:0/2151663145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc15bfe0 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.272+0000 7fe3e1346640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3dc0b2d90 0x7fe3dc1e4690 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00bb20 tx=0x7fe3cc0366a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:34.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.273+0000 7fe3d97fa640 1 -- 172.21.15.125:0/2151663145 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc036ce0 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.273+0000 7fe3d97fa640 1 -- 172.21.15.125:0/2151663145 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3cc004030 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.273+0000 7fe3d97fa640 1 -- 172.21.15.125:0/2151663145 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc004440 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.273+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc15c2c0 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.273+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc0af720 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.274+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc15c7e0 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.274+0000 7fe3d97fa640 1 -- 172.21.15.125:0/2151663145 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fe3cc0035f0 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.274+0000 7fe3d97fa640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe3bc06b040 0x7fe3bc06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:34.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.274+0000 7fe3d97fa640 1 -- 172.21.15.125:0/2151663145 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6932+0+0 (secure 0 0 0) 0x7fe3cc000e40 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.274+0000 7fe3e0b45640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe3bc06b040 0x7fe3bc06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:34.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.275+0000 7fe3e0b45640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe3bc06b040 0x7fe3bc06d500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40013d0 tx=0x7fe3d40021f0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:34.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.278+0000 7fe3d97fa640 1 -- 172.21.15.125:0/2151663145 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3dc15c7e0 con 0x7fe3dc0b2d90 2023-10-18T10:37:34.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.486+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3dc0d1910 con 0x7fe3bc06b040 2023-10-18T10:37:34.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.558+0000 7fe3d97fa640 1 -- 172.21.15.125:0/2151663145 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fe3d4003e00 con 0x7fe3bc06b040 2023-10-18T10:37:34.560 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-10-18T10:37:34.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe3bc06b040 msgr2=0x7fe3bc06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:34.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe3bc06b040 0x7fe3bc06d500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40013d0 tx=0x7fe3d40021f0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3dc0b2d90 msgr2=0x7fe3dc1e4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:34.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3dc0b2d90 0x7fe3dc1e4690 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00bb20 tx=0x7fe3cc0366a0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 shutdown_connections 2023-10-18T10:37:34.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe3bc06b040 0x7fe3bc06d500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3dc0b2d90 0x7fe3dc1e4690 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3dc0aeb90 0x7fe3dc0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 --2- 172.21.15.125:0/2151663145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3c8004d00 0x7fe3dc158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:34.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 >> 172.21.15.125:0/2151663145 conn(0x7fe3dc0d0d60 msgr2=0x7fe3dc15a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:34.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 shutdown_connections 2023-10-18T10:37:34.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:34.560+0000 7fe3e2da9640 1 -- 172.21.15.125:0/2151663145 wait complete. 2023-10-18T10:37:34.572 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-10-18T10:37:34.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.573 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-18T10:37:34.595 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:34.595 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:34.595 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:34.595 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:34.595 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:34.596 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:34.596 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:34.596 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:34.596 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:34.597 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:34.597 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:34.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:34.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.598 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:34.657 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:34.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.658 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:34.720 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:34.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.721 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:34.781 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:34.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.783 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:34.843 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:34.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.844 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:34.905 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:34.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.906 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:34.966 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:34.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:34.967 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:35.027 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:35.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.028 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:35.039 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:37:35.070 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:37:35.089 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:35.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.090 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:35.183 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:35.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.184 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:35.244 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:35.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.245 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:35.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.306 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-10-18T10:37:35.364 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1m 2023-10-18T10:37:35.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 -- 172.21.15.125:0/4267735453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 msgr2=0x7f9bf40703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:35.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/4267735453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 0x7f9bf40703e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cbd00 tx=0x7f9bf002bd10 comp rx=0 tx=0).stop 2023-10-18T10:37:35.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 -- 172.21.15.125:0/4267735453 shutdown_connections 2023-10-18T10:37:35.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/4267735453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 0x7f9bf40703e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/4267735453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf406c0a0 0x7f9bf4070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/4267735453 >> v2:172.21.15.139:3300/0 conn(0x7f9bf413e520 0x7f9bf406bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 -- 172.21.15.125:0/4267735453 >> 172.21.15.125:0/4267735453 conn(0x7f9bf4069930 msgr2=0x7f9bf4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:35.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 -- 172.21.15.125:0/4267735453 shutdown_connections 2023-10-18T10:37:35.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.515+0000 7f9bfc38e640 1 -- 172.21.15.125:0/4267735453 wait complete. 2023-10-18T10:37:35.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfc38e640 1 Processor -- start 2023-10-18T10:37:35.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfc38e640 1 -- start start 2023-10-18T10:37:35.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfc38e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf406c0a0 0x7f9bf413b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:35.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfc38e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf413e520 0x7f9bf41d9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:35.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfc38e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 0x7f9bf40c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:35.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfc38e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf41d94e0 con 0x7f9bf4141690 2023-10-18T10:37:35.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfc38e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf413b780 con 0x7f9bf413e520 2023-10-18T10:37:35.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfc38e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40c6840 con 0x7f9bf406c0a0 2023-10-18T10:37:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfa92b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 0x7f9bf40c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfa92b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 0x7f9bf40c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59272/0 (socket says 172.21.15.125:59272) 2023-10-18T10:37:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.517+0000 7f9bfa92b640 1 -- 172.21.15.125:0/3641214313 learned_addr learned my addr 172.21.15.125:0/3641214313 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfa92b640 1 -- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf406c0a0 msgr2=0x7f9bf413b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bf9929640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf413e520 0x7f9bf41d9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfa12a640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf406c0a0 0x7f9bf413b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfa92b640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf406c0a0 0x7f9bf413b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfa92b640 1 -- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf413e520 msgr2=0x7f9bf41d9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfa92b640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf413e520 0x7f9bf41d9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfa92b640 1 -- 172.21.15.125:0/3641214313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009560 con 0x7f9bf4141690 2023-10-18T10:37:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfa92b640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 0x7f9bf40c5da0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00eaa0 tx=0x7f9bec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9beaffd640 1 -- 172.21.15.125:0/3641214313 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00ce70 con 0x7f9bf4141690 2023-10-18T10:37:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9beaffd640 1 -- 172.21.15.125:0/3641214313 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9bec004c10 con 0x7f9bf4141690 2023-10-18T10:37:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf40c6b20 con 0x7f9bf4141690 2023-10-18T10:37:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9beaffd640 1 -- 172.21.15.125:0/3641214313 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec005020 con 0x7f9bf4141690 2023-10-18T10:37:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.518+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf41d99d0 con 0x7f9bf4141690 2023-10-18T10:37:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.519+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf40690e0 con 0x7f9bf4141690 2023-10-18T10:37:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.520+0000 7f9beaffd640 1 -- 172.21.15.125:0/3641214313 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f9bec0051a0 con 0x7f9bf4141690 2023-10-18T10:37:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.520+0000 7f9beaffd640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9bd806b040 0x7f9bd806d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.520+0000 7f9beaffd640 1 -- 172.21.15.125:0/3641214313 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f9bec014070 con 0x7f9bf4141690 2023-10-18T10:37:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.520+0000 7f9bfa12a640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9bd806b040 0x7f9bd806d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.521+0000 7f9bfa12a640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9bd806b040 0x7f9bd806d500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bf41e1170 tx=0x7f9be4005f90 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.524+0000 7f9beaffd640 1 -- 172.21.15.125:0/3641214313 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9bec08e030 con 0x7f9bf4141690 2023-10-18T10:37:35.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.736+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bf40c6d60 con 0x7f9bd806b040 2023-10-18T10:37:35.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.793+0000 7f9beaffd640 1 -- 172.21.15.125:0/3641214313 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f9bf40c6d60 con 0x7f9bd806b040 2023-10-18T10:37:35.795 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-10-18T10:37:35.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9bd806b040 msgr2=0x7f9bd806d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:35.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9bd806b040 0x7f9bd806d500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bf41e1170 tx=0x7f9be4005f90 comp rx=0 tx=0).stop 2023-10-18T10:37:35.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 msgr2=0x7f9bf40c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:35.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 0x7f9bf40c5da0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00eaa0 tx=0x7f9bec00ef60 comp rx=0 tx=0).stop 2023-10-18T10:37:35.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 shutdown_connections 2023-10-18T10:37:35.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f9bd806b040 0x7f9bd806d500 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bf413e520 0x7f9bf41d9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9bf4141690 0x7f9bf40c5da0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 --2- 172.21.15.125:0/3641214313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bf406c0a0 0x7f9bf413b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:35.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 >> 172.21.15.125:0/3641214313 conn(0x7f9bf4069930 msgr2=0x7f9bf413d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:35.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 shutdown_connections 2023-10-18T10:37:35.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:35.795+0000 7f9bfc38e640 1 -- 172.21.15.125:0/3641214313 wait complete. 2023-10-18T10:37:35.807 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-10-18T10:37:35.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.808 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-18T10:37:35.829 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:35.830 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:35.830 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:35.830 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:35.830 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:35.830 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:35.831 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:35.831 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_01_UTC 2023-10-18T10:37:35.831 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:35.832 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:35.832 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:35.832 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:35.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.833 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:35.891 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:35.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.892 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:35.957 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:35.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:35.958 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:36.018 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:36.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.019 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:36.080 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:36.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.081 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:36.142 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:36.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.143 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:36.203 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:36.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.204 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:36.265 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_34_01_UTC 2023-10-18T10:37:36.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.362 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_34_01_UTC) 2023-10-18T10:37:36.404 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:36.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.405 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:36.466 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:36.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.467 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:36.528 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:36.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.528 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:36.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:36.590 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-10-18T10:37:36.651 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1m 2023-10-18T10:37:36.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.794+0000 7f4b86b7d640 1 -- 172.21.15.125:0/3991655200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 msgr2=0x7f4b800b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:36.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.794+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/3991655200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 0x7f4b800b09a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4b800cc490 tx=0x7f4b7802c2f0 comp rx=0 tx=0).stop 2023-10-18T10:37:36.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.794+0000 7f4b86b7d640 1 -- 172.21.15.125:0/3991655200 shutdown_connections 2023-10-18T10:37:36.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.794+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/3991655200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 0x7f4b800b09a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:36.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.794+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/3991655200 >> v2:172.21.15.139:3300/0 conn(0x7f4b800afa20 0x7f4b80158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:36.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.794+0000 7f4b86b7d640 1 -- 172.21.15.125:0/3991655200 >> 172.21.15.125:0/3991655200 conn(0x7f4b80069930 msgr2=0x7f4b80069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:36.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.794+0000 7f4b86b7d640 1 -- 172.21.15.125:0/3991655200 shutdown_connections 2023-10-18T10:37:36.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.794+0000 7f4b86b7d640 1 -- 172.21.15.125:0/3991655200 wait complete. 2023-10-18T10:37:36.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.795+0000 7f4b86b7d640 1 Processor -- start 2023-10-18T10:37:36.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.795+0000 7f4b86b7d640 1 -- start start 2023-10-18T10:37:36.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b86b7d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b800afa20 0x7f4b80143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:36.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b86b7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 0x7f4b801e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:36.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b86b7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b800cbd20 0x7f4b801e3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:36.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b86b7d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801e3180 con 0x7f4b800b0530 2023-10-18T10:37:36.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b86b7d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801e2570 con 0x7f4b800cbd20 2023-10-18T10:37:36.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b86b7d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801e2850 con 0x7f4b800afa20 2023-10-18T10:37:36.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b77fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 0x7f4b801e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:36.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b77fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 0x7f4b801e1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59290/0 (socket says 172.21.15.125:59290) 2023-10-18T10:37:36.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.796+0000 7f4b77fff640 1 -- 172.21.15.125:0/2621355626 learned_addr learned my addr 172.21.15.125:0/2621355626 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:36.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b8511a640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b800cbd20 0x7f4b801e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:36.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b84919640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b800afa20 0x7f4b80143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:36.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b77fff640 1 -- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b800afa20 msgr2=0x7f4b80143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:36.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b77fff640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b800afa20 0x7f4b80143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:36.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b77fff640 1 -- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b800cbd20 msgr2=0x7f4b801e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:36.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b77fff640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b800cbd20 0x7f4b801e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:36.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b77fff640 1 -- 172.21.15.125:0/2621355626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78009900 con 0x7f4b800b0530 2023-10-18T10:37:36.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b84919640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b800afa20 0x7f4b80143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:36.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b77fff640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 0x7f4b801e1c20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000b4a0 tx=0x7f4b7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:36.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b757fa640 1 -- 172.21.15.125:0/2621355626 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70010610 con 0x7f4b800b0530 2023-10-18T10:37:36.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b801e2b30 con 0x7f4b800b0530 2023-10-18T10:37:36.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b757fa640 1 -- 172.21.15.125:0/2621355626 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b70010860 con 0x7f4b800b0530 2023-10-18T10:37:36.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b757fa640 1 -- 172.21.15.125:0/2621355626 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70010c70 con 0x7f4b800b0530 2023-10-18T10:37:36.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.797+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b80143870 con 0x7f4b800b0530 2023-10-18T10:37:36.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.798+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b801e32f0 con 0x7f4b800b0530 2023-10-18T10:37:36.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.798+0000 7f4b757fa640 1 -- 172.21.15.125:0/2621355626 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f4b70017360 con 0x7f4b800b0530 2023-10-18T10:37:36.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.799+0000 7f4b757fa640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f4b6c06b070 0x7f4b6c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.799+0000 7f4b757fa640 1 -- 172.21.15.125:0/2621355626 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f4b70089ca0 con 0x7f4b800b0530 2023-10-18T10:37:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.799+0000 7f4b84919640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f4b6c06b070 0x7f4b6c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.799+0000 7f4b84919640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f4b6c06b070 0x7f4b6c06d530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b800c8bb0 tx=0x7f4b7c009040 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.803+0000 7f4b757fa640 1 -- 172.21.15.125:0/2621355626 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b70054650 con 0x7f4b800b0530 2023-10-18T10:37:36.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:36.990+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b800cc100 con 0x7f4b6c06b070 2023-10-18T10:37:37.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.048+0000 7f4b757fa640 1 -- 172.21.15.125:0/2621355626 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f4b7c0050d0 con 0x7f4b6c06b070 2023-10-18T10:37:37.050 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-10-18T10:37:37.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f4b6c06b070 msgr2=0x7f4b6c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:37.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f4b6c06b070 0x7f4b6c06d530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b800c8bb0 tx=0x7f4b7c009040 comp rx=0 tx=0).stop 2023-10-18T10:37:37.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 msgr2=0x7f4b801e1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:37.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 0x7f4b801e1c20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000b4a0 tx=0x7f4b7000b960 comp rx=0 tx=0).stop 2023-10-18T10:37:37.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 shutdown_connections 2023-10-18T10:37:37.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f4b6c06b070 0x7f4b6c06d530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b800cbd20 0x7f4b801e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b800b0530 0x7f4b801e1c20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 --2- 172.21.15.125:0/2621355626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b800afa20 0x7f4b80143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 >> 172.21.15.125:0/2621355626 conn(0x7f4b80069930 msgr2=0x7f4b8006ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:37.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 shutdown_connections 2023-10-18T10:37:37.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.050+0000 7f4b86b7d640 1 -- 172.21.15.125:0/2621355626 wait complete. 2023-10-18T10:37:37.062 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-10-18T10:37:37.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.063 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-18T10:37:37.084 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:37.085 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:37.085 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:37.085 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:37.085 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:37.085 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:37.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:37.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:37.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:37.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:37.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:37.087 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:37.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.087 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:37.146 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:37.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.147 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:37.208 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:37.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.208 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:37.269 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:37.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.270 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:37.330 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:37.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.331 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:37.410 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:37.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.411 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:37.471 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:37.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.472 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:37.533 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:37.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.534 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:37.595 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:37.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.596 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:37.656 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:37.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.657 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:37.717 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:37.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.718 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:37.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:37.780 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-10-18T10:37:37.837 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1m 2023-10-18T10:37:37.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.979+0000 7f41a2fb0640 1 -- 172.21.15.125:0/3026015739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 msgr2=0x7f419c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:37.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.979+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/3026015739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 0x7f419c06b2e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4198009830 tx=0x7f419802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.980+0000 7f41a2fb0640 1 -- 172.21.15.125:0/3026015739 shutdown_connections 2023-10-18T10:37:37.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.980+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/3026015739 >> v2:172.21.15.139:3300/0 conn(0x7f419c0c96b0 0x7f419c0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.980+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/3026015739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 0x7f419c06b2e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.980+0000 7f41a2fb0640 1 -- 172.21.15.125:0/3026015739 >> 172.21.15.125:0/3026015739 conn(0x7f419c069930 msgr2=0x7f419c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:37.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.980+0000 7f41a2fb0640 1 -- 172.21.15.125:0/3026015739 shutdown_connections 2023-10-18T10:37:37.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.980+0000 7f41a2fb0640 1 -- 172.21.15.125:0/3026015739 wait complete. 2023-10-18T10:37:37.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.981+0000 7f41a2fb0640 1 Processor -- start 2023-10-18T10:37:37.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.981+0000 7f41a2fb0640 1 -- start start 2023-10-18T10:37:37.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a2fb0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 0x7f419c0aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:37.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a2fb0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c0ab500 0x7f419c13b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:37.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a2fb0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f419c0c96b0 0x7f419c0af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:37.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a2fb0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0b0b40 con 0x7f419c06af00 2023-10-18T10:37:37.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a2fb0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0afc90 con 0x7f419c0ab500 2023-10-18T10:37:37.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a2fb0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0aff30 con 0x7f419c0c96b0 2023-10-18T10:37:37.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 0x7f419c0aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:37.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 0x7f419c0aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52282/0 (socket says 172.21.15.125:52282) 2023-10-18T10:37:37.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 -- 172.21.15.125:0/4129881498 learned_addr learned my addr 172.21.15.125:0/4129881498 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:37.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f4193fff640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c0ab500 0x7f419c13b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:37.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a154d640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f419c0c96b0 0x7f419c0af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:37.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 -- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f419c0c96b0 msgr2=0x7f419c0af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:37.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f419c0c96b0 0x7f419c0af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 -- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c0ab500 msgr2=0x7f419c13b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:37.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c0ab500 0x7f419c13b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:37.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 -- 172.21.15.125:0/4129881498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4198009490 con 0x7f419c06af00 2023-10-18T10:37:37.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a0d4c640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 0x7f419c0aaf00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4198009780 tx=0x7f41980038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:37.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41917fa640 1 -- 172.21.15.125:0/4129881498 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198003ea0 con 0x7f419c06af00 2023-10-18T10:37:37.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41917fa640 1 -- 172.21.15.125:0/4129881498 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4198034550 con 0x7f419c06af00 2023-10-18T10:37:37.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c0b0210 con 0x7f419c06af00 2023-10-18T10:37:37.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41917fa640 1 -- 172.21.15.125:0/4129881498 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198034960 con 0x7f419c06af00 2023-10-18T10:37:37.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.982+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419c0b07d0 con 0x7f419c06af00 2023-10-18T10:37:37.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.983+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419c0690e0 con 0x7f419c06af00 2023-10-18T10:37:37.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.984+0000 7f41917fa640 1 -- 172.21.15.125:0/4129881498 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f419803b040 con 0x7f419c06af00 2023-10-18T10:37:37.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.984+0000 7f41917fa640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418806b070 0x7f418806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:37.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.984+0000 7f41917fa640 1 -- 172.21.15.125:0/4129881498 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6884+0+0 (secure 0 0 0) 0x7f41980ac250 con 0x7f419c06af00 2023-10-18T10:37:37.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.984+0000 7f4193fff640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418806b070 0x7f418806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:37.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.985+0000 7f4193fff640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418806b070 0x7f418806d530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f419c1f2920 tx=0x7f418c00b2e0 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:37.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:37.989+0000 7f41917fa640 1 -- 172.21.15.125:0/4129881498 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4198076c90 con 0x7f419c06af00 2023-10-18T10:37:38.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.173+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 --> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f419c159c50 con 0x7f418806b070 2023-10-18T10:37:38.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.235+0000 7f41917fa640 1 -- 172.21.15.125:0/4129881498 <== mgr.5063 v2:172.21.15.139:6832/609182116 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f418c015070 con 0x7f418806b070 2023-10-18T10:37:38.238 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-10-18T10:37:38.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.237+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418806b070 msgr2=0x7f418806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:38.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.237+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418806b070 0x7f418806d530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f419c1f2920 tx=0x7f418c00b2e0 comp rx=0 tx=0).stop 2023-10-18T10:37:38.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.237+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 msgr2=0x7f419c0aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:38.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.237+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 0x7f419c0aaf00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4198009780 tx=0x7f41980038a0 comp rx=0 tx=0).stop 2023-10-18T10:37:38.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.238+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 shutdown_connections 2023-10-18T10:37:38.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.238+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f418806b070 0x7f418806d530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:38.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.238+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f419c0ab500 0x7f419c13b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:38.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.238+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f419c0c96b0 0x7f419c0af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:38.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.238+0000 7f41a2fb0640 1 --2- 172.21.15.125:0/4129881498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f419c06af00 0x7f419c0aaf00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:38.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.238+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 >> 172.21.15.125:0/4129881498 conn(0x7f419c069930 msgr2=0x7f419c15c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:38.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.238+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 shutdown_connections 2023-10-18T10:37:38.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:38.238+0000 7f41a2fb0640 1 -- 172.21.15.125:0/4129881498 wait complete. 2023-10-18T10:37:38.249 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-10-18T10:37:38.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.250 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-18T10:37:38.272 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:38.272 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:38.272 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:38.273 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:38.273 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:38.273 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:38.273 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:38.273 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:38.274 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:38.274 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:38.274 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:38.274 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_27_00_UTC 2023-10-18T10:37:38.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.275 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_27_00_UTC) 2023-10-18T10:37:38.343 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_28_00_UTC 2023-10-18T10:37:38.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.344 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_28_00_UTC) 2023-10-18T10:37:38.405 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_29_00_UTC 2023-10-18T10:37:38.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.406 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_29_00_UTC) 2023-10-18T10:37:38.474 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_30_00_UTC 2023-10-18T10:37:38.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.475 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_30_00_UTC) 2023-10-18T10:37:38.536 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_31_00_UTC 2023-10-18T10:37:38.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.537 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_31_00_UTC) 2023-10-18T10:37:38.598 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_32_00_UTC 2023-10-18T10:37:38.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.599 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_32_00_UTC) 2023-10-18T10:37:38.661 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_33_00_UTC 2023-10-18T10:37:38.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.663 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_33_00_UTC) 2023-10-18T10:37:38.723 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_34_00_UTC 2023-10-18T10:37:38.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.724 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_34_00_UTC) 2023-10-18T10:37:38.787 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_35_00_UTC 2023-10-18T10:37:38.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.788 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_35_00_UTC) 2023-10-18T10:37:38.848 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_36_00_UTC 2023-10-18T10:37:38.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.849 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_36_00_UTC) 2023-10-18T10:37:38.912 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_37_00_UTC 2023-10-18T10:37:38.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.913 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-18-10_37_00_UTC) 2023-10-18T10:37:38.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:38.974 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-10-18T10:37:39.031 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T10:37:39.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.173+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/551335406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d406af00 msgr2=0x7fe8d406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:39.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.173+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/551335406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d406af00 0x7fe8d406b2e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009900 tx=0x7fe8c402be00 comp rx=0 tx=0).stop 2023-10-18T10:37:39.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.173+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/551335406 shutdown_connections 2023-10-18T10:37:39.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.173+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/551335406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8d40c96b0 0x7fe8d40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:39.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.173+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/551335406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d406af00 0x7fe8d406b2e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:39.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.173+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/551335406 >> 172.21.15.125:0/551335406 conn(0x7fe8d4069930 msgr2=0x7fe8d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:39.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.173+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/551335406 shutdown_connections 2023-10-18T10:37:39.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.173+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/551335406 wait complete. 2023-10-18T10:37:39.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.175+0000 7fe8d8dc9640 1 Processor -- start 2023-10-18T10:37:39.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.175+0000 7fe8d8dc9640 1 -- start start 2023-10-18T10:37:39.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.175+0000 7fe8d8dc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8d406af00 0x7fe8d413d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:39.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d8dc9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8d40ab500 0x7fe8d40aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:39.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d8dc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d40c96b0 0x7fe8d4139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:39.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d8dc9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40a8080 con 0x7fe8d40c96b0 2023-10-18T10:37:39.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d8dc9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40a82d0 con 0x7fe8d406af00 2023-10-18T10:37:39.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d8dc9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40a85b0 con 0x7fe8d40ab500 2023-10-18T10:37:39.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d40c96b0 0x7fe8d4139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:39.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d40c96b0 0x7fe8d4139e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52312/0 (socket says 172.21.15.125:52312) 2023-10-18T10:37:39.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 -- 172.21.15.125:0/3338322436 learned_addr learned my addr 172.21.15.125:0/3338322436 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:39.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8c9d9b640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8d40ab500 0x7fe8d40aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:39.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d259c640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8d406af00 0x7fe8d413d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:39.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 -- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8d40ab500 msgr2=0x7fe8d40aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:39.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8d40ab500 0x7fe8d40aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:39.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 -- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8d406af00 msgr2=0x7fe8d413d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:39.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8d406af00 0x7fe8d413d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:39.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 -- 172.21.15.125:0/3338322436 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c4009560 con 0x7fe8d40c96b0 2023-10-18T10:37:39.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d2d9d640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d40c96b0 0x7fe8d4139e90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc001560 tx=0x7fe8cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:39.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8cb7fe640 1 -- 172.21.15.125:0/3338322436 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc003da0 con 0x7fe8d40c96b0 2023-10-18T10:37:39.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8cb7fe640 1 -- 172.21.15.125:0/3338322436 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8cc005070 con 0x7fe8d40c96b0 2023-10-18T10:37:39.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d40a8890 con 0x7fe8d40c96b0 2023-10-18T10:37:39.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8cb7fe640 1 -- 172.21.15.125:0/3338322436 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc005460 con 0x7fe8d40c96b0 2023-10-18T10:37:39.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.176+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d413dbb0 con 0x7fe8d40c96b0 2023-10-18T10:37:39.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.177+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d40a8db0 con 0x7fe8d40c96b0 2023-10-18T10:37:39.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.178+0000 7fe8cb7fe640 1 -- 172.21.15.125:0/3338322436 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fe8cc0055c0 con 0x7fe8d40c96b0 2023-10-18T10:37:39.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.178+0000 7fe8cb7fe640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe8b406b040 0x7fe8b406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:39.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.178+0000 7fe8cb7fe640 1 -- 172.21.15.125:0/3338322436 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7440+0+0 (secure 0 0 0) 0x7fe8cc089bd0 con 0x7fe8d40c96b0 2023-10-18T10:37:39.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.178+0000 7fe8d259c640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe8b406b040 0x7fe8b406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:39.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.179+0000 7fe8d259c640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe8b406b040 0x7fe8b406d500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4035000 tx=0x7fe8c4004730 comp rx=0 tx=0).ready entity=mgr.5063 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:39.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.182+0000 7fe8cb7fe640 1 -- 172.21.15.125:0/3338322436 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8cc054340 con 0x7fe8d40c96b0 2023-10-18T10:37:39.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:39.406+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fe8d406c500 con 0x7fe8d40c96b0 2023-10-18T10:37:40.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.375+0000 7fe8cb7fe640 1 -- 172.21.15.125:0/3338322436 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v16) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe8cc087e60 con 0x7fe8d40c96b0 2023-10-18T10:37:40.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe8b406b040 msgr2=0x7fe8b406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:40.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe8b406b040 0x7fe8b406d500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4035000 tx=0x7fe8c4004730 comp rx=0 tx=0).stop 2023-10-18T10:37:40.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d40c96b0 msgr2=0x7fe8d4139e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:40.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d40c96b0 0x7fe8d4139e90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc001560 tx=0x7fe8cc00cc70 comp rx=0 tx=0).stop 2023-10-18T10:37:40.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 shutdown_connections 2023-10-18T10:37:40.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fe8b406b040 0x7fe8b406d500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8d40c96b0 0x7fe8d4139e90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8d40ab500 0x7fe8d40aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 --2- 172.21.15.125:0/3338322436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8d406af00 0x7fe8d413d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 >> 172.21.15.125:0/3338322436 conn(0x7fe8d4069930 msgr2=0x7fe8d413b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:40.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 shutdown_connections 2023-10-18T10:37:40.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.378+0000 7fe8d8dc9640 1 -- 172.21.15.125:0/3338322436 wait complete. 2023-10-18T10:37:40.394 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:37:40.395 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:37:40.435 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:37:40.484 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:37:40.484 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:37:40.485 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:37:40.501 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:37:40.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.663+0000 7f99adc7c640 1 -- 172.21.15.125:0/2449009770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80cf8b0 msgr2=0x7f99a80ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:40.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.663+0000 7f99adc7c640 1 --2- 172.21.15.125:0/2449009770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80cf8b0 0x7f99a80ce9a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f999c009830 tx=0x7f999c02bd60 comp rx=0 tx=0).stop 2023-10-18T10:37:40.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.663+0000 7f99adc7c640 1 -- 172.21.15.125:0/2449009770 shutdown_connections 2023-10-18T10:37:40.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.663+0000 7f99adc7c640 1 --2- 172.21.15.125:0/2449009770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80cf8b0 0x7f99a80ce9a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.663+0000 7f99adc7c640 1 -- 172.21.15.125:0/2449009770 >> 172.21.15.125:0/2449009770 conn(0x7f99a80d0c50 msgr2=0x7f99a80d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:40.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.663+0000 7f99adc7c640 1 -- 172.21.15.125:0/2449009770 shutdown_connections 2023-10-18T10:37:40.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.663+0000 7f99adc7c640 1 -- 172.21.15.125:0/2449009770 wait complete. 2023-10-18T10:37:40.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.665+0000 7f99adc7c640 1 Processor -- start 2023-10-18T10:37:40.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.665+0000 7f99adc7c640 1 -- start start 2023-10-18T10:37:40.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99adc7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80b1ca0 0x7f99a80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:40.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99adc7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a80cf8b0 0x7f99a80b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:40.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99adc7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a815e5e0 0x7f99a80c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:40.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99adc7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80b08b0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99adc7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80b00f0 con 0x7f99a815e5e0 2023-10-18T10:37:40.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99adc7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80ad9e0 con 0x7f99a80cf8b0 2023-10-18T10:37:40.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80b1ca0 0x7f99a80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:40.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80b1ca0 0x7f99a80b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52356/0 (socket says 172.21.15.125:52356) 2023-10-18T10:37:40.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 -- 172.21.15.125:0/541961207 learned_addr learned my addr 172.21.15.125:0/541961207 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:40.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a6ffd640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a80cf8b0 0x7f99a80b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:40.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a7fff640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a815e5e0 0x7f99a80c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:40.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 -- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a80cf8b0 msgr2=0x7f99a80b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:40.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a80cf8b0 0x7f99a80b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 -- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a815e5e0 msgr2=0x7f99a80c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:40.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a815e5e0 0x7f99a80c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 -- 172.21.15.125:0/541961207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c009490 con 0x7f99a80b1ca0 2023-10-18T10:37:40.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.666+0000 7f99a77fe640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80b1ca0 0x7f99a80b26d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f999800c3c0 tx=0x7f999800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:40.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.667+0000 7f998ffff640 1 -- 172.21.15.125:0/541961207 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980035f0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.667+0000 7f998ffff640 1 -- 172.21.15.125:0/541961207 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9998003770 con 0x7f99a80b1ca0 2023-10-18T10:37:40.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.667+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a80adcc0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.667+0000 7f998ffff640 1 -- 172.21.15.125:0/541961207 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998003b80 con 0x7f99a80b1ca0 2023-10-18T10:37:40.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.667+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a80adfa0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.668+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a80b05d0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.668+0000 7f998ffff640 1 -- 172.21.15.125:0/541961207 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f9998003ce0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.669+0000 7f998ffff640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f999406b020 0x7f999406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:40.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.669+0000 7f99a6ffd640 1 -- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f999406b020 msgr2=0x7f999406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:40.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.669+0000 7f998ffff640 1 -- 172.21.15.125:0/541961207 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f99980893b0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.669+0000 7f99a6ffd640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f999406b020 0x7f999406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:40.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.673+0000 7f998ffff640 1 -- 172.21.15.125:0/541961207 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99a80b05d0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.809 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:40.806+0000 7f73572b2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:37:40.819 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:40.817+0000 7fe78142b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:37:40.854 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:40.851+0000 7f73572b2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:37:40.864 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:40.862+0000 7fe78142b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:37:40.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.870+0000 7f99a6ffd640 1 -- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f999406b020 msgr2=0x7f999406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:40.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.870+0000 7f99a6ffd640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f999406b020 0x7f999406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:37:40.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.875+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99a80b05d0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.876+0000 7f998ffff640 1 -- 172.21.15.125:0/541961207 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18298 (secure 0 0 0) 0x7f99a80b05d0 con 0x7f99a80b1ca0 2023-10-18T10:37:40.878 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:40.878 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":51,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:40.328395+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:26:16.213313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-18T10:26:16.801795+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3880096480":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/1401100783":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/2631710418":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/557166225":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6833/2976686360":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6832/2976686360":"2023-10-19T10:26:33.551736+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":4,"length":2},{"begin":8,"length":1},{"begin":10,"length":3},{"begin":15,"length":2},{"begin":19,"length":1},{"begin":21,"length":3},{"begin":26,"length":2},{"begin":30,"length":1},{"begin":32,"length":2},{"begin":36,"length":2},{"begin":39,"length":2},{"begin":43,"length":2},{"begin":47,"length":2},{"begin":50,"length":2},{"begin":53,"length":2},{"begin":57,"length":2},{"begin":60,"length":2},{"begin":63,"length":2},{"begin":67,"length":2},{"begin":70,"length":2},{"begin":73,"length":2},{"begin":77,"length":2},{"begin":80,"length":3},{"begin":84,"length":2},{"begin":88,"length":2},{"begin":91,"length":3},{"begin":95,"length":2},{"begin":99,"length":2},{"begin":103,"length":2},{"begin":106,"length":2},{"begin":110,"length":1},{"begin":159,"length":62}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":16,"length":1},{"begin":27,"length":1},{"begin":37,"length":1},{"begin":48,"length":1},{"begin":58,"length":1},{"begin":68,"length":1},{"begin":78,"length":1},{"begin":89,"length":1},{"begin":100,"length":1},{"begin":211,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:37:40.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f999406b020 msgr2=0x7f999406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:40.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f999406b020 0x7f999406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80b1ca0 msgr2=0x7f99a80b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:40.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80b1ca0 0x7f99a80b26d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f999800c3c0 tx=0x7f999800c880 comp rx=0 tx=0).stop 2023-10-18T10:37:40.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 shutdown_connections 2023-10-18T10:37:40.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f999406b020 0x7f999406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a815e5e0 0x7f99a80c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a80cf8b0 0x7f99a80b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 --2- 172.21.15.125:0/541961207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99a80b1ca0 0x7f99a80b26d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:40.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 >> 172.21.15.125:0/541961207 conn(0x7f99a80d0c50 msgr2=0x7f99a815d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:40.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 shutdown_connections 2023-10-18T10:37:40.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.878+0000 7f99adc7c640 1 -- 172.21.15.125:0/541961207 wait complete. 2023-10-18T10:37:40.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:26:16.213313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:26:16.801795+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:37:40.892 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:37:41.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.998+0000 7faea8801640 1 -- 172.21.15.125:0/3872863603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea40705d0 msgr2=0x7faea40cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.998+0000 7faea8801640 1 --2- 172.21.15.125:0/3872863603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea40705d0 0x7faea40cc240 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005fd0 tx=0x7fae9c02c2e0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.998+0000 7faea8801640 1 -- 172.21.15.125:0/3872863603 shutdown_connections 2023-10-18T10:37:41.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.998+0000 7faea8801640 1 --2- 172.21.15.125:0/3872863603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea40705d0 0x7faea40cc240 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.998+0000 7faea8801640 1 --2- 172.21.15.125:0/3872863603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faea40b0870 0x7faea40cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.998+0000 7faea8801640 1 -- 172.21.15.125:0/3872863603 >> 172.21.15.125:0/3872863603 conn(0x7faea406a830 msgr2=0x7faea406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:41.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.998+0000 7faea8801640 1 -- 172.21.15.125:0/3872863603 shutdown_connections 2023-10-18T10:37:41.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.998+0000 7faea8801640 1 -- 172.21.15.125:0/3872863603 wait complete. 2023-10-18T10:37:41.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.999+0000 7faea8801640 1 Processor -- start 2023-10-18T10:37:41.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:40.999+0000 7faea8801640 1 -- start start 2023-10-18T10:37:41.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea8801640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea40705d0 0x7faea414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea8801640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea41589f0 0x7faea415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea8801640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faea415b7b0 0x7faea4151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea8801640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea4152e10 con 0x7faea41589f0 2023-10-18T10:37:41.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea8801640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea41530f0 con 0x7faea40705d0 2023-10-18T10:37:41.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea8801640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea41533d0 con 0x7faea415b7b0 2023-10-18T10:37:41.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea41589f0 0x7faea415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea41589f0 0x7faea415b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52382/0 (socket says 172.21.15.125:52382) 2023-10-18T10:37:41.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea259c640 1 -- 172.21.15.125:0/1062936526 learned_addr learned my addr 172.21.15.125:0/1062936526 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:41.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea359e640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faea415b7b0 0x7faea4151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea2d9d640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea40705d0 0x7faea414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea259c640 1 -- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faea415b7b0 msgr2=0x7faea4151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea259c640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faea415b7b0 0x7faea4151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea259c640 1 -- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea40705d0 msgr2=0x7faea414f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea259c640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea40705d0 0x7faea414f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea259c640 1 -- 172.21.15.125:0/1062936526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c009900 con 0x7faea41589f0 2023-10-18T10:37:41.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.001+0000 7faea359e640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faea415b7b0 0x7faea4151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:37:41.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.000+0000 7faea2d9d640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea40705d0 0x7faea414f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:37:41.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.001+0000 7faea259c640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea41589f0 0x7faea415b270 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d950 tx=0x7fae9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:41.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.001+0000 7fae8b7fe640 1 -- 172.21.15.125:0/1062936526 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94009c30 con 0x7faea41589f0 2023-10-18T10:37:41.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.001+0000 7fae8b7fe640 1 -- 172.21.15.125:0/1062936526 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae94009e80 con 0x7faea41589f0 2023-10-18T10:37:41.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.001+0000 7fae8b7fe640 1 -- 172.21.15.125:0/1062936526 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94004280 con 0x7faea41589f0 2023-10-18T10:37:41.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.001+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea41536b0 con 0x7faea41589f0 2023-10-18T10:37:41.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.001+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea4152340 con 0x7faea41589f0 2023-10-18T10:37:41.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.002+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea404da70 con 0x7faea41589f0 2023-10-18T10:37:41.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.003+0000 7fae8b7fe640 1 -- 172.21.15.125:0/1062936526 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fae940035f0 con 0x7faea41589f0 2023-10-18T10:37:41.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.003+0000 7fae8b7fe640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fae9006aff0 0x7fae9006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.003+0000 7faea2d9d640 1 -- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fae9006aff0 msgr2=0x7fae9006d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:41.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.003+0000 7fae8b7fe640 1 -- 172.21.15.125:0/1062936526 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7028+0+0 (secure 0 0 0) 0x7fae9408a0f0 con 0x7faea41589f0 2023-10-18T10:37:41.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.003+0000 7faea2d9d640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fae9006aff0 0x7fae9006d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:41.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.008+0000 7fae8b7fe640 1 -- 172.21.15.125:0/1062936526 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae940549a0 con 0x7faea41589f0 2023-10-18T10:37:41.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.180+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faea406a050 con 0x7faea41589f0 2023-10-18T10:37:41.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.180+0000 7fae8b7fe640 1 -- 172.21.15.125:0/1062936526 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae940200a0 con 0x7faea41589f0 2023-10-18T10:37:41.183 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:37:41.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fae9006aff0 msgr2=0x7fae9006d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:41.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fae9006aff0 0x7fae9006d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea41589f0 msgr2=0x7faea415b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea41589f0 0x7faea415b270 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d950 tx=0x7fae9400de10 comp rx=0 tx=0).stop 2023-10-18T10:37:41.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 shutdown_connections 2023-10-18T10:37:41.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fae9006aff0 0x7fae9006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faea415b7b0 0x7faea4151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faea41589f0 0x7faea415b270 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 --2- 172.21.15.125:0/1062936526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faea40705d0 0x7faea414f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.182+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 >> 172.21.15.125:0/1062936526 conn(0x7faea406a830 msgr2=0x7faea406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:41.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.183+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 shutdown_connections 2023-10-18T10:37:41.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.183+0000 7faea8801640 1 -- 172.21.15.125:0/1062936526 wait complete. 2023-10-18T10:37:41.195 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:37:41.243 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.240+0000 7f73572b2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:37:41.275 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.274+0000 7fe78142b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:37:41.287 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.284+0000 7f73572b2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:37:41.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.300+0000 7f3525f8a640 1 -- 172.21.15.125:0/327733999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 msgr2=0x7f352006ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.300+0000 7f3525f8a640 1 --2- 172.21.15.125:0/327733999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 0x7f352006ef80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f35200711a0 tx=0x7f351402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.300+0000 7f3525f8a640 1 -- 172.21.15.125:0/327733999 shutdown_connections 2023-10-18T10:37:41.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.300+0000 7f3525f8a640 1 --2- 172.21.15.125:0/327733999 >> v2:172.21.15.139:3300/0 conn(0x7f35200c9490 0x7f352006e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.300+0000 7f3525f8a640 1 --2- 172.21.15.125:0/327733999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 0x7f352006ef80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.300+0000 7f3525f8a640 1 -- 172.21.15.125:0/327733999 >> 172.21.15.125:0/327733999 conn(0x7f35200b06e0 msgr2=0x7f35200b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:41.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.300+0000 7f3525f8a640 1 -- 172.21.15.125:0/327733999 shutdown_connections 2023-10-18T10:37:41.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.300+0000 7f3525f8a640 1 -- 172.21.15.125:0/327733999 wait complete. 2023-10-18T10:37:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.301+0000 7f3525f8a640 1 Processor -- start 2023-10-18T10:37:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.302+0000 7f3525f8a640 1 -- start start 2023-10-18T10:37:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.302+0000 7f3525f8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f352006d2b0 0x7f3520152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.302+0000 7f3525f8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35200c9490 0x7f35200b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.302+0000 7f3525f8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 0x7f3520159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.302+0000 7f3525f8a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35200b30a0 con 0x7f352015c5b0 2023-10-18T10:37:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.302+0000 7f3525f8a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3520153430 con 0x7f352006d2b0 2023-10-18T10:37:41.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.302+0000 7f3525f8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3520152e90 con 0x7f35200c9490 2023-10-18T10:37:41.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.302+0000 7f351ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 0x7f3520159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 0x7f3520159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52398/0 (socket says 172.21.15.125:52398) 2023-10-18T10:37:41.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351ffff640 1 -- 172.21.15.125:0/709071589 learned_addr learned my addr 172.21.15.125:0/709071589 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:41.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351f7fe640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f352006d2b0 0x7f3520152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351effd640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35200c9490 0x7f35200b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351ffff640 1 -- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35200c9490 msgr2=0x7f35200b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351ffff640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35200c9490 0x7f35200b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351ffff640 1 -- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f352006d2b0 msgr2=0x7f3520152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351ffff640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f352006d2b0 0x7f3520152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351ffff640 1 -- 172.21.15.125:0/709071589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514009490 con 0x7f352015c5b0 2023-10-18T10:37:41.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f351ffff640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 0x7f3520159a80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f351800cbf0 tx=0x7f351800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:41.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f3507fff640 1 -- 172.21.15.125:0/709071589 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351800aca0 con 0x7f352015c5b0 2023-10-18T10:37:41.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f3507fff640 1 -- 172.21.15.125:0/709071589 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3518003970 con 0x7f352015c5b0 2023-10-18T10:37:41.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35200b2510 con 0x7f352015c5b0 2023-10-18T10:37:41.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f3507fff640 1 -- 172.21.15.125:0/709071589 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518003d80 con 0x7f352015c5b0 2023-10-18T10:37:41.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.303+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35200b29e0 con 0x7f352015c5b0 2023-10-18T10:37:41.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.305+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35200cc510 con 0x7f352015c5b0 2023-10-18T10:37:41.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.305+0000 7f3507fff640 1 -- 172.21.15.125:0/709071589 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f3518004030 con 0x7f352015c5b0 2023-10-18T10:37:41.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.305+0000 7f3507fff640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f350c06aff0 0x7f350c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.305+0000 7f351f7fe640 1 -- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f350c06aff0 msgr2=0x7f350c06d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:41.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.305+0000 7f3507fff640 1 -- 172.21.15.125:0/709071589 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f351808a820 con 0x7f352015c5b0 2023-10-18T10:37:41.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.305+0000 7f351f7fe640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f350c06aff0 0x7f350c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:41.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.309+0000 7f3507fff640 1 -- 172.21.15.125:0/709071589 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35180551e0 con 0x7f352015c5b0 2023-10-18T10:37:41.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.327+0000 7f73572b2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:37:41.330 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.329+0000 7fe78142b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:37:41.381 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.379+0000 7f73572b2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:37:41.386 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.384+0000 7fe78142b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:37:41.423 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.420+0000 7f73572b2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:37:41.439 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.438+0000 7fe78142b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:37:41.482 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.480+0000 7fe78142b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:37:41.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.506+0000 7f351f7fe640 1 -- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f350c06aff0 msgr2=0x7f350c06d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:41.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.506+0000 7f351f7fe640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f350c06aff0 0x7f350c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:37:41.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.507+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35200b31e0 con 0x7f352015c5b0 2023-10-18T10:37:41.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.508+0000 7f3507fff640 1 -- 172.21.15.125:0/709071589 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f35200b31e0 con 0x7f352015c5b0 2023-10-18T10:37:41.510 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:37:41.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f350c06aff0 msgr2=0x7f350c06d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:41.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f350c06aff0 0x7f350c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 msgr2=0x7f3520159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 0x7f3520159a80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f351800cbf0 tx=0x7f351800a060 comp rx=0 tx=0).stop 2023-10-18T10:37:41.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 shutdown_connections 2023-10-18T10:37:41.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f350c06aff0 0x7f350c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f352015c5b0 0x7f3520159a80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35200c9490 0x7f35200b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 --2- 172.21.15.125:0/709071589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f352006d2b0 0x7f3520152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 >> 172.21.15.125:0/709071589 conn(0x7f35200b06e0 msgr2=0x7f35200abbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:41.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 shutdown_connections 2023-10-18T10:37:41.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.510+0000 7f3525f8a640 1 -- 172.21.15.125:0/709071589 wait complete. 2023-10-18T10:37:41.520 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.518+0000 7f73572b2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:37:41.522 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:37:41.573 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.570+0000 7f73572b2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:37:41.585 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.583+0000 7fe78142b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:37:41.615 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.612+0000 7f73572b2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:37:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.628+0000 7f696c2da640 1 -- 172.21.15.125:0/1259344958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f696415b2a0 msgr2=0x7f69640cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.628+0000 7f696c2da640 1 --2- 172.21.15.125:0/1259344958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f696415b2a0 0x7f69640cf720 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6954005fd0 tx=0x7f695402bd90 comp rx=0 tx=0).stop 2023-10-18T10:37:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.628+0000 7f696c2da640 1 -- 172.21.15.125:0/1259344958 shutdown_connections 2023-10-18T10:37:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.628+0000 7f696c2da640 1 --2- 172.21.15.125:0/1259344958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f696415b2a0 0x7f69640cf720 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.628+0000 7f696c2da640 1 --2- 172.21.15.125:0/1259344958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69640cf340 0x7f69640d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.628+0000 7f696c2da640 1 -- 172.21.15.125:0/1259344958 >> 172.21.15.125:0/1259344958 conn(0x7f69640d0d60 msgr2=0x7f69640d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:41.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.628+0000 7f696c2da640 1 -- 172.21.15.125:0/1259344958 shutdown_connections 2023-10-18T10:37:41.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.628+0000 7f696c2da640 1 -- 172.21.15.125:0/1259344958 wait complete. 2023-10-18T10:37:41.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.629+0000 7f696c2da640 1 Processor -- start 2023-10-18T10:37:41.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.629+0000 7f696c2da640 1 -- start start 2023-10-18T10:37:41.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696c2da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69640ceb40 0x7f696415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696c2da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69640cf340 0x7f69640cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696c2da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f696415b2a0 0x7f69641f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696c2da640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69640cb0c0 con 0x7f69640ceb40 2023-10-18T10:37:41.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696c2da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696415d880 con 0x7f696415b2a0 2023-10-18T10:37:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696c2da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696415db60 con 0x7f69640cf340 2023-10-18T10:37:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69640ceb40 0x7f696415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69640ceb40 0x7f696415d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52404/0 (socket says 172.21.15.125:52404) 2023-10-18T10:37:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 -- 172.21.15.125:0/2484788491 learned_addr learned my addr 172.21.15.125:0/2484788491 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:41.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f6969875640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69640cf340 0x7f69640cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a877640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f696415b2a0 0x7f69641f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 -- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69640cf340 msgr2=0x7f69640cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69640cf340 0x7f69640cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 -- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f696415b2a0 msgr2=0x7f69641f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f696415b2a0 0x7f69641f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 -- 172.21.15.125:0/2484788491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954009560 con 0x7f69640ceb40 2023-10-18T10:37:41.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a877640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f696415b2a0 0x7f69641f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:41.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696a076640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69640ceb40 0x7f696415d230 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f696000e9f0 tx=0x7f696000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:41.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f695affd640 1 -- 172.21.15.125:0/2484788491 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960009700 con 0x7f69640ceb40 2023-10-18T10:37:41.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f695affd640 1 -- 172.21.15.125:0/2484788491 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f696000ce70 con 0x7f69640ceb40 2023-10-18T10:37:41.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696415de40 con 0x7f69640ceb40 2023-10-18T10:37:41.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f695affd640 1 -- 172.21.15.125:0/2484788491 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960010670 con 0x7f69640ceb40 2023-10-18T10:37:41.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.630+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69641f1430 con 0x7f69640ceb40 2023-10-18T10:37:41.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.631+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69640690e0 con 0x7f69640ceb40 2023-10-18T10:37:41.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.632+0000 7f695affd640 1 -- 172.21.15.125:0/2484788491 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f69600107d0 con 0x7f69640ceb40 2023-10-18T10:37:41.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.632+0000 7f695affd640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f694806aff0 0x7f694806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.632+0000 7f6969875640 1 -- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f694806aff0 msgr2=0x7f694806d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:41.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.633+0000 7f695affd640 1 -- 172.21.15.125:0/2484788491 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f6960014070 con 0x7f69640ceb40 2023-10-18T10:37:41.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.633+0000 7f6969875640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f694806aff0 0x7f694806d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:41.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.637+0000 7f695affd640 1 -- 172.21.15.125:0/2484788491 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6960051ca0 con 0x7f69640ceb40 2023-10-18T10:37:41.642 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.641+0000 7fe78142b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:37:41.664 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.662+0000 7f73572b2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:37:41.687 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.685+0000 7fe78142b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:37:41.709 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:41.704+0000 7f73572b2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:37:41.737 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.735+0000 7fe78142b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:37:41.781 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:41.779+0000 7fe78142b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:37:41.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.811+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6964147d80 con 0x7f69640ceb40 2023-10-18T10:37:41.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.811+0000 7f695affd640 1 -- 172.21.15.125:0/2484788491 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6960058ef0 con 0x7f69640ceb40 2023-10-18T10:37:41.813 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:37:41.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f694806aff0 msgr2=0x7f694806d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:41.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f694806aff0 0x7f694806d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69640ceb40 msgr2=0x7f696415d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69640ceb40 0x7f696415d230 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f696000e9f0 tx=0x7f696000eeb0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 shutdown_connections 2023-10-18T10:37:41.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f694806aff0 0x7f694806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f696415b2a0 0x7f69641f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69640cf340 0x7f69640cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 --2- 172.21.15.125:0/2484788491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69640ceb40 0x7f696415d230 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 >> 172.21.15.125:0/2484788491 conn(0x7f69640d0d60 msgr2=0x7f69640aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:41.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 shutdown_connections 2023-10-18T10:37:41.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.813+0000 7f696c2da640 1 -- 172.21.15.125:0/2484788491 wait complete. 2023-10-18T10:37:41.826 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:37:41.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.929+0000 7f99dab1d640 1 -- 172.21.15.125:0/763997888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 msgr2=0x7f99c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.929+0000 7f99dab1d640 1 --2- 172.21.15.125:0/763997888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 0x7f99c400b980 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00c640 tx=0x7f99c40057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.929+0000 7f99dab1d640 1 -- 172.21.15.125:0/763997888 shutdown_connections 2023-10-18T10:37:41.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.929+0000 7f99dab1d640 1 --2- 172.21.15.125:0/763997888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 0x7f99c400b980 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.929+0000 7f99dab1d640 1 -- 172.21.15.125:0/763997888 >> 172.21.15.125:0/763997888 conn(0x7f99d40d0d60 msgr2=0x7f99d40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:41.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.929+0000 7f99dab1d640 1 -- 172.21.15.125:0/763997888 shutdown_connections 2023-10-18T10:37:41.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.929+0000 7f99dab1d640 1 -- 172.21.15.125:0/763997888 wait complete. 2023-10-18T10:37:41.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.930+0000 7f99dab1d640 1 Processor -- start 2023-10-18T10:37:41.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.931+0000 7f99dab1d640 1 -- start start 2023-10-18T10:37:41.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.931+0000 7f99dab1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 0x7f99d4148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.931+0000 7f99dab1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d40c8060 0x7f99d40c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.931+0000 7f99dab1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d40b0ea0 0x7f99d40b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.931+0000 7f99dab1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d40c8a20 con 0x7f99c4005180 2023-10-18T10:37:41.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.931+0000 7f99dab1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d4148bc0 con 0x7f99d40b0ea0 2023-10-18T10:37:41.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.931+0000 7f99dab1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d40b1860 con 0x7f99d40c8060 2023-10-18T10:37:41.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 0x7f99d4148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 0x7f99d4148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52420/0 (socket says 172.21.15.125:52420) 2023-10-18T10:37:41.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 -- 172.21.15.125:0/3974604613 learned_addr learned my addr 172.21.15.125:0/3974604613 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:41.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d3fff640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d40c8060 0x7f99d40c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d90ba640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d40b0ea0 0x7f99d40b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:41.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 -- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d40c8060 msgr2=0x7f99d40c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d40c8060 0x7f99d40c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 -- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d40b0ea0 msgr2=0x7f99d40b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:41.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d40b0ea0 0x7f99d40b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:41.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 -- 172.21.15.125:0/3974604613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d4148e10 con 0x7f99c4005180 2023-10-18T10:37:41.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d3fff640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d40c8060 0x7f99d40c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:37:41.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d88b9640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 0x7f99d4148490 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f99c8011ab0 tx=0x7f99c8011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:41.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d17fa640 1 -- 172.21.15.125:0/3974604613 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8004460 con 0x7f99c4005180 2023-10-18T10:37:41.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d17fa640 1 -- 172.21.15.125:0/3974604613 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f99c800f9a0 con 0x7f99c4005180 2023-10-18T10:37:41.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d41490f0 con 0x7f99c4005180 2023-10-18T10:37:41.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99d17fa640 1 -- 172.21.15.125:0/3974604613 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c800fd90 con 0x7f99c4005180 2023-10-18T10:37:41.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.932+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d41496b0 con 0x7f99c4005180 2023-10-18T10:37:41.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.933+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d40690e0 con 0x7f99c4005180 2023-10-18T10:37:41.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.934+0000 7f99d17fa640 1 -- 172.21.15.125:0/3974604613 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f99c80035f0 con 0x7f99c4005180 2023-10-18T10:37:41.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.935+0000 7f99d17fa640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f99b806b020 0x7f99b806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:41.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.935+0000 7f99d3fff640 1 -- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f99b806b020 msgr2=0x7f99b806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:41.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.935+0000 7f99d17fa640 1 -- 172.21.15.125:0/3974604613 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f99c808b210 con 0x7f99c4005180 2023-10-18T10:37:41.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.935+0000 7f99d3fff640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f99b806b020 0x7f99b806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:41.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:41.938+0000 7f99d17fa640 1 -- 172.21.15.125:0/3974604613 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99c8055cd0 con 0x7f99c4005180 2023-10-18T10:37:42.022 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.020+0000 7f73572b2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:37:42.112 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.110+0000 7fe78142b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:37:42.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.124+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f99d40af990 con 0x7f99c4005180 2023-10-18T10:37:42.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.124+0000 7f99d17fa640 1 -- 172.21.15.125:0/3974604613 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v51) v1 ==== 80+0+359 (secure 0 0 0) 0x7f99c8028020 con 0x7f99c4005180 2023-10-18T10:37:42.127 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3880096480 2023-10-19T10:26:33.551736+0000 2023-10-18T10:37:42.127 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1401100783 2023-10-19T10:26:33.551736+0000 2023-10-18T10:37:42.127 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2631710418 2023-10-19T10:26:33.551736+0000 2023-10-18T10:37:42.127 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/557166225 2023-10-19T10:26:33.551736+0000 2023-10-18T10:37:42.128 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/2976686360 2023-10-19T10:26:33.551736+0000 2023-10-18T10:37:42.128 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/2976686360 2023-10-19T10:26:33.551736+0000 2023-10-18T10:37:42.129 INFO:teuthology.orchestra.run.smithi125.stderr:listed 6 entries 2023-10-18T10:37:42.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.126+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f99b806b020 msgr2=0x7f99b806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:42.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.126+0000 7f99dab1d640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f99b806b020 0x7f99b806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.126+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 msgr2=0x7f99d4148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:42.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.126+0000 7f99dab1d640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 0x7f99d4148490 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f99c8011ab0 tx=0x7f99c8011eb0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.127+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 shutdown_connections 2023-10-18T10:37:42.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.127+0000 7f99dab1d640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f99b806b020 0x7f99b806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.127+0000 7f99dab1d640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99d40b0ea0 0x7f99d40b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.127+0000 7f99dab1d640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99d40c8060 0x7f99d40c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.127+0000 7f99dab1d640 1 --2- 172.21.15.125:0/3974604613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99c4005180 0x7f99d4148490 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.127+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 >> 172.21.15.125:0/3974604613 conn(0x7f99d40d0d60 msgr2=0x7f99d40ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:42.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.127+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 shutdown_connections 2023-10-18T10:37:42.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.127+0000 7f99dab1d640 1 -- 172.21.15.125:0/3974604613 wait complete. 2023-10-18T10:37:42.139 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:37:42.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:42.140 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T10:37:42.151 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.150+0000 7f73572b2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:37:42.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[106073]: fuse finished with error 0 and tester_r 0 2023-10-18T10:37:42.167 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:37:42.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:42.168 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:37:42.242 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.241+0000 7f73572b2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:37:42.250 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.248+0000 7fe78142b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:37:42.254 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:37:42.254 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:37:42.270 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:37:42.270 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:37:42.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:42.271 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:37:42.286 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.285+0000 7f73572b2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:37:42.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:37:42.300 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T10:37:42.301 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:37:42.301 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:37:42.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:37:42.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:37:42.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:42.317 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:37:42.345 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.343+0000 7fe78142b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:37:42.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:37:42.360 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T10:37:42.360 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:37:42.361 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:37:42.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:37:42.376 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:37:42.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:37:42.377 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:37:42.385 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.383+0000 7f73572b2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:37:42.389 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.388+0000 7fe78142b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:37:42.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:37:42.399 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T10:37:42.400 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:37:42.478 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.476+0000 7f73572b2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:37:42.490 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.488+0000 7fe78142b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:37:42.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.503+0000 7f68cda96640 1 -- 172.21.15.125:0/1658935821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b00047c0 msgr2=0x7f68b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:42.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.503+0000 7f68cda96640 1 --2- 172.21.15.125:0/1658935821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b00047c0 0x7f68b0004c40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009b50 tx=0x7f68b00057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.503+0000 7f68cda96640 1 -- 172.21.15.125:0/1658935821 shutdown_connections 2023-10-18T10:37:42.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.503+0000 7f68cda96640 1 --2- 172.21.15.125:0/1658935821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b00047c0 0x7f68b0004c40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.503+0000 7f68cda96640 1 -- 172.21.15.125:0/1658935821 >> 172.21.15.125:0/1658935821 conn(0x7f68c80d0d60 msgr2=0x7f68c80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:42.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.503+0000 7f68cda96640 1 -- 172.21.15.125:0/1658935821 shutdown_connections 2023-10-18T10:37:42.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.503+0000 7f68cda96640 1 -- 172.21.15.125:0/1658935821 wait complete. 2023-10-18T10:37:42.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.504+0000 7f68cda96640 1 Processor -- start 2023-10-18T10:37:42.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68cda96640 1 -- start start 2023-10-18T10:37:42.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68cda96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b00047c0 0x7f68c814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:42.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68cda96640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80ca790 0x7f68c81f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68cda96640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c81f1960 0x7f68c806a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68cda96640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81f23c0 con 0x7f68c80ca790 2023-10-18T10:37:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68cda96640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81f1e00 con 0x7f68b00047c0 2023-10-18T10:37:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68cda96640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81f20e0 con 0x7f68c81f1960 2023-10-18T10:37:42.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68c6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80ca790 0x7f68c81f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:42.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68c6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80ca790 0x7f68c81f1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52446/0 (socket says 172.21.15.125:52446) 2023-10-18T10:37:42.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68c6ffd640 1 -- 172.21.15.125:0/3302441699 learned_addr learned my addr 172.21.15.125:0/3302441699 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:42.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68c77fe640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b00047c0 0x7f68c814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:42.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.505+0000 7f68c7fff640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c81f1960 0x7f68c806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:42.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68c6ffd640 1 -- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c81f1960 msgr2=0x7f68c806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:42.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68c6ffd640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c81f1960 0x7f68c806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68c6ffd640 1 -- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b00047c0 msgr2=0x7f68c814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:42.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68c6ffd640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b00047c0 0x7f68c814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68c6ffd640 1 -- 172.21.15.125:0/3302441699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c80b4b40 con 0x7f68c80ca790 2023-10-18T10:37:42.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68c6ffd640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80ca790 0x7f68c81f1420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f68b802cb70 tx=0x7f68b8033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:42.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68affff640 1 -- 172.21.15.125:0/3302441699 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004440 con 0x7f68c80ca790 2023-10-18T10:37:42.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68affff640 1 -- 172.21.15.125:0/3302441699 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f68b8033e40 con 0x7f68c80ca790 2023-10-18T10:37:42.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c80b4e20 con 0x7f68c80ca790 2023-10-18T10:37:42.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68affff640 1 -- 172.21.15.125:0/3302441699 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8003850 con 0x7f68c80ca790 2023-10-18T10:37:42.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.506+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c806acb0 con 0x7f68c80ca790 2023-10-18T10:37:42.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.507+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c80690e0 con 0x7f68c80ca790 2023-10-18T10:37:42.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.508+0000 7f68affff640 1 -- 172.21.15.125:0/3302441699 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f68b80039b0 con 0x7f68c80ca790 2023-10-18T10:37:42.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.508+0000 7f68affff640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f68b406aff0 0x7f68b406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:42.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.508+0000 7f68c77fe640 1 -- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f68b406aff0 msgr2=0x7f68b406d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:42.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.508+0000 7f68affff640 1 -- 172.21.15.125:0/3302441699 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f68b80acaa0 con 0x7f68c80ca790 2023-10-18T10:37:42.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.508+0000 7f68c77fe640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f68b406aff0 0x7f68b406d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:42.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.512+0000 7f68affff640 1 -- 172.21.15.125:0/3302441699 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68b8077480 con 0x7f68c80ca790 2023-10-18T10:37:42.586 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.585+0000 7fe78142b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:37:42.614 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.613+0000 7f73572b2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:37:42.655 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.654+0000 7f73572b2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:37:42.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.709+0000 7f68c77fe640 1 -- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f68b406aff0 msgr2=0x7f68b406d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:42.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.709+0000 7f68c77fe640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f68b406aff0 0x7f68b406d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:37:42.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.722+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68c80d0b40 con 0x7f68c80ca790 2023-10-18T10:37:42.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.723+0000 7f68affff640 1 -- 172.21.15.125:0/3302441699 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f68b8041090 con 0x7f68c80ca790 2023-10-18T10:37:42.725 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:42.725 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3511459076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3511459076},{"type":"v1","addr":"172.21.15.139:6836","nonce":3511459076}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/984169411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":984169411},{"type":"v1","addr":"172.21.15.139:6837","nonce":984169411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2755714237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2755714237},{"type":"v1","addr":"172.21.15.125:6834","nonce":2755714237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:26:17.831071+0000","modified":"2023-10-18T10:26:19.850357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1934222457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1934222457},{"type":"v1","addr":"172.21.15.125:6835","nonce":1934222457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:37:42.726 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 16 2023-10-18T10:37:42.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f68b406aff0 msgr2=0x7f68b406d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:42.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f68b406aff0 0x7f68b406d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80ca790 msgr2=0x7f68c81f1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:42.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80ca790 0x7f68c81f1420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f68b802cb70 tx=0x7f68b8033c10 comp rx=0 tx=0).stop 2023-10-18T10:37:42.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 shutdown_connections 2023-10-18T10:37:42.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f68b406aff0 0x7f68b406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68c81f1960 0x7f68c806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68c80ca790 0x7f68c81f1420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 --2- 172.21.15.125:0/3302441699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68b00047c0 0x7f68c814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 >> 172.21.15.125:0/3302441699 conn(0x7f68c80d0d60 msgr2=0x7f68c80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:42.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 shutdown_connections 2023-10-18T10:37:42.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.725+0000 7f68cda96640 1 -- 172.21.15.125:0/3302441699 wait complete. 2023-10-18T10:37:42.730 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.727+0000 7fe78142b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:37:42.738 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:37:42.759 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:42.758+0000 7f73572b2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:37:42.774 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.772+0000 7fe78142b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:37:42.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.841+0000 7f18793e7640 1 -- 172.21.15.125:0/2917757702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18741579f0 msgr2=0x7f18740b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:42.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.841+0000 7f18793e7640 1 --2- 172.21.15.125:0/2917757702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18741579f0 0x7f18740b0e90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1864005fd0 tx=0x7f186402be10 comp rx=0 tx=0).stop 2023-10-18T10:37:42.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.841+0000 7f18793e7640 1 -- 172.21.15.125:0/2917757702 shutdown_connections 2023-10-18T10:37:42.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.841+0000 7f18793e7640 1 --2- 172.21.15.125:0/2917757702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f187415c1f0 0x7f187406a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.841+0000 7f18793e7640 1 --2- 172.21.15.125:0/2917757702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18741579f0 0x7f18740b0e90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.841+0000 7f18793e7640 1 -- 172.21.15.125:0/2917757702 >> 172.21.15.125:0/2917757702 conn(0x7f187406fae0 msgr2=0x7f187406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:42.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.841+0000 7f18793e7640 1 -- 172.21.15.125:0/2917757702 shutdown_connections 2023-10-18T10:37:42.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.841+0000 7f18793e7640 1 -- 172.21.15.125:0/2917757702 wait complete. 2023-10-18T10:37:42.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.842+0000 7f18793e7640 1 Processor -- start 2023-10-18T10:37:42.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.842+0000 7f18793e7640 1 -- start start 2023-10-18T10:37:42.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f18793e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f187406ac70 0x7f187413b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:42.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f18793e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18741579f0 0x7f18741ee6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:42.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f18793e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f187415c1f0 0x7f18741eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:42.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f18793e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18741ef130 con 0x7f187406ac70 2023-10-18T10:37:42.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f18793e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1874153eb0 con 0x7f18741579f0 2023-10-18T10:37:42.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f18793e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1874157080 con 0x7f187415c1f0 2023-10-18T10:37:42.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f187406ac70 0x7f187413b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:42.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f187406ac70 0x7f187413b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52476/0 (socket says 172.21.15.125:52476) 2023-10-18T10:37:42.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 -- 172.21.15.125:0/370805825 learned_addr learned my addr 172.21.15.125:0/370805825 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:42.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f18737fe640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f187415c1f0 0x7f18741eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:42.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f186bfff640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18741579f0 0x7f18741ee6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:42.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 -- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f187415c1f0 msgr2=0x7f18741eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:42.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f187415c1f0 0x7f18741eebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 -- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18741579f0 msgr2=0x7f18741ee6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:42.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18741579f0 0x7f18741ee6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:42.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 -- 172.21.15.125:0/370805825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1864009560 con 0x7f187406ac70 2023-10-18T10:37:42.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.843+0000 7f1872ffd640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f187406ac70 0x7f187413b320 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1864036040 tx=0x7f186402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:42.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.844+0000 7f186b7fe640 1 -- 172.21.15.125:0/370805825 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640035f0 con 0x7f187406ac70 2023-10-18T10:37:42.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.844+0000 7f186b7fe640 1 -- 172.21.15.125:0/370805825 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1864003840 con 0x7f187406ac70 2023-10-18T10:37:42.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.844+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1874154170 con 0x7f187406ac70 2023-10-18T10:37:42.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.844+0000 7f186b7fe640 1 -- 172.21.15.125:0/370805825 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864003c50 con 0x7f187406ac70 2023-10-18T10:37:42.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.844+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187414c750 con 0x7f187406ac70 2023-10-18T10:37:42.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.845+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18740690e0 con 0x7f187406ac70 2023-10-18T10:37:42.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.845+0000 7f186b7fe640 1 -- 172.21.15.125:0/370805825 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f186403e450 con 0x7f187406ac70 2023-10-18T10:37:42.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.845+0000 7f186b7fe640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f185406aff0 0x7f185406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:42.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.845+0000 7f186b7fe640 1 -- 172.21.15.125:0/370805825 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f18640ac9b0 con 0x7f187406ac70 2023-10-18T10:37:42.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.845+0000 7f186bfff640 1 -- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f185406aff0 msgr2=0x7f185406d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:42.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.845+0000 7f186bfff640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f185406aff0 0x7f185406d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:42.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:42.849+0000 7f186b7fe640 1 -- 172.21.15.125:0/370805825 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1864077480 con 0x7f187406ac70 2023-10-18T10:37:42.886 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:42.884+0000 7fe78142b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:37:43.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.025+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18741595a0 con 0x7f187406ac70 2023-10-18T10:37:43.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.026+0000 7f186b7fe640 1 -- 172.21.15.125:0/370805825 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18298 (secure 0 0 0) 0x7f186407aaa0 con 0x7f187406ac70 2023-10-18T10:37:43.028 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:43.028 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":51,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:40.328395+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:26:16.213313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-18T10:26:16.801795+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3880096480":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/1401100783":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/2631710418":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/557166225":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6833/2976686360":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6832/2976686360":"2023-10-19T10:26:33.551736+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":4,"length":2},{"begin":8,"length":1},{"begin":10,"length":3},{"begin":15,"length":2},{"begin":19,"length":1},{"begin":21,"length":3},{"begin":26,"length":2},{"begin":30,"length":1},{"begin":32,"length":2},{"begin":36,"length":2},{"begin":39,"length":2},{"begin":43,"length":2},{"begin":47,"length":2},{"begin":50,"length":2},{"begin":53,"length":2},{"begin":57,"length":2},{"begin":60,"length":2},{"begin":63,"length":2},{"begin":67,"length":2},{"begin":70,"length":2},{"begin":73,"length":2},{"begin":77,"length":2},{"begin":80,"length":3},{"begin":84,"length":2},{"begin":88,"length":2},{"begin":91,"length":3},{"begin":95,"length":2},{"begin":99,"length":2},{"begin":103,"length":2},{"begin":106,"length":2},{"begin":110,"length":1},{"begin":159,"length":62}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":16,"length":1},{"begin":27,"length":1},{"begin":37,"length":1},{"begin":48,"length":1},{"begin":58,"length":1},{"begin":68,"length":1},{"begin":78,"length":1},{"begin":89,"length":1},{"begin":100,"length":1},{"begin":211,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:37:43.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f185406aff0 msgr2=0x7f185406d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:43.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f185406aff0 0x7f185406d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f187406ac70 msgr2=0x7f187413b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f187406ac70 0x7f187413b320 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1864036040 tx=0x7f186402dae0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 shutdown_connections 2023-10-18T10:37:43.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f185406aff0 0x7f185406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18741579f0 0x7f18741ee6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f187415c1f0 0x7f18741eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 --2- 172.21.15.125:0/370805825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f187406ac70 0x7f187413b320 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 >> 172.21.15.125:0/370805825 conn(0x7f187406fae0 msgr2=0x7f187414a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:43.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 shutdown_connections 2023-10-18T10:37:43.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.028+0000 7f18793e7640 1 -- 172.21.15.125:0/370805825 wait complete. 2023-10-18T10:37:43.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:26:16.213313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:26:16.801795+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:37:43.041 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:37:43.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 -- 172.21.15.125:0/1865916444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015ccf0 msgr2=0x7f306015d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 --2- 172.21.15.125:0/1865916444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015ccf0 0x7f306015d180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f30600714a0 tx=0x7f305802c1d0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 -- 172.21.15.125:0/1865916444 shutdown_connections 2023-10-18T10:37:43.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 --2- 172.21.15.125:0/1865916444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f306015ccf0 0x7f306015d180 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 --2- 172.21.15.125:0/1865916444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cdb30 0x7f30600cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 --2- 172.21.15.125:0/1865916444 >> v2:172.21.15.139:3300/0 conn(0x7f3060159060 0x7f306006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 -- 172.21.15.125:0/1865916444 >> 172.21.15.125:0/1865916444 conn(0x7f30600b0670 msgr2=0x7f30600b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:43.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 -- 172.21.15.125:0/1865916444 shutdown_connections 2023-10-18T10:37:43.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.185+0000 7f3066eb4640 1 -- 172.21.15.125:0/1865916444 wait complete. 2023-10-18T10:37:43.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.186+0000 7f3066eb4640 1 Processor -- start 2023-10-18T10:37:43.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.186+0000 7f3066eb4640 1 -- start start 2023-10-18T10:37:43.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3066eb4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cdb30 0x7f3060155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3066eb4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3060159060 0x7f30600b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3066eb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306015ccf0 0x7f30601e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3066eb4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060156220 con 0x7f3060159060 2023-10-18T10:37:43.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3066eb4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30601f3430 con 0x7f306015ccf0 2023-10-18T10:37:43.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3066eb4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30601f2670 con 0x7f30600cdb30 2023-10-18T10:37:43.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3060159060 0x7f30600b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3060159060 0x7f30600b2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52484/0 (socket says 172.21.15.125:52484) 2023-10-18T10:37:43.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 -- 172.21.15.125:0/2795116297 learned_addr learned my addr 172.21.15.125:0/2795116297 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:43.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3064c50640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cdb30 0x7f3060155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3065451640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306015ccf0 0x7f30601e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 -- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cdb30 msgr2=0x7f3060155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cdb30 0x7f3060155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 -- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306015ccf0 msgr2=0x7f30601e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306015ccf0 0x7f30601e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 -- 172.21.15.125:0/2795116297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3058009900 con 0x7f3060159060 2023-10-18T10:37:43.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305ffff640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3060159060 0x7f30600b2310 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f304c00d770 tx=0x7f304c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305d7fa640 1 -- 172.21.15.125:0/2795116297 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c009c30 con 0x7f3060159060 2023-10-18T10:37:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f305d7fa640 1 -- 172.21.15.125:0/2795116297 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f304c00de80 con 0x7f3060159060 2023-10-18T10:37:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.187+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30601f2950 con 0x7f3060159060 2023-10-18T10:37:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.188+0000 7f305d7fa640 1 -- 172.21.15.125:0/2795116297 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c005280 con 0x7f3060159060 2023-10-18T10:37:43.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.188+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30601f2f10 con 0x7f3060159060 2023-10-18T10:37:43.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.189+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30600675f0 con 0x7f3060159060 2023-10-18T10:37:43.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.189+0000 7f305d7fa640 1 -- 172.21.15.125:0/2795116297 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f304c0053e0 con 0x7f3060159060 2023-10-18T10:37:43.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.189+0000 7f305d7fa640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f304406b020 0x7f304406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.189+0000 7f305d7fa640 1 -- 172.21.15.125:0/2795116297 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f304c08a030 con 0x7f3060159060 2023-10-18T10:37:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.189+0000 7f3064c50640 1 -- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f304406b020 msgr2=0x7f304406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:43.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.189+0000 7f3064c50640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f304406b020 0x7f304406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:43.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.194+0000 7f305d7fa640 1 -- 172.21.15.125:0/2795116297 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f304c054980 con 0x7f3060159060 2023-10-18T10:37:43.372 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.370+0000 7f73572b2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:37:43.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.370+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30600b4aa0 con 0x7f3060159060 2023-10-18T10:37:43.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.370+0000 7f305d7fa640 1 -- 172.21.15.125:0/2795116297 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f304c0170a0 con 0x7f3060159060 2023-10-18T10:37:43.373 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:37:43.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f304406b020 msgr2=0x7f304406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f304406b020 0x7f304406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3060159060 msgr2=0x7f30600b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3060159060 0x7f30600b2310 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f304c00d770 tx=0x7f304c00dc30 comp rx=0 tx=0).stop 2023-10-18T10:37:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 shutdown_connections 2023-10-18T10:37:43.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7f304406b020 0x7f304406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306015ccf0 0x7f30601e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3060159060 0x7f30600b2310 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 --2- 172.21.15.125:0/2795116297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30600cdb30 0x7f3060155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.372+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 >> 172.21.15.125:0/2795116297 conn(0x7f30600b0670 msgr2=0x7f30600b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:43.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.373+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 shutdown_connections 2023-10-18T10:37:43.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.373+0000 7f3066eb4640 1 -- 172.21.15.125:0/2795116297 wait complete. 2023-10-18T10:37:43.385 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:37:43.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.490+0000 7fc4b8a01640 1 -- 172.21.15.125:0/1518407708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40cb690 msgr2=0x7fc4b40cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.490+0000 7fc4b8a01640 1 --2- 172.21.15.125:0/1518407708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40cb690 0x7fc4b40cf400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009be0 tx=0x7fc4ac02c160 comp rx=0 tx=0).stop 2023-10-18T10:37:43.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.490+0000 7fc4b8a01640 1 -- 172.21.15.125:0/1518407708 shutdown_connections 2023-10-18T10:37:43.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.490+0000 7fc4b8a01640 1 --2- 172.21.15.125:0/1518407708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b40cb690 0x7fc4b40cf400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.490+0000 7fc4b8a01640 1 -- 172.21.15.125:0/1518407708 >> 172.21.15.125:0/1518407708 conn(0x7fc4b40ce8e0 msgr2=0x7fc4b40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:43.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.490+0000 7fc4b8a01640 1 -- 172.21.15.125:0/1518407708 shutdown_connections 2023-10-18T10:37:43.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.490+0000 7fc4b8a01640 1 -- 172.21.15.125:0/1518407708 wait complete. 2023-10-18T10:37:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.492+0000 7fc4b8a01640 1 Processor -- start 2023-10-18T10:37:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.492+0000 7fc4b8a01640 1 -- start start 2023-10-18T10:37:43.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.492+0000 7fc4b8a01640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4b406d2a0 0x7fc4b4149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.492+0000 7fc4b8a01640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40cb690 0x7fc4b414a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.492+0000 7fc4b8a01640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b4159040 0x7fc4b415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.492+0000 7fc4b8a01640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40c6db0 con 0x7fc4b406d2a0 2023-10-18T10:37:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.492+0000 7fc4b8a01640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40c7090 con 0x7fc4b40cb690 2023-10-18T10:37:43.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.492+0000 7fc4b8a01640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40c7370 con 0x7fc4b4159040 2023-10-18T10:37:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4b406d2a0 0x7fc4b4149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4b406d2a0 0x7fc4b4149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52486/0 (socket says 172.21.15.125:52486) 2023-10-18T10:37:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 -- 172.21.15.125:0/3588781505 learned_addr learned my addr 172.21.15.125:0/3588781505 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b259c640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40cb690 0x7fc4b414a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b359e640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b4159040 0x7fc4b415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 -- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b4159040 msgr2=0x7fc4b415bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b4159040 0x7fc4b415bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 -- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40cb690 msgr2=0x7fc4b414a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40cb690 0x7fc4b414a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 -- 172.21.15.125:0/3588781505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac009840 con 0x7fc4b406d2a0 2023-10-18T10:37:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b359e640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b4159040 0x7fc4b415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:37:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b2d9d640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4b406d2a0 0x7fc4b4149ca0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800c410 tx=0x7fc4a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc49b7fe640 1 -- 172.21.15.125:0/3588781505 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80035f0 con 0x7fc4b406d2a0 2023-10-18T10:37:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc49b7fe640 1 -- 172.21.15.125:0/3588781505 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4a8003770 con 0x7fc4b406d2a0 2023-10-18T10:37:43.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b40c7650 con 0x7fc4b406d2a0 2023-10-18T10:37:43.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc49b7fe640 1 -- 172.21.15.125:0/3588781505 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8003b80 con 0x7fc4b406d2a0 2023-10-18T10:37:43.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.493+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b414a1e0 con 0x7fc4b406d2a0 2023-10-18T10:37:43.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.494+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b40c7b70 con 0x7fc4b406d2a0 2023-10-18T10:37:43.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.495+0000 7fc49b7fe640 1 -- 172.21.15.125:0/3588781505 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fc4a8003ce0 con 0x7fc4b406d2a0 2023-10-18T10:37:43.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.495+0000 7fc49b7fe640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc4a006b020 0x7fc4a006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.495+0000 7fc4b259c640 1 -- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc4a006b020 msgr2=0x7fc4a006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:43.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.495+0000 7fc49b7fe640 1 -- 172.21.15.125:0/3588781505 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7028+0+0 (secure 0 0 0) 0x7fc4a80893c0 con 0x7fc4b406d2a0 2023-10-18T10:37:43.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.495+0000 7fc4b259c640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc4a006b020 0x7fc4a006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:43.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.499+0000 7fc49b7fe640 1 -- 172.21.15.125:0/3588781505 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4b40c7b70 con 0x7fc4b406d2a0 2023-10-18T10:37:43.534 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:43.533+0000 7fe78142b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:37:43.642 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.640+0000 7f73572b2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:37:43.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.677+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4b40c7b70 con 0x7fc4b406d2a0 2023-10-18T10:37:43.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.677+0000 7fc49b7fe640 1 -- 172.21.15.125:0/3588781505 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4a8017020 con 0x7fc4b406d2a0 2023-10-18T10:37:43.679 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:37:43.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc4a006b020 msgr2=0x7fc4a006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:43.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc4a006b020 0x7fc4a006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4b406d2a0 msgr2=0x7fc4b4149ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4b406d2a0 0x7fc4b4149ca0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800c410 tx=0x7fc4a800c8d0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 shutdown_connections 2023-10-18T10:37:43.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7fc4a006b020 0x7fc4a006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40cb690 0x7fc4b414a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4b4159040 0x7fc4b415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 --2- 172.21.15.125:0/3588781505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4b406d2a0 0x7fc4b4149ca0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 >> 172.21.15.125:0/3588781505 conn(0x7fc4b40ce8e0 msgr2=0x7fc4b4156d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:43.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 shutdown_connections 2023-10-18T10:37:43.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.679+0000 7fc4b8a01640 1 -- 172.21.15.125:0/3588781505 wait complete. 2023-10-18T10:37:43.691 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.689+0000 7f73572b2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:37:43.692 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:37:43.790 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.788+0000 7f73572b2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:37:43.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.796+0000 7efec4ef7640 1 -- 172.21.15.125:0/1434376205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 msgr2=0x7efec00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.796+0000 7efec4ef7640 1 --2- 172.21.15.125:0/1434376205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 0x7efec00d1910 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efec015b680 tx=0x7efea802bdb0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.796+0000 7efec4ef7640 1 -- 172.21.15.125:0/1434376205 shutdown_connections 2023-10-18T10:37:43.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.796+0000 7efec4ef7640 1 --2- 172.21.15.125:0/1434376205 >> v2:172.21.15.139:3300/0 conn(0x7efec00ceb40 0x7efec00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.796+0000 7efec4ef7640 1 --2- 172.21.15.125:0/1434376205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 0x7efec00d1910 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.796+0000 7efec4ef7640 1 -- 172.21.15.125:0/1434376205 >> 172.21.15.125:0/1434376205 conn(0x7efec00d0d60 msgr2=0x7efec00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:43.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.796+0000 7efec4ef7640 1 -- 172.21.15.125:0/1434376205 shutdown_connections 2023-10-18T10:37:43.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.796+0000 7efec4ef7640 1 -- 172.21.15.125:0/1434376205 wait complete. 2023-10-18T10:37:43.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.797+0000 7efec4ef7640 1 Processor -- start 2023-10-18T10:37:43.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.797+0000 7efec4ef7640 1 -- start start 2023-10-18T10:37:43.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efec4ef7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec00ceb40 0x7efec0151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efec4ef7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec00cf340 0x7efec00b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efec4ef7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 0x7efec00b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efec4ef7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec0151960 con 0x7efec015b2a0 2023-10-18T10:37:43.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efec4ef7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec00b4d40 con 0x7efec00ceb40 2023-10-18T10:37:43.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efec4ef7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec00b5020 con 0x7efec00cf340 2023-10-18T10:37:43.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efebed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 0x7efec00b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efebed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 0x7efec00b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52504/0 (socket says 172.21.15.125:52504) 2023-10-18T10:37:43.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efebed9d640 1 -- 172.21.15.125:0/2323720839 learned_addr learned my addr 172.21.15.125:0/2323720839 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:43.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efebe59c640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec00ceb40 0x7efec0151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.798+0000 7efeb7fff640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec00cf340 0x7efec00b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:43.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efebed9d640 1 -- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec00cf340 msgr2=0x7efec00b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efebed9d640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec00cf340 0x7efec00b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efebed9d640 1 -- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec00ceb40 msgr2=0x7efec0151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efebed9d640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec00ceb40 0x7efec0151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efebed9d640 1 -- 172.21.15.125:0/2323720839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8009560 con 0x7efec015b2a0 2023-10-18T10:37:43.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efebed9d640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 0x7efec00b42f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efeb800da70 tx=0x7efeb800df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:43.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb800bdd0 con 0x7efec015b2a0 2023-10-18T10:37:43.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efeb8003a60 con 0x7efec015b2a0 2023-10-18T10:37:43.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec00b5300 con 0x7efec015b2a0 2023-10-18T10:37:43.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8003e70 con 0x7efec015b2a0 2023-10-18T10:37:43.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.799+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec00b58c0 con 0x7efec015b2a0 2023-10-18T10:37:43.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.800+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec00690e0 con 0x7efec015b2a0 2023-10-18T10:37:43.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.800+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 89400+0+0 (secure 0 0 0) 0x7efeb8003fd0 con 0x7efec015b2a0 2023-10-18T10:37:43.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.801+0000 7efeb6ffd640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7efea006aff0 0x7efea006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:43.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.801+0000 7efebe59c640 1 -- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7efea006aff0 msgr2=0x7efea006d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/609182116 2023-10-18T10:37:43.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.801+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7028+0+0 (secure 0 0 0) 0x7efeb808a430 con 0x7efec015b2a0 2023-10-18T10:37:43.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.801+0000 7efebe59c640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7efea006aff0 0x7efea006d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:37:43.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.809+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efeb8054d00 con 0x7efec015b2a0 2023-10-18T10:37:43.819 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:43.816+0000 7fe78142b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:37:43.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.822+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 17) v1 ==== 88940+0+0 (secure 0 0 0) 0x7efeb80550d0 con 0x7efec015b2a0 2023-10-18T10:37:43.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.822+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7efea006aff0 msgr2=0x7efea006d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:37:43.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.822+0000 7efeb6ffd640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7efea006aff0 0x7efea006d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.873 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:43.871+0000 7fe78142b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:37:43.880 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72dcd49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.881 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72dcd49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.881 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72dcd49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.881 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72dcd49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.881 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72dcd49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.882 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72d9d43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.882 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72d9d43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.882 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72d9d43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.883 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72d9d43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.883 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:37:43.879+0000 7f72d9d43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:37:43.978 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:37:43.976+0000 7fe78142b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:37:43.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.992+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efec00b0d00 con 0x7efec015b2a0 2023-10-18T10:37:43.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.993+0000 7efeb6ffd640 1 -- 172.21.15.125:0/2323720839 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7efeb8020070 con 0x7efec015b2a0 2023-10-18T10:37:43.995 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:37:43.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 msgr2=0x7efec00b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:43.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 0x7efec00b42f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efeb800da70 tx=0x7efeb800df30 comp rx=0 tx=0).stop 2023-10-18T10:37:43.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 shutdown_connections 2023-10-18T10:37:43.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:6832/609182116,v1:172.21.15.139:6833/609182116] conn(0x7efea006aff0 0x7efea006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efec015b2a0 0x7efec00b42f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec00cf340 0x7efec00b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 --2- 172.21.15.125:0/2323720839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec00ceb40 0x7efec0151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:43.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 >> 172.21.15.125:0/2323720839 conn(0x7efec00d0d60 msgr2=0x7efec006b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:43.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 shutdown_connections 2023-10-18T10:37:43.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:43.995+0000 7efec4ef7640 1 -- 172.21.15.125:0/2323720839 wait complete. 2023-10-18T10:37:44.007 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:37:44.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.108+0000 7f8caafe7640 1 -- 172.21.15.125:0/893210690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c94003fb0 msgr2=0x7f8c940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.108+0000 7f8caafe7640 1 --2- 172.21.15.125:0/893210690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c94003fb0 0x7f8c940047c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000bb20 tx=0x7f8c940057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.108+0000 7f8ca9584640 1 -- 172.21.15.125:0/893210690 reap_dead start 2023-10-18T10:37:44.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.108+0000 7f8caafe7640 1 -- 172.21.15.125:0/893210690 shutdown_connections 2023-10-18T10:37:44.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.108+0000 7f8caafe7640 1 -- 172.21.15.125:0/893210690 >> 172.21.15.125:0/893210690 conn(0x7f8ca4069930 msgr2=0x7f8ca4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:44.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.108+0000 7f8caafe7640 1 -- 172.21.15.125:0/893210690 shutdown_connections 2023-10-18T10:37:44.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.108+0000 7f8caafe7640 1 -- 172.21.15.125:0/893210690 wait complete. 2023-10-18T10:37:44.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.109+0000 7f8caafe7640 1 Processor -- start 2023-10-18T10:37:44.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.109+0000 7f8caafe7640 1 -- start start 2023-10-18T10:37:44.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8caafe7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c94003fb0 0x7f8ca40a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8caafe7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca40b1250 0x7f8ca40b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8caafe7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ca4155bc0 0x7f8ca4156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8caafe7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4131930 con 0x7f8ca4155bc0 2023-10-18T10:37:44.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8caafe7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40b1c10 con 0x7f8ca40b1250 2023-10-18T10:37:44.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8caafe7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4130700 con 0x7f8c94003fb0 2023-10-18T10:37:44.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ca4155bc0 0x7f8ca4156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ca4155bc0 0x7f8ca4156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52520/0 (socket says 172.21.15.125:52520) 2023-10-18T10:37:44.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 -- 172.21.15.125:0/1346522582 learned_addr learned my addr 172.21.15.125:0/1346522582 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:44.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca8d83640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c94003fb0 0x7f8ca40a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca3fff640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca40b1250 0x7f8ca40b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 -- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c94003fb0 msgr2=0x7f8ca40a6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c94003fb0 0x7f8ca40a6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 -- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca40b1250 msgr2=0x7f8ca40b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca40b1250 0x7f8ca40b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 -- 172.21.15.125:0/1346522582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca41309e0 con 0x7f8ca4155bc0 2023-10-18T10:37:44.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca3fff640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca40b1250 0x7f8ca40b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:44.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca8d83640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c94003fb0 0x7f8ca40a6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:44.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca9584640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ca4155bc0 0x7f8ca4156040 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40cc430 tx=0x7f8c9c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:44.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.110+0000 7f8ca17fa640 1 -- 172.21.15.125:0/1346522582 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0061e0 con 0x7f8ca4155bc0 2023-10-18T10:37:44.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.111+0000 7f8ca17fa640 1 -- 172.21.15.125:0/1346522582 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8c9c00c940 con 0x7f8ca4155bc0 2023-10-18T10:37:44.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.111+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca4130cc0 con 0x7f8ca4155bc0 2023-10-18T10:37:44.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.111+0000 7f8ca17fa640 1 -- 172.21.15.125:0/1346522582 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00cd50 con 0x7f8ca4155bc0 2023-10-18T10:37:44.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.111+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca40a7140 con 0x7f8ca4155bc0 2023-10-18T10:37:44.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.111+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca4155a10 con 0x7f8ca4155bc0 2023-10-18T10:37:44.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.112+0000 7f8ca17fa640 1 -- 172.21.15.125:0/1346522582 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8c9c01f460 con 0x7f8ca4155bc0 2023-10-18T10:37:44.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.112+0000 7f8ca17fa640 1 -- 172.21.15.125:0/1346522582 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6951+0+0 (secure 0 0 0) 0x7f8c9c089980 con 0x7f8ca4155bc0 2023-10-18T10:37:44.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.117+0000 7f8ca17fa640 1 -- 172.21.15.125:0/1346522582 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c9c0549a0 con 0x7f8ca4155bc0 2023-10-18T10:37:44.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.316+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ca40690e0 con 0x7f8ca4155bc0 2023-10-18T10:37:44.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.316+0000 7f8ca17fa640 1 -- 172.21.15.125:0/1346522582 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f8c9c057fc0 con 0x7f8ca4155bc0 2023-10-18T10:37:44.318 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:44.319 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3511459076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3511459076},{"type":"v1","addr":"172.21.15.139:6836","nonce":3511459076}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/984169411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":984169411},{"type":"v1","addr":"172.21.15.139:6837","nonce":984169411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2755714237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2755714237},{"type":"v1","addr":"172.21.15.125:6834","nonce":2755714237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:26:17.831071+0000","modified":"2023-10-18T10:26:19.850357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1934222457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1934222457},{"type":"v1","addr":"172.21.15.125:6835","nonce":1934222457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:37:44.320 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 16 2023-10-18T10:37:44.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ca4155bc0 msgr2=0x7f8ca4156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ca4155bc0 0x7f8ca4156040 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40cc430 tx=0x7f8c9c00ed10 comp rx=0 tx=0).stop 2023-10-18T10:37:44.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 shutdown_connections 2023-10-18T10:37:44.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ca40b1250 0x7f8ca40b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ca4155bc0 0x7f8ca4156040 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 --2- 172.21.15.125:0/1346522582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c94003fb0 0x7f8ca40a6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 >> 172.21.15.125:0/1346522582 conn(0x7f8ca4069930 msgr2=0x7f8ca41522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:44.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 shutdown_connections 2023-10-18T10:37:44.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.318+0000 7f8caafe7640 1 -- 172.21.15.125:0/1346522582 wait complete. 2023-10-18T10:37:44.330 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:37:44.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.431+0000 7fd41d256640 1 -- 172.21.15.125:0/2919431000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180cf340 msgr2=0x7fd4180d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.431+0000 7fd41d256640 1 --2- 172.21.15.125:0/2919431000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180cf340 0x7fd4180d1910 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd41808a810 tx=0x7fd40802bcf0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.432+0000 7fd41d256640 1 -- 172.21.15.125:0/2919431000 shutdown_connections 2023-10-18T10:37:44.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.432+0000 7fd41d256640 1 --2- 172.21.15.125:0/2919431000 >> v2:172.21.15.139:3300/0 conn(0x7fd41815b2a0 0x7fd4180cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.432+0000 7fd41d256640 1 --2- 172.21.15.125:0/2919431000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180cf340 0x7fd4180d1910 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.432+0000 7fd41d256640 1 -- 172.21.15.125:0/2919431000 >> 172.21.15.125:0/2919431000 conn(0x7fd4180d0d60 msgr2=0x7fd4180d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:44.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.432+0000 7fd41d256640 1 -- 172.21.15.125:0/2919431000 shutdown_connections 2023-10-18T10:37:44.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.432+0000 7fd41d256640 1 -- 172.21.15.125:0/2919431000 wait complete. 2023-10-18T10:37:44.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.433+0000 7fd41d256640 1 Processor -- start 2023-10-18T10:37:44.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.433+0000 7fd41d256640 1 -- start start 2023-10-18T10:37:44.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41d256640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180ceb40 0x7fd4180c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41d256640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4180cf340 0x7fd4180b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41d256640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd41815b2a0 0x7fd418156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41d256640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418157310 con 0x7fd4180ceb40 2023-10-18T10:37:44.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41d256640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180b3e90 con 0x7fd4180cf340 2023-10-18T10:37:44.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41d256640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd418157120 con 0x7fd41815b2a0 2023-10-18T10:37:44.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180ceb40 0x7fd4180c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180ceb40 0x7fd4180c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52534/0 (socket says 172.21.15.125:52534) 2023-10-18T10:37:44.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 -- 172.21.15.125:0/101121189 learned_addr learned my addr 172.21.15.125:0/101121189 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41659c640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4180cf340 0x7fd4180b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41759e640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd41815b2a0 0x7fd418156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 -- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd41815b2a0 msgr2=0x7fd418156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd41815b2a0 0x7fd418156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 -- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4180cf340 msgr2=0x7fd4180b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4180cf340 0x7fd4180b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 -- 172.21.15.125:0/101121189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd408009560 con 0x7fd4180ceb40 2023-10-18T10:37:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd41659c640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4180cf340 0x7fd4180b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd416d9d640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180ceb40 0x7fd4180c88d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd40802c300 tx=0x7fd40802da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:44.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.434+0000 7fd3ff7fe640 1 -- 172.21.15.125:0/101121189 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408003800 con 0x7fd4180ceb40 2023-10-18T10:37:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.435+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4181f1480 con 0x7fd4180ceb40 2023-10-18T10:37:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.435+0000 7fd3ff7fe640 1 -- 172.21.15.125:0/101121189 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd408003a50 con 0x7fd4180ceb40 2023-10-18T10:37:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.435+0000 7fd3ff7fe640 1 -- 172.21.15.125:0/101121189 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408003e60 con 0x7fd4180ceb40 2023-10-18T10:37:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.435+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4181f5620 con 0x7fd4180ceb40 2023-10-18T10:37:44.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.435+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4180690e0 con 0x7fd4180ceb40 2023-10-18T10:37:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.436+0000 7fd3ff7fe640 1 -- 172.21.15.125:0/101121189 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd40803e450 con 0x7fd4180ceb40 2023-10-18T10:37:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.436+0000 7fd3ff7fe640 1 -- 172.21.15.125:0/101121189 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6951+0+0 (secure 0 0 0) 0x7fd4080ac410 con 0x7fd4180ceb40 2023-10-18T10:37:44.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.441+0000 7fd3ff7fe640 1 -- 172.21.15.125:0/101121189 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4080774d0 con 0x7fd4180ceb40 2023-10-18T10:37:44.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.611+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4180cb9d0 con 0x7fd4180ceb40 2023-10-18T10:37:44.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.613+0000 7fd3ff7fe640 1 -- 172.21.15.125:0/101121189 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18169 (secure 0 0 0) 0x7fd40807aaf0 con 0x7fd4180ceb40 2023-10-18T10:37:44.615 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:44.615 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":53,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:43.810407+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:26:16.213313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-18T10:26:16.801795+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":53,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2073919910":"2023-10-19T10:37:43.810352+0000","172.21.15.139:6833/609182116":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/2631710418":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/1401100783":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/219007869":"2023-10-19T10:37:43.810352+0000","172.21.15.139:6833/2976686360":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/3880096480":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6832/2976686360":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/1021953536":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/336930685":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/3718327380":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/485385349":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/768306585":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/557166225":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6832/609182116":"2023-10-19T10:37:43.810352+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":16,"length":1},{"begin":27,"length":1},{"begin":37,"length":1},{"begin":48,"length":1},{"begin":58,"length":1},{"begin":68,"length":1},{"begin":78,"length":1},{"begin":89,"length":1},{"begin":100,"length":1},{"begin":111,"length":1},{"begin":204,"length":18}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":111,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:37:44.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180ceb40 msgr2=0x7fd4180c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180ceb40 0x7fd4180c88d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd40802c300 tx=0x7fd40802da90 comp rx=0 tx=0).stop 2023-10-18T10:37:44.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 shutdown_connections 2023-10-18T10:37:44.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4180cf340 0x7fd4180b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd41815b2a0 0x7fd418156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 --2- 172.21.15.125:0/101121189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4180ceb40 0x7fd4180c88d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 >> 172.21.15.125:0/101121189 conn(0x7fd4180d0d60 msgr2=0x7fd4180cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:44.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 shutdown_connections 2023-10-18T10:37:44.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.615+0000 7fd41d256640 1 -- 172.21.15.125:0/101121189 wait complete. 2023-10-18T10:37:44.627 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:37:44.628 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:37:44.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.731+0000 7f3b69c10640 1 -- 172.21.15.125:0/2084331338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 msgr2=0x7f3b6406bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.731+0000 7f3b69c10640 1 --2- 172.21.15.125:0/2084331338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 0x7f3b6406bb60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3b54005fd0 tx=0x7f3b5402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.731+0000 7f3b69c10640 1 -- 172.21.15.125:0/2084331338 shutdown_connections 2023-10-18T10:37:44.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.731+0000 7f3b69c10640 1 --2- 172.21.15.125:0/2084331338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b64141690 0x7f3b640703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.731+0000 7f3b69c10640 1 --2- 172.21.15.125:0/2084331338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 0x7f3b6406bb60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.731+0000 7f3b69c10640 1 -- 172.21.15.125:0/2084331338 >> 172.21.15.125:0/2084331338 conn(0x7f3b64069930 msgr2=0x7f3b64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:44.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.731+0000 7f3b69c10640 1 -- 172.21.15.125:0/2084331338 shutdown_connections 2023-10-18T10:37:44.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.731+0000 7f3b69c10640 1 -- 172.21.15.125:0/2084331338 wait complete. 2023-10-18T10:37:44.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.732+0000 7f3b69c10640 1 Processor -- start 2023-10-18T10:37:44.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.732+0000 7f3b69c10640 1 -- start start 2023-10-18T10:37:44.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b69c10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6406c0a0 0x7f3b641daaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b69c10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 0x7f3b641db5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b69c10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b64141690 0x7f3b641dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b69c10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b641e1310 con 0x7f3b6413e520 2023-10-18T10:37:44.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b69c10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b641e0420 con 0x7f3b6406c0a0 2023-10-18T10:37:44.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b69c10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b641e0700 con 0x7f3b64141690 2023-10-18T10:37:44.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 0x7f3b641db5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 0x7f3b641db5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52558/0 (socket says 172.21.15.125:52558) 2023-10-18T10:37:44.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 -- 172.21.15.125:0/3879132365 learned_addr learned my addr 172.21.15.125:0/3879132365 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:44.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b637fe640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6406c0a0 0x7f3b641daaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b63fff640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b64141690 0x7f3b641dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 -- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b64141690 msgr2=0x7f3b641dbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b64141690 0x7f3b641dbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 -- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6406c0a0 msgr2=0x7f3b641daaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6406c0a0 0x7f3b641daaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 -- 172.21.15.125:0/3879132365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b54009560 con 0x7f3b6413e520 2023-10-18T10:37:44.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b637fe640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6406c0a0 0x7f3b641daaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:44.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b63fff640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b64141690 0x7f3b641dbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:44.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.733+0000 7f3b62ffd640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 0x7f3b641db5a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800d990 tx=0x7f3b5800de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:44.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.734+0000 7f3b4bfff640 1 -- 172.21.15.125:0/3879132365 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800ed60 con 0x7f3b6413e520 2023-10-18T10:37:44.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.734+0000 7f3b4bfff640 1 -- 172.21.15.125:0/3879132365 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b58016440 con 0x7f3b6413e520 2023-10-18T10:37:44.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.734+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b641e09e0 con 0x7f3b6413e520 2023-10-18T10:37:44.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.734+0000 7f3b4bfff640 1 -- 172.21.15.125:0/3879132365 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58016850 con 0x7f3b6413e520 2023-10-18T10:37:44.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.734+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b641e0fa0 con 0x7f3b6413e520 2023-10-18T10:37:44.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.735+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b641e1480 con 0x7f3b6413e520 2023-10-18T10:37:44.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.735+0000 7f3b4bfff640 1 -- 172.21.15.125:0/3879132365 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 17) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f3b5801f3b0 con 0x7f3b6413e520 2023-10-18T10:37:44.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.736+0000 7f3b4bfff640 1 -- 172.21.15.125:0/3879132365 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6951+0+0 (secure 0 0 0) 0x7f3b5808b820 con 0x7f3b6413e520 2023-10-18T10:37:44.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.740+0000 7f3b4bfff640 1 -- 172.21.15.125:0/3879132365 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b58056840 con 0x7f3b6413e520 2023-10-18T10:37:44.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.820+0000 7f3b4bfff640 1 -- 172.21.15.125:0/3879132365 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 18) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3b580562f0 con 0x7f3b6413e520 2023-10-18T10:37:44.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.820+0000 7f3b4bfff640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b50070d80 0x7f3b50073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:44.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.823+0000 7f3b637fe640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b50070d80 0x7f3b50073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:44.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.824+0000 7f3b637fe640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b50070d80 0x7f3b50073170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b54035000 tx=0x7f3b54004730 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:44.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.945+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b640690e0 con 0x7f3b6413e520 2023-10-18T10:37:44.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.946+0000 7f3b4bfff640 1 -- 172.21.15.125:0/3879132365 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f3b58074380 con 0x7f3b6413e520 2023-10-18T10:37:44.948 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:44.948 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3511459076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3511459076},{"type":"v1","addr":"172.21.15.139:6836","nonce":3511459076}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/984169411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":984169411},{"type":"v1","addr":"172.21.15.139:6837","nonce":984169411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2755714237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2755714237},{"type":"v1","addr":"172.21.15.125:6834","nonce":2755714237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:26:17.831071+0000","modified":"2023-10-18T10:26:19.850357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1934222457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1934222457},{"type":"v1","addr":"172.21.15.125:6835","nonce":1934222457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:37:44.949 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 16 2023-10-18T10:37:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b50070d80 msgr2=0x7f3b50073170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b50070d80 0x7f3b50073170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b54035000 tx=0x7f3b54004730 comp rx=0 tx=0).stop 2023-10-18T10:37:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 msgr2=0x7f3b641db5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 0x7f3b641db5a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800d990 tx=0x7f3b5800de50 comp rx=0 tx=0).stop 2023-10-18T10:37:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 shutdown_connections 2023-10-18T10:37:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b50070d80 0x7f3b50073170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b64141690 0x7f3b641dbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b6413e520 0x7f3b641db5a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 --2- 172.21.15.125:0/3879132365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b6406c0a0 0x7f3b641daaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 >> 172.21.15.125:0/3879132365 conn(0x7f3b64069930 msgr2=0x7f3b640c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 shutdown_connections 2023-10-18T10:37:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:44.948+0000 7f3b69c10640 1 -- 172.21.15.125:0/3879132365 wait complete. 2023-10-18T10:37:44.960 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:37:45.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.061+0000 7fcc97cf2640 1 -- 172.21.15.125:0/2858109089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc90140660 msgr2=0x7fcc900c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.061+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/2858109089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc90140660 0x7fcc900c97c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcc900ca400 tx=0x7fcc8002c630 comp rx=0 tx=0).stop 2023-10-18T10:37:45.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.061+0000 7fcc97cf2640 1 -- 172.21.15.125:0/2858109089 shutdown_connections 2023-10-18T10:37:45.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.061+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/2858109089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc90140660 0x7fcc900c97c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.061+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/2858109089 >> v2:172.21.15.139:3300/0 conn(0x7fcc9013d360 0x7fcc9015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.061+0000 7fcc97cf2640 1 -- 172.21.15.125:0/2858109089 >> 172.21.15.125:0/2858109089 conn(0x7fcc90069930 msgr2=0x7fcc90069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:45.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.061+0000 7fcc97cf2640 1 -- 172.21.15.125:0/2858109089 shutdown_connections 2023-10-18T10:37:45.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.061+0000 7fcc97cf2640 1 -- 172.21.15.125:0/2858109089 wait complete. 2023-10-18T10:37:45.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.062+0000 7fcc97cf2640 1 Processor -- start 2023-10-18T10:37:45.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.062+0000 7fcc97cf2640 1 -- start start 2023-10-18T10:37:45.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.062+0000 7fcc97cf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900c9d00 0x7fcc900acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc97cf2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc9013d360 0x7fcc901f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc97cf2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90140660 0x7fcc901f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc97cf2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901efc10 con 0x7fcc9013d360 2023-10-18T10:37:45.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc97cf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901f3d10 con 0x7fcc900c9d00 2023-10-18T10:37:45.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc97cf2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901f3a30 con 0x7fcc90140660 2023-10-18T10:37:45.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc9013d360 0x7fcc901f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc9013d360 0x7fcc901f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52584/0 (socket says 172.21.15.125:52584) 2023-10-18T10:37:45.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 -- 172.21.15.125:0/1135516267 learned_addr learned my addr 172.21.15.125:0/1135516267 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:45.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc95a8e640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900c9d00 0x7fcc900acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9628f640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90140660 0x7fcc901f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 -- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90140660 msgr2=0x7fcc901f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90140660 0x7fcc901f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 -- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900c9d00 msgr2=0x7fcc900acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900c9d00 0x7fcc900acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 -- 172.21.15.125:0/1135516267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80009df0 con 0x7fcc9013d360 2023-10-18T10:37:45.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9628f640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90140660 0x7fcc901f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:45.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc9528d640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc9013d360 0x7fcc901f2e50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000b3c0 tx=0x7fcc800095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:45.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc867fc640 1 -- 172.21.15.125:0/1135516267 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800035f0 con 0x7fcc9013d360 2023-10-18T10:37:45.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc867fc640 1 -- 172.21.15.125:0/1135516267 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcc80003840 con 0x7fcc9013d360 2023-10-18T10:37:45.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc900ae120 con 0x7fcc9013d360 2023-10-18T10:37:45.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc867fc640 1 -- 172.21.15.125:0/1135516267 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80003c10 con 0x7fcc9013d360 2023-10-18T10:37:45.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.063+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc900ae6e0 con 0x7fcc9013d360 2023-10-18T10:37:45.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.064+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc900ae900 con 0x7fcc9013d360 2023-10-18T10:37:45.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.065+0000 7fcc867fc640 1 -- 172.21.15.125:0/1135516267 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fcc80003d70 con 0x7fcc9013d360 2023-10-18T10:37:45.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.065+0000 7fcc867fc640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fcc7406ac90 0x7fcc7406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.065+0000 7fcc867fc640 1 -- 172.21.15.125:0/1135516267 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6951+0+0 (secure 0 0 0) 0x7fcc800abe00 con 0x7fcc9013d360 2023-10-18T10:37:45.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.065+0000 7fcc95a8e640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fcc7406ac90 0x7fcc7406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.066+0000 7fcc95a8e640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fcc7406ac90 0x7fcc7406d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc90137270 tx=0x7fcc8c008040 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:45.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.069+0000 7fcc867fc640 1 -- 172.21.15.125:0/1135516267 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc80076d10 con 0x7fcc9013d360 2023-10-18T10:37:45.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.268+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc900ca150 con 0x7fcc9013d360 2023-10-18T10:37:45.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.268+0000 7fcc867fc640 1 -- 172.21.15.125:0/1135516267 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fcc900ca150 con 0x7fcc9013d360 2023-10-18T10:37:45.271 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:45.271 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3511459076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3511459076},{"type":"v1","addr":"172.21.15.139:6836","nonce":3511459076}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/984169411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":984169411},{"type":"v1","addr":"172.21.15.139:6837","nonce":984169411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2755714237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2755714237},{"type":"v1","addr":"172.21.15.125:6834","nonce":2755714237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:26:17.831071+0000","modified":"2023-10-18T10:26:19.850357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1934222457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1934222457},{"type":"v1","addr":"172.21.15.125:6835","nonce":1934222457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T10:37:45.272 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 16 2023-10-18T10:37:45.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fcc7406ac90 msgr2=0x7fcc7406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fcc7406ac90 0x7fcc7406d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc90137270 tx=0x7fcc8c008040 comp rx=0 tx=0).stop 2023-10-18T10:37:45.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc9013d360 msgr2=0x7fcc901f2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc9013d360 0x7fcc901f2e50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000b3c0 tx=0x7fcc800095a0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 shutdown_connections 2023-10-18T10:37:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fcc7406ac90 0x7fcc7406d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90140660 0x7fcc901f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc9013d360 0x7fcc901f2e50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 --2- 172.21.15.125:0/1135516267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900c9d00 0x7fcc900acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 >> 172.21.15.125:0/1135516267 conn(0x7fcc90069930 msgr2=0x7fcc90156350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:45.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 shutdown_connections 2023-10-18T10:37:45.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.270+0000 7fcc97cf2640 1 -- 172.21.15.125:0/1135516267 wait complete. 2023-10-18T10:37:45.283 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:37:45.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.385+0000 7fe0e9e56640 1 -- 172.21.15.125:0/2003220814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40cf340 msgr2=0x7fe0e40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.385+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/2003220814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40cf340 0x7fe0e40d1910 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005fd0 tx=0x7fe0d402bd90 comp rx=0 tx=0).stop 2023-10-18T10:37:45.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.385+0000 7fe0e9e56640 1 -- 172.21.15.125:0/2003220814 shutdown_connections 2023-10-18T10:37:45.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.385+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/2003220814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e415b2a0 0x7fe0e40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.385+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/2003220814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e40cf340 0x7fe0e40d1910 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.385+0000 7fe0e9e56640 1 -- 172.21.15.125:0/2003220814 >> 172.21.15.125:0/2003220814 conn(0x7fe0e40d0d60 msgr2=0x7fe0e40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:45.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.385+0000 7fe0e9e56640 1 -- 172.21.15.125:0/2003220814 shutdown_connections 2023-10-18T10:37:45.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.385+0000 7fe0e9e56640 1 -- 172.21.15.125:0/2003220814 wait complete. 2023-10-18T10:37:45.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.386+0000 7fe0e9e56640 1 Processor -- start 2023-10-18T10:37:45.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.386+0000 7fe0e9e56640 1 -- start start 2023-10-18T10:37:45.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e9e56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40ceb40 0x7fe0e40b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e9e56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cf340 0x7fe0e415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e9e56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e415b2a0 0x7fe0e4157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e9e56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e415bc10 con 0x7fe0e415b2a0 2023-10-18T10:37:45.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e9e56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40b67b0 con 0x7fe0e40ceb40 2023-10-18T10:37:45.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e9e56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e4158360 con 0x7fe0e40cf340 2023-10-18T10:37:45.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e415b2a0 0x7fe0e4157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e415b2a0 0x7fe0e4157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52610/0 (socket says 172.21.15.125:52610) 2023-10-18T10:37:45.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e3fff640 1 -- 172.21.15.125:0/25323608 learned_addr learned my addr 172.21.15.125:0/25323608 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:45.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e37fe640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40ceb40 0x7fe0e40b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e2ffd640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cf340 0x7fe0e415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e3fff640 1 -- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cf340 msgr2=0x7fe0e415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e3fff640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cf340 0x7fe0e415b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e3fff640 1 -- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40ceb40 msgr2=0x7fe0e40b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e3fff640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40ceb40 0x7fe0e40b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.387+0000 7fe0e3fff640 1 -- 172.21.15.125:0/25323608 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d4009560 con 0x7fe0e415b2a0 2023-10-18T10:37:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.388+0000 7fe0e3fff640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e415b2a0 0x7fe0e4157e20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c7d0 tx=0x7fe0dc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.388+0000 7fe0cbfff640 1 -- 172.21.15.125:0/25323608 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc003b70 con 0x7fe0e415b2a0 2023-10-18T10:37:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.388+0000 7fe0cbfff640 1 -- 172.21.15.125:0/25323608 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0dc003cf0 con 0x7fe0e415b2a0 2023-10-18T10:37:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.388+0000 7fe0cbfff640 1 -- 172.21.15.125:0/25323608 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc003ea0 con 0x7fe0e415b2a0 2023-10-18T10:37:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.388+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e415be40 con 0x7fe0e415b2a0 2023-10-18T10:37:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.388+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e415c400 con 0x7fe0e415b2a0 2023-10-18T10:37:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.389+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e40690e0 con 0x7fe0e415b2a0 2023-10-18T10:37:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.389+0000 7fe0cbfff640 1 -- 172.21.15.125:0/25323608 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe0dc00aa50 con 0x7fe0e415b2a0 2023-10-18T10:37:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.390+0000 7fe0cbfff640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe0d006ac90 0x7fe0d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.390+0000 7fe0cbfff640 1 -- 172.21.15.125:0/25323608 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6951+0+0 (secure 0 0 0) 0x7fe0dc0894c0 con 0x7fe0e415b2a0 2023-10-18T10:37:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.390+0000 7fe0e37fe640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe0d006ac90 0x7fe0d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.390+0000 7fe0e37fe640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe0d006ac90 0x7fe0d006d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4035000 tx=0x7fe0d4004730 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:45.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.394+0000 7fe0cbfff640 1 -- 172.21.15.125:0/25323608 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe0dc054480 con 0x7fe0e415b2a0 2023-10-18T10:37:45.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.564+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0e40d0b40 con 0x7fe0e415b2a0 2023-10-18T10:37:45.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.565+0000 7fe0cbfff640 1 -- 172.21.15.125:0/25323608 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18169 (secure 0 0 0) 0x7fe0dc057aa0 con 0x7fe0e415b2a0 2023-10-18T10:37:45.567 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:45.567 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":53,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:43.810407+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:26:16.213313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-18T10:26:16.801795+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":53,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2073919910":"2023-10-19T10:37:43.810352+0000","172.21.15.139:6833/609182116":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/2631710418":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/1401100783":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/219007869":"2023-10-19T10:37:43.810352+0000","172.21.15.139:6833/2976686360":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/3880096480":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6832/2976686360":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/1021953536":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/336930685":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/3718327380":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/485385349":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/768306585":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/557166225":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6832/609182116":"2023-10-19T10:37:43.810352+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":16,"length":1},{"begin":27,"length":1},{"begin":37,"length":1},{"begin":48,"length":1},{"begin":58,"length":1},{"begin":68,"length":1},{"begin":78,"length":1},{"begin":89,"length":1},{"begin":100,"length":1},{"begin":111,"length":1},{"begin":204,"length":18}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":111,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:37:45.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe0d006ac90 msgr2=0x7fe0d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe0d006ac90 0x7fe0d006d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4035000 tx=0x7fe0d4004730 comp rx=0 tx=0).stop 2023-10-18T10:37:45.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e415b2a0 msgr2=0x7fe0e4157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e415b2a0 0x7fe0e4157e20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c7d0 tx=0x7fe0dc00cc90 comp rx=0 tx=0).stop 2023-10-18T10:37:45.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 shutdown_connections 2023-10-18T10:37:45.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe0d006ac90 0x7fe0d006d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe0e415b2a0 0x7fe0e4157e20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0e40cf340 0x7fe0e415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 --2- 172.21.15.125:0/25323608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0e40ceb40 0x7fe0e40b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 >> 172.21.15.125:0/25323608 conn(0x7fe0e40d0d60 msgr2=0x7fe0e415a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 shutdown_connections 2023-10-18T10:37:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.567+0000 7fe0e9e56640 1 -- 172.21.15.125:0/25323608 wait complete. 2023-10-18T10:37:45.579 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:37:45.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.681+0000 7f9804ade640 1 -- 172.21.15.125:0/2364741910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 msgr2=0x7f97ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.681+0000 7f9804ade640 1 --2- 172.21.15.125:0/2364741910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 0x7f97ec004c40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009b50 tx=0x7f97ec0057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.681+0000 7f97fed9d640 1 -- 172.21.15.125:0/2364741910 reap_dead start 2023-10-18T10:37:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.681+0000 7f9804ade640 1 -- 172.21.15.125:0/2364741910 shutdown_connections 2023-10-18T10:37:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.681+0000 7f9804ade640 1 -- 172.21.15.125:0/2364741910 >> 172.21.15.125:0/2364741910 conn(0x7f9800069930 msgr2=0x7f9800069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.681+0000 7f9804ade640 1 -- 172.21.15.125:0/2364741910 shutdown_connections 2023-10-18T10:37:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.681+0000 7f9804ade640 1 -- 172.21.15.125:0/2364741910 wait complete. 2023-10-18T10:37:45.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.682+0000 7f9804ade640 1 Processor -- start 2023-10-18T10:37:45.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.682+0000 7f9804ade640 1 -- start start 2023-10-18T10:37:45.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f9804ade640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 0x7f980013eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f9804ade640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9800070a50 0x7f98000caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f9804ade640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9800151800 0x7f9800151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f9804ade640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980006f5b0 con 0x7f97ec0047c0 2023-10-18T10:37:45.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f9804ade640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800070610 con 0x7f9800151800 2023-10-18T10:37:45.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f9804ade640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980006f850 con 0x7f9800070a50 2023-10-18T10:37:45.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 0x7f980013eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 0x7f980013eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52614/0 (socket says 172.21.15.125:52614) 2023-10-18T10:37:45.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fe59c640 1 -- 172.21.15.125:0/505031796 learned_addr learned my addr 172.21.15.125:0/505031796 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:45.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fed9d640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9800151800 0x7f9800151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97f7fff640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9800070a50 0x7f98000caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fe59c640 1 -- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9800070a50 msgr2=0x7f98000caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fe59c640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9800070a50 0x7f98000caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fe59c640 1 -- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9800151800 msgr2=0x7f9800151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fe59c640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9800151800 0x7f9800151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.683+0000 7f97fe59c640 1 -- 172.21.15.125:0/505031796 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980006fb30 con 0x7f97ec0047c0 2023-10-18T10:37:45.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.684+0000 7f97fe59c640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 0x7f980013eb60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f97f0011ae0 tx=0x7f97f0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:45.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.684+0000 7f97f6ffd640 1 -- 172.21.15.125:0/505031796 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000fcb0 con 0x7f97ec0047c0 2023-10-18T10:37:45.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.684+0000 7f97f6ffd640 1 -- 172.21.15.125:0/505031796 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97f000fe10 con 0x7f97ec0047c0 2023-10-18T10:37:45.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.684+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980006fe10 con 0x7f97ec0047c0 2023-10-18T10:37:45.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.684+0000 7f97f6ffd640 1 -- 172.21.15.125:0/505031796 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0004280 con 0x7f97ec0047c0 2023-10-18T10:37:45.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.684+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98001521c0 con 0x7f97ec0047c0 2023-10-18T10:37:45.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.685+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98001f2470 con 0x7f97ec0047c0 2023-10-18T10:37:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.685+0000 7f97f6ffd640 1 -- 172.21.15.125:0/505031796 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f97f00035f0 con 0x7f97ec0047c0 2023-10-18T10:37:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.686+0000 7f97f6ffd640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f97e006ac90 0x7f97e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.686+0000 7f97f6ffd640 1 -- 172.21.15.125:0/505031796 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6951+0+0 (secure 0 0 0) 0x7f97f008a280 con 0x7f97ec0047c0 2023-10-18T10:37:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.686+0000 7f97f7fff640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f97e006ac90 0x7f97e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.686+0000 7f97f7fff640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f97e006ac90 0x7f97e006d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97e8036000 tx=0x7f97e80046c0 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.690+0000 7f97f6ffd640 1 -- 172.21.15.125:0/505031796 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97f00551c0 con 0x7f97ec0047c0 2023-10-18T10:37:45.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.895+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9800159a00 con 0x7f97ec0047c0 2023-10-18T10:37:45.897 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T10:37:45.896+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:37:45.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.897+0000 7f97f6ffd640 1 -- 172.21.15.125:0/505031796 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f97f00587e0 con 0x7f97ec0047c0 2023-10-18T10:37:45.899 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:37:45.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f97e006ac90 msgr2=0x7f97e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f97e006ac90 0x7f97e006d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97e8036000 tx=0x7f97e80046c0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 msgr2=0x7f980013eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:45.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 0x7f980013eb60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f97f0011ae0 tx=0x7f97f0009f10 comp rx=0 tx=0).stop 2023-10-18T10:37:45.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 shutdown_connections 2023-10-18T10:37:45.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f97e006ac90 0x7f97e006d150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9800151800 0x7f9800151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9800070a50 0x7f98000caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 --2- 172.21.15.125:0/505031796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97ec0047c0 0x7f980013eb60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:45.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 >> 172.21.15.125:0/505031796 conn(0x7f9800069930 msgr2=0x7f980015a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:45.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 shutdown_connections 2023-10-18T10:37:45.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:45.900+0000 7f9804ade640 1 -- 172.21.15.125:0/505031796 wait complete. 2023-10-18T10:37:45.913 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:37:45.935 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:37:46.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.015+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/896118577 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 msgr2=0x7fbfa006bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:46.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.015+0000 7fbfa4d9e640 1 --2- 172.21.15.125:0/896118577 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 0x7fbfa006bb60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009830 tx=0x7fbf9002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:37:46.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.016+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/896118577 shutdown_connections 2023-10-18T10:37:46.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.016+0000 7fbfa4d9e640 1 --2- 172.21.15.125:0/896118577 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 0x7fbfa006bb60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:46.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.016+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/896118577 >> 172.21.15.125:0/896118577 conn(0x7fbfa0069930 msgr2=0x7fbfa0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:46.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.016+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/896118577 shutdown_connections 2023-10-18T10:37:46.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.016+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/896118577 wait complete. 2023-10-18T10:37:46.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.017+0000 7fbfa4d9e640 1 Processor -- start 2023-10-18T10:37:46.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.017+0000 7fbfa4d9e640 1 -- start start 2023-10-18T10:37:46.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbfa4d9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbfa0070af0 0x7fbfa013af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:46.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbfa4d9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 0x7fbfa0158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:46.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbfa4d9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfa0141690 0x7fbfa0159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:46.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbfa4d9e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa0158dc0 con 0x7fbfa013e520 2023-10-18T10:37:46.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbfa4d9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01590a0 con 0x7fbfa0141690 2023-10-18T10:37:46.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbfa4d9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa0159380 con 0x7fbfa0070af0 2023-10-18T10:37:46.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 0x7fbfa0158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:46.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 0x7fbfa0158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52644/0 (socket says 172.21.15.125:52644) 2023-10-18T10:37:46.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 -- 172.21.15.125:0/3313538215 learned_addr learned my addr 172.21.15.125:0/3313538215 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:46.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf9ed9d640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfa0141690 0x7fbfa0159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:46.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf9e59c640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbfa0070af0 0x7fbfa013af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:46.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 -- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbfa0070af0 msgr2=0x7fbfa013af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:46.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbfa0070af0 0x7fbfa013af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:46.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 -- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfa0141690 msgr2=0x7fbfa0159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:46.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfa0141690 0x7fbfa0159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:46.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 -- 172.21.15.125:0/3313538215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90009490 con 0x7fbfa013e520 2023-10-18T10:37:46.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf9e59c640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbfa0070af0 0x7fbfa013af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:46.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf97fff640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 0x7fbfa0158450 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c3c0 tx=0x7fbf8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:46.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf96ffd640 1 -- 172.21.15.125:0/3313538215 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88004230 con 0x7fbfa013e520 2023-10-18T10:37:46.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf96ffd640 1 -- 172.21.15.125:0/3313538215 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbf880044a0 con 0x7fbfa013e520 2023-10-18T10:37:46.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa0159660 con 0x7fbfa013e520 2023-10-18T10:37:46.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbf96ffd640 1 -- 172.21.15.125:0/3313538215 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88003850 con 0x7fbfa013e520 2023-10-18T10:37:46.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.018+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa0158990 con 0x7fbfa013e520 2023-10-18T10:37:46.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.019+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa0159b80 con 0x7fbfa013e520 2023-10-18T10:37:46.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.020+0000 7fbf96ffd640 1 -- 172.21.15.125:0/3313538215 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 18) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbf880039b0 con 0x7fbfa013e520 2023-10-18T10:37:46.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.020+0000 7fbf96ffd640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fbf8006ac90 0x7fbf8006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:46.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.020+0000 7fbf96ffd640 1 -- 172.21.15.125:0/3313538215 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6993+0+0 (secure 0 0 0) 0x7fbf88088fa0 con 0x7fbfa013e520 2023-10-18T10:37:46.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.020+0000 7fbf9e59c640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fbf8006ac90 0x7fbf8006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:46.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.021+0000 7fbf9e59c640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fbf8006ac90 0x7fbf8006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbf90036000 tx=0x7fbf90004730 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:46.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.024+0000 7fbf96ffd640 1 -- 172.21.15.125:0/3313538215 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfa0159b80 con 0x7fbfa013e520 2023-10-18T10:37:46.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.229+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbfa0159b80 con 0x7fbfa013e520 2023-10-18T10:37:46.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.392+0000 7fbf96ffd640 1 -- 172.21.15.125:0/3313538215 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 19) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fbf88054180 con 0x7fbfa013e520 2023-10-18T10:37:46.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.899+0000 7fbf96ffd640 1 -- 172.21.15.125:0/3313538215 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf88073480 con 0x7fbfa013e520 2023-10-18T10:37:46.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fbf8006ac90 msgr2=0x7fbf8006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:46.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fbf8006ac90 0x7fbf8006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbf90036000 tx=0x7fbf90004730 comp rx=0 tx=0).stop 2023-10-18T10:37:46.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 msgr2=0x7fbfa0158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:46.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 0x7fbfa0158450 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c3c0 tx=0x7fbf8800c880 comp rx=0 tx=0).stop 2023-10-18T10:37:46.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 shutdown_connections 2023-10-18T10:37:46.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fbf8006ac90 0x7fbf8006d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfa0141690 0x7fbfa0159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbfa013e520 0x7fbfa0158450 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 --2- 172.21.15.125:0/3313538215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbfa0070af0 0x7fbfa013af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 >> 172.21.15.125:0/3313538215 conn(0x7fbfa0069930 msgr2=0x7fbfa015af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.901+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 shutdown_connections 2023-10-18T10:37:46.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:46.902+0000 7fbfa4d9e640 1 -- 172.21.15.125:0/3313538215 wait complete. 2023-10-18T10:37:46.917 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:37:47.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.068+0000 7f0f9b397640 1 -- 172.21.15.125:0/1812282770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f94140660 msgr2=0x7f0f940c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:47.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.068+0000 7f0f9b397640 1 --2- 172.21.15.125:0/1812282770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f94140660 0x7f0f940c97c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0f940ca400 tx=0x7f0f8802bdb0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.068+0000 7f0f9b397640 1 -- 172.21.15.125:0/1812282770 shutdown_connections 2023-10-18T10:37:47.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.068+0000 7f0f9b397640 1 --2- 172.21.15.125:0/1812282770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f94140660 0x7f0f940c97c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.068+0000 7f0f9b397640 1 --2- 172.21.15.125:0/1812282770 >> v2:172.21.15.139:3300/0 conn(0x7f0f9413d360 0x7f0f9415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.068+0000 7f0f9b397640 1 -- 172.21.15.125:0/1812282770 >> 172.21.15.125:0/1812282770 conn(0x7f0f94069930 msgr2=0x7f0f94069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:47.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.068+0000 7f0f9b397640 1 -- 172.21.15.125:0/1812282770 shutdown_connections 2023-10-18T10:37:47.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.068+0000 7f0f9b397640 1 -- 172.21.15.125:0/1812282770 wait complete. 2023-10-18T10:37:47.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.069+0000 7f0f9b397640 1 Processor -- start 2023-10-18T10:37:47.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.069+0000 7f0f9b397640 1 -- start start 2023-10-18T10:37:47.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f9b397640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f9406d170 0x7f0f94154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:47.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f9b397640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f9413d360 0x7f0f94133500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:47.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f9b397640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f94140660 0x7f0f94133a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:47.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f9b397640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94155350 con 0x7f0f9406d170 2023-10-18T10:37:47.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f9b397640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941367e0 con 0x7f0f9413d360 2023-10-18T10:37:47.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f9b397640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941555f0 con 0x7f0f94140660 2023-10-18T10:37:47.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f9406d170 0x7f0f94154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:47.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f9406d170 0x7f0f94154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52654/0 (socket says 172.21.15.125:52654) 2023-10-18T10:37:47.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 -- 172.21.15.125:0/96894553 learned_addr learned my addr 172.21.15.125:0/96894553 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:47.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f98932640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f9413d360 0x7f0f94133500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:47.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99934640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f94140660 0x7f0f94133a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 -- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f94140660 msgr2=0x7f0f94133a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f94140660 0x7f0f94133a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 -- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f9413d360 msgr2=0x7f0f94133500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f9413d360 0x7f0f94133500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 -- 172.21.15.125:0/96894553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009560 con 0x7f0f9406d170 2023-10-18T10:37:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99934640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f94140660 0x7f0f94133a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f99133640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f9406d170 0x7f0f94154b00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00d770 tx=0x7f0f8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.070+0000 7f0f91ffb640 1 -- 172.21.15.125:0/96894553 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c009c30 con 0x7f0f9406d170 2023-10-18T10:37:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.071+0000 7f0f91ffb640 1 -- 172.21.15.125:0/96894553 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f8c00de80 con 0x7f0f9406d170 2023-10-18T10:37:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.071+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f941558d0 con 0x7f0f9406d170 2023-10-18T10:37:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.071+0000 7f0f91ffb640 1 -- 172.21.15.125:0/96894553 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c005280 con 0x7f0f9406d170 2023-10-18T10:37:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.071+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f94136ac0 con 0x7f0f9406d170 2023-10-18T10:37:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.072+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f94155b10 con 0x7f0f9406d170 2023-10-18T10:37:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.072+0000 7f0f91ffb640 1 -- 172.21.15.125:0/96894553 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 19) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0f8c0053e0 con 0x7f0f9406d170 2023-10-18T10:37:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.072+0000 7f0f91ffb640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0f8006af00 0x7f0f8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.073+0000 7f0f91ffb640 1 -- 172.21.15.125:0/96894553 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6993+0+0 (secure 0 0 0) 0x7f0f8c089de0 con 0x7f0f9406d170 2023-10-18T10:37:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.073+0000 7f0f98932640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0f8006af00 0x7f0f8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.073+0000 7f0f98932640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0f8006af00 0x7f0f8006d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0f88035000 tx=0x7f0f88004730 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.077+0000 7f0f91ffb640 1 -- 172.21.15.125:0/96894553 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f8c054930 con 0x7f0f9406d170 2023-10-18T10:37:47.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.242+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f9415b970 con 0x7f0f9406d170 2023-10-18T10:37:47.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.900+0000 7f0f91ffb640 1 -- 172.21.15.125:0/96894553 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0f8c017020 con 0x7f0f9406d170 2023-10-18T10:37:47.902 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:37:47.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0f8006af00 msgr2=0x7f0f8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:47.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0f8006af00 0x7f0f8006d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0f88035000 tx=0x7f0f88004730 comp rx=0 tx=0).stop 2023-10-18T10:37:47.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f9406d170 msgr2=0x7f0f94154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:47.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f9406d170 0x7f0f94154b00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00d770 tx=0x7f0f8c00dc30 comp rx=0 tx=0).stop 2023-10-18T10:37:47.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 shutdown_connections 2023-10-18T10:37:47.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0f8006af00 0x7f0f8006d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f9413d360 0x7f0f94133500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f94140660 0x7f0f94133a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 --2- 172.21.15.125:0/96894553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0f9406d170 0x7f0f94154b00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:47.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 >> 172.21.15.125:0/96894553 conn(0x7f0f94069930 msgr2=0x7f0f941579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:47.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 shutdown_connections 2023-10-18T10:37:47.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:47.903+0000 7f0f9b397640 1 -- 172.21.15.125:0/96894553 wait complete. 2023-10-18T10:37:47.918 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:37:48.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.028+0000 7fca126b0640 1 -- 172.21.15.125:0/766029883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0cbd20 msgr2=0x7fca0c158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.028+0000 7fca126b0640 1 --2- 172.21.15.125:0/766029883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0cbd20 0x7fca0c158300 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00a940 tx=0x7fc9fc02d090 comp rx=0 tx=0).stop 2023-10-18T10:37:48.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.028+0000 7fca126b0640 1 -- 172.21.15.125:0/766029883 shutdown_connections 2023-10-18T10:37:48.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.028+0000 7fca126b0640 1 --2- 172.21.15.125:0/766029883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca0c0afa20 0x7fca0c0705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.028+0000 7fca126b0640 1 --2- 172.21.15.125:0/766029883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0cbd20 0x7fca0c158300 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.028+0000 7fca126b0640 1 -- 172.21.15.125:0/766029883 >> 172.21.15.125:0/766029883 conn(0x7fca0c069930 msgr2=0x7fca0c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:48.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.028+0000 7fca126b0640 1 -- 172.21.15.125:0/766029883 shutdown_connections 2023-10-18T10:37:48.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.028+0000 7fca126b0640 1 -- 172.21.15.125:0/766029883 wait complete. 2023-10-18T10:37:48.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.029+0000 7fca126b0640 1 Processor -- start 2023-10-18T10:37:48.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.029+0000 7fca126b0640 1 -- start start 2023-10-18T10:37:48.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca126b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c0afa20 0x7fca0c15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca126b0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0b0530 0x7fca0c1dfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca126b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca0c0cbd20 0x7fca0c1495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca126b0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c14a070 con 0x7fca0c0b0530 2023-10-18T10:37:48.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca126b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1e0210 con 0x7fca0c0afa20 2023-10-18T10:37:48.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca126b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1e1670 con 0x7fca0c0cbd20 2023-10-18T10:37:48.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0b0530 0x7fca0c1dfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0b0530 0x7fca0c1dfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52684/0 (socket says 172.21.15.125:52684) 2023-10-18T10:37:48.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 -- 172.21.15.125:0/2097584925 learned_addr learned my addr 172.21.15.125:0/2097584925 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:48.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0bfff640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c0afa20 0x7fca0c15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca10c4d640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca0c0cbd20 0x7fca0c1495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 -- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca0c0cbd20 msgr2=0x7fca0c1495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca0c0cbd20 0x7fca0c1495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 -- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c0afa20 msgr2=0x7fca0c15a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c0afa20 0x7fca0c15a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 -- 172.21.15.125:0/2097584925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc00a5a0 con 0x7fca0c0b0530 2023-10-18T10:37:48.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0bfff640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c0afa20 0x7fca0c15a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:48.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca10c4d640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca0c0cbd20 0x7fca0c1495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:48.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca0b7fe640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0b0530 0x7fca0c1dfb10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c890 tx=0x7fca0000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:48.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca08ff9640 1 -- 172.21.15.125:0/2097584925 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00004220 con 0x7fca0c0b0530 2023-10-18T10:37:48.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca08ff9640 1 -- 172.21.15.125:0/2097584925 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fca00004470 con 0x7fca0c0b0530 2023-10-18T10:37:48.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c070cf0 con 0x7fca0c0b0530 2023-10-18T10:37:48.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca08ff9640 1 -- 172.21.15.125:0/2097584925 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00007c10 con 0x7fca0c0b0530 2023-10-18T10:37:48.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.030+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c1e1910 con 0x7fca0c0b0530 2023-10-18T10:37:48.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.031+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c0690e0 con 0x7fca0c0b0530 2023-10-18T10:37:48.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.032+0000 7fca08ff9640 1 -- 172.21.15.125:0/2097584925 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 19) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fca00005020 con 0x7fca0c0b0530 2023-10-18T10:37:48.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.032+0000 7fca08ff9640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc9f806aed0 0x7fc9f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.032+0000 7fca08ff9640 1 -- 172.21.15.125:0/2097584925 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6555+0+0 (secure 0 0 0) 0x7fca0008a010 con 0x7fca0c0b0530 2023-10-18T10:37:48.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.033+0000 7fca0bfff640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc9f806aed0 0x7fc9f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.033+0000 7fca0bfff640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc9f806aed0 0x7fc9f806d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc037000 tx=0x7fc9fc004730 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.037+0000 7fca08ff9640 1 -- 172.21.15.125:0/2097584925 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca00054d00 con 0x7fca0c0b0530 2023-10-18T10:37:48.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.202+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fca0c06f700 con 0x7fca0c0b0530 2023-10-18T10:37:48.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.329+0000 7fca08ff9640 1 -- 172.21.15.125:0/2097584925 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7fca0001d020 con 0x7fca0c0b0530 2023-10-18T10:37:48.331 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T10:37:48.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc9f806aed0 msgr2=0x7fc9f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc9f806aed0 0x7fc9f806d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc037000 tx=0x7fc9fc004730 comp rx=0 tx=0).stop 2023-10-18T10:37:48.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0b0530 msgr2=0x7fca0c1dfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0b0530 0x7fca0c1dfb10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c890 tx=0x7fca0000cd50 comp rx=0 tx=0).stop 2023-10-18T10:37:48.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 shutdown_connections 2023-10-18T10:37:48.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc9f806aed0 0x7fc9f806d390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca0c0cbd20 0x7fca0c1495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca0c0b0530 0x7fca0c1dfb10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 --2- 172.21.15.125:0/2097584925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca0c0afa20 0x7fca0c15a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 >> 172.21.15.125:0/2097584925 conn(0x7fca0c069930 msgr2=0x7fca0c0ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:48.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 shutdown_connections 2023-10-18T10:37:48.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.331+0000 7fca126b0640 1 -- 172.21.15.125:0/2097584925 wait complete. 2023-10-18T10:37:48.351 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T10:37:48.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.454+0000 7fc08bf24640 1 -- 172.21.15.125:0/1141254482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0840cf340 msgr2=0x7fc0840d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.454+0000 7fc08bf24640 1 --2- 172.21.15.125:0/1141254482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0840cf340 0x7fc0840d1910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc080005fd0 tx=0x7fc08002c050 comp rx=0 tx=0).stop 2023-10-18T10:37:48.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.455+0000 7fc08bf24640 1 -- 172.21.15.125:0/1141254482 shutdown_connections 2023-10-18T10:37:48.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.455+0000 7fc08bf24640 1 --2- 172.21.15.125:0/1141254482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0840ceb40 0x7fc0840b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.455+0000 7fc08bf24640 1 --2- 172.21.15.125:0/1141254482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0840cf340 0x7fc0840d1910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.455+0000 7fc08bf24640 1 -- 172.21.15.125:0/1141254482 >> 172.21.15.125:0/1141254482 conn(0x7fc0840d0d60 msgr2=0x7fc0840d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:48.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.455+0000 7fc08bf24640 1 -- 172.21.15.125:0/1141254482 shutdown_connections 2023-10-18T10:37:48.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.455+0000 7fc08bf24640 1 -- 172.21.15.125:0/1141254482 wait complete. 2023-10-18T10:37:48.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.456+0000 7fc08bf24640 1 Processor -- start 2023-10-18T10:37:48.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.456+0000 7fc08bf24640 1 -- start start 2023-10-18T10:37:48.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.456+0000 7fc08bf24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0840ceb40 0x7fc0841f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.456+0000 7fc08bf24640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0840cf340 0x7fc0841f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.456+0000 7fc08bf24640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08415b2a0 0x7fc0840b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.456+0000 7fc08bf24640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840b49b0 con 0x7fc08415b2a0 2023-10-18T10:37:48.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.456+0000 7fc08bf24640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08415e610 con 0x7fc0840ceb40 2023-10-18T10:37:48.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.456+0000 7fc08bf24640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0841f2620 con 0x7fc0840cf340 2023-10-18T10:37:48.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08415b2a0 0x7fc0840b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08415b2a0 0x7fc0840b5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52696/0 (socket says 172.21.15.125:52696) 2023-10-18T10:37:48.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 -- 172.21.15.125:0/620404805 learned_addr learned my addr 172.21.15.125:0/620404805 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:48.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc089cc0640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0840ceb40 0x7fc0841f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc0894bf640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0840cf340 0x7fc0841f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 -- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0840cf340 msgr2=0x7fc0841f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0840cf340 0x7fc0841f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 -- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0840ceb40 msgr2=0x7fc0841f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0840ceb40 0x7fc0841f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 -- 172.21.15.125:0/620404805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c009490 con 0x7fc08415b2a0 2023-10-18T10:37:48.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08a4c1640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08415b2a0 0x7fc0840b5830 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e9b0 tx=0x7fc07c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:48.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc07a7fc640 1 -- 172.21.15.125:0/620404805 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00cd80 con 0x7fc08415b2a0 2023-10-18T10:37:48.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc07a7fc640 1 -- 172.21.15.125:0/620404805 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc07c004020 con 0x7fc08415b2a0 2023-10-18T10:37:48.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc080009840 con 0x7fc08415b2a0 2023-10-18T10:37:48.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc07a7fc640 1 -- 172.21.15.125:0/620404805 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00cee0 con 0x7fc08415b2a0 2023-10-18T10:37:48.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.457+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0840b0d50 con 0x7fc08415b2a0 2023-10-18T10:37:48.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.458+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0840b4b20 con 0x7fc08415b2a0 2023-10-18T10:37:48.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.459+0000 7fc07a7fc640 1 -- 172.21.15.125:0/620404805 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc07c0035f0 con 0x7fc08415b2a0 2023-10-18T10:37:48.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.459+0000 7fc07a7fc640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc06806af60 0x7fc06806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.459+0000 7fc07a7fc640 1 -- 172.21.15.125:0/620404805 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fc07c014070 con 0x7fc08415b2a0 2023-10-18T10:37:48.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.459+0000 7fc089cc0640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc06806af60 0x7fc06806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.460+0000 7fc089cc0640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc06806af60 0x7fc06806d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc080035000 tx=0x7fc080004ab0 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:48.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.463+0000 7fc07a7fc640 1 -- 172.21.15.125:0/620404805 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc07c058d80 con 0x7fc08415b2a0 2023-10-18T10:37:48.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.630+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7fc0840690e0 con 0x7fc08415b2a0 2023-10-18T10:37:48.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.634+0000 7fc07a7fc640 1 -- 172.21.15.125:0/620404805 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v34) v1 ==== 71+0+0 (secure 0 0 0) 0x7fc07c058910 con 0x7fc08415b2a0 2023-10-18T10:37:48.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc06806af60 msgr2=0x7fc06806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc06806af60 0x7fc06806d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc080035000 tx=0x7fc080004ab0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08415b2a0 msgr2=0x7fc0840b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08415b2a0 0x7fc0840b5830 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e9b0 tx=0x7fc07c00ee70 comp rx=0 tx=0).stop 2023-10-18T10:37:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 shutdown_connections 2023-10-18T10:37:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fc06806af60 0x7fc06806d420 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08415b2a0 0x7fc0840b5830 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0840cf340 0x7fc0841f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 --2- 172.21.15.125:0/620404805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0840ceb40 0x7fc0841f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 >> 172.21.15.125:0/620404805 conn(0x7fc0840d0d60 msgr2=0x7fc08406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:48.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 shutdown_connections 2023-10-18T10:37:48.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.636+0000 7fc08bf24640 1 -- 172.21.15.125:0/620404805 wait complete. 2023-10-18T10:37:48.649 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-18T10:37:48.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.752+0000 7fee7d9a9640 1 -- 172.21.15.125:0/852865102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780cf340 msgr2=0x7fee780d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.752+0000 7fee7d9a9640 1 --2- 172.21.15.125:0/852865102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780cf340 0x7fee780d1910 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fee68009830 tx=0x7fee6802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.753+0000 7fee7d9a9640 1 -- 172.21.15.125:0/852865102 shutdown_connections 2023-10-18T10:37:48.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.753+0000 7fee7d9a9640 1 --2- 172.21.15.125:0/852865102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780cf340 0x7fee780d1910 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.753+0000 7fee7d9a9640 1 -- 172.21.15.125:0/852865102 >> 172.21.15.125:0/852865102 conn(0x7fee780d0d60 msgr2=0x7fee780d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:48.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.753+0000 7fee7d9a9640 1 -- 172.21.15.125:0/852865102 shutdown_connections 2023-10-18T10:37:48.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.753+0000 7fee7d9a9640 1 -- 172.21.15.125:0/852865102 wait complete. 2023-10-18T10:37:48.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.754+0000 7fee7d9a9640 1 Processor -- start 2023-10-18T10:37:48.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.754+0000 7fee7d9a9640 1 -- start start 2023-10-18T10:37:48.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.754+0000 7fee7d9a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780ceb40 0x7fee7815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.754+0000 7fee7d9a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee780cf340 0x7fee781f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.754+0000 7fee7d9a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee7815b2a0 0x7fee7806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.754+0000 7fee7d9a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780b3af0 con 0x7fee780ceb40 2023-10-18T10:37:48.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.754+0000 7fee7d9a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7815c550 con 0x7fee7815b2a0 2023-10-18T10:37:48.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.754+0000 7fee7d9a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee780b3d40 con 0x7fee780cf340 2023-10-18T10:37:48.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780ceb40 0x7fee7815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780ceb40 0x7fee7815cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52706/0 (socket says 172.21.15.125:52706) 2023-10-18T10:37:48.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 -- 172.21.15.125:0/2974612709 learned_addr learned my addr 172.21.15.125:0/2974612709 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:48.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee777fe640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee7815b2a0 0x7fee7806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee6ffff640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee780cf340 0x7fee781f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 -- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee780cf340 msgr2=0x7fee781f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee780cf340 0x7fee781f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 -- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee7815b2a0 msgr2=0x7fee7806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:48.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee7815b2a0 0x7fee7806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:48.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 -- 172.21.15.125:0/2974612709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68009490 con 0x7fee780ceb40 2023-10-18T10:37:48.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee777fe640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee7815b2a0 0x7fee7806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:48.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee76ffd640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780ceb40 0x7fee7815cd20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fee68009950 tx=0x7fee680038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:48.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee74ff9640 1 -- 172.21.15.125:0/2974612709 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68003ea0 con 0x7fee780ceb40 2023-10-18T10:37:48.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee74ff9640 1 -- 172.21.15.125:0/2974612709 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee68005020 con 0x7fee780ceb40 2023-10-18T10:37:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee780b4020 con 0x7fee780ceb40 2023-10-18T10:37:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee74ff9640 1 -- 172.21.15.125:0/2974612709 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68005410 con 0x7fee780ceb40 2023-10-18T10:37:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.755+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7815d260 con 0x7fee780ceb40 2023-10-18T10:37:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.756+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee780b4540 con 0x7fee780ceb40 2023-10-18T10:37:48.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.757+0000 7fee74ff9640 1 -- 172.21.15.125:0/2974612709 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fee680047a0 con 0x7fee780ceb40 2023-10-18T10:37:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.757+0000 7fee74ff9640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fee5806af90 0x7fee5806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.757+0000 7fee74ff9640 1 -- 172.21.15.125:0/2974612709 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fee680ac570 con 0x7fee780ceb40 2023-10-18T10:37:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.757+0000 7fee6ffff640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fee5806af90 0x7fee5806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:48.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.758+0000 7fee6ffff640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fee5806af90 0x7fee5806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee7809e5a0 tx=0x7fee60009040 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:48.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.761+0000 7fee74ff9640 1 -- 172.21.15.125:0/2974612709 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee780b4540 con 0x7fee780ceb40 2023-10-18T10:37:48.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:48.997+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fee780b4540 con 0x7fee780ceb40 2023-10-18T10:37:49.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.909+0000 7fee74ff9640 1 -- 172.21.15.125:0/2974612709 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v460) v1 ==== 158+0+0 (secure 0 0 0) 0x7fee780b4540 con 0x7fee780ceb40 2023-10-18T10:37:49.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fee5806af90 msgr2=0x7fee5806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:49.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fee5806af90 0x7fee5806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee7809e5a0 tx=0x7fee60009040 comp rx=0 tx=0).stop 2023-10-18T10:37:49.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780ceb40 msgr2=0x7fee7815cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:49.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780ceb40 0x7fee7815cd20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fee68009950 tx=0x7fee680038a0 comp rx=0 tx=0).stop 2023-10-18T10:37:49.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 shutdown_connections 2023-10-18T10:37:49.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fee5806af90 0x7fee5806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:49.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee7815b2a0 0x7fee7806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:49.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee780cf340 0x7fee781f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:49.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 --2- 172.21.15.125:0/2974612709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee780ceb40 0x7fee7815cd20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:49.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 >> 172.21.15.125:0/2974612709 conn(0x7fee780d0d60 msgr2=0x7fee780ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:49.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 shutdown_connections 2023-10-18T10:37:49.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:49.912+0000 7fee7d9a9640 1 -- 172.21.15.125:0/2974612709 wait complete. 2023-10-18T10:37:49.930 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-10-18T10:37:49.930 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-10-18T10:37:49.931 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T10:37:49.931 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:37:50.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.038+0000 7f8f52490640 1 -- 172.21.15.125:0/3180125312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0cbd20 msgr2=0x7f8f4c158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.038+0000 7f8f52490640 1 --2- 172.21.15.125:0/3180125312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0cbd20 0x7f8f4c158300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005fd0 tx=0x7f8f3c02be10 comp rx=0 tx=0).stop 2023-10-18T10:37:50.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.038+0000 7f8f52490640 1 -- 172.21.15.125:0/3180125312 shutdown_connections 2023-10-18T10:37:50.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.038+0000 7f8f52490640 1 --2- 172.21.15.125:0/3180125312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f4c0afa20 0x7f8f4c0705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.038+0000 7f8f52490640 1 --2- 172.21.15.125:0/3180125312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0cbd20 0x7f8f4c158300 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.038+0000 7f8f52490640 1 -- 172.21.15.125:0/3180125312 >> 172.21.15.125:0/3180125312 conn(0x7f8f4c069930 msgr2=0x7f8f4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:50.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.038+0000 7f8f52490640 1 -- 172.21.15.125:0/3180125312 shutdown_connections 2023-10-18T10:37:50.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.038+0000 7f8f52490640 1 -- 172.21.15.125:0/3180125312 wait complete. 2023-10-18T10:37:50.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.039+0000 7f8f52490640 1 Processor -- start 2023-10-18T10:37:50.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.039+0000 7f8f52490640 1 -- start start 2023-10-18T10:37:50.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f52490640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0afa20 0x7f8f4c146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f52490640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f4c0b0530 0x7f8f4c1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f52490640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f4c0cbd20 0x7f8f4c1f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f52490640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c148300 con 0x7f8f4c0afa20 2023-10-18T10:37:50.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f52490640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1485e0 con 0x7f8f4c0b0530 2023-10-18T10:37:50.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f52490640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1488c0 con 0x7f8f4c0cbd20 2023-10-18T10:37:50.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0afa20 0x7f8f4c146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0afa20 0x7f8f4c146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52728/0 (socket says 172.21.15.125:52728) 2023-10-18T10:37:50.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4bfff640 1 -- 172.21.15.125:0/3381163435 learned_addr learned my addr 172.21.15.125:0/3381163435 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:50.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4b7fe640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f4c0b0530 0x7f8f4c1f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f50a2d640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f4c0cbd20 0x7f8f4c1f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4bfff640 1 -- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f4c0cbd20 msgr2=0x7f8f4c1f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4bfff640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f4c0cbd20 0x7f8f4c1f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4bfff640 1 -- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f4c0b0530 msgr2=0x7f8f4c1f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4bfff640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f4c0b0530 0x7f8f4c1f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f4bfff640 1 -- 172.21.15.125:0/3381163435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c009560 con 0x7f8f4c0afa20 2023-10-18T10:37:50.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.040+0000 7f8f50a2d640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f4c0cbd20 0x7f8f4c1f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:50.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.041+0000 7f8f4bfff640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0afa20 0x7f8f4c146310 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c006010 tx=0x7f8f3c02dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:50.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.041+0000 7f8f48ff9640 1 -- 172.21.15.125:0/3381163435 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0035f0 con 0x7f8f4c0afa20 2023-10-18T10:37:50.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.041+0000 7f8f48ff9640 1 -- 172.21.15.125:0/3381163435 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8f3c003840 con 0x7f8f4c0afa20 2023-10-18T10:37:50.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.041+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c070cf0 con 0x7f8f4c0afa20 2023-10-18T10:37:50.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.041+0000 7f8f48ff9640 1 -- 172.21.15.125:0/3381163435 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c003c50 con 0x7f8f4c0afa20 2023-10-18T10:37:50.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.041+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c148e80 con 0x7f8f4c0afa20 2023-10-18T10:37:50.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.042+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c148070 con 0x7f8f4c0afa20 2023-10-18T10:37:50.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.042+0000 7f8f48ff9640 1 -- 172.21.15.125:0/3381163435 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8f3c03e450 con 0x7f8f4c0afa20 2023-10-18T10:37:50.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.043+0000 7f8f48ff9640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8f3806af60 0x7f8f3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.043+0000 7f8f48ff9640 1 -- 172.21.15.125:0/3381163435 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f8f3c0ac5e0 con 0x7f8f4c0afa20 2023-10-18T10:37:50.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.043+0000 7f8f4b7fe640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8f3806af60 0x7f8f3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.043+0000 7f8f4b7fe640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8f3806af60 0x7f8f3806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005fd0 tx=0x7f8f40005850 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:50.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.047+0000 7f8f48ff9640 1 -- 172.21.15.125:0/3381163435 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f3c0774b0 con 0x7f8f4c0afa20 2023-10-18T10:37:50.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.213+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f4c0cc1e0 con 0x7f8f4c0afa20 2023-10-18T10:37:50.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.214+0000 7f8f48ff9640 1 -- 172.21.15.125:0/3381163435 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+14502 (secure 0 0 0) 0x7f8f3c003db0 con 0x7f8f4c0afa20 2023-10-18T10:37:50.216 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:50.216 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":56,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:48.328866+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6835/1934222457":"2023-10-19T10:37:45.896954+0000","172.21.15.139:0/2073919910":"2023-10-19T10:37:43.810352+0000","172.21.15.139:6833/609182116":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/2631710418":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/1401100783":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/219007869":"2023-10-19T10:37:43.810352+0000","172.21.15.139:6833/2976686360":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/3880096480":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6832/2976686360":"2023-10-19T10:26:33.551736+0000","172.21.15.139:0/1021953536":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/336930685":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/3718327380":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/485385349":"2023-10-19T10:37:43.810352+0000","172.21.15.139:0/768306585":"2023-10-19T10:37:43.810352+0000","172.21.15.125:6833/1934222457":"2023-10-19T10:37:45.896954+0000","172.21.15.139:0/557166225":"2023-10-19T10:26:33.551736+0000","172.21.15.139:6832/609182116":"2023-10-19T10:37:43.810352+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":16,"length":1},{"begin":27,"length":1},{"begin":37,"length":1},{"begin":48,"length":1},{"begin":58,"length":1},{"begin":68,"length":1},{"begin":78,"length":1},{"begin":89,"length":1},{"begin":100,"length":1},{"begin":111,"length":1},{"begin":204,"length":18}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:37:50.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8f3806af60 msgr2=0x7f8f3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8f3806af60 0x7f8f3806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005fd0 tx=0x7f8f40005850 comp rx=0 tx=0).stop 2023-10-18T10:37:50.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0afa20 msgr2=0x7f8f4c146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0afa20 0x7f8f4c146310 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c006010 tx=0x7f8f3c02dbb0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 shutdown_connections 2023-10-18T10:37:50.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8f3806af60 0x7f8f3806d420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f4c0b0530 0x7f8f4c1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f4c0cbd20 0x7f8f4c1f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 --2- 172.21.15.125:0/3381163435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f4c0afa20 0x7f8f4c146310 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 >> 172.21.15.125:0/3381163435 conn(0x7f8f4c069930 msgr2=0x7f8f4c157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:50.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 shutdown_connections 2023-10-18T10:37:50.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.216+0000 7f8f52490640 1 -- 172.21.15.125:0/3381163435 wait complete. 2023-10-18T10:37:50.228 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:37:50.229 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:37:50.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.332+0000 7fabcdf33640 1 -- 172.21.15.125:0/771446231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabb00052a0 msgr2=0x7fabb000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.332+0000 7fabcdf33640 1 --2- 172.21.15.125:0/771446231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabb00052a0 0x7fabb000b980 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fabc0006fa0 tx=0x7fabb00057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.333+0000 7fabcdf33640 1 -- 172.21.15.125:0/771446231 shutdown_connections 2023-10-18T10:37:50.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.333+0000 7fabcdf33640 1 --2- 172.21.15.125:0/771446231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabb00052a0 0x7fabb000b980 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.333+0000 7fabcdf33640 1 -- 172.21.15.125:0/771446231 >> 172.21.15.125:0/771446231 conn(0x7fabc80d0d60 msgr2=0x7fabc80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:50.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.333+0000 7fabcdf33640 1 -- 172.21.15.125:0/771446231 shutdown_connections 2023-10-18T10:37:50.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.333+0000 7fabcdf33640 1 -- 172.21.15.125:0/771446231 wait complete. 2023-10-18T10:37:50.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.334+0000 7fabcdf33640 1 Processor -- start 2023-10-18T10:37:50.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.334+0000 7fabcdf33640 1 -- start start 2023-10-18T10:37:50.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabcdf33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb00052a0 0x7fabc815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabcdf33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabc81577b0 0x7fabc80b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabcdf33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabc80b5220 0x7fabc80ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabcdf33640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc8157c30 con 0x7fabc80b5220 2023-10-18T10:37:50.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabcdf33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc815de00 con 0x7fabb00052a0 2023-10-18T10:37:50.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabcdf33640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc815d1f0 con 0x7fabc81577b0 2023-10-18T10:37:50.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabc80b5220 0x7fabc80ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabc80b5220 0x7fabc80ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52760/0 (socket says 172.21.15.125:52760) 2023-10-18T10:37:50.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 -- 172.21.15.125:0/2379475979 learned_addr learned my addr 172.21.15.125:0/2379475979 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:50.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc77fe640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb00052a0 0x7fabc815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc6ffd640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabc81577b0 0x7fabc80b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 -- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabc81577b0 msgr2=0x7fabc80b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabc81577b0 0x7fabc80b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 -- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb00052a0 msgr2=0x7fabc815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb00052a0 0x7fabc815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 -- 172.21.15.125:0/2379475979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc815d4d0 con 0x7fabc80b5220 2023-10-18T10:37:50.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc6ffd640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabc81577b0 0x7fabc80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:50.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabc7fff640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabc80b5220 0x7fabc80ae5d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fabc0006fd0 tx=0x7fabc0001580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:50.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabaffff640 1 -- 172.21.15.125:0/2379475979 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0006070 con 0x7fabc80b5220 2023-10-18T10:37:50.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabaffff640 1 -- 172.21.15.125:0/2379475979 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fabc00061d0 con 0x7fabc80b5220 2023-10-18T10:37:50.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc815d7b0 con 0x7fabc80b5220 2023-10-18T10:37:50.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabaffff640 1 -- 172.21.15.125:0/2379475979 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00065c0 con 0x7fabc80b5220 2023-10-18T10:37:50.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.335+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc815cd80 con 0x7fabc80b5220 2023-10-18T10:37:50.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.336+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc815df90 con 0x7fabc80b5220 2023-10-18T10:37:50.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.337+0000 7fabaffff640 1 -- 172.21.15.125:0/2379475979 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fabc0031030 con 0x7fabc80b5220 2023-10-18T10:37:50.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.337+0000 7fabaffff640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fabb406af90 0x7fabb406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.337+0000 7fabaffff640 1 -- 172.21.15.125:0/2379475979 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fabc00aa950 con 0x7fabc80b5220 2023-10-18T10:37:50.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.337+0000 7fabc77fe640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fabb406af90 0x7fabb406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.338+0000 7fabc77fe640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fabb406af90 0x7fabb406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fabb8006fd0 tx=0x7fabb8006d40 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:50.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.341+0000 7fabaffff640 1 -- 172.21.15.125:0/2379475979 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabc00755d0 con 0x7fabc80b5220 2023-10-18T10:37:50.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.509+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabc80690e0 con 0x7fabc80b5220 2023-10-18T10:37:50.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.509+0000 7fabaffff640 1 -- 172.21.15.125:0/2379475979 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabc0042020 con 0x7fabc80b5220 2023-10-18T10:37:50.511 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:37:50.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fabb406af90 msgr2=0x7fabb406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fabb406af90 0x7fabb406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fabb8006fd0 tx=0x7fabb8006d40 comp rx=0 tx=0).stop 2023-10-18T10:37:50.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabc80b5220 msgr2=0x7fabc80ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabc80b5220 0x7fabc80ae5d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fabc0006fd0 tx=0x7fabc0001580 comp rx=0 tx=0).stop 2023-10-18T10:37:50.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 shutdown_connections 2023-10-18T10:37:50.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fabb406af90 0x7fabb406d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabc80b5220 0x7fabc80ae5d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabc81577b0 0x7fabc80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 --2- 172.21.15.125:0/2379475979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabb00052a0 0x7fabc815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 >> 172.21.15.125:0/2379475979 conn(0x7fabc80d0d60 msgr2=0x7fabc80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:50.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 shutdown_connections 2023-10-18T10:37:50.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.511+0000 7fabcdf33640 1 -- 172.21.15.125:0/2379475979 wait complete. 2023-10-18T10:37:50.523 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-18T10:37:50.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.623+0000 7f7753fc0640 1 -- 172.21.15.125:0/605975718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 msgr2=0x7f774c0cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.623+0000 7f7753fc0640 1 --2- 172.21.15.125:0/605975718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 0x7f774c0cc370 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f774c0afcb0 tx=0x7f774802bda0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.624+0000 7f7753fc0640 1 -- 172.21.15.125:0/605975718 shutdown_connections 2023-10-18T10:37:50.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.624+0000 7f7753fc0640 1 --2- 172.21.15.125:0/605975718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f774c070a60 0x7f774c15c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.624+0000 7f7753fc0640 1 --2- 172.21.15.125:0/605975718 >> v2:172.21.15.139:3300/0 conn(0x7f774c0c9250 0x7f774c070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.624+0000 7f7753fc0640 1 --2- 172.21.15.125:0/605975718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 0x7f774c0cc370 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.624+0000 7f7753fc0640 1 -- 172.21.15.125:0/605975718 >> 172.21.15.125:0/605975718 conn(0x7f774c0aec40 msgr2=0x7f774c0af050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:50.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.624+0000 7f7753fc0640 1 -- 172.21.15.125:0/605975718 shutdown_connections 2023-10-18T10:37:50.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.624+0000 7f7753fc0640 1 -- 172.21.15.125:0/605975718 wait complete. 2023-10-18T10:37:50.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.625+0000 7f7753fc0640 1 Processor -- start 2023-10-18T10:37:50.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.625+0000 7f7753fc0640 1 -- start start 2023-10-18T10:37:50.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.625+0000 7f7753fc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f774c070a60 0x7f774c14b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.625+0000 7f7753fc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f774c0c9250 0x7f774c151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.625+0000 7f7753fc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 0x7f774c150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.625+0000 7f7753fc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c06c850 con 0x7f774c158aa0 2023-10-18T10:37:50.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.625+0000 7f7753fc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c06bbd0 con 0x7f774c070a60 2023-10-18T10:37:50.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.625+0000 7f7753fc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774c06beb0 con 0x7f774c0c9250 2023-10-18T10:37:50.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 0x7f774c150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 0x7f774c150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52786/0 (socket says 172.21.15.125:52786) 2023-10-18T10:37:50.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 -- 172.21.15.125:0/2427828953 learned_addr learned my addr 172.21.15.125:0/2427828953 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:50.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775155b640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f774c0c9250 0x7f774c151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f7751d5c640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f774c070a60 0x7f774c14b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 -- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f774c0c9250 msgr2=0x7f774c151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f774c0c9250 0x7f774c151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 -- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f774c070a60 msgr2=0x7f774c14b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f774c070a60 0x7f774c14b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 -- 172.21.15.125:0/2427828953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7748009560 con 0x7f774c158aa0 2023-10-18T10:37:50.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f7751d5c640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f774c070a60 0x7f774c14b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:50.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f775255d640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 0x7f774c150e80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f774400e980 tx=0x7f774400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:50.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f77427fc640 1 -- 172.21.15.125:0/2427828953 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744004d20 con 0x7f774c158aa0 2023-10-18T10:37:50.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f77427fc640 1 -- 172.21.15.125:0/2427828953 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f774400cb20 con 0x7f774c158aa0 2023-10-18T10:37:50.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774c06c190 con 0x7f774c158aa0 2023-10-18T10:37:50.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f77427fc640 1 -- 172.21.15.125:0/2427828953 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744005020 con 0x7f774c158aa0 2023-10-18T10:37:50.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.626+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c14b950 con 0x7f774c158aa0 2023-10-18T10:37:50.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.627+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774c06c990 con 0x7f774c158aa0 2023-10-18T10:37:50.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.628+0000 7f77427fc640 1 -- 172.21.15.125:0/2427828953 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7744005180 con 0x7f774c158aa0 2023-10-18T10:37:50.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.628+0000 7f77427fc640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f773006af60 0x7f773006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:50.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.628+0000 7f77427fc640 1 -- 172.21.15.125:0/2427828953 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f774408a620 con 0x7f774c158aa0 2023-10-18T10:37:50.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.628+0000 7f7751d5c640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f773006af60 0x7f773006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:50.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.629+0000 7f7751d5c640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f773006af60 0x7f773006d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7748035000 tx=0x7f7748004730 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:50.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.632+0000 7f77427fc640 1 -- 172.21.15.125:0/2427828953 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f774c06c990 con 0x7f774c158aa0 2023-10-18T10:37:50.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.859+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f774c06c990 con 0x7f774c158aa0 2023-10-18T10:37:50.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.910+0000 7f77427fc640 1 -- 172.21.15.125:0/2427828953 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v461) v1 ==== 157+0+0 (secure 0 0 0) 0x7f774c06c990 con 0x7f774c158aa0 2023-10-18T10:37:50.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f773006af60 msgr2=0x7f773006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f773006af60 0x7f773006d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7748035000 tx=0x7f7748004730 comp rx=0 tx=0).stop 2023-10-18T10:37:50.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 msgr2=0x7f774c150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:50.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 0x7f774c150e80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f774400e980 tx=0x7f774400ee40 comp rx=0 tx=0).stop 2023-10-18T10:37:50.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 shutdown_connections 2023-10-18T10:37:50.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f773006af60 0x7f773006d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f774c158aa0 0x7f774c150e80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f774c0c9250 0x7f774c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 --2- 172.21.15.125:0/2427828953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f774c070a60 0x7f774c14b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:50.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 >> 172.21.15.125:0/2427828953 conn(0x7f774c0aec40 msgr2=0x7f774c14d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:50.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 shutdown_connections 2023-10-18T10:37:50.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:50.913+0000 7f7753fc0640 1 -- 172.21.15.125:0/2427828953 wait complete. 2023-10-18T10:37:50.925 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T10:37:51.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.025+0000 7f716c0e5640 1 -- 172.21.15.125:0/1445282830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f716406af00 msgr2=0x7f716406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.025+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1445282830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f716406af00 0x7f716406b2e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f715c005fd0 tx=0x7f715c02c050 comp rx=0 tx=0).stop 2023-10-18T10:37:51.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.025+0000 7f716c0e5640 1 -- 172.21.15.125:0/1445282830 shutdown_connections 2023-10-18T10:37:51.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.025+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1445282830 >> v2:172.21.15.139:3300/0 conn(0x7f71640c96b0 0x7f71640c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.025+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1445282830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f716406af00 0x7f716406b2e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.025+0000 7f716c0e5640 1 -- 172.21.15.125:0/1445282830 >> 172.21.15.125:0/1445282830 conn(0x7f7164069930 msgr2=0x7f7164069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:51.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.025+0000 7f716c0e5640 1 -- 172.21.15.125:0/1445282830 shutdown_connections 2023-10-18T10:37:51.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.025+0000 7f716c0e5640 1 -- 172.21.15.125:0/1445282830 wait complete. 2023-10-18T10:37:51.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.026+0000 7f716c0e5640 1 Processor -- start 2023-10-18T10:37:51.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.026+0000 7f716c0e5640 1 -- start start 2023-10-18T10:37:51.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716c0e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f716406af00 0x7f71640a8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716c0e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71640ab500 0x7f71640b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716c0e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71640c96b0 0x7f716412f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716c0e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71640a6800 con 0x7f71640c96b0 2023-10-18T10:37:51.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716c0e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716412cc10 con 0x7f71640ab500 2023-10-18T10:37:51.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716c0e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71640aedb0 con 0x7f716406af00 2023-10-18T10:37:51.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716a682640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71640c96b0 0x7f716412f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716a682640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71640c96b0 0x7f716412f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52792/0 (socket says 172.21.15.125:52792) 2023-10-18T10:37:51.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716a682640 1 -- 172.21.15.125:0/1909051001 learned_addr learned my addr 172.21.15.125:0/1909051001 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:51.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f7169680640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71640ab500 0x7f71640b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716a682640 1 -- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f716406af00 msgr2=0x7f71640a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716a682640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f716406af00 0x7f71640a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716a682640 1 -- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71640ab500 msgr2=0x7f71640b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716a682640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71640ab500 0x7f71640b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.027+0000 7f716a682640 1 -- 172.21.15.125:0/1909051001 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c009840 con 0x7f71640c96b0 2023-10-18T10:37:51.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.028+0000 7f716a682640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71640c96b0 0x7f716412f0f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f716000c430 tx=0x7f716000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:51.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.028+0000 7f715a7fc640 1 -- 172.21.15.125:0/1909051001 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160004230 con 0x7f71640c96b0 2023-10-18T10:37:51.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.028+0000 7f715a7fc640 1 -- 172.21.15.125:0/1909051001 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f71600044a0 con 0x7f71640c96b0 2023-10-18T10:37:51.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.028+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71640a81a0 con 0x7f71640c96b0 2023-10-18T10:37:51.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.028+0000 7f715a7fc640 1 -- 172.21.15.125:0/1909051001 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160003850 con 0x7f71640c96b0 2023-10-18T10:37:51.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.028+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71640a7ec0 con 0x7f71640c96b0 2023-10-18T10:37:51.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.029+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71640690e0 con 0x7f71640c96b0 2023-10-18T10:37:51.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.029+0000 7f715a7fc640 1 -- 172.21.15.125:0/1909051001 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f71600039b0 con 0x7f71640c96b0 2023-10-18T10:37:51.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.029+0000 7f715a7fc640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f714806af60 0x7f714806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.030+0000 7f715a7fc640 1 -- 172.21.15.125:0/1909051001 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f7160089180 con 0x7f71640c96b0 2023-10-18T10:37:51.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.030+0000 7f7169e81640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f714806af60 0x7f714806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.030+0000 7f7169e81640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f714806af60 0x7f714806d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f715c035000 tx=0x7f715c004a40 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:51.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.034+0000 7f715a7fc640 1 -- 172.21.15.125:0/1909051001 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7160053f90 con 0x7f71640c96b0 2023-10-18T10:37:51.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.198+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f71640aeef0 con 0x7f71640c96b0 2023-10-18T10:37:51.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.199+0000 7f715a7fc640 1 -- 172.21.15.125:0/1909051001 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f71600178c0 con 0x7f71640c96b0 2023-10-18T10:37:51.201 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T10:37:51.201 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T10:37:51.201 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T10:37:51.202 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T10:37:51.202 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T10:37:51.202 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T10:37:51.202 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T10:37:51.202 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T10:37:51.203 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T10:37:51.203 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T10:37:51.203 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T10:37:51.203 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T10:37:51.203 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T10:37:51.204 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T10:37:51.204 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T10:37:51.204 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T10:37:51.204 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T10:37:51.204 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T10:37:51.205 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T10:37:51.205 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T10:37:51.205 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T10:37:51.205 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T10:37:51.205 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T10:37:51.206 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T10:37:51.206 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T10:37:51.206 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T10:37:51.206 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T10:37:51.206 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T10:37:51.207 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T10:37:51.207 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T10:37:51.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f714806af60 msgr2=0x7f714806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f714806af60 0x7f714806d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f715c035000 tx=0x7f715c004a40 comp rx=0 tx=0).stop 2023-10-18T10:37:51.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71640c96b0 msgr2=0x7f716412f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71640c96b0 0x7f716412f0f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f716000c430 tx=0x7f716000c8f0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 shutdown_connections 2023-10-18T10:37:51.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f714806af60 0x7f714806d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71640ab500 0x7f71640b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71640c96b0 0x7f716412f0f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 --2- 172.21.15.125:0/1909051001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f716406af00 0x7f71640a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 >> 172.21.15.125:0/1909051001 conn(0x7f7164069930 msgr2=0x7f716413b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:51.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 shutdown_connections 2023-10-18T10:37:51.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.201+0000 7f716c0e5640 1 -- 172.21.15.125:0/1909051001 wait complete. 2023-10-18T10:37:51.213 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:37:51.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.316+0000 7f4b084c9640 1 -- 172.21.15.125:0/3256576100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000b0530 msgr2=0x7f4b000b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.316+0000 7f4b084c9640 1 --2- 172.21.15.125:0/3256576100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000b0530 0x7f4b000b09c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009830 tx=0x7f4af802bd60 comp rx=0 tx=0).stop 2023-10-18T10:37:51.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.316+0000 7f4b084c9640 1 -- 172.21.15.125:0/3256576100 shutdown_connections 2023-10-18T10:37:51.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.316+0000 7f4b084c9640 1 --2- 172.21.15.125:0/3256576100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000b0530 0x7f4b000b09c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.316+0000 7f4b084c9640 1 --2- 172.21.15.125:0/3256576100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b000afa20 0x7f4b00158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.316+0000 7f4b084c9640 1 -- 172.21.15.125:0/3256576100 >> 172.21.15.125:0/3256576100 conn(0x7f4b00069930 msgr2=0x7f4b00069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:51.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.316+0000 7f4b084c9640 1 -- 172.21.15.125:0/3256576100 shutdown_connections 2023-10-18T10:37:51.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.316+0000 7f4b084c9640 1 -- 172.21.15.125:0/3256576100 wait complete. 2023-10-18T10:37:51.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.317+0000 7f4b084c9640 1 Processor -- start 2023-10-18T10:37:51.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.317+0000 7f4b084c9640 1 -- start start 2023-10-18T10:37:51.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b084c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000afa20 0x7f4b00125200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b084c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b000b0530 0x7f4b000ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b084c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b000cbd20 0x7f4b000ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b084c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b001266d0 con 0x7f4b000afa20 2023-10-18T10:37:51.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b084c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0013cfb0 con 0x7f4b000b0530 2023-10-18T10:37:51.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b084c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00125ac0 con 0x7f4b000cbd20 2023-10-18T10:37:51.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000afa20 0x7f4b00125200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000afa20 0x7f4b00125200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52806/0 (socket says 172.21.15.125:52806) 2023-10-18T10:37:51.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 -- 172.21.15.125:0/533224849 learned_addr learned my addr 172.21.15.125:0/533224849 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:51.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b05a64640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b000b0530 0x7f4b000ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06a66640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b000cbd20 0x7f4b000ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 -- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b000cbd20 msgr2=0x7f4b000ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b000cbd20 0x7f4b000ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 -- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b000b0530 msgr2=0x7f4b000ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b000b0530 0x7f4b000ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 -- 172.21.15.125:0/533224849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8009490 con 0x7f4b000afa20 2023-10-18T10:37:51.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b05a64640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b000b0530 0x7f4b000ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:51.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b06265640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000afa20 0x7f4b00125200 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00cb10 tx=0x7f4afc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:51.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4af6ffd640 1 -- 172.21.15.125:0/533224849 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00ac70 con 0x7f4b000afa20 2023-10-18T10:37:51.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4af6ffd640 1 -- 172.21.15.125:0/533224849 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4afc003970 con 0x7f4b000afa20 2023-10-18T10:37:51.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4af6ffd640 1 -- 172.21.15.125:0/533224849 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc003dc0 con 0x7f4b000afa20 2023-10-18T10:37:51.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b00125da0 con 0x7f4b000afa20 2023-10-18T10:37:51.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.318+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b00126360 con 0x7f4b000afa20 2023-10-18T10:37:51.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.319+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0013abe0 con 0x7f4b000afa20 2023-10-18T10:37:51.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.320+0000 7f4af6ffd640 1 -- 172.21.15.125:0/533224849 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4afc004030 con 0x7f4b000afa20 2023-10-18T10:37:51.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.320+0000 7f4af6ffd640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f4ae406af90 0x7f4ae406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.320+0000 7f4af6ffd640 1 -- 172.21.15.125:0/533224849 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f4afc08a2d0 con 0x7f4b000afa20 2023-10-18T10:37:51.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.320+0000 7f4b05a64640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f4ae406af90 0x7f4ae406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.321+0000 7f4b05a64640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f4ae406af90 0x7f4ae406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b0014b410 tx=0x7f4af000a5c0 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:51.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.324+0000 7f4af6ffd640 1 -- 172.21.15.125:0/533224849 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4afc055170 con 0x7f4b000afa20 2023-10-18T10:37:51.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.489+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4b000690e0 con 0x7f4b000afa20 2023-10-18T10:37:51.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.492+0000 7f4af6ffd640 1 -- 172.21.15.125:0/533224849 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4afc058790 con 0x7f4b000afa20 2023-10-18T10:37:51.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f4ae406af90 msgr2=0x7f4ae406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f4ae406af90 0x7f4ae406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b0014b410 tx=0x7f4af000a5c0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000afa20 msgr2=0x7f4b00125200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000afa20 0x7f4b00125200 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00cb10 tx=0x7f4afc00a060 comp rx=0 tx=0).stop 2023-10-18T10:37:51.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 shutdown_connections 2023-10-18T10:37:51.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f4ae406af90 0x7f4ae406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b000b0530 0x7f4b000ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b000cbd20 0x7f4b000ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 --2- 172.21.15.125:0/533224849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b000afa20 0x7f4b00125200 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 >> 172.21.15.125:0/533224849 conn(0x7f4b00069930 msgr2=0x7f4b000ac5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:51.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 shutdown_connections 2023-10-18T10:37:51.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.494+0000 7f4b084c9640 1 -- 172.21.15.125:0/533224849 wait complete. 2023-10-18T10:37:51.507 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:37:51.507 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:37:51.554 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:37:51.610 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:37:51.611 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:37:51.611 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:37:51.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:37:51.627 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:37:51.627 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:37:51.644 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:37:51.645 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:37:51.645 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:37:51.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:37:51.683 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:37:51.684 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:37:51.701 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:37:51.701 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:37:51.701 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:37:51.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.823+0000 7fb940f8a640 1 -- 172.21.15.125:0/4232868883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c06b030 msgr2=0x7fb93c06b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.823+0000 7fb940f8a640 1 --2- 172.21.15.125:0/4232868883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c06b030 0x7fb93c06b410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005fd0 tx=0x7fb92c02c050 comp rx=0 tx=0).stop 2023-10-18T10:37:51.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.823+0000 7fb940f8a640 1 -- 172.21.15.125:0/4232868883 shutdown_connections 2023-10-18T10:37:51.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.823+0000 7fb940f8a640 1 --2- 172.21.15.125:0/4232868883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c159bd0 0x7fb93c15c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.823+0000 7fb940f8a640 1 --2- 172.21.15.125:0/4232868883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c06b030 0x7fb93c06b410 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.823+0000 7fb940f8a640 1 -- 172.21.15.125:0/4232868883 >> 172.21.15.125:0/4232868883 conn(0x7fb93c069930 msgr2=0x7fb93c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:51.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.823+0000 7fb940f8a640 1 -- 172.21.15.125:0/4232868883 shutdown_connections 2023-10-18T10:37:51.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.823+0000 7fb940f8a640 1 -- 172.21.15.125:0/4232868883 wait complete. 2023-10-18T10:37:51.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.825+0000 7fb940f8a640 1 Processor -- start 2023-10-18T10:37:51.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.825+0000 7fb940f8a640 1 -- start start 2023-10-18T10:37:51.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.825+0000 7fb940f8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c06b030 0x7fb93c0a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.825+0000 7fb940f8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c159bd0 0x7fb93c0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.825+0000 7fb940f8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c15ccb0 0x7fb93c13e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.825+0000 7fb940f8a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1ef190 con 0x7fb93c06b030 2023-10-18T10:37:51.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.825+0000 7fb940f8a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0a80e0 con 0x7fb93c15ccb0 2023-10-18T10:37:51.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.825+0000 7fb940f8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0a83c0 con 0x7fb93c159bd0 2023-10-18T10:37:51.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c06b030 0x7fb93c0a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c06b030 0x7fb93c0a6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52822/0 (socket says 172.21.15.125:52822) 2023-10-18T10:37:51.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 -- 172.21.15.125:0/1845350208 learned_addr learned my addr 172.21.15.125:0/1845350208 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:51.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb939d9b640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c159bd0 0x7fb93c0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93ad9d640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c15ccb0 0x7fb93c13e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 -- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c159bd0 msgr2=0x7fb93c0c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c159bd0 0x7fb93c0c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 -- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c15ccb0 msgr2=0x7fb93c13e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:51.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c15ccb0 0x7fb93c13e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:51.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 -- 172.21.15.125:0/1845350208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c009840 con 0x7fb93c06b030 2023-10-18T10:37:51.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93ad9d640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c15ccb0 0x7fb93c13e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:51.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb93a59c640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c06b030 0x7fb93c0a6ec0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb92c036040 tx=0x7fb92c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:51.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb92affd640 1 -- 172.21.15.125:0/1845350208 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c03e460 con 0x7fb93c06b030 2023-10-18T10:37:51.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb92affd640 1 -- 172.21.15.125:0/1845350208 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb92c03e6b0 con 0x7fb93c06b030 2023-10-18T10:37:51.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c0a86a0 con 0x7fb93c06b030 2023-10-18T10:37:51.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb92affd640 1 -- 172.21.15.125:0/1845350208 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c03eaa0 con 0x7fb93c06b030 2023-10-18T10:37:51.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.826+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c0a8c60 con 0x7fb93c06b030 2023-10-18T10:37:51.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.827+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93c0690e0 con 0x7fb93c06b030 2023-10-18T10:37:51.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.828+0000 7fb92affd640 1 -- 172.21.15.125:0/1845350208 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb92c03ec00 con 0x7fb93c06b030 2023-10-18T10:37:51.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.828+0000 7fb92affd640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fb91c06af60 0x7fb91c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:51.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.828+0000 7fb92affd640 1 -- 172.21.15.125:0/1845350208 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fb92c0ac850 con 0x7fb93c06b030 2023-10-18T10:37:51.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.828+0000 7fb939d9b640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fb91c06af60 0x7fb91c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:51.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.829+0000 7fb939d9b640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fb91c06af60 0x7fb91c06d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0af9c0 tx=0x7fb930009260 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:51.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:51.832+0000 7fb92affd640 1 -- 172.21.15.125:0/1845350208 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb92c0776f0 con 0x7fb93c06b030 2023-10-18T10:37:52.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.035+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb93c0cc460 con 0x7fb93c06b030 2023-10-18T10:37:52.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.035+0000 7fb92affd640 1 -- 172.21.15.125:0/1845350208 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fb92c042020 con 0x7fb93c06b030 2023-10-18T10:37:52.038 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:52.038 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3511459076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3511459076},{"type":"v1","addr":"172.21.15.139:6836","nonce":3511459076}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/984169411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":984169411},{"type":"v1","addr":"172.21.15.139:6837","nonce":984169411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2755714237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2755714237},{"type":"v1","addr":"172.21.15.125:6834","nonce":2755714237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":6492,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/4118855557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":4118855557},{"type":"v1","addr":"172.21.15.125:6835","nonce":4118855557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2023-10-18T10:37:52.039 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 18 2023-10-18T10:37:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fb91c06af60 msgr2=0x7fb91c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fb91c06af60 0x7fb91c06d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0af9c0 tx=0x7fb930009260 comp rx=0 tx=0).stop 2023-10-18T10:37:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c06b030 msgr2=0x7fb93c0a6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c06b030 0x7fb93c0a6ec0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb92c036040 tx=0x7fb92c0042f0 comp rx=0 tx=0).stop 2023-10-18T10:37:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 shutdown_connections 2023-10-18T10:37:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fb91c06af60 0x7fb91c06d420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c15ccb0 0x7fb93c13e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c159bd0 0x7fb93c0c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 --2- 172.21.15.125:0/1845350208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c06b030 0x7fb93c0a6ec0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.037+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 >> 172.21.15.125:0/1845350208 conn(0x7fb93c069930 msgr2=0x7fb93c15c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.038+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 shutdown_connections 2023-10-18T10:37:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.038+0000 7fb940f8a640 1 -- 172.21.15.125:0/1845350208 wait complete. 2023-10-18T10:37:52.051 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:37:52.051 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:37:52.052 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:37:52.052 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:37:52.052 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:37:52.052 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:37:52.053 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:37:52.053 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:37:52.053 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:37:52.054 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:37:52.054 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:37:52.055 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:37:52.055 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:37:52.055 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:37:52.056 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:37:52.056 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:37:52.062 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:37:52.062 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:37:52.063 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:37:52.064 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:37:52.066 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:37:52.067 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:37:52.069 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:37:52.070 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:37:52.076 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:37:52.077 DEBUG:teuthology.parallel:result is None 2023-10-18T10:37:52.081 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:37:52.081 DEBUG:teuthology.parallel:result is None 2023-10-18T10:37:52.106 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:37:52.107 DEBUG:teuthology.parallel:result is None 2023-10-18T10:37:52.108 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:37:52.108 DEBUG:teuthology.parallel:result is None 2023-10-18T10:37:52.109 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:37:52.160 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:37:52.161 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:37:52.181 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:37:52.182 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:37:52.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.215+0000 7f1015997640 1 -- 172.21.15.125:0/3931650598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10100ceb40 msgr2=0x7f0ff8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:52.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.215+0000 7f1015997640 1 --2- 172.21.15.125:0/3931650598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10100ceb40 0x7f0ff8005180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1008009f00 tx=0x7f0ff80057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:52.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.215+0000 7f100f7fe640 1 -- 172.21.15.125:0/3931650598 reap_dead start 2023-10-18T10:37:52.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.215+0000 7f1015997640 1 -- 172.21.15.125:0/3931650598 shutdown_connections 2023-10-18T10:37:52.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.215+0000 7f1015997640 1 -- 172.21.15.125:0/3931650598 >> 172.21.15.125:0/3931650598 conn(0x7f10100d0d60 msgr2=0x7f10100d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:52.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.215+0000 7f1015997640 1 -- 172.21.15.125:0/3931650598 shutdown_connections 2023-10-18T10:37:52.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.215+0000 7f1015997640 1 -- 172.21.15.125:0/3931650598 wait complete. 2023-10-18T10:37:52.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.216+0000 7f1015997640 1 Processor -- start 2023-10-18T10:37:52.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.216+0000 7f1015997640 1 -- start start 2023-10-18T10:37:52.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f1015997640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1010150240 0x7f10100ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:52.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f1015997640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10100ca7d0 0x7f101015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:52.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f1015997640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f101015d3c0 0x7f10100c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:52.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f1015997640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101015c700 con 0x7f101015d3c0 2023-10-18T10:37:52.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f1015997640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100cac50 con 0x7f1010150240 2023-10-18T10:37:52.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f1015997640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101015b680 con 0x7f10100ca7d0 2023-10-18T10:37:52.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f101015d3c0 0x7f10100c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:52.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f101015d3c0 0x7f10100c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52854/0 (socket says 172.21.15.125:52854) 2023-10-18T10:37:52.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100f7fe640 1 -- 172.21.15.125:0/701052757 learned_addr learned my addr 172.21.15.125:0/701052757 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:52.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100effd640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1010150240 0x7f10100ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:52.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100e7fc640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10100ca7d0 0x7f101015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:52.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100f7fe640 1 -- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10100ca7d0 msgr2=0x7f101015ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:52.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100f7fe640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10100ca7d0 0x7f101015ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:52.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100f7fe640 1 -- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1010150240 msgr2=0x7f10100ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:52.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100f7fe640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1010150240 0x7f10100ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:52.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.217+0000 7f100f7fe640 1 -- 172.21.15.125:0/701052757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101015b940 con 0x7f101015d3c0 2023-10-18T10:37:52.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.218+0000 7f100f7fe640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f101015d3c0 0x7f10100c7e30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1008009f00 tx=0x7f1008030700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:52.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.218+0000 7f0ff7fff640 1 -- 172.21.15.125:0/701052757 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100803d070 con 0x7f101015d3c0 2023-10-18T10:37:52.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.218+0000 7f0ff7fff640 1 -- 172.21.15.125:0/701052757 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1008009200 con 0x7f101015d3c0 2023-10-18T10:37:52.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.218+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101015bc20 con 0x7f101015d3c0 2023-10-18T10:37:52.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.218+0000 7f0ff7fff640 1 -- 172.21.15.125:0/701052757 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008003bb0 con 0x7f101015d3c0 2023-10-18T10:37:52.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.218+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101015c1e0 con 0x7f101015d3c0 2023-10-18T10:37:52.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.219+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10100690e0 con 0x7f101015d3c0 2023-10-18T10:37:52.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.219+0000 7f0ff7fff640 1 -- 172.21.15.125:0/701052757 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f100802e040 con 0x7f101015d3c0 2023-10-18T10:37:52.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.220+0000 7f0ff7fff640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0ffc06af90 0x7f0ffc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:52.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.220+0000 7f0ff7fff640 1 -- 172.21.15.125:0/701052757 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f10080ab770 con 0x7f101015d3c0 2023-10-18T10:37:52.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.220+0000 7f100effd640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0ffc06af90 0x7f0ffc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:52.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.220+0000 7f100effd640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0ffc06af90 0x7f0ffc06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10100ca090 tx=0x7f1000005f90 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:52.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.224+0000 7f0ff7fff640 1 -- 172.21.15.125:0/701052757 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10080765e0 con 0x7f101015d3c0 2023-10-18T10:37:52.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:52.401+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f10100d0b40 con 0x7f101015d3c0 2023-10-18T10:37:53.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.515+0000 7f0ff7fff640 1 -- 172.21.15.125:0/701052757 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1008079c00 con 0x7f101015d3c0 2023-10-18T10:37:53.517 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T10:37:53.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0ffc06af90 msgr2=0x7f0ffc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:53.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0ffc06af90 0x7f0ffc06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10100ca090 tx=0x7f1000005f90 comp rx=0 tx=0).stop 2023-10-18T10:37:53.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f101015d3c0 msgr2=0x7f10100c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:53.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f101015d3c0 0x7f10100c7e30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1008009f00 tx=0x7f1008030700 comp rx=0 tx=0).stop 2023-10-18T10:37:53.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 shutdown_connections 2023-10-18T10:37:53.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f0ffc06af90 0x7f0ffc06d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f101015d3c0 0x7f10100c7e30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10100ca7d0 0x7f101015ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 --2- 172.21.15.125:0/701052757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1010150240 0x7f10100ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.518+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 >> 172.21.15.125:0/701052757 conn(0x7f10100d0d60 msgr2=0x7f10100ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:53.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.519+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 shutdown_connections 2023-10-18T10:37:53.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.519+0000 7f1015997640 1 -- 172.21.15.125:0/701052757 wait complete. 2023-10-18T10:37:53.534 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:37:53.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.639+0000 7f77f118a640 1 -- 172.21.15.125:0/3888070332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0cbd20 msgr2=0x7f77ec158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:53.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.639+0000 7f77f118a640 1 --2- 172.21.15.125:0/3888070332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0cbd20 0x7f77ec158300 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005fd0 tx=0x7f77dc02bd90 comp rx=0 tx=0).stop 2023-10-18T10:37:53.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.639+0000 7f77f118a640 1 -- 172.21.15.125:0/3888070332 shutdown_connections 2023-10-18T10:37:53.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.639+0000 7f77f118a640 1 --2- 172.21.15.125:0/3888070332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77ec0b0530 0x7f77ec0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.639+0000 7f77f118a640 1 --2- 172.21.15.125:0/3888070332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0cbd20 0x7f77ec158300 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.639+0000 7f77f118a640 1 -- 172.21.15.125:0/3888070332 >> 172.21.15.125:0/3888070332 conn(0x7f77ec069930 msgr2=0x7f77ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:53.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.639+0000 7f77f118a640 1 -- 172.21.15.125:0/3888070332 shutdown_connections 2023-10-18T10:37:53.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.639+0000 7f77f118a640 1 -- 172.21.15.125:0/3888070332 wait complete. 2023-10-18T10:37:53.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.640+0000 7f77f118a640 1 Processor -- start 2023-10-18T10:37:53.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.640+0000 7f77f118a640 1 -- start start 2023-10-18T10:37:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77f118a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0afa20 0x7f77ec156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77f118a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ec0b0530 0x7f77ec1504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77f118a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77ec0cbd20 0x7f77ec150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77f118a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec146e80 con 0x7f77ec0afa20 2023-10-18T10:37:53.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77f118a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec147160 con 0x7f77ec0b0530 2023-10-18T10:37:53.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77f118a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec147440 con 0x7f77ec0cbd20 2023-10-18T10:37:53.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0afa20 0x7f77ec156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:53.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0afa20 0x7f77ec156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52886/0 (socket says 172.21.15.125:52886) 2023-10-18T10:37:53.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 -- 172.21.15.125:0/3518807777 learned_addr learned my addr 172.21.15.125:0/3518807777 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:53.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 -- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77ec0cbd20 msgr2=0x7f77ec150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:53.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ea59c640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ec0b0530 0x7f77ec1504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:53.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77ec0cbd20 0x7f77ec150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 -- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ec0b0530 msgr2=0x7f77ec1504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:53.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ec0b0530 0x7f77ec1504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 -- 172.21.15.125:0/3518807777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc009560 con 0x7f77ec0afa20 2023-10-18T10:37:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.641+0000 7f77ead9d640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0afa20 0x7f77ec156530 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009850 tx=0x7f77dc02da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.642+0000 7f77d37fe640 1 -- 172.21.15.125:0/3518807777 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc02de60 con 0x7f77ec0afa20 2023-10-18T10:37:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.642+0000 7f77d37fe640 1 -- 172.21.15.125:0/3518807777 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f77dc0036e0 con 0x7f77ec0afa20 2023-10-18T10:37:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.642+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec147720 con 0x7f77ec0afa20 2023-10-18T10:37:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.642+0000 7f77d37fe640 1 -- 172.21.15.125:0/3518807777 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc003a90 con 0x7f77ec0afa20 2023-10-18T10:37:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.642+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ec156a70 con 0x7f77ec0afa20 2023-10-18T10:37:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.643+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ec147c40 con 0x7f77ec0afa20 2023-10-18T10:37:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.643+0000 7f77d37fe640 1 -- 172.21.15.125:0/3518807777 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f77dc003bf0 con 0x7f77ec0afa20 2023-10-18T10:37:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.643+0000 7f77d37fe640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f77d806af60 0x7f77d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:53.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.644+0000 7f77d37fe640 1 -- 172.21.15.125:0/3518807777 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f77dc0ac0c0 con 0x7f77ec0afa20 2023-10-18T10:37:53.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.644+0000 7f77ea59c640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f77d806af60 0x7f77d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:53.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.644+0000 7f77ea59c640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f77d806af60 0x7f77d806d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77ec157cf0 tx=0x7f77e0009040 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:53.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.648+0000 7f77d37fe640 1 -- 172.21.15.125:0/3518807777 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77ec147c40 con 0x7f77ec0afa20 2023-10-18T10:37:53.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.874+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f77ec147c40 con 0x7f77ec0afa20 2023-10-18T10:37:53.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.875+0000 7f77d37fe640 1 -- 172.21.15.125:0/3518807777 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f77ec147c40 con 0x7f77ec0afa20 2023-10-18T10:37:53.877 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:53.877 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:37:53.878 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T10:37:53.878 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.878 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T10:37:53.878 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T10:37:53.879 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.879 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:37:53.879 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:37:53.879 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:37:53.879 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.880 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.880 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.880 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.880 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T10:37:53.880 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T10:37:53.881 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.881 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:37:53.881 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:37:53.881 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:37:53.881 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.882 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.882 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.882 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.883 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T10:37:53.883 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T10:37:53.883 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.883 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:37:53.883 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:37:53.884 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:37:53.884 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.884 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.884 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.885 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.885 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T10:37:53.885 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T10:37:53.885 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.886 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:37:53.886 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:37:53.886 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:37:53.886 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.886 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.887 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.887 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.887 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T10:37:53.887 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T10:37:53.888 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.888 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:37:53.888 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:37:53.888 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.888 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.889 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.889 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.889 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T10:37:53.889 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T10:37:53.889 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.889 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:37:53.890 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:37:53.890 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.890 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.890 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.890 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.891 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T10:37:53.891 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T10:37:53.891 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.891 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:37:53.891 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:37:53.892 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.892 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.892 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.892 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.892 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T10:37:53.893 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T10:37:53.893 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.893 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:37:53.893 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:37:53.893 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.893 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.894 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.894 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.894 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T10:37:53.894 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T10:37:53.894 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.895 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:37:53.895 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:37:53.895 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.896 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.896 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.896 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.896 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T10:37:53.897 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T10:37:53.897 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.897 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:37:53.897 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:37:53.897 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.897 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.898 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.898 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.898 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T10:37:53.898 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T10:37:53.898 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.899 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:37:53.899 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:37:53.899 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.899 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.899 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.900 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.900 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T10:37:53.900 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T10:37:53.900 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.900 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:37:53.901 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:37:53.901 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.901 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.901 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.901 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.901 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T10:37:53.902 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T10:37:53.902 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.902 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:37:53.902 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T10:37:53.902 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T10:37:53.903 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.903 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.903 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.903 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T10:37:53.903 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T10:37:53.904 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.904 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:37:53.904 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:37:53.904 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:37:53.904 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:37:53.904 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.905 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.905 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.905 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T10:37:53.905 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T10:37:53.905 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.906 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:37:53.906 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:37:53.906 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:37:53.906 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:37:53.906 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.907 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.907 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.907 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T10:37:53.907 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T10:37:53.907 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.907 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:37:53.908 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:37:53.908 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:37:53.908 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:37:53.908 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.908 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.909 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.909 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T10:37:53.909 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T10:37:53.909 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.909 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:37:53.910 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:37:53.910 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:37:53.910 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.910 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.910 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.911 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.911 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T10:37:53.911 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T10:37:53.911 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.911 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:37:53.911 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:37:53.912 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.912 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.912 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:37:53.912 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:53.912 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T10:37:53.913 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T10:37:53.913 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:37:53.913 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:37:53.913 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:37:53.913 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:37:53.913 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.914 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:53.914 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T10:37:53.914 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:37:53.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f77d806af60 msgr2=0x7f77d806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:53.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f77d806af60 0x7f77d806d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77ec157cf0 tx=0x7f77e0009040 comp rx=0 tx=0).stop 2023-10-18T10:37:53.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0afa20 msgr2=0x7f77ec156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:53.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0afa20 0x7f77ec156530 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009850 tx=0x7f77dc02da60 comp rx=0 tx=0).stop 2023-10-18T10:37:53.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 shutdown_connections 2023-10-18T10:37:53.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f77d806af60 0x7f77d806d420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ec0b0530 0x7f77ec1504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77ec0cbd20 0x7f77ec150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 --2- 172.21.15.125:0/3518807777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ec0afa20 0x7f77ec156530 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:53.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 >> 172.21.15.125:0/3518807777 conn(0x7f77ec069930 msgr2=0x7f77ec14d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:53.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 shutdown_connections 2023-10-18T10:37:53.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:53.877+0000 7f77f118a640 1 -- 172.21.15.125:0/3518807777 wait complete. 2023-10-18T10:37:53.919 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:37:54.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.019+0000 7f093f64b640 1 -- 172.21.15.125:0/585823904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09240039a0 msgr2=0x7f0924003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.019+0000 7f093f64b640 1 --2- 172.21.15.125:0/585823904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09240039a0 0x7f0924003dc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0934001d80 tx=0x7f09240057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.019+0000 7f093dbe8640 1 -- 172.21.15.125:0/585823904 reap_dead start 2023-10-18T10:37:54.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.019+0000 7f093f64b640 1 -- 172.21.15.125:0/585823904 shutdown_connections 2023-10-18T10:37:54.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.019+0000 7f093f64b640 1 -- 172.21.15.125:0/585823904 >> 172.21.15.125:0/585823904 conn(0x7f0938069930 msgr2=0x7f0938069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:54.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.019+0000 7f093f64b640 1 -- 172.21.15.125:0/585823904 shutdown_connections 2023-10-18T10:37:54.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.019+0000 7f093f64b640 1 -- 172.21.15.125:0/585823904 wait complete. 2023-10-18T10:37:54.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093f64b640 1 Processor -- start 2023-10-18T10:37:54.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093f64b640 1 -- start start 2023-10-18T10:37:54.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093f64b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240039a0 0x7f09380a8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093f64b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f093813ef40 0x7f093815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093f64b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f093813f3a0 0x7f093815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093f64b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09380a7e20 con 0x7f093813f3a0 2023-10-18T10:37:54.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093f64b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09380a6ec0 con 0x7f09240039a0 2023-10-18T10:37:54.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093f64b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09380a71a0 con 0x7f093813ef40 2023-10-18T10:37:54.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093dbe8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f093813f3a0 0x7f093815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093dbe8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f093813f3a0 0x7f093815c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52900/0 (socket says 172.21.15.125:52900) 2023-10-18T10:37:54.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.021+0000 7f093dbe8640 1 -- 172.21.15.125:0/424858772 learned_addr learned my addr 172.21.15.125:0/424858772 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:54.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093cbe6640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f093813ef40 0x7f093815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093d3e7640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240039a0 0x7f09380a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093dbe8640 1 -- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f093813ef40 msgr2=0x7f093815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093dbe8640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f093813ef40 0x7f093815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093dbe8640 1 -- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240039a0 msgr2=0x7f09380a8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093dbe8640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240039a0 0x7f09380a8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093dbe8640 1 -- 172.21.15.125:0/424858772 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380a7480 con 0x7f093813f3a0 2023-10-18T10:37:54.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093cbe6640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f093813ef40 0x7f093815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:54.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093d3e7640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240039a0 0x7f09380a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:37:54.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093dbe8640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f093813f3a0 0x7f093815c170 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f093806d570 tx=0x7f093000fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:54.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f092dffb640 1 -- 172.21.15.125:0/424858772 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930004440 con 0x7f093813f3a0 2023-10-18T10:37:54.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f092dffb640 1 -- 172.21.15.125:0/424858772 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f09300036e0 con 0x7f093813f3a0 2023-10-18T10:37:54.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09380a7760 con 0x7f093813f3a0 2023-10-18T10:37:54.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f092dffb640 1 -- 172.21.15.125:0/424858772 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930003af0 con 0x7f093813f3a0 2023-10-18T10:37:54.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.022+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093815df20 con 0x7f093813f3a0 2023-10-18T10:37:54.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.023+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09380a7f60 con 0x7f093813f3a0 2023-10-18T10:37:54.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.024+0000 7f092dffb640 1 -- 172.21.15.125:0/424858772 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0930003c50 con 0x7f093813f3a0 2023-10-18T10:37:54.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.024+0000 7f092dffb640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f091c06af90 0x7f091c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.024+0000 7f092dffb640 1 -- 172.21.15.125:0/424858772 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0930089c50 con 0x7f093813f3a0 2023-10-18T10:37:54.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.024+0000 7f093d3e7640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f091c06af90 0x7f091c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.025+0000 7f093d3e7640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f091c06af90 0x7f091c06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0934034000 tx=0x7f0934001510 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:54.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.028+0000 7f092dffb640 1 -- 172.21.15.125:0/424858772 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09380a7f60 con 0x7f093813f3a0 2023-10-18T10:37:54.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.196+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09380a7f60 con 0x7f093813f3a0 2023-10-18T10:37:54.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.197+0000 7f092dffb640 1 -- 172.21.15.125:0/424858772 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13110 (secure 0 0 0) 0x7f09380a7f60 con 0x7f093813f3a0 2023-10-18T10:37:54.199 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:54.199 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":61,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:53.514367+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:37:54.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f091c06af90 msgr2=0x7f091c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f091c06af90 0x7f091c06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0934034000 tx=0x7f0934001510 comp rx=0 tx=0).stop 2023-10-18T10:37:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f093813f3a0 msgr2=0x7f093815c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f093813f3a0 0x7f093815c170 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f093806d570 tx=0x7f093000fa20 comp rx=0 tx=0).stop 2023-10-18T10:37:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 shutdown_connections 2023-10-18T10:37:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f091c06af90 0x7f091c06d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f093813f3a0 0x7f093815c170 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f093813ef40 0x7f093815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 --2- 172.21.15.125:0/424858772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09240039a0 0x7f09380a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 >> 172.21.15.125:0/424858772 conn(0x7f0938069930 msgr2=0x7f09380703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 shutdown_connections 2023-10-18T10:37:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.199+0000 7f093f64b640 1 -- 172.21.15.125:0/424858772 wait complete. 2023-10-18T10:37:54.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:37:54.212 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:37:54.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.316+0000 7f98fc04f640 1 -- 172.21.15.125:0/1183535909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98f413d360 msgr2=0x7f98f415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.316+0000 7f98fc04f640 1 --2- 172.21.15.125:0/1183535909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98f413d360 0x7f98f415a9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005fd0 tx=0x7f98ec02be10 comp rx=0 tx=0).stop 2023-10-18T10:37:54.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.316+0000 7f98fc04f640 1 -- 172.21.15.125:0/1183535909 shutdown_connections 2023-10-18T10:37:54.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.316+0000 7f98fc04f640 1 --2- 172.21.15.125:0/1183535909 >> v2:172.21.15.139:3300/0 conn(0x7f98f4140660 0x7f98f40c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.316+0000 7f98fc04f640 1 --2- 172.21.15.125:0/1183535909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98f413d360 0x7f98f415a9f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.316+0000 7f98fc04f640 1 -- 172.21.15.125:0/1183535909 >> 172.21.15.125:0/1183535909 conn(0x7f98f4069930 msgr2=0x7f98f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:54.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.316+0000 7f98fc04f640 1 -- 172.21.15.125:0/1183535909 shutdown_connections 2023-10-18T10:37:54.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.316+0000 7f98fc04f640 1 -- 172.21.15.125:0/1183535909 wait complete. 2023-10-18T10:37:54.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.317+0000 7f98fc04f640 1 Processor -- start 2023-10-18T10:37:54.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.317+0000 7f98fc04f640 1 -- start start 2023-10-18T10:37:54.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98fc04f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98f40c9d00 0x7f98f4157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98fc04f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98f413d360 0x7f98f4155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98fc04f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98f4140660 0x7f98f40b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98fc04f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f413a3e0 con 0x7f98f413d360 2023-10-18T10:37:54.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98fc04f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f413a630 con 0x7f98f40c9d00 2023-10-18T10:37:54.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98fc04f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f413a910 con 0x7f98f4140660 2023-10-18T10:37:54.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98f95ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98f413d360 0x7f98f4155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98f95ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98f413d360 0x7f98f4155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52914/0 (socket says 172.21.15.125:52914) 2023-10-18T10:37:54.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98f95ea640 1 -- 172.21.15.125:0/2777304756 learned_addr learned my addr 172.21.15.125:0/2777304756 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:54.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98fa5ec640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98f4140660 0x7f98f40b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.318+0000 7f98f9deb640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98f40c9d00 0x7f98f4157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f95ea640 1 -- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98f4140660 msgr2=0x7f98f40b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f95ea640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98f4140660 0x7f98f40b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f95ea640 1 -- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98f40c9d00 msgr2=0x7f98f4157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f95ea640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98f40c9d00 0x7f98f4157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f95ea640 1 -- 172.21.15.125:0/2777304756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec009560 con 0x7f98f413d360 2023-10-18T10:37:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f95ea640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98f413d360 0x7f98f4155020 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f98e80095a0 tx=0x7f98e8009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f27fc640 1 -- 172.21.15.125:0/2777304756 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8014820 con 0x7f98f413d360 2023-10-18T10:37:54.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f27fc640 1 -- 172.21.15.125:0/2777304756 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98e8010040 con 0x7f98f413d360 2023-10-18T10:37:54.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f413abf0 con 0x7f98f413d360 2023-10-18T10:37:54.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98f27fc640 1 -- 172.21.15.125:0/2777304756 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8014cd0 con 0x7f98f413d360 2023-10-18T10:37:54.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.319+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f413b1b0 con 0x7f98f413d360 2023-10-18T10:37:54.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.320+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f40690e0 con 0x7f98f413d360 2023-10-18T10:37:54.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.321+0000 7f98f27fc640 1 -- 172.21.15.125:0/2777304756 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f98e801d400 con 0x7f98f413d360 2023-10-18T10:37:54.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.321+0000 7f98f27fc640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f98e006af60 0x7f98e006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.321+0000 7f98f27fc640 1 -- 172.21.15.125:0/2777304756 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f98e8089ae0 con 0x7f98f413d360 2023-10-18T10:37:54.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.321+0000 7f98f9deb640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f98e006af60 0x7f98e006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.322+0000 7f98f9deb640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f98e006af60 0x7f98e006d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98ec035000 tx=0x7f98ec004730 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:54.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.325+0000 7f98f27fc640 1 -- 172.21.15.125:0/2777304756 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98e8054c30 con 0x7f98f413d360 2023-10-18T10:37:54.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.489+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98f415b970 con 0x7f98f413d360 2023-10-18T10:37:54.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.490+0000 7f98f27fc640 1 -- 172.21.15.125:0/2777304756 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98e8058250 con 0x7f98f413d360 2023-10-18T10:37:54.492 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:37:54.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f98e006af60 msgr2=0x7f98e006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f98e006af60 0x7f98e006d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98ec035000 tx=0x7f98ec004730 comp rx=0 tx=0).stop 2023-10-18T10:37:54.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98f413d360 msgr2=0x7f98f4155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98f413d360 0x7f98f4155020 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f98e80095a0 tx=0x7f98e8009a60 comp rx=0 tx=0).stop 2023-10-18T10:37:54.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 shutdown_connections 2023-10-18T10:37:54.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f98e006af60 0x7f98e006d420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98f4140660 0x7f98f40b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98f413d360 0x7f98f4155020 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 --2- 172.21.15.125:0/2777304756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98f40c9d00 0x7f98f4157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 >> 172.21.15.125:0/2777304756 conn(0x7f98f4069930 msgr2=0x7f98f413c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:54.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 shutdown_connections 2023-10-18T10:37:54.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.492+0000 7f98fc04f640 1 -- 172.21.15.125:0/2777304756 wait complete. 2023-10-18T10:37:54.504 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:37:54.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.606+0000 7f3127b4c640 1 -- 172.21.15.125:0/2030072401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f310c0047c0 msgr2=0x7f310c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.606+0000 7f3127b4c640 1 --2- 172.21.15.125:0/2030072401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f310c0047c0 0x7f310c004c40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3110006f80 tx=0x7f310c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.606+0000 7f3127b4c640 1 -- 172.21.15.125:0/2030072401 shutdown_connections 2023-10-18T10:37:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.606+0000 7f3127b4c640 1 --2- 172.21.15.125:0/2030072401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f310c0047c0 0x7f310c004c40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.606+0000 7f3127b4c640 1 -- 172.21.15.125:0/2030072401 >> 172.21.15.125:0/2030072401 conn(0x7f3120069930 msgr2=0x7f3120069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.606+0000 7f3127b4c640 1 -- 172.21.15.125:0/2030072401 shutdown_connections 2023-10-18T10:37:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.606+0000 7f3127b4c640 1 -- 172.21.15.125:0/2030072401 wait complete. 2023-10-18T10:37:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.607+0000 7f3127b4c640 1 Processor -- start 2023-10-18T10:37:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.607+0000 7f3127b4c640 1 -- start start 2023-10-18T10:37:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f3127b4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f310c0047c0 0x7f3120139290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f3127b4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312014ff30 0x7f31201f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f3127b4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31201e36b0 0x7f31201e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f3127b4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31201e40a0 con 0x7f31201e36b0 2023-10-18T10:37:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f3127b4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31201397d0 con 0x7f312014ff30 2023-10-18T10:37:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f3127b4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31201f3450 con 0x7f310c0047c0 2023-10-18T10:37:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31201e36b0 0x7f31201e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31201e36b0 0x7f31201e3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52936/0 (socket says 172.21.15.125:52936) 2023-10-18T10:37:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 -- 172.21.15.125:0/2320992756 learned_addr learned my addr 172.21.15.125:0/2320992756 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 -- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f310c0047c0 msgr2=0x7f3120139290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31258e8640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f310c0047c0 0x7f3120139290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31250e7640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312014ff30 0x7f31201f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f310c0047c0 0x7f3120139290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 -- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312014ff30 msgr2=0x7f31201f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312014ff30 0x7f31201f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 -- 172.21.15.125:0/2320992756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120150460 con 0x7f31201e36b0 2023-10-18T10:37:54.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31260e9640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31201e36b0 0x7f31201e3b60 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f31200cc150 tx=0x7f3118001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:54.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31167fc640 1 -- 172.21.15.125:0/2320992756 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011840 con 0x7f31201e36b0 2023-10-18T10:37:54.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31167fc640 1 -- 172.21.15.125:0/2320992756 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31180119c0 con 0x7f31201e36b0 2023-10-18T10:37:54.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3120150740 con 0x7f31201e36b0 2023-10-18T10:37:54.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f31167fc640 1 -- 172.21.15.125:0/2320992756 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011dd0 con 0x7f31201e36b0 2023-10-18T10:37:54.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.608+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3120150d00 con 0x7f31201e36b0 2023-10-18T10:37:54.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.609+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312006c7e0 con 0x7f31201e36b0 2023-10-18T10:37:54.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.610+0000 7f31167fc640 1 -- 172.21.15.125:0/2320992756 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f311801d460 con 0x7f31201e36b0 2023-10-18T10:37:54.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.610+0000 7f31167fc640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f310406af90 0x7f310406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.610+0000 7f31167fc640 1 -- 172.21.15.125:0/2320992756 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f311808a1a0 con 0x7f31201e36b0 2023-10-18T10:37:54.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.610+0000 7f31258e8640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f310406af90 0x7f310406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.611+0000 7f31258e8640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f310406af90 0x7f310406d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f312006c0f0 tx=0x7f311c005f90 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:54.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.614+0000 7f31167fc640 1 -- 172.21.15.125:0/2320992756 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3118055240 con 0x7f31201e36b0 2023-10-18T10:37:54.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.783+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f31200690e0 con 0x7f31201e36b0 2023-10-18T10:37:54.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.783+0000 7f31167fc640 1 -- 172.21.15.125:0/2320992756 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3118058860 con 0x7f31201e36b0 2023-10-18T10:37:54.785 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:54.786 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:37:54.786 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:37:54.786 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:37:54.786 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:37:54.786 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:37:54.787 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:37:54.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f310406af90 msgr2=0x7f310406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f310406af90 0x7f310406d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f312006c0f0 tx=0x7f311c005f90 comp rx=0 tx=0).stop 2023-10-18T10:37:54.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31201e36b0 msgr2=0x7f31201e3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31201e36b0 0x7f31201e3b60 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f31200cc150 tx=0x7f3118001c30 comp rx=0 tx=0).stop 2023-10-18T10:37:54.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 shutdown_connections 2023-10-18T10:37:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f310406af90 0x7f310406d450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312014ff30 0x7f31201f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31201e36b0 0x7f31201e3b60 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 --2- 172.21.15.125:0/2320992756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f310c0047c0 0x7f3120139290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 >> 172.21.15.125:0/2320992756 conn(0x7f3120069930 msgr2=0x7f312013b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:54.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 shutdown_connections 2023-10-18T10:37:54.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.785+0000 7f3127b4c640 1 -- 172.21.15.125:0/2320992756 wait complete. 2023-10-18T10:37:54.797 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:37:54.798 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:37:54.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.902+0000 7f46b381a640 1 -- 172.21.15.125:0/3113460121 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4698004d00 msgr2=0x7f4698005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.902+0000 7f46b381a640 1 --2- 172.21.15.125:0/3113460121 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4698004d00 0x7f4698005180 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0ca450 tx=0x7f46980057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.902+0000 7f46b1db7640 1 -- 172.21.15.125:0/3113460121 reap_dead start 2023-10-18T10:37:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.902+0000 7f46b381a640 1 -- 172.21.15.125:0/3113460121 shutdown_connections 2023-10-18T10:37:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.902+0000 7f46b381a640 1 -- 172.21.15.125:0/3113460121 >> 172.21.15.125:0/3113460121 conn(0x7f46ac069930 msgr2=0x7f46ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.902+0000 7f46b381a640 1 -- 172.21.15.125:0/3113460121 shutdown_connections 2023-10-18T10:37:54.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.902+0000 7f46b381a640 1 -- 172.21.15.125:0/3113460121 wait complete. 2023-10-18T10:37:54.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.903+0000 7f46b381a640 1 Processor -- start 2023-10-18T10:37:54.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b381a640 1 -- start start 2023-10-18T10:37:54.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b381a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4698004d00 0x7f46ac155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b381a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46ac138570 0x7f46ac131a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b381a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac158a00 0x7f46ac158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b381a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac1593f0 con 0x7f46ac138570 2023-10-18T10:37:54.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b381a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac138a10 con 0x7f46ac158a00 2023-10-18T10:37:54.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b381a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac138cf0 con 0x7f4698004d00 2023-10-18T10:37:54.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b0db5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46ac138570 0x7f46ac131a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b0db5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46ac138570 0x7f46ac131a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52954/0 (socket says 172.21.15.125:52954) 2023-10-18T10:37:54.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b0db5640 1 -- 172.21.15.125:0/1722986851 learned_addr learned my addr 172.21.15.125:0/1722986851 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:54.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b1db7640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac158a00 0x7f46ac158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.904+0000 7f46b15b6640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4698004d00 0x7f46ac155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46b0db5640 1 -- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4698004d00 msgr2=0x7f46ac155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46b0db5640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4698004d00 0x7f46ac155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46b0db5640 1 -- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac158a00 msgr2=0x7f46ac158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:54.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46b0db5640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac158a00 0x7f46ac158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:54.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46b0db5640 1 -- 172.21.15.125:0/1722986851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac138fd0 con 0x7f46ac138570 2023-10-18T10:37:54.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46b0db5640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46ac138570 0x7f46ac131a70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f469c003ed0 tx=0x7f469c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:54.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46a1ffb640 1 -- 172.21.15.125:0/1722986851 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c006110 con 0x7f46ac138570 2023-10-18T10:37:54.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46a1ffb640 1 -- 172.21.15.125:0/1722986851 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f469c011840 con 0x7f46ac138570 2023-10-18T10:37:54.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac1392b0 con 0x7f46ac138570 2023-10-18T10:37:54.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46a1ffb640 1 -- 172.21.15.125:0/1722986851 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c011c30 con 0x7f46ac138570 2023-10-18T10:37:54.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.905+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac155da0 con 0x7f46ac138570 2023-10-18T10:37:54.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.906+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac0690e0 con 0x7f46ac138570 2023-10-18T10:37:54.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.906+0000 7f46a1ffb640 1 -- 172.21.15.125:0/1722986851 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f469c0103c0 con 0x7f46ac138570 2023-10-18T10:37:54.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.907+0000 7f46a1ffb640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f469006af60 0x7f469006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:54.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.907+0000 7f46a1ffb640 1 -- 172.21.15.125:0/1722986851 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f469c08a5d0 con 0x7f46ac138570 2023-10-18T10:37:54.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.907+0000 7f46b15b6640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f469006af60 0x7f469006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:54.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.907+0000 7f46b15b6640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f469006af60 0x7f469006d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46a8001eb0 tx=0x7f46a8000ed0 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:54.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:54.911+0000 7f46a1ffb640 1 -- 172.21.15.125:0/1722986851 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f469c055650 con 0x7f46ac138570 2023-10-18T10:37:55.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.081+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f46ac15b940 con 0x7f46ac138570 2023-10-18T10:37:55.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.929+0000 7f46a1ffb640 1 -- 172.21.15.125:0/1722986851 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7f469c01f020 con 0x7f46ac138570 2023-10-18T10:37:55.931 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:37:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f469006af60 msgr2=0x7f469006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f469006af60 0x7f469006d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46a8001eb0 tx=0x7f46a8000ed0 comp rx=0 tx=0).stop 2023-10-18T10:37:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46ac138570 msgr2=0x7f46ac131a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:55.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46ac138570 0x7f46ac131a70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f469c003ed0 tx=0x7f469c001630 comp rx=0 tx=0).stop 2023-10-18T10:37:55.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 shutdown_connections 2023-10-18T10:37:55.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f469006af60 0x7f469006d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:55.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46ac158a00 0x7f46ac158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:55.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46ac138570 0x7f46ac131a70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:55.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 --2- 172.21.15.125:0/1722986851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4698004d00 0x7f46ac155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:55.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 >> 172.21.15.125:0/1722986851 conn(0x7f46ac069930 msgr2=0x7f46ac157470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:55.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 shutdown_connections 2023-10-18T10:37:55.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:55.932+0000 7f46b381a640 1 -- 172.21.15.125:0/1722986851 wait complete. 2023-10-18T10:37:55.951 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:37:56.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.063+0000 7f7abeabb640 1 -- 172.21.15.125:0/3042205709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab80c9750 msgr2=0x7f7ab80c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:56.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.063+0000 7f7abeabb640 1 --2- 172.21.15.125:0/3042205709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab80c9750 0x7f7ab80c9ba0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005fd0 tx=0x7f7ab002c240 comp rx=0 tx=0).stop 2023-10-18T10:37:56.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.063+0000 7f7abeabb640 1 -- 172.21.15.125:0/3042205709 shutdown_connections 2023-10-18T10:37:56.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.063+0000 7f7abeabb640 1 --2- 172.21.15.125:0/3042205709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab80c9750 0x7f7ab80c9ba0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.063+0000 7f7abeabb640 1 --2- 172.21.15.125:0/3042205709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ab806b030 0x7f7ab806b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.063+0000 7f7abeabb640 1 -- 172.21.15.125:0/3042205709 >> 172.21.15.125:0/3042205709 conn(0x7f7ab8069930 msgr2=0x7f7ab8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:56.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.063+0000 7f7abeabb640 1 -- 172.21.15.125:0/3042205709 shutdown_connections 2023-10-18T10:37:56.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.063+0000 7f7abeabb640 1 -- 172.21.15.125:0/3042205709 wait complete. 2023-10-18T10:37:56.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.064+0000 7f7abeabb640 1 Processor -- start 2023-10-18T10:37:56.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.064+0000 7f7abeabb640 1 -- start start 2023-10-18T10:37:56.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abeabb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab806b030 0x7f7ab8152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:56.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abeabb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ab80c9750 0x7f7ab80ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:56.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abeabb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ab8159bd0 0x7f7ab80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:56.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abeabb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8153180 con 0x7f7ab806b030 2023-10-18T10:37:56.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abeabb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8152120 con 0x7f7ab8159bd0 2023-10-18T10:37:56.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abeabb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8136910 con 0x7f7ab80c9750 2023-10-18T10:37:56.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab806b030 0x7f7ab8152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:56.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab806b030 0x7f7ab8152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52962/0 (socket says 172.21.15.125:52962) 2023-10-18T10:37:56.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 -- 172.21.15.125:0/1858111583 learned_addr learned my addr 172.21.15.125:0/1858111583 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:56.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 -- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ab80c9750 msgr2=0x7f7ab80ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:56.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ab80c9750 0x7f7ab80ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 -- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ab8159bd0 msgr2=0x7f7ab80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:56.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ab8159bd0 0x7f7ab80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 -- 172.21.15.125:0/1858111583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab0009900 con 0x7f7ab806b030 2023-10-18T10:37:56.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abc857640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab806b030 0x7f7ab8152c40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00d770 tx=0x7f7aac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:56.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7ab57fa640 1 -- 172.21.15.125:0/1858111583 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac009c30 con 0x7f7ab806b030 2023-10-18T10:37:56.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7ab57fa640 1 -- 172.21.15.125:0/1858111583 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7aac00de80 con 0x7f7ab806b030 2023-10-18T10:37:56.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab8136bf0 con 0x7f7ab806b030 2023-10-18T10:37:56.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7ab57fa640 1 -- 172.21.15.125:0/1858111583 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac005280 con 0x7f7ab806b030 2023-10-18T10:37:56.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.065+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab81371b0 con 0x7f7ab806b030 2023-10-18T10:37:56.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.066+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab80690e0 con 0x7f7ab806b030 2023-10-18T10:37:56.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.067+0000 7f7ab57fa640 1 -- 172.21.15.125:0/1858111583 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7aac0053e0 con 0x7f7ab806b030 2023-10-18T10:37:56.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.067+0000 7f7ab57fa640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7a9c06af60 0x7f7a9c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:56.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.068+0000 7f7ab57fa640 1 -- 172.21.15.125:0/1858111583 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f7aac089a60 con 0x7f7ab806b030 2023-10-18T10:37:56.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.068+0000 7f7ab7fff640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7a9c06af60 0x7f7a9c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:56.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.068+0000 7f7ab7fff640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7a9c06af60 0x7f7a9c06d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4000f60 tx=0x7f7aa40079d0 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:56.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.072+0000 7f7ab57fa640 1 -- 172.21.15.125:0/1858111583 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7aac054990 con 0x7f7ab806b030 2023-10-18T10:37:56.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.237+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7ab80cc460 con 0x7f7ab806b030 2023-10-18T10:37:56.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.931+0000 7f7ab57fa640 1 -- 172.21.15.125:0/1858111583 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7aac017020 con 0x7f7ab806b030 2023-10-18T10:37:56.933 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:37:56.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.933+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7a9c06af60 msgr2=0x7f7a9c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:56.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.933+0000 7f7abeabb640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7a9c06af60 0x7f7a9c06d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4000f60 tx=0x7f7aa40079d0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.933+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab806b030 msgr2=0x7f7ab8152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:56.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.933+0000 7f7abeabb640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab806b030 0x7f7ab8152c40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00d770 tx=0x7f7aac00dc30 comp rx=0 tx=0).stop 2023-10-18T10:37:56.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.934+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 shutdown_connections 2023-10-18T10:37:56.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.934+0000 7f7abeabb640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7a9c06af60 0x7f7a9c06d420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.934+0000 7f7abeabb640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ab8159bd0 0x7f7ab80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.934+0000 7f7abeabb640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ab80c9750 0x7f7ab80ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.934+0000 7f7abeabb640 1 --2- 172.21.15.125:0/1858111583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ab806b030 0x7f7ab8152c40 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:56.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.934+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 >> 172.21.15.125:0/1858111583 conn(0x7f7ab8069930 msgr2=0x7f7ab813a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:56.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.934+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 shutdown_connections 2023-10-18T10:37:56.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:56.934+0000 7f7abeabb640 1 -- 172.21.15.125:0/1858111583 wait complete. 2023-10-18T10:37:56.957 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:37:57.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.063+0000 7f94db905640 1 -- 172.21.15.125:0/3864706195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d406ae40 msgr2=0x7f94d406b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:57.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.063+0000 7f94db905640 1 --2- 172.21.15.125:0/3864706195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d406ae40 0x7f94d406b980 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f94d4066580 tx=0x7f94cc02be40 comp rx=0 tx=0).stop 2023-10-18T10:37:57.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.063+0000 7f94db905640 1 -- 172.21.15.125:0/3864706195 shutdown_connections 2023-10-18T10:37:57.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.063+0000 7f94db905640 1 --2- 172.21.15.125:0/3864706195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d406ae40 0x7f94d406b980 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:57.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.063+0000 7f94db905640 1 --2- 172.21.15.125:0/3864706195 >> v2:172.21.15.139:3300/0 conn(0x7f94d415ad70 0x7f94d40ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:57.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.063+0000 7f94db905640 1 -- 172.21.15.125:0/3864706195 >> 172.21.15.125:0/3864706195 conn(0x7f94d406fae0 msgr2=0x7f94d406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:57.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.063+0000 7f94db905640 1 -- 172.21.15.125:0/3864706195 shutdown_connections 2023-10-18T10:37:57.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.063+0000 7f94db905640 1 -- 172.21.15.125:0/3864706195 wait complete. 2023-10-18T10:37:57.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.064+0000 7f94db905640 1 Processor -- start 2023-10-18T10:37:57.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94db905640 1 -- start start 2023-10-18T10:37:57.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94db905640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406ae40 0x7f94d414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94db905640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d40acc70 0x7f94d414b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94db905640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415ad70 0x7f94d414b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94db905640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4149c40 con 0x7f94d415ad70 2023-10-18T10:37:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94db905640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d414bbd0 con 0x7f94d406ae40 2023-10-18T10:37:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94db905640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d414beb0 con 0x7f94d40acc70 2023-10-18T10:37:57.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94d9ea2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415ad70 0x7f94d414b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:57.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94d9ea2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415ad70 0x7f94d414b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52982/0 (socket says 172.21.15.125:52982) 2023-10-18T10:37:57.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.065+0000 7f94d9ea2640 1 -- 172.21.15.125:0/275917247 learned_addr learned my addr 172.21.15.125:0/275917247 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:57.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d96a1640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406ae40 0x7f94d414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:57.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d8ea0640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d40acc70 0x7f94d414b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:57.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d9ea2640 1 -- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d40acc70 msgr2=0x7f94d414b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:57.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d9ea2640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d40acc70 0x7f94d414b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:57.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d9ea2640 1 -- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406ae40 msgr2=0x7f94d414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:57.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d9ea2640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406ae40 0x7f94d414caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:57.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d9ea2640 1 -- 172.21.15.125:0/275917247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc009560 con 0x7f94d415ad70 2023-10-18T10:37:57.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d8ea0640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d40acc70 0x7f94d414b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:37:57.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94d9ea2640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415ad70 0x7f94d414b5a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f94cc036040 tx=0x7f94cc02db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:57.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94c9ffb640 1 -- 172.21.15.125:0/275917247 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc003800 con 0x7f94d415ad70 2023-10-18T10:37:57.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94c9ffb640 1 -- 172.21.15.125:0/275917247 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94cc003a50 con 0x7f94d415ad70 2023-10-18T10:37:57.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d414c190 con 0x7f94d415ad70 2023-10-18T10:37:57.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94c9ffb640 1 -- 172.21.15.125:0/275917247 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc003e60 con 0x7f94d415ad70 2023-10-18T10:37:57.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.066+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d414c470 con 0x7f94d415ad70 2023-10-18T10:37:57.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.067+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4147dc0 con 0x7f94d415ad70 2023-10-18T10:37:57.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.068+0000 7f94c9ffb640 1 -- 172.21.15.125:0/275917247 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f94cc02dd50 con 0x7f94d415ad70 2023-10-18T10:37:57.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.068+0000 7f94c9ffb640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f94b806af60 0x7f94b806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:57.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.068+0000 7f94c9ffb640 1 -- 172.21.15.125:0/275917247 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f94cc0ac280 con 0x7f94d415ad70 2023-10-18T10:37:57.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.068+0000 7f94d96a1640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f94b806af60 0x7f94b806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:57.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.069+0000 7f94d96a1640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f94b806af60 0x7f94b806d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94d00092f0 tx=0x7f94d0009260 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:57.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.073+0000 7f94c9ffb640 1 -- 172.21.15.125:0/275917247 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94d4147dc0 con 0x7f94d415ad70 2023-10-18T10:37:57.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.275+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f94d4147dc0 con 0x7f94d415ad70 2023-10-18T10:37:57.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.954+0000 7f94c9ffb640 1 -- 172.21.15.125:0/275917247 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f94d4147dc0 con 0x7f94d415ad70 2023-10-18T10:37:57.960 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 7 and data pool 8 2023-10-18T10:37:57.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.966+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f94b806af60 msgr2=0x7f94b806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:57.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.966+0000 7f94db905640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f94b806af60 0x7f94b806d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94d00092f0 tx=0x7f94d0009260 comp rx=0 tx=0).stop 2023-10-18T10:37:57.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.966+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415ad70 msgr2=0x7f94d414b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:57.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.966+0000 7f94db905640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415ad70 0x7f94d414b5a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f94cc036040 tx=0x7f94cc02db60 comp rx=0 tx=0).stop 2023-10-18T10:37:57.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.967+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 shutdown_connections 2023-10-18T10:37:57.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.967+0000 7f94db905640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f94b806af60 0x7f94b806d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:57.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.967+0000 7f94db905640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415ad70 0x7f94d414b5a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:57.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.967+0000 7f94db905640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d40acc70 0x7f94d414b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:57.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.967+0000 7f94db905640 1 --2- 172.21.15.125:0/275917247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406ae40 0x7f94d414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:57.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.967+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 >> 172.21.15.125:0/275917247 conn(0x7f94d406fae0 msgr2=0x7f94d40ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:57.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.970+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 shutdown_connections 2023-10-18T10:37:57.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:57.971+0000 7f94db905640 1 -- 172.21.15.125:0/275917247 wait complete. 2023-10-18T10:37:57.989 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:37:58.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 -- 172.21.15.125:0/4244144506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400cf340 msgr2=0x7fdd400d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/4244144506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400cf340 0x7fdd400d1910 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fdd4008a120 tx=0x7fdd3c02c1d0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 -- 172.21.15.125:0/4244144506 shutdown_connections 2023-10-18T10:37:58.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/4244144506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd400ceb40 0x7fdd400b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/4244144506 >> v2:172.21.15.139:3300/0 conn(0x7fdd4015b2a0 0x7fdd400cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/4244144506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400cf340 0x7fdd400d1910 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 -- 172.21.15.125:0/4244144506 >> 172.21.15.125:0/4244144506 conn(0x7fdd400d0d60 msgr2=0x7fdd400d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:58.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 -- 172.21.15.125:0/4244144506 shutdown_connections 2023-10-18T10:37:58.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.091+0000 7fdd47f2c640 1 -- 172.21.15.125:0/4244144506 wait complete. 2023-10-18T10:37:58.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.092+0000 7fdd47f2c640 1 Processor -- start 2023-10-18T10:37:58.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.092+0000 7fdd47f2c640 1 -- start start 2023-10-18T10:37:58.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd47f2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd4014ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd47f2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd400cf340 0x7fdd4015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd47f2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd4015b2a0 0x7fdd400b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd47f2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4015d130 con 0x7fdd400ceb40 2023-10-18T10:37:58.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd47f2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4015d410 con 0x7fdd400cf340 2023-10-18T10:37:58.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd47f2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4015d6f0 con 0x7fdd4015b2a0 2023-10-18T10:37:58.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd45cc8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd4014ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd45cc8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd4014ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55336/0 (socket says 172.21.15.125:55336) 2023-10-18T10:37:58.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd45cc8640 1 -- 172.21.15.125:0/3539368939 learned_addr learned my addr 172.21.15.125:0/3539368939 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:58.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd45cc8640 1 -- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd4015b2a0 msgr2=0x7fdd400b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd45cc8640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd4015b2a0 0x7fdd400b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd45cc8640 1 -- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd400cf340 msgr2=0x7fdd4015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd45cc8640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd400cf340 0x7fdd4015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.093+0000 7fdd45cc8640 1 -- 172.21.15.125:0/3539368939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009900 con 0x7fdd400ceb40 2023-10-18T10:37:58.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.094+0000 7fdd45cc8640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd4014ff10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c000f80 tx=0x7fdd3c035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:58.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.094+0000 7fdd367fc640 1 -- 172.21.15.125:0/3539368939 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c034440 con 0x7fdd400ceb40 2023-10-18T10:37:58.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.094+0000 7fdd367fc640 1 -- 172.21.15.125:0/3539368939 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd3c004580 con 0x7fdd400ceb40 2023-10-18T10:37:58.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.094+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4015d9d0 con 0x7fdd400ceb40 2023-10-18T10:37:58.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.094+0000 7fdd367fc640 1 -- 172.21.15.125:0/3539368939 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c040450 con 0x7fdd400ceb40 2023-10-18T10:37:58.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.094+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd40150670 con 0x7fdd400ceb40 2023-10-18T10:37:58.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.095+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd401508c0 con 0x7fdd400ceb40 2023-10-18T10:37:58.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.095+0000 7fdd367fc640 1 -- 172.21.15.125:0/3539368939 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdd3c00a040 con 0x7fdd400ceb40 2023-10-18T10:37:58.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.096+0000 7fdd367fc640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fdd2406af60 0x7fdd2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.096+0000 7fdd367fc640 1 -- 172.21.15.125:0/3539368939 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdd3c0acaf0 con 0x7fdd400ceb40 2023-10-18T10:37:58.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.096+0000 7fdd454c7640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fdd2406af60 0x7fdd2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.096+0000 7fdd454c7640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fdd2406af60 0x7fdd2406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd400c7390 tx=0x7fdd30005bd0 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:58.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.100+0000 7fdd367fc640 1 -- 172.21.15.125:0/3539368939 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd3c077950 con 0x7fdd400ceb40 2023-10-18T10:37:58.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.267+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd4015def0 con 0x7fdd400ceb40 2023-10-18T10:37:58.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.268+0000 7fdd367fc640 1 -- 172.21.15.125:0/3539368939 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16835 (secure 0 0 0) 0x7fdd3c039020 con 0x7fdd400ceb40 2023-10-18T10:37:58.270 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:58.270 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":64,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:57.277257+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:37:55.100958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-18T10:37:56.256769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:37:58.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fdd2406af60 msgr2=0x7fdd2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fdd2406af60 0x7fdd2406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd400c7390 tx=0x7fdd30005bd0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 msgr2=0x7fdd4014ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd4014ff10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c000f80 tx=0x7fdd3c035db0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 shutdown_connections 2023-10-18T10:37:58.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fdd2406af60 0x7fdd2406d420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd400cf340 0x7fdd4015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd4015b2a0 0x7fdd400b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 --2- 172.21.15.125:0/3539368939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd4014ff10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 >> 172.21.15.125:0/3539368939 conn(0x7fdd400d0d60 msgr2=0x7fdd400b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:58.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 shutdown_connections 2023-10-18T10:37:58.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.270+0000 7fdd47f2c640 1 -- 172.21.15.125:0/3539368939 wait complete. 2023-10-18T10:37:58.283 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:37:58.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.384+0000 7f679f6d7640 1 -- 172.21.15.125:0/1485323560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 msgr2=0x7f67980b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.384+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1485323560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 0x7f67980b0e90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f67980acd30 tx=0x7f678802bd70 comp rx=0 tx=0).stop 2023-10-18T10:37:58.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.384+0000 7f679f6d7640 1 -- 172.21.15.125:0/1485323560 shutdown_connections 2023-10-18T10:37:58.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.384+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1485323560 >> v2:172.21.15.139:3300/0 conn(0x7f679815c1f0 0x7f6798157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.384+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1485323560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 0x7f67980b0e90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.384+0000 7f679f6d7640 1 -- 172.21.15.125:0/1485323560 >> 172.21.15.125:0/1485323560 conn(0x7f679806fae0 msgr2=0x7f679806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:58.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.384+0000 7f679f6d7640 1 -- 172.21.15.125:0/1485323560 shutdown_connections 2023-10-18T10:37:58.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.384+0000 7f679f6d7640 1 -- 172.21.15.125:0/1485323560 wait complete. 2023-10-18T10:37:58.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.385+0000 7f679f6d7640 1 Processor -- start 2023-10-18T10:37:58.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.385+0000 7f679f6d7640 1 -- start start 2023-10-18T10:37:58.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.385+0000 7f679f6d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 0x7f679814a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679f6d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67981579f0 0x7f679813c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679f6d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f679815c1f0 0x7f679813c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679f6d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679814eb40 con 0x7f679806ae40 2023-10-18T10:37:58.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679f6d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6798155590 con 0x7f67981579f0 2023-10-18T10:37:58.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679f6d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6798155870 con 0x7f679815c1f0 2023-10-18T10:37:58.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 0x7f679814a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 0x7f679814a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55362/0 (socket says 172.21.15.125:55362) 2023-10-18T10:37:58.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 -- 172.21.15.125:0/1302369649 learned_addr learned my addr 172.21.15.125:0/1302369649 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:58.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679dc74640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f679815c1f0 0x7f679813c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679cc72640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67981579f0 0x7f679813c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 -- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f679815c1f0 msgr2=0x7f679813c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f679815c1f0 0x7f679813c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 -- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67981579f0 msgr2=0x7f679813c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67981579f0 0x7f679813c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 -- 172.21.15.125:0/1302369649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788009490 con 0x7f679806ae40 2023-10-18T10:37:58.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679d473640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 0x7f679814a210 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f679400c4b0 tx=0x7f679400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:58.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f678dffb640 1 -- 172.21.15.125:0/1302369649 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794005480 con 0x7f679806ae40 2023-10-18T10:37:58.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f678dffb640 1 -- 172.21.15.125:0/1302369649 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f67940036e0 con 0x7f679806ae40 2023-10-18T10:37:58.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67980b1de0 con 0x7f679806ae40 2023-10-18T10:37:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.386+0000 7f678dffb640 1 -- 172.21.15.125:0/1302369649 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794003b30 con 0x7f679806ae40 2023-10-18T10:37:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.387+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679814ee20 con 0x7f679806ae40 2023-10-18T10:37:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.387+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67981553f0 con 0x7f679806ae40 2023-10-18T10:37:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.388+0000 7f678dffb640 1 -- 172.21.15.125:0/1302369649 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6794003c90 con 0x7f679806ae40 2023-10-18T10:37:58.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.388+0000 7f678dffb640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f677c06af60 0x7f677c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.388+0000 7f678dffb640 1 -- 172.21.15.125:0/1302369649 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f67940890c0 con 0x7f679806ae40 2023-10-18T10:37:58.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.389+0000 7f679cc72640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f677c06af60 0x7f677c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.389+0000 7f679cc72640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f677c06af60 0x7f677c06d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6788036000 tx=0x7f6788004730 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:58.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.393+0000 7f678dffb640 1 -- 172.21.15.125:0/1302369649 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67981553f0 con 0x7f679806ae40 2023-10-18T10:37:58.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.560+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67981553f0 con 0x7f679806ae40 2023-10-18T10:37:58.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.561+0000 7f678dffb640 1 -- 172.21.15.125:0/1302369649 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16835 (secure 0 0 0) 0x7f67981553f0 con 0x7f679806ae40 2023-10-18T10:37:58.563 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:37:58.563 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":64,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:57.277257+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:37:55.100958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-18T10:37:56.256769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:37:58.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f677c06af60 msgr2=0x7f677c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f677c06af60 0x7f677c06d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6788036000 tx=0x7f6788004730 comp rx=0 tx=0).stop 2023-10-18T10:37:58.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 msgr2=0x7f679814a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 0x7f679814a210 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f679400c4b0 tx=0x7f679400c970 comp rx=0 tx=0).stop 2023-10-18T10:37:58.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 shutdown_connections 2023-10-18T10:37:58.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f677c06af60 0x7f677c06d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67981579f0 0x7f679813c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f679815c1f0 0x7f679813c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 --2- 172.21.15.125:0/1302369649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f679806ae40 0x7f679814a210 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 >> 172.21.15.125:0/1302369649 conn(0x7f679806fae0 msgr2=0x7f6798150e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:58.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 shutdown_connections 2023-10-18T10:37:58.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.563+0000 7f679f6d7640 1 -- 172.21.15.125:0/1302369649 wait complete. 2023-10-18T10:37:58.575 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:37:58.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.679+0000 7f8976867640 1 -- 172.21.15.125:0/3764703835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4a70 msgr2=0x7f896803a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.679+0000 7f8976867640 1 --2- 172.21.15.125:0/3764703835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4a70 0x7f896803a780 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8960009830 tx=0x7f896002bd60 comp rx=0 tx=0).stop 2023-10-18T10:37:58.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.679+0000 7f8976867640 1 -- 172.21.15.125:0/3764703835 shutdown_connections 2023-10-18T10:37:58.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.679+0000 7f8976867640 1 --2- 172.21.15.125:0/3764703835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4a70 0x7f896803a780 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.679+0000 7f8976867640 1 -- 172.21.15.125:0/3764703835 >> 172.21.15.125:0/3764703835 conn(0x7f896801e690 msgr2=0x7f89680553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:58.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.679+0000 7f8976867640 1 -- 172.21.15.125:0/3764703835 shutdown_connections 2023-10-18T10:37:58.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.679+0000 7f8976867640 1 -- 172.21.15.125:0/3764703835 wait complete. 2023-10-18T10:37:58.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.680+0000 7f8976867640 1 Processor -- start 2023-10-18T10:37:58.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.680+0000 7f8976867640 1 -- start start 2023-10-18T10:37:58.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8976867640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89680341b0 0x7f896804ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8976867640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4690 0x7f89680548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8976867640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89680e4a70 0x7f89680d1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8976867640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896804aa00 con 0x7f89680e4690 2023-10-18T10:37:58.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8976867640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8968053b10 con 0x7f89680e4a70 2023-10-18T10:37:58.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8976867640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8968053df0 con 0x7f89680341b0 2023-10-18T10:37:58.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4690 0x7f89680548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975865640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89680341b0 0x7f896804ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4690 0x7f89680548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55380/0 (socket says 172.21.15.125:55380) 2023-10-18T10:37:58.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 -- 172.21.15.125:0/1937875643 learned_addr learned my addr 172.21.15.125:0/1937875643 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:58.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8976066640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89680e4a70 0x7f89680d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 -- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89680341b0 msgr2=0x7f896804ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89680341b0 0x7f896804ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 -- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89680e4a70 msgr2=0x7f89680d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:58.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89680e4a70 0x7f89680d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:58.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 -- 172.21.15.125:0/1937875643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8960009490 con 0x7f89680e4690 2023-10-18T10:37:58.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.681+0000 7f8975064640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4690 0x7f89680548d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8960009950 tx=0x7f89600038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:58.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.682+0000 7f89667fc640 1 -- 172.21.15.125:0/1937875643 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960003ea0 con 0x7f89680e4690 2023-10-18T10:37:58.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.682+0000 7f89667fc640 1 -- 172.21.15.125:0/1937875643 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8960005020 con 0x7f89680e4690 2023-10-18T10:37:58.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.682+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89680540d0 con 0x7f89680e4690 2023-10-18T10:37:58.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.682+0000 7f89667fc640 1 -- 172.21.15.125:0/1937875643 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960005410 con 0x7f89680e4690 2023-10-18T10:37:58.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.682+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896804f1f0 con 0x7f89680e4690 2023-10-18T10:37:58.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.683+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89680545f0 con 0x7f89680e4690 2023-10-18T10:37:58.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.683+0000 7f89667fc640 1 -- 172.21.15.125:0/1937875643 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f89600047a0 con 0x7f89680e4690 2023-10-18T10:37:58.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.683+0000 7f89667fc640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f895406af60 0x7f895406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:58.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.683+0000 7f89667fc640 1 -- 172.21.15.125:0/1937875643 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f89600ac0a0 con 0x7f89680e4690 2023-10-18T10:37:58.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.684+0000 7f8975865640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f895406af60 0x7f895406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:58.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.684+0000 7f8975865640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f895406af60 0x7f895406d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f896c000f60 tx=0x7f896c009040 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:58.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.688+0000 7f89667fc640 1 -- 172.21.15.125:0/1937875643 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f89680545f0 con 0x7f89680e4690 2023-10-18T10:37:58.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:58.888+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89680545f0 con 0x7f89680e4690 2023-10-18T10:37:59.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.336+0000 7f89667fc640 1 -- 172.21.15.125:0/1937875643 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f89680545f0 con 0x7f89680e4690 2023-10-18T10:37:59.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f895406af60 msgr2=0x7f895406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:59.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f895406af60 0x7f895406d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f896c000f60 tx=0x7f896c009040 comp rx=0 tx=0).stop 2023-10-18T10:37:59.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4690 msgr2=0x7f89680548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:59.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4690 0x7f89680548d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8960009950 tx=0x7f89600038a0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 shutdown_connections 2023-10-18T10:37:59.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f895406af60 0x7f895406d420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89680e4a70 0x7f89680d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f89680e4690 0x7f89680548d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 --2- 172.21.15.125:0/1937875643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89680341b0 0x7f896804ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 >> 172.21.15.125:0/1937875643 conn(0x7f896801e690 msgr2=0x7f89680345e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:59.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 shutdown_connections 2023-10-18T10:37:59.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.338+0000 7f8976867640 1 -- 172.21.15.125:0/1937875643 wait complete. 2023-10-18T10:37:59.352 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:37:59.352 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:37:59.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.457+0000 7f8d50d7d640 1 -- 172.21.15.125:0/2548170109 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 msgr2=0x7f8d38004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:59.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.457+0000 7f8d50d7d640 1 --2- 172.21.15.125:0/2548170109 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 0x7f8d38004c40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8d34001680 tx=0x7f8d380057f0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.458+0000 7f8d50d7d640 1 -- 172.21.15.125:0/2548170109 shutdown_connections 2023-10-18T10:37:59.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.458+0000 7f8d50d7d640 1 --2- 172.21.15.125:0/2548170109 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 0x7f8d38004c40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.458+0000 7f8d50d7d640 1 -- 172.21.15.125:0/2548170109 >> 172.21.15.125:0/2548170109 conn(0x7f8d4c0d0d60 msgr2=0x7f8d4c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:37:59.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.458+0000 7f8d50d7d640 1 -- 172.21.15.125:0/2548170109 shutdown_connections 2023-10-18T10:37:59.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.458+0000 7f8d50d7d640 1 -- 172.21.15.125:0/2548170109 wait complete. 2023-10-18T10:37:59.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.459+0000 7f8d50d7d640 1 Processor -- start 2023-10-18T10:37:59.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.459+0000 7f8d50d7d640 1 -- start start 2023-10-18T10:37:59.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d50d7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 0x7f8d4c153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:59.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d50d7d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d4c1f4d30 0x7f8d4c1f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:59.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d50d7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d4c0aee90 0x7f8d4c0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:59.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d50d7d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1e57a0 con 0x7f8d380047c0 2023-10-18T10:37:59.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d50d7d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1f5c20 con 0x7f8d4c0aee90 2023-10-18T10:37:59.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d50d7d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0c85b0 con 0x7f8d4c1f4d30 2023-10-18T10:37:59.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 0x7f8d4c153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:59.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 0x7f8d4c153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55396/0 (socket says 172.21.15.125:55396) 2023-10-18T10:37:59.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 -- 172.21.15.125:0/1947852276 learned_addr learned my addr 172.21.15.125:0/1947852276 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:37:59.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 -- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d4c1f4d30 msgr2=0x7f8d4c1f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:59.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d4c1f4d30 0x7f8d4c1f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 -- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d4c0aee90 msgr2=0x7f8d4c0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:37:59.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d4c0aee90 0x7f8d4c0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:37:59.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 -- 172.21.15.125:0/1947852276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c1f56f0 con 0x7f8d380047c0 2023-10-18T10:37:59.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d4a59c640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 0x7f8d4c153bb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c011b10 tx=0x7f8d3c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:59.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d437fe640 1 -- 172.21.15.125:0/1947852276 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00fc60 con 0x7f8d380047c0 2023-10-18T10:37:59.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d437fe640 1 -- 172.21.15.125:0/1947852276 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d3c00fdc0 con 0x7f8d380047c0 2023-10-18T10:37:59.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c0af850 con 0x7f8d380047c0 2023-10-18T10:37:59.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d437fe640 1 -- 172.21.15.125:0/1947852276 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c004280 con 0x7f8d380047c0 2023-10-18T10:37:59.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.460+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c1f5940 con 0x7f8d380047c0 2023-10-18T10:37:59.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.462+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c0690e0 con 0x7f8d380047c0 2023-10-18T10:37:59.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.462+0000 7f8d437fe640 1 -- 172.21.15.125:0/1947852276 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8d3c0035f0 con 0x7f8d380047c0 2023-10-18T10:37:59.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.462+0000 7f8d437fe640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8d2c06af90 0x7f8d2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:37:59.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.463+0000 7f8d437fe640 1 -- 172.21.15.125:0/1947852276 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8d3c08a3b0 con 0x7f8d380047c0 2023-10-18T10:37:59.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.463+0000 7f8d41d9b640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8d2c06af90 0x7f8d2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:37:59.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.463+0000 7f8d41d9b640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8d2c06af90 0x7f8d2c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8d34034000 tx=0x7f8d34001120 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:37:59.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.467+0000 7f8d437fe640 1 -- 172.21.15.125:0/1947852276 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d3c055190 con 0x7f8d380047c0 2023-10-18T10:37:59.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:37:59.668+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8d4c0d0b40 con 0x7f8d380047c0 2023-10-18T10:38:00.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.343+0000 7f8d437fe640 1 -- 172.21.15.125:0/1947852276 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8d3c0587b0 con 0x7f8d380047c0 2023-10-18T10:38:00.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.345+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8d2c06af90 msgr2=0x7f8d2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.345+0000 7f8d50d7d640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8d2c06af90 0x7f8d2c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8d34034000 tx=0x7f8d34001120 comp rx=0 tx=0).stop 2023-10-18T10:38:00.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.345+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 msgr2=0x7f8d4c153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.345+0000 7f8d50d7d640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 0x7f8d4c153bb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c011b10 tx=0x7f8d3c009f10 comp rx=0 tx=0).stop 2023-10-18T10:38:00.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.346+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 shutdown_connections 2023-10-18T10:38:00.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.346+0000 7f8d50d7d640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f8d2c06af90 0x7f8d2c06d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.346+0000 7f8d50d7d640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d4c0aee90 0x7f8d4c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.346+0000 7f8d50d7d640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d4c1f4d30 0x7f8d4c1f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.346+0000 7f8d50d7d640 1 --2- 172.21.15.125:0/1947852276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d380047c0 0x7f8d4c153bb0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.346+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 >> 172.21.15.125:0/1947852276 conn(0x7f8d4c0d0d60 msgr2=0x7f8d4c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:00.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.346+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 shutdown_connections 2023-10-18T10:38:00.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.346+0000 7f8d50d7d640 1 -- 172.21.15.125:0/1947852276 wait complete. 2023-10-18T10:38:00.359 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:38:00.359 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:38:00.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.462+0000 7f089ae39640 1 -- 172.21.15.125:0/3117484080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0894070920 msgr2=0x7f08940affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.462+0000 7f089ae39640 1 --2- 172.21.15.125:0/3117484080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0894070920 0x7f08940affe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f088c005fd0 tx=0x7f088c02c050 comp rx=0 tx=0).stop 2023-10-18T10:38:00.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.462+0000 7f089ae39640 1 -- 172.21.15.125:0/3117484080 shutdown_connections 2023-10-18T10:38:00.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.462+0000 7f089ae39640 1 --2- 172.21.15.125:0/3117484080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0894070920 0x7f08940affe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.462+0000 7f089ae39640 1 --2- 172.21.15.125:0/3117484080 >> v2:172.21.15.139:3300/0 conn(0x7f0894141690 0x7f08940703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.462+0000 7f089ae39640 1 -- 172.21.15.125:0/3117484080 >> 172.21.15.125:0/3117484080 conn(0x7f0894069930 msgr2=0x7f0894069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:00.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.462+0000 7f089ae39640 1 -- 172.21.15.125:0/3117484080 shutdown_connections 2023-10-18T10:38:00.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.462+0000 7f089ae39640 1 -- 172.21.15.125:0/3117484080 wait complete. 2023-10-18T10:38:00.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.463+0000 7f089ae39640 1 Processor -- start 2023-10-18T10:38:00.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.463+0000 7f089ae39640 1 -- start start 2023-10-18T10:38:00.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f089ae39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0894070920 0x7f08941e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:00.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f089ae39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f089413e520 0x7f08941f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:00.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f089ae39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0894141690 0x7f08941f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:00.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f089ae39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08940ab1b0 con 0x7f0894141690 2023-10-18T10:38:00.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f089ae39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08940aa630 con 0x7f0894070920 2023-10-18T10:38:00.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f089ae39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08940aa910 con 0x7f089413e520 2023-10-18T10:38:00.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f08993d6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0894141690 0x7f08941f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:00.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f08993d6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0894141690 0x7f08941f2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55404/0 (socket says 172.21.15.125:55404) 2023-10-18T10:38:00.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f08993d6640 1 -- 172.21.15.125:0/322624026 learned_addr learned my addr 172.21.15.125:0/322624026 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:00.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f0893fff640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f089413e520 0x7f08941f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:00.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.464+0000 7f0898bd5640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0894070920 0x7f08941e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:00.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08993d6640 1 -- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f089413e520 msgr2=0x7f08941f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08993d6640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f089413e520 0x7f08941f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08993d6640 1 -- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0894070920 msgr2=0x7f08941e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08993d6640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0894070920 0x7f08941e4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08993d6640 1 -- 172.21.15.125:0/322624026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0880009560 con 0x7f0894141690 2023-10-18T10:38:00.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08993d6640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0894141690 0x7f08941f2f30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f088c009cd0 tx=0x7f088c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:00.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08917fa640 1 -- 172.21.15.125:0/322624026 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c03e460 con 0x7f0894141690 2023-10-18T10:38:00.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c009840 con 0x7f0894141690 2023-10-18T10:38:00.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08917fa640 1 -- 172.21.15.125:0/322624026 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f088c03e6b0 con 0x7f0894141690 2023-10-18T10:38:00.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f08917fa640 1 -- 172.21.15.125:0/322624026 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c03ea80 con 0x7f0894141690 2023-10-18T10:38:00.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.465+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08940aaed0 con 0x7f0894141690 2023-10-18T10:38:00.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.466+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08940690e0 con 0x7f0894141690 2023-10-18T10:38:00.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.466+0000 7f08917fa640 1 -- 172.21.15.125:0/322624026 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f088c03ebe0 con 0x7f0894141690 2023-10-18T10:38:00.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.467+0000 7f08917fa640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f087806af60 0x7f087806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:00.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.467+0000 7f08917fa640 1 -- 172.21.15.125:0/322624026 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f088c0ac890 con 0x7f0894141690 2023-10-18T10:38:00.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.467+0000 7f0898bd5640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f087806af60 0x7f087806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:00.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.468+0000 7f0898bd5640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f087806af60 0x7f087806d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08940c9190 tx=0x7f0888009040 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:00.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.471+0000 7f08917fa640 1 -- 172.21.15.125:0/322624026 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f088c077720 con 0x7f0894141690 2023-10-18T10:38:00.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.670+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08940c9870 con 0x7f0894141690 2023-10-18T10:38:00.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.671+0000 7f08917fa640 1 -- 172.21.15.125:0/322624026 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f088c042090 con 0x7f0894141690 2023-10-18T10:38:00.673 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:38:00.674 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1045257948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1045257948},{"type":"v1","addr":"172.21.15.139:6836","nonce":1045257948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/4183044691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4183044691},{"type":"v1","addr":"172.21.15.139:6837","nonce":4183044691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1083462482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1083462482},{"type":"v1","addr":"172.21.15.125:6835","nonce":1083462482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:37:57.277572+0000","modified":"2023-10-18T10:37:59.669781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/615856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":615856560},{"type":"v1","addr":"172.21.15.125:6834","nonce":615856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T10:38:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 23 2023-10-18T10:38:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f087806af60 msgr2=0x7f087806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f087806af60 0x7f087806d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08940c9190 tx=0x7f0888009040 comp rx=0 tx=0).stop 2023-10-18T10:38:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0894141690 msgr2=0x7f08941f2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0894141690 0x7f08941f2f30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f088c009cd0 tx=0x7f088c0042f0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 shutdown_connections 2023-10-18T10:38:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f087806af60 0x7f087806d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0894141690 0x7f08941f2f30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f089413e520 0x7f08941f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 --2- 172.21.15.125:0/322624026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0894070920 0x7f08941e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 >> 172.21.15.125:0/322624026 conn(0x7f0894069930 msgr2=0x7f089413d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 shutdown_connections 2023-10-18T10:38:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.673+0000 7f089ae39640 1 -- 172.21.15.125:0/322624026 wait complete. 2023-10-18T10:38:00.685 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:38:00.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 -- 172.21.15.125:0/3057264200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80ceb40 msgr2=0x7fe5e80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/3057264200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80ceb40 0x7fe5e80b0650 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe5e806c510 tx=0x7fe5e002c1d0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 -- 172.21.15.125:0/3057264200 shutdown_connections 2023-10-18T10:38:00.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/3057264200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80ceb40 0x7fe5e80b0650 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/3057264200 >> v2:172.21.15.139:3300/0 conn(0x7fe5e815b2a0 0x7fe5e80cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/3057264200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5e80cf340 0x7fe5e80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 -- 172.21.15.125:0/3057264200 >> 172.21.15.125:0/3057264200 conn(0x7fe5e80d0d60 msgr2=0x7fe5e80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:00.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 -- 172.21.15.125:0/3057264200 shutdown_connections 2023-10-18T10:38:00.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.786+0000 7fe5edf2c640 1 -- 172.21.15.125:0/3057264200 wait complete. 2023-10-18T10:38:00.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.787+0000 7fe5edf2c640 1 Processor -- start 2023-10-18T10:38:00.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.787+0000 7fe5edf2c640 1 -- start start 2023-10-18T10:38:00.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5edf2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5e80ceb40 0x7fe5e81e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:00.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5edf2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80cf340 0x7fe5e80ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:00.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5edf2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5e815b2a0 0x7fe5e80cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:00.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5edf2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80c9410 con 0x7fe5e80cf340 2023-10-18T10:38:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5edf2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e81e66d0 con 0x7fe5e815b2a0 2023-10-18T10:38:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5edf2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e81e63f0 con 0x7fe5e80ceb40 2023-10-18T10:38:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5deffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80cf340 0x7fe5e80ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5deffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80cf340 0x7fe5e80ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55428/0 (socket says 172.21.15.125:55428) 2023-10-18T10:38:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.788+0000 7fe5deffd640 1 -- 172.21.15.125:0/1308943985 learned_addr learned my addr 172.21.15.125:0/1308943985 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5deffd640 1 -- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5e80ceb40 msgr2=0x7fe5e81e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5deffd640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5e80ceb40 0x7fe5e81e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5deffd640 1 -- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5e815b2a0 msgr2=0x7fe5e80cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5deffd640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5e815b2a0 0x7fe5e80cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5deffd640 1 -- 172.21.15.125:0/1308943985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0009900 con 0x7fe5e80cf340 2023-10-18T10:38:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5deffd640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80cf340 0x7fe5e80ca120 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009fd0 tx=0x7fe5d000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5e4ff9640 1 -- 172.21.15.125:0/1308943985 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0009580 con 0x7fe5e80cf340 2023-10-18T10:38:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5e4ff9640 1 -- 172.21.15.125:0/1308943985 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe5d000ccc0 con 0x7fe5e80cf340 2023-10-18T10:38:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5e4ff9640 1 -- 172.21.15.125:0/1308943985 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0005280 con 0x7fe5e80cf340 2023-10-18T10:38:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e80c9640 con 0x7fe5e80cf340 2023-10-18T10:38:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.789+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e80c9c00 con 0x7fe5e80cf340 2023-10-18T10:38:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.790+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e81e6140 con 0x7fe5e80cf340 2023-10-18T10:38:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.790+0000 7fe5e4ff9640 1 -- 172.21.15.125:0/1308943985 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe5d00053e0 con 0x7fe5e80cf340 2023-10-18T10:38:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.791+0000 7fe5e4ff9640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe5c806af60 0x7fe5c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.791+0000 7fe5e4ff9640 1 -- 172.21.15.125:0/1308943985 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe5d0014070 con 0x7fe5e80cf340 2023-10-18T10:38:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.791+0000 7fe5e77fe640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe5c806af60 0x7fe5c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:00.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.791+0000 7fe5e77fe640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe5c806af60 0x7fe5c806d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009ad0 tx=0x7fe5d8009590 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:00.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.795+0000 7fe5e4ff9640 1 -- 172.21.15.125:0/1308943985 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5d0055060 con 0x7fe5e80cf340 2023-10-18T10:38:00.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.962+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e80690e0 con 0x7fe5e80cf340 2023-10-18T10:38:00.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.963+0000 7fe5e4ff9640 1 -- 172.21.15.125:0/1308943985 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16822 (secure 0 0 0) 0x7fe5d0058680 con 0x7fe5e80cf340 2023-10-18T10:38:00.965 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:38:00.965 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":65,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:37:58.952546+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:37:55.100958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-18T10:37:56.256769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:38:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe5c806af60 msgr2=0x7fe5c806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe5c806af60 0x7fe5c806d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009ad0 tx=0x7fe5d8009590 comp rx=0 tx=0).stop 2023-10-18T10:38:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80cf340 msgr2=0x7fe5e80ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80cf340 0x7fe5e80ca120 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009fd0 tx=0x7fe5d000ec20 comp rx=0 tx=0).stop 2023-10-18T10:38:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 shutdown_connections 2023-10-18T10:38:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fe5c806af60 0x7fe5c806d420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5e815b2a0 0x7fe5e80cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5e80cf340 0x7fe5e80ca120 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 --2- 172.21.15.125:0/1308943985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5e80ceb40 0x7fe5e81e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 >> 172.21.15.125:0/1308943985 conn(0x7fe5e80d0d60 msgr2=0x7fe5e815a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 shutdown_connections 2023-10-18T10:38:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:00.965+0000 7fe5edf2c640 1 -- 172.21.15.125:0/1308943985 wait complete. 2023-10-18T10:38:00.978 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:38:00.978 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:38:01.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.078+0000 7f1062e4b640 1 -- 172.21.15.125:0/3192474232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c06af00 msgr2=0x7f105c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.078+0000 7f1062e4b640 1 --2- 172.21.15.125:0/3192474232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c06af00 0x7f105c06b2e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1054009be0 tx=0x7f105402c0c0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.078+0000 7f1062e4b640 1 -- 172.21.15.125:0/3192474232 shutdown_connections 2023-10-18T10:38:01.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.078+0000 7f1062e4b640 1 --2- 172.21.15.125:0/3192474232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f105c0abc10 0x7f105c0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.078+0000 7f1062e4b640 1 --2- 172.21.15.125:0/3192474232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c06af00 0x7f105c06b2e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.078+0000 7f1062e4b640 1 -- 172.21.15.125:0/3192474232 >> 172.21.15.125:0/3192474232 conn(0x7f105c069930 msgr2=0x7f105c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:01.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.078+0000 7f1062e4b640 1 -- 172.21.15.125:0/3192474232 shutdown_connections 2023-10-18T10:38:01.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.078+0000 7f1062e4b640 1 -- 172.21.15.125:0/3192474232 wait complete. 2023-10-18T10:38:01.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.079+0000 7f1062e4b640 1 Processor -- start 2023-10-18T10:38:01.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.079+0000 7f1062e4b640 1 -- start start 2023-10-18T10:38:01.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1062e4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f105c06af00 0x7f105c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1062e4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0abc10 0x7f105c0a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1062e4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f105c0c96b0 0x7f105c0a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1062e4b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c1f3a60 con 0x7f105c06af00 2023-10-18T10:38:01.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1062e4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0a81f0 con 0x7f105c0abc10 2023-10-18T10:38:01.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1062e4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c15b480 con 0x7f105c0c96b0 2023-10-18T10:38:01.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1060be7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f105c06af00 0x7f105c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1060be7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f105c06af00 0x7f105c15b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55444/0 (socket says 172.21.15.125:55444) 2023-10-18T10:38:01.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1060be7640 1 -- 172.21.15.125:0/1060588250 learned_addr learned my addr 172.21.15.125:0/1060588250 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:01.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f10613e8640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f105c0c96b0 0x7f105c0a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1060be7640 1 -- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f105c0c96b0 msgr2=0x7f105c0a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1060be7640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f105c0c96b0 0x7f105c0a9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1060be7640 1 -- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0abc10 msgr2=0x7f105c0a7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f105bfff640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0abc10 0x7f105c0a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1060be7640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0abc10 0x7f105c0a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.080+0000 7f1060be7640 1 -- 172.21.15.125:0/1060588250 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1054009840 con 0x7f105c06af00 2023-10-18T10:38:01.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.081+0000 7f1060be7640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f105c06af00 0x7f105c15b910 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1054009d00 tx=0x7f10540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:01.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.081+0000 7f10597fa640 1 -- 172.21.15.125:0/1060588250 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540046f0 con 0x7f105c06af00 2023-10-18T10:38:01.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.081+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c15a870 con 0x7f105c06af00 2023-10-18T10:38:01.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.081+0000 7f10597fa640 1 -- 172.21.15.125:0/1060588250 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f10540345a0 con 0x7f105c06af00 2023-10-18T10:38:01.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.081+0000 7f10597fa640 1 -- 172.21.15.125:0/1060588250 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054034990 con 0x7f105c06af00 2023-10-18T10:38:01.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.081+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c15ae30 con 0x7f105c06af00 2023-10-18T10:38:01.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.082+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c13bad0 con 0x7f105c06af00 2023-10-18T10:38:01.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.082+0000 7f10597fa640 1 -- 172.21.15.125:0/1060588250 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1054034af0 con 0x7f105c06af00 2023-10-18T10:38:01.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.083+0000 7f10597fa640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f103c06af60 0x7f103c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.083+0000 7f10597fa640 1 -- 172.21.15.125:0/1060588250 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f10540ac9d0 con 0x7f105c06af00 2023-10-18T10:38:01.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.083+0000 7f105bfff640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f103c06af60 0x7f103c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.083+0000 7f105bfff640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f103c06af60 0x7f103c06d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f105c06d110 tx=0x7f1044009040 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:01.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.087+0000 7f10597fa640 1 -- 172.21.15.125:0/1060588250 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10540777b0 con 0x7f105c06af00 2023-10-18T10:38:01.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.249+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 --> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f105c159c50 con 0x7f103c06af60 2023-10-18T10:38:01.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.257+0000 7f10597fa640 1 -- 172.21.15.125:0/1060588250 <== mgr.5626 v2:172.21.15.139:6832/2140727580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289783 (secure 0 0 0) 0x7f10440123d0 con 0x7f103c06af60 2023-10-18T10:38:01.263 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:38:01.264 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:38:01.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f103c06af60 msgr2=0x7f103c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f103c06af60 0x7f103c06d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f105c06d110 tx=0x7f1044009040 comp rx=0 tx=0).stop 2023-10-18T10:38:01.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f105c06af00 msgr2=0x7f105c15b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f105c06af00 0x7f105c15b910 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1054009d00 tx=0x7f10540042f0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 shutdown_connections 2023-10-18T10:38:01.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f103c06af60 0x7f103c06d420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0abc10 0x7f105c0a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f105c0c96b0 0x7f105c0a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 --2- 172.21.15.125:0/1060588250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f105c06af00 0x7f105c15b910 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 >> 172.21.15.125:0/1060588250 conn(0x7f105c069930 msgr2=0x7f105c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 shutdown_connections 2023-10-18T10:38:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.260+0000 7f1062e4b640 1 -- 172.21.15.125:0/1060588250 wait complete. 2023-10-18T10:38:01.273 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:37:59.832855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297040,"kb_used_data":7304,"kb_used_omap":12,"kb_used_meta":289587,"kb_avail":754677680,"statfs":{"total":773094113280,"available":772789944320,"internally_reserved":0,"allocated":7479296,"data_stored":4220284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":296537656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000859"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319834+0000","last_change":"2023-10-18T10:37:57.953401+0000","last_active":"2023-10-18T10:37:59.319834+0000","last_peered":"2023-10-18T10:37:59.319834+0000","last_clean":"2023-10-18T10:37:59.319834+0000","last_became_active":"2023-10-18T10:37:57.953090+0000","last_became_peered":"2023-10-18T10:37:57.953090+0000","last_unstale":"2023-10-18T10:37:59.319834+0000","last_undegraded":"2023-10-18T10:37:59.319834+0000","last_fullsized":"2023-10-18T10:37:59.319834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:55:30.673197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320473+0000","last_change":"2023-10-18T10:37:57.966749+0000","last_active":"2023-10-18T10:37:59.320473+0000","last_peered":"2023-10-18T10:37:59.320473+0000","last_clean":"2023-10-18T10:37:59.320473+0000","last_became_active":"2023-10-18T10:37:57.966495+0000","last_became_peered":"2023-10-18T10:37:57.966495+0000","last_unstale":"2023-10-18T10:37:59.320473+0000","last_undegraded":"2023-10-18T10:37:59.320473+0000","last_fullsized":"2023-10-18T10:37:59.320473+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:49:09.644289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319428+0000","last_change":"2023-10-18T10:37:57.965988+0000","last_active":"2023-10-18T10:37:59.319428+0000","last_peered":"2023-10-18T10:37:59.319428+0000","last_clean":"2023-10-18T10:37:59.319428+0000","last_became_active":"2023-10-18T10:37:57.965152+0000","last_became_peered":"2023-10-18T10:37:57.965152+0000","last_unstale":"2023-10-18T10:37:59.319428+0000","last_undegraded":"2023-10-18T10:37:59.319428+0000","last_fullsized":"2023-10-18T10:37:59.319428+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:44.808490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320075+0000","last_change":"2023-10-18T10:37:57.977203+0000","last_active":"2023-10-18T10:37:59.320075+0000","last_peered":"2023-10-18T10:37:59.320075+0000","last_clean":"2023-10-18T10:37:59.320075+0000","last_became_active":"2023-10-18T10:37:57.976913+0000","last_became_peered":"2023-10-18T10:37:57.976913+0000","last_unstale":"2023-10-18T10:37:59.320075+0000","last_undegraded":"2023-10-18T10:37:59.320075+0000","last_fullsized":"2023-10-18T10:37:59.320075+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:44.808490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961527+0000","last_change":"2023-10-18T10:37:57.955122+0000","last_active":"2023-10-18T10:37:58.961527+0000","last_peered":"2023-10-18T10:37:58.961527+0000","last_clean":"2023-10-18T10:37:58.961527+0000","last_became_active":"2023-10-18T10:37:57.954807+0000","last_became_peered":"2023-10-18T10:37:57.954807+0000","last_unstale":"2023-10-18T10:37:58.961527+0000","last_undegraded":"2023-10-18T10:37:58.961527+0000","last_fullsized":"2023-10-18T10:37:58.961527+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:29.465806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319557+0000","last_change":"2023-10-18T10:37:57.962347+0000","last_active":"2023-10-18T10:37:59.319557+0000","last_peered":"2023-10-18T10:37:59.319557+0000","last_clean":"2023-10-18T10:37:59.319557+0000","last_became_active":"2023-10-18T10:37:57.962056+0000","last_became_peered":"2023-10-18T10:37:57.962056+0000","last_unstale":"2023-10-18T10:37:59.319557+0000","last_undegraded":"2023-10-18T10:37:59.319557+0000","last_fullsized":"2023-10-18T10:37:59.319557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:26.065661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319191+0000","last_change":"2023-10-18T10:37:57.960830+0000","last_active":"2023-10-18T10:37:59.319191+0000","last_peered":"2023-10-18T10:37:59.319191+0000","last_clean":"2023-10-18T10:37:59.319191+0000","last_became_active":"2023-10-18T10:37:57.959457+0000","last_became_peered":"2023-10-18T10:37:57.959457+0000","last_unstale":"2023-10-18T10:37:59.319191+0000","last_undegraded":"2023-10-18T10:37:59.319191+0000","last_fullsized":"2023-10-18T10:37:59.319191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:24.198102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320306+0000","last_change":"2023-10-18T10:37:57.971563+0000","last_active":"2023-10-18T10:37:59.320306+0000","last_peered":"2023-10-18T10:37:59.320306+0000","last_clean":"2023-10-18T10:37:59.320306+0000","last_became_active":"2023-10-18T10:37:57.968579+0000","last_became_peered":"2023-10-18T10:37:57.968579+0000","last_unstale":"2023-10-18T10:37:59.320306+0000","last_undegraded":"2023-10-18T10:37:59.320306+0000","last_fullsized":"2023-10-18T10:37:59.320306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:24.819764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320543+0000","last_change":"2023-10-18T10:37:57.975017+0000","last_active":"2023-10-18T10:37:59.320543+0000","last_peered":"2023-10-18T10:37:59.320543+0000","last_clean":"2023-10-18T10:37:59.320543+0000","last_became_active":"2023-10-18T10:37:57.974211+0000","last_became_peered":"2023-10-18T10:37:57.974211+0000","last_unstale":"2023-10-18T10:37:59.320543+0000","last_undegraded":"2023-10-18T10:37:59.320543+0000","last_fullsized":"2023-10-18T10:37:59.320543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:23:59.673836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319413+0000","last_change":"2023-10-18T10:37:57.953933+0000","last_active":"2023-10-18T10:37:59.319413+0000","last_peered":"2023-10-18T10:37:59.319413+0000","last_clean":"2023-10-18T10:37:59.319413+0000","last_became_active":"2023-10-18T10:37:57.953671+0000","last_became_peered":"2023-10-18T10:37:57.953671+0000","last_unstale":"2023-10-18T10:37:59.319413+0000","last_undegraded":"2023-10-18T10:37:59.319413+0000","last_fullsized":"2023-10-18T10:37:59.319413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:15.822428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.955554+0000","last_change":"2023-10-18T10:37:57.971723+0000","last_active":"2023-10-18T10:37:58.955554+0000","last_peered":"2023-10-18T10:37:58.955554+0000","last_clean":"2023-10-18T10:37:58.955554+0000","last_became_active":"2023-10-18T10:37:57.971444+0000","last_became_peered":"2023-10-18T10:37:57.971444+0000","last_unstale":"2023-10-18T10:37:58.955554+0000","last_undegraded":"2023-10-18T10:37:58.955554+0000","last_fullsized":"2023-10-18T10:37:58.955554+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:18:48.291519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320289+0000","last_change":"2023-10-18T10:37:57.976929+0000","last_active":"2023-10-18T10:37:59.320289+0000","last_peered":"2023-10-18T10:37:59.320289+0000","last_clean":"2023-10-18T10:37:59.320289+0000","last_became_active":"2023-10-18T10:37:57.976421+0000","last_became_peered":"2023-10-18T10:37:57.976421+0000","last_unstale":"2023-10-18T10:37:59.320289+0000","last_undegraded":"2023-10-18T10:37:59.320289+0000","last_fullsized":"2023-10-18T10:37:59.320289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:21.654897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319151+0000","last_change":"2023-10-18T10:37:57.955417+0000","last_active":"2023-10-18T10:37:59.319151+0000","last_peered":"2023-10-18T10:37:59.319151+0000","last_clean":"2023-10-18T10:37:59.319151+0000","last_became_active":"2023-10-18T10:37:57.955013+0000","last_became_peered":"2023-10-18T10:37:57.955013+0000","last_unstale":"2023-10-18T10:37:59.319151+0000","last_undegraded":"2023-10-18T10:37:59.319151+0000","last_fullsized":"2023-10-18T10:37:59.319151+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:50:02.468827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320414+0000","last_change":"2023-10-18T10:37:57.958294+0000","last_active":"2023-10-18T10:37:59.320414+0000","last_peered":"2023-10-18T10:37:59.320414+0000","last_clean":"2023-10-18T10:37:59.320414+0000","last_became_active":"2023-10-18T10:37:57.958050+0000","last_became_peered":"2023-10-18T10:37:57.958050+0000","last_unstale":"2023-10-18T10:37:59.320414+0000","last_undegraded":"2023-10-18T10:37:59.320414+0000","last_fullsized":"2023-10-18T10:37:59.320414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:11.915134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319505+0000","last_change":"2023-10-18T10:37:57.964193+0000","last_active":"2023-10-18T10:37:59.319505+0000","last_peered":"2023-10-18T10:37:59.319505+0000","last_clean":"2023-10-18T10:37:59.319505+0000","last_became_active":"2023-10-18T10:37:57.963911+0000","last_became_peered":"2023-10-18T10:37:57.963911+0000","last_unstale":"2023-10-18T10:37:59.319505+0000","last_undegraded":"2023-10-18T10:37:59.319505+0000","last_fullsized":"2023-10-18T10:37:59.319505+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:24.198102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320022+0000","last_change":"2023-10-18T10:37:57.954607+0000","last_active":"2023-10-18T10:37:59.320022+0000","last_peered":"2023-10-18T10:37:59.320022+0000","last_clean":"2023-10-18T10:37:59.320022+0000","last_became_active":"2023-10-18T10:37:57.954299+0000","last_became_peered":"2023-10-18T10:37:57.954299+0000","last_unstale":"2023-10-18T10:37:59.320022+0000","last_undegraded":"2023-10-18T10:37:59.320022+0000","last_fullsized":"2023-10-18T10:37:59.320022+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:01.709827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319932+0000","last_change":"2023-10-18T10:37:57.966451+0000","last_active":"2023-10-18T10:37:59.319932+0000","last_peered":"2023-10-18T10:37:59.319932+0000","last_clean":"2023-10-18T10:37:59.319932+0000","last_became_active":"2023-10-18T10:37:57.965325+0000","last_became_peered":"2023-10-18T10:37:57.965325+0000","last_unstale":"2023-10-18T10:37:59.319932+0000","last_undegraded":"2023-10-18T10:37:59.319932+0000","last_fullsized":"2023-10-18T10:37:59.319932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:05:52.337294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320569+0000","last_change":"2023-10-18T10:37:57.967333+0000","last_active":"2023-10-18T10:37:59.320569+0000","last_peered":"2023-10-18T10:37:59.320569+0000","last_clean":"2023-10-18T10:37:59.320569+0000","last_became_active":"2023-10-18T10:37:57.967087+0000","last_became_peered":"2023-10-18T10:37:57.967087+0000","last_unstale":"2023-10-18T10:37:59.320569+0000","last_undegraded":"2023-10-18T10:37:59.320569+0000","last_fullsized":"2023-10-18T10:37:59.320569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:42:01.890239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319547+0000","last_change":"2023-10-18T10:37:57.965744+0000","last_active":"2023-10-18T10:37:59.319547+0000","last_peered":"2023-10-18T10:37:59.319547+0000","last_clean":"2023-10-18T10:37:59.319547+0000","last_became_active":"2023-10-18T10:37:57.964795+0000","last_became_peered":"2023-10-18T10:37:57.964795+0000","last_unstale":"2023-10-18T10:37:59.319547+0000","last_undegraded":"2023-10-18T10:37:59.319547+0000","last_fullsized":"2023-10-18T10:37:59.319547+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:42:01.890239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319882+0000","last_change":"2023-10-18T10:37:57.965614+0000","last_active":"2023-10-18T10:37:59.319882+0000","last_peered":"2023-10-18T10:37:59.319882+0000","last_clean":"2023-10-18T10:37:59.319882+0000","last_became_active":"2023-10-18T10:37:57.965337+0000","last_became_peered":"2023-10-18T10:37:57.965337+0000","last_unstale":"2023-10-18T10:37:59.319882+0000","last_undegraded":"2023-10-18T10:37:59.319882+0000","last_fullsized":"2023-10-18T10:37:59.319882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:36.204628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319672+0000","last_change":"2023-10-18T10:37:57.966843+0000","last_active":"2023-10-18T10:37:59.319672+0000","last_peered":"2023-10-18T10:37:59.319672+0000","last_clean":"2023-10-18T10:37:59.319672+0000","last_became_active":"2023-10-18T10:37:57.966583+0000","last_became_peered":"2023-10-18T10:37:57.966583+0000","last_unstale":"2023-10-18T10:37:59.319672+0000","last_undegraded":"2023-10-18T10:37:59.319672+0000","last_fullsized":"2023-10-18T10:37:59.319672+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:42.559543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319241+0000","last_change":"2023-10-18T10:37:57.969205+0000","last_active":"2023-10-18T10:37:59.319241+0000","last_peered":"2023-10-18T10:37:59.319241+0000","last_clean":"2023-10-18T10:37:59.319241+0000","last_became_active":"2023-10-18T10:37:57.968910+0000","last_became_peered":"2023-10-18T10:37:57.968910+0000","last_unstale":"2023-10-18T10:37:59.319241+0000","last_undegraded":"2023-10-18T10:37:59.319241+0000","last_fullsized":"2023-10-18T10:37:59.319241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:52.175815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.955515+0000","last_change":"2023-10-18T10:37:57.960187+0000","last_active":"2023-10-18T10:37:58.955515+0000","last_peered":"2023-10-18T10:37:58.955515+0000","last_clean":"2023-10-18T10:37:58.955515+0000","last_became_active":"2023-10-18T10:37:57.959903+0000","last_became_peered":"2023-10-18T10:37:57.959903+0000","last_unstale":"2023-10-18T10:37:58.955515+0000","last_undegraded":"2023-10-18T10:37:58.955515+0000","last_fullsized":"2023-10-18T10:37:58.955515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:00:28.477842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320146+0000","last_change":"2023-10-18T10:37:57.956512+0000","last_active":"2023-10-18T10:37:59.320146+0000","last_peered":"2023-10-18T10:37:59.320146+0000","last_clean":"2023-10-18T10:37:59.320146+0000","last_became_active":"2023-10-18T10:37:57.956255+0000","last_became_peered":"2023-10-18T10:37:57.956255+0000","last_unstale":"2023-10-18T10:37:59.320146+0000","last_undegraded":"2023-10-18T10:37:59.320146+0000","last_fullsized":"2023-10-18T10:37:59.320146+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:30:31.277948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320164+0000","last_change":"2023-10-18T10:37:57.971446+0000","last_active":"2023-10-18T10:37:59.320164+0000","last_peered":"2023-10-18T10:37:59.320164+0000","last_clean":"2023-10-18T10:37:59.320164+0000","last_became_active":"2023-10-18T10:37:57.970902+0000","last_became_peered":"2023-10-18T10:37:57.970902+0000","last_unstale":"2023-10-18T10:37:59.320164+0000","last_undegraded":"2023-10-18T10:37:59.320164+0000","last_fullsized":"2023-10-18T10:37:59.320164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:21:19.628260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319576+0000","last_change":"2023-10-18T10:37:57.974213+0000","last_active":"2023-10-18T10:37:59.319576+0000","last_peered":"2023-10-18T10:37:59.319576+0000","last_clean":"2023-10-18T10:37:59.319576+0000","last_became_active":"2023-10-18T10:37:57.972067+0000","last_became_peered":"2023-10-18T10:37:57.972067+0000","last_unstale":"2023-10-18T10:37:59.319576+0000","last_undegraded":"2023-10-18T10:37:59.319576+0000","last_fullsized":"2023-10-18T10:37:59.319576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:30:31.277948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319761+0000","last_change":"2023-10-18T10:37:57.967112+0000","last_active":"2023-10-18T10:37:59.319761+0000","last_peered":"2023-10-18T10:37:59.319761+0000","last_clean":"2023-10-18T10:37:59.319761+0000","last_became_active":"2023-10-18T10:37:57.966819+0000","last_became_peered":"2023-10-18T10:37:57.966819+0000","last_unstale":"2023-10-18T10:37:59.319761+0000","last_undegraded":"2023-10-18T10:37:59.319761+0000","last_fullsized":"2023-10-18T10:37:59.319761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:47.370984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319957+0000","last_change":"2023-10-18T10:37:57.965077+0000","last_active":"2023-10-18T10:37:59.319957+0000","last_peered":"2023-10-18T10:37:59.319957+0000","last_clean":"2023-10-18T10:37:59.319957+0000","last_became_active":"2023-10-18T10:37:57.964785+0000","last_became_peered":"2023-10-18T10:37:57.964785+0000","last_unstale":"2023-10-18T10:37:59.319957+0000","last_undegraded":"2023-10-18T10:37:59.319957+0000","last_fullsized":"2023-10-18T10:37:59.319957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:59.117648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319799+0000","last_change":"2023-10-18T10:37:57.963167+0000","last_active":"2023-10-18T10:37:59.319799+0000","last_peered":"2023-10-18T10:37:59.319799+0000","last_clean":"2023-10-18T10:37:59.319799+0000","last_became_active":"2023-10-18T10:37:57.962849+0000","last_became_peered":"2023-10-18T10:37:57.962849+0000","last_unstale":"2023-10-18T10:37:59.319799+0000","last_undegraded":"2023-10-18T10:37:59.319799+0000","last_fullsized":"2023-10-18T10:37:59.319799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:13:05.525565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319620+0000","last_change":"2023-10-18T10:37:57.971806+0000","last_active":"2023-10-18T10:37:59.319620+0000","last_peered":"2023-10-18T10:37:59.319620+0000","last_clean":"2023-10-18T10:37:59.319620+0000","last_became_active":"2023-10-18T10:37:57.971530+0000","last_became_peered":"2023-10-18T10:37:57.971530+0000","last_unstale":"2023-10-18T10:37:59.319620+0000","last_undegraded":"2023-10-18T10:37:59.319620+0000","last_fullsized":"2023-10-18T10:37:59.319620+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:14.428640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319218+0000","last_change":"2023-10-18T10:37:57.956429+0000","last_active":"2023-10-18T10:37:59.319218+0000","last_peered":"2023-10-18T10:37:59.319218+0000","last_clean":"2023-10-18T10:37:59.319218+0000","last_became_active":"2023-10-18T10:37:57.956171+0000","last_became_peered":"2023-10-18T10:37:57.956171+0000","last_unstale":"2023-10-18T10:37:59.319218+0000","last_undegraded":"2023-10-18T10:37:59.319218+0000","last_fullsized":"2023-10-18T10:37:59.319218+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:14.428640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319350+0000","last_change":"2023-10-18T10:37:57.971096+0000","last_active":"2023-10-18T10:37:59.319350+0000","last_peered":"2023-10-18T10:37:59.319350+0000","last_clean":"2023-10-18T10:37:59.319350+0000","last_became_active":"2023-10-18T10:37:57.970788+0000","last_became_peered":"2023-10-18T10:37:57.970788+0000","last_unstale":"2023-10-18T10:37:59.319350+0000","last_undegraded":"2023-10-18T10:37:59.319350+0000","last_fullsized":"2023-10-18T10:37:59.319350+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:35.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319324+0000","last_change":"2023-10-18T10:37:57.966206+0000","last_active":"2023-10-18T10:37:59.319324+0000","last_peered":"2023-10-18T10:37:59.319324+0000","last_clean":"2023-10-18T10:37:59.319324+0000","last_became_active":"2023-10-18T10:37:57.965924+0000","last_became_peered":"2023-10-18T10:37:57.965924+0000","last_unstale":"2023-10-18T10:37:59.319324+0000","last_undegraded":"2023-10-18T10:37:59.319324+0000","last_fullsized":"2023-10-18T10:37:59.319324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:35.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"64'4","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319684+0000","last_change":"2023-10-18T10:37:56.945466+0000","last_active":"2023-10-18T10:37:59.319684+0000","last_peered":"2023-10-18T10:37:59.319684+0000","last_clean":"2023-10-18T10:37:59.319684+0000","last_became_active":"2023-10-18T10:37:56.945183+0000","last_became_peered":"2023-10-18T10:37:56.945183+0000","last_unstale":"2023-10-18T10:37:59.319684+0000","last_undegraded":"2023-10-18T10:37:59.319684+0000","last_fullsized":"2023-10-18T10:37:59.319684+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:33:55.760075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319734+0000","last_change":"2023-10-18T10:37:57.962326+0000","last_active":"2023-10-18T10:37:59.319734+0000","last_peered":"2023-10-18T10:37:59.319734+0000","last_clean":"2023-10-18T10:37:59.319734+0000","last_became_active":"2023-10-18T10:37:57.962021+0000","last_became_peered":"2023-10-18T10:37:57.962021+0000","last_unstale":"2023-10-18T10:37:59.319734+0000","last_undegraded":"2023-10-18T10:37:59.319734+0000","last_fullsized":"2023-10-18T10:37:59.319734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:29.010713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319490+0000","last_change":"2023-10-18T10:37:56.952243+0000","last_active":"2023-10-18T10:37:59.319490+0000","last_peered":"2023-10-18T10:37:59.319490+0000","last_clean":"2023-10-18T10:37:59.319490+0000","last_became_active":"2023-10-18T10:37:56.951937+0000","last_became_peered":"2023-10-18T10:37:56.951937+0000","last_unstale":"2023-10-18T10:37:59.319490+0000","last_undegraded":"2023-10-18T10:37:59.319490+0000","last_fullsized":"2023-10-18T10:37:59.319490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:02.773566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320032+0000","last_change":"2023-10-18T10:37:57.967696+0000","last_active":"2023-10-18T10:37:59.320032+0000","last_peered":"2023-10-18T10:37:59.320032+0000","last_clean":"2023-10-18T10:37:59.320032+0000","last_became_active":"2023-10-18T10:37:57.967387+0000","last_became_peered":"2023-10-18T10:37:57.967387+0000","last_unstale":"2023-10-18T10:37:59.320032+0000","last_undegraded":"2023-10-18T10:37:59.320032+0000","last_fullsized":"2023-10-18T10:37:59.320032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:29.465806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319849+0000","last_change":"2023-10-18T10:37:56.952879+0000","last_active":"2023-10-18T10:37:59.319849+0000","last_peered":"2023-10-18T10:37:59.319849+0000","last_clean":"2023-10-18T10:37:59.319849+0000","last_became_active":"2023-10-18T10:37:56.952340+0000","last_became_peered":"2023-10-18T10:37:56.952340+0000","last_unstale":"2023-10-18T10:37:59.319849+0000","last_undegraded":"2023-10-18T10:37:59.319849+0000","last_fullsized":"2023-10-18T10:37:59.319849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:35.197029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320105+0000","last_change":"2023-10-18T10:37:57.974667+0000","last_active":"2023-10-18T10:37:59.320105+0000","last_peered":"2023-10-18T10:37:59.320105+0000","last_clean":"2023-10-18T10:37:59.320105+0000","last_became_active":"2023-10-18T10:37:57.974350+0000","last_became_peered":"2023-10-18T10:37:57.974350+0000","last_unstale":"2023-10-18T10:37:59.320105+0000","last_undegraded":"2023-10-18T10:37:59.320105+0000","last_fullsized":"2023-10-18T10:37:59.320105+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:24:59.217657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320075+0000","last_change":"2023-10-18T10:37:56.951625+0000","last_active":"2023-10-18T10:37:59.320075+0000","last_peered":"2023-10-18T10:37:59.320075+0000","last_clean":"2023-10-18T10:37:59.320075+0000","last_became_active":"2023-10-18T10:37:56.951375+0000","last_became_peered":"2023-10-18T10:37:56.951375+0000","last_unstale":"2023-10-18T10:37:59.320075+0000","last_undegraded":"2023-10-18T10:37:59.320075+0000","last_fullsized":"2023-10-18T10:37:59.320075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:15.400131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961537+0000","last_change":"2023-10-18T10:37:57.957530+0000","last_active":"2023-10-18T10:37:58.961537+0000","last_peered":"2023-10-18T10:37:58.961537+0000","last_clean":"2023-10-18T10:37:58.961537+0000","last_became_active":"2023-10-18T10:37:57.957247+0000","last_became_peered":"2023-10-18T10:37:57.957247+0000","last_unstale":"2023-10-18T10:37:58.961537+0000","last_undegraded":"2023-10-18T10:37:58.961537+0000","last_fullsized":"2023-10-18T10:37:58.961537+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:51:51.427813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319890+0000","last_change":"2023-10-18T10:37:56.955873+0000","last_active":"2023-10-18T10:37:59.319890+0000","last_peered":"2023-10-18T10:37:59.319890+0000","last_clean":"2023-10-18T10:37:59.319890+0000","last_became_active":"2023-10-18T10:37:56.955625+0000","last_became_peered":"2023-10-18T10:37:56.955625+0000","last_unstale":"2023-10-18T10:37:59.319890+0000","last_undegraded":"2023-10-18T10:37:59.319890+0000","last_fullsized":"2023-10-18T10:37:59.319890+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:55.749193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320015+0000","last_change":"2023-10-18T10:37:57.972619+0000","last_active":"2023-10-18T10:37:59.320015+0000","last_peered":"2023-10-18T10:37:59.320015+0000","last_clean":"2023-10-18T10:37:59.320015+0000","last_became_active":"2023-10-18T10:37:57.972172+0000","last_became_peered":"2023-10-18T10:37:57.972172+0000","last_unstale":"2023-10-18T10:37:59.320015+0000","last_undegraded":"2023-10-18T10:37:59.320015+0000","last_fullsized":"2023-10-18T10:37:59.320015+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:29.010713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961305+0000","last_change":"2023-10-18T10:37:56.951459+0000","last_active":"2023-10-18T10:37:58.961305+0000","last_peered":"2023-10-18T10:37:58.961305+0000","last_clean":"2023-10-18T10:37:58.961305+0000","last_became_active":"2023-10-18T10:37:56.951186+0000","last_became_peered":"2023-10-18T10:37:56.951186+0000","last_unstale":"2023-10-18T10:37:58.961305+0000","last_undegraded":"2023-10-18T10:37:58.961305+0000","last_fullsized":"2023-10-18T10:37:58.961305+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:34.321126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319796+0000","last_change":"2023-10-18T10:37:57.974277+0000","last_active":"2023-10-18T10:37:59.319796+0000","last_peered":"2023-10-18T10:37:59.319796+0000","last_clean":"2023-10-18T10:37:59.319796+0000","last_became_active":"2023-10-18T10:37:57.972042+0000","last_became_peered":"2023-10-18T10:37:57.972042+0000","last_unstale":"2023-10-18T10:37:59.319796+0000","last_undegraded":"2023-10-18T10:37:59.319796+0000","last_fullsized":"2023-10-18T10:37:59.319796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:11.915134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319483+0000","last_change":"2023-10-18T10:37:56.949089+0000","last_active":"2023-10-18T10:37:59.319483+0000","last_peered":"2023-10-18T10:37:59.319483+0000","last_clean":"2023-10-18T10:37:59.319483+0000","last_became_active":"2023-10-18T10:37:56.948826+0000","last_became_peered":"2023-10-18T10:37:56.948826+0000","last_unstale":"2023-10-18T10:37:59.319483+0000","last_undegraded":"2023-10-18T10:37:59.319483+0000","last_fullsized":"2023-10-18T10:37:59.319483+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:10.729518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320575+0000","last_change":"2023-10-18T10:37:57.964429+0000","last_active":"2023-10-18T10:37:59.320575+0000","last_peered":"2023-10-18T10:37:59.320575+0000","last_clean":"2023-10-18T10:37:59.320575+0000","last_became_active":"2023-10-18T10:37:57.964171+0000","last_became_peered":"2023-10-18T10:37:57.964171+0000","last_unstale":"2023-10-18T10:37:59.320575+0000","last_undegraded":"2023-10-18T10:37:59.320575+0000","last_fullsized":"2023-10-18T10:37:59.320575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:10.675568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320223+0000","last_change":"2023-10-18T10:37:56.955114+0000","last_active":"2023-10-18T10:37:59.320223+0000","last_peered":"2023-10-18T10:37:59.320223+0000","last_clean":"2023-10-18T10:37:59.320223+0000","last_became_active":"2023-10-18T10:37:56.954823+0000","last_became_peered":"2023-10-18T10:37:56.954823+0000","last_unstale":"2023-10-18T10:37:59.320223+0000","last_undegraded":"2023-10-18T10:37:59.320223+0000","last_fullsized":"2023-10-18T10:37:59.320223+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:36.387677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961199+0000","last_change":"2023-10-18T10:37:57.970372+0000","last_active":"2023-10-18T10:37:58.961199+0000","last_peered":"2023-10-18T10:37:58.961199+0000","last_clean":"2023-10-18T10:37:58.961199+0000","last_became_active":"2023-10-18T10:37:57.970088+0000","last_became_peered":"2023-10-18T10:37:57.970088+0000","last_unstale":"2023-10-18T10:37:58.961199+0000","last_undegraded":"2023-10-18T10:37:58.961199+0000","last_fullsized":"2023-10-18T10:37:58.961199+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:59.117648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320614+0000","last_change":"2023-10-18T10:37:56.949701+0000","last_active":"2023-10-18T10:37:59.320614+0000","last_peered":"2023-10-18T10:37:59.320614+0000","last_clean":"2023-10-18T10:37:59.320614+0000","last_became_active":"2023-10-18T10:37:56.949443+0000","last_became_peered":"2023-10-18T10:37:56.949443+0000","last_unstale":"2023-10-18T10:37:59.320614+0000","last_undegraded":"2023-10-18T10:37:59.320614+0000","last_fullsized":"2023-10-18T10:37:59.320614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:01.287718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319600+0000","last_change":"2023-10-18T10:37:57.956703+0000","last_active":"2023-10-18T10:37:59.319600+0000","last_peered":"2023-10-18T10:37:59.319600+0000","last_clean":"2023-10-18T10:37:59.319600+0000","last_became_active":"2023-10-18T10:37:57.956395+0000","last_became_peered":"2023-10-18T10:37:57.956395+0000","last_unstale":"2023-10-18T10:37:59.319600+0000","last_undegraded":"2023-10-18T10:37:59.319600+0000","last_fullsized":"2023-10-18T10:37:59.319600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:14.428640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961447+0000","last_change":"2023-10-18T10:37:56.952611+0000","last_active":"2023-10-18T10:37:58.961447+0000","last_peered":"2023-10-18T10:37:58.961447+0000","last_clean":"2023-10-18T10:37:58.961447+0000","last_became_active":"2023-10-18T10:37:56.952343+0000","last_became_peered":"2023-10-18T10:37:56.952343+0000","last_unstale":"2023-10-18T10:37:58.961447+0000","last_undegraded":"2023-10-18T10:37:58.961447+0000","last_fullsized":"2023-10-18T10:37:58.961447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:41.551944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320431+0000","last_change":"2023-10-18T10:37:57.963394+0000","last_active":"2023-10-18T10:37:59.320431+0000","last_peered":"2023-10-18T10:37:59.320431+0000","last_clean":"2023-10-18T10:37:59.320431+0000","last_became_active":"2023-10-18T10:37:57.963123+0000","last_became_peered":"2023-10-18T10:37:57.963123+0000","last_unstale":"2023-10-18T10:37:59.320431+0000","last_undegraded":"2023-10-18T10:37:59.320431+0000","last_fullsized":"2023-10-18T10:37:59.320431+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:24:59.217657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"64'3","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320197+0000","last_change":"2023-10-18T10:37:56.954070+0000","last_active":"2023-10-18T10:37:59.320197+0000","last_peered":"2023-10-18T10:37:59.320197+0000","last_clean":"2023-10-18T10:37:59.320197+0000","last_became_active":"2023-10-18T10:37:56.953756+0000","last_became_peered":"2023-10-18T10:37:56.953756+0000","last_unstale":"2023-10-18T10:37:59.320197+0000","last_undegraded":"2023-10-18T10:37:59.320197+0000","last_fullsized":"2023-10-18T10:37:59.320197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:15.400131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961533+0000","last_change":"2023-10-18T10:37:57.953033+0000","last_active":"2023-10-18T10:37:58.961533+0000","last_peered":"2023-10-18T10:37:58.961533+0000","last_clean":"2023-10-18T10:37:58.961533+0000","last_became_active":"2023-10-18T10:37:57.952745+0000","last_became_peered":"2023-10-18T10:37:57.952745+0000","last_unstale":"2023-10-18T10:37:58.961533+0000","last_undegraded":"2023-10-18T10:37:58.961533+0000","last_fullsized":"2023-10-18T10:37:58.961533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:33:07.812082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"64'6","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320401+0000","last_change":"2023-10-18T10:37:56.954805+0000","last_active":"2023-10-18T10:37:59.320401+0000","last_peered":"2023-10-18T10:37:59.320401+0000","last_clean":"2023-10-18T10:37:59.320401+0000","last_became_active":"2023-10-18T10:37:56.954488+0000","last_became_peered":"2023-10-18T10:37:56.954488+0000","last_unstale":"2023-10-18T10:37:59.320401+0000","last_undegraded":"2023-10-18T10:37:59.320401+0000","last_fullsized":"2023-10-18T10:37:59.320401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:13.421041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319271+0000","last_change":"2023-10-18T10:37:57.961632+0000","last_active":"2023-10-18T10:37:59.319271+0000","last_peered":"2023-10-18T10:37:59.319271+0000","last_clean":"2023-10-18T10:37:59.319271+0000","last_became_active":"2023-10-18T10:37:57.960353+0000","last_became_peered":"2023-10-18T10:37:57.960353+0000","last_unstale":"2023-10-18T10:37:59.319271+0000","last_undegraded":"2023-10-18T10:37:59.319271+0000","last_fullsized":"2023-10-18T10:37:59.319271+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:00.458354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319982+0000","last_change":"2023-10-18T10:37:56.950895+0000","last_active":"2023-10-18T10:37:59.319982+0000","last_peered":"2023-10-18T10:37:59.319982+0000","last_clean":"2023-10-18T10:37:59.319982+0000","last_became_active":"2023-10-18T10:37:56.950623+0000","last_became_peered":"2023-10-18T10:37:56.950623+0000","last_unstale":"2023-10-18T10:37:59.319982+0000","last_undegraded":"2023-10-18T10:37:59.319982+0000","last_fullsized":"2023-10-18T10:37:59.319982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:18.359944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319668+0000","last_change":"2023-10-18T10:37:57.966150+0000","last_active":"2023-10-18T10:37:59.319668+0000","last_peered":"2023-10-18T10:37:59.319668+0000","last_clean":"2023-10-18T10:37:59.319668+0000","last_became_active":"2023-10-18T10:37:57.965481+0000","last_became_peered":"2023-10-18T10:37:57.965481+0000","last_unstale":"2023-10-18T10:37:59.319668+0000","last_undegraded":"2023-10-18T10:37:59.319668+0000","last_fullsized":"2023-10-18T10:37:59.319668+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:10.675568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320099+0000","last_change":"2023-10-18T10:37:56.952659+0000","last_active":"2023-10-18T10:37:59.320099+0000","last_peered":"2023-10-18T10:37:59.320099+0000","last_clean":"2023-10-18T10:37:59.320099+0000","last_became_active":"2023-10-18T10:37:56.952378+0000","last_became_peered":"2023-10-18T10:37:56.952378+0000","last_unstale":"2023-10-18T10:37:59.320099+0000","last_undegraded":"2023-10-18T10:37:59.320099+0000","last_fullsized":"2023-10-18T10:37:59.320099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:43:26.678464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319684+0000","last_change":"2023-10-18T10:37:57.968546+0000","last_active":"2023-10-18T10:37:59.319684+0000","last_peered":"2023-10-18T10:37:59.319684+0000","last_clean":"2023-10-18T10:37:59.319684+0000","last_became_active":"2023-10-18T10:37:57.968276+0000","last_became_peered":"2023-10-18T10:37:57.968276+0000","last_unstale":"2023-10-18T10:37:59.319684+0000","last_undegraded":"2023-10-18T10:37:59.319684+0000","last_fullsized":"2023-10-18T10:37:59.319684+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:21:19.628260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319681+0000","last_change":"2023-10-18T10:37:56.950945+0000","last_active":"2023-10-18T10:37:59.319681+0000","last_peered":"2023-10-18T10:37:59.319681+0000","last_clean":"2023-10-18T10:37:59.319681+0000","last_became_active":"2023-10-18T10:37:56.950664+0000","last_became_peered":"2023-10-18T10:37:56.950664+0000","last_unstale":"2023-10-18T10:37:59.319681+0000","last_undegraded":"2023-10-18T10:37:59.319681+0000","last_fullsized":"2023-10-18T10:37:59.319681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:18:10.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319941+0000","last_change":"2023-10-18T10:37:57.963804+0000","last_active":"2023-10-18T10:37:59.319941+0000","last_peered":"2023-10-18T10:37:59.319941+0000","last_clean":"2023-10-18T10:37:59.319941+0000","last_became_active":"2023-10-18T10:37:57.963503+0000","last_became_peered":"2023-10-18T10:37:57.963503+0000","last_unstale":"2023-10-18T10:37:59.319941+0000","last_undegraded":"2023-10-18T10:37:59.319941+0000","last_fullsized":"2023-10-18T10:37:59.319941+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:52.175815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"64'8","reported_seq":20,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320344+0000","last_change":"2023-10-18T10:37:56.950726+0000","last_active":"2023-10-18T10:37:59.320344+0000","last_peered":"2023-10-18T10:37:59.320344+0000","last_clean":"2023-10-18T10:37:59.320344+0000","last_became_active":"2023-10-18T10:37:56.949932+0000","last_became_peered":"2023-10-18T10:37:56.949932+0000","last_unstale":"2023-10-18T10:37:59.320344+0000","last_undegraded":"2023-10-18T10:37:59.320344+0000","last_fullsized":"2023-10-18T10:37:59.320344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:42:00.882640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320111+0000","last_change":"2023-10-18T10:37:57.969403+0000","last_active":"2023-10-18T10:37:59.320111+0000","last_peered":"2023-10-18T10:37:59.320111+0000","last_clean":"2023-10-18T10:37:59.320111+0000","last_became_active":"2023-10-18T10:37:57.968306+0000","last_became_peered":"2023-10-18T10:37:57.968306+0000","last_unstale":"2023-10-18T10:37:59.320111+0000","last_undegraded":"2023-10-18T10:37:59.320111+0000","last_fullsized":"2023-10-18T10:37:59.320111+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:19.454749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320051+0000","last_change":"2023-10-18T10:37:56.956875+0000","last_active":"2023-10-18T10:37:59.320051+0000","last_peered":"2023-10-18T10:37:59.320051+0000","last_clean":"2023-10-18T10:37:59.320051+0000","last_became_active":"2023-10-18T10:37:56.956636+0000","last_became_peered":"2023-10-18T10:37:56.956636+0000","last_unstale":"2023-10-18T10:37:59.320051+0000","last_undegraded":"2023-10-18T10:37:59.320051+0000","last_fullsized":"2023-10-18T10:37:59.320051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:47:59.450755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319205+0000","last_change":"2023-10-18T10:37:57.956559+0000","last_active":"2023-10-18T10:37:59.319205+0000","last_peered":"2023-10-18T10:37:59.319205+0000","last_clean":"2023-10-18T10:37:59.319205+0000","last_became_active":"2023-10-18T10:37:57.956323+0000","last_became_peered":"2023-10-18T10:37:57.956323+0000","last_unstale":"2023-10-18T10:37:59.319205+0000","last_undegraded":"2023-10-18T10:37:59.319205+0000","last_fullsized":"2023-10-18T10:37:59.319205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:43.446784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.955638+0000","last_change":"2023-10-18T10:37:56.949984+0000","last_active":"2023-10-18T10:37:58.955638+0000","last_peered":"2023-10-18T10:37:58.955638+0000","last_clean":"2023-10-18T10:37:58.955638+0000","last_became_active":"2023-10-18T10:37:56.949745+0000","last_became_peered":"2023-10-18T10:37:56.949745+0000","last_unstale":"2023-10-18T10:37:58.955638+0000","last_undegraded":"2023-10-18T10:37:58.955638+0000","last_fullsized":"2023-10-18T10:37:58.955638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:06.460580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961181+0000","last_change":"2023-10-18T10:37:57.955364+0000","last_active":"2023-10-18T10:37:58.961181+0000","last_peered":"2023-10-18T10:37:58.961181+0000","last_clean":"2023-10-18T10:37:58.961181+0000","last_became_active":"2023-10-18T10:37:57.955091+0000","last_became_peered":"2023-10-18T10:37:57.955091+0000","last_unstale":"2023-10-18T10:37:58.961181+0000","last_undegraded":"2023-10-18T10:37:58.961181+0000","last_fullsized":"2023-10-18T10:37:58.961181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:36.204628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319756+0000","last_change":"2023-10-18T10:37:56.953753+0000","last_active":"2023-10-18T10:37:59.319756+0000","last_peered":"2023-10-18T10:37:59.319756+0000","last_clean":"2023-10-18T10:37:59.319756+0000","last_became_active":"2023-10-18T10:37:56.953482+0000","last_became_peered":"2023-10-18T10:37:56.953482+0000","last_unstale":"2023-10-18T10:37:59.319756+0000","last_undegraded":"2023-10-18T10:37:59.319756+0000","last_fullsized":"2023-10-18T10:37:59.319756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:02.773566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319280+0000","last_change":"2023-10-18T10:37:57.955254+0000","last_active":"2023-10-18T10:37:59.319280+0000","last_peered":"2023-10-18T10:37:59.319280+0000","last_clean":"2023-10-18T10:37:59.319280+0000","last_became_active":"2023-10-18T10:37:57.954680+0000","last_became_peered":"2023-10-18T10:37:57.954680+0000","last_unstale":"2023-10-18T10:37:59.319280+0000","last_undegraded":"2023-10-18T10:37:59.319280+0000","last_fullsized":"2023-10-18T10:37:59.319280+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:10:56.717467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319972+0000","last_change":"2023-10-18T10:37:56.954521+0000","last_active":"2023-10-18T10:37:59.319972+0000","last_peered":"2023-10-18T10:37:59.319972+0000","last_clean":"2023-10-18T10:37:59.319972+0000","last_became_active":"2023-10-18T10:37:56.954246+0000","last_became_peered":"2023-10-18T10:37:56.954246+0000","last_unstale":"2023-10-18T10:37:59.319972+0000","last_undegraded":"2023-10-18T10:37:59.319972+0000","last_fullsized":"2023-10-18T10:37:59.319972+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:09:00.277886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319902+0000","last_change":"2023-10-18T10:37:57.956951+0000","last_active":"2023-10-18T10:37:59.319902+0000","last_peered":"2023-10-18T10:37:59.319902+0000","last_clean":"2023-10-18T10:37:59.319902+0000","last_became_active":"2023-10-18T10:37:57.956470+0000","last_became_peered":"2023-10-18T10:37:57.956470+0000","last_unstale":"2023-10-18T10:37:59.319902+0000","last_undegraded":"2023-10-18T10:37:59.319902+0000","last_fullsized":"2023-10-18T10:37:59.319902+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:44.808490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319366+0000","last_change":"2023-10-18T10:37:56.951564+0000","last_active":"2023-10-18T10:37:59.319366+0000","last_peered":"2023-10-18T10:37:59.319366+0000","last_clean":"2023-10-18T10:37:59.319366+0000","last_became_active":"2023-10-18T10:37:56.951290+0000","last_became_peered":"2023-10-18T10:37:56.951290+0000","last_unstale":"2023-10-18T10:37:59.319366+0000","last_undegraded":"2023-10-18T10:37:59.319366+0000","last_fullsized":"2023-10-18T10:37:59.319366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:58.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319670+0000","last_change":"2023-10-18T10:37:57.965698+0000","last_active":"2023-10-18T10:37:59.319670+0000","last_peered":"2023-10-18T10:37:59.319670+0000","last_clean":"2023-10-18T10:37:59.319670+0000","last_became_active":"2023-10-18T10:37:57.965414+0000","last_became_peered":"2023-10-18T10:37:57.965414+0000","last_unstale":"2023-10-18T10:37:59.319670+0000","last_undegraded":"2023-10-18T10:37:59.319670+0000","last_fullsized":"2023-10-18T10:37:59.319670+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:19.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319320+0000","last_change":"2023-10-18T10:37:56.953685+0000","last_active":"2023-10-18T10:37:59.319320+0000","last_peered":"2023-10-18T10:37:59.319320+0000","last_clean":"2023-10-18T10:37:59.319320+0000","last_became_active":"2023-10-18T10:37:56.953417+0000","last_became_peered":"2023-10-18T10:37:56.953417+0000","last_unstale":"2023-10-18T10:37:59.319320+0000","last_undegraded":"2023-10-18T10:37:59.319320+0000","last_fullsized":"2023-10-18T10:37:59.319320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:34.220180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319487+0000","last_change":"2023-10-18T10:37:57.962943+0000","last_active":"2023-10-18T10:37:59.319487+0000","last_peered":"2023-10-18T10:37:59.319487+0000","last_clean":"2023-10-18T10:37:59.319487+0000","last_became_active":"2023-10-18T10:37:57.962677+0000","last_became_peered":"2023-10-18T10:37:57.962677+0000","last_unstale":"2023-10-18T10:37:59.319487+0000","last_undegraded":"2023-10-18T10:37:59.319487+0000","last_fullsized":"2023-10-18T10:37:59.319487+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:49:09.644289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319275+0000","last_change":"2023-10-18T10:37:56.950058+0000","last_active":"2023-10-18T10:37:59.319275+0000","last_peered":"2023-10-18T10:37:59.319275+0000","last_clean":"2023-10-18T10:37:59.319275+0000","last_became_active":"2023-10-18T10:37:56.949797+0000","last_became_peered":"2023-10-18T10:37:56.949797+0000","last_unstale":"2023-10-18T10:37:59.319275+0000","last_undegraded":"2023-10-18T10:37:59.319275+0000","last_fullsized":"2023-10-18T10:37:59.319275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:15.400131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319950+0000","last_change":"2023-10-18T10:37:57.971233+0000","last_active":"2023-10-18T10:37:59.319950+0000","last_peered":"2023-10-18T10:37:59.319950+0000","last_clean":"2023-10-18T10:37:59.319950+0000","last_became_active":"2023-10-18T10:37:57.970922+0000","last_became_peered":"2023-10-18T10:37:57.970922+0000","last_unstale":"2023-10-18T10:37:59.319950+0000","last_undegraded":"2023-10-18T10:37:59.319950+0000","last_fullsized":"2023-10-18T10:37:59.319950+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:09:10.864067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.955384+0000","last_change":"2023-10-18T10:37:56.949441+0000","last_active":"2023-10-18T10:37:58.955384+0000","last_peered":"2023-10-18T10:37:58.955384+0000","last_clean":"2023-10-18T10:37:58.955384+0000","last_became_active":"2023-10-18T10:37:56.949178+0000","last_became_peered":"2023-10-18T10:37:56.949178+0000","last_unstale":"2023-10-18T10:37:58.955384+0000","last_undegraded":"2023-10-18T10:37:58.955384+0000","last_fullsized":"2023-10-18T10:37:58.955384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:47.581833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319441+0000","last_change":"2023-10-18T10:37:57.971742+0000","last_active":"2023-10-18T10:37:59.319441+0000","last_peered":"2023-10-18T10:37:59.319441+0000","last_clean":"2023-10-18T10:37:59.319441+0000","last_became_active":"2023-10-18T10:37:57.971478+0000","last_became_peered":"2023-10-18T10:37:57.971478+0000","last_unstale":"2023-10-18T10:37:59.319441+0000","last_undegraded":"2023-10-18T10:37:59.319441+0000","last_fullsized":"2023-10-18T10:37:59.319441+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:24.198102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319796+0000","last_change":"2023-10-18T10:37:56.953279+0000","last_active":"2023-10-18T10:37:59.319796+0000","last_peered":"2023-10-18T10:37:59.319796+0000","last_clean":"2023-10-18T10:37:59.319796+0000","last_became_active":"2023-10-18T10:37:56.953047+0000","last_became_peered":"2023-10-18T10:37:56.953047+0000","last_unstale":"2023-10-18T10:37:59.319796+0000","last_undegraded":"2023-10-18T10:37:59.319796+0000","last_fullsized":"2023-10-18T10:37:59.319796+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:30.885550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319683+0000","last_change":"2023-10-18T10:37:57.953909+0000","last_active":"2023-10-18T10:37:59.319683+0000","last_peered":"2023-10-18T10:37:59.319683+0000","last_clean":"2023-10-18T10:37:59.319683+0000","last_became_active":"2023-10-18T10:37:57.953638+0000","last_became_peered":"2023-10-18T10:37:57.953638+0000","last_unstale":"2023-10-18T10:37:59.319683+0000","last_undegraded":"2023-10-18T10:37:59.319683+0000","last_fullsized":"2023-10-18T10:37:59.319683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:23:10.167209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320289+0000","last_change":"2023-10-18T10:37:56.948678+0000","last_active":"2023-10-18T10:37:59.320289+0000","last_peered":"2023-10-18T10:37:59.320289+0000","last_clean":"2023-10-18T10:37:59.320289+0000","last_became_active":"2023-10-18T10:37:56.948004+0000","last_became_peered":"2023-10-18T10:37:56.948004+0000","last_unstale":"2023-10-18T10:37:59.320289+0000","last_undegraded":"2023-10-18T10:37:59.320289+0000","last_fullsized":"2023-10-18T10:37:59.320289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:17.386951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320220+0000","last_change":"2023-10-18T10:37:57.973958+0000","last_active":"2023-10-18T10:37:59.320220+0000","last_peered":"2023-10-18T10:37:59.320220+0000","last_clean":"2023-10-18T10:37:59.320220+0000","last_became_active":"2023-10-18T10:37:57.968934+0000","last_became_peered":"2023-10-18T10:37:57.968934+0000","last_unstale":"2023-10-18T10:37:59.320220+0000","last_undegraded":"2023-10-18T10:37:59.320220+0000","last_fullsized":"2023-10-18T10:37:59.320220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:42.536234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319810+0000","last_change":"2023-10-18T10:37:56.951389+0000","last_active":"2023-10-18T10:37:59.319810+0000","last_peered":"2023-10-18T10:37:59.319810+0000","last_clean":"2023-10-18T10:37:59.319810+0000","last_became_active":"2023-10-18T10:37:56.951138+0000","last_became_peered":"2023-10-18T10:37:56.951138+0000","last_unstale":"2023-10-18T10:37:59.319810+0000","last_undegraded":"2023-10-18T10:37:59.319810+0000","last_fullsized":"2023-10-18T10:37:59.319810+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:19:18.147109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319721+0000","last_change":"2023-10-18T10:37:57.978061+0000","last_active":"2023-10-18T10:37:59.319721+0000","last_peered":"2023-10-18T10:37:59.319721+0000","last_clean":"2023-10-18T10:37:59.319721+0000","last_became_active":"2023-10-18T10:37:57.977710+0000","last_became_peered":"2023-10-18T10:37:57.977710+0000","last_unstale":"2023-10-18T10:37:59.319721+0000","last_undegraded":"2023-10-18T10:37:59.319721+0000","last_fullsized":"2023-10-18T10:37:59.319721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:11.710841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320188+0000","last_change":"2023-10-18T10:37:56.954250+0000","last_active":"2023-10-18T10:37:59.320188+0000","last_peered":"2023-10-18T10:37:59.320188+0000","last_clean":"2023-10-18T10:37:59.320188+0000","last_became_active":"2023-10-18T10:37:56.953381+0000","last_became_peered":"2023-10-18T10:37:56.953381+0000","last_unstale":"2023-10-18T10:37:59.320188+0000","last_undegraded":"2023-10-18T10:37:59.320188+0000","last_fullsized":"2023-10-18T10:37:59.320188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:13:04.517966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320280+0000","last_change":"2023-10-18T10:37:57.965052+0000","last_active":"2023-10-18T10:37:59.320280+0000","last_peered":"2023-10-18T10:37:59.320280+0000","last_clean":"2023-10-18T10:37:59.320280+0000","last_became_active":"2023-10-18T10:37:57.964747+0000","last_became_peered":"2023-10-18T10:37:57.964747+0000","last_unstale":"2023-10-18T10:37:59.320280+0000","last_undegraded":"2023-10-18T10:37:59.320280+0000","last_fullsized":"2023-10-18T10:37:59.320280+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:49:09.644289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319395+0000","last_change":"2023-10-18T10:37:56.953872+0000","last_active":"2023-10-18T10:37:59.319395+0000","last_peered":"2023-10-18T10:37:59.319395+0000","last_clean":"2023-10-18T10:37:59.319395+0000","last_became_active":"2023-10-18T10:37:56.950129+0000","last_became_peered":"2023-10-18T10:37:56.950129+0000","last_unstale":"2023-10-18T10:37:59.319395+0000","last_undegraded":"2023-10-18T10:37:59.319395+0000","last_fullsized":"2023-10-18T10:37:59.319395+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:23.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319368+0000","last_change":"2023-10-18T10:37:57.958790+0000","last_active":"2023-10-18T10:37:59.319368+0000","last_peered":"2023-10-18T10:37:59.319368+0000","last_clean":"2023-10-18T10:37:59.319368+0000","last_became_active":"2023-10-18T10:37:57.958557+0000","last_became_peered":"2023-10-18T10:37:57.958557+0000","last_unstale":"2023-10-18T10:37:59.319368+0000","last_undegraded":"2023-10-18T10:37:59.319368+0000","last_fullsized":"2023-10-18T10:37:59.319368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:29.010713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961193+0000","last_change":"2023-10-18T10:37:56.955078+0000","last_active":"2023-10-18T10:37:58.961193+0000","last_peered":"2023-10-18T10:37:58.961193+0000","last_clean":"2023-10-18T10:37:58.961193+0000","last_became_active":"2023-10-18T10:37:56.954762+0000","last_became_peered":"2023-10-18T10:37:56.954762+0000","last_unstale":"2023-10-18T10:37:58.961193+0000","last_undegraded":"2023-10-18T10:37:58.961193+0000","last_fullsized":"2023-10-18T10:37:58.961193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:18.039701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320597+0000","last_change":"2023-10-18T10:37:57.975675+0000","last_active":"2023-10-18T10:37:59.320597+0000","last_peered":"2023-10-18T10:37:59.320597+0000","last_clean":"2023-10-18T10:37:59.320597+0000","last_became_active":"2023-10-18T10:37:57.974990+0000","last_became_peered":"2023-10-18T10:37:57.974990+0000","last_unstale":"2023-10-18T10:37:59.320597+0000","last_undegraded":"2023-10-18T10:37:59.320597+0000","last_fullsized":"2023-10-18T10:37:59.320597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:43.879323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319448+0000","last_change":"2023-10-18T10:37:56.950627+0000","last_active":"2023-10-18T10:37:59.319448+0000","last_peered":"2023-10-18T10:37:59.319448+0000","last_clean":"2023-10-18T10:37:59.319448+0000","last_became_active":"2023-10-18T10:37:56.950306+0000","last_became_peered":"2023-10-18T10:37:56.950306+0000","last_unstale":"2023-10-18T10:37:59.319448+0000","last_undegraded":"2023-10-18T10:37:59.319448+0000","last_fullsized":"2023-10-18T10:37:59.319448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:17.386951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319403+0000","last_change":"2023-10-18T10:37:57.966257+0000","last_active":"2023-10-18T10:37:59.319403+0000","last_peered":"2023-10-18T10:37:59.319403+0000","last_clean":"2023-10-18T10:37:59.319403+0000","last_became_active":"2023-10-18T10:37:57.965520+0000","last_became_peered":"2023-10-18T10:37:57.965520+0000","last_unstale":"2023-10-18T10:37:59.319403+0000","last_undegraded":"2023-10-18T10:37:59.319403+0000","last_fullsized":"2023-10-18T10:37:59.319403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:35.328724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319563+0000","last_change":"2023-10-18T10:37:56.950713+0000","last_active":"2023-10-18T10:37:59.319563+0000","last_peered":"2023-10-18T10:37:59.319563+0000","last_clean":"2023-10-18T10:37:59.319563+0000","last_became_active":"2023-10-18T10:37:56.950433+0000","last_became_peered":"2023-10-18T10:37:56.950433+0000","last_unstale":"2023-10-18T10:37:59.319563+0000","last_undegraded":"2023-10-18T10:37:59.319563+0000","last_fullsized":"2023-10-18T10:37:59.319563+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:45.317821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"53'139","reported_seq":288,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-18T10:37:57.955261+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:37:57.955261+0000","last_peered":"2023-10-18T10:37:57.955261+0000","last_clean":"2023-10-18T10:37:57.955261+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:37:57.955261+0000","last_undegraded":"2023-10-18T10:37:57.955261+0000","last_fullsized":"2023-10-18T10:37:57.955261+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":969833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.27900000000000003,"15min":0.23799999999999999},"max":{"1min":0.77800000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.56499999999999995},"min":{"1min":0.47399999999999998,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.55900000000000005},"min":{"1min":0.46700000000000003,"5min":0.38,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.45900000000000002,"5min":0.34899999999999998,"15min":0.33200000000000002},"max":{"1min":0.80400000000000005,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.625,"15min":0.58699999999999997},"min":{"1min":0.45500000000000002,"5min":0.36699999999999999,"15min":0.29399999999999998},"max":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.57699999999999996},"min":{"1min":0.46300000000000002,"5min":0.40200000000000002,"15min":0.33400000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.026},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60399999999999998,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.78800000000000003,"15min":0.81899999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.48399999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.68899999999999995,"5min":0.70499999999999996,"15min":0.76500000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.46000000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.91500000000000004},"last":0.86199999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.83999999999999997},"last":0.68300000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":1.052},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53700000000000003,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.95399999999999996},"last":0.91700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":379993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.71599999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78500000000000003,"5min":0.82399999999999995,"15min":1.016},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.28899999999999998},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.85299999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.29299999999999998},"max":{"1min":0.80900000000000005,"5min":0.81399999999999995,"15min":0.88800000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.67900000000000005,"5min":0.80800000000000005,"15min":0.85699999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.56299999999999994},"min":{"1min":0.503,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.875},"last":0.57899999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.318},"max":{"1min":0.73199999999999998,"5min":0.78700000000000003,"15min":0.82199999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.59299999999999997},"min":{"1min":0.45300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.84399999999999997,"15min":0.91800000000000004},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.437,"15min":0.45000000000000001},"min":{"1min":0.29199999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70199999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76600000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.62}]},{"osd":5,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45900000000000002,"15min":0.47499999999999998},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70899999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64600000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.436}]},{"osd":7,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48799999999999999,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.66500000000000004,"5min":0.78600000000000003,"15min":0.81100000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56999999999999995,"15min":0.58699999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.87,"15min":0.87},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":379341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.28199999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.57599999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.79500000000000004,"15min":0.80500000000000005},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.621,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61499999999999999,"15min":0.59199999999999997},"min":{"1min":0.53000000000000003,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60999999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.46999999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":0.84199999999999997,"15min":0.86399999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":1.0620000000000001},"last":0.57699999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.622,"15min":0.58999999999999997},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":0.83499999999999996,"15min":0.83699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.47399999999999998,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":1.1240000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.61299999999999999,"5min":0.70099999999999996,"15min":1.262},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":0.63600000000000001,"5min":0.78900000000000003,"15min":1.028},"last":0.51100000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.68999999999999995,"15min":1.1899999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.35099999999999998,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.80200000000000005,"15min":1.224},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.64800000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69299999999999995,"5min":0.79800000000000004,"15min":1.159},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":379923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.48199999999999998,"5min":0.33100000000000002,"15min":0.26000000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.84299999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.56899999999999995},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.33000000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.67300000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.57299999999999995},"min":{"1min":0.47399999999999998,"5min":0.376,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.81000000000000005,"15min":0.81899999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.58799999999999997},"min":{"1min":0.38500000000000001,"5min":0.372,"15min":0.29499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.59299999999999997},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59499999999999997,"15min":0.56399999999999995},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.313},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35799999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.32900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79100000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.50700000000000001},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.248},"max":{"1min":0.71299999999999997,"5min":0.78200000000000003,"15min":0.79600000000000004},"last":0.373}]},{"osd":6,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.502,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79400000000000004,"15min":0.80700000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.33800000000000002,"15min":0.28399999999999997},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.90400000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.84799999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.27100000000000002},"max":{"1min":0.80800000000000005,"5min":0.85899999999999999,"15min":0.879},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":794624,"data_stored":380975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.19500000000000001},"max":{"1min":0.46500000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.39400000000000002,"15min":0.378},"min":{"1min":0.254,"5min":0.254,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.57099999999999995,"15min":0.68999999999999995},"last":0.33400000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.248,"5min":0.248,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.70699999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42899999999999999,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.23000000000000001},"max":{"1min":0.624,"5min":0.65600000000000003,"15min":0.84799999999999998},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.41999999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.19900000000000001},"max":{"1min":0.60199999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45600000000000002,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.25600000000000001},"max":{"1min":0.60999999999999999,"5min":0.627,"15min":0.69299999999999995},"last":0.32200000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81999999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63600000000000001,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.504}]},{"osd":5,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.626},"min":{"1min":0.376,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.78300000000000003,"5min":1.002,"15min":1.002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.307},"max":{"1min":0.79300000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.77600000000000002,"5min":0.93700000000000006,"15min":0.96899999999999997},"last":0.442},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.88600000000000001,"15min":1.0349999999999999},"last":0.49099999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.38300000000000001},"max":{"1min":0.78800000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54168,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317672,"statfs":{"total":96636764160,"available":96581296128,"internally_reserved":0,"allocated":790528,"data_stored":380953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.36299999999999999},"min":{"1min":0.28399999999999997,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.219},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.252,"5min":0.192,"15min":0.192},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.63100000000000001},"last":0.318}]},{"osd":1,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40699999999999997,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.20899999999999999},"max":{"1min":0.54700000000000004,"5min":0.61399999999999999,"15min":0.82199999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.26100000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40699999999999997,"15min":0.38300000000000001},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.55100000000000005,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.34100000000000003,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.33800000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.63300000000000001},"min":{"1min":0.38500000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.81999999999999995,"5min":0.88800000000000001,"15min":1.458},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67300000000000004,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64400000000000002,"15min":0.61099999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91400000000000003,"5min":0.95699999999999996,"15min":0.997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63800000000000001,"15min":0.60299999999999998},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82499999999999996,"5min":0.93600000000000005,"15min":0.96699999999999997},"last":0.498},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.63500000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.84799999999999998,"5min":0.93700000000000006,"15min":1.028},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66900000000000004,"15min":0.63},"min":{"1min":0.41699999999999998,"5min":0.373,"15min":0.36399999999999999},"max":{"1min":0.81999999999999995,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.879,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":379501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.379,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56200000000000006,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.372},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.19800000000000001},"max":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.63300000000000001},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40899999999999997,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.25,"15min":0.186},"max":{"1min":0.54800000000000004,"5min":0.68899999999999995,"15min":0.70799999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23999999999999999},"max":{"1min":0.56100000000000005,"5min":0.64800000000000002,"15min":0.747},"last":0.496}]},{"osd":3,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.438,"15min":0.40699999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.61499999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.28199999999999997,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.60399999999999998},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.34100000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.753},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.61099999999999999},"min":{"1min":0.53400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.86599999999999999,"15min":0.92700000000000005},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.47399999999999998,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.90700000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.745},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.61199999999999999},"min":{"1min":0.47099999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65100000000000002,"15min":0.624},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89100000000000001,"5min":0.92000000000000004,"15min":0.94299999999999995},"last":0.69299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92400000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.371},"max":{"1min":0.93200000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1380352,"data_stored":969765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.224,"15min":0.192},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.71099999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.25900000000000001,"15min":0.17799999999999999},"max":{"1min":0.56299999999999994,"5min":0.61099999999999999,"15min":0.76900000000000002},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.39200000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.187},"max":{"1min":0.60399999999999998,"5min":0.63400000000000001,"15min":0.96099999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42299999999999999,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.23999999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.93700000000000006},"last":0.36499999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39700000000000002,"15min":0.38900000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.222},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.64400000000000002},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.45000000000000001,"15min":0.442},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.22800000000000001},"max":{"1min":0.54500000000000004,"5min":0.65500000000000003,"15min":0.91100000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.45300000000000001,"5min":0.40200000000000002,"15min":0.35699999999999998},"max":{"1min":0.92700000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.46100000000000002,"5min":0.45800000000000002,"15min":0.34699999999999998},"max":{"1min":0.92900000000000005,"5min":0.96199999999999997,"15min":0.96799999999999997},"last":0.58799999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.41299999999999998,"15min":0.371},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.46899999999999997,"5min":0.45200000000000001,"15min":0.41099999999999998},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66100000000000003,"15min":0.64800000000000002},"min":{"1min":0.498,"5min":0.432,"15min":0.41999999999999998},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.48199999999999998,"5min":0.44600000000000001,"15min":0.33200000000000002},"max":{"1min":0.88800000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:38:01.279 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:38:01.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.414+0000 7f14bd07c640 1 -- 172.21.15.125:0/378461671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b80cc090 msgr2=0x7f14b81581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.414+0000 7f14bd07c640 1 --2- 172.21.15.125:0/378461671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b80cc090 0x7f14b81581e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f14a0005fd0 tx=0x7f14a002be10 comp rx=0 tx=0).stop 2023-10-18T10:38:01.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.414+0000 7f14bd07c640 1 -- 172.21.15.125:0/378461671 shutdown_connections 2023-10-18T10:38:01.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.414+0000 7f14bd07c640 1 --2- 172.21.15.125:0/378461671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b815c450 0x7f14b8158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.414+0000 7f14bd07c640 1 --2- 172.21.15.125:0/378461671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b80cc090 0x7f14b81581e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.414+0000 7f14bd07c640 1 -- 172.21.15.125:0/378461671 >> 172.21.15.125:0/378461671 conn(0x7f14b8069930 msgr2=0x7f14b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:01.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.414+0000 7f14bd07c640 1 -- 172.21.15.125:0/378461671 shutdown_connections 2023-10-18T10:38:01.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.414+0000 7f14bd07c640 1 -- 172.21.15.125:0/378461671 wait complete. 2023-10-18T10:38:01.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.416+0000 7f14bd07c640 1 Processor -- start 2023-10-18T10:38:01.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.416+0000 7f14bd07c640 1 -- start start 2023-10-18T10:38:01.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.416+0000 7f14bd07c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80cc090 0x7f14b8137db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.416+0000 7f14bd07c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b81413a0 0x7f14b8135020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.416+0000 7f14bd07c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b815c450 0x7f14b8151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.416+0000 7f14bd07c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8154010 con 0x7f14b815c450 2023-10-18T10:38:01.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.416+0000 7f14bd07c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8152150 con 0x7f14b81413a0 2023-10-18T10:38:01.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.416+0000 7f14bd07c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8154260 con 0x7f14b80cc090 2023-10-18T10:38:01.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b815c450 0x7f14b8151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b815c450 0x7f14b8151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55464/0 (socket says 172.21.15.125:55464) 2023-10-18T10:38:01.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 -- 172.21.15.125:0/2788233592 learned_addr learned my addr 172.21.15.125:0/2788233592 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:01.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14affff640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b81413a0 0x7f14b8135020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b6d9d640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80cc090 0x7f14b8137db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 -- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80cc090 msgr2=0x7f14b8137db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80cc090 0x7f14b8137db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 -- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b81413a0 msgr2=0x7f14b8135020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b81413a0 0x7f14b8135020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 -- 172.21.15.125:0/2788233592 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a0009560 con 0x7f14b815c450 2023-10-18T10:38:01.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b6d9d640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80cc090 0x7f14b8137db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:38:01.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14b759e640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b815c450 0x7f14b8151010 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f14b000e960 tx=0x7f14b000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:01.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14af7fe640 1 -- 172.21.15.125:0/2788233592 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0004d20 con 0x7f14b815c450 2023-10-18T10:38:01.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14af7fe640 1 -- 172.21.15.125:0/2788233592 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f14b000cb20 con 0x7f14b815c450 2023-10-18T10:38:01.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b8154540 con 0x7f14b815c450 2023-10-18T10:38:01.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14af7fe640 1 -- 172.21.15.125:0/2788233592 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0005020 con 0x7f14b815c450 2023-10-18T10:38:01.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.417+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b81530c0 con 0x7f14b815c450 2023-10-18T10:38:01.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.418+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b80690e0 con 0x7f14b815c450 2023-10-18T10:38:01.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.419+0000 7f14af7fe640 1 -- 172.21.15.125:0/2788233592 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f14b0005180 con 0x7f14b815c450 2023-10-18T10:38:01.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.419+0000 7f14af7fe640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f149806af60 0x7f149806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.419+0000 7f14af7fe640 1 -- 172.21.15.125:0/2788233592 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f14b008a400 con 0x7f14b815c450 2023-10-18T10:38:01.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.419+0000 7f14b6d9d640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f149806af60 0x7f149806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.420+0000 7f14b6d9d640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f149806af60 0x7f149806d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f14a8001680 tx=0x7f14a8009040 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:01.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.423+0000 7f14af7fe640 1 -- 172.21.15.125:0/2788233592 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14b0055150 con 0x7f14b815c450 2023-10-18T10:38:01.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.586+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 --> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14b806ff90 con 0x7f149806af60 2023-10-18T10:38:01.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.594+0000 7f14af7fe640 1 -- 172.21.15.125:0/2788233592 <== mgr.5626 v2:172.21.15.139:6832/2140727580 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289783 (secure 0 0 0) 0x7f14b806ff90 con 0x7f149806af60 2023-10-18T10:38:01.599 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:38:01.601 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:38:01.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f149806af60 msgr2=0x7f149806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f149806af60 0x7f149806d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f14a8001680 tx=0x7f14a8009040 comp rx=0 tx=0).stop 2023-10-18T10:38:01.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b815c450 msgr2=0x7f14b8151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b815c450 0x7f14b8151010 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f14b000e960 tx=0x7f14b000ee20 comp rx=0 tx=0).stop 2023-10-18T10:38:01.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 shutdown_connections 2023-10-18T10:38:01.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f149806af60 0x7f149806d420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b81413a0 0x7f14b8135020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14b815c450 0x7f14b8151010 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 --2- 172.21.15.125:0/2788233592 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80cc090 0x7f14b8137db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 >> 172.21.15.125:0/2788233592 conn(0x7f14b8069930 msgr2=0x7f14b80b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:01.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 shutdown_connections 2023-10-18T10:38:01.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.597+0000 7f14bd07c640 1 -- 172.21.15.125:0/2788233592 wait complete. 2023-10-18T10:38:01.609 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:37:59.832855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297040,"kb_used_data":7304,"kb_used_omap":12,"kb_used_meta":289587,"kb_avail":754677680,"statfs":{"total":773094113280,"available":772789944320,"internally_reserved":0,"allocated":7479296,"data_stored":4220284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":296537656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000859"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319834+0000","last_change":"2023-10-18T10:37:57.953401+0000","last_active":"2023-10-18T10:37:59.319834+0000","last_peered":"2023-10-18T10:37:59.319834+0000","last_clean":"2023-10-18T10:37:59.319834+0000","last_became_active":"2023-10-18T10:37:57.953090+0000","last_became_peered":"2023-10-18T10:37:57.953090+0000","last_unstale":"2023-10-18T10:37:59.319834+0000","last_undegraded":"2023-10-18T10:37:59.319834+0000","last_fullsized":"2023-10-18T10:37:59.319834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:55:30.673197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320473+0000","last_change":"2023-10-18T10:37:57.966749+0000","last_active":"2023-10-18T10:37:59.320473+0000","last_peered":"2023-10-18T10:37:59.320473+0000","last_clean":"2023-10-18T10:37:59.320473+0000","last_became_active":"2023-10-18T10:37:57.966495+0000","last_became_peered":"2023-10-18T10:37:57.966495+0000","last_unstale":"2023-10-18T10:37:59.320473+0000","last_undegraded":"2023-10-18T10:37:59.320473+0000","last_fullsized":"2023-10-18T10:37:59.320473+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:49:09.644289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319428+0000","last_change":"2023-10-18T10:37:57.965988+0000","last_active":"2023-10-18T10:37:59.319428+0000","last_peered":"2023-10-18T10:37:59.319428+0000","last_clean":"2023-10-18T10:37:59.319428+0000","last_became_active":"2023-10-18T10:37:57.965152+0000","last_became_peered":"2023-10-18T10:37:57.965152+0000","last_unstale":"2023-10-18T10:37:59.319428+0000","last_undegraded":"2023-10-18T10:37:59.319428+0000","last_fullsized":"2023-10-18T10:37:59.319428+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:44.808490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320075+0000","last_change":"2023-10-18T10:37:57.977203+0000","last_active":"2023-10-18T10:37:59.320075+0000","last_peered":"2023-10-18T10:37:59.320075+0000","last_clean":"2023-10-18T10:37:59.320075+0000","last_became_active":"2023-10-18T10:37:57.976913+0000","last_became_peered":"2023-10-18T10:37:57.976913+0000","last_unstale":"2023-10-18T10:37:59.320075+0000","last_undegraded":"2023-10-18T10:37:59.320075+0000","last_fullsized":"2023-10-18T10:37:59.320075+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:44.808490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961527+0000","last_change":"2023-10-18T10:37:57.955122+0000","last_active":"2023-10-18T10:37:58.961527+0000","last_peered":"2023-10-18T10:37:58.961527+0000","last_clean":"2023-10-18T10:37:58.961527+0000","last_became_active":"2023-10-18T10:37:57.954807+0000","last_became_peered":"2023-10-18T10:37:57.954807+0000","last_unstale":"2023-10-18T10:37:58.961527+0000","last_undegraded":"2023-10-18T10:37:58.961527+0000","last_fullsized":"2023-10-18T10:37:58.961527+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:29.465806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319557+0000","last_change":"2023-10-18T10:37:57.962347+0000","last_active":"2023-10-18T10:37:59.319557+0000","last_peered":"2023-10-18T10:37:59.319557+0000","last_clean":"2023-10-18T10:37:59.319557+0000","last_became_active":"2023-10-18T10:37:57.962056+0000","last_became_peered":"2023-10-18T10:37:57.962056+0000","last_unstale":"2023-10-18T10:37:59.319557+0000","last_undegraded":"2023-10-18T10:37:59.319557+0000","last_fullsized":"2023-10-18T10:37:59.319557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:26.065661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319191+0000","last_change":"2023-10-18T10:37:57.960830+0000","last_active":"2023-10-18T10:37:59.319191+0000","last_peered":"2023-10-18T10:37:59.319191+0000","last_clean":"2023-10-18T10:37:59.319191+0000","last_became_active":"2023-10-18T10:37:57.959457+0000","last_became_peered":"2023-10-18T10:37:57.959457+0000","last_unstale":"2023-10-18T10:37:59.319191+0000","last_undegraded":"2023-10-18T10:37:59.319191+0000","last_fullsized":"2023-10-18T10:37:59.319191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:24.198102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320306+0000","last_change":"2023-10-18T10:37:57.971563+0000","last_active":"2023-10-18T10:37:59.320306+0000","last_peered":"2023-10-18T10:37:59.320306+0000","last_clean":"2023-10-18T10:37:59.320306+0000","last_became_active":"2023-10-18T10:37:57.968579+0000","last_became_peered":"2023-10-18T10:37:57.968579+0000","last_unstale":"2023-10-18T10:37:59.320306+0000","last_undegraded":"2023-10-18T10:37:59.320306+0000","last_fullsized":"2023-10-18T10:37:59.320306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:24.819764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320543+0000","last_change":"2023-10-18T10:37:57.975017+0000","last_active":"2023-10-18T10:37:59.320543+0000","last_peered":"2023-10-18T10:37:59.320543+0000","last_clean":"2023-10-18T10:37:59.320543+0000","last_became_active":"2023-10-18T10:37:57.974211+0000","last_became_peered":"2023-10-18T10:37:57.974211+0000","last_unstale":"2023-10-18T10:37:59.320543+0000","last_undegraded":"2023-10-18T10:37:59.320543+0000","last_fullsized":"2023-10-18T10:37:59.320543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:23:59.673836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319413+0000","last_change":"2023-10-18T10:37:57.953933+0000","last_active":"2023-10-18T10:37:59.319413+0000","last_peered":"2023-10-18T10:37:59.319413+0000","last_clean":"2023-10-18T10:37:59.319413+0000","last_became_active":"2023-10-18T10:37:57.953671+0000","last_became_peered":"2023-10-18T10:37:57.953671+0000","last_unstale":"2023-10-18T10:37:59.319413+0000","last_undegraded":"2023-10-18T10:37:59.319413+0000","last_fullsized":"2023-10-18T10:37:59.319413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:15.822428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.955554+0000","last_change":"2023-10-18T10:37:57.971723+0000","last_active":"2023-10-18T10:37:58.955554+0000","last_peered":"2023-10-18T10:37:58.955554+0000","last_clean":"2023-10-18T10:37:58.955554+0000","last_became_active":"2023-10-18T10:37:57.971444+0000","last_became_peered":"2023-10-18T10:37:57.971444+0000","last_unstale":"2023-10-18T10:37:58.955554+0000","last_undegraded":"2023-10-18T10:37:58.955554+0000","last_fullsized":"2023-10-18T10:37:58.955554+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:18:48.291519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320289+0000","last_change":"2023-10-18T10:37:57.976929+0000","last_active":"2023-10-18T10:37:59.320289+0000","last_peered":"2023-10-18T10:37:59.320289+0000","last_clean":"2023-10-18T10:37:59.320289+0000","last_became_active":"2023-10-18T10:37:57.976421+0000","last_became_peered":"2023-10-18T10:37:57.976421+0000","last_unstale":"2023-10-18T10:37:59.320289+0000","last_undegraded":"2023-10-18T10:37:59.320289+0000","last_fullsized":"2023-10-18T10:37:59.320289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:21.654897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319151+0000","last_change":"2023-10-18T10:37:57.955417+0000","last_active":"2023-10-18T10:37:59.319151+0000","last_peered":"2023-10-18T10:37:59.319151+0000","last_clean":"2023-10-18T10:37:59.319151+0000","last_became_active":"2023-10-18T10:37:57.955013+0000","last_became_peered":"2023-10-18T10:37:57.955013+0000","last_unstale":"2023-10-18T10:37:59.319151+0000","last_undegraded":"2023-10-18T10:37:59.319151+0000","last_fullsized":"2023-10-18T10:37:59.319151+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:50:02.468827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320414+0000","last_change":"2023-10-18T10:37:57.958294+0000","last_active":"2023-10-18T10:37:59.320414+0000","last_peered":"2023-10-18T10:37:59.320414+0000","last_clean":"2023-10-18T10:37:59.320414+0000","last_became_active":"2023-10-18T10:37:57.958050+0000","last_became_peered":"2023-10-18T10:37:57.958050+0000","last_unstale":"2023-10-18T10:37:59.320414+0000","last_undegraded":"2023-10-18T10:37:59.320414+0000","last_fullsized":"2023-10-18T10:37:59.320414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:11.915134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319505+0000","last_change":"2023-10-18T10:37:57.964193+0000","last_active":"2023-10-18T10:37:59.319505+0000","last_peered":"2023-10-18T10:37:59.319505+0000","last_clean":"2023-10-18T10:37:59.319505+0000","last_became_active":"2023-10-18T10:37:57.963911+0000","last_became_peered":"2023-10-18T10:37:57.963911+0000","last_unstale":"2023-10-18T10:37:59.319505+0000","last_undegraded":"2023-10-18T10:37:59.319505+0000","last_fullsized":"2023-10-18T10:37:59.319505+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:24.198102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320022+0000","last_change":"2023-10-18T10:37:57.954607+0000","last_active":"2023-10-18T10:37:59.320022+0000","last_peered":"2023-10-18T10:37:59.320022+0000","last_clean":"2023-10-18T10:37:59.320022+0000","last_became_active":"2023-10-18T10:37:57.954299+0000","last_became_peered":"2023-10-18T10:37:57.954299+0000","last_unstale":"2023-10-18T10:37:59.320022+0000","last_undegraded":"2023-10-18T10:37:59.320022+0000","last_fullsized":"2023-10-18T10:37:59.320022+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:01.709827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319932+0000","last_change":"2023-10-18T10:37:57.966451+0000","last_active":"2023-10-18T10:37:59.319932+0000","last_peered":"2023-10-18T10:37:59.319932+0000","last_clean":"2023-10-18T10:37:59.319932+0000","last_became_active":"2023-10-18T10:37:57.965325+0000","last_became_peered":"2023-10-18T10:37:57.965325+0000","last_unstale":"2023-10-18T10:37:59.319932+0000","last_undegraded":"2023-10-18T10:37:59.319932+0000","last_fullsized":"2023-10-18T10:37:59.319932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:05:52.337294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320569+0000","last_change":"2023-10-18T10:37:57.967333+0000","last_active":"2023-10-18T10:37:59.320569+0000","last_peered":"2023-10-18T10:37:59.320569+0000","last_clean":"2023-10-18T10:37:59.320569+0000","last_became_active":"2023-10-18T10:37:57.967087+0000","last_became_peered":"2023-10-18T10:37:57.967087+0000","last_unstale":"2023-10-18T10:37:59.320569+0000","last_undegraded":"2023-10-18T10:37:59.320569+0000","last_fullsized":"2023-10-18T10:37:59.320569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:42:01.890239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319547+0000","last_change":"2023-10-18T10:37:57.965744+0000","last_active":"2023-10-18T10:37:59.319547+0000","last_peered":"2023-10-18T10:37:59.319547+0000","last_clean":"2023-10-18T10:37:59.319547+0000","last_became_active":"2023-10-18T10:37:57.964795+0000","last_became_peered":"2023-10-18T10:37:57.964795+0000","last_unstale":"2023-10-18T10:37:59.319547+0000","last_undegraded":"2023-10-18T10:37:59.319547+0000","last_fullsized":"2023-10-18T10:37:59.319547+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:42:01.890239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319882+0000","last_change":"2023-10-18T10:37:57.965614+0000","last_active":"2023-10-18T10:37:59.319882+0000","last_peered":"2023-10-18T10:37:59.319882+0000","last_clean":"2023-10-18T10:37:59.319882+0000","last_became_active":"2023-10-18T10:37:57.965337+0000","last_became_peered":"2023-10-18T10:37:57.965337+0000","last_unstale":"2023-10-18T10:37:59.319882+0000","last_undegraded":"2023-10-18T10:37:59.319882+0000","last_fullsized":"2023-10-18T10:37:59.319882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:36.204628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319672+0000","last_change":"2023-10-18T10:37:57.966843+0000","last_active":"2023-10-18T10:37:59.319672+0000","last_peered":"2023-10-18T10:37:59.319672+0000","last_clean":"2023-10-18T10:37:59.319672+0000","last_became_active":"2023-10-18T10:37:57.966583+0000","last_became_peered":"2023-10-18T10:37:57.966583+0000","last_unstale":"2023-10-18T10:37:59.319672+0000","last_undegraded":"2023-10-18T10:37:59.319672+0000","last_fullsized":"2023-10-18T10:37:59.319672+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:42.559543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319241+0000","last_change":"2023-10-18T10:37:57.969205+0000","last_active":"2023-10-18T10:37:59.319241+0000","last_peered":"2023-10-18T10:37:59.319241+0000","last_clean":"2023-10-18T10:37:59.319241+0000","last_became_active":"2023-10-18T10:37:57.968910+0000","last_became_peered":"2023-10-18T10:37:57.968910+0000","last_unstale":"2023-10-18T10:37:59.319241+0000","last_undegraded":"2023-10-18T10:37:59.319241+0000","last_fullsized":"2023-10-18T10:37:59.319241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:52.175815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.955515+0000","last_change":"2023-10-18T10:37:57.960187+0000","last_active":"2023-10-18T10:37:58.955515+0000","last_peered":"2023-10-18T10:37:58.955515+0000","last_clean":"2023-10-18T10:37:58.955515+0000","last_became_active":"2023-10-18T10:37:57.959903+0000","last_became_peered":"2023-10-18T10:37:57.959903+0000","last_unstale":"2023-10-18T10:37:58.955515+0000","last_undegraded":"2023-10-18T10:37:58.955515+0000","last_fullsized":"2023-10-18T10:37:58.955515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:00:28.477842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320146+0000","last_change":"2023-10-18T10:37:57.956512+0000","last_active":"2023-10-18T10:37:59.320146+0000","last_peered":"2023-10-18T10:37:59.320146+0000","last_clean":"2023-10-18T10:37:59.320146+0000","last_became_active":"2023-10-18T10:37:57.956255+0000","last_became_peered":"2023-10-18T10:37:57.956255+0000","last_unstale":"2023-10-18T10:37:59.320146+0000","last_undegraded":"2023-10-18T10:37:59.320146+0000","last_fullsized":"2023-10-18T10:37:59.320146+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:30:31.277948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320164+0000","last_change":"2023-10-18T10:37:57.971446+0000","last_active":"2023-10-18T10:37:59.320164+0000","last_peered":"2023-10-18T10:37:59.320164+0000","last_clean":"2023-10-18T10:37:59.320164+0000","last_became_active":"2023-10-18T10:37:57.970902+0000","last_became_peered":"2023-10-18T10:37:57.970902+0000","last_unstale":"2023-10-18T10:37:59.320164+0000","last_undegraded":"2023-10-18T10:37:59.320164+0000","last_fullsized":"2023-10-18T10:37:59.320164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:21:19.628260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319576+0000","last_change":"2023-10-18T10:37:57.974213+0000","last_active":"2023-10-18T10:37:59.319576+0000","last_peered":"2023-10-18T10:37:59.319576+0000","last_clean":"2023-10-18T10:37:59.319576+0000","last_became_active":"2023-10-18T10:37:57.972067+0000","last_became_peered":"2023-10-18T10:37:57.972067+0000","last_unstale":"2023-10-18T10:37:59.319576+0000","last_undegraded":"2023-10-18T10:37:59.319576+0000","last_fullsized":"2023-10-18T10:37:59.319576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:30:31.277948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319761+0000","last_change":"2023-10-18T10:37:57.967112+0000","last_active":"2023-10-18T10:37:59.319761+0000","last_peered":"2023-10-18T10:37:59.319761+0000","last_clean":"2023-10-18T10:37:59.319761+0000","last_became_active":"2023-10-18T10:37:57.966819+0000","last_became_peered":"2023-10-18T10:37:57.966819+0000","last_unstale":"2023-10-18T10:37:59.319761+0000","last_undegraded":"2023-10-18T10:37:59.319761+0000","last_fullsized":"2023-10-18T10:37:59.319761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:47.370984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319957+0000","last_change":"2023-10-18T10:37:57.965077+0000","last_active":"2023-10-18T10:37:59.319957+0000","last_peered":"2023-10-18T10:37:59.319957+0000","last_clean":"2023-10-18T10:37:59.319957+0000","last_became_active":"2023-10-18T10:37:57.964785+0000","last_became_peered":"2023-10-18T10:37:57.964785+0000","last_unstale":"2023-10-18T10:37:59.319957+0000","last_undegraded":"2023-10-18T10:37:59.319957+0000","last_fullsized":"2023-10-18T10:37:59.319957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:59.117648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319799+0000","last_change":"2023-10-18T10:37:57.963167+0000","last_active":"2023-10-18T10:37:59.319799+0000","last_peered":"2023-10-18T10:37:59.319799+0000","last_clean":"2023-10-18T10:37:59.319799+0000","last_became_active":"2023-10-18T10:37:57.962849+0000","last_became_peered":"2023-10-18T10:37:57.962849+0000","last_unstale":"2023-10-18T10:37:59.319799+0000","last_undegraded":"2023-10-18T10:37:59.319799+0000","last_fullsized":"2023-10-18T10:37:59.319799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:13:05.525565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319620+0000","last_change":"2023-10-18T10:37:57.971806+0000","last_active":"2023-10-18T10:37:59.319620+0000","last_peered":"2023-10-18T10:37:59.319620+0000","last_clean":"2023-10-18T10:37:59.319620+0000","last_became_active":"2023-10-18T10:37:57.971530+0000","last_became_peered":"2023-10-18T10:37:57.971530+0000","last_unstale":"2023-10-18T10:37:59.319620+0000","last_undegraded":"2023-10-18T10:37:59.319620+0000","last_fullsized":"2023-10-18T10:37:59.319620+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:14.428640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319218+0000","last_change":"2023-10-18T10:37:57.956429+0000","last_active":"2023-10-18T10:37:59.319218+0000","last_peered":"2023-10-18T10:37:59.319218+0000","last_clean":"2023-10-18T10:37:59.319218+0000","last_became_active":"2023-10-18T10:37:57.956171+0000","last_became_peered":"2023-10-18T10:37:57.956171+0000","last_unstale":"2023-10-18T10:37:59.319218+0000","last_undegraded":"2023-10-18T10:37:59.319218+0000","last_fullsized":"2023-10-18T10:37:59.319218+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:14.428640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319350+0000","last_change":"2023-10-18T10:37:57.971096+0000","last_active":"2023-10-18T10:37:59.319350+0000","last_peered":"2023-10-18T10:37:59.319350+0000","last_clean":"2023-10-18T10:37:59.319350+0000","last_became_active":"2023-10-18T10:37:57.970788+0000","last_became_peered":"2023-10-18T10:37:57.970788+0000","last_unstale":"2023-10-18T10:37:59.319350+0000","last_undegraded":"2023-10-18T10:37:59.319350+0000","last_fullsized":"2023-10-18T10:37:59.319350+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:35.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319324+0000","last_change":"2023-10-18T10:37:57.966206+0000","last_active":"2023-10-18T10:37:59.319324+0000","last_peered":"2023-10-18T10:37:59.319324+0000","last_clean":"2023-10-18T10:37:59.319324+0000","last_became_active":"2023-10-18T10:37:57.965924+0000","last_became_peered":"2023-10-18T10:37:57.965924+0000","last_unstale":"2023-10-18T10:37:59.319324+0000","last_undegraded":"2023-10-18T10:37:59.319324+0000","last_fullsized":"2023-10-18T10:37:59.319324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:35.227778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"64'4","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319684+0000","last_change":"2023-10-18T10:37:56.945466+0000","last_active":"2023-10-18T10:37:59.319684+0000","last_peered":"2023-10-18T10:37:59.319684+0000","last_clean":"2023-10-18T10:37:59.319684+0000","last_became_active":"2023-10-18T10:37:56.945183+0000","last_became_peered":"2023-10-18T10:37:56.945183+0000","last_unstale":"2023-10-18T10:37:59.319684+0000","last_undegraded":"2023-10-18T10:37:59.319684+0000","last_fullsized":"2023-10-18T10:37:59.319684+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:33:55.760075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319734+0000","last_change":"2023-10-18T10:37:57.962326+0000","last_active":"2023-10-18T10:37:59.319734+0000","last_peered":"2023-10-18T10:37:59.319734+0000","last_clean":"2023-10-18T10:37:59.319734+0000","last_became_active":"2023-10-18T10:37:57.962021+0000","last_became_peered":"2023-10-18T10:37:57.962021+0000","last_unstale":"2023-10-18T10:37:59.319734+0000","last_undegraded":"2023-10-18T10:37:59.319734+0000","last_fullsized":"2023-10-18T10:37:59.319734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:29.010713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319490+0000","last_change":"2023-10-18T10:37:56.952243+0000","last_active":"2023-10-18T10:37:59.319490+0000","last_peered":"2023-10-18T10:37:59.319490+0000","last_clean":"2023-10-18T10:37:59.319490+0000","last_became_active":"2023-10-18T10:37:56.951937+0000","last_became_peered":"2023-10-18T10:37:56.951937+0000","last_unstale":"2023-10-18T10:37:59.319490+0000","last_undegraded":"2023-10-18T10:37:59.319490+0000","last_fullsized":"2023-10-18T10:37:59.319490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:02.773566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320032+0000","last_change":"2023-10-18T10:37:57.967696+0000","last_active":"2023-10-18T10:37:59.320032+0000","last_peered":"2023-10-18T10:37:59.320032+0000","last_clean":"2023-10-18T10:37:59.320032+0000","last_became_active":"2023-10-18T10:37:57.967387+0000","last_became_peered":"2023-10-18T10:37:57.967387+0000","last_unstale":"2023-10-18T10:37:59.320032+0000","last_undegraded":"2023-10-18T10:37:59.320032+0000","last_fullsized":"2023-10-18T10:37:59.320032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:04:29.465806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319849+0000","last_change":"2023-10-18T10:37:56.952879+0000","last_active":"2023-10-18T10:37:59.319849+0000","last_peered":"2023-10-18T10:37:59.319849+0000","last_clean":"2023-10-18T10:37:59.319849+0000","last_became_active":"2023-10-18T10:37:56.952340+0000","last_became_peered":"2023-10-18T10:37:56.952340+0000","last_unstale":"2023-10-18T10:37:59.319849+0000","last_undegraded":"2023-10-18T10:37:59.319849+0000","last_fullsized":"2023-10-18T10:37:59.319849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:35.197029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320105+0000","last_change":"2023-10-18T10:37:57.974667+0000","last_active":"2023-10-18T10:37:59.320105+0000","last_peered":"2023-10-18T10:37:59.320105+0000","last_clean":"2023-10-18T10:37:59.320105+0000","last_became_active":"2023-10-18T10:37:57.974350+0000","last_became_peered":"2023-10-18T10:37:57.974350+0000","last_unstale":"2023-10-18T10:37:59.320105+0000","last_undegraded":"2023-10-18T10:37:59.320105+0000","last_fullsized":"2023-10-18T10:37:59.320105+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:24:59.217657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320075+0000","last_change":"2023-10-18T10:37:56.951625+0000","last_active":"2023-10-18T10:37:59.320075+0000","last_peered":"2023-10-18T10:37:59.320075+0000","last_clean":"2023-10-18T10:37:59.320075+0000","last_became_active":"2023-10-18T10:37:56.951375+0000","last_became_peered":"2023-10-18T10:37:56.951375+0000","last_unstale":"2023-10-18T10:37:59.320075+0000","last_undegraded":"2023-10-18T10:37:59.320075+0000","last_fullsized":"2023-10-18T10:37:59.320075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:15.400131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961537+0000","last_change":"2023-10-18T10:37:57.957530+0000","last_active":"2023-10-18T10:37:58.961537+0000","last_peered":"2023-10-18T10:37:58.961537+0000","last_clean":"2023-10-18T10:37:58.961537+0000","last_became_active":"2023-10-18T10:37:57.957247+0000","last_became_peered":"2023-10-18T10:37:57.957247+0000","last_unstale":"2023-10-18T10:37:58.961537+0000","last_undegraded":"2023-10-18T10:37:58.961537+0000","last_fullsized":"2023-10-18T10:37:58.961537+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:51:51.427813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319890+0000","last_change":"2023-10-18T10:37:56.955873+0000","last_active":"2023-10-18T10:37:59.319890+0000","last_peered":"2023-10-18T10:37:59.319890+0000","last_clean":"2023-10-18T10:37:59.319890+0000","last_became_active":"2023-10-18T10:37:56.955625+0000","last_became_peered":"2023-10-18T10:37:56.955625+0000","last_unstale":"2023-10-18T10:37:59.319890+0000","last_undegraded":"2023-10-18T10:37:59.319890+0000","last_fullsized":"2023-10-18T10:37:59.319890+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:55.749193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320015+0000","last_change":"2023-10-18T10:37:57.972619+0000","last_active":"2023-10-18T10:37:59.320015+0000","last_peered":"2023-10-18T10:37:59.320015+0000","last_clean":"2023-10-18T10:37:59.320015+0000","last_became_active":"2023-10-18T10:37:57.972172+0000","last_became_peered":"2023-10-18T10:37:57.972172+0000","last_unstale":"2023-10-18T10:37:59.320015+0000","last_undegraded":"2023-10-18T10:37:59.320015+0000","last_fullsized":"2023-10-18T10:37:59.320015+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:29.010713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961305+0000","last_change":"2023-10-18T10:37:56.951459+0000","last_active":"2023-10-18T10:37:58.961305+0000","last_peered":"2023-10-18T10:37:58.961305+0000","last_clean":"2023-10-18T10:37:58.961305+0000","last_became_active":"2023-10-18T10:37:56.951186+0000","last_became_peered":"2023-10-18T10:37:56.951186+0000","last_unstale":"2023-10-18T10:37:58.961305+0000","last_undegraded":"2023-10-18T10:37:58.961305+0000","last_fullsized":"2023-10-18T10:37:58.961305+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:34.321126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319796+0000","last_change":"2023-10-18T10:37:57.974277+0000","last_active":"2023-10-18T10:37:59.319796+0000","last_peered":"2023-10-18T10:37:59.319796+0000","last_clean":"2023-10-18T10:37:59.319796+0000","last_became_active":"2023-10-18T10:37:57.972042+0000","last_became_peered":"2023-10-18T10:37:57.972042+0000","last_unstale":"2023-10-18T10:37:59.319796+0000","last_undegraded":"2023-10-18T10:37:59.319796+0000","last_fullsized":"2023-10-18T10:37:59.319796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:11.915134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319483+0000","last_change":"2023-10-18T10:37:56.949089+0000","last_active":"2023-10-18T10:37:59.319483+0000","last_peered":"2023-10-18T10:37:59.319483+0000","last_clean":"2023-10-18T10:37:59.319483+0000","last_became_active":"2023-10-18T10:37:56.948826+0000","last_became_peered":"2023-10-18T10:37:56.948826+0000","last_unstale":"2023-10-18T10:37:59.319483+0000","last_undegraded":"2023-10-18T10:37:59.319483+0000","last_fullsized":"2023-10-18T10:37:59.319483+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:10.729518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320575+0000","last_change":"2023-10-18T10:37:57.964429+0000","last_active":"2023-10-18T10:37:59.320575+0000","last_peered":"2023-10-18T10:37:59.320575+0000","last_clean":"2023-10-18T10:37:59.320575+0000","last_became_active":"2023-10-18T10:37:57.964171+0000","last_became_peered":"2023-10-18T10:37:57.964171+0000","last_unstale":"2023-10-18T10:37:59.320575+0000","last_undegraded":"2023-10-18T10:37:59.320575+0000","last_fullsized":"2023-10-18T10:37:59.320575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:10.675568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320223+0000","last_change":"2023-10-18T10:37:56.955114+0000","last_active":"2023-10-18T10:37:59.320223+0000","last_peered":"2023-10-18T10:37:59.320223+0000","last_clean":"2023-10-18T10:37:59.320223+0000","last_became_active":"2023-10-18T10:37:56.954823+0000","last_became_peered":"2023-10-18T10:37:56.954823+0000","last_unstale":"2023-10-18T10:37:59.320223+0000","last_undegraded":"2023-10-18T10:37:59.320223+0000","last_fullsized":"2023-10-18T10:37:59.320223+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:36.387677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961199+0000","last_change":"2023-10-18T10:37:57.970372+0000","last_active":"2023-10-18T10:37:58.961199+0000","last_peered":"2023-10-18T10:37:58.961199+0000","last_clean":"2023-10-18T10:37:58.961199+0000","last_became_active":"2023-10-18T10:37:57.970088+0000","last_became_peered":"2023-10-18T10:37:57.970088+0000","last_unstale":"2023-10-18T10:37:58.961199+0000","last_undegraded":"2023-10-18T10:37:58.961199+0000","last_fullsized":"2023-10-18T10:37:58.961199+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:59.117648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320614+0000","last_change":"2023-10-18T10:37:56.949701+0000","last_active":"2023-10-18T10:37:59.320614+0000","last_peered":"2023-10-18T10:37:59.320614+0000","last_clean":"2023-10-18T10:37:59.320614+0000","last_became_active":"2023-10-18T10:37:56.949443+0000","last_became_peered":"2023-10-18T10:37:56.949443+0000","last_unstale":"2023-10-18T10:37:59.320614+0000","last_undegraded":"2023-10-18T10:37:59.320614+0000","last_fullsized":"2023-10-18T10:37:59.320614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:01.287718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319600+0000","last_change":"2023-10-18T10:37:57.956703+0000","last_active":"2023-10-18T10:37:59.319600+0000","last_peered":"2023-10-18T10:37:59.319600+0000","last_clean":"2023-10-18T10:37:59.319600+0000","last_became_active":"2023-10-18T10:37:57.956395+0000","last_became_peered":"2023-10-18T10:37:57.956395+0000","last_unstale":"2023-10-18T10:37:59.319600+0000","last_undegraded":"2023-10-18T10:37:59.319600+0000","last_fullsized":"2023-10-18T10:37:59.319600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:14.428640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961447+0000","last_change":"2023-10-18T10:37:56.952611+0000","last_active":"2023-10-18T10:37:58.961447+0000","last_peered":"2023-10-18T10:37:58.961447+0000","last_clean":"2023-10-18T10:37:58.961447+0000","last_became_active":"2023-10-18T10:37:56.952343+0000","last_became_peered":"2023-10-18T10:37:56.952343+0000","last_unstale":"2023-10-18T10:37:58.961447+0000","last_undegraded":"2023-10-18T10:37:58.961447+0000","last_fullsized":"2023-10-18T10:37:58.961447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:41.551944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320431+0000","last_change":"2023-10-18T10:37:57.963394+0000","last_active":"2023-10-18T10:37:59.320431+0000","last_peered":"2023-10-18T10:37:59.320431+0000","last_clean":"2023-10-18T10:37:59.320431+0000","last_became_active":"2023-10-18T10:37:57.963123+0000","last_became_peered":"2023-10-18T10:37:57.963123+0000","last_unstale":"2023-10-18T10:37:59.320431+0000","last_undegraded":"2023-10-18T10:37:59.320431+0000","last_fullsized":"2023-10-18T10:37:59.320431+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:24:59.217657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"64'3","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320197+0000","last_change":"2023-10-18T10:37:56.954070+0000","last_active":"2023-10-18T10:37:59.320197+0000","last_peered":"2023-10-18T10:37:59.320197+0000","last_clean":"2023-10-18T10:37:59.320197+0000","last_became_active":"2023-10-18T10:37:56.953756+0000","last_became_peered":"2023-10-18T10:37:56.953756+0000","last_unstale":"2023-10-18T10:37:59.320197+0000","last_undegraded":"2023-10-18T10:37:59.320197+0000","last_fullsized":"2023-10-18T10:37:59.320197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:15.400131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961533+0000","last_change":"2023-10-18T10:37:57.953033+0000","last_active":"2023-10-18T10:37:58.961533+0000","last_peered":"2023-10-18T10:37:58.961533+0000","last_clean":"2023-10-18T10:37:58.961533+0000","last_became_active":"2023-10-18T10:37:57.952745+0000","last_became_peered":"2023-10-18T10:37:57.952745+0000","last_unstale":"2023-10-18T10:37:58.961533+0000","last_undegraded":"2023-10-18T10:37:58.961533+0000","last_fullsized":"2023-10-18T10:37:58.961533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:33:07.812082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"64'6","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320401+0000","last_change":"2023-10-18T10:37:56.954805+0000","last_active":"2023-10-18T10:37:59.320401+0000","last_peered":"2023-10-18T10:37:59.320401+0000","last_clean":"2023-10-18T10:37:59.320401+0000","last_became_active":"2023-10-18T10:37:56.954488+0000","last_became_peered":"2023-10-18T10:37:56.954488+0000","last_unstale":"2023-10-18T10:37:59.320401+0000","last_undegraded":"2023-10-18T10:37:59.320401+0000","last_fullsized":"2023-10-18T10:37:59.320401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:13.421041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319271+0000","last_change":"2023-10-18T10:37:57.961632+0000","last_active":"2023-10-18T10:37:59.319271+0000","last_peered":"2023-10-18T10:37:59.319271+0000","last_clean":"2023-10-18T10:37:59.319271+0000","last_became_active":"2023-10-18T10:37:57.960353+0000","last_became_peered":"2023-10-18T10:37:57.960353+0000","last_unstale":"2023-10-18T10:37:59.319271+0000","last_undegraded":"2023-10-18T10:37:59.319271+0000","last_fullsized":"2023-10-18T10:37:59.319271+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:00.458354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319982+0000","last_change":"2023-10-18T10:37:56.950895+0000","last_active":"2023-10-18T10:37:59.319982+0000","last_peered":"2023-10-18T10:37:59.319982+0000","last_clean":"2023-10-18T10:37:59.319982+0000","last_became_active":"2023-10-18T10:37:56.950623+0000","last_became_peered":"2023-10-18T10:37:56.950623+0000","last_unstale":"2023-10-18T10:37:59.319982+0000","last_undegraded":"2023-10-18T10:37:59.319982+0000","last_fullsized":"2023-10-18T10:37:59.319982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:18.359944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319668+0000","last_change":"2023-10-18T10:37:57.966150+0000","last_active":"2023-10-18T10:37:59.319668+0000","last_peered":"2023-10-18T10:37:59.319668+0000","last_clean":"2023-10-18T10:37:59.319668+0000","last_became_active":"2023-10-18T10:37:57.965481+0000","last_became_peered":"2023-10-18T10:37:57.965481+0000","last_unstale":"2023-10-18T10:37:59.319668+0000","last_undegraded":"2023-10-18T10:37:59.319668+0000","last_fullsized":"2023-10-18T10:37:59.319668+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:10.675568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320099+0000","last_change":"2023-10-18T10:37:56.952659+0000","last_active":"2023-10-18T10:37:59.320099+0000","last_peered":"2023-10-18T10:37:59.320099+0000","last_clean":"2023-10-18T10:37:59.320099+0000","last_became_active":"2023-10-18T10:37:56.952378+0000","last_became_peered":"2023-10-18T10:37:56.952378+0000","last_unstale":"2023-10-18T10:37:59.320099+0000","last_undegraded":"2023-10-18T10:37:59.320099+0000","last_fullsized":"2023-10-18T10:37:59.320099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:43:26.678464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319684+0000","last_change":"2023-10-18T10:37:57.968546+0000","last_active":"2023-10-18T10:37:59.319684+0000","last_peered":"2023-10-18T10:37:59.319684+0000","last_clean":"2023-10-18T10:37:59.319684+0000","last_became_active":"2023-10-18T10:37:57.968276+0000","last_became_peered":"2023-10-18T10:37:57.968276+0000","last_unstale":"2023-10-18T10:37:59.319684+0000","last_undegraded":"2023-10-18T10:37:59.319684+0000","last_fullsized":"2023-10-18T10:37:59.319684+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:21:19.628260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319681+0000","last_change":"2023-10-18T10:37:56.950945+0000","last_active":"2023-10-18T10:37:59.319681+0000","last_peered":"2023-10-18T10:37:59.319681+0000","last_clean":"2023-10-18T10:37:59.319681+0000","last_became_active":"2023-10-18T10:37:56.950664+0000","last_became_peered":"2023-10-18T10:37:56.950664+0000","last_unstale":"2023-10-18T10:37:59.319681+0000","last_undegraded":"2023-10-18T10:37:59.319681+0000","last_fullsized":"2023-10-18T10:37:59.319681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:18:10.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319941+0000","last_change":"2023-10-18T10:37:57.963804+0000","last_active":"2023-10-18T10:37:59.319941+0000","last_peered":"2023-10-18T10:37:59.319941+0000","last_clean":"2023-10-18T10:37:59.319941+0000","last_became_active":"2023-10-18T10:37:57.963503+0000","last_became_peered":"2023-10-18T10:37:57.963503+0000","last_unstale":"2023-10-18T10:37:59.319941+0000","last_undegraded":"2023-10-18T10:37:59.319941+0000","last_fullsized":"2023-10-18T10:37:59.319941+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:21:52.175815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"64'8","reported_seq":20,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320344+0000","last_change":"2023-10-18T10:37:56.950726+0000","last_active":"2023-10-18T10:37:59.320344+0000","last_peered":"2023-10-18T10:37:59.320344+0000","last_clean":"2023-10-18T10:37:59.320344+0000","last_became_active":"2023-10-18T10:37:56.949932+0000","last_became_peered":"2023-10-18T10:37:56.949932+0000","last_unstale":"2023-10-18T10:37:59.320344+0000","last_undegraded":"2023-10-18T10:37:59.320344+0000","last_fullsized":"2023-10-18T10:37:59.320344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:42:00.882640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320111+0000","last_change":"2023-10-18T10:37:57.969403+0000","last_active":"2023-10-18T10:37:59.320111+0000","last_peered":"2023-10-18T10:37:59.320111+0000","last_clean":"2023-10-18T10:37:59.320111+0000","last_became_active":"2023-10-18T10:37:57.968306+0000","last_became_peered":"2023-10-18T10:37:57.968306+0000","last_unstale":"2023-10-18T10:37:59.320111+0000","last_undegraded":"2023-10-18T10:37:59.320111+0000","last_fullsized":"2023-10-18T10:37:59.320111+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:19.454749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320051+0000","last_change":"2023-10-18T10:37:56.956875+0000","last_active":"2023-10-18T10:37:59.320051+0000","last_peered":"2023-10-18T10:37:59.320051+0000","last_clean":"2023-10-18T10:37:59.320051+0000","last_became_active":"2023-10-18T10:37:56.956636+0000","last_became_peered":"2023-10-18T10:37:56.956636+0000","last_unstale":"2023-10-18T10:37:59.320051+0000","last_undegraded":"2023-10-18T10:37:59.320051+0000","last_fullsized":"2023-10-18T10:37:59.320051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:47:59.450755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319205+0000","last_change":"2023-10-18T10:37:57.956559+0000","last_active":"2023-10-18T10:37:59.319205+0000","last_peered":"2023-10-18T10:37:59.319205+0000","last_clean":"2023-10-18T10:37:59.319205+0000","last_became_active":"2023-10-18T10:37:57.956323+0000","last_became_peered":"2023-10-18T10:37:57.956323+0000","last_unstale":"2023-10-18T10:37:59.319205+0000","last_undegraded":"2023-10-18T10:37:59.319205+0000","last_fullsized":"2023-10-18T10:37:59.319205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:43.446784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.955638+0000","last_change":"2023-10-18T10:37:56.949984+0000","last_active":"2023-10-18T10:37:58.955638+0000","last_peered":"2023-10-18T10:37:58.955638+0000","last_clean":"2023-10-18T10:37:58.955638+0000","last_became_active":"2023-10-18T10:37:56.949745+0000","last_became_peered":"2023-10-18T10:37:56.949745+0000","last_unstale":"2023-10-18T10:37:58.955638+0000","last_undegraded":"2023-10-18T10:37:58.955638+0000","last_fullsized":"2023-10-18T10:37:58.955638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:06.460580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961181+0000","last_change":"2023-10-18T10:37:57.955364+0000","last_active":"2023-10-18T10:37:58.961181+0000","last_peered":"2023-10-18T10:37:58.961181+0000","last_clean":"2023-10-18T10:37:58.961181+0000","last_became_active":"2023-10-18T10:37:57.955091+0000","last_became_peered":"2023-10-18T10:37:57.955091+0000","last_unstale":"2023-10-18T10:37:58.961181+0000","last_undegraded":"2023-10-18T10:37:58.961181+0000","last_fullsized":"2023-10-18T10:37:58.961181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:36.204628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"64'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319756+0000","last_change":"2023-10-18T10:37:56.953753+0000","last_active":"2023-10-18T10:37:59.319756+0000","last_peered":"2023-10-18T10:37:59.319756+0000","last_clean":"2023-10-18T10:37:59.319756+0000","last_became_active":"2023-10-18T10:37:56.953482+0000","last_became_peered":"2023-10-18T10:37:56.953482+0000","last_unstale":"2023-10-18T10:37:59.319756+0000","last_undegraded":"2023-10-18T10:37:59.319756+0000","last_fullsized":"2023-10-18T10:37:59.319756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:02.773566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319280+0000","last_change":"2023-10-18T10:37:57.955254+0000","last_active":"2023-10-18T10:37:59.319280+0000","last_peered":"2023-10-18T10:37:59.319280+0000","last_clean":"2023-10-18T10:37:59.319280+0000","last_became_active":"2023-10-18T10:37:57.954680+0000","last_became_peered":"2023-10-18T10:37:57.954680+0000","last_unstale":"2023-10-18T10:37:59.319280+0000","last_undegraded":"2023-10-18T10:37:59.319280+0000","last_fullsized":"2023-10-18T10:37:59.319280+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:10:56.717467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319972+0000","last_change":"2023-10-18T10:37:56.954521+0000","last_active":"2023-10-18T10:37:59.319972+0000","last_peered":"2023-10-18T10:37:59.319972+0000","last_clean":"2023-10-18T10:37:59.319972+0000","last_became_active":"2023-10-18T10:37:56.954246+0000","last_became_peered":"2023-10-18T10:37:56.954246+0000","last_unstale":"2023-10-18T10:37:59.319972+0000","last_undegraded":"2023-10-18T10:37:59.319972+0000","last_fullsized":"2023-10-18T10:37:59.319972+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:09:00.277886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319902+0000","last_change":"2023-10-18T10:37:57.956951+0000","last_active":"2023-10-18T10:37:59.319902+0000","last_peered":"2023-10-18T10:37:59.319902+0000","last_clean":"2023-10-18T10:37:59.319902+0000","last_became_active":"2023-10-18T10:37:57.956470+0000","last_became_peered":"2023-10-18T10:37:57.956470+0000","last_unstale":"2023-10-18T10:37:59.319902+0000","last_undegraded":"2023-10-18T10:37:59.319902+0000","last_fullsized":"2023-10-18T10:37:59.319902+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:44.808490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319366+0000","last_change":"2023-10-18T10:37:56.951564+0000","last_active":"2023-10-18T10:37:59.319366+0000","last_peered":"2023-10-18T10:37:59.319366+0000","last_clean":"2023-10-18T10:37:59.319366+0000","last_became_active":"2023-10-18T10:37:56.951290+0000","last_became_peered":"2023-10-18T10:37:56.951290+0000","last_unstale":"2023-10-18T10:37:59.319366+0000","last_undegraded":"2023-10-18T10:37:59.319366+0000","last_fullsized":"2023-10-18T10:37:59.319366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:58.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319670+0000","last_change":"2023-10-18T10:37:57.965698+0000","last_active":"2023-10-18T10:37:59.319670+0000","last_peered":"2023-10-18T10:37:59.319670+0000","last_clean":"2023-10-18T10:37:59.319670+0000","last_became_active":"2023-10-18T10:37:57.965414+0000","last_became_peered":"2023-10-18T10:37:57.965414+0000","last_unstale":"2023-10-18T10:37:59.319670+0000","last_undegraded":"2023-10-18T10:37:59.319670+0000","last_fullsized":"2023-10-18T10:37:59.319670+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:19.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319320+0000","last_change":"2023-10-18T10:37:56.953685+0000","last_active":"2023-10-18T10:37:59.319320+0000","last_peered":"2023-10-18T10:37:59.319320+0000","last_clean":"2023-10-18T10:37:59.319320+0000","last_became_active":"2023-10-18T10:37:56.953417+0000","last_became_peered":"2023-10-18T10:37:56.953417+0000","last_unstale":"2023-10-18T10:37:59.319320+0000","last_undegraded":"2023-10-18T10:37:59.319320+0000","last_fullsized":"2023-10-18T10:37:59.319320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:34.220180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319487+0000","last_change":"2023-10-18T10:37:57.962943+0000","last_active":"2023-10-18T10:37:59.319487+0000","last_peered":"2023-10-18T10:37:59.319487+0000","last_clean":"2023-10-18T10:37:59.319487+0000","last_became_active":"2023-10-18T10:37:57.962677+0000","last_became_peered":"2023-10-18T10:37:57.962677+0000","last_unstale":"2023-10-18T10:37:59.319487+0000","last_undegraded":"2023-10-18T10:37:59.319487+0000","last_fullsized":"2023-10-18T10:37:59.319487+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:49:09.644289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319275+0000","last_change":"2023-10-18T10:37:56.950058+0000","last_active":"2023-10-18T10:37:59.319275+0000","last_peered":"2023-10-18T10:37:59.319275+0000","last_clean":"2023-10-18T10:37:59.319275+0000","last_became_active":"2023-10-18T10:37:56.949797+0000","last_became_peered":"2023-10-18T10:37:56.949797+0000","last_unstale":"2023-10-18T10:37:59.319275+0000","last_undegraded":"2023-10-18T10:37:59.319275+0000","last_fullsized":"2023-10-18T10:37:59.319275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:15.400131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319950+0000","last_change":"2023-10-18T10:37:57.971233+0000","last_active":"2023-10-18T10:37:59.319950+0000","last_peered":"2023-10-18T10:37:59.319950+0000","last_clean":"2023-10-18T10:37:59.319950+0000","last_became_active":"2023-10-18T10:37:57.970922+0000","last_became_peered":"2023-10-18T10:37:57.970922+0000","last_unstale":"2023-10-18T10:37:59.319950+0000","last_undegraded":"2023-10-18T10:37:59.319950+0000","last_fullsized":"2023-10-18T10:37:59.319950+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:09:10.864067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.955384+0000","last_change":"2023-10-18T10:37:56.949441+0000","last_active":"2023-10-18T10:37:58.955384+0000","last_peered":"2023-10-18T10:37:58.955384+0000","last_clean":"2023-10-18T10:37:58.955384+0000","last_became_active":"2023-10-18T10:37:56.949178+0000","last_became_peered":"2023-10-18T10:37:56.949178+0000","last_unstale":"2023-10-18T10:37:58.955384+0000","last_undegraded":"2023-10-18T10:37:58.955384+0000","last_fullsized":"2023-10-18T10:37:58.955384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:47.581833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319441+0000","last_change":"2023-10-18T10:37:57.971742+0000","last_active":"2023-10-18T10:37:59.319441+0000","last_peered":"2023-10-18T10:37:59.319441+0000","last_clean":"2023-10-18T10:37:59.319441+0000","last_became_active":"2023-10-18T10:37:57.971478+0000","last_became_peered":"2023-10-18T10:37:57.971478+0000","last_unstale":"2023-10-18T10:37:59.319441+0000","last_undegraded":"2023-10-18T10:37:59.319441+0000","last_fullsized":"2023-10-18T10:37:59.319441+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:24.198102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"64'2","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319796+0000","last_change":"2023-10-18T10:37:56.953279+0000","last_active":"2023-10-18T10:37:59.319796+0000","last_peered":"2023-10-18T10:37:59.319796+0000","last_clean":"2023-10-18T10:37:59.319796+0000","last_became_active":"2023-10-18T10:37:56.953047+0000","last_became_peered":"2023-10-18T10:37:56.953047+0000","last_unstale":"2023-10-18T10:37:59.319796+0000","last_undegraded":"2023-10-18T10:37:59.319796+0000","last_fullsized":"2023-10-18T10:37:59.319796+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:30.885550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319683+0000","last_change":"2023-10-18T10:37:57.953909+0000","last_active":"2023-10-18T10:37:59.319683+0000","last_peered":"2023-10-18T10:37:59.319683+0000","last_clean":"2023-10-18T10:37:59.319683+0000","last_became_active":"2023-10-18T10:37:57.953638+0000","last_became_peered":"2023-10-18T10:37:57.953638+0000","last_unstale":"2023-10-18T10:37:59.319683+0000","last_undegraded":"2023-10-18T10:37:59.319683+0000","last_fullsized":"2023-10-18T10:37:59.319683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:23:10.167209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320289+0000","last_change":"2023-10-18T10:37:56.948678+0000","last_active":"2023-10-18T10:37:59.320289+0000","last_peered":"2023-10-18T10:37:59.320289+0000","last_clean":"2023-10-18T10:37:59.320289+0000","last_became_active":"2023-10-18T10:37:56.948004+0000","last_became_peered":"2023-10-18T10:37:56.948004+0000","last_unstale":"2023-10-18T10:37:59.320289+0000","last_undegraded":"2023-10-18T10:37:59.320289+0000","last_fullsized":"2023-10-18T10:37:59.320289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:17.386951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320220+0000","last_change":"2023-10-18T10:37:57.973958+0000","last_active":"2023-10-18T10:37:59.320220+0000","last_peered":"2023-10-18T10:37:59.320220+0000","last_clean":"2023-10-18T10:37:59.320220+0000","last_became_active":"2023-10-18T10:37:57.968934+0000","last_became_peered":"2023-10-18T10:37:57.968934+0000","last_unstale":"2023-10-18T10:37:59.320220+0000","last_undegraded":"2023-10-18T10:37:59.320220+0000","last_fullsized":"2023-10-18T10:37:59.320220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:42.536234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319810+0000","last_change":"2023-10-18T10:37:56.951389+0000","last_active":"2023-10-18T10:37:59.319810+0000","last_peered":"2023-10-18T10:37:59.319810+0000","last_clean":"2023-10-18T10:37:59.319810+0000","last_became_active":"2023-10-18T10:37:56.951138+0000","last_became_peered":"2023-10-18T10:37:56.951138+0000","last_unstale":"2023-10-18T10:37:59.319810+0000","last_undegraded":"2023-10-18T10:37:59.319810+0000","last_fullsized":"2023-10-18T10:37:59.319810+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:19:18.147109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319721+0000","last_change":"2023-10-18T10:37:57.978061+0000","last_active":"2023-10-18T10:37:59.319721+0000","last_peered":"2023-10-18T10:37:59.319721+0000","last_clean":"2023-10-18T10:37:59.319721+0000","last_became_active":"2023-10-18T10:37:57.977710+0000","last_became_peered":"2023-10-18T10:37:57.977710+0000","last_unstale":"2023-10-18T10:37:59.319721+0000","last_undegraded":"2023-10-18T10:37:59.319721+0000","last_fullsized":"2023-10-18T10:37:59.319721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:11.710841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320188+0000","last_change":"2023-10-18T10:37:56.954250+0000","last_active":"2023-10-18T10:37:59.320188+0000","last_peered":"2023-10-18T10:37:59.320188+0000","last_clean":"2023-10-18T10:37:59.320188+0000","last_became_active":"2023-10-18T10:37:56.953381+0000","last_became_peered":"2023-10-18T10:37:56.953381+0000","last_unstale":"2023-10-18T10:37:59.320188+0000","last_undegraded":"2023-10-18T10:37:59.320188+0000","last_fullsized":"2023-10-18T10:37:59.320188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:13:04.517966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320280+0000","last_change":"2023-10-18T10:37:57.965052+0000","last_active":"2023-10-18T10:37:59.320280+0000","last_peered":"2023-10-18T10:37:59.320280+0000","last_clean":"2023-10-18T10:37:59.320280+0000","last_became_active":"2023-10-18T10:37:57.964747+0000","last_became_peered":"2023-10-18T10:37:57.964747+0000","last_unstale":"2023-10-18T10:37:59.320280+0000","last_undegraded":"2023-10-18T10:37:59.320280+0000","last_fullsized":"2023-10-18T10:37:59.320280+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:49:09.644289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319395+0000","last_change":"2023-10-18T10:37:56.953872+0000","last_active":"2023-10-18T10:37:59.319395+0000","last_peered":"2023-10-18T10:37:59.319395+0000","last_clean":"2023-10-18T10:37:59.319395+0000","last_became_active":"2023-10-18T10:37:56.950129+0000","last_became_peered":"2023-10-18T10:37:56.950129+0000","last_unstale":"2023-10-18T10:37:59.319395+0000","last_undegraded":"2023-10-18T10:37:59.319395+0000","last_fullsized":"2023-10-18T10:37:59.319395+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:23.190504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319368+0000","last_change":"2023-10-18T10:37:57.958790+0000","last_active":"2023-10-18T10:37:59.319368+0000","last_peered":"2023-10-18T10:37:59.319368+0000","last_clean":"2023-10-18T10:37:59.319368+0000","last_became_active":"2023-10-18T10:37:57.958557+0000","last_became_peered":"2023-10-18T10:37:57.958557+0000","last_unstale":"2023-10-18T10:37:59.319368+0000","last_undegraded":"2023-10-18T10:37:59.319368+0000","last_fullsized":"2023-10-18T10:37:59.319368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:29.010713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:58.961193+0000","last_change":"2023-10-18T10:37:56.955078+0000","last_active":"2023-10-18T10:37:58.961193+0000","last_peered":"2023-10-18T10:37:58.961193+0000","last_clean":"2023-10-18T10:37:58.961193+0000","last_became_active":"2023-10-18T10:37:56.954762+0000","last_became_peered":"2023-10-18T10:37:56.954762+0000","last_unstale":"2023-10-18T10:37:58.961193+0000","last_undegraded":"2023-10-18T10:37:58.961193+0000","last_fullsized":"2023-10-18T10:37:58.961193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:18.039701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.320597+0000","last_change":"2023-10-18T10:37:57.975675+0000","last_active":"2023-10-18T10:37:59.320597+0000","last_peered":"2023-10-18T10:37:59.320597+0000","last_clean":"2023-10-18T10:37:59.320597+0000","last_became_active":"2023-10-18T10:37:57.974990+0000","last_became_peered":"2023-10-18T10:37:57.974990+0000","last_unstale":"2023-10-18T10:37:59.320597+0000","last_undegraded":"2023-10-18T10:37:59.320597+0000","last_fullsized":"2023-10-18T10:37:59.320597+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:43.879323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319448+0000","last_change":"2023-10-18T10:37:56.950627+0000","last_active":"2023-10-18T10:37:59.319448+0000","last_peered":"2023-10-18T10:37:59.319448+0000","last_clean":"2023-10-18T10:37:59.319448+0000","last_became_active":"2023-10-18T10:37:56.950306+0000","last_became_peered":"2023-10-18T10:37:56.950306+0000","last_unstale":"2023-10-18T10:37:59.319448+0000","last_undegraded":"2023-10-18T10:37:59.319448+0000","last_fullsized":"2023-10-18T10:37:59.319448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:17.386951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319403+0000","last_change":"2023-10-18T10:37:57.966257+0000","last_active":"2023-10-18T10:37:59.319403+0000","last_peered":"2023-10-18T10:37:59.319403+0000","last_clean":"2023-10-18T10:37:59.319403+0000","last_became_active":"2023-10-18T10:37:57.965520+0000","last_became_peered":"2023-10-18T10:37:57.965520+0000","last_unstale":"2023-10-18T10:37:59.319403+0000","last_undegraded":"2023-10-18T10:37:59.319403+0000","last_fullsized":"2023-10-18T10:37:59.319403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:56.930282+0000","last_clean_scrub_stamp":"2023-10-18T10:37:56.930282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:35.328724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T10:37:59.319563+0000","last_change":"2023-10-18T10:37:56.950713+0000","last_active":"2023-10-18T10:37:59.319563+0000","last_peered":"2023-10-18T10:37:59.319563+0000","last_clean":"2023-10-18T10:37:59.319563+0000","last_became_active":"2023-10-18T10:37:56.950433+0000","last_became_peered":"2023-10-18T10:37:56.950433+0000","last_unstale":"2023-10-18T10:37:59.319563+0000","last_undegraded":"2023-10-18T10:37:59.319563+0000","last_fullsized":"2023-10-18T10:37:59.319563+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:37:55.922684+0000","last_clean_scrub_stamp":"2023-10-18T10:37:55.922684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:45.317821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"53'139","reported_seq":288,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-18T10:37:57.955261+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:37:57.955261+0000","last_peered":"2023-10-18T10:37:57.955261+0000","last_clean":"2023-10-18T10:37:57.955261+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:37:57.955261+0000","last_undegraded":"2023-10-18T10:37:57.955261+0000","last_fullsized":"2023-10-18T10:37:57.955261+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":969833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.27900000000000003,"15min":0.23799999999999999},"max":{"1min":0.77800000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.56499999999999995},"min":{"1min":0.47399999999999998,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.55900000000000005},"min":{"1min":0.46700000000000003,"5min":0.38,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.45900000000000002,"5min":0.34899999999999998,"15min":0.33200000000000002},"max":{"1min":0.80400000000000005,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.625,"15min":0.58699999999999997},"min":{"1min":0.45500000000000002,"5min":0.36699999999999999,"15min":0.29399999999999998},"max":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.57699999999999996},"min":{"1min":0.46300000000000002,"5min":0.40200000000000002,"15min":0.33400000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.026},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60399999999999998,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.78800000000000003,"15min":0.81899999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.61099999999999999},"min":{"1min":0.48399999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.68899999999999995,"5min":0.70499999999999996,"15min":0.76500000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.46000000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26300000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.91500000000000004},"last":0.86199999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.83999999999999997},"last":0.68300000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":1.052},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53700000000000003,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.95399999999999996},"last":0.91700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":379993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.71599999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.46400000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78500000000000003,"5min":0.82399999999999995,"15min":1.016},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.28899999999999998},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.85299999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.29299999999999998},"max":{"1min":0.80900000000000005,"5min":0.81399999999999995,"15min":0.88800000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.67900000000000005,"5min":0.80800000000000005,"15min":0.85699999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.56299999999999994},"min":{"1min":0.503,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.875},"last":0.57899999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.318},"max":{"1min":0.73199999999999998,"5min":0.78700000000000003,"15min":0.82199999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.59299999999999997},"min":{"1min":0.45300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.84399999999999997,"15min":0.91800000000000004},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.437,"15min":0.45000000000000001},"min":{"1min":0.29199999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.70199999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76600000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.62}]},{"osd":5,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45900000000000002,"15min":0.47499999999999998},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70899999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64600000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.436}]},{"osd":7,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48799999999999999,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.66500000000000004,"5min":0.78600000000000003,"15min":0.81100000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56999999999999995,"15min":0.58699999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.87,"15min":0.87},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":379341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.28199999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.57599999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34999999999999998},"max":{"1min":0.73199999999999998,"5min":0.79500000000000004,"15min":0.80500000000000005},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.621,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61499999999999999,"15min":0.59199999999999997},"min":{"1min":0.53000000000000003,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60999999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.46999999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":0.84199999999999997,"15min":0.86399999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":1.0620000000000001},"last":0.57699999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.622,"15min":0.58999999999999997},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":0.83499999999999996,"15min":0.83699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.47399999999999998,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":1.1240000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.61299999999999999,"5min":0.70099999999999996,"15min":1.262},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":0.63600000000000001,"5min":0.78900000000000003,"15min":1.028},"last":0.51100000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47799999999999998,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.32300000000000001,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.68999999999999995,"15min":1.1899999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.35099999999999998,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.80200000000000005,"15min":1.224},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.64800000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69299999999999995,"5min":0.79800000000000004,"15min":1.159},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":379923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.48199999999999998,"5min":0.33100000000000002,"15min":0.26000000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.84299999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.56899999999999995},"min":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.33000000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.67300000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60799999999999998,"15min":0.57299999999999995},"min":{"1min":0.47399999999999998,"5min":0.376,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.81000000000000005,"15min":0.81899999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.58799999999999997},"min":{"1min":0.38500000000000001,"5min":0.372,"15min":0.29499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.59299999999999997},"min":{"1min":0.434,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.86199999999999999},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59499999999999997,"15min":0.56399999999999995},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.313},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.35799999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.32900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79100000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.50700000000000001},"min":{"1min":0.34300000000000003,"5min":0.314,"15min":0.248},"max":{"1min":0.71299999999999997,"5min":0.78200000000000003,"15min":0.79600000000000004},"last":0.373}]},{"osd":6,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.502,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":0.79400000000000004,"15min":0.80700000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.33800000000000002,"15min":0.28399999999999997},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.90400000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29299999999999998},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.84799999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.27100000000000002},"max":{"1min":0.80800000000000005,"5min":0.85899999999999999,"15min":0.879},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":794624,"data_stored":380975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.374,"15min":0.36799999999999999},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.19500000000000001},"max":{"1min":0.46500000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.39400000000000002,"15min":0.378},"min":{"1min":0.254,"5min":0.254,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.57099999999999995,"15min":0.68999999999999995},"last":0.33400000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.248,"5min":0.248,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.70699999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42899999999999999,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.23000000000000001},"max":{"1min":0.624,"5min":0.65600000000000003,"15min":0.84799999999999998},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44500000000000001,"15min":0.41999999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.19900000000000001},"max":{"1min":0.60199999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45600000000000002,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.25600000000000001},"max":{"1min":0.60999999999999999,"5min":0.627,"15min":0.69299999999999995},"last":0.32200000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81999999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63600000000000001,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.504}]},{"osd":5,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.626},"min":{"1min":0.376,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.78300000000000003,"5min":1.002,"15min":1.002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.307},"max":{"1min":0.79300000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.77600000000000002,"5min":0.93700000000000006,"15min":0.96899999999999997},"last":0.442},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.88600000000000001,"15min":1.0349999999999999},"last":0.49099999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.38300000000000001},"max":{"1min":0.78800000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54168,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317672,"statfs":{"total":96636764160,"available":96581296128,"internally_reserved":0,"allocated":790528,"data_stored":380953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38900000000000001,"15min":0.36299999999999999},"min":{"1min":0.28399999999999997,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.219},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.252,"5min":0.192,"15min":0.192},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.63100000000000001},"last":0.318}]},{"osd":1,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40699999999999997,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.20899999999999999},"max":{"1min":0.54700000000000004,"5min":0.61399999999999999,"15min":0.82199999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.39800000000000002},"min":{"1min":0.28999999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.26100000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40699999999999997,"15min":0.38300000000000001},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.55100000000000005,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46400000000000002,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.34100000000000003,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.33800000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.63300000000000001},"min":{"1min":0.38500000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.81999999999999995,"5min":0.88800000000000001,"15min":1.458},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67300000000000004,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64400000000000002,"15min":0.61099999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91400000000000003,"5min":0.95699999999999996,"15min":0.997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63800000000000001,"15min":0.60299999999999998},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82499999999999996,"5min":0.93600000000000005,"15min":0.96699999999999997},"last":0.498},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.63500000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.84799999999999998,"5min":0.93700000000000006,"15min":1.028},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:37:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66900000000000004,"15min":0.63},"min":{"1min":0.41699999999999998,"5min":0.373,"15min":0.36399999999999999},"max":{"1min":0.81999999999999995,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.879,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":379501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.379,"15min":0.35599999999999998},"min":{"1min":0.23200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56200000000000006,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.372},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.19800000000000001},"max":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.63300000000000001},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40899999999999997,"15min":0.38},"min":{"1min":0.28000000000000003,"5min":0.25,"15min":0.186},"max":{"1min":0.54800000000000004,"5min":0.68899999999999995,"15min":0.70799999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.23999999999999999},"max":{"1min":0.56100000000000005,"5min":0.64800000000000002,"15min":0.747},"last":0.496}]},{"osd":3,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.438,"15min":0.40699999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.61499999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.28199999999999997,"15min":0.25600000000000001},"max":{"1min":0.65100000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.57499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.60399999999999998},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.34100000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.753},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.61099999999999999},"min":{"1min":0.53400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.86599999999999999,"15min":0.92700000000000005},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.47399999999999998,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.90700000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.745},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.61199999999999999},"min":{"1min":0.47099999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65100000000000002,"15min":0.624},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89100000000000001,"5min":0.92000000000000004,"15min":0.94299999999999995},"last":0.69299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:37:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64800000000000002,"15min":0.63},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92400000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.371},"max":{"1min":0.93200000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1380352,"data_stored":969765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.37},"min":{"1min":0.22700000000000001,"5min":0.224,"15min":0.192},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.71099999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.25900000000000001,"15min":0.17799999999999999},"max":{"1min":0.56299999999999994,"5min":0.61099999999999999,"15min":0.76900000000000002},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.39200000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.187},"max":{"1min":0.60399999999999998,"5min":0.63400000000000001,"15min":0.96099999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42299999999999999,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.23999999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.93700000000000006},"last":0.36499999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39700000000000002,"15min":0.38900000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.222},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.64400000000000002},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.45000000000000001,"15min":0.442},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.22800000000000001},"max":{"1min":0.54500000000000004,"5min":0.65500000000000003,"15min":0.91100000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.60499999999999998},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.371},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63500000000000001,"15min":0.61499999999999999},"min":{"1min":0.45300000000000001,"5min":0.40200000000000002,"15min":0.35699999999999998},"max":{"1min":0.92700000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.46100000000000002,"5min":0.45800000000000002,"15min":0.34699999999999998},"max":{"1min":0.92900000000000005,"5min":0.96199999999999997,"15min":0.96799999999999997},"last":0.58799999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.41299999999999998,"15min":0.371},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.46899999999999997,"5min":0.45200000000000001,"15min":0.41099999999999998},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66100000000000003,"15min":0.64800000000000002},"min":{"1min":0.498,"5min":0.432,"15min":0.41999999999999998},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.48199999999999998,"5min":0.44600000000000001,"15min":0.33200000000000002},"max":{"1min":0.88800000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:38:01.616 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:38:01.617 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:38:01.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.758+0000 7f3787905640 1 -- 172.21.15.125:0/2484833659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 msgr2=0x7f37800b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.758+0000 7f3787905640 1 --2- 172.21.15.125:0/2484833659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 0x7f37800b09c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3778005fd0 tx=0x7f377802c2e0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.758+0000 7f3787905640 1 -- 172.21.15.125:0/2484833659 shutdown_connections 2023-10-18T10:38:01.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.758+0000 7f3787905640 1 --2- 172.21.15.125:0/2484833659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 0x7f37800b09c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.758+0000 7f3787905640 1 --2- 172.21.15.125:0/2484833659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37800cbd20 0x7f3780158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.758+0000 7f3787905640 1 -- 172.21.15.125:0/2484833659 >> 172.21.15.125:0/2484833659 conn(0x7f3780069930 msgr2=0x7f3780069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:01.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.759+0000 7f3787905640 1 -- 172.21.15.125:0/2484833659 shutdown_connections 2023-10-18T10:38:01.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.759+0000 7f3787905640 1 -- 172.21.15.125:0/2484833659 wait complete. 2023-10-18T10:38:01.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3787905640 1 Processor -- start 2023-10-18T10:38:01.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3787905640 1 -- start start 2023-10-18T10:38:01.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3787905640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37800afa20 0x7f3780134160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3787905640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 0x7f3780142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3787905640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37800cbd20 0x7f3780143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3787905640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37801346a0 con 0x7f37800b0530 2023-10-18T10:38:01.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3787905640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37801464a0 con 0x7f37800cbd20 2023-10-18T10:38:01.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3787905640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3780158980 con 0x7f37800afa20 2023-10-18T10:38:01.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3784ea0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 0x7f3780142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3784ea0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 0x7f3780142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55482/0 (socket says 172.21.15.125:55482) 2023-10-18T10:38:01.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.760+0000 7f3784ea0640 1 -- 172.21.15.125:0/950249626 learned_addr learned my addr 172.21.15.125:0/950249626 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:01.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f37856a1640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37800afa20 0x7f3780134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3785ea2640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37800cbd20 0x7f3780143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3784ea0640 1 -- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37800afa20 msgr2=0x7f3780134160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3784ea0640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37800afa20 0x7f3780134160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3784ea0640 1 -- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37800cbd20 msgr2=0x7f3780143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:01.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3784ea0640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37800cbd20 0x7f3780143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:01.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3784ea0640 1 -- 172.21.15.125:0/950249626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3778009900 con 0x7f37800b0530 2023-10-18T10:38:01.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f37856a1640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37800afa20 0x7f3780134160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:38:01.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3784ea0640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 0x7f3780142d30 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f377000c450 tx=0x7f377000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:01.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3775ffb640 1 -- 172.21.15.125:0/950249626 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37700052d0 con 0x7f37800b0530 2023-10-18T10:38:01.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3775ffb640 1 -- 172.21.15.125:0/950249626 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3770005430 con 0x7f37800b0530 2023-10-18T10:38:01.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37801466f0 con 0x7f37800b0530 2023-10-18T10:38:01.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3775ffb640 1 -- 172.21.15.125:0/950249626 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3770012890 con 0x7f37800b0530 2023-10-18T10:38:01.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.761+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3780146cb0 con 0x7f37800b0530 2023-10-18T10:38:01.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.762+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37800690e0 con 0x7f37800b0530 2023-10-18T10:38:01.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.763+0000 7f3775ffb640 1 -- 172.21.15.125:0/950249626 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f37700035f0 con 0x7f37800b0530 2023-10-18T10:38:01.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.763+0000 7f3775ffb640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f376406af60 0x7f376406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:01.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.763+0000 7f3775ffb640 1 -- 172.21.15.125:0/950249626 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f37700895c0 con 0x7f37800b0530 2023-10-18T10:38:01.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.763+0000 7f37856a1640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f376406af60 0x7f376406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:01.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.764+0000 7f37856a1640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f376406af60 0x7f376406d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f377c00bbc0 tx=0x7f377c005ef0 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:01.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.767+0000 7f3775ffb640 1 -- 172.21.15.125:0/950249626 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3770054420 con 0x7f37800b0530 2023-10-18T10:38:01.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.996+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3780151740 con 0x7f37800b0530 2023-10-18T10:38:02.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:01.998+0000 7f3775ffb640 1 -- 172.21.15.125:0/950249626 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3770057a40 con 0x7f37800b0530 2023-10-18T10:38:02.000 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:38:02.000 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:38:02.001 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T10:38:02.001 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T10:38:02.001 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:38:02.002 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T10:38:02.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f376406af60 msgr2=0x7f376406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f376406af60 0x7f376406d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f377c00bbc0 tx=0x7f377c005ef0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 msgr2=0x7f3780142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 0x7f3780142d30 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f377000c450 tx=0x7f377000c910 comp rx=0 tx=0).stop 2023-10-18T10:38:02.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 shutdown_connections 2023-10-18T10:38:02.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f376406af60 0x7f376406d420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37800cbd20 0x7f3780143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37800b0530 0x7f3780142d30 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 --2- 172.21.15.125:0/950249626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37800afa20 0x7f3780134160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 >> 172.21.15.125:0/950249626 conn(0x7f3780069930 msgr2=0x7f3780069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 shutdown_connections 2023-10-18T10:38:02.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.000+0000 7f3787905640 1 -- 172.21.15.125:0/950249626 wait complete. 2023-10-18T10:38:02.013 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:38:02.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.115+0000 7f7caca90640 1 -- 172.21.15.125:0/3575884417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 msgr2=0x7f7c90004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.115+0000 7f7caca90640 1 --2- 172.21.15.125:0/3575884417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 0x7f7c90004c40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c006fd0 tx=0x7f7c900057f0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.115+0000 7f7ca6d9d640 1 -- 172.21.15.125:0/3575884417 reap_dead start 2023-10-18T10:38:02.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.115+0000 7f7caca90640 1 -- 172.21.15.125:0/3575884417 shutdown_connections 2023-10-18T10:38:02.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.115+0000 7f7caca90640 1 -- 172.21.15.125:0/3575884417 >> 172.21.15.125:0/3575884417 conn(0x7f7ca80d0d60 msgr2=0x7f7ca80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.115+0000 7f7caca90640 1 -- 172.21.15.125:0/3575884417 shutdown_connections 2023-10-18T10:38:02.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.115+0000 7f7caca90640 1 -- 172.21.15.125:0/3575884417 wait complete. 2023-10-18T10:38:02.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.116+0000 7f7caca90640 1 Processor -- start 2023-10-18T10:38:02.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.116+0000 7f7caca90640 1 -- start start 2023-10-18T10:38:02.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7caca90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 0x7f7ca815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7caca90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca81f43b0 0x7f7ca815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7caca90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ca806b110 0x7f7ca806b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7caca90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca806bad0 con 0x7f7c900047c0 2023-10-18T10:38:02.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7caca90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca815cc50 con 0x7f7ca81f43b0 2023-10-18T10:38:02.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7caca90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca815cf30 con 0x7f7ca806b110 2023-10-18T10:38:02.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 0x7f7ca815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 0x7f7ca815c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55510/0 (socket says 172.21.15.125:55510) 2023-10-18T10:38:02.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 -- 172.21.15.125:0/1344409648 learned_addr learned my addr 172.21.15.125:0/1344409648 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:02.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca6d9d640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ca806b110 0x7f7ca806b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca5d9b640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca81f43b0 0x7f7ca815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 -- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ca806b110 msgr2=0x7f7ca806b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ca806b110 0x7f7ca806b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 -- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca81f43b0 msgr2=0x7f7ca815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca81f43b0 0x7f7ca815c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 -- 172.21.15.125:0/1344409648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca815d210 con 0x7f7c900047c0 2023-10-18T10:38:02.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7ca659c640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 0x7f7ca815c0e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7c98003fd0 tx=0x7f7c98001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:02.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7c96ffd640 1 -- 172.21.15.125:0/1344409648 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98003b20 con 0x7f7c900047c0 2023-10-18T10:38:02.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7c96ffd640 1 -- 172.21.15.125:0/1344409648 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7c9800fa60 con 0x7f7c900047c0 2023-10-18T10:38:02.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7c96ffd640 1 -- 172.21.15.125:0/1344409648 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800fe50 con 0x7f7c900047c0 2023-10-18T10:38:02.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.117+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca815d4f0 con 0x7f7c900047c0 2023-10-18T10:38:02.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.118+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca80ca090 con 0x7f7c900047c0 2023-10-18T10:38:02.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.118+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca81f12b0 con 0x7f7c900047c0 2023-10-18T10:38:02.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.119+0000 7f7c96ffd640 1 -- 172.21.15.125:0/1344409648 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7c98011430 con 0x7f7c900047c0 2023-10-18T10:38:02.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.119+0000 7f7c96ffd640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7c8806af90 0x7f7c8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.119+0000 7f7c96ffd640 1 -- 172.21.15.125:0/1344409648 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7c9808a940 con 0x7f7c900047c0 2023-10-18T10:38:02.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.120+0000 7f7ca5d9b640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7c8806af90 0x7f7c8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.120+0000 7f7ca5d9b640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7c8806af90 0x7f7c8806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c034000 tx=0x7f7c9c02e000 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:02.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.123+0000 7f7c96ffd640 1 -- 172.21.15.125:0/1344409648 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ca81f12b0 con 0x7f7c900047c0 2023-10-18T10:38:02.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.324+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ca81f12b0 con 0x7f7c900047c0 2023-10-18T10:38:02.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.325+0000 7f7c96ffd640 1 -- 172.21.15.125:0/1344409648 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f7ca81f12b0 con 0x7f7c900047c0 2023-10-18T10:38:02.327 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:38:02.327 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1045257948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1045257948},{"type":"v1","addr":"172.21.15.139:6836","nonce":1045257948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/4183044691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4183044691},{"type":"v1","addr":"172.21.15.139:6837","nonce":4183044691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1083462482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1083462482},{"type":"v1","addr":"172.21.15.125:6835","nonce":1083462482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:37:57.277572+0000","modified":"2023-10-18T10:37:59.669781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/615856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":615856560},{"type":"v1","addr":"172.21.15.125:6834","nonce":615856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T10:38:02.328 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 23 2023-10-18T10:38:02.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7c8806af90 msgr2=0x7f7c8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7c8806af90 0x7f7c8806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c034000 tx=0x7f7c9c02e000 comp rx=0 tx=0).stop 2023-10-18T10:38:02.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 msgr2=0x7f7ca815c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 0x7f7ca815c0e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7c98003fd0 tx=0x7f7c98001cf0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 shutdown_connections 2023-10-18T10:38:02.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f7c8806af90 0x7f7c8806d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ca81f43b0 0x7f7ca815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ca806b110 0x7f7ca806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 --2- 172.21.15.125:0/1344409648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c900047c0 0x7f7ca815c0e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.326+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 >> 172.21.15.125:0/1344409648 conn(0x7f7ca80d0d60 msgr2=0x7f7ca80b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.327+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 shutdown_connections 2023-10-18T10:38:02.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.327+0000 7f7caca90640 1 -- 172.21.15.125:0/1344409648 wait complete. 2023-10-18T10:38:02.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:37:57.277572+0000', 'modified': '2023-10-18T10:37:59.669781+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6591}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6591': {'gid': 6591, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6834/615856560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6832', 'nonce': 615856560}, {'type': 'v1', 'addr': '172.21.15.125:6834', 'nonce': 615856560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:38:02.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:38:02.744 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:38:02.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.848+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3043386266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ad0004d00 msgr2=0x7f9ad0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.848+0000 7f9aeb4b4640 1 --2- 172.21.15.125:0/3043386266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ad0004d00 0x7f9ad0005180 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009f00 tx=0x7f9ad00057f0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.848+0000 7f9ae9a51640 1 -- 172.21.15.125:0/3043386266 reap_dead start 2023-10-18T10:38:02.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.848+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3043386266 shutdown_connections 2023-10-18T10:38:02.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.848+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3043386266 >> 172.21.15.125:0/3043386266 conn(0x7f9ae4069930 msgr2=0x7f9ae4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.848+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3043386266 shutdown_connections 2023-10-18T10:38:02.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.848+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3043386266 wait complete. 2023-10-18T10:38:02.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.849+0000 7f9aeb4b4640 1 Processor -- start 2023-10-18T10:38:02.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.849+0000 7f9aeb4b4640 1 -- start start 2023-10-18T10:38:02.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9aeb4b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad0004d00 0x7f9ae414fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9aeb4b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ae413a230 0x7f9ae40b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9aeb4b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ae40b0d30 0x7f9ae4150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9aeb4b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40b11b0 con 0x7f9ae413a230 2023-10-18T10:38:02.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9aeb4b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae4150170 con 0x7f9ad0004d00 2023-10-18T10:38:02.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9aeb4b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae4151b60 con 0x7f9ae40b0d30 2023-10-18T10:38:02.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ae413a230 0x7f9ae40b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ae413a230 0x7f9ae40b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55532/0 (socket says 172.21.15.125:55532) 2023-10-18T10:38:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 -- 172.21.15.125:0/3807940681 learned_addr learned my addr 172.21.15.125:0/3807940681 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae9250640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad0004d00 0x7f9ae414fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ae40b0d30 0x7f9ae4150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 -- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ae40b0d30 msgr2=0x7f9ae4150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ae40b0d30 0x7f9ae4150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 -- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad0004d00 msgr2=0x7f9ae414fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad0004d00 0x7f9ae414fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 -- 172.21.15.125:0/3807940681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4159a20 con 0x7f9ae413a230 2023-10-18T10:38:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae9250640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad0004d00 0x7f9ae414fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:38:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ae413a230 0x7f9ae40b07f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0011a50 tx=0x7f9ae0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ad9ffb640 1 -- 172.21.15.125:0/3807940681 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000fc70 con 0x7f9ae413a230 2023-10-18T10:38:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ad9ffb640 1 -- 172.21.15.125:0/3807940681 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ae000fec0 con 0x7f9ae413a230 2023-10-18T10:38:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3807940681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae40a64b0 con 0x7f9ae413a230 2023-10-18T10:38:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.850+0000 7f9ad9ffb640 1 -- 172.21.15.125:0/3807940681 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0012690 con 0x7f9ae413a230 2023-10-18T10:38:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.851+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3807940681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae40a6a70 con 0x7f9ae413a230 2023-10-18T10:38:02.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.852+0000 7f9ad9ffb640 1 -- 172.21.15.125:0/3807940681 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9ae00127f0 con 0x7f9ae413a230 2023-10-18T10:38:02.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.852+0000 7f9ad9ffb640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f9ac806af90 0x7f9ac806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.852+0000 7f9ad9ffb640 1 -- 172.21.15.125:0/3807940681 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9ae008a6c0 con 0x7f9ae413a230 2023-10-18T10:38:02.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.853+0000 7f9ae9250640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f9ac806af90 0x7f9ac806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.853+0000 7f9ae9250640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f9ac806af90 0x7f9ac806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4006cb0 tx=0x7f9ad4006720 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.853+0000 7f9aee046740 1 Processor -- start 2023-10-18T10:38:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.853+0000 7f9aee046740 1 -- start start 2023-10-18T10:38:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9aee046740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x55747715eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9aee046740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55747718bf20 0x55747718e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9aee046740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747715f020 0x55747718ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9aee046740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557477070730 con 0x55747715f020 2023-10-18T10:38:02.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9aee046740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574770d1780 con 0x55747718bb40 2023-10-18T10:38:02.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9aee046740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574770cee70 con 0x55747718bf20 2023-10-18T10:38:02.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747715f020 0x55747718ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747715f020 0x55747718ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55544/0 (socket says 172.21.15.125:55544) 2023-10-18T10:38:02.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 -- 172.21.15.125:0/789466687 learned_addr learned my addr 172.21.15.125:0/789466687 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:02.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9250640 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x55747715eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55747718bf20 0x55747718e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 -- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55747718bf20 msgr2=0x55747718e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:38:02.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55747718bf20 0x55747718e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 -- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 msgr2=0x55747715eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:38:02.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x55747715eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 -- 172.21.15.125:0/789466687 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574770cf3d0 con 0x55747715f020 2023-10-18T10:38:02.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9250640 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x55747715eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:38:02.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55747718bf20 0x55747718e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:38:02.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.854+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747715f020 0x55747718ee10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9adc005fd0 tx=0x7f9adc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdf410136d01265 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:02.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aceffd640 1 -- 172.21.15.125:0/789466687 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc03d040 con 0x55747715f020 2023-10-18T10:38:02.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aceffd640 1 -- 172.21.15.125:0/789466687 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9adc0309a0 con 0x55747715f020 2023-10-18T10:38:02.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aceffd640 1 -- 172.21.15.125:0/789466687 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0306d0 con 0x55747715f020 2023-10-18T10:38:02.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 -- 172.21.15.125:0/789466687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747715f020 msgr2=0x55747718ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747715f020 0x55747718ee10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9adc005fd0 tx=0x7f9adc004a40 comp rx=0 tx=0).stop 2023-10-18T10:38:02.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 -- 172.21.15.125:0/789466687 shutdown_connections 2023-10-18T10:38:02.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747715f020 0x55747718ee10 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55747718bf20 0x55747718e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 --2- 172.21.15.125:0/789466687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x55747715eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 -- 172.21.15.125:0/789466687 >> 172.21.15.125:0/789466687 conn(0x557477160d50 msgr2=0x5574771a98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 -- 172.21.15.125:0/789466687 shutdown_connections 2023-10-18T10:38:02.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.855+0000 7f9aee046740 1 -- 172.21.15.125:0/789466687 wait complete. 2023-10-18T10:38:02.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.856+0000 7f9aee046740 1 -- start start 2023-10-18T10:38:02.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.856+0000 7f9aee046740 1 Processor -- start 2023-10-18T10:38:02.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9aee046740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771aa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9aee046740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 0x557477177e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9aee046740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x5574771787c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9aee046740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574770ca760 con 0x55747718bb40 2023-10-18T10:38:02.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9aee046740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574770858f0 con 0x55747718bf20 2023-10-18T10:38:02.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9aee046740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x557477085590 con 0x557477178340 2023-10-18T10:38:02.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9aee046740 10 client.0 _ll_register_callbacks cb 0x5574770c1ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:38:02.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771aa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55550/0 (socket says 172.21.15.125:55550) 2023-10-18T10:38:02.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 -- 172.21.15.125:0/2854350338 learned_addr learned my addr 172.21.15.125:0/2854350338 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:02.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 0x557477177e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x5574771787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 -- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 msgr2=0x5574771787c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:38:02.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x5574771787c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 -- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 msgr2=0x557477177e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:38:02.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 0x557477177e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 -- 172.21.15.125:0/2854350338 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad0003520 con 0x55747718bb40 2023-10-18T10:38:02.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x5574771787c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:38:02.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ae9250640 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771aa160 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4003ea0 tx=0x7f9ad4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8b911d419d2aa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:02.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ac3fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:38:02.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ac3fff640 1 -- 172.21.15.125:0/2854350338 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad401b610 con 0x55747718bb40 2023-10-18T10:38:02.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.857+0000 7f9ac3fff640 1 -- 172.21.15.125:0/2854350338 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9ad401b770 con 0x55747718bb40 2023-10-18T10:38:02.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 1 -- 172.21.15.125:0/2854350338 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574770c3740 con 0x55747718bb40 2023-10-18T10:38:02.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9ac3fff640 1 -- 172.21.15.125:0/2854350338 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ad401b930 con 0x55747718bb40 2023-10-18T10:38:02.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 10 client.6714 fetch_fsmap learned FSMap version 23 2023-10-18T10:38:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 1 -- 172.21.15.125:0/2854350338 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5574770c8de0 con 0x55747718bb40 2023-10-18T10:38:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9ac3fff640 1 -- 172.21.15.125:0/2854350338 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5574770c3740 con 0x55747718bb40 2023-10-18T10:38:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9ac3fff640 1 -- 172.21.15.125:0/2854350338 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9ad401ba90 con 0x55747718bb40 2023-10-18T10:38:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 10 client.6714 fetch_fsmap finished waiting for FSMap version 23 2023-10-18T10:38:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 10 client.6714 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:38:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 20 client.6714 populate_metadata read hostname 'smithi125' 2023-10-18T10:38:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] conn(0x5574771bcfd0 0x5574771bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 4 client.6714 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:38:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9aee046740 1 -- 172.21.15.125:0/2854350338 --> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574770c3460 con 0x5574771bcfd0 2023-10-18T10:38:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] conn(0x5574771bcfd0 0x5574771bd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] conn(0x5574771bcfd0 0x5574771bd3b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1a46eacfe51f43 server_cookie=d343355075ba4c4 in_seq=0 out_seq=0 2023-10-18T10:38:02.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.858+0000 7f9ac3fff640 10 client.6714 ms_handle_connect on v2:172.21.15.125:6832/615856560 2023-10-18T10:38:02.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9ac3fff640 1 -- 172.21.15.125:0/2854350338 <== mds.0 v2:172.21.15.125:6832/615856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5574770c3460 con 0x5574771bcfd0 2023-10-18T10:38:02.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9ac3fff640 10 client.6714 handle_command_reply: tid=0 2023-10-18T10:38:02.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 client.6714 shutdown 2023-10-18T10:38:02.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 -- 172.21.15.125:0/2854350338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 msgr2=0x5574771aa160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771aa160 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4003ea0 tx=0x7f9ad4001a00 comp rx=0 tx=0).stop 2023-10-18T10:38:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 -- 172.21.15.125:0/2854350338 shutdown_connections 2023-10-18T10:38:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] conn(0x5574771bcfd0 0x5574771bd3b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 0x557477177e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x5574771787c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 --2- 172.21.15.125:0/2854350338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771aa160 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9ac3fff640 0 client.6714 ms_handle_reset on v2:172.21.15.125:6832/615856560 2023-10-18T10:38:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 -- 172.21.15.125:0/2854350338 >> 172.21.15.125:0/2854350338 conn(0x55747715f020 msgr2=0x55747718b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 -- 172.21.15.125:0/2854350338 shutdown_connections 2023-10-18T10:38:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 1 -- 172.21.15.125:0/2854350338 wait complete. 2023-10-18T10:38:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.859+0000 7f9aee046740 20 client.6714 trim_cache size 0 max 16384 2023-10-18T10:38:02.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.874+0000 7f9aee046740 1 Processor -- start 2023-10-18T10:38:02.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.874+0000 7f9aee046740 1 -- start start 2023-10-18T10:38:02.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9aee046740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x55747718c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9aee046740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9aee046740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 0x5574771ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9aee046740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747704b270 con 0x55747718bb40 2023-10-18T10:38:02.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9aee046740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x557477174e30 con 0x55747718bf20 2023-10-18T10:38:02.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9aee046740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x557477162e40 con 0x557477178340 2023-10-18T10:38:02.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771768f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55564/0 (socket says 172.21.15.125:55564) 2023-10-18T10:38:02.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 -- 172.21.15.125:0/4265220431 learned_addr learned my addr 172.21.15.125:0/4265220431 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:02.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 -- 172.21.15.125:0/4265220431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 msgr2=0x55747718c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:38:02.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/4265220431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x55747718c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 -- 172.21.15.125:0/4265220431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 msgr2=0x5574771ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:38:02.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/4265220431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 0x5574771ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 -- 172.21.15.125:0/4265220431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557477067d00 con 0x55747718bb40 2023-10-18T10:38:02.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.875+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/4265220431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771768f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0088990 tx=0x7f9ae0026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdaeca5746479ef server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:02.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9acf7fe640 1 -- 172.21.15.125:0/4265220431 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0088aa0 con 0x55747718bb40 2023-10-18T10:38:02.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9acf7fe640 1 -- 172.21.15.125:0/4265220431 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ae00559a0 con 0x55747718bb40 2023-10-18T10:38:02.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9acf7fe640 1 -- 172.21.15.125:0/4265220431 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00589e0 con 0x55747718bb40 2023-10-18T10:38:02.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 -- 172.21.15.125:0/4265220431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 msgr2=0x5574771768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 --2- 172.21.15.125:0/4265220431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771768f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0088990 tx=0x7f9ae0026040 comp rx=0 tx=0).stop 2023-10-18T10:38:02.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 -- 172.21.15.125:0/4265220431 shutdown_connections 2023-10-18T10:38:02.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 --2- 172.21.15.125:0/4265220431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bf20 0x5574771ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 --2- 172.21.15.125:0/4265220431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bb40 0x5574771768f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 --2- 172.21.15.125:0/4265220431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x55747718c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 -- 172.21.15.125:0/4265220431 >> 172.21.15.125:0/4265220431 conn(0x55747715f020 msgr2=0x5574771626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 -- 172.21.15.125:0/4265220431 shutdown_connections 2023-10-18T10:38:02.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.876+0000 7f9aee046740 1 -- 172.21.15.125:0/4265220431 wait complete. 2023-10-18T10:38:02.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.877+0000 7f9aee046740 1 -- start start 2023-10-18T10:38:02.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.877+0000 7f9aee046740 1 Processor -- start 2023-10-18T10:38:02.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.877+0000 7f9aee046740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x557477212420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x557477189710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bf20 0x557477163800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557477194290 con 0x55747718bf20 2023-10-18T10:38:02.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x557477189e60 con 0x55747718bb40 2023-10-18T10:38:02.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x557477194040 con 0x557477178340 2023-10-18T10:38:02.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 10 client.0 _ll_register_callbacks cb 0x7f9ad00036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:38:02.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bf20 0x557477163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bf20 0x557477163800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55566/0 (socket says 172.21.15.125:55566) 2023-10-18T10:38:02.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 -- 172.21.15.125:0/4243797488 learned_addr learned my addr 172.21.15.125:0/4243797488 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:02.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9250640 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x557477212420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x557477189710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 -- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 msgr2=0x557477212420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:38:02.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x557477212420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 -- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 msgr2=0x557477189710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:38:02.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x557477189710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 -- 172.21.15.125:0/4243797488 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae40a6a70 con 0x55747718bf20 2023-10-18T10:38:02.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae8a4f640 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x557477189710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:38:02.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ae9a51640 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bf20 0x557477163800 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9adc02cf40 tx=0x7f9adc02d100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96a1885a33b183d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:02.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ac3fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:38:02.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ac3fff640 1 -- 172.21.15.125:0/4243797488 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc03d070 con 0x55747718bf20 2023-10-18T10:38:02.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ac3fff640 1 -- 172.21.15.125:0/4243797488 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9adc030d10 con 0x55747718bf20 2023-10-18T10:38:02.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 1 -- 172.21.15.125:0/4243797488 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9adc0309a0 con 0x55747718bf20 2023-10-18T10:38:02.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ac3fff640 1 -- 172.21.15.125:0/4243797488 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9adc003970 con 0x55747718bf20 2023-10-18T10:38:02.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 10 client.6720 fetch_fsmap learned FSMap version 23 2023-10-18T10:38:02.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9aee046740 1 -- 172.21.15.125:0/4243797488 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55747708d4f0 con 0x55747718bf20 2023-10-18T10:38:02.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ac3fff640 1 -- 172.21.15.125:0/4243797488 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9adc0309a0 con 0x55747718bf20 2023-10-18T10:38:02.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.878+0000 7f9ac3fff640 1 -- 172.21.15.125:0/4243797488 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9adc036020 con 0x55747718bf20 2023-10-18T10:38:02.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 10 client.6720 fetch_fsmap finished waiting for FSMap version 23 2023-10-18T10:38:02.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 10 client.6720 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:38:02.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 20 client.6720 populate_metadata read hostname 'smithi125' 2023-10-18T10:38:02.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] conn(0x5574771ab1c0 0x5574771ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:02.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 4 client.6720 mds_command: new command op to 6591 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:38:02.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 -- 172.21.15.125:0/4243797488 --> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557476dcc6c0 con 0x5574771ab1c0 2023-10-18T10:38:02.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9ae9250640 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] conn(0x5574771ab1c0 0x5574771ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:02.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9ac3fff640 10 client.6720 ms_handle_connect on v2:172.21.15.125:6832/615856560 2023-10-18T10:38:02.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9ae9250640 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] conn(0x5574771ab1c0 0x5574771ab5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9fdc4bcff0a5131 server_cookie=fd58e6edb9e4afb in_seq=0 out_seq=0 2023-10-18T10:38:02.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9ac3fff640 1 -- 172.21.15.125:0/4243797488 <== mds.0 v2:172.21.15.125:6832/615856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x557476dcc6c0 con 0x5574771ab1c0 2023-10-18T10:38:02.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9ac3fff640 10 client.6720 handle_command_reply: tid=0 2023-10-18T10:38:02.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 client.6720 shutdown 2023-10-18T10:38:02.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 -- 172.21.15.125:0/4243797488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bf20 msgr2=0x557477163800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bf20 0x557477163800 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9adc02cf40 tx=0x7f9adc02d100 comp rx=0 tx=0).stop 2023-10-18T10:38:02.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 -- 172.21.15.125:0/4243797488 shutdown_connections 2023-10-18T10:38:02.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.125:6832/615856560,v1:172.21.15.125:6834/615856560] conn(0x5574771ab1c0 0x5574771ab5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55747718bb40 0x557477189710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9ac3fff640 0 client.6720 ms_handle_reset on v2:172.21.15.125:6832/615856560 2023-10-18T10:38:02.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55747718bf20 0x557477163800 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 --2- 172.21.15.125:0/4243797488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x557477178340 0x557477212420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 -- 172.21.15.125:0/4243797488 >> 172.21.15.125:0/4243797488 conn(0x55747715f020 msgr2=0x55747715a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 -- 172.21.15.125:0/4243797488 shutdown_connections 2023-10-18T10:38:02.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.879+0000 7f9aee046740 1 -- 172.21.15.125:0/4243797488 wait complete. 2023-10-18T10:38:02.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aee046740 20 client.6720 trim_cache size 0 max 16384 2023-10-18T10:38:02.907 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:38:02.908 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:38:02.908 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:38:02.908 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 6591, 2023-10-18T10:38:02.908 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:38:02.908 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:38:02.909 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:38:02.909 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 4.925643913, 2023-10-18T10:38:02.909 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 23, 2023-10-18T10:38:02.909 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 65, 2023-10-18T10:38:02.909 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 65, 2023-10-18T10:38:02.910 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 10.698049922999999 2023-10-18T10:38:02.910 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:38:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f9ac806af90 msgr2=0x7f9ac806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f9ac806af90 0x7f9ac806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4006cb0 tx=0x7f9ad4006720 comp rx=0 tx=0).stop 2023-10-18T10:38:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3807940681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ae413a230 msgr2=0x7f9ae40b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ae413a230 0x7f9ae40b07f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0011a50 tx=0x7f9ae0011f10 comp rx=0 tx=0).stop 2023-10-18T10:38:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3807940681 shutdown_connections 2023-10-18T10:38:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f9ac806af90 0x7f9ac806d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ae40b0d30 0x7f9ae4150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ae413a230 0x7f9ae40b07f0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 --2- 172.21.15.125:0/3807940681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad0004d00 0x7f9ae414fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:02.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3807940681 >> 172.21.15.125:0/3807940681 conn(0x7f9ae4069930 msgr2=0x7f9ae4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:02.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3807940681 shutdown_connections 2023-10-18T10:38:02.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:02.880+0000 7f9aeb4b4640 1 -- 172.21.15.125:0/3807940681 wait complete. 2023-10-18T10:38:02.914 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:38:02.921 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T10:38:02.919+0000 7f5c227cf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:38:03.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.015+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1555046862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800047c0 msgr2=0x7f3b80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:03.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.015+0000 7f3b99ca2640 1 --2- 172.21.15.125:0/1555046862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800047c0 0x7f3b80004c40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c001680 tx=0x7f3b800057f0 comp rx=0 tx=0).stop 2023-10-18T10:38:03.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.015+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1555046862 shutdown_connections 2023-10-18T10:38:03.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.015+0000 7f3b99ca2640 1 --2- 172.21.15.125:0/1555046862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800047c0 0x7f3b80004c40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:03.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.015+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1555046862 >> 172.21.15.125:0/1555046862 conn(0x7f3b9409a030 msgr2=0x7f3b940d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:03.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.015+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1555046862 shutdown_connections 2023-10-18T10:38:03.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.015+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1555046862 wait complete. 2023-10-18T10:38:03.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.016+0000 7f3b99ca2640 1 Processor -- start 2023-10-18T10:38:03.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.016+0000 7f3b99ca2640 1 -- start start 2023-10-18T10:38:03.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b99ca2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800047c0 0x7f3b9414d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:03.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b99ca2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b940c8950 0x7f3b940c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b99ca2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b940ca060 0x7f3b940ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b99ca2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b940c8dd0 con 0x7f3b940c8950 2023-10-18T10:38:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b99ca2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b940c8570 con 0x7f3b940ca060 2023-10-18T10:38:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b99ca2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b940c8fc0 con 0x7f3b800047c0 2023-10-18T10:38:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b940c8950 0x7f3b940c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:03.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b940c8950 0x7f3b940c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55582/0 (socket says 172.21.15.125:55582) 2023-10-18T10:38:03.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 -- 172.21.15.125:0/1361144187 learned_addr learned my addr 172.21.15.125:0/1361144187 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:03.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b93fff640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b940ca060 0x7f3b940ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:03.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b937fe640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800047c0 0x7f3b9414d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:03.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 -- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800047c0 msgr2=0x7f3b9414d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:03.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800047c0 0x7f3b9414d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:03.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 -- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b940ca060 msgr2=0x7f3b940ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:03.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b940ca060 0x7f3b940ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:03.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 -- 172.21.15.125:0/1361144187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b940c92a0 con 0x7f3b940c8950 2023-10-18T10:38:03.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b937fe640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800047c0 0x7f3b9414d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:38:03.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.017+0000 7f3b92ffd640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b940c8950 0x7f3b940c8030 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c02c0d0 tx=0x7f3b7c006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:03.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.018+0000 7f3b8bfff640 1 -- 172.21.15.125:0/1361144187 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c006540 con 0x7f3b940c8950 2023-10-18T10:38:03.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.018+0000 7f3b8bfff640 1 -- 172.21.15.125:0/1361144187 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b7c003c70 con 0x7f3b940c8950 2023-10-18T10:38:03.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.018+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b940c9580 con 0x7f3b940c8950 2023-10-18T10:38:03.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.018+0000 7f3b8bfff640 1 -- 172.21.15.125:0/1361144187 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c02c790 con 0x7f3b940c8950 2023-10-18T10:38:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.018+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b940c9b40 con 0x7f3b940c8950 2023-10-18T10:38:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.019+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b940c9cd0 con 0x7f3b940c8950 2023-10-18T10:38:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.019+0000 7f3b8bfff640 1 -- 172.21.15.125:0/1361144187 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3b7c02c8f0 con 0x7f3b940c8950 2023-10-18T10:38:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.019+0000 7f3b8bfff640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b7406af60 0x7f3b7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.019+0000 7f3b8bfff640 1 -- 172.21.15.125:0/1361144187 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3b7c0aaa20 con 0x7f3b940c8950 2023-10-18T10:38:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.020+0000 7f3b937fe640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b7406af60 0x7f3b7406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:03.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.020+0000 7f3b937fe640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b7406af60 0x7f3b7406d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c005fd0 tx=0x7f3b8c009500 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:03.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.024+0000 7f3b8bfff640 1 -- 172.21.15.125:0/1361144187 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b7c075880 con 0x7f3b940c8950 2023-10-18T10:38:03.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.224+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3b940675f0 con 0x7f3b940c8950 2023-10-18T10:38:03.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.224+0000 7f3b8bfff640 1 -- 172.21.15.125:0/1361144187 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3b7c044070 con 0x7f3b940c8950 2023-10-18T10:38:03.227 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:38:03.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b7406af60 msgr2=0x7f3b7406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:03.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b7406af60 0x7f3b7406d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c005fd0 tx=0x7f3b8c009500 comp rx=0 tx=0).stop 2023-10-18T10:38:03.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b940c8950 msgr2=0x7f3b940c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:03.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b940c8950 0x7f3b940c8030 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c02c0d0 tx=0x7f3b7c006360 comp rx=0 tx=0).stop 2023-10-18T10:38:03.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 shutdown_connections 2023-10-18T10:38:03.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3b7406af60 0x7f3b7406d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:03.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b940ca060 0x7f3b940ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:03.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b940c8950 0x7f3b940c8030 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:03.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 --2- 172.21.15.125:0/1361144187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800047c0 0x7f3b9414d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:03.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 >> 172.21.15.125:0/1361144187 conn(0x7f3b9409a030 msgr2=0x7f3b9406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:03.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 shutdown_connections 2023-10-18T10:38:03.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:03.226+0000 7f3b99ca2640 1 -- 172.21.15.125:0/1361144187 wait complete. 2023-10-18T10:38:03.239 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:38:03.239 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:38:03.239 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:38:03.240 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:38:03.240 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:38:03.240 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:38:03.241 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:38:03.241 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:38:03.241 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:38:03.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:03.243 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:38:03.295 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:38:03.296 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:38:03.296 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:38:03.296 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:38:03.296 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:38:03.296 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:38:03.297 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:38:03.297 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:38:03.297 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:38:03.297 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:38:03.297 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:38:03.298 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:38:03.298 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:38:03.298 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:38:03.298 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:38:03.298 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:38:03.299 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:38:03.299 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 41941sec preferred_lft 41941sec 2023-10-18T10:38:03.299 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:38:03.299 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:38:03.300 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:38:03.300 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:38:03.300 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:38:03.300 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:38:03.300 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T10:38:03.301 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:38:03.301 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:38:03.301 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:38:03.301 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T10:38:03.301 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:38:03.302 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:38:03.302 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:38:03.302 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T10:38:03.302 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:38:03.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:03.303 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:38:03.361 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:38:03.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:03.364 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:38:03.419 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:38:03.419 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:38:03.420 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:38:03.420 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:38:03.420 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:38:03.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:03.421 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:38:03.421 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:38:03.422 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:38:03.423 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:38:03.423 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:38:03.424 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:38:03.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:03.514 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:38:03.529 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:38:03.529 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:38:03.529 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:38:03.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:03.530 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:38:03.596 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:38:03.597 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:38:03.597 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:38:03.651 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:38:03.651 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:38:03.652 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:38:03.706 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:38:03.774 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:38:03.829 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:38:03.829 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:38:03.829 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:38:03.830 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:38:03.830 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:38:03.830 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:38:03.830 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:38:03.830 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:38:03.831 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:38:03.831 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:38:03.831 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:38:03.831 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:38:03.831 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:38:03.832 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:38:03.832 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:38:03.832 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:38:03.832 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:38:03.832 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:38:03.833 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:38:03.833 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:38:03.833 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:38:03.833 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:38:03.833 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:38:03.833 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:38:03.834 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:38:03.834 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:38:03.834 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:38:03.834 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:38:03.834 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:38:03.835 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:38:03.835 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:38:03.835 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:38:03.835 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:38:03.835 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:38:03.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:03.836 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:38:03.885 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:38:03.886 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:38:03.926 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:38:03.953 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:38:04.008 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:38:04.008 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:38:04.009 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:38:04.009 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:38:04.009 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:38:04.009 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:38:04.009 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:38:04.010 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:38:04.010 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:38:04.010 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:38:04.010 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:38:04.011 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:38:04.011 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:38:04.011 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:38:04.011 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:38:04.011 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:38:04.012 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:38:04.012 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:38:04.012 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:38:04.012 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:38:04.013 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:38:04.013 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:38:04.013 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:38:04.013 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:38:04.014 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:38:04.014 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:38:04.014 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:38:04.014 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:38:04.015 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:38:04.015 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:38:04.015 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:38:04.015 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:38:04.016 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:38:04.016 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:38:04.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:04.017 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:38:04.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:38:04.009+0000 7f5372641480 -1 init, newargv = 0x55ec5de04990 newargc=15 2023-10-18T10:38:04.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[114100]: starting ceph client 2023-10-18T10:38:04.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[114100]: starting fuse 2023-10-18T10:38:04.032 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:38:04.032 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:38:04.033 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:38:04.033 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:38:04.033 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:38:04.034 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:38:04.034 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:38:04.035 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:38:04.035 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:38:04.035 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:38:04.036 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:38:04.036 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:38:04.037 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:38:04.037 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:38:04.037 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:38:04.038 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:38:04.038 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:38:04.038 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:38:04.039 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:38:04.039 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:38:04.039 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:38:04.040 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:38:04.040 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:38:04.041 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:38:04.041 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:38:04.042 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:38:04.042 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:38:04.042 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:38:04.043 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:38:04.151 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.114100.asok 2023-10-18T10:38:04.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:38:04.162 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.114100.asok 2023-10-18T10:38:04.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:04.163 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.114100.asok status 2023-10-18T10:38:04.297 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:38:04.298 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:38:04.298 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:38:04.298 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:38:04.298 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:38:04.299 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:38:04.299 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:38:04.299 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "114100", 2023-10-18T10:38:04.299 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:38:04.299 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:38:04.299 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:38:04.300 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:38:04.300 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 6732, 2023-10-18T10:38:04.300 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:38:04.300 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:38:04.300 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:38:04.301 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 6732 2023-10-18T10:38:04.301 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:38:04.301 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:38:04.301 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:38:04.301 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:38:04.302 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 1114294450 2023-10-18T10:38:04.302 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:38:04.302 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:38:04.302 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:38:04.302 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:38:04.303 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:38:04.303 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 1114294450 2023-10-18T10:38:04.303 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:38:04.303 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.6732 192.168.0.1:0/1114294450", 2023-10-18T10:38:04.303 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/1114294450", 2023-10-18T10:38:04.303 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:38:04.304 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 23, 2023-10-18T10:38:04.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 65, 2023-10-18T10:38:04.304 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:38:04.304 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:38:04.304 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:38:04.305 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:38:04.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:04.306 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:38:04.307 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:38:04.321 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:38:04.322 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:38:04.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:04.323 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:38:04.389 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T10:38:04.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.531+0000 7f3dbe264640 1 -- 172.21.15.125:0/1570696383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3db815b2a0 msgr2=0x7f3db80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:04.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.531+0000 7f3dbe264640 1 --2- 172.21.15.125:0/1570696383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3db815b2a0 0x7f3db80d1910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009830 tx=0x7f3dac02bcd0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.531+0000 7f3dbe264640 1 -- 172.21.15.125:0/1570696383 shutdown_connections 2023-10-18T10:38:04.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.531+0000 7f3dbe264640 1 --2- 172.21.15.125:0/1570696383 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3db80ceb40 0x7f3db80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.531+0000 7f3dbe264640 1 --2- 172.21.15.125:0/1570696383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3db815b2a0 0x7f3db80d1910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.531+0000 7f3dbe264640 1 -- 172.21.15.125:0/1570696383 >> 172.21.15.125:0/1570696383 conn(0x7f3db80d0d60 msgr2=0x7f3db80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:04.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.532+0000 7f3dbe264640 1 -- 172.21.15.125:0/1570696383 shutdown_connections 2023-10-18T10:38:04.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.532+0000 7f3dbe264640 1 -- 172.21.15.125:0/1570696383 wait complete. 2023-10-18T10:38:04.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.533+0000 7f3dbe264640 1 Processor -- start 2023-10-18T10:38:04.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.533+0000 7f3dbe264640 1 -- start start 2023-10-18T10:38:04.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbe264640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3db80ceb40 0x7f3db80ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:04.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbe264640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3db80cf340 0x7f3db80b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:04.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbe264640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3db815b2a0 0x7f3db815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:04.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbe264640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db815b6d0 con 0x7f3db80cf340 2023-10-18T10:38:04.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbe264640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80cac70 con 0x7f3db815b2a0 2023-10-18T10:38:04.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbe264640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db814f6e0 con 0x7f3db80ceb40 2023-10-18T10:38:04.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3db80cf340 0x7f3db80b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:04.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3db80cf340 0x7f3db80b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55586/0 (socket says 172.21.15.125:55586) 2023-10-18T10:38:04.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 -- 172.21.15.125:0/4276144802 learned_addr learned my addr 172.21.15.125:0/4276144802 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:04.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbc801640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3db815b2a0 0x7f3db815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:04.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db7fff640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3db80ceb40 0x7f3db80ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:04.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 -- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3db80ceb40 msgr2=0x7f3db80ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:04.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3db80ceb40 0x7f3db80ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 -- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3db815b2a0 msgr2=0x7f3db815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:04.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3db815b2a0 0x7f3db815c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 -- 172.21.15.125:0/4276144802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac009490 con 0x7f3db80cf340 2023-10-18T10:38:04.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db77fe640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3db80cf340 0x7f3db80b5c90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009780 tx=0x7f3dac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:04.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db4ff9640 1 -- 172.21.15.125:0/4276144802 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac003ea0 con 0x7f3db80cf340 2023-10-18T10:38:04.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db4ff9640 1 -- 172.21.15.125:0/4276144802 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3dac034440 con 0x7f3db80cf340 2023-10-18T10:38:04.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3db4ff9640 1 -- 172.21.15.125:0/4276144802 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac034850 con 0x7f3db80cf340 2023-10-18T10:38:04.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db814f9c0 con 0x7f3db80cf340 2023-10-18T10:38:04.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.534+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db815bc00 con 0x7f3db80cf340 2023-10-18T10:38:04.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.535+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db80d07d0 con 0x7f3db80cf340 2023-10-18T10:38:04.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.536+0000 7f3db4ff9640 1 -- 172.21.15.125:0/4276144802 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3dac03b040 con 0x7f3db80cf340 2023-10-18T10:38:04.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.536+0000 7f3db4ff9640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3da406af60 0x7f3da406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:04.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.536+0000 7f3db4ff9640 1 -- 172.21.15.125:0/4276144802 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3dac0abcb0 con 0x7f3db80cf340 2023-10-18T10:38:04.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.536+0000 7f3db7fff640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3da406af60 0x7f3da406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:04.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.537+0000 7f3db7fff640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3da406af60 0x7f3da406d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3da8007fd0 tx=0x7f3da8009040 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:04.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.540+0000 7f3db4ff9640 1 -- 172.21.15.125:0/4276144802 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dac076ba0 con 0x7f3db80cf340 2023-10-18T10:38:04.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.728+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 --> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3db80b0d30 con 0x7f3da406af60 2023-10-18T10:38:04.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.730+0000 7f3db4ff9640 1 -- 172.21.15.125:0/4276144802 <== mgr.5626 v2:172.21.15.139:6832/2140727580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3db80b0d30 con 0x7f3da406af60 2023-10-18T10:38:04.732 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:38:04.732 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:38:04.732 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T10:38:04.732 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:38:04.733 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:38:04.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3da406af60 msgr2=0x7f3da406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:04.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3da406af60 0x7f3da406d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3da8007fd0 tx=0x7f3da8009040 comp rx=0 tx=0).stop 2023-10-18T10:38:04.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3db80cf340 msgr2=0x7f3db80b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:04.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3db80cf340 0x7f3db80b5c90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009780 tx=0x7f3dac0038a0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 shutdown_connections 2023-10-18T10:38:04.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7f3da406af60 0x7f3da406d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3db815b2a0 0x7f3db815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3db80cf340 0x7f3db80b5c90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 --2- 172.21.15.125:0/4276144802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3db80ceb40 0x7f3db80ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 >> 172.21.15.125:0/4276144802 conn(0x7f3db80d0d60 msgr2=0x7f3db80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:04.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 shutdown_connections 2023-10-18T10:38:04.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.732+0000 7f3dbe264640 1 -- 172.21.15.125:0/4276144802 wait complete. 2023-10-18T10:38:04.745 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T10:38:04.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.848+0000 7ff5bf773640 1 -- 172.21.15.125:0/437355992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8070af0 msgr2=0x7ff5b80affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:04.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.848+0000 7ff5bf773640 1 --2- 172.21.15.125:0/437355992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8070af0 0x7ff5b80affe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80cbd00 tx=0x7ff5b002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.848+0000 7ff5bf773640 1 -- 172.21.15.125:0/437355992 shutdown_connections 2023-10-18T10:38:04.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.848+0000 7ff5bf773640 1 --2- 172.21.15.125:0/437355992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8070af0 0x7ff5b80affe0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.848+0000 7ff5bf773640 1 --2- 172.21.15.125:0/437355992 >> v2:172.21.15.139:3300/0 conn(0x7ff5b813e520 0x7ff5b806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.848+0000 7ff5bf773640 1 -- 172.21.15.125:0/437355992 >> 172.21.15.125:0/437355992 conn(0x7ff5b8069930 msgr2=0x7ff5b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:04.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.848+0000 7ff5bf773640 1 -- 172.21.15.125:0/437355992 shutdown_connections 2023-10-18T10:38:04.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.848+0000 7ff5bf773640 1 -- 172.21.15.125:0/437355992 wait complete. 2023-10-18T10:38:04.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.849+0000 7ff5bf773640 1 Processor -- start 2023-10-18T10:38:04.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.849+0000 7ff5bf773640 1 -- start start 2023-10-18T10:38:04.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bf773640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b8070af0 0x7ff5b80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:04.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bf773640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b813e520 0x7ff5b813ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:04.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bf773640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8141690 0x7ff5b813b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:04.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bf773640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b813b820 con 0x7ff5b813e520 2023-10-18T10:38:04.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bf773640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b815aca0 con 0x7ff5b8070af0 2023-10-18T10:38:04.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bf773640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80c7bf0 con 0x7ff5b8141690 2023-10-18T10:38:04.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b813e520 0x7ff5b813ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:04.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b813e520 0x7ff5b813ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55598/0 (socket says 172.21.15.125:55598) 2023-10-18T10:38:04.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 -- 172.21.15.125:0/1488281668 learned_addr learned my addr 172.21.15.125:0/1488281668 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:04.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bd50f640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b8070af0 0x7ff5b80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:04.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bdd10640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8141690 0x7ff5b813b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:04.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 -- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8141690 msgr2=0x7ff5b813b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:04.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8141690 0x7ff5b813b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 -- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b8070af0 msgr2=0x7ff5b80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:04.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b8070af0 0x7ff5b80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:04.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 -- 172.21.15.125:0/1488281668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0009490 con 0x7ff5b813e520 2023-10-18T10:38:04.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bdd10640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8141690 0x7ff5b813b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:38:04.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.850+0000 7ff5bcd0e640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b813e520 0x7ff5b813ada0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c410 tx=0x7ff5a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:04.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.851+0000 7ff5adffb640 1 -- 172.21.15.125:0/1488281668 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80035f0 con 0x7ff5b813e520 2023-10-18T10:38:04.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.851+0000 7ff5adffb640 1 -- 172.21.15.125:0/1488281668 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5a8003770 con 0x7ff5b813e520 2023-10-18T10:38:04.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.851+0000 7ff5adffb640 1 -- 172.21.15.125:0/1488281668 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8003b80 con 0x7ff5b813e520 2023-10-18T10:38:04.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.851+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b80c7ed0 con 0x7ff5b813e520 2023-10-18T10:38:04.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.851+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b80c8490 con 0x7ff5b813e520 2023-10-18T10:38:04.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.852+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b80b1bd0 con 0x7ff5b813e520 2023-10-18T10:38:04.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.852+0000 7ff5adffb640 1 -- 172.21.15.125:0/1488281668 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 20) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff5a8003ce0 con 0x7ff5b813e520 2023-10-18T10:38:04.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.852+0000 7ff5adffb640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7ff59c06af60 0x7ff59c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:04.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.852+0000 7ff5adffb640 1 -- 172.21.15.125:0/1488281668 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff5a8088f90 con 0x7ff5b813e520 2023-10-18T10:38:04.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.853+0000 7ff5bd50f640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7ff59c06af60 0x7ff59c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:04.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.853+0000 7ff5bd50f640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7ff59c06af60 0x7ff59c06d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005fd0 tx=0x7ff5b4009680 comp rx=0 tx=0).ready entity=mgr.5626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:04.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:04.857+0000 7ff5adffb640 1 -- 172.21.15.125:0/1488281668 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5a80bbca0 con 0x7ff5b813e520 2023-10-18T10:38:05.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.072+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7ff5b80690e0 con 0x7ff5b813e520 2023-10-18T10:38:05.124 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:38:05.127 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:38:05.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.336+0000 7ff5adffb640 1 -- 172.21.15.125:0/1488281668 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v21) v1 ==== 92+0+0 (secure 0 0 0) 0x7ff5a8017070 con 0x7ff5b813e520 2023-10-18T10:38:05.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7ff59c06af60 msgr2=0x7ff59c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7ff59c06af60 0x7ff59c06d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005fd0 tx=0x7ff5b4009680 comp rx=0 tx=0).stop 2023-10-18T10:38:05.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b813e520 msgr2=0x7ff5b813ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b813e520 0x7ff5b813ada0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c410 tx=0x7ff5a800c8d0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 shutdown_connections 2023-10-18T10:38:05.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7ff59c06af60 0x7ff59c06d420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8141690 0x7ff5b813b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b813e520 0x7ff5b813ada0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 --2- 172.21.15.125:0/1488281668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b8070af0 0x7ff5b80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 >> 172.21.15.125:0/1488281668 conn(0x7ff5b8069930 msgr2=0x7ff5b80ac670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:05.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 shutdown_connections 2023-10-18T10:38:05.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.339+0000 7ff5bf773640 1 -- 172.21.15.125:0/1488281668 wait complete. 2023-10-18T10:38:05.354 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T10:38:05.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.461+0000 7fa74b09b640 1 -- 172.21.15.125:0/1006643266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7440cf340 msgr2=0x7fa7440d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.461+0000 7fa74b09b640 1 --2- 172.21.15.125:0/1006643266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7440cf340 0x7fa7440d1910 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa738009830 tx=0x7fa73802bd60 comp rx=0 tx=0).stop 2023-10-18T10:38:05.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.461+0000 7fa74b09b640 1 -- 172.21.15.125:0/1006643266 shutdown_connections 2023-10-18T10:38:05.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.461+0000 7fa74b09b640 1 --2- 172.21.15.125:0/1006643266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7440cf340 0x7fa7440d1910 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.461+0000 7fa74b09b640 1 -- 172.21.15.125:0/1006643266 >> 172.21.15.125:0/1006643266 conn(0x7fa7440d0d60 msgr2=0x7fa7440d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:05.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.461+0000 7fa74b09b640 1 -- 172.21.15.125:0/1006643266 shutdown_connections 2023-10-18T10:38:05.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.461+0000 7fa74b09b640 1 -- 172.21.15.125:0/1006643266 wait complete. 2023-10-18T10:38:05.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.462+0000 7fa74b09b640 1 Processor -- start 2023-10-18T10:38:05.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.462+0000 7fa74b09b640 1 -- start start 2023-10-18T10:38:05.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa74b09b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7440ceb40 0x7fa74415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:05.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa74b09b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7440cf340 0x7fa74414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:05.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa74b09b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74415b2a0 0x7fa74415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:05.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa74b09b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa744151210 con 0x7fa74415b2a0 2023-10-18T10:38:05.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa74b09b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa744150600 con 0x7fa7440cf340 2023-10-18T10:38:05.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa74b09b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7441508e0 con 0x7fa7440ceb40 2023-10-18T10:38:05.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74415b2a0 0x7fa74415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:05.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74415b2a0 0x7fa74415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55638/0 (socket says 172.21.15.125:55638) 2023-10-18T10:38:05.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 -- 172.21.15.125:0/1734648398 learned_addr learned my addr 172.21.15.125:0/1734648398 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:05.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa748e37640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7440ceb40 0x7fa74415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:05.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa743fff640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7440cf340 0x7fa74414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:05.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 -- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7440ceb40 msgr2=0x7fa74415df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7440ceb40 0x7fa74415df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 -- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7440cf340 msgr2=0x7fa74414f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7440cf340 0x7fa74414f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 -- 172.21.15.125:0/1734648398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738009490 con 0x7fa74415b2a0 2023-10-18T10:38:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.463+0000 7fa749638640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74415b2a0 0x7fa74415c090 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00c760 tx=0x7fa73c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.464+0000 7fa7417fa640 1 -- 172.21.15.125:0/1734648398 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c003da0 con 0x7fa74415b2a0 2023-10-18T10:38:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.464+0000 7fa7417fa640 1 -- 172.21.15.125:0/1734648398 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa73c00ce70 con 0x7fa74415b2a0 2023-10-18T10:38:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.464+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa744150bc0 con 0x7fa74415b2a0 2023-10-18T10:38:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.464+0000 7fa7417fa640 1 -- 172.21.15.125:0/1734648398 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c0052b0 con 0x7fa74415b2a0 2023-10-18T10:38:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.464+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74414fe00 con 0x7fa74415b2a0 2023-10-18T10:38:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.465+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7441513c0 con 0x7fa74415b2a0 2023-10-18T10:38:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.465+0000 7fa7417fa640 1 -- 172.21.15.125:0/1734648398 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 21) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa73c005410 con 0x7fa74415b2a0 2023-10-18T10:38:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.465+0000 7fa7417fa640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa72806afb0 0x7fa72806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.466+0000 7fa7417fa640 1 -- 172.21.15.125:0/1734648398 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa73c0897b0 con 0x7fa74415b2a0 2023-10-18T10:38:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.466+0000 7fa748e37640 1 -- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa72806afb0 msgr2=0x7fa72806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2140727580 2023-10-18T10:38:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.466+0000 7fa748e37640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa72806afb0 0x7fa72806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:38:05.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.470+0000 7fa7417fa640 1 -- 172.21.15.125:0/1734648398 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7441513c0 con 0x7fa74415b2a0 2023-10-18T10:38:05.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.645+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fa7441513c0 con 0x7fa74415b2a0 2023-10-18T10:38:05.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.648+0000 7fa7417fa640 1 -- 172.21.15.125:0/1734648398 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v36)=0 v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa7441513c0 con 0x7fa74415b2a0 2023-10-18T10:38:05.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa72806afb0 msgr2=0x7fa72806d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:38:05.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa72806afb0 0x7fa72806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74415b2a0 msgr2=0x7fa74415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74415b2a0 0x7fa74415c090 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00c760 tx=0x7fa73c00cc20 comp rx=0 tx=0).stop 2023-10-18T10:38:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 shutdown_connections 2023-10-18T10:38:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa72806afb0 0x7fa72806d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7440cf340 0x7fa74414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74415b2a0 0x7fa74415c090 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 --2- 172.21.15.125:0/1734648398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7440ceb40 0x7fa74415df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 >> 172.21.15.125:0/1734648398 conn(0x7fa7440d0d60 msgr2=0x7fa744159190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 shutdown_connections 2023-10-18T10:38:05.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.650+0000 7fa74b09b640 1 -- 172.21.15.125:0/1734648398 wait complete. 2023-10-18T10:38:05.664 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T10:38:05.722 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:05.720+0000 7f457fa21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:38:05.732 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:05.731+0000 7f20c802e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:38:05.766 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:05.764+0000 7f457fa21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:38:05.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.769+0000 7fda06db7640 1 -- 172.21.15.125:0/216009242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000d1910 msgr2=0x7fda000b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.769+0000 7fda06db7640 1 --2- 172.21.15.125:0/216009242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000d1910 0x7fda000b0650 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8005fd0 tx=0x7fd9f802c2e0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.769+0000 7fda06db7640 1 -- 172.21.15.125:0/216009242 shutdown_connections 2023-10-18T10:38:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.769+0000 7fda06db7640 1 --2- 172.21.15.125:0/216009242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000d1910 0x7fda000b0650 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.769+0000 7fda06db7640 1 -- 172.21.15.125:0/216009242 >> 172.21.15.125:0/216009242 conn(0x7fda000d0d60 msgr2=0x7fda000d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.769+0000 7fda06db7640 1 -- 172.21.15.125:0/216009242 shutdown_connections 2023-10-18T10:38:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.769+0000 7fda06db7640 1 -- 172.21.15.125:0/216009242 wait complete. 2023-10-18T10:38:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.771+0000 7fda06db7640 1 Processor -- start 2023-10-18T10:38:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.771+0000 7fda06db7640 1 -- start start 2023-10-18T10:38:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.771+0000 7fda06db7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000cf340 0x7fda000cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.771+0000 7fda06db7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda000d1910 0x7fda000b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.771+0000 7fda06db7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda0015b2a0 0x7fda000b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.771+0000 7fda06db7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001f5640 con 0x7fda000cf340 2023-10-18T10:38:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.771+0000 7fda06db7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000cb090 con 0x7fda0015b2a0 2023-10-18T10:38:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.771+0000 7fda06db7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda001f49c0 con 0x7fda000d1910 2023-10-18T10:38:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000cf340 0x7fda000cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000cf340 0x7fda000cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55656/0 (socket says 172.21.15.125:55656) 2023-10-18T10:38:05.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 -- 172.21.15.125:0/2358080940 learned_addr learned my addr 172.21.15.125:0/2358080940 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:05.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda05354640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda0015b2a0 0x7fda000b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:05.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fd9fffff640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda000d1910 0x7fda000b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:05.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 -- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda000d1910 msgr2=0x7fda000b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda000d1910 0x7fda000b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 -- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda0015b2a0 msgr2=0x7fda000b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda0015b2a0 0x7fda000b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 -- 172.21.15.125:0/2358080940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8009900 con 0x7fda000cf340 2023-10-18T10:38:05.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda04b53640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000cf340 0x7fda000cab50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c450 tx=0x7fd9f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:05.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fd9fd7fa640 1 -- 172.21.15.125:0/2358080940 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40052d0 con 0x7fda000cf340 2023-10-18T10:38:05.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fd9fd7fa640 1 -- 172.21.15.125:0/2358080940 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9f4005430 con 0x7fda000cf340 2023-10-18T10:38:05.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda001f4ca0 con 0x7fda000cf340 2023-10-18T10:38:05.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fd9fd7fa640 1 -- 172.21.15.125:0/2358080940 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4012890 con 0x7fda000cf340 2023-10-18T10:38:05.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.772+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001f5260 con 0x7fda000cf340 2023-10-18T10:38:05.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.773+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda001f5780 con 0x7fda000cf340 2023-10-18T10:38:05.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.774+0000 7fd9fd7fa640 1 -- 172.21.15.125:0/2358080940 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 21) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fd9f40035f0 con 0x7fda000cf340 2023-10-18T10:38:05.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.774+0000 7fd9fd7fa640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fd9e406afe0 0x7fd9e406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:05.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.774+0000 7fd9fffff640 1 -- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fd9e406afe0 msgr2=0x7fd9e406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2140727580 2023-10-18T10:38:05.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.774+0000 7fd9fd7fa640 1 -- 172.21.15.125:0/2358080940 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd9f4089450 con 0x7fda000cf340 2023-10-18T10:38:05.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.774+0000 7fd9fffff640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fd9e406afe0 0x7fd9e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:38:05.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.778+0000 7fd9fd7fa640 1 -- 172.21.15.125:0/2358080940 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda001f5780 con 0x7fda000cf340 2023-10-18T10:38:05.782 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:05.780+0000 7f20c802e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:38:05.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.952+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fda001f5780 con 0x7fda000cf340 2023-10-18T10:38:05.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.955+0000 7fd9fd7fa640 1 -- 172.21.15.125:0/2358080940 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v37)=0 v37) v1 ==== 133+0+0 (secure 0 0 0) 0x7fda001f5780 con 0x7fda000cf340 2023-10-18T10:38:05.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fd9e406afe0 msgr2=0x7fd9e406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:38:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fd9e406afe0 0x7fd9e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000cf340 msgr2=0x7fda000cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000cf340 0x7fda000cab50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c450 tx=0x7fd9f400c910 comp rx=0 tx=0).stop 2023-10-18T10:38:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 shutdown_connections 2023-10-18T10:38:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fd9e406afe0 0x7fd9e406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda0015b2a0 0x7fda000b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda000d1910 0x7fda000b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 --2- 172.21.15.125:0/2358080940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda000cf340 0x7fda000cab50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:05.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 >> 172.21.15.125:0/2358080940 conn(0x7fda000d0d60 msgr2=0x7fda000ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:05.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 shutdown_connections 2023-10-18T10:38:05.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:05.958+0000 7fda06db7640 1 -- 172.21.15.125:0/2358080940 wait complete. 2023-10-18T10:38:05.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:05.972 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T10:38:05.992 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-18T10:38:06.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.138+0000 7fa477e75640 1 -- 172.21.15.125:0/1966068550 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa47015b180 msgr2=0x7fa4700cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:06.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.138+0000 7fa477e75640 1 --2- 172.21.15.125:0/1966068550 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa47015b180 0x7fa4700cf350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa460009860 tx=0x7fa46002be60 comp rx=0 tx=0).stop 2023-10-18T10:38:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.138+0000 7fa477e75640 1 -- 172.21.15.125:0/1966068550 shutdown_connections 2023-10-18T10:38:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.138+0000 7fa477e75640 1 --2- 172.21.15.125:0/1966068550 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa47015b180 0x7fa4700cf350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.138+0000 7fa477e75640 1 -- 172.21.15.125:0/1966068550 >> 172.21.15.125:0/1966068550 conn(0x7fa470099ff0 msgr2=0x7fa4700d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.138+0000 7fa477e75640 1 -- 172.21.15.125:0/1966068550 shutdown_connections 2023-10-18T10:38:06.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.138+0000 7fa477e75640 1 -- 172.21.15.125:0/1966068550 wait complete. 2023-10-18T10:38:06.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.139+0000 7fa477e75640 1 Processor -- start 2023-10-18T10:38:06.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.139+0000 7fa477e75640 1 -- start start 2023-10-18T10:38:06.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa477e75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700cddc0 0x7fa470158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:06.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa477e75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700ce420 0x7fa4701e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:06.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa477e75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa47015b180 0x7fa4700b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:06.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa477e75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701e5890 con 0x7fa4700ce420 2023-10-18T10:38:06.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa477e75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701e5b70 con 0x7fa47015b180 2023-10-18T10:38:06.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa477e75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701e5e50 con 0x7fa4700cddc0 2023-10-18T10:38:06.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475410640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700ce420 0x7fa4701e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:06.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475410640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700ce420 0x7fa4701e4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55658/0 (socket says 172.21.15.125:55658) 2023-10-18T10:38:06.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475410640 1 -- 172.21.15.125:0/1388648537 learned_addr learned my addr 172.21.15.125:0/1388648537 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:06.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475c11640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700cddc0 0x7fa470158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:06.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa476412640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa47015b180 0x7fa4700b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:06.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475410640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700cddc0 msgr2=0x7fa470158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:06.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475410640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700cddc0 0x7fa470158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:06.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475410640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa47015b180 msgr2=0x7fa4700b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:06.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475410640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa47015b180 0x7fa4700b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:06.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475410640 1 -- 172.21.15.125:0/1388648537 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa460009490 con 0x7fa4700ce420 2023-10-18T10:38:06.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.140+0000 7fa475c11640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4700cddc0 0x7fa470158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:38:06.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.141+0000 7fa475410640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700ce420 0x7fa4701e4f90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c500 tx=0x7fa46c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:06.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.141+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0052d0 con 0x7fa4700ce420 2023-10-18T10:38:06.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.141+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa46c005450 con 0x7fa4700ce420 2023-10-18T10:38:06.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.141+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4701e6130 con 0x7fa4700ce420 2023-10-18T10:38:06.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.141+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c012890 con 0x7fa4700ce420 2023-10-18T10:38:06.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.141+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4701e54d0 con 0x7fa4700ce420 2023-10-18T10:38:06.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.142+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa470066960 con 0x7fa4700ce420 2023-10-18T10:38:06.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.143+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 21) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa46c0035f0 con 0x7fa4700ce420 2023-10-18T10:38:06.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.143+0000 7fa4667fc640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 0x7fa45406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:06.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.143+0000 7fa475c11640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 msgr2=0x7fa45406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2140727580 2023-10-18T10:38:06.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.143+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa46c089520 con 0x7fa4700ce420 2023-10-18T10:38:06.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.143+0000 7fa475c11640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 0x7fa45406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:38:06.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.147+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa46c054440 con 0x7fa4700ce420 2023-10-18T10:38:06.161 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.159+0000 7f457fa21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:38:06.195 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.194+0000 7f20c802e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:38:06.205 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.203+0000 7f457fa21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:38:06.241 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.240+0000 7f20c802e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:38:06.247 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.245+0000 7f457fa21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:38:06.285 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.284+0000 7f20c802e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:38:06.298 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.297+0000 7f457fa21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:38:06.338 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.337+0000 7f20c802e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:38:06.339 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.338+0000 7f457fa21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:38:06.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.342+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 --> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4701e5ce0 con 0x7fa45406afe0 2023-10-18T10:38:06.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.344+0000 7fa475c11640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 msgr2=0x7fa45406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2140727580 2023-10-18T10:38:06.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.344+0000 7fa475c11640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 0x7fa45406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:38:06.381 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.380+0000 7f20c802e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:38:06.439 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.437+0000 7f457fa21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:38:06.479 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.477+0000 7f20c802e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:38:06.491 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.489+0000 7f457fa21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:38:06.531 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.530+0000 7f20c802e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:38:06.533 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.531+0000 7f457fa21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:38:06.573 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.571+0000 7f20c802e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:38:06.581 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.579+0000 7f457fa21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:38:06.620 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.619+0000 7f20c802e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:38:06.623 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.621+0000 7f457fa21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:38:06.662 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.661+0000 7f20c802e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:38:06.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.745+0000 7fa475c11640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 msgr2=0x7fa45406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2140727580 2023-10-18T10:38:06.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:06.745+0000 7fa475c11640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 0x7fa45406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:38:06.938 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:06.936+0000 7f457fa21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:38:06.977 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:06.975+0000 7f20c802e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:38:07.068 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:07.067+0000 7f457fa21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:38:07.108 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:07.106+0000 7f20c802e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:38:07.159 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:07.158+0000 7f457fa21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:38:07.200 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:07.198+0000 7f20c802e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:38:07.202 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:07.201+0000 7f457fa21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:38:07.241 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:07.240+0000 7f20c802e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:38:07.296 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:07.295+0000 7f457fa21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:38:07.336 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:07.334+0000 7f20c802e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:38:07.386 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:07.385+0000 7f457fa21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:38:07.427 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:07.425+0000 7f20c802e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:38:07.522 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:07.521+0000 7f457fa21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:38:07.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:07.546+0000 7fa475c11640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 msgr2=0x7fa45406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2140727580 2023-10-18T10:38:07.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:07.546+0000 7fa475c11640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 0x7fa45406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:38:07.563 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:07.561+0000 7f20c802e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:38:07.564 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:07.563+0000 7f457fa21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:38:07.605 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:07.603+0000 7f20c802e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:38:07.668 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:07.667+0000 7f457fa21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:38:07.708 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:07.707+0000 7f20c802e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:38:08.280 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.278+0000 7f457fa21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:38:08.327 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:08.317+0000 7f20c802e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:38:08.541 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.540+0000 7f457fa21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:38:08.579 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:08.578+0000 7f20c802e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:38:08.589 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.588+0000 7f457fa21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:38:08.628 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:08.626+0000 7f20c802e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:38:08.689 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.688+0000 7f457fa21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:38:08.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:08.708+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 22) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fa46c054810 con 0x7fa4700ce420 2023-10-18T10:38:08.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:08.708+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 msgr2=0x7fa45406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:38:08.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:08.708+0000 7fa4667fc640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/2140727580,v1:172.21.15.139:6833/2140727580] conn(0x7fa45406afe0 0x7fa45406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:08.732 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:38:08.731+0000 7f20c802e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:38:08.816 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44fc1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.816 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44fc1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.816 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44fc1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.817 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44fc1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.817 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44fc1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.817 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44ff209640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.817 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44ff209640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.817 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44ff209640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.818 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44ff209640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:08.818 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:38:08.814+0000 7f44ff209640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:38:09.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.713+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 23) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fa46c0878d0 con 0x7fa4700ce420 2023-10-18T10:38:09.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.713+0000 7fa4667fc640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fa454035880 0x7fa4540753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:09.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.713+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 --> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4701e5ce0 con 0x7fa454035880 2023-10-18T10:38:09.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.715+0000 7fa475c11640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fa454035880 0x7fa4540753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:09.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.717+0000 7fa475c11640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fa454035880 0x7fa4540753b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa460036000 tx=0x7fa460004730 comp rx=0 tx=0).ready entity=mgr.5956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:09.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.770+0000 7fa4667fc640 1 -- 172.21.15.125:0/1388648537 <== mgr.5956 v2:172.21.15.139:6832/397604228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa4701e5ce0 con 0x7fa454035880 2023-10-18T10:38:09.773 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1 2023-10-18T10:38:09.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.772+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fa454035880 msgr2=0x7fa4540753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:09.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.772+0000 7fa477e75640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fa454035880 0x7fa4540753b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa460036000 tx=0x7fa460004730 comp rx=0 tx=0).stop 2023-10-18T10:38:09.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.772+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700ce420 msgr2=0x7fa4701e4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:09.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.772+0000 7fa477e75640 1 --2- 172.21.15.125:0/1388648537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4700ce420 0x7fa4701e4f90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00c500 tx=0x7fa46c00c9c0 comp rx=0 tx=0).stop 2023-10-18T10:38:09.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.773+0000 7fa476412640 1 -- 172.21.15.125:0/1388648537 reap_dead start 2023-10-18T10:38:09.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.773+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 shutdown_connections 2023-10-18T10:38:09.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.773+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 >> 172.21.15.125:0/1388648537 conn(0x7fa470099ff0 msgr2=0x7fa47015c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:09.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.773+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 shutdown_connections 2023-10-18T10:38:09.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.773+0000 7fa477e75640 1 -- 172.21.15.125:0/1388648537 wait complete. 2023-10-18T10:38:09.786 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-18T10:38:09.786 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2m 2023-10-18T10:38:09.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.890+0000 7fb9a46e5640 1 -- 172.21.15.125:0/4007706070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c140620 msgr2=0x7fb99c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:09.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.890+0000 7fb9a46e5640 1 --2- 172.21.15.125:0/4007706070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c140620 0x7fb99c0703e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb998009830 tx=0x7fb99802bd60 comp rx=0 tx=0).stop 2023-10-18T10:38:09.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.891+0000 7fb9a46e5640 1 -- 172.21.15.125:0/4007706070 shutdown_connections 2023-10-18T10:38:09.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.891+0000 7fb9a46e5640 1 --2- 172.21.15.125:0/4007706070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c140620 0x7fb99c0703e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:09.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.891+0000 7fb9a46e5640 1 -- 172.21.15.125:0/4007706070 >> 172.21.15.125:0/4007706070 conn(0x7fb99c069930 msgr2=0x7fb99c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:09.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.891+0000 7fb9a46e5640 1 -- 172.21.15.125:0/4007706070 shutdown_connections 2023-10-18T10:38:09.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.891+0000 7fb9a46e5640 1 -- 172.21.15.125:0/4007706070 wait complete. 2023-10-18T10:38:09.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a46e5640 1 Processor -- start 2023-10-18T10:38:09.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a46e5640 1 -- start start 2023-10-18T10:38:09.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a46e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c06fab0 0x7fb99c1334e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:09.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a46e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb99c140620 0x7fb99c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:09.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a46e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb99c140a00 0x7fb99c0a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:09.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a46e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c133a20 con 0x7fb99c06fab0 2023-10-18T10:38:09.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a46e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c1eeec0 con 0x7fb99c140620 2023-10-18T10:38:09.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a46e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0aa010 con 0x7fb99c140a00 2023-10-18T10:38:09.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.892+0000 7fb9a2481640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c06fab0 0x7fb99c1334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:09.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2481640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c06fab0 0x7fb99c1334e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57268/0 (socket says 172.21.15.125:57268) 2023-10-18T10:38:09.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2481640 1 -- 172.21.15.125:0/39780895 learned_addr learned my addr 172.21.15.125:0/39780895 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:09.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a1c80640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb99c140620 0x7fb99c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:09.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2c82640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb99c140a00 0x7fb99c0a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:09.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2481640 1 -- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb99c140a00 msgr2=0x7fb99c0a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:09.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2481640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb99c140a00 0x7fb99c0a99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:09.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2481640 1 -- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb99c140620 msgr2=0x7fb99c0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:09.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2481640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb99c140620 0x7fb99c0a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:09.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2481640 1 -- 172.21.15.125:0/39780895 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb998009490 con 0x7fb99c06fab0 2023-10-18T10:38:09.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a2481640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c06fab0 0x7fb99c1334e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb998000c00 tx=0x7fb9980038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:09.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb992ffd640 1 -- 172.21.15.125:0/39780895 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998003ea0 con 0x7fb99c06fab0 2023-10-18T10:38:09.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb992ffd640 1 -- 172.21.15.125:0/39780895 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb998034530 con 0x7fb99c06fab0 2023-10-18T10:38:09.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb992ffd640 1 -- 172.21.15.125:0/39780895 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998034940 con 0x7fb99c06fab0 2023-10-18T10:38:09.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c0aa2f0 con 0x7fb99c06fab0 2023-10-18T10:38:09.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.893+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c0aa8b0 con 0x7fb99c06fab0 2023-10-18T10:38:09.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.894+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c0690e0 con 0x7fb99c06fab0 2023-10-18T10:38:09.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.895+0000 7fb992ffd640 1 -- 172.21.15.125:0/39780895 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 23) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fb998034aa0 con 0x7fb99c06fab0 2023-10-18T10:38:09.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.895+0000 7fb992ffd640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fb98006ad10 0x7fb98006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:09.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.895+0000 7fb992ffd640 1 -- 172.21.15.125:0/39780895 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6373+0+0 (secure 0 0 0) 0x7fb9980abea0 con 0x7fb99c06fab0 2023-10-18T10:38:09.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.895+0000 7fb9a1c80640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fb98006ad10 0x7fb98006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:09.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.896+0000 7fb9a1c80640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fb98006ad10 0x7fb98006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb98c006fd0 tx=0x7fb98c008040 comp rx=0 tx=0).ready entity=mgr.5956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:09.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:09.900+0000 7fb992ffd640 1 -- 172.21.15.125:0/39780895 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb998076ff0 con 0x7fb99c06fab0 2023-10-18T10:38:10.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.085+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 --> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb99c0ca100 con 0x7fb98006ad10 2023-10-18T10:38:10.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.135+0000 7fb992ffd640 1 -- 172.21.15.125:0/39780895 <== mgr.5956 v2:172.21.15.139:6832/397604228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb98c00bd80 con 0x7fb98006ad10 2023-10-18T10:38:10.137 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1 2023-10-18T10:38:10.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fb98006ad10 msgr2=0x7fb98006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:10.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fb98006ad10 0x7fb98006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb98c006fd0 tx=0x7fb98c008040 comp rx=0 tx=0).stop 2023-10-18T10:38:10.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c06fab0 msgr2=0x7fb99c1334e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:10.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c06fab0 0x7fb99c1334e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb998000c00 tx=0x7fb9980038a0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 shutdown_connections 2023-10-18T10:38:10.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fb98006ad10 0x7fb98006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb99c140620 0x7fb99c0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb99c140a00 0x7fb99c0a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 --2- 172.21.15.125:0/39780895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb99c06fab0 0x7fb99c1334e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 >> 172.21.15.125:0/39780895 conn(0x7fb99c069930 msgr2=0x7fb99c13c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:10.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 shutdown_connections 2023-10-18T10:38:10.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.137+0000 7fb9a46e5640 1 -- 172.21.15.125:0/39780895 wait complete. 2023-10-18T10:38:10.150 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-18T10:38:10.150 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-18-10_39_00 in ~50s... 2023-10-18T10:38:10.151 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-18-10_40_00 in ~110s... 2023-10-18T10:38:10.151 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m', '2m'], expected_retention: [] 2023-10-18T10:38:10.151 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-18T10:38:10.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.255+0000 7f3fc5090640 1 -- 172.21.15.125:0/1259095210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00c96b0 msgr2=0x7f3fc00c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:10.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.255+0000 7f3fc5090640 1 --2- 172.21.15.125:0/1259095210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00c96b0 0x7f3fc00c9b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009830 tx=0x7f3fa802bd60 comp rx=0 tx=0).stop 2023-10-18T10:38:10.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.255+0000 7f3fc5090640 1 -- 172.21.15.125:0/1259095210 shutdown_connections 2023-10-18T10:38:10.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.255+0000 7f3fc5090640 1 --2- 172.21.15.125:0/1259095210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00c96b0 0x7f3fc00c9b30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.255+0000 7f3fc5090640 1 --2- 172.21.15.125:0/1259095210 >> v2:172.21.15.139:3300/0 conn(0x7f3fc006af00 0x7f3fc006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.255+0000 7f3fc5090640 1 -- 172.21.15.125:0/1259095210 >> 172.21.15.125:0/1259095210 conn(0x7f3fc0069930 msgr2=0x7f3fc0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:10.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.255+0000 7f3fc5090640 1 -- 172.21.15.125:0/1259095210 shutdown_connections 2023-10-18T10:38:10.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.255+0000 7f3fc5090640 1 -- 172.21.15.125:0/1259095210 wait complete. 2023-10-18T10:38:10.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.256+0000 7f3fc5090640 1 Processor -- start 2023-10-18T10:38:10.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.256+0000 7f3fc5090640 1 -- start start 2023-10-18T10:38:10.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fc5090640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fc006af00 0x7f3fc00b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:10.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fc5090640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00ab500 0x7f3fc0138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:10.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fc5090640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00c96b0 0x7f3fc01e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:10.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fc5090640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc01e4700 con 0x7f3fc00c96b0 2023-10-18T10:38:10.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fc5090640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc01e49e0 con 0x7f3fc006af00 2023-10-18T10:38:10.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fc5090640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc01e4cc0 con 0x7f3fc00ab500 2023-10-18T10:38:10.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00c96b0 0x7f3fc01e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:10.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00c96b0 0x7f3fc01e3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57272/0 (socket says 172.21.15.125:57272) 2023-10-18T10:38:10.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 -- 172.21.15.125:0/103608393 learned_addr learned my addr 172.21.15.125:0/103608393 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:38:10.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbed9d640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fc006af00 0x7f3fc00b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:10.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fb7fff640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00ab500 0x7f3fc0138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:10.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 -- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00ab500 msgr2=0x7f3fc0138380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:10.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00ab500 0x7f3fc0138380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 -- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fc006af00 msgr2=0x7f3fc00b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:10.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fc006af00 0x7f3fc00b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 -- 172.21.15.125:0/103608393 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8009490 con 0x7f3fc00c96b0 2023-10-18T10:38:10.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fb7fff640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00ab500 0x7f3fc0138380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:38:10.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fbf59e640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00c96b0 0x7f3fc01e3e40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800cbc0 tx=0x7f3fb800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:10.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fb77fe640 1 -- 172.21.15.125:0/103608393 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800aca0 con 0x7f3fc00c96b0 2023-10-18T10:38:10.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc01e4fa0 con 0x7f3fc00c96b0 2023-10-18T10:38:10.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fb77fe640 1 -- 172.21.15.125:0/103608393 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3fb8003970 con 0x7f3fc00c96b0 2023-10-18T10:38:10.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.257+0000 7f3fb77fe640 1 -- 172.21.15.125:0/103608393 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8003d80 con 0x7f3fc00c96b0 2023-10-18T10:38:10.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.258+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc01388c0 con 0x7f3fc00c96b0 2023-10-18T10:38:10.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.258+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc006c7e0 con 0x7f3fc00c96b0 2023-10-18T10:38:10.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.259+0000 7f3fb77fe640 1 -- 172.21.15.125:0/103608393 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 23) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3fb8004030 con 0x7f3fc00c96b0 2023-10-18T10:38:10.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.259+0000 7f3fb77fe640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3fa006ace0 0x7f3fa006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:38:10.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.259+0000 7f3fb77fe640 1 -- 172.21.15.125:0/103608393 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f3fb8089f70 con 0x7f3fc00c96b0 2023-10-18T10:38:10.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.259+0000 7f3fbed9d640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3fa006ace0 0x7f3fa006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:38:10.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.260+0000 7f3fbed9d640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3fa006ace0 0x7f3fa006d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009bd0 tx=0x7f3fb0005f90 comp rx=0 tx=0).ready entity=mgr.5956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:38:10.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.264+0000 7f3fb77fe640 1 -- 172.21.15.125:0/103608393 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fb80bcca0 con 0x7f3fc00c96b0 2023-10-18T10:38:10.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.452+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 --> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fc0159c50 con 0x7f3fa006ace0 2023-10-18T10:38:10.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.460+0000 7f3fb77fe640 1 -- 172.21.15.125:0/103608393 <== mgr.5956 v2:172.21.15.139:6832/397604228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f3fb001b3f0 con 0x7f3fa006ace0 2023-10-18T10:38:10.462 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:38:10.462 INFO:teuthology.orchestra.run.smithi125.stdout:{"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-18T10:38:10.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3fa006ace0 msgr2=0x7f3fa006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:10.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3fa006ace0 0x7f3fa006d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009bd0 tx=0x7f3fb0005f90 comp rx=0 tx=0).stop 2023-10-18T10:38:10.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00c96b0 msgr2=0x7f3fc01e3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:38:10.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00c96b0 0x7f3fc01e3e40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800cbc0 tx=0x7f3fb800a060 comp rx=0 tx=0).stop 2023-10-18T10:38:10.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 shutdown_connections 2023-10-18T10:38:10.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3fa006ace0 0x7f3fa006d1a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00c96b0 0x7f3fc01e3e40 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fc00ab500 0x7f3fc0138380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 --2- 172.21.15.125:0/103608393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fc006af00 0x7f3fc00b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:38:10.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 >> 172.21.15.125:0/103608393 conn(0x7f3fc0069930 msgr2=0x7f3fc0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:38:10.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 shutdown_connections 2023-10-18T10:38:10.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:38:10.462+0000 7f3fc5090640 1 -- 172.21.15.125:0/103608393 wait complete. 2023-10-18T10:38:10.475 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-18T10:38:10.475 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m', '2m'], 'retention': [{}, {}]} 2023-10-18T10:38:10.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:10.476 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:10.495 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:10.495 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:10.495 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:11.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:11.497 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:11.520 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:11.520 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:11.521 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:12.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:12.523 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:12.546 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:12.546 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:12.546 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:13.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:13.548 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:13.572 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:13.572 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:13.572 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:14.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:14.574 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:14.598 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:14.599 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:14.599 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:15.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:15.601 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:15.624 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:15.624 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:15.624 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:16.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:16.626 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:16.649 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:16.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:16.650 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:17.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:17.651 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:17.674 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:17.675 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:17.675 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:18.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:18.677 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:18.701 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:18.702 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:18.702 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:19.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:19.703 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:19.726 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:19.726 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:19.726 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:20.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:20.728 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:20.751 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:20.751 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:20.751 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:21.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:21.754 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:21.777 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:21.777 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:21.777 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:22.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:22.779 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:22.802 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:22.803 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:22.803 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:23.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:23.805 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:23.828 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:23.828 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:23.828 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:24.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:24.830 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:24.853 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:24.854 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:24.854 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:25.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:25.856 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:25.878 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:25.879 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:25.879 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:26.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:26.881 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:26.904 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:26.904 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:26.905 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:27.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:27.906 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:27.929 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:27.929 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:27.930 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:28.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:28.932 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:28.956 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:28.957 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:28.957 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:29.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:29.959 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:29.982 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:29.983 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:29.983 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:30.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:30.985 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:31.008 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:31.008 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:31.009 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:32.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:32.011 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:32.033 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:32.034 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:32.034 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:33.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:33.036 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:33.058 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:33.059 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:33.059 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:34.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:34.061 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:34.084 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:34.085 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:34.085 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:35.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:35.087 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:35.110 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:35.110 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:35.110 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:35.187 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:38:35.190 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:38:36.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:36.112 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:36.135 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:36.135 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:36.136 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:37.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:37.137 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:37.160 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:37.161 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:37.161 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:38.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:38.163 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:38.185 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:38.185 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:38.186 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:39.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:39.188 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:39.210 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:39.211 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:39.211 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:40.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:40.213 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:40.235 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:40.236 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:40.236 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:41.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:41.238 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:41.261 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:41.262 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:41.262 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:42.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:42.263 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:42.286 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:42.287 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:42.287 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:43.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:43.289 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:43.312 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:43.312 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:43.313 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:44.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:44.315 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:44.337 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:44.338 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:44.338 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:45.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:45.339 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:45.362 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:45.363 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:45.363 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:46.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:46.365 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:46.389 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:46.389 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:46.390 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:47.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:47.392 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:47.414 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:47.414 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:47.415 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:48.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:48.417 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:48.439 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:48.440 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:48.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:49.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:49.442 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:49.464 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:49.465 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:49.465 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:50.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:50.466 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:50.489 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:50.489 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:50.489 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:51.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:51.491 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:51.513 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:51.513 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:51.513 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:52.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:52.515 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:52.537 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:52.537 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:52.538 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:53.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:53.540 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:53.562 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:53.563 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:53.563 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:54.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:54.565 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:54.589 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:54.664 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:54.664 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:55.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:55.667 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:55.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:55.689 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:55.690 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:56.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:56.691 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:56.714 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:56.715 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:56.715 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:57.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:57.716 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:57.739 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:57.740 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:57.740 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:58.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:58.742 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:58.765 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:58.765 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:58.765 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:38:59.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:38:59.767 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:38:59.790 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:38:59.790 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:38:59.790 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:00.148 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:39:00.146+0000 7f44ee1a7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:39:00.148 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:39:00.146+0000 7f44ee1a7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:39:00.148 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:39:00.146+0000 7f44ee1a7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:39:00.148 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:39:00.146+0000 7f44ee1a7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:39:00.149 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:39:00.146+0000 7f44ee1a7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:39:00.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:00.793 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:00.815 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:00.816 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:00.816 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:00.816 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:00.817 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-18-10_39_00_UTC'] 2023-10-18T10:39:00.817 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697625490.1507974, now = 1697625540.8175929, timo = 50 2023-10-18T10:39:01.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:01.819 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:01.841 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:01.842 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:01.842 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:01.843 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:02.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:02.844 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:02.867 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:02.867 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:02.867 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:02.868 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:03.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:03.869 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:03.892 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:03.893 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:03.893 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:03.893 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:04.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:04.895 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:04.918 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:04.919 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:04.919 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:04.919 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:05.326 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:39:05.329 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:39:05.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:05.921 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:05.944 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:05.944 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:05.944 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:05.945 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:06.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:06.947 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:06.970 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:06.970 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:06.970 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:06.971 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:07.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:07.973 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:07.996 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:07.996 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:07.997 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:07.997 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:08.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:08.999 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:09.022 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:09.022 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:09.023 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:09.023 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:10.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:10.025 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:10.047 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:10.047 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:10.048 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:10.048 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:11.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:11.050 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:11.074 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:11.074 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:11.075 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:11.075 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:12.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:12.078 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:12.101 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:12.101 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:12.102 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:12.102 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:13.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:13.104 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:13.126 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:13.127 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:13.127 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:13.127 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:14.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:14.130 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:14.151 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:14.153 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:14.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:14.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:15.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:15.155 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:15.177 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:15.179 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:15.179 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:15.179 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:16.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:16.181 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:16.204 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:16.204 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:16.205 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:16.205 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:17.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:17.210 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:17.232 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:17.233 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:17.234 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:17.234 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:18.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:18.235 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:18.259 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:18.259 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:18.259 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:18.260 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:19.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:19.261 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:19.283 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:19.284 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:19.284 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:19.285 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:20.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:20.294 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:20.317 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:20.317 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:20.318 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:20.318 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:21.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:21.320 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:21.341 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:21.342 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:21.342 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:21.343 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:22.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:22.344 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:22.367 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:22.368 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:22.368 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:22.369 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:23.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:23.370 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:23.392 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:23.393 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:23.393 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:23.394 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:24.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:24.395 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:24.418 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:24.418 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:24.419 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:24.419 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:25.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:25.420 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:25.444 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:25.445 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:25.446 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:25.446 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:26.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:26.449 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:26.471 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:26.472 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:26.472 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:26.472 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:27.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:27.474 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:27.497 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:27.497 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:27.498 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:27.498 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:28.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:28.500 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:28.523 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:28.523 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:28.524 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:28.524 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:29.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:29.526 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:29.549 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:29.549 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:29.549 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:29.550 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:30.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:30.551 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:30.573 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:30.574 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:30.574 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:30.574 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:31.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:31.576 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:31.598 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:31.599 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:31.599 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:31.599 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:32.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:32.601 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:32.622 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:32.623 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:32.624 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:32.624 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:33.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:33.626 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:33.648 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:33.649 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:33.650 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:33.650 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:34.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:34.652 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:34.674 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:34.674 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:34.675 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:34.675 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:35.388 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:39:35.391 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:39:35.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:35.676 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:35.697 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:35.698 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:35.698 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:35.698 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:36.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:36.700 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:36.722 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:36.723 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:36.723 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:36.723 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:37.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:37.725 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:37.747 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:37.747 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:37.748 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:37.748 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:38.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:38.750 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:38.772 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:38.773 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:38.773 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:38.773 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:39.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:39.775 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:39.797 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:39.797 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:39.798 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:39.798 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:40.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:40.800 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:40.823 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:40.823 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:40.824 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:40.824 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:41.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:41.825 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:41.847 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:41.848 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:41.848 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:41.849 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:42.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:42.850 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:42.873 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:42.874 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:42.874 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:42.875 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:43.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:43.877 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:43.899 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:43.899 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:43.900 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:43.900 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:44.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:44.902 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:44.924 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:44.924 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:44.925 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:44.925 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:45.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:45.927 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:45.949 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:45.950 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:45.951 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:45.951 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:46.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:46.953 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:46.975 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:46.976 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:46.976 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:46.977 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:47.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:47.978 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:48.002 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:48.002 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:48.002 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:48.003 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:49.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:49.005 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:49.028 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:49.028 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:49.029 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:49.029 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:50.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:50.031 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:50.054 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:50.054 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:50.055 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:50.055 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:51.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:51.057 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:51.079 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:51.080 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:51.080 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:51.080 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:52.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:52.082 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:52.105 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:52.106 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:52.106 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:52.106 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:53.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:53.108 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:53.131 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:53.132 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:53.132 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:53.132 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:54.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:54.134 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:54.156 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:54.157 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:54.157 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:54.157 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:55.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:55.160 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:55.183 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:55.183 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:55.184 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:55.184 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:56.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:56.186 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:56.208 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:56.208 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:56.209 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:56.209 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:57.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:57.211 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:57.234 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:57.234 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:57.235 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:57.235 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:58.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:58.237 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:58.260 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:58.260 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:58.260 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:58.261 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:39:59.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:39:59.263 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:39:59.284 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:39:59.285 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC'} 2023-10-18T10:39:59.286 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:39:59.286 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:40:00.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:00.288 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:40:00.308 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:40:00.308 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_40_00_UTC 2023-10-18T10:40:00.309 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_39_00_UTC', 'scheduled-2023-10-18-10_40_00_UTC'} 2023-10-18T10:40:00.309 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-18-10_40_00_UTC'} 2023-10-18T10:40:00.309 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:40:00.310 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-18-10_40_00_UTC'] 2023-10-18T10:40:00.310 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697625490.1507974, now = 1697625600.3108065, timo = 110 2023-10-18T10:40:01.312 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-18T10:40:01.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.437+0000 7fc6b4e32640 1 -- 172.21.15.125:0/696307657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6b0070350 msgr2=0x7fc6b00707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:01.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.437+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/696307657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6b0070350 0x7fc6b00707c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8005fd0 tx=0x7fc6a802bfa0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.437+0000 7fc6b4e32640 1 -- 172.21.15.125:0/696307657 shutdown_connections 2023-10-18T10:40:01.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.437+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/696307657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6b0070350 0x7fc6b00707c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.437+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/696307657 >> v2:172.21.15.139:3300/0 conn(0x7fc6b00cb980 0x7fc6b013db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.437+0000 7fc6b4e32640 1 -- 172.21.15.125:0/696307657 >> 172.21.15.125:0/696307657 conn(0x7fc6b0069930 msgr2=0x7fc6b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:01.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.437+0000 7fc6b4e32640 1 -- 172.21.15.125:0/696307657 shutdown_connections 2023-10-18T10:40:01.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.437+0000 7fc6b4e32640 1 -- 172.21.15.125:0/696307657 wait complete. 2023-10-18T10:40:01.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.439+0000 7fc6b4e32640 1 Processor -- start 2023-10-18T10:40:01.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.439+0000 7fc6b4e32640 1 -- start start 2023-10-18T10:40:01.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6b4e32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6b0070350 0x7fc6b01348f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:01.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6b4e32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6b00cb980 0x7fc6b0135d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:01.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6b4e32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6b0136240 0x7fc6b01567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:01.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6b4e32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0139a70 con 0x7fc6b0136240 2023-10-18T10:40:01.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6b4e32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0139790 con 0x7fc6b0070350 2023-10-18T10:40:01.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6b4e32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0135220 con 0x7fc6b00cb980 2023-10-18T10:40:01.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6aed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6b0136240 0x7fc6b01567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:01.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6aed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6b0136240 0x7fc6b01567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54876/0 (socket says 172.21.15.125:54876) 2023-10-18T10:40:01.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6aed9d640 1 -- 172.21.15.125:0/1629399712 learned_addr learned my addr 172.21.15.125:0/1629399712 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:01.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6ae59c640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6b0070350 0x7fc6b01348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:01.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6add9b640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6b00cb980 0x7fc6b0135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:01.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6aed9d640 1 -- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6b00cb980 msgr2=0x7fc6b0135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:01.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6aed9d640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6b00cb980 0x7fc6b0135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6aed9d640 1 -- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6b0070350 msgr2=0x7fc6b01348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:01.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6aed9d640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6b0070350 0x7fc6b01348f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.440+0000 7fc6aed9d640 1 -- 172.21.15.125:0/1629399712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8009840 con 0x7fc6b0136240 2023-10-18T10:40:01.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.441+0000 7fc6aed9d640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6b0136240 0x7fc6b01567f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8036040 tx=0x7fc6a8003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:01.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.441+0000 7fc6a6ffd640 1 -- 172.21.15.125:0/1629399712 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80043d0 con 0x7fc6b0136240 2023-10-18T10:40:01.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.441+0000 7fc6a6ffd640 1 -- 172.21.15.125:0/1629399712 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc6a8004640 con 0x7fc6b0136240 2023-10-18T10:40:01.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.441+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b0135500 con 0x7fc6b0136240 2023-10-18T10:40:01.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.441+0000 7fc6a6ffd640 1 -- 172.21.15.125:0/1629399712 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a803e450 con 0x7fc6b0136240 2023-10-18T10:40:01.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.441+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b0134e30 con 0x7fc6b0136240 2023-10-18T10:40:01.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.442+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b00690e0 con 0x7fc6b0136240 2023-10-18T10:40:01.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.443+0000 7fc6a6ffd640 1 -- 172.21.15.125:0/1629399712 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fc6a803e5b0 con 0x7fc6b0136240 2023-10-18T10:40:01.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.443+0000 7fc6a6ffd640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fc69006afb0 0x7fc69006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:01.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.443+0000 7fc6a6ffd640 1 -- 172.21.15.125:0/1629399712 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc6a80ac890 con 0x7fc6b0136240 2023-10-18T10:40:01.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.443+0000 7fc6ae59c640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fc69006afb0 0x7fc69006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:01.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.444+0000 7fc6ae59c640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fc69006afb0 0x7fc69006d470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009bd0 tx=0x7fc6a0005f90 comp rx=0 tx=0).ready entity=mgr.5956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:01.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.448+0000 7fc6a6ffd640 1 -- 172.21.15.125:0/1629399712 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6a80777c0 con 0x7fc6b0136240 2023-10-18T10:40:01.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.637+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 --> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6b0070d00 con 0x7fc69006afb0 2023-10-18T10:40:01.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.702+0000 7fc6a6ffd640 1 -- 172.21.15.125:0/1629399712 <== mgr.5956 v2:172.21.15.139:6832/397604228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc6a001b3f0 con 0x7fc69006afb0 2023-10-18T10:40:01.704 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path snap_test_dir1 2023-10-18T10:40:01.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fc69006afb0 msgr2=0x7fc69006d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:01.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fc69006afb0 0x7fc69006d470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009bd0 tx=0x7fc6a0005f90 comp rx=0 tx=0).stop 2023-10-18T10:40:01.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6b0136240 msgr2=0x7fc6b01567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:01.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6b0136240 0x7fc6b01567f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8036040 tx=0x7fc6a8003f20 comp rx=0 tx=0).stop 2023-10-18T10:40:01.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 shutdown_connections 2023-10-18T10:40:01.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fc69006afb0 0x7fc69006d470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6b0136240 0x7fc6b01567f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6b00cb980 0x7fc6b0135d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 --2- 172.21.15.125:0/1629399712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6b0070350 0x7fc6b01348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:01.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 >> 172.21.15.125:0/1629399712 conn(0x7fc6b0069930 msgr2=0x7fc6b00cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:01.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 shutdown_connections 2023-10-18T10:40:01.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:01.704+0000 7fc6b4e32640 1 -- 172.21.15.125:0/1629399712 wait complete. 2023-10-18T10:40:01.716 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-18T10:40:01.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:01.717 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:40:01.734 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:40:01.735 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_40_00_UTC 2023-10-18T10:40:01.735 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_39_00_UTC 2023-10-18T10:40:01.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:01.736 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_39_00_UTC) 2023-10-18T10:40:01.795 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_40_00_UTC 2023-10-18T10:40:01.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:01.796 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_40_00_UTC) 2023-10-18T10:40:01.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:01.855 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-18T10:40:01.913 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T10:40:02.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.056+0000 7f76f4af2640 1 -- 172.21.15.125:0/2067599252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 msgr2=0x7f76f00707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.056+0000 7f76f4af2640 1 --2- 172.21.15.125:0/2067599252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 0x7f76f00707e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005fd0 tx=0x7f76e802c240 comp rx=0 tx=0).stop 2023-10-18T10:40:02.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.056+0000 7f76f4af2640 1 -- 172.21.15.125:0/2067599252 shutdown_connections 2023-10-18T10:40:02.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.056+0000 7f76f4af2640 1 --2- 172.21.15.125:0/2067599252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 0x7f76f00707e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.056+0000 7f76f4af2640 1 --2- 172.21.15.125:0/2067599252 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00cb980 0x7f76f013db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.056+0000 7f76f4af2640 1 -- 172.21.15.125:0/2067599252 >> 172.21.15.125:0/2067599252 conn(0x7f76f0069930 msgr2=0x7f76f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:02.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.056+0000 7f76f4af2640 1 -- 172.21.15.125:0/2067599252 shutdown_connections 2023-10-18T10:40:02.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.056+0000 7f76f4af2640 1 -- 172.21.15.125:0/2067599252 wait complete. 2023-10-18T10:40:02.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.058+0000 7f76f4af2640 1 Processor -- start 2023-10-18T10:40:02.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.058+0000 7f76f4af2640 1 -- start start 2023-10-18T10:40:02.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.058+0000 7f76f4af2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 0x7f76f0134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.058+0000 7f76f4af2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00cb980 0x7f76f00caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.058+0000 7f76f4af2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00caff0 0x7f76f01409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.058+0000 7f76f4af2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00cb4a0 con 0x7f76f0070350 2023-10-18T10:40:02.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.058+0000 7f76f4af2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f013e600 con 0x7f76f00caff0 2023-10-18T10:40:02.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.058+0000 7f76f4af2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f014f410 con 0x7f76f00cb980 2023-10-18T10:40:02.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 0x7f76f0134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 0x7f76f0134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54898/0 (socket says 172.21.15.125:54898) 2023-10-18T10:40:02.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 -- 172.21.15.125:0/1382098722 learned_addr learned my addr 172.21.15.125:0/1382098722 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:02.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76eed9d640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00caff0 0x7f76f01409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76edd9b640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00cb980 0x7f76f00caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 -- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00cb980 msgr2=0x7f76f00caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00cb980 0x7f76f00caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 -- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00caff0 msgr2=0x7f76f01409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00caff0 0x7f76f01409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 -- 172.21.15.125:0/1382098722 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8009900 con 0x7f76f0070350 2023-10-18T10:40:02.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76eed9d640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00caff0 0x7f76f01409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:02.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76ee59c640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 0x7f76f0134140 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f76e400b4a0 tx=0x7f76e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:02.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76deffd640 1 -- 172.21.15.125:0/1382098722 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4010610 con 0x7f76f0070350 2023-10-18T10:40:02.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76deffd640 1 -- 172.21.15.125:0/1382098722 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76e4010860 con 0x7f76f0070350 2023-10-18T10:40:02.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f014f6f0 con 0x7f76f0070350 2023-10-18T10:40:02.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76deffd640 1 -- 172.21.15.125:0/1382098722 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4010c70 con 0x7f76f0070350 2023-10-18T10:40:02.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.059+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f014fcb0 con 0x7f76f0070350 2023-10-18T10:40:02.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.060+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f00690e0 con 0x7f76f0070350 2023-10-18T10:40:02.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.061+0000 7f76deffd640 1 -- 172.21.15.125:0/1382098722 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f76e4017360 con 0x7f76f0070350 2023-10-18T10:40:02.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.061+0000 7f76deffd640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f76d006afb0 0x7f76d006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.061+0000 7f76deffd640 1 -- 172.21.15.125:0/1382098722 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f76e40898b0 con 0x7f76f0070350 2023-10-18T10:40:02.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.061+0000 7f76edd9b640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f76d006afb0 0x7f76d006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.062+0000 7f76edd9b640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f76d006afb0 0x7f76d006d470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f76e0006fd0 tx=0x7f76e0008040 comp rx=0 tx=0).ready entity=mgr.5956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:02.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.065+0000 7f76deffd640 1 -- 172.21.15.125:0/1382098722 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76e40580a0 con 0x7f76f0070350 2023-10-18T10:40:02.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.277+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f76f006ff80 con 0x7f76f0070350 2023-10-18T10:40:02.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.379+0000 7f76deffd640 1 -- 172.21.15.125:0/1382098722 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v28) v1 ==== 93+0+0 (secure 0 0 0) 0x7f76e4057c30 con 0x7f76f0070350 2023-10-18T10:40:02.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f76d006afb0 msgr2=0x7f76d006d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f76d006afb0 0x7f76d006d470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f76e0006fd0 tx=0x7f76e0008040 comp rx=0 tx=0).stop 2023-10-18T10:40:02.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 msgr2=0x7f76f0134140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 0x7f76f0134140 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f76e400b4a0 tx=0x7f76e400b960 comp rx=0 tx=0).stop 2023-10-18T10:40:02.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 shutdown_connections 2023-10-18T10:40:02.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f76d006afb0 0x7f76d006d470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f00caff0 0x7f76f01409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76f00cb980 0x7f76f00caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 --2- 172.21.15.125:0/1382098722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0070350 0x7f76f0134140 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 >> 172.21.15.125:0/1382098722 conn(0x7f76f0069930 msgr2=0x7f76f0159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:02.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 shutdown_connections 2023-10-18T10:40:02.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.382+0000 7f76f4af2640 1 -- 172.21.15.125:0/1382098722 wait complete. 2023-10-18T10:40:02.395 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:40:02.396 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:40:02.436 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:40:02.474 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:40:02.475 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:40:02.475 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:02.491 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:02.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.635+0000 7fbd78f3c640 1 -- 172.21.15.125:0/1279057514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74070350 msgr2=0x7fbd74070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.635+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/1279057514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74070350 0x7fbd74070780 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fbd7406f4d0 tx=0x7fbd6c02c2f0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.635+0000 7fbd78f3c640 1 -- 172.21.15.125:0/1279057514 shutdown_connections 2023-10-18T10:40:02.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.635+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/1279057514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74070350 0x7fbd74070780 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.635+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/1279057514 >> v2:172.21.15.139:3300/0 conn(0x7fbd7413d350 0x7fbd740cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.635+0000 7fbd78f3c640 1 -- 172.21.15.125:0/1279057514 >> 172.21.15.125:0/1279057514 conn(0x7fbd74069930 msgr2=0x7fbd74069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:02.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.635+0000 7fbd78f3c640 1 -- 172.21.15.125:0/1279057514 shutdown_connections 2023-10-18T10:40:02.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.635+0000 7fbd78f3c640 1 -- 172.21.15.125:0/1279057514 wait complete. 2023-10-18T10:40:02.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.636+0000 7fbd78f3c640 1 Processor -- start 2023-10-18T10:40:02.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd78f3c640 1 -- start start 2023-10-18T10:40:02.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd78f3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd74070350 0x7fbd740b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd78f3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd740cb980 0x7fbd740ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd78f3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd741ee7d0 0x7fbd741eec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd78f3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740af040 con 0x7fbd741ee7d0 2023-10-18T10:40:02.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd78f3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741ef1c0 con 0x7fbd74070350 2023-10-18T10:40:02.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd78f3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741f36f0 con 0x7fbd740cb980 2023-10-18T10:40:02.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd72d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd741ee7d0 0x7fbd741eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd72d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd741ee7d0 0x7fbd741eec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54930/0 (socket says 172.21.15.125:54930) 2023-10-18T10:40:02.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd72d9d640 1 -- 172.21.15.125:0/941277067 learned_addr learned my addr 172.21.15.125:0/941277067 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:02.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.637+0000 7fbd71d9b640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd740cb980 0x7fbd740ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd7259c640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd74070350 0x7fbd740b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd72d9d640 1 -- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd740cb980 msgr2=0x7fbd740ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd72d9d640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd740cb980 0x7fbd740ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd72d9d640 1 -- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd74070350 msgr2=0x7fbd740b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd72d9d640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd74070350 0x7fbd740b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd72d9d640 1 -- 172.21.15.125:0/941277067 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c009900 con 0x7fbd741ee7d0 2023-10-18T10:40:02.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd72d9d640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd741ee7d0 0x7fbd741eec80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009d60 tx=0x7fbd6c035ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:02.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd62ffd640 1 -- 172.21.15.125:0/941277067 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c003e60 con 0x7fbd741ee7d0 2023-10-18T10:40:02.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd62ffd640 1 -- 172.21.15.125:0/941277067 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbd6c004340 con 0x7fbd741ee7d0 2023-10-18T10:40:02.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd740b07a0 con 0x7fbd741ee7d0 2023-10-18T10:40:02.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd62ffd640 1 -- 172.21.15.125:0/941277067 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c004750 con 0x7fbd741ee7d0 2023-10-18T10:40:02.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.638+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd740b0d60 con 0x7fbd741ee7d0 2023-10-18T10:40:02.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.639+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd740690e0 con 0x7fbd741ee7d0 2023-10-18T10:40:02.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.639+0000 7fbd62ffd640 1 -- 172.21.15.125:0/941277067 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fbd6c00a040 con 0x7fbd741ee7d0 2023-10-18T10:40:02.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.640+0000 7fbd62ffd640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fbd5406af60 0x7fbd5406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.640+0000 7fbd62ffd640 1 -- 172.21.15.125:0/941277067 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fbd6c0aca10 con 0x7fbd741ee7d0 2023-10-18T10:40:02.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.640+0000 7fbd7259c640 1 -- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fbd5406af60 msgr2=0x7fbd5406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:02.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.640+0000 7fbd7259c640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fbd5406af60 0x7fbd5406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:02.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.644+0000 7fbd62ffd640 1 -- 172.21.15.125:0/941277067 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd6c077850 con 0x7fbd741ee7d0 2023-10-18T10:40:02.771 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:02.769+0000 7f0061a3e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:40:02.776 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:02.774+0000 7fab56a25040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:40:02.815 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:02.814+0000 7f0061a3e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:40:02.822 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:02.821+0000 7fab56a25040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:40:02.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.824+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd74152530 con 0x7fbd741ee7d0 2023-10-18T10:40:02.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.825+0000 7fbd62ffd640 1 -- 172.21.15.125:0/941277067 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17525 (secure 0 0 0) 0x7fbd6c044090 con 0x7fbd741ee7d0 2023-10-18T10:40:02.827 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:02.828 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":69,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:02.368954+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:37:55.100958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-18T10:37:56.256769+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3263530731":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3471641794":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/2137279356":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/1884009041":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3401202005":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3643146558":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6833/2140727580":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6832/2140727580":"2023-10-19T10:38:08.702660+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:02.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fbd5406af60 msgr2=0x7fbd5406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:02.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fbd5406af60 0x7fbd5406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd741ee7d0 msgr2=0x7fbd741eec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd741ee7d0 0x7fbd741eec80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009d60 tx=0x7fbd6c035ed0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 shutdown_connections 2023-10-18T10:40:02.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fbd5406af60 0x7fbd5406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd741ee7d0 0x7fbd741eec80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd740cb980 0x7fbd740ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 --2- 172.21.15.125:0/941277067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd74070350 0x7fbd740b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 >> 172.21.15.125:0/941277067 conn(0x7fbd74069930 msgr2=0x7fbd74069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:02.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 shutdown_connections 2023-10-18T10:40:02.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.827+0000 7fbd78f3c640 1 -- 172.21.15.125:0/941277067 wait complete. 2023-10-18T10:40:02.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:37:55.100958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:37:56.256769+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:40:02.841 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:40:02.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.946+0000 7fed8e9f5640 1 -- 172.21.15.125:0/2291050307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 msgr2=0x7fed8815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.946+0000 7fed8e9f5640 1 --2- 172.21.15.125:0/2291050307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 0x7fed8815d400 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fed80005fd0 tx=0x7fed8002c050 comp rx=0 tx=0).stop 2023-10-18T10:40:02.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.946+0000 7fed8e9f5640 1 -- 172.21.15.125:0/2291050307 shutdown_connections 2023-10-18T10:40:02.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.946+0000 7fed8e9f5640 1 --2- 172.21.15.125:0/2291050307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 0x7fed8815d400 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.946+0000 7fed8e9f5640 1 -- 172.21.15.125:0/2291050307 >> 172.21.15.125:0/2291050307 conn(0x7fed880b0670 msgr2=0x7fed880b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:02.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.946+0000 7fed8e9f5640 1 -- 172.21.15.125:0/2291050307 shutdown_connections 2023-10-18T10:40:02.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.946+0000 7fed8e9f5640 1 -- 172.21.15.125:0/2291050307 wait complete. 2023-10-18T10:40:02.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.947+0000 7fed8e9f5640 1 Processor -- start 2023-10-18T10:40:02.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.947+0000 7fed8e9f5640 1 -- start start 2023-10-18T10:40:02.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed8e9f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 0x7fed880c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed8e9f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880ce240 0x7fed8815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed8e9f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88159060 0x7fed88158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed8e9f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8815c520 con 0x7fed8806d260 2023-10-18T10:40:02.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed8e9f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed881ef490 con 0x7fed88159060 2023-10-18T10:40:02.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed8e9f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c2c60 con 0x7fed880ce240 2023-10-18T10:40:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed87fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 0x7fed880c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed87fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 0x7fed880c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54942/0 (socket says 172.21.15.125:54942) 2023-10-18T10:40:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed87fff640 1 -- 172.21.15.125:0/207291314 learned_addr learned my addr 172.21.15.125:0/207291314 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed8cf92640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88159060 0x7fed88158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed877fe640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880ce240 0x7fed8815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed87fff640 1 -- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880ce240 msgr2=0x7fed8815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed87fff640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880ce240 0x7fed8815bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed87fff640 1 -- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88159060 msgr2=0x7fed88158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.948+0000 7fed87fff640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88159060 0x7fed88158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.949+0000 7fed87fff640 1 -- 172.21.15.125:0/207291314 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80009840 con 0x7fed8806d260 2023-10-18T10:40:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.949+0000 7fed87fff640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 0x7fed880c25d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00c3c0 tx=0x7fed7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.949+0000 7fed84ff9640 1 -- 172.21.15.125:0/207291314 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0035f0 con 0x7fed8806d260 2023-10-18T10:40:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.949+0000 7fed84ff9640 1 -- 172.21.15.125:0/207291314 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed7c003770 con 0x7fed8806d260 2023-10-18T10:40:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.949+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed880c2f40 con 0x7fed8806d260 2023-10-18T10:40:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.949+0000 7fed84ff9640 1 -- 172.21.15.125:0/207291314 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c003b80 con 0x7fed8806d260 2023-10-18T10:40:02.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.949+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed880c3500 con 0x7fed8806d260 2023-10-18T10:40:02.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.950+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed880675f0 con 0x7fed8806d260 2023-10-18T10:40:02.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.950+0000 7fed84ff9640 1 -- 172.21.15.125:0/207291314 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fed7c003ce0 con 0x7fed8806d260 2023-10-18T10:40:02.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.951+0000 7fed84ff9640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fed7406af60 0x7fed7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:02.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.951+0000 7fed877fe640 1 -- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fed7406af60 msgr2=0x7fed7406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:02.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.951+0000 7fed84ff9640 1 -- 172.21.15.125:0/207291314 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fed7c089170 con 0x7fed8806d260 2023-10-18T10:40:02.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.951+0000 7fed877fe640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fed7406af60 0x7fed7406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:02.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:02.955+0000 7fed84ff9640 1 -- 172.21.15.125:0/207291314 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed7c053e10 con 0x7fed8806d260 2023-10-18T10:40:03.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.128+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed880b4aa0 con 0x7fed8806d260 2023-10-18T10:40:03.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.128+0000 7fed84ff9640 1 -- 172.21.15.125:0/207291314 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed7c057430 con 0x7fed8806d260 2023-10-18T10:40:03.130 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:40:03.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fed7406af60 msgr2=0x7fed7406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:03.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fed7406af60 0x7fed7406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 msgr2=0x7fed880c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 0x7fed880c25d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00c3c0 tx=0x7fed7c00c880 comp rx=0 tx=0).stop 2023-10-18T10:40:03.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 shutdown_connections 2023-10-18T10:40:03.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7fed7406af60 0x7fed7406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed88159060 0x7fed88158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880ce240 0x7fed8815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 --2- 172.21.15.125:0/207291314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed8806d260 0x7fed880c25d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.130+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 >> 172.21.15.125:0/207291314 conn(0x7fed880b0670 msgr2=0x7fed880ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:03.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.131+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 shutdown_connections 2023-10-18T10:40:03.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.131+0000 7fed8e9f5640 1 -- 172.21.15.125:0/207291314 wait complete. 2023-10-18T10:40:03.143 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:40:03.206 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.204+0000 7f0061a3e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:40:03.234 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.232+0000 7fab56a25040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:40:03.249 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.248+0000 7f0061a3e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:40:03.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.251+0000 7f4095407640 1 -- 172.21.15.125:0/2006729950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4078006290 msgr2=0x7f407800ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.251+0000 7f4095407640 1 --2- 172.21.15.125:0/2006729950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4078006290 0x7f407800ca90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f408800c640 tx=0x7f4078006900 comp rx=0 tx=0).stop 2023-10-18T10:40:03.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.251+0000 7f4095407640 1 -- 172.21.15.125:0/2006729950 shutdown_connections 2023-10-18T10:40:03.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.251+0000 7f4095407640 1 --2- 172.21.15.125:0/2006729950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4078006290 0x7f407800ca90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.251+0000 7f4095407640 1 -- 172.21.15.125:0/2006729950 >> 172.21.15.125:0/2006729950 conn(0x7f4090069930 msgr2=0x7f4090069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:03.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.251+0000 7f4095407640 1 -- 172.21.15.125:0/2006729950 shutdown_connections 2023-10-18T10:40:03.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.251+0000 7f4095407640 1 -- 172.21.15.125:0/2006729950 wait complete. 2023-10-18T10:40:03.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.252+0000 7f4095407640 1 Processor -- start 2023-10-18T10:40:03.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.252+0000 7f4095407640 1 -- start start 2023-10-18T10:40:03.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f4095407640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4078006290 0x7f4090149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f4095407640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4090135ae0 0x7f4090136110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f4095407640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4090136650 0x7f409014c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f4095407640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409014b830 con 0x7f4090135ae0 2023-10-18T10:40:03.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f4095407640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409014bb10 con 0x7f4090136650 2023-10-18T10:40:03.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f4095407640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409014bdf0 con 0x7f4078006290 2023-10-18T10:40:03.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4090135ae0 0x7f4090136110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4090135ae0 0x7f4090136110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54960/0 (socket says 172.21.15.125:54960) 2023-10-18T10:40:03.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 -- 172.21.15.125:0/2493995078 learned_addr learned my addr 172.21.15.125:0/2493995078 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:03.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408effd640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4078006290 0x7f4090149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408f7fe640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4090136650 0x7f409014c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 -- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4078006290 msgr2=0x7f4090149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4078006290 0x7f4090149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 -- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4090136650 msgr2=0x7f409014c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4090136650 0x7f409014c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 -- 172.21.15.125:0/2493995078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409014c0d0 con 0x7f4090135ae0 2023-10-18T10:40:03.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f408e7fc640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4090135ae0 0x7f4090136110 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4084003ec0 tx=0x7f4084001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:03.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f40777fe640 1 -- 172.21.15.125:0/2493995078 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084001d40 con 0x7f4090135ae0 2023-10-18T10:40:03.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f40777fe640 1 -- 172.21.15.125:0/2493995078 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4084011640 con 0x7f4090135ae0 2023-10-18T10:40:03.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409014c3b0 con 0x7f4090135ae0 2023-10-18T10:40:03.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.253+0000 7f40777fe640 1 -- 172.21.15.125:0/2493995078 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084011a30 con 0x7f4090135ae0 2023-10-18T10:40:03.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.254+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409014ce10 con 0x7f4090135ae0 2023-10-18T10:40:03.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.255+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40900690e0 con 0x7f4090135ae0 2023-10-18T10:40:03.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.255+0000 7f40777fe640 1 -- 172.21.15.125:0/2493995078 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f4084011b90 con 0x7f4090135ae0 2023-10-18T10:40:03.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.256+0000 7f40777fe640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f407c06af60 0x7f407c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.256+0000 7f408effd640 1 -- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f407c06af60 msgr2=0x7f407c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:03.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.256+0000 7f40777fe640 1 -- 172.21.15.125:0/2493995078 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f408408a830 con 0x7f4090135ae0 2023-10-18T10:40:03.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.256+0000 7f408effd640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f407c06af60 0x7f407c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:03.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.260+0000 7f40777fe640 1 -- 172.21.15.125:0/2493995078 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4084055390 con 0x7f4090135ae0 2023-10-18T10:40:03.281 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.280+0000 7fab56a25040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:40:03.293 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.291+0000 7f0061a3e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:40:03.325 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.324+0000 7fab56a25040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:40:03.345 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.343+0000 7f0061a3e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:40:03.380 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.379+0000 7fab56a25040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:40:03.391 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.389+0000 7f0061a3e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:40:03.425 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.422+0000 7fab56a25040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:40:03.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.436+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f409006f700 con 0x7f4090135ae0 2023-10-18T10:40:03.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.437+0000 7f40777fe640 1 -- 172.21.15.125:0/2493995078 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4084015020 con 0x7f4090135ae0 2023-10-18T10:40:03.439 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:40:03.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f407c06af60 msgr2=0x7f407c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:03.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f407c06af60 0x7f407c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4090135ae0 msgr2=0x7f4090136110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4090135ae0 0x7f4090136110 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4084003ec0 tx=0x7f4084001630 comp rx=0 tx=0).stop 2023-10-18T10:40:03.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 shutdown_connections 2023-10-18T10:40:03.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f407c06af60 0x7f407c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4090136650 0x7f409014c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4090135ae0 0x7f4090136110 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 --2- 172.21.15.125:0/2493995078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4078006290 0x7f4090149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 >> 172.21.15.125:0/2493995078 conn(0x7f4090069930 msgr2=0x7f40901578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:03.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 shutdown_connections 2023-10-18T10:40:03.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.439+0000 7f4095407640 1 -- 172.21.15.125:0/2493995078 wait complete. 2023-10-18T10:40:03.452 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:40:03.489 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.488+0000 7f0061a3e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:40:03.527 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.526+0000 7fab56a25040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:40:03.542 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.541+0000 7f0061a3e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:03.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.558+0000 7f922e5a5640 1 -- 172.21.15.125:0/3214372290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92280ce240 msgr2=0x7f922806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.558+0000 7f922e5a5640 1 --2- 172.21.15.125:0/3214372290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92280ce240 0x7f922806d260 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f921c005fd0 tx=0x7f921c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:03.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.558+0000 7f922e5a5640 1 -- 172.21.15.125:0/3214372290 shutdown_connections 2023-10-18T10:40:03.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.558+0000 7f922e5a5640 1 --2- 172.21.15.125:0/3214372290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f922815ccf0 0x7f922815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.558+0000 7f922e5a5640 1 --2- 172.21.15.125:0/3214372290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92280ce240 0x7f922806d260 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.558+0000 7f922e5a5640 1 -- 172.21.15.125:0/3214372290 >> 172.21.15.125:0/3214372290 conn(0x7f92280b0670 msgr2=0x7f92280b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:03.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.558+0000 7f922e5a5640 1 -- 172.21.15.125:0/3214372290 shutdown_connections 2023-10-18T10:40:03.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.558+0000 7f922e5a5640 1 -- 172.21.15.125:0/3214372290 wait complete. 2023-10-18T10:40:03.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.560+0000 7f922e5a5640 1 Processor -- start 2023-10-18T10:40:03.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.560+0000 7f922e5a5640 1 -- start start 2023-10-18T10:40:03.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.560+0000 7f922e5a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92280ce240 0x7f9228156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.560+0000 7f922e5a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9228159060 0x7f92280c1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.560+0000 7f922e5a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f922815ccf0 0x7f92281f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.560+0000 7f922e5a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280b46a0 con 0x7f922815ccf0 2023-10-18T10:40:03.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.560+0000 7f922e5a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280b3ad0 con 0x7f9228159060 2023-10-18T10:40:03.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.560+0000 7f922e5a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280b3d70 con 0x7f92280ce240 2023-10-18T10:40:03.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f922815ccf0 0x7f92281f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f9227fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92280ce240 0x7f9228156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f92277fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9228159060 0x7f92280c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f922815ccf0 0x7f92281f1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54970/0 (socket says 172.21.15.125:54970) 2023-10-18T10:40:03.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f9227fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92280ce240 0x7f9228156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:39210/0 (socket says 172.21.15.125:39210) 2023-10-18T10:40:03.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 -- 172.21.15.125:0/2637447916 learned_addr learned my addr 172.21.15.125:0/2637447916 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:03.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 -- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92280ce240 msgr2=0x7f9228156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92280ce240 0x7f9228156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 -- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9228159060 msgr2=0x7f92280c1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9228159060 0x7f92280c1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 -- 172.21.15.125:0/2637447916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c009560 con 0x7f922815ccf0 2023-10-18T10:40:03.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f92277fe640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9228159060 0x7f92280c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:03.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922cb42640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f922815ccf0 0x7f92281f1a40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f922000ea80 tx=0x7f922000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:03.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f9224ff9640 1 -- 172.21.15.125:0/2637447916 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922000cea0 con 0x7f922815ccf0 2023-10-18T10:40:03.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f9224ff9640 1 -- 172.21.15.125:0/2637447916 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9220004b20 con 0x7f922815ccf0 2023-10-18T10:40:03.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f9224ff9640 1 -- 172.21.15.125:0/2637447916 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220005020 con 0x7f922815ccf0 2023-10-18T10:40:03.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92280b4050 con 0x7f922815ccf0 2023-10-18T10:40:03.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.561+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92281572e0 con 0x7f922815ccf0 2023-10-18T10:40:03.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.562+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92280cb860 con 0x7f922815ccf0 2023-10-18T10:40:03.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.563+0000 7f9224ff9640 1 -- 172.21.15.125:0/2637447916 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f9220005180 con 0x7f922815ccf0 2023-10-18T10:40:03.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.563+0000 7f9224ff9640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f921406af60 0x7f921406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.563+0000 7f9227fff640 1 -- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f921406af60 msgr2=0x7f921406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:03.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.563+0000 7f9224ff9640 1 -- 172.21.15.125:0/2637447916 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f9220014070 con 0x7f922815ccf0 2023-10-18T10:40:03.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.563+0000 7f9227fff640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f921406af60 0x7f921406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:03.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.568+0000 7f9224ff9640 1 -- 172.21.15.125:0/2637447916 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9220055400 con 0x7f922815ccf0 2023-10-18T10:40:03.585 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.584+0000 7f0061a3e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:40:03.586 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.584+0000 7fab56a25040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:03.629 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.627+0000 7fab56a25040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:40:03.633 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.631+0000 7f0061a3e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:40:03.675 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.673+0000 7f0061a3e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:40:03.679 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.677+0000 7fab56a25040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:40:03.722 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:03.721+0000 7fab56a25040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:40:03.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.741+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f922806d260 con 0x7f922815ccf0 2023-10-18T10:40:03.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.741+0000 7f9224ff9640 1 -- 172.21.15.125:0/2637447916 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9220058a20 con 0x7f922815ccf0 2023-10-18T10:40:03.743 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:40:03.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f921406af60 msgr2=0x7f921406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:03.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f921406af60 0x7f921406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f922815ccf0 msgr2=0x7f92281f1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f922815ccf0 0x7f92281f1a40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f922000ea80 tx=0x7f922000ef40 comp rx=0 tx=0).stop 2023-10-18T10:40:03.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 shutdown_connections 2023-10-18T10:40:03.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f921406af60 0x7f921406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9228159060 0x7f92280c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f922815ccf0 0x7f92281f1a40 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 --2- 172.21.15.125:0/2637447916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92280ce240 0x7f9228156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 >> 172.21.15.125:0/2637447916 conn(0x7f92280b0670 msgr2=0x7f92280ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:03.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 shutdown_connections 2023-10-18T10:40:03.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.743+0000 7f922e5a5640 1 -- 172.21.15.125:0/2637447916 wait complete. 2023-10-18T10:40:03.756 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:40:03.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.861+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/475194150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc0cf340 msgr2=0x7ff0dc0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.861+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/475194150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc0cf340 0x7ff0dc0d1910 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc08a120 tx=0x7ff0cc02bdb0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.861+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/475194150 shutdown_connections 2023-10-18T10:40:03.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.861+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/475194150 >> v2:172.21.15.139:3300/0 conn(0x7ff0dc15b2a0 0x7ff0dc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.861+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/475194150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc0cf340 0x7ff0dc0d1910 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.861+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/475194150 >> 172.21.15.125:0/475194150 conn(0x7ff0dc0d0d60 msgr2=0x7ff0dc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:03.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.861+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/475194150 shutdown_connections 2023-10-18T10:40:03.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.861+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/475194150 wait complete. 2023-10-18T10:40:03.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.862+0000 7ff0e0ba9640 1 Processor -- start 2023-10-18T10:40:03.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0e0ba9640 1 -- start start 2023-10-18T10:40:03.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0e0ba9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0dc0ceb40 0x7ff0dc157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0e0ba9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0dc0cf340 0x7ff0dc0cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0e0ba9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc15b2a0 0x7ff0dc0ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0e0ba9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc1e64d0 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0e0ba9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0c9650 con 0x7ff0dc0ceb40 2023-10-18T10:40:03.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0e0ba9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0c9930 con 0x7ff0dc0cf340 2023-10-18T10:40:03.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0dad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc15b2a0 0x7ff0dc0ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0dad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc15b2a0 0x7ff0dc0ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54986/0 (socket says 172.21.15.125:54986) 2023-10-18T10:40:03.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0dad9d640 1 -- 172.21.15.125:0/2132613969 learned_addr learned my addr 172.21.15.125:0/2132613969 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:03.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0d3fff640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0dc0cf340 0x7ff0dc0cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.863+0000 7ff0da59c640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0dc0ceb40 0x7ff0dc157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:03.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0dad9d640 1 -- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0dc0cf340 msgr2=0x7ff0dc0cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0dad9d640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0dc0cf340 0x7ff0dc0cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0dad9d640 1 -- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0dc0ceb40 msgr2=0x7ff0dc157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:03.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0dad9d640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0dc0ceb40 0x7ff0dc157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:03.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0dad9d640 1 -- 172.21.15.125:0/2132613969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc009560 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0dad9d640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc15b2a0 0x7ff0dc0ca130 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c730 tx=0x7ff0d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:03.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0d2ffd640 1 -- 172.21.15.125:0/2132613969 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4003b90 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0d2ffd640 1 -- 172.21.15.125:0/2132613969 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff0d4003cf0 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc0c9c10 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0d2ffd640 1 -- 172.21.15.125:0/2132613969 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400aab0 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.864+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc0c9300 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.865+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc0c9e00 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.865+0000 7ff0d2ffd640 1 -- 172.21.15.125:0/2132613969 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff0d400ac10 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.866+0000 7ff0d2ffd640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7ff0bc06af90 0x7ff0bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:03.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.866+0000 7ff0da59c640 1 -- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7ff0bc06af90 msgr2=0x7ff0bc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:03.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.866+0000 7ff0d2ffd640 1 -- 172.21.15.125:0/2132613969 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6503+0+0 (secure 0 0 0) 0x7ff0d4089930 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.866+0000 7ff0da59c640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7ff0bc06af90 0x7ff0bc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:03.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:03.870+0000 7ff0d2ffd640 1 -- 172.21.15.125:0/2132613969 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0d40545e0 con 0x7ff0dc15b2a0 2023-10-18T10:40:03.995 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:03.994+0000 7f0061a3e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:40:04.057 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.056+0000 7fab56a25040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:40:04.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.060+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff0dc0cb9d0 con 0x7ff0dc15b2a0 2023-10-18T10:40:04.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.060+0000 7ff0d2ffd640 1 -- 172.21.15.125:0/2132613969 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v70) v1 ==== 80+0+478 (secure 0 0 0) 0x7ff0d4057c00 con 0x7ff0dc15b2a0 2023-10-18T10:40:04.062 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3263530731 2023-10-19T10:38:08.702660+0000 2023-10-18T10:40:04.063 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3471641794 2023-10-19T10:38:08.702660+0000 2023-10-18T10:40:04.063 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2137279356 2023-10-19T10:38:08.702660+0000 2023-10-18T10:40:04.063 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1884009041 2023-10-19T10:38:08.702660+0000 2023-10-18T10:40:04.063 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3401202005 2023-10-19T10:38:08.702660+0000 2023-10-18T10:40:04.064 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3643146558 2023-10-19T10:38:08.702660+0000 2023-10-18T10:40:04.064 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/2140727580 2023-10-19T10:38:08.702660+0000 2023-10-18T10:40:04.064 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/2140727580 2023-10-19T10:38:08.702660+0000 2023-10-18T10:40:04.065 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T10:40:04.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7ff0bc06af90 msgr2=0x7ff0bc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:04.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7ff0bc06af90 0x7ff0bc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc15b2a0 msgr2=0x7ff0dc0ca130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:04.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc15b2a0 0x7ff0dc0ca130 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c730 tx=0x7ff0d400cbf0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 shutdown_connections 2023-10-18T10:40:04.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7ff0bc06af90 0x7ff0bc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0dc15b2a0 0x7ff0dc0ca130 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0dc0cf340 0x7ff0dc0cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 --2- 172.21.15.125:0/2132613969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0dc0ceb40 0x7ff0dc157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 >> 172.21.15.125:0/2132613969 conn(0x7ff0dc0d0d60 msgr2=0x7ff0dc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:04.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 shutdown_connections 2023-10-18T10:40:04.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.063+0000 7ff0e0ba9640 1 -- 172.21.15.125:0/2132613969 wait complete. 2023-10-18T10:40:04.075 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:40:04.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:04.076 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T10:40:04.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[114100]: fuse finished with error 0 and tester_r 0 2023-10-18T10:40:04.103 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:40:04.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:04.104 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:40:04.127 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:04.125+0000 7f0061a3e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:40:04.166 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:40:04.166 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:04.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:40:04.182 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:40:04.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:04.183 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:40:04.197 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.196+0000 7fab56a25040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:40:04.221 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:04.219+0000 7f0061a3e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:40:04.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:40:04.235 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T10:40:04.236 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:40:04.236 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:04.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:40:04.253 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:40:04.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:04.253 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:40:04.265 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:04.263+0000 7f0061a3e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:40:04.291 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.290+0000 7fab56a25040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:40:04.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:40:04.305 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T10:40:04.306 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:40:04.306 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:04.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:40:04.322 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:40:04.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:04.322 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:40:04.335 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.334+0000 7fab56a25040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:40:04.361 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:04.360+0000 7f0061a3e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:04.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:40:04.376 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T10:40:04.376 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:04.436 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.435+0000 7fab56a25040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:04.455 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:04.454+0000 7f0061a3e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:40:04.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.479+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3273885143 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d18070350 msgr2=0x7f3d18070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:04.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.479+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3273885143 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d18070350 0x7f3d18070780 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3d180cc560 tx=0x7f3d1002c2f0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.479+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3273885143 shutdown_connections 2023-10-18T10:40:04.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.479+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3273885143 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d18070350 0x7f3d18070780 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.479+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3273885143 >> v2:172.21.15.139:3300/0 conn(0x7f3d180cb980 0x7f3d1813db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.479+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3273885143 >> 172.21.15.125:0/3273885143 conn(0x7f3d18069930 msgr2=0x7f3d18069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:04.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.479+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3273885143 shutdown_connections 2023-10-18T10:40:04.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.479+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3273885143 wait complete. 2023-10-18T10:40:04.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.481+0000 7f3d1d6c2640 1 Processor -- start 2023-10-18T10:40:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.481+0000 7f3d1d6c2640 1 -- start start 2023-10-18T10:40:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.481+0000 7f3d1d6c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d18070350 0x7f3d1813a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.481+0000 7f3d1d6c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d180cb980 0x7f3d18135ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.481+0000 7f3d1d6c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d181f0fb0 0x7f3d1813bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.481+0000 7f3d1d6c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d18136420 con 0x7f3d180cb980 2023-10-18T10:40:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.481+0000 7f3d1d6c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d18135ca0 con 0x7f3d181f0fb0 2023-10-18T10:40:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.481+0000 7f3d1d6c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d181366a0 con 0x7f3d18070350 2023-10-18T10:40:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d167fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d180cb980 0x7f3d18135ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d167fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d180cb980 0x7f3d18135ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55012/0 (socket says 172.21.15.125:55012) 2023-10-18T10:40:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d167fc640 1 -- 172.21.15.125:0/3806233301 learned_addr learned my addr 172.21.15.125:0/3806233301 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:04.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d16ffd640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d18070350 0x7f3d1813a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:04.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d177fe640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d181f0fb0 0x7f3d1813bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:04.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d177fe640 1 -- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d18070350 msgr2=0x7f3d1813a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:04.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d177fe640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d18070350 0x7f3d1813a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d177fe640 1 -- 172.21.15.125:0/3806233301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d180cb980 msgr2=0x7f3d18135ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:04.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d177fe640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d180cb980 0x7f3d18135ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d177fe640 1 -- 172.21.15.125:0/3806233301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d10009900 con 0x7f3d181f0fb0 2023-10-18T10:40:04.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d167fc640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d180cb980 0x7f3d18135ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:04.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d177fe640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d181f0fb0 0x7f3d1813bfa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3d10033000 tx=0x7f3d10035e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:04.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3806233301 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1002cea0 con 0x7f3d181f0fb0 2023-10-18T10:40:04.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3806233301 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d10004250 con 0x7f3d181f0fb0 2023-10-18T10:40:04.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3806233301 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10003ea0 con 0x7f3d181f0fb0 2023-10-18T10:40:04.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d18134ee0 con 0x7f3d181f0fb0 2023-10-18T10:40:04.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.482+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d181354a0 con 0x7f3d181f0fb0 2023-10-18T10:40:04.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.483+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d180690e0 con 0x7f3d181f0fb0 2023-10-18T10:40:04.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.484+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3806233301 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f3d1003b040 con 0x7f3d181f0fb0 2023-10-18T10:40:04.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.484+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3d0406af60 0x7f3d0406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:04.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.484+0000 7f3d16ffd640 1 -- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3d0406af60 msgr2=0x7f3d0406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:04.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.484+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3806233301 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f3d100ac4b0 con 0x7f3d181f0fb0 2023-10-18T10:40:04.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.484+0000 7f3d16ffd640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3d0406af60 0x7f3d0406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:04.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.489+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3806233301 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d100771f0 con 0x7f3d181f0fb0 2023-10-18T10:40:04.533 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.531+0000 7fab56a25040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:40:04.593 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:04.591+0000 7f0061a3e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:04.634 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:04.633+0000 7f0061a3e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:04.674 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.672+0000 7fab56a25040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:04.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.685+0000 7f3d16ffd640 1 -- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3d0406af60 msgr2=0x7f3d0406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:04.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.685+0000 7f3d16ffd640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3d0406af60 0x7f3d0406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:40:04.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.699+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d180c9210 con 0x7f3d181f0fb0 2023-10-18T10:40:04.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.700+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3806233301 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f3d1007a810 con 0x7f3d181f0fb0 2023-10-18T10:40:04.702 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:04.703 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1045257948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1045257948},{"type":"v1","addr":"172.21.15.139:6836","nonce":1045257948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/4183044691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4183044691},{"type":"v1","addr":"172.21.15.139:6837","nonce":4183044691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1083462482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1083462482},{"type":"v1","addr":"172.21.15.125:6835","nonce":1083462482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:37:57.277572+0000","modified":"2023-10-18T10:37:59.669781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/615856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":615856560},{"type":"v1","addr":"172.21.15.125:6834","nonce":615856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T10:40:04.703 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 23 2023-10-18T10:40:04.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3d0406af60 msgr2=0x7f3d0406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:04.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3d0406af60 0x7f3d0406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d181f0fb0 msgr2=0x7f3d1813bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:04.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d181f0fb0 0x7f3d1813bfa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3d10033000 tx=0x7f3d10035e50 comp rx=0 tx=0).stop 2023-10-18T10:40:04.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 shutdown_connections 2023-10-18T10:40:04.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f3d0406af60 0x7f3d0406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d181f0fb0 0x7f3d1813bfa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d180cb980 0x7f3d18135ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 --2- 172.21.15.125:0/3806233301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d18070350 0x7f3d1813a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 >> 172.21.15.125:0/3806233301 conn(0x7f3d18069930 msgr2=0x7f3d18137200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:04.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 shutdown_connections 2023-10-18T10:40:04.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.702+0000 7f3d1d6c2640 1 -- 172.21.15.125:0/3806233301 wait complete. 2023-10-18T10:40:04.715 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:04.718 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.716+0000 7fab56a25040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:04.737 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:04.736+0000 7f0061a3e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:40:04.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.815+0000 7feec518e640 1 -- 172.21.15.125:0/2931969244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec00d1910 msgr2=0x7feec00b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:04.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.815+0000 7feec518e640 1 --2- 172.21.15.125:0/2931969244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec00d1910 0x7feec00b0650 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7feeb8005fd0 tx=0x7feeb802c0e0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.815+0000 7feec518e640 1 -- 172.21.15.125:0/2931969244 shutdown_connections 2023-10-18T10:40:04.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.815+0000 7feec518e640 1 --2- 172.21.15.125:0/2931969244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec00d1910 0x7feec00b0650 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.815+0000 7feec518e640 1 -- 172.21.15.125:0/2931969244 >> 172.21.15.125:0/2931969244 conn(0x7feec00d0d60 msgr2=0x7feec00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:04.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.815+0000 7feec518e640 1 -- 172.21.15.125:0/2931969244 shutdown_connections 2023-10-18T10:40:04.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.815+0000 7feec518e640 1 -- 172.21.15.125:0/2931969244 wait complete. 2023-10-18T10:40:04.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.817+0000 7feec518e640 1 Processor -- start 2023-10-18T10:40:04.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.817+0000 7feec518e640 1 -- start start 2023-10-18T10:40:04.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.817+0000 7feec518e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feec00cf340 0x7feec014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:04.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.817+0000 7feec518e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feec00d1910 0x7feec015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:04.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.817+0000 7feec518e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec015b2a0 0x7feec006aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:04.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.817+0000 7feec518e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec015c330 con 0x7feec015b2a0 2023-10-18T10:40:04.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.817+0000 7feec518e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec015e570 con 0x7feec00cf340 2023-10-18T10:40:04.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.817+0000 7feec518e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec015b720 con 0x7feec00d1910 2023-10-18T10:40:04.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec015b2a0 0x7feec006aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:04.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec015b2a0 0x7feec006aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55020/0 (socket says 172.21.15.125:55020) 2023-10-18T10:40:04.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 -- 172.21.15.125:0/229185653 learned_addr learned my addr 172.21.15.125:0/229185653 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:04.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebed9d640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feec00cf340 0x7feec014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:04.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feeb7fff640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feec00d1910 0x7feec015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:04.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 -- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feec00d1910 msgr2=0x7feec015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:04.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feec00d1910 0x7feec015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 -- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feec00cf340 msgr2=0x7feec014f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:04.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feec00cf340 0x7feec014f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:04.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 -- 172.21.15.125:0/229185653 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb8009840 con 0x7feec015b2a0 2023-10-18T10:40:04.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feebf59e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec015b2a0 0x7feec006aa40 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7feeb8036040 tx=0x7feeb8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:04.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feeb77fe640 1 -- 172.21.15.125:0/229185653 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8004670 con 0x7feec015b2a0 2023-10-18T10:40:04.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feeb77fe640 1 -- 172.21.15.125:0/229185653 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feeb80345a0 con 0x7feec015b2a0 2023-10-18T10:40:04.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feeb77fe640 1 -- 172.21.15.125:0/229185653 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb80349b0 con 0x7feec015b2a0 2023-10-18T10:40:04.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec015ba00 con 0x7feec015b2a0 2023-10-18T10:40:04.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.818+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec015bfc0 con 0x7feec015b2a0 2023-10-18T10:40:04.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.819+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec015c180 con 0x7feec015b2a0 2023-10-18T10:40:04.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.820+0000 7feeb77fe640 1 -- 172.21.15.125:0/229185653 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7feeb8034b10 con 0x7feec015b2a0 2023-10-18T10:40:04.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.820+0000 7feeb77fe640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feea006af60 0x7feea006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:04.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.820+0000 7feebed9d640 1 -- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feea006af60 msgr2=0x7feea006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:04.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.820+0000 7feeb77fe640 1 -- 172.21.15.125:0/229185653 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6503+0+0 (secure 0 0 0) 0x7feeb80acb40 con 0x7feec015b2a0 2023-10-18T10:40:04.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.820+0000 7feebed9d640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feea006af60 0x7feea006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:04.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.824+0000 7feeb77fe640 1 -- 172.21.15.125:0/229185653 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feeb80777d0 con 0x7feec015b2a0 2023-10-18T10:40:04.829 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:04.827+0000 7fab56a25040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:40:04.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.996+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feec00d04f0 con 0x7feec015b2a0 2023-10-18T10:40:04.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.997+0000 7feeb77fe640 1 -- 172.21.15.125:0/229185653 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17502 (secure 0 0 0) 0x7feeb8043090 con 0x7feec015b2a0 2023-10-18T10:40:04.999 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:05.000 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":70,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:03.370494+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:37:55.100958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-18T10:37:56.256769+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3263530731":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3471641794":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/2137279356":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/1884009041":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3401202005":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3643146558":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6833/2140727580":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6832/2140727580":"2023-10-19T10:38:08.702660+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:05.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feea006af60 msgr2=0x7feea006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feea006af60 0x7feea006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec015b2a0 msgr2=0x7feec006aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec015b2a0 0x7feec006aa40 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7feeb8036040 tx=0x7feeb8004270 comp rx=0 tx=0).stop 2023-10-18T10:40:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 shutdown_connections 2023-10-18T10:40:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feea006af60 0x7feea006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feec015b2a0 0x7feec006aa40 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feec00d1910 0x7feec015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 --2- 172.21.15.125:0/229185653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feec00cf340 0x7feec014f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:04.999+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 >> 172.21.15.125:0/229185653 conn(0x7feec00d0d60 msgr2=0x7feec006b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:05.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.000+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 shutdown_connections 2023-10-18T10:40:05.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.000+0000 7feec518e640 1 -- 172.21.15.125:0/229185653 wait complete. 2023-10-18T10:40:05.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:37:55.100958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:37:56.256769+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:40:05.013 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:40:05.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.156+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/832184688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc815b2a0 msgr2=0x7f5dc80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.156+0000 7f5dcd1a9640 1 --2- 172.21.15.125:0/832184688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc815b2a0 0x7f5dc80d1910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009830 tx=0x7f5dbc02bd60 comp rx=0 tx=0).stop 2023-10-18T10:40:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.157+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/832184688 shutdown_connections 2023-10-18T10:40:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.157+0000 7f5dcd1a9640 1 --2- 172.21.15.125:0/832184688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc815b2a0 0x7f5dc80d1910 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.157+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/832184688 >> 172.21.15.125:0/832184688 conn(0x7f5dc80d0d60 msgr2=0x7f5dc80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:05.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.157+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/832184688 shutdown_connections 2023-10-18T10:40:05.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.157+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/832184688 wait complete. 2023-10-18T10:40:05.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.158+0000 7f5dcd1a9640 1 Processor -- start 2023-10-18T10:40:05.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.158+0000 7f5dcd1a9640 1 -- start start 2023-10-18T10:40:05.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dcd1a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc80ceb40 0x7f5dc81e4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dcd1a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc80cf340 0x7f5dc8158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dcd1a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc815b2a0 0x7f5dc815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dcd1a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8158b40 con 0x7f5dc815b2a0 2023-10-18T10:40:05.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dcd1a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc815cf70 con 0x7f5dc80ceb40 2023-10-18T10:40:05.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dcd1a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc815d250 con 0x7f5dc80cf340 2023-10-18T10:40:05.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc815b2a0 0x7f5dc815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc815b2a0 0x7f5dc815c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55030/0 (socket says 172.21.15.125:55030) 2023-10-18T10:40:05.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc759e640 1 -- 172.21.15.125:0/4226104738 learned_addr learned my addr 172.21.15.125:0/4226104738 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:05.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc6d9d640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc80ceb40 0x7f5dc81e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc659c640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc80cf340 0x7f5dc8158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc6d9d640 1 -- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc80cf340 msgr2=0x7f5dc8158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc6d9d640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc80cf340 0x7f5dc8158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc6d9d640 1 -- 172.21.15.125:0/4226104738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc815b2a0 msgr2=0x7f5dc815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc6d9d640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc815b2a0 0x7f5dc815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc6d9d640 1 -- 172.21.15.125:0/4226104738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc009490 con 0x7f5dc80ceb40 2023-10-18T10:40:05.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc759e640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc815b2a0 0x7f5dc815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:05.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.159+0000 7f5dc6d9d640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc80ceb40 0x7f5dc81e4610 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c3c0 tx=0x7f5db800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:05.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.160+0000 7f5daf7fe640 1 -- 172.21.15.125:0/4226104738 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db80052d0 con 0x7f5dc80ceb40 2023-10-18T10:40:05.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.160+0000 7f5daf7fe640 1 -- 172.21.15.125:0/4226104738 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5db80035f0 con 0x7f5dc80ceb40 2023-10-18T10:40:05.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.160+0000 7f5daf7fe640 1 -- 172.21.15.125:0/4226104738 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8003a00 con 0x7f5dc80ceb40 2023-10-18T10:40:05.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.160+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc815d530 con 0x7f5dc80ceb40 2023-10-18T10:40:05.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.160+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc815daf0 con 0x7f5dc80ceb40 2023-10-18T10:40:05.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.161+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc80d04f0 con 0x7f5dc80ceb40 2023-10-18T10:40:05.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.161+0000 7f5daf7fe640 1 -- 172.21.15.125:0/4226104738 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f5db8003b60 con 0x7f5dc80ceb40 2023-10-18T10:40:05.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.162+0000 7f5daf7fe640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f5db406af90 0x7f5db406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.162+0000 7f5dc659c640 1 -- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f5db406af90 msgr2=0x7f5db406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:05.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.162+0000 7f5daf7fe640 1 -- 172.21.15.125:0/4226104738 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f5db80891c0 con 0x7f5dc80ceb40 2023-10-18T10:40:05.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.162+0000 7f5dc659c640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f5db406af90 0x7f5db406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:05.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.170+0000 7f5daf7fe640 1 -- 172.21.15.125:0/4226104738 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5db8053f00 con 0x7f5dc80ceb40 2023-10-18T10:40:05.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.343+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5dc80677b0 con 0x7f5dc80ceb40 2023-10-18T10:40:05.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.343+0000 7f5daf7fe640 1 -- 172.21.15.125:0/4226104738 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5db8057520 con 0x7f5dc80ceb40 2023-10-18T10:40:05.345 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:40:05.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.345+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f5db406af90 msgr2=0x7f5db406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:05.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.345+0000 7f5dcd1a9640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f5db406af90 0x7f5db406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.345+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc80ceb40 msgr2=0x7f5dc81e4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.345+0000 7f5dcd1a9640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc80ceb40 0x7f5dc81e4610 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c3c0 tx=0x7f5db800c880 comp rx=0 tx=0).stop 2023-10-18T10:40:05.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.346+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 shutdown_connections 2023-10-18T10:40:05.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.346+0000 7f5dcd1a9640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f5db406af90 0x7f5db406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.346+0000 7f5dcd1a9640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc815b2a0 0x7f5dc815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.346+0000 7f5dcd1a9640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc80cf340 0x7f5dc8158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.346+0000 7f5dcd1a9640 1 --2- 172.21.15.125:0/4226104738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc80ceb40 0x7f5dc81e4610 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.346+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 >> 172.21.15.125:0/4226104738 conn(0x7f5dc80d0d60 msgr2=0x7f5dc806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:05.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.346+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 shutdown_connections 2023-10-18T10:40:05.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.346+0000 7f5dcd1a9640 1 -- 172.21.15.125:0/4226104738 wait complete. 2023-10-18T10:40:05.350 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.347+0000 7f0061a3e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:40:05.358 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:40:05.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.463+0000 7f049671d640 1 -- 172.21.15.125:0/1708401866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f049015b2a0 msgr2=0x7f04900cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.463+0000 7f049671d640 1 --2- 172.21.15.125:0/1708401866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f049015b2a0 0x7f04900cf720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0480009830 tx=0x7f048002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.463+0000 7f049671d640 1 -- 172.21.15.125:0/1708401866 shutdown_connections 2023-10-18T10:40:05.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.463+0000 7f049671d640 1 --2- 172.21.15.125:0/1708401866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f049015b2a0 0x7f04900cf720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.463+0000 7f049671d640 1 --2- 172.21.15.125:0/1708401866 >> v2:172.21.15.139:3300/0 conn(0x7f04900cf340 0x7f04900d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.463+0000 7f049671d640 1 -- 172.21.15.125:0/1708401866 >> 172.21.15.125:0/1708401866 conn(0x7f04900d0d60 msgr2=0x7f04900d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:05.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.463+0000 7f049671d640 1 -- 172.21.15.125:0/1708401866 shutdown_connections 2023-10-18T10:40:05.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.463+0000 7f049671d640 1 -- 172.21.15.125:0/1708401866 wait complete. 2023-10-18T10:40:05.468 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:40:05.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.465+0000 7f049671d640 1 Processor -- start 2023-10-18T10:40:05.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.465+0000 7f049671d640 1 -- start start 2023-10-18T10:40:05.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.465+0000 7f049671d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04900ceb40 0x7f0490157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.465+0000 7f049671d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04900cf340 0x7f04900b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.465+0000 7f049671d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f049015b2a0 0x7f0490157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.465+0000 7f049671d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049015c850 con 0x7f04900ceb40 2023-10-18T10:40:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.465+0000 7f049671d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04900b5910 con 0x7f049015b2a0 2023-10-18T10:40:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.465+0000 7f049671d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04900b5bf0 con 0x7f04900cf340 2023-10-18T10:40:05.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04900ceb40 0x7f0490157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048f7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04900cf340 0x7f04900b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f0494cba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f049015b2a0 0x7f0490157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04900ceb40 0x7f0490157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55038/0 (socket says 172.21.15.125:55038) 2023-10-18T10:40:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048f7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04900cf340 0x7f04900b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:39256/0 (socket says 172.21.15.125:39256) 2023-10-18T10:40:05.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048ffff640 1 -- 172.21.15.125:0/3906383290 learned_addr learned my addr 172.21.15.125:0/3906383290 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f0494cba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f049015b2a0 0x7f0490157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:59822/0 (socket says 172.21.15.125:59822) 2023-10-18T10:40:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048f7fe640 1 -- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f049015b2a0 msgr2=0x7f0490157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048f7fe640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f049015b2a0 0x7f0490157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048f7fe640 1 -- 172.21.15.125:0/3906383290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04900ceb40 msgr2=0x7f0490157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048f7fe640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04900ceb40 0x7f0490157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048f7fe640 1 -- 172.21.15.125:0/3906383290 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0484009560 con 0x7f04900cf340 2023-10-18T10:40:05.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f0494cba640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f049015b2a0 0x7f0490157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:05.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048f7fe640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04900cf340 0x7f04900b5280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0480033000 tx=0x7f04800038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:05.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048cff9640 1 -- 172.21.15.125:0/3906383290 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480003ea0 con 0x7f04900cf340 2023-10-18T10:40:05.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048cff9640 1 -- 172.21.15.125:0/3906383290 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0480034440 con 0x7f04900cf340 2023-10-18T10:40:05.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0480009490 con 0x7f04900cf340 2023-10-18T10:40:05.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f048cff9640 1 -- 172.21.15.125:0/3906383290 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480034830 con 0x7f04900cf340 2023-10-18T10:40:05.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.466+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049015caa0 con 0x7f04900cf340 2023-10-18T10:40:05.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.468+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04900b6110 con 0x7f04900cf340 2023-10-18T10:40:05.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.468+0000 7f048cff9640 1 -- 172.21.15.125:0/3906383290 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f0480034990 con 0x7f04900cf340 2023-10-18T10:40:05.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.468+0000 7f048cff9640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f047c06af60 0x7f047c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.468+0000 7f048ffff640 1 -- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f047c06af60 msgr2=0x7f047c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:05.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.468+0000 7f048cff9640 1 -- 172.21.15.125:0/3906383290 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6503+0+0 (secure 0 0 0) 0x7f04800ac1a0 con 0x7f04900cf340 2023-10-18T10:40:05.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.468+0000 7f048ffff640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f047c06af60 0x7f047c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:05.478 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:40:05.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.473+0000 7f048cff9640 1 -- 172.21.15.125:0/3906383290 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0480076ed0 con 0x7f04900cf340 2023-10-18T10:40:05.479 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:05.475+0000 7fab56a25040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:40:05.619 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.616+0000 7f0061a3e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:40:05.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.646+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04900d04f0 con 0x7f04900cf340 2023-10-18T10:40:05.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.647+0000 7f048cff9640 1 -- 172.21.15.125:0/3906383290 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f048007a4f0 con 0x7f04900cf340 2023-10-18T10:40:05.651 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:40:05.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f047c06af60 msgr2=0x7f047c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:05.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f047c06af60 0x7f047c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04900cf340 msgr2=0x7f04900b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04900cf340 0x7f04900b5280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0480033000 tx=0x7f04800038a0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 shutdown_connections 2023-10-18T10:40:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7f047c06af60 0x7f047c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f049015b2a0 0x7f0490157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f04900cf340 0x7f04900b5280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 --2- 172.21.15.125:0/3906383290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04900ceb40 0x7f0490157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 >> 172.21.15.125:0/3906383290 conn(0x7f04900d0d60 msgr2=0x7f049015a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 shutdown_connections 2023-10-18T10:40:05.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.649+0000 7f049671d640 1 -- 172.21.15.125:0/3906383290 wait complete. 2023-10-18T10:40:05.662 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:40:05.667 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.665+0000 7f0061a3e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:40:05.751 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:05.749+0000 7fab56a25040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:40:05.766 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.763+0000 7f0061a3e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:40:05.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.766+0000 7feb97064640 1 -- 172.21.15.125:0/2880360560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb9015b2a0 msgr2=0x7feb900d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.766+0000 7feb97064640 1 --2- 172.21.15.125:0/2880360560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb9015b2a0 0x7feb900d1910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feb84009860 tx=0x7feb8402bd60 comp rx=0 tx=0).stop 2023-10-18T10:40:05.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.766+0000 7feb97064640 1 -- 172.21.15.125:0/2880360560 shutdown_connections 2023-10-18T10:40:05.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.766+0000 7feb97064640 1 --2- 172.21.15.125:0/2880360560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb9015b2a0 0x7feb900d1910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.766+0000 7feb97064640 1 -- 172.21.15.125:0/2880360560 >> 172.21.15.125:0/2880360560 conn(0x7feb900d0d60 msgr2=0x7feb900d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:05.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.766+0000 7feb97064640 1 -- 172.21.15.125:0/2880360560 shutdown_connections 2023-10-18T10:40:05.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.766+0000 7feb97064640 1 -- 172.21.15.125:0/2880360560 wait complete. 2023-10-18T10:40:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.767+0000 7feb97064640 1 Processor -- start 2023-10-18T10:40:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.767+0000 7feb97064640 1 -- start start 2023-10-18T10:40:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb97064640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb900ceb40 0x7feb901580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb97064640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb900cf340 0x7feb9014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb97064640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb9015b2a0 0x7feb900b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb97064640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb900c8550 con 0x7feb9015b2a0 2023-10-18T10:40:05.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb97064640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb900b5080 con 0x7feb900ceb40 2023-10-18T10:40:05.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb97064640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb900c7fd0 con 0x7feb900cf340 2023-10-18T10:40:05.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb95601640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb9015b2a0 0x7feb900b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb95601640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb9015b2a0 0x7feb900b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55054/0 (socket says 172.21.15.125:55054) 2023-10-18T10:40:05.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.768+0000 7feb95601640 1 -- 172.21.15.125:0/1379944021 learned_addr learned my addr 172.21.15.125:0/1379944021 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:05.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb8ffff640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb900cf340 0x7feb9014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb94e00640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb900ceb40 0x7feb901580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:05.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb95601640 1 -- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb900cf340 msgr2=0x7feb9014f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb95601640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb900cf340 0x7feb9014f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb95601640 1 -- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb900ceb40 msgr2=0x7feb901580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb95601640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb900ceb40 0x7feb901580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb95601640 1 -- 172.21.15.125:0/1379944021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb84009490 con 0x7feb9015b2a0 2023-10-18T10:40:05.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb8ffff640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb900cf340 0x7feb9014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb95601640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb9015b2a0 0x7feb900b4b40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7feb8800cbc0 tx=0x7feb8800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800ad40 con 0x7feb9015b2a0 2023-10-18T10:40:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb88012420 con 0x7feb9015b2a0 2023-10-18T10:40:05.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb900c8ec0 con 0x7feb9015b2a0 2023-10-18T10:40:05.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88012810 con 0x7feb9015b2a0 2023-10-18T10:40:05.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.769+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb900c8270 con 0x7feb9015b2a0 2023-10-18T10:40:05.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.770+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb900c7e30 con 0x7feb9015b2a0 2023-10-18T10:40:05.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.771+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7feb8801a3b0 con 0x7feb9015b2a0 2023-10-18T10:40:05.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.771+0000 7feb8d7fa640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feb7406af60 0x7feb7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:05.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.771+0000 7feb94e00640 1 -- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feb7406af60 msgr2=0x7feb7406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/397604228 2023-10-18T10:40:05.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.771+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6503+0+0 (secure 0 0 0) 0x7feb8808a130 con 0x7feb9015b2a0 2023-10-18T10:40:05.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.771+0000 7feb94e00640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feb7406af60 0x7feb7406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:05.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.775+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb900c7e30 con 0x7feb9015b2a0 2023-10-18T10:40:05.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.786+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 29) v1 ==== 88940+0+0 (secure 0 0 0) 0x7feb880552c0 con 0x7feb9015b2a0 2023-10-18T10:40:05.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.786+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feb7406af60 msgr2=0x7feb7406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:05.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.786+0000 7feb8d7fa640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feb7406af60 0x7feb7406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.807 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:05.805+0000 7fab56a25040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:40:05.841 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.838+0000 7effe3d6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.842 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.838+0000 7effe3d6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.842 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.838+0000 7effe3d6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.842 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.838+0000 7effe3d6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.842 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.838+0000 7effe3d6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.843 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.839+0000 7effe6d71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.843 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.839+0000 7effe6d71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.843 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.839+0000 7effe6d71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.843 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.839+0000 7effe6d71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.844 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:05.839+0000 7effe6d71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:05.912 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:05.910+0000 7fab56a25040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:40:05.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.955+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb9006af10 con 0x7feb9015b2a0 2023-10-18T10:40:05.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.955+0000 7feb8d7fa640 1 -- 172.21.15.125:0/1379944021 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7feb9006af10 con 0x7feb9015b2a0 2023-10-18T10:40:05.958 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:40:05.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.957+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb9015b2a0 msgr2=0x7feb900b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.957+0000 7feb97064640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb9015b2a0 0x7feb900b4b40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7feb8800cbc0 tx=0x7feb8800a430 comp rx=0 tx=0).stop 2023-10-18T10:40:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.958+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 shutdown_connections 2023-10-18T10:40:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.958+0000 7feb97064640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:6832/397604228,v1:172.21.15.139:6833/397604228] conn(0x7feb7406af60 0x7feb7406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.958+0000 7feb97064640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb9015b2a0 0x7feb900b4b40 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.958+0000 7feb97064640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb900cf340 0x7feb9014f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.958+0000 7feb97064640 1 --2- 172.21.15.125:0/1379944021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb900ceb40 0x7feb901580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:05.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.958+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 >> 172.21.15.125:0/1379944021 conn(0x7feb900d0d60 msgr2=0x7feb9015a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:05.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.958+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 shutdown_connections 2023-10-18T10:40:05.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:05.958+0000 7feb97064640 1 -- 172.21.15.125:0/1379944021 wait complete. 2023-10-18T10:40:05.970 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:06.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.072+0000 7f9284105640 1 -- 172.21.15.125:0/2453015937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f927c140660 msgr2=0x7f927c13d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.072+0000 7f9284105640 1 --2- 172.21.15.125:0/2453015937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f927c140660 0x7f927c13d360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f926c009830 tx=0x7f926c02bd60 comp rx=0 tx=0).stop 2023-10-18T10:40:06.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.072+0000 7f9284105640 1 -- 172.21.15.125:0/2453015937 shutdown_connections 2023-10-18T10:40:06.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.072+0000 7f9284105640 1 --2- 172.21.15.125:0/2453015937 >> v2:172.21.15.139:3300/0 conn(0x7f927c0c97c0 0x7f927c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.072+0000 7f9284105640 1 --2- 172.21.15.125:0/2453015937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f927c140660 0x7f927c13d360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.072+0000 7f9284105640 1 -- 172.21.15.125:0/2453015937 >> 172.21.15.125:0/2453015937 conn(0x7f927c069930 msgr2=0x7f927c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:06.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.072+0000 7f9284105640 1 -- 172.21.15.125:0/2453015937 shutdown_connections 2023-10-18T10:40:06.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.072+0000 7f9284105640 1 -- 172.21.15.125:0/2453015937 wait complete. 2023-10-18T10:40:06.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.073+0000 7f9284105640 1 Processor -- start 2023-10-18T10:40:06.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.073+0000 7f9284105640 1 -- start start 2023-10-18T10:40:06.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f9284105640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f927c06d170 0x7f927c1331a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f9284105640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f927c0c97c0 0x7f927c06b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f9284105640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f927c140660 0x7f927c06be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f9284105640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c13c090 con 0x7f927c140660 2023-10-18T10:40:06.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f9284105640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c13c370 con 0x7f927c0c97c0 2023-10-18T10:40:06.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f9284105640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c13c650 con 0x7f927c06d170 2023-10-18T10:40:06.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f927c140660 0x7f927c06be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f927c140660 0x7f927c06be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55058/0 (socket says 172.21.15.125:55058) 2023-10-18T10:40:06.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 -- 172.21.15.125:0/342782126 learned_addr learned my addr 172.21.15.125:0/342782126 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:06.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f9281ea1640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f927c06d170 0x7f927c1331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92816a0640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f927c0c97c0 0x7f927c06b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 -- 172.21.15.125:0/342782126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f927c06d170 msgr2=0x7f927c1331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f927c06d170 0x7f927c1331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 -- 172.21.15.125:0/342782126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f927c0c97c0 msgr2=0x7f927c06b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f927c0c97c0 0x7f927c06b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 -- 172.21.15.125:0/342782126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009840 con 0x7f927c140660 2023-10-18T10:40:06.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92826a2640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f927c140660 0x7f927c06be50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f927400efd0 tx=0x7f927400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:06.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92727fc640 1 -- 172.21.15.125:0/342782126 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274009b40 con 0x7f927c140660 2023-10-18T10:40:06.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.074+0000 7f92727fc640 1 -- 172.21.15.125:0/342782126 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9274003c60 con 0x7f927c140660 2023-10-18T10:40:06.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.075+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c009490 con 0x7f927c140660 2023-10-18T10:40:06.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.075+0000 7f92727fc640 1 -- 172.21.15.125:0/342782126 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274004020 con 0x7f927c140660 2023-10-18T10:40:06.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.075+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c13bd40 con 0x7f927c140660 2023-10-18T10:40:06.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.076+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927c0690e0 con 0x7f927c140660 2023-10-18T10:40:06.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.076+0000 7f92727fc640 1 -- 172.21.15.125:0/342782126 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 29) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f92740041a0 con 0x7f927c140660 2023-10-18T10:40:06.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.076+0000 7f92727fc640 1 -- 172.21.15.125:0/342782126 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f9274014070 con 0x7f927c140660 2023-10-18T10:40:06.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.080+0000 7f92727fc640 1 -- 172.21.15.125:0/342782126 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92740555f0 con 0x7f927c140660 2023-10-18T10:40:06.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.278+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f927c140200 con 0x7f927c140660 2023-10-18T10:40:06.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.279+0000 7f92727fc640 1 -- 172.21.15.125:0/342782126 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f9274058c10 con 0x7f927c140660 2023-10-18T10:40:06.281 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:06.282 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1045257948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1045257948},{"type":"v1","addr":"172.21.15.139:6836","nonce":1045257948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/4183044691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4183044691},{"type":"v1","addr":"172.21.15.139:6837","nonce":4183044691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1083462482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1083462482},{"type":"v1","addr":"172.21.15.125:6835","nonce":1083462482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:37:57.277572+0000","modified":"2023-10-18T10:37:59.669781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/615856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":615856560},{"type":"v1","addr":"172.21.15.125:6834","nonce":615856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T10:40:06.282 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 23 2023-10-18T10:40:06.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f927c140660 msgr2=0x7f927c06be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f927c140660 0x7f927c06be50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f927400efd0 tx=0x7f927400c6a0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 shutdown_connections 2023-10-18T10:40:06.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f927c0c97c0 0x7f927c06b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f927c140660 0x7f927c06be50 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 --2- 172.21.15.125:0/342782126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f927c06d170 0x7f927c1331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 >> 172.21.15.125:0/342782126 conn(0x7f927c069930 msgr2=0x7f927c157f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:06.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 shutdown_connections 2023-10-18T10:40:06.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.281+0000 7f9284105640 1 -- 172.21.15.125:0/342782126 wait complete. 2023-10-18T10:40:06.293 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:06.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.394+0000 7f9c89ef2640 1 -- 172.21.15.125:0/2264426820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 msgr2=0x7f9c6c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.394+0000 7f9c89ef2640 1 --2- 172.21.15.125:0/2264426820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 0x7f9c6c003d90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c290 tx=0x7f9c6c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.394+0000 7f9c83fff640 1 -- 172.21.15.125:0/2264426820 reap_dead start 2023-10-18T10:40:06.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.394+0000 7f9c89ef2640 1 -- 172.21.15.125:0/2264426820 shutdown_connections 2023-10-18T10:40:06.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.394+0000 7f9c89ef2640 1 -- 172.21.15.125:0/2264426820 >> 172.21.15.125:0/2264426820 conn(0x7f9c8406fae0 msgr2=0x7f9c8406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:06.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.394+0000 7f9c89ef2640 1 -- 172.21.15.125:0/2264426820 shutdown_connections 2023-10-18T10:40:06.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.394+0000 7f9c89ef2640 1 -- 172.21.15.125:0/2264426820 wait complete. 2023-10-18T10:40:06.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.395+0000 7f9c89ef2640 1 Processor -- start 2023-10-18T10:40:06.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.395+0000 7f9c89ef2640 1 -- start start 2023-10-18T10:40:06.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c89ef2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 0x7f9c8413ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c89ef2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c841e2a80 0x7f9c8414ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c89ef2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c841e2f00 0x7f9c8414f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c89ef2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8413f380 con 0x7f9c6c003970 2023-10-18T10:40:06.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c89ef2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841e1700 con 0x7f9c841e2a80 2023-10-18T10:40:06.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c89ef2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841e19e0 con 0x7f9c841e2f00 2023-10-18T10:40:06.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 0x7f9c8413ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 0x7f9c8413ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55080/0 (socket says 172.21.15.125:55080) 2023-10-18T10:40:06.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 -- 172.21.15.125:0/3391080008 learned_addr learned my addr 172.21.15.125:0/3391080008 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:06.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c83fff640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c841e2f00 0x7f9c8414f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c82ffd640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c841e2a80 0x7f9c8414ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 -- 172.21.15.125:0/3391080008 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c841e2f00 msgr2=0x7f9c8414f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c841e2f00 0x7f9c8414f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 -- 172.21.15.125:0/3391080008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c841e2a80 msgr2=0x7f9c8414ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c841e2a80 0x7f9c8414ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 -- 172.21.15.125:0/3391080008 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c840b2330 con 0x7f9c6c003970 2023-10-18T10:40:06.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.396+0000 7f9c837fe640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 0x7f9c8413ee40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9c74000c00 tx=0x7f9c74001660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:06.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.397+0000 7f9c6bfff640 1 -- 172.21.15.125:0/3391080008 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74006230 con 0x7f9c6c003970 2023-10-18T10:40:06.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.397+0000 7f9c6bfff640 1 -- 172.21.15.125:0/3391080008 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9c740063b0 con 0x7f9c6c003970 2023-10-18T10:40:06.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.397+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c841e1cc0 con 0x7f9c6c003970 2023-10-18T10:40:06.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.397+0000 7f9c6bfff640 1 -- 172.21.15.125:0/3391080008 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c740035f0 con 0x7f9c6c003970 2023-10-18T10:40:06.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.397+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c841e2280 con 0x7f9c6c003970 2023-10-18T10:40:06.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.398+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c840690e0 con 0x7f9c6c003970 2023-10-18T10:40:06.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.398+0000 7f9c6bfff640 1 -- 172.21.15.125:0/3391080008 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 29) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f9c74031030 con 0x7f9c6c003970 2023-10-18T10:40:06.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.399+0000 7f9c6bfff640 1 -- 172.21.15.125:0/3391080008 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f9c740aa1f0 con 0x7f9c6c003970 2023-10-18T10:40:06.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.403+0000 7f9c6bfff640 1 -- 172.21.15.125:0/3391080008 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c740752d0 con 0x7f9c6c003970 2023-10-18T10:40:06.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.571+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c840b08f0 con 0x7f9c6c003970 2023-10-18T10:40:06.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.573+0000 7f9c6bfff640 1 -- 172.21.15.125:0/3391080008 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17943 (secure 0 0 0) 0x7f9c740788f0 con 0x7f9c6c003970 2023-10-18T10:40:06.575 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:06.575 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":71,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:05.779594+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:37:55.100958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-18T10:37:56.256769+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3728065252":"2023-10-19T10:40:05.779561+0000","172.21.15.139:6832/397604228":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/1729948720":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2802553135":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2678895150":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/3263530731":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6833/397604228":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/3471641794":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/1871659361":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2137279356":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/1884009041":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3401202005":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3643146558":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6833/2140727580":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/751932982":"2023-10-19T10:40:05.779561+0000","172.21.15.139:6832/2140727580":"2023-10-19T10:38:08.702660+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:06.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 msgr2=0x7f9c8413ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 0x7f9c8413ee40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9c74000c00 tx=0x7f9c74001660 comp rx=0 tx=0).stop 2023-10-18T10:40:06.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 shutdown_connections 2023-10-18T10:40:06.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c841e2a80 0x7f9c8414ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c841e2f00 0x7f9c8414f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 --2- 172.21.15.125:0/3391080008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c6c003970 0x7f9c8413ee40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 >> 172.21.15.125:0/3391080008 conn(0x7f9c8406fae0 msgr2=0x7f9c8415a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:06.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 shutdown_connections 2023-10-18T10:40:06.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.575+0000 7f9c89ef2640 1 -- 172.21.15.125:0/3391080008 wait complete. 2023-10-18T10:40:06.588 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:40:06.588 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:06.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.694+0000 7f6984807640 1 -- 172.21.15.125:0/4151355745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69680047c0 msgr2=0x7f6968004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.694+0000 7f6984807640 1 --2- 172.21.15.125:0/4151355745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69680047c0 0x7f6968004c40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6974001680 tx=0x7f69680057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.694+0000 7f6984807640 1 -- 172.21.15.125:0/4151355745 shutdown_connections 2023-10-18T10:40:06.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.694+0000 7f6984807640 1 --2- 172.21.15.125:0/4151355745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69680047c0 0x7f6968004c40 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.694+0000 7f6984807640 1 -- 172.21.15.125:0/4151355745 >> 172.21.15.125:0/4151355745 conn(0x7f6980069930 msgr2=0x7f6980069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:06.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.694+0000 7f6984807640 1 -- 172.21.15.125:0/4151355745 shutdown_connections 2023-10-18T10:40:06.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.694+0000 7f6984807640 1 -- 172.21.15.125:0/4151355745 wait complete. 2023-10-18T10:40:06.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.695+0000 7f6984807640 1 Processor -- start 2023-10-18T10:40:06.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.695+0000 7f6984807640 1 -- start start 2023-10-18T10:40:06.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f6984807640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69680047c0 0x7f6980157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f6984807640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6980154210 0x7f69801552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f6984807640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6980155810 0x7f69800b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f6984807640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6980157df0 con 0x7f6980154210 2023-10-18T10:40:06.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f6984807640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69801578c0 con 0x7f6980155810 2023-10-18T10:40:06.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f6984807640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698012b2e0 con 0x7f69680047c0 2023-10-18T10:40:06.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6980154210 0x7f69801552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6980154210 0x7f69801552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55104/0 (socket says 172.21.15.125:55104) 2023-10-18T10:40:06.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697dd9b640 1 -- 172.21.15.125:0/433973433 learned_addr learned my addr 172.21.15.125:0/433973433 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:06.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697dd9b640 1 -- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69680047c0 msgr2=0x7f6980157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697dd9b640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69680047c0 0x7f6980157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697ed9d640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6980155810 0x7f69800b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697dd9b640 1 -- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6980155810 msgr2=0x7f69800b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697dd9b640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6980155810 0x7f69800b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.696+0000 7f697dd9b640 1 -- 172.21.15.125:0/433973433 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698012b5c0 con 0x7f6980154210 2023-10-18T10:40:06.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.697+0000 7f697dd9b640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6980154210 0x7f69801552d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f697402c0d0 tx=0x7f69740035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:06.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.697+0000 7f696effd640 1 -- 172.21.15.125:0/433973433 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974003bf0 con 0x7f6980154210 2023-10-18T10:40:06.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.697+0000 7f696effd640 1 -- 172.21.15.125:0/433973433 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6974003e40 con 0x7f6980154210 2023-10-18T10:40:06.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.697+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698012b8a0 con 0x7f6980154210 2023-10-18T10:40:06.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.697+0000 7f696effd640 1 -- 172.21.15.125:0/433973433 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697402c790 con 0x7f6980154210 2023-10-18T10:40:06.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.697+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698012be60 con 0x7f6980154210 2023-10-18T10:40:06.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.698+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69800690e0 con 0x7f6980154210 2023-10-18T10:40:06.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.698+0000 7f696effd640 1 -- 172.21.15.125:0/433973433 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 29) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f697402c8f0 con 0x7f6980154210 2023-10-18T10:40:06.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.698+0000 7f696effd640 1 -- 172.21.15.125:0/433973433 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f69740aa7c0 con 0x7f6980154210 2023-10-18T10:40:06.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.703+0000 7f696effd640 1 -- 172.21.15.125:0/433973433 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6974075780 con 0x7f6980154210 2023-10-18T10:40:06.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.787+0000 7f696effd640 1 -- 172.21.15.125:0/433973433 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 30) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6974075230 con 0x7f6980154210 2023-10-18T10:40:06.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.787+0000 7f696effd640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f6960070c80 0x7f6960073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:06.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.790+0000 7f697e59c640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f6960070c80 0x7f6960073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:06.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.791+0000 7f697e59c640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f6960070c80 0x7f6960073070 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6970001930 tx=0x7f6970001df0 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.907+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69800af590 con 0x7f6980154210 2023-10-18T10:40:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.908+0000 7f696effd640 1 -- 172.21.15.125:0/433973433 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f6974061830 con 0x7f6980154210 2023-10-18T10:40:06.910 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:06.910 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1045257948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1045257948},{"type":"v1","addr":"172.21.15.139:6836","nonce":1045257948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/4183044691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4183044691},{"type":"v1","addr":"172.21.15.139:6837","nonce":4183044691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1083462482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1083462482},{"type":"v1","addr":"172.21.15.125:6835","nonce":1083462482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:37:57.277572+0000","modified":"2023-10-18T10:37:59.669781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/615856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":615856560},{"type":"v1","addr":"172.21.15.125:6834","nonce":615856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T10:40:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 23 2023-10-18T10:40:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.909+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f6960070c80 msgr2=0x7f6960073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.909+0000 7f6984807640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f6960070c80 0x7f6960073070 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6970001930 tx=0x7f6970001df0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.909+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6980154210 msgr2=0x7f69801552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:06.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.909+0000 7f6984807640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6980154210 0x7f69801552d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f697402c0d0 tx=0x7f69740035f0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.910+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 shutdown_connections 2023-10-18T10:40:06.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.910+0000 7f6984807640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6980155810 0x7f69800b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.910+0000 7f6984807640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f6960070c80 0x7f6960073070 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.910+0000 7f6984807640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6980154210 0x7f69801552d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.910+0000 7f6984807640 1 --2- 172.21.15.125:0/433973433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69680047c0 0x7f6980157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:06.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.910+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 >> 172.21.15.125:0/433973433 conn(0x7f6980069930 msgr2=0x7f69800ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:06.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.910+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 shutdown_connections 2023-10-18T10:40:06.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:06.910+0000 7f6984807640 1 -- 172.21.15.125:0/433973433 wait complete. 2023-10-18T10:40:06.922 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:07.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.021+0000 7f4c92f73640 1 -- 172.21.15.125:0/2478742629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c13e520 msgr2=0x7f4c8c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.021+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2478742629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c13e520 0x7f4c8c06bb60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0b20f0 tx=0x7f4c8002bd30 comp rx=0 tx=0).stop 2023-10-18T10:40:07.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.022+0000 7f4c92f73640 1 -- 172.21.15.125:0/2478742629 shutdown_connections 2023-10-18T10:40:07.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.022+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2478742629 >> v2:172.21.15.139:3300/0 conn(0x7f4c8c070af0 0x7f4c8c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.022+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2478742629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c13e520 0x7f4c8c06bb60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.022+0000 7f4c92f73640 1 -- 172.21.15.125:0/2478742629 >> 172.21.15.125:0/2478742629 conn(0x7f4c8c069930 msgr2=0x7f4c8c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:07.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.022+0000 7f4c92f73640 1 -- 172.21.15.125:0/2478742629 shutdown_connections 2023-10-18T10:40:07.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.022+0000 7f4c92f73640 1 -- 172.21.15.125:0/2478742629 wait complete. 2023-10-18T10:40:07.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.023+0000 7f4c92f73640 1 Processor -- start 2023-10-18T10:40:07.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.023+0000 7f4c92f73640 1 -- start start 2023-10-18T10:40:07.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c92f73640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c8c070af0 0x7f4c8c13b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c92f73640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c8c13e520 0x7f4c8c13cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c92f73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c141690 0x7f4c8c152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c92f73640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c151ef0 con 0x7f4c8c141690 2023-10-18T10:40:07.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c92f73640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c1518c0 con 0x7f4c8c13e520 2023-10-18T10:40:07.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c92f73640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c13d880 con 0x7f4c8c070af0 2023-10-18T10:40:07.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c141690 0x7f4c8c152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c141690 0x7f4c8c152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55112/0 (socket says 172.21.15.125:55112) 2023-10-18T10:40:07.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 -- 172.21.15.125:0/2819427134 learned_addr learned my addr 172.21.15.125:0/2819427134 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:07.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c90d0f640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c8c070af0 0x7f4c8c13b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c8bfff640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c8c13e520 0x7f4c8c13cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 -- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c8c070af0 msgr2=0x7f4c8c13b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c8c070af0 0x7f4c8c13b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 -- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c8c13e520 msgr2=0x7f4c8c13cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c8c13e520 0x7f4c8c13cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 -- 172.21.15.125:0/2819427134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80009560 con 0x7f4c8c141690 2023-10-18T10:40:07.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c91510640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c141690 0x7f4c8c152310 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400eaa0 tx=0x7f4c8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:07.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c897fa640 1 -- 172.21.15.125:0/2819427134 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400cdb0 con 0x7f4c8c141690 2023-10-18T10:40:07.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c897fa640 1 -- 172.21.15.125:0/2819427134 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4c84004b20 con 0x7f4c8c141690 2023-10-18T10:40:07.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c897fa640 1 -- 172.21.15.125:0/2819427134 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84005020 con 0x7f4c8c141690 2023-10-18T10:40:07.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c13db60 con 0x7f4c8c141690 2023-10-18T10:40:07.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.024+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c13d500 con 0x7f4c8c141690 2023-10-18T10:40:07.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.025+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c152030 con 0x7f4c8c141690 2023-10-18T10:40:07.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.026+0000 7f4c897fa640 1 -- 172.21.15.125:0/2819427134 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 30) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4c84005180 con 0x7f4c8c141690 2023-10-18T10:40:07.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.026+0000 7f4c897fa640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f4c7006acc0 0x7f4c7006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.026+0000 7f4c897fa640 1 -- 172.21.15.125:0/2819427134 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f4c84089f40 con 0x7f4c8c141690 2023-10-18T10:40:07.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.026+0000 7f4c90d0f640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f4c7006acc0 0x7f4c7006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.027+0000 7f4c90d0f640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f4c7006acc0 0x7f4c7006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c80035000 tx=0x7f4c80004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:07.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.031+0000 7f4c897fa640 1 -- 172.21.15.125:0/2819427134 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c8c152030 con 0x7f4c8c141690 2023-10-18T10:40:07.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.231+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c8c152030 con 0x7f4c8c141690 2023-10-18T10:40:07.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.232+0000 7f4c897fa640 1 -- 172.21.15.125:0/2819427134 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f4c8c152030 con 0x7f4c8c141690 2023-10-18T10:40:07.234 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:07.235 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1045257948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1045257948},{"type":"v1","addr":"172.21.15.139:6836","nonce":1045257948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/4183044691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4183044691},{"type":"v1","addr":"172.21.15.139:6837","nonce":4183044691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1083462482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1083462482},{"type":"v1","addr":"172.21.15.125:6835","nonce":1083462482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:37:57.277572+0000","modified":"2023-10-18T10:37:59.669781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6591":{"gid":6591,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/615856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":615856560},{"type":"v1","addr":"172.21.15.125:6834","nonce":615856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T10:40:07.236 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 23 2023-10-18T10:40:07.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f4c7006acc0 msgr2=0x7f4c7006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f4c7006acc0 0x7f4c7006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c80035000 tx=0x7f4c80004730 comp rx=0 tx=0).stop 2023-10-18T10:40:07.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c141690 msgr2=0x7f4c8c152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c141690 0x7f4c8c152310 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400eaa0 tx=0x7f4c8400ef60 comp rx=0 tx=0).stop 2023-10-18T10:40:07.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 shutdown_connections 2023-10-18T10:40:07.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c8c13e520 0x7f4c8c13cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f4c7006acc0 0x7f4c7006d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c8c141690 0x7f4c8c152310 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 --2- 172.21.15.125:0/2819427134 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c8c070af0 0x7f4c8c13b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 >> 172.21.15.125:0/2819427134 conn(0x7f4c8c069930 msgr2=0x7f4c8c158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:07.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 shutdown_connections 2023-10-18T10:40:07.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.234+0000 7f4c92f73640 1 -- 172.21.15.125:0/2819427134 wait complete. 2023-10-18T10:40:07.246 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:07.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.347+0000 7f0557f6e640 1 -- 172.21.15.125:0/3161740574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05500cbd20 msgr2=0x7f0550158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.347+0000 7f0557f6e640 1 --2- 172.21.15.125:0/3161740574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05500cbd20 0x7f0550158300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0540009830 tx=0x7f054002bd60 comp rx=0 tx=0).stop 2023-10-18T10:40:07.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.347+0000 7f0557f6e640 1 -- 172.21.15.125:0/3161740574 shutdown_connections 2023-10-18T10:40:07.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.347+0000 7f0557f6e640 1 --2- 172.21.15.125:0/3161740574 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05500afa20 0x7f05500705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.347+0000 7f0557f6e640 1 --2- 172.21.15.125:0/3161740574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05500cbd20 0x7f0550158300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.347+0000 7f0557f6e640 1 -- 172.21.15.125:0/3161740574 >> 172.21.15.125:0/3161740574 conn(0x7f0550069930 msgr2=0x7f0550069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:07.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.347+0000 7f0557f6e640 1 -- 172.21.15.125:0/3161740574 shutdown_connections 2023-10-18T10:40:07.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.347+0000 7f0557f6e640 1 -- 172.21.15.125:0/3161740574 wait complete. 2023-10-18T10:40:07.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.348+0000 7f0557f6e640 1 Processor -- start 2023-10-18T10:40:07.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.348+0000 7f0557f6e640 1 -- start start 2023-10-18T10:40:07.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0557f6e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05500afa20 0x7f0550144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0557f6e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05500b0530 0x7f0550146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0557f6e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05500cbd20 0x7f0550146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0557f6e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0550142ec0 con 0x7f05500afa20 2023-10-18T10:40:07.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0557f6e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05501472e0 con 0x7f05500cbd20 2023-10-18T10:40:07.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0557f6e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0550147e10 con 0x7f05500b0530 2023-10-18T10:40:07.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05500afa20 0x7f0550144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05500afa20 0x7f0550144bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55126/0 (socket says 172.21.15.125:55126) 2023-10-18T10:40:07.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 -- 172.21.15.125:0/1247145478 learned_addr learned my addr 172.21.15.125:0/1247145478 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:07.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555509640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05500b0530 0x7f0550146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f055650b640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05500cbd20 0x7f0550146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 -- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05500b0530 msgr2=0x7f0550146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05500b0530 0x7f0550146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 -- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05500cbd20 msgr2=0x7f0550146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05500cbd20 0x7f0550146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 -- 172.21.15.125:0/1247145478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540009490 con 0x7f05500afa20 2023-10-18T10:40:07.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0555d0a640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05500afa20 0x7f0550144bb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0540000c00 tx=0x7f05400038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:07.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f05467fc640 1 -- 172.21.15.125:0/1247145478 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540003ea0 con 0x7f05500afa20 2023-10-18T10:40:07.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f05467fc640 1 -- 172.21.15.125:0/1247145478 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0540034440 con 0x7f05500afa20 2023-10-18T10:40:07.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0550070cf0 con 0x7f05500afa20 2023-10-18T10:40:07.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f05467fc640 1 -- 172.21.15.125:0/1247145478 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540034850 con 0x7f05500afa20 2023-10-18T10:40:07.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.349+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0550147850 con 0x7f05500afa20 2023-10-18T10:40:07.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.350+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05500690e0 con 0x7f05500afa20 2023-10-18T10:40:07.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.351+0000 7f05467fc640 1 -- 172.21.15.125:0/1247145478 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 30) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f05400349b0 con 0x7f05500afa20 2023-10-18T10:40:07.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.351+0000 7f05467fc640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f053406ac90 0x7f053406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.351+0000 7f05467fc640 1 -- 172.21.15.125:0/1247145478 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f05400abe60 con 0x7f05500afa20 2023-10-18T10:40:07.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.351+0000 7f0555509640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f053406ac90 0x7f053406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.352+0000 7f0555509640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f053406ac90 0x7f053406d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0550142aa0 tx=0x7f054c005f90 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:07.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.355+0000 7f05467fc640 1 -- 172.21.15.125:0/1247145478 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0540076e30 con 0x7f05500afa20 2023-10-18T10:40:07.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.523+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05500cc1e0 con 0x7f05500afa20 2023-10-18T10:40:07.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.524+0000 7f05467fc640 1 -- 172.21.15.125:0/1247145478 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17943 (secure 0 0 0) 0x7f054007a450 con 0x7f05500afa20 2023-10-18T10:40:07.526 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:07.526 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":71,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:05.779594+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:37:55.100958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-18T10:37:56.256769+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3728065252":"2023-10-19T10:40:05.779561+0000","172.21.15.139:6832/397604228":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/1729948720":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2802553135":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2678895150":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/3263530731":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6833/397604228":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/3471641794":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/1871659361":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2137279356":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/1884009041":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3401202005":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3643146558":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6833/2140727580":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/751932982":"2023-10-19T10:40:05.779561+0000","172.21.15.139:6832/2140727580":"2023-10-19T10:38:08.702660+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:07.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f053406ac90 msgr2=0x7f053406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f053406ac90 0x7f053406d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0550142aa0 tx=0x7f054c005f90 comp rx=0 tx=0).stop 2023-10-18T10:40:07.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05500afa20 msgr2=0x7f0550144bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05500afa20 0x7f0550144bb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0540000c00 tx=0x7f05400038a0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 shutdown_connections 2023-10-18T10:40:07.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f05500cbd20 0x7f0550146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f053406ac90 0x7f053406d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05500b0530 0x7f0550146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 --2- 172.21.15.125:0/1247145478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05500afa20 0x7f0550144bb0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 >> 172.21.15.125:0/1247145478 conn(0x7f0550069930 msgr2=0x7f05500ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:07.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 shutdown_connections 2023-10-18T10:40:07.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.526+0000 7f0557f6e640 1 -- 172.21.15.125:0/1247145478 wait complete. 2023-10-18T10:40:07.539 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:40:07.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 -- 172.21.15.125:0/658165238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 msgr2=0x7f02840c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 --2- 172.21.15.125:0/658165238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 0x7f02840c9b60 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0274005fd0 tx=0x7f027402bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:07.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 -- 172.21.15.125:0/658165238 shutdown_connections 2023-10-18T10:40:07.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 --2- 172.21.15.125:0/658165238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 0x7f02840c9b60 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 --2- 172.21.15.125:0/658165238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02840ab500 0x7f02840ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 --2- 172.21.15.125:0/658165238 >> v2:172.21.15.139:3300/0 conn(0x7f028406af00 0x7f028406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 -- 172.21.15.125:0/658165238 >> 172.21.15.125:0/658165238 conn(0x7f0284069930 msgr2=0x7f0284069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:07.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 -- 172.21.15.125:0/658165238 shutdown_connections 2023-10-18T10:40:07.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.640+0000 7f028c6a2640 1 -- 172.21.15.125:0/658165238 wait complete. 2023-10-18T10:40:07.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.641+0000 7f028c6a2640 1 Processor -- start 2023-10-18T10:40:07.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028c6a2640 1 -- start start 2023-10-18T10:40:07.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028c6a2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028406af00 0x7f028413a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028c6a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02840ab500 0x7f02840aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028c6a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 0x7f02840aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028c6a2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0284155510 con 0x7f02840c96b0 2023-10-18T10:40:07.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028c6a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02840710b0 con 0x7f02840ab500 2023-10-18T10:40:07.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028c6a2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02841569a0 con 0x7f028406af00 2023-10-18T10:40:07.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028ac3f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 0x7f02840aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028ac3f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 0x7f02840aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40682/0 (socket says 172.21.15.125:40682) 2023-10-18T10:40:07.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.642+0000 7f028ac3f640 1 -- 172.21.15.125:0/2540320246 learned_addr learned my addr 172.21.15.125:0/2540320246 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:07.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f028ac3f640 1 -- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028406af00 msgr2=0x7f028413a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f028ac3f640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028406af00 0x7f028413a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f028ac3f640 1 -- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02840ab500 msgr2=0x7f02840aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f028ac3f640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02840ab500 0x7f02840aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f028ac3f640 1 -- 172.21.15.125:0/2540320246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0274009560 con 0x7f02840c96b0 2023-10-18T10:40:07.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f028ac3f640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 0x7f02840aae00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f027c00c970 tx=0x7f027c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:07.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f027affd640 1 -- 172.21.15.125:0/2540320246 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c007bb0 con 0x7f02840c96b0 2023-10-18T10:40:07.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f027affd640 1 -- 172.21.15.125:0/2540320246 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f027c007e20 con 0x7f02840c96b0 2023-10-18T10:40:07.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f027affd640 1 -- 172.21.15.125:0/2540320246 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c005280 con 0x7f02840c96b0 2023-10-18T10:40:07.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028413aa20 con 0x7f02840c96b0 2023-10-18T10:40:07.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.643+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0284154c70 con 0x7f02840c96b0 2023-10-18T10:40:07.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.644+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0284156b60 con 0x7f02840c96b0 2023-10-18T10:40:07.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.644+0000 7f027affd640 1 -- 172.21.15.125:0/2540320246 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 30) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f027c0053e0 con 0x7f02840c96b0 2023-10-18T10:40:07.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.645+0000 7f027affd640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f026806ac90 0x7f026806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.645+0000 7f027affd640 1 -- 172.21.15.125:0/2540320246 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f027c089b20 con 0x7f02840c96b0 2023-10-18T10:40:07.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.645+0000 7f028a43e640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f026806ac90 0x7f026806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.645+0000 7f028a43e640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f026806ac90 0x7f026806d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0280007be0 tx=0x7f0280005f70 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:07.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.649+0000 7f027affd640 1 -- 172.21.15.125:0/2540320246 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f027c054ac0 con 0x7f02840c96b0 2023-10-18T10:40:07.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.852+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f02840abe50 con 0x7f02840c96b0 2023-10-18T10:40:07.854 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T10:40:07.852+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:40:07.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.853+0000 7f027affd640 1 -- 172.21.15.125:0/2540320246 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f027c01d070 con 0x7f02840c96b0 2023-10-18T10:40:07.855 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:40:07.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f026806ac90 msgr2=0x7f026806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f026806ac90 0x7f026806d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0280007be0 tx=0x7f0280005f70 comp rx=0 tx=0).stop 2023-10-18T10:40:07.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 msgr2=0x7f02840aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 0x7f02840aae00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f027c00c970 tx=0x7f027c00ce30 comp rx=0 tx=0).stop 2023-10-18T10:40:07.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 shutdown_connections 2023-10-18T10:40:07.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02840ab500 0x7f02840aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f026806ac90 0x7f026806d150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02840c96b0 0x7f02840aae00 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 --2- 172.21.15.125:0/2540320246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028406af00 0x7f028413a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 >> 172.21.15.125:0/2540320246 conn(0x7f0284069930 msgr2=0x7f02840ac320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:07.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 shutdown_connections 2023-10-18T10:40:07.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.856+0000 7f028c6a2640 1 -- 172.21.15.125:0/2540320246 wait complete. 2023-10-18T10:40:07.869 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:40:07.891 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:40:07.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.972+0000 7fe2ef765640 1 -- 172.21.15.125:0/3979433172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 msgr2=0x7fe2e80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.972+0000 7fe2ef765640 1 --2- 172.21.15.125:0/3979433172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 0x7fe2e80705a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4005fd0 tx=0x7fe2e402be10 comp rx=0 tx=0).stop 2023-10-18T10:40:07.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.972+0000 7fe2ef765640 1 -- 172.21.15.125:0/3979433172 shutdown_connections 2023-10-18T10:40:07.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.972+0000 7fe2ef765640 1 --2- 172.21.15.125:0/3979433172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 0x7fe2e80705a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.972+0000 7fe2ef765640 1 --2- 172.21.15.125:0/3979433172 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2e80cbd20 0x7fe2e8158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.972+0000 7fe2ef765640 1 -- 172.21.15.125:0/3979433172 >> 172.21.15.125:0/3979433172 conn(0x7fe2e8069930 msgr2=0x7fe2e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:07.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.972+0000 7fe2ef765640 1 -- 172.21.15.125:0/3979433172 shutdown_connections 2023-10-18T10:40:07.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.972+0000 7fe2ef765640 1 -- 172.21.15.125:0/3979433172 wait complete. 2023-10-18T10:40:07.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.973+0000 7fe2ef765640 1 Processor -- start 2023-10-18T10:40:07.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.973+0000 7fe2ef765640 1 -- start start 2023-10-18T10:40:07.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ef765640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 0x7fe2e81e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ef765640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2e80b0530 0x7fe2e8155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ef765640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2e80cbd20 0x7fe2e81442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ef765640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e8155520 con 0x7fe2e80afa20 2023-10-18T10:40:07.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ef765640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e8144810 con 0x7fe2e80cbd20 2023-10-18T10:40:07.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ef765640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e81548a0 con 0x7fe2e80b0530 2023-10-18T10:40:07.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 0x7fe2e81e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 0x7fe2e81e3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40708/0 (socket says 172.21.15.125:40708) 2023-10-18T10:40:07.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 -- 172.21.15.125:0/1703868982 learned_addr learned my addr 172.21.15.125:0/1703868982 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:07.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2edd02640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2e80cbd20 0x7fe2e81442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ecd00640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2e80b0530 0x7fe2e8155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 -- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2e80b0530 msgr2=0x7fe2e8155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2e80b0530 0x7fe2e8155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 -- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2e80cbd20 msgr2=0x7fe2e81442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:07.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2e80cbd20 0x7fe2e81442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:07.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 -- 172.21.15.125:0/1703868982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4009560 con 0x7fe2e80afa20 2023-10-18T10:40:07.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2edd02640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2e80cbd20 0x7fe2e81442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:07.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ecd00640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2e80b0530 0x7fe2e8155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:07.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.974+0000 7fe2ed501640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 0x7fe2e81e3840 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c970 tx=0x7fe2d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:07.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.975+0000 7fe2ddffb640 1 -- 172.21.15.125:0/1703868982 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8007bb0 con 0x7fe2e80afa20 2023-10-18T10:40:07.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.975+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e8070cf0 con 0x7fe2e80afa20 2023-10-18T10:40:07.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.975+0000 7fe2ddffb640 1 -- 172.21.15.125:0/1703868982 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe2d8007e00 con 0x7fe2e80afa20 2023-10-18T10:40:07.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.975+0000 7fe2ddffb640 1 -- 172.21.15.125:0/1703868982 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d800f6b0 con 0x7fe2e80afa20 2023-10-18T10:40:07.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.975+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e8154e60 con 0x7fe2e80afa20 2023-10-18T10:40:07.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.976+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e80690e0 con 0x7fe2e80afa20 2023-10-18T10:40:07.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.976+0000 7fe2ddffb640 1 -- 172.21.15.125:0/1703868982 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 30) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe2d800f810 con 0x7fe2e80afa20 2023-10-18T10:40:07.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.976+0000 7fe2ddffb640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe2cc06ac90 0x7fe2cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:07.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.977+0000 7fe2ddffb640 1 -- 172.21.15.125:0/1703868982 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6889+0+0 (secure 0 0 0) 0x7fe2d8089f00 con 0x7fe2e80afa20 2023-10-18T10:40:07.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.977+0000 7fe2ecd00640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe2cc06ac90 0x7fe2cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:07.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.977+0000 7fe2ecd00640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe2cc06ac90 0x7fe2cc06d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4035000 tx=0x7fe2e4004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:07.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:07.981+0000 7fe2ddffb640 1 -- 172.21.15.125:0/1703868982 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2d8054dd0 con 0x7fe2e80afa20 2023-10-18T10:40:08.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.185+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe2e80cc1e0 con 0x7fe2e80afa20 2023-10-18T10:40:08.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.799+0000 7fe2ddffb640 1 -- 172.21.15.125:0/1703868982 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 31) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe2d80551a0 con 0x7fe2e80afa20 2023-10-18T10:40:08.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.855+0000 7fe2ddffb640 1 -- 172.21.15.125:0/1703868982 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe2d8084010 con 0x7fe2e80afa20 2023-10-18T10:40:08.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe2cc06ac90 msgr2=0x7fe2cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:08.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe2cc06ac90 0x7fe2cc06d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4035000 tx=0x7fe2e4004730 comp rx=0 tx=0).stop 2023-10-18T10:40:08.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 msgr2=0x7fe2e81e3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:08.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 0x7fe2e81e3840 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c970 tx=0x7fe2d800ce30 comp rx=0 tx=0).stop 2023-10-18T10:40:08.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 shutdown_connections 2023-10-18T10:40:08.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2e80cbd20 0x7fe2e81442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:08.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe2cc06ac90 0x7fe2cc06d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:08.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe2e80b0530 0x7fe2e8155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:08.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 --2- 172.21.15.125:0/1703868982 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2e80afa20 0x7fe2e81e3840 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:08.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 >> 172.21.15.125:0/1703868982 conn(0x7fe2e8069930 msgr2=0x7fe2e8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:08.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 shutdown_connections 2023-10-18T10:40:08.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:08.858+0000 7fe2ef765640 1 -- 172.21.15.125:0/1703868982 wait complete. 2023-10-18T10:40:08.872 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:40:09.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.021+0000 7f7eb6a40640 1 -- 172.21.15.125:0/587425840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb0141690 msgr2=0x7f7eb00703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:09.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.021+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/587425840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb0141690 0x7f7eb00703e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4005fd0 tx=0x7f7ea402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.021+0000 7f7eb6a40640 1 -- 172.21.15.125:0/587425840 shutdown_connections 2023-10-18T10:40:09.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.021+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/587425840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb0141690 0x7f7eb00703e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.021+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/587425840 >> v2:172.21.15.139:3300/0 conn(0x7f7eb013e520 0x7f7eb006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.021+0000 7f7eb6a40640 1 -- 172.21.15.125:0/587425840 >> 172.21.15.125:0/587425840 conn(0x7f7eb0069930 msgr2=0x7f7eb0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:09.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.021+0000 7f7eb6a40640 1 -- 172.21.15.125:0/587425840 shutdown_connections 2023-10-18T10:40:09.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.021+0000 7f7eb6a40640 1 -- 172.21.15.125:0/587425840 wait complete. 2023-10-18T10:40:09.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.022+0000 7f7eb6a40640 1 Processor -- start 2023-10-18T10:40:09.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eb6a40640 1 -- start start 2023-10-18T10:40:09.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eb6a40640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7eb006c0a0 0x7f7eb01f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:09.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eb6a40640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb013e520 0x7f7eb01db950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:09.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eb6a40640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb0141690 0x7f7eb01d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:09.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eb6a40640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb01db090 con 0x7f7eb006c0a0 2023-10-18T10:40:09.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eb6a40640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb01da410 con 0x7f7eb013e520 2023-10-18T10:40:09.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eb6a40640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb01da6f0 con 0x7f7eb0141690 2023-10-18T10:40:09.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7eb006c0a0 0x7f7eb01f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:09.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7eb006c0a0 0x7f7eb01f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40720/0 (socket says 172.21.15.125:40720) 2023-10-18T10:40:09.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.023+0000 7f7eaffff640 1 -- 172.21.15.125:0/640895329 learned_addr learned my addr 172.21.15.125:0/640895329 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:09.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eb4fdd640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb0141690 0x7f7eb01d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:09.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eaf7fe640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb013e520 0x7f7eb01db950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:09.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eaffff640 1 -- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb0141690 msgr2=0x7f7eb01d77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:09.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eaffff640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb0141690 0x7f7eb01d77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eaffff640 1 -- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb013e520 msgr2=0x7f7eb01db950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:09.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eaffff640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb013e520 0x7f7eb01db950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eaffff640 1 -- 172.21.15.125:0/640895329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0009490 con 0x7f7eb006c0a0 2023-10-18T10:40:09.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eaf7fe640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb013e520 0x7f7eb01db950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:09.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eaffff640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7eb006c0a0 0x7f7eb01f1d40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ea60 tx=0x7f7ea000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:09.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eacff9640 1 -- 172.21.15.125:0/640895329 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000cde0 con 0x7f7eb006c0a0 2023-10-18T10:40:09.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eacff9640 1 -- 172.21.15.125:0/640895329 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ea0004020 con 0x7f7eb006c0a0 2023-10-18T10:40:09.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea4009560 con 0x7f7eb006c0a0 2023-10-18T10:40:09.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eacff9640 1 -- 172.21.15.125:0/640895329 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0004410 con 0x7f7eb006c0a0 2023-10-18T10:40:09.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.024+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb01dacb0 con 0x7f7eb006c0a0 2023-10-18T10:40:09.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.025+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb01ef1a0 con 0x7f7eb006c0a0 2023-10-18T10:40:09.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.026+0000 7f7eacff9640 1 -- 172.21.15.125:0/640895329 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 31) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7ea00035f0 con 0x7f7eb006c0a0 2023-10-18T10:40:09.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.026+0000 7f7eacff9640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7e9c06aed0 0x7f7e9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:09.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.026+0000 7f7eacff9640 1 -- 172.21.15.125:0/640895329 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6889+0+0 (secure 0 0 0) 0x7f7ea0014070 con 0x7f7eb006c0a0 2023-10-18T10:40:09.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.026+0000 7f7eaf7fe640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7e9c06aed0 0x7f7e9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:09.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.027+0000 7f7eaf7fe640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7e9c06aed0 0x7f7e9c06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4035000 tx=0x7f7ea4004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:09.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.030+0000 7f7eacff9640 1 -- 172.21.15.125:0/640895329 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ea00554f0 con 0x7f7eb006c0a0 2023-10-18T10:40:09.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.196+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7eb00690e0 con 0x7f7eb006c0a0 2023-10-18T10:40:09.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.856+0000 7f7eacff9640 1 -- 172.21.15.125:0/640895329 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7ea0058b10 con 0x7f7eb006c0a0 2023-10-18T10:40:09.859 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:40:09.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.859+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7e9c06aed0 msgr2=0x7f7e9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.859+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7e9c06aed0 0x7f7e9c06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4035000 tx=0x7f7ea4004730 comp rx=0 tx=0).stop 2023-10-18T10:40:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.859+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7eb006c0a0 msgr2=0x7f7eb01f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.859+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7eb006c0a0 0x7f7eb01f1d40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ea60 tx=0x7f7ea000ef20 comp rx=0 tx=0).stop 2023-10-18T10:40:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.860+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 shutdown_connections 2023-10-18T10:40:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.860+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb013e520 0x7f7eb01db950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.860+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7e9c06aed0 0x7f7e9c06d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.860+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb0141690 0x7f7eb01d77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.860+0000 7f7eb6a40640 1 --2- 172.21.15.125:0/640895329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7eb006c0a0 0x7f7eb01f1d40 secure :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ea60 tx=0x7f7ea000ef20 comp rx=0 tx=0).stop 2023-10-18T10:40:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.860+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 >> 172.21.15.125:0/640895329 conn(0x7f7eb0069930 msgr2=0x7f7eb0065490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:09.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.860+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 shutdown_connections 2023-10-18T10:40:09.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.860+0000 7f7eb6a40640 1 -- 172.21.15.125:0/640895329 wait complete. 2023-10-18T10:40:09.876 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:40:09.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.987+0000 7f0d85ec9640 1 -- 172.21.15.125:0/4089542977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d68004d00 msgr2=0x7f0d68005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:09.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.987+0000 7f0d85ec9640 1 --2- 172.21.15.125:0/4089542977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d68004d00 0x7f0d68005180 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0d78001d80 tx=0x7f0d680057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.987+0000 7f0d7ffff640 1 -- 172.21.15.125:0/4089542977 reap_dead start 2023-10-18T10:40:09.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.987+0000 7f0d85ec9640 1 -- 172.21.15.125:0/4089542977 shutdown_connections 2023-10-18T10:40:09.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.987+0000 7f0d85ec9640 1 -- 172.21.15.125:0/4089542977 >> 172.21.15.125:0/4089542977 conn(0x7f0d800d0d60 msgr2=0x7f0d800d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:09.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.987+0000 7f0d85ec9640 1 -- 172.21.15.125:0/4089542977 shutdown_connections 2023-10-18T10:40:09.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.987+0000 7f0d85ec9640 1 -- 172.21.15.125:0/4089542977 wait complete. 2023-10-18T10:40:09.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.988+0000 7f0d85ec9640 1 Processor -- start 2023-10-18T10:40:09.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.988+0000 7f0d85ec9640 1 -- start start 2023-10-18T10:40:09.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d85ec9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d68004d00 0x7f0d800b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:09.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d85ec9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800b6450 0x7f0d800ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:09.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d85ec9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800b68d0 0x7f0d800aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:09.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d85ec9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b3e90 con 0x7f0d800b6450 2023-10-18T10:40:09.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d85ec9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b5690 con 0x7f0d68004d00 2023-10-18T10:40:09.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d85ec9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b5970 con 0x7f0d800b68d0 2023-10-18T10:40:09.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800b6450 0x7f0d800ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:09.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800b6450 0x7f0d800ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40726/0 (socket says 172.21.15.125:40726) 2023-10-18T10:40:09.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7effd640 1 -- 172.21.15.125:0/976808310 learned_addr learned my addr 172.21.15.125:0/976808310 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:09.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7ffff640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800b68d0 0x7f0d800aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:09.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7f7fe640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d68004d00 0x7f0d800b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:09.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7effd640 1 -- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800b68d0 msgr2=0x7f0d800aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:09.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7effd640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800b68d0 0x7f0d800aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7effd640 1 -- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d68004d00 msgr2=0x7f0d800b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:09.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7effd640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d68004d00 0x7f0d800b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:09.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7effd640 1 -- 172.21.15.125:0/976808310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800b5c50 con 0x7f0d800b6450 2023-10-18T10:40:09.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7ffff640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800b68d0 0x7f0d800aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:09.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.989+0000 7f0d7f7fe640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d68004d00 0x7f0d800b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:09.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.990+0000 7f0d7effd640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800b6450 0x7f0d800ae7b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400e6d0 tx=0x7f0d7400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:09.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.990+0000 7f0d67fff640 1 -- 172.21.15.125:0/976808310 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400eea0 con 0x7f0d800b6450 2023-10-18T10:40:09.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.990+0000 7f0d67fff640 1 -- 172.21.15.125:0/976808310 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d7400c840 con 0x7f0d800b6450 2023-10-18T10:40:09.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.990+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d800b5f30 con 0x7f0d800b6450 2023-10-18T10:40:09.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.990+0000 7f0d67fff640 1 -- 172.21.15.125:0/976808310 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400cbf0 con 0x7f0d800b6450 2023-10-18T10:40:09.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.990+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d80144270 con 0x7f0d800b6450 2023-10-18T10:40:09.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.991+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d800b57d0 con 0x7f0d800b6450 2023-10-18T10:40:09.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.991+0000 7f0d67fff640 1 -- 172.21.15.125:0/976808310 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 31) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0d7400cd50 con 0x7f0d800b6450 2023-10-18T10:40:09.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.991+0000 7f0d67fff640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f0d6c06aed0 0x7f0d6c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:09.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.992+0000 7f0d67fff640 1 -- 172.21.15.125:0/976808310 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6423+0+0 (secure 0 0 0) 0x7f0d7408a190 con 0x7f0d800b6450 2023-10-18T10:40:09.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.992+0000 7f0d7f7fe640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f0d6c06aed0 0x7f0d6c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:10.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.992+0000 7f0d7f7fe640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f0d6c06aed0 0x7f0d6c06d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009c50 tx=0x7f0d70005f90 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:10.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:09.996+0000 7f0d67fff640 1 -- 172.21.15.125:0/976808310 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d800b57d0 con 0x7f0d800b6450 2023-10-18T10:40:10.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.161+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0d800b57d0 con 0x7f0d800b6450 2023-10-18T10:40:10.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.399+0000 7f0d67fff640 1 -- 172.21.15.125:0/976808310 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0d74055240 con 0x7f0d800b6450 2023-10-18T10:40:10.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.865+0000 7f0d67fff640 1 -- 172.21.15.125:0/976808310 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0d740730a0 con 0x7f0d800b6450 2023-10-18T10:40:10.867 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T10:40:10.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f0d6c06aed0 msgr2=0x7f0d6c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:10.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f0d6c06aed0 0x7f0d6c06d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009c50 tx=0x7f0d70005f90 comp rx=0 tx=0).stop 2023-10-18T10:40:10.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800b6450 msgr2=0x7f0d800ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:10.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800b6450 0x7f0d800ae7b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400e6d0 tx=0x7f0d7400eb90 comp rx=0 tx=0).stop 2023-10-18T10:40:10.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 shutdown_connections 2023-10-18T10:40:10.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f0d6c06aed0 0x7f0d6c06d390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800b68d0 0x7f0d800aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800b6450 0x7f0d800ae7b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 --2- 172.21.15.125:0/976808310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d68004d00 0x7f0d800b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 >> 172.21.15.125:0/976808310 conn(0x7f0d800d0d60 msgr2=0x7f0d8006b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:10.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.869+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 shutdown_connections 2023-10-18T10:40:10.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.870+0000 7f0d85ec9640 1 -- 172.21.15.125:0/976808310 wait complete. 2023-10-18T10:40:10.886 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T10:40:10.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.990+0000 7f31781f5640 1 -- 172.21.15.125:0/2171228248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31700b0530 msgr2=0x7f31700b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:10.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.990+0000 7f31781f5640 1 --2- 172.21.15.125:0/2171228248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31700b0530 0x7f31700b09c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3168005fd0 tx=0x7f316802c2e0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.990+0000 7f31781f5640 1 -- 172.21.15.125:0/2171228248 shutdown_connections 2023-10-18T10:40:10.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.990+0000 7f31781f5640 1 --2- 172.21.15.125:0/2171228248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31700b0530 0x7f31700b09c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.990+0000 7f31781f5640 1 --2- 172.21.15.125:0/2171228248 >> v2:172.21.15.139:3300/0 conn(0x7f31700afa20 0x7f3170158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.990+0000 7f31781f5640 1 -- 172.21.15.125:0/2171228248 >> 172.21.15.125:0/2171228248 conn(0x7f3170069930 msgr2=0x7f3170069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:10.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.990+0000 7f31781f5640 1 -- 172.21.15.125:0/2171228248 shutdown_connections 2023-10-18T10:40:10.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.990+0000 7f31781f5640 1 -- 172.21.15.125:0/2171228248 wait complete. 2023-10-18T10:40:10.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.991+0000 7f31781f5640 1 Processor -- start 2023-10-18T10:40:10.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.991+0000 7f31781f5640 1 -- start start 2023-10-18T10:40:10.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f31781f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31700afa20 0x7f31701f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:10.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f31781f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31700b0530 0x7f3170146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:10.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f31781f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31700cbd20 0x7f31701560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:10.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f31781f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31701ef950 con 0x7f31700b0530 2023-10-18T10:40:10.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f31781f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31701f23e0 con 0x7f31700afa20 2023-10-18T10:40:10.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f31781f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31701f26c0 con 0x7f31700cbd20 2023-10-18T10:40:10.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31700b0530 0x7f3170146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:10.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31700b0530 0x7f3170146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40740/0 (socket says 172.21.15.125:40740) 2023-10-18T10:40:10.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 -- 172.21.15.125:0/3339160291 learned_addr learned my addr 172.21.15.125:0/3339160291 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:10.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3176792640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31700cbd20 0x7f31701560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:10.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175f91640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31700afa20 0x7f31701f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:10.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 -- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31700cbd20 msgr2=0x7f31701560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:10.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31700cbd20 0x7f31701560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 -- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31700afa20 msgr2=0x7f31701f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:10.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31700afa20 0x7f31701f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:10.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 -- 172.21.15.125:0/3339160291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160009490 con 0x7f31700b0530 2023-10-18T10:40:10.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.992+0000 7f3175790640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31700b0530 0x7f3170146c60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f316000ea60 tx=0x7f316000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:10.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.993+0000 7f31667fc640 1 -- 172.21.15.125:0/3339160291 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316000cde0 con 0x7f31700b0530 2023-10-18T10:40:10.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.993+0000 7f31667fc640 1 -- 172.21.15.125:0/3339160291 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3160004020 con 0x7f31700b0530 2023-10-18T10:40:11.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.993+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3168009900 con 0x7f31700b0530 2023-10-18T10:40:11.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.993+0000 7f31667fc640 1 -- 172.21.15.125:0/3339160291 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160004410 con 0x7f31700b0530 2023-10-18T10:40:11.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.993+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31701477d0 con 0x7f31700b0530 2023-10-18T10:40:11.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.994+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31700690e0 con 0x7f31700b0530 2023-10-18T10:40:11.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.994+0000 7f31667fc640 1 -- 172.21.15.125:0/3339160291 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f31600035f0 con 0x7f31700b0530 2023-10-18T10:40:11.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.995+0000 7f31667fc640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f315406af60 0x7f315406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.995+0000 7f31667fc640 1 -- 172.21.15.125:0/3339160291 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f3160014070 con 0x7f31700b0530 2023-10-18T10:40:11.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.995+0000 7f3175f91640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f315406af60 0x7f315406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.995+0000 7f3175f91640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f315406af60 0x7f315406d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f316c0016e0 tx=0x7f316c009040 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:11.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:10.999+0000 7f31667fc640 1 -- 172.21.15.125:0/3339160291 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31600552c0 con 0x7f31700b0530 2023-10-18T10:40:11.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.166+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7f317006f700 con 0x7f31700b0530 2023-10-18T10:40:11.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.169+0000 7f31667fc640 1 -- 172.21.15.125:0/3339160291 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v38) v1 ==== 71+0+0 (secure 0 0 0) 0x7f31600588e0 con 0x7f31700b0530 2023-10-18T10:40:11.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f315406af60 msgr2=0x7f315406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f315406af60 0x7f315406d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f316c0016e0 tx=0x7f316c009040 comp rx=0 tx=0).stop 2023-10-18T10:40:11.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31700b0530 msgr2=0x7f3170146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31700b0530 0x7f3170146c60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f316000ea60 tx=0x7f316000ef20 comp rx=0 tx=0).stop 2023-10-18T10:40:11.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 shutdown_connections 2023-10-18T10:40:11.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f315406af60 0x7f315406d420 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31700cbd20 0x7f31701560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31700b0530 0x7f3170146c60 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 --2- 172.21.15.125:0/3339160291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31700afa20 0x7f31701f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 >> 172.21.15.125:0/3339160291 conn(0x7f3170069930 msgr2=0x7f3170069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:11.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 shutdown_connections 2023-10-18T10:40:11.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.171+0000 7f31781f5640 1 -- 172.21.15.125:0/3339160291 wait complete. 2023-10-18T10:40:11.183 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-18T10:40:11.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.283+0000 7fb6885fc640 1 -- 172.21.15.125:0/263359179 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb68006af00 msgr2=0x7fb68006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.283+0000 7fb6885fc640 1 --2- 172.21.15.125:0/263359179 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb68006af00 0x7fb68006b2e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb67c005fd0 tx=0x7fb67c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:11.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.284+0000 7fb6885fc640 1 -- 172.21.15.125:0/263359179 shutdown_connections 2023-10-18T10:40:11.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.284+0000 7fb6885fc640 1 --2- 172.21.15.125:0/263359179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6800ab500 0x7fb6800ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.284+0000 7fb6885fc640 1 --2- 172.21.15.125:0/263359179 >> v2:172.21.15.139:3300/0 conn(0x7fb6800c96b0 0x7fb6800c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.284+0000 7fb6885fc640 1 --2- 172.21.15.125:0/263359179 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb68006af00 0x7fb68006b2e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.284+0000 7fb6885fc640 1 -- 172.21.15.125:0/263359179 >> 172.21.15.125:0/263359179 conn(0x7fb680069930 msgr2=0x7fb680069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:11.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.284+0000 7fb6885fc640 1 -- 172.21.15.125:0/263359179 shutdown_connections 2023-10-18T10:40:11.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.284+0000 7fb6885fc640 1 -- 172.21.15.125:0/263359179 wait complete. 2023-10-18T10:40:11.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb6885fc640 1 Processor -- start 2023-10-18T10:40:11.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb6885fc640 1 -- start start 2023-10-18T10:40:11.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb6885fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb68006af00 0x7fb6800aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb6885fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6800ab500 0x7fb6800a8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb6885fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6800c96b0 0x7fb6801f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb6885fc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800a8770 con 0x7fb6800ab500 2023-10-18T10:40:11.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb6885fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800ab230 con 0x7fb6800c96b0 2023-10-18T10:40:11.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb6885fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800a6f60 con 0x7fb68006af00 2023-10-18T10:40:11.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb685b97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6800ab500 0x7fb6800a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb685b97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6800ab500 0x7fb6800a8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40752/0 (socket says 172.21.15.125:40752) 2023-10-18T10:40:11.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.285+0000 7fb685b97640 1 -- 172.21.15.125:0/962205470 learned_addr learned my addr 172.21.15.125:0/962205470 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:11.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb686b99640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6800c96b0 0x7fb6801f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb686398640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb68006af00 0x7fb6800aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb685b97640 1 -- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb68006af00 msgr2=0x7fb6800aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb685b97640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb68006af00 0x7fb6800aab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb685b97640 1 -- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6800c96b0 msgr2=0x7fb6801f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb685b97640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6800c96b0 0x7fb6801f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb685b97640 1 -- 172.21.15.125:0/962205470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c009560 con 0x7fb6800ab500 2023-10-18T10:40:11.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb685b97640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6800ab500 0x7fb6800a8230 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb67000d740 tx=0x7fb67000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:11.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb676ffd640 1 -- 172.21.15.125:0/962205470 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670009c30 con 0x7fb6800ab500 2023-10-18T10:40:11.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb676ffd640 1 -- 172.21.15.125:0/962205470 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb670009e80 con 0x7fb6800ab500 2023-10-18T10:40:11.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb676ffd640 1 -- 172.21.15.125:0/962205470 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700106c0 con 0x7fb6800ab500 2023-10-18T10:40:11.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6800a7240 con 0x7fb6800ab500 2023-10-18T10:40:11.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.286+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6800a7800 con 0x7fb6800ab500 2023-10-18T10:40:11.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.287+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6800a79c0 con 0x7fb6800ab500 2023-10-18T10:40:11.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.287+0000 7fb676ffd640 1 -- 172.21.15.125:0/962205470 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb670010820 con 0x7fb6800ab500 2023-10-18T10:40:11.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.288+0000 7fb676ffd640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb66406af60 0x7fb66406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.288+0000 7fb676ffd640 1 -- 172.21.15.125:0/962205470 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fb670089e30 con 0x7fb6800ab500 2023-10-18T10:40:11.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.288+0000 7fb686398640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb66406af60 0x7fb66406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.288+0000 7fb686398640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb66406af60 0x7fb66406d420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb67c035000 tx=0x7fb67c004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:11.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.292+0000 7fb676ffd640 1 -- 172.21.15.125:0/962205470 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb670054bd0 con 0x7fb6800ab500 2023-10-18T10:40:11.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.532+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fb6800690e0 con 0x7fb6800ab500 2023-10-18T10:40:11.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.867+0000 7fb676ffd640 1 -- 172.21.15.125:0/962205470 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v546) v1 ==== 154+0+0 (secure 0 0 0) 0x7fb6700210e0 con 0x7fb6800ab500 2023-10-18T10:40:11.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb66406af60 msgr2=0x7fb66406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb66406af60 0x7fb66406d420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb67c035000 tx=0x7fb67c004730 comp rx=0 tx=0).stop 2023-10-18T10:40:11.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6800ab500 msgr2=0x7fb6800a8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6800ab500 0x7fb6800a8230 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb67000d740 tx=0x7fb67000dc00 comp rx=0 tx=0).stop 2023-10-18T10:40:11.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 shutdown_connections 2023-10-18T10:40:11.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6800c96b0 0x7fb6801f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb66406af60 0x7fb66406d420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6800ab500 0x7fb6800a8230 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 --2- 172.21.15.125:0/962205470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb68006af00 0x7fb6800aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 >> 172.21.15.125:0/962205470 conn(0x7fb680069930 msgr2=0x7fb6800cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:11.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 shutdown_connections 2023-10-18T10:40:11.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.869+0000 7fb6885fc640 1 -- 172.21.15.125:0/962205470 wait complete. 2023-10-18T10:40:11.881 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-10-18T10:40:11.882 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-10-18T10:40:11.882 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T10:40:11.883 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:11.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.983+0000 7f5c8974c640 1 -- 172.21.15.125:0/1788756918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c6c003970 msgr2=0x7f5c6c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.984+0000 7f5c8974c640 1 --2- 172.21.15.125:0/1788756918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c6c003970 0x7f5c6c003d90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009ed0 tx=0x7f5c6c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.984+0000 7f5c8974c640 1 -- 172.21.15.125:0/1788756918 shutdown_connections 2023-10-18T10:40:11.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.984+0000 7f5c8974c640 1 --2- 172.21.15.125:0/1788756918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c6c003970 0x7f5c6c003d90 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.984+0000 7f5c8974c640 1 -- 172.21.15.125:0/1788756918 >> 172.21.15.125:0/1788756918 conn(0x7f5c84069930 msgr2=0x7f5c84069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:11.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.984+0000 7f5c8974c640 1 -- 172.21.15.125:0/1788756918 shutdown_connections 2023-10-18T10:40:11.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.984+0000 7f5c8974c640 1 -- 172.21.15.125:0/1788756918 wait complete. 2023-10-18T10:40:11.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.985+0000 7f5c8974c640 1 Processor -- start 2023-10-18T10:40:11.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.985+0000 7f5c8974c640 1 -- start start 2023-10-18T10:40:11.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c8974c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c6c003970 0x7f5c840ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c8974c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c840ab500 0x7f5c840ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c8974c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c84154cb0 0x7f5c84155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c8974c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c84130200 con 0x7f5c84154cb0 2023-10-18T10:40:11.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c8974c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840ac3a0 con 0x7f5c840ab500 2023-10-18T10:40:11.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c8974c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840abea0 con 0x7f5c6c003970 2023-10-18T10:40:11.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c84154cb0 0x7f5c84155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c84154cb0 0x7f5c84155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40762/0 (socket says 172.21.15.125:40762) 2023-10-18T10:40:11.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 -- 172.21.15.125:0/1809843355 learned_addr learned my addr 172.21.15.125:0/1809843355 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:11.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c82ffd640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c6c003970 0x7f5c840ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c827fc640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c840ab500 0x7f5c840ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 -- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c6c003970 msgr2=0x7f5c840ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c6c003970 0x7f5c840ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 -- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c840ab500 msgr2=0x7f5c840ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:11.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c840ab500 0x7f5c840ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:11.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 -- 172.21.15.125:0/1809843355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84155670 con 0x7f5c84154cb0 2023-10-18T10:40:11.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c827fc640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c840ab500 0x7f5c840ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:11.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c837fe640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c84154cb0 0x7f5c84155130 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5c840ac640 tx=0x7f5c7c00e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:11.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c6b7fe640 1 -- 172.21.15.125:0/1809843355 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c013070 con 0x7f5c84154cb0 2023-10-18T10:40:11.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c6b7fe640 1 -- 172.21.15.125:0/1809843355 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c7c006e60 con 0x7f5c84154cb0 2023-10-18T10:40:11.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c840ac0c0 con 0x7f5c84154cb0 2023-10-18T10:40:11.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c6b7fe640 1 -- 172.21.15.125:0/1809843355 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0016b0 con 0x7f5c84154cb0 2023-10-18T10:40:11.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.986+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8412fe20 con 0x7f5c84154cb0 2023-10-18T10:40:11.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.987+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c84130340 con 0x7f5c84154cb0 2023-10-18T10:40:11.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.988+0000 7f5c6b7fe640 1 -- 172.21.15.125:0/1809843355 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c7c001810 con 0x7f5c84154cb0 2023-10-18T10:40:11.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.988+0000 7f5c6b7fe640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c7006af90 0x7f5c7006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:11.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.988+0000 7f5c6b7fe640 1 -- 172.21.15.125:0/1809843355 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f5c7c08b2d0 con 0x7f5c84154cb0 2023-10-18T10:40:11.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.988+0000 7f5c82ffd640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c7006af90 0x7f5c7006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:11.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.989+0000 7f5c82ffd640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c7006af90 0x7f5c7006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5c74037000 tx=0x7f5c74004a40 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:11.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:11.992+0000 7f5c6b7fe640 1 -- 172.21.15.125:0/1809843355 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c84130340 con 0x7f5c84154cb0 2023-10-18T10:40:12.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.159+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c84130340 con 0x7f5c84154cb0 2023-10-18T10:40:12.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.160+0000 7f5c6b7fe640 1 -- 172.21.15.125:0/1809843355 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+14256 (secure 0 0 0) 0x7f5c84130340 con 0x7f5c84154cb0 2023-10-18T10:40:12.162 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:12.162 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":75,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:10.858112+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3728065252":"2023-10-19T10:40:05.779561+0000","172.21.15.139:6832/397604228":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/1729948720":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2802553135":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2678895150":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/3263530731":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6833/397604228":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/3471641794":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/1871659361":"2023-10-19T10:40:05.779561+0000","172.21.15.139:0/2137279356":"2023-10-19T10:38:08.702660+0000","172.21.15.125:6832/615856560":"2023-10-19T10:40:07.853402+0000","172.21.15.139:0/1884009041":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3401202005":"2023-10-19T10:38:08.702660+0000","172.21.15.139:0/3643146558":"2023-10-19T10:38:08.702660+0000","172.21.15.139:6833/2140727580":"2023-10-19T10:38:08.702660+0000","172.21.15.125:6834/615856560":"2023-10-19T10:40:07.853402+0000","172.21.15.139:0/751932982":"2023-10-19T10:40:05.779561+0000","172.21.15.139:6832/2140727580":"2023-10-19T10:38:08.702660+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:12.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c7006af90 msgr2=0x7f5c7006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c7006af90 0x7f5c7006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5c74037000 tx=0x7f5c74004a40 comp rx=0 tx=0).stop 2023-10-18T10:40:12.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c84154cb0 msgr2=0x7f5c84155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c84154cb0 0x7f5c84155130 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5c840ac640 tx=0x7f5c7c00e840 comp rx=0 tx=0).stop 2023-10-18T10:40:12.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 shutdown_connections 2023-10-18T10:40:12.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c840ab500 0x7f5c840ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c7006af90 0x7f5c7006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c84154cb0 0x7f5c84155130 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 --2- 172.21.15.125:0/1809843355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c6c003970 0x7f5c840ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 >> 172.21.15.125:0/1809843355 conn(0x7f5c84069930 msgr2=0x7f5c841586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:12.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 shutdown_connections 2023-10-18T10:40:12.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.162+0000 7f5c8974c640 1 -- 172.21.15.125:0/1809843355 wait complete. 2023-10-18T10:40:12.175 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:40:12.176 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:40:12.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.279+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/482295670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1b4005e10 msgr2=0x7fd1b4006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.279+0000 7fd1cd1d0640 1 --2- 172.21.15.125:0/482295670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1b4005e10 0x7fd1b4006290 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009ed0 tx=0x7fd1b4006900 comp rx=0 tx=0).stop 2023-10-18T10:40:12.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.279+0000 7fd1c759e640 1 -- 172.21.15.125:0/482295670 reap_dead start 2023-10-18T10:40:12.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.279+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/482295670 shutdown_connections 2023-10-18T10:40:12.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.279+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/482295670 >> 172.21.15.125:0/482295670 conn(0x7fd1c8069930 msgr2=0x7fd1c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:12.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.279+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/482295670 shutdown_connections 2023-10-18T10:40:12.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.279+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/482295670 wait complete. 2023-10-18T10:40:12.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.280+0000 7fd1cd1d0640 1 Processor -- start 2023-10-18T10:40:12.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.280+0000 7fd1cd1d0640 1 -- start start 2023-10-18T10:40:12.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1cd1d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1b4005e10 0x7fd1c8152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1cd1d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c813e140 0x7fd1c814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1cd1d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1c814ee40 0x7fd1c80ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1cd1d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8152bd0 con 0x7fd1c814ee40 2023-10-18T10:40:12.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1cd1d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8151f50 con 0x7fd1b4005e10 2023-10-18T10:40:12.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1cd1d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8152230 con 0x7fd1c813e140 2023-10-18T10:40:12.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1c814ee40 0x7fd1c80ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1c814ee40 0x7fd1c80ad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40794/0 (socket says 172.21.15.125:40794) 2023-10-18T10:40:12.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1c759e640 1 -- 172.21.15.125:0/2412589450 learned_addr learned my addr 172.21.15.125:0/2412589450 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:12.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1bffff640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c813e140 0x7fd1c814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.281+0000 7fd1c6d9d640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1b4005e10 0x7fd1c8152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1c759e640 1 -- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c813e140 msgr2=0x7fd1c814e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1c759e640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c813e140 0x7fd1c814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1c759e640 1 -- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1b4005e10 msgr2=0x7fd1c8152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1c759e640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1b4005e10 0x7fd1c8152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1c759e640 1 -- 172.21.15.125:0/2412589450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c8152510 con 0x7fd1c814ee40 2023-10-18T10:40:12.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1c6d9d640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1b4005e10 0x7fd1c8152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:12.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1c759e640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1c814ee40 0x7fd1c80ad780 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fd1c002c640 tx=0x7fd1c0030740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:12.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1bf7fe640 1 -- 172.21.15.125:0/2412589450 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c003d070 con 0x7fd1c814ee40 2023-10-18T10:40:12.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1bf7fe640 1 -- 172.21.15.125:0/2412589450 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd1c0030ce0 con 0x7fd1c814ee40 2023-10-18T10:40:12.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c81527f0 con 0x7fd1c814ee40 2023-10-18T10:40:12.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1bf7fe640 1 -- 172.21.15.125:0/2412589450 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0003bd0 con 0x7fd1c814ee40 2023-10-18T10:40:12.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.282+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c8149930 con 0x7fd1c814ee40 2023-10-18T10:40:12.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.283+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c80690e0 con 0x7fd1c814ee40 2023-10-18T10:40:12.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.283+0000 7fd1bf7fe640 1 -- 172.21.15.125:0/2412589450 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd1c0003d30 con 0x7fd1c814ee40 2023-10-18T10:40:12.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.284+0000 7fd1bf7fe640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd1a806af60 0x7fd1a806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.284+0000 7fd1bf7fe640 1 -- 172.21.15.125:0/2412589450 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fd1c00ab700 con 0x7fd1c814ee40 2023-10-18T10:40:12.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.284+0000 7fd1c6d9d640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd1a806af60 0x7fd1a806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.284+0000 7fd1c6d9d640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd1a806af60 0x7fd1a806d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8001eb0 tx=0x7fd1b8000ed0 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:12.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.288+0000 7fd1bf7fe640 1 -- 172.21.15.125:0/2412589450 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1c0076410 con 0x7fd1c814ee40 2023-10-18T10:40:12.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.452+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1c8149930 con 0x7fd1c814ee40 2023-10-18T10:40:12.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.453+0000 7fd1bf7fe640 1 -- 172.21.15.125:0/2412589450 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1c8149930 con 0x7fd1c814ee40 2023-10-18T10:40:12.455 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:40:12.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd1a806af60 msgr2=0x7fd1a806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd1a806af60 0x7fd1a806d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8001eb0 tx=0x7fd1b8000ed0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1c814ee40 msgr2=0x7fd1c80ad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1c814ee40 0x7fd1c80ad780 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fd1c002c640 tx=0x7fd1c0030740 comp rx=0 tx=0).stop 2023-10-18T10:40:12.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 shutdown_connections 2023-10-18T10:40:12.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd1a806af60 0x7fd1a806d420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1c814ee40 0x7fd1c80ad780 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1c813e140 0x7fd1c814e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 --2- 172.21.15.125:0/2412589450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1b4005e10 0x7fd1c8152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 >> 172.21.15.125:0/2412589450 conn(0x7fd1c8069930 msgr2=0x7fd1c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:12.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 shutdown_connections 2023-10-18T10:40:12.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.455+0000 7fd1cd1d0640 1 -- 172.21.15.125:0/2412589450 wait complete. 2023-10-18T10:40:12.468 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-18T10:40:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.569+0000 7fa21b4e5640 1 -- 172.21.15.125:0/2867531611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2000039a0 msgr2=0x7fa200003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.569+0000 7fa21b4e5640 1 --2- 172.21.15.125:0/2867531611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2000039a0 0x7fa200003dc0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa208001680 tx=0x7fa2000057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.569+0000 7fa219a82640 1 -- 172.21.15.125:0/2867531611 reap_dead start 2023-10-18T10:40:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.569+0000 7fa21b4e5640 1 -- 172.21.15.125:0/2867531611 shutdown_connections 2023-10-18T10:40:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.569+0000 7fa21b4e5640 1 -- 172.21.15.125:0/2867531611 >> 172.21.15.125:0/2867531611 conn(0x7fa214069930 msgr2=0x7fa214069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.569+0000 7fa21b4e5640 1 -- 172.21.15.125:0/2867531611 shutdown_connections 2023-10-18T10:40:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.569+0000 7fa21b4e5640 1 -- 172.21.15.125:0/2867531611 wait complete. 2023-10-18T10:40:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.570+0000 7fa21b4e5640 1 Processor -- start 2023-10-18T10:40:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.570+0000 7fa21b4e5640 1 -- start start 2023-10-18T10:40:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.570+0000 7fa21b4e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2000039a0 0x7fa214152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.570+0000 7fa21b4e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140a65b0 0x7fa2140ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.570+0000 7fa21b4e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa214130760 0x7fa2140cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.570+0000 7fa21b4e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa214152250 con 0x7fa214130760 2023-10-18T10:40:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.570+0000 7fa21b4e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa214151680 con 0x7fa2140a65b0 2023-10-18T10:40:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.570+0000 7fa21b4e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa214151920 con 0x7fa2000039a0 2023-10-18T10:40:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa214130760 0x7fa2140cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa214130760 0x7fa2140cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40808/0 (socket says 172.21.15.125:40808) 2023-10-18T10:40:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 -- 172.21.15.125:0/693394651 learned_addr learned my addr 172.21.15.125:0/693394651 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:12.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa218a80640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140a65b0 0x7fa2140ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219281640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2000039a0 0x7fa214152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 -- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2000039a0 msgr2=0x7fa214152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2000039a0 0x7fa214152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 -- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140a65b0 msgr2=0x7fa2140ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140a65b0 0x7fa2140ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 -- 172.21.15.125:0/693394651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa214151c00 con 0x7fa214130760 2023-10-18T10:40:12.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa219a82640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa214130760 0x7fa2140cadb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0013d0 tx=0x7fa20c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:12.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa205ffb640 1 -- 172.21.15.125:0/693394651 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00cc20 con 0x7fa214130760 2023-10-18T10:40:12.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa205ffb640 1 -- 172.21.15.125:0/693394651 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa20c00ce90 con 0x7fa214130760 2023-10-18T10:40:12.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa214151ee0 con 0x7fa214130760 2023-10-18T10:40:12.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa205ffb640 1 -- 172.21.15.125:0/693394651 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0116a0 con 0x7fa214130760 2023-10-18T10:40:12.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.571+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2140b07f0 con 0x7fa214130760 2023-10-18T10:40:12.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.572+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2140690e0 con 0x7fa214130760 2023-10-18T10:40:12.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.573+0000 7fa205ffb640 1 -- 172.21.15.125:0/693394651 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa20c011800 con 0x7fa214130760 2023-10-18T10:40:12.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.573+0000 7fa205ffb640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa1f806af60 0x7fa1f806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.573+0000 7fa205ffb640 1 -- 172.21.15.125:0/693394651 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fa20c089ff0 con 0x7fa214130760 2023-10-18T10:40:12.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.573+0000 7fa219281640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa1f806af60 0x7fa1f806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.574+0000 7fa219281640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa1f806af60 0x7fa1f806d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa208034000 tx=0x7fa208001190 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:12.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.578+0000 7fa205ffb640 1 -- 172.21.15.125:0/693394651 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa20c054e40 con 0x7fa214130760 2023-10-18T10:40:12.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.811+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fa2140cc420 con 0x7fa214130760 2023-10-18T10:40:12.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.868+0000 7fa205ffb640 1 -- 172.21.15.125:0/693394651 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v547) v1 ==== 159+0+0 (secure 0 0 0) 0x7fa20c058460 con 0x7fa214130760 2023-10-18T10:40:12.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.870+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa1f806af60 msgr2=0x7fa1f806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.870+0000 7fa21b4e5640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa1f806af60 0x7fa1f806d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa208034000 tx=0x7fa208001190 comp rx=0 tx=0).stop 2023-10-18T10:40:12.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.870+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa214130760 msgr2=0x7fa2140cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.870+0000 7fa21b4e5640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa214130760 0x7fa2140cadb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0013d0 tx=0x7fa20c00ed30 comp rx=0 tx=0).stop 2023-10-18T10:40:12.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.871+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 shutdown_connections 2023-10-18T10:40:12.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.871+0000 7fa21b4e5640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140a65b0 0x7fa2140ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.871+0000 7fa21b4e5640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa1f806af60 0x7fa1f806d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.871+0000 7fa21b4e5640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa214130760 0x7fa2140cadb0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.871+0000 7fa21b4e5640 1 --2- 172.21.15.125:0/693394651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2000039a0 0x7fa214152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.871+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 >> 172.21.15.125:0/693394651 conn(0x7fa214069930 msgr2=0x7fa214065490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.871+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 shutdown_connections 2023-10-18T10:40:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.871+0000 7fa21b4e5640 1 -- 172.21.15.125:0/693394651 wait complete. 2023-10-18T10:40:12.883 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T10:40:12.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.983+0000 7f163e4b4640 1 -- 172.21.15.125:0/3297620227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163813e520 msgr2=0x7f163806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.983+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3297620227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163813e520 0x7f163806bb60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f16380cbdc0 tx=0x7f162802bd30 comp rx=0 tx=0).stop 2023-10-18T10:40:12.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.983+0000 7f163e4b4640 1 -- 172.21.15.125:0/3297620227 shutdown_connections 2023-10-18T10:40:12.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.983+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3297620227 >> v2:172.21.15.139:3300/0 conn(0x7f1638141690 0x7f16380703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.983+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3297620227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163813e520 0x7f163806bb60 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.983+0000 7f163e4b4640 1 -- 172.21.15.125:0/3297620227 >> 172.21.15.125:0/3297620227 conn(0x7f1638069930 msgr2=0x7f1638069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:12.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.983+0000 7f163e4b4640 1 -- 172.21.15.125:0/3297620227 shutdown_connections 2023-10-18T10:40:12.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.983+0000 7f163e4b4640 1 -- 172.21.15.125:0/3297620227 wait complete. 2023-10-18T10:40:12.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.984+0000 7f163e4b4640 1 Processor -- start 2023-10-18T10:40:12.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f163e4b4640 1 -- start start 2023-10-18T10:40:12.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f163e4b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163806c0a0 0x7f16380a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f163e4b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f163813e520 0x7f163813b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f163e4b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1638141690 0x7f16380c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f163e4b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163813b7d0 con 0x7f163806c0a0 2023-10-18T10:40:12.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f163e4b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163814ae40 con 0x7f163813e520 2023-10-18T10:40:12.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f163e4b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163814b120 con 0x7f1638141690 2023-10-18T10:40:12.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f1637fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163806c0a0 0x7f16380a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f1637fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163806c0a0 0x7f16380a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40826/0 (socket says 172.21.15.125:40826) 2023-10-18T10:40:12.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.985+0000 7f1637fff640 1 -- 172.21.15.125:0/3935001026 learned_addr learned my addr 172.21.15.125:0/3935001026 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:12.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f16377fe640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f163813e520 0x7f163813b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f163ca51640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1638141690 0x7f16380c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1637fff640 1 -- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1638141690 msgr2=0x7f16380c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1637fff640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1638141690 0x7f16380c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1637fff640 1 -- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f163813e520 msgr2=0x7f163813b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:12.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1637fff640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f163813e520 0x7f163813b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:12.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1637fff640 1 -- 172.21.15.125:0/3935001026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1628009560 con 0x7f163806c0a0 2023-10-18T10:40:12.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f163ca51640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1638141690 0x7f16380c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:12.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1637fff640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163806c0a0 0x7f16380a9c50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1628009850 tx=0x7f162802da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:12.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1634ff9640 1 -- 172.21.15.125:0/3935001026 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628003800 con 0x7f163806c0a0 2023-10-18T10:40:12.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1634ff9640 1 -- 172.21.15.125:0/3935001026 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1628003a50 con 0x7f163806c0a0 2023-10-18T10:40:12.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163814b400 con 0x7f163806c0a0 2023-10-18T10:40:12.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f1634ff9640 1 -- 172.21.15.125:0/3935001026 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628003e20 con 0x7f163806c0a0 2023-10-18T10:40:12.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.986+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163814b9c0 con 0x7f163806c0a0 2023-10-18T10:40:12.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.987+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16380690e0 con 0x7f163806c0a0 2023-10-18T10:40:12.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.987+0000 7f1634ff9640 1 -- 172.21.15.125:0/3935001026 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f162803e450 con 0x7f163806c0a0 2023-10-18T10:40:12.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.988+0000 7f1634ff9640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f162406af60 0x7f162406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:12.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.988+0000 7f1634ff9640 1 -- 172.21.15.125:0/3935001026 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f16280ac3b0 con 0x7f163806c0a0 2023-10-18T10:40:12.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.988+0000 7f16377fe640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f162406af60 0x7f162406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:12.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.989+0000 7f16377fe640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f162406af60 0x7f162406d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f162c005fd0 tx=0x7f162c005850 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:12.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:12.992+0000 7f1634ff9640 1 -- 172.21.15.125:0/3935001026 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1628077260 con 0x7f163806c0a0 2023-10-18T10:40:13.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.159+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f16380c9200 con 0x7f163806c0a0 2023-10-18T10:40:13.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.160+0000 7f1634ff9640 1 -- 172.21.15.125:0/3935001026 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v38) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f16280428c0 con 0x7f163806c0a0 2023-10-18T10:40:13.162 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T10:40:13.162 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T10:40:13.162 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T10:40:13.163 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T10:40:13.163 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T10:40:13.163 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T10:40:13.163 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T10:40:13.163 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T10:40:13.164 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T10:40:13.164 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T10:40:13.164 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T10:40:13.164 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T10:40:13.164 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T10:40:13.165 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T10:40:13.165 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T10:40:13.165 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T10:40:13.165 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T10:40:13.166 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T10:40:13.166 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T10:40:13.166 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T10:40:13.166 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T10:40:13.166 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T10:40:13.167 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T10:40:13.167 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T10:40:13.167 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T10:40:13.167 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T10:40:13.167 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T10:40:13.168 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T10:40:13.168 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T10:40:13.168 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T10:40:13.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.162+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f162406af60 msgr2=0x7f162406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.162+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f162406af60 0x7f162406d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f162c005fd0 tx=0x7f162c005850 comp rx=0 tx=0).stop 2023-10-18T10:40:13.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.162+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163806c0a0 msgr2=0x7f16380a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163806c0a0 0x7f16380a9c50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1628009850 tx=0x7f162802da60 comp rx=0 tx=0).stop 2023-10-18T10:40:13.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 shutdown_connections 2023-10-18T10:40:13.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f163813e520 0x7f163813b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f162406af60 0x7f162406d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1638141690 0x7f16380c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 --2- 172.21.15.125:0/3935001026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163806c0a0 0x7f16380a9c50 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 >> 172.21.15.125:0/3935001026 conn(0x7f1638069930 msgr2=0x7f16380af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:13.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 shutdown_connections 2023-10-18T10:40:13.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.163+0000 7f163e4b4640 1 -- 172.21.15.125:0/3935001026 wait complete. 2023-10-18T10:40:13.175 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:40:13.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.278+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/431861197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 msgr2=0x7fe8c40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.278+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/431861197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 0x7fe8c40705a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005fd0 tx=0x7fe8b802bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:13.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.278+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/431861197 shutdown_connections 2023-10-18T10:40:13.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.278+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/431861197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 0x7fe8c40705a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.278+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/431861197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8c40cbd20 0x7fe8c4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.278+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/431861197 >> 172.21.15.125:0/431861197 conn(0x7fe8c4069930 msgr2=0x7fe8c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:13.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.278+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/431861197 shutdown_connections 2023-10-18T10:40:13.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.278+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/431861197 wait complete. 2023-10-18T10:40:13.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.280+0000 7fe8c8bd7640 1 Processor -- start 2023-10-18T10:40:13.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.280+0000 7fe8c8bd7640 1 -- start start 2023-10-18T10:40:13.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.280+0000 7fe8c8bd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 0x7fe8c4140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:13.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.280+0000 7fe8c8bd7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8c40b0530 0x7fe8c4143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:13.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.280+0000 7fe8c8bd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8c40cbd20 0x7fe8c41ef670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:13.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.280+0000 7fe8c8bd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40ad250 con 0x7fe8c40afa20 2023-10-18T10:40:13.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.280+0000 7fe8c8bd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c4141c70 con 0x7fe8c40cbd20 2023-10-18T10:40:13.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.280+0000 7fe8c8bd7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c4143d20 con 0x7fe8c40b0530 2023-10-18T10:40:13.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 0x7fe8c4140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:13.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 0x7fe8c4140300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40850/0 (socket says 172.21.15.125:40850) 2023-10-18T10:40:13.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 -- 172.21.15.125:0/3877642744 learned_addr learned my addr 172.21.15.125:0/3877642744 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:13.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c1d9b640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8c40b0530 0x7fe8c4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:13.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c2d9d640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8c40cbd20 0x7fe8c41ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:13.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 -- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8c40b0530 msgr2=0x7fe8c4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8c40b0530 0x7fe8c4143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 -- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8c40cbd20 msgr2=0x7fe8c41ef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8c40cbd20 0x7fe8c41ef670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 -- 172.21.15.125:0/3877642744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009560 con 0x7fe8c40afa20 2023-10-18T10:40:13.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c2d9d640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8c40cbd20 0x7fe8c41ef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:13.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c259c640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 0x7fe8c4140300 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400b410 tx=0x7fe8b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:13.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8b2ffd640 1 -- 172.21.15.125:0/3877642744 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4011830 con 0x7fe8c40afa20 2023-10-18T10:40:13.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c4070cf0 con 0x7fe8c40afa20 2023-10-18T10:40:13.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8b2ffd640 1 -- 172.21.15.125:0/3877642744 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8b4011aa0 con 0x7fe8c40afa20 2023-10-18T10:40:13.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8b2ffd640 1 -- 172.21.15.125:0/3877642744 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4010380 con 0x7fe8c40afa20 2023-10-18T10:40:13.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.281+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c41442e0 con 0x7fe8c40afa20 2023-10-18T10:40:13.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.282+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c40690e0 con 0x7fe8c40afa20 2023-10-18T10:40:13.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.283+0000 7fe8b2ffd640 1 -- 172.21.15.125:0/3877642744 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe8b40104e0 con 0x7fe8c40afa20 2023-10-18T10:40:13.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.283+0000 7fe8b2ffd640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe8a406af60 0x7fe8a406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:13.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.283+0000 7fe8b2ffd640 1 -- 172.21.15.125:0/3877642744 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fe8b4089d40 con 0x7fe8c40afa20 2023-10-18T10:40:13.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.283+0000 7fe8c1d9b640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe8a406af60 0x7fe8a406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:13.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.284+0000 7fe8c1d9b640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe8a406af60 0x7fe8a406d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8035000 tx=0x7fe8b8004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:13.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.287+0000 7fe8b2ffd640 1 -- 172.21.15.125:0/3877642744 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8b4054b70 con 0x7fe8c40afa20 2023-10-18T10:40:13.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.456+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe8c40cc1e0 con 0x7fe8c40afa20 2023-10-18T10:40:13.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.459+0000 7fe8b2ffd640 1 -- 172.21.15.125:0/3877642744 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v39)=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe8b4058190 con 0x7fe8c40afa20 2023-10-18T10:40:13.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe8a406af60 msgr2=0x7fe8a406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe8a406af60 0x7fe8a406d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8035000 tx=0x7fe8b8004730 comp rx=0 tx=0).stop 2023-10-18T10:40:13.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 msgr2=0x7fe8c4140300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 0x7fe8c4140300 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400b410 tx=0x7fe8b400b8d0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 shutdown_connections 2023-10-18T10:40:13.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8c40cbd20 0x7fe8c41ef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe8a406af60 0x7fe8a406d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8c40b0530 0x7fe8c4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 --2- 172.21.15.125:0/3877642744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c40afa20 0x7fe8c4140300 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 >> 172.21.15.125:0/3877642744 conn(0x7fe8c4069930 msgr2=0x7fe8c40ac590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:13.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 shutdown_connections 2023-10-18T10:40:13.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.461+0000 7fe8c8bd7640 1 -- 172.21.15.125:0/3877642744 wait complete. 2023-10-18T10:40:13.473 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:40:13.474 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:40:13.516 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:40:13.568 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:40:13.569 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:40:13.569 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:13.585 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:40:13.585 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:40:13.586 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:13.602 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:40:13.603 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:40:13.603 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:13.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:40:13.643 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:40:13.645 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:13.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:40:13.662 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:40:13.663 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:13.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 -- 172.21.15.125:0/790351022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec15b2a0 msgr2=0x7fb6ec0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/790351022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec15b2a0 0x7fb6ec0cf720 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec089ae0 tx=0x7fb6e002bd10 comp rx=0 tx=0).stop 2023-10-18T10:40:13.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 -- 172.21.15.125:0/790351022 shutdown_connections 2023-10-18T10:40:13.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/790351022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec15b2a0 0x7fb6ec0cf720 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/790351022 >> v2:172.21.15.139:3300/0 conn(0x7fb6ec0ceb40 0x7fb6ec0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/790351022 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6ec0cf340 0x7fb6ec0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 -- 172.21.15.125:0/790351022 >> 172.21.15.125:0/790351022 conn(0x7fb6ec0d0d60 msgr2=0x7fb6ec0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:13.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 -- 172.21.15.125:0/790351022 shutdown_connections 2023-10-18T10:40:13.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.782+0000 7fb6f0ada640 1 -- 172.21.15.125:0/790351022 wait complete. 2023-10-18T10:40:13.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.783+0000 7fb6f0ada640 1 Processor -- start 2023-10-18T10:40:13.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6f0ada640 1 -- start start 2023-10-18T10:40:13.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6f0ada640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6ec0ceb40 0x7fb6ec0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:13.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6f0ada640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec0cf340 0x7fb6ec159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:13.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6f0ada640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6ec15b2a0 0x7fb6ec09e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:13.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6f0ada640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b4c20 con 0x7fb6ec0cf340 2023-10-18T10:40:13.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6f0ada640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec159c80 con 0x7fb6ec15b2a0 2023-10-18T10:40:13.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6f0ada640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec09ea10 con 0x7fb6ec0ceb40 2023-10-18T10:40:13.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6e9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec0cf340 0x7fb6ec159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:13.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6e9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec0cf340 0x7fb6ec159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40864/0 (socket says 172.21.15.125:40864) 2023-10-18T10:40:13.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6e9d9b640 1 -- 172.21.15.125:0/816518724 learned_addr learned my addr 172.21.15.125:0/816518724 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:13.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6ead9d640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6ec15b2a0 0x7fb6ec09e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:13.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.784+0000 7fb6ea59c640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6ec0ceb40 0x7fb6ec0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:13.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6e9d9b640 1 -- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6ec0ceb40 msgr2=0x7fb6ec0b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6e9d9b640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6ec0ceb40 0x7fb6ec0b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6e9d9b640 1 -- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6ec15b2a0 msgr2=0x7fb6ec09e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:13.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6e9d9b640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6ec15b2a0 0x7fb6ec09e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:13.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6e9d9b640 1 -- 172.21.15.125:0/816518724 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e0009560 con 0x7fb6ec0cf340 2023-10-18T10:40:13.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6e9d9b640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec0cf340 0x7fb6ec159740 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0036040 tx=0x7fb6e002dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:13.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6daffd640 1 -- 172.21.15.125:0/816518724 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0005650 con 0x7fb6ec0cf340 2023-10-18T10:40:13.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6daffd640 1 -- 172.21.15.125:0/816518724 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb6e00038f0 con 0x7fb6ec0cf340 2023-10-18T10:40:13.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec0b4010 con 0x7fb6ec0cf340 2023-10-18T10:40:13.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6daffd640 1 -- 172.21.15.125:0/816518724 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0003ce0 con 0x7fb6ec0cf340 2023-10-18T10:40:13.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.785+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec0b45d0 con 0x7fb6ec0cf340 2023-10-18T10:40:13.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.786+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec0690e0 con 0x7fb6ec0cf340 2023-10-18T10:40:13.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.786+0000 7fb6daffd640 1 -- 172.21.15.125:0/816518724 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb6e003e450 con 0x7fb6ec0cf340 2023-10-18T10:40:13.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.787+0000 7fb6daffd640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb6cc06af60 0x7fb6cc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:13.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.787+0000 7fb6daffd640 1 -- 172.21.15.125:0/816518724 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fb6e00ac0f0 con 0x7fb6ec0cf340 2023-10-18T10:40:13.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.787+0000 7fb6ea59c640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb6cc06af60 0x7fb6cc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:13.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.787+0000 7fb6ea59c640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb6cc06af60 0x7fb6cc06d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005c40 tx=0x7fb6dc00a570 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:13.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.791+0000 7fb6daffd640 1 -- 172.21.15.125:0/816518724 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6e0077040 con 0x7fb6ec0cf340 2023-10-18T10:40:13.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.997+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6ec0d0b40 con 0x7fb6ec0cf340 2023-10-18T10:40:14.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:13.998+0000 7fb6daffd640 1 -- 172.21.15.125:0/816518724 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fb6e0042090 con 0x7fb6ec0cf340 2023-10-18T10:40:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:14.001 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1045257948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1045257948},{"type":"v1","addr":"172.21.15.139:6836","nonce":1045257948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/4183044691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4183044691},{"type":"v1","addr":"172.21.15.139:6837","nonce":4183044691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5824,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1083462482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1083462482},{"type":"v1","addr":"172.21.15.125:6835","nonce":1083462482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6996,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3675784681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3675784681},{"type":"v1","addr":"172.21.15.125:6834","nonce":3675784681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[]} 2023-10-18T10:40:14.002 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 25 2023-10-18T10:40:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb6cc06af60 msgr2=0x7fb6cc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb6cc06af60 0x7fb6cc06d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005c40 tx=0x7fb6dc00a570 comp rx=0 tx=0).stop 2023-10-18T10:40:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec0cf340 msgr2=0x7fb6ec159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:14.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec0cf340 0x7fb6ec159740 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0036040 tx=0x7fb6e002dce0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 shutdown_connections 2023-10-18T10:40:14.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6ec15b2a0 0x7fb6ec09e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb6cc06af60 0x7fb6cc06d420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6ec0cf340 0x7fb6ec159740 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 --2- 172.21.15.125:0/816518724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6ec0ceb40 0x7fb6ec0b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 >> 172.21.15.125:0/816518724 conn(0x7fb6ec0d0d60 msgr2=0x7fb6ec159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:14.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 shutdown_connections 2023-10-18T10:40:14.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.000+0000 7fb6f0ada640 1 -- 172.21.15.125:0/816518724 wait complete. 2023-10-18T10:40:14.013 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:40:14.013 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:40:14.013 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:40:14.014 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:40:14.014 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:40:14.014 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:40:14.015 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:40:14.015 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:40:14.015 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:40:14.016 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:40:14.016 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:40:14.016 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:40:14.017 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:40:14.017 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:40:14.018 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:40:14.018 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:40:14.022 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:40:14.023 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:40:14.024 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:40:14.025 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:40:14.028 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:40:14.028 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:40:14.029 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:40:14.030 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:40:14.067 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:40:14.067 DEBUG:teuthology.parallel:result is None 2023-10-18T10:40:14.069 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:40:14.069 DEBUG:teuthology.parallel:result is None 2023-10-18T10:40:14.072 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:40:14.072 DEBUG:teuthology.parallel:result is None 2023-10-18T10:40:14.073 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:40:14.073 DEBUG:teuthology.parallel:result is None 2023-10-18T10:40:14.074 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:40:14.141 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:40:14.142 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:40:14.142 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T10:40:14.140+0000 7ff1e5b7b640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-18T10:40:14.155 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:40:14.156 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:40:14.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.179+0000 7fef85c96640 1 -- 172.21.15.125:0/1531890941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 msgr2=0x7fef800d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:14.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.179+0000 7fef85c96640 1 --2- 172.21.15.125:0/1531890941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 0x7fef800d1910 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fef8015b680 tx=0x7fef7002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.179+0000 7fef85c96640 1 -- 172.21.15.125:0/1531890941 shutdown_connections 2023-10-18T10:40:14.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.179+0000 7fef85c96640 1 --2- 172.21.15.125:0/1531890941 >> v2:172.21.15.139:3300/0 conn(0x7fef800ceb40 0x7fef800b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.179+0000 7fef85c96640 1 --2- 172.21.15.125:0/1531890941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 0x7fef800d1910 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.179+0000 7fef85c96640 1 -- 172.21.15.125:0/1531890941 >> 172.21.15.125:0/1531890941 conn(0x7fef800d0d60 msgr2=0x7fef800d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:14.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.179+0000 7fef85c96640 1 -- 172.21.15.125:0/1531890941 shutdown_connections 2023-10-18T10:40:14.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.179+0000 7fef85c96640 1 -- 172.21.15.125:0/1531890941 wait complete. 2023-10-18T10:40:14.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.180+0000 7fef85c96640 1 Processor -- start 2023-10-18T10:40:14.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.180+0000 7fef85c96640 1 -- start start 2023-10-18T10:40:14.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef85c96640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef800ceb40 0x7fef800b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:14.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef85c96640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 0x7fef800c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:14.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef85c96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef8015b2a0 0x7fef800ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:14.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef85c96640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8015cba0 con 0x7fef800cf340 2023-10-18T10:40:14.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef85c96640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef801f11a0 con 0x7fef8015b2a0 2023-10-18T10:40:14.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef85c96640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef800c8d00 con 0x7fef800ceb40 2023-10-18T10:40:14.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef7effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 0x7fef800c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:14.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef7effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 0x7fef800c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40894/0 (socket says 172.21.15.125:40894) 2023-10-18T10:40:14.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef7effd640 1 -- 172.21.15.125:0/1702651104 learned_addr learned my addr 172.21.15.125:0/1702651104 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:14.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.181+0000 7fef7f7fe640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef800ceb40 0x7fef800b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:14.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7effd640 1 -- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef800ceb40 msgr2=0x7fef800b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:14.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7effd640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef800ceb40 0x7fef800b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7effd640 1 -- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef8015b2a0 msgr2=0x7fef800ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:14.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7ffff640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef8015b2a0 0x7fef800ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:14.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7effd640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef8015b2a0 0x7fef800ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:14.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7effd640 1 -- 172.21.15.125:0/1702651104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70009490 con 0x7fef800cf340 2023-10-18T10:40:14.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7f7fe640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef800ceb40 0x7fef800b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:14.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7ffff640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef8015b2a0 0x7fef800ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:14.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef7effd640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 0x7fef800c9b10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c410 tx=0x7fef7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:14.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef67fff640 1 -- 172.21.15.125:0/1702651104 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef740035f0 con 0x7fef800cf340 2023-10-18T10:40:14.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef67fff640 1 -- 172.21.15.125:0/1702651104 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef74003770 con 0x7fef800cf340 2023-10-18T10:40:14.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef67fff640 1 -- 172.21.15.125:0/1702651104 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74003b80 con 0x7fef800cf340 2023-10-18T10:40:14.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef800c8fe0 con 0x7fef800cf340 2023-10-18T10:40:14.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.182+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8015c7c0 con 0x7fef800cf340 2023-10-18T10:40:14.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.183+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8015cce0 con 0x7fef800cf340 2023-10-18T10:40:14.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.183+0000 7fef67fff640 1 -- 172.21.15.125:0/1702651104 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fef74003ce0 con 0x7fef800cf340 2023-10-18T10:40:14.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.184+0000 7fef67fff640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fef6c06af60 0x7fef6c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:14.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.184+0000 7fef67fff640 1 -- 172.21.15.125:0/1702651104 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fef74088fa0 con 0x7fef800cf340 2023-10-18T10:40:14.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.184+0000 7fef7f7fe640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fef6c06af60 0x7fef6c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:14.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.184+0000 7fef7f7fe640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fef6c06af60 0x7fef6c06d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fef70036000 tx=0x7fef70004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:14.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.188+0000 7fef67fff640 1 -- 172.21.15.125:0/1702651104 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef740bbca0 con 0x7fef800cf340 2023-10-18T10:40:14.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:14.366+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fef8015c950 con 0x7fef800cf340 2023-10-18T10:40:15.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.479+0000 7fef67fff640 1 -- 172.21.15.125:0/1702651104 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7fef74017070 con 0x7fef800cf340 2023-10-18T10:40:15.482 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T10:40:15.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fef6c06af60 msgr2=0x7fef6c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fef6c06af60 0x7fef6c06d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fef70036000 tx=0x7fef70004730 comp rx=0 tx=0).stop 2023-10-18T10:40:15.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 msgr2=0x7fef800c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 0x7fef800c9b10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c410 tx=0x7fef7400c8d0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 shutdown_connections 2023-10-18T10:40:15.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef8015b2a0 0x7fef800ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fef6c06af60 0x7fef6c06d420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef800cf340 0x7fef800c9b10 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 --2- 172.21.15.125:0/1702651104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef800ceb40 0x7fef800b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 >> 172.21.15.125:0/1702651104 conn(0x7fef800d0d60 msgr2=0x7fef800cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:15.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 shutdown_connections 2023-10-18T10:40:15.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.483+0000 7fef85c96640 1 -- 172.21.15.125:0/1702651104 wait complete. 2023-10-18T10:40:15.499 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:40:15.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.604+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/4221562839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d815ccb0 msgr2=0x7fb3d80c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.604+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/4221562839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d815ccb0 0x7fb3d80c9750 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80cc8a0 tx=0x7fb3d002c080 comp rx=0 tx=0).stop 2023-10-18T10:40:15.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.604+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/4221562839 shutdown_connections 2023-10-18T10:40:15.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.604+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/4221562839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d815ccb0 0x7fb3d80c9750 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.604+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/4221562839 >> v2:172.21.15.139:3300/0 conn(0x7fb3d806b410 0x7fb3d815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.604+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/4221562839 >> 172.21.15.125:0/4221562839 conn(0x7fb3d8069930 msgr2=0x7fb3d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:15.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.604+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/4221562839 shutdown_connections 2023-10-18T10:40:15.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.605+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/4221562839 wait complete. 2023-10-18T10:40:15.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.606+0000 7fb3dcb9e640 1 Processor -- start 2023-10-18T10:40:15.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.606+0000 7fb3dcb9e640 1 -- start start 2023-10-18T10:40:15.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.606+0000 7fb3dcb9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d806b030 0x7fb3d80a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:15.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.606+0000 7fb3dcb9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3d806b410 0x7fb3d8150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:15.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3dcb9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d815ccb0 0x7fb3d80a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:15.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3dcb9e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80a7700 con 0x7fb3d806b030 2023-10-18T10:40:15.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3dcb9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8139ad0 con 0x7fb3d815ccb0 2023-10-18T10:40:15.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3dcb9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d81513e0 con 0x7fb3d806b410 2023-10-18T10:40:15.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d806b030 0x7fb3d80a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:15.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d806b030 0x7fb3d80a71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40910/0 (socket says 172.21.15.125:40910) 2023-10-18T10:40:15.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 -- 172.21.15.125:0/2328584391 learned_addr learned my addr 172.21.15.125:0/2328584391 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:15.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d6d9d640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d815ccb0 0x7fb3d80a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:15.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3cffff640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3d806b410 0x7fb3d8150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:15.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 -- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3d806b410 msgr2=0x7fb3d8150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3d806b410 0x7fb3d8150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 -- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d815ccb0 msgr2=0x7fb3d80a8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d815ccb0 0x7fb3d80a8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 -- 172.21.15.125:0/2328584391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0009840 con 0x7fb3d806b030 2023-10-18T10:40:15.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3cffff640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3d806b410 0x7fb3d8150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:15.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3d659c640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d806b030 0x7fb3d80a71c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800c460 tx=0x7fb3c800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:15.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3ceffd640 1 -- 172.21.15.125:0/2328584391 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8004230 con 0x7fb3d806b030 2023-10-18T10:40:15.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3ceffd640 1 -- 172.21.15.125:0/2328584391 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3c80044a0 con 0x7fb3d806b030 2023-10-18T10:40:15.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3ceffd640 1 -- 172.21.15.125:0/2328584391 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8003850 con 0x7fb3d806b030 2023-10-18T10:40:15.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d8151660 con 0x7fb3d806b030 2023-10-18T10:40:15.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.607+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d8151c20 con 0x7fb3d806b030 2023-10-18T10:40:15.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.608+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d80690e0 con 0x7fb3d806b030 2023-10-18T10:40:15.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.609+0000 7fb3ceffd640 1 -- 172.21.15.125:0/2328584391 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb3c80039d0 con 0x7fb3d806b030 2023-10-18T10:40:15.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.609+0000 7fb3ceffd640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb3b806af60 0x7fb3b806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:15.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.609+0000 7fb3ceffd640 1 -- 172.21.15.125:0/2328584391 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb3c8088d90 con 0x7fb3d806b030 2023-10-18T10:40:15.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.609+0000 7fb3cffff640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb3b806af60 0x7fb3b806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:15.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.610+0000 7fb3cffff640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb3b806af60 0x7fb3b806d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0006fd0 tx=0x7fb3c0008040 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:15.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.613+0000 7fb3ceffd640 1 -- 172.21.15.125:0/2328584391 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3c8057a90 con 0x7fb3d806b030 2023-10-18T10:40:15.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.841+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb3d806d3b0 con 0x7fb3d806b030 2023-10-18T10:40:15.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.842+0000 7fb3ceffd640 1 -- 172.21.15.125:0/2328584391 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb3c8017090 con 0x7fb3d806b030 2023-10-18T10:40:15.844 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:15.844 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:40:15.844 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T10:40:15.845 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.845 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T10:40:15.845 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T10:40:15.845 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.845 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:15.846 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:15.846 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:15.846 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.846 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.846 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.846 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.847 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T10:40:15.847 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T10:40:15.847 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.847 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:15.848 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:15.848 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:15.848 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.848 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.848 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.849 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.849 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T10:40:15.849 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T10:40:15.849 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.849 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:15.850 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:15.850 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:15.850 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.850 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.850 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.851 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.851 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T10:40:15.851 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T10:40:15.851 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.851 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:15.851 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:15.852 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:15.852 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.852 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.852 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.852 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.852 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T10:40:15.853 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T10:40:15.853 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.853 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:15.853 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:15.853 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.853 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.854 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.854 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.854 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T10:40:15.854 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T10:40:15.854 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.855 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:15.855 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:15.855 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.855 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.855 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.856 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.856 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T10:40:15.856 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T10:40:15.856 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.856 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:15.856 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:15.857 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.857 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.857 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.857 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.857 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T10:40:15.858 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T10:40:15.858 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.858 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:15.858 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:15.858 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.859 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.859 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.859 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.859 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T10:40:15.859 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T10:40:15.859 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.860 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:15.860 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:15.860 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.860 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.860 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.861 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.861 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T10:40:15.861 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T10:40:15.861 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.861 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:15.861 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:15.862 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.862 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.862 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.862 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.862 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T10:40:15.863 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T10:40:15.863 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.863 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:15.863 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:15.863 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.863 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.864 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.864 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.864 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T10:40:15.864 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T10:40:15.864 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.864 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:15.865 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:15.865 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.865 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.865 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.865 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.865 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T10:40:15.866 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T10:40:15.866 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.866 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:15.866 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T10:40:15.866 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T10:40:15.867 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.867 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.867 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.867 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T10:40:15.867 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T10:40:15.868 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.868 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:15.868 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:40:15.868 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:40:15.868 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:40:15.868 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.869 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.869 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.869 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T10:40:15.869 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T10:40:15.869 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:15.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:40:15.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:40:15.870 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:40:15.870 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.871 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.871 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.871 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T10:40:15.871 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T10:40:15.871 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.871 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:15.872 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:40:15.872 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:40:15.872 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:40:15.872 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.872 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.872 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.873 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T10:40:15.873 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T10:40:15.873 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.873 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:40:15.873 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.874 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T10:40:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T10:40:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:40:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:40:15.875 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.876 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.876 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:15.876 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:15.876 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T10:40:15.876 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T10:40:15.876 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:15.877 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:40:15.877 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:40:15.877 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:15.877 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.877 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:15.877 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T10:40:15.878 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:40:15.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb3b806af60 msgr2=0x7fb3b806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb3b806af60 0x7fb3b806d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0006fd0 tx=0x7fb3c0008040 comp rx=0 tx=0).stop 2023-10-18T10:40:15.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d806b030 msgr2=0x7fb3d80a71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d806b030 0x7fb3d80a71c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800c460 tx=0x7fb3c800c920 comp rx=0 tx=0).stop 2023-10-18T10:40:15.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 shutdown_connections 2023-10-18T10:40:15.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d815ccb0 0x7fb3d80a8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb3b806af60 0x7fb3b806d420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3d806b410 0x7fb3d8150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 --2- 172.21.15.125:0/2328584391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3d806b030 0x7fb3d80a71c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 >> 172.21.15.125:0/2328584391 conn(0x7fb3d8069930 msgr2=0x7fb3d813b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:15.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 shutdown_connections 2023-10-18T10:40:15.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.844+0000 7fb3dcb9e640 1 -- 172.21.15.125:0/2328584391 wait complete. 2023-10-18T10:40:15.882 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.983+0000 7fc64dd5e640 1 -- 172.21.15.125:0/4016137909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 msgr2=0x7fc630004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.983+0000 7fc64dd5e640 1 --2- 172.21.15.125:0/4016137909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 0x7fc630004840 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009b50 tx=0x7fc6300057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.983+0000 7fc647fff640 1 -- 172.21.15.125:0/4016137909 reap_dead start 2023-10-18T10:40:15.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.983+0000 7fc64dd5e640 1 -- 172.21.15.125:0/4016137909 shutdown_connections 2023-10-18T10:40:15.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.983+0000 7fc64dd5e640 1 -- 172.21.15.125:0/4016137909 >> 172.21.15.125:0/4016137909 conn(0x7fc648069930 msgr2=0x7fc648069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:15.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.983+0000 7fc64dd5e640 1 -- 172.21.15.125:0/4016137909 shutdown_connections 2023-10-18T10:40:15.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.983+0000 7fc64dd5e640 1 -- 172.21.15.125:0/4016137909 wait complete. 2023-10-18T10:40:15.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.984+0000 7fc64dd5e640 1 Processor -- start 2023-10-18T10:40:15.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.984+0000 7fc64dd5e640 1 -- start start 2023-10-18T10:40:15.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc64dd5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 0x7fc6480a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:15.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc64dd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc648156860 0x7fc648140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:15.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc64dd5e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc648159f70 0x7fc64815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:15.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc64dd5e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64815a930 con 0x7fc630004360 2023-10-18T10:40:15.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc64dd5e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6481407d0 con 0x7fc648156860 2023-10-18T10:40:15.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc64dd5e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc648156ce0 con 0x7fc648159f70 2023-10-18T10:40:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc6477fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 0x7fc6480a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc6477fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 0x7fc6480a64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40942/0 (socket says 172.21.15.125:40942) 2023-10-18T10:40:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc6477fe640 1 -- 172.21.15.125:0/3575108142 learned_addr learned my addr 172.21.15.125:0/3575108142 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc647fff640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc648159f70 0x7fc64815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc646ffd640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc648156860 0x7fc648140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc6477fe640 1 -- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc648159f70 msgr2=0x7fc64815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc6477fe640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc648159f70 0x7fc64815a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc6477fe640 1 -- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc648156860 msgr2=0x7fc648140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc6477fe640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc648156860 0x7fc648140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.985+0000 7fc6477fe640 1 -- 172.21.15.125:0/3575108142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480a7b30 con 0x7fc630004360 2023-10-18T10:40:15.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.986+0000 7fc6477fe640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 0x7fc6480a64b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc63800e570 tx=0x7fc63800ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:15.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.986+0000 7fc62ffff640 1 -- 172.21.15.125:0/3575108142 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63800c8f0 con 0x7fc630004360 2023-10-18T10:40:15.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.986+0000 7fc62ffff640 1 -- 172.21.15.125:0/3575108142 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc63800cb40 con 0x7fc630004360 2023-10-18T10:40:15.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.986+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6480a6a90 con 0x7fc630004360 2023-10-18T10:40:15.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.986+0000 7fc62ffff640 1 -- 172.21.15.125:0/3575108142 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638011460 con 0x7fc630004360 2023-10-18T10:40:15.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.986+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6480a7050 con 0x7fc630004360 2023-10-18T10:40:15.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.987+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6480690e0 con 0x7fc630004360 2023-10-18T10:40:15.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.987+0000 7fc62ffff640 1 -- 172.21.15.125:0/3575108142 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc6380115c0 con 0x7fc630004360 2023-10-18T10:40:15.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.987+0000 7fc62ffff640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fc63406af90 0x7fc63406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:15.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.988+0000 7fc62ffff640 1 -- 172.21.15.125:0/3575108142 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc638089e60 con 0x7fc630004360 2023-10-18T10:40:15.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.988+0000 7fc646ffd640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fc63406af90 0x7fc63406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:15.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.988+0000 7fc646ffd640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fc63406af90 0x7fc63406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc63c036000 tx=0x7fc63c0046c0 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:15.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:15.992+0000 7fc62ffff640 1 -- 172.21.15.125:0/3575108142 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc638054f00 con 0x7fc630004360 2023-10-18T10:40:16.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.159+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc64806f700 con 0x7fc630004360 2023-10-18T10:40:16.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.160+0000 7fc62ffff640 1 -- 172.21.15.125:0/3575108142 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+13110 (secure 0 0 0) 0x7fc63801f020 con 0x7fc630004360 2023-10-18T10:40:16.162 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:16.162 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":80,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:15.478834+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:16.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.161+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fc63406af90 msgr2=0x7fc63406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.161+0000 7fc64dd5e640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fc63406af90 0x7fc63406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc63c036000 tx=0x7fc63c0046c0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.161+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 msgr2=0x7fc6480a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.161+0000 7fc64dd5e640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 0x7fc6480a64b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc63800e570 tx=0x7fc63800ea30 comp rx=0 tx=0).stop 2023-10-18T10:40:16.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.162+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 shutdown_connections 2023-10-18T10:40:16.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.162+0000 7fc64dd5e640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc648156860 0x7fc648140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.162+0000 7fc64dd5e640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fc63406af90 0x7fc63406d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.162+0000 7fc64dd5e640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc648159f70 0x7fc64815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.162+0000 7fc64dd5e640 1 --2- 172.21.15.125:0/3575108142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc630004360 0x7fc6480a64b0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.162+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 >> 172.21.15.125:0/3575108142 conn(0x7fc648069930 msgr2=0x7fc64806ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:16.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.162+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 shutdown_connections 2023-10-18T10:40:16.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.162+0000 7fc64dd5e640 1 -- 172.21.15.125:0/3575108142 wait complete. 2023-10-18T10:40:16.174 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:40:16.175 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:40:16.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.278+0000 7fe398339640 1 -- 172.21.15.125:0/1186696396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900cf340 msgr2=0x7fe3900d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.278+0000 7fe398339640 1 --2- 172.21.15.125:0/1186696396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900cf340 0x7fe3900d1910 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe39008a120 tx=0x7fe38c02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.278+0000 7fe398339640 1 -- 172.21.15.125:0/1186696396 shutdown_connections 2023-10-18T10:40:16.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.278+0000 7fe398339640 1 --2- 172.21.15.125:0/1186696396 >> v2:172.21.15.139:3300/0 conn(0x7fe39015b2a0 0x7fe3900cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.278+0000 7fe398339640 1 --2- 172.21.15.125:0/1186696396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900cf340 0x7fe3900d1910 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.278+0000 7fe398339640 1 -- 172.21.15.125:0/1186696396 >> 172.21.15.125:0/1186696396 conn(0x7fe3900d0d60 msgr2=0x7fe3900d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:16.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.278+0000 7fe398339640 1 -- 172.21.15.125:0/1186696396 shutdown_connections 2023-10-18T10:40:16.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.278+0000 7fe398339640 1 -- 172.21.15.125:0/1186696396 wait complete. 2023-10-18T10:40:16.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.279+0000 7fe398339640 1 Processor -- start 2023-10-18T10:40:16.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.279+0000 7fe398339640 1 -- start start 2023-10-18T10:40:16.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe398339640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900ceb40 0x7fe390149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe398339640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3900cf340 0x7fe3900b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe398339640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39015b2a0 0x7fe3900b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe398339640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3900b56e0 con 0x7fe3900ceb40 2023-10-18T10:40:16.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe398339640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3901f13a0 con 0x7fe39015b2a0 2023-10-18T10:40:16.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe398339640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39014ae50 con 0x7fe3900cf340 2023-10-18T10:40:16.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900ceb40 0x7fe390149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:16.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900ceb40 0x7fe390149d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40960/0 (socket says 172.21.15.125:40960) 2023-10-18T10:40:16.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 -- 172.21.15.125:0/768858182 learned_addr learned my addr 172.21.15.125:0/768858182 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:16.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 -- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3900cf340 msgr2=0x7fe3900b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3900cf340 0x7fe3900b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 -- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39015b2a0 msgr2=0x7fe3900b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39015b2a0 0x7fe3900b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 -- 172.21.15.125:0/768858182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c009490 con 0x7fe3900ceb40 2023-10-18T10:40:16.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.280+0000 7fe3960d5640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900ceb40 0x7fe390149d40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe38c02c300 tx=0x7fe38c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:16.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.281+0000 7fe386ffd640 1 -- 172.21.15.125:0/768858182 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c003ea0 con 0x7fe3900ceb40 2023-10-18T10:40:16.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.281+0000 7fe386ffd640 1 -- 172.21.15.125:0/768858182 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe38c034440 con 0x7fe3900ceb40 2023-10-18T10:40:16.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.281+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39014a520 con 0x7fe3900ceb40 2023-10-18T10:40:16.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.281+0000 7fe386ffd640 1 -- 172.21.15.125:0/768858182 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c0347d0 con 0x7fe3900ceb40 2023-10-18T10:40:16.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.281+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39014aae0 con 0x7fe3900ceb40 2023-10-18T10:40:16.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.282+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3900690e0 con 0x7fe3900ceb40 2023-10-18T10:40:16.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.282+0000 7fe386ffd640 1 -- 172.21.15.125:0/768858182 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe38c034930 con 0x7fe3900ceb40 2023-10-18T10:40:16.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.282+0000 7fe386ffd640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe37406af60 0x7fe37406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.283+0000 7fe386ffd640 1 -- 172.21.15.125:0/768858182 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe38c0ac2c0 con 0x7fe3900ceb40 2023-10-18T10:40:16.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.283+0000 7fe3958d4640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe37406af60 0x7fe37406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:16.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.283+0000 7fe3958d4640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe37406af60 0x7fe37406d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe39015a240 tx=0x7fe380005db0 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:16.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.287+0000 7fe386ffd640 1 -- 172.21.15.125:0/768858182 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe38c077470 con 0x7fe3900ceb40 2023-10-18T10:40:16.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.449+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3900cb980 con 0x7fe3900ceb40 2023-10-18T10:40:16.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.450+0000 7fe386ffd640 1 -- 172.21.15.125:0/768858182 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe38c07aa90 con 0x7fe3900ceb40 2023-10-18T10:40:16.452 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:40:16.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe37406af60 msgr2=0x7fe37406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe37406af60 0x7fe37406d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe39015a240 tx=0x7fe380005db0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900ceb40 msgr2=0x7fe390149d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900ceb40 0x7fe390149d40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe38c02c300 tx=0x7fe38c0038a0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 shutdown_connections 2023-10-18T10:40:16.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe39015b2a0 0x7fe3900b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe37406af60 0x7fe37406d420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3900cf340 0x7fe3900b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 --2- 172.21.15.125:0/768858182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3900ceb40 0x7fe390149d40 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 >> 172.21.15.125:0/768858182 conn(0x7fe3900d0d60 msgr2=0x7fe3900d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:16.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 shutdown_connections 2023-10-18T10:40:16.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.452+0000 7fe398339640 1 -- 172.21.15.125:0/768858182 wait complete. 2023-10-18T10:40:16.464 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:40:16.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.565+0000 7f3892ee5640 1 -- 172.21.15.125:0/2534264333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f387c003970 msgr2=0x7f387c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.565+0000 7f3892ee5640 1 --2- 172.21.15.125:0/2534264333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f387c003970 0x7f387c003d90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f388000baf0 tx=0x7f387c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.565+0000 7f3892ee5640 1 -- 172.21.15.125:0/2534264333 shutdown_connections 2023-10-18T10:40:16.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.565+0000 7f3892ee5640 1 --2- 172.21.15.125:0/2534264333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f387c003970 0x7f387c003d90 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.565+0000 7f3892ee5640 1 -- 172.21.15.125:0/2534264333 >> 172.21.15.125:0/2534264333 conn(0x7f388c069930 msgr2=0x7f388c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:16.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.565+0000 7f3892ee5640 1 -- 172.21.15.125:0/2534264333 shutdown_connections 2023-10-18T10:40:16.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.565+0000 7f3892ee5640 1 -- 172.21.15.125:0/2534264333 wait complete. 2023-10-18T10:40:16.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.566+0000 7f3892ee5640 1 Processor -- start 2023-10-18T10:40:16.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.566+0000 7f3892ee5640 1 -- start start 2023-10-18T10:40:16.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3892ee5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f387c003970 0x7f388c0ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3892ee5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f388c0ab790 0x7f388c0abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3892ee5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f388c12cba0 0x7f388c12d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3892ee5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c1e3330 con 0x7f388c12cba0 2023-10-18T10:40:16.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3892ee5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c1e2290 con 0x7f388c0ab790 2023-10-18T10:40:16.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3892ee5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c1e2570 con 0x7f387c003970 2023-10-18T10:40:16.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3891482640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f388c12cba0 0x7f388c12d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:16.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3891482640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f388c12cba0 0x7f388c12d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40972/0 (socket says 172.21.15.125:40972) 2023-10-18T10:40:16.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3891482640 1 -- 172.21.15.125:0/1230712390 learned_addr learned my addr 172.21.15.125:0/1230712390 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:16.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3891482640 1 -- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f387c003970 msgr2=0x7f388c0ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3890c81640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f387c003970 0x7f388c0ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:16.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f388bfff640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f388c0ab790 0x7f388c0abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3891482640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f387c003970 0x7f388c0ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3891482640 1 -- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f388c0ab790 msgr2=0x7f388c0abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3891482640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f388c0ab790 0x7f388c0abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.567+0000 7f3891482640 1 -- 172.21.15.125:0/1230712390 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c1e2850 con 0x7f388c12cba0 2023-10-18T10:40:16.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.568+0000 7f3891482640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f388c12cba0 0x7f388c12d020 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3884000f80 tx=0x7f388400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:16.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.568+0000 7f38897fa640 1 -- 172.21.15.125:0/1230712390 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388400c9a0 con 0x7f388c12cba0 2023-10-18T10:40:16.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.568+0000 7f38897fa640 1 -- 172.21.15.125:0/1230712390 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f388400cbf0 con 0x7f388c12cba0 2023-10-18T10:40:16.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.568+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c1e2b30 con 0x7f388c12cba0 2023-10-18T10:40:16.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.568+0000 7f38897fa640 1 -- 172.21.15.125:0/1230712390 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884011460 con 0x7f388c12cba0 2023-10-18T10:40:16.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.568+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c12d560 con 0x7f388c12cba0 2023-10-18T10:40:16.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.569+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c0690e0 con 0x7f388c12cba0 2023-10-18T10:40:16.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.569+0000 7f38897fa640 1 -- 172.21.15.125:0/1230712390 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f38840115c0 con 0x7f388c12cba0 2023-10-18T10:40:16.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.570+0000 7f38897fa640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f387006af60 0x7f387006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.570+0000 7f38897fa640 1 -- 172.21.15.125:0/1230712390 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3884089e80 con 0x7f388c12cba0 2023-10-18T10:40:16.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.570+0000 7f3890c81640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f387006af60 0x7f387006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:16.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.570+0000 7f3890c81640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f387006af60 0x7f387006d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3880037000 tx=0x7f388002e040 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:16.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.574+0000 7f38897fa640 1 -- 172.21.15.125:0/1230712390 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3884058920 con 0x7f388c12cba0 2023-10-18T10:40:16.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.743+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f388c13fc20 con 0x7f388c12cba0 2023-10-18T10:40:16.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.743+0000 7f38897fa640 1 -- 172.21.15.125:0/1230712390 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f38840584b0 con 0x7f388c12cba0 2023-10-18T10:40:16.746 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:16.746 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:40:16.746 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:16.746 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:40:16.747 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:40:16.747 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:16.747 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:40:16.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.745+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f387006af60 msgr2=0x7f387006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.745+0000 7f3892ee5640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f387006af60 0x7f387006d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3880037000 tx=0x7f388002e040 comp rx=0 tx=0).stop 2023-10-18T10:40:16.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.745+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f388c12cba0 msgr2=0x7f388c12d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.745+0000 7f3892ee5640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f388c12cba0 0x7f388c12d020 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3884000f80 tx=0x7f388400ed10 comp rx=0 tx=0).stop 2023-10-18T10:40:16.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.746+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 shutdown_connections 2023-10-18T10:40:16.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.746+0000 7f3892ee5640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f388c0ab790 0x7f388c0abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.746+0000 7f3892ee5640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f387006af60 0x7f387006d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.746+0000 7f3892ee5640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f388c12cba0 0x7f388c12d020 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.746+0000 7f3892ee5640 1 --2- 172.21.15.125:0/1230712390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f387c003970 0x7f388c0ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.746+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 >> 172.21.15.125:0/1230712390 conn(0x7f388c069930 msgr2=0x7f388c13d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:16.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.746+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 shutdown_connections 2023-10-18T10:40:16.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.746+0000 7f3892ee5640 1 -- 172.21.15.125:0/1230712390 wait complete. 2023-10-18T10:40:16.758 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:40:16.758 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:40:16.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.863+0000 7ff766ef7640 1 -- 172.21.15.125:0/1353088248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff760157280 msgr2=0x7ff7600b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.863+0000 7ff766ef7640 1 --2- 172.21.15.125:0/1353088248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff760157280 0x7ff7600b1da0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff750005fd0 tx=0x7ff75002d0d0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.863+0000 7ff766ef7640 1 -- 172.21.15.125:0/1353088248 shutdown_connections 2023-10-18T10:40:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.863+0000 7ff766ef7640 1 --2- 172.21.15.125:0/1353088248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff76006cee0 0x7ff760157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.863+0000 7ff766ef7640 1 --2- 172.21.15.125:0/1353088248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff760157280 0x7ff7600b1da0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.863+0000 7ff766ef7640 1 -- 172.21.15.125:0/1353088248 >> 172.21.15.125:0/1353088248 conn(0x7ff7600aec40 msgr2=0x7ff7600af050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:16.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.863+0000 7ff766ef7640 1 -- 172.21.15.125:0/1353088248 shutdown_connections 2023-10-18T10:40:16.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.863+0000 7ff766ef7640 1 -- 172.21.15.125:0/1353088248 wait complete. 2023-10-18T10:40:16.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.864+0000 7ff766ef7640 1 Processor -- start 2023-10-18T10:40:16.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.864+0000 7ff766ef7640 1 -- start start 2023-10-18T10:40:16.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff766ef7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff76006cee0 0x7ff7601ef550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff766ef7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff760157280 0x7ff760153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff766ef7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7601583c0 0x7ff76013d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff766ef7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760153c80 con 0x7ff76006cee0 2023-10-18T10:40:16.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff766ef7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76013d870 con 0x7ff760157280 2023-10-18T10:40:16.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff766ef7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76014c3b0 con 0x7ff7601583c0 2023-10-18T10:40:16.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff76006cee0 0x7ff7601ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:16.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff76006cee0 0x7ff7601ef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40994/0 (socket says 172.21.15.125:40994) 2023-10-18T10:40:16.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 -- 172.21.15.125:0/2693531181 learned_addr learned my addr 172.21.15.125:0/2693531181 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:16.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 -- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7601583c0 msgr2=0x7ff76013d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7601583c0 0x7ff76013d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 -- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff760157280 msgr2=0x7ff760153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff760157280 0x7ff760153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 -- 172.21.15.125:0/2693531181 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75000a670 con 0x7ff76006cee0 2023-10-18T10:40:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff764c93640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff76006cee0 0x7ff7601ef550 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff750037040 tx=0x7ff750004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.865+0000 7ff75d7fa640 1 -- 172.21.15.125:0/2693531181 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750004e20 con 0x7ff76006cee0 2023-10-18T10:40:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.866+0000 7ff75d7fa640 1 -- 172.21.15.125:0/2693531181 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff7500036e0 con 0x7ff76006cee0 2023-10-18T10:40:16.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.866+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff760156900 con 0x7ff76006cee0 2023-10-18T10:40:16.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.866+0000 7ff75d7fa640 1 -- 172.21.15.125:0/2693531181 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750003ab0 con 0x7ff76006cee0 2023-10-18T10:40:16.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.866+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76014c8e0 con 0x7ff76006cee0 2023-10-18T10:40:16.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.867+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7600690e0 con 0x7ff76006cee0 2023-10-18T10:40:16.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.867+0000 7ff75d7fa640 1 -- 172.21.15.125:0/2693531181 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff750003c10 con 0x7ff76006cee0 2023-10-18T10:40:16.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.867+0000 7ff75d7fa640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7ff74c06af90 0x7ff74c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:16.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.867+0000 7ff75d7fa640 1 -- 172.21.15.125:0/2693531181 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff7500ac870 con 0x7ff76006cee0 2023-10-18T10:40:16.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.868+0000 7ff75ffff640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7ff74c06af90 0x7ff74c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:16.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.868+0000 7ff75ffff640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7ff74c06af90 0x7ff74c06d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff754005fd0 tx=0x7ff754005850 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:16.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:16.872+0000 7ff75d7fa640 1 -- 172.21.15.125:0/2693531181 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff750077a20 con 0x7ff76006cee0 2023-10-18T10:40:17.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.041+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff7600cc8f0 con 0x7ff76006cee0 2023-10-18T10:40:17.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.883+0000 7ff75d7fa640 1 -- 172.21.15.125:0/2693531181 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff75007b040 con 0x7ff76006cee0 2023-10-18T10:40:17.885 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:40:17.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.886+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7ff74c06af90 msgr2=0x7ff74c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:17.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.886+0000 7ff766ef7640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7ff74c06af90 0x7ff74c06d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff754005fd0 tx=0x7ff754005850 comp rx=0 tx=0).stop 2023-10-18T10:40:17.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.886+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff76006cee0 msgr2=0x7ff7601ef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:17.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.886+0000 7ff766ef7640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff76006cee0 0x7ff7601ef550 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff750037040 tx=0x7ff750004820 comp rx=0 tx=0).stop 2023-10-18T10:40:17.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.887+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 shutdown_connections 2023-10-18T10:40:17.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.887+0000 7ff766ef7640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff760157280 0x7ff760153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:17.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.887+0000 7ff766ef7640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7ff74c06af90 0x7ff74c06d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:17.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.887+0000 7ff766ef7640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7601583c0 0x7ff76013d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:17.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.887+0000 7ff766ef7640 1 --2- 172.21.15.125:0/2693531181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff76006cee0 0x7ff7601ef550 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:17.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.887+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 >> 172.21.15.125:0/2693531181 conn(0x7ff7600aec40 msgr2=0x7ff7600af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:17.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.887+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 shutdown_connections 2023-10-18T10:40:17.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:17.887+0000 7ff766ef7640 1 -- 172.21.15.125:0/2693531181 wait complete. 2023-10-18T10:40:17.904 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:40:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.015+0000 7f948586f640 1 -- 172.21.15.125:0/310914793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94680047c0 msgr2=0x7f9468004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.015+0000 7f948586f640 1 --2- 172.21.15.125:0/310914793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94680047c0 0x7f9468004c40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9474001680 tx=0x7f94680057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.015+0000 7f947f7fe640 1 -- 172.21.15.125:0/310914793 reap_dead start 2023-10-18T10:40:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.015+0000 7f948586f640 1 -- 172.21.15.125:0/310914793 shutdown_connections 2023-10-18T10:40:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.015+0000 7f948586f640 1 -- 172.21.15.125:0/310914793 >> 172.21.15.125:0/310914793 conn(0x7f948009a010 msgr2=0x7f94800d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.015+0000 7f948586f640 1 -- 172.21.15.125:0/310914793 shutdown_connections 2023-10-18T10:40:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.015+0000 7f948586f640 1 -- 172.21.15.125:0/310914793 wait complete. 2023-10-18T10:40:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.016+0000 7f948586f640 1 Processor -- start 2023-10-18T10:40:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f948586f640 1 -- start start 2023-10-18T10:40:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f948586f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94680047c0 0x7f948014a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f948586f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b5910 0x7f94800b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f948586f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94800b4190 0x7f94800cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f948586f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94800c8050 con 0x7f94800b4190 2023-10-18T10:40:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f948586f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948014a6e0 con 0x7f94680047c0 2023-10-18T10:40:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f948586f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94800b5df0 con 0x7f94800b5910 2023-10-18T10:40:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94800b4190 0x7f94800cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:18.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94800b4190 0x7f94800cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40514/0 (socket says 172.21.15.125:40514) 2023-10-18T10:40:18.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.017+0000 7f947f7fe640 1 -- 172.21.15.125:0/3402390941 learned_addr learned my addr 172.21.15.125:0/3402390941 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:18.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947e7fc640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b5910 0x7f94800b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:18.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947effd640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94680047c0 0x7f948014a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:18.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947f7fe640 1 -- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b5910 msgr2=0x7f94800b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:18.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947f7fe640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b5910 0x7f94800b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:18.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947f7fe640 1 -- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94680047c0 msgr2=0x7f948014a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:18.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947f7fe640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94680047c0 0x7f948014a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:18.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947f7fe640 1 -- 172.21.15.125:0/3402390941 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800b60d0 con 0x7f94800b4190 2023-10-18T10:40:18.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947effd640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94680047c0 0x7f948014a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:18.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f947f7fe640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94800b4190 0x7f94800cc9a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f94800891a0 tx=0x7f947800e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:18.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f9467fff640 1 -- 172.21.15.125:0/3402390941 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478011870 con 0x7f94800b4190 2023-10-18T10:40:18.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f9467fff640 1 -- 172.21.15.125:0/3402390941 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9478011ae0 con 0x7f94800b4190 2023-10-18T10:40:18.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94800b63b0 con 0x7f94800b4190 2023-10-18T10:40:18.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f9467fff640 1 -- 172.21.15.125:0/3402390941 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780103c0 con 0x7f94800b4190 2023-10-18T10:40:18.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.018+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94800ccee0 con 0x7f94800b4190 2023-10-18T10:40:18.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.019+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94800690e0 con 0x7f94800b4190 2023-10-18T10:40:18.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.020+0000 7f9467fff640 1 -- 172.21.15.125:0/3402390941 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9478010520 con 0x7f94800b4190 2023-10-18T10:40:18.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.020+0000 7f9467fff640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f946c06af90 0x7f946c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:18.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.020+0000 7f9467fff640 1 -- 172.21.15.125:0/3402390941 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f9478089d60 con 0x7f94800b4190 2023-10-18T10:40:18.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.020+0000 7f947effd640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f946c06af90 0x7f946c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:18.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.021+0000 7f947effd640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f946c06af90 0x7f946c06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9470005fd0 tx=0x7f9470009500 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:18.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.025+0000 7f9467fff640 1 -- 172.21.15.125:0/3402390941 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9478054c90 con 0x7f94800b4190 2023-10-18T10:40:18.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.192+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f94800c7c70 con 0x7f94800b4190 2023-10-18T10:40:18.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.885+0000 7f9467fff640 1 -- 172.21.15.125:0/3402390941 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7f94780582b0 con 0x7f94800b4190 2023-10-18T10:40:18.888 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:40:18.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f946c06af90 msgr2=0x7f946c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:18.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f946c06af90 0x7f946c06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9470005fd0 tx=0x7f9470009500 comp rx=0 tx=0).stop 2023-10-18T10:40:18.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94800b4190 msgr2=0x7f94800cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:18.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94800b4190 0x7f94800cc9a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f94800891a0 tx=0x7f947800e980 comp rx=0 tx=0).stop 2023-10-18T10:40:18.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 shutdown_connections 2023-10-18T10:40:18.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f946c06af90 0x7f946c06d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:18.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94800b4190 0x7f94800cc9a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:18.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b5910 0x7f94800b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:18.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 --2- 172.21.15.125:0/3402390941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94680047c0 0x7f948014a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:18.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.888+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 >> 172.21.15.125:0/3402390941 conn(0x7f948009a010 msgr2=0x7f94800cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:18.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.889+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 shutdown_connections 2023-10-18T10:40:18.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:18.889+0000 7f948586f640 1 -- 172.21.15.125:0/3402390941 wait complete. 2023-10-18T10:40:18.914 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:40:19.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 -- 172.21.15.125:0/1604955548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a406af00 msgr2=0x7f30a406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:19.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 --2- 172.21.15.125:0/1604955548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a406af00 0x7f30a406b2e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f30a40c9420 tx=0x7f30a002bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:19.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 -- 172.21.15.125:0/1604955548 shutdown_connections 2023-10-18T10:40:19.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 --2- 172.21.15.125:0/1604955548 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30a40ab500 0x7f30a40ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 --2- 172.21.15.125:0/1604955548 >> v2:172.21.15.139:3300/0 conn(0x7f30a40c96b0 0x7f30a40c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 --2- 172.21.15.125:0/1604955548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a406af00 0x7f30a406b2e0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 -- 172.21.15.125:0/1604955548 >> 172.21.15.125:0/1604955548 conn(0x7f30a4069930 msgr2=0x7f30a4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:19.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 -- 172.21.15.125:0/1604955548 shutdown_connections 2023-10-18T10:40:19.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.021+0000 7f30ac644640 1 -- 172.21.15.125:0/1604955548 wait complete. 2023-10-18T10:40:19.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.022+0000 7f30ac644640 1 Processor -- start 2023-10-18T10:40:19.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.022+0000 7f30ac644640 1 -- start start 2023-10-18T10:40:19.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30ac644640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30a406af00 0x7f30a40ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30ac644640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30a40ab500 0x7f30a40aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30ac644640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a40c96b0 0x7f30a415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30ac644640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40ab3c0 con 0x7f30a40c96b0 2023-10-18T10:40:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30ac644640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40a75f0 con 0x7f30a40ab500 2023-10-18T10:40:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30ac644640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40a9000 con 0x7f30a406af00 2023-10-18T10:40:19.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a40c96b0 0x7f30a415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:19.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a40c96b0 0x7f30a415b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40532/0 (socket says 172.21.15.125:40532) 2023-10-18T10:40:19.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 -- 172.21.15.125:0/857260222 learned_addr learned my addr 172.21.15.125:0/857260222 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:19.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aa3e0640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30a406af00 0x7f30a40ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:19.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30a9bdf640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30a40ab500 0x7f30a40aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:19.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 -- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30a406af00 msgr2=0x7f30a40ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:19.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30a406af00 0x7f30a40ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 -- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30a40ab500 msgr2=0x7f30a40aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:19.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30a40ab500 0x7f30a40aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 -- 172.21.15.125:0/857260222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a0009560 con 0x7f30a40c96b0 2023-10-18T10:40:19.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30aabe1640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a40c96b0 0x7f30a415b930 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f309c00eaa0 tx=0x7f309c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:19.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f309affd640 1 -- 172.21.15.125:0/857260222 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00cdb0 con 0x7f30a40c96b0 2023-10-18T10:40:19.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f309affd640 1 -- 172.21.15.125:0/857260222 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f309c004b20 con 0x7f30a40c96b0 2023-10-18T10:40:19.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a40a92e0 con 0x7f30a40c96b0 2023-10-18T10:40:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f309affd640 1 -- 172.21.15.125:0/857260222 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c005020 con 0x7f30a40c96b0 2023-10-18T10:40:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.023+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a40a98a0 con 0x7f30a40c96b0 2023-10-18T10:40:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.024+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a40a7780 con 0x7f30a40c96b0 2023-10-18T10:40:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.025+0000 7f309affd640 1 -- 172.21.15.125:0/857260222 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f309c005180 con 0x7f30a40c96b0 2023-10-18T10:40:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.025+0000 7f309affd640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f308806af60 0x7f308806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:19.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.025+0000 7f309affd640 1 -- 172.21.15.125:0/857260222 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f309c08a2f0 con 0x7f30a40c96b0 2023-10-18T10:40:19.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.026+0000 7f30aa3e0640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f308806af60 0x7f308806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:19.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.026+0000 7f30aa3e0640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f308806af60 0x7f308806d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30a0035000 tx=0x7f30a0004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:19.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.030+0000 7f309affd640 1 -- 172.21.15.125:0/857260222 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f309c055150 con 0x7f30a40c96b0 2023-10-18T10:40:19.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.230+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f30a415a190 con 0x7f30a40c96b0 2023-10-18T10:40:19.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.904+0000 7f309affd640 1 -- 172.21.15.125:0/857260222 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7f309c058770 con 0x7f30a40c96b0 2023-10-18T10:40:19.907 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 9 and data pool 10 2023-10-18T10:40:19.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.914+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f308806af60 msgr2=0x7f308806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:19.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.914+0000 7f30ac644640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f308806af60 0x7f308806d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30a0035000 tx=0x7f30a0004730 comp rx=0 tx=0).stop 2023-10-18T10:40:19.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.914+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a40c96b0 msgr2=0x7f30a415b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:19.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.914+0000 7f30ac644640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a40c96b0 0x7f30a415b930 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f309c00eaa0 tx=0x7f309c00ef60 comp rx=0 tx=0).stop 2023-10-18T10:40:19.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.917+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 shutdown_connections 2023-10-18T10:40:19.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.917+0000 7f30ac644640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30a40ab500 0x7f30a40aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.918+0000 7f30ac644640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f308806af60 0x7f308806d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.918+0000 7f30ac644640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30a40c96b0 0x7f30a415b930 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.918+0000 7f30ac644640 1 --2- 172.21.15.125:0/857260222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30a406af00 0x7f30a40ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:19.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.918+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 >> 172.21.15.125:0/857260222 conn(0x7f30a4069930 msgr2=0x7f30a40b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:19.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.919+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 shutdown_connections 2023-10-18T10:40:19.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:19.919+0000 7f30ac644640 1 -- 172.21.15.125:0/857260222 wait complete. 2023-10-18T10:40:19.936 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:20.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.039+0000 7fce3d694640 1 -- 172.21.15.125:0/3546992562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce38158910 msgr2=0x7fce38158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.039+0000 7fce3d694640 1 --2- 172.21.15.125:0/3546992562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce38158910 0x7fce38158cf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fce28005fd0 tx=0x7fce2802bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:20.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.039+0000 7fce3d694640 1 -- 172.21.15.125:0/3546992562 shutdown_connections 2023-10-18T10:40:20.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.039+0000 7fce3d694640 1 --2- 172.21.15.125:0/3546992562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce380cd1c0 0x7fce3815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.039+0000 7fce3d694640 1 --2- 172.21.15.125:0/3546992562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce38158910 0x7fce38158cf0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.040+0000 7fce3d694640 1 -- 172.21.15.125:0/3546992562 >> 172.21.15.125:0/3546992562 conn(0x7fce380af7a0 msgr2=0x7fce380afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:20.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.040+0000 7fce3d694640 1 -- 172.21.15.125:0/3546992562 shutdown_connections 2023-10-18T10:40:20.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.040+0000 7fce3d694640 1 -- 172.21.15.125:0/3546992562 wait complete. 2023-10-18T10:40:20.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce3d694640 1 Processor -- start 2023-10-18T10:40:20.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce3d694640 1 -- start start 2023-10-18T10:40:20.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce3d694640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce380cd1c0 0x7fce3814be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce3d694640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce380cd9e0 0x7fce3814a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce3d694640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce38158910 0x7fce3815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce3d694640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3814e2c0 con 0x7fce380cd9e0 2023-10-18T10:40:20.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce3d694640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3814c360 con 0x7fce380cd1c0 2023-10-18T10:40:20.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce3d694640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce38153e60 con 0x7fce38158910 2023-10-18T10:40:20.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce2ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce380cd9e0 0x7fce3814a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce2ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce380cd9e0 0x7fce3814a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40554/0 (socket says 172.21.15.125:40554) 2023-10-18T10:40:20.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.041+0000 7fce2ffff640 1 -- 172.21.15.125:0/1263909659 learned_addr learned my addr 172.21.15.125:0/1263909659 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:20.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce377fe640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce38158910 0x7fce3815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce36ffd640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce380cd1c0 0x7fce3814be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2ffff640 1 -- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce38158910 msgr2=0x7fce3815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2ffff640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce38158910 0x7fce3815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2ffff640 1 -- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce380cd1c0 msgr2=0x7fce3814be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2ffff640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce380cd1c0 0x7fce3814be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2ffff640 1 -- 172.21.15.125:0/1263909659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce28009560 con 0x7fce380cd9e0 2023-10-18T10:40:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2ffff640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce380cd9e0 0x7fce3814a0f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fce2000d770 tx=0x7fce2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2f7fe640 1 -- 172.21.15.125:0/1263909659 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20009c30 con 0x7fce380cd9e0 2023-10-18T10:40:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2f7fe640 1 -- 172.21.15.125:0/1263909659 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce2000de80 con 0x7fce380cd9e0 2023-10-18T10:40:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce38154140 con 0x7fce380cd9e0 2023-10-18T10:40:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce2f7fe640 1 -- 172.21.15.125:0/1263909659 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20005280 con 0x7fce380cd9e0 2023-10-18T10:40:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.042+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce38154700 con 0x7fce380cd9e0 2023-10-18T10:40:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.043+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce38154920 con 0x7fce380cd9e0 2023-10-18T10:40:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.044+0000 7fce2f7fe640 1 -- 172.21.15.125:0/1263909659 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fce200053e0 con 0x7fce380cd9e0 2023-10-18T10:40:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.044+0000 7fce2f7fe640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce1806af90 0x7fce1806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.044+0000 7fce2f7fe640 1 -- 172.21.15.125:0/1263909659 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fce20089bb0 con 0x7fce380cd9e0 2023-10-18T10:40:20.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.044+0000 7fce36ffd640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce1806af90 0x7fce1806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.045+0000 7fce36ffd640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce1806af90 0x7fce1806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce28035000 tx=0x7fce28004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:20.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.048+0000 7fce2f7fe640 1 -- 172.21.15.125:0/1263909659 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce20054920 con 0x7fce380cd9e0 2023-10-18T10:40:20.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.218+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce380690e0 con 0x7fce380cd9e0 2023-10-18T10:40:20.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.219+0000 7fce2f7fe640 1 -- 172.21.15.125:0/1263909659 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fce20017070 con 0x7fce380cd9e0 2023-10-18T10:40:20.221 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:20.221 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":83,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:19.231507+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:17.060551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:18.211723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:20.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce1806af90 msgr2=0x7fce1806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce1806af90 0x7fce1806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce28035000 tx=0x7fce28004730 comp rx=0 tx=0).stop 2023-10-18T10:40:20.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce380cd9e0 msgr2=0x7fce3814a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce380cd9e0 0x7fce3814a0f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fce2000d770 tx=0x7fce2000dc30 comp rx=0 tx=0).stop 2023-10-18T10:40:20.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 shutdown_connections 2023-10-18T10:40:20.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce1806af90 0x7fce1806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce38158910 0x7fce3815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce380cd9e0 0x7fce3814a0f0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 --2- 172.21.15.125:0/1263909659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce380cd1c0 0x7fce3814be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 >> 172.21.15.125:0/1263909659 conn(0x7fce380af7a0 msgr2=0x7fce380cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:20.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 shutdown_connections 2023-10-18T10:40:20.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.221+0000 7fce3d694640 1 -- 172.21.15.125:0/1263909659 wait complete. 2023-10-18T10:40:20.233 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:20.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.334+0000 7f70b147c640 1 -- 172.21.15.125:0/871077144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 msgr2=0x7f70ac06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.334+0000 7f70b147c640 1 --2- 172.21.15.125:0/871077144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 0x7f70ac06bb60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f709c005fd0 tx=0x7f709c02c1b0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.334+0000 7f70b147c640 1 -- 172.21.15.125:0/871077144 shutdown_connections 2023-10-18T10:40:20.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.334+0000 7f70b147c640 1 --2- 172.21.15.125:0/871077144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70ac070af0 0x7f70ac0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.334+0000 7f70b147c640 1 --2- 172.21.15.125:0/871077144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 0x7f70ac06bb60 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.334+0000 7f70b147c640 1 -- 172.21.15.125:0/871077144 >> 172.21.15.125:0/871077144 conn(0x7f70ac069930 msgr2=0x7f70ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:20.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.334+0000 7f70b147c640 1 -- 172.21.15.125:0/871077144 shutdown_connections 2023-10-18T10:40:20.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.334+0000 7f70b147c640 1 -- 172.21.15.125:0/871077144 wait complete. 2023-10-18T10:40:20.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.335+0000 7f70b147c640 1 Processor -- start 2023-10-18T10:40:20.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.335+0000 7f70b147c640 1 -- start start 2023-10-18T10:40:20.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70b147c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70ac070af0 0x7f70ac1edf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70b147c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 0x7f70ac13bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70b147c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70ac141690 0x7f70ac151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70b147c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac13c990 con 0x7f70ac13e520 2023-10-18T10:40:20.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70b147c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac13cc70 con 0x7f70ac070af0 2023-10-18T10:40:20.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70b147c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac13cf50 con 0x7f70ac141690 2023-10-18T10:40:20.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70aa7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 0x7f70ac13bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70aa7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 0x7f70ac13bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40574/0 (socket says 172.21.15.125:40574) 2023-10-18T10:40:20.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70aa7fc640 1 -- 172.21.15.125:0/3645283886 learned_addr learned my addr 172.21.15.125:0/3645283886 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:20.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70ab7fe640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70ac141690 0x7f70ac151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.336+0000 7f70aaffd640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70ac070af0 0x7f70ac1edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70aa7fc640 1 -- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70ac141690 msgr2=0x7f70ac151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70aa7fc640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70ac141690 0x7f70ac151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70aa7fc640 1 -- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70ac070af0 msgr2=0x7f70ac1edf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70aa7fc640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70ac070af0 0x7f70ac1edf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70aa7fc640 1 -- 172.21.15.125:0/3645283886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c009900 con 0x7f70ac13e520 2023-10-18T10:40:20.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70aaffd640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70ac070af0 0x7f70ac1edf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:20.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70aa7fc640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 0x7f70ac13bf30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b3f0 tx=0x7f70a000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:20.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70937fe640 1 -- 172.21.15.125:0/3645283886 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011830 con 0x7f70ac13e520 2023-10-18T10:40:20.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70937fe640 1 -- 172.21.15.125:0/3645283886 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f70a0011a80 con 0x7f70ac13e520 2023-10-18T10:40:20.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac13d230 con 0x7f70ac13e520 2023-10-18T10:40:20.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70937fe640 1 -- 172.21.15.125:0/3645283886 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011e90 con 0x7f70ac13e520 2023-10-18T10:40:20.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.337+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac13da30 con 0x7f70ac13e520 2023-10-18T10:40:20.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.338+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac0690e0 con 0x7f70ac13e520 2023-10-18T10:40:20.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.338+0000 7f70937fe640 1 -- 172.21.15.125:0/3645283886 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f70a001a400 con 0x7f70ac13e520 2023-10-18T10:40:20.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.339+0000 7f70937fe640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f709806af90 0x7f709806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.339+0000 7f70937fe640 1 -- 172.21.15.125:0/3645283886 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f70a0089840 con 0x7f70ac13e520 2023-10-18T10:40:20.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.339+0000 7f70aaffd640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f709806af90 0x7f709806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.339+0000 7f70aaffd640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f709806af90 0x7f709806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f709c036000 tx=0x7f709c004ab0 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:20.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.343+0000 7f70937fe640 1 -- 172.21.15.125:0/3645283886 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70a0054730 con 0x7f70ac13e520 2023-10-18T10:40:20.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.511+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70ac0c98a0 con 0x7f70ac13e520 2023-10-18T10:40:20.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.512+0000 7f70937fe640 1 -- 172.21.15.125:0/3645283886 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f70a0015090 con 0x7f70ac13e520 2023-10-18T10:40:20.514 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:20.514 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":83,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:19.231507+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:17.060551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:18.211723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:20.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f709806af90 msgr2=0x7f709806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f709806af90 0x7f709806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f709c036000 tx=0x7f709c004ab0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 msgr2=0x7f70ac13bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 0x7f70ac13bf30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b3f0 tx=0x7f70a000b8b0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 shutdown_connections 2023-10-18T10:40:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f709806af90 0x7f709806d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70ac141690 0x7f70ac151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70ac13e520 0x7f70ac13bf30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 --2- 172.21.15.125:0/3645283886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70ac070af0 0x7f70ac1edf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 >> 172.21.15.125:0/3645283886 conn(0x7f70ac069930 msgr2=0x7f70ac158870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 shutdown_connections 2023-10-18T10:40:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.514+0000 7f70b147c640 1 -- 172.21.15.125:0/3645283886 wait complete. 2023-10-18T10:40:20.527 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:40:20.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.630+0000 7f11479b6640 1 -- 172.21.15.125:0/3965818987 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400cbd20 msgr2=0x7f1140158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.630+0000 7f11479b6640 1 --2- 172.21.15.125:0/3965818987 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400cbd20 0x7f1140158300 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f11400cc490 tx=0x7f113802be40 comp rx=0 tx=0).stop 2023-10-18T10:40:20.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.630+0000 7f11479b6640 1 -- 172.21.15.125:0/3965818987 shutdown_connections 2023-10-18T10:40:20.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.630+0000 7f11479b6640 1 --2- 172.21.15.125:0/3965818987 >> v2:172.21.15.139:3300/0 conn(0x7f11400afa20 0x7f11400705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.630+0000 7f11479b6640 1 --2- 172.21.15.125:0/3965818987 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400cbd20 0x7f1140158300 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.630+0000 7f11479b6640 1 -- 172.21.15.125:0/3965818987 >> 172.21.15.125:0/3965818987 conn(0x7f1140069930 msgr2=0x7f1140069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:20.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.630+0000 7f11479b6640 1 -- 172.21.15.125:0/3965818987 shutdown_connections 2023-10-18T10:40:20.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.630+0000 7f11479b6640 1 -- 172.21.15.125:0/3965818987 wait complete. 2023-10-18T10:40:20.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.631+0000 7f11479b6640 1 Processor -- start 2023-10-18T10:40:20.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.631+0000 7f11479b6640 1 -- start start 2023-10-18T10:40:20.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f11479b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400afa20 0x7f114014ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f11479b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11400b0530 0x7f114013c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f11479b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11400cbd20 0x7f11401599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f11479b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401e23b0 con 0x7f11400afa20 2023-10-18T10:40:20.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f11479b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401485e0 con 0x7f11400cbd20 2023-10-18T10:40:20.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f11479b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114014acd0 con 0x7f11400b0530 2023-10-18T10:40:20.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145752640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400afa20 0x7f114014ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145752640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400afa20 0x7f114014ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40582/0 (socket says 172.21.15.125:40582) 2023-10-18T10:40:20.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145752640 1 -- 172.21.15.125:0/798974361 learned_addr learned my addr 172.21.15.125:0/798974361 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:20.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145f53640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11400cbd20 0x7f11401599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1144f51640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11400b0530 0x7f114013c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145752640 1 -- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11400b0530 msgr2=0x7f114013c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145752640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11400b0530 0x7f114013c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145752640 1 -- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11400cbd20 msgr2=0x7f11401599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145752640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11400cbd20 0x7f11401599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1145752640 1 -- 172.21.15.125:0/798974361 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138009560 con 0x7f11400afa20 2023-10-18T10:40:20.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.632+0000 7f1144f51640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11400b0530 0x7f114013c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:20.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.633+0000 7f1145752640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400afa20 0x7f114014ba00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f11380099c0 tx=0x7f113802de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:20.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.633+0000 7f113dffb640 1 -- 172.21.15.125:0/798974361 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138005670 con 0x7f11400afa20 2023-10-18T10:40:20.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.633+0000 7f113dffb640 1 -- 172.21.15.125:0/798974361 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f11380038f0 con 0x7f11400afa20 2023-10-18T10:40:20.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.633+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11401e20d0 con 0x7f11400afa20 2023-10-18T10:40:20.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.633+0000 7f113dffb640 1 -- 172.21.15.125:0/798974361 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138003ce0 con 0x7f11400afa20 2023-10-18T10:40:20.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.633+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114014b200 con 0x7f11400afa20 2023-10-18T10:40:20.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.634+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11400690e0 con 0x7f11400afa20 2023-10-18T10:40:20.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.634+0000 7f113dffb640 1 -- 172.21.15.125:0/798974361 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f113803e450 con 0x7f11400afa20 2023-10-18T10:40:20.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.635+0000 7f113dffb640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f112c06af60 0x7f112c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:20.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.635+0000 7f113dffb640 1 -- 172.21.15.125:0/798974361 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f11380ac3e0 con 0x7f11400afa20 2023-10-18T10:40:20.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.635+0000 7f1144f51640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f112c06af60 0x7f112c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:20.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.635+0000 7f1144f51640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f112c06af60 0x7f112c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f114015a560 tx=0x7f1134005db0 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:20.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.639+0000 7f113dffb640 1 -- 172.21.15.125:0/798974361 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1138077240 con 0x7f11400afa20 2023-10-18T10:40:20.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.844+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f114006f700 con 0x7f11400afa20 2023-10-18T10:40:20.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.911+0000 7f113dffb640 1 -- 172.21.15.125:0/798974361 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7f113807a860 con 0x7f11400afa20 2023-10-18T10:40:20.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f112c06af60 msgr2=0x7f112c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f112c06af60 0x7f112c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f114015a560 tx=0x7f1134005db0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400afa20 msgr2=0x7f114014ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400afa20 0x7f114014ba00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f11380099c0 tx=0x7f113802de10 comp rx=0 tx=0).stop 2023-10-18T10:40:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 shutdown_connections 2023-10-18T10:40:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11400cbd20 0x7f11401599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f112c06af60 0x7f112c06d420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11400b0530 0x7f114013c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 --2- 172.21.15.125:0/798974361 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11400afa20 0x7f114014ba00 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 >> 172.21.15.125:0/798974361 conn(0x7f1140069930 msgr2=0x7f11400caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 shutdown_connections 2023-10-18T10:40:20.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:20.914+0000 7f11479b6640 1 -- 172.21.15.125:0/798974361 wait complete. 2023-10-18T10:40:20.926 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:40:20.927 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:40:21.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.030+0000 7f66b7e75640 1 -- 172.21.15.125:0/1796887562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00afa20 msgr2=0x7f66b00705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:21.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.030+0000 7f66b7e75640 1 --2- 172.21.15.125:0/1796887562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00afa20 0x7f66b00705a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f66a0005fd0 tx=0x7f66a002d0d0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.030+0000 7f66b7e75640 1 -- 172.21.15.125:0/1796887562 shutdown_connections 2023-10-18T10:40:21.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.030+0000 7f66b7e75640 1 --2- 172.21.15.125:0/1796887562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00afa20 0x7f66b00705a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.030+0000 7f66b7e75640 1 --2- 172.21.15.125:0/1796887562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66b00cbd20 0x7f66b0158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.031+0000 7f66b7e75640 1 -- 172.21.15.125:0/1796887562 >> 172.21.15.125:0/1796887562 conn(0x7f66b0069930 msgr2=0x7f66b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:21.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.031+0000 7f66b7e75640 1 -- 172.21.15.125:0/1796887562 shutdown_connections 2023-10-18T10:40:21.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.031+0000 7f66b7e75640 1 -- 172.21.15.125:0/1796887562 wait complete. 2023-10-18T10:40:21.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b7e75640 1 Processor -- start 2023-10-18T10:40:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b7e75640 1 -- start start 2023-10-18T10:40:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b7e75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66b00afa20 0x7f66b0144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b7e75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66b00b0530 0x7f66b00ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:21.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b7e75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00cbd20 0x7f66b00adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:21.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b7e75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b014c780 con 0x7f66b00cbd20 2023-10-18T10:40:21.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b7e75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b014bb70 con 0x7f66b00b0530 2023-10-18T10:40:21.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b7e75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b014be50 con 0x7f66b00afa20 2023-10-18T10:40:21.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.032+0000 7f66b6412640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00cbd20 0x7f66b00adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:21.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b6412640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00cbd20 0x7f66b00adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40608/0 (socket says 172.21.15.125:40608) 2023-10-18T10:40:21.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b6412640 1 -- 172.21.15.125:0/763472057 learned_addr learned my addr 172.21.15.125:0/763472057 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:21.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b6412640 1 -- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66b00afa20 msgr2=0x7f66b0144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:21.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b6412640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66b00afa20 0x7f66b0144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b6412640 1 -- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66b00b0530 msgr2=0x7f66b00ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:21.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b5410640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66b00b0530 0x7f66b00ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:21.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b6412640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66b00b0530 0x7f66b00ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b6412640 1 -- 172.21.15.125:0/763472057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a000a670 con 0x7f66b00cbd20 2023-10-18T10:40:21.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b6412640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00cbd20 0x7f66b00adfb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c780 tx=0x7f66a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:21.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66a67fc640 1 -- 172.21.15.125:0/763472057 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8004b60 con 0x7f66b00cbd20 2023-10-18T10:40:21.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66a67fc640 1 -- 172.21.15.125:0/763472057 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66a800ce90 con 0x7f66b00cbd20 2023-10-18T10:40:21.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b0070cf0 con 0x7f66b00cbd20 2023-10-18T10:40:21.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66a67fc640 1 -- 172.21.15.125:0/763472057 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800aab0 con 0x7f66b00cbd20 2023-10-18T10:40:21.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.033+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b014c410 con 0x7f66b00cbd20 2023-10-18T10:40:21.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.034+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b00690e0 con 0x7f66b00cbd20 2023-10-18T10:40:21.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.034+0000 7f66a67fc640 1 -- 172.21.15.125:0/763472057 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f66a800ac10 con 0x7f66b00cbd20 2023-10-18T10:40:21.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.035+0000 7f66a67fc640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f669406af60 0x7f669406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:21.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.035+0000 7f66a67fc640 1 -- 172.21.15.125:0/763472057 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f66a8089d10 con 0x7f66b00cbd20 2023-10-18T10:40:21.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.035+0000 7f66b5c11640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f669406af60 0x7f669406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:21.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.036+0000 7f66b5c11640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f669406af60 0x7f669406d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00a860 tx=0x7f66ac005b50 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:21.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.039+0000 7f66a67fc640 1 -- 172.21.15.125:0/763472057 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66a8054b70 con 0x7f66b00cbd20 2023-10-18T10:40:21.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.238+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f66b00cc1e0 con 0x7f66b00cbd20 2023-10-18T10:40:21.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.919+0000 7f66a67fc640 1 -- 172.21.15.125:0/763472057 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7f66a8058190 con 0x7f66b00cbd20 2023-10-18T10:40:21.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f669406af60 msgr2=0x7f669406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:21.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f669406af60 0x7f669406d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00a860 tx=0x7f66ac005b50 comp rx=0 tx=0).stop 2023-10-18T10:40:21.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00cbd20 msgr2=0x7f66b00adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:21.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00cbd20 0x7f66b00adfb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c780 tx=0x7f66a800cc40 comp rx=0 tx=0).stop 2023-10-18T10:40:21.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 shutdown_connections 2023-10-18T10:40:21.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66b00b0530 0x7f66b00ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f669406af60 0x7f669406d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66b00cbd20 0x7f66b00adfb0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 --2- 172.21.15.125:0/763472057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66b00afa20 0x7f66b0144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:21.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 >> 172.21.15.125:0/763472057 conn(0x7f66b0069930 msgr2=0x7f66b006beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:21.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 shutdown_connections 2023-10-18T10:40:21.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:21.922+0000 7f66b7e75640 1 -- 172.21.15.125:0/763472057 wait complete. 2023-10-18T10:40:21.937 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:40:21.938 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:22.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.045+0000 7fe50a368640 1 -- 172.21.15.125:0/1009578406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4ec0058d0 msgr2=0x7fe4ec005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.045+0000 7fe50a368640 1 --2- 172.21.15.125:0/1009578406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4ec0058d0 0x7fe4ec005d50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009b20 tx=0x7fe4ec006900 comp rx=0 tx=0).stop 2023-10-18T10:40:22.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.045+0000 7fe508905640 1 -- 172.21.15.125:0/1009578406 reap_dead start 2023-10-18T10:40:22.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.045+0000 7fe50a368640 1 -- 172.21.15.125:0/1009578406 shutdown_connections 2023-10-18T10:40:22.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.045+0000 7fe50a368640 1 -- 172.21.15.125:0/1009578406 >> 172.21.15.125:0/1009578406 conn(0x7fe504069930 msgr2=0x7fe504069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:22.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.045+0000 7fe50a368640 1 -- 172.21.15.125:0/1009578406 shutdown_connections 2023-10-18T10:40:22.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.045+0000 7fe50a368640 1 -- 172.21.15.125:0/1009578406 wait complete. 2023-10-18T10:40:22.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.046+0000 7fe50a368640 1 Processor -- start 2023-10-18T10:40:22.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe50a368640 1 -- start start 2023-10-18T10:40:22.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe50a368640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4ec0058d0 0x7fe50414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe50a368640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe504148000 0x7fe5041e33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe50a368640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5041e38f0 0x7fe504134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe50a368640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50414ce80 con 0x7fe5041e38f0 2023-10-18T10:40:22.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe50a368640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50414c0c0 con 0x7fe4ec0058d0 2023-10-18T10:40:22.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe50a368640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50414c3a0 con 0x7fe504148000 2023-10-18T10:40:22.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe508905640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5041e38f0 0x7fe504134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe508905640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5041e38f0 0x7fe504134fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40626/0 (socket says 172.21.15.125:40626) 2023-10-18T10:40:22.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.047+0000 7fe508905640 1 -- 172.21.15.125:0/3130045068 learned_addr learned my addr 172.21.15.125:0/3130045068 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:22.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe508905640 1 -- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe504148000 msgr2=0x7fe5041e33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe503fff640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4ec0058d0 0x7fe50414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe5037fe640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe504148000 0x7fe5041e33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe508905640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe504148000 0x7fe5041e33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe508905640 1 -- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4ec0058d0 msgr2=0x7fe50414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe508905640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4ec0058d0 0x7fe50414afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe508905640 1 -- 172.21.15.125:0/3130045068 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50414c680 con 0x7fe5041e38f0 2023-10-18T10:40:22.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe508905640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5041e38f0 0x7fe504134fc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe50406f6d0 tx=0x7fe4fc001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:22.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe500ff9640 1 -- 172.21.15.125:0/3130045068 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc011900 con 0x7fe5041e38f0 2023-10-18T10:40:22.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe500ff9640 1 -- 172.21.15.125:0/3130045068 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe4fc011a80 con 0x7fe5041e38f0 2023-10-18T10:40:22.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50414c960 con 0x7fe5041e38f0 2023-10-18T10:40:22.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe500ff9640 1 -- 172.21.15.125:0/3130045068 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc011e90 con 0x7fe5041e38f0 2023-10-18T10:40:22.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.048+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe504148480 con 0x7fe5041e38f0 2023-10-18T10:40:22.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.049+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5040690e0 con 0x7fe5041e38f0 2023-10-18T10:40:22.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.049+0000 7fe500ff9640 1 -- 172.21.15.125:0/3130045068 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe4fc01d460 con 0x7fe5041e38f0 2023-10-18T10:40:22.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.050+0000 7fe500ff9640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe4f006af90 0x7fe4f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.050+0000 7fe500ff9640 1 -- 172.21.15.125:0/3130045068 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe4fc08a550 con 0x7fe5041e38f0 2023-10-18T10:40:22.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.050+0000 7fe503fff640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe4f006af90 0x7fe4f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.051+0000 7fe503fff640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe4f006af90 0x7fe4f006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4001050 tx=0x7fe4f4001b90 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:22.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.054+0000 7fe500ff9640 1 -- 172.21.15.125:0/3130045068 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4fc055280 con 0x7fe5041e38f0 2023-10-18T10:40:22.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.254+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe50406f700 con 0x7fe5041e38f0 2023-10-18T10:40:22.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.254+0000 7fe500ff9640 1 -- 172.21.15.125:0/3130045068 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fe4fc0588a0 con 0x7fe5041e38f0 2023-10-18T10:40:22.257 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:22.257 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/693466412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":693466412},{"type":"v1","addr":"172.21.15.139:6836","nonce":693466412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6277,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/170638031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":170638031},{"type":"v1","addr":"172.21.15.139:6837","nonce":170638031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":7068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3024029084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3024029084},{"type":"v1","addr":"172.21.15.125:6834","nonce":3024029084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:19.231823+0000","modified":"2023-10-18T10:40:21.240206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1136111019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1136111019},{"type":"v1","addr":"172.21.15.125:6835","nonce":1136111019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T10:40:22.259 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 29 2023-10-18T10:40:22.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe4f006af90 msgr2=0x7fe4f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe4f006af90 0x7fe4f006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4001050 tx=0x7fe4f4001b90 comp rx=0 tx=0).stop 2023-10-18T10:40:22.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5041e38f0 msgr2=0x7fe504134fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5041e38f0 0x7fe504134fc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe50406f6d0 tx=0x7fe4fc001c30 comp rx=0 tx=0).stop 2023-10-18T10:40:22.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 shutdown_connections 2023-10-18T10:40:22.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fe4f006af90 0x7fe4f006d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5041e38f0 0x7fe504134fc0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe504148000 0x7fe5041e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 --2- 172.21.15.125:0/3130045068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4ec0058d0 0x7fe50414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 >> 172.21.15.125:0/3130045068 conn(0x7fe504069930 msgr2=0x7fe504065590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:22.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 shutdown_connections 2023-10-18T10:40:22.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.256+0000 7fe50a368640 1 -- 172.21.15.125:0/3130045068 wait complete. 2023-10-18T10:40:22.269 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:22.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.369+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1220587748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc15b2a0 msgr2=0x7fa9cc0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.369+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1220587748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc15b2a0 0x7fa9cc0cf720 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc005fd0 tx=0x7fa9bc02bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:22.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.369+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1220587748 shutdown_connections 2023-10-18T10:40:22.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.369+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1220587748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc15b2a0 0x7fa9cc0cf720 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.369+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1220587748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9cc0cf340 0x7fa9cc0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.369+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1220587748 >> 172.21.15.125:0/1220587748 conn(0x7fa9cc0d0d60 msgr2=0x7fa9cc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:22.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.369+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1220587748 shutdown_connections 2023-10-18T10:40:22.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.369+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1220587748 wait complete. 2023-10-18T10:40:22.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.370+0000 7fa9d3f2c640 1 Processor -- start 2023-10-18T10:40:22.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.370+0000 7fa9d3f2c640 1 -- start start 2023-10-18T10:40:22.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d3f2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9cc0ceb40 0x7fa9cc158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d3f2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc0cf340 0x7fa9cc158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d3f2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9cc15b2a0 0x7fa9cc159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d3f2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1f18a0 con 0x7fa9cc0cf340 2023-10-18T10:40:22.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d3f2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1f1b80 con 0x7fa9cc15b2a0 2023-10-18T10:40:22.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d3f2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1f1e60 con 0x7fa9cc0ceb40 2023-10-18T10:40:22.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d14c7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc0cf340 0x7fa9cc158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d14c7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc0cf340 0x7fa9cc158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40646/0 (socket says 172.21.15.125:40646) 2023-10-18T10:40:22.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d14c7640 1 -- 172.21.15.125:0/1754114935 learned_addr learned my addr 172.21.15.125:0/1754114935 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:22.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d1cc8640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9cc0ceb40 0x7fa9cc158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d24c9640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9cc15b2a0 0x7fa9cc159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d14c7640 1 -- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9cc0ceb40 msgr2=0x7fa9cc158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d14c7640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9cc0ceb40 0x7fa9cc158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d14c7640 1 -- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9cc15b2a0 msgr2=0x7fa9cc159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d14c7640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9cc15b2a0 0x7fa9cc159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d14c7640 1 -- 172.21.15.125:0/1754114935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc009560 con 0x7fa9cc0cf340 2023-10-18T10:40:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.371+0000 7fa9d24c9640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9cc15b2a0 0x7fa9cc159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.372+0000 7fa9d14c7640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc0cf340 0x7fa9cc158fd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc006010 tx=0x7fa9bc02d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:22.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.372+0000 7fa9c27fc640 1 -- 172.21.15.125:0/1754114935 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc02ddf0 con 0x7fa9cc0cf340 2023-10-18T10:40:22.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.372+0000 7fa9c27fc640 1 -- 172.21.15.125:0/1754114935 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa9bc0036e0 con 0x7fa9cc0cf340 2023-10-18T10:40:22.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.372+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc1f2140 con 0x7fa9cc0cf340 2023-10-18T10:40:22.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.372+0000 7fa9c27fc640 1 -- 172.21.15.125:0/1754114935 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc003af0 con 0x7fa9cc0cf340 2023-10-18T10:40:22.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.372+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc09e910 con 0x7fa9cc0cf340 2023-10-18T10:40:22.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.373+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc09eaa0 con 0x7fa9cc0cf340 2023-10-18T10:40:22.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.373+0000 7fa9c27fc640 1 -- 172.21.15.125:0/1754114935 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa9bc003c50 con 0x7fa9cc0cf340 2023-10-18T10:40:22.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.373+0000 7fa9c27fc640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa9b006af60 0x7fa9b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.373+0000 7fa9c27fc640 1 -- 172.21.15.125:0/1754114935 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa9bc0ac4e0 con 0x7fa9cc0cf340 2023-10-18T10:40:22.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.374+0000 7fa9d1cc8640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa9b006af60 0x7fa9b006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.374+0000 7fa9d1cc8640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa9b006af60 0x7fa9b006d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005fd0 tx=0x7fa9c8009500 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:22.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.377+0000 7fa9c27fc640 1 -- 172.21.15.125:0/1754114935 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9bc0773f0 con 0x7fa9cc0cf340 2023-10-18T10:40:22.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.546+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9cc1f2600 con 0x7fa9cc0cf340 2023-10-18T10:40:22.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.547+0000 7fa9c27fc640 1 -- 172.21.15.125:0/1754114935 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16764 (secure 0 0 0) 0x7fa9bc07aa10 con 0x7fa9cc0cf340 2023-10-18T10:40:22.549 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:22.549 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":84,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:20.895150+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:17.060551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:18.211723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:22.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa9b006af60 msgr2=0x7fa9b006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa9b006af60 0x7fa9b006d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005fd0 tx=0x7fa9c8009500 comp rx=0 tx=0).stop 2023-10-18T10:40:22.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc0cf340 msgr2=0x7fa9cc158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc0cf340 0x7fa9cc158fd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc006010 tx=0x7fa9bc02d9f0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 shutdown_connections 2023-10-18T10:40:22.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9cc15b2a0 0x7fa9cc159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fa9b006af60 0x7fa9b006d420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9cc0cf340 0x7fa9cc158fd0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 --2- 172.21.15.125:0/1754114935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9cc0ceb40 0x7fa9cc158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 >> 172.21.15.125:0/1754114935 conn(0x7fa9cc0d0d60 msgr2=0x7fa9cc15d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:22.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 shutdown_connections 2023-10-18T10:40:22.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.549+0000 7fa9d3f2c640 1 -- 172.21.15.125:0/1754114935 wait complete. 2023-10-18T10:40:22.561 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:40:22.562 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:22.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.662+0000 7efc218a2640 1 -- 172.21.15.125:0/1189645388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc040039a0 msgr2=0x7efc04003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.662+0000 7efc218a2640 1 --2- 172.21.15.125:0/1189645388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc040039a0 0x7efc04003d80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7efc1c061510 tx=0x7efc040057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.662+0000 7efc1b7fe640 1 -- 172.21.15.125:0/1189645388 reap_dead start 2023-10-18T10:40:22.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.662+0000 7efc218a2640 1 -- 172.21.15.125:0/1189645388 shutdown_connections 2023-10-18T10:40:22.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.662+0000 7efc218a2640 1 -- 172.21.15.125:0/1189645388 >> 172.21.15.125:0/1189645388 conn(0x7efc1c06fae0 msgr2=0x7efc1c06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:22.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.662+0000 7efc218a2640 1 -- 172.21.15.125:0/1189645388 shutdown_connections 2023-10-18T10:40:22.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.662+0000 7efc218a2640 1 -- 172.21.15.125:0/1189645388 wait complete. 2023-10-18T10:40:22.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.663+0000 7efc218a2640 1 Processor -- start 2023-10-18T10:40:22.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc218a2640 1 -- start start 2023-10-18T10:40:22.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc218a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc040039a0 0x7efc1c147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc218a2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc1c1565c0 0x7efc1c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc218a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc1c0ae770 0x7efc1c150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc218a2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c148670 con 0x7efc1c0ae770 2023-10-18T10:40:22.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc218a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c0ad790 con 0x7efc040039a0 2023-10-18T10:40:22.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc218a2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c154ca0 con 0x7efc1c1565c0 2023-10-18T10:40:22.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc1b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc1c0ae770 0x7efc1c150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc1b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc1c0ae770 0x7efc1c150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40676/0 (socket says 172.21.15.125:40676) 2023-10-18T10:40:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.664+0000 7efc1b7fe640 1 -- 172.21.15.125:0/2451400034 learned_addr learned my addr 172.21.15.125:0/2451400034 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1affd640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc040039a0 0x7efc1c147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1a7fc640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc1c1565c0 0x7efc1c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1b7fe640 1 -- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc1c1565c0 msgr2=0x7efc1c0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1b7fe640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc1c1565c0 0x7efc1c0ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1b7fe640 1 -- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc040039a0 msgr2=0x7efc1c147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1b7fe640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc040039a0 0x7efc1c147a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1b7fe640 1 -- 172.21.15.125:0/2451400034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0b2330 con 0x7efc1c0ae770 2023-10-18T10:40:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1affd640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc040039a0 0x7efc1c147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:22.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc1b7fe640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc1c0ae770 0x7efc1c150d10 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7efc1c071330 tx=0x7efc140039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:22.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc03fff640 1 -- 172.21.15.125:0/2451400034 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14011840 con 0x7efc1c0ae770 2023-10-18T10:40:22.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc03fff640 1 -- 172.21.15.125:0/2451400034 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc140119a0 con 0x7efc1c0ae770 2023-10-18T10:40:22.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc03fff640 1 -- 172.21.15.125:0/2451400034 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14011db0 con 0x7efc1c0ae770 2023-10-18T10:40:22.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1c154f80 con 0x7efc1c0ae770 2023-10-18T10:40:22.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.665+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1c0add10 con 0x7efc1c0ae770 2023-10-18T10:40:22.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.666+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1c155170 con 0x7efc1c0ae770 2023-10-18T10:40:22.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.666+0000 7efc03fff640 1 -- 172.21.15.125:0/2451400034 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efc1401d460 con 0x7efc1c0ae770 2023-10-18T10:40:22.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.667+0000 7efc03fff640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7efc0806af60 0x7efc0806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:22.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.667+0000 7efc03fff640 1 -- 172.21.15.125:0/2451400034 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efc1408a400 con 0x7efc1c0ae770 2023-10-18T10:40:22.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.667+0000 7efc1affd640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7efc0806af60 0x7efc0806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:22.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.667+0000 7efc1affd640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7efc0806af60 0x7efc0806d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efc0c036000 tx=0x7efc0c004a40 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:22.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.671+0000 7efc03fff640 1 -- 172.21.15.125:0/2451400034 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc140550c0 con 0x7efc1c0ae770 2023-10-18T10:40:22.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.834+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 --> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc1c0690e0 con 0x7efc0806af60 2023-10-18T10:40:22.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.843+0000 7efc03fff640 1 -- 172.21.15.125:0/2451400034 <== mgr.6055 v2:172.21.15.139:6832/2642591764 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278403 (secure 0 0 0) 0x7efc1c0690e0 con 0x7efc0806af60 2023-10-18T10:40:22.847 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:22.848 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:22.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7efc0806af60 msgr2=0x7efc0806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7efc0806af60 0x7efc0806d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efc0c036000 tx=0x7efc0c004a40 comp rx=0 tx=0).stop 2023-10-18T10:40:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc1c0ae770 msgr2=0x7efc1c150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc1c0ae770 0x7efc1c150d10 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7efc1c071330 tx=0x7efc140039c0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 shutdown_connections 2023-10-18T10:40:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7efc0806af60 0x7efc0806d420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc1c0ae770 0x7efc1c150d10 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc1c1565c0 0x7efc1c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 --2- 172.21.15.125:0/2451400034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc040039a0 0x7efc1c147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:22.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 >> 172.21.15.125:0/2451400034 conn(0x7efc1c06fae0 msgr2=0x7efc1c153150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:22.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 shutdown_connections 2023-10-18T10:40:22.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:22.846+0000 7efc218a2640 1 -- 172.21.15.125:0/2451400034 wait complete. 2023-10-18T10:40:22.858 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:40:21.796046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":249,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":302744,"kb_used_data":8844,"kb_used_omap":12,"kb_used_meta":293747,"kb_avail":754671976,"statfs":{"total":773094113280,"available":772784103424,"internally_reserved":0,"allocated":9056256,"data_stored":5064494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":300797496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001015"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919741+0000","last_change":"2023-10-18T10:40:19.912867+0000","last_active":"2023-10-18T10:40:19.919741+0000","last_peered":"2023-10-18T10:40:19.919741+0000","last_clean":"2023-10-18T10:40:19.919741+0000","last_became_active":"2023-10-18T10:40:19.912618+0000","last_became_peered":"2023-10-18T10:40:19.912618+0000","last_unstale":"2023-10-18T10:40:19.919741+0000","last_undegraded":"2023-10-18T10:40:19.919741+0000","last_fullsized":"2023-10-18T10:40:19.919741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:40:18.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917449+0000","last_change":"2023-10-18T10:40:19.912063+0000","last_active":"2023-10-18T10:40:19.917449+0000","last_peered":"2023-10-18T10:40:19.917449+0000","last_clean":"2023-10-18T10:40:19.917449+0000","last_became_active":"2023-10-18T10:40:19.911766+0000","last_became_peered":"2023-10-18T10:40:19.911766+0000","last_unstale":"2023-10-18T10:40:19.917449+0000","last_undegraded":"2023-10-18T10:40:19.917449+0000","last_fullsized":"2023-10-18T10:40:19.917449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:14.456342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.921101+0000","last_change":"2023-10-18T10:40:19.919930+0000","last_active":"2023-10-18T10:40:19.921101+0000","last_peered":"2023-10-18T10:40:19.921101+0000","last_clean":"2023-10-18T10:40:19.921101+0000","last_became_active":"2023-10-18T10:40:19.919664+0000","last_became_peered":"2023-10-18T10:40:19.919664+0000","last_unstale":"2023-10-18T10:40:19.921101+0000","last_undegraded":"2023-10-18T10:40:19.921101+0000","last_fullsized":"2023-10-18T10:40:19.921101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:04.430250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910585+0000","last_change":"2023-10-18T10:40:19.909713+0000","last_active":"2023-10-18T10:40:19.910585+0000","last_peered":"2023-10-18T10:40:19.910585+0000","last_clean":"2023-10-18T10:40:19.910585+0000","last_became_active":"2023-10-18T10:40:19.909464+0000","last_became_peered":"2023-10-18T10:40:19.909464+0000","last_unstale":"2023-10-18T10:40:19.910585+0000","last_undegraded":"2023-10-18T10:40:19.910585+0000","last_fullsized":"2023-10-18T10:40:19.910585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:55.604568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.904650+0000","last_change":"2023-10-18T10:40:19.902583+0000","last_active":"2023-10-18T10:40:19.904650+0000","last_peered":"2023-10-18T10:40:19.904650+0000","last_clean":"2023-10-18T10:40:19.904650+0000","last_became_active":"2023-10-18T10:40:19.902167+0000","last_became_peered":"2023-10-18T10:40:19.902167+0000","last_unstale":"2023-10-18T10:40:19.904650+0000","last_undegraded":"2023-10-18T10:40:19.904650+0000","last_fullsized":"2023-10-18T10:40:19.904650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:10.594385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.903183+0000","last_change":"2023-10-18T10:40:19.902535+0000","last_active":"2023-10-18T10:40:19.903183+0000","last_peered":"2023-10-18T10:40:19.903183+0000","last_clean":"2023-10-18T10:40:19.903183+0000","last_became_active":"2023-10-18T10:40:19.902248+0000","last_became_peered":"2023-10-18T10:40:19.902248+0000","last_unstale":"2023-10-18T10:40:19.903183+0000","last_undegraded":"2023-10-18T10:40:19.903183+0000","last_fullsized":"2023-10-18T10:40:19.903183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:56.313746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917829+0000","last_change":"2023-10-18T10:40:19.910004+0000","last_active":"2023-10-18T10:40:19.917829+0000","last_peered":"2023-10-18T10:40:19.917829+0000","last_clean":"2023-10-18T10:40:19.917829+0000","last_became_active":"2023-10-18T10:40:19.909737+0000","last_became_peered":"2023-10-18T10:40:19.909737+0000","last_unstale":"2023-10-18T10:40:19.917829+0000","last_undegraded":"2023-10-18T10:40:19.917829+0000","last_fullsized":"2023-10-18T10:40:19.917829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:36.377245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"83'4","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924752+0000","last_change":"2023-10-18T10:40:18.906118+0000","last_active":"2023-10-18T10:40:19.924752+0000","last_peered":"2023-10-18T10:40:19.924752+0000","last_clean":"2023-10-18T10:40:19.924752+0000","last_became_active":"2023-10-18T10:40:18.905861+0000","last_became_peered":"2023-10-18T10:40:18.905861+0000","last_unstale":"2023-10-18T10:40:19.924752+0000","last_undegraded":"2023-10-18T10:40:19.924752+0000","last_fullsized":"2023-10-18T10:40:19.924752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:12.674141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922912+0000","last_change":"2023-10-18T10:40:18.901139+0000","last_active":"2023-10-18T10:40:19.922912+0000","last_peered":"2023-10-18T10:40:19.922912+0000","last_clean":"2023-10-18T10:40:19.922912+0000","last_became_active":"2023-10-18T10:40:18.900880+0000","last_became_peered":"2023-10-18T10:40:18.900880+0000","last_unstale":"2023-10-18T10:40:19.922912+0000","last_undegraded":"2023-10-18T10:40:19.922912+0000","last_fullsized":"2023-10-18T10:40:19.922912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:57.088016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924051+0000","last_change":"2023-10-18T10:40:19.923120+0000","last_active":"2023-10-18T10:40:19.924051+0000","last_peered":"2023-10-18T10:40:19.924051+0000","last_clean":"2023-10-18T10:40:19.924051+0000","last_became_active":"2023-10-18T10:40:19.922847+0000","last_became_peered":"2023-10-18T10:40:19.922847+0000","last_unstale":"2023-10-18T10:40:19.924051+0000","last_undegraded":"2023-10-18T10:40:19.924051+0000","last_fullsized":"2023-10-18T10:40:19.924051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:04.526990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922729+0000","last_change":"2023-10-18T10:40:18.901404+0000","last_active":"2023-10-18T10:40:19.922729+0000","last_peered":"2023-10-18T10:40:19.922729+0000","last_clean":"2023-10-18T10:40:19.922729+0000","last_became_active":"2023-10-18T10:40:18.900949+0000","last_became_peered":"2023-10-18T10:40:18.900949+0000","last_unstale":"2023-10-18T10:40:19.922729+0000","last_undegraded":"2023-10-18T10:40:19.922729+0000","last_fullsized":"2023-10-18T10:40:19.922729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:54.300307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896846+0000","last_change":"2023-10-18T10:40:18.907834+0000","last_active":"2023-10-18T10:40:19.896846+0000","last_peered":"2023-10-18T10:40:19.896846+0000","last_clean":"2023-10-18T10:40:19.896846+0000","last_became_active":"2023-10-18T10:40:18.907243+0000","last_became_peered":"2023-10-18T10:40:18.907243+0000","last_unstale":"2023-10-18T10:40:19.896846+0000","last_undegraded":"2023-10-18T10:40:19.896846+0000","last_fullsized":"2023-10-18T10:40:19.896846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:00.888824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919903+0000","last_change":"2023-10-18T10:40:19.919367+0000","last_active":"2023-10-18T10:40:19.919903+0000","last_peered":"2023-10-18T10:40:19.919903+0000","last_clean":"2023-10-18T10:40:19.919903+0000","last_became_active":"2023-10-18T10:40:19.919114+0000","last_became_peered":"2023-10-18T10:40:19.919114+0000","last_unstale":"2023-10-18T10:40:19.919903+0000","last_undegraded":"2023-10-18T10:40:19.919903+0000","last_fullsized":"2023-10-18T10:40:19.919903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:44.547473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.916409+0000","last_change":"2023-10-18T10:40:19.907755+0000","last_active":"2023-10-18T10:40:19.916409+0000","last_peered":"2023-10-18T10:40:19.916409+0000","last_clean":"2023-10-18T10:40:19.916409+0000","last_became_active":"2023-10-18T10:40:19.907464+0000","last_became_peered":"2023-10-18T10:40:19.907464+0000","last_unstale":"2023-10-18T10:40:19.916409+0000","last_undegraded":"2023-10-18T10:40:19.916409+0000","last_fullsized":"2023-10-18T10:40:19.916409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:43.487087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.914067+0000","last_change":"2023-10-18T10:40:19.911151+0000","last_active":"2023-10-18T10:40:19.914067+0000","last_peered":"2023-10-18T10:40:19.914067+0000","last_clean":"2023-10-18T10:40:19.914067+0000","last_became_active":"2023-10-18T10:40:19.910476+0000","last_became_peered":"2023-10-18T10:40:19.910476+0000","last_unstale":"2023-10-18T10:40:19.914067+0000","last_undegraded":"2023-10-18T10:40:19.914067+0000","last_fullsized":"2023-10-18T10:40:19.914067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:43.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.928954+0000","last_change":"2023-10-18T10:40:18.905020+0000","last_active":"2023-10-18T10:40:19.928954+0000","last_peered":"2023-10-18T10:40:19.928954+0000","last_clean":"2023-10-18T10:40:19.928954+0000","last_became_active":"2023-10-18T10:40:18.904379+0000","last_became_peered":"2023-10-18T10:40:18.904379+0000","last_unstale":"2023-10-18T10:40:19.928954+0000","last_undegraded":"2023-10-18T10:40:19.928954+0000","last_fullsized":"2023-10-18T10:40:19.928954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:28.436049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917478+0000","last_change":"2023-10-18T10:40:19.909009+0000","last_active":"2023-10-18T10:40:19.917478+0000","last_peered":"2023-10-18T10:40:19.917478+0000","last_clean":"2023-10-18T10:40:19.917478+0000","last_became_active":"2023-10-18T10:40:19.908767+0000","last_became_peered":"2023-10-18T10:40:19.908767+0000","last_unstale":"2023-10-18T10:40:19.917478+0000","last_undegraded":"2023-10-18T10:40:19.917478+0000","last_fullsized":"2023-10-18T10:40:19.917478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:00.544158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897209+0000","last_change":"2023-10-18T10:40:18.902590+0000","last_active":"2023-10-18T10:40:19.897209+0000","last_peered":"2023-10-18T10:40:19.897209+0000","last_clean":"2023-10-18T10:40:19.897209+0000","last_became_active":"2023-10-18T10:40:18.902335+0000","last_became_peered":"2023-10-18T10:40:18.902335+0000","last_unstale":"2023-10-18T10:40:19.897209+0000","last_undegraded":"2023-10-18T10:40:19.897209+0000","last_fullsized":"2023-10-18T10:40:19.897209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:58:24.515902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.902230+0000","last_change":"2023-10-18T10:40:18.905086+0000","last_active":"2023-10-18T10:40:19.902230+0000","last_peered":"2023-10-18T10:40:19.902230+0000","last_clean":"2023-10-18T10:40:19.902230+0000","last_became_active":"2023-10-18T10:40:18.904715+0000","last_became_peered":"2023-10-18T10:40:18.904715+0000","last_unstale":"2023-10-18T10:40:19.902230+0000","last_undegraded":"2023-10-18T10:40:19.902230+0000","last_fullsized":"2023-10-18T10:40:19.902230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:44.769336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897010+0000","last_change":"2023-10-18T10:40:18.901223+0000","last_active":"2023-10-18T10:40:19.897010+0000","last_peered":"2023-10-18T10:40:19.897010+0000","last_clean":"2023-10-18T10:40:19.897010+0000","last_became_active":"2023-10-18T10:40:18.900638+0000","last_became_peered":"2023-10-18T10:40:18.900638+0000","last_unstale":"2023-10-18T10:40:19.897010+0000","last_undegraded":"2023-10-18T10:40:19.897010+0000","last_fullsized":"2023-10-18T10:40:19.897010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:04.547290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926391+0000","last_change":"2023-10-18T10:40:19.925927+0000","last_active":"2023-10-18T10:40:19.926391+0000","last_peered":"2023-10-18T10:40:19.926391+0000","last_clean":"2023-10-18T10:40:19.926391+0000","last_became_active":"2023-10-18T10:40:19.925668+0000","last_became_peered":"2023-10-18T10:40:19.925668+0000","last_unstale":"2023-10-18T10:40:19.926391+0000","last_undegraded":"2023-10-18T10:40:19.926391+0000","last_fullsized":"2023-10-18T10:40:19.926391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:09.139441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897265+0000","last_change":"2023-10-18T10:40:18.904336+0000","last_active":"2023-10-18T10:40:19.897265+0000","last_peered":"2023-10-18T10:40:19.897265+0000","last_clean":"2023-10-18T10:40:19.897265+0000","last_became_active":"2023-10-18T10:40:18.904085+0000","last_became_peered":"2023-10-18T10:40:18.904085+0000","last_unstale":"2023-10-18T10:40:19.897265+0000","last_undegraded":"2023-10-18T10:40:19.897265+0000","last_fullsized":"2023-10-18T10:40:19.897265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:32.124110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.900343+0000","last_change":"2023-10-18T10:40:18.902461+0000","last_active":"2023-10-18T10:40:19.900343+0000","last_peered":"2023-10-18T10:40:19.900343+0000","last_clean":"2023-10-18T10:40:19.900343+0000","last_became_active":"2023-10-18T10:40:18.902198+0000","last_became_peered":"2023-10-18T10:40:18.902198+0000","last_unstale":"2023-10-18T10:40:19.900343+0000","last_undegraded":"2023-10-18T10:40:19.900343+0000","last_fullsized":"2023-10-18T10:40:19.900343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:18.205683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896765+0000","last_change":"2023-10-18T10:40:18.901946+0000","last_active":"2023-10-18T10:40:19.896765+0000","last_peered":"2023-10-18T10:40:19.896765+0000","last_clean":"2023-10-18T10:40:19.896765+0000","last_became_active":"2023-10-18T10:40:18.901694+0000","last_became_peered":"2023-10-18T10:40:18.901694+0000","last_unstale":"2023-10-18T10:40:19.896765+0000","last_undegraded":"2023-10-18T10:40:19.896765+0000","last_fullsized":"2023-10-18T10:40:19.896765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:08.817426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'161","reported_seq":379,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-18T10:40:14.604954+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:14.604954+0000","last_peered":"2023-10-18T10:40:14.604954+0000","last_clean":"2023-10-18T10:40:14.604954+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:14.604954+0000","last_undegraded":"2023-10-18T10:40:14.604954+0000","last_fullsized":"2023-10-18T10:40:14.604954+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":22,"acting":22,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59084,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312756,"statfs":{"total":96636764160,"available":96576262144,"internally_reserved":0,"allocated":1564672,"data_stored":1070586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.876},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73199999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.85299999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.47599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.47499999999999998,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":1.016},"last":0.45000000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.436,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.495},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.52100000000000002,"5min":0.35499999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.50700000000000001,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.47899999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.875},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.48199999999999998,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.48399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.433}]},{"osd":4,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":0.81799999999999995,"15min":0.84999999999999998},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47399999999999998,"15min":0.47099999999999997},"min":{"1min":0.35099999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.32300000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.87},"last":0.54600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.47699999999999998,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.016},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.46200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.629},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.76200000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.48199999999999998,"5min":0.377,"15min":0.28599999999999998},"max":{"1min":0.746,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.51600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.49399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.57099999999999995,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.81000000000000005,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.72299999999999998,"5min":0.80100000000000005,"15min":1.262},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":1.0109999999999999,"15min":1.028},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72999999999999998,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":0.96099999999999997,"15min":1.224},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.73699999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.871,"15min":1.159},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.42299999999999999,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.33000000000000002},"max":{"1min":0.751,"5min":0.877,"15min":0.877},"last":0.374}]},{"osd":1,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70299999999999996,"5min":0.81000000000000005,"15min":0.81899999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.501,"5min":0.36699999999999999,"15min":0.29499999999999998},"max":{"1min":0.77700000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.48299999999999998,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.86199999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.57499999999999996},"min":{"1min":0.505,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.76800000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55300000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.34699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.505,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.746,"5min":0.77600000000000002,"15min":0.79600000000000004},"last":0.57099999999999995}]},{"osd":6,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.50600000000000001},"min":{"1min":0.32700000000000001,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.78800000000000003,"5min":1.446,"15min":1.446},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.90400000000000003},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.36899999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.74199999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.41799999999999998,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.81100000000000005,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":500037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54336,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317504,"statfs":{"total":96636764160,"available":96581124096,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.313,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.52300000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.32700000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40300000000000002},"min":{"1min":0.34100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60999999999999999,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.35599999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.377,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.373}]},{"osd":4,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.88200000000000001,"5min":0.88800000000000001,"15min":1.458},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.52600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.95699999999999996,"15min":0.997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.93700000000000006,"15min":1.028},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.63300000000000001},"min":{"1min":0.51000000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.88900000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.52500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84599999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"last":0.51000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":502413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.371},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.434}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.505}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":1070586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.28599999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.308},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:22.865 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:23.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.000+0000 7f4970155640 1 -- 172.21.15.125:0/723898173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 msgr2=0x7f49680d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.000+0000 7f4970155640 1 --2- 172.21.15.125:0/723898173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 0x7f49680d1910 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4964009830 tx=0x7f496402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.000+0000 7f4970155640 1 -- 172.21.15.125:0/723898173 shutdown_connections 2023-10-18T10:40:23.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.000+0000 7f4970155640 1 --2- 172.21.15.125:0/723898173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49680ceb40 0x7f49680b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.000+0000 7f4970155640 1 --2- 172.21.15.125:0/723898173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 0x7f49680d1910 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.000+0000 7f4970155640 1 -- 172.21.15.125:0/723898173 >> 172.21.15.125:0/723898173 conn(0x7f49680d0d60 msgr2=0x7f49680d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:23.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.000+0000 7f4970155640 1 -- 172.21.15.125:0/723898173 shutdown_connections 2023-10-18T10:40:23.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.000+0000 7f4970155640 1 -- 172.21.15.125:0/723898173 wait complete. 2023-10-18T10:40:23.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.001+0000 7f4970155640 1 Processor -- start 2023-10-18T10:40:23.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.001+0000 7f4970155640 1 -- start start 2023-10-18T10:40:23.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f4970155640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49680ceb40 0x7f49680ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f4970155640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49680cf340 0x7f49680b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f4970155640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 0x7f49680ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f4970155640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49680c85b0 con 0x7f496815b2a0 2023-10-18T10:40:23.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f4970155640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49680b6c30 con 0x7f49680ceb40 2023-10-18T10:40:23.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f4970155640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49680b6950 con 0x7f49680cf340 2023-10-18T10:40:23.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f496e6f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 0x7f49680ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f496e6f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 0x7f49680ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40690/0 (socket says 172.21.15.125:40690) 2023-10-18T10:40:23.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f496e6f2640 1 -- 172.21.15.125:0/3677413213 learned_addr learned my addr 172.21.15.125:0/3677413213 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:23.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f496d6f0640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49680cf340 0x7f49680b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f496e6f2640 1 -- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49680cf340 msgr2=0x7f49680b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.002+0000 7f496def1640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49680ceb40 0x7f49680ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f496e6f2640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49680cf340 0x7f49680b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f496e6f2640 1 -- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49680ceb40 msgr2=0x7f49680ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f496e6f2640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49680ceb40 0x7f49680ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f496e6f2640 1 -- 172.21.15.125:0/3677413213 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4964009490 con 0x7f496815b2a0 2023-10-18T10:40:23.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f496d6f0640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49680cf340 0x7f49680b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:23.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f496e6f2640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 0x7f49680ae280 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f496000cd10 tx=0x7f496000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:23.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f495e7fc640 1 -- 172.21.15.125:0/3677413213 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000f450 con 0x7f496815b2a0 2023-10-18T10:40:23.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f495e7fc640 1 -- 172.21.15.125:0/3677413213 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f496000f6a0 con 0x7f496815b2a0 2023-10-18T10:40:23.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49680c7f60 con 0x7f496815b2a0 2023-10-18T10:40:23.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f495e7fc640 1 -- 172.21.15.125:0/3677413213 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000fab0 con 0x7f496815b2a0 2023-10-18T10:40:23.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.003+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49680ca9f0 con 0x7f496815b2a0 2023-10-18T10:40:23.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.004+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49680c8760 con 0x7f496815b2a0 2023-10-18T10:40:23.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.004+0000 7f495e7fc640 1 -- 172.21.15.125:0/3677413213 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f496000fc10 con 0x7f496815b2a0 2023-10-18T10:40:23.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.005+0000 7f495e7fc640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f494c06af60 0x7f494c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.005+0000 7f495e7fc640 1 -- 172.21.15.125:0/3677413213 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f496008a5a0 con 0x7f496815b2a0 2023-10-18T10:40:23.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.005+0000 7f496def1640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f494c06af60 0x7f494c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.005+0000 7f496def1640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f494c06af60 0x7f494c06d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49580046b0 tx=0x7f4958009040 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:23.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.009+0000 7f495e7fc640 1 -- 172.21.15.125:0/3677413213 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49680c8760 con 0x7f496815b2a0 2023-10-18T10:40:23.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.173+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 --> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49680c8760 con 0x7f494c06af60 2023-10-18T10:40:23.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.181+0000 7f495e7fc640 1 -- 172.21.15.125:0/3677413213 <== mgr.6055 v2:172.21.15.139:6832/2642591764 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278403 (secure 0 0 0) 0x7f49680c8760 con 0x7f494c06af60 2023-10-18T10:40:23.186 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:23.188 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:23.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f494c06af60 msgr2=0x7f494c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f494c06af60 0x7f494c06d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49580046b0 tx=0x7f4958009040 comp rx=0 tx=0).stop 2023-10-18T10:40:23.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 msgr2=0x7f49680ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 0x7f49680ae280 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f496000cd10 tx=0x7f496000a620 comp rx=0 tx=0).stop 2023-10-18T10:40:23.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 shutdown_connections 2023-10-18T10:40:23.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f494c06af60 0x7f494c06d420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f496815b2a0 0x7f49680ae280 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49680cf340 0x7f49680b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 --2- 172.21.15.125:0/3677413213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49680ceb40 0x7f49680ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 >> 172.21.15.125:0/3677413213 conn(0x7f49680d0d60 msgr2=0x7f496815a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:23.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 shutdown_connections 2023-10-18T10:40:23.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.184+0000 7f4970155640 1 -- 172.21.15.125:0/3677413213 wait complete. 2023-10-18T10:40:23.196 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:40:21.796046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":249,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":302744,"kb_used_data":8844,"kb_used_omap":12,"kb_used_meta":293747,"kb_avail":754671976,"statfs":{"total":773094113280,"available":772784103424,"internally_reserved":0,"allocated":9056256,"data_stored":5064494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":300797496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001015"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919741+0000","last_change":"2023-10-18T10:40:19.912867+0000","last_active":"2023-10-18T10:40:19.919741+0000","last_peered":"2023-10-18T10:40:19.919741+0000","last_clean":"2023-10-18T10:40:19.919741+0000","last_became_active":"2023-10-18T10:40:19.912618+0000","last_became_peered":"2023-10-18T10:40:19.912618+0000","last_unstale":"2023-10-18T10:40:19.919741+0000","last_undegraded":"2023-10-18T10:40:19.919741+0000","last_fullsized":"2023-10-18T10:40:19.919741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:40:18.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917449+0000","last_change":"2023-10-18T10:40:19.912063+0000","last_active":"2023-10-18T10:40:19.917449+0000","last_peered":"2023-10-18T10:40:19.917449+0000","last_clean":"2023-10-18T10:40:19.917449+0000","last_became_active":"2023-10-18T10:40:19.911766+0000","last_became_peered":"2023-10-18T10:40:19.911766+0000","last_unstale":"2023-10-18T10:40:19.917449+0000","last_undegraded":"2023-10-18T10:40:19.917449+0000","last_fullsized":"2023-10-18T10:40:19.917449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:14.456342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.921101+0000","last_change":"2023-10-18T10:40:19.919930+0000","last_active":"2023-10-18T10:40:19.921101+0000","last_peered":"2023-10-18T10:40:19.921101+0000","last_clean":"2023-10-18T10:40:19.921101+0000","last_became_active":"2023-10-18T10:40:19.919664+0000","last_became_peered":"2023-10-18T10:40:19.919664+0000","last_unstale":"2023-10-18T10:40:19.921101+0000","last_undegraded":"2023-10-18T10:40:19.921101+0000","last_fullsized":"2023-10-18T10:40:19.921101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:04.430250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910585+0000","last_change":"2023-10-18T10:40:19.909713+0000","last_active":"2023-10-18T10:40:19.910585+0000","last_peered":"2023-10-18T10:40:19.910585+0000","last_clean":"2023-10-18T10:40:19.910585+0000","last_became_active":"2023-10-18T10:40:19.909464+0000","last_became_peered":"2023-10-18T10:40:19.909464+0000","last_unstale":"2023-10-18T10:40:19.910585+0000","last_undegraded":"2023-10-18T10:40:19.910585+0000","last_fullsized":"2023-10-18T10:40:19.910585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:55.604568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.904650+0000","last_change":"2023-10-18T10:40:19.902583+0000","last_active":"2023-10-18T10:40:19.904650+0000","last_peered":"2023-10-18T10:40:19.904650+0000","last_clean":"2023-10-18T10:40:19.904650+0000","last_became_active":"2023-10-18T10:40:19.902167+0000","last_became_peered":"2023-10-18T10:40:19.902167+0000","last_unstale":"2023-10-18T10:40:19.904650+0000","last_undegraded":"2023-10-18T10:40:19.904650+0000","last_fullsized":"2023-10-18T10:40:19.904650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:10.594385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.903183+0000","last_change":"2023-10-18T10:40:19.902535+0000","last_active":"2023-10-18T10:40:19.903183+0000","last_peered":"2023-10-18T10:40:19.903183+0000","last_clean":"2023-10-18T10:40:19.903183+0000","last_became_active":"2023-10-18T10:40:19.902248+0000","last_became_peered":"2023-10-18T10:40:19.902248+0000","last_unstale":"2023-10-18T10:40:19.903183+0000","last_undegraded":"2023-10-18T10:40:19.903183+0000","last_fullsized":"2023-10-18T10:40:19.903183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:56.313746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917829+0000","last_change":"2023-10-18T10:40:19.910004+0000","last_active":"2023-10-18T10:40:19.917829+0000","last_peered":"2023-10-18T10:40:19.917829+0000","last_clean":"2023-10-18T10:40:19.917829+0000","last_became_active":"2023-10-18T10:40:19.909737+0000","last_became_peered":"2023-10-18T10:40:19.909737+0000","last_unstale":"2023-10-18T10:40:19.917829+0000","last_undegraded":"2023-10-18T10:40:19.917829+0000","last_fullsized":"2023-10-18T10:40:19.917829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:36.377245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"83'4","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924752+0000","last_change":"2023-10-18T10:40:18.906118+0000","last_active":"2023-10-18T10:40:19.924752+0000","last_peered":"2023-10-18T10:40:19.924752+0000","last_clean":"2023-10-18T10:40:19.924752+0000","last_became_active":"2023-10-18T10:40:18.905861+0000","last_became_peered":"2023-10-18T10:40:18.905861+0000","last_unstale":"2023-10-18T10:40:19.924752+0000","last_undegraded":"2023-10-18T10:40:19.924752+0000","last_fullsized":"2023-10-18T10:40:19.924752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:12.674141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922912+0000","last_change":"2023-10-18T10:40:18.901139+0000","last_active":"2023-10-18T10:40:19.922912+0000","last_peered":"2023-10-18T10:40:19.922912+0000","last_clean":"2023-10-18T10:40:19.922912+0000","last_became_active":"2023-10-18T10:40:18.900880+0000","last_became_peered":"2023-10-18T10:40:18.900880+0000","last_unstale":"2023-10-18T10:40:19.922912+0000","last_undegraded":"2023-10-18T10:40:19.922912+0000","last_fullsized":"2023-10-18T10:40:19.922912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:57.088016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924051+0000","last_change":"2023-10-18T10:40:19.923120+0000","last_active":"2023-10-18T10:40:19.924051+0000","last_peered":"2023-10-18T10:40:19.924051+0000","last_clean":"2023-10-18T10:40:19.924051+0000","last_became_active":"2023-10-18T10:40:19.922847+0000","last_became_peered":"2023-10-18T10:40:19.922847+0000","last_unstale":"2023-10-18T10:40:19.924051+0000","last_undegraded":"2023-10-18T10:40:19.924051+0000","last_fullsized":"2023-10-18T10:40:19.924051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:04.526990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922729+0000","last_change":"2023-10-18T10:40:18.901404+0000","last_active":"2023-10-18T10:40:19.922729+0000","last_peered":"2023-10-18T10:40:19.922729+0000","last_clean":"2023-10-18T10:40:19.922729+0000","last_became_active":"2023-10-18T10:40:18.900949+0000","last_became_peered":"2023-10-18T10:40:18.900949+0000","last_unstale":"2023-10-18T10:40:19.922729+0000","last_undegraded":"2023-10-18T10:40:19.922729+0000","last_fullsized":"2023-10-18T10:40:19.922729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:54.300307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896846+0000","last_change":"2023-10-18T10:40:18.907834+0000","last_active":"2023-10-18T10:40:19.896846+0000","last_peered":"2023-10-18T10:40:19.896846+0000","last_clean":"2023-10-18T10:40:19.896846+0000","last_became_active":"2023-10-18T10:40:18.907243+0000","last_became_peered":"2023-10-18T10:40:18.907243+0000","last_unstale":"2023-10-18T10:40:19.896846+0000","last_undegraded":"2023-10-18T10:40:19.896846+0000","last_fullsized":"2023-10-18T10:40:19.896846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:00.888824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919903+0000","last_change":"2023-10-18T10:40:19.919367+0000","last_active":"2023-10-18T10:40:19.919903+0000","last_peered":"2023-10-18T10:40:19.919903+0000","last_clean":"2023-10-18T10:40:19.919903+0000","last_became_active":"2023-10-18T10:40:19.919114+0000","last_became_peered":"2023-10-18T10:40:19.919114+0000","last_unstale":"2023-10-18T10:40:19.919903+0000","last_undegraded":"2023-10-18T10:40:19.919903+0000","last_fullsized":"2023-10-18T10:40:19.919903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:44.547473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.916409+0000","last_change":"2023-10-18T10:40:19.907755+0000","last_active":"2023-10-18T10:40:19.916409+0000","last_peered":"2023-10-18T10:40:19.916409+0000","last_clean":"2023-10-18T10:40:19.916409+0000","last_became_active":"2023-10-18T10:40:19.907464+0000","last_became_peered":"2023-10-18T10:40:19.907464+0000","last_unstale":"2023-10-18T10:40:19.916409+0000","last_undegraded":"2023-10-18T10:40:19.916409+0000","last_fullsized":"2023-10-18T10:40:19.916409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:43.487087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.914067+0000","last_change":"2023-10-18T10:40:19.911151+0000","last_active":"2023-10-18T10:40:19.914067+0000","last_peered":"2023-10-18T10:40:19.914067+0000","last_clean":"2023-10-18T10:40:19.914067+0000","last_became_active":"2023-10-18T10:40:19.910476+0000","last_became_peered":"2023-10-18T10:40:19.910476+0000","last_unstale":"2023-10-18T10:40:19.914067+0000","last_undegraded":"2023-10-18T10:40:19.914067+0000","last_fullsized":"2023-10-18T10:40:19.914067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:43.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.928954+0000","last_change":"2023-10-18T10:40:18.905020+0000","last_active":"2023-10-18T10:40:19.928954+0000","last_peered":"2023-10-18T10:40:19.928954+0000","last_clean":"2023-10-18T10:40:19.928954+0000","last_became_active":"2023-10-18T10:40:18.904379+0000","last_became_peered":"2023-10-18T10:40:18.904379+0000","last_unstale":"2023-10-18T10:40:19.928954+0000","last_undegraded":"2023-10-18T10:40:19.928954+0000","last_fullsized":"2023-10-18T10:40:19.928954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:28.436049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917478+0000","last_change":"2023-10-18T10:40:19.909009+0000","last_active":"2023-10-18T10:40:19.917478+0000","last_peered":"2023-10-18T10:40:19.917478+0000","last_clean":"2023-10-18T10:40:19.917478+0000","last_became_active":"2023-10-18T10:40:19.908767+0000","last_became_peered":"2023-10-18T10:40:19.908767+0000","last_unstale":"2023-10-18T10:40:19.917478+0000","last_undegraded":"2023-10-18T10:40:19.917478+0000","last_fullsized":"2023-10-18T10:40:19.917478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:00.544158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897209+0000","last_change":"2023-10-18T10:40:18.902590+0000","last_active":"2023-10-18T10:40:19.897209+0000","last_peered":"2023-10-18T10:40:19.897209+0000","last_clean":"2023-10-18T10:40:19.897209+0000","last_became_active":"2023-10-18T10:40:18.902335+0000","last_became_peered":"2023-10-18T10:40:18.902335+0000","last_unstale":"2023-10-18T10:40:19.897209+0000","last_undegraded":"2023-10-18T10:40:19.897209+0000","last_fullsized":"2023-10-18T10:40:19.897209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:58:24.515902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.902230+0000","last_change":"2023-10-18T10:40:18.905086+0000","last_active":"2023-10-18T10:40:19.902230+0000","last_peered":"2023-10-18T10:40:19.902230+0000","last_clean":"2023-10-18T10:40:19.902230+0000","last_became_active":"2023-10-18T10:40:18.904715+0000","last_became_peered":"2023-10-18T10:40:18.904715+0000","last_unstale":"2023-10-18T10:40:19.902230+0000","last_undegraded":"2023-10-18T10:40:19.902230+0000","last_fullsized":"2023-10-18T10:40:19.902230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:44.769336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897010+0000","last_change":"2023-10-18T10:40:18.901223+0000","last_active":"2023-10-18T10:40:19.897010+0000","last_peered":"2023-10-18T10:40:19.897010+0000","last_clean":"2023-10-18T10:40:19.897010+0000","last_became_active":"2023-10-18T10:40:18.900638+0000","last_became_peered":"2023-10-18T10:40:18.900638+0000","last_unstale":"2023-10-18T10:40:19.897010+0000","last_undegraded":"2023-10-18T10:40:19.897010+0000","last_fullsized":"2023-10-18T10:40:19.897010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:04.547290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926391+0000","last_change":"2023-10-18T10:40:19.925927+0000","last_active":"2023-10-18T10:40:19.926391+0000","last_peered":"2023-10-18T10:40:19.926391+0000","last_clean":"2023-10-18T10:40:19.926391+0000","last_became_active":"2023-10-18T10:40:19.925668+0000","last_became_peered":"2023-10-18T10:40:19.925668+0000","last_unstale":"2023-10-18T10:40:19.926391+0000","last_undegraded":"2023-10-18T10:40:19.926391+0000","last_fullsized":"2023-10-18T10:40:19.926391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:09.139441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897265+0000","last_change":"2023-10-18T10:40:18.904336+0000","last_active":"2023-10-18T10:40:19.897265+0000","last_peered":"2023-10-18T10:40:19.897265+0000","last_clean":"2023-10-18T10:40:19.897265+0000","last_became_active":"2023-10-18T10:40:18.904085+0000","last_became_peered":"2023-10-18T10:40:18.904085+0000","last_unstale":"2023-10-18T10:40:19.897265+0000","last_undegraded":"2023-10-18T10:40:19.897265+0000","last_fullsized":"2023-10-18T10:40:19.897265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:32.124110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.900343+0000","last_change":"2023-10-18T10:40:18.902461+0000","last_active":"2023-10-18T10:40:19.900343+0000","last_peered":"2023-10-18T10:40:19.900343+0000","last_clean":"2023-10-18T10:40:19.900343+0000","last_became_active":"2023-10-18T10:40:18.902198+0000","last_became_peered":"2023-10-18T10:40:18.902198+0000","last_unstale":"2023-10-18T10:40:19.900343+0000","last_undegraded":"2023-10-18T10:40:19.900343+0000","last_fullsized":"2023-10-18T10:40:19.900343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:18.205683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896765+0000","last_change":"2023-10-18T10:40:18.901946+0000","last_active":"2023-10-18T10:40:19.896765+0000","last_peered":"2023-10-18T10:40:19.896765+0000","last_clean":"2023-10-18T10:40:19.896765+0000","last_became_active":"2023-10-18T10:40:18.901694+0000","last_became_peered":"2023-10-18T10:40:18.901694+0000","last_unstale":"2023-10-18T10:40:19.896765+0000","last_undegraded":"2023-10-18T10:40:19.896765+0000","last_fullsized":"2023-10-18T10:40:19.896765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:08.817426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'161","reported_seq":379,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-18T10:40:14.604954+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:14.604954+0000","last_peered":"2023-10-18T10:40:14.604954+0000","last_clean":"2023-10-18T10:40:14.604954+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:14.604954+0000","last_undegraded":"2023-10-18T10:40:14.604954+0000","last_fullsized":"2023-10-18T10:40:14.604954+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":22,"acting":22,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59084,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312756,"statfs":{"total":96636764160,"available":96576262144,"internally_reserved":0,"allocated":1564672,"data_stored":1070586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.876},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73199999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.85299999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.47599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.47499999999999998,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":1.016},"last":0.45000000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.436,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.495},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.52100000000000002,"5min":0.35499999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.50700000000000001,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.47899999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.875},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.48199999999999998,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.48399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.433}]},{"osd":4,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":0.81799999999999995,"15min":0.84999999999999998},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47399999999999998,"15min":0.47099999999999997},"min":{"1min":0.35099999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.32300000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.87},"last":0.54600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.47699999999999998,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.016},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.46200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.629},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.76200000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.48199999999999998,"5min":0.377,"15min":0.28599999999999998},"max":{"1min":0.746,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.51600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.49399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.57099999999999995,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.81000000000000005,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.72299999999999998,"5min":0.80100000000000005,"15min":1.262},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":1.0109999999999999,"15min":1.028},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72999999999999998,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":0.96099999999999997,"15min":1.224},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.73699999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.871,"15min":1.159},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.42299999999999999,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.33000000000000002},"max":{"1min":0.751,"5min":0.877,"15min":0.877},"last":0.374}]},{"osd":1,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70299999999999996,"5min":0.81000000000000005,"15min":0.81899999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.501,"5min":0.36699999999999999,"15min":0.29499999999999998},"max":{"1min":0.77700000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.48299999999999998,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.86199999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.57499999999999996},"min":{"1min":0.505,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.76800000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55300000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.34699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.505,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.746,"5min":0.77600000000000002,"15min":0.79600000000000004},"last":0.57099999999999995}]},{"osd":6,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.50600000000000001},"min":{"1min":0.32700000000000001,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.78800000000000003,"5min":1.446,"15min":1.446},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.90400000000000003},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.36899999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.74199999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.41799999999999998,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.81100000000000005,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":500037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54336,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317504,"statfs":{"total":96636764160,"available":96581124096,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.313,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.52300000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.32700000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40300000000000002},"min":{"1min":0.34100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60999999999999999,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.35599999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.377,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.373}]},{"osd":4,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.88200000000000001,"5min":0.88800000000000001,"15min":1.458},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.52600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.95699999999999996,"15min":0.997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.93700000000000006,"15min":1.028},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.63300000000000001},"min":{"1min":0.51000000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.88900000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.52500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84599999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"last":0.51000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":502413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.371},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.434}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.505}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":1070586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.28599999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.308},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:23.202 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:23.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.337+0000 7fcad1256640 1 -- 172.21.15.125:0/2748810567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcab8003970 msgr2=0x7fcab8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.337+0000 7fcad1256640 1 --2- 172.21.15.125:0/2748810567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcab8003970 0x7fcab8003d90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcabc001680 tx=0x7fcab80057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.337+0000 7fcad1256640 1 -- 172.21.15.125:0/2748810567 shutdown_connections 2023-10-18T10:40:23.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.337+0000 7fcad1256640 1 --2- 172.21.15.125:0/2748810567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcab8003970 0x7fcab8003d90 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.337+0000 7fcad1256640 1 -- 172.21.15.125:0/2748810567 >> 172.21.15.125:0/2748810567 conn(0x7fcacc069930 msgr2=0x7fcacc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:23.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.337+0000 7fcad1256640 1 -- 172.21.15.125:0/2748810567 shutdown_connections 2023-10-18T10:40:23.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.337+0000 7fcad1256640 1 -- 172.21.15.125:0/2748810567 wait complete. 2023-10-18T10:40:23.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.338+0000 7fcad1256640 1 Processor -- start 2023-10-18T10:40:23.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.338+0000 7fcad1256640 1 -- start start 2023-10-18T10:40:23.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcad1256640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcab8003970 0x7fcacc1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcad1256640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcacc13fa90 0x7fcacc13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcad1256640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcacc140450 0x7fcacc153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcad1256640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc1f0520 con 0x7fcacc13fa90 2023-10-18T10:40:23.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcad1256640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc140900 con 0x7fcab8003970 2023-10-18T10:40:23.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcad1256640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc153260 con 0x7fcacc140450 2023-10-18T10:40:23.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcac3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcacc13fa90 0x7fcacc13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcac3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcacc13fa90 0x7fcacc13ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40698/0 (socket says 172.21.15.125:40698) 2023-10-18T10:40:23.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcac3fff640 1 -- 172.21.15.125:0/3765322812 learned_addr learned my addr 172.21.15.125:0/3765322812 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:23.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcacb59e640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcacc140450 0x7fcacc153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcacad9d640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcab8003970 0x7fcacc1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcac3fff640 1 -- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcacc140450 msgr2=0x7fcacc153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.339+0000 7fcac3fff640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcacc140450 0x7fcacc153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcac3fff640 1 -- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcab8003970 msgr2=0x7fcacc1515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcac3fff640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcab8003970 0x7fcacc1515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcac3fff640 1 -- 172.21.15.125:0/3765322812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc1f0240 con 0x7fcacc13fa90 2023-10-18T10:40:23.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcacad9d640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcab8003970 0x7fcacc1515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:23.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcacb59e640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcacc140450 0x7fcacc153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:23.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcac3fff640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcacc13fa90 0x7fcacc13ff10 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcab400f9d0 tx=0x7fcab400fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:23.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcac37fe640 1 -- 172.21.15.125:0/3765322812 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab400bd10 con 0x7fcacc13fa90 2023-10-18T10:40:23.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcac37fe640 1 -- 172.21.15.125:0/3765322812 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcab4011440 con 0x7fcacc13fa90 2023-10-18T10:40:23.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc153500 con 0x7fcacc13fa90 2023-10-18T10:40:23.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcac37fe640 1 -- 172.21.15.125:0/3765322812 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011830 con 0x7fcacc13fa90 2023-10-18T10:40:23.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.340+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc153ac0 con 0x7fcacc13fa90 2023-10-18T10:40:23.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.341+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc1575f0 con 0x7fcacc13fa90 2023-10-18T10:40:23.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.341+0000 7fcac37fe640 1 -- 172.21.15.125:0/3765322812 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcab4011990 con 0x7fcacc13fa90 2023-10-18T10:40:23.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.342+0000 7fcac37fe640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fcaac06af60 0x7fcaac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.342+0000 7fcac37fe640 1 -- 172.21.15.125:0/3765322812 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcab408a9e0 con 0x7fcacc13fa90 2023-10-18T10:40:23.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.342+0000 7fcacad9d640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fcaac06af60 0x7fcaac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.343+0000 7fcacad9d640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fcaac06af60 0x7fcaac06d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcabc034000 tx=0x7fcabc02e040 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:23.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.346+0000 7fcac37fe640 1 -- 172.21.15.125:0/3765322812 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcab40557e0 con 0x7fcacc13fa90 2023-10-18T10:40:23.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.510+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 --> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcacc0cbd60 con 0x7fcaac06af60 2023-10-18T10:40:23.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.518+0000 7fcac37fe640 1 -- 172.21.15.125:0/3765322812 <== mgr.6055 v2:172.21.15.139:6832/2642591764 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278403 (secure 0 0 0) 0x7fcabc0035f0 con 0x7fcaac06af60 2023-10-18T10:40:23.522 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:23.523 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:23.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fcaac06af60 msgr2=0x7fcaac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fcaac06af60 0x7fcaac06d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcabc034000 tx=0x7fcabc02e040 comp rx=0 tx=0).stop 2023-10-18T10:40:23.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcacc13fa90 msgr2=0x7fcacc13ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcacc13fa90 0x7fcacc13ff10 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcab400f9d0 tx=0x7fcab400fdd0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 shutdown_connections 2023-10-18T10:40:23.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fcaac06af60 0x7fcaac06d420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcacc140450 0x7fcacc153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcacc13fa90 0x7fcacc13ff10 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 --2- 172.21.15.125:0/3765322812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcab8003970 0x7fcacc1515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 >> 172.21.15.125:0/3765322812 conn(0x7fcacc069930 msgr2=0x7fcacc136a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:23.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 shutdown_connections 2023-10-18T10:40:23.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.520+0000 7fcad1256640 1 -- 172.21.15.125:0/3765322812 wait complete. 2023-10-18T10:40:23.533 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:40:21.796046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":249,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":302744,"kb_used_data":8844,"kb_used_omap":12,"kb_used_meta":293747,"kb_avail":754671976,"statfs":{"total":773094113280,"available":772784103424,"internally_reserved":0,"allocated":9056256,"data_stored":5064494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":300797496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001015"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919741+0000","last_change":"2023-10-18T10:40:19.912867+0000","last_active":"2023-10-18T10:40:19.919741+0000","last_peered":"2023-10-18T10:40:19.919741+0000","last_clean":"2023-10-18T10:40:19.919741+0000","last_became_active":"2023-10-18T10:40:19.912618+0000","last_became_peered":"2023-10-18T10:40:19.912618+0000","last_unstale":"2023-10-18T10:40:19.919741+0000","last_undegraded":"2023-10-18T10:40:19.919741+0000","last_fullsized":"2023-10-18T10:40:19.919741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:40:18.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917449+0000","last_change":"2023-10-18T10:40:19.912063+0000","last_active":"2023-10-18T10:40:19.917449+0000","last_peered":"2023-10-18T10:40:19.917449+0000","last_clean":"2023-10-18T10:40:19.917449+0000","last_became_active":"2023-10-18T10:40:19.911766+0000","last_became_peered":"2023-10-18T10:40:19.911766+0000","last_unstale":"2023-10-18T10:40:19.917449+0000","last_undegraded":"2023-10-18T10:40:19.917449+0000","last_fullsized":"2023-10-18T10:40:19.917449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:14.456342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.921101+0000","last_change":"2023-10-18T10:40:19.919930+0000","last_active":"2023-10-18T10:40:19.921101+0000","last_peered":"2023-10-18T10:40:19.921101+0000","last_clean":"2023-10-18T10:40:19.921101+0000","last_became_active":"2023-10-18T10:40:19.919664+0000","last_became_peered":"2023-10-18T10:40:19.919664+0000","last_unstale":"2023-10-18T10:40:19.921101+0000","last_undegraded":"2023-10-18T10:40:19.921101+0000","last_fullsized":"2023-10-18T10:40:19.921101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:04.430250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910585+0000","last_change":"2023-10-18T10:40:19.909713+0000","last_active":"2023-10-18T10:40:19.910585+0000","last_peered":"2023-10-18T10:40:19.910585+0000","last_clean":"2023-10-18T10:40:19.910585+0000","last_became_active":"2023-10-18T10:40:19.909464+0000","last_became_peered":"2023-10-18T10:40:19.909464+0000","last_unstale":"2023-10-18T10:40:19.910585+0000","last_undegraded":"2023-10-18T10:40:19.910585+0000","last_fullsized":"2023-10-18T10:40:19.910585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:55.604568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.904650+0000","last_change":"2023-10-18T10:40:19.902583+0000","last_active":"2023-10-18T10:40:19.904650+0000","last_peered":"2023-10-18T10:40:19.904650+0000","last_clean":"2023-10-18T10:40:19.904650+0000","last_became_active":"2023-10-18T10:40:19.902167+0000","last_became_peered":"2023-10-18T10:40:19.902167+0000","last_unstale":"2023-10-18T10:40:19.904650+0000","last_undegraded":"2023-10-18T10:40:19.904650+0000","last_fullsized":"2023-10-18T10:40:19.904650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:10.594385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.903183+0000","last_change":"2023-10-18T10:40:19.902535+0000","last_active":"2023-10-18T10:40:19.903183+0000","last_peered":"2023-10-18T10:40:19.903183+0000","last_clean":"2023-10-18T10:40:19.903183+0000","last_became_active":"2023-10-18T10:40:19.902248+0000","last_became_peered":"2023-10-18T10:40:19.902248+0000","last_unstale":"2023-10-18T10:40:19.903183+0000","last_undegraded":"2023-10-18T10:40:19.903183+0000","last_fullsized":"2023-10-18T10:40:19.903183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:56.313746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917829+0000","last_change":"2023-10-18T10:40:19.910004+0000","last_active":"2023-10-18T10:40:19.917829+0000","last_peered":"2023-10-18T10:40:19.917829+0000","last_clean":"2023-10-18T10:40:19.917829+0000","last_became_active":"2023-10-18T10:40:19.909737+0000","last_became_peered":"2023-10-18T10:40:19.909737+0000","last_unstale":"2023-10-18T10:40:19.917829+0000","last_undegraded":"2023-10-18T10:40:19.917829+0000","last_fullsized":"2023-10-18T10:40:19.917829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:36.377245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"83'4","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924752+0000","last_change":"2023-10-18T10:40:18.906118+0000","last_active":"2023-10-18T10:40:19.924752+0000","last_peered":"2023-10-18T10:40:19.924752+0000","last_clean":"2023-10-18T10:40:19.924752+0000","last_became_active":"2023-10-18T10:40:18.905861+0000","last_became_peered":"2023-10-18T10:40:18.905861+0000","last_unstale":"2023-10-18T10:40:19.924752+0000","last_undegraded":"2023-10-18T10:40:19.924752+0000","last_fullsized":"2023-10-18T10:40:19.924752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:12.674141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922912+0000","last_change":"2023-10-18T10:40:18.901139+0000","last_active":"2023-10-18T10:40:19.922912+0000","last_peered":"2023-10-18T10:40:19.922912+0000","last_clean":"2023-10-18T10:40:19.922912+0000","last_became_active":"2023-10-18T10:40:18.900880+0000","last_became_peered":"2023-10-18T10:40:18.900880+0000","last_unstale":"2023-10-18T10:40:19.922912+0000","last_undegraded":"2023-10-18T10:40:19.922912+0000","last_fullsized":"2023-10-18T10:40:19.922912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:57.088016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924051+0000","last_change":"2023-10-18T10:40:19.923120+0000","last_active":"2023-10-18T10:40:19.924051+0000","last_peered":"2023-10-18T10:40:19.924051+0000","last_clean":"2023-10-18T10:40:19.924051+0000","last_became_active":"2023-10-18T10:40:19.922847+0000","last_became_peered":"2023-10-18T10:40:19.922847+0000","last_unstale":"2023-10-18T10:40:19.924051+0000","last_undegraded":"2023-10-18T10:40:19.924051+0000","last_fullsized":"2023-10-18T10:40:19.924051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:04.526990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922729+0000","last_change":"2023-10-18T10:40:18.901404+0000","last_active":"2023-10-18T10:40:19.922729+0000","last_peered":"2023-10-18T10:40:19.922729+0000","last_clean":"2023-10-18T10:40:19.922729+0000","last_became_active":"2023-10-18T10:40:18.900949+0000","last_became_peered":"2023-10-18T10:40:18.900949+0000","last_unstale":"2023-10-18T10:40:19.922729+0000","last_undegraded":"2023-10-18T10:40:19.922729+0000","last_fullsized":"2023-10-18T10:40:19.922729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:54.300307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896846+0000","last_change":"2023-10-18T10:40:18.907834+0000","last_active":"2023-10-18T10:40:19.896846+0000","last_peered":"2023-10-18T10:40:19.896846+0000","last_clean":"2023-10-18T10:40:19.896846+0000","last_became_active":"2023-10-18T10:40:18.907243+0000","last_became_peered":"2023-10-18T10:40:18.907243+0000","last_unstale":"2023-10-18T10:40:19.896846+0000","last_undegraded":"2023-10-18T10:40:19.896846+0000","last_fullsized":"2023-10-18T10:40:19.896846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:00.888824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919903+0000","last_change":"2023-10-18T10:40:19.919367+0000","last_active":"2023-10-18T10:40:19.919903+0000","last_peered":"2023-10-18T10:40:19.919903+0000","last_clean":"2023-10-18T10:40:19.919903+0000","last_became_active":"2023-10-18T10:40:19.919114+0000","last_became_peered":"2023-10-18T10:40:19.919114+0000","last_unstale":"2023-10-18T10:40:19.919903+0000","last_undegraded":"2023-10-18T10:40:19.919903+0000","last_fullsized":"2023-10-18T10:40:19.919903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:44.547473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.916409+0000","last_change":"2023-10-18T10:40:19.907755+0000","last_active":"2023-10-18T10:40:19.916409+0000","last_peered":"2023-10-18T10:40:19.916409+0000","last_clean":"2023-10-18T10:40:19.916409+0000","last_became_active":"2023-10-18T10:40:19.907464+0000","last_became_peered":"2023-10-18T10:40:19.907464+0000","last_unstale":"2023-10-18T10:40:19.916409+0000","last_undegraded":"2023-10-18T10:40:19.916409+0000","last_fullsized":"2023-10-18T10:40:19.916409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:43.487087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.914067+0000","last_change":"2023-10-18T10:40:19.911151+0000","last_active":"2023-10-18T10:40:19.914067+0000","last_peered":"2023-10-18T10:40:19.914067+0000","last_clean":"2023-10-18T10:40:19.914067+0000","last_became_active":"2023-10-18T10:40:19.910476+0000","last_became_peered":"2023-10-18T10:40:19.910476+0000","last_unstale":"2023-10-18T10:40:19.914067+0000","last_undegraded":"2023-10-18T10:40:19.914067+0000","last_fullsized":"2023-10-18T10:40:19.914067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:43.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.928954+0000","last_change":"2023-10-18T10:40:18.905020+0000","last_active":"2023-10-18T10:40:19.928954+0000","last_peered":"2023-10-18T10:40:19.928954+0000","last_clean":"2023-10-18T10:40:19.928954+0000","last_became_active":"2023-10-18T10:40:18.904379+0000","last_became_peered":"2023-10-18T10:40:18.904379+0000","last_unstale":"2023-10-18T10:40:19.928954+0000","last_undegraded":"2023-10-18T10:40:19.928954+0000","last_fullsized":"2023-10-18T10:40:19.928954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:28.436049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917478+0000","last_change":"2023-10-18T10:40:19.909009+0000","last_active":"2023-10-18T10:40:19.917478+0000","last_peered":"2023-10-18T10:40:19.917478+0000","last_clean":"2023-10-18T10:40:19.917478+0000","last_became_active":"2023-10-18T10:40:19.908767+0000","last_became_peered":"2023-10-18T10:40:19.908767+0000","last_unstale":"2023-10-18T10:40:19.917478+0000","last_undegraded":"2023-10-18T10:40:19.917478+0000","last_fullsized":"2023-10-18T10:40:19.917478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:00.544158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897209+0000","last_change":"2023-10-18T10:40:18.902590+0000","last_active":"2023-10-18T10:40:19.897209+0000","last_peered":"2023-10-18T10:40:19.897209+0000","last_clean":"2023-10-18T10:40:19.897209+0000","last_became_active":"2023-10-18T10:40:18.902335+0000","last_became_peered":"2023-10-18T10:40:18.902335+0000","last_unstale":"2023-10-18T10:40:19.897209+0000","last_undegraded":"2023-10-18T10:40:19.897209+0000","last_fullsized":"2023-10-18T10:40:19.897209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:58:24.515902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.902230+0000","last_change":"2023-10-18T10:40:18.905086+0000","last_active":"2023-10-18T10:40:19.902230+0000","last_peered":"2023-10-18T10:40:19.902230+0000","last_clean":"2023-10-18T10:40:19.902230+0000","last_became_active":"2023-10-18T10:40:18.904715+0000","last_became_peered":"2023-10-18T10:40:18.904715+0000","last_unstale":"2023-10-18T10:40:19.902230+0000","last_undegraded":"2023-10-18T10:40:19.902230+0000","last_fullsized":"2023-10-18T10:40:19.902230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:44.769336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:17.879176+0000","last_change":"2023-10-18T10:40:17.879176+0000","last_active":"2023-10-18T10:40:17.879176+0000","last_peered":"2023-10-18T10:40:17.879176+0000","last_clean":"2023-10-18T10:40:17.879176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:17.879176+0000","last_undegraded":"2023-10-18T10:40:17.879176+0000","last_fullsized":"2023-10-18T10:40:17.879176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897010+0000","last_change":"2023-10-18T10:40:18.901223+0000","last_active":"2023-10-18T10:40:19.897010+0000","last_peered":"2023-10-18T10:40:19.897010+0000","last_clean":"2023-10-18T10:40:19.897010+0000","last_became_active":"2023-10-18T10:40:18.900638+0000","last_became_peered":"2023-10-18T10:40:18.900638+0000","last_unstale":"2023-10-18T10:40:19.897010+0000","last_undegraded":"2023-10-18T10:40:19.897010+0000","last_fullsized":"2023-10-18T10:40:19.897010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:04.547290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926391+0000","last_change":"2023-10-18T10:40:19.925927+0000","last_active":"2023-10-18T10:40:19.926391+0000","last_peered":"2023-10-18T10:40:19.926391+0000","last_clean":"2023-10-18T10:40:19.926391+0000","last_became_active":"2023-10-18T10:40:19.925668+0000","last_became_peered":"2023-10-18T10:40:19.925668+0000","last_unstale":"2023-10-18T10:40:19.926391+0000","last_undegraded":"2023-10-18T10:40:19.926391+0000","last_fullsized":"2023-10-18T10:40:19.926391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:09.139441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897265+0000","last_change":"2023-10-18T10:40:18.904336+0000","last_active":"2023-10-18T10:40:19.897265+0000","last_peered":"2023-10-18T10:40:19.897265+0000","last_clean":"2023-10-18T10:40:19.897265+0000","last_became_active":"2023-10-18T10:40:18.904085+0000","last_became_peered":"2023-10-18T10:40:18.904085+0000","last_unstale":"2023-10-18T10:40:19.897265+0000","last_undegraded":"2023-10-18T10:40:19.897265+0000","last_fullsized":"2023-10-18T10:40:19.897265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:32.124110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.900343+0000","last_change":"2023-10-18T10:40:18.902461+0000","last_active":"2023-10-18T10:40:19.900343+0000","last_peered":"2023-10-18T10:40:19.900343+0000","last_clean":"2023-10-18T10:40:19.900343+0000","last_became_active":"2023-10-18T10:40:18.902198+0000","last_became_peered":"2023-10-18T10:40:18.902198+0000","last_unstale":"2023-10-18T10:40:19.900343+0000","last_undegraded":"2023-10-18T10:40:19.900343+0000","last_fullsized":"2023-10-18T10:40:19.900343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:18.205683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:18.884322+0000","last_change":"2023-10-18T10:40:18.884322+0000","last_active":"2023-10-18T10:40:18.884322+0000","last_peered":"2023-10-18T10:40:18.884322+0000","last_clean":"2023-10-18T10:40:18.884322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:18.884322+0000","last_undegraded":"2023-10-18T10:40:18.884322+0000","last_fullsized":"2023-10-18T10:40:18.884322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896765+0000","last_change":"2023-10-18T10:40:18.901946+0000","last_active":"2023-10-18T10:40:19.896765+0000","last_peered":"2023-10-18T10:40:19.896765+0000","last_clean":"2023-10-18T10:40:19.896765+0000","last_became_active":"2023-10-18T10:40:18.901694+0000","last_became_peered":"2023-10-18T10:40:18.901694+0000","last_unstale":"2023-10-18T10:40:19.896765+0000","last_undegraded":"2023-10-18T10:40:19.896765+0000","last_fullsized":"2023-10-18T10:40:19.896765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:08.817426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'161","reported_seq":379,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-18T10:40:14.604954+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:14.604954+0000","last_peered":"2023-10-18T10:40:14.604954+0000","last_clean":"2023-10-18T10:40:14.604954+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:14.604954+0000","last_undegraded":"2023-10-18T10:40:14.604954+0000","last_fullsized":"2023-10-18T10:40:14.604954+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":22,"acting":22,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59084,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312756,"statfs":{"total":96636764160,"available":96576262144,"internally_reserved":0,"allocated":1564672,"data_stored":1070586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.876},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73199999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.85299999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.47599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.47499999999999998,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":1.016},"last":0.45000000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.436,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.495},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.52100000000000002,"5min":0.35499999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.50700000000000001,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.47899999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.875},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.48199999999999998,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.48399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.433}]},{"osd":4,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":0.81799999999999995,"15min":0.84999999999999998},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47399999999999998,"15min":0.47099999999999997},"min":{"1min":0.35099999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.48799999999999999},"min":{"1min":0.34000000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.32300000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.313,"15min":0.313},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.87},"last":0.54600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.47699999999999998,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.016},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.46200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.629},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.76200000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.48199999999999998,"5min":0.377,"15min":0.28599999999999998},"max":{"1min":0.746,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.51600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.49399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.57099999999999995,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.81000000000000005,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.72299999999999998,"5min":0.80100000000000005,"15min":1.262},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":1.0109999999999999,"15min":1.028},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72999999999999998,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":0.96099999999999997,"15min":1.224},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.73699999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.871,"15min":1.159},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.42299999999999999,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.44600000000000001,"5min":0.39400000000000002,"15min":0.33000000000000002},"max":{"1min":0.751,"5min":0.877,"15min":0.877},"last":0.374}]},{"osd":1,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70299999999999996,"5min":0.81000000000000005,"15min":0.81899999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.501,"5min":0.36699999999999999,"15min":0.29499999999999998},"max":{"1min":0.77700000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.48299999999999998,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.86199999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.57499999999999996},"min":{"1min":0.505,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.76800000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55300000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.34699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.505,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.746,"5min":0.77600000000000002,"15min":0.79600000000000004},"last":0.57099999999999995}]},{"osd":6,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.50600000000000001},"min":{"1min":0.32700000000000001,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.78800000000000003,"5min":1.446,"15min":1.446},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.90400000000000003},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:39:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.36899999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.74199999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.41799999999999998,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.81100000000000005,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":500037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54336,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317504,"statfs":{"total":96636764160,"available":96581124096,"internally_reserved":0,"allocated":970752,"data_stored":480218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.313,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.52300000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.32700000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40300000000000002},"min":{"1min":0.34100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60999999999999999,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.35599999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.377,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.373}]},{"osd":4,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.88200000000000001,"5min":0.88800000000000001,"15min":1.458},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.52600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.95699999999999996,"15min":0.997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.93700000000000006,"15min":1.028},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.63300000000000001},"min":{"1min":0.51000000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.88900000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.52500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84599999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"last":0.51000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":502413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.371},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.434}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.505}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":1070586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.28599999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.308},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:23.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2023-10-18T10:40:23.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2023-10-18T10:40:23.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2023-10-18T10:40:23.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2023-10-18T10:40:23.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.544 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2023-10-18T10:40:23.544 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.545 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-10-18T10:40:23.545 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.546 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2023-10-18T10:40:23.546 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.547 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2023-10-18T10:40:23.548 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.548 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2023-10-18T10:40:23.549 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.549 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-10-18T10:40:23.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-10-18T10:40:23.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2023-10-18T10:40:23.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2023-10-18T10:40:23.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2023-10-18T10:40:23.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2023-10-18T10:40:23.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2023-10-18T10:40:23.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-10-18T10:40:23.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2023-10-18T10:40:23.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-10-18T10:40:23.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-10-18T10:40:23.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2023-10-18T10:40:23.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2023-10-18T10:40:23.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-10-18T10:40:23.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2023-10-18T10:40:23.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2023-10-18T10:40:23.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-10-18T10:40:23.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-10-18T10:40:23.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-10-18T10:40:23.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-10-18T10:40:23.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-10-18T10:40:23.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-10-18T10:40:23.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-10-18T10:40:23.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-10-18T10:40:23.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-10-18T10:40:23.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-10-18T10:40:23.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-10-18T10:40:23.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-10-18T10:40:23.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-10-18T10:40:23.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-10-18T10:40:23.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-10-18T10:40:23.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-10-18T10:40:23.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-10-18T10:40:23.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-10-18T10:40:23.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-10-18T10:40:23.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-10-18T10:40:23.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-10-18T10:40:23.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-10-18T10:40:23.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-10-18T10:40:23.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-10-18T10:40:23.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-10-18T10:40:23.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-10-18T10:40:23.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-10-18T10:40:23.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-10-18T10:40:23.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-10-18T10:40:23.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-10-18T10:40:23.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-10-18T10:40:23.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-10-18T10:40:23.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-10-18T10:40:23.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-10-18T10:40:23.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-10-18T10:40:23.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-10-18T10:40:23.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-10-18T10:40:23.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-10-18T10:40:23.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-10-18T10:40:23.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-10-18T10:40:23.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-10-18T10:40:23.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-10-18T10:40:23.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-10-18T10:40:23.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-10-18T10:40:23.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:17.879176+0000', 'last_change': '2023-10-18T10:40:17.879176+0000', 'last_active': '2023-10-18T10:40:17.879176+0000', 'last_peered': '2023-10-18T10:40:17.879176+0000', 'last_clean': '2023-10-18T10:40:17.879176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:17.879176+0000', 'last_undegraded': '2023-10-18T10:40:17.879176+0000', 'last_fullsized': '2023-10-18T10:40:17.879176+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:17.879176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-10-18T10:40:23.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-10-18T10:40:23.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-10-18T10:40:23.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:18.884322+0000', 'last_change': '2023-10-18T10:40:18.884322+0000', 'last_active': '2023-10-18T10:40:18.884322+0000', 'last_peered': '2023-10-18T10:40:18.884322+0000', 'last_clean': '2023-10-18T10:40:18.884322+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:18.884322+0000', 'last_undegraded': '2023-10-18T10:40:18.884322+0000', 'last_fullsized': '2023-10-18T10:40:18.884322+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:18.884322+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:23.615 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T10:40:23.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.714+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3917468129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 msgr2=0x7f6098003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.714+0000 7f60b1ff5640 1 --2- 172.21.15.125:0/3917468129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 0x7f6098003dc0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f609c009b50 tx=0x7f60980057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.714+0000 7f60abfff640 1 -- 172.21.15.125:0/3917468129 reap_dead start 2023-10-18T10:40:23.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.714+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3917468129 shutdown_connections 2023-10-18T10:40:23.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.714+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3917468129 >> 172.21.15.125:0/3917468129 conn(0x7f60ac069930 msgr2=0x7f60ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:23.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.714+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3917468129 shutdown_connections 2023-10-18T10:40:23.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.714+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3917468129 wait complete. 2023-10-18T10:40:23.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.716+0000 7f60b1ff5640 1 Processor -- start 2023-10-18T10:40:23.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.716+0000 7f60b1ff5640 1 -- start start 2023-10-18T10:40:23.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.716+0000 7f60b1ff5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 0x7f60ac13a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.716+0000 7f60b1ff5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60ac1e53a0 0x7f60ac0a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.716+0000 7f60b1ff5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60ac0a7400 0x7f60ac0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.716+0000 7f60b1ff5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac1e4f10 con 0x7f60980039a0 2023-10-18T10:40:23.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.716+0000 7f60b1ff5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac13ac30 con 0x7f60ac1e53a0 2023-10-18T10:40:23.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.716+0000 7f60b1ff5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac1e3e80 con 0x7f60ac0a7400 2023-10-18T10:40:23.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 0x7f60ac13a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 0x7f60ac13a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40712/0 (socket says 172.21.15.125:40712) 2023-10-18T10:40:23.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 -- 172.21.15.125:0/3264501949 learned_addr learned my addr 172.21.15.125:0/3264501949 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:23.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60abfff640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60ac0a7400 0x7f60ac0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60a2ffd640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60ac1e53a0 0x7f60ac0a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 -- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60ac0a7400 msgr2=0x7f60ac0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60ac0a7400 0x7f60ac0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 -- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60ac1e53a0 msgr2=0x7f60ac0a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60ac1e53a0 0x7f60ac0a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 -- 172.21.15.125:0/3264501949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac1e38c0 con 0x7f60980039a0 2023-10-18T10:40:23.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60abfff640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60ac0a7400 0x7f60ac0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:23.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60ab7fe640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 0x7f60ac13a6f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f609c009c70 tx=0x7f609c033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:23.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60a8ff9640 1 -- 172.21.15.125:0/3264501949 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c03e8f0 con 0x7f60980039a0 2023-10-18T10:40:23.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60a8ff9640 1 -- 172.21.15.125:0/3264501949 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f609c03ea50 con 0x7f60980039a0 2023-10-18T10:40:23.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60a8ff9640 1 -- 172.21.15.125:0/3264501949 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c03ee40 con 0x7f60980039a0 2023-10-18T10:40:23.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac1e3ba0 con 0x7f60980039a0 2023-10-18T10:40:23.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.717+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac13b5e0 con 0x7f60980039a0 2023-10-18T10:40:23.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.718+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac0690e0 con 0x7f60980039a0 2023-10-18T10:40:23.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.719+0000 7f60a8ff9640 1 -- 172.21.15.125:0/3264501949 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f609c04f3a0 con 0x7f60980039a0 2023-10-18T10:40:23.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.719+0000 7f60a8ff9640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f608c06af60 0x7f608c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:23.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.719+0000 7f60a8ff9640 1 -- 172.21.15.125:0/3264501949 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f609c033e40 con 0x7f60980039a0 2023-10-18T10:40:23.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.719+0000 7f60a2ffd640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f608c06af60 0x7f608c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:23.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.720+0000 7f60a2ffd640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f608c06af60 0x7f608c06d420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60940058c0 tx=0x7f6094005850 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:23.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.723+0000 7f60a8ff9640 1 -- 172.21.15.125:0/3264501949 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f609c077c40 con 0x7f60980039a0 2023-10-18T10:40:23.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.955+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f60ac1599c0 con 0x7f60980039a0 2023-10-18T10:40:23.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.956+0000 7f60a8ff9640 1 -- 172.21.15.125:0/3264501949 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1087 (secure 0 0 0) 0x7f609c04b020 con 0x7f60980039a0 2023-10-18T10:40:23.958 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:23.958 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":945,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":72},{"state_name":"active+clean","count":25}],"num_pgs":97,"num_pools":3,"num_objects":8,"data_bytes":592562,"bytes_used":310009856,"bytes_avail":772784103424,"bytes_total":773094113280,"unknown_pgs_ratio":0.74226802587509155,"write_bytes_sec":1023,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":29,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":7071}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-10-18T10:39:10.732193+0000","services":{}},"progress_events":{}} 2023-10-18T10:40:23.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f608c06af60 msgr2=0x7f608c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f608c06af60 0x7f608c06d420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60940058c0 tx=0x7f6094005850 comp rx=0 tx=0).stop 2023-10-18T10:40:23.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 msgr2=0x7f60ac13a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:23.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 0x7f60ac13a6f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f609c009c70 tx=0x7f609c033c50 comp rx=0 tx=0).stop 2023-10-18T10:40:23.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 shutdown_connections 2023-10-18T10:40:23.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60ac1e53a0 0x7f60ac0a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f608c06af60 0x7f608c06d420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60ac0a7400 0x7f60ac0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 --2- 172.21.15.125:0/3264501949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60980039a0 0x7f60ac13a6f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:23.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 >> 172.21.15.125:0/3264501949 conn(0x7f60ac069930 msgr2=0x7f60ac15cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:23.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 shutdown_connections 2023-10-18T10:40:23.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:23.958+0000 7f60b1ff5640 1 -- 172.21.15.125:0/3264501949 wait complete. 2023-10-18T10:40:23.970 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T10:40:23.971 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:24.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.071+0000 7fb2477f5640 1 -- 172.21.15.125:0/2643423965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2400704e0 msgr2=0x7fb2400cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:24.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.071+0000 7fb2477f5640 1 --2- 172.21.15.125:0/2643423965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2400704e0 0x7fb2400cc910 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb238005fd0 tx=0x7fb23802c2e0 comp rx=0 tx=0).stop 2023-10-18T10:40:24.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.071+0000 7fb2477f5640 1 -- 172.21.15.125:0/2643423965 shutdown_connections 2023-10-18T10:40:24.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.071+0000 7fb2477f5640 1 --2- 172.21.15.125:0/2643423965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2400704e0 0x7fb2400cc910 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:24.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.071+0000 7fb2477f5640 1 -- 172.21.15.125:0/2643423965 >> 172.21.15.125:0/2643423965 conn(0x7fb2400af730 msgr2=0x7fb2400afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:24.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.072+0000 7fb2477f5640 1 -- 172.21.15.125:0/2643423965 shutdown_connections 2023-10-18T10:40:24.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.072+0000 7fb2477f5640 1 -- 172.21.15.125:0/2643423965 wait complete. 2023-10-18T10:40:24.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.073+0000 7fb2477f5640 1 Processor -- start 2023-10-18T10:40:24.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.073+0000 7fb2477f5640 1 -- start start 2023-10-18T10:40:24.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.073+0000 7fb2477f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240070080 0x7fb2400b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:24.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.073+0000 7fb2477f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2400704e0 0x7fb240134470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:24.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.073+0000 7fb2477f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2401416f0 0x7fb240155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:24.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.073+0000 7fb2477f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb240155ca0 con 0x7fb2401416f0 2023-10-18T10:40:24.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.073+0000 7fb2477f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb240138580 con 0x7fb240070080 2023-10-18T10:40:24.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.073+0000 7fb2477f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb240138280 con 0x7fb2400704e0 2023-10-18T10:40:24.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2401416f0 0x7fb240155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:24.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2401416f0 0x7fb240155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40744/0 (socket says 172.21.15.125:40744) 2023-10-18T10:40:24.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 -- 172.21.15.125:0/415372146 learned_addr learned my addr 172.21.15.125:0/415372146 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:24.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245591640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240070080 0x7fb2400b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:24.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb244d90640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2400704e0 0x7fb240134470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:24.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 -- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2400704e0 msgr2=0x7fb240134470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:24.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2400704e0 0x7fb240134470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:24.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 -- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240070080 msgr2=0x7fb2400b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:24.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240070080 0x7fb2400b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:24.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 -- 172.21.15.125:0/415372146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb238009900 con 0x7fb2401416f0 2023-10-18T10:40:24.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb245d92640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2401416f0 0x7fb240155760 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb238009d90 tx=0x7fb238035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:24.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb235ffb640 1 -- 172.21.15.125:0/415372146 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238004240 con 0x7fb2401416f0 2023-10-18T10:40:24.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb235ffb640 1 -- 172.21.15.125:0/415372146 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb238004490 con 0x7fb2401416f0 2023-10-18T10:40:24.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb235ffb640 1 -- 172.21.15.125:0/415372146 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238004860 con 0x7fb2401416f0 2023-10-18T10:40:24.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2400b1cd0 con 0x7fb2401416f0 2023-10-18T10:40:24.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.074+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24015b300 con 0x7fb2401416f0 2023-10-18T10:40:24.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.075+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb240138730 con 0x7fb2401416f0 2023-10-18T10:40:24.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.076+0000 7fb235ffb640 1 -- 172.21.15.125:0/415372146 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb23803b050 con 0x7fb2401416f0 2023-10-18T10:40:24.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.076+0000 7fb235ffb640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb22406af90 0x7fb22406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:24.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.076+0000 7fb235ffb640 1 -- 172.21.15.125:0/415372146 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb2380ac630 con 0x7fb2401416f0 2023-10-18T10:40:24.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.076+0000 7fb245591640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb22406af90 0x7fb22406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:24.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.077+0000 7fb245591640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb22406af90 0x7fb22406d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb23c007fd0 tx=0x7fb23c009040 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:24.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.081+0000 7fb235ffb640 1 -- 172.21.15.125:0/415372146 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb240138730 con 0x7fb2401416f0 2023-10-18T10:40:24.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.245+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 --> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2400b1b50 con 0x7fb22406af90 2023-10-18T10:40:24.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.253+0000 7fb235ffb640 1 -- 172.21.15.125:0/415372146 <== mgr.6055 v2:172.21.15.139:6832/2642591764 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289432 (secure 0 0 0) 0x7fb23c001c40 con 0x7fb22406af90 2023-10-18T10:40:24.258 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:24.259 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:24.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb22406af90 msgr2=0x7fb22406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:24.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb22406af90 0x7fb22406d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb23c007fd0 tx=0x7fb23c009040 comp rx=0 tx=0).stop 2023-10-18T10:40:24.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2401416f0 msgr2=0x7fb240155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:24.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2401416f0 0x7fb240155760 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb238009d90 tx=0x7fb238035b50 comp rx=0 tx=0).stop 2023-10-18T10:40:24.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 shutdown_connections 2023-10-18T10:40:24.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fb22406af90 0x7fb22406d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:24.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2401416f0 0x7fb240155760 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:24.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2400704e0 0x7fb240134470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:24.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 --2- 172.21.15.125:0/415372146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb240070080 0x7fb2400b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:24.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 >> 172.21.15.125:0/415372146 conn(0x7fb2400af730 msgr2=0x7fb240069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:24.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.255+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 shutdown_connections 2023-10-18T10:40:24.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:24.256+0000 7fb2477f5640 1 -- 172.21.15.125:0/415372146 wait complete. 2023-10-18T10:40:24.268 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-18T10:40:23.796611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":303008,"kb_used_data":9108,"kb_used_omap":12,"kb_used_meta":293747,"kb_avail":754671712,"statfs":{"total":773094113280,"available":772783833088,"internally_reserved":0,"allocated":9326592,"data_stored":5192472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":300797496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.911416"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919741+0000","last_change":"2023-10-18T10:40:19.912867+0000","last_active":"2023-10-18T10:40:19.919741+0000","last_peered":"2023-10-18T10:40:19.919741+0000","last_clean":"2023-10-18T10:40:19.919741+0000","last_became_active":"2023-10-18T10:40:19.912618+0000","last_became_peered":"2023-10-18T10:40:19.912618+0000","last_unstale":"2023-10-18T10:40:19.919741+0000","last_undegraded":"2023-10-18T10:40:19.919741+0000","last_fullsized":"2023-10-18T10:40:19.919741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:40:18.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917449+0000","last_change":"2023-10-18T10:40:19.912063+0000","last_active":"2023-10-18T10:40:19.917449+0000","last_peered":"2023-10-18T10:40:19.917449+0000","last_clean":"2023-10-18T10:40:19.917449+0000","last_became_active":"2023-10-18T10:40:19.911766+0000","last_became_peered":"2023-10-18T10:40:19.911766+0000","last_unstale":"2023-10-18T10:40:19.917449+0000","last_undegraded":"2023-10-18T10:40:19.917449+0000","last_fullsized":"2023-10-18T10:40:19.917449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:14.456342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506537+0000","last_change":"2023-10-18T10:40:19.915439+0000","last_active":"2023-10-18T10:40:21.506537+0000","last_peered":"2023-10-18T10:40:21.506537+0000","last_clean":"2023-10-18T10:40:21.506537+0000","last_became_active":"2023-10-18T10:40:19.906290+0000","last_became_peered":"2023-10-18T10:40:19.906290+0000","last_unstale":"2023-10-18T10:40:21.506537+0000","last_undegraded":"2023-10-18T10:40:21.506537+0000","last_fullsized":"2023-10-18T10:40:21.506537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:05.552436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.925151+0000","last_change":"2023-10-18T10:40:19.919169+0000","last_active":"2023-10-18T10:40:19.925151+0000","last_peered":"2023-10-18T10:40:19.925151+0000","last_clean":"2023-10-18T10:40:19.925151+0000","last_became_active":"2023-10-18T10:40:19.918843+0000","last_became_peered":"2023-10-18T10:40:19.918843+0000","last_unstale":"2023-10-18T10:40:19.925151+0000","last_undegraded":"2023-10-18T10:40:19.925151+0000","last_fullsized":"2023-10-18T10:40:19.925151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:35:43.608938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902310+0000","last_change":"2023-10-18T10:40:19.918784+0000","last_active":"2023-10-18T10:40:20.902310+0000","last_peered":"2023-10-18T10:40:20.902310+0000","last_clean":"2023-10-18T10:40:20.902310+0000","last_became_active":"2023-10-18T10:40:19.918505+0000","last_became_peered":"2023-10-18T10:40:19.918505+0000","last_unstale":"2023-10-18T10:40:20.902310+0000","last_undegraded":"2023-10-18T10:40:20.902310+0000","last_fullsized":"2023-10-18T10:40:20.902310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:07:25.728867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898822+0000","last_change":"2023-10-18T10:40:19.909046+0000","last_active":"2023-10-18T10:40:20.898822+0000","last_peered":"2023-10-18T10:40:20.898822+0000","last_clean":"2023-10-18T10:40:20.898822+0000","last_became_active":"2023-10-18T10:40:19.908798+0000","last_became_peered":"2023-10-18T10:40:19.908798+0000","last_unstale":"2023-10-18T10:40:20.898822+0000","last_undegraded":"2023-10-18T10:40:20.898822+0000","last_fullsized":"2023-10-18T10:40:20.898822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:45:33.551804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.903510+0000","last_change":"2023-10-18T10:40:19.902502+0000","last_active":"2023-10-18T10:40:19.903510+0000","last_peered":"2023-10-18T10:40:19.903510+0000","last_clean":"2023-10-18T10:40:19.903510+0000","last_became_active":"2023-10-18T10:40:19.902241+0000","last_became_peered":"2023-10-18T10:40:19.902241+0000","last_unstale":"2023-10-18T10:40:19.903510+0000","last_undegraded":"2023-10-18T10:40:19.903510+0000","last_fullsized":"2023-10-18T10:40:19.903510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:37.704391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.797019+0000","last_change":"2023-10-18T10:40:19.912259+0000","last_active":"2023-10-18T10:40:21.797019+0000","last_peered":"2023-10-18T10:40:21.797019+0000","last_clean":"2023-10-18T10:40:21.797019+0000","last_became_active":"2023-10-18T10:40:19.911998+0000","last_became_peered":"2023-10-18T10:40:19.911998+0000","last_unstale":"2023-10-18T10:40:21.797019+0000","last_undegraded":"2023-10-18T10:40:21.797019+0000","last_fullsized":"2023-10-18T10:40:21.797019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:01.893970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506791+0000","last_change":"2023-10-18T10:40:19.920485+0000","last_active":"2023-10-18T10:40:21.506791+0000","last_peered":"2023-10-18T10:40:21.506791+0000","last_clean":"2023-10-18T10:40:21.506791+0000","last_became_active":"2023-10-18T10:40:19.920181+0000","last_became_peered":"2023-10-18T10:40:19.920181+0000","last_unstale":"2023-10-18T10:40:21.506791+0000","last_undegraded":"2023-10-18T10:40:21.506791+0000","last_fullsized":"2023-10-18T10:40:21.506791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:10.594385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506580+0000","last_change":"2023-10-18T10:40:19.915961+0000","last_active":"2023-10-18T10:40:21.506580+0000","last_peered":"2023-10-18T10:40:21.506580+0000","last_clean":"2023-10-18T10:40:21.506580+0000","last_became_active":"2023-10-18T10:40:19.915740+0000","last_became_peered":"2023-10-18T10:40:19.915740+0000","last_unstale":"2023-10-18T10:40:21.506580+0000","last_undegraded":"2023-10-18T10:40:21.506580+0000","last_fullsized":"2023-10-18T10:40:21.506580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:43.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.925217+0000","last_change":"2023-10-18T10:40:19.919420+0000","last_active":"2023-10-18T10:40:19.925217+0000","last_peered":"2023-10-18T10:40:19.925217+0000","last_clean":"2023-10-18T10:40:19.925217+0000","last_became_active":"2023-10-18T10:40:19.918484+0000","last_became_peered":"2023-10-18T10:40:19.918484+0000","last_unstale":"2023-10-18T10:40:19.925217+0000","last_undegraded":"2023-10-18T10:40:19.925217+0000","last_fullsized":"2023-10-18T10:40:19.925217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:05.833363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796759+0000","last_change":"2023-10-18T10:40:19.907774+0000","last_active":"2023-10-18T10:40:21.796759+0000","last_peered":"2023-10-18T10:40:21.796759+0000","last_clean":"2023-10-18T10:40:21.796759+0000","last_became_active":"2023-10-18T10:40:19.907511+0000","last_became_peered":"2023-10-18T10:40:19.907511+0000","last_unstale":"2023-10-18T10:40:21.796759+0000","last_undegraded":"2023-10-18T10:40:21.796759+0000","last_fullsized":"2023-10-18T10:40:21.796759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:19.230040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796778+0000","last_change":"2023-10-18T10:40:19.918526+0000","last_active":"2023-10-18T10:40:21.796778+0000","last_peered":"2023-10-18T10:40:21.796778+0000","last_clean":"2023-10-18T10:40:21.796778+0000","last_became_active":"2023-10-18T10:40:19.918271+0000","last_became_peered":"2023-10-18T10:40:19.918271+0000","last_unstale":"2023-10-18T10:40:21.796778+0000","last_undegraded":"2023-10-18T10:40:21.796778+0000","last_fullsized":"2023-10-18T10:40:21.796778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:47.354614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506233+0000","last_change":"2023-10-18T10:40:19.905868+0000","last_active":"2023-10-18T10:40:21.506233+0000","last_peered":"2023-10-18T10:40:21.506233+0000","last_clean":"2023-10-18T10:40:21.506233+0000","last_became_active":"2023-10-18T10:40:19.905593+0000","last_became_peered":"2023-10-18T10:40:19.905593+0000","last_unstale":"2023-10-18T10:40:21.506233+0000","last_undegraded":"2023-10-18T10:40:21.506233+0000","last_fullsized":"2023-10-18T10:40:21.506233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:55.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506063+0000","last_change":"2023-10-18T10:40:19.905714+0000","last_active":"2023-10-18T10:40:21.506063+0000","last_peered":"2023-10-18T10:40:21.506063+0000","last_clean":"2023-10-18T10:40:21.506063+0000","last_became_active":"2023-10-18T10:40:19.905464+0000","last_became_peered":"2023-10-18T10:40:19.905464+0000","last_unstale":"2023-10-18T10:40:21.506063+0000","last_undegraded":"2023-10-18T10:40:21.506063+0000","last_fullsized":"2023-10-18T10:40:21.506063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:01.893970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.908797+0000","last_change":"2023-10-18T10:40:19.907404+0000","last_active":"2023-10-18T10:40:19.908797+0000","last_peered":"2023-10-18T10:40:19.908797+0000","last_clean":"2023-10-18T10:40:19.908797+0000","last_became_active":"2023-10-18T10:40:19.907152+0000","last_became_peered":"2023-10-18T10:40:19.907152+0000","last_unstale":"2023-10-18T10:40:19.908797+0000","last_undegraded":"2023-10-18T10:40:19.908797+0000","last_fullsized":"2023-10-18T10:40:19.908797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:20.332693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.921101+0000","last_change":"2023-10-18T10:40:19.919930+0000","last_active":"2023-10-18T10:40:19.921101+0000","last_peered":"2023-10-18T10:40:19.921101+0000","last_clean":"2023-10-18T10:40:19.921101+0000","last_became_active":"2023-10-18T10:40:19.919664+0000","last_became_peered":"2023-10-18T10:40:19.919664+0000","last_unstale":"2023-10-18T10:40:19.921101+0000","last_undegraded":"2023-10-18T10:40:19.921101+0000","last_fullsized":"2023-10-18T10:40:19.921101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:04.430250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926304+0000","last_change":"2023-10-18T10:40:19.923065+0000","last_active":"2023-10-18T10:40:19.926304+0000","last_peered":"2023-10-18T10:40:19.926304+0000","last_clean":"2023-10-18T10:40:19.926304+0000","last_became_active":"2023-10-18T10:40:19.921641+0000","last_became_peered":"2023-10-18T10:40:19.921641+0000","last_unstale":"2023-10-18T10:40:19.926304+0000","last_undegraded":"2023-10-18T10:40:19.926304+0000","last_fullsized":"2023-10-18T10:40:19.926304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:31:07.900837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902717+0000","last_change":"2023-10-18T10:40:19.912926+0000","last_active":"2023-10-18T10:40:20.902717+0000","last_peered":"2023-10-18T10:40:20.902717+0000","last_clean":"2023-10-18T10:40:20.902717+0000","last_became_active":"2023-10-18T10:40:19.912676+0000","last_became_peered":"2023-10-18T10:40:19.912676+0000","last_unstale":"2023-10-18T10:40:20.902717+0000","last_undegraded":"2023-10-18T10:40:20.902717+0000","last_fullsized":"2023-10-18T10:40:20.902717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:34.217906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910585+0000","last_change":"2023-10-18T10:40:19.909713+0000","last_active":"2023-10-18T10:40:19.910585+0000","last_peered":"2023-10-18T10:40:19.910585+0000","last_clean":"2023-10-18T10:40:19.910585+0000","last_became_active":"2023-10-18T10:40:19.909464+0000","last_became_peered":"2023-10-18T10:40:19.909464+0000","last_unstale":"2023-10-18T10:40:19.910585+0000","last_undegraded":"2023-10-18T10:40:19.910585+0000","last_fullsized":"2023-10-18T10:40:19.910585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:55.604568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796869+0000","last_change":"2023-10-18T10:40:19.909983+0000","last_active":"2023-10-18T10:40:21.796869+0000","last_peered":"2023-10-18T10:40:21.796869+0000","last_clean":"2023-10-18T10:40:21.796869+0000","last_became_active":"2023-10-18T10:40:19.909687+0000","last_became_peered":"2023-10-18T10:40:19.909687+0000","last_unstale":"2023-10-18T10:40:21.796869+0000","last_undegraded":"2023-10-18T10:40:21.796869+0000","last_fullsized":"2023-10-18T10:40:21.796869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:09.822572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.904650+0000","last_change":"2023-10-18T10:40:19.902583+0000","last_active":"2023-10-18T10:40:19.904650+0000","last_peered":"2023-10-18T10:40:19.904650+0000","last_clean":"2023-10-18T10:40:19.904650+0000","last_became_active":"2023-10-18T10:40:19.902167+0000","last_became_peered":"2023-10-18T10:40:19.902167+0000","last_unstale":"2023-10-18T10:40:19.904650+0000","last_undegraded":"2023-10-18T10:40:19.904650+0000","last_fullsized":"2023-10-18T10:40:19.904650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:10.594385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.903183+0000","last_change":"2023-10-18T10:40:19.902535+0000","last_active":"2023-10-18T10:40:19.903183+0000","last_peered":"2023-10-18T10:40:19.903183+0000","last_clean":"2023-10-18T10:40:19.903183+0000","last_became_active":"2023-10-18T10:40:19.902248+0000","last_became_peered":"2023-10-18T10:40:19.902248+0000","last_unstale":"2023-10-18T10:40:19.903183+0000","last_undegraded":"2023-10-18T10:40:19.903183+0000","last_fullsized":"2023-10-18T10:40:19.903183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:56.313746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506047+0000","last_change":"2023-10-18T10:40:19.907648+0000","last_active":"2023-10-18T10:40:21.506047+0000","last_peered":"2023-10-18T10:40:21.506047+0000","last_clean":"2023-10-18T10:40:21.506047+0000","last_became_active":"2023-10-18T10:40:19.906778+0000","last_became_peered":"2023-10-18T10:40:19.906778+0000","last_unstale":"2023-10-18T10:40:21.506047+0000","last_undegraded":"2023-10-18T10:40:21.506047+0000","last_fullsized":"2023-10-18T10:40:21.506047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:09.822572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796772+0000","last_change":"2023-10-18T10:40:19.912919+0000","last_active":"2023-10-18T10:40:21.796772+0000","last_peered":"2023-10-18T10:40:21.796772+0000","last_clean":"2023-10-18T10:40:21.796772+0000","last_became_active":"2023-10-18T10:40:19.912669+0000","last_became_peered":"2023-10-18T10:40:19.912669+0000","last_unstale":"2023-10-18T10:40:21.796772+0000","last_undegraded":"2023-10-18T10:40:21.796772+0000","last_fullsized":"2023-10-18T10:40:21.796772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:08.877366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796691+0000","last_change":"2023-10-18T10:40:19.908565+0000","last_active":"2023-10-18T10:40:21.796691+0000","last_peered":"2023-10-18T10:40:21.796691+0000","last_clean":"2023-10-18T10:40:21.796691+0000","last_became_active":"2023-10-18T10:40:19.908239+0000","last_became_peered":"2023-10-18T10:40:19.908239+0000","last_unstale":"2023-10-18T10:40:21.796691+0000","last_undegraded":"2023-10-18T10:40:21.796691+0000","last_fullsized":"2023-10-18T10:40:21.796691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:09.515116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.920159+0000","last_change":"2023-10-18T10:40:19.917470+0000","last_active":"2023-10-18T10:40:19.920159+0000","last_peered":"2023-10-18T10:40:19.920159+0000","last_clean":"2023-10-18T10:40:19.920159+0000","last_became_active":"2023-10-18T10:40:19.917044+0000","last_became_peered":"2023-10-18T10:40:19.917044+0000","last_unstale":"2023-10-18T10:40:19.920159+0000","last_undegraded":"2023-10-18T10:40:19.920159+0000","last_fullsized":"2023-10-18T10:40:19.920159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:21.627876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917829+0000","last_change":"2023-10-18T10:40:19.910004+0000","last_active":"2023-10-18T10:40:19.917829+0000","last_peered":"2023-10-18T10:40:19.917829+0000","last_clean":"2023-10-18T10:40:19.917829+0000","last_became_active":"2023-10-18T10:40:19.909737+0000","last_became_peered":"2023-10-18T10:40:19.909737+0000","last_unstale":"2023-10-18T10:40:19.917829+0000","last_undegraded":"2023-10-18T10:40:19.917829+0000","last_fullsized":"2023-10-18T10:40:19.917829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:36.377245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506257+0000","last_change":"2023-10-18T10:40:19.921023+0000","last_active":"2023-10-18T10:40:21.506257+0000","last_peered":"2023-10-18T10:40:21.506257+0000","last_clean":"2023-10-18T10:40:21.506257+0000","last_became_active":"2023-10-18T10:40:19.919546+0000","last_became_peered":"2023-10-18T10:40:19.919546+0000","last_unstale":"2023-10-18T10:40:21.506257+0000","last_undegraded":"2023-10-18T10:40:21.506257+0000","last_fullsized":"2023-10-18T10:40:21.506257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:56.313746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796648+0000","last_change":"2023-10-18T10:40:19.907166+0000","last_active":"2023-10-18T10:40:21.796648+0000","last_peered":"2023-10-18T10:40:21.796648+0000","last_clean":"2023-10-18T10:40:21.796648+0000","last_became_active":"2023-10-18T10:40:19.906900+0000","last_became_peered":"2023-10-18T10:40:19.906900+0000","last_unstale":"2023-10-18T10:40:21.796648+0000","last_undegraded":"2023-10-18T10:40:21.796648+0000","last_fullsized":"2023-10-18T10:40:21.796648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:51.778136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.920080+0000","last_change":"2023-10-18T10:40:19.917537+0000","last_active":"2023-10-18T10:40:19.920080+0000","last_peered":"2023-10-18T10:40:19.920080+0000","last_clean":"2023-10-18T10:40:19.920080+0000","last_became_active":"2023-10-18T10:40:19.916919+0000","last_became_peered":"2023-10-18T10:40:19.916919+0000","last_unstale":"2023-10-18T10:40:19.920080+0000","last_undegraded":"2023-10-18T10:40:19.920080+0000","last_fullsized":"2023-10-18T10:40:19.920080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:41.408790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898663+0000","last_change":"2023-10-18T10:40:19.915799+0000","last_active":"2023-10-18T10:40:20.898663+0000","last_peered":"2023-10-18T10:40:20.898663+0000","last_clean":"2023-10-18T10:40:20.898663+0000","last_became_active":"2023-10-18T10:40:19.915535+0000","last_became_peered":"2023-10-18T10:40:19.915535+0000","last_unstale":"2023-10-18T10:40:20.898663+0000","last_undegraded":"2023-10-18T10:40:20.898663+0000","last_fullsized":"2023-10-18T10:40:20.898663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:33.129256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902670+0000","last_change":"2023-10-18T10:40:19.910827+0000","last_active":"2023-10-18T10:40:20.902670+0000","last_peered":"2023-10-18T10:40:20.902670+0000","last_clean":"2023-10-18T10:40:20.902670+0000","last_became_active":"2023-10-18T10:40:19.910578+0000","last_became_peered":"2023-10-18T10:40:19.910578+0000","last_unstale":"2023-10-18T10:40:20.902670+0000","last_undegraded":"2023-10-18T10:40:20.902670+0000","last_fullsized":"2023-10-18T10:40:20.902670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:49.380981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"83'4","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924752+0000","last_change":"2023-10-18T10:40:18.906118+0000","last_active":"2023-10-18T10:40:19.924752+0000","last_peered":"2023-10-18T10:40:19.924752+0000","last_clean":"2023-10-18T10:40:19.924752+0000","last_became_active":"2023-10-18T10:40:18.905861+0000","last_became_peered":"2023-10-18T10:40:18.905861+0000","last_unstale":"2023-10-18T10:40:19.924752+0000","last_undegraded":"2023-10-18T10:40:19.924752+0000","last_fullsized":"2023-10-18T10:40:19.924752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:12.674141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926449+0000","last_change":"2023-10-18T10:40:19.924903+0000","last_active":"2023-10-18T10:40:19.926449+0000","last_peered":"2023-10-18T10:40:19.926449+0000","last_clean":"2023-10-18T10:40:19.926449+0000","last_became_active":"2023-10-18T10:40:19.924531+0000","last_became_peered":"2023-10-18T10:40:19.924531+0000","last_unstale":"2023-10-18T10:40:19.926449+0000","last_undegraded":"2023-10-18T10:40:19.926449+0000","last_fullsized":"2023-10-18T10:40:19.926449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:46.773804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.899076+0000","last_change":"2023-10-18T10:40:18.899749+0000","last_active":"2023-10-18T10:40:20.899076+0000","last_peered":"2023-10-18T10:40:20.899076+0000","last_clean":"2023-10-18T10:40:20.899076+0000","last_became_active":"2023-10-18T10:40:18.899485+0000","last_became_peered":"2023-10-18T10:40:18.899485+0000","last_unstale":"2023-10-18T10:40:20.899076+0000","last_undegraded":"2023-10-18T10:40:20.899076+0000","last_fullsized":"2023-10-18T10:40:20.899076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:42.126241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.899018+0000","last_change":"2023-10-18T10:40:19.907847+0000","last_active":"2023-10-18T10:40:20.899018+0000","last_peered":"2023-10-18T10:40:20.899018+0000","last_clean":"2023-10-18T10:40:20.899018+0000","last_became_active":"2023-10-18T10:40:19.907296+0000","last_became_peered":"2023-10-18T10:40:19.907296+0000","last_unstale":"2023-10-18T10:40:20.899018+0000","last_undegraded":"2023-10-18T10:40:20.899018+0000","last_fullsized":"2023-10-18T10:40:20.899018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:24:07.576898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898947+0000","last_change":"2023-10-18T10:40:18.904926+0000","last_active":"2023-10-18T10:40:20.898947+0000","last_peered":"2023-10-18T10:40:20.898947+0000","last_clean":"2023-10-18T10:40:20.898947+0000","last_became_active":"2023-10-18T10:40:18.904685+0000","last_became_peered":"2023-10-18T10:40:18.904685+0000","last_unstale":"2023-10-18T10:40:20.898947+0000","last_undegraded":"2023-10-18T10:40:20.898947+0000","last_fullsized":"2023-10-18T10:40:20.898947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:04.547290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506691+0000","last_change":"2023-10-18T10:40:19.909120+0000","last_active":"2023-10-18T10:40:21.506691+0000","last_peered":"2023-10-18T10:40:21.506691+0000","last_clean":"2023-10-18T10:40:21.506691+0000","last_became_active":"2023-10-18T10:40:19.908791+0000","last_became_peered":"2023-10-18T10:40:19.908791+0000","last_unstale":"2023-10-18T10:40:21.506691+0000","last_undegraded":"2023-10-18T10:40:21.506691+0000","last_fullsized":"2023-10-18T10:40:21.506691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:44.547473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898954+0000","last_change":"2023-10-18T10:40:18.897418+0000","last_active":"2023-10-18T10:40:20.898954+0000","last_peered":"2023-10-18T10:40:20.898954+0000","last_clean":"2023-10-18T10:40:20.898954+0000","last_became_active":"2023-10-18T10:40:18.897166+0000","last_became_peered":"2023-10-18T10:40:18.897166+0000","last_unstale":"2023-10-18T10:40:20.898954+0000","last_undegraded":"2023-10-18T10:40:20.898954+0000","last_fullsized":"2023-10-18T10:40:20.898954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:55.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.909100+0000","last_change":"2023-10-18T10:40:19.908499+0000","last_active":"2023-10-18T10:40:19.909100+0000","last_peered":"2023-10-18T10:40:19.909100+0000","last_clean":"2023-10-18T10:40:19.909100+0000","last_became_active":"2023-10-18T10:40:19.907640+0000","last_became_peered":"2023-10-18T10:40:19.907640+0000","last_unstale":"2023-10-18T10:40:19.909100+0000","last_undegraded":"2023-10-18T10:40:19.909100+0000","last_fullsized":"2023-10-18T10:40:19.909100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:24.687875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922912+0000","last_change":"2023-10-18T10:40:18.901139+0000","last_active":"2023-10-18T10:40:19.922912+0000","last_peered":"2023-10-18T10:40:19.922912+0000","last_clean":"2023-10-18T10:40:19.922912+0000","last_became_active":"2023-10-18T10:40:18.900880+0000","last_became_peered":"2023-10-18T10:40:18.900880+0000","last_unstale":"2023-10-18T10:40:19.922912+0000","last_undegraded":"2023-10-18T10:40:19.922912+0000","last_fullsized":"2023-10-18T10:40:19.922912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:57.088016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924051+0000","last_change":"2023-10-18T10:40:19.923120+0000","last_active":"2023-10-18T10:40:19.924051+0000","last_peered":"2023-10-18T10:40:19.924051+0000","last_clean":"2023-10-18T10:40:19.924051+0000","last_became_active":"2023-10-18T10:40:19.922847+0000","last_became_peered":"2023-10-18T10:40:19.922847+0000","last_unstale":"2023-10-18T10:40:19.924051+0000","last_undegraded":"2023-10-18T10:40:19.924051+0000","last_fullsized":"2023-10-18T10:40:19.924051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:04.526990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922729+0000","last_change":"2023-10-18T10:40:18.901404+0000","last_active":"2023-10-18T10:40:19.922729+0000","last_peered":"2023-10-18T10:40:19.922729+0000","last_clean":"2023-10-18T10:40:19.922729+0000","last_became_active":"2023-10-18T10:40:18.900949+0000","last_became_peered":"2023-10-18T10:40:18.900949+0000","last_unstale":"2023-10-18T10:40:19.922729+0000","last_undegraded":"2023-10-18T10:40:19.922729+0000","last_fullsized":"2023-10-18T10:40:19.922729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:54.300307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902234+0000","last_change":"2023-10-18T10:40:19.910432+0000","last_active":"2023-10-18T10:40:20.902234+0000","last_peered":"2023-10-18T10:40:20.902234+0000","last_clean":"2023-10-18T10:40:20.902234+0000","last_became_active":"2023-10-18T10:40:19.910017+0000","last_became_peered":"2023-10-18T10:40:19.910017+0000","last_unstale":"2023-10-18T10:40:20.902234+0000","last_undegraded":"2023-10-18T10:40:20.902234+0000","last_fullsized":"2023-10-18T10:40:20.902234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:41.294716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896846+0000","last_change":"2023-10-18T10:40:18.907834+0000","last_active":"2023-10-18T10:40:19.896846+0000","last_peered":"2023-10-18T10:40:19.896846+0000","last_clean":"2023-10-18T10:40:19.896846+0000","last_became_active":"2023-10-18T10:40:18.907243+0000","last_became_peered":"2023-10-18T10:40:18.907243+0000","last_unstale":"2023-10-18T10:40:19.896846+0000","last_undegraded":"2023-10-18T10:40:19.896846+0000","last_fullsized":"2023-10-18T10:40:19.896846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:00.888824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919903+0000","last_change":"2023-10-18T10:40:19.919367+0000","last_active":"2023-10-18T10:40:19.919903+0000","last_peered":"2023-10-18T10:40:19.919903+0000","last_clean":"2023-10-18T10:40:19.919903+0000","last_became_active":"2023-10-18T10:40:19.919114+0000","last_became_peered":"2023-10-18T10:40:19.919114+0000","last_unstale":"2023-10-18T10:40:19.919903+0000","last_undegraded":"2023-10-18T10:40:19.919903+0000","last_fullsized":"2023-10-18T10:40:19.919903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:44.547473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902222+0000","last_change":"2023-10-18T10:40:18.904489+0000","last_active":"2023-10-18T10:40:20.902222+0000","last_peered":"2023-10-18T10:40:20.902222+0000","last_clean":"2023-10-18T10:40:20.902222+0000","last_became_active":"2023-10-18T10:40:18.904227+0000","last_became_peered":"2023-10-18T10:40:18.904227+0000","last_unstale":"2023-10-18T10:40:20.902222+0000","last_undegraded":"2023-10-18T10:40:20.902222+0000","last_fullsized":"2023-10-18T10:40:20.902222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:23:40.577154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.916409+0000","last_change":"2023-10-18T10:40:19.907755+0000","last_active":"2023-10-18T10:40:19.916409+0000","last_peered":"2023-10-18T10:40:19.916409+0000","last_clean":"2023-10-18T10:40:19.916409+0000","last_became_active":"2023-10-18T10:40:19.907464+0000","last_became_peered":"2023-10-18T10:40:19.907464+0000","last_unstale":"2023-10-18T10:40:19.916409+0000","last_undegraded":"2023-10-18T10:40:19.916409+0000","last_fullsized":"2023-10-18T10:40:19.916409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:43.487087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506791+0000","last_change":"2023-10-18T10:40:18.908362+0000","last_active":"2023-10-18T10:40:21.506791+0000","last_peered":"2023-10-18T10:40:21.506791+0000","last_clean":"2023-10-18T10:40:21.506791+0000","last_became_active":"2023-10-18T10:40:18.908086+0000","last_became_peered":"2023-10-18T10:40:18.908086+0000","last_unstale":"2023-10-18T10:40:21.506791+0000","last_undegraded":"2023-10-18T10:40:21.506791+0000","last_fullsized":"2023-10-18T10:40:21.506791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:46.349468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.914067+0000","last_change":"2023-10-18T10:40:19.911151+0000","last_active":"2023-10-18T10:40:19.914067+0000","last_peered":"2023-10-18T10:40:19.914067+0000","last_clean":"2023-10-18T10:40:19.914067+0000","last_became_active":"2023-10-18T10:40:19.910476+0000","last_became_peered":"2023-10-18T10:40:19.910476+0000","last_unstale":"2023-10-18T10:40:19.914067+0000","last_undegraded":"2023-10-18T10:40:19.914067+0000","last_fullsized":"2023-10-18T10:40:19.914067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:43.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910808+0000","last_change":"2023-10-18T10:40:18.902928+0000","last_active":"2023-10-18T10:40:19.910808+0000","last_peered":"2023-10-18T10:40:19.910808+0000","last_clean":"2023-10-18T10:40:19.910808+0000","last_became_active":"2023-10-18T10:40:18.900825+0000","last_became_peered":"2023-10-18T10:40:18.900825+0000","last_unstale":"2023-10-18T10:40:19.910808+0000","last_undegraded":"2023-10-18T10:40:19.910808+0000","last_fullsized":"2023-10-18T10:40:19.910808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:40.289570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902809+0000","last_change":"2023-10-18T10:40:19.916696+0000","last_active":"2023-10-18T10:40:20.902809+0000","last_peered":"2023-10-18T10:40:20.902809+0000","last_clean":"2023-10-18T10:40:20.902809+0000","last_became_active":"2023-10-18T10:40:19.916431+0000","last_became_peered":"2023-10-18T10:40:19.916431+0000","last_unstale":"2023-10-18T10:40:20.902809+0000","last_undegraded":"2023-10-18T10:40:20.902809+0000","last_fullsized":"2023-10-18T10:40:20.902809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:16.648782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"83'3","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506480+0000","last_change":"2023-10-18T10:40:18.905401+0000","last_active":"2023-10-18T10:40:21.506480+0000","last_peered":"2023-10-18T10:40:21.506480+0000","last_clean":"2023-10-18T10:40:21.506480+0000","last_became_active":"2023-10-18T10:40:18.905137+0000","last_became_peered":"2023-10-18T10:40:18.905137+0000","last_unstale":"2023-10-18T10:40:21.506480+0000","last_undegraded":"2023-10-18T10:40:21.506480+0000","last_fullsized":"2023-10-18T10:40:21.506480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:18.224893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919989+0000","last_change":"2023-10-18T10:40:19.917320+0000","last_active":"2023-10-18T10:40:19.919989+0000","last_peered":"2023-10-18T10:40:19.919989+0000","last_clean":"2023-10-18T10:40:19.919989+0000","last_became_active":"2023-10-18T10:40:19.916828+0000","last_became_peered":"2023-10-18T10:40:19.916828+0000","last_unstale":"2023-10-18T10:40:19.919989+0000","last_undegraded":"2023-10-18T10:40:19.919989+0000","last_fullsized":"2023-10-18T10:40:19.919989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:58:04.490274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.928954+0000","last_change":"2023-10-18T10:40:18.905020+0000","last_active":"2023-10-18T10:40:19.928954+0000","last_peered":"2023-10-18T10:40:19.928954+0000","last_clean":"2023-10-18T10:40:19.928954+0000","last_became_active":"2023-10-18T10:40:18.904379+0000","last_became_peered":"2023-10-18T10:40:18.904379+0000","last_unstale":"2023-10-18T10:40:19.928954+0000","last_undegraded":"2023-10-18T10:40:19.928954+0000","last_fullsized":"2023-10-18T10:40:19.928954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:28.436049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919896+0000","last_change":"2023-10-18T10:40:19.912403+0000","last_active":"2023-10-18T10:40:19.919896+0000","last_peered":"2023-10-18T10:40:19.919896+0000","last_clean":"2023-10-18T10:40:19.919896+0000","last_became_active":"2023-10-18T10:40:19.912123+0000","last_became_peered":"2023-10-18T10:40:19.912123+0000","last_unstale":"2023-10-18T10:40:19.919896+0000","last_undegraded":"2023-10-18T10:40:19.919896+0000","last_fullsized":"2023-10-18T10:40:19.919896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:23:41.582300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506310+0000","last_change":"2023-10-18T10:40:18.907634+0000","last_active":"2023-10-18T10:40:21.506310+0000","last_peered":"2023-10-18T10:40:21.506310+0000","last_clean":"2023-10-18T10:40:21.506310+0000","last_became_active":"2023-10-18T10:40:18.907350+0000","last_became_peered":"2023-10-18T10:40:18.907350+0000","last_unstale":"2023-10-18T10:40:21.506310+0000","last_undegraded":"2023-10-18T10:40:21.506310+0000","last_fullsized":"2023-10-18T10:40:21.506310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:53.817705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796678+0000","last_change":"2023-10-18T10:40:19.909879+0000","last_active":"2023-10-18T10:40:21.796678+0000","last_peered":"2023-10-18T10:40:21.796678+0000","last_clean":"2023-10-18T10:40:21.796678+0000","last_became_active":"2023-10-18T10:40:19.909602+0000","last_became_peered":"2023-10-18T10:40:19.909602+0000","last_unstale":"2023-10-18T10:40:21.796678+0000","last_undegraded":"2023-10-18T10:40:21.796678+0000","last_fullsized":"2023-10-18T10:40:21.796678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:40.764061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796570+0000","last_change":"2023-10-18T10:40:18.903374+0000","last_active":"2023-10-18T10:40:21.796570+0000","last_peered":"2023-10-18T10:40:21.796570+0000","last_clean":"2023-10-18T10:40:21.796570+0000","last_became_active":"2023-10-18T10:40:18.902956+0000","last_became_peered":"2023-10-18T10:40:18.902956+0000","last_unstale":"2023-10-18T10:40:21.796570+0000","last_undegraded":"2023-10-18T10:40:21.796570+0000","last_fullsized":"2023-10-18T10:40:21.796570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:35:55.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902416+0000","last_change":"2023-10-18T10:40:19.908861+0000","last_active":"2023-10-18T10:40:20.902416+0000","last_peered":"2023-10-18T10:40:20.902416+0000","last_clean":"2023-10-18T10:40:20.902416+0000","last_became_active":"2023-10-18T10:40:19.908585+0000","last_became_peered":"2023-10-18T10:40:19.908585+0000","last_unstale":"2023-10-18T10:40:20.902416+0000","last_undegraded":"2023-10-18T10:40:20.902416+0000","last_fullsized":"2023-10-18T10:40:20.902416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:27.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506173+0000","last_change":"2023-10-18T10:40:18.902520+0000","last_active":"2023-10-18T10:40:21.506173+0000","last_peered":"2023-10-18T10:40:21.506173+0000","last_clean":"2023-10-18T10:40:21.506173+0000","last_became_active":"2023-10-18T10:40:18.902149+0000","last_became_peered":"2023-10-18T10:40:18.902149+0000","last_unstale":"2023-10-18T10:40:21.506173+0000","last_undegraded":"2023-10-18T10:40:21.506173+0000","last_fullsized":"2023-10-18T10:40:21.506173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:28.391862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917478+0000","last_change":"2023-10-18T10:40:19.909009+0000","last_active":"2023-10-18T10:40:19.917478+0000","last_peered":"2023-10-18T10:40:19.917478+0000","last_clean":"2023-10-18T10:40:19.917478+0000","last_became_active":"2023-10-18T10:40:19.908767+0000","last_became_peered":"2023-10-18T10:40:19.908767+0000","last_unstale":"2023-10-18T10:40:19.917478+0000","last_undegraded":"2023-10-18T10:40:19.917478+0000","last_fullsized":"2023-10-18T10:40:19.917478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:00.544158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.929032+0000","last_change":"2023-10-18T10:40:18.904087+0000","last_active":"2023-10-18T10:40:19.929032+0000","last_peered":"2023-10-18T10:40:19.929032+0000","last_clean":"2023-10-18T10:40:19.929032+0000","last_became_active":"2023-10-18T10:40:18.903533+0000","last_became_peered":"2023-10-18T10:40:18.903533+0000","last_unstale":"2023-10-18T10:40:19.929032+0000","last_undegraded":"2023-10-18T10:40:19.929032+0000","last_fullsized":"2023-10-18T10:40:19.929032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:43:32.864029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506533+0000","last_change":"2023-10-18T10:40:19.920719+0000","last_active":"2023-10-18T10:40:21.506533+0000","last_peered":"2023-10-18T10:40:21.506533+0000","last_clean":"2023-10-18T10:40:21.506533+0000","last_became_active":"2023-10-18T10:40:19.919994+0000","last_became_peered":"2023-10-18T10:40:19.919994+0000","last_unstale":"2023-10-18T10:40:21.506533+0000","last_undegraded":"2023-10-18T10:40:21.506533+0000","last_fullsized":"2023-10-18T10:40:21.506533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:58.093162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796980+0000","last_change":"2023-10-18T10:40:18.904201+0000","last_active":"2023-10-18T10:40:21.796980+0000","last_peered":"2023-10-18T10:40:21.796980+0000","last_clean":"2023-10-18T10:40:21.796980+0000","last_became_active":"2023-10-18T10:40:18.902679+0000","last_became_peered":"2023-10-18T10:40:18.902679+0000","last_unstale":"2023-10-18T10:40:21.796980+0000","last_undegraded":"2023-10-18T10:40:21.796980+0000","last_fullsized":"2023-10-18T10:40:21.796980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:34:26.005003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902871+0000","last_change":"2023-10-18T10:40:19.918134+0000","last_active":"2023-10-18T10:40:20.902871+0000","last_peered":"2023-10-18T10:40:20.902871+0000","last_clean":"2023-10-18T10:40:20.902871+0000","last_became_active":"2023-10-18T10:40:19.917832+0000","last_became_peered":"2023-10-18T10:40:19.917832+0000","last_unstale":"2023-10-18T10:40:20.902871+0000","last_undegraded":"2023-10-18T10:40:20.902871+0000","last_fullsized":"2023-10-18T10:40:20.902871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:55.584201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506636+0000","last_change":"2023-10-18T10:40:18.909663+0000","last_active":"2023-10-18T10:40:21.506636+0000","last_peered":"2023-10-18T10:40:21.506636+0000","last_clean":"2023-10-18T10:40:21.506636+0000","last_became_active":"2023-10-18T10:40:18.909380+0000","last_became_peered":"2023-10-18T10:40:18.909380+0000","last_unstale":"2023-10-18T10:40:21.506636+0000","last_undegraded":"2023-10-18T10:40:21.506636+0000","last_fullsized":"2023-10-18T10:40:21.506636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:39.758915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926264+0000","last_change":"2023-10-18T10:40:19.921712+0000","last_active":"2023-10-18T10:40:19.926264+0000","last_peered":"2023-10-18T10:40:19.926264+0000","last_clean":"2023-10-18T10:40:19.926264+0000","last_became_active":"2023-10-18T10:40:19.921407+0000","last_became_peered":"2023-10-18T10:40:19.921407+0000","last_unstale":"2023-10-18T10:40:19.926264+0000","last_undegraded":"2023-10-18T10:40:19.926264+0000","last_fullsized":"2023-10-18T10:40:19.926264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:27:11.030985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506602+0000","last_change":"2023-10-18T10:40:18.908209+0000","last_active":"2023-10-18T10:40:21.506602+0000","last_peered":"2023-10-18T10:40:21.506602+0000","last_clean":"2023-10-18T10:40:21.506602+0000","last_became_active":"2023-10-18T10:40:18.907903+0000","last_became_peered":"2023-10-18T10:40:18.907903+0000","last_unstale":"2023-10-18T10:40:21.506602+0000","last_undegraded":"2023-10-18T10:40:21.506602+0000","last_fullsized":"2023-10-18T10:40:21.506602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:07.872220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902781+0000","last_change":"2023-10-18T10:40:19.911382+0000","last_active":"2023-10-18T10:40:20.902781+0000","last_peered":"2023-10-18T10:40:20.902781+0000","last_clean":"2023-10-18T10:40:20.902781+0000","last_became_active":"2023-10-18T10:40:19.911148+0000","last_became_peered":"2023-10-18T10:40:19.911148+0000","last_unstale":"2023-10-18T10:40:20.902781+0000","last_undegraded":"2023-10-18T10:40:20.902781+0000","last_fullsized":"2023-10-18T10:40:20.902781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:58:23.503790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897209+0000","last_change":"2023-10-18T10:40:18.902590+0000","last_active":"2023-10-18T10:40:19.897209+0000","last_peered":"2023-10-18T10:40:19.897209+0000","last_clean":"2023-10-18T10:40:19.897209+0000","last_became_active":"2023-10-18T10:40:18.902335+0000","last_became_peered":"2023-10-18T10:40:18.902335+0000","last_unstale":"2023-10-18T10:40:19.897209+0000","last_undegraded":"2023-10-18T10:40:19.897209+0000","last_fullsized":"2023-10-18T10:40:19.897209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:58:24.515902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898728+0000","last_change":"2023-10-18T10:40:19.914939+0000","last_active":"2023-10-18T10:40:20.898728+0000","last_peered":"2023-10-18T10:40:20.898728+0000","last_clean":"2023-10-18T10:40:20.898728+0000","last_became_active":"2023-10-18T10:40:19.914697+0000","last_became_peered":"2023-10-18T10:40:19.914697+0000","last_unstale":"2023-10-18T10:40:20.898728+0000","last_undegraded":"2023-10-18T10:40:20.898728+0000","last_fullsized":"2023-10-18T10:40:20.898728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:22:18.467561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898802+0000","last_change":"2023-10-18T10:40:18.906643+0000","last_active":"2023-10-18T10:40:20.898802+0000","last_peered":"2023-10-18T10:40:20.898802+0000","last_clean":"2023-10-18T10:40:20.898802+0000","last_became_active":"2023-10-18T10:40:18.906394+0000","last_became_peered":"2023-10-18T10:40:18.906394+0000","last_unstale":"2023-10-18T10:40:20.898802+0000","last_undegraded":"2023-10-18T10:40:20.898802+0000","last_fullsized":"2023-10-18T10:40:20.898802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:08.817426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796546+0000","last_change":"2023-10-18T10:40:19.905171+0000","last_active":"2023-10-18T10:40:21.796546+0000","last_peered":"2023-10-18T10:40:21.796546+0000","last_clean":"2023-10-18T10:40:21.796546+0000","last_became_active":"2023-10-18T10:40:19.904893+0000","last_became_peered":"2023-10-18T10:40:19.904893+0000","last_unstale":"2023-10-18T10:40:21.796546+0000","last_undegraded":"2023-10-18T10:40:21.796546+0000","last_fullsized":"2023-10-18T10:40:21.796546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:47:36.295066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796800+0000","last_change":"2023-10-18T10:40:18.903207+0000","last_active":"2023-10-18T10:40:21.796800+0000","last_peered":"2023-10-18T10:40:21.796800+0000","last_clean":"2023-10-18T10:40:21.796800+0000","last_became_active":"2023-10-18T10:40:18.902656+0000","last_became_peered":"2023-10-18T10:40:18.902656+0000","last_unstale":"2023-10-18T10:40:21.796800+0000","last_undegraded":"2023-10-18T10:40:21.796800+0000","last_fullsized":"2023-10-18T10:40:21.796800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:20.431457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902422+0000","last_change":"2023-10-18T10:40:19.911858+0000","last_active":"2023-10-18T10:40:20.902422+0000","last_peered":"2023-10-18T10:40:20.902422+0000","last_clean":"2023-10-18T10:40:20.902422+0000","last_became_active":"2023-10-18T10:40:19.910571+0000","last_became_peered":"2023-10-18T10:40:19.910571+0000","last_unstale":"2023-10-18T10:40:20.902422+0000","last_undegraded":"2023-10-18T10:40:20.902422+0000","last_fullsized":"2023-10-18T10:40:20.902422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:30.689854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898655+0000","last_change":"2023-10-18T10:40:18.899875+0000","last_active":"2023-10-18T10:40:20.898655+0000","last_peered":"2023-10-18T10:40:20.898655+0000","last_clean":"2023-10-18T10:40:20.898655+0000","last_became_active":"2023-10-18T10:40:18.899619+0000","last_became_peered":"2023-10-18T10:40:18.899619+0000","last_unstale":"2023-10-18T10:40:20.898655+0000","last_undegraded":"2023-10-18T10:40:20.898655+0000","last_fullsized":"2023-10-18T10:40:20.898655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:54.300307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898893+0000","last_change":"2023-10-18T10:40:19.919436+0000","last_active":"2023-10-18T10:40:20.898893+0000","last_peered":"2023-10-18T10:40:20.898893+0000","last_clean":"2023-10-18T10:40:20.898893+0000","last_became_active":"2023-10-18T10:40:19.919181+0000","last_became_peered":"2023-10-18T10:40:19.919181+0000","last_unstale":"2023-10-18T10:40:20.898893+0000","last_undegraded":"2023-10-18T10:40:20.898893+0000","last_fullsized":"2023-10-18T10:40:20.898893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:35.681517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.797039+0000","last_change":"2023-10-18T10:40:18.906951+0000","last_active":"2023-10-18T10:40:21.797039+0000","last_peered":"2023-10-18T10:40:21.797039+0000","last_clean":"2023-10-18T10:40:21.797039+0000","last_became_active":"2023-10-18T10:40:18.906455+0000","last_became_peered":"2023-10-18T10:40:18.906455+0000","last_unstale":"2023-10-18T10:40:21.797039+0000","last_undegraded":"2023-10-18T10:40:21.797039+0000","last_fullsized":"2023-10-18T10:40:21.797039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:28.391862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910555+0000","last_change":"2023-10-18T10:40:19.906881+0000","last_active":"2023-10-18T10:40:19.910555+0000","last_peered":"2023-10-18T10:40:19.910555+0000","last_clean":"2023-10-18T10:40:19.910555+0000","last_became_active":"2023-10-18T10:40:19.906627+0000","last_became_peered":"2023-10-18T10:40:19.906627+0000","last_unstale":"2023-10-18T10:40:19.910555+0000","last_undegraded":"2023-10-18T10:40:19.910555+0000","last_fullsized":"2023-10-18T10:40:19.910555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:33.017257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796915+0000","last_change":"2023-10-18T10:40:18.904182+0000","last_active":"2023-10-18T10:40:21.796915+0000","last_peered":"2023-10-18T10:40:21.796915+0000","last_clean":"2023-10-18T10:40:21.796915+0000","last_became_active":"2023-10-18T10:40:18.903869+0000","last_became_peered":"2023-10-18T10:40:18.903869+0000","last_unstale":"2023-10-18T10:40:21.796915+0000","last_undegraded":"2023-10-18T10:40:21.796915+0000","last_fullsized":"2023-10-18T10:40:21.796915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:05.894872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796812+0000","last_change":"2023-10-18T10:40:19.909239+0000","last_active":"2023-10-18T10:40:21.796812+0000","last_peered":"2023-10-18T10:40:21.796812+0000","last_clean":"2023-10-18T10:40:21.796812+0000","last_became_active":"2023-10-18T10:40:19.908949+0000","last_became_peered":"2023-10-18T10:40:19.908949+0000","last_unstale":"2023-10-18T10:40:21.796812+0000","last_undegraded":"2023-10-18T10:40:21.796812+0000","last_fullsized":"2023-10-18T10:40:21.796812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:44.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898995+0000","last_change":"2023-10-18T10:40:18.904341+0000","last_active":"2023-10-18T10:40:20.898995+0000","last_peered":"2023-10-18T10:40:20.898995+0000","last_clean":"2023-10-18T10:40:20.898995+0000","last_became_active":"2023-10-18T10:40:18.904111+0000","last_became_peered":"2023-10-18T10:40:18.904111+0000","last_unstale":"2023-10-18T10:40:20.898995+0000","last_undegraded":"2023-10-18T10:40:20.898995+0000","last_fullsized":"2023-10-18T10:40:20.898995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:43.542327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902850+0000","last_change":"2023-10-18T10:40:19.912475+0000","last_active":"2023-10-18T10:40:20.902850+0000","last_peered":"2023-10-18T10:40:20.902850+0000","last_clean":"2023-10-18T10:40:20.902850+0000","last_became_active":"2023-10-18T10:40:19.912112+0000","last_became_peered":"2023-10-18T10:40:19.912112+0000","last_unstale":"2023-10-18T10:40:20.902850+0000","last_undegraded":"2023-10-18T10:40:20.902850+0000","last_fullsized":"2023-10-18T10:40:20.902850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:09:51.716263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.902230+0000","last_change":"2023-10-18T10:40:18.905086+0000","last_active":"2023-10-18T10:40:19.902230+0000","last_peered":"2023-10-18T10:40:19.902230+0000","last_clean":"2023-10-18T10:40:19.902230+0000","last_became_active":"2023-10-18T10:40:18.904715+0000","last_became_peered":"2023-10-18T10:40:18.904715+0000","last_unstale":"2023-10-18T10:40:19.902230+0000","last_undegraded":"2023-10-18T10:40:19.902230+0000","last_fullsized":"2023-10-18T10:40:19.902230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:44.769336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902202+0000","last_change":"2023-10-18T10:40:19.911753+0000","last_active":"2023-10-18T10:40:20.902202+0000","last_peered":"2023-10-18T10:40:20.902202+0000","last_clean":"2023-10-18T10:40:20.902202+0000","last_became_active":"2023-10-18T10:40:19.911511+0000","last_became_peered":"2023-10-18T10:40:19.911511+0000","last_unstale":"2023-10-18T10:40:20.902202+0000","last_undegraded":"2023-10-18T10:40:20.902202+0000","last_fullsized":"2023-10-18T10:40:20.902202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:31:07.900837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506412+0000","last_change":"2023-10-18T10:40:18.902448+0000","last_active":"2023-10-18T10:40:21.506412+0000","last_peered":"2023-10-18T10:40:21.506412+0000","last_clean":"2023-10-18T10:40:21.506412+0000","last_became_active":"2023-10-18T10:40:18.902196+0000","last_became_peered":"2023-10-18T10:40:18.902196+0000","last_unstale":"2023-10-18T10:40:21.506412+0000","last_undegraded":"2023-10-18T10:40:21.506412+0000","last_fullsized":"2023-10-18T10:40:21.506412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:26.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.920063+0000","last_change":"2023-10-18T10:40:19.914096+0000","last_active":"2023-10-18T10:40:19.920063+0000","last_peered":"2023-10-18T10:40:19.920063+0000","last_clean":"2023-10-18T10:40:19.920063+0000","last_became_active":"2023-10-18T10:40:19.913844+0000","last_became_peered":"2023-10-18T10:40:19.913844+0000","last_unstale":"2023-10-18T10:40:19.920063+0000","last_undegraded":"2023-10-18T10:40:19.920063+0000","last_fullsized":"2023-10-18T10:40:19.920063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:21.171698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897010+0000","last_change":"2023-10-18T10:40:18.901223+0000","last_active":"2023-10-18T10:40:19.897010+0000","last_peered":"2023-10-18T10:40:19.897010+0000","last_clean":"2023-10-18T10:40:19.897010+0000","last_became_active":"2023-10-18T10:40:18.900638+0000","last_became_peered":"2023-10-18T10:40:18.900638+0000","last_unstale":"2023-10-18T10:40:19.897010+0000","last_undegraded":"2023-10-18T10:40:19.897010+0000","last_fullsized":"2023-10-18T10:40:19.897010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:04.547290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926391+0000","last_change":"2023-10-18T10:40:19.925927+0000","last_active":"2023-10-18T10:40:19.926391+0000","last_peered":"2023-10-18T10:40:19.926391+0000","last_clean":"2023-10-18T10:40:19.926391+0000","last_became_active":"2023-10-18T10:40:19.925668+0000","last_became_peered":"2023-10-18T10:40:19.925668+0000","last_unstale":"2023-10-18T10:40:19.926391+0000","last_undegraded":"2023-10-18T10:40:19.926391+0000","last_fullsized":"2023-10-18T10:40:19.926391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:09.139441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897265+0000","last_change":"2023-10-18T10:40:18.904336+0000","last_active":"2023-10-18T10:40:19.897265+0000","last_peered":"2023-10-18T10:40:19.897265+0000","last_clean":"2023-10-18T10:40:19.897265+0000","last_became_active":"2023-10-18T10:40:18.904085+0000","last_became_peered":"2023-10-18T10:40:18.904085+0000","last_unstale":"2023-10-18T10:40:19.897265+0000","last_undegraded":"2023-10-18T10:40:19.897265+0000","last_fullsized":"2023-10-18T10:40:19.897265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:32.124110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910680+0000","last_change":"2023-10-18T10:40:19.910079+0000","last_active":"2023-10-18T10:40:19.910680+0000","last_peered":"2023-10-18T10:40:19.910680+0000","last_clean":"2023-10-18T10:40:19.910680+0000","last_became_active":"2023-10-18T10:40:19.909833+0000","last_became_peered":"2023-10-18T10:40:19.909833+0000","last_unstale":"2023-10-18T10:40:19.910680+0000","last_undegraded":"2023-10-18T10:40:19.910680+0000","last_fullsized":"2023-10-18T10:40:19.910680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:43:23.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.900343+0000","last_change":"2023-10-18T10:40:18.902461+0000","last_active":"2023-10-18T10:40:19.900343+0000","last_peered":"2023-10-18T10:40:19.900343+0000","last_clean":"2023-10-18T10:40:19.900343+0000","last_became_active":"2023-10-18T10:40:18.902198+0000","last_became_peered":"2023-10-18T10:40:18.902198+0000","last_unstale":"2023-10-18T10:40:19.900343+0000","last_undegraded":"2023-10-18T10:40:19.900343+0000","last_fullsized":"2023-10-18T10:40:19.900343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:18.205683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902492+0000","last_change":"2023-10-18T10:40:19.915806+0000","last_active":"2023-10-18T10:40:20.902492+0000","last_peered":"2023-10-18T10:40:20.902492+0000","last_clean":"2023-10-18T10:40:20.902492+0000","last_became_active":"2023-10-18T10:40:19.915465+0000","last_became_peered":"2023-10-18T10:40:19.915465+0000","last_unstale":"2023-10-18T10:40:20.902492+0000","last_undegraded":"2023-10-18T10:40:20.902492+0000","last_fullsized":"2023-10-18T10:40:20.902492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:45.206294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896765+0000","last_change":"2023-10-18T10:40:18.901946+0000","last_active":"2023-10-18T10:40:19.896765+0000","last_peered":"2023-10-18T10:40:19.896765+0000","last_clean":"2023-10-18T10:40:19.896765+0000","last_became_active":"2023-10-18T10:40:18.901694+0000","last_became_peered":"2023-10-18T10:40:18.901694+0000","last_unstale":"2023-10-18T10:40:19.896765+0000","last_undegraded":"2023-10-18T10:40:19.896765+0000","last_fullsized":"2023-10-18T10:40:19.896765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:08.817426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'161","reported_seq":383,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.898971+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:19.898971+0000","last_peered":"2023-10-18T10:40:19.898971+0000","last_clean":"2023-10-18T10:40:19.898971+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:19.898971+0000","last_undegraded":"2023-10-18T10:40:19.898971+0000","last_fullsized":"2023-10-18T10:40:19.898971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575037,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59136,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312704,"statfs":{"total":96636764160,"available":96576208896,"internally_reserved":0,"allocated":1617920,"data_stored":1091181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.69199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1024000,"data_stored":506691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.436},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":1.016},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.80300000000000005,"15min":0.875},"last":0.37}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.872,"15min":0.872},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.54200000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66800000000000004,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.496}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":0.82199999999999995,"15min":0.87},"last":0.56299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834575037,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":500059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.47699999999999998,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.016},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.624}]},{"osd":1,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.46200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.76200000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.749}]},{"osd":2,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.48199999999999998,"5min":0.377,"15min":0.28599999999999998},"max":{"1min":0.746,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.51600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.49399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.57099999999999995,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.81000000000000005,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.72299999999999998,"5min":0.80100000000000005,"15min":1.262},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":1.0109999999999999,"15min":1.028},"last":0.33600000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72999999999999998,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":0.96099999999999997,"15min":1.224},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.73699999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.495},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.871,"15min":1.159},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575037,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":501649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.877,"15min":0.877},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51100000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.502,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.72099999999999997,"5min":0.746,"15min":0.79600000000000004},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75600000000000001,"5min":1.446,"15min":1.446},"last":0.495},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.90400000000000003},"last":0.376}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.82699999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.41999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":500037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54376,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317464,"statfs":{"total":96636764160,"available":96581083136,"internally_reserved":0,"allocated":1007616,"data_stored":500037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.313,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.52300000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.32700000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.39500000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.378},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40300000000000002},"min":{"1min":0.34100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60999999999999999,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.377,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.88200000000000001,"5min":0.88800000000000001,"15min":1.458},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.52600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.95699999999999996,"15min":0.997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.93700000000000006,"15min":1.028},"last":0.748}]},{"osd":7,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.63300000000000001},"min":{"1min":0.51000000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.88900000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.52500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84599999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"last":0.63200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":502413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.371},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.434}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.505}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1601536,"data_stored":1090405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.316}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.371},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.307}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41399999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.40500000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.438},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.45700000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.46700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:24.882 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T10:40:24.881+0000 7ff1dfb6f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:40:27.276 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:27.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.389+0000 7f331a98e640 1 -- 172.21.15.125:0/142501802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 msgr2=0x7f33140d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.389+0000 7f331a98e640 1 --2- 172.21.15.125:0/142501802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 0x7f33140d1910 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3304005fd0 tx=0x7f330402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.389+0000 7f331a98e640 1 -- 172.21.15.125:0/142501802 shutdown_connections 2023-10-18T10:40:27.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.389+0000 7f331a98e640 1 --2- 172.21.15.125:0/142501802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f331415b2a0 0x7f33140cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.389+0000 7f331a98e640 1 --2- 172.21.15.125:0/142501802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 0x7f33140d1910 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.389+0000 7f331a98e640 1 -- 172.21.15.125:0/142501802 >> 172.21.15.125:0/142501802 conn(0x7f33140d0d60 msgr2=0x7f33140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:27.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.389+0000 7f331a98e640 1 -- 172.21.15.125:0/142501802 shutdown_connections 2023-10-18T10:40:27.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.389+0000 7f331a98e640 1 -- 172.21.15.125:0/142501802 wait complete. 2023-10-18T10:40:27.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.390+0000 7f331a98e640 1 Processor -- start 2023-10-18T10:40:27.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f331a98e640 1 -- start start 2023-10-18T10:40:27.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f331a98e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33140ceb40 0x7f331415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:27.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f331a98e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 0x7f33141f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:27.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f331a98e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f331415b2a0 0x7f33141f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:27.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f331a98e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33141f2960 con 0x7f33140cf340 2023-10-18T10:40:27.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f331a98e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33141f1e90 con 0x7f33140ceb40 2023-10-18T10:40:27.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f331a98e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33140b3b10 con 0x7f331415b2a0 2023-10-18T10:40:27.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f33137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 0x7f33141f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:27.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f33137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 0x7f33141f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40764/0 (socket says 172.21.15.125:40764) 2023-10-18T10:40:27.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.391+0000 7f33137fe640 1 -- 172.21.15.125:0/970280027 learned_addr learned my addr 172.21.15.125:0/970280027 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:27.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f3313fff640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33140ceb40 0x7f331415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:27.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f3318f2b640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f331415b2a0 0x7f33141f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:27.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f33137fe640 1 -- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f331415b2a0 msgr2=0x7f33141f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f33137fe640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f331415b2a0 0x7f33141f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f33137fe640 1 -- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33140ceb40 msgr2=0x7f331415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f33137fe640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33140ceb40 0x7f331415e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f33137fe640 1 -- 172.21.15.125:0/970280027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304009560 con 0x7f33140cf340 2023-10-18T10:40:27.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f33137fe640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 0x7f33141f1950 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f330800d700 tx=0x7f330800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:27.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f3310ff9640 1 -- 172.21.15.125:0/970280027 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308005220 con 0x7f33140cf340 2023-10-18T10:40:27.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f3310ff9640 1 -- 172.21.15.125:0/970280027 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3308005470 con 0x7f33140cf340 2023-10-18T10:40:27.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f3310ff9640 1 -- 172.21.15.125:0/970280027 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308009a80 con 0x7f33140cf340 2023-10-18T10:40:27.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33141f20c0 con 0x7f33140cf340 2023-10-18T10:40:27.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.392+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33141f2680 con 0x7f33140cf340 2023-10-18T10:40:27.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.393+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33140690e0 con 0x7f33140cf340 2023-10-18T10:40:27.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.394+0000 7f3310ff9640 1 -- 172.21.15.125:0/970280027 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3308009be0 con 0x7f33140cf340 2023-10-18T10:40:27.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.394+0000 7f3310ff9640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f330006af60 0x7f330006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:27.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.394+0000 7f3310ff9640 1 -- 172.21.15.125:0/970280027 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3308089ac0 con 0x7f33140cf340 2023-10-18T10:40:27.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.394+0000 7f3313fff640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f330006af60 0x7f330006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:27.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.395+0000 7f3313fff640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f330006af60 0x7f330006d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3304035000 tx=0x7f3304004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:27.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.398+0000 7f3310ff9640 1 -- 172.21.15.125:0/970280027 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33080548c0 con 0x7f33140cf340 2023-10-18T10:40:27.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.561+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 --> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33140cc170 con 0x7f330006af60 2023-10-18T10:40:27.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.569+0000 7f3310ff9640 1 -- 172.21.15.125:0/970280027 <== mgr.6055 v2:172.21.15.139:6832/2642591764 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289432 (secure 0 0 0) 0x7f3304004210 con 0x7f330006af60 2023-10-18T10:40:27.574 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:27.579 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:27.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f330006af60 msgr2=0x7f330006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f330006af60 0x7f330006d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3304035000 tx=0x7f3304004730 comp rx=0 tx=0).stop 2023-10-18T10:40:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 msgr2=0x7f33141f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 0x7f33141f1950 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f330800d700 tx=0x7f330800dbc0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 shutdown_connections 2023-10-18T10:40:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f330006af60 0x7f330006d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f331415b2a0 0x7f33141f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33140cf340 0x7f33141f1950 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 --2- 172.21.15.125:0/970280027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33140ceb40 0x7f331415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 >> 172.21.15.125:0/970280027 conn(0x7f33140d0d60 msgr2=0x7f331415a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:27.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 shutdown_connections 2023-10-18T10:40:27.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.572+0000 7f331a98e640 1 -- 172.21.15.125:0/970280027 wait complete. 2023-10-18T10:40:27.585 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-18T10:40:25.797167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":303008,"kb_used_data":9108,"kb_used_omap":12,"kb_used_meta":293747,"kb_avail":754671712,"statfs":{"total":773094113280,"available":772783833088,"internally_reserved":0,"allocated":9326592,"data_stored":5192472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":300797496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.909732"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919741+0000","last_change":"2023-10-18T10:40:19.912867+0000","last_active":"2023-10-18T10:40:19.919741+0000","last_peered":"2023-10-18T10:40:19.919741+0000","last_clean":"2023-10-18T10:40:19.919741+0000","last_became_active":"2023-10-18T10:40:19.912618+0000","last_became_peered":"2023-10-18T10:40:19.912618+0000","last_unstale":"2023-10-18T10:40:19.919741+0000","last_undegraded":"2023-10-18T10:40:19.919741+0000","last_fullsized":"2023-10-18T10:40:19.919741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:40:18.272296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917449+0000","last_change":"2023-10-18T10:40:19.912063+0000","last_active":"2023-10-18T10:40:19.917449+0000","last_peered":"2023-10-18T10:40:19.917449+0000","last_clean":"2023-10-18T10:40:19.917449+0000","last_became_active":"2023-10-18T10:40:19.911766+0000","last_became_peered":"2023-10-18T10:40:19.911766+0000","last_unstale":"2023-10-18T10:40:19.917449+0000","last_undegraded":"2023-10-18T10:40:19.917449+0000","last_fullsized":"2023-10-18T10:40:19.917449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:14.456342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506537+0000","last_change":"2023-10-18T10:40:19.915439+0000","last_active":"2023-10-18T10:40:21.506537+0000","last_peered":"2023-10-18T10:40:21.506537+0000","last_clean":"2023-10-18T10:40:21.506537+0000","last_became_active":"2023-10-18T10:40:19.906290+0000","last_became_peered":"2023-10-18T10:40:19.906290+0000","last_unstale":"2023-10-18T10:40:21.506537+0000","last_undegraded":"2023-10-18T10:40:21.506537+0000","last_fullsized":"2023-10-18T10:40:21.506537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:05.552436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.925151+0000","last_change":"2023-10-18T10:40:19.919169+0000","last_active":"2023-10-18T10:40:19.925151+0000","last_peered":"2023-10-18T10:40:19.925151+0000","last_clean":"2023-10-18T10:40:19.925151+0000","last_became_active":"2023-10-18T10:40:19.918843+0000","last_became_peered":"2023-10-18T10:40:19.918843+0000","last_unstale":"2023-10-18T10:40:19.925151+0000","last_undegraded":"2023-10-18T10:40:19.925151+0000","last_fullsized":"2023-10-18T10:40:19.925151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:35:43.608938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902310+0000","last_change":"2023-10-18T10:40:19.918784+0000","last_active":"2023-10-18T10:40:20.902310+0000","last_peered":"2023-10-18T10:40:20.902310+0000","last_clean":"2023-10-18T10:40:20.902310+0000","last_became_active":"2023-10-18T10:40:19.918505+0000","last_became_peered":"2023-10-18T10:40:19.918505+0000","last_unstale":"2023-10-18T10:40:20.902310+0000","last_undegraded":"2023-10-18T10:40:20.902310+0000","last_fullsized":"2023-10-18T10:40:20.902310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:07:25.728867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898822+0000","last_change":"2023-10-18T10:40:19.909046+0000","last_active":"2023-10-18T10:40:20.898822+0000","last_peered":"2023-10-18T10:40:20.898822+0000","last_clean":"2023-10-18T10:40:20.898822+0000","last_became_active":"2023-10-18T10:40:19.908798+0000","last_became_peered":"2023-10-18T10:40:19.908798+0000","last_unstale":"2023-10-18T10:40:20.898822+0000","last_undegraded":"2023-10-18T10:40:20.898822+0000","last_fullsized":"2023-10-18T10:40:20.898822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:45:33.551804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.903510+0000","last_change":"2023-10-18T10:40:19.902502+0000","last_active":"2023-10-18T10:40:19.903510+0000","last_peered":"2023-10-18T10:40:19.903510+0000","last_clean":"2023-10-18T10:40:19.903510+0000","last_became_active":"2023-10-18T10:40:19.902241+0000","last_became_peered":"2023-10-18T10:40:19.902241+0000","last_unstale":"2023-10-18T10:40:19.903510+0000","last_undegraded":"2023-10-18T10:40:19.903510+0000","last_fullsized":"2023-10-18T10:40:19.903510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:37.704391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.797019+0000","last_change":"2023-10-18T10:40:19.912259+0000","last_active":"2023-10-18T10:40:21.797019+0000","last_peered":"2023-10-18T10:40:21.797019+0000","last_clean":"2023-10-18T10:40:21.797019+0000","last_became_active":"2023-10-18T10:40:19.911998+0000","last_became_peered":"2023-10-18T10:40:19.911998+0000","last_unstale":"2023-10-18T10:40:21.797019+0000","last_undegraded":"2023-10-18T10:40:21.797019+0000","last_fullsized":"2023-10-18T10:40:21.797019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:01.893970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506791+0000","last_change":"2023-10-18T10:40:19.920485+0000","last_active":"2023-10-18T10:40:21.506791+0000","last_peered":"2023-10-18T10:40:21.506791+0000","last_clean":"2023-10-18T10:40:21.506791+0000","last_became_active":"2023-10-18T10:40:19.920181+0000","last_became_peered":"2023-10-18T10:40:19.920181+0000","last_unstale":"2023-10-18T10:40:21.506791+0000","last_undegraded":"2023-10-18T10:40:21.506791+0000","last_fullsized":"2023-10-18T10:40:21.506791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:10.594385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506580+0000","last_change":"2023-10-18T10:40:19.915961+0000","last_active":"2023-10-18T10:40:21.506580+0000","last_peered":"2023-10-18T10:40:21.506580+0000","last_clean":"2023-10-18T10:40:21.506580+0000","last_became_active":"2023-10-18T10:40:19.915740+0000","last_became_peered":"2023-10-18T10:40:19.915740+0000","last_unstale":"2023-10-18T10:40:21.506580+0000","last_undegraded":"2023-10-18T10:40:21.506580+0000","last_fullsized":"2023-10-18T10:40:21.506580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:43.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.925217+0000","last_change":"2023-10-18T10:40:19.919420+0000","last_active":"2023-10-18T10:40:19.925217+0000","last_peered":"2023-10-18T10:40:19.925217+0000","last_clean":"2023-10-18T10:40:19.925217+0000","last_became_active":"2023-10-18T10:40:19.918484+0000","last_became_peered":"2023-10-18T10:40:19.918484+0000","last_unstale":"2023-10-18T10:40:19.925217+0000","last_undegraded":"2023-10-18T10:40:19.925217+0000","last_fullsized":"2023-10-18T10:40:19.925217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:05.833363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796759+0000","last_change":"2023-10-18T10:40:19.907774+0000","last_active":"2023-10-18T10:40:21.796759+0000","last_peered":"2023-10-18T10:40:21.796759+0000","last_clean":"2023-10-18T10:40:21.796759+0000","last_became_active":"2023-10-18T10:40:19.907511+0000","last_became_peered":"2023-10-18T10:40:19.907511+0000","last_unstale":"2023-10-18T10:40:21.796759+0000","last_undegraded":"2023-10-18T10:40:21.796759+0000","last_fullsized":"2023-10-18T10:40:21.796759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:19.230040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796778+0000","last_change":"2023-10-18T10:40:19.918526+0000","last_active":"2023-10-18T10:40:21.796778+0000","last_peered":"2023-10-18T10:40:21.796778+0000","last_clean":"2023-10-18T10:40:21.796778+0000","last_became_active":"2023-10-18T10:40:19.918271+0000","last_became_peered":"2023-10-18T10:40:19.918271+0000","last_unstale":"2023-10-18T10:40:21.796778+0000","last_undegraded":"2023-10-18T10:40:21.796778+0000","last_fullsized":"2023-10-18T10:40:21.796778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:47.354614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506233+0000","last_change":"2023-10-18T10:40:19.905868+0000","last_active":"2023-10-18T10:40:21.506233+0000","last_peered":"2023-10-18T10:40:21.506233+0000","last_clean":"2023-10-18T10:40:21.506233+0000","last_became_active":"2023-10-18T10:40:19.905593+0000","last_became_peered":"2023-10-18T10:40:19.905593+0000","last_unstale":"2023-10-18T10:40:21.506233+0000","last_undegraded":"2023-10-18T10:40:21.506233+0000","last_fullsized":"2023-10-18T10:40:21.506233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:55.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506063+0000","last_change":"2023-10-18T10:40:19.905714+0000","last_active":"2023-10-18T10:40:21.506063+0000","last_peered":"2023-10-18T10:40:21.506063+0000","last_clean":"2023-10-18T10:40:21.506063+0000","last_became_active":"2023-10-18T10:40:19.905464+0000","last_became_peered":"2023-10-18T10:40:19.905464+0000","last_unstale":"2023-10-18T10:40:21.506063+0000","last_undegraded":"2023-10-18T10:40:21.506063+0000","last_fullsized":"2023-10-18T10:40:21.506063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:01.893970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.908797+0000","last_change":"2023-10-18T10:40:19.907404+0000","last_active":"2023-10-18T10:40:19.908797+0000","last_peered":"2023-10-18T10:40:19.908797+0000","last_clean":"2023-10-18T10:40:19.908797+0000","last_became_active":"2023-10-18T10:40:19.907152+0000","last_became_peered":"2023-10-18T10:40:19.907152+0000","last_unstale":"2023-10-18T10:40:19.908797+0000","last_undegraded":"2023-10-18T10:40:19.908797+0000","last_fullsized":"2023-10-18T10:40:19.908797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:20.332693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.921101+0000","last_change":"2023-10-18T10:40:19.919930+0000","last_active":"2023-10-18T10:40:19.921101+0000","last_peered":"2023-10-18T10:40:19.921101+0000","last_clean":"2023-10-18T10:40:19.921101+0000","last_became_active":"2023-10-18T10:40:19.919664+0000","last_became_peered":"2023-10-18T10:40:19.919664+0000","last_unstale":"2023-10-18T10:40:19.921101+0000","last_undegraded":"2023-10-18T10:40:19.921101+0000","last_fullsized":"2023-10-18T10:40:19.921101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:04.430250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926304+0000","last_change":"2023-10-18T10:40:19.923065+0000","last_active":"2023-10-18T10:40:19.926304+0000","last_peered":"2023-10-18T10:40:19.926304+0000","last_clean":"2023-10-18T10:40:19.926304+0000","last_became_active":"2023-10-18T10:40:19.921641+0000","last_became_peered":"2023-10-18T10:40:19.921641+0000","last_unstale":"2023-10-18T10:40:19.926304+0000","last_undegraded":"2023-10-18T10:40:19.926304+0000","last_fullsized":"2023-10-18T10:40:19.926304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:31:07.900837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902717+0000","last_change":"2023-10-18T10:40:19.912926+0000","last_active":"2023-10-18T10:40:20.902717+0000","last_peered":"2023-10-18T10:40:20.902717+0000","last_clean":"2023-10-18T10:40:20.902717+0000","last_became_active":"2023-10-18T10:40:19.912676+0000","last_became_peered":"2023-10-18T10:40:19.912676+0000","last_unstale":"2023-10-18T10:40:20.902717+0000","last_undegraded":"2023-10-18T10:40:20.902717+0000","last_fullsized":"2023-10-18T10:40:20.902717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:51:34.217906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910585+0000","last_change":"2023-10-18T10:40:19.909713+0000","last_active":"2023-10-18T10:40:19.910585+0000","last_peered":"2023-10-18T10:40:19.910585+0000","last_clean":"2023-10-18T10:40:19.910585+0000","last_became_active":"2023-10-18T10:40:19.909464+0000","last_became_peered":"2023-10-18T10:40:19.909464+0000","last_unstale":"2023-10-18T10:40:19.910585+0000","last_undegraded":"2023-10-18T10:40:19.910585+0000","last_fullsized":"2023-10-18T10:40:19.910585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:55.604568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796869+0000","last_change":"2023-10-18T10:40:19.909983+0000","last_active":"2023-10-18T10:40:21.796869+0000","last_peered":"2023-10-18T10:40:21.796869+0000","last_clean":"2023-10-18T10:40:21.796869+0000","last_became_active":"2023-10-18T10:40:19.909687+0000","last_became_peered":"2023-10-18T10:40:19.909687+0000","last_unstale":"2023-10-18T10:40:21.796869+0000","last_undegraded":"2023-10-18T10:40:21.796869+0000","last_fullsized":"2023-10-18T10:40:21.796869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:09.822572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.904650+0000","last_change":"2023-10-18T10:40:19.902583+0000","last_active":"2023-10-18T10:40:19.904650+0000","last_peered":"2023-10-18T10:40:19.904650+0000","last_clean":"2023-10-18T10:40:19.904650+0000","last_became_active":"2023-10-18T10:40:19.902167+0000","last_became_peered":"2023-10-18T10:40:19.902167+0000","last_unstale":"2023-10-18T10:40:19.904650+0000","last_undegraded":"2023-10-18T10:40:19.904650+0000","last_fullsized":"2023-10-18T10:40:19.904650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:10.594385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.903183+0000","last_change":"2023-10-18T10:40:19.902535+0000","last_active":"2023-10-18T10:40:19.903183+0000","last_peered":"2023-10-18T10:40:19.903183+0000","last_clean":"2023-10-18T10:40:19.903183+0000","last_became_active":"2023-10-18T10:40:19.902248+0000","last_became_peered":"2023-10-18T10:40:19.902248+0000","last_unstale":"2023-10-18T10:40:19.903183+0000","last_undegraded":"2023-10-18T10:40:19.903183+0000","last_fullsized":"2023-10-18T10:40:19.903183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:56.313746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506047+0000","last_change":"2023-10-18T10:40:19.907648+0000","last_active":"2023-10-18T10:40:21.506047+0000","last_peered":"2023-10-18T10:40:21.506047+0000","last_clean":"2023-10-18T10:40:21.506047+0000","last_became_active":"2023-10-18T10:40:19.906778+0000","last_became_peered":"2023-10-18T10:40:19.906778+0000","last_unstale":"2023-10-18T10:40:21.506047+0000","last_undegraded":"2023-10-18T10:40:21.506047+0000","last_fullsized":"2023-10-18T10:40:21.506047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:09.822572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796772+0000","last_change":"2023-10-18T10:40:19.912919+0000","last_active":"2023-10-18T10:40:21.796772+0000","last_peered":"2023-10-18T10:40:21.796772+0000","last_clean":"2023-10-18T10:40:21.796772+0000","last_became_active":"2023-10-18T10:40:19.912669+0000","last_became_peered":"2023-10-18T10:40:19.912669+0000","last_unstale":"2023-10-18T10:40:21.796772+0000","last_undegraded":"2023-10-18T10:40:21.796772+0000","last_fullsized":"2023-10-18T10:40:21.796772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:08.877366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796691+0000","last_change":"2023-10-18T10:40:19.908565+0000","last_active":"2023-10-18T10:40:21.796691+0000","last_peered":"2023-10-18T10:40:21.796691+0000","last_clean":"2023-10-18T10:40:21.796691+0000","last_became_active":"2023-10-18T10:40:19.908239+0000","last_became_peered":"2023-10-18T10:40:19.908239+0000","last_unstale":"2023-10-18T10:40:21.796691+0000","last_undegraded":"2023-10-18T10:40:21.796691+0000","last_fullsized":"2023-10-18T10:40:21.796691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:09.515116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.920159+0000","last_change":"2023-10-18T10:40:19.917470+0000","last_active":"2023-10-18T10:40:19.920159+0000","last_peered":"2023-10-18T10:40:19.920159+0000","last_clean":"2023-10-18T10:40:19.920159+0000","last_became_active":"2023-10-18T10:40:19.917044+0000","last_became_peered":"2023-10-18T10:40:19.917044+0000","last_unstale":"2023-10-18T10:40:19.920159+0000","last_undegraded":"2023-10-18T10:40:19.920159+0000","last_fullsized":"2023-10-18T10:40:19.920159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:21.627876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917829+0000","last_change":"2023-10-18T10:40:19.910004+0000","last_active":"2023-10-18T10:40:19.917829+0000","last_peered":"2023-10-18T10:40:19.917829+0000","last_clean":"2023-10-18T10:40:19.917829+0000","last_became_active":"2023-10-18T10:40:19.909737+0000","last_became_peered":"2023-10-18T10:40:19.909737+0000","last_unstale":"2023-10-18T10:40:19.917829+0000","last_undegraded":"2023-10-18T10:40:19.917829+0000","last_fullsized":"2023-10-18T10:40:19.917829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:36.377245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506257+0000","last_change":"2023-10-18T10:40:19.921023+0000","last_active":"2023-10-18T10:40:21.506257+0000","last_peered":"2023-10-18T10:40:21.506257+0000","last_clean":"2023-10-18T10:40:21.506257+0000","last_became_active":"2023-10-18T10:40:19.919546+0000","last_became_peered":"2023-10-18T10:40:19.919546+0000","last_unstale":"2023-10-18T10:40:21.506257+0000","last_undegraded":"2023-10-18T10:40:21.506257+0000","last_fullsized":"2023-10-18T10:40:21.506257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:56.313746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796648+0000","last_change":"2023-10-18T10:40:19.907166+0000","last_active":"2023-10-18T10:40:21.796648+0000","last_peered":"2023-10-18T10:40:21.796648+0000","last_clean":"2023-10-18T10:40:21.796648+0000","last_became_active":"2023-10-18T10:40:19.906900+0000","last_became_peered":"2023-10-18T10:40:19.906900+0000","last_unstale":"2023-10-18T10:40:21.796648+0000","last_undegraded":"2023-10-18T10:40:21.796648+0000","last_fullsized":"2023-10-18T10:40:21.796648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:51.778136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.920080+0000","last_change":"2023-10-18T10:40:19.917537+0000","last_active":"2023-10-18T10:40:19.920080+0000","last_peered":"2023-10-18T10:40:19.920080+0000","last_clean":"2023-10-18T10:40:19.920080+0000","last_became_active":"2023-10-18T10:40:19.916919+0000","last_became_peered":"2023-10-18T10:40:19.916919+0000","last_unstale":"2023-10-18T10:40:19.920080+0000","last_undegraded":"2023-10-18T10:40:19.920080+0000","last_fullsized":"2023-10-18T10:40:19.920080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:41.408790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898663+0000","last_change":"2023-10-18T10:40:19.915799+0000","last_active":"2023-10-18T10:40:20.898663+0000","last_peered":"2023-10-18T10:40:20.898663+0000","last_clean":"2023-10-18T10:40:20.898663+0000","last_became_active":"2023-10-18T10:40:19.915535+0000","last_became_peered":"2023-10-18T10:40:19.915535+0000","last_unstale":"2023-10-18T10:40:20.898663+0000","last_undegraded":"2023-10-18T10:40:20.898663+0000","last_fullsized":"2023-10-18T10:40:20.898663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:33.129256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902670+0000","last_change":"2023-10-18T10:40:19.910827+0000","last_active":"2023-10-18T10:40:20.902670+0000","last_peered":"2023-10-18T10:40:20.902670+0000","last_clean":"2023-10-18T10:40:20.902670+0000","last_became_active":"2023-10-18T10:40:19.910578+0000","last_became_peered":"2023-10-18T10:40:19.910578+0000","last_unstale":"2023-10-18T10:40:20.902670+0000","last_undegraded":"2023-10-18T10:40:20.902670+0000","last_fullsized":"2023-10-18T10:40:20.902670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:49.380981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"83'4","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924752+0000","last_change":"2023-10-18T10:40:18.906118+0000","last_active":"2023-10-18T10:40:19.924752+0000","last_peered":"2023-10-18T10:40:19.924752+0000","last_clean":"2023-10-18T10:40:19.924752+0000","last_became_active":"2023-10-18T10:40:18.905861+0000","last_became_peered":"2023-10-18T10:40:18.905861+0000","last_unstale":"2023-10-18T10:40:19.924752+0000","last_undegraded":"2023-10-18T10:40:19.924752+0000","last_fullsized":"2023-10-18T10:40:19.924752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:12.674141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926449+0000","last_change":"2023-10-18T10:40:19.924903+0000","last_active":"2023-10-18T10:40:19.926449+0000","last_peered":"2023-10-18T10:40:19.926449+0000","last_clean":"2023-10-18T10:40:19.926449+0000","last_became_active":"2023-10-18T10:40:19.924531+0000","last_became_peered":"2023-10-18T10:40:19.924531+0000","last_unstale":"2023-10-18T10:40:19.926449+0000","last_undegraded":"2023-10-18T10:40:19.926449+0000","last_fullsized":"2023-10-18T10:40:19.926449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:46.773804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.899076+0000","last_change":"2023-10-18T10:40:18.899749+0000","last_active":"2023-10-18T10:40:20.899076+0000","last_peered":"2023-10-18T10:40:20.899076+0000","last_clean":"2023-10-18T10:40:20.899076+0000","last_became_active":"2023-10-18T10:40:18.899485+0000","last_became_peered":"2023-10-18T10:40:18.899485+0000","last_unstale":"2023-10-18T10:40:20.899076+0000","last_undegraded":"2023-10-18T10:40:20.899076+0000","last_fullsized":"2023-10-18T10:40:20.899076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:42.126241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.899018+0000","last_change":"2023-10-18T10:40:19.907847+0000","last_active":"2023-10-18T10:40:20.899018+0000","last_peered":"2023-10-18T10:40:20.899018+0000","last_clean":"2023-10-18T10:40:20.899018+0000","last_became_active":"2023-10-18T10:40:19.907296+0000","last_became_peered":"2023-10-18T10:40:19.907296+0000","last_unstale":"2023-10-18T10:40:20.899018+0000","last_undegraded":"2023-10-18T10:40:20.899018+0000","last_fullsized":"2023-10-18T10:40:20.899018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:24:07.576898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898947+0000","last_change":"2023-10-18T10:40:18.904926+0000","last_active":"2023-10-18T10:40:20.898947+0000","last_peered":"2023-10-18T10:40:20.898947+0000","last_clean":"2023-10-18T10:40:20.898947+0000","last_became_active":"2023-10-18T10:40:18.904685+0000","last_became_peered":"2023-10-18T10:40:18.904685+0000","last_unstale":"2023-10-18T10:40:20.898947+0000","last_undegraded":"2023-10-18T10:40:20.898947+0000","last_fullsized":"2023-10-18T10:40:20.898947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:04.547290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506691+0000","last_change":"2023-10-18T10:40:19.909120+0000","last_active":"2023-10-18T10:40:21.506691+0000","last_peered":"2023-10-18T10:40:21.506691+0000","last_clean":"2023-10-18T10:40:21.506691+0000","last_became_active":"2023-10-18T10:40:19.908791+0000","last_became_peered":"2023-10-18T10:40:19.908791+0000","last_unstale":"2023-10-18T10:40:21.506691+0000","last_undegraded":"2023-10-18T10:40:21.506691+0000","last_fullsized":"2023-10-18T10:40:21.506691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:44.547473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898954+0000","last_change":"2023-10-18T10:40:18.897418+0000","last_active":"2023-10-18T10:40:20.898954+0000","last_peered":"2023-10-18T10:40:20.898954+0000","last_clean":"2023-10-18T10:40:20.898954+0000","last_became_active":"2023-10-18T10:40:18.897166+0000","last_became_peered":"2023-10-18T10:40:18.897166+0000","last_unstale":"2023-10-18T10:40:20.898954+0000","last_undegraded":"2023-10-18T10:40:20.898954+0000","last_fullsized":"2023-10-18T10:40:20.898954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:55.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.909100+0000","last_change":"2023-10-18T10:40:19.908499+0000","last_active":"2023-10-18T10:40:19.909100+0000","last_peered":"2023-10-18T10:40:19.909100+0000","last_clean":"2023-10-18T10:40:19.909100+0000","last_became_active":"2023-10-18T10:40:19.907640+0000","last_became_peered":"2023-10-18T10:40:19.907640+0000","last_unstale":"2023-10-18T10:40:19.909100+0000","last_undegraded":"2023-10-18T10:40:19.909100+0000","last_fullsized":"2023-10-18T10:40:19.909100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:24.687875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922912+0000","last_change":"2023-10-18T10:40:18.901139+0000","last_active":"2023-10-18T10:40:19.922912+0000","last_peered":"2023-10-18T10:40:19.922912+0000","last_clean":"2023-10-18T10:40:19.922912+0000","last_became_active":"2023-10-18T10:40:18.900880+0000","last_became_peered":"2023-10-18T10:40:18.900880+0000","last_unstale":"2023-10-18T10:40:19.922912+0000","last_undegraded":"2023-10-18T10:40:19.922912+0000","last_fullsized":"2023-10-18T10:40:19.922912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:57.088016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.924051+0000","last_change":"2023-10-18T10:40:19.923120+0000","last_active":"2023-10-18T10:40:19.924051+0000","last_peered":"2023-10-18T10:40:19.924051+0000","last_clean":"2023-10-18T10:40:19.924051+0000","last_became_active":"2023-10-18T10:40:19.922847+0000","last_became_peered":"2023-10-18T10:40:19.922847+0000","last_unstale":"2023-10-18T10:40:19.924051+0000","last_undegraded":"2023-10-18T10:40:19.924051+0000","last_fullsized":"2023-10-18T10:40:19.924051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:04.526990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.922729+0000","last_change":"2023-10-18T10:40:18.901404+0000","last_active":"2023-10-18T10:40:19.922729+0000","last_peered":"2023-10-18T10:40:19.922729+0000","last_clean":"2023-10-18T10:40:19.922729+0000","last_became_active":"2023-10-18T10:40:18.900949+0000","last_became_peered":"2023-10-18T10:40:18.900949+0000","last_unstale":"2023-10-18T10:40:19.922729+0000","last_undegraded":"2023-10-18T10:40:19.922729+0000","last_fullsized":"2023-10-18T10:40:19.922729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:54.300307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902234+0000","last_change":"2023-10-18T10:40:19.910432+0000","last_active":"2023-10-18T10:40:20.902234+0000","last_peered":"2023-10-18T10:40:20.902234+0000","last_clean":"2023-10-18T10:40:20.902234+0000","last_became_active":"2023-10-18T10:40:19.910017+0000","last_became_peered":"2023-10-18T10:40:19.910017+0000","last_unstale":"2023-10-18T10:40:20.902234+0000","last_undegraded":"2023-10-18T10:40:20.902234+0000","last_fullsized":"2023-10-18T10:40:20.902234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:41.294716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896846+0000","last_change":"2023-10-18T10:40:18.907834+0000","last_active":"2023-10-18T10:40:19.896846+0000","last_peered":"2023-10-18T10:40:19.896846+0000","last_clean":"2023-10-18T10:40:19.896846+0000","last_became_active":"2023-10-18T10:40:18.907243+0000","last_became_peered":"2023-10-18T10:40:18.907243+0000","last_unstale":"2023-10-18T10:40:19.896846+0000","last_undegraded":"2023-10-18T10:40:19.896846+0000","last_fullsized":"2023-10-18T10:40:19.896846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:00.888824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919903+0000","last_change":"2023-10-18T10:40:19.919367+0000","last_active":"2023-10-18T10:40:19.919903+0000","last_peered":"2023-10-18T10:40:19.919903+0000","last_clean":"2023-10-18T10:40:19.919903+0000","last_became_active":"2023-10-18T10:40:19.919114+0000","last_became_peered":"2023-10-18T10:40:19.919114+0000","last_unstale":"2023-10-18T10:40:19.919903+0000","last_undegraded":"2023-10-18T10:40:19.919903+0000","last_fullsized":"2023-10-18T10:40:19.919903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:44.547473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902222+0000","last_change":"2023-10-18T10:40:18.904489+0000","last_active":"2023-10-18T10:40:20.902222+0000","last_peered":"2023-10-18T10:40:20.902222+0000","last_clean":"2023-10-18T10:40:20.902222+0000","last_became_active":"2023-10-18T10:40:18.904227+0000","last_became_peered":"2023-10-18T10:40:18.904227+0000","last_unstale":"2023-10-18T10:40:20.902222+0000","last_undegraded":"2023-10-18T10:40:20.902222+0000","last_fullsized":"2023-10-18T10:40:20.902222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:23:40.577154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.916409+0000","last_change":"2023-10-18T10:40:19.907755+0000","last_active":"2023-10-18T10:40:19.916409+0000","last_peered":"2023-10-18T10:40:19.916409+0000","last_clean":"2023-10-18T10:40:19.916409+0000","last_became_active":"2023-10-18T10:40:19.907464+0000","last_became_peered":"2023-10-18T10:40:19.907464+0000","last_unstale":"2023-10-18T10:40:19.916409+0000","last_undegraded":"2023-10-18T10:40:19.916409+0000","last_fullsized":"2023-10-18T10:40:19.916409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:43.487087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506791+0000","last_change":"2023-10-18T10:40:18.908362+0000","last_active":"2023-10-18T10:40:21.506791+0000","last_peered":"2023-10-18T10:40:21.506791+0000","last_clean":"2023-10-18T10:40:21.506791+0000","last_became_active":"2023-10-18T10:40:18.908086+0000","last_became_peered":"2023-10-18T10:40:18.908086+0000","last_unstale":"2023-10-18T10:40:21.506791+0000","last_undegraded":"2023-10-18T10:40:21.506791+0000","last_fullsized":"2023-10-18T10:40:21.506791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:46.349468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.914067+0000","last_change":"2023-10-18T10:40:19.911151+0000","last_active":"2023-10-18T10:40:19.914067+0000","last_peered":"2023-10-18T10:40:19.914067+0000","last_clean":"2023-10-18T10:40:19.914067+0000","last_became_active":"2023-10-18T10:40:19.910476+0000","last_became_peered":"2023-10-18T10:40:19.910476+0000","last_unstale":"2023-10-18T10:40:19.914067+0000","last_undegraded":"2023-10-18T10:40:19.914067+0000","last_fullsized":"2023-10-18T10:40:19.914067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:43.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910808+0000","last_change":"2023-10-18T10:40:18.902928+0000","last_active":"2023-10-18T10:40:19.910808+0000","last_peered":"2023-10-18T10:40:19.910808+0000","last_clean":"2023-10-18T10:40:19.910808+0000","last_became_active":"2023-10-18T10:40:18.900825+0000","last_became_peered":"2023-10-18T10:40:18.900825+0000","last_unstale":"2023-10-18T10:40:19.910808+0000","last_undegraded":"2023-10-18T10:40:19.910808+0000","last_fullsized":"2023-10-18T10:40:19.910808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:40.289570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902809+0000","last_change":"2023-10-18T10:40:19.916696+0000","last_active":"2023-10-18T10:40:20.902809+0000","last_peered":"2023-10-18T10:40:20.902809+0000","last_clean":"2023-10-18T10:40:20.902809+0000","last_became_active":"2023-10-18T10:40:19.916431+0000","last_became_peered":"2023-10-18T10:40:19.916431+0000","last_unstale":"2023-10-18T10:40:20.902809+0000","last_undegraded":"2023-10-18T10:40:20.902809+0000","last_fullsized":"2023-10-18T10:40:20.902809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:06:16.648782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"83'3","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506480+0000","last_change":"2023-10-18T10:40:18.905401+0000","last_active":"2023-10-18T10:40:21.506480+0000","last_peered":"2023-10-18T10:40:21.506480+0000","last_clean":"2023-10-18T10:40:21.506480+0000","last_became_active":"2023-10-18T10:40:18.905137+0000","last_became_peered":"2023-10-18T10:40:18.905137+0000","last_unstale":"2023-10-18T10:40:21.506480+0000","last_undegraded":"2023-10-18T10:40:21.506480+0000","last_fullsized":"2023-10-18T10:40:21.506480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:18.224893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919989+0000","last_change":"2023-10-18T10:40:19.917320+0000","last_active":"2023-10-18T10:40:19.919989+0000","last_peered":"2023-10-18T10:40:19.919989+0000","last_clean":"2023-10-18T10:40:19.919989+0000","last_became_active":"2023-10-18T10:40:19.916828+0000","last_became_peered":"2023-10-18T10:40:19.916828+0000","last_unstale":"2023-10-18T10:40:19.919989+0000","last_undegraded":"2023-10-18T10:40:19.919989+0000","last_fullsized":"2023-10-18T10:40:19.919989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:58:04.490274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.928954+0000","last_change":"2023-10-18T10:40:18.905020+0000","last_active":"2023-10-18T10:40:19.928954+0000","last_peered":"2023-10-18T10:40:19.928954+0000","last_clean":"2023-10-18T10:40:19.928954+0000","last_became_active":"2023-10-18T10:40:18.904379+0000","last_became_peered":"2023-10-18T10:40:18.904379+0000","last_unstale":"2023-10-18T10:40:19.928954+0000","last_undegraded":"2023-10-18T10:40:19.928954+0000","last_fullsized":"2023-10-18T10:40:19.928954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:28.436049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.919896+0000","last_change":"2023-10-18T10:40:19.912403+0000","last_active":"2023-10-18T10:40:19.919896+0000","last_peered":"2023-10-18T10:40:19.919896+0000","last_clean":"2023-10-18T10:40:19.919896+0000","last_became_active":"2023-10-18T10:40:19.912123+0000","last_became_peered":"2023-10-18T10:40:19.912123+0000","last_unstale":"2023-10-18T10:40:19.919896+0000","last_undegraded":"2023-10-18T10:40:19.919896+0000","last_fullsized":"2023-10-18T10:40:19.919896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:23:41.582300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506310+0000","last_change":"2023-10-18T10:40:18.907634+0000","last_active":"2023-10-18T10:40:21.506310+0000","last_peered":"2023-10-18T10:40:21.506310+0000","last_clean":"2023-10-18T10:40:21.506310+0000","last_became_active":"2023-10-18T10:40:18.907350+0000","last_became_peered":"2023-10-18T10:40:18.907350+0000","last_unstale":"2023-10-18T10:40:21.506310+0000","last_undegraded":"2023-10-18T10:40:21.506310+0000","last_fullsized":"2023-10-18T10:40:21.506310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:53.817705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796678+0000","last_change":"2023-10-18T10:40:19.909879+0000","last_active":"2023-10-18T10:40:21.796678+0000","last_peered":"2023-10-18T10:40:21.796678+0000","last_clean":"2023-10-18T10:40:21.796678+0000","last_became_active":"2023-10-18T10:40:19.909602+0000","last_became_peered":"2023-10-18T10:40:19.909602+0000","last_unstale":"2023-10-18T10:40:21.796678+0000","last_undegraded":"2023-10-18T10:40:21.796678+0000","last_fullsized":"2023-10-18T10:40:21.796678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:40.764061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796570+0000","last_change":"2023-10-18T10:40:18.903374+0000","last_active":"2023-10-18T10:40:21.796570+0000","last_peered":"2023-10-18T10:40:21.796570+0000","last_clean":"2023-10-18T10:40:21.796570+0000","last_became_active":"2023-10-18T10:40:18.902956+0000","last_became_peered":"2023-10-18T10:40:18.902956+0000","last_unstale":"2023-10-18T10:40:21.796570+0000","last_undegraded":"2023-10-18T10:40:21.796570+0000","last_fullsized":"2023-10-18T10:40:21.796570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:35:55.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902416+0000","last_change":"2023-10-18T10:40:19.908861+0000","last_active":"2023-10-18T10:40:20.902416+0000","last_peered":"2023-10-18T10:40:20.902416+0000","last_clean":"2023-10-18T10:40:20.902416+0000","last_became_active":"2023-10-18T10:40:19.908585+0000","last_became_peered":"2023-10-18T10:40:19.908585+0000","last_unstale":"2023-10-18T10:40:20.902416+0000","last_undegraded":"2023-10-18T10:40:20.902416+0000","last_fullsized":"2023-10-18T10:40:20.902416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:27.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506173+0000","last_change":"2023-10-18T10:40:18.902520+0000","last_active":"2023-10-18T10:40:21.506173+0000","last_peered":"2023-10-18T10:40:21.506173+0000","last_clean":"2023-10-18T10:40:21.506173+0000","last_became_active":"2023-10-18T10:40:18.902149+0000","last_became_peered":"2023-10-18T10:40:18.902149+0000","last_unstale":"2023-10-18T10:40:21.506173+0000","last_undegraded":"2023-10-18T10:40:21.506173+0000","last_fullsized":"2023-10-18T10:40:21.506173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:28.391862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.917478+0000","last_change":"2023-10-18T10:40:19.909009+0000","last_active":"2023-10-18T10:40:19.917478+0000","last_peered":"2023-10-18T10:40:19.917478+0000","last_clean":"2023-10-18T10:40:19.917478+0000","last_became_active":"2023-10-18T10:40:19.908767+0000","last_became_peered":"2023-10-18T10:40:19.908767+0000","last_unstale":"2023-10-18T10:40:19.917478+0000","last_undegraded":"2023-10-18T10:40:19.917478+0000","last_fullsized":"2023-10-18T10:40:19.917478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:00.544158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.929032+0000","last_change":"2023-10-18T10:40:18.904087+0000","last_active":"2023-10-18T10:40:19.929032+0000","last_peered":"2023-10-18T10:40:19.929032+0000","last_clean":"2023-10-18T10:40:19.929032+0000","last_became_active":"2023-10-18T10:40:18.903533+0000","last_became_peered":"2023-10-18T10:40:18.903533+0000","last_unstale":"2023-10-18T10:40:19.929032+0000","last_undegraded":"2023-10-18T10:40:19.929032+0000","last_fullsized":"2023-10-18T10:40:19.929032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:43:32.864029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506533+0000","last_change":"2023-10-18T10:40:19.920719+0000","last_active":"2023-10-18T10:40:21.506533+0000","last_peered":"2023-10-18T10:40:21.506533+0000","last_clean":"2023-10-18T10:40:21.506533+0000","last_became_active":"2023-10-18T10:40:19.919994+0000","last_became_peered":"2023-10-18T10:40:19.919994+0000","last_unstale":"2023-10-18T10:40:21.506533+0000","last_undegraded":"2023-10-18T10:40:21.506533+0000","last_fullsized":"2023-10-18T10:40:21.506533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:58.093162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796980+0000","last_change":"2023-10-18T10:40:18.904201+0000","last_active":"2023-10-18T10:40:21.796980+0000","last_peered":"2023-10-18T10:40:21.796980+0000","last_clean":"2023-10-18T10:40:21.796980+0000","last_became_active":"2023-10-18T10:40:18.902679+0000","last_became_peered":"2023-10-18T10:40:18.902679+0000","last_unstale":"2023-10-18T10:40:21.796980+0000","last_undegraded":"2023-10-18T10:40:21.796980+0000","last_fullsized":"2023-10-18T10:40:21.796980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:34:26.005003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902871+0000","last_change":"2023-10-18T10:40:19.918134+0000","last_active":"2023-10-18T10:40:20.902871+0000","last_peered":"2023-10-18T10:40:20.902871+0000","last_clean":"2023-10-18T10:40:20.902871+0000","last_became_active":"2023-10-18T10:40:19.917832+0000","last_became_peered":"2023-10-18T10:40:19.917832+0000","last_unstale":"2023-10-18T10:40:20.902871+0000","last_undegraded":"2023-10-18T10:40:20.902871+0000","last_fullsized":"2023-10-18T10:40:20.902871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:55.584201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506636+0000","last_change":"2023-10-18T10:40:18.909663+0000","last_active":"2023-10-18T10:40:21.506636+0000","last_peered":"2023-10-18T10:40:21.506636+0000","last_clean":"2023-10-18T10:40:21.506636+0000","last_became_active":"2023-10-18T10:40:18.909380+0000","last_became_peered":"2023-10-18T10:40:18.909380+0000","last_unstale":"2023-10-18T10:40:21.506636+0000","last_undegraded":"2023-10-18T10:40:21.506636+0000","last_fullsized":"2023-10-18T10:40:21.506636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:39.758915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926264+0000","last_change":"2023-10-18T10:40:19.921712+0000","last_active":"2023-10-18T10:40:19.926264+0000","last_peered":"2023-10-18T10:40:19.926264+0000","last_clean":"2023-10-18T10:40:19.926264+0000","last_became_active":"2023-10-18T10:40:19.921407+0000","last_became_peered":"2023-10-18T10:40:19.921407+0000","last_unstale":"2023-10-18T10:40:19.926264+0000","last_undegraded":"2023-10-18T10:40:19.926264+0000","last_fullsized":"2023-10-18T10:40:19.926264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:27:11.030985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506602+0000","last_change":"2023-10-18T10:40:18.908209+0000","last_active":"2023-10-18T10:40:21.506602+0000","last_peered":"2023-10-18T10:40:21.506602+0000","last_clean":"2023-10-18T10:40:21.506602+0000","last_became_active":"2023-10-18T10:40:18.907903+0000","last_became_peered":"2023-10-18T10:40:18.907903+0000","last_unstale":"2023-10-18T10:40:21.506602+0000","last_undegraded":"2023-10-18T10:40:21.506602+0000","last_fullsized":"2023-10-18T10:40:21.506602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:07.872220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902781+0000","last_change":"2023-10-18T10:40:19.911382+0000","last_active":"2023-10-18T10:40:20.902781+0000","last_peered":"2023-10-18T10:40:20.902781+0000","last_clean":"2023-10-18T10:40:20.902781+0000","last_became_active":"2023-10-18T10:40:19.911148+0000","last_became_peered":"2023-10-18T10:40:19.911148+0000","last_unstale":"2023-10-18T10:40:20.902781+0000","last_undegraded":"2023-10-18T10:40:20.902781+0000","last_fullsized":"2023-10-18T10:40:20.902781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:58:23.503790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897209+0000","last_change":"2023-10-18T10:40:18.902590+0000","last_active":"2023-10-18T10:40:19.897209+0000","last_peered":"2023-10-18T10:40:19.897209+0000","last_clean":"2023-10-18T10:40:19.897209+0000","last_became_active":"2023-10-18T10:40:18.902335+0000","last_became_peered":"2023-10-18T10:40:18.902335+0000","last_unstale":"2023-10-18T10:40:19.897209+0000","last_undegraded":"2023-10-18T10:40:19.897209+0000","last_fullsized":"2023-10-18T10:40:19.897209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:58:24.515902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898728+0000","last_change":"2023-10-18T10:40:19.914939+0000","last_active":"2023-10-18T10:40:20.898728+0000","last_peered":"2023-10-18T10:40:20.898728+0000","last_clean":"2023-10-18T10:40:20.898728+0000","last_became_active":"2023-10-18T10:40:19.914697+0000","last_became_peered":"2023-10-18T10:40:19.914697+0000","last_unstale":"2023-10-18T10:40:20.898728+0000","last_undegraded":"2023-10-18T10:40:20.898728+0000","last_fullsized":"2023-10-18T10:40:20.898728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:22:18.467561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898802+0000","last_change":"2023-10-18T10:40:18.906643+0000","last_active":"2023-10-18T10:40:20.898802+0000","last_peered":"2023-10-18T10:40:20.898802+0000","last_clean":"2023-10-18T10:40:20.898802+0000","last_became_active":"2023-10-18T10:40:18.906394+0000","last_became_peered":"2023-10-18T10:40:18.906394+0000","last_unstale":"2023-10-18T10:40:20.898802+0000","last_undegraded":"2023-10-18T10:40:20.898802+0000","last_fullsized":"2023-10-18T10:40:20.898802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:08.817426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796546+0000","last_change":"2023-10-18T10:40:19.905171+0000","last_active":"2023-10-18T10:40:21.796546+0000","last_peered":"2023-10-18T10:40:21.796546+0000","last_clean":"2023-10-18T10:40:21.796546+0000","last_became_active":"2023-10-18T10:40:19.904893+0000","last_became_peered":"2023-10-18T10:40:19.904893+0000","last_unstale":"2023-10-18T10:40:21.796546+0000","last_undegraded":"2023-10-18T10:40:21.796546+0000","last_fullsized":"2023-10-18T10:40:21.796546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:47:36.295066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796800+0000","last_change":"2023-10-18T10:40:18.903207+0000","last_active":"2023-10-18T10:40:21.796800+0000","last_peered":"2023-10-18T10:40:21.796800+0000","last_clean":"2023-10-18T10:40:21.796800+0000","last_became_active":"2023-10-18T10:40:18.902656+0000","last_became_peered":"2023-10-18T10:40:18.902656+0000","last_unstale":"2023-10-18T10:40:21.796800+0000","last_undegraded":"2023-10-18T10:40:21.796800+0000","last_fullsized":"2023-10-18T10:40:21.796800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:20.431457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902422+0000","last_change":"2023-10-18T10:40:19.911858+0000","last_active":"2023-10-18T10:40:20.902422+0000","last_peered":"2023-10-18T10:40:20.902422+0000","last_clean":"2023-10-18T10:40:20.902422+0000","last_became_active":"2023-10-18T10:40:19.910571+0000","last_became_peered":"2023-10-18T10:40:19.910571+0000","last_unstale":"2023-10-18T10:40:20.902422+0000","last_undegraded":"2023-10-18T10:40:20.902422+0000","last_fullsized":"2023-10-18T10:40:20.902422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:30.689854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898655+0000","last_change":"2023-10-18T10:40:18.899875+0000","last_active":"2023-10-18T10:40:20.898655+0000","last_peered":"2023-10-18T10:40:20.898655+0000","last_clean":"2023-10-18T10:40:20.898655+0000","last_became_active":"2023-10-18T10:40:18.899619+0000","last_became_peered":"2023-10-18T10:40:18.899619+0000","last_unstale":"2023-10-18T10:40:20.898655+0000","last_undegraded":"2023-10-18T10:40:20.898655+0000","last_fullsized":"2023-10-18T10:40:20.898655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:54.300307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898893+0000","last_change":"2023-10-18T10:40:19.919436+0000","last_active":"2023-10-18T10:40:20.898893+0000","last_peered":"2023-10-18T10:40:20.898893+0000","last_clean":"2023-10-18T10:40:20.898893+0000","last_became_active":"2023-10-18T10:40:19.919181+0000","last_became_peered":"2023-10-18T10:40:19.919181+0000","last_unstale":"2023-10-18T10:40:20.898893+0000","last_undegraded":"2023-10-18T10:40:20.898893+0000","last_fullsized":"2023-10-18T10:40:20.898893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:35.681517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.797039+0000","last_change":"2023-10-18T10:40:18.906951+0000","last_active":"2023-10-18T10:40:21.797039+0000","last_peered":"2023-10-18T10:40:21.797039+0000","last_clean":"2023-10-18T10:40:21.797039+0000","last_became_active":"2023-10-18T10:40:18.906455+0000","last_became_peered":"2023-10-18T10:40:18.906455+0000","last_unstale":"2023-10-18T10:40:21.797039+0000","last_undegraded":"2023-10-18T10:40:21.797039+0000","last_fullsized":"2023-10-18T10:40:21.797039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:28.391862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910555+0000","last_change":"2023-10-18T10:40:19.906881+0000","last_active":"2023-10-18T10:40:19.910555+0000","last_peered":"2023-10-18T10:40:19.910555+0000","last_clean":"2023-10-18T10:40:19.910555+0000","last_became_active":"2023-10-18T10:40:19.906627+0000","last_became_peered":"2023-10-18T10:40:19.906627+0000","last_unstale":"2023-10-18T10:40:19.910555+0000","last_undegraded":"2023-10-18T10:40:19.910555+0000","last_fullsized":"2023-10-18T10:40:19.910555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:33.017257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796915+0000","last_change":"2023-10-18T10:40:18.904182+0000","last_active":"2023-10-18T10:40:21.796915+0000","last_peered":"2023-10-18T10:40:21.796915+0000","last_clean":"2023-10-18T10:40:21.796915+0000","last_became_active":"2023-10-18T10:40:18.903869+0000","last_became_peered":"2023-10-18T10:40:18.903869+0000","last_unstale":"2023-10-18T10:40:21.796915+0000","last_undegraded":"2023-10-18T10:40:21.796915+0000","last_fullsized":"2023-10-18T10:40:21.796915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:05.894872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.796812+0000","last_change":"2023-10-18T10:40:19.909239+0000","last_active":"2023-10-18T10:40:21.796812+0000","last_peered":"2023-10-18T10:40:21.796812+0000","last_clean":"2023-10-18T10:40:21.796812+0000","last_became_active":"2023-10-18T10:40:19.908949+0000","last_became_peered":"2023-10-18T10:40:19.908949+0000","last_unstale":"2023-10-18T10:40:21.796812+0000","last_undegraded":"2023-10-18T10:40:21.796812+0000","last_fullsized":"2023-10-18T10:40:21.796812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:44.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.898995+0000","last_change":"2023-10-18T10:40:18.904341+0000","last_active":"2023-10-18T10:40:20.898995+0000","last_peered":"2023-10-18T10:40:20.898995+0000","last_clean":"2023-10-18T10:40:20.898995+0000","last_became_active":"2023-10-18T10:40:18.904111+0000","last_became_peered":"2023-10-18T10:40:18.904111+0000","last_unstale":"2023-10-18T10:40:20.898995+0000","last_undegraded":"2023-10-18T10:40:20.898995+0000","last_fullsized":"2023-10-18T10:40:20.898995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:43.542327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902850+0000","last_change":"2023-10-18T10:40:19.912475+0000","last_active":"2023-10-18T10:40:20.902850+0000","last_peered":"2023-10-18T10:40:20.902850+0000","last_clean":"2023-10-18T10:40:20.902850+0000","last_became_active":"2023-10-18T10:40:19.912112+0000","last_became_peered":"2023-10-18T10:40:19.912112+0000","last_unstale":"2023-10-18T10:40:20.902850+0000","last_undegraded":"2023-10-18T10:40:20.902850+0000","last_fullsized":"2023-10-18T10:40:20.902850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:09:51.716263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.902230+0000","last_change":"2023-10-18T10:40:18.905086+0000","last_active":"2023-10-18T10:40:19.902230+0000","last_peered":"2023-10-18T10:40:19.902230+0000","last_clean":"2023-10-18T10:40:19.902230+0000","last_became_active":"2023-10-18T10:40:18.904715+0000","last_became_peered":"2023-10-18T10:40:18.904715+0000","last_unstale":"2023-10-18T10:40:19.902230+0000","last_undegraded":"2023-10-18T10:40:19.902230+0000","last_fullsized":"2023-10-18T10:40:19.902230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:44.769336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902202+0000","last_change":"2023-10-18T10:40:19.911753+0000","last_active":"2023-10-18T10:40:20.902202+0000","last_peered":"2023-10-18T10:40:20.902202+0000","last_clean":"2023-10-18T10:40:20.902202+0000","last_became_active":"2023-10-18T10:40:19.911511+0000","last_became_peered":"2023-10-18T10:40:19.911511+0000","last_unstale":"2023-10-18T10:40:20.902202+0000","last_undegraded":"2023-10-18T10:40:20.902202+0000","last_fullsized":"2023-10-18T10:40:20.902202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:31:07.900837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:21.506412+0000","last_change":"2023-10-18T10:40:18.902448+0000","last_active":"2023-10-18T10:40:21.506412+0000","last_peered":"2023-10-18T10:40:21.506412+0000","last_clean":"2023-10-18T10:40:21.506412+0000","last_became_active":"2023-10-18T10:40:18.902196+0000","last_became_peered":"2023-10-18T10:40:18.902196+0000","last_unstale":"2023-10-18T10:40:21.506412+0000","last_undegraded":"2023-10-18T10:40:21.506412+0000","last_fullsized":"2023-10-18T10:40:21.506412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:26.979193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.920063+0000","last_change":"2023-10-18T10:40:19.914096+0000","last_active":"2023-10-18T10:40:19.920063+0000","last_peered":"2023-10-18T10:40:19.920063+0000","last_clean":"2023-10-18T10:40:19.920063+0000","last_became_active":"2023-10-18T10:40:19.913844+0000","last_became_peered":"2023-10-18T10:40:19.913844+0000","last_unstale":"2023-10-18T10:40:19.920063+0000","last_undegraded":"2023-10-18T10:40:19.920063+0000","last_fullsized":"2023-10-18T10:40:19.920063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:21.171698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897010+0000","last_change":"2023-10-18T10:40:18.901223+0000","last_active":"2023-10-18T10:40:19.897010+0000","last_peered":"2023-10-18T10:40:19.897010+0000","last_clean":"2023-10-18T10:40:19.897010+0000","last_became_active":"2023-10-18T10:40:18.900638+0000","last_became_peered":"2023-10-18T10:40:18.900638+0000","last_unstale":"2023-10-18T10:40:19.897010+0000","last_undegraded":"2023-10-18T10:40:19.897010+0000","last_fullsized":"2023-10-18T10:40:19.897010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:04.547290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.926391+0000","last_change":"2023-10-18T10:40:19.925927+0000","last_active":"2023-10-18T10:40:19.926391+0000","last_peered":"2023-10-18T10:40:19.926391+0000","last_clean":"2023-10-18T10:40:19.926391+0000","last_became_active":"2023-10-18T10:40:19.925668+0000","last_became_peered":"2023-10-18T10:40:19.925668+0000","last_unstale":"2023-10-18T10:40:19.926391+0000","last_undegraded":"2023-10-18T10:40:19.926391+0000","last_fullsized":"2023-10-18T10:40:19.926391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:09.139441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.897265+0000","last_change":"2023-10-18T10:40:18.904336+0000","last_active":"2023-10-18T10:40:19.897265+0000","last_peered":"2023-10-18T10:40:19.897265+0000","last_clean":"2023-10-18T10:40:19.897265+0000","last_became_active":"2023-10-18T10:40:18.904085+0000","last_became_peered":"2023-10-18T10:40:18.904085+0000","last_unstale":"2023-10-18T10:40:19.897265+0000","last_undegraded":"2023-10-18T10:40:19.897265+0000","last_fullsized":"2023-10-18T10:40:19.897265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:32.124110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.910680+0000","last_change":"2023-10-18T10:40:19.910079+0000","last_active":"2023-10-18T10:40:19.910680+0000","last_peered":"2023-10-18T10:40:19.910680+0000","last_clean":"2023-10-18T10:40:19.910680+0000","last_became_active":"2023-10-18T10:40:19.909833+0000","last_became_peered":"2023-10-18T10:40:19.909833+0000","last_unstale":"2023-10-18T10:40:19.910680+0000","last_undegraded":"2023-10-18T10:40:19.910680+0000","last_fullsized":"2023-10-18T10:40:19.910680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:43:23.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.900343+0000","last_change":"2023-10-18T10:40:18.902461+0000","last_active":"2023-10-18T10:40:19.900343+0000","last_peered":"2023-10-18T10:40:19.900343+0000","last_clean":"2023-10-18T10:40:19.900343+0000","last_became_active":"2023-10-18T10:40:18.902198+0000","last_became_peered":"2023-10-18T10:40:18.902198+0000","last_unstale":"2023-10-18T10:40:19.900343+0000","last_undegraded":"2023-10-18T10:40:19.900343+0000","last_fullsized":"2023-10-18T10:40:19.900343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:13:18.205683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T10:40:20.902492+0000","last_change":"2023-10-18T10:40:19.915806+0000","last_active":"2023-10-18T10:40:20.902492+0000","last_peered":"2023-10-18T10:40:20.902492+0000","last_clean":"2023-10-18T10:40:20.902492+0000","last_became_active":"2023-10-18T10:40:19.915465+0000","last_became_peered":"2023-10-18T10:40:19.915465+0000","last_unstale":"2023-10-18T10:40:20.902492+0000","last_undegraded":"2023-10-18T10:40:20.902492+0000","last_fullsized":"2023-10-18T10:40:20.902492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:18.884322+0000","last_clean_scrub_stamp":"2023-10-18T10:40:18.884322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:45.206294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.896765+0000","last_change":"2023-10-18T10:40:18.901946+0000","last_active":"2023-10-18T10:40:19.896765+0000","last_peered":"2023-10-18T10:40:19.896765+0000","last_clean":"2023-10-18T10:40:19.896765+0000","last_became_active":"2023-10-18T10:40:18.901694+0000","last_became_peered":"2023-10-18T10:40:18.901694+0000","last_unstale":"2023-10-18T10:40:19.896765+0000","last_undegraded":"2023-10-18T10:40:19.896765+0000","last_fullsized":"2023-10-18T10:40:19.896765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:17.879176+0000","last_clean_scrub_stamp":"2023-10-18T10:40:17.879176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:54:08.817426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'161","reported_seq":383,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-18T10:40:19.898971+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:19.898971+0000","last_peered":"2023-10-18T10:40:19.898971+0000","last_clean":"2023-10-18T10:40:19.898971+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:19.898971+0000","last_undegraded":"2023-10-18T10:40:19.898971+0000","last_fullsized":"2023-10-18T10:40:19.898971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575037,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59136,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312704,"statfs":{"total":96636764160,"available":96576208896,"internally_reserved":0,"allocated":1617920,"data_stored":1091181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.69199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1024000,"data_stored":506691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.436},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":1.016},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.80300000000000005,"15min":0.875},"last":0.37}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.872,"15min":0.872},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.54200000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66800000000000004,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.496}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":0.82199999999999995,"15min":0.87},"last":0.56299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834575037,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":500059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.47699999999999998,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.016},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.624}]},{"osd":1,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.46200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.76200000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.749}]},{"osd":2,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.48199999999999998,"5min":0.377,"15min":0.28599999999999998},"max":{"1min":0.746,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.51600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.49399999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.57099999999999995,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.81000000000000005,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.72299999999999998,"5min":0.80100000000000005,"15min":1.262},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":1.0109999999999999,"15min":1.028},"last":0.33600000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72999999999999998,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":0.96099999999999997,"15min":1.224},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:39:22 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.73699999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.495},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.871,"15min":1.159},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575037,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":501649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.877,"15min":0.877},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51100000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.502,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.72099999999999997,"5min":0.746,"15min":0.79600000000000004},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75600000000000001,"5min":1.446,"15min":1.446},"last":0.495},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.90400000000000003},"last":0.376}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.82699999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.41999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":500037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54376,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317464,"statfs":{"total":96636764160,"available":96581083136,"internally_reserved":0,"allocated":1007616,"data_stored":500037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.313,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.52300000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.32700000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.39500000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.378},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40300000000000002},"min":{"1min":0.34100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60999999999999999,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.53400000000000003,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.377,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66000000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.88200000000000001,"5min":0.88800000000000001,"15min":1.458},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.52600000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.95699999999999996,"15min":0.997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.82299999999999995,"5min":0.93700000000000006,"15min":1.028},"last":0.748}]},{"osd":7,"last update":"Wed Oct 18 10:39:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.63300000000000001},"min":{"1min":0.51000000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.88900000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.52500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84599999999999997,"5min":0.90200000000000002,"15min":0.92500000000000004},"last":0.63200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":502413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.371},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.434}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.505}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1601536,"data_stored":1090405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.316}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.371},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.307}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41399999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.40500000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.438},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.45700000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.46700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:27.599 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:40:27.602 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:40:27.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.734+0000 7f189de10640 1 -- 172.21.15.125:0/1221674139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18800052a0 msgr2=0x7f188000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.734+0000 7f189de10640 1 --2- 172.21.15.125:0/1221674139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18800052a0 0x7f188000b980 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f189000c610 tx=0x7f18800057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.734+0000 7f189de10640 1 -- 172.21.15.125:0/1221674139 shutdown_connections 2023-10-18T10:40:27.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.734+0000 7f189de10640 1 --2- 172.21.15.125:0/1221674139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18800052a0 0x7f188000b980 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.734+0000 7f189de10640 1 -- 172.21.15.125:0/1221674139 >> 172.21.15.125:0/1221674139 conn(0x7f18980d0d60 msgr2=0x7f18980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:27.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.734+0000 7f189de10640 1 -- 172.21.15.125:0/1221674139 shutdown_connections 2023-10-18T10:40:27.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.734+0000 7f189de10640 1 -- 172.21.15.125:0/1221674139 wait complete. 2023-10-18T10:40:27.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.735+0000 7f189de10640 1 Processor -- start 2023-10-18T10:40:27.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.735+0000 7f189de10640 1 -- start start 2023-10-18T10:40:27.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f189de10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18800052a0 0x7f18981591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:27.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f189de10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18980c8840 0x7f18980ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:27.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f189de10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189815dad0 0x7f189815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:27.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f189de10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189815e490 con 0x7f189815dad0 2023-10-18T10:40:27.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f189de10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189809c1d0 con 0x7f18980c8840 2023-10-18T10:40:27.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f189de10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18980c8e00 con 0x7f18800052a0 2023-10-18T10:40:27.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189815dad0 0x7f189815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:27.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189815dad0 0x7f189815df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49970/0 (socket says 172.21.15.125:49970) 2023-10-18T10:40:27.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 -- 172.21.15.125:0/3470305466 learned_addr learned my addr 172.21.15.125:0/3470305466 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:27.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f18977fe640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18800052a0 0x7f18981591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:27.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1896ffd640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18980c8840 0x7f18980ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:27.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 -- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18800052a0 msgr2=0x7f18981591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18800052a0 0x7f18981591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 -- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18980c8840 msgr2=0x7f18980ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18980c8840 0x7f18980ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 -- 172.21.15.125:0/3470305466 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980c90e0 con 0x7f189815dad0 2023-10-18T10:40:27.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.736+0000 7f1897fff640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189815dad0 0x7f189815df50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f189000c730 tx=0x7f1890001390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:27.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.737+0000 7f187ffff640 1 -- 172.21.15.125:0/3470305466 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18900019f0 con 0x7f189815dad0 2023-10-18T10:40:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.737+0000 7f187ffff640 1 -- 172.21.15.125:0/3470305466 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1890003970 con 0x7f189815dad0 2023-10-18T10:40:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.737+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18980c93c0 con 0x7f189815dad0 2023-10-18T10:40:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.737+0000 7f187ffff640 1 -- 172.21.15.125:0/3470305466 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890003d80 con 0x7f189815dad0 2023-10-18T10:40:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.737+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18980caf30 con 0x7f189815dad0 2023-10-18T10:40:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.738+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189809c390 con 0x7f189815dad0 2023-10-18T10:40:27.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.738+0000 7f187ffff640 1 -- 172.21.15.125:0/3470305466 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1890031030 con 0x7f189815dad0 2023-10-18T10:40:27.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.738+0000 7f187ffff640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f188406af60 0x7f188406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:27.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.738+0000 7f187ffff640 1 -- 172.21.15.125:0/3470305466 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f18900aa6a0 con 0x7f189815dad0 2023-10-18T10:40:27.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.739+0000 7f18977fe640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f188406af60 0x7f188406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:27.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.739+0000 7f18977fe640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f188406af60 0x7f188406d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1888000c00 tx=0x7f1888006b30 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:27.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.743+0000 7f187ffff640 1 -- 172.21.15.125:0/3470305466 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1890075480 con 0x7f189815dad0 2023-10-18T10:40:27.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.974+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f18980c98e0 con 0x7f189815dad0 2023-10-18T10:40:27.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.976+0000 7f187ffff640 1 -- 172.21.15.125:0/3470305466 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1890078aa0 con 0x7f189815dad0 2023-10-18T10:40:27.978 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:40:27.978 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:40:27.979 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T10:40:27.979 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T10:40:27.979 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:40:27.980 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T10:40:27.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f188406af60 msgr2=0x7f188406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f188406af60 0x7f188406d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1888000c00 tx=0x7f1888006b30 comp rx=0 tx=0).stop 2023-10-18T10:40:27.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189815dad0 msgr2=0x7f189815df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:27.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189815dad0 0x7f189815df50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f189000c730 tx=0x7f1890001390 comp rx=0 tx=0).stop 2023-10-18T10:40:27.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 shutdown_connections 2023-10-18T10:40:27.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18980c8840 0x7f18980ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f188406af60 0x7f188406d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189815dad0 0x7f189815df50 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 --2- 172.21.15.125:0/3470305466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18800052a0 0x7f18981591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:27.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 >> 172.21.15.125:0/3470305466 conn(0x7f18980d0d60 msgr2=0x7f189815a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:27.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 shutdown_connections 2023-10-18T10:40:27.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:27.978+0000 7f189de10640 1 -- 172.21.15.125:0/3470305466 wait complete. 2023-10-18T10:40:27.991 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:28.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 -- 172.21.15.125:0/2323390090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c5406af00 msgr2=0x7f5c5406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/2323390090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c5406af00 0x7f5c5406b2e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5c5406d590 tx=0x7f5c4402bda0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 -- 172.21.15.125:0/2323390090 shutdown_connections 2023-10-18T10:40:28.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/2323390090 >> v2:172.21.15.139:3300/0 conn(0x7f5c540ab500 0x7f5c540ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/2323390090 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c540c96b0 0x7f5c540c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/2323390090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c5406af00 0x7f5c5406b2e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 -- 172.21.15.125:0/2323390090 >> 172.21.15.125:0/2323390090 conn(0x7f5c54069930 msgr2=0x7f5c54069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 -- 172.21.15.125:0/2323390090 shutdown_connections 2023-10-18T10:40:28.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.092+0000 7f5c59aaf640 1 -- 172.21.15.125:0/2323390090 wait complete. 2023-10-18T10:40:28.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.093+0000 7f5c59aaf640 1 Processor -- start 2023-10-18T10:40:28.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.093+0000 7f5c59aaf640 1 -- start start 2023-10-18T10:40:28.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c59aaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c5406af00 0x7f5c540b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c59aaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ab500 0x7f5c54151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c59aaf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c540c96b0 0x7f5c540ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c59aaf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541553b0 con 0x7f5c540c96b0 2023-10-18T10:40:28.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c59aaf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540b06a0 con 0x7f5c540ab500 2023-10-18T10:40:28.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c59aaf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541523e0 con 0x7f5c5406af00 2023-10-18T10:40:28.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c540c96b0 0x7f5c540ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c540c96b0 0x7f5c540ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49988/0 (socket says 172.21.15.125:49988) 2023-10-18T10:40:28.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c53fff640 1 -- 172.21.15.125:0/11951423 learned_addr learned my addr 172.21.15.125:0/11951423 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:28.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c537fe640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c5406af00 0x7f5c540b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.094+0000 7f5c52ffd640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ab500 0x7f5c54151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c53fff640 1 -- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c5406af00 msgr2=0x7f5c540b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c53fff640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c5406af00 0x7f5c540b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c53fff640 1 -- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ab500 msgr2=0x7f5c54151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c53fff640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ab500 0x7f5c54151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c53fff640 1 -- 172.21.15.125:0/11951423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44009560 con 0x7f5c540c96b0 2023-10-18T10:40:28.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c52ffd640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ab500 0x7f5c54151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:28.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c537fe640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c5406af00 0x7f5c540b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:28.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c53fff640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c540c96b0 0x7f5c540ad520 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00b970 tx=0x7f5c4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:28.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c3bfff640 1 -- 172.21.15.125:0/11951423 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c00cae0 con 0x7f5c540c96b0 2023-10-18T10:40:28.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c3bfff640 1 -- 172.21.15.125:0/11951423 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c4c00cd50 con 0x7f5c540c96b0 2023-10-18T10:40:28.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c54153dd0 con 0x7f5c540c96b0 2023-10-18T10:40:28.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c3bfff640 1 -- 172.21.15.125:0/11951423 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0146a0 con 0x7f5c540c96b0 2023-10-18T10:40:28.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.095+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c54154390 con 0x7f5c540c96b0 2023-10-18T10:40:28.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.096+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c540abe50 con 0x7f5c540c96b0 2023-10-18T10:40:28.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.096+0000 7f5c3bfff640 1 -- 172.21.15.125:0/11951423 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c4c01d3b0 con 0x7f5c540c96b0 2023-10-18T10:40:28.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.097+0000 7f5c3bfff640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c4006af60 0x7f5c4006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.097+0000 7f5c3bfff640 1 -- 172.21.15.125:0/11951423 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5c4c089a40 con 0x7f5c540c96b0 2023-10-18T10:40:28.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.097+0000 7f5c537fe640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c4006af60 0x7f5c4006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.097+0000 7f5c537fe640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c4006af60 0x7f5c4006d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c44035000 tx=0x7f5c44004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:28.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.101+0000 7f5c3bfff640 1 -- 172.21.15.125:0/11951423 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c4c08be60 con 0x7f5c540c96b0 2023-10-18T10:40:28.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.304+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c540690e0 con 0x7f5c540c96b0 2023-10-18T10:40:28.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.305+0000 7f5c3bfff640 1 -- 172.21.15.125:0/11951423 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f5c4c019020 con 0x7f5c540c96b0 2023-10-18T10:40:28.307 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:28.307 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/693466412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":693466412},{"type":"v1","addr":"172.21.15.139:6836","nonce":693466412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6277,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/170638031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":170638031},{"type":"v1","addr":"172.21.15.139:6837","nonce":170638031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":7068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3024029084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3024029084},{"type":"v1","addr":"172.21.15.125:6834","nonce":3024029084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:19.231823+0000","modified":"2023-10-18T10:40:21.240206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1136111019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1136111019},{"type":"v1","addr":"172.21.15.125:6835","nonce":1136111019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T10:40:28.307 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 29 2023-10-18T10:40:28.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c4006af60 msgr2=0x7f5c4006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c4006af60 0x7f5c4006d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c44035000 tx=0x7f5c44004730 comp rx=0 tx=0).stop 2023-10-18T10:40:28.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c540c96b0 msgr2=0x7f5c540ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c540c96b0 0x7f5c540ad520 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00b970 tx=0x7f5c4c00be30 comp rx=0 tx=0).stop 2023-10-18T10:40:28.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 shutdown_connections 2023-10-18T10:40:28.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c540ab500 0x7f5c54151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f5c4006af60 0x7f5c4006d420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c540c96b0 0x7f5c540ad520 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 --2- 172.21.15.125:0/11951423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c5406af00 0x7f5c540b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.306+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 >> 172.21.15.125:0/11951423 conn(0x7f5c54069930 msgr2=0x7f5c54069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.307+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 shutdown_connections 2023-10-18T10:40:28.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.307+0000 7f5c59aaf640 1 -- 172.21.15.125:0/11951423 wait complete. 2023-10-18T10:40:28.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:40:19.231823+0000', 'modified': '2023-10-18T10:40:21.240206+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7071}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7071': {'gid': 7071, 'name': 'a', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/1136111019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 1136111019}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 1136111019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:40:28.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:40:28.723 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:40:28.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.827+0000 7fae4684f640 1 -- 172.21.15.125:0/2493719883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 msgr2=0x7fae400703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.827+0000 7fae4684f640 1 --2- 172.21.15.125:0/2493719883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 0x7fae400703e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fae34009830 tx=0x7fae3402bc30 comp rx=0 tx=0).stop 2023-10-18T10:40:28.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.827+0000 7fae4684f640 1 -- 172.21.15.125:0/2493719883 shutdown_connections 2023-10-18T10:40:28.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.827+0000 7fae4684f640 1 --2- 172.21.15.125:0/2493719883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 0x7fae400703e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.827+0000 7fae4684f640 1 --2- 172.21.15.125:0/2493719883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4013e520 0x7fae4006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.827+0000 7fae4684f640 1 -- 172.21.15.125:0/2493719883 >> 172.21.15.125:0/2493719883 conn(0x7fae40069930 msgr2=0x7fae40069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.827+0000 7fae4684f640 1 -- 172.21.15.125:0/2493719883 shutdown_connections 2023-10-18T10:40:28.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.827+0000 7fae4684f640 1 -- 172.21.15.125:0/2493719883 wait complete. 2023-10-18T10:40:28.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.828+0000 7fae4684f640 1 Processor -- start 2023-10-18T10:40:28.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.828+0000 7fae4684f640 1 -- start start 2023-10-18T10:40:28.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae4684f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4006c0a0 0x7fae401ef140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae4684f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4013e520 0x7fae400ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae4684f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 0x7fae400ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae4684f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae401e25c0 con 0x7fae40141690 2023-10-18T10:40:28.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae4684f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae401e28a0 con 0x7fae4013e520 2023-10-18T10:40:28.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae4684f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae401e2b80 con 0x7fae4006c0a0 2023-10-18T10:40:28.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 0x7fae400ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 0x7fae400ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50002/0 (socket says 172.21.15.125:50002) 2023-10-18T10:40:28.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 -- 172.21.15.125:0/840867836 learned_addr learned my addr 172.21.15.125:0/840867836 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:28.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae3ffff640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4006c0a0 0x7fae401ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae3f7fe640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4013e520 0x7fae400ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 -- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4006c0a0 msgr2=0x7fae401ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4006c0a0 0x7fae401ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 -- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4013e520 msgr2=0x7fae400ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4013e520 0x7fae400ab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 -- 172.21.15.125:0/840867836 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009490 con 0x7fae40141690 2023-10-18T10:40:28.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae44dec640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 0x7fae400ad980 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fae3800cb40 tx=0x7fae3800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:28.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae3cff9640 1 -- 172.21.15.125:0/840867836 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800aca0 con 0x7fae40141690 2023-10-18T10:40:28.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae3cff9640 1 -- 172.21.15.125:0/840867836 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae38012420 con 0x7fae40141690 2023-10-18T10:40:28.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae4684f640 1 -- 172.21.15.125:0/840867836 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae400b1320 con 0x7fae40141690 2023-10-18T10:40:28.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae3cff9640 1 -- 172.21.15.125:0/840867836 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38012850 con 0x7fae40141690 2023-10-18T10:40:28.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.829+0000 7fae4684f640 1 -- 172.21.15.125:0/840867836 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae401e3140 con 0x7fae40141690 2023-10-18T10:40:28.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.831+0000 7fae3cff9640 1 -- 172.21.15.125:0/840867836 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fae3801a3b0 con 0x7fae40141690 2023-10-18T10:40:28.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.831+0000 7fae3cff9640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fae2c06af60 0x7fae2c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.831+0000 7fae3cff9640 1 -- 172.21.15.125:0/840867836 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae3808a0a0 con 0x7fae40141690 2023-10-18T10:40:28.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.831+0000 7fae3ffff640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fae2c06af60 0x7fae2c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.832+0000 7fae3ffff640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fae2c06af60 0x7fae2c06d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae30005fd0 tx=0x7fae30005850 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:28.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.832+0000 7fae493e9740 1 Processor -- start 2023-10-18T10:40:28.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.832+0000 7fae493e9740 1 -- start start 2023-10-18T10:40:28.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae493e9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e55ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae493e9740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 0x5574f8e858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae493e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e56020 0x5574f8e85e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae493e9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f8d67730 con 0x5574f8e82b40 2023-10-18T10:40:28.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae493e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f8dc8780 con 0x5574f8e56020 2023-10-18T10:40:28.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae493e9740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574f8dc5e70 con 0x5574f8e82f20 2023-10-18T10:40:28.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e55ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e55ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50006/0 (socket says 172.21.15.125:50006) 2023-10-18T10:40:28.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae3ffff640 1 -- 172.21.15.125:0/3663640233 learned_addr learned my addr 172.21.15.125:0/3663640233 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:28.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae3ffff640 1 -- 172.21.15.125:0/3663640233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 msgr2=0x5574f8e858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:40:28.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae3ffff640 1 --2- 172.21.15.125:0/3663640233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 0x5574f8e858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae3ffff640 1 -- 172.21.15.125:0/3663640233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e56020 msgr2=0x5574f8e85e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:40:28.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae3ffff640 1 --2- 172.21.15.125:0/3663640233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e56020 0x5574f8e85e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.833+0000 7fae3ffff640 1 -- 172.21.15.125:0/3663640233 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f8dc63d0 con 0x5574f8e82b40 2023-10-18T10:40:28.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae3ffff640 1 --2- 172.21.15.125:0/3663640233 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e55ae0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fae3000fee0 tx=0x7fae30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af43594a202b6e5b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:28.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae25ffb640 1 -- 172.21.15.125:0/3663640233 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3001e650 con 0x5574f8e82b40 2023-10-18T10:40:28.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae25ffb640 1 -- 172.21.15.125:0/3663640233 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae3001e8a0 con 0x5574f8e82b40 2023-10-18T10:40:28.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae25ffb640 1 -- 172.21.15.125:0/3663640233 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3001ec70 con 0x5574f8e82b40 2023-10-18T10:40:28.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 -- 172.21.15.125:0/3663640233 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 msgr2=0x5574f8e55ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 --2- 172.21.15.125:0/3663640233 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e55ae0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fae3000fee0 tx=0x7fae30012000 comp rx=0 tx=0).stop 2023-10-18T10:40:28.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 -- 172.21.15.125:0/3663640233 shutdown_connections 2023-10-18T10:40:28.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 --2- 172.21.15.125:0/3663640233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e56020 0x5574f8e85e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 --2- 172.21.15.125:0/3663640233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 0x5574f8e858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 --2- 172.21.15.125:0/3663640233 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e55ae0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 -- 172.21.15.125:0/3663640233 >> 172.21.15.125:0/3663640233 conn(0x5574f8e57d50 msgr2=0x5574f8ea08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 -- 172.21.15.125:0/3663640233 shutdown_connections 2023-10-18T10:40:28.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.834+0000 7fae493e9740 1 -- 172.21.15.125:0/3663640233 wait complete. 2023-10-18T10:40:28.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.835+0000 7fae493e9740 1 -- start start 2023-10-18T10:40:28.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.835+0000 7fae493e9740 1 Processor -- start 2023-10-18T10:40:28.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae493e9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e89490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae493e9740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 0x5574f8e7fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae493e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e973f0 0x5574f8e99410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae493e9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f8dc1760 con 0x5574f8e82b40 2023-10-18T10:40:28.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae493e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f8d7c8f0 con 0x5574f8e973f0 2023-10-18T10:40:28.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae493e9740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574f8d7c590 con 0x5574f8e82f20 2023-10-18T10:40:28.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae493e9740 10 client.0 _ll_register_callbacks cb 0x5574f8db8ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:40:28.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e89490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e89490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50008/0 (socket says 172.21.15.125:50008) 2023-10-18T10:40:28.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 -- 172.21.15.125:0/3474218442 learned_addr learned my addr 172.21.15.125:0/3474218442 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:28.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3f7fe640 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 0x5574f8e7fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae44dec640 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e973f0 0x5574f8e99410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 -- 172.21.15.125:0/3474218442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 msgr2=0x5574f8e7fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:40:28.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 0x5574f8e7fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 -- 172.21.15.125:0/3474218442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e973f0 msgr2=0x5574f8e99410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:40:28.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e973f0 0x5574f8e99410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 -- 172.21.15.125:0/3474218442 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3000fb90 con 0x5574f8e82b40 2023-10-18T10:40:28.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae3ffff640 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e89490 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fae3000bfd0 tx=0x7fae3001e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cde4f2f7ac987ad server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:28.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.836+0000 7fae1effd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:40:28.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae1effd640 1 -- 172.21.15.125:0/3474218442 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3002d070 con 0x5574f8e82b40 2023-10-18T10:40:28.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae1effd640 1 -- 172.21.15.125:0/3474218442 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae3001eda0 con 0x5574f8e82b40 2023-10-18T10:40:28.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 1 -- 172.21.15.125:0/3474218442 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574f8dba740 con 0x5574f8e82b40 2023-10-18T10:40:28.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae1effd640 1 -- 172.21.15.125:0/3474218442 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae30027450 con 0x5574f8e82b40 2023-10-18T10:40:28.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 10 client.7248 fetch_fsmap learned FSMap version 29 2023-10-18T10:40:28.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 1 -- 172.21.15.125:0/3474218442 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5574f8dbfde0 con 0x5574f8e82b40 2023-10-18T10:40:28.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae1effd640 1 -- 172.21.15.125:0/3474218442 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5574f8dba740 con 0x5574f8e82b40 2023-10-18T10:40:28.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae1effd640 1 -- 172.21.15.125:0/3474218442 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fae300275b0 con 0x5574f8e82b40 2023-10-18T10:40:28.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 10 client.7248 fetch_fsmap finished waiting for FSMap version 29 2023-10-18T10:40:28.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 10 client.7248 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:40:28.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 20 client.7248 populate_metadata read hostname 'smithi125' 2023-10-18T10:40:28.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] conn(0x5574f8e869c0 0x5574f8e86da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 4 client.7248 mds_command: new command op to 7071 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:40:28.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae493e9740 1 -- 172.21.15.125:0/3474218442 --> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574f8dba460 con 0x5574f8e869c0 2023-10-18T10:40:28.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae3f7fe640 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] conn(0x5574f8e869c0 0x5574f8e86da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae1effd640 10 client.7248 ms_handle_connect on v2:172.21.15.125:6833/1136111019 2023-10-18T10:40:28.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.837+0000 7fae3f7fe640 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] conn(0x5574f8e869c0 0x5574f8e86da0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c0407eec940702 server_cookie=7ef16f7c3dd8b9f2 in_seq=0 out_seq=0 2023-10-18T10:40:28.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae1effd640 1 -- 172.21.15.125:0/3474218442 <== mds.0 v2:172.21.15.125:6833/1136111019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5574f8dba460 con 0x5574f8e869c0 2023-10-18T10:40:28.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae1effd640 10 client.7248 handle_command_reply: tid=0 2023-10-18T10:40:28.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 client.7248 shutdown 2023-10-18T10:40:28.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 -- 172.21.15.125:0/3474218442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 msgr2=0x5574f8e89490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e89490 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fae3000bfd0 tx=0x7fae3001e710 comp rx=0 tx=0).stop 2023-10-18T10:40:28.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 -- 172.21.15.125:0/3474218442 shutdown_connections 2023-10-18T10:40:28.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] conn(0x5574f8e869c0 0x5574f8e86da0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e973f0 0x5574f8e99410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82f20 0x5574f8e7fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae1effd640 0 client.7248 ms_handle_reset on v2:172.21.15.125:6833/1136111019 2023-10-18T10:40:28.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 --2- 172.21.15.125:0/3474218442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8e89490 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 -- 172.21.15.125:0/3474218442 >> 172.21.15.125:0/3474218442 conn(0x5574f8e56020 msgr2=0x5574f8e599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 -- 172.21.15.125:0/3474218442 shutdown_connections 2023-10-18T10:40:28.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 1 -- 172.21.15.125:0/3474218442 wait complete. 2023-10-18T10:40:28.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.838+0000 7fae493e9740 20 client.7248 trim_cache size 0 max 16384 2023-10-18T10:40:28.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae493e9740 1 Processor -- start 2023-10-18T10:40:28.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae493e9740 1 -- start start 2023-10-18T10:40:28.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae493e9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8ea3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae493e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 0x5574f8ea4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae493e9740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e973f0 0x5574f8e85310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae493e9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f8d42270 con 0x5574f8e82b40 2023-10-18T10:40:28.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae493e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f8e55270 con 0x5574f8e82f20 2023-10-18T10:40:28.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae493e9740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574f8e9b470 con 0x5574f8e973f0 2023-10-18T10:40:28.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8ea3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8ea3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50016/0 (socket says 172.21.15.125:50016) 2023-10-18T10:40:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.854+0000 7fae3ffff640 1 -- 172.21.15.125:0/1427779949 learned_addr learned my addr 172.21.15.125:0/1427779949 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae44dec640 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e973f0 0x5574f8e85310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae3ffff640 1 -- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e973f0 msgr2=0x5574f8e85310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:40:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae3f7fe640 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 0x5574f8ea4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae3ffff640 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e973f0 0x5574f8e85310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae3ffff640 1 -- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 msgr2=0x5574f8ea4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:40:28.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae3ffff640 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 0x5574f8ea4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae3ffff640 1 -- 172.21.15.125:0/1427779949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f8e9a3b0 con 0x5574f8e82b40 2023-10-18T10:40:28.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae44dec640 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e973f0 0x5574f8e85310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:28.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae3ffff640 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8ea3de0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fae300058c0 tx=0x7fae30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31bc4c5fbbbfebd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:28.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae267fc640 1 -- 172.21.15.125:0/1427779949 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30040070 con 0x5574f8e82b40 2023-10-18T10:40:28.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae267fc640 1 -- 172.21.15.125:0/1427779949 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae30027b10 con 0x5574f8e82b40 2023-10-18T10:40:28.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae267fc640 1 -- 172.21.15.125:0/1427779949 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30027690 con 0x5574f8e82b40 2023-10-18T10:40:28.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae493e9740 1 -- 172.21.15.125:0/1427779949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 msgr2=0x5574f8ea3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae493e9740 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8ea3de0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fae300058c0 tx=0x7fae30012000 comp rx=0 tx=0).stop 2023-10-18T10:40:28.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae493e9740 1 -- 172.21.15.125:0/1427779949 shutdown_connections 2023-10-18T10:40:28.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae493e9740 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 0x5574f8ea4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae493e9740 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e973f0 0x5574f8e85310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae493e9740 1 --2- 172.21.15.125:0/1427779949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e82b40 0x5574f8ea3de0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.855+0000 7fae493e9740 1 -- 172.21.15.125:0/1427779949 >> 172.21.15.125:0/1427779949 conn(0x5574f8e56020 msgr2=0x5574f8e7ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.856+0000 7fae493e9740 1 -- 172.21.15.125:0/1427779949 shutdown_connections 2023-10-18T10:40:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.856+0000 7fae493e9740 1 -- 172.21.15.125:0/1427779949 wait complete. 2023-10-18T10:40:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.856+0000 7fae493e9740 1 -- start start 2023-10-18T10:40:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.856+0000 7fae493e9740 1 Processor -- start 2023-10-18T10:40:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae493e9740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82b40 0x5574f8f08470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae493e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 0x5574f8e989c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae493e9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e973f0 0x5574f8e98f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae493e9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f8e81910 con 0x5574f8e973f0 2023-10-18T10:40:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae493e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f8f089b0 con 0x5574f8e82f20 2023-10-18T10:40:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae493e9740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574f8e5ae10 con 0x5574f8e82b40 2023-10-18T10:40:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae493e9740 10 client.0 _ll_register_callbacks cb 0x7fae30015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:40:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae44dec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e973f0 0x5574f8e98f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae44dec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e973f0 0x5574f8e98f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50020/0 (socket says 172.21.15.125:50020) 2023-10-18T10:40:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae44dec640 1 -- 172.21.15.125:0/259710542 learned_addr learned my addr 172.21.15.125:0/259710542 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae3ffff640 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82b40 0x5574f8f08470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae3f7fe640 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 0x5574f8e989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae44dec640 1 -- 172.21.15.125:0/259710542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82b40 msgr2=0x5574f8f08470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:40:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae44dec640 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82b40 0x5574f8f08470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae44dec640 1 -- 172.21.15.125:0/259710542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 msgr2=0x5574f8e989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:40:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae44dec640 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 0x5574f8e989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.857+0000 7fae44dec640 1 -- 172.21.15.125:0/259710542 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574f8dbfde0 con 0x5574f8e973f0 2023-10-18T10:40:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae44dec640 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e973f0 0x5574f8e98f00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fae38001800 tx=0x7fae38058c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5796471b380785b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae1effd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:40:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae1effd640 1 -- 172.21.15.125:0/259710542 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38060a30 con 0x5574f8e973f0 2023-10-18T10:40:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae1effd640 1 -- 172.21.15.125:0/259710542 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae38060b90 con 0x5574f8e973f0 2023-10-18T10:40:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 1 -- 172.21.15.125:0/259710542 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae3001e8a0 con 0x5574f8e973f0 2023-10-18T10:40:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae1effd640 1 -- 172.21.15.125:0/259710542 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae380880d0 con 0x5574f8e973f0 2023-10-18T10:40:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 10 client.7254 fetch_fsmap learned FSMap version 29 2023-10-18T10:40:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 1 -- 172.21.15.125:0/259710542 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5574f8d5ed00 con 0x5574f8e973f0 2023-10-18T10:40:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae1effd640 1 -- 172.21.15.125:0/259710542 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae38074080 con 0x5574f8e973f0 2023-10-18T10:40:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae1effd640 1 -- 172.21.15.125:0/259710542 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fae38022020 con 0x5574f8e973f0 2023-10-18T10:40:28.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 10 client.7254 fetch_fsmap finished waiting for FSMap version 29 2023-10-18T10:40:28.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 10 client.7254 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:40:28.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 20 client.7254 populate_metadata read hostname 'smithi125' 2023-10-18T10:40:28.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] conn(0x5574f8ea1d90 0x5574f8ea2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 4 client.7254 mds_command: new command op to 7071 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:40:28.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae493e9740 1 -- 172.21.15.125:0/259710542 --> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574f8ac36c0 con 0x5574f8ea1d90 2023-10-18T10:40:28.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae3ffff640 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] conn(0x5574f8ea1d90 0x5574f8ea2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.858+0000 7fae1effd640 10 client.7254 ms_handle_connect on v2:172.21.15.125:6833/1136111019 2023-10-18T10:40:28.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae3ffff640 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] conn(0x5574f8ea1d90 0x5574f8ea2170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274721dc40e4092e server_cookie=c54512d9dea01a35 in_seq=0 out_seq=0 2023-10-18T10:40:28.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae1effd640 1 -- 172.21.15.125:0/259710542 <== mds.0 v2:172.21.15.125:6833/1136111019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5574f8ac36c0 con 0x5574f8ea1d90 2023-10-18T10:40:28.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae1effd640 10 client.7254 handle_command_reply: tid=0 2023-10-18T10:40:28.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 client.7254 shutdown 2023-10-18T10:40:28.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 -- 172.21.15.125:0/259710542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e973f0 msgr2=0x5574f8e98f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e973f0 0x5574f8e98f00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fae38001800 tx=0x7fae38058c80 comp rx=0 tx=0).stop 2023-10-18T10:40:28.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 -- 172.21.15.125:0/259710542 shutdown_connections 2023-10-18T10:40:28.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.125:6833/1136111019,v1:172.21.15.125:6835/1136111019] conn(0x5574f8ea1d90 0x5574f8ea2170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5574f8e82f20 0x5574f8e989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae1effd640 0 client.7254 ms_handle_reset on v2:172.21.15.125:6833/1136111019 2023-10-18T10:40:28.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5574f8e973f0 0x5574f8e98f00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 --2- 172.21.15.125:0/259710542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5574f8e82b40 0x5574f8f08470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 -- 172.21.15.125:0/259710542 >> 172.21.15.125:0/259710542 conn(0x5574f8e56020 msgr2=0x5574f8e7ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 -- 172.21.15.125:0/259710542 shutdown_connections 2023-10-18T10:40:28.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 1 -- 172.21.15.125:0/259710542 wait complete. 2023-10-18T10:40:28.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae493e9740 20 client.7254 trim_cache size 0 max 16384 2023-10-18T10:40:28.883 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:40:28.884 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:40:28.884 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:40:28.884 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 7071, 2023-10-18T10:40:28.884 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:40:28.884 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:40:28.884 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:40:28.885 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.9532699549999997, 2023-10-18T10:40:28.885 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 29, 2023-10-18T10:40:28.885 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 84, 2023-10-18T10:40:28.885 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 84, 2023-10-18T10:40:28.885 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 14.717798349000001 2023-10-18T10:40:28.886 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:40:28.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae4684f640 1 -- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fae2c06af60 msgr2=0x7fae2c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae4684f640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fae2c06af60 0x7fae2c06d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae30005fd0 tx=0x7fae30005850 comp rx=0 tx=0).stop 2023-10-18T10:40:28.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae4684f640 1 -- 172.21.15.125:0/840867836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 msgr2=0x7fae400ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.859+0000 7fae4684f640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 0x7fae400ad980 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fae3800cb40 tx=0x7fae3800a060 comp rx=0 tx=0).stop 2023-10-18T10:40:28.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.860+0000 7fae4684f640 1 -- 172.21.15.125:0/840867836 shutdown_connections 2023-10-18T10:40:28.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.860+0000 7fae4684f640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4013e520 0x7fae400ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.860+0000 7fae4684f640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fae2c06af60 0x7fae2c06d420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.860+0000 7fae4684f640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae40141690 0x7fae400ad980 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.860+0000 7fae4684f640 1 --2- 172.21.15.125:0/840867836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4006c0a0 0x7fae401ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.860+0000 7fae4684f640 1 -- 172.21.15.125:0/840867836 >> 172.21.15.125:0/840867836 conn(0x7fae40069930 msgr2=0x7fae4013d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.860+0000 7fae4684f640 1 -- 172.21.15.125:0/840867836 shutdown_connections 2023-10-18T10:40:28.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.860+0000 7fae4684f640 1 -- 172.21.15.125:0/840867836 wait complete. 2023-10-18T10:40:28.890 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:40:28.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.990+0000 7fce54e87640 1 -- 172.21.15.125:0/2563215377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce5006af00 msgr2=0x7fce5006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.990+0000 7fce54e87640 1 --2- 172.21.15.125:0/2563215377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce5006af00 0x7fce5006b2e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fce44009900 tx=0x7fce4402be00 comp rx=0 tx=0).stop 2023-10-18T10:40:28.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.990+0000 7fce54e87640 1 -- 172.21.15.125:0/2563215377 shutdown_connections 2023-10-18T10:40:28.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.990+0000 7fce54e87640 1 --2- 172.21.15.125:0/2563215377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce500abc10 0x7fce500c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.990+0000 7fce54e87640 1 --2- 172.21.15.125:0/2563215377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce5006af00 0x7fce5006b2e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.990+0000 7fce54e87640 1 -- 172.21.15.125:0/2563215377 >> 172.21.15.125:0/2563215377 conn(0x7fce50069930 msgr2=0x7fce50069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:28.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.990+0000 7fce54e87640 1 -- 172.21.15.125:0/2563215377 shutdown_connections 2023-10-18T10:40:28.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.990+0000 7fce54e87640 1 -- 172.21.15.125:0/2563215377 wait complete. 2023-10-18T10:40:28.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce54e87640 1 Processor -- start 2023-10-18T10:40:28.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce54e87640 1 -- start start 2023-10-18T10:40:28.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce54e87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce5006af00 0x7fce5013a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce54e87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce500abc10 0x7fce50070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce54e87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce500c96b0 0x7fce501538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:28.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce54e87640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50071110 con 0x7fce500abc10 2023-10-18T10:40:28.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce54e87640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50070130 con 0x7fce500c96b0 2023-10-18T10:40:28.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce54e87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce50154300 con 0x7fce5006af00 2023-10-18T10:40:28.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce4dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce500abc10 0x7fce50070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:28.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce4dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce500abc10 0x7fce50070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50036/0 (socket says 172.21.15.125:50036) 2023-10-18T10:40:28.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.992+0000 7fce4dd9b640 1 -- 172.21.15.125:0/2657071514 learned_addr learned my addr 172.21.15.125:0/2657071514 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:28.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce4dd9b640 1 -- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce5006af00 msgr2=0x7fce5013a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce4dd9b640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce5006af00 0x7fce5013a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce4dd9b640 1 -- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce500c96b0 msgr2=0x7fce501538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:28.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce4dd9b640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce500c96b0 0x7fce501538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:28.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce4dd9b640 1 -- 172.21.15.125:0/2657071514 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44009560 con 0x7fce500abc10 2023-10-18T10:40:29.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce4dd9b640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce500abc10 0x7fce50070bd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c400 tx=0x7fce4000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:29.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce3effd640 1 -- 172.21.15.125:0/2657071514 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400052d0 con 0x7fce500abc10 2023-10-18T10:40:29.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce3effd640 1 -- 172.21.15.125:0/2657071514 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce400035f0 con 0x7fce500abc10 2023-10-18T10:40:29.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce500703d0 con 0x7fce500abc10 2023-10-18T10:40:29.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce3effd640 1 -- 172.21.15.125:0/2657071514 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40003a00 con 0x7fce500abc10 2023-10-18T10:40:29.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.993+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce50158dd0 con 0x7fce500abc10 2023-10-18T10:40:29.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.994+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce50158f60 con 0x7fce500abc10 2023-10-18T10:40:29.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.994+0000 7fce3effd640 1 -- 172.21.15.125:0/2657071514 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fce40003b60 con 0x7fce500abc10 2023-10-18T10:40:29.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.995+0000 7fce3effd640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce3006af60 0x7fce3006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:29.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.995+0000 7fce3effd640 1 -- 172.21.15.125:0/2657071514 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fce40089040 con 0x7fce500abc10 2023-10-18T10:40:29.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.995+0000 7fce4e59c640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce3006af60 0x7fce3006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:29.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.996+0000 7fce4e59c640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce3006af60 0x7fce3006d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce44035000 tx=0x7fce440046c0 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:29.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:28.999+0000 7fce3effd640 1 -- 172.21.15.125:0/2657071514 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce40053f30 con 0x7fce500abc10 2023-10-18T10:40:29.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.198+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fce5006bf40 con 0x7fce500abc10 2023-10-18T10:40:29.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.198+0000 7fce3effd640 1 -- 172.21.15.125:0/2657071514 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+73 (secure 0 0 0) 0x7fce40017060 con 0x7fce500abc10 2023-10-18T10:40:29.200 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:40:29.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce3006af60 msgr2=0x7fce3006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:29.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce3006af60 0x7fce3006d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce44035000 tx=0x7fce440046c0 comp rx=0 tx=0).stop 2023-10-18T10:40:29.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce500abc10 msgr2=0x7fce50070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:29.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce500abc10 0x7fce50070bd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c400 tx=0x7fce4000c8c0 comp rx=0 tx=0).stop 2023-10-18T10:40:29.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 shutdown_connections 2023-10-18T10:40:29.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce500c96b0 0x7fce501538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:29.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fce3006af60 0x7fce3006d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:29.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce500abc10 0x7fce50070bd0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:29.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 --2- 172.21.15.125:0/2657071514 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce5006af00 0x7fce5013a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:29.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 >> 172.21.15.125:0/2657071514 conn(0x7fce50069930 msgr2=0x7fce5013b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:29.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 shutdown_connections 2023-10-18T10:40:29.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:29.200+0000 7fce54e87640 1 -- 172.21.15.125:0/2657071514 wait complete. 2023-10-18T10:40:29.212 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:40:29.213 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:40:29.213 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:40:29.213 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:40:29.214 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:40:29.214 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:40:29.214 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:40:29.215 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:40:29.215 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:40:29.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:29.216 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:40:29.231 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:40:29.231 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:40:29.231 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:40:29.232 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:40:29.232 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:40:29.232 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:40:29.232 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:40:29.232 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:40:29.233 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:40:29.233 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:40:29.233 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:40:29.233 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:40:29.233 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:40:29.234 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:40:29.234 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:40:29.234 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:40:29.234 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:40:29.234 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 41795sec preferred_lft 41795sec 2023-10-18T10:40:29.235 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:40:29.235 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:40:29.235 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:40:29.235 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:40:29.235 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:40:29.235 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:40:29.236 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T10:40:29.236 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:40:29.236 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:40:29.236 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:40:29.236 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T10:40:29.237 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:40:29.237 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:40:29.237 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:40:29.237 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T10:40:29.237 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:40:29.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:29.238 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:40:29.296 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:40:29.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:29.298 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:40:29.353 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:40:29.354 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:40:29.354 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:40:29.354 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:40:29.354 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:40:29.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:29.355 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:40:29.356 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:40:29.356 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:40:29.357 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:40:29.357 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:40:29.357 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:40:29.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:29.449 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:40:29.464 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:40:29.464 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:40:29.465 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:40:29.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:29.465 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:40:29.531 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:40:29.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:40:29.532 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:40:29.586 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:40:29.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:40:29.587 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:40:29.641 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:40:29.707 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:40:29.763 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:40:29.763 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:40:29.764 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:40:29.764 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:40:29.764 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:40:29.764 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:40:29.764 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:40:29.765 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:40:29.765 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:40:29.765 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:40:29.765 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:40:29.765 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:40:29.766 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:40:29.766 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:40:29.766 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:40:29.766 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:40:29.766 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:40:29.767 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:40:29.767 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:40:29.767 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:40:29.767 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:40:29.767 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:40:29.767 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:40:29.768 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:40:29.768 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:40:29.768 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:40:29.768 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:40:29.768 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:40:29.769 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:40:29.769 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:40:29.769 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:40:29.769 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:40:29.769 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:40:29.770 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:40:29.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:29.770 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:40:29.818 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:40:29.818 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:40:29.861 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:40:29.888 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:40:29.943 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:40:29.943 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:40:29.943 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:40:29.944 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:40:29.944 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:40:29.944 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:40:29.944 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:40:29.944 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:40:29.945 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:40:29.945 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:40:29.945 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:40:29.945 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:40:29.945 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:40:29.946 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:40:29.946 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:40:29.946 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:40:29.946 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:40:29.946 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:40:29.947 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:40:29.947 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:40:29.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:40:29.947 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:40:29.947 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:40:29.947 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:40:29.948 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:40:29.948 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:40:29.948 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:40:29.948 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:40:29.948 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:40:29.949 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:40:29.949 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:40:29.949 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:40:29.949 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:40:29.949 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:40:29.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:29.950 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:40:29.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:40:29.944+0000 7fbb6e77c480 -1 init, newargv = 0x55f1c931ccf0 newargc=15 2023-10-18T10:40:29.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[119373]: starting ceph client 2023-10-18T10:40:29.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[119373]: starting fuse 2023-10-18T10:40:29.965 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:40:29.966 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:40:29.966 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:40:29.966 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:40:29.967 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:40:29.967 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:40:29.968 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:40:29.968 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:40:29.968 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:40:29.969 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:40:29.969 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:40:29.970 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:40:29.970 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:40:29.970 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:40:29.971 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:40:29.971 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:40:29.971 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:40:29.972 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:40:29.972 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:40:29.972 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:40:29.973 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:40:29.973 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:40:29.974 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:40:29.974 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:40:29.974 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:40:29.975 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:40:29.975 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:40:29.975 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:40:29.976 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:40:30.084 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.119373.asok 2023-10-18T10:40:30.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:40:30.095 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.119373.asok 2023-10-18T10:40:30.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:30.096 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.119373.asok status 2023-10-18T10:40:30.229 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:40:30.230 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:40:30.230 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:40:30.230 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:40:30.230 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:40:30.231 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:40:30.231 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:40:30.231 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "119373", 2023-10-18T10:40:30.231 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:40:30.231 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:30.232 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:40:30.232 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:40:30.232 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 7266, 2023-10-18T10:40:30.232 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:40:30.232 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:40:30.233 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:40:30.233 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 7266 2023-10-18T10:40:30.233 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:30.233 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:40:30.233 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:40:30.234 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:40:30.234 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 3615210459 2023-10-18T10:40:30.234 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:30.234 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:30.234 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:40:30.235 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:40:30.235 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:40:30.235 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 3615210459 2023-10-18T10:40:30.235 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:30.235 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.7266 192.168.0.1:0/3615210459", 2023-10-18T10:40:30.235 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/3615210459", 2023-10-18T10:40:30.236 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:40:30.236 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 29, 2023-10-18T10:40:30.236 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 84, 2023-10-18T10:40:30.236 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:40:30.236 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:40:30.237 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:40:30.237 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:40:30.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:30.238 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:40:30.241 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:40:30.257 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:40:30.257 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:40:30.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:30.258 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:40:30.325 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T10:40:30.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.468+0000 7f04b5ff5640 1 -- 172.21.15.125:0/498290583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0498003970 msgr2=0x7f0498003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:30.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.468+0000 7f04b5ff5640 1 --2- 172.21.15.125:0/498290583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0498003970 0x7f0498003d90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009b80 tx=0x7f04980057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.468+0000 7f04affff640 1 -- 172.21.15.125:0/498290583 reap_dead start 2023-10-18T10:40:30.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.468+0000 7f04b5ff5640 1 -- 172.21.15.125:0/498290583 shutdown_connections 2023-10-18T10:40:30.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.468+0000 7f04b5ff5640 1 -- 172.21.15.125:0/498290583 >> 172.21.15.125:0/498290583 conn(0x7f04b00d0d60 msgr2=0x7f04b00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:30.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.468+0000 7f04b5ff5640 1 -- 172.21.15.125:0/498290583 shutdown_connections 2023-10-18T10:40:30.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.468+0000 7f04b5ff5640 1 -- 172.21.15.125:0/498290583 wait complete. 2023-10-18T10:40:30.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.469+0000 7f04b5ff5640 1 Processor -- start 2023-10-18T10:40:30.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.469+0000 7f04b5ff5640 1 -- start start 2023-10-18T10:40:30.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04b5ff5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0498003970 0x7f04b00b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:30.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04b5ff5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b01f10f0 0x7f04b0157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:30.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04b5ff5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04b00b45d0 0x7f04b00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:30.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04b5ff5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00c8fb0 con 0x7f04b00b45d0 2023-10-18T10:40:30.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04b5ff5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b01576a0 con 0x7f04b01f10f0 2023-10-18T10:40:30.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04b5ff5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00c9250 con 0x7f0498003970 2023-10-18T10:40:30.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04b00b45d0 0x7f04b00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:30.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04b00b45d0 0x7f04b00cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50040/0 (socket says 172.21.15.125:50040) 2023-10-18T10:40:30.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 -- 172.21.15.125:0/12319767 learned_addr learned my addr 172.21.15.125:0/12319767 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:30.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04aeffd640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b01f10f0 0x7f04b0157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:30.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04af7fe640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0498003970 0x7f04b00b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:30.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 -- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0498003970 msgr2=0x7f04b00b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:30.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0498003970 0x7f04b00b6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 -- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b01f10f0 msgr2=0x7f04b0157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:30.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b01f10f0 0x7f04b0157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 -- 172.21.15.125:0/12319767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b00c9530 con 0x7f04b00b45d0 2023-10-18T10:40:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04aeffd640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b01f10f0 0x7f04b0157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04af7fe640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0498003970 0x7f04b00b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f04affff640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04b00b45d0 0x7f04b00cc0e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f04b00b6e10 tx=0x7f04a8010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.470+0000 7f0497fff640 1 -- 172.21.15.125:0/12319767 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a800eeb0 con 0x7f04b00b45d0 2023-10-18T10:40:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.471+0000 7f0497fff640 1 -- 172.21.15.125:0/12319767 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f04a8003a60 con 0x7f04b00b45d0 2023-10-18T10:40:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.471+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b00c9810 con 0x7f04b00b45d0 2023-10-18T10:40:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.471+0000 7f0497fff640 1 -- 172.21.15.125:0/12319767 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8003e70 con 0x7f04b00b45d0 2023-10-18T10:40:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.471+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b00b4a50 con 0x7f04b00b45d0 2023-10-18T10:40:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.471+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b00c9d30 con 0x7f04b00b45d0 2023-10-18T10:40:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.472+0000 7f0497fff640 1 -- 172.21.15.125:0/12319767 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f04a8019030 con 0x7f04b00b45d0 2023-10-18T10:40:30.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.472+0000 7f0497fff640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f049c06af90 0x7f049c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.472+0000 7f0497fff640 1 -- 172.21.15.125:0/12319767 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f04a8017070 con 0x7f04b00b45d0 2023-10-18T10:40:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.473+0000 7f04af7fe640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f049c06af90 0x7f049c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.473+0000 7f04af7fe640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f049c06af90 0x7f049c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04a0036000 tx=0x7f04a0035040 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.477+0000 7f0497fff640 1 -- 172.21.15.125:0/12319767 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04b00c9d30 con 0x7f04b00b45d0 2023-10-18T10:40:30.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.662+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 --> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f04b00c93c0 con 0x7f049c06af90 2023-10-18T10:40:30.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.665+0000 7f0497fff640 1 -- 172.21.15.125:0/12319767 <== mgr.6055 v2:172.21.15.139:6832/2642591764 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f04b00c93c0 con 0x7f049c06af90 2023-10-18T10:40:30.666 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:40:30.667 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:30.667 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T10:40:30.667 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:30.667 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:40:30.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f049c06af90 msgr2=0x7f049c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:30.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f049c06af90 0x7f049c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04a0036000 tx=0x7f04a0035040 comp rx=0 tx=0).stop 2023-10-18T10:40:30.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04b00b45d0 msgr2=0x7f04b00cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:30.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04b00b45d0 0x7f04b00cc0e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f04b00b6e10 tx=0x7f04a8010f40 comp rx=0 tx=0).stop 2023-10-18T10:40:30.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 shutdown_connections 2023-10-18T10:40:30.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04b01f10f0 0x7f04b0157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f049c06af90 0x7f049c06d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f04b00b45d0 0x7f04b00cc0e0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 --2- 172.21.15.125:0/12319767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0498003970 0x7f04b00b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 >> 172.21.15.125:0/12319767 conn(0x7f04b00d0d60 msgr2=0x7f04b00ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:30.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 shutdown_connections 2023-10-18T10:40:30.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.666+0000 7f04b5ff5640 1 -- 172.21.15.125:0/12319767 wait complete. 2023-10-18T10:40:30.678 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T10:40:30.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.780+0000 7f77c78e5640 1 -- 172.21.15.125:0/2398428221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 msgr2=0x7f77c0158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:30.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.780+0000 7f77c78e5640 1 --2- 172.21.15.125:0/2398428221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 0x7f77c0158300 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f77c00b10f0 tx=0x7f77bc02bdb0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.780+0000 7f77c78e5640 1 -- 172.21.15.125:0/2398428221 shutdown_connections 2023-10-18T10:40:30.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.780+0000 7f77c78e5640 1 --2- 172.21.15.125:0/2398428221 >> v2:172.21.15.139:3300/0 conn(0x7f77c00b0530 0x7f77c00b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.780+0000 7f77c78e5640 1 --2- 172.21.15.125:0/2398428221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 0x7f77c0158300 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.780+0000 7f77c78e5640 1 -- 172.21.15.125:0/2398428221 >> 172.21.15.125:0/2398428221 conn(0x7f77c0069930 msgr2=0x7f77c0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:30.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.780+0000 7f77c78e5640 1 -- 172.21.15.125:0/2398428221 shutdown_connections 2023-10-18T10:40:30.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.780+0000 7f77c78e5640 1 -- 172.21.15.125:0/2398428221 wait complete. 2023-10-18T10:40:30.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.781+0000 7f77c78e5640 1 Processor -- start 2023-10-18T10:40:30.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.781+0000 7f77c78e5640 1 -- start start 2023-10-18T10:40:30.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c78e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 0x7f77c014c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:30.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c78e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b0530 0x7f77c01ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:30.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c78e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c00cbd20 0x7f77c0156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:30.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c78e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c014d2e0 con 0x7f77c00afa20 2023-10-18T10:40:30.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c78e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c01ef7a0 con 0x7f77c00cbd20 2023-10-18T10:40:30.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c78e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c0156ec0 con 0x7f77c00b0530 2023-10-18T10:40:30.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c5681640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 0x7f77c014c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:30.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c5681640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 0x7f77c014c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50068/0 (socket says 172.21.15.125:50068) 2023-10-18T10:40:30.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c5681640 1 -- 172.21.15.125:0/3008899696 learned_addr learned my addr 172.21.15.125:0/3008899696 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:30.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c4e80640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b0530 0x7f77c01ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:30.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c5e82640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c00cbd20 0x7f77c0156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:30.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c5681640 1 -- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b0530 msgr2=0x7f77c01ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:30.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c5681640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b0530 0x7f77c01ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c5681640 1 -- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c00cbd20 msgr2=0x7f77c0156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.782+0000 7f77c5681640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c00cbd20 0x7f77c0156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.783+0000 7f77c5681640 1 -- 172.21.15.125:0/3008899696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc009560 con 0x7f77c00afa20 2023-10-18T10:40:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.783+0000 7f77c5681640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 0x7f77c014c3e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f77b000b3f0 tx=0x7f77b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.783+0000 7f77b5ffb640 1 -- 172.21.15.125:0/3008899696 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0011830 con 0x7f77c00afa20 2023-10-18T10:40:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.783+0000 7f77b5ffb640 1 -- 172.21.15.125:0/3008899696 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f77b0011a80 con 0x7f77c00afa20 2023-10-18T10:40:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.783+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c0153100 con 0x7f77c00afa20 2023-10-18T10:40:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.783+0000 7f77b5ffb640 1 -- 172.21.15.125:0/3008899696 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0011e90 con 0x7f77c00afa20 2023-10-18T10:40:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.783+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c0158cc0 con 0x7f77c00afa20 2023-10-18T10:40:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.784+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c00690e0 con 0x7f77c00afa20 2023-10-18T10:40:30.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.784+0000 7f77b5ffb640 1 -- 172.21.15.125:0/3008899696 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f77b001a400 con 0x7f77c00afa20 2023-10-18T10:40:30.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.785+0000 7f77b5ffb640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f77a406af60 0x7f77a406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:30.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.785+0000 7f77b5ffb640 1 -- 172.21.15.125:0/3008899696 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f77b00897c0 con 0x7f77c00afa20 2023-10-18T10:40:30.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.785+0000 7f77c4e80640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f77a406af60 0x7f77a406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:30.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.785+0000 7f77c4e80640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f77a406af60 0x7f77a406d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77bc035000 tx=0x7f77bc004730 comp rx=0 tx=0).ready entity=mgr.6055 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:30.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:30.789+0000 7f77b5ffb640 1 -- 172.21.15.125:0/3008899696 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77b0054760 con 0x7f77c00afa20 2023-10-18T10:40:31.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.000+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f77c006f700 con 0x7f77c00afa20 2023-10-18T10:40:31.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.382+0000 7f77b5ffb640 1 -- 172.21.15.125:0/3008899696 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v33) v1 ==== 92+0+0 (secure 0 0 0) 0x7f77b0057d80 con 0x7f77c00afa20 2023-10-18T10:40:31.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f77a406af60 msgr2=0x7f77a406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f77a406af60 0x7f77a406d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77bc035000 tx=0x7f77bc004730 comp rx=0 tx=0).stop 2023-10-18T10:40:31.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 msgr2=0x7f77c014c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 0x7f77c014c3e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f77b000b3f0 tx=0x7f77b000b8b0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 shutdown_connections 2023-10-18T10:40:31.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c00cbd20 0x7f77c0156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f77a406af60 0x7f77a406d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b0530 0x7f77c01ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 --2- 172.21.15.125:0/3008899696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77c00afa20 0x7f77c014c3e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 >> 172.21.15.125:0/3008899696 conn(0x7f77c0069930 msgr2=0x7f77c00c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:31.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 shutdown_connections 2023-10-18T10:40:31.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.385+0000 7f77c78e5640 1 -- 172.21.15.125:0/3008899696 wait complete. 2023-10-18T10:40:31.400 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T10:40:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.507+0000 7f7f80d3c640 1 -- 172.21.15.125:0/1237366315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0d1910 msgr2=0x7f7f7c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.507+0000 7f7f80d3c640 1 --2- 172.21.15.125:0/1237366315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0d1910 0x7f7f7c0b0650 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7f74005fd0 tx=0x7f7f7402c0e0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.507+0000 7f7f80d3c640 1 -- 172.21.15.125:0/1237366315 shutdown_connections 2023-10-18T10:40:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.507+0000 7f7f80d3c640 1 --2- 172.21.15.125:0/1237366315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0d1910 0x7f7f7c0b0650 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.507+0000 7f7f80d3c640 1 -- 172.21.15.125:0/1237366315 >> 172.21.15.125:0/1237366315 conn(0x7f7f7c0d0d60 msgr2=0x7f7f7c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:31.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.507+0000 7f7f80d3c640 1 -- 172.21.15.125:0/1237366315 shutdown_connections 2023-10-18T10:40:31.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.507+0000 7f7f80d3c640 1 -- 172.21.15.125:0/1237366315 wait complete. 2023-10-18T10:40:31.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.508+0000 7f7f80d3c640 1 Processor -- start 2023-10-18T10:40:31.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.508+0000 7f7f80d3c640 1 -- start start 2023-10-18T10:40:31.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f80d3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0cf340 0x7f7f7c158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:31.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f80d3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f7c0d1910 0x7f7f7c0b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:31.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f80d3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f7c15b2a0 0x7f7f7c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:31.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f80d3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c87f0 con 0x7f7f7c0cf340 2023-10-18T10:40:31.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f80d3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c158580 con 0x7f7f7c15b2a0 2023-10-18T10:40:31.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f80d3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c89e0 con 0x7f7f7c0d1910 2023-10-18T10:40:31.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0cf340 0x7f7f7c158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:31.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0cf340 0x7f7f7c158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50110/0 (socket says 172.21.15.125:50110) 2023-10-18T10:40:31.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 -- 172.21.15.125:0/2389768214 learned_addr learned my addr 172.21.15.125:0/2389768214 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:31.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f79d9b640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f7c0d1910 0x7f7f7c0b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:31.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7ad9d640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f7c15b2a0 0x7f7f7c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:31.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 -- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f7c0d1910 msgr2=0x7f7f7c0b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f7c0d1910 0x7f7f7c0b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 -- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f7c15b2a0 msgr2=0x7f7f7c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f7c15b2a0 0x7f7f7c0b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 -- 172.21.15.125:0/2389768214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009840 con 0x7f7f7c0cf340 2023-10-18T10:40:31.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f79d9b640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f7c0d1910 0x7f7f7c0b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:31.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7ad9d640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f7c15b2a0 0x7f7f7c0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:31.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.509+0000 7f7f7a59c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0cf340 0x7f7f7c158040 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00c400 tx=0x7f7f6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:31.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.510+0000 7f7f6affd640 1 -- 172.21.15.125:0/2389768214 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0052d0 con 0x7f7f7c0cf340 2023-10-18T10:40:31.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.510+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c0c8cc0 con 0x7f7f7c0cf340 2023-10-18T10:40:31.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.510+0000 7f7f6affd640 1 -- 172.21.15.125:0/2389768214 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7f6c007020 con 0x7f7f7c0cf340 2023-10-18T10:40:31.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.510+0000 7f7f6affd640 1 -- 172.21.15.125:0/2389768214 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c007470 con 0x7f7f7c0cf340 2023-10-18T10:40:31.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.510+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c0c9280 con 0x7f7f7c0cf340 2023-10-18T10:40:31.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.511+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c0690e0 con 0x7f7f7c0cf340 2023-10-18T10:40:31.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.512+0000 7f7f6affd640 1 -- 172.21.15.125:0/2389768214 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 33) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f7f6c0067d0 con 0x7f7f7c0cf340 2023-10-18T10:40:31.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.512+0000 7f7f6affd640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7f5c06afb0 0x7f7f5c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:31.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.512+0000 7f7f6affd640 1 -- 172.21.15.125:0/2389768214 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7f6c089040 con 0x7f7f7c0cf340 2023-10-18T10:40:31.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.512+0000 7f7f79d9b640 1 -- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7f5c06afb0 msgr2=0x7f7f5c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2642591764 2023-10-18T10:40:31.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.512+0000 7f7f79d9b640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7f5c06afb0 0x7f7f5c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:31.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.516+0000 7f7f6affd640 1 -- 172.21.15.125:0/2389768214 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f6c053e20 con 0x7f7f7c0cf340 2023-10-18T10:40:31.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.690+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7f7c0d0b40 con 0x7f7f7c0cf340 2023-10-18T10:40:31.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.695+0000 7f7f6affd640 1 -- 172.21.15.125:0/2389768214 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v40)=0 v40) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7f6c057440 con 0x7f7f7c0cf340 2023-10-18T10:40:31.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.697+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7f5c06afb0 msgr2=0x7f7f5c06d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:31.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.697+0000 7f7f80d3c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7f5c06afb0 0x7f7f5c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.697+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0cf340 msgr2=0x7f7f7c158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.697+0000 7f7f80d3c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0cf340 0x7f7f7c158040 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00c400 tx=0x7f7f6c00c8c0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.698+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 shutdown_connections 2023-10-18T10:40:31.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.698+0000 7f7f80d3c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f7c15b2a0 0x7f7f7c0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.698+0000 7f7f80d3c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f7f5c06afb0 0x7f7f5c06d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.698+0000 7f7f80d3c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f7c0d1910 0x7f7f7c0b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.698+0000 7f7f80d3c640 1 --2- 172.21.15.125:0/2389768214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f7c0cf340 0x7f7f7c158040 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.698+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 >> 172.21.15.125:0/2389768214 conn(0x7f7f7c0d0d60 msgr2=0x7f7f7c0b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:31.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.698+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 shutdown_connections 2023-10-18T10:40:31.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.698+0000 7f7f80d3c640 1 -- 172.21.15.125:0/2389768214 wait complete. 2023-10-18T10:40:31.710 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T10:40:31.771 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:31.769+0000 7f97a8c26040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:40:31.779 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:31.777+0000 7f6a3843a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:40:31.816 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:31.814+0000 7f97a8c26040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:40:31.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.816+0000 7fda1b939640 1 -- 172.21.15.125:0/942289522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda140cf340 msgr2=0x7fda140d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.816+0000 7fda1b939640 1 --2- 172.21.15.125:0/942289522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda140cf340 0x7fda140d1910 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fda1408a810 tx=0x7fda0402bcf0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.816+0000 7fda1b939640 1 -- 172.21.15.125:0/942289522 shutdown_connections 2023-10-18T10:40:31.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.816+0000 7fda1b939640 1 --2- 172.21.15.125:0/942289522 >> v2:172.21.15.139:3300/0 conn(0x7fda1415b2a0 0x7fda140cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.816+0000 7fda1b939640 1 --2- 172.21.15.125:0/942289522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda140cf340 0x7fda140d1910 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.816+0000 7fda1b939640 1 -- 172.21.15.125:0/942289522 >> 172.21.15.125:0/942289522 conn(0x7fda140d0d60 msgr2=0x7fda140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:31.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.816+0000 7fda1b939640 1 -- 172.21.15.125:0/942289522 shutdown_connections 2023-10-18T10:40:31.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.816+0000 7fda1b939640 1 -- 172.21.15.125:0/942289522 wait complete. 2023-10-18T10:40:31.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.817+0000 7fda1b939640 1 Processor -- start 2023-10-18T10:40:31.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.817+0000 7fda1b939640 1 -- start start 2023-10-18T10:40:31.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda1b939640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda140ceb40 0x7fda140b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:31.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda1b939640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda140cf340 0x7fda14158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:31.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda1b939640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda1415b2a0 0x7fda141585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:31.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda1b939640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141f5b70 con 0x7fda1415b2a0 2023-10-18T10:40:31.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda1b939640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141f4db0 con 0x7fda140cf340 2023-10-18T10:40:31.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda1b939640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda141f5090 con 0x7fda140ceb40 2023-10-18T10:40:31.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda19ed6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda1415b2a0 0x7fda141585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:31.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda19ed6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda1415b2a0 0x7fda141585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50128/0 (socket says 172.21.15.125:50128) 2023-10-18T10:40:31.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda19ed6640 1 -- 172.21.15.125:0/2825574441 learned_addr learned my addr 172.21.15.125:0/2825574441 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:31.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda196d5640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda140ceb40 0x7fda140b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:31.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda18ed4640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda140cf340 0x7fda14158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:31.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda19ed6640 1 -- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda140ceb40 msgr2=0x7fda140b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda19ed6640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda140ceb40 0x7fda140b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda19ed6640 1 -- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda140cf340 msgr2=0x7fda14158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:31.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda19ed6640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda140cf340 0x7fda14158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:31.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda19ed6640 1 -- 172.21.15.125:0/2825574441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04009560 con 0x7fda1415b2a0 2023-10-18T10:40:31.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.818+0000 7fda18ed4640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda140cf340 0x7fda14158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:31.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.819+0000 7fda19ed6640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda1415b2a0 0x7fda141585d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00c780 tx=0x7fda0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:31.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.819+0000 7fda09ffb640 1 -- 172.21.15.125:0/2825574441 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c004b60 con 0x7fda1415b2a0 2023-10-18T10:40:31.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.819+0000 7fda09ffb640 1 -- 172.21.15.125:0/2825574441 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda0c00ce90 con 0x7fda1415b2a0 2023-10-18T10:40:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.819+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda141f5370 con 0x7fda1415b2a0 2023-10-18T10:40:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.819+0000 7fda09ffb640 1 -- 172.21.15.125:0/2825574441 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00aab0 con 0x7fda1415b2a0 2023-10-18T10:40:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.819+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda141f2570 con 0x7fda1415b2a0 2023-10-18T10:40:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.820+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda141f5890 con 0x7fda1415b2a0 2023-10-18T10:40:31.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.820+0000 7fda09ffb640 1 -- 172.21.15.125:0/2825574441 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 33) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fda0c00ac10 con 0x7fda1415b2a0 2023-10-18T10:40:31.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.821+0000 7fda09ffb640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd9f806afb0 0x7fd9f806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:31.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.821+0000 7fda196d5640 1 -- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd9f806afb0 msgr2=0x7fd9f806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2642591764 2023-10-18T10:40:31.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.821+0000 7fda09ffb640 1 -- 172.21.15.125:0/2825574441 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fda0c089cb0 con 0x7fda1415b2a0 2023-10-18T10:40:31.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.821+0000 7fda196d5640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd9f806afb0 0x7fd9f806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:31.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:31.829+0000 7fda09ffb640 1 -- 172.21.15.125:0/2825574441 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda141f5890 con 0x7fda1415b2a0 2023-10-18T10:40:31.831 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:31.829+0000 7f6a3843a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:40:32.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.003+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fda141f5890 con 0x7fda1415b2a0 2023-10-18T10:40:32.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.008+0000 7fda09ffb640 1 -- 172.21.15.125:0/2825574441 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v41)=0 v41) v1 ==== 133+0+0 (secure 0 0 0) 0x7fda141f5890 con 0x7fda1415b2a0 2023-10-18T10:40:32.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.010+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd9f806afb0 msgr2=0x7fd9f806d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:32.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.010+0000 7fda1b939640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd9f806afb0 0x7fd9f806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.010+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda1415b2a0 msgr2=0x7fda141585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:32.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.010+0000 7fda1b939640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda1415b2a0 0x7fda141585d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00c780 tx=0x7fda0c00cc40 comp rx=0 tx=0).stop 2023-10-18T10:40:32.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.011+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 shutdown_connections 2023-10-18T10:40:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.011+0000 7fda1b939640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda140cf340 0x7fda14158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.011+0000 7fda1b939640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7fd9f806afb0 0x7fd9f806d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.011+0000 7fda1b939640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda1415b2a0 0x7fda141585d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.011+0000 7fda1b939640 1 --2- 172.21.15.125:0/2825574441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda140ceb40 0x7fda140b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.011+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 >> 172.21.15.125:0/2825574441 conn(0x7fda140d0d60 msgr2=0x7fda140ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:32.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.011+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 shutdown_connections 2023-10-18T10:40:32.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.011+0000 7fda1b939640 1 -- 172.21.15.125:0/2825574441 wait complete. 2023-10-18T10:40:32.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:32.024 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T10:40:32.043 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-18T10:40:32.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.188+0000 7f82bfa1a640 1 -- 172.21.15.125:0/3609879703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82b80cf2a0 msgr2=0x7f82b80cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:32.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.188+0000 7f82bfa1a640 1 --2- 172.21.15.125:0/3609879703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82b80cf2a0 0x7f82b80cdef0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f82a8005fd0 tx=0x7f82a802be10 comp rx=0 tx=0).stop 2023-10-18T10:40:32.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.188+0000 7f82bfa1a640 1 -- 172.21.15.125:0/3609879703 shutdown_connections 2023-10-18T10:40:32.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.188+0000 7f82bfa1a640 1 --2- 172.21.15.125:0/3609879703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82b80b1180 0x7f82b80b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.188+0000 7f82bfa1a640 1 --2- 172.21.15.125:0/3609879703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82b80cf2a0 0x7f82b80cdef0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.188+0000 7f82bfa1a640 1 -- 172.21.15.125:0/3609879703 >> 172.21.15.125:0/3609879703 conn(0x7f82b809a010 msgr2=0x7f82b80d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:32.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.188+0000 7f82bfa1a640 1 -- 172.21.15.125:0/3609879703 shutdown_connections 2023-10-18T10:40:32.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.188+0000 7f82bfa1a640 1 -- 172.21.15.125:0/3609879703 wait complete. 2023-10-18T10:40:32.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.189+0000 7f82bfa1a640 1 Processor -- start 2023-10-18T10:40:32.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bfa1a640 1 -- start start 2023-10-18T10:40:32.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bfa1a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82b80b1180 0x7f82b80c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:32.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bfa1a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b80cf2a0 0x7f82b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:32.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bfa1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82b815db10 0x7f82b80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:32.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bfa1a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80c4810 con 0x7f82b815db10 2023-10-18T10:40:32.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bfa1a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b814dcb0 con 0x7f82b80cf2a0 2023-10-18T10:40:32.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bfa1a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80c7780 con 0x7f82b80b1180 2023-10-18T10:40:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bdfb7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82b815db10 0x7f82b80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bdfb7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82b815db10 0x7f82b80c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50158/0 (socket says 172.21.15.125:50158) 2023-10-18T10:40:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bdfb7640 1 -- 172.21.15.125:0/1678019249 learned_addr learned my addr 172.21.15.125:0/1678019249 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bd7b6640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82b80b1180 0x7f82b80c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.190+0000 7f82bcfb5640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b80cf2a0 0x7f82b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82bdfb7640 1 -- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82b80b1180 msgr2=0x7f82b80c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82bdfb7640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82b80b1180 0x7f82b80c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82bdfb7640 1 -- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b80cf2a0 msgr2=0x7f82b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82bdfb7640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b80cf2a0 0x7f82b815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82bdfb7640 1 -- 172.21.15.125:0/1678019249 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8009560 con 0x7f82b815db10 2023-10-18T10:40:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82bdfb7640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82b815db10 0x7f82b80c7db0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f82b000da70 tx=0x7f82b000df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000bdd0 con 0x7f82b815db10 2023-10-18T10:40:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82b0003a60 con 0x7f82b815db10 2023-10-18T10:40:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82bfa1a640 1 -- 172.21.15.125:0/1678019249 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b8151dd0 con 0x7f82b815db10 2023-10-18T10:40:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0003e70 con 0x7f82b815db10 2023-10-18T10:40:32.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.191+0000 7f82bfa1a640 1 -- 172.21.15.125:0/1678019249 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b8152390 con 0x7f82b815db10 2023-10-18T10:40:32.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.192+0000 7f82bfa1a640 1 -- 172.21.15.125:0/1678019249 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b8066960 con 0x7f82b815db10 2023-10-18T10:40:32.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.193+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 33) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f82b0004000 con 0x7f82b815db10 2023-10-18T10:40:32.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.193+0000 7f82adffb640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 0x7f829c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:32.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.193+0000 7f82bd7b6640 1 -- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 msgr2=0x7f829c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2642591764 2023-10-18T10:40:32.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.193+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f82b0089f00 con 0x7f82b815db10 2023-10-18T10:40:32.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.193+0000 7f82bd7b6640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 0x7f829c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.201+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82b0054d00 con 0x7f82b815db10 2023-10-18T10:40:32.208 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.206+0000 7f97a8c26040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:40:32.247 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.246+0000 7f6a3843a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:40:32.252 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.251+0000 7f97a8c26040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:40:32.294 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.292+0000 7f6a3843a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:40:32.295 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.294+0000 7f97a8c26040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:40:32.338 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.336+0000 7f6a3843a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:40:32.347 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.345+0000 7f97a8c26040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:40:32.389 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.387+0000 7f97a8c26040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:40:32.391 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.389+0000 7f6a3843a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:40:32.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.394+0000 7f82bd7b6640 1 -- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 msgr2=0x7f829c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2642591764 2023-10-18T10:40:32.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.394+0000 7f82bd7b6640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 0x7f829c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:40:32.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.396+0000 7f82bfa1a640 1 -- 172.21.15.125:0/1678019249 --> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f82b80b1b50 con 0x7f829c06afb0 2023-10-18T10:40:32.434 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.432+0000 7f6a3843a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:40:32.490 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.486+0000 7f97a8c26040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:40:32.532 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.530+0000 7f6a3843a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:40:32.541 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.539+0000 7f97a8c26040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:32.583 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.581+0000 7f97a8c26040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:40:32.585 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.583+0000 7f6a3843a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:32.627 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.625+0000 7f6a3843a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:40:32.631 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.630+0000 7f97a8c26040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:40:32.674 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.672+0000 7f97a8c26040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:40:32.674 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.673+0000 7f6a3843a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:40:32.717 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:32.715+0000 7f6a3843a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:40:32.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.795+0000 7f82bd7b6640 1 -- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 msgr2=0x7f829c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2642591764 2023-10-18T10:40:32.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:32.795+0000 7f82bd7b6640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 0x7f829c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:40:32.990 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:32.989+0000 7f97a8c26040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:40:33.034 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.032+0000 7f6a3843a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:40:33.122 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:33.120+0000 7f97a8c26040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:40:33.164 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.162+0000 7f6a3843a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:40:33.213 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:33.212+0000 7f97a8c26040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:40:33.255 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:33.254+0000 7f97a8c26040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:40:33.257 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.256+0000 7f6a3843a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:40:33.300 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.298+0000 7f6a3843a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:40:33.350 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:33.348+0000 7f97a8c26040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:33.395 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.394+0000 7f6a3843a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:33.443 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:33.441+0000 7f97a8c26040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:40:33.486 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.485+0000 7f6a3843a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:40:33.580 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:33.578+0000 7f97a8c26040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:33.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:33.595+0000 7f82bd7b6640 1 -- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 msgr2=0x7f829c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2642591764 2023-10-18T10:40:33.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:33.595+0000 7f82bd7b6640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 0x7f829c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:40:33.622 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:33.620+0000 7f97a8c26040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:33.624 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.621+0000 7f6a3843a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:33.666 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.663+0000 7f6a3843a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:33.727 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:33.725+0000 7f97a8c26040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:40:33.770 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:33.768+0000 7f6a3843a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:40:34.341 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.339+0000 7f97a8c26040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:40:34.387 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:34.385+0000 7f6a3843a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:40:34.601 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.599+0000 7f97a8c26040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:40:34.648 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:34.645+0000 7f6a3843a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:40:34.649 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.647+0000 7f97a8c26040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:40:34.696 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:34.694+0000 7f6a3843a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:40:34.751 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.749+0000 7f97a8c26040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:40:34.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:34.770+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 34) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f82b00882e0 con 0x7f82b815db10 2023-10-18T10:40:34.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:34.771+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 msgr2=0x7f829c06d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:34.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:34.771+0000 7f82adffb640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/2642591764,v1:172.21.15.139:6833/2642591764] conn(0x7f829c06afb0 0x7f829c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:34.801 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:34.798+0000 7f6a3843a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:40:34.883 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f972831d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.883 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f972831d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f972831d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f9724ad6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f972831d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f972831d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f9724ad6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f9724ad6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f9724ad6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:34.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:34.881+0000 7f9724ad6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:35.540 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:40:35.543 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:40:35.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.772+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 35) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f82b00550d0 con 0x7f82b815db10 2023-10-18T10:40:35.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.772+0000 7f82adffb640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f829c035850 0x7f829c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:35.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.772+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 --> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f829c035c30 con 0x7f829c035850 2023-10-18T10:40:35.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.775+0000 7f82bd7b6640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f829c035850 0x7f829c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:35.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.777+0000 7f82bd7b6640 1 --2- 172.21.15.125:0/1678019249 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f829c035850 0x7f829c075380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f82a8035000 tx=0x7f82a8004730 comp rx=0 tx=0).ready entity=mgr.5912 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:35.786 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:35.784+0000 7f972f32b640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-18T10:40:35.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.785+0000 7f82adffb640 1 -- 172.21.15.125:0/1678019249 <== mgr.5912 v2:172.21.15.139:6832/3064166201 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f829c035c30 con 0x7f829c035850 2023-10-18T10:40:35.787 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-18T10:40:35.793 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T10:40:35.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:35.795 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-18T10:40:35.815 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T10:40:35.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.961+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3627809399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 msgr2=0x7f4a78158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:35.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.961+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3627809399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 0x7f4a78158300 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4a780cc490 tx=0x7f4a6802be40 comp rx=0 tx=0).stop 2023-10-18T10:40:35.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.961+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3627809399 shutdown_connections 2023-10-18T10:40:35.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.961+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3627809399 >> v2:172.21.15.139:3300/0 conn(0x7f4a780afa20 0x7f4a780705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:35.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.961+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3627809399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 0x7f4a78158300 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:35.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.961+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3627809399 >> 172.21.15.125:0/3627809399 conn(0x7f4a78069930 msgr2=0x7f4a78069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:35.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.961+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3627809399 shutdown_connections 2023-10-18T10:40:35.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.961+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3627809399 wait complete. 2023-10-18T10:40:35.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.962+0000 7f4a7dd97640 1 Processor -- start 2023-10-18T10:40:35.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a7dd97640 1 -- start start 2023-10-18T10:40:35.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a7dd97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a780afa20 0x7f4a78137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:35.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a7dd97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a780b0530 0x7f4a7813ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:35.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a7dd97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 0x7f4a781562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:35.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a7dd97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78150f20 con 0x7f4a780cbd20 2023-10-18T10:40:35.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a7dd97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78138040 con 0x7f4a780b0530 2023-10-18T10:40:35.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a7dd97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7814fe80 con 0x7f4a780afa20 2023-10-18T10:40:35.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a77fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 0x7f4a781562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:35.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a77fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 0x7f4a781562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50180/0 (socket says 172.21.15.125:50180) 2023-10-18T10:40:35.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a77fff640 1 -- 172.21.15.125:0/3230518931 learned_addr learned my addr 172.21.15.125:0/3230518931 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:35.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.963+0000 7f4a76ffd640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a780b0530 0x7f4a7813ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:35.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a777fe640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a780afa20 0x7f4a78137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:35.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a77fff640 1 -- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a780afa20 msgr2=0x7f4a78137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:35.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a77fff640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a780afa20 0x7f4a78137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:35.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a77fff640 1 -- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a780b0530 msgr2=0x7f4a7813ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:35.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a77fff640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a780b0530 0x7f4a7813ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:35.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a77fff640 1 -- 172.21.15.125:0/3230518931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68009560 con 0x7f4a780cbd20 2023-10-18T10:40:35.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a77fff640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 0x7f4a781562c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c7a0 tx=0x7f4a7000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:35.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a5ffff640 1 -- 172.21.15.125:0/3230518931 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70003b70 con 0x7f4a780cbd20 2023-10-18T10:40:35.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a5ffff640 1 -- 172.21.15.125:0/3230518931 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a70003cf0 con 0x7f4a780cbd20 2023-10-18T10:40:35.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a78150160 con 0x7f4a780cbd20 2023-10-18T10:40:35.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a5ffff640 1 -- 172.21.15.125:0/3230518931 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000aab0 con 0x7f4a780cbd20 2023-10-18T10:40:35.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.964+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a78150720 con 0x7f4a780cbd20 2023-10-18T10:40:35.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.965+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78072170 con 0x7f4a780cbd20 2023-10-18T10:40:35.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.966+0000 7f4a5ffff640 1 -- 172.21.15.125:0/3230518931 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 35) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f4a7000ac10 con 0x7f4a780cbd20 2023-10-18T10:40:35.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.966+0000 7f4a5ffff640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f4a6406ad10 0x7f4a6406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:35.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.966+0000 7f4a5ffff640 1 -- 172.21.15.125:0/3230518931 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f4a700893f0 con 0x7f4a780cbd20 2023-10-18T10:40:35.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.966+0000 7f4a777fe640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f4a6406ad10 0x7f4a6406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:35.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.967+0000 7f4a777fe640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f4a6406ad10 0x7f4a6406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a68035000 tx=0x7f4a68004730 comp rx=0 tx=0).ready entity=mgr.5912 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:35.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:35.970+0000 7f4a5ffff640 1 -- 172.21.15.125:0/3230518931 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a70054560 con 0x7f4a780cbd20 2023-10-18T10:40:36.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.183+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f4a7815abb0 con 0x7f4a780cbd20 2023-10-18T10:40:36.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.775+0000 7f4a5ffff640 1 -- 172.21.15.125:0/3230518931 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v36) v1 ==== 93+0+0 (secure 0 0 0) 0x7f4a70057b80 con 0x7f4a780cbd20 2023-10-18T10:40:36.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f4a6406ad10 msgr2=0x7f4a6406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:36.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f4a6406ad10 0x7f4a6406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a68035000 tx=0x7f4a68004730 comp rx=0 tx=0).stop 2023-10-18T10:40:36.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 msgr2=0x7f4a781562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:36.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 0x7f4a781562c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c7a0 tx=0x7f4a7000cc60 comp rx=0 tx=0).stop 2023-10-18T10:40:36.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 shutdown_connections 2023-10-18T10:40:36.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f4a6406ad10 0x7f4a6406d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:36.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a780b0530 0x7f4a7813ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:36.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780cbd20 0x7f4a781562c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:36.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 --2- 172.21.15.125:0/3230518931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a780afa20 0x7f4a78137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:36.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 >> 172.21.15.125:0/3230518931 conn(0x7f4a78069930 msgr2=0x7f4a780ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:36.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 shutdown_connections 2023-10-18T10:40:36.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:36.778+0000 7f4a7dd97640 1 -- 172.21.15.125:0/3230518931 wait complete. 2023-10-18T10:40:36.794 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:40:36.795 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:40:36.845 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:40:36.887 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:40:36.887 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:40:36.888 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:36.904 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:37.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.048+0000 7f2784ec2640 1 -- 172.21.15.125:0/2008145824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27800ceb40 msgr2=0x7f27800b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.048+0000 7f2784ec2640 1 --2- 172.21.15.125:0/2008145824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27800ceb40 0x7f27800b0650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2778005fd0 tx=0x7f277802c050 comp rx=0 tx=0).stop 2023-10-18T10:40:37.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.048+0000 7f2784ec2640 1 -- 172.21.15.125:0/2008145824 shutdown_connections 2023-10-18T10:40:37.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.048+0000 7f2784ec2640 1 --2- 172.21.15.125:0/2008145824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27800ceb40 0x7f27800b0650 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.048+0000 7f2784ec2640 1 --2- 172.21.15.125:0/2008145824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27800cf340 0x7f27800d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.048+0000 7f2784ec2640 1 -- 172.21.15.125:0/2008145824 >> 172.21.15.125:0/2008145824 conn(0x7f27800d0d60 msgr2=0x7f27800d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:37.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.048+0000 7f2784ec2640 1 -- 172.21.15.125:0/2008145824 shutdown_connections 2023-10-18T10:40:37.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.048+0000 7f2784ec2640 1 -- 172.21.15.125:0/2008145824 wait complete. 2023-10-18T10:40:37.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.049+0000 7f2784ec2640 1 Processor -- start 2023-10-18T10:40:37.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.049+0000 7f2784ec2640 1 -- start start 2023-10-18T10:40:37.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f2784ec2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27800ceb40 0x7f278015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f2784ec2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27800cf340 0x7f27801f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f2784ec2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f278015b2a0 0x7f27801f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f2784ec2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2780155f60 con 0x7f27800ceb40 2023-10-18T10:40:37.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f2784ec2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27801f58f0 con 0x7f27800cf340 2023-10-18T10:40:37.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f2784ec2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278015c6a0 con 0x7f278015b2a0 2023-10-18T10:40:37.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27800ceb40 0x7f278015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27800ceb40 0x7f278015c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50192/0 (socket says 172.21.15.125:50192) 2023-10-18T10:40:37.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277e59c640 1 -- 172.21.15.125:0/1770597172 learned_addr learned my addr 172.21.15.125:0/1770597172 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:37.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277ed9d640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f278015b2a0 0x7f27801f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f2777fff640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27800cf340 0x7f27801f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277e59c640 1 -- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f278015b2a0 msgr2=0x7f27801f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277e59c640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f278015b2a0 0x7f27801f0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277e59c640 1 -- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27800cf340 msgr2=0x7f27801f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277e59c640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27800cf340 0x7f27801f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.050+0000 7f277e59c640 1 -- 172.21.15.125:0/1770597172 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770009490 con 0x7f27800ceb40 2023-10-18T10:40:37.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.051+0000 7f277e59c640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27800ceb40 0x7f278015c0c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f277000ea60 tx=0x7f277000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:37.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.051+0000 7f2776ffd640 1 -- 172.21.15.125:0/1770597172 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000cde0 con 0x7f27800ceb40 2023-10-18T10:40:37.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.051+0000 7f2776ffd640 1 -- 172.21.15.125:0/1770597172 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2770004020 con 0x7f27800ceb40 2023-10-18T10:40:37.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.051+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2778009840 con 0x7f27800ceb40 2023-10-18T10:40:37.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.051+0000 7f2776ffd640 1 -- 172.21.15.125:0/1770597172 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770004410 con 0x7f27800ceb40 2023-10-18T10:40:37.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.051+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278015cc60 con 0x7f27800ceb40 2023-10-18T10:40:37.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.052+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27800690e0 con 0x7f27800ceb40 2023-10-18T10:40:37.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.052+0000 7f2776ffd640 1 -- 172.21.15.125:0/1770597172 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f27700035f0 con 0x7f27800ceb40 2023-10-18T10:40:37.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.053+0000 7f2776ffd640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f276006ac90 0x7f276006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.053+0000 7f2777fff640 1 -- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f276006ac90 msgr2=0x7f276006d150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:37.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.053+0000 7f2776ffd640 1 -- 172.21.15.125:0/1770597172 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f2770014070 con 0x7f27800ceb40 2023-10-18T10:40:37.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.053+0000 7f2777fff640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f276006ac90 0x7f276006d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:37.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.061+0000 7f2776ffd640 1 -- 172.21.15.125:0/1770597172 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27700551e0 con 0x7f27800ceb40 2023-10-18T10:40:37.161 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.160+0000 7f73fac25040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:40:37.176 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.175+0000 7fcec9e2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:40:37.206 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.204+0000 7f73fac25040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:40:37.223 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.221+0000 7fcec9e2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:40:37.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.235+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27800cb9b0 con 0x7f27800ceb40 2023-10-18T10:40:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.237+0000 7f2776ffd640 1 -- 172.21.15.125:0/1770597172 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17272 (secure 0 0 0) 0x7f2770058800 con 0x7f27800ceb40 2023-10-18T10:40:37.239 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:37.240 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":85,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:34.763640+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:17.060551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:18.211723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/1217443296":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2948747997":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/3110681241":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/1795019078":"2023-10-19T10:40:34.763606+0000","172.21.15.139:6833/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:6832/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/686709880":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2487642114":"2023-10-19T10:40:34.763606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f276006ac90 msgr2=0x7f276006d150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f276006ac90 0x7f276006d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27800ceb40 msgr2=0x7f278015c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27800ceb40 0x7f278015c0c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f277000ea60 tx=0x7f277000ef20 comp rx=0 tx=0).stop 2023-10-18T10:40:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 shutdown_connections 2023-10-18T10:40:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f276006ac90 0x7f276006d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27800cf340 0x7f27801f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f278015b2a0 0x7f27801f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 --2- 172.21.15.125:0/1770597172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27800ceb40 0x7f278015c0c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.239+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 >> 172.21.15.125:0/1770597172 conn(0x7f27800d0d60 msgr2=0x7f2780151040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.240+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 shutdown_connections 2023-10-18T10:40:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.240+0000 7f2784ec2640 1 -- 172.21.15.125:0/1770597172 wait complete. 2023-10-18T10:40:37.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:40:17.060551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:40:18.211723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:40:37.254 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:40:37.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.359+0000 7ff347924640 1 -- 172.21.15.125:0/2662485562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 msgr2=0x7ff340070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.359+0000 7ff347924640 1 --2- 172.21.15.125:0/2662485562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 0x7ff340070ce0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff3400b3100 tx=0x7ff33c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.360+0000 7ff347924640 1 -- 172.21.15.125:0/2662485562 shutdown_connections 2023-10-18T10:40:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.360+0000 7ff347924640 1 --2- 172.21.15.125:0/2662485562 >> v2:172.21.15.139:3300/0 conn(0x7ff34013ebe0 0x7ff340069bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.360+0000 7ff347924640 1 --2- 172.21.15.125:0/2662485562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 0x7ff340070ce0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.360+0000 7ff347924640 1 -- 172.21.15.125:0/2662485562 >> 172.21.15.125:0/2662485562 conn(0x7ff3400af730 msgr2=0x7ff3400afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:37.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.360+0000 7ff347924640 1 -- 172.21.15.125:0/2662485562 shutdown_connections 2023-10-18T10:40:37.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.360+0000 7ff347924640 1 -- 172.21.15.125:0/2662485562 wait complete. 2023-10-18T10:40:37.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.361+0000 7ff347924640 1 Processor -- start 2023-10-18T10:40:37.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.361+0000 7ff347924640 1 -- start start 2023-10-18T10:40:37.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.361+0000 7ff347924640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff340069720 0x7ff34014f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.361+0000 7ff347924640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 0x7ff3401efc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.361+0000 7ff347924640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff34013ebe0 0x7ff34013cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.361+0000 7ff347924640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401e3a00 con 0x7ff34013bd70 2023-10-18T10:40:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.361+0000 7ff347924640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401e3ce0 con 0x7ff340069720 2023-10-18T10:40:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.361+0000 7ff347924640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401e3fc0 con 0x7ff34013ebe0 2023-10-18T10:40:37.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 0x7ff3401efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 0x7ff3401efc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50202/0 (socket says 172.21.15.125:50202) 2023-10-18T10:40:37.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 -- 172.21.15.125:0/3967197415 learned_addr learned my addr 172.21.15.125:0/3967197415 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:37.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff3456c0640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff340069720 0x7ff34014f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff345ec1640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff34013ebe0 0x7ff34013cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 -- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff34013ebe0 msgr2=0x7ff34013cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff34013ebe0 0x7ff34013cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 -- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff340069720 msgr2=0x7ff34014f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff340069720 0x7ff34014f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 -- 172.21.15.125:0/3967197415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c009560 con 0x7ff34013bd70 2023-10-18T10:40:37.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff344ebf640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 0x7ff3401efc20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c430 tx=0x7ff33000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:37.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff335ffb640 1 -- 172.21.15.125:0/3967197415 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330011790 con 0x7ff34013bd70 2023-10-18T10:40:37.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff335ffb640 1 -- 172.21.15.125:0/3967197415 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff3300119e0 con 0x7ff34013bd70 2023-10-18T10:40:37.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3401e42a0 con 0x7ff34013bd70 2023-10-18T10:40:37.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.362+0000 7ff335ffb640 1 -- 172.21.15.125:0/3967197415 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330011e30 con 0x7ff34013bd70 2023-10-18T10:40:37.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.363+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34014f6b0 con 0x7ff34013bd70 2023-10-18T10:40:37.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.364+0000 7ff335ffb640 1 -- 172.21.15.125:0/3967197415 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff33001c460 con 0x7ff34013bd70 2023-10-18T10:40:37.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.364+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3401e47c0 con 0x7ff34013bd70 2023-10-18T10:40:37.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.364+0000 7ff335ffb640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7ff32406ac90 0x7ff32406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.364+0000 7ff3456c0640 1 -- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7ff32406ac90 msgr2=0x7ff32406d150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:37.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.364+0000 7ff335ffb640 1 -- 172.21.15.125:0/3967197415 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7ff3300895b0 con 0x7ff34013bd70 2023-10-18T10:40:37.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.364+0000 7ff3456c0640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7ff32406ac90 0x7ff32406d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:37.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.368+0000 7ff335ffb640 1 -- 172.21.15.125:0/3967197415 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3401e47c0 con 0x7ff34013bd70 2023-10-18T10:40:37.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.542+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3401e47c0 con 0x7ff34013bd70 2023-10-18T10:40:37.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.543+0000 7ff335ffb640 1 -- 172.21.15.125:0/3967197415 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3401e47c0 con 0x7ff34013bd70 2023-10-18T10:40:37.545 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:40:37.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7ff32406ac90 msgr2=0x7ff32406d150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:37.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7ff32406ac90 0x7ff32406d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 msgr2=0x7ff3401efc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 0x7ff3401efc20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c430 tx=0x7ff33000c8f0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 shutdown_connections 2023-10-18T10:40:37.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7ff32406ac90 0x7ff32406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff34013ebe0 0x7ff34013cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff34013bd70 0x7ff3401efc20 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 --2- 172.21.15.125:0/3967197415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff340069720 0x7ff34014f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 >> 172.21.15.125:0/3967197415 conn(0x7ff3400af730 msgr2=0x7ff3400afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:37.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 shutdown_connections 2023-10-18T10:40:37.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.545+0000 7ff347924640 1 -- 172.21.15.125:0/3967197415 wait complete. 2023-10-18T10:40:37.558 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:40:37.591 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.590+0000 7f73fac25040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:40:37.634 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.632+0000 7fcec9e2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:40:37.634 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.633+0000 7f73fac25040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:40:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.664+0000 7f7974d4c640 1 -- 172.21.15.125:0/3646565419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f797015b2a0 msgr2=0x7f79700cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.664+0000 7f7974d4c640 1 --2- 172.21.15.125:0/3646565419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f797015b2a0 0x7f79700cf720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7958009830 tx=0x7f795802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.664+0000 7f7974d4c640 1 -- 172.21.15.125:0/3646565419 shutdown_connections 2023-10-18T10:40:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.664+0000 7f7974d4c640 1 --2- 172.21.15.125:0/3646565419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f797015b2a0 0x7f79700cf720 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.664+0000 7f7974d4c640 1 --2- 172.21.15.125:0/3646565419 >> v2:172.21.15.139:3300/0 conn(0x7f79700cf340 0x7f79700d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.664+0000 7f7974d4c640 1 -- 172.21.15.125:0/3646565419 >> 172.21.15.125:0/3646565419 conn(0x7f79700d0d60 msgr2=0x7f79700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.665+0000 7f7974d4c640 1 -- 172.21.15.125:0/3646565419 shutdown_connections 2023-10-18T10:40:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.665+0000 7f7974d4c640 1 -- 172.21.15.125:0/3646565419 wait complete. 2023-10-18T10:40:37.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.666+0000 7f7974d4c640 1 Processor -- start 2023-10-18T10:40:37.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.666+0000 7f7974d4c640 1 -- start start 2023-10-18T10:40:37.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.666+0000 7f7974d4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79700ceb40 0x7f79701f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7974d4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79700cf340 0x7f7970158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7974d4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f797015b2a0 0x7f79700b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7974d4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701f2980 con 0x7f79700ceb40 2023-10-18T10:40:37.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7974d4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701f5380 con 0x7f797015b2a0 2023-10-18T10:40:37.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7974d4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79701f5660 con 0x7f79700cf340 2023-10-18T10:40:37.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f796e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79700ceb40 0x7f79701f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f796e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79700ceb40 0x7f79701f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60928/0 (socket says 172.21.15.125:60928) 2023-10-18T10:40:37.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f796e59c640 1 -- 172.21.15.125:0/36727983 learned_addr learned my addr 172.21.15.125:0/36727983 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:37.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7967fff640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79700cf340 0x7f7970158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f796ed9d640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f797015b2a0 0x7f79700b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7967fff640 1 -- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f797015b2a0 msgr2=0x7f79700b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7967fff640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f797015b2a0 0x7f79700b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7967fff640 1 -- 172.21.15.125:0/36727983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79700ceb40 msgr2=0x7f79701f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7967fff640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79700ceb40 0x7f79701f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7967fff640 1 -- 172.21.15.125:0/36727983 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7960009560 con 0x7f79700cf340 2023-10-18T10:40:37.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f796ed9d640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f797015b2a0 0x7f79700b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:37.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f796e59c640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79700ceb40 0x7f79701f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:37.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7967fff640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79700cf340 0x7f7970158200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7958033000 tx=0x7f79580038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:37.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7966ffd640 1 -- 172.21.15.125:0/36727983 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958003ea0 con 0x7f79700cf340 2023-10-18T10:40:37.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7966ffd640 1 -- 172.21.15.125:0/36727983 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7958034440 con 0x7f79700cf340 2023-10-18T10:40:37.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7958009490 con 0x7f79700cf340 2023-10-18T10:40:37.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.667+0000 7f7966ffd640 1 -- 172.21.15.125:0/36727983 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958034830 con 0x7f79700cf340 2023-10-18T10:40:37.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.668+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79701f5c20 con 0x7f79700cf340 2023-10-18T10:40:37.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.668+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797015a8d0 con 0x7f79700cf340 2023-10-18T10:40:37.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.669+0000 7f7966ffd640 1 -- 172.21.15.125:0/36727983 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f7958034990 con 0x7f79700cf340 2023-10-18T10:40:37.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.669+0000 7f7966ffd640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f795006ac90 0x7f795006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.669+0000 7f796e59c640 1 -- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f795006ac90 msgr2=0x7f795006d150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:37.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.669+0000 7f7966ffd640 1 -- 172.21.15.125:0/36727983 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f79580abd10 con 0x7f79700cf340 2023-10-18T10:40:37.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.670+0000 7f796e59c640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f795006ac90 0x7f795006d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:37.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.678+0000 7f7966ffd640 1 -- 172.21.15.125:0/36727983 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79580b1020 con 0x7f79700cf340 2023-10-18T10:40:37.681 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.679+0000 7f73fac25040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:40:37.681 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.679+0000 7fcec9e2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:40:37.725 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.724+0000 7fcec9e2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:40:37.732 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.730+0000 7f73fac25040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:40:37.772 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.771+0000 7f73fac25040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:40:37.779 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.778+0000 7fcec9e2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:40:37.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.784+0000 7f7966ffd640 1 -- 172.21.15.125:0/36727983 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f7958077190 con 0x7f79700cf340 2023-10-18T10:40:37.825 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.823+0000 7fcec9e2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:40:37.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.854+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79700690e0 con 0x7f79700cf340 2023-10-18T10:40:37.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.855+0000 7f7966ffd640 1 -- 172.21.15.125:0/36727983 <== mon.2 v2:172.21.15.139:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f795807e590 con 0x7f79700cf340 2023-10-18T10:40:37.857 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:40:37.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f795006ac90 msgr2=0x7f795006d150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:37.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f795006ac90 0x7f795006d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79700cf340 msgr2=0x7f7970158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79700cf340 0x7f7970158200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7958033000 tx=0x7f79580038a0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 shutdown_connections 2023-10-18T10:40:37.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f795006ac90 0x7f795006d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f797015b2a0 0x7f79700b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79700cf340 0x7f7970158200 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 --2- 172.21.15.125:0/36727983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79700ceb40 0x7f79701f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 >> 172.21.15.125:0/36727983 conn(0x7f79700d0d60 msgr2=0x7f79700b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:37.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 shutdown_connections 2023-10-18T10:40:37.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.857+0000 7f7974d4c640 1 -- 172.21.15.125:0/36727983 wait complete. 2023-10-18T10:40:37.870 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:40:37.872 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.871+0000 7f73fac25040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:40:37.925 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.923+0000 7f73fac25040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:37.928 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.927+0000 7fcec9e2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:40:37.965 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:37.964+0000 7f73fac25040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:40:37.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.975+0000 7f295359e640 1 -- 172.21.15.125:0/1326382213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 msgr2=0x7f2938004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.975+0000 7f295359e640 1 --2- 172.21.15.125:0/1326382213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 0x7f2938004c40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f294c0b00b0 tx=0x7f29380057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.975+0000 7f295359e640 1 -- 172.21.15.125:0/1326382213 shutdown_connections 2023-10-18T10:40:37.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.975+0000 7f295359e640 1 --2- 172.21.15.125:0/1326382213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 0x7f2938004c40 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.975+0000 7f295359e640 1 -- 172.21.15.125:0/1326382213 >> 172.21.15.125:0/1326382213 conn(0x7f294c0d0d60 msgr2=0x7f294c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:37.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.975+0000 7f295359e640 1 -- 172.21.15.125:0/1326382213 shutdown_connections 2023-10-18T10:40:37.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.975+0000 7f295359e640 1 -- 172.21.15.125:0/1326382213 wait complete. 2023-10-18T10:40:37.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.976+0000 7f295359e640 1 Processor -- start 2023-10-18T10:40:37.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.976+0000 7f295359e640 1 -- start start 2023-10-18T10:40:37.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 0x7f294c155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295359e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f294c0b2830 0x7f294c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294c0b5890 0x7f294c0b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295359e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0b6540 con 0x7f29380047c0 2023-10-18T10:40:37.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295359e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0b31f0 con 0x7f294c0b5890 2023-10-18T10:40:37.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295359e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c15b850 con 0x7f294c0b2830 2023-10-18T10:40:37.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295133a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 0x7f294c155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295133a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 0x7f294c155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60952/0 (socket says 172.21.15.125:60952) 2023-10-18T10:40:37.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295133a640 1 -- 172.21.15.125:0/3451389105 learned_addr learned my addr 172.21.15.125:0/3451389105 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:37.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f2950b39640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f294c0b2830 0x7f294c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f2951b3b640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294c0b5890 0x7f294c0b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:37.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295133a640 1 -- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f294c0b2830 msgr2=0x7f294c0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295133a640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f294c0b2830 0x7f294c0b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295133a640 1 -- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294c0b5890 msgr2=0x7f294c0b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:37.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295133a640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294c0b5890 0x7f294c0b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:37.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.977+0000 7f295133a640 1 -- 172.21.15.125:0/3451389105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c15bb30 con 0x7f29380047c0 2023-10-18T10:40:37.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.978+0000 7f295133a640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 0x7f294c155190 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2948003e40 tx=0x7f2948001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:37.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.978+0000 7f293dffb640 1 -- 172.21.15.125:0/3451389105 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948011840 con 0x7f29380047c0 2023-10-18T10:40:37.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.978+0000 7f293dffb640 1 -- 172.21.15.125:0/3451389105 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29480119a0 con 0x7f29380047c0 2023-10-18T10:40:37.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.978+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c15be10 con 0x7f29380047c0 2023-10-18T10:40:37.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.978+0000 7f293dffb640 1 -- 172.21.15.125:0/3451389105 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948011d90 con 0x7f29380047c0 2023-10-18T10:40:37.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.978+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c0b6250 con 0x7f29380047c0 2023-10-18T10:40:37.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.979+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c15ae00 con 0x7f29380047c0 2023-10-18T10:40:37.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.979+0000 7f293dffb640 1 -- 172.21.15.125:0/3451389105 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f29480103c0 con 0x7f29380047c0 2023-10-18T10:40:37.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.980+0000 7f293dffb640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f293006af60 0x7f293006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:37.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.980+0000 7f2950b39640 1 -- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f293006af60 msgr2=0x7f293006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:37.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.980+0000 7f293dffb640 1 -- 172.21.15.125:0/3451389105 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f294808a960 con 0x7f29380047c0 2023-10-18T10:40:37.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.980+0000 7f2950b39640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f293006af60 0x7f293006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:37.988 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:37.983+0000 7fcec9e2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:37.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:37.984+0000 7f293dffb640 1 -- 172.21.15.125:0/3451389105 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2948055690 con 0x7f29380047c0 2023-10-18T10:40:38.014 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.013+0000 7f73fac25040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:40:38.028 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.027+0000 7fcec9e2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:40:38.056 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.054+0000 7f73fac25040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:40:38.076 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.075+0000 7fcec9e2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:40:38.118 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.117+0000 7fcec9e2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:40:38.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.152+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f294c0690e0 con 0x7f29380047c0 2023-10-18T10:40:38.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.153+0000 7f293dffb640 1 -- 172.21.15.125:0/3451389105 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f294801f070 con 0x7f29380047c0 2023-10-18T10:40:38.155 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:40:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f293006af60 msgr2=0x7f293006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f293006af60 0x7f293006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 msgr2=0x7f294c155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 0x7f294c155190 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2948003e40 tx=0x7f2948001630 comp rx=0 tx=0).stop 2023-10-18T10:40:38.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 shutdown_connections 2023-10-18T10:40:38.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f293006af60 0x7f293006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294c0b5890 0x7f294c0b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f294c0b2830 0x7f294c0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 --2- 172.21.15.125:0/3451389105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29380047c0 0x7f294c155190 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 >> 172.21.15.125:0/3451389105 conn(0x7f294c0d0d60 msgr2=0x7f294c069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:38.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 shutdown_connections 2023-10-18T10:40:38.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.155+0000 7f295359e640 1 -- 172.21.15.125:0/3451389105 wait complete. 2023-10-18T10:40:38.168 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:40:38.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.272+0000 7f7219a21640 1 -- 172.21.15.125:0/255366113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214159060 msgr2=0x7f721406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:38.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.272+0000 7f7219a21640 1 --2- 172.21.15.125:0/255366113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214159060 0x7f721406d260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009860 tx=0x7f71fc02bcb0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.272+0000 7f7219a21640 1 -- 172.21.15.125:0/255366113 shutdown_connections 2023-10-18T10:40:38.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.272+0000 7f7219a21640 1 --2- 172.21.15.125:0/255366113 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f721415d400 0x7f72140cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.272+0000 7f7219a21640 1 --2- 172.21.15.125:0/255366113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7214159060 0x7f721406d260 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.272+0000 7f7219a21640 1 -- 172.21.15.125:0/255366113 >> 172.21.15.125:0/255366113 conn(0x7f72140b0670 msgr2=0x7f72140b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:38.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.272+0000 7f7219a21640 1 -- 172.21.15.125:0/255366113 shutdown_connections 2023-10-18T10:40:38.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.272+0000 7f7219a21640 1 -- 172.21.15.125:0/255366113 wait complete. 2023-10-18T10:40:38.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.273+0000 7f7219a21640 1 Processor -- start 2023-10-18T10:40:38.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.273+0000 7f7219a21640 1 -- start start 2023-10-18T10:40:38.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7219a21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72140cdb30 0x7f721415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:38.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7219a21640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7214159060 0x7f721414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:38.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7219a21640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f721415d400 0x7f72140b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:38.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7219a21640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721415a130 con 0x7f7214159060 2023-10-18T10:40:38.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7219a21640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140c2790 con 0x7f72140cdb30 2023-10-18T10:40:38.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7219a21640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721415a660 con 0x7f721415d400 2023-10-18T10:40:38.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f72127fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7214159060 0x7f721414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:38.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f72127fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7214159060 0x7f721414f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60958/0 (socket says 172.21.15.125:60958) 2023-10-18T10:40:38.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f72127fc640 1 -- 172.21.15.125:0/1721897516 learned_addr learned my addr 172.21.15.125:0/1721897516 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:38.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7212ffd640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72140cdb30 0x7f721415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:38.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f72137fe640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f721415d400 0x7f72140b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:38.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7212ffd640 1 -- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f721415d400 msgr2=0x7f72140b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:38.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7212ffd640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f721415d400 0x7f72140b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7212ffd640 1 -- 172.21.15.125:0/1721897516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7214159060 msgr2=0x7f721414f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:38.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.274+0000 7f7212ffd640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7214159060 0x7f721414f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.275+0000 7f7212ffd640 1 -- 172.21.15.125:0/1721897516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009490 con 0x7f72140cdb30 2023-10-18T10:40:38.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.275+0000 7f72127fc640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7214159060 0x7f721414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:38.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.275+0000 7f7212ffd640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72140cdb30 0x7f721415c6e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f71fc033000 tx=0x7f71fc0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:38.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.275+0000 7f720bfff640 1 -- 172.21.15.125:0/1721897516 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc003ea0 con 0x7f72140cdb30 2023-10-18T10:40:38.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.275+0000 7f720bfff640 1 -- 172.21.15.125:0/1721897516 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f71fc034440 con 0x7f72140cdb30 2023-10-18T10:40:38.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.275+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721415a380 con 0x7f72140cdb30 2023-10-18T10:40:38.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.275+0000 7f720bfff640 1 -- 172.21.15.125:0/1721897516 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc034830 con 0x7f72140cdb30 2023-10-18T10:40:38.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.275+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721414f920 con 0x7f72140cdb30 2023-10-18T10:40:38.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.276+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72140675f0 con 0x7f72140cdb30 2023-10-18T10:40:38.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.276+0000 7f720bfff640 1 -- 172.21.15.125:0/1721897516 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f71fc034990 con 0x7f72140cdb30 2023-10-18T10:40:38.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.277+0000 7f720bfff640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f71f406af90 0x7f71f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:38.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.277+0000 7f72127fc640 1 -- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f71f406af90 msgr2=0x7f71f406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:38.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.277+0000 7f720bfff640 1 -- 172.21.15.125:0/1721897516 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f71fc0ac040 con 0x7f72140cdb30 2023-10-18T10:40:38.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.277+0000 7f72127fc640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f71f406af90 0x7f71f406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:38.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.281+0000 7f720bfff640 1 -- 172.21.15.125:0/1721897516 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71fc076e00 con 0x7f72140cdb30 2023-10-18T10:40:38.373 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.371+0000 7f73fac25040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:40:38.451 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.450+0000 7fcec9e2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:40:38.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.468+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f72140b4aa0 con 0x7f72140cdb30 2023-10-18T10:40:38.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.469+0000 7f720bfff640 1 -- 172.21.15.125:0/1721897516 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v85) v1 ==== 80+0+477 (secure 0 0 0) 0x7f71fc07a420 con 0x7f72140cdb30 2023-10-18T10:40:38.471 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1217443296 2023-10-19T10:40:34.763606+0000 2023-10-18T10:40:38.471 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2948747997 2023-10-19T10:40:34.763606+0000 2023-10-18T10:40:38.471 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3110681241 2023-10-19T10:40:34.763606+0000 2023-10-18T10:40:38.471 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1795019078 2023-10-19T10:40:34.763606+0000 2023-10-18T10:40:38.472 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/2642591764 2023-10-19T10:40:34.763606+0000 2023-10-18T10:40:38.472 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/2642591764 2023-10-19T10:40:34.763606+0000 2023-10-18T10:40:38.472 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/686709880 2023-10-19T10:40:34.763606+0000 2023-10-18T10:40:38.472 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2487642114 2023-10-19T10:40:34.763606+0000 2023-10-18T10:40:38.473 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T10:40:38.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f71f406af90 msgr2=0x7f71f406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:38.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f71f406af90 0x7f71f406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72140cdb30 msgr2=0x7f721415c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:38.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72140cdb30 0x7f721415c6e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f71fc033000 tx=0x7f71fc0038a0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 shutdown_connections 2023-10-18T10:40:38.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f71f406af90 0x7f71f406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f721415d400 0x7f72140b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7214159060 0x7f721414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 --2- 172.21.15.125:0/1721897516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72140cdb30 0x7f721415c6e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 >> 172.21.15.125:0/1721897516 conn(0x7f72140b0670 msgr2=0x7f72140cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:38.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 shutdown_connections 2023-10-18T10:40:38.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.471+0000 7f7219a21640 1 -- 172.21.15.125:0/1721897516 wait complete. 2023-10-18T10:40:38.483 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:40:38.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:38.484 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T10:40:38.504 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.503+0000 7f73fac25040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:40:38.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[119373]: fuse finished with error 0 and tester_r 0 2023-10-18T10:40:38.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:40:38.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:38.512 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:40:38.591 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.590+0000 7fcec9e2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:40:38.595 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.594+0000 7f73fac25040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:40:38.598 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:40:38.599 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:38.614 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:40:38.615 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:40:38.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:38.616 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:40:38.640 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.639+0000 7f73fac25040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:40:38.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:40:38.655 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T10:40:38.655 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:40:38.656 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:38.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:40:38.671 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:40:38.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:38.673 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:40:38.686 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.685+0000 7fcec9e2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:40:38.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:40:38.701 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T10:40:38.701 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:40:38.702 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:38.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:40:38.717 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:40:38.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:40:38.718 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:40:38.731 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.729+0000 7fcec9e2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:40:38.739 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.738+0000 7f73fac25040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:38.752 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T10:40:38.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:40:38.754 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:38.831 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.829+0000 7fcec9e2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:38.833 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.831+0000 7f73fac25040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:40:38.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.857+0000 7f94cd705640 1 -- 172.21.15.125:0/4108189074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 msgr2=0x7f94c806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:38.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.857+0000 7f94cd705640 1 --2- 172.21.15.125:0/4108189074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 0x7f94c806d260 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009830 tx=0x7f94b802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.857+0000 7f94cd705640 1 -- 172.21.15.125:0/4108189074 shutdown_connections 2023-10-18T10:40:38.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.857+0000 7f94cd705640 1 --2- 172.21.15.125:0/4108189074 >> v2:172.21.15.139:3300/0 conn(0x7f94c815d400 0x7f94c80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.857+0000 7f94cd705640 1 --2- 172.21.15.125:0/4108189074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 0x7f94c806d260 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.857+0000 7f94cd705640 1 -- 172.21.15.125:0/4108189074 >> 172.21.15.125:0/4108189074 conn(0x7f94c80b0670 msgr2=0x7f94c80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:38.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.857+0000 7f94cd705640 1 -- 172.21.15.125:0/4108189074 shutdown_connections 2023-10-18T10:40:38.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.857+0000 7f94cd705640 1 -- 172.21.15.125:0/4108189074 wait complete. 2023-10-18T10:40:38.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.858+0000 7f94cd705640 1 Processor -- start 2023-10-18T10:40:38.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.858+0000 7f94cd705640 1 -- start start 2023-10-18T10:40:38.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94cd705640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c80cdb30 0x7f94c8151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:38.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94cd705640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 0x7f94c80b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:38.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94cd705640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c815d400 0x7f94c8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:38.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94cd705640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c81e56e0 con 0x7f94c8159060 2023-10-18T10:40:38.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94cd705640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c80b3650 con 0x7f94c815d400 2023-10-18T10:40:38.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94cd705640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c815a2f0 con 0x7f94c80cdb30 2023-10-18T10:40:38.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94bffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 0x7f94c80b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:38.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94bffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 0x7f94c80b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60962/0 (socket says 172.21.15.125:60962) 2023-10-18T10:40:38.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94bffff640 1 -- 172.21.15.125:0/1880044441 learned_addr learned my addr 172.21.15.125:0/1880044441 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:38.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94c77fe640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c815d400 0x7f94c8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:38.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94c6ffd640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c80cdb30 0x7f94c8151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:38.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94bffff640 1 -- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c80cdb30 msgr2=0x7f94c8151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:38.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94bffff640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c80cdb30 0x7f94c8151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94bffff640 1 -- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c815d400 msgr2=0x7f94c8155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:38.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94bffff640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c815d400 0x7f94c8155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:38.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.859+0000 7f94bffff640 1 -- 172.21.15.125:0/1880044441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009490 con 0x7f94c8159060 2023-10-18T10:40:38.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.860+0000 7f94bffff640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 0x7f94c80b2e20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c4b0 tx=0x7f94b000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:38.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.860+0000 7f94bf7fe640 1 -- 172.21.15.125:0/1880044441 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0005480 con 0x7f94c8159060 2023-10-18T10:40:38.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.860+0000 7f94bf7fe640 1 -- 172.21.15.125:0/1880044441 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94b00036e0 con 0x7f94c8159060 2023-10-18T10:40:38.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.860+0000 7f94bf7fe640 1 -- 172.21.15.125:0/1880044441 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0003b30 con 0x7f94c8159060 2023-10-18T10:40:38.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.860+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c815a5d0 con 0x7f94c8159060 2023-10-18T10:40:38.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.860+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c815ab90 con 0x7f94c8159060 2023-10-18T10:40:38.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.861+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c815adb0 con 0x7f94c8159060 2023-10-18T10:40:38.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.861+0000 7f94bf7fe640 1 -- 172.21.15.125:0/1880044441 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f94b0003c90 con 0x7f94c8159060 2023-10-18T10:40:38.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.862+0000 7f94bf7fe640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f94a806af60 0x7f94a806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:38.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.862+0000 7f94c6ffd640 1 -- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f94a806af60 msgr2=0x7f94a806d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:38.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.862+0000 7f94bf7fe640 1 -- 172.21.15.125:0/1880044441 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f94b0089110 con 0x7f94c8159060 2023-10-18T10:40:38.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.862+0000 7f94c6ffd640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f94a806af60 0x7f94a806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:38.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:38.866+0000 7f94bf7fe640 1 -- 172.21.15.125:0/1880044441 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94b0053f50 con 0x7f94c8159060 2023-10-18T10:40:38.929 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:38.927+0000 7fcec9e2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:40:38.970 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:38.969+0000 7f73fac25040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:39.012 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:39.010+0000 7f73fac25040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:39.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.063+0000 7f94c6ffd640 1 -- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f94a806af60 msgr2=0x7f94a806d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:39.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.063+0000 7f94c6ffd640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f94a806af60 0x7f94a806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:40:39.070 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:39.069+0000 7fcec9e2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:40:39.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.078+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94c80675f0 con 0x7f94c8159060 2023-10-18T10:40:39.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.078+0000 7f94bf7fe640 1 -- 172.21.15.125:0/1880044441 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f94b0017090 con 0x7f94c8159060 2023-10-18T10:40:39.080 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:39.080 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/693466412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":693466412},{"type":"v1","addr":"172.21.15.139:6836","nonce":693466412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6277,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/170638031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":170638031},{"type":"v1","addr":"172.21.15.139:6837","nonce":170638031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":7068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3024029084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3024029084},{"type":"v1","addr":"172.21.15.125:6834","nonce":3024029084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:19.231823+0000","modified":"2023-10-18T10:40:21.240206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1136111019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1136111019},{"type":"v1","addr":"172.21.15.125:6835","nonce":1136111019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T10:40:39.081 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 29 2023-10-18T10:40:39.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f94a806af60 msgr2=0x7f94a806d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f94a806af60 0x7f94a806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 msgr2=0x7f94c80b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 0x7f94c80b2e20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c4b0 tx=0x7f94b000c970 comp rx=0 tx=0).stop 2023-10-18T10:40:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 shutdown_connections 2023-10-18T10:40:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f94a806af60 0x7f94a806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c815d400 0x7f94c8155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c8159060 0x7f94c80b2e20 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 --2- 172.21.15.125:0/1880044441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c80cdb30 0x7f94c8151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 >> 172.21.15.125:0/1880044441 conn(0x7f94c80b0670 msgr2=0x7f94c80acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 shutdown_connections 2023-10-18T10:40:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.080+0000 7f94cd705640 1 -- 172.21.15.125:0/1880044441 wait complete. 2023-10-18T10:40:39.093 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:39.115 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:39.113+0000 7f73fac25040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:40:39.116 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:39.114+0000 7fcec9e2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:40:39.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.195+0000 7fb492826640 1 -- 172.21.15.125:0/2381174726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb48c15b2a0 msgr2=0x7fb48c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.195+0000 7fb492826640 1 --2- 172.21.15.125:0/2381174726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb48c15b2a0 0x7fb48c0d1910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb480009830 tx=0x7fb48002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.196+0000 7fb492826640 1 -- 172.21.15.125:0/2381174726 shutdown_connections 2023-10-18T10:40:39.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.196+0000 7fb492826640 1 --2- 172.21.15.125:0/2381174726 >> v2:172.21.15.139:3300/0 conn(0x7fb48c0ceb40 0x7fb48c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.196+0000 7fb492826640 1 --2- 172.21.15.125:0/2381174726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb48c15b2a0 0x7fb48c0d1910 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.196+0000 7fb492826640 1 -- 172.21.15.125:0/2381174726 >> 172.21.15.125:0/2381174726 conn(0x7fb48c0d0d60 msgr2=0x7fb48c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:39.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.196+0000 7fb492826640 1 -- 172.21.15.125:0/2381174726 shutdown_connections 2023-10-18T10:40:39.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.196+0000 7fb492826640 1 -- 172.21.15.125:0/2381174726 wait complete. 2023-10-18T10:40:39.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.197+0000 7fb492826640 1 Processor -- start 2023-10-18T10:40:39.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.198+0000 7fb492826640 1 -- start start 2023-10-18T10:40:39.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb492826640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb48c0ceb40 0x7fb48c15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb492826640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb48c0cf340 0x7fb48c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb492826640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb48c15b2a0 0x7fb48c0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb492826640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0c9420 con 0x7fb48c0cf340 2023-10-18T10:40:39.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb492826640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0c8810 con 0x7fb48c15b2a0 2023-10-18T10:40:39.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb492826640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0c8af0 con 0x7fb48c0ceb40 2023-10-18T10:40:39.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb48c0cf340 0x7fb48c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb48b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb48c0cf340 0x7fb48c0c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60978/0 (socket says 172.21.15.125:60978) 2023-10-18T10:40:39.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb48b7fe640 1 -- 172.21.15.125:0/1520032158 learned_addr learned my addr 172.21.15.125:0/1520032158 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:39.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb48bfff640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb48c0ceb40 0x7fb48c15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb490dc3640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb48c15b2a0 0x7fb48c0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb490dc3640 1 -- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb48c0ceb40 msgr2=0x7fb48c15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb490dc3640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb48c0ceb40 0x7fb48c15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb490dc3640 1 -- 172.21.15.125:0/1520032158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb48c0cf340 msgr2=0x7fb48c0c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb490dc3640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb48c0cf340 0x7fb48c0c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb490dc3640 1 -- 172.21.15.125:0/1520032158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484009840 con 0x7fb48c15b2a0 2023-10-18T10:40:39.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.199+0000 7fb48b7fe640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb48c0cf340 0x7fb48c0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:40:39.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.200+0000 7fb490dc3640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb48c15b2a0 0x7fb48c0c6540 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb48400ec50 tx=0x7fb48400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:39.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.200+0000 7fb488ff9640 1 -- 172.21.15.125:0/1520032158 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484003b70 con 0x7fb48c15b2a0 2023-10-18T10:40:39.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.200+0000 7fb488ff9640 1 -- 172.21.15.125:0/1520032158 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb484003cd0 con 0x7fb48c15b2a0 2023-10-18T10:40:39.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.200+0000 7fb488ff9640 1 -- 172.21.15.125:0/1520032158 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484004280 con 0x7fb48c15b2a0 2023-10-18T10:40:39.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.200+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb480009490 con 0x7fb48c15b2a0 2023-10-18T10:40:39.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.200+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c0c90b0 con 0x7fb48c15b2a0 2023-10-18T10:40:39.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.201+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c0c9590 con 0x7fb48c15b2a0 2023-10-18T10:40:39.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.201+0000 7fb488ff9640 1 -- 172.21.15.125:0/1520032158 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb4840043e0 con 0x7fb48c15b2a0 2023-10-18T10:40:39.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.201+0000 7fb488ff9640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7fb47806af90 0x7fb47806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.202+0000 7fb48bfff640 1 -- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7fb47806af90 msgr2=0x7fb47806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:39.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.202+0000 7fb488ff9640 1 -- 172.21.15.125:0/1520032158 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7fb484014070 con 0x7fb48c15b2a0 2023-10-18T10:40:39.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.202+0000 7fb48bfff640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7fb47806af90 0x7fb47806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:39.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.206+0000 7fb488ff9640 1 -- 172.21.15.125:0/1520032158 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb484055570 con 0x7fb48c15b2a0 2023-10-18T10:40:39.228 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:39.227+0000 7fcec9e2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:40:39.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.381+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb48c0690e0 con 0x7fb48c15b2a0 2023-10-18T10:40:39.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.382+0000 7fb488ff9640 1 -- 172.21.15.125:0/1520032158 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17272 (secure 0 0 0) 0x7fb484003e80 con 0x7fb48c15b2a0 2023-10-18T10:40:39.384 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:39.385 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":85,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:34.763640+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:17.060551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:18.211723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/1217443296":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2948747997":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/3110681241":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/1795019078":"2023-10-19T10:40:34.763606+0000","172.21.15.139:6833/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:6832/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/686709880":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2487642114":"2023-10-19T10:40:34.763606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:39.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7fb47806af90 msgr2=0x7fb47806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:39.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7fb47806af90 0x7fb47806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb48c15b2a0 msgr2=0x7fb48c0c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb48c15b2a0 0x7fb48c0c6540 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb48400ec50 tx=0x7fb48400c6a0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 shutdown_connections 2023-10-18T10:40:39.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7fb47806af90 0x7fb47806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb48c15b2a0 0x7fb48c0c6540 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb48c0cf340 0x7fb48c0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 --2- 172.21.15.125:0/1520032158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb48c0ceb40 0x7fb48c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 >> 172.21.15.125:0/1520032158 conn(0x7fb48c0d0d60 msgr2=0x7fb48c06b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:39.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 shutdown_connections 2023-10-18T10:40:39.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.384+0000 7fb492826640 1 -- 172.21.15.125:0/1520032158 wait complete. 2023-10-18T10:40:39.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:40:17.060551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:40:18.211723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:40:39.397 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:40:39.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.541+0000 7f96d4044640 1 -- 172.21.15.125:0/916958167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc0ceb40 msgr2=0x7f96cc0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.541+0000 7f96d4044640 1 --2- 172.21.15.125:0/916958167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc0ceb40 0x7f96cc0b0650 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f96c4005fd0 tx=0x7f96c402c240 comp rx=0 tx=0).stop 2023-10-18T10:40:39.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.541+0000 7f96d4044640 1 -- 172.21.15.125:0/916958167 shutdown_connections 2023-10-18T10:40:39.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.541+0000 7f96d4044640 1 --2- 172.21.15.125:0/916958167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc0ceb40 0x7f96cc0b0650 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.541+0000 7f96d4044640 1 --2- 172.21.15.125:0/916958167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc15b2a0 0x7f96cc0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.541+0000 7f96d4044640 1 -- 172.21.15.125:0/916958167 >> 172.21.15.125:0/916958167 conn(0x7f96cc0d0d60 msgr2=0x7f96cc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:39.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.541+0000 7f96d4044640 1 -- 172.21.15.125:0/916958167 shutdown_connections 2023-10-18T10:40:39.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.541+0000 7f96d4044640 1 -- 172.21.15.125:0/916958167 wait complete. 2023-10-18T10:40:39.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.542+0000 7f96d4044640 1 Processor -- start 2023-10-18T10:40:39.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.542+0000 7f96d4044640 1 -- start start 2023-10-18T10:40:39.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d4044640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc0ceb40 0x7f96cc1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d4044640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0cf340 0x7f96cc150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d4044640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc15b2a0 0x7f96cc15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d4044640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc1f49c0 con 0x7f96cc15b2a0 2023-10-18T10:40:39.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d4044640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0b3eb0 con 0x7f96cc0ceb40 2023-10-18T10:40:39.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d4044640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0b4190 con 0x7f96cc0cf340 2023-10-18T10:40:39.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d25e1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc15b2a0 0x7f96cc15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d25e1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc15b2a0 0x7f96cc15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32772/0 (socket says 172.21.15.125:32772) 2023-10-18T10:40:39.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d25e1640 1 -- 172.21.15.125:0/3563424554 learned_addr learned my addr 172.21.15.125:0/3563424554 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:39.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d15df640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0cf340 0x7f96cc150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.543+0000 7f96d1de0640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc0ceb40 0x7f96cc1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96d25e1640 1 -- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0cf340 msgr2=0x7f96cc150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96d25e1640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0cf340 0x7f96cc150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96d25e1640 1 -- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc0ceb40 msgr2=0x7f96cc1f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96d25e1640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc0ceb40 0x7f96cc1f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96d25e1640 1 -- 172.21.15.125:0/3563424554 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4009900 con 0x7f96cc15b2a0 2023-10-18T10:40:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96d25e1640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc15b2a0 0x7f96cc15ce90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009bf0 tx=0x7f96c4035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96be7fc640 1 -- 172.21.15.125:0/3563424554 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4034440 con 0x7f96cc15b2a0 2023-10-18T10:40:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96be7fc640 1 -- 172.21.15.125:0/3563424554 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96c4003e60 con 0x7f96cc15b2a0 2023-10-18T10:40:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96be7fc640 1 -- 172.21.15.125:0/3563424554 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40044b0 con 0x7f96cc15b2a0 2023-10-18T10:40:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc0b0d50 con 0x7f96cc15b2a0 2023-10-18T10:40:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.544+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc0b4750 con 0x7f96cc15b2a0 2023-10-18T10:40:39.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.545+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc0690e0 con 0x7f96cc15b2a0 2023-10-18T10:40:39.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.546+0000 7f96be7fc640 1 -- 172.21.15.125:0/3563424554 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f96c400a040 con 0x7f96cc15b2a0 2023-10-18T10:40:39.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.546+0000 7f96be7fc640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f96b006af60 0x7f96b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.546+0000 7f96d1de0640 1 -- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f96b006af60 msgr2=0x7f96b006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:39.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.546+0000 7f96be7fc640 1 -- 172.21.15.125:0/3563424554 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f96c40ac820 con 0x7f96cc15b2a0 2023-10-18T10:40:39.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.546+0000 7f96d1de0640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f96b006af60 0x7f96b006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:39.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.550+0000 7f96be7fc640 1 -- 172.21.15.125:0/3563424554 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96c40776f0 con 0x7f96cc15b2a0 2023-10-18T10:40:39.724 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:39.723+0000 7f73fac25040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:40:39.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.724+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96cc0d0b40 con 0x7f96cc15b2a0 2023-10-18T10:40:39.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.724+0000 7f96be7fc640 1 -- 172.21.15.125:0/3563424554 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96c40390a0 con 0x7f96cc15b2a0 2023-10-18T10:40:39.726 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:40:39.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f96b006af60 msgr2=0x7f96b006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:39.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f96b006af60 0x7f96b006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc15b2a0 msgr2=0x7f96cc15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc15b2a0 0x7f96cc15ce90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009bf0 tx=0x7f96c4035e30 comp rx=0 tx=0).stop 2023-10-18T10:40:39.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 shutdown_connections 2023-10-18T10:40:39.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f96b006af60 0x7f96b006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96cc15b2a0 0x7f96cc15ce90 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0cf340 0x7f96cc150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 --2- 172.21.15.125:0/3563424554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc0ceb40 0x7f96cc1f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 >> 172.21.15.125:0/3563424554 conn(0x7f96cc0d0d60 msgr2=0x7f96cc0ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:39.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 shutdown_connections 2023-10-18T10:40:39.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.726+0000 7f96d4044640 1 -- 172.21.15.125:0/3563424554 wait complete. 2023-10-18T10:40:39.738 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:40:39.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.844+0000 7f214064f640 1 -- 172.21.15.125:0/1539913199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f213815ccf0 msgr2=0x7f213815d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.844+0000 7f214064f640 1 --2- 172.21.15.125:0/1539913199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f213815ccf0 0x7f213815d100 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f21380706e0 tx=0x7f213002c260 comp rx=0 tx=0).stop 2023-10-18T10:40:39.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.844+0000 7f214064f640 1 -- 172.21.15.125:0/1539913199 shutdown_connections 2023-10-18T10:40:39.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.844+0000 7f214064f640 1 --2- 172.21.15.125:0/1539913199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f213815ccf0 0x7f213815d100 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.844+0000 7f214064f640 1 --2- 172.21.15.125:0/1539913199 >> v2:172.21.15.139:3300/0 conn(0x7f21380ce240 0x7f213806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.844+0000 7f214064f640 1 -- 172.21.15.125:0/1539913199 >> 172.21.15.125:0/1539913199 conn(0x7f21380b0670 msgr2=0x7f21380b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:39.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.844+0000 7f214064f640 1 -- 172.21.15.125:0/1539913199 shutdown_connections 2023-10-18T10:40:39.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.844+0000 7f214064f640 1 -- 172.21.15.125:0/1539913199 wait complete. 2023-10-18T10:40:39.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.845+0000 7f214064f640 1 Processor -- start 2023-10-18T10:40:39.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f214064f640 1 -- start start 2023-10-18T10:40:39.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f214064f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21380ce240 0x7f213815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f214064f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2138159060 0x7f213815a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f214064f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f213815ccf0 0x7f2138158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f214064f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21381e3600 con 0x7f2138159060 2023-10-18T10:40:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f214064f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213815b240 con 0x7f213815ccf0 2023-10-18T10:40:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f214064f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213815a9a0 con 0x7f21380ce240 2023-10-18T10:40:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f213dbea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2138159060 0x7f213815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f213dbea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2138159060 0x7f213815a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32782/0 (socket says 172.21.15.125:32782) 2023-10-18T10:40:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f213dbea640 1 -- 172.21.15.125:0/476590624 learned_addr learned my addr 172.21.15.125:0/476590624 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f213e3eb640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21380ce240 0x7f213815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.846+0000 7f213ebec640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f213815ccf0 0x7f2138158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f213dbea640 1 -- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21380ce240 msgr2=0x7f213815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f213dbea640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21380ce240 0x7f213815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f213dbea640 1 -- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f213815ccf0 msgr2=0x7f2138158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f213dbea640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f213815ccf0 0x7f2138158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f213dbea640 1 -- 172.21.15.125:0/476590624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130009900 con 0x7f2138159060 2023-10-18T10:40:39.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f213dbea640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2138159060 0x7f213815a150 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f213400e9f0 tx=0x7f213400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:39.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f212effd640 1 -- 172.21.15.125:0/476590624 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134009700 con 0x7f2138159060 2023-10-18T10:40:39.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213815ac80 con 0x7f2138159060 2023-10-18T10:40:39.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f212effd640 1 -- 172.21.15.125:0/476590624 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f213400ce70 con 0x7f2138159060 2023-10-18T10:40:39.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f212effd640 1 -- 172.21.15.125:0/476590624 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134010670 con 0x7f2138159060 2023-10-18T10:40:39.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.847+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21381e3830 con 0x7f2138159060 2023-10-18T10:40:39.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.848+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21381e39c0 con 0x7f2138159060 2023-10-18T10:40:39.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.848+0000 7f212effd640 1 -- 172.21.15.125:0/476590624 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 37) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f21340107d0 con 0x7f2138159060 2023-10-18T10:40:39.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.849+0000 7f212effd640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f211c06af60 0x7f211c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:39.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.849+0000 7f212effd640 1 -- 172.21.15.125:0/476590624 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f2134014070 con 0x7f2138159060 2023-10-18T10:40:39.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.849+0000 7f213e3eb640 1 -- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f211c06af60 msgr2=0x7f211c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3064166201 2023-10-18T10:40:39.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.849+0000 7f213e3eb640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f211c06af60 0x7f211c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:40:39.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:39.857+0000 7f212effd640 1 -- 172.21.15.125:0/476590624 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2134051ca0 con 0x7f2138159060 2023-10-18T10:40:39.881 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:39.880+0000 7fcec9e2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:40:39.986 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:39.984+0000 7f73fac25040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:40:40.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.030+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21380675f0 con 0x7f2138159060 2023-10-18T10:40:40.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.030+0000 7f212effd640 1 -- 172.21.15.125:0/476590624 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2134058eb0 con 0x7f2138159060 2023-10-18T10:40:40.032 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:40:40.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f211c06af60 msgr2=0x7f211c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:40:40.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f211c06af60 0x7f211c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2138159060 msgr2=0x7f213815a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2138159060 0x7f213815a150 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f213400e9f0 tx=0x7f213400eeb0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 shutdown_connections 2023-10-18T10:40:40.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:6832/3064166201,v1:172.21.15.139:6833/3064166201] conn(0x7f211c06af60 0x7f211c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f213815ccf0 0x7f2138158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2138159060 0x7f213815a150 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 --2- 172.21.15.125:0/476590624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21380ce240 0x7f213815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 >> 172.21.15.125:0/476590624 conn(0x7f21380b0670 msgr2=0x7f21380af990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:40.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 shutdown_connections 2023-10-18T10:40:40.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.032+0000 7f214064f640 1 -- 172.21.15.125:0/476590624 wait complete. 2023-10-18T10:40:40.037 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.032+0000 7f73fac25040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:40:40.045 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:40:40.133 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.131+0000 7f73fac25040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:40:40.157 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:40.156+0000 7fcec9e2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:40:40.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 -- 172.21.15.125:0/2504518292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 msgr2=0x7f44580cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 --2- 172.21.15.125:0/2504518292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 0x7f44580cf720 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f445808a810 tx=0x7f444002bd10 comp rx=0 tx=0).stop 2023-10-18T10:40:40.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 -- 172.21.15.125:0/2504518292 shutdown_connections 2023-10-18T10:40:40.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 --2- 172.21.15.125:0/2504518292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 0x7f44580cf720 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 --2- 172.21.15.125:0/2504518292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44580ceb40 0x7f44580b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 --2- 172.21.15.125:0/2504518292 >> v2:172.21.15.139:3300/0 conn(0x7f44580cf340 0x7f44580d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 -- 172.21.15.125:0/2504518292 >> 172.21.15.125:0/2504518292 conn(0x7f44580d0d60 msgr2=0x7f44580d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:40.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 -- 172.21.15.125:0/2504518292 shutdown_connections 2023-10-18T10:40:40.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.156+0000 7f445e54c640 1 -- 172.21.15.125:0/2504518292 wait complete. 2023-10-18T10:40:40.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.158+0000 7f445e54c640 1 Processor -- start 2023-10-18T10:40:40.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.158+0000 7f445e54c640 1 -- start start 2023-10-18T10:40:40.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445e54c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44580ceb40 0x7f44581e5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445e54c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44580cf340 0x7f4458158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445e54c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 0x7f445815d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445e54c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44581e65e0 con 0x7f445815b2a0 2023-10-18T10:40:40.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445e54c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44581e59d0 con 0x7f44580ceb40 2023-10-18T10:40:40.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445e54c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44581e5cb0 con 0x7f44580cf340 2023-10-18T10:40:40.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445cae9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 0x7f445815d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445cae9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 0x7f445815d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32798/0 (socket says 172.21.15.125:32798) 2023-10-18T10:40:40.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445cae9640 1 -- 172.21.15.125:0/39391797 learned_addr learned my addr 172.21.15.125:0/39391797 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:40.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f444ffff640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44580cf340 0x7f4458158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f4457fff640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44580ceb40 0x7f44581e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445cae9640 1 -- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44580cf340 msgr2=0x7f4458158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445cae9640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44580cf340 0x7f4458158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445cae9640 1 -- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44580ceb40 msgr2=0x7f44581e5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445cae9640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44580ceb40 0x7f44581e5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f445cae9640 1 -- 172.21.15.125:0/39391797 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440009560 con 0x7f445815b2a0 2023-10-18T10:40:40.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.159+0000 7f4457fff640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44580ceb40 0x7f44581e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:40.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.160+0000 7f445cae9640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 0x7f445815d770 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f445000eaa0 tx=0x7f445000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:40.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.160+0000 7f44557fa640 1 -- 172.21.15.125:0/39391797 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000ce70 con 0x7f445815b2a0 2023-10-18T10:40:40.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.160+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44581e5f90 con 0x7f445815b2a0 2023-10-18T10:40:40.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.160+0000 7f44557fa640 1 -- 172.21.15.125:0/39391797 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4450004c10 con 0x7f445815b2a0 2023-10-18T10:40:40.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.160+0000 7f44557fa640 1 -- 172.21.15.125:0/39391797 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450005020 con 0x7f445815b2a0 2023-10-18T10:40:40.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.160+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4458158940 con 0x7f445815b2a0 2023-10-18T10:40:40.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.161+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44580d04f0 con 0x7f445815b2a0 2023-10-18T10:40:40.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.161+0000 7f44557fa640 1 -- 172.21.15.125:0/39391797 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 38) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f44500051a0 con 0x7f445815b2a0 2023-10-18T10:40:40.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.162+0000 7f44557fa640 1 -- 172.21.15.125:0/39391797 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f4450014070 con 0x7f445815b2a0 2023-10-18T10:40:40.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.170+0000 7f44557fa640 1 -- 172.21.15.125:0/39391797 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4450055360 con 0x7f445815b2a0 2023-10-18T10:40:40.207 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.205+0000 7f737cc69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.208 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737cc69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.208 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737cc69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.208 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737cc69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.208 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737cc69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.209 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737fc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.209 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737fc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.209 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737fc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.209 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737fc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.209 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:40:40.206+0000 7f737fc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:40:40.215 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:40.214+0000 7fcec9e2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:40:40.320 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:40:40.318+0000 7fcec9e2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:40:40.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.346+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44580690e0 con 0x7f445815b2a0 2023-10-18T10:40:40.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.346+0000 7f44557fa640 1 -- 172.21.15.125:0/39391797 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f445001d020 con 0x7f445815b2a0 2023-10-18T10:40:40.348 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:40:40.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 msgr2=0x7f445815d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 0x7f445815d770 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f445000eaa0 tx=0x7f445000ef60 comp rx=0 tx=0).stop 2023-10-18T10:40:40.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 shutdown_connections 2023-10-18T10:40:40.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445815b2a0 0x7f445815d770 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44580cf340 0x7f4458158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 --2- 172.21.15.125:0/39391797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44580ceb40 0x7f44581e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 >> 172.21.15.125:0/39391797 conn(0x7f44580d0d60 msgr2=0x7f445815a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:40.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 shutdown_connections 2023-10-18T10:40:40.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.348+0000 7f445e54c640 1 -- 172.21.15.125:0/39391797 wait complete. 2023-10-18T10:40:40.361 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:40.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.462+0000 7fa2c9a10640 1 -- 172.21.15.125:0/3877010754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2ac003970 msgr2=0x7fa2ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.462+0000 7fa2c9a10640 1 --2- 172.21.15.125:0/3877010754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2ac003970 0x7fa2ac003d90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009ed0 tx=0x7fa2ac0057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.462+0000 7fa2c37fe640 1 -- 172.21.15.125:0/3877010754 reap_dead start 2023-10-18T10:40:40.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.462+0000 7fa2c9a10640 1 -- 172.21.15.125:0/3877010754 shutdown_connections 2023-10-18T10:40:40.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.462+0000 7fa2c9a10640 1 -- 172.21.15.125:0/3877010754 >> 172.21.15.125:0/3877010754 conn(0x7fa2c4069930 msgr2=0x7fa2c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:40.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.462+0000 7fa2c9a10640 1 -- 172.21.15.125:0/3877010754 shutdown_connections 2023-10-18T10:40:40.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.462+0000 7fa2c9a10640 1 -- 172.21.15.125:0/3877010754 wait complete. 2023-10-18T10:40:40.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.463+0000 7fa2c9a10640 1 Processor -- start 2023-10-18T10:40:40.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.464+0000 7fa2c9a10640 1 -- start start 2023-10-18T10:40:40.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.464+0000 7fa2c9a10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2ac003970 0x7fa2c40cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.464+0000 7fa2c9a10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c41e2300 0x7fa2c41e2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.464+0000 7fa2c9a10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c40a6f90 0x7fa2c40a7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.464+0000 7fa2c9a10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41e2cc0 con 0x7fa2c41e2300 2023-10-18T10:40:40.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.464+0000 7fa2c9a10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41dbc90 con 0x7fa2c40a6f90 2023-10-18T10:40:40.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.464+0000 7fa2c9a10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41dd130 con 0x7fa2ac003970 2023-10-18T10:40:40.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.464+0000 7fa2c27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c41e2300 0x7fa2c41e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c41e2300 0x7fa2c41e2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32822/0 (socket says 172.21.15.125:32822) 2023-10-18T10:40:40.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c27fc640 1 -- 172.21.15.125:0/1791737177 learned_addr learned my addr 172.21.15.125:0/1791737177 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:40.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c2ffd640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2ac003970 0x7fa2c40cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c37fe640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c40a6f90 0x7fa2c40a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c27fc640 1 -- 172.21.15.125:0/1791737177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2ac003970 msgr2=0x7fa2c40cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c27fc640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2ac003970 0x7fa2c40cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c27fc640 1 -- 172.21.15.125:0/1791737177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c40a6f90 msgr2=0x7fa2c40a7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c27fc640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c40a6f90 0x7fa2c40a7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c27fc640 1 -- 172.21.15.125:0/1791737177 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c41dd410 con 0x7fa2c41e2300 2023-10-18T10:40:40.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c27fc640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c41e2300 0x7fa2c41e2780 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8011b50 tx=0x7fa2b800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:40.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2abfff640 1 -- 172.21.15.125:0/1791737177 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800e7c0 con 0x7fa2c41e2300 2023-10-18T10:40:40.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2abfff640 1 -- 172.21.15.125:0/1791737177 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2b800e920 con 0x7fa2c41e2300 2023-10-18T10:40:40.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c41dd6f0 con 0x7fa2c41e2300 2023-10-18T10:40:40.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2abfff640 1 -- 172.21.15.125:0/1791737177 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800ecd0 con 0x7fa2c41e2300 2023-10-18T10:40:40.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.465+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c41ddcb0 con 0x7fa2c41e2300 2023-10-18T10:40:40.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.466+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c40b1bd0 con 0x7fa2c41e2300 2023-10-18T10:40:40.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.467+0000 7fa2abfff640 1 -- 172.21.15.125:0/1791737177 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 38) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa2b8019430 con 0x7fa2c41e2300 2023-10-18T10:40:40.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.467+0000 7fa2abfff640 1 -- 172.21.15.125:0/1791737177 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6717+0+0 (secure 0 0 0) 0x7fa2b808a680 con 0x7fa2c41e2300 2023-10-18T10:40:40.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.471+0000 7fa2abfff640 1 -- 172.21.15.125:0/1791737177 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2b8055840 con 0x7fa2c41e2300 2023-10-18T10:40:40.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.669+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2c40690e0 con 0x7fa2c41e2300 2023-10-18T10:40:40.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.670+0000 7fa2abfff640 1 -- 172.21.15.125:0/1791737177 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fa2b801e090 con 0x7fa2c41e2300 2023-10-18T10:40:40.672 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:40.672 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/693466412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":693466412},{"type":"v1","addr":"172.21.15.139:6836","nonce":693466412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6277,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/170638031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":170638031},{"type":"v1","addr":"172.21.15.139:6837","nonce":170638031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":7068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3024029084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3024029084},{"type":"v1","addr":"172.21.15.125:6834","nonce":3024029084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:19.231823+0000","modified":"2023-10-18T10:40:21.240206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1136111019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1136111019},{"type":"v1","addr":"172.21.15.125:6835","nonce":1136111019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T10:40:40.673 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 29 2023-10-18T10:40:40.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c41e2300 msgr2=0x7fa2c41e2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c41e2300 0x7fa2c41e2780 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8011b50 tx=0x7fa2b800f790 comp rx=0 tx=0).stop 2023-10-18T10:40:40.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 shutdown_connections 2023-10-18T10:40:40.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c40a6f90 0x7fa2c40a7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c41e2300 0x7fa2c41e2780 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 --2- 172.21.15.125:0/1791737177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2ac003970 0x7fa2c40cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 >> 172.21.15.125:0/1791737177 conn(0x7fa2c4069930 msgr2=0x7fa2c41df900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:40.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 shutdown_connections 2023-10-18T10:40:40.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.672+0000 7fa2c9a10640 1 -- 172.21.15.125:0/1791737177 wait complete. 2023-10-18T10:40:40.684 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:40.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.790+0000 7f6ad3402640 1 -- 172.21.15.125:0/1805896570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6acc0abc10 msgr2=0x7f6acc06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.790+0000 7f6ad3402640 1 --2- 172.21.15.125:0/1805896570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6acc0abc10 0x7f6acc06b2e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4005fd0 tx=0x7f6ac402c170 comp rx=0 tx=0).stop 2023-10-18T10:40:40.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.790+0000 7f6ad3402640 1 -- 172.21.15.125:0/1805896570 shutdown_connections 2023-10-18T10:40:40.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.790+0000 7f6ad3402640 1 --2- 172.21.15.125:0/1805896570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6acc0abc10 0x7f6acc06b2e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.790+0000 7f6ad3402640 1 -- 172.21.15.125:0/1805896570 >> 172.21.15.125:0/1805896570 conn(0x7f6acc069930 msgr2=0x7f6acc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:40.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.791+0000 7f6ad3402640 1 -- 172.21.15.125:0/1805896570 shutdown_connections 2023-10-18T10:40:40.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.791+0000 7f6ad3402640 1 -- 172.21.15.125:0/1805896570 wait complete. 2023-10-18T10:40:40.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad3402640 1 Processor -- start 2023-10-18T10:40:40.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad3402640 1 -- start start 2023-10-18T10:40:40.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad3402640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6acc06af00 0x7f6acc13a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad3402640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6acc0abc10 0x7f6acc13d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad3402640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6acc0c9dc0 0x7f6acc13d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:40.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad3402640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0b05a0 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad3402640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0a7cd0 con 0x7f6acc0abc10 2023-10-18T10:40:40.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad3402640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0a7710 con 0x7f6acc06af00 2023-10-18T10:40:40.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.792+0000 7f6ad199f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6acc0c9dc0 0x7f6acc13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad199f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6acc0c9dc0 0x7f6acc13d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32826/0 (socket says 172.21.15.125:32826) 2023-10-18T10:40:40.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad199f640 1 -- 172.21.15.125:0/113194636 learned_addr learned my addr 172.21.15.125:0/113194636 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:40.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad119e640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6acc06af00 0x7f6acc13a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad099d640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6acc0abc10 0x7f6acc13d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:40.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad199f640 1 -- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6acc06af00 msgr2=0x7f6acc13a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad199f640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6acc06af00 0x7f6acc13a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad199f640 1 -- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6acc0abc10 msgr2=0x7f6acc13d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad199f640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6acc0abc10 0x7f6acc13d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad199f640 1 -- 172.21.15.125:0/113194636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac4009840 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad119e640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6acc06af00 0x7f6acc13a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:40.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad199f640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6acc0c9dc0 0x7f6acc13d940 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4036040 tx=0x7f6ac4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:40.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ac1ffb640 1 -- 172.21.15.125:0/113194636 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4004670 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ac1ffb640 1 -- 172.21.15.125:0/113194636 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ac40345a0 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ac1ffb640 1 -- 172.21.15.125:0/113194636 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac40349b0 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc0a79f0 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.793+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc13a850 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.795+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc15a670 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.795+0000 7f6ac1ffb640 1 -- 172.21.15.125:0/113194636 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 38) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f6ac4034b10 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.795+0000 7f6ac1ffb640 1 -- 172.21.15.125:0/113194636 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f6ac40ac680 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.800+0000 7f6ac1ffb640 1 -- 172.21.15.125:0/113194636 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ac4077790 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.967+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6acc159a00 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.969+0000 7f6ac1ffb640 1 -- 172.21.15.125:0/113194636 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+17779 (secure 0 0 0) 0x7f6ac407adb0 con 0x7f6acc0c9dc0 2023-10-18T10:40:40.971 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:40.971 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":86,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:40.145597+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:17.060551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:18.211723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3479864824":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/3728504787":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/627668147":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/3234267299":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6832/3064166201":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/938869555":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/1217443296":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2948747997":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/3110681241":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/1795019078":"2023-10-19T10:40:34.763606+0000","172.21.15.139:6833/3064166201":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6833/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/578014551":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6832/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/686709880":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2487642114":"2023-10-19T10:40:34.763606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:40.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.970+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6acc0c9dc0 msgr2=0x7f6acc13d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:40.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.970+0000 7f6ad3402640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6acc0c9dc0 0x7f6acc13d940 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4036040 tx=0x7f6ac4004270 comp rx=0 tx=0).stop 2023-10-18T10:40:40.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.971+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 shutdown_connections 2023-10-18T10:40:40.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.971+0000 7f6ad3402640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6acc0abc10 0x7f6acc13d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.971+0000 7f6ad3402640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6acc0c9dc0 0x7f6acc13d940 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.971+0000 7f6ad3402640 1 --2- 172.21.15.125:0/113194636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6acc06af00 0x7f6acc13a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:40.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.971+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 >> 172.21.15.125:0/113194636 conn(0x7f6acc069930 msgr2=0x7f6acc069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:40.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.971+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 shutdown_connections 2023-10-18T10:40:40.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:40.971+0000 7f6ad3402640 1 -- 172.21.15.125:0/113194636 wait complete. 2023-10-18T10:40:40.983 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:40:40.984 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:41.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.084+0000 7f78b72f2640 1 -- 172.21.15.125:0/1320132044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00b0530 msgr2=0x7f78b00b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.084+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1320132044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00b0530 0x7f78b00b09a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f78b00cc490 tx=0x7f78a802be40 comp rx=0 tx=0).stop 2023-10-18T10:40:41.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.084+0000 7f78b72f2640 1 -- 172.21.15.125:0/1320132044 shutdown_connections 2023-10-18T10:40:41.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.084+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1320132044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00b0530 0x7f78b00b09a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.084+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1320132044 >> v2:172.21.15.139:3300/0 conn(0x7f78b00afa20 0x7f78b0158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.084+0000 7f78b72f2640 1 -- 172.21.15.125:0/1320132044 >> 172.21.15.125:0/1320132044 conn(0x7f78b0069930 msgr2=0x7f78b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:41.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.084+0000 7f78b72f2640 1 -- 172.21.15.125:0/1320132044 shutdown_connections 2023-10-18T10:40:41.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.084+0000 7f78b72f2640 1 -- 172.21.15.125:0/1320132044 wait complete. 2023-10-18T10:40:41.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.085+0000 7f78b72f2640 1 Processor -- start 2023-10-18T10:40:41.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.085+0000 7f78b72f2640 1 -- start start 2023-10-18T10:40:41.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b72f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78b00afa20 0x7f78b0147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b72f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78b00b0530 0x7f78b0141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b72f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00cbd20 0x7f78b0140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b72f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0145630 con 0x7f78b00cbd20 2023-10-18T10:40:41.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b72f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0145ed0 con 0x7f78b00afa20 2023-10-18T10:40:41.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b72f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01420e0 con 0x7f78b00b0530 2023-10-18T10:40:41.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00cbd20 0x7f78b0140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00cbd20 0x7f78b0140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32842/0 (socket says 172.21.15.125:32842) 2023-10-18T10:40:41.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 -- 172.21.15.125:0/1339468432 learned_addr learned my addr 172.21.15.125:0/1339468432 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:41.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b488d640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78b00b0530 0x7f78b0141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b508e640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78b00afa20 0x7f78b0147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 -- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78b00b0530 msgr2=0x7f78b0141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78b00b0530 0x7f78b0141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 -- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78b00afa20 msgr2=0x7f78b0147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78b00afa20 0x7f78b0147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 -- 172.21.15.125:0/1339468432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8009560 con 0x7f78b00cbd20 2023-10-18T10:40:41.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.086+0000 7f78b588f640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00cbd20 0x7f78b0140970 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f78a8036040 tx=0x7f78a80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:41.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.087+0000 7f78a5ffb640 1 -- 172.21.15.125:0/1339468432 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8003800 con 0x7f78b00cbd20 2023-10-18T10:40:41.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.087+0000 7f78a5ffb640 1 -- 172.21.15.125:0/1339468432 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78a8003a50 con 0x7f78b00cbd20 2023-10-18T10:40:41.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.087+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b0142380 con 0x7f78b00cbd20 2023-10-18T10:40:41.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.087+0000 7f78a5ffb640 1 -- 172.21.15.125:0/1339468432 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8003e40 con 0x7f78b00cbd20 2023-10-18T10:40:41.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.087+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b0145b60 con 0x7f78b00cbd20 2023-10-18T10:40:41.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.088+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b00690e0 con 0x7f78b00cbd20 2023-10-18T10:40:41.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.088+0000 7f78a5ffb640 1 -- 172.21.15.125:0/1339468432 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 38) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f78a8034440 con 0x7f78b00cbd20 2023-10-18T10:40:41.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.089+0000 7f78a5ffb640 1 -- 172.21.15.125:0/1339468432 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f78a80ae2b0 con 0x7f78b00cbd20 2023-10-18T10:40:41.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.093+0000 7f78a5ffb640 1 -- 172.21.15.125:0/1339468432 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78a80794e0 con 0x7f78b00cbd20 2023-10-18T10:40:41.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.155+0000 7f78a5ffb640 1 -- 172.21.15.125:0/1339468432 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f78a8078f90 con 0x7f78b00cbd20 2023-10-18T10:40:41.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.155+0000 7f78a5ffb640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7894070b20 0x7f7894072f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.156+0000 7f78b508e640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7894070b20 0x7f7894072f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.158+0000 7f78b508e640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7894070b20 0x7f7894072f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f78b00aeb70 tx=0x7f78ac009040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:41.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.300+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78b006f700 con 0x7f78b00cbd20 2023-10-18T10:40:41.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.301+0000 7f78a5ffb640 1 -- 172.21.15.125:0/1339468432 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f78a8092b70 con 0x7f78b00cbd20 2023-10-18T10:40:41.303 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:41.303 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/693466412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":693466412},{"type":"v1","addr":"172.21.15.139:6836","nonce":693466412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6277,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/170638031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":170638031},{"type":"v1","addr":"172.21.15.139:6837","nonce":170638031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":7068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3024029084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3024029084},{"type":"v1","addr":"172.21.15.125:6834","nonce":3024029084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:19.231823+0000","modified":"2023-10-18T10:40:21.240206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1136111019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1136111019},{"type":"v1","addr":"172.21.15.125:6835","nonce":1136111019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T10:40:41.304 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 29 2023-10-18T10:40:41.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7894070b20 msgr2=0x7f7894072f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7894070b20 0x7f7894072f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f78b00aeb70 tx=0x7f78ac009040 comp rx=0 tx=0).stop 2023-10-18T10:40:41.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00cbd20 msgr2=0x7f78b0140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00cbd20 0x7f78b0140970 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f78a8036040 tx=0x7f78a80045e0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 shutdown_connections 2023-10-18T10:40:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7894070b20 0x7f7894072f10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78b00cbd20 0x7f78b0140970 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78b00b0530 0x7f78b0141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 --2- 172.21.15.125:0/1339468432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78b00afa20 0x7f78b0147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 >> 172.21.15.125:0/1339468432 conn(0x7f78b0069930 msgr2=0x7f78b014baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 shutdown_connections 2023-10-18T10:40:41.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.303+0000 7f78b72f2640 1 -- 172.21.15.125:0/1339468432 wait complete. 2023-10-18T10:40:41.315 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:41.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.415+0000 7f6776264640 1 -- 172.21.15.125:0/292118978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 msgr2=0x7f67700affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.415+0000 7f6776264640 1 --2- 172.21.15.125:0/292118978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 0x7f67700affe0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f67700cbdc0 tx=0x7f676802c260 comp rx=0 tx=0).stop 2023-10-18T10:40:41.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.415+0000 7f6776264640 1 -- 172.21.15.125:0/292118978 shutdown_connections 2023-10-18T10:40:41.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.415+0000 7f6776264640 1 --2- 172.21.15.125:0/292118978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 0x7f67700affe0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.415+0000 7f6776264640 1 --2- 172.21.15.125:0/292118978 >> v2:172.21.15.139:3300/0 conn(0x7f6770141690 0x7f67700703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.415+0000 7f6776264640 1 -- 172.21.15.125:0/292118978 >> 172.21.15.125:0/292118978 conn(0x7f6770069930 msgr2=0x7f6770069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:41.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.415+0000 7f6776264640 1 -- 172.21.15.125:0/292118978 shutdown_connections 2023-10-18T10:40:41.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.415+0000 7f6776264640 1 -- 172.21.15.125:0/292118978 wait complete. 2023-10-18T10:40:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.417+0000 7f6776264640 1 Processor -- start 2023-10-18T10:40:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.417+0000 7f6776264640 1 -- start start 2023-10-18T10:40:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.417+0000 7f6776264640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 0x7f67701e2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.417+0000 7f6776264640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677013e520 0x7f67700a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.417+0000 7f6776264640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6770141690 0x7f67700a8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.417+0000 7f6776264640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700a7af0 con 0x7f6770070920 2023-10-18T10:40:41.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f6776264640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700abeb0 con 0x7f677013e520 2023-10-18T10:40:41.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f6776264640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700ac190 con 0x7f6770141690 2023-10-18T10:40:41.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 0x7f67701e2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 0x7f67701e2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32862/0 (socket says 172.21.15.125:32862) 2023-10-18T10:40:41.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 -- 172.21.15.125:0/2104025406 learned_addr learned my addr 172.21.15.125:0/2104025406 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:41.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676f7fe640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677013e520 0x7f67700a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f6774801640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6770141690 0x7f67700a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 -- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6770141690 msgr2=0x7f67700a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6770141690 0x7f67700a8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 -- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677013e520 msgr2=0x7f67700a7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677013e520 0x7f67700a7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 -- 172.21.15.125:0/2104025406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768009900 con 0x7f6770070920 2023-10-18T10:40:41.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676ffff640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 0x7f67701e2c20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f676000c450 tx=0x7f676000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:41.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676cff9640 1 -- 172.21.15.125:0/2104025406 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600052d0 con 0x7f6770070920 2023-10-18T10:40:41.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676cff9640 1 -- 172.21.15.125:0/2104025406 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6760005430 con 0x7f6770070920 2023-10-18T10:40:41.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67700a7430 con 0x7f6770070920 2023-10-18T10:40:41.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f676cff9640 1 -- 172.21.15.125:0/2104025406 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760012890 con 0x7f6770070920 2023-10-18T10:40:41.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.418+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67701dd7c0 con 0x7f6770070920 2023-10-18T10:40:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.419+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67700ac400 con 0x7f6770070920 2023-10-18T10:40:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.420+0000 7f676cff9640 1 -- 172.21.15.125:0/2104025406 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f67600035f0 con 0x7f6770070920 2023-10-18T10:40:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.420+0000 7f676cff9640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f675c06ac90 0x7f675c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.420+0000 7f676cff9640 1 -- 172.21.15.125:0/2104025406 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f67600891c0 con 0x7f6770070920 2023-10-18T10:40:41.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.420+0000 7f676f7fe640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f675c06ac90 0x7f675c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.421+0000 7f676f7fe640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f675c06ac90 0x7f675c06d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67701e3160 tx=0x7f6764005f90 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:41.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.425+0000 7f676cff9640 1 -- 172.21.15.125:0/2104025406 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67600541f0 con 0x7f6770070920 2023-10-18T10:40:41.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.624+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67700a7c30 con 0x7f6770070920 2023-10-18T10:40:41.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.625+0000 7f676cff9640 1 -- 172.21.15.125:0/2104025406 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f67700a7c30 con 0x7f6770070920 2023-10-18T10:40:41.627 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:41.627 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/693466412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":693466412},{"type":"v1","addr":"172.21.15.139:6836","nonce":693466412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6277,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/170638031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":170638031},{"type":"v1","addr":"172.21.15.139:6837","nonce":170638031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":7068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3024029084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3024029084},{"type":"v1","addr":"172.21.15.125:6834","nonce":3024029084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:19.231823+0000","modified":"2023-10-18T10:40:21.240206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7071":{"gid":7071,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1136111019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1136111019},{"type":"v1","addr":"172.21.15.125:6835","nonce":1136111019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T10:40:41.628 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 29 2023-10-18T10:40:41.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f675c06ac90 msgr2=0x7f675c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f675c06ac90 0x7f675c06d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67701e3160 tx=0x7f6764005f90 comp rx=0 tx=0).stop 2023-10-18T10:40:41.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 msgr2=0x7f67701e2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 0x7f67701e2c20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f676000c450 tx=0x7f676000c910 comp rx=0 tx=0).stop 2023-10-18T10:40:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 shutdown_connections 2023-10-18T10:40:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f675c06ac90 0x7f675c06d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677013e520 0x7f67700a7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6770141690 0x7f67700a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 --2- 172.21.15.125:0/2104025406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6770070920 0x7f67701e2c20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 >> 172.21.15.125:0/2104025406 conn(0x7f6770069930 msgr2=0x7f677013d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:41.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 shutdown_connections 2023-10-18T10:40:41.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.627+0000 7f6776264640 1 -- 172.21.15.125:0/2104025406 wait complete. 2023-10-18T10:40:41.639 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:41.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.740+0000 7fc11ed8e640 1 -- 172.21.15.125:0/1814097218 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc11813e520 msgr2=0x7fc11806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.740+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/1814097218 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc11813e520 0x7fc11806bb60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b20f0 tx=0x7fc10c02bdb0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.740+0000 7fc11ed8e640 1 -- 172.21.15.125:0/1814097218 shutdown_connections 2023-10-18T10:40:41.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.740+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/1814097218 >> v2:172.21.15.139:3300/0 conn(0x7fc118070af0 0x7fc1180affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.740+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/1814097218 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc11813e520 0x7fc11806bb60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.740+0000 7fc11ed8e640 1 -- 172.21.15.125:0/1814097218 >> 172.21.15.125:0/1814097218 conn(0x7fc118069930 msgr2=0x7fc118069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:41.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.741+0000 7fc11ed8e640 1 -- 172.21.15.125:0/1814097218 shutdown_connections 2023-10-18T10:40:41.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.741+0000 7fc11ed8e640 1 -- 172.21.15.125:0/1814097218 wait complete. 2023-10-18T10:40:41.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.742+0000 7fc11ed8e640 1 Processor -- start 2023-10-18T10:40:41.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.742+0000 7fc11ed8e640 1 -- start start 2023-10-18T10:40:41.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.742+0000 7fc11ed8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc118070af0 0x7fc118156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.742+0000 7fc11ed8e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11813e520 0x7fc11813af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11ed8e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc118141690 0x7fc118151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11ed8e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118151340 con 0x7fc118141690 2023-10-18T10:40:41.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11ed8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118156650 con 0x7fc118070af0 2023-10-18T10:40:41.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11ed8e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11813b4c0 con 0x7fc11813e520 2023-10-18T10:40:41.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc118141690 0x7fc118151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc118141690 0x7fc118151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32882/0 (socket says 172.21.15.125:32882) 2023-10-18T10:40:41.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 -- 172.21.15.125:0/3507243380 learned_addr learned my addr 172.21.15.125:0/3507243380 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:41.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc117fff640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11813e520 0x7fc11813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11cb2a640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc118070af0 0x7fc118156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 -- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11813e520 msgr2=0x7fc11813af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11813e520 0x7fc11813af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 -- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc118070af0 msgr2=0x7fc118156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc118070af0 0x7fc118156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 -- 172.21.15.125:0/3507243380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009560 con 0x7fc118141690 2023-10-18T10:40:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11cb2a640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc118070af0 0x7fc118156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11d32b640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc118141690 0x7fc118151ee0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc11000eaa0 tx=0x7fc11000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc1157fa640 1 -- 172.21.15.125:0/3507243380 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000cdb0 con 0x7fc118141690 2023-10-18T10:40:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc1157fa640 1 -- 172.21.15.125:0/3507243380 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc110004b20 con 0x7fc118141690 2023-10-18T10:40:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc118151060 con 0x7fc118141690 2023-10-18T10:40:41.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc1157fa640 1 -- 172.21.15.125:0/3507243380 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110005020 con 0x7fc118141690 2023-10-18T10:40:41.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.743+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc118155910 con 0x7fc118141690 2023-10-18T10:40:41.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.744+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1180b1bd0 con 0x7fc118141690 2023-10-18T10:40:41.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.745+0000 7fc1157fa640 1 -- 172.21.15.125:0/3507243380 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc110005180 con 0x7fc118141690 2023-10-18T10:40:41.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.745+0000 7fc1157fa640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc0fc06ac90 0x7fc0fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:41.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.745+0000 7fc1157fa640 1 -- 172.21.15.125:0/3507243380 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6717+0+0 (secure 0 0 0) 0x7fc110089f40 con 0x7fc118141690 2023-10-18T10:40:41.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.745+0000 7fc11cb2a640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc0fc06ac90 0x7fc0fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:41.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.746+0000 7fc11cb2a640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc0fc06ac90 0x7fc0fc06d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc10c035000 tx=0x7fc10c004730 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:41.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.749+0000 7fc1157fa640 1 -- 172.21.15.125:0/3507243380 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc110054ff0 con 0x7fc118141690 2023-10-18T10:40:41.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.917+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1180690e0 con 0x7fc118141690 2023-10-18T10:40:41.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.918+0000 7fc1157fa640 1 -- 172.21.15.125:0/3507243380 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+17779 (secure 0 0 0) 0x7fc11001f070 con 0x7fc118141690 2023-10-18T10:40:41.920 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:41.921 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":86,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:40.145597+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:17.060551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:18.211723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3479864824":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/3728504787":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/627668147":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/3234267299":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6832/3064166201":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/938869555":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/1217443296":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2948747997":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/3110681241":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/1795019078":"2023-10-19T10:40:34.763606+0000","172.21.15.139:6833/3064166201":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6833/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/578014551":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6832/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/686709880":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2487642114":"2023-10-19T10:40:34.763606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:41.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.920+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc0fc06ac90 msgr2=0x7fc0fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.920+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc0fc06ac90 0x7fc0fc06d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc10c035000 tx=0x7fc10c004730 comp rx=0 tx=0).stop 2023-10-18T10:40:41.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.920+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc118141690 msgr2=0x7fc118151ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:41.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.920+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc118141690 0x7fc118151ee0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc11000eaa0 tx=0x7fc11000ef60 comp rx=0 tx=0).stop 2023-10-18T10:40:41.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.921+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 shutdown_connections 2023-10-18T10:40:41.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.921+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc0fc06ac90 0x7fc0fc06d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.921+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc118141690 0x7fc118151ee0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.921+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc11813e520 0x7fc11813af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.921+0000 7fc11ed8e640 1 --2- 172.21.15.125:0/3507243380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc118070af0 0x7fc118156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:41.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.921+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 >> 172.21.15.125:0/3507243380 conn(0x7fc118069930 msgr2=0x7fc1180af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:41.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.921+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 shutdown_connections 2023-10-18T10:40:41.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:41.921+0000 7fc11ed8e640 1 -- 172.21.15.125:0/3507243380 wait complete. 2023-10-18T10:40:41.933 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:40:42.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.034+0000 7f0622505640 1 -- 172.21.15.125:0/3463076440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 msgr2=0x7f061c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:42.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.034+0000 7f0622505640 1 --2- 172.21.15.125:0/3463076440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c0d1910 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f060c005fd0 tx=0x7f060c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:42.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.034+0000 7f0622505640 1 -- 172.21.15.125:0/3463076440 shutdown_connections 2023-10-18T10:40:42.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.034+0000 7f0622505640 1 --2- 172.21.15.125:0/3463076440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c0d1910 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.034+0000 7f0622505640 1 -- 172.21.15.125:0/3463076440 >> 172.21.15.125:0/3463076440 conn(0x7f061c0d0d60 msgr2=0x7f061c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:42.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.034+0000 7f0622505640 1 -- 172.21.15.125:0/3463076440 shutdown_connections 2023-10-18T10:40:42.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.034+0000 7f0622505640 1 -- 172.21.15.125:0/3463076440 wait complete. 2023-10-18T10:40:42.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.035+0000 7f0622505640 1 Processor -- start 2023-10-18T10:40:42.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.035+0000 7f0622505640 1 -- start start 2023-10-18T10:40:42.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0622505640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:42.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0622505640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c0cf340 0x7f061c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:42.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0622505640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c15b2a0 0x7f061c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:42.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0622505640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0b6a70 con 0x7f061c15b2a0 2023-10-18T10:40:42.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0622505640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c15cee0 con 0x7f061c0ceb40 2023-10-18T10:40:42.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0622505640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c15d1c0 con 0x7f061c0cf340 2023-10-18T10:40:42.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0620aa2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c15b2a0 0x7f061c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:42.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0620aa2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c15b2a0 0x7f061c15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32904/0 (socket says 172.21.15.125:32904) 2023-10-18T10:40:42.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f0620aa2640 1 -- 172.21.15.125:0/306109844 learned_addr learned my addr 172.21.15.125:0/306109844 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:42.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f061b7fe640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c0cf340 0x7f061c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:42.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.036+0000 7f061bfff640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:42.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0620aa2640 1 -- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c0cf340 msgr2=0x7f061c0b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:42.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0620aa2640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c0cf340 0x7f061c0b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0620aa2640 1 -- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 msgr2=0x7f061c0b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0620aa2640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c0b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0620aa2640 1 -- 172.21.15.125:0/306109844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c009560 con 0x7f061c15b2a0 2023-10-18T10:40:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f061b7fe640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c0cf340 0x7f061c0b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f061bfff640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c0b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0620aa2640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c15b2a0 0x7f061c15b6d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f061400c710 tx=0x7f061400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0618ff9640 1 -- 172.21.15.125:0/306109844 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614006d50 con 0x7f061c15b2a0 2023-10-18T10:40:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0618ff9640 1 -- 172.21.15.125:0/306109844 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f061400ce20 con 0x7f061c15b2a0 2023-10-18T10:40:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c15d4a0 con 0x7f061c15b2a0 2023-10-18T10:40:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0618ff9640 1 -- 172.21.15.125:0/306109844 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614003bd0 con 0x7f061c15b2a0 2023-10-18T10:40:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.037+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c0b5570 con 0x7f061c15b2a0 2023-10-18T10:40:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.038+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c15d9c0 con 0x7f061c15b2a0 2023-10-18T10:40:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.038+0000 7f0618ff9640 1 -- 172.21.15.125:0/306109844 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0614003d30 con 0x7f061c15b2a0 2023-10-18T10:40:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.039+0000 7f0618ff9640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f060806acc0 0x7f060806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.039+0000 7f0618ff9640 1 -- 172.21.15.125:0/306109844 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f0614089420 con 0x7f061c15b2a0 2023-10-18T10:40:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.039+0000 7f061bfff640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f060806acc0 0x7f060806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:42.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.040+0000 7f061bfff640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f060806acc0 0x7f060806d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f060c035000 tx=0x7f060c004730 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:42.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.043+0000 7f0618ff9640 1 -- 172.21.15.125:0/306109844 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0614054420 con 0x7f061c15b2a0 2023-10-18T10:40:42.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.247+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f061c0690e0 con 0x7f061c15b2a0 2023-10-18T10:40:42.249 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T10:40:42.247+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:40:42.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.249+0000 7f0618ff9640 1 -- 172.21.15.125:0/306109844 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0614057a40 con 0x7f061c15b2a0 2023-10-18T10:40:42.251 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:40:42.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f060806acc0 msgr2=0x7f060806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:42.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f060806acc0 0x7f060806d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f060c035000 tx=0x7f060c004730 comp rx=0 tx=0).stop 2023-10-18T10:40:42.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c15b2a0 msgr2=0x7f061c15b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:42.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c15b2a0 0x7f061c15b6d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f061400c710 tx=0x7f061400cbd0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 shutdown_connections 2023-10-18T10:40:42.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f060806acc0 0x7f060806d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c15b2a0 0x7f061c15b6d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c0cf340 0x7f061c0b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 --2- 172.21.15.125:0/306109844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c0b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 >> 172.21.15.125:0/306109844 conn(0x7f061c0d0d60 msgr2=0x7f061c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:42.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 shutdown_connections 2023-10-18T10:40:42.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.251+0000 7f0622505640 1 -- 172.21.15.125:0/306109844 wait complete. 2023-10-18T10:40:42.256 INFO:tasks.ceph.mds.a.smithi125.stderr: -4007> 2023-10-18T10:40:14.140+0000 7ff1e5b7b640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-18T10:40:42.256 INFO:tasks.ceph.mds.a.smithi125.stderr: -2822> 2023-10-18T10:40:24.881+0000 7ff1dfb6f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:40:42.264 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:40:42.281 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:40:42.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.367+0000 7ff9ee15e640 1 -- 172.21.15.125:0/232248127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd1c0 msgr2=0x7ff9e815c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:42.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.367+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/232248127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd1c0 0x7ff9e815c0e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8073460 tx=0x7ff9dc02bda0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.368+0000 7ff9ee15e640 1 -- 172.21.15.125:0/232248127 shutdown_connections 2023-10-18T10:40:42.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.368+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/232248127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd1c0 0x7ff9e815c0e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.368+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/232248127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9e80cd9e0 0x7ff9e80cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.368+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/232248127 >> v2:172.21.15.139:3300/0 conn(0x7ff9e8158910 0x7ff9e8158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.368+0000 7ff9ee15e640 1 -- 172.21.15.125:0/232248127 >> 172.21.15.125:0/232248127 conn(0x7ff9e80af7a0 msgr2=0x7ff9e80afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:42.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.368+0000 7ff9ee15e640 1 -- 172.21.15.125:0/232248127 shutdown_connections 2023-10-18T10:40:42.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.368+0000 7ff9ee15e640 1 -- 172.21.15.125:0/232248127 wait complete. 2023-10-18T10:40:42.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.369+0000 7ff9ee15e640 1 Processor -- start 2023-10-18T10:40:42.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.369+0000 7ff9ee15e640 1 -- start start 2023-10-18T10:40:42.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9ee15e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e80cd1c0 0x7ff9e80aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:42.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9ee15e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd9e0 0x7ff9e80ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:42.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9ee15e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9e8158910 0x7ff9e80ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:42.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9ee15e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80aae00 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9ee15e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e81f5e60 con 0x7ff9e80cd1c0 2023-10-18T10:40:42.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9ee15e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e81f52e0 con 0x7ff9e8158910 2023-10-18T10:40:42.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd9e0 0x7ff9e80ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:42.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd9e0 0x7ff9e80ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32946/0 (socket says 172.21.15.125:32946) 2023-10-18T10:40:42.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 -- 172.21.15.125:0/2786136838 learned_addr learned my addr 172.21.15.125:0/2786136838 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:42.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e77fe640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e80cd1c0 0x7ff9e80aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:42.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e7fff640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9e8158910 0x7ff9e80ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:42.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 -- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9e8158910 msgr2=0x7ff9e80ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:42.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9e8158910 0x7ff9e80ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 -- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e80cd1c0 msgr2=0x7ff9e80aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:42.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e80cd1c0 0x7ff9e80aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:42.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 -- 172.21.15.125:0/2786136838 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc009560 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9e6ffd640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd9e0 0x7ff9e80ae870 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0099c0 tx=0x7ff9dc02db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:42.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9cffff640 1 -- 172.21.15.125:0/2786136838 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc003800 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9cffff640 1 -- 172.21.15.125:0/2786136838 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9dc003a50 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e81f55c0 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9cffff640 1 -- 172.21.15.125:0/2786136838 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc003e60 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.370+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e81f5b80 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.371+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e80690e0 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.372+0000 7ff9cffff640 1 -- 172.21.15.125:0/2786136838 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 39) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff9dc03e450 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.372+0000 7ff9cffff640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff9d406acc0 0x7ff9d406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:42.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.372+0000 7ff9cffff640 1 -- 172.21.15.125:0/2786136838 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6803+0+0 (secure 0 0 0) 0x7ff9dc0ac1b0 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.373+0000 7ff9e77fe640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff9d406acc0 0x7ff9d406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:42.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.373+0000 7ff9e77fe640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff9d406acc0 0x7ff9d406d180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80092f0 tx=0x7ff9d8009260 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:42.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.377+0000 7ff9cffff640 1 -- 172.21.15.125:0/2786136838 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9dc0771f0 con 0x7ff9e80cd9e0 2023-10-18T10:40:42.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:42.581+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff9e80ce5e0 con 0x7ff9e80cd9e0 2023-10-18T10:40:43.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.166+0000 7ff9cffff640 1 -- 172.21.15.125:0/2786136838 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 40) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ff9dc0775c0 con 0x7ff9e80cd9e0 2023-10-18T10:40:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.250+0000 7ff9cffff640 1 -- 172.21.15.125:0/2786136838 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff9dc091310 con 0x7ff9e80cd9e0 2023-10-18T10:40:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff9d406acc0 msgr2=0x7ff9d406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff9d406acc0 0x7ff9d406d180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80092f0 tx=0x7ff9d8009260 comp rx=0 tx=0).stop 2023-10-18T10:40:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd9e0 msgr2=0x7ff9e80ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd9e0 0x7ff9e80ae870 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0099c0 tx=0x7ff9dc02db40 comp rx=0 tx=0).stop 2023-10-18T10:40:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 shutdown_connections 2023-10-18T10:40:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff9d406acc0 0x7ff9d406d180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9e8158910 0x7ff9e80ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9e80cd9e0 0x7ff9e80ae870 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 --2- 172.21.15.125:0/2786136838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e80cd1c0 0x7ff9e80aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 >> 172.21.15.125:0/2786136838 conn(0x7ff9e80af7a0 msgr2=0x7ff9e8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 shutdown_connections 2023-10-18T10:40:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.254+0000 7ff9ee15e640 1 -- 172.21.15.125:0/2786136838 wait complete. 2023-10-18T10:40:43.269 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:40:43.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 -- 172.21.15.125:0/2043832331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 msgr2=0x7f81100ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:43.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 --2- 172.21.15.125:0/2043832331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 0x7f81100ca870 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f81100ca400 tx=0x7f810802bd90 comp rx=0 tx=0).stop 2023-10-18T10:40:43.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 -- 172.21.15.125:0/2043832331 shutdown_connections 2023-10-18T10:40:43.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 --2- 172.21.15.125:0/2043832331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 0x7f81100ca870 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 --2- 172.21.15.125:0/2043832331 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8110140660 0x7f81100c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 --2- 172.21.15.125:0/2043832331 >> v2:172.21.15.139:3300/0 conn(0x7f811013d360 0x7f811015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 -- 172.21.15.125:0/2043832331 >> 172.21.15.125:0/2043832331 conn(0x7f8110069930 msgr2=0x7f8110069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:43.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 -- 172.21.15.125:0/2043832331 shutdown_connections 2023-10-18T10:40:43.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.420+0000 7f8117844640 1 -- 172.21.15.125:0/2043832331 wait complete. 2023-10-18T10:40:43.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.421+0000 7f8117844640 1 Processor -- start 2023-10-18T10:40:43.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f8117844640 1 -- start start 2023-10-18T10:40:43.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f8117844640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 0x7f81101372c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:43.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f8117844640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811013d360 0x7f81101f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:43.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f8117844640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8110140660 0x7f811014aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:43.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f8117844640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811014e230 con 0x7f81100c9d00 2023-10-18T10:40:43.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f8117844640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811014db90 con 0x7f811013d360 2023-10-18T10:40:43.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f8117844640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8110137800 con 0x7f8110140660 2023-10-18T10:40:43.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f81155e0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 0x7f81101372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:43.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f81155e0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 0x7f81101372c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32962/0 (socket says 172.21.15.125:32962) 2023-10-18T10:40:43.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.422+0000 7f81155e0640 1 -- 172.21.15.125:0/3149650176 learned_addr learned my addr 172.21.15.125:0/3149650176 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:43.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f8114ddf640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811013d360 0x7f81101f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:43.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f8115de1640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8110140660 0x7f811014aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:43.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f81155e0640 1 -- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8110140660 msgr2=0x7f811014aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:43.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f81155e0640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8110140660 0x7f811014aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f81155e0640 1 -- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811013d360 msgr2=0x7f81101f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:43.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f81155e0640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811013d360 0x7f81101f1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:43.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f81155e0640 1 -- 172.21.15.125:0/3149650176 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8108009560 con 0x7f81100c9d00 2023-10-18T10:40:43.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f8115de1640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8110140660 0x7f811014aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:43.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f81155e0640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 0x7f81101372c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f810c00cce0 tx=0x7f810c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:43.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f8105ffb640 1 -- 172.21.15.125:0/3149650176 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00f650 con 0x7f81100c9d00 2023-10-18T10:40:43.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f8105ffb640 1 -- 172.21.15.125:0/3149650176 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f810c00ce90 con 0x7f81100c9d00 2023-10-18T10:40:43.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811006b4a0 con 0x7f81100c9d00 2023-10-18T10:40:43.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f8105ffb640 1 -- 172.21.15.125:0/3149650176 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00fb40 con 0x7f81100c9d00 2023-10-18T10:40:43.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.423+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811014a1a0 con 0x7f81100c9d00 2023-10-18T10:40:43.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.424+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81100690e0 con 0x7f81100c9d00 2023-10-18T10:40:43.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.424+0000 7f8105ffb640 1 -- 172.21.15.125:0/3149650176 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 40) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f810c00fca0 con 0x7f81100c9d00 2023-10-18T10:40:43.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.425+0000 7f8105ffb640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f80f406aed0 0x7f80f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:43.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.425+0000 7f8105ffb640 1 -- 172.21.15.125:0/3149650176 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f810c08a7b0 con 0x7f81100c9d00 2023-10-18T10:40:43.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.425+0000 7f8114ddf640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f80f406aed0 0x7f80f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:43.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.426+0000 7f8114ddf640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f80f406aed0 0x7f80f406d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f811006f9c0 tx=0x7f810000c420 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:43.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.429+0000 7f8105ffb640 1 -- 172.21.15.125:0/3149650176 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f810c055320 con 0x7f81100c9d00 2023-10-18T10:40:43.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:43.594+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8110157d90 con 0x7f81100c9d00 2023-10-18T10:40:44.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.374+0000 7f8105ffb640 1 -- 172.21.15.125:0/3149650176 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) v1 ==== 181+0+0 (secure 0 0 0) 0x7f810c058940 con 0x7f81100c9d00 2023-10-18T10:40:44.376 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:40:44.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f80f406aed0 msgr2=0x7f80f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:44.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f80f406aed0 0x7f80f406d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f811006f9c0 tx=0x7f810000c420 comp rx=0 tx=0).stop 2023-10-18T10:40:44.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 msgr2=0x7f81101372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:44.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 0x7f81101372c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f810c00cce0 tx=0x7f810c00a620 comp rx=0 tx=0).stop 2023-10-18T10:40:44.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 shutdown_connections 2023-10-18T10:40:44.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f80f406aed0 0x7f80f406d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:44.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811013d360 0x7f81101f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:44.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8110140660 0x7f811014aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:44.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 --2- 172.21.15.125:0/3149650176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81100c9d00 0x7f81101372c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:44.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.376+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 >> 172.21.15.125:0/3149650176 conn(0x7f8110069930 msgr2=0x7f8110157680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:44.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.377+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 shutdown_connections 2023-10-18T10:40:44.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.377+0000 7f8117844640 1 -- 172.21.15.125:0/3149650176 wait complete. 2023-10-18T10:40:44.390 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:40:44.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.498+0000 7fef23ff5640 1 -- 172.21.15.125:0/3698693416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c0cf340 msgr2=0x7fef1c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:44.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.498+0000 7fef23ff5640 1 --2- 172.21.15.125:0/3698693416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c0cf340 0x7fef1c0d1910 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fef18005fd0 tx=0x7fef1802c050 comp rx=0 tx=0).stop 2023-10-18T10:40:44.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.498+0000 7fef23ff5640 1 -- 172.21.15.125:0/3698693416 shutdown_connections 2023-10-18T10:40:44.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.498+0000 7fef23ff5640 1 --2- 172.21.15.125:0/3698693416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef1c15b2a0 0x7fef1c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:44.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.498+0000 7fef23ff5640 1 --2- 172.21.15.125:0/3698693416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c0cf340 0x7fef1c0d1910 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:44.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.498+0000 7fef23ff5640 1 -- 172.21.15.125:0/3698693416 >> 172.21.15.125:0/3698693416 conn(0x7fef1c0d0d60 msgr2=0x7fef1c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:44.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.498+0000 7fef23ff5640 1 -- 172.21.15.125:0/3698693416 shutdown_connections 2023-10-18T10:40:44.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.498+0000 7fef23ff5640 1 -- 172.21.15.125:0/3698693416 wait complete. 2023-10-18T10:40:44.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.499+0000 7fef23ff5640 1 Processor -- start 2023-10-18T10:40:44.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.499+0000 7fef23ff5640 1 -- start start 2023-10-18T10:40:44.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef23ff5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef1c0ceb40 0x7fef1c158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:44.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef23ff5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef1c0cf340 0x7fef1c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:44.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef23ff5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c15b2a0 0x7fef1c159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:44.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef23ff5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0b4e60 con 0x7fef1c15b2a0 2023-10-18T10:40:44.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef23ff5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0b3dc0 con 0x7fef1c0cf340 2023-10-18T10:40:44.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef23ff5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0b40a0 con 0x7fef1c0ceb40 2023-10-18T10:40:44.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c15b2a0 0x7fef1c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:44.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c15b2a0 0x7fef1c159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32978/0 (socket says 172.21.15.125:32978) 2023-10-18T10:40:44.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 -- 172.21.15.125:0/2213007883 learned_addr learned my addr 172.21.15.125:0/2213007883 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:44.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef21d91640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef1c0ceb40 0x7fef1c158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:44.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef21590640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef1c0cf340 0x7fef1c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:44.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 -- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef1c0ceb40 msgr2=0x7fef1c158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:44.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef1c0ceb40 0x7fef1c158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:44.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 -- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef1c0cf340 msgr2=0x7fef1c15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:44.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef1c0cf340 0x7fef1c15d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:44.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 -- 172.21.15.125:0/2213007883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef18009840 con 0x7fef1c15b2a0 2023-10-18T10:40:44.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.500+0000 7fef22592640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c15b2a0 0x7fef1c159510 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c430 tx=0x7fef1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:44.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.501+0000 7fef127fc640 1 -- 172.21.15.125:0/2213007883 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14011790 con 0x7fef1c15b2a0 2023-10-18T10:40:44.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.501+0000 7fef127fc640 1 -- 172.21.15.125:0/2213007883 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef140119e0 con 0x7fef1c15b2a0 2023-10-18T10:40:44.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.501+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c0b4380 con 0x7fef1c15b2a0 2023-10-18T10:40:44.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.501+0000 7fef127fc640 1 -- 172.21.15.125:0/2213007883 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14011e10 con 0x7fef1c15b2a0 2023-10-18T10:40:44.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.501+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c0b4940 con 0x7fef1c15b2a0 2023-10-18T10:40:44.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.501+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c0690e0 con 0x7fef1c15b2a0 2023-10-18T10:40:44.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.502+0000 7fef127fc640 1 -- 172.21.15.125:0/2213007883 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 40) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fef1401c460 con 0x7fef1c15b2a0 2023-10-18T10:40:44.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.502+0000 7fef127fc640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef0006af00 0x7fef0006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:44.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.503+0000 7fef127fc640 1 -- 172.21.15.125:0/2213007883 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6365+0+0 (secure 0 0 0) 0x7fef140898f0 con 0x7fef1c15b2a0 2023-10-18T10:40:44.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.503+0000 7fef21d91640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef0006af00 0x7fef0006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:44.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.503+0000 7fef21d91640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef0006af00 0x7fef0006d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef18035000 tx=0x7fef18004ab0 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:44.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.507+0000 7fef127fc640 1 -- 172.21.15.125:0/2213007883 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef140547b0 con 0x7fef1c15b2a0 2023-10-18T10:40:44.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.672+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef1c0af990 con 0x7fef1c15b2a0 2023-10-18T10:40:44.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:44.796+0000 7fef127fc640 1 -- 172.21.15.125:0/2213007883 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fef14054b80 con 0x7fef1c15b2a0 2023-10-18T10:40:45.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.375+0000 7fef127fc640 1 -- 172.21.15.125:0/2213007883 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7fef14072320 con 0x7fef1c15b2a0 2023-10-18T10:40:45.377 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T10:40:45.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.378+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef0006af00 msgr2=0x7fef0006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef0006af00 0x7fef0006d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef18035000 tx=0x7fef18004ab0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c15b2a0 msgr2=0x7fef1c159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c15b2a0 0x7fef1c159510 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c430 tx=0x7fef1400c8f0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 shutdown_connections 2023-10-18T10:40:45.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef0006af00 0x7fef0006d3c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef1c0cf340 0x7fef1c15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef1c15b2a0 0x7fef1c159510 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 --2- 172.21.15.125:0/2213007883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef1c0ceb40 0x7fef1c158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 >> 172.21.15.125:0/2213007883 conn(0x7fef1c0d0d60 msgr2=0x7fef1c06b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:45.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 shutdown_connections 2023-10-18T10:40:45.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.379+0000 7fef23ff5640 1 -- 172.21.15.125:0/2213007883 wait complete. 2023-10-18T10:40:45.397 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T10:40:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.504+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2626059446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd7c003970 msgr2=0x7fbd7c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.504+0000 7fbd9a2c9640 1 --2- 172.21.15.125:0/2626059446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd7c003970 0x7fbd7c003d90 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c260 tx=0x7fbd7c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.504+0000 7fbd98866640 1 -- 172.21.15.125:0/2626059446 reap_dead start 2023-10-18T10:40:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.504+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2626059446 shutdown_connections 2023-10-18T10:40:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.504+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2626059446 >> 172.21.15.125:0/2626059446 conn(0x7fbd94069930 msgr2=0x7fbd94069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.504+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2626059446 shutdown_connections 2023-10-18T10:40:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.504+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2626059446 wait complete. 2023-10-18T10:40:45.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.505+0000 7fbd9a2c9640 1 Processor -- start 2023-10-18T10:40:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.505+0000 7fbd9a2c9640 1 -- start start 2023-10-18T10:40:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd9a2c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd7c003970 0x7fbd9413ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd9a2c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9406f080 0x7fbd9406f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd9a2c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd940ade90 0x7fbd940ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd9a2c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9406fa40 con 0x7fbd940ade90 2023-10-18T10:40:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd9a2c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941f0de0 con 0x7fbd9406f080 2023-10-18T10:40:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd9a2c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9413b3a0 con 0x7fbd7c003970 2023-10-18T10:40:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd98866640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd940ade90 0x7fbd940ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd98866640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd940ade90 0x7fbd940ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33002/0 (socket says 172.21.15.125:33002) 2023-10-18T10:40:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd98866640 1 -- 172.21.15.125:0/2667362039 learned_addr learned my addr 172.21.15.125:0/2667362039 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd93fff640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd7c003970 0x7fbd9413ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd937fe640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9406f080 0x7fbd9406f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd98866640 1 -- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd7c003970 msgr2=0x7fbd9413ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd98866640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd7c003970 0x7fbd9413ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd98866640 1 -- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9406f080 msgr2=0x7fbd9406f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd98866640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9406f080 0x7fbd9406f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd98866640 1 -- 172.21.15.125:0/2667362039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd941f01d0 con 0x7fbd940ade90 2023-10-18T10:40:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.506+0000 7fbd93fff640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd7c003970 0x7fbd9413ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.507+0000 7fbd98866640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd940ade90 0x7fbd940ae340 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c005fd0 tx=0x7fbd8c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.507+0000 7fbd90ff9640 1 -- 172.21.15.125:0/2667362039 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c004b20 con 0x7fbd940ade90 2023-10-18T10:40:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.507+0000 7fbd90ff9640 1 -- 172.21.15.125:0/2667362039 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbd8c004c80 con 0x7fbd940ade90 2023-10-18T10:40:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.507+0000 7fbd90ff9640 1 -- 172.21.15.125:0/2667362039 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c003bd0 con 0x7fbd940ade90 2023-10-18T10:40:45.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.507+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd941f04b0 con 0x7fbd940ade90 2023-10-18T10:40:45.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.507+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd941f0a70 con 0x7fbd940ade90 2023-10-18T10:40:45.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.508+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd941f0c00 con 0x7fbd940ade90 2023-10-18T10:40:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.508+0000 7fbd90ff9640 1 -- 172.21.15.125:0/2667362039 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbd8c003d30 con 0x7fbd940ade90 2023-10-18T10:40:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.508+0000 7fbd90ff9640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fbd8006af90 0x7fbd8006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.509+0000 7fbd90ff9640 1 -- 172.21.15.125:0/2667362039 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fbd8c089d40 con 0x7fbd940ade90 2023-10-18T10:40:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.509+0000 7fbd93fff640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fbd8006af90 0x7fbd8006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:45.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.509+0000 7fbd93fff640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fbd8006af90 0x7fbd8006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbd84034000 tx=0x7fbd84036040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:45.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.513+0000 7fbd90ff9640 1 -- 172.21.15.125:0/2667362039 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd8c058570 con 0x7fbd940ade90 2023-10-18T10:40:45.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.688+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7fbd9406c3b0 con 0x7fbd940ade90 2023-10-18T10:40:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.692+0000 7fbd90ff9640 1 -- 172.21.15.125:0/2667362039 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v42) v1 ==== 71+0+0 (secure 0 0 0) 0x7fbd8c01e080 con 0x7fbd940ade90 2023-10-18T10:40:45.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fbd8006af90 msgr2=0x7fbd8006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fbd8006af90 0x7fbd8006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbd84034000 tx=0x7fbd84036040 comp rx=0 tx=0).stop 2023-10-18T10:40:45.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd940ade90 msgr2=0x7fbd940ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd940ade90 0x7fbd940ae340 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c005fd0 tx=0x7fbd8c00f3c0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 shutdown_connections 2023-10-18T10:40:45.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fbd8006af90 0x7fbd8006d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd9406f080 0x7fbd9406f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd940ade90 0x7fbd940ae340 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 --2- 172.21.15.125:0/2667362039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd7c003970 0x7fbd9413ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 >> 172.21.15.125:0/2667362039 conn(0x7fbd94069930 msgr2=0x7fbd9413fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 shutdown_connections 2023-10-18T10:40:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.694+0000 7fbd9a2c9640 1 -- 172.21.15.125:0/2667362039 wait complete. 2023-10-18T10:40:45.706 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-18T10:40:45.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.809+0000 7fa059b0c640 1 -- 172.21.15.125:0/3371317228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540b0530 msgr2=0x7fa0540b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.809+0000 7fa059b0c640 1 --2- 172.21.15.125:0/3371317228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540b0530 0x7fa0540b09c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa04c005fd0 tx=0x7fa04c02c2e0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.809+0000 7fa059b0c640 1 -- 172.21.15.125:0/3371317228 shutdown_connections 2023-10-18T10:40:45.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.809+0000 7fa059b0c640 1 --2- 172.21.15.125:0/3371317228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540b0530 0x7fa0540b09c0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.809+0000 7fa059b0c640 1 --2- 172.21.15.125:0/3371317228 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0540cbd20 0x7fa054158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.809+0000 7fa059b0c640 1 -- 172.21.15.125:0/3371317228 >> 172.21.15.125:0/3371317228 conn(0x7fa054069930 msgr2=0x7fa054069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:45.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.809+0000 7fa059b0c640 1 -- 172.21.15.125:0/3371317228 shutdown_connections 2023-10-18T10:40:45.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.809+0000 7fa059b0c640 1 -- 172.21.15.125:0/3371317228 wait complete. 2023-10-18T10:40:45.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.810+0000 7fa059b0c640 1 Processor -- start 2023-10-18T10:40:45.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa059b0c640 1 -- start start 2023-10-18T10:40:45.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa059b0c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540afa20 0x7fa0541487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:45.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa059b0c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0540b0530 0x7fa05414aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:45.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa059b0c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0540cbd20 0x7fa0540ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:45.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa059b0c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05414b430 con 0x7fa0540afa20 2023-10-18T10:40:45.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa059b0c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05414b710 con 0x7fa0540b0530 2023-10-18T10:40:45.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa059b0c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05414b9f0 con 0x7fa0540cbd20 2023-10-18T10:40:45.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa0537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540afa20 0x7fa0541487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:45.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa0537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540afa20 0x7fa0541487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33014/0 (socket says 172.21.15.125:33014) 2023-10-18T10:40:45.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.811+0000 7fa0537fe640 1 -- 172.21.15.125:0/2614222964 learned_addr learned my addr 172.21.15.125:0/2614222964 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:45.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa0537fe640 1 -- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0540cbd20 msgr2=0x7fa0540ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa0537fe640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0540cbd20 0x7fa0540ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa0537fe640 1 -- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0540b0530 msgr2=0x7fa05414aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:45.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa0537fe640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0540b0530 0x7fa05414aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:45.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa0537fe640 1 -- 172.21.15.125:0/2614222964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c009900 con 0x7fa0540afa20 2023-10-18T10:40:45.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa0537fe640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540afa20 0x7fa0541487d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa04400d6d0 tx=0x7fa04400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:45.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa050ff9640 1 -- 172.21.15.125:0/2614222964 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044005220 con 0x7fa0540afa20 2023-10-18T10:40:45.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa050ff9640 1 -- 172.21.15.125:0/2614222964 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa044005490 con 0x7fa0540afa20 2023-10-18T10:40:45.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05414bcd0 con 0x7fa0540afa20 2023-10-18T10:40:45.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa050ff9640 1 -- 172.21.15.125:0/2614222964 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044009a80 con 0x7fa0540afa20 2023-10-18T10:40:45.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.812+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0541545a0 con 0x7fa0540afa20 2023-10-18T10:40:45.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.813+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0541547c0 con 0x7fa0540afa20 2023-10-18T10:40:45.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.814+0000 7fa050ff9640 1 -- 172.21.15.125:0/2614222964 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa0440047a0 con 0x7fa0540afa20 2023-10-18T10:40:45.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.814+0000 7fa050ff9640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa03806af90 0x7fa03806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:45.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.814+0000 7fa050ff9640 1 -- 172.21.15.125:0/2614222964 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fa044089c60 con 0x7fa0540afa20 2023-10-18T10:40:45.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.814+0000 7fa04affd640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa03806af90 0x7fa03806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:45.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.815+0000 7fa04affd640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa03806af90 0x7fa03806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa03c000f60 tx=0x7fa03c0079b0 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:45.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:45.818+0000 7fa050ff9640 1 -- 172.21.15.125:0/2614222964 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa044054a90 con 0x7fa0540afa20 2023-10-18T10:40:46.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.048+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fa0540700d0 con 0x7fa0540afa20 2023-10-18T10:40:46.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.382+0000 7fa050ff9640 1 -- 172.21.15.125:0/2614222964 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v575) v1 ==== 156+0+0 (secure 0 0 0) 0x7fa044017020 con 0x7fa0540afa20 2023-10-18T10:40:46.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa03806af90 msgr2=0x7fa03806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa03806af90 0x7fa03806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa03c000f60 tx=0x7fa03c0079b0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540afa20 msgr2=0x7fa0541487d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540afa20 0x7fa0541487d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa04400d6d0 tx=0x7fa04400db90 comp rx=0 tx=0).stop 2023-10-18T10:40:46.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 shutdown_connections 2023-10-18T10:40:46.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa03806af90 0x7fa03806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0540b0530 0x7fa05414aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0540cbd20 0x7fa0540ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 --2- 172.21.15.125:0/2614222964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0540afa20 0x7fa0541487d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 >> 172.21.15.125:0/2614222964 conn(0x7fa054069930 msgr2=0x7fa0540ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:46.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 shutdown_connections 2023-10-18T10:40:46.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.384+0000 7fa059b0c640 1 -- 172.21.15.125:0/2614222964 wait complete. 2023-10-18T10:40:46.397 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-10-18T10:40:46.398 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-10-18T10:40:46.398 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T10:40:46.399 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:46.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.500+0000 7fad7a541640 1 -- 172.21.15.125:0/3290109914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5c003970 msgr2=0x7fad5c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.500+0000 7fad7a541640 1 --2- 172.21.15.125:0/3290109914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5c003970 0x7fad5c003d70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fad64006fc0 tx=0x7fad5c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.500+0000 7fad7a541640 1 -- 172.21.15.125:0/3290109914 shutdown_connections 2023-10-18T10:40:46.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.500+0000 7fad7a541640 1 --2- 172.21.15.125:0/3290109914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5c003970 0x7fad5c003d70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.500+0000 7fad7a541640 1 -- 172.21.15.125:0/3290109914 >> 172.21.15.125:0/3290109914 conn(0x7fad74069930 msgr2=0x7fad74069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:46.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.500+0000 7fad7a541640 1 -- 172.21.15.125:0/3290109914 shutdown_connections 2023-10-18T10:40:46.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.500+0000 7fad7a541640 1 -- 172.21.15.125:0/3290109914 wait complete. 2023-10-18T10:40:46.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.502+0000 7fad7a541640 1 Processor -- start 2023-10-18T10:40:46.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.502+0000 7fad7a541640 1 -- start start 2023-10-18T10:40:46.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.502+0000 7fad7a541640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad5c003970 0x7fad7412dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:46.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.502+0000 7fad7a541640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad74142bc0 0x7fad74158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:46.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.502+0000 7fad7a541640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad74158ec0 0x7fad74159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:46.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.502+0000 7fad7a541640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7412e2d0 con 0x7fad74158ec0 2023-10-18T10:40:46.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.502+0000 7fad7a541640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7412e5b0 con 0x7fad5c003970 2023-10-18T10:40:46.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.502+0000 7fad7a541640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7412e890 con 0x7fad74142bc0 2023-10-18T10:40:46.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad74158ec0 0x7fad74159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:46.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad74158ec0 0x7fad74159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33036/0 (socket says 172.21.15.125:33036) 2023-10-18T10:40:46.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 -- 172.21.15.125:0/2820471786 learned_addr learned my addr 172.21.15.125:0/2820471786 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:46.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad737fe640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad74142bc0 0x7fad74158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:46.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad73fff640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad5c003970 0x7fad7412dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:46.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 -- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad74142bc0 msgr2=0x7fad74158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad74142bc0 0x7fad74158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 -- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad5c003970 msgr2=0x7fad7412dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad5c003970 0x7fad7412dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 -- 172.21.15.125:0/2820471786 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7412eb70 con 0x7fad74158ec0 2023-10-18T10:40:46.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad78ade640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad74158ec0 0x7fad74159340 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fad740ca740 tx=0x7fad6c009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:46.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad70ff9640 1 -- 172.21.15.125:0/2820471786 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004d20 con 0x7fad74158ec0 2023-10-18T10:40:46.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad70ff9640 1 -- 172.21.15.125:0/2820471786 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad6c004e80 con 0x7fad74158ec0 2023-10-18T10:40:46.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7412ee50 con 0x7fad74158ec0 2023-10-18T10:40:46.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad70ff9640 1 -- 172.21.15.125:0/2820471786 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c012b70 con 0x7fad74158ec0 2023-10-18T10:40:46.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.503+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad74164220 con 0x7fad74158ec0 2023-10-18T10:40:46.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.504+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad740690e0 con 0x7fad74158ec0 2023-10-18T10:40:46.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.505+0000 7fad70ff9640 1 -- 172.21.15.125:0/2820471786 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fad6c012cd0 con 0x7fad74158ec0 2023-10-18T10:40:46.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.505+0000 7fad70ff9640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fad6006af90 0x7fad6006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:46.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.505+0000 7fad70ff9640 1 -- 172.21.15.125:0/2820471786 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fad6c08bf80 con 0x7fad74158ec0 2023-10-18T10:40:46.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.505+0000 7fad73fff640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fad6006af90 0x7fad6006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:46.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.506+0000 7fad73fff640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fad6006af90 0x7fad6006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad64034000 tx=0x7fad64039040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:46.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.509+0000 7fad70ff9640 1 -- 172.21.15.125:0/2820471786 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad6c056f70 con 0x7fad74158ec0 2023-10-18T10:40:46.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.676+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad740ae490 con 0x7fad74158ec0 2023-10-18T10:40:46.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.677+0000 7fad70ff9640 1 -- 172.21.15.125:0/2820471786 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+14258 (secure 0 0 0) 0x7fad6c05a590 con 0x7fad74158ec0 2023-10-18T10:40:46.679 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:46.680 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":89,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:45.374921+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6835/1136111019":"2023-10-19T10:40:42.248608+0000","172.21.15.139:0/3479864824":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/3728504787":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/627668147":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/3234267299":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6832/3064166201":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/938869555":"2023-10-19T10:40:40.145536+0000","172.21.15.139:0/1217443296":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2948747997":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/3110681241":"2023-10-19T10:40:34.763606+0000","172.21.15.125:6833/1136111019":"2023-10-19T10:40:42.248608+0000","172.21.15.139:0/1795019078":"2023-10-19T10:40:34.763606+0000","172.21.15.139:6833/3064166201":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6833/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/578014551":"2023-10-19T10:40:40.145536+0000","172.21.15.139:6832/2642591764":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/686709880":"2023-10-19T10:40:34.763606+0000","172.21.15.139:0/2487642114":"2023-10-19T10:40:34.763606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:46.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fad6006af90 msgr2=0x7fad6006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fad6006af90 0x7fad6006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad64034000 tx=0x7fad64039040 comp rx=0 tx=0).stop 2023-10-18T10:40:46.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad74158ec0 msgr2=0x7fad74159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad74158ec0 0x7fad74159340 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fad740ca740 tx=0x7fad6c009a40 comp rx=0 tx=0).stop 2023-10-18T10:40:46.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 shutdown_connections 2023-10-18T10:40:46.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fad6006af90 0x7fad6006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad74158ec0 0x7fad74159340 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad74142bc0 0x7fad74158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 --2- 172.21.15.125:0/2820471786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad5c003970 0x7fad7412dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 >> 172.21.15.125:0/2820471786 conn(0x7fad74069930 msgr2=0x7fad74070510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:46.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 shutdown_connections 2023-10-18T10:40:46.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.679+0000 7fad7a541640 1 -- 172.21.15.125:0/2820471786 wait complete. 2023-10-18T10:40:46.692 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:40:46.692 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:40:46.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.795+0000 7f09ce47c640 1 -- 172.21.15.125:0/2333946668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 msgr2=0x7f09c80b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.795+0000 7f09ce47c640 1 --2- 172.21.15.125:0/2333946668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 0x7f09c80b09c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f09c0005fd0 tx=0x7f09c002c240 comp rx=0 tx=0).stop 2023-10-18T10:40:46.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.795+0000 7f09ce47c640 1 -- 172.21.15.125:0/2333946668 shutdown_connections 2023-10-18T10:40:46.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.795+0000 7f09ce47c640 1 --2- 172.21.15.125:0/2333946668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 0x7f09c80b09c0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.795+0000 7f09ce47c640 1 --2- 172.21.15.125:0/2333946668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c80afa20 0x7f09c8158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.795+0000 7f09ce47c640 1 -- 172.21.15.125:0/2333946668 >> 172.21.15.125:0/2333946668 conn(0x7f09c8069930 msgr2=0x7f09c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:46.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.795+0000 7f09ce47c640 1 -- 172.21.15.125:0/2333946668 shutdown_connections 2023-10-18T10:40:46.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.795+0000 7f09ce47c640 1 -- 172.21.15.125:0/2333946668 wait complete. 2023-10-18T10:40:46.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.797+0000 7f09ce47c640 1 Processor -- start 2023-10-18T10:40:46.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.797+0000 7f09ce47c640 1 -- start start 2023-10-18T10:40:46.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09ce47c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c80afa20 0x7f09c8149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:46.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09ce47c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 0x7f09c81500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:46.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09ce47c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c80cbd20 0x7f09c814cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:46.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09ce47c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c814bc80 con 0x7f09c80b0530 2023-10-18T10:40:46.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09ce47c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c814bf60 con 0x7f09c80cbd20 2023-10-18T10:40:46.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09ce47c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c814c240 con 0x7f09c80afa20 2023-10-18T10:40:46.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 0x7f09c81500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:46.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 0x7f09c81500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33054/0 (socket says 172.21.15.125:33054) 2023-10-18T10:40:46.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 -- 172.21.15.125:0/1082570196 learned_addr learned my addr 172.21.15.125:0/1082570196 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:46.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c7fff640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c80afa20 0x7f09c8149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:46.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09cca19640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c80cbd20 0x7f09c814cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:46.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 -- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c80afa20 msgr2=0x7f09c8149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c80afa20 0x7f09c8149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 -- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c80cbd20 msgr2=0x7f09c814cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c80cbd20 0x7f09c814cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 -- 172.21.15.125:0/1082570196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0009900 con 0x7f09c80b0530 2023-10-18T10:40:46.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c77fe640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 0x7f09c81500a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00b4a0 tx=0x7f09bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:46.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c4ff9640 1 -- 172.21.15.125:0/1082570196 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc010610 con 0x7f09c80b0530 2023-10-18T10:40:46.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c4ff9640 1 -- 172.21.15.125:0/1082570196 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f09bc010860 con 0x7f09c80b0530 2023-10-18T10:40:46.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c814c520 con 0x7f09c80b0530 2023-10-18T10:40:46.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09c4ff9640 1 -- 172.21.15.125:0/1082570196 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc010c70 con 0x7f09c80b0530 2023-10-18T10:40:46.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.798+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c8149fc0 con 0x7f09c80b0530 2023-10-18T10:40:46.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.799+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c814ca40 con 0x7f09c80b0530 2023-10-18T10:40:46.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.800+0000 7f09c4ff9640 1 -- 172.21.15.125:0/1082570196 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f09bc017360 con 0x7f09c80b0530 2023-10-18T10:40:46.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.800+0000 7f09c4ff9640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f09b406af60 0x7f09b406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:46.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.800+0000 7f09c4ff9640 1 -- 172.21.15.125:0/1082570196 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f09bc0896c0 con 0x7f09c80b0530 2023-10-18T10:40:46.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.800+0000 7f09c7fff640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f09b406af60 0x7f09b406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:46.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.801+0000 7f09c7fff640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f09b406af60 0x7f09b406d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f09b8006fd0 tx=0x7f09b8009060 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:46.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.804+0000 7f09c4ff9640 1 -- 172.21.15.125:0/1082570196 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09c814ca40 con 0x7f09c80b0530 2023-10-18T10:40:46.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.971+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09c814ca40 con 0x7f09c80b0530 2023-10-18T10:40:46.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.971+0000 7f09c4ff9640 1 -- 172.21.15.125:0/1082570196 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09c814ca40 con 0x7f09c80b0530 2023-10-18T10:40:46.973 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:40:46.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f09b406af60 msgr2=0x7f09b406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f09b406af60 0x7f09b406d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f09b8006fd0 tx=0x7f09b8009060 comp rx=0 tx=0).stop 2023-10-18T10:40:46.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 msgr2=0x7f09c81500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:46.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 0x7f09c81500a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00b4a0 tx=0x7f09bc00b960 comp rx=0 tx=0).stop 2023-10-18T10:40:46.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 shutdown_connections 2023-10-18T10:40:46.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f09b406af60 0x7f09b406d420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c80cbd20 0x7f09c814cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c80b0530 0x7f09c81500a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 --2- 172.21.15.125:0/1082570196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c80afa20 0x7f09c8149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:46.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 >> 172.21.15.125:0/1082570196 conn(0x7f09c8069930 msgr2=0x7f09c80acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:46.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 shutdown_connections 2023-10-18T10:40:46.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:46.973+0000 7f09ce47c640 1 -- 172.21.15.125:0/1082570196 wait complete. 2023-10-18T10:40:46.985 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-18T10:40:47.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.086+0000 7f1b7f116640 1 -- 172.21.15.125:0/1276356001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b68003fd0 msgr2=0x7f1b680047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.086+0000 7f1b7f116640 1 --2- 172.21.15.125:0/1276356001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b68003fd0 0x7f1b680047c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1b64001680 tx=0x7f1b680057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.086+0000 7f1b7d6b3640 1 -- 172.21.15.125:0/1276356001 reap_dead start 2023-10-18T10:40:47.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.086+0000 7f1b7f116640 1 -- 172.21.15.125:0/1276356001 shutdown_connections 2023-10-18T10:40:47.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.086+0000 7f1b7f116640 1 -- 172.21.15.125:0/1276356001 >> 172.21.15.125:0/1276356001 conn(0x7f1b78069930 msgr2=0x7f1b78069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:47.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.086+0000 7f1b7f116640 1 -- 172.21.15.125:0/1276356001 shutdown_connections 2023-10-18T10:40:47.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.086+0000 7f1b7f116640 1 -- 172.21.15.125:0/1276356001 wait complete. 2023-10-18T10:40:47.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.087+0000 7f1b7f116640 1 Processor -- start 2023-10-18T10:40:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.087+0000 7f1b7f116640 1 -- start start 2023-10-18T10:40:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7f116640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b68003fd0 0x7f1b781ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7f116640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b781f0cf0 0x7f1b780c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7f116640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b7815a120 0x7f1b7815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7f116640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7815aae0 con 0x7f1b7815a120 2023-10-18T10:40:47.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7f116640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780a65c0 con 0x7f1b781f0cf0 2023-10-18T10:40:47.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7f116640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781f1170 con 0x7f1b68003fd0 2023-10-18T10:40:47.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7d6b3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b7815a120 0x7f1b7815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7d6b3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b7815a120 0x7f1b7815a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33084/0 (socket says 172.21.15.125:33084) 2023-10-18T10:40:47.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7d6b3640 1 -- 172.21.15.125:0/1225614558 learned_addr learned my addr 172.21.15.125:0/1225614558 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:47.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7ceb2640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b68003fd0 0x7f1b781ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b77fff640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b781f0cf0 0x7f1b780c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7d6b3640 1 -- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b68003fd0 msgr2=0x7f1b781ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7d6b3640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b68003fd0 0x7f1b781ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7d6b3640 1 -- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b781f0cf0 msgr2=0x7f1b780c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7d6b3640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b781f0cf0 0x7f1b780c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.088+0000 7f1b7d6b3640 1 -- 172.21.15.125:0/1225614558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7813ab70 con 0x7f1b7815a120 2023-10-18T10:40:47.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.089+0000 7f1b7d6b3640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b7815a120 0x7f1b7815a5a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005fd0 tx=0x7f1b70011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:47.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.089+0000 7f1b757fa640 1 -- 172.21.15.125:0/1225614558 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b70003e40 con 0x7f1b7815a120 2023-10-18T10:40:47.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.089+0000 7f1b757fa640 1 -- 172.21.15.125:0/1225614558 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b70004b20 con 0x7f1b7815a120 2023-10-18T10:40:47.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.089+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7813b580 con 0x7f1b7815a120 2023-10-18T10:40:47.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.089+0000 7f1b757fa640 1 -- 172.21.15.125:0/1225614558 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7000f920 con 0x7f1b7815a120 2023-10-18T10:40:47.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.089+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7815b400 con 0x7f1b7815a120 2023-10-18T10:40:47.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.090+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b780690e0 con 0x7f1b7815a120 2023-10-18T10:40:47.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.090+0000 7f1b757fa640 1 -- 172.21.15.125:0/1225614558 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1b70018030 con 0x7f1b7815a120 2023-10-18T10:40:47.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.091+0000 7f1b757fa640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f1b5c06af60 0x7f1b5c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.091+0000 7f1b757fa640 1 -- 172.21.15.125:0/1225614558 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f1b7008a230 con 0x7f1b7815a120 2023-10-18T10:40:47.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.091+0000 7f1b7ceb2640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f1b5c06af60 0x7f1b5c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.091+0000 7f1b7ceb2640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f1b5c06af60 0x7f1b5c06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0017a0 tx=0x7f1b6c006040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:47.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.095+0000 7f1b757fa640 1 -- 172.21.15.125:0/1225614558 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b70055100 con 0x7f1b7815a120 2023-10-18T10:40:47.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.324+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f1b7813ab50 con 0x7f1b7815a120 2023-10-18T10:40:47.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.383+0000 7f1b757fa640 1 -- 172.21.15.125:0/1225614558 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v576) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1b700200f0 con 0x7f1b7815a120 2023-10-18T10:40:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f1b5c06af60 msgr2=0x7f1b5c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f1b5c06af60 0x7f1b5c06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0017a0 tx=0x7f1b6c006040 comp rx=0 tx=0).stop 2023-10-18T10:40:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b7815a120 msgr2=0x7f1b7815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b7815a120 0x7f1b7815a5a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005fd0 tx=0x7f1b70011e60 comp rx=0 tx=0).stop 2023-10-18T10:40:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 shutdown_connections 2023-10-18T10:40:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f1b5c06af60 0x7f1b5c06d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b781f0cf0 0x7f1b780c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b7815a120 0x7f1b7815a5a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 --2- 172.21.15.125:0/1225614558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b68003fd0 0x7f1b781ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 >> 172.21.15.125:0/1225614558 conn(0x7f1b78069930 msgr2=0x7f1b7815e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:47.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 shutdown_connections 2023-10-18T10:40:47.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.386+0000 7f1b7f116640 1 -- 172.21.15.125:0/1225614558 wait complete. 2023-10-18T10:40:47.398 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T10:40:47.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.498+0000 7f685a04b640 1 -- 172.21.15.125:0/84743290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540b0c40 msgr2=0x7f6854158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.498+0000 7f685a04b640 1 --2- 172.21.15.125:0/84743290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540b0c40 0x7f6854158300 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f684c005fd0 tx=0x7f684c02c2e0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.498+0000 7f685a04b640 1 -- 172.21.15.125:0/84743290 shutdown_connections 2023-10-18T10:40:47.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.498+0000 7f685a04b640 1 --2- 172.21.15.125:0/84743290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540b0c40 0x7f6854158300 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.498+0000 7f685a04b640 1 -- 172.21.15.125:0/84743290 >> 172.21.15.125:0/84743290 conn(0x7f6854069930 msgr2=0x7f6854069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:47.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.498+0000 7f685a04b640 1 -- 172.21.15.125:0/84743290 shutdown_connections 2023-10-18T10:40:47.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.498+0000 7f685a04b640 1 -- 172.21.15.125:0/84743290 wait complete. 2023-10-18T10:40:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.499+0000 7f685a04b640 1 Processor -- start 2023-10-18T10:40:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.499+0000 7f685a04b640 1 -- start start 2023-10-18T10:40:47.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f685a04b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540afa20 0x7f685406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f685a04b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68540b0c40 0x7f6854144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f685a04b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68540cbd20 0x7f6854144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f685a04b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68541436c0 con 0x7f68540afa20 2023-10-18T10:40:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f685a04b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6854142af0 con 0x7f68540cbd20 2023-10-18T10:40:47.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f685a04b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6854142d20 con 0x7f68540b0c40 2023-10-18T10:40:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540afa20 0x7f685406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540afa20 0x7f685406b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56166/0 (socket says 172.21.15.125:56166) 2023-10-18T10:40:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 -- 172.21.15.125:0/2933124712 learned_addr learned my addr 172.21.15.125:0/2933124712 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f6852ffd640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68540b0c40 0x7f6854144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f6853fff640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68540cbd20 0x7f6854144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 -- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68540b0c40 msgr2=0x7f6854144410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68540b0c40 0x7f6854144410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 -- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68540cbd20 msgr2=0x7f6854144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68540cbd20 0x7f6854144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 -- 172.21.15.125:0/2933124712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c009900 con 0x7f68540afa20 2023-10-18T10:40:47.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f68537fe640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540afa20 0x7f685406b7c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f684400b3f0 tx=0x7f684400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:47.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f683bfff640 1 -- 172.21.15.125:0/2933124712 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844011830 con 0x7f68540afa20 2023-10-18T10:40:47.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f683bfff640 1 -- 172.21.15.125:0/2933124712 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6844011a80 con 0x7f68540afa20 2023-10-18T10:40:47.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.500+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6854143000 con 0x7f68540afa20 2023-10-18T10:40:47.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.501+0000 7f683bfff640 1 -- 172.21.15.125:0/2933124712 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844011e90 con 0x7f68540afa20 2023-10-18T10:40:47.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.501+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685406bd00 con 0x7f68540afa20 2023-10-18T10:40:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.501+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68540690e0 con 0x7f68540afa20 2023-10-18T10:40:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.502+0000 7f683bfff640 1 -- 172.21.15.125:0/2933124712 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f684401a400 con 0x7f68540afa20 2023-10-18T10:40:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.502+0000 7f683bfff640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f684006af60 0x7f684006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.502+0000 7f683bfff640 1 -- 172.21.15.125:0/2933124712 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f6844089760 con 0x7f68540afa20 2023-10-18T10:40:47.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.503+0000 7f6852ffd640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f684006af60 0x7f684006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.503+0000 7f6852ffd640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f684006af60 0x7f684006d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f685406c0c0 tx=0x7f6848009040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:47.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.506+0000 7f683bfff640 1 -- 172.21.15.125:0/2933124712 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6844054750 con 0x7f68540afa20 2023-10-18T10:40:47.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.673+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f685406f700 con 0x7f68540afa20 2023-10-18T10:40:47.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.673+0000 7f683bfff640 1 -- 172.21.15.125:0/2933124712 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v42) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f68440158c0 con 0x7f68540afa20 2023-10-18T10:40:47.676 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T10:40:47.676 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T10:40:47.676 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T10:40:47.676 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T10:40:47.676 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T10:40:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T10:40:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T10:40:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T10:40:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T10:40:47.677 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T10:40:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T10:40:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T10:40:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T10:40:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T10:40:47.678 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T10:40:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T10:40:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T10:40:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T10:40:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T10:40:47.679 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T10:40:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T10:40:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T10:40:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T10:40:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T10:40:47.680 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T10:40:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T10:40:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T10:40:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T10:40:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T10:40:47.681 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T10:40:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f684006af60 msgr2=0x7f684006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f684006af60 0x7f684006d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f685406c0c0 tx=0x7f6848009040 comp rx=0 tx=0).stop 2023-10-18T10:40:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540afa20 msgr2=0x7f685406b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540afa20 0x7f685406b7c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f684400b3f0 tx=0x7f684400b8b0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 shutdown_connections 2023-10-18T10:40:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f684006af60 0x7f684006d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68540cbd20 0x7f6854144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68540b0c40 0x7f6854144410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 --2- 172.21.15.125:0/2933124712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68540afa20 0x7f685406b7c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.675+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 >> 172.21.15.125:0/2933124712 conn(0x7f6854069930 msgr2=0x7f6854069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.676+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 shutdown_connections 2023-10-18T10:40:47.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.676+0000 7f685a04b640 1 -- 172.21.15.125:0/2933124712 wait complete. 2023-10-18T10:40:47.688 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:40:47.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.790+0000 7fee3f44f640 1 -- 172.21.15.125:0/3188041135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee380ab500 msgr2=0x7fee380ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.790+0000 7fee3f44f640 1 --2- 172.21.15.125:0/3188041135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee380ab500 0x7fee380ab990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fee380ca260 tx=0x7fee3402bdb0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.790+0000 7fee3f44f640 1 -- 172.21.15.125:0/3188041135 shutdown_connections 2023-10-18T10:40:47.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.790+0000 7fee3f44f640 1 --2- 172.21.15.125:0/3188041135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee380ab500 0x7fee380ab990 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.790+0000 7fee3f44f640 1 --2- 172.21.15.125:0/3188041135 >> v2:172.21.15.139:3300/0 conn(0x7fee380c9dc0 0x7fee3806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.790+0000 7fee3f44f640 1 -- 172.21.15.125:0/3188041135 >> 172.21.15.125:0/3188041135 conn(0x7fee38069930 msgr2=0x7fee38069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:47.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.791+0000 7fee3f44f640 1 -- 172.21.15.125:0/3188041135 shutdown_connections 2023-10-18T10:40:47.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.791+0000 7fee3f44f640 1 -- 172.21.15.125:0/3188041135 wait complete. 2023-10-18T10:40:47.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3f44f640 1 Processor -- start 2023-10-18T10:40:47.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3f44f640 1 -- start start 2023-10-18T10:40:47.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3f44f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3806af00 0x7fee3806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3f44f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee380ab500 0x7fee3814ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3f44f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee380c9dc0 0x7fee380aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3f44f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380b1fb0 con 0x7fee3806af00 2023-10-18T10:40:47.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3f44f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3806ffd0 con 0x7fee380ab500 2023-10-18T10:40:47.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3f44f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380aeb10 con 0x7fee380c9dc0 2023-10-18T10:40:47.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.792+0000 7fee3d1eb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3806af00 0x7fee3806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3d1eb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3806af00 0x7fee3806fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56180/0 (socket says 172.21.15.125:56180) 2023-10-18T10:40:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3d1eb640 1 -- 172.21.15.125:0/1118701038 learned_addr learned my addr 172.21.15.125:0/1118701038 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3d1eb640 1 -- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee380c9dc0 msgr2=0x7fee380aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3d1eb640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee380c9dc0 0x7fee380aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3d1eb640 1 -- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee380ab500 msgr2=0x7fee3814ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3d1eb640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee380ab500 0x7fee3814ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3d1eb640 1 -- 172.21.15.125:0/1118701038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34009560 con 0x7fee3806af00 2023-10-18T10:40:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3d1eb640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3806af00 0x7fee3806fa90 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c7c0 tx=0x7fee3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee2dffb640 1 -- 172.21.15.125:0/1118701038 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3000aa50 con 0x7fee3806af00 2023-10-18T10:40:47.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee2dffb640 1 -- 172.21.15.125:0/1118701038 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee3000aca0 con 0x7fee3806af00 2023-10-18T10:40:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee380701e0 con 0x7fee3806af00 2023-10-18T10:40:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee2dffb640 1 -- 172.21.15.125:0/1118701038 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30012680 con 0x7fee3806af00 2023-10-18T10:40:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.793+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee380ae7a0 con 0x7fee3806af00 2023-10-18T10:40:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.794+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee380cb6d0 con 0x7fee3806af00 2023-10-18T10:40:47.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.795+0000 7fee2dffb640 1 -- 172.21.15.125:0/1118701038 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fee300127e0 con 0x7fee3806af00 2023-10-18T10:40:47.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.795+0000 7fee2dffb640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fee1c06af60 0x7fee1c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:47.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.795+0000 7fee2dffb640 1 -- 172.21.15.125:0/1118701038 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fee300899f0 con 0x7fee3806af00 2023-10-18T10:40:47.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.795+0000 7fee3c9ea640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fee1c06af60 0x7fee1c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:47.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.796+0000 7fee3c9ea640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fee1c06af60 0x7fee1c06d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fee38071260 tx=0x7fee28005b50 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:47.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.799+0000 7fee2dffb640 1 -- 172.21.15.125:0/1118701038 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee30054930 con 0x7fee3806af00 2023-10-18T10:40:47.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.965+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fee380675f0 con 0x7fee3806af00 2023-10-18T10:40:47.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.968+0000 7fee2dffb640 1 -- 172.21.15.125:0/1118701038 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7fee30057f50 con 0x7fee3806af00 2023-10-18T10:40:47.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fee1c06af60 msgr2=0x7fee1c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fee1c06af60 0x7fee1c06d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fee38071260 tx=0x7fee28005b50 comp rx=0 tx=0).stop 2023-10-18T10:40:47.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3806af00 msgr2=0x7fee3806fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3806af00 0x7fee3806fa90 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c7c0 tx=0x7fee3000cc80 comp rx=0 tx=0).stop 2023-10-18T10:40:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 shutdown_connections 2023-10-18T10:40:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fee1c06af60 0x7fee1c06d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee380ab500 0x7fee3814ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee380c9dc0 0x7fee380aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 --2- 172.21.15.125:0/1118701038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee3806af00 0x7fee3806fa90 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 >> 172.21.15.125:0/1118701038 conn(0x7fee38069930 msgr2=0x7fee380cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 shutdown_connections 2023-10-18T10:40:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:47.970+0000 7fee3f44f640 1 -- 172.21.15.125:0/1118701038 wait complete. 2023-10-18T10:40:47.982 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:40:47.983 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:40:48.027 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:40:48.070 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:40:48.070 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:40:48.071 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:48.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:40:48.086 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:40:48.087 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:48.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:40:48.127 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:40:48.128 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:48.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:40:48.143 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:40:48.143 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:40:48.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:40:48.185 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:40:48.185 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:48.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.286+0000 7f3a5f661640 1 -- 172.21.15.125:0/2145504839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 msgr2=0x7f3a580b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:48.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.286+0000 7f3a5f661640 1 --2- 172.21.15.125:0/2145504839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 0x7f3a580b34c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3a580cfa20 tx=0x7f3a5402c110 comp rx=0 tx=0).stop 2023-10-18T10:40:48.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.286+0000 7f3a5f661640 1 -- 172.21.15.125:0/2145504839 shutdown_connections 2023-10-18T10:40:48.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.286+0000 7f3a5f661640 1 --2- 172.21.15.125:0/2145504839 >> v2:172.21.15.139:3300/0 conn(0x7f3a5806d2b0 0x7f3a5815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.286+0000 7f3a5f661640 1 --2- 172.21.15.125:0/2145504839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 0x7f3a580b34c0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.286+0000 7f3a5f661640 1 -- 172.21.15.125:0/2145504839 >> 172.21.15.125:0/2145504839 conn(0x7f3a580b3f00 msgr2=0x7f3a580b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:48.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.286+0000 7f3a5f661640 1 -- 172.21.15.125:0/2145504839 shutdown_connections 2023-10-18T10:40:48.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.286+0000 7f3a5f661640 1 -- 172.21.15.125:0/2145504839 wait complete. 2023-10-18T10:40:48.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.287+0000 7f3a5f661640 1 Processor -- start 2023-10-18T10:40:48.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.287+0000 7f3a5f661640 1 -- start start 2023-10-18T10:40:48.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5f661640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5806d2b0 0x7f3a58150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:48.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5f661640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 0x7f3a5814cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5f661640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5815c0d0 0x7f3a580b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5f661640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5814dfe0 con 0x7f3a5815b7b0 2023-10-18T10:40:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5f661640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a581512e0 con 0x7f3a5806d2b0 2023-10-18T10:40:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5f661640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a581515c0 con 0x7f3a5815c0d0 2023-10-18T10:40:48.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5cbfc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 0x7f3a5814cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5cbfc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 0x7f3a5814cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56194/0 (socket says 172.21.15.125:56194) 2023-10-18T10:40:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5cbfc640 1 -- 172.21.15.125:0/3437688668 learned_addr learned my addr 172.21.15.125:0/3437688668 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5cbfc640 1 -- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5815c0d0 msgr2=0x7f3a580b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:48.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5cbfc640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5815c0d0 0x7f3a580b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5cbfc640 1 -- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5806d2b0 msgr2=0x7f3a58150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5cbfc640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5806d2b0 0x7f3a58150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.288+0000 7f3a5cbfc640 1 -- 172.21.15.125:0/3437688668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54009840 con 0x7f3a5815b7b0 2023-10-18T10:40:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.289+0000 7f3a5cbfc640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 0x7f3a5814cec0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c890 tx=0x7f3a4800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:48.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.289+0000 7f3a4dffb640 1 -- 172.21.15.125:0/3437688668 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48004220 con 0x7f3a5815b7b0 2023-10-18T10:40:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.289+0000 7f3a4dffb640 1 -- 172.21.15.125:0/3437688668 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a48004470 con 0x7f3a5815b7b0 2023-10-18T10:40:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.289+0000 7f3a4dffb640 1 -- 172.21.15.125:0/3437688668 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48007c10 con 0x7f3a5815b7b0 2023-10-18T10:40:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.289+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a581518a0 con 0x7f3a5815b7b0 2023-10-18T10:40:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.289+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a58151e60 con 0x7f3a5815b7b0 2023-10-18T10:40:48.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.290+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a58152080 con 0x7f3a5815b7b0 2023-10-18T10:40:48.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.290+0000 7f3a4dffb640 1 -- 172.21.15.125:0/3437688668 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3a48005020 con 0x7f3a5815b7b0 2023-10-18T10:40:48.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.290+0000 7f3a4dffb640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3a3c06af60 0x7f3a3c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:48.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.290+0000 7f3a4dffb640 1 -- 172.21.15.125:0/3437688668 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f3a48089f00 con 0x7f3a5815b7b0 2023-10-18T10:40:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.291+0000 7f3a5d3fd640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3a3c06af60 0x7f3a3c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.291+0000 7f3a5d3fd640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3a3c06af60 0x7f3a3c06d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a54035000 tx=0x7f3a54004a40 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.295+0000 7f3a4dffb640 1 -- 172.21.15.125:0/3437688668 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a48054ca0 con 0x7f3a5815b7b0 2023-10-18T10:40:48.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.497+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a5806b7d0 con 0x7f3a5815b7b0 2023-10-18T10:40:48.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.497+0000 7f3a4dffb640 1 -- 172.21.15.125:0/3437688668 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f3a4801d070 con 0x7f3a5815b7b0 2023-10-18T10:40:48.500 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:48.500 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/693466412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":693466412},{"type":"v1","addr":"172.21.15.139:6836","nonce":693466412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6277,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/170638031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":170638031},{"type":"v1","addr":"172.21.15.139:6837","nonce":170638031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":7068,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3024029084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3024029084},{"type":"v1","addr":"172.21.15.125:6834","nonce":3024029084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":7491,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2211361664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2211361664},{"type":"v1","addr":"172.21.15.125:6835","nonce":2211361664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[]} 2023-10-18T10:40:48.500 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 31 2023-10-18T10:40:48.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3a3c06af60 msgr2=0x7f3a3c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3a3c06af60 0x7f3a3c06d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a54035000 tx=0x7f3a54004a40 comp rx=0 tx=0).stop 2023-10-18T10:40:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 msgr2=0x7f3a5814cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 0x7f3a5814cec0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c890 tx=0x7f3a4800cd50 comp rx=0 tx=0).stop 2023-10-18T10:40:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 shutdown_connections 2023-10-18T10:40:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3a3c06af60 0x7f3a3c06d420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5815c0d0 0x7f3a580b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5815b7b0 0x7f3a5814cec0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 --2- 172.21.15.125:0/3437688668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5806d2b0 0x7f3a58150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 >> 172.21.15.125:0/3437688668 conn(0x7f3a580b3f00 msgr2=0x7f3a581590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:48.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 shutdown_connections 2023-10-18T10:40:48.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.499+0000 7f3a5f661640 1 -- 172.21.15.125:0/3437688668 wait complete. 2023-10-18T10:40:48.512 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:40:48.512 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:40:48.513 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:40:48.513 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:40:48.513 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:40:48.514 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:40:48.514 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:40:48.514 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:40:48.514 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:40:48.515 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:40:48.515 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:40:48.515 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:40:48.516 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:40:48.516 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:40:48.516 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:40:48.516 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:40:48.522 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:40:48.522 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:40:48.523 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:40:48.523 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:40:48.527 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:40:48.527 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:40:48.528 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:40:48.528 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:40:48.564 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:40:48.564 DEBUG:teuthology.parallel:result is None 2023-10-18T10:40:48.565 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:40:48.565 DEBUG:teuthology.parallel:result is None 2023-10-18T10:40:48.570 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:40:48.570 DEBUG:teuthology.parallel:result is None 2023-10-18T10:40:48.571 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:40:48.571 DEBUG:teuthology.parallel:result is None 2023-10-18T10:40:48.572 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:40:48.638 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:40:48.639 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:40:48.653 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:40:48.654 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:40:48.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.677+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/92919802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c3806d2d0 msgr2=0x7f8c3806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:48.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.677+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/92919802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c3806d2d0 0x7f8c3806d6b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009830 tx=0x7f8c2802bd60 comp rx=0 tx=0).stop 2023-10-18T10:40:48.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.677+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/92919802 shutdown_connections 2023-10-18T10:40:48.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.677+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/92919802 >> v2:172.21.15.139:3300/0 conn(0x7f8c3806f0a0 0x7f8c3806dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.677+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/92919802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c3806d2d0 0x7f8c3806d6b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.677+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/92919802 >> 172.21.15.125:0/92919802 conn(0x7f8c380b0670 msgr2=0x7f8c380b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:48.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.677+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/92919802 shutdown_connections 2023-10-18T10:40:48.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.677+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/92919802 wait complete. 2023-10-18T10:40:48.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.678+0000 7f8c3d1ce640 1 Processor -- start 2023-10-18T10:40:48.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.678+0000 7f8c3d1ce640 1 -- start start 2023-10-18T10:40:48.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3d1ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c3806d2d0 0x7f8c38150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:48.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3d1ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c3806f0a0 0x7f8c380cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:48.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3d1ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380ab950 0x7f8c380c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:48.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3d1ce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c38150780 con 0x7f8c3806f0a0 2023-10-18T10:40:48.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3d1ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c38151940 con 0x7f8c380ab950 2023-10-18T10:40:48.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3d1ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c38151c20 con 0x7f8c3806d2d0 2023-10-18T10:40:48.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c3806f0a0 0x7f8c380cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:48.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c3806f0a0 0x7f8c380cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56250/0 (socket says 172.21.15.125:56250) 2023-10-18T10:40:48.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3659c640 1 -- 172.21.15.125:0/1826119683 learned_addr learned my addr 172.21.15.125:0/1826119683 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:48.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c36d9d640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c3806d2d0 0x7f8c38150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:48.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3759e640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380ab950 0x7f8c380c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:48.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3659c640 1 -- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c3806d2d0 msgr2=0x7f8c38150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:48.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3659c640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c3806d2d0 0x7f8c38150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3659c640 1 -- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380ab950 msgr2=0x7f8c380c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:48.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3659c640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380ab950 0x7f8c380c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:48.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.679+0000 7f8c3659c640 1 -- 172.21.15.125:0/1826119683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c009560 con 0x7f8c3806f0a0 2023-10-18T10:40:48.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.680+0000 7f8c3659c640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c3806f0a0 0x7f8c380cbbd0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00e9c0 tx=0x7f8c2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:48.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.680+0000 7f8c1f7fe640 1 -- 172.21.15.125:0/1826119683 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c00cde0 con 0x7f8c3806f0a0 2023-10-18T10:40:48.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.680+0000 7f8c1f7fe640 1 -- 172.21.15.125:0/1826119683 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8c2c004020 con 0x7f8c3806f0a0 2023-10-18T10:40:48.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.680+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c28009490 con 0x7f8c3806f0a0 2023-10-18T10:40:48.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.680+0000 7f8c1f7fe640 1 -- 172.21.15.125:0/1826119683 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c004410 con 0x7f8c3806f0a0 2023-10-18T10:40:48.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.680+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c381521e0 con 0x7f8c3806f0a0 2023-10-18T10:40:48.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.681+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c380ce1d0 con 0x7f8c3806f0a0 2023-10-18T10:40:48.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.681+0000 7f8c1f7fe640 1 -- 172.21.15.125:0/1826119683 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8c2c0035f0 con 0x7f8c3806f0a0 2023-10-18T10:40:48.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.682+0000 7f8c1f7fe640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8c2406af60 0x7f8c2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:48.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.682+0000 7f8c1f7fe640 1 -- 172.21.15.125:0/1826119683 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f8c2c014070 con 0x7f8c3806f0a0 2023-10-18T10:40:48.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.682+0000 7f8c36d9d640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8c2406af60 0x7f8c2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:48.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.682+0000 7f8c36d9d640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8c2406af60 0x7f8c2406d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8c28036000 tx=0x7f8c28004730 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:48.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.686+0000 7f8c1f7fe640 1 -- 172.21.15.125:0/1826119683 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c2c055270 con 0x7f8c3806f0a0 2023-10-18T10:40:48.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:48.865+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8c380690e0 con 0x7f8c3806f0a0 2023-10-18T10:40:49.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.984+0000 7f8c1f7fe640 1 -- 172.21.15.125:0/1826119683 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8c2c058890 con 0x7f8c3806f0a0 2023-10-18T10:40:49.986 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T10:40:49.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.986+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8c2406af60 msgr2=0x7f8c2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:49.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.986+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8c2406af60 0x7f8c2406d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8c28036000 tx=0x7f8c28004730 comp rx=0 tx=0).stop 2023-10-18T10:40:49.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c3806f0a0 msgr2=0x7f8c380cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:49.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c3806f0a0 0x7f8c380cbbd0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00e9c0 tx=0x7f8c2c00ee80 comp rx=0 tx=0).stop 2023-10-18T10:40:49.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 shutdown_connections 2023-10-18T10:40:49.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8c2406af60 0x7f8c2406d420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:49.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380ab950 0x7f8c380c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:49.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8c3806f0a0 0x7f8c380cbbd0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:49.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 --2- 172.21.15.125:0/1826119683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8c3806d2d0 0x7f8c38150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:49.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 >> 172.21.15.125:0/1826119683 conn(0x7f8c380b0670 msgr2=0x7f8c380af830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:49.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 shutdown_connections 2023-10-18T10:40:49.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:49.987+0000 7f8c3d1ce640 1 -- 172.21.15.125:0/1826119683 wait complete. 2023-10-18T10:40:50.001 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:40:50.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.108+0000 7f786616c640 1 -- 172.21.15.125:0/1806038273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7848005180 msgr2=0x7f784800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.108+0000 7f786616c640 1 --2- 172.21.15.125:0/1806038273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7848005180 0x7f784800b980 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f785800c610 tx=0x7f78480057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.108+0000 7f786616c640 1 -- 172.21.15.125:0/1806038273 shutdown_connections 2023-10-18T10:40:50.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.108+0000 7f786616c640 1 --2- 172.21.15.125:0/1806038273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7848005180 0x7f784800b980 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.108+0000 7f786616c640 1 -- 172.21.15.125:0/1806038273 >> 172.21.15.125:0/1806038273 conn(0x7f78600d0d60 msgr2=0x7f78600d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:50.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.108+0000 7f786616c640 1 -- 172.21.15.125:0/1806038273 shutdown_connections 2023-10-18T10:40:50.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.108+0000 7f786616c640 1 -- 172.21.15.125:0/1806038273 wait complete. 2023-10-18T10:40:50.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.109+0000 7f786616c640 1 Processor -- start 2023-10-18T10:40:50.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.109+0000 7f786616c640 1 -- start start 2023-10-18T10:40:50.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f786616c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7848005180 0x7f78601578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f786616c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78601e4990 0x7f7860157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f786616c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78601583b0 0x7f786015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f786616c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78601e4570 con 0x7f78601583b0 2023-10-18T10:40:50.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f786616c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78601e38f0 con 0x7f78601e4990 2023-10-18T10:40:50.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f786616c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78601e3bd0 con 0x7f7848005180 2023-10-18T10:40:50.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78601583b0 0x7f786015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78601583b0 0x7f786015d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56274/0 (socket says 172.21.15.125:56274) 2023-10-18T10:40:50.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785ffff640 1 -- 172.21.15.125:0/3507043628 learned_addr learned my addr 172.21.15.125:0/3507043628 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:50.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785effd640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78601e4990 0x7f7860157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785f7fe640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7848005180 0x7f78601578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785ffff640 1 -- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7848005180 msgr2=0x7f78601578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785ffff640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7848005180 0x7f78601578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785ffff640 1 -- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78601e4990 msgr2=0x7f7860157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785ffff640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78601e4990 0x7f7860157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.110+0000 7f785ffff640 1 -- 172.21.15.125:0/3507043628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78601e3eb0 con 0x7f78601583b0 2023-10-18T10:40:50.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.111+0000 7f785ffff640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78601583b0 0x7f786015d230 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f785800c730 tx=0x7f78580018e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:50.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.111+0000 7f7847fff640 1 -- 172.21.15.125:0/3507043628 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858001ad0 con 0x7f78601583b0 2023-10-18T10:40:50.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.111+0000 7f7847fff640 1 -- 172.21.15.125:0/3507043628 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7858003970 con 0x7f78601583b0 2023-10-18T10:40:50.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.111+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78601e4190 con 0x7f78601583b0 2023-10-18T10:40:50.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.111+0000 7f7847fff640 1 -- 172.21.15.125:0/3507043628 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858003d80 con 0x7f78601583b0 2023-10-18T10:40:50.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.111+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78601e3580 con 0x7f78601583b0 2023-10-18T10:40:50.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.112+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78600d07d0 con 0x7f78601583b0 2023-10-18T10:40:50.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.112+0000 7f7847fff640 1 -- 172.21.15.125:0/3507043628 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7858031030 con 0x7f78601583b0 2023-10-18T10:40:50.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.113+0000 7f7847fff640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f784c06af90 0x7f784c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.113+0000 7f7847fff640 1 -- 172.21.15.125:0/3507043628 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f78580a9fa0 con 0x7f78601583b0 2023-10-18T10:40:50.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.113+0000 7f785f7fe640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f784c06af90 0x7f784c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.113+0000 7f785f7fe640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f784c06af90 0x7f784c06d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78600b4180 tx=0x7f7850005f90 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:50.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.117+0000 7f7847fff640 1 -- 172.21.15.125:0/3507043628 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7858075180 con 0x7f78601583b0 2023-10-18T10:40:50.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.344+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f78601e4320 con 0x7f78601583b0 2023-10-18T10:40:50.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.344+0000 7f7847fff640 1 -- 172.21.15.125:0/3507043628 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7858042070 con 0x7f78601583b0 2023-10-18T10:40:50.346 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:50.347 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:40:50.347 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T10:40:50.347 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.347 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T10:40:50.348 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T10:40:50.348 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.348 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:50.348 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:50.348 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:50.349 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.349 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.349 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.349 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.349 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T10:40:50.350 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T10:40:50.350 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.350 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:50.350 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:50.350 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:50.351 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.351 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.351 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.351 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.351 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T10:40:50.352 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T10:40:50.352 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.352 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:50.352 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:50.352 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:50.353 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.353 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.353 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.353 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.353 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T10:40:50.354 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T10:40:50.354 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.354 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:50.354 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:50.355 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:50.355 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.355 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.355 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.355 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.355 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T10:40:50.356 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T10:40:50.356 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.356 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:50.356 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:50.356 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.357 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.357 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.357 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.357 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T10:40:50.357 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T10:40:50.358 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.358 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:50.358 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:50.358 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.359 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.359 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.359 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.359 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T10:40:50.359 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T10:40:50.360 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.360 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:50.360 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:50.360 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.360 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.361 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.361 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.361 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T10:40:50.361 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T10:40:50.361 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.361 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:50.362 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:50.362 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.362 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.362 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.362 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.363 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T10:40:50.363 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T10:40:50.363 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.363 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:50.363 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:50.364 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.364 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.364 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.364 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.364 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T10:40:50.365 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T10:40:50.365 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.365 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:50.365 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:50.365 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.366 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.366 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.366 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.366 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T10:40:50.366 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T10:40:50.367 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.367 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:50.367 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:50.367 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.367 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.367 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.368 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.368 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T10:40:50.368 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T10:40:50.368 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.368 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:40:50.369 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:40:50.369 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.369 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.369 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.369 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.370 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T10:40:50.370 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T10:40:50.370 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.370 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:50.370 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T10:40:50.371 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T10:40:50.371 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.371 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.371 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.371 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T10:40:50.372 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T10:40:50.372 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.372 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:50.372 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:40:50.372 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:40:50.373 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:40:50.373 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.373 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.373 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.373 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T10:40:50.373 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T10:40:50.374 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.374 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:50.374 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:40:50.374 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:40:50.374 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:40:50.375 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.375 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.375 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.375 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T10:40:50.375 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T10:40:50.376 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.376 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:40:50.376 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:40:50.376 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:40:50.376 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:40:50.376 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.377 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.377 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.377 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T10:40:50.377 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T10:40:50.377 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.377 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:40:50.378 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:40:50.378 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:40:50.378 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.378 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.378 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.379 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.379 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T10:40:50.379 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T10:40:50.379 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.379 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:40:50.380 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:40:50.380 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.380 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.380 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:40:50.380 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:50.381 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T10:40:50.381 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T10:40:50.381 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:40:50.381 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:40:50.381 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:40:50.382 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:40:50.382 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.382 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:50.382 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T10:40:50.382 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:40:50.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f784c06af90 msgr2=0x7f784c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f784c06af90 0x7f784c06d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78600b4180 tx=0x7f7850005f90 comp rx=0 tx=0).stop 2023-10-18T10:40:50.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78601583b0 msgr2=0x7f786015d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78601583b0 0x7f786015d230 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f785800c730 tx=0x7f78580018e0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 shutdown_connections 2023-10-18T10:40:50.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f784c06af90 0x7f784c06d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78601e4990 0x7f7860157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78601583b0 0x7f786015d230 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 --2- 172.21.15.125:0/3507043628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7848005180 0x7f78601578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 >> 172.21.15.125:0/3507043628 conn(0x7f78600d0d60 msgr2=0x7f786015a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:50.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 shutdown_connections 2023-10-18T10:40:50.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.346+0000 7f786616c640 1 -- 172.21.15.125:0/3507043628 wait complete. 2023-10-18T10:40:50.387 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:50.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 -- 172.21.15.125:0/2730344847 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 msgr2=0x7f39380b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 --2- 172.21.15.125:0/2730344847 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 0x7f39380b0fa0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f39380894b0 tx=0x7f393002bda0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 -- 172.21.15.125:0/2730344847 shutdown_connections 2023-10-18T10:40:50.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 --2- 172.21.15.125:0/2730344847 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 0x7f39380b0fa0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 --2- 172.21.15.125:0/2730344847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393815b640 0x7f39380cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 --2- 172.21.15.125:0/2730344847 >> v2:172.21.15.139:3300/0 conn(0x7f393815b260 0x7f39380cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 -- 172.21.15.125:0/2730344847 >> 172.21.15.125:0/2730344847 conn(0x7f393809a030 msgr2=0x7f39380d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:50.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 -- 172.21.15.125:0/2730344847 shutdown_connections 2023-10-18T10:40:50.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.487+0000 7f393d90f640 1 -- 172.21.15.125:0/2730344847 wait complete. 2023-10-18T10:40:50.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.488+0000 7f393d90f640 1 Processor -- start 2023-10-18T10:40:50.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.488+0000 7f393d90f640 1 -- start start 2023-10-18T10:40:50.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f393d90f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 0x7f393815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f393d90f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393815b260 0x7f393815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f393d90f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f393815b640 0x7f39380b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f393d90f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393815cf00 con 0x7f39380b0b10 2023-10-18T10:40:50.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f393d90f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393815d1e0 con 0x7f393815b640 2023-10-18T10:40:50.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f393d90f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393815d4c0 con 0x7f393815b260 2023-10-18T10:40:50.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f3936ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 0x7f393815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f3936ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 0x7f393815a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56296/0 (socket says 172.21.15.125:56296) 2023-10-18T10:40:50.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f3936ffd640 1 -- 172.21.15.125:0/2764566884 learned_addr learned my addr 172.21.15.125:0/2764566884 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:50.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f3936ffd640 1 -- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393815b260 msgr2=0x7f393815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f3936ffd640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393815b260 0x7f393815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f3936ffd640 1 -- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f393815b640 msgr2=0x7f39380b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f3936ffd640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f393815b640 0x7f39380b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.489+0000 7f3936ffd640 1 -- 172.21.15.125:0/2764566884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3930009560 con 0x7f39380b0b10 2023-10-18T10:40:50.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.490+0000 7f3936ffd640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 0x7f393815a000 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3928005fd0 tx=0x7f392800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:50.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.490+0000 7f391ffff640 1 -- 172.21.15.125:0/2764566884 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392800f650 con 0x7f39380b0b10 2023-10-18T10:40:50.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.490+0000 7f391ffff640 1 -- 172.21.15.125:0/2764566884 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f392800f8a0 con 0x7f39380b0b10 2023-10-18T10:40:50.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.490+0000 7f391ffff640 1 -- 172.21.15.125:0/2764566884 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392800fcd0 con 0x7f39380b0b10 2023-10-18T10:40:50.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.490+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393815d7a0 con 0x7f39380b0b10 2023-10-18T10:40:50.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.490+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393815ca70 con 0x7f39380b0b10 2023-10-18T10:40:50.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.491+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39380675f0 con 0x7f39380b0b10 2023-10-18T10:40:50.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.491+0000 7f391ffff640 1 -- 172.21.15.125:0/2764566884 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3928018400 con 0x7f39380b0b10 2023-10-18T10:40:50.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.491+0000 7f391ffff640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f392406af90 0x7f392406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.492+0000 7f391ffff640 1 -- 172.21.15.125:0/2764566884 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f392808a270 con 0x7f39380b0b10 2023-10-18T10:40:50.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.492+0000 7f39367fc640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f392406af90 0x7f392406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.492+0000 7f39367fc640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f392406af90 0x7f392406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39380af5b0 tx=0x7f392c005b50 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:50.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.496+0000 7f391ffff640 1 -- 172.21.15.125:0/2764566884 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3928055310 con 0x7f39380b0b10 2023-10-18T10:40:50.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.661+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39380cce10 con 0x7f39380b0b10 2023-10-18T10:40:50.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.662+0000 7f391ffff640 1 -- 172.21.15.125:0/2764566884 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13111 (secure 0 0 0) 0x7f392805de60 con 0x7f39380b0b10 2023-10-18T10:40:50.663 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:50.664 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":94,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:49.983498+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:50.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f392406af90 msgr2=0x7f392406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f392406af90 0x7f392406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39380af5b0 tx=0x7f392c005b50 comp rx=0 tx=0).stop 2023-10-18T10:40:50.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 msgr2=0x7f393815a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 0x7f393815a000 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3928005fd0 tx=0x7f392800a620 comp rx=0 tx=0).stop 2023-10-18T10:40:50.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 shutdown_connections 2023-10-18T10:40:50.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f392406af90 0x7f392406d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f393815b640 0x7f39380b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393815b260 0x7f393815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.663+0000 7f393d90f640 1 --2- 172.21.15.125:0/2764566884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39380b0b10 0x7f393815a000 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.664+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 >> 172.21.15.125:0/2764566884 conn(0x7f393809a030 msgr2=0x7f39380ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:50.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.664+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 shutdown_connections 2023-10-18T10:40:50.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.664+0000 7f393d90f640 1 -- 172.21.15.125:0/2764566884 wait complete. 2023-10-18T10:40:50.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:40:50.677 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:40:50.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.781+0000 7fe435b33640 1 -- 172.21.15.125:0/3037112976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 msgr2=0x7fe418004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.781+0000 7fe435b33640 1 --2- 172.21.15.125:0/3037112976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 0x7fe418004c40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe424009b50 tx=0x7fe4180057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.781+0000 7fe435b33640 1 -- 172.21.15.125:0/3037112976 shutdown_connections 2023-10-18T10:40:50.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.781+0000 7fe435b33640 1 --2- 172.21.15.125:0/3037112976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 0x7fe418004c40 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.781+0000 7fe435b33640 1 -- 172.21.15.125:0/3037112976 >> 172.21.15.125:0/3037112976 conn(0x7fe4300d0d60 msgr2=0x7fe4300d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:50.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.781+0000 7fe435b33640 1 -- 172.21.15.125:0/3037112976 shutdown_connections 2023-10-18T10:40:50.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.781+0000 7fe435b33640 1 -- 172.21.15.125:0/3037112976 wait complete. 2023-10-18T10:40:50.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.782+0000 7fe435b33640 1 Processor -- start 2023-10-18T10:40:50.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.782+0000 7fe435b33640 1 -- start start 2023-10-18T10:40:50.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe435b33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 0x7fe4300b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe435b33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4300c94a0 0x7fe43014f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe435b33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe43006a700 0x7fe43006ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe435b33640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43006b0c0 con 0x7fe4180047c0 2023-10-18T10:40:50.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe435b33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300c9920 con 0x7fe43006a700 2023-10-18T10:40:50.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe435b33640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300cb1e0 con 0x7fe4300c94a0 2023-10-18T10:40:50.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 0x7fe4300b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 0x7fe4300b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56314/0 (socket says 172.21.15.125:56314) 2023-10-18T10:40:50.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 -- 172.21.15.125:0/991144606 learned_addr learned my addr 172.21.15.125:0/991144606 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:50.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42ffff640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe43006a700 0x7fe43006ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 -- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4300c94a0 msgr2=0x7fe43014f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42effd640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4300c94a0 0x7fe43014f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4300c94a0 0x7fe43014f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 -- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe43006a700 msgr2=0x7fe43006ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe43006a700 0x7fe43006ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 -- 172.21.15.125:0/991144606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4300b59e0 con 0x7fe4180047c0 2023-10-18T10:40:50.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe42f7fe640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 0x7fe4300b5350 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe420006630 tx=0x7fe4200039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:50.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.783+0000 7fe417fff640 1 -- 172.21.15.125:0/991144606 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420011900 con 0x7fe4180047c0 2023-10-18T10:40:50.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.784+0000 7fe417fff640 1 -- 172.21.15.125:0/991144606 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe420011a60 con 0x7fe4180047c0 2023-10-18T10:40:50.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.784+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4300b5cc0 con 0x7fe4180047c0 2023-10-18T10:40:50.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.784+0000 7fe417fff640 1 -- 172.21.15.125:0/991144606 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420011e50 con 0x7fe4180047c0 2023-10-18T10:40:50.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.784+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4300b6280 con 0x7fe4180047c0 2023-10-18T10:40:50.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.784+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4300690e0 con 0x7fe4180047c0 2023-10-18T10:40:50.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.785+0000 7fe417fff640 1 -- 172.21.15.125:0/991144606 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe42000f430 con 0x7fe4180047c0 2023-10-18T10:40:50.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.785+0000 7fe417fff640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe41c06af60 0x7fe41c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:50.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.785+0000 7fe417fff640 1 -- 172.21.15.125:0/991144606 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe42008a490 con 0x7fe4180047c0 2023-10-18T10:40:50.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.786+0000 7fe42effd640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe41c06af60 0x7fe41c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:50.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.786+0000 7fe42effd640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe41c06af60 0x7fe41c06d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe424036000 tx=0x7fe424035040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:50.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.789+0000 7fe417fff640 1 -- 172.21.15.125:0/991144606 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe420055540 con 0x7fe4180047c0 2023-10-18T10:40:50.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.956+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4300d0b40 con 0x7fe4180047c0 2023-10-18T10:40:50.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.956+0000 7fe417fff640 1 -- 172.21.15.125:0/991144606 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe420020020 con 0x7fe4180047c0 2023-10-18T10:40:50.959 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:40:50.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe41c06af60 msgr2=0x7fe41c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe41c06af60 0x7fe41c06d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe424036000 tx=0x7fe424035040 comp rx=0 tx=0).stop 2023-10-18T10:40:50.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 msgr2=0x7fe4300b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:50.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 0x7fe4300b5350 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe420006630 tx=0x7fe4200039c0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 shutdown_connections 2023-10-18T10:40:50.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe41c06af60 0x7fe41c06d420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe43006a700 0x7fe43006ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4300c94a0 0x7fe43014f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 --2- 172.21.15.125:0/991144606 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4180047c0 0x7fe4300b5350 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:50.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 >> 172.21.15.125:0/991144606 conn(0x7fe4300d0d60 msgr2=0x7fe4300ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:50.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 shutdown_connections 2023-10-18T10:40:50.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:50.958+0000 7fe435b33640 1 -- 172.21.15.125:0/991144606 wait complete. 2023-10-18T10:40:50.971 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:40:51.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.073+0000 7fb7453a9640 1 -- 172.21.15.125:0/764860471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7400b1ca0 msgr2=0x7fb7400cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:51.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.073+0000 7fb7453a9640 1 --2- 172.21.15.125:0/764860471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7400b1ca0 0x7fb7400cf1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb728005fd0 tx=0x7fb72802be10 comp rx=0 tx=0).stop 2023-10-18T10:40:51.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.073+0000 7fb7453a9640 1 -- 172.21.15.125:0/764860471 shutdown_connections 2023-10-18T10:40:51.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.073+0000 7fb7453a9640 1 --2- 172.21.15.125:0/764860471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7400b1ca0 0x7fb7400cf1a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.073+0000 7fb7453a9640 1 --2- 172.21.15.125:0/764860471 >> v2:172.21.15.139:3300/0 conn(0x7fb74015e5e0 0x7fb7400ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.073+0000 7fb7453a9640 1 -- 172.21.15.125:0/764860471 >> 172.21.15.125:0/764860471 conn(0x7fb7400d0c50 msgr2=0x7fb7400d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:51.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.073+0000 7fb7453a9640 1 -- 172.21.15.125:0/764860471 shutdown_connections 2023-10-18T10:40:51.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.073+0000 7fb7453a9640 1 -- 172.21.15.125:0/764860471 wait complete. 2023-10-18T10:40:51.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.074+0000 7fb7453a9640 1 Processor -- start 2023-10-18T10:40:51.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.074+0000 7fb7453a9640 1 -- start start 2023-10-18T10:40:51.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7453a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7400b1ca0 0x7fb740156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:51.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7453a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7400cf6e0 0x7fb7401545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:51.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7453a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb74015e5e0 0x7fb740151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:51.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7453a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb740154af0 con 0x7fb7400cf6e0 2023-10-18T10:40:51.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7453a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb740153cb0 con 0x7fb7400b1ca0 2023-10-18T10:40:51.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7453a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb740153f90 con 0x7fb74015e5e0 2023-10-18T10:40:51.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7400cf6e0 0x7fb7401545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:51.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7400cf6e0 0x7fb7401545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56322/0 (socket says 172.21.15.125:56322) 2023-10-18T10:40:51.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 -- 172.21.15.125:0/2889515354 learned_addr learned my addr 172.21.15.125:0/2889515354 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:51.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb73effd640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7400b1ca0 0x7fb740156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:51.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb73f7fe640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb74015e5e0 0x7fb740151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:51.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 -- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb74015e5e0 msgr2=0x7fb740151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:51.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb74015e5e0 0x7fb740151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 -- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7400b1ca0 msgr2=0x7fb740156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:51.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7400b1ca0 0x7fb740156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 -- 172.21.15.125:0/2889515354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb730009490 con 0x7fb7400cf6e0 2023-10-18T10:40:51.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb737fff640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7400cf6e0 0x7fb7401545b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb728036040 tx=0x7fb72802da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:51.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7377fe640 1 -- 172.21.15.125:0/2889515354 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72802de70 con 0x7fb7400cf6e0 2023-10-18T10:40:51.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7377fe640 1 -- 172.21.15.125:0/2889515354 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb7280036e0 con 0x7fb7400cf6e0 2023-10-18T10:40:51.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.075+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb728009560 con 0x7fb7400cf6e0 2023-10-18T10:40:51.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.076+0000 7fb7377fe640 1 -- 172.21.15.125:0/2889515354 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728003af0 con 0x7fb7400cf6e0 2023-10-18T10:40:51.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.076+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7401f50b0 con 0x7fb7400cf6e0 2023-10-18T10:40:51.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.077+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7400690e0 con 0x7fb7400cf6e0 2023-10-18T10:40:51.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.077+0000 7fb7377fe640 1 -- 172.21.15.125:0/2889515354 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb728003c50 con 0x7fb7400cf6e0 2023-10-18T10:40:51.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.077+0000 7fb7377fe640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb72006af60 0x7fb72006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:51.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.077+0000 7fb7377fe640 1 -- 172.21.15.125:0/2889515354 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb7280ac040 con 0x7fb7400cf6e0 2023-10-18T10:40:51.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.078+0000 7fb73effd640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb72006af60 0x7fb72006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:51.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.078+0000 7fb73effd640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb72006af60 0x7fb72006d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb730009c50 tx=0x7fb730005f90 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:51.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.082+0000 7fb7377fe640 1 -- 172.21.15.125:0/2889515354 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7280771f0 con 0x7fb7400cf6e0 2023-10-18T10:40:51.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.250+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb7401f52a0 con 0x7fb7400cf6e0 2023-10-18T10:40:51.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.250+0000 7fb7377fe640 1 -- 172.21.15.125:0/2889515354 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v94) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb728043060 con 0x7fb7400cf6e0 2023-10-18T10:40:51.253 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:51.253 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:40:51.253 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:40:51.253 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:40:51.253 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:40:51.254 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:40:51.254 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:40:51.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb72006af60 msgr2=0x7fb72006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:51.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb72006af60 0x7fb72006d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb730009c50 tx=0x7fb730005f90 comp rx=0 tx=0).stop 2023-10-18T10:40:51.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7400cf6e0 msgr2=0x7fb7401545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:51.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7400cf6e0 0x7fb7401545b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb728036040 tx=0x7fb72802da70 comp rx=0 tx=0).stop 2023-10-18T10:40:51.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 shutdown_connections 2023-10-18T10:40:51.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb72006af60 0x7fb72006d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb74015e5e0 0x7fb740151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7400cf6e0 0x7fb7401545b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 --2- 172.21.15.125:0/2889515354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7400b1ca0 0x7fb740156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 >> 172.21.15.125:0/2889515354 conn(0x7fb7400d0c50 msgr2=0x7fb74015cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:51.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 shutdown_connections 2023-10-18T10:40:51.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.252+0000 7fb7453a9640 1 -- 172.21.15.125:0/2889515354 wait complete. 2023-10-18T10:40:51.265 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:40:51.265 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:40:51.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.369+0000 7f25c48de640 1 -- 172.21.15.125:0/2454733450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 msgr2=0x7f25c00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:51.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.369+0000 7f25c48de640 1 --2- 172.21.15.125:0/2454733450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 0x7f25c00d1910 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009830 tx=0x7f25b002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.369+0000 7f25c48de640 1 -- 172.21.15.125:0/2454733450 shutdown_connections 2023-10-18T10:40:51.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.369+0000 7f25c48de640 1 --2- 172.21.15.125:0/2454733450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25c00ceb40 0x7f25c00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.369+0000 7f25c48de640 1 --2- 172.21.15.125:0/2454733450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 0x7f25c00d1910 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.369+0000 7f25c48de640 1 -- 172.21.15.125:0/2454733450 >> 172.21.15.125:0/2454733450 conn(0x7f25c00d0d60 msgr2=0x7f25c00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:51.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.369+0000 7f25c48de640 1 -- 172.21.15.125:0/2454733450 shutdown_connections 2023-10-18T10:40:51.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.369+0000 7f25c48de640 1 -- 172.21.15.125:0/2454733450 wait complete. 2023-10-18T10:40:51.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.370+0000 7f25c48de640 1 Processor -- start 2023-10-18T10:40:51.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25c48de640 1 -- start start 2023-10-18T10:40:51.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25c48de640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25c00ceb40 0x7f25c009e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:51.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25c48de640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 0x7f25c01581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:51.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25c48de640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25c015b2a0 0x7f25c01f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:51.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25c48de640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c009eb20 con 0x7f25c00cf340 2023-10-18T10:40:51.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25c48de640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0158720 con 0x7f25c00ceb40 2023-10-18T10:40:51.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25c48de640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01f2370 con 0x7f25c015b2a0 2023-10-18T10:40:51.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 0x7f25c01581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:51.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 0x7f25c01581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56336/0 (socket says 172.21.15.125:56336) 2023-10-18T10:40:51.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.371+0000 7f25b7fff640 1 -- 172.21.15.125:0/120464512 learned_addr learned my addr 172.21.15.125:0/120464512 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:51.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25be59c640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25c00ceb40 0x7f25c009e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:51.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25bed9d640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25c015b2a0 0x7f25c01f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:51.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b7fff640 1 -- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25c015b2a0 msgr2=0x7f25c01f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:51.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b7fff640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25c015b2a0 0x7f25c01f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b7fff640 1 -- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25c00ceb40 msgr2=0x7f25c009e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:51.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b7fff640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25c00ceb40 0x7f25c009e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:51.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b7fff640 1 -- 172.21.15.125:0/120464512 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0009490 con 0x7f25c00cf340 2023-10-18T10:40:51.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25be59c640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25c00ceb40 0x7f25c009e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:51.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b7fff640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 0x7f25c01581e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f25a800c420 tx=0x7f25a800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:51.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b6ffd640 1 -- 172.21.15.125:0/120464512 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8004230 con 0x7f25c00cf340 2023-10-18T10:40:51.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b6ffd640 1 -- 172.21.15.125:0/120464512 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25a80044a0 con 0x7f25c00cf340 2023-10-18T10:40:51.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c00c9de0 con 0x7f25c00cf340 2023-10-18T10:40:51.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25b6ffd640 1 -- 172.21.15.125:0/120464512 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8003850 con 0x7f25c00cf340 2023-10-18T10:40:51.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.372+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c01f4990 con 0x7f25c00cf340 2023-10-18T10:40:51.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.373+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c00d1920 con 0x7f25c00cf340 2023-10-18T10:40:51.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.373+0000 7f25b6ffd640 1 -- 172.21.15.125:0/120464512 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f25a80039b0 con 0x7f25c00cf340 2023-10-18T10:40:51.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.374+0000 7f25b6ffd640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f25a006af60 0x7f25a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:51.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.374+0000 7f25b6ffd640 1 -- 172.21.15.125:0/120464512 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f25a8088cf0 con 0x7f25c00cf340 2023-10-18T10:40:51.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.374+0000 7f25be59c640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f25a006af60 0x7f25a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:51.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.374+0000 7f25be59c640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f25a006af60 0x7f25a006d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f25b0036000 tx=0x7f25b0004730 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:51.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.378+0000 7f25b6ffd640 1 -- 172.21.15.125:0/120464512 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25a808d070 con 0x7f25c00cf340 2023-10-18T10:40:51.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:51.544+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f25c00b0cd0 con 0x7f25c00cf340 2023-10-18T10:40:52.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.400+0000 7f25b6ffd640 1 -- 172.21.15.125:0/120464512 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v95) v1 ==== 138+0+0 (secure 0 0 0) 0x7f25a8017020 con 0x7f25c00cf340 2023-10-18T10:40:52.402 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:40:52.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f25a006af60 msgr2=0x7f25a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:52.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f25a006af60 0x7f25a006d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f25b0036000 tx=0x7f25b0004730 comp rx=0 tx=0).stop 2023-10-18T10:40:52.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 msgr2=0x7f25c01581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:52.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 0x7f25c01581e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f25a800c420 tx=0x7f25a800c8e0 comp rx=0 tx=0).stop 2023-10-18T10:40:52.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 shutdown_connections 2023-10-18T10:40:52.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f25a006af60 0x7f25a006d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:52.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25c015b2a0 0x7f25c01f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:52.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25c00cf340 0x7f25c01581e0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:52.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 --2- 172.21.15.125:0/120464512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25c00ceb40 0x7f25c009e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:52.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 >> 172.21.15.125:0/120464512 conn(0x7f25c00d0d60 msgr2=0x7f25c015a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:52.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 shutdown_connections 2023-10-18T10:40:52.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.403+0000 7f25c48de640 1 -- 172.21.15.125:0/120464512 wait complete. 2023-10-18T10:40:52.421 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:40:52.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.534+0000 7fef68bd0640 1 -- 172.21.15.125:0/4051672474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 msgr2=0x7fef50003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:52.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.534+0000 7fef68bd0640 1 --2- 172.21.15.125:0/4051672474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 0x7fef50003d90 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009ed0 tx=0x7fef500057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:52.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.534+0000 7fef62d9d640 1 -- 172.21.15.125:0/4051672474 reap_dead start 2023-10-18T10:40:52.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.534+0000 7fef68bd0640 1 -- 172.21.15.125:0/4051672474 shutdown_connections 2023-10-18T10:40:52.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.534+0000 7fef68bd0640 1 -- 172.21.15.125:0/4051672474 >> 172.21.15.125:0/4051672474 conn(0x7fef640d0d60 msgr2=0x7fef640d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:52.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.534+0000 7fef68bd0640 1 -- 172.21.15.125:0/4051672474 shutdown_connections 2023-10-18T10:40:52.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.534+0000 7fef68bd0640 1 -- 172.21.15.125:0/4051672474 wait complete. 2023-10-18T10:40:52.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.535+0000 7fef68bd0640 1 Processor -- start 2023-10-18T10:40:52.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.535+0000 7fef68bd0640 1 -- start start 2023-10-18T10:40:52.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef68bd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 0x7fef641574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:52.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef68bd0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef640b4040 0x7fef6415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:52.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef68bd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef6415bc10 0x7fef641f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:52.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef68bd0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6415dc50 con 0x7fef50003970 2023-10-18T10:40:52.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef68bd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6415c090 con 0x7fef6415bc10 2023-10-18T10:40:52.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef68bd0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef641f20a0 con 0x7fef640b4040 2023-10-18T10:40:52.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 0x7fef641574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:52.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 0x7fef641574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56362/0 (socket says 172.21.15.125:56362) 2023-10-18T10:40:52.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 -- 172.21.15.125:0/1886996938 learned_addr learned my addr 172.21.15.125:0/1886996938 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:52.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 -- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef640b4040 msgr2=0x7fef6415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:52.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef640b4040 0x7fef6415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:52.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 -- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef6415bc10 msgr2=0x7fef641f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:52.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef6415bc10 0x7fef641f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:52.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 -- 172.21.15.125:0/1886996938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef641f2380 con 0x7fef50003970 2023-10-18T10:40:52.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.536+0000 7fef6259c640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 0x7fef641574f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009e20 tx=0x7fef5c030990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:52.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.537+0000 7fef5affd640 1 -- 172.21.15.125:0/1886996938 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c003970 con 0x7fef50003970 2023-10-18T10:40:52.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.537+0000 7fef5affd640 1 -- 172.21.15.125:0/1886996938 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef5c003bc0 con 0x7fef50003970 2023-10-18T10:40:52.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.537+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6406a030 con 0x7fef50003970 2023-10-18T10:40:52.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.537+0000 7fef5affd640 1 -- 172.21.15.125:0/1886996938 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004030 con 0x7fef50003970 2023-10-18T10:40:52.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.537+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6406a5f0 con 0x7fef50003970 2023-10-18T10:40:52.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.537+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6406a780 con 0x7fef50003970 2023-10-18T10:40:52.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.538+0000 7fef5affd640 1 -- 172.21.15.125:0/1886996938 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fef5c004190 con 0x7fef50003970 2023-10-18T10:40:52.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.538+0000 7fef5affd640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef4406af90 0x7fef4406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:52.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.539+0000 7fef5affd640 1 -- 172.21.15.125:0/1886996938 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fef5c0abe80 con 0x7fef50003970 2023-10-18T10:40:52.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.539+0000 7fef61d9b640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef4406af90 0x7fef4406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:52.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.539+0000 7fef61d9b640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef4406af90 0x7fef4406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fef4c000c00 tx=0x7fef4c001110 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:52.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.543+0000 7fef5affd640 1 -- 172.21.15.125:0/1886996938 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef5c076e40 con 0x7fef50003970 2023-10-18T10:40:52.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:52.712+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fef640690e0 con 0x7fef50003970 2023-10-18T10:40:53.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.407+0000 7fef5affd640 1 -- 172.21.15.125:0/1886996938 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v96) v1 ==== 170+0+0 (secure 0 0 0) 0x7fef5c045070 con 0x7fef50003970 2023-10-18T10:40:53.410 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:40:53.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef4406af90 msgr2=0x7fef4406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:53.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef4406af90 0x7fef4406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fef4c000c00 tx=0x7fef4c001110 comp rx=0 tx=0).stop 2023-10-18T10:40:53.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 msgr2=0x7fef641574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:53.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 0x7fef641574f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009e20 tx=0x7fef5c030990 comp rx=0 tx=0).stop 2023-10-18T10:40:53.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 shutdown_connections 2023-10-18T10:40:53.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef4406af90 0x7fef4406d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:53.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef6415bc10 0x7fef641f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:53.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef640b4040 0x7fef6415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:53.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 --2- 172.21.15.125:0/1886996938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef50003970 0x7fef641574f0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:53.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 >> 172.21.15.125:0/1886996938 conn(0x7fef640d0d60 msgr2=0x7fef640ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:53.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.409+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 shutdown_connections 2023-10-18T10:40:53.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.410+0000 7fef68bd0640 1 -- 172.21.15.125:0/1886996938 wait complete. 2023-10-18T10:40:53.434 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:40:53.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.537+0000 7f0325dc7640 1 -- 172.21.15.125:0/446661185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f032015b180 msgr2=0x7f03200cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:53.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.537+0000 7f0325dc7640 1 --2- 172.21.15.125:0/446661185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f032015b180 0x7f03200cf350 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0310005fd0 tx=0x7f031002bed0 comp rx=0 tx=0).stop 2023-10-18T10:40:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.537+0000 7f0325dc7640 1 -- 172.21.15.125:0/446661185 shutdown_connections 2023-10-18T10:40:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.537+0000 7f0325dc7640 1 --2- 172.21.15.125:0/446661185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f032015b180 0x7f03200cf350 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.537+0000 7f0325dc7640 1 -- 172.21.15.125:0/446661185 >> 172.21.15.125:0/446661185 conn(0x7f0320099ff0 msgr2=0x7f03200d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:53.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.537+0000 7f0325dc7640 1 -- 172.21.15.125:0/446661185 shutdown_connections 2023-10-18T10:40:53.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.537+0000 7f0325dc7640 1 -- 172.21.15.125:0/446661185 wait complete. 2023-10-18T10:40:53.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.538+0000 7f0325dc7640 1 Processor -- start 2023-10-18T10:40:53.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.538+0000 7f0325dc7640 1 -- start start 2023-10-18T10:40:53.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f0325dc7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03200cddc0 0x7f0320158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:53.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f0325dc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03200ce420 0x7f03201e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:53.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f0325dc7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f032015b180 0x7f03201e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:53.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f0325dc7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201e4740 con 0x7f03200cddc0 2023-10-18T10:40:53.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f0325dc7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201e4a20 con 0x7f03200ce420 2023-10-18T10:40:53.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f0325dc7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03201e4d00 con 0x7f032015b180 2023-10-18T10:40:53.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03200cddc0 0x7f0320158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:53.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f031f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03200cddc0 0x7f0320158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56378/0 (socket says 172.21.15.125:56378) 2023-10-18T10:40:53.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f031f7fe640 1 -- 172.21.15.125:0/102836817 learned_addr learned my addr 172.21.15.125:0/102836817 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:53.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.539+0000 7f0316ffd640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03200ce420 0x7f03201e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:53.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031f7fe640 1 -- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f032015b180 msgr2=0x7f03201e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:53.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031ffff640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f032015b180 0x7f03201e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:53.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031f7fe640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f032015b180 0x7f03201e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:53.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031f7fe640 1 -- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03200ce420 msgr2=0x7f03201e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:53.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031f7fe640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03200ce420 0x7f03201e39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:53.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031f7fe640 1 -- 172.21.15.125:0/102836817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0310009560 con 0x7f03200cddc0 2023-10-18T10:40:53.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031ffff640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f032015b180 0x7f03201e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:53.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031f7fe640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03200cddc0 0x7f0320158310 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f03100099f0 tx=0x7f0310003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:53.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031cff9640 1 -- 172.21.15.125:0/102836817 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310003d10 con 0x7f03200cddc0 2023-10-18T10:40:53.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03201e4fe0 con 0x7f03200cddc0 2023-10-18T10:40:53.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031cff9640 1 -- 172.21.15.125:0/102836817 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0310005020 con 0x7f03200cddc0 2023-10-18T10:40:53.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f031cff9640 1 -- 172.21.15.125:0/102836817 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310005410 con 0x7f03200cddc0 2023-10-18T10:40:53.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.540+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201e4450 con 0x7f03200cddc0 2023-10-18T10:40:53.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.541+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0320066960 con 0x7f03200cddc0 2023-10-18T10:40:53.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.541+0000 7f031cff9640 1 -- 172.21.15.125:0/102836817 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0310005570 con 0x7f03200cddc0 2023-10-18T10:40:53.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.542+0000 7f031cff9640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f030006af60 0x7f030006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:53.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.542+0000 7f031cff9640 1 -- 172.21.15.125:0/102836817 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f03100ac690 con 0x7f03200cddc0 2023-10-18T10:40:53.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.542+0000 7f0316ffd640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f030006af60 0x7f030006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:53.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.543+0000 7f0316ffd640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f030006af60 0x7f030006d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03201f4880 tx=0x7f0308009040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:53.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.546+0000 7f031cff9640 1 -- 172.21.15.125:0/102836817 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0310077560 con 0x7f03200cddc0 2023-10-18T10:40:53.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:53.747+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f03201e5500 con 0x7f03200cddc0 2023-10-18T10:40:54.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.419+0000 7f031cff9640 1 -- 172.21.15.125:0/102836817 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v33) v1 ==== 174+0+0 (secure 0 0 0) 0x7f031007ab80 con 0x7f03200cddc0 2023-10-18T10:40:54.421 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 11 and data pool 12 2023-10-18T10:40:54.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.434+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f030006af60 msgr2=0x7f030006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.434+0000 7f0325dc7640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f030006af60 0x7f030006d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03201f4880 tx=0x7f0308009040 comp rx=0 tx=0).stop 2023-10-18T10:40:54.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.434+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03200cddc0 msgr2=0x7f0320158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.434+0000 7f0325dc7640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03200cddc0 0x7f0320158310 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f03100099f0 tx=0x7f0310003910 comp rx=0 tx=0).stop 2023-10-18T10:40:54.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.437+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 shutdown_connections 2023-10-18T10:40:54.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.437+0000 7f0325dc7640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f030006af60 0x7f030006d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.437+0000 7f0325dc7640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03200ce420 0x7f03201e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.437+0000 7f0325dc7640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f032015b180 0x7f03201e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.437+0000 7f0325dc7640 1 --2- 172.21.15.125:0/102836817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03200cddc0 0x7f0320158310 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.437+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 >> 172.21.15.125:0/102836817 conn(0x7f0320099ff0 msgr2=0x7f032015a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:54.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.437+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 shutdown_connections 2023-10-18T10:40:54.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.437+0000 7f0325dc7640 1 -- 172.21.15.125:0/102836817 wait complete. 2023-10-18T10:40:54.452 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:54.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 -- 172.21.15.125:0/1642542990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84000b1870 msgr2=0x7f8400158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 --2- 172.21.15.125:0/1642542990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84000b1870 0x7f8400158b20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8400072010 tx=0x7f83fc02c1d0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 -- 172.21.15.125:0/1642542990 shutdown_connections 2023-10-18T10:40:54.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 --2- 172.21.15.125:0/1642542990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840006c990 0x7f840006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 --2- 172.21.15.125:0/1642542990 >> v2:172.21.15.139:3300/0 conn(0x7f8400070130 0x7f84000b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 --2- 172.21.15.125:0/1642542990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84000b1870 0x7f8400158b20 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 -- 172.21.15.125:0/1642542990 >> 172.21.15.125:0/1642542990 conn(0x7f84000af830 msgr2=0x7f84000afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:54.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 -- 172.21.15.125:0/1642542990 shutdown_connections 2023-10-18T10:40:54.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.554+0000 7f8408061640 1 -- 172.21.15.125:0/1642542990 wait complete. 2023-10-18T10:40:54.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.555+0000 7f8408061640 1 Processor -- start 2023-10-18T10:40:54.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.555+0000 7f8408061640 1 -- start start 2023-10-18T10:40:54.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f8408061640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840006c990 0x7f840014b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:54.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f8408061640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8400070130 0x7f840014bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:54.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f8408061640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84000b1870 0x7f840014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:54.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f8408061640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c9b60 con 0x7f8400070130 2023-10-18T10:40:54.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f8408061640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c9e40 con 0x7f84000b1870 2023-10-18T10:40:54.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f8408061640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84000ca120 con 0x7f840006c990 2023-10-18T10:40:54.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f84055fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8400070130 0x7f840014bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:54.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f84055fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8400070130 0x7f840014bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56392/0 (socket says 172.21.15.125:56392) 2023-10-18T10:40:54.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f84055fc640 1 -- 172.21.15.125:0/4056394725 learned_addr learned my addr 172.21.15.125:0/4056394725 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:54.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f8405dfd640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840006c990 0x7f840014b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:54.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.556+0000 7f84065fe640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84000b1870 0x7f840014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:54.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f84055fc640 1 -- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840006c990 msgr2=0x7f840014b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f84055fc640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840006c990 0x7f840014b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f84055fc640 1 -- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84000b1870 msgr2=0x7f840014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f84055fc640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84000b1870 0x7f840014e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f84055fc640 1 -- 172.21.15.125:0/4056394725 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc009900 con 0x7f8400070130 2023-10-18T10:40:54.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f84065fe640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84000b1870 0x7f840014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:54.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f8405dfd640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840006c990 0x7f840014b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:54.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f84055fc640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8400070130 0x7f840014bf80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f83f000d950 tx=0x7f83f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:54.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f83f67fc640 1 -- 172.21.15.125:0/4056394725 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0009c30 con 0x7f8400070130 2023-10-18T10:40:54.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f83f67fc640 1 -- 172.21.15.125:0/4056394725 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83f0009e80 con 0x7f8400070130 2023-10-18T10:40:54.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84000c2aa0 con 0x7f8400070130 2023-10-18T10:40:54.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f83f67fc640 1 -- 172.21.15.125:0/4056394725 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f00106c0 con 0x7f8400070130 2023-10-18T10:40:54.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.557+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8400154810 con 0x7f8400070130 2023-10-18T10:40:54.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.558+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84000690e0 con 0x7f8400070130 2023-10-18T10:40:54.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.558+0000 7f83f67fc640 1 -- 172.21.15.125:0/4056394725 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f83f0010820 con 0x7f8400070130 2023-10-18T10:40:54.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.559+0000 7f83f67fc640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f83e406af60 0x7f83e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:54.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.559+0000 7f83f67fc640 1 -- 172.21.15.125:0/4056394725 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f83f0089e50 con 0x7f8400070130 2023-10-18T10:40:54.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.559+0000 7f8405dfd640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f83e406af60 0x7f83e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:54.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.559+0000 7f8405dfd640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f83e406af60 0x7f83e406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83fc036000 tx=0x7f83fc004ab0 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:54.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.563+0000 7f83f67fc640 1 -- 172.21.15.125:0/4056394725 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83f0054c30 con 0x7f8400070130 2023-10-18T10:40:54.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.729+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f840006dfd0 con 0x7f8400070130 2023-10-18T10:40:54.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.730+0000 7f83f67fc640 1 -- 172.21.15.125:0/4056394725 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16838 (secure 0 0 0) 0x7f83f0017090 con 0x7f8400070130 2023-10-18T10:40:54.732 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:54.733 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":97,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:53.749113+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:51.563590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:52.731689+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:54.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f83e406af60 msgr2=0x7f83e406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f83e406af60 0x7f83e406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83fc036000 tx=0x7f83fc004ab0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8400070130 msgr2=0x7f840014bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8400070130 0x7f840014bf80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f83f000d950 tx=0x7f83f000de10 comp rx=0 tx=0).stop 2023-10-18T10:40:54.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 shutdown_connections 2023-10-18T10:40:54.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f83e406af60 0x7f83e406d420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84000b1870 0x7f840014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8400070130 0x7f840014bf80 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 --2- 172.21.15.125:0/4056394725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840006c990 0x7f840014b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 >> 172.21.15.125:0/4056394725 conn(0x7f84000af830 msgr2=0x7f84000cc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:54.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 shutdown_connections 2023-10-18T10:40:54.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.732+0000 7f8408061640 1 -- 172.21.15.125:0/4056394725 wait complete. 2023-10-18T10:40:54.745 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:54.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.845+0000 7f978139e640 1 -- 172.21.15.125:0/4015248010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97640039a0 msgr2=0x7f9764003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.845+0000 7f978139e640 1 --2- 172.21.15.125:0/4015248010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97640039a0 0x7f9764003dc0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f976c009e20 tx=0x7f97640057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.845+0000 7f977b7fe640 1 -- 172.21.15.125:0/4015248010 reap_dead start 2023-10-18T10:40:54.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.845+0000 7f978139e640 1 -- 172.21.15.125:0/4015248010 shutdown_connections 2023-10-18T10:40:54.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.845+0000 7f978139e640 1 -- 172.21.15.125:0/4015248010 >> 172.21.15.125:0/4015248010 conn(0x7f977c069930 msgr2=0x7f977c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:54.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.845+0000 7f978139e640 1 -- 172.21.15.125:0/4015248010 shutdown_connections 2023-10-18T10:40:54.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.845+0000 7f978139e640 1 -- 172.21.15.125:0/4015248010 wait complete. 2023-10-18T10:40:54.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f978139e640 1 Processor -- start 2023-10-18T10:40:54.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f978139e640 1 -- start start 2023-10-18T10:40:54.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f978139e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97640039a0 0x7f977c0a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:54.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f978139e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f977c138e10 0x7f977c1384c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:54.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f978139e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f977c0aebb0 0x7f977c1e2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:54.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f978139e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0a7b60 con 0x7f977c0aebb0 2023-10-18T10:40:54.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f978139e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0adb10 con 0x7f97640039a0 2023-10-18T10:40:54.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f978139e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0addf0 con 0x7f977c138e10 2023-10-18T10:40:54.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.847+0000 7f977b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f977c0aebb0 0x7f977c1e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:54.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f977c0aebb0 0x7f977c1e2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56412/0 (socket says 172.21.15.125:56412) 2023-10-18T10:40:54.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977b7fe640 1 -- 172.21.15.125:0/428059730 learned_addr learned my addr 172.21.15.125:0/428059730 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:54.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977affd640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97640039a0 0x7f977c0a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:54.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977a7fc640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f977c138e10 0x7f977c1384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:54.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977b7fe640 1 -- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f977c138e10 msgr2=0x7f977c1384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977b7fe640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f977c138e10 0x7f977c1384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977b7fe640 1 -- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97640039a0 msgr2=0x7f977c0a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:54.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977b7fe640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97640039a0 0x7f977c0a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:54.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977b7fe640 1 -- 172.21.15.125:0/428059730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0ae0d0 con 0x7f977c0aebb0 2023-10-18T10:40:54.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977a7fc640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f977c138e10 0x7f977c1384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:54.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f977b7fe640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f977c0aebb0 0x7f977c1e2890 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f977c13f3f0 tx=0x7f977400e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:54.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f97637fe640 1 -- 172.21.15.125:0/428059730 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774011870 con 0x7f977c0aebb0 2023-10-18T10:40:54.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f97637fe640 1 -- 172.21.15.125:0/428059730 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9774011ae0 con 0x7f977c0aebb0 2023-10-18T10:40:54.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c0ae3b0 con 0x7f977c0aebb0 2023-10-18T10:40:54.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f97637fe640 1 -- 172.21.15.125:0/428059730 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740103c0 con 0x7f977c0aebb0 2023-10-18T10:40:54.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.848+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c138a00 con 0x7f977c0aebb0 2023-10-18T10:40:54.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.849+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c0ae8d0 con 0x7f977c0aebb0 2023-10-18T10:40:54.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.849+0000 7f97637fe640 1 -- 172.21.15.125:0/428059730 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9774010520 con 0x7f977c0aebb0 2023-10-18T10:40:54.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.850+0000 7f97637fe640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f976806af60 0x7f976806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:54.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.850+0000 7f97637fe640 1 -- 172.21.15.125:0/428059730 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9774089eb0 con 0x7f977c0aebb0 2023-10-18T10:40:54.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.850+0000 7f977affd640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f976806af60 0x7f976806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:54.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.851+0000 7f977affd640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f976806af60 0x7f976806d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f976c036000 tx=0x7f976c035040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:54.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:54.854+0000 7f97637fe640 1 -- 172.21.15.125:0/428059730 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f977c0ae8d0 con 0x7f977c0aebb0 2023-10-18T10:40:55.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.020+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f977c06bf40 con 0x7f977c0aebb0 2023-10-18T10:40:55.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.021+0000 7f97637fe640 1 -- 172.21.15.125:0/428059730 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16838 (secure 0 0 0) 0x7f9774054c00 con 0x7f977c0aebb0 2023-10-18T10:40:55.025 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:55.026 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":97,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:53.749113+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:51.563590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:52.731689+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:55.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f976806af60 msgr2=0x7f976806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f976806af60 0x7f976806d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f976c036000 tx=0x7f976c035040 comp rx=0 tx=0).stop 2023-10-18T10:40:55.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f977c0aebb0 msgr2=0x7f977c1e2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f977c0aebb0 0x7f977c1e2890 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f977c13f3f0 tx=0x7f977400e980 comp rx=0 tx=0).stop 2023-10-18T10:40:55.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 shutdown_connections 2023-10-18T10:40:55.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f976806af60 0x7f976806d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f977c0aebb0 0x7f977c1e2890 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f977c138e10 0x7f977c1384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 --2- 172.21.15.125:0/428059730 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97640039a0 0x7f977c0a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 >> 172.21.15.125:0/428059730 conn(0x7f977c069930 msgr2=0x7f977c13b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:55.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 shutdown_connections 2023-10-18T10:40:55.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.023+0000 7f978139e640 1 -- 172.21.15.125:0/428059730 wait complete. 2023-10-18T10:40:55.036 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:40:55.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.139+0000 7faa24fc0640 1 -- 172.21.15.125:0/3667809916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa08003fd0 msgr2=0x7faa080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.139+0000 7faa24fc0640 1 --2- 172.21.15.125:0/3667809916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa08003fd0 0x7faa080047c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7faa14006fd0 tx=0x7faa080057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.139+0000 7faa1ed9d640 1 -- 172.21.15.125:0/3667809916 reap_dead start 2023-10-18T10:40:55.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.139+0000 7faa24fc0640 1 -- 172.21.15.125:0/3667809916 shutdown_connections 2023-10-18T10:40:55.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.139+0000 7faa24fc0640 1 -- 172.21.15.125:0/3667809916 >> 172.21.15.125:0/3667809916 conn(0x7faa20069930 msgr2=0x7faa20069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:55.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.139+0000 7faa24fc0640 1 -- 172.21.15.125:0/3667809916 shutdown_connections 2023-10-18T10:40:55.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.139+0000 7faa24fc0640 1 -- 172.21.15.125:0/3667809916 wait complete. 2023-10-18T10:40:55.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.141+0000 7faa24fc0640 1 Processor -- start 2023-10-18T10:40:55.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.141+0000 7faa24fc0640 1 -- start start 2023-10-18T10:40:55.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.141+0000 7faa24fc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa08003fd0 0x7faa2013dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:55.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.141+0000 7faa24fc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa2013abc0 0x7faa200ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:55.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.141+0000 7faa24fc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2013c3e0 0x7faa200ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:55.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.141+0000 7faa24fc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2013d8c0 con 0x7faa2013c3e0 2023-10-18T10:40:55.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.141+0000 7faa24fc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200ac010 con 0x7faa2013abc0 2023-10-18T10:40:55.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.141+0000 7faa24fc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2013c860 con 0x7faa08003fd0 2023-10-18T10:40:55.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2013c3e0 0x7faa200ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:55.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2013c3e0 0x7faa200ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56436/0 (socket says 172.21.15.125:56436) 2023-10-18T10:40:55.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 -- 172.21.15.125:0/2758389453 learned_addr learned my addr 172.21.15.125:0/2758389453 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:55.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1dd9b640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa2013abc0 0x7faa200ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:55.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1e59c640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa08003fd0 0x7faa2013dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:55.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 -- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa08003fd0 msgr2=0x7faa2013dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa08003fd0 0x7faa2013dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 -- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa2013abc0 msgr2=0x7faa200ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa2013abc0 0x7faa200ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 -- 172.21.15.125:0/2758389453 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2013cf20 con 0x7faa2013c3e0 2023-10-18T10:40:55.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1dd9b640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa2013abc0 0x7faa200ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:55.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1e59c640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa08003fd0 0x7faa2013dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:40:55.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa1ed9d640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2013c3e0 0x7faa200ca870 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7faa200ac660 tx=0x7faa18001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:55.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa0effd640 1 -- 172.21.15.125:0/2758389453 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18011660 con 0x7faa2013c3e0 2023-10-18T10:40:55.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa0effd640 1 -- 172.21.15.125:0/2758389453 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa180118b0 con 0x7faa2013c3e0 2023-10-18T10:40:55.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2013d200 con 0x7faa2013c3e0 2023-10-18T10:40:55.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa0effd640 1 -- 172.21.15.125:0/2758389453 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18011cc0 con 0x7faa2013c3e0 2023-10-18T10:40:55.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.142+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa200cadb0 con 0x7faa2013c3e0 2023-10-18T10:40:55.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.143+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa200ac150 con 0x7faa2013c3e0 2023-10-18T10:40:55.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.144+0000 7faa0effd640 1 -- 172.21.15.125:0/2758389453 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faa1801a400 con 0x7faa2013c3e0 2023-10-18T10:40:55.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.144+0000 7faa0effd640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7faa0006af90 0x7faa0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:55.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.144+0000 7faa0effd640 1 -- 172.21.15.125:0/2758389453 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faa1808a3a0 con 0x7faa2013c3e0 2023-10-18T10:40:55.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.145+0000 7faa1e59c640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7faa0006af90 0x7faa0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:55.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.145+0000 7faa1e59c640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7faa0006af90 0x7faa0006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa10005fd0 tx=0x7faa10009560 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:55.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.149+0000 7faa0effd640 1 -- 172.21.15.125:0/2758389453 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa200ac150 con 0x7faa2013c3e0 2023-10-18T10:40:55.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.349+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faa2013da00 con 0x7faa2013c3e0 2023-10-18T10:40:55.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.420+0000 7faa0effd640 1 -- 172.21.15.125:0/2758389453 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7faa2013da00 con 0x7faa2013c3e0 2023-10-18T10:40:55.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7faa0006af90 msgr2=0x7faa0006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7faa0006af90 0x7faa0006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa10005fd0 tx=0x7faa10009560 comp rx=0 tx=0).stop 2023-10-18T10:40:55.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2013c3e0 msgr2=0x7faa200ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2013c3e0 0x7faa200ca870 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7faa200ac660 tx=0x7faa18001630 comp rx=0 tx=0).stop 2023-10-18T10:40:55.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 shutdown_connections 2023-10-18T10:40:55.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7faa0006af90 0x7faa0006d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa2013abc0 0x7faa200ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa2013c3e0 0x7faa200ca870 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 --2- 172.21.15.125:0/2758389453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa08003fd0 0x7faa2013dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 >> 172.21.15.125:0/2758389453 conn(0x7faa20069930 msgr2=0x7faa200703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:55.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 shutdown_connections 2023-10-18T10:40:55.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.422+0000 7faa24fc0640 1 -- 172.21.15.125:0/2758389453 wait complete. 2023-10-18T10:40:55.436 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:40:55.436 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:40:55.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.540+0000 7fb536b5e640 1 -- 172.21.15.125:0/3313951728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530070af0 msgr2=0x7fb5300affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.540+0000 7fb536b5e640 1 --2- 172.21.15.125:0/3313951728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530070af0 0x7fb5300affe0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb5300cbd00 tx=0x7fb52802c1d0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.540+0000 7fb536b5e640 1 -- 172.21.15.125:0/3313951728 shutdown_connections 2023-10-18T10:40:55.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.540+0000 7fb536b5e640 1 --2- 172.21.15.125:0/3313951728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530070af0 0x7fb5300affe0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.540+0000 7fb536b5e640 1 --2- 172.21.15.125:0/3313951728 >> v2:172.21.15.139:3300/0 conn(0x7fb53013e520 0x7fb53006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.540+0000 7fb536b5e640 1 -- 172.21.15.125:0/3313951728 >> 172.21.15.125:0/3313951728 conn(0x7fb530069930 msgr2=0x7fb530069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:55.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.540+0000 7fb536b5e640 1 -- 172.21.15.125:0/3313951728 shutdown_connections 2023-10-18T10:40:55.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.540+0000 7fb536b5e640 1 -- 172.21.15.125:0/3313951728 wait complete. 2023-10-18T10:40:55.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.541+0000 7fb536b5e640 1 Processor -- start 2023-10-18T10:40:55.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.541+0000 7fb536b5e640 1 -- start start 2023-10-18T10:40:55.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb536b5e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb530070af0 0x7fb53013b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:55.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb536b5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53013e520 0x7fb5301e2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:55.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb536b5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530141690 0x7fb5300ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:55.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb536b5e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5301e2a50 con 0x7fb530141690 2023-10-18T10:40:55.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb536b5e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5301e2d30 con 0x7fb53013e520 2023-10-18T10:40:55.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb536b5e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5301e3010 con 0x7fb530070af0 2023-10-18T10:40:55.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb5350fb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530141690 0x7fb5300ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:55.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb5350fb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530141690 0x7fb5300ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56458/0 (socket says 172.21.15.125:56458) 2023-10-18T10:40:55.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.542+0000 7fb5350fb640 1 -- 172.21.15.125:0/4047984908 learned_addr learned my addr 172.21.15.125:0/4047984908 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:55.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb5348fa640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb530070af0 0x7fb53013b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:55.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb52ffff640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53013e520 0x7fb5301e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:55.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb5350fb640 1 -- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb530070af0 msgr2=0x7fb53013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb5350fb640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb530070af0 0x7fb53013b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb5350fb640 1 -- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53013e520 msgr2=0x7fb5301e2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:55.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb5350fb640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53013e520 0x7fb5301e2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:55.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb5350fb640 1 -- 172.21.15.125:0/4047984908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528009900 con 0x7fb530141690 2023-10-18T10:40:55.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb5348fa640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb530070af0 0x7fb53013b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:55.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb52ffff640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53013e520 0x7fb5301e2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:55.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb5350fb640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530141690 0x7fb5300ca870 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb528009d60 tx=0x7fb528035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:55.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb52d7fa640 1 -- 172.21.15.125:0/4047984908 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528034440 con 0x7fb530141690 2023-10-18T10:40:55.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb52d7fa640 1 -- 172.21.15.125:0/4047984908 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb528004540 con 0x7fb530141690 2023-10-18T10:40:55.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5301e32f0 con 0x7fb530141690 2023-10-18T10:40:55.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb52d7fa640 1 -- 172.21.15.125:0/4047984908 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528040450 con 0x7fb530141690 2023-10-18T10:40:55.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.543+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53013b650 con 0x7fb530141690 2023-10-18T10:40:55.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.544+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5301ef040 con 0x7fb530141690 2023-10-18T10:40:55.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.544+0000 7fb52d7fa640 1 -- 172.21.15.125:0/4047984908 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb52800a040 con 0x7fb530141690 2023-10-18T10:40:55.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.545+0000 7fb52d7fa640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb51406af60 0x7fb51406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:55.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.545+0000 7fb52d7fa640 1 -- 172.21.15.125:0/4047984908 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb5280acc20 con 0x7fb530141690 2023-10-18T10:40:55.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.545+0000 7fb5348fa640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb51406af60 0x7fb51406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:55.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.546+0000 7fb5348fa640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb51406af60 0x7fb51406d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb52400baa0 tx=0x7fb524005d90 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:55.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.549+0000 7fb52d7fa640 1 -- 172.21.15.125:0/4047984908 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5301ef040 con 0x7fb530141690 2023-10-18T10:40:55.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:55.749+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb5301ef040 con 0x7fb530141690 2023-10-18T10:40:56.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.434+0000 7fb52d7fa640 1 -- 172.21.15.125:0/4047984908 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb5301ef040 con 0x7fb530141690 2023-10-18T10:40:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb51406af60 msgr2=0x7fb51406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb51406af60 0x7fb51406d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb52400baa0 tx=0x7fb524005d90 comp rx=0 tx=0).stop 2023-10-18T10:40:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530141690 msgr2=0x7fb5300ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530141690 0x7fb5300ca870 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb528009d60 tx=0x7fb528035e30 comp rx=0 tx=0).stop 2023-10-18T10:40:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 shutdown_connections 2023-10-18T10:40:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fb51406af60 0x7fb51406d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53013e520 0x7fb5301e2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb530141690 0x7fb5300ca870 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 --2- 172.21.15.125:0/4047984908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb530070af0 0x7fb53013b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 >> 172.21.15.125:0/4047984908 conn(0x7fb530069930 msgr2=0x7fb53013d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:56.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 shutdown_connections 2023-10-18T10:40:56.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.436+0000 7fb536b5e640 1 -- 172.21.15.125:0/4047984908 wait complete. 2023-10-18T10:40:56.450 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:40:56.450 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:40:56.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.554+0000 7f7d49864640 1 -- 172.21.15.125:0/2567038172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d44070c30 msgr2=0x7f7d440b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.554+0000 7f7d49864640 1 --2- 172.21.15.125:0/2567038172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d44070c30 0x7f7d440b0bc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7d4406ac90 tx=0x7f7d3402bdb0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.554+0000 7f7d49864640 1 -- 172.21.15.125:0/2567038172 shutdown_connections 2023-10-18T10:40:56.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.554+0000 7f7d49864640 1 --2- 172.21.15.125:0/2567038172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d44070c30 0x7f7d440b0bc0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.554+0000 7f7d49864640 1 --2- 172.21.15.125:0/2567038172 >> v2:172.21.15.139:3300/0 conn(0x7f7d44158aa0 0x7f7d440cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.554+0000 7f7d49864640 1 -- 172.21.15.125:0/2567038172 >> 172.21.15.125:0/2567038172 conn(0x7f7d440aec40 msgr2=0x7f7d440af050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:56.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.554+0000 7f7d49864640 1 -- 172.21.15.125:0/2567038172 shutdown_connections 2023-10-18T10:40:56.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.554+0000 7f7d49864640 1 -- 172.21.15.125:0/2567038172 wait complete. 2023-10-18T10:40:56.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.555+0000 7f7d49864640 1 Processor -- start 2023-10-18T10:40:56.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d49864640 1 -- start start 2023-10-18T10:40:56.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d49864640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d44070c30 0x7f7d441ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:56.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d49864640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d440c9250 0x7f7d4415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:56.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d49864640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d44158aa0 0x7f7d4415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:56.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d49864640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d441ef700 con 0x7f7d440c9250 2023-10-18T10:40:56.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d49864640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4415a070 con 0x7f7d44070c30 2023-10-18T10:40:56.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d49864640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4415bfe0 con 0x7f7d44158aa0 2023-10-18T10:40:56.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d427fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d440c9250 0x7f7d4415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:56.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d427fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d440c9250 0x7f7d4415a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56484/0 (socket says 172.21.15.125:56484) 2023-10-18T10:40:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.556+0000 7f7d427fc640 1 -- 172.21.15.125:0/3665602748 learned_addr learned my addr 172.21.15.125:0/3665602748 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d437fe640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d44158aa0 0x7f7d4415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d42ffd640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d44070c30 0x7f7d441ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d427fc640 1 -- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d44158aa0 msgr2=0x7f7d4415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d427fc640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d44158aa0 0x7f7d4415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d427fc640 1 -- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d44070c30 msgr2=0x7f7d441ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d427fc640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d44070c30 0x7f7d441ef1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d427fc640 1 -- 172.21.15.125:0/3665602748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34009560 con 0x7f7d440c9250 2023-10-18T10:40:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d427fc640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d440c9250 0x7f7d4415a500 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c430 tx=0x7f7d3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d2bfff640 1 -- 172.21.15.125:0/3665602748 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38011790 con 0x7f7d440c9250 2023-10-18T10:40:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d2bfff640 1 -- 172.21.15.125:0/3665602748 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7d380119e0 con 0x7f7d440c9250 2023-10-18T10:40:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4415c2c0 con 0x7f7d440c9250 2023-10-18T10:40:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d2bfff640 1 -- 172.21.15.125:0/3665602748 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38011e30 con 0x7f7d440c9250 2023-10-18T10:40:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.557+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4415aa40 con 0x7f7d440c9250 2023-10-18T10:40:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.558+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4415a1e0 con 0x7f7d440c9250 2023-10-18T10:40:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.559+0000 7f7d2bfff640 1 -- 172.21.15.125:0/3665602748 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7d3801c460 con 0x7f7d440c9250 2023-10-18T10:40:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.559+0000 7f7d2bfff640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7d3006af60 0x7f7d3006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.559+0000 7f7d2bfff640 1 -- 172.21.15.125:0/3665602748 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7d38089ae0 con 0x7f7d440c9250 2023-10-18T10:40:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.559+0000 7f7d42ffd640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7d3006af60 0x7f7d3006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:56.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.560+0000 7f7d42ffd640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7d3006af60 0x7f7d3006d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d4414d470 tx=0x7f7d3c009400 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:56.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.563+0000 7f7d2bfff640 1 -- 172.21.15.125:0/3665602748 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d38054950 con 0x7f7d440c9250 2023-10-18T10:40:56.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.764+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d4415c4b0 con 0x7f7d440c9250 2023-10-18T10:40:56.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.764+0000 7f7d2bfff640 1 -- 172.21.15.125:0/3665602748 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f7d38057f70 con 0x7f7d440c9250 2023-10-18T10:40:56.767 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:56.767 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3531064821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3531064821},{"type":"v1","addr":"172.21.15.139:6837","nonce":3531064821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2850027191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2850027191},{"type":"v1","addr":"172.21.15.139:6836","nonce":2850027191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7578,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3670537575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3670537575},{"type":"v1","addr":"172.21.15.125:6834","nonce":3670537575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:53.749412+0000","modified":"2023-10-18T10:40:55.750936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7581":{"gid":7581,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1041197980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1041197980},{"type":"v1","addr":"172.21.15.125:6835","nonce":1041197980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T10:40:56.767 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 35 2023-10-18T10:40:56.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7d3006af60 msgr2=0x7f7d3006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7d3006af60 0x7f7d3006d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d4414d470 tx=0x7f7d3c009400 comp rx=0 tx=0).stop 2023-10-18T10:40:56.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d440c9250 msgr2=0x7f7d4415a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d440c9250 0x7f7d4415a500 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c430 tx=0x7f7d3800c8f0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 shutdown_connections 2023-10-18T10:40:56.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f7d3006af60 0x7f7d3006d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d44158aa0 0x7f7d4415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d440c9250 0x7f7d4415a500 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 --2- 172.21.15.125:0/3665602748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d44070c30 0x7f7d441ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 >> 172.21.15.125:0/3665602748 conn(0x7f7d440aec40 msgr2=0x7f7d4414da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:56.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.766+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 shutdown_connections 2023-10-18T10:40:56.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.767+0000 7f7d49864640 1 -- 172.21.15.125:0/3665602748 wait complete. 2023-10-18T10:40:56.779 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:40:56.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.879+0000 7f30b70c9640 1 -- 172.21.15.125:0/3864341534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00b0530 msgr2=0x7f30b00b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.879+0000 7f30b70c9640 1 --2- 172.21.15.125:0/3864341534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00b0530 0x7f30b00b09c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f30a8005fd0 tx=0x7f30a802c0e0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.879+0000 7f30b70c9640 1 -- 172.21.15.125:0/3864341534 shutdown_connections 2023-10-18T10:40:56.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.879+0000 7f30b70c9640 1 --2- 172.21.15.125:0/3864341534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00b0530 0x7f30b00b09c0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.879+0000 7f30b70c9640 1 --2- 172.21.15.125:0/3864341534 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b00afa20 0x7f30b0158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.879+0000 7f30b70c9640 1 -- 172.21.15.125:0/3864341534 >> 172.21.15.125:0/3864341534 conn(0x7f30b0069930 msgr2=0x7f30b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:56.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.879+0000 7f30b70c9640 1 -- 172.21.15.125:0/3864341534 shutdown_connections 2023-10-18T10:40:56.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.879+0000 7f30b70c9640 1 -- 172.21.15.125:0/3864341534 wait complete. 2023-10-18T10:40:56.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.880+0000 7f30b70c9640 1 Processor -- start 2023-10-18T10:40:56.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.880+0000 7f30b70c9640 1 -- start start 2023-10-18T10:40:56.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b70c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b00afa20 0x7f30b0135900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:56.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b70c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b00b0530 0x7f30b01e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:56.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b70c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00cbd20 0x7f30b014d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:56.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b70c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01ef010 con 0x7f30b00cbd20 2023-10-18T10:40:56.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b70c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b014f8f0 con 0x7f30b00b0530 2023-10-18T10:40:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b70c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b014fbd0 con 0x7f30b00afa20 2023-10-18T10:40:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00cbd20 0x7f30b014d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00cbd20 0x7f30b014d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56500/0 (socket says 172.21.15.125:56500) 2023-10-18T10:40:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 -- 172.21.15.125:0/1675109817 learned_addr learned my addr 172.21.15.125:0/1675109817 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30affff640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b00b0530 0x7f30b01e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 -- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b00afa20 msgr2=0x7f30b0135900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b4e65640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b00afa20 0x7f30b0135900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b00afa20 0x7f30b0135900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 -- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b00b0530 msgr2=0x7f30b01e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b00b0530 0x7f30b01e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:56.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 -- 172.21.15.125:0/1675109817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a8009840 con 0x7f30b00cbd20 2023-10-18T10:40:56.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30affff640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b00b0530 0x7f30b01e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:56.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b4e65640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b00afa20 0x7f30b0135900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:56.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.881+0000 7f30b5666640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00cbd20 0x7f30b014d7a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f30a8005fd0 tx=0x7f30a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.882+0000 7f30ad7fa640 1 -- 172.21.15.125:0/1675109817 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80046f0 con 0x7f30b00cbd20 2023-10-18T10:40:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.882+0000 7f30ad7fa640 1 -- 172.21.15.125:0/1675109817 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f30a80345a0 con 0x7f30b00cbd20 2023-10-18T10:40:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.882+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b014feb0 con 0x7f30b00cbd20 2023-10-18T10:40:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.882+0000 7f30ad7fa640 1 -- 172.21.15.125:0/1675109817 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80349b0 con 0x7f30b00cbd20 2023-10-18T10:40:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.882+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b0150470 con 0x7f30b00cbd20 2023-10-18T10:40:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.883+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b0157b40 con 0x7f30b00cbd20 2023-10-18T10:40:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.883+0000 7f30ad7fa640 1 -- 172.21.15.125:0/1675109817 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f30a8034b10 con 0x7f30b00cbd20 2023-10-18T10:40:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.883+0000 7f30ad7fa640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f309c06af90 0x7f309c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.884+0000 7f30ad7fa640 1 -- 172.21.15.125:0/1675109817 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f30a80acb30 con 0x7f30b00cbd20 2023-10-18T10:40:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.884+0000 7f30b4e65640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f309c06af90 0x7f309c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:56.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.884+0000 7f30b4e65640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f309c06af90 0x7f309c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30a0000f60 tx=0x7f30a0009040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:56.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:56.888+0000 7f30ad7fa640 1 -- 172.21.15.125:0/1675109817 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30a8077910 con 0x7f30b00cbd20 2023-10-18T10:40:57.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.055+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30b006a270 con 0x7f30b00cbd20 2023-10-18T10:40:57.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.056+0000 7f30ad7fa640 1 -- 172.21.15.125:0/1675109817 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16825 (secure 0 0 0) 0x7f30a807af30 con 0x7f30b00cbd20 2023-10-18T10:40:57.058 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:57.058 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":98,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:40:55.420473+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:51.563590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:52.731689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:40:57.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f309c06af90 msgr2=0x7f309c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f309c06af90 0x7f309c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30a0000f60 tx=0x7f30a0009040 comp rx=0 tx=0).stop 2023-10-18T10:40:57.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00cbd20 msgr2=0x7f30b014d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00cbd20 0x7f30b014d7a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f30a8005fd0 tx=0x7f30a80042f0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 shutdown_connections 2023-10-18T10:40:57.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f309c06af90 0x7f309c06d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30b00b0530 0x7f30b01e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30b00cbd20 0x7f30b014d7a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 --2- 172.21.15.125:0/1675109817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30b00afa20 0x7f30b0135900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 >> 172.21.15.125:0/1675109817 conn(0x7f30b0069930 msgr2=0x7f30b00ad640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:57.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 shutdown_connections 2023-10-18T10:40:57.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.058+0000 7f30b70c9640 1 -- 172.21.15.125:0/1675109817 wait complete. 2023-10-18T10:40:57.071 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:40:57.071 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:57.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 -- 172.21.15.125:0/2213713555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d815b2a0 msgr2=0x7f82d80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2213713555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d815b2a0 0x7f82d80cf720 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f82d808a810 tx=0x7f82cc02bd10 comp rx=0 tx=0).stop 2023-10-18T10:40:57.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 -- 172.21.15.125:0/2213713555 shutdown_connections 2023-10-18T10:40:57.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2213713555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d815b2a0 0x7f82d80cf720 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2213713555 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82d80ceb40 0x7f82d80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2213713555 >> v2:172.21.15.139:3300/0 conn(0x7f82d80cf340 0x7f82d80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 -- 172.21.15.125:0/2213713555 >> 172.21.15.125:0/2213713555 conn(0x7f82d80d0d60 msgr2=0x7f82d80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:57.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 -- 172.21.15.125:0/2213713555 shutdown_connections 2023-10-18T10:40:57.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.171+0000 7f82dd64f640 1 -- 172.21.15.125:0/2213713555 wait complete. 2023-10-18T10:40:57.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.172+0000 7f82dd64f640 1 Processor -- start 2023-10-18T10:40:57.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.172+0000 7f82dd64f640 1 -- start start 2023-10-18T10:40:57.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82dd64f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82d80ceb40 0x7f82d8158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82dd64f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d80cf340 0x7f82d80b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82dd64f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82d815b2a0 0x7f82d80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82dd64f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80c7e30 con 0x7f82d80cf340 2023-10-18T10:40:57.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82dd64f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80b3ea0 con 0x7f82d80ceb40 2023-10-18T10:40:57.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82dd64f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80b4180 con 0x7f82d815b2a0 2023-10-18T10:40:57.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d80cf340 0x7f82d80b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d80cf340 0x7f82d80b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56520/0 (socket says 172.21.15.125:56520) 2023-10-18T10:40:57.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d67fc640 1 -- 172.21.15.125:0/2995337891 learned_addr learned my addr 172.21.15.125:0/2995337891 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:57.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d67fc640 1 -- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82d815b2a0 msgr2=0x7f82d80cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d67fc640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82d815b2a0 0x7f82d80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d67fc640 1 -- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82d80ceb40 msgr2=0x7f82d8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d6ffd640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82d80ceb40 0x7f82d8158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d67fc640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82d80ceb40 0x7f82d8158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.173+0000 7f82d67fc640 1 -- 172.21.15.125:0/2995337891 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc009560 con 0x7f82d80cf340 2023-10-18T10:40:57.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.174+0000 7f82d67fc640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d80cf340 0x7f82d80b64f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009850 tx=0x7f82cc02da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:57.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.174+0000 7f82bf7fe640 1 -- 172.21.15.125:0/2995337891 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc003800 con 0x7f82d80cf340 2023-10-18T10:40:57.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.174+0000 7f82bf7fe640 1 -- 172.21.15.125:0/2995337891 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82cc003a50 con 0x7f82d80cf340 2023-10-18T10:40:57.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.174+0000 7f82bf7fe640 1 -- 172.21.15.125:0/2995337891 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc003e20 con 0x7f82d80cf340 2023-10-18T10:40:57.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.174+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d80b4460 con 0x7f82d80cf340 2023-10-18T10:40:57.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.174+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d80b4a20 con 0x7f82d80cf340 2023-10-18T10:40:57.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.175+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d80690e0 con 0x7f82d80cf340 2023-10-18T10:40:57.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.175+0000 7f82bf7fe640 1 -- 172.21.15.125:0/2995337891 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f82cc03e450 con 0x7f82d80cf340 2023-10-18T10:40:57.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.176+0000 7f82bf7fe640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f82c406af60 0x7f82c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.176+0000 7f82bf7fe640 1 -- 172.21.15.125:0/2995337891 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f82cc0ac400 con 0x7f82d80cf340 2023-10-18T10:40:57.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.176+0000 7f82d6ffd640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f82c406af60 0x7f82c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.176+0000 7f82d6ffd640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f82c406af60 0x7f82c406d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005c40 tx=0x7f82c8005bd0 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:57.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.180+0000 7f82bf7fe640 1 -- 172.21.15.125:0/2995337891 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82cc077260 con 0x7f82d80cf340 2023-10-18T10:40:57.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.343+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 --> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82d80cc170 con 0x7f82c406af60 2023-10-18T10:40:57.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.351+0000 7f82bf7fe640 1 -- 172.21.15.125:0/2995337891 <== mgr.5972 v2:172.21.15.139:6832/143413153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284363 (secure 0 0 0) 0x7f82c8009a50 con 0x7f82c406af60 2023-10-18T10:40:57.356 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:57.357 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:57.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f82c406af60 msgr2=0x7f82c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f82c406af60 0x7f82c406d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005c40 tx=0x7f82c8005bd0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d80cf340 msgr2=0x7f82d80b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d80cf340 0x7f82d80b64f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009850 tx=0x7f82cc02da40 comp rx=0 tx=0).stop 2023-10-18T10:40:57.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 shutdown_connections 2023-10-18T10:40:57.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f82c406af60 0x7f82c406d420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82d815b2a0 0x7f82d80cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82d80cf340 0x7f82d80b64f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 --2- 172.21.15.125:0/2995337891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82d80ceb40 0x7f82d8158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 >> 172.21.15.125:0/2995337891 conn(0x7f82d80d0d60 msgr2=0x7f82d81e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:57.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 shutdown_connections 2023-10-18T10:40:57.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.354+0000 7f82dd64f640 1 -- 172.21.15.125:0/2995337891 wait complete. 2023-10-18T10:40:57.366 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:40:56.162612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":296,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":312608,"kb_used_data":10400,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754662112,"statfs":{"total":773094113280,"available":772774002688,"internally_reserved":0,"allocated":10649600,"data_stored":5879653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001032"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144511+0000","last_change":"2023-10-18T10:40:54.435140+0000","last_active":"2023-10-18T10:40:56.144511+0000","last_peered":"2023-10-18T10:40:56.144511+0000","last_clean":"2023-10-18T10:40:56.144511+0000","last_became_active":"2023-10-18T10:40:54.434657+0000","last_became_peered":"2023-10-18T10:40:54.434657+0000","last_unstale":"2023-10-18T10:40:56.144511+0000","last_undegraded":"2023-10-18T10:40:56.144511+0000","last_fullsized":"2023-10-18T10:40:56.144511+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.896314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500661+0000","last_change":"2023-10-18T10:40:54.428671+0000","last_active":"2023-10-18T10:40:55.500661+0000","last_peered":"2023-10-18T10:40:55.500661+0000","last_clean":"2023-10-18T10:40:55.500661+0000","last_became_active":"2023-10-18T10:40:54.428396+0000","last_became_peered":"2023-10-18T10:40:54.428396+0000","last_unstale":"2023-10-18T10:40:55.500661+0000","last_undegraded":"2023-10-18T10:40:55.500661+0000","last_fullsized":"2023-10-18T10:40:55.500661+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:39.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018418+0000","last_change":"2023-10-18T10:40:54.433351+0000","last_active":"2023-10-18T10:40:56.018418+0000","last_peered":"2023-10-18T10:40:56.018418+0000","last_clean":"2023-10-18T10:40:56.018418+0000","last_became_active":"2023-10-18T10:40:54.430314+0000","last_became_peered":"2023-10-18T10:40:54.430314+0000","last_unstale":"2023-10-18T10:40:56.018418+0000","last_undegraded":"2023-10-18T10:40:56.018418+0000","last_fullsized":"2023-10-18T10:40:56.018418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:48.808404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429115+0000","last_change":"2023-10-18T10:40:54.431510+0000","last_active":"2023-10-18T10:40:55.429115+0000","last_peered":"2023-10-18T10:40:55.429115+0000","last_clean":"2023-10-18T10:40:55.429115+0000","last_became_active":"2023-10-18T10:40:54.431184+0000","last_became_peered":"2023-10-18T10:40:54.431184+0000","last_unstale":"2023-10-18T10:40:55.429115+0000","last_undegraded":"2023-10-18T10:40:55.429115+0000","last_fullsized":"2023-10-18T10:40:55.429115+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:33.893983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144553+0000","last_change":"2023-10-18T10:40:54.434704+0000","last_active":"2023-10-18T10:40:56.144553+0000","last_peered":"2023-10-18T10:40:56.144553+0000","last_clean":"2023-10-18T10:40:56.144553+0000","last_became_active":"2023-10-18T10:40:54.434388+0000","last_became_peered":"2023-10-18T10:40:54.434388+0000","last_unstale":"2023-10-18T10:40:56.144553+0000","last_undegraded":"2023-10-18T10:40:56.144553+0000","last_fullsized":"2023-10-18T10:40:56.144553+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:54.346618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:54.427003+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:54.426743+0000","last_became_peered":"2023-10-18T10:40:54.426743+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:58.361349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500867+0000","last_change":"2023-10-18T10:40:54.433837+0000","last_active":"2023-10-18T10:40:55.500867+0000","last_peered":"2023-10-18T10:40:55.500867+0000","last_clean":"2023-10-18T10:40:55.500867+0000","last_became_active":"2023-10-18T10:40:54.433558+0000","last_became_peered":"2023-10-18T10:40:54.433558+0000","last_unstale":"2023-10-18T10:40:55.500867+0000","last_undegraded":"2023-10-18T10:40:55.500867+0000","last_fullsized":"2023-10-18T10:40:55.500867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:04.849763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144518+0000","last_change":"2023-10-18T10:40:54.429847+0000","last_active":"2023-10-18T10:40:56.144518+0000","last_peered":"2023-10-18T10:40:56.144518+0000","last_clean":"2023-10-18T10:40:56.144518+0000","last_became_active":"2023-10-18T10:40:54.429629+0000","last_became_peered":"2023-10-18T10:40:54.429629+0000","last_unstale":"2023-10-18T10:40:56.144518+0000","last_undegraded":"2023-10-18T10:40:56.144518+0000","last_fullsized":"2023-10-18T10:40:56.144518+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:29.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500804+0000","last_change":"2023-10-18T10:40:54.427881+0000","last_active":"2023-10-18T10:40:55.500804+0000","last_peered":"2023-10-18T10:40:55.500804+0000","last_clean":"2023-10-18T10:40:55.500804+0000","last_became_active":"2023-10-18T10:40:54.427423+0000","last_became_peered":"2023-10-18T10:40:54.427423+0000","last_unstale":"2023-10-18T10:40:55.500804+0000","last_undegraded":"2023-10-18T10:40:55.500804+0000","last_fullsized":"2023-10-18T10:40:55.500804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:07.682253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018526+0000","last_change":"2023-10-18T10:40:54.426415+0000","last_active":"2023-10-18T10:40:56.018526+0000","last_peered":"2023-10-18T10:40:56.018526+0000","last_clean":"2023-10-18T10:40:56.018526+0000","last_became_active":"2023-10-18T10:40:54.426129+0000","last_became_peered":"2023-10-18T10:40:54.426129+0000","last_unstale":"2023-10-18T10:40:56.018526+0000","last_undegraded":"2023-10-18T10:40:56.018526+0000","last_fullsized":"2023-10-18T10:40:56.018526+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:33:27.749058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429139+0000","last_change":"2023-10-18T10:40:54.430848+0000","last_active":"2023-10-18T10:40:55.429139+0000","last_peered":"2023-10-18T10:40:55.429139+0000","last_clean":"2023-10-18T10:40:55.429139+0000","last_became_active":"2023-10-18T10:40:54.430562+0000","last_became_peered":"2023-10-18T10:40:54.430562+0000","last_unstale":"2023-10-18T10:40:55.429139+0000","last_undegraded":"2023-10-18T10:40:55.429139+0000","last_fullsized":"2023-10-18T10:40:55.429139+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:05.564173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144611+0000","last_change":"2023-10-18T10:40:54.421817+0000","last_active":"2023-10-18T10:40:56.144611+0000","last_peered":"2023-10-18T10:40:56.144611+0000","last_clean":"2023-10-18T10:40:56.144611+0000","last_became_active":"2023-10-18T10:40:54.421613+0000","last_became_peered":"2023-10-18T10:40:54.421613+0000","last_unstale":"2023-10-18T10:40:56.144611+0000","last_undegraded":"2023-10-18T10:40:56.144611+0000","last_fullsized":"2023-10-18T10:40:56.144611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:36.706863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:54.435303+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:54.434740+0000","last_became_peered":"2023-10-18T10:40:54.434740+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.386244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429236+0000","last_change":"2023-10-18T10:40:54.432868+0000","last_active":"2023-10-18T10:40:55.429236+0000","last_peered":"2023-10-18T10:40:55.429236+0000","last_clean":"2023-10-18T10:40:55.429236+0000","last_became_active":"2023-10-18T10:40:54.432637+0000","last_became_peered":"2023-10-18T10:40:54.432637+0000","last_unstale":"2023-10-18T10:40:55.429236+0000","last_undegraded":"2023-10-18T10:40:55.429236+0000","last_fullsized":"2023-10-18T10:40:55.429236+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:46.951599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018489+0000","last_change":"2023-10-18T10:40:54.430205+0000","last_active":"2023-10-18T10:40:56.018489+0000","last_peered":"2023-10-18T10:40:56.018489+0000","last_clean":"2023-10-18T10:40:56.018489+0000","last_became_active":"2023-10-18T10:40:54.429898+0000","last_became_peered":"2023-10-18T10:40:54.429898+0000","last_unstale":"2023-10-18T10:40:56.018489+0000","last_undegraded":"2023-10-18T10:40:56.018489+0000","last_fullsized":"2023-10-18T10:40:56.018489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:48.646925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018318+0000","last_change":"2023-10-18T10:40:54.428004+0000","last_active":"2023-10-18T10:40:56.018318+0000","last_peered":"2023-10-18T10:40:56.018318+0000","last_clean":"2023-10-18T10:40:56.018318+0000","last_became_active":"2023-10-18T10:40:54.427533+0000","last_became_peered":"2023-10-18T10:40:54.427533+0000","last_unstale":"2023-10-18T10:40:56.018318+0000","last_undegraded":"2023-10-18T10:40:56.018318+0000","last_fullsized":"2023-10-18T10:40:56.018318+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:27:07.146678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143943+0000","last_change":"2023-10-18T10:40:54.424777+0000","last_active":"2023-10-18T10:40:56.143943+0000","last_peered":"2023-10-18T10:40:56.143943+0000","last_clean":"2023-10-18T10:40:56.143943+0000","last_became_active":"2023-10-18T10:40:54.424544+0000","last_became_peered":"2023-10-18T10:40:54.424544+0000","last_unstale":"2023-10-18T10:40:56.143943+0000","last_undegraded":"2023-10-18T10:40:56.143943+0000","last_fullsized":"2023-10-18T10:40:56.143943+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144475+0000","last_change":"2023-10-18T10:40:54.425355+0000","last_active":"2023-10-18T10:40:56.144475+0000","last_peered":"2023-10-18T10:40:56.144475+0000","last_clean":"2023-10-18T10:40:56.144475+0000","last_became_active":"2023-10-18T10:40:54.425123+0000","last_became_peered":"2023-10-18T10:40:54.425123+0000","last_unstale":"2023-10-18T10:40:56.144475+0000","last_undegraded":"2023-10-18T10:40:56.144475+0000","last_fullsized":"2023-10-18T10:40:56.144475+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:24.991706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500878+0000","last_change":"2023-10-18T10:40:54.426993+0000","last_active":"2023-10-18T10:40:55.500878+0000","last_peered":"2023-10-18T10:40:55.500878+0000","last_clean":"2023-10-18T10:40:55.500878+0000","last_became_active":"2023-10-18T10:40:54.426702+0000","last_became_peered":"2023-10-18T10:40:54.426702+0000","last_unstale":"2023-10-18T10:40:55.500878+0000","last_undegraded":"2023-10-18T10:40:55.500878+0000","last_fullsized":"2023-10-18T10:40:55.500878+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:57.667741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018550+0000","last_change":"2023-10-18T10:40:54.432406+0000","last_active":"2023-10-18T10:40:56.018550+0000","last_peered":"2023-10-18T10:40:56.018550+0000","last_clean":"2023-10-18T10:40:56.018550+0000","last_became_active":"2023-10-18T10:40:54.432113+0000","last_became_peered":"2023-10-18T10:40:54.432113+0000","last_unstale":"2023-10-18T10:40:56.018550+0000","last_undegraded":"2023-10-18T10:40:56.018550+0000","last_fullsized":"2023-10-18T10:40:56.018550+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:24:16.099369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144250+0000","last_change":"2023-10-18T10:40:54.431844+0000","last_active":"2023-10-18T10:40:56.144250+0000","last_peered":"2023-10-18T10:40:56.144250+0000","last_clean":"2023-10-18T10:40:56.144250+0000","last_became_active":"2023-10-18T10:40:54.431447+0000","last_became_peered":"2023-10-18T10:40:54.431447+0000","last_unstale":"2023-10-18T10:40:56.144250+0000","last_undegraded":"2023-10-18T10:40:56.144250+0000","last_fullsized":"2023-10-18T10:40:56.144250+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:30.965000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"97'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:53.417631+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:53.417381+0000","last_became_peered":"2023-10-18T10:40:53.417381+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:40.271808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144091+0000","last_change":"2023-10-18T10:40:54.420096+0000","last_active":"2023-10-18T10:40:56.144091+0000","last_peered":"2023-10-18T10:40:56.144091+0000","last_clean":"2023-10-18T10:40:56.144091+0000","last_became_active":"2023-10-18T10:40:54.419838+0000","last_became_peered":"2023-10-18T10:40:54.419838+0000","last_unstale":"2023-10-18T10:40:56.144091+0000","last_undegraded":"2023-10-18T10:40:56.144091+0000","last_fullsized":"2023-10-18T10:40:56.144091+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:17.947505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144791+0000","last_change":"2023-10-18T10:40:53.423111+0000","last_active":"2023-10-18T10:40:56.144791+0000","last_peered":"2023-10-18T10:40:56.144791+0000","last_clean":"2023-10-18T10:40:56.144791+0000","last_became_active":"2023-10-18T10:40:53.422848+0000","last_became_peered":"2023-10-18T10:40:53.422848+0000","last_unstale":"2023-10-18T10:40:56.144791+0000","last_undegraded":"2023-10-18T10:40:56.144791+0000","last_fullsized":"2023-10-18T10:40:56.144791+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:19.845504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144374+0000","last_change":"2023-10-18T10:40:54.435357+0000","last_active":"2023-10-18T10:40:56.144374+0000","last_peered":"2023-10-18T10:40:56.144374+0000","last_clean":"2023-10-18T10:40:56.144374+0000","last_became_active":"2023-10-18T10:40:54.435102+0000","last_became_peered":"2023-10-18T10:40:54.435102+0000","last_unstale":"2023-10-18T10:40:56.144374+0000","last_undegraded":"2023-10-18T10:40:56.144374+0000","last_fullsized":"2023-10-18T10:40:56.144374+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:28:23.536390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144627+0000","last_change":"2023-10-18T10:40:54.428864+0000","last_active":"2023-10-18T10:40:56.144627+0000","last_peered":"2023-10-18T10:40:56.144627+0000","last_clean":"2023-10-18T10:40:56.144627+0000","last_became_active":"2023-10-18T10:40:54.428412+0000","last_became_peered":"2023-10-18T10:40:54.428412+0000","last_unstale":"2023-10-18T10:40:56.144627+0000","last_undegraded":"2023-10-18T10:40:56.144627+0000","last_fullsized":"2023-10-18T10:40:56.144627+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:15.038645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500519+0000","last_change":"2023-10-18T10:40:53.420317+0000","last_active":"2023-10-18T10:40:55.500519+0000","last_peered":"2023-10-18T10:40:55.500519+0000","last_clean":"2023-10-18T10:40:55.500519+0000","last_became_active":"2023-10-18T10:40:53.420046+0000","last_became_peered":"2023-10-18T10:40:53.420046+0000","last_unstale":"2023-10-18T10:40:55.500519+0000","last_undegraded":"2023-10-18T10:40:55.500519+0000","last_fullsized":"2023-10-18T10:40:55.500519+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:36.623092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144181+0000","last_change":"2023-10-18T10:40:53.430760+0000","last_active":"2023-10-18T10:40:56.144181+0000","last_peered":"2023-10-18T10:40:56.144181+0000","last_clean":"2023-10-18T10:40:56.144181+0000","last_became_active":"2023-10-18T10:40:53.430469+0000","last_became_peered":"2023-10-18T10:40:53.430469+0000","last_unstale":"2023-10-18T10:40:56.144181+0000","last_undegraded":"2023-10-18T10:40:56.144181+0000","last_fullsized":"2023-10-18T10:40:56.144181+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:45.943807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144991+0000","last_change":"2023-10-18T10:40:54.435078+0000","last_active":"2023-10-18T10:40:56.144991+0000","last_peered":"2023-10-18T10:40:56.144991+0000","last_clean":"2023-10-18T10:40:56.144991+0000","last_became_active":"2023-10-18T10:40:54.434517+0000","last_became_peered":"2023-10-18T10:40:54.434517+0000","last_unstale":"2023-10-18T10:40:56.144991+0000","last_undegraded":"2023-10-18T10:40:56.144991+0000","last_fullsized":"2023-10-18T10:40:56.144991+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:40.219033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:53.415948+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:53.415668+0000","last_became_peered":"2023-10-18T10:40:53.415668+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:12.409497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144074+0000","last_change":"2023-10-18T10:40:54.426165+0000","last_active":"2023-10-18T10:40:56.144074+0000","last_peered":"2023-10-18T10:40:56.144074+0000","last_clean":"2023-10-18T10:40:56.144074+0000","last_became_active":"2023-10-18T10:40:54.425916+0000","last_became_peered":"2023-10-18T10:40:54.425916+0000","last_unstale":"2023-10-18T10:40:56.144074+0000","last_undegraded":"2023-10-18T10:40:56.144074+0000","last_fullsized":"2023-10-18T10:40:56.144074+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500662+0000","last_change":"2023-10-18T10:40:54.428238+0000","last_active":"2023-10-18T10:40:55.500662+0000","last_peered":"2023-10-18T10:40:55.500662+0000","last_clean":"2023-10-18T10:40:55.500662+0000","last_became_active":"2023-10-18T10:40:54.427404+0000","last_became_peered":"2023-10-18T10:40:54.427404+0000","last_unstale":"2023-10-18T10:40:55.500662+0000","last_undegraded":"2023-10-18T10:40:55.500662+0000","last_fullsized":"2023-10-18T10:40:55.500662+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:06.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500959+0000","last_change":"2023-10-18T10:40:53.423540+0000","last_active":"2023-10-18T10:40:55.500959+0000","last_peered":"2023-10-18T10:40:55.500959+0000","last_clean":"2023-10-18T10:40:55.500959+0000","last_became_active":"2023-10-18T10:40:53.423278+0000","last_became_peered":"2023-10-18T10:40:53.423278+0000","last_unstale":"2023-10-18T10:40:55.500959+0000","last_undegraded":"2023-10-18T10:40:55.500959+0000","last_fullsized":"2023-10-18T10:40:55.500959+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:15.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429349+0000","last_change":"2023-10-18T10:40:54.423285+0000","last_active":"2023-10-18T10:40:55.429349+0000","last_peered":"2023-10-18T10:40:55.429349+0000","last_clean":"2023-10-18T10:40:55.429349+0000","last_became_active":"2023-10-18T10:40:54.423013+0000","last_became_peered":"2023-10-18T10:40:54.423013+0000","last_unstale":"2023-10-18T10:40:55.429349+0000","last_undegraded":"2023-10-18T10:40:55.429349+0000","last_fullsized":"2023-10-18T10:40:55.429349+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"97'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018642+0000","last_change":"2023-10-18T10:40:53.424070+0000","last_active":"2023-10-18T10:40:56.018642+0000","last_peered":"2023-10-18T10:40:56.018642+0000","last_clean":"2023-10-18T10:40:56.018642+0000","last_became_active":"2023-10-18T10:40:53.423799+0000","last_became_peered":"2023-10-18T10:40:53.423799+0000","last_unstale":"2023-10-18T10:40:56.018642+0000","last_undegraded":"2023-10-18T10:40:56.018642+0000","last_fullsized":"2023-10-18T10:40:56.018642+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:09.891958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"97'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144832+0000","last_change":"2023-10-18T10:40:53.422980+0000","last_active":"2023-10-18T10:40:56.144832+0000","last_peered":"2023-10-18T10:40:56.144832+0000","last_clean":"2023-10-18T10:40:56.144832+0000","last_became_active":"2023-10-18T10:40:53.422696+0000","last_became_peered":"2023-10-18T10:40:53.422696+0000","last_unstale":"2023-10-18T10:40:56.144832+0000","last_undegraded":"2023-10-18T10:40:56.144832+0000","last_fullsized":"2023-10-18T10:40:56.144832+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:20.506905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500893+0000","last_change":"2023-10-18T10:40:54.423164+0000","last_active":"2023-10-18T10:40:55.500893+0000","last_peered":"2023-10-18T10:40:55.500893+0000","last_clean":"2023-10-18T10:40:55.500893+0000","last_became_active":"2023-10-18T10:40:54.422844+0000","last_became_peered":"2023-10-18T10:40:54.422844+0000","last_unstale":"2023-10-18T10:40:55.500893+0000","last_undegraded":"2023-10-18T10:40:55.500893+0000","last_fullsized":"2023-10-18T10:40:55.500893+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:50.691881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500565+0000","last_change":"2023-10-18T10:40:53.420785+0000","last_active":"2023-10-18T10:40:55.500565+0000","last_peered":"2023-10-18T10:40:55.500565+0000","last_clean":"2023-10-18T10:40:55.500565+0000","last_became_active":"2023-10-18T10:40:53.420530+0000","last_became_peered":"2023-10-18T10:40:53.420530+0000","last_unstale":"2023-10-18T10:40:55.500565+0000","last_undegraded":"2023-10-18T10:40:55.500565+0000","last_fullsized":"2023-10-18T10:40:55.500565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:44.748900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144877+0000","last_change":"2023-10-18T10:40:54.430742+0000","last_active":"2023-10-18T10:40:56.144877+0000","last_peered":"2023-10-18T10:40:56.144877+0000","last_clean":"2023-10-18T10:40:56.144877+0000","last_became_active":"2023-10-18T10:40:54.430455+0000","last_became_peered":"2023-10-18T10:40:54.430455+0000","last_unstale":"2023-10-18T10:40:56.144877+0000","last_undegraded":"2023-10-18T10:40:56.144877+0000","last_fullsized":"2023-10-18T10:40:56.144877+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:39.892546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144941+0000","last_change":"2023-10-18T10:40:54.430021+0000","last_active":"2023-10-18T10:40:56.144941+0000","last_peered":"2023-10-18T10:40:56.144941+0000","last_clean":"2023-10-18T10:40:56.144941+0000","last_became_active":"2023-10-18T10:40:54.429684+0000","last_became_peered":"2023-10-18T10:40:54.429684+0000","last_unstale":"2023-10-18T10:40:56.144941+0000","last_undegraded":"2023-10-18T10:40:56.144941+0000","last_fullsized":"2023-10-18T10:40:56.144941+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:03.648375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018334+0000","last_change":"2023-10-18T10:40:53.425774+0000","last_active":"2023-10-18T10:40:56.018334+0000","last_peered":"2023-10-18T10:40:56.018334+0000","last_clean":"2023-10-18T10:40:56.018334+0000","last_became_active":"2023-10-18T10:40:53.425518+0000","last_became_peered":"2023-10-18T10:40:53.425518+0000","last_unstale":"2023-10-18T10:40:56.018334+0000","last_undegraded":"2023-10-18T10:40:56.018334+0000","last_fullsized":"2023-10-18T10:40:56.018334+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:30.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145025+0000","last_change":"2023-10-18T10:40:54.432786+0000","last_active":"2023-10-18T10:40:56.145025+0000","last_peered":"2023-10-18T10:40:56.145025+0000","last_clean":"2023-10-18T10:40:56.145025+0000","last_became_active":"2023-10-18T10:40:54.432538+0000","last_became_peered":"2023-10-18T10:40:54.432538+0000","last_unstale":"2023-10-18T10:40:56.145025+0000","last_undegraded":"2023-10-18T10:40:56.145025+0000","last_fullsized":"2023-10-18T10:40:56.145025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:52:39.855703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"97'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144160+0000","last_change":"2023-10-18T10:40:53.426701+0000","last_active":"2023-10-18T10:40:56.144160+0000","last_peered":"2023-10-18T10:40:56.144160+0000","last_clean":"2023-10-18T10:40:56.144160+0000","last_became_active":"2023-10-18T10:40:53.426431+0000","last_became_peered":"2023-10-18T10:40:53.426431+0000","last_unstale":"2023-10-18T10:40:56.144160+0000","last_undegraded":"2023-10-18T10:40:56.144160+0000","last_fullsized":"2023-10-18T10:40:56.144160+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:31.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429351+0000","last_change":"2023-10-18T10:40:54.434075+0000","last_active":"2023-10-18T10:40:55.429351+0000","last_peered":"2023-10-18T10:40:55.429351+0000","last_clean":"2023-10-18T10:40:55.429351+0000","last_became_active":"2023-10-18T10:40:54.433835+0000","last_became_peered":"2023-10-18T10:40:54.433835+0000","last_unstale":"2023-10-18T10:40:55.429351+0000","last_undegraded":"2023-10-18T10:40:55.429351+0000","last_fullsized":"2023-10-18T10:40:55.429351+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018402+0000","last_change":"2023-10-18T10:40:54.435840+0000","last_active":"2023-10-18T10:40:56.018402+0000","last_peered":"2023-10-18T10:40:56.018402+0000","last_clean":"2023-10-18T10:40:56.018402+0000","last_became_active":"2023-10-18T10:40:54.435594+0000","last_became_peered":"2023-10-18T10:40:54.435594+0000","last_unstale":"2023-10-18T10:40:56.018402+0000","last_undegraded":"2023-10-18T10:40:56.018402+0000","last_fullsized":"2023-10-18T10:40:56.018402+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:16:01.996675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018391+0000","last_change":"2023-10-18T10:40:54.427794+0000","last_active":"2023-10-18T10:40:56.018391+0000","last_peered":"2023-10-18T10:40:56.018391+0000","last_clean":"2023-10-18T10:40:56.018391+0000","last_became_active":"2023-10-18T10:40:54.427255+0000","last_became_peered":"2023-10-18T10:40:54.427255+0000","last_unstale":"2023-10-18T10:40:56.018391+0000","last_undegraded":"2023-10-18T10:40:56.018391+0000","last_fullsized":"2023-10-18T10:40:56.018391+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:08.181951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018673+0000","last_change":"2023-10-18T10:40:53.416910+0000","last_active":"2023-10-18T10:40:56.018673+0000","last_peered":"2023-10-18T10:40:56.018673+0000","last_clean":"2023-10-18T10:40:56.018673+0000","last_became_active":"2023-10-18T10:40:53.416630+0000","last_became_peered":"2023-10-18T10:40:53.416630+0000","last_unstale":"2023-10-18T10:40:56.018673+0000","last_undegraded":"2023-10-18T10:40:56.018673+0000","last_fullsized":"2023-10-18T10:40:56.018673+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:24.474032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429270+0000","last_change":"2023-10-18T10:40:54.439124+0000","last_active":"2023-10-18T10:40:55.429270+0000","last_peered":"2023-10-18T10:40:55.429270+0000","last_clean":"2023-10-18T10:40:55.429270+0000","last_became_active":"2023-10-18T10:40:54.438869+0000","last_became_peered":"2023-10-18T10:40:54.438869+0000","last_unstale":"2023-10-18T10:40:55.429270+0000","last_undegraded":"2023-10-18T10:40:55.429270+0000","last_fullsized":"2023-10-18T10:40:55.429270+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:50.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144441+0000","last_change":"2023-10-18T10:40:54.420623+0000","last_active":"2023-10-18T10:40:56.144441+0000","last_peered":"2023-10-18T10:40:56.144441+0000","last_clean":"2023-10-18T10:40:56.144441+0000","last_became_active":"2023-10-18T10:40:54.420398+0000","last_became_peered":"2023-10-18T10:40:54.420398+0000","last_unstale":"2023-10-18T10:40:56.144441+0000","last_undegraded":"2023-10-18T10:40:56.144441+0000","last_fullsized":"2023-10-18T10:40:56.144441+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.611760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429356+0000","last_change":"2023-10-18T10:40:53.424629+0000","last_active":"2023-10-18T10:40:55.429356+0000","last_peered":"2023-10-18T10:40:55.429356+0000","last_clean":"2023-10-18T10:40:55.429356+0000","last_became_active":"2023-10-18T10:40:53.424387+0000","last_became_peered":"2023-10-18T10:40:53.424387+0000","last_unstale":"2023-10-18T10:40:55.429356+0000","last_undegraded":"2023-10-18T10:40:55.429356+0000","last_fullsized":"2023-10-18T10:40:55.429356+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:54.556080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144592+0000","last_change":"2023-10-18T10:40:54.427692+0000","last_active":"2023-10-18T10:40:56.144592+0000","last_peered":"2023-10-18T10:40:56.144592+0000","last_clean":"2023-10-18T10:40:56.144592+0000","last_became_active":"2023-10-18T10:40:54.427408+0000","last_became_peered":"2023-10-18T10:40:54.427408+0000","last_unstale":"2023-10-18T10:40:56.144592+0000","last_undegraded":"2023-10-18T10:40:56.144592+0000","last_fullsized":"2023-10-18T10:40:56.144592+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:47:48.326742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429051+0000","last_change":"2023-10-18T10:40:53.426213+0000","last_active":"2023-10-18T10:40:55.429051+0000","last_peered":"2023-10-18T10:40:55.429051+0000","last_clean":"2023-10-18T10:40:55.429051+0000","last_became_active":"2023-10-18T10:40:53.425947+0000","last_became_peered":"2023-10-18T10:40:53.425947+0000","last_unstale":"2023-10-18T10:40:55.429051+0000","last_undegraded":"2023-10-18T10:40:55.429051+0000","last_fullsized":"2023-10-18T10:40:55.429051+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:45:58.336437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145071+0000","last_change":"2023-10-18T10:40:54.429476+0000","last_active":"2023-10-18T10:40:56.145071+0000","last_peered":"2023-10-18T10:40:56.145071+0000","last_clean":"2023-10-18T10:40:56.145071+0000","last_became_active":"2023-10-18T10:40:54.429109+0000","last_became_peered":"2023-10-18T10:40:54.429109+0000","last_unstale":"2023-10-18T10:40:56.145071+0000","last_undegraded":"2023-10-18T10:40:56.145071+0000","last_fullsized":"2023-10-18T10:40:56.145071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:58.443341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.428986+0000","last_change":"2023-10-18T10:40:53.425821+0000","last_active":"2023-10-18T10:40:55.428986+0000","last_peered":"2023-10-18T10:40:55.428986+0000","last_clean":"2023-10-18T10:40:55.428986+0000","last_became_active":"2023-10-18T10:40:53.425567+0000","last_became_peered":"2023-10-18T10:40:53.425567+0000","last_unstale":"2023-10-18T10:40:55.428986+0000","last_undegraded":"2023-10-18T10:40:55.428986+0000","last_fullsized":"2023-10-18T10:40:55.428986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:18.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500839+0000","last_change":"2023-10-18T10:40:53.424258+0000","last_active":"2023-10-18T10:40:55.500839+0000","last_peered":"2023-10-18T10:40:55.500839+0000","last_clean":"2023-10-18T10:40:55.500839+0000","last_became_active":"2023-10-18T10:40:53.423997+0000","last_became_peered":"2023-10-18T10:40:53.423997+0000","last_unstale":"2023-10-18T10:40:55.500839+0000","last_undegraded":"2023-10-18T10:40:55.500839+0000","last_fullsized":"2023-10-18T10:40:55.500839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:05.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144724+0000","last_change":"2023-10-18T10:40:54.433951+0000","last_active":"2023-10-18T10:40:56.144724+0000","last_peered":"2023-10-18T10:40:56.144724+0000","last_clean":"2023-10-18T10:40:56.144724+0000","last_became_active":"2023-10-18T10:40:54.433371+0000","last_became_peered":"2023-10-18T10:40:54.433371+0000","last_unstale":"2023-10-18T10:40:56.144724+0000","last_undegraded":"2023-10-18T10:40:56.144724+0000","last_fullsized":"2023-10-18T10:40:56.144724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:04.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144585+0000","last_change":"2023-10-18T10:40:53.431024+0000","last_active":"2023-10-18T10:40:56.144585+0000","last_peered":"2023-10-18T10:40:56.144585+0000","last_clean":"2023-10-18T10:40:56.144585+0000","last_became_active":"2023-10-18T10:40:53.430743+0000","last_became_peered":"2023-10-18T10:40:53.430743+0000","last_unstale":"2023-10-18T10:40:56.144585+0000","last_undegraded":"2023-10-18T10:40:56.144585+0000","last_fullsized":"2023-10-18T10:40:56.144585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:32.886192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144379+0000","last_change":"2023-10-18T10:40:54.431077+0000","last_active":"2023-10-18T10:40:56.144379+0000","last_peered":"2023-10-18T10:40:56.144379+0000","last_clean":"2023-10-18T10:40:56.144379+0000","last_became_active":"2023-10-18T10:40:54.430805+0000","last_became_peered":"2023-10-18T10:40:54.430805+0000","last_unstale":"2023-10-18T10:40:56.144379+0000","last_undegraded":"2023-10-18T10:40:56.144379+0000","last_fullsized":"2023-10-18T10:40:56.144379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:49:55.716517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144673+0000","last_change":"2023-10-18T10:40:54.433504+0000","last_active":"2023-10-18T10:40:56.144673+0000","last_peered":"2023-10-18T10:40:56.144673+0000","last_clean":"2023-10-18T10:40:56.144673+0000","last_became_active":"2023-10-18T10:40:54.433264+0000","last_became_peered":"2023-10-18T10:40:54.433264+0000","last_unstale":"2023-10-18T10:40:56.144673+0000","last_undegraded":"2023-10-18T10:40:56.144673+0000","last_fullsized":"2023-10-18T10:40:56.144673+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:07.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144446+0000","last_change":"2023-10-18T10:40:53.424988+0000","last_active":"2023-10-18T10:40:56.144446+0000","last_peered":"2023-10-18T10:40:56.144446+0000","last_clean":"2023-10-18T10:40:56.144446+0000","last_became_active":"2023-10-18T10:40:53.423255+0000","last_became_peered":"2023-10-18T10:40:53.423255+0000","last_unstale":"2023-10-18T10:40:56.144446+0000","last_undegraded":"2023-10-18T10:40:56.144446+0000","last_fullsized":"2023-10-18T10:40:56.144446+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:30.233098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144920+0000","last_change":"2023-10-18T10:40:54.434657+0000","last_active":"2023-10-18T10:40:56.144920+0000","last_peered":"2023-10-18T10:40:56.144920+0000","last_clean":"2023-10-18T10:40:56.144920+0000","last_became_active":"2023-10-18T10:40:54.434344+0000","last_became_peered":"2023-10-18T10:40:54.434344+0000","last_unstale":"2023-10-18T10:40:56.144920+0000","last_undegraded":"2023-10-18T10:40:56.144920+0000","last_fullsized":"2023-10-18T10:40:56.144920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:49.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018412+0000","last_change":"2023-10-18T10:40:53.419056+0000","last_active":"2023-10-18T10:40:56.018412+0000","last_peered":"2023-10-18T10:40:56.018412+0000","last_clean":"2023-10-18T10:40:56.018412+0000","last_became_active":"2023-10-18T10:40:53.418815+0000","last_became_peered":"2023-10-18T10:40:53.418815+0000","last_unstale":"2023-10-18T10:40:56.018412+0000","last_undegraded":"2023-10-18T10:40:56.018412+0000","last_fullsized":"2023-10-18T10:40:56.018412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:13.857868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144758+0000","last_change":"2023-10-18T10:40:53.431352+0000","last_active":"2023-10-18T10:40:56.144758+0000","last_peered":"2023-10-18T10:40:56.144758+0000","last_clean":"2023-10-18T10:40:56.144758+0000","last_became_active":"2023-10-18T10:40:53.431047+0000","last_became_peered":"2023-10-18T10:40:53.431047+0000","last_unstale":"2023-10-18T10:40:56.144758+0000","last_undegraded":"2023-10-18T10:40:56.144758+0000","last_fullsized":"2023-10-18T10:40:56.144758+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:17.609983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429479+0000","last_change":"2023-10-18T10:40:53.425307+0000","last_active":"2023-10-18T10:40:55.429479+0000","last_peered":"2023-10-18T10:40:55.429479+0000","last_clean":"2023-10-18T10:40:55.429479+0000","last_became_active":"2023-10-18T10:40:53.424835+0000","last_became_peered":"2023-10-18T10:40:53.424835+0000","last_unstale":"2023-10-18T10:40:55.429479+0000","last_undegraded":"2023-10-18T10:40:55.429479+0000","last_fullsized":"2023-10-18T10:40:55.429479+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:04.973143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"86'172","reported_seq":429,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T10:40:50.192957+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:50.192957+0000","last_peered":"2023-10-18T10:40:50.192957+0000","last_clean":"2023-10-18T10:40:50.192957+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:50.192957+0000","last_undegraded":"2023-10-18T10:40:50.192957+0000","last_fullsized":"2023-10-18T10:40:50.192957+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":86,"acting":86,"num_store_stats":5},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":42,"acting":42,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59252,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312588,"statfs":{"total":96636764160,"available":96576090112,"internally_reserved":0,"allocated":1744896,"data_stored":1162487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.64700000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.47799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.627}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.44900000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.56499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":593558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":1.016},"last":0.52800000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.80300000000000005,"15min":0.875},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.504}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.872,"15min":0.872},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66800000000000004,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36599999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":0.82199999999999995,"15min":0.87},"last":0.44500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":1.016},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.498},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.81000000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38300000000000001,"15min":0.28599999999999998},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.314,"15min":0.28499999999999998},"max":{"1min":0.71199999999999997,"5min":0.80100000000000005,"15min":1.262},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":1.0109999999999999,"15min":1.028},"last":0.61399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68400000000000005,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77800000000000002,"5min":0.96099999999999997,"15min":1.224},"last":0.59599999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65600000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.438,"5min":0.307,"15min":0.307},"max":{"1min":0.85299999999999998,"5min":0.871,"15min":1.159},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":572119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.877,"15min":0.877},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.495}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.502,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.72099999999999997,"5min":0.746,"15min":0.79600000000000004},"last":0.629}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75600000000000001,"5min":1.446,"15min":1.446},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.90400000000000003},"last":0.40200000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.82699999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.47499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":599148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.315}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.436},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1212416,"data_stored":599308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.224,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.192},"max":{"1min":0.51900000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.54600000000000004,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.30399999999999999,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.60199999999999998,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.502},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.626,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.82399999999999995,"5min":0.88200000000000001,"15min":1.458},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.83799999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.746}]},{"osd":5,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.89000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.93700000000000006,"15min":1.028},"last":0.68400000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.625,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.879,"15min":0.92500000000000004},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":572119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.33900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.377},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.497}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.433},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.57499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1781760,"data_stored":1182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.25},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.27700000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.37},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.315}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:57.373 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:57.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.507+0000 7fef751d7640 1 -- 172.21.15.125:0/2853563449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef5c0039a0 msgr2=0x7fef5c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.507+0000 7fef751d7640 1 --2- 172.21.15.125:0/2853563449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef5c0039a0 0x7fef5c003dc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fef6000bb20 tx=0x7fef5c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.507+0000 7fef6f59e640 1 -- 172.21.15.125:0/2853563449 reap_dead start 2023-10-18T10:40:57.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.507+0000 7fef751d7640 1 -- 172.21.15.125:0/2853563449 shutdown_connections 2023-10-18T10:40:57.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.507+0000 7fef751d7640 1 -- 172.21.15.125:0/2853563449 >> 172.21.15.125:0/2853563449 conn(0x7fef70069930 msgr2=0x7fef70069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:57.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.507+0000 7fef751d7640 1 -- 172.21.15.125:0/2853563449 shutdown_connections 2023-10-18T10:40:57.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.507+0000 7fef751d7640 1 -- 172.21.15.125:0/2853563449 wait complete. 2023-10-18T10:40:57.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.508+0000 7fef751d7640 1 Processor -- start 2023-10-18T10:40:57.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.508+0000 7fef751d7640 1 -- start start 2023-10-18T10:40:57.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef751d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef5c0039a0 0x7fef70070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef751d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef70070530 0x7fef7013e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef751d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef700aa8c0 0x7fef700aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef751d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7013d170 con 0x7fef700aa8c0 2023-10-18T10:40:57.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef751d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7013dcf0 con 0x7fef70070530 2023-10-18T10:40:57.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef751d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef70070990 con 0x7fef5c0039a0 2023-10-18T10:40:57.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef700aa8c0 0x7fef700aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef700aa8c0 0x7fef700aad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43308/0 (socket says 172.21.15.125:43308) 2023-10-18T10:40:57.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6f59e640 1 -- 172.21.15.125:0/4092609323 learned_addr learned my addr 172.21.15.125:0/4092609323 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:57.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef67fff640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef70070530 0x7fef7013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6ed9d640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef5c0039a0 0x7fef70070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6f59e640 1 -- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef5c0039a0 msgr2=0x7fef70070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6f59e640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef5c0039a0 0x7fef70070b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6f59e640 1 -- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef70070530 msgr2=0x7fef7013e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6f59e640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef70070530 0x7fef7013e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.509+0000 7fef6f59e640 1 -- 172.21.15.125:0/4092609323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70153c00 con 0x7fef700aa8c0 2023-10-18T10:40:57.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.510+0000 7fef6f59e640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef700aa8c0 0x7fef700aad40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fef7013f3f0 tx=0x7fef6800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:57.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.510+0000 7fef677fe640 1 -- 172.21.15.125:0/4092609323 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6800cb40 con 0x7fef700aa8c0 2023-10-18T10:40:57.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.510+0000 7fef677fe640 1 -- 172.21.15.125:0/4092609323 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef6800cdd0 con 0x7fef700aa8c0 2023-10-18T10:40:57.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.510+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7013d350 con 0x7fef700aa8c0 2023-10-18T10:40:57.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.510+0000 7fef677fe640 1 -- 172.21.15.125:0/4092609323 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680116c0 con 0x7fef700aa8c0 2023-10-18T10:40:57.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.510+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7013d910 con 0x7fef700aa8c0 2023-10-18T10:40:57.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.511+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7013de30 con 0x7fef700aa8c0 2023-10-18T10:40:57.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.511+0000 7fef677fe640 1 -- 172.21.15.125:0/4092609323 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fef68011820 con 0x7fef700aa8c0 2023-10-18T10:40:57.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.511+0000 7fef677fe640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef5006af90 0x7fef5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.511+0000 7fef677fe640 1 -- 172.21.15.125:0/4092609323 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fef6808a150 con 0x7fef700aa8c0 2023-10-18T10:40:57.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.512+0000 7fef6ed9d640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef5006af90 0x7fef5006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.512+0000 7fef6ed9d640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef5006af90 0x7fef5006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef60037000 tx=0x7fef6002e040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:57.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.516+0000 7fef677fe640 1 -- 172.21.15.125:0/4092609323 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef7013de30 con 0x7fef700aa8c0 2023-10-18T10:40:57.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.678+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 --> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef700ab280 con 0x7fef5006af90 2023-10-18T10:40:57.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.686+0000 7fef677fe640 1 -- 172.21.15.125:0/4092609323 <== mgr.5972 v2:172.21.15.139:6832/143413153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284363 (secure 0 0 0) 0x7fef700ab280 con 0x7fef5006af90 2023-10-18T10:40:57.690 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:57.692 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:57.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.688+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef5006af90 msgr2=0x7fef5006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.688+0000 7fef751d7640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef5006af90 0x7fef5006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef60037000 tx=0x7fef6002e040 comp rx=0 tx=0).stop 2023-10-18T10:40:57.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.688+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef700aa8c0 msgr2=0x7fef700aad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.688+0000 7fef751d7640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef700aa8c0 0x7fef700aad40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fef7013f3f0 tx=0x7fef6800ed10 comp rx=0 tx=0).stop 2023-10-18T10:40:57.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.689+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 shutdown_connections 2023-10-18T10:40:57.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.689+0000 7fef751d7640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fef5006af90 0x7fef5006d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.689+0000 7fef751d7640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef70070530 0x7fef7013e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.689+0000 7fef751d7640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef700aa8c0 0x7fef700aad40 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.689+0000 7fef751d7640 1 --2- 172.21.15.125:0/4092609323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef5c0039a0 0x7fef70070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.689+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 >> 172.21.15.125:0/4092609323 conn(0x7fef70069930 msgr2=0x7fef701399d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:57.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.689+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 shutdown_connections 2023-10-18T10:40:57.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.689+0000 7fef751d7640 1 -- 172.21.15.125:0/4092609323 wait complete. 2023-10-18T10:40:57.701 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:40:56.162612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":296,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":312608,"kb_used_data":10400,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754662112,"statfs":{"total":773094113280,"available":772774002688,"internally_reserved":0,"allocated":10649600,"data_stored":5879653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001032"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144511+0000","last_change":"2023-10-18T10:40:54.435140+0000","last_active":"2023-10-18T10:40:56.144511+0000","last_peered":"2023-10-18T10:40:56.144511+0000","last_clean":"2023-10-18T10:40:56.144511+0000","last_became_active":"2023-10-18T10:40:54.434657+0000","last_became_peered":"2023-10-18T10:40:54.434657+0000","last_unstale":"2023-10-18T10:40:56.144511+0000","last_undegraded":"2023-10-18T10:40:56.144511+0000","last_fullsized":"2023-10-18T10:40:56.144511+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.896314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500661+0000","last_change":"2023-10-18T10:40:54.428671+0000","last_active":"2023-10-18T10:40:55.500661+0000","last_peered":"2023-10-18T10:40:55.500661+0000","last_clean":"2023-10-18T10:40:55.500661+0000","last_became_active":"2023-10-18T10:40:54.428396+0000","last_became_peered":"2023-10-18T10:40:54.428396+0000","last_unstale":"2023-10-18T10:40:55.500661+0000","last_undegraded":"2023-10-18T10:40:55.500661+0000","last_fullsized":"2023-10-18T10:40:55.500661+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:39.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018418+0000","last_change":"2023-10-18T10:40:54.433351+0000","last_active":"2023-10-18T10:40:56.018418+0000","last_peered":"2023-10-18T10:40:56.018418+0000","last_clean":"2023-10-18T10:40:56.018418+0000","last_became_active":"2023-10-18T10:40:54.430314+0000","last_became_peered":"2023-10-18T10:40:54.430314+0000","last_unstale":"2023-10-18T10:40:56.018418+0000","last_undegraded":"2023-10-18T10:40:56.018418+0000","last_fullsized":"2023-10-18T10:40:56.018418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:48.808404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429115+0000","last_change":"2023-10-18T10:40:54.431510+0000","last_active":"2023-10-18T10:40:55.429115+0000","last_peered":"2023-10-18T10:40:55.429115+0000","last_clean":"2023-10-18T10:40:55.429115+0000","last_became_active":"2023-10-18T10:40:54.431184+0000","last_became_peered":"2023-10-18T10:40:54.431184+0000","last_unstale":"2023-10-18T10:40:55.429115+0000","last_undegraded":"2023-10-18T10:40:55.429115+0000","last_fullsized":"2023-10-18T10:40:55.429115+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:33.893983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144553+0000","last_change":"2023-10-18T10:40:54.434704+0000","last_active":"2023-10-18T10:40:56.144553+0000","last_peered":"2023-10-18T10:40:56.144553+0000","last_clean":"2023-10-18T10:40:56.144553+0000","last_became_active":"2023-10-18T10:40:54.434388+0000","last_became_peered":"2023-10-18T10:40:54.434388+0000","last_unstale":"2023-10-18T10:40:56.144553+0000","last_undegraded":"2023-10-18T10:40:56.144553+0000","last_fullsized":"2023-10-18T10:40:56.144553+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:54.346618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:54.427003+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:54.426743+0000","last_became_peered":"2023-10-18T10:40:54.426743+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:58.361349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500867+0000","last_change":"2023-10-18T10:40:54.433837+0000","last_active":"2023-10-18T10:40:55.500867+0000","last_peered":"2023-10-18T10:40:55.500867+0000","last_clean":"2023-10-18T10:40:55.500867+0000","last_became_active":"2023-10-18T10:40:54.433558+0000","last_became_peered":"2023-10-18T10:40:54.433558+0000","last_unstale":"2023-10-18T10:40:55.500867+0000","last_undegraded":"2023-10-18T10:40:55.500867+0000","last_fullsized":"2023-10-18T10:40:55.500867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:04.849763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144518+0000","last_change":"2023-10-18T10:40:54.429847+0000","last_active":"2023-10-18T10:40:56.144518+0000","last_peered":"2023-10-18T10:40:56.144518+0000","last_clean":"2023-10-18T10:40:56.144518+0000","last_became_active":"2023-10-18T10:40:54.429629+0000","last_became_peered":"2023-10-18T10:40:54.429629+0000","last_unstale":"2023-10-18T10:40:56.144518+0000","last_undegraded":"2023-10-18T10:40:56.144518+0000","last_fullsized":"2023-10-18T10:40:56.144518+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:29.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500804+0000","last_change":"2023-10-18T10:40:54.427881+0000","last_active":"2023-10-18T10:40:55.500804+0000","last_peered":"2023-10-18T10:40:55.500804+0000","last_clean":"2023-10-18T10:40:55.500804+0000","last_became_active":"2023-10-18T10:40:54.427423+0000","last_became_peered":"2023-10-18T10:40:54.427423+0000","last_unstale":"2023-10-18T10:40:55.500804+0000","last_undegraded":"2023-10-18T10:40:55.500804+0000","last_fullsized":"2023-10-18T10:40:55.500804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:07.682253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018526+0000","last_change":"2023-10-18T10:40:54.426415+0000","last_active":"2023-10-18T10:40:56.018526+0000","last_peered":"2023-10-18T10:40:56.018526+0000","last_clean":"2023-10-18T10:40:56.018526+0000","last_became_active":"2023-10-18T10:40:54.426129+0000","last_became_peered":"2023-10-18T10:40:54.426129+0000","last_unstale":"2023-10-18T10:40:56.018526+0000","last_undegraded":"2023-10-18T10:40:56.018526+0000","last_fullsized":"2023-10-18T10:40:56.018526+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:33:27.749058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429139+0000","last_change":"2023-10-18T10:40:54.430848+0000","last_active":"2023-10-18T10:40:55.429139+0000","last_peered":"2023-10-18T10:40:55.429139+0000","last_clean":"2023-10-18T10:40:55.429139+0000","last_became_active":"2023-10-18T10:40:54.430562+0000","last_became_peered":"2023-10-18T10:40:54.430562+0000","last_unstale":"2023-10-18T10:40:55.429139+0000","last_undegraded":"2023-10-18T10:40:55.429139+0000","last_fullsized":"2023-10-18T10:40:55.429139+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:05.564173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144611+0000","last_change":"2023-10-18T10:40:54.421817+0000","last_active":"2023-10-18T10:40:56.144611+0000","last_peered":"2023-10-18T10:40:56.144611+0000","last_clean":"2023-10-18T10:40:56.144611+0000","last_became_active":"2023-10-18T10:40:54.421613+0000","last_became_peered":"2023-10-18T10:40:54.421613+0000","last_unstale":"2023-10-18T10:40:56.144611+0000","last_undegraded":"2023-10-18T10:40:56.144611+0000","last_fullsized":"2023-10-18T10:40:56.144611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:36.706863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:54.435303+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:54.434740+0000","last_became_peered":"2023-10-18T10:40:54.434740+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.386244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429236+0000","last_change":"2023-10-18T10:40:54.432868+0000","last_active":"2023-10-18T10:40:55.429236+0000","last_peered":"2023-10-18T10:40:55.429236+0000","last_clean":"2023-10-18T10:40:55.429236+0000","last_became_active":"2023-10-18T10:40:54.432637+0000","last_became_peered":"2023-10-18T10:40:54.432637+0000","last_unstale":"2023-10-18T10:40:55.429236+0000","last_undegraded":"2023-10-18T10:40:55.429236+0000","last_fullsized":"2023-10-18T10:40:55.429236+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:46.951599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018489+0000","last_change":"2023-10-18T10:40:54.430205+0000","last_active":"2023-10-18T10:40:56.018489+0000","last_peered":"2023-10-18T10:40:56.018489+0000","last_clean":"2023-10-18T10:40:56.018489+0000","last_became_active":"2023-10-18T10:40:54.429898+0000","last_became_peered":"2023-10-18T10:40:54.429898+0000","last_unstale":"2023-10-18T10:40:56.018489+0000","last_undegraded":"2023-10-18T10:40:56.018489+0000","last_fullsized":"2023-10-18T10:40:56.018489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:48.646925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018318+0000","last_change":"2023-10-18T10:40:54.428004+0000","last_active":"2023-10-18T10:40:56.018318+0000","last_peered":"2023-10-18T10:40:56.018318+0000","last_clean":"2023-10-18T10:40:56.018318+0000","last_became_active":"2023-10-18T10:40:54.427533+0000","last_became_peered":"2023-10-18T10:40:54.427533+0000","last_unstale":"2023-10-18T10:40:56.018318+0000","last_undegraded":"2023-10-18T10:40:56.018318+0000","last_fullsized":"2023-10-18T10:40:56.018318+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:27:07.146678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143943+0000","last_change":"2023-10-18T10:40:54.424777+0000","last_active":"2023-10-18T10:40:56.143943+0000","last_peered":"2023-10-18T10:40:56.143943+0000","last_clean":"2023-10-18T10:40:56.143943+0000","last_became_active":"2023-10-18T10:40:54.424544+0000","last_became_peered":"2023-10-18T10:40:54.424544+0000","last_unstale":"2023-10-18T10:40:56.143943+0000","last_undegraded":"2023-10-18T10:40:56.143943+0000","last_fullsized":"2023-10-18T10:40:56.143943+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144475+0000","last_change":"2023-10-18T10:40:54.425355+0000","last_active":"2023-10-18T10:40:56.144475+0000","last_peered":"2023-10-18T10:40:56.144475+0000","last_clean":"2023-10-18T10:40:56.144475+0000","last_became_active":"2023-10-18T10:40:54.425123+0000","last_became_peered":"2023-10-18T10:40:54.425123+0000","last_unstale":"2023-10-18T10:40:56.144475+0000","last_undegraded":"2023-10-18T10:40:56.144475+0000","last_fullsized":"2023-10-18T10:40:56.144475+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:24.991706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500878+0000","last_change":"2023-10-18T10:40:54.426993+0000","last_active":"2023-10-18T10:40:55.500878+0000","last_peered":"2023-10-18T10:40:55.500878+0000","last_clean":"2023-10-18T10:40:55.500878+0000","last_became_active":"2023-10-18T10:40:54.426702+0000","last_became_peered":"2023-10-18T10:40:54.426702+0000","last_unstale":"2023-10-18T10:40:55.500878+0000","last_undegraded":"2023-10-18T10:40:55.500878+0000","last_fullsized":"2023-10-18T10:40:55.500878+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:57.667741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018550+0000","last_change":"2023-10-18T10:40:54.432406+0000","last_active":"2023-10-18T10:40:56.018550+0000","last_peered":"2023-10-18T10:40:56.018550+0000","last_clean":"2023-10-18T10:40:56.018550+0000","last_became_active":"2023-10-18T10:40:54.432113+0000","last_became_peered":"2023-10-18T10:40:54.432113+0000","last_unstale":"2023-10-18T10:40:56.018550+0000","last_undegraded":"2023-10-18T10:40:56.018550+0000","last_fullsized":"2023-10-18T10:40:56.018550+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:24:16.099369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144250+0000","last_change":"2023-10-18T10:40:54.431844+0000","last_active":"2023-10-18T10:40:56.144250+0000","last_peered":"2023-10-18T10:40:56.144250+0000","last_clean":"2023-10-18T10:40:56.144250+0000","last_became_active":"2023-10-18T10:40:54.431447+0000","last_became_peered":"2023-10-18T10:40:54.431447+0000","last_unstale":"2023-10-18T10:40:56.144250+0000","last_undegraded":"2023-10-18T10:40:56.144250+0000","last_fullsized":"2023-10-18T10:40:56.144250+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:30.965000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"97'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:53.417631+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:53.417381+0000","last_became_peered":"2023-10-18T10:40:53.417381+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:40.271808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144091+0000","last_change":"2023-10-18T10:40:54.420096+0000","last_active":"2023-10-18T10:40:56.144091+0000","last_peered":"2023-10-18T10:40:56.144091+0000","last_clean":"2023-10-18T10:40:56.144091+0000","last_became_active":"2023-10-18T10:40:54.419838+0000","last_became_peered":"2023-10-18T10:40:54.419838+0000","last_unstale":"2023-10-18T10:40:56.144091+0000","last_undegraded":"2023-10-18T10:40:56.144091+0000","last_fullsized":"2023-10-18T10:40:56.144091+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:17.947505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144791+0000","last_change":"2023-10-18T10:40:53.423111+0000","last_active":"2023-10-18T10:40:56.144791+0000","last_peered":"2023-10-18T10:40:56.144791+0000","last_clean":"2023-10-18T10:40:56.144791+0000","last_became_active":"2023-10-18T10:40:53.422848+0000","last_became_peered":"2023-10-18T10:40:53.422848+0000","last_unstale":"2023-10-18T10:40:56.144791+0000","last_undegraded":"2023-10-18T10:40:56.144791+0000","last_fullsized":"2023-10-18T10:40:56.144791+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:19.845504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144374+0000","last_change":"2023-10-18T10:40:54.435357+0000","last_active":"2023-10-18T10:40:56.144374+0000","last_peered":"2023-10-18T10:40:56.144374+0000","last_clean":"2023-10-18T10:40:56.144374+0000","last_became_active":"2023-10-18T10:40:54.435102+0000","last_became_peered":"2023-10-18T10:40:54.435102+0000","last_unstale":"2023-10-18T10:40:56.144374+0000","last_undegraded":"2023-10-18T10:40:56.144374+0000","last_fullsized":"2023-10-18T10:40:56.144374+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:28:23.536390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144627+0000","last_change":"2023-10-18T10:40:54.428864+0000","last_active":"2023-10-18T10:40:56.144627+0000","last_peered":"2023-10-18T10:40:56.144627+0000","last_clean":"2023-10-18T10:40:56.144627+0000","last_became_active":"2023-10-18T10:40:54.428412+0000","last_became_peered":"2023-10-18T10:40:54.428412+0000","last_unstale":"2023-10-18T10:40:56.144627+0000","last_undegraded":"2023-10-18T10:40:56.144627+0000","last_fullsized":"2023-10-18T10:40:56.144627+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:15.038645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500519+0000","last_change":"2023-10-18T10:40:53.420317+0000","last_active":"2023-10-18T10:40:55.500519+0000","last_peered":"2023-10-18T10:40:55.500519+0000","last_clean":"2023-10-18T10:40:55.500519+0000","last_became_active":"2023-10-18T10:40:53.420046+0000","last_became_peered":"2023-10-18T10:40:53.420046+0000","last_unstale":"2023-10-18T10:40:55.500519+0000","last_undegraded":"2023-10-18T10:40:55.500519+0000","last_fullsized":"2023-10-18T10:40:55.500519+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:36.623092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144181+0000","last_change":"2023-10-18T10:40:53.430760+0000","last_active":"2023-10-18T10:40:56.144181+0000","last_peered":"2023-10-18T10:40:56.144181+0000","last_clean":"2023-10-18T10:40:56.144181+0000","last_became_active":"2023-10-18T10:40:53.430469+0000","last_became_peered":"2023-10-18T10:40:53.430469+0000","last_unstale":"2023-10-18T10:40:56.144181+0000","last_undegraded":"2023-10-18T10:40:56.144181+0000","last_fullsized":"2023-10-18T10:40:56.144181+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:45.943807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144991+0000","last_change":"2023-10-18T10:40:54.435078+0000","last_active":"2023-10-18T10:40:56.144991+0000","last_peered":"2023-10-18T10:40:56.144991+0000","last_clean":"2023-10-18T10:40:56.144991+0000","last_became_active":"2023-10-18T10:40:54.434517+0000","last_became_peered":"2023-10-18T10:40:54.434517+0000","last_unstale":"2023-10-18T10:40:56.144991+0000","last_undegraded":"2023-10-18T10:40:56.144991+0000","last_fullsized":"2023-10-18T10:40:56.144991+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:40.219033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:53.415948+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:53.415668+0000","last_became_peered":"2023-10-18T10:40:53.415668+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:12.409497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144074+0000","last_change":"2023-10-18T10:40:54.426165+0000","last_active":"2023-10-18T10:40:56.144074+0000","last_peered":"2023-10-18T10:40:56.144074+0000","last_clean":"2023-10-18T10:40:56.144074+0000","last_became_active":"2023-10-18T10:40:54.425916+0000","last_became_peered":"2023-10-18T10:40:54.425916+0000","last_unstale":"2023-10-18T10:40:56.144074+0000","last_undegraded":"2023-10-18T10:40:56.144074+0000","last_fullsized":"2023-10-18T10:40:56.144074+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500662+0000","last_change":"2023-10-18T10:40:54.428238+0000","last_active":"2023-10-18T10:40:55.500662+0000","last_peered":"2023-10-18T10:40:55.500662+0000","last_clean":"2023-10-18T10:40:55.500662+0000","last_became_active":"2023-10-18T10:40:54.427404+0000","last_became_peered":"2023-10-18T10:40:54.427404+0000","last_unstale":"2023-10-18T10:40:55.500662+0000","last_undegraded":"2023-10-18T10:40:55.500662+0000","last_fullsized":"2023-10-18T10:40:55.500662+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:06.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500959+0000","last_change":"2023-10-18T10:40:53.423540+0000","last_active":"2023-10-18T10:40:55.500959+0000","last_peered":"2023-10-18T10:40:55.500959+0000","last_clean":"2023-10-18T10:40:55.500959+0000","last_became_active":"2023-10-18T10:40:53.423278+0000","last_became_peered":"2023-10-18T10:40:53.423278+0000","last_unstale":"2023-10-18T10:40:55.500959+0000","last_undegraded":"2023-10-18T10:40:55.500959+0000","last_fullsized":"2023-10-18T10:40:55.500959+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:15.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429349+0000","last_change":"2023-10-18T10:40:54.423285+0000","last_active":"2023-10-18T10:40:55.429349+0000","last_peered":"2023-10-18T10:40:55.429349+0000","last_clean":"2023-10-18T10:40:55.429349+0000","last_became_active":"2023-10-18T10:40:54.423013+0000","last_became_peered":"2023-10-18T10:40:54.423013+0000","last_unstale":"2023-10-18T10:40:55.429349+0000","last_undegraded":"2023-10-18T10:40:55.429349+0000","last_fullsized":"2023-10-18T10:40:55.429349+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"97'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018642+0000","last_change":"2023-10-18T10:40:53.424070+0000","last_active":"2023-10-18T10:40:56.018642+0000","last_peered":"2023-10-18T10:40:56.018642+0000","last_clean":"2023-10-18T10:40:56.018642+0000","last_became_active":"2023-10-18T10:40:53.423799+0000","last_became_peered":"2023-10-18T10:40:53.423799+0000","last_unstale":"2023-10-18T10:40:56.018642+0000","last_undegraded":"2023-10-18T10:40:56.018642+0000","last_fullsized":"2023-10-18T10:40:56.018642+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:09.891958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"97'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144832+0000","last_change":"2023-10-18T10:40:53.422980+0000","last_active":"2023-10-18T10:40:56.144832+0000","last_peered":"2023-10-18T10:40:56.144832+0000","last_clean":"2023-10-18T10:40:56.144832+0000","last_became_active":"2023-10-18T10:40:53.422696+0000","last_became_peered":"2023-10-18T10:40:53.422696+0000","last_unstale":"2023-10-18T10:40:56.144832+0000","last_undegraded":"2023-10-18T10:40:56.144832+0000","last_fullsized":"2023-10-18T10:40:56.144832+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:20.506905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500893+0000","last_change":"2023-10-18T10:40:54.423164+0000","last_active":"2023-10-18T10:40:55.500893+0000","last_peered":"2023-10-18T10:40:55.500893+0000","last_clean":"2023-10-18T10:40:55.500893+0000","last_became_active":"2023-10-18T10:40:54.422844+0000","last_became_peered":"2023-10-18T10:40:54.422844+0000","last_unstale":"2023-10-18T10:40:55.500893+0000","last_undegraded":"2023-10-18T10:40:55.500893+0000","last_fullsized":"2023-10-18T10:40:55.500893+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:50.691881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500565+0000","last_change":"2023-10-18T10:40:53.420785+0000","last_active":"2023-10-18T10:40:55.500565+0000","last_peered":"2023-10-18T10:40:55.500565+0000","last_clean":"2023-10-18T10:40:55.500565+0000","last_became_active":"2023-10-18T10:40:53.420530+0000","last_became_peered":"2023-10-18T10:40:53.420530+0000","last_unstale":"2023-10-18T10:40:55.500565+0000","last_undegraded":"2023-10-18T10:40:55.500565+0000","last_fullsized":"2023-10-18T10:40:55.500565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:44.748900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144877+0000","last_change":"2023-10-18T10:40:54.430742+0000","last_active":"2023-10-18T10:40:56.144877+0000","last_peered":"2023-10-18T10:40:56.144877+0000","last_clean":"2023-10-18T10:40:56.144877+0000","last_became_active":"2023-10-18T10:40:54.430455+0000","last_became_peered":"2023-10-18T10:40:54.430455+0000","last_unstale":"2023-10-18T10:40:56.144877+0000","last_undegraded":"2023-10-18T10:40:56.144877+0000","last_fullsized":"2023-10-18T10:40:56.144877+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:39.892546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144941+0000","last_change":"2023-10-18T10:40:54.430021+0000","last_active":"2023-10-18T10:40:56.144941+0000","last_peered":"2023-10-18T10:40:56.144941+0000","last_clean":"2023-10-18T10:40:56.144941+0000","last_became_active":"2023-10-18T10:40:54.429684+0000","last_became_peered":"2023-10-18T10:40:54.429684+0000","last_unstale":"2023-10-18T10:40:56.144941+0000","last_undegraded":"2023-10-18T10:40:56.144941+0000","last_fullsized":"2023-10-18T10:40:56.144941+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:03.648375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018334+0000","last_change":"2023-10-18T10:40:53.425774+0000","last_active":"2023-10-18T10:40:56.018334+0000","last_peered":"2023-10-18T10:40:56.018334+0000","last_clean":"2023-10-18T10:40:56.018334+0000","last_became_active":"2023-10-18T10:40:53.425518+0000","last_became_peered":"2023-10-18T10:40:53.425518+0000","last_unstale":"2023-10-18T10:40:56.018334+0000","last_undegraded":"2023-10-18T10:40:56.018334+0000","last_fullsized":"2023-10-18T10:40:56.018334+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:30.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145025+0000","last_change":"2023-10-18T10:40:54.432786+0000","last_active":"2023-10-18T10:40:56.145025+0000","last_peered":"2023-10-18T10:40:56.145025+0000","last_clean":"2023-10-18T10:40:56.145025+0000","last_became_active":"2023-10-18T10:40:54.432538+0000","last_became_peered":"2023-10-18T10:40:54.432538+0000","last_unstale":"2023-10-18T10:40:56.145025+0000","last_undegraded":"2023-10-18T10:40:56.145025+0000","last_fullsized":"2023-10-18T10:40:56.145025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:52:39.855703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"97'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144160+0000","last_change":"2023-10-18T10:40:53.426701+0000","last_active":"2023-10-18T10:40:56.144160+0000","last_peered":"2023-10-18T10:40:56.144160+0000","last_clean":"2023-10-18T10:40:56.144160+0000","last_became_active":"2023-10-18T10:40:53.426431+0000","last_became_peered":"2023-10-18T10:40:53.426431+0000","last_unstale":"2023-10-18T10:40:56.144160+0000","last_undegraded":"2023-10-18T10:40:56.144160+0000","last_fullsized":"2023-10-18T10:40:56.144160+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:31.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429351+0000","last_change":"2023-10-18T10:40:54.434075+0000","last_active":"2023-10-18T10:40:55.429351+0000","last_peered":"2023-10-18T10:40:55.429351+0000","last_clean":"2023-10-18T10:40:55.429351+0000","last_became_active":"2023-10-18T10:40:54.433835+0000","last_became_peered":"2023-10-18T10:40:54.433835+0000","last_unstale":"2023-10-18T10:40:55.429351+0000","last_undegraded":"2023-10-18T10:40:55.429351+0000","last_fullsized":"2023-10-18T10:40:55.429351+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018402+0000","last_change":"2023-10-18T10:40:54.435840+0000","last_active":"2023-10-18T10:40:56.018402+0000","last_peered":"2023-10-18T10:40:56.018402+0000","last_clean":"2023-10-18T10:40:56.018402+0000","last_became_active":"2023-10-18T10:40:54.435594+0000","last_became_peered":"2023-10-18T10:40:54.435594+0000","last_unstale":"2023-10-18T10:40:56.018402+0000","last_undegraded":"2023-10-18T10:40:56.018402+0000","last_fullsized":"2023-10-18T10:40:56.018402+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:16:01.996675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018391+0000","last_change":"2023-10-18T10:40:54.427794+0000","last_active":"2023-10-18T10:40:56.018391+0000","last_peered":"2023-10-18T10:40:56.018391+0000","last_clean":"2023-10-18T10:40:56.018391+0000","last_became_active":"2023-10-18T10:40:54.427255+0000","last_became_peered":"2023-10-18T10:40:54.427255+0000","last_unstale":"2023-10-18T10:40:56.018391+0000","last_undegraded":"2023-10-18T10:40:56.018391+0000","last_fullsized":"2023-10-18T10:40:56.018391+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:08.181951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018673+0000","last_change":"2023-10-18T10:40:53.416910+0000","last_active":"2023-10-18T10:40:56.018673+0000","last_peered":"2023-10-18T10:40:56.018673+0000","last_clean":"2023-10-18T10:40:56.018673+0000","last_became_active":"2023-10-18T10:40:53.416630+0000","last_became_peered":"2023-10-18T10:40:53.416630+0000","last_unstale":"2023-10-18T10:40:56.018673+0000","last_undegraded":"2023-10-18T10:40:56.018673+0000","last_fullsized":"2023-10-18T10:40:56.018673+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:24.474032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429270+0000","last_change":"2023-10-18T10:40:54.439124+0000","last_active":"2023-10-18T10:40:55.429270+0000","last_peered":"2023-10-18T10:40:55.429270+0000","last_clean":"2023-10-18T10:40:55.429270+0000","last_became_active":"2023-10-18T10:40:54.438869+0000","last_became_peered":"2023-10-18T10:40:54.438869+0000","last_unstale":"2023-10-18T10:40:55.429270+0000","last_undegraded":"2023-10-18T10:40:55.429270+0000","last_fullsized":"2023-10-18T10:40:55.429270+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:50.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144441+0000","last_change":"2023-10-18T10:40:54.420623+0000","last_active":"2023-10-18T10:40:56.144441+0000","last_peered":"2023-10-18T10:40:56.144441+0000","last_clean":"2023-10-18T10:40:56.144441+0000","last_became_active":"2023-10-18T10:40:54.420398+0000","last_became_peered":"2023-10-18T10:40:54.420398+0000","last_unstale":"2023-10-18T10:40:56.144441+0000","last_undegraded":"2023-10-18T10:40:56.144441+0000","last_fullsized":"2023-10-18T10:40:56.144441+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.611760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429356+0000","last_change":"2023-10-18T10:40:53.424629+0000","last_active":"2023-10-18T10:40:55.429356+0000","last_peered":"2023-10-18T10:40:55.429356+0000","last_clean":"2023-10-18T10:40:55.429356+0000","last_became_active":"2023-10-18T10:40:53.424387+0000","last_became_peered":"2023-10-18T10:40:53.424387+0000","last_unstale":"2023-10-18T10:40:55.429356+0000","last_undegraded":"2023-10-18T10:40:55.429356+0000","last_fullsized":"2023-10-18T10:40:55.429356+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:54.556080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144592+0000","last_change":"2023-10-18T10:40:54.427692+0000","last_active":"2023-10-18T10:40:56.144592+0000","last_peered":"2023-10-18T10:40:56.144592+0000","last_clean":"2023-10-18T10:40:56.144592+0000","last_became_active":"2023-10-18T10:40:54.427408+0000","last_became_peered":"2023-10-18T10:40:54.427408+0000","last_unstale":"2023-10-18T10:40:56.144592+0000","last_undegraded":"2023-10-18T10:40:56.144592+0000","last_fullsized":"2023-10-18T10:40:56.144592+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:47:48.326742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429051+0000","last_change":"2023-10-18T10:40:53.426213+0000","last_active":"2023-10-18T10:40:55.429051+0000","last_peered":"2023-10-18T10:40:55.429051+0000","last_clean":"2023-10-18T10:40:55.429051+0000","last_became_active":"2023-10-18T10:40:53.425947+0000","last_became_peered":"2023-10-18T10:40:53.425947+0000","last_unstale":"2023-10-18T10:40:55.429051+0000","last_undegraded":"2023-10-18T10:40:55.429051+0000","last_fullsized":"2023-10-18T10:40:55.429051+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:45:58.336437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145071+0000","last_change":"2023-10-18T10:40:54.429476+0000","last_active":"2023-10-18T10:40:56.145071+0000","last_peered":"2023-10-18T10:40:56.145071+0000","last_clean":"2023-10-18T10:40:56.145071+0000","last_became_active":"2023-10-18T10:40:54.429109+0000","last_became_peered":"2023-10-18T10:40:54.429109+0000","last_unstale":"2023-10-18T10:40:56.145071+0000","last_undegraded":"2023-10-18T10:40:56.145071+0000","last_fullsized":"2023-10-18T10:40:56.145071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:58.443341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.428986+0000","last_change":"2023-10-18T10:40:53.425821+0000","last_active":"2023-10-18T10:40:55.428986+0000","last_peered":"2023-10-18T10:40:55.428986+0000","last_clean":"2023-10-18T10:40:55.428986+0000","last_became_active":"2023-10-18T10:40:53.425567+0000","last_became_peered":"2023-10-18T10:40:53.425567+0000","last_unstale":"2023-10-18T10:40:55.428986+0000","last_undegraded":"2023-10-18T10:40:55.428986+0000","last_fullsized":"2023-10-18T10:40:55.428986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:18.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500839+0000","last_change":"2023-10-18T10:40:53.424258+0000","last_active":"2023-10-18T10:40:55.500839+0000","last_peered":"2023-10-18T10:40:55.500839+0000","last_clean":"2023-10-18T10:40:55.500839+0000","last_became_active":"2023-10-18T10:40:53.423997+0000","last_became_peered":"2023-10-18T10:40:53.423997+0000","last_unstale":"2023-10-18T10:40:55.500839+0000","last_undegraded":"2023-10-18T10:40:55.500839+0000","last_fullsized":"2023-10-18T10:40:55.500839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:05.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144724+0000","last_change":"2023-10-18T10:40:54.433951+0000","last_active":"2023-10-18T10:40:56.144724+0000","last_peered":"2023-10-18T10:40:56.144724+0000","last_clean":"2023-10-18T10:40:56.144724+0000","last_became_active":"2023-10-18T10:40:54.433371+0000","last_became_peered":"2023-10-18T10:40:54.433371+0000","last_unstale":"2023-10-18T10:40:56.144724+0000","last_undegraded":"2023-10-18T10:40:56.144724+0000","last_fullsized":"2023-10-18T10:40:56.144724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:04.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144585+0000","last_change":"2023-10-18T10:40:53.431024+0000","last_active":"2023-10-18T10:40:56.144585+0000","last_peered":"2023-10-18T10:40:56.144585+0000","last_clean":"2023-10-18T10:40:56.144585+0000","last_became_active":"2023-10-18T10:40:53.430743+0000","last_became_peered":"2023-10-18T10:40:53.430743+0000","last_unstale":"2023-10-18T10:40:56.144585+0000","last_undegraded":"2023-10-18T10:40:56.144585+0000","last_fullsized":"2023-10-18T10:40:56.144585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:32.886192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144379+0000","last_change":"2023-10-18T10:40:54.431077+0000","last_active":"2023-10-18T10:40:56.144379+0000","last_peered":"2023-10-18T10:40:56.144379+0000","last_clean":"2023-10-18T10:40:56.144379+0000","last_became_active":"2023-10-18T10:40:54.430805+0000","last_became_peered":"2023-10-18T10:40:54.430805+0000","last_unstale":"2023-10-18T10:40:56.144379+0000","last_undegraded":"2023-10-18T10:40:56.144379+0000","last_fullsized":"2023-10-18T10:40:56.144379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:49:55.716517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144673+0000","last_change":"2023-10-18T10:40:54.433504+0000","last_active":"2023-10-18T10:40:56.144673+0000","last_peered":"2023-10-18T10:40:56.144673+0000","last_clean":"2023-10-18T10:40:56.144673+0000","last_became_active":"2023-10-18T10:40:54.433264+0000","last_became_peered":"2023-10-18T10:40:54.433264+0000","last_unstale":"2023-10-18T10:40:56.144673+0000","last_undegraded":"2023-10-18T10:40:56.144673+0000","last_fullsized":"2023-10-18T10:40:56.144673+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:07.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144446+0000","last_change":"2023-10-18T10:40:53.424988+0000","last_active":"2023-10-18T10:40:56.144446+0000","last_peered":"2023-10-18T10:40:56.144446+0000","last_clean":"2023-10-18T10:40:56.144446+0000","last_became_active":"2023-10-18T10:40:53.423255+0000","last_became_peered":"2023-10-18T10:40:53.423255+0000","last_unstale":"2023-10-18T10:40:56.144446+0000","last_undegraded":"2023-10-18T10:40:56.144446+0000","last_fullsized":"2023-10-18T10:40:56.144446+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:30.233098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144920+0000","last_change":"2023-10-18T10:40:54.434657+0000","last_active":"2023-10-18T10:40:56.144920+0000","last_peered":"2023-10-18T10:40:56.144920+0000","last_clean":"2023-10-18T10:40:56.144920+0000","last_became_active":"2023-10-18T10:40:54.434344+0000","last_became_peered":"2023-10-18T10:40:54.434344+0000","last_unstale":"2023-10-18T10:40:56.144920+0000","last_undegraded":"2023-10-18T10:40:56.144920+0000","last_fullsized":"2023-10-18T10:40:56.144920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:49.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018412+0000","last_change":"2023-10-18T10:40:53.419056+0000","last_active":"2023-10-18T10:40:56.018412+0000","last_peered":"2023-10-18T10:40:56.018412+0000","last_clean":"2023-10-18T10:40:56.018412+0000","last_became_active":"2023-10-18T10:40:53.418815+0000","last_became_peered":"2023-10-18T10:40:53.418815+0000","last_unstale":"2023-10-18T10:40:56.018412+0000","last_undegraded":"2023-10-18T10:40:56.018412+0000","last_fullsized":"2023-10-18T10:40:56.018412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:13.857868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144758+0000","last_change":"2023-10-18T10:40:53.431352+0000","last_active":"2023-10-18T10:40:56.144758+0000","last_peered":"2023-10-18T10:40:56.144758+0000","last_clean":"2023-10-18T10:40:56.144758+0000","last_became_active":"2023-10-18T10:40:53.431047+0000","last_became_peered":"2023-10-18T10:40:53.431047+0000","last_unstale":"2023-10-18T10:40:56.144758+0000","last_undegraded":"2023-10-18T10:40:56.144758+0000","last_fullsized":"2023-10-18T10:40:56.144758+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:17.609983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429479+0000","last_change":"2023-10-18T10:40:53.425307+0000","last_active":"2023-10-18T10:40:55.429479+0000","last_peered":"2023-10-18T10:40:55.429479+0000","last_clean":"2023-10-18T10:40:55.429479+0000","last_became_active":"2023-10-18T10:40:53.424835+0000","last_became_peered":"2023-10-18T10:40:53.424835+0000","last_unstale":"2023-10-18T10:40:55.429479+0000","last_undegraded":"2023-10-18T10:40:55.429479+0000","last_fullsized":"2023-10-18T10:40:55.429479+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:04.973143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"86'172","reported_seq":429,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T10:40:50.192957+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:50.192957+0000","last_peered":"2023-10-18T10:40:50.192957+0000","last_clean":"2023-10-18T10:40:50.192957+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:50.192957+0000","last_undegraded":"2023-10-18T10:40:50.192957+0000","last_fullsized":"2023-10-18T10:40:50.192957+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":86,"acting":86,"num_store_stats":5},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":42,"acting":42,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59252,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312588,"statfs":{"total":96636764160,"available":96576090112,"internally_reserved":0,"allocated":1744896,"data_stored":1162487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.64700000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.47799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.627}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.44900000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.56499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":593558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":1.016},"last":0.52800000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.80300000000000005,"15min":0.875},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.504}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.872,"15min":0.872},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66800000000000004,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36599999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":0.82199999999999995,"15min":0.87},"last":0.44500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":1.016},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.498},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.81000000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38300000000000001,"15min":0.28599999999999998},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.314,"15min":0.28499999999999998},"max":{"1min":0.71199999999999997,"5min":0.80100000000000005,"15min":1.262},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":1.0109999999999999,"15min":1.028},"last":0.61399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68400000000000005,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77800000000000002,"5min":0.96099999999999997,"15min":1.224},"last":0.59599999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65600000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.438,"5min":0.307,"15min":0.307},"max":{"1min":0.85299999999999998,"5min":0.871,"15min":1.159},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":572119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.877,"15min":0.877},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.495}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.502,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.72099999999999997,"5min":0.746,"15min":0.79600000000000004},"last":0.629}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75600000000000001,"5min":1.446,"15min":1.446},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.90400000000000003},"last":0.40200000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.82699999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.47499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":599148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.315}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.436},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1212416,"data_stored":599308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.224,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.192},"max":{"1min":0.51900000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.54600000000000004,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.30399999999999999,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.60199999999999998,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.502},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.626,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.82399999999999995,"5min":0.88200000000000001,"15min":1.458},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.83799999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.746}]},{"osd":5,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.89000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.93700000000000006,"15min":1.028},"last":0.68400000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.625,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.879,"15min":0.92500000000000004},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":572119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.33900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.377},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.497}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.433},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.57499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1781760,"data_stored":1182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.25},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.27700000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.37},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.315}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:57.708 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:57.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.842+0000 7fa2fb07c640 1 -- 172.21.15.125:0/3188333236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f40abc10 msgr2=0x7fa2f40c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.842+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/3188333236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f40abc10 0x7fa2f40c9b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009ca0 tx=0x7fa2ec02c2e0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.842+0000 7fa2fb07c640 1 -- 172.21.15.125:0/3188333236 shutdown_connections 2023-10-18T10:40:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.842+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/3188333236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f40abc10 0x7fa2f40c9b60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.842+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/3188333236 >> v2:172.21.15.139:3300/0 conn(0x7fa2f406af00 0x7fa2f406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.842+0000 7fa2fb07c640 1 -- 172.21.15.125:0/3188333236 >> 172.21.15.125:0/3188333236 conn(0x7fa2f4069930 msgr2=0x7fa2f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.842+0000 7fa2fb07c640 1 -- 172.21.15.125:0/3188333236 shutdown_connections 2023-10-18T10:40:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.842+0000 7fa2fb07c640 1 -- 172.21.15.125:0/3188333236 wait complete. 2023-10-18T10:40:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.843+0000 7fa2fb07c640 1 Processor -- start 2023-10-18T10:40:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.843+0000 7fa2fb07c640 1 -- start start 2023-10-18T10:40:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2fb07c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406af00 0x7fa2f414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2fb07c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f40abc10 0x7fa2f41e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2fb07c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f40c96b0 0x7fa2f41e3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2fb07c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40aff30 con 0x7fa2f40abc10 2023-10-18T10:40:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2fb07c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f4151fb0 con 0x7fa2f40c96b0 2023-10-18T10:40:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2fb07c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41e36c0 con 0x7fa2f406af00 2023-10-18T10:40:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f40abc10 0x7fa2f41e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f40abc10 0x7fa2f41e2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43318/0 (socket says 172.21.15.125:43318) 2023-10-18T10:40:57.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f3fff640 1 -- 172.21.15.125:0/4264148866 learned_addr learned my addr 172.21.15.125:0/4264148866 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:57.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f9619640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f40c96b0 0x7fa2f41e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f8e18640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406af00 0x7fa2f414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f3fff640 1 -- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406af00 msgr2=0x7fa2f414e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f3fff640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406af00 0x7fa2f414e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f3fff640 1 -- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f40c96b0 msgr2=0x7fa2f41e3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f3fff640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f40c96b0 0x7fa2f41e3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f3fff640 1 -- 172.21.15.125:0/4264148866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8009490 con 0x7fa2f40abc10 2023-10-18T10:40:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.844+0000 7fa2f8e18640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406af00 0x7fa2f414e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:40:57.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.845+0000 7fa2f3fff640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f40abc10 0x7fa2f41e2b20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c3c0 tx=0x7fa2e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:57.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.845+0000 7fa2f17fa640 1 -- 172.21.15.125:0/4264148866 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00052d0 con 0x7fa2f40abc10 2023-10-18T10:40:57.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.845+0000 7fa2f17fa640 1 -- 172.21.15.125:0/4264148866 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2e0007020 con 0x7fa2f40abc10 2023-10-18T10:40:57.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.845+0000 7fa2f17fa640 1 -- 172.21.15.125:0/4264148866 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0007430 con 0x7fa2f40abc10 2023-10-18T10:40:57.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.845+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec009900 con 0x7fa2f40abc10 2023-10-18T10:40:57.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.845+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f40af870 con 0x7fa2f40abc10 2023-10-18T10:40:57.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.846+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f40b0070 con 0x7fa2f40abc10 2023-10-18T10:40:57.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.846+0000 7fa2f17fa640 1 -- 172.21.15.125:0/4264148866 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa2e00067d0 con 0x7fa2f40abc10 2023-10-18T10:40:57.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.846+0000 7fa2f17fa640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa2d806af60 0x7fa2d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:57.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.847+0000 7fa2f17fa640 1 -- 172.21.15.125:0/4264148866 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa2e0088ee0 con 0x7fa2f40abc10 2023-10-18T10:40:57.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.847+0000 7fa2f8e18640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa2d806af60 0x7fa2d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:57.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.847+0000 7fa2f8e18640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa2d806af60 0x7fa2d806d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009ce0 tx=0x7fa2e8005f90 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:57.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:57.851+0000 7fa2f17fa640 1 -- 172.21.15.125:0/4264148866 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2e0053cc0 con 0x7fa2f40abc10 2023-10-18T10:40:58.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.015+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 --> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2f4159c50 con 0x7fa2d806af60 2023-10-18T10:40:58.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.023+0000 7fa2f17fa640 1 -- 172.21.15.125:0/4264148866 <== mgr.5972 v2:172.21.15.139:6832/143413153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284363 (secure 0 0 0) 0x7fa2e8009950 con 0x7fa2d806af60 2023-10-18T10:40:58.028 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:58.029 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:58.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa2d806af60 msgr2=0x7fa2d806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa2d806af60 0x7fa2d806d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009ce0 tx=0x7fa2e8005f90 comp rx=0 tx=0).stop 2023-10-18T10:40:58.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f40abc10 msgr2=0x7fa2f41e2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f40abc10 0x7fa2f41e2b20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c3c0 tx=0x7fa2e000c880 comp rx=0 tx=0).stop 2023-10-18T10:40:58.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 shutdown_connections 2023-10-18T10:40:58.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fa2d806af60 0x7fa2d806d420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f40c96b0 0x7fa2f41e3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f40abc10 0x7fa2f41e2b20 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.026+0000 7fa2fb07c640 1 --2- 172.21.15.125:0/4264148866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406af00 0x7fa2f414e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.027+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 >> 172.21.15.125:0/4264148866 conn(0x7fa2f4069930 msgr2=0x7fa2f4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:58.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.027+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 shutdown_connections 2023-10-18T10:40:58.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.027+0000 7fa2fb07c640 1 -- 172.21.15.125:0/4264148866 wait complete. 2023-10-18T10:40:58.039 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:40:56.162612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":296,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":312608,"kb_used_data":10400,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754662112,"statfs":{"total":773094113280,"available":772774002688,"internally_reserved":0,"allocated":10649600,"data_stored":5879653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001032"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144511+0000","last_change":"2023-10-18T10:40:54.435140+0000","last_active":"2023-10-18T10:40:56.144511+0000","last_peered":"2023-10-18T10:40:56.144511+0000","last_clean":"2023-10-18T10:40:56.144511+0000","last_became_active":"2023-10-18T10:40:54.434657+0000","last_became_peered":"2023-10-18T10:40:54.434657+0000","last_unstale":"2023-10-18T10:40:56.144511+0000","last_undegraded":"2023-10-18T10:40:56.144511+0000","last_fullsized":"2023-10-18T10:40:56.144511+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.896314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500661+0000","last_change":"2023-10-18T10:40:54.428671+0000","last_active":"2023-10-18T10:40:55.500661+0000","last_peered":"2023-10-18T10:40:55.500661+0000","last_clean":"2023-10-18T10:40:55.500661+0000","last_became_active":"2023-10-18T10:40:54.428396+0000","last_became_peered":"2023-10-18T10:40:54.428396+0000","last_unstale":"2023-10-18T10:40:55.500661+0000","last_undegraded":"2023-10-18T10:40:55.500661+0000","last_fullsized":"2023-10-18T10:40:55.500661+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:39.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018418+0000","last_change":"2023-10-18T10:40:54.433351+0000","last_active":"2023-10-18T10:40:56.018418+0000","last_peered":"2023-10-18T10:40:56.018418+0000","last_clean":"2023-10-18T10:40:56.018418+0000","last_became_active":"2023-10-18T10:40:54.430314+0000","last_became_peered":"2023-10-18T10:40:54.430314+0000","last_unstale":"2023-10-18T10:40:56.018418+0000","last_undegraded":"2023-10-18T10:40:56.018418+0000","last_fullsized":"2023-10-18T10:40:56.018418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:48.808404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429115+0000","last_change":"2023-10-18T10:40:54.431510+0000","last_active":"2023-10-18T10:40:55.429115+0000","last_peered":"2023-10-18T10:40:55.429115+0000","last_clean":"2023-10-18T10:40:55.429115+0000","last_became_active":"2023-10-18T10:40:54.431184+0000","last_became_peered":"2023-10-18T10:40:54.431184+0000","last_unstale":"2023-10-18T10:40:55.429115+0000","last_undegraded":"2023-10-18T10:40:55.429115+0000","last_fullsized":"2023-10-18T10:40:55.429115+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:33.893983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144553+0000","last_change":"2023-10-18T10:40:54.434704+0000","last_active":"2023-10-18T10:40:56.144553+0000","last_peered":"2023-10-18T10:40:56.144553+0000","last_clean":"2023-10-18T10:40:56.144553+0000","last_became_active":"2023-10-18T10:40:54.434388+0000","last_became_peered":"2023-10-18T10:40:54.434388+0000","last_unstale":"2023-10-18T10:40:56.144553+0000","last_undegraded":"2023-10-18T10:40:56.144553+0000","last_fullsized":"2023-10-18T10:40:56.144553+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:54.346618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:54.427003+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:54.426743+0000","last_became_peered":"2023-10-18T10:40:54.426743+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:58.361349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500867+0000","last_change":"2023-10-18T10:40:54.433837+0000","last_active":"2023-10-18T10:40:55.500867+0000","last_peered":"2023-10-18T10:40:55.500867+0000","last_clean":"2023-10-18T10:40:55.500867+0000","last_became_active":"2023-10-18T10:40:54.433558+0000","last_became_peered":"2023-10-18T10:40:54.433558+0000","last_unstale":"2023-10-18T10:40:55.500867+0000","last_undegraded":"2023-10-18T10:40:55.500867+0000","last_fullsized":"2023-10-18T10:40:55.500867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:04.849763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144518+0000","last_change":"2023-10-18T10:40:54.429847+0000","last_active":"2023-10-18T10:40:56.144518+0000","last_peered":"2023-10-18T10:40:56.144518+0000","last_clean":"2023-10-18T10:40:56.144518+0000","last_became_active":"2023-10-18T10:40:54.429629+0000","last_became_peered":"2023-10-18T10:40:54.429629+0000","last_unstale":"2023-10-18T10:40:56.144518+0000","last_undegraded":"2023-10-18T10:40:56.144518+0000","last_fullsized":"2023-10-18T10:40:56.144518+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:29.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500804+0000","last_change":"2023-10-18T10:40:54.427881+0000","last_active":"2023-10-18T10:40:55.500804+0000","last_peered":"2023-10-18T10:40:55.500804+0000","last_clean":"2023-10-18T10:40:55.500804+0000","last_became_active":"2023-10-18T10:40:54.427423+0000","last_became_peered":"2023-10-18T10:40:54.427423+0000","last_unstale":"2023-10-18T10:40:55.500804+0000","last_undegraded":"2023-10-18T10:40:55.500804+0000","last_fullsized":"2023-10-18T10:40:55.500804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:07.682253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018526+0000","last_change":"2023-10-18T10:40:54.426415+0000","last_active":"2023-10-18T10:40:56.018526+0000","last_peered":"2023-10-18T10:40:56.018526+0000","last_clean":"2023-10-18T10:40:56.018526+0000","last_became_active":"2023-10-18T10:40:54.426129+0000","last_became_peered":"2023-10-18T10:40:54.426129+0000","last_unstale":"2023-10-18T10:40:56.018526+0000","last_undegraded":"2023-10-18T10:40:56.018526+0000","last_fullsized":"2023-10-18T10:40:56.018526+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:33:27.749058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429139+0000","last_change":"2023-10-18T10:40:54.430848+0000","last_active":"2023-10-18T10:40:55.429139+0000","last_peered":"2023-10-18T10:40:55.429139+0000","last_clean":"2023-10-18T10:40:55.429139+0000","last_became_active":"2023-10-18T10:40:54.430562+0000","last_became_peered":"2023-10-18T10:40:54.430562+0000","last_unstale":"2023-10-18T10:40:55.429139+0000","last_undegraded":"2023-10-18T10:40:55.429139+0000","last_fullsized":"2023-10-18T10:40:55.429139+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:05.564173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144611+0000","last_change":"2023-10-18T10:40:54.421817+0000","last_active":"2023-10-18T10:40:56.144611+0000","last_peered":"2023-10-18T10:40:56.144611+0000","last_clean":"2023-10-18T10:40:56.144611+0000","last_became_active":"2023-10-18T10:40:54.421613+0000","last_became_peered":"2023-10-18T10:40:54.421613+0000","last_unstale":"2023-10-18T10:40:56.144611+0000","last_undegraded":"2023-10-18T10:40:56.144611+0000","last_fullsized":"2023-10-18T10:40:56.144611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:36.706863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:54.435303+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:54.434740+0000","last_became_peered":"2023-10-18T10:40:54.434740+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.386244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429236+0000","last_change":"2023-10-18T10:40:54.432868+0000","last_active":"2023-10-18T10:40:55.429236+0000","last_peered":"2023-10-18T10:40:55.429236+0000","last_clean":"2023-10-18T10:40:55.429236+0000","last_became_active":"2023-10-18T10:40:54.432637+0000","last_became_peered":"2023-10-18T10:40:54.432637+0000","last_unstale":"2023-10-18T10:40:55.429236+0000","last_undegraded":"2023-10-18T10:40:55.429236+0000","last_fullsized":"2023-10-18T10:40:55.429236+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:46.951599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018489+0000","last_change":"2023-10-18T10:40:54.430205+0000","last_active":"2023-10-18T10:40:56.018489+0000","last_peered":"2023-10-18T10:40:56.018489+0000","last_clean":"2023-10-18T10:40:56.018489+0000","last_became_active":"2023-10-18T10:40:54.429898+0000","last_became_peered":"2023-10-18T10:40:54.429898+0000","last_unstale":"2023-10-18T10:40:56.018489+0000","last_undegraded":"2023-10-18T10:40:56.018489+0000","last_fullsized":"2023-10-18T10:40:56.018489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:48.646925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018318+0000","last_change":"2023-10-18T10:40:54.428004+0000","last_active":"2023-10-18T10:40:56.018318+0000","last_peered":"2023-10-18T10:40:56.018318+0000","last_clean":"2023-10-18T10:40:56.018318+0000","last_became_active":"2023-10-18T10:40:54.427533+0000","last_became_peered":"2023-10-18T10:40:54.427533+0000","last_unstale":"2023-10-18T10:40:56.018318+0000","last_undegraded":"2023-10-18T10:40:56.018318+0000","last_fullsized":"2023-10-18T10:40:56.018318+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:27:07.146678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143943+0000","last_change":"2023-10-18T10:40:54.424777+0000","last_active":"2023-10-18T10:40:56.143943+0000","last_peered":"2023-10-18T10:40:56.143943+0000","last_clean":"2023-10-18T10:40:56.143943+0000","last_became_active":"2023-10-18T10:40:54.424544+0000","last_became_peered":"2023-10-18T10:40:54.424544+0000","last_unstale":"2023-10-18T10:40:56.143943+0000","last_undegraded":"2023-10-18T10:40:56.143943+0000","last_fullsized":"2023-10-18T10:40:56.143943+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144475+0000","last_change":"2023-10-18T10:40:54.425355+0000","last_active":"2023-10-18T10:40:56.144475+0000","last_peered":"2023-10-18T10:40:56.144475+0000","last_clean":"2023-10-18T10:40:56.144475+0000","last_became_active":"2023-10-18T10:40:54.425123+0000","last_became_peered":"2023-10-18T10:40:54.425123+0000","last_unstale":"2023-10-18T10:40:56.144475+0000","last_undegraded":"2023-10-18T10:40:56.144475+0000","last_fullsized":"2023-10-18T10:40:56.144475+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:24.991706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500878+0000","last_change":"2023-10-18T10:40:54.426993+0000","last_active":"2023-10-18T10:40:55.500878+0000","last_peered":"2023-10-18T10:40:55.500878+0000","last_clean":"2023-10-18T10:40:55.500878+0000","last_became_active":"2023-10-18T10:40:54.426702+0000","last_became_peered":"2023-10-18T10:40:54.426702+0000","last_unstale":"2023-10-18T10:40:55.500878+0000","last_undegraded":"2023-10-18T10:40:55.500878+0000","last_fullsized":"2023-10-18T10:40:55.500878+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:57.667741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018550+0000","last_change":"2023-10-18T10:40:54.432406+0000","last_active":"2023-10-18T10:40:56.018550+0000","last_peered":"2023-10-18T10:40:56.018550+0000","last_clean":"2023-10-18T10:40:56.018550+0000","last_became_active":"2023-10-18T10:40:54.432113+0000","last_became_peered":"2023-10-18T10:40:54.432113+0000","last_unstale":"2023-10-18T10:40:56.018550+0000","last_undegraded":"2023-10-18T10:40:56.018550+0000","last_fullsized":"2023-10-18T10:40:56.018550+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:24:16.099369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144250+0000","last_change":"2023-10-18T10:40:54.431844+0000","last_active":"2023-10-18T10:40:56.144250+0000","last_peered":"2023-10-18T10:40:56.144250+0000","last_clean":"2023-10-18T10:40:56.144250+0000","last_became_active":"2023-10-18T10:40:54.431447+0000","last_became_peered":"2023-10-18T10:40:54.431447+0000","last_unstale":"2023-10-18T10:40:56.144250+0000","last_undegraded":"2023-10-18T10:40:56.144250+0000","last_fullsized":"2023-10-18T10:40:56.144250+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:30.965000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"97'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:53.417631+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:53.417381+0000","last_became_peered":"2023-10-18T10:40:53.417381+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:40.271808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144091+0000","last_change":"2023-10-18T10:40:54.420096+0000","last_active":"2023-10-18T10:40:56.144091+0000","last_peered":"2023-10-18T10:40:56.144091+0000","last_clean":"2023-10-18T10:40:56.144091+0000","last_became_active":"2023-10-18T10:40:54.419838+0000","last_became_peered":"2023-10-18T10:40:54.419838+0000","last_unstale":"2023-10-18T10:40:56.144091+0000","last_undegraded":"2023-10-18T10:40:56.144091+0000","last_fullsized":"2023-10-18T10:40:56.144091+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:17.947505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144791+0000","last_change":"2023-10-18T10:40:53.423111+0000","last_active":"2023-10-18T10:40:56.144791+0000","last_peered":"2023-10-18T10:40:56.144791+0000","last_clean":"2023-10-18T10:40:56.144791+0000","last_became_active":"2023-10-18T10:40:53.422848+0000","last_became_peered":"2023-10-18T10:40:53.422848+0000","last_unstale":"2023-10-18T10:40:56.144791+0000","last_undegraded":"2023-10-18T10:40:56.144791+0000","last_fullsized":"2023-10-18T10:40:56.144791+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:19.845504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144374+0000","last_change":"2023-10-18T10:40:54.435357+0000","last_active":"2023-10-18T10:40:56.144374+0000","last_peered":"2023-10-18T10:40:56.144374+0000","last_clean":"2023-10-18T10:40:56.144374+0000","last_became_active":"2023-10-18T10:40:54.435102+0000","last_became_peered":"2023-10-18T10:40:54.435102+0000","last_unstale":"2023-10-18T10:40:56.144374+0000","last_undegraded":"2023-10-18T10:40:56.144374+0000","last_fullsized":"2023-10-18T10:40:56.144374+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:28:23.536390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144627+0000","last_change":"2023-10-18T10:40:54.428864+0000","last_active":"2023-10-18T10:40:56.144627+0000","last_peered":"2023-10-18T10:40:56.144627+0000","last_clean":"2023-10-18T10:40:56.144627+0000","last_became_active":"2023-10-18T10:40:54.428412+0000","last_became_peered":"2023-10-18T10:40:54.428412+0000","last_unstale":"2023-10-18T10:40:56.144627+0000","last_undegraded":"2023-10-18T10:40:56.144627+0000","last_fullsized":"2023-10-18T10:40:56.144627+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:15.038645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500519+0000","last_change":"2023-10-18T10:40:53.420317+0000","last_active":"2023-10-18T10:40:55.500519+0000","last_peered":"2023-10-18T10:40:55.500519+0000","last_clean":"2023-10-18T10:40:55.500519+0000","last_became_active":"2023-10-18T10:40:53.420046+0000","last_became_peered":"2023-10-18T10:40:53.420046+0000","last_unstale":"2023-10-18T10:40:55.500519+0000","last_undegraded":"2023-10-18T10:40:55.500519+0000","last_fullsized":"2023-10-18T10:40:55.500519+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:36.623092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144181+0000","last_change":"2023-10-18T10:40:53.430760+0000","last_active":"2023-10-18T10:40:56.144181+0000","last_peered":"2023-10-18T10:40:56.144181+0000","last_clean":"2023-10-18T10:40:56.144181+0000","last_became_active":"2023-10-18T10:40:53.430469+0000","last_became_peered":"2023-10-18T10:40:53.430469+0000","last_unstale":"2023-10-18T10:40:56.144181+0000","last_undegraded":"2023-10-18T10:40:56.144181+0000","last_fullsized":"2023-10-18T10:40:56.144181+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:45.943807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144991+0000","last_change":"2023-10-18T10:40:54.435078+0000","last_active":"2023-10-18T10:40:56.144991+0000","last_peered":"2023-10-18T10:40:56.144991+0000","last_clean":"2023-10-18T10:40:56.144991+0000","last_became_active":"2023-10-18T10:40:54.434517+0000","last_became_peered":"2023-10-18T10:40:54.434517+0000","last_unstale":"2023-10-18T10:40:56.144991+0000","last_undegraded":"2023-10-18T10:40:56.144991+0000","last_fullsized":"2023-10-18T10:40:56.144991+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:40.219033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:53.415948+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:53.415668+0000","last_became_peered":"2023-10-18T10:40:53.415668+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:12.409497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144074+0000","last_change":"2023-10-18T10:40:54.426165+0000","last_active":"2023-10-18T10:40:56.144074+0000","last_peered":"2023-10-18T10:40:56.144074+0000","last_clean":"2023-10-18T10:40:56.144074+0000","last_became_active":"2023-10-18T10:40:54.425916+0000","last_became_peered":"2023-10-18T10:40:54.425916+0000","last_unstale":"2023-10-18T10:40:56.144074+0000","last_undegraded":"2023-10-18T10:40:56.144074+0000","last_fullsized":"2023-10-18T10:40:56.144074+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500662+0000","last_change":"2023-10-18T10:40:54.428238+0000","last_active":"2023-10-18T10:40:55.500662+0000","last_peered":"2023-10-18T10:40:55.500662+0000","last_clean":"2023-10-18T10:40:55.500662+0000","last_became_active":"2023-10-18T10:40:54.427404+0000","last_became_peered":"2023-10-18T10:40:54.427404+0000","last_unstale":"2023-10-18T10:40:55.500662+0000","last_undegraded":"2023-10-18T10:40:55.500662+0000","last_fullsized":"2023-10-18T10:40:55.500662+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:06.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500959+0000","last_change":"2023-10-18T10:40:53.423540+0000","last_active":"2023-10-18T10:40:55.500959+0000","last_peered":"2023-10-18T10:40:55.500959+0000","last_clean":"2023-10-18T10:40:55.500959+0000","last_became_active":"2023-10-18T10:40:53.423278+0000","last_became_peered":"2023-10-18T10:40:53.423278+0000","last_unstale":"2023-10-18T10:40:55.500959+0000","last_undegraded":"2023-10-18T10:40:55.500959+0000","last_fullsized":"2023-10-18T10:40:55.500959+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:15.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429349+0000","last_change":"2023-10-18T10:40:54.423285+0000","last_active":"2023-10-18T10:40:55.429349+0000","last_peered":"2023-10-18T10:40:55.429349+0000","last_clean":"2023-10-18T10:40:55.429349+0000","last_became_active":"2023-10-18T10:40:54.423013+0000","last_became_peered":"2023-10-18T10:40:54.423013+0000","last_unstale":"2023-10-18T10:40:55.429349+0000","last_undegraded":"2023-10-18T10:40:55.429349+0000","last_fullsized":"2023-10-18T10:40:55.429349+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"97'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018642+0000","last_change":"2023-10-18T10:40:53.424070+0000","last_active":"2023-10-18T10:40:56.018642+0000","last_peered":"2023-10-18T10:40:56.018642+0000","last_clean":"2023-10-18T10:40:56.018642+0000","last_became_active":"2023-10-18T10:40:53.423799+0000","last_became_peered":"2023-10-18T10:40:53.423799+0000","last_unstale":"2023-10-18T10:40:56.018642+0000","last_undegraded":"2023-10-18T10:40:56.018642+0000","last_fullsized":"2023-10-18T10:40:56.018642+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:09.891958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"97'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144832+0000","last_change":"2023-10-18T10:40:53.422980+0000","last_active":"2023-10-18T10:40:56.144832+0000","last_peered":"2023-10-18T10:40:56.144832+0000","last_clean":"2023-10-18T10:40:56.144832+0000","last_became_active":"2023-10-18T10:40:53.422696+0000","last_became_peered":"2023-10-18T10:40:53.422696+0000","last_unstale":"2023-10-18T10:40:56.144832+0000","last_undegraded":"2023-10-18T10:40:56.144832+0000","last_fullsized":"2023-10-18T10:40:56.144832+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:20.506905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500893+0000","last_change":"2023-10-18T10:40:54.423164+0000","last_active":"2023-10-18T10:40:55.500893+0000","last_peered":"2023-10-18T10:40:55.500893+0000","last_clean":"2023-10-18T10:40:55.500893+0000","last_became_active":"2023-10-18T10:40:54.422844+0000","last_became_peered":"2023-10-18T10:40:54.422844+0000","last_unstale":"2023-10-18T10:40:55.500893+0000","last_undegraded":"2023-10-18T10:40:55.500893+0000","last_fullsized":"2023-10-18T10:40:55.500893+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:50.691881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500565+0000","last_change":"2023-10-18T10:40:53.420785+0000","last_active":"2023-10-18T10:40:55.500565+0000","last_peered":"2023-10-18T10:40:55.500565+0000","last_clean":"2023-10-18T10:40:55.500565+0000","last_became_active":"2023-10-18T10:40:53.420530+0000","last_became_peered":"2023-10-18T10:40:53.420530+0000","last_unstale":"2023-10-18T10:40:55.500565+0000","last_undegraded":"2023-10-18T10:40:55.500565+0000","last_fullsized":"2023-10-18T10:40:55.500565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:44.748900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144877+0000","last_change":"2023-10-18T10:40:54.430742+0000","last_active":"2023-10-18T10:40:56.144877+0000","last_peered":"2023-10-18T10:40:56.144877+0000","last_clean":"2023-10-18T10:40:56.144877+0000","last_became_active":"2023-10-18T10:40:54.430455+0000","last_became_peered":"2023-10-18T10:40:54.430455+0000","last_unstale":"2023-10-18T10:40:56.144877+0000","last_undegraded":"2023-10-18T10:40:56.144877+0000","last_fullsized":"2023-10-18T10:40:56.144877+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:39.892546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144941+0000","last_change":"2023-10-18T10:40:54.430021+0000","last_active":"2023-10-18T10:40:56.144941+0000","last_peered":"2023-10-18T10:40:56.144941+0000","last_clean":"2023-10-18T10:40:56.144941+0000","last_became_active":"2023-10-18T10:40:54.429684+0000","last_became_peered":"2023-10-18T10:40:54.429684+0000","last_unstale":"2023-10-18T10:40:56.144941+0000","last_undegraded":"2023-10-18T10:40:56.144941+0000","last_fullsized":"2023-10-18T10:40:56.144941+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:03.648375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018334+0000","last_change":"2023-10-18T10:40:53.425774+0000","last_active":"2023-10-18T10:40:56.018334+0000","last_peered":"2023-10-18T10:40:56.018334+0000","last_clean":"2023-10-18T10:40:56.018334+0000","last_became_active":"2023-10-18T10:40:53.425518+0000","last_became_peered":"2023-10-18T10:40:53.425518+0000","last_unstale":"2023-10-18T10:40:56.018334+0000","last_undegraded":"2023-10-18T10:40:56.018334+0000","last_fullsized":"2023-10-18T10:40:56.018334+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:30.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145025+0000","last_change":"2023-10-18T10:40:54.432786+0000","last_active":"2023-10-18T10:40:56.145025+0000","last_peered":"2023-10-18T10:40:56.145025+0000","last_clean":"2023-10-18T10:40:56.145025+0000","last_became_active":"2023-10-18T10:40:54.432538+0000","last_became_peered":"2023-10-18T10:40:54.432538+0000","last_unstale":"2023-10-18T10:40:56.145025+0000","last_undegraded":"2023-10-18T10:40:56.145025+0000","last_fullsized":"2023-10-18T10:40:56.145025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:52:39.855703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"97'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144160+0000","last_change":"2023-10-18T10:40:53.426701+0000","last_active":"2023-10-18T10:40:56.144160+0000","last_peered":"2023-10-18T10:40:56.144160+0000","last_clean":"2023-10-18T10:40:56.144160+0000","last_became_active":"2023-10-18T10:40:53.426431+0000","last_became_peered":"2023-10-18T10:40:53.426431+0000","last_unstale":"2023-10-18T10:40:56.144160+0000","last_undegraded":"2023-10-18T10:40:56.144160+0000","last_fullsized":"2023-10-18T10:40:56.144160+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:31.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429351+0000","last_change":"2023-10-18T10:40:54.434075+0000","last_active":"2023-10-18T10:40:55.429351+0000","last_peered":"2023-10-18T10:40:55.429351+0000","last_clean":"2023-10-18T10:40:55.429351+0000","last_became_active":"2023-10-18T10:40:54.433835+0000","last_became_peered":"2023-10-18T10:40:54.433835+0000","last_unstale":"2023-10-18T10:40:55.429351+0000","last_undegraded":"2023-10-18T10:40:55.429351+0000","last_fullsized":"2023-10-18T10:40:55.429351+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018402+0000","last_change":"2023-10-18T10:40:54.435840+0000","last_active":"2023-10-18T10:40:56.018402+0000","last_peered":"2023-10-18T10:40:56.018402+0000","last_clean":"2023-10-18T10:40:56.018402+0000","last_became_active":"2023-10-18T10:40:54.435594+0000","last_became_peered":"2023-10-18T10:40:54.435594+0000","last_unstale":"2023-10-18T10:40:56.018402+0000","last_undegraded":"2023-10-18T10:40:56.018402+0000","last_fullsized":"2023-10-18T10:40:56.018402+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:16:01.996675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018391+0000","last_change":"2023-10-18T10:40:54.427794+0000","last_active":"2023-10-18T10:40:56.018391+0000","last_peered":"2023-10-18T10:40:56.018391+0000","last_clean":"2023-10-18T10:40:56.018391+0000","last_became_active":"2023-10-18T10:40:54.427255+0000","last_became_peered":"2023-10-18T10:40:54.427255+0000","last_unstale":"2023-10-18T10:40:56.018391+0000","last_undegraded":"2023-10-18T10:40:56.018391+0000","last_fullsized":"2023-10-18T10:40:56.018391+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:08.181951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018673+0000","last_change":"2023-10-18T10:40:53.416910+0000","last_active":"2023-10-18T10:40:56.018673+0000","last_peered":"2023-10-18T10:40:56.018673+0000","last_clean":"2023-10-18T10:40:56.018673+0000","last_became_active":"2023-10-18T10:40:53.416630+0000","last_became_peered":"2023-10-18T10:40:53.416630+0000","last_unstale":"2023-10-18T10:40:56.018673+0000","last_undegraded":"2023-10-18T10:40:56.018673+0000","last_fullsized":"2023-10-18T10:40:56.018673+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:24.474032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429270+0000","last_change":"2023-10-18T10:40:54.439124+0000","last_active":"2023-10-18T10:40:55.429270+0000","last_peered":"2023-10-18T10:40:55.429270+0000","last_clean":"2023-10-18T10:40:55.429270+0000","last_became_active":"2023-10-18T10:40:54.438869+0000","last_became_peered":"2023-10-18T10:40:54.438869+0000","last_unstale":"2023-10-18T10:40:55.429270+0000","last_undegraded":"2023-10-18T10:40:55.429270+0000","last_fullsized":"2023-10-18T10:40:55.429270+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:50.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144441+0000","last_change":"2023-10-18T10:40:54.420623+0000","last_active":"2023-10-18T10:40:56.144441+0000","last_peered":"2023-10-18T10:40:56.144441+0000","last_clean":"2023-10-18T10:40:56.144441+0000","last_became_active":"2023-10-18T10:40:54.420398+0000","last_became_peered":"2023-10-18T10:40:54.420398+0000","last_unstale":"2023-10-18T10:40:56.144441+0000","last_undegraded":"2023-10-18T10:40:56.144441+0000","last_fullsized":"2023-10-18T10:40:56.144441+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.611760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429356+0000","last_change":"2023-10-18T10:40:53.424629+0000","last_active":"2023-10-18T10:40:55.429356+0000","last_peered":"2023-10-18T10:40:55.429356+0000","last_clean":"2023-10-18T10:40:55.429356+0000","last_became_active":"2023-10-18T10:40:53.424387+0000","last_became_peered":"2023-10-18T10:40:53.424387+0000","last_unstale":"2023-10-18T10:40:55.429356+0000","last_undegraded":"2023-10-18T10:40:55.429356+0000","last_fullsized":"2023-10-18T10:40:55.429356+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:54.556080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144592+0000","last_change":"2023-10-18T10:40:54.427692+0000","last_active":"2023-10-18T10:40:56.144592+0000","last_peered":"2023-10-18T10:40:56.144592+0000","last_clean":"2023-10-18T10:40:56.144592+0000","last_became_active":"2023-10-18T10:40:54.427408+0000","last_became_peered":"2023-10-18T10:40:54.427408+0000","last_unstale":"2023-10-18T10:40:56.144592+0000","last_undegraded":"2023-10-18T10:40:56.144592+0000","last_fullsized":"2023-10-18T10:40:56.144592+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:47:48.326742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429051+0000","last_change":"2023-10-18T10:40:53.426213+0000","last_active":"2023-10-18T10:40:55.429051+0000","last_peered":"2023-10-18T10:40:55.429051+0000","last_clean":"2023-10-18T10:40:55.429051+0000","last_became_active":"2023-10-18T10:40:53.425947+0000","last_became_peered":"2023-10-18T10:40:53.425947+0000","last_unstale":"2023-10-18T10:40:55.429051+0000","last_undegraded":"2023-10-18T10:40:55.429051+0000","last_fullsized":"2023-10-18T10:40:55.429051+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:45:58.336437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145071+0000","last_change":"2023-10-18T10:40:54.429476+0000","last_active":"2023-10-18T10:40:56.145071+0000","last_peered":"2023-10-18T10:40:56.145071+0000","last_clean":"2023-10-18T10:40:56.145071+0000","last_became_active":"2023-10-18T10:40:54.429109+0000","last_became_peered":"2023-10-18T10:40:54.429109+0000","last_unstale":"2023-10-18T10:40:56.145071+0000","last_undegraded":"2023-10-18T10:40:56.145071+0000","last_fullsized":"2023-10-18T10:40:56.145071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:58.443341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.428986+0000","last_change":"2023-10-18T10:40:53.425821+0000","last_active":"2023-10-18T10:40:55.428986+0000","last_peered":"2023-10-18T10:40:55.428986+0000","last_clean":"2023-10-18T10:40:55.428986+0000","last_became_active":"2023-10-18T10:40:53.425567+0000","last_became_peered":"2023-10-18T10:40:53.425567+0000","last_unstale":"2023-10-18T10:40:55.428986+0000","last_undegraded":"2023-10-18T10:40:55.428986+0000","last_fullsized":"2023-10-18T10:40:55.428986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:18.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500839+0000","last_change":"2023-10-18T10:40:53.424258+0000","last_active":"2023-10-18T10:40:55.500839+0000","last_peered":"2023-10-18T10:40:55.500839+0000","last_clean":"2023-10-18T10:40:55.500839+0000","last_became_active":"2023-10-18T10:40:53.423997+0000","last_became_peered":"2023-10-18T10:40:53.423997+0000","last_unstale":"2023-10-18T10:40:55.500839+0000","last_undegraded":"2023-10-18T10:40:55.500839+0000","last_fullsized":"2023-10-18T10:40:55.500839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:05.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144724+0000","last_change":"2023-10-18T10:40:54.433951+0000","last_active":"2023-10-18T10:40:56.144724+0000","last_peered":"2023-10-18T10:40:56.144724+0000","last_clean":"2023-10-18T10:40:56.144724+0000","last_became_active":"2023-10-18T10:40:54.433371+0000","last_became_peered":"2023-10-18T10:40:54.433371+0000","last_unstale":"2023-10-18T10:40:56.144724+0000","last_undegraded":"2023-10-18T10:40:56.144724+0000","last_fullsized":"2023-10-18T10:40:56.144724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:04.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144585+0000","last_change":"2023-10-18T10:40:53.431024+0000","last_active":"2023-10-18T10:40:56.144585+0000","last_peered":"2023-10-18T10:40:56.144585+0000","last_clean":"2023-10-18T10:40:56.144585+0000","last_became_active":"2023-10-18T10:40:53.430743+0000","last_became_peered":"2023-10-18T10:40:53.430743+0000","last_unstale":"2023-10-18T10:40:56.144585+0000","last_undegraded":"2023-10-18T10:40:56.144585+0000","last_fullsized":"2023-10-18T10:40:56.144585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:32.886192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144379+0000","last_change":"2023-10-18T10:40:54.431077+0000","last_active":"2023-10-18T10:40:56.144379+0000","last_peered":"2023-10-18T10:40:56.144379+0000","last_clean":"2023-10-18T10:40:56.144379+0000","last_became_active":"2023-10-18T10:40:54.430805+0000","last_became_peered":"2023-10-18T10:40:54.430805+0000","last_unstale":"2023-10-18T10:40:56.144379+0000","last_undegraded":"2023-10-18T10:40:56.144379+0000","last_fullsized":"2023-10-18T10:40:56.144379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:49:55.716517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144673+0000","last_change":"2023-10-18T10:40:54.433504+0000","last_active":"2023-10-18T10:40:56.144673+0000","last_peered":"2023-10-18T10:40:56.144673+0000","last_clean":"2023-10-18T10:40:56.144673+0000","last_became_active":"2023-10-18T10:40:54.433264+0000","last_became_peered":"2023-10-18T10:40:54.433264+0000","last_unstale":"2023-10-18T10:40:56.144673+0000","last_undegraded":"2023-10-18T10:40:56.144673+0000","last_fullsized":"2023-10-18T10:40:56.144673+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:07.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144446+0000","last_change":"2023-10-18T10:40:53.424988+0000","last_active":"2023-10-18T10:40:56.144446+0000","last_peered":"2023-10-18T10:40:56.144446+0000","last_clean":"2023-10-18T10:40:56.144446+0000","last_became_active":"2023-10-18T10:40:53.423255+0000","last_became_peered":"2023-10-18T10:40:53.423255+0000","last_unstale":"2023-10-18T10:40:56.144446+0000","last_undegraded":"2023-10-18T10:40:56.144446+0000","last_fullsized":"2023-10-18T10:40:56.144446+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:30.233098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144920+0000","last_change":"2023-10-18T10:40:54.434657+0000","last_active":"2023-10-18T10:40:56.144920+0000","last_peered":"2023-10-18T10:40:56.144920+0000","last_clean":"2023-10-18T10:40:56.144920+0000","last_became_active":"2023-10-18T10:40:54.434344+0000","last_became_peered":"2023-10-18T10:40:54.434344+0000","last_unstale":"2023-10-18T10:40:56.144920+0000","last_undegraded":"2023-10-18T10:40:56.144920+0000","last_fullsized":"2023-10-18T10:40:56.144920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:49.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018412+0000","last_change":"2023-10-18T10:40:53.419056+0000","last_active":"2023-10-18T10:40:56.018412+0000","last_peered":"2023-10-18T10:40:56.018412+0000","last_clean":"2023-10-18T10:40:56.018412+0000","last_became_active":"2023-10-18T10:40:53.418815+0000","last_became_peered":"2023-10-18T10:40:53.418815+0000","last_unstale":"2023-10-18T10:40:56.018412+0000","last_undegraded":"2023-10-18T10:40:56.018412+0000","last_fullsized":"2023-10-18T10:40:56.018412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:13.857868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144758+0000","last_change":"2023-10-18T10:40:53.431352+0000","last_active":"2023-10-18T10:40:56.144758+0000","last_peered":"2023-10-18T10:40:56.144758+0000","last_clean":"2023-10-18T10:40:56.144758+0000","last_became_active":"2023-10-18T10:40:53.431047+0000","last_became_peered":"2023-10-18T10:40:53.431047+0000","last_unstale":"2023-10-18T10:40:56.144758+0000","last_undegraded":"2023-10-18T10:40:56.144758+0000","last_fullsized":"2023-10-18T10:40:56.144758+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:17.609983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:52.393600+0000","last_change":"2023-10-18T10:40:52.393600+0000","last_active":"2023-10-18T10:40:52.393600+0000","last_peered":"2023-10-18T10:40:52.393600+0000","last_clean":"2023-10-18T10:40:52.393600+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:52.393600+0000","last_undegraded":"2023-10-18T10:40:52.393600+0000","last_fullsized":"2023-10-18T10:40:52.393600+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:40:53.401392+0000","last_change":"2023-10-18T10:40:53.401392+0000","last_active":"2023-10-18T10:40:53.401392+0000","last_peered":"2023-10-18T10:40:53.401392+0000","last_clean":"2023-10-18T10:40:53.401392+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:40:53.401392+0000","last_undegraded":"2023-10-18T10:40:53.401392+0000","last_fullsized":"2023-10-18T10:40:53.401392+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429479+0000","last_change":"2023-10-18T10:40:53.425307+0000","last_active":"2023-10-18T10:40:55.429479+0000","last_peered":"2023-10-18T10:40:55.429479+0000","last_clean":"2023-10-18T10:40:55.429479+0000","last_became_active":"2023-10-18T10:40:53.424835+0000","last_became_peered":"2023-10-18T10:40:53.424835+0000","last_unstale":"2023-10-18T10:40:55.429479+0000","last_undegraded":"2023-10-18T10:40:55.429479+0000","last_fullsized":"2023-10-18T10:40:55.429479+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:04.973143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"86'172","reported_seq":429,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-10-18T10:40:50.192957+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:50.192957+0000","last_peered":"2023-10-18T10:40:50.192957+0000","last_clean":"2023-10-18T10:40:50.192957+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:50.192957+0000","last_undegraded":"2023-10-18T10:40:50.192957+0000","last_fullsized":"2023-10-18T10:40:50.192957+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":86,"acting":86,"num_store_stats":5},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":42,"acting":42,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59252,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312588,"statfs":{"total":96636764160,"available":96576090112,"internally_reserved":0,"allocated":1744896,"data_stored":1162487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.64700000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.47799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.627}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.44900000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.56499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":593558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":1.016},"last":0.52800000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.80300000000000005,"15min":0.875},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.504}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.872,"15min":0.872},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66800000000000004,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36599999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":0.82199999999999995,"15min":0.87},"last":0.44500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":1.016},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.498},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.81000000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38300000000000001,"15min":0.28599999999999998},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.314,"15min":0.28499999999999998},"max":{"1min":0.71199999999999997,"5min":0.80100000000000005,"15min":1.262},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":1.0109999999999999,"15min":1.028},"last":0.61399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68400000000000005,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77800000000000002,"5min":0.96099999999999997,"15min":1.224},"last":0.59599999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65600000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.438,"5min":0.307,"15min":0.307},"max":{"1min":0.85299999999999998,"5min":0.871,"15min":1.159},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":572119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.877,"15min":0.877},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.495}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.502,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.72099999999999997,"5min":0.746,"15min":0.79600000000000004},"last":0.629}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75600000000000001,"5min":1.446,"15min":1.446},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.90400000000000003},"last":0.40200000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.82699999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.47499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":599148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.315}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.436},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1212416,"data_stored":599308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.224,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.192},"max":{"1min":0.51900000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.54600000000000004,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.30399999999999999,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.60199999999999998,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.502},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.626,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.82399999999999995,"5min":0.88200000000000001,"15min":1.458},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.83799999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.746}]},{"osd":5,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.89000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.93700000000000006,"15min":1.028},"last":0.68400000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.625,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.879,"15min":0.92500000000000004},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":572119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.33900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.377},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.497}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.433},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.57499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1781760,"data_stored":1182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.25},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.27700000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.37},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.315}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:58.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2023-10-18T10:40:58.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2023-10-18T10:40:58.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2023-10-18T10:40:58.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2023-10-18T10:40:58.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2023-10-18T10:40:58.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2023-10-18T10:40:58.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2023-10-18T10:40:58.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2023-10-18T10:40:58.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2023-10-18T10:40:58.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2023-10-18T10:40:58.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2023-10-18T10:40:58.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2023-10-18T10:40:58.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-10-18T10:40:58.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-10-18T10:40:58.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-10-18T10:40:58.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-10-18T10:40:58.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-10-18T10:40:58.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-10-18T10:40:58.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-10-18T10:40:58.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-10-18T10:40:58.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-10-18T10:40:58.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-10-18T10:40:58.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-10-18T10:40:58.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-10-18T10:40:58.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-10-18T10:40:58.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-10-18T10:40:58.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-10-18T10:40:58.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-10-18T10:40:58.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-10-18T10:40:58.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2023-10-18T10:40:58.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-10-18T10:40:58.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:52.393600+0000', 'last_change': '2023-10-18T10:40:52.393600+0000', 'last_active': '2023-10-18T10:40:52.393600+0000', 'last_peered': '2023-10-18T10:40:52.393600+0000', 'last_clean': '2023-10-18T10:40:52.393600+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:52.393600+0000', 'last_undegraded': '2023-10-18T10:40:52.393600+0000', 'last_fullsized': '2023-10-18T10:40:52.393600+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:52.393600+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-10-18T10:40:58.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:40:53.401392+0000', 'last_change': '2023-10-18T10:40:53.401392+0000', 'last_active': '2023-10-18T10:40:53.401392+0000', 'last_peered': '2023-10-18T10:40:53.401392+0000', 'last_clean': '2023-10-18T10:40:53.401392+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:40:53.401392+0000', 'last_undegraded': '2023-10-18T10:40:53.401392+0000', 'last_fullsized': '2023-10-18T10:40:53.401392+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'last_clean_scrub_stamp': '2023-10-18T10:40:53.401392+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:40:58.072 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T10:40:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.178+0000 7f3276610640 1 -- 172.21.15.125:0/3297076058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3258005180 msgr2=0x7f325800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.178+0000 7f3276610640 1 --2- 172.21.15.125:0/3297076058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3258005180 0x7f325800b980 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3268001d80 tx=0x7f32580057f0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.179+0000 7f3276610640 1 -- 172.21.15.125:0/3297076058 shutdown_connections 2023-10-18T10:40:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.179+0000 7f3276610640 1 --2- 172.21.15.125:0/3297076058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3258005180 0x7f325800b980 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.179+0000 7f3276610640 1 -- 172.21.15.125:0/3297076058 >> 172.21.15.125:0/3297076058 conn(0x7f32700d0d60 msgr2=0x7f32700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:58.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.179+0000 7f3276610640 1 -- 172.21.15.125:0/3297076058 shutdown_connections 2023-10-18T10:40:58.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.179+0000 7f3276610640 1 -- 172.21.15.125:0/3297076058 wait complete. 2023-10-18T10:40:58.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.180+0000 7f3276610640 1 Processor -- start 2023-10-18T10:40:58.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.180+0000 7f3276610640 1 -- start start 2023-10-18T10:40:58.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3276610640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3258005180 0x7f3270155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:58.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3276610640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3270157810 0x7f32700af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:58.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3276610640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32700b5960 0x7f32700b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:58.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3276610640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32700b6350 con 0x7f3270157810 2023-10-18T10:40:58.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3276610640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327015e4e0 con 0x7f3258005180 2023-10-18T10:40:58.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3276610640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32700cae40 con 0x7f32700b5960 2023-10-18T10:40:58.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3270157810 0x7f32700af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:58.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3270157810 0x7f32700af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43338/0 (socket says 172.21.15.125:43338) 2023-10-18T10:40:58.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 -- 172.21.15.125:0/3079897892 learned_addr learned my addr 172.21.15.125:0/3079897892 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326ffff640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3258005180 0x7f3270155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3274bad640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32700b5960 0x7f32700b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 -- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32700b5960 msgr2=0x7f32700b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32700b5960 0x7f32700b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 -- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3258005180 msgr2=0x7f3270155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3258005180 0x7f3270155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 -- 172.21.15.125:0/3079897892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32700ca230 con 0x7f3270157810 2023-10-18T10:40:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326f7fe640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3270157810 0x7f32700af250 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f326400e640 tx=0x7f326400eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326cff9640 1 -- 172.21.15.125:0/3079897892 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400c940 con 0x7f3270157810 2023-10-18T10:40:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326cff9640 1 -- 172.21.15.125:0/3079897892 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f326400cb90 con 0x7f3270157810 2023-10-18T10:40:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32700ca510 con 0x7f3270157810 2023-10-18T10:40:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f326cff9640 1 -- 172.21.15.125:0/3079897892 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264011460 con 0x7f3270157810 2023-10-18T10:40:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.181+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32700caad0 con 0x7f3270157810 2023-10-18T10:40:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.182+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32700af790 con 0x7f3270157810 2023-10-18T10:40:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.183+0000 7f326cff9640 1 -- 172.21.15.125:0/3079897892 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f32640115c0 con 0x7f3270157810 2023-10-18T10:40:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.183+0000 7f326cff9640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f325c06af90 0x7f325c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:58.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.183+0000 7f326cff9640 1 -- 172.21.15.125:0/3079897892 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f326408a090 con 0x7f3270157810 2023-10-18T10:40:58.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.183+0000 7f326ffff640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f325c06af90 0x7f325c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:58.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.184+0000 7f326ffff640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f325c06af90 0x7f325c06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3260009ad0 tx=0x7f3260005f90 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:58.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.187+0000 7f326cff9640 1 -- 172.21.15.125:0/3079897892 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3264054e90 con 0x7f3270157810 2023-10-18T10:40:58.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.419+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f32700690e0 con 0x7f3270157810 2023-10-18T10:40:58.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.419+0000 7f326cff9640 1 -- 172.21.15.125:0/3079897892 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1088 (secure 0 0 0) 0x7f326401f450 con 0x7f3270157810 2023-10-18T10:40:58.421 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:58.422 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":980,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":98,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":65},{"state_name":"unknown","count":32}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592756,"bytes_used":320110592,"bytes_avail":772774002688,"bytes_total":773094113280,"unknown_pgs_ratio":0.32989689707756042,"write_bytes_sec":3327,"read_op_per_sec":0,"write_op_per_sec":9},"fsmap":{"epoch":35,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":7581}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-10-18T10:39:10.732193+0000","services":{}},"progress_events":{}} 2023-10-18T10:40:58.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f325c06af90 msgr2=0x7f325c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f325c06af90 0x7f325c06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3260009ad0 tx=0x7f3260005f90 comp rx=0 tx=0).stop 2023-10-18T10:40:58.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3270157810 msgr2=0x7f32700af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3270157810 0x7f32700af250 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f326400e640 tx=0x7f326400eb00 comp rx=0 tx=0).stop 2023-10-18T10:40:58.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 shutdown_connections 2023-10-18T10:40:58.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f325c06af90 0x7f325c06d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32700b5960 0x7f32700b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3270157810 0x7f32700af250 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 --2- 172.21.15.125:0/3079897892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3258005180 0x7f3270155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 >> 172.21.15.125:0/3079897892 conn(0x7f32700d0d60 msgr2=0x7f327006b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:58.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 shutdown_connections 2023-10-18T10:40:58.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.421+0000 7f3276610640 1 -- 172.21.15.125:0/3079897892 wait complete. 2023-10-18T10:40:58.433 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T10:40:58.434 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:40:58.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.535+0000 7f3ff739e640 1 -- 172.21.15.125:0/2071754419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fdc006290 msgr2=0x7f3fdc00ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.535+0000 7f3ff739e640 1 --2- 172.21.15.125:0/2071754419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fdc006290 0x7f3fdc00ca90 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c640 tx=0x7f3fdc006900 comp rx=0 tx=0).stop 2023-10-18T10:40:58.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.535+0000 7f3ff739e640 1 -- 172.21.15.125:0/2071754419 shutdown_connections 2023-10-18T10:40:58.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.535+0000 7f3ff739e640 1 --2- 172.21.15.125:0/2071754419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fdc006290 0x7f3fdc00ca90 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.535+0000 7f3ff739e640 1 -- 172.21.15.125:0/2071754419 >> 172.21.15.125:0/2071754419 conn(0x7f3ff0069930 msgr2=0x7f3ff0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:58.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.535+0000 7f3ff739e640 1 -- 172.21.15.125:0/2071754419 shutdown_connections 2023-10-18T10:40:58.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.535+0000 7f3ff739e640 1 -- 172.21.15.125:0/2071754419 wait complete. 2023-10-18T10:40:58.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.536+0000 7f3ff739e640 1 Processor -- start 2023-10-18T10:40:58.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.536+0000 7f3ff739e640 1 -- start start 2023-10-18T10:40:58.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff739e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fdc006290 0x7f3ff0146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:58.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff739e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff014fdd0 0x7f3ff0150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:58.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff739e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff01eed70 0x7f3ff01ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:58.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff739e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0150790 con 0x7f3ff01eed70 2023-10-18T10:40:58.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff739e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01f0720 con 0x7f3ff014fdd0 2023-10-18T10:40:58.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff739e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01467a0 con 0x7f3fdc006290 2023-10-18T10:40:58.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff01eed70 0x7f3ff01ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:58.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff01eed70 0x7f3ff01ef1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43358/0 (socket says 172.21.15.125:43358) 2023-10-18T10:40:58.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 -- 172.21.15.125:0/3793032138 learned_addr learned my addr 172.21.15.125:0/3793032138 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:40:58.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff513a640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fdc006290 0x7f3ff0146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:58.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 -- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fdc006290 msgr2=0x7f3ff0146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fdc006290 0x7f3ff0146260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff4939640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff014fdd0 0x7f3ff0150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:58.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 -- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff014fdd0 msgr2=0x7f3ff0150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff014fdd0 0x7f3ff0150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 -- 172.21.15.125:0/3793032138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff01efad0 con 0x7f3ff01eed70 2023-10-18T10:40:58.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff593b640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff01eed70 0x7f3ff01ef1f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3fe802cd40 tx=0x7f3fe8004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:58.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3fe1ffb640 1 -- 172.21.15.125:0/3793032138 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80017b0 con 0x7f3ff01eed70 2023-10-18T10:40:58.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3fe1ffb640 1 -- 172.21.15.125:0/3793032138 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3fe8001910 con 0x7f3ff01eed70 2023-10-18T10:40:58.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff01f00d0 con 0x7f3ff01eed70 2023-10-18T10:40:58.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.537+0000 7f3fe1ffb640 1 -- 172.21.15.125:0/3793032138 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80062d0 con 0x7f3ff01eed70 2023-10-18T10:40:58.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.538+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff01ef730 con 0x7f3ff01eed70 2023-10-18T10:40:58.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.539+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff01eff30 con 0x7f3ff01eed70 2023-10-18T10:40:58.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.539+0000 7f3fe1ffb640 1 -- 172.21.15.125:0/3793032138 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3fe8031030 con 0x7f3ff01eed70 2023-10-18T10:40:58.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.539+0000 7f3fe1ffb640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3fd406af60 0x7f3fd406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:40:58.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.539+0000 7f3fe1ffb640 1 -- 172.21.15.125:0/3793032138 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3fe80aac20 con 0x7f3ff01eed70 2023-10-18T10:40:58.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.540+0000 7f3ff513a640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3fd406af60 0x7f3fd406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:40:58.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.540+0000 7f3ff513a640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3fd406af60 0x7f3fd406d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0151350 tx=0x7f3fec006070 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:40:58.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.544+0000 7f3fe1ffb640 1 -- 172.21.15.125:0/3793032138 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ff01eff30 con 0x7f3ff01eed70 2023-10-18T10:40:58.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.707+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 --> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ff01eff30 con 0x7f3fd406af60 2023-10-18T10:40:58.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.715+0000 7f3fe1ffb640 1 -- 172.21.15.125:0/3793032138 <== mgr.5972 v2:172.21.15.139:6832/143413153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289473 (secure 0 0 0) 0x7f3ff01eff30 con 0x7f3fd406af60 2023-10-18T10:40:58.720 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:40:58.722 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:40:58.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3fd406af60 msgr2=0x7f3fd406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3fd406af60 0x7f3fd406d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0151350 tx=0x7f3fec006070 comp rx=0 tx=0).stop 2023-10-18T10:40:58.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff01eed70 msgr2=0x7f3ff01ef1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:40:58.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff01eed70 0x7f3ff01ef1f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3fe802cd40 tx=0x7f3fe8004a00 comp rx=0 tx=0).stop 2023-10-18T10:40:58.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 shutdown_connections 2023-10-18T10:40:58.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f3fd406af60 0x7f3fd406d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ff014fdd0 0x7f3ff0150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff01eed70 0x7f3ff01ef1f0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 --2- 172.21.15.125:0/3793032138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fdc006290 0x7f3ff0146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:40:58.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 >> 172.21.15.125:0/3793032138 conn(0x7f3ff0069930 msgr2=0x7f3ff0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:40:58.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 shutdown_connections 2023-10-18T10:40:58.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:40:58.718+0000 7f3ff739e640 1 -- 172.21.15.125:0/3793032138 wait complete. 2023-10-18T10:40:58.731 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:40:58.163105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":303,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":312740,"kb_used_data":10540,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754661980,"statfs":{"total":773094113280,"available":772773867520,"internally_reserved":0,"allocated":10792960,"data_stored":5940984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.760327"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144661+0000","last_change":"2023-10-18T10:40:54.429498+0000","last_active":"2023-10-18T10:40:56.144661+0000","last_peered":"2023-10-18T10:40:56.144661+0000","last_clean":"2023-10-18T10:40:56.144661+0000","last_became_active":"2023-10-18T10:40:54.429010+0000","last_became_peered":"2023-10-18T10:40:54.429010+0000","last_unstale":"2023-10-18T10:40:56.144661+0000","last_undegraded":"2023-10-18T10:40:56.144661+0000","last_fullsized":"2023-10-18T10:40:56.144661+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144511+0000","last_change":"2023-10-18T10:40:54.435140+0000","last_active":"2023-10-18T10:40:56.144511+0000","last_peered":"2023-10-18T10:40:56.144511+0000","last_clean":"2023-10-18T10:40:56.144511+0000","last_became_active":"2023-10-18T10:40:54.434657+0000","last_became_peered":"2023-10-18T10:40:54.434657+0000","last_unstale":"2023-10-18T10:40:56.144511+0000","last_undegraded":"2023-10-18T10:40:56.144511+0000","last_fullsized":"2023-10-18T10:40:56.144511+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.896314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144046+0000","last_change":"2023-10-18T10:40:54.424056+0000","last_active":"2023-10-18T10:40:56.144046+0000","last_peered":"2023-10-18T10:40:56.144046+0000","last_clean":"2023-10-18T10:40:56.144046+0000","last_became_active":"2023-10-18T10:40:54.423789+0000","last_became_peered":"2023-10-18T10:40:54.423789+0000","last_unstale":"2023-10-18T10:40:56.144046+0000","last_undegraded":"2023-10-18T10:40:56.144046+0000","last_fullsized":"2023-10-18T10:40:56.144046+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:29.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500661+0000","last_change":"2023-10-18T10:40:54.428671+0000","last_active":"2023-10-18T10:40:55.500661+0000","last_peered":"2023-10-18T10:40:55.500661+0000","last_clean":"2023-10-18T10:40:55.500661+0000","last_became_active":"2023-10-18T10:40:54.428396+0000","last_became_peered":"2023-10-18T10:40:54.428396+0000","last_unstale":"2023-10-18T10:40:55.500661+0000","last_undegraded":"2023-10-18T10:40:55.500661+0000","last_fullsized":"2023-10-18T10:40:55.500661+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:39.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018418+0000","last_change":"2023-10-18T10:40:54.433351+0000","last_active":"2023-10-18T10:40:56.018418+0000","last_peered":"2023-10-18T10:40:56.018418+0000","last_clean":"2023-10-18T10:40:56.018418+0000","last_became_active":"2023-10-18T10:40:54.430314+0000","last_became_peered":"2023-10-18T10:40:54.430314+0000","last_unstale":"2023-10-18T10:40:56.018418+0000","last_undegraded":"2023-10-18T10:40:56.018418+0000","last_fullsized":"2023-10-18T10:40:56.018418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:48.808404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429115+0000","last_change":"2023-10-18T10:40:54.431510+0000","last_active":"2023-10-18T10:40:55.429115+0000","last_peered":"2023-10-18T10:40:55.429115+0000","last_clean":"2023-10-18T10:40:55.429115+0000","last_became_active":"2023-10-18T10:40:54.431184+0000","last_became_peered":"2023-10-18T10:40:54.431184+0000","last_unstale":"2023-10-18T10:40:55.429115+0000","last_undegraded":"2023-10-18T10:40:55.429115+0000","last_fullsized":"2023-10-18T10:40:55.429115+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:33.893983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144553+0000","last_change":"2023-10-18T10:40:54.434704+0000","last_active":"2023-10-18T10:40:56.144553+0000","last_peered":"2023-10-18T10:40:56.144553+0000","last_clean":"2023-10-18T10:40:56.144553+0000","last_became_active":"2023-10-18T10:40:54.434388+0000","last_became_peered":"2023-10-18T10:40:54.434388+0000","last_unstale":"2023-10-18T10:40:56.144553+0000","last_undegraded":"2023-10-18T10:40:56.144553+0000","last_fullsized":"2023-10-18T10:40:56.144553+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:54.346618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:54.427003+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:54.426743+0000","last_became_peered":"2023-10-18T10:40:54.426743+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:58.361349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434857+0000","last_change":"2023-10-18T10:40:54.424669+0000","last_active":"2023-10-18T10:40:55.434857+0000","last_peered":"2023-10-18T10:40:55.434857+0000","last_clean":"2023-10-18T10:40:55.434857+0000","last_became_active":"2023-10-18T10:40:54.424408+0000","last_became_peered":"2023-10-18T10:40:54.424408+0000","last_unstale":"2023-10-18T10:40:55.434857+0000","last_undegraded":"2023-10-18T10:40:55.434857+0000","last_fullsized":"2023-10-18T10:40:55.434857+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:07.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500867+0000","last_change":"2023-10-18T10:40:54.433837+0000","last_active":"2023-10-18T10:40:55.500867+0000","last_peered":"2023-10-18T10:40:55.500867+0000","last_clean":"2023-10-18T10:40:55.500867+0000","last_became_active":"2023-10-18T10:40:54.433558+0000","last_became_peered":"2023-10-18T10:40:54.433558+0000","last_unstale":"2023-10-18T10:40:55.500867+0000","last_undegraded":"2023-10-18T10:40:55.500867+0000","last_fullsized":"2023-10-18T10:40:55.500867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:04.849763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144518+0000","last_change":"2023-10-18T10:40:54.429847+0000","last_active":"2023-10-18T10:40:56.144518+0000","last_peered":"2023-10-18T10:40:56.144518+0000","last_clean":"2023-10-18T10:40:56.144518+0000","last_became_active":"2023-10-18T10:40:54.429629+0000","last_became_peered":"2023-10-18T10:40:54.429629+0000","last_unstale":"2023-10-18T10:40:56.144518+0000","last_undegraded":"2023-10-18T10:40:56.144518+0000","last_fullsized":"2023-10-18T10:40:56.144518+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:29.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500804+0000","last_change":"2023-10-18T10:40:54.427881+0000","last_active":"2023-10-18T10:40:55.500804+0000","last_peered":"2023-10-18T10:40:55.500804+0000","last_clean":"2023-10-18T10:40:55.500804+0000","last_became_active":"2023-10-18T10:40:54.427423+0000","last_became_peered":"2023-10-18T10:40:54.427423+0000","last_unstale":"2023-10-18T10:40:55.500804+0000","last_undegraded":"2023-10-18T10:40:55.500804+0000","last_fullsized":"2023-10-18T10:40:55.500804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:07.682253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018526+0000","last_change":"2023-10-18T10:40:54.426415+0000","last_active":"2023-10-18T10:40:56.018526+0000","last_peered":"2023-10-18T10:40:56.018526+0000","last_clean":"2023-10-18T10:40:56.018526+0000","last_became_active":"2023-10-18T10:40:54.426129+0000","last_became_peered":"2023-10-18T10:40:54.426129+0000","last_unstale":"2023-10-18T10:40:56.018526+0000","last_undegraded":"2023-10-18T10:40:56.018526+0000","last_fullsized":"2023-10-18T10:40:56.018526+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:33:27.749058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429139+0000","last_change":"2023-10-18T10:40:54.430848+0000","last_active":"2023-10-18T10:40:55.429139+0000","last_peered":"2023-10-18T10:40:55.429139+0000","last_clean":"2023-10-18T10:40:55.429139+0000","last_became_active":"2023-10-18T10:40:54.430562+0000","last_became_peered":"2023-10-18T10:40:54.430562+0000","last_unstale":"2023-10-18T10:40:55.429139+0000","last_undegraded":"2023-10-18T10:40:55.429139+0000","last_fullsized":"2023-10-18T10:40:55.429139+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:05.564173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144611+0000","last_change":"2023-10-18T10:40:54.421817+0000","last_active":"2023-10-18T10:40:56.144611+0000","last_peered":"2023-10-18T10:40:56.144611+0000","last_clean":"2023-10-18T10:40:56.144611+0000","last_became_active":"2023-10-18T10:40:54.421613+0000","last_became_peered":"2023-10-18T10:40:54.421613+0000","last_unstale":"2023-10-18T10:40:56.144611+0000","last_undegraded":"2023-10-18T10:40:56.144611+0000","last_fullsized":"2023-10-18T10:40:56.144611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:36.706863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144136+0000","last_change":"2023-10-18T10:40:54.434433+0000","last_active":"2023-10-18T10:40:56.144136+0000","last_peered":"2023-10-18T10:40:56.144136+0000","last_clean":"2023-10-18T10:40:56.144136+0000","last_became_active":"2023-10-18T10:40:54.433953+0000","last_became_peered":"2023-10-18T10:40:54.433953+0000","last_unstale":"2023-10-18T10:40:56.144136+0000","last_undegraded":"2023-10-18T10:40:56.144136+0000","last_fullsized":"2023-10-18T10:40:56.144136+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:32.284117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:54.435303+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:54.434740+0000","last_became_peered":"2023-10-18T10:40:54.434740+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.386244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144183+0000","last_change":"2023-10-18T10:40:54.427685+0000","last_active":"2023-10-18T10:40:56.144183+0000","last_peered":"2023-10-18T10:40:56.144183+0000","last_clean":"2023-10-18T10:40:56.144183+0000","last_became_active":"2023-10-18T10:40:54.427436+0000","last_became_peered":"2023-10-18T10:40:54.427436+0000","last_unstale":"2023-10-18T10:40:56.144183+0000","last_undegraded":"2023-10-18T10:40:56.144183+0000","last_fullsized":"2023-10-18T10:40:56.144183+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:02.501409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429236+0000","last_change":"2023-10-18T10:40:54.432868+0000","last_active":"2023-10-18T10:40:55.429236+0000","last_peered":"2023-10-18T10:40:55.429236+0000","last_clean":"2023-10-18T10:40:55.429236+0000","last_became_active":"2023-10-18T10:40:54.432637+0000","last_became_peered":"2023-10-18T10:40:54.432637+0000","last_unstale":"2023-10-18T10:40:55.429236+0000","last_undegraded":"2023-10-18T10:40:55.429236+0000","last_fullsized":"2023-10-18T10:40:55.429236+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:46.951599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018489+0000","last_change":"2023-10-18T10:40:54.430205+0000","last_active":"2023-10-18T10:40:56.018489+0000","last_peered":"2023-10-18T10:40:56.018489+0000","last_clean":"2023-10-18T10:40:56.018489+0000","last_became_active":"2023-10-18T10:40:54.429898+0000","last_became_peered":"2023-10-18T10:40:54.429898+0000","last_unstale":"2023-10-18T10:40:56.018489+0000","last_undegraded":"2023-10-18T10:40:56.018489+0000","last_fullsized":"2023-10-18T10:40:56.018489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:48.646925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434605+0000","last_change":"2023-10-18T10:40:54.424296+0000","last_active":"2023-10-18T10:40:55.434605+0000","last_peered":"2023-10-18T10:40:55.434605+0000","last_clean":"2023-10-18T10:40:55.434605+0000","last_became_active":"2023-10-18T10:40:54.423967+0000","last_became_peered":"2023-10-18T10:40:54.423967+0000","last_unstale":"2023-10-18T10:40:55.434605+0000","last_undegraded":"2023-10-18T10:40:55.434605+0000","last_fullsized":"2023-10-18T10:40:55.434605+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:31.240890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018318+0000","last_change":"2023-10-18T10:40:54.428004+0000","last_active":"2023-10-18T10:40:56.018318+0000","last_peered":"2023-10-18T10:40:56.018318+0000","last_clean":"2023-10-18T10:40:56.018318+0000","last_became_active":"2023-10-18T10:40:54.427533+0000","last_became_peered":"2023-10-18T10:40:54.427533+0000","last_unstale":"2023-10-18T10:40:56.018318+0000","last_undegraded":"2023-10-18T10:40:56.018318+0000","last_fullsized":"2023-10-18T10:40:56.018318+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:27:07.146678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434483+0000","last_change":"2023-10-18T10:40:54.420300+0000","last_active":"2023-10-18T10:40:55.434483+0000","last_peered":"2023-10-18T10:40:55.434483+0000","last_clean":"2023-10-18T10:40:55.434483+0000","last_became_active":"2023-10-18T10:40:54.420008+0000","last_became_peered":"2023-10-18T10:40:54.420008+0000","last_unstale":"2023-10-18T10:40:55.434483+0000","last_undegraded":"2023-10-18T10:40:55.434483+0000","last_fullsized":"2023-10-18T10:40:55.434483+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:20.853296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143943+0000","last_change":"2023-10-18T10:40:54.424777+0000","last_active":"2023-10-18T10:40:56.143943+0000","last_peered":"2023-10-18T10:40:56.143943+0000","last_clean":"2023-10-18T10:40:56.143943+0000","last_became_active":"2023-10-18T10:40:54.424544+0000","last_became_peered":"2023-10-18T10:40:54.424544+0000","last_unstale":"2023-10-18T10:40:56.143943+0000","last_undegraded":"2023-10-18T10:40:56.143943+0000","last_fullsized":"2023-10-18T10:40:56.143943+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144475+0000","last_change":"2023-10-18T10:40:54.425355+0000","last_active":"2023-10-18T10:40:56.144475+0000","last_peered":"2023-10-18T10:40:56.144475+0000","last_clean":"2023-10-18T10:40:56.144475+0000","last_became_active":"2023-10-18T10:40:54.425123+0000","last_became_peered":"2023-10-18T10:40:54.425123+0000","last_unstale":"2023-10-18T10:40:56.144475+0000","last_undegraded":"2023-10-18T10:40:56.144475+0000","last_fullsized":"2023-10-18T10:40:56.144475+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:24.991706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500878+0000","last_change":"2023-10-18T10:40:54.426993+0000","last_active":"2023-10-18T10:40:55.500878+0000","last_peered":"2023-10-18T10:40:55.500878+0000","last_clean":"2023-10-18T10:40:55.500878+0000","last_became_active":"2023-10-18T10:40:54.426702+0000","last_became_peered":"2023-10-18T10:40:54.426702+0000","last_unstale":"2023-10-18T10:40:55.500878+0000","last_undegraded":"2023-10-18T10:40:55.500878+0000","last_fullsized":"2023-10-18T10:40:55.500878+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:57.667741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144401+0000","last_change":"2023-10-18T10:40:54.430464+0000","last_active":"2023-10-18T10:40:56.144401+0000","last_peered":"2023-10-18T10:40:56.144401+0000","last_clean":"2023-10-18T10:40:56.144401+0000","last_became_active":"2023-10-18T10:40:54.430070+0000","last_became_peered":"2023-10-18T10:40:54.430070+0000","last_unstale":"2023-10-18T10:40:56.144401+0000","last_undegraded":"2023-10-18T10:40:56.144401+0000","last_fullsized":"2023-10-18T10:40:56.144401+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:46.951599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018550+0000","last_change":"2023-10-18T10:40:54.432406+0000","last_active":"2023-10-18T10:40:56.018550+0000","last_peered":"2023-10-18T10:40:56.018550+0000","last_clean":"2023-10-18T10:40:56.018550+0000","last_became_active":"2023-10-18T10:40:54.432113+0000","last_became_peered":"2023-10-18T10:40:54.432113+0000","last_unstale":"2023-10-18T10:40:56.018550+0000","last_undegraded":"2023-10-18T10:40:56.018550+0000","last_fullsized":"2023-10-18T10:40:56.018550+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:24:16.099369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144250+0000","last_change":"2023-10-18T10:40:54.431844+0000","last_active":"2023-10-18T10:40:56.144250+0000","last_peered":"2023-10-18T10:40:56.144250+0000","last_clean":"2023-10-18T10:40:56.144250+0000","last_became_active":"2023-10-18T10:40:54.431447+0000","last_became_peered":"2023-10-18T10:40:54.431447+0000","last_unstale":"2023-10-18T10:40:56.144250+0000","last_undegraded":"2023-10-18T10:40:56.144250+0000","last_fullsized":"2023-10-18T10:40:56.144250+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:30.965000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144201+0000","last_change":"2023-10-18T10:40:54.424148+0000","last_active":"2023-10-18T10:40:56.144201+0000","last_peered":"2023-10-18T10:40:56.144201+0000","last_clean":"2023-10-18T10:40:56.144201+0000","last_became_active":"2023-10-18T10:40:54.423867+0000","last_became_peered":"2023-10-18T10:40:54.423867+0000","last_unstale":"2023-10-18T10:40:56.144201+0000","last_undegraded":"2023-10-18T10:40:56.144201+0000","last_fullsized":"2023-10-18T10:40:56.144201+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:33.893983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143940+0000","last_change":"2023-10-18T10:40:54.433301+0000","last_active":"2023-10-18T10:40:56.143940+0000","last_peered":"2023-10-18T10:40:56.143940+0000","last_clean":"2023-10-18T10:40:56.143940+0000","last_became_active":"2023-10-18T10:40:54.430606+0000","last_became_peered":"2023-10-18T10:40:54.430606+0000","last_unstale":"2023-10-18T10:40:56.143940+0000","last_undegraded":"2023-10-18T10:40:56.143940+0000","last_fullsized":"2023-10-18T10:40:56.143940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:03.914078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144239+0000","last_change":"2023-10-18T10:40:54.431569+0000","last_active":"2023-10-18T10:40:56.144239+0000","last_peered":"2023-10-18T10:40:56.144239+0000","last_clean":"2023-10-18T10:40:56.144239+0000","last_became_active":"2023-10-18T10:40:54.431255+0000","last_became_peered":"2023-10-18T10:40:54.431255+0000","last_unstale":"2023-10-18T10:40:56.144239+0000","last_undegraded":"2023-10-18T10:40:56.144239+0000","last_fullsized":"2023-10-18T10:40:56.144239+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:05.564173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144039+0000","last_change":"2023-10-18T10:40:54.428297+0000","last_active":"2023-10-18T10:40:56.144039+0000","last_peered":"2023-10-18T10:40:56.144039+0000","last_clean":"2023-10-18T10:40:56.144039+0000","last_became_active":"2023-10-18T10:40:54.428053+0000","last_became_peered":"2023-10-18T10:40:54.428053+0000","last_unstale":"2023-10-18T10:40:56.144039+0000","last_undegraded":"2023-10-18T10:40:56.144039+0000","last_fullsized":"2023-10-18T10:40:56.144039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:41.417088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"97'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:53.417631+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:53.417381+0000","last_became_peered":"2023-10-18T10:40:53.417381+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:40.271808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144091+0000","last_change":"2023-10-18T10:40:54.420096+0000","last_active":"2023-10-18T10:40:56.144091+0000","last_peered":"2023-10-18T10:40:56.144091+0000","last_clean":"2023-10-18T10:40:56.144091+0000","last_became_active":"2023-10-18T10:40:54.419838+0000","last_became_peered":"2023-10-18T10:40:54.419838+0000","last_unstale":"2023-10-18T10:40:56.144091+0000","last_undegraded":"2023-10-18T10:40:56.144091+0000","last_fullsized":"2023-10-18T10:40:56.144091+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:17.947505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144489+0000","last_change":"2023-10-18T10:40:53.426284+0000","last_active":"2023-10-18T10:40:56.144489+0000","last_peered":"2023-10-18T10:40:56.144489+0000","last_clean":"2023-10-18T10:40:56.144489+0000","last_became_active":"2023-10-18T10:40:53.425978+0000","last_became_peered":"2023-10-18T10:40:53.425978+0000","last_unstale":"2023-10-18T10:40:56.144489+0000","last_undegraded":"2023-10-18T10:40:56.144489+0000","last_fullsized":"2023-10-18T10:40:56.144489+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:28.892929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434484+0000","last_change":"2023-10-18T10:40:54.425573+0000","last_active":"2023-10-18T10:40:55.434484+0000","last_peered":"2023-10-18T10:40:55.434484+0000","last_clean":"2023-10-18T10:40:55.434484+0000","last_became_active":"2023-10-18T10:40:54.425311+0000","last_became_peered":"2023-10-18T10:40:54.425311+0000","last_unstale":"2023-10-18T10:40:55.434484+0000","last_undegraded":"2023-10-18T10:40:55.434484+0000","last_fullsized":"2023-10-18T10:40:55.434484+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:21.514697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144286+0000","last_change":"2023-10-18T10:40:53.430881+0000","last_active":"2023-10-18T10:40:56.144286+0000","last_peered":"2023-10-18T10:40:56.144286+0000","last_clean":"2023-10-18T10:40:56.144286+0000","last_became_active":"2023-10-18T10:40:53.430566+0000","last_became_peered":"2023-10-18T10:40:53.430566+0000","last_unstale":"2023-10-18T10:40:56.144286+0000","last_undegraded":"2023-10-18T10:40:56.144286+0000","last_fullsized":"2023-10-18T10:40:56.144286+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:49.684089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434675+0000","last_change":"2023-10-18T10:40:54.423819+0000","last_active":"2023-10-18T10:40:55.434675+0000","last_peered":"2023-10-18T10:40:55.434675+0000","last_clean":"2023-10-18T10:40:55.434675+0000","last_became_active":"2023-10-18T10:40:54.423497+0000","last_became_peered":"2023-10-18T10:40:54.423497+0000","last_unstale":"2023-10-18T10:40:55.434675+0000","last_undegraded":"2023-10-18T10:40:55.434675+0000","last_fullsized":"2023-10-18T10:40:55.434675+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:18.617775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144791+0000","last_change":"2023-10-18T10:40:53.423111+0000","last_active":"2023-10-18T10:40:56.144791+0000","last_peered":"2023-10-18T10:40:56.144791+0000","last_clean":"2023-10-18T10:40:56.144791+0000","last_became_active":"2023-10-18T10:40:53.422848+0000","last_became_peered":"2023-10-18T10:40:53.422848+0000","last_unstale":"2023-10-18T10:40:56.144791+0000","last_undegraded":"2023-10-18T10:40:56.144791+0000","last_fullsized":"2023-10-18T10:40:56.144791+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:19.845504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144374+0000","last_change":"2023-10-18T10:40:54.435357+0000","last_active":"2023-10-18T10:40:56.144374+0000","last_peered":"2023-10-18T10:40:56.144374+0000","last_clean":"2023-10-18T10:40:56.144374+0000","last_became_active":"2023-10-18T10:40:54.435102+0000","last_became_peered":"2023-10-18T10:40:54.435102+0000","last_unstale":"2023-10-18T10:40:56.144374+0000","last_undegraded":"2023-10-18T10:40:56.144374+0000","last_fullsized":"2023-10-18T10:40:56.144374+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:28:23.536390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434703+0000","last_change":"2023-10-18T10:40:53.421367+0000","last_active":"2023-10-18T10:40:55.434703+0000","last_peered":"2023-10-18T10:40:55.434703+0000","last_clean":"2023-10-18T10:40:55.434703+0000","last_became_active":"2023-10-18T10:40:53.421030+0000","last_became_peered":"2023-10-18T10:40:53.421030+0000","last_unstale":"2023-10-18T10:40:55.434703+0000","last_undegraded":"2023-10-18T10:40:55.434703+0000","last_fullsized":"2023-10-18T10:40:55.434703+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:59.291678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144627+0000","last_change":"2023-10-18T10:40:54.428864+0000","last_active":"2023-10-18T10:40:56.144627+0000","last_peered":"2023-10-18T10:40:56.144627+0000","last_clean":"2023-10-18T10:40:56.144627+0000","last_became_active":"2023-10-18T10:40:54.428412+0000","last_became_peered":"2023-10-18T10:40:54.428412+0000","last_unstale":"2023-10-18T10:40:56.144627+0000","last_undegraded":"2023-10-18T10:40:56.144627+0000","last_fullsized":"2023-10-18T10:40:56.144627+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:15.038645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500519+0000","last_change":"2023-10-18T10:40:53.420317+0000","last_active":"2023-10-18T10:40:55.500519+0000","last_peered":"2023-10-18T10:40:55.500519+0000","last_clean":"2023-10-18T10:40:55.500519+0000","last_became_active":"2023-10-18T10:40:53.420046+0000","last_became_peered":"2023-10-18T10:40:53.420046+0000","last_unstale":"2023-10-18T10:40:55.500519+0000","last_undegraded":"2023-10-18T10:40:55.500519+0000","last_fullsized":"2023-10-18T10:40:55.500519+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:36.623092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434874+0000","last_change":"2023-10-18T10:40:54.426193+0000","last_active":"2023-10-18T10:40:55.434874+0000","last_peered":"2023-10-18T10:40:55.434874+0000","last_clean":"2023-10-18T10:40:55.434874+0000","last_became_active":"2023-10-18T10:40:54.425894+0000","last_became_peered":"2023-10-18T10:40:54.425894+0000","last_unstale":"2023-10-18T10:40:55.434874+0000","last_undegraded":"2023-10-18T10:40:55.434874+0000","last_fullsized":"2023-10-18T10:40:55.434874+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:04.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144181+0000","last_change":"2023-10-18T10:40:53.430760+0000","last_active":"2023-10-18T10:40:56.144181+0000","last_peered":"2023-10-18T10:40:56.144181+0000","last_clean":"2023-10-18T10:40:56.144181+0000","last_became_active":"2023-10-18T10:40:53.430469+0000","last_became_peered":"2023-10-18T10:40:53.430469+0000","last_unstale":"2023-10-18T10:40:56.144181+0000","last_undegraded":"2023-10-18T10:40:56.144181+0000","last_fullsized":"2023-10-18T10:40:56.144181+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:45.943807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144991+0000","last_change":"2023-10-18T10:40:54.435078+0000","last_active":"2023-10-18T10:40:56.144991+0000","last_peered":"2023-10-18T10:40:56.144991+0000","last_clean":"2023-10-18T10:40:56.144991+0000","last_became_active":"2023-10-18T10:40:54.434517+0000","last_became_peered":"2023-10-18T10:40:54.434517+0000","last_unstale":"2023-10-18T10:40:56.144991+0000","last_undegraded":"2023-10-18T10:40:56.144991+0000","last_fullsized":"2023-10-18T10:40:56.144991+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:40.219033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:53.415948+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:53.415668+0000","last_became_peered":"2023-10-18T10:40:53.415668+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:12.409497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144074+0000","last_change":"2023-10-18T10:40:54.426165+0000","last_active":"2023-10-18T10:40:56.144074+0000","last_peered":"2023-10-18T10:40:56.144074+0000","last_clean":"2023-10-18T10:40:56.144074+0000","last_became_active":"2023-10-18T10:40:54.425916+0000","last_became_peered":"2023-10-18T10:40:54.425916+0000","last_unstale":"2023-10-18T10:40:56.144074+0000","last_undegraded":"2023-10-18T10:40:56.144074+0000","last_fullsized":"2023-10-18T10:40:56.144074+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144650+0000","last_change":"2023-10-18T10:40:53.428251+0000","last_active":"2023-10-18T10:40:56.144650+0000","last_peered":"2023-10-18T10:40:56.144650+0000","last_clean":"2023-10-18T10:40:56.144650+0000","last_became_active":"2023-10-18T10:40:53.427975+0000","last_became_peered":"2023-10-18T10:40:53.427975+0000","last_unstale":"2023-10-18T10:40:56.144650+0000","last_undegraded":"2023-10-18T10:40:56.144650+0000","last_fullsized":"2023-10-18T10:40:56.144650+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:45:58.336437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500662+0000","last_change":"2023-10-18T10:40:54.428238+0000","last_active":"2023-10-18T10:40:55.500662+0000","last_peered":"2023-10-18T10:40:55.500662+0000","last_clean":"2023-10-18T10:40:55.500662+0000","last_became_active":"2023-10-18T10:40:54.427404+0000","last_became_peered":"2023-10-18T10:40:54.427404+0000","last_unstale":"2023-10-18T10:40:55.500662+0000","last_undegraded":"2023-10-18T10:40:55.500662+0000","last_fullsized":"2023-10-18T10:40:55.500662+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:06.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500959+0000","last_change":"2023-10-18T10:40:53.423540+0000","last_active":"2023-10-18T10:40:55.500959+0000","last_peered":"2023-10-18T10:40:55.500959+0000","last_clean":"2023-10-18T10:40:55.500959+0000","last_became_active":"2023-10-18T10:40:53.423278+0000","last_became_peered":"2023-10-18T10:40:53.423278+0000","last_unstale":"2023-10-18T10:40:55.500959+0000","last_undegraded":"2023-10-18T10:40:55.500959+0000","last_fullsized":"2023-10-18T10:40:55.500959+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:15.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429349+0000","last_change":"2023-10-18T10:40:54.423285+0000","last_active":"2023-10-18T10:40:55.429349+0000","last_peered":"2023-10-18T10:40:55.429349+0000","last_clean":"2023-10-18T10:40:55.429349+0000","last_became_active":"2023-10-18T10:40:54.423013+0000","last_became_peered":"2023-10-18T10:40:54.423013+0000","last_unstale":"2023-10-18T10:40:55.429349+0000","last_undegraded":"2023-10-18T10:40:55.429349+0000","last_fullsized":"2023-10-18T10:40:55.429349+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"97'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018642+0000","last_change":"2023-10-18T10:40:53.424070+0000","last_active":"2023-10-18T10:40:56.018642+0000","last_peered":"2023-10-18T10:40:56.018642+0000","last_clean":"2023-10-18T10:40:56.018642+0000","last_became_active":"2023-10-18T10:40:53.423799+0000","last_became_peered":"2023-10-18T10:40:53.423799+0000","last_unstale":"2023-10-18T10:40:56.018642+0000","last_undegraded":"2023-10-18T10:40:56.018642+0000","last_fullsized":"2023-10-18T10:40:56.018642+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:09.891958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143935+0000","last_change":"2023-10-18T10:40:54.426135+0000","last_active":"2023-10-18T10:40:56.143935+0000","last_peered":"2023-10-18T10:40:56.143935+0000","last_clean":"2023-10-18T10:40:56.143935+0000","last_became_active":"2023-10-18T10:40:54.425409+0000","last_became_peered":"2023-10-18T10:40:54.425409+0000","last_unstale":"2023-10-18T10:40:56.143935+0000","last_undegraded":"2023-10-18T10:40:56.143935+0000","last_fullsized":"2023-10-18T10:40:56.143935+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:01.527219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"97'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144832+0000","last_change":"2023-10-18T10:40:53.422980+0000","last_active":"2023-10-18T10:40:56.144832+0000","last_peered":"2023-10-18T10:40:56.144832+0000","last_clean":"2023-10-18T10:40:56.144832+0000","last_became_active":"2023-10-18T10:40:53.422696+0000","last_became_peered":"2023-10-18T10:40:53.422696+0000","last_unstale":"2023-10-18T10:40:56.144832+0000","last_undegraded":"2023-10-18T10:40:56.144832+0000","last_fullsized":"2023-10-18T10:40:56.144832+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:20.506905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500893+0000","last_change":"2023-10-18T10:40:54.423164+0000","last_active":"2023-10-18T10:40:55.500893+0000","last_peered":"2023-10-18T10:40:55.500893+0000","last_clean":"2023-10-18T10:40:55.500893+0000","last_became_active":"2023-10-18T10:40:54.422844+0000","last_became_peered":"2023-10-18T10:40:54.422844+0000","last_unstale":"2023-10-18T10:40:55.500893+0000","last_undegraded":"2023-10-18T10:40:55.500893+0000","last_fullsized":"2023-10-18T10:40:55.500893+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:50.691881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500565+0000","last_change":"2023-10-18T10:40:53.420785+0000","last_active":"2023-10-18T10:40:55.500565+0000","last_peered":"2023-10-18T10:40:55.500565+0000","last_clean":"2023-10-18T10:40:55.500565+0000","last_became_active":"2023-10-18T10:40:53.420530+0000","last_became_peered":"2023-10-18T10:40:53.420530+0000","last_unstale":"2023-10-18T10:40:55.500565+0000","last_undegraded":"2023-10-18T10:40:55.500565+0000","last_fullsized":"2023-10-18T10:40:55.500565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:44.748900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144877+0000","last_change":"2023-10-18T10:40:54.430742+0000","last_active":"2023-10-18T10:40:56.144877+0000","last_peered":"2023-10-18T10:40:56.144877+0000","last_clean":"2023-10-18T10:40:56.144877+0000","last_became_active":"2023-10-18T10:40:54.430455+0000","last_became_peered":"2023-10-18T10:40:54.430455+0000","last_unstale":"2023-10-18T10:40:56.144877+0000","last_undegraded":"2023-10-18T10:40:56.144877+0000","last_fullsized":"2023-10-18T10:40:56.144877+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:39.892546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144118+0000","last_change":"2023-10-18T10:40:53.428666+0000","last_active":"2023-10-18T10:40:56.144118+0000","last_peered":"2023-10-18T10:40:56.144118+0000","last_clean":"2023-10-18T10:40:56.144118+0000","last_became_active":"2023-10-18T10:40:53.428330+0000","last_became_peered":"2023-10-18T10:40:53.428330+0000","last_unstale":"2023-10-18T10:40:56.144118+0000","last_undegraded":"2023-10-18T10:40:56.144118+0000","last_fullsized":"2023-10-18T10:40:56.144118+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:06.674462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144941+0000","last_change":"2023-10-18T10:40:54.430021+0000","last_active":"2023-10-18T10:40:56.144941+0000","last_peered":"2023-10-18T10:40:56.144941+0000","last_clean":"2023-10-18T10:40:56.144941+0000","last_became_active":"2023-10-18T10:40:54.429684+0000","last_became_peered":"2023-10-18T10:40:54.429684+0000","last_unstale":"2023-10-18T10:40:56.144941+0000","last_undegraded":"2023-10-18T10:40:56.144941+0000","last_fullsized":"2023-10-18T10:40:56.144941+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:03.648375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018334+0000","last_change":"2023-10-18T10:40:53.425774+0000","last_active":"2023-10-18T10:40:56.018334+0000","last_peered":"2023-10-18T10:40:56.018334+0000","last_clean":"2023-10-18T10:40:56.018334+0000","last_became_active":"2023-10-18T10:40:53.425518+0000","last_became_peered":"2023-10-18T10:40:53.425518+0000","last_unstale":"2023-10-18T10:40:56.018334+0000","last_undegraded":"2023-10-18T10:40:56.018334+0000","last_fullsized":"2023-10-18T10:40:56.018334+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:30.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145025+0000","last_change":"2023-10-18T10:40:54.432786+0000","last_active":"2023-10-18T10:40:56.145025+0000","last_peered":"2023-10-18T10:40:56.145025+0000","last_clean":"2023-10-18T10:40:56.145025+0000","last_became_active":"2023-10-18T10:40:54.432538+0000","last_became_peered":"2023-10-18T10:40:54.432538+0000","last_unstale":"2023-10-18T10:40:56.145025+0000","last_undegraded":"2023-10-18T10:40:56.145025+0000","last_fullsized":"2023-10-18T10:40:56.145025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:52:39.855703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"97'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144160+0000","last_change":"2023-10-18T10:40:53.426701+0000","last_active":"2023-10-18T10:40:56.144160+0000","last_peered":"2023-10-18T10:40:56.144160+0000","last_clean":"2023-10-18T10:40:56.144160+0000","last_became_active":"2023-10-18T10:40:53.426431+0000","last_became_peered":"2023-10-18T10:40:53.426431+0000","last_unstale":"2023-10-18T10:40:56.144160+0000","last_undegraded":"2023-10-18T10:40:56.144160+0000","last_fullsized":"2023-10-18T10:40:56.144160+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:31.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429351+0000","last_change":"2023-10-18T10:40:54.434075+0000","last_active":"2023-10-18T10:40:55.429351+0000","last_peered":"2023-10-18T10:40:55.429351+0000","last_clean":"2023-10-18T10:40:55.429351+0000","last_became_active":"2023-10-18T10:40:54.433835+0000","last_became_peered":"2023-10-18T10:40:54.433835+0000","last_unstale":"2023-10-18T10:40:55.429351+0000","last_undegraded":"2023-10-18T10:40:55.429351+0000","last_fullsized":"2023-10-18T10:40:55.429351+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144738+0000","last_change":"2023-10-18T10:40:53.428407+0000","last_active":"2023-10-18T10:40:56.144738+0000","last_peered":"2023-10-18T10:40:56.144738+0000","last_clean":"2023-10-18T10:40:56.144738+0000","last_became_active":"2023-10-18T10:40:53.428139+0000","last_became_peered":"2023-10-18T10:40:53.428139+0000","last_unstale":"2023-10-18T10:40:56.144738+0000","last_undegraded":"2023-10-18T10:40:56.144738+0000","last_fullsized":"2023-10-18T10:40:56.144738+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:54.556080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018402+0000","last_change":"2023-10-18T10:40:54.435840+0000","last_active":"2023-10-18T10:40:56.018402+0000","last_peered":"2023-10-18T10:40:56.018402+0000","last_clean":"2023-10-18T10:40:56.018402+0000","last_became_active":"2023-10-18T10:40:54.435594+0000","last_became_peered":"2023-10-18T10:40:54.435594+0000","last_unstale":"2023-10-18T10:40:56.018402+0000","last_undegraded":"2023-10-18T10:40:56.018402+0000","last_fullsized":"2023-10-18T10:40:56.018402+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:16:01.996675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434903+0000","last_change":"2023-10-18T10:40:53.424603+0000","last_active":"2023-10-18T10:40:55.434903+0000","last_peered":"2023-10-18T10:40:55.434903+0000","last_clean":"2023-10-18T10:40:55.434903+0000","last_became_active":"2023-10-18T10:40:53.424328+0000","last_became_peered":"2023-10-18T10:40:53.424328+0000","last_unstale":"2023-10-18T10:40:55.434903+0000","last_undegraded":"2023-10-18T10:40:55.434903+0000","last_fullsized":"2023-10-18T10:40:55.434903+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:49.348865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018391+0000","last_change":"2023-10-18T10:40:54.427794+0000","last_active":"2023-10-18T10:40:56.018391+0000","last_peered":"2023-10-18T10:40:56.018391+0000","last_clean":"2023-10-18T10:40:56.018391+0000","last_became_active":"2023-10-18T10:40:54.427255+0000","last_became_peered":"2023-10-18T10:40:54.427255+0000","last_unstale":"2023-10-18T10:40:56.018391+0000","last_undegraded":"2023-10-18T10:40:56.018391+0000","last_fullsized":"2023-10-18T10:40:56.018391+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:08.181951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018673+0000","last_change":"2023-10-18T10:40:53.416910+0000","last_active":"2023-10-18T10:40:56.018673+0000","last_peered":"2023-10-18T10:40:56.018673+0000","last_clean":"2023-10-18T10:40:56.018673+0000","last_became_active":"2023-10-18T10:40:53.416630+0000","last_became_peered":"2023-10-18T10:40:53.416630+0000","last_unstale":"2023-10-18T10:40:56.018673+0000","last_undegraded":"2023-10-18T10:40:56.018673+0000","last_fullsized":"2023-10-18T10:40:56.018673+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:24.474032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429270+0000","last_change":"2023-10-18T10:40:54.439124+0000","last_active":"2023-10-18T10:40:55.429270+0000","last_peered":"2023-10-18T10:40:55.429270+0000","last_clean":"2023-10-18T10:40:55.429270+0000","last_became_active":"2023-10-18T10:40:54.438869+0000","last_became_peered":"2023-10-18T10:40:54.438869+0000","last_unstale":"2023-10-18T10:40:55.429270+0000","last_undegraded":"2023-10-18T10:40:55.429270+0000","last_fullsized":"2023-10-18T10:40:55.429270+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:50.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143888+0000","last_change":"2023-10-18T10:40:53.429322+0000","last_active":"2023-10-18T10:40:56.143888+0000","last_peered":"2023-10-18T10:40:56.143888+0000","last_clean":"2023-10-18T10:40:56.143888+0000","last_became_active":"2023-10-18T10:40:53.429093+0000","last_became_peered":"2023-10-18T10:40:53.429093+0000","last_unstale":"2023-10-18T10:40:56.143888+0000","last_undegraded":"2023-10-18T10:40:56.143888+0000","last_fullsized":"2023-10-18T10:40:56.143888+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:14.928492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144441+0000","last_change":"2023-10-18T10:40:54.420623+0000","last_active":"2023-10-18T10:40:56.144441+0000","last_peered":"2023-10-18T10:40:56.144441+0000","last_clean":"2023-10-18T10:40:56.144441+0000","last_became_active":"2023-10-18T10:40:54.420398+0000","last_became_peered":"2023-10-18T10:40:54.420398+0000","last_unstale":"2023-10-18T10:40:56.144441+0000","last_undegraded":"2023-10-18T10:40:56.144441+0000","last_fullsized":"2023-10-18T10:40:56.144441+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.611760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429356+0000","last_change":"2023-10-18T10:40:53.424629+0000","last_active":"2023-10-18T10:40:55.429356+0000","last_peered":"2023-10-18T10:40:55.429356+0000","last_clean":"2023-10-18T10:40:55.429356+0000","last_became_active":"2023-10-18T10:40:53.424387+0000","last_became_peered":"2023-10-18T10:40:53.424387+0000","last_unstale":"2023-10-18T10:40:55.429356+0000","last_undegraded":"2023-10-18T10:40:55.429356+0000","last_fullsized":"2023-10-18T10:40:55.429356+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:54.556080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144592+0000","last_change":"2023-10-18T10:40:54.427692+0000","last_active":"2023-10-18T10:40:56.144592+0000","last_peered":"2023-10-18T10:40:56.144592+0000","last_clean":"2023-10-18T10:40:56.144592+0000","last_became_active":"2023-10-18T10:40:54.427408+0000","last_became_peered":"2023-10-18T10:40:54.427408+0000","last_unstale":"2023-10-18T10:40:56.144592+0000","last_undegraded":"2023-10-18T10:40:56.144592+0000","last_fullsized":"2023-10-18T10:40:56.144592+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:47:48.326742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429051+0000","last_change":"2023-10-18T10:40:53.426213+0000","last_active":"2023-10-18T10:40:55.429051+0000","last_peered":"2023-10-18T10:40:55.429051+0000","last_clean":"2023-10-18T10:40:55.429051+0000","last_became_active":"2023-10-18T10:40:53.425947+0000","last_became_peered":"2023-10-18T10:40:53.425947+0000","last_unstale":"2023-10-18T10:40:55.429051+0000","last_undegraded":"2023-10-18T10:40:55.429051+0000","last_fullsized":"2023-10-18T10:40:55.429051+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:45:58.336437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145071+0000","last_change":"2023-10-18T10:40:54.429476+0000","last_active":"2023-10-18T10:40:56.145071+0000","last_peered":"2023-10-18T10:40:56.145071+0000","last_clean":"2023-10-18T10:40:56.145071+0000","last_became_active":"2023-10-18T10:40:54.429109+0000","last_became_peered":"2023-10-18T10:40:54.429109+0000","last_unstale":"2023-10-18T10:40:56.145071+0000","last_undegraded":"2023-10-18T10:40:56.145071+0000","last_fullsized":"2023-10-18T10:40:56.145071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:58.443341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.428986+0000","last_change":"2023-10-18T10:40:53.425821+0000","last_active":"2023-10-18T10:40:55.428986+0000","last_peered":"2023-10-18T10:40:55.428986+0000","last_clean":"2023-10-18T10:40:55.428986+0000","last_became_active":"2023-10-18T10:40:53.425567+0000","last_became_peered":"2023-10-18T10:40:53.425567+0000","last_unstale":"2023-10-18T10:40:55.428986+0000","last_undegraded":"2023-10-18T10:40:55.428986+0000","last_fullsized":"2023-10-18T10:40:55.428986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:18.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434418+0000","last_change":"2023-10-18T10:40:54.430249+0000","last_active":"2023-10-18T10:40:55.434418+0000","last_peered":"2023-10-18T10:40:55.434418+0000","last_clean":"2023-10-18T10:40:55.434418+0000","last_became_active":"2023-10-18T10:40:54.429992+0000","last_became_peered":"2023-10-18T10:40:54.429992+0000","last_unstale":"2023-10-18T10:40:55.434418+0000","last_undegraded":"2023-10-18T10:40:55.434418+0000","last_fullsized":"2023-10-18T10:40:55.434418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:26.621442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500839+0000","last_change":"2023-10-18T10:40:53.424258+0000","last_active":"2023-10-18T10:40:55.500839+0000","last_peered":"2023-10-18T10:40:55.500839+0000","last_clean":"2023-10-18T10:40:55.500839+0000","last_became_active":"2023-10-18T10:40:53.423997+0000","last_became_peered":"2023-10-18T10:40:53.423997+0000","last_unstale":"2023-10-18T10:40:55.500839+0000","last_undegraded":"2023-10-18T10:40:55.500839+0000","last_fullsized":"2023-10-18T10:40:55.500839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:05.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144724+0000","last_change":"2023-10-18T10:40:54.433951+0000","last_active":"2023-10-18T10:40:56.144724+0000","last_peered":"2023-10-18T10:40:56.144724+0000","last_clean":"2023-10-18T10:40:56.144724+0000","last_became_active":"2023-10-18T10:40:54.433371+0000","last_became_peered":"2023-10-18T10:40:54.433371+0000","last_unstale":"2023-10-18T10:40:56.144724+0000","last_undegraded":"2023-10-18T10:40:56.144724+0000","last_fullsized":"2023-10-18T10:40:56.144724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:04.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144585+0000","last_change":"2023-10-18T10:40:53.431024+0000","last_active":"2023-10-18T10:40:56.144585+0000","last_peered":"2023-10-18T10:40:56.144585+0000","last_clean":"2023-10-18T10:40:56.144585+0000","last_became_active":"2023-10-18T10:40:53.430743+0000","last_became_peered":"2023-10-18T10:40:53.430743+0000","last_unstale":"2023-10-18T10:40:56.144585+0000","last_undegraded":"2023-10-18T10:40:56.144585+0000","last_fullsized":"2023-10-18T10:40:56.144585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:32.886192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434756+0000","last_change":"2023-10-18T10:40:54.425246+0000","last_active":"2023-10-18T10:40:55.434756+0000","last_peered":"2023-10-18T10:40:55.434756+0000","last_clean":"2023-10-18T10:40:55.434756+0000","last_became_active":"2023-10-18T10:40:54.424838+0000","last_became_peered":"2023-10-18T10:40:54.424838+0000","last_unstale":"2023-10-18T10:40:55.434756+0000","last_undegraded":"2023-10-18T10:40:55.434756+0000","last_fullsized":"2023-10-18T10:40:55.434756+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:54.346618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144500+0000","last_change":"2023-10-18T10:40:53.428598+0000","last_active":"2023-10-18T10:40:56.144500+0000","last_peered":"2023-10-18T10:40:56.144500+0000","last_clean":"2023-10-18T10:40:56.144500+0000","last_became_active":"2023-10-18T10:40:53.428102+0000","last_became_peered":"2023-10-18T10:40:53.428102+0000","last_unstale":"2023-10-18T10:40:56.144500+0000","last_undegraded":"2023-10-18T10:40:56.144500+0000","last_fullsized":"2023-10-18T10:40:56.144500+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:51:40.263102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144379+0000","last_change":"2023-10-18T10:40:54.431077+0000","last_active":"2023-10-18T10:40:56.144379+0000","last_peered":"2023-10-18T10:40:56.144379+0000","last_clean":"2023-10-18T10:40:56.144379+0000","last_became_active":"2023-10-18T10:40:54.430805+0000","last_became_peered":"2023-10-18T10:40:54.430805+0000","last_unstale":"2023-10-18T10:40:56.144379+0000","last_undegraded":"2023-10-18T10:40:56.144379+0000","last_fullsized":"2023-10-18T10:40:56.144379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:49:55.716517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144226+0000","last_change":"2023-10-18T10:40:53.426405+0000","last_active":"2023-10-18T10:40:56.144226+0000","last_peered":"2023-10-18T10:40:56.144226+0000","last_clean":"2023-10-18T10:40:56.144226+0000","last_became_active":"2023-10-18T10:40:53.426121+0000","last_became_peered":"2023-10-18T10:40:53.426121+0000","last_unstale":"2023-10-18T10:40:56.144226+0000","last_undegraded":"2023-10-18T10:40:56.144226+0000","last_fullsized":"2023-10-18T10:40:56.144226+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:03.841971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144673+0000","last_change":"2023-10-18T10:40:54.433504+0000","last_active":"2023-10-18T10:40:56.144673+0000","last_peered":"2023-10-18T10:40:56.144673+0000","last_clean":"2023-10-18T10:40:56.144673+0000","last_became_active":"2023-10-18T10:40:54.433264+0000","last_became_peered":"2023-10-18T10:40:54.433264+0000","last_unstale":"2023-10-18T10:40:56.144673+0000","last_undegraded":"2023-10-18T10:40:56.144673+0000","last_fullsized":"2023-10-18T10:40:56.144673+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:07.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144446+0000","last_change":"2023-10-18T10:40:53.424988+0000","last_active":"2023-10-18T10:40:56.144446+0000","last_peered":"2023-10-18T10:40:56.144446+0000","last_clean":"2023-10-18T10:40:56.144446+0000","last_became_active":"2023-10-18T10:40:53.423255+0000","last_became_peered":"2023-10-18T10:40:53.423255+0000","last_unstale":"2023-10-18T10:40:56.144446+0000","last_undegraded":"2023-10-18T10:40:56.144446+0000","last_fullsized":"2023-10-18T10:40:56.144446+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:30.233098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144920+0000","last_change":"2023-10-18T10:40:54.434657+0000","last_active":"2023-10-18T10:40:56.144920+0000","last_peered":"2023-10-18T10:40:56.144920+0000","last_clean":"2023-10-18T10:40:56.144920+0000","last_became_active":"2023-10-18T10:40:54.434344+0000","last_became_peered":"2023-10-18T10:40:54.434344+0000","last_unstale":"2023-10-18T10:40:56.144920+0000","last_undegraded":"2023-10-18T10:40:56.144920+0000","last_fullsized":"2023-10-18T10:40:56.144920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:49.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018412+0000","last_change":"2023-10-18T10:40:53.419056+0000","last_active":"2023-10-18T10:40:56.018412+0000","last_peered":"2023-10-18T10:40:56.018412+0000","last_clean":"2023-10-18T10:40:56.018412+0000","last_became_active":"2023-10-18T10:40:53.418815+0000","last_became_peered":"2023-10-18T10:40:53.418815+0000","last_unstale":"2023-10-18T10:40:56.018412+0000","last_undegraded":"2023-10-18T10:40:56.018412+0000","last_fullsized":"2023-10-18T10:40:56.018412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:13.857868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144317+0000","last_change":"2023-10-18T10:40:54.423909+0000","last_active":"2023-10-18T10:40:56.144317+0000","last_peered":"2023-10-18T10:40:56.144317+0000","last_clean":"2023-10-18T10:40:56.144317+0000","last_became_active":"2023-10-18T10:40:54.423659+0000","last_became_peered":"2023-10-18T10:40:54.423659+0000","last_unstale":"2023-10-18T10:40:56.144317+0000","last_undegraded":"2023-10-18T10:40:56.144317+0000","last_fullsized":"2023-10-18T10:40:56.144317+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:49:26.295205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144758+0000","last_change":"2023-10-18T10:40:53.431352+0000","last_active":"2023-10-18T10:40:56.144758+0000","last_peered":"2023-10-18T10:40:56.144758+0000","last_clean":"2023-10-18T10:40:56.144758+0000","last_became_active":"2023-10-18T10:40:53.431047+0000","last_became_peered":"2023-10-18T10:40:53.431047+0000","last_unstale":"2023-10-18T10:40:56.144758+0000","last_undegraded":"2023-10-18T10:40:56.144758+0000","last_fullsized":"2023-10-18T10:40:56.144758+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:17.609983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144229+0000","last_change":"2023-10-18T10:40:54.427209+0000","last_active":"2023-10-18T10:40:56.144229+0000","last_peered":"2023-10-18T10:40:56.144229+0000","last_clean":"2023-10-18T10:40:56.144229+0000","last_became_active":"2023-10-18T10:40:54.426950+0000","last_became_peered":"2023-10-18T10:40:54.426950+0000","last_unstale":"2023-10-18T10:40:56.144229+0000","last_undegraded":"2023-10-18T10:40:56.144229+0000","last_fullsized":"2023-10-18T10:40:56.144229+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:10.812136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143765+0000","last_change":"2023-10-18T10:40:53.426790+0000","last_active":"2023-10-18T10:40:56.143765+0000","last_peered":"2023-10-18T10:40:56.143765+0000","last_clean":"2023-10-18T10:40:56.143765+0000","last_became_active":"2023-10-18T10:40:53.426556+0000","last_became_peered":"2023-10-18T10:40:53.426556+0000","last_unstale":"2023-10-18T10:40:56.143765+0000","last_undegraded":"2023-10-18T10:40:56.143765+0000","last_fullsized":"2023-10-18T10:40:56.143765+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:03.450696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144433+0000","last_change":"2023-10-18T10:40:54.430137+0000","last_active":"2023-10-18T10:40:56.144433+0000","last_peered":"2023-10-18T10:40:56.144433+0000","last_clean":"2023-10-18T10:40:56.144433+0000","last_became_active":"2023-10-18T10:40:54.429879+0000","last_became_peered":"2023-10-18T10:40:54.429879+0000","last_unstale":"2023-10-18T10:40:56.144433+0000","last_undegraded":"2023-10-18T10:40:56.144433+0000","last_fullsized":"2023-10-18T10:40:56.144433+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:50.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429479+0000","last_change":"2023-10-18T10:40:53.425307+0000","last_active":"2023-10-18T10:40:55.429479+0000","last_peered":"2023-10-18T10:40:55.429479+0000","last_clean":"2023-10-18T10:40:55.429479+0000","last_became_active":"2023-10-18T10:40:53.424835+0000","last_became_peered":"2023-10-18T10:40:53.424835+0000","last_unstale":"2023-10-18T10:40:55.429479+0000","last_undegraded":"2023-10-18T10:40:55.429479+0000","last_fullsized":"2023-10-18T10:40:55.429479+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:04.973143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"86'172","reported_seq":433,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-18T10:40:54.422694+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:54.422694+0000","last_peered":"2023-10-18T10:40:54.422694+0000","last_clean":"2023-10-18T10:40:54.422694+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:54.422694+0000","last_undegraded":"2023-10-18T10:40:54.422694+0000","last_fullsized":"2023-10-18T10:40:54.422694+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59296,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312544,"statfs":{"total":96636764160,"available":96576045056,"internally_reserved":0,"allocated":1785856,"data_stored":1182336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.433},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.33300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.34999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.50900000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.38},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.48999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":593558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":1.016},"last":0.52800000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.80300000000000005,"15min":0.875},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.504}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.872,"15min":0.872},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66800000000000004,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36599999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":0.82199999999999995,"15min":0.87},"last":0.44500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":1.016},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.498},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.81000000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38300000000000001,"15min":0.28599999999999998},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.314,"15min":0.28499999999999998},"max":{"1min":0.71199999999999997,"5min":0.80100000000000005,"15min":1.262},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":1.0109999999999999,"15min":1.028},"last":0.61399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68400000000000005,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77800000000000002,"5min":0.96099999999999997,"15min":1.224},"last":0.59599999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65600000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.438,"5min":0.307,"15min":0.307},"max":{"1min":0.85299999999999998,"5min":0.871,"15min":1.159},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":593662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.877,"15min":0.877},"last":0.623}]},{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69199999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.502,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.72099999999999997,"5min":0.746,"15min":0.79600000000000004},"last":0.32700000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75600000000000001,"5min":1.446,"15min":1.446},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.90400000000000003},"last":0.38600000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.82699999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.44500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":599148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.315}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.436},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1212416,"data_stored":599308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.224,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.192},"max":{"1min":0.51900000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.54600000000000004,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.30399999999999999,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.60199999999999998,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.502},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.626,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.82399999999999995,"5min":0.88200000000000001,"15min":1.458},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.83799999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.746}]},{"osd":5,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.89000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.93700000000000006,"15min":1.028},"last":0.68400000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.625,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.879,"15min":0.92500000000000004},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575043,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":592058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.311}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.29599999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.55700000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65200000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.59099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1781760,"data_stored":1182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.25},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.27700000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.37},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.315}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:40:59.394 INFO:tasks.ceph.mds.c.smithi125.stderr:2023-10-18T10:40:59.392+0000 7f00a3ba1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:41:01.738 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:41:01.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.851+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/1441902510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00cf340 msgr2=0x7ff1e00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:01.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.851+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/1441902510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00cf340 0x7ff1e00d1910 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ff1e008a120 tx=0x7ff1d002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:41:01.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.851+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/1441902510 shutdown_connections 2023-10-18T10:41:01.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.851+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/1441902510 >> v2:172.21.15.139:3300/0 conn(0x7ff1e015b2a0 0x7ff1e00cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:01.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.851+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/1441902510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00cf340 0x7ff1e00d1910 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:01.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.851+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/1441902510 >> 172.21.15.125:0/1441902510 conn(0x7ff1e00d0d60 msgr2=0x7ff1e00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:01.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.851+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/1441902510 shutdown_connections 2023-10-18T10:41:01.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.851+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/1441902510 wait complete. 2023-10-18T10:41:01.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.852+0000 7ff1e5f9e640 1 Processor -- start 2023-10-18T10:41:01.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1e5f9e640 1 -- start start 2023-10-18T10:41:01.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1e5f9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00ceb40 0x7ff1e00b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:01.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1e5f9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e00cf340 0x7ff1e00c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:01.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1e5f9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1e015b2a0 0x7ff1e0154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:01.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1e5f9e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00c8530 con 0x7ff1e00ceb40 2023-10-18T10:41:01.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1e5f9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00c8810 con 0x7ff1e015b2a0 2023-10-18T10:41:01.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1e5f9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00c8af0 con 0x7ff1e00cf340 2023-10-18T10:41:01.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1df7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00ceb40 0x7ff1e00b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:01.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1df7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00ceb40 0x7ff1e00b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43378/0 (socket says 172.21.15.125:43378) 2023-10-18T10:41:01.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.853+0000 7ff1df7fe640 1 -- 172.21.15.125:0/710212735 learned_addr learned my addr 172.21.15.125:0/710212735 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:01.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1df7fe640 1 -- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e00cf340 msgr2=0x7ff1e00c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:01.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1dffff640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1e015b2a0 0x7ff1e0154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:01.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1df7fe640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e00cf340 0x7ff1e00c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:01.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1df7fe640 1 -- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1e015b2a0 msgr2=0x7ff1e0154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:01.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1df7fe640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1e015b2a0 0x7ff1e0154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:01.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1df7fe640 1 -- 172.21.15.125:0/710212735 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d0009490 con 0x7ff1e00ceb40 2023-10-18T10:41:01.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1df7fe640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00ceb40 0x7ff1e00b18b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ff1d002c300 tx=0x7ff1d00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:01.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1c7fff640 1 -- 172.21.15.125:0/710212735 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0003ea0 con 0x7ff1e00ceb40 2023-10-18T10:41:01.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1c7fff640 1 -- 172.21.15.125:0/710212735 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1d0034440 con 0x7ff1e00ceb40 2023-10-18T10:41:01.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e00c8dd0 con 0x7ff1e00ceb40 2023-10-18T10:41:01.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1c7fff640 1 -- 172.21.15.125:0/710212735 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0034810 con 0x7ff1e00ceb40 2023-10-18T10:41:01.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.854+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e0154cd0 con 0x7ff1e00ceb40 2023-10-18T10:41:01.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.855+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e0154e60 con 0x7ff1e00ceb40 2023-10-18T10:41:01.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.855+0000 7ff1c7fff640 1 -- 172.21.15.125:0/710212735 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff1d0034970 con 0x7ff1e00ceb40 2023-10-18T10:41:01.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.856+0000 7ff1c7fff640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff1cc06af60 0x7ff1cc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:01.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.856+0000 7ff1c7fff640 1 -- 172.21.15.125:0/710212735 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff1d00ac970 con 0x7ff1e00ceb40 2023-10-18T10:41:01.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.856+0000 7ff1deffd640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff1cc06af60 0x7ff1cc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:01.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.857+0000 7ff1deffd640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff1cc06af60 0x7ff1cc06d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00b22b0 tx=0x7ff1d4006bf0 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:01.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:01.860+0000 7ff1c7fff640 1 -- 172.21.15.125:0/710212735 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1d0077860 con 0x7ff1e00ceb40 2023-10-18T10:41:02.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.025+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 --> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1e00cb980 con 0x7ff1cc06af60 2023-10-18T10:41:02.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.033+0000 7ff1c7fff640 1 -- 172.21.15.125:0/710212735 <== mgr.5972 v2:172.21.15.139:6832/143413153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289473 (secure 0 0 0) 0x7ff1e00cb980 con 0x7ff1cc06af60 2023-10-18T10:41:02.038 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:02.039 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:41:02.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff1cc06af60 msgr2=0x7ff1cc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff1cc06af60 0x7ff1cc06d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00b22b0 tx=0x7ff1d4006bf0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00ceb40 msgr2=0x7ff1e00b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00ceb40 0x7ff1e00b18b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ff1d002c300 tx=0x7ff1d00038a0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 shutdown_connections 2023-10-18T10:41:02.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff1cc06af60 0x7ff1cc06d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1e015b2a0 0x7ff1e0154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e00cf340 0x7ff1e00c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 --2- 172.21.15.125:0/710212735 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff1e00ceb40 0x7ff1e00b18b0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 >> 172.21.15.125:0/710212735 conn(0x7ff1e00d0d60 msgr2=0x7ff1e00ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:02.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.036+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 shutdown_connections 2023-10-18T10:41:02.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.037+0000 7ff1e5f9e640 1 -- 172.21.15.125:0/710212735 wait complete. 2023-10-18T10:41:02.049 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-18T10:41:00.163543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":303,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":312740,"kb_used_data":10540,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754661980,"statfs":{"total":773094113280,"available":772773867520,"internally_reserved":0,"allocated":10792960,"data_stored":5940984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.754008"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144661+0000","last_change":"2023-10-18T10:40:54.429498+0000","last_active":"2023-10-18T10:40:56.144661+0000","last_peered":"2023-10-18T10:40:56.144661+0000","last_clean":"2023-10-18T10:40:56.144661+0000","last_became_active":"2023-10-18T10:40:54.429010+0000","last_became_peered":"2023-10-18T10:40:54.429010+0000","last_unstale":"2023-10-18T10:40:56.144661+0000","last_undegraded":"2023-10-18T10:40:56.144661+0000","last_fullsized":"2023-10-18T10:40:56.144661+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144511+0000","last_change":"2023-10-18T10:40:54.435140+0000","last_active":"2023-10-18T10:40:56.144511+0000","last_peered":"2023-10-18T10:40:56.144511+0000","last_clean":"2023-10-18T10:40:56.144511+0000","last_became_active":"2023-10-18T10:40:54.434657+0000","last_became_peered":"2023-10-18T10:40:54.434657+0000","last_unstale":"2023-10-18T10:40:56.144511+0000","last_undegraded":"2023-10-18T10:40:56.144511+0000","last_fullsized":"2023-10-18T10:40:56.144511+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.896314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144046+0000","last_change":"2023-10-18T10:40:54.424056+0000","last_active":"2023-10-18T10:40:56.144046+0000","last_peered":"2023-10-18T10:40:56.144046+0000","last_clean":"2023-10-18T10:40:56.144046+0000","last_became_active":"2023-10-18T10:40:54.423789+0000","last_became_peered":"2023-10-18T10:40:54.423789+0000","last_unstale":"2023-10-18T10:40:56.144046+0000","last_undegraded":"2023-10-18T10:40:56.144046+0000","last_fullsized":"2023-10-18T10:40:56.144046+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:29.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500661+0000","last_change":"2023-10-18T10:40:54.428671+0000","last_active":"2023-10-18T10:40:55.500661+0000","last_peered":"2023-10-18T10:40:55.500661+0000","last_clean":"2023-10-18T10:40:55.500661+0000","last_became_active":"2023-10-18T10:40:54.428396+0000","last_became_peered":"2023-10-18T10:40:54.428396+0000","last_unstale":"2023-10-18T10:40:55.500661+0000","last_undegraded":"2023-10-18T10:40:55.500661+0000","last_fullsized":"2023-10-18T10:40:55.500661+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:39.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018418+0000","last_change":"2023-10-18T10:40:54.433351+0000","last_active":"2023-10-18T10:40:56.018418+0000","last_peered":"2023-10-18T10:40:56.018418+0000","last_clean":"2023-10-18T10:40:56.018418+0000","last_became_active":"2023-10-18T10:40:54.430314+0000","last_became_peered":"2023-10-18T10:40:54.430314+0000","last_unstale":"2023-10-18T10:40:56.018418+0000","last_undegraded":"2023-10-18T10:40:56.018418+0000","last_fullsized":"2023-10-18T10:40:56.018418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:48.808404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429115+0000","last_change":"2023-10-18T10:40:54.431510+0000","last_active":"2023-10-18T10:40:55.429115+0000","last_peered":"2023-10-18T10:40:55.429115+0000","last_clean":"2023-10-18T10:40:55.429115+0000","last_became_active":"2023-10-18T10:40:54.431184+0000","last_became_peered":"2023-10-18T10:40:54.431184+0000","last_unstale":"2023-10-18T10:40:55.429115+0000","last_undegraded":"2023-10-18T10:40:55.429115+0000","last_fullsized":"2023-10-18T10:40:55.429115+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:33.893983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144553+0000","last_change":"2023-10-18T10:40:54.434704+0000","last_active":"2023-10-18T10:40:56.144553+0000","last_peered":"2023-10-18T10:40:56.144553+0000","last_clean":"2023-10-18T10:40:56.144553+0000","last_became_active":"2023-10-18T10:40:54.434388+0000","last_became_peered":"2023-10-18T10:40:54.434388+0000","last_unstale":"2023-10-18T10:40:56.144553+0000","last_undegraded":"2023-10-18T10:40:56.144553+0000","last_fullsized":"2023-10-18T10:40:56.144553+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:54.346618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:54.427003+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:54.426743+0000","last_became_peered":"2023-10-18T10:40:54.426743+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:58.361349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434857+0000","last_change":"2023-10-18T10:40:54.424669+0000","last_active":"2023-10-18T10:40:55.434857+0000","last_peered":"2023-10-18T10:40:55.434857+0000","last_clean":"2023-10-18T10:40:55.434857+0000","last_became_active":"2023-10-18T10:40:54.424408+0000","last_became_peered":"2023-10-18T10:40:54.424408+0000","last_unstale":"2023-10-18T10:40:55.434857+0000","last_undegraded":"2023-10-18T10:40:55.434857+0000","last_fullsized":"2023-10-18T10:40:55.434857+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:07.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500867+0000","last_change":"2023-10-18T10:40:54.433837+0000","last_active":"2023-10-18T10:40:55.500867+0000","last_peered":"2023-10-18T10:40:55.500867+0000","last_clean":"2023-10-18T10:40:55.500867+0000","last_became_active":"2023-10-18T10:40:54.433558+0000","last_became_peered":"2023-10-18T10:40:54.433558+0000","last_unstale":"2023-10-18T10:40:55.500867+0000","last_undegraded":"2023-10-18T10:40:55.500867+0000","last_fullsized":"2023-10-18T10:40:55.500867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:04.849763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144518+0000","last_change":"2023-10-18T10:40:54.429847+0000","last_active":"2023-10-18T10:40:56.144518+0000","last_peered":"2023-10-18T10:40:56.144518+0000","last_clean":"2023-10-18T10:40:56.144518+0000","last_became_active":"2023-10-18T10:40:54.429629+0000","last_became_peered":"2023-10-18T10:40:54.429629+0000","last_unstale":"2023-10-18T10:40:56.144518+0000","last_undegraded":"2023-10-18T10:40:56.144518+0000","last_fullsized":"2023-10-18T10:40:56.144518+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:29.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500804+0000","last_change":"2023-10-18T10:40:54.427881+0000","last_active":"2023-10-18T10:40:55.500804+0000","last_peered":"2023-10-18T10:40:55.500804+0000","last_clean":"2023-10-18T10:40:55.500804+0000","last_became_active":"2023-10-18T10:40:54.427423+0000","last_became_peered":"2023-10-18T10:40:54.427423+0000","last_unstale":"2023-10-18T10:40:55.500804+0000","last_undegraded":"2023-10-18T10:40:55.500804+0000","last_fullsized":"2023-10-18T10:40:55.500804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:07.682253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018526+0000","last_change":"2023-10-18T10:40:54.426415+0000","last_active":"2023-10-18T10:40:56.018526+0000","last_peered":"2023-10-18T10:40:56.018526+0000","last_clean":"2023-10-18T10:40:56.018526+0000","last_became_active":"2023-10-18T10:40:54.426129+0000","last_became_peered":"2023-10-18T10:40:54.426129+0000","last_unstale":"2023-10-18T10:40:56.018526+0000","last_undegraded":"2023-10-18T10:40:56.018526+0000","last_fullsized":"2023-10-18T10:40:56.018526+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:33:27.749058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429139+0000","last_change":"2023-10-18T10:40:54.430848+0000","last_active":"2023-10-18T10:40:55.429139+0000","last_peered":"2023-10-18T10:40:55.429139+0000","last_clean":"2023-10-18T10:40:55.429139+0000","last_became_active":"2023-10-18T10:40:54.430562+0000","last_became_peered":"2023-10-18T10:40:54.430562+0000","last_unstale":"2023-10-18T10:40:55.429139+0000","last_undegraded":"2023-10-18T10:40:55.429139+0000","last_fullsized":"2023-10-18T10:40:55.429139+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:05.564173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144611+0000","last_change":"2023-10-18T10:40:54.421817+0000","last_active":"2023-10-18T10:40:56.144611+0000","last_peered":"2023-10-18T10:40:56.144611+0000","last_clean":"2023-10-18T10:40:56.144611+0000","last_became_active":"2023-10-18T10:40:54.421613+0000","last_became_peered":"2023-10-18T10:40:54.421613+0000","last_unstale":"2023-10-18T10:40:56.144611+0000","last_undegraded":"2023-10-18T10:40:56.144611+0000","last_fullsized":"2023-10-18T10:40:56.144611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:36.706863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144136+0000","last_change":"2023-10-18T10:40:54.434433+0000","last_active":"2023-10-18T10:40:56.144136+0000","last_peered":"2023-10-18T10:40:56.144136+0000","last_clean":"2023-10-18T10:40:56.144136+0000","last_became_active":"2023-10-18T10:40:54.433953+0000","last_became_peered":"2023-10-18T10:40:54.433953+0000","last_unstale":"2023-10-18T10:40:56.144136+0000","last_undegraded":"2023-10-18T10:40:56.144136+0000","last_fullsized":"2023-10-18T10:40:56.144136+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:32.284117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:54.435303+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:54.434740+0000","last_became_peered":"2023-10-18T10:40:54.434740+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:08.386244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144183+0000","last_change":"2023-10-18T10:40:54.427685+0000","last_active":"2023-10-18T10:40:56.144183+0000","last_peered":"2023-10-18T10:40:56.144183+0000","last_clean":"2023-10-18T10:40:56.144183+0000","last_became_active":"2023-10-18T10:40:54.427436+0000","last_became_peered":"2023-10-18T10:40:54.427436+0000","last_unstale":"2023-10-18T10:40:56.144183+0000","last_undegraded":"2023-10-18T10:40:56.144183+0000","last_fullsized":"2023-10-18T10:40:56.144183+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:26:02.501409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429236+0000","last_change":"2023-10-18T10:40:54.432868+0000","last_active":"2023-10-18T10:40:55.429236+0000","last_peered":"2023-10-18T10:40:55.429236+0000","last_clean":"2023-10-18T10:40:55.429236+0000","last_became_active":"2023-10-18T10:40:54.432637+0000","last_became_peered":"2023-10-18T10:40:54.432637+0000","last_unstale":"2023-10-18T10:40:55.429236+0000","last_undegraded":"2023-10-18T10:40:55.429236+0000","last_fullsized":"2023-10-18T10:40:55.429236+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:46.951599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018489+0000","last_change":"2023-10-18T10:40:54.430205+0000","last_active":"2023-10-18T10:40:56.018489+0000","last_peered":"2023-10-18T10:40:56.018489+0000","last_clean":"2023-10-18T10:40:56.018489+0000","last_became_active":"2023-10-18T10:40:54.429898+0000","last_became_peered":"2023-10-18T10:40:54.429898+0000","last_unstale":"2023-10-18T10:40:56.018489+0000","last_undegraded":"2023-10-18T10:40:56.018489+0000","last_fullsized":"2023-10-18T10:40:56.018489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:48.646925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434605+0000","last_change":"2023-10-18T10:40:54.424296+0000","last_active":"2023-10-18T10:40:55.434605+0000","last_peered":"2023-10-18T10:40:55.434605+0000","last_clean":"2023-10-18T10:40:55.434605+0000","last_became_active":"2023-10-18T10:40:54.423967+0000","last_became_peered":"2023-10-18T10:40:54.423967+0000","last_unstale":"2023-10-18T10:40:55.434605+0000","last_undegraded":"2023-10-18T10:40:55.434605+0000","last_fullsized":"2023-10-18T10:40:55.434605+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:31.240890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018318+0000","last_change":"2023-10-18T10:40:54.428004+0000","last_active":"2023-10-18T10:40:56.018318+0000","last_peered":"2023-10-18T10:40:56.018318+0000","last_clean":"2023-10-18T10:40:56.018318+0000","last_became_active":"2023-10-18T10:40:54.427533+0000","last_became_peered":"2023-10-18T10:40:54.427533+0000","last_unstale":"2023-10-18T10:40:56.018318+0000","last_undegraded":"2023-10-18T10:40:56.018318+0000","last_fullsized":"2023-10-18T10:40:56.018318+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:27:07.146678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434483+0000","last_change":"2023-10-18T10:40:54.420300+0000","last_active":"2023-10-18T10:40:55.434483+0000","last_peered":"2023-10-18T10:40:55.434483+0000","last_clean":"2023-10-18T10:40:55.434483+0000","last_became_active":"2023-10-18T10:40:54.420008+0000","last_became_peered":"2023-10-18T10:40:54.420008+0000","last_unstale":"2023-10-18T10:40:55.434483+0000","last_undegraded":"2023-10-18T10:40:55.434483+0000","last_fullsized":"2023-10-18T10:40:55.434483+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:20.853296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143943+0000","last_change":"2023-10-18T10:40:54.424777+0000","last_active":"2023-10-18T10:40:56.143943+0000","last_peered":"2023-10-18T10:40:56.143943+0000","last_clean":"2023-10-18T10:40:56.143943+0000","last_became_active":"2023-10-18T10:40:54.424544+0000","last_became_peered":"2023-10-18T10:40:54.424544+0000","last_unstale":"2023-10-18T10:40:56.143943+0000","last_undegraded":"2023-10-18T10:40:56.143943+0000","last_fullsized":"2023-10-18T10:40:56.143943+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144475+0000","last_change":"2023-10-18T10:40:54.425355+0000","last_active":"2023-10-18T10:40:56.144475+0000","last_peered":"2023-10-18T10:40:56.144475+0000","last_clean":"2023-10-18T10:40:56.144475+0000","last_became_active":"2023-10-18T10:40:54.425123+0000","last_became_peered":"2023-10-18T10:40:54.425123+0000","last_unstale":"2023-10-18T10:40:56.144475+0000","last_undegraded":"2023-10-18T10:40:56.144475+0000","last_fullsized":"2023-10-18T10:40:56.144475+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:19:24.991706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500878+0000","last_change":"2023-10-18T10:40:54.426993+0000","last_active":"2023-10-18T10:40:55.500878+0000","last_peered":"2023-10-18T10:40:55.500878+0000","last_clean":"2023-10-18T10:40:55.500878+0000","last_became_active":"2023-10-18T10:40:54.426702+0000","last_became_peered":"2023-10-18T10:40:54.426702+0000","last_unstale":"2023-10-18T10:40:55.500878+0000","last_undegraded":"2023-10-18T10:40:55.500878+0000","last_fullsized":"2023-10-18T10:40:55.500878+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:57.667741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144401+0000","last_change":"2023-10-18T10:40:54.430464+0000","last_active":"2023-10-18T10:40:56.144401+0000","last_peered":"2023-10-18T10:40:56.144401+0000","last_clean":"2023-10-18T10:40:56.144401+0000","last_became_active":"2023-10-18T10:40:54.430070+0000","last_became_peered":"2023-10-18T10:40:54.430070+0000","last_unstale":"2023-10-18T10:40:56.144401+0000","last_undegraded":"2023-10-18T10:40:56.144401+0000","last_fullsized":"2023-10-18T10:40:56.144401+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:46.951599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018550+0000","last_change":"2023-10-18T10:40:54.432406+0000","last_active":"2023-10-18T10:40:56.018550+0000","last_peered":"2023-10-18T10:40:56.018550+0000","last_clean":"2023-10-18T10:40:56.018550+0000","last_became_active":"2023-10-18T10:40:54.432113+0000","last_became_peered":"2023-10-18T10:40:54.432113+0000","last_unstale":"2023-10-18T10:40:56.018550+0000","last_undegraded":"2023-10-18T10:40:56.018550+0000","last_fullsized":"2023-10-18T10:40:56.018550+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:24:16.099369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144250+0000","last_change":"2023-10-18T10:40:54.431844+0000","last_active":"2023-10-18T10:40:56.144250+0000","last_peered":"2023-10-18T10:40:56.144250+0000","last_clean":"2023-10-18T10:40:56.144250+0000","last_became_active":"2023-10-18T10:40:54.431447+0000","last_became_peered":"2023-10-18T10:40:54.431447+0000","last_unstale":"2023-10-18T10:40:56.144250+0000","last_undegraded":"2023-10-18T10:40:56.144250+0000","last_fullsized":"2023-10-18T10:40:56.144250+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:30.965000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144201+0000","last_change":"2023-10-18T10:40:54.424148+0000","last_active":"2023-10-18T10:40:56.144201+0000","last_peered":"2023-10-18T10:40:56.144201+0000","last_clean":"2023-10-18T10:40:56.144201+0000","last_became_active":"2023-10-18T10:40:54.423867+0000","last_became_peered":"2023-10-18T10:40:54.423867+0000","last_unstale":"2023-10-18T10:40:56.144201+0000","last_undegraded":"2023-10-18T10:40:56.144201+0000","last_fullsized":"2023-10-18T10:40:56.144201+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:33.893983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143940+0000","last_change":"2023-10-18T10:40:54.433301+0000","last_active":"2023-10-18T10:40:56.143940+0000","last_peered":"2023-10-18T10:40:56.143940+0000","last_clean":"2023-10-18T10:40:56.143940+0000","last_became_active":"2023-10-18T10:40:54.430606+0000","last_became_peered":"2023-10-18T10:40:54.430606+0000","last_unstale":"2023-10-18T10:40:56.143940+0000","last_undegraded":"2023-10-18T10:40:56.143940+0000","last_fullsized":"2023-10-18T10:40:56.143940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:03.914078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144239+0000","last_change":"2023-10-18T10:40:54.431569+0000","last_active":"2023-10-18T10:40:56.144239+0000","last_peered":"2023-10-18T10:40:56.144239+0000","last_clean":"2023-10-18T10:40:56.144239+0000","last_became_active":"2023-10-18T10:40:54.431255+0000","last_became_peered":"2023-10-18T10:40:54.431255+0000","last_unstale":"2023-10-18T10:40:56.144239+0000","last_undegraded":"2023-10-18T10:40:56.144239+0000","last_fullsized":"2023-10-18T10:40:56.144239+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:05.564173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144039+0000","last_change":"2023-10-18T10:40:54.428297+0000","last_active":"2023-10-18T10:40:56.144039+0000","last_peered":"2023-10-18T10:40:56.144039+0000","last_clean":"2023-10-18T10:40:56.144039+0000","last_became_active":"2023-10-18T10:40:54.428053+0000","last_became_peered":"2023-10-18T10:40:54.428053+0000","last_unstale":"2023-10-18T10:40:56.144039+0000","last_undegraded":"2023-10-18T10:40:56.144039+0000","last_fullsized":"2023-10-18T10:40:56.144039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:41.417088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"97'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018641+0000","last_change":"2023-10-18T10:40:53.417631+0000","last_active":"2023-10-18T10:40:56.018641+0000","last_peered":"2023-10-18T10:40:56.018641+0000","last_clean":"2023-10-18T10:40:56.018641+0000","last_became_active":"2023-10-18T10:40:53.417381+0000","last_became_peered":"2023-10-18T10:40:53.417381+0000","last_unstale":"2023-10-18T10:40:56.018641+0000","last_undegraded":"2023-10-18T10:40:56.018641+0000","last_fullsized":"2023-10-18T10:40:56.018641+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:40.271808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144091+0000","last_change":"2023-10-18T10:40:54.420096+0000","last_active":"2023-10-18T10:40:56.144091+0000","last_peered":"2023-10-18T10:40:56.144091+0000","last_clean":"2023-10-18T10:40:56.144091+0000","last_became_active":"2023-10-18T10:40:54.419838+0000","last_became_peered":"2023-10-18T10:40:54.419838+0000","last_unstale":"2023-10-18T10:40:56.144091+0000","last_undegraded":"2023-10-18T10:40:56.144091+0000","last_fullsized":"2023-10-18T10:40:56.144091+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:17.947505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144489+0000","last_change":"2023-10-18T10:40:53.426284+0000","last_active":"2023-10-18T10:40:56.144489+0000","last_peered":"2023-10-18T10:40:56.144489+0000","last_clean":"2023-10-18T10:40:56.144489+0000","last_became_active":"2023-10-18T10:40:53.425978+0000","last_became_peered":"2023-10-18T10:40:53.425978+0000","last_unstale":"2023-10-18T10:40:56.144489+0000","last_undegraded":"2023-10-18T10:40:56.144489+0000","last_fullsized":"2023-10-18T10:40:56.144489+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:28.892929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434484+0000","last_change":"2023-10-18T10:40:54.425573+0000","last_active":"2023-10-18T10:40:55.434484+0000","last_peered":"2023-10-18T10:40:55.434484+0000","last_clean":"2023-10-18T10:40:55.434484+0000","last_became_active":"2023-10-18T10:40:54.425311+0000","last_became_peered":"2023-10-18T10:40:54.425311+0000","last_unstale":"2023-10-18T10:40:55.434484+0000","last_undegraded":"2023-10-18T10:40:55.434484+0000","last_fullsized":"2023-10-18T10:40:55.434484+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:21.514697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144286+0000","last_change":"2023-10-18T10:40:53.430881+0000","last_active":"2023-10-18T10:40:56.144286+0000","last_peered":"2023-10-18T10:40:56.144286+0000","last_clean":"2023-10-18T10:40:56.144286+0000","last_became_active":"2023-10-18T10:40:53.430566+0000","last_became_peered":"2023-10-18T10:40:53.430566+0000","last_unstale":"2023-10-18T10:40:56.144286+0000","last_undegraded":"2023-10-18T10:40:56.144286+0000","last_fullsized":"2023-10-18T10:40:56.144286+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:49.684089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434675+0000","last_change":"2023-10-18T10:40:54.423819+0000","last_active":"2023-10-18T10:40:55.434675+0000","last_peered":"2023-10-18T10:40:55.434675+0000","last_clean":"2023-10-18T10:40:55.434675+0000","last_became_active":"2023-10-18T10:40:54.423497+0000","last_became_peered":"2023-10-18T10:40:54.423497+0000","last_unstale":"2023-10-18T10:40:55.434675+0000","last_undegraded":"2023-10-18T10:40:55.434675+0000","last_fullsized":"2023-10-18T10:40:55.434675+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:18.617775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144791+0000","last_change":"2023-10-18T10:40:53.423111+0000","last_active":"2023-10-18T10:40:56.144791+0000","last_peered":"2023-10-18T10:40:56.144791+0000","last_clean":"2023-10-18T10:40:56.144791+0000","last_became_active":"2023-10-18T10:40:53.422848+0000","last_became_peered":"2023-10-18T10:40:53.422848+0000","last_unstale":"2023-10-18T10:40:56.144791+0000","last_undegraded":"2023-10-18T10:40:56.144791+0000","last_fullsized":"2023-10-18T10:40:56.144791+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:19.845504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144374+0000","last_change":"2023-10-18T10:40:54.435357+0000","last_active":"2023-10-18T10:40:56.144374+0000","last_peered":"2023-10-18T10:40:56.144374+0000","last_clean":"2023-10-18T10:40:56.144374+0000","last_became_active":"2023-10-18T10:40:54.435102+0000","last_became_peered":"2023-10-18T10:40:54.435102+0000","last_unstale":"2023-10-18T10:40:56.144374+0000","last_undegraded":"2023-10-18T10:40:56.144374+0000","last_fullsized":"2023-10-18T10:40:56.144374+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:28:23.536390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434703+0000","last_change":"2023-10-18T10:40:53.421367+0000","last_active":"2023-10-18T10:40:55.434703+0000","last_peered":"2023-10-18T10:40:55.434703+0000","last_clean":"2023-10-18T10:40:55.434703+0000","last_became_active":"2023-10-18T10:40:53.421030+0000","last_became_peered":"2023-10-18T10:40:53.421030+0000","last_unstale":"2023-10-18T10:40:55.434703+0000","last_undegraded":"2023-10-18T10:40:55.434703+0000","last_fullsized":"2023-10-18T10:40:55.434703+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:59.291678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144627+0000","last_change":"2023-10-18T10:40:54.428864+0000","last_active":"2023-10-18T10:40:56.144627+0000","last_peered":"2023-10-18T10:40:56.144627+0000","last_clean":"2023-10-18T10:40:56.144627+0000","last_became_active":"2023-10-18T10:40:54.428412+0000","last_became_peered":"2023-10-18T10:40:54.428412+0000","last_unstale":"2023-10-18T10:40:56.144627+0000","last_undegraded":"2023-10-18T10:40:56.144627+0000","last_fullsized":"2023-10-18T10:40:56.144627+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:15.038645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500519+0000","last_change":"2023-10-18T10:40:53.420317+0000","last_active":"2023-10-18T10:40:55.500519+0000","last_peered":"2023-10-18T10:40:55.500519+0000","last_clean":"2023-10-18T10:40:55.500519+0000","last_became_active":"2023-10-18T10:40:53.420046+0000","last_became_peered":"2023-10-18T10:40:53.420046+0000","last_unstale":"2023-10-18T10:40:55.500519+0000","last_undegraded":"2023-10-18T10:40:55.500519+0000","last_fullsized":"2023-10-18T10:40:55.500519+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:36.623092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434874+0000","last_change":"2023-10-18T10:40:54.426193+0000","last_active":"2023-10-18T10:40:55.434874+0000","last_peered":"2023-10-18T10:40:55.434874+0000","last_clean":"2023-10-18T10:40:55.434874+0000","last_became_active":"2023-10-18T10:40:54.425894+0000","last_became_peered":"2023-10-18T10:40:54.425894+0000","last_unstale":"2023-10-18T10:40:55.434874+0000","last_undegraded":"2023-10-18T10:40:55.434874+0000","last_fullsized":"2023-10-18T10:40:55.434874+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:04.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144181+0000","last_change":"2023-10-18T10:40:53.430760+0000","last_active":"2023-10-18T10:40:56.144181+0000","last_peered":"2023-10-18T10:40:56.144181+0000","last_clean":"2023-10-18T10:40:56.144181+0000","last_became_active":"2023-10-18T10:40:53.430469+0000","last_became_peered":"2023-10-18T10:40:53.430469+0000","last_unstale":"2023-10-18T10:40:56.144181+0000","last_undegraded":"2023-10-18T10:40:56.144181+0000","last_fullsized":"2023-10-18T10:40:56.144181+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:45.943807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144991+0000","last_change":"2023-10-18T10:40:54.435078+0000","last_active":"2023-10-18T10:40:56.144991+0000","last_peered":"2023-10-18T10:40:56.144991+0000","last_clean":"2023-10-18T10:40:56.144991+0000","last_became_active":"2023-10-18T10:40:54.434517+0000","last_became_peered":"2023-10-18T10:40:54.434517+0000","last_unstale":"2023-10-18T10:40:56.144991+0000","last_undegraded":"2023-10-18T10:40:56.144991+0000","last_fullsized":"2023-10-18T10:40:56.144991+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:40.219033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018282+0000","last_change":"2023-10-18T10:40:53.415948+0000","last_active":"2023-10-18T10:40:56.018282+0000","last_peered":"2023-10-18T10:40:56.018282+0000","last_clean":"2023-10-18T10:40:56.018282+0000","last_became_active":"2023-10-18T10:40:53.415668+0000","last_became_peered":"2023-10-18T10:40:53.415668+0000","last_unstale":"2023-10-18T10:40:56.018282+0000","last_undegraded":"2023-10-18T10:40:56.018282+0000","last_fullsized":"2023-10-18T10:40:56.018282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:12.409497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144074+0000","last_change":"2023-10-18T10:40:54.426165+0000","last_active":"2023-10-18T10:40:56.144074+0000","last_peered":"2023-10-18T10:40:56.144074+0000","last_clean":"2023-10-18T10:40:56.144074+0000","last_became_active":"2023-10-18T10:40:54.425916+0000","last_became_peered":"2023-10-18T10:40:54.425916+0000","last_unstale":"2023-10-18T10:40:56.144074+0000","last_undegraded":"2023-10-18T10:40:56.144074+0000","last_fullsized":"2023-10-18T10:40:56.144074+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144650+0000","last_change":"2023-10-18T10:40:53.428251+0000","last_active":"2023-10-18T10:40:56.144650+0000","last_peered":"2023-10-18T10:40:56.144650+0000","last_clean":"2023-10-18T10:40:56.144650+0000","last_became_active":"2023-10-18T10:40:53.427975+0000","last_became_peered":"2023-10-18T10:40:53.427975+0000","last_unstale":"2023-10-18T10:40:56.144650+0000","last_undegraded":"2023-10-18T10:40:56.144650+0000","last_fullsized":"2023-10-18T10:40:56.144650+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:45:58.336437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500662+0000","last_change":"2023-10-18T10:40:54.428238+0000","last_active":"2023-10-18T10:40:55.500662+0000","last_peered":"2023-10-18T10:40:55.500662+0000","last_clean":"2023-10-18T10:40:55.500662+0000","last_became_active":"2023-10-18T10:40:54.427404+0000","last_became_peered":"2023-10-18T10:40:54.427404+0000","last_unstale":"2023-10-18T10:40:55.500662+0000","last_undegraded":"2023-10-18T10:40:55.500662+0000","last_fullsized":"2023-10-18T10:40:55.500662+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:06.729491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500959+0000","last_change":"2023-10-18T10:40:53.423540+0000","last_active":"2023-10-18T10:40:55.500959+0000","last_peered":"2023-10-18T10:40:55.500959+0000","last_clean":"2023-10-18T10:40:55.500959+0000","last_became_active":"2023-10-18T10:40:53.423278+0000","last_became_peered":"2023-10-18T10:40:53.423278+0000","last_unstale":"2023-10-18T10:40:55.500959+0000","last_undegraded":"2023-10-18T10:40:55.500959+0000","last_fullsized":"2023-10-18T10:40:55.500959+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:15.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429349+0000","last_change":"2023-10-18T10:40:54.423285+0000","last_active":"2023-10-18T10:40:55.429349+0000","last_peered":"2023-10-18T10:40:55.429349+0000","last_clean":"2023-10-18T10:40:55.429349+0000","last_became_active":"2023-10-18T10:40:54.423013+0000","last_became_peered":"2023-10-18T10:40:54.423013+0000","last_unstale":"2023-10-18T10:40:55.429349+0000","last_undegraded":"2023-10-18T10:40:55.429349+0000","last_fullsized":"2023-10-18T10:40:55.429349+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:54.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"97'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018642+0000","last_change":"2023-10-18T10:40:53.424070+0000","last_active":"2023-10-18T10:40:56.018642+0000","last_peered":"2023-10-18T10:40:56.018642+0000","last_clean":"2023-10-18T10:40:56.018642+0000","last_became_active":"2023-10-18T10:40:53.423799+0000","last_became_peered":"2023-10-18T10:40:53.423799+0000","last_unstale":"2023-10-18T10:40:56.018642+0000","last_undegraded":"2023-10-18T10:40:56.018642+0000","last_fullsized":"2023-10-18T10:40:56.018642+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:09.891958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143935+0000","last_change":"2023-10-18T10:40:54.426135+0000","last_active":"2023-10-18T10:40:56.143935+0000","last_peered":"2023-10-18T10:40:56.143935+0000","last_clean":"2023-10-18T10:40:56.143935+0000","last_became_active":"2023-10-18T10:40:54.425409+0000","last_became_peered":"2023-10-18T10:40:54.425409+0000","last_unstale":"2023-10-18T10:40:56.143935+0000","last_undegraded":"2023-10-18T10:40:56.143935+0000","last_fullsized":"2023-10-18T10:40:56.143935+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:01.527219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"97'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144832+0000","last_change":"2023-10-18T10:40:53.422980+0000","last_active":"2023-10-18T10:40:56.144832+0000","last_peered":"2023-10-18T10:40:56.144832+0000","last_clean":"2023-10-18T10:40:56.144832+0000","last_became_active":"2023-10-18T10:40:53.422696+0000","last_became_peered":"2023-10-18T10:40:53.422696+0000","last_unstale":"2023-10-18T10:40:56.144832+0000","last_undegraded":"2023-10-18T10:40:56.144832+0000","last_fullsized":"2023-10-18T10:40:56.144832+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:20.506905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500893+0000","last_change":"2023-10-18T10:40:54.423164+0000","last_active":"2023-10-18T10:40:55.500893+0000","last_peered":"2023-10-18T10:40:55.500893+0000","last_clean":"2023-10-18T10:40:55.500893+0000","last_became_active":"2023-10-18T10:40:54.422844+0000","last_became_peered":"2023-10-18T10:40:54.422844+0000","last_unstale":"2023-10-18T10:40:55.500893+0000","last_undegraded":"2023-10-18T10:40:55.500893+0000","last_fullsized":"2023-10-18T10:40:55.500893+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:50.691881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500565+0000","last_change":"2023-10-18T10:40:53.420785+0000","last_active":"2023-10-18T10:40:55.500565+0000","last_peered":"2023-10-18T10:40:55.500565+0000","last_clean":"2023-10-18T10:40:55.500565+0000","last_became_active":"2023-10-18T10:40:53.420530+0000","last_became_peered":"2023-10-18T10:40:53.420530+0000","last_unstale":"2023-10-18T10:40:55.500565+0000","last_undegraded":"2023-10-18T10:40:55.500565+0000","last_fullsized":"2023-10-18T10:40:55.500565+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:44.748900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144877+0000","last_change":"2023-10-18T10:40:54.430742+0000","last_active":"2023-10-18T10:40:56.144877+0000","last_peered":"2023-10-18T10:40:56.144877+0000","last_clean":"2023-10-18T10:40:56.144877+0000","last_became_active":"2023-10-18T10:40:54.430455+0000","last_became_peered":"2023-10-18T10:40:54.430455+0000","last_unstale":"2023-10-18T10:40:56.144877+0000","last_undegraded":"2023-10-18T10:40:56.144877+0000","last_fullsized":"2023-10-18T10:40:56.144877+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:39.892546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144118+0000","last_change":"2023-10-18T10:40:53.428666+0000","last_active":"2023-10-18T10:40:56.144118+0000","last_peered":"2023-10-18T10:40:56.144118+0000","last_clean":"2023-10-18T10:40:56.144118+0000","last_became_active":"2023-10-18T10:40:53.428330+0000","last_became_peered":"2023-10-18T10:40:53.428330+0000","last_unstale":"2023-10-18T10:40:56.144118+0000","last_undegraded":"2023-10-18T10:40:56.144118+0000","last_fullsized":"2023-10-18T10:40:56.144118+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:06.674462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144941+0000","last_change":"2023-10-18T10:40:54.430021+0000","last_active":"2023-10-18T10:40:56.144941+0000","last_peered":"2023-10-18T10:40:56.144941+0000","last_clean":"2023-10-18T10:40:56.144941+0000","last_became_active":"2023-10-18T10:40:54.429684+0000","last_became_peered":"2023-10-18T10:40:54.429684+0000","last_unstale":"2023-10-18T10:40:56.144941+0000","last_undegraded":"2023-10-18T10:40:56.144941+0000","last_fullsized":"2023-10-18T10:40:56.144941+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:03.648375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018334+0000","last_change":"2023-10-18T10:40:53.425774+0000","last_active":"2023-10-18T10:40:56.018334+0000","last_peered":"2023-10-18T10:40:56.018334+0000","last_clean":"2023-10-18T10:40:56.018334+0000","last_became_active":"2023-10-18T10:40:53.425518+0000","last_became_peered":"2023-10-18T10:40:53.425518+0000","last_unstale":"2023-10-18T10:40:56.018334+0000","last_undegraded":"2023-10-18T10:40:56.018334+0000","last_fullsized":"2023-10-18T10:40:56.018334+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:30.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145025+0000","last_change":"2023-10-18T10:40:54.432786+0000","last_active":"2023-10-18T10:40:56.145025+0000","last_peered":"2023-10-18T10:40:56.145025+0000","last_clean":"2023-10-18T10:40:56.145025+0000","last_became_active":"2023-10-18T10:40:54.432538+0000","last_became_peered":"2023-10-18T10:40:54.432538+0000","last_unstale":"2023-10-18T10:40:56.145025+0000","last_undegraded":"2023-10-18T10:40:56.145025+0000","last_fullsized":"2023-10-18T10:40:56.145025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:52:39.855703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"97'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144160+0000","last_change":"2023-10-18T10:40:53.426701+0000","last_active":"2023-10-18T10:40:56.144160+0000","last_peered":"2023-10-18T10:40:56.144160+0000","last_clean":"2023-10-18T10:40:56.144160+0000","last_became_active":"2023-10-18T10:40:53.426431+0000","last_became_peered":"2023-10-18T10:40:53.426431+0000","last_unstale":"2023-10-18T10:40:56.144160+0000","last_undegraded":"2023-10-18T10:40:56.144160+0000","last_fullsized":"2023-10-18T10:40:56.144160+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:31.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429351+0000","last_change":"2023-10-18T10:40:54.434075+0000","last_active":"2023-10-18T10:40:55.429351+0000","last_peered":"2023-10-18T10:40:55.429351+0000","last_clean":"2023-10-18T10:40:55.429351+0000","last_became_active":"2023-10-18T10:40:54.433835+0000","last_became_peered":"2023-10-18T10:40:54.433835+0000","last_unstale":"2023-10-18T10:40:55.429351+0000","last_undegraded":"2023-10-18T10:40:55.429351+0000","last_fullsized":"2023-10-18T10:40:55.429351+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:31.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144738+0000","last_change":"2023-10-18T10:40:53.428407+0000","last_active":"2023-10-18T10:40:56.144738+0000","last_peered":"2023-10-18T10:40:56.144738+0000","last_clean":"2023-10-18T10:40:56.144738+0000","last_became_active":"2023-10-18T10:40:53.428139+0000","last_became_peered":"2023-10-18T10:40:53.428139+0000","last_unstale":"2023-10-18T10:40:56.144738+0000","last_undegraded":"2023-10-18T10:40:56.144738+0000","last_fullsized":"2023-10-18T10:40:56.144738+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:54.556080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018402+0000","last_change":"2023-10-18T10:40:54.435840+0000","last_active":"2023-10-18T10:40:56.018402+0000","last_peered":"2023-10-18T10:40:56.018402+0000","last_clean":"2023-10-18T10:40:56.018402+0000","last_became_active":"2023-10-18T10:40:54.435594+0000","last_became_peered":"2023-10-18T10:40:54.435594+0000","last_unstale":"2023-10-18T10:40:56.018402+0000","last_undegraded":"2023-10-18T10:40:56.018402+0000","last_fullsized":"2023-10-18T10:40:56.018402+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:16:01.996675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434903+0000","last_change":"2023-10-18T10:40:53.424603+0000","last_active":"2023-10-18T10:40:55.434903+0000","last_peered":"2023-10-18T10:40:55.434903+0000","last_clean":"2023-10-18T10:40:55.434903+0000","last_became_active":"2023-10-18T10:40:53.424328+0000","last_became_peered":"2023-10-18T10:40:53.424328+0000","last_unstale":"2023-10-18T10:40:55.434903+0000","last_undegraded":"2023-10-18T10:40:55.434903+0000","last_fullsized":"2023-10-18T10:40:55.434903+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:49.348865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018391+0000","last_change":"2023-10-18T10:40:54.427794+0000","last_active":"2023-10-18T10:40:56.018391+0000","last_peered":"2023-10-18T10:40:56.018391+0000","last_clean":"2023-10-18T10:40:56.018391+0000","last_became_active":"2023-10-18T10:40:54.427255+0000","last_became_peered":"2023-10-18T10:40:54.427255+0000","last_unstale":"2023-10-18T10:40:56.018391+0000","last_undegraded":"2023-10-18T10:40:56.018391+0000","last_fullsized":"2023-10-18T10:40:56.018391+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:08.181951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"97'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018673+0000","last_change":"2023-10-18T10:40:53.416910+0000","last_active":"2023-10-18T10:40:56.018673+0000","last_peered":"2023-10-18T10:40:56.018673+0000","last_clean":"2023-10-18T10:40:56.018673+0000","last_became_active":"2023-10-18T10:40:53.416630+0000","last_became_peered":"2023-10-18T10:40:53.416630+0000","last_unstale":"2023-10-18T10:40:56.018673+0000","last_undegraded":"2023-10-18T10:40:56.018673+0000","last_fullsized":"2023-10-18T10:40:56.018673+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:19:24.474032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429270+0000","last_change":"2023-10-18T10:40:54.439124+0000","last_active":"2023-10-18T10:40:55.429270+0000","last_peered":"2023-10-18T10:40:55.429270+0000","last_clean":"2023-10-18T10:40:55.429270+0000","last_became_active":"2023-10-18T10:40:54.438869+0000","last_became_peered":"2023-10-18T10:40:54.438869+0000","last_unstale":"2023-10-18T10:40:55.429270+0000","last_undegraded":"2023-10-18T10:40:55.429270+0000","last_fullsized":"2023-10-18T10:40:55.429270+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:50.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143888+0000","last_change":"2023-10-18T10:40:53.429322+0000","last_active":"2023-10-18T10:40:56.143888+0000","last_peered":"2023-10-18T10:40:56.143888+0000","last_clean":"2023-10-18T10:40:56.143888+0000","last_became_active":"2023-10-18T10:40:53.429093+0000","last_became_peered":"2023-10-18T10:40:53.429093+0000","last_unstale":"2023-10-18T10:40:56.143888+0000","last_undegraded":"2023-10-18T10:40:56.143888+0000","last_fullsized":"2023-10-18T10:40:56.143888+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:14.928492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144441+0000","last_change":"2023-10-18T10:40:54.420623+0000","last_active":"2023-10-18T10:40:56.144441+0000","last_peered":"2023-10-18T10:40:56.144441+0000","last_clean":"2023-10-18T10:40:56.144441+0000","last_became_active":"2023-10-18T10:40:54.420398+0000","last_became_peered":"2023-10-18T10:40:54.420398+0000","last_unstale":"2023-10-18T10:40:56.144441+0000","last_undegraded":"2023-10-18T10:40:56.144441+0000","last_fullsized":"2023-10-18T10:40:56.144441+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:43.611760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429356+0000","last_change":"2023-10-18T10:40:53.424629+0000","last_active":"2023-10-18T10:40:55.429356+0000","last_peered":"2023-10-18T10:40:55.429356+0000","last_clean":"2023-10-18T10:40:55.429356+0000","last_became_active":"2023-10-18T10:40:53.424387+0000","last_became_peered":"2023-10-18T10:40:53.424387+0000","last_unstale":"2023-10-18T10:40:55.429356+0000","last_undegraded":"2023-10-18T10:40:55.429356+0000","last_fullsized":"2023-10-18T10:40:55.429356+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:54.556080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144592+0000","last_change":"2023-10-18T10:40:54.427692+0000","last_active":"2023-10-18T10:40:56.144592+0000","last_peered":"2023-10-18T10:40:56.144592+0000","last_clean":"2023-10-18T10:40:56.144592+0000","last_became_active":"2023-10-18T10:40:54.427408+0000","last_became_peered":"2023-10-18T10:40:54.427408+0000","last_unstale":"2023-10-18T10:40:56.144592+0000","last_undegraded":"2023-10-18T10:40:56.144592+0000","last_fullsized":"2023-10-18T10:40:56.144592+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:47:48.326742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429051+0000","last_change":"2023-10-18T10:40:53.426213+0000","last_active":"2023-10-18T10:40:55.429051+0000","last_peered":"2023-10-18T10:40:55.429051+0000","last_clean":"2023-10-18T10:40:55.429051+0000","last_became_active":"2023-10-18T10:40:53.425947+0000","last_became_peered":"2023-10-18T10:40:53.425947+0000","last_unstale":"2023-10-18T10:40:55.429051+0000","last_undegraded":"2023-10-18T10:40:55.429051+0000","last_fullsized":"2023-10-18T10:40:55.429051+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:45:58.336437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.145071+0000","last_change":"2023-10-18T10:40:54.429476+0000","last_active":"2023-10-18T10:40:56.145071+0000","last_peered":"2023-10-18T10:40:56.145071+0000","last_clean":"2023-10-18T10:40:56.145071+0000","last_became_active":"2023-10-18T10:40:54.429109+0000","last_became_peered":"2023-10-18T10:40:54.429109+0000","last_unstale":"2023-10-18T10:40:56.145071+0000","last_undegraded":"2023-10-18T10:40:56.145071+0000","last_fullsized":"2023-10-18T10:40:56.145071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:58.443341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.428986+0000","last_change":"2023-10-18T10:40:53.425821+0000","last_active":"2023-10-18T10:40:55.428986+0000","last_peered":"2023-10-18T10:40:55.428986+0000","last_clean":"2023-10-18T10:40:55.428986+0000","last_became_active":"2023-10-18T10:40:53.425567+0000","last_became_peered":"2023-10-18T10:40:53.425567+0000","last_unstale":"2023-10-18T10:40:55.428986+0000","last_undegraded":"2023-10-18T10:40:55.428986+0000","last_fullsized":"2023-10-18T10:40:55.428986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:18.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434418+0000","last_change":"2023-10-18T10:40:54.430249+0000","last_active":"2023-10-18T10:40:55.434418+0000","last_peered":"2023-10-18T10:40:55.434418+0000","last_clean":"2023-10-18T10:40:55.434418+0000","last_became_active":"2023-10-18T10:40:54.429992+0000","last_became_peered":"2023-10-18T10:40:54.429992+0000","last_unstale":"2023-10-18T10:40:55.434418+0000","last_undegraded":"2023-10-18T10:40:55.434418+0000","last_fullsized":"2023-10-18T10:40:55.434418+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:26.621442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"97'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.500839+0000","last_change":"2023-10-18T10:40:53.424258+0000","last_active":"2023-10-18T10:40:55.500839+0000","last_peered":"2023-10-18T10:40:55.500839+0000","last_clean":"2023-10-18T10:40:55.500839+0000","last_became_active":"2023-10-18T10:40:53.423997+0000","last_became_peered":"2023-10-18T10:40:53.423997+0000","last_unstale":"2023-10-18T10:40:55.500839+0000","last_undegraded":"2023-10-18T10:40:55.500839+0000","last_fullsized":"2023-10-18T10:40:55.500839+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:05.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144724+0000","last_change":"2023-10-18T10:40:54.433951+0000","last_active":"2023-10-18T10:40:56.144724+0000","last_peered":"2023-10-18T10:40:56.144724+0000","last_clean":"2023-10-18T10:40:56.144724+0000","last_became_active":"2023-10-18T10:40:54.433371+0000","last_became_peered":"2023-10-18T10:40:54.433371+0000","last_unstale":"2023-10-18T10:40:56.144724+0000","last_undegraded":"2023-10-18T10:40:56.144724+0000","last_fullsized":"2023-10-18T10:40:56.144724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:04.260422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144585+0000","last_change":"2023-10-18T10:40:53.431024+0000","last_active":"2023-10-18T10:40:56.144585+0000","last_peered":"2023-10-18T10:40:56.144585+0000","last_clean":"2023-10-18T10:40:56.144585+0000","last_became_active":"2023-10-18T10:40:53.430743+0000","last_became_peered":"2023-10-18T10:40:53.430743+0000","last_unstale":"2023-10-18T10:40:56.144585+0000","last_undegraded":"2023-10-18T10:40:56.144585+0000","last_fullsized":"2023-10-18T10:40:56.144585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:32.886192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.434756+0000","last_change":"2023-10-18T10:40:54.425246+0000","last_active":"2023-10-18T10:40:55.434756+0000","last_peered":"2023-10-18T10:40:55.434756+0000","last_clean":"2023-10-18T10:40:55.434756+0000","last_became_active":"2023-10-18T10:40:54.424838+0000","last_became_peered":"2023-10-18T10:40:54.424838+0000","last_unstale":"2023-10-18T10:40:55.434756+0000","last_undegraded":"2023-10-18T10:40:55.434756+0000","last_fullsized":"2023-10-18T10:40:55.434756+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:54.346618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144500+0000","last_change":"2023-10-18T10:40:53.428598+0000","last_active":"2023-10-18T10:40:56.144500+0000","last_peered":"2023-10-18T10:40:56.144500+0000","last_clean":"2023-10-18T10:40:56.144500+0000","last_became_active":"2023-10-18T10:40:53.428102+0000","last_became_peered":"2023-10-18T10:40:53.428102+0000","last_unstale":"2023-10-18T10:40:56.144500+0000","last_undegraded":"2023-10-18T10:40:56.144500+0000","last_fullsized":"2023-10-18T10:40:56.144500+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:51:40.263102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144379+0000","last_change":"2023-10-18T10:40:54.431077+0000","last_active":"2023-10-18T10:40:56.144379+0000","last_peered":"2023-10-18T10:40:56.144379+0000","last_clean":"2023-10-18T10:40:56.144379+0000","last_became_active":"2023-10-18T10:40:54.430805+0000","last_became_peered":"2023-10-18T10:40:54.430805+0000","last_unstale":"2023-10-18T10:40:56.144379+0000","last_undegraded":"2023-10-18T10:40:56.144379+0000","last_fullsized":"2023-10-18T10:40:56.144379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:49:55.716517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144226+0000","last_change":"2023-10-18T10:40:53.426405+0000","last_active":"2023-10-18T10:40:56.144226+0000","last_peered":"2023-10-18T10:40:56.144226+0000","last_clean":"2023-10-18T10:40:56.144226+0000","last_became_active":"2023-10-18T10:40:53.426121+0000","last_became_peered":"2023-10-18T10:40:53.426121+0000","last_unstale":"2023-10-18T10:40:56.144226+0000","last_undegraded":"2023-10-18T10:40:56.144226+0000","last_fullsized":"2023-10-18T10:40:56.144226+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:03.841971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144673+0000","last_change":"2023-10-18T10:40:54.433504+0000","last_active":"2023-10-18T10:40:56.144673+0000","last_peered":"2023-10-18T10:40:56.144673+0000","last_clean":"2023-10-18T10:40:56.144673+0000","last_became_active":"2023-10-18T10:40:54.433264+0000","last_became_peered":"2023-10-18T10:40:54.433264+0000","last_unstale":"2023-10-18T10:40:56.144673+0000","last_undegraded":"2023-10-18T10:40:56.144673+0000","last_fullsized":"2023-10-18T10:40:56.144673+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:07.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144446+0000","last_change":"2023-10-18T10:40:53.424988+0000","last_active":"2023-10-18T10:40:56.144446+0000","last_peered":"2023-10-18T10:40:56.144446+0000","last_clean":"2023-10-18T10:40:56.144446+0000","last_became_active":"2023-10-18T10:40:53.423255+0000","last_became_peered":"2023-10-18T10:40:53.423255+0000","last_unstale":"2023-10-18T10:40:56.144446+0000","last_undegraded":"2023-10-18T10:40:56.144446+0000","last_fullsized":"2023-10-18T10:40:56.144446+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:30.233098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144920+0000","last_change":"2023-10-18T10:40:54.434657+0000","last_active":"2023-10-18T10:40:56.144920+0000","last_peered":"2023-10-18T10:40:56.144920+0000","last_clean":"2023-10-18T10:40:56.144920+0000","last_became_active":"2023-10-18T10:40:54.434344+0000","last_became_peered":"2023-10-18T10:40:54.434344+0000","last_unstale":"2023-10-18T10:40:56.144920+0000","last_undegraded":"2023-10-18T10:40:56.144920+0000","last_fullsized":"2023-10-18T10:40:56.144920+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:49.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.018412+0000","last_change":"2023-10-18T10:40:53.419056+0000","last_active":"2023-10-18T10:40:56.018412+0000","last_peered":"2023-10-18T10:40:56.018412+0000","last_clean":"2023-10-18T10:40:56.018412+0000","last_became_active":"2023-10-18T10:40:53.418815+0000","last_became_peered":"2023-10-18T10:40:53.418815+0000","last_unstale":"2023-10-18T10:40:56.018412+0000","last_undegraded":"2023-10-18T10:40:56.018412+0000","last_fullsized":"2023-10-18T10:40:56.018412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:13.857868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144317+0000","last_change":"2023-10-18T10:40:54.423909+0000","last_active":"2023-10-18T10:40:56.144317+0000","last_peered":"2023-10-18T10:40:56.144317+0000","last_clean":"2023-10-18T10:40:56.144317+0000","last_became_active":"2023-10-18T10:40:54.423659+0000","last_became_peered":"2023-10-18T10:40:54.423659+0000","last_unstale":"2023-10-18T10:40:56.144317+0000","last_undegraded":"2023-10-18T10:40:56.144317+0000","last_fullsized":"2023-10-18T10:40:56.144317+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:49:26.295205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144758+0000","last_change":"2023-10-18T10:40:53.431352+0000","last_active":"2023-10-18T10:40:56.144758+0000","last_peered":"2023-10-18T10:40:56.144758+0000","last_clean":"2023-10-18T10:40:56.144758+0000","last_became_active":"2023-10-18T10:40:53.431047+0000","last_became_peered":"2023-10-18T10:40:53.431047+0000","last_unstale":"2023-10-18T10:40:56.144758+0000","last_undegraded":"2023-10-18T10:40:56.144758+0000","last_fullsized":"2023-10-18T10:40:56.144758+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:17.609983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144229+0000","last_change":"2023-10-18T10:40:54.427209+0000","last_active":"2023-10-18T10:40:56.144229+0000","last_peered":"2023-10-18T10:40:56.144229+0000","last_clean":"2023-10-18T10:40:56.144229+0000","last_became_active":"2023-10-18T10:40:54.426950+0000","last_became_peered":"2023-10-18T10:40:54.426950+0000","last_unstale":"2023-10-18T10:40:56.144229+0000","last_undegraded":"2023-10-18T10:40:56.144229+0000","last_fullsized":"2023-10-18T10:40:56.144229+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:10.812136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.143765+0000","last_change":"2023-10-18T10:40:53.426790+0000","last_active":"2023-10-18T10:40:56.143765+0000","last_peered":"2023-10-18T10:40:56.143765+0000","last_clean":"2023-10-18T10:40:56.143765+0000","last_became_active":"2023-10-18T10:40:53.426556+0000","last_became_peered":"2023-10-18T10:40:53.426556+0000","last_unstale":"2023-10-18T10:40:56.143765+0000","last_undegraded":"2023-10-18T10:40:56.143765+0000","last_fullsized":"2023-10-18T10:40:56.143765+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:03.450696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:56.144433+0000","last_change":"2023-10-18T10:40:54.430137+0000","last_active":"2023-10-18T10:40:56.144433+0000","last_peered":"2023-10-18T10:40:56.144433+0000","last_clean":"2023-10-18T10:40:56.144433+0000","last_became_active":"2023-10-18T10:40:54.429879+0000","last_became_peered":"2023-10-18T10:40:54.429879+0000","last_unstale":"2023-10-18T10:40:56.144433+0000","last_undegraded":"2023-10-18T10:40:56.144433+0000","last_fullsized":"2023-10-18T10:40:56.144433+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:53.401392+0000","last_clean_scrub_stamp":"2023-10-18T10:40:53.401392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:50.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-18T10:40:55.429479+0000","last_change":"2023-10-18T10:40:53.425307+0000","last_active":"2023-10-18T10:40:55.429479+0000","last_peered":"2023-10-18T10:40:55.429479+0000","last_clean":"2023-10-18T10:40:55.429479+0000","last_became_active":"2023-10-18T10:40:53.424835+0000","last_became_peered":"2023-10-18T10:40:53.424835+0000","last_unstale":"2023-10-18T10:40:55.429479+0000","last_undegraded":"2023-10-18T10:40:55.429479+0000","last_fullsized":"2023-10-18T10:40:55.429479+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:40:52.393600+0000","last_clean_scrub_stamp":"2023-10-18T10:40:52.393600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:04.973143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"86'172","reported_seq":433,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-18T10:40:54.422694+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:40:54.422694+0000","last_peered":"2023-10-18T10:40:54.422694+0000","last_clean":"2023-10-18T10:40:54.422694+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:40:54.422694+0000","last_undegraded":"2023-10-18T10:40:54.422694+0000","last_fullsized":"2023-10-18T10:40:54.422694+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59296,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312544,"statfs":{"total":96636764160,"available":96576045056,"internally_reserved":0,"allocated":1785856,"data_stored":1182336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.23799999999999999},"max":{"1min":0.76300000000000001,"5min":0.80100000000000005,"15min":0.88300000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.27800000000000002},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.29899999999999999},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.433},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73499999999999999,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.85299999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":1.026},"last":0.33300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.81899999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60299999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.34300000000000003},"max":{"1min":0.77100000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.311,"5min":0.27100000000000002,"15min":0.254},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.34999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77600000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.50900000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.052},"last":0.38},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.48999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":593558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":1.016},"last":0.52800000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.85299999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.88800000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.85699999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":0.80300000000000005,"15min":0.875},"last":0.61599999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.78400000000000003,"15min":0.82199999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.91800000000000004},"last":0.504}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.872,"15min":0.872},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.36699999999999999,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66800000000000004,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36599999999999999}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.747,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":0.82199999999999995,"15min":0.87},"last":0.44500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":1.016},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82199999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.498},{"interface":"front","average":{"1min":0.625,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.81000000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38300000000000001,"15min":0.28599999999999998},"max":{"1min":0.69999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":1.0620000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.378},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.314,"15min":0.28499999999999998},"max":{"1min":0.71199999999999997,"5min":0.80100000000000005,"15min":1.262},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.496,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":1.0109999999999999,"15min":1.028},"last":0.61399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68400000000000005,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77800000000000002,"5min":0.96099999999999997,"15min":1.224},"last":0.59599999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65600000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.438,"5min":0.307,"15min":0.307},"max":{"1min":0.85299999999999998,"5min":0.871,"15min":1.159},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":593662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.877,"15min":0.877},"last":0.623}]},{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69199999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.85899999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.29899999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38200000000000001,"5min":0.36199999999999999,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.35799999999999998},"max":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.502,"15min":0.501},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.248},"max":{"1min":0.72099999999999997,"5min":0.746,"15min":0.79600000000000004},"last":0.32700000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.502},"min":{"1min":0.36699999999999999,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.75600000000000001,"5min":1.446,"15min":1.446},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32500000000000001,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.82199999999999995,"15min":0.90400000000000003},"last":0.38600000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.82699999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.44500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":599148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.73099999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.25600000000000001,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.54700000000000004,"5min":0.59299999999999997,"15min":0.68999999999999995},"last":0.315}]},{"osd":1,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57399999999999995,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.57299999999999995,"5min":0.67000000000000004,"15min":0.84799999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.29299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89500000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.625},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.94699999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.441,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.85999999999999999,"5min":1.002,"15min":1.002},"last":0.436},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.37},"max":{"1min":0.91500000000000004,"5min":1.018,"15min":1.018},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.0349999999999999},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:40:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1212416,"data_stored":599308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.224,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.192},"max":{"1min":0.51900000000000002,"5min":0.59899999999999998,"15min":0.63100000000000001},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":0.54600000000000004,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41399999999999998,"15min":0.40500000000000003},"min":{"1min":0.30399999999999999,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.60199999999999998,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.24199999999999999},"max":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.61599999999999999},"last":0.502},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.626,"5min":0.64800000000000002,"15min":0.92600000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.82399999999999995,"5min":0.88200000000000001,"15min":1.458},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.83799999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.746}]},{"osd":5,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.89000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.93700000000000006,"15min":1.028},"last":0.68400000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:40:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.625,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.879,"15min":0.92500000000000004},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575043,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":592058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.39800000000000002,"15min":0.38900000000000001},"min":{"1min":0.247,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.311}]},{"osd":2,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.318,"5min":0.24299999999999999,"15min":0.186},"max":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.70799999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.64800000000000002,"15min":0.747},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.29599999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.55700000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65200000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.48399999999999999,"5min":0.40699999999999997,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.94299999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46100000000000002,"15min":0.371},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.59099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1781760,"data_stored":1182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.376},"min":{"1min":0.249,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.25},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.76900000000000002},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.187},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.93700000000000006},"last":0.27700000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39200000000000002},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.58999999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.37},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.308,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.315}]},{"osd":4,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88,"5min":0.92000000000000004,"15min":0.95699999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.61899999999999999},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.34699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0920000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.339,"15min":1.339},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.44700000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:41:02.055 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:41:02.055 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:41:02.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.197+0000 7f440d705640 1 -- 172.21.15.125:0/3604322794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44080ac870 msgr2=0x7f440806b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.197+0000 7f440d705640 1 --2- 172.21.15.125:0/3604322794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44080ac870 0x7f440806b980 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f43fc005fd0 tx=0x7f43fc02be20 comp rx=0 tx=0).stop 2023-10-18T10:41:02.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.197+0000 7f440d705640 1 -- 172.21.15.125:0/3604322794 shutdown_connections 2023-10-18T10:41:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.197+0000 7f440d705640 1 --2- 172.21.15.125:0/3604322794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44080ac870 0x7f440806b980 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.197+0000 7f440d705640 1 -- 172.21.15.125:0/3604322794 >> 172.21.15.125:0/3604322794 conn(0x7f440806fae0 msgr2=0x7f440806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.197+0000 7f440d705640 1 -- 172.21.15.125:0/3604322794 shutdown_connections 2023-10-18T10:41:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.197+0000 7f440d705640 1 -- 172.21.15.125:0/3604322794 wait complete. 2023-10-18T10:41:02.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.198+0000 7f440d705640 1 Processor -- start 2023-10-18T10:41:02.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.198+0000 7f440d705640 1 -- start start 2023-10-18T10:41:02.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f440d705640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440806bec0 0x7f44080ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:02.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f440d705640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44080ac870 0x7f4408144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:02.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f440d705640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4408156790 0x7f440815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:02.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f440d705640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4408152470 con 0x7f440806bec0 2023-10-18T10:41:02.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f440d705640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44080adf80 con 0x7f44080ac870 2023-10-18T10:41:02.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f440d705640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44080ad3e0 con 0x7f4408156790 2023-10-18T10:41:02.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440806bec0 0x7f44080ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:02.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440806bec0 0x7f44080ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43406/0 (socket says 172.21.15.125:43406) 2023-10-18T10:41:02.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 -- 172.21.15.125:0/4269215768 learned_addr learned my addr 172.21.15.125:0/4269215768 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:02.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 -- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4408156790 msgr2=0x7f440815bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f44067fc640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44080ac870 0x7f4408144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:02.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4408156790 0x7f440815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 -- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44080ac870 msgr2=0x7f4408144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44080ac870 0x7f4408144790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 -- 172.21.15.125:0/4269215768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc009560 con 0x7f440806bec0 2023-10-18T10:41:02.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.199+0000 7f4406ffd640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440806bec0 0x7f44080ae410 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f43f800c7f0 tx=0x7f43f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:02.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.200+0000 7f43ef7fe640 1 -- 172.21.15.125:0/4269215768 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f800aa50 con 0x7f440806bec0 2023-10-18T10:41:02.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.200+0000 7f43ef7fe640 1 -- 172.21.15.125:0/4269215768 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f43f800aca0 con 0x7f440806bec0 2023-10-18T10:41:02.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.200+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44080ad650 con 0x7f440806bec0 2023-10-18T10:41:02.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.200+0000 7f43ef7fe640 1 -- 172.21.15.125:0/4269215768 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8012680 con 0x7f440806bec0 2023-10-18T10:41:02.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.200+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44080adc10 con 0x7f440806bec0 2023-10-18T10:41:02.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.201+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44080690e0 con 0x7f440806bec0 2023-10-18T10:41:02.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.201+0000 7f43ef7fe640 1 -- 172.21.15.125:0/4269215768 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f43f801a3b0 con 0x7f440806bec0 2023-10-18T10:41:02.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.201+0000 7f43ef7fe640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f43f406af60 0x7f43f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:02.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.202+0000 7f43ef7fe640 1 -- 172.21.15.125:0/4269215768 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f43f8089e50 con 0x7f440806bec0 2023-10-18T10:41:02.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.202+0000 7f44067fc640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f43f406af60 0x7f43f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:02.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.202+0000 7f44067fc640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f43f406af60 0x7f43f406d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f43fc035000 tx=0x7f43fc004730 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:02.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.206+0000 7f43ef7fe640 1 -- 172.21.15.125:0/4269215768 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43f8054c20 con 0x7f440806bec0 2023-10-18T10:41:02.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.440+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f44080cb190 con 0x7f440806bec0 2023-10-18T10:41:02.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.442+0000 7f43ef7fe640 1 -- 172.21.15.125:0/4269215768 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7f43f8016020 con 0x7f440806bec0 2023-10-18T10:41:02.444 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:41:02.444 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:41:02.444 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T10:41:02.444 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T10:41:02.445 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:41:02.445 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T10:41:02.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f43f406af60 msgr2=0x7f43f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f43f406af60 0x7f43f406d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f43fc035000 tx=0x7f43fc004730 comp rx=0 tx=0).stop 2023-10-18T10:41:02.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440806bec0 msgr2=0x7f44080ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440806bec0 0x7f44080ae410 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f43f800c7f0 tx=0x7f43f800ccb0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 shutdown_connections 2023-10-18T10:41:02.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f43f406af60 0x7f43f406d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44080ac870 0x7f4408144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4408156790 0x7f440815bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 --2- 172.21.15.125:0/4269215768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f440806bec0 0x7f44080ae410 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 >> 172.21.15.125:0/4269215768 conn(0x7f440806fae0 msgr2=0x7f4408154d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:02.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 shutdown_connections 2023-10-18T10:41:02.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.444+0000 7f440d705640 1 -- 172.21.15.125:0/4269215768 wait complete. 2023-10-18T10:41:02.457 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:41:02.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.557+0000 7fc9baa32640 1 -- 172.21.15.125:0/2869489962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b40abc10 msgr2=0x7fc9b40c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.557+0000 7fc9baa32640 1 --2- 172.21.15.125:0/2869489962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b40abc10 0x7fc9b40c9b30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009ca0 tx=0x7fc9ac02c2e0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.557+0000 7fc9baa32640 1 -- 172.21.15.125:0/2869489962 shutdown_connections 2023-10-18T10:41:02.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.557+0000 7fc9baa32640 1 --2- 172.21.15.125:0/2869489962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b40abc10 0x7fc9b40c9b30 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.557+0000 7fc9baa32640 1 --2- 172.21.15.125:0/2869489962 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9b406af00 0x7fc9b406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.557+0000 7fc9baa32640 1 -- 172.21.15.125:0/2869489962 >> 172.21.15.125:0/2869489962 conn(0x7fc9b4069930 msgr2=0x7fc9b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:02.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.557+0000 7fc9baa32640 1 -- 172.21.15.125:0/2869489962 shutdown_connections 2023-10-18T10:41:02.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.557+0000 7fc9baa32640 1 -- 172.21.15.125:0/2869489962 wait complete. 2023-10-18T10:41:02.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.558+0000 7fc9baa32640 1 Processor -- start 2023-10-18T10:41:02.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.558+0000 7fc9baa32640 1 -- start start 2023-10-18T10:41:02.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9baa32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b406af00 0x7fc9b41f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:02.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9baa32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9b40abc10 0x7fc9b4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:02.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9baa32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9b40c96b0 0x7fc9b415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:02.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9baa32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b4153ec0 con 0x7fc9b406af00 2023-10-18T10:41:02.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9baa32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b40af6a0 con 0x7fc9b40c96b0 2023-10-18T10:41:02.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9baa32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b41325d0 con 0x7fc9b40abc10 2023-10-18T10:41:02.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b406af00 0x7fc9b41f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:02.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b406af00 0x7fc9b41f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43412/0 (socket says 172.21.15.125:43412) 2023-10-18T10:41:02.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 -- 172.21.15.125:0/4258860358 learned_addr learned my addr 172.21.15.125:0/4258860358 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:02.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b8fcf640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9b40c96b0 0x7fc9b415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:02.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b37fe640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9b40abc10 0x7fc9b4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:02.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 -- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9b40abc10 msgr2=0x7fc9b4151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9b40abc10 0x7fc9b4151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 -- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9b40c96b0 msgr2=0x7fc9b415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9b40c96b0 0x7fc9b415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 -- 172.21.15.125:0/4258860358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac009900 con 0x7fc9b406af00 2023-10-18T10:41:02.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b8fcf640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9b40c96b0 0x7fc9b415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:02.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b3fff640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b406af00 0x7fc9b41f57b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400d7a0 tx=0x7fc9a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:02.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b0ff9640 1 -- 172.21.15.125:0/4258860358 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4009c30 con 0x7fc9b406af00 2023-10-18T10:41:02.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b0ff9640 1 -- 172.21.15.125:0/4258860358 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc9a4009e80 con 0x7fc9b406af00 2023-10-18T10:41:02.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b41f5cf0 con 0x7fc9b406af00 2023-10-18T10:41:02.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9b0ff9640 1 -- 172.21.15.125:0/4258860358 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4005280 con 0x7fc9b406af00 2023-10-18T10:41:02.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.559+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b4131d70 con 0x7fc9b406af00 2023-10-18T10:41:02.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.560+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b40c8e50 con 0x7fc9b406af00 2023-10-18T10:41:02.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.561+0000 7fc9b0ff9640 1 -- 172.21.15.125:0/4258860358 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc9a40053e0 con 0x7fc9b406af00 2023-10-18T10:41:02.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.561+0000 7fc9b0ff9640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc9a006af90 0x7fc9a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:02.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.561+0000 7fc9b0ff9640 1 -- 172.21.15.125:0/4258860358 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc9a4089be0 con 0x7fc9b406af00 2023-10-18T10:41:02.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.561+0000 7fc9b37fe640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc9a006af90 0x7fc9a006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:02.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.562+0000 7fc9b37fe640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc9a006af90 0x7fc9a006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40af050 tx=0x7fc9a8009040 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:02.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.565+0000 7fc9b0ff9640 1 -- 172.21.15.125:0/4258860358 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9a4054930 con 0x7fc9b406af00 2023-10-18T10:41:02.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.765+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9b40690e0 con 0x7fc9b406af00 2023-10-18T10:41:02.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.766+0000 7fc9b0ff9640 1 -- 172.21.15.125:0/4258860358 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fc9a4017020 con 0x7fc9b406af00 2023-10-18T10:41:02.768 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:02.768 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3531064821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3531064821},{"type":"v1","addr":"172.21.15.139:6837","nonce":3531064821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2850027191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2850027191},{"type":"v1","addr":"172.21.15.139:6836","nonce":2850027191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7578,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3670537575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3670537575},{"type":"v1","addr":"172.21.15.125:6834","nonce":3670537575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:53.749412+0000","modified":"2023-10-18T10:40:55.750936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7581":{"gid":7581,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1041197980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1041197980},{"type":"v1","addr":"172.21.15.125:6835","nonce":1041197980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T10:41:02.768 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 35 2023-10-18T10:41:02.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc9a006af90 msgr2=0x7fc9a006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc9a006af90 0x7fc9a006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40af050 tx=0x7fc9a8009040 comp rx=0 tx=0).stop 2023-10-18T10:41:02.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b406af00 msgr2=0x7fc9b41f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:02.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b406af00 0x7fc9b41f57b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400d7a0 tx=0x7fc9a400dc60 comp rx=0 tx=0).stop 2023-10-18T10:41:02.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 shutdown_connections 2023-10-18T10:41:02.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fc9a006af90 0x7fc9a006d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9b40c96b0 0x7fc9b415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9b40abc10 0x7fc9b4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 --2- 172.21.15.125:0/4258860358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9b406af00 0x7fc9b41f57b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:02.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.767+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 >> 172.21.15.125:0/4258860358 conn(0x7fc9b4069930 msgr2=0x7fc9b4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:02.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.768+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 shutdown_connections 2023-10-18T10:41:02.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:02.768+0000 7fc9baa32640 1 -- 172.21.15.125:0/4258860358 wait complete. 2023-10-18T10:41:03.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:40:53.749412+0000', 'modified': '2023-10-18T10:40:55.750936+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7581': {'gid': 7581, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/1041197980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 1041197980}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 1041197980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:41:03.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:41:03.184 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T10:41:03.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.293+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4199841534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7840047c0 msgr2=0x7fe784004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.293+0000 7fe7a03c0640 1 --2- 172.21.15.125:0/4199841534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7840047c0 0x7fe784004c40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe788001680 tx=0x7fe7840057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.293+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4199841534 shutdown_connections 2023-10-18T10:41:03.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.293+0000 7fe7a03c0640 1 --2- 172.21.15.125:0/4199841534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7840047c0 0x7fe784004c40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.293+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4199841534 >> 172.21.15.125:0/4199841534 conn(0x7fe7980d0d60 msgr2=0x7fe7980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:03.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.293+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4199841534 shutdown_connections 2023-10-18T10:41:03.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.293+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4199841534 wait complete. 2023-10-18T10:41:03.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.294+0000 7fe7a03c0640 1 Processor -- start 2023-10-18T10:41:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.294+0000 7fe7a03c0640 1 -- start start 2023-10-18T10:41:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe7a03c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7840047c0 0x7fe7980c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe7a03c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980cabc0 0x7fe798145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe7a03c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7981455a0 0x7fe79806ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe7a03c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980b5550 con 0x7fe7980cabc0 2023-10-18T10:41:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe7a03c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980c7e30 con 0x7fe7981455a0 2023-10-18T10:41:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe7a03c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980c8d80 con 0x7fe7840047c0 2023-10-18T10:41:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980cabc0 0x7fe798145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980cabc0 0x7fe798145060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43432/0 (socket says 172.21.15.125:43432) 2023-10-18T10:41:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 -- 172.21.15.125:0/4098894119 learned_addr learned my addr 172.21.15.125:0/4098894119 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:03.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79e15c640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7840047c0 0x7fe7980c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79e95d640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7981455a0 0x7fe79806ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 -- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7840047c0 msgr2=0x7fe7980c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7840047c0 0x7fe7980c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 -- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7981455a0 msgr2=0x7fe79806ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7981455a0 0x7fe79806ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 -- 172.21.15.125:0/4098894119 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798145b20 con 0x7fe7980cabc0 2023-10-18T10:41:03.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79e15c640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7840047c0 0x7fe7980c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:03.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79e95d640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7981455a0 0x7fe79806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:03.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe79d95b640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980cabc0 0x7fe798145060 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe78802c0d0 tx=0x7fe7880035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:03.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe78effd640 1 -- 172.21.15.125:0/4098894119 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788003bf0 con 0x7fe7980cabc0 2023-10-18T10:41:03.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe78effd640 1 -- 172.21.15.125:0/4098894119 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe788003e40 con 0x7fe7980cabc0 2023-10-18T10:41:03.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4098894119 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe798145e00 con 0x7fe7980cabc0 2023-10-18T10:41:03.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe78effd640 1 -- 172.21.15.125:0/4098894119 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78802c790 con 0x7fe7980cabc0 2023-10-18T10:41:03.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.295+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4098894119 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7980c8320 con 0x7fe7980cabc0 2023-10-18T10:41:03.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.297+0000 7fe78effd640 1 -- 172.21.15.125:0/4098894119 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe78802c8f0 con 0x7fe7980cabc0 2023-10-18T10:41:03.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.297+0000 7fe78effd640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe77c06af90 0x7fe77c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.298+0000 7fe78effd640 1 -- 172.21.15.125:0/4098894119 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe7880aa810 con 0x7fe7980cabc0 2023-10-18T10:41:03.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.298+0000 7fe79e15c640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe77c06af90 0x7fe77c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.298+0000 7fe79e15c640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe77c06af90 0x7fe77c06d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe7940013d0 tx=0x7fe794001f30 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:03.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.298+0000 7fe7a302b740 1 Processor -- start 2023-10-18T10:41:03.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe7a302b740 1 -- start start 2023-10-18T10:41:03.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9755aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9755b020 0x55ac9758ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe7a302b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9746c730 con 0x55ac97587b40 2023-10-18T10:41:03.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe7a302b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac974cd780 con 0x55ac97587f20 2023-10-18T10:41:03.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe7a302b740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac974cae70 con 0x55ac9755b020 2023-10-18T10:41:03.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe79e15c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9755aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe79e15c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9755aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43436/0 (socket says 172.21.15.125:43436) 2023-10-18T10:41:03.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe79e15c640 1 -- 172.21.15.125:0/2838141436 learned_addr learned my addr 172.21.15.125:0/2838141436 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.299+0000 7fe79d95b640 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe79e95d640 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9755b020 0x55ac9758ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe79e15c640 1 -- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9755b020 msgr2=0x55ac9758ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9755b020 0x55ac9758ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe79e15c640 1 -- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 msgr2=0x55ac9758a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe79e15c640 1 -- 172.21.15.125:0/2838141436 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac974cb3d0 con 0x55ac97587b40 2023-10-18T10:41:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe79d95b640 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9755aae0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe794006820 tx=0x7fe794014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4268f761e8efb166 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe77bfff640 1 -- 172.21.15.125:0/2838141436 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794012bc0 con 0x55ac97587b40 2023-10-18T10:41:03.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe77bfff640 1 -- 172.21.15.125:0/2838141436 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe794016070 con 0x55ac97587b40 2023-10-18T10:41:03.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.300+0000 7fe77bfff640 1 -- 172.21.15.125:0/2838141436 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940017d0 con 0x55ac97587b40 2023-10-18T10:41:03.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 -- 172.21.15.125:0/2838141436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 msgr2=0x55ac9755aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9755aae0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe794006820 tx=0x7fe794014000 comp rx=0 tx=0).stop 2023-10-18T10:41:03.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 -- 172.21.15.125:0/2838141436 shutdown_connections 2023-10-18T10:41:03.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9755b020 0x55ac9758ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2838141436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9755aae0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 -- 172.21.15.125:0/2838141436 >> 172.21.15.125:0/2838141436 conn(0x55ac9755cd50 msgr2=0x55ac975a58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:03.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 -- 172.21.15.125:0/2838141436 shutdown_connections 2023-10-18T10:41:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 -- 172.21.15.125:0/2838141436 wait complete. 2023-10-18T10:41:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 -- start start 2023-10-18T10:41:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.301+0000 7fe7a302b740 1 Processor -- start 2023-10-18T10:41:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac975b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9759ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9759e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe7a302b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac974c6760 con 0x55ac97587b40 2023-10-18T10:41:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe7a302b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac974818f0 con 0x55ac97587f20 2023-10-18T10:41:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe7a302b740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac97481590 con 0x55ac9757b8b0 2023-10-18T10:41:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe7a302b740 10 client.0 _ll_register_callbacks cb 0x55ac974bdee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:41:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe79e15c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac975b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe79e15c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac975b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43444/0 (socket says 172.21.15.125:43444) 2023-10-18T10:41:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.302+0000 7fe79e15c640 1 -- 172.21.15.125:0/2494158357 learned_addr learned my addr 172.21.15.125:0/2494158357 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79d95b640 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9759ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79e95d640 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9759e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79e15c640 1 -- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 msgr2=0x55ac9759e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9759e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79e15c640 1 -- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 msgr2=0x55ac9759ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9759ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79e15c640 1 -- 172.21.15.125:0/2494158357 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7980ce490 con 0x55ac97587b40 2023-10-18T10:41:03.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79e95d640 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9759e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:03.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac975b5860 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe794007010 tx=0x7fe794012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18b8541d7886073 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:03.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7797fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:41:03.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7797fa640 1 -- 172.21.15.125:0/2494158357 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79402d070 con 0x55ac97587b40 2023-10-18T10:41:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7797fa640 1 -- 172.21.15.125:0/2494158357 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe794034c60 con 0x55ac97587b40 2023-10-18T10:41:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 1 -- 172.21.15.125:0/2494158357 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac974bf740 con 0x55ac97587b40 2023-10-18T10:41:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7797fa640 1 -- 172.21.15.125:0/2494158357 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe794034e20 con 0x55ac97587b40 2023-10-18T10:41:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 10 client.7743 fetch_fsmap learned FSMap version 35 2023-10-18T10:41:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 1 -- 172.21.15.125:0/2494158357 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55ac974c4de0 con 0x55ac97587b40 2023-10-18T10:41:03.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7797fa640 1 -- 172.21.15.125:0/2494158357 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55ac974bf740 con 0x55ac97587b40 2023-10-18T10:41:03.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7797fa640 1 -- 172.21.15.125:0/2494158357 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe79403d420 con 0x55ac97587b40 2023-10-18T10:41:03.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 10 client.7743 fetch_fsmap finished waiting for FSMap version 35 2023-10-18T10:41:03.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 10 client.7743 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T10:41:03.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 20 client.7743 populate_metadata read hostname 'smithi125' 2023-10-18T10:41:03.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] conn(0x55ac975c0fd0 0x55ac975c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 4 client.7743 mds_command: new command op to 7581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:41:03.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.303+0000 7fe7a302b740 1 -- 172.21.15.125:0/2494158357 --> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac974bf460 con 0x55ac975c0fd0 2023-10-18T10:41:03.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.304+0000 7fe79d95b640 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] conn(0x55ac975c0fd0 0x55ac975c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.304+0000 7fe7797fa640 10 client.7743 ms_handle_connect on v2:172.21.15.125:6833/1041197980 2023-10-18T10:41:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.304+0000 7fe79d95b640 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] conn(0x55ac975c0fd0 0x55ac975c33c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03aca1cc66ea977 server_cookie=9642f6cc33a8624a in_seq=0 out_seq=0 2023-10-18T10:41:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.304+0000 7fe7797fa640 1 -- 172.21.15.125:0/2494158357 <== mds.0 v2:172.21.15.125:6833/1041197980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac974bf460 con 0x55ac975c0fd0 2023-10-18T10:41:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.304+0000 7fe7797fa640 10 client.7743 handle_command_reply: tid=0 2023-10-18T10:41:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.304+0000 7fe7a302b740 1 client.7743 shutdown 2023-10-18T10:41:03.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 -- 172.21.15.125:0/2494158357 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 msgr2=0x55ac975b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac975b5860 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe794007010 tx=0x7fe794012e70 comp rx=0 tx=0).stop 2023-10-18T10:41:03.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 -- 172.21.15.125:0/2494158357 shutdown_connections 2023-10-18T10:41:03.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] conn(0x55ac975c0fd0 0x55ac975c33c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9759ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9759e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7797fa640 0 client.7743 ms_handle_reset on v2:172.21.15.125:6833/1041197980 2023-10-18T10:41:03.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2494158357 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac975b5860 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 -- 172.21.15.125:0/2494158357 >> 172.21.15.125:0/2494158357 conn(0x55ac9755b020 msgr2=0x55ac9758a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:03.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 -- 172.21.15.125:0/2494158357 shutdown_connections 2023-10-18T10:41:03.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 1 -- 172.21.15.125:0/2494158357 wait complete. 2023-10-18T10:41:03.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.305+0000 7fe7a302b740 20 client.7743 trim_cache size 0 max 16384 2023-10-18T10:41:03.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.320+0000 7fe7a302b740 1 Processor -- start 2023-10-18T10:41:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.320+0000 7fe7a302b740 1 -- start start 2023-10-18T10:41:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac9757b8b0 0x55ac9759d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587b40 0x55ac9755e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac97587f20 0x55ac97579bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe7a302b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac97447270 con 0x55ac9757b8b0 2023-10-18T10:41:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe7a302b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9759ef50 con 0x55ac97587b40 2023-10-18T10:41:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe7a302b740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac9758ec30 con 0x55ac97587f20 2023-10-18T10:41:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac9757b8b0 0x55ac9759d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac9757b8b0 0x55ac9759d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43450/0 (socket says 172.21.15.125:43450) 2023-10-18T10:41:03.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 -- 172.21.15.125:0/1363139224 learned_addr learned my addr 172.21.15.125:0/1363139224 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:03.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79d95b640 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587b40 0x55ac9755e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e95d640 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac97587f20 0x55ac97579bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 -- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac97587f20 msgr2=0x55ac97579bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:03.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac97587f20 0x55ac97579bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 -- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587b40 msgr2=0x55ac9755e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:03.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587b40 0x55ac9755e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 -- 172.21.15.125:0/1363139224 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac97463d00 con 0x55ac9757b8b0 2023-10-18T10:41:03.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79d95b640 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587b40 0x55ac9755e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:03.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.321+0000 7fe79e15c640 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac9757b8b0 0x55ac9759d270 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe794012d50 tx=0x7fe79400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929dee15b27e34a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:03.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe78ce00640 1 -- 172.21.15.125:0/1363139224 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79403d9b0 con 0x55ac9757b8b0 2023-10-18T10:41:03.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe78ce00640 1 -- 172.21.15.125:0/1363139224 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe79403dc60 con 0x55ac9757b8b0 2023-10-18T10:41:03.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe78ce00640 1 -- 172.21.15.125:0/1363139224 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794027620 con 0x55ac9757b8b0 2023-10-18T10:41:03.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 -- 172.21.15.125:0/1363139224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac9757b8b0 msgr2=0x55ac9759d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac9757b8b0 0x55ac9759d270 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe794012d50 tx=0x7fe79400c000 comp rx=0 tx=0).stop 2023-10-18T10:41:03.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 -- 172.21.15.125:0/1363139224 shutdown_connections 2023-10-18T10:41:03.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587b40 0x55ac9755e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac97587f20 0x55ac97579bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 --2- 172.21.15.125:0/1363139224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac9757b8b0 0x55ac9759d270 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 -- 172.21.15.125:0/1363139224 >> 172.21.15.125:0/1363139224 conn(0x55ac9755b020 msgr2=0x55ac975a8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:03.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 -- 172.21.15.125:0/1363139224 shutdown_connections 2023-10-18T10:41:03.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.322+0000 7fe7a302b740 1 -- 172.21.15.125:0/1363139224 wait complete. 2023-10-18T10:41:03.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.323+0000 7fe7a302b740 1 -- start start 2023-10-18T10:41:03.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.323+0000 7fe7a302b740 1 Processor -- start 2023-10-18T10:41:03.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9758ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9760d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7a302b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7a302b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac975c7bf0 con 0x55ac97587b40 2023-10-18T10:41:03.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7a302b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac975bf9e0 con 0x55ac97587f20 2023-10-18T10:41:03.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7a302b740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac9758b260 con 0x55ac9757b8b0 2023-10-18T10:41:03.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7a302b740 10 client.0 _ll_register_callbacks cb 0x7fe790009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:41:03.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9760d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9760d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43458/0 (socket says 172.21.15.125:43458) 2023-10-18T10:41:03.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 -- 172.21.15.125:0/2176637111 learned_addr learned my addr 172.21.15.125:0/2176637111 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:03.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79e95d640 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9758ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 -- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 msgr2=0x55ac9758ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:03.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9758ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 -- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 msgr2=0x55ac9758bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:03.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 -- 172.21.15.125:0/2176637111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac974c4de0 con 0x55ac97587b40 2023-10-18T10:41:03.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9758ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:03.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe79d95b640 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9760d320 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe78802c1a0 tx=0x7fe788079ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e072e16baa92e3f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:03.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7797fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:41:03.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7797fa640 1 -- 172.21.15.125:0/2176637111 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7880a6070 con 0x55ac97587b40 2023-10-18T10:41:03.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7797fa640 1 -- 172.21.15.125:0/2176637111 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe788081590 con 0x55ac97587b40 2023-10-18T10:41:03.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.324+0000 7fe7a302b740 1 -- 172.21.15.125:0/2176637111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac97586c50 con 0x55ac97587b40 2023-10-18T10:41:03.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7797fa640 1 -- 172.21.15.125:0/2176637111 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe788077630 con 0x55ac97587b40 2023-10-18T10:41:03.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7a302b740 10 client.7749 fetch_fsmap learned FSMap version 35 2023-10-18T10:41:03.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7a302b740 1 -- 172.21.15.125:0/2176637111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55ac974894f0 con 0x55ac97587b40 2023-10-18T10:41:03.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7797fa640 1 -- 172.21.15.125:0/2176637111 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55ac97586c50 con 0x55ac97587b40 2023-10-18T10:41:03.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7797fa640 1 -- 172.21.15.125:0/2176637111 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe78803c070 con 0x55ac97587b40 2023-10-18T10:41:03.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7a302b740 10 client.7749 fetch_fsmap finished waiting for FSMap version 35 2023-10-18T10:41:03.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7a302b740 10 client.7749 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T10:41:03.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7a302b740 20 client.7749 populate_metadata read hostname 'smithi125' 2023-10-18T10:41:03.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] conn(0x55ac975a6b20 0x55ac975a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7a302b740 4 client.7749 mds_command: new command op to 7581 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:41:03.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7a302b740 1 -- 172.21.15.125:0/2176637111 --> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fe794016070 con 0x55ac975a6b20 2023-10-18T10:41:03.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] conn(0x55ac975a6b20 0x55ac975a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe7797fa640 10 client.7749 ms_handle_connect on v2:172.21.15.125:6833/1041197980 2023-10-18T10:41:03.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.325+0000 7fe79e15c640 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] conn(0x55ac975a6b20 0x55ac975a6f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295f9177dd11226d server_cookie=3cd0ce10edcf0598 in_seq=0 out_seq=0 2023-10-18T10:41:03.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7797fa640 1 -- 172.21.15.125:0/2176637111 <== mds.0 v2:172.21.15.125:6833/1041197980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7fe794016070 con 0x55ac975a6b20 2023-10-18T10:41:03.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7797fa640 10 client.7749 handle_command_reply: tid=0 2023-10-18T10:41:03.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 client.7749 shutdown 2023-10-18T10:41:03.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 -- 172.21.15.125:0/2176637111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 msgr2=0x55ac9760d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9760d320 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe78802c1a0 tx=0x7fe788079ac0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 -- 172.21.15.125:0/2176637111 shutdown_connections 2023-10-18T10:41:03.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.125:6833/1041197980,v1:172.21.15.125:6835/1041197980] conn(0x55ac975a6b20 0x55ac975a6f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ac97587f20 0x55ac9758bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7797fa640 0 client.7749 ms_handle_reset on v2:172.21.15.125:6833/1041197980 2023-10-18T10:41:03.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ac97587b40 0x55ac9760d320 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 --2- 172.21.15.125:0/2176637111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ac9757b8b0 0x55ac9758ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 -- 172.21.15.125:0/2176637111 >> 172.21.15.125:0/2176637111 conn(0x55ac9755b020 msgr2=0x55ac975b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:03.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 -- 172.21.15.125:0/2176637111 shutdown_connections 2023-10-18T10:41:03.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 1 -- 172.21.15.125:0/2176637111 wait complete. 2023-10-18T10:41:03.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a302b740 20 client.7749 trim_cache size 0 max 16384 2023-10-18T10:41:03.352 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:41:03.352 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:41:03.352 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:41:03.352 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 7581, 2023-10-18T10:41:03.353 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:41:03.353 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:41:03.353 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:41:03.353 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.9058827689999998, 2023-10-18T10:41:03.353 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 35, 2023-10-18T10:41:03.354 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 98, 2023-10-18T10:41:03.354 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 98, 2023-10-18T10:41:03.354 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 14.68733162 2023-10-18T10:41:03.354 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:41:03.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe77c06af90 msgr2=0x7fe77c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe77c06af90 0x7fe77c06d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe7940013d0 tx=0x7fe794001f30 comp rx=0 tx=0).stop 2023-10-18T10:41:03.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4098894119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980cabc0 msgr2=0x7fe798145060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980cabc0 0x7fe798145060 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe78802c0d0 tx=0x7fe7880035f0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4098894119 shutdown_connections 2023-10-18T10:41:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fe77c06af90 0x7fe77c06d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7981455a0 0x7fe79806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980cabc0 0x7fe798145060 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 --2- 172.21.15.125:0/4098894119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7840047c0 0x7fe7980c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.326+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4098894119 >> 172.21.15.125:0/4098894119 conn(0x7fe7980d0d60 msgr2=0x7fe7980ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:03.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.327+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4098894119 shutdown_connections 2023-10-18T10:41:03.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.327+0000 7fe7a03c0640 1 -- 172.21.15.125:0/4098894119 wait complete. 2023-10-18T10:41:03.358 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:41:03.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.461+0000 7ff5ec139640 1 -- 172.21.15.125:0/2218277352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 msgr2=0x7ff5e4158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.461+0000 7ff5ec139640 1 --2- 172.21.15.125:0/2218277352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 0x7ff5e4158300 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40b10f0 tx=0x7ff5d402d160 comp rx=0 tx=0).stop 2023-10-18T10:41:03.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.461+0000 7ff5ec139640 1 -- 172.21.15.125:0/2218277352 shutdown_connections 2023-10-18T10:41:03.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.461+0000 7ff5ec139640 1 --2- 172.21.15.125:0/2218277352 >> v2:172.21.15.139:3300/0 conn(0x7ff5e40b0530 0x7ff5e40b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.461+0000 7ff5ec139640 1 --2- 172.21.15.125:0/2218277352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 0x7ff5e4158300 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.461+0000 7ff5ec139640 1 -- 172.21.15.125:0/2218277352 >> 172.21.15.125:0/2218277352 conn(0x7ff5e4069930 msgr2=0x7ff5e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:03.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.461+0000 7ff5ec139640 1 -- 172.21.15.125:0/2218277352 shutdown_connections 2023-10-18T10:41:03.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.461+0000 7ff5ec139640 1 -- 172.21.15.125:0/2218277352 wait complete. 2023-10-18T10:41:03.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5ec139640 1 Processor -- start 2023-10-18T10:41:03.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5ec139640 1 -- start start 2023-10-18T10:41:03.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5ec139640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 0x7ff5e41529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5ec139640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40b0530 0x7ff5e4155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5ec139640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e40cbd20 0x7ff5e413b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5ec139640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4156e70 con 0x7ff5e40afa20 2023-10-18T10:41:03.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5ec139640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e41ef580 con 0x7ff5e40b0530 2023-10-18T10:41:03.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5ec139640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4152f00 con 0x7ff5e40cbd20 2023-10-18T10:41:03.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5e9ed5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 0x7ff5e41529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5e9ed5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 0x7ff5e41529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43478/0 (socket says 172.21.15.125:43478) 2023-10-18T10:41:03.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.463+0000 7ff5e9ed5640 1 -- 172.21.15.125:0/3819476750 learned_addr learned my addr 172.21.15.125:0/3819476750 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:03.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5e96d4640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40b0530 0x7ff5e4155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5ea6d6640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e40cbd20 0x7ff5e413b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5e9ed5640 1 -- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e40cbd20 msgr2=0x7ff5e413b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5e9ed5640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e40cbd20 0x7ff5e413b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5e9ed5640 1 -- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40b0530 msgr2=0x7ff5e4155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5e9ed5640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40b0530 0x7ff5e4155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5e9ed5640 1 -- 172.21.15.125:0/3819476750 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d400a670 con 0x7ff5e40afa20 2023-10-18T10:41:03.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5e96d4640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40b0530 0x7ff5e4155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:03.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5e9ed5640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 0x7ff5e41529c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c400 tx=0x7ff5e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:03.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5da7fc640 1 -- 172.21.15.125:0/3819476750 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00052d0 con 0x7ff5e40afa20 2023-10-18T10:41:03.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5da7fc640 1 -- 172.21.15.125:0/3819476750 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5e00035f0 con 0x7ff5e40afa20 2023-10-18T10:41:03.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e4156370 con 0x7ff5e40afa20 2023-10-18T10:41:03.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5da7fc640 1 -- 172.21.15.125:0/3819476750 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0003a00 con 0x7ff5e40afa20 2023-10-18T10:41:03.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.464+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e4156b90 con 0x7ff5e40afa20 2023-10-18T10:41:03.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.465+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e40690e0 con 0x7ff5e40afa20 2023-10-18T10:41:03.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.465+0000 7ff5da7fc640 1 -- 172.21.15.125:0/3819476750 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff5e0003b60 con 0x7ff5e40afa20 2023-10-18T10:41:03.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.466+0000 7ff5da7fc640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff5c806af60 0x7ff5c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:03.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.466+0000 7ff5da7fc640 1 -- 172.21.15.125:0/3819476750 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff5e0089040 con 0x7ff5e40afa20 2023-10-18T10:41:03.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.466+0000 7ff5e96d4640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff5c806af60 0x7ff5c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:03.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.467+0000 7ff5e96d4640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff5c806af60 0x7ff5c806d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4036000 tx=0x7ff5d4004730 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:03.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.470+0000 7ff5da7fc640 1 -- 172.21.15.125:0/3819476750 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5e0053f30 con 0x7ff5e40afa20 2023-10-18T10:41:03.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.670+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5e406f700 con 0x7ff5e40afa20 2023-10-18T10:41:03.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.671+0000 7ff5da7fc640 1 -- 172.21.15.125:0/3819476750 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff5e0017060 con 0x7ff5e40afa20 2023-10-18T10:41:03.673 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:41:03.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff5c806af60 msgr2=0x7ff5c806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff5c806af60 0x7ff5c806d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4036000 tx=0x7ff5d4004730 comp rx=0 tx=0).stop 2023-10-18T10:41:03.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 msgr2=0x7ff5e41529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:03.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 0x7ff5e41529c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c400 tx=0x7ff5e000c8c0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 shutdown_connections 2023-10-18T10:41:03.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7ff5c806af60 0x7ff5c806d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40b0530 0x7ff5e4155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e40cbd20 0x7ff5e413b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 --2- 172.21.15.125:0/3819476750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e40afa20 0x7ff5e41529c0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:03.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 >> 172.21.15.125:0/3819476750 conn(0x7ff5e4069930 msgr2=0x7ff5e4157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:03.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 shutdown_connections 2023-10-18T10:41:03.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:03.673+0000 7ff5ec139640 1 -- 172.21.15.125:0/3819476750 wait complete. 2023-10-18T10:41:03.685 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:41:03.685 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:41:03.686 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:41:03.686 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:41:03.686 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:03.687 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:41:03.687 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:41:03.687 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:41:03.688 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:41:03.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:03.689 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:41:03.704 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:41:03.704 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:41:03.704 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:41:03.704 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:03.705 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:41:03.705 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:03.705 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:41:03.705 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:03.705 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:41:03.705 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:41:03.706 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:03.706 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:41:03.706 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:41:03.706 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:03.706 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:41:03.707 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:03.707 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:41:03.707 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 41760sec preferred_lft 41760sec 2023-10-18T10:41:03.707 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:41:03.707 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:03.708 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:41:03.708 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:03.708 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:41:03.708 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:03.708 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T10:41:03.709 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:03.709 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:41:03.709 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:03.709 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T10:41:03.710 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:03.710 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:41:03.710 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:03.710 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T10:41:03.710 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:03.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:03.711 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:41:03.770 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:41:03.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:03.772 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:41:03.828 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:41:03.829 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:41:03.829 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:41:03.829 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:41:03.829 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:41:03.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:03.830 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:41:03.831 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:41:03.831 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:41:03.832 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:41:03.832 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:41:03.832 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:41:03.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:03.924 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:41:03.938 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:41:03.939 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:41:03.939 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:41:03.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:03.940 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:41:04.005 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:41:04.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:41:04.006 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:04.060 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:41:04.061 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:41:04.061 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:04.115 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:41:04.181 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:41:04.237 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:41:04.237 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:41:04.238 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:41:04.238 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:41:04.238 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:41:04.238 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:41:04.238 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:41:04.239 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:41:04.239 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:41:04.239 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:41:04.239 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:41:04.240 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:41:04.240 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:41:04.240 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:41:04.240 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:41:04.240 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:41:04.241 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:41:04.241 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:41:04.241 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:41:04.241 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:41:04.241 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:41:04.241 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:41:04.242 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:41:04.242 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:41:04.242 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:41:04.242 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:41:04.242 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:41:04.243 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:41:04.243 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:41:04.243 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:41:04.243 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:04.243 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:04.244 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:04.244 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:04.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:04.244 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:41:04.292 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:41:04.292 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:41:04.334 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:41:04.361 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:41:04.416 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:41:04.417 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:41:04.417 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:41:04.417 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:41:04.417 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:41:04.418 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:41:04.418 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:41:04.418 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:41:04.419 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:41:04.419 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:41:04.419 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:41:04.419 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:41:04.419 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:41:04.420 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:41:04.420 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:41:04.420 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:41:04.420 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:41:04.420 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:41:04.421 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:41:04.421 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:41:04.421 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:41:04.421 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:41:04.421 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:41:04.422 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:41:04.422 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:41:04.422 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:41:04.422 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:41:04.422 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:41:04.423 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:41:04.423 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:41:04.423 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:04.423 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:04.423 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:04.424 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:04.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:04.424 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:41:04.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:41:04.418+0000 7f95a340d480 -1 init, newargv = 0x56461b4c7d70 newargc=15 2023-10-18T10:41:04.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[122557]: starting ceph client 2023-10-18T10:41:04.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[122557]: starting fuse 2023-10-18T10:41:04.440 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:41:04.440 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:41:04.441 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:41:04.441 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:41:04.442 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:41:04.442 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:41:04.443 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:41:04.443 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:41:04.443 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:41:04.444 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:41:04.444 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:41:04.444 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:41:04.445 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:41:04.445 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:41:04.446 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:41:04.446 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:41:04.446 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:41:04.447 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:41:04.447 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:41:04.447 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:41:04.448 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:41:04.448 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:41:04.449 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:41:04.449 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:41:04.449 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:41:04.450 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:41:04.450 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:41:04.451 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:41:04.451 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:41:04.559 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.122557.asok 2023-10-18T10:41:04.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:41:04.569 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.122557.asok 2023-10-18T10:41:04.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:04.571 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.122557.asok status 2023-10-18T10:41:04.703 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:41:04.703 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:41:04.704 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:41:04.704 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:41:04.704 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:41:04.704 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:41:04.705 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:41:04.705 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "122557", 2023-10-18T10:41:04.705 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:41:04.705 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:04.705 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:41:04.706 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:41:04.706 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 7764, 2023-10-18T10:41:04.706 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:41:04.707 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:41:04.707 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:41:04.707 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 7764 2023-10-18T10:41:04.707 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:04.708 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:41:04.708 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:41:04.708 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:41:04.708 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 71389258 2023-10-18T10:41:04.708 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:04.709 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:04.709 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:41:04.709 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:41:04.709 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:41:04.709 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 71389258 2023-10-18T10:41:04.710 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:04.710 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.7764 192.168.0.1:0/71389258", 2023-10-18T10:41:04.710 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/71389258", 2023-10-18T10:41:04.710 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:41:04.711 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 35, 2023-10-18T10:41:04.711 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 98, 2023-10-18T10:41:04.711 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:41:04.711 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:41:04.711 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:41:04.712 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:41:04.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:04.713 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:41:04.714 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:04.729 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:41:04.729 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:04.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:04.730 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:04.796 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T10:41:04.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.937+0000 7f8124632640 1 -- 172.21.15.125:0/447581714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c06af00 msgr2=0x7f811c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:04.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.937+0000 7f8124632640 1 --2- 172.21.15.125:0/447581714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c06af00 0x7f811c06b2e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8114009ca0 tx=0x7f811402c230 comp rx=0 tx=0).stop 2023-10-18T10:41:04.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.937+0000 7f8124632640 1 -- 172.21.15.125:0/447581714 shutdown_connections 2023-10-18T10:41:04.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.937+0000 7f8124632640 1 --2- 172.21.15.125:0/447581714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f811c0abc10 0x7f811c0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:04.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.937+0000 7f8124632640 1 --2- 172.21.15.125:0/447581714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c06af00 0x7f811c06b2e0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:04.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.937+0000 7f8124632640 1 -- 172.21.15.125:0/447581714 >> 172.21.15.125:0/447581714 conn(0x7f811c069930 msgr2=0x7f811c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:04.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.937+0000 7f8124632640 1 -- 172.21.15.125:0/447581714 shutdown_connections 2023-10-18T10:41:04.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.937+0000 7f8124632640 1 -- 172.21.15.125:0/447581714 wait complete. 2023-10-18T10:41:04.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.938+0000 7f8124632640 1 Processor -- start 2023-10-18T10:41:04.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.938+0000 7f8124632640 1 -- start start 2023-10-18T10:41:04.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8124632640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811c06af00 0x7f811c1f0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:04.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8124632640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f811c0abc10 0x7f811c158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:04.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8124632640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c0c96b0 0x7f811c13cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:04.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8124632640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1f0860 con 0x7f811c0c96b0 2023-10-18T10:41:04.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8124632640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c159030 con 0x7f811c06af00 2023-10-18T10:41:04.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8124632640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c153be0 con 0x7f811c0abc10 2023-10-18T10:41:04.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c0c96b0 0x7f811c13cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:04.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c0c96b0 0x7f811c13cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43508/0 (socket says 172.21.15.125:43508) 2023-10-18T10:41:04.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 -- 172.21.15.125:0/3038962777 learned_addr learned my addr 172.21.15.125:0/3038962777 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:04.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8121bcd640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f811c0abc10 0x7f811c158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:04.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f81223ce640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811c06af00 0x7f811c1f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:04.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 -- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f811c0abc10 msgr2=0x7f811c158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:04.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f811c0abc10 0x7f811c158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:04.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 -- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811c06af00 msgr2=0x7f811c1f0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:04.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811c06af00 0x7f811c1f0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:04.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 -- 172.21.15.125:0/3038962777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114009900 con 0x7f811c0c96b0 2023-10-18T10:41:04.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8122bcf640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c0c96b0 0x7f811c13cef0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f811800d780 tx=0x7f811800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:04.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.939+0000 7f8112ffd640 1 -- 172.21.15.125:0/3038962777 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118009bd0 con 0x7f811c0c96b0 2023-10-18T10:41:04.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.940+0000 7f8112ffd640 1 -- 172.21.15.125:0/3038962777 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8118009e20 con 0x7f811c0c96b0 2023-10-18T10:41:04.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.940+0000 7f8112ffd640 1 -- 172.21.15.125:0/3038962777 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118005280 con 0x7f811c0c96b0 2023-10-18T10:41:04.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.940+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c12e340 con 0x7f811c0c96b0 2023-10-18T10:41:04.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.940+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c12e900 con 0x7f811c0c96b0 2023-10-18T10:41:04.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.941+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c0690e0 con 0x7f811c0c96b0 2023-10-18T10:41:04.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.941+0000 7f8112ffd640 1 -- 172.21.15.125:0/3038962777 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f81180053e0 con 0x7f811c0c96b0 2023-10-18T10:41:04.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.941+0000 7f8112ffd640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f810006af60 0x7f810006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:04.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.942+0000 7f8112ffd640 1 -- 172.21.15.125:0/3038962777 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8118089b10 con 0x7f811c0c96b0 2023-10-18T10:41:04.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.942+0000 7f81223ce640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f810006af60 0x7f810006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:04.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.942+0000 7f81223ce640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f810006af60 0x7f810006d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8114036000 tx=0x7f8114004ab0 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:04.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:04.946+0000 7f8112ffd640 1 -- 172.21.15.125:0/3038962777 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81180548f0 con 0x7f811c0c96b0 2023-10-18T10:41:05.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.131+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 --> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f811c159c50 con 0x7f810006af60 2023-10-18T10:41:05.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.133+0000 7f8112ffd640 1 -- 172.21.15.125:0/3038962777 <== mgr.5972 v2:172.21.15.139:6832/143413153 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8114003b90 con 0x7f810006af60 2023-10-18T10:41:05.135 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:41:05.135 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:05.136 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T10:41:05.136 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:05.136 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:41:05.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f810006af60 msgr2=0x7f810006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:05.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f810006af60 0x7f810006d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8114036000 tx=0x7f8114004ab0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c0c96b0 msgr2=0x7f811c13cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:05.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c0c96b0 0x7f811c13cef0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f811800d780 tx=0x7f811800dc40 comp rx=0 tx=0).stop 2023-10-18T10:41:05.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 shutdown_connections 2023-10-18T10:41:05.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f810006af60 0x7f810006d420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f811c0c96b0 0x7f811c13cef0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f811c0abc10 0x7f811c158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 --2- 172.21.15.125:0/3038962777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f811c06af00 0x7f811c1f0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 >> 172.21.15.125:0/3038962777 conn(0x7f811c069930 msgr2=0x7f811c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:05.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 shutdown_connections 2023-10-18T10:41:05.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.135+0000 7f8124632640 1 -- 172.21.15.125:0/3038962777 wait complete. 2023-10-18T10:41:05.147 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T10:41:05.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.250+0000 7f47bd79e640 1 -- 172.21.15.125:0/513258506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47b8158910 msgr2=0x7f47b8158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:05.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.250+0000 7f47bd79e640 1 --2- 172.21.15.125:0/513258506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47b8158910 0x7f47b8158cf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f47a8005fd0 tx=0x7f47a802c0e0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.250+0000 7f47bd79e640 1 -- 172.21.15.125:0/513258506 shutdown_connections 2023-10-18T10:41:05.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.250+0000 7f47bd79e640 1 --2- 172.21.15.125:0/513258506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b80cd9e0 0x7f47b80cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.250+0000 7f47bd79e640 1 --2- 172.21.15.125:0/513258506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47b8158910 0x7f47b8158cf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.250+0000 7f47bd79e640 1 -- 172.21.15.125:0/513258506 >> 172.21.15.125:0/513258506 conn(0x7f47b80af7a0 msgr2=0x7f47b80afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:05.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.250+0000 7f47bd79e640 1 -- 172.21.15.125:0/513258506 shutdown_connections 2023-10-18T10:41:05.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.250+0000 7f47bd79e640 1 -- 172.21.15.125:0/513258506 wait complete. 2023-10-18T10:41:05.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47bd79e640 1 Processor -- start 2023-10-18T10:41:05.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47bd79e640 1 -- start start 2023-10-18T10:41:05.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47bd79e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b80cd1c0 0x7f47b81567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:05.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47bd79e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b80cd9e0 0x7f47b814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:05.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47bd79e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47b8158910 0x7f47b814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:05.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47bd79e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8154810 con 0x7f47b80cd9e0 2023-10-18T10:41:05.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47bd79e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b814da80 con 0x7f47b8158910 2023-10-18T10:41:05.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47bd79e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b814dd60 con 0x7f47b80cd1c0 2023-10-18T10:41:05.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47b67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b80cd9e0 0x7f47b814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:05.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.252+0000 7f47b67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b80cd9e0 0x7f47b814cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43524/0 (socket says 172.21.15.125:43524) 2023-10-18T10:41:05.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b67fc640 1 -- 172.21.15.125:0/562295975 learned_addr learned my addr 172.21.15.125:0/562295975 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:05.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b77fe640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47b8158910 0x7f47b814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:05.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b6ffd640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b80cd1c0 0x7f47b81567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:05.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b67fc640 1 -- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b80cd1c0 msgr2=0x7f47b81567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:05.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b67fc640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b80cd1c0 0x7f47b81567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b67fc640 1 -- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47b8158910 msgr2=0x7f47b814d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:05.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b67fc640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47b8158910 0x7f47b814d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:05.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b67fc640 1 -- 172.21.15.125:0/562295975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8009840 con 0x7f47b80cd9e0 2023-10-18T10:41:05.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47b67fc640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b80cd9e0 0x7f47b814cea0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c750 tx=0x7f47ac00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:05.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f479f7fe640 1 -- 172.21.15.125:0/562295975 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac005270 con 0x7f47b80cd9e0 2023-10-18T10:41:05.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f479f7fe640 1 -- 172.21.15.125:0/562295975 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47ac00ce90 con 0x7f47b80cd9e0 2023-10-18T10:41:05.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b814e040 con 0x7f47b80cd9e0 2023-10-18T10:41:05.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f479f7fe640 1 -- 172.21.15.125:0/562295975 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00a8a0 con 0x7f47b80cd9e0 2023-10-18T10:41:05.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.253+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b814e600 con 0x7f47b80cd9e0 2023-10-18T10:41:05.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.254+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b814e820 con 0x7f47b80cd9e0 2023-10-18T10:41:05.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.255+0000 7f479f7fe640 1 -- 172.21.15.125:0/562295975 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 41) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f47ac00aa00 con 0x7f47b80cd9e0 2023-10-18T10:41:05.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.255+0000 7f479f7fe640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f47a406af60 0x7f47a406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:05.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.255+0000 7f479f7fe640 1 -- 172.21.15.125:0/562295975 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f47ac088d80 con 0x7f47b80cd9e0 2023-10-18T10:41:05.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.255+0000 7f47b6ffd640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f47a406af60 0x7f47a406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:05.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.256+0000 7f47b6ffd640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f47a406af60 0x7f47a406d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47a8035000 tx=0x7f47a8004ab0 comp rx=0 tx=0).ready entity=mgr.5972 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:05.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.259+0000 7f479f7fe640 1 -- 172.21.15.125:0/562295975 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47ac053d20 con 0x7f47b80cd9e0 2023-10-18T10:41:05.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:05.472+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f47b815c7f0 con 0x7f47b80cd9e0 2023-10-18T10:41:05.669 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:41:05.672 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:41:06.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.463+0000 7f479f7fe640 1 -- 172.21.15.125:0/562295975 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v42) v1 ==== 92+0+0 (secure 0 0 0) 0x7f47b815c7f0 con 0x7f47b80cd9e0 2023-10-18T10:41:06.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f47a406af60 msgr2=0x7f47a406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f47a406af60 0x7f47a406d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47a8035000 tx=0x7f47a8004ab0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b80cd9e0 msgr2=0x7f47b814cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b80cd9e0 0x7f47b814cea0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c750 tx=0x7f47ac00cb50 comp rx=0 tx=0).stop 2023-10-18T10:41:06.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 shutdown_connections 2023-10-18T10:41:06.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f47a406af60 0x7f47a406d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47b8158910 0x7f47b814d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47b80cd9e0 0x7f47b814cea0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 --2- 172.21.15.125:0/562295975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47b80cd1c0 0x7f47b81567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 >> 172.21.15.125:0/562295975 conn(0x7f47b80af7a0 msgr2=0x7f47b815ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:06.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 shutdown_connections 2023-10-18T10:41:06.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.466+0000 7f47bd79e640 1 -- 172.21.15.125:0/562295975 wait complete. 2023-10-18T10:41:06.481 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T10:41:06.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.590+0000 7f8bd894c640 1 -- 172.21.15.125:0/1561143774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 msgr2=0x7f8bd40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.590+0000 7f8bd894c640 1 --2- 172.21.15.125:0/1561143774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 0x7f8bd40cf720 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8bd408a810 tx=0x7f8bbc02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.590+0000 7f8bd894c640 1 -- 172.21.15.125:0/1561143774 shutdown_connections 2023-10-18T10:41:06.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.590+0000 7f8bd894c640 1 --2- 172.21.15.125:0/1561143774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 0x7f8bd40cf720 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.590+0000 7f8bd894c640 1 --2- 172.21.15.125:0/1561143774 >> v2:172.21.15.139:3300/0 conn(0x7f8bd40cf340 0x7f8bd40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.590+0000 7f8bd894c640 1 -- 172.21.15.125:0/1561143774 >> 172.21.15.125:0/1561143774 conn(0x7f8bd40d0d60 msgr2=0x7f8bd40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:06.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.590+0000 7f8bd894c640 1 -- 172.21.15.125:0/1561143774 shutdown_connections 2023-10-18T10:41:06.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.590+0000 7f8bd894c640 1 -- 172.21.15.125:0/1561143774 wait complete. 2023-10-18T10:41:06.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.592+0000 7f8bd894c640 1 Processor -- start 2023-10-18T10:41:06.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.592+0000 7f8bd894c640 1 -- start start 2023-10-18T10:41:06.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd894c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bd40ceb40 0x7f8bd409e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:06.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd894c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bd40cf340 0x7f8bd40aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:06.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd894c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 0x7f8bd4157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:06.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd894c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40af4b0 con 0x7f8bd415b2a0 2023-10-18T10:41:06.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd894c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd409e920 con 0x7f8bd40ceb40 2023-10-18T10:41:06.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd894c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40c8fb0 con 0x7f8bd40cf340 2023-10-18T10:41:06.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 0x7f8bd4157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:06.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 0x7f8bd4157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43542/0 (socket says 172.21.15.125:43542) 2023-10-18T10:41:06.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 -- 172.21.15.125:0/25684569 learned_addr learned my addr 172.21.15.125:0/25684569 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:06.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd259c640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bd40ceb40 0x7f8bd409e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:06.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bc9d9b640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bd40cf340 0x7f8bd40aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:06.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 -- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bd40cf340 msgr2=0x7f8bd40aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bd40cf340 0x7f8bd40aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 -- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bd40ceb40 msgr2=0x7f8bd409e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bd40ceb40 0x7f8bd409e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 -- 172.21.15.125:0/25684569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc009490 con 0x7f8bd415b2a0 2023-10-18T10:41:06.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bd2d9d640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 0x7f8bd4157eb0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00cbc0 tx=0x7f8bcc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:06.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/25684569 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00ad40 con 0x7f8bd415b2a0 2023-10-18T10:41:06.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.593+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/25684569 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bcc012420 con 0x7f8bd415b2a0 2023-10-18T10:41:06.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.594+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd40c9290 con 0x7f8bd415b2a0 2023-10-18T10:41:06.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.594+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/25684569 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc012830 con 0x7f8bd415b2a0 2023-10-18T10:41:06.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.594+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd40c9850 con 0x7f8bd415b2a0 2023-10-18T10:41:06.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.594+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd40d04f0 con 0x7f8bd415b2a0 2023-10-18T10:41:06.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.595+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/25684569 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 42) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f8bcc01a3b0 con 0x7f8bd415b2a0 2023-10-18T10:41:06.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.596+0000 7f8bcb7fe640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8bb406afb0 0x7f8bb406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:06.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.596+0000 7f8bd259c640 1 -- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8bb406afb0 msgr2=0x7f8bb406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/143413153 2023-10-18T10:41:06.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.596+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/25684569 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8bcc089f70 con 0x7f8bd415b2a0 2023-10-18T10:41:06.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.596+0000 7f8bd259c640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8bb406afb0 0x7f8bb406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:06.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.600+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/25684569 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bcc054e70 con 0x7f8bd415b2a0 2023-10-18T10:41:06.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.775+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f8bd40cb980 con 0x7f8bd415b2a0 2023-10-18T10:41:06.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.778+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/25684569 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v44)=0 v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8bcc058490 con 0x7f8bd415b2a0 2023-10-18T10:41:06.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8bb406afb0 msgr2=0x7f8bb406d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:06.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8bb406afb0 0x7f8bb406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 msgr2=0x7f8bd4157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 0x7f8bd4157eb0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00cbc0 tx=0x7f8bcc00a430 comp rx=0 tx=0).stop 2023-10-18T10:41:06.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 shutdown_connections 2023-10-18T10:41:06.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f8bb406afb0 0x7f8bb406d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bd415b2a0 0x7f8bd4157eb0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bd40cf340 0x7f8bd40aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 --2- 172.21.15.125:0/25684569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bd40ceb40 0x7f8bd409e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 >> 172.21.15.125:0/25684569 conn(0x7f8bd40d0d60 msgr2=0x7f8bd40ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:06.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 shutdown_connections 2023-10-18T10:41:06.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.780+0000 7f8bd894c640 1 -- 172.21.15.125:0/25684569 wait complete. 2023-10-18T10:41:06.793 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T10:41:06.849 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:06.848+0000 7fca0e012040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:41:06.860 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:06.859+0000 7fbd79432040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:41:06.894 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:06.892+0000 7fca0e012040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:41:06.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 -- 172.21.15.125:0/3139923180 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8159060 msgr2=0x7fcfd806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 --2- 172.21.15.125:0/3139923180 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8159060 0x7fcfd806d260 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80ceab0 tx=0x7fcfcc02bd10 comp rx=0 tx=0).stop 2023-10-18T10:41:06.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 -- 172.21.15.125:0/3139923180 shutdown_connections 2023-10-18T10:41:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 --2- 172.21.15.125:0/3139923180 >> v2:172.21.15.139:3300/0 conn(0x7fcfd815ccf0 0x7fcfd815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 --2- 172.21.15.125:0/3139923180 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfd80cdb30 0x7fcfd80cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 --2- 172.21.15.125:0/3139923180 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd8159060 0x7fcfd806d260 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 -- 172.21.15.125:0/3139923180 >> 172.21.15.125:0/3139923180 conn(0x7fcfd80b0670 msgr2=0x7fcfd80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 -- 172.21.15.125:0/3139923180 shutdown_connections 2023-10-18T10:41:06.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.898+0000 7fcfde705640 1 -- 172.21.15.125:0/3139923180 wait complete. 2023-10-18T10:41:06.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.899+0000 7fcfde705640 1 Processor -- start 2023-10-18T10:41:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.899+0000 7fcfde705640 1 -- start start 2023-10-18T10:41:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfde705640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd80cdb30 0x7fcfd80b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfde705640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfd8159060 0x7fcfd81e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfde705640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfd815ccf0 0x7fcfd80c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfde705640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80b3b20 con 0x7fcfd80cdb30 2023-10-18T10:41:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfde705640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81e3fe0 con 0x7fcfd815ccf0 2023-10-18T10:41:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfde705640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80b2ea0 con 0x7fcfd8159060 2023-10-18T10:41:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd80cdb30 0x7fcfd80b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd80cdb30 0x7fcfd80b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43560/0 (socket says 172.21.15.125:43560) 2023-10-18T10:41:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfd7fff640 1 -- 172.21.15.125:0/3918593618 learned_addr learned my addr 172.21.15.125:0/3918593618 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfd77fe640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfd8159060 0x7fcfd81e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.900+0000 7fcfdcca2640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfd815ccf0 0x7fcfd80c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd7fff640 1 -- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfd8159060 msgr2=0x7fcfd81e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd7fff640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfd8159060 0x7fcfd81e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd7fff640 1 -- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfd815ccf0 msgr2=0x7fcfd80c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd7fff640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfd815ccf0 0x7fcfd80c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd7fff640 1 -- 172.21.15.125:0/3918593618 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc009560 con 0x7fcfd80cdb30 2023-10-18T10:41:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd7fff640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd80cdb30 0x7fcfd80b2890 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc000c00 tx=0x7fcfcc02d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd4ff9640 1 -- 172.21.15.125:0/3918593618 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc02dd70 con 0x7fcfd80cdb30 2023-10-18T10:41:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd4ff9640 1 -- 172.21.15.125:0/3918593618 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcfcc0035f0 con 0x7fcfd80cdb30 2023-10-18T10:41:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfd4ff9640 1 -- 172.21.15.125:0/3918593618 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc003a00 con 0x7fcfd80cdb30 2023-10-18T10:41:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd80b3180 con 0x7fcfd80cdb30 2023-10-18T10:41:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.901+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd80b3740 con 0x7fcfd80cdb30 2023-10-18T10:41:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.902+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd80675f0 con 0x7fcfd80cdb30 2023-10-18T10:41:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.902+0000 7fcfd4ff9640 1 -- 172.21.15.125:0/3918593618 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 42) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fcfcc003b60 con 0x7fcfd80cdb30 2023-10-18T10:41:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.903+0000 7fcfd4ff9640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fcfc406afe0 0x7fcfc406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.903+0000 7fcfd77fe640 1 -- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fcfc406afe0 msgr2=0x7fcfc406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/143413153 2023-10-18T10:41:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.903+0000 7fcfd4ff9640 1 -- 172.21.15.125:0/3918593618 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcfcc0abfe0 con 0x7fcfd80cdb30 2023-10-18T10:41:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.903+0000 7fcfd77fe640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fcfc406afe0 0x7fcfc406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:06.912 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:06.907+0000 7fbd79432040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:41:06.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:06.907+0000 7fcfd4ff9640 1 -- 172.21.15.125:0/3918593618 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfcc076fb0 con 0x7fcfd80cdb30 2023-10-18T10:41:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.080+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fcfd80b4aa0 con 0x7fcfd80cdb30 2023-10-18T10:41:07.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.085+0000 7fcfd4ff9640 1 -- 172.21.15.125:0/3918593618 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v45)=0 v45) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcfcc042020 con 0x7fcfd80cdb30 2023-10-18T10:41:07.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.087+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fcfc406afe0 msgr2=0x7fcfc406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:07.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.087+0000 7fcfde705640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fcfc406afe0 0x7fcfc406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.087+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd80cdb30 msgr2=0x7fcfd80b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:07.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.087+0000 7fcfde705640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd80cdb30 0x7fcfd80b2890 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc000c00 tx=0x7fcfcc02d970 comp rx=0 tx=0).stop 2023-10-18T10:41:07.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.088+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 shutdown_connections 2023-10-18T10:41:07.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.088+0000 7fcfde705640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7fcfc406afe0 0x7fcfc406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.088+0000 7fcfde705640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfd815ccf0 0x7fcfd80c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.088+0000 7fcfde705640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfd8159060 0x7fcfd81e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.088+0000 7fcfde705640 1 --2- 172.21.15.125:0/3918593618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcfd80cdb30 0x7fcfd80b2890 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.088+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 >> 172.21.15.125:0/3918593618 conn(0x7fcfd80b0670 msgr2=0x7fcfd80ac5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:07.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.088+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 shutdown_connections 2023-10-18T10:41:07.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.088+0000 7fcfde705640 1 -- 172.21.15.125:0/3918593618 wait complete. 2023-10-18T10:41:07.100 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-18T10:41:07.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.204+0000 7f0570e90640 1 -- 172.21.15.125:0/3639927870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f056c0cb960 msgr2=0x7f056c06da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:07.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.204+0000 7f0570e90640 1 --2- 172.21.15.125:0/3639927870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f056c0cb960 0x7f056c06da80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0564005fd0 tx=0x7f056402c2e0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.204+0000 7f0570e90640 1 -- 172.21.15.125:0/3639927870 shutdown_connections 2023-10-18T10:41:07.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.204+0000 7f0570e90640 1 --2- 172.21.15.125:0/3639927870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f056c0cb960 0x7f056c06da80 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.204+0000 7f0570e90640 1 --2- 172.21.15.125:0/3639927870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f056c0708c0 0x7f056c158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.204+0000 7f0570e90640 1 -- 172.21.15.125:0/3639927870 >> 172.21.15.125:0/3639927870 conn(0x7f056c0af810 msgr2=0x7f056c0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:07.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.204+0000 7f0570e90640 1 -- 172.21.15.125:0/3639927870 shutdown_connections 2023-10-18T10:41:07.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.204+0000 7f0570e90640 1 -- 172.21.15.125:0/3639927870 wait complete. 2023-10-18T10:41:07.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.205+0000 7f0570e90640 1 Processor -- start 2023-10-18T10:41:07.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.205+0000 7f0570e90640 1 -- start start 2023-10-18T10:41:07.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f0570e90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f056c0708c0 0x7f056c13fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:07.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f0570e90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f056c0cb960 0x7f056c13b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:07.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f0570e90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f056c141460 0x7f056c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:07.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f0570e90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0c97d0 con 0x7f056c141460 2023-10-18T10:41:07.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f0570e90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0c8940 con 0x7f056c0cb960 2023-10-18T10:41:07.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f0570e90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0c8b50 con 0x7f056c0708c0 2023-10-18T10:41:07.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f056c141460 0x7f056c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:07.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f056c141460 0x7f056c13b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43576/0 (socket says 172.21.15.125:43576) 2023-10-18T10:41:07.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056ad9d640 1 -- 172.21.15.125:0/1527322705 learned_addr learned my addr 172.21.15.125:0/1527322705 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:07.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056a59c640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f056c0708c0 0x7f056c13fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:07.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f0563fff640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f056c0cb960 0x7f056c13b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:07.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056a59c640 1 -- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f056c0cb960 msgr2=0x7f056c13b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:07.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056a59c640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f056c0cb960 0x7f056c13b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056a59c640 1 -- 172.21.15.125:0/1527322705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f056c141460 msgr2=0x7f056c13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:07.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056a59c640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f056c141460 0x7f056c13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:07.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.206+0000 7f056a59c640 1 -- 172.21.15.125:0/1527322705 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564009900 con 0x7f056c0708c0 2023-10-18T10:41:07.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.207+0000 7f056a59c640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f056c0708c0 0x7f056c13fba0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c390 tx=0x7f055c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:07.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.207+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0052d0 con 0x7f056c0708c0 2023-10-18T10:41:07.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.207+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f055c0035f0 con 0x7f056c0708c0 2023-10-18T10:41:07.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.207+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c003a00 con 0x7f056c0708c0 2023-10-18T10:41:07.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.207+0000 7f0570e90640 1 -- 172.21.15.125:0/1527322705 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c0c8e30 con 0x7f056c0708c0 2023-10-18T10:41:07.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.207+0000 7f0570e90640 1 -- 172.21.15.125:0/1527322705 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c0c93f0 con 0x7f056c0708c0 2023-10-18T10:41:07.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.208+0000 7f0570e90640 1 -- 172.21.15.125:0/1527322705 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c0675f0 con 0x7f056c0708c0 2023-10-18T10:41:07.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.208+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 42) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f055c003b60 con 0x7f056c0708c0 2023-10-18T10:41:07.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.209+0000 7f0562ffd640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 0x7f054c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:07.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.209+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f055c088f50 con 0x7f056c0708c0 2023-10-18T10:41:07.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.209+0000 7f0563fff640 1 -- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 msgr2=0x7f054c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/143413153 2023-10-18T10:41:07.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.209+0000 7f0563fff640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 0x7f054c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:07.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.213+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f055c053ef0 con 0x7f056c0708c0 2023-10-18T10:41:07.286 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.284+0000 7fca0e012040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:41:07.313 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.312+0000 7fbd79432040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:41:07.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.327+0000 7fca0e012040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:41:07.357 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.355+0000 7fbd79432040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:41:07.370 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.368+0000 7fca0e012040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:41:07.399 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.397+0000 7fbd79432040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:41:07.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.402+0000 7f0570e90640 1 -- 172.21.15.125:0/1527322705 --> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f056c15afc0 con 0x7f054c06afb0 2023-10-18T10:41:07.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.410+0000 7f0563fff640 1 -- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 msgr2=0x7f054c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/143413153 2023-10-18T10:41:07.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.410+0000 7f0563fff640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 0x7f054c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:41:07.421 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.420+0000 7fca0e012040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:41:07.450 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.449+0000 7fbd79432040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:41:07.462 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.461+0000 7fca0e012040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:41:07.493 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.491+0000 7fbd79432040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:41:07.560 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.558+0000 7fca0e012040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:41:07.590 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.589+0000 7fbd79432040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:41:07.612 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.610+0000 7fca0e012040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:07.643 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.641+0000 7fbd79432040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:07.653 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.652+0000 7fca0e012040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:41:07.684 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.682+0000 7fbd79432040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:41:07.701 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.699+0000 7fca0e012040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:41:07.731 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.729+0000 7fbd79432040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:41:07.743 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:07.742+0000 7fca0e012040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:41:07.773 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:07.772+0000 7fbd79432040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:41:07.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.811+0000 7f0563fff640 1 -- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 msgr2=0x7f054c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/143413153 2023-10-18T10:41:07.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:07.811+0000 7f0563fff640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 0x7f054c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:41:08.056 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.055+0000 7fca0e012040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:41:08.087 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.085+0000 7fbd79432040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:41:08.185 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.184+0000 7fca0e012040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:41:08.217 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.215+0000 7fbd79432040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:41:08.276 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.274+0000 7fca0e012040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:41:08.310 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.308+0000 7fbd79432040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:41:08.318 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.316+0000 7fca0e012040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:41:08.352 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.350+0000 7fbd79432040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:41:08.414 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.413+0000 7fca0e012040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:08.447 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.446+0000 7fbd79432040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:08.506 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.505+0000 7fca0e012040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:41:08.538 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.536+0000 7fbd79432040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:41:08.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:08.612+0000 7f0563fff640 1 -- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 msgr2=0x7f054c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/143413153 2023-10-18T10:41:08.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:08.612+0000 7f0563fff640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 0x7f054c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:41:08.642 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.641+0000 7fca0e012040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:08.673 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.671+0000 7fbd79432040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:08.685 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.683+0000 7fca0e012040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:08.715 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.713+0000 7fbd79432040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:08.789 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:08.788+0000 7fca0e012040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:41:08.819 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:08.817+0000 7fbd79432040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:41:09.401 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.400+0000 7fca0e012040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:41:09.431 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:09.430+0000 7fbd79432040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:41:09.662 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.661+0000 7fca0e012040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:41:09.691 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:09.690+0000 7fbd79432040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:41:09.710 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.708+0000 7fca0e012040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:41:09.739 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:09.738+0000 7fbd79432040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:41:09.811 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.809+0000 7fca0e012040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:41:09.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:09.831+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mgrmap(e 43) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f055c0542c0 con 0x7f056c0708c0 2023-10-18T10:41:09.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:09.831+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 msgr2=0x7f054c06d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:09.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:09.831+0000 7f0562ffd640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/143413153,v1:172.21.15.139:6833/143413153] conn(0x7f054c06afb0 0x7f054c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:09.843 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:09.841+0000 7fbd79432040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:41:09.935 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98a6a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.935 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98a6a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.935 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98a6a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.936 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98a6a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.936 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98a6a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.936 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98d6ef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.936 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98d6ef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.936 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98d6ef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.937 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98d6ef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:09.937 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:09.933+0000 7fc98d6ef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:10.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.832+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mon.2 v2:172.21.15.139:3301/0 8 ==== mgrmap(e 44) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f055c087300 con 0x7f056c0708c0 2023-10-18T10:41:10.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.833+0000 7f0562ffd640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f054c035850 0x7f054c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:10.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.833+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 --> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f056c15afc0 con 0x7f054c035850 2023-10-18T10:41:10.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.833+0000 7f0563fff640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f054c035850 0x7f054c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:10.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.835+0000 7f0563fff640 1 --2- 172.21.15.125:0/1527322705 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f054c035850 0x7f054c075380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f056c0c5e80 tx=0x7f0554005f90 comp rx=0 tx=0).ready entity=mgr.6823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:10.845 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:10.844+0000 7fc9946fd640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-18T10:41:10.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.844+0000 7f0562ffd640 1 -- 172.21.15.125:0/1527322705 <== mgr.6823 v2:172.21.15.139:6832/1652676135 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f056c15afc0 con 0x7f054c035850 2023-10-18T10:41:10.846 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-18T10:41:10.851 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T10:41:10.852 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T10:41:10.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 -- 172.21.15.125:0/3032140706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f857806ffb0 msgr2=0x7f85780c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:10.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 --2- 172.21.15.125:0/3032140706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f857806ffb0 0x7f85780c96b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f85780ca460 tx=0x7f856c02bda0 comp rx=0 tx=0).stop 2023-10-18T10:41:10.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 -- 172.21.15.125:0/3032140706 shutdown_connections 2023-10-18T10:41:10.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 --2- 172.21.15.125:0/3032140706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f857806ffb0 0x7f85780c96b0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:10.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 --2- 172.21.15.125:0/3032140706 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85780c9bf0 0x7f8578154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:10.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 --2- 172.21.15.125:0/3032140706 >> v2:172.21.15.139:3300/0 conn(0x7f85780b1920 0x7f857806fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:10.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 -- 172.21.15.125:0/3032140706 >> 172.21.15.125:0/3032140706 conn(0x7f8578069930 msgr2=0x7f8578069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:10.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 -- 172.21.15.125:0/3032140706 shutdown_connections 2023-10-18T10:41:10.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.956+0000 7f857c844640 1 -- 172.21.15.125:0/3032140706 wait complete. 2023-10-18T10:41:10.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.957+0000 7f857c844640 1 Processor -- start 2023-10-18T10:41:10.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.957+0000 7f857c844640 1 -- start start 2023-10-18T10:41:10.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f857c844640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f857806ffb0 0x7f857814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:10.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f857c844640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85780b1920 0x7f85781570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:10.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f857c844640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85780c9bf0 0x7f857806b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:10.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f857c844640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857814ab70 con 0x7f85780c9bf0 2023-10-18T10:41:10.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f857c844640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857815d3c0 con 0x7f857806ffb0 2023-10-18T10:41:10.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f857c844640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85781575e0 con 0x7f85780b1920 2023-10-18T10:41:10.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f8576d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85780c9bf0 0x7f857806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:10.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f8576d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85780c9bf0 0x7f857806b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33270/0 (socket says 172.21.15.125:33270) 2023-10-18T10:41:10.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f8576d9d640 1 -- 172.21.15.125:0/1736387130 learned_addr learned my addr 172.21.15.125:0/1736387130 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:10.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f857659c640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f857806ffb0 0x7f857814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:10.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.958+0000 7f8575d9b640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85780b1920 0x7f85781570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:10.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8576d9d640 1 -- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85780b1920 msgr2=0x7f85781570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:10.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8576d9d640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85780b1920 0x7f85781570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:10.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8576d9d640 1 -- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f857806ffb0 msgr2=0x7f857814de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:10.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8576d9d640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f857806ffb0 0x7f857814de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:10.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8576d9d640 1 -- 172.21.15.125:0/1736387130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c009560 con 0x7f85780c9bf0 2023-10-18T10:41:10.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8575d9b640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85780b1920 0x7f85781570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:41:10.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f857659c640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f857806ffb0 0x7f857814de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:41:10.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8576d9d640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85780c9bf0 0x7f857806b940 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f857000e740 tx=0x7f857000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:10.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8566ffd640 1 -- 172.21.15.125:0/1736387130 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85700095a0 con 0x7f85780c9bf0 2023-10-18T10:41:10.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8566ffd640 1 -- 172.21.15.125:0/1736387130 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f857000cce0 con 0x7f85780c9bf0 2023-10-18T10:41:10.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85781f55f0 con 0x7f85780c9bf0 2023-10-18T10:41:10.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f8566ffd640 1 -- 172.21.15.125:0/1736387130 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570005280 con 0x7f85780c9bf0 2023-10-18T10:41:10.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.959+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857815d940 con 0x7f85780c9bf0 2023-10-18T10:41:10.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.960+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85780710a0 con 0x7f85780c9bf0 2023-10-18T10:41:10.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.960+0000 7f8566ffd640 1 -- 172.21.15.125:0/1736387130 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 44) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f85700053e0 con 0x7f85780c9bf0 2023-10-18T10:41:10.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.961+0000 7f8566ffd640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f855806ace0 0x7f855806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:10.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.961+0000 7f8566ffd640 1 -- 172.21.15.125:0/1736387130 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f8570014070 con 0x7f85780c9bf0 2023-10-18T10:41:10.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.961+0000 7f857659c640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f855806ace0 0x7f855806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:10.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.961+0000 7f857659c640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f855806ace0 0x7f855806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8568005fd0 tx=0x7f8568009500 comp rx=0 tx=0).ready entity=mgr.6823 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:10.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:10.965+0000 7f8566ffd640 1 -- 172.21.15.125:0/1736387130 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8570058930 con 0x7f85780c9bf0 2023-10-18T10:41:11.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.177+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f85780690e0 con 0x7f85780c9bf0 2023-10-18T10:41:11.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.887+0000 7f8566ffd640 1 -- 172.21.15.125:0/1736387130 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v45) v1 ==== 93+0+0 (secure 0 0 0) 0x7f85700584c0 con 0x7f85780c9bf0 2023-10-18T10:41:11.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f855806ace0 msgr2=0x7f855806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:11.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f855806ace0 0x7f855806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8568005fd0 tx=0x7f8568009500 comp rx=0 tx=0).stop 2023-10-18T10:41:11.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85780c9bf0 msgr2=0x7f857806b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:11.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85780c9bf0 0x7f857806b940 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f857000e740 tx=0x7f857000ec00 comp rx=0 tx=0).stop 2023-10-18T10:41:11.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 shutdown_connections 2023-10-18T10:41:11.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f855806ace0 0x7f855806d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:11.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85780c9bf0 0x7f857806b940 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:11.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85780b1920 0x7f85781570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:11.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 --2- 172.21.15.125:0/1736387130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f857806ffb0 0x7f857814de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:11.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.890+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 >> 172.21.15.125:0/1736387130 conn(0x7f8578069930 msgr2=0x7f857815a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:11.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.891+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 shutdown_connections 2023-10-18T10:41:11.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:11.891+0000 7f857c844640 1 -- 172.21.15.125:0/1736387130 wait complete. 2023-10-18T10:41:11.905 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:41:11.905 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:41:11.942 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:41:11.986 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:41:11.987 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:41:11.987 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:41:12.004 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:12.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.147+0000 7f06d30c9640 1 -- 172.21.15.125:0/1754540316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06cc159060 msgr2=0x7f06cc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.147+0000 7f06d30c9640 1 --2- 172.21.15.125:0/1754540316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06cc159060 0x7f06cc06d260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009830 tx=0x7f06c002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.147+0000 7f06d30c9640 1 -- 172.21.15.125:0/1754540316 shutdown_connections 2023-10-18T10:41:12.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.147+0000 7f06d30c9640 1 --2- 172.21.15.125:0/1754540316 >> v2:172.21.15.139:3300/0 conn(0x7f06cc15d400 0x7f06cc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.147+0000 7f06d30c9640 1 --2- 172.21.15.125:0/1754540316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06cc159060 0x7f06cc06d260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.147+0000 7f06d30c9640 1 -- 172.21.15.125:0/1754540316 >> 172.21.15.125:0/1754540316 conn(0x7f06cc0b0670 msgr2=0x7f06cc0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:12.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.147+0000 7f06d30c9640 1 -- 172.21.15.125:0/1754540316 shutdown_connections 2023-10-18T10:41:12.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.147+0000 7f06d30c9640 1 -- 172.21.15.125:0/1754540316 wait complete. 2023-10-18T10:41:12.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.148+0000 7f06d30c9640 1 Processor -- start 2023-10-18T10:41:12.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.148+0000 7f06d30c9640 1 -- start start 2023-10-18T10:41:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d30c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06cc0cdb30 0x7f06cc1eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d30c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06cc159060 0x7f06cc0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d30c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06cc15d400 0x7f06cc0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d30c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0b14d0 con 0x7f06cc15d400 2023-10-18T10:41:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d30c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0b17b0 con 0x7f06cc0cdb30 2023-10-18T10:41:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d30c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc0b1a90 con 0x7f06cc159060 2023-10-18T10:41:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06cc15d400 0x7f06cc0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06cc15d400 0x7f06cc0b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33302/0 (socket says 172.21.15.125:33302) 2023-10-18T10:41:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d0e65640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06cc0cdb30 0x7f06cc1eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 -- 172.21.15.125:0/3770813864 learned_addr learned my addr 172.21.15.125:0/3770813864 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06cbfff640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06cc159060 0x7f06cc0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 -- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06cc159060 msgr2=0x7f06cc0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06cc159060 0x7f06cc0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 -- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06cc0cdb30 msgr2=0x7f06cc1eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06cc0cdb30 0x7f06cc1eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 -- 172.21.15.125:0/3770813864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c4009840 con 0x7f06cc15d400 2023-10-18T10:41:12.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d1666640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06cc15d400 0x7f06cc0b2570 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f06c400ec20 tx=0x7f06c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:12.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06c97fa640 1 -- 172.21.15.125:0/3770813864 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4009aa0 con 0x7f06cc15d400 2023-10-18T10:41:12.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06c97fa640 1 -- 172.21.15.125:0/3770813864 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f06c4003c60 con 0x7f06cc15d400 2023-10-18T10:41:12.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c0009490 con 0x7f06cc15d400 2023-10-18T10:41:12.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.149+0000 7f06c97fa640 1 -- 172.21.15.125:0/3770813864 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4004020 con 0x7f06cc15d400 2023-10-18T10:41:12.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.150+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc0b2050 con 0x7f06cc15d400 2023-10-18T10:41:12.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.151+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc0b2270 con 0x7f06cc15d400 2023-10-18T10:41:12.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.151+0000 7f06c97fa640 1 -- 172.21.15.125:0/3770813864 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f06c40041a0 con 0x7f06cc15d400 2023-10-18T10:41:12.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.152+0000 7f06c97fa640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06b006af60 0x7f06b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.152+0000 7f06d0e65640 1 -- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06b006af60 msgr2=0x7f06b006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:12.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.152+0000 7f06c97fa640 1 -- 172.21.15.125:0/3770813864 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f06c4014070 con 0x7f06cc15d400 2023-10-18T10:41:12.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.152+0000 7f06d0e65640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06b006af60 0x7f06b006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:12.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.156+0000 7f06c97fa640 1 -- 172.21.15.125:0/3770813864 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06c4055710 con 0x7f06cc15d400 2023-10-18T10:41:12.273 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:12.271+0000 7f1283245040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:41:12.281 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:12.280+0000 7fe82be43040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:41:12.317 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:12.316+0000 7f1283245040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:41:12.327 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:12.326+0000 7fe82be43040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:41:12.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.331+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06cc0675f0 con 0x7f06cc15d400 2023-10-18T10:41:12.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.332+0000 7f06c97fa640 1 -- 172.21.15.125:0/3770813864 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17331 (secure 0 0 0) 0x7f06c4058d30 con 0x7f06cc15d400 2023-10-18T10:41:12.334 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:12.334 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":99,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:09.823682+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:51.563590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:52.731689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2731662835":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/3976511705":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/1104476358":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2807756531":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6833/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6832/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/264862829":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2537511727":"2023-10-19T10:41:09.823648+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:12.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06b006af60 msgr2=0x7f06b006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:12.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06b006af60 0x7f06b006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06cc15d400 msgr2=0x7f06cc0b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06cc15d400 0x7f06cc0b2570 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f06c400ec20 tx=0x7f06c400c6a0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 shutdown_connections 2023-10-18T10:41:12.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06b006af60 0x7f06b006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06cc15d400 0x7f06cc0b2570 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06cc159060 0x7f06cc0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 --2- 172.21.15.125:0/3770813864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06cc0cdb30 0x7f06cc1eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 >> 172.21.15.125:0/3770813864 conn(0x7f06cc0b0670 msgr2=0x7f06cc0afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:12.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 shutdown_connections 2023-10-18T10:41:12.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.334+0000 7f06d30c9640 1 -- 172.21.15.125:0/3770813864 wait complete. 2023-10-18T10:41:12.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:40:51.563590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:40:52.731689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:41:12.348 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:41:12.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.454+0000 7f49b964b640 1 -- 172.21.15.125:0/1739599578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b41416f0 msgr2=0x7f49b40b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.454+0000 7f49b964b640 1 --2- 172.21.15.125:0/1739599578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b41416f0 0x7f49b40b1440 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f49b40b06d0 tx=0x7f49a402be40 comp rx=0 tx=0).stop 2023-10-18T10:41:12.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.454+0000 7f49b964b640 1 -- 172.21.15.125:0/1739599578 shutdown_connections 2023-10-18T10:41:12.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.454+0000 7f49b964b640 1 --2- 172.21.15.125:0/1739599578 >> v2:172.21.15.139:3300/0 conn(0x7f49b40b1980 0x7f49b4070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.454+0000 7f49b964b640 1 --2- 172.21.15.125:0/1739599578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b41416f0 0x7f49b40b1440 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.454+0000 7f49b964b640 1 -- 172.21.15.125:0/1739599578 >> 172.21.15.125:0/1739599578 conn(0x7f49b40af730 msgr2=0x7f49b40afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:12.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.454+0000 7f49b964b640 1 -- 172.21.15.125:0/1739599578 shutdown_connections 2023-10-18T10:41:12.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.454+0000 7f49b964b640 1 -- 172.21.15.125:0/1739599578 wait complete. 2023-10-18T10:41:12.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.456+0000 7f49b964b640 1 Processor -- start 2023-10-18T10:41:12.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.456+0000 7f49b964b640 1 -- start start 2023-10-18T10:41:12.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49b964b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49b40705c0 0x7f49b413fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49b964b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b40b1980 0x7f49b415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49b964b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49b41416f0 0x7f49b4137020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49b964b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4155d60 con 0x7f49b40b1980 2023-10-18T10:41:12.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49b964b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b41f34b0 con 0x7f49b40705c0 2023-10-18T10:41:12.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49b964b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b413b870 con 0x7f49b41416f0 2023-10-18T10:41:12.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49abfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b40b1980 0x7f49b415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49abfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b40b1980 0x7f49b415a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33328/0 (socket says 172.21.15.125:33328) 2023-10-18T10:41:12.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49abfff640 1 -- 172.21.15.125:0/2868601316 learned_addr learned my addr 172.21.15.125:0/2868601316 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:12.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49b2ffd640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49b40705c0 0x7f49b413fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.457+0000 7f49b37fe640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49b41416f0 0x7f49b4137020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49abfff640 1 -- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49b41416f0 msgr2=0x7f49b4137020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49abfff640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49b41416f0 0x7f49b4137020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49abfff640 1 -- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49b40705c0 msgr2=0x7f49b413fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49abfff640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49b40705c0 0x7f49b413fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49abfff640 1 -- 172.21.15.125:0/2868601316 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4009560 con 0x7f49b40b1980 2023-10-18T10:41:12.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49b37fe640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49b41416f0 0x7f49b4137020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:12.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49abfff640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b40b1980 0x7f49b415a640 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d980 tx=0x7f499c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:12.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49ab7fe640 1 -- 172.21.15.125:0/2868601316 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c009c30 con 0x7f49b40b1980 2023-10-18T10:41:12.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49ab7fe640 1 -- 172.21.15.125:0/2868601316 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f499c009e80 con 0x7f49b40b1980 2023-10-18T10:41:12.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49ab7fe640 1 -- 172.21.15.125:0/2868601316 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c0106a0 con 0x7f49b40b1980 2023-10-18T10:41:12.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b41f2600 con 0x7f49b40b1980 2023-10-18T10:41:12.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.458+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b41f2e60 con 0x7f49b40b1980 2023-10-18T10:41:12.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.459+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b40675f0 con 0x7f49b40b1980 2023-10-18T10:41:12.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.460+0000 7f49ab7fe640 1 -- 172.21.15.125:0/2868601316 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f499c010800 con 0x7f49b40b1980 2023-10-18T10:41:12.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.460+0000 7f49ab7fe640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f499406af60 0x7f499406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.460+0000 7f49b2ffd640 1 -- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f499406af60 msgr2=0x7f499406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:12.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.460+0000 7f49ab7fe640 1 -- 172.21.15.125:0/2868601316 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f499c089ee0 con 0x7f49b40b1980 2023-10-18T10:41:12.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.460+0000 7f49b2ffd640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f499406af60 0x7f499406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:12.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.464+0000 7f49ab7fe640 1 -- 172.21.15.125:0/2868601316 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f499c054b80 con 0x7f49b40b1980 2023-10-18T10:41:12.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.637+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49b40b2150 con 0x7f49b40b1980 2023-10-18T10:41:12.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.638+0000 7f49ab7fe640 1 -- 172.21.15.125:0/2868601316 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f499c0200a0 con 0x7f49b40b1980 2023-10-18T10:41:12.640 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:41:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f499406af60 msgr2=0x7f499406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f499406af60 0x7f499406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b40b1980 msgr2=0x7f49b415a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b40b1980 0x7f49b415a640 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d980 tx=0x7f499c00de40 comp rx=0 tx=0).stop 2023-10-18T10:41:12.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 shutdown_connections 2023-10-18T10:41:12.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f499406af60 0x7f499406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49b41416f0 0x7f49b4137020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49b40b1980 0x7f49b415a640 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 --2- 172.21.15.125:0/2868601316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49b40705c0 0x7f49b413fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 >> 172.21.15.125:0/2868601316 conn(0x7f49b40af730 msgr2=0x7f49b4157a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:12.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 shutdown_connections 2023-10-18T10:41:12.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.640+0000 7f49b964b640 1 -- 172.21.15.125:0/2868601316 wait complete. 2023-10-18T10:41:12.653 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:41:12.705 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:12.704+0000 7f1283245040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:41:12.737 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:12.736+0000 7fe82be43040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:41:12.749 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:12.748+0000 7f1283245040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:41:12.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 -- 172.21.15.125:0/2153145956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98500cdb30 msgr2=0x7f98500cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 --2- 172.21.15.125:0/2153145956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98500cdb30 0x7f98500cdfe0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9840005fd0 tx=0x7f984002bd10 comp rx=0 tx=0).stop 2023-10-18T10:41:12.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 -- 172.21.15.125:0/2153145956 shutdown_connections 2023-10-18T10:41:12.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 --2- 172.21.15.125:0/2153145956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f98500cdb30 0x7f98500cdfe0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 --2- 172.21.15.125:0/2153145956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f985015ccf0 0x7f985015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 --2- 172.21.15.125:0/2153145956 >> v2:172.21.15.139:3300/0 conn(0x7f9850159060 0x7f985006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 -- 172.21.15.125:0/2153145956 >> 172.21.15.125:0/2153145956 conn(0x7f98500b0670 msgr2=0x7f98500b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:12.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 -- 172.21.15.125:0/2153145956 shutdown_connections 2023-10-18T10:41:12.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.758+0000 7f9856839640 1 -- 172.21.15.125:0/2153145956 wait complete. 2023-10-18T10:41:12.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.759+0000 7f9856839640 1 Processor -- start 2023-10-18T10:41:12.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.759+0000 7f9856839640 1 -- start start 2023-10-18T10:41:12.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9856839640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98500cdb30 0x7f98500b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9856839640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9850159060 0x7f98500c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9856839640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f985015ccf0 0x7f9850147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9856839640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98500c8380 con 0x7f985015ccf0 2023-10-18T10:41:12.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9856839640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98500c7700 con 0x7f9850159060 2023-10-18T10:41:12.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9856839640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98500c79e0 con 0x7f98500cdb30 2023-10-18T10:41:12.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9854dd6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f985015ccf0 0x7f9850147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9854dd6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f985015ccf0 0x7f9850147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33336/0 (socket says 172.21.15.125:33336) 2023-10-18T10:41:12.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9854dd6640 1 -- 172.21.15.125:0/1131081013 learned_addr learned my addr 172.21.15.125:0/1131081013 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:12.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f984f7fe640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9850159060 0x7f98500c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f984ffff640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98500cdb30 0x7f98500b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:12.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9854dd6640 1 -- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98500cdb30 msgr2=0x7f98500b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9854dd6640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98500cdb30 0x7f98500b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9854dd6640 1 -- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9850159060 msgr2=0x7f98500c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9854dd6640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9850159060 0x7f98500c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.760+0000 7f9854dd6640 1 -- 172.21.15.125:0/1131081013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840009560 con 0x7f985015ccf0 2023-10-18T10:41:12.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.761+0000 7f9854dd6640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f985015ccf0 0x7f9850147d50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f984800ea80 tx=0x7f984800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:12.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.761+0000 7f984cff9640 1 -- 172.21.15.125:0/1131081013 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984800cea0 con 0x7f985015ccf0 2023-10-18T10:41:12.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.761+0000 7f984cff9640 1 -- 172.21.15.125:0/1131081013 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9848004b20 con 0x7f985015ccf0 2023-10-18T10:41:12.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.761+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98500c7cc0 con 0x7f985015ccf0 2023-10-18T10:41:12.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.761+0000 7f984cff9640 1 -- 172.21.15.125:0/1131081013 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848005020 con 0x7f985015ccf0 2023-10-18T10:41:12.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.761+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98500b1f10 con 0x7f985015ccf0 2023-10-18T10:41:12.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.762+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98500b2100 con 0x7f985015ccf0 2023-10-18T10:41:12.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.762+0000 7f984cff9640 1 -- 172.21.15.125:0/1131081013 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f9848005180 con 0x7f985015ccf0 2023-10-18T10:41:12.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.763+0000 7f984cff9640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f983c06af90 0x7f983c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:12.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.763+0000 7f984ffff640 1 -- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f983c06af90 msgr2=0x7f983c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:12.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.763+0000 7f984cff9640 1 -- 172.21.15.125:0/1131081013 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f9848014070 con 0x7f985015ccf0 2023-10-18T10:41:12.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.763+0000 7f984ffff640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f983c06af90 0x7f983c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:12.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.767+0000 7f984cff9640 1 -- 172.21.15.125:0/1131081013 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98480552b0 con 0x7f985015ccf0 2023-10-18T10:41:12.784 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:12.783+0000 7fe82be43040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:41:12.792 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:12.790+0000 7f1283245040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:41:12.828 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:12.827+0000 7fe82be43040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:41:12.844 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:12.843+0000 7f1283245040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:41:12.882 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:12.880+0000 7fe82be43040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:41:12.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:12.884+0000 7f1283245040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:41:12.925 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:12.924+0000 7fe82be43040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:41:12.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.943+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98500675f0 con 0x7f985015ccf0 2023-10-18T10:41:12.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.943+0000 7f984cff9640 1 -- 172.21.15.125:0/1131081013 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f98480588d0 con 0x7f985015ccf0 2023-10-18T10:41:12.945 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:41:12.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f983c06af90 msgr2=0x7f983c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:12.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f983c06af90 0x7f983c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f985015ccf0 msgr2=0x7f9850147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f985015ccf0 0x7f9850147d50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f984800ea80 tx=0x7f984800ef40 comp rx=0 tx=0).stop 2023-10-18T10:41:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 shutdown_connections 2023-10-18T10:41:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f983c06af90 0x7f983c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9850159060 0x7f98500c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f985015ccf0 0x7f9850147d50 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 --2- 172.21.15.125:0/1131081013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98500cdb30 0x7f98500b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:12.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 >> 172.21.15.125:0/1131081013 conn(0x7f98500b0670 msgr2=0x7f9850157fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:12.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 shutdown_connections 2023-10-18T10:41:12.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:12.945+0000 7f9856839640 1 -- 172.21.15.125:0/1131081013 wait complete. 2023-10-18T10:41:12.958 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:41:12.984 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:12.983+0000 7f1283245040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:41:13.029 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.027+0000 7fe82be43040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:41:13.036 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.035+0000 7f1283245040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:13.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 -- 172.21.15.125:0/131096452 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380cf340 msgr2=0x7f8e380d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/131096452 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380cf340 0x7f8e380d1910 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8e3808a120 tx=0x7f8e2802bd10 comp rx=0 tx=0).stop 2023-10-18T10:41:13.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 -- 172.21.15.125:0/131096452 shutdown_connections 2023-10-18T10:41:13.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/131096452 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380ceb40 0x7f8e380b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/131096452 >> v2:172.21.15.139:3300/0 conn(0x7f8e3815b2a0 0x7f8e380cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/131096452 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380cf340 0x7f8e380d1910 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 -- 172.21.15.125:0/131096452 >> 172.21.15.125:0/131096452 conn(0x7f8e380d0d60 msgr2=0x7f8e380d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:13.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 -- 172.21.15.125:0/131096452 shutdown_connections 2023-10-18T10:41:13.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.063+0000 7f8e3cebb640 1 -- 172.21.15.125:0/131096452 wait complete. 2023-10-18T10:41:13.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.064+0000 7f8e3cebb640 1 Processor -- start 2023-10-18T10:41:13.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.064+0000 7f8e3cebb640 1 -- start start 2023-10-18T10:41:13.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3cebb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380ceb40 0x7f8e38158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3cebb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e380cf340 0x7f8e380b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3cebb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e3815b2a0 0x7f8e3815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3cebb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380b4130 con 0x7f8e380ceb40 2023-10-18T10:41:13.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3cebb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380b53a0 con 0x7f8e380cf340 2023-10-18T10:41:13.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3cebb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380b3e90 con 0x7f8e3815b2a0 2023-10-18T10:41:13.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380ceb40 0x7f8e38158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380ceb40 0x7f8e38158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33362/0 (socket says 172.21.15.125:33362) 2023-10-18T10:41:13.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e3659c640 1 -- 172.21.15.125:0/159040713 learned_addr learned my addr 172.21.15.125:0/159040713 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:13.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e2ffff640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e380cf340 0x7f8e380b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.065+0000 7f8e36d9d640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e3815b2a0 0x7f8e3815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e3659c640 1 -- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e3815b2a0 msgr2=0x7f8e3815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e3659c640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e3815b2a0 0x7f8e3815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e3659c640 1 -- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e380cf340 msgr2=0x7f8e380b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e3659c640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e380cf340 0x7f8e380b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e3659c640 1 -- 172.21.15.125:0/159040713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e28009560 con 0x7f8e380ceb40 2023-10-18T10:41:13.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e36d9d640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e3815b2a0 0x7f8e3815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:13.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e3659c640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380ceb40 0x7f8e38158400 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8e280099c0 tx=0x7f8e2802dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:13.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e2effd640 1 -- 172.21.15.125:0/159040713 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28003800 con 0x7f8e380ceb40 2023-10-18T10:41:13.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e2effd640 1 -- 172.21.15.125:0/159040713 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8e28003a50 con 0x7f8e380ceb40 2023-10-18T10:41:13.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e380b4a00 con 0x7f8e380ceb40 2023-10-18T10:41:13.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e2effd640 1 -- 172.21.15.125:0/159040713 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28003e60 con 0x7f8e380ceb40 2023-10-18T10:41:13.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.066+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e380b4fc0 con 0x7f8e380ceb40 2023-10-18T10:41:13.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.067+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e380690e0 con 0x7f8e380ceb40 2023-10-18T10:41:13.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.067+0000 7f8e2effd640 1 -- 172.21.15.125:0/159040713 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8e2803e450 con 0x7f8e380ceb40 2023-10-18T10:41:13.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.068+0000 7f8e2effd640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f8e1806af60 0x7f8e1806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.068+0000 7f8e2ffff640 1 -- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f8e1806af60 msgr2=0x7f8e1806d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:13.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.068+0000 7f8e2effd640 1 -- 172.21.15.125:0/159040713 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f8e280ac550 con 0x7f8e380ceb40 2023-10-18T10:41:13.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.068+0000 7f8e2ffff640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f8e1806af60 0x7f8e1806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:13.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.072+0000 7f8e2effd640 1 -- 172.21.15.125:0/159040713 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e280772e0 con 0x7f8e380ceb40 2023-10-18T10:41:13.078 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.077+0000 7f1283245040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:41:13.086 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.084+0000 7fe82be43040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:13.126 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.124+0000 7f1283245040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:41:13.128 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.127+0000 7fe82be43040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:41:13.168 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.166+0000 7f1283245040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:41:13.178 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.177+0000 7fe82be43040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:41:13.222 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.220+0000 7fe82be43040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:41:13.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.244+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e380d0b40 con 0x7f8e380ceb40 2023-10-18T10:41:13.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.245+0000 7f8e2effd640 1 -- 172.21.15.125:0/159040713 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e2807a900 con 0x7f8e380ceb40 2023-10-18T10:41:13.247 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:41:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f8e1806af60 msgr2=0x7f8e1806d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f8e1806af60 0x7f8e1806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380ceb40 msgr2=0x7f8e38158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380ceb40 0x7f8e38158400 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8e280099c0 tx=0x7f8e2802dab0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 shutdown_connections 2023-10-18T10:41:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f8e1806af60 0x7f8e1806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e380cf340 0x7f8e380b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e3815b2a0 0x7f8e3815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 --2- 172.21.15.125:0/159040713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e380ceb40 0x7f8e38158400 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 >> 172.21.15.125:0/159040713 conn(0x7f8e380d0d60 msgr2=0x7f8e380cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:13.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 shutdown_connections 2023-10-18T10:41:13.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.247+0000 7f8e3cebb640 1 -- 172.21.15.125:0/159040713 wait complete. 2023-10-18T10:41:13.259 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:41:13.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.364+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/1365124591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a04004d00 msgr2=0x7f5a04005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.364+0000 7f5a1c9b6640 1 --2- 172.21.15.125:0/1365124591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a04004d00 0x7f5a04005180 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5a10001e90 tx=0x7f5a040057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.364+0000 7f5a16d9d640 1 -- 172.21.15.125:0/1365124591 reap_dead start 2023-10-18T10:41:13.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.364+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/1365124591 shutdown_connections 2023-10-18T10:41:13.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.364+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/1365124591 >> 172.21.15.125:0/1365124591 conn(0x7f5a180d0d60 msgr2=0x7f5a180d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:13.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.364+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/1365124591 shutdown_connections 2023-10-18T10:41:13.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.364+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/1365124591 wait complete. 2023-10-18T10:41:13.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.366+0000 7f5a1c9b6640 1 Processor -- start 2023-10-18T10:41:13.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.366+0000 7f5a1c9b6640 1 -- start start 2023-10-18T10:41:13.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.366+0000 7f5a1c9b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a04004d00 0x7f5a180af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.366+0000 7f5a1c9b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a181585b0 0x7f5a1806b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.366+0000 7f5a1c9b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1806b850 0x7f5a1815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.366+0000 7f5a1c9b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a18158a30 con 0x7f5a181585b0 2023-10-18T10:41:13.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.366+0000 7f5a1c9b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1806bcd0 con 0x7f5a04004d00 2023-10-18T10:41:13.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.366+0000 7f5a1c9b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180b5670 con 0x7f5a1806b850 2023-10-18T10:41:13.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a181585b0 0x7f5a1806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a181585b0 0x7f5a1806b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33374/0 (socket says 172.21.15.125:33374) 2023-10-18T10:41:13.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 -- 172.21.15.125:0/326636541 learned_addr learned my addr 172.21.15.125:0/326636541 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:13.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a1659c640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a04004d00 0x7f5a180af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a16d9d640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1806b850 0x7f5a1815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 -- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1806b850 msgr2=0x7f5a1815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1806b850 0x7f5a1815b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 -- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a04004d00 msgr2=0x7f5a180af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a04004d00 0x7f5a180af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 -- 172.21.15.125:0/326636541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180b5b00 con 0x7f5a181585b0 2023-10-18T10:41:13.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0ffff640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a181585b0 0x7f5a1806b310 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5a00011ae0 tx=0x7f5a00009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:13.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0effd640 1 -- 172.21.15.125:0/326636541 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000fe70 con 0x7f5a181585b0 2023-10-18T10:41:13.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0effd640 1 -- 172.21.15.125:0/326636541 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a00004050 con 0x7f5a181585b0 2023-10-18T10:41:13.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a0effd640 1 -- 172.21.15.125:0/326636541 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00004460 con 0x7f5a181585b0 2023-10-18T10:41:13.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a180b6720 con 0x7f5a181585b0 2023-10-18T10:41:13.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.367+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a180b4d40 con 0x7f5a181585b0 2023-10-18T10:41:13.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.368+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a180677b0 con 0x7f5a181585b0 2023-10-18T10:41:13.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.369+0000 7f5a0effd640 1 -- 172.21.15.125:0/326636541 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f5a000035f0 con 0x7f5a181585b0 2023-10-18T10:41:13.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.369+0000 7f5a0effd640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f59f806af90 0x7f59f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.369+0000 7f5a1659c640 1 -- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f59f806af90 msgr2=0x7f59f806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:13.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.370+0000 7f5a0effd640 1 -- 172.21.15.125:0/326636541 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f5a0008a940 con 0x7f5a181585b0 2023-10-18T10:41:13.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.370+0000 7f5a1659c640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f59f806af90 0x7f59f806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:13.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.373+0000 7f5a0effd640 1 -- 172.21.15.125:0/326636541 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a00055600 con 0x7f5a181585b0 2023-10-18T10:41:13.484 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.482+0000 7f1283245040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:41:13.555 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.553+0000 7fe82be43040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:41:13.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.559+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5a180690e0 con 0x7f5a181585b0 2023-10-18T10:41:13.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.560+0000 7f5a0effd640 1 -- 172.21.15.125:0/326636541 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v99) v1 ==== 80+0+475 (secure 0 0 0) 0x7f5a0001e080 con 0x7f5a181585b0 2023-10-18T10:41:13.562 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2731662835 2023-10-19T10:41:09.823648+0000 2023-10-18T10:41:13.562 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3976511705 2023-10-19T10:41:09.823648+0000 2023-10-18T10:41:13.562 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1104476358 2023-10-19T10:41:09.823648+0000 2023-10-18T10:41:13.563 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2807756531 2023-10-19T10:41:09.823648+0000 2023-10-18T10:41:13.563 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/143413153 2023-10-19T10:41:09.823648+0000 2023-10-18T10:41:13.563 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/143413153 2023-10-19T10:41:09.823648+0000 2023-10-18T10:41:13.563 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/264862829 2023-10-19T10:41:09.823648+0000 2023-10-18T10:41:13.563 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2537511727 2023-10-19T10:41:09.823648+0000 2023-10-18T10:41:13.564 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T10:41:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f59f806af90 msgr2=0x7f59f806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f59f806af90 0x7f59f806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a181585b0 msgr2=0x7f5a1806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a181585b0 0x7f5a1806b310 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5a00011ae0 tx=0x7f5a00009e90 comp rx=0 tx=0).stop 2023-10-18T10:41:13.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 shutdown_connections 2023-10-18T10:41:13.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f59f806af90 0x7f59f806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a1806b850 0x7f5a1815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a181585b0 0x7f5a1806b310 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 --2- 172.21.15.125:0/326636541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a04004d00 0x7f5a180af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 >> 172.21.15.125:0/326636541 conn(0x7f5a180d0d60 msgr2=0x7f5a180c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:13.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 shutdown_connections 2023-10-18T10:41:13.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.562+0000 7f5a1c9b6640 1 -- 172.21.15.125:0/326636541 wait complete. 2023-10-18T10:41:13.574 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:41:13.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:13.575 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:13.601 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[122557]: fuse finished with error 0 and tester_r 0 2023-10-18T10:41:13.602 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:41:13.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:13.603 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:41:13.615 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.614+0000 7f1283245040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:41:13.673 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:41:13.673 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:41:13.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:41:13.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:41:13.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:13.691 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:41:13.692 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.691+0000 7fe82be43040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:41:13.710 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.708+0000 7f1283245040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:41:13.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:41:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T10:41:13.725 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:41:13.725 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:41:13.742 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:41:13.742 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:41:13.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:13.743 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:41:13.754 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.753+0000 7f1283245040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:41:13.788 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.786+0000 7fe82be43040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:41:13.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:41:13.797 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T10:41:13.798 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:41:13.798 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:41:13.814 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:41:13.815 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:41:13.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:13.816 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:41:13.832 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.830+0000 7fe82be43040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:41:13.852 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.850+0000 7f1283245040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:13.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:41:13.867 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T10:41:13.868 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:41:13.930 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:13.928+0000 7fe82be43040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:13.946 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:13.945+0000 7f1283245040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:41:13.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.970+0000 7fac585e9640 1 -- 172.21.15.125:0/3704505427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac500ceb40 msgr2=0x7fac500b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.970+0000 7fac585e9640 1 --2- 172.21.15.125:0/3704505427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac500ceb40 0x7fac500b0650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fac48005fd0 tx=0x7fac4802c050 comp rx=0 tx=0).stop 2023-10-18T10:41:13.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.970+0000 7fac585e9640 1 -- 172.21.15.125:0/3704505427 shutdown_connections 2023-10-18T10:41:13.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.970+0000 7fac585e9640 1 --2- 172.21.15.125:0/3704505427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac500ceb40 0x7fac500b0650 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.970+0000 7fac585e9640 1 --2- 172.21.15.125:0/3704505427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac500cf340 0x7fac500d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.970+0000 7fac585e9640 1 -- 172.21.15.125:0/3704505427 >> 172.21.15.125:0/3704505427 conn(0x7fac500d0d60 msgr2=0x7fac500d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:13.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.970+0000 7fac585e9640 1 -- 172.21.15.125:0/3704505427 shutdown_connections 2023-10-18T10:41:13.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.970+0000 7fac585e9640 1 -- 172.21.15.125:0/3704505427 wait complete. 2023-10-18T10:41:13.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.971+0000 7fac585e9640 1 Processor -- start 2023-10-18T10:41:13.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.971+0000 7fac585e9640 1 -- start start 2023-10-18T10:41:13.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac585e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac500ceb40 0x7fac501e4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac585e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac500cf340 0x7fac500caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac585e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac5015b2a0 0x7fac5015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac585e9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac501e54e0 con 0x7fac500ceb40 2023-10-18T10:41:13.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac585e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac500cafa0 con 0x7fac5015b2a0 2023-10-18T10:41:13.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac585e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac501e48d0 con 0x7fac500cf340 2023-10-18T10:41:13.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56385640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac500ceb40 0x7fac501e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56385640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac500ceb40 0x7fac501e4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33382/0 (socket says 172.21.15.125:33382) 2023-10-18T10:41:13.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56385640 1 -- 172.21.15.125:0/3547426065 learned_addr learned my addr 172.21.15.125:0/3547426065 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:13.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56b86640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac5015b2a0 0x7fac5015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac55b84640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac500cf340 0x7fac500caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:13.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56b86640 1 -- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac500cf340 msgr2=0x7fac500caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56b86640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac500cf340 0x7fac500caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56b86640 1 -- 172.21.15.125:0/3547426065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac500ceb40 msgr2=0x7fac501e4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:13.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56b86640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac500ceb40 0x7fac501e4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:13.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56b86640 1 -- 172.21.15.125:0/3547426065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac48009840 con 0x7fac5015b2a0 2023-10-18T10:41:13.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56385640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac500ceb40 0x7fac501e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:41:13.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.972+0000 7fac56b86640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac5015b2a0 0x7fac5015d700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fac50069e00 tx=0x7fac480042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:13.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.973+0000 7fac46ffd640 1 -- 172.21.15.125:0/3547426065 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4803d430 con 0x7fac5015b2a0 2023-10-18T10:41:13.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.973+0000 7fac46ffd640 1 -- 172.21.15.125:0/3547426065 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac4803d680 con 0x7fac5015b2a0 2023-10-18T10:41:13.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.973+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac501e4bb0 con 0x7fac5015b2a0 2023-10-18T10:41:13.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.973+0000 7fac46ffd640 1 -- 172.21.15.125:0/3547426065 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4803da90 con 0x7fac5015b2a0 2023-10-18T10:41:13.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.973+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac501e5170 con 0x7fac5015b2a0 2023-10-18T10:41:13.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.974+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac501e5690 con 0x7fac5015b2a0 2023-10-18T10:41:13.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.974+0000 7fac46ffd640 1 -- 172.21.15.125:0/3547426065 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fac4803dbf0 con 0x7fac5015b2a0 2023-10-18T10:41:13.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.975+0000 7fac46ffd640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7fac3406af60 0x7fac3406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:13.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.975+0000 7fac56385640 1 -- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7fac3406af60 msgr2=0x7fac3406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:13.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.975+0000 7fac46ffd640 1 -- 172.21.15.125:0/3547426065 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7fac480ac740 con 0x7fac5015b2a0 2023-10-18T10:41:13.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.975+0000 7fac56385640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7fac3406af60 0x7fac3406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:13.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:13.983+0000 7fac46ffd640 1 -- 172.21.15.125:0/3547426065 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac501e5690 con 0x7fac5015b2a0 2023-10-18T10:41:14.024 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:14.023+0000 7fe82be43040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:41:14.086 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:14.084+0000 7f1283245040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:14.127 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:14.126+0000 7f1283245040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:14.160 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:14.159+0000 7fe82be43040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:14.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.176+0000 7fac56385640 1 -- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7fac3406af60 msgr2=0x7fac3406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:14.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.176+0000 7fac56385640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7fac3406af60 0x7fac3406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:41:14.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.187+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac501e5690 con 0x7fac5015b2a0 2023-10-18T10:41:14.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.188+0000 7fac46ffd640 1 -- 172.21.15.125:0/3547426065 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fac501e5690 con 0x7fac5015b2a0 2023-10-18T10:41:14.191 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:14.191 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3531064821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3531064821},{"type":"v1","addr":"172.21.15.139:6837","nonce":3531064821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2850027191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2850027191},{"type":"v1","addr":"172.21.15.139:6836","nonce":2850027191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7578,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3670537575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3670537575},{"type":"v1","addr":"172.21.15.125:6834","nonce":3670537575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:53.749412+0000","modified":"2023-10-18T10:40:55.750936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7581":{"gid":7581,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1041197980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1041197980},{"type":"v1","addr":"172.21.15.125:6835","nonce":1041197980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T10:41:14.193 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 35 2023-10-18T10:41:14.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7fac3406af60 msgr2=0x7fac3406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:14.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7fac3406af60 0x7fac3406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac5015b2a0 msgr2=0x7fac5015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac5015b2a0 0x7fac5015d700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fac50069e00 tx=0x7fac480042f0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 shutdown_connections 2023-10-18T10:41:14.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7fac3406af60 0x7fac3406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac5015b2a0 0x7fac5015d700 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac500cf340 0x7fac500caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 --2- 172.21.15.125:0/3547426065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac500ceb40 0x7fac501e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 >> 172.21.15.125:0/3547426065 conn(0x7fac500d0d60 msgr2=0x7fac5006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:14.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 shutdown_connections 2023-10-18T10:41:14.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.190+0000 7fac585e9640 1 -- 172.21.15.125:0/3547426065 wait complete. 2023-10-18T10:41:14.203 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:14.205 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:14.202+0000 7fe82be43040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:14.231 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:14.229+0000 7f1283245040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:41:14.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.307+0000 7f8154f5a640 1 -- 172.21.15.125:0/1802415952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 msgr2=0x7f815006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.307+0000 7f8154f5a640 1 --2- 172.21.15.125:0/1802415952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 0x7f815006d260 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f8150071150 tx=0x7f814002bdb0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.307+0000 7f8154f5a640 1 -- 172.21.15.125:0/1802415952 shutdown_connections 2023-10-18T10:41:14.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.307+0000 7f8154f5a640 1 --2- 172.21.15.125:0/1802415952 >> v2:172.21.15.139:3300/0 conn(0x7f815015ccf0 0x7f815015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.307+0000 7f8154f5a640 1 --2- 172.21.15.125:0/1802415952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 0x7f815006d260 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.307+0000 7f8154f5a640 1 -- 172.21.15.125:0/1802415952 >> 172.21.15.125:0/1802415952 conn(0x7f81500b0670 msgr2=0x7f81500b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:14.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.307+0000 7f8154f5a640 1 -- 172.21.15.125:0/1802415952 shutdown_connections 2023-10-18T10:41:14.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.307+0000 7f8154f5a640 1 -- 172.21.15.125:0/1802415952 wait complete. 2023-10-18T10:41:14.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.308+0000 7f8154f5a640 1 Processor -- start 2023-10-18T10:41:14.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f8154f5a640 1 -- start start 2023-10-18T10:41:14.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f8154f5a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 0x7f8150157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f8154f5a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8150159060 0x7f81500c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f8154f5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815015ccf0 0x7f81500c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f8154f5a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500b2480 con 0x7f81500ce240 2023-10-18T10:41:14.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f8154f5a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500b2760 con 0x7f815015ccf0 2023-10-18T10:41:14.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f8154f5a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81500b2a40 con 0x7f8150159060 2023-10-18T10:41:14.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f814e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 0x7f8150157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f814e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 0x7f8150157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33390/0 (socket says 172.21.15.125:33390) 2023-10-18T10:41:14.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f814e59c640 1 -- 172.21.15.125:0/3927251524 learned_addr learned my addr 172.21.15.125:0/3927251524 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:14.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f814ed9d640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815015ccf0 0x7f81500c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.309+0000 7f814dd9b640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8150159060 0x7f81500c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f814e59c640 1 -- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8150159060 msgr2=0x7f81500c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f814e59c640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8150159060 0x7f81500c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f814e59c640 1 -- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815015ccf0 msgr2=0x7f81500c18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f814e59c640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815015ccf0 0x7f81500c18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f814e59c640 1 -- 172.21.15.125:0/3927251524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140009560 con 0x7f81500ce240 2023-10-18T10:41:14.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f814dd9b640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8150159060 0x7f81500c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:14.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f814e59c640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 0x7f8150157fc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f813800c4b0 tx=0x7f813800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:14.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f8146ffd640 1 -- 172.21.15.125:0/3927251524 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138005480 con 0x7f81500ce240 2023-10-18T10:41:14.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f8146ffd640 1 -- 172.21.15.125:0/3927251524 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f81380036e0 con 0x7f81500ce240 2023-10-18T10:41:14.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81500b2d20 con 0x7f81500ce240 2023-10-18T10:41:14.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f8146ffd640 1 -- 172.21.15.125:0/3927251524 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138003b30 con 0x7f81500ce240 2023-10-18T10:41:14.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.310+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8150158500 con 0x7f81500ce240 2023-10-18T10:41:14.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.311+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81500b2f60 con 0x7f81500ce240 2023-10-18T10:41:14.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.311+0000 7f8146ffd640 1 -- 172.21.15.125:0/3927251524 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8138003c90 con 0x7f81500ce240 2023-10-18T10:41:14.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.312+0000 7f8146ffd640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f813006af60 0x7f813006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.312+0000 7f814dd9b640 1 -- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f813006af60 msgr2=0x7f813006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:14.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.312+0000 7f8146ffd640 1 -- 172.21.15.125:0/3927251524 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f8138089180 con 0x7f81500ce240 2023-10-18T10:41:14.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.312+0000 7f814dd9b640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f813006af60 0x7f813006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:14.320 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:14.314+0000 7fe82be43040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:41:14.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.316+0000 7f8146ffd640 1 -- 172.21.15.125:0/3927251524 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81500b2f60 con 0x7f81500ce240 2023-10-18T10:41:14.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.492+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81500b2f60 con 0x7f81500ce240 2023-10-18T10:41:14.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.493+0000 7f8146ffd640 1 -- 172.21.15.125:0/3927251524 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17331 (secure 0 0 0) 0x7f81500b2f60 con 0x7f81500ce240 2023-10-18T10:41:14.495 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:14.495 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":99,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:09.823682+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:51.563590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:52.731689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2731662835":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/3976511705":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/1104476358":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2807756531":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6833/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6832/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/264862829":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2537511727":"2023-10-19T10:41:09.823648+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:14.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f813006af60 msgr2=0x7f813006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:14.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f813006af60 0x7f813006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 msgr2=0x7f8150157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 0x7f8150157fc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f813800c4b0 tx=0x7f813800c970 comp rx=0 tx=0).stop 2023-10-18T10:41:14.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 shutdown_connections 2023-10-18T10:41:14.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f813006af60 0x7f813006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f815015ccf0 0x7f81500c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8150159060 0x7f81500c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 --2- 172.21.15.125:0/3927251524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81500ce240 0x7f8150157fc0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 >> 172.21.15.125:0/3927251524 conn(0x7f81500b0670 msgr2=0x7f81500ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:14.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 shutdown_connections 2023-10-18T10:41:14.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.495+0000 7f8154f5a640 1 -- 172.21.15.125:0/3927251524 wait complete. 2023-10-18T10:41:14.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:40:51.563590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:40:52.731689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:41:14.509 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:41:14.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.615+0000 7f0719a44640 1 -- 172.21.15.125:0/318074469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f071415b2a0 msgr2=0x7f07140d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.615+0000 7f0719a44640 1 --2- 172.21.15.125:0/318074469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f071415b2a0 0x7f07140d1910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009830 tx=0x7f06fc02bc50 comp rx=0 tx=0).stop 2023-10-18T10:41:14.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.615+0000 7f0719a44640 1 -- 172.21.15.125:0/318074469 shutdown_connections 2023-10-18T10:41:14.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.615+0000 7f0719a44640 1 --2- 172.21.15.125:0/318074469 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f071415b2a0 0x7f07140d1910 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.615+0000 7f0719a44640 1 -- 172.21.15.125:0/318074469 >> 172.21.15.125:0/318074469 conn(0x7f07140d0d60 msgr2=0x7f07140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:14.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.616+0000 7f0719a44640 1 -- 172.21.15.125:0/318074469 shutdown_connections 2023-10-18T10:41:14.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.616+0000 7f0719a44640 1 -- 172.21.15.125:0/318074469 wait complete. 2023-10-18T10:41:14.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f0719a44640 1 Processor -- start 2023-10-18T10:41:14.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f0719a44640 1 -- start start 2023-10-18T10:41:14.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f0719a44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07140ceb40 0x7f0714157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f0719a44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07140cf340 0x7f07140ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f0719a44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f071415b2a0 0x7f07140ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f0719a44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071415ce90 con 0x7f071415b2a0 2023-10-18T10:41:14.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f0719a44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0714157e80 con 0x7f07140ceb40 2023-10-18T10:41:14.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f0719a44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07140c9830 con 0x7f07140cf340 2023-10-18T10:41:14.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.617+0000 7f07137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f071415b2a0 0x7f07140ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f07137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f071415b2a0 0x7f07140ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33406/0 (socket says 172.21.15.125:33406) 2023-10-18T10:41:14.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f07137fe640 1 -- 172.21.15.125:0/1783079756 learned_addr learned my addr 172.21.15.125:0/1783079756 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:14.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f0712ffd640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07140ceb40 0x7f0714157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f070bfff640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07140cf340 0x7f07140ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f07137fe640 1 -- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07140cf340 msgr2=0x7f07140ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f07137fe640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07140cf340 0x7f07140ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f07137fe640 1 -- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07140ceb40 msgr2=0x7f0714157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f07137fe640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07140ceb40 0x7f0714157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f07137fe640 1 -- 172.21.15.125:0/1783079756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc009490 con 0x7f071415b2a0 2023-10-18T10:41:14.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f07137fe640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f071415b2a0 0x7f07140ae900 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f070c00c790 tx=0x7f070c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:14.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f0710ff9640 1 -- 172.21.15.125:0/1783079756 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00cea0 con 0x7f071415b2a0 2023-10-18T10:41:14.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f0710ff9640 1 -- 172.21.15.125:0/1783079756 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f070c003b70 con 0x7f071415b2a0 2023-10-18T10:41:14.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07140c8f90 con 0x7f071415b2a0 2023-10-18T10:41:14.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f0710ff9640 1 -- 172.21.15.125:0/1783079756 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00a850 con 0x7f071415b2a0 2023-10-18T10:41:14.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.618+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07140c9550 con 0x7f071415b2a0 2023-10-18T10:41:14.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.619+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071415ccf0 con 0x7f071415b2a0 2023-10-18T10:41:14.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.620+0000 7f0710ff9640 1 -- 172.21.15.125:0/1783079756 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f070c00a9b0 con 0x7f071415b2a0 2023-10-18T10:41:14.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.620+0000 7f0710ff9640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06f406af60 0x7f06f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.620+0000 7f0712ffd640 1 -- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06f406af60 msgr2=0x7f06f406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:14.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.620+0000 7f0710ff9640 1 -- 172.21.15.125:0/1783079756 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f070c089950 con 0x7f071415b2a0 2023-10-18T10:41:14.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.620+0000 7f0712ffd640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06f406af60 0x7f06f406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:14.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.624+0000 7f0710ff9640 1 -- 172.21.15.125:0/1783079756 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f071415ccf0 con 0x7f071415b2a0 2023-10-18T10:41:14.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.799+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f071415ccf0 con 0x7f071415b2a0 2023-10-18T10:41:14.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.800+0000 7f0710ff9640 1 -- 172.21.15.125:0/1783079756 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f071415ccf0 con 0x7f071415b2a0 2023-10-18T10:41:14.802 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:41:14.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06f406af60 msgr2=0x7f06f406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:14.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06f406af60 0x7f06f406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f071415b2a0 msgr2=0x7f07140ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f071415b2a0 0x7f07140ae900 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f070c00c790 tx=0x7f070c00cc50 comp rx=0 tx=0).stop 2023-10-18T10:41:14.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 shutdown_connections 2023-10-18T10:41:14.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f06f406af60 0x7f06f406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f071415b2a0 0x7f07140ae900 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07140cf340 0x7f07140ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 --2- 172.21.15.125:0/1783079756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07140ceb40 0x7f0714157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 >> 172.21.15.125:0/1783079756 conn(0x7f07140d0d60 msgr2=0x7f07140ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:14.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 shutdown_connections 2023-10-18T10:41:14.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.802+0000 7f0719a44640 1 -- 172.21.15.125:0/1783079756 wait complete. 2023-10-18T10:41:14.814 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:41:14.842 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:14.840+0000 7f1283245040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:41:14.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.921+0000 7f39c447c640 1 -- 172.21.15.125:0/3617553086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc13ebe0 msgr2=0x7f39bc070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.921+0000 7f39c447c640 1 --2- 172.21.15.125:0/3617553086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc13ebe0 0x7f39bc070ce0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f39b4005fd0 tx=0x7f39b402c170 comp rx=0 tx=0).stop 2023-10-18T10:41:14.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.921+0000 7f39c447c640 1 -- 172.21.15.125:0/3617553086 shutdown_connections 2023-10-18T10:41:14.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.921+0000 7f39c447c640 1 --2- 172.21.15.125:0/3617553086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc13ebe0 0x7f39bc070ce0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.921+0000 7f39c447c640 1 -- 172.21.15.125:0/3617553086 >> 172.21.15.125:0/3617553086 conn(0x7f39bc0af730 msgr2=0x7f39bc0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:14.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.922+0000 7f39c447c640 1 -- 172.21.15.125:0/3617553086 shutdown_connections 2023-10-18T10:41:14.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.922+0000 7f39c447c640 1 -- 172.21.15.125:0/3617553086 wait complete. 2023-10-18T10:41:14.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c447c640 1 Processor -- start 2023-10-18T10:41:14.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c447c640 1 -- start start 2023-10-18T10:41:14.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c447c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39bc069e30 0x7f39bc13da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c447c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39bc13bd70 0x7f39bc137430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c447c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc13ebe0 0x7f39bc13aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c447c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc13df70 con 0x7f39bc069e30 2023-10-18T10:41:14.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c447c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc137970 con 0x7f39bc13ebe0 2023-10-18T10:41:14.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c447c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc155b00 con 0x7f39bc13bd70 2023-10-18T10:41:14.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c2218640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39bc069e30 0x7f39bc13da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c2218640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39bc069e30 0x7f39bc13da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33422/0 (socket says 172.21.15.125:33422) 2023-10-18T10:41:14.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.923+0000 7f39c2218640 1 -- 172.21.15.125:0/1799270847 learned_addr learned my addr 172.21.15.125:0/1799270847 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:14.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c2a19640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc13ebe0 0x7f39bc13aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c1a17640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39bc13bd70 0x7f39bc137430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:14.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c2218640 1 -- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39bc13bd70 msgr2=0x7f39bc137430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c2218640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39bc13bd70 0x7f39bc137430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c2218640 1 -- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc13ebe0 msgr2=0x7f39bc13aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:14.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c2218640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc13ebe0 0x7f39bc13aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:14.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c2218640 1 -- 172.21.15.125:0/1799270847 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4009840 con 0x7f39bc069e30 2023-10-18T10:41:14.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c2218640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39bc069e30 0x7f39bc13da30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c490 tx=0x7f39b800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:14.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39b2ffd640 1 -- 172.21.15.125:0/1799270847 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8012830 con 0x7f39bc069e30 2023-10-18T10:41:14.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39b2ffd640 1 -- 172.21.15.125:0/1799270847 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f39b8012a80 con 0x7f39bc069e30 2023-10-18T10:41:14.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc13afa0 con 0x7f39bc069e30 2023-10-18T10:41:14.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39b2ffd640 1 -- 172.21.15.125:0/1799270847 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8012e90 con 0x7f39bc069e30 2023-10-18T10:41:14.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.924+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc13e440 con 0x7f39bc069e30 2023-10-18T10:41:14.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.925+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc0675f0 con 0x7f39bc069e30 2023-10-18T10:41:14.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.925+0000 7f39b2ffd640 1 -- 172.21.15.125:0/1799270847 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f39b801b400 con 0x7f39bc069e30 2023-10-18T10:41:14.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.926+0000 7f39b2ffd640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f39a006af60 0x7f39a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.926+0000 7f39c1a17640 1 -- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f39a006af60 msgr2=0x7f39a006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.926+0000 7f39b2ffd640 1 -- 172.21.15.125:0/1799270847 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f39b8089af0 con 0x7f39bc069e30 2023-10-18T10:41:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.926+0000 7f39c1a17640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f39a006af60 0x7f39a006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:14.930+0000 7f39b2ffd640 1 -- 172.21.15.125:0/1799270847 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39b8058340 con 0x7f39bc069e30 2023-10-18T10:41:14.970 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:14.968+0000 7fe82be43040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:41:15.104 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.102+0000 7f1283245040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:41:15.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.103+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39bc06bf40 con 0x7f39bc069e30 2023-10-18T10:41:15.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.103+0000 7f39b2ffd640 1 -- 172.21.15.125:0/1799270847 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39b8057ed0 con 0x7f39bc069e30 2023-10-18T10:41:15.105 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:41:15.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f39a006af60 msgr2=0x7f39a006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:15.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f39a006af60 0x7f39a006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39bc069e30 msgr2=0x7f39bc13da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39bc069e30 0x7f39bc13da30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c490 tx=0x7f39b800c950 comp rx=0 tx=0).stop 2023-10-18T10:41:15.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 shutdown_connections 2023-10-18T10:41:15.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f39a006af60 0x7f39a006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc13ebe0 0x7f39bc13aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39bc13bd70 0x7f39bc137430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 --2- 172.21.15.125:0/1799270847 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39bc069e30 0x7f39bc13da30 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 >> 172.21.15.125:0/1799270847 conn(0x7f39bc0af730 msgr2=0x7f39bc159460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 shutdown_connections 2023-10-18T10:41:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.105+0000 7f39c447c640 1 -- 172.21.15.125:0/1799270847 wait complete. 2023-10-18T10:41:15.118 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:41:15.152 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.150+0000 7f1283245040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:41:15.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.223+0000 7f929067c640 1 -- 172.21.15.125:0/421239478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f928815b2a0 msgr2=0x7f92880cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.223+0000 7f929067c640 1 --2- 172.21.15.125:0/421239478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f928815b2a0 0x7f92880cf720 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f927c005fd0 tx=0x7f927c02c050 comp rx=0 tx=0).stop 2023-10-18T10:41:15.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.224+0000 7f929067c640 1 -- 172.21.15.125:0/421239478 shutdown_connections 2023-10-18T10:41:15.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.224+0000 7f929067c640 1 --2- 172.21.15.125:0/421239478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f928815b2a0 0x7f92880cf720 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.224+0000 7f929067c640 1 --2- 172.21.15.125:0/421239478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92880cf340 0x7f92880d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.224+0000 7f929067c640 1 -- 172.21.15.125:0/421239478 >> 172.21.15.125:0/421239478 conn(0x7f92880d0d60 msgr2=0x7f92880d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:15.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.224+0000 7f929067c640 1 -- 172.21.15.125:0/421239478 shutdown_connections 2023-10-18T10:41:15.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.224+0000 7f929067c640 1 -- 172.21.15.125:0/421239478 wait complete. 2023-10-18T10:41:15.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.225+0000 7f929067c640 1 Processor -- start 2023-10-18T10:41:15.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.225+0000 7f929067c640 1 -- start start 2023-10-18T10:41:15.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.225+0000 7f929067c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92880ceb40 0x7f92880cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.225+0000 7f929067c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92880cf340 0x7f9288159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.225+0000 7f929067c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f928815b2a0 0x7f92880c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.225+0000 7f929067c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880c8290 con 0x7f92880cf340 2023-10-18T10:41:15.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.225+0000 7f929067c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880c90b0 con 0x7f928815b2a0 2023-10-18T10:41:15.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.225+0000 7f929067c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92880cb120 con 0x7f92880ceb40 2023-10-18T10:41:15.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928dc17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92880cf340 0x7f9288159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928e418640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92880ceb40 0x7f92880cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928dc17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92880cf340 0x7f9288159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33440/0 (socket says 172.21.15.125:33440) 2023-10-18T10:41:15.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928dc17640 1 -- 172.21.15.125:0/3554798104 learned_addr learned my addr 172.21.15.125:0/3554798104 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:15.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928ec19640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f928815b2a0 0x7f92880c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928e418640 1 -- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f928815b2a0 msgr2=0x7f92880c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928e418640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f928815b2a0 0x7f92880c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928e418640 1 -- 172.21.15.125:0/3554798104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92880cf340 msgr2=0x7f9288159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928e418640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92880cf340 0x7f9288159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928e418640 1 -- 172.21.15.125:0/3554798104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c009840 con 0x7f92880ceb40 2023-10-18T10:41:15.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928dc17640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92880cf340 0x7f9288159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:41:15.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f928e418640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92880ceb40 0x7f92880cabe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f928000e9f0 tx=0x7f928000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:15.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928000ccc0 con 0x7f92880ceb40 2023-10-18T10:41:15.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9280010430 con 0x7f92880ceb40 2023-10-18T10:41:15.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92880c84a0 con 0x7f92880ceb40 2023-10-18T10:41:15.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280010820 con 0x7f92880ceb40 2023-10-18T10:41:15.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.226+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92880c8a60 con 0x7f92880ceb40 2023-10-18T10:41:15.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.227+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92880690e0 con 0x7f92880ceb40 2023-10-18T10:41:15.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.228+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 45) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f9280010980 con 0x7f92880ceb40 2023-10-18T10:41:15.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.228+0000 7f9286ffd640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f927406af60 0x7f927406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.228+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f9280014070 con 0x7f92880ceb40 2023-10-18T10:41:15.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.228+0000 7f928dc17640 1 -- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f927406af60 msgr2=0x7f927406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1652676135 2023-10-18T10:41:15.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.228+0000 7f928dc17640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f927406af60 0x7f927406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:15.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.233+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9280051ca0 con 0x7f92880ceb40 2023-10-18T10:41:15.246 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:15.244+0000 7fe82be43040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:41:15.251 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.250+0000 7f1283245040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:41:15.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.271+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mgrmap(e 46) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f9280088830 con 0x7f92880ceb40 2023-10-18T10:41:15.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.271+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f927406af60 msgr2=0x7f927406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:15.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.271+0000 7f9286ffd640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f927406af60 0x7f927406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.300 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:15.298+0000 7fe82be43040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:41:15.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.325+0000 7f12055a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.325+0000 7f12055a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.325+0000 7f12055a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.325+0000 7f12055a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.325+0000 7f12055a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.325+0000 7f12085a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.326+0000 7f12085a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.326+0000 7f12085a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.326+0000 7f12085a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:15.326+0000 7f12085a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:15.402 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:15.400+0000 7fe82be43040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:41:15.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.409+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92880d0b40 con 0x7f92880ceb40 2023-10-18T10:41:15.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.410+0000 7f9286ffd640 1 -- 172.21.15.125:0/3554798104 <== mon.2 v2:172.21.15.139:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9280084e80 con 0x7f92880ceb40 2023-10-18T10:41:15.412 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:41:15.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92880ceb40 msgr2=0x7f92880cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92880ceb40 0x7f92880cabe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f928000e9f0 tx=0x7f928000eeb0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 shutdown_connections 2023-10-18T10:41:15.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:6832/1652676135,v1:172.21.15.139:6833/1652676135] conn(0x7f927406af60 0x7f927406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f928815b2a0 0x7f92880c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92880cf340 0x7f9288159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 --2- 172.21.15.125:0/3554798104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92880ceb40 0x7f92880cabe0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 >> 172.21.15.125:0/3554798104 conn(0x7f92880d0d60 msgr2=0x7f9288159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:15.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 shutdown_connections 2023-10-18T10:41:15.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.412+0000 7f929067c640 1 -- 172.21.15.125:0/3554798104 wait complete. 2023-10-18T10:41:15.425 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:41:15.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.524+0000 7f7f39464640 1 -- 172.21.15.125:0/4009736182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 msgr2=0x7f7f340cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.524+0000 7f7f39464640 1 --2- 172.21.15.125:0/4009736182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 0x7f7f340cde70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c005fd0 tx=0x7f7f2c02c240 comp rx=0 tx=0).stop 2023-10-18T10:41:15.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.525+0000 7f7f39464640 1 -- 172.21.15.125:0/4009736182 shutdown_connections 2023-10-18T10:41:15.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.525+0000 7f7f39464640 1 --2- 172.21.15.125:0/4009736182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 0x7f7f340cde70 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.525+0000 7f7f39464640 1 --2- 172.21.15.125:0/4009736182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f34158910 0x7f7f34158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.525+0000 7f7f39464640 1 -- 172.21.15.125:0/4009736182 >> 172.21.15.125:0/4009736182 conn(0x7f7f340af7a0 msgr2=0x7f7f340afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:15.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.525+0000 7f7f39464640 1 -- 172.21.15.125:0/4009736182 shutdown_connections 2023-10-18T10:41:15.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.525+0000 7f7f39464640 1 -- 172.21.15.125:0/4009736182 wait complete. 2023-10-18T10:41:15.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f39464640 1 Processor -- start 2023-10-18T10:41:15.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f39464640 1 -- start start 2023-10-18T10:41:15.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f39464640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f340cd1c0 0x7f7f341f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f39464640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 0x7f7f340acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f39464640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f34158910 0x7f7f34156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f39464640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3414a7f0 con 0x7f7f340cd9e0 2023-10-18T10:41:15.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f39464640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3414aa40 con 0x7f7f340cd1c0 2023-10-18T10:41:15.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f39464640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3414ad20 con 0x7f7f34158910 2023-10-18T10:41:15.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.526+0000 7f7f2bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 0x7f7f340acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 0x7f7f340acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33458/0 (socket says 172.21.15.125:33458) 2023-10-18T10:41:15.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2bfff640 1 -- 172.21.15.125:0/3639493244 learned_addr learned my addr 172.21.15.125:0/3639493244 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:15.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2bfff640 1 -- 172.21.15.125:0/3639493244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f34158910 msgr2=0x7f7f34156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f32ffd640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f340cd1c0 0x7f7f341f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f337fe640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f34158910 0x7f7f34156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2bfff640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f34158910 0x7f7f34156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2bfff640 1 -- 172.21.15.125:0/3639493244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f340cd1c0 msgr2=0x7f7f341f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2bfff640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f340cd1c0 0x7f7f341f0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2bfff640 1 -- 172.21.15.125:0/3639493244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c009900 con 0x7f7f340cd9e0 2023-10-18T10:41:15.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2bfff640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 0x7f7f340acca0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00c410 tx=0x7f7f1c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:15.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2b7fe640 1 -- 172.21.15.125:0/3639493244 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0035f0 con 0x7f7f340cd9e0 2023-10-18T10:41:15.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2b7fe640 1 -- 172.21.15.125:0/3639493244 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7f1c003750 con 0x7f7f340cd9e0 2023-10-18T10:41:15.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f2b7fe640 1 -- 172.21.15.125:0/3639493244 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c003ba0 con 0x7f7f340cd9e0 2023-10-18T10:41:15.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3414b000 con 0x7f7f340cd9e0 2023-10-18T10:41:15.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.527+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3414b5c0 con 0x7f7f340cd9e0 2023-10-18T10:41:15.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.528+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f340690e0 con 0x7f7f340cd9e0 2023-10-18T10:41:15.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.529+0000 7f7f2b7fe640 1 -- 172.21.15.125:0/3639493244 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 46) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f7f1c003d00 con 0x7f7f340cd9e0 2023-10-18T10:41:15.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.529+0000 7f7f2b7fe640 1 -- 172.21.15.125:0/3639493244 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f7f1c088820 con 0x7f7f340cd9e0 2023-10-18T10:41:15.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.533+0000 7f7f2b7fe640 1 -- 172.21.15.125:0/3639493244 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f1c053930 con 0x7f7f340cd9e0 2023-10-18T10:41:15.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.733+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f340ce5e0 con 0x7f7f340cd9e0 2023-10-18T10:41:15.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.734+0000 7f7f2b7fe640 1 -- 172.21.15.125:0/3639493244 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f7f1c056f50 con 0x7f7f340cd9e0 2023-10-18T10:41:15.736 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:15.736 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3531064821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3531064821},{"type":"v1","addr":"172.21.15.139:6837","nonce":3531064821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2850027191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2850027191},{"type":"v1","addr":"172.21.15.139:6836","nonce":2850027191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7578,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3670537575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3670537575},{"type":"v1","addr":"172.21.15.125:6834","nonce":3670537575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:53.749412+0000","modified":"2023-10-18T10:40:55.750936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7581":{"gid":7581,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1041197980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1041197980},{"type":"v1","addr":"172.21.15.125:6835","nonce":1041197980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T10:41:15.737 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 35 2023-10-18T10:41:15.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 msgr2=0x7f7f340acca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 0x7f7f340acca0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00c410 tx=0x7f7f1c00c8d0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 shutdown_connections 2023-10-18T10:41:15.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f34158910 0x7f7f34156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f340cd9e0 0x7f7f340acca0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 --2- 172.21.15.125:0/3639493244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f340cd1c0 0x7f7f341f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 >> 172.21.15.125:0/3639493244 conn(0x7f7f340af7a0 msgr2=0x7f7f3415ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:15.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 shutdown_connections 2023-10-18T10:41:15.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.736+0000 7f7f39464640 1 -- 172.21.15.125:0/3639493244 wait complete. 2023-10-18T10:41:15.748 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:15.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 -- 172.21.15.125:0/343175456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939006c0a0 msgr2=0x7f9390070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 --2- 172.21.15.125:0/343175456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939006c0a0 0x7f9390070920 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f939006a780 tx=0x7f938802c140 comp rx=0 tx=0).stop 2023-10-18T10:41:15.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 -- 172.21.15.125:0/343175456 shutdown_connections 2023-10-18T10:41:15.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 --2- 172.21.15.125:0/343175456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939006c0a0 0x7f9390070920 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 --2- 172.21.15.125:0/343175456 >> v2:172.21.15.139:3300/0 conn(0x7f9390141690 0x7f93900703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 --2- 172.21.15.125:0/343175456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f939013e520 0x7f939006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 -- 172.21.15.125:0/343175456 >> 172.21.15.125:0/343175456 conn(0x7f9390069930 msgr2=0x7f9390069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:15.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 -- 172.21.15.125:0/343175456 shutdown_connections 2023-10-18T10:41:15.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.850+0000 7f9397ac2640 1 -- 172.21.15.125:0/343175456 wait complete. 2023-10-18T10:41:15.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f9397ac2640 1 Processor -- start 2023-10-18T10:41:15.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f9397ac2640 1 -- start start 2023-10-18T10:41:15.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f9397ac2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939006c0a0 0x7f93900aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f9397ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939013e520 0x7f93900ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f9397ac2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9390141690 0x7f939013b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:15.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f9397ac2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93901e22b0 con 0x7f939013e520 2023-10-18T10:41:15.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f9397ac2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900ab820 con 0x7f939006c0a0 2023-10-18T10:41:15.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f9397ac2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93901e2dc0 con 0x7f9390141690 2023-10-18T10:41:15.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f939505d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939013e520 0x7f93900ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f939505d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939013e520 0x7f93900ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33484/0 (socket says 172.21.15.125:33484) 2023-10-18T10:41:15.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.852+0000 7f939505d640 1 -- 172.21.15.125:0/954028688 learned_addr learned my addr 172.21.15.125:0/954028688 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:15.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f939605f640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9390141690 0x7f939013b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f939585e640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939006c0a0 0x7f93900aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:15.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f939505d640 1 -- 172.21.15.125:0/954028688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9390141690 msgr2=0x7f939013b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f939505d640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9390141690 0x7f939013b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f939505d640 1 -- 172.21.15.125:0/954028688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939006c0a0 msgr2=0x7f93900aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:15.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f939505d640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939006c0a0 0x7f93900aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:15.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f939505d640 1 -- 172.21.15.125:0/954028688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388009900 con 0x7f939013e520 2023-10-18T10:41:15.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f939505d640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939013e520 0x7f93900ab2e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f938c00d750 tx=0x7f938c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:15.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f93867fc640 1 -- 172.21.15.125:0/954028688 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c004220 con 0x7f939013e520 2023-10-18T10:41:15.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f93867fc640 1 -- 172.21.15.125:0/954028688 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f938c004470 con 0x7f939013e520 2023-10-18T10:41:15.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939013b780 con 0x7f939013e520 2023-10-18T10:41:15.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f93867fc640 1 -- 172.21.15.125:0/954028688 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c009c30 con 0x7f939013e520 2023-10-18T10:41:15.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.853+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93901e2770 con 0x7f939013e520 2023-10-18T10:41:15.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.854+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93900690e0 con 0x7f939013e520 2023-10-18T10:41:15.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.854+0000 7f93867fc640 1 -- 172.21.15.125:0/954028688 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 46) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f938c005020 con 0x7f939013e520 2023-10-18T10:41:15.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.855+0000 7f93867fc640 1 -- 172.21.15.125:0/954028688 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f938c0896d0 con 0x7f939013e520 2023-10-18T10:41:15.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:15.859+0000 7f93867fc640 1 -- 172.21.15.125:0/954028688 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f938c054890 con 0x7f939013e520 2023-10-18T10:41:16.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.025+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93901e2f00 con 0x7f939013e520 2023-10-18T10:41:16.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.027+0000 7f93867fc640 1 -- 172.21.15.125:0/954028688 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17839 (secure 0 0 0) 0x7f938c017020 con 0x7f939013e520 2023-10-18T10:41:16.029 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:16.029 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":100,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:15.264487+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:51.563590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:52.731689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/495291025":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/1376126096":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/2074636940":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/905146753":"2023-10-19T10:41:15.264454+0000","172.21.15.139:6832/1652676135":"2023-10-19T10:41:15.264454+0000","172.21.15.139:6833/1652676135":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/383144741":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/3534247944":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/2731662835":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/3976511705":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/1104476358":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2807756531":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6833/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6832/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/264862829":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2537511727":"2023-10-19T10:41:09.823648+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.028+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939013e520 msgr2=0x7f93900ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.028+0000 7f9397ac2640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939013e520 0x7f93900ab2e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f938c00d750 tx=0x7f938c00dc10 comp rx=0 tx=0).stop 2023-10-18T10:41:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.029+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 shutdown_connections 2023-10-18T10:41:16.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.029+0000 7f9397ac2640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9390141690 0x7f939013b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.029+0000 7f9397ac2640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f939013e520 0x7f93900ab2e0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.029+0000 7f9397ac2640 1 --2- 172.21.15.125:0/954028688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939006c0a0 0x7f93900aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.029+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 >> 172.21.15.125:0/954028688 conn(0x7f9390069930 msgr2=0x7f93900ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.029+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 shutdown_connections 2023-10-18T10:41:16.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.029+0000 7f9397ac2640 1 -- 172.21.15.125:0/954028688 wait complete. 2023-10-18T10:41:16.041 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:41:16.042 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:41:16.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.142+0000 7ffa90ad7640 1 -- 172.21.15.125:0/3580063237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 msgr2=0x7ffa74003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.142+0000 7ffa90ad7640 1 --2- 172.21.15.125:0/3580063237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 0x7ffa74003d80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c640 tx=0x7ffa740057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.142+0000 7ffa8ad9d640 1 -- 172.21.15.125:0/3580063237 reap_dead start 2023-10-18T10:41:16.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.142+0000 7ffa90ad7640 1 -- 172.21.15.125:0/3580063237 shutdown_connections 2023-10-18T10:41:16.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.142+0000 7ffa90ad7640 1 -- 172.21.15.125:0/3580063237 >> 172.21.15.125:0/3580063237 conn(0x7ffa8c069930 msgr2=0x7ffa8c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:16.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.142+0000 7ffa90ad7640 1 -- 172.21.15.125:0/3580063237 shutdown_connections 2023-10-18T10:41:16.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.142+0000 7ffa90ad7640 1 -- 172.21.15.125:0/3580063237 wait complete. 2023-10-18T10:41:16.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.143+0000 7ffa90ad7640 1 Processor -- start 2023-10-18T10:41:16.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.143+0000 7ffa90ad7640 1 -- start start 2023-10-18T10:41:16.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa90ad7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 0x7ffa8c14f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa90ad7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8c136280 0x7ffa8c136700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa90ad7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8c151670 0x7ffa8c151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa90ad7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c151250 con 0x7ffa740039a0 2023-10-18T10:41:16.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa90ad7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c136c40 con 0x7ffa8c151670 2023-10-18T10:41:16.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa90ad7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c15afb0 con 0x7ffa8c136280 2023-10-18T10:41:16.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 0x7ffa8c14f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 0x7ffa8c14f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33502/0 (socket says 172.21.15.125:33502) 2023-10-18T10:41:16.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 -- 172.21.15.125:0/2457045990 learned_addr learned my addr 172.21.15.125:0/2457045990 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:16.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa89d9b640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8c136280 0x7ffa8c136700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8ad9d640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8c151670 0x7ffa8c151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 -- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8c136280 msgr2=0x7ffa8c136700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8c136280 0x7ffa8c136700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 -- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8c151670 msgr2=0x7ffa8c151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8c151670 0x7ffa8c151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 -- 172.21.15.125:0/2457045990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c152030 con 0x7ffa740039a0 2023-10-18T10:41:16.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa89d9b640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8c136280 0x7ffa8c136700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:16.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa8a59c640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 0x7ffa8c14f8b0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02cbe0 tx=0x7ffa7c001840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:16.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.144+0000 7ffa7affd640 1 -- 172.21.15.125:0/2457045990 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c003b70 con 0x7ffa740039a0 2023-10-18T10:41:16.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.145+0000 7ffa7affd640 1 -- 172.21.15.125:0/2457045990 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa7c003cf0 con 0x7ffa740039a0 2023-10-18T10:41:16.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.145+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8c150330 con 0x7ffa740039a0 2023-10-18T10:41:16.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.145+0000 7ffa7affd640 1 -- 172.21.15.125:0/2457045990 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c003ea0 con 0x7ffa740039a0 2023-10-18T10:41:16.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.145+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c150b90 con 0x7ffa740039a0 2023-10-18T10:41:16.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.146+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c0690e0 con 0x7ffa740039a0 2023-10-18T10:41:16.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.146+0000 7ffa7affd640 1 -- 172.21.15.125:0/2457045990 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 46) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ffa7c031030 con 0x7ffa740039a0 2023-10-18T10:41:16.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.146+0000 7ffa7affd640 1 -- 172.21.15.125:0/2457045990 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6717+0+0 (secure 0 0 0) 0x7ffa7c0aa5d0 con 0x7ffa740039a0 2023-10-18T10:41:16.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.151+0000 7ffa7affd640 1 -- 172.21.15.125:0/2457045990 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa7c075780 con 0x7ffa740039a0 2023-10-18T10:41:16.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.273+0000 7ffa7affd640 1 -- 172.21.15.125:0/2457045990 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ffa7c075230 con 0x7ffa740039a0 2023-10-18T10:41:16.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.274+0000 7ffa7affd640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ffa6c070af0 0x7ffa6c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.275+0000 7ffa89d9b640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ffa6c070af0 0x7ffa6c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.276+0000 7ffa89d9b640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ffa6c070af0 0x7ffa6c072ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c1396a0 tx=0x7ffa80005f90 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:16.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.355+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa8c152030 con 0x7ffa740039a0 2023-10-18T10:41:16.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.355+0000 7ffa7affd640 1 -- 172.21.15.125:0/2457045990 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4927 (secure 0 0 0) 0x7ffa8c152030 con 0x7ffa740039a0 2023-10-18T10:41:16.358 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:16.358 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3531064821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3531064821},{"type":"v1","addr":"172.21.15.139:6837","nonce":3531064821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2850027191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2850027191},{"type":"v1","addr":"172.21.15.139:6836","nonce":2850027191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7578,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3670537575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3670537575},{"type":"v1","addr":"172.21.15.125:6834","nonce":3670537575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:53.749412+0000","modified":"2023-10-18T10:40:55.750936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7581":{"gid":7581,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1041197980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1041197980},{"type":"v1","addr":"172.21.15.125:6835","nonce":1041197980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T10:41:16.359 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 35 2023-10-18T10:41:16.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ffa6c070af0 msgr2=0x7ffa6c072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ffa6c070af0 0x7ffa6c072ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c1396a0 tx=0x7ffa80005f90 comp rx=0 tx=0).stop 2023-10-18T10:41:16.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 msgr2=0x7ffa8c14f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 0x7ffa8c14f8b0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02cbe0 tx=0x7ffa7c001840 comp rx=0 tx=0).stop 2023-10-18T10:41:16.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 shutdown_connections 2023-10-18T10:41:16.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ffa6c070af0 0x7ffa6c072ee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8c151670 0x7ffa8c151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8c136280 0x7ffa8c136700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 --2- 172.21.15.125:0/2457045990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa740039a0 0x7ffa8c14f8b0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 >> 172.21.15.125:0/2457045990 conn(0x7ffa8c069930 msgr2=0x7ffa8c13b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:16.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 shutdown_connections 2023-10-18T10:41:16.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.357+0000 7ffa90ad7640 1 -- 172.21.15.125:0/2457045990 wait complete. 2023-10-18T10:41:16.370 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:41:16.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.471+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/3291147398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 msgr2=0x7f4e140c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.471+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/3291147398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 0x7f4e140c97c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009830 tx=0x7f4e0402bd60 comp rx=0 tx=0).stop 2023-10-18T10:41:16.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.471+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/3291147398 shutdown_connections 2023-10-18T10:41:16.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.471+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/3291147398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e1406d170 0x7f4e140c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.471+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/3291147398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 0x7f4e140c97c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.471+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/3291147398 >> 172.21.15.125:0/3291147398 conn(0x7f4e14069930 msgr2=0x7f4e14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:16.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.471+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/3291147398 shutdown_connections 2023-10-18T10:41:16.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.471+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/3291147398 wait complete. 2023-10-18T10:41:16.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.472+0000 7f4e1bfc0640 1 Processor -- start 2023-10-18T10:41:16.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.472+0000 7f4e1bfc0640 1 -- start start 2023-10-18T10:41:16.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1bfc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e1406d170 0x7f4e140b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1bfc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e1413d360 0x7f4e140af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1bfc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 0x7f4e140afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1bfc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140acdf0 con 0x7f4e14140660 2023-10-18T10:41:16.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1bfc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1413e8e0 con 0x7f4e1413d360 2023-10-18T10:41:16.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1bfc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140b0bf0 con 0x7f4e1406d170 2023-10-18T10:41:16.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 0x7f4e140afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 0x7f4e140afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33522/0 (socket says 172.21.15.125:33522) 2023-10-18T10:41:16.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 -- 172.21.15.125:0/74547675 learned_addr learned my addr 172.21.15.125:0/74547675 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:16.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 -- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e1406d170 msgr2=0x7f4e140b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e1406d170 0x7f4e140b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 -- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e1413d360 msgr2=0x7f4e140af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e1413d360 0x7f4e140af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 -- 172.21.15.125:0/74547675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04009490 con 0x7f4e14140660 2023-10-18T10:41:16.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1a55d640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 0x7f4e140afa70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00d750 tx=0x7f4e0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:16.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e0a7fc640 1 -- 172.21.15.125:0/74547675 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c009bd0 con 0x7f4e14140660 2023-10-18T10:41:16.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e0a7fc640 1 -- 172.21.15.125:0/74547675 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e0c009e40 con 0x7f4e14140660 2023-10-18T10:41:16.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e140ad040 con 0x7f4e14140660 2023-10-18T10:41:16.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.473+0000 7f4e0a7fc640 1 -- 172.21.15.125:0/74547675 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c005280 con 0x7f4e14140660 2023-10-18T10:41:16.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.474+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e140ad600 con 0x7f4e14140660 2023-10-18T10:41:16.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.475+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e140ad7f0 con 0x7f4e14140660 2023-10-18T10:41:16.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.475+0000 7f4e0a7fc640 1 -- 172.21.15.125:0/74547675 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4e0c0053e0 con 0x7f4e14140660 2023-10-18T10:41:16.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.475+0000 7f4e0a7fc640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4df806ac90 0x7f4df806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.476+0000 7f4e0a7fc640 1 -- 172.21.15.125:0/74547675 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f4e0c089830 con 0x7f4e14140660 2023-10-18T10:41:16.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.476+0000 7f4e19d5c640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4df806ac90 0x7f4df806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.476+0000 7f4e19d5c640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4df806ac90 0x7f4df806d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4e10004a30 tx=0x7f4e100016d0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:16.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.480+0000 7f4e0a7fc640 1 -- 172.21.15.125:0/74547675 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e0c054830 con 0x7f4e14140660 2023-10-18T10:41:16.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.679+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e140690e0 con 0x7f4e14140660 2023-10-18T10:41:16.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.680+0000 7f4e0a7fc640 1 -- 172.21.15.125:0/74547675 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f4e0c017090 con 0x7f4e14140660 2023-10-18T10:41:16.682 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:16.682 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3531064821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3531064821},{"type":"v1","addr":"172.21.15.139:6837","nonce":3531064821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2850027191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2850027191},{"type":"v1","addr":"172.21.15.139:6836","nonce":2850027191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7578,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3670537575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3670537575},{"type":"v1","addr":"172.21.15.125:6834","nonce":3670537575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:40:53.749412+0000","modified":"2023-10-18T10:40:55.750936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7581":{"gid":7581,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1041197980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1041197980},{"type":"v1","addr":"172.21.15.125:6835","nonce":1041197980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T10:41:16.683 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 35 2023-10-18T10:41:16.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4df806ac90 msgr2=0x7f4df806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4df806ac90 0x7f4df806d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4e10004a30 tx=0x7f4e100016d0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 msgr2=0x7f4e140afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 0x7f4e140afa70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00d750 tx=0x7f4e0c00dc10 comp rx=0 tx=0).stop 2023-10-18T10:41:16.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 shutdown_connections 2023-10-18T10:41:16.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4df806ac90 0x7f4df806d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e1413d360 0x7f4e140af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e14140660 0x7f4e140afa70 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 --2- 172.21.15.125:0/74547675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e1406d170 0x7f4e140b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 >> 172.21.15.125:0/74547675 conn(0x7f4e14069930 msgr2=0x7f4e1413f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:16.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 shutdown_connections 2023-10-18T10:41:16.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.682+0000 7f4e1bfc0640 1 -- 172.21.15.125:0/74547675 wait complete. 2023-10-18T10:41:16.694 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:16.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.795+0000 7feec3e56640 1 -- 172.21.15.125:0/3105292371 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0afa20 msgr2=0x7feebc158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.795+0000 7feec3e56640 1 --2- 172.21.15.125:0/3105292371 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0afa20 0x7feebc158300 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7feebc0b10f0 tx=0x7feeac02bdb0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.795+0000 7feec3e56640 1 -- 172.21.15.125:0/3105292371 shutdown_connections 2023-10-18T10:41:16.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.795+0000 7feec3e56640 1 --2- 172.21.15.125:0/3105292371 >> v2:172.21.15.139:3300/0 conn(0x7feebc0b0530 0x7feebc0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.795+0000 7feec3e56640 1 --2- 172.21.15.125:0/3105292371 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0afa20 0x7feebc158300 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.795+0000 7feec3e56640 1 -- 172.21.15.125:0/3105292371 >> 172.21.15.125:0/3105292371 conn(0x7feebc069930 msgr2=0x7feebc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:16.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.795+0000 7feec3e56640 1 -- 172.21.15.125:0/3105292371 shutdown_connections 2023-10-18T10:41:16.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.795+0000 7feec3e56640 1 -- 172.21.15.125:0/3105292371 wait complete. 2023-10-18T10:41:16.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.796+0000 7feec3e56640 1 Processor -- start 2023-10-18T10:41:16.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.796+0000 7feec3e56640 1 -- start start 2023-10-18T10:41:16.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.796+0000 7feec3e56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc0afa20 0x7feebc143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.796+0000 7feec3e56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0b0530 0x7feebc1409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.796+0000 7feec3e56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0cbd20 0x7feebc140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.796+0000 7feec3e56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc144050 con 0x7feebc0b0530 2023-10-18T10:41:16.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.796+0000 7feec3e56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc144330 con 0x7feebc0afa20 2023-10-18T10:41:16.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.796+0000 7feec3e56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc144610 con 0x7feebc0cbd20 2023-10-18T10:41:16.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0b0530 0x7feebc1409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0b0530 0x7feebc1409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33538/0 (socket says 172.21.15.125:33538) 2023-10-18T10:41:16.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 -- 172.21.15.125:0/1977315211 learned_addr learned my addr 172.21.15.125:0/1977315211 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:16.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec23f3640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0cbd20 0x7feebc140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec1bf2640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc0afa20 0x7feebc143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 -- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0cbd20 msgr2=0x7feebc140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0cbd20 0x7feebc140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 -- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc0afa20 msgr2=0x7feebc143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc0afa20 0x7feebc143700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 -- 172.21.15.125:0/1977315211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac009560 con 0x7feebc0b0530 2023-10-18T10:41:16.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec23f3640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0cbd20 0x7feebc140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:16.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec1bf2640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc0afa20 0x7feebc143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:16.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec13f1640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0b0530 0x7feebc1409b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7feeac036040 tx=0x7feeac02dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:16.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feeb27fc640 1 -- 172.21.15.125:0/1977315211 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac005d50 con 0x7feebc0b0530 2023-10-18T10:41:16.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feeb27fc640 1 -- 172.21.15.125:0/1977315211 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feeac003910 con 0x7feebc0b0530 2023-10-18T10:41:16.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feeb27fc640 1 -- 172.21.15.125:0/1977315211 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac003d20 con 0x7feebc0b0530 2023-10-18T10:41:16.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc1448f0 con 0x7feebc0b0530 2023-10-18T10:41:16.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.797+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc143c40 con 0x7feebc0b0530 2023-10-18T10:41:16.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.798+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc144e10 con 0x7feebc0b0530 2023-10-18T10:41:16.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.799+0000 7feeb27fc640 1 -- 172.21.15.125:0/1977315211 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7feeac03e450 con 0x7feebc0b0530 2023-10-18T10:41:16.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.799+0000 7feeb27fc640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7feea006ac90 0x7feea006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:16.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.799+0000 7feeb27fc640 1 -- 172.21.15.125:0/1977315211 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6717+0+0 (secure 0 0 0) 0x7feeac0ac030 con 0x7feebc0b0530 2023-10-18T10:41:16.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.799+0000 7feec1bf2640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7feea006ac90 0x7feea006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:16.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.800+0000 7feec1bf2640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7feea006ac90 0x7feea006d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7feeb800a830 tx=0x7feeb8008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:16.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.803+0000 7feeb27fc640 1 -- 172.21.15.125:0/1977315211 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feebc144e10 con 0x7feebc0b0530 2023-10-18T10:41:16.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.972+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feebc144e10 con 0x7feebc0b0530 2023-10-18T10:41:16.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.973+0000 7feeb27fc640 1 -- 172.21.15.125:0/1977315211 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17839 (secure 0 0 0) 0x7feebc144e10 con 0x7feebc0b0530 2023-10-18T10:41:16.975 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:16.975 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":100,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:15.264487+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:40:51.563590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-18T10:40:52.731689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/495291025":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/1376126096":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/2074636940":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/905146753":"2023-10-19T10:41:15.264454+0000","172.21.15.139:6832/1652676135":"2023-10-19T10:41:15.264454+0000","172.21.15.139:6833/1652676135":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/383144741":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/3534247944":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/2731662835":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/3976511705":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/1104476358":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2807756531":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6833/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6832/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/264862829":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2537511727":"2023-10-19T10:41:09.823648+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:16.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7feea006ac90 msgr2=0x7feea006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7feea006ac90 0x7feea006d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7feeb800a830 tx=0x7feeb8008040 comp rx=0 tx=0).stop 2023-10-18T10:41:16.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0b0530 msgr2=0x7feebc1409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:16.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0b0530 0x7feebc1409b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7feeac036040 tx=0x7feeac02dd80 comp rx=0 tx=0).stop 2023-10-18T10:41:16.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 shutdown_connections 2023-10-18T10:41:16.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7feea006ac90 0x7feea006d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0cbd20 0x7feebc140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0b0530 0x7feebc1409b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 --2- 172.21.15.125:0/1977315211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc0afa20 0x7feebc143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 >> 172.21.15.125:0/1977315211 conn(0x7feebc069930 msgr2=0x7feebc06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 shutdown_connections 2023-10-18T10:41:16.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:16.975+0000 7feec3e56640 1 -- 172.21.15.125:0/1977315211 wait complete. 2023-10-18T10:41:16.987 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:41:17.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.089+0000 7f8211a00640 1 -- 172.21.15.125:0/2893618187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81f4003f20 msgr2=0x7f81f40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:17.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.089+0000 7f8211a00640 1 --2- 172.21.15.125:0/2893618187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f81f4003f20 0x7f81f40047c0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f8200006fd0 tx=0x7f81f40057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.089+0000 7f820b7fe640 1 -- 172.21.15.125:0/2893618187 reap_dead start 2023-10-18T10:41:17.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.089+0000 7f8211a00640 1 -- 172.21.15.125:0/2893618187 shutdown_connections 2023-10-18T10:41:17.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.089+0000 7f8211a00640 1 -- 172.21.15.125:0/2893618187 >> 172.21.15.125:0/2893618187 conn(0x7f820c0af7a0 msgr2=0x7f820c0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:17.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.089+0000 7f8211a00640 1 -- 172.21.15.125:0/2893618187 shutdown_connections 2023-10-18T10:41:17.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.089+0000 7f8211a00640 1 -- 172.21.15.125:0/2893618187 wait complete. 2023-10-18T10:41:17.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.091+0000 7f8211a00640 1 Processor -- start 2023-10-18T10:41:17.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.091+0000 7f8211a00640 1 -- start start 2023-10-18T10:41:17.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.091+0000 7f8211a00640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81f4003f20 0x7f820c0702a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:17.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.091+0000 7f8211a00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c156760 0x7f820c14a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:17.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.091+0000 7f8211a00640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f820c156be0 0x7f820c14a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:17.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.091+0000 7f8211a00640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c156340 con 0x7f820c156be0 2023-10-18T10:41:17.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.091+0000 7f8211a00640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c157c90 con 0x7f820c156760 2023-10-18T10:41:17.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.091+0000 7f8211a00640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c1556c0 con 0x7f81f4003f20 2023-10-18T10:41:17.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f820c156be0 0x7f820c14a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:17.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c156760 0x7f820c14a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:17.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81f4003f20 0x7f820c0702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:17.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f820c156be0 0x7f820c14a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33552/0 (socket says 172.21.15.125:33552) 2023-10-18T10:41:17.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 -- 172.21.15.125:0/1020315908 learned_addr learned my addr 172.21.15.125:0/1020315908 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:17.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 -- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81f4003f20 msgr2=0x7f820c0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:17.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81f4003f20 0x7f820c0702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 -- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c156760 msgr2=0x7f820c14a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:17.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c156760 0x7f820c14a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 -- 172.21.15.125:0/1020315908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c1559a0 con 0x7f820c156be0 2023-10-18T10:41:17.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f820b7fe640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f820c156be0 0x7f820c14a6e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f820c071f00 tx=0x7f8204001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:17.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f81f3fff640 1 -- 172.21.15.125:0/1020315908 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204017030 con 0x7f820c156be0 2023-10-18T10:41:17.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f81f3fff640 1 -- 172.21.15.125:0/1020315908 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8204011630 con 0x7f820c156be0 2023-10-18T10:41:17.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f81f3fff640 1 -- 172.21.15.125:0/1020315908 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204011a40 con 0x7f820c156be0 2023-10-18T10:41:17.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c155c80 con 0x7f820c156be0 2023-10-18T10:41:17.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.092+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c0707e0 con 0x7f820c156be0 2023-10-18T10:41:17.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.093+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c156480 con 0x7f820c156be0 2023-10-18T10:41:17.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.094+0000 7f81f3fff640 1 -- 172.21.15.125:0/1020315908 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f820401a070 con 0x7f820c156be0 2023-10-18T10:41:17.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.094+0000 7f81f3fff640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f81f806ac90 0x7f81f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:17.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.094+0000 7f81f3fff640 1 -- 172.21.15.125:0/1020315908 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6717+0+0 (secure 0 0 0) 0x7f8204018070 con 0x7f820c156be0 2023-10-18T10:41:17.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.094+0000 7f820affd640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f81f806ac90 0x7f81f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:17.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.095+0000 7f820affd640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f81f806ac90 0x7f81f806d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005fd0 tx=0x7f81fc009500 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:17.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.098+0000 7f81f3fff640 1 -- 172.21.15.125:0/1020315908 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f820c156480 con 0x7f820c156be0 2023-10-18T10:41:17.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.299+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f820c156480 con 0x7f820c156be0 2023-10-18T10:41:17.302 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T10:41:17.300+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:41:17.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.303+0000 7f81f3fff640 1 -- 172.21.15.125:0/1020315908 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f820c156480 con 0x7f820c156be0 2023-10-18T10:41:17.305 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:41:17.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f81f806ac90 msgr2=0x7f81f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:17.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f81f806ac90 0x7f81f806d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005fd0 tx=0x7f81fc009500 comp rx=0 tx=0).stop 2023-10-18T10:41:17.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f820c156be0 msgr2=0x7f820c14a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:17.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f820c156be0 0x7f820c14a6e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f820c071f00 tx=0x7f8204001630 comp rx=0 tx=0).stop 2023-10-18T10:41:17.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 shutdown_connections 2023-10-18T10:41:17.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f81f806ac90 0x7f81f806d150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c156760 0x7f820c14a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f820c156be0 0x7f820c14a6e0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 --2- 172.21.15.125:0/1020315908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81f4003f20 0x7f820c0702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.305+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 >> 172.21.15.125:0/1020315908 conn(0x7f820c0af7a0 msgr2=0x7f820c14c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.306+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 shutdown_connections 2023-10-18T10:41:17.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.306+0000 7f8211a00640 1 -- 172.21.15.125:0/1020315908 wait complete. 2023-10-18T10:41:17.310 INFO:tasks.ceph.mds.c.smithi125.stderr: -1475> 2023-10-18T10:40:59.392+0000 7f00a3ba1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:41:17.320 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:41:17.333 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:41:17.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.424+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4078994185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2006af00 msgr2=0x7f4d2006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:17.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.424+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4078994185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2006af00 0x7f4d2006b2e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009860 tx=0x7f4d1002bd60 comp rx=0 tx=0).stop 2023-10-18T10:41:17.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.424+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4078994185 shutdown_connections 2023-10-18T10:41:17.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.424+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4078994185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d200abc10 0x7f4d200c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.424+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4078994185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d2006af00 0x7f4d2006b2e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.424+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4078994185 >> 172.21.15.125:0/4078994185 conn(0x7f4d20069930 msgr2=0x7f4d20069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.424+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4078994185 shutdown_connections 2023-10-18T10:41:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.424+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4078994185 wait complete. 2023-10-18T10:41:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.426+0000 7f4d25cf2640 1 Processor -- start 2023-10-18T10:41:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.426+0000 7f4d25cf2640 1 -- start start 2023-10-18T10:41:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d25cf2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2006af00 0x7f4d200a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d25cf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d200abc10 0x7f4d20151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d25cf2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d200c96b0 0x7f4d2015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d25cf2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d200a7a40 con 0x7f4d2006af00 2023-10-18T10:41:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d25cf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20152390 con 0x7f4d200abc10 2023-10-18T10:41:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d25cf2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2012aaf0 con 0x7f4d200c96b0 2023-10-18T10:41:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2006af00 0x7f4d200a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2006af00 0x7f4d200a7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33574/0 (socket says 172.21.15.125:33574) 2023-10-18T10:41:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 -- 172.21.15.125:0/4031645707 learned_addr learned my addr 172.21.15.125:0/4031645707 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:17.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1ffff640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d200c96b0 0x7f4d2015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:17.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d16ffd640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d200abc10 0x7f4d20151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:17.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 -- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d200c96b0 msgr2=0x7f4d2015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:17.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d200c96b0 0x7f4d2015b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 -- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d200abc10 msgr2=0x7f4d20151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d200abc10 0x7f4d20151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 -- 172.21.15.125:0/4031645707 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18009900 con 0x7f4d2006af00 2023-10-18T10:41:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.427+0000 7f4d1f7fe640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2006af00 0x7f4d200a7500 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009780 tx=0x7f4d100038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.428+0000 7f4d1cff9640 1 -- 172.21.15.125:0/4031645707 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10003ea0 con 0x7f4d2006af00 2023-10-18T10:41:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.428+0000 7f4d1cff9640 1 -- 172.21.15.125:0/4031645707 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d10034440 con 0x7f4d2006af00 2023-10-18T10:41:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.428+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d10009490 con 0x7f4d2006af00 2023-10-18T10:41:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.428+0000 7f4d1cff9640 1 -- 172.21.15.125:0/4031645707 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10034850 con 0x7f4d2006af00 2023-10-18T10:41:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.428+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2012b0b0 con 0x7f4d2006af00 2023-10-18T10:41:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.429+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d200690e0 con 0x7f4d2006af00 2023-10-18T10:41:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.429+0000 7f4d1cff9640 1 -- 172.21.15.125:0/4031645707 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4d1003b040 con 0x7f4d2006af00 2023-10-18T10:41:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.429+0000 7f4d1cff9640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4d0006ac90 0x7f4d0006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.430+0000 7f4d1cff9640 1 -- 172.21.15.125:0/4031645707 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f4d100ab930 con 0x7f4d2006af00 2023-10-18T10:41:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.430+0000 7f4d16ffd640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4d0006ac90 0x7f4d0006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:17.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.430+0000 7f4d16ffd640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4d0006ac90 0x7f4d0006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d200b1fb0 tx=0x7f4d08009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:17.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.434+0000 7f4d1cff9640 1 -- 172.21.15.125:0/4031645707 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d10076980 con 0x7f4d2006af00 2023-10-18T10:41:17.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:17.638+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4d200cc420 con 0x7f4d2006af00 2023-10-18T10:41:18.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.285+0000 7f4d1cff9640 1 -- 172.21.15.125:0/4031645707 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4d10076d50 con 0x7f4d2006af00 2023-10-18T10:41:18.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.304+0000 7f4d1cff9640 1 -- 172.21.15.125:0/4031645707 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4d10090d30 con 0x7f4d2006af00 2023-10-18T10:41:18.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4d0006ac90 msgr2=0x7f4d0006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:18.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4d0006ac90 0x7f4d0006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d200b1fb0 tx=0x7f4d08009040 comp rx=0 tx=0).stop 2023-10-18T10:41:18.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2006af00 msgr2=0x7f4d200a7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:18.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2006af00 0x7f4d200a7500 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009780 tx=0x7f4d100038a0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 shutdown_connections 2023-10-18T10:41:18.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4d0006ac90 0x7f4d0006d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d200abc10 0x7f4d20151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d200c96b0 0x7f4d2015b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 --2- 172.21.15.125:0/4031645707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d2006af00 0x7f4d200a7500 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 >> 172.21.15.125:0/4031645707 conn(0x7f4d20069930 msgr2=0x7f4d200cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:18.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 shutdown_connections 2023-10-18T10:41:18.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.307+0000 7f4d25cf2640 1 -- 172.21.15.125:0/4031645707 wait complete. 2023-10-18T10:41:18.322 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:41:18.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.472+0000 7f60ba2fc640 1 -- 172.21.15.125:0/2435751867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40ceb40 msgr2=0x7f60b40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:18.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.472+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/2435751867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40ceb40 0x7f60b40b0650 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f60ac005fd0 tx=0x7f60ac02c1b0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.472+0000 7f60ba2fc640 1 -- 172.21.15.125:0/2435751867 shutdown_connections 2023-10-18T10:41:18.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.472+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/2435751867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40ceb40 0x7f60b40b0650 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.472+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/2435751867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60b415b2a0 0x7f60b40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.472+0000 7f60ba2fc640 1 -- 172.21.15.125:0/2435751867 >> 172.21.15.125:0/2435751867 conn(0x7f60b40d0d60 msgr2=0x7f60b40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:18.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.472+0000 7f60ba2fc640 1 -- 172.21.15.125:0/2435751867 shutdown_connections 2023-10-18T10:41:18.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.472+0000 7f60ba2fc640 1 -- 172.21.15.125:0/2435751867 wait complete. 2023-10-18T10:41:18.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.473+0000 7f60ba2fc640 1 Processor -- start 2023-10-18T10:41:18.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.473+0000 7f60ba2fc640 1 -- start start 2023-10-18T10:41:18.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60ba2fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b40ceb40 0x7f60b41571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:18.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60ba2fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40cf340 0x7f60b41e5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:18.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60ba2fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60b415b2a0 0x7f60b40b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:18.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60ba2fc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40b19b0 con 0x7f60b40cf340 2023-10-18T10:41:18.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60ba2fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b41e5a20 con 0x7f60b40ceb40 2023-10-18T10:41:18.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60ba2fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b41e5d00 con 0x7f60b415b2a0 2023-10-18T10:41:18.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40cf340 0x7f60b41e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:18.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40cf340 0x7f60b41e5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45014/0 (socket says 172.21.15.125:45014) 2023-10-18T10:41:18.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b37fe640 1 -- 172.21.15.125:0/4262481789 learned_addr learned my addr 172.21.15.125:0/4262481789 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:18.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b8899640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60b415b2a0 0x7f60b40b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:18.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b3fff640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b40ceb40 0x7f60b41571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:18.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b37fe640 1 -- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60b415b2a0 msgr2=0x7f60b40b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:18.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b37fe640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60b415b2a0 0x7f60b40b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b37fe640 1 -- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b40ceb40 msgr2=0x7f60b41571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:18.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.474+0000 7f60b37fe640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b40ceb40 0x7f60b41571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:18.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.475+0000 7f60b37fe640 1 -- 172.21.15.125:0/4262481789 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac009900 con 0x7f60b40cf340 2023-10-18T10:41:18.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.475+0000 7f60b37fe640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40cf340 0x7f60b41e5340 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f60a800b410 tx=0x7f60a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:18.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.475+0000 7f60b0ff9640 1 -- 172.21.15.125:0/4262481789 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8011830 con 0x7f60b40cf340 2023-10-18T10:41:18.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.475+0000 7f60b0ff9640 1 -- 172.21.15.125:0/4262481789 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f60a8011aa0 con 0x7f60b40cf340 2023-10-18T10:41:18.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.475+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b41e5fe0 con 0x7f60b40cf340 2023-10-18T10:41:18.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.475+0000 7f60b0ff9640 1 -- 172.21.15.125:0/4262481789 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8010380 con 0x7f60b40cf340 2023-10-18T10:41:18.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.475+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b40c9e40 con 0x7f60b40cf340 2023-10-18T10:41:18.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.476+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b41e6500 con 0x7f60b40cf340 2023-10-18T10:41:18.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.476+0000 7f60b0ff9640 1 -- 172.21.15.125:0/4262481789 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f60a80104e0 con 0x7f60b40cf340 2023-10-18T10:41:18.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.477+0000 7f60b0ff9640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f60a006aed0 0x7f60a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:18.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.477+0000 7f60b0ff9640 1 -- 172.21.15.125:0/4262481789 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f60a8089d60 con 0x7f60b40cf340 2023-10-18T10:41:18.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.477+0000 7f60b3fff640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f60a006aed0 0x7f60a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:18.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.477+0000 7f60b3fff640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f60a006aed0 0x7f60a006d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f60a4004ea0 tx=0x7f60a4009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:18.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.481+0000 7f60b0ff9640 1 -- 172.21.15.125:0/4262481789 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60b41e6500 con 0x7f60b40cf340 2023-10-18T10:41:18.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:18.648+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60b41e6500 con 0x7f60b40cf340 2023-10-18T10:41:19.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.381+0000 7f60b0ff9640 1 -- 172.21.15.125:0/4262481789 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7f60b41e6500 con 0x7f60b40cf340 2023-10-18T10:41:19.383 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:41:19.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.384+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f60a006aed0 msgr2=0x7f60a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:19.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.384+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f60a006aed0 0x7f60a006d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f60a4004ea0 tx=0x7f60a4009040 comp rx=0 tx=0).stop 2023-10-18T10:41:19.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40cf340 msgr2=0x7f60b41e5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:19.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40cf340 0x7f60b41e5340 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f60a800b410 tx=0x7f60a800b8d0 comp rx=0 tx=0).stop 2023-10-18T10:41:19.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 shutdown_connections 2023-10-18T10:41:19.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f60a006aed0 0x7f60a006d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:19.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60b415b2a0 0x7f60b40b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:19.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60b40cf340 0x7f60b41e5340 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:19.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 --2- 172.21.15.125:0/4262481789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b40ceb40 0x7f60b41571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:19.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 >> 172.21.15.125:0/4262481789 conn(0x7f60b40d0d60 msgr2=0x7f60b415a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:19.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 shutdown_connections 2023-10-18T10:41:19.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.385+0000 7f60ba2fc640 1 -- 172.21.15.125:0/4262481789 wait complete. 2023-10-18T10:41:19.400 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:41:19.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.511+0000 7fd2ad294640 1 -- 172.21.15.125:0/368195881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd290004d00 msgr2=0x7fd290005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:19.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.511+0000 7fd2ad294640 1 --2- 172.21.15.125:0/368195881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd290004d00 0x7fd290005180 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b50 tx=0x7fd2900057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:19.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.511+0000 7fd2a77fe640 1 -- 172.21.15.125:0/368195881 reap_dead start 2023-10-18T10:41:19.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.511+0000 7fd2ad294640 1 -- 172.21.15.125:0/368195881 shutdown_connections 2023-10-18T10:41:19.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.511+0000 7fd2ad294640 1 -- 172.21.15.125:0/368195881 >> 172.21.15.125:0/368195881 conn(0x7fd2a809a030 msgr2=0x7fd2a80d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:19.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.511+0000 7fd2ad294640 1 -- 172.21.15.125:0/368195881 shutdown_connections 2023-10-18T10:41:19.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.511+0000 7fd2ad294640 1 -- 172.21.15.125:0/368195881 wait complete. 2023-10-18T10:41:19.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.512+0000 7fd2ad294640 1 Processor -- start 2023-10-18T10:41:19.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.512+0000 7fd2ad294640 1 -- start start 2023-10-18T10:41:19.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2ad294640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd290004d00 0x7fd2a815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:19.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2ad294640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2a815d150 0x7fd2a80c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:19.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2ad294640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2a81f11b0 0x7fd2a80c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:19.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2ad294640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80c9690 con 0x7fd2a815d150 2023-10-18T10:41:19.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2ad294640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a815c0b0 con 0x7fd290004d00 2023-10-18T10:41:19.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2ad294640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a815c390 con 0x7fd2a81f11b0 2023-10-18T10:41:19.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2a815d150 0x7fd2a80c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:19.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2a815d150 0x7fd2a80c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45034/0 (socket says 172.21.15.125:45034) 2023-10-18T10:41:19.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 -- 172.21.15.125:0/3828272870 learned_addr learned my addr 172.21.15.125:0/3828272870 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:19.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a6ffd640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd290004d00 0x7fd2a815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:19.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a77fe640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2a81f11b0 0x7fd2a80c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:19.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 -- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2a81f11b0 msgr2=0x7fd2a80c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:19.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2a81f11b0 0x7fd2a80c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:19.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 -- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd290004d00 msgr2=0x7fd2a815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:19.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd290004d00 0x7fd2a815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:19.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 -- 172.21.15.125:0/3828272870 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a815c670 con 0x7fd2a815d150 2023-10-18T10:41:19.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a6ffd640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd290004d00 0x7fd2a815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:19.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2a67fc640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2a815d150 0x7fd2a80c9150 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00e700 tx=0x7fd29c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:19.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd28f7fe640 1 -- 172.21.15.125:0/3828272870 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c004460 con 0x7fd2a815d150 2023-10-18T10:41:19.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd28f7fe640 1 -- 172.21.15.125:0/3828272870 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd29c00c840 con 0x7fd2a815d150 2023-10-18T10:41:19.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a815c950 con 0x7fd2a815d150 2023-10-18T10:41:19.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd28f7fe640 1 -- 172.21.15.125:0/3828272870 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00cc50 con 0x7fd2a815d150 2023-10-18T10:41:19.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.513+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a81f1630 con 0x7fd2a815d150 2023-10-18T10:41:19.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.514+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a80675f0 con 0x7fd2a815d150 2023-10-18T10:41:19.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.515+0000 7fd28f7fe640 1 -- 172.21.15.125:0/3828272870 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 48) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd29c0035f0 con 0x7fd2a815d150 2023-10-18T10:41:19.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.515+0000 7fd28f7fe640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fd29406af00 0x7fd29406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:19.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.515+0000 7fd28f7fe640 1 -- 172.21.15.125:0/3828272870 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6365+0+0 (secure 0 0 0) 0x7fd29c08a4a0 con 0x7fd2a815d150 2023-10-18T10:41:19.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.515+0000 7fd2a6ffd640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fd29406af00 0x7fd29406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:19.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.516+0000 7fd2a6ffd640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fd29406af00 0x7fd29406d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd298005cc0 tx=0x7fd298005c50 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:19.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.519+0000 7fd28f7fe640 1 -- 172.21.15.125:0/3828272870 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd29c055130 con 0x7fd2a815d150 2023-10-18T10:41:19.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.685+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2a806bd40 con 0x7fd2a815d150 2023-10-18T10:41:19.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:19.905+0000 7fd28f7fe640 1 -- 172.21.15.125:0/3828272870 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd29c055500 con 0x7fd2a815d150 2023-10-18T10:41:20.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.383+0000 7fd28f7fe640 1 -- 172.21.15.125:0/3828272870 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd29c072480 con 0x7fd2a815d150 2023-10-18T10:41:20.385 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T10:41:20.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.386+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fd29406af00 msgr2=0x7fd29406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.386+0000 7fd2ad294640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fd29406af00 0x7fd29406d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd298005cc0 tx=0x7fd298005c50 comp rx=0 tx=0).stop 2023-10-18T10:41:20.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.386+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2a815d150 msgr2=0x7fd2a80c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.386+0000 7fd2ad294640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2a815d150 0x7fd2a80c9150 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00e700 tx=0x7fd29c00ebc0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.386+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 shutdown_connections 2023-10-18T10:41:20.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.387+0000 7fd2ad294640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fd29406af00 0x7fd29406d3c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.387+0000 7fd2ad294640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2a81f11b0 0x7fd2a80c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.387+0000 7fd2ad294640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2a815d150 0x7fd2a80c9150 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.387+0000 7fd2ad294640 1 --2- 172.21.15.125:0/3828272870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd290004d00 0x7fd2a815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.387+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 >> 172.21.15.125:0/3828272870 conn(0x7fd2a809a030 msgr2=0x7fd2a80aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:20.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.387+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 shutdown_connections 2023-10-18T10:41:20.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.387+0000 7fd2ad294640 1 -- 172.21.15.125:0/3828272870 wait complete. 2023-10-18T10:41:20.403 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T10:41:20.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.508+0000 7f6d4921a640 1 -- 172.21.15.125:0/2917766967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 msgr2=0x7f6d2c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.508+0000 7f6d4921a640 1 --2- 172.21.15.125:0/2917766967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 0x7f6d2c003d90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f6d34001d80 tx=0x7f6d2c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.508+0000 7f6d4921a640 1 -- 172.21.15.125:0/2917766967 shutdown_connections 2023-10-18T10:41:20.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.508+0000 7f6d4921a640 1 --2- 172.21.15.125:0/2917766967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 0x7f6d2c003d90 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.508+0000 7f6d4921a640 1 -- 172.21.15.125:0/2917766967 >> 172.21.15.125:0/2917766967 conn(0x7f6d440d0d60 msgr2=0x7f6d440d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:20.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.508+0000 7f6d4921a640 1 -- 172.21.15.125:0/2917766967 shutdown_connections 2023-10-18T10:41:20.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.509+0000 7f6d4921a640 1 -- 172.21.15.125:0/2917766967 wait complete. 2023-10-18T10:41:20.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.510+0000 7f6d4921a640 1 Processor -- start 2023-10-18T10:41:20.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.510+0000 7f6d4921a640 1 -- start start 2023-10-18T10:41:20.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d4921a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 0x7f6d441511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:20.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d4921a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d441f5a00 0x7f6d4415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:20.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d4921a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d441f1710 0x7f6d440b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:20.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d4921a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d44152780 con 0x7f6d2c003970 2023-10-18T10:41:20.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d4921a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4415d850 con 0x7f6d441f5a00 2023-10-18T10:41:20.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d4921a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d44152130 con 0x7f6d441f1710 2023-10-18T10:41:20.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 0x7f6d441511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:20.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 0x7f6d441511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45038/0 (socket says 172.21.15.125:45038) 2023-10-18T10:41:20.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 -- 172.21.15.125:0/3557151938 learned_addr learned my addr 172.21.15.125:0/3557151938 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d4259c640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d441f5a00 0x7f6d4415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 -- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d441f1710 msgr2=0x7f6d440b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d441f1710 0x7f6d440b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 -- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d441f5a00 msgr2=0x7f6d4415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d441f5a00 0x7f6d4415ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 -- 172.21.15.125:0/3557151938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44152410 con 0x7f6d2c003970 2023-10-18T10:41:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d42d9d640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 0x7f6d441511e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000f80 tx=0x7f6d34003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d2b7fe640 1 -- 172.21.15.125:0/3557151938 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3402ce30 con 0x7f6d2c003970 2023-10-18T10:41:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d2b7fe640 1 -- 172.21.15.125:0/3557151938 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d34006140 con 0x7f6d2c003970 2023-10-18T10:41:20.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d44151720 con 0x7f6d2c003970 2023-10-18T10:41:20.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.511+0000 7f6d2b7fe640 1 -- 172.21.15.125:0/3557151938 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34006550 con 0x7f6d2c003970 2023-10-18T10:41:20.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.512+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4415d5d0 con 0x7f6d2c003970 2023-10-18T10:41:20.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.513+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d44151f90 con 0x7f6d2c003970 2023-10-18T10:41:20.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.513+0000 7f6d2b7fe640 1 -- 172.21.15.125:0/3557151938 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6d3403d070 con 0x7f6d2c003970 2023-10-18T10:41:20.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.513+0000 7f6d2b7fe640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6d3006af90 0x7f6d3006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:20.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.513+0000 7f6d2b7fe640 1 -- 172.21.15.125:0/3557151938 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f6d340aa1e0 con 0x7f6d2c003970 2023-10-18T10:41:20.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.514+0000 7f6d4259c640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6d3006af90 0x7f6d3006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:20.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.514+0000 7f6d4259c640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6d3006af90 0x7f6d3006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6d38000c00 tx=0x7f6d38000fe0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:20.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.517+0000 7f6d2b7fe640 1 -- 172.21.15.125:0/3557151938 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d44151f90 con 0x7f6d2c003970 2023-10-18T10:41:20.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.684+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7f6d44152270 con 0x7f6d2c003970 2023-10-18T10:41:20.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.688+0000 7f6d2b7fe640 1 -- 172.21.15.125:0/3557151938 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v46) v1 ==== 71+0+0 (secure 0 0 0) 0x7f6d44152270 con 0x7f6d2c003970 2023-10-18T10:41:20.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6d3006af90 msgr2=0x7f6d3006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6d3006af90 0x7f6d3006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6d38000c00 tx=0x7f6d38000fe0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 msgr2=0x7f6d441511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 0x7f6d441511e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000f80 tx=0x7f6d34003c10 comp rx=0 tx=0).stop 2023-10-18T10:41:20.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 shutdown_connections 2023-10-18T10:41:20.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6d3006af90 0x7f6d3006d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d441f5a00 0x7f6d4415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d441f1710 0x7f6d440b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 --2- 172.21.15.125:0/3557151938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d2c003970 0x7f6d441511e0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 >> 172.21.15.125:0/3557151938 conn(0x7f6d440d0d60 msgr2=0x7f6d440b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:20.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 shutdown_connections 2023-10-18T10:41:20.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.690+0000 7f6d4921a640 1 -- 172.21.15.125:0/3557151938 wait complete. 2023-10-18T10:41:20.702 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-18T10:41:20.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.803+0000 7fac1a287640 1 -- 172.21.15.125:0/3407005002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 msgr2=0x7fabfc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.803+0000 7fac1a287640 1 --2- 172.21.15.125:0/3407005002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 0x7fabfc003d90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fac04009b50 tx=0x7fabfc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.803+0000 7fac18824640 1 -- 172.21.15.125:0/3407005002 reap_dead start 2023-10-18T10:41:20.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.804+0000 7fac1a287640 1 -- 172.21.15.125:0/3407005002 shutdown_connections 2023-10-18T10:41:20.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.804+0000 7fac1a287640 1 -- 172.21.15.125:0/3407005002 >> 172.21.15.125:0/3407005002 conn(0x7fac14069930 msgr2=0x7fac14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:20.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.804+0000 7fac1a287640 1 -- 172.21.15.125:0/3407005002 shutdown_connections 2023-10-18T10:41:20.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.804+0000 7fac1a287640 1 -- 172.21.15.125:0/3407005002 wait complete. 2023-10-18T10:41:20.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.805+0000 7fac1a287640 1 Processor -- start 2023-10-18T10:41:20.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.805+0000 7fac1a287640 1 -- start start 2023-10-18T10:41:20.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac1a287640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 0x7fac1414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:20.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac1a287640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac140aa3f0 0x7fac140a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:20.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac1a287640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac140aa870 0x7fac141562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:20.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac1a287640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac140a6f70 con 0x7fabfc003970 2023-10-18T10:41:20.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac1a287640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac140a7250 con 0x7fac140aa870 2023-10-18T10:41:20.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac1a287640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac140a7530 con 0x7fac140aa3f0 2023-10-18T10:41:20.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 0x7fac1414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:20.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 0x7fac1414cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45054/0 (socket says 172.21.15.125:45054) 2023-10-18T10:41:20.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 -- 172.21.15.125:0/357802284 learned_addr learned my addr 172.21.15.125:0/357802284 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:20.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac18824640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac140aa870 0x7fac141562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:20.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 -- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac140aa3f0 msgr2=0x7fac140a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac140aa3f0 0x7fac140a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 -- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac140aa870 msgr2=0x7fac141562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:20.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac140aa870 0x7fac141562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:20.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 -- 172.21.15.125:0/357802284 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14158980 con 0x7fabfc003970 2023-10-18T10:41:20.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac13fff640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 0x7fac1414cc50 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fac04009b50 tx=0x7fac04033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:20.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac10ff9640 1 -- 172.21.15.125:0/357802284 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04004440 con 0x7fabfc003970 2023-10-18T10:41:20.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac10ff9640 1 -- 172.21.15.125:0/357802284 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac04033e60 con 0x7fabfc003970 2023-10-18T10:41:20.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac14158c60 con 0x7fabfc003970 2023-10-18T10:41:20.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac10ff9640 1 -- 172.21.15.125:0/357802284 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04003830 con 0x7fabfc003970 2023-10-18T10:41:20.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.806+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1415a430 con 0x7fabfc003970 2023-10-18T10:41:20.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.807+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac14158df0 con 0x7fabfc003970 2023-10-18T10:41:20.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.808+0000 7fac10ff9640 1 -- 172.21.15.125:0/357802284 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fac04003990 con 0x7fabfc003970 2023-10-18T10:41:20.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.808+0000 7fac10ff9640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fac0006af90 0x7fac0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:20.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.808+0000 7fac10ff9640 1 -- 172.21.15.125:0/357802284 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fac040acab0 con 0x7fabfc003970 2023-10-18T10:41:20.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.808+0000 7fac137fe640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fac0006af90 0x7fac0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:20.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.809+0000 7fac137fe640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fac0006af90 0x7fac0006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fac08001eb0 tx=0x7fac08001cd0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:20.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:20.812+0000 7fac10ff9640 1 -- 172.21.15.125:0/357802284 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac04077850 con 0x7fabfc003970 2023-10-18T10:41:21.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.045+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7fac140690e0 con 0x7fabfc003970 2023-10-18T10:41:21.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.391+0000 7fac10ff9640 1 -- 172.21.15.125:0/357802284 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v607) v1 ==== 166+0+0 (secure 0 0 0) 0x7fac0404c020 con 0x7fabfc003970 2023-10-18T10:41:21.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fac0006af90 msgr2=0x7fac0006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fac0006af90 0x7fac0006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fac08001eb0 tx=0x7fac08001cd0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 msgr2=0x7fac1414cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 0x7fac1414cc50 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fac04009b50 tx=0x7fac04033c30 comp rx=0 tx=0).stop 2023-10-18T10:41:21.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 shutdown_connections 2023-10-18T10:41:21.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fac0006af90 0x7fac0006d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac140aa870 0x7fac141562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac140aa3f0 0x7fac140a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 --2- 172.21.15.125:0/357802284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabfc003970 0x7fac1414cc50 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 >> 172.21.15.125:0/357802284 conn(0x7fac14069930 msgr2=0x7fac1414ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:21.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 shutdown_connections 2023-10-18T10:41:21.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.393+0000 7fac1a287640 1 -- 172.21.15.125:0/357802284 wait complete. 2023-10-18T10:41:21.406 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-10-18T10:41:21.407 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-10-18T10:41:21.407 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T10:41:21.408 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:21.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.509+0000 7fec143a9640 1 -- 172.21.15.125:0/3003422265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 msgr2=0x7fec0c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.509+0000 7fec143a9640 1 --2- 172.21.15.125:0/3003422265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 0x7fec0c0d1910 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fec0c15b680 tx=0x7fec0802c1d0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.509+0000 7fec143a9640 1 -- 172.21.15.125:0/3003422265 shutdown_connections 2023-10-18T10:41:21.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.509+0000 7fec143a9640 1 --2- 172.21.15.125:0/3003422265 >> v2:172.21.15.139:3300/0 conn(0x7fec0c0ceb40 0x7fec0c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.509+0000 7fec143a9640 1 --2- 172.21.15.125:0/3003422265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 0x7fec0c0d1910 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.509+0000 7fec143a9640 1 -- 172.21.15.125:0/3003422265 >> 172.21.15.125:0/3003422265 conn(0x7fec0c0d0d60 msgr2=0x7fec0c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:21.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.509+0000 7fec143a9640 1 -- 172.21.15.125:0/3003422265 shutdown_connections 2023-10-18T10:41:21.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.509+0000 7fec143a9640 1 -- 172.21.15.125:0/3003422265 wait complete. 2023-10-18T10:41:21.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.510+0000 7fec143a9640 1 Processor -- start 2023-10-18T10:41:21.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.510+0000 7fec143a9640 1 -- start start 2023-10-18T10:41:21.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec143a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec0c0ceb40 0x7fec0c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:21.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec143a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 0x7fec0c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:21.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec143a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec0c15b2a0 0x7fec0c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:21.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec143a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0b4b50 con 0x7fec0c0cf340 2023-10-18T10:41:21.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec143a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c15bd80 con 0x7fec0c0ceb40 2023-10-18T10:41:21.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec143a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c15b770 con 0x7fec0c15b2a0 2023-10-18T10:41:21.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec11944640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 0x7fec0c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:21.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec11944640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 0x7fec0c0b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45072/0 (socket says 172.21.15.125:45072) 2023-10-18T10:41:21.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec11944640 1 -- 172.21.15.125:0/961017615 learned_addr learned my addr 172.21.15.125:0/961017615 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:21.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec12145640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec0c0ceb40 0x7fec0c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:21.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec11944640 1 -- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec0c15b2a0 msgr2=0x7fec0c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec12946640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec0c15b2a0 0x7fec0c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:21.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec11944640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec0c15b2a0 0x7fec0c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec11944640 1 -- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec0c0ceb40 msgr2=0x7fec0c1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec11944640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec0c0ceb40 0x7fec0c1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec11944640 1 -- 172.21.15.125:0/961017615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec08009900 con 0x7fec0c0cf340 2023-10-18T10:41:21.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec12145640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec0c0ceb40 0x7fec0c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:21.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.511+0000 7fec12946640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec0c15b2a0 0x7fec0c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:21.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.512+0000 7fec11944640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 0x7fec0c0b40d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7febfc00b3e0 tx=0x7febfc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:21.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.512+0000 7fec02ffd640 1 -- 172.21.15.125:0/961017615 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc011830 con 0x7fec0c0cf340 2023-10-18T10:41:21.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.512+0000 7fec02ffd640 1 -- 172.21.15.125:0/961017615 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7febfc011aa0 con 0x7fec0c0cf340 2023-10-18T10:41:21.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.512+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c0af670 con 0x7fec0c0cf340 2023-10-18T10:41:21.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.512+0000 7fec02ffd640 1 -- 172.21.15.125:0/961017615 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc010380 con 0x7fec0c0cf340 2023-10-18T10:41:21.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.512+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c0af950 con 0x7fec0c0cf340 2023-10-18T10:41:21.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.513+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c15bf10 con 0x7fec0c0cf340 2023-10-18T10:41:21.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.513+0000 7fec02ffd640 1 -- 172.21.15.125:0/961017615 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7febfc0104e0 con 0x7fec0c0cf340 2023-10-18T10:41:21.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.514+0000 7fec02ffd640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7febf006af90 0x7febf006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:21.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.514+0000 7fec02ffd640 1 -- 172.21.15.125:0/961017615 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7febfc089bb0 con 0x7fec0c0cf340 2023-10-18T10:41:21.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.514+0000 7fec12145640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7febf006af90 0x7febf006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:21.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.514+0000 7fec12145640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7febf006af90 0x7febf006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fec08036000 tx=0x7fec08004ab0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:21.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.518+0000 7fec02ffd640 1 -- 172.21.15.125:0/961017615 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7febfc054af0 con 0x7fec0c0cf340 2023-10-18T10:41:21.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.684+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec0c0b0d00 con 0x7fec0c0cf340 2023-10-18T10:41:21.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.685+0000 7fec02ffd640 1 -- 172.21.15.125:0/961017615 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+14257 (secure 0 0 0) 0x7febfc015090 con 0x7fec0c0cf340 2023-10-18T10:41:21.687 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:21.687 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":103,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:20.382251+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6835/1041197980":"2023-10-19T10:41:17.300892+0000","172.21.15.125:6833/1041197980":"2023-10-19T10:41:17.300892+0000","172.21.15.139:0/495291025":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/1376126096":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/2074636940":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/905146753":"2023-10-19T10:41:15.264454+0000","172.21.15.139:6832/1652676135":"2023-10-19T10:41:15.264454+0000","172.21.15.139:6833/1652676135":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/383144741":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/3534247944":"2023-10-19T10:41:15.264454+0000","172.21.15.139:0/2731662835":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/3976511705":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/1104476358":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2807756531":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6833/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:6832/143413153":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/264862829":"2023-10-19T10:41:09.823648+0000","172.21.15.139:0/2537511727":"2023-10-19T10:41:09.823648+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:21.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7febf006af90 msgr2=0x7febf006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7febf006af90 0x7febf006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fec08036000 tx=0x7fec08004ab0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 msgr2=0x7fec0c0b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 0x7fec0c0b40d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7febfc00b3e0 tx=0x7febfc00b8a0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 shutdown_connections 2023-10-18T10:41:21.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7febf006af90 0x7febf006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec0c15b2a0 0x7fec0c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec0c0cf340 0x7fec0c0b40d0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 --2- 172.21.15.125:0/961017615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec0c0ceb40 0x7fec0c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.686+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 >> 172.21.15.125:0/961017615 conn(0x7fec0c0d0d60 msgr2=0x7fec0c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:21.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.687+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 shutdown_connections 2023-10-18T10:41:21.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.687+0000 7fec143a9640 1 -- 172.21.15.125:0/961017615 wait complete. 2023-10-18T10:41:21.700 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:41:21.700 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:41:21.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 -- 172.21.15.125:0/849197363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0cb980 msgr2=0x7f25fc13db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 --2- 172.21.15.125:0/849197363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0cb980 0x7f25fc13db80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f25fc06fa40 tx=0x7f25ec02bda0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 -- 172.21.15.125:0/849197363 shutdown_connections 2023-10-18T10:41:21.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 --2- 172.21.15.125:0/849197363 >> v2:172.21.15.139:3300/0 conn(0x7f25fc0c96b0 0x7f25fc0c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 --2- 172.21.15.125:0/849197363 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25fc13d350 0x7f25fc0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 --2- 172.21.15.125:0/849197363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0cb980 0x7f25fc13db80 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 -- 172.21.15.125:0/849197363 >> 172.21.15.125:0/849197363 conn(0x7f25fc069930 msgr2=0x7f25fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:21.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 -- 172.21.15.125:0/849197363 shutdown_connections 2023-10-18T10:41:21.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.804+0000 7f2601ee5640 1 -- 172.21.15.125:0/849197363 wait complete. 2023-10-18T10:41:21.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.805+0000 7f2601ee5640 1 Processor -- start 2023-10-18T10:41:21.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.805+0000 7f2601ee5640 1 -- start start 2023-10-18T10:41:21.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f2601ee5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25fc0c96b0 0x7f25fc137bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:21.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f2601ee5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25fc0cb980 0x7f25fc159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:21.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f2601ee5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0707f0 0x7f25fc1386d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:21.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f2601ee5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc139750 con 0x7f25fc0707f0 2023-10-18T10:41:21.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f2601ee5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc14f1d0 con 0x7f25fc0cb980 2023-10-18T10:41:21.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f2601ee5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc14f700 con 0x7f25fc0c96b0 2023-10-18T10:41:21.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0707f0 0x7f25fc1386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:21.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0707f0 0x7f25fc1386d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45088/0 (socket says 172.21.15.125:45088) 2023-10-18T10:41:21.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 -- 172.21.15.125:0/1070119198 learned_addr learned my addr 172.21.15.125:0/1070119198 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:21.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fb7fe640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25fc0c96b0 0x7f25fc137bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:21.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25faffd640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25fc0cb980 0x7f25fc159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:21.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 -- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25fc0c96b0 msgr2=0x7f25fc137bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25fc0c96b0 0x7f25fc137bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 -- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25fc0cb980 msgr2=0x7f25fc159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25fc0cb980 0x7f25fc159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 -- 172.21.15.125:0/1070119198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec009560 con 0x7f25fc0707f0 2023-10-18T10:41:21.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.806+0000 7f25fbfff640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0707f0 0x7f25fc1386d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f25f400caa0 tx=0x7f25f400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:21.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.807+0000 7f25e3fff640 1 -- 172.21.15.125:0/1070119198 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f400adf0 con 0x7f25fc0707f0 2023-10-18T10:41:21.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.807+0000 7f25e3fff640 1 -- 172.21.15.125:0/1070119198 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25f4004c10 con 0x7f25fc0707f0 2023-10-18T10:41:21.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.807+0000 7f25e3fff640 1 -- 172.21.15.125:0/1070119198 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4005020 con 0x7f25fc0707f0 2023-10-18T10:41:21.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.807+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc1399a0 con 0x7f25fc0707f0 2023-10-18T10:41:21.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.807+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc13a110 con 0x7f25fc0707f0 2023-10-18T10:41:21.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.808+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25fc0690e0 con 0x7f25fc0707f0 2023-10-18T10:41:21.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.808+0000 7f25e3fff640 1 -- 172.21.15.125:0/1070119198 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f25f4005180 con 0x7f25fc0707f0 2023-10-18T10:41:21.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.808+0000 7f25e3fff640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f25e806af60 0x7f25e806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:21.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.809+0000 7f25e3fff640 1 -- 172.21.15.125:0/1070119198 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f25f408a040 con 0x7f25fc0707f0 2023-10-18T10:41:21.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.809+0000 7f25fb7fe640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f25e806af60 0x7f25e806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:21.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.809+0000 7f25fb7fe640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f25e806af60 0x7f25e806d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f25ec035000 tx=0x7f25ec004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:21.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.813+0000 7f25e3fff640 1 -- 172.21.15.125:0/1070119198 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25f4054f20 con 0x7f25fc0707f0 2023-10-18T10:41:21.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.978+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25fc06ff80 con 0x7f25fc0707f0 2023-10-18T10:41:21.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.978+0000 7f25e3fff640 1 -- 172.21.15.125:0/1070119198 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25f401d020 con 0x7f25fc0707f0 2023-10-18T10:41:21.980 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:41:21.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f25e806af60 msgr2=0x7f25e806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f25e806af60 0x7f25e806d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f25ec035000 tx=0x7f25ec004730 comp rx=0 tx=0).stop 2023-10-18T10:41:21.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0707f0 msgr2=0x7f25fc1386d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0707f0 0x7f25fc1386d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f25f400caa0 tx=0x7f25f400cf60 comp rx=0 tx=0).stop 2023-10-18T10:41:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 shutdown_connections 2023-10-18T10:41:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f25e806af60 0x7f25e806d420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25fc0cb980 0x7f25fc159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25fc0707f0 0x7f25fc1386d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 --2- 172.21.15.125:0/1070119198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25fc0c96b0 0x7f25fc137bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 >> 172.21.15.125:0/1070119198 conn(0x7f25fc069930 msgr2=0x7f25fc06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 shutdown_connections 2023-10-18T10:41:21.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:21.980+0000 7f2601ee5640 1 -- 172.21.15.125:0/1070119198 wait complete. 2023-10-18T10:41:21.992 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-18T10:41:22.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 -- 172.21.15.125:0/3298499610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa800140660 msgr2=0x7fa8000c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 --2- 172.21.15.125:0/3298499610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa800140660 0x7fa8000c97c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa8000ca400 tx=0x7fa7f002bda0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 -- 172.21.15.125:0/3298499610 shutdown_connections 2023-10-18T10:41:22.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 --2- 172.21.15.125:0/3298499610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa800140660 0x7fa8000c97c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 --2- 172.21.15.125:0/3298499610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8000c9d00 0x7fa8000ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 --2- 172.21.15.125:0/3298499610 >> v2:172.21.15.139:3300/0 conn(0x7fa80013d360 0x7fa80015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 -- 172.21.15.125:0/3298499610 >> 172.21.15.125:0/3298499610 conn(0x7fa800069930 msgr2=0x7fa800069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:22.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 -- 172.21.15.125:0/3298499610 shutdown_connections 2023-10-18T10:41:22.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.093+0000 7fa8073a5640 1 -- 172.21.15.125:0/3298499610 wait complete. 2023-10-18T10:41:22.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.094+0000 7fa8073a5640 1 Processor -- start 2023-10-18T10:41:22.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.094+0000 7fa8073a5640 1 -- start start 2023-10-18T10:41:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa8073a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8000c9d00 0x7fa800156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa8073a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80013d360 0x7fa80013c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa8073a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa800140660 0x7fa800139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa8073a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800156990 con 0x7fa8000c9d00 2023-10-18T10:41:22.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa8073a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800131dd0 con 0x7fa80013d360 2023-10-18T10:41:22.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa8073a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8001320b0 con 0x7fa800140660 2023-10-18T10:41:22.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8000c9d00 0x7fa800156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8000c9d00 0x7fa800156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45106/0 (socket says 172.21.15.125:45106) 2023-10-18T10:41:22.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 -- 172.21.15.125:0/135357613 learned_addr learned my addr 172.21.15.125:0/135357613 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:22.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa804940640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80013d360 0x7fa80013c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805942640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa800140660 0x7fa800139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 -- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa800140660 msgr2=0x7fa800139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa800140660 0x7fa800139d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 -- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80013d360 msgr2=0x7fa80013c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80013d360 0x7fa80013c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 -- 172.21.15.125:0/135357613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f0009560 con 0x7fa8000c9d00 2023-10-18T10:41:22.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.095+0000 7fa805141640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8000c9d00 0x7fa800156450 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00eab0 tx=0x7fa7fc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:22.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.096+0000 7fa7f5ffb640 1 -- 172.21.15.125:0/135357613 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00ce40 con 0x7fa8000c9d00 2023-10-18T10:41:22.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.096+0000 7fa7f5ffb640 1 -- 172.21.15.125:0/135357613 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa7fc011560 con 0x7fa8000c9d00 2023-10-18T10:41:22.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.096+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa800132390 con 0x7fa8000c9d00 2023-10-18T10:41:22.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.096+0000 7fa7f5ffb640 1 -- 172.21.15.125:0/135357613 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc011910 con 0x7fa8000c9d00 2023-10-18T10:41:22.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.096+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa800132950 con 0x7fa8000c9d00 2023-10-18T10:41:22.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.097+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa800132ae0 con 0x7fa8000c9d00 2023-10-18T10:41:22.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.097+0000 7fa7f5ffb640 1 -- 172.21.15.125:0/135357613 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa7fc011a70 con 0x7fa8000c9d00 2023-10-18T10:41:22.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.097+0000 7fa7f5ffb640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa7e406af90 0x7fa7e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.097+0000 7fa7f5ffb640 1 -- 172.21.15.125:0/135357613 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fa7fc08a610 con 0x7fa8000c9d00 2023-10-18T10:41:22.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.098+0000 7fa804940640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa7e406af90 0x7fa7e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.098+0000 7fa804940640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa7e406af90 0x7fa7e406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0035000 tx=0x7fa7f0004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:22.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.102+0000 7fa7f5ffb640 1 -- 172.21.15.125:0/135357613 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7fc0553b0 con 0x7fa8000c9d00 2023-10-18T10:41:22.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.339+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fa8000690e0 con 0x7fa8000c9d00 2023-10-18T10:41:22.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.392+0000 7fa7f5ffb640 1 -- 172.21.15.125:0/135357613 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v608) v1 ==== 186+0+0 (secure 0 0 0) 0x7fa7fc0589d0 con 0x7fa8000c9d00 2023-10-18T10:41:22.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa7e406af90 msgr2=0x7fa7e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa7e406af90 0x7fa7e406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0035000 tx=0x7fa7f0004730 comp rx=0 tx=0).stop 2023-10-18T10:41:22.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8000c9d00 msgr2=0x7fa800156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8000c9d00 0x7fa800156450 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00eab0 tx=0x7fa7fc00ef70 comp rx=0 tx=0).stop 2023-10-18T10:41:22.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 shutdown_connections 2023-10-18T10:41:22.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa7e406af90 0x7fa7e406d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80013d360 0x7fa80013c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa800140660 0x7fa800139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 --2- 172.21.15.125:0/135357613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8000c9d00 0x7fa800156450 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 >> 172.21.15.125:0/135357613 conn(0x7fa800069930 msgr2=0x7fa800157eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:22.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 shutdown_connections 2023-10-18T10:41:22.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.394+0000 7fa8073a5640 1 -- 172.21.15.125:0/135357613 wait complete. 2023-10-18T10:41:22.407 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T10:41:22.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.508+0000 7f568f610640 1 -- 172.21.15.125:0/3795147784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 msgr2=0x7f567400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.508+0000 7f568f610640 1 --2- 172.21.15.125:0/3795147784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 0x7f567400b980 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f56800043b0 tx=0x7f56740057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.508+0000 7f568f610640 1 -- 172.21.15.125:0/3795147784 shutdown_connections 2023-10-18T10:41:22.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.508+0000 7f568f610640 1 --2- 172.21.15.125:0/3795147784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 0x7f567400b980 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.508+0000 7f568f610640 1 -- 172.21.15.125:0/3795147784 >> 172.21.15.125:0/3795147784 conn(0x7f5688069930 msgr2=0x7f5688069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:22.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.508+0000 7f568f610640 1 -- 172.21.15.125:0/3795147784 shutdown_connections 2023-10-18T10:41:22.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.508+0000 7f568f610640 1 -- 172.21.15.125:0/3795147784 wait complete. 2023-10-18T10:41:22.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.509+0000 7f568f610640 1 Processor -- start 2023-10-18T10:41:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.509+0000 7f568f610640 1 -- start start 2023-10-18T10:41:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568f610640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 0x7f56880ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568f610640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5688070570 0x7f5688151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568f610640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56880aa8c0 0x7f56880aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568f610640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880ab2b0 con 0x7f56740052a0 2023-10-18T10:41:22.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568f610640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880afdd0 con 0x7f56880aa8c0 2023-10-18T10:41:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568f610640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5688152240 con 0x7f5688070570 2023-10-18T10:41:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 0x7f56880ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 0x7f56880ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45122/0 (socket says 172.21.15.125:45122) 2023-10-18T10:41:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 -- 172.21.15.125:0/3982066478 learned_addr learned my addr 172.21.15.125:0/3982066478 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:22.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 -- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5688070570 msgr2=0x7f5688151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5688070570 0x7f5688151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 -- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56880aa8c0 msgr2=0x7f56880aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56880aa8c0 0x7f56880aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 -- 172.21.15.125:0/3982066478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880aed30 con 0x7f56740052a0 2023-10-18T10:41:22.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568d3ac640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 0x7f56880ae700 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f567c00e660 tx=0x7f567c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:22.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f5679ffb640 1 -- 172.21.15.125:0/3982066478 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c004250 con 0x7f56740052a0 2023-10-18T10:41:22.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f5679ffb640 1 -- 172.21.15.125:0/3982066478 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f567c0044a0 con 0x7f56740052a0 2023-10-18T10:41:22.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f5679ffb640 1 -- 172.21.15.125:0/3982066478 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00caa0 con 0x7f56740052a0 2023-10-18T10:41:22.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.510+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56880af010 con 0x7f56740052a0 2023-10-18T10:41:22.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.511+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56880af5d0 con 0x7f56740052a0 2023-10-18T10:41:22.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.512+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56880690e0 con 0x7f56740052a0 2023-10-18T10:41:22.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.512+0000 7f5679ffb640 1 -- 172.21.15.125:0/3982066478 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f567c00cc00 con 0x7f56740052a0 2023-10-18T10:41:22.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.512+0000 7f5679ffb640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f566c06af60 0x7f566c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.512+0000 7f5679ffb640 1 -- 172.21.15.125:0/3982066478 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f567c08a0a0 con 0x7f56740052a0 2023-10-18T10:41:22.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.513+0000 7f568cbab640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f566c06af60 0x7f566c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.513+0000 7f568cbab640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f566c06af60 0x7f566c06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5684005940 tx=0x7f56840058b0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:22.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.516+0000 7f5679ffb640 1 -- 172.21.15.125:0/3982066478 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f567c054ed0 con 0x7f56740052a0 2023-10-18T10:41:22.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.683+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f56880cc420 con 0x7f56740052a0 2023-10-18T10:41:22.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.683+0000 7f5679ffb640 1 -- 172.21.15.125:0/3982066478 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v46) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f567c0158c0 con 0x7f56740052a0 2023-10-18T10:41:22.686 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T10:41:22.686 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T10:41:22.686 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T10:41:22.686 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T10:41:22.686 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T10:41:22.687 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T10:41:22.687 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T10:41:22.688 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T10:41:22.688 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T10:41:22.688 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T10:41:22.688 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T10:41:22.688 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T10:41:22.689 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T10:41:22.689 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T10:41:22.689 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T10:41:22.689 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T10:41:22.689 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T10:41:22.690 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T10:41:22.690 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T10:41:22.690 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T10:41:22.690 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T10:41:22.690 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T10:41:22.691 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T10:41:22.691 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T10:41:22.691 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T10:41:22.691 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T10:41:22.691 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T10:41:22.692 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T10:41:22.692 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T10:41:22.692 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T10:41:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f566c06af60 msgr2=0x7f566c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f566c06af60 0x7f566c06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5684005940 tx=0x7f56840058b0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 msgr2=0x7f56880ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 0x7f56880ae700 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f567c00e660 tx=0x7f567c00eb20 comp rx=0 tx=0).stop 2023-10-18T10:41:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 shutdown_connections 2023-10-18T10:41:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f566c06af60 0x7f566c06d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56880aa8c0 0x7f56880aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5688070570 0x7f5688151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 --2- 172.21.15.125:0/3982066478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f56740052a0 0x7f56880ae700 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 >> 172.21.15.125:0/3982066478 conn(0x7f5688069930 msgr2=0x7f56880ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 shutdown_connections 2023-10-18T10:41:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.685+0000 7f568f610640 1 -- 172.21.15.125:0/3982066478 wait complete. 2023-10-18T10:41:22.698 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:41:22.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 -- 172.21.15.125:0/3544818105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b8406af00 msgr2=0x7f7b8406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 --2- 172.21.15.125:0/3544818105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b8406af00 0x7f7b8406b2e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7b8406d590 tx=0x7f7b7402bda0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 -- 172.21.15.125:0/3544818105 shutdown_connections 2023-10-18T10:41:22.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 --2- 172.21.15.125:0/3544818105 >> v2:172.21.15.139:3300/0 conn(0x7f7b840ab500 0x7f7b840ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 --2- 172.21.15.125:0/3544818105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b840c96b0 0x7f7b840c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 --2- 172.21.15.125:0/3544818105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b8406af00 0x7f7b8406b2e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 -- 172.21.15.125:0/3544818105 >> 172.21.15.125:0/3544818105 conn(0x7f7b84069930 msgr2=0x7f7b84069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:22.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 -- 172.21.15.125:0/3544818105 shutdown_connections 2023-10-18T10:41:22.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.802+0000 7f7b89e56640 1 -- 172.21.15.125:0/3544818105 wait complete. 2023-10-18T10:41:22.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.803+0000 7f7b89e56640 1 Processor -- start 2023-10-18T10:41:22.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.803+0000 7f7b89e56640 1 -- start start 2023-10-18T10:41:22.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b89e56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b8406af00 0x7f7b840af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b89e56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b840ab500 0x7f7b84070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b89e56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b840c96b0 0x7f7b841383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b89e56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840af830 con 0x7f7b840c96b0 2023-10-18T10:41:22.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b89e56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8406f080 con 0x7f7b8406af00 2023-10-18T10:41:22.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b89e56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b841f5010 con 0x7f7b840ab500 2023-10-18T10:41:22.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b840c96b0 0x7f7b841383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b840c96b0 0x7f7b841383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45136/0 (socket says 172.21.15.125:45136) 2023-10-18T10:41:22.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 -- 172.21.15.125:0/2328937776 learned_addr learned my addr 172.21.15.125:0/2328937776 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:22.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b837fe640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b8406af00 0x7f7b840af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b82ffd640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b840ab500 0x7f7b84070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 -- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b840ab500 msgr2=0x7f7b84070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b840ab500 0x7f7b84070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 -- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b8406af00 msgr2=0x7f7b840af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b8406af00 0x7f7b840af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 -- 172.21.15.125:0/2328937776 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74009560 con 0x7f7b840c96b0 2023-10-18T10:41:22.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b83fff640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b840c96b0 0x7f7b841383a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00cce0 tx=0x7f7b7c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:22.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b6bfff640 1 -- 172.21.15.125:0/2328937776 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00f450 con 0x7f7b840c96b0 2023-10-18T10:41:22.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b6bfff640 1 -- 172.21.15.125:0/2328937776 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b7c00f6c0 con 0x7f7b840c96b0 2023-10-18T10:41:22.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b6bfff640 1 -- 172.21.15.125:0/2328937776 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00fad0 con 0x7f7b840c96b0 2023-10-18T10:41:22.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.804+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b841f52f0 con 0x7f7b840c96b0 2023-10-18T10:41:22.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.805+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b841f58b0 con 0x7f7b840c96b0 2023-10-18T10:41:22.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.805+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b841f5ad0 con 0x7f7b840c96b0 2023-10-18T10:41:22.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.806+0000 7f7b6bfff640 1 -- 172.21.15.125:0/2328937776 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7b7c00fc30 con 0x7f7b840c96b0 2023-10-18T10:41:22.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.806+0000 7f7b6bfff640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f7b7006af60 0x7f7b7006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:22.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.806+0000 7f7b6bfff640 1 -- 172.21.15.125:0/2328937776 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f7b7c08a580 con 0x7f7b840c96b0 2023-10-18T10:41:22.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.806+0000 7f7b837fe640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f7b7006af60 0x7f7b7006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:22.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.807+0000 7f7b837fe640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f7b7006af60 0x7f7b7006d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b74035000 tx=0x7f7b74004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:22.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.810+0000 7f7b6bfff640 1 -- 172.21.15.125:0/2328937776 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b7c055320 con 0x7f7b840c96b0 2023-10-18T10:41:22.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.976+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7b8406f1f0 con 0x7f7b840c96b0 2023-10-18T10:41:22.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.980+0000 7f7b6bfff640 1 -- 172.21.15.125:0/2328937776 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v47)=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7b8406f1f0 con 0x7f7b840c96b0 2023-10-18T10:41:22.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.981+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f7b7006af60 msgr2=0x7f7b7006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.981+0000 7f7b89e56640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f7b7006af60 0x7f7b7006d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b74035000 tx=0x7f7b74004730 comp rx=0 tx=0).stop 2023-10-18T10:41:22.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.981+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b840c96b0 msgr2=0x7f7b841383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:22.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b840c96b0 0x7f7b841383a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00cce0 tx=0x7f7b7c00a620 comp rx=0 tx=0).stop 2023-10-18T10:41:22.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 shutdown_connections 2023-10-18T10:41:22.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f7b7006af60 0x7f7b7006d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b840c96b0 0x7f7b841383a0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b840ab500 0x7f7b84070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 --2- 172.21.15.125:0/2328937776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b8406af00 0x7f7b840af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:22.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 >> 172.21.15.125:0/2328937776 conn(0x7f7b84069930 msgr2=0x7f7b840cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:22.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 shutdown_connections 2023-10-18T10:41:22.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:22.982+0000 7f7b89e56640 1 -- 172.21.15.125:0/2328937776 wait complete. 2023-10-18T10:41:22.994 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:41:22.995 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:41:23.044 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:41:23.088 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:41:23.089 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:41:23.089 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:41:23.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:41:23.105 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:41:23.106 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:41:23.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:41:23.146 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:41:23.146 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:41:23.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:41:23.162 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:41:23.162 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:41:23.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:41:23.202 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:41:23.203 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:41:23.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.304+0000 7fdded207640 1 -- 172.21.15.125:0/1180949422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde80cf340 msgr2=0x7fdde80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:23.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.304+0000 7fdded207640 1 --2- 172.21.15.125:0/1180949422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde80cf340 0x7fdde80d1910 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005fd0 tx=0x7fddd802c1b0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.304+0000 7fdded207640 1 -- 172.21.15.125:0/1180949422 shutdown_connections 2023-10-18T10:41:23.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.304+0000 7fdded207640 1 --2- 172.21.15.125:0/1180949422 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdde80ceb40 0x7fdde80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.304+0000 7fdded207640 1 --2- 172.21.15.125:0/1180949422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde80cf340 0x7fdde80d1910 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.304+0000 7fdded207640 1 -- 172.21.15.125:0/1180949422 >> 172.21.15.125:0/1180949422 conn(0x7fdde80d0d60 msgr2=0x7fdde80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:23.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.304+0000 7fdded207640 1 -- 172.21.15.125:0/1180949422 shutdown_connections 2023-10-18T10:41:23.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.304+0000 7fdded207640 1 -- 172.21.15.125:0/1180949422 wait complete. 2023-10-18T10:41:23.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.305+0000 7fdded207640 1 Processor -- start 2023-10-18T10:41:23.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.305+0000 7fdded207640 1 -- start start 2023-10-18T10:41:23.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdded207640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde80ceb40 0x7fdde81500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:23.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdded207640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdde80cf340 0x7fdde80b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:23.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdded207640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde815b2a0 0x7fdde80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:23.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdded207640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde80b4ed0 con 0x7fdde815b2a0 2023-10-18T10:41:23.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdded207640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde80b42c0 con 0x7fdde80ceb40 2023-10-18T10:41:23.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdded207640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde80b45a0 con 0x7fdde80cf340 2023-10-18T10:41:23.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde815b2a0 0x7fdde80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:23.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde815b2a0 0x7fdde80cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45142/0 (socket says 172.21.15.125:45142) 2023-10-18T10:41:23.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde759e640 1 -- 172.21.15.125:0/3663393133 learned_addr learned my addr 172.21.15.125:0/3663393133 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:23.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde6d9d640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde80ceb40 0x7fdde81500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:23.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde659c640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdde80cf340 0x7fdde80b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:23.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde759e640 1 -- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdde80cf340 msgr2=0x7fdde80b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:23.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde759e640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdde80cf340 0x7fdde80b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde759e640 1 -- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde80ceb40 msgr2=0x7fdde81500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:23.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde759e640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde80ceb40 0x7fdde81500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde759e640 1 -- 172.21.15.125:0/3663393133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd8009900 con 0x7fdde815b2a0 2023-10-18T10:41:23.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.306+0000 7fdde659c640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdde80cf340 0x7fdde80b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:23.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.307+0000 7fdde759e640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde815b2a0 0x7fdde80cc8c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d750 tx=0x7fdde000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:23.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.307+0000 7fddcf7fe640 1 -- 172.21.15.125:0/3663393133 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0009bd0 con 0x7fdde815b2a0 2023-10-18T10:41:23.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.307+0000 7fddcf7fe640 1 -- 172.21.15.125:0/3663393133 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdde0009e20 con 0x7fdde815b2a0 2023-10-18T10:41:23.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.307+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde80b4880 con 0x7fdde815b2a0 2023-10-18T10:41:23.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.307+0000 7fddcf7fe640 1 -- 172.21.15.125:0/3663393133 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0005280 con 0x7fdde815b2a0 2023-10-18T10:41:23.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.307+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde8150600 con 0x7fdde815b2a0 2023-10-18T10:41:23.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.308+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde81f1aa0 con 0x7fdde815b2a0 2023-10-18T10:41:23.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.308+0000 7fddcf7fe640 1 -- 172.21.15.125:0/3663393133 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdde00053e0 con 0x7fdde815b2a0 2023-10-18T10:41:23.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.309+0000 7fddcf7fe640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fddd406af90 0x7fddd406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:23.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.309+0000 7fddcf7fe640 1 -- 172.21.15.125:0/3663393133 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7fdde00899b0 con 0x7fdde815b2a0 2023-10-18T10:41:23.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.309+0000 7fdde6d9d640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fddd406af90 0x7fddd406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:23.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.310+0000 7fdde6d9d640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fddd406af90 0x7fddd406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fddd8036000 tx=0x7fddd8004ab0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:23.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.313+0000 7fddcf7fe640 1 -- 172.21.15.125:0/3663393133 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdde81f1aa0 con 0x7fdde815b2a0 2023-10-18T10:41:23.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.518+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdde81f1aa0 con 0x7fdde815b2a0 2023-10-18T10:41:23.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.518+0000 7fddcf7fe640 1 -- 172.21.15.125:0/3663393133 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fdde81f1aa0 con 0x7fdde815b2a0 2023-10-18T10:41:23.520 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:23.521 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3531064821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3531064821},{"type":"v1","addr":"172.21.15.139:6837","nonce":3531064821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2850027191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2850027191},{"type":"v1","addr":"172.21.15.139:6836","nonce":2850027191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7578,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3670537575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3670537575},{"type":"v1","addr":"172.21.15.125:6834","nonce":3670537575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3747395779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3747395779},{"type":"v1","addr":"172.21.15.125:6835","nonce":3747395779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[]} 2023-10-18T10:41:23.521 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 37 2023-10-18T10:41:23.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fddd406af90 msgr2=0x7fddd406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:23.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fddd406af90 0x7fddd406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fddd8036000 tx=0x7fddd8004ab0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde815b2a0 msgr2=0x7fdde80cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:23.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde815b2a0 0x7fdde80cc8c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d750 tx=0x7fdde000dc10 comp rx=0 tx=0).stop 2023-10-18T10:41:23.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 shutdown_connections 2023-10-18T10:41:23.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fddd406af90 0x7fddd406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdde815b2a0 0x7fdde80cc8c0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdde80cf340 0x7fdde80b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 --2- 172.21.15.125:0/3663393133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdde80ceb40 0x7fdde81500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 >> 172.21.15.125:0/3663393133 conn(0x7fdde80d0d60 msgr2=0x7fdde806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:23.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 shutdown_connections 2023-10-18T10:41:23.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.520+0000 7fdded207640 1 -- 172.21.15.125:0/3663393133 wait complete. 2023-10-18T10:41:23.533 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:41:23.533 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:41:23.533 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:41:23.534 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:41:23.534 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:41:23.534 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:41:23.535 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:41:23.535 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:41:23.535 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:41:23.535 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:41:23.536 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:41:23.536 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:41:23.536 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:41:23.536 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:41:23.537 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:41:23.537 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:41:23.542 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:41:23.543 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:41:23.544 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:41:23.544 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:41:23.547 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:41:23.547 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:41:23.548 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:41:23.548 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:41:23.586 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:41:23.587 DEBUG:teuthology.parallel:result is None 2023-10-18T10:41:23.588 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:41:23.588 DEBUG:teuthology.parallel:result is None 2023-10-18T10:41:23.591 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:41:23.591 DEBUG:teuthology.parallel:result is None 2023-10-18T10:41:23.592 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:41:23.592 DEBUG:teuthology.parallel:result is None 2023-10-18T10:41:23.593 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:41:23.661 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:41:23.662 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:41:23.672 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:41:23.672 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:41:23.673 INFO:tasks.ceph.mds.b.smithi139.stderr:2023-10-18T10:41:23.671+0000 7f24bb077640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-18T10:41:23.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.698+0000 7f94b477d640 1 -- 172.21.15.125:0/3245741933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9498004d00 msgr2=0x7f9498005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.698+0000 7f94b477d640 1 --2- 172.21.15.125:0/3245741933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9498004d00 0x7f9498005180 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f94a400c640 tx=0x7f94980057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.698+0000 7f94b2d1a640 1 -- 172.21.15.125:0/3245741933 reap_dead start 2023-10-18T10:41:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.699+0000 7f94b477d640 1 -- 172.21.15.125:0/3245741933 shutdown_connections 2023-10-18T10:41:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.699+0000 7f94b477d640 1 -- 172.21.15.125:0/3245741933 >> 172.21.15.125:0/3245741933 conn(0x7f94ac0d0d60 msgr2=0x7f94ac0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:23.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.699+0000 7f94b477d640 1 -- 172.21.15.125:0/3245741933 shutdown_connections 2023-10-18T10:41:23.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.699+0000 7f94b477d640 1 -- 172.21.15.125:0/3245741933 wait complete. 2023-10-18T10:41:23.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.700+0000 7f94b477d640 1 Processor -- start 2023-10-18T10:41:23.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.700+0000 7f94b477d640 1 -- start start 2023-10-18T10:41:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.700+0000 7f94b477d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9498004d00 0x7f94ac15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.700+0000 7f94b477d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94ac0b3950 0x7f94ac1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.700+0000 7f94b477d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94ac0b5060 0x7f94ac0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.700+0000 7f94b477d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0b3dd0 con 0x7f94ac0b3950 2023-10-18T10:41:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.700+0000 7f94b477d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac09e460 con 0x7f9498004d00 2023-10-18T10:41:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.700+0000 7f94b477d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0b3fc0 con 0x7f94ac0b5060 2023-10-18T10:41:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94ac0b3950 0x7f94ac1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94ac0b3950 0x7f94ac1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45188/0 (socket says 172.21.15.125:45188) 2023-10-18T10:41:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 -- 172.21.15.125:0/1015691587 learned_addr learned my addr 172.21.15.125:0/1015691587 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b2519640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9498004d00 0x7f94ac15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b2d1a640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94ac0b5060 0x7f94ac0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 -- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94ac0b5060 msgr2=0x7f94ac0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94ac0b5060 0x7f94ac0c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 -- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9498004d00 msgr2=0x7f94ac15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9498004d00 0x7f94ac15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 -- 172.21.15.125:0/1015691587 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0b42a0 con 0x7f94ac0b3950 2023-10-18T10:41:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b2d1a640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94ac0b5060 0x7f94ac0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b1d18640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94ac0b3950 0x7f94ac1582e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f949c011ab0 tx=0x7f949c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94a2ffd640 1 -- 172.21.15.125:0/1015691587 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c00fc70 con 0x7f94ac0b3950 2023-10-18T10:41:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94a2ffd640 1 -- 172.21.15.125:0/1015691587 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f949c00fdd0 con 0x7f94ac0b3950 2023-10-18T10:41:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac0b4580 con 0x7f94ac0b3950 2023-10-18T10:41:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94a2ffd640 1 -- 172.21.15.125:0/1015691587 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c019670 con 0x7f94ac0b3950 2023-10-18T10:41:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.701+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac0b4b40 con 0x7f94ac0b3950 2023-10-18T10:41:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.702+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0690e0 con 0x7f94ac0b3950 2023-10-18T10:41:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.703+0000 7f94a2ffd640 1 -- 172.21.15.125:0/1015691587 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f949c0197d0 con 0x7f94ac0b3950 2023-10-18T10:41:23.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.703+0000 7f94a2ffd640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f949006af90 0x7f949006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:23.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.703+0000 7f94a2ffd640 1 -- 172.21.15.125:0/1015691587 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f949c08a4f0 con 0x7f94ac0b3950 2023-10-18T10:41:23.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.703+0000 7f94b2519640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f949006af90 0x7f949006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:23.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.704+0000 7f94b2519640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f949006af90 0x7f949006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009b90 tx=0x7f94a8005f90 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:23.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.707+0000 7f94a2ffd640 1 -- 172.21.15.125:0/1015691587 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f949c0553d0 con 0x7f94ac0b3950 2023-10-18T10:41:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:23.884+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f94ac0d0b40 con 0x7f94ac0b3950 2023-10-18T10:41:25.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.006+0000 7f94a2ffd640 1 -- 172.21.15.125:0/1015691587 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) v1 ==== 97+0+0 (secure 0 0 0) 0x7f949c0589f0 con 0x7f94ac0b3950 2023-10-18T10:41:25.008 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T10:41:25.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f949006af90 msgr2=0x7f949006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f949006af90 0x7f949006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009b90 tx=0x7f94a8005f90 comp rx=0 tx=0).stop 2023-10-18T10:41:25.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94ac0b3950 msgr2=0x7f94ac1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94ac0b3950 0x7f94ac1582e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f949c011ab0 tx=0x7f949c011f70 comp rx=0 tx=0).stop 2023-10-18T10:41:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 shutdown_connections 2023-10-18T10:41:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f949006af90 0x7f949006d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94ac0b5060 0x7f94ac0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94ac0b3950 0x7f94ac1582e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 --2- 172.21.15.125:0/1015691587 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9498004d00 0x7f94ac15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 >> 172.21.15.125:0/1015691587 conn(0x7f94ac0d0d60 msgr2=0x7f94ac15a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:25.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 shutdown_connections 2023-10-18T10:41:25.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.009+0000 7f94b477d640 1 -- 172.21.15.125:0/1015691587 wait complete. 2023-10-18T10:41:25.024 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:41:25.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.129+0000 7f6bbb856640 1 -- 172.21.15.125:0/1739969022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb406af00 msgr2=0x7f6bb406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.129+0000 7f6bbb856640 1 --2- 172.21.15.125:0/1739969022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb406af00 0x7f6bb406b2e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009860 tx=0x7f6bac02bcd0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.129+0000 7f6bbb856640 1 -- 172.21.15.125:0/1739969022 shutdown_connections 2023-10-18T10:41:25.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.129+0000 7f6bbb856640 1 --2- 172.21.15.125:0/1739969022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb406af00 0x7f6bb406b2e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.129+0000 7f6bbb856640 1 -- 172.21.15.125:0/1739969022 >> 172.21.15.125:0/1739969022 conn(0x7f6bb4069930 msgr2=0x7f6bb4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:25.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.130+0000 7f6bbb856640 1 -- 172.21.15.125:0/1739969022 shutdown_connections 2023-10-18T10:41:25.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.130+0000 7f6bbb856640 1 -- 172.21.15.125:0/1739969022 wait complete. 2023-10-18T10:41:25.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.131+0000 7f6bbb856640 1 Processor -- start 2023-10-18T10:41:25.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.131+0000 7f6bbb856640 1 -- start start 2023-10-18T10:41:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.131+0000 7f6bbb856640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bb406af00 0x7f6bb4139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.131+0000 7f6bbb856640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bb40abc10 0x7f6bb40b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.131+0000 7f6bbb856640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb40c96b0 0x7f6bb40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.131+0000 7f6bbb856640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb4139b80 con 0x7f6bb40c96b0 2023-10-18T10:41:25.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.131+0000 7f6bbb856640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40b06a0 con 0x7f6bb40abc10 2023-10-18T10:41:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.131+0000 7f6bbb856640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40a8250 con 0x7f6bb406af00 2023-10-18T10:41:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb40c96b0 0x7f6bb40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb40c96b0 0x7f6bb40acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45218/0 (socket says 172.21.15.125:45218) 2023-10-18T10:41:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 -- 172.21.15.125:0/2455877387 learned_addr learned my addr 172.21.15.125:0/2455877387 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb95f2640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bb406af00 0x7f6bb4139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb8df1640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bb40abc10 0x7f6bb40b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 -- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bb406af00 msgr2=0x7f6bb4139640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bb406af00 0x7f6bb4139640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 -- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bb40abc10 msgr2=0x7f6bb40b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bb40abc10 0x7f6bb40b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 -- 172.21.15.125:0/2455877387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac009490 con 0x7f6bb40c96b0 2023-10-18T10:41:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb9df3640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb40c96b0 0x7f6bb40acd10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c430 tx=0x7f6ba400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb1ffb640 1 -- 172.21.15.125:0/2455877387 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4011790 con 0x7f6bb40c96b0 2023-10-18T10:41:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb1ffb640 1 -- 172.21.15.125:0/2455877387 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ba40119e0 con 0x7f6bb40c96b0 2023-10-18T10:41:25.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb40a8510 con 0x7f6bb40c96b0 2023-10-18T10:41:25.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bb1ffb640 1 -- 172.21.15.125:0/2455877387 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4011dd0 con 0x7f6bb40c96b0 2023-10-18T10:41:25.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.132+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40a8ad0 con 0x7f6bb40c96b0 2023-10-18T10:41:25.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.133+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb406c7e0 con 0x7f6bb40c96b0 2023-10-18T10:41:25.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.134+0000 7f6bb1ffb640 1 -- 172.21.15.125:0/2455877387 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6ba401c460 con 0x7f6bb40c96b0 2023-10-18T10:41:25.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.134+0000 7f6bb1ffb640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6ba006af60 0x7f6ba006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.134+0000 7f6bb1ffb640 1 -- 172.21.15.125:0/2455877387 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6ba40894b0 con 0x7f6bb40c96b0 2023-10-18T10:41:25.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.135+0000 7f6bb95f2640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6ba006af60 0x7f6ba006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.135+0000 7f6bb95f2640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6ba006af60 0x7f6ba006d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6bac036000 tx=0x7f6bac004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:25.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.139+0000 7f6bb1ffb640 1 -- 172.21.15.125:0/2455877387 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ba40547a0 con 0x7f6bb40c96b0 2023-10-18T10:41:25.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.369+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6bb40690e0 con 0x7f6bb40c96b0 2023-10-18T10:41:25.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.370+0000 7f6bb1ffb640 1 -- 172.21.15.125:0/2455877387 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6ba4057dc0 con 0x7f6bb40c96b0 2023-10-18T10:41:25.372 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:25.372 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:41:25.372 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T10:41:25.373 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.373 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T10:41:25.373 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T10:41:25.373 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.374 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:41:25.374 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:41:25.374 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:41:25.374 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.375 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.375 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.375 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.375 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T10:41:25.375 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T10:41:25.376 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.376 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:41:25.376 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:41:25.376 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:41:25.376 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.377 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.377 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.377 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.377 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T10:41:25.377 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T10:41:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:41:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:41:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:41:25.378 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.379 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.379 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.379 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.379 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T10:41:25.380 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T10:41:25.380 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.380 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:41:25.380 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:41:25.381 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:41:25.381 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.381 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.381 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.381 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.382 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T10:41:25.382 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T10:41:25.382 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.382 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:41:25.383 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:41:25.383 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.383 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.383 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.383 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.384 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T10:41:25.384 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T10:41:25.384 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.385 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:41:25.385 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:41:25.385 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.385 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.385 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.386 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.386 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T10:41:25.386 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T10:41:25.386 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.387 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:41:25.387 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:41:25.387 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.387 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.387 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.388 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.388 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T10:41:25.388 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T10:41:25.388 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.389 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:41:25.389 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:41:25.389 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.389 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.390 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.390 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.390 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T10:41:25.390 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T10:41:25.391 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.391 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:41:25.391 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:41:25.391 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.392 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.392 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.392 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.392 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T10:41:25.393 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T10:41:25.393 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.393 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:41:25.393 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:41:25.394 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.394 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.394 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.395 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.395 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T10:41:25.395 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T10:41:25.395 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.396 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:41:25.396 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:41:25.396 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.396 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.396 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.397 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.397 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T10:41:25.397 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T10:41:25.397 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.397 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:41:25.398 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:41:25.398 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.398 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.398 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.399 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.399 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T10:41:25.399 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T10:41:25.399 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.399 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:41:25.400 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T10:41:25.400 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T10:41:25.400 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.400 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.401 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.401 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T10:41:25.401 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T10:41:25.401 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.401 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:41:25.402 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:41:25.402 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:41:25.402 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:41:25.402 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.402 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.403 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.403 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T10:41:25.403 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T10:41:25.403 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.404 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:41:25.404 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:41:25.404 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:41:25.404 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:41:25.404 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.405 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.405 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.405 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T10:41:25.405 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T10:41:25.405 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.406 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:41:25.406 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:41:25.406 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:41:25.406 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:41:25.407 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.407 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.407 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.407 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T10:41:25.407 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T10:41:25.408 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.408 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:41:25.408 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:41:25.408 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:41:25.409 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.409 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.409 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.409 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.409 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T10:41:25.410 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T10:41:25.410 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.410 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:41:25.410 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:41:25.411 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.411 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.411 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:25.411 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:25.411 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T10:41:25.412 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T10:41:25.412 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:41:25.412 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:41:25.412 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:41:25.413 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:41:25.413 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.413 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:25.413 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T10:41:25.413 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:41:25.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.371+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6ba006af60 msgr2=0x7f6ba006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.371+0000 7f6bbb856640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6ba006af60 0x7f6ba006d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6bac036000 tx=0x7f6bac004730 comp rx=0 tx=0).stop 2023-10-18T10:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.371+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb40c96b0 msgr2=0x7f6bb40acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.371+0000 7f6bbb856640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb40c96b0 0x7f6bb40acd10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c430 tx=0x7f6ba400c8f0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.371+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 shutdown_connections 2023-10-18T10:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.371+0000 7f6bbb856640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6ba006af60 0x7f6ba006d420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.371+0000 7f6bbb856640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bb40abc10 0x7f6bb40b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.372+0000 7f6bbb856640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bb40c96b0 0x7f6bb40acd10 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.372+0000 7f6bbb856640 1 --2- 172.21.15.125:0/2455877387 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bb406af00 0x7f6bb4139640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.372+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 >> 172.21.15.125:0/2455877387 conn(0x7f6bb4069930 msgr2=0x7f6bb4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:25.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.372+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 shutdown_connections 2023-10-18T10:41:25.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.372+0000 7f6bbb856640 1 -- 172.21.15.125:0/2455877387 wait complete. 2023-10-18T10:41:25.419 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:25.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.520+0000 7f81a260e640 1 -- 172.21.15.125:0/1350611515 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c06af00 msgr2=0x7f819c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.520+0000 7f81a260e640 1 --2- 172.21.15.125:0/1350611515 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c06af00 0x7f819c06b2e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f818c005fd0 tx=0x7f818c02be10 comp rx=0 tx=0).stop 2023-10-18T10:41:25.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.520+0000 7f81a260e640 1 -- 172.21.15.125:0/1350611515 shutdown_connections 2023-10-18T10:41:25.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.520+0000 7f81a260e640 1 --2- 172.21.15.125:0/1350611515 >> v2:172.21.15.139:3300/0 conn(0x7f819c0abc10 0x7f819c0c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.520+0000 7f81a260e640 1 --2- 172.21.15.125:0/1350611515 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c06af00 0x7f819c06b2e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.520+0000 7f81a260e640 1 -- 172.21.15.125:0/1350611515 >> 172.21.15.125:0/1350611515 conn(0x7f819c069930 msgr2=0x7f819c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:25.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.520+0000 7f81a260e640 1 -- 172.21.15.125:0/1350611515 shutdown_connections 2023-10-18T10:41:25.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.520+0000 7f81a260e640 1 -- 172.21.15.125:0/1350611515 wait complete. 2023-10-18T10:41:25.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.521+0000 7f81a260e640 1 Processor -- start 2023-10-18T10:41:25.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.521+0000 7f81a260e640 1 -- start start 2023-10-18T10:41:25.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f81a260e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f819c06af00 0x7f819c139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f81a260e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f819c0abc10 0x7f819c070910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f81a260e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c0c96b0 0x7f819c0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f81a260e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0ac9c0 con 0x7f819c0abc10 2023-10-18T10:41:25.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f81a260e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c152810 con 0x7f819c06af00 2023-10-18T10:41:25.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f81a260e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0ac670 con 0x7f819c0c96b0 2023-10-18T10:41:25.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f819c0abc10 0x7f819c070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f819c0abc10 0x7f819c070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45230/0 (socket says 172.21.15.125:45230) 2023-10-18T10:41:25.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 -- 172.21.15.125:0/277085027 learned_addr learned my addr 172.21.15.125:0/277085027 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:25.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819bfff640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f819c06af00 0x7f819c139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f81a0bab640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c0c96b0 0x7f819c0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 -- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c0c96b0 msgr2=0x7f819c0aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c0c96b0 0x7f819c0aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 -- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f819c06af00 msgr2=0x7f819c139800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f819c06af00 0x7f819c139800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 -- 172.21.15.125:0/277085027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c009560 con 0x7f819c0abc10 2023-10-18T10:41:25.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f81a0bab640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c0c96b0 0x7f819c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:25.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f819b7fe640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f819c0abc10 0x7f819c070910 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f819000c400 tx=0x7f819000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:25.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.522+0000 7f8198ff9640 1 -- 172.21.15.125:0/277085027 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81900052d0 con 0x7f819c0abc10 2023-10-18T10:41:25.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.523+0000 7f8198ff9640 1 -- 172.21.15.125:0/277085027 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f81900035f0 con 0x7f819c0abc10 2023-10-18T10:41:25.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.523+0000 7f8198ff9640 1 -- 172.21.15.125:0/277085027 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190003a00 con 0x7f819c0abc10 2023-10-18T10:41:25.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.523+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c150650 con 0x7f819c0abc10 2023-10-18T10:41:25.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.523+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c152d40 con 0x7f819c0abc10 2023-10-18T10:41:25.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.524+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c0acb00 con 0x7f819c0abc10 2023-10-18T10:41:25.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.524+0000 7f8198ff9640 1 -- 172.21.15.125:0/277085027 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8190003b60 con 0x7f819c0abc10 2023-10-18T10:41:25.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.524+0000 7f8198ff9640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f818806af60 0x7f818806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.524+0000 7f8198ff9640 1 -- 172.21.15.125:0/277085027 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8190088d60 con 0x7f819c0abc10 2023-10-18T10:41:25.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.525+0000 7f819bfff640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f818806af60 0x7f818806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.525+0000 7f819bfff640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f818806af60 0x7f818806d420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f818c005fd0 tx=0x7f818c004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:25.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.528+0000 7f8198ff9640 1 -- 172.21.15.125:0/277085027 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f819c0acb00 con 0x7f819c0abc10 2023-10-18T10:41:25.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.695+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f819c0acb00 con 0x7f819c0abc10 2023-10-18T10:41:25.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.696+0000 7f8198ff9640 1 -- 172.21.15.125:0/277085027 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+13112 (secure 0 0 0) 0x7f819c0acb00 con 0x7f819c0abc10 2023-10-18T10:41:25.698 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:25.699 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":108,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:25.005747+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:25.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f818806af60 msgr2=0x7f818806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f818806af60 0x7f818806d420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f818c005fd0 tx=0x7f818c004730 comp rx=0 tx=0).stop 2023-10-18T10:41:25.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f819c0abc10 msgr2=0x7f819c070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f819c0abc10 0x7f819c070910 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f819000c400 tx=0x7f819000c8c0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 shutdown_connections 2023-10-18T10:41:25.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f818806af60 0x7f818806d420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819c0c96b0 0x7f819c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f819c0abc10 0x7f819c070910 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 --2- 172.21.15.125:0/277085027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f819c06af00 0x7f819c139800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 >> 172.21.15.125:0/277085027 conn(0x7f819c069930 msgr2=0x7f819c0cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:25.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 shutdown_connections 2023-10-18T10:41:25.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.698+0000 7f81a260e640 1 -- 172.21.15.125:0/277085027 wait complete. 2023-10-18T10:41:25.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:41:25.711 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:41:25.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.815+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/630478377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e806cf30 msgr2=0x7fc9e806bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.815+0000 7fc9eeaf2640 1 --2- 172.21.15.125:0/630478377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e806cf30 0x7fc9e806bca0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc005fd0 tx=0x7fc9dc02bd90 comp rx=0 tx=0).stop 2023-10-18T10:41:25.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.815+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/630478377 shutdown_connections 2023-10-18T10:41:25.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.815+0000 7fc9eeaf2640 1 --2- 172.21.15.125:0/630478377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e806cf30 0x7fc9e806bca0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.815+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/630478377 >> 172.21.15.125:0/630478377 conn(0x7fc9e8069930 msgr2=0x7fc9e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:25.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.815+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/630478377 shutdown_connections 2023-10-18T10:41:25.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.815+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/630478377 wait complete. 2023-10-18T10:41:25.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.816+0000 7fc9eeaf2640 1 Processor -- start 2023-10-18T10:41:25.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.816+0000 7fc9eeaf2640 1 -- start start 2023-10-18T10:41:25.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9eeaf2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9e806cf30 0x7fc9e813e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9eeaf2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e80afa20 0x7fc9e80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9eeaf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e8157220 0x7fc9e81e5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9eeaf2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e806b880 con 0x7fc9e80afa20 2023-10-18T10:41:25.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9eeaf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80cc170 con 0x7fc9e8157220 2023-10-18T10:41:25.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9eeaf2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e813e820 con 0x7fc9e806cf30 2023-10-18T10:41:25.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9e7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e80afa20 0x7fc9e80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9e7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e80afa20 0x7fc9e80cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45252/0 (socket says 172.21.15.125:45252) 2023-10-18T10:41:25.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9e7fff640 1 -- 172.21.15.125:0/1457875079 learned_addr learned my addr 172.21.15.125:0/1457875079 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:25.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9ed08f640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e8157220 0x7fc9e81e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9ec88e640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9e806cf30 0x7fc9e813e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9e7fff640 1 -- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9e806cf30 msgr2=0x7fc9e813e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9e7fff640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9e806cf30 0x7fc9e813e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9e7fff640 1 -- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e8157220 msgr2=0x7fc9e81e5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9e7fff640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e8157220 0x7fc9e81e5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9e7fff640 1 -- 172.21.15.125:0/1457875079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc009560 con 0x7fc9e80afa20 2023-10-18T10:41:25.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.817+0000 7fc9ed08f640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e8157220 0x7fc9e81e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:25.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.818+0000 7fc9ec88e640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9e806cf30 0x7fc9e813e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:25.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.818+0000 7fc9e7fff640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e80afa20 0x7fc9e80cb790 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc006010 tx=0x7fc9dc02dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:25.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.818+0000 7fc9e57fa640 1 -- 172.21.15.125:0/1457875079 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc003800 con 0x7fc9e80afa20 2023-10-18T10:41:25.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.818+0000 7fc9e57fa640 1 -- 172.21.15.125:0/1457875079 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc9dc003a50 con 0x7fc9e80afa20 2023-10-18T10:41:25.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.818+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e81549b0 con 0x7fc9e80afa20 2023-10-18T10:41:25.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.818+0000 7fc9e57fa640 1 -- 172.21.15.125:0/1457875079 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc003de0 con 0x7fc9e80afa20 2023-10-18T10:41:25.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.818+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e8154f70 con 0x7fc9e80afa20 2023-10-18T10:41:25.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.819+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e80690e0 con 0x7fc9e80afa20 2023-10-18T10:41:25.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.819+0000 7fc9e57fa640 1 -- 172.21.15.125:0/1457875079 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc9dc03e450 con 0x7fc9e80afa20 2023-10-18T10:41:25.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.820+0000 7fc9e57fa640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fc9d406af60 0x7fc9d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:25.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.820+0000 7fc9e57fa640 1 -- 172.21.15.125:0/1457875079 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc9dc0ac280 con 0x7fc9e80afa20 2023-10-18T10:41:25.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.820+0000 7fc9ec88e640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fc9d406af60 0x7fc9d406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:25.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.820+0000 7fc9ec88e640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fc9d406af60 0x7fc9d406d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8000f60 tx=0x7fc9d8009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:25.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.824+0000 7fc9e57fa640 1 -- 172.21.15.125:0/1457875079 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9dc0775e0 con 0x7fc9e80afa20 2023-10-18T10:41:25.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.990+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9e80ca2e0 con 0x7fc9e80afa20 2023-10-18T10:41:25.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.991+0000 7fc9e57fa640 1 -- 172.21.15.125:0/1457875079 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9dc0430a0 con 0x7fc9e80afa20 2023-10-18T10:41:25.993 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:41:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fc9d406af60 msgr2=0x7fc9d406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fc9d406af60 0x7fc9d406d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8000f60 tx=0x7fc9d8009040 comp rx=0 tx=0).stop 2023-10-18T10:41:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e80afa20 msgr2=0x7fc9e80cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e80afa20 0x7fc9e80cb790 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc006010 tx=0x7fc9dc02dd60 comp rx=0 tx=0).stop 2023-10-18T10:41:25.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 shutdown_connections 2023-10-18T10:41:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fc9d406af60 0x7fc9d406d420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e8157220 0x7fc9e81e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9e80afa20 0x7fc9e80cb790 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 --2- 172.21.15.125:0/1457875079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9e806cf30 0x7fc9e813e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:25.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 >> 172.21.15.125:0/1457875079 conn(0x7fc9e8069930 msgr2=0x7fc9e8159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:25.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 shutdown_connections 2023-10-18T10:41:25.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:25.993+0000 7fc9eeaf2640 1 -- 172.21.15.125:0/1457875079 wait complete. 2023-10-18T10:41:26.005 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:41:26.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 -- 172.21.15.125:0/1180281428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40ceb40 msgr2=0x7fdfd40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:26.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 --2- 172.21.15.125:0/1180281428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40ceb40 0x7fdfd40b0650 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406c510 tx=0x7fdfcc02bd10 comp rx=0 tx=0).stop 2023-10-18T10:41:26.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 -- 172.21.15.125:0/1180281428 shutdown_connections 2023-10-18T10:41:26.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 --2- 172.21.15.125:0/1180281428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40ceb40 0x7fdfd40b0650 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 --2- 172.21.15.125:0/1180281428 >> v2:172.21.15.139:3300/0 conn(0x7fdfd415b2a0 0x7fdfd40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 --2- 172.21.15.125:0/1180281428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd40cf340 0x7fdfd40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 -- 172.21.15.125:0/1180281428 >> 172.21.15.125:0/1180281428 conn(0x7fdfd40d0d60 msgr2=0x7fdfd40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:26.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 -- 172.21.15.125:0/1180281428 shutdown_connections 2023-10-18T10:41:26.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.106+0000 7fdfdc797640 1 -- 172.21.15.125:0/1180281428 wait complete. 2023-10-18T10:41:26.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.107+0000 7fdfdc797640 1 Processor -- start 2023-10-18T10:41:26.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.107+0000 7fdfdc797640 1 -- start start 2023-10-18T10:41:26.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfdc797640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40ceb40 0x7fdfd40ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:26.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfdc797640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40cf340 0x7fdfd40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:26.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfdc797640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd415b2a0 0x7fdfd415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:26.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfdc797640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40b5020 con 0x7fdfd40cf340 2023-10-18T10:41:26.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfdc797640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40b43a0 con 0x7fdfd40ceb40 2023-10-18T10:41:26.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfdc797640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40b4680 con 0x7fdfd415b2a0 2023-10-18T10:41:26.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfd9d32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40cf340 0x7fdfd40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:26.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfd9d32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40cf340 0x7fdfd40b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45266/0 (socket says 172.21.15.125:45266) 2023-10-18T10:41:26.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfd9d32640 1 -- 172.21.15.125:0/372681335 learned_addr learned my addr 172.21.15.125:0/372681335 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:26.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfdad34640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd415b2a0 0x7fdfd415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:26.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfda533640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40ceb40 0x7fdfd40ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:26.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfd9d32640 1 -- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd415b2a0 msgr2=0x7fdfd415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:26.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfd9d32640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd415b2a0 0x7fdfd415dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfd9d32640 1 -- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40ceb40 msgr2=0x7fdfd40ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:26.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfd9d32640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40ceb40 0x7fdfd40ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfd9d32640 1 -- 172.21.15.125:0/372681335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc009560 con 0x7fdfd40cf340 2023-10-18T10:41:26.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.108+0000 7fdfda533640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40ceb40 0x7fdfd40ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:26.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.109+0000 7fdfd9d32640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40cf340 0x7fdfd40b3950 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c970 tx=0x7fdfc400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:26.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.109+0000 7fdfcaffd640 1 -- 172.21.15.125:0/372681335 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4007bb0 con 0x7fdfd40cf340 2023-10-18T10:41:26.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.109+0000 7fdfcaffd640 1 -- 172.21.15.125:0/372681335 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdfc4007e00 con 0x7fdfd40cf340 2023-10-18T10:41:26.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.109+0000 7fdfcaffd640 1 -- 172.21.15.125:0/372681335 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400f6b0 con 0x7fdfd40cf340 2023-10-18T10:41:26.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.109+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd40b4960 con 0x7fdfd40cf340 2023-10-18T10:41:26.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.109+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd41e60d0 con 0x7fdfd40cf340 2023-10-18T10:41:26.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.110+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd40690e0 con 0x7fdfd40cf340 2023-10-18T10:41:26.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.110+0000 7fdfcaffd640 1 -- 172.21.15.125:0/372681335 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdfc400f810 con 0x7fdfd40cf340 2023-10-18T10:41:26.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.110+0000 7fdfcaffd640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fdfb806af60 0x7fdfb806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:26.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.111+0000 7fdfcaffd640 1 -- 172.21.15.125:0/372681335 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdfc4089e10 con 0x7fdfd40cf340 2023-10-18T10:41:26.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.111+0000 7fdfda533640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fdfb806af60 0x7fdfb806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:26.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.111+0000 7fdfda533640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fdfb806af60 0x7fdfb806d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406b250 tx=0x7fdfd00093c0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:26.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.115+0000 7fdfcaffd640 1 -- 172.21.15.125:0/372681335 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfc4054f40 con 0x7fdfd40cf340 2023-10-18T10:41:26.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.284+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdfd40b6bf0 con 0x7fdfd40cf340 2023-10-18T10:41:26.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.284+0000 7fdfcaffd640 1 -- 172.21.15.125:0/372681335 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdfc4058560 con 0x7fdfd40cf340 2023-10-18T10:41:26.287 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:26.287 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:41:26.287 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:26.287 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:41:26.287 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:41:26.288 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:26.288 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:41:26.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fdfb806af60 msgr2=0x7fdfb806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:26.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fdfb806af60 0x7fdfb806d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406b250 tx=0x7fdfd00093c0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40cf340 msgr2=0x7fdfd40b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:26.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40cf340 0x7fdfd40b3950 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c970 tx=0x7fdfc400ce30 comp rx=0 tx=0).stop 2023-10-18T10:41:26.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 shutdown_connections 2023-10-18T10:41:26.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fdfb806af60 0x7fdfb806d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd415b2a0 0x7fdfd415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfd40cf340 0x7fdfd40b3950 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 --2- 172.21.15.125:0/372681335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40ceb40 0x7fdfd40ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 >> 172.21.15.125:0/372681335 conn(0x7fdfd40d0d60 msgr2=0x7fdfd406a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:26.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 shutdown_connections 2023-10-18T10:41:26.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.286+0000 7fdfdc797640 1 -- 172.21.15.125:0/372681335 wait complete. 2023-10-18T10:41:26.298 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:41:26.299 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:41:26.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.408+0000 7f48a2f4c640 1 -- 172.21.15.125:0/1311311872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4884003970 msgr2=0x7f4884003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:26.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.408+0000 7f48a2f4c640 1 --2- 172.21.15.125:0/1311311872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4884003970 0x7f4884003d90 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f488c009b50 tx=0x7f48840057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.408+0000 7f48a2f4c640 1 -- 172.21.15.125:0/1311311872 shutdown_connections 2023-10-18T10:41:26.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.408+0000 7f48a2f4c640 1 --2- 172.21.15.125:0/1311311872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4884003970 0x7f4884003d90 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.408+0000 7f48a2f4c640 1 -- 172.21.15.125:0/1311311872 >> 172.21.15.125:0/1311311872 conn(0x7f489c069930 msgr2=0x7f489c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:26.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.408+0000 7f48a2f4c640 1 -- 172.21.15.125:0/1311311872 shutdown_connections 2023-10-18T10:41:26.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.408+0000 7f48a2f4c640 1 -- 172.21.15.125:0/1311311872 wait complete. 2023-10-18T10:41:26.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.409+0000 7f48a2f4c640 1 Processor -- start 2023-10-18T10:41:26.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.409+0000 7f48a2f4c640 1 -- start start 2023-10-18T10:41:26.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a2f4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4884003970 0x7f489c159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:26.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a2f4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f489c15eae0 0x7f489c166dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:26.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a2f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f489c167310 0x7f489c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:26.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a2f4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c166050 con 0x7f489c15eae0 2023-10-18T10:41:26.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a2f4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c1662f0 con 0x7f489c167310 2023-10-18T10:41:26.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a2f4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c1665d0 con 0x7f4884003970 2023-10-18T10:41:26.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f489c15eae0 0x7f489c166dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:26.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f489c15eae0 0x7f489c166dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:45282/0 (socket says 172.21.15.125:45282) 2023-10-18T10:41:26.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 -- 172.21.15.125:0/3936162182 learned_addr learned my addr 172.21.15.125:0/3936162182 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:26.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a14e9640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f489c167310 0x7f489c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:26.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a0ce8640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4884003970 0x7f489c159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:26.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 -- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4884003970 msgr2=0x7f489c159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:26.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4884003970 0x7f489c159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 -- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f489c167310 msgr2=0x7f489c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:26.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f489c167310 0x7f489c15af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:26.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 -- 172.21.15.125:0/3936162182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c1668b0 con 0x7f489c15eae0 2023-10-18T10:41:26.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a0ce8640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4884003970 0x7f489c159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:26.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a14e9640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f489c167310 0x7f489c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:26.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f489bfff640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f489c15eae0 0x7f489c166dd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f48900011e0 tx=0x7f48900039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:26.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48997fa640 1 -- 172.21.15.125:0/3936162182 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890011900 con 0x7f489c15eae0 2023-10-18T10:41:26.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48997fa640 1 -- 172.21.15.125:0/3936162182 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4890011a60 con 0x7f489c15eae0 2023-10-18T10:41:26.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c15bdc0 con 0x7f489c15eae0 2023-10-18T10:41:26.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48997fa640 1 -- 172.21.15.125:0/3936162182 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890011e70 con 0x7f489c15eae0 2023-10-18T10:41:26.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.410+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c15ef60 con 0x7f489c15eae0 2023-10-18T10:41:26.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.411+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c166aa0 con 0x7f489c15eae0 2023-10-18T10:41:26.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.412+0000 7f48997fa640 1 -- 172.21.15.125:0/3936162182 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f48900103c0 con 0x7f489c15eae0 2023-10-18T10:41:26.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.412+0000 7f48997fa640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f488806af90 0x7f488806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:26.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.412+0000 7f48997fa640 1 -- 172.21.15.125:0/3936162182 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f489008a390 con 0x7f489c15eae0 2023-10-18T10:41:26.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.412+0000 7f48a0ce8640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f488806af90 0x7f488806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:26.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.413+0000 7f48a0ce8640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f488806af90 0x7f488806d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f488c036000 tx=0x7f488c035040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:26.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.416+0000 7f48997fa640 1 -- 172.21.15.125:0/3936162182 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48900554c0 con 0x7f489c15eae0 2023-10-18T10:41:26.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:26.583+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f489c0690e0 con 0x7f489c15eae0 2023-10-18T10:41:27.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.413+0000 7f48997fa640 1 -- 172.21.15.125:0/3936162182 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) v1 ==== 138+0+0 (secure 0 0 0) 0x7f48900150b0 con 0x7f489c15eae0 2023-10-18T10:41:27.415 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:41:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f488806af90 msgr2=0x7f488806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f488806af90 0x7f488806d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f488c036000 tx=0x7f488c035040 comp rx=0 tx=0).stop 2023-10-18T10:41:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f489c15eae0 msgr2=0x7f489c166dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:27.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f489c15eae0 0x7f489c166dd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f48900011e0 tx=0x7f48900039c0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 shutdown_connections 2023-10-18T10:41:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f488806af90 0x7f488806d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f489c167310 0x7f489c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f489c15eae0 0x7f489c166dd0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 --2- 172.21.15.125:0/3936162182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4884003970 0x7f489c159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 >> 172.21.15.125:0/3936162182 conn(0x7f489c069930 msgr2=0x7f489c15d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:27.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 shutdown_connections 2023-10-18T10:41:27.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.416+0000 7f48a2f4c640 1 -- 172.21.15.125:0/3936162182 wait complete. 2023-10-18T10:41:27.434 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:41:27.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.546+0000 7f8520439640 1 -- 172.21.15.125:0/2535554942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85180cf340 msgr2=0x7f85180d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:27.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.546+0000 7f8520439640 1 --2- 172.21.15.125:0/2535554942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85180cf340 0x7f85180d1910 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f851815b680 tx=0x7f851402c1d0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.546+0000 7f8520439640 1 -- 172.21.15.125:0/2535554942 shutdown_connections 2023-10-18T10:41:27.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.546+0000 7f8520439640 1 --2- 172.21.15.125:0/2535554942 >> v2:172.21.15.139:3300/0 conn(0x7f85180ceb40 0x7f85180b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.546+0000 7f8520439640 1 --2- 172.21.15.125:0/2535554942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85180cf340 0x7f85180d1910 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.546+0000 7f8520439640 1 -- 172.21.15.125:0/2535554942 >> 172.21.15.125:0/2535554942 conn(0x7f85180d0d60 msgr2=0x7f85180d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:27.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.546+0000 7f8520439640 1 -- 172.21.15.125:0/2535554942 shutdown_connections 2023-10-18T10:41:27.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.546+0000 7f8520439640 1 -- 172.21.15.125:0/2535554942 wait complete. 2023-10-18T10:41:27.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.547+0000 7f8520439640 1 Processor -- start 2023-10-18T10:41:27.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.547+0000 7f8520439640 1 -- start start 2023-10-18T10:41:27.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f8520439640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85180ceb40 0x7f8518156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:27.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f8520439640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85180cf340 0x7f851815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:27.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f8520439640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f851815b2a0 0x7f85180ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:27.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f8520439640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c9950 con 0x7f851815b2a0 2023-10-18T10:41:27.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f8520439640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c8cd0 con 0x7f85180ceb40 2023-10-18T10:41:27.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f8520439640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c8fb0 con 0x7f85180cf340 2023-10-18T10:41:27.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e9d6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f851815b2a0 0x7f85180ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:27.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e9d6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f851815b2a0 0x7f85180ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50752/0 (socket says 172.21.15.125:50752) 2023-10-18T10:41:27.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e9d6640 1 -- 172.21.15.125:0/1201537576 learned_addr learned my addr 172.21.15.125:0/1201537576 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:27.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e1d5640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85180ceb40 0x7f8518156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:27.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851d9d4640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85180cf340 0x7f851815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:27.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e9d6640 1 -- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85180cf340 msgr2=0x7f851815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:27.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e9d6640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85180cf340 0x7f851815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e9d6640 1 -- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85180ceb40 msgr2=0x7f8518156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:27.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e9d6640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85180ceb40 0x7f8518156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:27.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.548+0000 7f851e9d6640 1 -- 172.21.15.125:0/1201537576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8514009900 con 0x7f851815b2a0 2023-10-18T10:41:27.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.549+0000 7f851e9d6640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f851815b2a0 0x7f85180ae1f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f851000d750 tx=0x7f851000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:27.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.549+0000 7f850effd640 1 -- 172.21.15.125:0/1201537576 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510009bd0 con 0x7f851815b2a0 2023-10-18T10:41:27.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.549+0000 7f850effd640 1 -- 172.21.15.125:0/1201537576 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8510009e20 con 0x7f851815b2a0 2023-10-18T10:41:27.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.549+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180c9290 con 0x7f851815b2a0 2023-10-18T10:41:27.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.549+0000 7f850effd640 1 -- 172.21.15.125:0/1201537576 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510005280 con 0x7f851815b2a0 2023-10-18T10:41:27.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.549+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8518156660 con 0x7f851815b2a0 2023-10-18T10:41:27.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.550+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85180c9a90 con 0x7f851815b2a0 2023-10-18T10:41:27.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.551+0000 7f850effd640 1 -- 172.21.15.125:0/1201537576 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f85100053e0 con 0x7f851815b2a0 2023-10-18T10:41:27.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.551+0000 7f850effd640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f84fc06af60 0x7f84fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:27.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.551+0000 7f850effd640 1 -- 172.21.15.125:0/1201537576 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f8510089ad0 con 0x7f851815b2a0 2023-10-18T10:41:27.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.551+0000 7f851e1d5640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f84fc06af60 0x7f84fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:27.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.552+0000 7f851e1d5640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f84fc06af60 0x7f84fc06d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8514036000 tx=0x7f8514004ab0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:27.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.555+0000 7f850effd640 1 -- 172.21.15.125:0/1201537576 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8510054b90 con 0x7f851815b2a0 2023-10-18T10:41:27.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:27.723+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f85180c9a90 con 0x7f851815b2a0 2023-10-18T10:41:28.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.417+0000 7f850effd640 1 -- 172.21.15.125:0/1201537576 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) v1 ==== 170+0+0 (secure 0 0 0) 0x7f85180c9a90 con 0x7f851815b2a0 2023-10-18T10:41:28.420 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:41:28.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f84fc06af60 msgr2=0x7f84fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:28.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f84fc06af60 0x7f84fc06d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8514036000 tx=0x7f8514004ab0 comp rx=0 tx=0).stop 2023-10-18T10:41:28.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f851815b2a0 msgr2=0x7f85180ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:28.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f851815b2a0 0x7f85180ae1f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f851000d750 tx=0x7f851000dc10 comp rx=0 tx=0).stop 2023-10-18T10:41:28.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 shutdown_connections 2023-10-18T10:41:28.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f84fc06af60 0x7f84fc06d420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:28.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f851815b2a0 0x7f85180ae1f0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:28.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85180cf340 0x7f851815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:28.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 --2- 172.21.15.125:0/1201537576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85180ceb40 0x7f8518156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:28.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 >> 172.21.15.125:0/1201537576 conn(0x7f85180d0d60 msgr2=0x7f85181e4380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:28.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.419+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 shutdown_connections 2023-10-18T10:41:28.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.420+0000 7f8520439640 1 -- 172.21.15.125:0/1201537576 wait complete. 2023-10-18T10:41:28.445 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:41:28.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.551+0000 7f853660e640 1 -- 172.21.15.125:0/1409201766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85300b0c40 msgr2=0x7f853006bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:28.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.551+0000 7f853660e640 1 --2- 172.21.15.125:0/1409201766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85300b0c40 0x7f853006bba0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8528005fd0 tx=0x7f852802d410 comp rx=0 tx=0).stop 2023-10-18T10:41:28.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.551+0000 7f853660e640 1 -- 172.21.15.125:0/1409201766 shutdown_connections 2023-10-18T10:41:28.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.551+0000 7f853660e640 1 --2- 172.21.15.125:0/1409201766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85300b0c40 0x7f853006bba0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:28.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.551+0000 7f853660e640 1 -- 172.21.15.125:0/1409201766 >> 172.21.15.125:0/1409201766 conn(0x7f8530069930 msgr2=0x7f8530069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:28.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.551+0000 7f853660e640 1 -- 172.21.15.125:0/1409201766 shutdown_connections 2023-10-18T10:41:28.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.551+0000 7f853660e640 1 -- 172.21.15.125:0/1409201766 wait complete. 2023-10-18T10:41:28.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.552+0000 7f853660e640 1 Processor -- start 2023-10-18T10:41:28.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.552+0000 7f853660e640 1 -- start start 2023-10-18T10:41:28.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f853660e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85300afa20 0x7f85300c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:28.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f853660e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85300b0c40 0x7f85301f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:28.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f853660e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8530157220 0x7f85301f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:28.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f853660e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8530156130 con 0x7f8530157220 2023-10-18T10:41:28.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f853660e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8530155520 con 0x7f85300afa20 2023-10-18T10:41:28.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f853660e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8530155800 con 0x7f85300b0c40 2023-10-18T10:41:28.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8530157220 0x7f85301f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:28.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8530157220 0x7f85301f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50768/0 (socket says 172.21.15.125:50768) 2023-10-18T10:41:28.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 -- 172.21.15.125:0/1637747624 learned_addr learned my addr 172.21.15.125:0/1637747624 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:28.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f852f7fe640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85300b0c40 0x7f85301f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:28.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f852ffff640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85300afa20 0x7f85300c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:28.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 -- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85300b0c40 msgr2=0x7f85301f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:28.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85300b0c40 0x7f85301f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:28.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 -- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85300afa20 msgr2=0x7f85300c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:28.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85300afa20 0x7f85300c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:28.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 -- 172.21.15.125:0/1637747624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852800a950 con 0x7f8530157220 2023-10-18T10:41:28.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f8534bab640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8530157220 0x7f85301f4af0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f852800ade0 tx=0x7f8528032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:28.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f852cff9640 1 -- 172.21.15.125:0/1637747624 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528004240 con 0x7f8530157220 2023-10-18T10:41:28.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f852cff9640 1 -- 172.21.15.125:0/1637747624 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8528004490 con 0x7f8530157220 2023-10-18T10:41:28.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8530155ae0 con 0x7f8530157220 2023-10-18T10:41:28.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f852cff9640 1 -- 172.21.15.125:0/1637747624 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528004800 con 0x7f8530157220 2023-10-18T10:41:28.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.553+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8530155140 con 0x7f8530157220 2023-10-18T10:41:28.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.554+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8530145520 con 0x7f8530157220 2023-10-18T10:41:28.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.555+0000 7f852cff9640 1 -- 172.21.15.125:0/1637747624 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f852803b070 con 0x7f8530157220 2023-10-18T10:41:28.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.555+0000 7f852cff9640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f851c06af60 0x7f851c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:28.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.555+0000 7f852cff9640 1 -- 172.21.15.125:0/1637747624 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f85280ad180 con 0x7f8530157220 2023-10-18T10:41:28.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.555+0000 7f852ffff640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f851c06af60 0x7f851c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:28.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.556+0000 7f852ffff640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f851c06af60 0x7f851c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8520000f60 tx=0x7f8520009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:28.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.560+0000 7f852cff9640 1 -- 172.21.15.125:0/1637747624 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8530145520 con 0x7f8530157220 2023-10-18T10:41:28.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:28.759+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8530145520 con 0x7f8530157220 2023-10-18T10:41:29.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.444+0000 7f852cff9640 1 -- 172.21.15.125:0/1637747624 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v39) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8530145520 con 0x7f8530157220 2023-10-18T10:41:29.448 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 13 and data pool 14 2023-10-18T10:41:29.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.452+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f851c06af60 msgr2=0x7f851c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.452+0000 7f853660e640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f851c06af60 0x7f851c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8520000f60 tx=0x7f8520009040 comp rx=0 tx=0).stop 2023-10-18T10:41:29.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.452+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8530157220 msgr2=0x7f85301f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.453+0000 7f853660e640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8530157220 0x7f85301f4af0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f852800ade0 tx=0x7f8528032c10 comp rx=0 tx=0).stop 2023-10-18T10:41:29.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.455+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 shutdown_connections 2023-10-18T10:41:29.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.455+0000 7f853660e640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f851c06af60 0x7f851c06d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.455+0000 7f853660e640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8530157220 0x7f85301f4af0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.455+0000 7f853660e640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85300b0c40 0x7f85301f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.455+0000 7f853660e640 1 --2- 172.21.15.125:0/1637747624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85300afa20 0x7f85300c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.455+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 >> 172.21.15.125:0/1637747624 conn(0x7f8530069930 msgr2=0x7f8530159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:29.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.455+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 shutdown_connections 2023-10-18T10:41:29.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.456+0000 7f853660e640 1 -- 172.21.15.125:0/1637747624 wait complete. 2023-10-18T10:41:29.472 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:29.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.574+0000 7f2fda0a2640 1 -- 172.21.15.125:0/4133446250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 msgr2=0x7f2fc0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.574+0000 7f2fda0a2640 1 --2- 172.21.15.125:0/4133446250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 0x7f2fc0004c40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009b50 tx=0x7f2fc00057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.574+0000 7f2fd3fff640 1 -- 172.21.15.125:0/4133446250 reap_dead start 2023-10-18T10:41:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.574+0000 7f2fda0a2640 1 -- 172.21.15.125:0/4133446250 shutdown_connections 2023-10-18T10:41:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.574+0000 7f2fda0a2640 1 -- 172.21.15.125:0/4133446250 >> 172.21.15.125:0/4133446250 conn(0x7f2fd4069930 msgr2=0x7f2fd4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.574+0000 7f2fda0a2640 1 -- 172.21.15.125:0/4133446250 shutdown_connections 2023-10-18T10:41:29.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.574+0000 7f2fda0a2640 1 -- 172.21.15.125:0/4133446250 wait complete. 2023-10-18T10:41:29.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.575+0000 7f2fda0a2640 1 Processor -- start 2023-10-18T10:41:29.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.575+0000 7f2fda0a2640 1 -- start start 2023-10-18T10:41:29.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fda0a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 0x7f2fd41362f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:29.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fda0a2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40afbc0 0x7f2fd40b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:29.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fda0a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd40b0660 0x7f2fd40b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:29.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fda0a2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40b1020 con 0x7f2fc00047c0 2023-10-18T10:41:29.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fda0a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4139310 con 0x7f2fd40b0660 2023-10-18T10:41:29.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fda0a2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4138a70 con 0x7f2fd40afbc0 2023-10-18T10:41:29.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 0x7f2fd41362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 0x7f2fd41362f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50788/0 (socket says 172.21.15.125:50788) 2023-10-18T10:41:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 -- 172.21.15.125:0/419711330 learned_addr learned my addr 172.21.15.125:0/419711330 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd2ffd640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40afbc0 0x7f2fd40b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd3fff640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd40b0660 0x7f2fd40b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:29.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 -- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40afbc0 msgr2=0x7f2fd40b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40afbc0 0x7f2fd40b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 -- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd40b0660 msgr2=0x7f2fd40b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd40b0660 0x7f2fd40b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 -- 172.21.15.125:0/419711330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4138d50 con 0x7f2fc00047c0 2023-10-18T10:41:29.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd2ffd640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40afbc0 0x7f2fd40b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:29.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd3fff640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd40b0660 0x7f2fd40b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fd37fe640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 0x7f2fd41362f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e590 tx=0x7f2fbc00ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.576+0000 7f2fcbfff640 1 -- 172.21.15.125:0/419711330 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc010890 con 0x7f2fc00047c0 2023-10-18T10:41:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.577+0000 7f2fcbfff640 1 -- 172.21.15.125:0/419711330 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2fbc010ae0 con 0x7f2fc00047c0 2023-10-18T10:41:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.577+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd4139030 con 0x7f2fc00047c0 2023-10-18T10:41:29.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.577+0000 7f2fcbfff640 1 -- 172.21.15.125:0/419711330 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc011460 con 0x7f2fc00047c0 2023-10-18T10:41:29.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.577+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd4150a50 con 0x7f2fc00047c0 2023-10-18T10:41:29.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.578+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd40690e0 con 0x7f2fc00047c0 2023-10-18T10:41:29.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.578+0000 7f2fcbfff640 1 -- 172.21.15.125:0/419711330 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2fbc0115c0 con 0x7f2fc00047c0 2023-10-18T10:41:29.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.578+0000 7f2fcbfff640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f2fb406af60 0x7f2fb406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:29.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.578+0000 7f2fcbfff640 1 -- 172.21.15.125:0/419711330 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2fbc08a170 con 0x7f2fc00047c0 2023-10-18T10:41:29.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.579+0000 7f2fd2ffd640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f2fb406af60 0x7f2fb406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:29.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.579+0000 7f2fd2ffd640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f2fb406af60 0x7f2fb406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009b50 tx=0x7f2fc40046c0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:29.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.583+0000 7f2fcbfff640 1 -- 172.21.15.125:0/419711330 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fbc054f50 con 0x7f2fc00047c0 2023-10-18T10:41:29.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.748+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fd406ff80 con 0x7f2fc00047c0 2023-10-18T10:41:29.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.749+0000 7f2fcbfff640 1 -- 172.21.15.125:0/419711330 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16789 (secure 0 0 0) 0x7f2fbc015090 con 0x7f2fc00047c0 2023-10-18T10:41:29.751 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:29.752 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":111,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:28.760580+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:41:26.603027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-18T10:41:27.743317+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:29.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f2fb406af60 msgr2=0x7f2fb406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f2fb406af60 0x7f2fb406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009b50 tx=0x7f2fc40046c0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 msgr2=0x7f2fd41362f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 0x7f2fd41362f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e590 tx=0x7f2fbc00ea50 comp rx=0 tx=0).stop 2023-10-18T10:41:29.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 shutdown_connections 2023-10-18T10:41:29.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f2fb406af60 0x7f2fb406d420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd40b0660 0x7f2fd40b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40afbc0 0x7f2fd40b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 --2- 172.21.15.125:0/419711330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fc00047c0 0x7f2fd41362f0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 >> 172.21.15.125:0/419711330 conn(0x7f2fd4069930 msgr2=0x7f2fd41568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:29.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 shutdown_connections 2023-10-18T10:41:29.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.751+0000 7f2fda0a2640 1 -- 172.21.15.125:0/419711330 wait complete. 2023-10-18T10:41:29.764 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:29.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.864+0000 7ff9c255e640 1 -- 172.21.15.125:0/3028413961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc141690 msgr2=0x7ff9bc0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.864+0000 7ff9c255e640 1 --2- 172.21.15.125:0/3028413961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc141690 0x7ff9bc0703e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009830 tx=0x7ff9ac02bcd0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.864+0000 7ff9c255e640 1 -- 172.21.15.125:0/3028413961 shutdown_connections 2023-10-18T10:41:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.864+0000 7ff9c255e640 1 --2- 172.21.15.125:0/3028413961 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9bc070920 0x7ff9bc0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.864+0000 7ff9c255e640 1 --2- 172.21.15.125:0/3028413961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc141690 0x7ff9bc0703e0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.864+0000 7ff9c255e640 1 -- 172.21.15.125:0/3028413961 >> 172.21.15.125:0/3028413961 conn(0x7ff9bc069930 msgr2=0x7ff9bc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:29.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.864+0000 7ff9c255e640 1 -- 172.21.15.125:0/3028413961 shutdown_connections 2023-10-18T10:41:29.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.864+0000 7ff9c255e640 1 -- 172.21.15.125:0/3028413961 wait complete. 2023-10-18T10:41:29.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.865+0000 7ff9c255e640 1 Processor -- start 2023-10-18T10:41:29.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.865+0000 7ff9c255e640 1 -- start start 2023-10-18T10:41:29.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9c255e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc070920 0x7ff9bc0aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:29.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9c255e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9bc13e520 0x7ff9bc1f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:29.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9c255e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9bc141690 0x7ff9bc0a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:29.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9c255e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0a7280 con 0x7ff9bc070920 2023-10-18T10:41:29.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9c255e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0a6670 con 0x7ff9bc13e520 2023-10-18T10:41:29.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9c255e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0a6950 con 0x7ff9bc141690 2023-10-18T10:41:29.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9bbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc070920 0x7ff9bc0aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:29.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9bbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc070920 0x7ff9bc0aae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50806/0 (socket says 172.21.15.125:50806) 2023-10-18T10:41:29.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9bbfff640 1 -- 172.21.15.125:0/446053527 learned_addr learned my addr 172.21.15.125:0/446053527 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:29.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9c0afb640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9bc141690 0x7ff9bc0a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:29.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.866+0000 7ff9bb7fe640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9bc13e520 0x7ff9bc1f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:29.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9bbfff640 1 -- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9bc141690 msgr2=0x7ff9bc0a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9bbfff640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9bc141690 0x7ff9bc0a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9bbfff640 1 -- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9bc13e520 msgr2=0x7ff9bc1f1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:29.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9bbfff640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9bc13e520 0x7ff9bc1f1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:29.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9bbfff640 1 -- 172.21.15.125:0/446053527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac009490 con 0x7ff9bc070920 2023-10-18T10:41:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9bb7fe640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9bc13e520 0x7ff9bc1f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9bbfff640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc070920 0x7ff9bc0aae60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000c500 tx=0x7ff9b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9b8ff9640 1 -- 172.21.15.125:0/446053527 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0012830 con 0x7ff9bc070920 2023-10-18T10:41:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9b8ff9640 1 -- 172.21.15.125:0/446053527 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9b0012a80 con 0x7ff9bc070920 2023-10-18T10:41:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc0a6c30 con 0x7ff9bc070920 2023-10-18T10:41:29.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9b8ff9640 1 -- 172.21.15.125:0/446053527 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0012e90 con 0x7ff9bc070920 2023-10-18T10:41:29.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.867+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc0a9580 con 0x7ff9bc070920 2023-10-18T10:41:29.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.868+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc0690e0 con 0x7ff9bc070920 2023-10-18T10:41:29.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.868+0000 7ff9b8ff9640 1 -- 172.21.15.125:0/446053527 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff9b001a3b0 con 0x7ff9bc070920 2023-10-18T10:41:29.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.869+0000 7ff9b8ff9640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff9a806af60 0x7ff9a806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:29.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.869+0000 7ff9b8ff9640 1 -- 172.21.15.125:0/446053527 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff9b0089ad0 con 0x7ff9bc070920 2023-10-18T10:41:29.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.869+0000 7ff9bb7fe640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff9a806af60 0x7ff9a806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:29.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.870+0000 7ff9bb7fe640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff9a806af60 0x7ff9a806d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac036000 tx=0x7ff9ac004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:29.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:29.873+0000 7ff9b8ff9640 1 -- 172.21.15.125:0/446053527 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9b00548b0 con 0x7ff9bc070920 2023-10-18T10:41:30.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.040+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9bc070580 con 0x7ff9bc070920 2023-10-18T10:41:30.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.041+0000 7ff9b8ff9640 1 -- 172.21.15.125:0/446053527 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16789 (secure 0 0 0) 0x7ff9b0016090 con 0x7ff9bc070920 2023-10-18T10:41:30.044 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:30.044 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":111,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:28.760580+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:41:26.603027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-18T10:41:27.743317+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:30.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff9a806af60 msgr2=0x7ff9a806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff9a806af60 0x7ff9a806d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac036000 tx=0x7ff9ac004730 comp rx=0 tx=0).stop 2023-10-18T10:41:30.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc070920 msgr2=0x7ff9bc0aae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc070920 0x7ff9bc0aae60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000c500 tx=0x7ff9b000c9c0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 shutdown_connections 2023-10-18T10:41:30.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff9a806af60 0x7ff9a806d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9bc13e520 0x7ff9bc1f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9bc141690 0x7ff9bc0a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 --2- 172.21.15.125:0/446053527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9bc070920 0x7ff9bc0aae60 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 >> 172.21.15.125:0/446053527 conn(0x7ff9bc069930 msgr2=0x7ff9bc069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:30.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 shutdown_connections 2023-10-18T10:41:30.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.043+0000 7ff9c255e640 1 -- 172.21.15.125:0/446053527 wait complete. 2023-10-18T10:41:30.056 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:41:30.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.159+0000 7f9c7fab0640 1 -- 172.21.15.125:0/4270489137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 msgr2=0x7f9c780b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.159+0000 7f9c7fab0640 1 --2- 172.21.15.125:0/4270489137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 0x7f9c780b0650 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9c70005fd0 tx=0x7f9c7002c050 comp rx=0 tx=0).stop 2023-10-18T10:41:30.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.159+0000 7f9c7fab0640 1 -- 172.21.15.125:0/4270489137 shutdown_connections 2023-10-18T10:41:30.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.159+0000 7f9c7fab0640 1 --2- 172.21.15.125:0/4270489137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 0x7f9c780b0650 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.159+0000 7f9c7fab0640 1 -- 172.21.15.125:0/4270489137 >> 172.21.15.125:0/4270489137 conn(0x7f9c780d0d60 msgr2=0x7f9c780d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:30.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.159+0000 7f9c7fab0640 1 -- 172.21.15.125:0/4270489137 shutdown_connections 2023-10-18T10:41:30.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.159+0000 7f9c7fab0640 1 -- 172.21.15.125:0/4270489137 wait complete. 2023-10-18T10:41:30.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.160+0000 7f9c7fab0640 1 Processor -- start 2023-10-18T10:41:30.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.160+0000 7f9c7fab0640 1 -- start start 2023-10-18T10:41:30.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7fab0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c780cf340 0x7f9c78158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:30.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7fab0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 0x7f9c780c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:30.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7fab0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c7815b2a0 0x7f9c780ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:30.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7fab0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781e5430 con 0x7f9c780d1910 2023-10-18T10:41:30.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7fab0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780c9ec0 con 0x7f9c780cf340 2023-10-18T10:41:30.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7fab0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781e4820 con 0x7f9c7815b2a0 2023-10-18T10:41:30.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d04b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 0x7f9c780c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:30.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d04b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 0x7f9c780c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50832/0 (socket says 172.21.15.125:50832) 2023-10-18T10:41:30.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d04b640 1 -- 172.21.15.125:0/2680986909 learned_addr learned my addr 172.21.15.125:0/2680986909 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:30.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d04b640 1 -- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c7815b2a0 msgr2=0x7f9c780ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7e04d640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c7815b2a0 0x7f9c780ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:30.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d84c640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c780cf340 0x7f9c78158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:30.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d04b640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c7815b2a0 0x7f9c780ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d04b640 1 -- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c780cf340 msgr2=0x7f9c78158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d04b640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c780cf340 0x7f9c78158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.161+0000 7f9c7d04b640 1 -- 172.21.15.125:0/2680986909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70009840 con 0x7f9c780d1910 2023-10-18T10:41:30.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.162+0000 7f9c7d04b640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 0x7f9c780c9980 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800b3e0 tx=0x7f9c6800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:30.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.162+0000 7f9c6e7fc640 1 -- 172.21.15.125:0/2680986909 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68011830 con 0x7f9c780d1910 2023-10-18T10:41:30.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.162+0000 7f9c6e7fc640 1 -- 172.21.15.125:0/2680986909 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9c68011aa0 con 0x7f9c780d1910 2023-10-18T10:41:30.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.162+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c781e4b00 con 0x7f9c780d1910 2023-10-18T10:41:30.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.162+0000 7f9c6e7fc640 1 -- 172.21.15.125:0/2680986909 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68010380 con 0x7f9c780d1910 2023-10-18T10:41:30.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.162+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c781e50c0 con 0x7f9c780d1910 2023-10-18T10:41:30.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.163+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c780b3d00 con 0x7f9c780d1910 2023-10-18T10:41:30.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.163+0000 7f9c6e7fc640 1 -- 172.21.15.125:0/2680986909 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9c680104e0 con 0x7f9c780d1910 2023-10-18T10:41:30.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.164+0000 7f9c6e7fc640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c5c06af60 0x7f9c5c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:30.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.164+0000 7f9c6e7fc640 1 -- 172.21.15.125:0/2680986909 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9c68089d00 con 0x7f9c780d1910 2023-10-18T10:41:30.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.164+0000 7f9c7d84c640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c5c06af60 0x7f9c5c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:30.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.164+0000 7f9c7d84c640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c5c06af60 0x7f9c5c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c740046b0 tx=0x7f9c74009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:30.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.168+0000 7f9c6e7fc640 1 -- 172.21.15.125:0/2680986909 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c68054b60 con 0x7f9c780d1910 2023-10-18T10:41:30.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.368+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9c781e5570 con 0x7f9c780d1910 2023-10-18T10:41:30.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.442+0000 7f9c6e7fc640 1 -- 172.21.15.125:0/2680986909 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9c781e5570 con 0x7f9c780d1910 2023-10-18T10:41:30.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c5c06af60 msgr2=0x7f9c5c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c5c06af60 0x7f9c5c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c740046b0 tx=0x7f9c74009040 comp rx=0 tx=0).stop 2023-10-18T10:41:30.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 msgr2=0x7f9c780c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 0x7f9c780c9980 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800b3e0 tx=0x7f9c6800b8a0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 shutdown_connections 2023-10-18T10:41:30.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c5c06af60 0x7f9c5c06d420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c7815b2a0 0x7f9c780ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c780d1910 0x7f9c780c9980 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 --2- 172.21.15.125:0/2680986909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c780cf340 0x7f9c78158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 >> 172.21.15.125:0/2680986909 conn(0x7f9c780d0d60 msgr2=0x7f9c7815a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:30.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 shutdown_connections 2023-10-18T10:41:30.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.444+0000 7f9c7fab0640 1 -- 172.21.15.125:0/2680986909 wait complete. 2023-10-18T10:41:30.457 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:41:30.457 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:41:30.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 -- 172.21.15.125:0/3918644074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09740ceb40 msgr2=0x7f09740b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 --2- 172.21.15.125:0/3918644074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09740ceb40 0x7f09740b0650 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f097408a810 tx=0x7f096c02bd10 comp rx=0 tx=0).stop 2023-10-18T10:41:30.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 -- 172.21.15.125:0/3918644074 shutdown_connections 2023-10-18T10:41:30.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 --2- 172.21.15.125:0/3918644074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09740ceb40 0x7f09740b0650 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 --2- 172.21.15.125:0/3918644074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f097415b2a0 0x7f09740cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 --2- 172.21.15.125:0/3918644074 >> v2:172.21.15.139:3300/0 conn(0x7f09740cf340 0x7f09740d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 -- 172.21.15.125:0/3918644074 >> 172.21.15.125:0/3918644074 conn(0x7f09740d0d60 msgr2=0x7f09740d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:30.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 -- 172.21.15.125:0/3918644074 shutdown_connections 2023-10-18T10:41:30.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.561+0000 7f097be7c640 1 -- 172.21.15.125:0/3918644074 wait complete. 2023-10-18T10:41:30.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.562+0000 7f097be7c640 1 Processor -- start 2023-10-18T10:41:30.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.562+0000 7f097be7c640 1 -- start start 2023-10-18T10:41:30.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097be7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09740ceb40 0x7f0974158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:30.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097be7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09740cf340 0x7f09740b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:30.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097be7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f097415b2a0 0x7f0974159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:30.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097be7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741f47e0 con 0x7f097415b2a0 2023-10-18T10:41:30.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097be7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741f4a30 con 0x7f09740cf340 2023-10-18T10:41:30.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097be7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09741f4d10 con 0x7f09740ceb40 2023-10-18T10:41:30.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097a419640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f097415b2a0 0x7f0974159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:30.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097a419640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f097415b2a0 0x7f0974159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50850/0 (socket says 172.21.15.125:50850) 2023-10-18T10:41:30.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097a419640 1 -- 172.21.15.125:0/1173330117 learned_addr learned my addr 172.21.15.125:0/1173330117 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097a419640 1 -- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09740ceb40 msgr2=0x7f0974158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097a419640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09740ceb40 0x7f0974158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097a419640 1 -- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09740cf340 msgr2=0x7f09740b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097a419640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09740cf340 0x7f09740b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.563+0000 7f097a419640 1 -- 172.21.15.125:0/1173330117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c009560 con 0x7f097415b2a0 2023-10-18T10:41:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.564+0000 7f097a419640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f097415b2a0 0x7f0974159df0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f096c036040 tx=0x7f096c02da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.564+0000 7f096a7fc640 1 -- 172.21.15.125:0/1173330117 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c003800 con 0x7f097415b2a0 2023-10-18T10:41:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.564+0000 7f096a7fc640 1 -- 172.21.15.125:0/1173330117 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f096c003a50 con 0x7f097415b2a0 2023-10-18T10:41:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.564+0000 7f096a7fc640 1 -- 172.21.15.125:0/1173330117 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c003e40 con 0x7f097415b2a0 2023-10-18T10:41:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.564+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09741f4ff0 con 0x7f097415b2a0 2023-10-18T10:41:30.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.564+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09740caa60 con 0x7f097415b2a0 2023-10-18T10:41:30.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.565+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09740690e0 con 0x7f097415b2a0 2023-10-18T10:41:30.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.565+0000 7f096a7fc640 1 -- 172.21.15.125:0/1173330117 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f096c03e450 con 0x7f097415b2a0 2023-10-18T10:41:30.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.565+0000 7f096a7fc640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:30.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.566+0000 7f096a7fc640 1 -- 172.21.15.125:0/1173330117 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f096c0ac3f0 con 0x7f097415b2a0 2023-10-18T10:41:30.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.566+0000 7f0979c18640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:30.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.566+0000 7f0979c18640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f095806af60 0x7f095806d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09700092f0 tx=0x7f0970009220 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:30.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.570+0000 7f096a7fc640 1 -- 172.21.15.125:0/1173330117 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f096c0772e0 con 0x7f097415b2a0 2023-10-18T10:41:30.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:30.772+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f09740d0b40 con 0x7f097415b2a0 2023-10-18T10:41:31.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.451+0000 7f096a7fc640 1 -- 172.21.15.125:0/1173330117 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f096c07a900 con 0x7f097415b2a0 2023-10-18T10:41:31.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f095806af60 msgr2=0x7f095806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f095806af60 0x7f095806d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09700092f0 tx=0x7f0970009220 comp rx=0 tx=0).stop 2023-10-18T10:41:31.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f097415b2a0 msgr2=0x7f0974159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f097415b2a0 0x7f0974159df0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f096c036040 tx=0x7f096c02da30 comp rx=0 tx=0).stop 2023-10-18T10:41:31.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 shutdown_connections 2023-10-18T10:41:31.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09740cf340 0x7f09740b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f097415b2a0 0x7f0974159df0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 --2- 172.21.15.125:0/1173330117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09740ceb40 0x7f0974158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 >> 172.21.15.125:0/1173330117 conn(0x7f09740d0d60 msgr2=0x7f09740ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:31.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 shutdown_connections 2023-10-18T10:41:31.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.454+0000 7f097be7c640 1 -- 172.21.15.125:0/1173330117 wait complete. 2023-10-18T10:41:31.468 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:41:31.469 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:41:31.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.575+0000 7f162ea75640 1 -- 172.21.15.125:0/4211504774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 msgr2=0x7f162815c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.575+0000 7f162ea75640 1 --2- 172.21.15.125:0/4211504774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 0x7f162815c770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f162815d3c0 tx=0x7f161402bcb0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.575+0000 7f162ea75640 1 -- 172.21.15.125:0/4211504774 shutdown_connections 2023-10-18T10:41:31.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.575+0000 7f162ea75640 1 --2- 172.21.15.125:0/4211504774 >> v2:172.21.15.139:3300/0 conn(0x7f162815ccb0 0x7f16280c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.575+0000 7f162ea75640 1 --2- 172.21.15.125:0/4211504774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 0x7f162815c770 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.575+0000 7f162ea75640 1 -- 172.21.15.125:0/4211504774 >> 172.21.15.125:0/4211504774 conn(0x7f1628069930 msgr2=0x7f1628069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:31.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.575+0000 7f162ea75640 1 -- 172.21.15.125:0/4211504774 shutdown_connections 2023-10-18T10:41:31.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.575+0000 7f162ea75640 1 -- 172.21.15.125:0/4211504774 wait complete. 2023-10-18T10:41:31.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.576+0000 7f162ea75640 1 Processor -- start 2023-10-18T10:41:31.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.576+0000 7f162ea75640 1 -- start start 2023-10-18T10:41:31.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162ea75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f162806b030 0x7f16280a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:31.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162ea75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 0x7f16280a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:31.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162ea75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162815ccb0 0x7f16280a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:31.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162ea75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1628151cc0 con 0x7f162806b030 2023-10-18T10:41:31.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162ea75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16280a8470 con 0x7f162815ccb0 2023-10-18T10:41:31.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162ea75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1628151390 con 0x7f162806b410 2023-10-18T10:41:31.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162c811640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f162806b030 0x7f16280a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:31.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162c811640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f162806b030 0x7f16280a7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50858/0 (socket says 172.21.15.125:50858) 2023-10-18T10:41:31.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162c811640 1 -- 172.21.15.125:0/3043788840 learned_addr learned my addr 172.21.15.125:0/3043788840 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:31.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f162d012640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162815ccb0 0x7f16280a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:31.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.577+0000 7f1627fff640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 0x7f16280a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:31.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f162c811640 1 -- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 msgr2=0x7f16280a92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f162c811640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 0x7f16280a92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f162c811640 1 -- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162815ccb0 msgr2=0x7f16280a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f162c811640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162815ccb0 0x7f16280a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f162c811640 1 -- 172.21.15.125:0/3043788840 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1620009840 con 0x7f162806b030 2023-10-18T10:41:31.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f1627fff640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 0x7f16280a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:31.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f162c811640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f162806b030 0x7f16280a7a90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c430 tx=0x7f161c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:31.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f16257fa640 1 -- 172.21.15.125:0/3043788840 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c004230 con 0x7f162806b030 2023-10-18T10:41:31.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f16257fa640 1 -- 172.21.15.125:0/3043788840 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f161c0044a0 con 0x7f162806b030 2023-10-18T10:41:31.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1614009490 con 0x7f162806b030 2023-10-18T10:41:31.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f16257fa640 1 -- 172.21.15.125:0/3043788840 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c003850 con 0x7f162806b030 2023-10-18T10:41:31.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.578+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1628151950 con 0x7f162806b030 2023-10-18T10:41:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.579+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16280690e0 con 0x7f162806b030 2023-10-18T10:41:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.579+0000 7f16257fa640 1 -- 172.21.15.125:0/3043788840 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f161c003a30 con 0x7f162806b030 2023-10-18T10:41:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.580+0000 7f16257fa640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f160c06af60 0x7f160c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.580+0000 7f16257fa640 1 -- 172.21.15.125:0/3043788840 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f161c0891e0 con 0x7f162806b030 2023-10-18T10:41:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.580+0000 7f1627fff640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f160c06af60 0x7f160c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:31.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.580+0000 7f1627fff640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f160c06af60 0x7f160c06d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1614036000 tx=0x7f1614004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:31.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.584+0000 7f16257fa640 1 -- 172.21.15.125:0/3043788840 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f161c057b10 con 0x7f162806b030 2023-10-18T10:41:31.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.788+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1628153590 con 0x7f162806b030 2023-10-18T10:41:31.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.788+0000 7f16257fa640 1 -- 172.21.15.125:0/3043788840 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f161c0576a0 con 0x7f162806b030 2023-10-18T10:41:31.791 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:31.791 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":41,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2381546668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2381546668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2381546668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/193356301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":193356301},{"type":"v1","addr":"172.21.15.139:6836","nonce":193356301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":8073,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3589330474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3589330474},{"type":"v1","addr":"172.21.15.125:6834","nonce":3589330474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:41:28.760907+0000","modified":"2023-10-18T10:41:30.774086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8076":{"gid":8076,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/3342443842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3342443842},{"type":"v1","addr":"172.21.15.125:6835","nonce":3342443842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T10:41:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 41 2023-10-18T10:41:31.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f160c06af60 msgr2=0x7f160c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f160c06af60 0x7f160c06d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1614036000 tx=0x7f1614004730 comp rx=0 tx=0).stop 2023-10-18T10:41:31.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f162806b030 msgr2=0x7f16280a7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f162806b030 0x7f16280a7a90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c430 tx=0x7f161c00c8f0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 shutdown_connections 2023-10-18T10:41:31.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f160c06af60 0x7f160c06d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162815ccb0 0x7f16280a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f162806b410 0x7f16280a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 --2- 172.21.15.125:0/3043788840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f162806b030 0x7f16280a7a90 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 >> 172.21.15.125:0/3043788840 conn(0x7f1628069930 msgr2=0x7f16280b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:31.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 shutdown_connections 2023-10-18T10:41:31.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.790+0000 7f162ea75640 1 -- 172.21.15.125:0/3043788840 wait complete. 2023-10-18T10:41:31.803 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:41:31.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.903+0000 7fedad16e640 1 -- 172.21.15.125:0/1227972351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 msgr2=0x7fed94003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.903+0000 7fedad16e640 1 --2- 172.21.15.125:0/1227972351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 0x7fed94003d90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7feda0001d80 tx=0x7fed940057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.903+0000 7fedad16e640 1 -- 172.21.15.125:0/1227972351 shutdown_connections 2023-10-18T10:41:31.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.903+0000 7fedad16e640 1 --2- 172.21.15.125:0/1227972351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 0x7fed94003d90 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.903+0000 7fedad16e640 1 -- 172.21.15.125:0/1227972351 >> 172.21.15.125:0/1227972351 conn(0x7feda806fae0 msgr2=0x7feda806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:31.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.903+0000 7fedad16e640 1 -- 172.21.15.125:0/1227972351 shutdown_connections 2023-10-18T10:41:31.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.903+0000 7fedad16e640 1 -- 172.21.15.125:0/1227972351 wait complete. 2023-10-18T10:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.905+0000 7fedad16e640 1 Processor -- start 2023-10-18T10:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.905+0000 7fedad16e640 1 -- start start 2023-10-18T10:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.905+0000 7fedad16e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 0x7feda81594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.905+0000 7fedad16e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feda813bac0 0x7feda813bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.905+0000 7fedad16e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda81f2cb0 0x7feda81f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.905+0000 7fedad16e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda813d380 con 0x7fed94003970 2023-10-18T10:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.905+0000 7fedad16e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda813c460 con 0x7feda81f2cb0 2023-10-18T10:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.905+0000 7fedad16e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda813c700 con 0x7feda813bac0 2023-10-18T10:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 0x7feda81594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 0x7feda81594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50874/0 (socket says 172.21.15.125:50874) 2023-10-18T10:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 -- 172.21.15.125:0/2364547530 learned_addr learned my addr 172.21.15.125:0/2364547530 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda659c640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feda813bac0 0x7feda813bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda759e640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda81f2cb0 0x7feda81f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 -- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feda813bac0 msgr2=0x7feda813bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feda813bac0 0x7feda813bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 -- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda81f2cb0 msgr2=0x7feda81f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:31.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda81f2cb0 0x7feda81f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:31.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 -- 172.21.15.125:0/2364547530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda80b2330 con 0x7fed94003970 2023-10-18T10:41:31.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda659c640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feda813bac0 0x7feda813bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:31.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda759e640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda81f2cb0 0x7feda81f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:31.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7feda6d9d640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 0x7feda81594e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7feda0000f80 tx=0x7feda002c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:31.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7fed9f7fe640 1 -- 172.21.15.125:0/2364547530 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda002cea0 con 0x7fed94003970 2023-10-18T10:41:31.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7fed9f7fe640 1 -- 172.21.15.125:0/2364547530 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feda0006140 con 0x7fed94003970 2023-10-18T10:41:31.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda813c9e0 con 0x7fed94003970 2023-10-18T10:41:31.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7fed9f7fe640 1 -- 172.21.15.125:0/2364547530 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0006550 con 0x7fed94003970 2023-10-18T10:41:31.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.906+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda813cfa0 con 0x7fed94003970 2023-10-18T10:41:31.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.907+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda80690e0 con 0x7fed94003970 2023-10-18T10:41:31.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.908+0000 7fed9f7fe640 1 -- 172.21.15.125:0/2364547530 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feda00066b0 con 0x7fed94003970 2023-10-18T10:41:31.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.908+0000 7fed9f7fe640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fed8806af90 0x7fed8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:31.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.908+0000 7fed9f7fe640 1 -- 172.21.15.125:0/2364547530 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7feda00aad20 con 0x7fed94003970 2023-10-18T10:41:31.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.908+0000 7feda659c640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fed8806af90 0x7fed8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:31.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.909+0000 7feda659c640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fed8806af90 0x7fed8806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feda81589c0 tx=0x7fed90005f90 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:31.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:31.912+0000 7fed9f7fe640 1 -- 172.21.15.125:0/2364547530 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feda0075b80 con 0x7fed94003970 2023-10-18T10:41:32.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.079+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feda81589f0 con 0x7fed94003970 2023-10-18T10:41:32.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.080+0000 7fed9f7fe640 1 -- 172.21.15.125:0/2364547530 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16776 (secure 0 0 0) 0x7feda00791a0 con 0x7fed94003970 2023-10-18T10:41:32.083 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:32.083 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":112,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:41:30.431129+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:41:26.603027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-18T10:41:27.743317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:41:32.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fed8806af90 msgr2=0x7fed8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fed8806af90 0x7fed8806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feda81589c0 tx=0x7fed90005f90 comp rx=0 tx=0).stop 2023-10-18T10:41:32.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 msgr2=0x7feda81594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 0x7feda81594e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7feda0000f80 tx=0x7feda002c8a0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 shutdown_connections 2023-10-18T10:41:32.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fed8806af90 0x7fed8806d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feda81f2cb0 0x7feda81f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feda813bac0 0x7feda813bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 --2- 172.21.15.125:0/2364547530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed94003970 0x7feda81594e0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 >> 172.21.15.125:0/2364547530 conn(0x7feda806fae0 msgr2=0x7feda81532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:32.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 shutdown_connections 2023-10-18T10:41:32.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.082+0000 7fedad16e640 1 -- 172.21.15.125:0/2364547530 wait complete. 2023-10-18T10:41:32.095 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:41:32.095 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:41:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.196+0000 7f6ba9c75640 1 -- 172.21.15.125:0/1875365625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 msgr2=0x7f6ba4158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.196+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/1875365625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 0x7f6ba4158300 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f6b94005fd0 tx=0x7f6b9402be10 comp rx=0 tx=0).stop 2023-10-18T10:41:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.196+0000 7f6ba9c75640 1 -- 172.21.15.125:0/1875365625 shutdown_connections 2023-10-18T10:41:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.196+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/1875365625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba40afa20 0x7f6ba40705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.196+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/1875365625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 0x7f6ba4158300 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.196+0000 7f6ba9c75640 1 -- 172.21.15.125:0/1875365625 >> 172.21.15.125:0/1875365625 conn(0x7f6ba4069930 msgr2=0x7f6ba4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.196+0000 7f6ba9c75640 1 -- 172.21.15.125:0/1875365625 shutdown_connections 2023-10-18T10:41:32.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.196+0000 7f6ba9c75640 1 -- 172.21.15.125:0/1875365625 wait complete. 2023-10-18T10:41:32.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.197+0000 7f6ba9c75640 1 Processor -- start 2023-10-18T10:41:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.197+0000 7f6ba9c75640 1 -- start start 2023-10-18T10:41:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba9c75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba40afa20 0x7f6ba41393b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba9c75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba40b0530 0x7f6ba41efed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba9c75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 0x7f6ba41f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba9c75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba4152760 con 0x7f6ba40cbd20 2023-10-18T10:41:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba9c75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba41f0a10 con 0x7f6ba40b0530 2023-10-18T10:41:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba9c75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba41f0c70 con 0x7f6ba40afa20 2023-10-18T10:41:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 0x7f6ba41f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 0x7f6ba41f04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50890/0 (socket says 172.21.15.125:50890) 2023-10-18T10:41:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 -- 172.21.15.125:0/4246724140 learned_addr learned my addr 172.21.15.125:0/4246724140 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba37fe640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba40afa20 0x7f6ba41393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba2ffd640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba40b0530 0x7f6ba41efed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 -- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba40afa20 msgr2=0x7f6ba41393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba40afa20 0x7f6ba41393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 -- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba40b0530 msgr2=0x7f6ba41efed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba40b0530 0x7f6ba41efed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 -- 172.21.15.125:0/4246724140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b94009560 con 0x7f6ba40cbd20 2023-10-18T10:41:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba37fe640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba40afa20 0x7f6ba41393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba2ffd640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba40b0530 0x7f6ba41efed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.198+0000 7f6ba3fff640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 0x7f6ba41f04d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c710 tx=0x7f6b9c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.199+0000 7f6b8bfff640 1 -- 172.21.15.125:0/4246724140 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c006d50 con 0x7f6ba40cbd20 2023-10-18T10:41:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.199+0000 7f6b8bfff640 1 -- 172.21.15.125:0/4246724140 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b9c00ce20 con 0x7f6ba40cbd20 2023-10-18T10:41:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.199+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba4070cf0 con 0x7f6ba40cbd20 2023-10-18T10:41:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.199+0000 7f6b8bfff640 1 -- 172.21.15.125:0/4246724140 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c003bd0 con 0x7f6ba40cbd20 2023-10-18T10:41:32.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.199+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba41f1230 con 0x7f6ba40cbd20 2023-10-18T10:41:32.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.200+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba40690e0 con 0x7f6ba40cbd20 2023-10-18T10:41:32.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.200+0000 7f6b8bfff640 1 -- 172.21.15.125:0/4246724140 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6b9c003d30 con 0x7f6ba40cbd20 2023-10-18T10:41:32.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.200+0000 7f6b8bfff640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6b9006af60 0x7f6b9006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.200+0000 7f6b8bfff640 1 -- 172.21.15.125:0/4246724140 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6b9c089730 con 0x7f6ba40cbd20 2023-10-18T10:41:32.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.201+0000 7f6ba37fe640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6b9006af60 0x7f6b9006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.201+0000 7f6ba37fe640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6b9006af60 0x7f6b9006d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6b94035000 tx=0x7f6b94004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:32.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.205+0000 7f6b8bfff640 1 -- 172.21.15.125:0/4246724140 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b9c0545a0 con 0x7f6ba40cbd20 2023-10-18T10:41:32.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.367+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 --> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ba40cc1e0 con 0x7f6b9006af60 2023-10-18T10:41:32.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.376+0000 7f6b8bfff640 1 -- 172.21.15.125:0/4246724140 <== mgr.6299 v2:172.21.15.139:6832/1523403591 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283643 (secure 0 0 0) 0x7f6ba40cc1e0 con 0x7f6b9006af60 2023-10-18T10:41:32.380 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:32.381 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:41:32.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6b9006af60 msgr2=0x7f6b9006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6b9006af60 0x7f6b9006d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6b94035000 tx=0x7f6b94004730 comp rx=0 tx=0).stop 2023-10-18T10:41:32.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 msgr2=0x7f6ba41f04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 0x7f6ba41f04d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c710 tx=0x7f6b9c00cbd0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 shutdown_connections 2023-10-18T10:41:32.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f6b9006af60 0x7f6b9006d420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba40b0530 0x7f6ba41efed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6ba40cbd20 0x7f6ba41f04d0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 --2- 172.21.15.125:0/4246724140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba40afa20 0x7f6ba41393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.378+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 >> 172.21.15.125:0/4246724140 conn(0x7f6ba4069930 msgr2=0x7f6ba414e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:32.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.379+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 shutdown_connections 2023-10-18T10:41:32.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.379+0000 7f6ba9c75640 1 -- 172.21.15.125:0/4246724140 wait complete. 2023-10-18T10:41:32.391 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:41:31.281902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":314,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":116,"acting":116,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":313984,"kb_used_data":11780,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754660736,"statfs":{"total":773094113280,"available":772772593664,"internally_reserved":0,"allocated":12062720,"data_stored":6601716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000876"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434717+0000","last_change":"2023-10-18T10:41:29.447539+0000","last_active":"2023-10-18T10:41:30.434717+0000","last_peered":"2023-10-18T10:41:30.434717+0000","last_clean":"2023-10-18T10:41:30.434717+0000","last_became_active":"2023-10-18T10:41:29.447288+0000","last_became_peered":"2023-10-18T10:41:29.447288+0000","last_unstale":"2023-10-18T10:41:30.434717+0000","last_undegraded":"2023-10-18T10:41:30.434717+0000","last_fullsized":"2023-10-18T10:41:30.434717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:30.647260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750857+0000","last_change":"2023-10-18T10:41:29.442252+0000","last_active":"2023-10-18T10:41:30.750857+0000","last_peered":"2023-10-18T10:41:30.750857+0000","last_clean":"2023-10-18T10:41:30.750857+0000","last_became_active":"2023-10-18T10:41:29.441955+0000","last_became_peered":"2023-10-18T10:41:29.441955+0000","last_unstale":"2023-10-18T10:41:30.750857+0000","last_undegraded":"2023-10-18T10:41:30.750857+0000","last_fullsized":"2023-10-18T10:41:30.750857+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:23.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434545+0000","last_change":"2023-10-18T10:41:29.452284+0000","last_active":"2023-10-18T10:41:30.434545+0000","last_peered":"2023-10-18T10:41:30.434545+0000","last_clean":"2023-10-18T10:41:30.434545+0000","last_became_active":"2023-10-18T10:41:29.452037+0000","last_became_peered":"2023-10-18T10:41:29.452037+0000","last_unstale":"2023-10-18T10:41:30.434545+0000","last_undegraded":"2023-10-18T10:41:30.434545+0000","last_fullsized":"2023-10-18T10:41:30.434545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.154689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750025+0000","last_change":"2023-10-18T10:41:29.453178+0000","last_active":"2023-10-18T10:41:30.750025+0000","last_peered":"2023-10-18T10:41:30.750025+0000","last_clean":"2023-10-18T10:41:30.750025+0000","last_became_active":"2023-10-18T10:41:29.452902+0000","last_became_peered":"2023-10-18T10:41:29.452902+0000","last_unstale":"2023-10-18T10:41:30.750025+0000","last_undegraded":"2023-10-18T10:41:30.750025+0000","last_fullsized":"2023-10-18T10:41:30.750025+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:23.594087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750972+0000","last_change":"2023-10-18T10:41:29.450362+0000","last_active":"2023-10-18T10:41:30.750972+0000","last_peered":"2023-10-18T10:41:30.750972+0000","last_clean":"2023-10-18T10:41:30.750972+0000","last_became_active":"2023-10-18T10:41:29.449972+0000","last_became_peered":"2023-10-18T10:41:29.449972+0000","last_unstale":"2023-10-18T10:41:30.750972+0000","last_undegraded":"2023-10-18T10:41:30.750972+0000","last_fullsized":"2023-10-18T10:41:30.750972+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:59.437842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749904+0000","last_change":"2023-10-18T10:41:29.449393+0000","last_active":"2023-10-18T10:41:30.749904+0000","last_peered":"2023-10-18T10:41:30.749904+0000","last_clean":"2023-10-18T10:41:30.749904+0000","last_became_active":"2023-10-18T10:41:29.449086+0000","last_became_peered":"2023-10-18T10:41:29.449086+0000","last_unstale":"2023-10-18T10:41:30.749904+0000","last_undegraded":"2023-10-18T10:41:30.749904+0000","last_fullsized":"2023-10-18T10:41:30.749904+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:41.061091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751632+0000","last_change":"2023-10-18T10:41:29.444304+0000","last_active":"2023-10-18T10:41:30.751632+0000","last_peered":"2023-10-18T10:41:30.751632+0000","last_clean":"2023-10-18T10:41:30.751632+0000","last_became_active":"2023-10-18T10:41:29.444057+0000","last_became_peered":"2023-10-18T10:41:29.444057+0000","last_unstale":"2023-10-18T10:41:30.751632+0000","last_undegraded":"2023-10-18T10:41:30.751632+0000","last_fullsized":"2023-10-18T10:41:30.751632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:23.209276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751364+0000","last_change":"2023-10-18T10:41:29.443320+0000","last_active":"2023-10-18T10:41:30.751364+0000","last_peered":"2023-10-18T10:41:30.751364+0000","last_clean":"2023-10-18T10:41:30.751364+0000","last_became_active":"2023-10-18T10:41:29.443029+0000","last_became_peered":"2023-10-18T10:41:29.443029+0000","last_unstale":"2023-10-18T10:41:30.751364+0000","last_undegraded":"2023-10-18T10:41:30.751364+0000","last_fullsized":"2023-10-18T10:41:30.751364+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:27.802284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750628+0000","last_change":"2023-10-18T10:41:29.448605+0000","last_active":"2023-10-18T10:41:30.750628+0000","last_peered":"2023-10-18T10:41:30.750628+0000","last_clean":"2023-10-18T10:41:30.750628+0000","last_became_active":"2023-10-18T10:41:29.448348+0000","last_became_peered":"2023-10-18T10:41:29.448348+0000","last_unstale":"2023-10-18T10:41:30.750628+0000","last_undegraded":"2023-10-18T10:41:30.750628+0000","last_fullsized":"2023-10-18T10:41:30.750628+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:42.695172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750043+0000","last_change":"2023-10-18T10:41:29.444407+0000","last_active":"2023-10-18T10:41:30.750043+0000","last_peered":"2023-10-18T10:41:30.750043+0000","last_clean":"2023-10-18T10:41:30.750043+0000","last_became_active":"2023-10-18T10:41:29.444169+0000","last_became_peered":"2023-10-18T10:41:29.444169+0000","last_unstale":"2023-10-18T10:41:30.750043+0000","last_undegraded":"2023-10-18T10:41:30.750043+0000","last_fullsized":"2023-10-18T10:41:30.750043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:13.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750781+0000","last_change":"2023-10-18T10:41:29.454015+0000","last_active":"2023-10-18T10:41:30.750781+0000","last_peered":"2023-10-18T10:41:30.750781+0000","last_clean":"2023-10-18T10:41:30.750781+0000","last_became_active":"2023-10-18T10:41:29.451538+0000","last_became_peered":"2023-10-18T10:41:29.451538+0000","last_unstale":"2023-10-18T10:41:30.750781+0000","last_undegraded":"2023-10-18T10:41:30.750781+0000","last_fullsized":"2023-10-18T10:41:30.750781+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:52.418194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751142+0000","last_change":"2023-10-18T10:41:29.449763+0000","last_active":"2023-10-18T10:41:30.751142+0000","last_peered":"2023-10-18T10:41:30.751142+0000","last_clean":"2023-10-18T10:41:30.751142+0000","last_became_active":"2023-10-18T10:41:29.449498+0000","last_became_peered":"2023-10-18T10:41:29.449498+0000","last_unstale":"2023-10-18T10:41:30.751142+0000","last_undegraded":"2023-10-18T10:41:30.751142+0000","last_fullsized":"2023-10-18T10:41:30.751142+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:30.606377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434672+0000","last_change":"2023-10-18T10:41:29.448915+0000","last_active":"2023-10-18T10:41:30.434672+0000","last_peered":"2023-10-18T10:41:30.434672+0000","last_clean":"2023-10-18T10:41:30.434672+0000","last_became_active":"2023-10-18T10:41:29.448650+0000","last_became_peered":"2023-10-18T10:41:29.448650+0000","last_unstale":"2023-10-18T10:41:30.434672+0000","last_undegraded":"2023-10-18T10:41:30.434672+0000","last_fullsized":"2023-10-18T10:41:30.434672+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:41.152281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750152+0000","last_change":"2023-10-18T10:41:29.455557+0000","last_active":"2023-10-18T10:41:30.750152+0000","last_peered":"2023-10-18T10:41:30.750152+0000","last_clean":"2023-10-18T10:41:30.750152+0000","last_became_active":"2023-10-18T10:41:29.455296+0000","last_became_peered":"2023-10-18T10:41:29.455296+0000","last_unstale":"2023-10-18T10:41:30.750152+0000","last_undegraded":"2023-10-18T10:41:30.750152+0000","last_fullsized":"2023-10-18T10:41:30.750152+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:51.749366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434493+0000","last_change":"2023-10-18T10:41:29.453702+0000","last_active":"2023-10-18T10:41:30.434493+0000","last_peered":"2023-10-18T10:41:30.434493+0000","last_clean":"2023-10-18T10:41:30.434493+0000","last_became_active":"2023-10-18T10:41:29.453445+0000","last_became_peered":"2023-10-18T10:41:29.453445+0000","last_unstale":"2023-10-18T10:41:30.434493+0000","last_undegraded":"2023-10-18T10:41:30.434493+0000","last_fullsized":"2023-10-18T10:41:30.434493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:28.572719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750696+0000","last_change":"2023-10-18T10:41:29.451267+0000","last_active":"2023-10-18T10:41:30.750696+0000","last_peered":"2023-10-18T10:41:30.750696+0000","last_clean":"2023-10-18T10:41:30.750696+0000","last_became_active":"2023-10-18T10:41:29.450963+0000","last_became_peered":"2023-10-18T10:41:29.450963+0000","last_unstale":"2023-10-18T10:41:30.750696+0000","last_undegraded":"2023-10-18T10:41:30.750696+0000","last_fullsized":"2023-10-18T10:41:30.750696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:01.499934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751548+0000","last_change":"2023-10-18T10:41:29.453816+0000","last_active":"2023-10-18T10:41:30.751548+0000","last_peered":"2023-10-18T10:41:30.751548+0000","last_clean":"2023-10-18T10:41:30.751548+0000","last_became_active":"2023-10-18T10:41:29.453581+0000","last_became_peered":"2023-10-18T10:41:29.453581+0000","last_unstale":"2023-10-18T10:41:30.751548+0000","last_undegraded":"2023-10-18T10:41:30.751548+0000","last_fullsized":"2023-10-18T10:41:30.751548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:18.669429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750180+0000","last_change":"2023-10-18T10:41:29.454500+0000","last_active":"2023-10-18T10:41:30.750180+0000","last_peered":"2023-10-18T10:41:30.750180+0000","last_clean":"2023-10-18T10:41:30.750180+0000","last_became_active":"2023-10-18T10:41:29.454231+0000","last_became_peered":"2023-10-18T10:41:29.454231+0000","last_unstale":"2023-10-18T10:41:30.750180+0000","last_undegraded":"2023-10-18T10:41:30.750180+0000","last_fullsized":"2023-10-18T10:41:30.750180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:14.224881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750335+0000","last_change":"2023-10-18T10:41:29.453993+0000","last_active":"2023-10-18T10:41:30.750335+0000","last_peered":"2023-10-18T10:41:30.750335+0000","last_clean":"2023-10-18T10:41:30.750335+0000","last_became_active":"2023-10-18T10:41:29.453659+0000","last_became_peered":"2023-10-18T10:41:29.453659+0000","last_unstale":"2023-10-18T10:41:30.750335+0000","last_undegraded":"2023-10-18T10:41:30.750335+0000","last_fullsized":"2023-10-18T10:41:30.750335+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:50.051564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750971+0000","last_change":"2023-10-18T10:41:29.445142+0000","last_active":"2023-10-18T10:41:30.750971+0000","last_peered":"2023-10-18T10:41:30.750971+0000","last_clean":"2023-10-18T10:41:30.750971+0000","last_became_active":"2023-10-18T10:41:29.444871+0000","last_became_peered":"2023-10-18T10:41:29.444871+0000","last_unstale":"2023-10-18T10:41:30.750971+0000","last_undegraded":"2023-10-18T10:41:30.750971+0000","last_fullsized":"2023-10-18T10:41:30.750971+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:31:23.805032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750791+0000","last_change":"2023-10-18T10:41:29.448568+0000","last_active":"2023-10-18T10:41:30.750791+0000","last_peered":"2023-10-18T10:41:30.750791+0000","last_clean":"2023-10-18T10:41:30.750791+0000","last_became_active":"2023-10-18T10:41:29.448318+0000","last_became_peered":"2023-10-18T10:41:29.448318+0000","last_unstale":"2023-10-18T10:41:30.750791+0000","last_undegraded":"2023-10-18T10:41:30.750791+0000","last_fullsized":"2023-10-18T10:41:30.750791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:29.589441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751302+0000","last_change":"2023-10-18T10:41:29.445971+0000","last_active":"2023-10-18T10:41:30.751302+0000","last_peered":"2023-10-18T10:41:30.751302+0000","last_clean":"2023-10-18T10:41:30.751302+0000","last_became_active":"2023-10-18T10:41:29.445723+0000","last_became_peered":"2023-10-18T10:41:29.445723+0000","last_unstale":"2023-10-18T10:41:30.751302+0000","last_undegraded":"2023-10-18T10:41:30.751302+0000","last_fullsized":"2023-10-18T10:41:30.751302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:35:45.397768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751225+0000","last_change":"2023-10-18T10:41:29.451021+0000","last_active":"2023-10-18T10:41:30.751225+0000","last_peered":"2023-10-18T10:41:30.751225+0000","last_clean":"2023-10-18T10:41:30.751225+0000","last_became_active":"2023-10-18T10:41:29.450779+0000","last_became_peered":"2023-10-18T10:41:29.450779+0000","last_unstale":"2023-10-18T10:41:30.751225+0000","last_undegraded":"2023-10-18T10:41:30.751225+0000","last_fullsized":"2023-10-18T10:41:30.751225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:36.701370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749949+0000","last_change":"2023-10-18T10:41:29.443511+0000","last_active":"2023-10-18T10:41:30.749949+0000","last_peered":"2023-10-18T10:41:30.749949+0000","last_clean":"2023-10-18T10:41:30.749949+0000","last_became_active":"2023-10-18T10:41:29.443232+0000","last_became_peered":"2023-10-18T10:41:29.443232+0000","last_unstale":"2023-10-18T10:41:30.749949+0000","last_undegraded":"2023-10-18T10:41:30.749949+0000","last_fullsized":"2023-10-18T10:41:30.749949+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:29.869647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750830+0000","last_change":"2023-10-18T10:41:28.439965+0000","last_active":"2023-10-18T10:41:30.750830+0000","last_peered":"2023-10-18T10:41:30.750830+0000","last_clean":"2023-10-18T10:41:30.750830+0000","last_became_active":"2023-10-18T10:41:28.439700+0000","last_became_peered":"2023-10-18T10:41:28.439700+0000","last_unstale":"2023-10-18T10:41:30.750830+0000","last_undegraded":"2023-10-18T10:41:30.750830+0000","last_fullsized":"2023-10-18T10:41:30.750830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:44.202065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750979+0000","last_change":"2023-10-18T10:41:29.450500+0000","last_active":"2023-10-18T10:41:30.750979+0000","last_peered":"2023-10-18T10:41:30.750979+0000","last_clean":"2023-10-18T10:41:30.750979+0000","last_became_active":"2023-10-18T10:41:29.450234+0000","last_became_peered":"2023-10-18T10:41:29.450234+0000","last_unstale":"2023-10-18T10:41:30.750979+0000","last_undegraded":"2023-10-18T10:41:30.750979+0000","last_fullsized":"2023-10-18T10:41:30.750979+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:08.866019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434561+0000","last_change":"2023-10-18T10:41:28.435875+0000","last_active":"2023-10-18T10:41:30.434561+0000","last_peered":"2023-10-18T10:41:30.434561+0000","last_clean":"2023-10-18T10:41:30.434561+0000","last_became_active":"2023-10-18T10:41:28.435623+0000","last_became_peered":"2023-10-18T10:41:28.435623+0000","last_unstale":"2023-10-18T10:41:30.434561+0000","last_undegraded":"2023-10-18T10:41:30.434561+0000","last_fullsized":"2023-10-18T10:41:30.434561+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:22.897591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750655+0000","last_change":"2023-10-18T10:41:29.451882+0000","last_active":"2023-10-18T10:41:30.750655+0000","last_peered":"2023-10-18T10:41:30.750655+0000","last_clean":"2023-10-18T10:41:30.750655+0000","last_became_active":"2023-10-18T10:41:29.451623+0000","last_became_peered":"2023-10-18T10:41:29.451623+0000","last_unstale":"2023-10-18T10:41:30.750655+0000","last_undegraded":"2023-10-18T10:41:30.750655+0000","last_fullsized":"2023-10-18T10:41:30.750655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:39.862682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750926+0000","last_change":"2023-10-18T10:41:28.436855+0000","last_active":"2023-10-18T10:41:30.750926+0000","last_peered":"2023-10-18T10:41:30.750926+0000","last_clean":"2023-10-18T10:41:30.750926+0000","last_became_active":"2023-10-18T10:41:28.436585+0000","last_became_peered":"2023-10-18T10:41:28.436585+0000","last_unstale":"2023-10-18T10:41:30.750926+0000","last_undegraded":"2023-10-18T10:41:30.750926+0000","last_fullsized":"2023-10-18T10:41:30.750926+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.155833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434722+0000","last_change":"2023-10-18T10:41:28.440969+0000","last_active":"2023-10-18T10:41:30.434722+0000","last_peered":"2023-10-18T10:41:30.434722+0000","last_clean":"2023-10-18T10:41:30.434722+0000","last_became_active":"2023-10-18T10:41:28.440739+0000","last_became_peered":"2023-10-18T10:41:28.440739+0000","last_unstale":"2023-10-18T10:41:30.434722+0000","last_undegraded":"2023-10-18T10:41:30.434722+0000","last_fullsized":"2023-10-18T10:41:30.434722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.620461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434696+0000","last_change":"2023-10-18T10:41:29.456187+0000","last_active":"2023-10-18T10:41:30.434696+0000","last_peered":"2023-10-18T10:41:30.434696+0000","last_clean":"2023-10-18T10:41:30.434696+0000","last_became_active":"2023-10-18T10:41:29.455945+0000","last_became_peered":"2023-10-18T10:41:29.455945+0000","last_unstale":"2023-10-18T10:41:30.434696+0000","last_undegraded":"2023-10-18T10:41:30.434696+0000","last_fullsized":"2023-10-18T10:41:30.434696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:40.835626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750493+0000","last_change":"2023-10-18T10:41:29.449408+0000","last_active":"2023-10-18T10:41:30.750493+0000","last_peered":"2023-10-18T10:41:30.750493+0000","last_clean":"2023-10-18T10:41:30.750493+0000","last_became_active":"2023-10-18T10:41:29.449161+0000","last_became_peered":"2023-10-18T10:41:29.449161+0000","last_unstale":"2023-10-18T10:41:30.750493+0000","last_undegraded":"2023-10-18T10:41:30.750493+0000","last_fullsized":"2023-10-18T10:41:30.750493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:41.742410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"111'6","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751094+0000","last_change":"2023-10-18T10:41:28.427403+0000","last_active":"2023-10-18T10:41:30.751094+0000","last_peered":"2023-10-18T10:41:30.751094+0000","last_clean":"2023-10-18T10:41:30.751094+0000","last_became_active":"2023-10-18T10:41:28.427119+0000","last_became_peered":"2023-10-18T10:41:28.427119+0000","last_unstale":"2023-10-18T10:41:30.751094+0000","last_undegraded":"2023-10-18T10:41:30.751094+0000","last_fullsized":"2023-10-18T10:41:30.751094+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:11.156072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750239+0000","last_change":"2023-10-18T10:41:29.448792+0000","last_active":"2023-10-18T10:41:30.750239+0000","last_peered":"2023-10-18T10:41:30.750239+0000","last_clean":"2023-10-18T10:41:30.750239+0000","last_became_active":"2023-10-18T10:41:29.448536+0000","last_became_peered":"2023-10-18T10:41:29.448536+0000","last_unstale":"2023-10-18T10:41:30.750239+0000","last_undegraded":"2023-10-18T10:41:30.750239+0000","last_fullsized":"2023-10-18T10:41:30.750239+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:14.300602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751060+0000","last_change":"2023-10-18T10:41:28.433856+0000","last_active":"2023-10-18T10:41:30.751060+0000","last_peered":"2023-10-18T10:41:30.751060+0000","last_clean":"2023-10-18T10:41:30.751060+0000","last_became_active":"2023-10-18T10:41:28.433584+0000","last_became_peered":"2023-10-18T10:41:28.433584+0000","last_unstale":"2023-10-18T10:41:30.751060+0000","last_undegraded":"2023-10-18T10:41:30.751060+0000","last_fullsized":"2023-10-18T10:41:30.751060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:06:07.806734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750178+0000","last_change":"2023-10-18T10:41:29.439852+0000","last_active":"2023-10-18T10:41:30.750178+0000","last_peered":"2023-10-18T10:41:30.750178+0000","last_clean":"2023-10-18T10:41:30.750178+0000","last_became_active":"2023-10-18T10:41:29.439612+0000","last_became_peered":"2023-10-18T10:41:29.439612+0000","last_unstale":"2023-10-18T10:41:30.750178+0000","last_undegraded":"2023-10-18T10:41:30.750178+0000","last_fullsized":"2023-10-18T10:41:30.750178+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:10.074146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750194+0000","last_change":"2023-10-18T10:41:28.431914+0000","last_active":"2023-10-18T10:41:30.750194+0000","last_peered":"2023-10-18T10:41:30.750194+0000","last_clean":"2023-10-18T10:41:30.750194+0000","last_became_active":"2023-10-18T10:41:28.431664+0000","last_became_peered":"2023-10-18T10:41:28.431664+0000","last_unstale":"2023-10-18T10:41:30.750194+0000","last_undegraded":"2023-10-18T10:41:30.750194+0000","last_fullsized":"2023-10-18T10:41:30.750194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:41.649804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751367+0000","last_change":"2023-10-18T10:41:29.453041+0000","last_active":"2023-10-18T10:41:30.751367+0000","last_peered":"2023-10-18T10:41:30.751367+0000","last_clean":"2023-10-18T10:41:30.751367+0000","last_became_active":"2023-10-18T10:41:29.452788+0000","last_became_peered":"2023-10-18T10:41:29.452788+0000","last_unstale":"2023-10-18T10:41:30.751367+0000","last_undegraded":"2023-10-18T10:41:30.751367+0000","last_fullsized":"2023-10-18T10:41:30.751367+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:28.740818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"111'8","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750045+0000","last_change":"2023-10-18T10:41:28.440009+0000","last_active":"2023-10-18T10:41:30.750045+0000","last_peered":"2023-10-18T10:41:30.750045+0000","last_clean":"2023-10-18T10:41:30.750045+0000","last_became_active":"2023-10-18T10:41:28.439742+0000","last_became_peered":"2023-10-18T10:41:28.439742+0000","last_unstale":"2023-10-18T10:41:30.750045+0000","last_undegraded":"2023-10-18T10:41:30.750045+0000","last_fullsized":"2023-10-18T10:41:30.750045+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:42.072352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751005+0000","last_change":"2023-10-18T10:41:29.448022+0000","last_active":"2023-10-18T10:41:30.751005+0000","last_peered":"2023-10-18T10:41:30.751005+0000","last_clean":"2023-10-18T10:41:30.751005+0000","last_became_active":"2023-10-18T10:41:29.447771+0000","last_became_peered":"2023-10-18T10:41:29.447771+0000","last_unstale":"2023-10-18T10:41:30.751005+0000","last_undegraded":"2023-10-18T10:41:30.751005+0000","last_fullsized":"2023-10-18T10:41:30.751005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:13.373299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751109+0000","last_change":"2023-10-18T10:41:29.450342+0000","last_active":"2023-10-18T10:41:30.751109+0000","last_peered":"2023-10-18T10:41:30.751109+0000","last_clean":"2023-10-18T10:41:30.751109+0000","last_became_active":"2023-10-18T10:41:29.450078+0000","last_became_peered":"2023-10-18T10:41:29.450078+0000","last_unstale":"2023-10-18T10:41:30.751109+0000","last_undegraded":"2023-10-18T10:41:30.751109+0000","last_fullsized":"2023-10-18T10:41:30.751109+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:32.976432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750276+0000","last_change":"2023-10-18T10:41:28.438968+0000","last_active":"2023-10-18T10:41:30.750276+0000","last_peered":"2023-10-18T10:41:30.750276+0000","last_clean":"2023-10-18T10:41:30.750276+0000","last_became_active":"2023-10-18T10:41:28.438628+0000","last_became_peered":"2023-10-18T10:41:28.438628+0000","last_unstale":"2023-10-18T10:41:30.750276+0000","last_undegraded":"2023-10-18T10:41:30.750276+0000","last_fullsized":"2023-10-18T10:41:30.750276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:51.950983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750427+0000","last_change":"2023-10-18T10:41:29.442931+0000","last_active":"2023-10-18T10:41:30.750427+0000","last_peered":"2023-10-18T10:41:30.750427+0000","last_clean":"2023-10-18T10:41:30.750427+0000","last_became_active":"2023-10-18T10:41:29.442573+0000","last_became_peered":"2023-10-18T10:41:29.442573+0000","last_unstale":"2023-10-18T10:41:30.750427+0000","last_undegraded":"2023-10-18T10:41:30.750427+0000","last_fullsized":"2023-10-18T10:41:30.750427+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:36.667760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750311+0000","last_change":"2023-10-18T10:41:28.437156+0000","last_active":"2023-10-18T10:41:30.750311+0000","last_peered":"2023-10-18T10:41:30.750311+0000","last_clean":"2023-10-18T10:41:30.750311+0000","last_became_active":"2023-10-18T10:41:28.434657+0000","last_became_peered":"2023-10-18T10:41:28.434657+0000","last_unstale":"2023-10-18T10:41:30.750311+0000","last_undegraded":"2023-10-18T10:41:30.750311+0000","last_fullsized":"2023-10-18T10:41:30.750311+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:38:04.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750603+0000","last_change":"2023-10-18T10:41:29.452874+0000","last_active":"2023-10-18T10:41:30.750603+0000","last_peered":"2023-10-18T10:41:30.750603+0000","last_clean":"2023-10-18T10:41:30.750603+0000","last_became_active":"2023-10-18T10:41:29.452561+0000","last_became_peered":"2023-10-18T10:41:29.452561+0000","last_unstale":"2023-10-18T10:41:30.750603+0000","last_undegraded":"2023-10-18T10:41:30.750603+0000","last_fullsized":"2023-10-18T10:41:30.750603+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:26.000624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750422+0000","last_change":"2023-10-18T10:41:28.435576+0000","last_active":"2023-10-18T10:41:30.750422+0000","last_peered":"2023-10-18T10:41:30.750422+0000","last_clean":"2023-10-18T10:41:30.750422+0000","last_became_active":"2023-10-18T10:41:28.435286+0000","last_became_peered":"2023-10-18T10:41:28.435286+0000","last_unstale":"2023-10-18T10:41:30.750422+0000","last_undegraded":"2023-10-18T10:41:30.750422+0000","last_fullsized":"2023-10-18T10:41:30.750422+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:29.719996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750887+0000","last_change":"2023-10-18T10:41:29.446250+0000","last_active":"2023-10-18T10:41:30.750887+0000","last_peered":"2023-10-18T10:41:30.750887+0000","last_clean":"2023-10-18T10:41:30.750887+0000","last_became_active":"2023-10-18T10:41:29.445881+0000","last_became_peered":"2023-10-18T10:41:29.445881+0000","last_unstale":"2023-10-18T10:41:30.750887+0000","last_undegraded":"2023-10-18T10:41:30.750887+0000","last_fullsized":"2023-10-18T10:41:30.750887+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:28.061711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750340+0000","last_change":"2023-10-18T10:41:28.438325+0000","last_active":"2023-10-18T10:41:30.750340+0000","last_peered":"2023-10-18T10:41:30.750340+0000","last_clean":"2023-10-18T10:41:30.750340+0000","last_became_active":"2023-10-18T10:41:28.438056+0000","last_became_peered":"2023-10-18T10:41:28.438056+0000","last_unstale":"2023-10-18T10:41:30.750340+0000","last_undegraded":"2023-10-18T10:41:30.750340+0000","last_fullsized":"2023-10-18T10:41:30.750340+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:42.738454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751473+0000","last_change":"2023-10-18T10:41:29.449610+0000","last_active":"2023-10-18T10:41:30.751473+0000","last_peered":"2023-10-18T10:41:30.751473+0000","last_clean":"2023-10-18T10:41:30.751473+0000","last_became_active":"2023-10-18T10:41:29.449354+0000","last_became_peered":"2023-10-18T10:41:29.449354+0000","last_unstale":"2023-10-18T10:41:30.751473+0000","last_undegraded":"2023-10-18T10:41:30.751473+0000","last_fullsized":"2023-10-18T10:41:30.751473+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:01.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751349+0000","last_change":"2023-10-18T10:41:29.447215+0000","last_active":"2023-10-18T10:41:30.751349+0000","last_peered":"2023-10-18T10:41:30.751349+0000","last_clean":"2023-10-18T10:41:30.751349+0000","last_became_active":"2023-10-18T10:41:29.446983+0000","last_became_peered":"2023-10-18T10:41:29.446983+0000","last_unstale":"2023-10-18T10:41:30.751349+0000","last_undegraded":"2023-10-18T10:41:30.751349+0000","last_fullsized":"2023-10-18T10:41:30.751349+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750837+0000","last_change":"2023-10-18T10:41:29.448872+0000","last_active":"2023-10-18T10:41:30.750837+0000","last_peered":"2023-10-18T10:41:30.750837+0000","last_clean":"2023-10-18T10:41:30.750837+0000","last_became_active":"2023-10-18T10:41:29.448574+0000","last_became_peered":"2023-10-18T10:41:29.448574+0000","last_unstale":"2023-10-18T10:41:30.750837+0000","last_undegraded":"2023-10-18T10:41:30.750837+0000","last_fullsized":"2023-10-18T10:41:30.750837+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:40.934129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749895+0000","last_change":"2023-10-18T10:41:29.442254+0000","last_active":"2023-10-18T10:41:30.749895+0000","last_peered":"2023-10-18T10:41:30.749895+0000","last_clean":"2023-10-18T10:41:30.749895+0000","last_became_active":"2023-10-18T10:41:29.441995+0000","last_became_peered":"2023-10-18T10:41:29.441995+0000","last_unstale":"2023-10-18T10:41:30.749895+0000","last_undegraded":"2023-10-18T10:41:30.749895+0000","last_fullsized":"2023-10-18T10:41:30.749895+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:13.960239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751093+0000","last_change":"2023-10-18T10:41:28.434524+0000","last_active":"2023-10-18T10:41:30.751093+0000","last_peered":"2023-10-18T10:41:30.751093+0000","last_clean":"2023-10-18T10:41:30.751093+0000","last_became_active":"2023-10-18T10:41:28.434260+0000","last_became_peered":"2023-10-18T10:41:28.434260+0000","last_unstale":"2023-10-18T10:41:30.751093+0000","last_undegraded":"2023-10-18T10:41:30.751093+0000","last_fullsized":"2023-10-18T10:41:30.751093+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:26:18.323130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434509+0000","last_change":"2023-10-18T10:41:29.455708+0000","last_active":"2023-10-18T10:41:30.434509+0000","last_peered":"2023-10-18T10:41:30.434509+0000","last_clean":"2023-10-18T10:41:30.434509+0000","last_became_active":"2023-10-18T10:41:29.455459+0000","last_became_peered":"2023-10-18T10:41:29.455459+0000","last_unstale":"2023-10-18T10:41:30.434509+0000","last_undegraded":"2023-10-18T10:41:30.434509+0000","last_fullsized":"2023-10-18T10:41:30.434509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:01.952901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750593+0000","last_change":"2023-10-18T10:41:28.439218+0000","last_active":"2023-10-18T10:41:30.750593+0000","last_peered":"2023-10-18T10:41:30.750593+0000","last_clean":"2023-10-18T10:41:30.750593+0000","last_became_active":"2023-10-18T10:41:28.438612+0000","last_became_peered":"2023-10-18T10:41:28.438612+0000","last_unstale":"2023-10-18T10:41:30.750593+0000","last_undegraded":"2023-10-18T10:41:30.750593+0000","last_fullsized":"2023-10-18T10:41:30.750593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:22.330220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751017+0000","last_change":"2023-10-18T10:41:29.446112+0000","last_active":"2023-10-18T10:41:30.751017+0000","last_peered":"2023-10-18T10:41:30.751017+0000","last_clean":"2023-10-18T10:41:30.751017+0000","last_became_active":"2023-10-18T10:41:29.445569+0000","last_became_peered":"2023-10-18T10:41:29.445569+0000","last_unstale":"2023-10-18T10:41:30.751017+0000","last_undegraded":"2023-10-18T10:41:30.751017+0000","last_fullsized":"2023-10-18T10:41:30.751017+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:23.963420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750265+0000","last_change":"2023-10-18T10:41:28.440086+0000","last_active":"2023-10-18T10:41:30.750265+0000","last_peered":"2023-10-18T10:41:30.750265+0000","last_clean":"2023-10-18T10:41:30.750265+0000","last_became_active":"2023-10-18T10:41:28.439814+0000","last_became_peered":"2023-10-18T10:41:28.439814+0000","last_unstale":"2023-10-18T10:41:30.750265+0000","last_undegraded":"2023-10-18T10:41:30.750265+0000","last_fullsized":"2023-10-18T10:41:30.750265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:19.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"101'192","reported_seq":513,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T10:41:24.714727+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:41:24.714727+0000","last_peered":"2023-10-18T10:41:24.714727+0000","last_clean":"2023-10-18T10:41:24.714727+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:41:24.714727+0000","last_undegraded":"2023-10-18T10:41:24.714727+0000","last_fullsized":"2023-10-18T10:41:24.714727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":30,"acting":30,"num_store_stats":5},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":1912832,"data_stored":1248389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.876,"5min":0.876,"15min":0.88300000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68200000000000005,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73199999999999998,"5min":0.84199999999999997,"15min":0.84899999999999998},"last":0.51900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.627}]},{"osd":4,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.495,"15min":0.48799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.875,"15min":0.875},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78400000000000003,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.69999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32492,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1331200,"data_stored":664028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.34499999999999997,"15min":0.28699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.78500000000000003,"15min":0.82599999999999996},"last":0.438}]},{"osd":1,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.79500000000000004,"15min":0.83399999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.80900000000000005,"15min":0.82699999999999996},"last":0.621}]},{"osd":2,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55800000000000005,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65000000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.64900000000000002,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.66900000000000004,"5min":0.75600000000000001,"15min":0.84399999999999997},"last":0.69699999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.441,"15min":0.44},"min":{"1min":0.30399999999999999,"5min":0.29199999999999998,"15min":0.24199999999999999},"max":{"1min":0.76500000000000001,"5min":0.872,"15min":0.872},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45600000000000002,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65500000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48199999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82499999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.68300000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.749},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72199999999999998,"5min":0.82199999999999995,"15min":0.87},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57199999999999995,"15min":0.58399999999999996},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69599999999999995,"5min":0.82499999999999996,"15min":1.016},"last":0.434},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.70099999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.628},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":1.0620000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.80100000000000005,"15min":1.262},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73099999999999998,"5min":0.96099999999999997,"15min":1.224},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":0.871,"15min":1.159},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":664028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.877,"15min":0.877},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.76900000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.63100000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.70899999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.79600000000000004},"last":0.67500000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.32600000000000001,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.82999999999999996,"5min":1.446,"15min":1.446},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.80300000000000005,"5min":0.81599999999999995,"15min":0.89700000000000002},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.85199999999999998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.88600000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35199999999999998,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.183,"15min":0.183},"max":{"1min":0.46100000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.254,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.50700000000000001,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.376,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.55000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41699999999999998,"15min":0.42999999999999999},"min":{"1min":0.29299999999999998,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47699999999999998,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.308,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.879,"15min":0.94699999999999995},"last":0.622}]},{"osd":5,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":1.002,"15min":1.002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.37},"max":{"1min":0.85499999999999998,"5min":1.018,"15min":1.018},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.63500000000000001},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.873,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.68899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":690475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.376,"15min":0.371},"min":{"1min":0.22600000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.49099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.47299999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.39200000000000002,"15min":0.40300000000000002},"min":{"1min":0.27300000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55300000000000005,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.437},{"interface":"front","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.40500000000000003},"min":{"1min":0.27400000000000002,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.24299999999999999},"max":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.92600000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.624,"15min":0.64100000000000001},"min":{"1min":0.432,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":1.458},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79300000000000004,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":1.028},"last":0.80900000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81999999999999995,"5min":0.879,"15min":0.92500000000000004},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1372160,"data_stored":683925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.432,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.38800000000000001,"15min":0.38900000000000001},"min":{"1min":0.23799999999999999,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.51100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.38800000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.40200000000000002,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":0.627,"15min":0.64800000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26000000000000001},"max":{"1min":0.495,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.26800000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52300000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.73399999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.81799999999999995,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.624,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.374},"max":{"1min":0.871,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.371},"max":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575049,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1998848,"data_stored":1283161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.35999999999999999,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.50600000000000001,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.308},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.377,"15min":0.38600000000000001},"min":{"1min":0.26000000000000001,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.48899999999999999,"5min":0.62,"15min":0.76900000000000002},"last":0.34100000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.187},"max":{"1min":0.46700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39200000000000002,"15min":0.41499999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46200000000000002,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.28799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38300000000000001,"15min":0.39000000000000001},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.41799999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.307,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51300000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0649999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34699999999999998},"max":{"1min":0.98199999999999998,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":1.026,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.439}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.339,"15min":1.339},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.628,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:41:32.397 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:41:32.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.533+0000 7f999bf25640 1 -- 172.21.15.125:0/2815842153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9980005180 msgr2=0x7f998000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.533+0000 7f999bf25640 1 --2- 172.21.15.125:0/2815842153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9980005180 0x7f998000b980 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f998c009ed0 tx=0x7f99800057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.533+0000 7f999bf25640 1 -- 172.21.15.125:0/2815842153 shutdown_connections 2023-10-18T10:41:32.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.533+0000 7f999bf25640 1 --2- 172.21.15.125:0/2815842153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9980005180 0x7f998000b980 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.533+0000 7f999bf25640 1 -- 172.21.15.125:0/2815842153 >> 172.21.15.125:0/2815842153 conn(0x7f9994069930 msgr2=0x7f9994069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:32.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.533+0000 7f999bf25640 1 -- 172.21.15.125:0/2815842153 shutdown_connections 2023-10-18T10:41:32.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.533+0000 7f999bf25640 1 -- 172.21.15.125:0/2815842153 wait complete. 2023-10-18T10:41:32.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.534+0000 7f999bf25640 1 Processor -- start 2023-10-18T10:41:32.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.534+0000 7f999bf25640 1 -- start start 2023-10-18T10:41:32.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f999bf25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9980005180 0x7f99940ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f999bf25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99940b1310 0x7f9994152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f999bf25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99940aa8c0 0x7f99940aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f999bf25640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940ab280 con 0x7f99940b1310 2023-10-18T10:41:32.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f999bf25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999412f910 con 0x7f99940aa8c0 2023-10-18T10:41:32.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f999bf25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9994130490 con 0x7f9980005180 2023-10-18T10:41:32.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f99994c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99940b1310 0x7f9994152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f99994c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99940b1310 0x7f9994152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50898/0 (socket says 172.21.15.125:50898) 2023-10-18T10:41:32.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f99994c0640 1 -- 172.21.15.125:0/4152892397 learned_addr learned my addr 172.21.15.125:0/4152892397 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:32.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f9999cc1640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9980005180 0x7f99940ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f999a4c2640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99940aa8c0 0x7f99940aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f99994c0640 1 -- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9980005180 msgr2=0x7f99940ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f99994c0640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9980005180 0x7f99940ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f99994c0640 1 -- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99940aa8c0 msgr2=0x7f99940aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f99994c0640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99940aa8c0 0x7f99940aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f99994c0640 1 -- 172.21.15.125:0/4152892397 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999412fb60 con 0x7f99940b1310 2023-10-18T10:41:32.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.535+0000 7f9999cc1640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9980005180 0x7f99940ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:32.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.536+0000 7f99994c0640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99940b1310 0x7f9994152b40 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f998400e570 tx=0x7f998400ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:32.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.536+0000 7f998a7fc640 1 -- 172.21.15.125:0/4152892397 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998400c8f0 con 0x7f99940b1310 2023-10-18T10:41:32.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.536+0000 7f998a7fc640 1 -- 172.21.15.125:0/4152892397 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f998400cb40 con 0x7f99940b1310 2023-10-18T10:41:32.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.536+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999412fe40 con 0x7f99940b1310 2023-10-18T10:41:32.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.536+0000 7f998a7fc640 1 -- 172.21.15.125:0/4152892397 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984011460 con 0x7f99940b1310 2023-10-18T10:41:32.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.536+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99941512d0 con 0x7f99940b1310 2023-10-18T10:41:32.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.537+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999406c7e0 con 0x7f99940b1310 2023-10-18T10:41:32.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.537+0000 7f998a7fc640 1 -- 172.21.15.125:0/4152892397 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f99840115c0 con 0x7f99940b1310 2023-10-18T10:41:32.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.538+0000 7f998a7fc640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f997806af90 0x7f997806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.538+0000 7f998a7fc640 1 -- 172.21.15.125:0/4152892397 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f998408a080 con 0x7f99940b1310 2023-10-18T10:41:32.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.538+0000 7f9999cc1640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f997806af90 0x7f997806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.538+0000 7f9999cc1640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f997806af90 0x7f997806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9990000c00 tx=0x7f99900061b0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:32.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.542+0000 7f998a7fc640 1 -- 172.21.15.125:0/4152892397 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9984054e60 con 0x7f99940b1310 2023-10-18T10:41:32.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.705+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 --> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f999406c500 con 0x7f997806af90 2023-10-18T10:41:32.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.713+0000 7f998a7fc640 1 -- 172.21.15.125:0/4152892397 <== mgr.6299 v2:172.21.15.139:6832/1523403591 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283643 (secure 0 0 0) 0x7f999406c500 con 0x7f997806af90 2023-10-18T10:41:32.717 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:32.718 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:41:32.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f997806af90 msgr2=0x7f997806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f997806af90 0x7f997806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9990000c00 tx=0x7f99900061b0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99940b1310 msgr2=0x7f9994152b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99940b1310 0x7f9994152b40 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f998400e570 tx=0x7f998400ea30 comp rx=0 tx=0).stop 2023-10-18T10:41:32.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 shutdown_connections 2023-10-18T10:41:32.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f997806af90 0x7f997806d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99940aa8c0 0x7f99940aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99940b1310 0x7f9994152b40 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 --2- 172.21.15.125:0/4152892397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9980005180 0x7f99940ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 >> 172.21.15.125:0/4152892397 conn(0x7f9994069930 msgr2=0x7f99940b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:32.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 shutdown_connections 2023-10-18T10:41:32.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.715+0000 7f999bf25640 1 -- 172.21.15.125:0/4152892397 wait complete. 2023-10-18T10:41:32.727 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:41:31.281902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":314,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":116,"acting":116,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":313984,"kb_used_data":11780,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754660736,"statfs":{"total":773094113280,"available":772772593664,"internally_reserved":0,"allocated":12062720,"data_stored":6601716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000876"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434717+0000","last_change":"2023-10-18T10:41:29.447539+0000","last_active":"2023-10-18T10:41:30.434717+0000","last_peered":"2023-10-18T10:41:30.434717+0000","last_clean":"2023-10-18T10:41:30.434717+0000","last_became_active":"2023-10-18T10:41:29.447288+0000","last_became_peered":"2023-10-18T10:41:29.447288+0000","last_unstale":"2023-10-18T10:41:30.434717+0000","last_undegraded":"2023-10-18T10:41:30.434717+0000","last_fullsized":"2023-10-18T10:41:30.434717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:30.647260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750857+0000","last_change":"2023-10-18T10:41:29.442252+0000","last_active":"2023-10-18T10:41:30.750857+0000","last_peered":"2023-10-18T10:41:30.750857+0000","last_clean":"2023-10-18T10:41:30.750857+0000","last_became_active":"2023-10-18T10:41:29.441955+0000","last_became_peered":"2023-10-18T10:41:29.441955+0000","last_unstale":"2023-10-18T10:41:30.750857+0000","last_undegraded":"2023-10-18T10:41:30.750857+0000","last_fullsized":"2023-10-18T10:41:30.750857+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:23.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434545+0000","last_change":"2023-10-18T10:41:29.452284+0000","last_active":"2023-10-18T10:41:30.434545+0000","last_peered":"2023-10-18T10:41:30.434545+0000","last_clean":"2023-10-18T10:41:30.434545+0000","last_became_active":"2023-10-18T10:41:29.452037+0000","last_became_peered":"2023-10-18T10:41:29.452037+0000","last_unstale":"2023-10-18T10:41:30.434545+0000","last_undegraded":"2023-10-18T10:41:30.434545+0000","last_fullsized":"2023-10-18T10:41:30.434545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.154689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750025+0000","last_change":"2023-10-18T10:41:29.453178+0000","last_active":"2023-10-18T10:41:30.750025+0000","last_peered":"2023-10-18T10:41:30.750025+0000","last_clean":"2023-10-18T10:41:30.750025+0000","last_became_active":"2023-10-18T10:41:29.452902+0000","last_became_peered":"2023-10-18T10:41:29.452902+0000","last_unstale":"2023-10-18T10:41:30.750025+0000","last_undegraded":"2023-10-18T10:41:30.750025+0000","last_fullsized":"2023-10-18T10:41:30.750025+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:23.594087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750972+0000","last_change":"2023-10-18T10:41:29.450362+0000","last_active":"2023-10-18T10:41:30.750972+0000","last_peered":"2023-10-18T10:41:30.750972+0000","last_clean":"2023-10-18T10:41:30.750972+0000","last_became_active":"2023-10-18T10:41:29.449972+0000","last_became_peered":"2023-10-18T10:41:29.449972+0000","last_unstale":"2023-10-18T10:41:30.750972+0000","last_undegraded":"2023-10-18T10:41:30.750972+0000","last_fullsized":"2023-10-18T10:41:30.750972+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:59.437842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749904+0000","last_change":"2023-10-18T10:41:29.449393+0000","last_active":"2023-10-18T10:41:30.749904+0000","last_peered":"2023-10-18T10:41:30.749904+0000","last_clean":"2023-10-18T10:41:30.749904+0000","last_became_active":"2023-10-18T10:41:29.449086+0000","last_became_peered":"2023-10-18T10:41:29.449086+0000","last_unstale":"2023-10-18T10:41:30.749904+0000","last_undegraded":"2023-10-18T10:41:30.749904+0000","last_fullsized":"2023-10-18T10:41:30.749904+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:41.061091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751632+0000","last_change":"2023-10-18T10:41:29.444304+0000","last_active":"2023-10-18T10:41:30.751632+0000","last_peered":"2023-10-18T10:41:30.751632+0000","last_clean":"2023-10-18T10:41:30.751632+0000","last_became_active":"2023-10-18T10:41:29.444057+0000","last_became_peered":"2023-10-18T10:41:29.444057+0000","last_unstale":"2023-10-18T10:41:30.751632+0000","last_undegraded":"2023-10-18T10:41:30.751632+0000","last_fullsized":"2023-10-18T10:41:30.751632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:23.209276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751364+0000","last_change":"2023-10-18T10:41:29.443320+0000","last_active":"2023-10-18T10:41:30.751364+0000","last_peered":"2023-10-18T10:41:30.751364+0000","last_clean":"2023-10-18T10:41:30.751364+0000","last_became_active":"2023-10-18T10:41:29.443029+0000","last_became_peered":"2023-10-18T10:41:29.443029+0000","last_unstale":"2023-10-18T10:41:30.751364+0000","last_undegraded":"2023-10-18T10:41:30.751364+0000","last_fullsized":"2023-10-18T10:41:30.751364+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:27.802284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750628+0000","last_change":"2023-10-18T10:41:29.448605+0000","last_active":"2023-10-18T10:41:30.750628+0000","last_peered":"2023-10-18T10:41:30.750628+0000","last_clean":"2023-10-18T10:41:30.750628+0000","last_became_active":"2023-10-18T10:41:29.448348+0000","last_became_peered":"2023-10-18T10:41:29.448348+0000","last_unstale":"2023-10-18T10:41:30.750628+0000","last_undegraded":"2023-10-18T10:41:30.750628+0000","last_fullsized":"2023-10-18T10:41:30.750628+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:42.695172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750043+0000","last_change":"2023-10-18T10:41:29.444407+0000","last_active":"2023-10-18T10:41:30.750043+0000","last_peered":"2023-10-18T10:41:30.750043+0000","last_clean":"2023-10-18T10:41:30.750043+0000","last_became_active":"2023-10-18T10:41:29.444169+0000","last_became_peered":"2023-10-18T10:41:29.444169+0000","last_unstale":"2023-10-18T10:41:30.750043+0000","last_undegraded":"2023-10-18T10:41:30.750043+0000","last_fullsized":"2023-10-18T10:41:30.750043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:13.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750781+0000","last_change":"2023-10-18T10:41:29.454015+0000","last_active":"2023-10-18T10:41:30.750781+0000","last_peered":"2023-10-18T10:41:30.750781+0000","last_clean":"2023-10-18T10:41:30.750781+0000","last_became_active":"2023-10-18T10:41:29.451538+0000","last_became_peered":"2023-10-18T10:41:29.451538+0000","last_unstale":"2023-10-18T10:41:30.750781+0000","last_undegraded":"2023-10-18T10:41:30.750781+0000","last_fullsized":"2023-10-18T10:41:30.750781+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:52.418194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751142+0000","last_change":"2023-10-18T10:41:29.449763+0000","last_active":"2023-10-18T10:41:30.751142+0000","last_peered":"2023-10-18T10:41:30.751142+0000","last_clean":"2023-10-18T10:41:30.751142+0000","last_became_active":"2023-10-18T10:41:29.449498+0000","last_became_peered":"2023-10-18T10:41:29.449498+0000","last_unstale":"2023-10-18T10:41:30.751142+0000","last_undegraded":"2023-10-18T10:41:30.751142+0000","last_fullsized":"2023-10-18T10:41:30.751142+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:30.606377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434672+0000","last_change":"2023-10-18T10:41:29.448915+0000","last_active":"2023-10-18T10:41:30.434672+0000","last_peered":"2023-10-18T10:41:30.434672+0000","last_clean":"2023-10-18T10:41:30.434672+0000","last_became_active":"2023-10-18T10:41:29.448650+0000","last_became_peered":"2023-10-18T10:41:29.448650+0000","last_unstale":"2023-10-18T10:41:30.434672+0000","last_undegraded":"2023-10-18T10:41:30.434672+0000","last_fullsized":"2023-10-18T10:41:30.434672+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:41.152281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750152+0000","last_change":"2023-10-18T10:41:29.455557+0000","last_active":"2023-10-18T10:41:30.750152+0000","last_peered":"2023-10-18T10:41:30.750152+0000","last_clean":"2023-10-18T10:41:30.750152+0000","last_became_active":"2023-10-18T10:41:29.455296+0000","last_became_peered":"2023-10-18T10:41:29.455296+0000","last_unstale":"2023-10-18T10:41:30.750152+0000","last_undegraded":"2023-10-18T10:41:30.750152+0000","last_fullsized":"2023-10-18T10:41:30.750152+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:51.749366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434493+0000","last_change":"2023-10-18T10:41:29.453702+0000","last_active":"2023-10-18T10:41:30.434493+0000","last_peered":"2023-10-18T10:41:30.434493+0000","last_clean":"2023-10-18T10:41:30.434493+0000","last_became_active":"2023-10-18T10:41:29.453445+0000","last_became_peered":"2023-10-18T10:41:29.453445+0000","last_unstale":"2023-10-18T10:41:30.434493+0000","last_undegraded":"2023-10-18T10:41:30.434493+0000","last_fullsized":"2023-10-18T10:41:30.434493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:28.572719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750696+0000","last_change":"2023-10-18T10:41:29.451267+0000","last_active":"2023-10-18T10:41:30.750696+0000","last_peered":"2023-10-18T10:41:30.750696+0000","last_clean":"2023-10-18T10:41:30.750696+0000","last_became_active":"2023-10-18T10:41:29.450963+0000","last_became_peered":"2023-10-18T10:41:29.450963+0000","last_unstale":"2023-10-18T10:41:30.750696+0000","last_undegraded":"2023-10-18T10:41:30.750696+0000","last_fullsized":"2023-10-18T10:41:30.750696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:01.499934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751548+0000","last_change":"2023-10-18T10:41:29.453816+0000","last_active":"2023-10-18T10:41:30.751548+0000","last_peered":"2023-10-18T10:41:30.751548+0000","last_clean":"2023-10-18T10:41:30.751548+0000","last_became_active":"2023-10-18T10:41:29.453581+0000","last_became_peered":"2023-10-18T10:41:29.453581+0000","last_unstale":"2023-10-18T10:41:30.751548+0000","last_undegraded":"2023-10-18T10:41:30.751548+0000","last_fullsized":"2023-10-18T10:41:30.751548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:18.669429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750180+0000","last_change":"2023-10-18T10:41:29.454500+0000","last_active":"2023-10-18T10:41:30.750180+0000","last_peered":"2023-10-18T10:41:30.750180+0000","last_clean":"2023-10-18T10:41:30.750180+0000","last_became_active":"2023-10-18T10:41:29.454231+0000","last_became_peered":"2023-10-18T10:41:29.454231+0000","last_unstale":"2023-10-18T10:41:30.750180+0000","last_undegraded":"2023-10-18T10:41:30.750180+0000","last_fullsized":"2023-10-18T10:41:30.750180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:14.224881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750335+0000","last_change":"2023-10-18T10:41:29.453993+0000","last_active":"2023-10-18T10:41:30.750335+0000","last_peered":"2023-10-18T10:41:30.750335+0000","last_clean":"2023-10-18T10:41:30.750335+0000","last_became_active":"2023-10-18T10:41:29.453659+0000","last_became_peered":"2023-10-18T10:41:29.453659+0000","last_unstale":"2023-10-18T10:41:30.750335+0000","last_undegraded":"2023-10-18T10:41:30.750335+0000","last_fullsized":"2023-10-18T10:41:30.750335+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:50.051564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750971+0000","last_change":"2023-10-18T10:41:29.445142+0000","last_active":"2023-10-18T10:41:30.750971+0000","last_peered":"2023-10-18T10:41:30.750971+0000","last_clean":"2023-10-18T10:41:30.750971+0000","last_became_active":"2023-10-18T10:41:29.444871+0000","last_became_peered":"2023-10-18T10:41:29.444871+0000","last_unstale":"2023-10-18T10:41:30.750971+0000","last_undegraded":"2023-10-18T10:41:30.750971+0000","last_fullsized":"2023-10-18T10:41:30.750971+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:31:23.805032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750791+0000","last_change":"2023-10-18T10:41:29.448568+0000","last_active":"2023-10-18T10:41:30.750791+0000","last_peered":"2023-10-18T10:41:30.750791+0000","last_clean":"2023-10-18T10:41:30.750791+0000","last_became_active":"2023-10-18T10:41:29.448318+0000","last_became_peered":"2023-10-18T10:41:29.448318+0000","last_unstale":"2023-10-18T10:41:30.750791+0000","last_undegraded":"2023-10-18T10:41:30.750791+0000","last_fullsized":"2023-10-18T10:41:30.750791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:29.589441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751302+0000","last_change":"2023-10-18T10:41:29.445971+0000","last_active":"2023-10-18T10:41:30.751302+0000","last_peered":"2023-10-18T10:41:30.751302+0000","last_clean":"2023-10-18T10:41:30.751302+0000","last_became_active":"2023-10-18T10:41:29.445723+0000","last_became_peered":"2023-10-18T10:41:29.445723+0000","last_unstale":"2023-10-18T10:41:30.751302+0000","last_undegraded":"2023-10-18T10:41:30.751302+0000","last_fullsized":"2023-10-18T10:41:30.751302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:35:45.397768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751225+0000","last_change":"2023-10-18T10:41:29.451021+0000","last_active":"2023-10-18T10:41:30.751225+0000","last_peered":"2023-10-18T10:41:30.751225+0000","last_clean":"2023-10-18T10:41:30.751225+0000","last_became_active":"2023-10-18T10:41:29.450779+0000","last_became_peered":"2023-10-18T10:41:29.450779+0000","last_unstale":"2023-10-18T10:41:30.751225+0000","last_undegraded":"2023-10-18T10:41:30.751225+0000","last_fullsized":"2023-10-18T10:41:30.751225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:36.701370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749949+0000","last_change":"2023-10-18T10:41:29.443511+0000","last_active":"2023-10-18T10:41:30.749949+0000","last_peered":"2023-10-18T10:41:30.749949+0000","last_clean":"2023-10-18T10:41:30.749949+0000","last_became_active":"2023-10-18T10:41:29.443232+0000","last_became_peered":"2023-10-18T10:41:29.443232+0000","last_unstale":"2023-10-18T10:41:30.749949+0000","last_undegraded":"2023-10-18T10:41:30.749949+0000","last_fullsized":"2023-10-18T10:41:30.749949+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:29.869647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750830+0000","last_change":"2023-10-18T10:41:28.439965+0000","last_active":"2023-10-18T10:41:30.750830+0000","last_peered":"2023-10-18T10:41:30.750830+0000","last_clean":"2023-10-18T10:41:30.750830+0000","last_became_active":"2023-10-18T10:41:28.439700+0000","last_became_peered":"2023-10-18T10:41:28.439700+0000","last_unstale":"2023-10-18T10:41:30.750830+0000","last_undegraded":"2023-10-18T10:41:30.750830+0000","last_fullsized":"2023-10-18T10:41:30.750830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:44.202065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750979+0000","last_change":"2023-10-18T10:41:29.450500+0000","last_active":"2023-10-18T10:41:30.750979+0000","last_peered":"2023-10-18T10:41:30.750979+0000","last_clean":"2023-10-18T10:41:30.750979+0000","last_became_active":"2023-10-18T10:41:29.450234+0000","last_became_peered":"2023-10-18T10:41:29.450234+0000","last_unstale":"2023-10-18T10:41:30.750979+0000","last_undegraded":"2023-10-18T10:41:30.750979+0000","last_fullsized":"2023-10-18T10:41:30.750979+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:08.866019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434561+0000","last_change":"2023-10-18T10:41:28.435875+0000","last_active":"2023-10-18T10:41:30.434561+0000","last_peered":"2023-10-18T10:41:30.434561+0000","last_clean":"2023-10-18T10:41:30.434561+0000","last_became_active":"2023-10-18T10:41:28.435623+0000","last_became_peered":"2023-10-18T10:41:28.435623+0000","last_unstale":"2023-10-18T10:41:30.434561+0000","last_undegraded":"2023-10-18T10:41:30.434561+0000","last_fullsized":"2023-10-18T10:41:30.434561+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:22.897591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750655+0000","last_change":"2023-10-18T10:41:29.451882+0000","last_active":"2023-10-18T10:41:30.750655+0000","last_peered":"2023-10-18T10:41:30.750655+0000","last_clean":"2023-10-18T10:41:30.750655+0000","last_became_active":"2023-10-18T10:41:29.451623+0000","last_became_peered":"2023-10-18T10:41:29.451623+0000","last_unstale":"2023-10-18T10:41:30.750655+0000","last_undegraded":"2023-10-18T10:41:30.750655+0000","last_fullsized":"2023-10-18T10:41:30.750655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:39.862682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750926+0000","last_change":"2023-10-18T10:41:28.436855+0000","last_active":"2023-10-18T10:41:30.750926+0000","last_peered":"2023-10-18T10:41:30.750926+0000","last_clean":"2023-10-18T10:41:30.750926+0000","last_became_active":"2023-10-18T10:41:28.436585+0000","last_became_peered":"2023-10-18T10:41:28.436585+0000","last_unstale":"2023-10-18T10:41:30.750926+0000","last_undegraded":"2023-10-18T10:41:30.750926+0000","last_fullsized":"2023-10-18T10:41:30.750926+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.155833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434722+0000","last_change":"2023-10-18T10:41:28.440969+0000","last_active":"2023-10-18T10:41:30.434722+0000","last_peered":"2023-10-18T10:41:30.434722+0000","last_clean":"2023-10-18T10:41:30.434722+0000","last_became_active":"2023-10-18T10:41:28.440739+0000","last_became_peered":"2023-10-18T10:41:28.440739+0000","last_unstale":"2023-10-18T10:41:30.434722+0000","last_undegraded":"2023-10-18T10:41:30.434722+0000","last_fullsized":"2023-10-18T10:41:30.434722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.620461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434696+0000","last_change":"2023-10-18T10:41:29.456187+0000","last_active":"2023-10-18T10:41:30.434696+0000","last_peered":"2023-10-18T10:41:30.434696+0000","last_clean":"2023-10-18T10:41:30.434696+0000","last_became_active":"2023-10-18T10:41:29.455945+0000","last_became_peered":"2023-10-18T10:41:29.455945+0000","last_unstale":"2023-10-18T10:41:30.434696+0000","last_undegraded":"2023-10-18T10:41:30.434696+0000","last_fullsized":"2023-10-18T10:41:30.434696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:40.835626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750493+0000","last_change":"2023-10-18T10:41:29.449408+0000","last_active":"2023-10-18T10:41:30.750493+0000","last_peered":"2023-10-18T10:41:30.750493+0000","last_clean":"2023-10-18T10:41:30.750493+0000","last_became_active":"2023-10-18T10:41:29.449161+0000","last_became_peered":"2023-10-18T10:41:29.449161+0000","last_unstale":"2023-10-18T10:41:30.750493+0000","last_undegraded":"2023-10-18T10:41:30.750493+0000","last_fullsized":"2023-10-18T10:41:30.750493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:41.742410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"111'6","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751094+0000","last_change":"2023-10-18T10:41:28.427403+0000","last_active":"2023-10-18T10:41:30.751094+0000","last_peered":"2023-10-18T10:41:30.751094+0000","last_clean":"2023-10-18T10:41:30.751094+0000","last_became_active":"2023-10-18T10:41:28.427119+0000","last_became_peered":"2023-10-18T10:41:28.427119+0000","last_unstale":"2023-10-18T10:41:30.751094+0000","last_undegraded":"2023-10-18T10:41:30.751094+0000","last_fullsized":"2023-10-18T10:41:30.751094+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:11.156072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750239+0000","last_change":"2023-10-18T10:41:29.448792+0000","last_active":"2023-10-18T10:41:30.750239+0000","last_peered":"2023-10-18T10:41:30.750239+0000","last_clean":"2023-10-18T10:41:30.750239+0000","last_became_active":"2023-10-18T10:41:29.448536+0000","last_became_peered":"2023-10-18T10:41:29.448536+0000","last_unstale":"2023-10-18T10:41:30.750239+0000","last_undegraded":"2023-10-18T10:41:30.750239+0000","last_fullsized":"2023-10-18T10:41:30.750239+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:14.300602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751060+0000","last_change":"2023-10-18T10:41:28.433856+0000","last_active":"2023-10-18T10:41:30.751060+0000","last_peered":"2023-10-18T10:41:30.751060+0000","last_clean":"2023-10-18T10:41:30.751060+0000","last_became_active":"2023-10-18T10:41:28.433584+0000","last_became_peered":"2023-10-18T10:41:28.433584+0000","last_unstale":"2023-10-18T10:41:30.751060+0000","last_undegraded":"2023-10-18T10:41:30.751060+0000","last_fullsized":"2023-10-18T10:41:30.751060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:06:07.806734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750178+0000","last_change":"2023-10-18T10:41:29.439852+0000","last_active":"2023-10-18T10:41:30.750178+0000","last_peered":"2023-10-18T10:41:30.750178+0000","last_clean":"2023-10-18T10:41:30.750178+0000","last_became_active":"2023-10-18T10:41:29.439612+0000","last_became_peered":"2023-10-18T10:41:29.439612+0000","last_unstale":"2023-10-18T10:41:30.750178+0000","last_undegraded":"2023-10-18T10:41:30.750178+0000","last_fullsized":"2023-10-18T10:41:30.750178+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:10.074146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750194+0000","last_change":"2023-10-18T10:41:28.431914+0000","last_active":"2023-10-18T10:41:30.750194+0000","last_peered":"2023-10-18T10:41:30.750194+0000","last_clean":"2023-10-18T10:41:30.750194+0000","last_became_active":"2023-10-18T10:41:28.431664+0000","last_became_peered":"2023-10-18T10:41:28.431664+0000","last_unstale":"2023-10-18T10:41:30.750194+0000","last_undegraded":"2023-10-18T10:41:30.750194+0000","last_fullsized":"2023-10-18T10:41:30.750194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:41.649804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751367+0000","last_change":"2023-10-18T10:41:29.453041+0000","last_active":"2023-10-18T10:41:30.751367+0000","last_peered":"2023-10-18T10:41:30.751367+0000","last_clean":"2023-10-18T10:41:30.751367+0000","last_became_active":"2023-10-18T10:41:29.452788+0000","last_became_peered":"2023-10-18T10:41:29.452788+0000","last_unstale":"2023-10-18T10:41:30.751367+0000","last_undegraded":"2023-10-18T10:41:30.751367+0000","last_fullsized":"2023-10-18T10:41:30.751367+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:28.740818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"111'8","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750045+0000","last_change":"2023-10-18T10:41:28.440009+0000","last_active":"2023-10-18T10:41:30.750045+0000","last_peered":"2023-10-18T10:41:30.750045+0000","last_clean":"2023-10-18T10:41:30.750045+0000","last_became_active":"2023-10-18T10:41:28.439742+0000","last_became_peered":"2023-10-18T10:41:28.439742+0000","last_unstale":"2023-10-18T10:41:30.750045+0000","last_undegraded":"2023-10-18T10:41:30.750045+0000","last_fullsized":"2023-10-18T10:41:30.750045+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:42.072352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751005+0000","last_change":"2023-10-18T10:41:29.448022+0000","last_active":"2023-10-18T10:41:30.751005+0000","last_peered":"2023-10-18T10:41:30.751005+0000","last_clean":"2023-10-18T10:41:30.751005+0000","last_became_active":"2023-10-18T10:41:29.447771+0000","last_became_peered":"2023-10-18T10:41:29.447771+0000","last_unstale":"2023-10-18T10:41:30.751005+0000","last_undegraded":"2023-10-18T10:41:30.751005+0000","last_fullsized":"2023-10-18T10:41:30.751005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:13.373299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751109+0000","last_change":"2023-10-18T10:41:29.450342+0000","last_active":"2023-10-18T10:41:30.751109+0000","last_peered":"2023-10-18T10:41:30.751109+0000","last_clean":"2023-10-18T10:41:30.751109+0000","last_became_active":"2023-10-18T10:41:29.450078+0000","last_became_peered":"2023-10-18T10:41:29.450078+0000","last_unstale":"2023-10-18T10:41:30.751109+0000","last_undegraded":"2023-10-18T10:41:30.751109+0000","last_fullsized":"2023-10-18T10:41:30.751109+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:32.976432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750276+0000","last_change":"2023-10-18T10:41:28.438968+0000","last_active":"2023-10-18T10:41:30.750276+0000","last_peered":"2023-10-18T10:41:30.750276+0000","last_clean":"2023-10-18T10:41:30.750276+0000","last_became_active":"2023-10-18T10:41:28.438628+0000","last_became_peered":"2023-10-18T10:41:28.438628+0000","last_unstale":"2023-10-18T10:41:30.750276+0000","last_undegraded":"2023-10-18T10:41:30.750276+0000","last_fullsized":"2023-10-18T10:41:30.750276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:51.950983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750427+0000","last_change":"2023-10-18T10:41:29.442931+0000","last_active":"2023-10-18T10:41:30.750427+0000","last_peered":"2023-10-18T10:41:30.750427+0000","last_clean":"2023-10-18T10:41:30.750427+0000","last_became_active":"2023-10-18T10:41:29.442573+0000","last_became_peered":"2023-10-18T10:41:29.442573+0000","last_unstale":"2023-10-18T10:41:30.750427+0000","last_undegraded":"2023-10-18T10:41:30.750427+0000","last_fullsized":"2023-10-18T10:41:30.750427+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:36.667760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750311+0000","last_change":"2023-10-18T10:41:28.437156+0000","last_active":"2023-10-18T10:41:30.750311+0000","last_peered":"2023-10-18T10:41:30.750311+0000","last_clean":"2023-10-18T10:41:30.750311+0000","last_became_active":"2023-10-18T10:41:28.434657+0000","last_became_peered":"2023-10-18T10:41:28.434657+0000","last_unstale":"2023-10-18T10:41:30.750311+0000","last_undegraded":"2023-10-18T10:41:30.750311+0000","last_fullsized":"2023-10-18T10:41:30.750311+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:38:04.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750603+0000","last_change":"2023-10-18T10:41:29.452874+0000","last_active":"2023-10-18T10:41:30.750603+0000","last_peered":"2023-10-18T10:41:30.750603+0000","last_clean":"2023-10-18T10:41:30.750603+0000","last_became_active":"2023-10-18T10:41:29.452561+0000","last_became_peered":"2023-10-18T10:41:29.452561+0000","last_unstale":"2023-10-18T10:41:30.750603+0000","last_undegraded":"2023-10-18T10:41:30.750603+0000","last_fullsized":"2023-10-18T10:41:30.750603+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:26.000624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750422+0000","last_change":"2023-10-18T10:41:28.435576+0000","last_active":"2023-10-18T10:41:30.750422+0000","last_peered":"2023-10-18T10:41:30.750422+0000","last_clean":"2023-10-18T10:41:30.750422+0000","last_became_active":"2023-10-18T10:41:28.435286+0000","last_became_peered":"2023-10-18T10:41:28.435286+0000","last_unstale":"2023-10-18T10:41:30.750422+0000","last_undegraded":"2023-10-18T10:41:30.750422+0000","last_fullsized":"2023-10-18T10:41:30.750422+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:29.719996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750887+0000","last_change":"2023-10-18T10:41:29.446250+0000","last_active":"2023-10-18T10:41:30.750887+0000","last_peered":"2023-10-18T10:41:30.750887+0000","last_clean":"2023-10-18T10:41:30.750887+0000","last_became_active":"2023-10-18T10:41:29.445881+0000","last_became_peered":"2023-10-18T10:41:29.445881+0000","last_unstale":"2023-10-18T10:41:30.750887+0000","last_undegraded":"2023-10-18T10:41:30.750887+0000","last_fullsized":"2023-10-18T10:41:30.750887+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:28.061711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750340+0000","last_change":"2023-10-18T10:41:28.438325+0000","last_active":"2023-10-18T10:41:30.750340+0000","last_peered":"2023-10-18T10:41:30.750340+0000","last_clean":"2023-10-18T10:41:30.750340+0000","last_became_active":"2023-10-18T10:41:28.438056+0000","last_became_peered":"2023-10-18T10:41:28.438056+0000","last_unstale":"2023-10-18T10:41:30.750340+0000","last_undegraded":"2023-10-18T10:41:30.750340+0000","last_fullsized":"2023-10-18T10:41:30.750340+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:42.738454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751473+0000","last_change":"2023-10-18T10:41:29.449610+0000","last_active":"2023-10-18T10:41:30.751473+0000","last_peered":"2023-10-18T10:41:30.751473+0000","last_clean":"2023-10-18T10:41:30.751473+0000","last_became_active":"2023-10-18T10:41:29.449354+0000","last_became_peered":"2023-10-18T10:41:29.449354+0000","last_unstale":"2023-10-18T10:41:30.751473+0000","last_undegraded":"2023-10-18T10:41:30.751473+0000","last_fullsized":"2023-10-18T10:41:30.751473+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:01.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751349+0000","last_change":"2023-10-18T10:41:29.447215+0000","last_active":"2023-10-18T10:41:30.751349+0000","last_peered":"2023-10-18T10:41:30.751349+0000","last_clean":"2023-10-18T10:41:30.751349+0000","last_became_active":"2023-10-18T10:41:29.446983+0000","last_became_peered":"2023-10-18T10:41:29.446983+0000","last_unstale":"2023-10-18T10:41:30.751349+0000","last_undegraded":"2023-10-18T10:41:30.751349+0000","last_fullsized":"2023-10-18T10:41:30.751349+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750837+0000","last_change":"2023-10-18T10:41:29.448872+0000","last_active":"2023-10-18T10:41:30.750837+0000","last_peered":"2023-10-18T10:41:30.750837+0000","last_clean":"2023-10-18T10:41:30.750837+0000","last_became_active":"2023-10-18T10:41:29.448574+0000","last_became_peered":"2023-10-18T10:41:29.448574+0000","last_unstale":"2023-10-18T10:41:30.750837+0000","last_undegraded":"2023-10-18T10:41:30.750837+0000","last_fullsized":"2023-10-18T10:41:30.750837+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:40.934129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749895+0000","last_change":"2023-10-18T10:41:29.442254+0000","last_active":"2023-10-18T10:41:30.749895+0000","last_peered":"2023-10-18T10:41:30.749895+0000","last_clean":"2023-10-18T10:41:30.749895+0000","last_became_active":"2023-10-18T10:41:29.441995+0000","last_became_peered":"2023-10-18T10:41:29.441995+0000","last_unstale":"2023-10-18T10:41:30.749895+0000","last_undegraded":"2023-10-18T10:41:30.749895+0000","last_fullsized":"2023-10-18T10:41:30.749895+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:13.960239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751093+0000","last_change":"2023-10-18T10:41:28.434524+0000","last_active":"2023-10-18T10:41:30.751093+0000","last_peered":"2023-10-18T10:41:30.751093+0000","last_clean":"2023-10-18T10:41:30.751093+0000","last_became_active":"2023-10-18T10:41:28.434260+0000","last_became_peered":"2023-10-18T10:41:28.434260+0000","last_unstale":"2023-10-18T10:41:30.751093+0000","last_undegraded":"2023-10-18T10:41:30.751093+0000","last_fullsized":"2023-10-18T10:41:30.751093+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:26:18.323130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434509+0000","last_change":"2023-10-18T10:41:29.455708+0000","last_active":"2023-10-18T10:41:30.434509+0000","last_peered":"2023-10-18T10:41:30.434509+0000","last_clean":"2023-10-18T10:41:30.434509+0000","last_became_active":"2023-10-18T10:41:29.455459+0000","last_became_peered":"2023-10-18T10:41:29.455459+0000","last_unstale":"2023-10-18T10:41:30.434509+0000","last_undegraded":"2023-10-18T10:41:30.434509+0000","last_fullsized":"2023-10-18T10:41:30.434509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:01.952901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750593+0000","last_change":"2023-10-18T10:41:28.439218+0000","last_active":"2023-10-18T10:41:30.750593+0000","last_peered":"2023-10-18T10:41:30.750593+0000","last_clean":"2023-10-18T10:41:30.750593+0000","last_became_active":"2023-10-18T10:41:28.438612+0000","last_became_peered":"2023-10-18T10:41:28.438612+0000","last_unstale":"2023-10-18T10:41:30.750593+0000","last_undegraded":"2023-10-18T10:41:30.750593+0000","last_fullsized":"2023-10-18T10:41:30.750593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:22.330220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751017+0000","last_change":"2023-10-18T10:41:29.446112+0000","last_active":"2023-10-18T10:41:30.751017+0000","last_peered":"2023-10-18T10:41:30.751017+0000","last_clean":"2023-10-18T10:41:30.751017+0000","last_became_active":"2023-10-18T10:41:29.445569+0000","last_became_peered":"2023-10-18T10:41:29.445569+0000","last_unstale":"2023-10-18T10:41:30.751017+0000","last_undegraded":"2023-10-18T10:41:30.751017+0000","last_fullsized":"2023-10-18T10:41:30.751017+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:23.963420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750265+0000","last_change":"2023-10-18T10:41:28.440086+0000","last_active":"2023-10-18T10:41:30.750265+0000","last_peered":"2023-10-18T10:41:30.750265+0000","last_clean":"2023-10-18T10:41:30.750265+0000","last_became_active":"2023-10-18T10:41:28.439814+0000","last_became_peered":"2023-10-18T10:41:28.439814+0000","last_unstale":"2023-10-18T10:41:30.750265+0000","last_undegraded":"2023-10-18T10:41:30.750265+0000","last_fullsized":"2023-10-18T10:41:30.750265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:19.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"101'192","reported_seq":513,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T10:41:24.714727+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:41:24.714727+0000","last_peered":"2023-10-18T10:41:24.714727+0000","last_clean":"2023-10-18T10:41:24.714727+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:41:24.714727+0000","last_undegraded":"2023-10-18T10:41:24.714727+0000","last_fullsized":"2023-10-18T10:41:24.714727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":30,"acting":30,"num_store_stats":5},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":1912832,"data_stored":1248389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.876,"5min":0.876,"15min":0.88300000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68200000000000005,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73199999999999998,"5min":0.84199999999999997,"15min":0.84899999999999998},"last":0.51900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.627}]},{"osd":4,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.495,"15min":0.48799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.875,"15min":0.875},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78400000000000003,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.69999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32492,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1331200,"data_stored":664028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.34499999999999997,"15min":0.28699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.78500000000000003,"15min":0.82599999999999996},"last":0.438}]},{"osd":1,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.79500000000000004,"15min":0.83399999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.80900000000000005,"15min":0.82699999999999996},"last":0.621}]},{"osd":2,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55800000000000005,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65000000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.64900000000000002,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.66900000000000004,"5min":0.75600000000000001,"15min":0.84399999999999997},"last":0.69699999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.441,"15min":0.44},"min":{"1min":0.30399999999999999,"5min":0.29199999999999998,"15min":0.24199999999999999},"max":{"1min":0.76500000000000001,"5min":0.872,"15min":0.872},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45600000000000002,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65500000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48199999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82499999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.68300000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.749},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72199999999999998,"5min":0.82199999999999995,"15min":0.87},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57199999999999995,"15min":0.58399999999999996},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69599999999999995,"5min":0.82499999999999996,"15min":1.016},"last":0.434},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.70099999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.628},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":1.0620000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.80100000000000005,"15min":1.262},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73099999999999998,"5min":0.96099999999999997,"15min":1.224},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":0.871,"15min":1.159},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":664028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.877,"15min":0.877},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.76900000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.63100000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.70899999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.79600000000000004},"last":0.67500000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.32600000000000001,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.82999999999999996,"5min":1.446,"15min":1.446},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.80300000000000005,"5min":0.81599999999999995,"15min":0.89700000000000002},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.85199999999999998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.88600000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35199999999999998,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.183,"15min":0.183},"max":{"1min":0.46100000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.254,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.50700000000000001,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.376,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.55000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41699999999999998,"15min":0.42999999999999999},"min":{"1min":0.29299999999999998,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47699999999999998,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.308,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.879,"15min":0.94699999999999995},"last":0.622}]},{"osd":5,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":1.002,"15min":1.002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.37},"max":{"1min":0.85499999999999998,"5min":1.018,"15min":1.018},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.63500000000000001},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.873,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.68899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":690475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.376,"15min":0.371},"min":{"1min":0.22600000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.49099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.47299999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.39200000000000002,"15min":0.40300000000000002},"min":{"1min":0.27300000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55300000000000005,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.437},{"interface":"front","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.40500000000000003},"min":{"1min":0.27400000000000002,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.24299999999999999},"max":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.92600000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.624,"15min":0.64100000000000001},"min":{"1min":0.432,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":1.458},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79300000000000004,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":1.028},"last":0.80900000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81999999999999995,"5min":0.879,"15min":0.92500000000000004},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1372160,"data_stored":683925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.432,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.38800000000000001,"15min":0.38900000000000001},"min":{"1min":0.23799999999999999,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.51100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.38800000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.40200000000000002,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":0.627,"15min":0.64800000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26000000000000001},"max":{"1min":0.495,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.26800000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52300000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.73399999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.81799999999999995,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.624,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.374},"max":{"1min":0.871,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.371},"max":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575049,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1998848,"data_stored":1283161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.35999999999999999,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.50600000000000001,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.308},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.377,"15min":0.38600000000000001},"min":{"1min":0.26000000000000001,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.48899999999999999,"5min":0.62,"15min":0.76900000000000002},"last":0.34100000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.187},"max":{"1min":0.46700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39200000000000002,"15min":0.41499999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46200000000000002,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.28799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38300000000000001,"15min":0.39000000000000001},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.41799999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.307,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51300000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0649999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34699999999999998},"max":{"1min":0.98199999999999998,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":1.026,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.439}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.339,"15min":1.339},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.628,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:41:32.733 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:41:32.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.868+0000 7f0781ff0640 1 -- 172.21.15.125:0/288552806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 msgr2=0x7f077c15c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.868+0000 7f0781ff0640 1 --2- 172.21.15.125:0/288552806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 0x7f077c15c770 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cb9e0 tx=0x7f077002bdb0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.868+0000 7f0781ff0640 1 -- 172.21.15.125:0/288552806 shutdown_connections 2023-10-18T10:41:32.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.868+0000 7f0781ff0640 1 --2- 172.21.15.125:0/288552806 >> v2:172.21.15.139:3300/0 conn(0x7f077c15ccb0 0x7f077c0c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.868+0000 7f0781ff0640 1 --2- 172.21.15.125:0/288552806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 0x7f077c15c770 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.868+0000 7f0781ff0640 1 -- 172.21.15.125:0/288552806 >> 172.21.15.125:0/288552806 conn(0x7f077c069930 msgr2=0x7f077c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:32.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.868+0000 7f0781ff0640 1 -- 172.21.15.125:0/288552806 shutdown_connections 2023-10-18T10:41:32.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.868+0000 7f0781ff0640 1 -- 172.21.15.125:0/288552806 wait complete. 2023-10-18T10:41:32.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.869+0000 7f0781ff0640 1 Processor -- start 2023-10-18T10:41:32.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.869+0000 7f0781ff0640 1 -- start start 2023-10-18T10:41:32.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0781ff0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f077c06b030 0x7f077c1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0781ff0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 0x7f077c0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0781ff0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f077c15ccb0 0x7f077c0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0781ff0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c151420 con 0x7f077c06b410 2023-10-18T10:41:32.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0781ff0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c151700 con 0x7f077c15ccb0 2023-10-18T10:41:32.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0781ff0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c1519e0 con 0x7f077c06b030 2023-10-18T10:41:32.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 0x7f077c0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 0x7f077c0aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50918/0 (socket says 172.21.15.125:50918) 2023-10-18T10:41:32.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 -- 172.21.15.125:0/2005944688 learned_addr learned my addr 172.21.15.125:0/2005944688 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:32.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077b7fe640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f077c06b030 0x7f077c1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 -- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f077c06b030 msgr2=0x7f077c1f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f077c06b030 0x7f077c1f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 -- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f077c15ccb0 msgr2=0x7f077c0aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:32.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077bfff640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f077c15ccb0 0x7f077c0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f077c15ccb0 0x7f077c0aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 -- 172.21.15.125:0/2005944688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770009560 con 0x7f077c06b410 2023-10-18T10:41:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f077affd640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 0x7f077c0aa8c0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0770036040 tx=0x7f077002dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0763fff640 1 -- 172.21.15.125:0/2005944688 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770003800 con 0x7f077c06b410 2023-10-18T10:41:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0763fff640 1 -- 172.21.15.125:0/2005944688 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0770003a50 con 0x7f077c06b410 2023-10-18T10:41:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c151cc0 con 0x7f077c06b410 2023-10-18T10:41:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.870+0000 7f0763fff640 1 -- 172.21.15.125:0/2005944688 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770003de0 con 0x7f077c06b410 2023-10-18T10:41:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.871+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c1f44d0 con 0x7f077c06b410 2023-10-18T10:41:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.872+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c151b20 con 0x7f077c06b410 2023-10-18T10:41:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.872+0000 7f0763fff640 1 -- 172.21.15.125:0/2005944688 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f077003e450 con 0x7f077c06b410 2023-10-18T10:41:32.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.872+0000 7f0763fff640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f076806af60 0x7f076806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:32.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.872+0000 7f0763fff640 1 -- 172.21.15.125:0/2005944688 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f07700ac3e0 con 0x7f077c06b410 2023-10-18T10:41:32.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.873+0000 7f077b7fe640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f076806af60 0x7f076806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:32.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.873+0000 7f077b7fe640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f076806af60 0x7f076806d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f076c000f60 tx=0x7f076c009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:32.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:32.877+0000 7f0763fff640 1 -- 172.21.15.125:0/2005944688 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f077c151b20 con 0x7f077c06b410 2023-10-18T10:41:33.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.039+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 --> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f077c1521e0 con 0x7f076806af60 2023-10-18T10:41:33.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.047+0000 7f0763fff640 1 -- 172.21.15.125:0/2005944688 <== mgr.6299 v2:172.21.15.139:6832/1523403591 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283643 (secure 0 0 0) 0x7f077c1521e0 con 0x7f076806af60 2023-10-18T10:41:33.051 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:33.053 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:41:33.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f076806af60 msgr2=0x7f076806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f076806af60 0x7f076806d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f076c000f60 tx=0x7f076c009040 comp rx=0 tx=0).stop 2023-10-18T10:41:33.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 msgr2=0x7f077c0aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 0x7f077c0aa8c0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0770036040 tx=0x7f077002dad0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 shutdown_connections 2023-10-18T10:41:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f076806af60 0x7f076806d420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f077c15ccb0 0x7f077c0aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f077c06b410 0x7f077c0aa8c0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 --2- 172.21.15.125:0/2005944688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f077c06b030 0x7f077c1f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 >> 172.21.15.125:0/2005944688 conn(0x7f077c069930 msgr2=0x7f077c0ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:33.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 shutdown_connections 2023-10-18T10:41:33.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.050+0000 7f0781ff0640 1 -- 172.21.15.125:0/2005944688 wait complete. 2023-10-18T10:41:33.063 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:41:31.281902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":314,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":116,"acting":116,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":313984,"kb_used_data":11780,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754660736,"statfs":{"total":773094113280,"available":772772593664,"internally_reserved":0,"allocated":12062720,"data_stored":6601716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000876"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434717+0000","last_change":"2023-10-18T10:41:29.447539+0000","last_active":"2023-10-18T10:41:30.434717+0000","last_peered":"2023-10-18T10:41:30.434717+0000","last_clean":"2023-10-18T10:41:30.434717+0000","last_became_active":"2023-10-18T10:41:29.447288+0000","last_became_peered":"2023-10-18T10:41:29.447288+0000","last_unstale":"2023-10-18T10:41:30.434717+0000","last_undegraded":"2023-10-18T10:41:30.434717+0000","last_fullsized":"2023-10-18T10:41:30.434717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:30.647260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750857+0000","last_change":"2023-10-18T10:41:29.442252+0000","last_active":"2023-10-18T10:41:30.750857+0000","last_peered":"2023-10-18T10:41:30.750857+0000","last_clean":"2023-10-18T10:41:30.750857+0000","last_became_active":"2023-10-18T10:41:29.441955+0000","last_became_peered":"2023-10-18T10:41:29.441955+0000","last_unstale":"2023-10-18T10:41:30.750857+0000","last_undegraded":"2023-10-18T10:41:30.750857+0000","last_fullsized":"2023-10-18T10:41:30.750857+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:23.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434545+0000","last_change":"2023-10-18T10:41:29.452284+0000","last_active":"2023-10-18T10:41:30.434545+0000","last_peered":"2023-10-18T10:41:30.434545+0000","last_clean":"2023-10-18T10:41:30.434545+0000","last_became_active":"2023-10-18T10:41:29.452037+0000","last_became_peered":"2023-10-18T10:41:29.452037+0000","last_unstale":"2023-10-18T10:41:30.434545+0000","last_undegraded":"2023-10-18T10:41:30.434545+0000","last_fullsized":"2023-10-18T10:41:30.434545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.154689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750025+0000","last_change":"2023-10-18T10:41:29.453178+0000","last_active":"2023-10-18T10:41:30.750025+0000","last_peered":"2023-10-18T10:41:30.750025+0000","last_clean":"2023-10-18T10:41:30.750025+0000","last_became_active":"2023-10-18T10:41:29.452902+0000","last_became_peered":"2023-10-18T10:41:29.452902+0000","last_unstale":"2023-10-18T10:41:30.750025+0000","last_undegraded":"2023-10-18T10:41:30.750025+0000","last_fullsized":"2023-10-18T10:41:30.750025+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:23.594087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750972+0000","last_change":"2023-10-18T10:41:29.450362+0000","last_active":"2023-10-18T10:41:30.750972+0000","last_peered":"2023-10-18T10:41:30.750972+0000","last_clean":"2023-10-18T10:41:30.750972+0000","last_became_active":"2023-10-18T10:41:29.449972+0000","last_became_peered":"2023-10-18T10:41:29.449972+0000","last_unstale":"2023-10-18T10:41:30.750972+0000","last_undegraded":"2023-10-18T10:41:30.750972+0000","last_fullsized":"2023-10-18T10:41:30.750972+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:59.437842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749904+0000","last_change":"2023-10-18T10:41:29.449393+0000","last_active":"2023-10-18T10:41:30.749904+0000","last_peered":"2023-10-18T10:41:30.749904+0000","last_clean":"2023-10-18T10:41:30.749904+0000","last_became_active":"2023-10-18T10:41:29.449086+0000","last_became_peered":"2023-10-18T10:41:29.449086+0000","last_unstale":"2023-10-18T10:41:30.749904+0000","last_undegraded":"2023-10-18T10:41:30.749904+0000","last_fullsized":"2023-10-18T10:41:30.749904+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:41.061091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751632+0000","last_change":"2023-10-18T10:41:29.444304+0000","last_active":"2023-10-18T10:41:30.751632+0000","last_peered":"2023-10-18T10:41:30.751632+0000","last_clean":"2023-10-18T10:41:30.751632+0000","last_became_active":"2023-10-18T10:41:29.444057+0000","last_became_peered":"2023-10-18T10:41:29.444057+0000","last_unstale":"2023-10-18T10:41:30.751632+0000","last_undegraded":"2023-10-18T10:41:30.751632+0000","last_fullsized":"2023-10-18T10:41:30.751632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:23.209276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751364+0000","last_change":"2023-10-18T10:41:29.443320+0000","last_active":"2023-10-18T10:41:30.751364+0000","last_peered":"2023-10-18T10:41:30.751364+0000","last_clean":"2023-10-18T10:41:30.751364+0000","last_became_active":"2023-10-18T10:41:29.443029+0000","last_became_peered":"2023-10-18T10:41:29.443029+0000","last_unstale":"2023-10-18T10:41:30.751364+0000","last_undegraded":"2023-10-18T10:41:30.751364+0000","last_fullsized":"2023-10-18T10:41:30.751364+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:27.802284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750628+0000","last_change":"2023-10-18T10:41:29.448605+0000","last_active":"2023-10-18T10:41:30.750628+0000","last_peered":"2023-10-18T10:41:30.750628+0000","last_clean":"2023-10-18T10:41:30.750628+0000","last_became_active":"2023-10-18T10:41:29.448348+0000","last_became_peered":"2023-10-18T10:41:29.448348+0000","last_unstale":"2023-10-18T10:41:30.750628+0000","last_undegraded":"2023-10-18T10:41:30.750628+0000","last_fullsized":"2023-10-18T10:41:30.750628+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:42.695172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750043+0000","last_change":"2023-10-18T10:41:29.444407+0000","last_active":"2023-10-18T10:41:30.750043+0000","last_peered":"2023-10-18T10:41:30.750043+0000","last_clean":"2023-10-18T10:41:30.750043+0000","last_became_active":"2023-10-18T10:41:29.444169+0000","last_became_peered":"2023-10-18T10:41:29.444169+0000","last_unstale":"2023-10-18T10:41:30.750043+0000","last_undegraded":"2023-10-18T10:41:30.750043+0000","last_fullsized":"2023-10-18T10:41:30.750043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:13.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750781+0000","last_change":"2023-10-18T10:41:29.454015+0000","last_active":"2023-10-18T10:41:30.750781+0000","last_peered":"2023-10-18T10:41:30.750781+0000","last_clean":"2023-10-18T10:41:30.750781+0000","last_became_active":"2023-10-18T10:41:29.451538+0000","last_became_peered":"2023-10-18T10:41:29.451538+0000","last_unstale":"2023-10-18T10:41:30.750781+0000","last_undegraded":"2023-10-18T10:41:30.750781+0000","last_fullsized":"2023-10-18T10:41:30.750781+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:52.418194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751142+0000","last_change":"2023-10-18T10:41:29.449763+0000","last_active":"2023-10-18T10:41:30.751142+0000","last_peered":"2023-10-18T10:41:30.751142+0000","last_clean":"2023-10-18T10:41:30.751142+0000","last_became_active":"2023-10-18T10:41:29.449498+0000","last_became_peered":"2023-10-18T10:41:29.449498+0000","last_unstale":"2023-10-18T10:41:30.751142+0000","last_undegraded":"2023-10-18T10:41:30.751142+0000","last_fullsized":"2023-10-18T10:41:30.751142+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:30.606377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434672+0000","last_change":"2023-10-18T10:41:29.448915+0000","last_active":"2023-10-18T10:41:30.434672+0000","last_peered":"2023-10-18T10:41:30.434672+0000","last_clean":"2023-10-18T10:41:30.434672+0000","last_became_active":"2023-10-18T10:41:29.448650+0000","last_became_peered":"2023-10-18T10:41:29.448650+0000","last_unstale":"2023-10-18T10:41:30.434672+0000","last_undegraded":"2023-10-18T10:41:30.434672+0000","last_fullsized":"2023-10-18T10:41:30.434672+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:41.152281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750152+0000","last_change":"2023-10-18T10:41:29.455557+0000","last_active":"2023-10-18T10:41:30.750152+0000","last_peered":"2023-10-18T10:41:30.750152+0000","last_clean":"2023-10-18T10:41:30.750152+0000","last_became_active":"2023-10-18T10:41:29.455296+0000","last_became_peered":"2023-10-18T10:41:29.455296+0000","last_unstale":"2023-10-18T10:41:30.750152+0000","last_undegraded":"2023-10-18T10:41:30.750152+0000","last_fullsized":"2023-10-18T10:41:30.750152+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:51.749366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434493+0000","last_change":"2023-10-18T10:41:29.453702+0000","last_active":"2023-10-18T10:41:30.434493+0000","last_peered":"2023-10-18T10:41:30.434493+0000","last_clean":"2023-10-18T10:41:30.434493+0000","last_became_active":"2023-10-18T10:41:29.453445+0000","last_became_peered":"2023-10-18T10:41:29.453445+0000","last_unstale":"2023-10-18T10:41:30.434493+0000","last_undegraded":"2023-10-18T10:41:30.434493+0000","last_fullsized":"2023-10-18T10:41:30.434493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:28.572719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750696+0000","last_change":"2023-10-18T10:41:29.451267+0000","last_active":"2023-10-18T10:41:30.750696+0000","last_peered":"2023-10-18T10:41:30.750696+0000","last_clean":"2023-10-18T10:41:30.750696+0000","last_became_active":"2023-10-18T10:41:29.450963+0000","last_became_peered":"2023-10-18T10:41:29.450963+0000","last_unstale":"2023-10-18T10:41:30.750696+0000","last_undegraded":"2023-10-18T10:41:30.750696+0000","last_fullsized":"2023-10-18T10:41:30.750696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:01.499934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751548+0000","last_change":"2023-10-18T10:41:29.453816+0000","last_active":"2023-10-18T10:41:30.751548+0000","last_peered":"2023-10-18T10:41:30.751548+0000","last_clean":"2023-10-18T10:41:30.751548+0000","last_became_active":"2023-10-18T10:41:29.453581+0000","last_became_peered":"2023-10-18T10:41:29.453581+0000","last_unstale":"2023-10-18T10:41:30.751548+0000","last_undegraded":"2023-10-18T10:41:30.751548+0000","last_fullsized":"2023-10-18T10:41:30.751548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:18.669429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750180+0000","last_change":"2023-10-18T10:41:29.454500+0000","last_active":"2023-10-18T10:41:30.750180+0000","last_peered":"2023-10-18T10:41:30.750180+0000","last_clean":"2023-10-18T10:41:30.750180+0000","last_became_active":"2023-10-18T10:41:29.454231+0000","last_became_peered":"2023-10-18T10:41:29.454231+0000","last_unstale":"2023-10-18T10:41:30.750180+0000","last_undegraded":"2023-10-18T10:41:30.750180+0000","last_fullsized":"2023-10-18T10:41:30.750180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:14.224881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750335+0000","last_change":"2023-10-18T10:41:29.453993+0000","last_active":"2023-10-18T10:41:30.750335+0000","last_peered":"2023-10-18T10:41:30.750335+0000","last_clean":"2023-10-18T10:41:30.750335+0000","last_became_active":"2023-10-18T10:41:29.453659+0000","last_became_peered":"2023-10-18T10:41:29.453659+0000","last_unstale":"2023-10-18T10:41:30.750335+0000","last_undegraded":"2023-10-18T10:41:30.750335+0000","last_fullsized":"2023-10-18T10:41:30.750335+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:50.051564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750971+0000","last_change":"2023-10-18T10:41:29.445142+0000","last_active":"2023-10-18T10:41:30.750971+0000","last_peered":"2023-10-18T10:41:30.750971+0000","last_clean":"2023-10-18T10:41:30.750971+0000","last_became_active":"2023-10-18T10:41:29.444871+0000","last_became_peered":"2023-10-18T10:41:29.444871+0000","last_unstale":"2023-10-18T10:41:30.750971+0000","last_undegraded":"2023-10-18T10:41:30.750971+0000","last_fullsized":"2023-10-18T10:41:30.750971+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:31:23.805032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750791+0000","last_change":"2023-10-18T10:41:29.448568+0000","last_active":"2023-10-18T10:41:30.750791+0000","last_peered":"2023-10-18T10:41:30.750791+0000","last_clean":"2023-10-18T10:41:30.750791+0000","last_became_active":"2023-10-18T10:41:29.448318+0000","last_became_peered":"2023-10-18T10:41:29.448318+0000","last_unstale":"2023-10-18T10:41:30.750791+0000","last_undegraded":"2023-10-18T10:41:30.750791+0000","last_fullsized":"2023-10-18T10:41:30.750791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:29.589441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751302+0000","last_change":"2023-10-18T10:41:29.445971+0000","last_active":"2023-10-18T10:41:30.751302+0000","last_peered":"2023-10-18T10:41:30.751302+0000","last_clean":"2023-10-18T10:41:30.751302+0000","last_became_active":"2023-10-18T10:41:29.445723+0000","last_became_peered":"2023-10-18T10:41:29.445723+0000","last_unstale":"2023-10-18T10:41:30.751302+0000","last_undegraded":"2023-10-18T10:41:30.751302+0000","last_fullsized":"2023-10-18T10:41:30.751302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:35:45.397768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751225+0000","last_change":"2023-10-18T10:41:29.451021+0000","last_active":"2023-10-18T10:41:30.751225+0000","last_peered":"2023-10-18T10:41:30.751225+0000","last_clean":"2023-10-18T10:41:30.751225+0000","last_became_active":"2023-10-18T10:41:29.450779+0000","last_became_peered":"2023-10-18T10:41:29.450779+0000","last_unstale":"2023-10-18T10:41:30.751225+0000","last_undegraded":"2023-10-18T10:41:30.751225+0000","last_fullsized":"2023-10-18T10:41:30.751225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:36.701370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749949+0000","last_change":"2023-10-18T10:41:29.443511+0000","last_active":"2023-10-18T10:41:30.749949+0000","last_peered":"2023-10-18T10:41:30.749949+0000","last_clean":"2023-10-18T10:41:30.749949+0000","last_became_active":"2023-10-18T10:41:29.443232+0000","last_became_peered":"2023-10-18T10:41:29.443232+0000","last_unstale":"2023-10-18T10:41:30.749949+0000","last_undegraded":"2023-10-18T10:41:30.749949+0000","last_fullsized":"2023-10-18T10:41:30.749949+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:29.869647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750830+0000","last_change":"2023-10-18T10:41:28.439965+0000","last_active":"2023-10-18T10:41:30.750830+0000","last_peered":"2023-10-18T10:41:30.750830+0000","last_clean":"2023-10-18T10:41:30.750830+0000","last_became_active":"2023-10-18T10:41:28.439700+0000","last_became_peered":"2023-10-18T10:41:28.439700+0000","last_unstale":"2023-10-18T10:41:30.750830+0000","last_undegraded":"2023-10-18T10:41:30.750830+0000","last_fullsized":"2023-10-18T10:41:30.750830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:44.202065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750979+0000","last_change":"2023-10-18T10:41:29.450500+0000","last_active":"2023-10-18T10:41:30.750979+0000","last_peered":"2023-10-18T10:41:30.750979+0000","last_clean":"2023-10-18T10:41:30.750979+0000","last_became_active":"2023-10-18T10:41:29.450234+0000","last_became_peered":"2023-10-18T10:41:29.450234+0000","last_unstale":"2023-10-18T10:41:30.750979+0000","last_undegraded":"2023-10-18T10:41:30.750979+0000","last_fullsized":"2023-10-18T10:41:30.750979+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:08.866019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434561+0000","last_change":"2023-10-18T10:41:28.435875+0000","last_active":"2023-10-18T10:41:30.434561+0000","last_peered":"2023-10-18T10:41:30.434561+0000","last_clean":"2023-10-18T10:41:30.434561+0000","last_became_active":"2023-10-18T10:41:28.435623+0000","last_became_peered":"2023-10-18T10:41:28.435623+0000","last_unstale":"2023-10-18T10:41:30.434561+0000","last_undegraded":"2023-10-18T10:41:30.434561+0000","last_fullsized":"2023-10-18T10:41:30.434561+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:22.897591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750655+0000","last_change":"2023-10-18T10:41:29.451882+0000","last_active":"2023-10-18T10:41:30.750655+0000","last_peered":"2023-10-18T10:41:30.750655+0000","last_clean":"2023-10-18T10:41:30.750655+0000","last_became_active":"2023-10-18T10:41:29.451623+0000","last_became_peered":"2023-10-18T10:41:29.451623+0000","last_unstale":"2023-10-18T10:41:30.750655+0000","last_undegraded":"2023-10-18T10:41:30.750655+0000","last_fullsized":"2023-10-18T10:41:30.750655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:39.862682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750926+0000","last_change":"2023-10-18T10:41:28.436855+0000","last_active":"2023-10-18T10:41:30.750926+0000","last_peered":"2023-10-18T10:41:30.750926+0000","last_clean":"2023-10-18T10:41:30.750926+0000","last_became_active":"2023-10-18T10:41:28.436585+0000","last_became_peered":"2023-10-18T10:41:28.436585+0000","last_unstale":"2023-10-18T10:41:30.750926+0000","last_undegraded":"2023-10-18T10:41:30.750926+0000","last_fullsized":"2023-10-18T10:41:30.750926+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.155833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434722+0000","last_change":"2023-10-18T10:41:28.440969+0000","last_active":"2023-10-18T10:41:30.434722+0000","last_peered":"2023-10-18T10:41:30.434722+0000","last_clean":"2023-10-18T10:41:30.434722+0000","last_became_active":"2023-10-18T10:41:28.440739+0000","last_became_peered":"2023-10-18T10:41:28.440739+0000","last_unstale":"2023-10-18T10:41:30.434722+0000","last_undegraded":"2023-10-18T10:41:30.434722+0000","last_fullsized":"2023-10-18T10:41:30.434722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.620461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434696+0000","last_change":"2023-10-18T10:41:29.456187+0000","last_active":"2023-10-18T10:41:30.434696+0000","last_peered":"2023-10-18T10:41:30.434696+0000","last_clean":"2023-10-18T10:41:30.434696+0000","last_became_active":"2023-10-18T10:41:29.455945+0000","last_became_peered":"2023-10-18T10:41:29.455945+0000","last_unstale":"2023-10-18T10:41:30.434696+0000","last_undegraded":"2023-10-18T10:41:30.434696+0000","last_fullsized":"2023-10-18T10:41:30.434696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:40.835626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750493+0000","last_change":"2023-10-18T10:41:29.449408+0000","last_active":"2023-10-18T10:41:30.750493+0000","last_peered":"2023-10-18T10:41:30.750493+0000","last_clean":"2023-10-18T10:41:30.750493+0000","last_became_active":"2023-10-18T10:41:29.449161+0000","last_became_peered":"2023-10-18T10:41:29.449161+0000","last_unstale":"2023-10-18T10:41:30.750493+0000","last_undegraded":"2023-10-18T10:41:30.750493+0000","last_fullsized":"2023-10-18T10:41:30.750493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:41.742410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"111'6","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751094+0000","last_change":"2023-10-18T10:41:28.427403+0000","last_active":"2023-10-18T10:41:30.751094+0000","last_peered":"2023-10-18T10:41:30.751094+0000","last_clean":"2023-10-18T10:41:30.751094+0000","last_became_active":"2023-10-18T10:41:28.427119+0000","last_became_peered":"2023-10-18T10:41:28.427119+0000","last_unstale":"2023-10-18T10:41:30.751094+0000","last_undegraded":"2023-10-18T10:41:30.751094+0000","last_fullsized":"2023-10-18T10:41:30.751094+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:11.156072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750239+0000","last_change":"2023-10-18T10:41:29.448792+0000","last_active":"2023-10-18T10:41:30.750239+0000","last_peered":"2023-10-18T10:41:30.750239+0000","last_clean":"2023-10-18T10:41:30.750239+0000","last_became_active":"2023-10-18T10:41:29.448536+0000","last_became_peered":"2023-10-18T10:41:29.448536+0000","last_unstale":"2023-10-18T10:41:30.750239+0000","last_undegraded":"2023-10-18T10:41:30.750239+0000","last_fullsized":"2023-10-18T10:41:30.750239+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:14.300602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751060+0000","last_change":"2023-10-18T10:41:28.433856+0000","last_active":"2023-10-18T10:41:30.751060+0000","last_peered":"2023-10-18T10:41:30.751060+0000","last_clean":"2023-10-18T10:41:30.751060+0000","last_became_active":"2023-10-18T10:41:28.433584+0000","last_became_peered":"2023-10-18T10:41:28.433584+0000","last_unstale":"2023-10-18T10:41:30.751060+0000","last_undegraded":"2023-10-18T10:41:30.751060+0000","last_fullsized":"2023-10-18T10:41:30.751060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:06:07.806734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750178+0000","last_change":"2023-10-18T10:41:29.439852+0000","last_active":"2023-10-18T10:41:30.750178+0000","last_peered":"2023-10-18T10:41:30.750178+0000","last_clean":"2023-10-18T10:41:30.750178+0000","last_became_active":"2023-10-18T10:41:29.439612+0000","last_became_peered":"2023-10-18T10:41:29.439612+0000","last_unstale":"2023-10-18T10:41:30.750178+0000","last_undegraded":"2023-10-18T10:41:30.750178+0000","last_fullsized":"2023-10-18T10:41:30.750178+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:10.074146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750194+0000","last_change":"2023-10-18T10:41:28.431914+0000","last_active":"2023-10-18T10:41:30.750194+0000","last_peered":"2023-10-18T10:41:30.750194+0000","last_clean":"2023-10-18T10:41:30.750194+0000","last_became_active":"2023-10-18T10:41:28.431664+0000","last_became_peered":"2023-10-18T10:41:28.431664+0000","last_unstale":"2023-10-18T10:41:30.750194+0000","last_undegraded":"2023-10-18T10:41:30.750194+0000","last_fullsized":"2023-10-18T10:41:30.750194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:41.649804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751367+0000","last_change":"2023-10-18T10:41:29.453041+0000","last_active":"2023-10-18T10:41:30.751367+0000","last_peered":"2023-10-18T10:41:30.751367+0000","last_clean":"2023-10-18T10:41:30.751367+0000","last_became_active":"2023-10-18T10:41:29.452788+0000","last_became_peered":"2023-10-18T10:41:29.452788+0000","last_unstale":"2023-10-18T10:41:30.751367+0000","last_undegraded":"2023-10-18T10:41:30.751367+0000","last_fullsized":"2023-10-18T10:41:30.751367+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:28.740818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"111'8","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750045+0000","last_change":"2023-10-18T10:41:28.440009+0000","last_active":"2023-10-18T10:41:30.750045+0000","last_peered":"2023-10-18T10:41:30.750045+0000","last_clean":"2023-10-18T10:41:30.750045+0000","last_became_active":"2023-10-18T10:41:28.439742+0000","last_became_peered":"2023-10-18T10:41:28.439742+0000","last_unstale":"2023-10-18T10:41:30.750045+0000","last_undegraded":"2023-10-18T10:41:30.750045+0000","last_fullsized":"2023-10-18T10:41:30.750045+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:42.072352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751005+0000","last_change":"2023-10-18T10:41:29.448022+0000","last_active":"2023-10-18T10:41:30.751005+0000","last_peered":"2023-10-18T10:41:30.751005+0000","last_clean":"2023-10-18T10:41:30.751005+0000","last_became_active":"2023-10-18T10:41:29.447771+0000","last_became_peered":"2023-10-18T10:41:29.447771+0000","last_unstale":"2023-10-18T10:41:30.751005+0000","last_undegraded":"2023-10-18T10:41:30.751005+0000","last_fullsized":"2023-10-18T10:41:30.751005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:13.373299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751109+0000","last_change":"2023-10-18T10:41:29.450342+0000","last_active":"2023-10-18T10:41:30.751109+0000","last_peered":"2023-10-18T10:41:30.751109+0000","last_clean":"2023-10-18T10:41:30.751109+0000","last_became_active":"2023-10-18T10:41:29.450078+0000","last_became_peered":"2023-10-18T10:41:29.450078+0000","last_unstale":"2023-10-18T10:41:30.751109+0000","last_undegraded":"2023-10-18T10:41:30.751109+0000","last_fullsized":"2023-10-18T10:41:30.751109+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:32.976432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750276+0000","last_change":"2023-10-18T10:41:28.438968+0000","last_active":"2023-10-18T10:41:30.750276+0000","last_peered":"2023-10-18T10:41:30.750276+0000","last_clean":"2023-10-18T10:41:30.750276+0000","last_became_active":"2023-10-18T10:41:28.438628+0000","last_became_peered":"2023-10-18T10:41:28.438628+0000","last_unstale":"2023-10-18T10:41:30.750276+0000","last_undegraded":"2023-10-18T10:41:30.750276+0000","last_fullsized":"2023-10-18T10:41:30.750276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:51.950983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750427+0000","last_change":"2023-10-18T10:41:29.442931+0000","last_active":"2023-10-18T10:41:30.750427+0000","last_peered":"2023-10-18T10:41:30.750427+0000","last_clean":"2023-10-18T10:41:30.750427+0000","last_became_active":"2023-10-18T10:41:29.442573+0000","last_became_peered":"2023-10-18T10:41:29.442573+0000","last_unstale":"2023-10-18T10:41:30.750427+0000","last_undegraded":"2023-10-18T10:41:30.750427+0000","last_fullsized":"2023-10-18T10:41:30.750427+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:36.667760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750311+0000","last_change":"2023-10-18T10:41:28.437156+0000","last_active":"2023-10-18T10:41:30.750311+0000","last_peered":"2023-10-18T10:41:30.750311+0000","last_clean":"2023-10-18T10:41:30.750311+0000","last_became_active":"2023-10-18T10:41:28.434657+0000","last_became_peered":"2023-10-18T10:41:28.434657+0000","last_unstale":"2023-10-18T10:41:30.750311+0000","last_undegraded":"2023-10-18T10:41:30.750311+0000","last_fullsized":"2023-10-18T10:41:30.750311+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:38:04.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750603+0000","last_change":"2023-10-18T10:41:29.452874+0000","last_active":"2023-10-18T10:41:30.750603+0000","last_peered":"2023-10-18T10:41:30.750603+0000","last_clean":"2023-10-18T10:41:30.750603+0000","last_became_active":"2023-10-18T10:41:29.452561+0000","last_became_peered":"2023-10-18T10:41:29.452561+0000","last_unstale":"2023-10-18T10:41:30.750603+0000","last_undegraded":"2023-10-18T10:41:30.750603+0000","last_fullsized":"2023-10-18T10:41:30.750603+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:26.000624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750422+0000","last_change":"2023-10-18T10:41:28.435576+0000","last_active":"2023-10-18T10:41:30.750422+0000","last_peered":"2023-10-18T10:41:30.750422+0000","last_clean":"2023-10-18T10:41:30.750422+0000","last_became_active":"2023-10-18T10:41:28.435286+0000","last_became_peered":"2023-10-18T10:41:28.435286+0000","last_unstale":"2023-10-18T10:41:30.750422+0000","last_undegraded":"2023-10-18T10:41:30.750422+0000","last_fullsized":"2023-10-18T10:41:30.750422+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:29.719996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750887+0000","last_change":"2023-10-18T10:41:29.446250+0000","last_active":"2023-10-18T10:41:30.750887+0000","last_peered":"2023-10-18T10:41:30.750887+0000","last_clean":"2023-10-18T10:41:30.750887+0000","last_became_active":"2023-10-18T10:41:29.445881+0000","last_became_peered":"2023-10-18T10:41:29.445881+0000","last_unstale":"2023-10-18T10:41:30.750887+0000","last_undegraded":"2023-10-18T10:41:30.750887+0000","last_fullsized":"2023-10-18T10:41:30.750887+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:28.061711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750340+0000","last_change":"2023-10-18T10:41:28.438325+0000","last_active":"2023-10-18T10:41:30.750340+0000","last_peered":"2023-10-18T10:41:30.750340+0000","last_clean":"2023-10-18T10:41:30.750340+0000","last_became_active":"2023-10-18T10:41:28.438056+0000","last_became_peered":"2023-10-18T10:41:28.438056+0000","last_unstale":"2023-10-18T10:41:30.750340+0000","last_undegraded":"2023-10-18T10:41:30.750340+0000","last_fullsized":"2023-10-18T10:41:30.750340+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:42.738454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751473+0000","last_change":"2023-10-18T10:41:29.449610+0000","last_active":"2023-10-18T10:41:30.751473+0000","last_peered":"2023-10-18T10:41:30.751473+0000","last_clean":"2023-10-18T10:41:30.751473+0000","last_became_active":"2023-10-18T10:41:29.449354+0000","last_became_peered":"2023-10-18T10:41:29.449354+0000","last_unstale":"2023-10-18T10:41:30.751473+0000","last_undegraded":"2023-10-18T10:41:30.751473+0000","last_fullsized":"2023-10-18T10:41:30.751473+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:01.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751349+0000","last_change":"2023-10-18T10:41:29.447215+0000","last_active":"2023-10-18T10:41:30.751349+0000","last_peered":"2023-10-18T10:41:30.751349+0000","last_clean":"2023-10-18T10:41:30.751349+0000","last_became_active":"2023-10-18T10:41:29.446983+0000","last_became_peered":"2023-10-18T10:41:29.446983+0000","last_unstale":"2023-10-18T10:41:30.751349+0000","last_undegraded":"2023-10-18T10:41:30.751349+0000","last_fullsized":"2023-10-18T10:41:30.751349+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750837+0000","last_change":"2023-10-18T10:41:29.448872+0000","last_active":"2023-10-18T10:41:30.750837+0000","last_peered":"2023-10-18T10:41:30.750837+0000","last_clean":"2023-10-18T10:41:30.750837+0000","last_became_active":"2023-10-18T10:41:29.448574+0000","last_became_peered":"2023-10-18T10:41:29.448574+0000","last_unstale":"2023-10-18T10:41:30.750837+0000","last_undegraded":"2023-10-18T10:41:30.750837+0000","last_fullsized":"2023-10-18T10:41:30.750837+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:40.934129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749895+0000","last_change":"2023-10-18T10:41:29.442254+0000","last_active":"2023-10-18T10:41:30.749895+0000","last_peered":"2023-10-18T10:41:30.749895+0000","last_clean":"2023-10-18T10:41:30.749895+0000","last_became_active":"2023-10-18T10:41:29.441995+0000","last_became_peered":"2023-10-18T10:41:29.441995+0000","last_unstale":"2023-10-18T10:41:30.749895+0000","last_undegraded":"2023-10-18T10:41:30.749895+0000","last_fullsized":"2023-10-18T10:41:30.749895+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:13.960239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751093+0000","last_change":"2023-10-18T10:41:28.434524+0000","last_active":"2023-10-18T10:41:30.751093+0000","last_peered":"2023-10-18T10:41:30.751093+0000","last_clean":"2023-10-18T10:41:30.751093+0000","last_became_active":"2023-10-18T10:41:28.434260+0000","last_became_peered":"2023-10-18T10:41:28.434260+0000","last_unstale":"2023-10-18T10:41:30.751093+0000","last_undegraded":"2023-10-18T10:41:30.751093+0000","last_fullsized":"2023-10-18T10:41:30.751093+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:26:18.323130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434509+0000","last_change":"2023-10-18T10:41:29.455708+0000","last_active":"2023-10-18T10:41:30.434509+0000","last_peered":"2023-10-18T10:41:30.434509+0000","last_clean":"2023-10-18T10:41:30.434509+0000","last_became_active":"2023-10-18T10:41:29.455459+0000","last_became_peered":"2023-10-18T10:41:29.455459+0000","last_unstale":"2023-10-18T10:41:30.434509+0000","last_undegraded":"2023-10-18T10:41:30.434509+0000","last_fullsized":"2023-10-18T10:41:30.434509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:01.952901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750593+0000","last_change":"2023-10-18T10:41:28.439218+0000","last_active":"2023-10-18T10:41:30.750593+0000","last_peered":"2023-10-18T10:41:30.750593+0000","last_clean":"2023-10-18T10:41:30.750593+0000","last_became_active":"2023-10-18T10:41:28.438612+0000","last_became_peered":"2023-10-18T10:41:28.438612+0000","last_unstale":"2023-10-18T10:41:30.750593+0000","last_undegraded":"2023-10-18T10:41:30.750593+0000","last_fullsized":"2023-10-18T10:41:30.750593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:22.330220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751017+0000","last_change":"2023-10-18T10:41:29.446112+0000","last_active":"2023-10-18T10:41:30.751017+0000","last_peered":"2023-10-18T10:41:30.751017+0000","last_clean":"2023-10-18T10:41:30.751017+0000","last_became_active":"2023-10-18T10:41:29.445569+0000","last_became_peered":"2023-10-18T10:41:29.445569+0000","last_unstale":"2023-10-18T10:41:30.751017+0000","last_undegraded":"2023-10-18T10:41:30.751017+0000","last_fullsized":"2023-10-18T10:41:30.751017+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:23.963420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:27.404871+0000","last_change":"2023-10-18T10:41:27.404871+0000","last_active":"2023-10-18T10:41:27.404871+0000","last_peered":"2023-10-18T10:41:27.404871+0000","last_clean":"2023-10-18T10:41:27.404871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:27.404871+0000","last_undegraded":"2023-10-18T10:41:27.404871+0000","last_fullsized":"2023-10-18T10:41:27.404871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:41:28.414311+0000","last_change":"2023-10-18T10:41:28.414311+0000","last_active":"2023-10-18T10:41:28.414311+0000","last_peered":"2023-10-18T10:41:28.414311+0000","last_clean":"2023-10-18T10:41:28.414311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:41:28.414311+0000","last_undegraded":"2023-10-18T10:41:28.414311+0000","last_fullsized":"2023-10-18T10:41:28.414311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750265+0000","last_change":"2023-10-18T10:41:28.440086+0000","last_active":"2023-10-18T10:41:30.750265+0000","last_peered":"2023-10-18T10:41:30.750265+0000","last_clean":"2023-10-18T10:41:30.750265+0000","last_became_active":"2023-10-18T10:41:28.439814+0000","last_became_peered":"2023-10-18T10:41:28.439814+0000","last_unstale":"2023-10-18T10:41:30.750265+0000","last_undegraded":"2023-10-18T10:41:30.750265+0000","last_fullsized":"2023-10-18T10:41:30.750265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:19.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"101'192","reported_seq":513,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T10:41:24.714727+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:41:24.714727+0000","last_peered":"2023-10-18T10:41:24.714727+0000","last_clean":"2023-10-18T10:41:24.714727+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:41:24.714727+0000","last_undegraded":"2023-10-18T10:41:24.714727+0000","last_fullsized":"2023-10-18T10:41:24.714727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":30,"acting":30,"num_store_stats":5},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":1912832,"data_stored":1248389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.876,"5min":0.876,"15min":0.88300000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68200000000000005,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73199999999999998,"5min":0.84199999999999997,"15min":0.84899999999999998},"last":0.51900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.627}]},{"osd":4,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.495,"15min":0.48799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.875,"15min":0.875},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78400000000000003,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.69999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32492,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1331200,"data_stored":664028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.34499999999999997,"15min":0.28699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.78500000000000003,"15min":0.82599999999999996},"last":0.438}]},{"osd":1,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.79500000000000004,"15min":0.83399999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.80900000000000005,"15min":0.82699999999999996},"last":0.621}]},{"osd":2,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55800000000000005,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65000000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.64900000000000002,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.66900000000000004,"5min":0.75600000000000001,"15min":0.84399999999999997},"last":0.69699999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.441,"15min":0.44},"min":{"1min":0.30399999999999999,"5min":0.29199999999999998,"15min":0.24199999999999999},"max":{"1min":0.76500000000000001,"5min":0.872,"15min":0.872},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45600000000000002,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65500000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48199999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82499999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.68300000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.749},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72199999999999998,"5min":0.82199999999999995,"15min":0.87},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57199999999999995,"15min":0.58399999999999996},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69599999999999995,"5min":0.82499999999999996,"15min":1.016},"last":0.434},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.70099999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.628},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":1.0620000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.80100000000000005,"15min":1.262},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73099999999999998,"5min":0.96099999999999997,"15min":1.224},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":0.871,"15min":1.159},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575049,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":664028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.877,"15min":0.877},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.76900000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.63100000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.70899999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.79600000000000004},"last":0.67500000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.32600000000000001,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.82999999999999996,"5min":1.446,"15min":1.446},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.80300000000000005,"5min":0.81599999999999995,"15min":0.89700000000000002},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.85199999999999998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.88600000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35199999999999998,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.183,"15min":0.183},"max":{"1min":0.46100000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.254,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.50700000000000001,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.376,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.55000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41699999999999998,"15min":0.42999999999999999},"min":{"1min":0.29299999999999998,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47699999999999998,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.308,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.879,"15min":0.94699999999999995},"last":0.622}]},{"osd":5,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":1.002,"15min":1.002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.37},"max":{"1min":0.85499999999999998,"5min":1.018,"15min":1.018},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.63500000000000001},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.873,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.68899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":690475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.376,"15min":0.371},"min":{"1min":0.22600000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.49099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.47299999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.39200000000000002,"15min":0.40300000000000002},"min":{"1min":0.27300000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55300000000000005,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.437},{"interface":"front","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.40500000000000003},"min":{"1min":0.27400000000000002,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.24299999999999999},"max":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.92600000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.624,"15min":0.64100000000000001},"min":{"1min":0.432,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":1.458},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79300000000000004,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":1.028},"last":0.80900000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81999999999999995,"5min":0.879,"15min":0.92500000000000004},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1372160,"data_stored":683925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.432,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.38800000000000001,"15min":0.38900000000000001},"min":{"1min":0.23799999999999999,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.51100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.38800000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.40200000000000002,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":0.627,"15min":0.64800000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26000000000000001},"max":{"1min":0.495,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.26800000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52300000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.73399999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.81799999999999995,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.624,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.374},"max":{"1min":0.871,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.371},"max":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575049,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1998848,"data_stored":1283161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.35999999999999999,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.50600000000000001,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.308},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.377,"15min":0.38600000000000001},"min":{"1min":0.26000000000000001,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.48899999999999999,"5min":0.62,"15min":0.76900000000000002},"last":0.34100000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.187},"max":{"1min":0.46700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39200000000000002,"15min":0.41499999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46200000000000002,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.28799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38300000000000001,"15min":0.39000000000000001},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.41799999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.307,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51300000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0649999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34699999999999998},"max":{"1min":0.98199999999999998,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":1.026,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.439}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.339,"15min":1.339},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.628,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:41:33.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2023-10-18T10:41:33.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2023-10-18T10:41:33.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2023-10-18T10:41:33.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2023-10-18T10:41:33.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2023-10-18T10:41:33.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2023-10-18T10:41:33.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2023-10-18T10:41:33.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2023-10-18T10:41:33.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2023-10-18T10:41:33.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2023-10-18T10:41:33.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2023-10-18T10:41:33.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2023-10-18T10:41:33.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2023-10-18T10:41:33.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-10-18T10:41:33.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2023-10-18T10:41:33.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2023-10-18T10:41:33.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-10-18T10:41:33.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-10-18T10:41:33.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2023-10-18T10:41:33.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2023-10-18T10:41:33.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2023-10-18T10:41:33.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2023-10-18T10:41:33.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2023-10-18T10:41:33.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2023-10-18T10:41:33.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2023-10-18T10:41:33.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-10-18T10:41:33.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2023-10-18T10:41:33.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-10-18T10:41:33.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2023-10-18T10:41:33.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-10-18T10:41:33.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2023-10-18T10:41:33.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2023-10-18T10:41:33.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2023-10-18T10:41:33.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-10-18T10:41:33.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2023-10-18T10:41:33.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2023-10-18T10:41:33.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2023-10-18T10:41:33.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2023-10-18T10:41:33.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:27.404871+0000', 'last_change': '2023-10-18T10:41:27.404871+0000', 'last_active': '2023-10-18T10:41:27.404871+0000', 'last_peered': '2023-10-18T10:41:27.404871+0000', 'last_clean': '2023-10-18T10:41:27.404871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:27.404871+0000', 'last_undegraded': '2023-10-18T10:41:27.404871+0000', 'last_fullsized': '2023-10-18T10:41:27.404871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:27.404871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2023-10-18T10:41:33.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:41:28.414311+0000', 'last_change': '2023-10-18T10:41:28.414311+0000', 'last_active': '2023-10-18T10:41:28.414311+0000', 'last_peered': '2023-10-18T10:41:28.414311+0000', 'last_clean': '2023-10-18T10:41:28.414311+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:41:28.414311+0000', 'last_undegraded': '2023-10-18T10:41:28.414311+0000', 'last_fullsized': '2023-10-18T10:41:28.414311+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'last_clean_scrub_stamp': '2023-10-18T10:41:28.414311+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:41:33.105 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T10:41:33.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.206+0000 7fa61dee5640 1 -- 172.21.15.125:0/3778120718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 msgr2=0x7fa5f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.206+0000 7fa61dee5640 1 --2- 172.21.15.125:0/3778120718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 0x7fa5f8003d90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa608009b50 tx=0x7fa5f80057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.206+0000 7fa617fff640 1 -- 172.21.15.125:0/3778120718 reap_dead start 2023-10-18T10:41:33.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.206+0000 7fa61dee5640 1 -- 172.21.15.125:0/3778120718 shutdown_connections 2023-10-18T10:41:33.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.206+0000 7fa61dee5640 1 -- 172.21.15.125:0/3778120718 >> 172.21.15.125:0/3778120718 conn(0x7fa618069930 msgr2=0x7fa618069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:33.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.206+0000 7fa61dee5640 1 -- 172.21.15.125:0/3778120718 shutdown_connections 2023-10-18T10:41:33.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.206+0000 7fa61dee5640 1 -- 172.21.15.125:0/3778120718 wait complete. 2023-10-18T10:41:33.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.207+0000 7fa61dee5640 1 Processor -- start 2023-10-18T10:41:33.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.207+0000 7fa61dee5640 1 -- start start 2023-10-18T10:41:33.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa61dee5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 0x7fa6181565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:33.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa61dee5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa618154bd0 0x7fa618143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:33.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa61dee5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6180abac0 0x7fa6181470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:33.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa61dee5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618156b10 con 0x7fa5f8003970 2023-10-18T10:41:33.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa61dee5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618156d20 con 0x7fa618154bd0 2023-10-18T10:41:33.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa61dee5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa618157000 con 0x7fa6180abac0 2023-10-18T10:41:33.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 0x7fa6181565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:33.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 0x7fa6181565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50934/0 (socket says 172.21.15.125:50934) 2023-10-18T10:41:33.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 -- 172.21.15.125:0/1141782997 learned_addr learned my addr 172.21.15.125:0/1141782997 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:33.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 -- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6180abac0 msgr2=0x7fa6181470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6180abac0 0x7fa6181470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 -- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa618154bd0 msgr2=0x7fa618143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa618154bd0 0x7fa618143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 -- 172.21.15.125:0/1141782997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6181572e0 con 0x7fa5f8003970 2023-10-18T10:41:33.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.208+0000 7fa6177fe640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 0x7fa6181565d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa608009b50 tx=0x7fa608033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:33.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.209+0000 7fa614ff9640 1 -- 172.21.15.125:0/1141782997 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608004230 con 0x7fa5f8003970 2023-10-18T10:41:33.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.209+0000 7fa614ff9640 1 -- 172.21.15.125:0/1141782997 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa608004390 con 0x7fa5f8003970 2023-10-18T10:41:33.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.209+0000 7fa614ff9640 1 -- 172.21.15.125:0/1141782997 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608003850 con 0x7fa5f8003970 2023-10-18T10:41:33.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.209+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa618152120 con 0x7fa5f8003970 2023-10-18T10:41:33.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.209+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa618144530 con 0x7fa5f8003970 2023-10-18T10:41:33.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.210+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6180690e0 con 0x7fa5f8003970 2023-10-18T10:41:33.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.210+0000 7fa614ff9640 1 -- 172.21.15.125:0/1141782997 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa608003bc0 con 0x7fa5f8003970 2023-10-18T10:41:33.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.210+0000 7fa614ff9640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa5fc06af60 0x7fa5fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:33.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.210+0000 7fa614ff9640 1 -- 172.21.15.125:0/1141782997 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa6080ac870 con 0x7fa5f8003970 2023-10-18T10:41:33.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.211+0000 7fa60effd640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa5fc06af60 0x7fa5fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:33.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.211+0000 7fa60effd640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa5fc06af60 0x7fa5fc06d420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6180ae2f0 tx=0x7fa600005b30 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:33.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.214+0000 7fa614ff9640 1 -- 172.21.15.125:0/1141782997 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa608077650 con 0x7fa5f8003970 2023-10-18T10:41:33.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.444+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa61806f700 con 0x7fa5f8003970 2023-10-18T10:41:33.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.444+0000 7fa614ff9640 1 -- 172.21.15.125:0/1141782997 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1089 (secure 0 0 0) 0x7fa60804b080 con 0x7fa5f8003970 2023-10-18T10:41:33.449 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:33.450 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1015,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":58},{"state_name":"unknown","count":39}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":591980,"bytes_used":321519616,"bytes_avail":772772593664,"bytes_total":773094113280,"unknown_pgs_ratio":0.40206184983253479,"write_bytes_sec":767,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":41,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":8076}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-10-18T10:39:10.732193+0000","services":{}},"progress_events":{}} 2023-10-18T10:41:33.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa5fc06af60 msgr2=0x7fa5fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa5fc06af60 0x7fa5fc06d420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6180ae2f0 tx=0x7fa600005b30 comp rx=0 tx=0).stop 2023-10-18T10:41:33.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 msgr2=0x7fa6181565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 0x7fa6181565d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa608009b50 tx=0x7fa608033c30 comp rx=0 tx=0).stop 2023-10-18T10:41:33.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 shutdown_connections 2023-10-18T10:41:33.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fa5fc06af60 0x7fa5fc06d420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa618154bd0 0x7fa618143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6180abac0 0x7fa6181470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 --2- 172.21.15.125:0/1141782997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f8003970 0x7fa6181565d0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 >> 172.21.15.125:0/1141782997 conn(0x7fa618069930 msgr2=0x7fa61806bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:33.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 shutdown_connections 2023-10-18T10:41:33.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.446+0000 7fa61dee5640 1 -- 172.21.15.125:0/1141782997 wait complete. 2023-10-18T10:41:33.459 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T10:41:33.460 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:41:33.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.560+0000 7fe25eeaf640 1 -- 172.21.15.125:0/108316424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe25815d400 msgr2=0x7fe2580cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.560+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/108316424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe25815d400 0x7fe2580cdfb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe250005fd0 tx=0x7fe25002c050 comp rx=0 tx=0).stop 2023-10-18T10:41:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.560+0000 7fe25eeaf640 1 -- 172.21.15.125:0/108316424 shutdown_connections 2023-10-18T10:41:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.560+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/108316424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe25815d400 0x7fe2580cdfb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.560+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/108316424 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe258159060 0x7fe25806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.560+0000 7fe25eeaf640 1 -- 172.21.15.125:0/108316424 >> 172.21.15.125:0/108316424 conn(0x7fe2580b0670 msgr2=0x7fe2580b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:33.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.560+0000 7fe25eeaf640 1 -- 172.21.15.125:0/108316424 shutdown_connections 2023-10-18T10:41:33.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.560+0000 7fe25eeaf640 1 -- 172.21.15.125:0/108316424 wait complete. 2023-10-18T10:41:33.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.561+0000 7fe25eeaf640 1 Processor -- start 2023-10-18T10:41:33.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.561+0000 7fe25eeaf640 1 -- start start 2023-10-18T10:41:33.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25eeaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2580cdb30 0x7fe25815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:33.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25eeaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe258159060 0x7fe2580c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:33.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25eeaf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe25815d400 0x7fe2580c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:33.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25eeaf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25815b580 con 0x7fe25815d400 2023-10-18T10:41:33.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25eeaf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25815b860 con 0x7fe2580cdb30 2023-10-18T10:41:33.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25eeaf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25815bb40 con 0x7fe258159060 2023-10-18T10:41:33.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25d44c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe25815d400 0x7fe2580c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:33.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25d44c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe25815d400 0x7fe2580c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50946/0 (socket says 172.21.15.125:50946) 2023-10-18T10:41:33.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.562+0000 7fe25d44c640 1 -- 172.21.15.125:0/3506120620 learned_addr learned my addr 172.21.15.125:0/3506120620 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:33.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25cc4b640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2580cdb30 0x7fe25815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:33.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe257fff640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe258159060 0x7fe2580c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:33.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25d44c640 1 -- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe258159060 msgr2=0x7fe2580c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25d44c640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe258159060 0x7fe2580c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25d44c640 1 -- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2580cdb30 msgr2=0x7fe25815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25d44c640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2580cdb30 0x7fe25815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25d44c640 1 -- 172.21.15.125:0/3506120620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe248009560 con 0x7fe25815d400 2023-10-18T10:41:33.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25d44c640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe25815d400 0x7fe2580c4c20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe250009cd0 tx=0x7fe2500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:33.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe2557fa640 1 -- 172.21.15.125:0/3506120620 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25003e460 con 0x7fe25815d400 2023-10-18T10:41:33.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe250009840 con 0x7fe25815d400 2023-10-18T10:41:33.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe2557fa640 1 -- 172.21.15.125:0/3506120620 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe25003e6b0 con 0x7fe25815d400 2023-10-18T10:41:33.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe2557fa640 1 -- 172.21.15.125:0/3506120620 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25003ea80 con 0x7fe25815d400 2023-10-18T10:41:33.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.563+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2580c21b0 con 0x7fe25815d400 2023-10-18T10:41:33.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.564+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25815c000 con 0x7fe25815d400 2023-10-18T10:41:33.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.565+0000 7fe2557fa640 1 -- 172.21.15.125:0/3506120620 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe25003ebe0 con 0x7fe25815d400 2023-10-18T10:41:33.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.565+0000 7fe2557fa640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe24406af60 0x7fe24406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:33.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.565+0000 7fe2557fa640 1 -- 172.21.15.125:0/3506120620 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe2500ac890 con 0x7fe25815d400 2023-10-18T10:41:33.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.565+0000 7fe25cc4b640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe24406af60 0x7fe24406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:33.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.566+0000 7fe25cc4b640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe24406af60 0x7fe24406d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe248009530 tx=0x7fe248009260 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:33.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.569+0000 7fe2557fa640 1 -- 172.21.15.125:0/3506120620 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe250077720 con 0x7fe25815d400 2023-10-18T10:41:33.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.733+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 --> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe25815b9d0 con 0x7fe24406af60 2023-10-18T10:41:33.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.741+0000 7fe2557fa640 1 -- 172.21.15.125:0/3506120620 <== mgr.6299 v2:172.21.15.139:6832/1523403591 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289698 (secure 0 0 0) 0x7fe25815b9d0 con 0x7fe24406af60 2023-10-18T10:41:33.746 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:33.747 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:41:33.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe24406af60 msgr2=0x7fe24406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe24406af60 0x7fe24406d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe248009530 tx=0x7fe248009260 comp rx=0 tx=0).stop 2023-10-18T10:41:33.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe25815d400 msgr2=0x7fe2580c4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe25815d400 0x7fe2580c4c20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe250009cd0 tx=0x7fe2500042f0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 shutdown_connections 2023-10-18T10:41:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe24406af60 0x7fe24406d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe25815d400 0x7fe2580c4c20 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe258159060 0x7fe2580c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 --2- 172.21.15.125:0/3506120620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2580cdb30 0x7fe25815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:33.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 >> 172.21.15.125:0/3506120620 conn(0x7fe2580b0670 msgr2=0x7fe2580aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:33.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 shutdown_connections 2023-10-18T10:41:33.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:33.744+0000 7fe25eeaf640 1 -- 172.21.15.125:0/3506120620 wait complete. 2023-10-18T10:41:33.756 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:41:33.282422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":339,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314144,"kb_used_data":11948,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754660576,"statfs":{"total":773094113280,"available":772772429824,"internally_reserved":0,"allocated":12234752,"data_stored":6676256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.866409"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434717+0000","last_change":"2023-10-18T10:41:29.447539+0000","last_active":"2023-10-18T10:41:30.434717+0000","last_peered":"2023-10-18T10:41:30.434717+0000","last_clean":"2023-10-18T10:41:30.434717+0000","last_became_active":"2023-10-18T10:41:29.447288+0000","last_became_peered":"2023-10-18T10:41:29.447288+0000","last_unstale":"2023-10-18T10:41:30.434717+0000","last_undegraded":"2023-10-18T10:41:30.434717+0000","last_fullsized":"2023-10-18T10:41:30.434717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:30.647260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750857+0000","last_change":"2023-10-18T10:41:29.442252+0000","last_active":"2023-10-18T10:41:30.750857+0000","last_peered":"2023-10-18T10:41:30.750857+0000","last_clean":"2023-10-18T10:41:30.750857+0000","last_became_active":"2023-10-18T10:41:29.441955+0000","last_became_peered":"2023-10-18T10:41:29.441955+0000","last_unstale":"2023-10-18T10:41:30.750857+0000","last_undegraded":"2023-10-18T10:41:30.750857+0000","last_fullsized":"2023-10-18T10:41:30.750857+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:23.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434545+0000","last_change":"2023-10-18T10:41:29.452284+0000","last_active":"2023-10-18T10:41:30.434545+0000","last_peered":"2023-10-18T10:41:30.434545+0000","last_clean":"2023-10-18T10:41:30.434545+0000","last_became_active":"2023-10-18T10:41:29.452037+0000","last_became_peered":"2023-10-18T10:41:29.452037+0000","last_unstale":"2023-10-18T10:41:30.434545+0000","last_undegraded":"2023-10-18T10:41:30.434545+0000","last_fullsized":"2023-10-18T10:41:30.434545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.154689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750025+0000","last_change":"2023-10-18T10:41:29.453178+0000","last_active":"2023-10-18T10:41:30.750025+0000","last_peered":"2023-10-18T10:41:30.750025+0000","last_clean":"2023-10-18T10:41:30.750025+0000","last_became_active":"2023-10-18T10:41:29.452902+0000","last_became_peered":"2023-10-18T10:41:29.452902+0000","last_unstale":"2023-10-18T10:41:30.750025+0000","last_undegraded":"2023-10-18T10:41:30.750025+0000","last_fullsized":"2023-10-18T10:41:30.750025+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:23.594087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437984+0000","last_change":"2023-10-18T10:41:29.452729+0000","last_active":"2023-10-18T10:41:30.437984+0000","last_peered":"2023-10-18T10:41:30.437984+0000","last_clean":"2023-10-18T10:41:30.437984+0000","last_became_active":"2023-10-18T10:41:29.452415+0000","last_became_peered":"2023-10-18T10:41:29.452415+0000","last_unstale":"2023-10-18T10:41:30.437984+0000","last_undegraded":"2023-10-18T10:41:30.437984+0000","last_fullsized":"2023-10-18T10:41:30.437984+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:25:17.106887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750972+0000","last_change":"2023-10-18T10:41:29.450362+0000","last_active":"2023-10-18T10:41:30.750972+0000","last_peered":"2023-10-18T10:41:30.750972+0000","last_clean":"2023-10-18T10:41:30.750972+0000","last_became_active":"2023-10-18T10:41:29.449972+0000","last_became_peered":"2023-10-18T10:41:29.449972+0000","last_unstale":"2023-10-18T10:41:30.750972+0000","last_undegraded":"2023-10-18T10:41:30.750972+0000","last_fullsized":"2023-10-18T10:41:30.750972+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:59.437842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750535+0000","last_change":"2023-10-18T10:41:29.450173+0000","last_active":"2023-10-18T10:41:30.750535+0000","last_peered":"2023-10-18T10:41:30.750535+0000","last_clean":"2023-10-18T10:41:30.750535+0000","last_became_active":"2023-10-18T10:41:29.449859+0000","last_became_peered":"2023-10-18T10:41:29.449859+0000","last_unstale":"2023-10-18T10:41:30.750535+0000","last_undegraded":"2023-10-18T10:41:30.750535+0000","last_fullsized":"2023-10-18T10:41:30.750535+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.385853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749904+0000","last_change":"2023-10-18T10:41:29.449393+0000","last_active":"2023-10-18T10:41:30.749904+0000","last_peered":"2023-10-18T10:41:30.749904+0000","last_clean":"2023-10-18T10:41:30.749904+0000","last_became_active":"2023-10-18T10:41:29.449086+0000","last_became_peered":"2023-10-18T10:41:29.449086+0000","last_unstale":"2023-10-18T10:41:30.749904+0000","last_undegraded":"2023-10-18T10:41:30.749904+0000","last_fullsized":"2023-10-18T10:41:30.749904+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:41.061091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438599+0000","last_change":"2023-10-18T10:41:29.447548+0000","last_active":"2023-10-18T10:41:30.438599+0000","last_peered":"2023-10-18T10:41:30.438599+0000","last_clean":"2023-10-18T10:41:30.438599+0000","last_became_active":"2023-10-18T10:41:29.447242+0000","last_became_peered":"2023-10-18T10:41:29.447242+0000","last_unstale":"2023-10-18T10:41:30.438599+0000","last_undegraded":"2023-10-18T10:41:30.438599+0000","last_fullsized":"2023-10-18T10:41:30.438599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:41.061091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751632+0000","last_change":"2023-10-18T10:41:29.444304+0000","last_active":"2023-10-18T10:41:30.751632+0000","last_peered":"2023-10-18T10:41:30.751632+0000","last_clean":"2023-10-18T10:41:30.751632+0000","last_became_active":"2023-10-18T10:41:29.444057+0000","last_became_peered":"2023-10-18T10:41:29.444057+0000","last_unstale":"2023-10-18T10:41:30.751632+0000","last_undegraded":"2023-10-18T10:41:30.751632+0000","last_fullsized":"2023-10-18T10:41:30.751632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:23.209276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438263+0000","last_change":"2023-10-18T10:41:29.448537+0000","last_active":"2023-10-18T10:41:30.438263+0000","last_peered":"2023-10-18T10:41:30.438263+0000","last_clean":"2023-10-18T10:41:30.438263+0000","last_became_active":"2023-10-18T10:41:29.447727+0000","last_became_peered":"2023-10-18T10:41:29.447727+0000","last_unstale":"2023-10-18T10:41:30.438263+0000","last_undegraded":"2023-10-18T10:41:30.438263+0000","last_fullsized":"2023-10-18T10:41:30.438263+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:05.122562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751364+0000","last_change":"2023-10-18T10:41:29.443320+0000","last_active":"2023-10-18T10:41:30.751364+0000","last_peered":"2023-10-18T10:41:30.751364+0000","last_clean":"2023-10-18T10:41:30.751364+0000","last_became_active":"2023-10-18T10:41:29.443029+0000","last_became_peered":"2023-10-18T10:41:29.443029+0000","last_unstale":"2023-10-18T10:41:30.751364+0000","last_undegraded":"2023-10-18T10:41:30.751364+0000","last_fullsized":"2023-10-18T10:41:30.751364+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:27.802284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750628+0000","last_change":"2023-10-18T10:41:29.448605+0000","last_active":"2023-10-18T10:41:30.750628+0000","last_peered":"2023-10-18T10:41:30.750628+0000","last_clean":"2023-10-18T10:41:30.750628+0000","last_became_active":"2023-10-18T10:41:29.448348+0000","last_became_peered":"2023-10-18T10:41:29.448348+0000","last_unstale":"2023-10-18T10:41:30.750628+0000","last_undegraded":"2023-10-18T10:41:30.750628+0000","last_fullsized":"2023-10-18T10:41:30.750628+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:42.695172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437791+0000","last_change":"2023-10-18T10:41:29.455958+0000","last_active":"2023-10-18T10:41:30.437791+0000","last_peered":"2023-10-18T10:41:30.437791+0000","last_clean":"2023-10-18T10:41:30.437791+0000","last_became_active":"2023-10-18T10:41:29.455667+0000","last_became_peered":"2023-10-18T10:41:29.455667+0000","last_unstale":"2023-10-18T10:41:30.437791+0000","last_undegraded":"2023-10-18T10:41:30.437791+0000","last_fullsized":"2023-10-18T10:41:30.437791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:05.134557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750043+0000","last_change":"2023-10-18T10:41:29.444407+0000","last_active":"2023-10-18T10:41:30.750043+0000","last_peered":"2023-10-18T10:41:30.750043+0000","last_clean":"2023-10-18T10:41:30.750043+0000","last_became_active":"2023-10-18T10:41:29.444169+0000","last_became_peered":"2023-10-18T10:41:29.444169+0000","last_unstale":"2023-10-18T10:41:30.750043+0000","last_undegraded":"2023-10-18T10:41:30.750043+0000","last_fullsized":"2023-10-18T10:41:30.750043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:13.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750781+0000","last_change":"2023-10-18T10:41:29.454015+0000","last_active":"2023-10-18T10:41:30.750781+0000","last_peered":"2023-10-18T10:41:30.750781+0000","last_clean":"2023-10-18T10:41:30.750781+0000","last_became_active":"2023-10-18T10:41:29.451538+0000","last_became_peered":"2023-10-18T10:41:29.451538+0000","last_unstale":"2023-10-18T10:41:30.750781+0000","last_undegraded":"2023-10-18T10:41:30.750781+0000","last_fullsized":"2023-10-18T10:41:30.750781+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:52.418194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751142+0000","last_change":"2023-10-18T10:41:29.449763+0000","last_active":"2023-10-18T10:41:30.751142+0000","last_peered":"2023-10-18T10:41:30.751142+0000","last_clean":"2023-10-18T10:41:30.751142+0000","last_became_active":"2023-10-18T10:41:29.449498+0000","last_became_peered":"2023-10-18T10:41:29.449498+0000","last_unstale":"2023-10-18T10:41:30.751142+0000","last_undegraded":"2023-10-18T10:41:30.751142+0000","last_fullsized":"2023-10-18T10:41:30.751142+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:30.606377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434672+0000","last_change":"2023-10-18T10:41:29.448915+0000","last_active":"2023-10-18T10:41:30.434672+0000","last_peered":"2023-10-18T10:41:30.434672+0000","last_clean":"2023-10-18T10:41:30.434672+0000","last_became_active":"2023-10-18T10:41:29.448650+0000","last_became_peered":"2023-10-18T10:41:29.448650+0000","last_unstale":"2023-10-18T10:41:30.434672+0000","last_undegraded":"2023-10-18T10:41:30.434672+0000","last_fullsized":"2023-10-18T10:41:30.434672+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:41.152281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438286+0000","last_change":"2023-10-18T10:41:29.455993+0000","last_active":"2023-10-18T10:41:30.438286+0000","last_peered":"2023-10-18T10:41:30.438286+0000","last_clean":"2023-10-18T10:41:30.438286+0000","last_became_active":"2023-10-18T10:41:29.455678+0000","last_became_peered":"2023-10-18T10:41:29.455678+0000","last_unstale":"2023-10-18T10:41:30.438286+0000","last_undegraded":"2023-10-18T10:41:30.438286+0000","last_fullsized":"2023-10-18T10:41:30.438286+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:20.769610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750152+0000","last_change":"2023-10-18T10:41:29.455557+0000","last_active":"2023-10-18T10:41:30.750152+0000","last_peered":"2023-10-18T10:41:30.750152+0000","last_clean":"2023-10-18T10:41:30.750152+0000","last_became_active":"2023-10-18T10:41:29.455296+0000","last_became_peered":"2023-10-18T10:41:29.455296+0000","last_unstale":"2023-10-18T10:41:30.750152+0000","last_undegraded":"2023-10-18T10:41:30.750152+0000","last_fullsized":"2023-10-18T10:41:30.750152+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:51.749366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437980+0000","last_change":"2023-10-18T10:41:29.447863+0000","last_active":"2023-10-18T10:41:30.437980+0000","last_peered":"2023-10-18T10:41:30.437980+0000","last_clean":"2023-10-18T10:41:30.437980+0000","last_became_active":"2023-10-18T10:41:29.447353+0000","last_became_peered":"2023-10-18T10:41:29.447353+0000","last_unstale":"2023-10-18T10:41:30.437980+0000","last_undegraded":"2023-10-18T10:41:30.437980+0000","last_fullsized":"2023-10-18T10:41:30.437980+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:45.211505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437796+0000","last_change":"2023-10-18T10:41:29.455453+0000","last_active":"2023-10-18T10:41:30.437796+0000","last_peered":"2023-10-18T10:41:30.437796+0000","last_clean":"2023-10-18T10:41:30.437796+0000","last_became_active":"2023-10-18T10:41:29.455076+0000","last_became_peered":"2023-10-18T10:41:29.455076+0000","last_unstale":"2023-10-18T10:41:30.437796+0000","last_undegraded":"2023-10-18T10:41:30.437796+0000","last_fullsized":"2023-10-18T10:41:30.437796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:42.659244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750459+0000","last_change":"2023-10-18T10:41:29.453973+0000","last_active":"2023-10-18T10:41:30.750459+0000","last_peered":"2023-10-18T10:41:30.750459+0000","last_clean":"2023-10-18T10:41:30.750459+0000","last_became_active":"2023-10-18T10:41:29.453686+0000","last_became_peered":"2023-10-18T10:41:29.453686+0000","last_unstale":"2023-10-18T10:41:30.750459+0000","last_undegraded":"2023-10-18T10:41:30.750459+0000","last_fullsized":"2023-10-18T10:41:30.750459+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:05:15.082425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434493+0000","last_change":"2023-10-18T10:41:29.453702+0000","last_active":"2023-10-18T10:41:30.434493+0000","last_peered":"2023-10-18T10:41:30.434493+0000","last_clean":"2023-10-18T10:41:30.434493+0000","last_became_active":"2023-10-18T10:41:29.453445+0000","last_became_peered":"2023-10-18T10:41:29.453445+0000","last_unstale":"2023-10-18T10:41:30.434493+0000","last_undegraded":"2023-10-18T10:41:30.434493+0000","last_fullsized":"2023-10-18T10:41:30.434493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:28.572719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750696+0000","last_change":"2023-10-18T10:41:29.451267+0000","last_active":"2023-10-18T10:41:30.750696+0000","last_peered":"2023-10-18T10:41:30.750696+0000","last_clean":"2023-10-18T10:41:30.750696+0000","last_became_active":"2023-10-18T10:41:29.450963+0000","last_became_peered":"2023-10-18T10:41:29.450963+0000","last_unstale":"2023-10-18T10:41:30.750696+0000","last_undegraded":"2023-10-18T10:41:30.750696+0000","last_fullsized":"2023-10-18T10:41:30.750696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:01.499934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438439+0000","last_change":"2023-10-18T10:41:29.456047+0000","last_active":"2023-10-18T10:41:30.438439+0000","last_peered":"2023-10-18T10:41:30.438439+0000","last_clean":"2023-10-18T10:41:30.438439+0000","last_became_active":"2023-10-18T10:41:29.455742+0000","last_became_peered":"2023-10-18T10:41:29.455742+0000","last_unstale":"2023-10-18T10:41:30.438439+0000","last_undegraded":"2023-10-18T10:41:30.438439+0000","last_fullsized":"2023-10-18T10:41:30.438439+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:46:06.183343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751548+0000","last_change":"2023-10-18T10:41:29.453816+0000","last_active":"2023-10-18T10:41:30.751548+0000","last_peered":"2023-10-18T10:41:30.751548+0000","last_clean":"2023-10-18T10:41:30.751548+0000","last_became_active":"2023-10-18T10:41:29.453581+0000","last_became_peered":"2023-10-18T10:41:29.453581+0000","last_unstale":"2023-10-18T10:41:30.751548+0000","last_undegraded":"2023-10-18T10:41:30.751548+0000","last_fullsized":"2023-10-18T10:41:30.751548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:18.669429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750180+0000","last_change":"2023-10-18T10:41:29.454500+0000","last_active":"2023-10-18T10:41:30.750180+0000","last_peered":"2023-10-18T10:41:30.750180+0000","last_clean":"2023-10-18T10:41:30.750180+0000","last_became_active":"2023-10-18T10:41:29.454231+0000","last_became_peered":"2023-10-18T10:41:29.454231+0000","last_unstale":"2023-10-18T10:41:30.750180+0000","last_undegraded":"2023-10-18T10:41:30.750180+0000","last_fullsized":"2023-10-18T10:41:30.750180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:14.224881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751332+0000","last_change":"2023-10-18T10:41:29.453392+0000","last_active":"2023-10-18T10:41:30.751332+0000","last_peered":"2023-10-18T10:41:30.751332+0000","last_clean":"2023-10-18T10:41:30.751332+0000","last_became_active":"2023-10-18T10:41:29.453089+0000","last_became_peered":"2023-10-18T10:41:29.453089+0000","last_unstale":"2023-10-18T10:41:30.751332+0000","last_undegraded":"2023-10-18T10:41:30.751332+0000","last_fullsized":"2023-10-18T10:41:30.751332+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:52.483119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437981+0000","last_change":"2023-10-18T10:41:29.448083+0000","last_active":"2023-10-18T10:41:30.437981+0000","last_peered":"2023-10-18T10:41:30.437981+0000","last_clean":"2023-10-18T10:41:30.437981+0000","last_became_active":"2023-10-18T10:41:29.447584+0000","last_became_peered":"2023-10-18T10:41:29.447584+0000","last_unstale":"2023-10-18T10:41:30.437981+0000","last_undegraded":"2023-10-18T10:41:30.437981+0000","last_fullsized":"2023-10-18T10:41:30.437981+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:43.081792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750335+0000","last_change":"2023-10-18T10:41:29.453993+0000","last_active":"2023-10-18T10:41:30.750335+0000","last_peered":"2023-10-18T10:41:30.750335+0000","last_clean":"2023-10-18T10:41:30.750335+0000","last_became_active":"2023-10-18T10:41:29.453659+0000","last_became_peered":"2023-10-18T10:41:29.453659+0000","last_unstale":"2023-10-18T10:41:30.750335+0000","last_undegraded":"2023-10-18T10:41:30.750335+0000","last_fullsized":"2023-10-18T10:41:30.750335+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:50.051564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750971+0000","last_change":"2023-10-18T10:41:29.445142+0000","last_active":"2023-10-18T10:41:30.750971+0000","last_peered":"2023-10-18T10:41:30.750971+0000","last_clean":"2023-10-18T10:41:30.750971+0000","last_became_active":"2023-10-18T10:41:29.444871+0000","last_became_peered":"2023-10-18T10:41:29.444871+0000","last_unstale":"2023-10-18T10:41:30.750971+0000","last_undegraded":"2023-10-18T10:41:30.750971+0000","last_fullsized":"2023-10-18T10:41:30.750971+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:31:23.805032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750791+0000","last_change":"2023-10-18T10:41:29.448568+0000","last_active":"2023-10-18T10:41:30.750791+0000","last_peered":"2023-10-18T10:41:30.750791+0000","last_clean":"2023-10-18T10:41:30.750791+0000","last_became_active":"2023-10-18T10:41:29.448318+0000","last_became_peered":"2023-10-18T10:41:29.448318+0000","last_unstale":"2023-10-18T10:41:30.750791+0000","last_undegraded":"2023-10-18T10:41:30.750791+0000","last_fullsized":"2023-10-18T10:41:30.750791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:29.589441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"111'4","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751169+0000","last_change":"2023-10-18T10:41:28.435122+0000","last_active":"2023-10-18T10:41:30.751169+0000","last_peered":"2023-10-18T10:41:30.751169+0000","last_clean":"2023-10-18T10:41:30.751169+0000","last_became_active":"2023-10-18T10:41:28.434844+0000","last_became_peered":"2023-10-18T10:41:28.434844+0000","last_unstale":"2023-10-18T10:41:30.751169+0000","last_undegraded":"2023-10-18T10:41:30.751169+0000","last_fullsized":"2023-10-18T10:41:30.751169+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:04.834294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751302+0000","last_change":"2023-10-18T10:41:29.445971+0000","last_active":"2023-10-18T10:41:30.751302+0000","last_peered":"2023-10-18T10:41:30.751302+0000","last_clean":"2023-10-18T10:41:30.751302+0000","last_became_active":"2023-10-18T10:41:29.445723+0000","last_became_peered":"2023-10-18T10:41:29.445723+0000","last_unstale":"2023-10-18T10:41:30.751302+0000","last_undegraded":"2023-10-18T10:41:30.751302+0000","last_fullsized":"2023-10-18T10:41:30.751302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:35:45.397768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438376+0000","last_change":"2023-10-18T10:41:28.436698+0000","last_active":"2023-10-18T10:41:30.438376+0000","last_peered":"2023-10-18T10:41:30.438376+0000","last_clean":"2023-10-18T10:41:30.438376+0000","last_became_active":"2023-10-18T10:41:28.436411+0000","last_became_peered":"2023-10-18T10:41:28.436411+0000","last_unstale":"2023-10-18T10:41:30.438376+0000","last_undegraded":"2023-10-18T10:41:30.438376+0000","last_fullsized":"2023-10-18T10:41:30.438376+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:06.613711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751070+0000","last_change":"2023-10-18T10:41:29.457858+0000","last_active":"2023-10-18T10:41:30.751070+0000","last_peered":"2023-10-18T10:41:30.751070+0000","last_clean":"2023-10-18T10:41:30.751070+0000","last_became_active":"2023-10-18T10:41:29.457555+0000","last_became_peered":"2023-10-18T10:41:29.457555+0000","last_unstale":"2023-10-18T10:41:30.751070+0000","last_undegraded":"2023-10-18T10:41:30.751070+0000","last_fullsized":"2023-10-18T10:41:30.751070+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:12.165512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750907+0000","last_change":"2023-10-18T10:41:28.432593+0000","last_active":"2023-10-18T10:41:30.750907+0000","last_peered":"2023-10-18T10:41:30.750907+0000","last_clean":"2023-10-18T10:41:30.750907+0000","last_became_active":"2023-10-18T10:41:28.432296+0000","last_became_peered":"2023-10-18T10:41:28.432296+0000","last_unstale":"2023-10-18T10:41:30.750907+0000","last_undegraded":"2023-10-18T10:41:30.750907+0000","last_fullsized":"2023-10-18T10:41:30.750907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:27.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751225+0000","last_change":"2023-10-18T10:41:29.451021+0000","last_active":"2023-10-18T10:41:30.751225+0000","last_peered":"2023-10-18T10:41:30.751225+0000","last_clean":"2023-10-18T10:41:30.751225+0000","last_became_active":"2023-10-18T10:41:29.450779+0000","last_became_peered":"2023-10-18T10:41:29.450779+0000","last_unstale":"2023-10-18T10:41:30.751225+0000","last_undegraded":"2023-10-18T10:41:30.751225+0000","last_fullsized":"2023-10-18T10:41:30.751225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:36.701370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751378+0000","last_change":"2023-10-18T10:41:28.429164+0000","last_active":"2023-10-18T10:41:30.751378+0000","last_peered":"2023-10-18T10:41:30.751378+0000","last_clean":"2023-10-18T10:41:30.751378+0000","last_became_active":"2023-10-18T10:41:28.428856+0000","last_became_peered":"2023-10-18T10:41:28.428856+0000","last_unstale":"2023-10-18T10:41:30.751378+0000","last_undegraded":"2023-10-18T10:41:30.751378+0000","last_fullsized":"2023-10-18T10:41:30.751378+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:55:18.343120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749949+0000","last_change":"2023-10-18T10:41:29.443511+0000","last_active":"2023-10-18T10:41:30.749949+0000","last_peered":"2023-10-18T10:41:30.749949+0000","last_clean":"2023-10-18T10:41:30.749949+0000","last_became_active":"2023-10-18T10:41:29.443232+0000","last_became_peered":"2023-10-18T10:41:29.443232+0000","last_unstale":"2023-10-18T10:41:30.749949+0000","last_undegraded":"2023-10-18T10:41:30.749949+0000","last_fullsized":"2023-10-18T10:41:30.749949+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:29.869647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438137+0000","last_change":"2023-10-18T10:41:28.441255+0000","last_active":"2023-10-18T10:41:30.438137+0000","last_peered":"2023-10-18T10:41:30.438137+0000","last_clean":"2023-10-18T10:41:30.438137+0000","last_became_active":"2023-10-18T10:41:28.440983+0000","last_became_peered":"2023-10-18T10:41:28.440983+0000","last_unstale":"2023-10-18T10:41:30.438137+0000","last_undegraded":"2023-10-18T10:41:30.438137+0000","last_fullsized":"2023-10-18T10:41:30.438137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:34.041596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750604+0000","last_change":"2023-10-18T10:41:29.448123+0000","last_active":"2023-10-18T10:41:30.750604+0000","last_peered":"2023-10-18T10:41:30.750604+0000","last_clean":"2023-10-18T10:41:30.750604+0000","last_became_active":"2023-10-18T10:41:29.447856+0000","last_became_peered":"2023-10-18T10:41:29.447856+0000","last_unstale":"2023-10-18T10:41:30.750604+0000","last_undegraded":"2023-10-18T10:41:30.750604+0000","last_fullsized":"2023-10-18T10:41:30.750604+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:52.661375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750830+0000","last_change":"2023-10-18T10:41:28.439965+0000","last_active":"2023-10-18T10:41:30.750830+0000","last_peered":"2023-10-18T10:41:30.750830+0000","last_clean":"2023-10-18T10:41:30.750830+0000","last_became_active":"2023-10-18T10:41:28.439700+0000","last_became_peered":"2023-10-18T10:41:28.439700+0000","last_unstale":"2023-10-18T10:41:30.750830+0000","last_undegraded":"2023-10-18T10:41:30.750830+0000","last_fullsized":"2023-10-18T10:41:30.750830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:44.202065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750979+0000","last_change":"2023-10-18T10:41:29.450500+0000","last_active":"2023-10-18T10:41:30.750979+0000","last_peered":"2023-10-18T10:41:30.750979+0000","last_clean":"2023-10-18T10:41:30.750979+0000","last_became_active":"2023-10-18T10:41:29.450234+0000","last_became_peered":"2023-10-18T10:41:29.450234+0000","last_unstale":"2023-10-18T10:41:30.750979+0000","last_undegraded":"2023-10-18T10:41:30.750979+0000","last_fullsized":"2023-10-18T10:41:30.750979+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:08.866019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434561+0000","last_change":"2023-10-18T10:41:28.435875+0000","last_active":"2023-10-18T10:41:30.434561+0000","last_peered":"2023-10-18T10:41:30.434561+0000","last_clean":"2023-10-18T10:41:30.434561+0000","last_became_active":"2023-10-18T10:41:28.435623+0000","last_became_peered":"2023-10-18T10:41:28.435623+0000","last_unstale":"2023-10-18T10:41:30.434561+0000","last_undegraded":"2023-10-18T10:41:30.434561+0000","last_fullsized":"2023-10-18T10:41:30.434561+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:22.897591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750655+0000","last_change":"2023-10-18T10:41:29.451882+0000","last_active":"2023-10-18T10:41:30.750655+0000","last_peered":"2023-10-18T10:41:30.750655+0000","last_clean":"2023-10-18T10:41:30.750655+0000","last_became_active":"2023-10-18T10:41:29.451623+0000","last_became_peered":"2023-10-18T10:41:29.451623+0000","last_unstale":"2023-10-18T10:41:30.750655+0000","last_undegraded":"2023-10-18T10:41:30.750655+0000","last_fullsized":"2023-10-18T10:41:30.750655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:39.862682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750687+0000","last_change":"2023-10-18T10:41:28.436040+0000","last_active":"2023-10-18T10:41:30.750687+0000","last_peered":"2023-10-18T10:41:30.750687+0000","last_clean":"2023-10-18T10:41:30.750687+0000","last_became_active":"2023-10-18T10:41:28.434223+0000","last_became_peered":"2023-10-18T10:41:28.434223+0000","last_unstale":"2023-10-18T10:41:30.750687+0000","last_undegraded":"2023-10-18T10:41:30.750687+0000","last_fullsized":"2023-10-18T10:41:30.750687+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.620461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438253+0000","last_change":"2023-10-18T10:41:29.452229+0000","last_active":"2023-10-18T10:41:30.438253+0000","last_peered":"2023-10-18T10:41:30.438253+0000","last_clean":"2023-10-18T10:41:30.438253+0000","last_became_active":"2023-10-18T10:41:29.451930+0000","last_became_peered":"2023-10-18T10:41:29.451930+0000","last_unstale":"2023-10-18T10:41:30.438253+0000","last_undegraded":"2023-10-18T10:41:30.438253+0000","last_fullsized":"2023-10-18T10:41:30.438253+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:12.643803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750926+0000","last_change":"2023-10-18T10:41:28.436855+0000","last_active":"2023-10-18T10:41:30.750926+0000","last_peered":"2023-10-18T10:41:30.750926+0000","last_clean":"2023-10-18T10:41:30.750926+0000","last_became_active":"2023-10-18T10:41:28.436585+0000","last_became_peered":"2023-10-18T10:41:28.436585+0000","last_unstale":"2023-10-18T10:41:30.750926+0000","last_undegraded":"2023-10-18T10:41:30.750926+0000","last_fullsized":"2023-10-18T10:41:30.750926+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.155833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750812+0000","last_change":"2023-10-18T10:41:29.450388+0000","last_active":"2023-10-18T10:41:30.750812+0000","last_peered":"2023-10-18T10:41:30.750812+0000","last_clean":"2023-10-18T10:41:30.750812+0000","last_became_active":"2023-10-18T10:41:29.450092+0000","last_became_peered":"2023-10-18T10:41:29.450092+0000","last_unstale":"2023-10-18T10:41:30.750812+0000","last_undegraded":"2023-10-18T10:41:30.750812+0000","last_fullsized":"2023-10-18T10:41:30.750812+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:02:27.937026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434722+0000","last_change":"2023-10-18T10:41:28.440969+0000","last_active":"2023-10-18T10:41:30.434722+0000","last_peered":"2023-10-18T10:41:30.434722+0000","last_clean":"2023-10-18T10:41:30.434722+0000","last_became_active":"2023-10-18T10:41:28.440739+0000","last_became_peered":"2023-10-18T10:41:28.440739+0000","last_unstale":"2023-10-18T10:41:30.434722+0000","last_undegraded":"2023-10-18T10:41:30.434722+0000","last_fullsized":"2023-10-18T10:41:30.434722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.620461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434696+0000","last_change":"2023-10-18T10:41:29.456187+0000","last_active":"2023-10-18T10:41:30.434696+0000","last_peered":"2023-10-18T10:41:30.434696+0000","last_clean":"2023-10-18T10:41:30.434696+0000","last_became_active":"2023-10-18T10:41:29.455945+0000","last_became_peered":"2023-10-18T10:41:29.455945+0000","last_unstale":"2023-10-18T10:41:30.434696+0000","last_undegraded":"2023-10-18T10:41:30.434696+0000","last_fullsized":"2023-10-18T10:41:30.434696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:40.835626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"111'3","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438389+0000","last_change":"2023-10-18T10:41:28.440316+0000","last_active":"2023-10-18T10:41:30.438389+0000","last_peered":"2023-10-18T10:41:30.438389+0000","last_clean":"2023-10-18T10:41:30.438389+0000","last_became_active":"2023-10-18T10:41:28.440024+0000","last_became_peered":"2023-10-18T10:41:28.440024+0000","last_unstale":"2023-10-18T10:41:30.438389+0000","last_undegraded":"2023-10-18T10:41:30.438389+0000","last_fullsized":"2023-10-18T10:41:30.438389+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:51.651935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750493+0000","last_change":"2023-10-18T10:41:29.449408+0000","last_active":"2023-10-18T10:41:30.750493+0000","last_peered":"2023-10-18T10:41:30.750493+0000","last_clean":"2023-10-18T10:41:30.750493+0000","last_became_active":"2023-10-18T10:41:29.449161+0000","last_became_peered":"2023-10-18T10:41:29.449161+0000","last_unstale":"2023-10-18T10:41:30.750493+0000","last_undegraded":"2023-10-18T10:41:30.750493+0000","last_fullsized":"2023-10-18T10:41:30.750493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:41.742410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"111'6","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751094+0000","last_change":"2023-10-18T10:41:28.427403+0000","last_active":"2023-10-18T10:41:30.751094+0000","last_peered":"2023-10-18T10:41:30.751094+0000","last_clean":"2023-10-18T10:41:30.751094+0000","last_became_active":"2023-10-18T10:41:28.427119+0000","last_became_peered":"2023-10-18T10:41:28.427119+0000","last_unstale":"2023-10-18T10:41:30.751094+0000","last_undegraded":"2023-10-18T10:41:30.751094+0000","last_fullsized":"2023-10-18T10:41:30.751094+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:11.156072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750239+0000","last_change":"2023-10-18T10:41:29.448792+0000","last_active":"2023-10-18T10:41:30.750239+0000","last_peered":"2023-10-18T10:41:30.750239+0000","last_clean":"2023-10-18T10:41:30.750239+0000","last_became_active":"2023-10-18T10:41:29.448536+0000","last_became_peered":"2023-10-18T10:41:29.448536+0000","last_unstale":"2023-10-18T10:41:30.750239+0000","last_undegraded":"2023-10-18T10:41:30.750239+0000","last_fullsized":"2023-10-18T10:41:30.750239+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:14.300602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751307+0000","last_change":"2023-10-18T10:41:28.434350+0000","last_active":"2023-10-18T10:41:30.751307+0000","last_peered":"2023-10-18T10:41:30.751307+0000","last_clean":"2023-10-18T10:41:30.751307+0000","last_became_active":"2023-10-18T10:41:28.434068+0000","last_became_peered":"2023-10-18T10:41:28.434068+0000","last_unstale":"2023-10-18T10:41:30.751307+0000","last_undegraded":"2023-10-18T10:41:30.751307+0000","last_fullsized":"2023-10-18T10:41:30.751307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:39.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437865+0000","last_change":"2023-10-18T10:41:29.447478+0000","last_active":"2023-10-18T10:41:30.437865+0000","last_peered":"2023-10-18T10:41:30.437865+0000","last_clean":"2023-10-18T10:41:30.437865+0000","last_became_active":"2023-10-18T10:41:29.447163+0000","last_became_peered":"2023-10-18T10:41:29.447163+0000","last_unstale":"2023-10-18T10:41:30.437865+0000","last_undegraded":"2023-10-18T10:41:30.437865+0000","last_fullsized":"2023-10-18T10:41:30.437865+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:27.997549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751060+0000","last_change":"2023-10-18T10:41:28.433856+0000","last_active":"2023-10-18T10:41:30.751060+0000","last_peered":"2023-10-18T10:41:30.751060+0000","last_clean":"2023-10-18T10:41:30.751060+0000","last_became_active":"2023-10-18T10:41:28.433584+0000","last_became_peered":"2023-10-18T10:41:28.433584+0000","last_unstale":"2023-10-18T10:41:30.751060+0000","last_undegraded":"2023-10-18T10:41:30.751060+0000","last_fullsized":"2023-10-18T10:41:30.751060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:06:07.806734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750178+0000","last_change":"2023-10-18T10:41:29.439852+0000","last_active":"2023-10-18T10:41:30.750178+0000","last_peered":"2023-10-18T10:41:30.750178+0000","last_clean":"2023-10-18T10:41:30.750178+0000","last_became_active":"2023-10-18T10:41:29.439612+0000","last_became_peered":"2023-10-18T10:41:29.439612+0000","last_unstale":"2023-10-18T10:41:30.750178+0000","last_undegraded":"2023-10-18T10:41:30.750178+0000","last_fullsized":"2023-10-18T10:41:30.750178+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:10.074146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750194+0000","last_change":"2023-10-18T10:41:28.431914+0000","last_active":"2023-10-18T10:41:30.750194+0000","last_peered":"2023-10-18T10:41:30.750194+0000","last_clean":"2023-10-18T10:41:30.750194+0000","last_became_active":"2023-10-18T10:41:28.431664+0000","last_became_peered":"2023-10-18T10:41:28.431664+0000","last_unstale":"2023-10-18T10:41:30.750194+0000","last_undegraded":"2023-10-18T10:41:30.750194+0000","last_fullsized":"2023-10-18T10:41:30.750194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:41.649804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751367+0000","last_change":"2023-10-18T10:41:29.453041+0000","last_active":"2023-10-18T10:41:30.751367+0000","last_peered":"2023-10-18T10:41:30.751367+0000","last_clean":"2023-10-18T10:41:30.751367+0000","last_became_active":"2023-10-18T10:41:29.452788+0000","last_became_peered":"2023-10-18T10:41:29.452788+0000","last_unstale":"2023-10-18T10:41:30.751367+0000","last_undegraded":"2023-10-18T10:41:30.751367+0000","last_fullsized":"2023-10-18T10:41:30.751367+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:28.740818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"111'8","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750045+0000","last_change":"2023-10-18T10:41:28.440009+0000","last_active":"2023-10-18T10:41:30.750045+0000","last_peered":"2023-10-18T10:41:30.750045+0000","last_clean":"2023-10-18T10:41:30.750045+0000","last_became_active":"2023-10-18T10:41:28.439742+0000","last_became_peered":"2023-10-18T10:41:28.439742+0000","last_unstale":"2023-10-18T10:41:30.750045+0000","last_undegraded":"2023-10-18T10:41:30.750045+0000","last_fullsized":"2023-10-18T10:41:30.750045+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:42.072352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751005+0000","last_change":"2023-10-18T10:41:29.448022+0000","last_active":"2023-10-18T10:41:30.751005+0000","last_peered":"2023-10-18T10:41:30.751005+0000","last_clean":"2023-10-18T10:41:30.751005+0000","last_became_active":"2023-10-18T10:41:29.447771+0000","last_became_peered":"2023-10-18T10:41:29.447771+0000","last_unstale":"2023-10-18T10:41:30.751005+0000","last_undegraded":"2023-10-18T10:41:30.751005+0000","last_fullsized":"2023-10-18T10:41:30.751005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:13.373299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751383+0000","last_change":"2023-10-18T10:41:28.430372+0000","last_active":"2023-10-18T10:41:30.751383+0000","last_peered":"2023-10-18T10:41:30.751383+0000","last_clean":"2023-10-18T10:41:30.751383+0000","last_became_active":"2023-10-18T10:41:28.430110+0000","last_became_peered":"2023-10-18T10:41:28.430110+0000","last_unstale":"2023-10-18T10:41:30.751383+0000","last_undegraded":"2023-10-18T10:41:30.751383+0000","last_fullsized":"2023-10-18T10:41:30.751383+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.826001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750892+0000","last_change":"2023-10-18T10:41:29.449385+0000","last_active":"2023-10-18T10:41:30.750892+0000","last_peered":"2023-10-18T10:41:30.750892+0000","last_clean":"2023-10-18T10:41:30.750892+0000","last_became_active":"2023-10-18T10:41:29.449126+0000","last_became_peered":"2023-10-18T10:41:29.449126+0000","last_unstale":"2023-10-18T10:41:30.750892+0000","last_undegraded":"2023-10-18T10:41:30.750892+0000","last_fullsized":"2023-10-18T10:41:30.750892+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:20.124373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751437+0000","last_change":"2023-10-18T10:41:28.430749+0000","last_active":"2023-10-18T10:41:30.751437+0000","last_peered":"2023-10-18T10:41:30.751437+0000","last_clean":"2023-10-18T10:41:30.751437+0000","last_became_active":"2023-10-18T10:41:28.430495+0000","last_became_peered":"2023-10-18T10:41:28.430495+0000","last_unstale":"2023-10-18T10:41:30.751437+0000","last_undegraded":"2023-10-18T10:41:30.751437+0000","last_fullsized":"2023-10-18T10:41:30.751437+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:00.943461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438254+0000","last_change":"2023-10-18T10:41:29.447633+0000","last_active":"2023-10-18T10:41:30.438254+0000","last_peered":"2023-10-18T10:41:30.438254+0000","last_clean":"2023-10-18T10:41:30.438254+0000","last_became_active":"2023-10-18T10:41:29.446778+0000","last_became_peered":"2023-10-18T10:41:29.446778+0000","last_unstale":"2023-10-18T10:41:30.438254+0000","last_undegraded":"2023-10-18T10:41:30.438254+0000","last_fullsized":"2023-10-18T10:41:30.438254+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:48:55.670658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438448+0000","last_change":"2023-10-18T10:41:28.438164+0000","last_active":"2023-10-18T10:41:30.438448+0000","last_peered":"2023-10-18T10:41:30.438448+0000","last_clean":"2023-10-18T10:41:30.438448+0000","last_became_active":"2023-10-18T10:41:28.437873+0000","last_became_peered":"2023-10-18T10:41:28.437873+0000","last_unstale":"2023-10-18T10:41:30.438448+0000","last_undegraded":"2023-10-18T10:41:30.438448+0000","last_fullsized":"2023-10-18T10:41:30.438448+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:53.068021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751109+0000","last_change":"2023-10-18T10:41:29.450342+0000","last_active":"2023-10-18T10:41:30.751109+0000","last_peered":"2023-10-18T10:41:30.751109+0000","last_clean":"2023-10-18T10:41:30.751109+0000","last_became_active":"2023-10-18T10:41:29.450078+0000","last_became_peered":"2023-10-18T10:41:29.450078+0000","last_unstale":"2023-10-18T10:41:30.751109+0000","last_undegraded":"2023-10-18T10:41:30.751109+0000","last_fullsized":"2023-10-18T10:41:30.751109+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:32.976432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750276+0000","last_change":"2023-10-18T10:41:28.438968+0000","last_active":"2023-10-18T10:41:30.750276+0000","last_peered":"2023-10-18T10:41:30.750276+0000","last_clean":"2023-10-18T10:41:30.750276+0000","last_became_active":"2023-10-18T10:41:28.438628+0000","last_became_peered":"2023-10-18T10:41:28.438628+0000","last_unstale":"2023-10-18T10:41:30.750276+0000","last_undegraded":"2023-10-18T10:41:30.750276+0000","last_fullsized":"2023-10-18T10:41:30.750276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:51.950983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750427+0000","last_change":"2023-10-18T10:41:29.442931+0000","last_active":"2023-10-18T10:41:30.750427+0000","last_peered":"2023-10-18T10:41:30.750427+0000","last_clean":"2023-10-18T10:41:30.750427+0000","last_became_active":"2023-10-18T10:41:29.442573+0000","last_became_peered":"2023-10-18T10:41:29.442573+0000","last_unstale":"2023-10-18T10:41:30.750427+0000","last_undegraded":"2023-10-18T10:41:30.750427+0000","last_fullsized":"2023-10-18T10:41:30.750427+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:36.667760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750311+0000","last_change":"2023-10-18T10:41:28.437156+0000","last_active":"2023-10-18T10:41:30.750311+0000","last_peered":"2023-10-18T10:41:30.750311+0000","last_clean":"2023-10-18T10:41:30.750311+0000","last_became_active":"2023-10-18T10:41:28.434657+0000","last_became_peered":"2023-10-18T10:41:28.434657+0000","last_unstale":"2023-10-18T10:41:30.750311+0000","last_undegraded":"2023-10-18T10:41:30.750311+0000","last_fullsized":"2023-10-18T10:41:30.750311+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:38:04.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750603+0000","last_change":"2023-10-18T10:41:29.452874+0000","last_active":"2023-10-18T10:41:30.750603+0000","last_peered":"2023-10-18T10:41:30.750603+0000","last_clean":"2023-10-18T10:41:30.750603+0000","last_became_active":"2023-10-18T10:41:29.452561+0000","last_became_peered":"2023-10-18T10:41:29.452561+0000","last_unstale":"2023-10-18T10:41:30.750603+0000","last_undegraded":"2023-10-18T10:41:30.750603+0000","last_fullsized":"2023-10-18T10:41:30.750603+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:26.000624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750422+0000","last_change":"2023-10-18T10:41:28.435576+0000","last_active":"2023-10-18T10:41:30.750422+0000","last_peered":"2023-10-18T10:41:30.750422+0000","last_clean":"2023-10-18T10:41:30.750422+0000","last_became_active":"2023-10-18T10:41:28.435286+0000","last_became_peered":"2023-10-18T10:41:28.435286+0000","last_unstale":"2023-10-18T10:41:30.750422+0000","last_undegraded":"2023-10-18T10:41:30.750422+0000","last_fullsized":"2023-10-18T10:41:30.750422+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:29.719996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750887+0000","last_change":"2023-10-18T10:41:29.446250+0000","last_active":"2023-10-18T10:41:30.750887+0000","last_peered":"2023-10-18T10:41:30.750887+0000","last_clean":"2023-10-18T10:41:30.750887+0000","last_became_active":"2023-10-18T10:41:29.445881+0000","last_became_peered":"2023-10-18T10:41:29.445881+0000","last_unstale":"2023-10-18T10:41:30.750887+0000","last_undegraded":"2023-10-18T10:41:30.750887+0000","last_fullsized":"2023-10-18T10:41:30.750887+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:28.061711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750340+0000","last_change":"2023-10-18T10:41:28.438325+0000","last_active":"2023-10-18T10:41:30.750340+0000","last_peered":"2023-10-18T10:41:30.750340+0000","last_clean":"2023-10-18T10:41:30.750340+0000","last_became_active":"2023-10-18T10:41:28.438056+0000","last_became_peered":"2023-10-18T10:41:28.438056+0000","last_unstale":"2023-10-18T10:41:30.750340+0000","last_undegraded":"2023-10-18T10:41:30.750340+0000","last_fullsized":"2023-10-18T10:41:30.750340+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:42.738454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751354+0000","last_change":"2023-10-18T10:41:29.449925+0000","last_active":"2023-10-18T10:41:30.751354+0000","last_peered":"2023-10-18T10:41:30.751354+0000","last_clean":"2023-10-18T10:41:30.751354+0000","last_became_active":"2023-10-18T10:41:29.449632+0000","last_became_peered":"2023-10-18T10:41:29.449632+0000","last_unstale":"2023-10-18T10:41:30.751354+0000","last_undegraded":"2023-10-18T10:41:30.751354+0000","last_fullsized":"2023-10-18T10:41:30.751354+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:35.051036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751184+0000","last_change":"2023-10-18T10:41:28.428975+0000","last_active":"2023-10-18T10:41:30.751184+0000","last_peered":"2023-10-18T10:41:30.751184+0000","last_clean":"2023-10-18T10:41:30.751184+0000","last_became_active":"2023-10-18T10:41:28.428703+0000","last_became_peered":"2023-10-18T10:41:28.428703+0000","last_unstale":"2023-10-18T10:41:30.751184+0000","last_undegraded":"2023-10-18T10:41:30.751184+0000","last_fullsized":"2023-10-18T10:41:30.751184+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:52.305622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751473+0000","last_change":"2023-10-18T10:41:29.449610+0000","last_active":"2023-10-18T10:41:30.751473+0000","last_peered":"2023-10-18T10:41:30.751473+0000","last_clean":"2023-10-18T10:41:30.751473+0000","last_became_active":"2023-10-18T10:41:29.449354+0000","last_became_peered":"2023-10-18T10:41:29.449354+0000","last_unstale":"2023-10-18T10:41:30.751473+0000","last_undegraded":"2023-10-18T10:41:30.751473+0000","last_fullsized":"2023-10-18T10:41:30.751473+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:01.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438483+0000","last_change":"2023-10-18T10:41:28.430655+0000","last_active":"2023-10-18T10:41:30.438483+0000","last_peered":"2023-10-18T10:41:30.438483+0000","last_clean":"2023-10-18T10:41:30.438483+0000","last_became_active":"2023-10-18T10:41:28.430389+0000","last_became_peered":"2023-10-18T10:41:28.430389+0000","last_unstale":"2023-10-18T10:41:30.438483+0000","last_undegraded":"2023-10-18T10:41:30.438483+0000","last_fullsized":"2023-10-18T10:41:30.438483+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:05:14.072985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751349+0000","last_change":"2023-10-18T10:41:29.447215+0000","last_active":"2023-10-18T10:41:30.751349+0000","last_peered":"2023-10-18T10:41:30.751349+0000","last_clean":"2023-10-18T10:41:30.751349+0000","last_became_active":"2023-10-18T10:41:29.446983+0000","last_became_peered":"2023-10-18T10:41:29.446983+0000","last_unstale":"2023-10-18T10:41:30.751349+0000","last_undegraded":"2023-10-18T10:41:30.751349+0000","last_fullsized":"2023-10-18T10:41:30.751349+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438513+0000","last_change":"2023-10-18T10:41:28.439870+0000","last_active":"2023-10-18T10:41:30.438513+0000","last_peered":"2023-10-18T10:41:30.438513+0000","last_clean":"2023-10-18T10:41:30.438513+0000","last_became_active":"2023-10-18T10:41:28.439587+0000","last_became_peered":"2023-10-18T10:41:28.439587+0000","last_unstale":"2023-10-18T10:41:30.438513+0000","last_undegraded":"2023-10-18T10:41:30.438513+0000","last_fullsized":"2023-10-18T10:41:30.438513+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:19.114933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750837+0000","last_change":"2023-10-18T10:41:29.448872+0000","last_active":"2023-10-18T10:41:30.750837+0000","last_peered":"2023-10-18T10:41:30.750837+0000","last_clean":"2023-10-18T10:41:30.750837+0000","last_became_active":"2023-10-18T10:41:29.448574+0000","last_became_peered":"2023-10-18T10:41:29.448574+0000","last_unstale":"2023-10-18T10:41:30.750837+0000","last_undegraded":"2023-10-18T10:41:30.750837+0000","last_fullsized":"2023-10-18T10:41:30.750837+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:40.934129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751444+0000","last_change":"2023-10-18T10:41:28.439553+0000","last_active":"2023-10-18T10:41:30.751444+0000","last_peered":"2023-10-18T10:41:30.751444+0000","last_clean":"2023-10-18T10:41:30.751444+0000","last_became_active":"2023-10-18T10:41:28.439237+0000","last_became_peered":"2023-10-18T10:41:28.439237+0000","last_unstale":"2023-10-18T10:41:30.751444+0000","last_undegraded":"2023-10-18T10:41:30.751444+0000","last_fullsized":"2023-10-18T10:41:30.751444+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:10.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749895+0000","last_change":"2023-10-18T10:41:29.442254+0000","last_active":"2023-10-18T10:41:30.749895+0000","last_peered":"2023-10-18T10:41:30.749895+0000","last_clean":"2023-10-18T10:41:30.749895+0000","last_became_active":"2023-10-18T10:41:29.441995+0000","last_became_peered":"2023-10-18T10:41:29.441995+0000","last_unstale":"2023-10-18T10:41:30.749895+0000","last_undegraded":"2023-10-18T10:41:30.749895+0000","last_fullsized":"2023-10-18T10:41:30.749895+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:13.960239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751093+0000","last_change":"2023-10-18T10:41:28.434524+0000","last_active":"2023-10-18T10:41:30.751093+0000","last_peered":"2023-10-18T10:41:30.751093+0000","last_clean":"2023-10-18T10:41:30.751093+0000","last_became_active":"2023-10-18T10:41:28.434260+0000","last_became_peered":"2023-10-18T10:41:28.434260+0000","last_unstale":"2023-10-18T10:41:30.751093+0000","last_undegraded":"2023-10-18T10:41:30.751093+0000","last_fullsized":"2023-10-18T10:41:30.751093+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:26:18.323130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434509+0000","last_change":"2023-10-18T10:41:29.455708+0000","last_active":"2023-10-18T10:41:30.434509+0000","last_peered":"2023-10-18T10:41:30.434509+0000","last_clean":"2023-10-18T10:41:30.434509+0000","last_became_active":"2023-10-18T10:41:29.455459+0000","last_became_peered":"2023-10-18T10:41:29.455459+0000","last_unstale":"2023-10-18T10:41:30.434509+0000","last_undegraded":"2023-10-18T10:41:30.434509+0000","last_fullsized":"2023-10-18T10:41:30.434509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:01.952901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750593+0000","last_change":"2023-10-18T10:41:28.439218+0000","last_active":"2023-10-18T10:41:30.750593+0000","last_peered":"2023-10-18T10:41:30.750593+0000","last_clean":"2023-10-18T10:41:30.750593+0000","last_became_active":"2023-10-18T10:41:28.438612+0000","last_became_peered":"2023-10-18T10:41:28.438612+0000","last_unstale":"2023-10-18T10:41:30.750593+0000","last_undegraded":"2023-10-18T10:41:30.750593+0000","last_fullsized":"2023-10-18T10:41:30.750593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:22.330220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750696+0000","last_change":"2023-10-18T10:41:29.455263+0000","last_active":"2023-10-18T10:41:30.750696+0000","last_peered":"2023-10-18T10:41:30.750696+0000","last_clean":"2023-10-18T10:41:30.750696+0000","last_became_active":"2023-10-18T10:41:29.454955+0000","last_became_peered":"2023-10-18T10:41:29.454955+0000","last_unstale":"2023-10-18T10:41:30.750696+0000","last_undegraded":"2023-10-18T10:41:30.750696+0000","last_fullsized":"2023-10-18T10:41:30.750696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:54.077461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750720+0000","last_change":"2023-10-18T10:41:28.440128+0000","last_active":"2023-10-18T10:41:30.750720+0000","last_peered":"2023-10-18T10:41:30.750720+0000","last_clean":"2023-10-18T10:41:30.750720+0000","last_became_active":"2023-10-18T10:41:28.439291+0000","last_became_peered":"2023-10-18T10:41:28.439291+0000","last_unstale":"2023-10-18T10:41:30.750720+0000","last_undegraded":"2023-10-18T10:41:30.750720+0000","last_fullsized":"2023-10-18T10:41:30.750720+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:52.966951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751017+0000","last_change":"2023-10-18T10:41:29.446112+0000","last_active":"2023-10-18T10:41:30.751017+0000","last_peered":"2023-10-18T10:41:30.751017+0000","last_clean":"2023-10-18T10:41:30.751017+0000","last_became_active":"2023-10-18T10:41:29.445569+0000","last_became_peered":"2023-10-18T10:41:29.445569+0000","last_unstale":"2023-10-18T10:41:30.751017+0000","last_undegraded":"2023-10-18T10:41:30.751017+0000","last_fullsized":"2023-10-18T10:41:30.751017+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:23.963420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750595+0000","last_change":"2023-10-18T10:41:28.442720+0000","last_active":"2023-10-18T10:41:30.750595+0000","last_peered":"2023-10-18T10:41:30.750595+0000","last_clean":"2023-10-18T10:41:30.750595+0000","last_became_active":"2023-10-18T10:41:28.442384+0000","last_became_peered":"2023-10-18T10:41:28.442384+0000","last_unstale":"2023-10-18T10:41:30.750595+0000","last_undegraded":"2023-10-18T10:41:30.750595+0000","last_fullsized":"2023-10-18T10:41:30.750595+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:48.359996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751004+0000","last_change":"2023-10-18T10:41:29.449909+0000","last_active":"2023-10-18T10:41:30.751004+0000","last_peered":"2023-10-18T10:41:30.751004+0000","last_clean":"2023-10-18T10:41:30.751004+0000","last_became_active":"2023-10-18T10:41:29.449538+0000","last_became_peered":"2023-10-18T10:41:29.449538+0000","last_unstale":"2023-10-18T10:41:30.751004+0000","last_undegraded":"2023-10-18T10:41:30.751004+0000","last_fullsized":"2023-10-18T10:41:30.751004+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:54.170399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750265+0000","last_change":"2023-10-18T10:41:28.440086+0000","last_active":"2023-10-18T10:41:30.750265+0000","last_peered":"2023-10-18T10:41:30.750265+0000","last_clean":"2023-10-18T10:41:30.750265+0000","last_became_active":"2023-10-18T10:41:28.439814+0000","last_became_peered":"2023-10-18T10:41:28.439814+0000","last_unstale":"2023-10-18T10:41:30.750265+0000","last_undegraded":"2023-10-18T10:41:30.750265+0000","last_fullsized":"2023-10-18T10:41:30.750265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:19.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"101'192","reported_seq":517,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-18T10:41:29.442308+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:41:29.442308+0000","last_peered":"2023-10-18T10:41:29.442308+0000","last_clean":"2023-10-18T10:41:29.442308+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:41:29.442308+0000","last_undegraded":"2023-10-18T10:41:29.442308+0000","last_fullsized":"2023-10-18T10:41:29.442308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575050,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59480,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312360,"statfs":{"total":96636764160,"available":96575856640,"internally_reserved":0,"allocated":1970176,"data_stored":1274279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.876,"5min":0.876,"15min":0.88300000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68200000000000005,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73199999999999998,"5min":0.84199999999999997,"15min":0.84899999999999998},"last":0.51900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.627}]},{"osd":4,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.495,"15min":0.48799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.875,"15min":0.875},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78400000000000003,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.69999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575050,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":692691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.34499999999999997,"15min":0.28699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.78500000000000003,"15min":0.82599999999999996},"last":0.61499999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.79500000000000004,"15min":0.83399999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.80900000000000005,"15min":0.82699999999999996},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55800000000000005,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65000000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.627},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.64900000000000002,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.66900000000000004,"5min":0.75600000000000001,"15min":0.84399999999999997},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.441,"15min":0.44},"min":{"1min":0.30399999999999999,"5min":0.29199999999999998,"15min":0.24199999999999999},"max":{"1min":0.76500000000000001,"5min":0.872,"15min":0.872},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45600000000000002,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65500000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48199999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82499999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.68300000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72199999999999998,"5min":0.82199999999999995,"15min":0.87},"last":0.69899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57199999999999995,"15min":0.58399999999999996},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69599999999999995,"5min":0.82499999999999996,"15min":1.016},"last":0.434},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.70099999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.628},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":1.0620000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.80100000000000005,"15min":1.262},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73099999999999998,"5min":0.96099999999999997,"15min":1.224},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":0.871,"15min":1.159},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575050,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":684015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.877,"15min":0.877},"last":0.58599999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.628},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.76900000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.53700000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.437},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.70899999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.79600000000000004},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.32600000000000001,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.82999999999999996,"5min":1.446,"15min":1.446},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.80300000000000005,"5min":0.81599999999999995,"15min":0.89700000000000002},"last":0.79600000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.85199999999999998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.88600000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.879}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35199999999999998,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.183,"15min":0.183},"max":{"1min":0.46100000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.254,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.50700000000000001,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.376,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.55000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41699999999999998,"15min":0.42999999999999999},"min":{"1min":0.29299999999999998,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47699999999999998,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.308,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.879,"15min":0.94699999999999995},"last":0.622}]},{"osd":5,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":1.002,"15min":1.002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.37},"max":{"1min":0.85499999999999998,"5min":1.018,"15min":1.018},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.63500000000000001},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.873,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.68899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":690475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.376,"15min":0.371},"min":{"1min":0.22600000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.49099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.47299999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.39200000000000002,"15min":0.40300000000000002},"min":{"1min":0.27300000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55300000000000005,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.437},{"interface":"front","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.40500000000000003},"min":{"1min":0.27400000000000002,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.24299999999999999},"max":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.92600000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.624,"15min":0.64100000000000001},"min":{"1min":0.432,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":1.458},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79300000000000004,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":1.028},"last":0.80900000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81999999999999995,"5min":0.879,"15min":0.92500000000000004},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1372160,"data_stored":683925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.432,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.38800000000000001,"15min":0.38900000000000001},"min":{"1min":0.23799999999999999,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.51100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.38800000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.40200000000000002,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":0.627,"15min":0.64800000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26000000000000001},"max":{"1min":0.495,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.26800000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52300000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.73399999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.81799999999999995,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.624,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.374},"max":{"1min":0.871,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.371},"max":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575049,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1998848,"data_stored":1283161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.35999999999999999,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.50600000000000001,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.308},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.377,"15min":0.38600000000000001},"min":{"1min":0.26000000000000001,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.48899999999999999,"5min":0.62,"15min":0.76900000000000002},"last":0.34100000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.187},"max":{"1min":0.46700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39200000000000002,"15min":0.41499999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46200000000000002,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.28799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38300000000000001,"15min":0.39000000000000001},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.41799999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.307,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51300000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0649999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34699999999999998},"max":{"1min":0.98199999999999998,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":1.026,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.439}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.339,"15min":1.339},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.628,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:41:34.404 INFO:tasks.ceph.mds.c.smithi125.stderr:2023-10-18T10:41:34.403+0000 7f90be568640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:41:35.816 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:41:35.820 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:41:36.764 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:41:36.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.877+0000 7f9c3a697640 1 -- 172.21.15.125:0/2029462161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 msgr2=0x7f9c1c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:36.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.877+0000 7f9c3a697640 1 --2- 172.21.15.125:0/2029462161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 0x7f9c1c00b980 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c001d80 tx=0x7f9c1c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:36.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.877+0000 7f9c3a697640 1 -- 172.21.15.125:0/2029462161 shutdown_connections 2023-10-18T10:41:36.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.877+0000 7f9c3a697640 1 --2- 172.21.15.125:0/2029462161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 0x7f9c1c00b980 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:36.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.877+0000 7f9c3a697640 1 -- 172.21.15.125:0/2029462161 >> 172.21.15.125:0/2029462161 conn(0x7f9c340d0d60 msgr2=0x7f9c340d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:36.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.877+0000 7f9c3a697640 1 -- 172.21.15.125:0/2029462161 shutdown_connections 2023-10-18T10:41:36.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.877+0000 7f9c3a697640 1 -- 172.21.15.125:0/2029462161 wait complete. 2023-10-18T10:41:36.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.878+0000 7f9c3a697640 1 Processor -- start 2023-10-18T10:41:36.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.878+0000 7f9c3a697640 1 -- start start 2023-10-18T10:41:36.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c3a697640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 0x7f9c341f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:36.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c3a697640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c340c8c20 0x7f9c340b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:36.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c3a697640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c340b6470 0x7f9c3406a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:36.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c3a697640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3414f8d0 con 0x7f9c1c005180 2023-10-18T10:41:36.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c3a697640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340b6900 con 0x7f9c340b6470 2023-10-18T10:41:36.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c3a697640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340c7ed0 con 0x7f9c340c8c20 2023-10-18T10:41:36.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c33fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 0x7f9c341f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:36.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c33fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 0x7f9c341f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50962/0 (socket says 172.21.15.125:50962) 2023-10-18T10:41:36.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c33fff640 1 -- 172.21.15.125:0/1671209215 learned_addr learned my addr 172.21.15.125:0/1671209215 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:36.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c337fe640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c340c8c20 0x7f9c340b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:36.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.879+0000 7f9c38c34640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c340b6470 0x7f9c3406a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:36.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c33fff640 1 -- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c340c8c20 msgr2=0x7f9c340b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:36.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c33fff640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c340c8c20 0x7f9c340b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:36.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c33fff640 1 -- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c340b6470 msgr2=0x7f9c3406a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:36.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c33fff640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c340b6470 0x7f9c3406a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:36.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c33fff640 1 -- 172.21.15.125:0/1671209215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340c8140 con 0x7f9c1c005180 2023-10-18T10:41:36.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c38c34640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c340b6470 0x7f9c3406a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:36.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c33fff640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 0x7f9c341f0ed0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9c24011ae0 tx=0x7f9c24009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:36.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c30ff9640 1 -- 172.21.15.125:0/1671209215 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24004460 con 0x7f9c1c005180 2023-10-18T10:41:36.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c30ff9640 1 -- 172.21.15.125:0/1671209215 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9c2400fa00 con 0x7f9c1c005180 2023-10-18T10:41:36.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c340c8420 con 0x7f9c1c005180 2023-10-18T10:41:36.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c30ff9640 1 -- 172.21.15.125:0/1671209215 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2400fe10 con 0x7f9c1c005180 2023-10-18T10:41:36.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.880+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c340b6b30 con 0x7f9c1c005180 2023-10-18T10:41:36.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.881+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c340690e0 con 0x7f9c1c005180 2023-10-18T10:41:36.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.881+0000 7f9c30ff9640 1 -- 172.21.15.125:0/1671209215 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9c240035f0 con 0x7f9c1c005180 2023-10-18T10:41:36.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.882+0000 7f9c30ff9640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c2006af90 0x7f9c2006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:36.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.882+0000 7f9c30ff9640 1 -- 172.21.15.125:0/1671209215 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9c2408a670 con 0x7f9c1c005180 2023-10-18T10:41:36.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.882+0000 7f9c337fe640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c2006af90 0x7f9c2006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:36.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.882+0000 7f9c337fe640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c2006af90 0x7f9c2006d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9c280017e0 tx=0x7f9c28000f10 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:36.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:36.886+0000 7f9c30ff9640 1 -- 172.21.15.125:0/1671209215 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c24055500 con 0x7f9c1c005180 2023-10-18T10:41:37.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.048+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 --> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c340caf70 con 0x7f9c2006af90 2023-10-18T10:41:37.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.056+0000 7f9c30ff9640 1 -- 172.21.15.125:0/1671209215 <== mgr.6299 v2:172.21.15.139:6832/1523403591 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289698 (secure 0 0 0) 0x7f9c340caf70 con 0x7f9c2006af90 2023-10-18T10:41:37.061 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:37.062 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:41:37.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.058+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c2006af90 msgr2=0x7f9c2006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.058+0000 7f9c3a697640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c2006af90 0x7f9c2006d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9c280017e0 tx=0x7f9c28000f10 comp rx=0 tx=0).stop 2023-10-18T10:41:37.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.058+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 msgr2=0x7f9c341f0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.058+0000 7f9c3a697640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 0x7f9c341f0ed0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9c24011ae0 tx=0x7f9c24009e40 comp rx=0 tx=0).stop 2023-10-18T10:41:37.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.058+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 shutdown_connections 2023-10-18T10:41:37.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.058+0000 7f9c3a697640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f9c2006af90 0x7f9c2006d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.058+0000 7f9c3a697640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c340b6470 0x7f9c3406a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.059+0000 7f9c3a697640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c340c8c20 0x7f9c340b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.059+0000 7f9c3a697640 1 --2- 172.21.15.125:0/1671209215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c1c005180 0x7f9c341f0ed0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.059+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 >> 172.21.15.125:0/1671209215 conn(0x7f9c340d0d60 msgr2=0x7f9c3415d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:37.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.059+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 shutdown_connections 2023-10-18T10:41:37.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.059+0000 7f9c3a697640 1 -- 172.21.15.125:0/1671209215 wait complete. 2023-10-18T10:41:37.071 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-18T10:41:35.283009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":339,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314144,"kb_used_data":11948,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754660576,"statfs":{"total":773094113280,"available":772772429824,"internally_reserved":0,"allocated":12234752,"data_stored":6676256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.862953"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434717+0000","last_change":"2023-10-18T10:41:29.447539+0000","last_active":"2023-10-18T10:41:30.434717+0000","last_peered":"2023-10-18T10:41:30.434717+0000","last_clean":"2023-10-18T10:41:30.434717+0000","last_became_active":"2023-10-18T10:41:29.447288+0000","last_became_peered":"2023-10-18T10:41:29.447288+0000","last_unstale":"2023-10-18T10:41:30.434717+0000","last_undegraded":"2023-10-18T10:41:30.434717+0000","last_fullsized":"2023-10-18T10:41:30.434717+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:30.647260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750857+0000","last_change":"2023-10-18T10:41:29.442252+0000","last_active":"2023-10-18T10:41:30.750857+0000","last_peered":"2023-10-18T10:41:30.750857+0000","last_clean":"2023-10-18T10:41:30.750857+0000","last_became_active":"2023-10-18T10:41:29.441955+0000","last_became_peered":"2023-10-18T10:41:29.441955+0000","last_unstale":"2023-10-18T10:41:30.750857+0000","last_undegraded":"2023-10-18T10:41:30.750857+0000","last_fullsized":"2023-10-18T10:41:30.750857+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:23.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434545+0000","last_change":"2023-10-18T10:41:29.452284+0000","last_active":"2023-10-18T10:41:30.434545+0000","last_peered":"2023-10-18T10:41:30.434545+0000","last_clean":"2023-10-18T10:41:30.434545+0000","last_became_active":"2023-10-18T10:41:29.452037+0000","last_became_peered":"2023-10-18T10:41:29.452037+0000","last_unstale":"2023-10-18T10:41:30.434545+0000","last_undegraded":"2023-10-18T10:41:30.434545+0000","last_fullsized":"2023-10-18T10:41:30.434545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:50.154689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750025+0000","last_change":"2023-10-18T10:41:29.453178+0000","last_active":"2023-10-18T10:41:30.750025+0000","last_peered":"2023-10-18T10:41:30.750025+0000","last_clean":"2023-10-18T10:41:30.750025+0000","last_became_active":"2023-10-18T10:41:29.452902+0000","last_became_peered":"2023-10-18T10:41:29.452902+0000","last_unstale":"2023-10-18T10:41:30.750025+0000","last_undegraded":"2023-10-18T10:41:30.750025+0000","last_fullsized":"2023-10-18T10:41:30.750025+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:53:23.594087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437984+0000","last_change":"2023-10-18T10:41:29.452729+0000","last_active":"2023-10-18T10:41:30.437984+0000","last_peered":"2023-10-18T10:41:30.437984+0000","last_clean":"2023-10-18T10:41:30.437984+0000","last_became_active":"2023-10-18T10:41:29.452415+0000","last_became_peered":"2023-10-18T10:41:29.452415+0000","last_unstale":"2023-10-18T10:41:30.437984+0000","last_undegraded":"2023-10-18T10:41:30.437984+0000","last_fullsized":"2023-10-18T10:41:30.437984+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:25:17.106887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750972+0000","last_change":"2023-10-18T10:41:29.450362+0000","last_active":"2023-10-18T10:41:30.750972+0000","last_peered":"2023-10-18T10:41:30.750972+0000","last_clean":"2023-10-18T10:41:30.750972+0000","last_became_active":"2023-10-18T10:41:29.449972+0000","last_became_peered":"2023-10-18T10:41:29.449972+0000","last_unstale":"2023-10-18T10:41:30.750972+0000","last_undegraded":"2023-10-18T10:41:30.750972+0000","last_fullsized":"2023-10-18T10:41:30.750972+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:59.437842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750535+0000","last_change":"2023-10-18T10:41:29.450173+0000","last_active":"2023-10-18T10:41:30.750535+0000","last_peered":"2023-10-18T10:41:30.750535+0000","last_clean":"2023-10-18T10:41:30.750535+0000","last_became_active":"2023-10-18T10:41:29.449859+0000","last_became_peered":"2023-10-18T10:41:29.449859+0000","last_unstale":"2023-10-18T10:41:30.750535+0000","last_undegraded":"2023-10-18T10:41:30.750535+0000","last_fullsized":"2023-10-18T10:41:30.750535+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:34.385853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749904+0000","last_change":"2023-10-18T10:41:29.449393+0000","last_active":"2023-10-18T10:41:30.749904+0000","last_peered":"2023-10-18T10:41:30.749904+0000","last_clean":"2023-10-18T10:41:30.749904+0000","last_became_active":"2023-10-18T10:41:29.449086+0000","last_became_peered":"2023-10-18T10:41:29.449086+0000","last_unstale":"2023-10-18T10:41:30.749904+0000","last_undegraded":"2023-10-18T10:41:30.749904+0000","last_fullsized":"2023-10-18T10:41:30.749904+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:41.061091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438599+0000","last_change":"2023-10-18T10:41:29.447548+0000","last_active":"2023-10-18T10:41:30.438599+0000","last_peered":"2023-10-18T10:41:30.438599+0000","last_clean":"2023-10-18T10:41:30.438599+0000","last_became_active":"2023-10-18T10:41:29.447242+0000","last_became_peered":"2023-10-18T10:41:29.447242+0000","last_unstale":"2023-10-18T10:41:30.438599+0000","last_undegraded":"2023-10-18T10:41:30.438599+0000","last_fullsized":"2023-10-18T10:41:30.438599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:41.061091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751632+0000","last_change":"2023-10-18T10:41:29.444304+0000","last_active":"2023-10-18T10:41:30.751632+0000","last_peered":"2023-10-18T10:41:30.751632+0000","last_clean":"2023-10-18T10:41:30.751632+0000","last_became_active":"2023-10-18T10:41:29.444057+0000","last_became_peered":"2023-10-18T10:41:29.444057+0000","last_unstale":"2023-10-18T10:41:30.751632+0000","last_undegraded":"2023-10-18T10:41:30.751632+0000","last_fullsized":"2023-10-18T10:41:30.751632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:23.209276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438263+0000","last_change":"2023-10-18T10:41:29.448537+0000","last_active":"2023-10-18T10:41:30.438263+0000","last_peered":"2023-10-18T10:41:30.438263+0000","last_clean":"2023-10-18T10:41:30.438263+0000","last_became_active":"2023-10-18T10:41:29.447727+0000","last_became_peered":"2023-10-18T10:41:29.447727+0000","last_unstale":"2023-10-18T10:41:30.438263+0000","last_undegraded":"2023-10-18T10:41:30.438263+0000","last_fullsized":"2023-10-18T10:41:30.438263+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:05.122562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751364+0000","last_change":"2023-10-18T10:41:29.443320+0000","last_active":"2023-10-18T10:41:30.751364+0000","last_peered":"2023-10-18T10:41:30.751364+0000","last_clean":"2023-10-18T10:41:30.751364+0000","last_became_active":"2023-10-18T10:41:29.443029+0000","last_became_peered":"2023-10-18T10:41:29.443029+0000","last_unstale":"2023-10-18T10:41:30.751364+0000","last_undegraded":"2023-10-18T10:41:30.751364+0000","last_fullsized":"2023-10-18T10:41:30.751364+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:27.802284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750628+0000","last_change":"2023-10-18T10:41:29.448605+0000","last_active":"2023-10-18T10:41:30.750628+0000","last_peered":"2023-10-18T10:41:30.750628+0000","last_clean":"2023-10-18T10:41:30.750628+0000","last_became_active":"2023-10-18T10:41:29.448348+0000","last_became_peered":"2023-10-18T10:41:29.448348+0000","last_unstale":"2023-10-18T10:41:30.750628+0000","last_undegraded":"2023-10-18T10:41:30.750628+0000","last_fullsized":"2023-10-18T10:41:30.750628+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:42.695172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437791+0000","last_change":"2023-10-18T10:41:29.455958+0000","last_active":"2023-10-18T10:41:30.437791+0000","last_peered":"2023-10-18T10:41:30.437791+0000","last_clean":"2023-10-18T10:41:30.437791+0000","last_became_active":"2023-10-18T10:41:29.455667+0000","last_became_peered":"2023-10-18T10:41:29.455667+0000","last_unstale":"2023-10-18T10:41:30.437791+0000","last_undegraded":"2023-10-18T10:41:30.437791+0000","last_fullsized":"2023-10-18T10:41:30.437791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:05.134557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750043+0000","last_change":"2023-10-18T10:41:29.444407+0000","last_active":"2023-10-18T10:41:30.750043+0000","last_peered":"2023-10-18T10:41:30.750043+0000","last_clean":"2023-10-18T10:41:30.750043+0000","last_became_active":"2023-10-18T10:41:29.444169+0000","last_became_peered":"2023-10-18T10:41:29.444169+0000","last_unstale":"2023-10-18T10:41:30.750043+0000","last_undegraded":"2023-10-18T10:41:30.750043+0000","last_fullsized":"2023-10-18T10:41:30.750043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:13.954884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750781+0000","last_change":"2023-10-18T10:41:29.454015+0000","last_active":"2023-10-18T10:41:30.750781+0000","last_peered":"2023-10-18T10:41:30.750781+0000","last_clean":"2023-10-18T10:41:30.750781+0000","last_became_active":"2023-10-18T10:41:29.451538+0000","last_became_peered":"2023-10-18T10:41:29.451538+0000","last_unstale":"2023-10-18T10:41:30.750781+0000","last_undegraded":"2023-10-18T10:41:30.750781+0000","last_fullsized":"2023-10-18T10:41:30.750781+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:52.418194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751142+0000","last_change":"2023-10-18T10:41:29.449763+0000","last_active":"2023-10-18T10:41:30.751142+0000","last_peered":"2023-10-18T10:41:30.751142+0000","last_clean":"2023-10-18T10:41:30.751142+0000","last_became_active":"2023-10-18T10:41:29.449498+0000","last_became_peered":"2023-10-18T10:41:29.449498+0000","last_unstale":"2023-10-18T10:41:30.751142+0000","last_undegraded":"2023-10-18T10:41:30.751142+0000","last_fullsized":"2023-10-18T10:41:30.751142+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:30.606377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434672+0000","last_change":"2023-10-18T10:41:29.448915+0000","last_active":"2023-10-18T10:41:30.434672+0000","last_peered":"2023-10-18T10:41:30.434672+0000","last_clean":"2023-10-18T10:41:30.434672+0000","last_became_active":"2023-10-18T10:41:29.448650+0000","last_became_peered":"2023-10-18T10:41:29.448650+0000","last_unstale":"2023-10-18T10:41:30.434672+0000","last_undegraded":"2023-10-18T10:41:30.434672+0000","last_fullsized":"2023-10-18T10:41:30.434672+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:39:41.152281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438286+0000","last_change":"2023-10-18T10:41:29.455993+0000","last_active":"2023-10-18T10:41:30.438286+0000","last_peered":"2023-10-18T10:41:30.438286+0000","last_clean":"2023-10-18T10:41:30.438286+0000","last_became_active":"2023-10-18T10:41:29.455678+0000","last_became_peered":"2023-10-18T10:41:29.455678+0000","last_unstale":"2023-10-18T10:41:30.438286+0000","last_undegraded":"2023-10-18T10:41:30.438286+0000","last_fullsized":"2023-10-18T10:41:30.438286+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:20.769610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750152+0000","last_change":"2023-10-18T10:41:29.455557+0000","last_active":"2023-10-18T10:41:30.750152+0000","last_peered":"2023-10-18T10:41:30.750152+0000","last_clean":"2023-10-18T10:41:30.750152+0000","last_became_active":"2023-10-18T10:41:29.455296+0000","last_became_peered":"2023-10-18T10:41:29.455296+0000","last_unstale":"2023-10-18T10:41:30.750152+0000","last_undegraded":"2023-10-18T10:41:30.750152+0000","last_fullsized":"2023-10-18T10:41:30.750152+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:51.749366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437980+0000","last_change":"2023-10-18T10:41:29.447863+0000","last_active":"2023-10-18T10:41:30.437980+0000","last_peered":"2023-10-18T10:41:30.437980+0000","last_clean":"2023-10-18T10:41:30.437980+0000","last_became_active":"2023-10-18T10:41:29.447353+0000","last_became_peered":"2023-10-18T10:41:29.447353+0000","last_unstale":"2023-10-18T10:41:30.437980+0000","last_undegraded":"2023-10-18T10:41:30.437980+0000","last_fullsized":"2023-10-18T10:41:30.437980+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:45.211505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437796+0000","last_change":"2023-10-18T10:41:29.455453+0000","last_active":"2023-10-18T10:41:30.437796+0000","last_peered":"2023-10-18T10:41:30.437796+0000","last_clean":"2023-10-18T10:41:30.437796+0000","last_became_active":"2023-10-18T10:41:29.455076+0000","last_became_peered":"2023-10-18T10:41:29.455076+0000","last_unstale":"2023-10-18T10:41:30.437796+0000","last_undegraded":"2023-10-18T10:41:30.437796+0000","last_fullsized":"2023-10-18T10:41:30.437796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:42.659244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750459+0000","last_change":"2023-10-18T10:41:29.453973+0000","last_active":"2023-10-18T10:41:30.750459+0000","last_peered":"2023-10-18T10:41:30.750459+0000","last_clean":"2023-10-18T10:41:30.750459+0000","last_became_active":"2023-10-18T10:41:29.453686+0000","last_became_peered":"2023-10-18T10:41:29.453686+0000","last_unstale":"2023-10-18T10:41:30.750459+0000","last_undegraded":"2023-10-18T10:41:30.750459+0000","last_fullsized":"2023-10-18T10:41:30.750459+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:05:15.082425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434493+0000","last_change":"2023-10-18T10:41:29.453702+0000","last_active":"2023-10-18T10:41:30.434493+0000","last_peered":"2023-10-18T10:41:30.434493+0000","last_clean":"2023-10-18T10:41:30.434493+0000","last_became_active":"2023-10-18T10:41:29.453445+0000","last_became_peered":"2023-10-18T10:41:29.453445+0000","last_unstale":"2023-10-18T10:41:30.434493+0000","last_undegraded":"2023-10-18T10:41:30.434493+0000","last_fullsized":"2023-10-18T10:41:30.434493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:28.572719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750696+0000","last_change":"2023-10-18T10:41:29.451267+0000","last_active":"2023-10-18T10:41:30.750696+0000","last_peered":"2023-10-18T10:41:30.750696+0000","last_clean":"2023-10-18T10:41:30.750696+0000","last_became_active":"2023-10-18T10:41:29.450963+0000","last_became_peered":"2023-10-18T10:41:29.450963+0000","last_unstale":"2023-10-18T10:41:30.750696+0000","last_undegraded":"2023-10-18T10:41:30.750696+0000","last_fullsized":"2023-10-18T10:41:30.750696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:01.499934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438439+0000","last_change":"2023-10-18T10:41:29.456047+0000","last_active":"2023-10-18T10:41:30.438439+0000","last_peered":"2023-10-18T10:41:30.438439+0000","last_clean":"2023-10-18T10:41:30.438439+0000","last_became_active":"2023-10-18T10:41:29.455742+0000","last_became_peered":"2023-10-18T10:41:29.455742+0000","last_unstale":"2023-10-18T10:41:30.438439+0000","last_undegraded":"2023-10-18T10:41:30.438439+0000","last_fullsized":"2023-10-18T10:41:30.438439+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:46:06.183343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751548+0000","last_change":"2023-10-18T10:41:29.453816+0000","last_active":"2023-10-18T10:41:30.751548+0000","last_peered":"2023-10-18T10:41:30.751548+0000","last_clean":"2023-10-18T10:41:30.751548+0000","last_became_active":"2023-10-18T10:41:29.453581+0000","last_became_peered":"2023-10-18T10:41:29.453581+0000","last_unstale":"2023-10-18T10:41:30.751548+0000","last_undegraded":"2023-10-18T10:41:30.751548+0000","last_fullsized":"2023-10-18T10:41:30.751548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:18.669429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750180+0000","last_change":"2023-10-18T10:41:29.454500+0000","last_active":"2023-10-18T10:41:30.750180+0000","last_peered":"2023-10-18T10:41:30.750180+0000","last_clean":"2023-10-18T10:41:30.750180+0000","last_became_active":"2023-10-18T10:41:29.454231+0000","last_became_peered":"2023-10-18T10:41:29.454231+0000","last_unstale":"2023-10-18T10:41:30.750180+0000","last_undegraded":"2023-10-18T10:41:30.750180+0000","last_fullsized":"2023-10-18T10:41:30.750180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:14.224881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751332+0000","last_change":"2023-10-18T10:41:29.453392+0000","last_active":"2023-10-18T10:41:30.751332+0000","last_peered":"2023-10-18T10:41:30.751332+0000","last_clean":"2023-10-18T10:41:30.751332+0000","last_became_active":"2023-10-18T10:41:29.453089+0000","last_became_peered":"2023-10-18T10:41:29.453089+0000","last_unstale":"2023-10-18T10:41:30.751332+0000","last_undegraded":"2023-10-18T10:41:30.751332+0000","last_fullsized":"2023-10-18T10:41:30.751332+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:52.483119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437981+0000","last_change":"2023-10-18T10:41:29.448083+0000","last_active":"2023-10-18T10:41:30.437981+0000","last_peered":"2023-10-18T10:41:30.437981+0000","last_clean":"2023-10-18T10:41:30.437981+0000","last_became_active":"2023-10-18T10:41:29.447584+0000","last_became_peered":"2023-10-18T10:41:29.447584+0000","last_unstale":"2023-10-18T10:41:30.437981+0000","last_undegraded":"2023-10-18T10:41:30.437981+0000","last_fullsized":"2023-10-18T10:41:30.437981+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:43.081792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750335+0000","last_change":"2023-10-18T10:41:29.453993+0000","last_active":"2023-10-18T10:41:30.750335+0000","last_peered":"2023-10-18T10:41:30.750335+0000","last_clean":"2023-10-18T10:41:30.750335+0000","last_became_active":"2023-10-18T10:41:29.453659+0000","last_became_peered":"2023-10-18T10:41:29.453659+0000","last_unstale":"2023-10-18T10:41:30.750335+0000","last_undegraded":"2023-10-18T10:41:30.750335+0000","last_fullsized":"2023-10-18T10:41:30.750335+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:50.051564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750971+0000","last_change":"2023-10-18T10:41:29.445142+0000","last_active":"2023-10-18T10:41:30.750971+0000","last_peered":"2023-10-18T10:41:30.750971+0000","last_clean":"2023-10-18T10:41:30.750971+0000","last_became_active":"2023-10-18T10:41:29.444871+0000","last_became_peered":"2023-10-18T10:41:29.444871+0000","last_unstale":"2023-10-18T10:41:30.750971+0000","last_undegraded":"2023-10-18T10:41:30.750971+0000","last_fullsized":"2023-10-18T10:41:30.750971+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:31:23.805032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750791+0000","last_change":"2023-10-18T10:41:29.448568+0000","last_active":"2023-10-18T10:41:30.750791+0000","last_peered":"2023-10-18T10:41:30.750791+0000","last_clean":"2023-10-18T10:41:30.750791+0000","last_became_active":"2023-10-18T10:41:29.448318+0000","last_became_peered":"2023-10-18T10:41:29.448318+0000","last_unstale":"2023-10-18T10:41:30.750791+0000","last_undegraded":"2023-10-18T10:41:30.750791+0000","last_fullsized":"2023-10-18T10:41:30.750791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:29.589441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"111'4","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751169+0000","last_change":"2023-10-18T10:41:28.435122+0000","last_active":"2023-10-18T10:41:30.751169+0000","last_peered":"2023-10-18T10:41:30.751169+0000","last_clean":"2023-10-18T10:41:30.751169+0000","last_became_active":"2023-10-18T10:41:28.434844+0000","last_became_peered":"2023-10-18T10:41:28.434844+0000","last_unstale":"2023-10-18T10:41:30.751169+0000","last_undegraded":"2023-10-18T10:41:30.751169+0000","last_fullsized":"2023-10-18T10:41:30.751169+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:04.834294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751302+0000","last_change":"2023-10-18T10:41:29.445971+0000","last_active":"2023-10-18T10:41:30.751302+0000","last_peered":"2023-10-18T10:41:30.751302+0000","last_clean":"2023-10-18T10:41:30.751302+0000","last_became_active":"2023-10-18T10:41:29.445723+0000","last_became_peered":"2023-10-18T10:41:29.445723+0000","last_unstale":"2023-10-18T10:41:30.751302+0000","last_undegraded":"2023-10-18T10:41:30.751302+0000","last_fullsized":"2023-10-18T10:41:30.751302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:35:45.397768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438376+0000","last_change":"2023-10-18T10:41:28.436698+0000","last_active":"2023-10-18T10:41:30.438376+0000","last_peered":"2023-10-18T10:41:30.438376+0000","last_clean":"2023-10-18T10:41:30.438376+0000","last_became_active":"2023-10-18T10:41:28.436411+0000","last_became_peered":"2023-10-18T10:41:28.436411+0000","last_unstale":"2023-10-18T10:41:30.438376+0000","last_undegraded":"2023-10-18T10:41:30.438376+0000","last_fullsized":"2023-10-18T10:41:30.438376+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:06.613711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751070+0000","last_change":"2023-10-18T10:41:29.457858+0000","last_active":"2023-10-18T10:41:30.751070+0000","last_peered":"2023-10-18T10:41:30.751070+0000","last_clean":"2023-10-18T10:41:30.751070+0000","last_became_active":"2023-10-18T10:41:29.457555+0000","last_became_peered":"2023-10-18T10:41:29.457555+0000","last_unstale":"2023-10-18T10:41:30.751070+0000","last_undegraded":"2023-10-18T10:41:30.751070+0000","last_fullsized":"2023-10-18T10:41:30.751070+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:12.165512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750907+0000","last_change":"2023-10-18T10:41:28.432593+0000","last_active":"2023-10-18T10:41:30.750907+0000","last_peered":"2023-10-18T10:41:30.750907+0000","last_clean":"2023-10-18T10:41:30.750907+0000","last_became_active":"2023-10-18T10:41:28.432296+0000","last_became_peered":"2023-10-18T10:41:28.432296+0000","last_unstale":"2023-10-18T10:41:30.750907+0000","last_undegraded":"2023-10-18T10:41:30.750907+0000","last_fullsized":"2023-10-18T10:41:30.750907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:27.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751225+0000","last_change":"2023-10-18T10:41:29.451021+0000","last_active":"2023-10-18T10:41:30.751225+0000","last_peered":"2023-10-18T10:41:30.751225+0000","last_clean":"2023-10-18T10:41:30.751225+0000","last_became_active":"2023-10-18T10:41:29.450779+0000","last_became_peered":"2023-10-18T10:41:29.450779+0000","last_unstale":"2023-10-18T10:41:30.751225+0000","last_undegraded":"2023-10-18T10:41:30.751225+0000","last_fullsized":"2023-10-18T10:41:30.751225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:36.701370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751378+0000","last_change":"2023-10-18T10:41:28.429164+0000","last_active":"2023-10-18T10:41:30.751378+0000","last_peered":"2023-10-18T10:41:30.751378+0000","last_clean":"2023-10-18T10:41:30.751378+0000","last_became_active":"2023-10-18T10:41:28.428856+0000","last_became_peered":"2023-10-18T10:41:28.428856+0000","last_unstale":"2023-10-18T10:41:30.751378+0000","last_undegraded":"2023-10-18T10:41:30.751378+0000","last_fullsized":"2023-10-18T10:41:30.751378+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:55:18.343120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749949+0000","last_change":"2023-10-18T10:41:29.443511+0000","last_active":"2023-10-18T10:41:30.749949+0000","last_peered":"2023-10-18T10:41:30.749949+0000","last_clean":"2023-10-18T10:41:30.749949+0000","last_became_active":"2023-10-18T10:41:29.443232+0000","last_became_peered":"2023-10-18T10:41:29.443232+0000","last_unstale":"2023-10-18T10:41:30.749949+0000","last_undegraded":"2023-10-18T10:41:30.749949+0000","last_fullsized":"2023-10-18T10:41:30.749949+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:29.869647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438137+0000","last_change":"2023-10-18T10:41:28.441255+0000","last_active":"2023-10-18T10:41:30.438137+0000","last_peered":"2023-10-18T10:41:30.438137+0000","last_clean":"2023-10-18T10:41:30.438137+0000","last_became_active":"2023-10-18T10:41:28.440983+0000","last_became_peered":"2023-10-18T10:41:28.440983+0000","last_unstale":"2023-10-18T10:41:30.438137+0000","last_undegraded":"2023-10-18T10:41:30.438137+0000","last_fullsized":"2023-10-18T10:41:30.438137+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:34.041596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750604+0000","last_change":"2023-10-18T10:41:29.448123+0000","last_active":"2023-10-18T10:41:30.750604+0000","last_peered":"2023-10-18T10:41:30.750604+0000","last_clean":"2023-10-18T10:41:30.750604+0000","last_became_active":"2023-10-18T10:41:29.447856+0000","last_became_peered":"2023-10-18T10:41:29.447856+0000","last_unstale":"2023-10-18T10:41:30.750604+0000","last_undegraded":"2023-10-18T10:41:30.750604+0000","last_fullsized":"2023-10-18T10:41:30.750604+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:52.661375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750830+0000","last_change":"2023-10-18T10:41:28.439965+0000","last_active":"2023-10-18T10:41:30.750830+0000","last_peered":"2023-10-18T10:41:30.750830+0000","last_clean":"2023-10-18T10:41:30.750830+0000","last_became_active":"2023-10-18T10:41:28.439700+0000","last_became_peered":"2023-10-18T10:41:28.439700+0000","last_unstale":"2023-10-18T10:41:30.750830+0000","last_undegraded":"2023-10-18T10:41:30.750830+0000","last_fullsized":"2023-10-18T10:41:30.750830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:44.202065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750979+0000","last_change":"2023-10-18T10:41:29.450500+0000","last_active":"2023-10-18T10:41:30.750979+0000","last_peered":"2023-10-18T10:41:30.750979+0000","last_clean":"2023-10-18T10:41:30.750979+0000","last_became_active":"2023-10-18T10:41:29.450234+0000","last_became_peered":"2023-10-18T10:41:29.450234+0000","last_unstale":"2023-10-18T10:41:30.750979+0000","last_undegraded":"2023-10-18T10:41:30.750979+0000","last_fullsized":"2023-10-18T10:41:30.750979+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:08.866019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434561+0000","last_change":"2023-10-18T10:41:28.435875+0000","last_active":"2023-10-18T10:41:30.434561+0000","last_peered":"2023-10-18T10:41:30.434561+0000","last_clean":"2023-10-18T10:41:30.434561+0000","last_became_active":"2023-10-18T10:41:28.435623+0000","last_became_peered":"2023-10-18T10:41:28.435623+0000","last_unstale":"2023-10-18T10:41:30.434561+0000","last_undegraded":"2023-10-18T10:41:30.434561+0000","last_fullsized":"2023-10-18T10:41:30.434561+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:22.897591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750655+0000","last_change":"2023-10-18T10:41:29.451882+0000","last_active":"2023-10-18T10:41:30.750655+0000","last_peered":"2023-10-18T10:41:30.750655+0000","last_clean":"2023-10-18T10:41:30.750655+0000","last_became_active":"2023-10-18T10:41:29.451623+0000","last_became_peered":"2023-10-18T10:41:29.451623+0000","last_unstale":"2023-10-18T10:41:30.750655+0000","last_undegraded":"2023-10-18T10:41:30.750655+0000","last_fullsized":"2023-10-18T10:41:30.750655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:39.862682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750687+0000","last_change":"2023-10-18T10:41:28.436040+0000","last_active":"2023-10-18T10:41:30.750687+0000","last_peered":"2023-10-18T10:41:30.750687+0000","last_clean":"2023-10-18T10:41:30.750687+0000","last_became_active":"2023-10-18T10:41:28.434223+0000","last_became_peered":"2023-10-18T10:41:28.434223+0000","last_unstale":"2023-10-18T10:41:30.750687+0000","last_undegraded":"2023-10-18T10:41:30.750687+0000","last_fullsized":"2023-10-18T10:41:30.750687+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.620461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438253+0000","last_change":"2023-10-18T10:41:29.452229+0000","last_active":"2023-10-18T10:41:30.438253+0000","last_peered":"2023-10-18T10:41:30.438253+0000","last_clean":"2023-10-18T10:41:30.438253+0000","last_became_active":"2023-10-18T10:41:29.451930+0000","last_became_peered":"2023-10-18T10:41:29.451930+0000","last_unstale":"2023-10-18T10:41:30.438253+0000","last_undegraded":"2023-10-18T10:41:30.438253+0000","last_fullsized":"2023-10-18T10:41:30.438253+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:12.643803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750926+0000","last_change":"2023-10-18T10:41:28.436855+0000","last_active":"2023-10-18T10:41:30.750926+0000","last_peered":"2023-10-18T10:41:30.750926+0000","last_clean":"2023-10-18T10:41:30.750926+0000","last_became_active":"2023-10-18T10:41:28.436585+0000","last_became_peered":"2023-10-18T10:41:28.436585+0000","last_unstale":"2023-10-18T10:41:30.750926+0000","last_undegraded":"2023-10-18T10:41:30.750926+0000","last_fullsized":"2023-10-18T10:41:30.750926+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.155833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750812+0000","last_change":"2023-10-18T10:41:29.450388+0000","last_active":"2023-10-18T10:41:30.750812+0000","last_peered":"2023-10-18T10:41:30.750812+0000","last_clean":"2023-10-18T10:41:30.750812+0000","last_became_active":"2023-10-18T10:41:29.450092+0000","last_became_peered":"2023-10-18T10:41:29.450092+0000","last_unstale":"2023-10-18T10:41:30.750812+0000","last_undegraded":"2023-10-18T10:41:30.750812+0000","last_fullsized":"2023-10-18T10:41:30.750812+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:02:27.937026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434722+0000","last_change":"2023-10-18T10:41:28.440969+0000","last_active":"2023-10-18T10:41:30.434722+0000","last_peered":"2023-10-18T10:41:30.434722+0000","last_clean":"2023-10-18T10:41:30.434722+0000","last_became_active":"2023-10-18T10:41:28.440739+0000","last_became_peered":"2023-10-18T10:41:28.440739+0000","last_unstale":"2023-10-18T10:41:30.434722+0000","last_undegraded":"2023-10-18T10:41:30.434722+0000","last_fullsized":"2023-10-18T10:41:30.434722+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.620461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434696+0000","last_change":"2023-10-18T10:41:29.456187+0000","last_active":"2023-10-18T10:41:30.434696+0000","last_peered":"2023-10-18T10:41:30.434696+0000","last_clean":"2023-10-18T10:41:30.434696+0000","last_became_active":"2023-10-18T10:41:29.455945+0000","last_became_peered":"2023-10-18T10:41:29.455945+0000","last_unstale":"2023-10-18T10:41:30.434696+0000","last_undegraded":"2023-10-18T10:41:30.434696+0000","last_fullsized":"2023-10-18T10:41:30.434696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:40.835626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"111'3","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438389+0000","last_change":"2023-10-18T10:41:28.440316+0000","last_active":"2023-10-18T10:41:30.438389+0000","last_peered":"2023-10-18T10:41:30.438389+0000","last_clean":"2023-10-18T10:41:30.438389+0000","last_became_active":"2023-10-18T10:41:28.440024+0000","last_became_peered":"2023-10-18T10:41:28.440024+0000","last_unstale":"2023-10-18T10:41:30.438389+0000","last_undegraded":"2023-10-18T10:41:30.438389+0000","last_fullsized":"2023-10-18T10:41:30.438389+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:51.651935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750493+0000","last_change":"2023-10-18T10:41:29.449408+0000","last_active":"2023-10-18T10:41:30.750493+0000","last_peered":"2023-10-18T10:41:30.750493+0000","last_clean":"2023-10-18T10:41:30.750493+0000","last_became_active":"2023-10-18T10:41:29.449161+0000","last_became_peered":"2023-10-18T10:41:29.449161+0000","last_unstale":"2023-10-18T10:41:30.750493+0000","last_undegraded":"2023-10-18T10:41:30.750493+0000","last_fullsized":"2023-10-18T10:41:30.750493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:41.742410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"111'6","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751094+0000","last_change":"2023-10-18T10:41:28.427403+0000","last_active":"2023-10-18T10:41:30.751094+0000","last_peered":"2023-10-18T10:41:30.751094+0000","last_clean":"2023-10-18T10:41:30.751094+0000","last_became_active":"2023-10-18T10:41:28.427119+0000","last_became_peered":"2023-10-18T10:41:28.427119+0000","last_unstale":"2023-10-18T10:41:30.751094+0000","last_undegraded":"2023-10-18T10:41:30.751094+0000","last_fullsized":"2023-10-18T10:41:30.751094+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:11.156072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750239+0000","last_change":"2023-10-18T10:41:29.448792+0000","last_active":"2023-10-18T10:41:30.750239+0000","last_peered":"2023-10-18T10:41:30.750239+0000","last_clean":"2023-10-18T10:41:30.750239+0000","last_became_active":"2023-10-18T10:41:29.448536+0000","last_became_peered":"2023-10-18T10:41:29.448536+0000","last_unstale":"2023-10-18T10:41:30.750239+0000","last_undegraded":"2023-10-18T10:41:30.750239+0000","last_fullsized":"2023-10-18T10:41:30.750239+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:14.300602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751307+0000","last_change":"2023-10-18T10:41:28.434350+0000","last_active":"2023-10-18T10:41:30.751307+0000","last_peered":"2023-10-18T10:41:30.751307+0000","last_clean":"2023-10-18T10:41:30.751307+0000","last_became_active":"2023-10-18T10:41:28.434068+0000","last_became_peered":"2023-10-18T10:41:28.434068+0000","last_unstale":"2023-10-18T10:41:30.751307+0000","last_undegraded":"2023-10-18T10:41:30.751307+0000","last_fullsized":"2023-10-18T10:41:30.751307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:39.826186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.437865+0000","last_change":"2023-10-18T10:41:29.447478+0000","last_active":"2023-10-18T10:41:30.437865+0000","last_peered":"2023-10-18T10:41:30.437865+0000","last_clean":"2023-10-18T10:41:30.437865+0000","last_became_active":"2023-10-18T10:41:29.447163+0000","last_became_peered":"2023-10-18T10:41:29.447163+0000","last_unstale":"2023-10-18T10:41:30.437865+0000","last_undegraded":"2023-10-18T10:41:30.437865+0000","last_fullsized":"2023-10-18T10:41:30.437865+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:27.997549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751060+0000","last_change":"2023-10-18T10:41:28.433856+0000","last_active":"2023-10-18T10:41:30.751060+0000","last_peered":"2023-10-18T10:41:30.751060+0000","last_clean":"2023-10-18T10:41:30.751060+0000","last_became_active":"2023-10-18T10:41:28.433584+0000","last_became_peered":"2023-10-18T10:41:28.433584+0000","last_unstale":"2023-10-18T10:41:30.751060+0000","last_undegraded":"2023-10-18T10:41:30.751060+0000","last_fullsized":"2023-10-18T10:41:30.751060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:06:07.806734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750178+0000","last_change":"2023-10-18T10:41:29.439852+0000","last_active":"2023-10-18T10:41:30.750178+0000","last_peered":"2023-10-18T10:41:30.750178+0000","last_clean":"2023-10-18T10:41:30.750178+0000","last_became_active":"2023-10-18T10:41:29.439612+0000","last_became_peered":"2023-10-18T10:41:29.439612+0000","last_unstale":"2023-10-18T10:41:30.750178+0000","last_undegraded":"2023-10-18T10:41:30.750178+0000","last_fullsized":"2023-10-18T10:41:30.750178+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:10.074146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750194+0000","last_change":"2023-10-18T10:41:28.431914+0000","last_active":"2023-10-18T10:41:30.750194+0000","last_peered":"2023-10-18T10:41:30.750194+0000","last_clean":"2023-10-18T10:41:30.750194+0000","last_became_active":"2023-10-18T10:41:28.431664+0000","last_became_peered":"2023-10-18T10:41:28.431664+0000","last_unstale":"2023-10-18T10:41:30.750194+0000","last_undegraded":"2023-10-18T10:41:30.750194+0000","last_fullsized":"2023-10-18T10:41:30.750194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:41.649804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751367+0000","last_change":"2023-10-18T10:41:29.453041+0000","last_active":"2023-10-18T10:41:30.751367+0000","last_peered":"2023-10-18T10:41:30.751367+0000","last_clean":"2023-10-18T10:41:30.751367+0000","last_became_active":"2023-10-18T10:41:29.452788+0000","last_became_peered":"2023-10-18T10:41:29.452788+0000","last_unstale":"2023-10-18T10:41:30.751367+0000","last_undegraded":"2023-10-18T10:41:30.751367+0000","last_fullsized":"2023-10-18T10:41:30.751367+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:14:28.740818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"111'8","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750045+0000","last_change":"2023-10-18T10:41:28.440009+0000","last_active":"2023-10-18T10:41:30.750045+0000","last_peered":"2023-10-18T10:41:30.750045+0000","last_clean":"2023-10-18T10:41:30.750045+0000","last_became_active":"2023-10-18T10:41:28.439742+0000","last_became_peered":"2023-10-18T10:41:28.439742+0000","last_unstale":"2023-10-18T10:41:30.750045+0000","last_undegraded":"2023-10-18T10:41:30.750045+0000","last_fullsized":"2023-10-18T10:41:30.750045+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:42.072352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751005+0000","last_change":"2023-10-18T10:41:29.448022+0000","last_active":"2023-10-18T10:41:30.751005+0000","last_peered":"2023-10-18T10:41:30.751005+0000","last_clean":"2023-10-18T10:41:30.751005+0000","last_became_active":"2023-10-18T10:41:29.447771+0000","last_became_peered":"2023-10-18T10:41:29.447771+0000","last_unstale":"2023-10-18T10:41:30.751005+0000","last_undegraded":"2023-10-18T10:41:30.751005+0000","last_fullsized":"2023-10-18T10:41:30.751005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:13.373299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751383+0000","last_change":"2023-10-18T10:41:28.430372+0000","last_active":"2023-10-18T10:41:30.751383+0000","last_peered":"2023-10-18T10:41:30.751383+0000","last_clean":"2023-10-18T10:41:30.751383+0000","last_became_active":"2023-10-18T10:41:28.430110+0000","last_became_peered":"2023-10-18T10:41:28.430110+0000","last_unstale":"2023-10-18T10:41:30.751383+0000","last_undegraded":"2023-10-18T10:41:30.751383+0000","last_fullsized":"2023-10-18T10:41:30.751383+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.826001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750892+0000","last_change":"2023-10-18T10:41:29.449385+0000","last_active":"2023-10-18T10:41:30.750892+0000","last_peered":"2023-10-18T10:41:30.750892+0000","last_clean":"2023-10-18T10:41:30.750892+0000","last_became_active":"2023-10-18T10:41:29.449126+0000","last_became_peered":"2023-10-18T10:41:29.449126+0000","last_unstale":"2023-10-18T10:41:30.750892+0000","last_undegraded":"2023-10-18T10:41:30.750892+0000","last_fullsized":"2023-10-18T10:41:30.750892+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:20.124373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751437+0000","last_change":"2023-10-18T10:41:28.430749+0000","last_active":"2023-10-18T10:41:30.751437+0000","last_peered":"2023-10-18T10:41:30.751437+0000","last_clean":"2023-10-18T10:41:30.751437+0000","last_became_active":"2023-10-18T10:41:28.430495+0000","last_became_peered":"2023-10-18T10:41:28.430495+0000","last_unstale":"2023-10-18T10:41:30.751437+0000","last_undegraded":"2023-10-18T10:41:30.751437+0000","last_fullsized":"2023-10-18T10:41:30.751437+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:00.943461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438254+0000","last_change":"2023-10-18T10:41:29.447633+0000","last_active":"2023-10-18T10:41:30.438254+0000","last_peered":"2023-10-18T10:41:30.438254+0000","last_clean":"2023-10-18T10:41:30.438254+0000","last_became_active":"2023-10-18T10:41:29.446778+0000","last_became_peered":"2023-10-18T10:41:29.446778+0000","last_unstale":"2023-10-18T10:41:30.438254+0000","last_undegraded":"2023-10-18T10:41:30.438254+0000","last_fullsized":"2023-10-18T10:41:30.438254+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:48:55.670658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"111'1","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438448+0000","last_change":"2023-10-18T10:41:28.438164+0000","last_active":"2023-10-18T10:41:30.438448+0000","last_peered":"2023-10-18T10:41:30.438448+0000","last_clean":"2023-10-18T10:41:30.438448+0000","last_became_active":"2023-10-18T10:41:28.437873+0000","last_became_peered":"2023-10-18T10:41:28.437873+0000","last_unstale":"2023-10-18T10:41:30.438448+0000","last_undegraded":"2023-10-18T10:41:30.438448+0000","last_fullsized":"2023-10-18T10:41:30.438448+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:53.068021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751109+0000","last_change":"2023-10-18T10:41:29.450342+0000","last_active":"2023-10-18T10:41:30.751109+0000","last_peered":"2023-10-18T10:41:30.751109+0000","last_clean":"2023-10-18T10:41:30.751109+0000","last_became_active":"2023-10-18T10:41:29.450078+0000","last_became_peered":"2023-10-18T10:41:29.450078+0000","last_unstale":"2023-10-18T10:41:30.751109+0000","last_undegraded":"2023-10-18T10:41:30.751109+0000","last_fullsized":"2023-10-18T10:41:30.751109+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:32.976432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750276+0000","last_change":"2023-10-18T10:41:28.438968+0000","last_active":"2023-10-18T10:41:30.750276+0000","last_peered":"2023-10-18T10:41:30.750276+0000","last_clean":"2023-10-18T10:41:30.750276+0000","last_became_active":"2023-10-18T10:41:28.438628+0000","last_became_peered":"2023-10-18T10:41:28.438628+0000","last_unstale":"2023-10-18T10:41:30.750276+0000","last_undegraded":"2023-10-18T10:41:30.750276+0000","last_fullsized":"2023-10-18T10:41:30.750276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:45:51.950983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750427+0000","last_change":"2023-10-18T10:41:29.442931+0000","last_active":"2023-10-18T10:41:30.750427+0000","last_peered":"2023-10-18T10:41:30.750427+0000","last_clean":"2023-10-18T10:41:30.750427+0000","last_became_active":"2023-10-18T10:41:29.442573+0000","last_became_peered":"2023-10-18T10:41:29.442573+0000","last_unstale":"2023-10-18T10:41:30.750427+0000","last_undegraded":"2023-10-18T10:41:30.750427+0000","last_fullsized":"2023-10-18T10:41:30.750427+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:36.667760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750311+0000","last_change":"2023-10-18T10:41:28.437156+0000","last_active":"2023-10-18T10:41:30.750311+0000","last_peered":"2023-10-18T10:41:30.750311+0000","last_clean":"2023-10-18T10:41:30.750311+0000","last_became_active":"2023-10-18T10:41:28.434657+0000","last_became_peered":"2023-10-18T10:41:28.434657+0000","last_unstale":"2023-10-18T10:41:30.750311+0000","last_undegraded":"2023-10-18T10:41:30.750311+0000","last_fullsized":"2023-10-18T10:41:30.750311+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:38:04.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750603+0000","last_change":"2023-10-18T10:41:29.452874+0000","last_active":"2023-10-18T10:41:30.750603+0000","last_peered":"2023-10-18T10:41:30.750603+0000","last_clean":"2023-10-18T10:41:30.750603+0000","last_became_active":"2023-10-18T10:41:29.452561+0000","last_became_peered":"2023-10-18T10:41:29.452561+0000","last_unstale":"2023-10-18T10:41:30.750603+0000","last_undegraded":"2023-10-18T10:41:30.750603+0000","last_fullsized":"2023-10-18T10:41:30.750603+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:26.000624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750422+0000","last_change":"2023-10-18T10:41:28.435576+0000","last_active":"2023-10-18T10:41:30.750422+0000","last_peered":"2023-10-18T10:41:30.750422+0000","last_clean":"2023-10-18T10:41:30.750422+0000","last_became_active":"2023-10-18T10:41:28.435286+0000","last_became_peered":"2023-10-18T10:41:28.435286+0000","last_unstale":"2023-10-18T10:41:30.750422+0000","last_undegraded":"2023-10-18T10:41:30.750422+0000","last_fullsized":"2023-10-18T10:41:30.750422+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:29.719996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750887+0000","last_change":"2023-10-18T10:41:29.446250+0000","last_active":"2023-10-18T10:41:30.750887+0000","last_peered":"2023-10-18T10:41:30.750887+0000","last_clean":"2023-10-18T10:41:30.750887+0000","last_became_active":"2023-10-18T10:41:29.445881+0000","last_became_peered":"2023-10-18T10:41:29.445881+0000","last_unstale":"2023-10-18T10:41:30.750887+0000","last_undegraded":"2023-10-18T10:41:30.750887+0000","last_fullsized":"2023-10-18T10:41:30.750887+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:28.061711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750340+0000","last_change":"2023-10-18T10:41:28.438325+0000","last_active":"2023-10-18T10:41:30.750340+0000","last_peered":"2023-10-18T10:41:30.750340+0000","last_clean":"2023-10-18T10:41:30.750340+0000","last_became_active":"2023-10-18T10:41:28.438056+0000","last_became_peered":"2023-10-18T10:41:28.438056+0000","last_unstale":"2023-10-18T10:41:30.750340+0000","last_undegraded":"2023-10-18T10:41:30.750340+0000","last_fullsized":"2023-10-18T10:41:30.750340+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:42.738454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751354+0000","last_change":"2023-10-18T10:41:29.449925+0000","last_active":"2023-10-18T10:41:30.751354+0000","last_peered":"2023-10-18T10:41:30.751354+0000","last_clean":"2023-10-18T10:41:30.751354+0000","last_became_active":"2023-10-18T10:41:29.449632+0000","last_became_peered":"2023-10-18T10:41:29.449632+0000","last_unstale":"2023-10-18T10:41:30.751354+0000","last_undegraded":"2023-10-18T10:41:30.751354+0000","last_fullsized":"2023-10-18T10:41:30.751354+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:35.051036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"111'2","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751184+0000","last_change":"2023-10-18T10:41:28.428975+0000","last_active":"2023-10-18T10:41:30.751184+0000","last_peered":"2023-10-18T10:41:30.751184+0000","last_clean":"2023-10-18T10:41:30.751184+0000","last_became_active":"2023-10-18T10:41:28.428703+0000","last_became_peered":"2023-10-18T10:41:28.428703+0000","last_unstale":"2023-10-18T10:41:30.751184+0000","last_undegraded":"2023-10-18T10:41:30.751184+0000","last_fullsized":"2023-10-18T10:41:30.751184+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:44:52.305622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751473+0000","last_change":"2023-10-18T10:41:29.449610+0000","last_active":"2023-10-18T10:41:30.751473+0000","last_peered":"2023-10-18T10:41:30.751473+0000","last_clean":"2023-10-18T10:41:30.751473+0000","last_became_active":"2023-10-18T10:41:29.449354+0000","last_became_peered":"2023-10-18T10:41:29.449354+0000","last_unstale":"2023-10-18T10:41:30.751473+0000","last_undegraded":"2023-10-18T10:41:30.751473+0000","last_fullsized":"2023-10-18T10:41:30.751473+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:01.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438483+0000","last_change":"2023-10-18T10:41:28.430655+0000","last_active":"2023-10-18T10:41:30.438483+0000","last_peered":"2023-10-18T10:41:30.438483+0000","last_clean":"2023-10-18T10:41:30.438483+0000","last_became_active":"2023-10-18T10:41:28.430389+0000","last_became_peered":"2023-10-18T10:41:28.430389+0000","last_unstale":"2023-10-18T10:41:30.438483+0000","last_undegraded":"2023-10-18T10:41:30.438483+0000","last_fullsized":"2023-10-18T10:41:30.438483+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:05:14.072985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751349+0000","last_change":"2023-10-18T10:41:29.447215+0000","last_active":"2023-10-18T10:41:30.751349+0000","last_peered":"2023-10-18T10:41:30.751349+0000","last_clean":"2023-10-18T10:41:30.751349+0000","last_became_active":"2023-10-18T10:41:29.446983+0000","last_became_peered":"2023-10-18T10:41:29.446983+0000","last_unstale":"2023-10-18T10:41:30.751349+0000","last_undegraded":"2023-10-18T10:41:30.751349+0000","last_fullsized":"2023-10-18T10:41:30.751349+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.438513+0000","last_change":"2023-10-18T10:41:28.439870+0000","last_active":"2023-10-18T10:41:30.438513+0000","last_peered":"2023-10-18T10:41:30.438513+0000","last_clean":"2023-10-18T10:41:30.438513+0000","last_became_active":"2023-10-18T10:41:28.439587+0000","last_became_peered":"2023-10-18T10:41:28.439587+0000","last_unstale":"2023-10-18T10:41:30.438513+0000","last_undegraded":"2023-10-18T10:41:30.438513+0000","last_fullsized":"2023-10-18T10:41:30.438513+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:19.114933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750837+0000","last_change":"2023-10-18T10:41:29.448872+0000","last_active":"2023-10-18T10:41:30.750837+0000","last_peered":"2023-10-18T10:41:30.750837+0000","last_clean":"2023-10-18T10:41:30.750837+0000","last_became_active":"2023-10-18T10:41:29.448574+0000","last_became_peered":"2023-10-18T10:41:29.448574+0000","last_unstale":"2023-10-18T10:41:30.750837+0000","last_undegraded":"2023-10-18T10:41:30.750837+0000","last_fullsized":"2023-10-18T10:41:30.750837+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:40.934129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751444+0000","last_change":"2023-10-18T10:41:28.439553+0000","last_active":"2023-10-18T10:41:30.751444+0000","last_peered":"2023-10-18T10:41:30.751444+0000","last_clean":"2023-10-18T10:41:30.751444+0000","last_became_active":"2023-10-18T10:41:28.439237+0000","last_became_peered":"2023-10-18T10:41:28.439237+0000","last_unstale":"2023-10-18T10:41:30.751444+0000","last_undegraded":"2023-10-18T10:41:30.751444+0000","last_fullsized":"2023-10-18T10:41:30.751444+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:10.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.749895+0000","last_change":"2023-10-18T10:41:29.442254+0000","last_active":"2023-10-18T10:41:30.749895+0000","last_peered":"2023-10-18T10:41:30.749895+0000","last_clean":"2023-10-18T10:41:30.749895+0000","last_became_active":"2023-10-18T10:41:29.441995+0000","last_became_peered":"2023-10-18T10:41:29.441995+0000","last_unstale":"2023-10-18T10:41:30.749895+0000","last_undegraded":"2023-10-18T10:41:30.749895+0000","last_fullsized":"2023-10-18T10:41:30.749895+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:13.960239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751093+0000","last_change":"2023-10-18T10:41:28.434524+0000","last_active":"2023-10-18T10:41:30.751093+0000","last_peered":"2023-10-18T10:41:30.751093+0000","last_clean":"2023-10-18T10:41:30.751093+0000","last_became_active":"2023-10-18T10:41:28.434260+0000","last_became_peered":"2023-10-18T10:41:28.434260+0000","last_unstale":"2023-10-18T10:41:30.751093+0000","last_undegraded":"2023-10-18T10:41:30.751093+0000","last_fullsized":"2023-10-18T10:41:30.751093+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:26:18.323130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.434509+0000","last_change":"2023-10-18T10:41:29.455708+0000","last_active":"2023-10-18T10:41:30.434509+0000","last_peered":"2023-10-18T10:41:30.434509+0000","last_clean":"2023-10-18T10:41:30.434509+0000","last_became_active":"2023-10-18T10:41:29.455459+0000","last_became_peered":"2023-10-18T10:41:29.455459+0000","last_unstale":"2023-10-18T10:41:30.434509+0000","last_undegraded":"2023-10-18T10:41:30.434509+0000","last_fullsized":"2023-10-18T10:41:30.434509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:01.952901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750593+0000","last_change":"2023-10-18T10:41:28.439218+0000","last_active":"2023-10-18T10:41:30.750593+0000","last_peered":"2023-10-18T10:41:30.750593+0000","last_clean":"2023-10-18T10:41:30.750593+0000","last_became_active":"2023-10-18T10:41:28.438612+0000","last_became_peered":"2023-10-18T10:41:28.438612+0000","last_unstale":"2023-10-18T10:41:30.750593+0000","last_undegraded":"2023-10-18T10:41:30.750593+0000","last_fullsized":"2023-10-18T10:41:30.750593+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:22.330220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750696+0000","last_change":"2023-10-18T10:41:29.455263+0000","last_active":"2023-10-18T10:41:30.750696+0000","last_peered":"2023-10-18T10:41:30.750696+0000","last_clean":"2023-10-18T10:41:30.750696+0000","last_became_active":"2023-10-18T10:41:29.454955+0000","last_became_peered":"2023-10-18T10:41:29.454955+0000","last_unstale":"2023-10-18T10:41:30.750696+0000","last_undegraded":"2023-10-18T10:41:30.750696+0000","last_fullsized":"2023-10-18T10:41:30.750696+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:10:54.077461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750720+0000","last_change":"2023-10-18T10:41:28.440128+0000","last_active":"2023-10-18T10:41:30.750720+0000","last_peered":"2023-10-18T10:41:30.750720+0000","last_clean":"2023-10-18T10:41:30.750720+0000","last_became_active":"2023-10-18T10:41:28.439291+0000","last_became_peered":"2023-10-18T10:41:28.439291+0000","last_unstale":"2023-10-18T10:41:30.750720+0000","last_undegraded":"2023-10-18T10:41:30.750720+0000","last_fullsized":"2023-10-18T10:41:30.750720+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:52.966951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751017+0000","last_change":"2023-10-18T10:41:29.446112+0000","last_active":"2023-10-18T10:41:30.751017+0000","last_peered":"2023-10-18T10:41:30.751017+0000","last_clean":"2023-10-18T10:41:30.751017+0000","last_became_active":"2023-10-18T10:41:29.445569+0000","last_became_peered":"2023-10-18T10:41:29.445569+0000","last_unstale":"2023-10-18T10:41:30.751017+0000","last_undegraded":"2023-10-18T10:41:30.751017+0000","last_fullsized":"2023-10-18T10:41:30.751017+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:23.963420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750595+0000","last_change":"2023-10-18T10:41:28.442720+0000","last_active":"2023-10-18T10:41:30.750595+0000","last_peered":"2023-10-18T10:41:30.750595+0000","last_clean":"2023-10-18T10:41:30.750595+0000","last_became_active":"2023-10-18T10:41:28.442384+0000","last_became_peered":"2023-10-18T10:41:28.442384+0000","last_unstale":"2023-10-18T10:41:30.750595+0000","last_undegraded":"2023-10-18T10:41:30.750595+0000","last_fullsized":"2023-10-18T10:41:30.750595+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:48.359996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.751004+0000","last_change":"2023-10-18T10:41:29.449909+0000","last_active":"2023-10-18T10:41:30.751004+0000","last_peered":"2023-10-18T10:41:30.751004+0000","last_clean":"2023-10-18T10:41:30.751004+0000","last_became_active":"2023-10-18T10:41:29.449538+0000","last_became_peered":"2023-10-18T10:41:29.449538+0000","last_unstale":"2023-10-18T10:41:30.751004+0000","last_undegraded":"2023-10-18T10:41:30.751004+0000","last_fullsized":"2023-10-18T10:41:30.751004+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:28.414311+0000","last_clean_scrub_stamp":"2023-10-18T10:41:28.414311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:54.170399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-18T10:41:30.750265+0000","last_change":"2023-10-18T10:41:28.440086+0000","last_active":"2023-10-18T10:41:30.750265+0000","last_peered":"2023-10-18T10:41:30.750265+0000","last_clean":"2023-10-18T10:41:30.750265+0000","last_became_active":"2023-10-18T10:41:28.439814+0000","last_became_peered":"2023-10-18T10:41:28.439814+0000","last_unstale":"2023-10-18T10:41:30.750265+0000","last_undegraded":"2023-10-18T10:41:30.750265+0000","last_fullsized":"2023-10-18T10:41:30.750265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:41:27.404871+0000","last_clean_scrub_stamp":"2023-10-18T10:41:27.404871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:19.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"101'192","reported_seq":517,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-18T10:41:29.442308+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:41:29.442308+0000","last_peered":"2023-10-18T10:41:29.442308+0000","last_clean":"2023-10-18T10:41:29.442308+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:41:29.442308+0000","last_undegraded":"2023-10-18T10:41:29.442308+0000","last_fullsized":"2023-10-18T10:41:29.442308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":770,"num_read_kb":1527,"num_write":295,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575050,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59480,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312360,"statfs":{"total":96636764160,"available":96575856640,"internally_reserved":0,"allocated":1970176,"data_stored":1274279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56999999999999995,"15min":0.58199999999999996},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.876,"5min":0.876,"15min":0.88300000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57599999999999996,"15min":0.58399999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.85799999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68200000000000005,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73199999999999998,"5min":0.84199999999999997,"15min":0.84899999999999998},"last":0.51900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.753,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.627}]},{"osd":4,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.71699999999999997,"15min":0.76500000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":0.86199999999999999,"15min":0.91500000000000004},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.495,"15min":0.48799999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.875,"15min":0.875},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78400000000000003,"5min":0.91700000000000004,"15min":0.95399999999999996},"last":0.69999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575050,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1400832,"data_stored":692691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.34499999999999997,"15min":0.28699999999999998},"max":{"1min":0.70999999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.78500000000000003,"15min":0.82599999999999996},"last":0.61499999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.79500000000000004,"15min":0.83399999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57899999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.80900000000000005,"15min":0.82699999999999996},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.55800000000000005,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65000000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.627},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.64900000000000002,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.66900000000000004,"5min":0.75600000000000001,"15min":0.84399999999999997},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.441,"15min":0.44},"min":{"1min":0.30399999999999999,"5min":0.29199999999999998,"15min":0.24199999999999999},"max":{"1min":0.76500000000000001,"5min":0.872,"15min":0.872},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.80300000000000005,"15min":0.84999999999999998},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45600000000000002,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65500000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.48299999999999998,"15min":0.48199999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82499999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:41:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.68300000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72199999999999998,"5min":0.82199999999999995,"15min":0.87},"last":0.69899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57199999999999995,"15min":0.58399999999999996},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69599999999999995,"5min":0.82499999999999996,"15min":1.016},"last":0.434},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71099999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.70099999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.628},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74199999999999999,"5min":0.79100000000000004,"15min":1.0620000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":0.90000000000000002,"15min":1.1240000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.80100000000000005,"15min":1.262},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.83299999999999996,"15min":1.1899999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73099999999999998,"5min":0.96099999999999997,"15min":1.224},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":0.871,"15min":1.159},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575050,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":684015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.877,"15min":0.877},"last":0.58599999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.628},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.34300000000000003,"15min":0.29499999999999998},"max":{"1min":0.76900000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.53700000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.437},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.873,"15min":0.873},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.70899999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":0.79100000000000004,"15min":0.82399999999999995},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.79600000000000004},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.32600000000000001,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.82999999999999996,"5min":1.446,"15min":1.446},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.80300000000000005,"5min":0.81599999999999995,"15min":0.89700000000000002},"last":0.79600000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.85199999999999998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.88600000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.879}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":683855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35199999999999998,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.183,"15min":0.183},"max":{"1min":0.46100000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.254,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.50700000000000001,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.376,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.55000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41699999999999998,"15min":0.42999999999999999},"min":{"1min":0.29299999999999998,"5min":0.25900000000000001,"15min":0.19900000000000001},"max":{"1min":0.47699999999999998,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.438,"15min":0.44700000000000001},"min":{"1min":0.308,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.879,"15min":0.94699999999999995},"last":0.622}]},{"osd":5,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":1.002,"15min":1.002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.37},"max":{"1min":0.85499999999999998,"5min":1.018,"15min":1.018},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.63500000000000001},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.38300000000000001},"max":{"1min":0.81899999999999995,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.873,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.68899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54736,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317104,"statfs":{"total":96636764160,"available":96580714496,"internally_reserved":0,"allocated":1380352,"data_stored":690475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.376,"15min":0.371},"min":{"1min":0.22600000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.49099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.47299999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.39200000000000002,"15min":0.40300000000000002},"min":{"1min":0.27300000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55300000000000005,"5min":0.64800000000000002,"15min":0.82199999999999995},"last":0.437},{"interface":"front","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.40500000000000003},"min":{"1min":0.27400000000000002,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.55100000000000005,"5min":0.63600000000000001,"15min":0.77300000000000002},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.24299999999999999},"max":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.92600000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.624,"15min":0.64100000000000001},"min":{"1min":0.432,"5min":0.38500000000000001,"15min":0.254},"max":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":1.458},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.441,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59499999999999997,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79300000000000004,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":1.028},"last":0.80900000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80600000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81999999999999995,"5min":0.879,"15min":0.92500000000000004},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1372160,"data_stored":683925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.29099999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.432,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.38800000000000001,"15min":0.38900000000000001},"min":{"1min":0.23799999999999999,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.51100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.38800000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.40200000000000002,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":0.627,"15min":0.64800000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.42599999999999999,"15min":0.42899999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26000000000000001},"max":{"1min":0.495,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.26800000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52300000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.73399999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.81799999999999995,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.624,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.374},"max":{"1min":0.871,"5min":0.90700000000000003,"15min":0.95999999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:41:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.92400000000000004,"15min":0.93500000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.371},"max":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575049,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1998848,"data_stored":1283161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.35999999999999999,"15min":0.375},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.50600000000000001,"5min":0.65900000000000003,"15min":0.71099999999999997},"last":0.308},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.377,"15min":0.38600000000000001},"min":{"1min":0.26000000000000001,"5min":0.19800000000000001,"15min":0.17799999999999999},"max":{"1min":0.48899999999999999,"5min":0.62,"15min":0.76900000000000002},"last":0.34100000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.248,"15min":0.187},"max":{"1min":0.46700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39200000000000002,"15min":0.41499999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46200000000000002,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.28799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.38300000000000001,"15min":0.39000000000000001},"min":{"1min":0.27500000000000002,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":0.41799999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.307,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51300000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59399999999999997,"15min":0.61299999999999999},"min":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.376,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0649999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34699999999999998},"max":{"1min":0.98199999999999998,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":1.026,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.439}]},{"osd":7,"last update":"Wed Oct 18 10:41:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":1.339,"15min":1.339},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.628,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:41:37.077 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:41:37.077 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:41:37.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.218+0000 7f6434c9b640 1 -- 172.21.15.125:0/199243563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 msgr2=0x7f643006bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.218+0000 7f6434c9b640 1 --2- 172.21.15.125:0/199243563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 0x7f643006bb60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642002c050 comp rx=0 tx=0).stop 2023-10-18T10:41:37.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.218+0000 7f6434c9b640 1 -- 172.21.15.125:0/199243563 shutdown_connections 2023-10-18T10:41:37.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.218+0000 7f6434c9b640 1 --2- 172.21.15.125:0/199243563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 0x7f643006bb60 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.218+0000 7f6434c9b640 1 -- 172.21.15.125:0/199243563 >> 172.21.15.125:0/199243563 conn(0x7f6430069930 msgr2=0x7f6430069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:37.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.218+0000 7f6434c9b640 1 -- 172.21.15.125:0/199243563 shutdown_connections 2023-10-18T10:41:37.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.218+0000 7f6434c9b640 1 -- 172.21.15.125:0/199243563 wait complete. 2023-10-18T10:41:37.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.219+0000 7f6434c9b640 1 Processor -- start 2023-10-18T10:41:37.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.219+0000 7f6434c9b640 1 -- start start 2023-10-18T10:41:37.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f6434c9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6430070af0 0x7f64301303b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:37.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f6434c9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 0x7f6430136ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:37.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f6434c9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6430141690 0x7f643013f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:37.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f6434c9b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6430132020 con 0x7f643013e520 2023-10-18T10:41:37.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f6434c9b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6430131410 con 0x7f6430141690 2023-10-18T10:41:37.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f6434c9b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64301316f0 con 0x7f6430070af0 2023-10-18T10:41:37.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f642dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 0x7f6430136ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:37.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f642dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 0x7f6430136ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50974/0 (socket says 172.21.15.125:50974) 2023-10-18T10:41:37.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.220+0000 7f642dd9b640 1 -- 172.21.15.125:0/1799322424 learned_addr learned my addr 172.21.15.125:0/1799322424 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:37.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642e59c640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6430070af0 0x7f64301303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:37.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642ed9d640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6430141690 0x7f643013f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:37.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642dd9b640 1 -- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6430070af0 msgr2=0x7f64301303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642dd9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6430070af0 0x7f64301303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642dd9b640 1 -- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6430141690 msgr2=0x7f643013f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642dd9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6430141690 0x7f643013f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642dd9b640 1 -- 172.21.15.125:0/1799322424 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6420009840 con 0x7f643013e520 2023-10-18T10:41:37.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642ed9d640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6430141690 0x7f643013f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:37.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642e59c640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6430070af0 0x7f64301303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:37.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f642dd9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 0x7f6430136ae0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f642400b3e0 tx=0x7f642400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:37.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f641effd640 1 -- 172.21.15.125:0/1799322424 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424011830 con 0x7f643013e520 2023-10-18T10:41:37.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f641effd640 1 -- 172.21.15.125:0/1799322424 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6424011aa0 con 0x7f643013e520 2023-10-18T10:41:37.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f641effd640 1 -- 172.21.15.125:0/1799322424 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424010380 con 0x7f643013e520 2023-10-18T10:41:37.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64301319d0 con 0x7f643013e520 2023-10-18T10:41:37.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.221+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64301308f0 con 0x7f643013e520 2023-10-18T10:41:37.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.222+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64300b1610 con 0x7f643013e520 2023-10-18T10:41:37.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.222+0000 7f641effd640 1 -- 172.21.15.125:0/1799322424 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f64240104e0 con 0x7f643013e520 2023-10-18T10:41:37.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.223+0000 7f641effd640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f641006af60 0x7f641006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:37.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.223+0000 7f641effd640 1 -- 172.21.15.125:0/1799322424 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6424089cc0 con 0x7f643013e520 2023-10-18T10:41:37.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.223+0000 7f642e59c640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f641006af60 0x7f641006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:37.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.224+0000 7f642e59c640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f641006af60 0x7f641006d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6420035000 tx=0x7f6420004ab0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:37.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.227+0000 7f641effd640 1 -- 172.21.15.125:0/1799322424 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6424054b20 con 0x7f643013e520 2023-10-18T10:41:37.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.460+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f64300690e0 con 0x7f643013e520 2023-10-18T10:41:37.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.463+0000 7f641effd640 1 -- 172.21.15.125:0/1799322424 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7f64240150e0 con 0x7f643013e520 2023-10-18T10:41:37.465 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:41:37.465 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:41:37.465 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T10:41:37.466 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T10:41:37.466 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:41:37.467 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T10:41:37.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f641006af60 msgr2=0x7f641006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f641006af60 0x7f641006d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6420035000 tx=0x7f6420004ab0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 msgr2=0x7f6430136ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 0x7f6430136ae0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f642400b3e0 tx=0x7f642400b8a0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 shutdown_connections 2023-10-18T10:41:37.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f641006af60 0x7f641006d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6430141690 0x7f643013f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f643013e520 0x7f6430136ae0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 --2- 172.21.15.125:0/1799322424 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6430070af0 0x7f64301303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 >> 172.21.15.125:0/1799322424 conn(0x7f6430069930 msgr2=0x7f643013b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:37.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 shutdown_connections 2023-10-18T10:41:37.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.465+0000 7f6434c9b640 1 -- 172.21.15.125:0/1799322424 wait complete. 2023-10-18T10:41:37.477 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:41:37.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.578+0000 7f9961d6c640 1 -- 172.21.15.125:0/4144406483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0afa20 msgr2=0x7f995c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.578+0000 7f9961d6c640 1 --2- 172.21.15.125:0/4144406483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0afa20 0x7f995c0705a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f995000a940 tx=0x7f995002d090 comp rx=0 tx=0).stop 2023-10-18T10:41:37.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.578+0000 7f9961d6c640 1 -- 172.21.15.125:0/4144406483 shutdown_connections 2023-10-18T10:41:37.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.578+0000 7f9961d6c640 1 --2- 172.21.15.125:0/4144406483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0afa20 0x7f995c0705a0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.578+0000 7f9961d6c640 1 --2- 172.21.15.125:0/4144406483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f995c0cbd20 0x7f995c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.578+0000 7f9961d6c640 1 -- 172.21.15.125:0/4144406483 >> 172.21.15.125:0/4144406483 conn(0x7f995c069930 msgr2=0x7f995c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:37.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.578+0000 7f9961d6c640 1 -- 172.21.15.125:0/4144406483 shutdown_connections 2023-10-18T10:41:37.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.578+0000 7f9961d6c640 1 -- 172.21.15.125:0/4144406483 wait complete. 2023-10-18T10:41:37.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.579+0000 7f9961d6c640 1 Processor -- start 2023-10-18T10:41:37.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.579+0000 7f9961d6c640 1 -- start start 2023-10-18T10:41:37.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f9961d6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f995c0afa20 0x7f995c141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:37.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f9961d6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0b0530 0x7f995c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:37.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f9961d6c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f995c0cbd20 0x7f995c155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:37.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f9961d6c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c155f70 con 0x7f995c0b0530 2023-10-18T10:41:37.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f9961d6c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c156250 con 0x7f995c0cbd20 2023-10-18T10:41:37.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f9961d6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c156530 con 0x7f995c0afa20 2023-10-18T10:41:37.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0b0530 0x7f995c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:37.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0b0530 0x7f995c1553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52132/0 (socket says 172.21.15.125:52132) 2023-10-18T10:41:37.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995affd640 1 -- 172.21.15.125:0/1824409594 learned_addr learned my addr 172.21.15.125:0/1824409594 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:37.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995b7fe640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f995c0afa20 0x7f995c141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:37.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995bfff640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f995c0cbd20 0x7f995c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:37.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995affd640 1 -- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f995c0afa20 msgr2=0x7f995c141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995affd640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f995c0afa20 0x7f995c141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995affd640 1 -- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f995c0cbd20 msgr2=0x7f995c155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995affd640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f995c0cbd20 0x7f995c155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995affd640 1 -- 172.21.15.125:0/1824409594 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995000a5a0 con 0x7f995c0b0530 2023-10-18T10:41:37.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.580+0000 7f995b7fe640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f995c0afa20 0x7f995c141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:41:37.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.581+0000 7f995affd640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0b0530 0x7f995c1553f0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f995000aa60 tx=0x7f9950036cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:37.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.581+0000 7f9943fff640 1 -- 172.21.15.125:0/1824409594 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950005220 con 0x7f995c0b0530 2023-10-18T10:41:37.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.581+0000 7f9943fff640 1 -- 172.21.15.125:0/1824409594 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9950005470 con 0x7f995c0b0530 2023-10-18T10:41:37.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.581+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c156810 con 0x7f995c0b0530 2023-10-18T10:41:37.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.581+0000 7f9943fff640 1 -- 172.21.15.125:0/1824409594 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950003a60 con 0x7f995c0b0530 2023-10-18T10:41:37.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.581+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c148760 con 0x7f995c0b0530 2023-10-18T10:41:37.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.582+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995c156d30 con 0x7f995c0b0530 2023-10-18T10:41:37.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.582+0000 7f9943fff640 1 -- 172.21.15.125:0/1824409594 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f995003c040 con 0x7f995c0b0530 2023-10-18T10:41:37.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.582+0000 7f9943fff640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f994806af60 0x7f994806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:37.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.583+0000 7f9943fff640 1 -- 172.21.15.125:0/1824409594 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f99500ac400 con 0x7f995c0b0530 2023-10-18T10:41:37.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.583+0000 7f995b7fe640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f994806af60 0x7f994806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:37.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.583+0000 7f995b7fe640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f994806af60 0x7f994806d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f994c009aa0 tx=0x7f994c005db0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:37.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.587+0000 7f9943fff640 1 -- 172.21.15.125:0/1824409594 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f995c156d30 con 0x7f995c0b0530 2023-10-18T10:41:37.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.787+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f995c156d30 con 0x7f995c0b0530 2023-10-18T10:41:37.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.788+0000 7f9943fff640 1 -- 172.21.15.125:0/1824409594 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f995c156d30 con 0x7f995c0b0530 2023-10-18T10:41:37.790 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:37.791 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":41,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2381546668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2381546668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2381546668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/193356301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":193356301},{"type":"v1","addr":"172.21.15.139:6836","nonce":193356301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":8073,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3589330474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3589330474},{"type":"v1","addr":"172.21.15.125:6834","nonce":3589330474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:41:28.760907+0000","modified":"2023-10-18T10:41:30.774086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8076":{"gid":8076,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/3342443842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3342443842},{"type":"v1","addr":"172.21.15.125:6835","nonce":3342443842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T10:41:37.791 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 41 2023-10-18T10:41:37.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f994806af60 msgr2=0x7f994806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f994806af60 0x7f994806d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f994c009aa0 tx=0x7f994c005db0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0b0530 msgr2=0x7f995c1553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:37.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0b0530 0x7f995c1553f0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f995000aa60 tx=0x7f9950036cd0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 shutdown_connections 2023-10-18T10:41:37.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f994806af60 0x7f994806d420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f995c0cbd20 0x7f995c155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f995c0b0530 0x7f995c1553f0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 --2- 172.21.15.125:0/1824409594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f995c0afa20 0x7f995c141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:37.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 >> 172.21.15.125:0/1824409594 conn(0x7f995c069930 msgr2=0x7f995c15a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:37.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 shutdown_connections 2023-10-18T10:41:37.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:37.790+0000 7f9961d6c640 1 -- 172.21.15.125:0/1824409594 wait complete. 2023-10-18T10:41:38.204 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:41:28.760907+0000', 'modified': '2023-10-18T10:41:30.774086+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8076}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8076': {'gid': 8076, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/3342443842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 3342443842}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 3342443842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:41:38.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:41:38.205 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T10:41:38.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.308+0000 7f3d05765640 1 -- 172.21.15.125:0/2802286338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cec003970 msgr2=0x7f3cec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.308+0000 7f3d05765640 1 --2- 172.21.15.125:0/2802286338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cec003970 0x7f3cec003d90 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009b80 tx=0x7f3cec0057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.308+0000 7f3cff7fe640 1 -- 172.21.15.125:0/2802286338 reap_dead start 2023-10-18T10:41:38.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.308+0000 7f3d05765640 1 -- 172.21.15.125:0/2802286338 shutdown_connections 2023-10-18T10:41:38.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.308+0000 7f3d05765640 1 -- 172.21.15.125:0/2802286338 >> 172.21.15.125:0/2802286338 conn(0x7f3d00069930 msgr2=0x7f3d00069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:38.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.308+0000 7f3d05765640 1 -- 172.21.15.125:0/2802286338 shutdown_connections 2023-10-18T10:41:38.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.308+0000 7f3d05765640 1 -- 172.21.15.125:0/2802286338 wait complete. 2023-10-18T10:41:38.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.309+0000 7f3d05765640 1 Processor -- start 2023-10-18T10:41:38.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.309+0000 7f3d05765640 1 -- start start 2023-10-18T10:41:38.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3d05765640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec003970 0x7f3d00145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3d05765640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d001f2510 0x7f3d001f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3d05765640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d001f2ed0 0x7f3d001eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3d05765640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00145ba0 con 0x7f3d001f2ed0 2023-10-18T10:41:38.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3d05765640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d001f3350 con 0x7f3cec003970 2023-10-18T10:41:38.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3d05765640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00133f80 con 0x7f3d001f2510 2023-10-18T10:41:38.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cff7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d001f2ed0 0x7f3d001eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cff7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d001f2ed0 0x7f3d001eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52146/0 (socket says 172.21.15.125:52146) 2023-10-18T10:41:38.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cff7fe640 1 -- 172.21.15.125:0/4237307265 learned_addr learned my addr 172.21.15.125:0/4237307265 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:38.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec003970 0x7f3d00145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d001f2510 0x7f3d001f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cff7fe640 1 -- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d001f2510 msgr2=0x7f3d001f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d001f2510 0x7f3d001f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cff7fe640 1 -- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec003970 msgr2=0x7f3d00145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec003970 0x7f3d00145660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cff7fe640 1 -- 172.21.15.125:0/4237307265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0015b0c0 con 0x7f3d001f2ed0 2023-10-18T10:41:38.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.310+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec003970 0x7f3d00145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:38.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.311+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d001f2ed0 0x7f3d001eeee0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8001330 tx=0x7f3cf800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:38.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.311+0000 7f3cf77fe640 1 -- 172.21.15.125:0/4237307265 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800ca20 con 0x7f3d001f2ed0 2023-10-18T10:41:38.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.311+0000 7f3cf77fe640 1 -- 172.21.15.125:0/4237307265 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3cf800cc90 con 0x7f3d001f2ed0 2023-10-18T10:41:38.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.311+0000 7f3d05765640 1 -- 172.21.15.125:0/4237307265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0015b3a0 con 0x7f3d001f2ed0 2023-10-18T10:41:38.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.311+0000 7f3cf77fe640 1 -- 172.21.15.125:0/4237307265 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8011460 con 0x7f3d001f2ed0 2023-10-18T10:41:38.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.311+0000 7f3d05765640 1 -- 172.21.15.125:0/4237307265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0015b960 con 0x7f3d001f2ed0 2023-10-18T10:41:38.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.312+0000 7f3cf77fe640 1 -- 172.21.15.125:0/4237307265 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3cf80115c0 con 0x7f3d001f2ed0 2023-10-18T10:41:38.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.312+0000 7f3cf77fe640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f3ce006af90 0x7f3ce006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.313+0000 7f3cf77fe640 1 -- 172.21.15.125:0/4237307265 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3cf808a0f0 con 0x7f3d001f2ed0 2023-10-18T10:41:38.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.313+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f3ce006af90 0x7f3ce006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.313+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f3ce006af90 0x7f3ce006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0036000 tx=0x7f3cf0035040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.313+0000 7f3d083cc740 1 Processor -- start 2023-10-18T10:41:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.314+0000 7f3d083cc740 1 -- start start 2023-10-18T10:41:38.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.314+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee671ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.314+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.314+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee672020 0x562eee6a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.314+0000 7f3d083cc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eee583730 con 0x562eee69ef20 2023-10-18T10:41:38.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.314+0000 7f3d083cc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eee5e4780 con 0x562eee69eb40 2023-10-18T10:41:38.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.314+0000 7f3d083cc740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eee5e1e70 con 0x562eee672020 2023-10-18T10:41:38.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cfeffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee671ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52158/0 (socket says 172.21.15.125:52158) 2023-10-18T10:41:38.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 -- 172.21.15.125:0/258301100 learned_addr learned my addr 172.21.15.125:0/258301100 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:38.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee672020 0x562eee6a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 -- 172.21.15.125:0/258301100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee672020 msgr2=0x562eee6a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:38.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee672020 0x562eee6a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 -- 172.21.15.125:0/258301100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 msgr2=0x562eee671ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:38.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee671ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 -- 172.21.15.125:0/258301100 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eee5e23d0 con 0x562eee69ef20 2023-10-18T10:41:38.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee671ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:41:38.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a18d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009b20 tx=0x7f3ce8013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a214c8df7d403a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:38.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf4ff9640 1 -- 172.21.15.125:0/258301100 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8005220 con 0x562eee69ef20 2023-10-18T10:41:38.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf4ff9640 1 -- 172.21.15.125:0/258301100 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ce8005380 con 0x562eee69ef20 2023-10-18T10:41:38.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.315+0000 7f3cf4ff9640 1 -- 172.21.15.125:0/258301100 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce80047a0 con 0x562eee69ef20 2023-10-18T10:41:38.330 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 -- 172.21.15.125:0/258301100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 msgr2=0x562eee6a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a18d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009b20 tx=0x7f3ce8013460 comp rx=0 tx=0).stop 2023-10-18T10:41:38.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 -- 172.21.15.125:0/258301100 shutdown_connections 2023-10-18T10:41:38.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee672020 0x562eee6a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a18d0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 --2- 172.21.15.125:0/258301100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee671ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 -- 172.21.15.125:0/258301100 >> 172.21.15.125:0/258301100 conn(0x562eee673d50 msgr2=0x562eee6bc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:38.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 -- 172.21.15.125:0/258301100 shutdown_connections 2023-10-18T10:41:38.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 -- 172.21.15.125:0/258301100 wait complete. 2023-10-18T10:41:38.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 -- start start 2023-10-18T10:41:38.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.316+0000 7f3d083cc740 1 Processor -- start 2023-10-18T10:41:38.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee692fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3d083cc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eee5dd760 con 0x562eee69ef20 2023-10-18T10:41:38.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3d083cc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eee5988f0 con 0x562eee69eb40 2023-10-18T10:41:38.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3d083cc740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eee598590 con 0x562eee6a4810 2023-10-18T10:41:38.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3d083cc740 10 client.0 _ll_register_callbacks cb 0x562eee5d4ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:41:38.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3cf7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3cf7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52166/0 (socket says 172.21.15.125:52166) 2023-10-18T10:41:38.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3cfeffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee692fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3cf7fff640 1 -- 172.21.15.125:0/2433584687 learned_addr learned my addr 172.21.15.125:0/2433584687 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:38.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.317+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cf7fff640 1 -- 172.21.15.125:0/2433584687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 msgr2=0x562eee6b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:38.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cf7fff640 1 -- 172.21.15.125:0/2433584687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 msgr2=0x562eee692fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:38.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee692fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cf7fff640 1 -- 172.21.15.125:0/2433584687 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce8009720 con 0x562eee69ef20 2023-10-18T10:41:38.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee692fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:41:38.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a42d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000c00 tx=0x7f3ce800f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123208575cbb841c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:38.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cddffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:41:38.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cddffb640 1 -- 172.21.15.125:0/2433584687 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8004030 con 0x562eee69ef20 2023-10-18T10:41:38.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cddffb640 1 -- 172.21.15.125:0/2433584687 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3ce8004190 con 0x562eee69ef20 2023-10-18T10:41:38.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 1 -- 172.21.15.125:0/2433584687 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eee5d6740 con 0x562eee69ef20 2023-10-18T10:41:38.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cddffb640 1 -- 172.21.15.125:0/2433584687 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce8004390 con 0x562eee69ef20 2023-10-18T10:41:38.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 10 client.8241 fetch_fsmap learned FSMap version 41 2023-10-18T10:41:38.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 1 -- 172.21.15.125:0/2433584687 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x562eee5dbde0 con 0x562eee69ef20 2023-10-18T10:41:38.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cddffb640 1 -- 172.21.15.125:0/2433584687 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x562eee5d6740 con 0x562eee69ef20 2023-10-18T10:41:38.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3cddffb640 1 -- 172.21.15.125:0/2433584687 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3ce802b060 con 0x562eee69ef20 2023-10-18T10:41:38.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 10 client.8241 fetch_fsmap finished waiting for FSMap version 41 2023-10-18T10:41:38.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 10 client.8241 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T10:41:38.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 20 client.8241 populate_metadata read hostname 'smithi125' 2023-10-18T10:41:38.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] conn(0x562eee6b4a10 0x562eee6b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 4 client.8241 mds_command: new command op to 8076 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:41:38.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.318+0000 7f3d083cc740 1 -- 172.21.15.125:0/2433584687 --> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eee5d6460 con 0x562eee6b4a10 2023-10-18T10:41:38.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.319+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] conn(0x562eee6b4a10 0x562eee6b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.319+0000 7f3cddffb640 10 client.8241 ms_handle_connect on v2:172.21.15.125:6833/3342443842 2023-10-18T10:41:38.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.319+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] conn(0x562eee6b4a10 0x562eee6b4df0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e585039b695a204 server_cookie=1982f89f6515d86d in_seq=0 out_seq=0 2023-10-18T10:41:38.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.319+0000 7f3cddffb640 1 -- 172.21.15.125:0/2433584687 <== mds.0 v2:172.21.15.125:6833/3342443842 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562eee5d6460 con 0x562eee6b4a10 2023-10-18T10:41:38.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.319+0000 7f3cddffb640 10 client.8241 handle_command_reply: tid=0 2023-10-18T10:41:38.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.319+0000 7f3d083cc740 1 client.8241 shutdown 2023-10-18T10:41:38.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.319+0000 7f3d083cc740 1 -- 172.21.15.125:0/2433584687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 msgr2=0x562eee6a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.319+0000 7f3d083cc740 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a42d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000c00 tx=0x7f3ce800f830 comp rx=0 tx=0).stop 2023-10-18T10:41:38.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 1 -- 172.21.15.125:0/2433584687 shutdown_connections 2023-10-18T10:41:38.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] conn(0x562eee6b4a10 0x562eee6b4df0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee6a42d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3cddffb640 0 client.8241 ms_handle_reset on v2:172.21.15.125:6833/3342443842 2023-10-18T10:41:38.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 1 --2- 172.21.15.125:0/2433584687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee692fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 1 -- 172.21.15.125:0/2433584687 >> 172.21.15.125:0/2433584687 conn(0x562eee672020 msgr2=0x562eee6759c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:38.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 1 -- 172.21.15.125:0/2433584687 shutdown_connections 2023-10-18T10:41:38.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 1 -- 172.21.15.125:0/2433584687 wait complete. 2023-10-18T10:41:38.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.320+0000 7f3d083cc740 20 client.8241 trim_cache size 0 max 16384 2023-10-18T10:41:38.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.335+0000 7f3d083cc740 1 Processor -- start 2023-10-18T10:41:38.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.335+0000 7f3d083cc740 1 -- start start 2023-10-18T10:41:38.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.335+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee69eb40 0x562eee66d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.335+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69ef20 0x562eee693150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.335+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee6a4810 0x562eee6a4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.335+0000 7f3d083cc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eee55e270 con 0x562eee6a4810 2023-10-18T10:41:38.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.335+0000 7f3d083cc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eee6a4550 con 0x562eee69ef20 2023-10-18T10:41:38.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.335+0000 7f3d083cc740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eee6710c0 con 0x562eee69eb40 2023-10-18T10:41:38.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee6a4810 0x562eee6a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee6a4810 0x562eee6a4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52172/0 (socket says 172.21.15.125:52172) 2023-10-18T10:41:38.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3145401076 learned_addr learned my addr 172.21.15.125:0/3145401076 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:38.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee69eb40 0x562eee66d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69ef20 0x562eee693150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee69eb40 msgr2=0x562eee66d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:38.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee69eb40 0x562eee66d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69ef20 msgr2=0x562eee693150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:38.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69ef20 0x562eee693150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 -- 172.21.15.125:0/3145401076 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eee57ad00 con 0x562eee6a4810 2023-10-18T10:41:38.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee69eb40 0x562eee66d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:38.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee6a4810 0x562eee6a4010 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8058ad0 tx=0x7f3cf801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d80b44ea3a048d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:38.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cf57fa640 1 -- 172.21.15.125:0/3145401076 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80880c0 con 0x562eee6a4810 2023-10-18T10:41:38.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cf57fa640 1 -- 172.21.15.125:0/3145401076 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3cf8088600 con 0x562eee6a4810 2023-10-18T10:41:38.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.336+0000 7f3cf57fa640 1 -- 172.21.15.125:0/3145401076 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8060a00 con 0x562eee6a4810 2023-10-18T10:41:38.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 -- 172.21.15.125:0/3145401076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee6a4810 msgr2=0x562eee6a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee6a4810 0x562eee6a4010 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8058ad0 tx=0x7f3cf801c040 comp rx=0 tx=0).stop 2023-10-18T10:41:38.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 -- 172.21.15.125:0/3145401076 shutdown_connections 2023-10-18T10:41:38.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69ef20 0x562eee693150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee6a4810 0x562eee6a4010 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 --2- 172.21.15.125:0/3145401076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee69eb40 0x562eee66d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 -- 172.21.15.125:0/3145401076 >> 172.21.15.125:0/3145401076 conn(0x562eee672020 msgr2=0x562eee6b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:38.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 -- 172.21.15.125:0/3145401076 shutdown_connections 2023-10-18T10:41:38.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 -- 172.21.15.125:0/3145401076 wait complete. 2023-10-18T10:41:38.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 -- start start 2023-10-18T10:41:38.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.337+0000 7f3d083cc740 1 Processor -- start 2023-10-18T10:41:38.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee722540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee676990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3d083cc740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3d083cc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eee7261f0 con 0x562eee69ef20 2023-10-18T10:41:38.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3d083cc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eee6a1aa0 con 0x562eee69eb40 2023-10-18T10:41:38.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3d083cc740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eee6bcdb0 con 0x562eee6a4810 2023-10-18T10:41:38.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3d083cc740 10 client.0 _ll_register_callbacks cb 0x7f3ce8009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:41:38.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee676990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee676990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52180/0 (socket says 172.21.15.125:52180) 2023-10-18T10:41:38.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 -- 172.21.15.125:0/374407203 learned_addr learned my addr 172.21.15.125:0/374407203 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:38.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee722540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 -- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 msgr2=0x562eee6bc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:38.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6bc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 -- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 msgr2=0x562eee722540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:41:38.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee722540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 -- 172.21.15.125:0/374407203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0015b960 con 0x562eee69ef20 2023-10-18T10:41:38.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cff7fe640 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6bc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:38.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cf7fff640 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee676990 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800fad0 tx=0x7f3ce8004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283d600409e079bb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:38.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.338+0000 7f3cddffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:41:38.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3cddffb640 1 -- 172.21.15.125:0/374407203 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800fb50 con 0x562eee69ef20 2023-10-18T10:41:38.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3cddffb640 1 -- 172.21.15.125:0/374407203 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3ce8000e40 con 0x562eee69ef20 2023-10-18T10:41:38.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 1 -- 172.21.15.125:0/374407203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ce8005380 con 0x562eee69ef20 2023-10-18T10:41:38.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3cddffb640 1 -- 172.21.15.125:0/374407203 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce800fcb0 con 0x562eee69ef20 2023-10-18T10:41:38.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 10 client.8247 fetch_fsmap learned FSMap version 41 2023-10-18T10:41:38.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 1 -- 172.21.15.125:0/374407203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x562eee5a04f0 con 0x562eee69ef20 2023-10-18T10:41:38.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3cddffb640 1 -- 172.21.15.125:0/374407203 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ce8005380 con 0x562eee69ef20 2023-10-18T10:41:38.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3cddffb640 1 -- 172.21.15.125:0/374407203 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3ce8004da0 con 0x562eee69ef20 2023-10-18T10:41:38.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 10 client.8247 fetch_fsmap finished waiting for FSMap version 41 2023-10-18T10:41:38.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 10 client.8247 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T10:41:38.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 20 client.8247 populate_metadata read hostname 'smithi125' 2023-10-18T10:41:38.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] conn(0x562eee6bfde0 0x562eee6c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 4 client.8247 mds_command: new command op to 8076 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:41:38.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3d083cc740 1 -- 172.21.15.125:0/374407203 --> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562eee2df6c0 con 0x562eee6bfde0 2023-10-18T10:41:38.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] conn(0x562eee6bfde0 0x562eee6c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3cddffb640 10 client.8247 ms_handle_connect on v2:172.21.15.125:6833/3342443842 2023-10-18T10:41:38.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.339+0000 7f3cfeffd640 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] conn(0x562eee6bfde0 0x562eee6c01c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778b924bfa36f0d6 server_cookie=2cb715c99d1319e9 in_seq=0 out_seq=0 2023-10-18T10:41:38.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3cddffb640 1 -- 172.21.15.125:0/374407203 <== mds.0 v2:172.21.15.125:6833/3342443842 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x562eee2df6c0 con 0x562eee6bfde0 2023-10-18T10:41:38.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3cddffb640 10 client.8247 handle_command_reply: tid=0 2023-10-18T10:41:38.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 client.8247 shutdown 2023-10-18T10:41:38.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 -- 172.21.15.125:0/374407203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 msgr2=0x562eee676990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee676990 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800fad0 tx=0x7f3ce8004bb0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 -- 172.21.15.125:0/374407203 shutdown_connections 2023-10-18T10:41:38.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.125:6833/3342443842,v1:172.21.15.125:6835/3342443842] conn(0x562eee6bfde0 0x562eee6c01c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562eee6a4810 0x562eee6bc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3cddffb640 0 client.8247 ms_handle_reset on v2:172.21.15.125:6833/3342443842 2023-10-18T10:41:38.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562eee69ef20 0x562eee676990 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 --2- 172.21.15.125:0/374407203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562eee69eb40 0x562eee722540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 -- 172.21.15.125:0/374407203 >> 172.21.15.125:0/374407203 conn(0x562eee672020 msgr2=0x562eee6cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:38.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 -- 172.21.15.125:0/374407203 shutdown_connections 2023-10-18T10:41:38.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 1 -- 172.21.15.125:0/374407203 wait complete. 2023-10-18T10:41:38.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d083cc740 20 client.8247 trim_cache size 0 max 16384 2023-10-18T10:41:38.366 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:41:38.366 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:41:38.366 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:41:38.366 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 8076, 2023-10-18T10:41:38.367 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:41:38.367 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:41:38.367 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:41:38.367 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.8966078559999993, 2023-10-18T10:41:38.367 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 41, 2023-10-18T10:41:38.368 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 112, 2023-10-18T10:41:38.368 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 112, 2023-10-18T10:41:38.368 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 14.678357436000001 2023-10-18T10:41:38.368 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:41:38.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 -- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f3ce006af90 msgr2=0x7f3ce006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f3ce006af90 0x7f3ce006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0036000 tx=0x7f3cf0035040 comp rx=0 tx=0).stop 2023-10-18T10:41:38.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 -- 172.21.15.125:0/4237307265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d001f2ed0 msgr2=0x7f3d001eeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d001f2ed0 0x7f3d001eeee0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8001330 tx=0x7f3cf800ed10 comp rx=0 tx=0).stop 2023-10-18T10:41:38.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 -- 172.21.15.125:0/4237307265 shutdown_connections 2023-10-18T10:41:38.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f3ce006af90 0x7f3ce006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d001f2ed0 0x7f3d001eeee0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d001f2510 0x7f3d001f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 --2- 172.21.15.125:0/4237307265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec003970 0x7f3d00145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 -- 172.21.15.125:0/4237307265 >> 172.21.15.125:0/4237307265 conn(0x7f3d00069930 msgr2=0x7f3d00069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:38.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 -- 172.21.15.125:0/4237307265 shutdown_connections 2023-10-18T10:41:38.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.340+0000 7f3d05765640 1 -- 172.21.15.125:0/4237307265 wait complete. 2023-10-18T10:41:38.372 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:41:38.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 -- 172.21.15.125:0/3661076444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 msgr2=0x7f3228158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 --2- 172.21.15.125:0/3661076444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 0x7f3228158300 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f32280b1450 tx=0x7f321802d150 comp rx=0 tx=0).stop 2023-10-18T10:41:38.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 -- 172.21.15.125:0/3661076444 shutdown_connections 2023-10-18T10:41:38.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 --2- 172.21.15.125:0/3661076444 >> v2:172.21.15.139:3300/0 conn(0x7f32280b0530 0x7f32280b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 --2- 172.21.15.125:0/3661076444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32280afa20 0x7f32280705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 --2- 172.21.15.125:0/3661076444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 0x7f3228158300 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 -- 172.21.15.125:0/3661076444 >> 172.21.15.125:0/3661076444 conn(0x7f3228069930 msgr2=0x7f3228069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:38.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 -- 172.21.15.125:0/3661076444 shutdown_connections 2023-10-18T10:41:38.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.473+0000 7f322c807640 1 -- 172.21.15.125:0/3661076444 wait complete. 2023-10-18T10:41:38.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.474+0000 7f322c807640 1 Processor -- start 2023-10-18T10:41:38.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.474+0000 7f322c807640 1 -- start start 2023-10-18T10:41:38.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f322c807640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32280afa20 0x7f32280a7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f322c807640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32280b0530 0x7f32281f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f322c807640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 0x7f3228145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f322c807640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32281f2b50 con 0x7f32280cbd20 2023-10-18T10:41:38.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f322c807640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32281f2e30 con 0x7f32280afa20 2023-10-18T10:41:38.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f322c807640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32281f3110 con 0x7f32280b0530 2023-10-18T10:41:38.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3226d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 0x7f3228145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3226d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 0x7f3228145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52200/0 (socket says 172.21.15.125:52200) 2023-10-18T10:41:38.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3226d9d640 1 -- 172.21.15.125:0/2060685299 learned_addr learned my addr 172.21.15.125:0/2060685299 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:38.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f322659c640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32280afa20 0x7f32280a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3225d9b640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32280b0530 0x7f32281f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3226d9d640 1 -- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32280b0530 msgr2=0x7f32281f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3226d9d640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32280b0530 0x7f32281f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3226d9d640 1 -- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32280afa20 msgr2=0x7f32280a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3226d9d640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32280afa20 0x7f32280a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3226d9d640 1 -- 172.21.15.125:0/2060685299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321800aa10 con 0x7f32280cbd20 2023-10-18T10:41:38.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.475+0000 7f3225d9b640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32280b0530 0x7f32281f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:38.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.476+0000 7f3226d9d640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 0x7f3228145270 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f322000d750 tx=0x7f322000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:38.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.476+0000 7f3216ffd640 1 -- 172.21.15.125:0/2060685299 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220009bd0 con 0x7f32280cbd20 2023-10-18T10:41:38.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.476+0000 7f3216ffd640 1 -- 172.21.15.125:0/2060685299 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3220009e20 con 0x7f32280cbd20 2023-10-18T10:41:38.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.476+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32281f33f0 con 0x7f32280cbd20 2023-10-18T10:41:38.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.476+0000 7f3216ffd640 1 -- 172.21.15.125:0/2060685299 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220005280 con 0x7f32280cbd20 2023-10-18T10:41:38.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.476+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32281f2810 con 0x7f32280cbd20 2023-10-18T10:41:38.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.477+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32281f3910 con 0x7f32280cbd20 2023-10-18T10:41:38.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.477+0000 7f3216ffd640 1 -- 172.21.15.125:0/2060685299 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f32200053e0 con 0x7f32280cbd20 2023-10-18T10:41:38.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.477+0000 7f3216ffd640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f320806af60 0x7f320806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:38.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.478+0000 7f3216ffd640 1 -- 172.21.15.125:0/2060685299 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3220089b30 con 0x7f32280cbd20 2023-10-18T10:41:38.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.478+0000 7f322659c640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f320806af60 0x7f320806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:38.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.478+0000 7f322659c640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f320806af60 0x7f320806d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3218037000 tx=0x7f3218004ab0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:38.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.482+0000 7f3216ffd640 1 -- 172.21.15.125:0/2060685299 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3220054930 con 0x7f32280cbd20 2023-10-18T10:41:38.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.688+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f32280c96b0 con 0x7f32280cbd20 2023-10-18T10:41:38.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.688+0000 7f3216ffd640 1 -- 172.21.15.125:0/2060685299 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3220017020 con 0x7f32280cbd20 2023-10-18T10:41:38.691 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:41:38.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f320806af60 msgr2=0x7f320806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f320806af60 0x7f320806d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3218037000 tx=0x7f3218004ab0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 msgr2=0x7f3228145270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:38.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 0x7f3228145270 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f322000d750 tx=0x7f322000dc10 comp rx=0 tx=0).stop 2023-10-18T10:41:38.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 shutdown_connections 2023-10-18T10:41:38.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f320806af60 0x7f320806d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32280cbd20 0x7f3228145270 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32280b0530 0x7f32281f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 --2- 172.21.15.125:0/2060685299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32280afa20 0x7f32280a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:38.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 >> 172.21.15.125:0/2060685299 conn(0x7f3228069930 msgr2=0x7f32280ac5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:38.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 shutdown_connections 2023-10-18T10:41:38.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:38.690+0000 7f322c807640 1 -- 172.21.15.125:0/2060685299 wait complete. 2023-10-18T10:41:38.702 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:41:38.703 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:41:38.703 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:41:38.703 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:41:38.704 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:38.704 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:41:38.704 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:41:38.705 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:41:38.705 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:41:38.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:38.706 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:41:38.721 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:41:38.721 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:41:38.721 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:41:38.722 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:38.722 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:41:38.722 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:38.722 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:41:38.723 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:38.723 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:41:38.723 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:41:38.723 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:38.723 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:41:38.723 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:41:38.724 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:38.724 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:41:38.724 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:38.724 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:41:38.724 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 41725sec preferred_lft 41725sec 2023-10-18T10:41:38.725 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:41:38.725 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:38.725 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:41:38.725 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:41:38.725 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:41:38.726 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:38.726 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T10:41:38.726 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:38.726 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:41:38.726 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:38.727 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T10:41:38.727 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:38.727 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:41:38.727 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:38.727 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T10:41:38.727 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:41:38.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:38.728 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:41:38.787 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:41:38.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:38.789 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:41:38.844 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:41:38.844 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:41:38.844 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:41:38.845 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:41:38.845 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:41:38.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:38.846 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:41:38.846 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:41:38.847 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:41:38.847 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:41:38.847 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:41:38.848 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:41:38.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:38.940 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:41:38.955 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:41:38.955 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:41:38.955 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:41:38.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:38.956 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:41:39.021 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:41:39.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:41:39.022 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:39.076 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:41:39.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:41:39.077 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:39.131 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:41:39.197 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:41:39.253 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:41:39.253 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:41:39.253 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:41:39.254 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:41:39.254 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:41:39.254 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:41:39.254 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:41:39.254 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:41:39.255 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:41:39.255 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:41:39.255 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:41:39.255 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:41:39.255 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:41:39.256 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:41:39.256 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:41:39.256 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:41:39.256 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:41:39.256 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:41:39.257 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:41:39.257 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:41:39.257 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:41:39.257 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:41:39.257 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:41:39.258 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:41:39.258 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:41:39.258 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:41:39.258 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:41:39.258 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:41:39.259 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:41:39.259 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:41:39.259 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:39.259 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:39.259 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:39.260 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:39.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:39.260 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:41:39.307 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:41:39.308 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:41:39.350 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:41:39.377 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:41:39.433 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:41:39.433 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:41:39.433 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:41:39.434 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:41:39.434 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:41:39.434 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:41:39.434 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:41:39.434 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:41:39.435 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:41:39.435 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:41:39.435 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:41:39.435 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:41:39.435 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:41:39.436 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:41:39.436 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:41:39.436 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:41:39.436 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:41:39.436 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:41:39.437 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:41:39.437 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:41:39.437 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:41:39.437 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:41:39.437 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:41:39.438 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:41:39.438 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:41:39.438 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:41:39.438 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:41:39.438 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:41:39.439 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:41:39.439 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:41:39.439 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:39.439 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:41:39.439 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:39.440 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:41:39.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:39.440 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:41:39.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:41:39.433+0000 7f8c5e43d480 -1 init, newargv = 0x560a332ae270 newargc=15 2023-10-18T10:41:39.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[125724]: starting ceph client 2023-10-18T10:41:39.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[125724]: starting fuse 2023-10-18T10:41:39.456 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:41:39.456 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:41:39.457 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:41:39.457 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:41:39.457 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:41:39.458 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:41:39.458 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:41:39.458 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:41:39.459 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:41:39.459 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:41:39.460 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:41:39.460 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:41:39.460 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:41:39.461 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:41:39.461 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:41:39.462 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:41:39.462 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:41:39.462 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:41:39.463 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:41:39.463 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:41:39.464 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:41:39.464 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:41:39.464 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:41:39.465 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:41:39.465 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:41:39.465 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:41:39.466 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:41:39.466 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:41:39.467 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:41:39.576 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.125724.asok 2023-10-18T10:41:39.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:41:39.586 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.125724.asok 2023-10-18T10:41:39.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:39.587 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.125724.asok status 2023-10-18T10:41:39.720 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:41:39.721 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:41:39.721 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:41:39.721 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:41:39.721 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:41:39.722 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:41:39.722 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:41:39.722 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "125724", 2023-10-18T10:41:39.722 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:41:39.722 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:39.723 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:41:39.723 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:41:39.723 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 8259, 2023-10-18T10:41:39.723 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:41:39.723 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:41:39.724 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:41:39.724 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 8259 2023-10-18T10:41:39.724 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:39.724 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:41:39.724 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:41:39.725 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:41:39.725 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 1874217052 2023-10-18T10:41:39.725 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:39.725 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:39.725 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:41:39.725 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:41:39.726 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:41:39.726 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 1874217052 2023-10-18T10:41:39.726 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:41:39.726 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.8259 192.168.0.1:0/1874217052", 2023-10-18T10:41:39.726 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/1874217052", 2023-10-18T10:41:39.727 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:41:39.727 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 41, 2023-10-18T10:41:39.727 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 112, 2023-10-18T10:41:39.727 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:41:39.727 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:41:39.727 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:41:39.728 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:41:39.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:39.729 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:41:39.730 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:39.745 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:41:39.745 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:39.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:41:39.746 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:41:39.812 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T10:41:39.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.952+0000 7f4af2fce640 1 -- 172.21.15.125:0/1227960442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0afa20 msgr2=0x7f4aec0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:39.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.952+0000 7f4af2fce640 1 --2- 172.21.15.125:0/1227960442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0afa20 0x7f4aec0705a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009830 tx=0x7f4ad802bd60 comp rx=0 tx=0).stop 2023-10-18T10:41:39.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.952+0000 7f4af2fce640 1 -- 172.21.15.125:0/1227960442 shutdown_connections 2023-10-18T10:41:39.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.952+0000 7f4af2fce640 1 --2- 172.21.15.125:0/1227960442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0afa20 0x7f4aec0705a0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.952+0000 7f4af2fce640 1 --2- 172.21.15.125:0/1227960442 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4aec0cbd20 0x7f4aec158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.952+0000 7f4af2fce640 1 -- 172.21.15.125:0/1227960442 >> 172.21.15.125:0/1227960442 conn(0x7f4aec069930 msgr2=0x7f4aec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.952+0000 7f4af2fce640 1 -- 172.21.15.125:0/1227960442 shutdown_connections 2023-10-18T10:41:39.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.953+0000 7f4af2fce640 1 -- 172.21.15.125:0/1227960442 wait complete. 2023-10-18T10:41:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.954+0000 7f4af2fce640 1 Processor -- start 2023-10-18T10:41:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.954+0000 7f4af2fce640 1 -- start start 2023-10-18T10:41:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.954+0000 7f4af2fce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aec0afa20 0x7f4aec140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:39.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.954+0000 7f4af2fce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0b0530 0x7f4aec142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.954+0000 7f4af2fce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4aec0cbd20 0x7f4aec159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.954+0000 7f4af2fce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec141290 con 0x7f4aec0b0530 2023-10-18T10:41:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.954+0000 7f4af2fce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec129d40 con 0x7f4aec0afa20 2023-10-18T10:41:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.954+0000 7f4af2fce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec12aba0 con 0x7f4aec0cbd20 2023-10-18T10:41:39.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0b0530 0x7f4aec142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:39.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0b0530 0x7f4aec142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52206/0 (socket says 172.21.15.125:52206) 2023-10-18T10:41:39.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 -- 172.21.15.125:0/2428649737 learned_addr learned my addr 172.21.15.125:0/2428649737 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:39.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4af0d6a640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aec0afa20 0x7f4aec140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:39.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4af156b640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4aec0cbd20 0x7f4aec159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:39.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 -- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4aec0cbd20 msgr2=0x7f4aec159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:39.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4aec0cbd20 0x7f4aec159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:39.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 -- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aec0afa20 msgr2=0x7f4aec140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:39.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aec0afa20 0x7f4aec140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:39.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 -- 172.21.15.125:0/2428649737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8009490 con 0x7f4aec0b0530 2023-10-18T10:41:39.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4af0d6a640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aec0afa20 0x7f4aec140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:39.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4aebfff640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0b0530 0x7f4aec142c80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4ad802c370 tx=0x7f4ad80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:39.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4ae97fa640 1 -- 172.21.15.125:0/2428649737 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8003ea0 con 0x7f4aec0b0530 2023-10-18T10:41:39.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4ae97fa640 1 -- 172.21.15.125:0/2428649737 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4ad8005020 con 0x7f4aec0b0530 2023-10-18T10:41:39.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec070cf0 con 0x7f4aec0b0530 2023-10-18T10:41:39.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4ae97fa640 1 -- 172.21.15.125:0/2428649737 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8005410 con 0x7f4aec0b0530 2023-10-18T10:41:39.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.955+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec12a270 con 0x7f4aec0b0530 2023-10-18T10:41:39.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.956+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec0690e0 con 0x7f4aec0b0530 2023-10-18T10:41:39.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.957+0000 7f4ae97fa640 1 -- 172.21.15.125:0/2428649737 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4ad80047a0 con 0x7f4aec0b0530 2023-10-18T10:41:39.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.957+0000 7f4ae97fa640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4ad006af60 0x7f4ad006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:39.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.957+0000 7f4ae97fa640 1 -- 172.21.15.125:0/2428649737 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4ad80ac5d0 con 0x7f4aec0b0530 2023-10-18T10:41:39.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.957+0000 7f4af0d6a640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4ad006af60 0x7f4ad006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:39.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.958+0000 7f4af0d6a640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4ad006af60 0x7f4ad006d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0005fd0 tx=0x7f4ae0005ea0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:39.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:39.961+0000 7f4ae97fa640 1 -- 172.21.15.125:0/2428649737 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ad8077440 con 0x7f4aec0b0530 2023-10-18T10:41:40.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.149+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 --> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4aec0cc1e0 con 0x7f4ad006af60 2023-10-18T10:41:40.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.150+0000 7f4ae97fa640 1 -- 172.21.15.125:0/2428649737 <== mgr.6299 v2:172.21.15.139:6832/1523403591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4aec0cc1e0 con 0x7f4ad006af60 2023-10-18T10:41:40.152 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:41:40.153 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:41:40.153 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T10:41:40.153 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:41:40.153 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:41:40.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4ad006af60 msgr2=0x7f4ad006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4ad006af60 0x7f4ad006d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0005fd0 tx=0x7f4ae0005ea0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0b0530 msgr2=0x7f4aec142c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0b0530 0x7f4aec142c80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4ad802c370 tx=0x7f4ad80038a0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 shutdown_connections 2023-10-18T10:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f4ad006af60 0x7f4ad006d420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4aec0cbd20 0x7f4aec159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4aec0b0530 0x7f4aec142c80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 --2- 172.21.15.125:0/2428649737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aec0afa20 0x7f4aec140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.152+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 >> 172.21.15.125:0/2428649737 conn(0x7f4aec069930 msgr2=0x7f4aec147ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.153+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 shutdown_connections 2023-10-18T10:41:40.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.153+0000 7f4af2fce640 1 -- 172.21.15.125:0/2428649737 wait complete. 2023-10-18T10:41:40.164 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T10:41:40.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.266+0000 7f393ab7d640 1 -- 172.21.15.125:0/2175567307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934141690 msgr2=0x7f39340703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.266+0000 7f393ab7d640 1 --2- 172.21.15.125:0/2175567307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934141690 0x7f39340703e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f39340b20f0 tx=0x7f392002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.266+0000 7f393ab7d640 1 -- 172.21.15.125:0/2175567307 shutdown_connections 2023-10-18T10:41:40.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.266+0000 7f393ab7d640 1 --2- 172.21.15.125:0/2175567307 >> v2:172.21.15.139:3300/0 conn(0x7f3934070920 0x7f39340affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.266+0000 7f393ab7d640 1 --2- 172.21.15.125:0/2175567307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934141690 0x7f39340703e0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.266+0000 7f393ab7d640 1 -- 172.21.15.125:0/2175567307 >> 172.21.15.125:0/2175567307 conn(0x7f3934069930 msgr2=0x7f3934069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:40.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.266+0000 7f393ab7d640 1 -- 172.21.15.125:0/2175567307 shutdown_connections 2023-10-18T10:41:40.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.267+0000 7f393ab7d640 1 -- 172.21.15.125:0/2175567307 wait complete. 2023-10-18T10:41:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.268+0000 7f393ab7d640 1 Processor -- start 2023-10-18T10:41:40.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.268+0000 7f393ab7d640 1 -- start start 2023-10-18T10:41:40.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.268+0000 7f393ab7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934070920 0x7f393413acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.268+0000 7f393ab7d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393413e520 0x7f39340abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.268+0000 7f393ab7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3934141690 0x7f39341e2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.268+0000 7f393ab7d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393413b210 con 0x7f3934070920 2023-10-18T10:41:40.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.268+0000 7f393ab7d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39340a7480 con 0x7f3934141690 2023-10-18T10:41:40.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.268+0000 7f393ab7d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39340a7760 con 0x7f393413e520 2023-10-18T10:41:40.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934070920 0x7f393413acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934070920 0x7f393413acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52222/0 (socket says 172.21.15.125:52222) 2023-10-18T10:41:40.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 -- 172.21.15.125:0/3497469038 learned_addr learned my addr 172.21.15.125:0/3497469038 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3933fff640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393413e520 0x7f39340abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f393911a640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3934141690 0x7f39341e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 -- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393413e520 msgr2=0x7f39340abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393413e520 0x7f39340abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 -- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3934141690 msgr2=0x7f39341e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3934141690 0x7f39341e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 -- 172.21.15.125:0/3497469038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3920009490 con 0x7f3934070920 2023-10-18T10:41:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3933fff640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393413e520 0x7f39340abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:41:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f3938919640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934070920 0x7f393413acd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f392800c500 tx=0x7f392800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f39317fa640 1 -- 172.21.15.125:0/3497469038 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928012830 con 0x7f3934070920 2023-10-18T10:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f39317fa640 1 -- 172.21.15.125:0/3497469038 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3928012a80 con 0x7f3934070920 2023-10-18T10:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39340a7a40 con 0x7f3934070920 2023-10-18T10:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f39317fa640 1 -- 172.21.15.125:0/3497469038 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928012e90 con 0x7f3934070920 2023-10-18T10:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.269+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39340a8000 con 0x7f3934070920 2023-10-18T10:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.270+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393413e0d0 con 0x7f3934070920 2023-10-18T10:41:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.271+0000 7f39317fa640 1 -- 172.21.15.125:0/3497469038 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 49) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f392801a3b0 con 0x7f3934070920 2023-10-18T10:41:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.271+0000 7f39317fa640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f391806af60 0x7f391806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.271+0000 7f39317fa640 1 -- 172.21.15.125:0/3497469038 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3928089770 con 0x7f3934070920 2023-10-18T10:41:40.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.272+0000 7f3933fff640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f391806af60 0x7f391806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.272+0000 7f3933fff640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f391806af60 0x7f391806d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3920036000 tx=0x7f3920004730 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:40.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.276+0000 7f39317fa640 1 -- 172.21.15.125:0/3497469038 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39280545e0 con 0x7f3934070920 2023-10-18T10:41:40.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.489+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f39340b1bd0 con 0x7f3934070920 2023-10-18T10:41:40.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.522+0000 7f39317fa640 1 -- 172.21.15.125:0/3497469038 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v50) v1 ==== 92+0+0 (secure 0 0 0) 0x7f3928057c00 con 0x7f3934070920 2023-10-18T10:41:40.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f391806af60 msgr2=0x7f391806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f391806af60 0x7f391806d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3920036000 tx=0x7f3920004730 comp rx=0 tx=0).stop 2023-10-18T10:41:40.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934070920 msgr2=0x7f393413acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934070920 0x7f393413acd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f392800c500 tx=0x7f392800c9c0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 shutdown_connections 2023-10-18T10:41:40.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f391806af60 0x7f391806d420 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3920036000 tx=0x7f3920004730 comp rx=0 tx=0).stop 2023-10-18T10:41:40.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3934141690 0x7f39341e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f393413e520 0x7f39340abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 --2- 172.21.15.125:0/3497469038 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3934070920 0x7f393413acd0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 >> 172.21.15.125:0/3497469038 conn(0x7f3934069930 msgr2=0x7f393406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:40.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 shutdown_connections 2023-10-18T10:41:40.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.525+0000 7f393ab7d640 1 -- 172.21.15.125:0/3497469038 wait complete. 2023-10-18T10:41:40.539 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T10:41:40.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.645+0000 7f1637883640 1 -- 172.21.15.125:0/466491559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163015b2a0 msgr2=0x7f16300cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.645+0000 7f1637883640 1 --2- 172.21.15.125:0/466491559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163015b2a0 0x7f16300cf720 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f162c005fd0 tx=0x7f162c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:41:40.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.645+0000 7f1637883640 1 -- 172.21.15.125:0/466491559 shutdown_connections 2023-10-18T10:41:40.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.645+0000 7f1637883640 1 --2- 172.21.15.125:0/466491559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f163015b2a0 0x7f16300cf720 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.645+0000 7f1637883640 1 --2- 172.21.15.125:0/466491559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16300cf340 0x7f16300d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.645+0000 7f1637883640 1 -- 172.21.15.125:0/466491559 >> 172.21.15.125:0/466491559 conn(0x7f16300d0d60 msgr2=0x7f16300d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:40.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.646+0000 7f1637883640 1 -- 172.21.15.125:0/466491559 shutdown_connections 2023-10-18T10:41:40.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.646+0000 7f1637883640 1 -- 172.21.15.125:0/466491559 wait complete. 2023-10-18T10:41:40.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.647+0000 7f1637883640 1 Processor -- start 2023-10-18T10:41:40.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.647+0000 7f1637883640 1 -- start start 2023-10-18T10:41:40.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.647+0000 7f1637883640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16300ceb40 0x7f16300c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.647+0000 7f1637883640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16300cf340 0x7f16300b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.647+0000 7f1637883640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f163015b2a0 0x7f16300c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.647+0000 7f1637883640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b3b20 con 0x7f16300cf340 2023-10-18T10:41:40.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.647+0000 7f1637883640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b3e00 con 0x7f16300ceb40 2023-10-18T10:41:40.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.647+0000 7f1637883640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b40e0 con 0x7f163015b2a0 2023-10-18T10:41:40.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16300cf340 0x7f16300b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16300cf340 0x7f16300b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52266/0 (socket says 172.21.15.125:52266) 2023-10-18T10:41:40.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 -- 172.21.15.125:0/4115843898 learned_addr learned my addr 172.21.15.125:0/4115843898 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:40.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f163561f640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16300ceb40 0x7f16300c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1635e20640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f163015b2a0 0x7f16300c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 -- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f163015b2a0 msgr2=0x7f16300c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f163015b2a0 0x7f16300c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 -- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16300ceb40 msgr2=0x7f16300c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16300ceb40 0x7f16300c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 -- 172.21.15.125:0/4115843898 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c009560 con 0x7f16300cf340 2023-10-18T10:41:40.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1634e1e640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16300cf340 0x7f16300b4bc0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f162c009850 tx=0x7f162c02d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:40.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1625ffb640 1 -- 172.21.15.125:0/4115843898 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c02ddd0 con 0x7f16300cf340 2023-10-18T10:41:40.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1625ffb640 1 -- 172.21.15.125:0/4115843898 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f162c0036e0 con 0x7f16300cf340 2023-10-18T10:41:40.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16300b43c0 con 0x7f16300cf340 2023-10-18T10:41:40.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1625ffb640 1 -- 172.21.15.125:0/4115843898 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c003a90 con 0x7f16300cf340 2023-10-18T10:41:40.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.648+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16300c8d80 con 0x7f16300cf340 2023-10-18T10:41:40.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.649+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16300690e0 con 0x7f16300cf340 2023-10-18T10:41:40.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.650+0000 7f1625ffb640 1 -- 172.21.15.125:0/4115843898 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 50) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f162c003bf0 con 0x7f16300cf340 2023-10-18T10:41:40.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.650+0000 7f1625ffb640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f161406afb0 0x7f161406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.650+0000 7f1625ffb640 1 -- 172.21.15.125:0/4115843898 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f162c0ac300 con 0x7f16300cf340 2023-10-18T10:41:40.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.650+0000 7f163561f640 1 -- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f161406afb0 msgr2=0x7f161406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1523403591 2023-10-18T10:41:40.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.650+0000 7f163561f640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f161406afb0 0x7f161406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:40.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.654+0000 7f1625ffb640 1 -- 172.21.15.125:0/4115843898 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f162c077330 con 0x7f16300cf340 2023-10-18T10:41:40.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.829+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f16300d0b40 con 0x7f16300cf340 2023-10-18T10:41:40.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.832+0000 7f1625ffb640 1 -- 172.21.15.125:0/4115843898 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v48)=0 v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f162c07a950 con 0x7f16300cf340 2023-10-18T10:41:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f161406afb0 msgr2=0x7f161406d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f161406afb0 0x7f161406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16300cf340 msgr2=0x7f16300b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16300cf340 0x7f16300b4bc0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f162c009850 tx=0x7f162c02d9d0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 shutdown_connections 2023-10-18T10:41:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7f161406afb0 0x7f161406d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f163015b2a0 0x7f16300c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16300cf340 0x7f16300b4bc0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 --2- 172.21.15.125:0/4115843898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16300ceb40 0x7f16300c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 >> 172.21.15.125:0/4115843898 conn(0x7f16300d0d60 msgr2=0x7f16300cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 shutdown_connections 2023-10-18T10:41:40.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.834+0000 7f1637883640 1 -- 172.21.15.125:0/4115843898 wait complete. 2023-10-18T10:41:40.847 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T10:41:40.905 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:40.903+0000 7f1f59045040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:41:40.916 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:40.915+0000 7f4871811040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:41:40.950 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:40.948+0000 7f1f59045040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:41:40.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.954+0000 7ff8570e0640 1 -- 172.21.15.125:0/2268590937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 msgr2=0x7ff83800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.954+0000 7ff8570e0640 1 --2- 172.21.15.125:0/2268590937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 0x7ff83800b980 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ff848001d80 tx=0x7ff8380057f0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.954+0000 7ff8570e0640 1 -- 172.21.15.125:0/2268590937 shutdown_connections 2023-10-18T10:41:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.954+0000 7ff8570e0640 1 --2- 172.21.15.125:0/2268590937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 0x7ff83800b980 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.954+0000 7ff8570e0640 1 -- 172.21.15.125:0/2268590937 >> 172.21.15.125:0/2268590937 conn(0x7ff8500d0d60 msgr2=0x7ff8500d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.954+0000 7ff8570e0640 1 -- 172.21.15.125:0/2268590937 shutdown_connections 2023-10-18T10:41:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.954+0000 7ff8570e0640 1 -- 172.21.15.125:0/2268590937 wait complete. 2023-10-18T10:41:40.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.956+0000 7ff8570e0640 1 Processor -- start 2023-10-18T10:41:40.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.956+0000 7ff8570e0640 1 -- start start 2023-10-18T10:41:40.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.957+0000 7ff8570e0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 0x7ff850151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.957+0000 7ff8570e0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8500c87f0 0x7ff8500ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.957+0000 7ff8570e0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8500c8c50 0x7ff850155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.957+0000 7ff8570e0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff850151f30 con 0x7ff838005180 2023-10-18T10:41:40.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.957+0000 7ff8570e0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500c91a0 con 0x7ff8500c8c50 2023-10-18T10:41:40.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.957+0000 7ff8570e0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500c9480 con 0x7ff8500c87f0 2023-10-18T10:41:40.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 0x7ff850151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 0x7ff850151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52294/0 (socket says 172.21.15.125:52294) 2023-10-18T10:41:40.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 -- 172.21.15.125:0/2844479758 learned_addr learned my addr 172.21.15.125:0/2844479758 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:40.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff847fff640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8500c87f0 0x7ff8500ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff85567d640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8500c8c50 0x7ff850155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:40.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 -- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8500c87f0 msgr2=0x7ff8500ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8500c87f0 0x7ff8500ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 -- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8500c8c50 msgr2=0x7ff850155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:40.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8500c8c50 0x7ff850155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:40.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 -- 172.21.15.125:0/2844479758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500c9760 con 0x7ff838005180 2023-10-18T10:41:40.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff854e7c640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 0x7ff850151590 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ff84c011ab0 tx=0x7ff84c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:40.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff8457fa640 1 -- 172.21.15.125:0/2844479758 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00fd90 con 0x7ff838005180 2023-10-18T10:41:40.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff8457fa640 1 -- 172.21.15.125:0/2844479758 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff84c019410 con 0x7ff838005180 2023-10-18T10:41:40.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff8457fa640 1 -- 172.21.15.125:0/2844479758 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c019800 con 0x7ff838005180 2023-10-18T10:41:40.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8500c9a40 con 0x7ff838005180 2023-10-18T10:41:40.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.958+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8500ca000 con 0x7ff838005180 2023-10-18T10:41:40.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.959+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8500ca1c0 con 0x7ff838005180 2023-10-18T10:41:40.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.960+0000 7ff8457fa640 1 -- 172.21.15.125:0/2844479758 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 50) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff84c019960 con 0x7ff838005180 2023-10-18T10:41:40.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.960+0000 7ff8457fa640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff83c06afe0 0x7ff83c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:40.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.960+0000 7ff847fff640 1 -- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff83c06afe0 msgr2=0x7ff83c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1523403591 2023-10-18T10:41:40.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.960+0000 7ff8457fa640 1 -- 172.21.15.125:0/2844479758 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff84c08a810 con 0x7ff838005180 2023-10-18T10:41:40.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.960+0000 7ff847fff640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff83c06afe0 0x7ff83c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:40.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:40.964+0000 7ff8457fa640 1 -- 172.21.15.125:0/2844479758 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff84c055500 con 0x7ff838005180 2023-10-18T10:41:40.967 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:40.964+0000 7f4871811040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:41:41.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.137+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff8500690e0 con 0x7ff838005180 2023-10-18T10:41:41.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.141+0000 7ff8457fa640 1 -- 172.21.15.125:0/2844479758 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v49)=0 v49) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff84c01e020 con 0x7ff838005180 2023-10-18T10:41:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff83c06afe0 msgr2=0x7ff83c06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff83c06afe0 0x7ff83c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 msgr2=0x7ff850151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 0x7ff850151590 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ff84c011ab0 tx=0x7ff84c011f70 comp rx=0 tx=0).stop 2023-10-18T10:41:41.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 shutdown_connections 2023-10-18T10:41:41.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7ff83c06afe0 0x7ff83c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8500c8c50 0x7ff850155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8500c87f0 0x7ff8500ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 --2- 172.21.15.125:0/2844479758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff838005180 0x7ff850151590 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 >> 172.21.15.125:0/2844479758 conn(0x7ff8500d0d60 msgr2=0x7ff85006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:41.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 shutdown_connections 2023-10-18T10:41:41.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.143+0000 7ff8570e0640 1 -- 172.21.15.125:0/2844479758 wait complete. 2023-10-18T10:41:41.155 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1m 2023-10-18T10:41:41.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.260+0000 7fe79f0e5640 1 -- 172.21.15.125:0/3931425376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980ceb40 msgr2=0x7fe7980b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:41.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.260+0000 7fe79f0e5640 1 --2- 172.21.15.125:0/3931425376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980ceb40 0x7fe7980b0650 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fe790005fd0 tx=0x7fe79002c240 comp rx=0 tx=0).stop 2023-10-18T10:41:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.260+0000 7fe79f0e5640 1 -- 172.21.15.125:0/3931425376 shutdown_connections 2023-10-18T10:41:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.260+0000 7fe79f0e5640 1 --2- 172.21.15.125:0/3931425376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7980ceb40 0x7fe7980b0650 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.260+0000 7fe79f0e5640 1 --2- 172.21.15.125:0/3931425376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7980cf340 0x7fe7980d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.260+0000 7fe79f0e5640 1 -- 172.21.15.125:0/3931425376 >> 172.21.15.125:0/3931425376 conn(0x7fe7980d0d60 msgr2=0x7fe7980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.260+0000 7fe79f0e5640 1 -- 172.21.15.125:0/3931425376 shutdown_connections 2023-10-18T10:41:41.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.260+0000 7fe79f0e5640 1 -- 172.21.15.125:0/3931425376 wait complete. 2023-10-18T10:41:41.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.261+0000 7fe79f0e5640 1 Processor -- start 2023-10-18T10:41:41.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.261+0000 7fe79f0e5640 1 -- start start 2023-10-18T10:41:41.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79f0e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7980ceb40 0x7fe7981f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:41.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79f0e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7980cf340 0x7fe7981f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:41.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79f0e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79815b2a0 0x7fe7980b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:41.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79f0e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980b4bb0 con 0x7fe79815b2a0 2023-10-18T10:41:41.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79f0e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980b4e90 con 0x7fe7980cf340 2023-10-18T10:41:41.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79f0e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980b5170 con 0x7fe7980ceb40 2023-10-18T10:41:41.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79d682640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79815b2a0 0x7fe7980b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:41.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79d682640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79815b2a0 0x7fe7980b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52320/0 (socket says 172.21.15.125:52320) 2023-10-18T10:41:41.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79d682640 1 -- 172.21.15.125:0/1254409241 learned_addr learned my addr 172.21.15.125:0/1254409241 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:41.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe79ce81640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7980ceb40 0x7fe7981f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:41.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.262+0000 7fe797fff640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7980cf340 0x7fe7981f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:41.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe79d682640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7980ceb40 msgr2=0x7fe7981f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe79d682640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7980ceb40 0x7fe7981f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe79d682640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7980cf340 msgr2=0x7fe7981f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe79d682640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7980cf340 0x7fe7981f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe79d682640 1 -- 172.21.15.125:0/1254409241 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe790009900 con 0x7fe79815b2a0 2023-10-18T10:41:41.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe79d682640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79815b2a0 0x7fe7980b0da0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe790037040 tx=0x7fe790035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790034440 con 0x7fe79815b2a0 2023-10-18T10:41:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe790004330 con 0x7fe79815b2a0 2023-10-18T10:41:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7980b5450 con 0x7fe79815b2a0 2023-10-18T10:41:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7900046c0 con 0x7fe79815b2a0 2023-10-18T10:41:41.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.263+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7981f4af0 con 0x7fe79815b2a0 2023-10-18T10:41:41.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.264+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7980b5970 con 0x7fe79815b2a0 2023-10-18T10:41:41.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.265+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 50) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fe79003b040 con 0x7fe79815b2a0 2023-10-18T10:41:41.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.265+0000 7fe7957fa640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 0x7fe77c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:41.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.265+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe7900ac370 con 0x7fe79815b2a0 2023-10-18T10:41:41.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.265+0000 7fe79ce81640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 msgr2=0x7fe77c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1523403591 2023-10-18T10:41:41.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.265+0000 7fe79ce81640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 0x7fe77c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:41:41.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.269+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7980b5970 con 0x7fe79815b2a0 2023-10-18T10:41:41.340 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.339+0000 7f1f59045040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:41:41.373 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.372+0000 7f4871811040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:41:41.384 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.382+0000 7f1f59045040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:41:41.419 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.417+0000 7f4871811040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:41:41.426 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.424+0000 7f1f59045040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:41:41.463 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.461+0000 7f4871811040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:41:41.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.464+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 --> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7980b5970 con 0x7fe77c06afb0 2023-10-18T10:41:41.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.466+0000 7fe79ce81640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 msgr2=0x7fe77c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1523403591 2023-10-18T10:41:41.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.466+0000 7fe79ce81640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 0x7fe77c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:41:41.476 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.475+0000 7f1f59045040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:41:41.514 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.513+0000 7f4871811040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:41:41.517 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.516+0000 7f1f59045040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:41:41.557 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.555+0000 7f4871811040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:41:41.618 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.616+0000 7f1f59045040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:41:41.655 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.653+0000 7f4871811040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:41:41.672 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.670+0000 7f1f59045040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:41.708 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.706+0000 7f4871811040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:41.713 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.711+0000 7f1f59045040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:41:41.749 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.747+0000 7f4871811040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:41:41.761 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.759+0000 7f1f59045040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:41:41.797 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.795+0000 7f4871811040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:41:41.803 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:41.802+0000 7f1f59045040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:41:41.839 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:41.837+0000 7f4871811040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:41:41.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.867+0000 7fe79ce81640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 msgr2=0x7fe77c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1523403591 2023-10-18T10:41:41.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:41.867+0000 7fe79ce81640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 0x7fe77c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:41:42.119 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.117+0000 7f1f59045040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:41:42.152 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.150+0000 7f4871811040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:41:42.247 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.245+0000 7f1f59045040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:41:42.281 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.279+0000 7f4871811040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:41:42.337 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.335+0000 7f1f59045040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:41:42.371 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.370+0000 7f4871811040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:41:42.378 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.377+0000 7f1f59045040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:41:42.413 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.411+0000 7f4871811040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:41:42.472 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.471+0000 7f1f59045040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:42.509 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.507+0000 7f4871811040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:42.562 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.560+0000 7f1f59045040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:41:42.599 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.597+0000 7f4871811040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:41:42.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:42.668+0000 7fe79ce81640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 msgr2=0x7fe77c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1523403591 2023-10-18T10:41:42.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:42.668+0000 7fe79ce81640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 0x7fe77c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:41:42.698 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.696+0000 7f1f59045040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:42.735 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.734+0000 7f4871811040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:41:42.740 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.737+0000 7f1f59045040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:42.776 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.775+0000 7f4871811040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:41:42.845 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:42.842+0000 7f1f59045040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:41:42.879 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:42.877+0000 7f4871811040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:41:43.462 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.460+0000 7f1f59045040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:41:43.497 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:43.495+0000 7f4871811040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:41:43.727 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.724+0000 7f1f59045040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:41:43.759 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:43.757+0000 7f4871811040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:41:43.776 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.773+0000 7f1f59045040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:41:43.807 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:43.805+0000 7f4871811040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:41:43.875 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.873+0000 7f1f59045040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:41:43.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:43.897+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 51) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fe79000a040 con 0x7fe79815b2a0 2023-10-18T10:41:43.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:43.898+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 msgr2=0x7fe77c06d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:41:43.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:43.898+0000 7fe7957fa640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/1523403591,v1:172.21.15.139:6833/1523403591] conn(0x7fe77c06afb0 0x7fe77c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:43.909 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:41:43.907+0000 7f4871811040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:41:44.002 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.998+0000 7f1ed5aef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.002 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.998+0000 7f1ed5aef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.002 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.998+0000 7f1ed5aef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.003 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.998+0000 7f1ed5aef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.003 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.998+0000 7f1ed5aef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.003 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.999+0000 7f1ed8b35640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.003 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.999+0000 7f1ed8b35640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.004 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.999+0000 7f1ed8b35640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.004 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.999+0000 7f1ed8b35640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.004 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:41:43.999+0000 7f1ed8b35640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:41:44.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.901+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 52) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fe790077780 con 0x7fe79815b2a0 2023-10-18T10:41:44.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.902+0000 7fe7957fa640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fe77c035850 0x7fe77c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:44.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.902+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 --> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7980b5970 con 0x7fe77c035850 2023-10-18T10:41:44.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.903+0000 7fe79ce81640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fe77c035850 0x7fe77c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:44.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.906+0000 7fe79ce81640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fe77c035850 0x7fe77c075380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00b690 tx=0x7fe78c00b390 comp rx=0 tx=0).ready entity=mgr.6569 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:44.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.956+0000 7fe7957fa640 1 -- 172.21.15.125:0/1254409241 <== mgr.6569 v2:172.21.15.139:6832/4208657878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fe7980b5970 con 0x7fe77c035850 2023-10-18T10:41:44.958 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /bad-path 2023-10-18T10:41:44.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fe77c035850 msgr2=0x7fe77c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:44.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79f0e5640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fe77c035850 0x7fe77c075380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00b690 tx=0x7fe78c00b390 comp rx=0 tx=0).stop 2023-10-18T10:41:44.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79815b2a0 msgr2=0x7fe7980b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:44.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79f0e5640 1 --2- 172.21.15.125:0/1254409241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79815b2a0 0x7fe7980b0da0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe790037040 tx=0x7fe790035a40 comp rx=0 tx=0).stop 2023-10-18T10:41:44.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79d682640 1 -- 172.21.15.125:0/1254409241 reap_dead start 2023-10-18T10:41:44.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 shutdown_connections 2023-10-18T10:41:44.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 >> 172.21.15.125:0/1254409241 conn(0x7fe7980d0d60 msgr2=0x7fe79815a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:44.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 shutdown_connections 2023-10-18T10:41:44.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:44.958+0000 7fe79f0e5640 1 -- 172.21.15.125:0/1254409241 wait complete. 2023-10-18T10:41:44.971 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-10-18T10:41:44.972 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-18T10:41:45.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.076+0000 7f33626c2640 1 -- 172.21.15.125:0/1262507072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 msgr2=0x7f335c0cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:45.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.076+0000 7f33626c2640 1 --2- 172.21.15.125:0/1262507072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 0x7f335c0cbd20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f335c0b10f0 tx=0x7f334c02be20 comp rx=0 tx=0).stop 2023-10-18T10:41:45.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.076+0000 7f33626c2640 1 -- 172.21.15.125:0/1262507072 shutdown_connections 2023-10-18T10:41:45.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.076+0000 7f33626c2640 1 --2- 172.21.15.125:0/1262507072 >> v2:172.21.15.139:3300/0 conn(0x7f335c0705a0 0x7f335c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.076+0000 7f33626c2640 1 --2- 172.21.15.125:0/1262507072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 0x7f335c0cbd20 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.076+0000 7f33626c2640 1 -- 172.21.15.125:0/1262507072 >> 172.21.15.125:0/1262507072 conn(0x7f335c069930 msgr2=0x7f335c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:45.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.076+0000 7f33626c2640 1 -- 172.21.15.125:0/1262507072 shutdown_connections 2023-10-18T10:41:45.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.076+0000 7f33626c2640 1 -- 172.21.15.125:0/1262507072 wait complete. 2023-10-18T10:41:45.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.077+0000 7f33626c2640 1 Processor -- start 2023-10-18T10:41:45.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.077+0000 7f33626c2640 1 -- start start 2023-10-18T10:41:45.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f33626c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f335c0705a0 0x7f335c166d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:45.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f33626c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 0x7f335c13ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:45.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f33626c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f335c158300 0x7f335c134010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:45.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f33626c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c1668a0 con 0x7f335c0afa20 2023-10-18T10:41:45.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f33626c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c166570 con 0x7f335c0705a0 2023-10-18T10:41:45.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f33626c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c15a550 con 0x7f335c158300 2023-10-18T10:41:45.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 0x7f335c13ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:45.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 0x7f335c13ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52334/0 (socket says 172.21.15.125:52334) 2023-10-18T10:41:45.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 -- 172.21.15.125:0/812685293 learned_addr learned my addr 172.21.15.125:0/812685293 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:41:45.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f3360c5f640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f335c158300 0x7f335c134010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:45.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335bfff640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f335c0705a0 0x7f335c166d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:45.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 -- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f335c158300 msgr2=0x7f335c134010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:45.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f335c158300 0x7f335c134010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 -- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f335c0705a0 msgr2=0x7f335c166d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:45.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f335c0705a0 0x7f335c166d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 -- 172.21.15.125:0/812685293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c009560 con 0x7f335c0afa20 2023-10-18T10:41:45.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.078+0000 7f335b7fe640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 0x7f335c13ae70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f335000c400 tx=0x7f335000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:45.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.079+0000 7f3358ff9640 1 -- 172.21.15.125:0/812685293 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33500052d0 con 0x7f335c0afa20 2023-10-18T10:41:45.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.079+0000 7f3358ff9640 1 -- 172.21.15.125:0/812685293 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33500035f0 con 0x7f335c0afa20 2023-10-18T10:41:45.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.079+0000 7f3358ff9640 1 -- 172.21.15.125:0/812685293 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350003a00 con 0x7f335c0afa20 2023-10-18T10:41:45.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.079+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c06fe50 con 0x7f335c0afa20 2023-10-18T10:41:45.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.079+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c15ab10 con 0x7f335c0afa20 2023-10-18T10:41:45.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.080+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c0690e0 con 0x7f335c0afa20 2023-10-18T10:41:45.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.080+0000 7f3358ff9640 1 -- 172.21.15.125:0/812685293 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 52) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3350003b60 con 0x7f335c0afa20 2023-10-18T10:41:45.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.080+0000 7f3358ff9640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f334806ace0 0x7f334806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:41:45.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.081+0000 7f3358ff9640 1 -- 172.21.15.125:0/812685293 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f3350088d00 con 0x7f335c0afa20 2023-10-18T10:41:45.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.081+0000 7f335bfff640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f334806ace0 0x7f334806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:41:45.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.081+0000 7f335bfff640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f334806ace0 0x7f334806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f334c035000 tx=0x7f334c004730 comp rx=0 tx=0).ready entity=mgr.6569 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:41:45.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.085+0000 7f3358ff9640 1 -- 172.21.15.125:0/812685293 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3350053e70 con 0x7f335c0afa20 2023-10-18T10:41:45.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.269+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 --> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f335c0b0a70 con 0x7f334806ace0 2023-10-18T10:41:45.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.275+0000 7f3358ff9640 1 -- 172.21.15.125:0/812685293 <== mgr.6569 v2:172.21.15.139:6832/4208657878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7f334c004260 con 0x7f334806ace0 2023-10-18T10:41:45.277 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:41:45.277 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:41:44", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-18T10:41:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f334806ace0 msgr2=0x7f334806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f334806ace0 0x7f334806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f334c035000 tx=0x7f334c004730 comp rx=0 tx=0).stop 2023-10-18T10:41:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 msgr2=0x7f335c13ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:41:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 0x7f335c13ae70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f335000c400 tx=0x7f335000c8c0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 shutdown_connections 2023-10-18T10:41:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f334806ace0 0x7f334806d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f335c158300 0x7f335c134010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f335c0afa20 0x7f335c13ae70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 --2- 172.21.15.125:0/812685293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f335c0705a0 0x7f335c166d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:41:45.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 >> 172.21.15.125:0/812685293 conn(0x7f335c069930 msgr2=0x7f335c15b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:41:45.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 shutdown_connections 2023-10-18T10:41:45.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:41:45.277+0000 7f33626c2640 1 -- 172.21.15.125:0/812685293 wait complete. 2023-10-18T10:41:45.289 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:41:44", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-18T10:42:00.966 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:00.964+0000 7f1ec82d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:00.967 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:00.965+0000 7f1ec82d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:00.967 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:00.965+0000 7f1ec82d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:00.967 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:00.965+0000 7f1ec82d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:00.967 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:00.965+0000 7f1ec82d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:05.970 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:42:05.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:42:36.036 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:42:36.039 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:42:45.290 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-18T10:42:45.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.415+0000 7f3914264640 1 -- 172.21.15.125:0/758342409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c0b0290 msgr2=0x7f390c0cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.415+0000 7f3914264640 1 --2- 172.21.15.125:0/758342409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c0b0290 0x7f390c0cc590 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3904009860 tx=0x7f390402bd60 comp rx=0 tx=0).stop 2023-10-18T10:42:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.415+0000 7f3914264640 1 -- 172.21.15.125:0/758342409 shutdown_connections 2023-10-18T10:42:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.415+0000 7f3914264640 1 --2- 172.21.15.125:0/758342409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c0b0290 0x7f390c0cc590 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.415+0000 7f3914264640 1 -- 172.21.15.125:0/758342409 >> 172.21.15.125:0/758342409 conn(0x7f390c0af7a0 msgr2=0x7f390c0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:45.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.415+0000 7f3914264640 1 -- 172.21.15.125:0/758342409 shutdown_connections 2023-10-18T10:42:45.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.415+0000 7f3914264640 1 -- 172.21.15.125:0/758342409 wait complete. 2023-10-18T10:42:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.416+0000 7f3914264640 1 Processor -- start 2023-10-18T10:42:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.416+0000 7f3914264640 1 -- start start 2023-10-18T10:42:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3914264640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c069e30 0x7f390c1e2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3914264640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f390c0b0290 0x7f390c1e5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3914264640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c15b270 0x7f390c151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3914264640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1e68f0 con 0x7f390c0b0290 2023-10-18T10:42:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3914264640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1e5db0 con 0x7f390c15b270 2023-10-18T10:42:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3914264640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1f0f20 con 0x7f390c069e30 2023-10-18T10:42:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f39117ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f390c0b0290 0x7f390c1e5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f39117ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f390c0b0290 0x7f390c1e5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42794/0 (socket says 172.21.15.125:42794) 2023-10-18T10:42:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f39117ff640 1 -- 172.21.15.125:0/3668177287 learned_addr learned my addr 172.21.15.125:0/3668177287 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3912801640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c15b270 0x7f390c151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3912000640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c069e30 0x7f390c1e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f39117ff640 1 -- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c069e30 msgr2=0x7f390c1e2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f39117ff640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c069e30 0x7f390c1e2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f39117ff640 1 -- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c15b270 msgr2=0x7f390c151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f39117ff640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c15b270 0x7f390c151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f39117ff640 1 -- 172.21.15.125:0/3668177287 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904009490 con 0x7f390c0b0290 2023-10-18T10:42:45.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.417+0000 7f3912801640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c15b270 0x7f390c151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:42:45.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.418+0000 7f3912000640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c069e30 0x7f390c1e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:42:45.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.418+0000 7f39117ff640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f390c0b0290 0x7f390c1e5870 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c490 tx=0x7f38fc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:45.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.418+0000 7f39027fc640 1 -- 172.21.15.125:0/3668177287 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0052d0 con 0x7f390c0b0290 2023-10-18T10:42:45.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.418+0000 7f39027fc640 1 -- 172.21.15.125:0/3668177287 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f38fc005430 con 0x7f390c0b0290 2023-10-18T10:42:45.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.418+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c1e6050 con 0x7f390c0b0290 2023-10-18T10:42:45.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.418+0000 7f39027fc640 1 -- 172.21.15.125:0/3668177287 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc012890 con 0x7f390c0b0290 2023-10-18T10:42:45.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.418+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c1e6610 con 0x7f390c0b0290 2023-10-18T10:42:45.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.419+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c0690e0 con 0x7f390c0b0290 2023-10-18T10:42:45.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.420+0000 7f39027fc640 1 -- 172.21.15.125:0/3668177287 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 55) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f38fc0035f0 con 0x7f390c0b0290 2023-10-18T10:42:45.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.420+0000 7f39027fc640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f38f006b0d0 0x7f38f006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:45.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.420+0000 7f39027fc640 1 -- 172.21.15.125:0/3668177287 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f38fc089b40 con 0x7f390c0b0290 2023-10-18T10:42:45.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.420+0000 7f3912000640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f38f006b0d0 0x7f38f006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:45.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.421+0000 7f3912000640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f38f006b0d0 0x7f38f006d590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f390c0cb440 tx=0x7f3908009040 comp rx=0 tx=0).ready entity=mgr.6569 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:45.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.425+0000 7f39027fc640 1 -- 172.21.15.125:0/3668177287 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38fc054590 con 0x7f390c0b0290 2023-10-18T10:42:45.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.619+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 --> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f390c06d100 con 0x7f38f006b0d0 2023-10-18T10:42:45.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.627+0000 7f39027fc640 1 -- 172.21.15.125:0/3668177287 <== mgr.6569 v2:172.21.15.139:6832/4208657878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f390800dc60 con 0x7f38f006b0d0 2023-10-18T10:42:45.629 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:45.629 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:41:44", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-18T10:42:45.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f38f006b0d0 msgr2=0x7f38f006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f38f006b0d0 0x7f38f006d590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f390c0cb440 tx=0x7f3908009040 comp rx=0 tx=0).stop 2023-10-18T10:42:45.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f390c0b0290 msgr2=0x7f390c1e5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f390c0b0290 0x7f390c1e5870 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c490 tx=0x7f38fc00c950 comp rx=0 tx=0).stop 2023-10-18T10:42:45.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 shutdown_connections 2023-10-18T10:42:45.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f38f006b0d0 0x7f38f006d590 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c15b270 0x7f390c151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f390c0b0290 0x7f390c1e5870 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 --2- 172.21.15.125:0/3668177287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c069e30 0x7f390c1e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 >> 172.21.15.125:0/3668177287 conn(0x7f390c0af7a0 msgr2=0x7f390c0ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:45.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 shutdown_connections 2023-10-18T10:42:45.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.629+0000 7f3914264640 1 -- 172.21.15.125:0/3668177287 wait complete. 2023-10-18T10:42:45.641 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:41:44", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-18T10:42:45.642 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2023-10-18T10:42:45.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.744+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/321573363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340cb980 msgr2=0x7fcf3413db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.744+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/321573363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340cb980 0x7fcf3413db80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcf3406d4f0 tx=0x7fcf3002bdb0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.744+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/321573363 shutdown_connections 2023-10-18T10:42:45.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.744+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/321573363 >> v2:172.21.15.139:3300/0 conn(0x7fcf34070350 0x7fcf34070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.744+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/321573363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340cb980 0x7fcf3413db80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.744+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/321573363 >> 172.21.15.125:0/321573363 conn(0x7fcf34069930 msgr2=0x7fcf34069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:45.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.744+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/321573363 shutdown_connections 2023-10-18T10:42:45.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.744+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/321573363 wait complete. 2023-10-18T10:42:45.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.745+0000 7fcf3c1b6640 1 Processor -- start 2023-10-18T10:42:45.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.745+0000 7fcf3c1b6640 1 -- start start 2023-10-18T10:42:45.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3c1b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf34070350 0x7fcf34140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:45.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3c1b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf340cb980 0x7fcf340ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:45.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3c1b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340aee70 0x7fcf340b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:45.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3c1b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf341379b0 con 0x7fcf340aee70 2023-10-18T10:42:45.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3c1b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf341384f0 con 0x7fcf340cb980 2023-10-18T10:42:45.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3c1b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340af2f0 con 0x7fcf34070350 2023-10-18T10:42:45.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3a753640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340aee70 0x7fcf340b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:45.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3a753640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340aee70 0x7fcf340b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42822/0 (socket says 172.21.15.125:42822) 2023-10-18T10:42:45.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf3a753640 1 -- 172.21.15.125:0/3872664759 learned_addr learned my addr 172.21.15.125:0/3872664759 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:45.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf39f52640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf34070350 0x7fcf34140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:45.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.746+0000 7fcf39751640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf340cb980 0x7fcf340ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:45.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf3a753640 1 -- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf34070350 msgr2=0x7fcf34140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf3a753640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf34070350 0x7fcf34140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf3a753640 1 -- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf340cb980 msgr2=0x7fcf340ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf3a753640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf340cb980 0x7fcf340ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:45.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf3a753640 1 -- 172.21.15.125:0/3872664759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30009560 con 0x7fcf340aee70 2023-10-18T10:42:45.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf39f52640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf34070350 0x7fcf34140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:42:45.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf39751640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf340cb980 0x7fcf340ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:42:45.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf3a753640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340aee70 0x7fcf340b0980 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00eaa0 tx=0x7fcf2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:45.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf2a7fc640 1 -- 172.21.15.125:0/3872664759 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00cdb0 con 0x7fcf340aee70 2023-10-18T10:42:45.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf2a7fc640 1 -- 172.21.15.125:0/3872664759 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf2c004b20 con 0x7fcf340aee70 2023-10-18T10:42:45.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf34137bc0 con 0x7fcf340aee70 2023-10-18T10:42:45.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf2a7fc640 1 -- 172.21.15.125:0/3872664759 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c005020 con 0x7fcf340aee70 2023-10-18T10:42:45.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.747+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf34138180 con 0x7fcf340aee70 2023-10-18T10:42:45.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.748+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf340690e0 con 0x7fcf340aee70 2023-10-18T10:42:45.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.748+0000 7fcf2a7fc640 1 -- 172.21.15.125:0/3872664759 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 55) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fcf2c005180 con 0x7fcf340aee70 2023-10-18T10:42:45.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.749+0000 7fcf2a7fc640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fcf1806b0d0 0x7fcf1806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:45.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.749+0000 7fcf2a7fc640 1 -- 172.21.15.125:0/3872664759 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fcf2c08a8b0 con 0x7fcf340aee70 2023-10-18T10:42:45.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.749+0000 7fcf39f52640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fcf1806b0d0 0x7fcf1806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:45.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.749+0000 7fcf39f52640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fcf1806b0d0 0x7fcf1806d590 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf30035000 tx=0x7fcf30004730 comp rx=0 tx=0).ready entity=mgr.6569 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:45.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.753+0000 7fcf2a7fc640 1 -- 172.21.15.125:0/3872664759 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf2c0551e0 con 0x7fcf340aee70 2023-10-18T10:42:45.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.938+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 --> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf34070cc0 con 0x7fcf1806b0d0 2023-10-18T10:42:45.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.994+0000 7fcf2a7fc640 1 -- 172.21.15.125:0/3872664759 <== mgr.6569 v2:172.21.15.139:6832/4208657878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7fcf30004280 con 0x7fcf1806b0d0 2023-10-18T10:42:45.997 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /bad-path 2023-10-18T10:42:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.996+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fcf1806b0d0 msgr2=0x7fcf1806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fcf1806b0d0 0x7fcf1806d590 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf30035000 tx=0x7fcf30004730 comp rx=0 tx=0).stop 2023-10-18T10:42:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340aee70 msgr2=0x7fcf340b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340aee70 0x7fcf340b0980 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00eaa0 tx=0x7fcf2c00ef60 comp rx=0 tx=0).stop 2023-10-18T10:42:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 shutdown_connections 2023-10-18T10:42:46.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fcf1806b0d0 0x7fcf1806d590 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf340cb980 0x7fcf340ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf340aee70 0x7fcf340b0980 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 --2- 172.21.15.125:0/3872664759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf34070350 0x7fcf34140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 >> 172.21.15.125:0/3872664759 conn(0x7fcf34069930 msgr2=0x7fcf340ac590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:46.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 shutdown_connections 2023-10-18T10:42:46.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:45.997+0000 7fcf3c1b6640 1 -- 172.21.15.125:0/3872664759 wait complete. 2023-10-18T10:42:46.009 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-10-18T10:42:46.009 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T10:42:46.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.111+0000 7f8495c83640 1 -- 172.21.15.125:0/1139251597 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8490070af0 msgr2=0x7f84900affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.111+0000 7f8495c83640 1 --2- 172.21.15.125:0/1139251597 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8490070af0 0x7f84900affe0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8488009830 tx=0x7f848802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.111+0000 7f8495c83640 1 -- 172.21.15.125:0/1139251597 shutdown_connections 2023-10-18T10:42:46.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.111+0000 7f8495c83640 1 --2- 172.21.15.125:0/1139251597 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8490070af0 0x7f84900affe0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.111+0000 7f8495c83640 1 --2- 172.21.15.125:0/1139251597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f849013e520 0x7f849006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.111+0000 7f8495c83640 1 -- 172.21.15.125:0/1139251597 >> 172.21.15.125:0/1139251597 conn(0x7f8490069930 msgr2=0x7f8490069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:46.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.111+0000 7f8495c83640 1 -- 172.21.15.125:0/1139251597 shutdown_connections 2023-10-18T10:42:46.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.111+0000 7f8495c83640 1 -- 172.21.15.125:0/1139251597 wait complete. 2023-10-18T10:42:46.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.112+0000 7f8495c83640 1 Processor -- start 2023-10-18T10:42:46.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f8495c83640 1 -- start start 2023-10-18T10:42:46.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f8495c83640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8490070af0 0x7f849013b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:46.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f8495c83640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f849013e520 0x7f8490140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:46.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f8495c83640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8490141690 0x7f84901dc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:46.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f8495c83640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849013f770 con 0x7f849013e520 2023-10-18T10:42:46.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f8495c83640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84901409a0 con 0x7f8490141690 2023-10-18T10:42:46.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f8495c83640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849013eb30 con 0x7f8490070af0 2023-10-18T10:42:46.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f848effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f849013e520 0x7f8490140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:46.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f848effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f849013e520 0x7f8490140460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42842/0 (socket says 172.21.15.125:42842) 2023-10-18T10:42:46.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.113+0000 7f848effd640 1 -- 172.21.15.125:0/3223760601 learned_addr learned my addr 172.21.15.125:0/3223760601 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:46.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f848f7fe640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8490070af0 0x7f849013b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:46.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f848ffff640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8490141690 0x7f84901dc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:46.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f848effd640 1 -- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8490070af0 msgr2=0x7f849013b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f848effd640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8490070af0 0x7f849013b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f848effd640 1 -- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8490141690 msgr2=0x7f84901dc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f848effd640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8490141690 0x7f84901dc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f848effd640 1 -- 172.21.15.125:0/3223760601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488009490 con 0x7f849013e520 2023-10-18T10:42:46.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f848effd640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f849013e520 0x7f8490140460 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f847800e6b0 tx=0x7f847800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:46.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f8487fff640 1 -- 172.21.15.125:0/3223760601 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478005270 con 0x7f849013e520 2023-10-18T10:42:46.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f8487fff640 1 -- 172.21.15.125:0/3223760601 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84780054c0 con 0x7f849013e520 2023-10-18T10:42:46.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849013edd0 con 0x7f849013e520 2023-10-18T10:42:46.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f8487fff640 1 -- 172.21.15.125:0/3223760601 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847800ca90 con 0x7f849013e520 2023-10-18T10:42:46.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.114+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849013f390 con 0x7f849013e520 2023-10-18T10:42:46.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.115+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849013f8b0 con 0x7f849013e520 2023-10-18T10:42:46.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.116+0000 7f8487fff640 1 -- 172.21.15.125:0/3223760601 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 55) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f847800cbf0 con 0x7f849013e520 2023-10-18T10:42:46.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.116+0000 7f8487fff640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f847006b0d0 0x7f847006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:46.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.116+0000 7f8487fff640 1 -- 172.21.15.125:0/3223760601 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f847808c560 con 0x7f849013e520 2023-10-18T10:42:46.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.116+0000 7f848f7fe640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f847006b0d0 0x7f847006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:46.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.117+0000 7f848f7fe640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f847006b0d0 0x7f847006d590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8480005c40 tx=0x7f848000a3b0 comp rx=0 tx=0).ready entity=mgr.6569 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:46.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.120+0000 7f8487fff640 1 -- 172.21.15.125:0/3223760601 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f849013f8b0 con 0x7f849013e520 2023-10-18T10:42:46.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.330+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f849013f8b0 con 0x7f849013e520 2023-10-18T10:42:46.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.500+0000 7f8487fff640 1 -- 172.21.15.125:0/3223760601 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v56) v1 ==== 93+0+0 (secure 0 0 0) 0x7f849013f8b0 con 0x7f849013e520 2023-10-18T10:42:46.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f847006b0d0 msgr2=0x7f847006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f847006b0d0 0x7f847006d590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8480005c40 tx=0x7f848000a3b0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f849013e520 msgr2=0x7f8490140460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f849013e520 0x7f8490140460 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f847800e6b0 tx=0x7f847800eb70 comp rx=0 tx=0).stop 2023-10-18T10:42:46.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 shutdown_connections 2023-10-18T10:42:46.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f847006b0d0 0x7f847006d590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8490141690 0x7f84901dc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f849013e520 0x7f8490140460 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 --2- 172.21.15.125:0/3223760601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8490070af0 0x7f849013b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 >> 172.21.15.125:0/3223760601 conn(0x7f8490069930 msgr2=0x7f84900acb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:46.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 shutdown_connections 2023-10-18T10:42:46.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.503+0000 7f8495c83640 1 -- 172.21.15.125:0/3223760601 wait complete. 2023-10-18T10:42:46.518 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:42:46.519 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:42:46.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:42:46.612 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:42:46.613 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:42:46.613 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:42:46.629 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:42:46.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.772+0000 7f2ef97c0640 1 -- 172.21.15.125:0/194135365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee00047c0 msgr2=0x7f2ee0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.772+0000 7f2ef97c0640 1 --2- 172.21.15.125:0/194135365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee00047c0 0x7f2ee0004c40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2ee00057f0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.772+0000 7f2ef97c0640 1 -- 172.21.15.125:0/194135365 shutdown_connections 2023-10-18T10:42:46.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.772+0000 7f2ef97c0640 1 --2- 172.21.15.125:0/194135365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee00047c0 0x7f2ee0004c40 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.772+0000 7f2ef97c0640 1 -- 172.21.15.125:0/194135365 >> 172.21.15.125:0/194135365 conn(0x7f2ef409a030 msgr2=0x7f2ef40d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:46.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.772+0000 7f2ef97c0640 1 -- 172.21.15.125:0/194135365 shutdown_connections 2023-10-18T10:42:46.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.772+0000 7f2ef97c0640 1 -- 172.21.15.125:0/194135365 wait complete. 2023-10-18T10:42:46.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.773+0000 7f2ef97c0640 1 Processor -- start 2023-10-18T10:42:46.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.773+0000 7f2ef97c0640 1 -- start start 2023-10-18T10:42:46.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.773+0000 7f2ef97c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00047c0 0x7f2ef415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:46.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.773+0000 7f2ef97c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ef40c5b40 0x7f2ef40c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:46.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2ef97c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef40c64e0 0x7f2ef40ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:46.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2ef97c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40c6960 con 0x7f2ef40c5b40 2023-10-18T10:42:46.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2ef97c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40c89c0 con 0x7f2ee00047c0 2023-10-18T10:42:46.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2ef97c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40c8ca0 con 0x7f2ef40c64e0 2023-10-18T10:42:46.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ef40c5b40 0x7f2ef40c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:46.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ef40c5b40 0x7f2ef40c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42872/0 (socket says 172.21.15.125:42872) 2023-10-18T10:42:46.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 -- 172.21.15.125:0/948760148 learned_addr learned my addr 172.21.15.125:0/948760148 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:46.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2ef2ffd640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00047c0 0x7f2ef415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:46.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2ef37fe640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef40c64e0 0x7f2ef40ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:46.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 -- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef40c64e0 msgr2=0x7f2ef40ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef40c64e0 0x7f2ef40ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 -- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00047c0 msgr2=0x7f2ef415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00047c0 0x7f2ef415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 -- 172.21.15.125:0/948760148 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef40b48c0 con 0x7f2ef40c5b40 2023-10-18T10:42:46.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eebfff640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ef40c5b40 0x7f2ef40c5fa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2edc001930 tx=0x7f2edc02dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:46.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eeb7fe640 1 -- 172.21.15.125:0/948760148 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc03c040 con 0x7f2ef40c5b40 2023-10-18T10:42:46.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eeb7fe640 1 -- 172.21.15.125:0/948760148 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2edc003810 con 0x7f2ef40c5b40 2023-10-18T10:42:46.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2eeb7fe640 1 -- 172.21.15.125:0/948760148 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc003c20 con 0x7f2ef40c5b40 2023-10-18T10:42:46.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef40b4ba0 con 0x7f2ef40c5b40 2023-10-18T10:42:46.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.774+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef415a720 con 0x7f2ef40c5b40 2023-10-18T10:42:46.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.775+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef40675f0 con 0x7f2ef40c5b40 2023-10-18T10:42:46.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.776+0000 7f2eeb7fe640 1 -- 172.21.15.125:0/948760148 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f2edc032070 con 0x7f2ef40c5b40 2023-10-18T10:42:46.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.776+0000 7f2eeb7fe640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f2ed406b080 0x7f2ed406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:46.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.776+0000 7f2ef2ffd640 1 -- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f2ed406b080 msgr2=0x7f2ed406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:46.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.776+0000 7f2eeb7fe640 1 -- 172.21.15.125:0/948760148 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f2edc0ab4c0 con 0x7f2ef40c5b40 2023-10-18T10:42:46.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.776+0000 7f2ef2ffd640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f2ed406b080 0x7f2ed406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:46.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.780+0000 7f2eeb7fe640 1 -- 172.21.15.125:0/948760148 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2edc075e90 con 0x7f2ef40c5b40 2023-10-18T10:42:46.889 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:46.888+0000 7fa833e2f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:42:46.890 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:46.889+0000 7fc37a811040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:42:46.933 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:46.932+0000 7fa833e2f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:42:46.936 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:46.934+0000 7fc37a811040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:42:46.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.949+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ef406add0 con 0x7f2ef40c5b40 2023-10-18T10:42:46.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.951+0000 7f2eeb7fe640 1 -- 172.21.15.125:0/948760148 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+17478 (secure 0 0 0) 0x7f2edc046090 con 0x7f2ef40c5b40 2023-10-18T10:42:46.953 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:46.953 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":115,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:42:45.484333+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:41:26.603027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-18T10:41:27.743317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3890203496":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2970354256":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6833/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6832/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2064702534":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2430123574":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/3646248611":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/170859299":"2023-10-19T10:41:43.888680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:42:46.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.952+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f2ed406b080 msgr2=0x7f2ed406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:46.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.952+0000 7f2ef97c0640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f2ed406b080 0x7f2ed406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.952+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ef40c5b40 msgr2=0x7f2ef40c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:46.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.952+0000 7f2ef97c0640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ef40c5b40 0x7f2ef40c5fa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2edc001930 tx=0x7f2edc02dfa0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.953+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 shutdown_connections 2023-10-18T10:42:46.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.953+0000 7f2ef97c0640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f2ed406b080 0x7f2ed406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.953+0000 7f2ef97c0640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef40c64e0 0x7f2ef40ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.953+0000 7f2ef97c0640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ef40c5b40 0x7f2ef40c5fa0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.953+0000 7f2ef97c0640 1 --2- 172.21.15.125:0/948760148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00047c0 0x7f2ef415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:46.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.953+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 >> 172.21.15.125:0/948760148 conn(0x7f2ef409a030 msgr2=0x7f2ef40af050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:46.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.953+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 shutdown_connections 2023-10-18T10:42:46.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:46.953+0000 7f2ef97c0640 1 -- 172.21.15.125:0/948760148 wait complete. 2023-10-18T10:42:46.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:41:26.603027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:41:27.743317+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:42:46.967 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:42:47.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.071+0000 7f7a7d075640 1 -- 172.21.15.125:0/1599286703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a64004070 msgr2=0x7f7a640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.071+0000 7f7a7d075640 1 --2- 172.21.15.125:0/1599286703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a64004070 0x7f7a640047c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006f80 tx=0x7f7a640057f0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.071+0000 7f7a7759e640 1 -- 172.21.15.125:0/1599286703 reap_dead start 2023-10-18T10:42:47.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.071+0000 7f7a7d075640 1 -- 172.21.15.125:0/1599286703 shutdown_connections 2023-10-18T10:42:47.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.071+0000 7f7a7d075640 1 -- 172.21.15.125:0/1599286703 >> 172.21.15.125:0/1599286703 conn(0x7f7a780b0670 msgr2=0x7f7a780b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:47.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.071+0000 7f7a7d075640 1 -- 172.21.15.125:0/1599286703 shutdown_connections 2023-10-18T10:42:47.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.071+0000 7f7a7d075640 1 -- 172.21.15.125:0/1599286703 wait complete. 2023-10-18T10:42:47.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.072+0000 7f7a7d075640 1 Processor -- start 2023-10-18T10:42:47.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.072+0000 7f7a7d075640 1 -- start start 2023-10-18T10:42:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7d075640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a64004070 0x7f7a7815c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7d075640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a781f2d40 0x7f7a780c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7d075640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a7815a490 0x7f7a7815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7d075640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7815caf0 con 0x7f7a7815a490 2023-10-18T10:42:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7d075640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c5390 con 0x7f7a64004070 2023-10-18T10:42:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7d075640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a781f32b0 con 0x7f7a781f2d40 2023-10-18T10:42:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a7815a490 0x7f7a7815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a7815a490 0x7f7a7815a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42890/0 (socket says 172.21.15.125:42890) 2023-10-18T10:42:47.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 -- 172.21.15.125:0/2557414373 learned_addr learned my addr 172.21.15.125:0/2557414373 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a76d9d640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a64004070 0x7f7a7815c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a6ffff640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a781f2d40 0x7f7a780c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 -- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a781f2d40 msgr2=0x7f7a780c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a781f2d40 0x7f7a780c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 -- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a64004070 msgr2=0x7f7a7815c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a64004070 0x7f7a7815c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 -- 172.21.15.125:0/2557414373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a781f3590 con 0x7f7a7815a490 2023-10-18T10:42:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a76d9d640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a64004070 0x7f7a7815c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:42:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.073+0000 7f7a7759e640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a7815a490 0x7f7a7815a940 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7a780708a0 tx=0x7f7a70001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.074+0000 7f7a6f7fe640 1 -- 172.21.15.125:0/2557414373 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70010890 con 0x7f7a7815a490 2023-10-18T10:42:47.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.074+0000 7f7a6f7fe640 1 -- 172.21.15.125:0/2557414373 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7a70010ae0 con 0x7f7a7815a490 2023-10-18T10:42:47.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.074+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a781f3870 con 0x7f7a7815a490 2023-10-18T10:42:47.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.074+0000 7f7a6f7fe640 1 -- 172.21.15.125:0/2557414373 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70011460 con 0x7f7a7815a490 2023-10-18T10:42:47.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.074+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a780c36c0 con 0x7f7a7815a490 2023-10-18T10:42:47.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.075+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a780675f0 con 0x7f7a7815a490 2023-10-18T10:42:47.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.075+0000 7f7a6f7fe640 1 -- 172.21.15.125:0/2557414373 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f7a700115c0 con 0x7f7a7815a490 2023-10-18T10:42:47.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.075+0000 7f7a6f7fe640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f7a5806b0b0 0x7f7a5806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.076+0000 7f7a76d9d640 1 -- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f7a5806b0b0 msgr2=0x7f7a5806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:47.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.076+0000 7f7a6f7fe640 1 -- 172.21.15.125:0/2557414373 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f7a7008a7a0 con 0x7f7a7815a490 2023-10-18T10:42:47.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.076+0000 7f7a76d9d640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f7a5806b0b0 0x7f7a5806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:47.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.084+0000 7f7a6f7fe640 1 -- 172.21.15.125:0/2557414373 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a70055210 con 0x7f7a7815a490 2023-10-18T10:42:47.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.253+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a780b4aa0 con 0x7f7a7815a490 2023-10-18T10:42:47.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.253+0000 7f7a6f7fe640 1 -- 172.21.15.125:0/2557414373 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a70058830 con 0x7f7a7815a490 2023-10-18T10:42:47.256 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:42:47.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f7a5806b0b0 msgr2=0x7f7a5806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:47.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f7a5806b0b0 0x7f7a5806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a7815a490 msgr2=0x7f7a7815a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a7815a490 0x7f7a7815a940 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f7a780708a0 tx=0x7f7a70001af0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 shutdown_connections 2023-10-18T10:42:47.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f7a5806b0b0 0x7f7a5806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7a7815a490 0x7f7a7815a940 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a781f2d40 0x7f7a780c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 --2- 172.21.15.125:0/2557414373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a64004070 0x7f7a7815c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 >> 172.21.15.125:0/2557414373 conn(0x7f7a780b0670 msgr2=0x7f7a780af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:47.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.255+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 shutdown_connections 2023-10-18T10:42:47.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.256+0000 7f7a7d075640 1 -- 172.21.15.125:0/2557414373 wait complete. 2023-10-18T10:42:47.268 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:42:47.320 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.318+0000 7fa833e2f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:42:47.345 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.343+0000 7fc37a811040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:42:47.363 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.361+0000 7fa833e2f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:42:47.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.373+0000 7fb40459e640 1 -- 172.21.15.125:0/2034723359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13bd70 msgr2=0x7fb3fc070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.373+0000 7fb40459e640 1 --2- 172.21.15.125:0/2034723359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13bd70 0x7fb3fc070ce0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec005fd0 tx=0x7fb3ec02be10 comp rx=0 tx=0).stop 2023-10-18T10:42:47.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.373+0000 7fb40459e640 1 -- 172.21.15.125:0/2034723359 shutdown_connections 2023-10-18T10:42:47.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.373+0000 7fb40459e640 1 --2- 172.21.15.125:0/2034723359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3fc069720 0x7fb3fc069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.373+0000 7fb40459e640 1 --2- 172.21.15.125:0/2034723359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13bd70 0x7fb3fc070ce0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.373+0000 7fb40459e640 1 -- 172.21.15.125:0/2034723359 >> 172.21.15.125:0/2034723359 conn(0x7fb3fc0af730 msgr2=0x7fb3fc0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:47.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.373+0000 7fb40459e640 1 -- 172.21.15.125:0/2034723359 shutdown_connections 2023-10-18T10:42:47.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.373+0000 7fb40459e640 1 -- 172.21.15.125:0/2034723359 wait complete. 2023-10-18T10:42:47.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.374+0000 7fb40459e640 1 Processor -- start 2023-10-18T10:42:47.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.374+0000 7fb40459e640 1 -- start start 2023-10-18T10:42:47.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb40459e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3fc069720 0x7fb3fc135420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb40459e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3fc13bd70 0x7fb3fc15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb40459e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13e4d0 0x7fb3fc139f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb40459e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc159c00 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb40459e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc1f3d20 con 0x7fb3fc069720 2023-10-18T10:42:47.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb40459e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc1592d0 con 0x7fb3fc13bd70 2023-10-18T10:42:47.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb402b3b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13e4d0 0x7fb3fc139f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb402b3b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13e4d0 0x7fb3fc139f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42920/0 (socket says 172.21.15.125:42920) 2023-10-18T10:42:47.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb402b3b640 1 -- 172.21.15.125:0/2025714285 learned_addr learned my addr 172.21.15.125:0/2025714285 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:47.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb401b39640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3fc13bd70 0x7fb3fc15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.375+0000 7fb40233a640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3fc069720 0x7fb3fc135420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb402b3b640 1 -- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3fc13bd70 msgr2=0x7fb3fc15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb402b3b640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3fc13bd70 0x7fb3fc15b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb402b3b640 1 -- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3fc069720 msgr2=0x7fb3fc135420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb402b3b640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3fc069720 0x7fb3fc135420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb402b3b640 1 -- 172.21.15.125:0/2025714285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec009560 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb402b3b640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13e4d0 0x7fb3fc139f10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c6b0 tx=0x7fb3f400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:47.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb3f2ffd640 1 -- 172.21.15.125:0/2025714285 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4003b70 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb3f2ffd640 1 -- 172.21.15.125:0/2025714285 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3f4003cf0 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb3f2ffd640 1 -- 172.21.15.125:0/2025714285 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4005330 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc1595b0 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.376+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc15bbb0 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.377+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc0b0800 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.377+0000 7fb3f2ffd640 1 -- 172.21.15.125:0/2025714285 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fb3f4005490 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.378+0000 7fb3f2ffd640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fb3e006b080 0x7fb3e006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.378+0000 7fb3f2ffd640 1 -- 172.21.15.125:0/2025714285 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fb3f4089b50 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.378+0000 7fb40233a640 1 -- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fb3e006b080 msgr2=0x7fb3e006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:47.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.378+0000 7fb40233a640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fb3e006b080 0x7fb3e006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:47.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.382+0000 7fb3f2ffd640 1 -- 172.21.15.125:0/2025714285 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3f4054640 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.392 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.390+0000 7fc37a811040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:42:47.405 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.404+0000 7fa833e2f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:42:47.435 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.433+0000 7fc37a811040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:42:47.456 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.454+0000 7fa833e2f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:42:47.487 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.485+0000 7fc37a811040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:42:47.497 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.495+0000 7fa833e2f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:42:47.529 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.528+0000 7fc37a811040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:42:47.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.550+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3fc069190 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.550+0000 7fb3f2ffd640 1 -- 172.21.15.125:0/2025714285 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3f4057c60 con 0x7fb3fc13e4d0 2023-10-18T10:42:47.553 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:42:47.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fb3e006b080 msgr2=0x7fb3e006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:47.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fb3e006b080 0x7fb3e006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13e4d0 msgr2=0x7fb3fc139f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13e4d0 0x7fb3fc139f10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c6b0 tx=0x7fb3f400cb70 comp rx=0 tx=0).stop 2023-10-18T10:42:47.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 shutdown_connections 2023-10-18T10:42:47.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fb3e006b080 0x7fb3e006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3fc13e4d0 0x7fb3fc139f10 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3fc13bd70 0x7fb3fc15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 --2- 172.21.15.125:0/2025714285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3fc069720 0x7fb3fc135420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 >> 172.21.15.125:0/2025714285 conn(0x7fb3fc0af730 msgr2=0x7fb3fc06d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:47.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 shutdown_connections 2023-10-18T10:42:47.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.553+0000 7fb40459e640 1 -- 172.21.15.125:0/2025714285 wait complete. 2023-10-18T10:42:47.565 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:42:47.596 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.595+0000 7fa833e2f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:42:47.633 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.631+0000 7fc37a811040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:42:47.649 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.647+0000 7fa833e2f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:42:47.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.670+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3603343976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab80d1910 msgr2=0x7f0ab80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.670+0000 7f0ac03d0640 1 --2- 172.21.15.125:0/3603343976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab80d1910 0x7f0ab80b0650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0005fd0 tx=0x7f0ab002bfc0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.670+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3603343976 shutdown_connections 2023-10-18T10:42:47.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.670+0000 7f0ac03d0640 1 --2- 172.21.15.125:0/3603343976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab80d1910 0x7f0ab80b0650 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.670+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3603343976 >> 172.21.15.125:0/3603343976 conn(0x7f0ab80d0d60 msgr2=0x7f0ab80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:47.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.670+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3603343976 shutdown_connections 2023-10-18T10:42:47.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.670+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3603343976 wait complete. 2023-10-18T10:42:47.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.671+0000 7f0ac03d0640 1 Processor -- start 2023-10-18T10:42:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.671+0000 7f0ac03d0640 1 -- start start 2023-10-18T10:42:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0ac03d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ab80cf340 0x7f0ab81f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0ac03d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ab80d1910 0x7f0ab80ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0ac03d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab815b2a0 0x7f0ab80b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0ac03d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81f1410 con 0x7f0ab80cf340 2023-10-18T10:42:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0ac03d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81f4b10 con 0x7f0ab815b2a0 2023-10-18T10:42:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0ac03d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81f4db0 con 0x7f0ab80d1910 2023-10-18T10:42:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0abe16c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ab80cf340 0x7f0ab81f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0abe16c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ab80cf340 0x7f0ab81f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50220/0 (socket says 172.21.15.125:50220) 2023-10-18T10:42:47.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0abe16c640 1 -- 172.21.15.125:0/3502400328 learned_addr learned my addr 172.21.15.125:0/3502400328 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:47.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0abe96d640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab815b2a0 0x7f0ab80b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.672+0000 7f0abd96b640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ab80d1910 0x7f0ab80ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0abe96d640 1 -- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ab80d1910 msgr2=0x7f0ab80ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0abe96d640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ab80d1910 0x7f0ab80ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0abe96d640 1 -- 172.21.15.125:0/3502400328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ab80cf340 msgr2=0x7f0ab81f0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0abe96d640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ab80cf340 0x7f0ab81f0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0abe96d640 1 -- 172.21.15.125:0/3502400328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0009840 con 0x7f0ab815b2a0 2023-10-18T10:42:47.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0abe16c640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ab80cf340 0x7f0ab81f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:42:47.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0abe96d640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab815b2a0 0x7f0ab80b58f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009000 tx=0x7f0ab0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0aaeffd640 1 -- 172.21.15.125:0/3502400328 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00043d0 con 0x7f0ab815b2a0 2023-10-18T10:42:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0aaeffd640 1 -- 172.21.15.125:0/3502400328 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ab0004640 con 0x7f0ab815b2a0 2023-10-18T10:42:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab81f5090 con 0x7f0ab815b2a0 2023-10-18T10:42:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0aaeffd640 1 -- 172.21.15.125:0/3502400328 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab003e450 con 0x7f0ab815b2a0 2023-10-18T10:42:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.673+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab81f5650 con 0x7f0ab815b2a0 2023-10-18T10:42:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.674+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab81f4ef0 con 0x7f0ab815b2a0 2023-10-18T10:42:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.674+0000 7f0aaeffd640 1 -- 172.21.15.125:0/3502400328 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f0ab003e5b0 con 0x7f0ab815b2a0 2023-10-18T10:42:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.675+0000 7f0aaeffd640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0a9c06b080 0x7f0a9c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.675+0000 7f0abe16c640 1 -- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0a9c06b080 msgr2=0x7f0a9c06d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.675+0000 7f0aaeffd640 1 -- 172.21.15.125:0/3502400328 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f0ab00acb30 con 0x7f0ab815b2a0 2023-10-18T10:42:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.675+0000 7f0abe16c640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0a9c06b080 0x7f0a9c06d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.679+0000 7f0aaeffd640 1 -- 172.21.15.125:0/3502400328 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ab81f4ef0 con 0x7f0ab815b2a0 2023-10-18T10:42:47.689 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.688+0000 7fc37a811040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:42:47.691 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.690+0000 7fa833e2f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:42:47.732 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.730+0000 7fc37a811040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:42:47.740 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.738+0000 7fa833e2f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:42:47.781 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.779+0000 7fc37a811040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:42:47.782 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:47.780+0000 7fa833e2f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:42:47.824 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:47.822+0000 7fc37a811040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:42:47.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.848+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ab81f4ef0 con 0x7f0ab815b2a0 2023-10-18T10:42:47.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.849+0000 7f0aaeffd640 1 -- 172.21.15.125:0/3502400328 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0ab81f4ef0 con 0x7f0ab815b2a0 2023-10-18T10:42:47.851 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:42:47.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0a9c06b080 msgr2=0x7f0a9c06d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:47.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0a9c06b080 0x7f0a9c06d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab815b2a0 msgr2=0x7f0ab80b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab815b2a0 0x7f0ab80b58f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009000 tx=0x7f0ab0009310 comp rx=0 tx=0).stop 2023-10-18T10:42:47.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 shutdown_connections 2023-10-18T10:42:47.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0a9c06b080 0x7f0a9c06d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ab815b2a0 0x7f0ab80b58f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ab80d1910 0x7f0ab80ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 --2- 172.21.15.125:0/3502400328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ab80cf340 0x7f0ab81f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 >> 172.21.15.125:0/3502400328 conn(0x7f0ab80d0d60 msgr2=0x7f0ab80ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:47.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 shutdown_connections 2023-10-18T10:42:47.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.851+0000 7f0ac03d0640 1 -- 172.21.15.125:0/3502400328 wait complete. 2023-10-18T10:42:47.863 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:42:47.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.968+0000 7fc98e56e640 1 -- 172.21.15.125:0/1926277711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9700047c0 msgr2=0x7fc970004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.968+0000 7fc98e56e640 1 --2- 172.21.15.125:0/1926277711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9700047c0 0x7fc970004c40 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fc978009b50 tx=0x7fc9700057f0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.968+0000 7fc98e56e640 1 -- 172.21.15.125:0/1926277711 shutdown_connections 2023-10-18T10:42:47.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.968+0000 7fc98e56e640 1 --2- 172.21.15.125:0/1926277711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9700047c0 0x7fc970004c40 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.968+0000 7fc98e56e640 1 -- 172.21.15.125:0/1926277711 >> 172.21.15.125:0/1926277711 conn(0x7fc9880d0d60 msgr2=0x7fc9880d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:47.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.968+0000 7fc98e56e640 1 -- 172.21.15.125:0/1926277711 shutdown_connections 2023-10-18T10:42:47.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.968+0000 7fc98e56e640 1 -- 172.21.15.125:0/1926277711 wait complete. 2023-10-18T10:42:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.969+0000 7fc98e56e640 1 Processor -- start 2023-10-18T10:42:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.969+0000 7fc98e56e640 1 -- start start 2023-10-18T10:42:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98e56e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9700047c0 0x7fc9880b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98e56e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9880b2d90 0x7fc98814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98e56e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98814d8c0 0x7fc9880b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98e56e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880b3210 con 0x7fc98814d8c0 2023-10-18T10:42:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98e56e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880cad60 con 0x7fc9700047c0 2023-10-18T10:42:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98e56e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98814b0c0 con 0x7fc9880b2d90 2023-10-18T10:42:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98cb0b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98814d8c0 0x7fc9880b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98cb0b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98814d8c0 0x7fc9880b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50232/0 (socket says 172.21.15.125:50232) 2023-10-18T10:42:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98cb0b640 1 -- 172.21.15.125:0/360839199 learned_addr learned my addr 172.21.15.125:0/360839199 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc987fff640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9700047c0 0x7fc9880b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc9877fe640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9880b2d90 0x7fc98814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:47.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98cb0b640 1 -- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9880b2d90 msgr2=0x7fc98814d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98cb0b640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9880b2d90 0x7fc98814d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98cb0b640 1 -- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9700047c0 msgr2=0x7fc9880b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:47.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98cb0b640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9700047c0 0x7fc9880b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:47.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.970+0000 7fc98cb0b640 1 -- 172.21.15.125:0/360839199 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98814b3a0 con 0x7fc98814d8c0 2023-10-18T10:42:47.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.971+0000 7fc98cb0b640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98814d8c0 0x7fc9880b4a60 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fc9880b6e10 tx=0x7fc980006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:47.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.971+0000 7fc984ff9640 1 -- 172.21.15.125:0/360839199 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800119c0 con 0x7fc98814d8c0 2023-10-18T10:42:47.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.971+0000 7fc984ff9640 1 -- 172.21.15.125:0/360839199 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc980011b20 con 0x7fc98814d8c0 2023-10-18T10:42:47.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.971+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98814b680 con 0x7fc98814d8c0 2023-10-18T10:42:47.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.971+0000 7fc984ff9640 1 -- 172.21.15.125:0/360839199 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800103c0 con 0x7fc98814d8c0 2023-10-18T10:42:47.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.971+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9881591f0 con 0x7fc98814d8c0 2023-10-18T10:42:47.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.972+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98814af20 con 0x7fc98814d8c0 2023-10-18T10:42:47.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.972+0000 7fc984ff9640 1 -- 172.21.15.125:0/360839199 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fc980010520 con 0x7fc98814d8c0 2023-10-18T10:42:47.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.973+0000 7fc984ff9640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fc97406b0b0 0x7fc97406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:47.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.973+0000 7fc987fff640 1 -- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fc97406b0b0 msgr2=0x7fc97406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:47.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.973+0000 7fc984ff9640 1 -- 172.21.15.125:0/360839199 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fc98008a440 con 0x7fc98814d8c0 2023-10-18T10:42:47.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.973+0000 7fc987fff640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fc97406b0b0 0x7fc97406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:47.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:47.977+0000 7fc984ff9640 1 -- 172.21.15.125:0/360839199 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc98814af20 con 0x7fc98814d8c0 2023-10-18T10:42:48.098 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.096+0000 7fa833e2f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:42:48.151 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.149+0000 7fc37a811040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:42:48.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.155+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc98814af20 con 0x7fc98814d8c0 2023-10-18T10:42:48.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.155+0000 7fc984ff9640 1 -- 172.21.15.125:0/360839199 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v115) v1 ==== 80+0+477 (secure 0 0 0) 0x7fc98814af20 con 0x7fc98814d8c0 2023-10-18T10:42:48.157 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3890203496 2023-10-19T10:41:43.888680+0000 2023-10-18T10:42:48.158 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2970354256 2023-10-19T10:41:43.888680+0000 2023-10-18T10:42:48.158 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/1523403591 2023-10-19T10:41:43.888680+0000 2023-10-18T10:42:48.158 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/1523403591 2023-10-19T10:41:43.888680+0000 2023-10-18T10:42:48.159 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2064702534 2023-10-19T10:41:43.888680+0000 2023-10-18T10:42:48.160 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2430123574 2023-10-19T10:41:43.888680+0000 2023-10-18T10:42:48.160 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3646248611 2023-10-19T10:41:43.888680+0000 2023-10-18T10:42:48.160 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/170859299 2023-10-19T10:41:43.888680+0000 2023-10-18T10:42:48.161 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T10:42:48.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fc97406b0b0 msgr2=0x7fc97406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:48.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fc97406b0b0 0x7fc97406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98814d8c0 msgr2=0x7fc9880b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:48.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98814d8c0 0x7fc9880b4a60 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fc9880b6e10 tx=0x7fc980006430 comp rx=0 tx=0).stop 2023-10-18T10:42:48.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 shutdown_connections 2023-10-18T10:42:48.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7fc97406b0b0 0x7fc97406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98814d8c0 0x7fc9880b4a60 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9880b2d90 0x7fc98814d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 --2- 172.21.15.125:0/360839199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9700047c0 0x7fc9880b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.157+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 >> 172.21.15.125:0/360839199 conn(0x7fc9880d0d60 msgr2=0x7fc98815a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:48.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.158+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 shutdown_connections 2023-10-18T10:42:48.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.158+0000 7fc98e56e640 1 -- 172.21.15.125:0/360839199 wait complete. 2023-10-18T10:42:48.170 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:42:48.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:42:48.171 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T10:42:48.196 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[125724]: fuse finished with error 0 and tester_r 0 2023-10-18T10:42:48.198 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:42:48.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:42:48.199 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:42:48.228 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.226+0000 7fa833e2f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:42:48.288 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.287+0000 7fc37a811040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:42:48.318 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.316+0000 7fa833e2f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:42:48.319 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:42:48.320 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:42:48.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:42:48.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:42:48.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:42:48.336 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:42:48.362 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.360+0000 7fa833e2f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:42:48.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:42:48.376 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T10:42:48.376 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:42:48.377 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:42:48.382 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.381+0000 7fc37a811040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:42:48.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:42:48.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:42:48.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:42:48.393 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:42:48.427 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.425+0000 7fc37a811040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:42:48.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:42:48.441 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T10:42:48.442 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:42:48.442 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:42:48.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:42:48.458 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:42:48.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:42:48.459 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:42:48.462 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.460+0000 7fa833e2f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:42:48.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:42:48.476 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T10:42:48.476 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:42:48.528 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.526+0000 7fc37a811040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:42:48.555 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.554+0000 7fa833e2f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:42:48.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.584+0000 7f6d87573640 1 -- 172.21.15.125:0/1898368904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d800cf340 msgr2=0x7f6d800d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:48.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.584+0000 7f6d87573640 1 --2- 172.21.15.125:0/1898368904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d800cf340 0x7f6d800d1910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009830 tx=0x7f6d7802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.584+0000 7f6d87573640 1 -- 172.21.15.125:0/1898368904 shutdown_connections 2023-10-18T10:42:48.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.584+0000 7f6d87573640 1 --2- 172.21.15.125:0/1898368904 >> v2:172.21.15.139:3300/0 conn(0x7f6d8015b2a0 0x7f6d800cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.584+0000 7f6d87573640 1 --2- 172.21.15.125:0/1898368904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d800cf340 0x7f6d800d1910 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.584+0000 7f6d87573640 1 -- 172.21.15.125:0/1898368904 >> 172.21.15.125:0/1898368904 conn(0x7f6d800d0d60 msgr2=0x7f6d800d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:48.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.585+0000 7f6d87573640 1 -- 172.21.15.125:0/1898368904 shutdown_connections 2023-10-18T10:42:48.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.585+0000 7f6d87573640 1 -- 172.21.15.125:0/1898368904 wait complete. 2023-10-18T10:42:48.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.586+0000 7f6d87573640 1 Processor -- start 2023-10-18T10:42:48.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.586+0000 7f6d87573640 1 -- start start 2023-10-18T10:42:48.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.586+0000 7f6d87573640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d800ceb40 0x7f6d800ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:48.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.586+0000 7f6d87573640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d800cf340 0x7f6d80154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:48.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.586+0000 7f6d87573640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d8015b2a0 0x7f6d80154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:48.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.586+0000 7f6d87573640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d80159510 con 0x7f6d800ceb40 2023-10-18T10:42:48.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.586+0000 7f6d87573640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8006bd90 con 0x7f6d8015b2a0 2023-10-18T10:42:48.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.586+0000 7f6d87573640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800b64e0 con 0x7f6d800cf340 2023-10-18T10:42:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d800ceb40 0x7f6d800ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d800ceb40 0x7f6d800ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50236/0 (socket says 172.21.15.125:50236) 2023-10-18T10:42:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 -- 172.21.15.125:0/182828472 learned_addr learned my addr 172.21.15.125:0/182828472 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d84b0e640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d800cf340 0x7f6d80154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:48.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d85b10640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d8015b2a0 0x7f6d80154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 -- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d800cf340 msgr2=0x7f6d80154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d800cf340 0x7f6d80154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 -- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d8015b2a0 msgr2=0x7f6d80154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d8015b2a0 0x7f6d80154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 -- 172.21.15.125:0/182828472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70009490 con 0x7f6d800ceb40 2023-10-18T10:42:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d8530f640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d800ceb40 0x7f6d800ca270 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f6d78000c00 tx=0x7f6d78003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d75ffb640 1 -- 172.21.15.125:0/182828472 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78003e20 con 0x7f6d800ceb40 2023-10-18T10:42:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d75ffb640 1 -- 172.21.15.125:0/182828472 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d78034440 con 0x7f6d800ceb40 2023-10-18T10:42:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d75ffb640 1 -- 172.21.15.125:0/182828472 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78034850 con 0x7f6d800ceb40 2023-10-18T10:42:48.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d78009490 con 0x7f6d800ceb40 2023-10-18T10:42:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.587+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d800b6a10 con 0x7f6d800ceb40 2023-10-18T10:42:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.588+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d800690e0 con 0x7f6d800ceb40 2023-10-18T10:42:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.589+0000 7f6d75ffb640 1 -- 172.21.15.125:0/182828472 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f6d780349b0 con 0x7f6d800ceb40 2023-10-18T10:42:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.589+0000 7f6d75ffb640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f6d6406b080 0x7f6d6406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:48.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.589+0000 7f6d75ffb640 1 -- 172.21.15.125:0/182828472 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f6d780ac380 con 0x7f6d800ceb40 2023-10-18T10:42:48.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.589+0000 7f6d84b0e640 1 -- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f6d6406b080 msgr2=0x7f6d6406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:48.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.589+0000 7f6d84b0e640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f6d6406b080 0x7f6d6406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:48.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.594+0000 7f6d75ffb640 1 -- 172.21.15.125:0/182828472 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d78076e70 con 0x7f6d800ceb40 2023-10-18T10:42:48.625 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.623+0000 7fc37a811040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:42:48.693 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.691+0000 7fa833e2f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:42:48.735 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.733+0000 7fa833e2f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:42:48.767 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.765+0000 7fc37a811040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:42:48.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.790+0000 7f6d84b0e640 1 -- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f6d6406b080 msgr2=0x7f6d6406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:48.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.790+0000 7f6d84b0e640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f6d6406b080 0x7f6d6406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:42:48.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.805+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d800d0b40 con 0x7f6d800ceb40 2023-10-18T10:42:48.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.805+0000 7f6d75ffb640 1 -- 172.21.15.125:0/182828472 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f6d7807a490 con 0x7f6d800ceb40 2023-10-18T10:42:48.807 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:48.808 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":41,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2381546668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2381546668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2381546668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/193356301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":193356301},{"type":"v1","addr":"172.21.15.139:6836","nonce":193356301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":8073,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3589330474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3589330474},{"type":"v1","addr":"172.21.15.125:6834","nonce":3589330474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:41:28.760907+0000","modified":"2023-10-18T10:41:30.774086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8076":{"gid":8076,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/3342443842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3342443842},{"type":"v1","addr":"172.21.15.125:6835","nonce":3342443842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T10:42:48.808 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 41 2023-10-18T10:42:48.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f6d6406b080 msgr2=0x7f6d6406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:48.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f6d6406b080 0x7f6d6406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d800ceb40 msgr2=0x7f6d800ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:48.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d800ceb40 0x7f6d800ca270 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f6d78000c00 tx=0x7f6d78003820 comp rx=0 tx=0).stop 2023-10-18T10:42:48.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 shutdown_connections 2023-10-18T10:42:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f6d6406b080 0x7f6d6406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d8015b2a0 0x7f6d80154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d800cf340 0x7f6d80154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 --2- 172.21.15.125:0/182828472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d800ceb40 0x7f6d800ca270 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 >> 172.21.15.125:0/182828472 conn(0x7f6d800d0d60 msgr2=0x7f6d801e5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:48.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 shutdown_connections 2023-10-18T10:42:48.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.807+0000 7f6d87573640 1 -- 172.21.15.125:0/182828472 wait complete. 2023-10-18T10:42:48.812 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.809+0000 7fc37a811040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:42:48.820 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:42:48.840 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:48.839+0000 7fa833e2f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:42:48.923 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:48.921+0000 7fc37a811040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:42:48.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.922+0000 7f54b8d3c640 1 -- 172.21.15.125:0/803540675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b40ceb40 msgr2=0x7f54b40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:48.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.922+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/803540675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b40ceb40 0x7f54b40b0650 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f54b408a120 tx=0x7f54ac02c260 comp rx=0 tx=0).stop 2023-10-18T10:42:48.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.922+0000 7f54b8d3c640 1 -- 172.21.15.125:0/803540675 shutdown_connections 2023-10-18T10:42:48.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.922+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/803540675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b40ceb40 0x7f54b40b0650 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.922+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/803540675 >> v2:172.21.15.139:3300/0 conn(0x7f54b415b2a0 0x7f54b40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.922+0000 7f54b8d3c640 1 -- 172.21.15.125:0/803540675 >> 172.21.15.125:0/803540675 conn(0x7f54b40d0d60 msgr2=0x7f54b40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:48.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.922+0000 7f54b8d3c640 1 -- 172.21.15.125:0/803540675 shutdown_connections 2023-10-18T10:42:48.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.922+0000 7f54b8d3c640 1 -- 172.21.15.125:0/803540675 wait complete. 2023-10-18T10:42:48.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.924+0000 7f54b8d3c640 1 Processor -- start 2023-10-18T10:42:48.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.924+0000 7f54b8d3c640 1 -- start start 2023-10-18T10:42:48.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.924+0000 7f54b8d3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54b40ceb40 0x7f54b414f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:48.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.924+0000 7f54b8d3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54b40cf340 0x7f54b415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:48.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.924+0000 7f54b8d3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b415b2a0 0x7f54b40c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:48.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.924+0000 7f54b8d3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b415c130 con 0x7f54b415b2a0 2023-10-18T10:42:48.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.924+0000 7f54b8d3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b415d190 con 0x7f54b40cf340 2023-10-18T10:42:48.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.924+0000 7f54b8d3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b415c3d0 con 0x7f54b40ceb40 2023-10-18T10:42:48.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b415b2a0 0x7f54b40c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:48.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b415b2a0 0x7f54b40c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50258/0 (socket says 172.21.15.125:50258) 2023-10-18T10:42:48.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 -- 172.21.15.125:0/159206405 learned_addr learned my addr 172.21.15.125:0/159206405 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:48.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54abfff640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54b40cf340 0x7f54b415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:48.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b259c640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54b40ceb40 0x7f54b414f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:48.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 -- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54b40ceb40 msgr2=0x7f54b414f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:48.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54b40ceb40 0x7f54b414f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 -- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54b40cf340 msgr2=0x7f54b415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:48.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54b40cf340 0x7f54b415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:48.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 -- 172.21.15.125:0/159206405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac009900 con 0x7f54b415b2a0 2023-10-18T10:42:48.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b2d9d640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b415b2a0 0x7f54b40c8290 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009d60 tx=0x7f54ac003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:48.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54aaffd640 1 -- 172.21.15.125:0/159206405 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac004450 con 0x7f54b415b2a0 2023-10-18T10:42:48.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54aaffd640 1 -- 172.21.15.125:0/159206405 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f54ac0046a0 con 0x7f54b415b2a0 2023-10-18T10:42:48.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54aaffd640 1 -- 172.21.15.125:0/159206405 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0406b0 con 0x7f54b415b2a0 2023-10-18T10:42:48.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b415c6b0 con 0x7f54b415b2a0 2023-10-18T10:42:48.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.925+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b415cc70 con 0x7f54b415b2a0 2023-10-18T10:42:48.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.926+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b415ce60 con 0x7f54b415b2a0 2023-10-18T10:42:48.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.927+0000 7f54aaffd640 1 -- 172.21.15.125:0/159206405 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f54ac00a040 con 0x7f54b415b2a0 2023-10-18T10:42:48.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.927+0000 7f54aaffd640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f549406b0b0 0x7f549406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:48.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.927+0000 7f54b259c640 1 -- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f549406b0b0 msgr2=0x7f549406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:48.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.927+0000 7f54aaffd640 1 -- 172.21.15.125:0/159206405 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f54ac0acfd0 con 0x7f54b415b2a0 2023-10-18T10:42:48.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.927+0000 7f54b259c640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f549406b0b0 0x7f549406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:48.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:48.935+0000 7f54aaffd640 1 -- 172.21.15.125:0/159206405 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54ac077a20 con 0x7f54b415b2a0 2023-10-18T10:42:49.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.109+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54b40690e0 con 0x7f54b415b2a0 2023-10-18T10:42:49.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.111+0000 7f54aaffd640 1 -- 172.21.15.125:0/159206405 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+17478 (secure 0 0 0) 0x7f54ac07b040 con 0x7f54b415b2a0 2023-10-18T10:42:49.114 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:49.114 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":115,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:42:45.484333+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:41:26.603027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-18T10:41:27.743317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3890203496":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2970354256":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6833/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6832/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2064702534":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2430123574":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/3646248611":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/170859299":"2023-10-19T10:41:43.888680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:42:49.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f549406b0b0 msgr2=0x7f549406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:49.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f549406b0b0 0x7f549406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b415b2a0 msgr2=0x7f54b40c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b415b2a0 0x7f54b40c8290 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009d60 tx=0x7f54ac003c20 comp rx=0 tx=0).stop 2023-10-18T10:42:49.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 shutdown_connections 2023-10-18T10:42:49.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f549406b0b0 0x7f549406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54b40cf340 0x7f54b415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54b415b2a0 0x7f54b40c8290 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 --2- 172.21.15.125:0/159206405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54b40ceb40 0x7f54b414f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 >> 172.21.15.125:0/159206405 conn(0x7f54b40d0d60 msgr2=0x7f54b4151240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:49.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 shutdown_connections 2023-10-18T10:42:49.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.113+0000 7f54b8d3c640 1 -- 172.21.15.125:0/159206405 wait complete. 2023-10-18T10:42:49.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:41:26.603027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:41:27.743317+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:42:49.127 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:42:49.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.233+0000 7f77c7918640 1 -- 172.21.15.125:0/3322916418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 msgr2=0x7f77ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.233+0000 7f77c7918640 1 --2- 172.21.15.125:0/3322916418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 0x7f77ac00b980 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f77b8001d80 tx=0x7f77ac0057f0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.233+0000 7f77c7918640 1 -- 172.21.15.125:0/3322916418 shutdown_connections 2023-10-18T10:42:49.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.233+0000 7f77c7918640 1 --2- 172.21.15.125:0/3322916418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 0x7f77ac00b980 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.233+0000 7f77c7918640 1 -- 172.21.15.125:0/3322916418 >> 172.21.15.125:0/3322916418 conn(0x7f77c00d0d60 msgr2=0x7f77c00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:49.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.233+0000 7f77c7918640 1 -- 172.21.15.125:0/3322916418 shutdown_connections 2023-10-18T10:42:49.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.233+0000 7f77c7918640 1 -- 172.21.15.125:0/3322916418 wait complete. 2023-10-18T10:42:49.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.234+0000 7f77c7918640 1 Processor -- start 2023-10-18T10:42:49.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.234+0000 7f77c7918640 1 -- start start 2023-10-18T10:42:49.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c7918640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 0x7f77c01f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c7918640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c0158bf0 0x7f77c015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c7918640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b38e0 0x7f77c015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c7918640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c0157aa0 con 0x7f77ac005180 2023-10-18T10:42:49.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c7918640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c015c5d0 con 0x7f77c0158bf0 2023-10-18T10:42:49.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c7918640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c00b5b10 con 0x7f77c00b38e0 2023-10-18T10:42:49.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 0x7f77c01f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 0x7f77c01f0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50266/0 (socket says 172.21.15.125:50266) 2023-10-18T10:42:49.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 -- 172.21.15.125:0/1251535966 learned_addr learned my addr 172.21.15.125:0/1251535966 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:49.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c4eb3640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c0158bf0 0x7f77c015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c5eb5640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b38e0 0x7f77c015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 -- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b38e0 msgr2=0x7f77c015ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b38e0 0x7f77c015ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 -- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c0158bf0 msgr2=0x7f77c015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c0158bf0 0x7f77c015c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 -- 172.21.15.125:0/1251535966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c00b5df0 con 0x7f77ac005180 2023-10-18T10:42:49.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77c56b4640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 0x7f77c01f0d10 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0017a0 tx=0x7f77bc003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:49.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.235+0000 7f77b1ffb640 1 -- 172.21.15.125:0/1251535966 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc006a90 con 0x7f77ac005180 2023-10-18T10:42:49.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.236+0000 7f77b1ffb640 1 -- 172.21.15.125:0/1251535966 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f77bc00fad0 con 0x7f77ac005180 2023-10-18T10:42:49.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.236+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c00b60d0 con 0x7f77ac005180 2023-10-18T10:42:49.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.236+0000 7f77b1ffb640 1 -- 172.21.15.125:0/1251535966 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc00fe60 con 0x7f77ac005180 2023-10-18T10:42:49.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.236+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c00b3d60 con 0x7f77ac005180 2023-10-18T10:42:49.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.237+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c00690e0 con 0x7f77ac005180 2023-10-18T10:42:49.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.237+0000 7f77b1ffb640 1 -- 172.21.15.125:0/1251535966 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f77bc011430 con 0x7f77ac005180 2023-10-18T10:42:49.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.238+0000 7f77b1ffb640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f77a406b0b0 0x7f77a406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.238+0000 7f77c4eb3640 1 -- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f77a406b0b0 msgr2=0x7f77a406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:49.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.238+0000 7f77b1ffb640 1 -- 172.21.15.125:0/1251535966 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f77bc08aeb0 con 0x7f77ac005180 2023-10-18T10:42:49.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.238+0000 7f77c4eb3640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f77a406b0b0 0x7f77a406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:49.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.242+0000 7f77b1ffb640 1 -- 172.21.15.125:0/1251535966 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77bc055810 con 0x7f77ac005180 2023-10-18T10:42:49.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.416+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77c00cd840 con 0x7f77ac005180 2023-10-18T10:42:49.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.416+0000 7f77b1ffb640 1 -- 172.21.15.125:0/1251535966 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77bc0200a0 con 0x7f77ac005180 2023-10-18T10:42:49.418 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:42:49.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f77a406b0b0 msgr2=0x7f77a406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:49.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f77a406b0b0 0x7f77a406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 msgr2=0x7f77c01f0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 0x7f77c01f0d10 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0017a0 tx=0x7f77bc003cb0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 shutdown_connections 2023-10-18T10:42:49.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f77a406b0b0 0x7f77a406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77c0158bf0 0x7f77c015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77c00b38e0 0x7f77c015ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 --2- 172.21.15.125:0/1251535966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77ac005180 0x7f77c01f0d10 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 >> 172.21.15.125:0/1251535966 conn(0x7f77c00d0d60 msgr2=0x7f77c006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:49.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 shutdown_connections 2023-10-18T10:42:49.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.418+0000 7f77c7918640 1 -- 172.21.15.125:0/1251535966 wait complete. 2023-10-18T10:42:49.431 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:42:49.456 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.454+0000 7fa833e2f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:42:49.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.539+0000 7f5aad773640 1 -- 172.21.15.125:0/717052048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aa80b1980 msgr2=0x7f5aa8070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.539+0000 7f5aad773640 1 --2- 172.21.15.125:0/717052048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aa80b1980 0x7f5aa8070080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009830 tx=0x7f5a9802bd60 comp rx=0 tx=0).stop 2023-10-18T10:42:49.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.539+0000 7f5aad773640 1 -- 172.21.15.125:0/717052048 shutdown_connections 2023-10-18T10:42:49.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.539+0000 7f5aad773640 1 --2- 172.21.15.125:0/717052048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aa80b1980 0x7f5aa8070080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.539+0000 7f5aad773640 1 --2- 172.21.15.125:0/717052048 >> v2:172.21.15.139:3300/0 conn(0x7f5aa81416f0 0x7f5aa80b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.539+0000 7f5aad773640 1 -- 172.21.15.125:0/717052048 >> 172.21.15.125:0/717052048 conn(0x7f5aa80af730 msgr2=0x7f5aa80afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:49.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.539+0000 7f5aad773640 1 -- 172.21.15.125:0/717052048 shutdown_connections 2023-10-18T10:42:49.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.539+0000 7f5aad773640 1 -- 172.21.15.125:0/717052048 wait complete. 2023-10-18T10:42:49.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aad773640 1 Processor -- start 2023-10-18T10:42:49.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aad773640 1 -- start start 2023-10-18T10:42:49.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aad773640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5aa80705c0 0x7f5aa80acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aad773640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aa80b1980 0x7f5aa81f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aad773640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5aa81416f0 0x7f5aa8158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aad773640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa81f5440 con 0x7f5aa80705c0 2023-10-18T10:42:49.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aad773640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa815a980 con 0x7f5aa81416f0 2023-10-18T10:42:49.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aad773640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa815ac60 con 0x7f5aa80b1980 2023-10-18T10:42:49.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aa6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5aa80705c0 0x7f5aa80acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aa6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5aa80705c0 0x7f5aa80acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50272/0 (socket says 172.21.15.125:50272) 2023-10-18T10:42:49.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aa6ffd640 1 -- 172.21.15.125:0/4207351106 learned_addr learned my addr 172.21.15.125:0/4207351106 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:49.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aa67fc640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aa80b1980 0x7f5aa81f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.541+0000 7f5aa77fe640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5aa81416f0 0x7f5aa8158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5aa6ffd640 1 -- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aa80b1980 msgr2=0x7f5aa81f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5aa6ffd640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aa80b1980 0x7f5aa81f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5aa6ffd640 1 -- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5aa81416f0 msgr2=0x7f5aa8158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5aa6ffd640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5aa81416f0 0x7f5aa8158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5aa6ffd640 1 -- 172.21.15.125:0/4207351106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98009490 con 0x7f5aa80705c0 2023-10-18T10:42:49.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5aa6ffd640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5aa80705c0 0x7f5aa80acd60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00e790 tx=0x7f5a9c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:49.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5a8f7fe640 1 -- 172.21.15.125:0/4207351106 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c009580 con 0x7f5aa80705c0 2023-10-18T10:42:49.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5a8f7fe640 1 -- 172.21.15.125:0/4207351106 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a9c00cdb0 con 0x7f5aa80705c0 2023-10-18T10:42:49.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa815af40 con 0x7f5aa80705c0 2023-10-18T10:42:49.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5a8f7fe640 1 -- 172.21.15.125:0/4207351106 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c010690 con 0x7f5aa80705c0 2023-10-18T10:42:49.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.542+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa80ad2a0 con 0x7f5aa80705c0 2023-10-18T10:42:49.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.543+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa80b1e60 con 0x7f5aa80705c0 2023-10-18T10:42:49.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.544+0000 7f5a8f7fe640 1 -- 172.21.15.125:0/4207351106 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f5a9c0107f0 con 0x7f5aa80705c0 2023-10-18T10:42:49.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.544+0000 7f5a8f7fe640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f5a9406b080 0x7f5a9406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.544+0000 7f5a8f7fe640 1 -- 172.21.15.125:0/4207351106 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f5a9c014070 con 0x7f5aa80705c0 2023-10-18T10:42:49.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.544+0000 7f5aa67fc640 1 -- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f5a9406b080 msgr2=0x7f5a9406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:49.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.544+0000 7f5aa67fc640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f5a9406b080 0x7f5a9406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:49.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.552+0000 7f5a8f7fe640 1 -- 172.21.15.125:0/4207351106 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a9c051ca0 con 0x7f5aa80705c0 2023-10-18T10:42:49.589 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:49.587+0000 7fc37a811040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:42:49.718 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.716+0000 7fa833e2f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:42:49.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.719+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5aa80675f0 con 0x7f5aa80705c0 2023-10-18T10:42:49.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.720+0000 7f5a8f7fe640 1 -- 172.21.15.125:0/4207351106 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a9c055240 con 0x7f5aa80705c0 2023-10-18T10:42:49.722 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:42:49.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f5a9406b080 msgr2=0x7f5a9406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:49.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f5a9406b080 0x7f5a9406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5aa80705c0 msgr2=0x7f5aa80acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5aa80705c0 0x7f5aa80acd60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00e790 tx=0x7f5a9c00ec50 comp rx=0 tx=0).stop 2023-10-18T10:42:49.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 shutdown_connections 2023-10-18T10:42:49.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f5a9406b080 0x7f5a9406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5aa81416f0 0x7f5aa8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aa80b1980 0x7f5aa81f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 --2- 172.21.15.125:0/4207351106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5aa80705c0 0x7f5aa80acd60 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 >> 172.21.15.125:0/4207351106 conn(0x7f5aa80af730 msgr2=0x7f5aa8157600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:49.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 shutdown_connections 2023-10-18T10:42:49.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.722+0000 7f5aad773640 1 -- 172.21.15.125:0/4207351106 wait complete. 2023-10-18T10:42:49.735 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:42:49.766 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.764+0000 7fa833e2f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:42:49.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.838+0000 7f0e5afdc640 1 -- 172.21.15.125:0/803014783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e5415b2a0 msgr2=0x7f0e540d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.838+0000 7f0e5afdc640 1 --2- 172.21.15.125:0/803014783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e5415b2a0 0x7f0e540d1910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009830 tx=0x7f0e4802bc30 comp rx=0 tx=0).stop 2023-10-18T10:42:49.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.839+0000 7f0e5afdc640 1 -- 172.21.15.125:0/803014783 shutdown_connections 2023-10-18T10:42:49.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.839+0000 7f0e5afdc640 1 --2- 172.21.15.125:0/803014783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e540ceb40 0x7f0e540b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.839+0000 7f0e5afdc640 1 --2- 172.21.15.125:0/803014783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e5415b2a0 0x7f0e540d1910 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.839+0000 7f0e5afdc640 1 -- 172.21.15.125:0/803014783 >> 172.21.15.125:0/803014783 conn(0x7f0e540d0d60 msgr2=0x7f0e540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:49.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.839+0000 7f0e5afdc640 1 -- 172.21.15.125:0/803014783 shutdown_connections 2023-10-18T10:42:49.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.839+0000 7f0e5afdc640 1 -- 172.21.15.125:0/803014783 wait complete. 2023-10-18T10:42:49.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.840+0000 7f0e5afdc640 1 Processor -- start 2023-10-18T10:42:49.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.840+0000 7f0e5afdc640 1 -- start start 2023-10-18T10:42:49.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e5afdc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e540ceb40 0x7f0e541581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e5afdc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e540cf340 0x7f0e5415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e5afdc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e5415b2a0 0x7f0e540cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e5afdc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5415c480 con 0x7f0e5415b2a0 2023-10-18T10:42:49.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e5afdc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5415b800 con 0x7f0e540ceb40 2023-10-18T10:42:49.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e5afdc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5415bae0 con 0x7f0e540cf340 2023-10-18T10:42:49.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e5415b2a0 0x7f0e540cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e5415b2a0 0x7f0e540cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50282/0 (socket says 172.21.15.125:50282) 2023-10-18T10:42:49.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e53fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e540cf340 0x7f0e5415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 -- 172.21.15.125:0/697672440 learned_addr learned my addr 172.21.15.125:0/697672440 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:49.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e58d78640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e540ceb40 0x7f0e541581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:49.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 -- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e540cf340 msgr2=0x7f0e5415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e540cf340 0x7f0e5415cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 -- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e540ceb40 msgr2=0x7f0e541581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:49.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e540ceb40 0x7f0e541581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 -- 172.21.15.125:0/697672440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44009560 con 0x7f0e5415b2a0 2023-10-18T10:42:49.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e59579640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e5415b2a0 0x7f0e540cc880 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400e940 tx=0x7f0e4400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:49.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4400cd80 con 0x7f0e5415b2a0 2023-10-18T10:42:49.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0e44004030 con 0x7f0e5415b2a0 2023-10-18T10:42:49.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e48009490 con 0x7f0e5415b2a0 2023-10-18T10:42:49.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44004440 con 0x7f0e5415b2a0 2023-10-18T10:42:49.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.841+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5415c0a0 con 0x7f0e5415b2a0 2023-10-18T10:42:49.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.842+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5415c5c0 con 0x7f0e5415b2a0 2023-10-18T10:42:49.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.843+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 56) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f0e440035f0 con 0x7f0e5415b2a0 2023-10-18T10:42:49.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.843+0000 7f0e517fa640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0e4006b080 0x7f0e4006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:49.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.843+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f0e44014070 con 0x7f0e5415b2a0 2023-10-18T10:42:49.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.844+0000 7f0e58d78640 1 -- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0e4006b080 msgr2=0x7f0e4006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4208657878 2023-10-18T10:42:49.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.844+0000 7f0e58d78640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0e4006b080 0x7f0e4006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:42:49.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.847+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e5415c5c0 con 0x7f0e5415b2a0 2023-10-18T10:42:49.858 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:49.857+0000 7fc37a811040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:42:49.866 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.865+0000 7fa833e2f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:42:49.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.886+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 57) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f0e44055770 con 0x7f0e5415b2a0 2023-10-18T10:42:49.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.886+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0e4006b080 msgr2=0x7f0e4006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:42:49.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:49.886+0000 7f0e517fa640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0e4006b080 0x7f0e4006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:49.913 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:49.911+0000 7fc37a811040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:42:49.943 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b88ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.943 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b88ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.944 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b88ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.944 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b88ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.944 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b88ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.944 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b58b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.944 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b58b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.945 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b58b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.945 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b58b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:49.945 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:42:49.941+0000 7fa7b58b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:42:50.018 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:42:50.016+0000 7fc37a811040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:42:50.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.024+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e5415c5c0 con 0x7f0e5415b2a0 2023-10-18T10:42:50.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.025+0000 7f0e517fa640 1 -- 172.21.15.125:0/697672440 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e5415c5c0 con 0x7f0e5415b2a0 2023-10-18T10:42:50.027 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:42:50.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e5415b2a0 msgr2=0x7f0e540cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e5415b2a0 0x7f0e540cc880 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400e940 tx=0x7f0e4400ee00 comp rx=0 tx=0).stop 2023-10-18T10:42:50.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 shutdown_connections 2023-10-18T10:42:50.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:6832/4208657878,v1:172.21.15.139:6833/4208657878] conn(0x7f0e4006b080 0x7f0e4006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e5415b2a0 0x7f0e540cc880 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e540cf340 0x7f0e5415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 --2- 172.21.15.125:0/697672440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e540ceb40 0x7f0e541581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 >> 172.21.15.125:0/697672440 conn(0x7f0e540d0d60 msgr2=0x7f0e5415a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:50.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 shutdown_connections 2023-10-18T10:42:50.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.027+0000 7f0e5afdc640 1 -- 172.21.15.125:0/697672440 wait complete. 2023-10-18T10:42:50.039 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:42:50.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.141+0000 7fb442e44640 1 -- 172.21.15.125:0/2910475024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 msgr2=0x7fb43c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.141+0000 7fb442e44640 1 --2- 172.21.15.125:0/2910475024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 0x7fb43c0d1910 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb430009830 tx=0x7fb43002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.141+0000 7fb442e44640 1 -- 172.21.15.125:0/2910475024 shutdown_connections 2023-10-18T10:42:50.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.141+0000 7fb442e44640 1 --2- 172.21.15.125:0/2910475024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 0x7fb43c0d1910 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.141+0000 7fb442e44640 1 -- 172.21.15.125:0/2910475024 >> 172.21.15.125:0/2910475024 conn(0x7fb43c0d0d60 msgr2=0x7fb43c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:50.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.141+0000 7fb442e44640 1 -- 172.21.15.125:0/2910475024 shutdown_connections 2023-10-18T10:42:50.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.141+0000 7fb442e44640 1 -- 172.21.15.125:0/2910475024 wait complete. 2023-10-18T10:42:50.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.142+0000 7fb442e44640 1 Processor -- start 2023-10-18T10:42:50.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.142+0000 7fb442e44640 1 -- start start 2023-10-18T10:42:50.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb442e44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c0ceb40 0x7fb43c0b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb442e44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 0x7fb43c0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb442e44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb43c15b2a0 0x7fb43c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb442e44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c14e530 con 0x7fb43c0cf340 2023-10-18T10:42:50.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb442e44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c14db90 con 0x7fb43c0ceb40 2023-10-18T10:42:50.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb442e44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c14de70 con 0x7fb43c15b2a0 2023-10-18T10:42:50.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb43bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 0x7fb43c0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb43bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 0x7fb43c0b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50286/0 (socket says 172.21.15.125:50286) 2023-10-18T10:42:50.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.143+0000 7fb43bfff640 1 -- 172.21.15.125:0/1490490758 learned_addr learned my addr 172.21.15.125:0/1490490758 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:50.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb43bfff640 1 -- 172.21.15.125:0/1490490758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb43c15b2a0 msgr2=0x7fb43c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb4413e1640 1 --2- 172.21.15.125:0/1490490758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb43c15b2a0 0x7fb43c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb43bfff640 1 --2- 172.21.15.125:0/1490490758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb43c15b2a0 0x7fb43c0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb43bfff640 1 -- 172.21.15.125:0/1490490758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c0ceb40 msgr2=0x7fb43c0b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb43bfff640 1 --2- 172.21.15.125:0/1490490758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c0ceb40 0x7fb43c0b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb43bfff640 1 -- 172.21.15.125:0/1490490758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb430009490 con 0x7fb43c0cf340 2023-10-18T10:42:50.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb43bfff640 1 --2- 172.21.15.125:0/1490490758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 0x7fb43c0b4fe0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b3f0 tx=0x7fb42800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:50.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb4397fa640 1 -- 172.21.15.125:0/1490490758 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428011830 con 0x7fb43c0cf340 2023-10-18T10:42:50.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb4397fa640 1 -- 172.21.15.125:0/1490490758 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb428011a80 con 0x7fb43c0cf340 2023-10-18T10:42:50.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb4397fa640 1 -- 172.21.15.125:0/1490490758 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428011e90 con 0x7fb43c0cf340 2023-10-18T10:42:50.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c14e150 con 0x7fb43c0cf340 2023-10-18T10:42:50.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.144+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c15df30 con 0x7fb43c0cf340 2023-10-18T10:42:50.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.145+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c0d04f0 con 0x7fb43c0cf340 2023-10-18T10:42:50.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.145+0000 7fb4397fa640 1 -- 172.21.15.125:0/1490490758 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 57) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fb42801a400 con 0x7fb43c0cf340 2023-10-18T10:42:50.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.146+0000 7fb4397fa640 1 -- 172.21.15.125:0/1490490758 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6919+0+0 (secure 0 0 0) 0x7fb428089670 con 0x7fb43c0cf340 2023-10-18T10:42:50.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.150+0000 7fb4397fa640 1 -- 172.21.15.125:0/1490490758 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb4280546b0 con 0x7fb43c0cf340 2023-10-18T10:42:50.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.347+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb43c158e50 con 0x7fb43c0cf340 2023-10-18T10:42:50.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.347+0000 7fb4397fa640 1 -- 172.21.15.125:0/1490490758 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fb428057cd0 con 0x7fb43c0cf340 2023-10-18T10:42:50.349 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:50.350 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":41,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2381546668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2381546668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2381546668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/193356301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":193356301},{"type":"v1","addr":"172.21.15.139:6836","nonce":193356301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":8073,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3589330474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3589330474},{"type":"v1","addr":"172.21.15.125:6834","nonce":3589330474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:41:28.760907+0000","modified":"2023-10-18T10:41:30.774086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8076":{"gid":8076,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/3342443842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3342443842},{"type":"v1","addr":"172.21.15.125:6835","nonce":3342443842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T10:42:50.351 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 41 2023-10-18T10:42:50.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 msgr2=0x7fb43c0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 --2- 172.21.15.125:0/1490490758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 0x7fb43c0b4fe0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b3f0 tx=0x7fb42800b8b0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 shutdown_connections 2023-10-18T10:42:50.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 --2- 172.21.15.125:0/1490490758 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb43c15b2a0 0x7fb43c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 --2- 172.21.15.125:0/1490490758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb43c0cf340 0x7fb43c0b4fe0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 --2- 172.21.15.125:0/1490490758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c0ceb40 0x7fb43c0b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 >> 172.21.15.125:0/1490490758 conn(0x7fb43c0d0d60 msgr2=0x7fb43c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:50.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 shutdown_connections 2023-10-18T10:42:50.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.349+0000 7fb442e44640 1 -- 172.21.15.125:0/1490490758 wait complete. 2023-10-18T10:42:50.362 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:42:50.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.462+0000 7f2b36d05640 1 -- 172.21.15.125:0/375994375 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b180047c0 msgr2=0x7f2b18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.462+0000 7f2b36d05640 1 --2- 172.21.15.125:0/375994375 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b180047c0 0x7f2b18004c40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2b24001680 tx=0x7f2b180057f0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.462+0000 7f2b352a2640 1 -- 172.21.15.125:0/375994375 reap_dead start 2023-10-18T10:42:50.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.462+0000 7f2b36d05640 1 -- 172.21.15.125:0/375994375 shutdown_connections 2023-10-18T10:42:50.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.462+0000 7f2b36d05640 1 -- 172.21.15.125:0/375994375 >> 172.21.15.125:0/375994375 conn(0x7f2b30069930 msgr2=0x7f2b30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:50.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.462+0000 7f2b36d05640 1 -- 172.21.15.125:0/375994375 shutdown_connections 2023-10-18T10:42:50.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.462+0000 7f2b36d05640 1 -- 172.21.15.125:0/375994375 wait complete. 2023-10-18T10:42:50.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.463+0000 7f2b36d05640 1 Processor -- start 2023-10-18T10:42:50.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.463+0000 7f2b36d05640 1 -- start start 2023-10-18T10:42:50.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b36d05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b180047c0 0x7f2b3015be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b36d05640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b3013ad30 0x7f2b3013a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b36d05640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b3015d5e0 0x7f2b3015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b36d05640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3013b5b0 con 0x7f2b3013ad30 2023-10-18T10:42:50.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b36d05640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3013ab00 con 0x7f2b180047c0 2023-10-18T10:42:50.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b36d05640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3015c540 con 0x7f2b3015d5e0 2023-10-18T10:42:50.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b2ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b3013ad30 0x7f2b3013a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b2ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b3013ad30 0x7f2b3013a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50304/0 (socket says 172.21.15.125:50304) 2023-10-18T10:42:50.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.464+0000 7f2b2ffff640 1 -- 172.21.15.125:0/2335106154 learned_addr learned my addr 172.21.15.125:0/2335106154 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:50.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b352a2640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b3015d5e0 0x7f2b3015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b34aa1640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b180047c0 0x7f2b3015be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2ffff640 1 -- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b3015d5e0 msgr2=0x7f2b3015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2ffff640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b3015d5e0 0x7f2b3015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2ffff640 1 -- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b180047c0 msgr2=0x7f2b3015be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2ffff640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b180047c0 0x7f2b3015be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2ffff640 1 -- 172.21.15.125:0/2335106154 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3015c820 con 0x7f2b3013ad30 2023-10-18T10:42:50.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b34aa1640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b180047c0 0x7f2b3015be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:42:50.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2ffff640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b3013ad30 0x7f2b3013a5c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2b240017a0 tx=0x7f2b240035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:50.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2d7fa640 1 -- 172.21.15.125:0/2335106154 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24003bf0 con 0x7f2b3013ad30 2023-10-18T10:42:50.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2d7fa640 1 -- 172.21.15.125:0/2335106154 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2b24003e40 con 0x7f2b3013ad30 2023-10-18T10:42:50.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3015cb00 con 0x7f2b3013ad30 2023-10-18T10:42:50.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b2d7fa640 1 -- 172.21.15.125:0/2335106154 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2402c7b0 con 0x7f2b3013ad30 2023-10-18T10:42:50.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.465+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3015d0c0 con 0x7f2b3013ad30 2023-10-18T10:42:50.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.466+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b300690e0 con 0x7f2b3013ad30 2023-10-18T10:42:50.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.466+0000 7f2b2d7fa640 1 -- 172.21.15.125:0/2335106154 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 57) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f2b2402c910 con 0x7f2b3013ad30 2023-10-18T10:42:50.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.467+0000 7f2b2d7fa640 1 -- 172.21.15.125:0/2335106154 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f2b240aa9a0 con 0x7f2b3013ad30 2023-10-18T10:42:50.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.471+0000 7f2b2d7fa640 1 -- 172.21.15.125:0/2335106154 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b24075860 con 0x7f2b3013ad30 2023-10-18T10:42:50.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.638+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b300b0dd0 con 0x7f2b3013ad30 2023-10-18T10:42:50.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.640+0000 7f2b2d7fa640 1 -- 172.21.15.125:0/2335106154 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+18113 (secure 0 0 0) 0x7f2b24043090 con 0x7f2b3013ad30 2023-10-18T10:42:50.642 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:50.642 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":116,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:42:49.879712+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:41:26.603027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-18T10:41:27.743317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/1923522040":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3280996584":"2023-10-19T10:42:49.879669+0000","172.21.15.139:6832/4208657878":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2295776957":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2846025337":"2023-10-19T10:42:49.879669+0000","172.21.15.139:6833/4208657878":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2600842084":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2652835634":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/397760040":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3890203496":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2970354256":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6833/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6832/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2064702534":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2430123574":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2593987833":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3646248611":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/170859299":"2023-10-19T10:41:43.888680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:42:50.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.641+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b3013ad30 msgr2=0x7f2b3013a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.641+0000 7f2b36d05640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b3013ad30 0x7f2b3013a5c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2b240017a0 tx=0x7f2b240035f0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.642+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 shutdown_connections 2023-10-18T10:42:50.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.642+0000 7f2b36d05640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b3015d5e0 0x7f2b3015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.642+0000 7f2b36d05640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b3013ad30 0x7f2b3013a5c0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.642+0000 7f2b36d05640 1 --2- 172.21.15.125:0/2335106154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b180047c0 0x7f2b3015be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.642+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 >> 172.21.15.125:0/2335106154 conn(0x7f2b30069930 msgr2=0x7f2b30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:50.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.642+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 shutdown_connections 2023-10-18T10:42:50.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.642+0000 7f2b36d05640 1 -- 172.21.15.125:0/2335106154 wait complete. 2023-10-18T10:42:50.654 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:42:50.655 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:42:50.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.755+0000 7f72eefa9640 1 -- 172.21.15.125:0/795018035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72d4005180 msgr2=0x7f72d400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.755+0000 7f72eefa9640 1 --2- 172.21.15.125:0/795018035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72d4005180 0x7f72d400b980 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f72d800bb20 tx=0x7f72d40057f0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.755+0000 7f72eefa9640 1 -- 172.21.15.125:0/795018035 shutdown_connections 2023-10-18T10:42:50.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.755+0000 7f72eefa9640 1 --2- 172.21.15.125:0/795018035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72d4005180 0x7f72d400b980 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.755+0000 7f72eefa9640 1 -- 172.21.15.125:0/795018035 >> 172.21.15.125:0/795018035 conn(0x7f72e8069930 msgr2=0x7f72e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:50.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.755+0000 7f72eefa9640 1 -- 172.21.15.125:0/795018035 shutdown_connections 2023-10-18T10:42:50.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.755+0000 7f72eefa9640 1 -- 172.21.15.125:0/795018035 wait complete. 2023-10-18T10:42:50.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.756+0000 7f72eefa9640 1 Processor -- start 2023-10-18T10:42:50.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.756+0000 7f72eefa9640 1 -- start start 2023-10-18T10:42:50.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72eefa9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72d4005180 0x7f72e80b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72eefa9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72e81526c0 0x7f72e8151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72eefa9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72e8152b20 0x7f72e81518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72eefa9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e813b4f0 con 0x7f72e8152b20 2023-10-18T10:42:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72eefa9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e812a0e0 con 0x7f72e81526c0 2023-10-18T10:42:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72eefa9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e80aef10 con 0x7f72d4005180 2023-10-18T10:42:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72e8152b20 0x7f72e81518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72e8152b20 0x7f72e81518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50322/0 (socket says 172.21.15.125:50322) 2023-10-18T10:42:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 -- 172.21.15.125:0/4194905884 learned_addr learned my addr 172.21.15.125:0/4194905884 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72e7fff640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72e81526c0 0x7f72e8151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ecd45640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72d4005180 0x7f72e80b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 -- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72d4005180 msgr2=0x7f72e80b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72d4005180 0x7f72e80b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 -- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72e81526c0 msgr2=0x7f72e8151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72e81526c0 0x7f72e8151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 -- 172.21.15.125:0/4194905884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e80b1d90 con 0x7f72e8152b20 2023-10-18T10:42:50.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72e7fff640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72e81526c0 0x7f72e8151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:42:50.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.757+0000 7f72ed546640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72e8152b20 0x7f72e81518c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f72d802c7c0 tx=0x7f72d802ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:50.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.758+0000 7f72e57fa640 1 -- 172.21.15.125:0/4194905884 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8003bc0 con 0x7f72e8152b20 2023-10-18T10:42:50.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.758+0000 7f72e57fa640 1 -- 172.21.15.125:0/4194905884 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f72d8003e10 con 0x7f72e8152b20 2023-10-18T10:42:50.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.758+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e812a330 con 0x7f72e8152b20 2023-10-18T10:42:50.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.758+0000 7f72e57fa640 1 -- 172.21.15.125:0/4194905884 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8036670 con 0x7f72e8152b20 2023-10-18T10:42:50.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.758+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e812a8f0 con 0x7f72e8152b20 2023-10-18T10:42:50.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.759+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e812ab10 con 0x7f72e8152b20 2023-10-18T10:42:50.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.759+0000 7f72e57fa640 1 -- 172.21.15.125:0/4194905884 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 57) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f72d80367d0 con 0x7f72e8152b20 2023-10-18T10:42:50.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.759+0000 7f72e57fa640 1 -- 172.21.15.125:0/4194905884 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f72d80aac50 con 0x7f72e8152b20 2023-10-18T10:42:50.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.764+0000 7f72e57fa640 1 -- 172.21.15.125:0/4194905884 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72d8075c90 con 0x7f72e8152b20 2023-10-18T10:42:50.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.888+0000 7f72e57fa640 1 -- 172.21.15.125:0/4194905884 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 58) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f72d8075740 con 0x7f72e8152b20 2023-10-18T10:42:50.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.888+0000 7f72e57fa640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f72c8070d70 0x7f72c8073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:50.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.891+0000 7f72ecd45640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f72c8070d70 0x7f72c8073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:50.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.893+0000 7f72ecd45640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f72c8070d70 0x7f72c8073160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f72e0000f80 tx=0x7f72e0006eb0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:50.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.968+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72e806c240 con 0x7f72e8152b20 2023-10-18T10:42:50.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.968+0000 7f72e57fa640 1 -- 172.21.15.125:0/4194905884 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f72d8032070 con 0x7f72e8152b20 2023-10-18T10:42:50.970 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:50.971 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":41,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2381546668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2381546668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2381546668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/193356301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":193356301},{"type":"v1","addr":"172.21.15.139:6836","nonce":193356301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":8073,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3589330474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3589330474},{"type":"v1","addr":"172.21.15.125:6834","nonce":3589330474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:41:28.760907+0000","modified":"2023-10-18T10:41:30.774086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8076":{"gid":8076,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/3342443842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3342443842},{"type":"v1","addr":"172.21.15.125:6835","nonce":3342443842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T10:42:50.971 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 41 2023-10-18T10:42:50.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f72c8070d70 msgr2=0x7f72c8073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f72c8070d70 0x7f72c8073160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f72e0000f80 tx=0x7f72e0006eb0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72e8152b20 msgr2=0x7f72e81518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:50.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72e8152b20 0x7f72e81518c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f72d802c7c0 tx=0x7f72d802ce40 comp rx=0 tx=0).stop 2023-10-18T10:42:50.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 shutdown_connections 2023-10-18T10:42:50.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f72c8070d70 0x7f72c8073160 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72e81526c0 0x7f72e8151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72e8152b20 0x7f72e81518c0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 --2- 172.21.15.125:0/4194905884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72d4005180 0x7f72e80b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:50.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 >> 172.21.15.125:0/4194905884 conn(0x7f72e8069930 msgr2=0x7f72e815bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:50.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 shutdown_connections 2023-10-18T10:42:50.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:50.970+0000 7f72eefa9640 1 -- 172.21.15.125:0/4194905884 wait complete. 2023-10-18T10:42:50.982 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:42:51.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.083+0000 7fc41fc21640 1 -- 172.21.15.125:0/3100512428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 msgr2=0x7fc4180ab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.083+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3100512428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 0x7fc4180ab970 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fc4180713a0 tx=0x7fc41002c1d0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.083+0000 7fc41fc21640 1 -- 172.21.15.125:0/3100512428 shutdown_connections 2023-10-18T10:42:51.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.083+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3100512428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 0x7fc4180ab970 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.083+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3100512428 >> v2:172.21.15.139:3300/0 conn(0x7fc41806ae00 0x7fc41806b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.083+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3100512428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4180c96b0 0x7fc4180c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.083+0000 7fc41fc21640 1 -- 172.21.15.125:0/3100512428 >> 172.21.15.125:0/3100512428 conn(0x7fc418069930 msgr2=0x7fc418069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:51.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.084+0000 7fc41fc21640 1 -- 172.21.15.125:0/3100512428 shutdown_connections 2023-10-18T10:42:51.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.084+0000 7fc41fc21640 1 -- 172.21.15.125:0/3100512428 wait complete. 2023-10-18T10:42:51.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41fc21640 1 Processor -- start 2023-10-18T10:42:51.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41fc21640 1 -- start start 2023-10-18T10:42:51.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41fc21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41806ae00 0x7fc41813e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41fc21640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 0x7fc41813a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41fc21640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4180c96b0 0x7fc41813aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41fc21640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc418150300 con 0x7fc4180ab500 2023-10-18T10:42:51.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41fc21640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc418155690 con 0x7fc41806ae00 2023-10-18T10:42:51.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41fc21640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc418155970 con 0x7fc4180c96b0 2023-10-18T10:42:51.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41d1bc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 0x7fc41813a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41d1bc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 0x7fc41813a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50328/0 (socket says 172.21.15.125:50328) 2023-10-18T10:42:51.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.085+0000 7fc41d1bc640 1 -- 172.21.15.125:0/3328965265 learned_addr learned my addr 172.21.15.125:0/3328965265 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:51.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41e1be640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4180c96b0 0x7fc41813aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41d9bd640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41806ae00 0x7fc41813e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41d1bc640 1 -- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4180c96b0 msgr2=0x7fc41813aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41d1bc640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4180c96b0 0x7fc41813aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41d1bc640 1 -- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41806ae00 msgr2=0x7fc41813e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41d1bc640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41806ae00 0x7fc41813e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41d1bc640 1 -- 172.21.15.125:0/3328965265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410009900 con 0x7fc4180ab500 2023-10-18T10:42:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41d9bd640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41806ae00 0x7fc41813e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:42:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41d1bc640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 0x7fc41813a5a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d950 tx=0x7fc40800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc40e7fc640 1 -- 172.21.15.125:0/3328965265 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408009c30 con 0x7fc4180ab500 2023-10-18T10:42:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc40e7fc640 1 -- 172.21.15.125:0/3328965265 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc408009e80 con 0x7fc4180ab500 2023-10-18T10:42:51.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc40e7fc640 1 -- 172.21.15.125:0/3328965265 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4080106c0 con 0x7fc4180ab500 2023-10-18T10:42:51.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc418155c50 con 0x7fc4180ab500 2023-10-18T10:42:51.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.086+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41813c680 con 0x7fc4180ab500 2023-10-18T10:42:51.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.087+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41813c810 con 0x7fc4180ab500 2023-10-18T10:42:51.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.088+0000 7fc40e7fc640 1 -- 172.21.15.125:0/3328965265 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 58) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc408010820 con 0x7fc4180ab500 2023-10-18T10:42:51.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.088+0000 7fc40e7fc640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc3fc06ac90 0x7fc3fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.088+0000 7fc40e7fc640 1 -- 172.21.15.125:0/3328965265 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6919+0+0 (secure 0 0 0) 0x7fc408089be0 con 0x7fc4180ab500 2023-10-18T10:42:51.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.088+0000 7fc41d9bd640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc3fc06ac90 0x7fc3fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.089+0000 7fc41d9bd640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc3fc06ac90 0x7fc3fc06d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc41400ba00 tx=0x7fc414005c80 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:51.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.092+0000 7fc40e7fc640 1 -- 172.21.15.125:0/3328965265 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc408054b10 con 0x7fc4180ab500 2023-10-18T10:42:51.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.291+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4180cbd80 con 0x7fc4180ab500 2023-10-18T10:42:51.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.291+0000 7fc40e7fc640 1 -- 172.21.15.125:0/3328965265 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fc408017090 con 0x7fc4180ab500 2023-10-18T10:42:51.294 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:51.294 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":41,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2381546668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2381546668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2381546668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/193356301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":193356301},{"type":"v1","addr":"172.21.15.139:6836","nonce":193356301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":8073,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3589330474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3589330474},{"type":"v1","addr":"172.21.15.125:6834","nonce":3589330474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:41:28.760907+0000","modified":"2023-10-18T10:41:30.774086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8076":{"gid":8076,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/3342443842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3342443842},{"type":"v1","addr":"172.21.15.125:6835","nonce":3342443842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T10:42:51.294 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 41 2023-10-18T10:42:51.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc3fc06ac90 msgr2=0x7fc3fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc3fc06ac90 0x7fc3fc06d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc41400ba00 tx=0x7fc414005c80 comp rx=0 tx=0).stop 2023-10-18T10:42:51.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 msgr2=0x7fc41813a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 0x7fc41813a5a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d950 tx=0x7fc40800de10 comp rx=0 tx=0).stop 2023-10-18T10:42:51.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 shutdown_connections 2023-10-18T10:42:51.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc3fc06ac90 0x7fc3fc06d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4180c96b0 0x7fc41813aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4180ab500 0x7fc41813a5a0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 --2- 172.21.15.125:0/3328965265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41806ae00 0x7fc41813e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 >> 172.21.15.125:0/3328965265 conn(0x7fc418069930 msgr2=0x7fc418069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:51.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 shutdown_connections 2023-10-18T10:42:51.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.293+0000 7fc41fc21640 1 -- 172.21.15.125:0/3328965265 wait complete. 2023-10-18T10:42:51.306 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:42:51.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.407+0000 7f7ef8333640 1 -- 172.21.15.125:0/542318421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 msgr2=0x7f7edc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.407+0000 7f7ef8333640 1 --2- 172.21.15.125:0/542318421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 0x7f7edc004c40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0006fc0 tx=0x7f7edc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.407+0000 7f7ef8333640 1 -- 172.21.15.125:0/542318421 shutdown_connections 2023-10-18T10:42:51.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.407+0000 7f7ef8333640 1 --2- 172.21.15.125:0/542318421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 0x7f7edc004c40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.407+0000 7f7ef8333640 1 -- 172.21.15.125:0/542318421 >> 172.21.15.125:0/542318421 conn(0x7f7ef00d0d60 msgr2=0x7f7ef00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:51.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.407+0000 7f7ef8333640 1 -- 172.21.15.125:0/542318421 shutdown_connections 2023-10-18T10:42:51.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.407+0000 7f7ef8333640 1 -- 172.21.15.125:0/542318421 wait complete. 2023-10-18T10:42:51.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.408+0000 7f7ef8333640 1 Processor -- start 2023-10-18T10:42:51.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.408+0000 7f7ef8333640 1 -- start start 2023-10-18T10:42:51.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef8333640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 0x7f7ef0148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef8333640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ef00b53d0 0x7f7ef00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef8333640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef00b3e90 0x7f7ef006a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef8333640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0149500 con 0x7f7edc0047c0 2023-10-18T10:42:51.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef8333640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef00b4330 con 0x7f7ef00b3e90 2023-10-18T10:42:51.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef8333640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef00b4610 con 0x7f7ef00b53d0 2023-10-18T10:42:51.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 0x7f7ef0148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 0x7f7ef0148fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50346/0 (socket says 172.21.15.125:50346) 2023-10-18T10:42:51.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 -- 172.21.15.125:0/622906733 learned_addr learned my addr 172.21.15.125:0/622906733 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:51.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef58ce640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ef00b53d0 0x7f7ef00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef68d0640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef00b3e90 0x7f7ef006a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 -- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ef00b53d0 msgr2=0x7f7ef00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ef00b53d0 0x7f7ef00b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 -- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef00b3e90 msgr2=0x7f7ef006a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef00b3e90 0x7f7ef006a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 -- 172.21.15.125:0/622906733 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef00b48f0 con 0x7f7edc0047c0 2023-10-18T10:42:51.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef60cf640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 0x7f7ef0148fc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7eec003e40 tx=0x7f7eec001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:51.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ee6ffd640 1 -- 172.21.15.125:0/622906733 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011840 con 0x7f7edc0047c0 2023-10-18T10:42:51.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ee6ffd640 1 -- 172.21.15.125:0/622906733 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7eec0119a0 con 0x7f7edc0047c0 2023-10-18T10:42:51.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef00b4bd0 con 0x7f7edc0047c0 2023-10-18T10:42:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ee6ffd640 1 -- 172.21.15.125:0/622906733 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011d90 con 0x7f7edc0047c0 2023-10-18T10:42:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.409+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef01510c0 con 0x7f7edc0047c0 2023-10-18T10:42:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.410+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef01512b0 con 0x7f7edc0047c0 2023-10-18T10:42:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.411+0000 7f7ee6ffd640 1 -- 172.21.15.125:0/622906733 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 58) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f7eec0103c0 con 0x7f7edc0047c0 2023-10-18T10:42:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.411+0000 7f7ee6ffd640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f7ed406ac90 0x7f7ed406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.411+0000 7f7ee6ffd640 1 -- 172.21.15.125:0/622906733 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f7eec08a630 con 0x7f7edc0047c0 2023-10-18T10:42:51.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.411+0000 7f7ef58ce640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f7ed406ac90 0x7f7ed406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.412+0000 7f7ef58ce640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f7ed406ac90 0x7f7ed406d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0034000 tx=0x7f7ee0039040 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:51.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.416+0000 7f7ee6ffd640 1 -- 172.21.15.125:0/622906733 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ef01512b0 con 0x7f7edc0047c0 2023-10-18T10:42:51.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.584+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ef01512b0 con 0x7f7edc0047c0 2023-10-18T10:42:51.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.585+0000 7f7ee6ffd640 1 -- 172.21.15.125:0/622906733 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+18113 (secure 0 0 0) 0x7f7ef01512b0 con 0x7f7edc0047c0 2023-10-18T10:42:51.587 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:51.588 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":116,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:42:49.879712+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:41:26.603027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-18T10:41:27.743317+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":2}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/1923522040":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3280996584":"2023-10-19T10:42:49.879669+0000","172.21.15.139:6832/4208657878":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2295776957":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2846025337":"2023-10-19T10:42:49.879669+0000","172.21.15.139:6833/4208657878":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2600842084":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2652835634":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/397760040":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3890203496":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2970354256":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6833/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6832/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2064702534":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2430123574":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2593987833":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3646248611":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/170859299":"2023-10-19T10:41:43.888680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:42:51.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f7ed406ac90 msgr2=0x7f7ed406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f7ed406ac90 0x7f7ed406d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0034000 tx=0x7f7ee0039040 comp rx=0 tx=0).stop 2023-10-18T10:42:51.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 msgr2=0x7f7ef0148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 0x7f7ef0148fc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7eec003e40 tx=0x7f7eec001630 comp rx=0 tx=0).stop 2023-10-18T10:42:51.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 shutdown_connections 2023-10-18T10:42:51.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f7ed406ac90 0x7f7ed406d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef00b3e90 0x7f7ef006a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ef00b53d0 0x7f7ef00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 --2- 172.21.15.125:0/622906733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7edc0047c0 0x7f7ef0148fc0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 >> 172.21.15.125:0/622906733 conn(0x7f7ef00d0d60 msgr2=0x7f7ef00c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:51.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 shutdown_connections 2023-10-18T10:42:51.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.587+0000 7f7ef8333640 1 -- 172.21.15.125:0/622906733 wait complete. 2023-10-18T10:42:51.600 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:42:51.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.702+0000 7f7626090640 1 -- 172.21.15.125:0/2779300531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76080047c0 msgr2=0x7f7608004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.702+0000 7f7626090640 1 --2- 172.21.15.125:0/2779300531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76080047c0 0x7f7608004c40 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7614001680 tx=0x7f76080057f0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.702+0000 7f761ffff640 1 -- 172.21.15.125:0/2779300531 reap_dead start 2023-10-18T10:42:51.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.702+0000 7f7626090640 1 -- 172.21.15.125:0/2779300531 shutdown_connections 2023-10-18T10:42:51.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.702+0000 7f7626090640 1 -- 172.21.15.125:0/2779300531 >> 172.21.15.125:0/2779300531 conn(0x7f7620069930 msgr2=0x7f7620069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:51.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.702+0000 7f7626090640 1 -- 172.21.15.125:0/2779300531 shutdown_connections 2023-10-18T10:42:51.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.702+0000 7f7626090640 1 -- 172.21.15.125:0/2779300531 wait complete. 2023-10-18T10:42:51.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.703+0000 7f7626090640 1 Processor -- start 2023-10-18T10:42:51.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f7626090640 1 -- start start 2023-10-18T10:42:51.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f7626090640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76080047c0 0x7f762013a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f7626090640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f762013b130 0x7f762015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f7626090640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762013d1f0 0x7f762015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f7626090640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200abc80 con 0x7f762013b130 2023-10-18T10:42:51.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f7626090640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200ab370 con 0x7f762013d1f0 2023-10-18T10:42:51.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f7626090640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762013aee0 con 0x7f76080047c0 2023-10-18T10:42:51.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f761effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f762013b130 0x7f762015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f761effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f762013b130 0x7f762015f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50356/0 (socket says 172.21.15.125:50356) 2023-10-18T10:42:51.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.704+0000 7f761effd640 1 -- 172.21.15.125:0/4109950165 learned_addr learned my addr 172.21.15.125:0/4109950165 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:51.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f761f7fe640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76080047c0 0x7f762013a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f761ffff640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762013d1f0 0x7f762015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f761effd640 1 -- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76080047c0 msgr2=0x7f762013a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f761effd640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76080047c0 0x7f762013a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f761effd640 1 -- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762013d1f0 msgr2=0x7f762015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f761effd640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762013d1f0 0x7f762015fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f761effd640 1 -- 172.21.15.125:0/4109950165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762013d7b0 con 0x7f762013b130 2023-10-18T10:42:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f761effd640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f762013b130 0x7f762015f710 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f76140017a0 tx=0x7f76140035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f7607fff640 1 -- 172.21.15.125:0/4109950165 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614003bf0 con 0x7f762013b130 2023-10-18T10:42:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f7607fff640 1 -- 172.21.15.125:0/4109950165 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7614003e40 con 0x7f762013b130 2023-10-18T10:42:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f7607fff640 1 -- 172.21.15.125:0/4109950165 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761402c7b0 con 0x7f762013b130 2023-10-18T10:42:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76200ab630 con 0x7f762013b130 2023-10-18T10:42:51.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.705+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76200ac110 con 0x7f762013b130 2023-10-18T10:42:51.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.706+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76200690e0 con 0x7f762013b130 2023-10-18T10:42:51.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.707+0000 7f7607fff640 1 -- 172.21.15.125:0/4109950165 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 58) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f761402c910 con 0x7f762013b130 2023-10-18T10:42:51.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.707+0000 7f7607fff640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f760c06acc0 0x7f760c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:51.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.707+0000 7f7607fff640 1 -- 172.21.15.125:0/4109950165 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f76140aa820 con 0x7f762013b130 2023-10-18T10:42:51.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.707+0000 7f761f7fe640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f760c06acc0 0x7f760c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:51.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.708+0000 7f761f7fe640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f760c06acc0 0x7f760c06d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7610000fb0 tx=0x7f7610001e30 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:51.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.711+0000 7f7607fff640 1 -- 172.21.15.125:0/4109950165 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76140756d0 con 0x7f762013b130 2023-10-18T10:42:51.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.911+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f762006bd00 con 0x7f762013b130 2023-10-18T10:42:51.914 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T10:42:51.912+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:42:51.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.913+0000 7f7607fff640 1 -- 172.21.15.125:0/4109950165 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7614043090 con 0x7f762013b130 2023-10-18T10:42:51.915 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:42:51.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f760c06acc0 msgr2=0x7f760c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f760c06acc0 0x7f760c06d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7610000fb0 tx=0x7f7610001e30 comp rx=0 tx=0).stop 2023-10-18T10:42:51.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f762013b130 msgr2=0x7f762015f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:51.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f762013b130 0x7f762015f710 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f76140017a0 tx=0x7f76140035f0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 shutdown_connections 2023-10-18T10:42:51.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f760c06acc0 0x7f760c06d180 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7610000fb0 tx=0x7f7610001e30 comp rx=0 tx=0).stop 2023-10-18T10:42:51.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f762013d1f0 0x7f762015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f762013b130 0x7f762015f710 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 --2- 172.21.15.125:0/4109950165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76080047c0 0x7f762013a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:51.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 >> 172.21.15.125:0/4109950165 conn(0x7f7620069930 msgr2=0x7f76200703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:51.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 shutdown_connections 2023-10-18T10:42:51.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:51.916+0000 7f7626090640 1 -- 172.21.15.125:0/4109950165 wait complete. 2023-10-18T10:42:51.920 INFO:tasks.ceph.mds.c.smithi125.stderr: -6165> 2023-10-18T10:41:34.403+0000 7f90be568640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:42:51.929 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:42:51.950 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:42:52.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.032+0000 7f5fd41e7640 1 -- 172.21.15.125:0/3943625402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 msgr2=0x7f5fcc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:52.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.032+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/3943625402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 0x7f5fcc06d260 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009830 tx=0x7f5fbc02bcd0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.032+0000 7f5fd41e7640 1 -- 172.21.15.125:0/3943625402 shutdown_connections 2023-10-18T10:42:52.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.032+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/3943625402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc0cdb30 0x7f5fcc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.032+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/3943625402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 0x7f5fcc06d260 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.032+0000 7f5fd41e7640 1 -- 172.21.15.125:0/3943625402 >> 172.21.15.125:0/3943625402 conn(0x7f5fcc0b0670 msgr2=0x7f5fcc0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:52.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.032+0000 7f5fd41e7640 1 -- 172.21.15.125:0/3943625402 shutdown_connections 2023-10-18T10:42:52.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.032+0000 7f5fd41e7640 1 -- 172.21.15.125:0/3943625402 wait complete. 2023-10-18T10:42:52.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd41e7640 1 Processor -- start 2023-10-18T10:42:52.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd41e7640 1 -- start start 2023-10-18T10:42:52.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd41e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc0cdb30 0x7f5fcc15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd41e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 0x7f5fcc0c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd41e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fcc15ccf0 0x7f5fcc156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd41e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0b2980 con 0x7f5fcc159060 2023-10-18T10:42:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd41e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc15c9c0 con 0x7f5fcc15ccf0 2023-10-18T10:42:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd41e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0b2b90 con 0x7f5fcc0cdb30 2023-10-18T10:42:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd1782640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 0x7f5fcc0c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd1782640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 0x7f5fcc0c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50374/0 (socket says 172.21.15.125:50374) 2023-10-18T10:42:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.034+0000 7f5fd1782640 1 -- 172.21.15.125:0/393137454 learned_addr learned my addr 172.21.15.125:0/393137454 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd1782640 1 -- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc0cdb30 msgr2=0x7f5fcc15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd1782640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc0cdb30 0x7f5fcc15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd1782640 1 -- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fcc15ccf0 msgr2=0x7f5fcc156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd2784640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fcc15ccf0 0x7f5fcc156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd1782640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fcc15ccf0 0x7f5fcc156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd1782640 1 -- 172.21.15.125:0/393137454 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc009490 con 0x7f5fcc159060 2023-10-18T10:42:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd1782640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 0x7f5fcc0c7c70 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800b4a0 tx=0x7f5fc800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fc27fc640 1 -- 172.21.15.125:0/393137454 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8010610 con 0x7f5fcc159060 2023-10-18T10:42:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fc27fc640 1 -- 172.21.15.125:0/393137454 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5fc8010860 con 0x7f5fcc159060 2023-10-18T10:42:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fc27fc640 1 -- 172.21.15.125:0/393137454 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8010c70 con 0x7f5fcc159060 2023-10-18T10:42:52.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc0b2e70 con 0x7f5fcc159060 2023-10-18T10:42:52.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.035+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc0b3430 con 0x7f5fcc159060 2023-10-18T10:42:52.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.036+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc0b3650 con 0x7f5fcc159060 2023-10-18T10:42:52.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.036+0000 7f5fc27fc640 1 -- 172.21.15.125:0/393137454 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 58) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f5fc8017360 con 0x7f5fcc159060 2023-10-18T10:42:52.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.037+0000 7f5fc27fc640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fb006ac90 0x7f5fb006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:52.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.037+0000 7f5fc27fc640 1 -- 172.21.15.125:0/393137454 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 7005+0+0 (secure 0 0 0) 0x7f5fc8089640 con 0x7f5fcc159060 2023-10-18T10:42:52.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.037+0000 7f5fd1f83640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fb006ac90 0x7f5fb006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:52.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.038+0000 7f5fd1f83640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fb006ac90 0x7f5fb006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc036000 tx=0x7f5fbc0046c0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:52.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.041+0000 7f5fc27fc640 1 -- 172.21.15.125:0/393137454 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fc8054520 con 0x7f5fcc159060 2023-10-18T10:42:52.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.242+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5fcc0675f0 con 0x7f5fcc159060 2023-10-18T10:42:52.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.898+0000 7f5fc27fc640 1 -- 172.21.15.125:0/393137454 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 59) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5fc80548f0 con 0x7f5fcc159060 2023-10-18T10:42:52.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.914+0000 7f5fc27fc640 1 -- 172.21.15.125:0/393137454 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5fc8072050 con 0x7f5fcc159060 2023-10-18T10:42:52.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fb006ac90 msgr2=0x7f5fb006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:52.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fb006ac90 0x7f5fb006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc036000 tx=0x7f5fbc0046c0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 msgr2=0x7f5fcc0c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:52.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 0x7f5fcc0c7c70 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800b4a0 tx=0x7f5fc800b960 comp rx=0 tx=0).stop 2023-10-18T10:42:52.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 shutdown_connections 2023-10-18T10:42:52.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fb006ac90 0x7f5fb006d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fcc15ccf0 0x7f5fcc156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc159060 0x7f5fcc0c7c70 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 --2- 172.21.15.125:0/393137454 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc0cdb30 0x7f5fcc15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:52.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 >> 172.21.15.125:0/393137454 conn(0x7f5fcc0b0670 msgr2=0x7f5fcc157380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:52.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 shutdown_connections 2023-10-18T10:42:52.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:52.917+0000 7f5fd41e7640 1 -- 172.21.15.125:0/393137454 wait complete. 2023-10-18T10:42:52.931 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:42:53.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.078+0000 7f51c5832640 1 -- 172.21.15.125:0/1844181854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c015b640 msgr2=0x7f51c00cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:53.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.078+0000 7f51c5832640 1 --2- 172.21.15.125:0/1844181854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c015b640 0x7f51c00cbf20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009830 tx=0x7f51b802bd60 comp rx=0 tx=0).stop 2023-10-18T10:42:53.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.078+0000 7f51c5832640 1 -- 172.21.15.125:0/1844181854 shutdown_connections 2023-10-18T10:42:53.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.078+0000 7f51c5832640 1 --2- 172.21.15.125:0/1844181854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c015b640 0x7f51c00cbf20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:53.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.078+0000 7f51c5832640 1 --2- 172.21.15.125:0/1844181854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51c00cfa20 0x7f51c00cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:53.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.078+0000 7f51c5832640 1 -- 172.21.15.125:0/1844181854 >> 172.21.15.125:0/1844181854 conn(0x7f51c009a030 msgr2=0x7f51c00d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:53.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.079+0000 7f51c5832640 1 -- 172.21.15.125:0/1844181854 shutdown_connections 2023-10-18T10:42:53.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.079+0000 7f51c5832640 1 -- 172.21.15.125:0/1844181854 wait complete. 2023-10-18T10:42:53.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.080+0000 7f51c5832640 1 Processor -- start 2023-10-18T10:42:53.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.080+0000 7f51c5832640 1 -- start start 2023-10-18T10:42:53.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51c5832640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51c00cfa20 0x7f51c01522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:53.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51c5832640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c015b260 0x7f51c015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:53.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51c5832640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51c015b640 0x7f51c00adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:53.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51c5832640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c7e90 con 0x7f51c00cfa20 2023-10-18T10:42:53.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51c5832640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c8150 con 0x7f51c015b260 2023-10-18T10:42:53.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51c5832640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c8430 con 0x7f51c015b640 2023-10-18T10:42:53.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51c00cfa20 0x7f51c01522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:53.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51c00cfa20 0x7f51c01522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50386/0 (socket says 172.21.15.125:50386) 2023-10-18T10:42:53.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 -- 172.21.15.125:0/3383785529 learned_addr learned my addr 172.21.15.125:0/3383785529 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:53.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51bf7fe640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51c015b640 0x7f51c00adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:53.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51be7fc640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c015b260 0x7f51c015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:53.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 -- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51c015b640 msgr2=0x7f51c00adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:53.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51c015b640 0x7f51c00adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:53.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 -- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c015b260 msgr2=0x7f51c015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:53.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c015b260 0x7f51c015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:53.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 -- 172.21.15.125:0/3383785529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009560 con 0x7f51c00cfa20 2023-10-18T10:42:53.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51beffd640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51c00cfa20 0x7f51c01522b0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f51b000c870 tx=0x7f51b000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:53.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51a7fff640 1 -- 172.21.15.125:0/3383785529 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0003b70 con 0x7f51c00cfa20 2023-10-18T10:42:53.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51a7fff640 1 -- 172.21.15.125:0/3383785529 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f51b0003cf0 con 0x7f51c00cfa20 2023-10-18T10:42:53.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b8009490 con 0x7f51c00cfa20 2023-10-18T10:42:53.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51a7fff640 1 -- 172.21.15.125:0/3383785529 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0003ea0 con 0x7f51c00cfa20 2023-10-18T10:42:53.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.081+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c00c89f0 con 0x7f51c00cfa20 2023-10-18T10:42:53.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.082+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c00675f0 con 0x7f51c00cfa20 2023-10-18T10:42:53.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.083+0000 7f51a7fff640 1 -- 172.21.15.125:0/3383785529 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 59) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f51b000aab0 con 0x7f51c00cfa20 2023-10-18T10:42:53.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.083+0000 7f51a7fff640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f51ac06af00 0x7f51ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:53.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.083+0000 7f51a7fff640 1 -- 172.21.15.125:0/3383785529 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 7005+0+0 (secure 0 0 0) 0x7f51b0089ae0 con 0x7f51c00cfa20 2023-10-18T10:42:53.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.083+0000 7f51be7fc640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f51ac06af00 0x7f51ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:53.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.084+0000 7f51be7fc640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f51ac06af00 0x7f51ac06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51c00ae830 tx=0x7f51b4009260 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:53.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.087+0000 7f51a7fff640 1 -- 172.21.15.125:0/3383785529 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51b00546c0 con 0x7f51c00cfa20 2023-10-18T10:42:53.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.255+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f51c00cf7b0 con 0x7f51c00cfa20 2023-10-18T10:42:53.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.917+0000 7f51a7fff640 1 -- 172.21.15.125:0/3383785529 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) v1 ==== 181+0+0 (secure 0 0 0) 0x7f51b0057ce0 con 0x7f51c00cfa20 2023-10-18T10:42:53.919 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:42:53.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f51ac06af00 msgr2=0x7f51ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:53.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f51ac06af00 0x7f51ac06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51c00ae830 tx=0x7f51b4009260 comp rx=0 tx=0).stop 2023-10-18T10:42:53.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51c00cfa20 msgr2=0x7f51c01522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:53.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51c00cfa20 0x7f51c01522b0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f51b000c870 tx=0x7f51b000cd30 comp rx=0 tx=0).stop 2023-10-18T10:42:53.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 shutdown_connections 2023-10-18T10:42:53.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f51ac06af00 0x7f51ac06d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:53.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c015b260 0x7f51c015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:53.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51c015b640 0x7f51c00adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:53.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 --2- 172.21.15.125:0/3383785529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51c00cfa20 0x7f51c01522b0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:53.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 >> 172.21.15.125:0/3383785529 conn(0x7f51c009a030 msgr2=0x7f51c006a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:53.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 shutdown_connections 2023-10-18T10:42:53.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:53.920+0000 7f51c5832640 1 -- 172.21.15.125:0/3383785529 wait complete. 2023-10-18T10:42:53.935 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:42:54.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.044+0000 7f5ff6c75640 1 -- 172.21.15.125:0/2910599824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00cbd20 msgr2=0x7f5ff0158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:54.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.044+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/2910599824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00cbd20 0x7f5ff0158300 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005fd0 tx=0x7f5fe402be10 comp rx=0 tx=0).stop 2023-10-18T10:42:54.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.044+0000 7f5ff6c75640 1 -- 172.21.15.125:0/2910599824 shutdown_connections 2023-10-18T10:42:54.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.044+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/2910599824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ff00afa20 0x7f5ff00705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:54.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.044+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/2910599824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00cbd20 0x7f5ff0158300 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:54.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.044+0000 7f5ff6c75640 1 -- 172.21.15.125:0/2910599824 >> 172.21.15.125:0/2910599824 conn(0x7f5ff0069930 msgr2=0x7f5ff0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:54.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.044+0000 7f5ff6c75640 1 -- 172.21.15.125:0/2910599824 shutdown_connections 2023-10-18T10:42:54.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.044+0000 7f5ff6c75640 1 -- 172.21.15.125:0/2910599824 wait complete. 2023-10-18T10:42:54.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.045+0000 7f5ff6c75640 1 Processor -- start 2023-10-18T10:42:54.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.046+0000 7f5ff6c75640 1 -- start start 2023-10-18T10:42:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.046+0000 7f5ff6c75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ff00afa20 0x7f5ff01e4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.046+0000 7f5ff6c75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00b0530 0x7f5ff013bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.046+0000 7f5ff6c75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff00cbd20 0x7f5ff01e2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.046+0000 7f5ff6c75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01e3970 con 0x7f5ff00b0530 2023-10-18T10:42:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.046+0000 7f5ff6c75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01e2cf0 con 0x7f5ff00cbd20 2023-10-18T10:42:54.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.046+0000 7f5ff6c75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01e2fd0 con 0x7f5ff00afa20 2023-10-18T10:42:54.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.046+0000 7f5feffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00b0530 0x7f5ff013bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:54.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5feffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00b0530 0x7f5ff013bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50408/0 (socket says 172.21.15.125:50408) 2023-10-18T10:42:54.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5feffff640 1 -- 172.21.15.125:0/988373281 learned_addr learned my addr 172.21.15.125:0/988373281 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:54.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5ff4a11640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ff00afa20 0x7f5ff01e4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:54.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5ff5212640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff00cbd20 0x7f5ff01e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:54.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5feffff640 1 -- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ff00afa20 msgr2=0x7f5ff01e4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:54.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5feffff640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ff00afa20 0x7f5ff01e4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:54.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5feffff640 1 -- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff00cbd20 msgr2=0x7f5ff01e2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:54.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5feffff640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff00cbd20 0x7f5ff01e2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:54.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5feffff640 1 -- 172.21.15.125:0/988373281 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4009560 con 0x7f5ff00b0530 2023-10-18T10:42:54.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5ff4a11640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ff00afa20 0x7f5ff01e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:42:54.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5feffff640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00b0530 0x7f5ff013bc90 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00ddc0 tx=0x7f5fdc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:54.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5fed7fa640 1 -- 172.21.15.125:0/988373281 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00e650 con 0x7f5ff00b0530 2023-10-18T10:42:54.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5fed7fa640 1 -- 172.21.15.125:0/988373281 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5fdc00e8a0 con 0x7f5ff00b0530 2023-10-18T10:42:54.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5fed7fa640 1 -- 172.21.15.125:0/988373281 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00ecb0 con 0x7f5ff00b0530 2023-10-18T10:42:54.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff0070cf0 con 0x7f5ff00b0530 2023-10-18T10:42:54.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.047+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff01e3590 con 0x7f5ff00b0530 2023-10-18T10:42:54.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.048+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff00690e0 con 0x7f5ff00b0530 2023-10-18T10:42:54.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.049+0000 7f5fed7fa640 1 -- 172.21.15.125:0/988373281 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 59) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5fdc016440 con 0x7f5ff00b0530 2023-10-18T10:42:54.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.049+0000 7f5fed7fa640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fd406aed0 0x7f5fd406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:54.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.049+0000 7f5fed7fa640 1 -- 172.21.15.125:0/988373281 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6567+0+0 (secure 0 0 0) 0x7f5fdc08a1c0 con 0x7f5ff00b0530 2023-10-18T10:42:54.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.049+0000 7f5ff4a11640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fd406aed0 0x7f5fd406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:54.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.050+0000 7f5ff4a11640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fd406aed0 0x7f5fd406d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4035000 tx=0x7f5fe4004730 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:54.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.053+0000 7f5fed7fa640 1 -- 172.21.15.125:0/988373281 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fdc054fd0 con 0x7f5ff00b0530 2023-10-18T10:42:54.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.224+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5ff00cc1e0 con 0x7f5ff00b0530 2023-10-18T10:42:54.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.521+0000 7f5fed7fa640 1 -- 172.21.15.125:0/988373281 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5fdc0553a0 con 0x7f5ff00b0530 2023-10-18T10:42:54.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.919+0000 7f5fed7fa640 1 -- 172.21.15.125:0/988373281 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5fdc070c50 con 0x7f5ff00b0530 2023-10-18T10:42:54.921 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T10:42:54.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fd406aed0 msgr2=0x7f5fd406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:54.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fd406aed0 0x7f5fd406d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4035000 tx=0x7f5fe4004730 comp rx=0 tx=0).stop 2023-10-18T10:42:54.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00b0530 msgr2=0x7f5ff013bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:54.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00b0530 0x7f5ff013bc90 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00ddc0 tx=0x7f5fdc00a620 comp rx=0 tx=0).stop 2023-10-18T10:42:54.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 shutdown_connections 2023-10-18T10:42:54.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5fd406aed0 0x7f5fd406d390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:54.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff00cbd20 0x7f5ff01e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:54.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ff00b0530 0x7f5ff013bc90 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:54.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 --2- 172.21.15.125:0/988373281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ff00afa20 0x7f5ff01e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:54.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 >> 172.21.15.125:0/988373281 conn(0x7f5ff0069930 msgr2=0x7f5ff0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:54.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 shutdown_connections 2023-10-18T10:42:54.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:54.922+0000 7f5ff6c75640 1 -- 172.21.15.125:0/988373281 wait complete. 2023-10-18T10:42:54.937 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T10:42:55.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.042+0000 7ffb72f00640 1 -- 172.21.15.125:0/3633138293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0afa20 msgr2=0x7ffb6c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.042+0000 7ffb72f00640 1 --2- 172.21.15.125:0/3633138293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0afa20 0x7ffb6c0705a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0cc7e0 tx=0x7ffb6002bd70 comp rx=0 tx=0).stop 2023-10-18T10:42:55.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.042+0000 7ffb72f00640 1 -- 172.21.15.125:0/3633138293 shutdown_connections 2023-10-18T10:42:55.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.042+0000 7ffb72f00640 1 --2- 172.21.15.125:0/3633138293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0afa20 0x7ffb6c0705a0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.042+0000 7ffb72f00640 1 --2- 172.21.15.125:0/3633138293 >> v2:172.21.15.139:3300/0 conn(0x7ffb6c0cbd20 0x7ffb6c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.042+0000 7ffb72f00640 1 -- 172.21.15.125:0/3633138293 >> 172.21.15.125:0/3633138293 conn(0x7ffb6c069930 msgr2=0x7ffb6c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:55.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.042+0000 7ffb72f00640 1 -- 172.21.15.125:0/3633138293 shutdown_connections 2023-10-18T10:42:55.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.042+0000 7ffb72f00640 1 -- 172.21.15.125:0/3633138293 wait complete. 2023-10-18T10:42:55.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.043+0000 7ffb72f00640 1 Processor -- start 2023-10-18T10:42:55.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.043+0000 7ffb72f00640 1 -- start start 2023-10-18T10:42:55.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb72f00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6c0afa20 0x7ffb6c13af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:55.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb72f00640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0b0530 0x7ffb6c163600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:55.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb72f00640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb6c0cbd20 0x7ffb6c1615b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:55.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb72f00640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c163f40 con 0x7ffb6c0b0530 2023-10-18T10:42:55.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb72f00640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c164220 con 0x7ffb6c0afa20 2023-10-18T10:42:55.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb72f00640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c164500 con 0x7ffb6c0cbd20 2023-10-18T10:42:55.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb6bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0b0530 0x7ffb6c163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:55.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb6bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0b0530 0x7ffb6c163600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50430/0 (socket says 172.21.15.125:50430) 2023-10-18T10:42:55.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb6bfff640 1 -- 172.21.15.125:0/417818859 learned_addr learned my addr 172.21.15.125:0/417818859 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:55.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb70c9c640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6c0afa20 0x7ffb6c13af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:55.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb7149d640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb6c0cbd20 0x7ffb6c1615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:55.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb6bfff640 1 -- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb6c0cbd20 msgr2=0x7ffb6c1615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb6bfff640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb6c0cbd20 0x7ffb6c1615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb6bfff640 1 -- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6c0afa20 msgr2=0x7ffb6c13af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.044+0000 7ffb6bfff640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6c0afa20 0x7ffb6c13af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.045+0000 7ffb6bfff640 1 -- 172.21.15.125:0/417818859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60009490 con 0x7ffb6c0b0530 2023-10-18T10:42:55.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.045+0000 7ffb70c9c640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6c0afa20 0x7ffb6c13af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:42:55.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.045+0000 7ffb6bfff640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0b0530 0x7ffb6c163600 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002c380 tx=0x7ffb600038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:55.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.045+0000 7ffb697fa640 1 -- 172.21.15.125:0/417818859 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60003ea0 con 0x7ffb6c0b0530 2023-10-18T10:42:55.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.045+0000 7ffb697fa640 1 -- 172.21.15.125:0/417818859 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffb60034440 con 0x7ffb6c0b0530 2023-10-18T10:42:55.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.045+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c13cbb0 con 0x7ffb6c0b0530 2023-10-18T10:42:55.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.045+0000 7ffb697fa640 1 -- 172.21.15.125:0/417818859 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60034810 con 0x7ffb6c0b0530 2023-10-18T10:42:55.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.045+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c13b4c0 con 0x7ffb6c0b0530 2023-10-18T10:42:55.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.046+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c13ca10 con 0x7ffb6c0b0530 2023-10-18T10:42:55.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.046+0000 7ffb697fa640 1 -- 172.21.15.125:0/417818859 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffb60034970 con 0x7ffb6c0b0530 2023-10-18T10:42:55.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.047+0000 7ffb697fa640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7ffb5806af60 0x7ffb5806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:55.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.047+0000 7ffb697fa640 1 -- 172.21.15.125:0/417818859 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7ffb600ac980 con 0x7ffb6c0b0530 2023-10-18T10:42:55.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.047+0000 7ffb70c9c640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7ffb5806af60 0x7ffb5806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.047+0000 7ffb70c9c640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7ffb5806af60 0x7ffb5806d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00ba60 tx=0x7ffb5c005cf0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.051+0000 7ffb697fa640 1 -- 172.21.15.125:0/417818859 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb6c13ca10 con 0x7ffb6c0b0530 2023-10-18T10:42:55.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.219+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7ffb6c13d0d0 con 0x7ffb6c0b0530 2023-10-18T10:42:55.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.222+0000 7ffb697fa640 1 -- 172.21.15.125:0/417818859 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v50) v1 ==== 71+0+0 (secure 0 0 0) 0x7ffb6c13d0d0 con 0x7ffb6c0b0530 2023-10-18T10:42:55.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7ffb5806af60 msgr2=0x7ffb5806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7ffb5806af60 0x7ffb5806d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00ba60 tx=0x7ffb5c005cf0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0b0530 msgr2=0x7ffb6c163600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0b0530 0x7ffb6c163600 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002c380 tx=0x7ffb600038a0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 shutdown_connections 2023-10-18T10:42:55.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7ffb5806af60 0x7ffb5806d420 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb6c0cbd20 0x7ffb6c1615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb6c0b0530 0x7ffb6c163600 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 --2- 172.21.15.125:0/417818859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb6c0afa20 0x7ffb6c13af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 >> 172.21.15.125:0/417818859 conn(0x7ffb6c069930 msgr2=0x7ffb6c157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:55.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 shutdown_connections 2023-10-18T10:42:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.224+0000 7ffb72f00640 1 -- 172.21.15.125:0/417818859 wait complete. 2023-10-18T10:42:55.237 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-18T10:42:55.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.337+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/4182612921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f006af00 msgr2=0x7fe8f006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.337+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/4182612921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f006af00 0x7fe8f006b2e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005fd0 tx=0x7fe8e002bd10 comp rx=0 tx=0).stop 2023-10-18T10:42:55.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.337+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/4182612921 shutdown_connections 2023-10-18T10:42:55.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.337+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/4182612921 >> v2:172.21.15.139:3300/0 conn(0x7fe8f00abc10 0x7fe8f00c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.337+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/4182612921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f006af00 0x7fe8f006b2e0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.337+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/4182612921 >> 172.21.15.125:0/4182612921 conn(0x7fe8f0069930 msgr2=0x7fe8f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:55.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.338+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/4182612921 shutdown_connections 2023-10-18T10:42:55.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.338+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/4182612921 wait complete. 2023-10-18T10:42:55.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.339+0000 7fe8f4d8a640 1 Processor -- start 2023-10-18T10:42:55.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.339+0000 7fe8f4d8a640 1 -- start start 2023-10-18T10:42:55.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.339+0000 7fe8f4d8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8f006af00 0x7fe8f012e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:55.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.339+0000 7fe8f4d8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f00abc10 0x7fe8f00b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:55.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.339+0000 7fe8f4d8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8f00c96b0 0x7fe8f00b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:55.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.339+0000 7fe8f4d8a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f012bbe0 con 0x7fe8f00abc10 2023-10-18T10:42:55.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.339+0000 7fe8f4d8a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f012e860 con 0x7fe8f006af00 2023-10-18T10:42:55.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.339+0000 7fe8f4d8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f012afd0 con 0x7fe8f00c96b0 2023-10-18T10:42:55.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f00abc10 0x7fe8f00b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:55.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f00abc10 0x7fe8f00b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50456/0 (socket says 172.21.15.125:50456) 2023-10-18T10:42:55.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 -- 172.21.15.125:0/977608063 learned_addr learned my addr 172.21.15.125:0/977608063 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:55.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8ee59c640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8f006af00 0x7fe8f012e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:55.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8eed9d640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8f00c96b0 0x7fe8f00b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:55.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 -- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8f00c96b0 msgr2=0x7fe8f00b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8f00c96b0 0x7fe8f00b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 -- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8f006af00 msgr2=0x7fe8f012e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8f006af00 0x7fe8f012e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 -- 172.21.15.125:0/977608063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0009560 con 0x7fe8f00abc10 2023-10-18T10:42:55.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8eed9d640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8f00c96b0 0x7fe8f00b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:42:55.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e7fff640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f00abc10 0x7fe8f00b08d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800b410 tx=0x7fe8d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:55.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e6ffd640 1 -- 172.21.15.125:0/977608063 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8011830 con 0x7fe8f00abc10 2023-10-18T10:42:55.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e6ffd640 1 -- 172.21.15.125:0/977608063 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8d8011a80 con 0x7fe8f00abc10 2023-10-18T10:42:55.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f012b2b0 con 0x7fe8f00abc10 2023-10-18T10:42:55.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8e6ffd640 1 -- 172.21.15.125:0/977608063 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8011e90 con 0x7fe8f00abc10 2023-10-18T10:42:55.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.340+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f012b870 con 0x7fe8f00abc10 2023-10-18T10:42:55.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.341+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f012bd90 con 0x7fe8f00abc10 2023-10-18T10:42:55.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.342+0000 7fe8e6ffd640 1 -- 172.21.15.125:0/977608063 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe8d801a400 con 0x7fe8f00abc10 2023-10-18T10:42:55.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.342+0000 7fe8e6ffd640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fe8d006af60 0x7fe8d006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:55.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.342+0000 7fe8e6ffd640 1 -- 172.21.15.125:0/977608063 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7fe8d8089c60 con 0x7fe8f00abc10 2023-10-18T10:42:55.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.342+0000 7fe8ee59c640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fe8d006af60 0x7fe8d006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:55.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.343+0000 7fe8ee59c640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fe8d006af60 0x7fe8d006d420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0035000 tx=0x7fe8e0004730 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:55.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.346+0000 7fe8e6ffd640 1 -- 172.21.15.125:0/977608063 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8f012bd90 con 0x7fe8f00abc10 2023-10-18T10:42:55.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.579+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fe8f00690e0 con 0x7fe8f00abc10 2023-10-18T10:42:55.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.927+0000 7fe8e6ffd640 1 -- 172.21.15.125:0/977608063 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v670) v1 ==== 183+0+0 (secure 0 0 0) 0x7fe8d8015020 con 0x7fe8f00abc10 2023-10-18T10:42:55.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.929+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fe8d006af60 msgr2=0x7fe8d006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.929+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fe8d006af60 0x7fe8d006d420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0035000 tx=0x7fe8e0004730 comp rx=0 tx=0).stop 2023-10-18T10:42:55.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.929+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f00abc10 msgr2=0x7fe8f00b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:55.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.929+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f00abc10 0x7fe8f00b08d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800b410 tx=0x7fe8d800b8d0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.929+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 shutdown_connections 2023-10-18T10:42:55.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.929+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fe8d006af60 0x7fe8d006d420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.930+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8f00c96b0 0x7fe8f00b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.930+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8f00abc10 0x7fe8f00b08d0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.930+0000 7fe8f4d8a640 1 --2- 172.21.15.125:0/977608063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8f006af00 0x7fe8f012e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:55.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.930+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 >> 172.21.15.125:0/977608063 conn(0x7fe8f0069930 msgr2=0x7fe8f0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:55.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.930+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 shutdown_connections 2023-10-18T10:42:55.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:55.930+0000 7fe8f4d8a640 1 -- 172.21.15.125:0/977608063 wait complete. 2023-10-18T10:42:55.942 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-10-18T10:42:55.943 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-10-18T10:42:55.943 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T10:42:55.944 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:42:56.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.048+0000 7f6e3687c640 1 -- 172.21.15.125:0/448675049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300ceb40 msgr2=0x7f6e300b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.048+0000 7f6e3687c640 1 --2- 172.21.15.125:0/448675049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300ceb40 0x7f6e300b0650 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6e3008a810 tx=0x7f6e2802c080 comp rx=0 tx=0).stop 2023-10-18T10:42:56.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.048+0000 7f6e3687c640 1 -- 172.21.15.125:0/448675049 shutdown_connections 2023-10-18T10:42:56.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.048+0000 7f6e3687c640 1 --2- 172.21.15.125:0/448675049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300ceb40 0x7f6e300b0650 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.048+0000 7f6e3687c640 1 --2- 172.21.15.125:0/448675049 >> v2:172.21.15.139:3300/0 conn(0x7f6e300cf340 0x7f6e300d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.048+0000 7f6e3687c640 1 -- 172.21.15.125:0/448675049 >> 172.21.15.125:0/448675049 conn(0x7f6e300d0d60 msgr2=0x7f6e300d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:56.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.048+0000 7f6e3687c640 1 -- 172.21.15.125:0/448675049 shutdown_connections 2023-10-18T10:42:56.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.048+0000 7f6e3687c640 1 -- 172.21.15.125:0/448675049 wait complete. 2023-10-18T10:42:56.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.049+0000 7f6e3687c640 1 Processor -- start 2023-10-18T10:42:56.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.049+0000 7f6e3687c640 1 -- start start 2023-10-18T10:42:56.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e3687c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e300ceb40 0x7f6e30156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e3687c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300cf340 0x7f6e30157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e3687c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e3015b2a0 0x7f6e3009e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e3687c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30156b90 con 0x7f6e300cf340 2023-10-18T10:42:56.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e3687c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30156970 con 0x7f6e300ceb40 2023-10-18T10:42:56.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e3687c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30156de0 con 0x7f6e3015b2a0 2023-10-18T10:42:56.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e2f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300cf340 0x7f6e30157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e2f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300cf340 0x7f6e30157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50478/0 (socket says 172.21.15.125:50478) 2023-10-18T10:42:56.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e2f7fe640 1 -- 172.21.15.125:0/964878742 learned_addr learned my addr 172.21.15.125:0/964878742 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:56.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e34e19640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e3015b2a0 0x7f6e3009e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e2ffff640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e300ceb40 0x7f6e30156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e2f7fe640 1 -- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e3015b2a0 msgr2=0x7f6e3009e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e2f7fe640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e3015b2a0 0x7f6e3009e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e2f7fe640 1 -- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e300ceb40 msgr2=0x7f6e30156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.050+0000 7f6e2f7fe640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e300ceb40 0x7f6e30156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.051+0000 7f6e2f7fe640 1 -- 172.21.15.125:0/964878742 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e28009840 con 0x7f6e300cf340 2023-10-18T10:42:56.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.051+0000 7f6e2f7fe640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300cf340 0x7f6e30157e80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c3c0 tx=0x7f6e2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.051+0000 7f6e2cff9640 1 -- 172.21.15.125:0/964878742 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24004230 con 0x7f6e300cf340 2023-10-18T10:42:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.051+0000 7f6e2cff9640 1 -- 172.21.15.125:0/964878742 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e240044a0 con 0x7f6e300cf340 2023-10-18T10:42:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.051+0000 7f6e2cff9640 1 -- 172.21.15.125:0/964878742 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24003850 con 0x7f6e300cf340 2023-10-18T10:42:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.051+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e301570c0 con 0x7f6e300cf340 2023-10-18T10:42:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.051+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30157680 con 0x7f6e300cf340 2023-10-18T10:42:56.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.052+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e300690e0 con 0x7f6e300cf340 2023-10-18T10:42:56.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.052+0000 7f6e2cff9640 1 -- 172.21.15.125:0/964878742 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6e240039b0 con 0x7f6e300cf340 2023-10-18T10:42:56.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.053+0000 7f6e2cff9640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6e1c06af60 0x7f6e1c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.053+0000 7f6e2cff9640 1 -- 172.21.15.125:0/964878742 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7f6e24089190 con 0x7f6e300cf340 2023-10-18T10:42:56.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.053+0000 7f6e2ffff640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6e1c06af60 0x7f6e1c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.053+0000 7f6e2ffff640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6e1c06af60 0x7f6e1c06d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e20005fd0 tx=0x7f6e20005850 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:56.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.057+0000 7f6e2cff9640 1 -- 172.21.15.125:0/964878742 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e24053ed0 con 0x7f6e300cf340 2023-10-18T10:42:56.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.224+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e300b0d00 con 0x7f6e300cf340 2023-10-18T10:42:56.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.225+0000 7f6e2cff9640 1 -- 172.21.15.125:0/964878742 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+14395 (secure 0 0 0) 0x7f6e24017090 con 0x7f6e300cf340 2023-10-18T10:42:56.227 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:56.227 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":119,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:42:54.918427+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6833/3342443842":"2023-10-19T10:42:51.912992+0000","172.21.15.139:0/1923522040":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3280996584":"2023-10-19T10:42:49.879669+0000","172.21.15.139:6832/4208657878":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2295776957":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2846025337":"2023-10-19T10:42:49.879669+0000","172.21.15.125:6835/3342443842":"2023-10-19T10:42:51.912992+0000","172.21.15.139:6833/4208657878":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2600842084":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/2652835634":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/397760040":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3890203496":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2970354256":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6833/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:6832/1523403591":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2064702534":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2430123574":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/2593987833":"2023-10-19T10:42:49.879669+0000","172.21.15.139:0/3646248611":"2023-10-19T10:41:43.888680+0000","172.21.15.139:0/170859299":"2023-10-19T10:41:43.888680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:42:56.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6e1c06af60 msgr2=0x7f6e1c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6e1c06af60 0x7f6e1c06d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e20005fd0 tx=0x7f6e20005850 comp rx=0 tx=0).stop 2023-10-18T10:42:56.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300cf340 msgr2=0x7f6e30157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300cf340 0x7f6e30157e80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c3c0 tx=0x7f6e2400c880 comp rx=0 tx=0).stop 2023-10-18T10:42:56.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 shutdown_connections 2023-10-18T10:42:56.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6e1c06af60 0x7f6e1c06d420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e3015b2a0 0x7f6e3009e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e300cf340 0x7f6e30157e80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 --2- 172.21.15.125:0/964878742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e300ceb40 0x7f6e30156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 >> 172.21.15.125:0/964878742 conn(0x7f6e300d0d60 msgr2=0x7f6e30159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 shutdown_connections 2023-10-18T10:42:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.227+0000 7f6e3687c640 1 -- 172.21.15.125:0/964878742 wait complete. 2023-10-18T10:42:56.239 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:42:56.240 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:42:56.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 -- 172.21.15.125:0/2502017931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c15b2a0 msgr2=0x7f895c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 --2- 172.21.15.125:0/2502017931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c15b2a0 0x7f895c0cf720 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f895c08a810 tx=0x7f894c02bd10 comp rx=0 tx=0).stop 2023-10-18T10:42:56.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 -- 172.21.15.125:0/2502017931 shutdown_connections 2023-10-18T10:42:56.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 --2- 172.21.15.125:0/2502017931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c15b2a0 0x7f895c0cf720 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 --2- 172.21.15.125:0/2502017931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f895c0ceb40 0x7f895c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 --2- 172.21.15.125:0/2502017931 >> v2:172.21.15.139:3300/0 conn(0x7f895c0cf340 0x7f895c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 -- 172.21.15.125:0/2502017931 >> 172.21.15.125:0/2502017931 conn(0x7f895c0d0d60 msgr2=0x7f895c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:56.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 -- 172.21.15.125:0/2502017931 shutdown_connections 2023-10-18T10:42:56.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.344+0000 7f8964384640 1 -- 172.21.15.125:0/2502017931 wait complete. 2023-10-18T10:42:56.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.345+0000 7f8964384640 1 Processor -- start 2023-10-18T10:42:56.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.345+0000 7f8964384640 1 -- start start 2023-10-18T10:42:56.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8964384640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f895c0ceb40 0x7f895c1f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8964384640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c0cf340 0x7f895c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8964384640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f895c15b2a0 0x7f895c1f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8964384640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c1f2490 con 0x7f895c0cf340 2023-10-18T10:42:56.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8964384640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b5c90 con 0x7f895c15b2a0 2023-10-18T10:42:56.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8964384640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b4e70 con 0x7f895c0ceb40 2023-10-18T10:42:56.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f896191f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c0cf340 0x7f895c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f896191f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c0cf340 0x7f895c0b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50500/0 (socket says 172.21.15.125:50500) 2023-10-18T10:42:56.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f896191f640 1 -- 172.21.15.125:0/3439770929 learned_addr learned my addr 172.21.15.125:0/3439770929 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:56.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8962921640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f895c15b2a0 0x7f895c1f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8962120640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f895c0ceb40 0x7f895c1f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f896191f640 1 -- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f895c0ceb40 msgr2=0x7f895c1f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f896191f640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f895c0ceb40 0x7f895c1f1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f896191f640 1 -- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f895c15b2a0 msgr2=0x7f895c1f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f896191f640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f895c15b2a0 0x7f895c1f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f896191f640 1 -- 172.21.15.125:0/3439770929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009560 con 0x7f895c0cf340 2023-10-18T10:42:56.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.346+0000 7f8962921640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f895c15b2a0 0x7f895c1f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:42:56.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.347+0000 7f896191f640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c0cf340 0x7f895c0b4930 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f894c009850 tx=0x7f894c02dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:56.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.347+0000 7f8952ffd640 1 -- 172.21.15.125:0/3439770929 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c005650 con 0x7f895c0cf340 2023-10-18T10:42:56.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.347+0000 7f8952ffd640 1 -- 172.21.15.125:0/3439770929 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f894c0038f0 con 0x7f895c0cf340 2023-10-18T10:42:56.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.347+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c0b5110 con 0x7f895c0cf340 2023-10-18T10:42:56.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.347+0000 7f8952ffd640 1 -- 172.21.15.125:0/3439770929 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c003ce0 con 0x7f895c0cf340 2023-10-18T10:42:56.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.347+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c0b56d0 con 0x7f895c0cf340 2023-10-18T10:42:56.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.348+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c0690e0 con 0x7f895c0cf340 2023-10-18T10:42:56.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.348+0000 7f8952ffd640 1 -- 172.21.15.125:0/3439770929 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f894c03e450 con 0x7f895c0cf340 2023-10-18T10:42:56.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.349+0000 7f8952ffd640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f894006af60 0x7f894006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.349+0000 7f8952ffd640 1 -- 172.21.15.125:0/3439770929 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7f894c0ac1d0 con 0x7f895c0cf340 2023-10-18T10:42:56.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.349+0000 7f8962120640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f894006af60 0x7f894006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.349+0000 7f8962120640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f894006af60 0x7f894006d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f895800a810 tx=0x7f8958008040 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:56.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.353+0000 7f8952ffd640 1 -- 172.21.15.125:0/3439770929 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f894c0770b0 con 0x7f895c0cf340 2023-10-18T10:42:56.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.521+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f895c0d0b40 con 0x7f895c0cf340 2023-10-18T10:42:56.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.521+0000 7f8952ffd640 1 -- 172.21.15.125:0/3439770929 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f894c07a6d0 con 0x7f895c0cf340 2023-10-18T10:42:56.523 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:42:56.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f894006af60 msgr2=0x7f894006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f894006af60 0x7f894006d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f895800a810 tx=0x7f8958008040 comp rx=0 tx=0).stop 2023-10-18T10:42:56.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c0cf340 msgr2=0x7f895c0b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c0cf340 0x7f895c0b4930 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f894c009850 tx=0x7f894c02dce0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 shutdown_connections 2023-10-18T10:42:56.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f894006af60 0x7f894006d420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f895c15b2a0 0x7f895c1f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f895c0cf340 0x7f895c0b4930 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 --2- 172.21.15.125:0/3439770929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f895c0ceb40 0x7f895c1f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 >> 172.21.15.125:0/3439770929 conn(0x7f895c0d0d60 msgr2=0x7f895c069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:56.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 shutdown_connections 2023-10-18T10:42:56.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.523+0000 7f8964384640 1 -- 172.21.15.125:0/3439770929 wait complete. 2023-10-18T10:42:56.535 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-18T10:42:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.635+0000 7f2df061a640 1 -- 172.21.15.125:0/2069174832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de80abc10 msgr2=0x7f2de80c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.635+0000 7f2df061a640 1 --2- 172.21.15.125:0/2069174832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de80abc10 0x7f2de80c9b30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009be0 tx=0x7f2de002c0c0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.635+0000 7f2df061a640 1 -- 172.21.15.125:0/2069174832 shutdown_connections 2023-10-18T10:42:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.635+0000 7f2df061a640 1 --2- 172.21.15.125:0/2069174832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de80abc10 0x7f2de80c9b30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.635+0000 7f2df061a640 1 --2- 172.21.15.125:0/2069174832 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2de806af00 0x7f2de806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.635+0000 7f2df061a640 1 -- 172.21.15.125:0/2069174832 >> 172.21.15.125:0/2069174832 conn(0x7f2de8069930 msgr2=0x7f2de8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.635+0000 7f2df061a640 1 -- 172.21.15.125:0/2069174832 shutdown_connections 2023-10-18T10:42:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.635+0000 7f2df061a640 1 -- 172.21.15.125:0/2069174832 wait complete. 2023-10-18T10:42:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.636+0000 7f2df061a640 1 Processor -- start 2023-10-18T10:42:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.636+0000 7f2df061a640 1 -- start start 2023-10-18T10:42:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2df061a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2de806af00 0x7f2de815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2df061a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2de80abc10 0x7f2de8152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2df061a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de80c96b0 0x7f2de80ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2df061a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de81530f0 con 0x7f2de80abc10 2023-10-18T10:42:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2df061a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8139770 con 0x7f2de80c96b0 2023-10-18T10:42:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2df061a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de806f5d0 con 0x7f2de806af00 2023-10-18T10:42:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dedbb5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2de80abc10 0x7f2de8152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dedbb5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2de80abc10 0x7f2de8152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50514/0 (socket says 172.21.15.125:50514) 2023-10-18T10:42:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dedbb5640 1 -- 172.21.15.125:0/595140559 learned_addr learned my addr 172.21.15.125:0/595140559 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dee3b6640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2de806af00 0x7f2de815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2deebb7640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de80c96b0 0x7f2de80ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dedbb5640 1 -- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2de806af00 msgr2=0x7f2de815ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dedbb5640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2de806af00 0x7f2de815ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dedbb5640 1 -- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de80c96b0 msgr2=0x7f2de80ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dedbb5640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de80c96b0 0x7f2de80ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.637+0000 7f2dedbb5640 1 -- 172.21.15.125:0/595140559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4009560 con 0x7f2de80abc10 2023-10-18T10:42:56.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.638+0000 7f2dedbb5640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2de80abc10 0x7f2de8152bb0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c430 tx=0x7f2dd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:56.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.638+0000 7f2ddeffd640 1 -- 172.21.15.125:0/595140559 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80052d0 con 0x7f2de80abc10 2023-10-18T10:42:56.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.638+0000 7f2ddeffd640 1 -- 172.21.15.125:0/595140559 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2dd80035f0 con 0x7f2de80abc10 2023-10-18T10:42:56.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.638+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de0009840 con 0x7f2de80abc10 2023-10-18T10:42:56.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.638+0000 7f2ddeffd640 1 -- 172.21.15.125:0/595140559 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8003a00 con 0x7f2de80abc10 2023-10-18T10:42:56.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.638+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de806fb90 con 0x7f2de80abc10 2023-10-18T10:42:56.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.639+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de80690e0 con 0x7f2de80abc10 2023-10-18T10:42:56.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.639+0000 7f2ddeffd640 1 -- 172.21.15.125:0/595140559 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2dd8003b60 con 0x7f2de80abc10 2023-10-18T10:42:56.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.640+0000 7f2ddeffd640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2dcc06af60 0x7f2dcc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:56.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.640+0000 7f2ddeffd640 1 -- 172.21.15.125:0/595140559 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7f2dd8089060 con 0x7f2de80abc10 2023-10-18T10:42:56.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.640+0000 7f2dee3b6640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2dcc06af60 0x7f2dcc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:56.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.640+0000 7f2dee3b6640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2dcc06af60 0x7f2dcc06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009530 tx=0x7f2de4009260 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:56.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.644+0000 7f2ddeffd640 1 -- 172.21.15.125:0/595140559 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2dd8053ee0 con 0x7f2de80abc10 2023-10-18T10:42:56.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.872+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f2de80cc420 con 0x7f2de80abc10 2023-10-18T10:42:56.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.929+0000 7f2ddeffd640 1 -- 172.21.15.125:0/595140559 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v671) v1 ==== 151+0+0 (secure 0 0 0) 0x7f2dd8057500 con 0x7f2de80abc10 2023-10-18T10:42:56.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2dcc06af60 msgr2=0x7f2dcc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2dcc06af60 0x7f2dcc06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009530 tx=0x7f2de4009260 comp rx=0 tx=0).stop 2023-10-18T10:42:56.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2de80abc10 msgr2=0x7f2de8152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:56.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2de80abc10 0x7f2de8152bb0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c430 tx=0x7f2dd800c8f0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 shutdown_connections 2023-10-18T10:42:56.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2dcc06af60 0x7f2dcc06d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de80c96b0 0x7f2de80ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2de80abc10 0x7f2de8152bb0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 --2- 172.21.15.125:0/595140559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2de806af00 0x7f2de815ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:56.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 >> 172.21.15.125:0/595140559 conn(0x7f2de8069930 msgr2=0x7f2de8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:56.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 shutdown_connections 2023-10-18T10:42:56.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:56.931+0000 7f2df061a640 1 -- 172.21.15.125:0/595140559 wait complete. 2023-10-18T10:42:56.943 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T10:42:57.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.043+0000 7efef7b4c640 1 -- 172.21.15.125:0/3102937285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00cbd20 msgr2=0x7efef0158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.043+0000 7efef7b4c640 1 --2- 172.21.15.125:0/3102937285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00cbd20 0x7efef0158300 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7efeec005fd0 tx=0x7efeec02bed0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.043+0000 7efef7b4c640 1 -- 172.21.15.125:0/3102937285 shutdown_connections 2023-10-18T10:42:57.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.043+0000 7efef7b4c640 1 --2- 172.21.15.125:0/3102937285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00cbd20 0x7efef0158300 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.043+0000 7efef7b4c640 1 -- 172.21.15.125:0/3102937285 >> 172.21.15.125:0/3102937285 conn(0x7efef0069930 msgr2=0x7efef0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:57.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.043+0000 7efef7b4c640 1 -- 172.21.15.125:0/3102937285 shutdown_connections 2023-10-18T10:42:57.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.043+0000 7efef7b4c640 1 -- 172.21.15.125:0/3102937285 wait complete. 2023-10-18T10:42:57.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef7b4c640 1 Processor -- start 2023-10-18T10:42:57.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef7b4c640 1 -- start start 2023-10-18T10:42:57.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef7b4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00705a0 0x7efef01593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef7b4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efef00afa20 0x7efef0145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef7b4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efef00cbd20 0x7efef01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef7b4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0135020 con 0x7efef00705a0 2023-10-18T10:42:57.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef7b4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0159900 con 0x7efef00cbd20 2023-10-18T10:42:57.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef7b4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef015a530 con 0x7efef00afa20 2023-10-18T10:42:57.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef58e8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00705a0 0x7efef01593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef58e8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00705a0 0x7efef01593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50536/0 (socket says 172.21.15.125:50536) 2023-10-18T10:42:57.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.045+0000 7efef58e8640 1 -- 172.21.15.125:0/4286513434 learned_addr learned my addr 172.21.15.125:0/4286513434 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:57.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef60e9640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efef00cbd20 0x7efef01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef50e7640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efef00afa20 0x7efef0145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef58e8640 1 -- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efef00afa20 msgr2=0x7efef0145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef58e8640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efef00afa20 0x7efef0145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef58e8640 1 -- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efef00cbd20 msgr2=0x7efef01562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef58e8640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efef00cbd20 0x7efef01562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef58e8640 1 -- 172.21.15.125:0/4286513434 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec009560 con 0x7efef00705a0 2023-10-18T10:42:57.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef58e8640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00705a0 0x7efef01593c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7efee400c450 tx=0x7efee400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:57.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efee27fc640 1 -- 172.21.15.125:0/4286513434 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee40052d0 con 0x7efef00705a0 2023-10-18T10:42:57.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efee27fc640 1 -- 172.21.15.125:0/4286513434 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efee4005430 con 0x7efef00705a0 2023-10-18T10:42:57.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efee27fc640 1 -- 172.21.15.125:0/4286513434 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4012890 con 0x7efef00705a0 2023-10-18T10:42:57.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef015a810 con 0x7efef00705a0 2023-10-18T10:42:57.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.046+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef0135550 con 0x7efef00705a0 2023-10-18T10:42:57.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.047+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef00690e0 con 0x7efef00705a0 2023-10-18T10:42:57.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.048+0000 7efee27fc640 1 -- 172.21.15.125:0/4286513434 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efee40035f0 con 0x7efef00705a0 2023-10-18T10:42:57.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.048+0000 7efee27fc640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7efed406af60 0x7efed406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.048+0000 7efee27fc640 1 -- 172.21.15.125:0/4286513434 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7efee4089590 con 0x7efef00705a0 2023-10-18T10:42:57.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.048+0000 7efef50e7640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7efed406af60 0x7efed406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.049+0000 7efef50e7640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7efed406af60 0x7efed406d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efeec035000 tx=0x7efeec02e040 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:57.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.052+0000 7efee27fc640 1 -- 172.21.15.125:0/4286513434 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efee40543e0 con 0x7efef00705a0 2023-10-18T10:42:57.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.218+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7efef006f700 con 0x7efef00705a0 2023-10-18T10:42:57.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.219+0000 7efee27fc640 1 -- 172.21.15.125:0/4286513434 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v50) v1 ==== 59+0+2220 (secure 0 0 0) 0x7efee4057a00 con 0x7efef00705a0 2023-10-18T10:42:57.221 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T10:42:57.221 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T10:42:57.222 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T10:42:57.222 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T10:42:57.222 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T10:42:57.222 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T10:42:57.222 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T10:42:57.223 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T10:42:57.223 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T10:42:57.223 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T10:42:57.223 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T10:42:57.223 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T10:42:57.224 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T10:42:57.224 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T10:42:57.224 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T10:42:57.224 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T10:42:57.224 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T10:42:57.225 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T10:42:57.225 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T10:42:57.225 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T10:42:57.225 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T10:42:57.225 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T10:42:57.226 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T10:42:57.226 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T10:42:57.226 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T10:42:57.226 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T10:42:57.226 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T10:42:57.227 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T10:42:57.227 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T10:42:57.227 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T10:42:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7efed406af60 msgr2=0x7efed406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7efed406af60 0x7efed406d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efeec035000 tx=0x7efeec02e040 comp rx=0 tx=0).stop 2023-10-18T10:42:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00705a0 msgr2=0x7efef01593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00705a0 0x7efef01593c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7efee400c450 tx=0x7efee400c910 comp rx=0 tx=0).stop 2023-10-18T10:42:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 shutdown_connections 2023-10-18T10:42:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7efed406af60 0x7efed406d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efef00cbd20 0x7efef01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efef00afa20 0x7efef0145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 --2- 172.21.15.125:0/4286513434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efef00705a0 0x7efef01593c0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 >> 172.21.15.125:0/4286513434 conn(0x7efef0069930 msgr2=0x7efef0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:57.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 shutdown_connections 2023-10-18T10:42:57.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.221+0000 7efef7b4c640 1 -- 172.21.15.125:0/4286513434 wait complete. 2023-10-18T10:42:57.233 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:42:57.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.336+0000 7f911d4e7640 1 -- 172.21.15.125:0/1071806881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 msgr2=0x7f91180cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.336+0000 7f911d4e7640 1 --2- 172.21.15.125:0/1071806881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 0x7f91180cc990 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f9118061510 tx=0x7f910802c250 comp rx=0 tx=0).stop 2023-10-18T10:42:57.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.336+0000 7f911d4e7640 1 -- 172.21.15.125:0/1071806881 shutdown_connections 2023-10-18T10:42:57.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.336+0000 7f911d4e7640 1 --2- 172.21.15.125:0/1071806881 >> v2:172.21.15.139:3300/0 conn(0x7f91180ac870 0x7f911806b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.336+0000 7f911d4e7640 1 --2- 172.21.15.125:0/1071806881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 0x7f91180cc990 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.336+0000 7f911d4e7640 1 -- 172.21.15.125:0/1071806881 >> 172.21.15.125:0/1071806881 conn(0x7f911806fae0 msgr2=0x7f911806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:57.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.336+0000 7f911d4e7640 1 -- 172.21.15.125:0/1071806881 shutdown_connections 2023-10-18T10:42:57.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.336+0000 7f911d4e7640 1 -- 172.21.15.125:0/1071806881 wait complete. 2023-10-18T10:42:57.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.337+0000 7f911d4e7640 1 Processor -- start 2023-10-18T10:42:57.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.337+0000 7f911d4e7640 1 -- start start 2023-10-18T10:42:57.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f911d4e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f911806bec0 0x7f9118152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f911d4e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91180ac870 0x7f911815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f911d4e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 0x7f91180a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f911d4e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911815a510 con 0x7f9118156790 2023-10-18T10:42:57.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f911d4e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91181530b0 con 0x7f911806bec0 2023-10-18T10:42:57.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f911d4e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91181e44d0 con 0x7f91180ac870 2023-10-18T10:42:57.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 0x7f91180a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 0x7f91180a67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50550/0 (socket says 172.21.15.125:50550) 2023-10-18T10:42:57.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 -- 172.21.15.125:0/199298120 learned_addr learned my addr 172.21.15.125:0/199298120 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:57.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f9116ffd640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f911806bec0 0x7f9118152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 -- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91180ac870 msgr2=0x7f911815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91180ac870 0x7f911815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 -- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f911806bec0 msgr2=0x7f9118152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f911806bec0 0x7f9118152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 -- 172.21.15.125:0/199298120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9108009900 con 0x7f9118156790 2023-10-18T10:42:57.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f91177fe640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 0x7f91180a67c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f911000b3e0 tx=0x7f911000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:57.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f90ff7fe640 1 -- 172.21.15.125:0/199298120 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110004020 con 0x7f9118156790 2023-10-18T10:42:57.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f90ff7fe640 1 -- 172.21.15.125:0/199298120 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9110004270 con 0x7f9118156790 2023-10-18T10:42:57.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.338+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91181e47b0 con 0x7f9118156790 2023-10-18T10:42:57.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.339+0000 7f90ff7fe640 1 -- 172.21.15.125:0/199298120 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110011890 con 0x7f9118156790 2023-10-18T10:42:57.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.339+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91181e4d70 con 0x7f9118156790 2023-10-18T10:42:57.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.340+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91180b37a0 con 0x7f9118156790 2023-10-18T10:42:57.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.340+0000 7f90ff7fe640 1 -- 172.21.15.125:0/199298120 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f91100035f0 con 0x7f9118156790 2023-10-18T10:42:57.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.340+0000 7f90ff7fe640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f910406af60 0x7f910406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.341+0000 7f90ff7fe640 1 -- 172.21.15.125:0/199298120 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7f9110089770 con 0x7f9118156790 2023-10-18T10:42:57.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.341+0000 7f9116ffd640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f910406af60 0x7f910406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.341+0000 7f9116ffd640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f910406af60 0x7f910406d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9108036000 tx=0x7f9108004ab0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:57.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.345+0000 7f90ff7fe640 1 -- 172.21.15.125:0/199298120 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91100545c0 con 0x7f9118156790 2023-10-18T10:42:57.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.513+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f911815a6d0 con 0x7f9118156790 2023-10-18T10:42:57.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.516+0000 7f90ff7fe640 1 -- 172.21.15.125:0/199298120 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91100044a0 con 0x7f9118156790 2023-10-18T10:42:57.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f910406af60 msgr2=0x7f910406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f910406af60 0x7f910406d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9108036000 tx=0x7f9108004ab0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 msgr2=0x7f91180a67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 0x7f91180a67c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f911000b3e0 tx=0x7f911000b8a0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 shutdown_connections 2023-10-18T10:42:57.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f910406af60 0x7f910406d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9118156790 0x7f91180a67c0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91180ac870 0x7f911815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 --2- 172.21.15.125:0/199298120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f911806bec0 0x7f9118152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 >> 172.21.15.125:0/199298120 conn(0x7f911806fae0 msgr2=0x7f9118155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:57.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 shutdown_connections 2023-10-18T10:42:57.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.518+0000 7f911d4e7640 1 -- 172.21.15.125:0/199298120 wait complete. 2023-10-18T10:42:57.531 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:42:57.531 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:42:57.568 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:42:57.620 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:42:57.621 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:42:57.621 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:42:57.636 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:42:57.637 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:42:57.637 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:42:57.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:42:57.655 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:42:57.655 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:42:57.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:42:57.692 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:42:57.692 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:42:57.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:42:57.712 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:42:57.713 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:42:57.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.831+0000 7fdcc79dc640 1 -- 172.21.15.125:0/2246594210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 msgr2=0x7fdcc00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.831+0000 7fdcc79dc640 1 --2- 172.21.15.125:0/2246594210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 0x7fdcc00d1910 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009830 tx=0x7fdcb002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.831+0000 7fdcc79dc640 1 -- 172.21.15.125:0/2246594210 shutdown_connections 2023-10-18T10:42:57.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.831+0000 7fdcc79dc640 1 --2- 172.21.15.125:0/2246594210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 0x7fdcc00d1910 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.831+0000 7fdcc79dc640 1 -- 172.21.15.125:0/2246594210 >> 172.21.15.125:0/2246594210 conn(0x7fdcc00d0d60 msgr2=0x7fdcc00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:57.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.831+0000 7fdcc79dc640 1 -- 172.21.15.125:0/2246594210 shutdown_connections 2023-10-18T10:42:57.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.831+0000 7fdcc79dc640 1 -- 172.21.15.125:0/2246594210 wait complete. 2023-10-18T10:42:57.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.832+0000 7fdcc79dc640 1 Processor -- start 2023-10-18T10:42:57.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.832+0000 7fdcc79dc640 1 -- start start 2023-10-18T10:42:57.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc79dc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc00ceb40 0x7fdcc00c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc79dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc00cf340 0x7fdcc01f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc79dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 0x7fdcc00ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc79dc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00b69c0 con 0x7fdcc015b2a0 2023-10-18T10:42:57.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc79dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00c7e30 con 0x7fdcc00cf340 2023-10-18T10:42:57.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc79dc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00c9380 con 0x7fdcc00ceb40 2023-10-18T10:42:57.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc5f79640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 0x7fdcc00ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc5f79640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 0x7fdcc00ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48706/0 (socket says 172.21.15.125:48706) 2023-10-18T10:42:57.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc5f79640 1 -- 172.21.15.125:0/3434489677 learned_addr learned my addr 172.21.15.125:0/3434489677 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:57.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.833+0000 7fdcc5f79640 1 -- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc00ceb40 msgr2=0x7fdcc00c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcc5f79640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc00ceb40 0x7fdcc00c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcc5f79640 1 -- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc00cf340 msgr2=0x7fdcc01f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcc5f79640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc00cf340 0x7fdcc01f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcc5f79640 1 -- 172.21.15.125:0/3434489677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb0009490 con 0x7fdcc015b2a0 2023-10-18T10:42:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcc5f79640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 0x7fdcc00ca140 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c780 tx=0x7fdcb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcb5ffb640 1 -- 172.21.15.125:0/3434489677 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8004b60 con 0x7fdcc015b2a0 2023-10-18T10:42:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcb5ffb640 1 -- 172.21.15.125:0/3434489677 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdcb800ce90 con 0x7fdcc015b2a0 2023-10-18T10:42:57.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc00c8080 con 0x7fdcc015b2a0 2023-10-18T10:42:57.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcb5ffb640 1 -- 172.21.15.125:0/3434489677 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800aab0 con 0x7fdcc015b2a0 2023-10-18T10:42:57.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.834+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc00c8640 con 0x7fdcc015b2a0 2023-10-18T10:42:57.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.835+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc00690e0 con 0x7fdcc015b2a0 2023-10-18T10:42:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.835+0000 7fdcb5ffb640 1 -- 172.21.15.125:0/3434489677 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdcb800ac10 con 0x7fdcc015b2a0 2023-10-18T10:42:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.836+0000 7fdcb5ffb640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fdca406af60 0x7fdca406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.836+0000 7fdcb5ffb640 1 -- 172.21.15.125:0/3434489677 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7fdcb8089cf0 con 0x7fdcc015b2a0 2023-10-18T10:42:57.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.836+0000 7fdcc5778640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fdca406af60 0x7fdca406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:57.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.837+0000 7fdcc5778640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fdca406af60 0x7fdca406d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc001600 tx=0x7fdcbc007910 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:57.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:57.840+0000 7fdcb5ffb640 1 -- 172.21.15.125:0/3434489677 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcb808e070 con 0x7fdcc015b2a0 2023-10-18T10:42:58.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.045+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcc00d0b40 con 0x7fdcc015b2a0 2023-10-18T10:42:58.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.045+0000 7fdcb5ffb640 1 -- 172.21.15.125:0/3434489677 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fdcb8054bd0 con 0x7fdcc015b2a0 2023-10-18T10:42:58.048 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:58.048 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2381546668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2381546668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2381546668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/193356301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":193356301},{"type":"v1","addr":"172.21.15.139:6836","nonce":193356301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":8073,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3589330474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3589330474},{"type":"v1","addr":"172.21.15.125:6834","nonce":3589330474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/4188117303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":4188117303},{"type":"v1","addr":"172.21.15.125:6835","nonce":4188117303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2023-10-18T10:42:58.049 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 43 2023-10-18T10:42:58.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fdca406af60 msgr2=0x7fdca406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:58.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fdca406af60 0x7fdca406d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc001600 tx=0x7fdcbc007910 comp rx=0 tx=0).stop 2023-10-18T10:42:58.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 msgr2=0x7fdcc00ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:58.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 0x7fdcc00ca140 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c780 tx=0x7fdcb800cc40 comp rx=0 tx=0).stop 2023-10-18T10:42:58.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 shutdown_connections 2023-10-18T10:42:58.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fdca406af60 0x7fdca406d420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc00cf340 0x7fdcc01f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc015b2a0 0x7fdcc00ca140 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 --2- 172.21.15.125:0/3434489677 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc00ceb40 0x7fdcc00c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.047+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 >> 172.21.15.125:0/3434489677 conn(0x7fdcc00d0d60 msgr2=0x7fdcc0069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:58.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.048+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 shutdown_connections 2023-10-18T10:42:58.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.048+0000 7fdcc79dc640 1 -- 172.21.15.125:0/3434489677 wait complete. 2023-10-18T10:42:58.060 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:42:58.060 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:42:58.061 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:42:58.061 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:42:58.061 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:42:58.062 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:42:58.062 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:42:58.062 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:42:58.062 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:42:58.063 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:42:58.063 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:42:58.063 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:42:58.064 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:42:58.064 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:42:58.064 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:42:58.064 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:42:58.069 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:42:58.070 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:42:58.071 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:42:58.072 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:42:58.074 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:42:58.075 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:42:58.076 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:42:58.076 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:42:58.113 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:42:58.114 DEBUG:teuthology.parallel:result is None 2023-10-18T10:42:58.115 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:42:58.115 DEBUG:teuthology.parallel:result is None 2023-10-18T10:42:58.118 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:42:58.118 DEBUG:teuthology.parallel:result is None 2023-10-18T10:42:58.119 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:42:58.119 DEBUG:teuthology.parallel:result is None 2023-10-18T10:42:58.120 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:42:58.188 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:42:58.189 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:42:58.189 INFO:tasks.ceph.mds.c.smithi125.stderr:2023-10-18T10:42:58.187+0000 7f214bd6d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-18T10:42:58.201 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:42:58.201 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:42:58.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.226+0000 7f94fafc0640 1 -- 172.21.15.125:0/1986664026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 msgr2=0x7f94f40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:58.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.226+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1986664026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 0x7f94f40cf720 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f94f408a810 tx=0x7f94e002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.226+0000 7f94fafc0640 1 -- 172.21.15.125:0/1986664026 shutdown_connections 2023-10-18T10:42:58.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.226+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1986664026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 0x7f94f40cf720 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.226+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1986664026 >> v2:172.21.15.139:3300/0 conn(0x7f94f40cf340 0x7f94f40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.226+0000 7f94fafc0640 1 -- 172.21.15.125:0/1986664026 >> 172.21.15.125:0/1986664026 conn(0x7f94f40d0d60 msgr2=0x7f94f40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:58.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.226+0000 7f94fafc0640 1 -- 172.21.15.125:0/1986664026 shutdown_connections 2023-10-18T10:42:58.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.226+0000 7f94fafc0640 1 -- 172.21.15.125:0/1986664026 wait complete. 2023-10-18T10:42:58.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.227+0000 7f94fafc0640 1 Processor -- start 2023-10-18T10:42:58.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.227+0000 7f94fafc0640 1 -- start start 2023-10-18T10:42:58.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94fafc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94f40ceb40 0x7f94f41501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:58.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94fafc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94f40cf340 0x7f94f40c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:58.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94fafc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 0x7f94f40c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:58.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94fafc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40ca140 con 0x7f94f415b2a0 2023-10-18T10:42:58.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94fafc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40c92c0 con 0x7f94f40cf340 2023-10-18T10:42:58.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94fafc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f4150710 con 0x7f94f40ceb40 2023-10-18T10:42:58.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f955d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 0x7f94f40c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:58.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f955d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 0x7f94f40c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48760/0 (socket says 172.21.15.125:48760) 2023-10-18T10:42:58.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f955d640 1 -- 172.21.15.125:0/1890939578 learned_addr learned my addr 172.21.15.125:0/1890939578 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:58.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f3fff640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94f40cf340 0x7f94f40c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:58.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f8d5c640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94f40ceb40 0x7f94f41501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:58.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f3fff640 1 -- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94f40ceb40 msgr2=0x7f94f41501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:58.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f3fff640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94f40ceb40 0x7f94f41501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f3fff640 1 -- 172.21.15.125:0/1890939578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 msgr2=0x7f94f40c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:58.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f3fff640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 0x7f94f40c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:58.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.228+0000 7f94f3fff640 1 -- 172.21.15.125:0/1890939578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e0009490 con 0x7f94f40cf340 2023-10-18T10:42:58.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.229+0000 7f94f955d640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 0x7f94f40c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:42:58.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.229+0000 7f94f3fff640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94f40cf340 0x7f94f40c8840 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94e0033000 tx=0x7f94e00045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:58.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.229+0000 7f94f17fa640 1 -- 172.21.15.125:0/1890939578 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0003a00 con 0x7f94f40cf340 2023-10-18T10:42:58.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.229+0000 7f94f17fa640 1 -- 172.21.15.125:0/1890939578 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94e0003c90 con 0x7f94f40cf340 2023-10-18T10:42:58.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.229+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f40c94c0 con 0x7f94f40cf340 2023-10-18T10:42:58.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.229+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f40c9a80 con 0x7f94f40cf340 2023-10-18T10:42:58.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.229+0000 7f94f17fa640 1 -- 172.21.15.125:0/1890939578 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0034440 con 0x7f94f40cf340 2023-10-18T10:42:58.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.230+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f40690e0 con 0x7f94f40cf340 2023-10-18T10:42:58.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.230+0000 7f94f17fa640 1 -- 172.21.15.125:0/1890939578 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f94e003b040 con 0x7f94f40cf340 2023-10-18T10:42:58.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.231+0000 7f94f17fa640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f94d806af90 0x7f94d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:58.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.231+0000 7f94f17fa640 1 -- 172.21.15.125:0/1890939578 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6037+0+0 (secure 0 0 0) 0x7f94e00ab830 con 0x7f94f40cf340 2023-10-18T10:42:58.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.231+0000 7f94f8d5c640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f94d806af90 0x7f94d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:58.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.231+0000 7f94f8d5c640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f94d806af90 0x7f94d806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94e80058c0 tx=0x7f94e800a2d0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:58.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.235+0000 7f94f17fa640 1 -- 172.21.15.125:0/1890939578 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94e0076730 con 0x7f94f40cf340 2023-10-18T10:42:58.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:58.415+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f94f40d0b40 con 0x7f94f40cf340 2023-10-18T10:42:59.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.435+0000 7f94f17fa640 1 -- 172.21.15.125:0/1890939578 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v124) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94e0039070 con 0x7f94f40cf340 2023-10-18T10:42:59.438 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T10:42:59.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f94d806af90 msgr2=0x7f94d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f94d806af90 0x7f94d806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94e80058c0 tx=0x7f94e800a2d0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94f40cf340 msgr2=0x7f94f40c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94f40cf340 0x7f94f40c8840 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94e0033000 tx=0x7f94e00045e0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 shutdown_connections 2023-10-18T10:42:59.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f94d806af90 0x7f94d806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94f40cf340 0x7f94f40c8840 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94f415b2a0 0x7f94f40c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 --2- 172.21.15.125:0/1890939578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94f40ceb40 0x7f94f41501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 >> 172.21.15.125:0/1890939578 conn(0x7f94f40d0d60 msgr2=0x7f94f40d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:59.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 shutdown_connections 2023-10-18T10:42:59.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.439+0000 7f94fafc0640 1 -- 172.21.15.125:0/1890939578 wait complete. 2023-10-18T10:42:59.454 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:42:59.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 -- 172.21.15.125:0/3461034141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c0ceb40 msgr2=0x7f337c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 --2- 172.21.15.125:0/3461034141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c0ceb40 0x7f337c0b0650 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f337c08a810 tx=0x7f337402bd10 comp rx=0 tx=0).stop 2023-10-18T10:42:59.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 -- 172.21.15.125:0/3461034141 shutdown_connections 2023-10-18T10:42:59.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 --2- 172.21.15.125:0/3461034141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c0ceb40 0x7f337c0b0650 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 --2- 172.21.15.125:0/3461034141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337c15b2a0 0x7f337c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 --2- 172.21.15.125:0/3461034141 >> v2:172.21.15.139:3300/0 conn(0x7f337c0cf340 0x7f337c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 -- 172.21.15.125:0/3461034141 >> 172.21.15.125:0/3461034141 conn(0x7f337c0d0d60 msgr2=0x7f337c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:59.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 -- 172.21.15.125:0/3461034141 shutdown_connections 2023-10-18T10:42:59.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.559+0000 7f33838c2640 1 -- 172.21.15.125:0/3461034141 wait complete. 2023-10-18T10:42:59.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.560+0000 7f33838c2640 1 Processor -- start 2023-10-18T10:42:59.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.560+0000 7f33838c2640 1 -- start start 2023-10-18T10:42:59.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f33838c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f337c0ceb40 0x7f337c1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:59.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f33838c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337c0cf340 0x7f337c1e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:59.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f33838c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c15b2a0 0x7f337c1e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:59.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f33838c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c153d90 con 0x7f337c15b2a0 2023-10-18T10:42:59.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f33838c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c158770 con 0x7f337c0ceb40 2023-10-18T10:42:59.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f33838c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c157b60 con 0x7f337c0cf340 2023-10-18T10:42:59.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f3381e5f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c15b2a0 0x7f337c1e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:59.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f3381e5f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c15b2a0 0x7f337c1e5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48780/0 (socket says 172.21.15.125:48780) 2023-10-18T10:42:59.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f3381e5f640 1 -- 172.21.15.125:0/3612629790 learned_addr learned my addr 172.21.15.125:0/3612629790 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:59.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f338165e640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f337c0ceb40 0x7f337c1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:59.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f3380e5d640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337c0cf340 0x7f337c1e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:59.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f3381e5f640 1 -- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337c0cf340 msgr2=0x7f337c1e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.561+0000 7f3381e5f640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337c0cf340 0x7f337c1e5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f3381e5f640 1 -- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f337c0ceb40 msgr2=0x7f337c1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f3381e5f640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f337c0ceb40 0x7f337c1574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f3381e5f640 1 -- 172.21.15.125:0/3612629790 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3374009560 con 0x7f337c15b2a0 2023-10-18T10:42:59.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f3381e5f640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c15b2a0 0x7f337c1e5dc0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f3374036040 tx=0x7f337402dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:59.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f3371ffb640 1 -- 172.21.15.125:0/3612629790 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374003800 con 0x7f337c15b2a0 2023-10-18T10:42:59.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f3371ffb640 1 -- 172.21.15.125:0/3612629790 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3374003a50 con 0x7f337c15b2a0 2023-10-18T10:42:59.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f3371ffb640 1 -- 172.21.15.125:0/3612629790 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374003ea0 con 0x7f337c15b2a0 2023-10-18T10:42:59.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337c157e40 con 0x7f337c15b2a0 2023-10-18T10:42:59.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.562+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c158400 con 0x7f337c15b2a0 2023-10-18T10:42:59.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.563+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337c0690e0 con 0x7f337c15b2a0 2023-10-18T10:42:59.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.563+0000 7f3371ffb640 1 -- 172.21.15.125:0/3612629790 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f337403e450 con 0x7f337c15b2a0 2023-10-18T10:42:59.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.564+0000 7f3371ffb640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f336006af60 0x7f336006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:59.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.564+0000 7f3371ffb640 1 -- 172.21.15.125:0/3612629790 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f33740ac1f0 con 0x7f337c15b2a0 2023-10-18T10:42:59.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.564+0000 7f338165e640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f336006af60 0x7f336006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:59.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.565+0000 7f338165e640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f336006af60 0x7f336006d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3378005fd0 tx=0x7f3378008230 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:59.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.568+0000 7f3371ffb640 1 -- 172.21.15.125:0/3612629790 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33740773b0 con 0x7f337c15b2a0 2023-10-18T10:42:59.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.796+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f337c06b190 con 0x7f337c15b2a0 2023-10-18T10:42:59.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.797+0000 7f3371ffb640 1 -- 172.21.15.125:0/3612629790 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3374042090 con 0x7f337c15b2a0 2023-10-18T10:42:59.799 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:42:59.800 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:42:59.800 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T10:42:59.800 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.800 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T10:42:59.800 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T10:42:59.801 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.801 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:42:59.801 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:42:59.801 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:42:59.801 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.802 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.802 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.802 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.802 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T10:42:59.802 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T10:42:59.803 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.803 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:42:59.803 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:42:59.803 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:42:59.803 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.804 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.804 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.804 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.804 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T10:42:59.804 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T10:42:59.805 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.805 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:42:59.805 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:42:59.805 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:42:59.805 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.806 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.806 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.806 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.806 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T10:42:59.806 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T10:42:59.807 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.807 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:42:59.807 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:42:59.807 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:42:59.807 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.807 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.808 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.808 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.808 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T10:42:59.808 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T10:42:59.808 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.809 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:42:59.809 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:42:59.809 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.809 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.809 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.810 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.810 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T10:42:59.810 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T10:42:59.810 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.810 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:42:59.811 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:42:59.811 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.811 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.811 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.811 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.812 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T10:42:59.812 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T10:42:59.812 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.812 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:42:59.812 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:42:59.812 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.813 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.813 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.813 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.813 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T10:42:59.813 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T10:42:59.814 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.814 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:42:59.814 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:42:59.814 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.814 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.815 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.815 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.815 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T10:42:59.815 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T10:42:59.815 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.816 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:42:59.816 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:42:59.816 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.816 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.816 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.817 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.817 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T10:42:59.817 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T10:42:59.817 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.817 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:42:59.817 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:42:59.818 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.818 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.818 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.818 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.818 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T10:42:59.819 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T10:42:59.819 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.819 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:42:59.819 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:42:59.819 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.820 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.820 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.820 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.820 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T10:42:59.820 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T10:42:59.821 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.821 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:42:59.821 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:42:59.821 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.821 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.821 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.822 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.822 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T10:42:59.822 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T10:42:59.822 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.822 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:42:59.823 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T10:42:59.823 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T10:42:59.823 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.823 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.823 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.824 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T10:42:59.824 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T10:42:59.824 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.824 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:42:59.824 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:42:59.825 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:42:59.825 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:42:59.825 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.825 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.825 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.825 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T10:42:59.826 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T10:42:59.826 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.826 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:42:59.826 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:42:59.826 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:42:59.827 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:42:59.827 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.827 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.827 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.827 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T10:42:59.828 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T10:42:59.828 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.828 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:42:59.828 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:42:59.828 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:42:59.829 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:42:59.829 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.829 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.829 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.829 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T10:42:59.830 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T10:42:59.830 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.830 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:42:59.830 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:42:59.830 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:42:59.831 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.831 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.831 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.831 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.831 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T10:42:59.831 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T10:42:59.832 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.832 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:42:59.832 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:42:59.832 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.832 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.833 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:42:59.833 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:42:59.833 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T10:42:59.833 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T10:42:59.833 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:42:59.834 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:42:59.834 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:42:59.834 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:42:59.834 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.834 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:42:59.835 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T10:42:59.835 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:42:59.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f336006af60 msgr2=0x7f336006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f336006af60 0x7f336006d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3378005fd0 tx=0x7f3378008230 comp rx=0 tx=0).stop 2023-10-18T10:42:59.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c15b2a0 msgr2=0x7f337c1e5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c15b2a0 0x7f337c1e5dc0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f3374036040 tx=0x7f337402dad0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 shutdown_connections 2023-10-18T10:42:59.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f336006af60 0x7f336006d420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f337c15b2a0 0x7f337c1e5dc0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337c0cf340 0x7f337c1e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 --2- 172.21.15.125:0/3612629790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f337c0ceb40 0x7f337c1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 >> 172.21.15.125:0/3612629790 conn(0x7f337c0d0d60 msgr2=0x7f337c0af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:59.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 shutdown_connections 2023-10-18T10:42:59.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.799+0000 7f33838c2640 1 -- 172.21.15.125:0/3612629790 wait complete. 2023-10-18T10:42:59.839 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:42:59.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.940+0000 7f77f9683640 1 -- 172.21.15.125:0/526684915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77f413e520 msgr2=0x7f77f406bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.940+0000 7f77f9683640 1 --2- 172.21.15.125:0/526684915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77f413e520 0x7f77f406bb60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f77e4005fd0 tx=0x7f77e402bd90 comp rx=0 tx=0).stop 2023-10-18T10:42:59.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.940+0000 7f77f9683640 1 -- 172.21.15.125:0/526684915 shutdown_connections 2023-10-18T10:42:59.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.940+0000 7f77f9683640 1 --2- 172.21.15.125:0/526684915 >> v2:172.21.15.139:3300/0 conn(0x7f77f4070af0 0x7f77f40affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.940+0000 7f77f9683640 1 --2- 172.21.15.125:0/526684915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77f413e520 0x7f77f406bb60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.940+0000 7f77f9683640 1 -- 172.21.15.125:0/526684915 >> 172.21.15.125:0/526684915 conn(0x7f77f4069930 msgr2=0x7f77f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:42:59.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.940+0000 7f77f9683640 1 -- 172.21.15.125:0/526684915 shutdown_connections 2023-10-18T10:42:59.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.940+0000 7f77f9683640 1 -- 172.21.15.125:0/526684915 wait complete. 2023-10-18T10:42:59.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f9683640 1 Processor -- start 2023-10-18T10:42:59.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f9683640 1 -- start start 2023-10-18T10:42:59.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f9683640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77f4070af0 0x7f77f413b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:59.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f9683640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f413e520 0x7f77f41db730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:59.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f9683640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77f4141690 0x7f77f40ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:59.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f9683640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40ad070 con 0x7f77f4070af0 2023-10-18T10:42:59.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f9683640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41e1720 con 0x7f77f413e520 2023-10-18T10:42:59.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f9683640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40ad2f0 con 0x7f77f4141690 2023-10-18T10:42:59.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77f4070af0 0x7f77f413b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:59.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77f4070af0 0x7f77f413b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48796/0 (socket says 172.21.15.125:48796) 2023-10-18T10:42:59.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f2ffd640 1 -- 172.21.15.125:0/2872872421 learned_addr learned my addr 172.21.15.125:0/2872872421 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:42:59.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f37fe640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77f4141690 0x7f77f40ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:59.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.942+0000 7f77f27fc640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f413e520 0x7f77f41db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:59.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.943+0000 7f77f2ffd640 1 -- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77f4141690 msgr2=0x7f77f40ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.943+0000 7f77f2ffd640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77f4141690 0x7f77f40ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.943+0000 7f77f2ffd640 1 -- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f413e520 msgr2=0x7f77f41db730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:42:59.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.943+0000 7f77f2ffd640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f413e520 0x7f77f41db730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:42:59.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.943+0000 7f77f2ffd640 1 -- 172.21.15.125:0/2872872421 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4009560 con 0x7f77f4070af0 2023-10-18T10:42:59.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.943+0000 7f77f2ffd640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77f4070af0 0x7f77f413b1b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e402d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:59.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.944+0000 7f77db7fe640 1 -- 172.21.15.125:0/2872872421 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e402ddf0 con 0x7f77f4070af0 2023-10-18T10:42:59.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.944+0000 7f77db7fe640 1 -- 172.21.15.125:0/2872872421 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f77e40036e0 con 0x7f77f4070af0 2023-10-18T10:42:59.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.944+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f40ad5d0 con 0x7f77f4070af0 2023-10-18T10:42:59.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.944+0000 7f77db7fe640 1 -- 172.21.15.125:0/2872872421 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4003af0 con 0x7f77f4070af0 2023-10-18T10:42:59.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.944+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f40adb90 con 0x7f77f4070af0 2023-10-18T10:42:59.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.945+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f41e1890 con 0x7f77f4070af0 2023-10-18T10:42:59.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.945+0000 7f77db7fe640 1 -- 172.21.15.125:0/2872872421 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f77e4003c50 con 0x7f77f4070af0 2023-10-18T10:42:59.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.946+0000 7f77db7fe640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f77e006af90 0x7f77e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:42:59.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.946+0000 7f77db7fe640 1 -- 172.21.15.125:0/2872872421 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f77e40abf30 con 0x7f77f4070af0 2023-10-18T10:42:59.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.946+0000 7f77f27fc640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f77e006af90 0x7f77e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:42:59.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.946+0000 7f77f27fc640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f77e006af90 0x7f77e006d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77f40cb950 tx=0x7f77e8009040 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:42:59.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:42:59.950+0000 7f77db7fe640 1 -- 172.21.15.125:0/2872872421 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77f41e1890 con 0x7f77f4070af0 2023-10-18T10:43:00.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.119+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77f41e1890 con 0x7f77f4070af0 2023-10-18T10:43:00.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.119+0000 7f77db7fe640 1 -- 172.21.15.125:0/2872872421 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+13120 (secure 0 0 0) 0x7f77f41e1890 con 0x7f77f4070af0 2023-10-18T10:43:00.122 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:00.122 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":124,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:42:59.429463+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:43:00.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f77e006af90 msgr2=0x7f77e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f77e006af90 0x7f77e006d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77f40cb950 tx=0x7f77e8009040 comp rx=0 tx=0).stop 2023-10-18T10:43:00.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77f4070af0 msgr2=0x7f77f413b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77f4070af0 0x7f77f413b1b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e402d9f0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 shutdown_connections 2023-10-18T10:43:00.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f77e006af90 0x7f77e006d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f413e520 0x7f77f41db730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77f4141690 0x7f77f40ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 --2- 172.21.15.125:0/2872872421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77f4070af0 0x7f77f413b1b0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 >> 172.21.15.125:0/2872872421 conn(0x7f77f4069930 msgr2=0x7f77f41df990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:00.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 shutdown_connections 2023-10-18T10:43:00.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.121+0000 7f77f9683640 1 -- 172.21.15.125:0/2872872421 wait complete. 2023-10-18T10:43:00.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:43:00.135 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:43:00.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.238+0000 7f0216410640 1 -- 172.21.15.125:0/3769882777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02100ceb40 msgr2=0x7f02100b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.238+0000 7f0216410640 1 --2- 172.21.15.125:0/3769882777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02100ceb40 0x7f02100b0650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0208005fd0 tx=0x7f020802c240 comp rx=0 tx=0).stop 2023-10-18T10:43:00.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.238+0000 7f0216410640 1 -- 172.21.15.125:0/3769882777 shutdown_connections 2023-10-18T10:43:00.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.238+0000 7f0216410640 1 --2- 172.21.15.125:0/3769882777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02100ceb40 0x7f02100b0650 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.238+0000 7f0216410640 1 --2- 172.21.15.125:0/3769882777 >> v2:172.21.15.139:3300/0 conn(0x7f02100cf340 0x7f02100d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.238+0000 7f0216410640 1 -- 172.21.15.125:0/3769882777 >> 172.21.15.125:0/3769882777 conn(0x7f02100d0d60 msgr2=0x7f02100d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:00.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.239+0000 7f0216410640 1 -- 172.21.15.125:0/3769882777 shutdown_connections 2023-10-18T10:43:00.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.239+0000 7f0216410640 1 -- 172.21.15.125:0/3769882777 wait complete. 2023-10-18T10:43:00.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.240+0000 7f0216410640 1 Processor -- start 2023-10-18T10:43:00.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.240+0000 7f0216410640 1 -- start start 2023-10-18T10:43:00.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.240+0000 7f0216410640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02100ceb40 0x7f021015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.240+0000 7f0216410640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02100cf340 0x7f02100ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.240+0000 7f0216410640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021015b2a0 0x7f0210146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.240+0000 7f0216410640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021015dca0 con 0x7f021015b2a0 2023-10-18T10:43:00.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.240+0000 7f0216410640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021015e0c0 con 0x7f02100ceb40 2023-10-18T10:43:00.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.240+0000 7f0216410640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021015d020 con 0x7f02100cf340 2023-10-18T10:43:00.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021015b2a0 0x7f0210146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021015b2a0 0x7f0210146f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48806/0 (socket says 172.21.15.125:48806) 2023-10-18T10:43:00.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 -- 172.21.15.125:0/3175050794 learned_addr learned my addr 172.21.15.125:0/3175050794 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:00.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f020f7fe640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02100cf340 0x7f02100ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f020ffff640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02100ceb40 0x7f021015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 -- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02100cf340 msgr2=0x7f02100ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02100cf340 0x7f02100ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 -- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02100ceb40 msgr2=0x7f021015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02100ceb40 0x7f021015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 -- 172.21.15.125:0/3175050794 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0200009490 con 0x7f021015b2a0 2023-10-18T10:43:00.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f02149ad640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021015b2a0 0x7f0210146f90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0208009bf0 tx=0x7f0208035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:00.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f020cff9640 1 -- 172.21.15.125:0/3175050794 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208034440 con 0x7f021015b2a0 2023-10-18T10:43:00.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f020cff9640 1 -- 172.21.15.125:0/3175050794 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0208003e60 con 0x7f021015b2a0 2023-10-18T10:43:00.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f020cff9640 1 -- 172.21.15.125:0/3175050794 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02080044b0 con 0x7f021015b2a0 2023-10-18T10:43:00.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0208009900 con 0x7f021015b2a0 2023-10-18T10:43:00.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.241+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021015d5e0 con 0x7f021015b2a0 2023-10-18T10:43:00.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.242+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021015dde0 con 0x7f021015b2a0 2023-10-18T10:43:00.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.243+0000 7f020cff9640 1 -- 172.21.15.125:0/3175050794 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f020800a040 con 0x7f021015b2a0 2023-10-18T10:43:00.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.243+0000 7f020cff9640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f01f006af60 0x7f01f006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.243+0000 7f020cff9640 1 -- 172.21.15.125:0/3175050794 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f02080ac3e0 con 0x7f021015b2a0 2023-10-18T10:43:00.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.243+0000 7f020ffff640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f01f006af60 0x7f01f006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.244+0000 7f020ffff640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f01f006af60 0x7f01f006d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0200009cf0 tx=0x7f0200005f90 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:00.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.247+0000 7f020cff9640 1 -- 172.21.15.125:0/3175050794 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f021015dde0 con 0x7f021015b2a0 2023-10-18T10:43:00.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.410+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f021015dde0 con 0x7f021015b2a0 2023-10-18T10:43:00.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.411+0000 7f020cff9640 1 -- 172.21.15.125:0/3175050794 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0208044020 con 0x7f021015b2a0 2023-10-18T10:43:00.413 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:43:00.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f01f006af60 msgr2=0x7f01f006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f01f006af60 0x7f01f006d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0200009cf0 tx=0x7f0200005f90 comp rx=0 tx=0).stop 2023-10-18T10:43:00.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021015b2a0 msgr2=0x7f0210146f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021015b2a0 0x7f0210146f90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0208009bf0 tx=0x7f0208035e30 comp rx=0 tx=0).stop 2023-10-18T10:43:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 shutdown_connections 2023-10-18T10:43:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f01f006af60 0x7f01f006d420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f021015b2a0 0x7f0210146f90 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02100cf340 0x7f02100ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 --2- 172.21.15.125:0/3175050794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02100ceb40 0x7f021015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 >> 172.21.15.125:0/3175050794 conn(0x7f02100d0d60 msgr2=0x7f02100b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:00.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 shutdown_connections 2023-10-18T10:43:00.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.413+0000 7f0216410640 1 -- 172.21.15.125:0/3175050794 wait complete. 2023-10-18T10:43:00.425 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:43:00.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.527+0000 7f252b044640 1 -- 172.21.15.125:0/2096325769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2514005180 msgr2=0x7f251400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.527+0000 7f252b044640 1 --2- 172.21.15.125:0/2096325769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2514005180 0x7f251400b980 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f251c00ed10 tx=0x7f25140057f0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.527+0000 7f252b044640 1 -- 172.21.15.125:0/2096325769 shutdown_connections 2023-10-18T10:43:00.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.527+0000 7f252b044640 1 --2- 172.21.15.125:0/2096325769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2514005180 0x7f251400b980 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.527+0000 7f252b044640 1 -- 172.21.15.125:0/2096325769 >> 172.21.15.125:0/2096325769 conn(0x7f25240b0670 msgr2=0x7f25240b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:00.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.527+0000 7f252b044640 1 -- 172.21.15.125:0/2096325769 shutdown_connections 2023-10-18T10:43:00.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.527+0000 7f252b044640 1 -- 172.21.15.125:0/2096325769 wait complete. 2023-10-18T10:43:00.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.528+0000 7f252b044640 1 Processor -- start 2023-10-18T10:43:00.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.528+0000 7f252b044640 1 -- start start 2023-10-18T10:43:00.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f252b044640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2514005180 0x7f25241575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f252b044640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2524152310 0x7f25240c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f252b044640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25240c8740 0x7f25240c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f252b044640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524157b10 con 0x7f2524152310 2023-10-18T10:43:00.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f252b044640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524157140 con 0x7f25240c8740 2023-10-18T10:43:00.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f252b044640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2524156530 con 0x7f2514005180 2023-10-18T10:43:00.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2524152310 0x7f25240c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2524152310 0x7f25240c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48824/0 (socket says 172.21.15.125:48824) 2023-10-18T10:43:00.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 -- 172.21.15.125:0/445237549 learned_addr learned my addr 172.21.15.125:0/445237549 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:00.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f25295e1640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25240c8740 0x7f25240c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2528de0640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2514005180 0x7f25241575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 -- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2514005180 msgr2=0x7f25241575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2514005180 0x7f25241575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 -- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25240c8740 msgr2=0x7f25240c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25240c8740 0x7f25240c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 -- 172.21.15.125:0/445237549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524156810 con 0x7f2524152310 2023-10-18T10:43:00.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f2523fff640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2524152310 0x7f25240c8200 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2510011ab0 tx=0x7f2510011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:00.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.529+0000 7f25217fa640 1 -- 172.21.15.125:0/445237549 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251000fe70 con 0x7f2524152310 2023-10-18T10:43:00.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.530+0000 7f25217fa640 1 -- 172.21.15.125:0/445237549 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2510004050 con 0x7f2524152310 2023-10-18T10:43:00.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.530+0000 7f25217fa640 1 -- 172.21.15.125:0/445237549 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510004440 con 0x7f2524152310 2023-10-18T10:43:00.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.530+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2524156af0 con 0x7f2524152310 2023-10-18T10:43:00.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.530+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2524156150 con 0x7f2524152310 2023-10-18T10:43:00.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.530+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25240c8b20 con 0x7f2524152310 2023-10-18T10:43:00.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.531+0000 7f25217fa640 1 -- 172.21.15.125:0/445237549 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f25100035f0 con 0x7f2524152310 2023-10-18T10:43:00.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.531+0000 7f25217fa640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f250806af60 0x7f250806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.531+0000 7f25217fa640 1 -- 172.21.15.125:0/445237549 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f251008a410 con 0x7f2524152310 2023-10-18T10:43:00.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.531+0000 7f2528de0640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f250806af60 0x7f250806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.532+0000 7f2528de0640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f250806af60 0x7f250806d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25180017a0 tx=0x7f2518001c30 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:00.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.535+0000 7f25217fa640 1 -- 172.21.15.125:0/445237549 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25100554d0 con 0x7f2524152310 2023-10-18T10:43:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.705+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25240675f0 con 0x7f2524152310 2023-10-18T10:43:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.705+0000 7f25217fa640 1 -- 172.21.15.125:0/445237549 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v124) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2510058af0 con 0x7f2524152310 2023-10-18T10:43:00.707 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:00.708 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:43:00.708 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:43:00.708 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:43:00.708 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:43:00.709 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:43:00.709 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:43:00.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f250806af60 msgr2=0x7f250806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f250806af60 0x7f250806d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25180017a0 tx=0x7f2518001c30 comp rx=0 tx=0).stop 2023-10-18T10:43:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2524152310 msgr2=0x7f25240c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2524152310 0x7f25240c8200 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2510011ab0 tx=0x7f2510011f70 comp rx=0 tx=0).stop 2023-10-18T10:43:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 shutdown_connections 2023-10-18T10:43:00.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f250806af60 0x7f250806d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25240c8740 0x7f25240c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2524152310 0x7f25240c8200 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 --2- 172.21.15.125:0/445237549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2514005180 0x7f25241575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 >> 172.21.15.125:0/445237549 conn(0x7f25240b0670 msgr2=0x7f252406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:00.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 shutdown_connections 2023-10-18T10:43:00.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.707+0000 7f252b044640 1 -- 172.21.15.125:0/445237549 wait complete. 2023-10-18T10:43:00.719 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:43:00.720 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:43:00.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.825+0000 7fcd5ac56640 1 -- 172.21.15.125:0/2214178750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd540b03a0 msgr2=0x7fcd540b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.825+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/2214178750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd540b03a0 0x7fcd540b07d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c005fd0 tx=0x7fcd4c02c2e0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.825+0000 7fcd5ac56640 1 -- 172.21.15.125:0/2214178750 shutdown_connections 2023-10-18T10:43:00.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.825+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/2214178750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd540b03a0 0x7fcd540b07d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.825+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/2214178750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd541519d0 0x7fcd54158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.825+0000 7fcd5ac56640 1 -- 172.21.15.125:0/2214178750 >> 172.21.15.125:0/2214178750 conn(0x7fcd54069930 msgr2=0x7fcd54069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.825+0000 7fcd5ac56640 1 -- 172.21.15.125:0/2214178750 shutdown_connections 2023-10-18T10:43:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.825+0000 7fcd5ac56640 1 -- 172.21.15.125:0/2214178750 wait complete. 2023-10-18T10:43:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.826+0000 7fcd5ac56640 1 Processor -- start 2023-10-18T10:43:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.826+0000 7fcd5ac56640 1 -- start start 2023-10-18T10:43:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd5ac56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd5406cf80 0x7fcd540aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd5ac56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd540b03a0 0x7fcd5414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd5ac56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd541519d0 0x7fcd54148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd5ac56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd54156e40 con 0x7fcd541519d0 2023-10-18T10:43:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd5ac56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5414cb50 con 0x7fcd540b03a0 2023-10-18T10:43:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd5ac56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540aaf50 con 0x7fcd5406cf80 2023-10-18T10:43:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd541519d0 0x7fcd54148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd541519d0 0x7fcd54148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48826/0 (socket says 172.21.15.125:48826) 2023-10-18T10:43:00.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 -- 172.21.15.125:0/3225324315 learned_addr learned my addr 172.21.15.125:0/3225324315 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:00.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd589f2640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd5406cf80 0x7fcd540aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd53fff640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd540b03a0 0x7fcd5414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 -- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd5406cf80 msgr2=0x7fcd540aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd5406cf80 0x7fcd540aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 -- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd540b03a0 msgr2=0x7fcd5414c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:00.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd540b03a0 0x7fcd5414c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:00.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 -- 172.21.15.125:0/3225324315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c009900 con 0x7fcd541519d0 2023-10-18T10:43:00.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd591f3640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd541519d0 0x7fcd54148b80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009bf0 tx=0x7fcd4c035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:00.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd517fa640 1 -- 172.21.15.125:0/3225324315 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c004240 con 0x7fcd541519d0 2023-10-18T10:43:00.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd517fa640 1 -- 172.21.15.125:0/3225324315 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd4c004490 con 0x7fcd541519d0 2023-10-18T10:43:00.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd517fa640 1 -- 172.21.15.125:0/3225324315 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c004860 con 0x7fcd541519d0 2023-10-18T10:43:00.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd540ab230 con 0x7fcd541519d0 2023-10-18T10:43:00.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.827+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd540ab7f0 con 0x7fcd541519d0 2023-10-18T10:43:00.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.828+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd540b1660 con 0x7fcd541519d0 2023-10-18T10:43:00.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.829+0000 7fcd517fa640 1 -- 172.21.15.125:0/3225324315 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcd4c03b050 con 0x7fcd541519d0 2023-10-18T10:43:00.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.829+0000 7fcd517fa640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd3806af60 0x7fcd3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:00.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.829+0000 7fcd517fa640 1 -- 172.21.15.125:0/3225324315 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fcd4c0abea0 con 0x7fcd541519d0 2023-10-18T10:43:00.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.829+0000 7fcd589f2640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd3806af60 0x7fcd3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:00.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.830+0000 7fcd589f2640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd3806af60 0x7fcd3806d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009b30 tx=0x7fcd48005f90 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:00.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:00.833+0000 7fcd517fa640 1 -- 172.21.15.125:0/3225324315 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd4c077080 con 0x7fcd541519d0 2023-10-18T10:43:01.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.001+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcd540690e0 con 0x7fcd541519d0 2023-10-18T10:43:01.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.947+0000 7fcd517fa640 1 -- 172.21.15.125:0/3225324315 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v125) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcd4c003e60 con 0x7fcd541519d0 2023-10-18T10:43:01.949 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:43:01.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd3806af60 msgr2=0x7fcd3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd3806af60 0x7fcd3806d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009b30 tx=0x7fcd48005f90 comp rx=0 tx=0).stop 2023-10-18T10:43:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd541519d0 msgr2=0x7fcd54148b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd541519d0 0x7fcd54148b80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009bf0 tx=0x7fcd4c035b50 comp rx=0 tx=0).stop 2023-10-18T10:43:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 shutdown_connections 2023-10-18T10:43:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd3806af60 0x7fcd3806d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:01.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd540b03a0 0x7fcd5414c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:01.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd541519d0 0x7fcd54148b80 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:01.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 --2- 172.21.15.125:0/3225324315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd5406cf80 0x7fcd540aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:01.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 >> 172.21.15.125:0/3225324315 conn(0x7fcd54069930 msgr2=0x7fcd540aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:01.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 shutdown_connections 2023-10-18T10:43:01.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:01.950+0000 7fcd5ac56640 1 -- 172.21.15.125:0/3225324315 wait complete. 2023-10-18T10:43:01.968 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:43:02.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.080+0000 7f3f5d77d640 1 -- 172.21.15.125:0/1513059669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 msgr2=0x7f3f580ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:02.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.080+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/1513059669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 0x7f3f580ce9a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3f58072c60 tx=0x7f3f4802bdb0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.080+0000 7f3f5d77d640 1 -- 172.21.15.125:0/1513059669 shutdown_connections 2023-10-18T10:43:02.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.081+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/1513059669 >> v2:172.21.15.139:3300/0 conn(0x7f3f580b1ca0 0x7f3f580cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.081+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/1513059669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 0x7f3f580ce9a0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.081+0000 7f3f5d77d640 1 -- 172.21.15.125:0/1513059669 >> 172.21.15.125:0/1513059669 conn(0x7f3f580d0c50 msgr2=0x7f3f580d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:02.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.081+0000 7f3f5d77d640 1 -- 172.21.15.125:0/1513059669 shutdown_connections 2023-10-18T10:43:02.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.081+0000 7f3f5d77d640 1 -- 172.21.15.125:0/1513059669 wait complete. 2023-10-18T10:43:02.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.082+0000 7f3f5d77d640 1 Processor -- start 2023-10-18T10:43:02.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.082+0000 7f3f5d77d640 1 -- start start 2023-10-18T10:43:02.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.082+0000 7f3f5d77d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f580b1ca0 0x7f3f5809d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:02.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.082+0000 7f3f5d77d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f580cf6e0 0x7f3f580c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:02.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.082+0000 7f3f5d77d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 0x7f3f581f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:02.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.082+0000 7f3f5d77d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5814db00 con 0x7f3f5815e5e0 2023-10-18T10:43:02.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.082+0000 7f3f5d77d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580ca190 con 0x7f3f580cf6e0 2023-10-18T10:43:02.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.082+0000 7f3f5d77d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580ca470 con 0x7f3f580b1ca0 2023-10-18T10:43:02.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 0x7f3f581f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:02.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 0x7f3f581f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48836/0 (socket says 172.21.15.125:48836) 2023-10-18T10:43:02.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 -- 172.21.15.125:0/2397839049 learned_addr learned my addr 172.21.15.125:0/2397839049 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:02.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f56ffd640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f580b1ca0 0x7f3f5809d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:02.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f4ffff640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f580cf6e0 0x7f3f580c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:02.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 -- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f580b1ca0 msgr2=0x7f3f5809d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:02.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f580b1ca0 0x7f3f5809d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 -- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f580cf6e0 msgr2=0x7f3f580c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:02.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f580cf6e0 0x7f3f580c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 -- 172.21.15.125:0/2397839049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48009560 con 0x7f3f5815e5e0 2023-10-18T10:43:02.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f577fe640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 0x7f3f581f4d70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000c780 tx=0x7f3f5000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:02.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f4f7fe640 1 -- 172.21.15.125:0/2397839049 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50004b60 con 0x7f3f5815e5e0 2023-10-18T10:43:02.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f4f7fe640 1 -- 172.21.15.125:0/2397839049 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f5000ce90 con 0x7f3f5815e5e0 2023-10-18T10:43:02.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5814dd10 con 0x7f3f5815e5e0 2023-10-18T10:43:02.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f4f7fe640 1 -- 172.21.15.125:0/2397839049 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000aab0 con 0x7f3f5815e5e0 2023-10-18T10:43:02.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.083+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5806b1c0 con 0x7f3f5815e5e0 2023-10-18T10:43:02.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.084+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5814e230 con 0x7f3f5815e5e0 2023-10-18T10:43:02.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.085+0000 7f3f4f7fe640 1 -- 172.21.15.125:0/2397839049 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3f5000ac10 con 0x7f3f5815e5e0 2023-10-18T10:43:02.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.085+0000 7f3f4f7fe640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f3f3806af60 0x7f3f3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:02.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.085+0000 7f3f4f7fe640 1 -- 172.21.15.125:0/2397839049 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f3f50089a80 con 0x7f3f5815e5e0 2023-10-18T10:43:02.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.085+0000 7f3f56ffd640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f3f3806af60 0x7f3f3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:02.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.086+0000 7f3f56ffd640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f3f3806af60 0x7f3f3806d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f48035000 tx=0x7f3f48004730 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:02.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.089+0000 7f3f4f7fe640 1 -- 172.21.15.125:0/2397839049 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f5814e230 con 0x7f3f5815e5e0 2023-10-18T10:43:02.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.259+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3f5814e230 con 0x7f3f5815e5e0 2023-10-18T10:43:02.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.949+0000 7f3f4f7fe640 1 -- 172.21.15.125:0/2397839049 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v126) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3f5814e230 con 0x7f3f5815e5e0 2023-10-18T10:43:02.952 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:43:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f3f3806af60 msgr2=0x7f3f3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f3f3806af60 0x7f3f3806d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f48035000 tx=0x7f3f48004730 comp rx=0 tx=0).stop 2023-10-18T10:43:02.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 msgr2=0x7f3f581f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 0x7f3f581f4d70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000c780 tx=0x7f3f5000cc40 comp rx=0 tx=0).stop 2023-10-18T10:43:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 shutdown_connections 2023-10-18T10:43:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f3f3806af60 0x7f3f3806d420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f580cf6e0 0x7f3f580c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f5815e5e0 0x7f3f581f4d70 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 --2- 172.21.15.125:0/2397839049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f580b1ca0 0x7f3f5809d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 >> 172.21.15.125:0/2397839049 conn(0x7f3f580d0c50 msgr2=0x7f3f580b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 shutdown_connections 2023-10-18T10:43:02.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:02.953+0000 7f3f5d77d640 1 -- 172.21.15.125:0/2397839049 wait complete. 2023-10-18T10:43:02.978 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:43:03.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.086+0000 7f829fa21640 1 -- 172.21.15.125:0/699689537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829813d450 msgr2=0x7f82981585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:03.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.086+0000 7f829fa21640 1 --2- 172.21.15.125:0/699689537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829813d450 0x7f82981585b0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8294005fd0 tx=0x7f829402c2e0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.086+0000 7f829fa21640 1 -- 172.21.15.125:0/699689537 shutdown_connections 2023-10-18T10:43:03.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.086+0000 7f829fa21640 1 --2- 172.21.15.125:0/699689537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8298158af0 0x7f829806c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.086+0000 7f829fa21640 1 --2- 172.21.15.125:0/699689537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829813d450 0x7f82981585b0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.086+0000 7f829fa21640 1 -- 172.21.15.125:0/699689537 >> 172.21.15.125:0/699689537 conn(0x7f8298069930 msgr2=0x7f8298069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:03.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.086+0000 7f829fa21640 1 -- 172.21.15.125:0/699689537 shutdown_connections 2023-10-18T10:43:03.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.086+0000 7f829fa21640 1 -- 172.21.15.125:0/699689537 wait complete. 2023-10-18T10:43:03.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.087+0000 7f829fa21640 1 Processor -- start 2023-10-18T10:43:03.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.087+0000 7f829fa21640 1 -- start start 2023-10-18T10:43:03.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829fa21640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829806d120 0x7f8298130bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:03.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829fa21640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f829813d450 0x7f82980b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:03.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829fa21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8298158af0 0x7f829812cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:03.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829fa21640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980aee40 con 0x7f829806d120 2023-10-18T10:43:03.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829fa21640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980ae880 con 0x7f8298158af0 2023-10-18T10:43:03.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829fa21640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980aeb60 con 0x7f829813d450 2023-10-18T10:43:03.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829806d120 0x7f8298130bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:03.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829806d120 0x7f8298130bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48856/0 (socket says 172.21.15.125:48856) 2023-10-18T10:43:03.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 -- 172.21.15.125:0/1605707197 learned_addr learned my addr 172.21.15.125:0/1605707197 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:03.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829dfbe640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8298158af0 0x7f829812cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:03.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829cfbc640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f829813d450 0x7f82980b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:03.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 -- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f829813d450 msgr2=0x7f82980b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:03.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f829813d450 0x7f82980b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 -- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8298158af0 msgr2=0x7f829812cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:03.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8298158af0 0x7f829812cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 -- 172.21.15.125:0/1605707197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294009900 con 0x7f829806d120 2023-10-18T10:43:03.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829dfbe640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8298158af0 0x7f829812cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:43:03.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.088+0000 7f829d7bd640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829806d120 0x7f8298130bf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8294006010 tx=0x7f8294035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:03.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.089+0000 7f828dffb640 1 -- 172.21.15.125:0/1605707197 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294004240 con 0x7f829806d120 2023-10-18T10:43:03.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.089+0000 7f828dffb640 1 -- 172.21.15.125:0/1605707197 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8294004490 con 0x7f829806d120 2023-10-18T10:43:03.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.089+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82980af230 con 0x7f829806d120 2023-10-18T10:43:03.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.089+0000 7f828dffb640 1 -- 172.21.15.125:0/1605707197 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294004860 con 0x7f829806d120 2023-10-18T10:43:03.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.089+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8298131130 con 0x7f829806d120 2023-10-18T10:43:03.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.090+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82980af090 con 0x7f829806d120 2023-10-18T10:43:03.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.090+0000 7f828dffb640 1 -- 172.21.15.125:0/1605707197 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f829403b040 con 0x7f829806d120 2023-10-18T10:43:03.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.091+0000 7f828dffb640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f827c06af60 0x7f827c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:03.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.091+0000 7f828dffb640 1 -- 172.21.15.125:0/1605707197 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f82940ac320 con 0x7f829806d120 2023-10-18T10:43:03.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.091+0000 7f829cfbc640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f827c06af60 0x7f827c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:03.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.092+0000 7f829cfbc640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f827c06af60 0x7f827c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8288005fd0 tx=0x7f8288005850 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:03.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.095+0000 7f828dffb640 1 -- 172.21.15.125:0/1605707197 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82980af090 con 0x7f829806d120 2023-10-18T10:43:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.297+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f82980af090 con 0x7f829806d120 2023-10-18T10:43:03.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.965+0000 7f828dffb640 1 -- 172.21.15.125:0/1605707197 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7f82980af090 con 0x7f829806d120 2023-10-18T10:43:03.968 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 15 and data pool 16 2023-10-18T10:43:03.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f827c06af60 msgr2=0x7f827c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:03.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f827c06af60 0x7f827c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8288005fd0 tx=0x7f8288005850 comp rx=0 tx=0).stop 2023-10-18T10:43:03.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829806d120 msgr2=0x7f8298130bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:03.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829806d120 0x7f8298130bf0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8294006010 tx=0x7f8294035b50 comp rx=0 tx=0).stop 2023-10-18T10:43:03.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 shutdown_connections 2023-10-18T10:43:03.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f827c06af60 0x7f827c06d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8298158af0 0x7f829812cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f829813d450 0x7f82980b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 --2- 172.21.15.125:0/1605707197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f829806d120 0x7f8298130bf0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:03.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.968+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 >> 172.21.15.125:0/1605707197 conn(0x7f8298069930 msgr2=0x7f82980adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:03.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.969+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 shutdown_connections 2023-10-18T10:43:03.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:03.969+0000 7f829fa21640 1 -- 172.21.15.125:0/1605707197 wait complete. 2023-10-18T10:43:03.985 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:43:04.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.086+0000 7fde4d075640 1 -- 172.21.15.125:0/3378173315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 msgr2=0x7fde480d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.086+0000 7fde4d075640 1 --2- 172.21.15.125:0/3378173315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 0x7fde480d1910 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005fd0 tx=0x7fde3c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:43:04.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.086+0000 7fde4d075640 1 -- 172.21.15.125:0/3378173315 shutdown_connections 2023-10-18T10:43:04.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.086+0000 7fde4d075640 1 --2- 172.21.15.125:0/3378173315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde480ceb40 0x7fde480b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.086+0000 7fde4d075640 1 --2- 172.21.15.125:0/3378173315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 0x7fde480d1910 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.086+0000 7fde4d075640 1 -- 172.21.15.125:0/3378173315 >> 172.21.15.125:0/3378173315 conn(0x7fde480d0d60 msgr2=0x7fde480d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:04.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.086+0000 7fde4d075640 1 -- 172.21.15.125:0/3378173315 shutdown_connections 2023-10-18T10:43:04.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.086+0000 7fde4d075640 1 -- 172.21.15.125:0/3378173315 wait complete. 2023-10-18T10:43:04.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4d075640 1 Processor -- start 2023-10-18T10:43:04.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4d075640 1 -- start start 2023-10-18T10:43:04.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4d075640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde480ceb40 0x7fde480b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4d075640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde480cf340 0x7fde481543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4d075640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 0x7fde480b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4d075640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c91e0 con 0x7fde4815b2a0 2023-10-18T10:43:04.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4d075640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48158600 con 0x7fde480cf340 2023-10-18T10:43:04.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4d075640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde481f57a0 con 0x7fde480ceb40 2023-10-18T10:43:04.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 0x7fde480b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 0x7fde480b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48874/0 (socket says 172.21.15.125:48874) 2023-10-18T10:43:04.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.088+0000 7fde4759e640 1 -- 172.21.15.125:0/1765234175 learned_addr learned my addr 172.21.15.125:0/1765234175 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:04.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde46d9d640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde480ceb40 0x7fde480b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4659c640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde480cf340 0x7fde481543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4759e640 1 -- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde480ceb40 msgr2=0x7fde480b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4759e640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde480ceb40 0x7fde480b6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4759e640 1 -- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde480cf340 msgr2=0x7fde481543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4759e640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde480cf340 0x7fde481543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4759e640 1 -- 172.21.15.125:0/1765234175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c009560 con 0x7fde4815b2a0 2023-10-18T10:43:04.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4759e640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 0x7fde480b38e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d750 tx=0x7fde4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:04.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde2f7fe640 1 -- 172.21.15.125:0/1765234175 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40009bd0 con 0x7fde4815b2a0 2023-10-18T10:43:04.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde2f7fe640 1 -- 172.21.15.125:0/1765234175 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde40009e20 con 0x7fde4815b2a0 2023-10-18T10:43:04.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde2f7fe640 1 -- 172.21.15.125:0/1765234175 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40005280 con 0x7fde4815b2a0 2023-10-18T10:43:04.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde480b0d50 con 0x7fde4815b2a0 2023-10-18T10:43:04.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.089+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde480c8c20 con 0x7fde4815b2a0 2023-10-18T10:43:04.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.090+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde48158770 con 0x7fde4815b2a0 2023-10-18T10:43:04.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.090+0000 7fde2f7fe640 1 -- 172.21.15.125:0/1765234175 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fde400053e0 con 0x7fde4815b2a0 2023-10-18T10:43:04.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.091+0000 7fde2f7fe640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fde3406af60 0x7fde3406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.091+0000 7fde2f7fe640 1 -- 172.21.15.125:0/1765234175 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fde40089b50 con 0x7fde4815b2a0 2023-10-18T10:43:04.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.091+0000 7fde46d9d640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fde3406af60 0x7fde3406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.092+0000 7fde46d9d640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fde3406af60 0x7fde3406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde380012e0 tx=0x7fde38009040 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:04.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.095+0000 7fde2f7fe640 1 -- 172.21.15.125:0/1765234175 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde40054930 con 0x7fde4815b2a0 2023-10-18T10:43:04.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.262+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde480690e0 con 0x7fde4815b2a0 2023-10-18T10:43:04.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.263+0000 7fde2f7fe640 1 -- 172.21.15.125:0/1765234175 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fde40017020 con 0x7fde4815b2a0 2023-10-18T10:43:04.265 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:04.266 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":127,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:43:03.298971+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:43:01.020853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-18T10:43:02.278684+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:43:04.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fde3406af60 msgr2=0x7fde3406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fde3406af60 0x7fde3406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde380012e0 tx=0x7fde38009040 comp rx=0 tx=0).stop 2023-10-18T10:43:04.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 msgr2=0x7fde480b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 0x7fde480b38e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d750 tx=0x7fde4000dc10 comp rx=0 tx=0).stop 2023-10-18T10:43:04.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 shutdown_connections 2023-10-18T10:43:04.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fde3406af60 0x7fde3406d420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde480cf340 0x7fde481543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4815b2a0 0x7fde480b38e0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 --2- 172.21.15.125:0/1765234175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde480ceb40 0x7fde480b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 >> 172.21.15.125:0/1765234175 conn(0x7fde480d0d60 msgr2=0x7fde480ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:04.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 shutdown_connections 2023-10-18T10:43:04.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.265+0000 7fde4d075640 1 -- 172.21.15.125:0/1765234175 wait complete. 2023-10-18T10:43:04.278 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:43:04.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.378+0000 7f2d4b183640 1 -- 172.21.15.125:0/3027379506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d440abc10 msgr2=0x7f2d440c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.378+0000 7f2d4b183640 1 --2- 172.21.15.125:0/3027379506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d440abc10 0x7f2d440c9b60 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c005fd0 tx=0x7f2d3c02c050 comp rx=0 tx=0).stop 2023-10-18T10:43:04.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.378+0000 7f2d4b183640 1 -- 172.21.15.125:0/3027379506 shutdown_connections 2023-10-18T10:43:04.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.378+0000 7f2d4b183640 1 --2- 172.21.15.125:0/3027379506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d440abc10 0x7f2d440c9b60 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.378+0000 7f2d4b183640 1 --2- 172.21.15.125:0/3027379506 >> v2:172.21.15.139:3300/0 conn(0x7f2d4406af00 0x7f2d4406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.378+0000 7f2d4b183640 1 -- 172.21.15.125:0/3027379506 >> 172.21.15.125:0/3027379506 conn(0x7f2d44069930 msgr2=0x7f2d44069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:04.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.378+0000 7f2d4b183640 1 -- 172.21.15.125:0/3027379506 shutdown_connections 2023-10-18T10:43:04.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.378+0000 7f2d4b183640 1 -- 172.21.15.125:0/3027379506 wait complete. 2023-10-18T10:43:04.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.379+0000 7f2d4b183640 1 Processor -- start 2023-10-18T10:43:04.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.379+0000 7f2d4b183640 1 -- start start 2023-10-18T10:43:04.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d4b183640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d4406af00 0x7f2d440708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d4b183640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d440abc10 0x7f2d44139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d4b183640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d440c96b0 0x7f2d4413da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d4b183640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4413d600 con 0x7f2d4406af00 2023-10-18T10:43:04.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d4b183640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d44139980 con 0x7f2d440c96b0 2023-10-18T10:43:04.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d4b183640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4413df60 con 0x7f2d440abc10 2023-10-18T10:43:04.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d48f1f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d4406af00 0x7f2d440708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d48f1f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d4406af00 0x7f2d440708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48902/0 (socket says 172.21.15.125:48902) 2023-10-18T10:43:04.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d48f1f640 1 -- 172.21.15.125:0/2468313341 learned_addr learned my addr 172.21.15.125:0/2468313341 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:04.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d48f1f640 1 -- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d440abc10 msgr2=0x7f2d44139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d48f1f640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d440abc10 0x7f2d44139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d48f1f640 1 -- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d440c96b0 msgr2=0x7f2d4413da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d48f1f640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d440c96b0 0x7f2d4413da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.380+0000 7f2d48f1f640 1 -- 172.21.15.125:0/2468313341 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c009840 con 0x7f2d4406af00 2023-10-18T10:43:04.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.381+0000 7f2d48f1f640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d4406af00 0x7f2d440708a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c890 tx=0x7f2d3800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:04.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.381+0000 7f2d417fa640 1 -- 172.21.15.125:0/2468313341 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38004220 con 0x7f2d4406af00 2023-10-18T10:43:04.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.381+0000 7f2d417fa640 1 -- 172.21.15.125:0/2468313341 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2d38004470 con 0x7f2d4406af00 2023-10-18T10:43:04.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.381+0000 7f2d417fa640 1 -- 172.21.15.125:0/2468313341 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38007c10 con 0x7f2d4406af00 2023-10-18T10:43:04.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.381+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d44070e80 con 0x7f2d4406af00 2023-10-18T10:43:04.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.381+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4413d220 con 0x7f2d4406af00 2023-10-18T10:43:04.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.382+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4413d870 con 0x7f2d4406af00 2023-10-18T10:43:04.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.382+0000 7f2d417fa640 1 -- 172.21.15.125:0/2468313341 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2d38005020 con 0x7f2d4406af00 2023-10-18T10:43:04.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.382+0000 7f2d417fa640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2d2806af90 0x7f2d2806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.383+0000 7f2d417fa640 1 -- 172.21.15.125:0/2468313341 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2d38089ec0 con 0x7f2d4406af00 2023-10-18T10:43:04.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.383+0000 7f2d43fff640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2d2806af90 0x7f2d2806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.383+0000 7f2d43fff640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2d2806af90 0x7f2d2806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d440add30 tx=0x7f2d30007930 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:04.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.387+0000 7f2d417fa640 1 -- 172.21.15.125:0/2468313341 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d38054ca0 con 0x7f2d4406af00 2023-10-18T10:43:04.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.557+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d440aba30 con 0x7f2d4406af00 2023-10-18T10:43:04.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.558+0000 7f2d417fa640 1 -- 172.21.15.125:0/2468313341 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16764 (secure 0 0 0) 0x7f2d3801d070 con 0x7f2d4406af00 2023-10-18T10:43:04.560 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:04.561 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":128,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:43:04.402713+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:43:01.020853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-18T10:43:02.278684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:43:04.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2d2806af90 msgr2=0x7f2d2806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2d2806af90 0x7f2d2806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d440add30 tx=0x7f2d30007930 comp rx=0 tx=0).stop 2023-10-18T10:43:04.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d4406af00 msgr2=0x7f2d440708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d4406af00 0x7f2d440708a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c890 tx=0x7f2d3800cd50 comp rx=0 tx=0).stop 2023-10-18T10:43:04.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 shutdown_connections 2023-10-18T10:43:04.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2d2806af90 0x7f2d2806d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d440c96b0 0x7f2d4413da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d440abc10 0x7f2d44139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 --2- 172.21.15.125:0/2468313341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d4406af00 0x7f2d440708a0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 >> 172.21.15.125:0/2468313341 conn(0x7f2d44069930 msgr2=0x7f2d44069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:04.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 shutdown_connections 2023-10-18T10:43:04.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.560+0000 7f2d4b183640 1 -- 172.21.15.125:0/2468313341 wait complete. 2023-10-18T10:43:04.574 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:43:04.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.679+0000 7fa42fce0640 1 -- 172.21.15.125:0/1185468157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 msgr2=0x7fa414003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.679+0000 7fa42fce0640 1 --2- 172.21.15.125:0/1185468157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 0x7fa414003d90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa424009b50 tx=0x7fa4140057f0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.679+0000 7fa42fce0640 1 -- 172.21.15.125:0/1185468157 shutdown_connections 2023-10-18T10:43:04.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.679+0000 7fa42fce0640 1 --2- 172.21.15.125:0/1185468157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 0x7fa414003d90 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.679+0000 7fa42fce0640 1 -- 172.21.15.125:0/1185468157 >> 172.21.15.125:0/1185468157 conn(0x7fa4280d0d60 msgr2=0x7fa4280d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:04.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.679+0000 7fa42fce0640 1 -- 172.21.15.125:0/1185468157 shutdown_connections 2023-10-18T10:43:04.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.679+0000 7fa42fce0640 1 -- 172.21.15.125:0/1185468157 wait complete. 2023-10-18T10:43:04.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.680+0000 7fa42fce0640 1 Processor -- start 2023-10-18T10:43:04.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.680+0000 7fa42fce0640 1 -- start start 2023-10-18T10:43:04.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42fce0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 0x7fa4280b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42fce0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa428156c10 0x7fa428157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42fce0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4280b4360 0x7fa4280b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42fce0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42814bc80 con 0x7fa414003970 2023-10-18T10:43:04.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42fce0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280b4d20 con 0x7fa4280b4360 2023-10-18T10:43:04.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42fce0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280cac00 con 0x7fa428156c10 2023-10-18T10:43:04.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 0x7fa4280b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 0x7fa4280b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48926/0 (socket says 172.21.15.125:48926) 2023-10-18T10:43:04.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 -- 172.21.15.125:0/3076699787 learned_addr learned my addr 172.21.15.125:0/3076699787 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:04.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42e27d640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4280b4360 0x7fa4280b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42d27b640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa428156c10 0x7fa428157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 -- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa428156c10 msgr2=0x7fa428157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa428156c10 0x7fa428157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 -- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4280b4360 msgr2=0x7fa4280b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4280b4360 0x7fa4280b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 -- 172.21.15.125:0/3076699787 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280c7fb0 con 0x7fa414003970 2023-10-18T10:43:04.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42da7c640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 0x7fa4280b5c00 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa424009b50 tx=0x7fa424033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:04.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa41e7fc640 1 -- 172.21.15.125:0/3076699787 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424004460 con 0x7fa414003970 2023-10-18T10:43:04.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa41e7fc640 1 -- 172.21.15.125:0/3076699787 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa424033e40 con 0x7fa414003970 2023-10-18T10:43:04.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4280c8290 con 0x7fa414003970 2023-10-18T10:43:04.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa41e7fc640 1 -- 172.21.15.125:0/3076699787 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424003850 con 0x7fa414003970 2023-10-18T10:43:04.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.681+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4280c8850 con 0x7fa414003970 2023-10-18T10:43:04.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.682+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4280690e0 con 0x7fa414003970 2023-10-18T10:43:04.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.683+0000 7fa41e7fc640 1 -- 172.21.15.125:0/3076699787 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa4240039b0 con 0x7fa414003970 2023-10-18T10:43:04.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.683+0000 7fa41e7fc640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa40c06af90 0x7fa40c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:04.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.683+0000 7fa41e7fc640 1 -- 172.21.15.125:0/3076699787 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa4240ac820 con 0x7fa414003970 2023-10-18T10:43:04.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.683+0000 7fa42d27b640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa40c06af90 0x7fa40c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:04.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.684+0000 7fa42d27b640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa40c06af90 0x7fa40c06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4280c7780 tx=0x7fa418006ae0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:04.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.687+0000 7fa41e7fc640 1 -- 172.21.15.125:0/3076699787 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa424077570 con 0x7fa414003970 2023-10-18T10:43:04.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.892+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa4280d0b40 con 0x7fa414003970 2023-10-18T10:43:04.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.974+0000 7fa41e7fc640 1 -- 172.21.15.125:0/3076699787 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa42404a070 con 0x7fa414003970 2023-10-18T10:43:04.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa40c06af90 msgr2=0x7fa40c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa40c06af90 0x7fa40c06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4280c7780 tx=0x7fa418006ae0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 msgr2=0x7fa4280b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:04.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 0x7fa4280b5c00 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa424009b50 tx=0x7fa424033c10 comp rx=0 tx=0).stop 2023-10-18T10:43:04.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 shutdown_connections 2023-10-18T10:43:04.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa40c06af90 0x7fa40c06d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4280b4360 0x7fa4280b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa428156c10 0x7fa428157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 --2- 172.21.15.125:0/3076699787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa414003970 0x7fa4280b5c00 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:04.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.976+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 >> 172.21.15.125:0/3076699787 conn(0x7fa4280d0d60 msgr2=0x7fa42806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:04.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.977+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 shutdown_connections 2023-10-18T10:43:04.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:04.977+0000 7fa42fce0640 1 -- 172.21.15.125:0/3076699787 wait complete. 2023-10-18T10:43:04.989 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:43:04.989 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:43:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.092+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/2468879748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080afa20 msgr2=0x7f6c08158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.092+0000 7f6c0f0a9640 1 --2- 172.21.15.125:0/2468879748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080afa20 0x7f6c08158300 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009830 tx=0x7f6bf402bd60 comp rx=0 tx=0).stop 2023-10-18T10:43:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.093+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/2468879748 shutdown_connections 2023-10-18T10:43:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.093+0000 7f6c0f0a9640 1 --2- 172.21.15.125:0/2468879748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080afa20 0x7f6c08158300 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:05.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.093+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/2468879748 >> 172.21.15.125:0/2468879748 conn(0x7f6c08069930 msgr2=0x7f6c08069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:05.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.093+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/2468879748 shutdown_connections 2023-10-18T10:43:05.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.093+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/2468879748 wait complete. 2023-10-18T10:43:05.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.094+0000 7f6c0f0a9640 1 Processor -- start 2023-10-18T10:43:05.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.094+0000 7f6c0f0a9640 1 -- start start 2023-10-18T10:43:05.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0f0a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c080afa20 0x7f6c08146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:05.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0f0a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080b0530 0x7f6c08150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:05.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0f0a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c080cbd20 0x7f6c0814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:05.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0f0a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c081efe30 con 0x7f6c080b0530 2023-10-18T10:43:05.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0f0a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c081f0110 con 0x7f6c080afa20 2023-10-18T10:43:05.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0f0a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c081f03f0 con 0x7f6c080cbd20 2023-10-18T10:43:05.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080b0530 0x7f6c08150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:05.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080b0530 0x7f6c08150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48954/0 (socket says 172.21.15.125:48954) 2023-10-18T10:43:05.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 -- 172.21.15.125:0/293504960 learned_addr learned my addr 172.21.15.125:0/293504960 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:05.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0ce45640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c080afa20 0x7f6c08146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:05.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0d646640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c080cbd20 0x7f6c0814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:05.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 -- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c080cbd20 msgr2=0x7f6c0814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:05.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c080cbd20 0x7f6c0814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:05.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 -- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c080afa20 msgr2=0x7f6c08146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:05.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c080afa20 0x7f6c08146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:05.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 -- 172.21.15.125:0/293504960 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf4009490 con 0x7f6c080b0530 2023-10-18T10:43:05.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0ce45640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c080afa20 0x7f6c08146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:43:05.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0d646640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c080cbd20 0x7f6c0814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:43:05.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c07fff640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080b0530 0x7f6c08150ae0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009950 tx=0x7f6bf40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:05.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c057fa640 1 -- 172.21.15.125:0/293504960 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4003ea0 con 0x7f6c080b0530 2023-10-18T10:43:05.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c057fa640 1 -- 172.21.15.125:0/293504960 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6bf4005020 con 0x7f6c080b0530 2023-10-18T10:43:05.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c081f06d0 con 0x7f6c080b0530 2023-10-18T10:43:05.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c057fa640 1 -- 172.21.15.125:0/293504960 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4005410 con 0x7f6c080b0530 2023-10-18T10:43:05.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.095+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c08151020 con 0x7f6c080b0530 2023-10-18T10:43:05.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.096+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c081f0860 con 0x7f6c080b0530 2023-10-18T10:43:05.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.097+0000 7f6c057fa640 1 -- 172.21.15.125:0/293504960 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6bf40047a0 con 0x7f6c080b0530 2023-10-18T10:43:05.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.097+0000 7f6c057fa640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6bec06af60 0x7f6bec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:05.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.097+0000 7f6c057fa640 1 -- 172.21.15.125:0/293504960 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6bf40ac610 con 0x7f6c080b0530 2023-10-18T10:43:05.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.097+0000 7f6c0ce45640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6bec06af60 0x7f6bec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:05.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.098+0000 7f6c0ce45640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6bec06af60 0x7f6bec06d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc000f60 tx=0x7f6bfc009040 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:05.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.101+0000 7f6c057fa640 1 -- 172.21.15.125:0/293504960 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bf4077470 con 0x7f6c080b0530 2023-10-18T10:43:05.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.300+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6c0806c7d0 con 0x7f6c080b0530 2023-10-18T10:43:05.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.979+0000 7f6c057fa640 1 -- 172.21.15.125:0/293504960 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6bf407aa90 con 0x7f6c080b0530 2023-10-18T10:43:05.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6bec06af60 msgr2=0x7f6bec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:05.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6bec06af60 0x7f6bec06d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc000f60 tx=0x7f6bfc009040 comp rx=0 tx=0).stop 2023-10-18T10:43:05.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080b0530 msgr2=0x7f6c08150ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:05.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080b0530 0x7f6c08150ae0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009950 tx=0x7f6bf40038a0 comp rx=0 tx=0).stop 2023-10-18T10:43:05.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 shutdown_connections 2023-10-18T10:43:05.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f6bec06af60 0x7f6bec06d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:05.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c080cbd20 0x7f6c0814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:05.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c080b0530 0x7f6c08150ae0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:05.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 --2- 172.21.15.125:0/293504960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c080afa20 0x7f6c08146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:05.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 >> 172.21.15.125:0/293504960 conn(0x7f6c08069930 msgr2=0x7f6c08069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:05.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 shutdown_connections 2023-10-18T10:43:05.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:05.981+0000 7f6c0f0a9640 1 -- 172.21.15.125:0/293504960 wait complete. 2023-10-18T10:43:05.995 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:43:05.995 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:43:06.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.097+0000 7f4a43e44640 1 -- 172.21.15.125:0/718916995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 msgr2=0x7f4a28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.097+0000 7f4a43e44640 1 --2- 172.21.15.125:0/718916995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 0x7f4a28005180 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400c640 tx=0x7f4a280057f0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.097+0000 7f4a423e1640 1 -- 172.21.15.125:0/718916995 reap_dead start 2023-10-18T10:43:06.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.097+0000 7f4a43e44640 1 -- 172.21.15.125:0/718916995 shutdown_connections 2023-10-18T10:43:06.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.097+0000 7f4a43e44640 1 -- 172.21.15.125:0/718916995 >> 172.21.15.125:0/718916995 conn(0x7f4a3c069930 msgr2=0x7f4a3c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:06.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.097+0000 7f4a43e44640 1 -- 172.21.15.125:0/718916995 shutdown_connections 2023-10-18T10:43:06.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.097+0000 7f4a43e44640 1 -- 172.21.15.125:0/718916995 wait complete. 2023-10-18T10:43:06.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.098+0000 7f4a43e44640 1 Processor -- start 2023-10-18T10:43:06.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.098+0000 7f4a43e44640 1 -- start start 2023-10-18T10:43:06.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a43e44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 0x7f4a3c152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a43e44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a3c0b0120 0x7f4a3c0a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a43e44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a3c0b05a0 0x7f4a3c0a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a43e44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0b0b60 con 0x7f4a28004d00 2023-10-18T10:43:06.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a43e44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0b0e40 con 0x7f4a3c0b0120 2023-10-18T10:43:06.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a43e44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0b1120 con 0x7f4a3c0b05a0 2023-10-18T10:43:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a41be0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 0x7f4a3c152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a41be0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 0x7f4a3c152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48972/0 (socket says 172.21.15.125:48972) 2023-10-18T10:43:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a41be0640 1 -- 172.21.15.125:0/3947101133 learned_addr learned my addr 172.21.15.125:0/3947101133 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a41be0640 1 -- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a3c0b05a0 msgr2=0x7f4a3c0a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a41be0640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a3c0b05a0 0x7f4a3c0a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a41be0640 1 -- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a3c0b0120 msgr2=0x7f4a3c0a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a41be0640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a3c0b0120 0x7f4a3c0a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.099+0000 7f4a41be0640 1 -- 172.21.15.125:0/3947101133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0b1400 con 0x7f4a28004d00 2023-10-18T10:43:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.100+0000 7f4a41be0640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 0x7f4a3c152550 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f4a38011a60 tx=0x7f4a38011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.100+0000 7f4a327fc640 1 -- 172.21.15.125:0/3947101133 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3800fc40 con 0x7f4a28004d00 2023-10-18T10:43:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.100+0000 7f4a327fc640 1 -- 172.21.15.125:0/3947101133 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a3800fda0 con 0x7f4a28004d00 2023-10-18T10:43:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.100+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c0b16e0 con 0x7f4a28004d00 2023-10-18T10:43:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.100+0000 7f4a327fc640 1 -- 172.21.15.125:0/3947101133 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38004280 con 0x7f4a28004d00 2023-10-18T10:43:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.100+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c0b1c00 con 0x7f4a28004d00 2023-10-18T10:43:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.101+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c0690e0 con 0x7f4a28004d00 2023-10-18T10:43:06.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.101+0000 7f4a327fc640 1 -- 172.21.15.125:0/3947101133 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4a380035f0 con 0x7f4a28004d00 2023-10-18T10:43:06.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.101+0000 7f4a327fc640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f4a2006af60 0x7f4a2006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.102+0000 7f4a327fc640 1 -- 172.21.15.125:0/3947101133 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4a3808a410 con 0x7f4a28004d00 2023-10-18T10:43:06.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.102+0000 7f4a413df640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f4a2006af60 0x7f4a2006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:06.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.102+0000 7f4a413df640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f4a2006af60 0x7f4a2006d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0ca110 tx=0x7f4a2c009530 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:06.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.106+0000 7f4a327fc640 1 -- 172.21.15.125:0/3947101133 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a380551f0 con 0x7f4a28004d00 2023-10-18T10:43:06.113 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:43:06.116 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:43:06.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.312+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a3c0cc420 con 0x7f4a28004d00 2023-10-18T10:43:06.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.313+0000 7f4a327fc640 1 -- 172.21.15.125:0/3947101133 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f4a38058810 con 0x7f4a28004d00 2023-10-18T10:43:06.316 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:06.316 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":47,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/4122761343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4122761343},{"type":"v1","addr":"172.21.15.139:6836","nonce":4122761343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6785,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1014994333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1014994333},{"type":"v1","addr":"172.21.15.139:6837","nonce":1014994333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3090232356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3090232356},{"type":"v1","addr":"172.21.15.125:6834","nonce":3090232356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:43:03.299305+0000","modified":"2023-10-18T10:43:05.301917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8613":{"gid":8613,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2199142702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2199142702},{"type":"v1","addr":"172.21.15.125:6835","nonce":2199142702}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T10:43:06.317 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 47 2023-10-18T10:43:06.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f4a2006af60 msgr2=0x7f4a2006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f4a2006af60 0x7f4a2006d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0ca110 tx=0x7f4a2c009530 comp rx=0 tx=0).stop 2023-10-18T10:43:06.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 msgr2=0x7f4a3c152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 0x7f4a3c152550 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f4a38011a60 tx=0x7f4a38011f20 comp rx=0 tx=0).stop 2023-10-18T10:43:06.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 shutdown_connections 2023-10-18T10:43:06.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f4a2006af60 0x7f4a2006d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a3c0b0120 0x7f4a3c0a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a3c0b05a0 0x7f4a3c0a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 --2- 172.21.15.125:0/3947101133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a28004d00 0x7f4a3c152550 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 >> 172.21.15.125:0/3947101133 conn(0x7f4a3c069930 msgr2=0x7f4a3c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:06.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 shutdown_connections 2023-10-18T10:43:06.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.315+0000 7f4a43e44640 1 -- 172.21.15.125:0/3947101133 wait complete. 2023-10-18T10:43:06.327 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:43:06.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.427+0000 7f392c8f2640 1 -- 172.21.15.125:0/1343429795 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280cf340 msgr2=0x7f39280d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.427+0000 7f392c8f2640 1 --2- 172.21.15.125:0/1343429795 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280cf340 0x7f39280d1910 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3918005fd0 tx=0x7f391802c050 comp rx=0 tx=0).stop 2023-10-18T10:43:06.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.427+0000 7f392c8f2640 1 -- 172.21.15.125:0/1343429795 shutdown_connections 2023-10-18T10:43:06.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.427+0000 7f392c8f2640 1 --2- 172.21.15.125:0/1343429795 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280cf340 0x7f39280d1910 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.427+0000 7f392c8f2640 1 -- 172.21.15.125:0/1343429795 >> 172.21.15.125:0/1343429795 conn(0x7f39280d0d60 msgr2=0x7f39280d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:06.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.427+0000 7f392c8f2640 1 -- 172.21.15.125:0/1343429795 shutdown_connections 2023-10-18T10:43:06.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.427+0000 7f392c8f2640 1 -- 172.21.15.125:0/1343429795 wait complete. 2023-10-18T10:43:06.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.428+0000 7f392c8f2640 1 Processor -- start 2023-10-18T10:43:06.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.428+0000 7f392c8f2640 1 -- start start 2023-10-18T10:43:06.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392c8f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39280ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392c8f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39280cf340 0x7f39281501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392c8f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f392815b2a0 0x7f39280b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392c8f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39280cac70 con 0x7f39280ceb40 2023-10-18T10:43:06.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392c8f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3928150730 con 0x7f392815b2a0 2023-10-18T10:43:06.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392c8f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39280c8bb0 con 0x7f39280cf340 2023-10-18T10:43:06.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39280ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:06.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39280ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48988/0 (socket says 172.21.15.125:48988) 2023-10-18T10:43:06.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.429+0000 7f392659c640 1 -- 172.21.15.125:0/4027628432 learned_addr learned my addr 172.21.15.125:0/4027628432 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:06.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f392659c640 1 -- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39280cf340 msgr2=0x7f39281501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f392659c640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39280cf340 0x7f39281501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f392659c640 1 -- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f392815b2a0 msgr2=0x7f39280b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f392659c640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f392815b2a0 0x7f39280b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f392659c640 1 -- 172.21.15.125:0/4027628432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3918009840 con 0x7f39280ceb40 2023-10-18T10:43:06.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f392659c640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39280ca730 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3918009cd0 tx=0x7f3918004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:06.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f3916ffd640 1 -- 172.21.15.125:0/4027628432 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391803e460 con 0x7f39280ceb40 2023-10-18T10:43:06.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f3916ffd640 1 -- 172.21.15.125:0/4027628432 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f391803e6b0 con 0x7f39280ceb40 2023-10-18T10:43:06.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f3916ffd640 1 -- 172.21.15.125:0/4027628432 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391803ea80 con 0x7f39280ceb40 2023-10-18T10:43:06.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39280c8e90 con 0x7f39280ceb40 2023-10-18T10:43:06.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.430+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39280c9450 con 0x7f39280ceb40 2023-10-18T10:43:06.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.431+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39280690e0 con 0x7f39280ceb40 2023-10-18T10:43:06.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.431+0000 7f3916ffd640 1 -- 172.21.15.125:0/4027628432 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f391803ebe0 con 0x7f39280ceb40 2023-10-18T10:43:06.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.432+0000 7f3916ffd640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f390806af60 0x7f390806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.432+0000 7f3916ffd640 1 -- 172.21.15.125:0/4027628432 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f39180acad0 con 0x7f39280ceb40 2023-10-18T10:43:06.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.432+0000 7f3925d9b640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f390806af60 0x7f390806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:06.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.432+0000 7f3925d9b640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f390806af60 0x7f390806d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f391c000f60 tx=0x7f391c007a30 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:06.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.436+0000 7f3916ffd640 1 -- 172.21.15.125:0/4027628432 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3918077940 con 0x7f39280ceb40 2023-10-18T10:43:06.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.602+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39280d0b40 con 0x7f39280ceb40 2023-10-18T10:43:06.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.603+0000 7f3916ffd640 1 -- 172.21.15.125:0/4027628432 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16764 (secure 0 0 0) 0x7f391807af60 con 0x7f39280ceb40 2023-10-18T10:43:06.605 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:06.605 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":128,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:43:04.402713+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:43:01.020853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-18T10:43:02.278684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:43:06.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f390806af60 msgr2=0x7f390806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f390806af60 0x7f390806d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f391c000f60 tx=0x7f391c007a30 comp rx=0 tx=0).stop 2023-10-18T10:43:06.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 msgr2=0x7f39280ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39280ca730 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3918009cd0 tx=0x7f3918004360 comp rx=0 tx=0).stop 2023-10-18T10:43:06.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 shutdown_connections 2023-10-18T10:43:06.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f390806af60 0x7f390806d420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f392815b2a0 0x7f39280b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39280cf340 0x7f39281501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 --2- 172.21.15.125:0/4027628432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39280ca730 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 >> 172.21.15.125:0/4027628432 conn(0x7f39280d0d60 msgr2=0x7f392806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:06.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 shutdown_connections 2023-10-18T10:43:06.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.605+0000 7f392c8f2640 1 -- 172.21.15.125:0/4027628432 wait complete. 2023-10-18T10:43:06.618 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:43:06.618 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:43:06.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.718+0000 7fcd23032640 1 -- 172.21.15.125:0/4090867503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 msgr2=0x7fcd0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.718+0000 7fcd23032640 1 --2- 172.21.15.125:0/4090867503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 0x7fcd0c004c40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800bb20 tx=0x7fcd0c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.718+0000 7fcd215cf640 1 -- 172.21.15.125:0/4090867503 reap_dead start 2023-10-18T10:43:06.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.718+0000 7fcd23032640 1 -- 172.21.15.125:0/4090867503 shutdown_connections 2023-10-18T10:43:06.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.718+0000 7fcd23032640 1 -- 172.21.15.125:0/4090867503 >> 172.21.15.125:0/4090867503 conn(0x7fcd1c069930 msgr2=0x7fcd1c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:06.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.718+0000 7fcd23032640 1 -- 172.21.15.125:0/4090867503 shutdown_connections 2023-10-18T10:43:06.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.718+0000 7fcd23032640 1 -- 172.21.15.125:0/4090867503 wait complete. 2023-10-18T10:43:06.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.719+0000 7fcd23032640 1 Processor -- start 2023-10-18T10:43:06.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.719+0000 7fcd23032640 1 -- start start 2023-10-18T10:43:06.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd23032640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 0x7fcd1c15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd23032640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15b850 0x7fcd1c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd23032640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c0b0600 0x7fcd1c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd23032640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c15bcd0 con 0x7fcd0c0047c0 2023-10-18T10:43:06.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd23032640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1441a0 con 0x7fcd1c0b0600 2023-10-18T10:43:06.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd23032640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c143590 con 0x7fcd1c15b850 2023-10-18T10:43:06.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 0x7fcd1c15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:06.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 0x7fcd1c15b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49006/0 (socket says 172.21.15.125:49006) 2023-10-18T10:43:06.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 -- 172.21.15.125:0/847590517 learned_addr learned my addr 172.21.15.125:0/847590517 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:06.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 -- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15b850 msgr2=0x7fcd1c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15b850 0x7fcd1c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 -- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c0b0600 msgr2=0x7fcd1c0b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c0b0600 0x7fcd1c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 -- 172.21.15.125:0/847590517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c143870 con 0x7fcd0c0047c0 2023-10-18T10:43:06.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.720+0000 7fcd20dce640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 0x7fcd1c15b310 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ea60 tx=0x7fcd1000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:06.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.721+0000 7fcd197fa640 1 -- 172.21.15.125:0/847590517 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000ad40 con 0x7fcd0c0047c0 2023-10-18T10:43:06.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.721+0000 7fcd197fa640 1 -- 172.21.15.125:0/847590517 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd10004030 con 0x7fcd0c0047c0 2023-10-18T10:43:06.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.721+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c143b50 con 0x7fcd0c0047c0 2023-10-18T10:43:06.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.721+0000 7fcd197fa640 1 -- 172.21.15.125:0/847590517 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10004420 con 0x7fcd0c0047c0 2023-10-18T10:43:06.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.721+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c134d50 con 0x7fcd0c0047c0 2023-10-18T10:43:06.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.721+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c06c850 con 0x7fcd0c0047c0 2023-10-18T10:43:06.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.722+0000 7fcd197fa640 1 -- 172.21.15.125:0/847590517 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcd10005020 con 0x7fcd0c0047c0 2023-10-18T10:43:06.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.722+0000 7fcd197fa640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd0006af90 0x7fcd0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:06.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.722+0000 7fcd197fa640 1 -- 172.21.15.125:0/847590517 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcd1008a040 con 0x7fcd0c0047c0 2023-10-18T10:43:06.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.723+0000 7fcd1bfff640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd0006af90 0x7fcd0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:06.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.723+0000 7fcd1bfff640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd0006af90 0x7fcd0006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd08037000 tx=0x7fcd0802e040 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:06.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.727+0000 7fcd197fa640 1 -- 172.21.15.125:0/847590517 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd100588d0 con 0x7fcd0c0047c0 2023-10-18T10:43:06.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.889+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 --> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd1c15c770 con 0x7fcd0006af90 2023-10-18T10:43:06.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.897+0000 7fcd197fa640 1 -- 172.21.15.125:0/847590517 <== mgr.7360 v2:172.21.15.139:6832/1511464628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287849 (secure 0 0 0) 0x7fcd0803f440 con 0x7fcd0006af90 2023-10-18T10:43:06.902 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:06.904 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:43:06.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.900+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd0006af90 msgr2=0x7fcd0006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.900+0000 7fcd23032640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd0006af90 0x7fcd0006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd08037000 tx=0x7fcd0802e040 comp rx=0 tx=0).stop 2023-10-18T10:43:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.900+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 msgr2=0x7fcd1c15b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.900+0000 7fcd23032640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 0x7fcd1c15b310 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ea60 tx=0x7fcd1000ef20 comp rx=0 tx=0).stop 2023-10-18T10:43:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.901+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 shutdown_connections 2023-10-18T10:43:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.901+0000 7fcd23032640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fcd0006af90 0x7fcd0006d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.901+0000 7fcd23032640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c0b0600 0x7fcd1c0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.901+0000 7fcd23032640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15b850 0x7fcd1c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.901+0000 7fcd23032640 1 --2- 172.21.15.125:0/847590517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0047c0 0x7fcd1c15b310 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.901+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 >> 172.21.15.125:0/847590517 conn(0x7fcd1c069930 msgr2=0x7fcd1c0ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.901+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 shutdown_connections 2023-10-18T10:43:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:06.901+0000 7fcd23032640 1 -- 172.21.15.125:0/847590517 wait complete. 2023-10-18T10:43:06.913 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:43:05.899805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":364,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":315772,"kb_used_data":13540,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754658948,"statfs":{"total":773094113280,"available":772770762752,"internally_reserved":0,"allocated":13864960,"data_stored":7534012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001056"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564947+0000","last_change":"2023-10-18T10:43:03.420057+0000","last_active":"2023-10-18T10:43:04.564947+0000","last_peered":"2023-10-18T10:43:04.564947+0000","last_clean":"2023-10-18T10:43:04.564947+0000","last_became_active":"2023-10-18T10:43:03.419809+0000","last_became_peered":"2023-10-18T10:43:03.419809+0000","last_unstale":"2023-10-18T10:43:04.564947+0000","last_undegraded":"2023-10-18T10:43:04.564947+0000","last_fullsized":"2023-10-18T10:43:04.564947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:51.310959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565191+0000","last_change":"2023-10-18T10:43:03.422800+0000","last_active":"2023-10-18T10:43:04.565191+0000","last_peered":"2023-10-18T10:43:04.565191+0000","last_clean":"2023-10-18T10:43:04.565191+0000","last_became_active":"2023-10-18T10:43:03.422037+0000","last_became_peered":"2023-10-18T10:43:03.422037+0000","last_unstale":"2023-10-18T10:43:04.565191+0000","last_undegraded":"2023-10-18T10:43:04.565191+0000","last_fullsized":"2023-10-18T10:43:04.565191+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:03.106690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564044+0000","last_change":"2023-10-18T10:43:03.424158+0000","last_active":"2023-10-18T10:43:04.564044+0000","last_peered":"2023-10-18T10:43:04.564044+0000","last_clean":"2023-10-18T10:43:04.564044+0000","last_became_active":"2023-10-18T10:43:03.423915+0000","last_became_peered":"2023-10-18T10:43:03.423915+0000","last_unstale":"2023-10-18T10:43:04.564044+0000","last_undegraded":"2023-10-18T10:43:04.564044+0000","last_fullsized":"2023-10-18T10:43:04.564044+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:49.732624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408937+0000","last_change":"2023-10-18T10:43:03.415009+0000","last_active":"2023-10-18T10:43:04.408937+0000","last_peered":"2023-10-18T10:43:04.408937+0000","last_clean":"2023-10-18T10:43:04.408937+0000","last_became_active":"2023-10-18T10:43:03.414710+0000","last_became_peered":"2023-10-18T10:43:03.414710+0000","last_unstale":"2023-10-18T10:43:04.408937+0000","last_undegraded":"2023-10-18T10:43:04.408937+0000","last_fullsized":"2023-10-18T10:43:04.408937+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:21.227528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563635+0000","last_change":"2023-10-18T10:43:03.411644+0000","last_active":"2023-10-18T10:43:04.563635+0000","last_peered":"2023-10-18T10:43:04.563635+0000","last_clean":"2023-10-18T10:43:04.563635+0000","last_became_active":"2023-10-18T10:43:03.411381+0000","last_became_peered":"2023-10-18T10:43:03.411381+0000","last_unstale":"2023-10-18T10:43:04.563635+0000","last_undegraded":"2023-10-18T10:43:04.563635+0000","last_fullsized":"2023-10-18T10:43:04.563635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:28.704370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564515+0000","last_change":"2023-10-18T10:43:03.420380+0000","last_active":"2023-10-18T10:43:04.564515+0000","last_peered":"2023-10-18T10:43:04.564515+0000","last_clean":"2023-10-18T10:43:04.564515+0000","last_became_active":"2023-10-18T10:43:03.420112+0000","last_became_peered":"2023-10-18T10:43:03.420112+0000","last_unstale":"2023-10-18T10:43:04.564515+0000","last_undegraded":"2023-10-18T10:43:04.564515+0000","last_fullsized":"2023-10-18T10:43:04.564515+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:15.468100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564645+0000","last_change":"2023-10-18T10:43:03.417853+0000","last_active":"2023-10-18T10:43:04.564645+0000","last_peered":"2023-10-18T10:43:04.564645+0000","last_clean":"2023-10-18T10:43:04.564645+0000","last_became_active":"2023-10-18T10:43:03.417585+0000","last_became_peered":"2023-10-18T10:43:03.417585+0000","last_unstale":"2023-10-18T10:43:04.564645+0000","last_undegraded":"2023-10-18T10:43:04.564645+0000","last_fullsized":"2023-10-18T10:43:04.564645+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:57.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563425+0000","last_change":"2023-10-18T10:43:03.420896+0000","last_active":"2023-10-18T10:43:04.563425+0000","last_peered":"2023-10-18T10:43:04.563425+0000","last_clean":"2023-10-18T10:43:04.563425+0000","last_became_active":"2023-10-18T10:43:03.420633+0000","last_became_peered":"2023-10-18T10:43:03.420633+0000","last_unstale":"2023-10-18T10:43:04.563425+0000","last_undegraded":"2023-10-18T10:43:04.563425+0000","last_fullsized":"2023-10-18T10:43:04.563425+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:54.658345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564188+0000","last_change":"2023-10-18T10:43:03.423222+0000","last_active":"2023-10-18T10:43:04.564188+0000","last_peered":"2023-10-18T10:43:04.564188+0000","last_clean":"2023-10-18T10:43:04.564188+0000","last_became_active":"2023-10-18T10:43:03.422095+0000","last_became_peered":"2023-10-18T10:43:03.422095+0000","last_unstale":"2023-10-18T10:43:04.564188+0000","last_undegraded":"2023-10-18T10:43:04.564188+0000","last_fullsized":"2023-10-18T10:43:04.564188+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:36.033906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564111+0000","last_change":"2023-10-18T10:43:03.420436+0000","last_active":"2023-10-18T10:43:04.564111+0000","last_peered":"2023-10-18T10:43:04.564111+0000","last_clean":"2023-10-18T10:43:04.564111+0000","last_became_active":"2023-10-18T10:43:03.419414+0000","last_became_peered":"2023-10-18T10:43:03.419414+0000","last_unstale":"2023-10-18T10:43:04.564111+0000","last_undegraded":"2023-10-18T10:43:04.564111+0000","last_fullsized":"2023-10-18T10:43:04.564111+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:27.017090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564170+0000","last_change":"2023-10-18T10:43:03.425870+0000","last_active":"2023-10-18T10:43:04.564170+0000","last_peered":"2023-10-18T10:43:04.564170+0000","last_clean":"2023-10-18T10:43:04.564170+0000","last_became_active":"2023-10-18T10:43:03.425583+0000","last_became_peered":"2023-10-18T10:43:03.425583+0000","last_unstale":"2023-10-18T10:43:04.564170+0000","last_undegraded":"2023-10-18T10:43:04.564170+0000","last_fullsized":"2023-10-18T10:43:04.564170+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:07.373278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408816+0000","last_change":"2023-10-18T10:43:03.421921+0000","last_active":"2023-10-18T10:43:04.408816+0000","last_peered":"2023-10-18T10:43:04.408816+0000","last_clean":"2023-10-18T10:43:04.408816+0000","last_became_active":"2023-10-18T10:43:03.421644+0000","last_became_peered":"2023-10-18T10:43:03.421644+0000","last_unstale":"2023-10-18T10:43:04.408816+0000","last_undegraded":"2023-10-18T10:43:04.408816+0000","last_fullsized":"2023-10-18T10:43:04.408816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:58.741901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563868+0000","last_change":"2023-10-18T10:43:03.415137+0000","last_active":"2023-10-18T10:43:04.563868+0000","last_peered":"2023-10-18T10:43:04.563868+0000","last_clean":"2023-10-18T10:43:04.563868+0000","last_became_active":"2023-10-18T10:43:03.414871+0000","last_became_peered":"2023-10-18T10:43:03.414871+0000","last_unstale":"2023-10-18T10:43:04.563868+0000","last_undegraded":"2023-10-18T10:43:04.563868+0000","last_fullsized":"2023-10-18T10:43:04.563868+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:33.182433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563789+0000","last_change":"2023-10-18T10:43:03.420511+0000","last_active":"2023-10-18T10:43:04.563789+0000","last_peered":"2023-10-18T10:43:04.563789+0000","last_clean":"2023-10-18T10:43:04.563789+0000","last_became_active":"2023-10-18T10:43:03.420241+0000","last_became_peered":"2023-10-18T10:43:03.420241+0000","last_unstale":"2023-10-18T10:43:04.563789+0000","last_undegraded":"2023-10-18T10:43:04.563789+0000","last_fullsized":"2023-10-18T10:43:04.563789+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:38.074311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563696+0000","last_change":"2023-10-18T10:43:03.417552+0000","last_active":"2023-10-18T10:43:04.563696+0000","last_peered":"2023-10-18T10:43:04.563696+0000","last_clean":"2023-10-18T10:43:04.563696+0000","last_became_active":"2023-10-18T10:43:03.417291+0000","last_became_peered":"2023-10-18T10:43:03.417291+0000","last_unstale":"2023-10-18T10:43:04.563696+0000","last_undegraded":"2023-10-18T10:43:04.563696+0000","last_fullsized":"2023-10-18T10:43:04.563696+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:18.941147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.425285+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.424971+0000","last_became_peered":"2023-10-18T10:43:03.424971+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:00.092514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565204+0000","last_change":"2023-10-18T10:43:03.416033+0000","last_active":"2023-10-18T10:43:04.565204+0000","last_peered":"2023-10-18T10:43:04.565204+0000","last_clean":"2023-10-18T10:43:04.565204+0000","last_became_active":"2023-10-18T10:43:03.415763+0000","last_became_peered":"2023-10-18T10:43:03.415763+0000","last_unstale":"2023-10-18T10:43:04.565204+0000","last_undegraded":"2023-10-18T10:43:04.565204+0000","last_fullsized":"2023-10-18T10:43:04.565204+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:36.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564171+0000","last_change":"2023-10-18T10:43:03.418626+0000","last_active":"2023-10-18T10:43:04.564171+0000","last_peered":"2023-10-18T10:43:04.564171+0000","last_clean":"2023-10-18T10:43:04.564171+0000","last_became_active":"2023-10-18T10:43:03.418344+0000","last_became_peered":"2023-10-18T10:43:03.418344+0000","last_unstale":"2023-10-18T10:43:04.564171+0000","last_undegraded":"2023-10-18T10:43:04.564171+0000","last_fullsized":"2023-10-18T10:43:04.564171+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:55.314298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564184+0000","last_change":"2023-10-18T10:43:03.421228+0000","last_active":"2023-10-18T10:43:04.564184+0000","last_peered":"2023-10-18T10:43:04.564184+0000","last_clean":"2023-10-18T10:43:04.564184+0000","last_became_active":"2023-10-18T10:43:03.420908+0000","last_became_peered":"2023-10-18T10:43:03.420908+0000","last_unstale":"2023-10-18T10:43:04.564184+0000","last_undegraded":"2023-10-18T10:43:04.564184+0000","last_fullsized":"2023-10-18T10:43:04.564184+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:19.745476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564166+0000","last_change":"2023-10-18T10:43:03.421765+0000","last_active":"2023-10-18T10:43:04.564166+0000","last_peered":"2023-10-18T10:43:04.564166+0000","last_clean":"2023-10-18T10:43:04.564166+0000","last_became_active":"2023-10-18T10:43:03.421501+0000","last_became_peered":"2023-10-18T10:43:03.421501+0000","last_unstale":"2023-10-18T10:43:04.564166+0000","last_undegraded":"2023-10-18T10:43:04.564166+0000","last_fullsized":"2023-10-18T10:43:04.564166+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:06.267958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.424152+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.423890+0000","last_became_peered":"2023-10-18T10:43:03.423890+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:26.100932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564263+0000","last_change":"2023-10-18T10:43:03.417440+0000","last_active":"2023-10-18T10:43:04.564263+0000","last_peered":"2023-10-18T10:43:04.564263+0000","last_clean":"2023-10-18T10:43:04.564263+0000","last_became_active":"2023-10-18T10:43:03.417201+0000","last_became_peered":"2023-10-18T10:43:03.417201+0000","last_unstale":"2023-10-18T10:43:04.564263+0000","last_undegraded":"2023-10-18T10:43:04.564263+0000","last_fullsized":"2023-10-18T10:43:04.564263+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:16.834481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564364+0000","last_change":"2023-10-18T10:43:03.419666+0000","last_active":"2023-10-18T10:43:04.564364+0000","last_peered":"2023-10-18T10:43:04.564364+0000","last_clean":"2023-10-18T10:43:04.564364+0000","last_became_active":"2023-10-18T10:43:03.418843+0000","last_became_peered":"2023-10-18T10:43:03.418843+0000","last_unstale":"2023-10-18T10:43:04.564364+0000","last_undegraded":"2023-10-18T10:43:04.564364+0000","last_fullsized":"2023-10-18T10:43:04.564364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:25.903261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565004+0000","last_change":"2023-10-18T10:43:03.423183+0000","last_active":"2023-10-18T10:43:04.565004+0000","last_peered":"2023-10-18T10:43:04.565004+0000","last_clean":"2023-10-18T10:43:04.565004+0000","last_became_active":"2023-10-18T10:43:03.422819+0000","last_became_peered":"2023-10-18T10:43:03.422819+0000","last_unstale":"2023-10-18T10:43:04.565004+0000","last_undegraded":"2023-10-18T10:43:04.565004+0000","last_fullsized":"2023-10-18T10:43:04.565004+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:34.324413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564301+0000","last_change":"2023-10-18T10:43:03.417982+0000","last_active":"2023-10-18T10:43:04.564301+0000","last_peered":"2023-10-18T10:43:04.564301+0000","last_clean":"2023-10-18T10:43:04.564301+0000","last_became_active":"2023-10-18T10:43:03.417736+0000","last_became_peered":"2023-10-18T10:43:03.417736+0000","last_unstale":"2023-10-18T10:43:04.564301+0000","last_undegraded":"2023-10-18T10:43:04.564301+0000","last_fullsized":"2023-10-18T10:43:04.564301+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:51.640858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564774+0000","last_change":"2023-10-18T10:43:03.416816+0000","last_active":"2023-10-18T10:43:04.564774+0000","last_peered":"2023-10-18T10:43:04.564774+0000","last_clean":"2023-10-18T10:43:04.564774+0000","last_became_active":"2023-10-18T10:43:03.416441+0000","last_became_peered":"2023-10-18T10:43:03.416441+0000","last_unstale":"2023-10-18T10:43:04.564774+0000","last_undegraded":"2023-10-18T10:43:04.564774+0000","last_fullsized":"2023-10-18T10:43:04.564774+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:23.903407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563541+0000","last_change":"2023-10-18T10:43:03.423095+0000","last_active":"2023-10-18T10:43:04.563541+0000","last_peered":"2023-10-18T10:43:04.563541+0000","last_clean":"2023-10-18T10:43:04.563541+0000","last_became_active":"2023-10-18T10:43:03.422840+0000","last_became_peered":"2023-10-18T10:43:03.422840+0000","last_unstale":"2023-10-18T10:43:04.563541+0000","last_undegraded":"2023-10-18T10:43:04.563541+0000","last_fullsized":"2023-10-18T10:43:04.563541+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:09.585007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564675+0000","last_change":"2023-10-18T10:43:03.423114+0000","last_active":"2023-10-18T10:43:04.564675+0000","last_peered":"2023-10-18T10:43:04.564675+0000","last_clean":"2023-10-18T10:43:04.564675+0000","last_became_active":"2023-10-18T10:43:03.422862+0000","last_became_peered":"2023-10-18T10:43:03.422862+0000","last_unstale":"2023-10-18T10:43:04.564675+0000","last_undegraded":"2023-10-18T10:43:04.564675+0000","last_fullsized":"2023-10-18T10:43:04.564675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:40:54.685089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"127'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408994+0000","last_change":"2023-10-18T10:43:02.963797+0000","last_active":"2023-10-18T10:43:04.408994+0000","last_peered":"2023-10-18T10:43:04.408994+0000","last_clean":"2023-10-18T10:43:04.408994+0000","last_became_active":"2023-10-18T10:43:02.963525+0000","last_became_peered":"2023-10-18T10:43:02.963525+0000","last_unstale":"2023-10-18T10:43:04.408994+0000","last_undegraded":"2023-10-18T10:43:04.408994+0000","last_fullsized":"2023-10-18T10:43:04.408994+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:18:22.535089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563557+0000","last_change":"2023-10-18T10:43:03.419783+0000","last_active":"2023-10-18T10:43:04.563557+0000","last_peered":"2023-10-18T10:43:04.563557+0000","last_clean":"2023-10-18T10:43:04.563557+0000","last_became_active":"2023-10-18T10:43:03.419543+0000","last_became_peered":"2023-10-18T10:43:03.419543+0000","last_unstale":"2023-10-18T10:43:04.563557+0000","last_undegraded":"2023-10-18T10:43:04.563557+0000","last_fullsized":"2023-10-18T10:43:04.563557+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:39.668528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564617+0000","last_change":"2023-10-18T10:43:02.965235+0000","last_active":"2023-10-18T10:43:04.564617+0000","last_peered":"2023-10-18T10:43:04.564617+0000","last_clean":"2023-10-18T10:43:04.564617+0000","last_became_active":"2023-10-18T10:43:02.964880+0000","last_became_peered":"2023-10-18T10:43:02.964880+0000","last_unstale":"2023-10-18T10:43:04.564617+0000","last_undegraded":"2023-10-18T10:43:04.564617+0000","last_fullsized":"2023-10-18T10:43:04.564617+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:48.610823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564182+0000","last_change":"2023-10-18T10:43:03.416904+0000","last_active":"2023-10-18T10:43:04.564182+0000","last_peered":"2023-10-18T10:43:04.564182+0000","last_clean":"2023-10-18T10:43:04.564182+0000","last_became_active":"2023-10-18T10:43:03.416489+0000","last_became_peered":"2023-10-18T10:43:03.416489+0000","last_unstale":"2023-10-18T10:43:04.564182+0000","last_undegraded":"2023-10-18T10:43:04.564182+0000","last_fullsized":"2023-10-18T10:43:04.564182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:36.136092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564337+0000","last_change":"2023-10-18T10:43:02.969842+0000","last_active":"2023-10-18T10:43:04.564337+0000","last_peered":"2023-10-18T10:43:04.564337+0000","last_clean":"2023-10-18T10:43:04.564337+0000","last_became_active":"2023-10-18T10:43:02.969620+0000","last_became_peered":"2023-10-18T10:43:02.969620+0000","last_unstale":"2023-10-18T10:43:04.564337+0000","last_undegraded":"2023-10-18T10:43:04.564337+0000","last_fullsized":"2023-10-18T10:43:04.564337+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:23:14.364024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409001+0000","last_change":"2023-10-18T10:43:03.417690+0000","last_active":"2023-10-18T10:43:04.409001+0000","last_peered":"2023-10-18T10:43:04.409001+0000","last_clean":"2023-10-18T10:43:04.409001+0000","last_became_active":"2023-10-18T10:43:03.417452+0000","last_became_peered":"2023-10-18T10:43:03.417452+0000","last_unstale":"2023-10-18T10:43:04.409001+0000","last_undegraded":"2023-10-18T10:43:04.409001+0000","last_fullsized":"2023-10-18T10:43:04.409001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:28.022888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563896+0000","last_change":"2023-10-18T10:43:02.965103+0000","last_active":"2023-10-18T10:43:04.563896+0000","last_peered":"2023-10-18T10:43:04.563896+0000","last_clean":"2023-10-18T10:43:04.563896+0000","last_became_active":"2023-10-18T10:43:02.964812+0000","last_became_peered":"2023-10-18T10:43:02.964812+0000","last_unstale":"2023-10-18T10:43:04.563896+0000","last_undegraded":"2023-10-18T10:43:04.563896+0000","last_fullsized":"2023-10-18T10:43:04.563896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:08.011562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564304+0000","last_change":"2023-10-18T10:43:03.424682+0000","last_active":"2023-10-18T10:43:04.564304+0000","last_peered":"2023-10-18T10:43:04.564304+0000","last_clean":"2023-10-18T10:43:04.564304+0000","last_became_active":"2023-10-18T10:43:03.424416+0000","last_became_peered":"2023-10-18T10:43:03.424416+0000","last_unstale":"2023-10-18T10:43:04.564304+0000","last_undegraded":"2023-10-18T10:43:04.564304+0000","last_fullsized":"2023-10-18T10:43:04.564304+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:29.729372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564479+0000","last_change":"2023-10-18T10:43:02.966346+0000","last_active":"2023-10-18T10:43:04.564479+0000","last_peered":"2023-10-18T10:43:04.564479+0000","last_clean":"2023-10-18T10:43:04.564479+0000","last_became_active":"2023-10-18T10:43:02.966041+0000","last_became_peered":"2023-10-18T10:43:02.966041+0000","last_unstale":"2023-10-18T10:43:04.564479+0000","last_undegraded":"2023-10-18T10:43:04.564479+0000","last_fullsized":"2023-10-18T10:43:04.564479+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:04.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565269+0000","last_change":"2023-10-18T10:43:03.424161+0000","last_active":"2023-10-18T10:43:04.565269+0000","last_peered":"2023-10-18T10:43:04.565269+0000","last_clean":"2023-10-18T10:43:04.565269+0000","last_became_active":"2023-10-18T10:43:03.423854+0000","last_became_peered":"2023-10-18T10:43:03.423854+0000","last_unstale":"2023-10-18T10:43:04.565269+0000","last_undegraded":"2023-10-18T10:43:04.565269+0000","last_fullsized":"2023-10-18T10:43:04.565269+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:24.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409024+0000","last_change":"2023-10-18T10:43:03.417182+0000","last_active":"2023-10-18T10:43:04.409024+0000","last_peered":"2023-10-18T10:43:04.409024+0000","last_clean":"2023-10-18T10:43:04.409024+0000","last_became_active":"2023-10-18T10:43:03.416917+0000","last_became_peered":"2023-10-18T10:43:03.416917+0000","last_unstale":"2023-10-18T10:43:04.409024+0000","last_undegraded":"2023-10-18T10:43:04.409024+0000","last_fullsized":"2023-10-18T10:43:04.409024+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:49.418978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408682+0000","last_change":"2023-10-18T10:43:02.969298+0000","last_active":"2023-10-18T10:43:04.408682+0000","last_peered":"2023-10-18T10:43:04.408682+0000","last_clean":"2023-10-18T10:43:04.408682+0000","last_became_active":"2023-10-18T10:43:02.969045+0000","last_became_peered":"2023-10-18T10:43:02.969045+0000","last_unstale":"2023-10-18T10:43:04.408682+0000","last_undegraded":"2023-10-18T10:43:04.408682+0000","last_fullsized":"2023-10-18T10:43:04.408682+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:24.897688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564560+0000","last_change":"2023-10-18T10:43:03.424618+0000","last_active":"2023-10-18T10:43:04.564560+0000","last_peered":"2023-10-18T10:43:04.564560+0000","last_clean":"2023-10-18T10:43:04.564560+0000","last_became_active":"2023-10-18T10:43:03.424369+0000","last_became_peered":"2023-10-18T10:43:03.424369+0000","last_unstale":"2023-10-18T10:43:04.564560+0000","last_undegraded":"2023-10-18T10:43:04.564560+0000","last_fullsized":"2023-10-18T10:43:04.564560+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:25.195869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563691+0000","last_change":"2023-10-18T10:43:03.414895+0000","last_active":"2023-10-18T10:43:04.563691+0000","last_peered":"2023-10-18T10:43:04.563691+0000","last_clean":"2023-10-18T10:43:04.563691+0000","last_became_active":"2023-10-18T10:43:03.414661+0000","last_became_peered":"2023-10-18T10:43:03.414661+0000","last_unstale":"2023-10-18T10:43:04.563691+0000","last_undegraded":"2023-10-18T10:43:04.563691+0000","last_fullsized":"2023-10-18T10:43:04.563691+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:46.699483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564739+0000","last_change":"2023-10-18T10:43:02.971336+0000","last_active":"2023-10-18T10:43:04.564739+0000","last_peered":"2023-10-18T10:43:04.564739+0000","last_clean":"2023-10-18T10:43:04.564739+0000","last_became_active":"2023-10-18T10:43:02.971030+0000","last_became_peered":"2023-10-18T10:43:02.971030+0000","last_unstale":"2023-10-18T10:43:04.564739+0000","last_undegraded":"2023-10-18T10:43:04.564739+0000","last_fullsized":"2023-10-18T10:43:04.564739+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:32:47.348309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563840+0000","last_change":"2023-10-18T10:43:03.420254+0000","last_active":"2023-10-18T10:43:04.563840+0000","last_peered":"2023-10-18T10:43:04.563840+0000","last_clean":"2023-10-18T10:43:04.563840+0000","last_became_active":"2023-10-18T10:43:03.420019+0000","last_became_peered":"2023-10-18T10:43:03.420019+0000","last_unstale":"2023-10-18T10:43:04.563840+0000","last_undegraded":"2023-10-18T10:43:04.563840+0000","last_fullsized":"2023-10-18T10:43:04.563840+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:02.531521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"127'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564189+0000","last_change":"2023-10-18T10:43:02.974197+0000","last_active":"2023-10-18T10:43:04.564189+0000","last_peered":"2023-10-18T10:43:04.564189+0000","last_clean":"2023-10-18T10:43:04.564189+0000","last_became_active":"2023-10-18T10:43:02.973961+0000","last_became_peered":"2023-10-18T10:43:02.973961+0000","last_unstale":"2023-10-18T10:43:04.564189+0000","last_undegraded":"2023-10-18T10:43:04.564189+0000","last_fullsized":"2023-10-18T10:43:04.564189+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:57.736328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564658+0000","last_change":"2023-10-18T10:43:03.417830+0000","last_active":"2023-10-18T10:43:04.564658+0000","last_peered":"2023-10-18T10:43:04.564658+0000","last_clean":"2023-10-18T10:43:04.564658+0000","last_became_active":"2023-10-18T10:43:03.417031+0000","last_became_peered":"2023-10-18T10:43:03.417031+0000","last_unstale":"2023-10-18T10:43:04.564658+0000","last_undegraded":"2023-10-18T10:43:04.564658+0000","last_fullsized":"2023-10-18T10:43:04.564658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:56.478327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"127'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565099+0000","last_change":"2023-10-18T10:43:02.959201+0000","last_active":"2023-10-18T10:43:04.565099+0000","last_peered":"2023-10-18T10:43:04.565099+0000","last_clean":"2023-10-18T10:43:04.565099+0000","last_became_active":"2023-10-18T10:43:02.958888+0000","last_became_peered":"2023-10-18T10:43:02.958888+0000","last_unstale":"2023-10-18T10:43:04.565099+0000","last_undegraded":"2023-10-18T10:43:04.565099+0000","last_fullsized":"2023-10-18T10:43:04.565099+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:04.257834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564597+0000","last_change":"2023-10-18T10:43:03.419410+0000","last_active":"2023-10-18T10:43:04.564597+0000","last_peered":"2023-10-18T10:43:04.564597+0000","last_clean":"2023-10-18T10:43:04.564597+0000","last_became_active":"2023-10-18T10:43:03.419055+0000","last_became_peered":"2023-10-18T10:43:03.419055+0000","last_unstale":"2023-10-18T10:43:04.564597+0000","last_undegraded":"2023-10-18T10:43:04.564597+0000","last_fullsized":"2023-10-18T10:43:04.564597+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:06.895973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564742+0000","last_change":"2023-10-18T10:43:02.972054+0000","last_active":"2023-10-18T10:43:04.564742+0000","last_peered":"2023-10-18T10:43:04.564742+0000","last_clean":"2023-10-18T10:43:04.564742+0000","last_became_active":"2023-10-18T10:43:02.971783+0000","last_became_peered":"2023-10-18T10:43:02.971783+0000","last_unstale":"2023-10-18T10:43:04.564742+0000","last_undegraded":"2023-10-18T10:43:04.564742+0000","last_fullsized":"2023-10-18T10:43:04.564742+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:07.914251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565070+0000","last_change":"2023-10-18T10:43:03.424942+0000","last_active":"2023-10-18T10:43:04.565070+0000","last_peered":"2023-10-18T10:43:04.565070+0000","last_clean":"2023-10-18T10:43:04.565070+0000","last_became_active":"2023-10-18T10:43:03.424610+0000","last_became_peered":"2023-10-18T10:43:03.424610+0000","last_unstale":"2023-10-18T10:43:04.565070+0000","last_undegraded":"2023-10-18T10:43:04.565070+0000","last_fullsized":"2023-10-18T10:43:04.565070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:58.441002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408796+0000","last_change":"2023-10-18T10:43:02.962770+0000","last_active":"2023-10-18T10:43:04.408796+0000","last_peered":"2023-10-18T10:43:04.408796+0000","last_clean":"2023-10-18T10:43:04.408796+0000","last_became_active":"2023-10-18T10:43:02.962492+0000","last_became_peered":"2023-10-18T10:43:02.962492+0000","last_unstale":"2023-10-18T10:43:04.408796+0000","last_undegraded":"2023-10-18T10:43:04.408796+0000","last_fullsized":"2023-10-18T10:43:04.408796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:04.206263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565182+0000","last_change":"2023-10-18T10:43:03.417626+0000","last_active":"2023-10-18T10:43:04.565182+0000","last_peered":"2023-10-18T10:43:04.565182+0000","last_clean":"2023-10-18T10:43:04.565182+0000","last_became_active":"2023-10-18T10:43:03.417374+0000","last_became_peered":"2023-10-18T10:43:03.417374+0000","last_unstale":"2023-10-18T10:43:04.565182+0000","last_undegraded":"2023-10-18T10:43:04.565182+0000","last_fullsized":"2023-10-18T10:43:04.565182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:39.656533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563634+0000","last_change":"2023-10-18T10:43:02.966586+0000","last_active":"2023-10-18T10:43:04.563634+0000","last_peered":"2023-10-18T10:43:04.563634+0000","last_clean":"2023-10-18T10:43:04.563634+0000","last_became_active":"2023-10-18T10:43:02.966340+0000","last_became_peered":"2023-10-18T10:43:02.966340+0000","last_unstale":"2023-10-18T10:43:04.563634+0000","last_undegraded":"2023-10-18T10:43:04.563634+0000","last_fullsized":"2023-10-18T10:43:04.563634+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.288426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565086+0000","last_change":"2023-10-18T10:43:03.425524+0000","last_active":"2023-10-18T10:43:04.565086+0000","last_peered":"2023-10-18T10:43:04.565086+0000","last_clean":"2023-10-18T10:43:04.565086+0000","last_became_active":"2023-10-18T10:43:03.425299+0000","last_became_peered":"2023-10-18T10:43:03.425299+0000","last_unstale":"2023-10-18T10:43:04.565086+0000","last_undegraded":"2023-10-18T10:43:04.565086+0000","last_fullsized":"2023-10-18T10:43:04.565086+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:18.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563658+0000","last_change":"2023-10-18T10:43:03.415655+0000","last_active":"2023-10-18T10:43:04.563658+0000","last_peered":"2023-10-18T10:43:04.563658+0000","last_clean":"2023-10-18T10:43:04.563658+0000","last_became_active":"2023-10-18T10:43:03.415397+0000","last_became_peered":"2023-10-18T10:43:03.415397+0000","last_unstale":"2023-10-18T10:43:04.563658+0000","last_undegraded":"2023-10-18T10:43:04.563658+0000","last_fullsized":"2023-10-18T10:43:04.563658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:17.193216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564284+0000","last_change":"2023-10-18T10:43:02.964287+0000","last_active":"2023-10-18T10:43:04.564284+0000","last_peered":"2023-10-18T10:43:04.564284+0000","last_clean":"2023-10-18T10:43:04.564284+0000","last_became_active":"2023-10-18T10:43:02.964033+0000","last_became_peered":"2023-10-18T10:43:02.964033+0000","last_unstale":"2023-10-18T10:43:04.564284+0000","last_undegraded":"2023-10-18T10:43:04.564284+0000","last_fullsized":"2023-10-18T10:43:04.564284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:33.533023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564963+0000","last_change":"2023-10-18T10:43:03.419976+0000","last_active":"2023-10-18T10:43:04.564963+0000","last_peered":"2023-10-18T10:43:04.564963+0000","last_clean":"2023-10-18T10:43:04.564963+0000","last_became_active":"2023-10-18T10:43:03.419611+0000","last_became_peered":"2023-10-18T10:43:03.419611+0000","last_unstale":"2023-10-18T10:43:04.564963+0000","last_undegraded":"2023-10-18T10:43:04.564963+0000","last_fullsized":"2023-10-18T10:43:04.564963+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:57.163873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564210+0000","last_change":"2023-10-18T10:43:02.964463+0000","last_active":"2023-10-18T10:43:04.564210+0000","last_peered":"2023-10-18T10:43:04.564210+0000","last_clean":"2023-10-18T10:43:04.564210+0000","last_became_active":"2023-10-18T10:43:02.964036+0000","last_became_peered":"2023-10-18T10:43:02.964036+0000","last_unstale":"2023-10-18T10:43:04.564210+0000","last_undegraded":"2023-10-18T10:43:04.564210+0000","last_fullsized":"2023-10-18T10:43:04.564210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:27.017315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564542+0000","last_change":"2023-10-18T10:43:03.416778+0000","last_active":"2023-10-18T10:43:04.564542+0000","last_peered":"2023-10-18T10:43:04.564542+0000","last_clean":"2023-10-18T10:43:04.564542+0000","last_became_active":"2023-10-18T10:43:03.416492+0000","last_became_peered":"2023-10-18T10:43:03.416492+0000","last_unstale":"2023-10-18T10:43:04.564542+0000","last_undegraded":"2023-10-18T10:43:04.564542+0000","last_fullsized":"2023-10-18T10:43:04.564542+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:29.332215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564773+0000","last_change":"2023-10-18T10:43:02.965244+0000","last_active":"2023-10-18T10:43:04.564773+0000","last_peered":"2023-10-18T10:43:04.564773+0000","last_clean":"2023-10-18T10:43:04.564773+0000","last_became_active":"2023-10-18T10:43:02.964962+0000","last_became_peered":"2023-10-18T10:43:02.964962+0000","last_unstale":"2023-10-18T10:43:04.564773+0000","last_undegraded":"2023-10-18T10:43:04.564773+0000","last_fullsized":"2023-10-18T10:43:04.564773+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:10.229768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563816+0000","last_change":"2023-10-18T10:43:03.419372+0000","last_active":"2023-10-18T10:43:04.563816+0000","last_peered":"2023-10-18T10:43:04.563816+0000","last_clean":"2023-10-18T10:43:04.563816+0000","last_became_active":"2023-10-18T10:43:03.419133+0000","last_became_peered":"2023-10-18T10:43:03.419133+0000","last_unstale":"2023-10-18T10:43:04.563816+0000","last_undegraded":"2023-10-18T10:43:04.563816+0000","last_fullsized":"2023-10-18T10:43:04.563816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:25.080148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564382+0000","last_change":"2023-10-18T10:43:02.963180+0000","last_active":"2023-10-18T10:43:04.564382+0000","last_peered":"2023-10-18T10:43:04.564382+0000","last_clean":"2023-10-18T10:43:04.564382+0000","last_became_active":"2023-10-18T10:43:02.962924+0000","last_became_peered":"2023-10-18T10:43:02.962924+0000","last_unstale":"2023-10-18T10:43:04.564382+0000","last_undegraded":"2023-10-18T10:43:04.564382+0000","last_fullsized":"2023-10-18T10:43:04.564382+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:27.605859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564165+0000","last_change":"2023-10-18T10:43:03.424395+0000","last_active":"2023-10-18T10:43:04.564165+0000","last_peered":"2023-10-18T10:43:04.564165+0000","last_clean":"2023-10-18T10:43:04.564165+0000","last_became_active":"2023-10-18T10:43:03.424101+0000","last_became_peered":"2023-10-18T10:43:03.424101+0000","last_unstale":"2023-10-18T10:43:04.564165+0000","last_undegraded":"2023-10-18T10:43:04.564165+0000","last_fullsized":"2023-10-18T10:43:04.564165+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:43.317979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563722+0000","last_change":"2023-10-18T10:43:02.964389+0000","last_active":"2023-10-18T10:43:04.563722+0000","last_peered":"2023-10-18T10:43:04.563722+0000","last_clean":"2023-10-18T10:43:04.563722+0000","last_became_active":"2023-10-18T10:43:02.964134+0000","last_became_peered":"2023-10-18T10:43:02.964134+0000","last_unstale":"2023-10-18T10:43:04.563722+0000","last_undegraded":"2023-10-18T10:43:04.563722+0000","last_fullsized":"2023-10-18T10:43:04.563722+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:27.504789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408932+0000","last_change":"2023-10-18T10:43:03.417105+0000","last_active":"2023-10-18T10:43:04.408932+0000","last_peered":"2023-10-18T10:43:04.408932+0000","last_clean":"2023-10-18T10:43:04.408932+0000","last_became_active":"2023-10-18T10:43:03.416864+0000","last_became_peered":"2023-10-18T10:43:03.416864+0000","last_unstale":"2023-10-18T10:43:04.408932+0000","last_undegraded":"2023-10-18T10:43:04.408932+0000","last_fullsized":"2023-10-18T10:43:04.408932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:09.017135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564862+0000","last_change":"2023-10-18T10:43:02.967733+0000","last_active":"2023-10-18T10:43:04.564862+0000","last_peered":"2023-10-18T10:43:04.564862+0000","last_clean":"2023-10-18T10:43:04.564862+0000","last_became_active":"2023-10-18T10:43:02.967444+0000","last_became_peered":"2023-10-18T10:43:02.967444+0000","last_unstale":"2023-10-18T10:43:04.564862+0000","last_undegraded":"2023-10-18T10:43:04.564862+0000","last_fullsized":"2023-10-18T10:43:04.564862+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:59.437631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564910+0000","last_change":"2023-10-18T10:43:03.418676+0000","last_active":"2023-10-18T10:43:04.564910+0000","last_peered":"2023-10-18T10:43:04.564910+0000","last_clean":"2023-10-18T10:43:04.564910+0000","last_became_active":"2023-10-18T10:43:03.418441+0000","last_became_peered":"2023-10-18T10:43:03.418441+0000","last_unstale":"2023-10-18T10:43:04.564910+0000","last_undegraded":"2023-10-18T10:43:04.564910+0000","last_fullsized":"2023-10-18T10:43:04.564910+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:30.204629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563690+0000","last_change":"2023-10-18T10:43:02.963728+0000","last_active":"2023-10-18T10:43:04.563690+0000","last_peered":"2023-10-18T10:43:04.563690+0000","last_clean":"2023-10-18T10:43:04.563690+0000","last_became_active":"2023-10-18T10:43:02.963467+0000","last_became_peered":"2023-10-18T10:43:02.963467+0000","last_unstale":"2023-10-18T10:43:04.563690+0000","last_undegraded":"2023-10-18T10:43:04.563690+0000","last_fullsized":"2023-10-18T10:43:04.563690+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:38.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564445+0000","last_change":"2023-10-18T10:43:03.421100+0000","last_active":"2023-10-18T10:43:04.564445+0000","last_peered":"2023-10-18T10:43:04.564445+0000","last_clean":"2023-10-18T10:43:04.564445+0000","last_became_active":"2023-10-18T10:43:03.420845+0000","last_became_peered":"2023-10-18T10:43:03.420845+0000","last_unstale":"2023-10-18T10:43:04.564445+0000","last_undegraded":"2023-10-18T10:43:04.564445+0000","last_fullsized":"2023-10-18T10:43:04.564445+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:32:19.619118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564074+0000","last_change":"2023-10-18T10:43:02.971128+0000","last_active":"2023-10-18T10:43:04.564074+0000","last_peered":"2023-10-18T10:43:04.564074+0000","last_clean":"2023-10-18T10:43:04.564074+0000","last_became_active":"2023-10-18T10:43:02.970845+0000","last_became_peered":"2023-10-18T10:43:02.970845+0000","last_unstale":"2023-10-18T10:43:04.564074+0000","last_undegraded":"2023-10-18T10:43:04.564074+0000","last_fullsized":"2023-10-18T10:43:04.564074+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:57.435429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564812+0000","last_change":"2023-10-18T10:43:02.972906+0000","last_active":"2023-10-18T10:43:04.564812+0000","last_peered":"2023-10-18T10:43:04.564812+0000","last_clean":"2023-10-18T10:43:04.564812+0000","last_became_active":"2023-10-18T10:43:02.972652+0000","last_became_peered":"2023-10-18T10:43:02.972652+0000","last_unstale":"2023-10-18T10:43:04.564812+0000","last_undegraded":"2023-10-18T10:43:04.564812+0000","last_fullsized":"2023-10-18T10:43:04.564812+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:09.602406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408779+0000","last_change":"2023-10-18T10:43:03.416563+0000","last_active":"2023-10-18T10:43:04.408779+0000","last_peered":"2023-10-18T10:43:04.408779+0000","last_clean":"2023-10-18T10:43:04.408779+0000","last_became_active":"2023-10-18T10:43:03.416321+0000","last_became_peered":"2023-10-18T10:43:03.416321+0000","last_unstale":"2023-10-18T10:43:04.408779+0000","last_undegraded":"2023-10-18T10:43:04.408779+0000","last_fullsized":"2023-10-18T10:43:04.408779+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:57.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565167+0000","last_change":"2023-10-18T10:43:02.962995+0000","last_active":"2023-10-18T10:43:04.565167+0000","last_peered":"2023-10-18T10:43:04.565167+0000","last_clean":"2023-10-18T10:43:04.565167+0000","last_became_active":"2023-10-18T10:43:02.962723+0000","last_became_peered":"2023-10-18T10:43:02.962723+0000","last_unstale":"2023-10-18T10:43:04.565167+0000","last_undegraded":"2023-10-18T10:43:04.565167+0000","last_fullsized":"2023-10-18T10:43:04.565167+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:45.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564038+0000","last_change":"2023-10-18T10:43:03.424431+0000","last_active":"2023-10-18T10:43:04.564038+0000","last_peered":"2023-10-18T10:43:04.564038+0000","last_clean":"2023-10-18T10:43:04.564038+0000","last_became_active":"2023-10-18T10:43:03.424173+0000","last_became_peered":"2023-10-18T10:43:03.424173+0000","last_unstale":"2023-10-18T10:43:04.564038+0000","last_undegraded":"2023-10-18T10:43:04.564038+0000","last_fullsized":"2023-10-18T10:43:04.564038+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:19.931739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564415+0000","last_change":"2023-10-18T10:43:02.968945+0000","last_active":"2023-10-18T10:43:04.564415+0000","last_peered":"2023-10-18T10:43:04.564415+0000","last_clean":"2023-10-18T10:43:04.564415+0000","last_became_active":"2023-10-18T10:43:02.968666+0000","last_became_peered":"2023-10-18T10:43:02.968666+0000","last_unstale":"2023-10-18T10:43:04.564415+0000","last_undegraded":"2023-10-18T10:43:04.564415+0000","last_fullsized":"2023-10-18T10:43:04.564415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:26.843460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564480+0000","last_change":"2023-10-18T10:43:03.421780+0000","last_active":"2023-10-18T10:43:04.564480+0000","last_peered":"2023-10-18T10:43:04.564480+0000","last_clean":"2023-10-18T10:43:04.564480+0000","last_became_active":"2023-10-18T10:43:03.421547+0000","last_became_peered":"2023-10-18T10:43:03.421547+0000","last_unstale":"2023-10-18T10:43:04.564480+0000","last_undegraded":"2023-10-18T10:43:04.564480+0000","last_fullsized":"2023-10-18T10:43:04.564480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:54:47.177774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563859+0000","last_change":"2023-10-18T10:43:03.423032+0000","last_active":"2023-10-18T10:43:04.563859+0000","last_peered":"2023-10-18T10:43:04.563859+0000","last_clean":"2023-10-18T10:43:04.563859+0000","last_became_active":"2023-10-18T10:43:03.422767+0000","last_became_peered":"2023-10-18T10:43:03.422767+0000","last_unstale":"2023-10-18T10:43:04.563859+0000","last_undegraded":"2023-10-18T10:43:04.563859+0000","last_fullsized":"2023-10-18T10:43:04.563859+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:17.615763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563911+0000","last_change":"2023-10-18T10:43:02.965062+0000","last_active":"2023-10-18T10:43:04.563911+0000","last_peered":"2023-10-18T10:43:04.563911+0000","last_clean":"2023-10-18T10:43:04.563911+0000","last_became_active":"2023-10-18T10:43:02.964753+0000","last_became_peered":"2023-10-18T10:43:02.964753+0000","last_unstale":"2023-10-18T10:43:04.563911+0000","last_undegraded":"2023-10-18T10:43:04.563911+0000","last_fullsized":"2023-10-18T10:43:04.563911+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:41.151549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564340+0000","last_change":"2023-10-18T10:43:03.421221+0000","last_active":"2023-10-18T10:43:04.564340+0000","last_peered":"2023-10-18T10:43:04.564340+0000","last_clean":"2023-10-18T10:43:04.564340+0000","last_became_active":"2023-10-18T10:43:03.420954+0000","last_became_peered":"2023-10-18T10:43:03.420954+0000","last_unstale":"2023-10-18T10:43:04.564340+0000","last_undegraded":"2023-10-18T10:43:04.564340+0000","last_fullsized":"2023-10-18T10:43:04.564340+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:42.945030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564932+0000","last_change":"2023-10-18T10:43:02.962374+0000","last_active":"2023-10-18T10:43:04.564932+0000","last_peered":"2023-10-18T10:43:04.564932+0000","last_clean":"2023-10-18T10:43:04.564932+0000","last_became_active":"2023-10-18T10:43:02.962065+0000","last_became_peered":"2023-10-18T10:43:02.962065+0000","last_unstale":"2023-10-18T10:43:04.564932+0000","last_undegraded":"2023-10-18T10:43:04.564932+0000","last_fullsized":"2023-10-18T10:43:04.564932+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:37.693671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564809+0000","last_change":"2023-10-18T10:43:03.418250+0000","last_active":"2023-10-18T10:43:04.564809+0000","last_peered":"2023-10-18T10:43:04.564809+0000","last_clean":"2023-10-18T10:43:04.564809+0000","last_became_active":"2023-10-18T10:43:03.417934+0000","last_became_peered":"2023-10-18T10:43:03.417934+0000","last_unstale":"2023-10-18T10:43:04.564809+0000","last_undegraded":"2023-10-18T10:43:04.564809+0000","last_fullsized":"2023-10-18T10:43:04.564809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:09.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563736+0000","last_change":"2023-10-18T10:43:02.965897+0000","last_active":"2023-10-18T10:43:04.563736+0000","last_peered":"2023-10-18T10:43:04.563736+0000","last_clean":"2023-10-18T10:43:04.563736+0000","last_became_active":"2023-10-18T10:43:02.965636+0000","last_became_peered":"2023-10-18T10:43:02.965636+0000","last_unstale":"2023-10-18T10:43:04.563736+0000","last_undegraded":"2023-10-18T10:43:04.563736+0000","last_fullsized":"2023-10-18T10:43:04.563736+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:10:39.372132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564606+0000","last_change":"2023-10-18T10:43:03.420416+0000","last_active":"2023-10-18T10:43:04.564606+0000","last_peered":"2023-10-18T10:43:04.564606+0000","last_clean":"2023-10-18T10:43:04.564606+0000","last_became_active":"2023-10-18T10:43:03.420064+0000","last_became_peered":"2023-10-18T10:43:03.420064+0000","last_unstale":"2023-10-18T10:43:04.564606+0000","last_undegraded":"2023-10-18T10:43:04.564606+0000","last_fullsized":"2023-10-18T10:43:04.564606+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:01.892160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564707+0000","last_change":"2023-10-18T10:43:02.962533+0000","last_active":"2023-10-18T10:43:04.564707+0000","last_peered":"2023-10-18T10:43:04.564707+0000","last_clean":"2023-10-18T10:43:04.564707+0000","last_became_active":"2023-10-18T10:43:02.962275+0000","last_became_peered":"2023-10-18T10:43:02.962275+0000","last_unstale":"2023-10-18T10:43:04.564707+0000","last_undegraded":"2023-10-18T10:43:04.564707+0000","last_fullsized":"2023-10-18T10:43:04.564707+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:26.189773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"117'214","reported_seq":611,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563741+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:43:04.563741+0000","last_peered":"2023-10-18T10:43:04.563741+0000","last_clean":"2023-10-18T10:43:04.563741+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:43:04.563741+0000","last_undegraded":"2023-10-18T10:43:04.563741+0000","last_fullsized":"2023-10-18T10:43:04.563741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575068,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59692,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312148,"statfs":{"total":96636764160,"available":96575639552,"internally_reserved":0,"allocated":2187264,"data_stored":1388162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.58299999999999996},"min":{"1min":0.36099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.754,"5min":0.876,"15min":0.88300000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63200000000000001,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70299999999999996,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.60299999999999998},"min":{"1min":0.42199999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.76000000000000001,"5min":0.79800000000000004,"15min":0.85799999999999998},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.68899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.315},"max":{"1min":0.78500000000000003,"5min":0.875,"15min":0.875},"last":0.504}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575068,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":799850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82599999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.69499999999999995,"5min":0.77700000000000002,"15min":0.82699999999999996},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65800000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.432}]},{"osd":3,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.78400000000000003,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.748,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.80300000000000005,"5min":0.82199999999999995,"15min":0.87},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":797634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.39600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88400000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81399999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.81999999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.41199999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.47599999999999998},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71599999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.375},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.496},"min":{"1min":0.373,"5min":0.31,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.38200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77200000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.312,"15min":0.31},"max":{"1min":0.84299999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.443},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.44700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.871,"15min":0.871},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32680,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1523712,"data_stored":765110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57699999999999996,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.77000000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.54700000000000004,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.753,"15min":0.877},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.89300000000000002},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73499999999999999,"5min":0.873,"15min":0.873},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71899999999999997,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.78200000000000003},"last":0.62}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.33200000000000002,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.79200000000000004,"5min":1.446,"15min":1.446},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.81999999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.879,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.52800000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.499,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.624,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.59299999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53400000000000003,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.379},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.434,"15min":0.45100000000000001},"min":{"1min":0.28599999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.80200000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.879,"15min":0.94699999999999995},"last":0.5}]},{"osd":5,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":1.002,"15min":1.002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88900000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.629,"15min":0.63800000000000001},"min":{"1min":0.434,"5min":0.38,"15min":0.37},"max":{"1min":0.82499999999999996,"5min":1.018,"15min":1.018},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.751,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.63900000000000001},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.627,"15min":0.65300000000000002},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54940,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316900,"statfs":{"total":96636764160,"available":96580505600,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.371,"15min":0.374},"min":{"1min":0.252,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.626},"last":0.377}]},{"osd":1,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29899999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.57899999999999996,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.315}]},{"osd":3,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.24299999999999999},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.434,"15min":0.44700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.41899999999999998,"15min":0.254},"max":{"1min":0.86499999999999999,"5min":0.88200000000000001,"15min":0.88800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.625,"15min":0.64500000000000002},"min":{"1min":0.45300000000000001,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.95699999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.63},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.93700000000000006},"last":0.57299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.84599999999999997,"15min":0.92500000000000004},"last":0.54100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36199999999999999,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.48999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.32000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.39200000000000002},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.374},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.42099999999999999},"min":{"1min":0.314,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53600000000000003,"5min":0.627,"15min":0.64800000000000002},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.33100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59499999999999997,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.33500000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.83999999999999997,"15min":0.95599999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.623},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.753,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.88100000000000001,"15min":0.95999999999999996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60199999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.76200000000000001,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.63700000000000001},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.78000000000000003,"5min":1.03,"15min":1.03},"last":0.749}]},{"osd":7,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.63},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78800000000000003,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575067,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2195456,"data_stored":1389704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.39000000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.54600000000000004,"5min":0.62,"15min":0.76900000000000002},"last":0.214}]},{"osd":2,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.187},"max":{"1min":0.58099999999999996,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.41999999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.29299999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.39500000000000002},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56599999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.312}]},{"osd":4,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.61699999999999999},"min":{"1min":0.372,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81999999999999995,"5min":0.996,"15min":0.996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90300000000000002,"5min":1.038,"15min":1.038},"last":0.34200000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.89200000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93700000000000006,"5min":1.026,"15min":1.3700000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88100000000000001,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.64300000000000002},"min":{"1min":0.41199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.05,"15min":1.339},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.64600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:43:06.920 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:43:07.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.054+0000 7f5ac567c640 1 -- 172.21.15.125:0/951660272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00d1970 msgr2=0x7f5ac00ae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.054+0000 7f5ac567c640 1 --2- 172.21.15.125:0/951660272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00d1970 0x7f5ac00ae1b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005fd0 tx=0x7f5ab802c2e0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.054+0000 7f5ac567c640 1 -- 172.21.15.125:0/951660272 shutdown_connections 2023-10-18T10:43:07.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.054+0000 7f5ac567c640 1 --2- 172.21.15.125:0/951660272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00d1970 0x7f5ac00ae1b0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.054+0000 7f5ac567c640 1 --2- 172.21.15.125:0/951660272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ac00cf710 0x7f5ac00ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.054+0000 7f5ac567c640 1 -- 172.21.15.125:0/951660272 >> 172.21.15.125:0/951660272 conn(0x7f5ac00d0d70 msgr2=0x7f5ac00d1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:07.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.054+0000 7f5ac567c640 1 -- 172.21.15.125:0/951660272 shutdown_connections 2023-10-18T10:43:07.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.054+0000 7f5ac567c640 1 -- 172.21.15.125:0/951660272 wait complete. 2023-10-18T10:43:07.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5ac567c640 1 Processor -- start 2023-10-18T10:43:07.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5ac567c640 1 -- start start 2023-10-18T10:43:07.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5ac567c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00cf710 0x7f5ac01f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5ac567c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ac00d1970 0x7f5ac0157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5ac567c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ac015b0f0 0x7f5ac015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5ac567c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01f1180 con 0x7f5ac00cf710 2023-10-18T10:43:07.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5ac567c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01f5660 con 0x7f5ac015b0f0 2023-10-18T10:43:07.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5ac567c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01f20d0 con 0x7f5ac00d1970 2023-10-18T10:43:07.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5abeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00cf710 0x7f5ac01f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5abeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00cf710 0x7f5ac01f1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49034/0 (socket says 172.21.15.125:49034) 2023-10-18T10:43:07.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.056+0000 7f5abeffd640 1 -- 172.21.15.125:0/4104144106 learned_addr learned my addr 172.21.15.125:0/4104144106 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:07.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5abe7fc640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ac00d1970 0x7f5ac0157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5abeffd640 1 -- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ac00d1970 msgr2=0x7f5ac0157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5abeffd640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ac00d1970 0x7f5ac0157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5abeffd640 1 -- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ac015b0f0 msgr2=0x7f5ac015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5abeffd640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ac015b0f0 0x7f5ac015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5abeffd640 1 -- 172.21.15.125:0/4104144106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab8009900 con 0x7f5ac00cf710 2023-10-18T10:43:07.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5abeffd640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00cf710 0x7f5ac01f1b90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c440 tx=0x7f5ab000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:07.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5aa77fe640 1 -- 172.21.15.125:0/4104144106 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0005480 con 0x7f5ac00cf710 2023-10-18T10:43:07.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac015d3e0 con 0x7f5ac00cf710 2023-10-18T10:43:07.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5aa77fe640 1 -- 172.21.15.125:0/4104144106 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ab00035f0 con 0x7f5ac00cf710 2023-10-18T10:43:07.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5aa77fe640 1 -- 172.21.15.125:0/4104144106 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0003a00 con 0x7f5ac00cf710 2023-10-18T10:43:07.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.057+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac015d9a0 con 0x7f5ac00cf710 2023-10-18T10:43:07.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.058+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac015dbc0 con 0x7f5ac00cf710 2023-10-18T10:43:07.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.058+0000 7f5aa77fe640 1 -- 172.21.15.125:0/4104144106 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5ab0003b60 con 0x7f5ac00cf710 2023-10-18T10:43:07.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.059+0000 7f5aa77fe640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5aac06af60 0x7f5aac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.059+0000 7f5aa77fe640 1 -- 172.21.15.125:0/4104144106 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5ab00890d0 con 0x7f5ac00cf710 2023-10-18T10:43:07.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.059+0000 7f5abe7fc640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5aac06af60 0x7f5aac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.060+0000 7f5abe7fc640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5aac06af60 0x7f5aac06d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ac015ce30 tx=0x7f5ab4005c60 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:07.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.063+0000 7f5aa77fe640 1 -- 172.21.15.125:0/4104144106 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ab0057940 con 0x7f5ac00cf710 2023-10-18T10:43:07.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.227+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 --> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ac00b5aa0 con 0x7f5aac06af60 2023-10-18T10:43:07.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.235+0000 7f5aa77fe640 1 -- 172.21.15.125:0/4104144106 <== mgr.7360 v2:172.21.15.139:6832/1511464628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287849 (secure 0 0 0) 0x7f5ac00b5aa0 con 0x7f5aac06af60 2023-10-18T10:43:07.240 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:07.241 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:43:07.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.238+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5aac06af60 msgr2=0x7f5aac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.238+0000 7f5ac567c640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5aac06af60 0x7f5aac06d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ac015ce30 tx=0x7f5ab4005c60 comp rx=0 tx=0).stop 2023-10-18T10:43:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.238+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00cf710 msgr2=0x7f5ac01f1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.238+0000 7f5ac567c640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00cf710 0x7f5ac01f1b90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c440 tx=0x7f5ab000c900 comp rx=0 tx=0).stop 2023-10-18T10:43:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.239+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 shutdown_connections 2023-10-18T10:43:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.239+0000 7f5ac567c640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f5aac06af60 0x7f5aac06d420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.239+0000 7f5ac567c640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ac015b0f0 0x7f5ac015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.239+0000 7f5ac567c640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ac00d1970 0x7f5ac0157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.239+0000 7f5ac567c640 1 --2- 172.21.15.125:0/4104144106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ac00cf710 0x7f5ac01f1b90 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.239+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 >> 172.21.15.125:0/4104144106 conn(0x7f5ac00d0d70 msgr2=0x7f5ac015a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.239+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 shutdown_connections 2023-10-18T10:43:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.239+0000 7f5ac567c640 1 -- 172.21.15.125:0/4104144106 wait complete. 2023-10-18T10:43:07.251 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:43:05.899805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":364,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":315772,"kb_used_data":13540,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754658948,"statfs":{"total":773094113280,"available":772770762752,"internally_reserved":0,"allocated":13864960,"data_stored":7534012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001056"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564947+0000","last_change":"2023-10-18T10:43:03.420057+0000","last_active":"2023-10-18T10:43:04.564947+0000","last_peered":"2023-10-18T10:43:04.564947+0000","last_clean":"2023-10-18T10:43:04.564947+0000","last_became_active":"2023-10-18T10:43:03.419809+0000","last_became_peered":"2023-10-18T10:43:03.419809+0000","last_unstale":"2023-10-18T10:43:04.564947+0000","last_undegraded":"2023-10-18T10:43:04.564947+0000","last_fullsized":"2023-10-18T10:43:04.564947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:51.310959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565191+0000","last_change":"2023-10-18T10:43:03.422800+0000","last_active":"2023-10-18T10:43:04.565191+0000","last_peered":"2023-10-18T10:43:04.565191+0000","last_clean":"2023-10-18T10:43:04.565191+0000","last_became_active":"2023-10-18T10:43:03.422037+0000","last_became_peered":"2023-10-18T10:43:03.422037+0000","last_unstale":"2023-10-18T10:43:04.565191+0000","last_undegraded":"2023-10-18T10:43:04.565191+0000","last_fullsized":"2023-10-18T10:43:04.565191+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:03.106690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564044+0000","last_change":"2023-10-18T10:43:03.424158+0000","last_active":"2023-10-18T10:43:04.564044+0000","last_peered":"2023-10-18T10:43:04.564044+0000","last_clean":"2023-10-18T10:43:04.564044+0000","last_became_active":"2023-10-18T10:43:03.423915+0000","last_became_peered":"2023-10-18T10:43:03.423915+0000","last_unstale":"2023-10-18T10:43:04.564044+0000","last_undegraded":"2023-10-18T10:43:04.564044+0000","last_fullsized":"2023-10-18T10:43:04.564044+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:49.732624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408937+0000","last_change":"2023-10-18T10:43:03.415009+0000","last_active":"2023-10-18T10:43:04.408937+0000","last_peered":"2023-10-18T10:43:04.408937+0000","last_clean":"2023-10-18T10:43:04.408937+0000","last_became_active":"2023-10-18T10:43:03.414710+0000","last_became_peered":"2023-10-18T10:43:03.414710+0000","last_unstale":"2023-10-18T10:43:04.408937+0000","last_undegraded":"2023-10-18T10:43:04.408937+0000","last_fullsized":"2023-10-18T10:43:04.408937+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:21.227528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563635+0000","last_change":"2023-10-18T10:43:03.411644+0000","last_active":"2023-10-18T10:43:04.563635+0000","last_peered":"2023-10-18T10:43:04.563635+0000","last_clean":"2023-10-18T10:43:04.563635+0000","last_became_active":"2023-10-18T10:43:03.411381+0000","last_became_peered":"2023-10-18T10:43:03.411381+0000","last_unstale":"2023-10-18T10:43:04.563635+0000","last_undegraded":"2023-10-18T10:43:04.563635+0000","last_fullsized":"2023-10-18T10:43:04.563635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:28.704370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564515+0000","last_change":"2023-10-18T10:43:03.420380+0000","last_active":"2023-10-18T10:43:04.564515+0000","last_peered":"2023-10-18T10:43:04.564515+0000","last_clean":"2023-10-18T10:43:04.564515+0000","last_became_active":"2023-10-18T10:43:03.420112+0000","last_became_peered":"2023-10-18T10:43:03.420112+0000","last_unstale":"2023-10-18T10:43:04.564515+0000","last_undegraded":"2023-10-18T10:43:04.564515+0000","last_fullsized":"2023-10-18T10:43:04.564515+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:15.468100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564645+0000","last_change":"2023-10-18T10:43:03.417853+0000","last_active":"2023-10-18T10:43:04.564645+0000","last_peered":"2023-10-18T10:43:04.564645+0000","last_clean":"2023-10-18T10:43:04.564645+0000","last_became_active":"2023-10-18T10:43:03.417585+0000","last_became_peered":"2023-10-18T10:43:03.417585+0000","last_unstale":"2023-10-18T10:43:04.564645+0000","last_undegraded":"2023-10-18T10:43:04.564645+0000","last_fullsized":"2023-10-18T10:43:04.564645+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:57.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563425+0000","last_change":"2023-10-18T10:43:03.420896+0000","last_active":"2023-10-18T10:43:04.563425+0000","last_peered":"2023-10-18T10:43:04.563425+0000","last_clean":"2023-10-18T10:43:04.563425+0000","last_became_active":"2023-10-18T10:43:03.420633+0000","last_became_peered":"2023-10-18T10:43:03.420633+0000","last_unstale":"2023-10-18T10:43:04.563425+0000","last_undegraded":"2023-10-18T10:43:04.563425+0000","last_fullsized":"2023-10-18T10:43:04.563425+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:54.658345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564188+0000","last_change":"2023-10-18T10:43:03.423222+0000","last_active":"2023-10-18T10:43:04.564188+0000","last_peered":"2023-10-18T10:43:04.564188+0000","last_clean":"2023-10-18T10:43:04.564188+0000","last_became_active":"2023-10-18T10:43:03.422095+0000","last_became_peered":"2023-10-18T10:43:03.422095+0000","last_unstale":"2023-10-18T10:43:04.564188+0000","last_undegraded":"2023-10-18T10:43:04.564188+0000","last_fullsized":"2023-10-18T10:43:04.564188+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:36.033906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564111+0000","last_change":"2023-10-18T10:43:03.420436+0000","last_active":"2023-10-18T10:43:04.564111+0000","last_peered":"2023-10-18T10:43:04.564111+0000","last_clean":"2023-10-18T10:43:04.564111+0000","last_became_active":"2023-10-18T10:43:03.419414+0000","last_became_peered":"2023-10-18T10:43:03.419414+0000","last_unstale":"2023-10-18T10:43:04.564111+0000","last_undegraded":"2023-10-18T10:43:04.564111+0000","last_fullsized":"2023-10-18T10:43:04.564111+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:27.017090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564170+0000","last_change":"2023-10-18T10:43:03.425870+0000","last_active":"2023-10-18T10:43:04.564170+0000","last_peered":"2023-10-18T10:43:04.564170+0000","last_clean":"2023-10-18T10:43:04.564170+0000","last_became_active":"2023-10-18T10:43:03.425583+0000","last_became_peered":"2023-10-18T10:43:03.425583+0000","last_unstale":"2023-10-18T10:43:04.564170+0000","last_undegraded":"2023-10-18T10:43:04.564170+0000","last_fullsized":"2023-10-18T10:43:04.564170+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:07.373278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408816+0000","last_change":"2023-10-18T10:43:03.421921+0000","last_active":"2023-10-18T10:43:04.408816+0000","last_peered":"2023-10-18T10:43:04.408816+0000","last_clean":"2023-10-18T10:43:04.408816+0000","last_became_active":"2023-10-18T10:43:03.421644+0000","last_became_peered":"2023-10-18T10:43:03.421644+0000","last_unstale":"2023-10-18T10:43:04.408816+0000","last_undegraded":"2023-10-18T10:43:04.408816+0000","last_fullsized":"2023-10-18T10:43:04.408816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:58.741901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563868+0000","last_change":"2023-10-18T10:43:03.415137+0000","last_active":"2023-10-18T10:43:04.563868+0000","last_peered":"2023-10-18T10:43:04.563868+0000","last_clean":"2023-10-18T10:43:04.563868+0000","last_became_active":"2023-10-18T10:43:03.414871+0000","last_became_peered":"2023-10-18T10:43:03.414871+0000","last_unstale":"2023-10-18T10:43:04.563868+0000","last_undegraded":"2023-10-18T10:43:04.563868+0000","last_fullsized":"2023-10-18T10:43:04.563868+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:33.182433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563789+0000","last_change":"2023-10-18T10:43:03.420511+0000","last_active":"2023-10-18T10:43:04.563789+0000","last_peered":"2023-10-18T10:43:04.563789+0000","last_clean":"2023-10-18T10:43:04.563789+0000","last_became_active":"2023-10-18T10:43:03.420241+0000","last_became_peered":"2023-10-18T10:43:03.420241+0000","last_unstale":"2023-10-18T10:43:04.563789+0000","last_undegraded":"2023-10-18T10:43:04.563789+0000","last_fullsized":"2023-10-18T10:43:04.563789+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:38.074311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563696+0000","last_change":"2023-10-18T10:43:03.417552+0000","last_active":"2023-10-18T10:43:04.563696+0000","last_peered":"2023-10-18T10:43:04.563696+0000","last_clean":"2023-10-18T10:43:04.563696+0000","last_became_active":"2023-10-18T10:43:03.417291+0000","last_became_peered":"2023-10-18T10:43:03.417291+0000","last_unstale":"2023-10-18T10:43:04.563696+0000","last_undegraded":"2023-10-18T10:43:04.563696+0000","last_fullsized":"2023-10-18T10:43:04.563696+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:18.941147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.425285+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.424971+0000","last_became_peered":"2023-10-18T10:43:03.424971+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:00.092514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565204+0000","last_change":"2023-10-18T10:43:03.416033+0000","last_active":"2023-10-18T10:43:04.565204+0000","last_peered":"2023-10-18T10:43:04.565204+0000","last_clean":"2023-10-18T10:43:04.565204+0000","last_became_active":"2023-10-18T10:43:03.415763+0000","last_became_peered":"2023-10-18T10:43:03.415763+0000","last_unstale":"2023-10-18T10:43:04.565204+0000","last_undegraded":"2023-10-18T10:43:04.565204+0000","last_fullsized":"2023-10-18T10:43:04.565204+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:36.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564171+0000","last_change":"2023-10-18T10:43:03.418626+0000","last_active":"2023-10-18T10:43:04.564171+0000","last_peered":"2023-10-18T10:43:04.564171+0000","last_clean":"2023-10-18T10:43:04.564171+0000","last_became_active":"2023-10-18T10:43:03.418344+0000","last_became_peered":"2023-10-18T10:43:03.418344+0000","last_unstale":"2023-10-18T10:43:04.564171+0000","last_undegraded":"2023-10-18T10:43:04.564171+0000","last_fullsized":"2023-10-18T10:43:04.564171+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:55.314298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564184+0000","last_change":"2023-10-18T10:43:03.421228+0000","last_active":"2023-10-18T10:43:04.564184+0000","last_peered":"2023-10-18T10:43:04.564184+0000","last_clean":"2023-10-18T10:43:04.564184+0000","last_became_active":"2023-10-18T10:43:03.420908+0000","last_became_peered":"2023-10-18T10:43:03.420908+0000","last_unstale":"2023-10-18T10:43:04.564184+0000","last_undegraded":"2023-10-18T10:43:04.564184+0000","last_fullsized":"2023-10-18T10:43:04.564184+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:19.745476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564166+0000","last_change":"2023-10-18T10:43:03.421765+0000","last_active":"2023-10-18T10:43:04.564166+0000","last_peered":"2023-10-18T10:43:04.564166+0000","last_clean":"2023-10-18T10:43:04.564166+0000","last_became_active":"2023-10-18T10:43:03.421501+0000","last_became_peered":"2023-10-18T10:43:03.421501+0000","last_unstale":"2023-10-18T10:43:04.564166+0000","last_undegraded":"2023-10-18T10:43:04.564166+0000","last_fullsized":"2023-10-18T10:43:04.564166+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:06.267958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.424152+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.423890+0000","last_became_peered":"2023-10-18T10:43:03.423890+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:26.100932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564263+0000","last_change":"2023-10-18T10:43:03.417440+0000","last_active":"2023-10-18T10:43:04.564263+0000","last_peered":"2023-10-18T10:43:04.564263+0000","last_clean":"2023-10-18T10:43:04.564263+0000","last_became_active":"2023-10-18T10:43:03.417201+0000","last_became_peered":"2023-10-18T10:43:03.417201+0000","last_unstale":"2023-10-18T10:43:04.564263+0000","last_undegraded":"2023-10-18T10:43:04.564263+0000","last_fullsized":"2023-10-18T10:43:04.564263+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:16.834481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564364+0000","last_change":"2023-10-18T10:43:03.419666+0000","last_active":"2023-10-18T10:43:04.564364+0000","last_peered":"2023-10-18T10:43:04.564364+0000","last_clean":"2023-10-18T10:43:04.564364+0000","last_became_active":"2023-10-18T10:43:03.418843+0000","last_became_peered":"2023-10-18T10:43:03.418843+0000","last_unstale":"2023-10-18T10:43:04.564364+0000","last_undegraded":"2023-10-18T10:43:04.564364+0000","last_fullsized":"2023-10-18T10:43:04.564364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:25.903261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565004+0000","last_change":"2023-10-18T10:43:03.423183+0000","last_active":"2023-10-18T10:43:04.565004+0000","last_peered":"2023-10-18T10:43:04.565004+0000","last_clean":"2023-10-18T10:43:04.565004+0000","last_became_active":"2023-10-18T10:43:03.422819+0000","last_became_peered":"2023-10-18T10:43:03.422819+0000","last_unstale":"2023-10-18T10:43:04.565004+0000","last_undegraded":"2023-10-18T10:43:04.565004+0000","last_fullsized":"2023-10-18T10:43:04.565004+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:34.324413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564301+0000","last_change":"2023-10-18T10:43:03.417982+0000","last_active":"2023-10-18T10:43:04.564301+0000","last_peered":"2023-10-18T10:43:04.564301+0000","last_clean":"2023-10-18T10:43:04.564301+0000","last_became_active":"2023-10-18T10:43:03.417736+0000","last_became_peered":"2023-10-18T10:43:03.417736+0000","last_unstale":"2023-10-18T10:43:04.564301+0000","last_undegraded":"2023-10-18T10:43:04.564301+0000","last_fullsized":"2023-10-18T10:43:04.564301+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:51.640858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564774+0000","last_change":"2023-10-18T10:43:03.416816+0000","last_active":"2023-10-18T10:43:04.564774+0000","last_peered":"2023-10-18T10:43:04.564774+0000","last_clean":"2023-10-18T10:43:04.564774+0000","last_became_active":"2023-10-18T10:43:03.416441+0000","last_became_peered":"2023-10-18T10:43:03.416441+0000","last_unstale":"2023-10-18T10:43:04.564774+0000","last_undegraded":"2023-10-18T10:43:04.564774+0000","last_fullsized":"2023-10-18T10:43:04.564774+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:23.903407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563541+0000","last_change":"2023-10-18T10:43:03.423095+0000","last_active":"2023-10-18T10:43:04.563541+0000","last_peered":"2023-10-18T10:43:04.563541+0000","last_clean":"2023-10-18T10:43:04.563541+0000","last_became_active":"2023-10-18T10:43:03.422840+0000","last_became_peered":"2023-10-18T10:43:03.422840+0000","last_unstale":"2023-10-18T10:43:04.563541+0000","last_undegraded":"2023-10-18T10:43:04.563541+0000","last_fullsized":"2023-10-18T10:43:04.563541+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:09.585007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564675+0000","last_change":"2023-10-18T10:43:03.423114+0000","last_active":"2023-10-18T10:43:04.564675+0000","last_peered":"2023-10-18T10:43:04.564675+0000","last_clean":"2023-10-18T10:43:04.564675+0000","last_became_active":"2023-10-18T10:43:03.422862+0000","last_became_peered":"2023-10-18T10:43:03.422862+0000","last_unstale":"2023-10-18T10:43:04.564675+0000","last_undegraded":"2023-10-18T10:43:04.564675+0000","last_fullsized":"2023-10-18T10:43:04.564675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:40:54.685089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"127'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408994+0000","last_change":"2023-10-18T10:43:02.963797+0000","last_active":"2023-10-18T10:43:04.408994+0000","last_peered":"2023-10-18T10:43:04.408994+0000","last_clean":"2023-10-18T10:43:04.408994+0000","last_became_active":"2023-10-18T10:43:02.963525+0000","last_became_peered":"2023-10-18T10:43:02.963525+0000","last_unstale":"2023-10-18T10:43:04.408994+0000","last_undegraded":"2023-10-18T10:43:04.408994+0000","last_fullsized":"2023-10-18T10:43:04.408994+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:18:22.535089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563557+0000","last_change":"2023-10-18T10:43:03.419783+0000","last_active":"2023-10-18T10:43:04.563557+0000","last_peered":"2023-10-18T10:43:04.563557+0000","last_clean":"2023-10-18T10:43:04.563557+0000","last_became_active":"2023-10-18T10:43:03.419543+0000","last_became_peered":"2023-10-18T10:43:03.419543+0000","last_unstale":"2023-10-18T10:43:04.563557+0000","last_undegraded":"2023-10-18T10:43:04.563557+0000","last_fullsized":"2023-10-18T10:43:04.563557+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:39.668528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564617+0000","last_change":"2023-10-18T10:43:02.965235+0000","last_active":"2023-10-18T10:43:04.564617+0000","last_peered":"2023-10-18T10:43:04.564617+0000","last_clean":"2023-10-18T10:43:04.564617+0000","last_became_active":"2023-10-18T10:43:02.964880+0000","last_became_peered":"2023-10-18T10:43:02.964880+0000","last_unstale":"2023-10-18T10:43:04.564617+0000","last_undegraded":"2023-10-18T10:43:04.564617+0000","last_fullsized":"2023-10-18T10:43:04.564617+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:48.610823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564182+0000","last_change":"2023-10-18T10:43:03.416904+0000","last_active":"2023-10-18T10:43:04.564182+0000","last_peered":"2023-10-18T10:43:04.564182+0000","last_clean":"2023-10-18T10:43:04.564182+0000","last_became_active":"2023-10-18T10:43:03.416489+0000","last_became_peered":"2023-10-18T10:43:03.416489+0000","last_unstale":"2023-10-18T10:43:04.564182+0000","last_undegraded":"2023-10-18T10:43:04.564182+0000","last_fullsized":"2023-10-18T10:43:04.564182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:36.136092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564337+0000","last_change":"2023-10-18T10:43:02.969842+0000","last_active":"2023-10-18T10:43:04.564337+0000","last_peered":"2023-10-18T10:43:04.564337+0000","last_clean":"2023-10-18T10:43:04.564337+0000","last_became_active":"2023-10-18T10:43:02.969620+0000","last_became_peered":"2023-10-18T10:43:02.969620+0000","last_unstale":"2023-10-18T10:43:04.564337+0000","last_undegraded":"2023-10-18T10:43:04.564337+0000","last_fullsized":"2023-10-18T10:43:04.564337+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:23:14.364024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409001+0000","last_change":"2023-10-18T10:43:03.417690+0000","last_active":"2023-10-18T10:43:04.409001+0000","last_peered":"2023-10-18T10:43:04.409001+0000","last_clean":"2023-10-18T10:43:04.409001+0000","last_became_active":"2023-10-18T10:43:03.417452+0000","last_became_peered":"2023-10-18T10:43:03.417452+0000","last_unstale":"2023-10-18T10:43:04.409001+0000","last_undegraded":"2023-10-18T10:43:04.409001+0000","last_fullsized":"2023-10-18T10:43:04.409001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:28.022888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563896+0000","last_change":"2023-10-18T10:43:02.965103+0000","last_active":"2023-10-18T10:43:04.563896+0000","last_peered":"2023-10-18T10:43:04.563896+0000","last_clean":"2023-10-18T10:43:04.563896+0000","last_became_active":"2023-10-18T10:43:02.964812+0000","last_became_peered":"2023-10-18T10:43:02.964812+0000","last_unstale":"2023-10-18T10:43:04.563896+0000","last_undegraded":"2023-10-18T10:43:04.563896+0000","last_fullsized":"2023-10-18T10:43:04.563896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:08.011562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564304+0000","last_change":"2023-10-18T10:43:03.424682+0000","last_active":"2023-10-18T10:43:04.564304+0000","last_peered":"2023-10-18T10:43:04.564304+0000","last_clean":"2023-10-18T10:43:04.564304+0000","last_became_active":"2023-10-18T10:43:03.424416+0000","last_became_peered":"2023-10-18T10:43:03.424416+0000","last_unstale":"2023-10-18T10:43:04.564304+0000","last_undegraded":"2023-10-18T10:43:04.564304+0000","last_fullsized":"2023-10-18T10:43:04.564304+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:29.729372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564479+0000","last_change":"2023-10-18T10:43:02.966346+0000","last_active":"2023-10-18T10:43:04.564479+0000","last_peered":"2023-10-18T10:43:04.564479+0000","last_clean":"2023-10-18T10:43:04.564479+0000","last_became_active":"2023-10-18T10:43:02.966041+0000","last_became_peered":"2023-10-18T10:43:02.966041+0000","last_unstale":"2023-10-18T10:43:04.564479+0000","last_undegraded":"2023-10-18T10:43:04.564479+0000","last_fullsized":"2023-10-18T10:43:04.564479+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:04.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565269+0000","last_change":"2023-10-18T10:43:03.424161+0000","last_active":"2023-10-18T10:43:04.565269+0000","last_peered":"2023-10-18T10:43:04.565269+0000","last_clean":"2023-10-18T10:43:04.565269+0000","last_became_active":"2023-10-18T10:43:03.423854+0000","last_became_peered":"2023-10-18T10:43:03.423854+0000","last_unstale":"2023-10-18T10:43:04.565269+0000","last_undegraded":"2023-10-18T10:43:04.565269+0000","last_fullsized":"2023-10-18T10:43:04.565269+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:24.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409024+0000","last_change":"2023-10-18T10:43:03.417182+0000","last_active":"2023-10-18T10:43:04.409024+0000","last_peered":"2023-10-18T10:43:04.409024+0000","last_clean":"2023-10-18T10:43:04.409024+0000","last_became_active":"2023-10-18T10:43:03.416917+0000","last_became_peered":"2023-10-18T10:43:03.416917+0000","last_unstale":"2023-10-18T10:43:04.409024+0000","last_undegraded":"2023-10-18T10:43:04.409024+0000","last_fullsized":"2023-10-18T10:43:04.409024+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:49.418978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408682+0000","last_change":"2023-10-18T10:43:02.969298+0000","last_active":"2023-10-18T10:43:04.408682+0000","last_peered":"2023-10-18T10:43:04.408682+0000","last_clean":"2023-10-18T10:43:04.408682+0000","last_became_active":"2023-10-18T10:43:02.969045+0000","last_became_peered":"2023-10-18T10:43:02.969045+0000","last_unstale":"2023-10-18T10:43:04.408682+0000","last_undegraded":"2023-10-18T10:43:04.408682+0000","last_fullsized":"2023-10-18T10:43:04.408682+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:24.897688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564560+0000","last_change":"2023-10-18T10:43:03.424618+0000","last_active":"2023-10-18T10:43:04.564560+0000","last_peered":"2023-10-18T10:43:04.564560+0000","last_clean":"2023-10-18T10:43:04.564560+0000","last_became_active":"2023-10-18T10:43:03.424369+0000","last_became_peered":"2023-10-18T10:43:03.424369+0000","last_unstale":"2023-10-18T10:43:04.564560+0000","last_undegraded":"2023-10-18T10:43:04.564560+0000","last_fullsized":"2023-10-18T10:43:04.564560+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:25.195869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563691+0000","last_change":"2023-10-18T10:43:03.414895+0000","last_active":"2023-10-18T10:43:04.563691+0000","last_peered":"2023-10-18T10:43:04.563691+0000","last_clean":"2023-10-18T10:43:04.563691+0000","last_became_active":"2023-10-18T10:43:03.414661+0000","last_became_peered":"2023-10-18T10:43:03.414661+0000","last_unstale":"2023-10-18T10:43:04.563691+0000","last_undegraded":"2023-10-18T10:43:04.563691+0000","last_fullsized":"2023-10-18T10:43:04.563691+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:46.699483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564739+0000","last_change":"2023-10-18T10:43:02.971336+0000","last_active":"2023-10-18T10:43:04.564739+0000","last_peered":"2023-10-18T10:43:04.564739+0000","last_clean":"2023-10-18T10:43:04.564739+0000","last_became_active":"2023-10-18T10:43:02.971030+0000","last_became_peered":"2023-10-18T10:43:02.971030+0000","last_unstale":"2023-10-18T10:43:04.564739+0000","last_undegraded":"2023-10-18T10:43:04.564739+0000","last_fullsized":"2023-10-18T10:43:04.564739+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:32:47.348309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563840+0000","last_change":"2023-10-18T10:43:03.420254+0000","last_active":"2023-10-18T10:43:04.563840+0000","last_peered":"2023-10-18T10:43:04.563840+0000","last_clean":"2023-10-18T10:43:04.563840+0000","last_became_active":"2023-10-18T10:43:03.420019+0000","last_became_peered":"2023-10-18T10:43:03.420019+0000","last_unstale":"2023-10-18T10:43:04.563840+0000","last_undegraded":"2023-10-18T10:43:04.563840+0000","last_fullsized":"2023-10-18T10:43:04.563840+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:02.531521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"127'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564189+0000","last_change":"2023-10-18T10:43:02.974197+0000","last_active":"2023-10-18T10:43:04.564189+0000","last_peered":"2023-10-18T10:43:04.564189+0000","last_clean":"2023-10-18T10:43:04.564189+0000","last_became_active":"2023-10-18T10:43:02.973961+0000","last_became_peered":"2023-10-18T10:43:02.973961+0000","last_unstale":"2023-10-18T10:43:04.564189+0000","last_undegraded":"2023-10-18T10:43:04.564189+0000","last_fullsized":"2023-10-18T10:43:04.564189+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:57.736328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564658+0000","last_change":"2023-10-18T10:43:03.417830+0000","last_active":"2023-10-18T10:43:04.564658+0000","last_peered":"2023-10-18T10:43:04.564658+0000","last_clean":"2023-10-18T10:43:04.564658+0000","last_became_active":"2023-10-18T10:43:03.417031+0000","last_became_peered":"2023-10-18T10:43:03.417031+0000","last_unstale":"2023-10-18T10:43:04.564658+0000","last_undegraded":"2023-10-18T10:43:04.564658+0000","last_fullsized":"2023-10-18T10:43:04.564658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:56.478327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"127'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565099+0000","last_change":"2023-10-18T10:43:02.959201+0000","last_active":"2023-10-18T10:43:04.565099+0000","last_peered":"2023-10-18T10:43:04.565099+0000","last_clean":"2023-10-18T10:43:04.565099+0000","last_became_active":"2023-10-18T10:43:02.958888+0000","last_became_peered":"2023-10-18T10:43:02.958888+0000","last_unstale":"2023-10-18T10:43:04.565099+0000","last_undegraded":"2023-10-18T10:43:04.565099+0000","last_fullsized":"2023-10-18T10:43:04.565099+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:04.257834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564597+0000","last_change":"2023-10-18T10:43:03.419410+0000","last_active":"2023-10-18T10:43:04.564597+0000","last_peered":"2023-10-18T10:43:04.564597+0000","last_clean":"2023-10-18T10:43:04.564597+0000","last_became_active":"2023-10-18T10:43:03.419055+0000","last_became_peered":"2023-10-18T10:43:03.419055+0000","last_unstale":"2023-10-18T10:43:04.564597+0000","last_undegraded":"2023-10-18T10:43:04.564597+0000","last_fullsized":"2023-10-18T10:43:04.564597+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:06.895973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564742+0000","last_change":"2023-10-18T10:43:02.972054+0000","last_active":"2023-10-18T10:43:04.564742+0000","last_peered":"2023-10-18T10:43:04.564742+0000","last_clean":"2023-10-18T10:43:04.564742+0000","last_became_active":"2023-10-18T10:43:02.971783+0000","last_became_peered":"2023-10-18T10:43:02.971783+0000","last_unstale":"2023-10-18T10:43:04.564742+0000","last_undegraded":"2023-10-18T10:43:04.564742+0000","last_fullsized":"2023-10-18T10:43:04.564742+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:07.914251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565070+0000","last_change":"2023-10-18T10:43:03.424942+0000","last_active":"2023-10-18T10:43:04.565070+0000","last_peered":"2023-10-18T10:43:04.565070+0000","last_clean":"2023-10-18T10:43:04.565070+0000","last_became_active":"2023-10-18T10:43:03.424610+0000","last_became_peered":"2023-10-18T10:43:03.424610+0000","last_unstale":"2023-10-18T10:43:04.565070+0000","last_undegraded":"2023-10-18T10:43:04.565070+0000","last_fullsized":"2023-10-18T10:43:04.565070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:58.441002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408796+0000","last_change":"2023-10-18T10:43:02.962770+0000","last_active":"2023-10-18T10:43:04.408796+0000","last_peered":"2023-10-18T10:43:04.408796+0000","last_clean":"2023-10-18T10:43:04.408796+0000","last_became_active":"2023-10-18T10:43:02.962492+0000","last_became_peered":"2023-10-18T10:43:02.962492+0000","last_unstale":"2023-10-18T10:43:04.408796+0000","last_undegraded":"2023-10-18T10:43:04.408796+0000","last_fullsized":"2023-10-18T10:43:04.408796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:04.206263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565182+0000","last_change":"2023-10-18T10:43:03.417626+0000","last_active":"2023-10-18T10:43:04.565182+0000","last_peered":"2023-10-18T10:43:04.565182+0000","last_clean":"2023-10-18T10:43:04.565182+0000","last_became_active":"2023-10-18T10:43:03.417374+0000","last_became_peered":"2023-10-18T10:43:03.417374+0000","last_unstale":"2023-10-18T10:43:04.565182+0000","last_undegraded":"2023-10-18T10:43:04.565182+0000","last_fullsized":"2023-10-18T10:43:04.565182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:39.656533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563634+0000","last_change":"2023-10-18T10:43:02.966586+0000","last_active":"2023-10-18T10:43:04.563634+0000","last_peered":"2023-10-18T10:43:04.563634+0000","last_clean":"2023-10-18T10:43:04.563634+0000","last_became_active":"2023-10-18T10:43:02.966340+0000","last_became_peered":"2023-10-18T10:43:02.966340+0000","last_unstale":"2023-10-18T10:43:04.563634+0000","last_undegraded":"2023-10-18T10:43:04.563634+0000","last_fullsized":"2023-10-18T10:43:04.563634+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.288426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565086+0000","last_change":"2023-10-18T10:43:03.425524+0000","last_active":"2023-10-18T10:43:04.565086+0000","last_peered":"2023-10-18T10:43:04.565086+0000","last_clean":"2023-10-18T10:43:04.565086+0000","last_became_active":"2023-10-18T10:43:03.425299+0000","last_became_peered":"2023-10-18T10:43:03.425299+0000","last_unstale":"2023-10-18T10:43:04.565086+0000","last_undegraded":"2023-10-18T10:43:04.565086+0000","last_fullsized":"2023-10-18T10:43:04.565086+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:18.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563658+0000","last_change":"2023-10-18T10:43:03.415655+0000","last_active":"2023-10-18T10:43:04.563658+0000","last_peered":"2023-10-18T10:43:04.563658+0000","last_clean":"2023-10-18T10:43:04.563658+0000","last_became_active":"2023-10-18T10:43:03.415397+0000","last_became_peered":"2023-10-18T10:43:03.415397+0000","last_unstale":"2023-10-18T10:43:04.563658+0000","last_undegraded":"2023-10-18T10:43:04.563658+0000","last_fullsized":"2023-10-18T10:43:04.563658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:17.193216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564284+0000","last_change":"2023-10-18T10:43:02.964287+0000","last_active":"2023-10-18T10:43:04.564284+0000","last_peered":"2023-10-18T10:43:04.564284+0000","last_clean":"2023-10-18T10:43:04.564284+0000","last_became_active":"2023-10-18T10:43:02.964033+0000","last_became_peered":"2023-10-18T10:43:02.964033+0000","last_unstale":"2023-10-18T10:43:04.564284+0000","last_undegraded":"2023-10-18T10:43:04.564284+0000","last_fullsized":"2023-10-18T10:43:04.564284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:33.533023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564963+0000","last_change":"2023-10-18T10:43:03.419976+0000","last_active":"2023-10-18T10:43:04.564963+0000","last_peered":"2023-10-18T10:43:04.564963+0000","last_clean":"2023-10-18T10:43:04.564963+0000","last_became_active":"2023-10-18T10:43:03.419611+0000","last_became_peered":"2023-10-18T10:43:03.419611+0000","last_unstale":"2023-10-18T10:43:04.564963+0000","last_undegraded":"2023-10-18T10:43:04.564963+0000","last_fullsized":"2023-10-18T10:43:04.564963+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:57.163873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564210+0000","last_change":"2023-10-18T10:43:02.964463+0000","last_active":"2023-10-18T10:43:04.564210+0000","last_peered":"2023-10-18T10:43:04.564210+0000","last_clean":"2023-10-18T10:43:04.564210+0000","last_became_active":"2023-10-18T10:43:02.964036+0000","last_became_peered":"2023-10-18T10:43:02.964036+0000","last_unstale":"2023-10-18T10:43:04.564210+0000","last_undegraded":"2023-10-18T10:43:04.564210+0000","last_fullsized":"2023-10-18T10:43:04.564210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:27.017315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564542+0000","last_change":"2023-10-18T10:43:03.416778+0000","last_active":"2023-10-18T10:43:04.564542+0000","last_peered":"2023-10-18T10:43:04.564542+0000","last_clean":"2023-10-18T10:43:04.564542+0000","last_became_active":"2023-10-18T10:43:03.416492+0000","last_became_peered":"2023-10-18T10:43:03.416492+0000","last_unstale":"2023-10-18T10:43:04.564542+0000","last_undegraded":"2023-10-18T10:43:04.564542+0000","last_fullsized":"2023-10-18T10:43:04.564542+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:29.332215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564773+0000","last_change":"2023-10-18T10:43:02.965244+0000","last_active":"2023-10-18T10:43:04.564773+0000","last_peered":"2023-10-18T10:43:04.564773+0000","last_clean":"2023-10-18T10:43:04.564773+0000","last_became_active":"2023-10-18T10:43:02.964962+0000","last_became_peered":"2023-10-18T10:43:02.964962+0000","last_unstale":"2023-10-18T10:43:04.564773+0000","last_undegraded":"2023-10-18T10:43:04.564773+0000","last_fullsized":"2023-10-18T10:43:04.564773+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:10.229768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563816+0000","last_change":"2023-10-18T10:43:03.419372+0000","last_active":"2023-10-18T10:43:04.563816+0000","last_peered":"2023-10-18T10:43:04.563816+0000","last_clean":"2023-10-18T10:43:04.563816+0000","last_became_active":"2023-10-18T10:43:03.419133+0000","last_became_peered":"2023-10-18T10:43:03.419133+0000","last_unstale":"2023-10-18T10:43:04.563816+0000","last_undegraded":"2023-10-18T10:43:04.563816+0000","last_fullsized":"2023-10-18T10:43:04.563816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:25.080148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564382+0000","last_change":"2023-10-18T10:43:02.963180+0000","last_active":"2023-10-18T10:43:04.564382+0000","last_peered":"2023-10-18T10:43:04.564382+0000","last_clean":"2023-10-18T10:43:04.564382+0000","last_became_active":"2023-10-18T10:43:02.962924+0000","last_became_peered":"2023-10-18T10:43:02.962924+0000","last_unstale":"2023-10-18T10:43:04.564382+0000","last_undegraded":"2023-10-18T10:43:04.564382+0000","last_fullsized":"2023-10-18T10:43:04.564382+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:27.605859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564165+0000","last_change":"2023-10-18T10:43:03.424395+0000","last_active":"2023-10-18T10:43:04.564165+0000","last_peered":"2023-10-18T10:43:04.564165+0000","last_clean":"2023-10-18T10:43:04.564165+0000","last_became_active":"2023-10-18T10:43:03.424101+0000","last_became_peered":"2023-10-18T10:43:03.424101+0000","last_unstale":"2023-10-18T10:43:04.564165+0000","last_undegraded":"2023-10-18T10:43:04.564165+0000","last_fullsized":"2023-10-18T10:43:04.564165+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:43.317979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563722+0000","last_change":"2023-10-18T10:43:02.964389+0000","last_active":"2023-10-18T10:43:04.563722+0000","last_peered":"2023-10-18T10:43:04.563722+0000","last_clean":"2023-10-18T10:43:04.563722+0000","last_became_active":"2023-10-18T10:43:02.964134+0000","last_became_peered":"2023-10-18T10:43:02.964134+0000","last_unstale":"2023-10-18T10:43:04.563722+0000","last_undegraded":"2023-10-18T10:43:04.563722+0000","last_fullsized":"2023-10-18T10:43:04.563722+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:27.504789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408932+0000","last_change":"2023-10-18T10:43:03.417105+0000","last_active":"2023-10-18T10:43:04.408932+0000","last_peered":"2023-10-18T10:43:04.408932+0000","last_clean":"2023-10-18T10:43:04.408932+0000","last_became_active":"2023-10-18T10:43:03.416864+0000","last_became_peered":"2023-10-18T10:43:03.416864+0000","last_unstale":"2023-10-18T10:43:04.408932+0000","last_undegraded":"2023-10-18T10:43:04.408932+0000","last_fullsized":"2023-10-18T10:43:04.408932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:09.017135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564862+0000","last_change":"2023-10-18T10:43:02.967733+0000","last_active":"2023-10-18T10:43:04.564862+0000","last_peered":"2023-10-18T10:43:04.564862+0000","last_clean":"2023-10-18T10:43:04.564862+0000","last_became_active":"2023-10-18T10:43:02.967444+0000","last_became_peered":"2023-10-18T10:43:02.967444+0000","last_unstale":"2023-10-18T10:43:04.564862+0000","last_undegraded":"2023-10-18T10:43:04.564862+0000","last_fullsized":"2023-10-18T10:43:04.564862+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:59.437631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564910+0000","last_change":"2023-10-18T10:43:03.418676+0000","last_active":"2023-10-18T10:43:04.564910+0000","last_peered":"2023-10-18T10:43:04.564910+0000","last_clean":"2023-10-18T10:43:04.564910+0000","last_became_active":"2023-10-18T10:43:03.418441+0000","last_became_peered":"2023-10-18T10:43:03.418441+0000","last_unstale":"2023-10-18T10:43:04.564910+0000","last_undegraded":"2023-10-18T10:43:04.564910+0000","last_fullsized":"2023-10-18T10:43:04.564910+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:30.204629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563690+0000","last_change":"2023-10-18T10:43:02.963728+0000","last_active":"2023-10-18T10:43:04.563690+0000","last_peered":"2023-10-18T10:43:04.563690+0000","last_clean":"2023-10-18T10:43:04.563690+0000","last_became_active":"2023-10-18T10:43:02.963467+0000","last_became_peered":"2023-10-18T10:43:02.963467+0000","last_unstale":"2023-10-18T10:43:04.563690+0000","last_undegraded":"2023-10-18T10:43:04.563690+0000","last_fullsized":"2023-10-18T10:43:04.563690+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:38.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564445+0000","last_change":"2023-10-18T10:43:03.421100+0000","last_active":"2023-10-18T10:43:04.564445+0000","last_peered":"2023-10-18T10:43:04.564445+0000","last_clean":"2023-10-18T10:43:04.564445+0000","last_became_active":"2023-10-18T10:43:03.420845+0000","last_became_peered":"2023-10-18T10:43:03.420845+0000","last_unstale":"2023-10-18T10:43:04.564445+0000","last_undegraded":"2023-10-18T10:43:04.564445+0000","last_fullsized":"2023-10-18T10:43:04.564445+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:32:19.619118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564074+0000","last_change":"2023-10-18T10:43:02.971128+0000","last_active":"2023-10-18T10:43:04.564074+0000","last_peered":"2023-10-18T10:43:04.564074+0000","last_clean":"2023-10-18T10:43:04.564074+0000","last_became_active":"2023-10-18T10:43:02.970845+0000","last_became_peered":"2023-10-18T10:43:02.970845+0000","last_unstale":"2023-10-18T10:43:04.564074+0000","last_undegraded":"2023-10-18T10:43:04.564074+0000","last_fullsized":"2023-10-18T10:43:04.564074+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:57.435429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564812+0000","last_change":"2023-10-18T10:43:02.972906+0000","last_active":"2023-10-18T10:43:04.564812+0000","last_peered":"2023-10-18T10:43:04.564812+0000","last_clean":"2023-10-18T10:43:04.564812+0000","last_became_active":"2023-10-18T10:43:02.972652+0000","last_became_peered":"2023-10-18T10:43:02.972652+0000","last_unstale":"2023-10-18T10:43:04.564812+0000","last_undegraded":"2023-10-18T10:43:04.564812+0000","last_fullsized":"2023-10-18T10:43:04.564812+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:09.602406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408779+0000","last_change":"2023-10-18T10:43:03.416563+0000","last_active":"2023-10-18T10:43:04.408779+0000","last_peered":"2023-10-18T10:43:04.408779+0000","last_clean":"2023-10-18T10:43:04.408779+0000","last_became_active":"2023-10-18T10:43:03.416321+0000","last_became_peered":"2023-10-18T10:43:03.416321+0000","last_unstale":"2023-10-18T10:43:04.408779+0000","last_undegraded":"2023-10-18T10:43:04.408779+0000","last_fullsized":"2023-10-18T10:43:04.408779+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:57.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565167+0000","last_change":"2023-10-18T10:43:02.962995+0000","last_active":"2023-10-18T10:43:04.565167+0000","last_peered":"2023-10-18T10:43:04.565167+0000","last_clean":"2023-10-18T10:43:04.565167+0000","last_became_active":"2023-10-18T10:43:02.962723+0000","last_became_peered":"2023-10-18T10:43:02.962723+0000","last_unstale":"2023-10-18T10:43:04.565167+0000","last_undegraded":"2023-10-18T10:43:04.565167+0000","last_fullsized":"2023-10-18T10:43:04.565167+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:45.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564038+0000","last_change":"2023-10-18T10:43:03.424431+0000","last_active":"2023-10-18T10:43:04.564038+0000","last_peered":"2023-10-18T10:43:04.564038+0000","last_clean":"2023-10-18T10:43:04.564038+0000","last_became_active":"2023-10-18T10:43:03.424173+0000","last_became_peered":"2023-10-18T10:43:03.424173+0000","last_unstale":"2023-10-18T10:43:04.564038+0000","last_undegraded":"2023-10-18T10:43:04.564038+0000","last_fullsized":"2023-10-18T10:43:04.564038+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:19.931739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564415+0000","last_change":"2023-10-18T10:43:02.968945+0000","last_active":"2023-10-18T10:43:04.564415+0000","last_peered":"2023-10-18T10:43:04.564415+0000","last_clean":"2023-10-18T10:43:04.564415+0000","last_became_active":"2023-10-18T10:43:02.968666+0000","last_became_peered":"2023-10-18T10:43:02.968666+0000","last_unstale":"2023-10-18T10:43:04.564415+0000","last_undegraded":"2023-10-18T10:43:04.564415+0000","last_fullsized":"2023-10-18T10:43:04.564415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:26.843460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564480+0000","last_change":"2023-10-18T10:43:03.421780+0000","last_active":"2023-10-18T10:43:04.564480+0000","last_peered":"2023-10-18T10:43:04.564480+0000","last_clean":"2023-10-18T10:43:04.564480+0000","last_became_active":"2023-10-18T10:43:03.421547+0000","last_became_peered":"2023-10-18T10:43:03.421547+0000","last_unstale":"2023-10-18T10:43:04.564480+0000","last_undegraded":"2023-10-18T10:43:04.564480+0000","last_fullsized":"2023-10-18T10:43:04.564480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:54:47.177774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563859+0000","last_change":"2023-10-18T10:43:03.423032+0000","last_active":"2023-10-18T10:43:04.563859+0000","last_peered":"2023-10-18T10:43:04.563859+0000","last_clean":"2023-10-18T10:43:04.563859+0000","last_became_active":"2023-10-18T10:43:03.422767+0000","last_became_peered":"2023-10-18T10:43:03.422767+0000","last_unstale":"2023-10-18T10:43:04.563859+0000","last_undegraded":"2023-10-18T10:43:04.563859+0000","last_fullsized":"2023-10-18T10:43:04.563859+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:17.615763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563911+0000","last_change":"2023-10-18T10:43:02.965062+0000","last_active":"2023-10-18T10:43:04.563911+0000","last_peered":"2023-10-18T10:43:04.563911+0000","last_clean":"2023-10-18T10:43:04.563911+0000","last_became_active":"2023-10-18T10:43:02.964753+0000","last_became_peered":"2023-10-18T10:43:02.964753+0000","last_unstale":"2023-10-18T10:43:04.563911+0000","last_undegraded":"2023-10-18T10:43:04.563911+0000","last_fullsized":"2023-10-18T10:43:04.563911+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:41.151549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564340+0000","last_change":"2023-10-18T10:43:03.421221+0000","last_active":"2023-10-18T10:43:04.564340+0000","last_peered":"2023-10-18T10:43:04.564340+0000","last_clean":"2023-10-18T10:43:04.564340+0000","last_became_active":"2023-10-18T10:43:03.420954+0000","last_became_peered":"2023-10-18T10:43:03.420954+0000","last_unstale":"2023-10-18T10:43:04.564340+0000","last_undegraded":"2023-10-18T10:43:04.564340+0000","last_fullsized":"2023-10-18T10:43:04.564340+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:42.945030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564932+0000","last_change":"2023-10-18T10:43:02.962374+0000","last_active":"2023-10-18T10:43:04.564932+0000","last_peered":"2023-10-18T10:43:04.564932+0000","last_clean":"2023-10-18T10:43:04.564932+0000","last_became_active":"2023-10-18T10:43:02.962065+0000","last_became_peered":"2023-10-18T10:43:02.962065+0000","last_unstale":"2023-10-18T10:43:04.564932+0000","last_undegraded":"2023-10-18T10:43:04.564932+0000","last_fullsized":"2023-10-18T10:43:04.564932+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:37.693671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564809+0000","last_change":"2023-10-18T10:43:03.418250+0000","last_active":"2023-10-18T10:43:04.564809+0000","last_peered":"2023-10-18T10:43:04.564809+0000","last_clean":"2023-10-18T10:43:04.564809+0000","last_became_active":"2023-10-18T10:43:03.417934+0000","last_became_peered":"2023-10-18T10:43:03.417934+0000","last_unstale":"2023-10-18T10:43:04.564809+0000","last_undegraded":"2023-10-18T10:43:04.564809+0000","last_fullsized":"2023-10-18T10:43:04.564809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:09.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563736+0000","last_change":"2023-10-18T10:43:02.965897+0000","last_active":"2023-10-18T10:43:04.563736+0000","last_peered":"2023-10-18T10:43:04.563736+0000","last_clean":"2023-10-18T10:43:04.563736+0000","last_became_active":"2023-10-18T10:43:02.965636+0000","last_became_peered":"2023-10-18T10:43:02.965636+0000","last_unstale":"2023-10-18T10:43:04.563736+0000","last_undegraded":"2023-10-18T10:43:04.563736+0000","last_fullsized":"2023-10-18T10:43:04.563736+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:10:39.372132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564606+0000","last_change":"2023-10-18T10:43:03.420416+0000","last_active":"2023-10-18T10:43:04.564606+0000","last_peered":"2023-10-18T10:43:04.564606+0000","last_clean":"2023-10-18T10:43:04.564606+0000","last_became_active":"2023-10-18T10:43:03.420064+0000","last_became_peered":"2023-10-18T10:43:03.420064+0000","last_unstale":"2023-10-18T10:43:04.564606+0000","last_undegraded":"2023-10-18T10:43:04.564606+0000","last_fullsized":"2023-10-18T10:43:04.564606+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:01.892160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564707+0000","last_change":"2023-10-18T10:43:02.962533+0000","last_active":"2023-10-18T10:43:04.564707+0000","last_peered":"2023-10-18T10:43:04.564707+0000","last_clean":"2023-10-18T10:43:04.564707+0000","last_became_active":"2023-10-18T10:43:02.962275+0000","last_became_peered":"2023-10-18T10:43:02.962275+0000","last_unstale":"2023-10-18T10:43:04.564707+0000","last_undegraded":"2023-10-18T10:43:04.564707+0000","last_fullsized":"2023-10-18T10:43:04.564707+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:26.189773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"117'214","reported_seq":611,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563741+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:43:04.563741+0000","last_peered":"2023-10-18T10:43:04.563741+0000","last_clean":"2023-10-18T10:43:04.563741+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:43:04.563741+0000","last_undegraded":"2023-10-18T10:43:04.563741+0000","last_fullsized":"2023-10-18T10:43:04.563741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575068,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59692,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312148,"statfs":{"total":96636764160,"available":96575639552,"internally_reserved":0,"allocated":2187264,"data_stored":1388162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.58299999999999996},"min":{"1min":0.36099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.754,"5min":0.876,"15min":0.88300000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63200000000000001,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70299999999999996,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.60299999999999998},"min":{"1min":0.42199999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.76000000000000001,"5min":0.79800000000000004,"15min":0.85799999999999998},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.68899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.315},"max":{"1min":0.78500000000000003,"5min":0.875,"15min":0.875},"last":0.504}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575068,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":799850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82599999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.69499999999999995,"5min":0.77700000000000002,"15min":0.82699999999999996},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65800000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.432}]},{"osd":3,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.78400000000000003,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.748,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.80300000000000005,"5min":0.82199999999999995,"15min":0.87},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":797634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.39600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88400000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81399999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.81999999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.41199999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.47599999999999998},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71599999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.375},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.496},"min":{"1min":0.373,"5min":0.31,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.38200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77200000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.312,"15min":0.31},"max":{"1min":0.84299999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.443},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.44700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.871,"15min":0.871},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32680,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1523712,"data_stored":765110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57699999999999996,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.77000000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.54700000000000004,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.753,"15min":0.877},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.89300000000000002},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73499999999999999,"5min":0.873,"15min":0.873},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71899999999999997,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.78200000000000003},"last":0.62}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.33200000000000002,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.79200000000000004,"5min":1.446,"15min":1.446},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.81999999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.879,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.52800000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.499,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.624,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.59299999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53400000000000003,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.379},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.434,"15min":0.45100000000000001},"min":{"1min":0.28599999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.80200000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.879,"15min":0.94699999999999995},"last":0.5}]},{"osd":5,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":1.002,"15min":1.002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88900000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.629,"15min":0.63800000000000001},"min":{"1min":0.434,"5min":0.38,"15min":0.37},"max":{"1min":0.82499999999999996,"5min":1.018,"15min":1.018},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.751,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.63900000000000001},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.627,"15min":0.65300000000000002},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54940,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316900,"statfs":{"total":96636764160,"available":96580505600,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.371,"15min":0.374},"min":{"1min":0.252,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.626},"last":0.377}]},{"osd":1,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29899999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.57899999999999996,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.315}]},{"osd":3,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.24299999999999999},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.434,"15min":0.44700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.41899999999999998,"15min":0.254},"max":{"1min":0.86499999999999999,"5min":0.88200000000000001,"15min":0.88800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.625,"15min":0.64500000000000002},"min":{"1min":0.45300000000000001,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.95699999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.63},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.93700000000000006},"last":0.57299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.84599999999999997,"15min":0.92500000000000004},"last":0.54100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36199999999999999,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.48999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.32000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.39200000000000002},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.374},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.42099999999999999},"min":{"1min":0.314,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53600000000000003,"5min":0.627,"15min":0.64800000000000002},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.33100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59499999999999997,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.33500000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.83999999999999997,"15min":0.95599999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.623},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.753,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.88100000000000001,"15min":0.95999999999999996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60199999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.76200000000000001,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.63700000000000001},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.78000000000000003,"5min":1.03,"15min":1.03},"last":0.749}]},{"osd":7,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.63},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78800000000000003,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575067,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2195456,"data_stored":1389704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.39000000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.54600000000000004,"5min":0.62,"15min":0.76900000000000002},"last":0.214}]},{"osd":2,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.187},"max":{"1min":0.58099999999999996,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.41999999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.29299999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.39500000000000002},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56599999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.312}]},{"osd":4,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.61699999999999999},"min":{"1min":0.372,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81999999999999995,"5min":0.996,"15min":0.996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90300000000000002,"5min":1.038,"15min":1.038},"last":0.34200000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.89200000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93700000000000006,"5min":1.026,"15min":1.3700000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88100000000000001,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.64300000000000002},"min":{"1min":0.41199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.05,"15min":1.339},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.64600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:43:07.257 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:43:07.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.392+0000 7fa84a690640 1 -- 172.21.15.125:0/2691371323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa844070af0 msgr2=0x7fa8440affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.392+0000 7fa84a690640 1 --2- 172.21.15.125:0/2691371323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa844070af0 0x7fa8440affe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa83c005fd0 tx=0x7fa83c02c050 comp rx=0 tx=0).stop 2023-10-18T10:43:07.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.392+0000 7fa84a690640 1 -- 172.21.15.125:0/2691371323 shutdown_connections 2023-10-18T10:43:07.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.392+0000 7fa84a690640 1 --2- 172.21.15.125:0/2691371323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa844070af0 0x7fa8440affe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.392+0000 7fa84a690640 1 --2- 172.21.15.125:0/2691371323 >> v2:172.21.15.139:3300/0 conn(0x7fa84413e520 0x7fa84406bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.392+0000 7fa84a690640 1 -- 172.21.15.125:0/2691371323 >> 172.21.15.125:0/2691371323 conn(0x7fa844069930 msgr2=0x7fa844069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:07.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.392+0000 7fa84a690640 1 -- 172.21.15.125:0/2691371323 shutdown_connections 2023-10-18T10:43:07.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.393+0000 7fa84a690640 1 -- 172.21.15.125:0/2691371323 wait complete. 2023-10-18T10:43:07.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.394+0000 7fa84a690640 1 Processor -- start 2023-10-18T10:43:07.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.394+0000 7fa84a690640 1 -- start start 2023-10-18T10:43:07.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.394+0000 7fa84a690640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa844070af0 0x7fa844153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.394+0000 7fa84a690640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84413e520 0x7fa8441555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.394+0000 7fa84a690640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa844141690 0x7fa844155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.394+0000 7fa84a690640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844159a00 con 0x7fa844141690 2023-10-18T10:43:07.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.394+0000 7fa84a690640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844153f00 con 0x7fa844070af0 2023-10-18T10:43:07.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.394+0000 7fa84a690640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa844159ca0 con 0x7fa84413e520 2023-10-18T10:43:07.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa844141690 0x7fa844155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa844141690 0x7fa844155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49038/0 (socket says 172.21.15.125:49038) 2023-10-18T10:43:07.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 -- 172.21.15.125:0/2422911118 learned_addr learned my addr 172.21.15.125:0/2422911118 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:07.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa843fff640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa844070af0 0x7fa844153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa8437fe640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84413e520 0x7fa8441555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 -- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84413e520 msgr2=0x7fa8441555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84413e520 0x7fa8441555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 -- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa844070af0 msgr2=0x7fa844153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa844070af0 0x7fa844153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 -- 172.21.15.125:0/2422911118 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa838009560 con 0x7fa844141690 2023-10-18T10:43:07.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa848c2d640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa844141690 0x7fa844155ae0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa83c006010 tx=0x7fa83c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:07.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa840ff9640 1 -- 172.21.15.125:0/2422911118 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004880 con 0x7fa844141690 2023-10-18T10:43:07.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa840ff9640 1 -- 172.21.15.125:0/2422911118 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa83c03d430 con 0x7fa844141690 2023-10-18T10:43:07.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c009840 con 0x7fa844141690 2023-10-18T10:43:07.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa840ff9640 1 -- 172.21.15.125:0/2422911118 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c03d840 con 0x7fa844141690 2023-10-18T10:43:07.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.395+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84415a260 con 0x7fa844141690 2023-10-18T10:43:07.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.396+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8440690e0 con 0x7fa844141690 2023-10-18T10:43:07.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.397+0000 7fa840ff9640 1 -- 172.21.15.125:0/2422911118 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa83c03d9a0 con 0x7fa844141690 2023-10-18T10:43:07.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.397+0000 7fa840ff9640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa83006af60 0x7fa83006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.397+0000 7fa840ff9640 1 -- 172.21.15.125:0/2422911118 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa83c0ac5b0 con 0x7fa844141690 2023-10-18T10:43:07.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.397+0000 7fa843fff640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa83006af60 0x7fa83006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.398+0000 7fa843fff640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa83006af60 0x7fa83006d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8440af120 tx=0x7fa838009260 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:07.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.401+0000 7fa840ff9640 1 -- 172.21.15.125:0/2422911118 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa83c0774c0 con 0x7fa844141690 2023-10-18T10:43:07.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.563+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 --> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa844159e10 con 0x7fa83006af60 2023-10-18T10:43:07.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.571+0000 7fa840ff9640 1 -- 172.21.15.125:0/2422911118 <== mgr.7360 v2:172.21.15.139:6832/1511464628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287849 (secure 0 0 0) 0x7fa844159e10 con 0x7fa83006af60 2023-10-18T10:43:07.576 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:07.577 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:43:07.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa83006af60 msgr2=0x7fa83006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa83006af60 0x7fa83006d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8440af120 tx=0x7fa838009260 comp rx=0 tx=0).stop 2023-10-18T10:43:07.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa844141690 msgr2=0x7fa844155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa844141690 0x7fa844155ae0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa83c006010 tx=0x7fa83c005c80 comp rx=0 tx=0).stop 2023-10-18T10:43:07.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 shutdown_connections 2023-10-18T10:43:07.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa83006af60 0x7fa83006d420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa844141690 0x7fa844155ae0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84413e520 0x7fa8441555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 --2- 172.21.15.125:0/2422911118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa844070af0 0x7fa844153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 >> 172.21.15.125:0/2422911118 conn(0x7fa844069930 msgr2=0x7fa8440b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:07.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 shutdown_connections 2023-10-18T10:43:07.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.574+0000 7fa84a690640 1 -- 172.21.15.125:0/2422911118 wait complete. 2023-10-18T10:43:07.587 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:43:05.899805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":364,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":315772,"kb_used_data":13540,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754658948,"statfs":{"total":773094113280,"available":772770762752,"internally_reserved":0,"allocated":13864960,"data_stored":7534012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001056"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564947+0000","last_change":"2023-10-18T10:43:03.420057+0000","last_active":"2023-10-18T10:43:04.564947+0000","last_peered":"2023-10-18T10:43:04.564947+0000","last_clean":"2023-10-18T10:43:04.564947+0000","last_became_active":"2023-10-18T10:43:03.419809+0000","last_became_peered":"2023-10-18T10:43:03.419809+0000","last_unstale":"2023-10-18T10:43:04.564947+0000","last_undegraded":"2023-10-18T10:43:04.564947+0000","last_fullsized":"2023-10-18T10:43:04.564947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:51.310959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565191+0000","last_change":"2023-10-18T10:43:03.422800+0000","last_active":"2023-10-18T10:43:04.565191+0000","last_peered":"2023-10-18T10:43:04.565191+0000","last_clean":"2023-10-18T10:43:04.565191+0000","last_became_active":"2023-10-18T10:43:03.422037+0000","last_became_peered":"2023-10-18T10:43:03.422037+0000","last_unstale":"2023-10-18T10:43:04.565191+0000","last_undegraded":"2023-10-18T10:43:04.565191+0000","last_fullsized":"2023-10-18T10:43:04.565191+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:03.106690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564044+0000","last_change":"2023-10-18T10:43:03.424158+0000","last_active":"2023-10-18T10:43:04.564044+0000","last_peered":"2023-10-18T10:43:04.564044+0000","last_clean":"2023-10-18T10:43:04.564044+0000","last_became_active":"2023-10-18T10:43:03.423915+0000","last_became_peered":"2023-10-18T10:43:03.423915+0000","last_unstale":"2023-10-18T10:43:04.564044+0000","last_undegraded":"2023-10-18T10:43:04.564044+0000","last_fullsized":"2023-10-18T10:43:04.564044+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:49.732624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408937+0000","last_change":"2023-10-18T10:43:03.415009+0000","last_active":"2023-10-18T10:43:04.408937+0000","last_peered":"2023-10-18T10:43:04.408937+0000","last_clean":"2023-10-18T10:43:04.408937+0000","last_became_active":"2023-10-18T10:43:03.414710+0000","last_became_peered":"2023-10-18T10:43:03.414710+0000","last_unstale":"2023-10-18T10:43:04.408937+0000","last_undegraded":"2023-10-18T10:43:04.408937+0000","last_fullsized":"2023-10-18T10:43:04.408937+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:21.227528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563635+0000","last_change":"2023-10-18T10:43:03.411644+0000","last_active":"2023-10-18T10:43:04.563635+0000","last_peered":"2023-10-18T10:43:04.563635+0000","last_clean":"2023-10-18T10:43:04.563635+0000","last_became_active":"2023-10-18T10:43:03.411381+0000","last_became_peered":"2023-10-18T10:43:03.411381+0000","last_unstale":"2023-10-18T10:43:04.563635+0000","last_undegraded":"2023-10-18T10:43:04.563635+0000","last_fullsized":"2023-10-18T10:43:04.563635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:28.704370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564515+0000","last_change":"2023-10-18T10:43:03.420380+0000","last_active":"2023-10-18T10:43:04.564515+0000","last_peered":"2023-10-18T10:43:04.564515+0000","last_clean":"2023-10-18T10:43:04.564515+0000","last_became_active":"2023-10-18T10:43:03.420112+0000","last_became_peered":"2023-10-18T10:43:03.420112+0000","last_unstale":"2023-10-18T10:43:04.564515+0000","last_undegraded":"2023-10-18T10:43:04.564515+0000","last_fullsized":"2023-10-18T10:43:04.564515+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:15.468100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564645+0000","last_change":"2023-10-18T10:43:03.417853+0000","last_active":"2023-10-18T10:43:04.564645+0000","last_peered":"2023-10-18T10:43:04.564645+0000","last_clean":"2023-10-18T10:43:04.564645+0000","last_became_active":"2023-10-18T10:43:03.417585+0000","last_became_peered":"2023-10-18T10:43:03.417585+0000","last_unstale":"2023-10-18T10:43:04.564645+0000","last_undegraded":"2023-10-18T10:43:04.564645+0000","last_fullsized":"2023-10-18T10:43:04.564645+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:57.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563425+0000","last_change":"2023-10-18T10:43:03.420896+0000","last_active":"2023-10-18T10:43:04.563425+0000","last_peered":"2023-10-18T10:43:04.563425+0000","last_clean":"2023-10-18T10:43:04.563425+0000","last_became_active":"2023-10-18T10:43:03.420633+0000","last_became_peered":"2023-10-18T10:43:03.420633+0000","last_unstale":"2023-10-18T10:43:04.563425+0000","last_undegraded":"2023-10-18T10:43:04.563425+0000","last_fullsized":"2023-10-18T10:43:04.563425+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:54.658345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564188+0000","last_change":"2023-10-18T10:43:03.423222+0000","last_active":"2023-10-18T10:43:04.564188+0000","last_peered":"2023-10-18T10:43:04.564188+0000","last_clean":"2023-10-18T10:43:04.564188+0000","last_became_active":"2023-10-18T10:43:03.422095+0000","last_became_peered":"2023-10-18T10:43:03.422095+0000","last_unstale":"2023-10-18T10:43:04.564188+0000","last_undegraded":"2023-10-18T10:43:04.564188+0000","last_fullsized":"2023-10-18T10:43:04.564188+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:36.033906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564111+0000","last_change":"2023-10-18T10:43:03.420436+0000","last_active":"2023-10-18T10:43:04.564111+0000","last_peered":"2023-10-18T10:43:04.564111+0000","last_clean":"2023-10-18T10:43:04.564111+0000","last_became_active":"2023-10-18T10:43:03.419414+0000","last_became_peered":"2023-10-18T10:43:03.419414+0000","last_unstale":"2023-10-18T10:43:04.564111+0000","last_undegraded":"2023-10-18T10:43:04.564111+0000","last_fullsized":"2023-10-18T10:43:04.564111+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:27.017090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564170+0000","last_change":"2023-10-18T10:43:03.425870+0000","last_active":"2023-10-18T10:43:04.564170+0000","last_peered":"2023-10-18T10:43:04.564170+0000","last_clean":"2023-10-18T10:43:04.564170+0000","last_became_active":"2023-10-18T10:43:03.425583+0000","last_became_peered":"2023-10-18T10:43:03.425583+0000","last_unstale":"2023-10-18T10:43:04.564170+0000","last_undegraded":"2023-10-18T10:43:04.564170+0000","last_fullsized":"2023-10-18T10:43:04.564170+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:07.373278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408816+0000","last_change":"2023-10-18T10:43:03.421921+0000","last_active":"2023-10-18T10:43:04.408816+0000","last_peered":"2023-10-18T10:43:04.408816+0000","last_clean":"2023-10-18T10:43:04.408816+0000","last_became_active":"2023-10-18T10:43:03.421644+0000","last_became_peered":"2023-10-18T10:43:03.421644+0000","last_unstale":"2023-10-18T10:43:04.408816+0000","last_undegraded":"2023-10-18T10:43:04.408816+0000","last_fullsized":"2023-10-18T10:43:04.408816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:58.741901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563868+0000","last_change":"2023-10-18T10:43:03.415137+0000","last_active":"2023-10-18T10:43:04.563868+0000","last_peered":"2023-10-18T10:43:04.563868+0000","last_clean":"2023-10-18T10:43:04.563868+0000","last_became_active":"2023-10-18T10:43:03.414871+0000","last_became_peered":"2023-10-18T10:43:03.414871+0000","last_unstale":"2023-10-18T10:43:04.563868+0000","last_undegraded":"2023-10-18T10:43:04.563868+0000","last_fullsized":"2023-10-18T10:43:04.563868+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:33.182433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563789+0000","last_change":"2023-10-18T10:43:03.420511+0000","last_active":"2023-10-18T10:43:04.563789+0000","last_peered":"2023-10-18T10:43:04.563789+0000","last_clean":"2023-10-18T10:43:04.563789+0000","last_became_active":"2023-10-18T10:43:03.420241+0000","last_became_peered":"2023-10-18T10:43:03.420241+0000","last_unstale":"2023-10-18T10:43:04.563789+0000","last_undegraded":"2023-10-18T10:43:04.563789+0000","last_fullsized":"2023-10-18T10:43:04.563789+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:38.074311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563696+0000","last_change":"2023-10-18T10:43:03.417552+0000","last_active":"2023-10-18T10:43:04.563696+0000","last_peered":"2023-10-18T10:43:04.563696+0000","last_clean":"2023-10-18T10:43:04.563696+0000","last_became_active":"2023-10-18T10:43:03.417291+0000","last_became_peered":"2023-10-18T10:43:03.417291+0000","last_unstale":"2023-10-18T10:43:04.563696+0000","last_undegraded":"2023-10-18T10:43:04.563696+0000","last_fullsized":"2023-10-18T10:43:04.563696+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:18.941147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.425285+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.424971+0000","last_became_peered":"2023-10-18T10:43:03.424971+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:00.092514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565204+0000","last_change":"2023-10-18T10:43:03.416033+0000","last_active":"2023-10-18T10:43:04.565204+0000","last_peered":"2023-10-18T10:43:04.565204+0000","last_clean":"2023-10-18T10:43:04.565204+0000","last_became_active":"2023-10-18T10:43:03.415763+0000","last_became_peered":"2023-10-18T10:43:03.415763+0000","last_unstale":"2023-10-18T10:43:04.565204+0000","last_undegraded":"2023-10-18T10:43:04.565204+0000","last_fullsized":"2023-10-18T10:43:04.565204+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:36.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564171+0000","last_change":"2023-10-18T10:43:03.418626+0000","last_active":"2023-10-18T10:43:04.564171+0000","last_peered":"2023-10-18T10:43:04.564171+0000","last_clean":"2023-10-18T10:43:04.564171+0000","last_became_active":"2023-10-18T10:43:03.418344+0000","last_became_peered":"2023-10-18T10:43:03.418344+0000","last_unstale":"2023-10-18T10:43:04.564171+0000","last_undegraded":"2023-10-18T10:43:04.564171+0000","last_fullsized":"2023-10-18T10:43:04.564171+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:55.314298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564184+0000","last_change":"2023-10-18T10:43:03.421228+0000","last_active":"2023-10-18T10:43:04.564184+0000","last_peered":"2023-10-18T10:43:04.564184+0000","last_clean":"2023-10-18T10:43:04.564184+0000","last_became_active":"2023-10-18T10:43:03.420908+0000","last_became_peered":"2023-10-18T10:43:03.420908+0000","last_unstale":"2023-10-18T10:43:04.564184+0000","last_undegraded":"2023-10-18T10:43:04.564184+0000","last_fullsized":"2023-10-18T10:43:04.564184+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:19.745476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564166+0000","last_change":"2023-10-18T10:43:03.421765+0000","last_active":"2023-10-18T10:43:04.564166+0000","last_peered":"2023-10-18T10:43:04.564166+0000","last_clean":"2023-10-18T10:43:04.564166+0000","last_became_active":"2023-10-18T10:43:03.421501+0000","last_became_peered":"2023-10-18T10:43:03.421501+0000","last_unstale":"2023-10-18T10:43:04.564166+0000","last_undegraded":"2023-10-18T10:43:04.564166+0000","last_fullsized":"2023-10-18T10:43:04.564166+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:06.267958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.424152+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.423890+0000","last_became_peered":"2023-10-18T10:43:03.423890+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:26.100932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564263+0000","last_change":"2023-10-18T10:43:03.417440+0000","last_active":"2023-10-18T10:43:04.564263+0000","last_peered":"2023-10-18T10:43:04.564263+0000","last_clean":"2023-10-18T10:43:04.564263+0000","last_became_active":"2023-10-18T10:43:03.417201+0000","last_became_peered":"2023-10-18T10:43:03.417201+0000","last_unstale":"2023-10-18T10:43:04.564263+0000","last_undegraded":"2023-10-18T10:43:04.564263+0000","last_fullsized":"2023-10-18T10:43:04.564263+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:16.834481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564364+0000","last_change":"2023-10-18T10:43:03.419666+0000","last_active":"2023-10-18T10:43:04.564364+0000","last_peered":"2023-10-18T10:43:04.564364+0000","last_clean":"2023-10-18T10:43:04.564364+0000","last_became_active":"2023-10-18T10:43:03.418843+0000","last_became_peered":"2023-10-18T10:43:03.418843+0000","last_unstale":"2023-10-18T10:43:04.564364+0000","last_undegraded":"2023-10-18T10:43:04.564364+0000","last_fullsized":"2023-10-18T10:43:04.564364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:25.903261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565004+0000","last_change":"2023-10-18T10:43:03.423183+0000","last_active":"2023-10-18T10:43:04.565004+0000","last_peered":"2023-10-18T10:43:04.565004+0000","last_clean":"2023-10-18T10:43:04.565004+0000","last_became_active":"2023-10-18T10:43:03.422819+0000","last_became_peered":"2023-10-18T10:43:03.422819+0000","last_unstale":"2023-10-18T10:43:04.565004+0000","last_undegraded":"2023-10-18T10:43:04.565004+0000","last_fullsized":"2023-10-18T10:43:04.565004+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:34.324413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564301+0000","last_change":"2023-10-18T10:43:03.417982+0000","last_active":"2023-10-18T10:43:04.564301+0000","last_peered":"2023-10-18T10:43:04.564301+0000","last_clean":"2023-10-18T10:43:04.564301+0000","last_became_active":"2023-10-18T10:43:03.417736+0000","last_became_peered":"2023-10-18T10:43:03.417736+0000","last_unstale":"2023-10-18T10:43:04.564301+0000","last_undegraded":"2023-10-18T10:43:04.564301+0000","last_fullsized":"2023-10-18T10:43:04.564301+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:51.640858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564774+0000","last_change":"2023-10-18T10:43:03.416816+0000","last_active":"2023-10-18T10:43:04.564774+0000","last_peered":"2023-10-18T10:43:04.564774+0000","last_clean":"2023-10-18T10:43:04.564774+0000","last_became_active":"2023-10-18T10:43:03.416441+0000","last_became_peered":"2023-10-18T10:43:03.416441+0000","last_unstale":"2023-10-18T10:43:04.564774+0000","last_undegraded":"2023-10-18T10:43:04.564774+0000","last_fullsized":"2023-10-18T10:43:04.564774+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:23.903407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563541+0000","last_change":"2023-10-18T10:43:03.423095+0000","last_active":"2023-10-18T10:43:04.563541+0000","last_peered":"2023-10-18T10:43:04.563541+0000","last_clean":"2023-10-18T10:43:04.563541+0000","last_became_active":"2023-10-18T10:43:03.422840+0000","last_became_peered":"2023-10-18T10:43:03.422840+0000","last_unstale":"2023-10-18T10:43:04.563541+0000","last_undegraded":"2023-10-18T10:43:04.563541+0000","last_fullsized":"2023-10-18T10:43:04.563541+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:09.585007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564675+0000","last_change":"2023-10-18T10:43:03.423114+0000","last_active":"2023-10-18T10:43:04.564675+0000","last_peered":"2023-10-18T10:43:04.564675+0000","last_clean":"2023-10-18T10:43:04.564675+0000","last_became_active":"2023-10-18T10:43:03.422862+0000","last_became_peered":"2023-10-18T10:43:03.422862+0000","last_unstale":"2023-10-18T10:43:04.564675+0000","last_undegraded":"2023-10-18T10:43:04.564675+0000","last_fullsized":"2023-10-18T10:43:04.564675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:40:54.685089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"127'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408994+0000","last_change":"2023-10-18T10:43:02.963797+0000","last_active":"2023-10-18T10:43:04.408994+0000","last_peered":"2023-10-18T10:43:04.408994+0000","last_clean":"2023-10-18T10:43:04.408994+0000","last_became_active":"2023-10-18T10:43:02.963525+0000","last_became_peered":"2023-10-18T10:43:02.963525+0000","last_unstale":"2023-10-18T10:43:04.408994+0000","last_undegraded":"2023-10-18T10:43:04.408994+0000","last_fullsized":"2023-10-18T10:43:04.408994+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:18:22.535089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563557+0000","last_change":"2023-10-18T10:43:03.419783+0000","last_active":"2023-10-18T10:43:04.563557+0000","last_peered":"2023-10-18T10:43:04.563557+0000","last_clean":"2023-10-18T10:43:04.563557+0000","last_became_active":"2023-10-18T10:43:03.419543+0000","last_became_peered":"2023-10-18T10:43:03.419543+0000","last_unstale":"2023-10-18T10:43:04.563557+0000","last_undegraded":"2023-10-18T10:43:04.563557+0000","last_fullsized":"2023-10-18T10:43:04.563557+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:39.668528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564617+0000","last_change":"2023-10-18T10:43:02.965235+0000","last_active":"2023-10-18T10:43:04.564617+0000","last_peered":"2023-10-18T10:43:04.564617+0000","last_clean":"2023-10-18T10:43:04.564617+0000","last_became_active":"2023-10-18T10:43:02.964880+0000","last_became_peered":"2023-10-18T10:43:02.964880+0000","last_unstale":"2023-10-18T10:43:04.564617+0000","last_undegraded":"2023-10-18T10:43:04.564617+0000","last_fullsized":"2023-10-18T10:43:04.564617+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:48.610823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564182+0000","last_change":"2023-10-18T10:43:03.416904+0000","last_active":"2023-10-18T10:43:04.564182+0000","last_peered":"2023-10-18T10:43:04.564182+0000","last_clean":"2023-10-18T10:43:04.564182+0000","last_became_active":"2023-10-18T10:43:03.416489+0000","last_became_peered":"2023-10-18T10:43:03.416489+0000","last_unstale":"2023-10-18T10:43:04.564182+0000","last_undegraded":"2023-10-18T10:43:04.564182+0000","last_fullsized":"2023-10-18T10:43:04.564182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:36.136092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564337+0000","last_change":"2023-10-18T10:43:02.969842+0000","last_active":"2023-10-18T10:43:04.564337+0000","last_peered":"2023-10-18T10:43:04.564337+0000","last_clean":"2023-10-18T10:43:04.564337+0000","last_became_active":"2023-10-18T10:43:02.969620+0000","last_became_peered":"2023-10-18T10:43:02.969620+0000","last_unstale":"2023-10-18T10:43:04.564337+0000","last_undegraded":"2023-10-18T10:43:04.564337+0000","last_fullsized":"2023-10-18T10:43:04.564337+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:23:14.364024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409001+0000","last_change":"2023-10-18T10:43:03.417690+0000","last_active":"2023-10-18T10:43:04.409001+0000","last_peered":"2023-10-18T10:43:04.409001+0000","last_clean":"2023-10-18T10:43:04.409001+0000","last_became_active":"2023-10-18T10:43:03.417452+0000","last_became_peered":"2023-10-18T10:43:03.417452+0000","last_unstale":"2023-10-18T10:43:04.409001+0000","last_undegraded":"2023-10-18T10:43:04.409001+0000","last_fullsized":"2023-10-18T10:43:04.409001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:28.022888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563896+0000","last_change":"2023-10-18T10:43:02.965103+0000","last_active":"2023-10-18T10:43:04.563896+0000","last_peered":"2023-10-18T10:43:04.563896+0000","last_clean":"2023-10-18T10:43:04.563896+0000","last_became_active":"2023-10-18T10:43:02.964812+0000","last_became_peered":"2023-10-18T10:43:02.964812+0000","last_unstale":"2023-10-18T10:43:04.563896+0000","last_undegraded":"2023-10-18T10:43:04.563896+0000","last_fullsized":"2023-10-18T10:43:04.563896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:08.011562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564304+0000","last_change":"2023-10-18T10:43:03.424682+0000","last_active":"2023-10-18T10:43:04.564304+0000","last_peered":"2023-10-18T10:43:04.564304+0000","last_clean":"2023-10-18T10:43:04.564304+0000","last_became_active":"2023-10-18T10:43:03.424416+0000","last_became_peered":"2023-10-18T10:43:03.424416+0000","last_unstale":"2023-10-18T10:43:04.564304+0000","last_undegraded":"2023-10-18T10:43:04.564304+0000","last_fullsized":"2023-10-18T10:43:04.564304+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:29.729372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564479+0000","last_change":"2023-10-18T10:43:02.966346+0000","last_active":"2023-10-18T10:43:04.564479+0000","last_peered":"2023-10-18T10:43:04.564479+0000","last_clean":"2023-10-18T10:43:04.564479+0000","last_became_active":"2023-10-18T10:43:02.966041+0000","last_became_peered":"2023-10-18T10:43:02.966041+0000","last_unstale":"2023-10-18T10:43:04.564479+0000","last_undegraded":"2023-10-18T10:43:04.564479+0000","last_fullsized":"2023-10-18T10:43:04.564479+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:04.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565269+0000","last_change":"2023-10-18T10:43:03.424161+0000","last_active":"2023-10-18T10:43:04.565269+0000","last_peered":"2023-10-18T10:43:04.565269+0000","last_clean":"2023-10-18T10:43:04.565269+0000","last_became_active":"2023-10-18T10:43:03.423854+0000","last_became_peered":"2023-10-18T10:43:03.423854+0000","last_unstale":"2023-10-18T10:43:04.565269+0000","last_undegraded":"2023-10-18T10:43:04.565269+0000","last_fullsized":"2023-10-18T10:43:04.565269+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:24.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409024+0000","last_change":"2023-10-18T10:43:03.417182+0000","last_active":"2023-10-18T10:43:04.409024+0000","last_peered":"2023-10-18T10:43:04.409024+0000","last_clean":"2023-10-18T10:43:04.409024+0000","last_became_active":"2023-10-18T10:43:03.416917+0000","last_became_peered":"2023-10-18T10:43:03.416917+0000","last_unstale":"2023-10-18T10:43:04.409024+0000","last_undegraded":"2023-10-18T10:43:04.409024+0000","last_fullsized":"2023-10-18T10:43:04.409024+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:49.418978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408682+0000","last_change":"2023-10-18T10:43:02.969298+0000","last_active":"2023-10-18T10:43:04.408682+0000","last_peered":"2023-10-18T10:43:04.408682+0000","last_clean":"2023-10-18T10:43:04.408682+0000","last_became_active":"2023-10-18T10:43:02.969045+0000","last_became_peered":"2023-10-18T10:43:02.969045+0000","last_unstale":"2023-10-18T10:43:04.408682+0000","last_undegraded":"2023-10-18T10:43:04.408682+0000","last_fullsized":"2023-10-18T10:43:04.408682+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:24.897688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564560+0000","last_change":"2023-10-18T10:43:03.424618+0000","last_active":"2023-10-18T10:43:04.564560+0000","last_peered":"2023-10-18T10:43:04.564560+0000","last_clean":"2023-10-18T10:43:04.564560+0000","last_became_active":"2023-10-18T10:43:03.424369+0000","last_became_peered":"2023-10-18T10:43:03.424369+0000","last_unstale":"2023-10-18T10:43:04.564560+0000","last_undegraded":"2023-10-18T10:43:04.564560+0000","last_fullsized":"2023-10-18T10:43:04.564560+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:25.195869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563691+0000","last_change":"2023-10-18T10:43:03.414895+0000","last_active":"2023-10-18T10:43:04.563691+0000","last_peered":"2023-10-18T10:43:04.563691+0000","last_clean":"2023-10-18T10:43:04.563691+0000","last_became_active":"2023-10-18T10:43:03.414661+0000","last_became_peered":"2023-10-18T10:43:03.414661+0000","last_unstale":"2023-10-18T10:43:04.563691+0000","last_undegraded":"2023-10-18T10:43:04.563691+0000","last_fullsized":"2023-10-18T10:43:04.563691+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:46.699483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564739+0000","last_change":"2023-10-18T10:43:02.971336+0000","last_active":"2023-10-18T10:43:04.564739+0000","last_peered":"2023-10-18T10:43:04.564739+0000","last_clean":"2023-10-18T10:43:04.564739+0000","last_became_active":"2023-10-18T10:43:02.971030+0000","last_became_peered":"2023-10-18T10:43:02.971030+0000","last_unstale":"2023-10-18T10:43:04.564739+0000","last_undegraded":"2023-10-18T10:43:04.564739+0000","last_fullsized":"2023-10-18T10:43:04.564739+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:32:47.348309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563840+0000","last_change":"2023-10-18T10:43:03.420254+0000","last_active":"2023-10-18T10:43:04.563840+0000","last_peered":"2023-10-18T10:43:04.563840+0000","last_clean":"2023-10-18T10:43:04.563840+0000","last_became_active":"2023-10-18T10:43:03.420019+0000","last_became_peered":"2023-10-18T10:43:03.420019+0000","last_unstale":"2023-10-18T10:43:04.563840+0000","last_undegraded":"2023-10-18T10:43:04.563840+0000","last_fullsized":"2023-10-18T10:43:04.563840+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:02.531521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"127'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564189+0000","last_change":"2023-10-18T10:43:02.974197+0000","last_active":"2023-10-18T10:43:04.564189+0000","last_peered":"2023-10-18T10:43:04.564189+0000","last_clean":"2023-10-18T10:43:04.564189+0000","last_became_active":"2023-10-18T10:43:02.973961+0000","last_became_peered":"2023-10-18T10:43:02.973961+0000","last_unstale":"2023-10-18T10:43:04.564189+0000","last_undegraded":"2023-10-18T10:43:04.564189+0000","last_fullsized":"2023-10-18T10:43:04.564189+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:57.736328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564658+0000","last_change":"2023-10-18T10:43:03.417830+0000","last_active":"2023-10-18T10:43:04.564658+0000","last_peered":"2023-10-18T10:43:04.564658+0000","last_clean":"2023-10-18T10:43:04.564658+0000","last_became_active":"2023-10-18T10:43:03.417031+0000","last_became_peered":"2023-10-18T10:43:03.417031+0000","last_unstale":"2023-10-18T10:43:04.564658+0000","last_undegraded":"2023-10-18T10:43:04.564658+0000","last_fullsized":"2023-10-18T10:43:04.564658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:56.478327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"127'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565099+0000","last_change":"2023-10-18T10:43:02.959201+0000","last_active":"2023-10-18T10:43:04.565099+0000","last_peered":"2023-10-18T10:43:04.565099+0000","last_clean":"2023-10-18T10:43:04.565099+0000","last_became_active":"2023-10-18T10:43:02.958888+0000","last_became_peered":"2023-10-18T10:43:02.958888+0000","last_unstale":"2023-10-18T10:43:04.565099+0000","last_undegraded":"2023-10-18T10:43:04.565099+0000","last_fullsized":"2023-10-18T10:43:04.565099+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:04.257834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564597+0000","last_change":"2023-10-18T10:43:03.419410+0000","last_active":"2023-10-18T10:43:04.564597+0000","last_peered":"2023-10-18T10:43:04.564597+0000","last_clean":"2023-10-18T10:43:04.564597+0000","last_became_active":"2023-10-18T10:43:03.419055+0000","last_became_peered":"2023-10-18T10:43:03.419055+0000","last_unstale":"2023-10-18T10:43:04.564597+0000","last_undegraded":"2023-10-18T10:43:04.564597+0000","last_fullsized":"2023-10-18T10:43:04.564597+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:06.895973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564742+0000","last_change":"2023-10-18T10:43:02.972054+0000","last_active":"2023-10-18T10:43:04.564742+0000","last_peered":"2023-10-18T10:43:04.564742+0000","last_clean":"2023-10-18T10:43:04.564742+0000","last_became_active":"2023-10-18T10:43:02.971783+0000","last_became_peered":"2023-10-18T10:43:02.971783+0000","last_unstale":"2023-10-18T10:43:04.564742+0000","last_undegraded":"2023-10-18T10:43:04.564742+0000","last_fullsized":"2023-10-18T10:43:04.564742+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:07.914251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565070+0000","last_change":"2023-10-18T10:43:03.424942+0000","last_active":"2023-10-18T10:43:04.565070+0000","last_peered":"2023-10-18T10:43:04.565070+0000","last_clean":"2023-10-18T10:43:04.565070+0000","last_became_active":"2023-10-18T10:43:03.424610+0000","last_became_peered":"2023-10-18T10:43:03.424610+0000","last_unstale":"2023-10-18T10:43:04.565070+0000","last_undegraded":"2023-10-18T10:43:04.565070+0000","last_fullsized":"2023-10-18T10:43:04.565070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:58.441002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408796+0000","last_change":"2023-10-18T10:43:02.962770+0000","last_active":"2023-10-18T10:43:04.408796+0000","last_peered":"2023-10-18T10:43:04.408796+0000","last_clean":"2023-10-18T10:43:04.408796+0000","last_became_active":"2023-10-18T10:43:02.962492+0000","last_became_peered":"2023-10-18T10:43:02.962492+0000","last_unstale":"2023-10-18T10:43:04.408796+0000","last_undegraded":"2023-10-18T10:43:04.408796+0000","last_fullsized":"2023-10-18T10:43:04.408796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:04.206263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565182+0000","last_change":"2023-10-18T10:43:03.417626+0000","last_active":"2023-10-18T10:43:04.565182+0000","last_peered":"2023-10-18T10:43:04.565182+0000","last_clean":"2023-10-18T10:43:04.565182+0000","last_became_active":"2023-10-18T10:43:03.417374+0000","last_became_peered":"2023-10-18T10:43:03.417374+0000","last_unstale":"2023-10-18T10:43:04.565182+0000","last_undegraded":"2023-10-18T10:43:04.565182+0000","last_fullsized":"2023-10-18T10:43:04.565182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:39.656533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563634+0000","last_change":"2023-10-18T10:43:02.966586+0000","last_active":"2023-10-18T10:43:04.563634+0000","last_peered":"2023-10-18T10:43:04.563634+0000","last_clean":"2023-10-18T10:43:04.563634+0000","last_became_active":"2023-10-18T10:43:02.966340+0000","last_became_peered":"2023-10-18T10:43:02.966340+0000","last_unstale":"2023-10-18T10:43:04.563634+0000","last_undegraded":"2023-10-18T10:43:04.563634+0000","last_fullsized":"2023-10-18T10:43:04.563634+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.288426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565086+0000","last_change":"2023-10-18T10:43:03.425524+0000","last_active":"2023-10-18T10:43:04.565086+0000","last_peered":"2023-10-18T10:43:04.565086+0000","last_clean":"2023-10-18T10:43:04.565086+0000","last_became_active":"2023-10-18T10:43:03.425299+0000","last_became_peered":"2023-10-18T10:43:03.425299+0000","last_unstale":"2023-10-18T10:43:04.565086+0000","last_undegraded":"2023-10-18T10:43:04.565086+0000","last_fullsized":"2023-10-18T10:43:04.565086+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:18.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563658+0000","last_change":"2023-10-18T10:43:03.415655+0000","last_active":"2023-10-18T10:43:04.563658+0000","last_peered":"2023-10-18T10:43:04.563658+0000","last_clean":"2023-10-18T10:43:04.563658+0000","last_became_active":"2023-10-18T10:43:03.415397+0000","last_became_peered":"2023-10-18T10:43:03.415397+0000","last_unstale":"2023-10-18T10:43:04.563658+0000","last_undegraded":"2023-10-18T10:43:04.563658+0000","last_fullsized":"2023-10-18T10:43:04.563658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:17.193216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564284+0000","last_change":"2023-10-18T10:43:02.964287+0000","last_active":"2023-10-18T10:43:04.564284+0000","last_peered":"2023-10-18T10:43:04.564284+0000","last_clean":"2023-10-18T10:43:04.564284+0000","last_became_active":"2023-10-18T10:43:02.964033+0000","last_became_peered":"2023-10-18T10:43:02.964033+0000","last_unstale":"2023-10-18T10:43:04.564284+0000","last_undegraded":"2023-10-18T10:43:04.564284+0000","last_fullsized":"2023-10-18T10:43:04.564284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:33.533023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564963+0000","last_change":"2023-10-18T10:43:03.419976+0000","last_active":"2023-10-18T10:43:04.564963+0000","last_peered":"2023-10-18T10:43:04.564963+0000","last_clean":"2023-10-18T10:43:04.564963+0000","last_became_active":"2023-10-18T10:43:03.419611+0000","last_became_peered":"2023-10-18T10:43:03.419611+0000","last_unstale":"2023-10-18T10:43:04.564963+0000","last_undegraded":"2023-10-18T10:43:04.564963+0000","last_fullsized":"2023-10-18T10:43:04.564963+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:57.163873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564210+0000","last_change":"2023-10-18T10:43:02.964463+0000","last_active":"2023-10-18T10:43:04.564210+0000","last_peered":"2023-10-18T10:43:04.564210+0000","last_clean":"2023-10-18T10:43:04.564210+0000","last_became_active":"2023-10-18T10:43:02.964036+0000","last_became_peered":"2023-10-18T10:43:02.964036+0000","last_unstale":"2023-10-18T10:43:04.564210+0000","last_undegraded":"2023-10-18T10:43:04.564210+0000","last_fullsized":"2023-10-18T10:43:04.564210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:27.017315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564542+0000","last_change":"2023-10-18T10:43:03.416778+0000","last_active":"2023-10-18T10:43:04.564542+0000","last_peered":"2023-10-18T10:43:04.564542+0000","last_clean":"2023-10-18T10:43:04.564542+0000","last_became_active":"2023-10-18T10:43:03.416492+0000","last_became_peered":"2023-10-18T10:43:03.416492+0000","last_unstale":"2023-10-18T10:43:04.564542+0000","last_undegraded":"2023-10-18T10:43:04.564542+0000","last_fullsized":"2023-10-18T10:43:04.564542+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:29.332215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564773+0000","last_change":"2023-10-18T10:43:02.965244+0000","last_active":"2023-10-18T10:43:04.564773+0000","last_peered":"2023-10-18T10:43:04.564773+0000","last_clean":"2023-10-18T10:43:04.564773+0000","last_became_active":"2023-10-18T10:43:02.964962+0000","last_became_peered":"2023-10-18T10:43:02.964962+0000","last_unstale":"2023-10-18T10:43:04.564773+0000","last_undegraded":"2023-10-18T10:43:04.564773+0000","last_fullsized":"2023-10-18T10:43:04.564773+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:10.229768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563816+0000","last_change":"2023-10-18T10:43:03.419372+0000","last_active":"2023-10-18T10:43:04.563816+0000","last_peered":"2023-10-18T10:43:04.563816+0000","last_clean":"2023-10-18T10:43:04.563816+0000","last_became_active":"2023-10-18T10:43:03.419133+0000","last_became_peered":"2023-10-18T10:43:03.419133+0000","last_unstale":"2023-10-18T10:43:04.563816+0000","last_undegraded":"2023-10-18T10:43:04.563816+0000","last_fullsized":"2023-10-18T10:43:04.563816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:25.080148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564382+0000","last_change":"2023-10-18T10:43:02.963180+0000","last_active":"2023-10-18T10:43:04.564382+0000","last_peered":"2023-10-18T10:43:04.564382+0000","last_clean":"2023-10-18T10:43:04.564382+0000","last_became_active":"2023-10-18T10:43:02.962924+0000","last_became_peered":"2023-10-18T10:43:02.962924+0000","last_unstale":"2023-10-18T10:43:04.564382+0000","last_undegraded":"2023-10-18T10:43:04.564382+0000","last_fullsized":"2023-10-18T10:43:04.564382+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:27.605859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564165+0000","last_change":"2023-10-18T10:43:03.424395+0000","last_active":"2023-10-18T10:43:04.564165+0000","last_peered":"2023-10-18T10:43:04.564165+0000","last_clean":"2023-10-18T10:43:04.564165+0000","last_became_active":"2023-10-18T10:43:03.424101+0000","last_became_peered":"2023-10-18T10:43:03.424101+0000","last_unstale":"2023-10-18T10:43:04.564165+0000","last_undegraded":"2023-10-18T10:43:04.564165+0000","last_fullsized":"2023-10-18T10:43:04.564165+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:43.317979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563722+0000","last_change":"2023-10-18T10:43:02.964389+0000","last_active":"2023-10-18T10:43:04.563722+0000","last_peered":"2023-10-18T10:43:04.563722+0000","last_clean":"2023-10-18T10:43:04.563722+0000","last_became_active":"2023-10-18T10:43:02.964134+0000","last_became_peered":"2023-10-18T10:43:02.964134+0000","last_unstale":"2023-10-18T10:43:04.563722+0000","last_undegraded":"2023-10-18T10:43:04.563722+0000","last_fullsized":"2023-10-18T10:43:04.563722+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:27.504789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408932+0000","last_change":"2023-10-18T10:43:03.417105+0000","last_active":"2023-10-18T10:43:04.408932+0000","last_peered":"2023-10-18T10:43:04.408932+0000","last_clean":"2023-10-18T10:43:04.408932+0000","last_became_active":"2023-10-18T10:43:03.416864+0000","last_became_peered":"2023-10-18T10:43:03.416864+0000","last_unstale":"2023-10-18T10:43:04.408932+0000","last_undegraded":"2023-10-18T10:43:04.408932+0000","last_fullsized":"2023-10-18T10:43:04.408932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:09.017135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564862+0000","last_change":"2023-10-18T10:43:02.967733+0000","last_active":"2023-10-18T10:43:04.564862+0000","last_peered":"2023-10-18T10:43:04.564862+0000","last_clean":"2023-10-18T10:43:04.564862+0000","last_became_active":"2023-10-18T10:43:02.967444+0000","last_became_peered":"2023-10-18T10:43:02.967444+0000","last_unstale":"2023-10-18T10:43:04.564862+0000","last_undegraded":"2023-10-18T10:43:04.564862+0000","last_fullsized":"2023-10-18T10:43:04.564862+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:59.437631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564910+0000","last_change":"2023-10-18T10:43:03.418676+0000","last_active":"2023-10-18T10:43:04.564910+0000","last_peered":"2023-10-18T10:43:04.564910+0000","last_clean":"2023-10-18T10:43:04.564910+0000","last_became_active":"2023-10-18T10:43:03.418441+0000","last_became_peered":"2023-10-18T10:43:03.418441+0000","last_unstale":"2023-10-18T10:43:04.564910+0000","last_undegraded":"2023-10-18T10:43:04.564910+0000","last_fullsized":"2023-10-18T10:43:04.564910+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:30.204629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563690+0000","last_change":"2023-10-18T10:43:02.963728+0000","last_active":"2023-10-18T10:43:04.563690+0000","last_peered":"2023-10-18T10:43:04.563690+0000","last_clean":"2023-10-18T10:43:04.563690+0000","last_became_active":"2023-10-18T10:43:02.963467+0000","last_became_peered":"2023-10-18T10:43:02.963467+0000","last_unstale":"2023-10-18T10:43:04.563690+0000","last_undegraded":"2023-10-18T10:43:04.563690+0000","last_fullsized":"2023-10-18T10:43:04.563690+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:38.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564445+0000","last_change":"2023-10-18T10:43:03.421100+0000","last_active":"2023-10-18T10:43:04.564445+0000","last_peered":"2023-10-18T10:43:04.564445+0000","last_clean":"2023-10-18T10:43:04.564445+0000","last_became_active":"2023-10-18T10:43:03.420845+0000","last_became_peered":"2023-10-18T10:43:03.420845+0000","last_unstale":"2023-10-18T10:43:04.564445+0000","last_undegraded":"2023-10-18T10:43:04.564445+0000","last_fullsized":"2023-10-18T10:43:04.564445+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:32:19.619118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564074+0000","last_change":"2023-10-18T10:43:02.971128+0000","last_active":"2023-10-18T10:43:04.564074+0000","last_peered":"2023-10-18T10:43:04.564074+0000","last_clean":"2023-10-18T10:43:04.564074+0000","last_became_active":"2023-10-18T10:43:02.970845+0000","last_became_peered":"2023-10-18T10:43:02.970845+0000","last_unstale":"2023-10-18T10:43:04.564074+0000","last_undegraded":"2023-10-18T10:43:04.564074+0000","last_fullsized":"2023-10-18T10:43:04.564074+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:57.435429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:02.948282+0000","last_change":"2023-10-18T10:43:02.948282+0000","last_active":"2023-10-18T10:43:02.948282+0000","last_peered":"2023-10-18T10:43:02.948282+0000","last_clean":"2023-10-18T10:43:02.948282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:02.948282+0000","last_undegraded":"2023-10-18T10:43:02.948282+0000","last_fullsized":"2023-10-18T10:43:02.948282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564812+0000","last_change":"2023-10-18T10:43:02.972906+0000","last_active":"2023-10-18T10:43:04.564812+0000","last_peered":"2023-10-18T10:43:04.564812+0000","last_clean":"2023-10-18T10:43:04.564812+0000","last_became_active":"2023-10-18T10:43:02.972652+0000","last_became_peered":"2023-10-18T10:43:02.972652+0000","last_unstale":"2023-10-18T10:43:04.564812+0000","last_undegraded":"2023-10-18T10:43:04.564812+0000","last_fullsized":"2023-10-18T10:43:04.564812+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:09.602406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408779+0000","last_change":"2023-10-18T10:43:03.416563+0000","last_active":"2023-10-18T10:43:04.408779+0000","last_peered":"2023-10-18T10:43:04.408779+0000","last_clean":"2023-10-18T10:43:04.408779+0000","last_became_active":"2023-10-18T10:43:03.416321+0000","last_became_peered":"2023-10-18T10:43:03.416321+0000","last_unstale":"2023-10-18T10:43:04.408779+0000","last_undegraded":"2023-10-18T10:43:04.408779+0000","last_fullsized":"2023-10-18T10:43:04.408779+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:57.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565167+0000","last_change":"2023-10-18T10:43:02.962995+0000","last_active":"2023-10-18T10:43:04.565167+0000","last_peered":"2023-10-18T10:43:04.565167+0000","last_clean":"2023-10-18T10:43:04.565167+0000","last_became_active":"2023-10-18T10:43:02.962723+0000","last_became_peered":"2023-10-18T10:43:02.962723+0000","last_unstale":"2023-10-18T10:43:04.565167+0000","last_undegraded":"2023-10-18T10:43:04.565167+0000","last_fullsized":"2023-10-18T10:43:04.565167+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:45.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564038+0000","last_change":"2023-10-18T10:43:03.424431+0000","last_active":"2023-10-18T10:43:04.564038+0000","last_peered":"2023-10-18T10:43:04.564038+0000","last_clean":"2023-10-18T10:43:04.564038+0000","last_became_active":"2023-10-18T10:43:03.424173+0000","last_became_peered":"2023-10-18T10:43:03.424173+0000","last_unstale":"2023-10-18T10:43:04.564038+0000","last_undegraded":"2023-10-18T10:43:04.564038+0000","last_fullsized":"2023-10-18T10:43:04.564038+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:19.931739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564415+0000","last_change":"2023-10-18T10:43:02.968945+0000","last_active":"2023-10-18T10:43:04.564415+0000","last_peered":"2023-10-18T10:43:04.564415+0000","last_clean":"2023-10-18T10:43:04.564415+0000","last_became_active":"2023-10-18T10:43:02.968666+0000","last_became_peered":"2023-10-18T10:43:02.968666+0000","last_unstale":"2023-10-18T10:43:04.564415+0000","last_undegraded":"2023-10-18T10:43:04.564415+0000","last_fullsized":"2023-10-18T10:43:04.564415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:26.843460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564480+0000","last_change":"2023-10-18T10:43:03.421780+0000","last_active":"2023-10-18T10:43:04.564480+0000","last_peered":"2023-10-18T10:43:04.564480+0000","last_clean":"2023-10-18T10:43:04.564480+0000","last_became_active":"2023-10-18T10:43:03.421547+0000","last_became_peered":"2023-10-18T10:43:03.421547+0000","last_unstale":"2023-10-18T10:43:04.564480+0000","last_undegraded":"2023-10-18T10:43:04.564480+0000","last_fullsized":"2023-10-18T10:43:04.564480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:54:47.177774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:43:01.942709+0000","last_change":"2023-10-18T10:43:01.942709+0000","last_active":"2023-10-18T10:43:01.942709+0000","last_peered":"2023-10-18T10:43:01.942709+0000","last_clean":"2023-10-18T10:43:01.942709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:43:01.942709+0000","last_undegraded":"2023-10-18T10:43:01.942709+0000","last_fullsized":"2023-10-18T10:43:01.942709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563859+0000","last_change":"2023-10-18T10:43:03.423032+0000","last_active":"2023-10-18T10:43:04.563859+0000","last_peered":"2023-10-18T10:43:04.563859+0000","last_clean":"2023-10-18T10:43:04.563859+0000","last_became_active":"2023-10-18T10:43:03.422767+0000","last_became_peered":"2023-10-18T10:43:03.422767+0000","last_unstale":"2023-10-18T10:43:04.563859+0000","last_undegraded":"2023-10-18T10:43:04.563859+0000","last_fullsized":"2023-10-18T10:43:04.563859+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:17.615763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563911+0000","last_change":"2023-10-18T10:43:02.965062+0000","last_active":"2023-10-18T10:43:04.563911+0000","last_peered":"2023-10-18T10:43:04.563911+0000","last_clean":"2023-10-18T10:43:04.563911+0000","last_became_active":"2023-10-18T10:43:02.964753+0000","last_became_peered":"2023-10-18T10:43:02.964753+0000","last_unstale":"2023-10-18T10:43:04.563911+0000","last_undegraded":"2023-10-18T10:43:04.563911+0000","last_fullsized":"2023-10-18T10:43:04.563911+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:41.151549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564340+0000","last_change":"2023-10-18T10:43:03.421221+0000","last_active":"2023-10-18T10:43:04.564340+0000","last_peered":"2023-10-18T10:43:04.564340+0000","last_clean":"2023-10-18T10:43:04.564340+0000","last_became_active":"2023-10-18T10:43:03.420954+0000","last_became_peered":"2023-10-18T10:43:03.420954+0000","last_unstale":"2023-10-18T10:43:04.564340+0000","last_undegraded":"2023-10-18T10:43:04.564340+0000","last_fullsized":"2023-10-18T10:43:04.564340+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:42.945030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564932+0000","last_change":"2023-10-18T10:43:02.962374+0000","last_active":"2023-10-18T10:43:04.564932+0000","last_peered":"2023-10-18T10:43:04.564932+0000","last_clean":"2023-10-18T10:43:04.564932+0000","last_became_active":"2023-10-18T10:43:02.962065+0000","last_became_peered":"2023-10-18T10:43:02.962065+0000","last_unstale":"2023-10-18T10:43:04.564932+0000","last_undegraded":"2023-10-18T10:43:04.564932+0000","last_fullsized":"2023-10-18T10:43:04.564932+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:37.693671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564809+0000","last_change":"2023-10-18T10:43:03.418250+0000","last_active":"2023-10-18T10:43:04.564809+0000","last_peered":"2023-10-18T10:43:04.564809+0000","last_clean":"2023-10-18T10:43:04.564809+0000","last_became_active":"2023-10-18T10:43:03.417934+0000","last_became_peered":"2023-10-18T10:43:03.417934+0000","last_unstale":"2023-10-18T10:43:04.564809+0000","last_undegraded":"2023-10-18T10:43:04.564809+0000","last_fullsized":"2023-10-18T10:43:04.564809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:09.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563736+0000","last_change":"2023-10-18T10:43:02.965897+0000","last_active":"2023-10-18T10:43:04.563736+0000","last_peered":"2023-10-18T10:43:04.563736+0000","last_clean":"2023-10-18T10:43:04.563736+0000","last_became_active":"2023-10-18T10:43:02.965636+0000","last_became_peered":"2023-10-18T10:43:02.965636+0000","last_unstale":"2023-10-18T10:43:04.563736+0000","last_undegraded":"2023-10-18T10:43:04.563736+0000","last_fullsized":"2023-10-18T10:43:04.563736+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:10:39.372132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564606+0000","last_change":"2023-10-18T10:43:03.420416+0000","last_active":"2023-10-18T10:43:04.564606+0000","last_peered":"2023-10-18T10:43:04.564606+0000","last_clean":"2023-10-18T10:43:04.564606+0000","last_became_active":"2023-10-18T10:43:03.420064+0000","last_became_peered":"2023-10-18T10:43:03.420064+0000","last_unstale":"2023-10-18T10:43:04.564606+0000","last_undegraded":"2023-10-18T10:43:04.564606+0000","last_fullsized":"2023-10-18T10:43:04.564606+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:01.892160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564707+0000","last_change":"2023-10-18T10:43:02.962533+0000","last_active":"2023-10-18T10:43:04.564707+0000","last_peered":"2023-10-18T10:43:04.564707+0000","last_clean":"2023-10-18T10:43:04.564707+0000","last_became_active":"2023-10-18T10:43:02.962275+0000","last_became_peered":"2023-10-18T10:43:02.962275+0000","last_unstale":"2023-10-18T10:43:04.564707+0000","last_undegraded":"2023-10-18T10:43:04.564707+0000","last_fullsized":"2023-10-18T10:43:04.564707+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:26.189773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"117'214","reported_seq":611,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563741+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:43:04.563741+0000","last_peered":"2023-10-18T10:43:04.563741+0000","last_clean":"2023-10-18T10:43:04.563741+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:43:04.563741+0000","last_undegraded":"2023-10-18T10:43:04.563741+0000","last_fullsized":"2023-10-18T10:43:04.563741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575068,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59692,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312148,"statfs":{"total":96636764160,"available":96575639552,"internally_reserved":0,"allocated":2187264,"data_stored":1388162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.58299999999999996},"min":{"1min":0.36099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.754,"5min":0.876,"15min":0.88300000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63200000000000001,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70299999999999996,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.60299999999999998},"min":{"1min":0.42199999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.76000000000000001,"5min":0.79800000000000004,"15min":0.85799999999999998},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.68899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.315},"max":{"1min":0.78500000000000003,"5min":0.875,"15min":0.875},"last":0.504}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575068,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":799850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82599999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.69499999999999995,"5min":0.77700000000000002,"15min":0.82699999999999996},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65800000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.432}]},{"osd":3,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.78400000000000003,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.748,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.80300000000000005,"5min":0.82199999999999995,"15min":0.87},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":797634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.39600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88400000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81399999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.81999999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.41199999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.47599999999999998},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71599999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.375},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.496},"min":{"1min":0.373,"5min":0.31,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.38200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77200000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.312,"15min":0.31},"max":{"1min":0.84299999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.443},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.44700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.871,"15min":0.871},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32680,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1523712,"data_stored":765110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57699999999999996,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.77000000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.54700000000000004,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.753,"15min":0.877},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.89300000000000002},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73499999999999999,"5min":0.873,"15min":0.873},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71899999999999997,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.78200000000000003},"last":0.62}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.33200000000000002,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.79200000000000004,"5min":1.446,"15min":1.446},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.81999999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.879,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.52800000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.499,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.624,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.59299999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53400000000000003,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.379},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.434,"15min":0.45100000000000001},"min":{"1min":0.28599999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.80200000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.879,"15min":0.94699999999999995},"last":0.5}]},{"osd":5,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":1.002,"15min":1.002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88900000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.629,"15min":0.63800000000000001},"min":{"1min":0.434,"5min":0.38,"15min":0.37},"max":{"1min":0.82499999999999996,"5min":1.018,"15min":1.018},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.751,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.63900000000000001},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.627,"15min":0.65300000000000002},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54940,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316900,"statfs":{"total":96636764160,"available":96580505600,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.371,"15min":0.374},"min":{"1min":0.252,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.626},"last":0.377}]},{"osd":1,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29899999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.57899999999999996,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.315}]},{"osd":3,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.24299999999999999},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.434,"15min":0.44700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.41899999999999998,"15min":0.254},"max":{"1min":0.86499999999999999,"5min":0.88200000000000001,"15min":0.88800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.625,"15min":0.64500000000000002},"min":{"1min":0.45300000000000001,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.95699999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.63},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.93700000000000006},"last":0.57299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.84599999999999997,"15min":0.92500000000000004},"last":0.54100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36199999999999999,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.48999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.32000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.39200000000000002},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.374},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.42099999999999999},"min":{"1min":0.314,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53600000000000003,"5min":0.627,"15min":0.64800000000000002},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.33100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59499999999999997,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.33500000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.83999999999999997,"15min":0.95599999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.623},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.753,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.88100000000000001,"15min":0.95999999999999996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60199999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.76200000000000001,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.63700000000000001},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.78000000000000003,"5min":1.03,"15min":1.03},"last":0.749}]},{"osd":7,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.63},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78800000000000003,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575067,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2195456,"data_stored":1389704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.39000000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.54600000000000004,"5min":0.62,"15min":0.76900000000000002},"last":0.214}]},{"osd":2,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.187},"max":{"1min":0.58099999999999996,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.41999999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.29299999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.39500000000000002},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56599999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.312}]},{"osd":4,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.61699999999999999},"min":{"1min":0.372,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81999999999999995,"5min":0.996,"15min":0.996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90300000000000002,"5min":1.038,"15min":1.038},"last":0.34200000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.89200000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93700000000000006,"5min":1.026,"15min":1.3700000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88100000000000001,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.64300000000000002},"min":{"1min":0.41199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.05,"15min":1.339},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.64600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:43:07.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2023-10-18T10:43:07.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:02.948282+0000', 'last_change': '2023-10-18T10:43:02.948282+0000', 'last_active': '2023-10-18T10:43:02.948282+0000', 'last_peered': '2023-10-18T10:43:02.948282+0000', 'last_clean': '2023-10-18T10:43:02.948282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:02.948282+0000', 'last_undegraded': '2023-10-18T10:43:02.948282+0000', 'last_fullsized': '2023-10-18T10:43:02.948282+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2023-10-18T10:43:07.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:02.948282+0000', 'last_change': '2023-10-18T10:43:02.948282+0000', 'last_active': '2023-10-18T10:43:02.948282+0000', 'last_peered': '2023-10-18T10:43:02.948282+0000', 'last_clean': '2023-10-18T10:43:02.948282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:02.948282+0000', 'last_undegraded': '2023-10-18T10:43:02.948282+0000', 'last_fullsized': '2023-10-18T10:43:02.948282+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2023-10-18T10:43:07.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:02.948282+0000', 'last_change': '2023-10-18T10:43:02.948282+0000', 'last_active': '2023-10-18T10:43:02.948282+0000', 'last_peered': '2023-10-18T10:43:02.948282+0000', 'last_clean': '2023-10-18T10:43:02.948282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:02.948282+0000', 'last_undegraded': '2023-10-18T10:43:02.948282+0000', 'last_fullsized': '2023-10-18T10:43:02.948282+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2023-10-18T10:43:07.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:02.948282+0000', 'last_change': '2023-10-18T10:43:02.948282+0000', 'last_active': '2023-10-18T10:43:02.948282+0000', 'last_peered': '2023-10-18T10:43:02.948282+0000', 'last_clean': '2023-10-18T10:43:02.948282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:02.948282+0000', 'last_undegraded': '2023-10-18T10:43:02.948282+0000', 'last_fullsized': '2023-10-18T10:43:02.948282+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2023-10-18T10:43:07.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:02.948282+0000', 'last_change': '2023-10-18T10:43:02.948282+0000', 'last_active': '2023-10-18T10:43:02.948282+0000', 'last_peered': '2023-10-18T10:43:02.948282+0000', 'last_clean': '2023-10-18T10:43:02.948282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:02.948282+0000', 'last_undegraded': '2023-10-18T10:43:02.948282+0000', 'last_fullsized': '2023-10-18T10:43:02.948282+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-10-18T10:43:07.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:02.948282+0000', 'last_change': '2023-10-18T10:43:02.948282+0000', 'last_active': '2023-10-18T10:43:02.948282+0000', 'last_peered': '2023-10-18T10:43:02.948282+0000', 'last_clean': '2023-10-18T10:43:02.948282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:02.948282+0000', 'last_undegraded': '2023-10-18T10:43:02.948282+0000', 'last_fullsized': '2023-10-18T10:43:02.948282+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-10-18T10:43:07.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:01.942709+0000', 'last_change': '2023-10-18T10:43:01.942709+0000', 'last_active': '2023-10-18T10:43:01.942709+0000', 'last_peered': '2023-10-18T10:43:01.942709+0000', 'last_clean': '2023-10-18T10:43:01.942709+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:01.942709+0000', 'last_undegraded': '2023-10-18T10:43:01.942709+0000', 'last_fullsized': '2023-10-18T10:43:01.942709+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-10-18T10:43:07.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:01.942709+0000', 'last_change': '2023-10-18T10:43:01.942709+0000', 'last_active': '2023-10-18T10:43:01.942709+0000', 'last_peered': '2023-10-18T10:43:01.942709+0000', 'last_clean': '2023-10-18T10:43:01.942709+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:01.942709+0000', 'last_undegraded': '2023-10-18T10:43:01.942709+0000', 'last_fullsized': '2023-10-18T10:43:01.942709+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-10-18T10:43:07.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:01.942709+0000', 'last_change': '2023-10-18T10:43:01.942709+0000', 'last_active': '2023-10-18T10:43:01.942709+0000', 'last_peered': '2023-10-18T10:43:01.942709+0000', 'last_clean': '2023-10-18T10:43:01.942709+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:01.942709+0000', 'last_undegraded': '2023-10-18T10:43:01.942709+0000', 'last_fullsized': '2023-10-18T10:43:01.942709+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-10-18T10:43:07.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:01.942709+0000', 'last_change': '2023-10-18T10:43:01.942709+0000', 'last_active': '2023-10-18T10:43:01.942709+0000', 'last_peered': '2023-10-18T10:43:01.942709+0000', 'last_clean': '2023-10-18T10:43:01.942709+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:01.942709+0000', 'last_undegraded': '2023-10-18T10:43:01.942709+0000', 'last_fullsized': '2023-10-18T10:43:01.942709+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-10-18T10:43:07.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:02.948282+0000', 'last_change': '2023-10-18T10:43:02.948282+0000', 'last_active': '2023-10-18T10:43:02.948282+0000', 'last_peered': '2023-10-18T10:43:02.948282+0000', 'last_clean': '2023-10-18T10:43:02.948282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:02.948282+0000', 'last_undegraded': '2023-10-18T10:43:02.948282+0000', 'last_fullsized': '2023-10-18T10:43:02.948282+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:02.948282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-10-18T10:43:07.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:43:01.942709+0000', 'last_change': '2023-10-18T10:43:01.942709+0000', 'last_active': '2023-10-18T10:43:01.942709+0000', 'last_peered': '2023-10-18T10:43:01.942709+0000', 'last_clean': '2023-10-18T10:43:01.942709+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:43:01.942709+0000', 'last_undegraded': '2023-10-18T10:43:01.942709+0000', 'last_fullsized': '2023-10-18T10:43:01.942709+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'last_clean_scrub_stamp': '2023-10-18T10:43:01.942709+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:43:07.603 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T10:43:07.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.727+0000 7f9481f3c640 1 -- 172.21.15.125:0/693022223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9464004d00 msgr2=0x7f9464005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.727+0000 7f9481f3c640 1 --2- 172.21.15.125:0/693022223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9464004d00 0x7f9464005180 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9474009ed0 tx=0x7f94640057f0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.727+0000 7f947bfff640 1 -- 172.21.15.125:0/693022223 reap_dead start 2023-10-18T10:43:07.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.727+0000 7f9481f3c640 1 -- 172.21.15.125:0/693022223 shutdown_connections 2023-10-18T10:43:07.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.727+0000 7f9481f3c640 1 -- 172.21.15.125:0/693022223 >> 172.21.15.125:0/693022223 conn(0x7f947c0d0d60 msgr2=0x7f947c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:07.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.727+0000 7f9481f3c640 1 -- 172.21.15.125:0/693022223 shutdown_connections 2023-10-18T10:43:07.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.728+0000 7f9481f3c640 1 -- 172.21.15.125:0/693022223 wait complete. 2023-10-18T10:43:07.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.729+0000 7f9481f3c640 1 Processor -- start 2023-10-18T10:43:07.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.729+0000 7f9481f3c640 1 -- start start 2023-10-18T10:43:07.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.729+0000 7f9481f3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9464004d00 0x7f947c14f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.729+0000 7f9481f3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f947c15d1b0 0x7f947c0b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.729+0000 7f9481f3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947c15c780 0x7f947c15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.729+0000 7f9481f3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c15b720 con 0x7f947c15d1b0 2023-10-18T10:43:07.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.729+0000 7f9481f3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c15b9c0 con 0x7f947c15c780 2023-10-18T10:43:07.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.729+0000 7f9481f3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c15bca0 con 0x7f9464004d00 2023-10-18T10:43:07.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f947c15d1b0 0x7f947c0b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f947c15d1b0 0x7f947c0b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54532/0 (socket says 172.21.15.125:54532) 2023-10-18T10:43:07.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 -- 172.21.15.125:0/316573043 learned_addr learned my addr 172.21.15.125:0/316573043 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:07.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947b7fe640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9464004d00 0x7f947c14f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947bfff640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947c15c780 0x7f947c15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 -- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9464004d00 msgr2=0x7f947c14f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9464004d00 0x7f947c14f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 -- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947c15c780 msgr2=0x7f947c15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947c15c780 0x7f947c15d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 -- 172.21.15.125:0/316573043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c15bf80 con 0x7f947c15d1b0 2023-10-18T10:43:07.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947b7fe640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9464004d00 0x7f947c14f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:43:07.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f947affd640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f947c15d1b0 0x7f947c0b6040 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9470001050 tx=0x7f9470003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:07.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f9463fff640 1 -- 172.21.15.125:0/316573043 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000f960 con 0x7f947c15d1b0 2023-10-18T10:43:07.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f9463fff640 1 -- 172.21.15.125:0/316573043 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f947000fac0 con 0x7f947c15d1b0 2023-10-18T10:43:07.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f9463fff640 1 -- 172.21.15.125:0/316573043 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000fe90 con 0x7f947c15d1b0 2023-10-18T10:43:07.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c15c260 con 0x7f947c15d1b0 2023-10-18T10:43:07.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.730+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c14f9e0 con 0x7f947c15d1b0 2023-10-18T10:43:07.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.731+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c15c4a0 con 0x7f947c15d1b0 2023-10-18T10:43:07.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.732+0000 7f9463fff640 1 -- 172.21.15.125:0/316573043 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9470011430 con 0x7f947c15d1b0 2023-10-18T10:43:07.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.732+0000 7f9463fff640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f946806af90 0x7f946806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:07.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.732+0000 7f9463fff640 1 -- 172.21.15.125:0/316573043 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f947008a810 con 0x7f947c15d1b0 2023-10-18T10:43:07.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.732+0000 7f947b7fe640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f946806af90 0x7f946806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:07.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.733+0000 7f947b7fe640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f946806af90 0x7f946806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f947c06b460 tx=0x7f946c005f90 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:07.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.736+0000 7f9463fff640 1 -- 172.21.15.125:0/316573043 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f947c15c4a0 con 0x7f947c15d1b0 2023-10-18T10:43:07.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.964+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f947c15c4a0 con 0x7f947c15d1b0 2023-10-18T10:43:07.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.965+0000 7f9463fff640 1 -- 172.21.15.125:0/316573043 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1090 (secure 0 0 0) 0x7f947c15c4a0 con 0x7f947c15d1b0 2023-10-18T10:43:07.967 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:07.967 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1109,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":128,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":85},{"state_name":"unknown","count":12}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592778,"bytes_used":323350528,"bytes_avail":772770762752,"bytes_total":773094113280,"unknown_pgs_ratio":0.12371134012937546,"write_bytes_sec":3583,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":47,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8613}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-18T10:42:45.920324+0000","services":{}},"progress_events":{}} 2023-10-18T10:43:07.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f946806af90 msgr2=0x7f946806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f946806af90 0x7f946806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f947c06b460 tx=0x7f946c005f90 comp rx=0 tx=0).stop 2023-10-18T10:43:07.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f947c15d1b0 msgr2=0x7f947c0b6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:07.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f947c15d1b0 0x7f947c0b6040 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9470001050 tx=0x7f9470003a80 comp rx=0 tx=0).stop 2023-10-18T10:43:07.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 shutdown_connections 2023-10-18T10:43:07.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f946806af90 0x7f946806d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f947c15c780 0x7f947c15d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f947c15d1b0 0x7f947c0b6040 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 --2- 172.21.15.125:0/316573043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9464004d00 0x7f947c14f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:07.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 >> 172.21.15.125:0/316573043 conn(0x7f947c0d0d60 msgr2=0x7f947c06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:07.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 shutdown_connections 2023-10-18T10:43:07.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:07.967+0000 7f9481f3c640 1 -- 172.21.15.125:0/316573043 wait complete. 2023-10-18T10:43:07.979 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T10:43:07.980 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:43:08.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.080+0000 7f42020f2640 1 -- 172.21.15.125:0/4150285430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc15b2a0 msgr2=0x7f41fc0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:08.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.080+0000 7f42020f2640 1 --2- 172.21.15.125:0/4150285430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc15b2a0 0x7f41fc0cf720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005fd0 tx=0x7f41f002bd90 comp rx=0 tx=0).stop 2023-10-18T10:43:08.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.080+0000 7f42020f2640 1 -- 172.21.15.125:0/4150285430 shutdown_connections 2023-10-18T10:43:08.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.080+0000 7f42020f2640 1 --2- 172.21.15.125:0/4150285430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc15b2a0 0x7f41fc0cf720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.080+0000 7f42020f2640 1 --2- 172.21.15.125:0/4150285430 >> v2:172.21.15.139:3300/0 conn(0x7f41fc0cf340 0x7f41fc0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.080+0000 7f42020f2640 1 -- 172.21.15.125:0/4150285430 >> 172.21.15.125:0/4150285430 conn(0x7f41fc0d0d60 msgr2=0x7f41fc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:08.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.080+0000 7f42020f2640 1 -- 172.21.15.125:0/4150285430 shutdown_connections 2023-10-18T10:43:08.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.080+0000 7f42020f2640 1 -- 172.21.15.125:0/4150285430 wait complete. 2023-10-18T10:43:08.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.081+0000 7f42020f2640 1 Processor -- start 2023-10-18T10:43:08.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.082+0000 7f42020f2640 1 -- start start 2023-10-18T10:43:08.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.082+0000 7f42020f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc0ceb40 0x7f41fc14a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:08.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.082+0000 7f42020f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41fc0cf340 0x7f41fc06b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:08.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.082+0000 7f42020f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc15b2a0 0x7f41fc0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:08.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.082+0000 7f42020f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0b4fd0 con 0x7f41fc0cf340 2023-10-18T10:43:08.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.082+0000 7f42020f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc15e130 con 0x7f41fc15b2a0 2023-10-18T10:43:08.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.082+0000 7f42020f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc15e940 con 0x7f41fc0ceb40 2023-10-18T10:43:08.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41fc0cf340 0x7f41fc06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:08.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41fc0cf340 0x7f41fc06b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54540/0 (socket says 172.21.15.125:54540) 2023-10-18T10:43:08.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 -- 172.21.15.125:0/1103979980 learned_addr learned my addr 172.21.15.125:0/1103979980 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:08.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41fb7fe640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc0ceb40 0x7f41fc14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:08.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41fbfff640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc15b2a0 0x7f41fc0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:08.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 -- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc0ceb40 msgr2=0x7f41fc14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:08.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc0ceb40 0x7f41fc14a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 -- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc15b2a0 msgr2=0x7f41fc0c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:08.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc15b2a0 0x7f41fc0c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 -- 172.21.15.125:0/1103979980 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0009560 con 0x7f41fc0cf340 2023-10-18T10:43:08.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41fbfff640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc15b2a0 0x7f41fc0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:43:08.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41fb7fe640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc0ceb40 0x7f41fc14a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:43:08.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41faffd640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41fc0cf340 0x7f41fc06b450 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006010 tx=0x7f41f002d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:08.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41e3fff640 1 -- 172.21.15.125:0/1103979980 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f002ddf0 con 0x7f41fc0cf340 2023-10-18T10:43:08.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41e3fff640 1 -- 172.21.15.125:0/1103979980 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f41f00036e0 con 0x7f41fc0cf340 2023-10-18T10:43:08.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc15e380 con 0x7f41fc0cf340 2023-10-18T10:43:08.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f41e3fff640 1 -- 172.21.15.125:0/1103979980 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0003af0 con 0x7f41fc0cf340 2023-10-18T10:43:08.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.083+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc0b5200 con 0x7f41fc0cf340 2023-10-18T10:43:08.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.084+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc0b5420 con 0x7f41fc0cf340 2023-10-18T10:43:08.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.085+0000 7f41e3fff640 1 -- 172.21.15.125:0/1103979980 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f41f0003c50 con 0x7f41fc0cf340 2023-10-18T10:43:08.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.085+0000 7f41e3fff640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f41e806af60 0x7f41e806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:08.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.085+0000 7f41e3fff640 1 -- 172.21.15.125:0/1103979980 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f41f00ac3c0 con 0x7f41fc0cf340 2023-10-18T10:43:08.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.085+0000 7f41fb7fe640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f41e806af60 0x7f41e806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:08.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.086+0000 7f41fb7fe640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f41e806af60 0x7f41e806d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009c20 tx=0x7f41ec005f90 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:08.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.089+0000 7f41e3fff640 1 -- 172.21.15.125:0/1103979980 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41f0077480 con 0x7f41fc0cf340 2023-10-18T10:43:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.252+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 --> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41fc0cc170 con 0x7f41e806af60 2023-10-18T10:43:08.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.260+0000 7f41e3fff640 1 -- 172.21.15.125:0/1103979980 <== mgr.7360 v2:172.21.15.139:6832/1511464628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289729 (secure 0 0 0) 0x7f41ec009950 con 0x7f41e806af60 2023-10-18T10:43:08.264 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:08.266 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:43:08.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f41e806af60 msgr2=0x7f41e806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:08.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f41e806af60 0x7f41e806d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009c20 tx=0x7f41ec005f90 comp rx=0 tx=0).stop 2023-10-18T10:43:08.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41fc0cf340 msgr2=0x7f41fc06b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:08.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41fc0cf340 0x7f41fc06b450 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006010 tx=0x7f41f002d9f0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 shutdown_connections 2023-10-18T10:43:08.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f41e806af60 0x7f41e806d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc15b2a0 0x7f41fc0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41fc0cf340 0x7f41fc06b450 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 --2- 172.21.15.125:0/1103979980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41fc0ceb40 0x7f41fc14a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:08.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 >> 172.21.15.125:0/1103979980 conn(0x7f41fc0d0d60 msgr2=0x7f41fc0adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:08.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 shutdown_connections 2023-10-18T10:43:08.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:08.263+0000 7f42020f2640 1 -- 172.21.15.125:0/1103979980 wait complete. 2023-10-18T10:43:08.276 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:43:07.900322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":379,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315840,"kb_used_data":13608,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754658880,"statfs":{"total":773094113280,"available":772770693120,"internally_reserved":0,"allocated":13934592,"data_stored":7566628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.949019"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564947+0000","last_change":"2023-10-18T10:43:03.420057+0000","last_active":"2023-10-18T10:43:04.564947+0000","last_peered":"2023-10-18T10:43:04.564947+0000","last_clean":"2023-10-18T10:43:04.564947+0000","last_became_active":"2023-10-18T10:43:03.419809+0000","last_became_peered":"2023-10-18T10:43:03.419809+0000","last_unstale":"2023-10-18T10:43:04.564947+0000","last_undegraded":"2023-10-18T10:43:04.564947+0000","last_fullsized":"2023-10-18T10:43:04.564947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:51.310959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565191+0000","last_change":"2023-10-18T10:43:03.422800+0000","last_active":"2023-10-18T10:43:04.565191+0000","last_peered":"2023-10-18T10:43:04.565191+0000","last_clean":"2023-10-18T10:43:04.565191+0000","last_became_active":"2023-10-18T10:43:03.422037+0000","last_became_peered":"2023-10-18T10:43:03.422037+0000","last_unstale":"2023-10-18T10:43:04.565191+0000","last_undegraded":"2023-10-18T10:43:04.565191+0000","last_fullsized":"2023-10-18T10:43:04.565191+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:03.106690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564044+0000","last_change":"2023-10-18T10:43:03.424158+0000","last_active":"2023-10-18T10:43:04.564044+0000","last_peered":"2023-10-18T10:43:04.564044+0000","last_clean":"2023-10-18T10:43:04.564044+0000","last_became_active":"2023-10-18T10:43:03.423915+0000","last_became_peered":"2023-10-18T10:43:03.423915+0000","last_unstale":"2023-10-18T10:43:04.564044+0000","last_undegraded":"2023-10-18T10:43:04.564044+0000","last_fullsized":"2023-10-18T10:43:04.564044+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:49.732624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408937+0000","last_change":"2023-10-18T10:43:03.415009+0000","last_active":"2023-10-18T10:43:04.408937+0000","last_peered":"2023-10-18T10:43:04.408937+0000","last_clean":"2023-10-18T10:43:04.408937+0000","last_became_active":"2023-10-18T10:43:03.414710+0000","last_became_peered":"2023-10-18T10:43:03.414710+0000","last_unstale":"2023-10-18T10:43:04.408937+0000","last_undegraded":"2023-10-18T10:43:04.408937+0000","last_fullsized":"2023-10-18T10:43:04.408937+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:21.227528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563635+0000","last_change":"2023-10-18T10:43:03.411644+0000","last_active":"2023-10-18T10:43:04.563635+0000","last_peered":"2023-10-18T10:43:04.563635+0000","last_clean":"2023-10-18T10:43:04.563635+0000","last_became_active":"2023-10-18T10:43:03.411381+0000","last_became_peered":"2023-10-18T10:43:03.411381+0000","last_unstale":"2023-10-18T10:43:04.563635+0000","last_undegraded":"2023-10-18T10:43:04.563635+0000","last_fullsized":"2023-10-18T10:43:04.563635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:28.704370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564515+0000","last_change":"2023-10-18T10:43:03.420380+0000","last_active":"2023-10-18T10:43:04.564515+0000","last_peered":"2023-10-18T10:43:04.564515+0000","last_clean":"2023-10-18T10:43:04.564515+0000","last_became_active":"2023-10-18T10:43:03.420112+0000","last_became_peered":"2023-10-18T10:43:03.420112+0000","last_unstale":"2023-10-18T10:43:04.564515+0000","last_undegraded":"2023-10-18T10:43:04.564515+0000","last_fullsized":"2023-10-18T10:43:04.564515+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:15.468100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564645+0000","last_change":"2023-10-18T10:43:03.417853+0000","last_active":"2023-10-18T10:43:04.564645+0000","last_peered":"2023-10-18T10:43:04.564645+0000","last_clean":"2023-10-18T10:43:04.564645+0000","last_became_active":"2023-10-18T10:43:03.417585+0000","last_became_peered":"2023-10-18T10:43:03.417585+0000","last_unstale":"2023-10-18T10:43:04.564645+0000","last_undegraded":"2023-10-18T10:43:04.564645+0000","last_fullsized":"2023-10-18T10:43:04.564645+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:57.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563425+0000","last_change":"2023-10-18T10:43:03.420896+0000","last_active":"2023-10-18T10:43:04.563425+0000","last_peered":"2023-10-18T10:43:04.563425+0000","last_clean":"2023-10-18T10:43:04.563425+0000","last_became_active":"2023-10-18T10:43:03.420633+0000","last_became_peered":"2023-10-18T10:43:03.420633+0000","last_unstale":"2023-10-18T10:43:04.563425+0000","last_undegraded":"2023-10-18T10:43:04.563425+0000","last_fullsized":"2023-10-18T10:43:04.563425+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:54.658345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564188+0000","last_change":"2023-10-18T10:43:03.423222+0000","last_active":"2023-10-18T10:43:04.564188+0000","last_peered":"2023-10-18T10:43:04.564188+0000","last_clean":"2023-10-18T10:43:04.564188+0000","last_became_active":"2023-10-18T10:43:03.422095+0000","last_became_peered":"2023-10-18T10:43:03.422095+0000","last_unstale":"2023-10-18T10:43:04.564188+0000","last_undegraded":"2023-10-18T10:43:04.564188+0000","last_fullsized":"2023-10-18T10:43:04.564188+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:36.033906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564111+0000","last_change":"2023-10-18T10:43:03.420436+0000","last_active":"2023-10-18T10:43:04.564111+0000","last_peered":"2023-10-18T10:43:04.564111+0000","last_clean":"2023-10-18T10:43:04.564111+0000","last_became_active":"2023-10-18T10:43:03.419414+0000","last_became_peered":"2023-10-18T10:43:03.419414+0000","last_unstale":"2023-10-18T10:43:04.564111+0000","last_undegraded":"2023-10-18T10:43:04.564111+0000","last_fullsized":"2023-10-18T10:43:04.564111+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:27.017090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564170+0000","last_change":"2023-10-18T10:43:03.425870+0000","last_active":"2023-10-18T10:43:04.564170+0000","last_peered":"2023-10-18T10:43:04.564170+0000","last_clean":"2023-10-18T10:43:04.564170+0000","last_became_active":"2023-10-18T10:43:03.425583+0000","last_became_peered":"2023-10-18T10:43:03.425583+0000","last_unstale":"2023-10-18T10:43:04.564170+0000","last_undegraded":"2023-10-18T10:43:04.564170+0000","last_fullsized":"2023-10-18T10:43:04.564170+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:07.373278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414558+0000","last_change":"2023-10-18T10:43:03.421791+0000","last_active":"2023-10-18T10:43:04.414558+0000","last_peered":"2023-10-18T10:43:04.414558+0000","last_clean":"2023-10-18T10:43:04.414558+0000","last_became_active":"2023-10-18T10:43:03.421296+0000","last_became_peered":"2023-10-18T10:43:03.421296+0000","last_unstale":"2023-10-18T10:43:04.414558+0000","last_undegraded":"2023-10-18T10:43:04.414558+0000","last_fullsized":"2023-10-18T10:43:04.414558+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:23:15.369598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408816+0000","last_change":"2023-10-18T10:43:03.421921+0000","last_active":"2023-10-18T10:43:04.408816+0000","last_peered":"2023-10-18T10:43:04.408816+0000","last_clean":"2023-10-18T10:43:04.408816+0000","last_became_active":"2023-10-18T10:43:03.421644+0000","last_became_peered":"2023-10-18T10:43:03.421644+0000","last_unstale":"2023-10-18T10:43:04.408816+0000","last_undegraded":"2023-10-18T10:43:04.408816+0000","last_fullsized":"2023-10-18T10:43:04.408816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:58.741901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563868+0000","last_change":"2023-10-18T10:43:03.415137+0000","last_active":"2023-10-18T10:43:04.563868+0000","last_peered":"2023-10-18T10:43:04.563868+0000","last_clean":"2023-10-18T10:43:04.563868+0000","last_became_active":"2023-10-18T10:43:03.414871+0000","last_became_peered":"2023-10-18T10:43:03.414871+0000","last_unstale":"2023-10-18T10:43:04.563868+0000","last_undegraded":"2023-10-18T10:43:04.563868+0000","last_fullsized":"2023-10-18T10:43:04.563868+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:33.182433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563789+0000","last_change":"2023-10-18T10:43:03.420511+0000","last_active":"2023-10-18T10:43:04.563789+0000","last_peered":"2023-10-18T10:43:04.563789+0000","last_clean":"2023-10-18T10:43:04.563789+0000","last_became_active":"2023-10-18T10:43:03.420241+0000","last_became_peered":"2023-10-18T10:43:03.420241+0000","last_unstale":"2023-10-18T10:43:04.563789+0000","last_undegraded":"2023-10-18T10:43:04.563789+0000","last_fullsized":"2023-10-18T10:43:04.563789+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:38.074311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563696+0000","last_change":"2023-10-18T10:43:03.417552+0000","last_active":"2023-10-18T10:43:04.563696+0000","last_peered":"2023-10-18T10:43:04.563696+0000","last_clean":"2023-10-18T10:43:04.563696+0000","last_became_active":"2023-10-18T10:43:03.417291+0000","last_became_peered":"2023-10-18T10:43:03.417291+0000","last_unstale":"2023-10-18T10:43:04.563696+0000","last_undegraded":"2023-10-18T10:43:04.563696+0000","last_fullsized":"2023-10-18T10:43:04.563696+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:18.941147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.425285+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.424971+0000","last_became_peered":"2023-10-18T10:43:03.424971+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:00.092514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565204+0000","last_change":"2023-10-18T10:43:03.416033+0000","last_active":"2023-10-18T10:43:04.565204+0000","last_peered":"2023-10-18T10:43:04.565204+0000","last_clean":"2023-10-18T10:43:04.565204+0000","last_became_active":"2023-10-18T10:43:03.415763+0000","last_became_peered":"2023-10-18T10:43:03.415763+0000","last_unstale":"2023-10-18T10:43:04.565204+0000","last_undegraded":"2023-10-18T10:43:04.565204+0000","last_fullsized":"2023-10-18T10:43:04.565204+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:36.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564171+0000","last_change":"2023-10-18T10:43:03.418626+0000","last_active":"2023-10-18T10:43:04.564171+0000","last_peered":"2023-10-18T10:43:04.564171+0000","last_clean":"2023-10-18T10:43:04.564171+0000","last_became_active":"2023-10-18T10:43:03.418344+0000","last_became_peered":"2023-10-18T10:43:03.418344+0000","last_unstale":"2023-10-18T10:43:04.564171+0000","last_undegraded":"2023-10-18T10:43:04.564171+0000","last_fullsized":"2023-10-18T10:43:04.564171+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:55.314298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414494+0000","last_change":"2023-10-18T10:43:03.416156+0000","last_active":"2023-10-18T10:43:04.414494+0000","last_peered":"2023-10-18T10:43:04.414494+0000","last_clean":"2023-10-18T10:43:04.414494+0000","last_became_active":"2023-10-18T10:43:03.415839+0000","last_became_peered":"2023-10-18T10:43:03.415839+0000","last_unstale":"2023-10-18T10:43:04.414494+0000","last_undegraded":"2023-10-18T10:43:04.414494+0000","last_fullsized":"2023-10-18T10:43:04.414494+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:24.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564184+0000","last_change":"2023-10-18T10:43:03.421228+0000","last_active":"2023-10-18T10:43:04.564184+0000","last_peered":"2023-10-18T10:43:04.564184+0000","last_clean":"2023-10-18T10:43:04.564184+0000","last_became_active":"2023-10-18T10:43:03.420908+0000","last_became_peered":"2023-10-18T10:43:03.420908+0000","last_unstale":"2023-10-18T10:43:04.564184+0000","last_undegraded":"2023-10-18T10:43:04.564184+0000","last_fullsized":"2023-10-18T10:43:04.564184+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:19.745476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564166+0000","last_change":"2023-10-18T10:43:03.421765+0000","last_active":"2023-10-18T10:43:04.564166+0000","last_peered":"2023-10-18T10:43:04.564166+0000","last_clean":"2023-10-18T10:43:04.564166+0000","last_became_active":"2023-10-18T10:43:03.421501+0000","last_became_peered":"2023-10-18T10:43:03.421501+0000","last_unstale":"2023-10-18T10:43:04.564166+0000","last_undegraded":"2023-10-18T10:43:04.564166+0000","last_fullsized":"2023-10-18T10:43:04.564166+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:06.267958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.424152+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.423890+0000","last_became_peered":"2023-10-18T10:43:03.423890+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:26.100932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564263+0000","last_change":"2023-10-18T10:43:03.417440+0000","last_active":"2023-10-18T10:43:04.564263+0000","last_peered":"2023-10-18T10:43:04.564263+0000","last_clean":"2023-10-18T10:43:04.564263+0000","last_became_active":"2023-10-18T10:43:03.417201+0000","last_became_peered":"2023-10-18T10:43:03.417201+0000","last_unstale":"2023-10-18T10:43:04.564263+0000","last_undegraded":"2023-10-18T10:43:04.564263+0000","last_fullsized":"2023-10-18T10:43:04.564263+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:16.834481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414558+0000","last_change":"2023-10-18T10:43:03.417026+0000","last_active":"2023-10-18T10:43:04.414558+0000","last_peered":"2023-10-18T10:43:04.414558+0000","last_clean":"2023-10-18T10:43:04.414558+0000","last_became_active":"2023-10-18T10:43:03.416727+0000","last_became_peered":"2023-10-18T10:43:03.416727+0000","last_unstale":"2023-10-18T10:43:04.414558+0000","last_undegraded":"2023-10-18T10:43:04.414558+0000","last_fullsized":"2023-10-18T10:43:04.414558+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:05.181231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414718+0000","last_change":"2023-10-18T10:43:03.418317+0000","last_active":"2023-10-18T10:43:04.414718+0000","last_peered":"2023-10-18T10:43:04.414718+0000","last_clean":"2023-10-18T10:43:04.414718+0000","last_became_active":"2023-10-18T10:43:03.418040+0000","last_became_peered":"2023-10-18T10:43:03.418040+0000","last_unstale":"2023-10-18T10:43:04.414718+0000","last_undegraded":"2023-10-18T10:43:04.414718+0000","last_fullsized":"2023-10-18T10:43:04.414718+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:27.849033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564364+0000","last_change":"2023-10-18T10:43:03.419666+0000","last_active":"2023-10-18T10:43:04.564364+0000","last_peered":"2023-10-18T10:43:04.564364+0000","last_clean":"2023-10-18T10:43:04.564364+0000","last_became_active":"2023-10-18T10:43:03.418843+0000","last_became_peered":"2023-10-18T10:43:03.418843+0000","last_unstale":"2023-10-18T10:43:04.564364+0000","last_undegraded":"2023-10-18T10:43:04.564364+0000","last_fullsized":"2023-10-18T10:43:04.564364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:25.903261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565004+0000","last_change":"2023-10-18T10:43:03.423183+0000","last_active":"2023-10-18T10:43:04.565004+0000","last_peered":"2023-10-18T10:43:04.565004+0000","last_clean":"2023-10-18T10:43:04.565004+0000","last_became_active":"2023-10-18T10:43:03.422819+0000","last_became_peered":"2023-10-18T10:43:03.422819+0000","last_unstale":"2023-10-18T10:43:04.565004+0000","last_undegraded":"2023-10-18T10:43:04.565004+0000","last_fullsized":"2023-10-18T10:43:04.565004+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:34.324413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564301+0000","last_change":"2023-10-18T10:43:03.417982+0000","last_active":"2023-10-18T10:43:04.564301+0000","last_peered":"2023-10-18T10:43:04.564301+0000","last_clean":"2023-10-18T10:43:04.564301+0000","last_became_active":"2023-10-18T10:43:03.417736+0000","last_became_peered":"2023-10-18T10:43:03.417736+0000","last_unstale":"2023-10-18T10:43:04.564301+0000","last_undegraded":"2023-10-18T10:43:04.564301+0000","last_fullsized":"2023-10-18T10:43:04.564301+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:51.640858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414313+0000","last_change":"2023-10-18T10:43:03.422207+0000","last_active":"2023-10-18T10:43:04.414313+0000","last_peered":"2023-10-18T10:43:04.414313+0000","last_clean":"2023-10-18T10:43:04.414313+0000","last_became_active":"2023-10-18T10:43:03.421941+0000","last_became_peered":"2023-10-18T10:43:03.421941+0000","last_unstale":"2023-10-18T10:43:04.414313+0000","last_undegraded":"2023-10-18T10:43:04.414313+0000","last_fullsized":"2023-10-18T10:43:04.414313+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:36.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564774+0000","last_change":"2023-10-18T10:43:03.416816+0000","last_active":"2023-10-18T10:43:04.564774+0000","last_peered":"2023-10-18T10:43:04.564774+0000","last_clean":"2023-10-18T10:43:04.564774+0000","last_became_active":"2023-10-18T10:43:03.416441+0000","last_became_peered":"2023-10-18T10:43:03.416441+0000","last_unstale":"2023-10-18T10:43:04.564774+0000","last_undegraded":"2023-10-18T10:43:04.564774+0000","last_fullsized":"2023-10-18T10:43:04.564774+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:23.903407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563541+0000","last_change":"2023-10-18T10:43:03.423095+0000","last_active":"2023-10-18T10:43:04.563541+0000","last_peered":"2023-10-18T10:43:04.563541+0000","last_clean":"2023-10-18T10:43:04.563541+0000","last_became_active":"2023-10-18T10:43:03.422840+0000","last_became_peered":"2023-10-18T10:43:03.422840+0000","last_unstale":"2023-10-18T10:43:04.563541+0000","last_undegraded":"2023-10-18T10:43:04.563541+0000","last_fullsized":"2023-10-18T10:43:04.563541+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:09.585007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564675+0000","last_change":"2023-10-18T10:43:03.423114+0000","last_active":"2023-10-18T10:43:04.564675+0000","last_peered":"2023-10-18T10:43:04.564675+0000","last_clean":"2023-10-18T10:43:04.564675+0000","last_became_active":"2023-10-18T10:43:03.422862+0000","last_became_peered":"2023-10-18T10:43:03.422862+0000","last_unstale":"2023-10-18T10:43:04.564675+0000","last_undegraded":"2023-10-18T10:43:04.564675+0000","last_fullsized":"2023-10-18T10:43:04.564675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:40:54.685089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"127'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408994+0000","last_change":"2023-10-18T10:43:02.963797+0000","last_active":"2023-10-18T10:43:04.408994+0000","last_peered":"2023-10-18T10:43:04.408994+0000","last_clean":"2023-10-18T10:43:04.408994+0000","last_became_active":"2023-10-18T10:43:02.963525+0000","last_became_peered":"2023-10-18T10:43:02.963525+0000","last_unstale":"2023-10-18T10:43:04.408994+0000","last_undegraded":"2023-10-18T10:43:04.408994+0000","last_fullsized":"2023-10-18T10:43:04.408994+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:18:22.535089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563557+0000","last_change":"2023-10-18T10:43:03.419783+0000","last_active":"2023-10-18T10:43:04.563557+0000","last_peered":"2023-10-18T10:43:04.563557+0000","last_clean":"2023-10-18T10:43:04.563557+0000","last_became_active":"2023-10-18T10:43:03.419543+0000","last_became_peered":"2023-10-18T10:43:03.419543+0000","last_unstale":"2023-10-18T10:43:04.563557+0000","last_undegraded":"2023-10-18T10:43:04.563557+0000","last_fullsized":"2023-10-18T10:43:04.563557+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:39.668528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564617+0000","last_change":"2023-10-18T10:43:02.965235+0000","last_active":"2023-10-18T10:43:04.564617+0000","last_peered":"2023-10-18T10:43:04.564617+0000","last_clean":"2023-10-18T10:43:04.564617+0000","last_became_active":"2023-10-18T10:43:02.964880+0000","last_became_peered":"2023-10-18T10:43:02.964880+0000","last_unstale":"2023-10-18T10:43:04.564617+0000","last_undegraded":"2023-10-18T10:43:04.564617+0000","last_fullsized":"2023-10-18T10:43:04.564617+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:48.610823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414516+0000","last_change":"2023-10-18T10:43:03.408432+0000","last_active":"2023-10-18T10:43:04.414516+0000","last_peered":"2023-10-18T10:43:04.414516+0000","last_clean":"2023-10-18T10:43:04.414516+0000","last_became_active":"2023-10-18T10:43:03.408124+0000","last_became_peered":"2023-10-18T10:43:03.408124+0000","last_unstale":"2023-10-18T10:43:04.414516+0000","last_undegraded":"2023-10-18T10:43:04.414516+0000","last_fullsized":"2023-10-18T10:43:04.414516+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:15.686252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414444+0000","last_change":"2023-10-18T10:43:02.970656+0000","last_active":"2023-10-18T10:43:04.414444+0000","last_peered":"2023-10-18T10:43:04.414444+0000","last_clean":"2023-10-18T10:43:04.414444+0000","last_became_active":"2023-10-18T10:43:02.970399+0000","last_became_peered":"2023-10-18T10:43:02.970399+0000","last_unstale":"2023-10-18T10:43:04.414444+0000","last_undegraded":"2023-10-18T10:43:04.414444+0000","last_fullsized":"2023-10-18T10:43:04.414444+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:22.897834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564182+0000","last_change":"2023-10-18T10:43:03.416904+0000","last_active":"2023-10-18T10:43:04.564182+0000","last_peered":"2023-10-18T10:43:04.564182+0000","last_clean":"2023-10-18T10:43:04.564182+0000","last_became_active":"2023-10-18T10:43:03.416489+0000","last_became_peered":"2023-10-18T10:43:03.416489+0000","last_unstale":"2023-10-18T10:43:04.564182+0000","last_undegraded":"2023-10-18T10:43:04.564182+0000","last_fullsized":"2023-10-18T10:43:04.564182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:36.136092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564337+0000","last_change":"2023-10-18T10:43:02.969842+0000","last_active":"2023-10-18T10:43:04.564337+0000","last_peered":"2023-10-18T10:43:04.564337+0000","last_clean":"2023-10-18T10:43:04.564337+0000","last_became_active":"2023-10-18T10:43:02.969620+0000","last_became_peered":"2023-10-18T10:43:02.969620+0000","last_unstale":"2023-10-18T10:43:04.564337+0000","last_undegraded":"2023-10-18T10:43:04.564337+0000","last_fullsized":"2023-10-18T10:43:04.564337+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:23:14.364024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409001+0000","last_change":"2023-10-18T10:43:03.417690+0000","last_active":"2023-10-18T10:43:04.409001+0000","last_peered":"2023-10-18T10:43:04.409001+0000","last_clean":"2023-10-18T10:43:04.409001+0000","last_became_active":"2023-10-18T10:43:03.417452+0000","last_became_peered":"2023-10-18T10:43:03.417452+0000","last_unstale":"2023-10-18T10:43:04.409001+0000","last_undegraded":"2023-10-18T10:43:04.409001+0000","last_fullsized":"2023-10-18T10:43:04.409001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:28.022888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563896+0000","last_change":"2023-10-18T10:43:02.965103+0000","last_active":"2023-10-18T10:43:04.563896+0000","last_peered":"2023-10-18T10:43:04.563896+0000","last_clean":"2023-10-18T10:43:04.563896+0000","last_became_active":"2023-10-18T10:43:02.964812+0000","last_became_peered":"2023-10-18T10:43:02.964812+0000","last_unstale":"2023-10-18T10:43:04.563896+0000","last_undegraded":"2023-10-18T10:43:04.563896+0000","last_fullsized":"2023-10-18T10:43:04.563896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:08.011562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564304+0000","last_change":"2023-10-18T10:43:03.424682+0000","last_active":"2023-10-18T10:43:04.564304+0000","last_peered":"2023-10-18T10:43:04.564304+0000","last_clean":"2023-10-18T10:43:04.564304+0000","last_became_active":"2023-10-18T10:43:03.424416+0000","last_became_peered":"2023-10-18T10:43:03.424416+0000","last_unstale":"2023-10-18T10:43:04.564304+0000","last_undegraded":"2023-10-18T10:43:04.564304+0000","last_fullsized":"2023-10-18T10:43:04.564304+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:29.729372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564479+0000","last_change":"2023-10-18T10:43:02.966346+0000","last_active":"2023-10-18T10:43:04.564479+0000","last_peered":"2023-10-18T10:43:04.564479+0000","last_clean":"2023-10-18T10:43:04.564479+0000","last_became_active":"2023-10-18T10:43:02.966041+0000","last_became_peered":"2023-10-18T10:43:02.966041+0000","last_unstale":"2023-10-18T10:43:04.564479+0000","last_undegraded":"2023-10-18T10:43:04.564479+0000","last_fullsized":"2023-10-18T10:43:04.564479+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:04.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565269+0000","last_change":"2023-10-18T10:43:03.424161+0000","last_active":"2023-10-18T10:43:04.565269+0000","last_peered":"2023-10-18T10:43:04.565269+0000","last_clean":"2023-10-18T10:43:04.565269+0000","last_became_active":"2023-10-18T10:43:03.423854+0000","last_became_peered":"2023-10-18T10:43:03.423854+0000","last_unstale":"2023-10-18T10:43:04.565269+0000","last_undegraded":"2023-10-18T10:43:04.565269+0000","last_fullsized":"2023-10-18T10:43:04.565269+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:24.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414223+0000","last_change":"2023-10-18T10:43:02.970449+0000","last_active":"2023-10-18T10:43:04.414223+0000","last_peered":"2023-10-18T10:43:04.414223+0000","last_clean":"2023-10-18T10:43:04.414223+0000","last_became_active":"2023-10-18T10:43:02.970145+0000","last_became_peered":"2023-10-18T10:43:02.970145+0000","last_unstale":"2023-10-18T10:43:04.414223+0000","last_undegraded":"2023-10-18T10:43:04.414223+0000","last_fullsized":"2023-10-18T10:43:04.414223+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:56.158300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409024+0000","last_change":"2023-10-18T10:43:03.417182+0000","last_active":"2023-10-18T10:43:04.409024+0000","last_peered":"2023-10-18T10:43:04.409024+0000","last_clean":"2023-10-18T10:43:04.409024+0000","last_became_active":"2023-10-18T10:43:03.416917+0000","last_became_peered":"2023-10-18T10:43:03.416917+0000","last_unstale":"2023-10-18T10:43:04.409024+0000","last_undegraded":"2023-10-18T10:43:04.409024+0000","last_fullsized":"2023-10-18T10:43:04.409024+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:49.418978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408682+0000","last_change":"2023-10-18T10:43:02.969298+0000","last_active":"2023-10-18T10:43:04.408682+0000","last_peered":"2023-10-18T10:43:04.408682+0000","last_clean":"2023-10-18T10:43:04.408682+0000","last_became_active":"2023-10-18T10:43:02.969045+0000","last_became_peered":"2023-10-18T10:43:02.969045+0000","last_unstale":"2023-10-18T10:43:04.408682+0000","last_undegraded":"2023-10-18T10:43:04.408682+0000","last_fullsized":"2023-10-18T10:43:04.408682+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:24.897688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564560+0000","last_change":"2023-10-18T10:43:03.424618+0000","last_active":"2023-10-18T10:43:04.564560+0000","last_peered":"2023-10-18T10:43:04.564560+0000","last_clean":"2023-10-18T10:43:04.564560+0000","last_became_active":"2023-10-18T10:43:03.424369+0000","last_became_peered":"2023-10-18T10:43:03.424369+0000","last_unstale":"2023-10-18T10:43:04.564560+0000","last_undegraded":"2023-10-18T10:43:04.564560+0000","last_fullsized":"2023-10-18T10:43:04.564560+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:25.195869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414588+0000","last_change":"2023-10-18T10:43:02.969790+0000","last_active":"2023-10-18T10:43:04.414588+0000","last_peered":"2023-10-18T10:43:04.414588+0000","last_clean":"2023-10-18T10:43:04.414588+0000","last_became_active":"2023-10-18T10:43:02.969434+0000","last_became_peered":"2023-10-18T10:43:02.969434+0000","last_unstale":"2023-10-18T10:43:04.414588+0000","last_undegraded":"2023-10-18T10:43:04.414588+0000","last_fullsized":"2023-10-18T10:43:04.414588+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:33.318840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563691+0000","last_change":"2023-10-18T10:43:03.414895+0000","last_active":"2023-10-18T10:43:04.563691+0000","last_peered":"2023-10-18T10:43:04.563691+0000","last_clean":"2023-10-18T10:43:04.563691+0000","last_became_active":"2023-10-18T10:43:03.414661+0000","last_became_peered":"2023-10-18T10:43:03.414661+0000","last_unstale":"2023-10-18T10:43:04.563691+0000","last_undegraded":"2023-10-18T10:43:04.563691+0000","last_fullsized":"2023-10-18T10:43:04.563691+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:46.699483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564739+0000","last_change":"2023-10-18T10:43:02.971336+0000","last_active":"2023-10-18T10:43:04.564739+0000","last_peered":"2023-10-18T10:43:04.564739+0000","last_clean":"2023-10-18T10:43:04.564739+0000","last_became_active":"2023-10-18T10:43:02.971030+0000","last_became_peered":"2023-10-18T10:43:02.971030+0000","last_unstale":"2023-10-18T10:43:04.564739+0000","last_undegraded":"2023-10-18T10:43:04.564739+0000","last_fullsized":"2023-10-18T10:43:04.564739+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:32:47.348309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563840+0000","last_change":"2023-10-18T10:43:03.420254+0000","last_active":"2023-10-18T10:43:04.563840+0000","last_peered":"2023-10-18T10:43:04.563840+0000","last_clean":"2023-10-18T10:43:04.563840+0000","last_became_active":"2023-10-18T10:43:03.420019+0000","last_became_peered":"2023-10-18T10:43:03.420019+0000","last_unstale":"2023-10-18T10:43:04.563840+0000","last_undegraded":"2023-10-18T10:43:04.563840+0000","last_fullsized":"2023-10-18T10:43:04.563840+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:02.531521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"127'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564189+0000","last_change":"2023-10-18T10:43:02.974197+0000","last_active":"2023-10-18T10:43:04.564189+0000","last_peered":"2023-10-18T10:43:04.564189+0000","last_clean":"2023-10-18T10:43:04.564189+0000","last_became_active":"2023-10-18T10:43:02.973961+0000","last_became_peered":"2023-10-18T10:43:02.973961+0000","last_unstale":"2023-10-18T10:43:04.564189+0000","last_undegraded":"2023-10-18T10:43:04.564189+0000","last_fullsized":"2023-10-18T10:43:04.564189+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:57.736328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564658+0000","last_change":"2023-10-18T10:43:03.417830+0000","last_active":"2023-10-18T10:43:04.564658+0000","last_peered":"2023-10-18T10:43:04.564658+0000","last_clean":"2023-10-18T10:43:04.564658+0000","last_became_active":"2023-10-18T10:43:03.417031+0000","last_became_peered":"2023-10-18T10:43:03.417031+0000","last_unstale":"2023-10-18T10:43:04.564658+0000","last_undegraded":"2023-10-18T10:43:04.564658+0000","last_fullsized":"2023-10-18T10:43:04.564658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:56.478327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"127'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565099+0000","last_change":"2023-10-18T10:43:02.959201+0000","last_active":"2023-10-18T10:43:04.565099+0000","last_peered":"2023-10-18T10:43:04.565099+0000","last_clean":"2023-10-18T10:43:04.565099+0000","last_became_active":"2023-10-18T10:43:02.958888+0000","last_became_peered":"2023-10-18T10:43:02.958888+0000","last_unstale":"2023-10-18T10:43:04.565099+0000","last_undegraded":"2023-10-18T10:43:04.565099+0000","last_fullsized":"2023-10-18T10:43:04.565099+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:04.257834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564597+0000","last_change":"2023-10-18T10:43:03.419410+0000","last_active":"2023-10-18T10:43:04.564597+0000","last_peered":"2023-10-18T10:43:04.564597+0000","last_clean":"2023-10-18T10:43:04.564597+0000","last_became_active":"2023-10-18T10:43:03.419055+0000","last_became_peered":"2023-10-18T10:43:03.419055+0000","last_unstale":"2023-10-18T10:43:04.564597+0000","last_undegraded":"2023-10-18T10:43:04.564597+0000","last_fullsized":"2023-10-18T10:43:04.564597+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:06.895973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564742+0000","last_change":"2023-10-18T10:43:02.972054+0000","last_active":"2023-10-18T10:43:04.564742+0000","last_peered":"2023-10-18T10:43:04.564742+0000","last_clean":"2023-10-18T10:43:04.564742+0000","last_became_active":"2023-10-18T10:43:02.971783+0000","last_became_peered":"2023-10-18T10:43:02.971783+0000","last_unstale":"2023-10-18T10:43:04.564742+0000","last_undegraded":"2023-10-18T10:43:04.564742+0000","last_fullsized":"2023-10-18T10:43:04.564742+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:07.914251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565070+0000","last_change":"2023-10-18T10:43:03.424942+0000","last_active":"2023-10-18T10:43:04.565070+0000","last_peered":"2023-10-18T10:43:04.565070+0000","last_clean":"2023-10-18T10:43:04.565070+0000","last_became_active":"2023-10-18T10:43:03.424610+0000","last_became_peered":"2023-10-18T10:43:03.424610+0000","last_unstale":"2023-10-18T10:43:04.565070+0000","last_undegraded":"2023-10-18T10:43:04.565070+0000","last_fullsized":"2023-10-18T10:43:04.565070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:58.441002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408796+0000","last_change":"2023-10-18T10:43:02.962770+0000","last_active":"2023-10-18T10:43:04.408796+0000","last_peered":"2023-10-18T10:43:04.408796+0000","last_clean":"2023-10-18T10:43:04.408796+0000","last_became_active":"2023-10-18T10:43:02.962492+0000","last_became_peered":"2023-10-18T10:43:02.962492+0000","last_unstale":"2023-10-18T10:43:04.408796+0000","last_undegraded":"2023-10-18T10:43:04.408796+0000","last_fullsized":"2023-10-18T10:43:04.408796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:04.206263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565182+0000","last_change":"2023-10-18T10:43:03.417626+0000","last_active":"2023-10-18T10:43:04.565182+0000","last_peered":"2023-10-18T10:43:04.565182+0000","last_clean":"2023-10-18T10:43:04.565182+0000","last_became_active":"2023-10-18T10:43:03.417374+0000","last_became_peered":"2023-10-18T10:43:03.417374+0000","last_unstale":"2023-10-18T10:43:04.565182+0000","last_undegraded":"2023-10-18T10:43:04.565182+0000","last_fullsized":"2023-10-18T10:43:04.565182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:39.656533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563634+0000","last_change":"2023-10-18T10:43:02.966586+0000","last_active":"2023-10-18T10:43:04.563634+0000","last_peered":"2023-10-18T10:43:04.563634+0000","last_clean":"2023-10-18T10:43:04.563634+0000","last_became_active":"2023-10-18T10:43:02.966340+0000","last_became_peered":"2023-10-18T10:43:02.966340+0000","last_unstale":"2023-10-18T10:43:04.563634+0000","last_undegraded":"2023-10-18T10:43:04.563634+0000","last_fullsized":"2023-10-18T10:43:04.563634+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.288426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565086+0000","last_change":"2023-10-18T10:43:03.425524+0000","last_active":"2023-10-18T10:43:04.565086+0000","last_peered":"2023-10-18T10:43:04.565086+0000","last_clean":"2023-10-18T10:43:04.565086+0000","last_became_active":"2023-10-18T10:43:03.425299+0000","last_became_peered":"2023-10-18T10:43:03.425299+0000","last_unstale":"2023-10-18T10:43:04.565086+0000","last_undegraded":"2023-10-18T10:43:04.565086+0000","last_fullsized":"2023-10-18T10:43:04.565086+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:18.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"127'8","reported_seq":20,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414095+0000","last_change":"2023-10-18T10:43:02.969709+0000","last_active":"2023-10-18T10:43:04.414095+0000","last_peered":"2023-10-18T10:43:04.414095+0000","last_clean":"2023-10-18T10:43:04.414095+0000","last_became_active":"2023-10-18T10:43:02.969463+0000","last_became_peered":"2023-10-18T10:43:02.969463+0000","last_unstale":"2023-10-18T10:43:04.414095+0000","last_undegraded":"2023-10-18T10:43:04.414095+0000","last_fullsized":"2023-10-18T10:43:04.414095+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:45.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563658+0000","last_change":"2023-10-18T10:43:03.415655+0000","last_active":"2023-10-18T10:43:04.563658+0000","last_peered":"2023-10-18T10:43:04.563658+0000","last_clean":"2023-10-18T10:43:04.563658+0000","last_became_active":"2023-10-18T10:43:03.415397+0000","last_became_peered":"2023-10-18T10:43:03.415397+0000","last_unstale":"2023-10-18T10:43:04.563658+0000","last_undegraded":"2023-10-18T10:43:04.563658+0000","last_fullsized":"2023-10-18T10:43:04.563658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:17.193216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564284+0000","last_change":"2023-10-18T10:43:02.964287+0000","last_active":"2023-10-18T10:43:04.564284+0000","last_peered":"2023-10-18T10:43:04.564284+0000","last_clean":"2023-10-18T10:43:04.564284+0000","last_became_active":"2023-10-18T10:43:02.964033+0000","last_became_peered":"2023-10-18T10:43:02.964033+0000","last_unstale":"2023-10-18T10:43:04.564284+0000","last_undegraded":"2023-10-18T10:43:04.564284+0000","last_fullsized":"2023-10-18T10:43:04.564284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:33.533023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564963+0000","last_change":"2023-10-18T10:43:03.419976+0000","last_active":"2023-10-18T10:43:04.564963+0000","last_peered":"2023-10-18T10:43:04.564963+0000","last_clean":"2023-10-18T10:43:04.564963+0000","last_became_active":"2023-10-18T10:43:03.419611+0000","last_became_peered":"2023-10-18T10:43:03.419611+0000","last_unstale":"2023-10-18T10:43:04.564963+0000","last_undegraded":"2023-10-18T10:43:04.564963+0000","last_fullsized":"2023-10-18T10:43:04.564963+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:57.163873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564210+0000","last_change":"2023-10-18T10:43:02.964463+0000","last_active":"2023-10-18T10:43:04.564210+0000","last_peered":"2023-10-18T10:43:04.564210+0000","last_clean":"2023-10-18T10:43:04.564210+0000","last_became_active":"2023-10-18T10:43:02.964036+0000","last_became_peered":"2023-10-18T10:43:02.964036+0000","last_unstale":"2023-10-18T10:43:04.564210+0000","last_undegraded":"2023-10-18T10:43:04.564210+0000","last_fullsized":"2023-10-18T10:43:04.564210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:27.017315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564542+0000","last_change":"2023-10-18T10:43:03.416778+0000","last_active":"2023-10-18T10:43:04.564542+0000","last_peered":"2023-10-18T10:43:04.564542+0000","last_clean":"2023-10-18T10:43:04.564542+0000","last_became_active":"2023-10-18T10:43:03.416492+0000","last_became_peered":"2023-10-18T10:43:03.416492+0000","last_unstale":"2023-10-18T10:43:04.564542+0000","last_undegraded":"2023-10-18T10:43:04.564542+0000","last_fullsized":"2023-10-18T10:43:04.564542+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:29.332215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564773+0000","last_change":"2023-10-18T10:43:02.965244+0000","last_active":"2023-10-18T10:43:04.564773+0000","last_peered":"2023-10-18T10:43:04.564773+0000","last_clean":"2023-10-18T10:43:04.564773+0000","last_became_active":"2023-10-18T10:43:02.964962+0000","last_became_peered":"2023-10-18T10:43:02.964962+0000","last_unstale":"2023-10-18T10:43:04.564773+0000","last_undegraded":"2023-10-18T10:43:04.564773+0000","last_fullsized":"2023-10-18T10:43:04.564773+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:10.229768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563816+0000","last_change":"2023-10-18T10:43:03.419372+0000","last_active":"2023-10-18T10:43:04.563816+0000","last_peered":"2023-10-18T10:43:04.563816+0000","last_clean":"2023-10-18T10:43:04.563816+0000","last_became_active":"2023-10-18T10:43:03.419133+0000","last_became_peered":"2023-10-18T10:43:03.419133+0000","last_unstale":"2023-10-18T10:43:04.563816+0000","last_undegraded":"2023-10-18T10:43:04.563816+0000","last_fullsized":"2023-10-18T10:43:04.563816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:25.080148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564382+0000","last_change":"2023-10-18T10:43:02.963180+0000","last_active":"2023-10-18T10:43:04.564382+0000","last_peered":"2023-10-18T10:43:04.564382+0000","last_clean":"2023-10-18T10:43:04.564382+0000","last_became_active":"2023-10-18T10:43:02.962924+0000","last_became_peered":"2023-10-18T10:43:02.962924+0000","last_unstale":"2023-10-18T10:43:04.564382+0000","last_undegraded":"2023-10-18T10:43:04.564382+0000","last_fullsized":"2023-10-18T10:43:04.564382+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:27.605859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564165+0000","last_change":"2023-10-18T10:43:03.424395+0000","last_active":"2023-10-18T10:43:04.564165+0000","last_peered":"2023-10-18T10:43:04.564165+0000","last_clean":"2023-10-18T10:43:04.564165+0000","last_became_active":"2023-10-18T10:43:03.424101+0000","last_became_peered":"2023-10-18T10:43:03.424101+0000","last_unstale":"2023-10-18T10:43:04.564165+0000","last_undegraded":"2023-10-18T10:43:04.564165+0000","last_fullsized":"2023-10-18T10:43:04.564165+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:43.317979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563722+0000","last_change":"2023-10-18T10:43:02.964389+0000","last_active":"2023-10-18T10:43:04.563722+0000","last_peered":"2023-10-18T10:43:04.563722+0000","last_clean":"2023-10-18T10:43:04.563722+0000","last_became_active":"2023-10-18T10:43:02.964134+0000","last_became_peered":"2023-10-18T10:43:02.964134+0000","last_unstale":"2023-10-18T10:43:04.563722+0000","last_undegraded":"2023-10-18T10:43:04.563722+0000","last_fullsized":"2023-10-18T10:43:04.563722+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:27.504789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408932+0000","last_change":"2023-10-18T10:43:03.417105+0000","last_active":"2023-10-18T10:43:04.408932+0000","last_peered":"2023-10-18T10:43:04.408932+0000","last_clean":"2023-10-18T10:43:04.408932+0000","last_became_active":"2023-10-18T10:43:03.416864+0000","last_became_peered":"2023-10-18T10:43:03.416864+0000","last_unstale":"2023-10-18T10:43:04.408932+0000","last_undegraded":"2023-10-18T10:43:04.408932+0000","last_fullsized":"2023-10-18T10:43:04.408932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:09.017135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564862+0000","last_change":"2023-10-18T10:43:02.967733+0000","last_active":"2023-10-18T10:43:04.564862+0000","last_peered":"2023-10-18T10:43:04.564862+0000","last_clean":"2023-10-18T10:43:04.564862+0000","last_became_active":"2023-10-18T10:43:02.967444+0000","last_became_peered":"2023-10-18T10:43:02.967444+0000","last_unstale":"2023-10-18T10:43:04.564862+0000","last_undegraded":"2023-10-18T10:43:04.564862+0000","last_fullsized":"2023-10-18T10:43:04.564862+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:59.437631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564910+0000","last_change":"2023-10-18T10:43:03.418676+0000","last_active":"2023-10-18T10:43:04.564910+0000","last_peered":"2023-10-18T10:43:04.564910+0000","last_clean":"2023-10-18T10:43:04.564910+0000","last_became_active":"2023-10-18T10:43:03.418441+0000","last_became_peered":"2023-10-18T10:43:03.418441+0000","last_unstale":"2023-10-18T10:43:04.564910+0000","last_undegraded":"2023-10-18T10:43:04.564910+0000","last_fullsized":"2023-10-18T10:43:04.564910+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:30.204629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563690+0000","last_change":"2023-10-18T10:43:02.963728+0000","last_active":"2023-10-18T10:43:04.563690+0000","last_peered":"2023-10-18T10:43:04.563690+0000","last_clean":"2023-10-18T10:43:04.563690+0000","last_became_active":"2023-10-18T10:43:02.963467+0000","last_became_peered":"2023-10-18T10:43:02.963467+0000","last_unstale":"2023-10-18T10:43:04.563690+0000","last_undegraded":"2023-10-18T10:43:04.563690+0000","last_fullsized":"2023-10-18T10:43:04.563690+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:38.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564445+0000","last_change":"2023-10-18T10:43:03.421100+0000","last_active":"2023-10-18T10:43:04.564445+0000","last_peered":"2023-10-18T10:43:04.564445+0000","last_clean":"2023-10-18T10:43:04.564445+0000","last_became_active":"2023-10-18T10:43:03.420845+0000","last_became_peered":"2023-10-18T10:43:03.420845+0000","last_unstale":"2023-10-18T10:43:04.564445+0000","last_undegraded":"2023-10-18T10:43:04.564445+0000","last_fullsized":"2023-10-18T10:43:04.564445+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:32:19.619118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564074+0000","last_change":"2023-10-18T10:43:02.971128+0000","last_active":"2023-10-18T10:43:04.564074+0000","last_peered":"2023-10-18T10:43:04.564074+0000","last_clean":"2023-10-18T10:43:04.564074+0000","last_became_active":"2023-10-18T10:43:02.970845+0000","last_became_peered":"2023-10-18T10:43:02.970845+0000","last_unstale":"2023-10-18T10:43:04.564074+0000","last_undegraded":"2023-10-18T10:43:04.564074+0000","last_fullsized":"2023-10-18T10:43:04.564074+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:57.435429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414114+0000","last_change":"2023-10-18T10:43:03.414544+0000","last_active":"2023-10-18T10:43:04.414114+0000","last_peered":"2023-10-18T10:43:04.414114+0000","last_clean":"2023-10-18T10:43:04.414114+0000","last_became_active":"2023-10-18T10:43:03.414270+0000","last_became_peered":"2023-10-18T10:43:03.414270+0000","last_unstale":"2023-10-18T10:43:04.414114+0000","last_undegraded":"2023-10-18T10:43:04.414114+0000","last_fullsized":"2023-10-18T10:43:04.414114+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:03.106690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564812+0000","last_change":"2023-10-18T10:43:02.972906+0000","last_active":"2023-10-18T10:43:04.564812+0000","last_peered":"2023-10-18T10:43:04.564812+0000","last_clean":"2023-10-18T10:43:04.564812+0000","last_became_active":"2023-10-18T10:43:02.972652+0000","last_became_peered":"2023-10-18T10:43:02.972652+0000","last_unstale":"2023-10-18T10:43:04.564812+0000","last_undegraded":"2023-10-18T10:43:04.564812+0000","last_fullsized":"2023-10-18T10:43:04.564812+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:09.602406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408779+0000","last_change":"2023-10-18T10:43:03.416563+0000","last_active":"2023-10-18T10:43:04.408779+0000","last_peered":"2023-10-18T10:43:04.408779+0000","last_clean":"2023-10-18T10:43:04.408779+0000","last_became_active":"2023-10-18T10:43:03.416321+0000","last_became_peered":"2023-10-18T10:43:03.416321+0000","last_unstale":"2023-10-18T10:43:04.408779+0000","last_undegraded":"2023-10-18T10:43:04.408779+0000","last_fullsized":"2023-10-18T10:43:04.408779+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:57.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565167+0000","last_change":"2023-10-18T10:43:02.962995+0000","last_active":"2023-10-18T10:43:04.565167+0000","last_peered":"2023-10-18T10:43:04.565167+0000","last_clean":"2023-10-18T10:43:04.565167+0000","last_became_active":"2023-10-18T10:43:02.962723+0000","last_became_peered":"2023-10-18T10:43:02.962723+0000","last_unstale":"2023-10-18T10:43:04.565167+0000","last_undegraded":"2023-10-18T10:43:04.565167+0000","last_fullsized":"2023-10-18T10:43:04.565167+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:45.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564038+0000","last_change":"2023-10-18T10:43:03.424431+0000","last_active":"2023-10-18T10:43:04.564038+0000","last_peered":"2023-10-18T10:43:04.564038+0000","last_clean":"2023-10-18T10:43:04.564038+0000","last_became_active":"2023-10-18T10:43:03.424173+0000","last_became_peered":"2023-10-18T10:43:03.424173+0000","last_unstale":"2023-10-18T10:43:04.564038+0000","last_undegraded":"2023-10-18T10:43:04.564038+0000","last_fullsized":"2023-10-18T10:43:04.564038+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:19.931739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564415+0000","last_change":"2023-10-18T10:43:02.968945+0000","last_active":"2023-10-18T10:43:04.564415+0000","last_peered":"2023-10-18T10:43:04.564415+0000","last_clean":"2023-10-18T10:43:04.564415+0000","last_became_active":"2023-10-18T10:43:02.968666+0000","last_became_peered":"2023-10-18T10:43:02.968666+0000","last_unstale":"2023-10-18T10:43:04.564415+0000","last_undegraded":"2023-10-18T10:43:04.564415+0000","last_fullsized":"2023-10-18T10:43:04.564415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:26.843460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564480+0000","last_change":"2023-10-18T10:43:03.421780+0000","last_active":"2023-10-18T10:43:04.564480+0000","last_peered":"2023-10-18T10:43:04.564480+0000","last_clean":"2023-10-18T10:43:04.564480+0000","last_became_active":"2023-10-18T10:43:03.421547+0000","last_became_peered":"2023-10-18T10:43:03.421547+0000","last_unstale":"2023-10-18T10:43:04.564480+0000","last_undegraded":"2023-10-18T10:43:04.564480+0000","last_fullsized":"2023-10-18T10:43:04.564480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:54:47.177774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414258+0000","last_change":"2023-10-18T10:43:02.965516+0000","last_active":"2023-10-18T10:43:04.414258+0000","last_peered":"2023-10-18T10:43:04.414258+0000","last_clean":"2023-10-18T10:43:04.414258+0000","last_became_active":"2023-10-18T10:43:02.965251+0000","last_became_peered":"2023-10-18T10:43:02.965251+0000","last_unstale":"2023-10-18T10:43:04.414258+0000","last_undegraded":"2023-10-18T10:43:04.414258+0000","last_fullsized":"2023-10-18T10:43:04.414258+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:59.529022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563859+0000","last_change":"2023-10-18T10:43:03.423032+0000","last_active":"2023-10-18T10:43:04.563859+0000","last_peered":"2023-10-18T10:43:04.563859+0000","last_clean":"2023-10-18T10:43:04.563859+0000","last_became_active":"2023-10-18T10:43:03.422767+0000","last_became_peered":"2023-10-18T10:43:03.422767+0000","last_unstale":"2023-10-18T10:43:04.563859+0000","last_undegraded":"2023-10-18T10:43:04.563859+0000","last_fullsized":"2023-10-18T10:43:04.563859+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:17.615763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563911+0000","last_change":"2023-10-18T10:43:02.965062+0000","last_active":"2023-10-18T10:43:04.563911+0000","last_peered":"2023-10-18T10:43:04.563911+0000","last_clean":"2023-10-18T10:43:04.563911+0000","last_became_active":"2023-10-18T10:43:02.964753+0000","last_became_peered":"2023-10-18T10:43:02.964753+0000","last_unstale":"2023-10-18T10:43:04.563911+0000","last_undegraded":"2023-10-18T10:43:04.563911+0000","last_fullsized":"2023-10-18T10:43:04.563911+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:41.151549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564340+0000","last_change":"2023-10-18T10:43:03.421221+0000","last_active":"2023-10-18T10:43:04.564340+0000","last_peered":"2023-10-18T10:43:04.564340+0000","last_clean":"2023-10-18T10:43:04.564340+0000","last_became_active":"2023-10-18T10:43:03.420954+0000","last_became_peered":"2023-10-18T10:43:03.420954+0000","last_unstale":"2023-10-18T10:43:04.564340+0000","last_undegraded":"2023-10-18T10:43:04.564340+0000","last_fullsized":"2023-10-18T10:43:04.564340+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:42.945030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564932+0000","last_change":"2023-10-18T10:43:02.962374+0000","last_active":"2023-10-18T10:43:04.564932+0000","last_peered":"2023-10-18T10:43:04.564932+0000","last_clean":"2023-10-18T10:43:04.564932+0000","last_became_active":"2023-10-18T10:43:02.962065+0000","last_became_peered":"2023-10-18T10:43:02.962065+0000","last_unstale":"2023-10-18T10:43:04.564932+0000","last_undegraded":"2023-10-18T10:43:04.564932+0000","last_fullsized":"2023-10-18T10:43:04.564932+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:37.693671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564809+0000","last_change":"2023-10-18T10:43:03.418250+0000","last_active":"2023-10-18T10:43:04.564809+0000","last_peered":"2023-10-18T10:43:04.564809+0000","last_clean":"2023-10-18T10:43:04.564809+0000","last_became_active":"2023-10-18T10:43:03.417934+0000","last_became_peered":"2023-10-18T10:43:03.417934+0000","last_unstale":"2023-10-18T10:43:04.564809+0000","last_undegraded":"2023-10-18T10:43:04.564809+0000","last_fullsized":"2023-10-18T10:43:04.564809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:09.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563736+0000","last_change":"2023-10-18T10:43:02.965897+0000","last_active":"2023-10-18T10:43:04.563736+0000","last_peered":"2023-10-18T10:43:04.563736+0000","last_clean":"2023-10-18T10:43:04.563736+0000","last_became_active":"2023-10-18T10:43:02.965636+0000","last_became_peered":"2023-10-18T10:43:02.965636+0000","last_unstale":"2023-10-18T10:43:04.563736+0000","last_undegraded":"2023-10-18T10:43:04.563736+0000","last_fullsized":"2023-10-18T10:43:04.563736+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:10:39.372132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564606+0000","last_change":"2023-10-18T10:43:03.420416+0000","last_active":"2023-10-18T10:43:04.564606+0000","last_peered":"2023-10-18T10:43:04.564606+0000","last_clean":"2023-10-18T10:43:04.564606+0000","last_became_active":"2023-10-18T10:43:03.420064+0000","last_became_peered":"2023-10-18T10:43:03.420064+0000","last_unstale":"2023-10-18T10:43:04.564606+0000","last_undegraded":"2023-10-18T10:43:04.564606+0000","last_fullsized":"2023-10-18T10:43:04.564606+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:01.892160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564707+0000","last_change":"2023-10-18T10:43:02.962533+0000","last_active":"2023-10-18T10:43:04.564707+0000","last_peered":"2023-10-18T10:43:04.564707+0000","last_clean":"2023-10-18T10:43:04.564707+0000","last_became_active":"2023-10-18T10:43:02.962275+0000","last_became_peered":"2023-10-18T10:43:02.962275+0000","last_unstale":"2023-10-18T10:43:04.564707+0000","last_undegraded":"2023-10-18T10:43:04.564707+0000","last_fullsized":"2023-10-18T10:43:04.564707+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:26.189773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"117'214","reported_seq":611,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563741+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:43:04.563741+0000","last_peered":"2023-10-18T10:43:04.563741+0000","last_clean":"2023-10-18T10:43:04.563741+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:43:04.563741+0000","last_undegraded":"2023-10-18T10:43:04.563741+0000","last_fullsized":"2023-10-18T10:43:04.563741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575068,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59692,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312148,"statfs":{"total":96636764160,"available":96575639552,"internally_reserved":0,"allocated":2187264,"data_stored":1388162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.58299999999999996},"min":{"1min":0.36099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.754,"5min":0.876,"15min":0.88300000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63200000000000001,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70299999999999996,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.60299999999999998},"min":{"1min":0.42199999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.76000000000000001,"5min":0.79800000000000004,"15min":0.85799999999999998},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.68899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.315},"max":{"1min":0.78500000000000003,"5min":0.875,"15min":0.875},"last":0.504}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575068,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":799850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82599999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.69499999999999995,"5min":0.77700000000000002,"15min":0.82699999999999996},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65800000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.432}]},{"osd":3,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.78400000000000003,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.748,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.80300000000000005,"5min":0.82199999999999995,"15min":0.87},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":797634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.39600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88400000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81399999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.81999999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.41199999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.47599999999999998},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71599999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.375},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.496},"min":{"1min":0.373,"5min":0.31,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.38200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77200000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.312,"15min":0.31},"max":{"1min":0.84299999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.443},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.44700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.871,"15min":0.871},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575068,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1593344,"data_stored":797726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57699999999999996,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.77000000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.501,"5min":0.54700000000000004,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.753,"15min":0.877},"last":0.63300000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.89300000000000002},"last":0.439}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73499999999999999,"5min":0.873,"15min":0.873},"last":0.76700000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71899999999999997,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.78200000000000003},"last":0.81100000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.33200000000000002,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.79200000000000004,"5min":1.446,"15min":1.446},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.81999999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.879,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.874}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.52800000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.499,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.624,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.59299999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53400000000000003,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.379},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.434,"15min":0.45100000000000001},"min":{"1min":0.28599999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.80200000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.879,"15min":0.94699999999999995},"last":0.5}]},{"osd":5,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":1.002,"15min":1.002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88900000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.629,"15min":0.63800000000000001},"min":{"1min":0.434,"5min":0.38,"15min":0.37},"max":{"1min":0.82499999999999996,"5min":1.018,"15min":1.018},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.751,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.63900000000000001},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.627,"15min":0.65300000000000002},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54940,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316900,"statfs":{"total":96636764160,"available":96580505600,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.371,"15min":0.374},"min":{"1min":0.252,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.626},"last":0.377}]},{"osd":1,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29899999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.57899999999999996,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.315}]},{"osd":3,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.24299999999999999},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.434,"15min":0.44700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.41899999999999998,"15min":0.254},"max":{"1min":0.86499999999999999,"5min":0.88200000000000001,"15min":0.88800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.625,"15min":0.64500000000000002},"min":{"1min":0.45300000000000001,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.95699999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.63},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.93700000000000006},"last":0.57299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.84599999999999997,"15min":0.92500000000000004},"last":0.54100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36199999999999999,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.48999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.32000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.39200000000000002},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.374},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.42099999999999999},"min":{"1min":0.314,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53600000000000003,"5min":0.627,"15min":0.64800000000000002},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.33100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59499999999999997,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.33500000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.83999999999999997,"15min":0.95599999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.623},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.753,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.88100000000000001,"15min":0.95999999999999996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60199999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.76200000000000001,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.63700000000000001},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.78000000000000003,"5min":1.03,"15min":1.03},"last":0.749}]},{"osd":7,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.63},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78800000000000003,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575067,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2195456,"data_stored":1389704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.39000000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.54600000000000004,"5min":0.62,"15min":0.76900000000000002},"last":0.214}]},{"osd":2,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.187},"max":{"1min":0.58099999999999996,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.41999999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.29299999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.39500000000000002},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56599999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.312}]},{"osd":4,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.61699999999999999},"min":{"1min":0.372,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81999999999999995,"5min":0.996,"15min":0.996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90300000000000002,"5min":1.038,"15min":1.038},"last":0.34200000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.89200000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93700000000000006,"5min":1.026,"15min":1.3700000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88100000000000001,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.64300000000000002},"min":{"1min":0.41199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.05,"15min":1.339},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.64600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:43:08.940 INFO:tasks.ceph.mds.c.smithi125.stderr:2023-10-18T10:43:08.939+0000 7f2145d61640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:43:11.282 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:43:11.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.393+0000 7f1f8b94c640 1 -- 172.21.15.125:0/1251130642 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f8413d360 msgr2=0x7f1f8415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.393+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/1251130642 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f8413d360 0x7f1f8415a9f0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f1f80005fd0 tx=0x7f1f8002c2e0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.393+0000 7f1f8b94c640 1 -- 172.21.15.125:0/1251130642 shutdown_connections 2023-10-18T10:43:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.393+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/1251130642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f84140660 0x7f1f840c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.393+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/1251130642 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f8413d360 0x7f1f8415a9f0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.393+0000 7f1f8b94c640 1 -- 172.21.15.125:0/1251130642 >> 172.21.15.125:0/1251130642 conn(0x7f1f84069930 msgr2=0x7f1f84069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:11.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.393+0000 7f1f8b94c640 1 -- 172.21.15.125:0/1251130642 shutdown_connections 2023-10-18T10:43:11.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.393+0000 7f1f8b94c640 1 -- 172.21.15.125:0/1251130642 wait complete. 2023-10-18T10:43:11.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.394+0000 7f1f8b94c640 1 Processor -- start 2023-10-18T10:43:11.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.394+0000 7f1f8b94c640 1 -- start start 2023-10-18T10:43:11.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f8b94c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f840c9d00 0x7f1f84156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:11.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f8b94c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8413d360 0x7f1f840ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:11.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f8b94c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f84140660 0x7f1f840ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:11.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f8b94c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840ae5c0 con 0x7f1f84140660 2023-10-18T10:43:11.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f8b94c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8414eb90 con 0x7f1f8413d360 2023-10-18T10:43:11.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f8b94c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8414cf30 con 0x7f1f840c9d00 2023-10-18T10:43:11.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f89ee9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f84140660 0x7f1f840ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:11.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f89ee9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f84140660 0x7f1f840ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54552/0 (socket says 172.21.15.125:54552) 2023-10-18T10:43:11.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f89ee9640 1 -- 172.21.15.125:0/645180438 learned_addr learned my addr 172.21.15.125:0/645180438 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:11.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f896e8640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f840c9d00 0x7f1f84156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:11.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f89ee9640 1 -- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f840c9d00 msgr2=0x7f1f84156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f89ee9640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f840c9d00 0x7f1f84156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f88ee7640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8413d360 0x7f1f840ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:11.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f89ee9640 1 -- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8413d360 msgr2=0x7f1f840ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f89ee9640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8413d360 0x7f1f840ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f89ee9640 1 -- 172.21.15.125:0/645180438 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009900 con 0x7f1f84140660 2023-10-18T10:43:11.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f896e8640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f840c9d00 0x7f1f84156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:43:11.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.395+0000 7f1f88ee7640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8413d360 0x7f1f840ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:43:11.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.396+0000 7f1f89ee9640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f84140660 0x7f1f840ac590 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c3f0 tx=0x7f1f7c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.396+0000 7f1f79ffb640 1 -- 172.21.15.125:0/645180438 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0052d0 con 0x7f1f84140660 2023-10-18T10:43:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.396+0000 7f1f79ffb640 1 -- 172.21.15.125:0/645180438 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f7c007020 con 0x7f1f84140660 2023-10-18T10:43:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.396+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8414cc00 con 0x7f1f84140660 2023-10-18T10:43:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.396+0000 7f1f79ffb640 1 -- 172.21.15.125:0/645180438 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c007430 con 0x7f1f84140660 2023-10-18T10:43:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.396+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f840acad0 con 0x7f1f84140660 2023-10-18T10:43:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.397+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f840690e0 con 0x7f1f84140660 2023-10-18T10:43:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.397+0000 7f1f79ffb640 1 -- 172.21.15.125:0/645180438 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1f7c007590 con 0x7f1f84140660 2023-10-18T10:43:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.398+0000 7f1f79ffb640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f1f6806af90 0x7f1f6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.398+0000 7f1f79ffb640 1 -- 172.21.15.125:0/645180438 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1f7c088e60 con 0x7f1f84140660 2023-10-18T10:43:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.398+0000 7f1f896e8640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f1f6806af90 0x7f1f6806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:11.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.398+0000 7f1f896e8640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f1f6806af90 0x7f1f6806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f80036000 tx=0x7f1f80004ab0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:11.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.402+0000 7f1f79ffb640 1 -- 172.21.15.125:0/645180438 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f7c053cc0 con 0x7f1f84140660 2023-10-18T10:43:11.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.570+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 --> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f8415b970 con 0x7f1f6806af90 2023-10-18T10:43:11.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.578+0000 7f1f79ffb640 1 -- 172.21.15.125:0/645180438 <== mgr.7360 v2:172.21.15.139:6832/1511464628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289729 (secure 0 0 0) 0x7f1f8415b970 con 0x7f1f6806af90 2023-10-18T10:43:11.583 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:11.584 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:43:11.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f1f6806af90 msgr2=0x7f1f6806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f1f6806af90 0x7f1f6806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f80036000 tx=0x7f1f80004ab0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f84140660 msgr2=0x7f1f840ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f84140660 0x7f1f840ac590 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c3f0 tx=0x7f1f7c00c8b0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 shutdown_connections 2023-10-18T10:43:11.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f1f6806af90 0x7f1f6806d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8413d360 0x7f1f840ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f84140660 0x7f1f840ac590 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 --2- 172.21.15.125:0/645180438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f840c9d00 0x7f1f84156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 >> 172.21.15.125:0/645180438 conn(0x7f1f84069930 msgr2=0x7f1f8406c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:11.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 shutdown_connections 2023-10-18T10:43:11.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.581+0000 7f1f8b94c640 1 -- 172.21.15.125:0/645180438 wait complete. 2023-10-18T10:43:11.594 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-18T10:43:09.900856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":379,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315840,"kb_used_data":13608,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754658880,"statfs":{"total":773094113280,"available":772770693120,"internally_reserved":0,"allocated":13934592,"data_stored":7566628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.949124"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564947+0000","last_change":"2023-10-18T10:43:03.420057+0000","last_active":"2023-10-18T10:43:04.564947+0000","last_peered":"2023-10-18T10:43:04.564947+0000","last_clean":"2023-10-18T10:43:04.564947+0000","last_became_active":"2023-10-18T10:43:03.419809+0000","last_became_peered":"2023-10-18T10:43:03.419809+0000","last_unstale":"2023-10-18T10:43:04.564947+0000","last_undegraded":"2023-10-18T10:43:04.564947+0000","last_fullsized":"2023-10-18T10:43:04.564947+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:51.310959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565191+0000","last_change":"2023-10-18T10:43:03.422800+0000","last_active":"2023-10-18T10:43:04.565191+0000","last_peered":"2023-10-18T10:43:04.565191+0000","last_clean":"2023-10-18T10:43:04.565191+0000","last_became_active":"2023-10-18T10:43:03.422037+0000","last_became_peered":"2023-10-18T10:43:03.422037+0000","last_unstale":"2023-10-18T10:43:04.565191+0000","last_undegraded":"2023-10-18T10:43:04.565191+0000","last_fullsized":"2023-10-18T10:43:04.565191+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:03.106690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564044+0000","last_change":"2023-10-18T10:43:03.424158+0000","last_active":"2023-10-18T10:43:04.564044+0000","last_peered":"2023-10-18T10:43:04.564044+0000","last_clean":"2023-10-18T10:43:04.564044+0000","last_became_active":"2023-10-18T10:43:03.423915+0000","last_became_peered":"2023-10-18T10:43:03.423915+0000","last_unstale":"2023-10-18T10:43:04.564044+0000","last_undegraded":"2023-10-18T10:43:04.564044+0000","last_fullsized":"2023-10-18T10:43:04.564044+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:49.732624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408937+0000","last_change":"2023-10-18T10:43:03.415009+0000","last_active":"2023-10-18T10:43:04.408937+0000","last_peered":"2023-10-18T10:43:04.408937+0000","last_clean":"2023-10-18T10:43:04.408937+0000","last_became_active":"2023-10-18T10:43:03.414710+0000","last_became_peered":"2023-10-18T10:43:03.414710+0000","last_unstale":"2023-10-18T10:43:04.408937+0000","last_undegraded":"2023-10-18T10:43:04.408937+0000","last_fullsized":"2023-10-18T10:43:04.408937+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:21.227528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563635+0000","last_change":"2023-10-18T10:43:03.411644+0000","last_active":"2023-10-18T10:43:04.563635+0000","last_peered":"2023-10-18T10:43:04.563635+0000","last_clean":"2023-10-18T10:43:04.563635+0000","last_became_active":"2023-10-18T10:43:03.411381+0000","last_became_peered":"2023-10-18T10:43:03.411381+0000","last_unstale":"2023-10-18T10:43:04.563635+0000","last_undegraded":"2023-10-18T10:43:04.563635+0000","last_fullsized":"2023-10-18T10:43:04.563635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:28.704370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564515+0000","last_change":"2023-10-18T10:43:03.420380+0000","last_active":"2023-10-18T10:43:04.564515+0000","last_peered":"2023-10-18T10:43:04.564515+0000","last_clean":"2023-10-18T10:43:04.564515+0000","last_became_active":"2023-10-18T10:43:03.420112+0000","last_became_peered":"2023-10-18T10:43:03.420112+0000","last_unstale":"2023-10-18T10:43:04.564515+0000","last_undegraded":"2023-10-18T10:43:04.564515+0000","last_fullsized":"2023-10-18T10:43:04.564515+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:22:15.468100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564645+0000","last_change":"2023-10-18T10:43:03.417853+0000","last_active":"2023-10-18T10:43:04.564645+0000","last_peered":"2023-10-18T10:43:04.564645+0000","last_clean":"2023-10-18T10:43:04.564645+0000","last_became_active":"2023-10-18T10:43:03.417585+0000","last_became_peered":"2023-10-18T10:43:03.417585+0000","last_unstale":"2023-10-18T10:43:04.564645+0000","last_undegraded":"2023-10-18T10:43:04.564645+0000","last_fullsized":"2023-10-18T10:43:04.564645+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:57.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563425+0000","last_change":"2023-10-18T10:43:03.420896+0000","last_active":"2023-10-18T10:43:04.563425+0000","last_peered":"2023-10-18T10:43:04.563425+0000","last_clean":"2023-10-18T10:43:04.563425+0000","last_became_active":"2023-10-18T10:43:03.420633+0000","last_became_peered":"2023-10-18T10:43:03.420633+0000","last_unstale":"2023-10-18T10:43:04.563425+0000","last_undegraded":"2023-10-18T10:43:04.563425+0000","last_fullsized":"2023-10-18T10:43:04.563425+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:54.658345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564188+0000","last_change":"2023-10-18T10:43:03.423222+0000","last_active":"2023-10-18T10:43:04.564188+0000","last_peered":"2023-10-18T10:43:04.564188+0000","last_clean":"2023-10-18T10:43:04.564188+0000","last_became_active":"2023-10-18T10:43:03.422095+0000","last_became_peered":"2023-10-18T10:43:03.422095+0000","last_unstale":"2023-10-18T10:43:04.564188+0000","last_undegraded":"2023-10-18T10:43:04.564188+0000","last_fullsized":"2023-10-18T10:43:04.564188+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:49:36.033906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564111+0000","last_change":"2023-10-18T10:43:03.420436+0000","last_active":"2023-10-18T10:43:04.564111+0000","last_peered":"2023-10-18T10:43:04.564111+0000","last_clean":"2023-10-18T10:43:04.564111+0000","last_became_active":"2023-10-18T10:43:03.419414+0000","last_became_peered":"2023-10-18T10:43:03.419414+0000","last_unstale":"2023-10-18T10:43:04.564111+0000","last_undegraded":"2023-10-18T10:43:04.564111+0000","last_fullsized":"2023-10-18T10:43:04.564111+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:27.017090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564170+0000","last_change":"2023-10-18T10:43:03.425870+0000","last_active":"2023-10-18T10:43:04.564170+0000","last_peered":"2023-10-18T10:43:04.564170+0000","last_clean":"2023-10-18T10:43:04.564170+0000","last_became_active":"2023-10-18T10:43:03.425583+0000","last_became_peered":"2023-10-18T10:43:03.425583+0000","last_unstale":"2023-10-18T10:43:04.564170+0000","last_undegraded":"2023-10-18T10:43:04.564170+0000","last_fullsized":"2023-10-18T10:43:04.564170+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:07.373278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414558+0000","last_change":"2023-10-18T10:43:03.421791+0000","last_active":"2023-10-18T10:43:04.414558+0000","last_peered":"2023-10-18T10:43:04.414558+0000","last_clean":"2023-10-18T10:43:04.414558+0000","last_became_active":"2023-10-18T10:43:03.421296+0000","last_became_peered":"2023-10-18T10:43:03.421296+0000","last_unstale":"2023-10-18T10:43:04.414558+0000","last_undegraded":"2023-10-18T10:43:04.414558+0000","last_fullsized":"2023-10-18T10:43:04.414558+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:23:15.369598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408816+0000","last_change":"2023-10-18T10:43:03.421921+0000","last_active":"2023-10-18T10:43:04.408816+0000","last_peered":"2023-10-18T10:43:04.408816+0000","last_clean":"2023-10-18T10:43:04.408816+0000","last_became_active":"2023-10-18T10:43:03.421644+0000","last_became_peered":"2023-10-18T10:43:03.421644+0000","last_unstale":"2023-10-18T10:43:04.408816+0000","last_undegraded":"2023-10-18T10:43:04.408816+0000","last_fullsized":"2023-10-18T10:43:04.408816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:58.741901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563868+0000","last_change":"2023-10-18T10:43:03.415137+0000","last_active":"2023-10-18T10:43:04.563868+0000","last_peered":"2023-10-18T10:43:04.563868+0000","last_clean":"2023-10-18T10:43:04.563868+0000","last_became_active":"2023-10-18T10:43:03.414871+0000","last_became_peered":"2023-10-18T10:43:03.414871+0000","last_unstale":"2023-10-18T10:43:04.563868+0000","last_undegraded":"2023-10-18T10:43:04.563868+0000","last_fullsized":"2023-10-18T10:43:04.563868+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:33.182433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563789+0000","last_change":"2023-10-18T10:43:03.420511+0000","last_active":"2023-10-18T10:43:04.563789+0000","last_peered":"2023-10-18T10:43:04.563789+0000","last_clean":"2023-10-18T10:43:04.563789+0000","last_became_active":"2023-10-18T10:43:03.420241+0000","last_became_peered":"2023-10-18T10:43:03.420241+0000","last_unstale":"2023-10-18T10:43:04.563789+0000","last_undegraded":"2023-10-18T10:43:04.563789+0000","last_fullsized":"2023-10-18T10:43:04.563789+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:38.074311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563696+0000","last_change":"2023-10-18T10:43:03.417552+0000","last_active":"2023-10-18T10:43:04.563696+0000","last_peered":"2023-10-18T10:43:04.563696+0000","last_clean":"2023-10-18T10:43:04.563696+0000","last_became_active":"2023-10-18T10:43:03.417291+0000","last_became_peered":"2023-10-18T10:43:03.417291+0000","last_unstale":"2023-10-18T10:43:04.563696+0000","last_undegraded":"2023-10-18T10:43:04.563696+0000","last_fullsized":"2023-10-18T10:43:04.563696+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:18.941147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.425285+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.424971+0000","last_became_peered":"2023-10-18T10:43:03.424971+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:00.092514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565204+0000","last_change":"2023-10-18T10:43:03.416033+0000","last_active":"2023-10-18T10:43:04.565204+0000","last_peered":"2023-10-18T10:43:04.565204+0000","last_clean":"2023-10-18T10:43:04.565204+0000","last_became_active":"2023-10-18T10:43:03.415763+0000","last_became_peered":"2023-10-18T10:43:03.415763+0000","last_unstale":"2023-10-18T10:43:04.565204+0000","last_undegraded":"2023-10-18T10:43:04.565204+0000","last_fullsized":"2023-10-18T10:43:04.565204+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:36.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564171+0000","last_change":"2023-10-18T10:43:03.418626+0000","last_active":"2023-10-18T10:43:04.564171+0000","last_peered":"2023-10-18T10:43:04.564171+0000","last_clean":"2023-10-18T10:43:04.564171+0000","last_became_active":"2023-10-18T10:43:03.418344+0000","last_became_peered":"2023-10-18T10:43:03.418344+0000","last_unstale":"2023-10-18T10:43:04.564171+0000","last_undegraded":"2023-10-18T10:43:04.564171+0000","last_fullsized":"2023-10-18T10:43:04.564171+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:55.314298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414494+0000","last_change":"2023-10-18T10:43:03.416156+0000","last_active":"2023-10-18T10:43:04.414494+0000","last_peered":"2023-10-18T10:43:04.414494+0000","last_clean":"2023-10-18T10:43:04.414494+0000","last_became_active":"2023-10-18T10:43:03.415839+0000","last_became_peered":"2023-10-18T10:43:03.415839+0000","last_unstale":"2023-10-18T10:43:04.414494+0000","last_undegraded":"2023-10-18T10:43:04.414494+0000","last_fullsized":"2023-10-18T10:43:04.414494+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:24.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564184+0000","last_change":"2023-10-18T10:43:03.421228+0000","last_active":"2023-10-18T10:43:04.564184+0000","last_peered":"2023-10-18T10:43:04.564184+0000","last_clean":"2023-10-18T10:43:04.564184+0000","last_became_active":"2023-10-18T10:43:03.420908+0000","last_became_peered":"2023-10-18T10:43:03.420908+0000","last_unstale":"2023-10-18T10:43:04.564184+0000","last_undegraded":"2023-10-18T10:43:04.564184+0000","last_fullsized":"2023-10-18T10:43:04.564184+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:19.745476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564166+0000","last_change":"2023-10-18T10:43:03.421765+0000","last_active":"2023-10-18T10:43:04.564166+0000","last_peered":"2023-10-18T10:43:04.564166+0000","last_clean":"2023-10-18T10:43:04.564166+0000","last_became_active":"2023-10-18T10:43:03.421501+0000","last_became_peered":"2023-10-18T10:43:03.421501+0000","last_unstale":"2023-10-18T10:43:04.564166+0000","last_undegraded":"2023-10-18T10:43:04.564166+0000","last_fullsized":"2023-10-18T10:43:04.564166+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:06.267958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564959+0000","last_change":"2023-10-18T10:43:03.424152+0000","last_active":"2023-10-18T10:43:04.564959+0000","last_peered":"2023-10-18T10:43:04.564959+0000","last_clean":"2023-10-18T10:43:04.564959+0000","last_became_active":"2023-10-18T10:43:03.423890+0000","last_became_peered":"2023-10-18T10:43:03.423890+0000","last_unstale":"2023-10-18T10:43:04.564959+0000","last_undegraded":"2023-10-18T10:43:04.564959+0000","last_fullsized":"2023-10-18T10:43:04.564959+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:26.100932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564263+0000","last_change":"2023-10-18T10:43:03.417440+0000","last_active":"2023-10-18T10:43:04.564263+0000","last_peered":"2023-10-18T10:43:04.564263+0000","last_clean":"2023-10-18T10:43:04.564263+0000","last_became_active":"2023-10-18T10:43:03.417201+0000","last_became_peered":"2023-10-18T10:43:03.417201+0000","last_unstale":"2023-10-18T10:43:04.564263+0000","last_undegraded":"2023-10-18T10:43:04.564263+0000","last_fullsized":"2023-10-18T10:43:04.564263+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:16.834481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414558+0000","last_change":"2023-10-18T10:43:03.417026+0000","last_active":"2023-10-18T10:43:04.414558+0000","last_peered":"2023-10-18T10:43:04.414558+0000","last_clean":"2023-10-18T10:43:04.414558+0000","last_became_active":"2023-10-18T10:43:03.416727+0000","last_became_peered":"2023-10-18T10:43:03.416727+0000","last_unstale":"2023-10-18T10:43:04.414558+0000","last_undegraded":"2023-10-18T10:43:04.414558+0000","last_fullsized":"2023-10-18T10:43:04.414558+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:05.181231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414718+0000","last_change":"2023-10-18T10:43:03.418317+0000","last_active":"2023-10-18T10:43:04.414718+0000","last_peered":"2023-10-18T10:43:04.414718+0000","last_clean":"2023-10-18T10:43:04.414718+0000","last_became_active":"2023-10-18T10:43:03.418040+0000","last_became_peered":"2023-10-18T10:43:03.418040+0000","last_unstale":"2023-10-18T10:43:04.414718+0000","last_undegraded":"2023-10-18T10:43:04.414718+0000","last_fullsized":"2023-10-18T10:43:04.414718+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:27.849033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564364+0000","last_change":"2023-10-18T10:43:03.419666+0000","last_active":"2023-10-18T10:43:04.564364+0000","last_peered":"2023-10-18T10:43:04.564364+0000","last_clean":"2023-10-18T10:43:04.564364+0000","last_became_active":"2023-10-18T10:43:03.418843+0000","last_became_peered":"2023-10-18T10:43:03.418843+0000","last_unstale":"2023-10-18T10:43:04.564364+0000","last_undegraded":"2023-10-18T10:43:04.564364+0000","last_fullsized":"2023-10-18T10:43:04.564364+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:25.903261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565004+0000","last_change":"2023-10-18T10:43:03.423183+0000","last_active":"2023-10-18T10:43:04.565004+0000","last_peered":"2023-10-18T10:43:04.565004+0000","last_clean":"2023-10-18T10:43:04.565004+0000","last_became_active":"2023-10-18T10:43:03.422819+0000","last_became_peered":"2023-10-18T10:43:03.422819+0000","last_unstale":"2023-10-18T10:43:04.565004+0000","last_undegraded":"2023-10-18T10:43:04.565004+0000","last_fullsized":"2023-10-18T10:43:04.565004+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:34.324413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564301+0000","last_change":"2023-10-18T10:43:03.417982+0000","last_active":"2023-10-18T10:43:04.564301+0000","last_peered":"2023-10-18T10:43:04.564301+0000","last_clean":"2023-10-18T10:43:04.564301+0000","last_became_active":"2023-10-18T10:43:03.417736+0000","last_became_peered":"2023-10-18T10:43:03.417736+0000","last_unstale":"2023-10-18T10:43:04.564301+0000","last_undegraded":"2023-10-18T10:43:04.564301+0000","last_fullsized":"2023-10-18T10:43:04.564301+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:51.640858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414313+0000","last_change":"2023-10-18T10:43:03.422207+0000","last_active":"2023-10-18T10:43:04.414313+0000","last_peered":"2023-10-18T10:43:04.414313+0000","last_clean":"2023-10-18T10:43:04.414313+0000","last_became_active":"2023-10-18T10:43:03.421941+0000","last_became_peered":"2023-10-18T10:43:03.421941+0000","last_unstale":"2023-10-18T10:43:04.414313+0000","last_undegraded":"2023-10-18T10:43:04.414313+0000","last_fullsized":"2023-10-18T10:43:04.414313+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:36.486872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564774+0000","last_change":"2023-10-18T10:43:03.416816+0000","last_active":"2023-10-18T10:43:04.564774+0000","last_peered":"2023-10-18T10:43:04.564774+0000","last_clean":"2023-10-18T10:43:04.564774+0000","last_became_active":"2023-10-18T10:43:03.416441+0000","last_became_peered":"2023-10-18T10:43:03.416441+0000","last_unstale":"2023-10-18T10:43:04.564774+0000","last_undegraded":"2023-10-18T10:43:04.564774+0000","last_fullsized":"2023-10-18T10:43:04.564774+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:23.903407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563541+0000","last_change":"2023-10-18T10:43:03.423095+0000","last_active":"2023-10-18T10:43:04.563541+0000","last_peered":"2023-10-18T10:43:04.563541+0000","last_clean":"2023-10-18T10:43:04.563541+0000","last_became_active":"2023-10-18T10:43:03.422840+0000","last_became_peered":"2023-10-18T10:43:03.422840+0000","last_unstale":"2023-10-18T10:43:04.563541+0000","last_undegraded":"2023-10-18T10:43:04.563541+0000","last_fullsized":"2023-10-18T10:43:04.563541+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:01:09.585007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564675+0000","last_change":"2023-10-18T10:43:03.423114+0000","last_active":"2023-10-18T10:43:04.564675+0000","last_peered":"2023-10-18T10:43:04.564675+0000","last_clean":"2023-10-18T10:43:04.564675+0000","last_became_active":"2023-10-18T10:43:03.422862+0000","last_became_peered":"2023-10-18T10:43:03.422862+0000","last_unstale":"2023-10-18T10:43:04.564675+0000","last_undegraded":"2023-10-18T10:43:04.564675+0000","last_fullsized":"2023-10-18T10:43:04.564675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:40:54.685089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"127'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408994+0000","last_change":"2023-10-18T10:43:02.963797+0000","last_active":"2023-10-18T10:43:04.408994+0000","last_peered":"2023-10-18T10:43:04.408994+0000","last_clean":"2023-10-18T10:43:04.408994+0000","last_became_active":"2023-10-18T10:43:02.963525+0000","last_became_peered":"2023-10-18T10:43:02.963525+0000","last_unstale":"2023-10-18T10:43:04.408994+0000","last_undegraded":"2023-10-18T10:43:04.408994+0000","last_fullsized":"2023-10-18T10:43:04.408994+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:18:22.535089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563557+0000","last_change":"2023-10-18T10:43:03.419783+0000","last_active":"2023-10-18T10:43:04.563557+0000","last_peered":"2023-10-18T10:43:04.563557+0000","last_clean":"2023-10-18T10:43:04.563557+0000","last_became_active":"2023-10-18T10:43:03.419543+0000","last_became_peered":"2023-10-18T10:43:03.419543+0000","last_unstale":"2023-10-18T10:43:04.563557+0000","last_undegraded":"2023-10-18T10:43:04.563557+0000","last_fullsized":"2023-10-18T10:43:04.563557+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:39.668528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564617+0000","last_change":"2023-10-18T10:43:02.965235+0000","last_active":"2023-10-18T10:43:04.564617+0000","last_peered":"2023-10-18T10:43:04.564617+0000","last_clean":"2023-10-18T10:43:04.564617+0000","last_became_active":"2023-10-18T10:43:02.964880+0000","last_became_peered":"2023-10-18T10:43:02.964880+0000","last_unstale":"2023-10-18T10:43:04.564617+0000","last_undegraded":"2023-10-18T10:43:04.564617+0000","last_fullsized":"2023-10-18T10:43:04.564617+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:48.610823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414516+0000","last_change":"2023-10-18T10:43:03.408432+0000","last_active":"2023-10-18T10:43:04.414516+0000","last_peered":"2023-10-18T10:43:04.414516+0000","last_clean":"2023-10-18T10:43:04.414516+0000","last_became_active":"2023-10-18T10:43:03.408124+0000","last_became_peered":"2023-10-18T10:43:03.408124+0000","last_unstale":"2023-10-18T10:43:04.414516+0000","last_undegraded":"2023-10-18T10:43:04.414516+0000","last_fullsized":"2023-10-18T10:43:04.414516+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:15.686252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414444+0000","last_change":"2023-10-18T10:43:02.970656+0000","last_active":"2023-10-18T10:43:04.414444+0000","last_peered":"2023-10-18T10:43:04.414444+0000","last_clean":"2023-10-18T10:43:04.414444+0000","last_became_active":"2023-10-18T10:43:02.970399+0000","last_became_peered":"2023-10-18T10:43:02.970399+0000","last_unstale":"2023-10-18T10:43:04.414444+0000","last_undegraded":"2023-10-18T10:43:04.414444+0000","last_fullsized":"2023-10-18T10:43:04.414444+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:22.897834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564182+0000","last_change":"2023-10-18T10:43:03.416904+0000","last_active":"2023-10-18T10:43:04.564182+0000","last_peered":"2023-10-18T10:43:04.564182+0000","last_clean":"2023-10-18T10:43:04.564182+0000","last_became_active":"2023-10-18T10:43:03.416489+0000","last_became_peered":"2023-10-18T10:43:03.416489+0000","last_unstale":"2023-10-18T10:43:04.564182+0000","last_undegraded":"2023-10-18T10:43:04.564182+0000","last_fullsized":"2023-10-18T10:43:04.564182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:36.136092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564337+0000","last_change":"2023-10-18T10:43:02.969842+0000","last_active":"2023-10-18T10:43:04.564337+0000","last_peered":"2023-10-18T10:43:04.564337+0000","last_clean":"2023-10-18T10:43:04.564337+0000","last_became_active":"2023-10-18T10:43:02.969620+0000","last_became_peered":"2023-10-18T10:43:02.969620+0000","last_unstale":"2023-10-18T10:43:04.564337+0000","last_undegraded":"2023-10-18T10:43:04.564337+0000","last_fullsized":"2023-10-18T10:43:04.564337+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:23:14.364024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409001+0000","last_change":"2023-10-18T10:43:03.417690+0000","last_active":"2023-10-18T10:43:04.409001+0000","last_peered":"2023-10-18T10:43:04.409001+0000","last_clean":"2023-10-18T10:43:04.409001+0000","last_became_active":"2023-10-18T10:43:03.417452+0000","last_became_peered":"2023-10-18T10:43:03.417452+0000","last_unstale":"2023-10-18T10:43:04.409001+0000","last_undegraded":"2023-10-18T10:43:04.409001+0000","last_fullsized":"2023-10-18T10:43:04.409001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:28.022888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563896+0000","last_change":"2023-10-18T10:43:02.965103+0000","last_active":"2023-10-18T10:43:04.563896+0000","last_peered":"2023-10-18T10:43:04.563896+0000","last_clean":"2023-10-18T10:43:04.563896+0000","last_became_active":"2023-10-18T10:43:02.964812+0000","last_became_peered":"2023-10-18T10:43:02.964812+0000","last_unstale":"2023-10-18T10:43:04.563896+0000","last_undegraded":"2023-10-18T10:43:04.563896+0000","last_fullsized":"2023-10-18T10:43:04.563896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:08.011562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564304+0000","last_change":"2023-10-18T10:43:03.424682+0000","last_active":"2023-10-18T10:43:04.564304+0000","last_peered":"2023-10-18T10:43:04.564304+0000","last_clean":"2023-10-18T10:43:04.564304+0000","last_became_active":"2023-10-18T10:43:03.424416+0000","last_became_peered":"2023-10-18T10:43:03.424416+0000","last_unstale":"2023-10-18T10:43:04.564304+0000","last_undegraded":"2023-10-18T10:43:04.564304+0000","last_fullsized":"2023-10-18T10:43:04.564304+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:29.729372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564479+0000","last_change":"2023-10-18T10:43:02.966346+0000","last_active":"2023-10-18T10:43:04.564479+0000","last_peered":"2023-10-18T10:43:04.564479+0000","last_clean":"2023-10-18T10:43:04.564479+0000","last_became_active":"2023-10-18T10:43:02.966041+0000","last_became_peered":"2023-10-18T10:43:02.966041+0000","last_unstale":"2023-10-18T10:43:04.564479+0000","last_undegraded":"2023-10-18T10:43:04.564479+0000","last_fullsized":"2023-10-18T10:43:04.564479+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:04.134775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565269+0000","last_change":"2023-10-18T10:43:03.424161+0000","last_active":"2023-10-18T10:43:04.565269+0000","last_peered":"2023-10-18T10:43:04.565269+0000","last_clean":"2023-10-18T10:43:04.565269+0000","last_became_active":"2023-10-18T10:43:03.423854+0000","last_became_peered":"2023-10-18T10:43:03.423854+0000","last_unstale":"2023-10-18T10:43:04.565269+0000","last_undegraded":"2023-10-18T10:43:04.565269+0000","last_fullsized":"2023-10-18T10:43:04.565269+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:24.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414223+0000","last_change":"2023-10-18T10:43:02.970449+0000","last_active":"2023-10-18T10:43:04.414223+0000","last_peered":"2023-10-18T10:43:04.414223+0000","last_clean":"2023-10-18T10:43:04.414223+0000","last_became_active":"2023-10-18T10:43:02.970145+0000","last_became_peered":"2023-10-18T10:43:02.970145+0000","last_unstale":"2023-10-18T10:43:04.414223+0000","last_undegraded":"2023-10-18T10:43:04.414223+0000","last_fullsized":"2023-10-18T10:43:04.414223+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:56.158300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.409024+0000","last_change":"2023-10-18T10:43:03.417182+0000","last_active":"2023-10-18T10:43:04.409024+0000","last_peered":"2023-10-18T10:43:04.409024+0000","last_clean":"2023-10-18T10:43:04.409024+0000","last_became_active":"2023-10-18T10:43:03.416917+0000","last_became_peered":"2023-10-18T10:43:03.416917+0000","last_unstale":"2023-10-18T10:43:04.409024+0000","last_undegraded":"2023-10-18T10:43:04.409024+0000","last_fullsized":"2023-10-18T10:43:04.409024+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:49.418978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408682+0000","last_change":"2023-10-18T10:43:02.969298+0000","last_active":"2023-10-18T10:43:04.408682+0000","last_peered":"2023-10-18T10:43:04.408682+0000","last_clean":"2023-10-18T10:43:04.408682+0000","last_became_active":"2023-10-18T10:43:02.969045+0000","last_became_peered":"2023-10-18T10:43:02.969045+0000","last_unstale":"2023-10-18T10:43:04.408682+0000","last_undegraded":"2023-10-18T10:43:04.408682+0000","last_fullsized":"2023-10-18T10:43:04.408682+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:24.897688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564560+0000","last_change":"2023-10-18T10:43:03.424618+0000","last_active":"2023-10-18T10:43:04.564560+0000","last_peered":"2023-10-18T10:43:04.564560+0000","last_clean":"2023-10-18T10:43:04.564560+0000","last_became_active":"2023-10-18T10:43:03.424369+0000","last_became_peered":"2023-10-18T10:43:03.424369+0000","last_unstale":"2023-10-18T10:43:04.564560+0000","last_undegraded":"2023-10-18T10:43:04.564560+0000","last_fullsized":"2023-10-18T10:43:04.564560+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:25.195869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414588+0000","last_change":"2023-10-18T10:43:02.969790+0000","last_active":"2023-10-18T10:43:04.414588+0000","last_peered":"2023-10-18T10:43:04.414588+0000","last_clean":"2023-10-18T10:43:04.414588+0000","last_became_active":"2023-10-18T10:43:02.969434+0000","last_became_peered":"2023-10-18T10:43:02.969434+0000","last_unstale":"2023-10-18T10:43:04.414588+0000","last_undegraded":"2023-10-18T10:43:04.414588+0000","last_fullsized":"2023-10-18T10:43:04.414588+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:33.318840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563691+0000","last_change":"2023-10-18T10:43:03.414895+0000","last_active":"2023-10-18T10:43:04.563691+0000","last_peered":"2023-10-18T10:43:04.563691+0000","last_clean":"2023-10-18T10:43:04.563691+0000","last_became_active":"2023-10-18T10:43:03.414661+0000","last_became_peered":"2023-10-18T10:43:03.414661+0000","last_unstale":"2023-10-18T10:43:04.563691+0000","last_undegraded":"2023-10-18T10:43:04.563691+0000","last_fullsized":"2023-10-18T10:43:04.563691+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:46.699483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564739+0000","last_change":"2023-10-18T10:43:02.971336+0000","last_active":"2023-10-18T10:43:04.564739+0000","last_peered":"2023-10-18T10:43:04.564739+0000","last_clean":"2023-10-18T10:43:04.564739+0000","last_became_active":"2023-10-18T10:43:02.971030+0000","last_became_peered":"2023-10-18T10:43:02.971030+0000","last_unstale":"2023-10-18T10:43:04.564739+0000","last_undegraded":"2023-10-18T10:43:04.564739+0000","last_fullsized":"2023-10-18T10:43:04.564739+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:32:47.348309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563840+0000","last_change":"2023-10-18T10:43:03.420254+0000","last_active":"2023-10-18T10:43:04.563840+0000","last_peered":"2023-10-18T10:43:04.563840+0000","last_clean":"2023-10-18T10:43:04.563840+0000","last_became_active":"2023-10-18T10:43:03.420019+0000","last_became_peered":"2023-10-18T10:43:03.420019+0000","last_unstale":"2023-10-18T10:43:04.563840+0000","last_undegraded":"2023-10-18T10:43:04.563840+0000","last_fullsized":"2023-10-18T10:43:04.563840+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:02.531521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"127'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564189+0000","last_change":"2023-10-18T10:43:02.974197+0000","last_active":"2023-10-18T10:43:04.564189+0000","last_peered":"2023-10-18T10:43:04.564189+0000","last_clean":"2023-10-18T10:43:04.564189+0000","last_became_active":"2023-10-18T10:43:02.973961+0000","last_became_peered":"2023-10-18T10:43:02.973961+0000","last_unstale":"2023-10-18T10:43:04.564189+0000","last_undegraded":"2023-10-18T10:43:04.564189+0000","last_fullsized":"2023-10-18T10:43:04.564189+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:57.736328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564658+0000","last_change":"2023-10-18T10:43:03.417830+0000","last_active":"2023-10-18T10:43:04.564658+0000","last_peered":"2023-10-18T10:43:04.564658+0000","last_clean":"2023-10-18T10:43:04.564658+0000","last_became_active":"2023-10-18T10:43:03.417031+0000","last_became_peered":"2023-10-18T10:43:03.417031+0000","last_unstale":"2023-10-18T10:43:04.564658+0000","last_undegraded":"2023-10-18T10:43:04.564658+0000","last_fullsized":"2023-10-18T10:43:04.564658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:56.478327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"127'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565099+0000","last_change":"2023-10-18T10:43:02.959201+0000","last_active":"2023-10-18T10:43:04.565099+0000","last_peered":"2023-10-18T10:43:04.565099+0000","last_clean":"2023-10-18T10:43:04.565099+0000","last_became_active":"2023-10-18T10:43:02.958888+0000","last_became_peered":"2023-10-18T10:43:02.958888+0000","last_unstale":"2023-10-18T10:43:04.565099+0000","last_undegraded":"2023-10-18T10:43:04.565099+0000","last_fullsized":"2023-10-18T10:43:04.565099+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:04.257834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564597+0000","last_change":"2023-10-18T10:43:03.419410+0000","last_active":"2023-10-18T10:43:04.564597+0000","last_peered":"2023-10-18T10:43:04.564597+0000","last_clean":"2023-10-18T10:43:04.564597+0000","last_became_active":"2023-10-18T10:43:03.419055+0000","last_became_peered":"2023-10-18T10:43:03.419055+0000","last_unstale":"2023-10-18T10:43:04.564597+0000","last_undegraded":"2023-10-18T10:43:04.564597+0000","last_fullsized":"2023-10-18T10:43:04.564597+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:06.895973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564742+0000","last_change":"2023-10-18T10:43:02.972054+0000","last_active":"2023-10-18T10:43:04.564742+0000","last_peered":"2023-10-18T10:43:04.564742+0000","last_clean":"2023-10-18T10:43:04.564742+0000","last_became_active":"2023-10-18T10:43:02.971783+0000","last_became_peered":"2023-10-18T10:43:02.971783+0000","last_unstale":"2023-10-18T10:43:04.564742+0000","last_undegraded":"2023-10-18T10:43:04.564742+0000","last_fullsized":"2023-10-18T10:43:04.564742+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:07.914251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565070+0000","last_change":"2023-10-18T10:43:03.424942+0000","last_active":"2023-10-18T10:43:04.565070+0000","last_peered":"2023-10-18T10:43:04.565070+0000","last_clean":"2023-10-18T10:43:04.565070+0000","last_became_active":"2023-10-18T10:43:03.424610+0000","last_became_peered":"2023-10-18T10:43:03.424610+0000","last_unstale":"2023-10-18T10:43:04.565070+0000","last_undegraded":"2023-10-18T10:43:04.565070+0000","last_fullsized":"2023-10-18T10:43:04.565070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:58.441002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408796+0000","last_change":"2023-10-18T10:43:02.962770+0000","last_active":"2023-10-18T10:43:04.408796+0000","last_peered":"2023-10-18T10:43:04.408796+0000","last_clean":"2023-10-18T10:43:04.408796+0000","last_became_active":"2023-10-18T10:43:02.962492+0000","last_became_peered":"2023-10-18T10:43:02.962492+0000","last_unstale":"2023-10-18T10:43:04.408796+0000","last_undegraded":"2023-10-18T10:43:04.408796+0000","last_fullsized":"2023-10-18T10:43:04.408796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:04.206263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565182+0000","last_change":"2023-10-18T10:43:03.417626+0000","last_active":"2023-10-18T10:43:04.565182+0000","last_peered":"2023-10-18T10:43:04.565182+0000","last_clean":"2023-10-18T10:43:04.565182+0000","last_became_active":"2023-10-18T10:43:03.417374+0000","last_became_peered":"2023-10-18T10:43:03.417374+0000","last_unstale":"2023-10-18T10:43:04.565182+0000","last_undegraded":"2023-10-18T10:43:04.565182+0000","last_fullsized":"2023-10-18T10:43:04.565182+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:39.656533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563634+0000","last_change":"2023-10-18T10:43:02.966586+0000","last_active":"2023-10-18T10:43:04.563634+0000","last_peered":"2023-10-18T10:43:04.563634+0000","last_clean":"2023-10-18T10:43:04.563634+0000","last_became_active":"2023-10-18T10:43:02.966340+0000","last_became_peered":"2023-10-18T10:43:02.966340+0000","last_unstale":"2023-10-18T10:43:04.563634+0000","last_undegraded":"2023-10-18T10:43:04.563634+0000","last_fullsized":"2023-10-18T10:43:04.563634+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.288426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565086+0000","last_change":"2023-10-18T10:43:03.425524+0000","last_active":"2023-10-18T10:43:04.565086+0000","last_peered":"2023-10-18T10:43:04.565086+0000","last_clean":"2023-10-18T10:43:04.565086+0000","last_became_active":"2023-10-18T10:43:03.425299+0000","last_became_peered":"2023-10-18T10:43:03.425299+0000","last_unstale":"2023-10-18T10:43:04.565086+0000","last_undegraded":"2023-10-18T10:43:04.565086+0000","last_fullsized":"2023-10-18T10:43:04.565086+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:18.551908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"127'8","reported_seq":20,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414095+0000","last_change":"2023-10-18T10:43:02.969709+0000","last_active":"2023-10-18T10:43:04.414095+0000","last_peered":"2023-10-18T10:43:04.414095+0000","last_clean":"2023-10-18T10:43:04.414095+0000","last_became_active":"2023-10-18T10:43:02.969463+0000","last_became_peered":"2023-10-18T10:43:02.969463+0000","last_unstale":"2023-10-18T10:43:04.414095+0000","last_undegraded":"2023-10-18T10:43:04.414095+0000","last_fullsized":"2023-10-18T10:43:04.414095+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:45.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563658+0000","last_change":"2023-10-18T10:43:03.415655+0000","last_active":"2023-10-18T10:43:04.563658+0000","last_peered":"2023-10-18T10:43:04.563658+0000","last_clean":"2023-10-18T10:43:04.563658+0000","last_became_active":"2023-10-18T10:43:03.415397+0000","last_became_peered":"2023-10-18T10:43:03.415397+0000","last_unstale":"2023-10-18T10:43:04.563658+0000","last_undegraded":"2023-10-18T10:43:04.563658+0000","last_fullsized":"2023-10-18T10:43:04.563658+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:17.193216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564284+0000","last_change":"2023-10-18T10:43:02.964287+0000","last_active":"2023-10-18T10:43:04.564284+0000","last_peered":"2023-10-18T10:43:04.564284+0000","last_clean":"2023-10-18T10:43:04.564284+0000","last_became_active":"2023-10-18T10:43:02.964033+0000","last_became_peered":"2023-10-18T10:43:02.964033+0000","last_unstale":"2023-10-18T10:43:04.564284+0000","last_undegraded":"2023-10-18T10:43:04.564284+0000","last_fullsized":"2023-10-18T10:43:04.564284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:33.533023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564963+0000","last_change":"2023-10-18T10:43:03.419976+0000","last_active":"2023-10-18T10:43:04.564963+0000","last_peered":"2023-10-18T10:43:04.564963+0000","last_clean":"2023-10-18T10:43:04.564963+0000","last_became_active":"2023-10-18T10:43:03.419611+0000","last_became_peered":"2023-10-18T10:43:03.419611+0000","last_unstale":"2023-10-18T10:43:04.564963+0000","last_undegraded":"2023-10-18T10:43:04.564963+0000","last_fullsized":"2023-10-18T10:43:04.564963+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:57.163873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"127'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564210+0000","last_change":"2023-10-18T10:43:02.964463+0000","last_active":"2023-10-18T10:43:04.564210+0000","last_peered":"2023-10-18T10:43:04.564210+0000","last_clean":"2023-10-18T10:43:04.564210+0000","last_became_active":"2023-10-18T10:43:02.964036+0000","last_became_peered":"2023-10-18T10:43:02.964036+0000","last_unstale":"2023-10-18T10:43:04.564210+0000","last_undegraded":"2023-10-18T10:43:04.564210+0000","last_fullsized":"2023-10-18T10:43:04.564210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:27.017315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564542+0000","last_change":"2023-10-18T10:43:03.416778+0000","last_active":"2023-10-18T10:43:04.564542+0000","last_peered":"2023-10-18T10:43:04.564542+0000","last_clean":"2023-10-18T10:43:04.564542+0000","last_became_active":"2023-10-18T10:43:03.416492+0000","last_became_peered":"2023-10-18T10:43:03.416492+0000","last_unstale":"2023-10-18T10:43:04.564542+0000","last_undegraded":"2023-10-18T10:43:04.564542+0000","last_fullsized":"2023-10-18T10:43:04.564542+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:29.332215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564773+0000","last_change":"2023-10-18T10:43:02.965244+0000","last_active":"2023-10-18T10:43:04.564773+0000","last_peered":"2023-10-18T10:43:04.564773+0000","last_clean":"2023-10-18T10:43:04.564773+0000","last_became_active":"2023-10-18T10:43:02.964962+0000","last_became_peered":"2023-10-18T10:43:02.964962+0000","last_unstale":"2023-10-18T10:43:04.564773+0000","last_undegraded":"2023-10-18T10:43:04.564773+0000","last_fullsized":"2023-10-18T10:43:04.564773+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:10.229768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563816+0000","last_change":"2023-10-18T10:43:03.419372+0000","last_active":"2023-10-18T10:43:04.563816+0000","last_peered":"2023-10-18T10:43:04.563816+0000","last_clean":"2023-10-18T10:43:04.563816+0000","last_became_active":"2023-10-18T10:43:03.419133+0000","last_became_peered":"2023-10-18T10:43:03.419133+0000","last_unstale":"2023-10-18T10:43:04.563816+0000","last_undegraded":"2023-10-18T10:43:04.563816+0000","last_fullsized":"2023-10-18T10:43:04.563816+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:34:25.080148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564382+0000","last_change":"2023-10-18T10:43:02.963180+0000","last_active":"2023-10-18T10:43:04.564382+0000","last_peered":"2023-10-18T10:43:04.564382+0000","last_clean":"2023-10-18T10:43:04.564382+0000","last_became_active":"2023-10-18T10:43:02.962924+0000","last_became_peered":"2023-10-18T10:43:02.962924+0000","last_unstale":"2023-10-18T10:43:04.564382+0000","last_undegraded":"2023-10-18T10:43:04.564382+0000","last_fullsized":"2023-10-18T10:43:04.564382+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:27.605859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564165+0000","last_change":"2023-10-18T10:43:03.424395+0000","last_active":"2023-10-18T10:43:04.564165+0000","last_peered":"2023-10-18T10:43:04.564165+0000","last_clean":"2023-10-18T10:43:04.564165+0000","last_became_active":"2023-10-18T10:43:03.424101+0000","last_became_peered":"2023-10-18T10:43:03.424101+0000","last_unstale":"2023-10-18T10:43:04.564165+0000","last_undegraded":"2023-10-18T10:43:04.564165+0000","last_fullsized":"2023-10-18T10:43:04.564165+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:43.317979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563722+0000","last_change":"2023-10-18T10:43:02.964389+0000","last_active":"2023-10-18T10:43:04.563722+0000","last_peered":"2023-10-18T10:43:04.563722+0000","last_clean":"2023-10-18T10:43:04.563722+0000","last_became_active":"2023-10-18T10:43:02.964134+0000","last_became_peered":"2023-10-18T10:43:02.964134+0000","last_unstale":"2023-10-18T10:43:04.563722+0000","last_undegraded":"2023-10-18T10:43:04.563722+0000","last_fullsized":"2023-10-18T10:43:04.563722+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:27.504789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408932+0000","last_change":"2023-10-18T10:43:03.417105+0000","last_active":"2023-10-18T10:43:04.408932+0000","last_peered":"2023-10-18T10:43:04.408932+0000","last_clean":"2023-10-18T10:43:04.408932+0000","last_became_active":"2023-10-18T10:43:03.416864+0000","last_became_peered":"2023-10-18T10:43:03.416864+0000","last_unstale":"2023-10-18T10:43:04.408932+0000","last_undegraded":"2023-10-18T10:43:04.408932+0000","last_fullsized":"2023-10-18T10:43:04.408932+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:09.017135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564862+0000","last_change":"2023-10-18T10:43:02.967733+0000","last_active":"2023-10-18T10:43:04.564862+0000","last_peered":"2023-10-18T10:43:04.564862+0000","last_clean":"2023-10-18T10:43:04.564862+0000","last_became_active":"2023-10-18T10:43:02.967444+0000","last_became_peered":"2023-10-18T10:43:02.967444+0000","last_unstale":"2023-10-18T10:43:04.564862+0000","last_undegraded":"2023-10-18T10:43:04.564862+0000","last_fullsized":"2023-10-18T10:43:04.564862+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:59.437631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564910+0000","last_change":"2023-10-18T10:43:03.418676+0000","last_active":"2023-10-18T10:43:04.564910+0000","last_peered":"2023-10-18T10:43:04.564910+0000","last_clean":"2023-10-18T10:43:04.564910+0000","last_became_active":"2023-10-18T10:43:03.418441+0000","last_became_peered":"2023-10-18T10:43:03.418441+0000","last_unstale":"2023-10-18T10:43:04.564910+0000","last_undegraded":"2023-10-18T10:43:04.564910+0000","last_fullsized":"2023-10-18T10:43:04.564910+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:50:30.204629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563690+0000","last_change":"2023-10-18T10:43:02.963728+0000","last_active":"2023-10-18T10:43:04.563690+0000","last_peered":"2023-10-18T10:43:04.563690+0000","last_clean":"2023-10-18T10:43:04.563690+0000","last_became_active":"2023-10-18T10:43:02.963467+0000","last_became_peered":"2023-10-18T10:43:02.963467+0000","last_unstale":"2023-10-18T10:43:04.563690+0000","last_undegraded":"2023-10-18T10:43:04.563690+0000","last_fullsized":"2023-10-18T10:43:04.563690+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:38.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564445+0000","last_change":"2023-10-18T10:43:03.421100+0000","last_active":"2023-10-18T10:43:04.564445+0000","last_peered":"2023-10-18T10:43:04.564445+0000","last_clean":"2023-10-18T10:43:04.564445+0000","last_became_active":"2023-10-18T10:43:03.420845+0000","last_became_peered":"2023-10-18T10:43:03.420845+0000","last_unstale":"2023-10-18T10:43:04.564445+0000","last_undegraded":"2023-10-18T10:43:04.564445+0000","last_fullsized":"2023-10-18T10:43:04.564445+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:32:19.619118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564074+0000","last_change":"2023-10-18T10:43:02.971128+0000","last_active":"2023-10-18T10:43:04.564074+0000","last_peered":"2023-10-18T10:43:04.564074+0000","last_clean":"2023-10-18T10:43:04.564074+0000","last_became_active":"2023-10-18T10:43:02.970845+0000","last_became_peered":"2023-10-18T10:43:02.970845+0000","last_unstale":"2023-10-18T10:43:04.564074+0000","last_undegraded":"2023-10-18T10:43:04.564074+0000","last_fullsized":"2023-10-18T10:43:04.564074+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:57.435429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414114+0000","last_change":"2023-10-18T10:43:03.414544+0000","last_active":"2023-10-18T10:43:04.414114+0000","last_peered":"2023-10-18T10:43:04.414114+0000","last_clean":"2023-10-18T10:43:04.414114+0000","last_became_active":"2023-10-18T10:43:03.414270+0000","last_became_peered":"2023-10-18T10:43:03.414270+0000","last_unstale":"2023-10-18T10:43:04.414114+0000","last_undegraded":"2023-10-18T10:43:04.414114+0000","last_fullsized":"2023-10-18T10:43:04.414114+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:03.106690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564812+0000","last_change":"2023-10-18T10:43:02.972906+0000","last_active":"2023-10-18T10:43:04.564812+0000","last_peered":"2023-10-18T10:43:04.564812+0000","last_clean":"2023-10-18T10:43:04.564812+0000","last_became_active":"2023-10-18T10:43:02.972652+0000","last_became_peered":"2023-10-18T10:43:02.972652+0000","last_unstale":"2023-10-18T10:43:04.564812+0000","last_undegraded":"2023-10-18T10:43:04.564812+0000","last_fullsized":"2023-10-18T10:43:04.564812+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:09.602406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.408779+0000","last_change":"2023-10-18T10:43:03.416563+0000","last_active":"2023-10-18T10:43:04.408779+0000","last_peered":"2023-10-18T10:43:04.408779+0000","last_clean":"2023-10-18T10:43:04.408779+0000","last_became_active":"2023-10-18T10:43:03.416321+0000","last_became_peered":"2023-10-18T10:43:03.416321+0000","last_unstale":"2023-10-18T10:43:04.408779+0000","last_undegraded":"2023-10-18T10:43:04.408779+0000","last_fullsized":"2023-10-18T10:43:04.408779+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:57.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.565167+0000","last_change":"2023-10-18T10:43:02.962995+0000","last_active":"2023-10-18T10:43:04.565167+0000","last_peered":"2023-10-18T10:43:04.565167+0000","last_clean":"2023-10-18T10:43:04.565167+0000","last_became_active":"2023-10-18T10:43:02.962723+0000","last_became_peered":"2023-10-18T10:43:02.962723+0000","last_unstale":"2023-10-18T10:43:04.565167+0000","last_undegraded":"2023-10-18T10:43:04.565167+0000","last_fullsized":"2023-10-18T10:43:04.565167+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:45.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564038+0000","last_change":"2023-10-18T10:43:03.424431+0000","last_active":"2023-10-18T10:43:04.564038+0000","last_peered":"2023-10-18T10:43:04.564038+0000","last_clean":"2023-10-18T10:43:04.564038+0000","last_became_active":"2023-10-18T10:43:03.424173+0000","last_became_peered":"2023-10-18T10:43:03.424173+0000","last_unstale":"2023-10-18T10:43:04.564038+0000","last_undegraded":"2023-10-18T10:43:04.564038+0000","last_fullsized":"2023-10-18T10:43:04.564038+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:19.931739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564415+0000","last_change":"2023-10-18T10:43:02.968945+0000","last_active":"2023-10-18T10:43:04.564415+0000","last_peered":"2023-10-18T10:43:04.564415+0000","last_clean":"2023-10-18T10:43:04.564415+0000","last_became_active":"2023-10-18T10:43:02.968666+0000","last_became_peered":"2023-10-18T10:43:02.968666+0000","last_unstale":"2023-10-18T10:43:04.564415+0000","last_undegraded":"2023-10-18T10:43:04.564415+0000","last_fullsized":"2023-10-18T10:43:04.564415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:26.843460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564480+0000","last_change":"2023-10-18T10:43:03.421780+0000","last_active":"2023-10-18T10:43:04.564480+0000","last_peered":"2023-10-18T10:43:04.564480+0000","last_clean":"2023-10-18T10:43:04.564480+0000","last_became_active":"2023-10-18T10:43:03.421547+0000","last_became_peered":"2023-10-18T10:43:03.421547+0000","last_unstale":"2023-10-18T10:43:04.564480+0000","last_undegraded":"2023-10-18T10:43:04.564480+0000","last_fullsized":"2023-10-18T10:43:04.564480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:54:47.177774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.414258+0000","last_change":"2023-10-18T10:43:02.965516+0000","last_active":"2023-10-18T10:43:04.414258+0000","last_peered":"2023-10-18T10:43:04.414258+0000","last_clean":"2023-10-18T10:43:04.414258+0000","last_became_active":"2023-10-18T10:43:02.965251+0000","last_became_peered":"2023-10-18T10:43:02.965251+0000","last_unstale":"2023-10-18T10:43:04.414258+0000","last_undegraded":"2023-10-18T10:43:04.414258+0000","last_fullsized":"2023-10-18T10:43:04.414258+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:59.529022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563859+0000","last_change":"2023-10-18T10:43:03.423032+0000","last_active":"2023-10-18T10:43:04.563859+0000","last_peered":"2023-10-18T10:43:04.563859+0000","last_clean":"2023-10-18T10:43:04.563859+0000","last_became_active":"2023-10-18T10:43:03.422767+0000","last_became_peered":"2023-10-18T10:43:03.422767+0000","last_unstale":"2023-10-18T10:43:04.563859+0000","last_undegraded":"2023-10-18T10:43:04.563859+0000","last_fullsized":"2023-10-18T10:43:04.563859+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:17.615763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563911+0000","last_change":"2023-10-18T10:43:02.965062+0000","last_active":"2023-10-18T10:43:04.563911+0000","last_peered":"2023-10-18T10:43:04.563911+0000","last_clean":"2023-10-18T10:43:04.563911+0000","last_became_active":"2023-10-18T10:43:02.964753+0000","last_became_peered":"2023-10-18T10:43:02.964753+0000","last_unstale":"2023-10-18T10:43:04.563911+0000","last_undegraded":"2023-10-18T10:43:04.563911+0000","last_fullsized":"2023-10-18T10:43:04.563911+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:41.151549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564340+0000","last_change":"2023-10-18T10:43:03.421221+0000","last_active":"2023-10-18T10:43:04.564340+0000","last_peered":"2023-10-18T10:43:04.564340+0000","last_clean":"2023-10-18T10:43:04.564340+0000","last_became_active":"2023-10-18T10:43:03.420954+0000","last_became_peered":"2023-10-18T10:43:03.420954+0000","last_unstale":"2023-10-18T10:43:04.564340+0000","last_undegraded":"2023-10-18T10:43:04.564340+0000","last_fullsized":"2023-10-18T10:43:04.564340+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:42.945030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564932+0000","last_change":"2023-10-18T10:43:02.962374+0000","last_active":"2023-10-18T10:43:04.564932+0000","last_peered":"2023-10-18T10:43:04.564932+0000","last_clean":"2023-10-18T10:43:04.564932+0000","last_became_active":"2023-10-18T10:43:02.962065+0000","last_became_peered":"2023-10-18T10:43:02.962065+0000","last_unstale":"2023-10-18T10:43:04.564932+0000","last_undegraded":"2023-10-18T10:43:04.564932+0000","last_fullsized":"2023-10-18T10:43:04.564932+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:37.693671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564809+0000","last_change":"2023-10-18T10:43:03.418250+0000","last_active":"2023-10-18T10:43:04.564809+0000","last_peered":"2023-10-18T10:43:04.564809+0000","last_clean":"2023-10-18T10:43:04.564809+0000","last_became_active":"2023-10-18T10:43:03.417934+0000","last_became_peered":"2023-10-18T10:43:03.417934+0000","last_unstale":"2023-10-18T10:43:04.564809+0000","last_undegraded":"2023-10-18T10:43:04.564809+0000","last_fullsized":"2023-10-18T10:43:04.564809+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:09.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563736+0000","last_change":"2023-10-18T10:43:02.965897+0000","last_active":"2023-10-18T10:43:04.563736+0000","last_peered":"2023-10-18T10:43:04.563736+0000","last_clean":"2023-10-18T10:43:04.563736+0000","last_became_active":"2023-10-18T10:43:02.965636+0000","last_became_peered":"2023-10-18T10:43:02.965636+0000","last_unstale":"2023-10-18T10:43:04.563736+0000","last_undegraded":"2023-10-18T10:43:04.563736+0000","last_fullsized":"2023-10-18T10:43:04.563736+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:10:39.372132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564606+0000","last_change":"2023-10-18T10:43:03.420416+0000","last_active":"2023-10-18T10:43:04.564606+0000","last_peered":"2023-10-18T10:43:04.564606+0000","last_clean":"2023-10-18T10:43:04.564606+0000","last_became_active":"2023-10-18T10:43:03.420064+0000","last_became_peered":"2023-10-18T10:43:03.420064+0000","last_unstale":"2023-10-18T10:43:04.564606+0000","last_undegraded":"2023-10-18T10:43:04.564606+0000","last_fullsized":"2023-10-18T10:43:04.564606+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:02.948282+0000","last_clean_scrub_stamp":"2023-10-18T10:43:02.948282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:01.892160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"127'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.564707+0000","last_change":"2023-10-18T10:43:02.962533+0000","last_active":"2023-10-18T10:43:04.564707+0000","last_peered":"2023-10-18T10:43:04.564707+0000","last_clean":"2023-10-18T10:43:04.564707+0000","last_became_active":"2023-10-18T10:43:02.962275+0000","last_became_peered":"2023-10-18T10:43:02.962275+0000","last_unstale":"2023-10-18T10:43:04.564707+0000","last_undegraded":"2023-10-18T10:43:04.564707+0000","last_fullsized":"2023-10-18T10:43:04.564707+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:43:01.942709+0000","last_clean_scrub_stamp":"2023-10-18T10:43:01.942709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:26.189773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"117'214","reported_seq":611,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-18T10:43:04.563741+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:43:04.563741+0000","last_peered":"2023-10-18T10:43:04.563741+0000","last_clean":"2023-10-18T10:43:04.563741+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:43:04.563741+0000","last_undegraded":"2023-10-18T10:43:04.563741+0000","last_fullsized":"2023-10-18T10:43:04.563741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":962,"num_read_kb":1939,"num_write":335,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575068,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59692,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312148,"statfs":{"total":96636764160,"available":96575639552,"internally_reserved":0,"allocated":2187264,"data_stored":1388162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.58299999999999996},"min":{"1min":0.36099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.754,"5min":0.876,"15min":0.88300000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.78400000000000003,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.33800000000000002,"15min":0.32400000000000001},"max":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63200000000000001,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70299999999999996,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.60299999999999998},"min":{"1min":0.42199999999999999,"5min":0.36599999999999999,"15min":0.34300000000000003},"max":{"1min":0.76000000000000001,"5min":0.79800000000000004,"15min":0.85799999999999998},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.68899999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.315},"max":{"1min":0.78500000000000003,"5min":0.875,"15min":0.875},"last":0.504}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.73499999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575068,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":799850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82599999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.28899999999999998},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.36399999999999999,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.69499999999999995,"5min":0.77700000000000002,"15min":0.82699999999999996},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65800000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.432}]},{"osd":3,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.32000000000000001,"15min":0.318},"max":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.78400000000000003,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76700000000000002,"5min":0.77900000000000003,"15min":0.79100000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.748,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:42:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.80300000000000005,"5min":0.82199999999999995,"15min":0.87},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":797634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.39600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88400000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81399999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.81999999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.41199999999999998,"5min":0.40999999999999998,"15min":0.378},"max":{"1min":0.83399999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.47599999999999998},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71599999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.375},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.496},"min":{"1min":0.373,"5min":0.31,"15min":0.29699999999999999},"max":{"1min":0.69799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.38900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.38200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77200000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:42:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.312,"15min":0.31},"max":{"1min":0.84299999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.443},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.44700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":0.871,"15min":0.871},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575068,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1593344,"data_stored":797726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57699999999999996,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.26000000000000001},"max":{"1min":0.77000000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.501,"5min":0.54700000000000004,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.753,"15min":0.877},"last":0.63300000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.89300000000000002},"last":0.439}]},{"osd":2,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.73499999999999999,"5min":0.873,"15min":0.873},"last":0.76700000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71899999999999997,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.78200000000000003},"last":0.81100000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.33200000000000002,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.79200000000000004,"5min":1.446,"15min":1.446},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.79600000000000004,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30099999999999999,"15min":0.29299999999999998},"max":{"1min":0.81999999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.27100000000000002},"max":{"1min":0.879,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.874}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35299999999999998,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.52800000000000002,"5min":0.58999999999999997,"15min":0.62},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.499,"5min":0.59299999999999997,"15min":0.59799999999999998},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.379,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.624,"5min":0.64100000000000001,"15min":0.70699999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.59299999999999997,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.30199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53400000000000003,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.379},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.434,"15min":0.45100000000000001},"min":{"1min":0.28599999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.80200000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.879,"15min":0.94699999999999995},"last":0.5}]},{"osd":5,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":1.002,"15min":1.002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.60999999999999999,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88900000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.629,"15min":0.63800000000000001},"min":{"1min":0.434,"5min":0.38,"15min":0.37},"max":{"1min":0.82499999999999996,"5min":1.018,"15min":1.018},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.751,"5min":0.89100000000000001,"15min":0.92600000000000005},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:42:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.63900000000000001},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85299999999999998,"5min":0.96699999999999997,"15min":0.96799999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.627,"15min":0.65300000000000002},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54940,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316900,"statfs":{"total":96636764160,"available":96580505600,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.371,"15min":0.374},"min":{"1min":0.252,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.626},"last":0.377}]},{"osd":1,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29899999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.57899999999999996,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.315}]},{"osd":3,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.24299999999999999},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.434,"15min":0.44700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.48399999999999999,"5min":0.41899999999999998,"15min":0.254},"max":{"1min":0.86499999999999999,"5min":0.88200000000000001,"15min":0.88800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.625,"15min":0.64500000000000002},"min":{"1min":0.45300000000000001,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.61099999999999999},"min":{"1min":0.46200000000000002,"5min":0.38100000000000001,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.89800000000000002,"15min":0.95699999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.96699999999999997},"last":0.63},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.93700000000000006},"last":0.57299999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:42:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36399999999999999},"max":{"1min":0.80700000000000005,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.84599999999999997,"15min":0.92500000000000004},"last":0.54100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1589248,"data_stored":797668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36199999999999999,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.254,"5min":0.217,"15min":0.20100000000000001},"max":{"1min":0.48999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.32000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.39200000000000002},"min":{"1min":0.26300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.374},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.42099999999999999},"min":{"1min":0.314,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53600000000000003,"5min":0.627,"15min":0.64800000000000002},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.33100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59499999999999997,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.27500000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.33500000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.83999999999999997,"15min":0.95599999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.623},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.753,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.374},"max":{"1min":0.71799999999999997,"5min":0.88100000000000001,"15min":0.95999999999999996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60199999999999998,"15min":0.623},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.316},"max":{"1min":0.76200000000000001,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.63700000000000001},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.78000000000000003,"5min":1.03,"15min":1.03},"last":0.749}]},{"osd":7,"last update":"Wed Oct 18 10:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61399999999999999,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":0.74399999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.63},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78800000000000003,"5min":0.93300000000000005,"15min":0.98199999999999998},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575067,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2195456,"data_stored":1389704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.20999999999999999,"15min":0.192},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.39000000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.54600000000000004,"5min":0.62,"15min":0.76900000000000002},"last":0.214}]},{"osd":2,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.40699999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.187},"max":{"1min":0.58099999999999996,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.41999999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.29299999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.39500000000000002},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56599999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44,"15min":0.45200000000000001},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.71999999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.312}]},{"osd":4,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.61699999999999999},"min":{"1min":0.372,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81999999999999995,"5min":0.996,"15min":0.996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90300000000000002,"5min":1.038,"15min":1.038},"last":0.34200000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95099999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.89200000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93700000000000006,"5min":1.026,"15min":1.3700000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.64000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88100000000000001,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:42:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.64300000000000002},"min":{"1min":0.41199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":1.05,"15min":1.339},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.64600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:43:11.600 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:43:11.600 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:43:11.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.740+0000 7f40dcb4c640 1 -- 172.21.15.125:0/820387702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80c9dc0 msgr2=0x7f40d806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.740+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/820387702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80c9dc0 0x7f40d806b2e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f40cc005fd0 tx=0x7f40cc02be10 comp rx=0 tx=0).stop 2023-10-18T10:43:11.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.740+0000 7f40dcb4c640 1 -- 172.21.15.125:0/820387702 shutdown_connections 2023-10-18T10:43:11.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.740+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/820387702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40d80ab500 0x7f40d80ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.740+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/820387702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80c9dc0 0x7f40d806b2e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.740+0000 7f40dcb4c640 1 -- 172.21.15.125:0/820387702 >> 172.21.15.125:0/820387702 conn(0x7f40d8069930 msgr2=0x7f40d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:11.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.740+0000 7f40dcb4c640 1 -- 172.21.15.125:0/820387702 shutdown_connections 2023-10-18T10:43:11.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.740+0000 7f40dcb4c640 1 -- 172.21.15.125:0/820387702 wait complete. 2023-10-18T10:43:11.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.741+0000 7f40dcb4c640 1 Processor -- start 2023-10-18T10:43:11.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.741+0000 7f40dcb4c640 1 -- start start 2023-10-18T10:43:11.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40dcb4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40d806af00 0x7f40d813a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:11.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40dcb4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80ab500 0x7f40d80a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:11.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40dcb4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40d80c9dc0 0x7f40d80ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:11.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40dcb4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d813ac50 con 0x7f40d80ab500 2023-10-18T10:43:11.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40dcb4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80a9ba0 con 0x7f40d806af00 2023-10-18T10:43:11.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40dcb4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d813dd70 con 0x7f40d80c9dc0 2023-10-18T10:43:11.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80ab500 0x7f40d80a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:11.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80ab500 0x7f40d80a6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54572/0 (socket says 172.21.15.125:54572) 2023-10-18T10:43:11.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 -- 172.21.15.125:0/823221246 learned_addr learned my addr 172.21.15.125:0/823221246 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:11.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d659c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40d806af00 0x7f40d813a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:11.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d6d9d640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40d80c9dc0 0x7f40d80ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:11.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 -- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40d80c9dc0 msgr2=0x7f40d80ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40d80c9dc0 0x7f40d80ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 -- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40d806af00 msgr2=0x7f40d813a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40d806af00 0x7f40d813a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 -- 172.21.15.125:0/823221246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc009560 con 0x7f40d80ab500 2023-10-18T10:43:11.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d659c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40d806af00 0x7f40d813a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:43:11.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d6d9d640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40d80c9dc0 0x7f40d80ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:43:11.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40d5d9b640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80ab500 0x7f40d80a6600 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009850 tx=0x7f40cc02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:11.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.742+0000 7f40c6ffd640 1 -- 172.21.15.125:0/823221246 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0035f0 con 0x7f40d80ab500 2023-10-18T10:43:11.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.743+0000 7f40c6ffd640 1 -- 172.21.15.125:0/823221246 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f40cc003840 con 0x7f40d80ab500 2023-10-18T10:43:11.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.743+0000 7f40c6ffd640 1 -- 172.21.15.125:0/823221246 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc003bf0 con 0x7f40d80ab500 2023-10-18T10:43:11.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.743+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d80a9e20 con 0x7f40d80ab500 2023-10-18T10:43:11.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.743+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d80aa3e0 con 0x7f40d80ab500 2023-10-18T10:43:11.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.744+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d80aa570 con 0x7f40d80ab500 2023-10-18T10:43:11.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.744+0000 7f40c6ffd640 1 -- 172.21.15.125:0/823221246 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f40cc003d50 con 0x7f40d80ab500 2023-10-18T10:43:11.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.744+0000 7f40c6ffd640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f40b806af60 0x7f40b806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:11.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.745+0000 7f40c6ffd640 1 -- 172.21.15.125:0/823221246 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f40cc0ac5f0 con 0x7f40d80ab500 2023-10-18T10:43:11.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.745+0000 7f40d659c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f40b806af60 0x7f40b806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:11.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.745+0000 7f40d659c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f40b806af60 0x7f40b806d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40d00012e0 tx=0x7f40d0009040 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:11.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.749+0000 7f40c6ffd640 1 -- 172.21.15.125:0/823221246 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40cc077570 con 0x7f40d80ab500 2023-10-18T10:43:11.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.977+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f40d80690e0 con 0x7f40d80ab500 2023-10-18T10:43:11.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.979+0000 7f40c6ffd640 1 -- 172.21.15.125:0/823221246 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+147 (secure 0 0 0) 0x7f40cc0430b0 con 0x7f40d80ab500 2023-10-18T10:43:11.981 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:43:11.982 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:43:11.982 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T10:43:11.982 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T10:43:11.982 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:43:11.983 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T10:43:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f40b806af60 msgr2=0x7f40b806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f40b806af60 0x7f40b806d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40d00012e0 tx=0x7f40d0009040 comp rx=0 tx=0).stop 2023-10-18T10:43:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80ab500 msgr2=0x7f40d80a6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80ab500 0x7f40d80a6600 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009850 tx=0x7f40cc02dae0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 shutdown_connections 2023-10-18T10:43:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f40b806af60 0x7f40b806d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40d80c9dc0 0x7f40d80ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f40d80ab500 0x7f40d80a6600 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 --2- 172.21.15.125:0/823221246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40d806af00 0x7f40d813a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.981+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 >> 172.21.15.125:0/823221246 conn(0x7f40d8069930 msgr2=0x7f40d80c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:11.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.982+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 shutdown_connections 2023-10-18T10:43:11.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:11.982+0000 7f40dcb4c640 1 -- 172.21.15.125:0/823221246 wait complete. 2023-10-18T10:43:11.994 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:43:12.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 -- 172.21.15.125:0/2002010365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a40c96b0 msgr2=0x7fa7a40c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/2002010365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a40c96b0 0x7fa7a40c9b60 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005fd0 tx=0x7fa78c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:43:12.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 -- 172.21.15.125:0/2002010365 shutdown_connections 2023-10-18T10:43:12.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/2002010365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a40c96b0 0x7fa7a40c9b60 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/2002010365 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a40ab500 0x7fa7a40ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/2002010365 >> v2:172.21.15.139:3300/0 conn(0x7fa7a406af00 0x7fa7a406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 -- 172.21.15.125:0/2002010365 >> 172.21.15.125:0/2002010365 conn(0x7fa7a4069930 msgr2=0x7fa7a4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 -- 172.21.15.125:0/2002010365 shutdown_connections 2023-10-18T10:43:12.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.094+0000 7fa7a8e75640 1 -- 172.21.15.125:0/2002010365 wait complete. 2023-10-18T10:43:12.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.095+0000 7fa7a8e75640 1 Processor -- start 2023-10-18T10:43:12.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.095+0000 7fa7a8e75640 1 -- start start 2023-10-18T10:43:12.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a8e75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a406af00 0x7fa7a4151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a8e75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a40ab500 0x7fa7a415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a8e75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a40c96b0 0x7fa7a415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a8e75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4130ec0 con 0x7fa7a406af00 2023-10-18T10:43:12.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a8e75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a412fe20 con 0x7fa7a40c96b0 2023-10-18T10:43:12.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a8e75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4130100 con 0x7fa7a40ab500 2023-10-18T10:43:12.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a406af00 0x7fa7a4151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a406af00 0x7fa7a4151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54580/0 (socket says 172.21.15.125:54580) 2023-10-18T10:43:12.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 -- 172.21.15.125:0/3479403818 learned_addr learned my addr 172.21.15.125:0/3479403818 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:12.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 -- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a40ab500 msgr2=0x7fa7a415b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a40ab500 0x7fa7a415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a2d9d640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a40c96b0 0x7fa7a415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 -- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a40c96b0 msgr2=0x7fa7a415bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a40c96b0 0x7fa7a415bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 -- 172.21.15.125:0/3479403818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c009560 con 0x7fa7a406af00 2023-10-18T10:43:12.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a259c640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a406af00 0x7fa7a4151fc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fa79400d950 tx=0x7fa79400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:12.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa79affd640 1 -- 172.21.15.125:0/3479403818 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794009c30 con 0x7fa7a406af00 2023-10-18T10:43:12.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa79affd640 1 -- 172.21.15.125:0/3479403818 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa794009e80 con 0x7fa7a406af00 2023-10-18T10:43:12.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa79affd640 1 -- 172.21.15.125:0/3479403818 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7940106c0 con 0x7fa7a406af00 2023-10-18T10:43:12.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a41303e0 con 0x7fa7a406af00 2023-10-18T10:43:12.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.096+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a41309a0 con 0x7fa7a406af00 2023-10-18T10:43:12.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.097+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a40abe50 con 0x7fa7a406af00 2023-10-18T10:43:12.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.098+0000 7fa79affd640 1 -- 172.21.15.125:0/3479403818 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa794010820 con 0x7fa7a406af00 2023-10-18T10:43:12.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.098+0000 7fa79affd640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa78406af60 0x7fa78406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.098+0000 7fa79affd640 1 -- 172.21.15.125:0/3479403818 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa794089ed0 con 0x7fa7a406af00 2023-10-18T10:43:12.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.098+0000 7fa79bfff640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa78406af60 0x7fa78406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.099+0000 7fa79bfff640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa78406af60 0x7fa78406d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa78c035000 tx=0x7fa78c0046c0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:12.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.102+0000 7fa79affd640 1 -- 172.21.15.125:0/3479403818 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa794054c20 con 0x7fa7a406af00 2023-10-18T10:43:12.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.303+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7a40690e0 con 0x7fa7a406af00 2023-10-18T10:43:12.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.303+0000 7fa79affd640 1 -- 172.21.15.125:0/3479403818 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fa794017090 con 0x7fa7a406af00 2023-10-18T10:43:12.305 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:12.306 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":47,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/4122761343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4122761343},{"type":"v1","addr":"172.21.15.139:6836","nonce":4122761343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6785,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1014994333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1014994333},{"type":"v1","addr":"172.21.15.139:6837","nonce":1014994333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3090232356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3090232356},{"type":"v1","addr":"172.21.15.125:6834","nonce":3090232356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:43:03.299305+0000","modified":"2023-10-18T10:43:05.301917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8613":{"gid":8613,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2199142702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2199142702},{"type":"v1","addr":"172.21.15.125:6835","nonce":2199142702}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T10:43:12.307 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 47 2023-10-18T10:43:12.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa78406af60 msgr2=0x7fa78406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa78406af60 0x7fa78406d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa78c035000 tx=0x7fa78c0046c0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a406af00 msgr2=0x7fa7a4151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a406af00 0x7fa7a4151fc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fa79400d950 tx=0x7fa79400de10 comp rx=0 tx=0).stop 2023-10-18T10:43:12.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 shutdown_connections 2023-10-18T10:43:12.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fa78406af60 0x7fa78406d420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a40c96b0 0x7fa7a415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a40ab500 0x7fa7a415b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 --2- 172.21.15.125:0/3479403818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7a406af00 0x7fa7a4151fc0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 >> 172.21.15.125:0/3479403818 conn(0x7fa7a4069930 msgr2=0x7fa7a40b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 shutdown_connections 2023-10-18T10:43:12.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.305+0000 7fa7a8e75640 1 -- 172.21.15.125:0/3479403818 wait complete. 2023-10-18T10:43:12.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:43:03.299305+0000', 'modified': '2023-10-18T10:43:05.301917+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8613}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8613': {'gid': 8613, 'name': 'c', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/2199142702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 2199142702}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 2199142702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:43:12.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:43:12.720 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T10:43:12.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.825+0000 7f1313f8e640 1 -- 172.21.15.125:0/3952555056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 msgr2=0x7f12f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.825+0000 7f1313f8e640 1 --2- 172.21.15.125:0/3952555056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 0x7f12f8003d90 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1308001650 tx=0x7f12f80057f0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.825+0000 7f1313f8e640 1 -- 172.21.15.125:0/3952555056 shutdown_connections 2023-10-18T10:43:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.825+0000 7f1313f8e640 1 --2- 172.21.15.125:0/3952555056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 0x7f12f8003d90 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.825+0000 7f1313f8e640 1 -- 172.21.15.125:0/3952555056 >> 172.21.15.125:0/3952555056 conn(0x7f130c069930 msgr2=0x7f130c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.825+0000 7f1313f8e640 1 -- 172.21.15.125:0/3952555056 shutdown_connections 2023-10-18T10:43:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.825+0000 7f1313f8e640 1 -- 172.21.15.125:0/3952555056 wait complete. 2023-10-18T10:43:12.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.826+0000 7f1313f8e640 1 Processor -- start 2023-10-18T10:43:12.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.826+0000 7f1313f8e640 1 -- start start 2023-10-18T10:43:12.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1313f8e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 0x7f130c158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1313f8e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f130c1592c0 0x7f130c0ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1313f8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130c0ab820 0x7f130c0aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1313f8e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0abcd0 con 0x7f12f8003970 2023-10-18T10:43:12.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1313f8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c130be0 con 0x7f130c0ab820 2023-10-18T10:43:12.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1313f8e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c130df0 con 0x7f130c1592c0 2023-10-18T10:43:12.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 0x7f130c158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 0x7f130c158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54608/0 (socket says 172.21.15.125:54608) 2023-10-18T10:43:12.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 -- 172.21.15.125:0/533451620 learned_addr learned my addr 172.21.15.125:0/533451620 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:12.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 -- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f130c1592c0 msgr2=0x7f130c0ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311529640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f130c1592c0 0x7f130c0ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f131252b640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130c0ab820 0x7f130c0aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f130c1592c0 0x7f130c0ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 -- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130c0ab820 msgr2=0x7f130c0aa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130c0ab820 0x7f130c0aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 -- 172.21.15.125:0/533451620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c1310d0 con 0x7f12f8003970 2023-10-18T10:43:12.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.827+0000 7f1311d2a640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 0x7f130c158d80 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f130802c090 tx=0x7f1308003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:12.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.828+0000 7f13027fc640 1 -- 172.21.15.125:0/533451620 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130803b070 con 0x7f12f8003970 2023-10-18T10:43:12.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.828+0000 7f13027fc640 1 -- 172.21.15.125:0/533451620 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1308006260 con 0x7f12f8003970 2023-10-18T10:43:12.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.828+0000 7f1313f8e640 1 -- 172.21.15.125:0/533451620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c1313b0 con 0x7f12f8003970 2023-10-18T10:43:12.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.828+0000 7f13027fc640 1 -- 172.21.15.125:0/533451620 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130802d740 con 0x7f12f8003970 2023-10-18T10:43:12.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.828+0000 7f1313f8e640 1 -- 172.21.15.125:0/533451620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c131970 con 0x7f12f8003970 2023-10-18T10:43:12.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.829+0000 7f13027fc640 1 -- 172.21.15.125:0/533451620 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f130802d8a0 con 0x7f12f8003970 2023-10-18T10:43:12.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.829+0000 7f13027fc640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f12f006af90 0x7f12f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.830+0000 7f13027fc640 1 -- 172.21.15.125:0/533451620 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f13080ab7b0 con 0x7f12f8003970 2023-10-18T10:43:12.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.830+0000 7f1311529640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f12f006af90 0x7f12f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.830+0000 7f1311529640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f12f006af90 0x7f12f006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0017a0 tx=0x7f12fc001df0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:12.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1316bf7740 1 Processor -- start 2023-10-18T10:43:12.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1316bf7740 1 -- start start 2023-10-18T10:43:12.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f709298ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 0x55f7092c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f709299020 0x55f7092c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1316bf7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7091aa730 con 0x55f7092c5b40 2023-10-18T10:43:12.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1316bf7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f70920b780 con 0x55f709299020 2023-10-18T10:43:12.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1316bf7740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f709208e70 con 0x55f7092c5f20 2023-10-18T10:43:12.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.831+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f709298ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f709298ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54610/0 (socket says 172.21.15.125:54610) 2023-10-18T10:43:12.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311d2a640 1 -- 172.21.15.125:0/2423603017 learned_addr learned my addr 172.21.15.125:0/2423603017 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:12.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311529640 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 0x55f7092c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f131252b640 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f709299020 0x55f7092c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311d2a640 1 -- 172.21.15.125:0/2423603017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 msgr2=0x55f7092c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:43:12.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311d2a640 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 0x55f7092c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311d2a640 1 -- 172.21.15.125:0/2423603017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f709299020 msgr2=0x55f7092c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:43:12.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311d2a640 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f709299020 0x55f7092c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311d2a640 1 -- 172.21.15.125:0/2423603017 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7092093d0 con 0x55f7092c5b40 2023-10-18T10:43:12.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f1311d2a640 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f709298ae0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1308081ef0 tx=0x7f1308079b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd0f34a8ae614eb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:12.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f12ef7fe640 1 -- 172.21.15.125:0/2423603017 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130807a120 con 0x55f7092c5b40 2023-10-18T10:43:12.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f12ef7fe640 1 -- 172.21.15.125:0/2423603017 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f130807e270 con 0x55f7092c5b40 2023-10-18T10:43:12.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.832+0000 7f12ef7fe640 1 -- 172.21.15.125:0/2423603017 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308082420 con 0x55f7092c5b40 2023-10-18T10:43:12.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 -- 172.21.15.125:0/2423603017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 msgr2=0x55f709298ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f709298ae0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1308081ef0 tx=0x7f1308079b40 comp rx=0 tx=0).stop 2023-10-18T10:43:12.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 -- 172.21.15.125:0/2423603017 shutdown_connections 2023-10-18T10:43:12.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f709299020 0x55f7092c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 0x55f7092c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 --2- 172.21.15.125:0/2423603017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f709298ae0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 -- 172.21.15.125:0/2423603017 >> 172.21.15.125:0/2423603017 conn(0x55f70929ad50 msgr2=0x55f7092e38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 -- 172.21.15.125:0/2423603017 shutdown_connections 2023-10-18T10:43:12.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 -- 172.21.15.125:0/2423603017 wait complete. 2023-10-18T10:43:12.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 -- start start 2023-10-18T10:43:12.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.833+0000 7f1316bf7740 1 Processor -- start 2023-10-18T10:43:12.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f7092b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 0x55f7092b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092e6de0 0x55f7092e7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1316bf7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f709204760 con 0x55f7092c5b40 2023-10-18T10:43:12.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1316bf7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7091bf8f0 con 0x55f7092e6de0 2023-10-18T10:43:12.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1316bf7740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7091bf590 con 0x55f7092c5f20 2023-10-18T10:43:12.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1316bf7740 10 client.0 _ll_register_callbacks cb 0x55f7091fbee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:43:12.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f7092b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f7092b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54624/0 (socket says 172.21.15.125:54624) 2023-10-18T10:43:12.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.834+0000 7f1311d2a640 1 -- 172.21.15.125:0/3099794801 learned_addr learned my addr 172.21.15.125:0/3099794801 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:12.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1311529640 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 0x55f7092b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f131252b640 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092e6de0 0x55f7092e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1311d2a640 1 -- 172.21.15.125:0/3099794801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 msgr2=0x55f7092b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:43:12.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1311d2a640 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 0x55f7092b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1311d2a640 1 -- 172.21.15.125:0/3099794801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092e6de0 msgr2=0x55f7092e7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:43:12.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1311d2a640 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092e6de0 0x55f7092e7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1311d2a640 1 -- 172.21.15.125:0/3099794801 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c131970 con 0x55f7092c5b40 2023-10-18T10:43:12.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1311d2a640 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f7092b2630 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1308076620 tx=0x7f130807a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b8abeed160cf4c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:12.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f12ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:43:12.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f12ecff9640 1 -- 172.21.15.125:0/3099794801 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130807a120 con 0x55f7092c5b40 2023-10-18T10:43:12.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f12ecff9640 1 -- 172.21.15.125:0/3099794801 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1308082420 con 0x55f7092c5b40 2023-10-18T10:43:12.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1316bf7740 1 -- 172.21.15.125:0/3099794801 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7091fd740 con 0x55f7092c5b40 2023-10-18T10:43:12.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f12ecff9640 1 -- 172.21.15.125:0/3099794801 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13080a9af0 con 0x55f7092c5b40 2023-10-18T10:43:12.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1316bf7740 10 client.8778 fetch_fsmap learned FSMap version 47 2023-10-18T10:43:12.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1316bf7740 1 -- 172.21.15.125:0/3099794801 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55f709202de0 con 0x55f7092c5b40 2023-10-18T10:43:12.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f12ecff9640 1 -- 172.21.15.125:0/3099794801 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55f7091fd740 con 0x55f7092c5b40 2023-10-18T10:43:12.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f12ecff9640 1 -- 172.21.15.125:0/3099794801 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1308034030 con 0x55f7092c5b40 2023-10-18T10:43:12.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1316bf7740 10 client.8778 fetch_fsmap finished waiting for FSMap version 47 2023-10-18T10:43:12.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.835+0000 7f1316bf7740 10 client.8778 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T10:43:12.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f1316bf7740 20 client.8778 populate_metadata read hostname 'smithi125' 2023-10-18T10:43:12.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] conn(0x55f7092b3c50 0x55f7092b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f1316bf7740 4 client.8778 mds_command: new command op to 8613 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:43:12.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f1316bf7740 1 -- 172.21.15.125:0/3099794801 --> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7091fd460 con 0x55f7092b3c50 2023-10-18T10:43:12.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f1311529640 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] conn(0x55f7092b3c50 0x55f7092b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f12ecff9640 10 client.8778 ms_handle_connect on v2:172.21.15.125:6833/2199142702 2023-10-18T10:43:12.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f1311529640 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] conn(0x55f7092b3c50 0x55f7092b4030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b203fd22ffb2d8e8 server_cookie=fd9879c354661346 in_seq=0 out_seq=0 2023-10-18T10:43:12.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f12ecff9640 1 -- 172.21.15.125:0/3099794801 <== mds.0 v2:172.21.15.125:6833/2199142702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f7091fd460 con 0x55f7092b3c50 2023-10-18T10:43:12.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f12ecff9640 10 client.8778 handle_command_reply: tid=0 2023-10-18T10:43:12.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.836+0000 7f1316bf7740 1 client.8778 shutdown 2023-10-18T10:43:12.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 -- 172.21.15.125:0/3099794801 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 msgr2=0x55f7092b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f7092b2630 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1308076620 tx=0x7f130807a8a0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 -- 172.21.15.125:0/3099794801 shutdown_connections 2023-10-18T10:43:12.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] conn(0x55f7092b3c50 0x55f7092b4030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092e6de0 0x55f7092e7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f12ecff9640 0 client.8778 ms_handle_reset on v2:172.21.15.125:6833/2199142702 2023-10-18T10:43:12.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092c5f20 0x55f7092b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3099794801 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f7092b2630 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 -- 172.21.15.125:0/3099794801 >> 172.21.15.125:0/3099794801 conn(0x55f709299020 msgr2=0x55f7092e4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 -- 172.21.15.125:0/3099794801 shutdown_connections 2023-10-18T10:43:12.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 1 -- 172.21.15.125:0/3099794801 wait complete. 2023-10-18T10:43:12.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.837+0000 7f1316bf7740 20 client.8778 trim_cache size 0 max 16384 2023-10-18T10:43:12.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.852+0000 7f1316bf7740 1 Processor -- start 2023-10-18T10:43:12.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.852+0000 7f1316bf7740 1 -- start start 2023-10-18T10:43:12.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.852+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70929c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.852+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f7092c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1316bf7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f709185270 con 0x55f7092c5b40 2023-10-18T10:43:12.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1316bf7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f709305cb0 con 0x55f7092c5f20 2023-10-18T10:43:12.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1316bf7740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7092f3170 con 0x55f7092e6de0 2023-10-18T10:43:12.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70929c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70929c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54632/0 (socket says 172.21.15.125:54632) 2023-10-18T10:43:12.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 -- 172.21.15.125:0/174543055 learned_addr learned my addr 172.21.15.125:0/174543055 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:12.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f131252b640 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311529640 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f7092c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 -- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 msgr2=0x55f7092c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:43:12.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 -- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 msgr2=0x55f7092c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:43:12.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f7092c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 -- 172.21.15.125:0/174543055 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7091a1d00 con 0x55f7092c5b40 2023-10-18T10:43:12.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f131252b640 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:43:12.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311529640 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f7092c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:43:12.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f1311d2a640 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70929c540 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f130807a9d0 tx=0x7f1308076460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7527f6815487e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:12.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f12effff640 1 -- 172.21.15.125:0/174543055 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130807a120 con 0x55f7092c5b40 2023-10-18T10:43:12.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f12effff640 1 -- 172.21.15.125:0/174543055 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1308082420 con 0x55f7092c5b40 2023-10-18T10:43:12.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.853+0000 7f12effff640 1 -- 172.21.15.125:0/174543055 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308080120 con 0x55f7092c5b40 2023-10-18T10:43:12.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 -- 172.21.15.125:0/174543055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 msgr2=0x55f70929c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70929c540 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f130807a9d0 tx=0x7f1308076460 comp rx=0 tx=0).stop 2023-10-18T10:43:12.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 -- 172.21.15.125:0/174543055 shutdown_connections 2023-10-18T10:43:12.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f7092c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 --2- 172.21.15.125:0/174543055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70929c540 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 -- 172.21.15.125:0/174543055 >> 172.21.15.125:0/174543055 conn(0x55f709299020 msgr2=0x55f709297ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 -- 172.21.15.125:0/174543055 shutdown_connections 2023-10-18T10:43:12.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 -- 172.21.15.125:0/174543055 wait complete. 2023-10-18T10:43:12.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 -- start start 2023-10-18T10:43:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.854+0000 7f1316bf7740 1 Processor -- start 2023-10-18T10:43:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70934f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f70929d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1316bf7740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092db270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1316bf7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f70929ed90 con 0x55f7092c5b40 2023-10-18T10:43:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1316bf7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7092dbe10 con 0x55f7092c5f20 2023-10-18T10:43:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1316bf7740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7092b4440 con 0x55f7092e6de0 2023-10-18T10:43:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1316bf7740 10 client.0 _ll_register_callbacks cb 0x7f1308040eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:43:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70934f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311d2a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70934f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54642/0 (socket says 172.21.15.125:54642) 2023-10-18T10:43:12.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311d2a640 1 -- 172.21.15.125:0/3766959444 learned_addr learned my addr 172.21.15.125:0/3766959444 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:12.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311529640 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f70929d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f131252b640 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092db270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311d2a640 1 -- 172.21.15.125:0/3766959444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 msgr2=0x55f7092db270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:43:12.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311d2a640 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092db270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311d2a640 1 -- 172.21.15.125:0/3766959444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 msgr2=0x55f70929d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:43:12.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311d2a640 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f70929d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.855+0000 7f1311d2a640 1 -- 172.21.15.125:0/3766959444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7092093d0 con 0x55f7092c5b40 2023-10-18T10:43:12.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1311d2a640 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70934f2c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1308079e20 tx=0x7f13080a9950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47331608390bc5f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:12.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f12ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:43:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f12ecff9640 1 -- 172.21.15.125:0/3766959444 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13080800d0 con 0x55f7092c5b40 2023-10-18T10:43:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f12ecff9640 1 -- 172.21.15.125:0/3766959444 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1308080230 con 0x55f7092c5b40 2023-10-18T10:43:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 1 -- 172.21.15.125:0/3766959444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f130807e270 con 0x55f7092c5b40 2023-10-18T10:43:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f12ecff9640 1 -- 172.21.15.125:0/3766959444 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f130802db50 con 0x55f7092c5b40 2023-10-18T10:43:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 10 client.8784 fetch_fsmap learned FSMap version 47 2023-10-18T10:43:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 1 -- 172.21.15.125:0/3766959444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55f7091c74f0 con 0x55f7092c5b40 2023-10-18T10:43:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f12ecff9640 1 -- 172.21.15.125:0/3766959444 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f130807e270 con 0x55f7092c5b40 2023-10-18T10:43:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f12ecff9640 1 -- 172.21.15.125:0/3766959444 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f130802dcb0 con 0x55f7092c5b40 2023-10-18T10:43:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 10 client.8784 fetch_fsmap finished waiting for FSMap version 47 2023-10-18T10:43:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 10 client.8784 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T10:43:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 20 client.8784 populate_metadata read hostname 'smithi125' 2023-10-18T10:43:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] conn(0x55f7092e5eb0 0x55f7092f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 4 client.8784 mds_command: new command op to 8613 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:43:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1316bf7740 1 -- 172.21.15.125:0/3766959444 --> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f708f066c0 con 0x55f7092e5eb0 2023-10-18T10:43:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1311529640 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] conn(0x55f7092e5eb0 0x55f7092f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f12ecff9640 10 client.8784 ms_handle_connect on v2:172.21.15.125:6833/2199142702 2023-10-18T10:43:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.856+0000 7f1311529640 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] conn(0x55f7092e5eb0 0x55f7092f1eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0db39fece0f0350 server_cookie=165f89295ace3b19 in_seq=0 out_seq=0 2023-10-18T10:43:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f12ecff9640 1 -- 172.21.15.125:0/3766959444 <== mds.0 v2:172.21.15.125:6833/2199142702 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f708f066c0 con 0x55f7092e5eb0 2023-10-18T10:43:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f12ecff9640 10 client.8784 handle_command_reply: tid=0 2023-10-18T10:43:12.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 client.8784 shutdown 2023-10-18T10:43:12.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 -- 172.21.15.125:0/3766959444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 msgr2=0x55f70934f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70934f2c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1308079e20 tx=0x7f13080a9950 comp rx=0 tx=0).stop 2023-10-18T10:43:12.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 -- 172.21.15.125:0/3766959444 shutdown_connections 2023-10-18T10:43:12.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.125:6833/2199142702,v1:172.21.15.125:6835/2199142702] conn(0x55f7092e5eb0 0x55f7092f1eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f7092c5f20 0x55f70929d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55f7092e6de0 0x55f7092db270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 --2- 172.21.15.125:0/3766959444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f7092c5b40 0x55f70934f2c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f12ecff9640 0 client.8784 ms_handle_reset on v2:172.21.15.125:6833/2199142702 2023-10-18T10:43:12.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 -- 172.21.15.125:0/3766959444 >> 172.21.15.125:0/3766959444 conn(0x55f709299020 msgr2=0x55f7092b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 -- 172.21.15.125:0/3766959444 shutdown_connections 2023-10-18T10:43:12.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 1 -- 172.21.15.125:0/3766959444 wait complete. 2023-10-18T10:43:12.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1316bf7740 20 client.8784 trim_cache size 0 max 16384 2023-10-18T10:43:12.885 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:43:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:43:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:43:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 8613, 2023-10-18T10:43:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:43:12.886 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:43:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:43:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.8907421269999993, 2023-10-18T10:43:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 47, 2023-10-18T10:43:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 128, 2023-10-18T10:43:12.887 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 128, 2023-10-18T10:43:12.888 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 14.668774033 2023-10-18T10:43:12.888 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:43:12.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 -- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f12f006af90 msgr2=0x7f12f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f12f006af90 0x7f12f006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0017a0 tx=0x7f12fc001df0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 -- 172.21.15.125:0/533451620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 msgr2=0x7f130c158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 0x7f130c158d80 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f130802c090 tx=0x7f1308003d30 comp rx=0 tx=0).stop 2023-10-18T10:43:12.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 -- 172.21.15.125:0/533451620 shutdown_connections 2023-10-18T10:43:12.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f12f006af90 0x7f12f006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130c0ab820 0x7f130c0aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f130c1592c0 0x7f130c0ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.857+0000 7f1313f8e640 1 --2- 172.21.15.125:0/533451620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12f8003970 0x7f130c158d80 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.858+0000 7f1313f8e640 1 -- 172.21.15.125:0/533451620 >> 172.21.15.125:0/533451620 conn(0x7f130c069930 msgr2=0x7f130c0ac910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.858+0000 7f1313f8e640 1 -- 172.21.15.125:0/533451620 shutdown_connections 2023-10-18T10:43:12.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.858+0000 7f1313f8e640 1 -- 172.21.15.125:0/533451620 wait complete. 2023-10-18T10:43:12.892 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:43:12.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.993+0000 7fb9f42a9640 1 -- 172.21.15.125:0/491725256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec0cf340 msgr2=0x7fb9ec0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:12.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.993+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/491725256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec0cf340 0x7fb9ec0d1910 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc005fd0 tx=0x7fb9dc02c240 comp rx=0 tx=0).stop 2023-10-18T10:43:12.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.993+0000 7fb9f42a9640 1 -- 172.21.15.125:0/491725256 shutdown_connections 2023-10-18T10:43:12.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.993+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/491725256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9ec15b2a0 0x7fb9ec0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.993+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/491725256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec0cf340 0x7fb9ec0d1910 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:12.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.993+0000 7fb9f42a9640 1 -- 172.21.15.125:0/491725256 >> 172.21.15.125:0/491725256 conn(0x7fb9ec0d0d60 msgr2=0x7fb9ec0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:12.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.993+0000 7fb9f42a9640 1 -- 172.21.15.125:0/491725256 shutdown_connections 2023-10-18T10:43:12.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.993+0000 7fb9f42a9640 1 -- 172.21.15.125:0/491725256 wait complete. 2023-10-18T10:43:12.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.994+0000 7fb9f42a9640 1 Processor -- start 2023-10-18T10:43:12.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.994+0000 7fb9f42a9640 1 -- start start 2023-10-18T10:43:13.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f42a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ec0ceb40 0x7fb9ec1f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:13.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f42a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9ec0cf340 0x7fb9ec0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:13.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f42a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec15b2a0 0x7fb9ec15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:13.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f42a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec157010 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f42a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec15c120 con 0x7fb9ec0ceb40 2023-10-18T10:43:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f42a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec15c400 con 0x7fb9ec0cf340 2023-10-18T10:43:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec15b2a0 0x7fb9ec15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec15b2a0 0x7fb9ec15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54656/0 (socket says 172.21.15.125:54656) 2023-10-18T10:43:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 -- 172.21.15.125:0/2104758498 learned_addr learned my addr 172.21.15.125:0/2104758498 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:13.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 -- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9ec0cf340 msgr2=0x7fb9ec0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f1844640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9ec0cf340 0x7fb9ec0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2045640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ec0ceb40 0x7fb9ec1f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9ec0cf340 0x7fb9ec0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 -- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ec0ceb40 msgr2=0x7fb9ec1f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:13.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ec0ceb40 0x7fb9ec1f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 -- 172.21.15.125:0/2104758498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc009900 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f1844640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9ec0cf340 0x7fb9ec0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:43:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2045640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ec0ceb40 0x7fb9ec1f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:43:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.995+0000 7fb9f2846640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec15b2a0 0x7fb9ec15b6d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4001eb0 tx=0x7fb9e400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:13.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.996+0000 7fb9eaffd640 1 -- 172.21.15.125:0/2104758498 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40052d0 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.996+0000 7fb9eaffd640 1 -- 172.21.15.125:0/2104758498 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb9e40035f0 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.996+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec156ca0 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.996+0000 7fb9eaffd640 1 -- 172.21.15.125:0/2104758498 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40039e0 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.996+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec15dde0 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.997+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec156b00 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.997+0000 7fb9eaffd640 1 -- 172.21.15.125:0/2104758498 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb9e4003b40 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.998+0000 7fb9eaffd640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fb9d806af60 0x7fb9d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:13.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.998+0000 7fb9eaffd640 1 -- 172.21.15.125:0/2104758498 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb9e40890f0 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.998+0000 7fb9f2045640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fb9d806af60 0x7fb9d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:13.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:12.998+0000 7fb9f2045640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fb9d806af60 0x7fb9d806d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc036000 tx=0x7fb9dc004ab0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:13.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.002+0000 7fb9eaffd640 1 -- 172.21.15.125:0/2104758498 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9ec156b00 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.201+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb9ec1571c0 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.201+0000 7fb9eaffd640 1 -- 172.21.15.125:0/2104758498 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb9ec1571c0 con 0x7fb9ec15b2a0 2023-10-18T10:43:13.204 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:43:13.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fb9d806af60 msgr2=0x7fb9d806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:13.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fb9d806af60 0x7fb9d806d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc036000 tx=0x7fb9dc004ab0 comp rx=0 tx=0).stop 2023-10-18T10:43:13.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec15b2a0 msgr2=0x7fb9ec15b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:13.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec15b2a0 0x7fb9ec15b6d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4001eb0 tx=0x7fb9e400c890 comp rx=0 tx=0).stop 2023-10-18T10:43:13.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 shutdown_connections 2023-10-18T10:43:13.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fb9d806af60 0x7fb9d806d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:13.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9ec15b2a0 0x7fb9ec15b6d0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:13.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9ec0cf340 0x7fb9ec0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:13.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 --2- 172.21.15.125:0/2104758498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9ec0ceb40 0x7fb9ec1f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:13.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 >> 172.21.15.125:0/2104758498 conn(0x7fb9ec0d0d60 msgr2=0x7fb9ec15a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:13.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 shutdown_connections 2023-10-18T10:43:13.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:13.203+0000 7fb9f42a9640 1 -- 172.21.15.125:0/2104758498 wait complete. 2023-10-18T10:43:13.215 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:43:13.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:43:13.216 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:43:13.216 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:43:13.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:43:13.217 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:43:13.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:43:13.218 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:43:13.218 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:43:13.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:13.219 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:43:13.235 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:43:13.235 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:43:13.235 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:43:13.235 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:43:13.235 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:43:13.236 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:43:13.236 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:43:13.236 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:43:13.236 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:43:13.236 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:43:13.237 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:43:13.237 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:43:13.237 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:43:13.237 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:43:13.237 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:43:13.238 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:43:13.238 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:43:13.238 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 41631sec preferred_lft 41631sec 2023-10-18T10:43:13.238 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:43:13.238 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:43:13.239 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:43:13.239 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:43:13.239 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:43:13.239 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:43:13.239 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T10:43:13.240 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:43:13.240 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:43:13.240 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:43:13.240 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T10:43:13.240 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:43:13.241 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:43:13.241 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:43:13.241 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T10:43:13.241 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:43:13.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:13.242 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:43:13.300 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:43:13.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:13.302 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:43:13.356 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:43:13.356 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:43:13.357 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:43:13.357 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:43:13.357 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:43:13.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:13.358 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:43:13.358 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:43:13.359 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:43:13.359 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:43:13.360 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:43:13.360 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:43:13.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:13.452 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:43:13.467 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:43:13.467 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:43:13.467 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:43:13.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:13.468 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:43:13.534 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:43:13.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:43:13.535 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:43:13.589 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:43:13.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:43:13.590 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:43:13.644 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:43:13.711 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:43:13.767 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:43:13.767 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:43:13.767 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:43:13.768 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:43:13.768 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:43:13.768 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:43:13.768 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:43:13.768 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:43:13.769 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:43:13.769 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:43:13.769 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:43:13.769 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:43:13.769 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:43:13.770 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:43:13.770 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:43:13.770 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:43:13.770 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:43:13.770 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:43:13.771 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:43:13.771 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:43:13.771 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:43:13.771 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:43:13.771 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:43:13.772 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:43:13.772 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:43:13.772 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:43:13.772 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:43:13.772 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:43:13.773 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:43:13.773 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:43:13.773 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:43:13.773 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:43:13.773 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:43:13.774 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:43:13.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:13.774 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:43:13.822 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:43:13.822 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:43:13.864 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:43:13.891 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:43:13.946 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:43:13.946 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:43:13.946 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:43:13.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:43:13.947 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:43:13.947 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:43:13.947 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:43:13.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:43:13.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:43:13.948 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:43:13.948 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:43:13.948 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:43:13.948 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:43:13.949 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:43:13.949 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:43:13.949 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:43:13.949 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:43:13.949 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:43:13.949 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:43:13.950 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:43:13.950 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:43:13.950 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:43:13.950 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:43:13.950 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:43:13.951 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:43:13.951 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:43:13.951 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:43:13.951 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:43:13.951 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:43:13.952 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:43:13.952 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:43:13.952 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:43:13.952 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:43:13.952 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:43:13.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:13.953 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:43:13.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:43:13.947+0000 7f5958579480 -1 init, newargv = 0x56072d116c70 newargc=15 2023-10-18T10:43:13.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[129036]: starting ceph client 2023-10-18T10:43:13.955 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[129036]: starting fuse 2023-10-18T10:43:13.968 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:43:13.968 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:43:13.969 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:43:13.969 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:43:13.970 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:43:13.970 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:43:13.971 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:43:13.971 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:43:13.971 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:43:13.972 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:43:13.972 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:43:13.973 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:43:13.973 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:43:13.973 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:43:13.974 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:43:13.974 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:43:13.974 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:43:13.975 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:43:13.975 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:43:13.976 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:43:13.976 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:43:13.976 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:43:13.977 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:43:13.977 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:43:13.977 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:43:13.978 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:43:13.978 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:43:13.979 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:43:13.979 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:43:14.087 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.129036.asok 2023-10-18T10:43:14.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:43:14.097 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.129036.asok 2023-10-18T10:43:14.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:14.098 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.129036.asok status 2023-10-18T10:43:14.232 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:43:14.233 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:43:14.233 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:43:14.233 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:43:14.233 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:43:14.233 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:43:14.234 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:43:14.234 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "129036", 2023-10-18T10:43:14.234 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:43:14.234 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:43:14.235 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:43:14.235 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:43:14.235 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 8799, 2023-10-18T10:43:14.235 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:43:14.236 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:43:14.236 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:43:14.236 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 8799 2023-10-18T10:43:14.236 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:43:14.236 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:43:14.237 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:43:14.237 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:43:14.237 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 846276804 2023-10-18T10:43:14.237 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:43:14.237 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:43:14.238 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:43:14.238 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:43:14.238 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:43:14.238 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 846276804 2023-10-18T10:43:14.239 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:43:14.239 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.8799 192.168.0.1:0/846276804", 2023-10-18T10:43:14.239 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/846276804", 2023-10-18T10:43:14.239 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:43:14.239 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 47, 2023-10-18T10:43:14.240 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 128, 2023-10-18T10:43:14.240 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:43:14.240 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:43:14.240 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:43:14.240 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:43:14.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:14.242 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:43:14.243 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:43:14.258 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:43:14.258 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:43:14.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:14.259 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:43:14.325 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T10:43:14.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.468+0000 7f08cd325640 1 -- 172.21.15.125:0/2841922528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c80cf340 msgr2=0x7f08c80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:14.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.468+0000 7f08cd325640 1 --2- 172.21.15.125:0/2841922528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c80cf340 0x7f08c80d1910 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f08b8005fd0 tx=0x7f08b802c050 comp rx=0 tx=0).stop 2023-10-18T10:43:14.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.468+0000 7f08cd325640 1 -- 172.21.15.125:0/2841922528 shutdown_connections 2023-10-18T10:43:14.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.468+0000 7f08cd325640 1 --2- 172.21.15.125:0/2841922528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08c815b2a0 0x7f08c80cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.468+0000 7f08cd325640 1 --2- 172.21.15.125:0/2841922528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c80cf340 0x7f08c80d1910 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.468+0000 7f08cd325640 1 -- 172.21.15.125:0/2841922528 >> 172.21.15.125:0/2841922528 conn(0x7f08c80d0d60 msgr2=0x7f08c80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:14.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.468+0000 7f08cd325640 1 -- 172.21.15.125:0/2841922528 shutdown_connections 2023-10-18T10:43:14.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.468+0000 7f08cd325640 1 -- 172.21.15.125:0/2841922528 wait complete. 2023-10-18T10:43:14.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.469+0000 7f08cd325640 1 Processor -- start 2023-10-18T10:43:14.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.469+0000 7f08cd325640 1 -- start start 2023-10-18T10:43:14.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08cd325640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08c80ceb40 0x7f08c80b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:14.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08cd325640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08c80cf340 0x7f08c815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:14.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08cd325640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c815b2a0 0x7f08c815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:14.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08cd325640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c81f46d0 con 0x7f08c815b2a0 2023-10-18T10:43:14.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08cd325640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c80b38e0 con 0x7f08c80cf340 2023-10-18T10:43:14.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08cd325640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c815c0e0 con 0x7f08c80ceb40 2023-10-18T10:43:14.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c815b2a0 0x7f08c815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:14.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c815b2a0 0x7f08c815d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54670/0 (socket says 172.21.15.125:54670) 2023-10-18T10:43:14.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08c77fe640 1 -- 172.21.15.125:0/2434228732 learned_addr learned my addr 172.21.15.125:0/2434228732 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:14.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08c67fc640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08c80cf340 0x7f08c815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:14.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08c6ffd640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08c80ceb40 0x7f08c80b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:14.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.470+0000 7f08c77fe640 1 -- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08c80ceb40 msgr2=0x7f08c80b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:14.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08c77fe640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08c80ceb40 0x7f08c80b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08c77fe640 1 -- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08c80cf340 msgr2=0x7f08c815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:14.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08c77fe640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08c80cf340 0x7f08c815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08c77fe640 1 -- 172.21.15.125:0/2434228732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009840 con 0x7f08c815b2a0 2023-10-18T10:43:14.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08c67fc640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08c80cf340 0x7f08c815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:43:14.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08c77fe640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c815b2a0 0x7f08c815d430 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f08c000c430 tx=0x7f08c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:14.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08af7fe640 1 -- 172.21.15.125:0/2434228732 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0011790 con 0x7f08c815b2a0 2023-10-18T10:43:14.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08af7fe640 1 -- 172.21.15.125:0/2434228732 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f08c00119e0 con 0x7f08c815b2a0 2023-10-18T10:43:14.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c80b6010 con 0x7f08c815b2a0 2023-10-18T10:43:14.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08af7fe640 1 -- 172.21.15.125:0/2434228732 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0011e10 con 0x7f08c815b2a0 2023-10-18T10:43:14.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.471+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c80b68e0 con 0x7f08c815b2a0 2023-10-18T10:43:14.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.472+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c8158410 con 0x7f08c815b2a0 2023-10-18T10:43:14.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.472+0000 7f08af7fe640 1 -- 172.21.15.125:0/2434228732 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f08c001c460 con 0x7f08c815b2a0 2023-10-18T10:43:14.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.473+0000 7f08af7fe640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f08b406af90 0x7f08b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:14.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.473+0000 7f08af7fe640 1 -- 172.21.15.125:0/2434228732 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f08c0089950 con 0x7f08c815b2a0 2023-10-18T10:43:14.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.473+0000 7f08c6ffd640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f08b406af90 0x7f08b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:14.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.474+0000 7f08c6ffd640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f08b406af90 0x7f08b406d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08b8035000 tx=0x7f08b8004ab0 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:14.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.477+0000 7f08af7fe640 1 -- 172.21.15.125:0/2434228732 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08c8158410 con 0x7f08c815b2a0 2023-10-18T10:43:14.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.665+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 --> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f08c80cc170 con 0x7f08b406af90 2023-10-18T10:43:14.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.667+0000 7f08af7fe640 1 -- 172.21.15.125:0/2434228732 <== mgr.7360 v2:172.21.15.139:6832/1511464628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f08b8003970 con 0x7f08b406af90 2023-10-18T10:43:14.669 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:43:14.669 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:43:14.669 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T10:43:14.670 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:43:14.670 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:43:14.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f08b406af90 msgr2=0x7f08b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:14.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f08b406af90 0x7f08b406d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08b8035000 tx=0x7f08b8004ab0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c815b2a0 msgr2=0x7f08c815d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:14.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c815b2a0 0x7f08c815d430 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f08c000c430 tx=0x7f08c000c8f0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 shutdown_connections 2023-10-18T10:43:14.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f08b406af90 0x7f08b406d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08c80cf340 0x7f08c815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08c815b2a0 0x7f08c815d430 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.668+0000 7f08cd325640 1 --2- 172.21.15.125:0/2434228732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08c80ceb40 0x7f08c80b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.669+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 >> 172.21.15.125:0/2434228732 conn(0x7f08c80d0d60 msgr2=0x7f08c80cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:14.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.669+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 shutdown_connections 2023-10-18T10:43:14.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.669+0000 7f08cd325640 1 -- 172.21.15.125:0/2434228732 wait complete. 2023-10-18T10:43:14.681 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T10:43:14.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.782+0000 7fc170f8e640 1 -- 172.21.15.125:0/2474994728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 msgr2=0x7fc16c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:14.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.782+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2474994728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 0x7fc16c0b0650 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fc16c08a120 tx=0x7fc16402c080 comp rx=0 tx=0).stop 2023-10-18T10:43:14.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.782+0000 7fc170f8e640 1 -- 172.21.15.125:0/2474994728 shutdown_connections 2023-10-18T10:43:14.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.782+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2474994728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 0x7fc16c0b0650 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.782+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2474994728 >> v2:172.21.15.139:3300/0 conn(0x7fc16c15b2a0 0x7fc16c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.782+0000 7fc170f8e640 1 -- 172.21.15.125:0/2474994728 >> 172.21.15.125:0/2474994728 conn(0x7fc16c0d0d60 msgr2=0x7fc16c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:14.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.782+0000 7fc170f8e640 1 -- 172.21.15.125:0/2474994728 shutdown_connections 2023-10-18T10:43:14.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.782+0000 7fc170f8e640 1 -- 172.21.15.125:0/2474994728 wait complete. 2023-10-18T10:43:14.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.783+0000 7fc170f8e640 1 Processor -- start 2023-10-18T10:43:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc170f8e640 1 -- start start 2023-10-18T10:43:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc170f8e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 0x7fc16c1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc170f8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc16c0cf340 0x7fc16c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc170f8e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc16c15b2a0 0x7fc16c159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:14.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc170f8e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0b4e50 con 0x7fc16c0ceb40 2023-10-18T10:43:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc170f8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0b3e20 con 0x7fc16c0cf340 2023-10-18T10:43:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc170f8e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0b4090 con 0x7fc16c15b2a0 2023-10-18T10:43:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc16a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 0x7fc16c1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc16a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 0x7fc16c1524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54692/0 (socket says 172.21.15.125:54692) 2023-10-18T10:43:14.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.784+0000 7fc16a59c640 1 -- 172.21.15.125:0/2846264968 learned_addr learned my addr 172.21.15.125:0/2846264968 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:14.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc16a59c640 1 -- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc16c15b2a0 msgr2=0x7fc16c159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:14.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc16a59c640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc16c15b2a0 0x7fc16c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc16a59c640 1 -- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc16c0cf340 msgr2=0x7fc16c159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:14.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc16a59c640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc16c0cf340 0x7fc16c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:14.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc16a59c640 1 -- 172.21.15.125:0/2846264968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164009840 con 0x7fc16c0ceb40 2023-10-18T10:43:14.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc16a59c640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 0x7fc16c1524a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00c4b0 tx=0x7fc15c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:14.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc162ffd640 1 -- 172.21.15.125:0/2846264968 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c005480 con 0x7fc16c0ceb40 2023-10-18T10:43:14.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc162ffd640 1 -- 172.21.15.125:0/2846264968 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc15c0036e0 con 0x7fc16c0ceb40 2023-10-18T10:43:14.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c0b4370 con 0x7fc16c0ceb40 2023-10-18T10:43:14.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc162ffd640 1 -- 172.21.15.125:0/2846264968 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c003b30 con 0x7fc16c0ceb40 2023-10-18T10:43:14.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.785+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c0b4930 con 0x7fc16c0ceb40 2023-10-18T10:43:14.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.786+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c0690e0 con 0x7fc16c0ceb40 2023-10-18T10:43:14.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.786+0000 7fc162ffd640 1 -- 172.21.15.125:0/2846264968 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 60) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc15c003c90 con 0x7fc16c0ceb40 2023-10-18T10:43:14.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.787+0000 7fc162ffd640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc14c06af60 0x7fc14c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:14.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.787+0000 7fc162ffd640 1 -- 172.21.15.125:0/2846264968 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc15c089160 con 0x7fc16c0ceb40 2023-10-18T10:43:14.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.787+0000 7fc163fff640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc14c06af60 0x7fc14c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:14.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.787+0000 7fc163fff640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc14c06af60 0x7fc14c06d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc16c157570 tx=0x7fc154005b50 comp rx=0 tx=0).ready entity=mgr.7360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:14.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:14.791+0000 7fc162ffd640 1 -- 172.21.15.125:0/2846264968 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc15c053f40 con 0x7fc16c0ceb40 2023-10-18T10:43:15.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.000+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc16c0b0d00 con 0x7fc16c0ceb40 2023-10-18T10:43:15.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.417+0000 7fc162ffd640 1 -- 172.21.15.125:0/2846264968 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v61) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc15c057560 con 0x7fc16c0ceb40 2023-10-18T10:43:15.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.419+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc14c06af60 msgr2=0x7fc14c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.419+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc14c06af60 0x7fc14c06d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc16c157570 tx=0x7fc154005b50 comp rx=0 tx=0).stop 2023-10-18T10:43:15.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.419+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 msgr2=0x7fc16c1524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.419+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 0x7fc16c1524a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00c4b0 tx=0x7fc15c00c970 comp rx=0 tx=0).stop 2023-10-18T10:43:15.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.420+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 shutdown_connections 2023-10-18T10:43:15.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.420+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7fc14c06af60 0x7fc14c06d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.420+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc16c0cf340 0x7fc16c159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.420+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc16c15b2a0 0x7fc16c159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.420+0000 7fc170f8e640 1 --2- 172.21.15.125:0/2846264968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc16c0ceb40 0x7fc16c1524a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.420+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 >> 172.21.15.125:0/2846264968 conn(0x7fc16c0d0d60 msgr2=0x7fc16c06b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:15.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.421+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 shutdown_connections 2023-10-18T10:43:15.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.421+0000 7fc170f8e640 1 -- 172.21.15.125:0/2846264968 wait complete. 2023-10-18T10:43:15.435 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T10:43:15.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.580+0000 7f08b1097640 1 -- 172.21.15.125:0/2759496198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 msgr2=0x7f08ac0c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.580+0000 7f08b1097640 1 --2- 172.21.15.125:0/2759496198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 0x7f08ac0c9b60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009be0 tx=0x7f08a402c0c0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.580+0000 7f08b1097640 1 -- 172.21.15.125:0/2759496198 shutdown_connections 2023-10-18T10:43:15.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.580+0000 7f08b1097640 1 --2- 172.21.15.125:0/2759496198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 0x7f08ac0c9b60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.580+0000 7f08b1097640 1 --2- 172.21.15.125:0/2759496198 >> v2:172.21.15.139:3300/0 conn(0x7f08ac06af00 0x7f08ac06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.580+0000 7f08b1097640 1 -- 172.21.15.125:0/2759496198 >> 172.21.15.125:0/2759496198 conn(0x7f08ac069930 msgr2=0x7f08ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:15.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.580+0000 7f08b1097640 1 -- 172.21.15.125:0/2759496198 shutdown_connections 2023-10-18T10:43:15.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.580+0000 7f08b1097640 1 -- 172.21.15.125:0/2759496198 wait complete. 2023-10-18T10:43:15.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.582+0000 7f08b1097640 1 Processor -- start 2023-10-18T10:43:15.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.582+0000 7f08b1097640 1 -- start start 2023-10-18T10:43:15.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08b1097640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08ac06af00 0x7f08ac137d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:15.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08b1097640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 0x7f08ac0ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:15.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08b1097640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac0c96b0 0x7f08ac0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:15.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08b1097640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0af440 con 0x7f08ac06af00 2023-10-18T10:43:15.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08b1097640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0ae830 con 0x7f08ac0c96b0 2023-10-18T10:43:15.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08b1097640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0aeb10 con 0x7f08ac0abc10 2023-10-18T10:43:15.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08ac06af00 0x7f08ac137d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:15.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08ac06af00 0x7f08ac137d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54704/0 (socket says 172.21.15.125:54704) 2023-10-18T10:43:15.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 -- 172.21.15.125:0/2592624153 learned_addr learned my addr 172.21.15.125:0/2592624153 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:15.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08a259c640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 0x7f08ac0ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:15.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08ab59e640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac0c96b0 0x7f08ac0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:15.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 -- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 msgr2=0x7f08ac0ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 0x7f08ac0ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 -- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac0c96b0 msgr2=0x7f08ac0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac0c96b0 0x7f08ac0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 -- 172.21.15.125:0/2592624153 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009840 con 0x7f08ac06af00 2023-10-18T10:43:15.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08a259c640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 0x7f08ac0ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:43:15.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.583+0000 7f08aad9d640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08ac06af00 0x7f08ac137d60 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f089c00e9f0 tx=0x7f089c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:15.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.584+0000 7f08a3fff640 1 -- 172.21.15.125:0/2592624153 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c009700 con 0x7f08ac06af00 2023-10-18T10:43:15.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.584+0000 7f08a3fff640 1 -- 172.21.15.125:0/2592624153 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f089c00ce70 con 0x7f08ac06af00 2023-10-18T10:43:15.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.584+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac0aedf0 con 0x7f08ac06af00 2023-10-18T10:43:15.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.584+0000 7f08a3fff640 1 -- 172.21.15.125:0/2592624153 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c010670 con 0x7f08ac06af00 2023-10-18T10:43:15.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.584+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac1382a0 con 0x7f08ac06af00 2023-10-18T10:43:15.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.585+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac138430 con 0x7f08ac06af00 2023-10-18T10:43:15.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.586+0000 7f08a3fff640 1 -- 172.21.15.125:0/2592624153 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 61) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f089c0107d0 con 0x7f08ac06af00 2023-10-18T10:43:15.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.586+0000 7f08a3fff640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f088c06afb0 0x7f088c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:15.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.586+0000 7f08a259c640 1 -- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f088c06afb0 msgr2=0x7f088c06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1511464628 2023-10-18T10:43:15.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.586+0000 7f08a3fff640 1 -- 172.21.15.125:0/2592624153 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f089c014070 con 0x7f08ac06af00 2023-10-18T10:43:15.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.586+0000 7f08a259c640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f088c06afb0 0x7f088c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:43:15.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.590+0000 7f08a3fff640 1 -- 172.21.15.125:0/2592624153 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f089c051ca0 con 0x7f08ac06af00 2023-10-18T10:43:15.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.761+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f08ac0690e0 con 0x7f08ac06af00 2023-10-18T10:43:15.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.764+0000 7f08a3fff640 1 -- 172.21.15.125:0/2592624153 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v52)=0 v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7f089c058f10 con 0x7f08ac06af00 2023-10-18T10:43:15.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f088c06afb0 msgr2=0x7f088c06d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:43:15.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f088c06afb0 0x7f088c06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08ac06af00 msgr2=0x7f08ac137d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08ac06af00 0x7f08ac137d60 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f089c00e9f0 tx=0x7f089c00eeb0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 shutdown_connections 2023-10-18T10:43:15.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f088c06afb0 0x7f088c06d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac0c96b0 0x7f08ac0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0abc10 0x7f08ac0ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 --2- 172.21.15.125:0/2592624153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08ac06af00 0x7f08ac137d60 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 >> 172.21.15.125:0/2592624153 conn(0x7f08ac069930 msgr2=0x7f08ac069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:15.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 shutdown_connections 2023-10-18T10:43:15.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.766+0000 7f08b1097640 1 -- 172.21.15.125:0/2592624153 wait complete. 2023-10-18T10:43:15.783 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T10:43:15.803 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:15.801+0000 7fdc5ae14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:43:15.848 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:15.846+0000 7fdc5ae14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:43:15.852 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:15.850+0000 7f133b64c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:43:15.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.888+0000 7f5547e1a640 1 -- 172.21.15.125:0/86706424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f552c003970 msgr2=0x7f552c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.888+0000 7f5547e1a640 1 --2- 172.21.15.125:0/86706424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f552c003970 0x7f552c003d90 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f552c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.888+0000 7f5547e1a640 1 -- 172.21.15.125:0/86706424 shutdown_connections 2023-10-18T10:43:15.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.888+0000 7f5547e1a640 1 --2- 172.21.15.125:0/86706424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f552c003970 0x7f552c003d90 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.888+0000 7f5547e1a640 1 -- 172.21.15.125:0/86706424 >> 172.21.15.125:0/86706424 conn(0x7f55400b0670 msgr2=0x7f55400b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:15.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.888+0000 7f5547e1a640 1 -- 172.21.15.125:0/86706424 shutdown_connections 2023-10-18T10:43:15.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.888+0000 7f5547e1a640 1 -- 172.21.15.125:0/86706424 wait complete. 2023-10-18T10:43:15.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.889+0000 7f5547e1a640 1 Processor -- start 2023-10-18T10:43:15.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.889+0000 7f5547e1a640 1 -- start start 2023-10-18T10:43:15.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f5547e1a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f552c003970 0x7f55400b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:15.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f5547e1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55400c7db0 0x7f55400c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:15.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f5547e1a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55401506f0 0x7f5540150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:15.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f5547e1a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c6b40 con 0x7f55400c7db0 2023-10-18T10:43:15.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f5547e1a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c6d30 con 0x7f55401506f0 2023-10-18T10:43:15.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f5547e1a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c7010 con 0x7f552c003970 2023-10-18T10:43:15.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55400c7db0 0x7f55400c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:15.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55400c7db0 0x7f55400c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54718/0 (socket says 172.21.15.125:54718) 2023-10-18T10:43:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 -- 172.21.15.125:0/1779899951 learned_addr learned my addr 172.21.15.125:0/1779899951 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f5545bb6640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f552c003970 0x7f55400b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55463b7640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55401506f0 0x7f5540150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 -- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f552c003970 msgr2=0x7f55400b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f552c003970 0x7f55400b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 -- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55401506f0 msgr2=0x7f5540150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55401506f0 0x7f5540150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 -- 172.21.15.125:0/1779899951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400c72f0 con 0x7f55400c7db0 2023-10-18T10:43:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55453b5640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55400c7db0 0x7f55400c6600 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f553c011ae0 tx=0x7f553c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55367fc640 1 -- 172.21.15.125:0/1779899951 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00fcb0 con 0x7f55400c7db0 2023-10-18T10:43:15.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55367fc640 1 -- 172.21.15.125:0/1779899951 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f553c00fe10 con 0x7f55400c7db0 2023-10-18T10:43:15.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.890+0000 7f55367fc640 1 -- 172.21.15.125:0/1779899951 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c004280 con 0x7f55400c7db0 2023-10-18T10:43:15.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.891+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55400c75d0 con 0x7f55400c7db0 2023-10-18T10:43:15.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.891+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5540156400 con 0x7f55400c7db0 2023-10-18T10:43:15.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.891+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55400675f0 con 0x7f55400c7db0 2023-10-18T10:43:15.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.892+0000 7f55367fc640 1 -- 172.21.15.125:0/1779899951 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 61) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f553c0035f0 con 0x7f55400c7db0 2023-10-18T10:43:15.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.892+0000 7f55367fc640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f552406afb0 0x7f552406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:15.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.893+0000 7f5545bb6640 1 -- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f552406afb0 msgr2=0x7f552406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1511464628 2023-10-18T10:43:15.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.893+0000 7f55367fc640 1 -- 172.21.15.125:0/1779899951 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f553c08a510 con 0x7f55400c7db0 2023-10-18T10:43:15.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.893+0000 7f5545bb6640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f552406afb0 0x7f552406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:43:15.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:15.897+0000 7f55367fc640 1 -- 172.21.15.125:0/1779899951 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f553c055270 con 0x7f55400c7db0 2023-10-18T10:43:15.902 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:15.898+0000 7f133b64c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:43:16.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.065+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f55400b0a50 con 0x7f55400c7db0 2023-10-18T10:43:16.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.069+0000 7f55367fc640 1 -- 172.21.15.125:0/1779899951 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v53)=0 v53) v1 ==== 133+0+0 (secure 0 0 0) 0x7f553c058890 con 0x7f55400c7db0 2023-10-18T10:43:16.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f552406afb0 msgr2=0x7f552406d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:43:16.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f552406afb0 0x7f552406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55400c7db0 msgr2=0x7f55400c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:16.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55400c7db0 0x7f55400c6600 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f553c011ae0 tx=0x7f553c009f10 comp rx=0 tx=0).stop 2023-10-18T10:43:16.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 shutdown_connections 2023-10-18T10:43:16.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f552406afb0 0x7f552406d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55401506f0 0x7f5540150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55400c7db0 0x7f55400c6600 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 --2- 172.21.15.125:0/1779899951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f552c003970 0x7f55400b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 >> 172.21.15.125:0/1779899951 conn(0x7f55400b0670 msgr2=0x7f55400ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:16.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 shutdown_connections 2023-10-18T10:43:16.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.071+0000 7f5547e1a640 1 -- 172.21.15.125:0/1779899951 wait complete. 2023-10-18T10:43:16.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:16.084 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T10:43:16.106 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-18T10:43:16.240 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.239+0000 7fdc5ae14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:43:16.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.251+0000 7f2cc697d640 1 -- 172.21.15.125:0/3238823520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc00ce420 msgr2=0x7f2cc00cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:16.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.251+0000 7f2cc697d640 1 --2- 172.21.15.125:0/3238823520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc00ce420 0x7f2cc00cd6b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00cd450 tx=0x7f2cb002be00 comp rx=0 tx=0).stop 2023-10-18T10:43:16.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.251+0000 7f2cc697d640 1 -- 172.21.15.125:0/3238823520 shutdown_connections 2023-10-18T10:43:16.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.251+0000 7f2cc697d640 1 --2- 172.21.15.125:0/3238823520 >> v2:172.21.15.139:3300/0 conn(0x7f2cc00cdbf0 0x7f2cc00ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.251+0000 7f2cc697d640 1 --2- 172.21.15.125:0/3238823520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc00ce420 0x7f2cc00cd6b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.251+0000 7f2cc697d640 1 -- 172.21.15.125:0/3238823520 >> 172.21.15.125:0/3238823520 conn(0x7f2cc0099ff0 msgr2=0x7f2cc00d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:16.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.251+0000 7f2cc697d640 1 -- 172.21.15.125:0/3238823520 shutdown_connections 2023-10-18T10:43:16.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.251+0000 7f2cc697d640 1 -- 172.21.15.125:0/3238823520 wait complete. 2023-10-18T10:43:16.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.252+0000 7f2cc697d640 1 Processor -- start 2023-10-18T10:43:16.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.253+0000 7f2cc697d640 1 -- start start 2023-10-18T10:43:16.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.253+0000 7f2cc697d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cc00cdbf0 0x7f2cc0158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:16.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.253+0000 7f2cc697d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cc00ce420 0x7f2cc00b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:16.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.253+0000 7f2cc697d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc015b180 0x7f2cc00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:16.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.253+0000 7f2cc697d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0159510 con 0x7f2cc015b180 2023-10-18T10:43:16.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.253+0000 7f2cc697d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc00c7fe0 con 0x7f2cc00ce420 2023-10-18T10:43:16.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.253+0000 7f2cc697d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc00c82c0 con 0x7f2cc00cdbf0 2023-10-18T10:43:16.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc015b180 0x7f2cc00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:16.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc015b180 0x7f2cc00c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54726/0 (socket says 172.21.15.125:54726) 2023-10-18T10:43:16.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 -- 172.21.15.125:0/205376057 learned_addr learned my addr 172.21.15.125:0/205376057 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:16.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cbf7fe640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cc00ce420 0x7f2cc00b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:16.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cbffff640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cc00cdbf0 0x7f2cc0158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:16.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cc00cdbf0 msgr2=0x7f2cc0158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:16.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cc00cdbf0 0x7f2cc0158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cc00ce420 msgr2=0x7f2cc00b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:16.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cc00ce420 0x7f2cc00b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:16.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 -- 172.21.15.125:0/205376057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb0009490 con 0x7f2cc015b180 2023-10-18T10:43:16.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cbf7fe640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cc00ce420 0x7f2cc00b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:43:16.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc4f1a640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc015b180 0x7f2cc00c9080 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800eb10 tx=0x7f2cb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:16.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004b20 con 0x7f2cc015b180 2023-10-18T10:43:16.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2cb8004ca0 con 0x7f2cc015b180 2023-10-18T10:43:16.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc00c85a0 con 0x7f2cc015b180 2023-10-18T10:43:16.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8005280 con 0x7f2cc015b180 2023-10-18T10:43:16.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.254+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc00c8b60 con 0x7f2cc015b180 2023-10-18T10:43:16.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.255+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc0066960 con 0x7f2cc015b180 2023-10-18T10:43:16.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.256+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 61) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f2cb80053e0 con 0x7f2cc015b180 2023-10-18T10:43:16.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.256+0000 7f2cbcff9640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 0x7f2cac06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:16.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.256+0000 7f2cbffff640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 msgr2=0x7f2cac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1511464628 2023-10-18T10:43:16.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.256+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2cb8014070 con 0x7f2cc015b180 2023-10-18T10:43:16.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.256+0000 7f2cbffff640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 0x7f2cac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:43:16.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.260+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2cb8055480 con 0x7f2cc015b180 2023-10-18T10:43:16.285 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.283+0000 7fdc5ae14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:43:16.304 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.302+0000 7f133b64c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:43:16.326 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.325+0000 7fdc5ae14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:43:16.348 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.347+0000 7f133b64c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:43:16.377 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.375+0000 7fdc5ae14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:43:16.390 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.389+0000 7f133b64c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:43:16.418 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.417+0000 7fdc5ae14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:43:16.443 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.442+0000 7f133b64c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:43:16.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.450+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 --> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cc00690e0 con 0x7f2cac06afe0 2023-10-18T10:43:16.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.457+0000 7f2cbffff640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 msgr2=0x7f2cac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1511464628 2023-10-18T10:43:16.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.457+0000 7f2cbffff640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 0x7f2cac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:43:16.485 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.483+0000 7f133b64c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:43:16.517 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.515+0000 7fdc5ae14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:43:16.569 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.568+0000 7fdc5ae14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:43:16.582 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.580+0000 7f133b64c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:43:16.611 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.609+0000 7fdc5ae14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:43:16.635 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.633+0000 7f133b64c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:43:16.659 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.658+0000 7fdc5ae14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:43:16.675 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.674+0000 7f133b64c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:43:16.701 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:16.699+0000 7fdc5ae14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:43:16.723 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.721+0000 7f133b64c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:43:16.765 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:16.763+0000 7f133b64c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:43:16.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.858+0000 7f2cbffff640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 msgr2=0x7f2cac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1511464628 2023-10-18T10:43:16.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:16.858+0000 7f2cbffff640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 0x7f2cac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:43:17.015 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.013+0000 7fdc5ae14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:43:17.076 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.075+0000 7f133b64c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:43:17.144 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.142+0000 7fdc5ae14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:43:17.205 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.203+0000 7f133b64c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:43:17.235 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.233+0000 7fdc5ae14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:43:17.277 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.275+0000 7fdc5ae14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:43:17.294 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.293+0000 7f133b64c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:43:17.338 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.335+0000 7f133b64c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:43:17.370 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.368+0000 7fdc5ae14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:43:17.430 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.428+0000 7f133b64c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:43:17.460 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.458+0000 7fdc5ae14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:43:17.520 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.518+0000 7f133b64c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:43:17.595 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.593+0000 7fdc5ae14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:43:17.636 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.634+0000 7fdc5ae14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:43:17.654 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.652+0000 7f133b64c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:43:17.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:17.659+0000 7f2cbffff640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 msgr2=0x7f2cac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1511464628 2023-10-18T10:43:17.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:17.659+0000 7f2cbffff640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 0x7f2cac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:43:17.696 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.694+0000 7f133b64c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:43:17.739 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:17.738+0000 7fdc5ae14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:43:17.799 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:17.797+0000 7f133b64c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:43:18.351 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.350+0000 7fdc5ae14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:43:18.413 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:18.411+0000 7f133b64c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:43:18.613 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.612+0000 7fdc5ae14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:43:18.662 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.660+0000 7fdc5ae14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:43:18.676 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:18.674+0000 7f133b64c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:43:18.725 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:18.723+0000 7f133b64c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:43:18.762 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.761+0000 7fdc5ae14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:43:18.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:18.782+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 62) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f2cb8088a60 con 0x7f2cc015b180 2023-10-18T10:43:18.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:18.782+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 msgr2=0x7f2cac06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:43:18.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:18.782+0000 7f2cbcff9640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/1511464628,v1:172.21.15.139:6833/1511464628] conn(0x7f2cac06afe0 0x7f2cac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:18.831 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:43:18.829+0000 7f133b64c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:43:18.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd75bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd75bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd75bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd75bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd75bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd9e01640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd9e01640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd9e01640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd9e01640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:18.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:43:18.882+0000 7fdbd9e01640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:43:19.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.786+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 63) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2cb8055850 con 0x7f2cc015b180 2023-10-18T10:43:19.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.786+0000 7f2cbcff9640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2cac035880 0x7f2cac0753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:19.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.786+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 --> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cc00690e0 con 0x7f2cac035880 2023-10-18T10:43:19.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.788+0000 7f2cbffff640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2cac035880 0x7f2cac0753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:19.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.790+0000 7f2cbffff640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2cac035880 0x7f2cac0753b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40011e0 tx=0x7f2cb4007d20 comp rx=0 tx=0).ready entity=mgr.6878 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:19.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.840+0000 7f2cbcff9640 1 -- 172.21.15.125:0/205376057 <== mgr.6878 v2:172.21.15.139:6832/3564227248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2cc00690e0 con 0x7f2cac035880 2023-10-18T10:43:19.843 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1 2023-10-18T10:43:19.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.842+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2cac035880 msgr2=0x7f2cac0753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:19.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.842+0000 7f2cc697d640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2cac035880 0x7f2cac0753b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40011e0 tx=0x7f2cb4007d20 comp rx=0 tx=0).stop 2023-10-18T10:43:19.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.842+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc015b180 msgr2=0x7f2cc00c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:19.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.842+0000 7f2cc697d640 1 --2- 172.21.15.125:0/205376057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2cc015b180 0x7f2cc00c9080 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800eb10 tx=0x7f2cb800c2d0 comp rx=0 tx=0).stop 2023-10-18T10:43:19.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.842+0000 7f2cc4f1a640 1 -- 172.21.15.125:0/205376057 reap_dead start 2023-10-18T10:43:19.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.842+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 shutdown_connections 2023-10-18T10:43:19.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.842+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 >> 172.21.15.125:0/205376057 conn(0x7f2cc0099ff0 msgr2=0x7f2cc015a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:19.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.843+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 shutdown_connections 2023-10-18T10:43:19.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.843+0000 7f2cc697d640 1 -- 172.21.15.125:0/205376057 wait complete. 2023-10-18T10:43:19.857 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-18T10:43:19.858 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-18-10_44_00 in ~41s... 2023-10-18T10:43:19.858 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-18T10:43:19.859 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-18T10:43:19.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.964+0000 7f65fd79e640 1 -- 172.21.15.125:0/4055315907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80cbd20 msgr2=0x7f65f8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:19.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.964+0000 7f65fd79e640 1 --2- 172.21.15.125:0/4055315907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80cbd20 0x7f65f8158300 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f65e8005fd0 tx=0x7f65e802d410 comp rx=0 tx=0).stop 2023-10-18T10:43:19.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.965+0000 7f65fd79e640 1 -- 172.21.15.125:0/4055315907 shutdown_connections 2023-10-18T10:43:19.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.965+0000 7f65fd79e640 1 --2- 172.21.15.125:0/4055315907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80cbd20 0x7f65f8158300 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:19.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.965+0000 7f65fd79e640 1 -- 172.21.15.125:0/4055315907 >> 172.21.15.125:0/4055315907 conn(0x7f65f8069930 msgr2=0x7f65f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:19.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.965+0000 7f65fd79e640 1 -- 172.21.15.125:0/4055315907 shutdown_connections 2023-10-18T10:43:19.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.965+0000 7f65fd79e640 1 -- 172.21.15.125:0/4055315907 wait complete. 2023-10-18T10:43:19.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.966+0000 7f65fd79e640 1 Processor -- start 2023-10-18T10:43:19.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.966+0000 7f65fd79e640 1 -- start start 2023-10-18T10:43:19.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.966+0000 7f65fd79e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65f80afa20 0x7f65f81496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:19.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65fd79e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80b0530 0x7f65f813a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:19.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65fd79e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65f80cbd20 0x7f65f81562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:19.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65fd79e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f813a7e0 con 0x7f65f80b0530 2023-10-18T10:43:19.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65fd79e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80ae1e0 con 0x7f65f80afa20 2023-10-18T10:43:19.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65fd79e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f8149df0 con 0x7f65f80cbd20 2023-10-18T10:43:19.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80b0530 0x7f65f813a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:19.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80b0530 0x7f65f813a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:47062/0 (socket says 172.21.15.125:47062) 2023-10-18T10:43:19.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 -- 172.21.15.125:0/380745781 learned_addr learned my addr 172.21.15.125:0/380745781 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:43:19.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f6ffd640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65f80afa20 0x7f65f81496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:19.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f77fe640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65f80cbd20 0x7f65f81562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:19.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 -- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65f80cbd20 msgr2=0x7f65f81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:19.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65f80cbd20 0x7f65f81562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:19.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 -- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65f80afa20 msgr2=0x7f65f81496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:19.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65f80afa20 0x7f65f81496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:19.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 -- 172.21.15.125:0/380745781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e800a950 con 0x7f65f80b0530 2023-10-18T10:43:19.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65f67fc640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80b0530 0x7f65f813a2a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00c400 tx=0x7f65ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:19.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65df7fe640 1 -- 172.21.15.125:0/380745781 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec004230 con 0x7f65f80b0530 2023-10-18T10:43:19.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65df7fe640 1 -- 172.21.15.125:0/380745781 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f65ec0044a0 con 0x7f65f80b0530 2023-10-18T10:43:19.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f814a0d0 con 0x7f65f80b0530 2023-10-18T10:43:19.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65df7fe640 1 -- 172.21.15.125:0/380745781 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec003850 con 0x7f65f80b0530 2023-10-18T10:43:19.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.967+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f814a690 con 0x7f65f80b0530 2023-10-18T10:43:19.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.968+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f8158a00 con 0x7f65f80b0530 2023-10-18T10:43:19.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.969+0000 7f65df7fe640 1 -- 172.21.15.125:0/380745781 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 63) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f65ec0039b0 con 0x7f65f80b0530 2023-10-18T10:43:19.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.969+0000 7f65df7fe640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f65e406ace0 0x7f65e406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:43:19.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.969+0000 7f65df7fe640 1 -- 172.21.15.125:0/380745781 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f65ec088cd0 con 0x7f65f80b0530 2023-10-18T10:43:19.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.969+0000 7f65f6ffd640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f65e406ace0 0x7f65e406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:43:19.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.970+0000 7f65f6ffd640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f65e406ace0 0x7f65e406d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f65e8036000 tx=0x7f65e8004ab0 comp rx=0 tx=0).ready entity=mgr.6878 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:43:19.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:19.973+0000 7f65df7fe640 1 -- 172.21.15.125:0/380745781 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65ec053ed0 con 0x7f65f80b0530 2023-10-18T10:43:20.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.159+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 --> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65f80690e0 con 0x7f65e406ace0 2023-10-18T10:43:20.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.164+0000 7f65df7fe640 1 -- 172.21.15.125:0/380745781 <== mgr.6878 v2:172.21.15.139:6832/3564227248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f65f80690e0 con 0x7f65e406ace0 2023-10-18T10:43:20.167 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:43:20.167 INFO:teuthology.orchestra.run.smithi125.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-18T10:43:20.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f65e406ace0 msgr2=0x7f65e406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:20.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f65e406ace0 0x7f65e406d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f65e8036000 tx=0x7f65e8004ab0 comp rx=0 tx=0).stop 2023-10-18T10:43:20.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80b0530 msgr2=0x7f65f813a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:43:20.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80b0530 0x7f65f813a2a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00c400 tx=0x7f65ec00c8c0 comp rx=0 tx=0).stop 2023-10-18T10:43:20.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 shutdown_connections 2023-10-18T10:43:20.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f65e406ace0 0x7f65e406d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:20.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65f80cbd20 0x7f65f81562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:20.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65f80b0530 0x7f65f813a2a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:20.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 --2- 172.21.15.125:0/380745781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65f80afa20 0x7f65f81496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:43:20.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 >> 172.21.15.125:0/380745781 conn(0x7f65f8069930 msgr2=0x7f65f80ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:43:20.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 shutdown_connections 2023-10-18T10:43:20.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:43:20.166+0000 7f65fd79e640 1 -- 172.21.15.125:0/380745781 wait complete. 2023-10-18T10:43:20.179 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-18T10:43:20.179 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{}]} 2023-10-18T10:43:20.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:20.180 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:20.202 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:20.203 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:20.203 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:21.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:21.205 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:21.225 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:21.226 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:21.226 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:22.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:22.227 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:22.250 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:22.250 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:22.251 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:23.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:23.252 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:23.274 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:23.275 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:23.275 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:24.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:24.276 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:24.299 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:24.300 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:24.300 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:25.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:25.302 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:25.324 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:25.325 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:25.325 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:26.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:26.327 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:26.350 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:26.350 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:26.351 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:27.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:27.353 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:27.376 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:27.376 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:27.376 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:28.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:28.379 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:28.401 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:28.402 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:28.402 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:29.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:29.404 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:29.427 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:29.428 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:29.428 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:30.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:30.430 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:30.453 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:30.453 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:30.453 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:31.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:31.455 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:31.477 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:31.478 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:31.478 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:32.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:32.480 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:32.502 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:32.503 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:32.503 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:33.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:33.505 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:33.528 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:33.528 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:33.529 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:34.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:34.530 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:34.555 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:34.556 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:34.556 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:35.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:35.559 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:35.590 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:35.590 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:35.591 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:36.177 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:43:36.180 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:43:36.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:36.593 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:36.615 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:36.615 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:36.615 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:37.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:37.618 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:37.641 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:37.641 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:37.641 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:38.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:38.643 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:38.666 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:38.666 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:38.667 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:39.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:39.668 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:39.691 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:39.692 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:39.692 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:40.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:40.694 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:40.719 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:40.719 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:40.720 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:41.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:41.721 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:41.745 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:41.745 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:41.745 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:42.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:42.747 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:42.770 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:42.771 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:42.771 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:43.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:43.773 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:43.796 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:43.796 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:43.796 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:44.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:44.798 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:44.821 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:44.821 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:44.821 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:45.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:45.824 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:45.850 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:45.850 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:45.850 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:46.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:46.852 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:46.876 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:46.876 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:46.876 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:47.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:47.878 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:47.901 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:47.901 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:47.902 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:48.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:48.904 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:48.928 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:48.928 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:48.929 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:49.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:49.930 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:49.953 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:49.954 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:49.954 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:50.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:50.956 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:50.979 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:50.979 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:50.980 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:51.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:51.981 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:52.004 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:52.004 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:52.005 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:53.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:53.007 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:53.030 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:53.031 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:53.032 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:54.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:54.034 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:54.056 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:54.057 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:54.057 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:55.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:55.058 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:55.081 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:55.081 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:55.082 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:56.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:56.084 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:56.111 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:56.112 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:56.112 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:57.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:57.114 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:57.136 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:57.136 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:57.136 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:58.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:58.138 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:58.160 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:58.161 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:58.161 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:43:59.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:43:59.163 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:43:59.186 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:43:59.186 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:43:59.187 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:44:00.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:00.188 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:44:00.211 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T10:44:00.211 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T10:44:00.212 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:44:00.852 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:00.850+0000 7fdbc9da1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:00.852 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:00.850+0000 7fdbc9da1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:00.852 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:00.850+0000 7fdbc9da1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:00.853 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:00.850+0000 7fdbc9da1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:00.853 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:00.850+0000 7fdbc9da1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:01.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:01.214 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:44:01.236 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_44_00_UTC 2023-10-18T10:44:01.237 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-10_44_00_UTC'} 2023-10-18T10:44:01.237 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-18-10_44_00_UTC'} 2023-10-18T10:44:01.238 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T10:44:01.238 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-18-10_44_00_UTC'] 2023-10-18T10:44:01.238 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697625799.85816, now = 1697625841.2389631, timo = 41 2023-10-18T10:44:02.239 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-18T10:44:02.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.354+0000 7f7316f68640 1 -- 172.21.15.125:0/309435855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73000039a0 msgr2=0x7f7300003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:02.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.354+0000 7f7316f68640 1 --2- 172.21.15.125:0/309435855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73000039a0 0x7f7300003dc0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f7304006fd0 tx=0x7f73000057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.354+0000 7f7315505640 1 -- 172.21.15.125:0/309435855 reap_dead start 2023-10-18T10:44:02.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.354+0000 7f7316f68640 1 -- 172.21.15.125:0/309435855 shutdown_connections 2023-10-18T10:44:02.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.354+0000 7f7316f68640 1 -- 172.21.15.125:0/309435855 >> 172.21.15.125:0/309435855 conn(0x7f7310069930 msgr2=0x7f7310069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:02.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.354+0000 7f7316f68640 1 -- 172.21.15.125:0/309435855 shutdown_connections 2023-10-18T10:44:02.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.354+0000 7f7316f68640 1 -- 172.21.15.125:0/309435855 wait complete. 2023-10-18T10:44:02.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.355+0000 7f7316f68640 1 Processor -- start 2023-10-18T10:44:02.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.355+0000 7f7316f68640 1 -- start start 2023-10-18T10:44:02.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7316f68640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73101370d0 0x7f7310150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:02.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7316f68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73101506c0 0x7f73101e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:02.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7316f68640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73101e11d0 0x7f731014ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:02.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7316f68640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73101e2520 con 0x7f73101370d0 2023-10-18T10:44:02.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7316f68640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7310150b20 con 0x7f73101506c0 2023-10-18T10:44:02.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7316f68640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73101e1650 con 0x7f73101e11d0 2023-10-18T10:44:02.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7314d04640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73101370d0 0x7f7310150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7314d04640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73101370d0 0x7f7310150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44440/0 (socket says 172.21.15.125:44440) 2023-10-18T10:44:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7314d04640 1 -- 172.21.15.125:0/724767298 learned_addr learned my addr 172.21.15.125:0/724767298 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7315505640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73101e11d0 0x7f731014ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f730ffff640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73101506c0 0x7f73101e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.356+0000 7f7314d04640 1 -- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73101e11d0 msgr2=0x7f731014ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:02.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f7314d04640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73101e11d0 0x7f731014ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f7314d04640 1 -- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73101506c0 msgr2=0x7f73101e0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:02.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f7314d04640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73101506c0 0x7f73101e0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f7314d04640 1 -- 172.21.15.125:0/724767298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731014c460 con 0x7f73101370d0 2023-10-18T10:44:02.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f730ffff640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73101506c0 0x7f73101e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:02.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f7314d04640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73101370d0 0x7f7310150180 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f7304000c00 tx=0x7f7304001540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:02.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f730d7fa640 1 -- 172.21.15.125:0/724767298 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304006460 con 0x7f73101370d0 2023-10-18T10:44:02.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f730d7fa640 1 -- 172.21.15.125:0/724767298 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7304001730 con 0x7f73101370d0 2023-10-18T10:44:02.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73101e18a0 con 0x7f73101370d0 2023-10-18T10:44:02.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f730d7fa640 1 -- 172.21.15.125:0/724767298 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73040065c0 con 0x7f73101370d0 2023-10-18T10:44:02.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.357+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73101e1e60 con 0x7f73101370d0 2023-10-18T10:44:02.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.358+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73100690e0 con 0x7f73101370d0 2023-10-18T10:44:02.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.359+0000 7f730d7fa640 1 -- 172.21.15.125:0/724767298 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 66) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f7304031030 con 0x7f73101370d0 2023-10-18T10:44:02.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.359+0000 7f730d7fa640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f72f406b100 0x7f72f406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.359+0000 7f730d7fa640 1 -- 172.21.15.125:0/724767298 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f73040ab620 con 0x7f73101370d0 2023-10-18T10:44:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.359+0000 7f730ffff640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f72f406b100 0x7f72f406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.360+0000 7f730ffff640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f72f406b100 0x7f72f406d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005d00 tx=0x7f72fc005ab0 comp rx=0 tx=0).ready entity=mgr.6878 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.363+0000 7f730d7fa640 1 -- 172.21.15.125:0/724767298 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7304076000 con 0x7f73101370d0 2023-10-18T10:44:02.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.550+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 --> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f731006f700 con 0x7f72f406b100 2023-10-18T10:44:02.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.612+0000 7f730d7fa640 1 -- 172.21.15.125:0/724767298 <== mgr.6878 v2:172.21.15.139:6832/3564227248 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f731006f700 con 0x7f72f406b100 2023-10-18T10:44:02.615 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path snap_test_dir1 2023-10-18T10:44:02.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f72f406b100 msgr2=0x7f72f406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f72f406b100 0x7f72f406d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005d00 tx=0x7f72fc005ab0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73101370d0 msgr2=0x7f7310150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73101370d0 0x7f7310150180 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f7304000c00 tx=0x7f7304001540 comp rx=0 tx=0).stop 2023-10-18T10:44:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 shutdown_connections 2023-10-18T10:44:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f72f406b100 0x7f72f406d5c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73101506c0 0x7f73101e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73101e11d0 0x7f731014ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 --2- 172.21.15.125:0/724767298 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73101370d0 0x7f7310150180 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 >> 172.21.15.125:0/724767298 conn(0x7f7310069930 msgr2=0x7f7310065490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 shutdown_connections 2023-10-18T10:44:02.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.614+0000 7f7316f68640 1 -- 172.21.15.125:0/724767298 wait complete. 2023-10-18T10:44:02.627 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-18T10:44:02.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:02.628 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T10:44:02.646 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_44_00_UTC 2023-10-18T10:44:02.646 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_44_00_UTC 2023-10-18T10:44:02.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:02.647 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_44_00_UTC) 2023-10-18T10:44:02.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:02.708 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-18T10:44:02.765 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T10:44:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.909+0000 7f2b63d9e640 1 -- 172.21.15.125:0/687787980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c0abae0 msgr2=0x7f2b5c0abf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.909+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/687787980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c0abae0 0x7f2b5c0abf70 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009830 tx=0x7f2b5402bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.909+0000 7f2b63d9e640 1 -- 172.21.15.125:0/687787980 shutdown_connections 2023-10-18T10:44:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.909+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/687787980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c0abae0 0x7f2b5c0abf70 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.909+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/687787980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b5c06a730 0x7f2b5c06ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.909+0000 7f2b63d9e640 1 -- 172.21.15.125:0/687787980 >> 172.21.15.125:0/687787980 conn(0x7f2b5c06fae0 msgr2=0x7f2b5c06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:02.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.909+0000 7f2b63d9e640 1 -- 172.21.15.125:0/687787980 shutdown_connections 2023-10-18T10:44:02.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.909+0000 7f2b63d9e640 1 -- 172.21.15.125:0/687787980 wait complete. 2023-10-18T10:44:02.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.910+0000 7f2b63d9e640 1 Processor -- start 2023-10-18T10:44:02.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.910+0000 7f2b63d9e640 1 -- start start 2023-10-18T10:44:02.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b63d9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b5c06a730 0x7f2b5c15bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:02.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b63d9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5c0abae0 0x7f2b5c14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:02.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b63d9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c155580 0x7f2b5c14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:02.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b63d9e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c15bff0 con 0x7f2b5c155580 2023-10-18T10:44:02.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b63d9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c14d5c0 con 0x7f2b5c0abae0 2023-10-18T10:44:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b63d9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c14d8a0 con 0x7f2b5c06a730 2023-10-18T10:44:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c155580 0x7f2b5c14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c155580 0x7f2b5c14e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44454/0 (socket says 172.21.15.125:44454) 2023-10-18T10:44:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 -- 172.21.15.125:0/1018263203 learned_addr learned my addr 172.21.15.125:0/1018263203 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:02.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 -- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b5c06a730 msgr2=0x7f2b5c15bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:02.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b5c06a730 0x7f2b5c15bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 -- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5c0abae0 msgr2=0x7f2b5c14cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:02.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b61339640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5c0abae0 0x7f2b5c14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:02.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5c0abae0 0x7f2b5c14cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:02.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 -- 172.21.15.125:0/1018263203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54009490 con 0x7f2b5c155580 2023-10-18T10:44:02.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.911+0000 7f2b6233b640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c155580 0x7f2b5c14e830 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009780 tx=0x7f2b54003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:02.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.912+0000 7f2b527fc640 1 -- 172.21.15.125:0/1018263203 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54003e20 con 0x7f2b5c155580 2023-10-18T10:44:02.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.912+0000 7f2b527fc640 1 -- 172.21.15.125:0/1018263203 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2b54034530 con 0x7f2b5c155580 2023-10-18T10:44:02.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.912+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c14db80 con 0x7f2b5c155580 2023-10-18T10:44:02.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.912+0000 7f2b527fc640 1 -- 172.21.15.125:0/1018263203 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54034940 con 0x7f2b5c155580 2023-10-18T10:44:02.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.912+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c14e140 con 0x7f2b5c155580 2023-10-18T10:44:02.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.913+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c14e330 con 0x7f2b5c155580 2023-10-18T10:44:02.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.913+0000 7f2b527fc640 1 -- 172.21.15.125:0/1018263203 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 66) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f2b5403b040 con 0x7f2b5c155580 2023-10-18T10:44:02.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.913+0000 7f2b527fc640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2b4006b100 0x7f2b4006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:02.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.913+0000 7f2b527fc640 1 -- 172.21.15.125:0/1018263203 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f2b540ac290 con 0x7f2b5c155580 2023-10-18T10:44:02.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.913+0000 7f2b61b3a640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2b4006b100 0x7f2b4006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:02.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.914+0000 7f2b61b3a640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2b4006b100 0x7f2b4006d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b580012e0 tx=0x7f2b58001630 comp rx=0 tx=0).ready entity=mgr.6878 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:02.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:02.917+0000 7f2b527fc640 1 -- 172.21.15.125:0/1018263203 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b54076d40 con 0x7f2b5c155580 2023-10-18T10:44:03.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.130+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f2b5c0690e0 con 0x7f2b5c155580 2023-10-18T10:44:03.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.874+0000 7f2b527fc640 1 -- 172.21.15.125:0/1018263203 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v67) v1 ==== 93+0+0 (secure 0 0 0) 0x7f2b5407a360 con 0x7f2b5c155580 2023-10-18T10:44:03.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2b4006b100 msgr2=0x7f2b4006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:03.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2b4006b100 0x7f2b4006d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b580012e0 tx=0x7f2b58001630 comp rx=0 tx=0).stop 2023-10-18T10:44:03.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c155580 msgr2=0x7f2b5c14e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:03.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c155580 0x7f2b5c14e830 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009780 tx=0x7f2b54003820 comp rx=0 tx=0).stop 2023-10-18T10:44:03.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 shutdown_connections 2023-10-18T10:44:03.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f2b4006b100 0x7f2b4006d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:03.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5c0abae0 0x7f2b5c14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:03.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b5c155580 0x7f2b5c14e830 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:03.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 --2- 172.21.15.125:0/1018263203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b5c06a730 0x7f2b5c15bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:03.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 >> 172.21.15.125:0/1018263203 conn(0x7f2b5c06fae0 msgr2=0x7f2b5c065640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:03.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 shutdown_connections 2023-10-18T10:44:03.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:03.877+0000 7f2b63d9e640 1 -- 172.21.15.125:0/1018263203 wait complete. 2023-10-18T10:44:03.892 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:44:03.893 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:44:03.942 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:44:04.009 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:44:04.010 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:44:04.010 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:04.026 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:04.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.169+0000 7fdd8294c640 1 -- 172.21.15.125:0/2130331429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c15b2a0 msgr2=0x7fdd7c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.169+0000 7fdd8294c640 1 --2- 172.21.15.125:0/2130331429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c15b2a0 0x7fdd7c0d1910 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c005fd0 tx=0x7fdd6c02bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:04.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.169+0000 7fdd8294c640 1 -- 172.21.15.125:0/2130331429 shutdown_connections 2023-10-18T10:44:04.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.169+0000 7fdd8294c640 1 --2- 172.21.15.125:0/2130331429 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd7c0ceb40 0x7fdd7c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.169+0000 7fdd8294c640 1 --2- 172.21.15.125:0/2130331429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c15b2a0 0x7fdd7c0d1910 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.169+0000 7fdd8294c640 1 -- 172.21.15.125:0/2130331429 >> 172.21.15.125:0/2130331429 conn(0x7fdd7c0d0d60 msgr2=0x7fdd7c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:04.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.169+0000 7fdd8294c640 1 -- 172.21.15.125:0/2130331429 shutdown_connections 2023-10-18T10:44:04.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.169+0000 7fdd8294c640 1 -- 172.21.15.125:0/2130331429 wait complete. 2023-10-18T10:44:04.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.171+0000 7fdd8294c640 1 Processor -- start 2023-10-18T10:44:04.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.171+0000 7fdd8294c640 1 -- start start 2023-10-18T10:44:04.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd8294c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c0ceb40 0x7fdd7c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd8294c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd7c0cf340 0x7fdd7c156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd8294c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c15b2a0 0x7fdd7c149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd8294c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c1572d0 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd8294c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c1574e0 con 0x7fdd7c15b2a0 2023-10-18T10:44:04.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd8294c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c1577c0 con 0x7fdd7c0cf340 2023-10-18T10:44:04.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd7bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c0ceb40 0x7fdd7c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd7bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c0ceb40 0x7fdd7c0c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44480/0 (socket says 172.21.15.125:44480) 2023-10-18T10:44:04.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd7bfff640 1 -- 172.21.15.125:0/1293530350 learned_addr learned my addr 172.21.15.125:0/1293530350 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:04.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd80ee9640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c15b2a0 0x7fdd7c149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.172+0000 7fdd7b7fe640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd7c0cf340 0x7fdd7c156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd7bfff640 1 -- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd7c0cf340 msgr2=0x7fdd7c156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd7bfff640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd7c0cf340 0x7fdd7c156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd7bfff640 1 -- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c15b2a0 msgr2=0x7fdd7c149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd7bfff640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c15b2a0 0x7fdd7c149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd7bfff640 1 -- 172.21.15.125:0/1293530350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009560 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd7bfff640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c0ceb40 0x7fdd7c0c7e30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c400 tx=0x7fdd7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:04.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd78ff9640 1 -- 172.21.15.125:0/1293530350 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700052d0 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd78ff9640 1 -- 172.21.15.125:0/1293530350 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd70007020 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c0b0d50 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd78ff9640 1 -- 172.21.15.125:0/1293530350 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70007470 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.173+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c157d80 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.174+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7c0690e0 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.175+0000 7fdd78ff9640 1 -- 172.21.15.125:0/1293530350 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fdd700067d0 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.175+0000 7fdd78ff9640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fdd6806b080 0x7fdd6806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.175+0000 7fdd7b7fe640 1 -- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fdd6806b080 msgr2=0x7fdd6806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:04.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.175+0000 7fdd78ff9640 1 -- 172.21.15.125:0/1293530350 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7fdd70089200 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.175+0000 7fdd7b7fe640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fdd6806b080 0x7fdd6806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:04.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.179+0000 7fdd78ff9640 1 -- 172.21.15.125:0/1293530350 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd70053e30 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.266 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:04.264+0000 7fa4ec64c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:44:04.270 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:04.268+0000 7fd9276b2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:44:04.310 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:04.308+0000 7fa4ec64c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:44:04.316 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:04.315+0000 7fd9276b2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:44:04.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.355+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd7c0d0b40 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.356+0000 7fdd78ff9640 1 -- 172.21.15.125:0/1293530350 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+17269 (secure 0 0 0) 0x7fdd70057450 con 0x7fdd7c0ceb40 2023-10-18T10:44:04.358 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:04.358 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":129,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:43:18.775585+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:43:01.020853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-18T10:43:02.278684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2083342052":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/1711146434":"2023-10-19T10:43:18.775528+0000","172.21.15.139:6833/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:6832/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/3838090415":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/214738857":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/188703397":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/84406598":"2023-10-19T10:43:18.775528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:04.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fdd6806b080 msgr2=0x7fdd6806d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:04.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fdd6806b080 0x7fdd6806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c0ceb40 msgr2=0x7fdd7c0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c0ceb40 0x7fdd7c0c7e30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c400 tx=0x7fdd7000c8c0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 shutdown_connections 2023-10-18T10:44:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fdd6806b080 0x7fdd6806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c15b2a0 0x7fdd7c149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd7c0cf340 0x7fdd7c156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 --2- 172.21.15.125:0/1293530350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c0ceb40 0x7fdd7c0c7e30 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 >> 172.21.15.125:0/1293530350 conn(0x7fdd7c0d0d60 msgr2=0x7fdd7c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:04.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 shutdown_connections 2023-10-18T10:44:04.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.358+0000 7fdd8294c640 1 -- 172.21.15.125:0/1293530350 wait complete. 2023-10-18T10:44:04.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:43:01.020853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:43:02.278684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:44:04.372 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:44:04.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.478+0000 7fb43246f640 1 -- 172.21.15.125:0/2145388314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c159060 msgr2=0x7fb42c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.478+0000 7fb43246f640 1 --2- 172.21.15.125:0/2145388314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c159060 0x7fb42c06d260 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb420009860 tx=0x7fb42002bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:04.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.478+0000 7fb43246f640 1 -- 172.21.15.125:0/2145388314 shutdown_connections 2023-10-18T10:44:04.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.478+0000 7fb43246f640 1 --2- 172.21.15.125:0/2145388314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c159060 0x7fb42c06d260 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.478+0000 7fb43246f640 1 -- 172.21.15.125:0/2145388314 >> 172.21.15.125:0/2145388314 conn(0x7fb42c0b0670 msgr2=0x7fb42c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:04.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.478+0000 7fb43246f640 1 -- 172.21.15.125:0/2145388314 shutdown_connections 2023-10-18T10:44:04.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.478+0000 7fb43246f640 1 -- 172.21.15.125:0/2145388314 wait complete. 2023-10-18T10:44:04.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.479+0000 7fb43246f640 1 Processor -- start 2023-10-18T10:44:04.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb43246f640 1 -- start start 2023-10-18T10:44:04.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb43246f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0cdb30 0x7fb42c14c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb43246f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb42c159060 0x7fb42c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb43246f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c15d400 0x7fb42c14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb43246f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c14e6d0 con 0x7fb42c159060 2023-10-18T10:44:04.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb43246f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c14e110 con 0x7fb42c0cdb30 2023-10-18T10:44:04.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb43246f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c14e3f0 con 0x7fb42c15d400 2023-10-18T10:44:04.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb42b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb42c159060 0x7fb42c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb42b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb42c159060 0x7fb42c0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44496/0 (socket says 172.21.15.125:44496) 2023-10-18T10:44:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb42b7fe640 1 -- 172.21.15.125:0/2027636583 learned_addr learned my addr 172.21.15.125:0/2027636583 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.480+0000 7fb42bfff640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0cdb30 0x7fb42c14c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb430a0c640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c15d400 0x7fb42c14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb42b7fe640 1 -- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c15d400 msgr2=0x7fb42c14d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb42b7fe640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c15d400 0x7fb42c14d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb42b7fe640 1 -- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0cdb30 msgr2=0x7fb42c14c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb42b7fe640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0cdb30 0x7fb42c14c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb42b7fe640 1 -- 172.21.15.125:0/2027636583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb420009490 con 0x7fb42c159060 2023-10-18T10:44:04.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb42b7fe640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb42c159060 0x7fb42c0c7c00 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c490 tx=0x7fb41c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:04.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb428ff9640 1 -- 172.21.15.125:0/2027636583 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c012830 con 0x7fb42c159060 2023-10-18T10:44:04.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb428ff9640 1 -- 172.21.15.125:0/2027636583 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb41c012a80 con 0x7fb42c159060 2023-10-18T10:44:04.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c0c7400 con 0x7fb42c159060 2023-10-18T10:44:04.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb428ff9640 1 -- 172.21.15.125:0/2027636583 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c011380 con 0x7fb42c159060 2023-10-18T10:44:04.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.481+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c0c8140 con 0x7fb42c159060 2023-10-18T10:44:04.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.482+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c0675f0 con 0x7fb42c159060 2023-10-18T10:44:04.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.483+0000 7fb428ff9640 1 -- 172.21.15.125:0/2027636583 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fb41c0114e0 con 0x7fb42c159060 2023-10-18T10:44:04.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.483+0000 7fb428ff9640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fb41806b080 0x7fb41806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.483+0000 7fb42bfff640 1 -- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fb41806b080 msgr2=0x7fb41806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:04.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.483+0000 7fb428ff9640 1 -- 172.21.15.125:0/2027636583 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7fb41c089d10 con 0x7fb42c159060 2023-10-18T10:44:04.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.483+0000 7fb42bfff640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fb41806b080 0x7fb41806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:04.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.487+0000 7fb428ff9640 1 -- 172.21.15.125:0/2027636583 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb41c0583d0 con 0x7fb42c159060 2023-10-18T10:44:04.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.659+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb42c0b4aa0 con 0x7fb42c159060 2023-10-18T10:44:04.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.660+0000 7fb428ff9640 1 -- 172.21.15.125:0/2027636583 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb41c0160a0 con 0x7fb42c159060 2023-10-18T10:44:04.662 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:44:04.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fb41806b080 msgr2=0x7fb41806d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:04.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fb41806b080 0x7fb41806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb42c159060 msgr2=0x7fb42c0c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb42c159060 0x7fb42c0c7c00 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c490 tx=0x7fb41c00c950 comp rx=0 tx=0).stop 2023-10-18T10:44:04.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 shutdown_connections 2023-10-18T10:44:04.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fb41806b080 0x7fb41806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c15d400 0x7fb42c14d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb42c159060 0x7fb42c0c7c00 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 --2- 172.21.15.125:0/2027636583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0cdb30 0x7fb42c14c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 >> 172.21.15.125:0/2027636583 conn(0x7fb42c0b0670 msgr2=0x7fb42c0ada00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:04.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 shutdown_connections 2023-10-18T10:44:04.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.662+0000 7fb43246f640 1 -- 172.21.15.125:0/2027636583 wait complete. 2023-10-18T10:44:04.675 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:44:04.698 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:04.697+0000 7fa4ec64c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:44:04.730 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:04.728+0000 7fd9276b2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:44:04.741 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:04.740+0000 7fa4ec64c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:44:04.776 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:04.774+0000 7fd9276b2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:44:04.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 -- 172.21.15.125:0/2080517537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c0ceb40 msgr2=0x7ff28c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 --2- 172.21.15.125:0/2080517537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c0ceb40 0x7ff28c0b0650 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff28c08a810 tx=0x7ff28402c1d0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 -- 172.21.15.125:0/2080517537 shutdown_connections 2023-10-18T10:44:04.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 --2- 172.21.15.125:0/2080517537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c0ceb40 0x7ff28c0b0650 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 --2- 172.21.15.125:0/2080517537 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff28c15b2a0 0x7ff28c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 --2- 172.21.15.125:0/2080517537 >> v2:172.21.15.139:3300/0 conn(0x7ff28c0cf340 0x7ff28c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 -- 172.21.15.125:0/2080517537 >> 172.21.15.125:0/2080517537 conn(0x7ff28c0d0d60 msgr2=0x7ff28c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:04.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 -- 172.21.15.125:0/2080517537 shutdown_connections 2023-10-18T10:44:04.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.781+0000 7ff292397640 1 -- 172.21.15.125:0/2080517537 wait complete. 2023-10-18T10:44:04.786 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:04.782+0000 7fa4ec64c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:44:04.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.782+0000 7ff292397640 1 Processor -- start 2023-10-18T10:44:04.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.782+0000 7ff292397640 1 -- start start 2023-10-18T10:44:04.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff292397640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff28c0ceb40 0x7ff28c156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff292397640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff28c0cf340 0x7ff28c06b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff292397640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c15b2a0 0x7ff28c06b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff292397640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c15d350 con 0x7ff28c15b2a0 2023-10-18T10:44:04.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff292397640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c158500 con 0x7ff28c0ceb40 2023-10-18T10:44:04.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff292397640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c158220 con 0x7ff28c0cf340 2023-10-18T10:44:04.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff290934640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c15b2a0 0x7ff28c06b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff290934640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c15b2a0 0x7ff28c06b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44514/0 (socket says 172.21.15.125:44514) 2023-10-18T10:44:04.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff290934640 1 -- 172.21.15.125:0/1094198950 learned_addr learned my addr 172.21.15.125:0/1094198950 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:04.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff28bfff640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff28c0ceb40 0x7ff28c156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.783+0000 7ff28b7fe640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff28c0cf340 0x7ff28c06b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:04.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff290934640 1 -- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff28c0cf340 msgr2=0x7ff28c06b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff290934640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff28c0cf340 0x7ff28c06b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff290934640 1 -- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff28c0ceb40 msgr2=0x7ff28c156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff290934640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff28c0ceb40 0x7ff28c156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff290934640 1 -- 172.21.15.125:0/1094198950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284009900 con 0x7ff28c15b2a0 2023-10-18T10:44:04.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff290934640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c15b2a0 0x7ff28c06b780 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff284009d60 tx=0x7ff284035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:04.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff288ff9640 1 -- 172.21.15.125:0/1094198950 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284034440 con 0x7ff28c15b2a0 2023-10-18T10:44:04.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff288ff9640 1 -- 172.21.15.125:0/1094198950 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff284004540 con 0x7ff28c15b2a0 2023-10-18T10:44:04.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff288ff9640 1 -- 172.21.15.125:0/1094198950 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284003e60 con 0x7ff28c15b2a0 2023-10-18T10:44:04.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28c0c7ed0 con 0x7ff28c15b2a0 2023-10-18T10:44:04.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.784+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c0c8490 con 0x7ff28c15b2a0 2023-10-18T10:44:04.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.785+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28c0c86b0 con 0x7ff28c15b2a0 2023-10-18T10:44:04.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.786+0000 7ff288ff9640 1 -- 172.21.15.125:0/1094198950 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7ff28400a040 con 0x7ff28c15b2a0 2023-10-18T10:44:04.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.786+0000 7ff288ff9640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff27806b080 0x7ff27806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:04.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.786+0000 7ff288ff9640 1 -- 172.21.15.125:0/1094198950 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7ff2840acc70 con 0x7ff28c15b2a0 2023-10-18T10:44:04.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.786+0000 7ff28bfff640 1 -- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff27806b080 msgr2=0x7ff27806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:04.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.786+0000 7ff28bfff640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff27806b080 0x7ff27806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:04.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.790+0000 7ff288ff9640 1 -- 172.21.15.125:0/1094198950 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2840778f0 con 0x7ff28c15b2a0 2023-10-18T10:44:04.823 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:04.821+0000 7fd9276b2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:44:04.838 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:04.834+0000 7fa4ec64c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:44:04.877 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:04.875+0000 7fa4ec64c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:44:04.877 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:04.875+0000 7fd9276b2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:44:04.920 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:04.918+0000 7fd9276b2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:44:04.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.964+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff28c0690e0 con 0x7ff28c15b2a0 2023-10-18T10:44:04.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.964+0000 7ff288ff9640 1 -- 172.21.15.125:0/1094198950 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff284039020 con 0x7ff28c15b2a0 2023-10-18T10:44:04.967 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:44:04.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff27806b080 msgr2=0x7ff27806d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:04.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff27806b080 0x7ff27806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c15b2a0 msgr2=0x7ff28c06b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:04.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c15b2a0 0x7ff28c06b780 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff284009d60 tx=0x7ff284035d80 comp rx=0 tx=0).stop 2023-10-18T10:44:04.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 shutdown_connections 2023-10-18T10:44:04.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff27806b080 0x7ff27806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff28c15b2a0 0x7ff28c06b780 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff28c0cf340 0x7ff28c06b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 --2- 172.21.15.125:0/1094198950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff28c0ceb40 0x7ff28c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:04.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 >> 172.21.15.125:0/1094198950 conn(0x7ff28c0d0d60 msgr2=0x7ff28c0ae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:04.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 shutdown_connections 2023-10-18T10:44:04.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:04.966+0000 7ff292397640 1 -- 172.21.15.125:0/1094198950 wait complete. 2023-10-18T10:44:04.974 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:04.972+0000 7fa4ec64c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:44:04.979 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:44:05.023 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.021+0000 7fd9276b2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:44:05.026 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.025+0000 7fa4ec64c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:05.068 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.066+0000 7fa4ec64c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:44:05.078 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.077+0000 7fd9276b2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:05.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.084+0000 7f0fff27c640 1 -- 172.21.15.125:0/1725745081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff806d260 msgr2=0x7f0ff815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:05.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.084+0000 7f0fff27c640 1 --2- 172.21.15.125:0/1725745081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff806d260 0x7f0ff815d400 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005fd0 tx=0x7f0ff002c250 comp rx=0 tx=0).stop 2023-10-18T10:44:05.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.084+0000 7f0fff27c640 1 -- 172.21.15.125:0/1725745081 shutdown_connections 2023-10-18T10:44:05.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.084+0000 7f0fff27c640 1 --2- 172.21.15.125:0/1725745081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff806d260 0x7f0ff815d400 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.084+0000 7f0fff27c640 1 -- 172.21.15.125:0/1725745081 >> 172.21.15.125:0/1725745081 conn(0x7f0ff80b0670 msgr2=0x7f0ff80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:05.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.085+0000 7f0fff27c640 1 -- 172.21.15.125:0/1725745081 shutdown_connections 2023-10-18T10:44:05.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.085+0000 7f0fff27c640 1 -- 172.21.15.125:0/1725745081 wait complete. 2023-10-18T10:44:05.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0fff27c640 1 Processor -- start 2023-10-18T10:44:05.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0fff27c640 1 -- start start 2023-10-18T10:44:05.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0fff27c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ff806d260 0x7f0ff80c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:05.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0fff27c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff80ce240 0x7f0ff806a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:05.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0fff27c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff8159060 0x7f0ff80b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:05.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0fff27c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80b38a0 con 0x7f0ff8159060 2023-10-18T10:44:05.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0fff27c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80c7c00 con 0x7f0ff80ce240 2023-10-18T10:44:05.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0fff27c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80c78d0 con 0x7f0ff806d260 2023-10-18T10:44:05.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.086+0000 7f0ffd819640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff8159060 0x7f0ff80b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:05.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd819640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff8159060 0x7f0ff80b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44532/0 (socket says 172.21.15.125:44532) 2023-10-18T10:44:05.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd819640 1 -- 172.21.15.125:0/905952028 learned_addr learned my addr 172.21.15.125:0/905952028 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:05.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffc817640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff80ce240 0x7f0ff806a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:05.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd018640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ff806d260 0x7f0ff80c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:05.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd819640 1 -- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ff806d260 msgr2=0x7f0ff80c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:05.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd819640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ff806d260 0x7f0ff80c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd819640 1 -- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff80ce240 msgr2=0x7f0ff806a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd819640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff80ce240 0x7f0ff806a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd819640 1 -- 172.21.15.125:0/905952028 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009900 con 0x7f0ff8159060 2023-10-18T10:44:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd018640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ff806d260 0x7f0ff80c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:05.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0ffd819640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff8159060 0x7f0ff80b27b0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009bf0 tx=0x7f0ff0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:05.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0fedffb640 1 -- 172.21.15.125:0/905952028 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0004450 con 0x7f0ff8159060 2023-10-18T10:44:05.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0fedffb640 1 -- 172.21.15.125:0/905952028 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ff00046a0 con 0x7f0ff8159060 2023-10-18T10:44:05.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0fedffb640 1 -- 172.21.15.125:0/905952028 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00406b0 con 0x7f0ff8159060 2023-10-18T10:44:05.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff80b3000 con 0x7f0ff8159060 2023-10-18T10:44:05.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.087+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff80b35c0 con 0x7f0ff8159060 2023-10-18T10:44:05.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.088+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff80c7db0 con 0x7f0ff8159060 2023-10-18T10:44:05.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.089+0000 7f0fedffb640 1 -- 172.21.15.125:0/905952028 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f0ff000a040 con 0x7f0ff8159060 2023-10-18T10:44:05.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.089+0000 7f0fedffb640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0fdc06b080 0x7f0fdc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:05.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.089+0000 7f0fedffb640 1 -- 172.21.15.125:0/905952028 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f0ff00acf00 con 0x7f0ff8159060 2023-10-18T10:44:05.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.089+0000 7f0ffd018640 1 -- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0fdc06b080 msgr2=0x7f0fdc06d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:05.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.089+0000 7f0ffd018640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0fdc06b080 0x7f0fdc06d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:05.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.093+0000 7f0fedffb640 1 -- 172.21.15.125:0/905952028 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ff80c7db0 con 0x7f0ff8159060 2023-10-18T10:44:05.115 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.114+0000 7fa4ec64c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:44:05.123 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.122+0000 7fd9276b2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:44:05.158 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.157+0000 7fa4ec64c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:44:05.174 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.172+0000 7fd9276b2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:44:05.218 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.216+0000 7fd9276b2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:44:05.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.267+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ff80c7db0 con 0x7f0ff8159060 2023-10-18T10:44:05.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.268+0000 7f0fedffb640 1 -- 172.21.15.125:0/905952028 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0ff80c7db0 con 0x7f0ff8159060 2023-10-18T10:44:05.270 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:44:05.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0fdc06b080 msgr2=0x7f0fdc06d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:05.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0fdc06b080 0x7f0fdc06d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff8159060 msgr2=0x7f0ff80b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:05.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff8159060 0x7f0ff80b27b0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009bf0 tx=0x7f0ff0003c20 comp rx=0 tx=0).stop 2023-10-18T10:44:05.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 shutdown_connections 2023-10-18T10:44:05.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0fdc06b080 0x7f0fdc06d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff80ce240 0x7f0ff806a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ff8159060 0x7f0ff80b27b0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 --2- 172.21.15.125:0/905952028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ff806d260 0x7f0ff80c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 >> 172.21.15.125:0/905952028 conn(0x7f0ff80b0670 msgr2=0x7f0ff815d160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:05.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 shutdown_connections 2023-10-18T10:44:05.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.270+0000 7f0fff27c640 1 -- 172.21.15.125:0/905952028 wait complete. 2023-10-18T10:44:05.283 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:44:05.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.426+0000 7ff4933a9640 1 -- 172.21.15.125:0/3124137690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c15b2a0 msgr2=0x7ff48c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:05.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.426+0000 7ff4933a9640 1 --2- 172.21.15.125:0/3124137690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c15b2a0 0x7ff48c0d1910 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff480009830 tx=0x7ff48002bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:05.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.426+0000 7ff4933a9640 1 -- 172.21.15.125:0/3124137690 shutdown_connections 2023-10-18T10:44:05.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.426+0000 7ff4933a9640 1 --2- 172.21.15.125:0/3124137690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c15b2a0 0x7ff48c0d1910 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.426+0000 7ff4933a9640 1 -- 172.21.15.125:0/3124137690 >> 172.21.15.125:0/3124137690 conn(0x7ff48c0d0d60 msgr2=0x7ff48c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:05.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.426+0000 7ff4933a9640 1 -- 172.21.15.125:0/3124137690 shutdown_connections 2023-10-18T10:44:05.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.426+0000 7ff4933a9640 1 -- 172.21.15.125:0/3124137690 wait complete. 2023-10-18T10:44:05.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.427+0000 7ff4933a9640 1 Processor -- start 2023-10-18T10:44:05.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.427+0000 7ff4933a9640 1 -- start start 2023-10-18T10:44:05.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff4933a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c0ceb40 0x7ff48c153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:05.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff4933a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff48c0cf340 0x7ff48c0b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:05.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff4933a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c15b2a0 0x7ff48c154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:05.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff4933a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0c9900 con 0x7ff48c0ceb40 2023-10-18T10:44:05.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff4933a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0b3950 con 0x7ff48c15b2a0 2023-10-18T10:44:05.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff4933a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0b4ea0 con 0x7ff48c0cf340 2023-10-18T10:44:05.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff491145640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c0ceb40 0x7ff48c153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:05.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff491145640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c0ceb40 0x7ff48c153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44552/0 (socket says 172.21.15.125:44552) 2023-10-18T10:44:05.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff491145640 1 -- 172.21.15.125:0/2792727126 learned_addr learned my addr 172.21.15.125:0/2792727126 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:05.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff491946640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c15b2a0 0x7ff48c154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:05.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.428+0000 7ff490944640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff48c0cf340 0x7ff48c0b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:05.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff491145640 1 -- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff48c0cf340 msgr2=0x7ff48c0b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:05.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff491145640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff48c0cf340 0x7ff48c0b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff491145640 1 -- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c15b2a0 msgr2=0x7ff48c154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:05.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff491145640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c15b2a0 0x7ff48c154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff491145640 1 -- 172.21.15.125:0/2792727126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480009490 con 0x7ff48c0ceb40 2023-10-18T10:44:05.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff491145640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c0ceb40 0x7ff48c153de0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff48800c450 tx=0x7ff48800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:05.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff47dffb640 1 -- 172.21.15.125:0/2792727126 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880052d0 con 0x7ff48c0ceb40 2023-10-18T10:44:05.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff47dffb640 1 -- 172.21.15.125:0/2792727126 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff488005430 con 0x7ff48c0ceb40 2023-10-18T10:44:05.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c0b3ba0 con 0x7ff48c0ceb40 2023-10-18T10:44:05.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff47dffb640 1 -- 172.21.15.125:0/2792727126 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488012890 con 0x7ff48c0ceb40 2023-10-18T10:44:05.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.429+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c0b4160 con 0x7ff48c0ceb40 2023-10-18T10:44:05.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.430+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c0cb270 con 0x7ff48c0ceb40 2023-10-18T10:44:05.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.431+0000 7ff47dffb640 1 -- 172.21.15.125:0/2792727126 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7ff4880129f0 con 0x7ff48c0ceb40 2023-10-18T10:44:05.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.431+0000 7ff47dffb640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff47006b0b0 0x7ff47006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:05.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.431+0000 7ff490944640 1 -- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff47006b0b0 msgr2=0x7ff47006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:05.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.432+0000 7ff47dffb640 1 -- 172.21.15.125:0/2792727126 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7ff488089770 con 0x7ff48c0ceb40 2023-10-18T10:44:05.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.432+0000 7ff490944640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff47006b0b0 0x7ff47006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:05.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.436+0000 7ff47dffb640 1 -- 172.21.15.125:0/2792727126 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff488054360 con 0x7ff48c0ceb40 2023-10-18T10:44:05.475 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.473+0000 7fa4ec64c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:44:05.550 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.549+0000 7fd9276b2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:44:05.604 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.603+0000 7fa4ec64c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:44:05.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.621+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff48c0690e0 con 0x7ff48c0ceb40 2023-10-18T10:44:05.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.621+0000 7ff47dffb640 1 -- 172.21.15.125:0/2792727126 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v129) v1 ==== 80+0+474 (secure 0 0 0) 0x7ff488057980 con 0x7ff48c0ceb40 2023-10-18T10:44:05.624 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2083342052 2023-10-19T10:43:18.775528+0000 2023-10-18T10:44:05.624 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1711146434 2023-10-19T10:43:18.775528+0000 2023-10-18T10:44:05.625 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/1511464628 2023-10-19T10:43:18.775528+0000 2023-10-18T10:44:05.625 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/1511464628 2023-10-19T10:43:18.775528+0000 2023-10-18T10:44:05.625 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3838090415 2023-10-19T10:43:18.775528+0000 2023-10-18T10:44:05.625 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/214738857 2023-10-19T10:43:18.775528+0000 2023-10-18T10:44:05.625 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/188703397 2023-10-19T10:43:18.775528+0000 2023-10-18T10:44:05.626 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/84406598 2023-10-19T10:43:18.775528+0000 2023-10-18T10:44:05.627 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T10:44:05.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff47006b0b0 msgr2=0x7ff47006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:05.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff47006b0b0 0x7ff47006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c0ceb40 msgr2=0x7ff48c153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:05.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c0ceb40 0x7ff48c153de0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff48800c450 tx=0x7ff48800c910 comp rx=0 tx=0).stop 2023-10-18T10:44:05.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 shutdown_connections 2023-10-18T10:44:05.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7ff47006b0b0 0x7ff47006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c15b2a0 0x7ff48c154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff48c0cf340 0x7ff48c0b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 --2- 172.21.15.125:0/2792727126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff48c0ceb40 0x7ff48c153de0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:05.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 >> 172.21.15.125:0/2792727126 conn(0x7ff48c0d0d60 msgr2=0x7ff48c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:05.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 shutdown_connections 2023-10-18T10:44:05.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:05.624+0000 7ff4933a9640 1 -- 172.21.15.125:0/2792727126 wait complete. 2023-10-18T10:44:05.636 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:44:05.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:05.637 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T10:44:05.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[129036]: fuse finished with error 0 and tester_r 0 2023-10-18T10:44:05.664 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:44:05.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:05.665 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:44:05.688 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.686+0000 7fd9276b2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:44:05.695 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.694+0000 7fa4ec64c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:44:05.737 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.736+0000 7fa4ec64c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:44:05.782 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.781+0000 7fd9276b2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:44:05.808 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:44:05.808 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:05.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:44:05.825 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:44:05.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:05.825 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:44:05.826 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.824+0000 7fd9276b2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:44:05.834 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.832+0000 7fa4ec64c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:05.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:05.848 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T10:44:05.848 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:44:05.849 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:05.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:44:05.864 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:44:05.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:05.865 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:44:05.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T10:44:05.920 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:44:05.921 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:05.925 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:05.924+0000 7fd9276b2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:05.928 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:05.927+0000 7fa4ec64c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:44:05.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:44:05.939 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:44:05.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:05.940 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:44:05.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:05.994 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T10:44:05.994 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:06.018 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:06.017+0000 7fd9276b2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:44:06.070 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:06.068+0000 7fa4ec64c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:06.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.097+0000 7f395b464640 1 -- 172.21.15.125:0/3006553814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f395415b2a0 msgr2=0x7f39540cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.097+0000 7f395b464640 1 --2- 172.21.15.125:0/3006553814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f395415b2a0 0x7f39540cf720 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3944009830 tx=0x7f394402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.097+0000 7f395b464640 1 -- 172.21.15.125:0/3006553814 shutdown_connections 2023-10-18T10:44:06.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.097+0000 7f395b464640 1 --2- 172.21.15.125:0/3006553814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f395415b2a0 0x7f39540cf720 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.097+0000 7f395b464640 1 --2- 172.21.15.125:0/3006553814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39540cf340 0x7f39540d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.097+0000 7f395b464640 1 -- 172.21.15.125:0/3006553814 >> 172.21.15.125:0/3006553814 conn(0x7f39540d0d60 msgr2=0x7f39540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:06.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.097+0000 7f395b464640 1 -- 172.21.15.125:0/3006553814 shutdown_connections 2023-10-18T10:44:06.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.097+0000 7f395b464640 1 -- 172.21.15.125:0/3006553814 wait complete. 2023-10-18T10:44:06.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.098+0000 7f395b464640 1 Processor -- start 2023-10-18T10:44:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f395b464640 1 -- start start 2023-10-18T10:44:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f395b464640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39540ceb40 0x7f39541f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f395b464640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540cf340 0x7f39541e4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f395b464640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395415b2a0 0x7f395415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f395b464640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b4510 con 0x7f39540ceb40 2023-10-18T10:44:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f395b464640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b3f20 con 0x7f39540cf340 2023-10-18T10:44:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f395b464640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39541e52a0 con 0x7f395415b2a0 2023-10-18T10:44:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f3959200640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39540ceb40 0x7f39541f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f3959200640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39540ceb40 0x7f39541f0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44568/0 (socket says 172.21.15.125:44568) 2023-10-18T10:44:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f3959200640 1 -- 172.21.15.125:0/2126215333 learned_addr learned my addr 172.21.15.125:0/2126215333 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f3959a01640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395415b2a0 0x7f395415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.099+0000 7f39589ff640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540cf340 0x7f39541e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3959200640 1 -- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395415b2a0 msgr2=0x7f395415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3959200640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395415b2a0 0x7f395415c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3959200640 1 -- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540cf340 msgr2=0x7f39541e4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3959200640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540cf340 0x7f39541e4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3959200640 1 -- 172.21.15.125:0/2126215333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3950009560 con 0x7f39540ceb40 2023-10-18T10:44:06.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3959200640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39540ceb40 0x7f39541f0900 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f395000e9f0 tx=0x7f395000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:06.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3949ffb640 1 -- 172.21.15.125:0/2126215333 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395000cde0 con 0x7f39540ceb40 2023-10-18T10:44:06.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3949ffb640 1 -- 172.21.15.125:0/2126215333 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3950004020 con 0x7f39540ceb40 2023-10-18T10:44:06.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3944009490 con 0x7f39540ceb40 2023-10-18T10:44:06.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f3949ffb640 1 -- 172.21.15.125:0/2126215333 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950004410 con 0x7f39540ceb40 2023-10-18T10:44:06.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.100+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39541e4fc0 con 0x7f39540ceb40 2023-10-18T10:44:06.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.101+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39540690e0 con 0x7f39540ceb40 2023-10-18T10:44:06.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.102+0000 7f3949ffb640 1 -- 172.21.15.125:0/2126215333 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f39500035f0 con 0x7f39540ceb40 2023-10-18T10:44:06.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.102+0000 7f3949ffb640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f393806b080 0x7f393806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.102+0000 7f3949ffb640 1 -- 172.21.15.125:0/2126215333 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f3950014070 con 0x7f39540ceb40 2023-10-18T10:44:06.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.102+0000 7f39589ff640 1 -- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f393806b080 msgr2=0x7f393806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:06.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.102+0000 7f39589ff640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f393806b080 0x7f393806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:06.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.106+0000 7f3949ffb640 1 -- 172.21.15.125:0/2126215333 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3950055340 con 0x7f39540ceb40 2023-10-18T10:44:06.113 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:06.111+0000 7fa4ec64c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:06.164 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:06.161+0000 7fd9276b2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:06.207 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:06.205+0000 7fd9276b2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:06.221 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:06.215+0000 7fa4ec64c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:44:06.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.303+0000 7f39589ff640 1 -- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f393806b080 msgr2=0x7f393806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:06.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.303+0000 7f39589ff640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f393806b080 0x7f393806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:44:06.312 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:06.311+0000 7fd9276b2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:44:06.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.312+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39540d0b40 con 0x7f39540ceb40 2023-10-18T10:44:06.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.313+0000 7f3949ffb640 1 -- 172.21.15.125:0/2126215333 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f3950058960 con 0x7f39540ceb40 2023-10-18T10:44:06.315 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:06.316 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":47,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/4122761343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4122761343},{"type":"v1","addr":"172.21.15.139:6836","nonce":4122761343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6785,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1014994333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1014994333},{"type":"v1","addr":"172.21.15.139:6837","nonce":1014994333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3090232356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3090232356},{"type":"v1","addr":"172.21.15.125:6834","nonce":3090232356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:43:03.299305+0000","modified":"2023-10-18T10:43:05.301917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8613":{"gid":8613,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2199142702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2199142702},{"type":"v1","addr":"172.21.15.125:6835","nonce":2199142702}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T10:44:06.318 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 47 2023-10-18T10:44:06.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f393806b080 msgr2=0x7f393806d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:06.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f393806b080 0x7f393806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39540ceb40 msgr2=0x7f39541f0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39540ceb40 0x7f39541f0900 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f395000e9f0 tx=0x7f395000eeb0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 shutdown_connections 2023-10-18T10:44:06.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f393806b080 0x7f393806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540cf340 0x7f39541e4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395415b2a0 0x7f395415c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 --2- 172.21.15.125:0/2126215333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39540ceb40 0x7f39541f0900 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 >> 172.21.15.125:0/2126215333 conn(0x7f39540d0d60 msgr2=0x7f39540b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:06.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 shutdown_connections 2023-10-18T10:44:06.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.315+0000 7f395b464640 1 -- 172.21.15.125:0/2126215333 wait complete. 2023-10-18T10:44:06.324 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:44:06.327 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:44:06.328 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:06.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 -- 172.21.15.125:0/1378956598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a40ceb40 msgr2=0x7f71a40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 --2- 172.21.15.125:0/1378956598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a40ceb40 0x7f71a40b0650 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f71a406c510 tx=0x7f719c02c1d0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 -- 172.21.15.125:0/1378956598 shutdown_connections 2023-10-18T10:44:06.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 --2- 172.21.15.125:0/1378956598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a40ceb40 0x7f71a40b0650 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 --2- 172.21.15.125:0/1378956598 >> v2:172.21.15.139:3300/0 conn(0x7f71a415b2a0 0x7f71a40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 --2- 172.21.15.125:0/1378956598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71a40cf340 0x7f71a40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 -- 172.21.15.125:0/1378956598 >> 172.21.15.125:0/1378956598 conn(0x7f71a40d0d60 msgr2=0x7f71a40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:06.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 -- 172.21.15.125:0/1378956598 shutdown_connections 2023-10-18T10:44:06.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.471+0000 7f71a918a640 1 -- 172.21.15.125:0/1378956598 wait complete. 2023-10-18T10:44:06.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.472+0000 7f71a918a640 1 Processor -- start 2023-10-18T10:44:06.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.472+0000 7f71a918a640 1 -- start start 2023-10-18T10:44:06.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a918a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71a40ceb40 0x7f71a40c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a918a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71a40cf340 0x7f71a40b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a918a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a415b2a0 0x7f71a415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a918a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40b4360 con 0x7f71a415b2a0 2023-10-18T10:44:06.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a918a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a414afa0 con 0x7f71a40ceb40 2023-10-18T10:44:06.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a918a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a414b280 con 0x7f71a40cf340 2023-10-18T10:44:06.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a415b2a0 0x7f71a415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a415b2a0 0x7f71a415d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44584/0 (socket says 172.21.15.125:44584) 2023-10-18T10:44:06.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 -- 172.21.15.125:0/2176574049 learned_addr learned my addr 172.21.15.125:0/2176574049 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:06.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a2d9d640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71a40ceb40 0x7f71a40c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a259c640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71a40cf340 0x7f71a40b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 -- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71a40cf340 msgr2=0x7f71a40b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71a40cf340 0x7f71a40b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 -- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71a40ceb40 msgr2=0x7f71a40c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71a40ceb40 0x7f71a40c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 -- 172.21.15.125:0/2176574049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c009900 con 0x7f71a415b2a0 2023-10-18T10:44:06.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.473+0000 7f71a359e640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a415b2a0 0x7f71a415d910 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f719c009d60 tx=0x7f719c035d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:06.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.474+0000 7f718b7fe640 1 -- 172.21.15.125:0/2176574049 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c034440 con 0x7f71a415b2a0 2023-10-18T10:44:06.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.474+0000 7f718b7fe640 1 -- 172.21.15.125:0/2176574049 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f719c004540 con 0x7f71a415b2a0 2023-10-18T10:44:06.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.474+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a40b6380 con 0x7f71a415b2a0 2023-10-18T10:44:06.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.474+0000 7f718b7fe640 1 -- 172.21.15.125:0/2176574049 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c003e60 con 0x7f71a415b2a0 2023-10-18T10:44:06.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.474+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a40b6940 con 0x7f71a415b2a0 2023-10-18T10:44:06.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.475+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a414b410 con 0x7f71a415b2a0 2023-10-18T10:44:06.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.475+0000 7f718b7fe640 1 -- 172.21.15.125:0/2176574049 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f719c03b040 con 0x7f71a415b2a0 2023-10-18T10:44:06.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.476+0000 7f718b7fe640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f719006b080 0x7f719006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.476+0000 7f71a2d9d640 1 -- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f719006b080 msgr2=0x7f719006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:06.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.476+0000 7f718b7fe640 1 -- 172.21.15.125:0/2176574049 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f719c0ac670 con 0x7f71a415b2a0 2023-10-18T10:44:06.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.476+0000 7f71a2d9d640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f719006b080 0x7f719006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:06.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.480+0000 7f718b7fe640 1 -- 172.21.15.125:0/2176574049 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f719c0771c0 con 0x7f71a415b2a0 2023-10-18T10:44:06.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.649+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71a40d04f0 con 0x7f71a415b2a0 2023-10-18T10:44:06.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.651+0000 7f718b7fe640 1 -- 172.21.15.125:0/2176574049 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+17269 (secure 0 0 0) 0x7f719c07a7e0 con 0x7f71a415b2a0 2023-10-18T10:44:06.654 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:06.654 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":129,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:43:18.775585+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:43:01.020853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-18T10:43:02.278684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2083342052":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/1711146434":"2023-10-19T10:43:18.775528+0000","172.21.15.139:6833/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:6832/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/3838090415":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/214738857":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/188703397":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/84406598":"2023-10-19T10:43:18.775528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:06.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f719006b080 msgr2=0x7f719006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:06.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f719006b080 0x7f719006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a415b2a0 msgr2=0x7f71a415d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a415b2a0 0x7f71a415d910 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f719c009d60 tx=0x7f719c035d00 comp rx=0 tx=0).stop 2023-10-18T10:44:06.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 shutdown_connections 2023-10-18T10:44:06.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f719006b080 0x7f719006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71a415b2a0 0x7f71a415d910 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71a40cf340 0x7f71a40b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 --2- 172.21.15.125:0/2176574049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71a40ceb40 0x7f71a40c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 >> 172.21.15.125:0/2176574049 conn(0x7f71a40d0d60 msgr2=0x7f71a415a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:06.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 shutdown_connections 2023-10-18T10:44:06.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.653+0000 7f71a918a640 1 -- 172.21.15.125:0/2176574049 wait complete. 2023-10-18T10:44:06.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:43:01.020853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:43:02.278684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:44:06.667 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:44:06.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.776+0000 7f0d38525640 1 -- 172.21.15.125:0/1779582341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30141690 msgr2=0x7f0d300703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.776+0000 7f0d38525640 1 --2- 172.21.15.125:0/1779582341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30141690 0x7f0d300703e0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005fd0 tx=0x7f0d2002bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:06.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.776+0000 7f0d38525640 1 -- 172.21.15.125:0/1779582341 shutdown_connections 2023-10-18T10:44:06.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.776+0000 7f0d38525640 1 --2- 172.21.15.125:0/1779582341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d30070920 0x7f0d300affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.776+0000 7f0d38525640 1 --2- 172.21.15.125:0/1779582341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30141690 0x7f0d300703e0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.776+0000 7f0d38525640 1 -- 172.21.15.125:0/1779582341 >> 172.21.15.125:0/1779582341 conn(0x7f0d30069930 msgr2=0x7f0d30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:06.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.776+0000 7f0d38525640 1 -- 172.21.15.125:0/1779582341 shutdown_connections 2023-10-18T10:44:06.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.776+0000 7f0d38525640 1 -- 172.21.15.125:0/1779582341 wait complete. 2023-10-18T10:44:06.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.777+0000 7f0d38525640 1 Processor -- start 2023-10-18T10:44:06.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.777+0000 7f0d38525640 1 -- start start 2023-10-18T10:44:06.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d38525640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30070920 0x7f0d301e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d38525640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d3013e520 0x7f0d3006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d38525640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d30141690 0x7f0d3006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d38525640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301e4470 con 0x7f0d30070920 2023-10-18T10:44:06.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d38525640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301e2e90 con 0x7f0d3013e520 2023-10-18T10:44:06.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d38525640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301e3170 con 0x7f0d30141690 2023-10-18T10:44:06.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d362c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30070920 0x7f0d301e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d362c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30070920 0x7f0d301e3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44602/0 (socket says 172.21.15.125:44602) 2023-10-18T10:44:06.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d362c1640 1 -- 172.21.15.125:0/3886551683 learned_addr learned my addr 172.21.15.125:0/3886551683 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:06.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d35ac0640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d3013e520 0x7f0d3006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.778+0000 7f0d36ac2640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d30141690 0x7f0d3006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:06.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d362c1640 1 -- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d30141690 msgr2=0x7f0d3006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d362c1640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d30141690 0x7f0d3006f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d362c1640 1 -- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d3013e520 msgr2=0x7f0d3006f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d362c1640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d3013e520 0x7f0d3006f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d362c1640 1 -- 172.21.15.125:0/3886551683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20009560 con 0x7f0d30070920 2023-10-18T10:44:06.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d35ac0640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d3013e520 0x7f0d3006f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:44:06.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d362c1640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30070920 0x7f0d301e3f30 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800c400 tx=0x7f0d2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:06.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d26ffd640 1 -- 172.21.15.125:0/3886551683 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d280052d0 con 0x7f0d30070920 2023-10-18T10:44:06.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d26ffd640 1 -- 172.21.15.125:0/3886551683 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d28007020 con 0x7f0d30070920 2023-10-18T10:44:06.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d301e3450 con 0x7f0d30070920 2023-10-18T10:44:06.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d26ffd640 1 -- 172.21.15.125:0/3886551683 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28007470 con 0x7f0d30070920 2023-10-18T10:44:06.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.779+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d301e3a10 con 0x7f0d30070920 2023-10-18T10:44:06.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.780+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d300b0fc0 con 0x7f0d30070920 2023-10-18T10:44:06.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.781+0000 7f0d26ffd640 1 -- 172.21.15.125:0/3886551683 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f0d280067d0 con 0x7f0d30070920 2023-10-18T10:44:06.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.782+0000 7f0d26ffd640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0d1406b080 0x7f0d1406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:06.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.782+0000 7f0d35ac0640 1 -- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0d1406b080 msgr2=0x7f0d1406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:06.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.782+0000 7f0d26ffd640 1 -- 172.21.15.125:0/3886551683 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f0d28089320 con 0x7f0d30070920 2023-10-18T10:44:06.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.782+0000 7f0d35ac0640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0d1406b080 0x7f0d1406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:06.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.790+0000 7f0d26ffd640 1 -- 172.21.15.125:0/3886551683 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d28053e30 con 0x7f0d30070920 2023-10-18T10:44:06.837 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:06.835+0000 7fa4ec64c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:44:06.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.960+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d30067930 con 0x7f0d30070920 2023-10-18T10:44:06.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.960+0000 7f0d26ffd640 1 -- 172.21.15.125:0/3886551683 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d28057450 con 0x7f0d30070920 2023-10-18T10:44:06.963 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:44:06.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0d1406b080 msgr2=0x7f0d1406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:06.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0d1406b080 0x7f0d1406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30070920 msgr2=0x7f0d301e3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:06.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30070920 0x7f0d301e3f30 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800c400 tx=0x7f0d2800c8c0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 shutdown_connections 2023-10-18T10:44:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7f0d1406b080 0x7f0d1406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d3013e520 0x7f0d3006f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d30141690 0x7f0d3006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 --2- 172.21.15.125:0/3886551683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d30070920 0x7f0d301e3f30 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 >> 172.21.15.125:0/3886551683 conn(0x7f0d30069930 msgr2=0x7f0d30069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 shutdown_connections 2023-10-18T10:44:06.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:06.962+0000 7f0d38525640 1 -- 172.21.15.125:0/3886551683 wait complete. 2023-10-18T10:44:06.976 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:44:06.977 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:06.974+0000 7fd9276b2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:44:07.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 -- 172.21.15.125:0/116917638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc11815ccf0 msgr2=0x7fc11815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/116917638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc11815ccf0 0x7fc11815d180 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fc1180714a0 tx=0x7fc11002c1d0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 -- 172.21.15.125:0/116917638 shutdown_connections 2023-10-18T10:44:07.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/116917638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc11815ccf0 0x7fc11815d180 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/116917638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1180cdb30 0x7fc1180cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/116917638 >> v2:172.21.15.139:3300/0 conn(0x7fc118159060 0x7fc11806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 -- 172.21.15.125:0/116917638 >> 172.21.15.125:0/116917638 conn(0x7fc1180b0670 msgr2=0x7fc1180b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:07.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 -- 172.21.15.125:0/116917638 shutdown_connections 2023-10-18T10:44:07.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.081+0000 7fc11d4b4640 1 -- 172.21.15.125:0/116917638 wait complete. 2023-10-18T10:44:07.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.082+0000 7fc11d4b4640 1 Processor -- start 2023-10-18T10:44:07.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.082+0000 7fc11d4b4640 1 -- start start 2023-10-18T10:44:07.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc11d4b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1180cdb30 0x7fc118157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc11d4b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc118159060 0x7fc1181e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc11d4b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11815ccf0 0x7fc1180c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc11d4b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118157e60 con 0x7fc1180cdb30 2023-10-18T10:44:07.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc11d4b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181e4210 con 0x7fc11815ccf0 2023-10-18T10:44:07.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc11d4b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181e44f0 con 0x7fc118159060 2023-10-18T10:44:07.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc116ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1180cdb30 0x7fc118157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc116ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1180cdb30 0x7fc118157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44616/0 (socket says 172.21.15.125:44616) 2023-10-18T10:44:07.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc116ffd640 1 -- 172.21.15.125:0/43082006 learned_addr learned my addr 172.21.15.125:0/43082006 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:07.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc10ffff640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc118159060 0x7fc1181e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc1177fe640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11815ccf0 0x7fc1180c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc116ffd640 1 -- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc118159060 msgr2=0x7fc1181e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc116ffd640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc118159060 0x7fc1181e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc116ffd640 1 -- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11815ccf0 msgr2=0x7fc1180c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc116ffd640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11815ccf0 0x7fc1180c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.083+0000 7fc116ffd640 1 -- 172.21.15.125:0/43082006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110009900 con 0x7fc1180cdb30 2023-10-18T10:44:07.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.084+0000 7fc116ffd640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1180cdb30 0x7fc118157920 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e9f0 tx=0x7fc10800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:07.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.084+0000 7fc10f7fe640 1 -- 172.21.15.125:0/43082006 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108009700 con 0x7fc1180cdb30 2023-10-18T10:44:07.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.084+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1181e47d0 con 0x7fc1180cdb30 2023-10-18T10:44:07.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.084+0000 7fc10f7fe640 1 -- 172.21.15.125:0/43082006 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc10800ce70 con 0x7fc1180cdb30 2023-10-18T10:44:07.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.084+0000 7fc10f7fe640 1 -- 172.21.15.125:0/43082006 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108010670 con 0x7fc1180cdb30 2023-10-18T10:44:07.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.084+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1181e4d90 con 0x7fc1180cdb30 2023-10-18T10:44:07.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.085+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1180675f0 con 0x7fc1180cdb30 2023-10-18T10:44:07.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.085+0000 7fc10f7fe640 1 -- 172.21.15.125:0/43082006 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 67) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fc1080107d0 con 0x7fc1180cdb30 2023-10-18T10:44:07.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.086+0000 7fc10f7fe640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fc0f806b080 0x7fc0f806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.086+0000 7fc10ffff640 1 -- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fc0f806b080 msgr2=0x7fc0f806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3564227248 2023-10-18T10:44:07.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.086+0000 7fc10f7fe640 1 -- 172.21.15.125:0/43082006 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6373+0+0 (secure 0 0 0) 0x7fc108014070 con 0x7fc1180cdb30 2023-10-18T10:44:07.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.086+0000 7fc10ffff640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fc0f806b080 0x7fc0f806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:07.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.090+0000 7fc10f7fe640 1 -- 172.21.15.125:0/43082006 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc108051ca0 con 0x7fc1180cdb30 2023-10-18T10:44:07.105 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.102+0000 7fa4ec64c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:44:07.152 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.150+0000 7fa4ec64c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:44:07.246 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:07.244+0000 7fd9276b2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:44:07.251 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.249+0000 7fa4ec64c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:44:07.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.256+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1180b4aa0 con 0x7fc1180cdb30 2023-10-18T10:44:07.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.257+0000 7fc10f7fe640 1 -- 172.21.15.125:0/43082006 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1080553d0 con 0x7fc1180cdb30 2023-10-18T10:44:07.259 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:44:07.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fc0f806b080 msgr2=0x7fc0f806d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:07.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fc0f806b080 0x7fc0f806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1180cdb30 msgr2=0x7fc118157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1180cdb30 0x7fc118157920 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e9f0 tx=0x7fc10800eeb0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 shutdown_connections 2023-10-18T10:44:07.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:6832/3564227248,v1:172.21.15.139:6833/3564227248] conn(0x7fc0f806b080 0x7fc0f806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11815ccf0 0x7fc1180c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc118159060 0x7fc1181e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 --2- 172.21.15.125:0/43082006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1180cdb30 0x7fc118157920 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 >> 172.21.15.125:0/43082006 conn(0x7fc1180b0670 msgr2=0x7fc1180cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:07.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 shutdown_connections 2023-10-18T10:44:07.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.259+0000 7fc11d4b4640 1 -- 172.21.15.125:0/43082006 wait complete. 2023-10-18T10:44:07.275 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:44:07.302 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:07.300+0000 7fd9276b2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:44:07.326 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa46e98c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa46e98c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa46e98c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa46e98c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa46e98c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa471992640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa471992640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa471992640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa471992640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:07.324+0000 7fa471992640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:07.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.380+0000 7f600ae32640 1 -- 172.21.15.125:0/1628703642 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040cf340 msgr2=0x7f60040d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.380+0000 7f600ae32640 1 --2- 172.21.15.125:0/1628703642 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040cf340 0x7f60040d1910 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f600408a120 tx=0x7f5ff802bdb0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.380+0000 7f600ae32640 1 -- 172.21.15.125:0/1628703642 shutdown_connections 2023-10-18T10:44:07.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.380+0000 7f600ae32640 1 --2- 172.21.15.125:0/1628703642 >> v2:172.21.15.139:3300/0 conn(0x7f600415b2a0 0x7f60040cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.381+0000 7f600ae32640 1 --2- 172.21.15.125:0/1628703642 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040cf340 0x7f60040d1910 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.381+0000 7f600ae32640 1 -- 172.21.15.125:0/1628703642 >> 172.21.15.125:0/1628703642 conn(0x7f60040d0d60 msgr2=0x7f60040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:07.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.381+0000 7f600ae32640 1 -- 172.21.15.125:0/1628703642 shutdown_connections 2023-10-18T10:44:07.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.381+0000 7f600ae32640 1 -- 172.21.15.125:0/1628703642 wait complete. 2023-10-18T10:44:07.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.382+0000 7f600ae32640 1 Processor -- start 2023-10-18T10:44:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.382+0000 7f600ae32640 1 -- start start 2023-10-18T10:44:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.382+0000 7f600ae32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040ceb40 0x7f60041f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f600ae32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60040cf340 0x7f60041580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f600ae32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f600415b2a0 0x7f600406b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f600ae32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60041f4ca0 con 0x7f60040ceb40 2023-10-18T10:44:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f600ae32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040b3b60 con 0x7f60040cf340 2023-10-18T10:44:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f600ae32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040b6500 con 0x7f600415b2a0 2023-10-18T10:44:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040ceb40 0x7f60041f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040ceb40 0x7f60041f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44632/0 (socket says 172.21.15.125:44632) 2023-10-18T10:44:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 -- 172.21.15.125:0/849560523 learned_addr learned my addr 172.21.15.125:0/849560523 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6003fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60040cf340 0x7f60041580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f60093cf640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f600415b2a0 0x7f600406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 -- 172.21.15.125:0/849560523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f600415b2a0 msgr2=0x7f600406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f600415b2a0 0x7f600406b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 -- 172.21.15.125:0/849560523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60040cf340 msgr2=0x7f60041580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60040cf340 0x7f60041580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 -- 172.21.15.125:0/849560523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8009560 con 0x7f60040ceb40 2023-10-18T10:44:07.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6003fff640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60040cf340 0x7f60041580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:44:07.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f6008bce640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040ceb40 0x7f60041f1ff0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80099c0 tx=0x7f5ff802da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:07.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f60017fa640 1 -- 172.21.15.125:0/849560523 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8003800 con 0x7f60040ceb40 2023-10-18T10:44:07.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f60017fa640 1 -- 172.21.15.125:0/849560523 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ff8003a50 con 0x7f60040ceb40 2023-10-18T10:44:07.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600406b540 con 0x7f60040ceb40 2023-10-18T10:44:07.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f60017fa640 1 -- 172.21.15.125:0/849560523 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8003e60 con 0x7f60040ceb40 2023-10-18T10:44:07.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.383+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60040b6a30 con 0x7f60040ceb40 2023-10-18T10:44:07.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.385+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60040690e0 con 0x7f60040ceb40 2023-10-18T10:44:07.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.385+0000 7f60017fa640 1 -- 172.21.15.125:0/849560523 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 68) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f5ff803e450 con 0x7f60040ceb40 2023-10-18T10:44:07.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.385+0000 7f60017fa640 1 -- 172.21.15.125:0/849560523 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f5ff80abf00 con 0x7f60040ceb40 2023-10-18T10:44:07.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.394+0000 7f60017fa640 1 -- 172.21.15.125:0/849560523 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ff8077060 con 0x7f60040ceb40 2023-10-18T10:44:07.412 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:07.410+0000 7fd9276b2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:44:07.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.562+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60040cb9d0 con 0x7f60040ceb40 2023-10-18T10:44:07.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.563+0000 7f60017fa640 1 -- 172.21.15.125:0/849560523 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ff8042020 con 0x7f60040ceb40 2023-10-18T10:44:07.565 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:44:07.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040ceb40 msgr2=0x7f60041f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040ceb40 0x7f60041f1ff0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80099c0 tx=0x7f5ff802da80 comp rx=0 tx=0).stop 2023-10-18T10:44:07.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 shutdown_connections 2023-10-18T10:44:07.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60040cf340 0x7f60041580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f600415b2a0 0x7f600406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 --2- 172.21.15.125:0/849560523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60040ceb40 0x7f60041f1ff0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 >> 172.21.15.125:0/849560523 conn(0x7f60040d0d60 msgr2=0x7f600415a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 shutdown_connections 2023-10-18T10:44:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.565+0000 7f600ae32640 1 -- 172.21.15.125:0/849560523 wait complete. 2023-10-18T10:44:07.577 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:07.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.718+0000 7f3f7087c640 1 -- 172.21.15.125:0/3288884277 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 msgr2=0x7f3f54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.718+0000 7f3f7087c640 1 --2- 172.21.15.125:0/3288884277 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 0x7f3f54004c40 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009b50 tx=0x7f3f540057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.718+0000 7f3f6ad9d640 1 -- 172.21.15.125:0/3288884277 reap_dead start 2023-10-18T10:44:07.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.718+0000 7f3f7087c640 1 -- 172.21.15.125:0/3288884277 shutdown_connections 2023-10-18T10:44:07.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.718+0000 7f3f7087c640 1 -- 172.21.15.125:0/3288884277 >> 172.21.15.125:0/3288884277 conn(0x7f3f6c069930 msgr2=0x7f3f6c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:07.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.718+0000 7f3f7087c640 1 -- 172.21.15.125:0/3288884277 shutdown_connections 2023-10-18T10:44:07.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.718+0000 7f3f7087c640 1 -- 172.21.15.125:0/3288884277 wait complete. 2023-10-18T10:44:07.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.719+0000 7f3f7087c640 1 Processor -- start 2023-10-18T10:44:07.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.719+0000 7f3f7087c640 1 -- start start 2023-10-18T10:44:07.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f7087c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 0x7f3f6c0abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f7087c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6c13ad60 0x7f3f6c13b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f7087c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f6c1dc780 0x7f3f6c1dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:07.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f7087c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c13b720 con 0x7f3f540047c0 2023-10-18T10:44:07.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f7087c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1dd5b0 con 0x7f3f6c1dc780 2023-10-18T10:44:07.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f7087c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1dd890 con 0x7f3f6c13ad60 2023-10-18T10:44:07.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 0x7f3f6c0abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f6a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 0x7f3f6c0abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36700/0 (socket says 172.21.15.125:36700) 2023-10-18T10:44:07.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f6a59c640 1 -- 172.21.15.125:0/1540819621 learned_addr learned my addr 172.21.15.125:0/1540819621 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:07.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f6ad9d640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f6c1dc780 0x7f3f6c1dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f69d9b640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6c13ad60 0x7f3f6c13b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:07.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f6a59c640 1 -- 172.21.15.125:0/1540819621 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6c13ad60 msgr2=0x7f3f6c13b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f6a59c640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6c13ad60 0x7f3f6c13b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f6a59c640 1 -- 172.21.15.125:0/1540819621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f6c1dc780 msgr2=0x7f3f6c1dcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.720+0000 7f3f6a59c640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f6c1dc780 0x7f3f6c1dcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.721+0000 7f3f6a59c640 1 -- 172.21.15.125:0/1540819621 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c1ddb70 con 0x7f3f540047c0 2023-10-18T10:44:07.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.721+0000 7f3f6a59c640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 0x7f3f6c0abbe0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f3f600011e0 tx=0x7f3f600039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:07.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.721+0000 7f3f5affd640 1 -- 172.21.15.125:0/1540819621 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60011900 con 0x7f3f540047c0 2023-10-18T10:44:07.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.721+0000 7f3f5affd640 1 -- 172.21.15.125:0/1540819621 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f60011a60 con 0x7f3f540047c0 2023-10-18T10:44:07.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.721+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c1d7a20 con 0x7f3f540047c0 2023-10-18T10:44:07.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.721+0000 7f3f5affd640 1 -- 172.21.15.125:0/1540819621 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60011e70 con 0x7f3f540047c0 2023-10-18T10:44:07.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.721+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c0ac120 con 0x7f3f540047c0 2023-10-18T10:44:07.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.722+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6c0ac2b0 con 0x7f3f540047c0 2023-10-18T10:44:07.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.722+0000 7f3f5affd640 1 -- 172.21.15.125:0/1540819621 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 68) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f3f600103c0 con 0x7f3f540047c0 2023-10-18T10:44:07.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.723+0000 7f3f5affd640 1 -- 172.21.15.125:0/1540819621 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f3f6008a390 con 0x7f3f540047c0 2023-10-18T10:44:07.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.727+0000 7f3f5affd640 1 -- 172.21.15.125:0/1540819621 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f600553c0 con 0x7f3f540047c0 2023-10-18T10:44:07.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.930+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f6c0690e0 con 0x7f3f540047c0 2023-10-18T10:44:07.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.930+0000 7f3f5affd640 1 -- 172.21.15.125:0/1540819621 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f3f600589e0 con 0x7f3f540047c0 2023-10-18T10:44:07.933 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:07.934 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":47,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/4122761343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4122761343},{"type":"v1","addr":"172.21.15.139:6836","nonce":4122761343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6785,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1014994333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1014994333},{"type":"v1","addr":"172.21.15.139:6837","nonce":1014994333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3090232356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3090232356},{"type":"v1","addr":"172.21.15.125:6834","nonce":3090232356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:43:03.299305+0000","modified":"2023-10-18T10:43:05.301917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8613":{"gid":8613,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2199142702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2199142702},{"type":"v1","addr":"172.21.15.125:6835","nonce":2199142702}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T10:44:07.936 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 47 2023-10-18T10:44:07.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 msgr2=0x7f3f6c0abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:07.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 0x7f3f6c0abbe0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f3f600011e0 tx=0x7f3f600039c0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 shutdown_connections 2023-10-18T10:44:07.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f6c1dc780 0x7f3f6c1dcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6c13ad60 0x7f3f6c13b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 --2- 172.21.15.125:0/1540819621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f540047c0 0x7f3f6c0abbe0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:07.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 >> 172.21.15.125:0/1540819621 conn(0x7f3f6c069930 msgr2=0x7f3f6c1de510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:07.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 shutdown_connections 2023-10-18T10:44:07.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:07.932+0000 7f3f7087c640 1 -- 172.21.15.125:0/1540819621 wait complete. 2023-10-18T10:44:07.945 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:08.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 -- 172.21.15.125:0/1342837286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300ab500 msgr2=0x7f18300ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 --2- 172.21.15.125:0/1342837286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300ab500 0x7f18300ab990 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f18300713a0 tx=0x7f182802c250 comp rx=0 tx=0).stop 2023-10-18T10:44:08.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 -- 172.21.15.125:0/1342837286 shutdown_connections 2023-10-18T10:44:08.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 --2- 172.21.15.125:0/1342837286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300ab500 0x7f18300ab990 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 --2- 172.21.15.125:0/1342837286 >> v2:172.21.15.139:3300/0 conn(0x7f18300c96b0 0x7f18300c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 --2- 172.21.15.125:0/1342837286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183006af00 0x7f183006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 -- 172.21.15.125:0/1342837286 >> 172.21.15.125:0/1342837286 conn(0x7f1830069930 msgr2=0x7f1830069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:08.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 -- 172.21.15.125:0/1342837286 shutdown_connections 2023-10-18T10:44:08.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.046+0000 7f18386a9640 1 -- 172.21.15.125:0/1342837286 wait complete. 2023-10-18T10:44:08.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f18386a9640 1 Processor -- start 2023-10-18T10:44:08.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f18386a9640 1 -- start start 2023-10-18T10:44:08.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f18386a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183006af00 0x7f18300a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f18386a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18300ab500 0x7f183013a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f18386a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300c96b0 0x7f18300b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f18386a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300b1cd0 con 0x7f18300c96b0 2023-10-18T10:44:08.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f18386a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300b0c30 con 0x7f183006af00 2023-10-18T10:44:08.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f18386a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18300b0f10 con 0x7f18300ab500 2023-10-18T10:44:08.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f1836c46640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300c96b0 0x7f18300b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f1836c46640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300c96b0 0x7f18300b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36718/0 (socket says 172.21.15.125:36718) 2023-10-18T10:44:08.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.048+0000 7f1836c46640 1 -- 172.21.15.125:0/3064253698 learned_addr learned my addr 172.21.15.125:0/3064253698 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:08.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1836445640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183006af00 0x7f18300a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1835c44640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18300ab500 0x7f183013a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1836c46640 1 -- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18300ab500 msgr2=0x7f183013a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1836c46640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18300ab500 0x7f183013a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1836c46640 1 -- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183006af00 msgr2=0x7f18300a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1836c46640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183006af00 0x7f18300a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1836c46640 1 -- 172.21.15.125:0/3064253698 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828009900 con 0x7f18300c96b0 2023-10-18T10:44:08.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1836445640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183006af00 0x7f18300a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:08.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1836c46640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300c96b0 0x7f18300b0330 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1828009d60 tx=0x7f1828035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:08.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1826ffd640 1 -- 172.21.15.125:0/3064253698 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828034440 con 0x7f18300c96b0 2023-10-18T10:44:08.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1826ffd640 1 -- 172.21.15.125:0/3064253698 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1828004450 con 0x7f18300c96b0 2023-10-18T10:44:08.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18300b11f0 con 0x7f18300c96b0 2023-10-18T10:44:08.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f1826ffd640 1 -- 172.21.15.125:0/3064253698 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828004860 con 0x7f18300c96b0 2023-10-18T10:44:08.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.049+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18300b17b0 con 0x7f18300c96b0 2023-10-18T10:44:08.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.050+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18300690e0 con 0x7f18300c96b0 2023-10-18T10:44:08.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.050+0000 7f1826ffd640 1 -- 172.21.15.125:0/3064253698 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 68) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f182800a040 con 0x7f18300c96b0 2023-10-18T10:44:08.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.051+0000 7f1826ffd640 1 -- 172.21.15.125:0/3064253698 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6803+0+0 (secure 0 0 0) 0x7f18280ac5f0 con 0x7f18300c96b0 2023-10-18T10:44:08.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.055+0000 7f1826ffd640 1 -- 172.21.15.125:0/3064253698 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18280776a0 con 0x7f18300c96b0 2023-10-18T10:44:08.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.223+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18300cc420 con 0x7f18300c96b0 2023-10-18T10:44:08.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.224+0000 7f1826ffd640 1 -- 172.21.15.125:0/3064253698 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+17905 (secure 0 0 0) 0x7f1828044090 con 0x7f18300c96b0 2023-10-18T10:44:08.226 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:08.226 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":130,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:07.263773+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:43:01.020853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-18T10:43:02.278684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3654883250":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6833/3564227248":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6832/3564227248":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2081490431":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/1435227047":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2662580042":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/1570574864":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/3882029760":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2083342052":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/1711146434":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/2016185684":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6833/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:6832/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/3171153406":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/3838090415":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/214738857":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/188703397":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/84406598":"2023-10-19T10:43:18.775528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:08.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300c96b0 msgr2=0x7f18300b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300c96b0 0x7f18300b0330 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1828009d60 tx=0x7f1828035e30 comp rx=0 tx=0).stop 2023-10-18T10:44:08.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 shutdown_connections 2023-10-18T10:44:08.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18300c96b0 0x7f18300b0330 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18300ab500 0x7f183013a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 --2- 172.21.15.125:0/3064253698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183006af00 0x7f18300a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 >> 172.21.15.125:0/3064253698 conn(0x7f1830069930 msgr2=0x7f1830069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:08.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 shutdown_connections 2023-10-18T10:44:08.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.226+0000 7f18386a9640 1 -- 172.21.15.125:0/3064253698 wait complete. 2023-10-18T10:44:08.238 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:44:08.239 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:08.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.348+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/2265894757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980cf340 msgr2=0x7f2b980d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.348+0000 7f2b9cbe2640 1 --2- 172.21.15.125:0/2265894757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980cf340 0x7f2b980d1910 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005fd0 tx=0x7f2b8802c2e0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.348+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/2265894757 shutdown_connections 2023-10-18T10:44:08.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.348+0000 7f2b9cbe2640 1 --2- 172.21.15.125:0/2265894757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980cf340 0x7f2b980d1910 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.348+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/2265894757 >> 172.21.15.125:0/2265894757 conn(0x7f2b980d0d60 msgr2=0x7f2b980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:08.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.348+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/2265894757 shutdown_connections 2023-10-18T10:44:08.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.348+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/2265894757 wait complete. 2023-10-18T10:44:08.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.349+0000 7f2b9cbe2640 1 Processor -- start 2023-10-18T10:44:08.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.349+0000 7f2b9cbe2640 1 -- start start 2023-10-18T10:44:08.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9cbe2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980ceb40 0x7f2b98158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9cbe2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b980cf340 0x7f2b981f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9cbe2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b9815b2a0 0x7f2b9809df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9cbe2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b6780 con 0x7f2b980ceb40 2023-10-18T10:44:08.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9cbe2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b38e0 con 0x7f2b980cf340 2023-10-18T10:44:08.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9cbe2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b3ea0 con 0x7f2b9815b2a0 2023-10-18T10:44:08.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980ceb40 0x7f2b98158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980ceb40 0x7f2b98158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36734/0 (socket says 172.21.15.125:36734) 2023-10-18T10:44:08.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.350+0000 7f2b9659c640 1 -- 172.21.15.125:0/788058430 learned_addr learned my addr 172.21.15.125:0/788058430 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:08.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b9659c640 1 -- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b9815b2a0 msgr2=0x7f2b9809df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b9659c640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b9815b2a0 0x7f2b9809df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b9659c640 1 -- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b980cf340 msgr2=0x7f2b981f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b9659c640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b980cf340 0x7f2b981f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b9659c640 1 -- 172.21.15.125:0/788058430 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b88009900 con 0x7f2b980ceb40 2023-10-18T10:44:08.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b9659c640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980ceb40 0x7f2b98158990 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006010 tx=0x7f2b88035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:08.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b86ffd640 1 -- 172.21.15.125:0/788058430 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88004240 con 0x7f2b980ceb40 2023-10-18T10:44:08.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b86ffd640 1 -- 172.21.15.125:0/788058430 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2b88004490 con 0x7f2b980ceb40 2023-10-18T10:44:08.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b98158ed0 con 0x7f2b980ceb40 2023-10-18T10:44:08.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b86ffd640 1 -- 172.21.15.125:0/788058430 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88004860 con 0x7f2b980ceb40 2023-10-18T10:44:08.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.351+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b981f2740 con 0x7f2b980ceb40 2023-10-18T10:44:08.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.352+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b980690e0 con 0x7f2b980ceb40 2023-10-18T10:44:08.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.352+0000 7f2b86ffd640 1 -- 172.21.15.125:0/788058430 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 69) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f2b8803b040 con 0x7f2b980ceb40 2023-10-18T10:44:08.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.353+0000 7f2b86ffd640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2b7806ac90 0x7f2b7806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.353+0000 7f2b86ffd640 1 -- 172.21.15.125:0/788058430 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6859+0+0 (secure 0 0 0) 0x7f2b880ac3a0 con 0x7f2b980ceb40 2023-10-18T10:44:08.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.353+0000 7f2b95d9b640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2b7806ac90 0x7f2b7806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.353+0000 7f2b95d9b640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2b7806ac90 0x7f2b7806d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c000f60 tx=0x7f2b8c0079b0 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:08.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.357+0000 7f2b86ffd640 1 -- 172.21.15.125:0/788058430 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b88077310 con 0x7f2b980ceb40 2023-10-18T10:44:08.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.557+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b980d0b40 con 0x7f2b980ceb40 2023-10-18T10:44:08.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.558+0000 7f2b86ffd640 1 -- 172.21.15.125:0/788058430 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f2b8807fe60 con 0x7f2b980ceb40 2023-10-18T10:44:08.561 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:08.561 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":47,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/4122761343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4122761343},{"type":"v1","addr":"172.21.15.139:6836","nonce":4122761343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6785,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1014994333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1014994333},{"type":"v1","addr":"172.21.15.139:6837","nonce":1014994333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3090232356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3090232356},{"type":"v1","addr":"172.21.15.125:6834","nonce":3090232356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:43:03.299305+0000","modified":"2023-10-18T10:43:05.301917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8613":{"gid":8613,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2199142702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2199142702},{"type":"v1","addr":"172.21.15.125:6835","nonce":2199142702}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T10:44:08.562 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 47 2023-10-18T10:44:08.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2b7806ac90 msgr2=0x7f2b7806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2b7806ac90 0x7f2b7806d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c000f60 tx=0x7f2b8c0079b0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980ceb40 msgr2=0x7f2b98158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980ceb40 0x7f2b98158990 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006010 tx=0x7f2b88035b50 comp rx=0 tx=0).stop 2023-10-18T10:44:08.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 shutdown_connections 2023-10-18T10:44:08.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2b7806ac90 0x7f2b7806d150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b980cf340 0x7f2b981f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b9815b2a0 0x7f2b9809df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 --2- 172.21.15.125:0/788058430 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b980ceb40 0x7f2b98158990 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 >> 172.21.15.125:0/788058430 conn(0x7f2b980d0d60 msgr2=0x7f2b980b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:08.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 shutdown_connections 2023-10-18T10:44:08.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.560+0000 7f2b9cbe2640 1 -- 172.21.15.125:0/788058430 wait complete. 2023-10-18T10:44:08.572 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:08.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.673+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1064084992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac141690 msgr2=0x7fc3ac0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.673+0000 7fc3b2e90640 1 --2- 172.21.15.125:0/1064084992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac141690 0x7fc3ac0703e0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009830 tx=0x7fc3a002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.673+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1064084992 shutdown_connections 2023-10-18T10:44:08.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.673+0000 7fc3b2e90640 1 --2- 172.21.15.125:0/1064084992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac141690 0x7fc3ac0703e0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.673+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1064084992 >> 172.21.15.125:0/1064084992 conn(0x7fc3ac069930 msgr2=0x7fc3ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:08.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.673+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1064084992 shutdown_connections 2023-10-18T10:44:08.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.673+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1064084992 wait complete. 2023-10-18T10:44:08.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.674+0000 7fc3b2e90640 1 Processor -- start 2023-10-18T10:44:08.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.674+0000 7fc3b2e90640 1 -- start start 2023-10-18T10:44:08.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b2e90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3ac070920 0x7fc3ac13ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b2e90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac13e520 0x7fc3ac13cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b2e90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3ac141690 0x7fc3ac13d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b2e90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1e3600 con 0x7fc3ac13e520 2023-10-18T10:44:08.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b2e90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac0a9360 con 0x7fc3ac070920 2023-10-18T10:44:08.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b2e90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac13b6b0 con 0x7fc3ac141690 2023-10-18T10:44:08.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac13e520 0x7fc3ac13cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac13e520 0x7fc3ac13cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36748/0 (socket says 172.21.15.125:36748) 2023-10-18T10:44:08.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 -- 172.21.15.125:0/1219946010 learned_addr learned my addr 172.21.15.125:0/1219946010 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:08.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b142d640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3ac141690 0x7fc3ac13d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 -- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3ac141690 msgr2=0x7fc3ac13d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b0c2c640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3ac070920 0x7fc3ac13ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3ac141690 0x7fc3ac13d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 -- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3ac070920 msgr2=0x7fc3ac13ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3ac070920 0x7fc3ac13ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 -- 172.21.15.125:0/1219946010 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0009490 con 0x7fc3ac13e520 2023-10-18T10:44:08.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b142d640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3ac141690 0x7fc3ac13d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:08.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3b0c2c640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3ac070920 0x7fc3ac13ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:08.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3abfff640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac13e520 0x7fc3ac13cfb0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009780 tx=0x7fc3a00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:08.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.675+0000 7fc3a97fa640 1 -- 172.21.15.125:0/1219946010 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0003ea0 con 0x7fc3ac13e520 2023-10-18T10:44:08.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.676+0000 7fc3a97fa640 1 -- 172.21.15.125:0/1219946010 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc3a0034550 con 0x7fc3ac13e520 2023-10-18T10:44:08.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.676+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ac13b180 con 0x7fc3ac13e520 2023-10-18T10:44:08.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.676+0000 7fc3a97fa640 1 -- 172.21.15.125:0/1219946010 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0034960 con 0x7fc3ac13e520 2023-10-18T10:44:08.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.676+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ac1e3040 con 0x7fc3ac13e520 2023-10-18T10:44:08.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.677+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ac0a8e90 con 0x7fc3ac13e520 2023-10-18T10:44:08.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.677+0000 7fc3a97fa640 1 -- 172.21.15.125:0/1219946010 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 69) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc3a003b040 con 0x7fc3ac13e520 2023-10-18T10:44:08.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.677+0000 7fc3a97fa640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc39806acc0 0x7fc39806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:08.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.677+0000 7fc3a97fa640 1 -- 172.21.15.125:0/1219946010 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6859+0+0 (secure 0 0 0) 0x7fc3a00abd80 con 0x7fc3ac13e520 2023-10-18T10:44:08.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.678+0000 7fc3b0c2c640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc39806acc0 0x7fc39806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:08.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.678+0000 7fc3b0c2c640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc39806acc0 0x7fc39806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0046b0 tx=0x7fc39c009040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:08.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.682+0000 7fc3a97fa640 1 -- 172.21.15.125:0/1219946010 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3ac0a8e90 con 0x7fc3ac13e520 2023-10-18T10:44:08.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.884+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3ac0a8e90 con 0x7fc3ac13e520 2023-10-18T10:44:08.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.885+0000 7fc3a97fa640 1 -- 172.21.15.125:0/1219946010 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fc3ac0a8e90 con 0x7fc3ac13e520 2023-10-18T10:44:08.887 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:08.888 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":47,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/4122761343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4122761343},{"type":"v1","addr":"172.21.15.139:6836","nonce":4122761343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6785,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1014994333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1014994333},{"type":"v1","addr":"172.21.15.139:6837","nonce":1014994333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3090232356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3090232356},{"type":"v1","addr":"172.21.15.125:6834","nonce":3090232356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:43:03.299305+0000","modified":"2023-10-18T10:43:05.301917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8613},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8613":{"gid":8613,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2199142702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2199142702},{"type":"v1","addr":"172.21.15.125:6835","nonce":2199142702}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T10:44:08.888 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 47 2023-10-18T10:44:08.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc39806acc0 msgr2=0x7fc39806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc39806acc0 0x7fc39806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0046b0 tx=0x7fc39c009040 comp rx=0 tx=0).stop 2023-10-18T10:44:08.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac13e520 msgr2=0x7fc3ac13cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:08.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac13e520 0x7fc3ac13cfb0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009780 tx=0x7fc3a00038a0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 shutdown_connections 2023-10-18T10:44:08.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc39806acc0 0x7fc39806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3ac141690 0x7fc3ac13d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3ac13e520 0x7fc3ac13cfb0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 --2- 172.21.15.125:0/1219946010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3ac070920 0x7fc3ac13ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:08.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 >> 172.21.15.125:0/1219946010 conn(0x7fc3ac069930 msgr2=0x7fc3ac0af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:08.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 shutdown_connections 2023-10-18T10:44:08.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:08.887+0000 7fc3b2e90640 1 -- 172.21.15.125:0/1219946010 wait complete. 2023-10-18T10:44:08.899 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:09.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.000+0000 7f63a18da640 1 -- 172.21.15.125:0/2767272187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63840047c0 msgr2=0x7f6384004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.000+0000 7f63a18da640 1 --2- 172.21.15.125:0/2767272187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63840047c0 0x7f6384004c40 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f639000c260 tx=0x7f63840057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.000+0000 7f639b7fe640 1 -- 172.21.15.125:0/2767272187 reap_dead start 2023-10-18T10:44:09.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.000+0000 7f63a18da640 1 -- 172.21.15.125:0/2767272187 shutdown_connections 2023-10-18T10:44:09.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.000+0000 7f63a18da640 1 -- 172.21.15.125:0/2767272187 >> 172.21.15.125:0/2767272187 conn(0x7f639c0d0d60 msgr2=0x7f639c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:09.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.000+0000 7f63a18da640 1 -- 172.21.15.125:0/2767272187 shutdown_connections 2023-10-18T10:44:09.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.000+0000 7f63a18da640 1 -- 172.21.15.125:0/2767272187 wait complete. 2023-10-18T10:44:09.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.001+0000 7f63a18da640 1 Processor -- start 2023-10-18T10:44:09.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.001+0000 7f63a18da640 1 -- start start 2023-10-18T10:44:09.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f63a18da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63840047c0 0x7f639c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f63a18da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f639c1e3850 0x7f639c1e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f63a18da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c0aeb90 0x7f639c0af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f63a18da640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c0af550 con 0x7f639c1e3850 2023-10-18T10:44:09.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f63a18da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c1e42d0 con 0x7f639c0aeb90 2023-10-18T10:44:09.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f63a18da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c1e45b0 con 0x7f63840047c0 2023-10-18T10:44:09.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f639c1e3850 0x7f639c1e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f639c1e3850 0x7f639c1e3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36774/0 (socket says 172.21.15.125:36774) 2023-10-18T10:44:09.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 -- 172.21.15.125:0/3180307124 learned_addr learned my addr 172.21.15.125:0/3180307124 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:09.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639b7fe640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c0aeb90 0x7f639c0af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639affd640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63840047c0 0x7f639c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 -- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63840047c0 msgr2=0x7f639c14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63840047c0 0x7f639c14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 -- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c0aeb90 msgr2=0x7f639c0af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c0aeb90 0x7f639c0af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 -- 172.21.15.125:0/3180307124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c1e4890 con 0x7f639c1e3850 2023-10-18T10:44:09.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.002+0000 7f639a7fc640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f639c1e3850 0x7f639c1e3cd0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f639000c380 tx=0x7f6390001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:09.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.003+0000 7f6383fff640 1 -- 172.21.15.125:0/3180307124 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390006230 con 0x7f639c1e3850 2023-10-18T10:44:09.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.003+0000 7f6383fff640 1 -- 172.21.15.125:0/3180307124 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6390006390 con 0x7f639c1e3850 2023-10-18T10:44:09.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.003+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c1e4b70 con 0x7f639c1e3850 2023-10-18T10:44:09.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.003+0000 7f6383fff640 1 -- 172.21.15.125:0/3180307124 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900035f0 con 0x7f639c1e3850 2023-10-18T10:44:09.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.003+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c14f7b0 con 0x7f639c1e3850 2023-10-18T10:44:09.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.004+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c0690e0 con 0x7f639c1e3850 2023-10-18T10:44:09.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.004+0000 7f6383fff640 1 -- 172.21.15.125:0/3180307124 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 69) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6390031030 con 0x7f639c1e3850 2023-10-18T10:44:09.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.004+0000 7f6383fff640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f638806ac90 0x7f638806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.005+0000 7f6383fff640 1 -- 172.21.15.125:0/3180307124 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6859+0+0 (secure 0 0 0) 0x7f63900aa460 con 0x7f639c1e3850 2023-10-18T10:44:09.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.005+0000 7f639affd640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f638806ac90 0x7f638806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.005+0000 7f639affd640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f638806ac90 0x7f638806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f638c005fd0 tx=0x7f638c009500 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:09.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.009+0000 7f6383fff640 1 -- 172.21.15.125:0/3180307124 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63900753d0 con 0x7f639c1e3850 2023-10-18T10:44:09.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.173+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f639c0d0b40 con 0x7f639c1e3850 2023-10-18T10:44:09.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.175+0000 7f6383fff640 1 -- 172.21.15.125:0/3180307124 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+18016 (secure 0 0 0) 0x7f63900789f0 con 0x7f639c1e3850 2023-10-18T10:44:09.177 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:09.177 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":131,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:08.266010+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:43:01.020853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-18T10:43:02.278684+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3654883250":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6833/3564227248":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6832/3564227248":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2081490431":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/1435227047":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2662580042":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/1570574864":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/3882029760":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2083342052":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/1711146434":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/2016185684":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6833/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:6832/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/3171153406":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/3838090415":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/214738857":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/188703397":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/84406598":"2023-10-19T10:43:18.775528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:09.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f638806ac90 msgr2=0x7f638806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f638806ac90 0x7f638806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f638c005fd0 tx=0x7f638c009500 comp rx=0 tx=0).stop 2023-10-18T10:44:09.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f639c1e3850 msgr2=0x7f639c1e3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f639c1e3850 0x7f639c1e3cd0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f639000c380 tx=0x7f6390001700 comp rx=0 tx=0).stop 2023-10-18T10:44:09.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 shutdown_connections 2023-10-18T10:44:09.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f638806ac90 0x7f638806d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f639c0aeb90 0x7f639c0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f639c1e3850 0x7f639c1e3cd0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 --2- 172.21.15.125:0/3180307124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63840047c0 0x7f639c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.176+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 >> 172.21.15.125:0/3180307124 conn(0x7f639c0d0d60 msgr2=0x7f639c159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:09.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.177+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 shutdown_connections 2023-10-18T10:44:09.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.177+0000 7f63a18da640 1 -- 172.21.15.125:0/3180307124 wait complete. 2023-10-18T10:44:09.189 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:44:09.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.290+0000 7f7e1a890640 1 -- 172.21.15.125:0/976238395 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e140c96b0 msgr2=0x7f7e140c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.290+0000 7f7e1a890640 1 --2- 172.21.15.125:0/976238395 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e140c96b0 0x7f7e140c9b30 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7e1406d590 tx=0x7f7e0802bda0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.291+0000 7f7e1a890640 1 -- 172.21.15.125:0/976238395 shutdown_connections 2023-10-18T10:44:09.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.291+0000 7f7e1a890640 1 --2- 172.21.15.125:0/976238395 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e140c96b0 0x7f7e140c9b30 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.291+0000 7f7e1a890640 1 --2- 172.21.15.125:0/976238395 >> v2:172.21.15.139:3300/0 conn(0x7f7e140ab500 0x7f7e140ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.291+0000 7f7e1a890640 1 --2- 172.21.15.125:0/976238395 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e1406af00 0x7f7e1406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.291+0000 7f7e1a890640 1 -- 172.21.15.125:0/976238395 >> 172.21.15.125:0/976238395 conn(0x7f7e14069930 msgr2=0x7f7e14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:09.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.291+0000 7f7e1a890640 1 -- 172.21.15.125:0/976238395 shutdown_connections 2023-10-18T10:44:09.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.291+0000 7f7e1a890640 1 -- 172.21.15.125:0/976238395 wait complete. 2023-10-18T10:44:09.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.292+0000 7f7e1a890640 1 Processor -- start 2023-10-18T10:44:09.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.292+0000 7f7e1a890640 1 -- start start 2023-10-18T10:44:09.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e1a890640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1406af00 0x7f7e141f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e1a890640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e140ab500 0x7f7e140aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e1a890640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e140c96b0 0x7f7e141ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e1a890640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e140ab200 con 0x7f7e1406af00 2023-10-18T10:44:09.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e1a890640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e140a64b0 con 0x7f7e140ab500 2023-10-18T10:44:09.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e1a890640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e140a6790 con 0x7f7e140c96b0 2023-10-18T10:44:09.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1406af00 0x7f7e141f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1406af00 0x7f7e141f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36796/0 (socket says 172.21.15.125:36796) 2023-10-18T10:44:09.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 -- 172.21.15.125:0/1511613439 learned_addr learned my addr 172.21.15.125:0/1511613439 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:09.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e137fe640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e140ab500 0x7f7e140aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e18e2d640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e140c96b0 0x7f7e141ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 -- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e140c96b0 msgr2=0x7f7e141ef7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e140c96b0 0x7f7e141ef7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 -- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e140ab500 msgr2=0x7f7e140aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e140ab500 0x7f7e140aacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 -- 172.21.15.125:0/1511613439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08009560 con 0x7f7e1406af00 2023-10-18T10:44:09.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e18e2d640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e140c96b0 0x7f7e141ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:09.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e13fff640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1406af00 0x7f7e141f35a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400d950 tx=0x7f7e0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:09.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e10ff9640 1 -- 172.21.15.125:0/1511613439 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04009c30 con 0x7f7e1406af00 2023-10-18T10:44:09.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e10ff9640 1 -- 172.21.15.125:0/1511613439 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7e04009e80 con 0x7f7e1406af00 2023-10-18T10:44:09.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e10ff9640 1 -- 172.21.15.125:0/1511613439 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040106c0 con 0x7f7e1406af00 2023-10-18T10:44:09.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e140a6a70 con 0x7f7e1406af00 2023-10-18T10:44:09.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.293+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e140a7030 con 0x7f7e1406af00 2023-10-18T10:44:09.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.294+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e140690e0 con 0x7f7e1406af00 2023-10-18T10:44:09.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.295+0000 7f7e10ff9640 1 -- 172.21.15.125:0/1511613439 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 69) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f7e04010820 con 0x7f7e1406af00 2023-10-18T10:44:09.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.295+0000 7f7e10ff9640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f7e0006ac90 0x7f7e0006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.295+0000 7f7e10ff9640 1 -- 172.21.15.125:0/1511613439 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6859+0+0 (secure 0 0 0) 0x7f7e04089af0 con 0x7f7e1406af00 2023-10-18T10:44:09.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.295+0000 7f7e137fe640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f7e0006ac90 0x7f7e0006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.296+0000 7f7e137fe640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f7e0006ac90 0x7f7e0006d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7e08035000 tx=0x7f7e08004730 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:09.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.299+0000 7f7e10ff9640 1 -- 172.21.15.125:0/1511613439 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e04054b10 con 0x7f7e1406af00 2023-10-18T10:44:09.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.503+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7e140ae460 con 0x7f7e1406af00 2023-10-18T10:44:09.505 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T10:44:09.504+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:44:09.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.505+0000 7f7e10ff9640 1 -- 172.21.15.125:0/1511613439 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7e04020090 con 0x7f7e1406af00 2023-10-18T10:44:09.507 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:44:09.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f7e0006ac90 msgr2=0x7f7e0006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f7e0006ac90 0x7f7e0006d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7e08035000 tx=0x7f7e08004730 comp rx=0 tx=0).stop 2023-10-18T10:44:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1406af00 msgr2=0x7f7e141f35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1406af00 0x7f7e141f35a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400d950 tx=0x7f7e0400de10 comp rx=0 tx=0).stop 2023-10-18T10:44:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 shutdown_connections 2023-10-18T10:44:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f7e0006ac90 0x7f7e0006d150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7e140ab500 0x7f7e140aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7e140c96b0 0x7f7e141ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 --2- 172.21.15.125:0/1511613439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e1406af00 0x7f7e141f35a0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.507+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 >> 172.21.15.125:0/1511613439 conn(0x7f7e14069930 msgr2=0x7f7e140adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.508+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 shutdown_connections 2023-10-18T10:44:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.508+0000 7f7e1a890640 1 -- 172.21.15.125:0/1511613439 wait complete. 2023-10-18T10:44:09.520 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:44:09.543 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:44:09.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.623+0000 7f51febd0640 1 -- 172.21.15.125:0/1859528405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 msgr2=0x7f51f80707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.623+0000 7f51febd0640 1 --2- 172.21.15.125:0/1859528405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 0x7f51f80707e0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005fd0 tx=0x7f51f002c0e0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.623+0000 7f51febd0640 1 -- 172.21.15.125:0/1859528405 shutdown_connections 2023-10-18T10:44:09.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.623+0000 7f51febd0640 1 --2- 172.21.15.125:0/1859528405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 0x7f51f80707e0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.623+0000 7f51febd0640 1 --2- 172.21.15.125:0/1859528405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f80cb980 0x7f51f813db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.623+0000 7f51febd0640 1 -- 172.21.15.125:0/1859528405 >> 172.21.15.125:0/1859528405 conn(0x7f51f8069930 msgr2=0x7f51f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:09.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.623+0000 7f51febd0640 1 -- 172.21.15.125:0/1859528405 shutdown_connections 2023-10-18T10:44:09.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.623+0000 7f51febd0640 1 -- 172.21.15.125:0/1859528405 wait complete. 2023-10-18T10:44:09.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51febd0640 1 Processor -- start 2023-10-18T10:44:09.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51febd0640 1 -- start start 2023-10-18T10:44:09.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51febd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 0x7f51f8139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51febd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f80cb980 0x7f51f8138a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51febd0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f813a0d0 0x7f51f8138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51febd0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81548e0 con 0x7f51f8070350 2023-10-18T10:44:09.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51febd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8153fd0 con 0x7f51f80cb980 2023-10-18T10:44:09.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51febd0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8154290 con 0x7f51f813a0d0 2023-10-18T10:44:09.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.625+0000 7f51fc96c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 0x7f51f8139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fc96c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 0x7f51f8139b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36834/0 (socket says 172.21.15.125:36834) 2023-10-18T10:44:09.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fc96c640 1 -- 172.21.15.125:0/2222190348 learned_addr learned my addr 172.21.15.125:0/2222190348 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:09.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51f7fff640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f80cb980 0x7f51f8138a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fd16d640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f813a0d0 0x7f51f8138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fc96c640 1 -- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f813a0d0 msgr2=0x7f51f8138f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fc96c640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f813a0d0 0x7f51f8138f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fc96c640 1 -- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f80cb980 msgr2=0x7f51f8138a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:09.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fc96c640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f80cb980 0x7f51f8138a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:09.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fc96c640 1 -- 172.21.15.125:0/2222190348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0009840 con 0x7f51f8070350 2023-10-18T10:44:09.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51fc96c640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 0x7f51f8139b90 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c970 tx=0x7f51ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:09.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51f57fa640 1 -- 172.21.15.125:0/2222190348 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec007bb0 con 0x7f51f8070350 2023-10-18T10:44:09.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51f57fa640 1 -- 172.21.15.125:0/2222190348 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f51ec007e20 con 0x7f51f8070350 2023-10-18T10:44:09.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f8154570 con 0x7f51f8070350 2023-10-18T10:44:09.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51f57fa640 1 -- 172.21.15.125:0/2222190348 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec005280 con 0x7f51f8070350 2023-10-18T10:44:09.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.626+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f8154f50 con 0x7f51f8070350 2023-10-18T10:44:09.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.627+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f80690e0 con 0x7f51f8070350 2023-10-18T10:44:09.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.628+0000 7f51f57fa640 1 -- 172.21.15.125:0/2222190348 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 69) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f51ec0053e0 con 0x7f51f8070350 2023-10-18T10:44:09.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.628+0000 7f51f57fa640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f51dc06ac90 0x7f51dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:09.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.628+0000 7f51f57fa640 1 -- 172.21.15.125:0/2222190348 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f51ec089c40 con 0x7f51f8070350 2023-10-18T10:44:09.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.628+0000 7f51f7fff640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f51dc06ac90 0x7f51dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:09.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.629+0000 7f51f7fff640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f51dc06ac90 0x7f51dc06d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51e4007fd0 tx=0x7f51e4009040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:09.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.632+0000 7f51f57fa640 1 -- 172.21.15.125:0/2222190348 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51ec08f020 con 0x7f51f8070350 2023-10-18T10:44:09.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.835+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f51f806ff80 con 0x7f51f8070350 2023-10-18T10:44:09.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:09.897+0000 7f51f57fa640 1 -- 172.21.15.125:0/2222190348 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 70) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f51ec054f40 con 0x7f51f8070350 2023-10-18T10:44:10.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.507+0000 7f51f57fa640 1 -- 172.21.15.125:0/2222190348 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f51ec0726c0 con 0x7f51f8070350 2023-10-18T10:44:10.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f51dc06ac90 msgr2=0x7f51dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:10.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f51dc06ac90 0x7f51dc06d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51e4007fd0 tx=0x7f51e4009040 comp rx=0 tx=0).stop 2023-10-18T10:44:10.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 msgr2=0x7f51f8139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:10.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 0x7f51f8139b90 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c970 tx=0x7f51ec00ce30 comp rx=0 tx=0).stop 2023-10-18T10:44:10.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 shutdown_connections 2023-10-18T10:44:10.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f51dc06ac90 0x7f51dc06d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f80cb980 0x7f51f8138a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f813a0d0 0x7f51f8138f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 --2- 172.21.15.125:0/2222190348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51f8070350 0x7f51f8139b90 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 >> 172.21.15.125:0/2222190348 conn(0x7f51f8069930 msgr2=0x7f51f8137a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:10.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 shutdown_connections 2023-10-18T10:44:10.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.509+0000 7f51febd0640 1 -- 172.21.15.125:0/2222190348 wait complete. 2023-10-18T10:44:10.524 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:44:10.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.673+0000 7f392dbb6640 1 -- 172.21.15.125:0/4042364576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280cf340 msgr2=0x7f39280d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:10.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.673+0000 7f392dbb6640 1 --2- 172.21.15.125:0/4042364576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280cf340 0x7f39280d1910 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f3918005fd0 tx=0x7f391802bcf0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.673+0000 7f392dbb6640 1 -- 172.21.15.125:0/4042364576 shutdown_connections 2023-10-18T10:44:10.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.673+0000 7f392dbb6640 1 --2- 172.21.15.125:0/4042364576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39280ceb40 0x7f39280b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.673+0000 7f392dbb6640 1 --2- 172.21.15.125:0/4042364576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280cf340 0x7f39280d1910 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.673+0000 7f392dbb6640 1 -- 172.21.15.125:0/4042364576 >> 172.21.15.125:0/4042364576 conn(0x7f39280d0d60 msgr2=0x7f39280d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:10.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.673+0000 7f392dbb6640 1 -- 172.21.15.125:0/4042364576 shutdown_connections 2023-10-18T10:44:10.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.673+0000 7f392dbb6640 1 -- 172.21.15.125:0/4042364576 wait complete. 2023-10-18T10:44:10.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.674+0000 7f392dbb6640 1 Processor -- start 2023-10-18T10:44:10.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.674+0000 7f392dbb6640 1 -- start start 2023-10-18T10:44:10.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f392dbb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39281f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:10.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f392dbb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39280cf340 0x7f39280ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:10.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f392dbb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f392815b2a0 0x7f39280ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:10.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f392dbb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39280b3a10 con 0x7f39280ceb40 2023-10-18T10:44:10.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f392dbb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39280b3cf0 con 0x7f39280cf340 2023-10-18T10:44:10.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f392dbb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39280b3fd0 con 0x7f392815b2a0 2023-10-18T10:44:10.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f39277fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39281f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:10.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f39277fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39281f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36852/0 (socket says 172.21.15.125:36852) 2023-10-18T10:44:10.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f39277fe640 1 -- 172.21.15.125:0/3937253019 learned_addr learned my addr 172.21.15.125:0/3937253019 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:10.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f3927fff640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f392815b2a0 0x7f39280ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:10.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f3926ffd640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39280cf340 0x7f39280ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:10.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f39277fe640 1 -- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f392815b2a0 msgr2=0x7f39280ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:10.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f39277fe640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f392815b2a0 0x7f39280ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f39277fe640 1 -- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39280cf340 msgr2=0x7f39280ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:10.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f39277fe640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39280cf340 0x7f39280ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:10.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.675+0000 7f39277fe640 1 -- 172.21.15.125:0/3937253019 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3918009560 con 0x7f39280ceb40 2023-10-18T10:44:10.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.676+0000 7f39277fe640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39281f1ff0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f3918006010 tx=0x7f391802da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:10.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.676+0000 7f390ffff640 1 -- 172.21.15.125:0/3937253019 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391802de90 con 0x7f39280ceb40 2023-10-18T10:44:10.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.676+0000 7f390ffff640 1 -- 172.21.15.125:0/3937253019 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f39180036e0 con 0x7f39280ceb40 2023-10-18T10:44:10.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.676+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39280b0d50 con 0x7f39280ceb40 2023-10-18T10:44:10.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.676+0000 7f390ffff640 1 -- 172.21.15.125:0/3937253019 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918003af0 con 0x7f39280ceb40 2023-10-18T10:44:10.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.676+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39280b4590 con 0x7f39280ceb40 2023-10-18T10:44:10.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.677+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39280b4780 con 0x7f39280ceb40 2023-10-18T10:44:10.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.677+0000 7f390ffff640 1 -- 172.21.15.125:0/3937253019 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 70) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3918003c50 con 0x7f39280ceb40 2023-10-18T10:44:10.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.678+0000 7f390ffff640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f391406aed0 0x7f391406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:10.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.678+0000 7f390ffff640 1 -- 172.21.15.125:0/3937253019 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f39180ac5b0 con 0x7f39280ceb40 2023-10-18T10:44:10.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.678+0000 7f3926ffd640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f391406aed0 0x7f391406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:10.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.678+0000 7f3926ffd640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f391406aed0 0x7f391406d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f391c006fd0 tx=0x7f391c008040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:10.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.682+0000 7f390ffff640 1 -- 172.21.15.125:0/3937253019 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39180772e0 con 0x7f39280ceb40 2023-10-18T10:44:10.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:10.849+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f39280690e0 con 0x7f39280ceb40 2023-10-18T10:44:11.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.508+0000 7f390ffff640 1 -- 172.21.15.125:0/3937253019 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 181+0+0 (secure 0 0 0) 0x7f391807a900 con 0x7f39280ceb40 2023-10-18T10:44:11.511 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:44:11.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.512+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f391406aed0 msgr2=0x7f391406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:11.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.512+0000 7f392dbb6640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f391406aed0 0x7f391406d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f391c006fd0 tx=0x7f391c008040 comp rx=0 tx=0).stop 2023-10-18T10:44:11.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.512+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 msgr2=0x7f39281f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:11.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.512+0000 7f392dbb6640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39281f1ff0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f3918006010 tx=0x7f391802da90 comp rx=0 tx=0).stop 2023-10-18T10:44:11.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.512+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 shutdown_connections 2023-10-18T10:44:11.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.512+0000 7f392dbb6640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f391406aed0 0x7f391406d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.513+0000 7f392dbb6640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39280cf340 0x7f39280ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.513+0000 7f392dbb6640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f392815b2a0 0x7f39280ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.513+0000 7f392dbb6640 1 --2- 172.21.15.125:0/3937253019 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39280ceb40 0x7f39281f1ff0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.513+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 >> 172.21.15.125:0/3937253019 conn(0x7f39280d0d60 msgr2=0x7f39280ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:11.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.513+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 shutdown_connections 2023-10-18T10:44:11.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.513+0000 7f392dbb6640 1 -- 172.21.15.125:0/3937253019 wait complete. 2023-10-18T10:44:11.528 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:44:11.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.638+0000 7f589f2e5640 1 -- 172.21.15.125:0/2430713527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 msgr2=0x7f58980d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:11.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.638+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2430713527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 0x7f58980d1910 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f5894005fd0 tx=0x7f589402c1b0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.638+0000 7f589f2e5640 1 -- 172.21.15.125:0/2430713527 shutdown_connections 2023-10-18T10:44:11.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.638+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2430713527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58980ceb40 0x7f58980b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.638+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2430713527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 0x7f58980d1910 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.638+0000 7f589f2e5640 1 -- 172.21.15.125:0/2430713527 >> 172.21.15.125:0/2430713527 conn(0x7f58980d0d60 msgr2=0x7f58980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:11.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.638+0000 7f589f2e5640 1 -- 172.21.15.125:0/2430713527 shutdown_connections 2023-10-18T10:44:11.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.638+0000 7f589f2e5640 1 -- 172.21.15.125:0/2430713527 wait complete. 2023-10-18T10:44:11.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.639+0000 7f589f2e5640 1 Processor -- start 2023-10-18T10:44:11.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.639+0000 7f589f2e5640 1 -- start start 2023-10-18T10:44:11.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589f2e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58980ceb40 0x7f58981572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:11.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589f2e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 0x7f58980c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:11.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589f2e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f589815b2a0 0x7f58980b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:11.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589f2e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980c8e20 con 0x7f58980cf340 2023-10-18T10:44:11.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589f2e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980c8860 con 0x7f58980ceb40 2023-10-18T10:44:11.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589f2e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58980c8b40 con 0x7f589815b2a0 2023-10-18T10:44:11.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589c880640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 0x7f58980c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:11.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589c880640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 0x7f58980c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36864/0 (socket says 172.21.15.125:36864) 2023-10-18T10:44:11.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589c880640 1 -- 172.21.15.125:0/2478595762 learned_addr learned my addr 172.21.15.125:0/2478595762 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:11.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589c880640 1 -- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f589815b2a0 msgr2=0x7f58980b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:11.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589c880640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f589815b2a0 0x7f58980b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589c880640 1 -- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58980ceb40 msgr2=0x7f58981572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:11.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589c880640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58980ceb40 0x7f58981572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:11.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.640+0000 7f589c880640 1 -- 172.21.15.125:0/2478595762 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5894009900 con 0x7f58980cf340 2023-10-18T10:44:11.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.641+0000 7f589c880640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 0x7f58980c7e30 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f588800b3e0 tx=0x7f588800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:11.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.641+0000 7f588dffb640 1 -- 172.21.15.125:0/2478595762 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888011830 con 0x7f58980cf340 2023-10-18T10:44:11.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.641+0000 7f588dffb640 1 -- 172.21.15.125:0/2478595762 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5888011aa0 con 0x7f58980cf340 2023-10-18T10:44:11.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.641+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58980c9340 con 0x7f58980cf340 2023-10-18T10:44:11.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.641+0000 7f588dffb640 1 -- 172.21.15.125:0/2478595762 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888010380 con 0x7f58980cf340 2023-10-18T10:44:11.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.641+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58980b67d0 con 0x7f58980cf340 2023-10-18T10:44:11.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.642+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58980c9860 con 0x7f58980cf340 2023-10-18T10:44:11.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.642+0000 7f588dffb640 1 -- 172.21.15.125:0/2478595762 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 70) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f58880104e0 con 0x7f58980cf340 2023-10-18T10:44:11.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.642+0000 7f588dffb640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f587c06aed0 0x7f587c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:11.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.643+0000 7f588dffb640 1 -- 172.21.15.125:0/2478595762 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6479+0+0 (secure 0 0 0) 0x7f5888089d00 con 0x7f58980cf340 2023-10-18T10:44:11.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.643+0000 7f589d081640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f587c06aed0 0x7f587c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:11.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.643+0000 7f589d081640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f587c06aed0 0x7f587c06d390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5894036000 tx=0x7f5894004a40 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:11.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.647+0000 7f588dffb640 1 -- 172.21.15.125:0/2478595762 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58980c9860 con 0x7f58980cf340 2023-10-18T10:44:11.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.814+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58980c9860 con 0x7f58980cf340 2023-10-18T10:44:11.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:11.907+0000 7f588dffb640 1 -- 172.21.15.125:0/2478595762 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5888054e90 con 0x7f58980cf340 2023-10-18T10:44:12.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.514+0000 7f588dffb640 1 -- 172.21.15.125:0/2478595762 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5888071d80 con 0x7f58980cf340 2023-10-18T10:44:12.516 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T10:44:12.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.517+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f587c06aed0 msgr2=0x7f587c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.517+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f587c06aed0 0x7f587c06d390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5894036000 tx=0x7f5894004a40 comp rx=0 tx=0).stop 2023-10-18T10:44:12.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.517+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 msgr2=0x7f58980c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.517+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 0x7f58980c7e30 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f588800b3e0 tx=0x7f588800b8a0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.518+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 shutdown_connections 2023-10-18T10:44:12.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.518+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f587c06aed0 0x7f587c06d390 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.518+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f589815b2a0 0x7f58980b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.518+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58980cf340 0x7f58980c7e30 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.518+0000 7f589f2e5640 1 --2- 172.21.15.125:0/2478595762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58980ceb40 0x7f58981572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.518+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 >> 172.21.15.125:0/2478595762 conn(0x7f58980d0d60 msgr2=0x7f589806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:12.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.518+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 shutdown_connections 2023-10-18T10:44:12.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.518+0000 7f589f2e5640 1 -- 172.21.15.125:0/2478595762 wait complete. 2023-10-18T10:44:12.533 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T10:44:12.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.639+0000 7fcee97a5640 1 -- 172.21.15.125:0/545539455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcee40cf340 msgr2=0x7fcee40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.639+0000 7fcee97a5640 1 --2- 172.21.15.125:0/545539455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcee40cf340 0x7fcee40d1910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fced4009830 tx=0x7fced402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.639+0000 7fcee97a5640 1 -- 172.21.15.125:0/545539455 shutdown_connections 2023-10-18T10:44:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.639+0000 7fcee97a5640 1 --2- 172.21.15.125:0/545539455 >> v2:172.21.15.139:3300/0 conn(0x7fcee415b2a0 0x7fcee40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.639+0000 7fcee97a5640 1 --2- 172.21.15.125:0/545539455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcee40cf340 0x7fcee40d1910 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.639+0000 7fcee97a5640 1 -- 172.21.15.125:0/545539455 >> 172.21.15.125:0/545539455 conn(0x7fcee40d0d60 msgr2=0x7fcee40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:12.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.639+0000 7fcee97a5640 1 -- 172.21.15.125:0/545539455 shutdown_connections 2023-10-18T10:44:12.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.639+0000 7fcee97a5640 1 -- 172.21.15.125:0/545539455 wait complete. 2023-10-18T10:44:12.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.640+0000 7fcee97a5640 1 Processor -- start 2023-10-18T10:44:12.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.640+0000 7fcee97a5640 1 -- start start 2023-10-18T10:44:12.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee97a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcee40ceb40 0x7fcee40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:12.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee97a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee40cf340 0x7fcee414f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:12.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee97a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcee415b2a0 0x7fcee40ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:12.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee97a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40ca3f0 con 0x7fcee40ceb40 2023-10-18T10:44:12.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee97a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee414f820 con 0x7fcee40cf340 2023-10-18T10:44:12.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee97a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40c9d20 con 0x7fcee415b2a0 2023-10-18T10:44:12.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcee40ceb40 0x7fcee40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:12.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcee40ceb40 0x7fcee40b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36876/0 (socket says 172.21.15.125:36876) 2023-10-18T10:44:12.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee2ffd640 1 -- 172.21.15.125:0/2382256467 learned_addr learned my addr 172.21.15.125:0/2382256467 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:12.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee37fe640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcee415b2a0 0x7fcee40ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:12.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.641+0000 7fcee27fc640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee40cf340 0x7fcee414f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:12.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcee2ffd640 1 -- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcee415b2a0 msgr2=0x7fcee40ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcee2ffd640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcee415b2a0 0x7fcee40ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcee2ffd640 1 -- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee40cf340 msgr2=0x7fcee414f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcee2ffd640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee40cf340 0x7fcee414f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcee2ffd640 1 -- 172.21.15.125:0/2382256467 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009490 con 0x7fcee40ceb40 2023-10-18T10:44:12.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcee2ffd640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcee40ceb40 0x7fcee40b3950 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fced4000c00 tx=0x7fced40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:12.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcecb7fe640 1 -- 172.21.15.125:0/2382256467 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4003ea0 con 0x7fcee40ceb40 2023-10-18T10:44:12.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcecb7fe640 1 -- 172.21.15.125:0/2382256467 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fced4034440 con 0x7fcee40ceb40 2023-10-18T10:44:12.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcecb7fe640 1 -- 172.21.15.125:0/2382256467 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4034850 con 0x7fcee40ceb40 2023-10-18T10:44:12.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced4009490 con 0x7fcee40ceb40 2023-10-18T10:44:12.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.642+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee40c9a40 con 0x7fcee40ceb40 2023-10-18T10:44:12.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.643+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee40690e0 con 0x7fcee40ceb40 2023-10-18T10:44:12.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.643+0000 7fcecb7fe640 1 -- 172.21.15.125:0/2382256467 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fced40349b0 con 0x7fcee40ceb40 2023-10-18T10:44:12.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.644+0000 7fcecb7fe640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fced006af60 0x7fced006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:12.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.644+0000 7fcecb7fe640 1 -- 172.21.15.125:0/2382256467 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7fced40abf50 con 0x7fcee40ceb40 2023-10-18T10:44:12.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.644+0000 7fcee27fc640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fced006af60 0x7fced006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:12.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.644+0000 7fcee27fc640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fced006af60 0x7fced006d420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcee40b0c20 tx=0x7fced8005f90 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:12.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.648+0000 7fcecb7fe640 1 -- 172.21.15.125:0/2382256467 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fced4076ec0 con 0x7fcee40ceb40 2023-10-18T10:44:12.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.814+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7fcee40d0b40 con 0x7fcee40ceb40 2023-10-18T10:44:12.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.817+0000 7fcecb7fe640 1 -- 172.21.15.125:0/2382256467 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v54) v1 ==== 71+0+0 (secure 0 0 0) 0x7fced4042080 con 0x7fcee40ceb40 2023-10-18T10:44:12.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fced006af60 msgr2=0x7fced006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fced006af60 0x7fced006d420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcee40b0c20 tx=0x7fced8005f90 comp rx=0 tx=0).stop 2023-10-18T10:44:12.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcee40ceb40 msgr2=0x7fcee40b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcee40ceb40 0x7fcee40b3950 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fced4000c00 tx=0x7fced40038a0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 shutdown_connections 2023-10-18T10:44:12.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fced006af60 0x7fced006d420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcee40cf340 0x7fcee414f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcee415b2a0 0x7fcee40ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 --2- 172.21.15.125:0/2382256467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcee40ceb40 0x7fcee40b3950 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 >> 172.21.15.125:0/2382256467 conn(0x7fcee40d0d60 msgr2=0x7fcee40b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.820+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 shutdown_connections 2023-10-18T10:44:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.821+0000 7fcee97a5640 1 -- 172.21.15.125:0/2382256467 wait complete. 2023-10-18T10:44:12.834 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-18T10:44:12.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.935+0000 7f3a627ce640 1 -- 172.21.15.125:0/3778437001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c06af00 msgr2=0x7f3a5c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.935+0000 7f3a627ce640 1 --2- 172.21.15.125:0/3778437001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c06af00 0x7f3a5c06b2e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009900 tx=0x7f3a4c02be00 comp rx=0 tx=0).stop 2023-10-18T10:44:12.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.935+0000 7f3a627ce640 1 -- 172.21.15.125:0/3778437001 shutdown_connections 2023-10-18T10:44:12.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.935+0000 7f3a627ce640 1 --2- 172.21.15.125:0/3778437001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5c0c96b0 0x7f3a5c0c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.935+0000 7f3a627ce640 1 --2- 172.21.15.125:0/3778437001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c06af00 0x7f3a5c06b2e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.935+0000 7f3a627ce640 1 -- 172.21.15.125:0/3778437001 >> 172.21.15.125:0/3778437001 conn(0x7f3a5c069930 msgr2=0x7f3a5c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:12.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.935+0000 7f3a627ce640 1 -- 172.21.15.125:0/3778437001 shutdown_connections 2023-10-18T10:44:12.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.935+0000 7f3a627ce640 1 -- 172.21.15.125:0/3778437001 wait complete. 2023-10-18T10:44:12.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.936+0000 7f3a627ce640 1 Processor -- start 2023-10-18T10:44:12.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a627ce640 1 -- start start 2023-10-18T10:44:12.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a627ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5c06af00 0x7f3a5c139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:12.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a627ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5c0ab500 0x7f3a5c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:12.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a627ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c0c96b0 0x7f3a5c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:12.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a627ce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c13a320 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a627ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0b10e0 con 0x7f3a5c0ab500 2023-10-18T10:44:12.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a627ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0a9ab0 con 0x7f3a5c06af00 2023-10-18T10:44:12.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a60d6b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c0c96b0 0x7f3a5c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:12.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a60d6b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c0c96b0 0x7f3a5c15a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36880/0 (socket says 172.21.15.125:36880) 2023-10-18T10:44:12.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.937+0000 7f3a60d6b640 1 -- 172.21.15.125:0/526139579 learned_addr learned my addr 172.21.15.125:0/526139579 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:12.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a5bfff640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5c06af00 0x7f3a5c139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:12.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a60d6b640 1 -- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5c06af00 msgr2=0x7f3a5c139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a60d6b640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5c06af00 0x7f3a5c139de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a5b7fe640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5c0ab500 0x7f3a5c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:12.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a60d6b640 1 -- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5c0ab500 msgr2=0x7f3a5c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:12.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a60d6b640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5c0ab500 0x7f3a5c0b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:12.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a60d6b640 1 -- 172.21.15.125:0/526139579 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c009560 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a60d6b640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c0c96b0 0x7f3a5c15a900 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f3a54001560 tx=0x7f3a5400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:12.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a58ff9640 1 -- 172.21.15.125:0/526139579 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54003da0 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a58ff9640 1 -- 172.21.15.125:0/526139579 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a54005070 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a58ff9640 1 -- 172.21.15.125:0/526139579 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54005460 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c0a6550 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.938+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c0a6b10 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.939+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c0a6d00 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.939+0000 7f3a58ff9640 1 -- 172.21.15.125:0/526139579 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3a540055c0 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.940+0000 7f3a58ff9640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f3a4806af60 0x7f3a4806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.940+0000 7f3a58ff9640 1 -- 172.21.15.125:0/526139579 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7f3a540893a0 con 0x7f3a5c0c96b0 2023-10-18T10:44:12.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.940+0000 7f3a5bfff640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f3a4806af60 0x7f3a4806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:12.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.940+0000 7f3a5bfff640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f3a4806af60 0x7f3a4806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c035000 tx=0x7f3a4c004730 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:12.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:12.944+0000 7f3a58ff9640 1 -- 172.21.15.125:0/526139579 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a54054260 con 0x7f3a5c0c96b0 2023-10-18T10:44:13.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.172+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f3a5c0690e0 con 0x7f3a5c0c96b0 2023-10-18T10:44:13.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.515+0000 7f3a58ff9640 1 -- 172.21.15.125:0/526139579 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v723) v1 ==== 148+0+0 (secure 0 0 0) 0x7f3a54057880 con 0x7f3a5c0c96b0 2023-10-18T10:44:13.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f3a4806af60 msgr2=0x7f3a4806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f3a4806af60 0x7f3a4806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c035000 tx=0x7f3a4c004730 comp rx=0 tx=0).stop 2023-10-18T10:44:13.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c0c96b0 msgr2=0x7f3a5c15a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c0c96b0 0x7f3a5c15a900 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f3a54001560 tx=0x7f3a5400cc70 comp rx=0 tx=0).stop 2023-10-18T10:44:13.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 shutdown_connections 2023-10-18T10:44:13.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f3a4806af60 0x7f3a4806d420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a5c0ab500 0x7f3a5c0b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a5c0c96b0 0x7f3a5c15a900 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 --2- 172.21.15.125:0/526139579 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a5c06af00 0x7f3a5c139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 >> 172.21.15.125:0/526139579 conn(0x7f3a5c069930 msgr2=0x7f3a5c0cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:13.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 shutdown_connections 2023-10-18T10:44:13.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.518+0000 7f3a627ce640 1 -- 172.21.15.125:0/526139579 wait complete. 2023-10-18T10:44:13.531 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-10-18T10:44:13.531 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T10:44:13.532 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:13.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.632+0000 7feb47d97640 1 -- 172.21.15.125:0/2532311693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb400c9dc0 msgr2=0x7feb4006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.632+0000 7feb47d97640 1 --2- 172.21.15.125:0/2532311693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb400c9dc0 0x7feb4006b2e0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7feb30005fd0 tx=0x7feb3002bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:13.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.632+0000 7feb47d97640 1 -- 172.21.15.125:0/2532311693 shutdown_connections 2023-10-18T10:44:13.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.632+0000 7feb47d97640 1 --2- 172.21.15.125:0/2532311693 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb400ab500 0x7feb400ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.632+0000 7feb47d97640 1 --2- 172.21.15.125:0/2532311693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb400c9dc0 0x7feb4006b2e0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.632+0000 7feb47d97640 1 -- 172.21.15.125:0/2532311693 >> 172.21.15.125:0/2532311693 conn(0x7feb40069930 msgr2=0x7feb40069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:13.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.632+0000 7feb47d97640 1 -- 172.21.15.125:0/2532311693 shutdown_connections 2023-10-18T10:44:13.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.632+0000 7feb47d97640 1 -- 172.21.15.125:0/2532311693 wait complete. 2023-10-18T10:44:13.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.633+0000 7feb47d97640 1 Processor -- start 2023-10-18T10:44:13.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.633+0000 7feb47d97640 1 -- start start 2023-10-18T10:44:13.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb47d97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb4006af00 0x7feb400a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:13.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb47d97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb400ab500 0x7feb4013a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:13.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb47d97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb400c9dc0 0x7feb400aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:13.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb47d97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400aa600 con 0x7feb4006af00 2023-10-18T10:44:13.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb47d97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400a73e0 con 0x7feb400c9dc0 2023-10-18T10:44:13.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb47d97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4013a800 con 0x7feb400ab500 2023-10-18T10:44:13.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb4006af00 0x7feb400a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:13.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb4006af00 0x7feb400a6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36896/0 (socket says 172.21.15.125:36896) 2023-10-18T10:44:13.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 -- 172.21.15.125:0/967168476 learned_addr learned my addr 172.21.15.125:0/967168476 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:13.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 -- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb400ab500 msgr2=0x7feb4013a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb400ab500 0x7feb4013a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 -- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb400c9dc0 msgr2=0x7feb400aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb400c9dc0 0x7feb400aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 -- 172.21.15.125:0/967168476 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30009560 con 0x7feb4006af00 2023-10-18T10:44:13.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.634+0000 7feb45b33640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb4006af00 0x7feb400a6ea0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00c7c0 tx=0x7feb3c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:13.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.635+0000 7feb367fc640 1 -- 172.21.15.125:0/967168476 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00aa50 con 0x7feb4006af00 2023-10-18T10:44:13.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.635+0000 7feb367fc640 1 -- 172.21.15.125:0/967168476 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb3c00aca0 con 0x7feb4006af00 2023-10-18T10:44:13.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.635+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb400a99f0 con 0x7feb4006af00 2023-10-18T10:44:13.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.635+0000 7feb367fc640 1 -- 172.21.15.125:0/967168476 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c012680 con 0x7feb4006af00 2023-10-18T10:44:13.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.635+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb400a9fb0 con 0x7feb4006af00 2023-10-18T10:44:13.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.635+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb400690e0 con 0x7feb4006af00 2023-10-18T10:44:13.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.636+0000 7feb367fc640 1 -- 172.21.15.125:0/967168476 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb3c01a3b0 con 0x7feb4006af00 2023-10-18T10:44:13.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.636+0000 7feb367fc640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7feb2406af60 0x7feb2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:13.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.637+0000 7feb367fc640 1 -- 172.21.15.125:0/967168476 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7feb3c089d70 con 0x7feb4006af00 2023-10-18T10:44:13.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.637+0000 7feb45332640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7feb2406af60 0x7feb2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:13.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.637+0000 7feb45332640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7feb2406af60 0x7feb2406d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feb30035000 tx=0x7feb300046c0 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:13.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.641+0000 7feb367fc640 1 -- 172.21.15.125:0/967168476 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb3c054c30 con 0x7feb4006af00 2023-10-18T10:44:13.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.808+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb4006bf00 con 0x7feb4006af00 2023-10-18T10:44:13.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.809+0000 7feb367fc640 1 -- 172.21.15.125:0/967168476 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+14437 (secure 0 0 0) 0x7feb3c058250 con 0x7feb4006af00 2023-10-18T10:44:13.811 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:13.812 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":134,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:12.510208+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6835/2199142702":"2023-10-19T10:44:09.504870+0000","172.21.15.125:6833/2199142702":"2023-10-19T10:44:09.504870+0000","172.21.15.139:0/3654883250":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6833/3564227248":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6832/3564227248":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2081490431":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/1435227047":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2662580042":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/1570574864":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/3882029760":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/2083342052":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/1711146434":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/2016185684":"2023-10-19T10:44:07.263735+0000","172.21.15.139:6833/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:6832/1511464628":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/3171153406":"2023-10-19T10:44:07.263735+0000","172.21.15.139:0/3838090415":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/214738857":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/188703397":"2023-10-19T10:43:18.775528+0000","172.21.15.139:0/84406598":"2023-10-19T10:43:18.775528+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:13.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7feb2406af60 msgr2=0x7feb2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7feb2406af60 0x7feb2406d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feb30035000 tx=0x7feb300046c0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb4006af00 msgr2=0x7feb400a6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb4006af00 0x7feb400a6ea0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00c7c0 tx=0x7feb3c00cc80 comp rx=0 tx=0).stop 2023-10-18T10:44:13.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 shutdown_connections 2023-10-18T10:44:13.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7feb2406af60 0x7feb2406d420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb400c9dc0 0x7feb400aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb400ab500 0x7feb4013a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 --2- 172.21.15.125:0/967168476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb4006af00 0x7feb400a6ea0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 >> 172.21.15.125:0/967168476 conn(0x7feb40069930 msgr2=0x7feb40069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:13.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 shutdown_connections 2023-10-18T10:44:13.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.811+0000 7feb47d97640 1 -- 172.21.15.125:0/967168476 wait complete. 2023-10-18T10:44:13.824 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:44:13.825 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:44:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.929+0000 7ff3b1a32640 1 -- 172.21.15.125:0/3512989528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 msgr2=0x7ff394003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.929+0000 7ff3b1a32640 1 --2- 172.21.15.125:0/3512989528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 0x7ff394003d90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff39c001680 tx=0x7ff3940057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.929+0000 7ff3b1a32640 1 -- 172.21.15.125:0/3512989528 shutdown_connections 2023-10-18T10:44:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.929+0000 7ff3b1a32640 1 --2- 172.21.15.125:0/3512989528 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 0x7ff394003d90 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.929+0000 7ff3b1a32640 1 -- 172.21.15.125:0/3512989528 >> 172.21.15.125:0/3512989528 conn(0x7ff3ac0b0670 msgr2=0x7ff3ac0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.929+0000 7ff3b1a32640 1 -- 172.21.15.125:0/3512989528 shutdown_connections 2023-10-18T10:44:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.929+0000 7ff3b1a32640 1 -- 172.21.15.125:0/3512989528 wait complete. 2023-10-18T10:44:13.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.930+0000 7ff3b1a32640 1 Processor -- start 2023-10-18T10:44:13.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.930+0000 7ff3b1a32640 1 -- start start 2023-10-18T10:44:13.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3b1a32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 0x7ff3ac0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3b1a32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ac155630 0x7ff3ac155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3b1a32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ac14f9d0 0x7ff3ac14fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3b1a32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cacd0 con 0x7ff394003970 2023-10-18T10:44:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3b1a32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1503c0 con 0x7ff3ac155630 2023-10-18T10:44:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3b1a32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0ca0c0 con 0x7ff3ac14f9d0 2023-10-18T10:44:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 0x7ff3ac0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 0x7ff3ac0ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36926/0 (socket says 172.21.15.125:36926) 2023-10-18T10:44:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 -- 172.21.15.125:0/1848036718 learned_addr learned my addr 172.21.15.125:0/1848036718 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 -- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ac14f9d0 msgr2=0x7ff3ac14fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ac14f9d0 0x7ff3ac14fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 -- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ac155630 msgr2=0x7ff3ac155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:13.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ac155630 0x7ff3ac155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:13.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 -- 172.21.15.125:0/1848036718 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0ca3a0 con 0x7ff394003970 2023-10-18T10:44:13.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3aaffd640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 0x7ff3ac0ae100 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0017a0 tx=0x7ff39c006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:13.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff393fff640 1 -- 172.21.15.125:0/1848036718 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c006540 con 0x7ff394003970 2023-10-18T10:44:13.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff393fff640 1 -- 172.21.15.125:0/1848036718 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff39c003bf0 con 0x7ff394003970 2023-10-18T10:44:13.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.931+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac0ca680 con 0x7ff394003970 2023-10-18T10:44:13.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.932+0000 7ff393fff640 1 -- 172.21.15.125:0/1848036718 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c02c530 con 0x7ff394003970 2023-10-18T10:44:13.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.932+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac1569d0 con 0x7ff394003970 2023-10-18T10:44:13.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.933+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac0ce1d0 con 0x7ff394003970 2023-10-18T10:44:13.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.933+0000 7ff393fff640 1 -- 172.21.15.125:0/1848036718 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff39c02c690 con 0x7ff394003970 2023-10-18T10:44:13.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.933+0000 7ff393fff640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff39806af90 0x7ff39806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:13.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.933+0000 7ff393fff640 1 -- 172.21.15.125:0/1848036718 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7ff39c0aadb0 con 0x7ff394003970 2023-10-18T10:44:13.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.934+0000 7ff3aa7fc640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff39806af90 0x7ff39806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:13.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.934+0000 7ff3aa7fc640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff39806af90 0x7ff39806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0c9000 tx=0x7ff3a0009530 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:13.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:13.938+0000 7ff393fff640 1 -- 172.21.15.125:0/1848036718 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff39c075be0 con 0x7ff394003970 2023-10-18T10:44:14.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.101+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3ac0690e0 con 0x7ff394003970 2023-10-18T10:44:14.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.101+0000 7ff393fff640 1 -- 172.21.15.125:0/1848036718 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff39c044020 con 0x7ff394003970 2023-10-18T10:44:14.103 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:44:14.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff39806af90 msgr2=0x7ff39806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff39806af90 0x7ff39806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0c9000 tx=0x7ff3a0009530 comp rx=0 tx=0).stop 2023-10-18T10:44:14.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 msgr2=0x7ff3ac0ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 0x7ff3ac0ae100 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0017a0 tx=0x7ff39c006360 comp rx=0 tx=0).stop 2023-10-18T10:44:14.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 shutdown_connections 2023-10-18T10:44:14.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff39806af90 0x7ff39806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ac155630 0x7ff3ac155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ac14f9d0 0x7ff3ac14fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 --2- 172.21.15.125:0/1848036718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff394003970 0x7ff3ac0ae100 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 >> 172.21.15.125:0/1848036718 conn(0x7ff3ac0b0670 msgr2=0x7ff3ac15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:14.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 shutdown_connections 2023-10-18T10:44:14.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.103+0000 7ff3b1a32640 1 -- 172.21.15.125:0/1848036718 wait complete. 2023-10-18T10:44:14.115 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-18T10:44:14.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.216+0000 7f6b5f305640 1 -- 172.21.15.125:0/3237958289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 msgr2=0x7f6b44003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.216+0000 7f6b5f305640 1 --2- 172.21.15.125:0/3237958289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 0x7f6b44003d70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000ade0 tx=0x7f6b440057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.216+0000 7f6b5f305640 1 -- 172.21.15.125:0/3237958289 shutdown_connections 2023-10-18T10:44:14.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.216+0000 7f6b5f305640 1 --2- 172.21.15.125:0/3237958289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 0x7f6b44003d70 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.216+0000 7f6b5f305640 1 -- 172.21.15.125:0/3237958289 >> 172.21.15.125:0/3237958289 conn(0x7f6b580d0d60 msgr2=0x7f6b580d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:14.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.217+0000 7f6b5f305640 1 -- 172.21.15.125:0/3237958289 shutdown_connections 2023-10-18T10:44:14.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.217+0000 7f6b5f305640 1 -- 172.21.15.125:0/3237958289 wait complete. 2023-10-18T10:44:14.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5f305640 1 Processor -- start 2023-10-18T10:44:14.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5f305640 1 -- start start 2023-10-18T10:44:14.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5f305640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 0x7f6b580cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5f305640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b58158250 0x7f6b580c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5f305640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b581e3510 0x7f6b581e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5f305640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580c8f00 con 0x7f6b44003970 2023-10-18T10:44:14.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5f305640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580c9170 con 0x7f6b581e3510 2023-10-18T10:44:14.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5f305640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580c9450 con 0x7f6b58158250 2023-10-18T10:44:14.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5d0a1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 0x7f6b580cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.218+0000 7f6b5d0a1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 0x7f6b580cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36940/0 (socket says 172.21.15.125:36940) 2023-10-18T10:44:14.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d0a1640 1 -- 172.21.15.125:0/1344162908 learned_addr learned my addr 172.21.15.125:0/1344162908 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:14.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d8a2640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b581e3510 0x7f6b581e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5c8a0640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b58158250 0x7f6b580c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d0a1640 1 -- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b58158250 msgr2=0x7f6b580c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d0a1640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b58158250 0x7f6b580c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d0a1640 1 -- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b581e3510 msgr2=0x7f6b581e39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d0a1640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b581e3510 0x7f6b581e39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d0a1640 1 -- 172.21.15.125:0/1344162908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b580c9730 con 0x7f6b44003970 2023-10-18T10:44:14.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d8a2640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b581e3510 0x7f6b581e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:14.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5c8a0640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b58158250 0x7f6b580c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:14.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5d0a1640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 0x7f6b580cad70 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f6b50036040 tx=0x7f6b50034f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:14.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b4dffb640 1 -- 172.21.15.125:0/1344162908 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50032910 con 0x7f6b44003970 2023-10-18T10:44:14.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b4dffb640 1 -- 172.21.15.125:0/1344162908 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b50032b80 con 0x7f6b44003970 2023-10-18T10:44:14.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b580c9a10 con 0x7f6b44003970 2023-10-18T10:44:14.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b4dffb640 1 -- 172.21.15.125:0/1344162908 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5003e420 con 0x7f6b44003970 2023-10-18T10:44:14.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.219+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b581586b0 con 0x7f6b44003970 2023-10-18T10:44:14.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.220+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b580690e0 con 0x7f6b44003970 2023-10-18T10:44:14.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.221+0000 7f6b4dffb640 1 -- 172.21.15.125:0/1344162908 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6b5003e580 con 0x7f6b44003970 2023-10-18T10:44:14.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.221+0000 7f6b4dffb640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f6b3c06af90 0x7f6b3c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.221+0000 7f6b4dffb640 1 -- 172.21.15.125:0/1344162908 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7f6b500acb40 con 0x7f6b44003970 2023-10-18T10:44:14.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.221+0000 7f6b5c8a0640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f6b3c06af90 0x7f6b3c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.222+0000 7f6b5c8a0640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f6b3c06af90 0x7f6b3c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006fd0 tx=0x7f6b48006b90 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:14.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.225+0000 7f6b4dffb640 1 -- 172.21.15.125:0/1344162908 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b50077980 con 0x7f6b44003970 2023-10-18T10:44:14.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.452+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7f6b580c8f00 con 0x7f6b44003970 2023-10-18T10:44:14.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.517+0000 7f6b4dffb640 1 -- 172.21.15.125:0/1344162908 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v724) v1 ==== 163+0+0 (secure 0 0 0) 0x7f6b5007afa0 con 0x7f6b44003970 2023-10-18T10:44:14.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f6b3c06af90 msgr2=0x7f6b3c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f6b3c06af90 0x7f6b3c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006fd0 tx=0x7f6b48006b90 comp rx=0 tx=0).stop 2023-10-18T10:44:14.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 msgr2=0x7f6b580cad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 0x7f6b580cad70 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f6b50036040 tx=0x7f6b50034f00 comp rx=0 tx=0).stop 2023-10-18T10:44:14.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 shutdown_connections 2023-10-18T10:44:14.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f6b3c06af90 0x7f6b3c06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b581e3510 0x7f6b581e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b58158250 0x7f6b580c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 --2- 172.21.15.125:0/1344162908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b44003970 0x7f6b580cad70 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 >> 172.21.15.125:0/1344162908 conn(0x7f6b580d0d60 msgr2=0x7f6b5815a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:14.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 shutdown_connections 2023-10-18T10:44:14.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.520+0000 7f6b5f305640 1 -- 172.21.15.125:0/1344162908 wait complete. 2023-10-18T10:44:14.532 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T10:44:14.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.633+0000 7f5f19dce640 1 -- 172.21.15.125:0/1023297415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 msgr2=0x7f5efc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.633+0000 7f5f19dce640 1 --2- 172.21.15.125:0/1023297415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 0x7f5efc003d90 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c260 tx=0x7f5efc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.633+0000 7f5f13fff640 1 -- 172.21.15.125:0/1023297415 reap_dead start 2023-10-18T10:44:14.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.633+0000 7f5f19dce640 1 -- 172.21.15.125:0/1023297415 shutdown_connections 2023-10-18T10:44:14.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.633+0000 7f5f19dce640 1 -- 172.21.15.125:0/1023297415 >> 172.21.15.125:0/1023297415 conn(0x7f5f14069930 msgr2=0x7f5f14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:14.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.634+0000 7f5f19dce640 1 -- 172.21.15.125:0/1023297415 shutdown_connections 2023-10-18T10:44:14.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.634+0000 7f5f19dce640 1 -- 172.21.15.125:0/1023297415 wait complete. 2023-10-18T10:44:14.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.635+0000 7f5f19dce640 1 Processor -- start 2023-10-18T10:44:14.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.635+0000 7f5f19dce640 1 -- start start 2023-10-18T10:44:14.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f19dce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 0x7f5f14135020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f19dce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f14159390 0x7f5f140ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f19dce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f141597f0 0x7f5f140cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f19dce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14149e00 con 0x7f5efc003970 2023-10-18T10:44:14.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f19dce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14135560 con 0x7f5f14159390 2023-10-18T10:44:14.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f19dce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14159d50 con 0x7f5f141597f0 2023-10-18T10:44:14.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 0x7f5f14135020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 0x7f5f14135020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36948/0 (socket says 172.21.15.125:36948) 2023-10-18T10:44:14.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 -- 172.21.15.125:0/2527003851 learned_addr learned my addr 172.21.15.125:0/2527003851 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:14.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f12ffd640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f14159390 0x7f5f140ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f13fff640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f141597f0 0x7f5f140cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 -- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f141597f0 msgr2=0x7f5f140cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f141597f0 0x7f5f140cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 -- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f14159390 msgr2=0x7f5f140ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f14159390 0x7f5f140ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 -- 172.21.15.125:0/2527003851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1415a030 con 0x7f5efc003970 2023-10-18T10:44:14.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f13fff640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f141597f0 0x7f5f140cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:14.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f137fe640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 0x7f5f14135020 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5f04000c00 tx=0x7f5f04001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:14.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5efbfff640 1 -- 172.21.15.125:0/2527003851 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04006460 con 0x7f5efc003970 2023-10-18T10:44:14.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5efbfff640 1 -- 172.21.15.125:0/2527003851 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f040065c0 con 0x7f5efc003970 2023-10-18T10:44:14.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1414a050 con 0x7f5efc003970 2023-10-18T10:44:14.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5efbfff640 1 -- 172.21.15.125:0/2527003851 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04003850 con 0x7f5efc003970 2023-10-18T10:44:14.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.636+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f14144160 con 0x7f5efc003970 2023-10-18T10:44:14.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.637+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f14144350 con 0x7f5efc003970 2023-10-18T10:44:14.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.638+0000 7f5efbfff640 1 -- 172.21.15.125:0/2527003851 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5f04031030 con 0x7f5efc003970 2023-10-18T10:44:14.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.638+0000 7f5efbfff640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5f0006af90 0x7f5f0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.638+0000 7f5efbfff640 1 -- 172.21.15.125:0/2527003851 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7f5f040aad50 con 0x7f5efc003970 2023-10-18T10:44:14.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.638+0000 7f5f12ffd640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5f0006af90 0x7f5f0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.639+0000 7f5f12ffd640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5f0006af90 0x7f5f0006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f1406c220 tx=0x7f5f08005f90 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:14.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.642+0000 7f5efbfff640 1 -- 172.21.15.125:0/2527003851 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f040ae030 con 0x7f5efc003970 2023-10-18T10:44:14.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.807+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f5f140acb30 con 0x7f5efc003970 2023-10-18T10:44:14.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.807+0000 7f5efbfff640 1 -- 172.21.15.125:0/2527003851 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v54) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f5f04045020 con 0x7f5efc003970 2023-10-18T10:44:14.810 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T10:44:14.810 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T10:44:14.810 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T10:44:14.811 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T10:44:14.811 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T10:44:14.811 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T10:44:14.811 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T10:44:14.811 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T10:44:14.812 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T10:44:14.812 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T10:44:14.812 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T10:44:14.812 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T10:44:14.812 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T10:44:14.813 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T10:44:14.813 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T10:44:14.813 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T10:44:14.813 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T10:44:14.814 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T10:44:14.814 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T10:44:14.814 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T10:44:14.814 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T10:44:14.814 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T10:44:14.815 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T10:44:14.815 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T10:44:14.815 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T10:44:14.815 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T10:44:14.815 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T10:44:14.816 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T10:44:14.816 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T10:44:14.816 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T10:44:14.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5f0006af90 msgr2=0x7f5f0006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5f0006af90 0x7f5f0006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f1406c220 tx=0x7f5f08005f90 comp rx=0 tx=0).stop 2023-10-18T10:44:14.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 msgr2=0x7f5f14135020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 0x7f5f14135020 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5f04000c00 tx=0x7f5f04001610 comp rx=0 tx=0).stop 2023-10-18T10:44:14.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 shutdown_connections 2023-10-18T10:44:14.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5f0006af90 0x7f5f0006d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f14159390 0x7f5f140ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f141597f0 0x7f5f140cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 --2- 172.21.15.125:0/2527003851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5efc003970 0x7f5f14135020 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 >> 172.21.15.125:0/2527003851 conn(0x7f5f14069930 msgr2=0x7f5f14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:14.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.809+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 shutdown_connections 2023-10-18T10:44:14.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.810+0000 7f5f19dce640 1 -- 172.21.15.125:0/2527003851 wait complete. 2023-10-18T10:44:14.822 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:44:14.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.925+0000 7f210e1a9640 1 -- 172.21.15.125:0/4130132744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f20f0003ef0 msgr2=0x7f20f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.925+0000 7f210e1a9640 1 --2- 172.21.15.125:0/4130132744 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f20f0003ef0 0x7f20f00047c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00c260 tx=0x7f20f00057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.925+0000 7f2107fff640 1 -- 172.21.15.125:0/4130132744 reap_dead start 2023-10-18T10:44:14.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.925+0000 7f210e1a9640 1 -- 172.21.15.125:0/4130132744 shutdown_connections 2023-10-18T10:44:14.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.925+0000 7f210e1a9640 1 -- 172.21.15.125:0/4130132744 >> 172.21.15.125:0/4130132744 conn(0x7f2108069930 msgr2=0x7f2108069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:14.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.925+0000 7f210e1a9640 1 -- 172.21.15.125:0/4130132744 shutdown_connections 2023-10-18T10:44:14.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.925+0000 7f210e1a9640 1 -- 172.21.15.125:0/4130132744 wait complete. 2023-10-18T10:44:14.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.926+0000 7f210e1a9640 1 Processor -- start 2023-10-18T10:44:14.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f210e1a9640 1 -- start start 2023-10-18T10:44:14.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f210e1a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f0003ef0 0x7f21080addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f210e1a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2108154f60 0x7f21081553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f210e1a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2108152770 0x7f2108152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f210e1a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210814f2e0 con 0x7f2108154f60 2023-10-18T10:44:14.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f210e1a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210814f5c0 con 0x7f20f0003ef0 2023-10-18T10:44:14.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f210e1a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210814f8a0 con 0x7f2108152770 2023-10-18T10:44:14.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2108154f60 0x7f21081553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f2106ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2108154f60 0x7f21081553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36970/0 (socket says 172.21.15.125:36970) 2023-10-18T10:44:14.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.927+0000 7f2106ffd640 1 -- 172.21.15.125:0/375623596 learned_addr learned my addr 172.21.15.125:0/375623596 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:14.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f2107fff640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2108152770 0x7f2108152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f21077fe640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f0003ef0 0x7f21080addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f2106ffd640 1 -- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2108152770 msgr2=0x7f2108152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f2106ffd640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2108152770 0x7f2108152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f2106ffd640 1 -- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f0003ef0 msgr2=0x7f21080addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:14.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f2106ffd640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f0003ef0 0x7f21080addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:14.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f2106ffd640 1 -- 172.21.15.125:0/375623596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210814fb80 con 0x7f2108154f60 2023-10-18T10:44:14.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f21077fe640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f0003ef0 0x7f21080addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:14.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f2106ffd640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2108154f60 0x7f21081553c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00c380 tx=0x7f20fc001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:14.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f20effff640 1 -- 172.21.15.125:0/375623596 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc006230 con 0x7f2108154f60 2023-10-18T10:44:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f20effff640 1 -- 172.21.15.125:0/375623596 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f20fc006390 con 0x7f2108154f60 2023-10-18T10:44:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210814fe60 con 0x7f2108154f60 2023-10-18T10:44:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f20effff640 1 -- 172.21.15.125:0/375623596 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0035f0 con 0x7f2108154f60 2023-10-18T10:44:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.928+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21080ae330 con 0x7f2108154f60 2023-10-18T10:44:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.929+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21080ae520 con 0x7f2108154f60 2023-10-18T10:44:14.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.930+0000 7f20effff640 1 -- 172.21.15.125:0/375623596 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f20fc031030 con 0x7f2108154f60 2023-10-18T10:44:14.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.930+0000 7f20effff640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f20f406af90 0x7f20f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:14.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.930+0000 7f20effff640 1 -- 172.21.15.125:0/375623596 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7f20fc0aa8c0 con 0x7f2108154f60 2023-10-18T10:44:14.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.930+0000 7f21077fe640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f20f406af90 0x7f20f406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:14.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.931+0000 7f21077fe640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f20f406af90 0x7f20f406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009bc0 tx=0x7f20f8005f90 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:14.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:14.934+0000 7f20effff640 1 -- 172.21.15.125:0/375623596 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20fc078fe0 con 0x7f2108154f60 2023-10-18T10:44:15.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.101+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2108067930 con 0x7f2108154f60 2023-10-18T10:44:15.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.104+0000 7f20effff640 1 -- 172.21.15.125:0/375623596 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v55)=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7f20fc043020 con 0x7f2108154f60 2023-10-18T10:44:15.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f20f406af90 msgr2=0x7f20f406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f20f406af90 0x7f20f406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009bc0 tx=0x7f20f8005f90 comp rx=0 tx=0).stop 2023-10-18T10:44:15.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2108154f60 msgr2=0x7f21081553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2108154f60 0x7f21081553c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00c380 tx=0x7f20fc001700 comp rx=0 tx=0).stop 2023-10-18T10:44:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 shutdown_connections 2023-10-18T10:44:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f20f406af90 0x7f20f406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2108152770 0x7f2108152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2108154f60 0x7f21081553c0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 --2- 172.21.15.125:0/375623596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f0003ef0 0x7f21080addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 >> 172.21.15.125:0/375623596 conn(0x7f2108069930 msgr2=0x7f21080ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:15.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 shutdown_connections 2023-10-18T10:44:15.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.106+0000 7f210e1a9640 1 -- 172.21.15.125:0/375623596 wait complete. 2023-10-18T10:44:15.119 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:44:15.119 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:44:15.169 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:44:15.232 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:44:15.233 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:44:15.233 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:15.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:44:15.249 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:44:15.249 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:15.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:44:15.268 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:44:15.269 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:15.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:44:15.304 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:44:15.304 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:15.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:44:15.325 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:44:15.326 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:15.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.445+0000 7f24b9875640 1 -- 172.21.15.125:0/2352616989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b40b1ca0 msgr2=0x7f24b40b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.445+0000 7f24b9875640 1 --2- 172.21.15.125:0/2352616989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b40b1ca0 0x7f24b40b42d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009830 tx=0x7f24a402bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:15.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.445+0000 7f24b9875640 1 -- 172.21.15.125:0/2352616989 shutdown_connections 2023-10-18T10:44:15.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.445+0000 7f24b9875640 1 --2- 172.21.15.125:0/2352616989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b40b1ca0 0x7f24b40b42d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.445+0000 7f24b9875640 1 --2- 172.21.15.125:0/2352616989 >> v2:172.21.15.139:3300/0 conn(0x7f24b40cf8b0 0x7f24b40ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.445+0000 7f24b9875640 1 -- 172.21.15.125:0/2352616989 >> 172.21.15.125:0/2352616989 conn(0x7f24b40d0c50 msgr2=0x7f24b40d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:15.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.445+0000 7f24b9875640 1 -- 172.21.15.125:0/2352616989 shutdown_connections 2023-10-18T10:44:15.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.445+0000 7f24b9875640 1 -- 172.21.15.125:0/2352616989 wait complete. 2023-10-18T10:44:15.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.446+0000 7f24b9875640 1 Processor -- start 2023-10-18T10:44:15.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.446+0000 7f24b9875640 1 -- start start 2023-10-18T10:44:15.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b9875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24b40b1ca0 0x7f24b414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:15.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b9875640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b40cf8b0 0x7f24b41597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:15.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b9875640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b415e5e0 0x7f24b4159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:15.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b9875640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40c5bc0 con 0x7f24b40b1ca0 2023-10-18T10:44:15.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b9875640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b415a340 con 0x7f24b415e5e0 2023-10-18T10:44:15.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b9875640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b415a620 con 0x7f24b40cf8b0 2023-10-18T10:44:15.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24b40b1ca0 0x7f24b414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:15.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24b40b1ca0 0x7f24b414a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36972/0 (socket says 172.21.15.125:36972) 2023-10-18T10:44:15.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 -- 172.21.15.125:0/3571684924 learned_addr learned my addr 172.21.15.125:0/3571684924 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:15.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b27fc640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b40cf8b0 0x7f24b41597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:15.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b37fe640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b415e5e0 0x7f24b4159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:15.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 -- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b40cf8b0 msgr2=0x7f24b41597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b40cf8b0 0x7f24b41597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 -- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b415e5e0 msgr2=0x7f24b4159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b415e5e0 0x7f24b4159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 -- 172.21.15.125:0/3571684924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8009840 con 0x7f24b40b1ca0 2023-10-18T10:44:15.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.447+0000 7f24b2ffd640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24b40b1ca0 0x7f24b414a080 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00c390 tx=0x7f24ac00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:15.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.448+0000 7f249bfff640 1 -- 172.21.15.125:0/3571684924 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac004230 con 0x7f24b40b1ca0 2023-10-18T10:44:15.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.448+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a4009490 con 0x7f24b40b1ca0 2023-10-18T10:44:15.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.448+0000 7f249bfff640 1 -- 172.21.15.125:0/3571684924 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24ac0044a0 con 0x7f24b40b1ca0 2023-10-18T10:44:15.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.448+0000 7f249bfff640 1 -- 172.21.15.125:0/3571684924 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac003850 con 0x7f24b40b1ca0 2023-10-18T10:44:15.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.448+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b40c5850 con 0x7f24b40b1ca0 2023-10-18T10:44:15.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.449+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b40690e0 con 0x7f24b40b1ca0 2023-10-18T10:44:15.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.449+0000 7f249bfff640 1 -- 172.21.15.125:0/3571684924 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f24ac0039f0 con 0x7f24b40b1ca0 2023-10-18T10:44:15.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.449+0000 7f249bfff640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f24a006af60 0x7f24a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:15.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.449+0000 7f249bfff640 1 -- 172.21.15.125:0/3571684924 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7f24ac089050 con 0x7f24b40b1ca0 2023-10-18T10:44:15.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.450+0000 7f24b27fc640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f24a006af60 0x7f24a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:15.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.450+0000 7f24b27fc640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f24a006af60 0x7f24a006d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24b40cf580 tx=0x7f24a8009500 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:15.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.454+0000 7f249bfff640 1 -- 172.21.15.125:0/3571684924 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24ac054000 con 0x7f24b40b1ca0 2023-10-18T10:44:15.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.654+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24b4069ca0 con 0x7f24b40b1ca0 2023-10-18T10:44:15.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.655+0000 7f249bfff640 1 -- 172.21.15.125:0/3571684924 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f24ac057620 con 0x7f24b40b1ca0 2023-10-18T10:44:15.657 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:15.657 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/4122761343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4122761343},{"type":"v1","addr":"172.21.15.139:6836","nonce":4122761343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6785,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1014994333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1014994333},{"type":"v1","addr":"172.21.15.139:6837","nonce":1014994333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3090232356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3090232356},{"type":"v1","addr":"172.21.15.125:6834","nonce":3090232356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":9078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/536374845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":536374845},{"type":"v1","addr":"172.21.15.125:6835","nonce":536374845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[]} 2023-10-18T10:44:15.658 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 49 2023-10-18T10:44:15.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f24a006af60 msgr2=0x7f24a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f24a006af60 0x7f24a006d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24b40cf580 tx=0x7f24a8009500 comp rx=0 tx=0).stop 2023-10-18T10:44:15.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24b40b1ca0 msgr2=0x7f24b414a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24b40b1ca0 0x7f24b414a080 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00c390 tx=0x7f24ac00c850 comp rx=0 tx=0).stop 2023-10-18T10:44:15.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 shutdown_connections 2023-10-18T10:44:15.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f24a006af60 0x7f24a006d420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b415e5e0 0x7f24b4159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b40cf8b0 0x7f24b41597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 --2- 172.21.15.125:0/3571684924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24b40b1ca0 0x7f24b414a080 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 >> 172.21.15.125:0/3571684924 conn(0x7f24b40d0c50 msgr2=0x7f24b415b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:15.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 shutdown_connections 2023-10-18T10:44:15.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.657+0000 7f24b9875640 1 -- 172.21.15.125:0/3571684924 wait complete. 2023-10-18T10:44:15.670 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:44:15.670 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:44:15.670 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:44:15.671 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:44:15.671 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:44:15.671 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:44:15.672 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:44:15.672 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:44:15.672 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:44:15.672 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:44:15.673 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:44:15.673 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:44:15.673 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:44:15.673 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:44:15.674 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:44:15.674 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:44:15.680 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:44:15.681 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:44:15.682 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:44:15.682 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:44:15.684 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:44:15.684 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:44:15.685 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:44:15.685 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:44:15.724 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:44:15.725 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:15.726 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:44:15.727 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:15.727 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:44:15.728 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:15.728 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:44:15.729 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:15.729 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:44:15.799 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:44:15.800 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:44:15.808 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:44:15.808 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:44:15.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.837+0000 7f21e53f5640 1 -- 172.21.15.125:0/237228351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21cc0047c0 msgr2=0x7f21cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.837+0000 7f21e53f5640 1 --2- 172.21.15.125:0/237228351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21cc0047c0 0x7f21cc004c40 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009bc0 tx=0x7f21cc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.837+0000 7f21df7fe640 1 -- 172.21.15.125:0/237228351 reap_dead start 2023-10-18T10:44:15.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.837+0000 7f21e53f5640 1 -- 172.21.15.125:0/237228351 shutdown_connections 2023-10-18T10:44:15.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.837+0000 7f21e53f5640 1 -- 172.21.15.125:0/237228351 >> 172.21.15.125:0/237228351 conn(0x7f21e00d0d60 msgr2=0x7f21e00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:15.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.837+0000 7f21e53f5640 1 -- 172.21.15.125:0/237228351 shutdown_connections 2023-10-18T10:44:15.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.837+0000 7f21e53f5640 1 -- 172.21.15.125:0/237228351 wait complete. 2023-10-18T10:44:15.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.838+0000 7f21e53f5640 1 Processor -- start 2023-10-18T10:44:15.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.838+0000 7f21e53f5640 1 -- start start 2023-10-18T10:44:15.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21e53f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc0047c0 0x7f21e01f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:15.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21e53f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21e006aab0 0x7f21e0156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:15.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21e53f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21e00c8840 0x7f21e00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:15.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21e53f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c9cd0 con 0x7f21e006aab0 2023-10-18T10:44:15.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21e53f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c90c0 con 0x7f21e00c8840 2023-10-18T10:44:15.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21e53f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c93a0 con 0x7f21cc0047c0 2023-10-18T10:44:15.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21e006aab0 0x7f21e0156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:15.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21e006aab0 0x7f21e0156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37022/0 (socket says 172.21.15.125:37022) 2023-10-18T10:44:15.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 -- 172.21.15.125:0/1698022914 learned_addr learned my addr 172.21.15.125:0/1698022914 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:15.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21df7fe640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21e00c8840 0x7f21e00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:15.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21deffd640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc0047c0 0x7f21e01f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:15.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 -- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc0047c0 msgr2=0x7f21e01f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc0047c0 0x7f21e01f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 -- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21e00c8840 msgr2=0x7f21e00b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:15.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21e00c8840 0x7f21e00b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:15.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 -- 172.21.15.125:0/1698022914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e00c9680 con 0x7f21e006aab0 2023-10-18T10:44:15.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.839+0000 7f21d7fff640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21e006aab0 0x7f21e0156e10 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009ce0 tx=0x7f21c8033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:15.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.840+0000 7f21d77fe640 1 -- 172.21.15.125:0/1698022914 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8004220 con 0x7f21e006aab0 2023-10-18T10:44:15.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.840+0000 7f21d77fe640 1 -- 172.21.15.125:0/1698022914 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f21c8004470 con 0x7f21e006aab0 2023-10-18T10:44:15.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.840+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e00c9960 con 0x7f21e006aab0 2023-10-18T10:44:15.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.840+0000 7f21d77fe640 1 -- 172.21.15.125:0/1698022914 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8005260 con 0x7f21e006aab0 2023-10-18T10:44:15.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.840+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e00c8ca0 con 0x7f21e006aab0 2023-10-18T10:44:15.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.841+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e00c9e80 con 0x7f21e006aab0 2023-10-18T10:44:15.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.841+0000 7f21d77fe640 1 -- 172.21.15.125:0/1698022914 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f21c80053c0 con 0x7f21e006aab0 2023-10-18T10:44:15.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.841+0000 7f21d77fe640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21c006af60 0x7f21c006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:15.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.841+0000 7f21d77fe640 1 -- 172.21.15.125:0/1698022914 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6065+0+0 (secure 0 0 0) 0x7f21c80ac4d0 con 0x7f21e006aab0 2023-10-18T10:44:15.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.842+0000 7f21deffd640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21c006af60 0x7f21c006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:15.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.842+0000 7f21deffd640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21c006af60 0x7f21c006d420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f21d0001200 tx=0x7f21d0001e50 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:15.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:15.846+0000 7f21d77fe640 1 -- 172.21.15.125:0/1698022914 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21e00c9e80 con 0x7f21e006aab0 2023-10-18T10:44:16.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:16.024+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f21e00690e0 con 0x7f21e006aab0 2023-10-18T10:44:17.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.126+0000 7f21d77fe640 1 -- 172.21.15.125:0/1698022914 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) v1 ==== 97+0+0 (secure 0 0 0) 0x7f21c8077220 con 0x7f21e006aab0 2023-10-18T10:44:17.129 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T10:44:17.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21c006af60 msgr2=0x7f21c006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21c006af60 0x7f21c006d420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f21d0001200 tx=0x7f21d0001e50 comp rx=0 tx=0).stop 2023-10-18T10:44:17.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21e006aab0 msgr2=0x7f21e0156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21e006aab0 0x7f21e0156e10 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009ce0 tx=0x7f21c8033c30 comp rx=0 tx=0).stop 2023-10-18T10:44:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 shutdown_connections 2023-10-18T10:44:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21c006af60 0x7f21c006d420 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21e00c8840 0x7f21e00b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21e006aab0 0x7f21e0156e10 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 --2- 172.21.15.125:0/1698022914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc0047c0 0x7f21e01f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 >> 172.21.15.125:0/1698022914 conn(0x7f21e00d0d60 msgr2=0x7f21e015a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:17.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 shutdown_connections 2023-10-18T10:44:17.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.130+0000 7f21e53f5640 1 -- 172.21.15.125:0/1698022914 wait complete. 2023-10-18T10:44:17.145 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:44:17.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.250+0000 7faaa9b55640 1 -- 172.21.15.125:0/2340609587 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa40d1910 msgr2=0x7faaa40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.250+0000 7faaa9b55640 1 --2- 172.21.15.125:0/2340609587 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa40d1910 0x7faaa40b0650 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7faa9c005fd0 tx=0x7faa9c02c050 comp rx=0 tx=0).stop 2023-10-18T10:44:17.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.250+0000 7faaa9b55640 1 -- 172.21.15.125:0/2340609587 shutdown_connections 2023-10-18T10:44:17.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.250+0000 7faaa9b55640 1 --2- 172.21.15.125:0/2340609587 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa40d1910 0x7faaa40b0650 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.250+0000 7faaa9b55640 1 -- 172.21.15.125:0/2340609587 >> 172.21.15.125:0/2340609587 conn(0x7faaa40d0d60 msgr2=0x7faaa40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:17.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.250+0000 7faaa9b55640 1 -- 172.21.15.125:0/2340609587 shutdown_connections 2023-10-18T10:44:17.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.250+0000 7faaa9b55640 1 -- 172.21.15.125:0/2340609587 wait complete. 2023-10-18T10:44:17.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.251+0000 7faaa9b55640 1 Processor -- start 2023-10-18T10:44:17.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa9b55640 1 -- start start 2023-10-18T10:44:17.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa9b55640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa40cf340 0x7faaa415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa9b55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa40d1910 0x7faaa40c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa9b55640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa415b2a0 0x7faaa414bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa9b55640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa415b820 con 0x7faaa415b2a0 2023-10-18T10:44:17.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa9b55640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa415bb00 con 0x7faaa40d1910 2023-10-18T10:44:17.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa9b55640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa415bde0 con 0x7faaa40cf340 2023-10-18T10:44:17.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa415b2a0 0x7faaa414bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa415b2a0 0x7faaa414bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37042/0 (socket says 172.21.15.125:37042) 2023-10-18T10:44:17.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.252+0000 7faaa3fff640 1 -- 172.21.15.125:0/3927063409 learned_addr learned my addr 172.21.15.125:0/3927063409 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:17.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faa9affd640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa40d1910 0x7faaa40c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa37fe640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa40cf340 0x7faaa415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa3fff640 1 -- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa40cf340 msgr2=0x7faaa415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa3fff640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa40cf340 0x7faaa415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa3fff640 1 -- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa40d1910 msgr2=0x7faaa40c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa3fff640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa40d1910 0x7faaa40c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa3fff640 1 -- 172.21.15.125:0/3927063409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c009840 con 0x7faaa415b2a0 2023-10-18T10:44:17.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faa9affd640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa40d1910 0x7faaa40c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa3fff640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa415b2a0 0x7faaa414bd60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7faa9c036040 tx=0x7faa9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa0ff9640 1 -- 172.21.15.125:0/3927063409 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c03e460 con 0x7faaa415b2a0 2023-10-18T10:44:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa0ff9640 1 -- 172.21.15.125:0/3927063409 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa9c03e6b0 con 0x7faaa415b2a0 2023-10-18T10:44:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa0ff9640 1 -- 172.21.15.125:0/3927063409 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c03eac0 con 0x7faaa415b2a0 2023-10-18T10:44:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa415c0c0 con 0x7faaa415b2a0 2023-10-18T10:44:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.253+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa4154ae0 con 0x7faaa415b2a0 2023-10-18T10:44:17.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.254+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa40677b0 con 0x7faaa415b2a0 2023-10-18T10:44:17.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.254+0000 7faaa0ff9640 1 -- 172.21.15.125:0/3927063409 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faa9c03ec20 con 0x7faaa415b2a0 2023-10-18T10:44:17.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.255+0000 7faaa0ff9640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7faa8406af60 0x7faa8406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.255+0000 7faaa0ff9640 1 -- 172.21.15.125:0/3927063409 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5205+0+0 (secure 0 0 0) 0x7faa9c0ac6e0 con 0x7faaa415b2a0 2023-10-18T10:44:17.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.255+0000 7faaa37fe640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7faa8406af60 0x7faa8406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.256+0000 7faaa37fe640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7faa8406af60 0x7faa8406d420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa94004ea0 tx=0x7faa94009040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:17.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.259+0000 7faaa0ff9640 1 -- 172.21.15.125:0/3927063409 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa9c077900 con 0x7faaa415b2a0 2023-10-18T10:44:17.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.487+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faaa415c580 con 0x7faaa415b2a0 2023-10-18T10:44:17.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.488+0000 7faaa0ff9640 1 -- 172.21.15.125:0/3927063409 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7faa9c042090 con 0x7faaa415b2a0 2023-10-18T10:44:17.491 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:17.491 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:44:17.491 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T10:44:17.492 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.492 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T10:44:17.492 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T10:44:17.493 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.493 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:17.493 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:17.493 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:17.493 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.494 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.494 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.494 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.494 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T10:44:17.494 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T10:44:17.495 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.495 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:17.495 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:17.495 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:17.495 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.496 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.496 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.496 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.496 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T10:44:17.497 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T10:44:17.497 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.497 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:17.497 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:17.497 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:17.498 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.498 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.498 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.498 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.498 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T10:44:17.499 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T10:44:17.499 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.499 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:17.499 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:17.499 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:17.500 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.500 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.500 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.500 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.500 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T10:44:17.501 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T10:44:17.501 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.501 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:17.501 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:17.501 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.502 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.502 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.502 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.502 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T10:44:17.502 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T10:44:17.503 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.503 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:17.503 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:17.503 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.503 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.504 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.504 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.504 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T10:44:17.504 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T10:44:17.504 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.505 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:17.505 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:17.505 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.505 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.505 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.505 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.506 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T10:44:17.506 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T10:44:17.506 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.506 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:17.507 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:17.507 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.507 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.507 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.507 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.508 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T10:44:17.508 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T10:44:17.508 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.508 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:17.509 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:17.509 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.509 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.509 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.510 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.510 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T10:44:17.510 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T10:44:17.510 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.510 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:17.511 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:17.511 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.511 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.511 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.511 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.512 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T10:44:17.512 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T10:44:17.512 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.512 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:17.512 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:17.513 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.513 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.513 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.513 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.513 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T10:44:17.513 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T10:44:17.514 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.514 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:17.514 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:17.514 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.514 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.515 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.515 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.515 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T10:44:17.515 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T10:44:17.515 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.516 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:17.516 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T10:44:17.516 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T10:44:17.516 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.516 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.517 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.517 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T10:44:17.517 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T10:44:17.517 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.517 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:17.518 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:44:17.518 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:44:17.518 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:44:17.518 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.518 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.519 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.519 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T10:44:17.519 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T10:44:17.519 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.519 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:17.519 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:44:17.520 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:44:17.520 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:44:17.520 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.520 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.520 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.521 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T10:44:17.521 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T10:44:17.521 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.521 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:17.521 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:44:17.522 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:44:17.522 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:44:17.522 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.522 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.522 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.523 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T10:44:17.523 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T10:44:17.523 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.523 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:44:17.523 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:17.524 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:17.524 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.524 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.524 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.524 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.524 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T10:44:17.525 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T10:44:17.525 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.525 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:44:17.525 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:44:17.525 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.526 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.526 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:17.526 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:17.526 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T10:44:17.526 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T10:44:17.527 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:17.527 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:44:17.527 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:44:17.527 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:17.527 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.528 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:17.528 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T10:44:17.528 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:44:17.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7faa8406af60 msgr2=0x7faa8406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7faa8406af60 0x7faa8406d420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa94004ea0 tx=0x7faa94009040 comp rx=0 tx=0).stop 2023-10-18T10:44:17.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa415b2a0 msgr2=0x7faaa414bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa415b2a0 0x7faaa414bd60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7faa9c036040 tx=0x7faa9c0042f0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 shutdown_connections 2023-10-18T10:44:17.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7faa8406af60 0x7faa8406d420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa40d1910 0x7faaa40c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa415b2a0 0x7faaa414bd60 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 --2- 172.21.15.125:0/3927063409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa40cf340 0x7faaa415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.489+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 >> 172.21.15.125:0/3927063409 conn(0x7faaa40d0d60 msgr2=0x7faaa40ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:17.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.490+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 shutdown_connections 2023-10-18T10:44:17.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.490+0000 7faaa9b55640 1 -- 172.21.15.125:0/3927063409 wait complete. 2023-10-18T10:44:17.532 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:17.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.632+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/3069888887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 msgr2=0x7fdbe40ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.632+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/3069888887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 0x7fdbe40ab990 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40ca260 tx=0x7fdbdc02c110 comp rx=0 tx=0).stop 2023-10-18T10:44:17.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.632+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/3069888887 shutdown_connections 2023-10-18T10:44:17.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.632+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/3069888887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 0x7fdbe40ab990 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.632+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/3069888887 >> v2:172.21.15.139:3300/0 conn(0x7fdbe40c9dc0 0x7fdbe406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.632+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/3069888887 >> 172.21.15.125:0/3069888887 conn(0x7fdbe4069930 msgr2=0x7fdbe4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:17.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.632+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/3069888887 shutdown_connections 2023-10-18T10:44:17.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.632+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/3069888887 wait complete. 2023-10-18T10:44:17.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.634+0000 7fdbe8fc0640 1 Processor -- start 2023-10-18T10:44:17.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.634+0000 7fdbe8fc0640 1 -- start start 2023-10-18T10:44:17.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.634+0000 7fdbe8fc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe406af00 0x7fdbe4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.634+0000 7fdbe8fc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 0x7fdbe413a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.634+0000 7fdbe8fc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbe40c9dc0 0x7fdbe40aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.634+0000 7fdbe8fc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40afac0 con 0x7fdbe40ab500 2023-10-18T10:44:17.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.634+0000 7fdbe8fc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe4159070 con 0x7fdbe406af00 2023-10-18T10:44:17.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.634+0000 7fdbe8fc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe413a850 con 0x7fdbe40c9dc0 2023-10-18T10:44:17.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 0x7fdbe413a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 0x7fdbe413a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55430/0 (socket says 172.21.15.125:55430) 2023-10-18T10:44:17.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 -- 172.21.15.125:0/2642493915 learned_addr learned my addr 172.21.15.125:0/2642493915 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:17.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbe2d9d640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbe40c9dc0 0x7fdbe40aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbe259c640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe406af00 0x7fdbe4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 -- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbe40c9dc0 msgr2=0x7fdbe40aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbe40c9dc0 0x7fdbe40aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 -- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe406af00 msgr2=0x7fdbe4158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe406af00 0x7fdbe4158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 -- 172.21.15.125:0/2642493915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc009840 con 0x7fdbe40ab500 2023-10-18T10:44:17.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbe259c640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe406af00 0x7fdbe4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:17.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdbfff640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 0x7fdbe413a310 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00d770 tx=0x7fdbcc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:17.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdaffd640 1 -- 172.21.15.125:0/2642493915 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc009c30 con 0x7fdbe40ab500 2023-10-18T10:44:17.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdaffd640 1 -- 172.21.15.125:0/2642493915 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdbcc00de80 con 0x7fdbe40ab500 2023-10-18T10:44:17.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbdaffd640 1 -- 172.21.15.125:0/2642493915 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc004260 con 0x7fdbe40ab500 2023-10-18T10:44:17.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe413d920 con 0x7fdbe40ab500 2023-10-18T10:44:17.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.635+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe40af230 con 0x7fdbe40ab500 2023-10-18T10:44:17.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.636+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe40690e0 con 0x7fdbe40ab500 2023-10-18T10:44:17.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.637+0000 7fdbdaffd640 1 -- 172.21.15.125:0/2642493915 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdbcc005020 con 0x7fdbe40ab500 2023-10-18T10:44:17.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.637+0000 7fdbdaffd640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdbc406af60 0x7fdbc406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.637+0000 7fdbdaffd640 1 -- 172.21.15.125:0/2642493915 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5205+0+0 (secure 0 0 0) 0x7fdbcc0899a0 con 0x7fdbe40ab500 2023-10-18T10:44:17.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.637+0000 7fdbe259c640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdbc406af60 0x7fdbc406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.638+0000 7fdbe259c640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdbc406af60 0x7fdbc406d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4000f60 tx=0x7fdbd4009040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:17.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.641+0000 7fdbdaffd640 1 -- 172.21.15.125:0/2642493915 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdbcc054a30 con 0x7fdbe40ab500 2023-10-18T10:44:17.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.810+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbe40675f0 con 0x7fdbe40ab500 2023-10-18T10:44:17.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.811+0000 7fdbdaffd640 1 -- 172.21.15.125:0/2642493915 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+13164 (secure 0 0 0) 0x7fdbcc017090 con 0x7fdbe40ab500 2023-10-18T10:44:17.813 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:17.813 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":139,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:17.126123+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:17.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdbc406af60 msgr2=0x7fdbc406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdbc406af60 0x7fdbc406d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4000f60 tx=0x7fdbd4009040 comp rx=0 tx=0).stop 2023-10-18T10:44:17.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 msgr2=0x7fdbe413a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 0x7fdbe413a310 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00d770 tx=0x7fdbcc00dc30 comp rx=0 tx=0).stop 2023-10-18T10:44:17.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 shutdown_connections 2023-10-18T10:44:17.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdbc406af60 0x7fdbc406d420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbe40c9dc0 0x7fdbe40aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbe40ab500 0x7fdbe413a310 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 --2- 172.21.15.125:0/2642493915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe406af00 0x7fdbe4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 >> 172.21.15.125:0/2642493915 conn(0x7fdbe4069930 msgr2=0x7fdbe4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:17.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 shutdown_connections 2023-10-18T10:44:17.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.813+0000 7fdbe8fc0640 1 -- 172.21.15.125:0/2642493915 wait complete. 2023-10-18T10:44:17.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:44:17.827 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:44:17.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 -- 172.21.15.125:0/2738805311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b640 msgr2=0x7effe80cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 --2- 172.21.15.125:0/2738805311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b640 0x7effe80cf850 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7effe80cb6e0 tx=0x7effdc02bda0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 -- 172.21.15.125:0/2738805311 shutdown_connections 2023-10-18T10:44:17.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 --2- 172.21.15.125:0/2738805311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b640 0x7effe80cf850 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 --2- 172.21.15.125:0/2738805311 >> v2:172.21.15.139:3300/0 conn(0x7effe80b0b10 0x7effe80b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 --2- 172.21.15.125:0/2738805311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effe815b260 0x7effe80cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 -- 172.21.15.125:0/2738805311 >> 172.21.15.125:0/2738805311 conn(0x7effe809a030 msgr2=0x7effe80d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:17.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 -- 172.21.15.125:0/2738805311 shutdown_connections 2023-10-18T10:44:17.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.930+0000 7effed316640 1 -- 172.21.15.125:0/2738805311 wait complete. 2023-10-18T10:44:17.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.931+0000 7effed316640 1 Processor -- start 2023-10-18T10:44:17.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.931+0000 7effed316640 1 -- start start 2023-10-18T10:44:17.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effed316640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effe80b0b10 0x7effe806ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effed316640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b260 0x7effe80c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effed316640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effe815b640 0x7effe80c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effed316640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80c9570 con 0x7effe815b260 2023-10-18T10:44:17.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effed316640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80c9a00 con 0x7effe80b0b10 2023-10-18T10:44:17.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effed316640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe80c8960 con 0x7effe815b640 2023-10-18T10:44:17.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b260 0x7effe80c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b260 0x7effe80c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55446/0 (socket says 172.21.15.125:55446) 2023-10-18T10:44:17.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe67fc640 1 -- 172.21.15.125:0/913114116 learned_addr learned my addr 172.21.15.125:0/913114116 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:17.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe77fe640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effe815b640 0x7effe80c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe6ffd640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effe80b0b10 0x7effe806ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe67fc640 1 -- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effe815b640 msgr2=0x7effe80c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe67fc640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effe815b640 0x7effe80c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe67fc640 1 -- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effe80b0b10 msgr2=0x7effe806ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:17.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe67fc640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effe80b0b10 0x7effe806ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:17.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.932+0000 7effe67fc640 1 -- 172.21.15.125:0/913114116 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc009560 con 0x7effe815b260 2023-10-18T10:44:17.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.933+0000 7effe67fc640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b260 0x7effe80c7df0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7effdc000c00 tx=0x7effdc02dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:17.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.933+0000 7effcf7fe640 1 -- 172.21.15.125:0/913114116 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc005670 con 0x7effe815b260 2023-10-18T10:44:17.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.933+0000 7effcf7fe640 1 -- 172.21.15.125:0/913114116 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7effdc0038f0 con 0x7effe815b260 2023-10-18T10:44:17.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.933+0000 7effed316640 1 -- 172.21.15.125:0/913114116 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe80c8c40 con 0x7effe815b260 2023-10-18T10:44:17.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.933+0000 7effcf7fe640 1 -- 172.21.15.125:0/913114116 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc003ce0 con 0x7effe815b260 2023-10-18T10:44:17.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.933+0000 7effed316640 1 -- 172.21.15.125:0/913114116 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe80c9200 con 0x7effe815b260 2023-10-18T10:44:17.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.934+0000 7effed316640 1 -- 172.21.15.125:0/913114116 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe80c9720 con 0x7effe815b260 2023-10-18T10:44:17.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.934+0000 7effcf7fe640 1 -- 172.21.15.125:0/913114116 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7effdc03e450 con 0x7effe815b260 2023-10-18T10:44:17.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.935+0000 7effcf7fe640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7effd406af60 0x7effd406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:17.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.935+0000 7effcf7fe640 1 -- 172.21.15.125:0/913114116 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5205+0+0 (secure 0 0 0) 0x7effdc0abf00 con 0x7effe815b260 2023-10-18T10:44:17.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.935+0000 7effe6ffd640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7effd406af60 0x7effd406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:17.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.935+0000 7effe6ffd640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7effd406af60 0x7effd406d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effe80af980 tx=0x7effd800d040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:17.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:17.939+0000 7effcf7fe640 1 -- 172.21.15.125:0/913114116 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effe80c9720 con 0x7effe815b260 2023-10-18T10:44:18.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.109+0000 7effed316640 1 -- 172.21.15.125:0/913114116 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effe80675f0 con 0x7effe815b260 2023-10-18T10:44:18.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.109+0000 7effcf7fe640 1 -- 172.21.15.125:0/913114116 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7effdc077260 con 0x7effe815b260 2023-10-18T10:44:18.112 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:44:18.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 -- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7effd406af60 msgr2=0x7effd406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7effd406af60 0x7effd406d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effe80af980 tx=0x7effd800d040 comp rx=0 tx=0).stop 2023-10-18T10:44:18.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 -- 172.21.15.125:0/913114116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b260 msgr2=0x7effe80c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b260 0x7effe80c7df0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7effdc000c00 tx=0x7effdc02dd70 comp rx=0 tx=0).stop 2023-10-18T10:44:18.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 -- 172.21.15.125:0/913114116 shutdown_connections 2023-10-18T10:44:18.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7effd406af60 0x7effd406d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effe815b640 0x7effe80c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effe815b260 0x7effe80c7df0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 --2- 172.21.15.125:0/913114116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effe80b0b10 0x7effe806ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 -- 172.21.15.125:0/913114116 >> 172.21.15.125:0/913114116 conn(0x7effe809a030 msgr2=0x7effe8069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:18.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 -- 172.21.15.125:0/913114116 shutdown_connections 2023-10-18T10:44:18.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.111+0000 7effed316640 1 -- 172.21.15.125:0/913114116 wait complete. 2023-10-18T10:44:18.124 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:44:18.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.224+0000 7fe329f97640 1 -- 172.21.15.125:0/3326759623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 msgr2=0x7fe310004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.224+0000 7fe329f97640 1 --2- 172.21.15.125:0/3326759623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 0x7fe310004c40 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009b50 tx=0x7fe3100057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.224+0000 7fe323fff640 1 -- 172.21.15.125:0/3326759623 reap_dead start 2023-10-18T10:44:18.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.224+0000 7fe329f97640 1 -- 172.21.15.125:0/3326759623 shutdown_connections 2023-10-18T10:44:18.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.224+0000 7fe329f97640 1 -- 172.21.15.125:0/3326759623 >> 172.21.15.125:0/3326759623 conn(0x7fe324069930 msgr2=0x7fe324069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:18.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.224+0000 7fe329f97640 1 -- 172.21.15.125:0/3326759623 shutdown_connections 2023-10-18T10:44:18.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.224+0000 7fe329f97640 1 -- 172.21.15.125:0/3326759623 wait complete. 2023-10-18T10:44:18.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.225+0000 7fe329f97640 1 Processor -- start 2023-10-18T10:44:18.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.225+0000 7fe329f97640 1 -- start start 2023-10-18T10:44:18.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe329f97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 0x7fe3240ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:18.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe329f97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3240a8770 0x7fe3240a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:18.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe329f97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe32413eba0 0x7fe32413f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:18.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe329f97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32413f560 con 0x7fe3100047c0 2023-10-18T10:44:18.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe329f97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240a74c0 con 0x7fe3240a8770 2023-10-18T10:44:18.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe329f97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240a82e0 con 0x7fe32413eba0 2023-10-18T10:44:18.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 0x7fe3240ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:18.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 0x7fe3240ab920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55466/0 (socket says 172.21.15.125:55466) 2023-10-18T10:44:18.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 -- 172.21.15.125:0/3896662976 learned_addr learned my addr 172.21.15.125:0/3896662976 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:18.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 -- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe32413eba0 msgr2=0x7fe32413f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe32413eba0 0x7fe32413f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 -- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3240a8770 msgr2=0x7fe3240a8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe322ffd640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3240a8770 0x7fe3240a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:18.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3240a8770 0x7fe3240a8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 -- 172.21.15.125:0/3896662976 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3240a79b0 con 0x7fe3100047c0 2023-10-18T10:44:18.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe322ffd640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3240a8770 0x7fe3240a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:18.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe3237fe640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 0x7fe3240ab920 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fe314011bf0 tx=0x7fe31400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:18.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe31bfff640 1 -- 172.21.15.125:0/3896662976 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314003e40 con 0x7fe3100047c0 2023-10-18T10:44:18.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe31bfff640 1 -- 172.21.15.125:0/3896662976 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe314004b20 con 0x7fe3100047c0 2023-10-18T10:44:18.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3240a7c90 con 0x7fe3100047c0 2023-10-18T10:44:18.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.226+0000 7fe31bfff640 1 -- 172.21.15.125:0/3896662976 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314004050 con 0x7fe3100047c0 2023-10-18T10:44:18.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.227+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3240aa4a0 con 0x7fe3100047c0 2023-10-18T10:44:18.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.227+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3240690e0 con 0x7fe3100047c0 2023-10-18T10:44:18.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.228+0000 7fe31bfff640 1 -- 172.21.15.125:0/3896662976 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe3140041b0 con 0x7fe3100047c0 2023-10-18T10:44:18.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.228+0000 7fe31bfff640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fe30406af60 0x7fe30406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:18.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.229+0000 7fe31bfff640 1 -- 172.21.15.125:0/3896662976 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5205+0+0 (secure 0 0 0) 0x7fe31408a800 con 0x7fe3100047c0 2023-10-18T10:44:18.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.229+0000 7fe322ffd640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fe30406af60 0x7fe30406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:18.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.229+0000 7fe322ffd640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fe30406af60 0x7fe30406d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe30c036000 tx=0x7fe30c035040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:18.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.233+0000 7fe31bfff640 1 -- 172.21.15.125:0/3896662976 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe314055910 con 0x7fe3100047c0 2023-10-18T10:44:18.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.402+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe3241e2de0 con 0x7fe3100047c0 2023-10-18T10:44:18.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.402+0000 7fe31bfff640 1 -- 172.21.15.125:0/3896662976 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v139) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe314058f30 con 0x7fe3100047c0 2023-10-18T10:44:18.404 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:18.405 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:44:18.405 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:18.405 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:44:18.405 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:44:18.405 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:18.406 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:44:18.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fe30406af60 msgr2=0x7fe30406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fe30406af60 0x7fe30406d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe30c036000 tx=0x7fe30c035040 comp rx=0 tx=0).stop 2023-10-18T10:44:18.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 msgr2=0x7fe3240ab920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 0x7fe3240ab920 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fe314011bf0 tx=0x7fe31400f790 comp rx=0 tx=0).stop 2023-10-18T10:44:18.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 shutdown_connections 2023-10-18T10:44:18.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fe30406af60 0x7fe30406d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3240a8770 0x7fe3240a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe32413eba0 0x7fe32413f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 --2- 172.21.15.125:0/3896662976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3100047c0 0x7fe3240ab920 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 >> 172.21.15.125:0/3896662976 conn(0x7fe324069930 msgr2=0x7fe3241df7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:18.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 shutdown_connections 2023-10-18T10:44:18.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.404+0000 7fe329f97640 1 -- 172.21.15.125:0/3896662976 wait complete. 2023-10-18T10:44:18.417 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:44:18.417 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:44:18.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.522+0000 7f50df6c9640 1 -- 172.21.15.125:0/2276562004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50c40047c0 msgr2=0x7f50c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.522+0000 7f50df6c9640 1 --2- 172.21.15.125:0/2276562004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50c40047c0 0x7f50c4004c40 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f50c8006fd0 tx=0x7f50c40057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.522+0000 7f50df6c9640 1 -- 172.21.15.125:0/2276562004 shutdown_connections 2023-10-18T10:44:18.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.522+0000 7f50df6c9640 1 --2- 172.21.15.125:0/2276562004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50c40047c0 0x7f50c4004c40 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.522+0000 7f50df6c9640 1 -- 172.21.15.125:0/2276562004 >> 172.21.15.125:0/2276562004 conn(0x7f50d80d0d60 msgr2=0x7f50d80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:18.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.522+0000 7f50df6c9640 1 -- 172.21.15.125:0/2276562004 shutdown_connections 2023-10-18T10:44:18.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.522+0000 7f50df6c9640 1 -- 172.21.15.125:0/2276562004 wait complete. 2023-10-18T10:44:18.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.523+0000 7f50df6c9640 1 Processor -- start 2023-10-18T10:44:18.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.523+0000 7f50df6c9640 1 -- start start 2023-10-18T10:44:18.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50df6c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c40047c0 0x7f50d80c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:18.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50df6c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d8157560 0x7f50d8156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:18.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50df6c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d80b0ea0 0x7f50d80b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:18.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50df6c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80b1840 con 0x7f50d8157560 2023-10-18T10:44:18.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50df6c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80c8e80 con 0x7f50c40047c0 2023-10-18T10:44:18.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50df6c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81572f0 con 0x7f50d80b0ea0 2023-10-18T10:44:18.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d8157560 0x7f50d8156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:18.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d8157560 0x7f50d8156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55486/0 (socket says 172.21.15.125:55486) 2023-10-18T10:44:18.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 -- 172.21.15.125:0/3490793912 learned_addr learned my addr 172.21.15.125:0/3490793912 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:18.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dd465640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c40047c0 0x7f50d80c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:18.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50ddc66640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d80b0ea0 0x7f50d80b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:18.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 -- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d80b0ea0 msgr2=0x7f50d80b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d80b0ea0 0x7f50d80b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 -- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c40047c0 msgr2=0x7f50d80c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:18.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c40047c0 0x7f50d80c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:18.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 -- 172.21.15.125:0/3490793912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80c8520 con 0x7f50d8157560 2023-10-18T10:44:18.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50dcc64640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d8157560 0x7f50d8156710 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f50c802c900 tx=0x7f50c80015a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:18.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50cdffb640 1 -- 172.21.15.125:0/3490793912 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8006230 con 0x7f50d8157560 2023-10-18T10:44:18.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50cdffb640 1 -- 172.21.15.125:0/3490793912 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f50c8006390 con 0x7f50d8157560 2023-10-18T10:44:18.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d80c7f60 con 0x7f50d8157560 2023-10-18T10:44:18.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.524+0000 7f50cdffb640 1 -- 172.21.15.125:0/3490793912 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c80035f0 con 0x7f50d8157560 2023-10-18T10:44:18.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.525+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d80b4a00 con 0x7f50d8157560 2023-10-18T10:44:18.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.526+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d80690e0 con 0x7f50d8157560 2023-10-18T10:44:18.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.526+0000 7f50cdffb640 1 -- 172.21.15.125:0/3490793912 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f50c8031030 con 0x7f50d8157560 2023-10-18T10:44:18.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.526+0000 7f50cdffb640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f50bc06af90 0x7f50bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:18.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.526+0000 7f50cdffb640 1 -- 172.21.15.125:0/3490793912 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f50c80aa2c0 con 0x7f50d8157560 2023-10-18T10:44:18.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.527+0000 7f50dd465640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f50bc06af90 0x7f50bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:18.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.527+0000 7f50dd465640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f50bc06af90 0x7f50bc06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f50d4005fd0 tx=0x7f50d4009500 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:18.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.531+0000 7f50cdffb640 1 -- 172.21.15.125:0/3490793912 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50c8075480 con 0x7f50d8157560 2023-10-18T10:44:18.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:18.697+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f50d80af990 con 0x7f50d8157560 2023-10-18T10:44:19.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.537+0000 7f50cdffb640 1 -- 172.21.15.125:0/3490793912 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v140) v1 ==== 138+0+0 (secure 0 0 0) 0x7f50c8078aa0 con 0x7f50d8157560 2023-10-18T10:44:19.539 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:44:19.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f50bc06af90 msgr2=0x7f50bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:19.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f50bc06af90 0x7f50bc06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f50d4005fd0 tx=0x7f50d4009500 comp rx=0 tx=0).stop 2023-10-18T10:44:19.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d8157560 msgr2=0x7f50d8156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:19.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d8157560 0x7f50d8156710 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f50c802c900 tx=0x7f50c80015a0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 shutdown_connections 2023-10-18T10:44:19.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f50bc06af90 0x7f50bc06d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d80b0ea0 0x7f50d80b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d8157560 0x7f50d8156710 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 --2- 172.21.15.125:0/3490793912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c40047c0 0x7f50d80c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 >> 172.21.15.125:0/3490793912 conn(0x7f50d80d0d60 msgr2=0x7f50d80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:19.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 shutdown_connections 2023-10-18T10:44:19.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.540+0000 7f50df6c9640 1 -- 172.21.15.125:0/3490793912 wait complete. 2023-10-18T10:44:19.557 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:44:19.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.666+0000 7f21f78bb640 1 -- 172.21.15.125:0/3721963706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 msgr2=0x7f21f00affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:19.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.666+0000 7f21f78bb640 1 --2- 172.21.15.125:0/3721963706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 0x7f21f00affe0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f21f00cbd00 tx=0x7f21e802c1d0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.666+0000 7f21f78bb640 1 -- 172.21.15.125:0/3721963706 shutdown_connections 2023-10-18T10:44:19.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.666+0000 7f21f78bb640 1 --2- 172.21.15.125:0/3721963706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 0x7f21f00affe0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.666+0000 7f21f78bb640 1 --2- 172.21.15.125:0/3721963706 >> v2:172.21.15.139:3300/0 conn(0x7f21f013e520 0x7f21f006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.666+0000 7f21f78bb640 1 -- 172.21.15.125:0/3721963706 >> 172.21.15.125:0/3721963706 conn(0x7f21f0069930 msgr2=0x7f21f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:19.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.666+0000 7f21f78bb640 1 -- 172.21.15.125:0/3721963706 shutdown_connections 2023-10-18T10:44:19.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.666+0000 7f21f78bb640 1 -- 172.21.15.125:0/3721963706 wait complete. 2023-10-18T10:44:19.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.667+0000 7f21f78bb640 1 Processor -- start 2023-10-18T10:44:19.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.667+0000 7f21f78bb640 1 -- start start 2023-10-18T10:44:19.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.667+0000 7f21f78bb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 0x7f21f01566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:19.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.667+0000 7f21f78bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21f013e520 0x7f21f012fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:19.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.667+0000 7f21f78bb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21f0141690 0x7f21f00ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:19.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.667+0000 7f21f78bb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f0156c10 con 0x7f21f0070af0 2023-10-18T10:44:19.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.667+0000 7f21f78bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f0130540 con 0x7f21f013e520 2023-10-18T10:44:19.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.667+0000 7f21f78bb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f0130820 con 0x7f21f0141690 2023-10-18T10:44:19.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 0x7f21f01566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:19.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 0x7f21f01566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55508/0 (socket says 172.21.15.125:55508) 2023-10-18T10:44:19.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 -- 172.21.15.125:0/4158108363 learned_addr learned my addr 172.21.15.125:0/4158108363 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:19.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 -- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21f0141690 msgr2=0x7f21f00ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:19.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5e58640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21f0141690 0x7f21f00ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:19.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f4e56640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21f013e520 0x7f21f012fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:19.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21f0141690 0x7f21f00ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 -- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21f013e520 msgr2=0x7f21f012fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:19.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21f013e520 0x7f21f012fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:19.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 -- 172.21.15.125:0/4158108363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8009900 con 0x7f21f0070af0 2023-10-18T10:44:19.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f4e56640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21f013e520 0x7f21f012fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:19.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f5657640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 0x7f21f01566d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c920 tx=0x7f21ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:19.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21e5ffb640 1 -- 172.21.15.125:0/4158108363 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec007c70 con 0x7f21f0070af0 2023-10-18T10:44:19.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21e5ffb640 1 -- 172.21.15.125:0/4158108363 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f21ec00f450 con 0x7f21f0070af0 2023-10-18T10:44:19.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f0130b00 con 0x7f21f0070af0 2023-10-18T10:44:19.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21e5ffb640 1 -- 172.21.15.125:0/4158108363 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00f860 con 0x7f21f0070af0 2023-10-18T10:44:19.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.668+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f01310c0 con 0x7f21f0070af0 2023-10-18T10:44:19.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.669+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f006bb60 con 0x7f21f0070af0 2023-10-18T10:44:19.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.670+0000 7f21e5ffb640 1 -- 172.21.15.125:0/4158108363 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f21ec00f9c0 con 0x7f21f0070af0 2023-10-18T10:44:19.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.670+0000 7f21e5ffb640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21d406af60 0x7f21d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:19.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.670+0000 7f21e5ffb640 1 -- 172.21.15.125:0/4158108363 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5575+0+0 (secure 0 0 0) 0x7f21ec08a0b0 con 0x7f21f0070af0 2023-10-18T10:44:19.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.670+0000 7f21f4e56640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21d406af60 0x7f21d406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:19.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.671+0000 7f21f4e56640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21d406af60 0x7f21d406d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f21f00ad6f0 tx=0x7f21e0008040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:19.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.675+0000 7f21e5ffb640 1 -- 172.21.15.125:0/4158108363 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21ec055050 con 0x7f21f0070af0 2023-10-18T10:44:19.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:19.841+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f21f00690e0 con 0x7f21f0070af0 2023-10-18T10:44:20.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.539+0000 7f21e5ffb640 1 -- 172.21.15.125:0/4158108363 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v141) v1 ==== 170+0+0 (secure 0 0 0) 0x7f21ec058670 con 0x7f21f0070af0 2023-10-18T10:44:20.542 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:44:20.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21d406af60 msgr2=0x7f21d406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:20.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21d406af60 0x7f21d406d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f21f00ad6f0 tx=0x7f21e0008040 comp rx=0 tx=0).stop 2023-10-18T10:44:20.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 msgr2=0x7f21f01566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:20.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 0x7f21f01566d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c920 tx=0x7f21ec00cde0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 shutdown_connections 2023-10-18T10:44:20.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f21d406af60 0x7f21d406d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21f013e520 0x7f21f012fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21f0141690 0x7f21f00ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 --2- 172.21.15.125:0/4158108363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f0070af0 0x7f21f01566d0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.543+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 >> 172.21.15.125:0/4158108363 conn(0x7f21f0069930 msgr2=0x7f21f0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:20.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.544+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 shutdown_connections 2023-10-18T10:44:20.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.544+0000 7f21f78bb640 1 -- 172.21.15.125:0/4158108363 wait complete. 2023-10-18T10:44:20.573 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:44:20.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.679+0000 7f66e0061640 1 -- 172.21.15.125:0/3765631919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d80b18f0 msgr2=0x7f66d806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.679+0000 7f66e0061640 1 --2- 172.21.15.125:0/3765631919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d80b18f0 0x7f66d806ce70 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005fd0 tx=0x7f66d002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.679+0000 7f66e0061640 1 -- 172.21.15.125:0/3765631919 shutdown_connections 2023-10-18T10:44:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.679+0000 7f66e0061640 1 --2- 172.21.15.125:0/3765631919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d80b18f0 0x7f66d806ce70 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.679+0000 7f66e0061640 1 --2- 172.21.15.125:0/3765631919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66d8158be0 0x7f66d80c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.679+0000 7f66e0061640 1 -- 172.21.15.125:0/3765631919 >> 172.21.15.125:0/3765631919 conn(0x7f66d80aec40 msgr2=0x7f66d80af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:20.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.679+0000 7f66e0061640 1 -- 172.21.15.125:0/3765631919 shutdown_connections 2023-10-18T10:44:20.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.679+0000 7f66e0061640 1 -- 172.21.15.125:0/3765631919 wait complete. 2023-10-18T10:44:20.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.680+0000 7f66e0061640 1 Processor -- start 2023-10-18T10:44:20.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.680+0000 7f66e0061640 1 -- start start 2023-10-18T10:44:20.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66e0061640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d806ca10 0x7f66d8143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:20.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66e0061640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66d80b18f0 0x7f66d8146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:20.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66e0061640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66d8158be0 0x7f66d81529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:20.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66e0061640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8147410 con 0x7f66d806ca10 2023-10-18T10:44:20.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66e0061640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d81443e0 con 0x7f66d80b18f0 2023-10-18T10:44:20.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66e0061640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8146e50 con 0x7f66d8158be0 2023-10-18T10:44:20.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66dddfd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d806ca10 0x7f66d8143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:20.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66dddfd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d806ca10 0x7f66d8143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55524/0 (socket says 172.21.15.125:55524) 2023-10-18T10:44:20.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66dddfd640 1 -- 172.21.15.125:0/2961495242 learned_addr learned my addr 172.21.15.125:0/2961495242 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:20.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66de5fe640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66d8158be0 0x7f66d81529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:20.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.681+0000 7f66dd5fc640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66d80b18f0 0x7f66d8146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:20.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66dddfd640 1 -- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66d8158be0 msgr2=0x7f66d81529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:20.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66dddfd640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66d8158be0 0x7f66d81529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66dddfd640 1 -- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66d80b18f0 msgr2=0x7f66d8146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:20.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66dddfd640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66d80b18f0 0x7f66d8146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:20.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66dddfd640 1 -- 172.21.15.125:0/2961495242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d0009560 con 0x7f66d806ca10 2023-10-18T10:44:20.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66dddfd640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d806ca10 0x7f66d8143da0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f66c800cce0 tx=0x7f66c80075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:20.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66d67fc640 1 -- 172.21.15.125:0/2961495242 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8004b20 con 0x7f66d806ca10 2023-10-18T10:44:20.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66d67fc640 1 -- 172.21.15.125:0/2961495242 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66c8004d70 con 0x7f66d806ca10 2023-10-18T10:44:20.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d8147130 con 0x7f66d806ca10 2023-10-18T10:44:20.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66d67fc640 1 -- 172.21.15.125:0/2961495242 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8005280 con 0x7f66d806ca10 2023-10-18T10:44:20.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.682+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d80aad40 con 0x7f66d806ca10 2023-10-18T10:44:20.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.683+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d80aa8c0 con 0x7f66d806ca10 2023-10-18T10:44:20.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.684+0000 7f66d67fc640 1 -- 172.21.15.125:0/2961495242 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f66c80053e0 con 0x7f66d806ca10 2023-10-18T10:44:20.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.684+0000 7f66d67fc640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f66c406af90 0x7f66c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:20.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.684+0000 7f66d67fc640 1 -- 172.21.15.125:0/2961495242 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f66c808a110 con 0x7f66d806ca10 2023-10-18T10:44:20.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.684+0000 7f66dd5fc640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f66c406af90 0x7f66c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:20.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.685+0000 7f66dd5fc640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f66c406af90 0x7f66c406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f66d8153550 tx=0x7f66cc007e30 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:20.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.688+0000 7f66d67fc640 1 -- 172.21.15.125:0/2961495242 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66d80aa8c0 con 0x7f66d806ca10 2023-10-18T10:44:20.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:20.889+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f66d80aa8c0 con 0x7f66d806ca10 2023-10-18T10:44:21.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.556+0000 7f66d67fc640 1 -- 172.21.15.125:0/2961495242 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7f66d80aa8c0 con 0x7f66d806ca10 2023-10-18T10:44:21.560 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 17 and data pool 18 2023-10-18T10:44:21.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.565+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f66c406af90 msgr2=0x7f66c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:21.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.566+0000 7f66e0061640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f66c406af90 0x7f66c406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f66d8153550 tx=0x7f66cc007e30 comp rx=0 tx=0).stop 2023-10-18T10:44:21.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.566+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d806ca10 msgr2=0x7f66d8143da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:21.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.566+0000 7f66e0061640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d806ca10 0x7f66d8143da0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f66c800cce0 tx=0x7f66c80075f0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.568+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 shutdown_connections 2023-10-18T10:44:21.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.568+0000 7f66e0061640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f66c406af90 0x7f66c406d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.568+0000 7f66e0061640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66d80b18f0 0x7f66d8146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.568+0000 7f66e0061640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66d8158be0 0x7f66d81529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.568+0000 7f66e0061640 1 --2- 172.21.15.125:0/2961495242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66d806ca10 0x7f66d8143da0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.568+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 >> 172.21.15.125:0/2961495242 conn(0x7f66d80aec40 msgr2=0x7f66d80b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:21.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.572+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 shutdown_connections 2023-10-18T10:44:21.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.576+0000 7f66e0061640 1 -- 172.21.15.125:0/2961495242 wait complete. 2023-10-18T10:44:21.592 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:21.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.700+0000 7f23584cf640 1 -- 172.21.15.125:0/3005596895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f233c0047c0 msgr2=0x7f233c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:21.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.700+0000 7f23584cf640 1 --2- 172.21.15.125:0/3005596895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f233c0047c0 0x7f233c004c40 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f2340001650 tx=0x7f233c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.700+0000 7f2356a6c640 1 -- 172.21.15.125:0/3005596895 reap_dead start 2023-10-18T10:44:21.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.701+0000 7f23584cf640 1 -- 172.21.15.125:0/3005596895 shutdown_connections 2023-10-18T10:44:21.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.701+0000 7f23584cf640 1 -- 172.21.15.125:0/3005596895 >> 172.21.15.125:0/3005596895 conn(0x7f2350069930 msgr2=0x7f2350069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:21.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.701+0000 7f23584cf640 1 -- 172.21.15.125:0/3005596895 shutdown_connections 2023-10-18T10:44:21.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.701+0000 7f23584cf640 1 -- 172.21.15.125:0/3005596895 wait complete. 2023-10-18T10:44:21.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f23584cf640 1 Processor -- start 2023-10-18T10:44:21.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f23584cf640 1 -- start start 2023-10-18T10:44:21.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f23584cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c0047c0 0x7f235013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:21.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f23584cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23500aa440 0x7f23500aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:21.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f23584cf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23500ae840 0x7f23500aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:21.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f23584cf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500af230 con 0x7f23500aa440 2023-10-18T10:44:21.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f23584cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23501f2500 con 0x7f233c0047c0 2023-10-18T10:44:21.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f23584cf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23500aae00 con 0x7f23500ae840 2023-10-18T10:44:21.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.702+0000 7f2355a6a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23500aa440 0x7f23500aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:21.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2355a6a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23500aa440 0x7f23500aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55544/0 (socket says 172.21.15.125:55544) 2023-10-18T10:44:21.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2355a6a640 1 -- 172.21.15.125:0/4226356237 learned_addr learned my addr 172.21.15.125:0/4226356237 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:21.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2356a6c640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23500ae840 0x7f23500aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:21.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f235626b640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c0047c0 0x7f235013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:21.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2355a6a640 1 -- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23500ae840 msgr2=0x7f23500aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:21.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2355a6a640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23500ae840 0x7f23500aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2355a6a640 1 -- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c0047c0 msgr2=0x7f235013adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:21.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2355a6a640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c0047c0 0x7f235013adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2355a6a640 1 -- 172.21.15.125:0/4226356237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235013b330 con 0x7f23500aa440 2023-10-18T10:44:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2355a6a640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23500aa440 0x7f23500aa8c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f234002c090 tx=0x7f2340003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2346ffd640 1 -- 172.21.15.125:0/4226356237 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234003b070 con 0x7f23500aa440 2023-10-18T10:44:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2346ffd640 1 -- 172.21.15.125:0/4226356237 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2340006260 con 0x7f23500aa440 2023-10-18T10:44:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23500a83e0 con 0x7f23500aa440 2023-10-18T10:44:21.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f2346ffd640 1 -- 172.21.15.125:0/4226356237 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234002d760 con 0x7f23500aa440 2023-10-18T10:44:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.703+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23500a89a0 con 0x7f23500aa440 2023-10-18T10:44:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.704+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23500af230 con 0x7f23500aa440 2023-10-18T10:44:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.704+0000 7f2346ffd640 1 -- 172.21.15.125:0/4226356237 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f234002d8c0 con 0x7f23500aa440 2023-10-18T10:44:21.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.705+0000 7f2346ffd640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f233406af60 0x7f233406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:21.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.705+0000 7f2346ffd640 1 -- 172.21.15.125:0/4226356237 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f23400ab7c0 con 0x7f23500aa440 2023-10-18T10:44:21.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.705+0000 7f235626b640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f233406af60 0x7f233406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:21.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.706+0000 7f235626b640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f233406af60 0x7f233406d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23500c9400 tx=0x7f234c009500 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:21.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.709+0000 7f2346ffd640 1 -- 172.21.15.125:0/4226356237 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23500af230 con 0x7f23500aa440 2023-10-18T10:44:21.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.875+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23500af230 con 0x7f23500aa440 2023-10-18T10:44:21.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.877+0000 7f2346ffd640 1 -- 172.21.15.125:0/4226356237 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f23500af230 con 0x7f23500aa440 2023-10-18T10:44:21.884 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:21.885 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":142,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:20.891203+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:18.717169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:19.861005+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:21.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f233406af60 msgr2=0x7f233406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:21.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f233406af60 0x7f233406d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23500c9400 tx=0x7f234c009500 comp rx=0 tx=0).stop 2023-10-18T10:44:21.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23500aa440 msgr2=0x7f23500aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:21.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23500aa440 0x7f23500aa8c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f234002c090 tx=0x7f2340003de0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 shutdown_connections 2023-10-18T10:44:21.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f233406af60 0x7f233406d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23500ae840 0x7f23500aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23500aa440 0x7f23500aa8c0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 --2- 172.21.15.125:0/4226356237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f233c0047c0 0x7f235013adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 >> 172.21.15.125:0/4226356237 conn(0x7f2350069930 msgr2=0x7f2350152770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:21.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 shutdown_connections 2023-10-18T10:44:21.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.879+0000 7f23584cf640 1 -- 172.21.15.125:0/4226356237 wait complete. 2023-10-18T10:44:21.892 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:21.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.993+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/1808879009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdca4004d00 msgr2=0x7fdca4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:21.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.993+0000 7fdcbd8d4640 1 --2- 172.21.15.125:0/1808879009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdca4004d00 0x7fdca4005180 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000c670 tx=0x7fdca40057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:21.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.993+0000 7fdcb77fe640 1 -- 172.21.15.125:0/1808879009 reap_dead start 2023-10-18T10:44:21.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.993+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/1808879009 shutdown_connections 2023-10-18T10:44:21.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.993+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/1808879009 >> 172.21.15.125:0/1808879009 conn(0x7fdcb809a030 msgr2=0x7fdcb80d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:21.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.993+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/1808879009 shutdown_connections 2023-10-18T10:44:21.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.993+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/1808879009 wait complete. 2023-10-18T10:44:21.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.994+0000 7fdcbd8d4640 1 Processor -- start 2023-10-18T10:44:22.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.994+0000 7fdcbd8d4640 1 -- start start 2023-10-18T10:44:22.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcbd8d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcb81521d0 0x7fdcb80aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcbd8d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcb80c8d30 0x7fdcb80af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcbd8d4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcb815dbf0 0x7fdcb815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcbd8d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c7f70 con 0x7fdcb81521d0 2023-10-18T10:44:22.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcbd8d4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c8250 con 0x7fdcb80c8d30 2023-10-18T10:44:22.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcbd8d4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c8530 con 0x7fdcb815dbf0 2023-10-18T10:44:22.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcb81521d0 0x7fdcb80aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcb81521d0 0x7fdcb80aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55554/0 (socket says 172.21.15.125:55554) 2023-10-18T10:44:22.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb6ffd640 1 -- 172.21.15.125:0/2257382741 learned_addr learned my addr 172.21.15.125:0/2257382741 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:22.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb77fe640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcb815dbf0 0x7fdcb815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcaffff640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcb80c8d30 0x7fdcb80af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb6ffd640 1 -- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcb815dbf0 msgr2=0x7fdcb815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb6ffd640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcb815dbf0 0x7fdcb815e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb6ffd640 1 -- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcb80c8d30 msgr2=0x7fdcb80af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb6ffd640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcb80c8d30 0x7fdcb80af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.995+0000 7fdcb6ffd640 1 -- 172.21.15.125:0/2257382741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80c8810 con 0x7fdcb81521d0 2023-10-18T10:44:22.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.996+0000 7fdcb6ffd640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcb81521d0 0x7fdcb80aebf0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fdca8004650 tx=0x7fdca800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:22.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.996+0000 7fdcb4ff9640 1 -- 172.21.15.125:0/2257382741 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8010850 con 0x7fdcb81521d0 2023-10-18T10:44:22.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.996+0000 7fdcb4ff9640 1 -- 172.21.15.125:0/2257382741 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdca8015070 con 0x7fdcb81521d0 2023-10-18T10:44:22.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.996+0000 7fdcb4ff9640 1 -- 172.21.15.125:0/2257382741 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8010c50 con 0x7fdcb81521d0 2023-10-18T10:44:22.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.996+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb815e5e0 con 0x7fdcb81521d0 2023-10-18T10:44:22.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.996+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb8152790 con 0x7fdcb81521d0 2023-10-18T10:44:22.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.997+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb80675f0 con 0x7fdcb81521d0 2023-10-18T10:44:22.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.997+0000 7fdcb4ff9640 1 -- 172.21.15.125:0/2257382741 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdca8017030 con 0x7fdcb81521d0 2023-10-18T10:44:22.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.998+0000 7fdcb4ff9640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdc9806af90 0x7fdc9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.998+0000 7fdcb4ff9640 1 -- 172.21.15.125:0/2257382741 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fdca808a310 con 0x7fdcb81521d0 2023-10-18T10:44:22.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.998+0000 7fdcaffff640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdc9806af90 0x7fdc9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:21.998+0000 7fdcaffff640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdc9806af90 0x7fdc9806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdca00058c0 tx=0x7fdca0005850 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:22.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.002+0000 7fdcb4ff9640 1 -- 172.21.15.125:0/2257382741 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdca8055160 con 0x7fdcb81521d0 2023-10-18T10:44:22.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.167+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcb80c80b0 con 0x7fdcb81521d0 2023-10-18T10:44:22.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.168+0000 7fdcb4ff9640 1 -- 172.21.15.125:0/2257382741 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16889 (secure 0 0 0) 0x7fdcb80c80b0 con 0x7fdcb81521d0 2023-10-18T10:44:22.170 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:22.171 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":142,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:20.891203+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:18.717169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:19.861005+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:22.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdc9806af90 msgr2=0x7fdc9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdc9806af90 0x7fdc9806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdca00058c0 tx=0x7fdca0005850 comp rx=0 tx=0).stop 2023-10-18T10:44:22.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcb81521d0 msgr2=0x7fdcb80aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcb81521d0 0x7fdcb80aebf0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fdca8004650 tx=0x7fdca800ba40 comp rx=0 tx=0).stop 2023-10-18T10:44:22.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 shutdown_connections 2023-10-18T10:44:22.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fdc9806af90 0x7fdc9806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcb80c8d30 0x7fdcb80af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcb815dbf0 0x7fdcb815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 --2- 172.21.15.125:0/2257382741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcb81521d0 0x7fdcb80aebf0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 >> 172.21.15.125:0/2257382741 conn(0x7fdcb809a030 msgr2=0x7fdcb80cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:22.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 shutdown_connections 2023-10-18T10:44:22.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.170+0000 7fdcbd8d4640 1 -- 172.21.15.125:0/2257382741 wait complete. 2023-10-18T10:44:22.183 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:44:22.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.287+0000 7f09723d0640 1 -- 172.21.15.125:0/2415506140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 msgr2=0x7f096c0b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.287+0000 7f09723d0640 1 --2- 172.21.15.125:0/2415506140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 0x7f096c0b09a0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f096c0cc490 tx=0x7f096402c110 comp rx=0 tx=0).stop 2023-10-18T10:44:22.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.287+0000 7f09723d0640 1 -- 172.21.15.125:0/2415506140 shutdown_connections 2023-10-18T10:44:22.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.287+0000 7f09723d0640 1 --2- 172.21.15.125:0/2415506140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 0x7f096c0b09a0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.287+0000 7f09723d0640 1 --2- 172.21.15.125:0/2415506140 >> v2:172.21.15.139:3300/0 conn(0x7f096c0afa20 0x7f096c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.287+0000 7f09723d0640 1 -- 172.21.15.125:0/2415506140 >> 172.21.15.125:0/2415506140 conn(0x7f096c069930 msgr2=0x7f096c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:22.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.287+0000 7f09723d0640 1 -- 172.21.15.125:0/2415506140 shutdown_connections 2023-10-18T10:44:22.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.287+0000 7f09723d0640 1 -- 172.21.15.125:0/2415506140 wait complete. 2023-10-18T10:44:22.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.288+0000 7f09723d0640 1 Processor -- start 2023-10-18T10:44:22.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.288+0000 7f09723d0640 1 -- start start 2023-10-18T10:44:22.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f09723d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c0afa20 0x7f096c13d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f09723d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 0x7f096c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f09723d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c0cbd20 0x7f096c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f09723d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1efb20 con 0x7f096c0b0530 2023-10-18T10:44:22.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f09723d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1eef10 con 0x7f096c0cbd20 2023-10-18T10:44:22.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f09723d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1ef1f0 con 0x7f096c0afa20 2023-10-18T10:44:22.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 0x7f096c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 0x7f096c152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55584/0 (socket says 172.21.15.125:55584) 2023-10-18T10:44:22.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 -- 172.21.15.125:0/1206092868 learned_addr learned my addr 172.21.15.125:0/1206092868 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:22.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f097096d640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c0cbd20 0x7f096c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096bfff640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c0afa20 0x7f096c13d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 -- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c0afa20 msgr2=0x7f096c13d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c0afa20 0x7f096c13d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 -- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c0cbd20 msgr2=0x7f096c0ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c0cbd20 0x7f096c0ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 -- 172.21.15.125:0/1206092868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0964009840 con 0x7f096c0b0530 2023-10-18T10:44:22.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f097096d640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c0cbd20 0x7f096c0ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:22.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f096b7fe640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 0x7f096c152c60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f0960009970 tx=0x7f0960009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:22.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f0968ff9640 1 -- 172.21.15.125:0/1206092868 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960012070 con 0x7f096c0b0530 2023-10-18T10:44:22.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f0968ff9640 1 -- 172.21.15.125:0/1206092868 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f096000d020 con 0x7f096c0b0530 2023-10-18T10:44:22.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.289+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c1ef4d0 con 0x7f096c0b0530 2023-10-18T10:44:22.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.290+0000 7f0968ff9640 1 -- 172.21.15.125:0/1206092868 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960007860 con 0x7f096c0b0530 2023-10-18T10:44:22.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.290+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c1eb7f0 con 0x7f096c0b0530 2023-10-18T10:44:22.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.290+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c1efc90 con 0x7f096c0b0530 2023-10-18T10:44:22.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.291+0000 7f0968ff9640 1 -- 172.21.15.125:0/1206092868 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0960016070 con 0x7f096c0b0530 2023-10-18T10:44:22.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.291+0000 7f0968ff9640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.291+0000 7f0968ff9640 1 -- 172.21.15.125:0/1206092868 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f096008b190 con 0x7f096c0b0530 2023-10-18T10:44:22.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.292+0000 7f096bfff640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.292+0000 7f096bfff640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f095806af60 0x7f095806d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f095c007fd0 tx=0x7f095c009040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:22.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.296+0000 7f0968ff9640 1 -- 172.21.15.125:0/1206092868 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0960059960 con 0x7f096c0b0530 2023-10-18T10:44:22.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.494+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f096c0690e0 con 0x7f096c0b0530 2023-10-18T10:44:22.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.565+0000 7f0968ff9640 1 -- 172.21.15.125:0/1206092868 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f09600594f0 con 0x7f096c0b0530 2023-10-18T10:44:22.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f095806af60 msgr2=0x7f095806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f095806af60 0x7f095806d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f095c007fd0 tx=0x7f095c009040 comp rx=0 tx=0).stop 2023-10-18T10:44:22.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 msgr2=0x7f096c152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 0x7f096c152c60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f0960009970 tx=0x7f0960009e30 comp rx=0 tx=0).stop 2023-10-18T10:44:22.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 shutdown_connections 2023-10-18T10:44:22.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c0cbd20 0x7f096c0ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f096c0b0530 0x7f096c152c60 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 --2- 172.21.15.125:0/1206092868 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c0afa20 0x7f096c13d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 >> 172.21.15.125:0/1206092868 conn(0x7f096c069930 msgr2=0x7f096c0c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:22.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 shutdown_connections 2023-10-18T10:44:22.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.568+0000 7f09723d0640 1 -- 172.21.15.125:0/1206092868 wait complete. 2023-10-18T10:44:22.581 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:44:22.582 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:44:22.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.685+0000 7f33de1a5640 1 -- 172.21.15.125:0/1695140879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33c40047c0 msgr2=0x7f33c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.685+0000 7f33de1a5640 1 --2- 172.21.15.125:0/1695140879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33c40047c0 0x7f33c4004c40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009b50 tx=0x7f33c40057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.685+0000 7f33de1a5640 1 -- 172.21.15.125:0/1695140879 shutdown_connections 2023-10-18T10:44:22.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.685+0000 7f33de1a5640 1 --2- 172.21.15.125:0/1695140879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33c40047c0 0x7f33c4004c40 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.685+0000 7f33de1a5640 1 -- 172.21.15.125:0/1695140879 >> 172.21.15.125:0/1695140879 conn(0x7f33d80d0d60 msgr2=0x7f33d80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:22.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.686+0000 7f33de1a5640 1 -- 172.21.15.125:0/1695140879 shutdown_connections 2023-10-18T10:44:22.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.686+0000 7f33de1a5640 1 -- 172.21.15.125:0/1695140879 wait complete. 2023-10-18T10:44:22.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.687+0000 7f33de1a5640 1 Processor -- start 2023-10-18T10:44:22.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.687+0000 7f33de1a5640 1 -- start start 2023-10-18T10:44:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.687+0000 7f33de1a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33c40047c0 0x7f33d80b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.687+0000 7f33de1a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33d81f4a50 0x7f33d814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.687+0000 7f33de1a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d80ae460 0x7f33d80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.687+0000 7f33de1a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c9350 con 0x7f33d80ae460 2023-10-18T10:44:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.687+0000 7f33de1a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80aee20 con 0x7f33d81f4a50 2023-10-18T10:44:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.687+0000 7f33de1a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c95f0 con 0x7f33c40047c0 2023-10-18T10:44:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d80ae460 0x7f33d80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d80ae460 0x7f33d80ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55598/0 (socket says 172.21.15.125:55598) 2023-10-18T10:44:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 -- 172.21.15.125:0/3192693173 learned_addr learned my addr 172.21.15.125:0/3192693173 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33ceffd640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33d81f4a50 0x7f33d814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d77fe640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33c40047c0 0x7f33d80b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 -- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33c40047c0 msgr2=0x7f33d80b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33c40047c0 0x7f33d80b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 -- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33d81f4a50 msgr2=0x7f33d814c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33d81f4a50 0x7f33d814c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 -- 172.21.15.125:0/3192693173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d814c6f0 con 0x7f33d80ae460 2023-10-18T10:44:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33ceffd640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33d81f4a50 0x7f33d814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d7fff640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d80ae460 0x7f33d80ae8e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f33d80b6e10 tx=0x7f33d00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d4ff9640 1 -- 172.21.15.125:0/3192693173 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00107d0 con 0x7f33d80ae460 2023-10-18T10:44:22.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d4ff9640 1 -- 172.21.15.125:0/3192693173 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33d0010a20 con 0x7f33d80ae460 2023-10-18T10:44:22.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33d4ff9640 1 -- 172.21.15.125:0/3192693173 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0010e30 con 0x7f33d80ae460 2023-10-18T10:44:22.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d81f51f0 con 0x7f33d80ae460 2023-10-18T10:44:22.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.688+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d81f57b0 con 0x7f33d80ae460 2023-10-18T10:44:22.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.689+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d80690e0 con 0x7f33d80ae460 2023-10-18T10:44:22.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.690+0000 7f33d4ff9640 1 -- 172.21.15.125:0/3192693173 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f33d001b3b0 con 0x7f33d80ae460 2023-10-18T10:44:22.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.690+0000 7f33d4ff9640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f33b806af90 0x7f33b806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:22.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.690+0000 7f33d4ff9640 1 -- 172.21.15.125:0/3192693173 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f33d00898f0 con 0x7f33d80ae460 2023-10-18T10:44:22.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.690+0000 7f33d77fe640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f33b806af90 0x7f33b806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:22.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.691+0000 7f33d77fe640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f33b806af90 0x7f33b806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33c8001ce0 tx=0x7f33c8006040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:22.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.694+0000 7f33d4ff9640 1 -- 172.21.15.125:0/3192693173 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33d00546d0 con 0x7f33d80ae460 2023-10-18T10:44:22.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:22.893+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f33d80d0b40 con 0x7f33d80ae460 2023-10-18T10:44:23.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.572+0000 7f33d4ff9640 1 -- 172.21.15.125:0/3192693173 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f33d0017070 con 0x7f33d80ae460 2023-10-18T10:44:23.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f33b806af90 msgr2=0x7f33b806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:23.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f33b806af90 0x7f33b806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33c8001ce0 tx=0x7f33c8006040 comp rx=0 tx=0).stop 2023-10-18T10:44:23.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d80ae460 msgr2=0x7f33d80ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:23.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d80ae460 0x7f33d80ae8e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f33d80b6e10 tx=0x7f33d00019b0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 shutdown_connections 2023-10-18T10:44:23.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f33b806af90 0x7f33b806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33d81f4a50 0x7f33d814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d80ae460 0x7f33d80ae8e0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 --2- 172.21.15.125:0/3192693173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33c40047c0 0x7f33d80b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 >> 172.21.15.125:0/3192693173 conn(0x7f33d80d0d60 msgr2=0x7f33d80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:23.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 shutdown_connections 2023-10-18T10:44:23.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.576+0000 7f33de1a5640 1 -- 172.21.15.125:0/3192693173 wait complete. 2023-10-18T10:44:23.591 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:44:23.591 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:23.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.698+0000 7ff506b0f640 1 -- 172.21.15.125:0/1949366210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5000c9dc0 msgr2=0x7ff50006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:23.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.698+0000 7ff506b0f640 1 --2- 172.21.15.125:0/1949366210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5000c9dc0 0x7ff50006b2e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009830 tx=0x7ff4ec02bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.698+0000 7ff506b0f640 1 -- 172.21.15.125:0/1949366210 shutdown_connections 2023-10-18T10:44:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.698+0000 7ff506b0f640 1 --2- 172.21.15.125:0/1949366210 >> v2:172.21.15.139:3300/0 conn(0x7ff5000ab500 0x7ff5000ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.698+0000 7ff506b0f640 1 --2- 172.21.15.125:0/1949366210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5000c9dc0 0x7ff50006b2e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.698+0000 7ff506b0f640 1 -- 172.21.15.125:0/1949366210 >> 172.21.15.125:0/1949366210 conn(0x7ff500069930 msgr2=0x7ff500069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.698+0000 7ff506b0f640 1 -- 172.21.15.125:0/1949366210 shutdown_connections 2023-10-18T10:44:23.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.698+0000 7ff506b0f640 1 -- 172.21.15.125:0/1949366210 wait complete. 2023-10-18T10:44:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.699+0000 7ff506b0f640 1 Processor -- start 2023-10-18T10:44:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.699+0000 7ff506b0f640 1 -- start start 2023-10-18T10:44:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff506b0f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff50006af00 0x7ff5000af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff506b0f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5000ab500 0x7ff5000a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff506b0f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5000c9dc0 0x7ff50013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff506b0f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000a7130 con 0x7ff50006af00 2023-10-18T10:44:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff506b0f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000af7d0 con 0x7ff5000ab500 2023-10-18T10:44:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff506b0f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000afab0 con 0x7ff5000c9dc0 2023-10-18T10:44:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5048ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff50006af00 0x7ff5000af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5048ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff50006af00 0x7ff5000af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55610/0 (socket says 172.21.15.125:55610) 2023-10-18T10:44:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5048ab640 1 -- 172.21.15.125:0/890823381 learned_addr learned my addr 172.21.15.125:0/890823381 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5050ac640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5000c9dc0 0x7ff50013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff4fffff640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5000ab500 0x7ff5000a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5048ab640 1 -- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5000c9dc0 msgr2=0x7ff50013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5048ab640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5000c9dc0 0x7ff50013a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5048ab640 1 -- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5000ab500 msgr2=0x7ff5000a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5048ab640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5000ab500 0x7ff5000a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.700+0000 7ff5048ab640 1 -- 172.21.15.125:0/890823381 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f8009900 con 0x7ff50006af00 2023-10-18T10:44:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.701+0000 7ff5048ab640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff50006af00 0x7ff5000af110 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400c400 tx=0x7ff4f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.701+0000 7ff4fd7fa640 1 -- 172.21.15.125:0/890823381 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f40052d0 con 0x7ff50006af00 2023-10-18T10:44:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.701+0000 7ff4fd7fa640 1 -- 172.21.15.125:0/890823381 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4f4007020 con 0x7ff50006af00 2023-10-18T10:44:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.701+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec009490 con 0x7ff50006af00 2023-10-18T10:44:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.701+0000 7ff4fd7fa640 1 -- 172.21.15.125:0/890823381 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4007470 con 0x7ff50006af00 2023-10-18T10:44:23.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.701+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5000b0070 con 0x7ff50006af00 2023-10-18T10:44:23.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.702+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5000c8dc0 con 0x7ff50006af00 2023-10-18T10:44:23.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.702+0000 7ff4fd7fa640 1 -- 172.21.15.125:0/890823381 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff4f40067d0 con 0x7ff50006af00 2023-10-18T10:44:23.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.702+0000 7ff4fd7fa640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff4e406af90 0x7ff4e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:23.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.703+0000 7ff4fd7fa640 1 -- 172.21.15.125:0/890823381 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7ff4f4088e90 con 0x7ff50006af00 2023-10-18T10:44:23.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.703+0000 7ff4fffff640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff4e406af90 0x7ff4e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:23.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.703+0000 7ff4fffff640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff4e406af90 0x7ff4e406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec036000 tx=0x7ff4ec004730 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:23.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.707+0000 7ff4fd7fa640 1 -- 172.21.15.125:0/890823381 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4f4053d80 con 0x7ff50006af00 2023-10-18T10:44:23.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.908+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff50013baf0 con 0x7ff50006af00 2023-10-18T10:44:23.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.909+0000 7ff4fd7fa640 1 -- 172.21.15.125:0/890823381 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4924 (secure 0 0 0) 0x7ff4f40573a0 con 0x7ff50006af00 2023-10-18T10:44:23.911 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:23.911 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":53,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7112,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/585529921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":585529921},{"type":"v1","addr":"172.21.15.125:6835","nonce":585529921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1680315524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1680315524},{"type":"v1","addr":"172.21.15.139:6837","nonce":1680315524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1127901065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1127901065},{"type":"v1","addr":"172.21.15.139:6836","nonce":1127901065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:20.891510+0000","modified":"2023-10-18T10:44:22.895207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9159":{"gid":9159,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2651847004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2651847004},{"type":"v1","addr":"172.21.15.125:6834","nonce":2651847004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T10:44:23.912 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 53 2023-10-18T10:44:23.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff4e406af90 msgr2=0x7ff4e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:23.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff4e406af90 0x7ff4e406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec036000 tx=0x7ff4ec004730 comp rx=0 tx=0).stop 2023-10-18T10:44:23.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff50006af00 msgr2=0x7ff5000af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:23.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff50006af00 0x7ff5000af110 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400c400 tx=0x7ff4f400c8c0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 shutdown_connections 2023-10-18T10:44:23.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff4e406af90 0x7ff4e406d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5000ab500 0x7ff5000a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5000c9dc0 0x7ff50013a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 --2- 172.21.15.125:0/890823381 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff50006af00 0x7ff5000af110 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:23.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 >> 172.21.15.125:0/890823381 conn(0x7ff500069930 msgr2=0x7ff5000cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:23.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 shutdown_connections 2023-10-18T10:44:23.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:23.911+0000 7ff506b0f640 1 -- 172.21.15.125:0/890823381 wait complete. 2023-10-18T10:44:23.924 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:24.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.023+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1133805099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 msgr2=0x7fb49c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.023+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1133805099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 0x7fb49c06bb60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009830 tx=0x7fb48c02bc30 comp rx=0 tx=0).stop 2023-10-18T10:44:24.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.023+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1133805099 shutdown_connections 2023-10-18T10:44:24.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.023+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1133805099 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb49c141690 0x7fb49c0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.023+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1133805099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 0x7fb49c06bb60 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.023+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1133805099 >> 172.21.15.125:0/1133805099 conn(0x7fb49c069930 msgr2=0x7fb49c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:24.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.023+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1133805099 shutdown_connections 2023-10-18T10:44:24.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.023+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1133805099 wait complete. 2023-10-18T10:44:24.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.024+0000 7fb4a21b6640 1 Processor -- start 2023-10-18T10:44:24.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb4a21b6640 1 -- start start 2023-10-18T10:44:24.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb4a21b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb49c06c0a0 0x7fb49c13adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb4a21b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 0x7fb49c1e3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb4a21b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb49c141690 0x7fb49c1e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb4a21b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c13de10 con 0x7fb49c13e520 2023-10-18T10:44:24.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb4a21b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c13b330 con 0x7fb49c141690 2023-10-18T10:44:24.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb4a21b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c1ef2b0 con 0x7fb49c06c0a0 2023-10-18T10:44:24.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb49affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 0x7fb49c1e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb49affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 0x7fb49c1e3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55626/0 (socket says 172.21.15.125:55626) 2023-10-18T10:44:24.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.025+0000 7fb49affd640 1 -- 172.21.15.125:0/1722682861 learned_addr learned my addr 172.21.15.125:0/1722682861 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:24.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb49b7fe640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb49c06c0a0 0x7fb49c13adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb49bfff640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb49c141690 0x7fb49c1e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb49affd640 1 -- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb49c06c0a0 msgr2=0x7fb49c13adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb49affd640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb49c06c0a0 0x7fb49c13adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb49affd640 1 -- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb49c141690 msgr2=0x7fb49c1e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb49affd640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb49c141690 0x7fb49c1e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb49affd640 1 -- 172.21.15.125:0/1722682861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c009490 con 0x7fb49c13e520 2023-10-18T10:44:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb49affd640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 0x7fb49c1e3220 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fb49000d740 tx=0x7fb49000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb483fff640 1 -- 172.21.15.125:0/1722682861 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490009c30 con 0x7fb49c13e520 2023-10-18T10:44:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb483fff640 1 -- 172.21.15.125:0/1722682861 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb490009e80 con 0x7fb49c13e520 2023-10-18T10:44:24.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c1ef590 con 0x7fb49c13e520 2023-10-18T10:44:24.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb483fff640 1 -- 172.21.15.125:0/1722682861 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900106c0 con 0x7fb49c13e520 2023-10-18T10:44:24.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.026+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c1efb50 con 0x7fb49c13e520 2023-10-18T10:44:24.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.027+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c1efd10 con 0x7fb49c13e520 2023-10-18T10:44:24.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.028+0000 7fb483fff640 1 -- 172.21.15.125:0/1722682861 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb490010820 con 0x7fb49c13e520 2023-10-18T10:44:24.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.028+0000 7fb483fff640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb48806af60 0x7fb48806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.028+0000 7fb483fff640 1 -- 172.21.15.125:0/1722682861 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fb490089ee0 con 0x7fb49c13e520 2023-10-18T10:44:24.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.028+0000 7fb49b7fe640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb48806af60 0x7fb48806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.029+0000 7fb49b7fe640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb48806af60 0x7fb48806d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb48c036000 tx=0x7fb48c004730 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:24.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.032+0000 7fb483fff640 1 -- 172.21.15.125:0/1722682861 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb490054c10 con 0x7fb49c13e520 2023-10-18T10:44:24.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.198+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb49c0690e0 con 0x7fb49c13e520 2023-10-18T10:44:24.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.199+0000 7fb483fff640 1 -- 172.21.15.125:0/1722682861 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16876 (secure 0 0 0) 0x7fb490021090 con 0x7fb49c13e520 2023-10-18T10:44:24.201 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:24.202 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":143,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:22.556110+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:18.717169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:19.861005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:24.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb48806af60 msgr2=0x7fb48806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb48806af60 0x7fb48806d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb48c036000 tx=0x7fb48c004730 comp rx=0 tx=0).stop 2023-10-18T10:44:24.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 msgr2=0x7fb49c1e3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 0x7fb49c1e3220 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fb49000d740 tx=0x7fb49000dc00 comp rx=0 tx=0).stop 2023-10-18T10:44:24.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 shutdown_connections 2023-10-18T10:44:24.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb48806af60 0x7fb48806d420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb49c141690 0x7fb49c1e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb49c13e520 0x7fb49c1e3220 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 --2- 172.21.15.125:0/1722682861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb49c06c0a0 0x7fb49c13adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 >> 172.21.15.125:0/1722682861 conn(0x7fb49c069930 msgr2=0x7fb49c1d9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:24.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 shutdown_connections 2023-10-18T10:44:24.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.201+0000 7fb4a21b6640 1 -- 172.21.15.125:0/1722682861 wait complete. 2023-10-18T10:44:24.214 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:44:24.215 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:44:24.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.315+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/4141813924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4c4003ee0 msgr2=0x7fc4c40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.315+0000 7fc4e2cc9640 1 --2- 172.21.15.125:0/4141813924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4c4003ee0 0x7fc4c40047c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0001790 tx=0x7fc4c40057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.315+0000 7fc4e1266640 1 -- 172.21.15.125:0/4141813924 reap_dead start 2023-10-18T10:44:24.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.315+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/4141813924 shutdown_connections 2023-10-18T10:44:24.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.315+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/4141813924 >> 172.21.15.125:0/4141813924 conn(0x7fc4dc069930 msgr2=0x7fc4dc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:24.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.316+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/4141813924 shutdown_connections 2023-10-18T10:44:24.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.316+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/4141813924 wait complete. 2023-10-18T10:44:24.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.317+0000 7fc4e2cc9640 1 Processor -- start 2023-10-18T10:44:24.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.317+0000 7fc4e2cc9640 1 -- start start 2023-10-18T10:44:24.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.317+0000 7fc4e2cc9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c4003ee0 0x7fc4dc149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.317+0000 7fc4e2cc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4dc1f2f00 0x7fc4dc1461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.317+0000 7fc4e2cc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4dc0acc10 0x7fc4dc0ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.317+0000 7fc4e2cc9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc0ad5b0 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.317+0000 7fc4e2cc9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc14a450 con 0x7fc4dc0acc10 2023-10-18T10:44:24.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.317+0000 7fc4e2cc9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc1f3400 con 0x7fc4c4003ee0 2023-10-18T10:44:24.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4dc1f2f00 0x7fc4dc1461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4dc1f2f00 0x7fc4dc1461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55646/0 (socket says 172.21.15.125:55646) 2023-10-18T10:44:24.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 -- 172.21.15.125:0/534892860 learned_addr learned my addr 172.21.15.125:0/534892860 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:24.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4e1266640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4dc0acc10 0x7fc4dc0ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4e0a65640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c4003ee0 0x7fc4dc149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 -- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c4003ee0 msgr2=0x7fc4dc149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c4003ee0 0x7fc4dc149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 -- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4dc0acc10 msgr2=0x7fc4dc0ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4dc0acc10 0x7fc4dc0ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 -- 172.21.15.125:0/534892860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc1f36e0 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4e1266640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4dc0acc10 0x7fc4dc0ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:24.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4dbfff640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4dc1f2f00 0x7fc4dc1461c0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0001790 tx=0x7fc4d0006290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:24.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4d97fa640 1 -- 172.21.15.125:0/534892860 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0003ac0 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4d97fa640 1 -- 172.21.15.125:0/534892860 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4d0003d10 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4d97fa640 1 -- 172.21.15.125:0/534892860 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00366b0 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4dc1f39c0 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.318+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc14a0d0 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.319+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc14a5c0 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.320+0000 7fc4d97fa640 1 -- 172.21.15.125:0/534892860 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc4d0033070 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.320+0000 7fc4d97fa640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc4c806af90 0x7fc4c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.320+0000 7fc4d97fa640 1 -- 172.21.15.125:0/534892860 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fc4d003d070 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.321+0000 7fc4e0a65640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc4c806af90 0x7fc4c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.321+0000 7fc4e0a65640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc4c806af90 0x7fc4c806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc001d00 tx=0x7fc4cc001bf0 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:24.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.325+0000 7fc4d97fa640 1 -- 172.21.15.125:0/534892860 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4d00761a0 con 0x7fc4dc1f2f00 2023-10-18T10:44:24.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.488+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 --> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4dc0690e0 con 0x7fc4c806af90 2023-10-18T10:44:24.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.496+0000 7fc4d97fa640 1 -- 172.21.15.125:0/534892860 <== mgr.7786 v2:172.21.15.139:6832/4144373589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274560 (secure 0 0 0) 0x7fc4dc0690e0 con 0x7fc4c806af90 2023-10-18T10:44:24.501 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:24.502 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:44:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc4c806af90 msgr2=0x7fc4c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc4c806af90 0x7fc4c806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc001d00 tx=0x7fc4cc001bf0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4dc1f2f00 msgr2=0x7fc4dc1461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4dc1f2f00 0x7fc4dc1461c0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0001790 tx=0x7fc4d0006290 comp rx=0 tx=0).stop 2023-10-18T10:44:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 shutdown_connections 2023-10-18T10:44:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fc4c806af90 0x7fc4c806d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4dc0acc10 0x7fc4dc0ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4dc1f2f00 0x7fc4dc1461c0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 --2- 172.21.15.125:0/534892860 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c4003ee0 0x7fc4dc149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 >> 172.21.15.125:0/534892860 conn(0x7fc4dc069930 msgr2=0x7fc4dc06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:24.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 shutdown_connections 2023-10-18T10:44:24.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.499+0000 7fc4e2cc9640 1 -- 172.21.15.125:0/534892860 wait complete. 2023-10-18T10:44:24.512 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:44:23.281677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":334924,"kb_used_data":14652,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754639796,"statfs":{"total":773094113280,"available":772751151104,"internally_reserved":0,"allocated":15003648,"data_stored":8147767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":327798328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001043"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"131'236","reported_seq":697,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-18T10:44:16.178354+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:44:16.178354+0000","last_peered":"2023-10-18T10:44:16.178354+0000","last_clean":"2023-10-18T10:44:16.178354+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:44:16.178354+0000","last_undegraded":"2023-10-18T10:44:16.178354+0000","last_fullsized":"2023-10-18T10:44:16.178354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575081,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59820,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312020,"statfs":{"total":96636764160,"available":96575508480,"internally_reserved":0,"allocated":2322432,"data_stored":1462010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.31,"5min":0.217,"15min":0.217},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.88300000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.746,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.44},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.73099999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.496}]},{"osd":3,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.71399999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77800000000000002,"5min":0.79800000000000004,"15min":0.85799999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45600000000000002,"15min":0.439},"min":{"1min":0.28799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48099999999999998,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.627,"5min":0.70699999999999996,"15min":0.86199999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.74299999999999999,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.76500000000000001,"5min":0.875,"15min":0.875},"last":0.58199999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.73199999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.27000000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.91700000000000004},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72099999999999997,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.66600000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.74199999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.748,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.70099999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.75700000000000001,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.68200000000000005}]},{"osd":4,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.66200000000000003,"5min":0.872,"15min":0.872},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.747},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.87},"last":0.73199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50924,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320916,"statfs":{"total":96636764160,"available":96584617984,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.38800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.58799999999999997}]},{"osd":2,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.93600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.66500000000000004,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.84199999999999997},"last":0.498},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.75600000000000001,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.72299999999999998,"15min":0.80100000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.38900000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.83299999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.31},"max":{"1min":0.71099999999999997,"5min":0.84299999999999997,"15min":0.89300000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.85299999999999998,"15min":0.871},"last":0.44800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.76200000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.877},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83199999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.89300000000000002},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81999999999999995,"15min":0.85899999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.46500000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.873,"15min":0.873},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74199999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.505,"15min":0.498},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.86399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":1.446},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.84799999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":1.0469999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":0.874,"5min":0.88600000000000001,"15min":1.0880000000000001},"last":0.75900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.36699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.183},"max":{"1min":0.52900000000000003,"5min":0.58999999999999997,"15min":0.62},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.22800000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.27300000000000002,"5min":0.249,"15min":0.20799999999999999},"max":{"1min":0.55200000000000005,"5min":0.624,"15min":0.70699999999999996},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38600000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.61599999999999999,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.432,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23300000000000001},"max":{"1min":0.61599999999999999,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.28100000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47999999999999998,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.46800000000000003,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.621,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86599999999999999,"5min":1.002,"15min":1.002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.63},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.89500000000000002,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.37},"max":{"1min":0.78100000000000003,"5min":1.018,"15min":1.018},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.44900000000000001,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":0.89800000000000002,"15min":0.96799999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.57599999999999996,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.95799999999999996,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.67200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55080,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316760,"statfs":{"total":96636764160,"available":96580362240,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.375,"15min":0.375},"min":{"1min":0.223,"5min":0.223,"15min":0.17599999999999999},"max":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.54100000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.40300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.503,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42099999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.61699999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.95699999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.87,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.46600000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.91400000000000003,"15min":0.93600000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.46000000000000002,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.441,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.45100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.59199999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.376}]},{"osd":2,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64800000000000002},"last":0.33800000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56999999999999995,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.35399999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38,"5min":0.377,"15min":0.374},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.44}]},{"osd":6,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.32400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63200000000000001},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.77800000000000002,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.46899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575080,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37600,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334240,"statfs":{"total":96636764160,"available":96598261760,"internally_reserved":0,"allocated":2310144,"data_stored":1455905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36799999999999999,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.46600000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.62,"15min":0.76900000000000002},"last":0.374}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39700000000000002,"15min":0.40699999999999997},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.56399999999999995,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39500000000000002,"15min":0.41499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.69699999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.59199999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.438,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.89200000000000002,"5min":0.996,"15min":0.996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.92400000000000004,"5min":1.038,"15min":1.038},"last":0.66100000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.85799999999999998,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94099999999999995,"5min":1.026,"15min":1.3700000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94099999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.621}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.64300000000000002},"min":{"1min":0.46999999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97499999999999998,"5min":1.05,"15min":1.339},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":1.01,"15min":1.01},"last":0.57399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:44:24.518 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:44:24.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.652+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2663365665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 msgr2=0x7fd8580c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.652+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2663365665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 0x7fd8580c97c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd848005fd0 tx=0x7fd84802be10 comp rx=0 tx=0).stop 2023-10-18T10:44:24.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.652+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2663365665 shutdown_connections 2023-10-18T10:44:24.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.652+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2663365665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 0x7fd8580c97c0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.652+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2663365665 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd85813d360 0x7fd85815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.652+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2663365665 >> 172.21.15.125:0/2663365665 conn(0x7fd858069930 msgr2=0x7fd858069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:24.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.652+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2663365665 shutdown_connections 2023-10-18T10:44:24.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.652+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2663365665 wait complete. 2023-10-18T10:44:24.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.653+0000 7fd85f5f5640 1 Processor -- start 2023-10-18T10:44:24.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.653+0000 7fd85f5f5640 1 -- start start 2023-10-18T10:44:24.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85f5f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8580c9d00 0x7fd8581f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85f5f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd85813d360 0x7fd8581348a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85f5f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 0x7fd8580b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85f5f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd858137a50 con 0x7fd858140660 2023-10-18T10:44:24.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85f5f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581f4750 con 0x7fd85813d360 2023-10-18T10:44:24.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85f5f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd858137120 con 0x7fd8580c9d00 2023-10-18T10:44:24.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 0x7fd8580b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 0x7fd8580b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55678/0 (socket says 172.21.15.125:55678) 2023-10-18T10:44:24.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 -- 172.21.15.125:0/2391986235 learned_addr learned my addr 172.21.15.125:0/2391986235 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:24.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85cb90640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd85813d360 0x7fd8581348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85d391640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8580c9d00 0x7fd8581f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 -- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8580c9d00 msgr2=0x7fd8581f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8580c9d00 0x7fd8581f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 -- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd85813d360 msgr2=0x7fd8581348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd85813d360 0x7fd8581348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 -- 172.21.15.125:0/2391986235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848009560 con 0x7fd858140660 2023-10-18T10:44:24.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85d391640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8580c9d00 0x7fd8581f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:24.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd85db92640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 0x7fd8580b09a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fd85000c460 tx=0x7fd85000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:24.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.654+0000 7fd84dffb640 1 -- 172.21.15.125:0/2391986235 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500035f0 con 0x7fd858140660 2023-10-18T10:44:24.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.655+0000 7fd84dffb640 1 -- 172.21.15.125:0/2391986235 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd850003770 con 0x7fd858140660 2023-10-18T10:44:24.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.655+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd858137390 con 0x7fd858140660 2023-10-18T10:44:24.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.655+0000 7fd84dffb640 1 -- 172.21.15.125:0/2391986235 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850003b80 con 0x7fd858140660 2023-10-18T10:44:24.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.655+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8581f57e0 con 0x7fd858140660 2023-10-18T10:44:24.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.656+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd858137b90 con 0x7fd858140660 2023-10-18T10:44:24.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.656+0000 7fd84dffb640 1 -- 172.21.15.125:0/2391986235 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd850003ce0 con 0x7fd858140660 2023-10-18T10:44:24.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.656+0000 7fd84dffb640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fd83c06af90 0x7fd83c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.656+0000 7fd84dffb640 1 -- 172.21.15.125:0/2391986235 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fd850088f60 con 0x7fd858140660 2023-10-18T10:44:24.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.657+0000 7fd85d391640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fd83c06af90 0x7fd83c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.657+0000 7fd85d391640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fd83c06af90 0x7fd83c06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd854005fd0 tx=0x7fd85400a4d0 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:24.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.660+0000 7fd84dffb640 1 -- 172.21.15.125:0/2391986235 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd858137b90 con 0x7fd858140660 2023-10-18T10:44:24.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.822+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 --> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd858137b90 con 0x7fd83c06af90 2023-10-18T10:44:24.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.830+0000 7fd84dffb640 1 -- 172.21.15.125:0/2391986235 <== mgr.7786 v2:172.21.15.139:6832/4144373589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274560 (secure 0 0 0) 0x7fd858137b90 con 0x7fd83c06af90 2023-10-18T10:44:24.834 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:24.835 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:44:24.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fd83c06af90 msgr2=0x7fd83c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fd83c06af90 0x7fd83c06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd854005fd0 tx=0x7fd85400a4d0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 msgr2=0x7fd8580b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 0x7fd8580b09a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fd85000c460 tx=0x7fd85000c920 comp rx=0 tx=0).stop 2023-10-18T10:44:24.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 shutdown_connections 2023-10-18T10:44:24.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fd83c06af90 0x7fd83c06d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd85813d360 0x7fd8581348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858140660 0x7fd8580b09a0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 --2- 172.21.15.125:0/2391986235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8580c9d00 0x7fd8581f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 >> 172.21.15.125:0/2391986235 conn(0x7fd858069930 msgr2=0x7fd8580cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:24.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 shutdown_connections 2023-10-18T10:44:24.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.833+0000 7fd85f5f5640 1 -- 172.21.15.125:0/2391986235 wait complete. 2023-10-18T10:44:24.845 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:44:23.281677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":334924,"kb_used_data":14652,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754639796,"statfs":{"total":773094113280,"available":772751151104,"internally_reserved":0,"allocated":15003648,"data_stored":8147767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":327798328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001043"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"131'236","reported_seq":697,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-18T10:44:16.178354+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:44:16.178354+0000","last_peered":"2023-10-18T10:44:16.178354+0000","last_clean":"2023-10-18T10:44:16.178354+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:44:16.178354+0000","last_undegraded":"2023-10-18T10:44:16.178354+0000","last_fullsized":"2023-10-18T10:44:16.178354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575081,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59820,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312020,"statfs":{"total":96636764160,"available":96575508480,"internally_reserved":0,"allocated":2322432,"data_stored":1462010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.31,"5min":0.217,"15min":0.217},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.88300000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.746,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.44},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.73099999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.496}]},{"osd":3,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.71399999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77800000000000002,"5min":0.79800000000000004,"15min":0.85799999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45600000000000002,"15min":0.439},"min":{"1min":0.28799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48099999999999998,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.627,"5min":0.70699999999999996,"15min":0.86199999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.74299999999999999,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.76500000000000001,"5min":0.875,"15min":0.875},"last":0.58199999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.73199999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.27000000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.91700000000000004},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72099999999999997,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.66600000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.74199999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.748,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.70099999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.75700000000000001,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.68200000000000005}]},{"osd":4,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.66200000000000003,"5min":0.872,"15min":0.872},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.747},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.87},"last":0.73199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50924,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320916,"statfs":{"total":96636764160,"available":96584617984,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.38800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.58799999999999997}]},{"osd":2,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.93600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.66500000000000004,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.84199999999999997},"last":0.498},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.75600000000000001,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.72299999999999998,"15min":0.80100000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.38900000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.83299999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.31},"max":{"1min":0.71099999999999997,"5min":0.84299999999999997,"15min":0.89300000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.85299999999999998,"15min":0.871},"last":0.44800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.76200000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.877},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83199999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.89300000000000002},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81999999999999995,"15min":0.85899999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.46500000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.873,"15min":0.873},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74199999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.505,"15min":0.498},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.86399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":1.446},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.84799999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":1.0469999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":0.874,"5min":0.88600000000000001,"15min":1.0880000000000001},"last":0.75900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.36699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.183},"max":{"1min":0.52900000000000003,"5min":0.58999999999999997,"15min":0.62},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.22800000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.27300000000000002,"5min":0.249,"15min":0.20799999999999999},"max":{"1min":0.55200000000000005,"5min":0.624,"15min":0.70699999999999996},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38600000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.61599999999999999,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.432,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23300000000000001},"max":{"1min":0.61599999999999999,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.28100000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47999999999999998,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.46800000000000003,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.621,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86599999999999999,"5min":1.002,"15min":1.002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.63},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.89500000000000002,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.37},"max":{"1min":0.78100000000000003,"5min":1.018,"15min":1.018},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.44900000000000001,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":0.89800000000000002,"15min":0.96799999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.57599999999999996,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.95799999999999996,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.67200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55080,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316760,"statfs":{"total":96636764160,"available":96580362240,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.375,"15min":0.375},"min":{"1min":0.223,"5min":0.223,"15min":0.17599999999999999},"max":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.54100000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.40300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.503,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42099999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.61699999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.95699999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.87,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.46600000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.91400000000000003,"15min":0.93600000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.46000000000000002,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.441,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.45100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.59199999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.376}]},{"osd":2,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64800000000000002},"last":0.33800000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56999999999999995,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.35399999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38,"5min":0.377,"15min":0.374},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.44}]},{"osd":6,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.32400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63200000000000001},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.77800000000000002,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.46899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575080,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37600,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334240,"statfs":{"total":96636764160,"available":96598261760,"internally_reserved":0,"allocated":2310144,"data_stored":1455905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36799999999999999,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.46600000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.62,"15min":0.76900000000000002},"last":0.374}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39700000000000002,"15min":0.40699999999999997},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.56399999999999995,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39500000000000002,"15min":0.41499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.69699999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.59199999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.438,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.89200000000000002,"5min":0.996,"15min":0.996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.92400000000000004,"5min":1.038,"15min":1.038},"last":0.66100000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.85799999999999998,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94099999999999995,"5min":1.026,"15min":1.3700000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94099999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.621}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.64300000000000002},"min":{"1min":0.46999999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97499999999999998,"5min":1.05,"15min":1.339},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":1.01,"15min":1.01},"last":0.57399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:44:24.852 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:44:24.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.986+0000 7f3218294640 1 -- 172.21.15.125:0/3342783223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 msgr2=0x7f32100ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.986+0000 7f3218294640 1 --2- 172.21.15.125:0/3342783223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 0x7f32100ab9b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3208005fd0 tx=0x7f320802c2e0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.986+0000 7f3218294640 1 -- 172.21.15.125:0/3342783223 shutdown_connections 2023-10-18T10:44:24.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.986+0000 7f3218294640 1 --2- 172.21.15.125:0/3342783223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 0x7f32100ab9b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.986+0000 7f3218294640 1 --2- 172.21.15.125:0/3342783223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32100c9dc0 0x7f321006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.986+0000 7f3218294640 1 -- 172.21.15.125:0/3342783223 >> 172.21.15.125:0/3342783223 conn(0x7f3210069930 msgr2=0x7f3210069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:24.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.986+0000 7f3218294640 1 -- 172.21.15.125:0/3342783223 shutdown_connections 2023-10-18T10:44:24.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.986+0000 7f3218294640 1 -- 172.21.15.125:0/3342783223 wait complete. 2023-10-18T10:44:24.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.987+0000 7f3218294640 1 Processor -- start 2023-10-18T10:44:24.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.987+0000 7f3218294640 1 -- start start 2023-10-18T10:44:24.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3218294640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f321006af00 0x7f32101f0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3218294640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 0x7f32101efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3218294640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32100c9dc0 0x7f3210150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3218294640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32101521d0 con 0x7f32100c9dc0 2023-10-18T10:44:24.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3218294640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3210151820 con 0x7f32100ab500 2023-10-18T10:44:24.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3218294640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210151ba0 con 0x7f321006af00 2023-10-18T10:44:24.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32100c9dc0 0x7f3210150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32100c9dc0 0x7f3210150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55686/0 (socket says 172.21.15.125:55686) 2023-10-18T10:44:24.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 -- 172.21.15.125:0/1427200661 learned_addr learned my addr 172.21.15.125:0/1427200661 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:24.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f321582f640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 0x7f32101efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216030640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f321006af00 0x7f32101f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 -- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f321006af00 msgr2=0x7f32101f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f321006af00 0x7f32101f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 -- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 msgr2=0x7f32101efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:24.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 0x7f32101efe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:24.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 -- 172.21.15.125:0/1427200661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3208009900 con 0x7f32100c9dc0 2023-10-18T10:44:24.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f321582f640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 0x7f32101efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:44:24.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3216831640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32100c9dc0 0x7f3210150330 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f3208009bf0 tx=0x7f3208035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:24.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.988+0000 7f3202ffd640 1 -- 172.21.15.125:0/1427200661 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208004240 con 0x7f32100c9dc0 2023-10-18T10:44:24.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.989+0000 7f3202ffd640 1 -- 172.21.15.125:0/1427200661 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3208004490 con 0x7f32100c9dc0 2023-10-18T10:44:24.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.989+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3210150ef0 con 0x7f32100c9dc0 2023-10-18T10:44:24.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.989+0000 7f3202ffd640 1 -- 172.21.15.125:0/1427200661 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208004860 con 0x7f32100c9dc0 2023-10-18T10:44:24.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.989+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3210151e60 con 0x7f32100c9dc0 2023-10-18T10:44:24.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.989+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3210152380 con 0x7f32100c9dc0 2023-10-18T10:44:24.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.990+0000 7f3202ffd640 1 -- 172.21.15.125:0/1427200661 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f320803b050 con 0x7f32100c9dc0 2023-10-18T10:44:24.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.990+0000 7f3202ffd640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f31f406af60 0x7f31f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:24.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.990+0000 7f3202ffd640 1 -- 172.21.15.125:0/1427200661 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f32080ac4e0 con 0x7f32100c9dc0 2023-10-18T10:44:24.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.991+0000 7f3216030640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f31f406af60 0x7f31f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:24.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.991+0000 7f3216030640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f31f406af60 0x7f31f406d420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f320c001780 tx=0x7f320c009040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:25.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:24.994+0000 7f3202ffd640 1 -- 172.21.15.125:0/1427200661 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3210152380 con 0x7f32100c9dc0 2023-10-18T10:44:25.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.157+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 --> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3210159c50 con 0x7f31f406af60 2023-10-18T10:44:25.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.165+0000 7f3202ffd640 1 -- 172.21.15.125:0/1427200661 <== mgr.7786 v2:172.21.15.139:6832/4144373589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274560 (secure 0 0 0) 0x7f320c001420 con 0x7f31f406af60 2023-10-18T10:44:25.169 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:25.171 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:44:25.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f31f406af60 msgr2=0x7f31f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f31f406af60 0x7f31f406d420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f320c001780 tx=0x7f320c009040 comp rx=0 tx=0).stop 2023-10-18T10:44:25.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32100c9dc0 msgr2=0x7f3210150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32100c9dc0 0x7f3210150330 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f3208009bf0 tx=0x7f3208035b50 comp rx=0 tx=0).stop 2023-10-18T10:44:25.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 shutdown_connections 2023-10-18T10:44:25.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f31f406af60 0x7f31f406d420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32100ab500 0x7f32101efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32100c9dc0 0x7f3210150330 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 --2- 172.21.15.125:0/1427200661 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f321006af00 0x7f32101f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 >> 172.21.15.125:0/1427200661 conn(0x7f3210069930 msgr2=0x7f3210069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:25.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 shutdown_connections 2023-10-18T10:44:25.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.168+0000 7f3218294640 1 -- 172.21.15.125:0/1427200661 wait complete. 2023-10-18T10:44:25.180 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:44:23.281677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":334924,"kb_used_data":14652,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754639796,"statfs":{"total":773094113280,"available":772751151104,"internally_reserved":0,"allocated":15003648,"data_stored":8147767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":327798328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001043"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:20.538434+0000","last_change":"2023-10-18T10:44:20.538434+0000","last_active":"2023-10-18T10:44:20.538434+0000","last_peered":"2023-10-18T10:44:20.538434+0000","last_clean":"2023-10-18T10:44:20.538434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:20.538434+0000","last_undegraded":"2023-10-18T10:44:20.538434+0000","last_fullsized":"2023-10-18T10:44:20.538434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:19.533540+0000","last_change":"2023-10-18T10:44:19.533540+0000","last_active":"2023-10-18T10:44:19.533540+0000","last_peered":"2023-10-18T10:44:19.533540+0000","last_clean":"2023-10-18T10:44:19.533540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:19.533540+0000","last_undegraded":"2023-10-18T10:44:19.533540+0000","last_fullsized":"2023-10-18T10:44:19.533540+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"131'236","reported_seq":697,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-18T10:44:16.178354+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:44:16.178354+0000","last_peered":"2023-10-18T10:44:16.178354+0000","last_clean":"2023-10-18T10:44:16.178354+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:44:16.178354+0000","last_undegraded":"2023-10-18T10:44:16.178354+0000","last_fullsized":"2023-10-18T10:44:16.178354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575081,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59820,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312020,"statfs":{"total":96636764160,"available":96575508480,"internally_reserved":0,"allocated":2322432,"data_stored":1462010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.31,"5min":0.217,"15min":0.217},"max":{"1min":0.80100000000000005,"5min":0.876,"15min":0.88300000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.746,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.44},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.90700000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.73099999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.496}]},{"osd":3,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.71399999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77800000000000002,"5min":0.79800000000000004,"15min":0.85799999999999998},"last":0.57199999999999995}]},{"osd":4,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45600000000000002,"15min":0.439},"min":{"1min":0.28799999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48099999999999998,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.627,"5min":0.70699999999999996,"15min":0.86199999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.74299999999999999,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.76500000000000001,"5min":0.875,"15min":0.875},"last":0.58199999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.33200000000000002},"max":{"1min":0.73199999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.27000000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.91700000000000004},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72099999999999997,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.66600000000000004}]},{"osd":2,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.74199999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.748,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.70099999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.75700000000000001,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.68200000000000005}]},{"osd":4,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.66200000000000003,"5min":0.872,"15min":0.872},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.747},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.87},"last":0.73199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50924,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320916,"statfs":{"total":96636764160,"available":96584617984,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.38800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.58799999999999997}]},{"osd":2,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.93600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.66500000000000004,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.60499999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.84199999999999997},"last":0.498},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.75600000000000001,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.72299999999999998,"15min":0.80100000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.38900000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.83299999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.31},"max":{"1min":0.71099999999999997,"5min":0.84299999999999997,"15min":0.89300000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.85299999999999998,"15min":0.871},"last":0.44800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.76200000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.877},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83199999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.89300000000000002},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81999999999999995,"15min":0.85899999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.46500000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.873,"15min":0.873},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74199999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.505,"15min":0.498},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.86399999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":1.446},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.84799999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":1.0469999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":0.874,"5min":0.88600000000000001,"15min":1.0880000000000001},"last":0.75900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.36699999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.183},"max":{"1min":0.52900000000000003,"5min":0.58999999999999997,"15min":0.62},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.22800000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.27300000000000002,"5min":0.249,"15min":0.20799999999999999},"max":{"1min":0.55200000000000005,"5min":0.624,"15min":0.70699999999999996},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38600000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.434},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.61599999999999999,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.432,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23300000000000001},"max":{"1min":0.61599999999999999,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.28100000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47999999999999998,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.76600000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.46800000000000003,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.621,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86599999999999999,"5min":1.002,"15min":1.002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.63},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.371},"max":{"1min":0.89500000000000002,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.37},"max":{"1min":0.78100000000000003,"5min":1.018,"15min":1.018},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.626,"15min":0.63900000000000001},"min":{"1min":0.45400000000000001,"5min":0.44900000000000001,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":0.89800000000000002,"15min":0.96799999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.57599999999999996,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.95799999999999996,"5min":0.96699999999999997,"15min":0.98899999999999999},"last":0.67200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55080,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316760,"statfs":{"total":96636764160,"available":96580362240,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.375,"15min":0.375},"min":{"1min":0.223,"5min":0.223,"15min":0.17599999999999999},"max":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.54100000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.40300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.503,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.42099999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.61699999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.95699999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.87,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.46600000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.91400000000000003,"15min":0.93600000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.46000000000000002,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.441,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.45100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":871642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.59199999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.376}]},{"osd":2,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64800000000000002},"last":0.33800000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56999999999999995,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.35399999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38,"5min":0.377,"15min":0.374},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.44}]},{"osd":6,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.32400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63200000000000001},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.77800000000000002,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.46899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575080,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37600,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334240,"statfs":{"total":96636764160,"available":96598261760,"internally_reserved":0,"allocated":2310144,"data_stored":1455905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36799999999999999,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.46600000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.62,"15min":0.76900000000000002},"last":0.374}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39700000000000002,"15min":0.40699999999999997},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.56399999999999995,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39500000000000002,"15min":0.41499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.69699999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.59199999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.438,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.89200000000000002,"5min":0.996,"15min":0.996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.92400000000000004,"5min":1.038,"15min":1.038},"last":0.66100000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.85799999999999998,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94099999999999995,"5min":1.026,"15min":1.3700000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94099999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.621}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.64300000000000002},"min":{"1min":0.46999999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97499999999999998,"5min":1.05,"15min":1.339},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":1.01,"15min":1.01},"last":0.57399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:44:25.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3f is not active+clean 2023-10-18T10:44:25.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3e is not active+clean 2023-10-18T10:44:25.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3d is not active+clean 2023-10-18T10:44:25.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3c is not active+clean 2023-10-18T10:44:25.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3b is not active+clean 2023-10-18T10:44:25.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3a is not active+clean 2023-10-18T10:44:25.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.39 is not active+clean 2023-10-18T10:44:25.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.38 is not active+clean 2023-10-18T10:44:25.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.37 is not active+clean 2023-10-18T10:44:25.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.36 is not active+clean 2023-10-18T10:44:25.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.35 is not active+clean 2023-10-18T10:44:25.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.34 is not active+clean 2023-10-18T10:44:25.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.33 is not active+clean 2023-10-18T10:44:25.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.32 is not active+clean 2023-10-18T10:44:25.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.31 is not active+clean 2023-10-18T10:44:25.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.30 is not active+clean 2023-10-18T10:44:25.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2f is not active+clean 2023-10-18T10:44:25.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2e is not active+clean 2023-10-18T10:44:25.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2d is not active+clean 2023-10-18T10:44:25.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2c is not active+clean 2023-10-18T10:44:25.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2b is not active+clean 2023-10-18T10:44:25.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2a is not active+clean 2023-10-18T10:44:25.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.29 is not active+clean 2023-10-18T10:44:25.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.28 is not active+clean 2023-10-18T10:44:25.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.27 is not active+clean 2023-10-18T10:44:25.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.26 is not active+clean 2023-10-18T10:44:25.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.25 is not active+clean 2023-10-18T10:44:25.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.24 is not active+clean 2023-10-18T10:44:25.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.23 is not active+clean 2023-10-18T10:44:25.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.22 is not active+clean 2023-10-18T10:44:25.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.21 is not active+clean 2023-10-18T10:44:25.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.20 is not active+clean 2023-10-18T10:44:25.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2023-10-18T10:44:25.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2023-10-18T10:44:25.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2023-10-18T10:44:25.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2023-10-18T10:44:25.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2023-10-18T10:44:25.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2023-10-18T10:44:25.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2023-10-18T10:44:25.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2023-10-18T10:44:25.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2023-10-18T10:44:25.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2023-10-18T10:44:25.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2023-10-18T10:44:25.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-10-18T10:44:25.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2023-10-18T10:44:25.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-10-18T10:44:25.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2023-10-18T10:44:25.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-10-18T10:44:25.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2023-10-18T10:44:25.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2023-10-18T10:44:25.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2023-10-18T10:44:25.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-10-18T10:44:25.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2023-10-18T10:44:25.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-10-18T10:44:25.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2023-10-18T10:44:25.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2023-10-18T10:44:25.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2023-10-18T10:44:25.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2023-10-18T10:44:25.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2023-10-18T10:44:25.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2023-10-18T10:44:25.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2023-10-18T10:44:25.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2023-10-18T10:44:25.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2023-10-18T10:44:25.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2023-10-18T10:44:25.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2023-10-18T10:44:25.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2023-10-18T10:44:25.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2023-10-18T10:44:25.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2023-10-18T10:44:25.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2023-10-18T10:44:25.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2023-10-18T10:44:25.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2023-10-18T10:44:25.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-10-18T10:44:25.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2023-10-18T10:44:25.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2023-10-18T10:44:25.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2023-10-18T10:44:25.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2023-10-18T10:44:25.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2023-10-18T10:44:25.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2023-10-18T10:44:25.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2023-10-18T10:44:25.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-10-18T10:44:25.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2023-10-18T10:44:25.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2023-10-18T10:44:25.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2023-10-18T10:44:25.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2023-10-18T10:44:25.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2023-10-18T10:44:25.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-10-18T10:44:25.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2023-10-18T10:44:25.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2023-10-18T10:44:25.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2023-10-18T10:44:25.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2023-10-18T10:44:25.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2023-10-18T10:44:25.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-10-18T10:44:25.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2023-10-18T10:44:25.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2023-10-18T10:44:25.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2023-10-18T10:44:25.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:20.538434+0000', 'last_change': '2023-10-18T10:44:20.538434+0000', 'last_active': '2023-10-18T10:44:20.538434+0000', 'last_peered': '2023-10-18T10:44:20.538434+0000', 'last_clean': '2023-10-18T10:44:20.538434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:20.538434+0000', 'last_undegraded': '2023-10-18T10:44:20.538434+0000', 'last_fullsized': '2023-10-18T10:44:20.538434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:20.538434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2023-10-18T10:44:25.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:19.533540+0000', 'last_change': '2023-10-18T10:44:19.533540+0000', 'last_active': '2023-10-18T10:44:19.533540+0000', 'last_peered': '2023-10-18T10:44:19.533540+0000', 'last_clean': '2023-10-18T10:44:19.533540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:19.533540+0000', 'last_undegraded': '2023-10-18T10:44:19.533540+0000', 'last_fullsized': '2023-10-18T10:44:19.533540+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:19.533540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:44:25.258 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T10:44:25.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.357+0000 7f96f99d7640 1 -- 172.21.15.125:0/3505491128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f406ae40 msgr2=0x7f96f40b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.357+0000 7f96f99d7640 1 --2- 172.21.15.125:0/3505491128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f406ae40 0x7f96f40b0e90 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f96e8005fd0 tx=0x7f96e802be10 comp rx=0 tx=0).stop 2023-10-18T10:44:25.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.358+0000 7f96f99d7640 1 -- 172.21.15.125:0/3505491128 shutdown_connections 2023-10-18T10:44:25.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.358+0000 7f96f99d7640 1 --2- 172.21.15.125:0/3505491128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96f415c1f0 0x7f96f4157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.358+0000 7f96f99d7640 1 --2- 172.21.15.125:0/3505491128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f406ae40 0x7f96f40b0e90 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.358+0000 7f96f99d7640 1 -- 172.21.15.125:0/3505491128 >> 172.21.15.125:0/3505491128 conn(0x7f96f406fae0 msgr2=0x7f96f406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:25.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.358+0000 7f96f99d7640 1 -- 172.21.15.125:0/3505491128 shutdown_connections 2023-10-18T10:44:25.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.358+0000 7f96f99d7640 1 -- 172.21.15.125:0/3505491128 wait complete. 2023-10-18T10:44:25.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.359+0000 7f96f99d7640 1 Processor -- start 2023-10-18T10:44:25.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.359+0000 7f96f99d7640 1 -- start start 2023-10-18T10:44:25.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.359+0000 7f96f99d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96f406ae40 0x7f96f4153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:25.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f99d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f41579f0 0x7f96f413a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:25.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f99d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96f415c1f0 0x7f96f4156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:25.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f99d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f413ae80 con 0x7f96f41579f0 2023-10-18T10:44:25.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f99d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f413b360 con 0x7f96f406ae40 2023-10-18T10:44:25.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f99d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f41571e0 con 0x7f96f415c1f0 2023-10-18T10:44:25.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f41579f0 0x7f96f413a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:25.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f41579f0 0x7f96f413a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55696/0 (socket says 172.21.15.125:55696) 2023-10-18T10:44:25.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 -- 172.21.15.125:0/2580951456 learned_addr learned my addr 172.21.15.125:0/2580951456 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:25.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f37fe640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96f415c1f0 0x7f96f4156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:25.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f2ffd640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96f406ae40 0x7f96f4153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:25.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 -- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96f415c1f0 msgr2=0x7f96f4156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96f415c1f0 0x7f96f4156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 -- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96f406ae40 msgr2=0x7f96f4153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96f406ae40 0x7f96f4153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 -- 172.21.15.125:0/2580951456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e8009560 con 0x7f96f41579f0 2023-10-18T10:44:25.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f27fc640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f41579f0 0x7f96f413a940 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f96e8006010 tx=0x7f96e802dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:25.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96dbfff640 1 -- 172.21.15.125:0/2580951456 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e80035f0 con 0x7f96f41579f0 2023-10-18T10:44:25.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96dbfff640 1 -- 172.21.15.125:0/2580951456 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96e8003840 con 0x7f96f41579f0 2023-10-18T10:44:25.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f4154940 con 0x7f96f41579f0 2023-10-18T10:44:25.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96dbfff640 1 -- 172.21.15.125:0/2580951456 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8003c50 con 0x7f96f41579f0 2023-10-18T10:44:25.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.360+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f4145710 con 0x7f96f41579f0 2023-10-18T10:44:25.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.361+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f4145570 con 0x7f96f41579f0 2023-10-18T10:44:25.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.362+0000 7f96dbfff640 1 -- 172.21.15.125:0/2580951456 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f96e803e450 con 0x7f96f41579f0 2023-10-18T10:44:25.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.362+0000 7f96dbfff640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f96e006af60 0x7f96e006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:25.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.362+0000 7f96dbfff640 1 -- 172.21.15.125:0/2580951456 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f96e80ac360 con 0x7f96f41579f0 2023-10-18T10:44:25.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.362+0000 7f96f2ffd640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f96e006af60 0x7f96e006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:25.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.363+0000 7f96f2ffd640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f96e006af60 0x7f96e006d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f96e4001f10 tx=0x7f96e4009040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:25.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.366+0000 7f96dbfff640 1 -- 172.21.15.125:0/2580951456 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96f4145570 con 0x7f96f41579f0 2023-10-18T10:44:25.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.598+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f96f4145570 con 0x7f96f41579f0 2023-10-18T10:44:25.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.598+0000 7f96dbfff640 1 -- 172.21.15.125:0/2580951456 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1014 (secure 0 0 0) 0x7f96f4145570 con 0x7f96f41579f0 2023-10-18T10:44:25.601 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:25.601 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1187,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":143,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":347615232,"bytes_avail":772746498048,"bytes_total":773094113280,"write_bytes_sec":2496,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":53,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":9159}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-18T10:42:45.920324+0000","services":{}},"progress_events":{}} 2023-10-18T10:44:25.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f96e006af60 msgr2=0x7f96e006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f96e006af60 0x7f96e006d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f96e4001f10 tx=0x7f96e4009040 comp rx=0 tx=0).stop 2023-10-18T10:44:25.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f41579f0 msgr2=0x7f96f413a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f41579f0 0x7f96f413a940 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f96e8006010 tx=0x7f96e802dbb0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 shutdown_connections 2023-10-18T10:44:25.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f96e006af60 0x7f96e006d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96f415c1f0 0x7f96f4156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96f41579f0 0x7f96f413a940 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 --2- 172.21.15.125:0/2580951456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96f406ae40 0x7f96f4153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 >> 172.21.15.125:0/2580951456 conn(0x7f96f406fae0 msgr2=0x7f96f4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:25.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 shutdown_connections 2023-10-18T10:44:25.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.601+0000 7f96f99d7640 1 -- 172.21.15.125:0/2580951456 wait complete. 2023-10-18T10:44:25.613 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T10:44:25.614 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:44:25.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.713+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/4047446432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd5413e520 msgr2=0x7fcd5406bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.713+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/4047446432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd5413e520 0x7fcd5406bb60 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009830 tx=0x7fcd5002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.713+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/4047446432 shutdown_connections 2023-10-18T10:44:25.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.713+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/4047446432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd54070af0 0x7fcd540affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.713+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/4047446432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd5413e520 0x7fcd5406bb60 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.713+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/4047446432 >> 172.21.15.125:0/4047446432 conn(0x7fcd54069930 msgr2=0x7fcd54069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:25.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.714+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/4047446432 shutdown_connections 2023-10-18T10:44:25.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.714+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/4047446432 wait complete. 2023-10-18T10:44:25.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5c5a9640 1 Processor -- start 2023-10-18T10:44:25.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5c5a9640 1 -- start start 2023-10-18T10:44:25.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5c5a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd54070af0 0x7fcd541ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:25.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5c5a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd5413e520 0x7fcd540a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:25.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5c5a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd54141690 0x7fcd541e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:25.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5c5a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540a7a30 con 0x7fcd54070af0 2023-10-18T10:44:25.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5c5a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540a6b10 con 0x7fcd5413e520 2023-10-18T10:44:25.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5c5a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540a6db0 con 0x7fcd54141690 2023-10-18T10:44:25.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.715+0000 7fcd5a345640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd54070af0 0x7fcd541ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:25.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5a345640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd54070af0 0x7fcd541ddae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55706/0 (socket says 172.21.15.125:55706) 2023-10-18T10:44:25.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5a345640 1 -- 172.21.15.125:0/62836219 learned_addr learned my addr 172.21.15.125:0/62836219 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:25.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5ab46640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd54141690 0x7fcd541e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:25.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5a345640 1 -- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd54141690 msgr2=0x7fcd541e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd59b44640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd5413e520 0x7fcd540a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:25.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5a345640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd54141690 0x7fcd541e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5a345640 1 -- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd5413e520 msgr2=0x7fcd540a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5a345640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd5413e520 0x7fcd540a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5a345640 1 -- 172.21.15.125:0/62836219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009490 con 0x7fcd54070af0 2023-10-18T10:44:25.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5ab46640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd54141690 0x7fcd541e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:25.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd59b44640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd5413e520 0x7fcd540a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:25.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5a345640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd54070af0 0x7fcd541ddae0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009780 tx=0x7fcd500038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:25.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd4affd640 1 -- 172.21.15.125:0/62836219 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50003ea0 con 0x7fcd54070af0 2023-10-18T10:44:25.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd4affd640 1 -- 172.21.15.125:0/62836219 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd50034550 con 0x7fcd54070af0 2023-10-18T10:44:25.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd540a7090 con 0x7fcd54070af0 2023-10-18T10:44:25.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd4affd640 1 -- 172.21.15.125:0/62836219 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50034960 con 0x7fcd54070af0 2023-10-18T10:44:25.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.716+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd540a7650 con 0x7fcd54070af0 2023-10-18T10:44:25.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.717+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd540a9330 con 0x7fcd54070af0 2023-10-18T10:44:25.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.718+0000 7fcd4affd640 1 -- 172.21.15.125:0/62836219 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcd5003b040 con 0x7fcd54070af0 2023-10-18T10:44:25.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.718+0000 7fcd4affd640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fcd3806af60 0x7fcd3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:25.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.718+0000 7fcd4affd640 1 -- 172.21.15.125:0/62836219 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fcd500abea0 con 0x7fcd54070af0 2023-10-18T10:44:25.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.718+0000 7fcd59b44640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fcd3806af60 0x7fcd3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:25.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.719+0000 7fcd59b44640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fcd3806af60 0x7fcd3806d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd540c8ee0 tx=0x7fcd44009040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:25.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.722+0000 7fcd4affd640 1 -- 172.21.15.125:0/62836219 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd50076ce0 con 0x7fcd54070af0 2023-10-18T10:44:25.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.885+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 --> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd540a7ca0 con 0x7fcd3806af60 2023-10-18T10:44:25.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.894+0000 7fcd4affd640 1 -- 172.21.15.125:0/62836219 <== mgr.7786 v2:172.21.15.139:6832/4144373589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289577 (secure 0 0 0) 0x7fcd540a7ca0 con 0x7fcd3806af60 2023-10-18T10:44:25.898 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:25.899 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:44:25.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.896+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fcd3806af60 msgr2=0x7fcd3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.896+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fcd3806af60 0x7fcd3806d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd540c8ee0 tx=0x7fcd44009040 comp rx=0 tx=0).stop 2023-10-18T10:44:25.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.896+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd54070af0 msgr2=0x7fcd541ddae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:25.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.896+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd54070af0 0x7fcd541ddae0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009780 tx=0x7fcd500038a0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.897+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 shutdown_connections 2023-10-18T10:44:25.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.897+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fcd3806af60 0x7fcd3806d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.897+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd5413e520 0x7fcd540a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.897+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd54141690 0x7fcd541e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.897+0000 7fcd5c5a9640 1 --2- 172.21.15.125:0/62836219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd54070af0 0x7fcd541ddae0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:25.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.897+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 >> 172.21.15.125:0/62836219 conn(0x7fcd54069930 msgr2=0x7fcd541df6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:25.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.897+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 shutdown_connections 2023-10-18T10:44:25.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:25.897+0000 7fcd5c5a9640 1 -- 172.21.15.125:0/62836219 wait complete. 2023-10-18T10:44:25.909 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-18T10:44:25.282166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":419,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339468,"kb_used_data":15052,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754635252,"statfs":{"total":773094113280,"available":772746498048,"internally_reserved":0,"allocated":15413248,"data_stored":8337924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":332058168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742877"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564850+0000","last_change":"2023-10-18T10:44:21.564341+0000","last_active":"2023-10-18T10:44:22.564850+0000","last_peered":"2023-10-18T10:44:22.564850+0000","last_clean":"2023-10-18T10:44:22.564850+0000","last_became_active":"2023-10-18T10:44:21.564034+0000","last_became_peered":"2023-10-18T10:44:21.564034+0000","last_unstale":"2023-10-18T10:44:22.564850+0000","last_undegraded":"2023-10-18T10:44:22.564850+0000","last_fullsized":"2023-10-18T10:44:22.564850+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:00.535183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578575+0000","last_change":"2023-10-18T10:44:21.556185+0000","last_active":"2023-10-18T10:44:22.578575+0000","last_peered":"2023-10-18T10:44:22.578575+0000","last_clean":"2023-10-18T10:44:22.578575+0000","last_became_active":"2023-10-18T10:44:21.555918+0000","last_became_peered":"2023-10-18T10:44:21.555918+0000","last_unstale":"2023-10-18T10:44:22.578575+0000","last_undegraded":"2023-10-18T10:44:22.578575+0000","last_fullsized":"2023-10-18T10:44:22.578575+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:14.477242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578619+0000","last_change":"2023-10-18T10:44:21.560236+0000","last_active":"2023-10-18T10:44:22.578619+0000","last_peered":"2023-10-18T10:44:22.578619+0000","last_clean":"2023-10-18T10:44:22.578619+0000","last_became_active":"2023-10-18T10:44:21.559948+0000","last_became_peered":"2023-10-18T10:44:21.559948+0000","last_unstale":"2023-10-18T10:44:22.578619+0000","last_undegraded":"2023-10-18T10:44:22.578619+0000","last_fullsized":"2023-10-18T10:44:22.578619+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:25.847090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230045+0000","last_change":"2023-10-18T10:44:21.566112+0000","last_active":"2023-10-18T10:44:23.230045+0000","last_peered":"2023-10-18T10:44:23.230045+0000","last_clean":"2023-10-18T10:44:23.230045+0000","last_became_active":"2023-10-18T10:44:21.565678+0000","last_became_peered":"2023-10-18T10:44:21.565678+0000","last_unstale":"2023-10-18T10:44:23.230045+0000","last_undegraded":"2023-10-18T10:44:23.230045+0000","last_fullsized":"2023-10-18T10:44:23.230045+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:49.126268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564305+0000","last_change":"2023-10-18T10:44:21.566246+0000","last_active":"2023-10-18T10:44:22.564305+0000","last_peered":"2023-10-18T10:44:22.564305+0000","last_clean":"2023-10-18T10:44:22.564305+0000","last_became_active":"2023-10-18T10:44:21.565923+0000","last_became_peered":"2023-10-18T10:44:21.565923+0000","last_unstale":"2023-10-18T10:44:22.564305+0000","last_undegraded":"2023-10-18T10:44:22.564305+0000","last_fullsized":"2023-10-18T10:44:22.564305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:46.001986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229705+0000","last_change":"2023-10-18T10:44:21.573416+0000","last_active":"2023-10-18T10:44:23.229705+0000","last_peered":"2023-10-18T10:44:23.229705+0000","last_clean":"2023-10-18T10:44:23.229705+0000","last_became_active":"2023-10-18T10:44:21.573170+0000","last_became_peered":"2023-10-18T10:44:21.573170+0000","last_unstale":"2023-10-18T10:44:23.229705+0000","last_undegraded":"2023-10-18T10:44:23.229705+0000","last_fullsized":"2023-10-18T10:44:23.229705+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:15.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229492+0000","last_change":"2023-10-18T10:44:21.574273+0000","last_active":"2023-10-18T10:44:23.229492+0000","last_peered":"2023-10-18T10:44:23.229492+0000","last_clean":"2023-10-18T10:44:23.229492+0000","last_became_active":"2023-10-18T10:44:21.574036+0000","last_became_peered":"2023-10-18T10:44:21.574036+0000","last_unstale":"2023-10-18T10:44:23.229492+0000","last_undegraded":"2023-10-18T10:44:23.229492+0000","last_fullsized":"2023-10-18T10:44:23.229492+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:29:43.710418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578168+0000","last_change":"2023-10-18T10:44:21.560023+0000","last_active":"2023-10-18T10:44:22.578168+0000","last_peered":"2023-10-18T10:44:22.578168+0000","last_clean":"2023-10-18T10:44:22.578168+0000","last_became_active":"2023-10-18T10:44:21.559761+0000","last_became_peered":"2023-10-18T10:44:21.559761+0000","last_unstale":"2023-10-18T10:44:22.578168+0000","last_undegraded":"2023-10-18T10:44:22.578168+0000","last_fullsized":"2023-10-18T10:44:22.578168+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:37.267378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230398+0000","last_change":"2023-10-18T10:44:21.570058+0000","last_active":"2023-10-18T10:44:23.230398+0000","last_peered":"2023-10-18T10:44:23.230398+0000","last_clean":"2023-10-18T10:44:23.230398+0000","last_became_active":"2023-10-18T10:44:21.569716+0000","last_became_peered":"2023-10-18T10:44:21.569716+0000","last_unstale":"2023-10-18T10:44:23.230398+0000","last_undegraded":"2023-10-18T10:44:23.230398+0000","last_fullsized":"2023-10-18T10:44:23.230398+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:01.830388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229428+0000","last_change":"2023-10-18T10:44:21.577695+0000","last_active":"2023-10-18T10:44:23.229428+0000","last_peered":"2023-10-18T10:44:23.229428+0000","last_clean":"2023-10-18T10:44:23.229428+0000","last_became_active":"2023-10-18T10:44:21.577436+0000","last_became_peered":"2023-10-18T10:44:21.577436+0000","last_unstale":"2023-10-18T10:44:23.229428+0000","last_undegraded":"2023-10-18T10:44:23.229428+0000","last_fullsized":"2023-10-18T10:44:23.229428+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:34:05.944034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564344+0000","last_change":"2023-10-18T10:44:21.572558+0000","last_active":"2023-10-18T10:44:22.564344+0000","last_peered":"2023-10-18T10:44:22.564344+0000","last_clean":"2023-10-18T10:44:22.564344+0000","last_became_active":"2023-10-18T10:44:21.571984+0000","last_became_peered":"2023-10-18T10:44:21.571984+0000","last_unstale":"2023-10-18T10:44:22.564344+0000","last_undegraded":"2023-10-18T10:44:22.564344+0000","last_fullsized":"2023-10-18T10:44:22.564344+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:42:55.664463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229952+0000","last_change":"2023-10-18T10:44:21.571295+0000","last_active":"2023-10-18T10:44:23.229952+0000","last_peered":"2023-10-18T10:44:23.229952+0000","last_clean":"2023-10-18T10:44:23.229952+0000","last_became_active":"2023-10-18T10:44:21.570928+0000","last_became_peered":"2023-10-18T10:44:21.570928+0000","last_unstale":"2023-10-18T10:44:23.229952+0000","last_undegraded":"2023-10-18T10:44:23.229952+0000","last_fullsized":"2023-10-18T10:44:23.229952+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:53:26.906540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229641+0000","last_change":"2023-10-18T10:44:21.574413+0000","last_active":"2023-10-18T10:44:23.229641+0000","last_peered":"2023-10-18T10:44:23.229641+0000","last_clean":"2023-10-18T10:44:23.229641+0000","last_became_active":"2023-10-18T10:44:21.574117+0000","last_became_peered":"2023-10-18T10:44:21.574117+0000","last_unstale":"2023-10-18T10:44:23.229641+0000","last_undegraded":"2023-10-18T10:44:23.229641+0000","last_fullsized":"2023-10-18T10:44:23.229641+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:12.650215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578339+0000","last_change":"2023-10-18T10:44:21.567413+0000","last_active":"2023-10-18T10:44:22.578339+0000","last_peered":"2023-10-18T10:44:22.578339+0000","last_clean":"2023-10-18T10:44:22.578339+0000","last_became_active":"2023-10-18T10:44:21.567166+0000","last_became_peered":"2023-10-18T10:44:21.567166+0000","last_unstale":"2023-10-18T10:44:22.578339+0000","last_undegraded":"2023-10-18T10:44:22.578339+0000","last_fullsized":"2023-10-18T10:44:22.578339+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:15.656446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564059+0000","last_change":"2023-10-18T10:44:21.572580+0000","last_active":"2023-10-18T10:44:22.564059+0000","last_peered":"2023-10-18T10:44:22.564059+0000","last_clean":"2023-10-18T10:44:22.564059+0000","last_became_active":"2023-10-18T10:44:21.572256+0000","last_became_peered":"2023-10-18T10:44:21.572256+0000","last_unstale":"2023-10-18T10:44:22.564059+0000","last_undegraded":"2023-10-18T10:44:22.564059+0000","last_fullsized":"2023-10-18T10:44:22.564059+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:45.613040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229560+0000","last_change":"2023-10-18T10:44:21.569232+0000","last_active":"2023-10-18T10:44:23.229560+0000","last_peered":"2023-10-18T10:44:23.229560+0000","last_clean":"2023-10-18T10:44:23.229560+0000","last_became_active":"2023-10-18T10:44:21.569008+0000","last_became_peered":"2023-10-18T10:44:21.569008+0000","last_unstale":"2023-10-18T10:44:23.229560+0000","last_undegraded":"2023-10-18T10:44:23.229560+0000","last_fullsized":"2023-10-18T10:44:23.229560+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:08.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229558+0000","last_change":"2023-10-18T10:44:21.573756+0000","last_active":"2023-10-18T10:44:23.229558+0000","last_peered":"2023-10-18T10:44:23.229558+0000","last_clean":"2023-10-18T10:44:23.229558+0000","last_became_active":"2023-10-18T10:44:21.573513+0000","last_became_peered":"2023-10-18T10:44:21.573513+0000","last_unstale":"2023-10-18T10:44:23.229558+0000","last_undegraded":"2023-10-18T10:44:23.229558+0000","last_fullsized":"2023-10-18T10:44:23.229558+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:36.510592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578668+0000","last_change":"2023-10-18T10:44:21.556821+0000","last_active":"2023-10-18T10:44:22.578668+0000","last_peered":"2023-10-18T10:44:22.578668+0000","last_clean":"2023-10-18T10:44:22.578668+0000","last_became_active":"2023-10-18T10:44:21.556568+0000","last_became_peered":"2023-10-18T10:44:21.556568+0000","last_unstale":"2023-10-18T10:44:22.578668+0000","last_undegraded":"2023-10-18T10:44:22.578668+0000","last_fullsized":"2023-10-18T10:44:22.578668+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:58:34.456605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229729+0000","last_change":"2023-10-18T10:44:21.565663+0000","last_active":"2023-10-18T10:44:23.229729+0000","last_peered":"2023-10-18T10:44:23.229729+0000","last_clean":"2023-10-18T10:44:23.229729+0000","last_became_active":"2023-10-18T10:44:21.565411+0000","last_became_peered":"2023-10-18T10:44:21.565411+0000","last_unstale":"2023-10-18T10:44:23.229729+0000","last_undegraded":"2023-10-18T10:44:23.229729+0000","last_fullsized":"2023-10-18T10:44:23.229729+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:38.817680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229141+0000","last_change":"2023-10-18T10:44:21.571929+0000","last_active":"2023-10-18T10:44:23.229141+0000","last_peered":"2023-10-18T10:44:23.229141+0000","last_clean":"2023-10-18T10:44:23.229141+0000","last_became_active":"2023-10-18T10:44:21.571674+0000","last_became_peered":"2023-10-18T10:44:21.571674+0000","last_unstale":"2023-10-18T10:44:23.229141+0000","last_undegraded":"2023-10-18T10:44:23.229141+0000","last_fullsized":"2023-10-18T10:44:23.229141+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:15.415236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230333+0000","last_change":"2023-10-18T10:44:21.563434+0000","last_active":"2023-10-18T10:44:23.230333+0000","last_peered":"2023-10-18T10:44:23.230333+0000","last_clean":"2023-10-18T10:44:23.230333+0000","last_became_active":"2023-10-18T10:44:21.563140+0000","last_became_peered":"2023-10-18T10:44:21.563140+0000","last_unstale":"2023-10-18T10:44:23.230333+0000","last_undegraded":"2023-10-18T10:44:23.230333+0000","last_fullsized":"2023-10-18T10:44:23.230333+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:37.968875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564486+0000","last_change":"2023-10-18T10:44:21.567529+0000","last_active":"2023-10-18T10:44:22.564486+0000","last_peered":"2023-10-18T10:44:22.564486+0000","last_clean":"2023-10-18T10:44:22.564486+0000","last_became_active":"2023-10-18T10:44:21.567226+0000","last_became_peered":"2023-10-18T10:44:21.567226+0000","last_unstale":"2023-10-18T10:44:22.564486+0000","last_undegraded":"2023-10-18T10:44:22.564486+0000","last_fullsized":"2023-10-18T10:44:22.564486+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:50.772585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229682+0000","last_change":"2023-10-18T10:44:21.571205+0000","last_active":"2023-10-18T10:44:23.229682+0000","last_peered":"2023-10-18T10:44:23.229682+0000","last_clean":"2023-10-18T10:44:23.229682+0000","last_became_active":"2023-10-18T10:44:21.570639+0000","last_became_peered":"2023-10-18T10:44:21.570639+0000","last_unstale":"2023-10-18T10:44:23.229682+0000","last_undegraded":"2023-10-18T10:44:23.229682+0000","last_fullsized":"2023-10-18T10:44:23.229682+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:28.045604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564020+0000","last_change":"2023-10-18T10:44:21.576465+0000","last_active":"2023-10-18T10:44:22.564020+0000","last_peered":"2023-10-18T10:44:22.564020+0000","last_clean":"2023-10-18T10:44:22.564020+0000","last_became_active":"2023-10-18T10:44:21.576113+0000","last_became_peered":"2023-10-18T10:44:21.576113+0000","last_unstale":"2023-10-18T10:44:22.564020+0000","last_undegraded":"2023-10-18T10:44:22.564020+0000","last_fullsized":"2023-10-18T10:44:22.564020+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:08.901111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564621+0000","last_change":"2023-10-18T10:44:21.573810+0000","last_active":"2023-10-18T10:44:22.564621+0000","last_peered":"2023-10-18T10:44:22.564621+0000","last_clean":"2023-10-18T10:44:22.564621+0000","last_became_active":"2023-10-18T10:44:21.571738+0000","last_became_peered":"2023-10-18T10:44:21.571738+0000","last_unstale":"2023-10-18T10:44:22.564621+0000","last_undegraded":"2023-10-18T10:44:22.564621+0000","last_fullsized":"2023-10-18T10:44:22.564621+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:25.282137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229144+0000","last_change":"2023-10-18T10:44:21.564526+0000","last_active":"2023-10-18T10:44:23.229144+0000","last_peered":"2023-10-18T10:44:23.229144+0000","last_clean":"2023-10-18T10:44:23.229144+0000","last_became_active":"2023-10-18T10:44:21.564253+0000","last_became_peered":"2023-10-18T10:44:21.564253+0000","last_unstale":"2023-10-18T10:44:23.229144+0000","last_undegraded":"2023-10-18T10:44:23.229144+0000","last_fullsized":"2023-10-18T10:44:23.229144+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:41.493559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229842+0000","last_change":"2023-10-18T10:44:21.566265+0000","last_active":"2023-10-18T10:44:23.229842+0000","last_peered":"2023-10-18T10:44:23.229842+0000","last_clean":"2023-10-18T10:44:23.229842+0000","last_became_active":"2023-10-18T10:44:21.566015+0000","last_became_peered":"2023-10-18T10:44:21.566015+0000","last_unstale":"2023-10-18T10:44:23.229842+0000","last_undegraded":"2023-10-18T10:44:23.229842+0000","last_fullsized":"2023-10-18T10:44:23.229842+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:45.613040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230026+0000","last_change":"2023-10-18T10:44:21.573779+0000","last_active":"2023-10-18T10:44:23.230026+0000","last_peered":"2023-10-18T10:44:23.230026+0000","last_clean":"2023-10-18T10:44:23.230026+0000","last_became_active":"2023-10-18T10:44:21.573438+0000","last_became_peered":"2023-10-18T10:44:21.573438+0000","last_unstale":"2023-10-18T10:44:23.230026+0000","last_undegraded":"2023-10-18T10:44:23.230026+0000","last_fullsized":"2023-10-18T10:44:23.230026+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:02.736471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229096+0000","last_change":"2023-10-18T10:44:21.560905+0000","last_active":"2023-10-18T10:44:23.229096+0000","last_peered":"2023-10-18T10:44:23.229096+0000","last_clean":"2023-10-18T10:44:23.229096+0000","last_became_active":"2023-10-18T10:44:21.560646+0000","last_became_peered":"2023-10-18T10:44:21.560646+0000","last_unstale":"2023-10-18T10:44:23.229096+0000","last_undegraded":"2023-10-18T10:44:23.229096+0000","last_fullsized":"2023-10-18T10:44:23.229096+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:33.058253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578375+0000","last_change":"2023-10-18T10:44:21.562213+0000","last_active":"2023-10-18T10:44:22.578375+0000","last_peered":"2023-10-18T10:44:22.578375+0000","last_clean":"2023-10-18T10:44:22.578375+0000","last_became_active":"2023-10-18T10:44:21.561968+0000","last_became_peered":"2023-10-18T10:44:21.561968+0000","last_unstale":"2023-10-18T10:44:22.578375+0000","last_undegraded":"2023-10-18T10:44:22.578375+0000","last_fullsized":"2023-10-18T10:44:22.578375+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:51:43.685746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.565002+0000","last_change":"2023-10-18T10:44:21.571801+0000","last_active":"2023-10-18T10:44:22.565002+0000","last_peered":"2023-10-18T10:44:22.565002+0000","last_clean":"2023-10-18T10:44:22.565002+0000","last_became_active":"2023-10-18T10:44:21.571493+0000","last_became_peered":"2023-10-18T10:44:21.571493+0000","last_unstale":"2023-10-18T10:44:22.565002+0000","last_undegraded":"2023-10-18T10:44:22.565002+0000","last_fullsized":"2023-10-18T10:44:22.565002+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:20.385907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564100+0000","last_change":"2023-10-18T10:44:21.576395+0000","last_active":"2023-10-18T10:44:22.564100+0000","last_peered":"2023-10-18T10:44:22.564100+0000","last_clean":"2023-10-18T10:44:22.564100+0000","last_became_active":"2023-10-18T10:44:21.576070+0000","last_became_peered":"2023-10-18T10:44:21.576070+0000","last_unstale":"2023-10-18T10:44:22.564100+0000","last_undegraded":"2023-10-18T10:44:22.564100+0000","last_fullsized":"2023-10-18T10:44:22.564100+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:43.691084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229598+0000","last_change":"2023-10-18T10:44:21.574452+0000","last_active":"2023-10-18T10:44:23.229598+0000","last_peered":"2023-10-18T10:44:23.229598+0000","last_clean":"2023-10-18T10:44:23.229598+0000","last_became_active":"2023-10-18T10:44:21.574176+0000","last_became_peered":"2023-10-18T10:44:21.574176+0000","last_unstale":"2023-10-18T10:44:23.229598+0000","last_undegraded":"2023-10-18T10:44:23.229598+0000","last_fullsized":"2023-10-18T10:44:23.229598+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:28.198131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"142'4","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229601+0000","last_change":"2023-10-18T10:44:20.554401+0000","last_active":"2023-10-18T10:44:23.229601+0000","last_peered":"2023-10-18T10:44:23.229601+0000","last_clean":"2023-10-18T10:44:23.229601+0000","last_became_active":"2023-10-18T10:44:20.554147+0000","last_became_peered":"2023-10-18T10:44:20.554147+0000","last_unstale":"2023-10-18T10:44:23.229601+0000","last_undegraded":"2023-10-18T10:44:23.229601+0000","last_fullsized":"2023-10-18T10:44:23.229601+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:40.125919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230036+0000","last_change":"2023-10-18T10:44:21.571376+0000","last_active":"2023-10-18T10:44:23.230036+0000","last_peered":"2023-10-18T10:44:23.230036+0000","last_clean":"2023-10-18T10:44:23.230036+0000","last_became_active":"2023-10-18T10:44:21.570991+0000","last_became_peered":"2023-10-18T10:44:21.570991+0000","last_unstale":"2023-10-18T10:44:23.230036+0000","last_undegraded":"2023-10-18T10:44:23.230036+0000","last_fullsized":"2023-10-18T10:44:23.230036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:51:03.439091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229510+0000","last_change":"2023-10-18T10:44:20.549263+0000","last_active":"2023-10-18T10:44:23.229510+0000","last_peered":"2023-10-18T10:44:23.229510+0000","last_clean":"2023-10-18T10:44:23.229510+0000","last_became_active":"2023-10-18T10:44:20.549001+0000","last_became_peered":"2023-10-18T10:44:20.549001+0000","last_unstale":"2023-10-18T10:44:23.229510+0000","last_undegraded":"2023-10-18T10:44:23.229510+0000","last_fullsized":"2023-10-18T10:44:23.229510+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:27.133011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564566+0000","last_change":"2023-10-18T10:44:21.576865+0000","last_active":"2023-10-18T10:44:22.564566+0000","last_peered":"2023-10-18T10:44:22.564566+0000","last_clean":"2023-10-18T10:44:22.564566+0000","last_became_active":"2023-10-18T10:44:21.576238+0000","last_became_peered":"2023-10-18T10:44:21.576238+0000","last_unstale":"2023-10-18T10:44:22.564566+0000","last_undegraded":"2023-10-18T10:44:22.564566+0000","last_fullsized":"2023-10-18T10:44:22.564566+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:22:49.158268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230246+0000","last_change":"2023-10-18T10:44:20.552971+0000","last_active":"2023-10-18T10:44:23.230246+0000","last_peered":"2023-10-18T10:44:23.230246+0000","last_clean":"2023-10-18T10:44:23.230246+0000","last_became_active":"2023-10-18T10:44:20.551870+0000","last_became_peered":"2023-10-18T10:44:20.551870+0000","last_unstale":"2023-10-18T10:44:23.230246+0000","last_undegraded":"2023-10-18T10:44:23.230246+0000","last_fullsized":"2023-10-18T10:44:23.230246+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:24.277243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230436+0000","last_change":"2023-10-18T10:44:21.563435+0000","last_active":"2023-10-18T10:44:23.230436+0000","last_peered":"2023-10-18T10:44:23.230436+0000","last_clean":"2023-10-18T10:44:23.230436+0000","last_became_active":"2023-10-18T10:44:21.563155+0000","last_became_peered":"2023-10-18T10:44:21.563155+0000","last_unstale":"2023-10-18T10:44:23.230436+0000","last_undegraded":"2023-10-18T10:44:23.230436+0000","last_fullsized":"2023-10-18T10:44:23.230436+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:15.656446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229620+0000","last_change":"2023-10-18T10:44:20.554735+0000","last_active":"2023-10-18T10:44:23.229620+0000","last_peered":"2023-10-18T10:44:23.229620+0000","last_clean":"2023-10-18T10:44:23.229620+0000","last_became_active":"2023-10-18T10:44:20.554456+0000","last_became_peered":"2023-10-18T10:44:20.554456+0000","last_unstale":"2023-10-18T10:44:23.229620+0000","last_undegraded":"2023-10-18T10:44:23.229620+0000","last_fullsized":"2023-10-18T10:44:23.229620+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:55.061568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229349+0000","last_change":"2023-10-18T10:44:21.564071+0000","last_active":"2023-10-18T10:44:23.229349+0000","last_peered":"2023-10-18T10:44:23.229349+0000","last_clean":"2023-10-18T10:44:23.229349+0000","last_became_active":"2023-10-18T10:44:21.563810+0000","last_became_peered":"2023-10-18T10:44:21.563810+0000","last_unstale":"2023-10-18T10:44:23.229349+0000","last_undegraded":"2023-10-18T10:44:23.229349+0000","last_fullsized":"2023-10-18T10:44:23.229349+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:32.562901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564409+0000","last_change":"2023-10-18T10:44:20.558619+0000","last_active":"2023-10-18T10:44:22.564409+0000","last_peered":"2023-10-18T10:44:22.564409+0000","last_clean":"2023-10-18T10:44:22.564409+0000","last_became_active":"2023-10-18T10:44:20.557889+0000","last_became_peered":"2023-10-18T10:44:20.557889+0000","last_unstale":"2023-10-18T10:44:22.564409+0000","last_undegraded":"2023-10-18T10:44:22.564409+0000","last_fullsized":"2023-10-18T10:44:22.564409+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:47.550038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564220+0000","last_change":"2023-10-18T10:44:21.568245+0000","last_active":"2023-10-18T10:44:22.564220+0000","last_peered":"2023-10-18T10:44:22.564220+0000","last_clean":"2023-10-18T10:44:22.564220+0000","last_became_active":"2023-10-18T10:44:21.567982+0000","last_became_peered":"2023-10-18T10:44:21.567982+0000","last_unstale":"2023-10-18T10:44:22.564220+0000","last_undegraded":"2023-10-18T10:44:22.564220+0000","last_fullsized":"2023-10-18T10:44:22.564220+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:07.009131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229791+0000","last_change":"2023-10-18T10:44:20.556161+0000","last_active":"2023-10-18T10:44:23.229791+0000","last_peered":"2023-10-18T10:44:23.229791+0000","last_clean":"2023-10-18T10:44:23.229791+0000","last_became_active":"2023-10-18T10:44:20.555910+0000","last_became_peered":"2023-10-18T10:44:20.555910+0000","last_unstale":"2023-10-18T10:44:23.229791+0000","last_undegraded":"2023-10-18T10:44:23.229791+0000","last_fullsized":"2023-10-18T10:44:23.229791+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:08.315986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578409+0000","last_change":"2023-10-18T10:44:21.559359+0000","last_active":"2023-10-18T10:44:22.578409+0000","last_peered":"2023-10-18T10:44:22.578409+0000","last_clean":"2023-10-18T10:44:22.578409+0000","last_became_active":"2023-10-18T10:44:21.559109+0000","last_became_peered":"2023-10-18T10:44:21.559109+0000","last_unstale":"2023-10-18T10:44:22.578409+0000","last_undegraded":"2023-10-18T10:44:22.578409+0000","last_fullsized":"2023-10-18T10:44:22.578409+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:20.603381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564906+0000","last_change":"2023-10-18T10:44:20.555390+0000","last_active":"2023-10-18T10:44:22.564906+0000","last_peered":"2023-10-18T10:44:22.564906+0000","last_clean":"2023-10-18T10:44:22.564906+0000","last_became_active":"2023-10-18T10:44:20.555150+0000","last_became_peered":"2023-10-18T10:44:20.555150+0000","last_unstale":"2023-10-18T10:44:22.564906+0000","last_undegraded":"2023-10-18T10:44:22.564906+0000","last_fullsized":"2023-10-18T10:44:22.564906+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:35:41.665405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229705+0000","last_change":"2023-10-18T10:44:21.567559+0000","last_active":"2023-10-18T10:44:23.229705+0000","last_peered":"2023-10-18T10:44:23.229705+0000","last_clean":"2023-10-18T10:44:23.229705+0000","last_became_active":"2023-10-18T10:44:21.567301+0000","last_became_peered":"2023-10-18T10:44:21.567301+0000","last_unstale":"2023-10-18T10:44:23.229705+0000","last_undegraded":"2023-10-18T10:44:23.229705+0000","last_fullsized":"2023-10-18T10:44:23.229705+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:04.527071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229782+0000","last_change":"2023-10-18T10:44:20.561064+0000","last_active":"2023-10-18T10:44:23.229782+0000","last_peered":"2023-10-18T10:44:23.229782+0000","last_clean":"2023-10-18T10:44:23.229782+0000","last_became_active":"2023-10-18T10:44:20.560807+0000","last_became_peered":"2023-10-18T10:44:20.560807+0000","last_unstale":"2023-10-18T10:44:23.229782+0000","last_undegraded":"2023-10-18T10:44:23.229782+0000","last_fullsized":"2023-10-18T10:44:23.229782+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:24:47.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229953+0000","last_change":"2023-10-18T10:44:21.560982+0000","last_active":"2023-10-18T10:44:23.229953+0000","last_peered":"2023-10-18T10:44:23.229953+0000","last_clean":"2023-10-18T10:44:23.229953+0000","last_became_active":"2023-10-18T10:44:21.560705+0000","last_became_peered":"2023-10-18T10:44:21.560705+0000","last_unstale":"2023-10-18T10:44:23.229953+0000","last_undegraded":"2023-10-18T10:44:23.229953+0000","last_fullsized":"2023-10-18T10:44:23.229953+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:07.009131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229691+0000","last_change":"2023-10-18T10:44:20.558136+0000","last_active":"2023-10-18T10:44:23.229691+0000","last_peered":"2023-10-18T10:44:23.229691+0000","last_clean":"2023-10-18T10:44:23.229691+0000","last_became_active":"2023-10-18T10:44:20.557868+0000","last_became_peered":"2023-10-18T10:44:20.557868+0000","last_unstale":"2023-10-18T10:44:23.229691+0000","last_undegraded":"2023-10-18T10:44:23.229691+0000","last_fullsized":"2023-10-18T10:44:23.229691+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:13.933951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564849+0000","last_change":"2023-10-18T10:44:21.572721+0000","last_active":"2023-10-18T10:44:22.564849+0000","last_peered":"2023-10-18T10:44:22.564849+0000","last_clean":"2023-10-18T10:44:22.564849+0000","last_became_active":"2023-10-18T10:44:21.572452+0000","last_became_peered":"2023-10-18T10:44:21.572452+0000","last_unstale":"2023-10-18T10:44:22.564849+0000","last_undegraded":"2023-10-18T10:44:22.564849+0000","last_fullsized":"2023-10-18T10:44:22.564849+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:11.608888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564656+0000","last_change":"2023-10-18T10:44:20.554698+0000","last_active":"2023-10-18T10:44:22.564656+0000","last_peered":"2023-10-18T10:44:22.564656+0000","last_clean":"2023-10-18T10:44:22.564656+0000","last_became_active":"2023-10-18T10:44:20.554438+0000","last_became_peered":"2023-10-18T10:44:20.554438+0000","last_unstale":"2023-10-18T10:44:22.564656+0000","last_undegraded":"2023-10-18T10:44:22.564656+0000","last_fullsized":"2023-10-18T10:44:22.564656+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:46.314630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578684+0000","last_change":"2023-10-18T10:44:21.561055+0000","last_active":"2023-10-18T10:44:22.578684+0000","last_peered":"2023-10-18T10:44:22.578684+0000","last_clean":"2023-10-18T10:44:22.578684+0000","last_became_active":"2023-10-18T10:44:21.560278+0000","last_became_peered":"2023-10-18T10:44:21.560278+0000","last_unstale":"2023-10-18T10:44:22.578684+0000","last_undegraded":"2023-10-18T10:44:22.578684+0000","last_fullsized":"2023-10-18T10:44:22.578684+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:53:01.261584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"142'3","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564965+0000","last_change":"2023-10-18T10:44:20.553723+0000","last_active":"2023-10-18T10:44:22.564965+0000","last_peered":"2023-10-18T10:44:22.564965+0000","last_clean":"2023-10-18T10:44:22.564965+0000","last_became_active":"2023-10-18T10:44:20.553398+0000","last_became_peered":"2023-10-18T10:44:20.553398+0000","last_unstale":"2023-10-18T10:44:22.564965+0000","last_undegraded":"2023-10-18T10:44:22.564965+0000","last_fullsized":"2023-10-18T10:44:22.564965+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:11.270346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564776+0000","last_change":"2023-10-18T10:44:21.573464+0000","last_active":"2023-10-18T10:44:22.564776+0000","last_peered":"2023-10-18T10:44:22.564776+0000","last_clean":"2023-10-18T10:44:22.564776+0000","last_became_active":"2023-10-18T10:44:21.572310+0000","last_became_peered":"2023-10-18T10:44:21.572310+0000","last_unstale":"2023-10-18T10:44:22.564776+0000","last_undegraded":"2023-10-18T10:44:22.564776+0000","last_fullsized":"2023-10-18T10:44:22.564776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:11.670089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"142'6","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578498+0000","last_change":"2023-10-18T10:44:20.560661+0000","last_active":"2023-10-18T10:44:22.578498+0000","last_peered":"2023-10-18T10:44:22.578498+0000","last_clean":"2023-10-18T10:44:22.578498+0000","last_became_active":"2023-10-18T10:44:20.560167+0000","last_became_peered":"2023-10-18T10:44:20.560167+0000","last_unstale":"2023-10-18T10:44:22.578498+0000","last_undegraded":"2023-10-18T10:44:22.578498+0000","last_fullsized":"2023-10-18T10:44:22.578498+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:42.686189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229978+0000","last_change":"2023-10-18T10:44:21.558514+0000","last_active":"2023-10-18T10:44:23.229978+0000","last_peered":"2023-10-18T10:44:23.229978+0000","last_clean":"2023-10-18T10:44:23.229978+0000","last_became_active":"2023-10-18T10:44:21.558237+0000","last_became_peered":"2023-10-18T10:44:21.558237+0000","last_unstale":"2023-10-18T10:44:23.229978+0000","last_undegraded":"2023-10-18T10:44:23.229978+0000","last_fullsized":"2023-10-18T10:44:23.229978+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:46.001986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578398+0000","last_change":"2023-10-18T10:44:20.552670+0000","last_active":"2023-10-18T10:44:22.578398+0000","last_peered":"2023-10-18T10:44:22.578398+0000","last_clean":"2023-10-18T10:44:22.578398+0000","last_became_active":"2023-10-18T10:44:20.552408+0000","last_became_peered":"2023-10-18T10:44:20.552408+0000","last_unstale":"2023-10-18T10:44:22.578398+0000","last_undegraded":"2023-10-18T10:44:22.578398+0000","last_fullsized":"2023-10-18T10:44:22.578398+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:10.665194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229402+0000","last_change":"2023-10-18T10:44:21.566860+0000","last_active":"2023-10-18T10:44:23.229402+0000","last_peered":"2023-10-18T10:44:23.229402+0000","last_clean":"2023-10-18T10:44:23.229402+0000","last_became_active":"2023-10-18T10:44:21.566602+0000","last_became_peered":"2023-10-18T10:44:21.566602+0000","last_unstale":"2023-10-18T10:44:23.229402+0000","last_undegraded":"2023-10-18T10:44:23.229402+0000","last_fullsized":"2023-10-18T10:44:23.229402+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:29.254273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578516+0000","last_change":"2023-10-18T10:44:20.557841+0000","last_active":"2023-10-18T10:44:22.578516+0000","last_peered":"2023-10-18T10:44:22.578516+0000","last_clean":"2023-10-18T10:44:22.578516+0000","last_became_active":"2023-10-18T10:44:20.557579+0000","last_became_peered":"2023-10-18T10:44:20.557579+0000","last_unstale":"2023-10-18T10:44:22.578516+0000","last_undegraded":"2023-10-18T10:44:22.578516+0000","last_fullsized":"2023-10-18T10:44:22.578516+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:27.040709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.228997+0000","last_change":"2023-10-18T10:44:21.561345+0000","last_active":"2023-10-18T10:44:23.228997+0000","last_peered":"2023-10-18T10:44:23.228997+0000","last_clean":"2023-10-18T10:44:23.228997+0000","last_became_active":"2023-10-18T10:44:21.561090+0000","last_became_peered":"2023-10-18T10:44:21.561090+0000","last_unstale":"2023-10-18T10:44:23.228997+0000","last_undegraded":"2023-10-18T10:44:23.228997+0000","last_fullsized":"2023-10-18T10:44:23.228997+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:53.624058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578287+0000","last_change":"2023-10-18T10:44:20.560994+0000","last_active":"2023-10-18T10:44:22.578287+0000","last_peered":"2023-10-18T10:44:22.578287+0000","last_clean":"2023-10-18T10:44:22.578287+0000","last_became_active":"2023-10-18T10:44:20.560402+0000","last_became_peered":"2023-10-18T10:44:20.560402+0000","last_unstale":"2023-10-18T10:44:22.578287+0000","last_undegraded":"2023-10-18T10:44:22.578287+0000","last_fullsized":"2023-10-18T10:44:22.578287+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:36.301690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564425+0000","last_change":"2023-10-18T10:44:21.573842+0000","last_active":"2023-10-18T10:44:22.564425+0000","last_peered":"2023-10-18T10:44:22.564425+0000","last_clean":"2023-10-18T10:44:22.564425+0000","last_became_active":"2023-10-18T10:44:21.573572+0000","last_became_peered":"2023-10-18T10:44:21.573572+0000","last_unstale":"2023-10-18T10:44:22.564425+0000","last_undegraded":"2023-10-18T10:44:22.564425+0000","last_fullsized":"2023-10-18T10:44:22.564425+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:36.142060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229650+0000","last_change":"2023-10-18T10:44:20.558709+0000","last_active":"2023-10-18T10:44:23.229650+0000","last_peered":"2023-10-18T10:44:23.229650+0000","last_clean":"2023-10-18T10:44:23.229650+0000","last_became_active":"2023-10-18T10:44:20.558454+0000","last_became_peered":"2023-10-18T10:44:20.558454+0000","last_unstale":"2023-10-18T10:44:23.229650+0000","last_undegraded":"2023-10-18T10:44:23.229650+0000","last_fullsized":"2023-10-18T10:44:23.229650+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:35:25.000743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564329+0000","last_change":"2023-10-18T10:44:21.571099+0000","last_active":"2023-10-18T10:44:22.564329+0000","last_peered":"2023-10-18T10:44:22.564329+0000","last_clean":"2023-10-18T10:44:22.564329+0000","last_became_active":"2023-10-18T10:44:21.570843+0000","last_became_peered":"2023-10-18T10:44:21.570843+0000","last_unstale":"2023-10-18T10:44:22.564329+0000","last_undegraded":"2023-10-18T10:44:22.564329+0000","last_fullsized":"2023-10-18T10:44:22.564329+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:36.531300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229275+0000","last_change":"2023-10-18T10:44:20.557537+0000","last_active":"2023-10-18T10:44:23.229275+0000","last_peered":"2023-10-18T10:44:23.229275+0000","last_clean":"2023-10-18T10:44:23.229275+0000","last_became_active":"2023-10-18T10:44:20.557279+0000","last_became_peered":"2023-10-18T10:44:20.557279+0000","last_unstale":"2023-10-18T10:44:23.229275+0000","last_undegraded":"2023-10-18T10:44:23.229275+0000","last_fullsized":"2023-10-18T10:44:23.229275+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:17.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230128+0000","last_change":"2023-10-18T10:44:21.569266+0000","last_active":"2023-10-18T10:44:23.230128+0000","last_peered":"2023-10-18T10:44:23.230128+0000","last_clean":"2023-10-18T10:44:23.230128+0000","last_became_active":"2023-10-18T10:44:21.568991+0000","last_became_peered":"2023-10-18T10:44:21.568991+0000","last_unstale":"2023-10-18T10:44:23.230128+0000","last_undegraded":"2023-10-18T10:44:23.230128+0000","last_fullsized":"2023-10-18T10:44:23.230128+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:14.757285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230308+0000","last_change":"2023-10-18T10:44:20.556381+0000","last_active":"2023-10-18T10:44:23.230308+0000","last_peered":"2023-10-18T10:44:23.230308+0000","last_clean":"2023-10-18T10:44:23.230308+0000","last_became_active":"2023-10-18T10:44:20.555808+0000","last_became_peered":"2023-10-18T10:44:20.555808+0000","last_unstale":"2023-10-18T10:44:23.230308+0000","last_undegraded":"2023-10-18T10:44:23.230308+0000","last_fullsized":"2023-10-18T10:44:23.230308+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:10.665194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564870+0000","last_change":"2023-10-18T10:44:21.572780+0000","last_active":"2023-10-18T10:44:22.564870+0000","last_peered":"2023-10-18T10:44:22.564870+0000","last_clean":"2023-10-18T10:44:22.564870+0000","last_became_active":"2023-10-18T10:44:21.571305+0000","last_became_peered":"2023-10-18T10:44:21.571305+0000","last_unstale":"2023-10-18T10:44:22.564870+0000","last_undegraded":"2023-10-18T10:44:22.564870+0000","last_fullsized":"2023-10-18T10:44:22.564870+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:37.306584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230003+0000","last_change":"2023-10-18T10:44:20.560300+0000","last_active":"2023-10-18T10:44:23.230003+0000","last_peered":"2023-10-18T10:44:23.230003+0000","last_clean":"2023-10-18T10:44:23.230003+0000","last_became_active":"2023-10-18T10:44:20.560024+0000","last_became_peered":"2023-10-18T10:44:20.560024+0000","last_unstale":"2023-10-18T10:44:23.230003+0000","last_undegraded":"2023-10-18T10:44:23.230003+0000","last_fullsized":"2023-10-18T10:44:23.230003+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:54:41.781127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229707+0000","last_change":"2023-10-18T10:44:21.569228+0000","last_active":"2023-10-18T10:44:23.229707+0000","last_peered":"2023-10-18T10:44:23.229707+0000","last_clean":"2023-10-18T10:44:23.229707+0000","last_became_active":"2023-10-18T10:44:21.568808+0000","last_became_peered":"2023-10-18T10:44:21.568808+0000","last_unstale":"2023-10-18T10:44:23.229707+0000","last_undegraded":"2023-10-18T10:44:23.229707+0000","last_fullsized":"2023-10-18T10:44:23.229707+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:57:44.839359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229830+0000","last_change":"2023-10-18T10:44:20.560128+0000","last_active":"2023-10-18T10:44:23.229830+0000","last_peered":"2023-10-18T10:44:23.229830+0000","last_clean":"2023-10-18T10:44:23.229830+0000","last_became_active":"2023-10-18T10:44:20.559860+0000","last_became_peered":"2023-10-18T10:44:20.559860+0000","last_unstale":"2023-10-18T10:44:23.229830+0000","last_undegraded":"2023-10-18T10:44:23.229830+0000","last_fullsized":"2023-10-18T10:44:23.229830+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:22.637653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578197+0000","last_change":"2023-10-18T10:44:21.577386+0000","last_active":"2023-10-18T10:44:22.578197+0000","last_peered":"2023-10-18T10:44:22.578197+0000","last_clean":"2023-10-18T10:44:22.578197+0000","last_became_active":"2023-10-18T10:44:21.577122+0000","last_became_peered":"2023-10-18T10:44:21.577122+0000","last_unstale":"2023-10-18T10:44:22.578197+0000","last_undegraded":"2023-10-18T10:44:22.578197+0000","last_fullsized":"2023-10-18T10:44:22.578197+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:37.968875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229944+0000","last_change":"2023-10-18T10:44:20.557928+0000","last_active":"2023-10-18T10:44:23.229944+0000","last_peered":"2023-10-18T10:44:23.229944+0000","last_clean":"2023-10-18T10:44:23.229944+0000","last_became_active":"2023-10-18T10:44:20.557670+0000","last_became_peered":"2023-10-18T10:44:20.557670+0000","last_unstale":"2023-10-18T10:44:23.229944+0000","last_undegraded":"2023-10-18T10:44:23.229944+0000","last_fullsized":"2023-10-18T10:44:23.229944+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:19.381013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578543+0000","last_change":"2023-10-18T10:44:21.566860+0000","last_active":"2023-10-18T10:44:22.578543+0000","last_peered":"2023-10-18T10:44:22.578543+0000","last_clean":"2023-10-18T10:44:22.578543+0000","last_became_active":"2023-10-18T10:44:21.566409+0000","last_became_peered":"2023-10-18T10:44:21.566409+0000","last_unstale":"2023-10-18T10:44:22.578543+0000","last_undegraded":"2023-10-18T10:44:22.578543+0000","last_fullsized":"2023-10-18T10:44:22.578543+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:12.324740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578307+0000","last_change":"2023-10-18T10:44:20.558099+0000","last_active":"2023-10-18T10:44:22.578307+0000","last_peered":"2023-10-18T10:44:22.578307+0000","last_clean":"2023-10-18T10:44:22.578307+0000","last_became_active":"2023-10-18T10:44:20.557837+0000","last_became_peered":"2023-10-18T10:44:20.557837+0000","last_unstale":"2023-10-18T10:44:22.578307+0000","last_undegraded":"2023-10-18T10:44:22.578307+0000","last_fullsized":"2023-10-18T10:44:22.578307+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:56.009593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564355+0000","last_change":"2023-10-18T10:44:21.572333+0000","last_active":"2023-10-18T10:44:22.564355+0000","last_peered":"2023-10-18T10:44:22.564355+0000","last_clean":"2023-10-18T10:44:22.564355+0000","last_became_active":"2023-10-18T10:44:21.572035+0000","last_became_peered":"2023-10-18T10:44:21.572035+0000","last_unstale":"2023-10-18T10:44:22.564355+0000","last_undegraded":"2023-10-18T10:44:22.564355+0000","last_fullsized":"2023-10-18T10:44:22.564355+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:17.682666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229348+0000","last_change":"2023-10-18T10:44:20.555014+0000","last_active":"2023-10-18T10:44:23.229348+0000","last_peered":"2023-10-18T10:44:23.229348+0000","last_clean":"2023-10-18T10:44:23.229348+0000","last_became_active":"2023-10-18T10:44:20.554187+0000","last_became_peered":"2023-10-18T10:44:20.554187+0000","last_unstale":"2023-10-18T10:44:23.229348+0000","last_undegraded":"2023-10-18T10:44:23.229348+0000","last_fullsized":"2023-10-18T10:44:23.229348+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:07.896216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230157+0000","last_change":"2023-10-18T10:44:21.573151+0000","last_active":"2023-10-18T10:44:23.230157+0000","last_peered":"2023-10-18T10:44:23.230157+0000","last_clean":"2023-10-18T10:44:23.230157+0000","last_became_active":"2023-10-18T10:44:21.572419+0000","last_became_peered":"2023-10-18T10:44:21.572419+0000","last_unstale":"2023-10-18T10:44:23.230157+0000","last_undegraded":"2023-10-18T10:44:23.230157+0000","last_fullsized":"2023-10-18T10:44:23.230157+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:51:53.373910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230113+0000","last_change":"2023-10-18T10:44:20.560435+0000","last_active":"2023-10-18T10:44:23.230113+0000","last_peered":"2023-10-18T10:44:23.230113+0000","last_clean":"2023-10-18T10:44:23.230113+0000","last_became_active":"2023-10-18T10:44:20.560180+0000","last_became_peered":"2023-10-18T10:44:20.560180+0000","last_unstale":"2023-10-18T10:44:23.230113+0000","last_undegraded":"2023-10-18T10:44:23.230113+0000","last_fullsized":"2023-10-18T10:44:23.230113+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:02.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229141+0000","last_change":"2023-10-18T10:44:21.563539+0000","last_active":"2023-10-18T10:44:23.229141+0000","last_peered":"2023-10-18T10:44:23.229141+0000","last_clean":"2023-10-18T10:44:23.229141+0000","last_became_active":"2023-10-18T10:44:21.563281+0000","last_became_peered":"2023-10-18T10:44:21.563281+0000","last_unstale":"2023-10-18T10:44:23.229141+0000","last_undegraded":"2023-10-18T10:44:23.229141+0000","last_fullsized":"2023-10-18T10:44:23.229141+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:29:42.989075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229801+0000","last_change":"2023-10-18T10:44:20.558728+0000","last_active":"2023-10-18T10:44:23.229801+0000","last_peered":"2023-10-18T10:44:23.229801+0000","last_clean":"2023-10-18T10:44:23.229801+0000","last_became_active":"2023-10-18T10:44:20.558466+0000","last_became_peered":"2023-10-18T10:44:20.558466+0000","last_unstale":"2023-10-18T10:44:23.229801+0000","last_undegraded":"2023-10-18T10:44:23.229801+0000","last_fullsized":"2023-10-18T10:44:23.229801+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:15.327159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230308+0000","last_change":"2023-10-18T10:44:21.565365+0000","last_active":"2023-10-18T10:44:23.230308+0000","last_peered":"2023-10-18T10:44:23.230308+0000","last_clean":"2023-10-18T10:44:23.230308+0000","last_became_active":"2023-10-18T10:44:21.565087+0000","last_became_peered":"2023-10-18T10:44:21.565087+0000","last_unstale":"2023-10-18T10:44:23.230308+0000","last_undegraded":"2023-10-18T10:44:23.230308+0000","last_fullsized":"2023-10-18T10:44:23.230308+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:37.267378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229858+0000","last_change":"2023-10-18T10:44:20.556980+0000","last_active":"2023-10-18T10:44:23.229858+0000","last_peered":"2023-10-18T10:44:23.229858+0000","last_clean":"2023-10-18T10:44:23.229858+0000","last_became_active":"2023-10-18T10:44:20.556696+0000","last_became_peered":"2023-10-18T10:44:20.556696+0000","last_unstale":"2023-10-18T10:44:23.229858+0000","last_undegraded":"2023-10-18T10:44:23.229858+0000","last_fullsized":"2023-10-18T10:44:23.229858+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:37:25.602393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229211+0000","last_change":"2023-10-18T10:44:21.572221+0000","last_active":"2023-10-18T10:44:23.229211+0000","last_peered":"2023-10-18T10:44:23.229211+0000","last_clean":"2023-10-18T10:44:23.229211+0000","last_became_active":"2023-10-18T10:44:21.571952+0000","last_became_peered":"2023-10-18T10:44:21.571952+0000","last_unstale":"2023-10-18T10:44:23.229211+0000","last_undegraded":"2023-10-18T10:44:23.229211+0000","last_fullsized":"2023-10-18T10:44:23.229211+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:14:14.754025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230218+0000","last_change":"2023-10-18T10:44:20.562360+0000","last_active":"2023-10-18T10:44:23.230218+0000","last_peered":"2023-10-18T10:44:23.230218+0000","last_clean":"2023-10-18T10:44:23.230218+0000","last_became_active":"2023-10-18T10:44:20.562101+0000","last_became_peered":"2023-10-18T10:44:20.562101+0000","last_unstale":"2023-10-18T10:44:23.230218+0000","last_undegraded":"2023-10-18T10:44:23.230218+0000","last_fullsized":"2023-10-18T10:44:23.230218+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:54.222737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229767+0000","last_change":"2023-10-18T10:44:21.569070+0000","last_active":"2023-10-18T10:44:23.229767+0000","last_peered":"2023-10-18T10:44:23.229767+0000","last_clean":"2023-10-18T10:44:23.229767+0000","last_became_active":"2023-10-18T10:44:21.568704+0000","last_became_peered":"2023-10-18T10:44:21.568704+0000","last_unstale":"2023-10-18T10:44:23.229767+0000","last_undegraded":"2023-10-18T10:44:23.229767+0000","last_fullsized":"2023-10-18T10:44:23.229767+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:06.144512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564507+0000","last_change":"2023-10-18T10:44:20.559585+0000","last_active":"2023-10-18T10:44:22.564507+0000","last_peered":"2023-10-18T10:44:22.564507+0000","last_clean":"2023-10-18T10:44:22.564507+0000","last_became_active":"2023-10-18T10:44:20.559293+0000","last_became_peered":"2023-10-18T10:44:20.559293+0000","last_unstale":"2023-10-18T10:44:22.564507+0000","last_undegraded":"2023-10-18T10:44:22.564507+0000","last_fullsized":"2023-10-18T10:44:22.564507+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:21.797094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578233+0000","last_change":"2023-10-18T10:44:21.567212+0000","last_active":"2023-10-18T10:44:22.578233+0000","last_peered":"2023-10-18T10:44:22.578233+0000","last_clean":"2023-10-18T10:44:22.578233+0000","last_became_active":"2023-10-18T10:44:21.566945+0000","last_became_peered":"2023-10-18T10:44:21.566945+0000","last_unstale":"2023-10-18T10:44:22.578233+0000","last_undegraded":"2023-10-18T10:44:22.578233+0000","last_fullsized":"2023-10-18T10:44:22.578233+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:01.830388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564631+0000","last_change":"2023-10-18T10:44:20.560139+0000","last_active":"2023-10-18T10:44:22.564631+0000","last_peered":"2023-10-18T10:44:22.564631+0000","last_clean":"2023-10-18T10:44:22.564631+0000","last_became_active":"2023-10-18T10:44:20.559766+0000","last_became_peered":"2023-10-18T10:44:20.559766+0000","last_unstale":"2023-10-18T10:44:22.564631+0000","last_undegraded":"2023-10-18T10:44:22.564631+0000","last_fullsized":"2023-10-18T10:44:22.564631+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:37:25.602393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229550+0000","last_change":"2023-10-18T10:44:21.570144+0000","last_active":"2023-10-18T10:44:23.229550+0000","last_peered":"2023-10-18T10:44:23.229550+0000","last_clean":"2023-10-18T10:44:23.229550+0000","last_became_active":"2023-10-18T10:44:21.569829+0000","last_became_peered":"2023-10-18T10:44:21.569829+0000","last_unstale":"2023-10-18T10:44:23.229550+0000","last_undegraded":"2023-10-18T10:44:23.229550+0000","last_fullsized":"2023-10-18T10:44:23.229550+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:53.278742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229571+0000","last_change":"2023-10-18T10:44:20.561863+0000","last_active":"2023-10-18T10:44:23.229571+0000","last_peered":"2023-10-18T10:44:23.229571+0000","last_clean":"2023-10-18T10:44:23.229571+0000","last_became_active":"2023-10-18T10:44:20.561608+0000","last_became_peered":"2023-10-18T10:44:20.561608+0000","last_unstale":"2023-10-18T10:44:23.229571+0000","last_undegraded":"2023-10-18T10:44:23.229571+0000","last_fullsized":"2023-10-18T10:44:23.229571+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:46.320265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229764+0000","last_change":"2023-10-18T10:44:21.566242+0000","last_active":"2023-10-18T10:44:23.229764+0000","last_peered":"2023-10-18T10:44:23.229764+0000","last_clean":"2023-10-18T10:44:23.229764+0000","last_became_active":"2023-10-18T10:44:21.566000+0000","last_became_peered":"2023-10-18T10:44:21.566000+0000","last_unstale":"2023-10-18T10:44:23.229764+0000","last_undegraded":"2023-10-18T10:44:23.229764+0000","last_fullsized":"2023-10-18T10:44:23.229764+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:30.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229345+0000","last_change":"2023-10-18T10:44:20.550433+0000","last_active":"2023-10-18T10:44:23.229345+0000","last_peered":"2023-10-18T10:44:23.229345+0000","last_clean":"2023-10-18T10:44:23.229345+0000","last_became_active":"2023-10-18T10:44:20.550162+0000","last_became_peered":"2023-10-18T10:44:20.550162+0000","last_unstale":"2023-10-18T10:44:23.229345+0000","last_undegraded":"2023-10-18T10:44:23.229345+0000","last_fullsized":"2023-10-18T10:44:23.229345+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:16.677771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230436+0000","last_change":"2023-10-18T10:44:21.564597+0000","last_active":"2023-10-18T10:44:23.230436+0000","last_peered":"2023-10-18T10:44:23.230436+0000","last_clean":"2023-10-18T10:44:23.230436+0000","last_became_active":"2023-10-18T10:44:21.564330+0000","last_became_peered":"2023-10-18T10:44:21.564330+0000","last_unstale":"2023-10-18T10:44:23.230436+0000","last_undegraded":"2023-10-18T10:44:23.230436+0000","last_fullsized":"2023-10-18T10:44:23.230436+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:51:43.685746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"142'8","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578191+0000","last_change":"2023-10-18T10:44:20.552906+0000","last_active":"2023-10-18T10:44:22.578191+0000","last_peered":"2023-10-18T10:44:22.578191+0000","last_clean":"2023-10-18T10:44:22.578191+0000","last_became_active":"2023-10-18T10:44:20.552648+0000","last_became_peered":"2023-10-18T10:44:20.552648+0000","last_unstale":"2023-10-18T10:44:22.578191+0000","last_undegraded":"2023-10-18T10:44:22.578191+0000","last_fullsized":"2023-10-18T10:44:22.578191+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:46.314630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"131'236","reported_seq":701,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T10:44:21.553098+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:44:21.553098+0000","last_peered":"2023-10-18T10:44:21.553098+0000","last_clean":"2023-10-18T10:44:21.553098+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:44:21.553098+0000","last_undegraded":"2023-10-18T10:44:21.553098+0000","last_fullsized":"2023-10-18T10:44:21.553098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575082,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2375680,"data_stored":1482697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.217,"15min":0.217},"max":{"1min":0.77900000000000003,"5min":0.876,"15min":0.88300000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.434},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.90700000000000003},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77900000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.505},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60099999999999998},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63400000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.86199999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575082,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1769472,"data_stored":892135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.502},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72099999999999997,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.74199999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.748,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.53200000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.75700000000000001,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.54200000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.66200000000000003,"5min":0.872,"15min":0.872},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.51800000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40500000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.87},"last":0.57199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575082,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50976,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320864,"statfs":{"total":96636764160,"available":96584564736,"internally_reserved":0,"allocated":1785856,"data_stored":898361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.38800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.81299999999999994}]},{"osd":1,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.93600000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.66500000000000004,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.52300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.84199999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.75600000000000001,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.69999999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.72299999999999998,"15min":0.80100000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.83299999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.83099999999999996}]},{"osd":7,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.31},"max":{"1min":0.71099999999999997,"5min":0.84299999999999997,"15min":0.89300000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.85299999999999998,"15min":0.871},"last":0.85899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575082,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37092,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334748,"statfs":{"total":96636764160,"available":96598781952,"internally_reserved":0,"allocated":1789952,"data_stored":899813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.76200000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.877},"last":0.57699999999999996}]},{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83199999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.89300000000000002},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81999999999999995,"15min":0.85899999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.46500000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.873,"15min":0.873},"last":0.629}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74199999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.505,"15min":0.498},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35999999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":1.446},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":1.0469999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":0.874,"5min":0.88600000000000001,"15min":1.0880000000000001},"last":0.64200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575082,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":899847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.36499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.62},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.59299999999999997},"last":0.29499999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20799999999999999},"max":{"1min":0.504,"5min":0.624,"15min":0.69099999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.59299999999999997,"15min":0.67000000000000004},"last":0.318}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.98299999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.59599999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86599999999999999,"15min":1.002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.88100000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.53400000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38,"15min":0.37},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.018},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.504}]},{"osd":7,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.879,"15min":0.96799999999999997},"last":0.442},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":0.98899999999999999},"last":0.51900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575082,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55120,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316720,"statfs":{"total":96636764160,"available":96580321280,"internally_reserved":0,"allocated":1765376,"data_stored":891553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.375,"15min":0.375},"min":{"1min":0.223,"5min":0.223,"15min":0.17599999999999999},"max":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.54100000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.311}]},{"osd":1,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.40300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.503,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.33500000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.372},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.30399999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.75700000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.95699999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.87,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.91400000000000003,"15min":0.93600000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.625}]},{"osd":7,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.46000000000000002,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.441,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1769472,"data_stored":891575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.59199999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64800000000000002},"last":0.39400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56999999999999995,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.442}]},{"osd":4,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38,"5min":0.377,"15min":0.374},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.875},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.32400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63200000000000001},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.77800000000000002,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.72499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575081,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37656,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334184,"statfs":{"total":96636764160,"available":96598204416,"internally_reserved":0,"allocated":2363392,"data_stored":1481943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36799999999999999,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.46600000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.37,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.62,"15min":0.76900000000000002},"last":0.495}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39700000000000002,"15min":0.40699999999999997},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.56399999999999995,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39500000000000002,"15min":0.41499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.54900000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.69699999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.59199999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53200000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.438,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.89200000000000002,"5min":0.996,"15min":0.996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.92400000000000004,"5min":1.038,"15min":1.038},"last":0.69899999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.85799999999999998,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94099999999999995,"5min":1.026,"15min":1.3700000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94099999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.81399999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.64300000000000002},"min":{"1min":0.46999999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97499999999999998,"5min":1.05,"15min":1.339},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":1.01,"15min":1.01},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:44:26.527 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T10:44:26.525+0000 7fc383161640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:44:28.915 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:44:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.026+0000 7f27f3b1d640 1 -- 172.21.15.125:0/4268655690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27d8003970 msgr2=0x7f27d8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.026+0000 7f27f3b1d640 1 --2- 172.21.15.125:0/4268655690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27d8003970 0x7f27d8003d90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f27e800afc0 tx=0x7f27d80057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.026+0000 7f27f20ba640 1 -- 172.21.15.125:0/4268655690 reap_dead start 2023-10-18T10:44:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.027+0000 7f27f3b1d640 1 -- 172.21.15.125:0/4268655690 shutdown_connections 2023-10-18T10:44:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.027+0000 7f27f3b1d640 1 -- 172.21.15.125:0/4268655690 >> 172.21.15.125:0/4268655690 conn(0x7f27ec069930 msgr2=0x7f27ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.027+0000 7f27f3b1d640 1 -- 172.21.15.125:0/4268655690 shutdown_connections 2023-10-18T10:44:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.027+0000 7f27f3b1d640 1 -- 172.21.15.125:0/4268655690 wait complete. 2023-10-18T10:44:29.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.028+0000 7f27f3b1d640 1 Processor -- start 2023-10-18T10:44:29.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.028+0000 7f27f3b1d640 1 -- start start 2023-10-18T10:44:29.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f3b1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d8003970 0x7f27ec1f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f3b1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27ec1f0970 0x7f27ec13b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f3b1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec13b5e0 0x7f27ec13ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f3b1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec14ed50 con 0x7f27ec1f0970 2023-10-18T10:44:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f3b1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec13c500 con 0x7f27d8003970 2023-10-18T10:44:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f3b1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec1f0df0 con 0x7f27ec13b5e0 2023-10-18T10:44:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27ec1f0970 0x7f27ec13b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27ec1f0970 0x7f27ec13b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33950/0 (socket says 172.21.15.125:33950) 2023-10-18T10:44:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 -- 172.21.15.125:0/3641415780 learned_addr learned my addr 172.21.15.125:0/3641415780 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f20ba640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec13b5e0 0x7f27ec13ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f18b9640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d8003970 0x7f27ec1f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 -- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec13b5e0 msgr2=0x7f27ec13ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec13b5e0 0x7f27ec13ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 -- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d8003970 msgr2=0x7f27ec1f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d8003970 0x7f27ec1f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 -- 172.21.15.125:0/3641415780 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec13c6f0 con 0x7f27ec1f0970 2023-10-18T10:44:29.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f18b9640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d8003970 0x7f27ec1f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:29.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f20ba640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec13b5e0 0x7f27ec13ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:29.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f10b8640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27ec1f0970 0x7f27ec13b0a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006680 tx=0x7f27dc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:29.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27e27fc640 1 -- 172.21.15.125:0/3641415780 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc011900 con 0x7f27ec1f0970 2023-10-18T10:44:29.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27e27fc640 1 -- 172.21.15.125:0/3641415780 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f27dc011a60 con 0x7f27ec1f0970 2023-10-18T10:44:29.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27e27fc640 1 -- 172.21.15.125:0/3641415780 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc011e70 con 0x7f27ec1f0970 2023-10-18T10:44:29.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec1391b0 con 0x7f27ec1f0970 2023-10-18T10:44:29.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.029+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec139770 con 0x7f27ec1f0970 2023-10-18T10:44:29.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.031+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec0690e0 con 0x7f27ec1f0970 2023-10-18T10:44:29.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.031+0000 7f27e27fc640 1 -- 172.21.15.125:0/3641415780 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f27dc0103c0 con 0x7f27ec1f0970 2023-10-18T10:44:29.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.031+0000 7f27e27fc640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f27d006af60 0x7f27d006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.031+0000 7f27e27fc640 1 -- 172.21.15.125:0/3641415780 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f27dc08a780 con 0x7f27ec1f0970 2023-10-18T10:44:29.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.032+0000 7f27f18b9640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f27d006af60 0x7f27d006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.032+0000 7f27f18b9640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f27d006af60 0x7f27d006d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27e8037000 tx=0x7f27e8030040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:29.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.036+0000 7f27e27fc640 1 -- 172.21.15.125:0/3641415780 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27dc0554c0 con 0x7f27ec1f0970 2023-10-18T10:44:29.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.196+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 --> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27ec0cc100 con 0x7f27d006af60 2023-10-18T10:44:29.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.205+0000 7f27e27fc640 1 -- 172.21.15.125:0/3641415780 <== mgr.7786 v2:172.21.15.139:6832/4144373589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289577 (secure 0 0 0) 0x7f27e802ead0 con 0x7f27d006af60 2023-10-18T10:44:29.209 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:29.211 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:44:29.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f27d006af60 msgr2=0x7f27d006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f27d006af60 0x7f27d006d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27e8037000 tx=0x7f27e8030040 comp rx=0 tx=0).stop 2023-10-18T10:44:29.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27ec1f0970 msgr2=0x7f27ec13b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27ec1f0970 0x7f27ec13b0a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006680 tx=0x7f27dc0039c0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 shutdown_connections 2023-10-18T10:44:29.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f27d006af60 0x7f27d006d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec13b5e0 0x7f27ec13ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27ec1f0970 0x7f27ec13b0a0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 --2- 172.21.15.125:0/3641415780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d8003970 0x7f27ec1f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 >> 172.21.15.125:0/3641415780 conn(0x7f27ec069930 msgr2=0x7f27ec1578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:29.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 shutdown_connections 2023-10-18T10:44:29.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.208+0000 7f27f3b1d640 1 -- 172.21.15.125:0/3641415780 wait complete. 2023-10-18T10:44:29.220 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-18T10:44:27.282878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":419,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339468,"kb_used_data":15052,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754635252,"statfs":{"total":773094113280,"available":772746498048,"internally_reserved":0,"allocated":15413248,"data_stored":8337924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":332058168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.741086"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564850+0000","last_change":"2023-10-18T10:44:21.564341+0000","last_active":"2023-10-18T10:44:22.564850+0000","last_peered":"2023-10-18T10:44:22.564850+0000","last_clean":"2023-10-18T10:44:22.564850+0000","last_became_active":"2023-10-18T10:44:21.564034+0000","last_became_peered":"2023-10-18T10:44:21.564034+0000","last_unstale":"2023-10-18T10:44:22.564850+0000","last_undegraded":"2023-10-18T10:44:22.564850+0000","last_fullsized":"2023-10-18T10:44:22.564850+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:00.535183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578575+0000","last_change":"2023-10-18T10:44:21.556185+0000","last_active":"2023-10-18T10:44:22.578575+0000","last_peered":"2023-10-18T10:44:22.578575+0000","last_clean":"2023-10-18T10:44:22.578575+0000","last_became_active":"2023-10-18T10:44:21.555918+0000","last_became_peered":"2023-10-18T10:44:21.555918+0000","last_unstale":"2023-10-18T10:44:22.578575+0000","last_undegraded":"2023-10-18T10:44:22.578575+0000","last_fullsized":"2023-10-18T10:44:22.578575+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:14.477242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578619+0000","last_change":"2023-10-18T10:44:21.560236+0000","last_active":"2023-10-18T10:44:22.578619+0000","last_peered":"2023-10-18T10:44:22.578619+0000","last_clean":"2023-10-18T10:44:22.578619+0000","last_became_active":"2023-10-18T10:44:21.559948+0000","last_became_peered":"2023-10-18T10:44:21.559948+0000","last_unstale":"2023-10-18T10:44:22.578619+0000","last_undegraded":"2023-10-18T10:44:22.578619+0000","last_fullsized":"2023-10-18T10:44:22.578619+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:25.847090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230045+0000","last_change":"2023-10-18T10:44:21.566112+0000","last_active":"2023-10-18T10:44:23.230045+0000","last_peered":"2023-10-18T10:44:23.230045+0000","last_clean":"2023-10-18T10:44:23.230045+0000","last_became_active":"2023-10-18T10:44:21.565678+0000","last_became_peered":"2023-10-18T10:44:21.565678+0000","last_unstale":"2023-10-18T10:44:23.230045+0000","last_undegraded":"2023-10-18T10:44:23.230045+0000","last_fullsized":"2023-10-18T10:44:23.230045+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:49.126268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564305+0000","last_change":"2023-10-18T10:44:21.566246+0000","last_active":"2023-10-18T10:44:22.564305+0000","last_peered":"2023-10-18T10:44:22.564305+0000","last_clean":"2023-10-18T10:44:22.564305+0000","last_became_active":"2023-10-18T10:44:21.565923+0000","last_became_peered":"2023-10-18T10:44:21.565923+0000","last_unstale":"2023-10-18T10:44:22.564305+0000","last_undegraded":"2023-10-18T10:44:22.564305+0000","last_fullsized":"2023-10-18T10:44:22.564305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:46.001986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229705+0000","last_change":"2023-10-18T10:44:21.573416+0000","last_active":"2023-10-18T10:44:23.229705+0000","last_peered":"2023-10-18T10:44:23.229705+0000","last_clean":"2023-10-18T10:44:23.229705+0000","last_became_active":"2023-10-18T10:44:21.573170+0000","last_became_peered":"2023-10-18T10:44:21.573170+0000","last_unstale":"2023-10-18T10:44:23.229705+0000","last_undegraded":"2023-10-18T10:44:23.229705+0000","last_fullsized":"2023-10-18T10:44:23.229705+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:15.531869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229492+0000","last_change":"2023-10-18T10:44:21.574273+0000","last_active":"2023-10-18T10:44:23.229492+0000","last_peered":"2023-10-18T10:44:23.229492+0000","last_clean":"2023-10-18T10:44:23.229492+0000","last_became_active":"2023-10-18T10:44:21.574036+0000","last_became_peered":"2023-10-18T10:44:21.574036+0000","last_unstale":"2023-10-18T10:44:23.229492+0000","last_undegraded":"2023-10-18T10:44:23.229492+0000","last_fullsized":"2023-10-18T10:44:23.229492+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:29:43.710418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578168+0000","last_change":"2023-10-18T10:44:21.560023+0000","last_active":"2023-10-18T10:44:22.578168+0000","last_peered":"2023-10-18T10:44:22.578168+0000","last_clean":"2023-10-18T10:44:22.578168+0000","last_became_active":"2023-10-18T10:44:21.559761+0000","last_became_peered":"2023-10-18T10:44:21.559761+0000","last_unstale":"2023-10-18T10:44:22.578168+0000","last_undegraded":"2023-10-18T10:44:22.578168+0000","last_fullsized":"2023-10-18T10:44:22.578168+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:37.267378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230398+0000","last_change":"2023-10-18T10:44:21.570058+0000","last_active":"2023-10-18T10:44:23.230398+0000","last_peered":"2023-10-18T10:44:23.230398+0000","last_clean":"2023-10-18T10:44:23.230398+0000","last_became_active":"2023-10-18T10:44:21.569716+0000","last_became_peered":"2023-10-18T10:44:21.569716+0000","last_unstale":"2023-10-18T10:44:23.230398+0000","last_undegraded":"2023-10-18T10:44:23.230398+0000","last_fullsized":"2023-10-18T10:44:23.230398+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:01.830388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229428+0000","last_change":"2023-10-18T10:44:21.577695+0000","last_active":"2023-10-18T10:44:23.229428+0000","last_peered":"2023-10-18T10:44:23.229428+0000","last_clean":"2023-10-18T10:44:23.229428+0000","last_became_active":"2023-10-18T10:44:21.577436+0000","last_became_peered":"2023-10-18T10:44:21.577436+0000","last_unstale":"2023-10-18T10:44:23.229428+0000","last_undegraded":"2023-10-18T10:44:23.229428+0000","last_fullsized":"2023-10-18T10:44:23.229428+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:34:05.944034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564344+0000","last_change":"2023-10-18T10:44:21.572558+0000","last_active":"2023-10-18T10:44:22.564344+0000","last_peered":"2023-10-18T10:44:22.564344+0000","last_clean":"2023-10-18T10:44:22.564344+0000","last_became_active":"2023-10-18T10:44:21.571984+0000","last_became_peered":"2023-10-18T10:44:21.571984+0000","last_unstale":"2023-10-18T10:44:22.564344+0000","last_undegraded":"2023-10-18T10:44:22.564344+0000","last_fullsized":"2023-10-18T10:44:22.564344+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:42:55.664463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229952+0000","last_change":"2023-10-18T10:44:21.571295+0000","last_active":"2023-10-18T10:44:23.229952+0000","last_peered":"2023-10-18T10:44:23.229952+0000","last_clean":"2023-10-18T10:44:23.229952+0000","last_became_active":"2023-10-18T10:44:21.570928+0000","last_became_peered":"2023-10-18T10:44:21.570928+0000","last_unstale":"2023-10-18T10:44:23.229952+0000","last_undegraded":"2023-10-18T10:44:23.229952+0000","last_fullsized":"2023-10-18T10:44:23.229952+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:53:26.906540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229641+0000","last_change":"2023-10-18T10:44:21.574413+0000","last_active":"2023-10-18T10:44:23.229641+0000","last_peered":"2023-10-18T10:44:23.229641+0000","last_clean":"2023-10-18T10:44:23.229641+0000","last_became_active":"2023-10-18T10:44:21.574117+0000","last_became_peered":"2023-10-18T10:44:21.574117+0000","last_unstale":"2023-10-18T10:44:23.229641+0000","last_undegraded":"2023-10-18T10:44:23.229641+0000","last_fullsized":"2023-10-18T10:44:23.229641+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:12.650215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578339+0000","last_change":"2023-10-18T10:44:21.567413+0000","last_active":"2023-10-18T10:44:22.578339+0000","last_peered":"2023-10-18T10:44:22.578339+0000","last_clean":"2023-10-18T10:44:22.578339+0000","last_became_active":"2023-10-18T10:44:21.567166+0000","last_became_peered":"2023-10-18T10:44:21.567166+0000","last_unstale":"2023-10-18T10:44:22.578339+0000","last_undegraded":"2023-10-18T10:44:22.578339+0000","last_fullsized":"2023-10-18T10:44:22.578339+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:15.656446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564059+0000","last_change":"2023-10-18T10:44:21.572580+0000","last_active":"2023-10-18T10:44:22.564059+0000","last_peered":"2023-10-18T10:44:22.564059+0000","last_clean":"2023-10-18T10:44:22.564059+0000","last_became_active":"2023-10-18T10:44:21.572256+0000","last_became_peered":"2023-10-18T10:44:21.572256+0000","last_unstale":"2023-10-18T10:44:22.564059+0000","last_undegraded":"2023-10-18T10:44:22.564059+0000","last_fullsized":"2023-10-18T10:44:22.564059+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:45.613040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229560+0000","last_change":"2023-10-18T10:44:21.569232+0000","last_active":"2023-10-18T10:44:23.229560+0000","last_peered":"2023-10-18T10:44:23.229560+0000","last_clean":"2023-10-18T10:44:23.229560+0000","last_became_active":"2023-10-18T10:44:21.569008+0000","last_became_peered":"2023-10-18T10:44:21.569008+0000","last_unstale":"2023-10-18T10:44:23.229560+0000","last_undegraded":"2023-10-18T10:44:23.229560+0000","last_fullsized":"2023-10-18T10:44:23.229560+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:08.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229558+0000","last_change":"2023-10-18T10:44:21.573756+0000","last_active":"2023-10-18T10:44:23.229558+0000","last_peered":"2023-10-18T10:44:23.229558+0000","last_clean":"2023-10-18T10:44:23.229558+0000","last_became_active":"2023-10-18T10:44:21.573513+0000","last_became_peered":"2023-10-18T10:44:21.573513+0000","last_unstale":"2023-10-18T10:44:23.229558+0000","last_undegraded":"2023-10-18T10:44:23.229558+0000","last_fullsized":"2023-10-18T10:44:23.229558+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:59:36.510592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578668+0000","last_change":"2023-10-18T10:44:21.556821+0000","last_active":"2023-10-18T10:44:22.578668+0000","last_peered":"2023-10-18T10:44:22.578668+0000","last_clean":"2023-10-18T10:44:22.578668+0000","last_became_active":"2023-10-18T10:44:21.556568+0000","last_became_peered":"2023-10-18T10:44:21.556568+0000","last_unstale":"2023-10-18T10:44:22.578668+0000","last_undegraded":"2023-10-18T10:44:22.578668+0000","last_fullsized":"2023-10-18T10:44:22.578668+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:58:34.456605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229729+0000","last_change":"2023-10-18T10:44:21.565663+0000","last_active":"2023-10-18T10:44:23.229729+0000","last_peered":"2023-10-18T10:44:23.229729+0000","last_clean":"2023-10-18T10:44:23.229729+0000","last_became_active":"2023-10-18T10:44:21.565411+0000","last_became_peered":"2023-10-18T10:44:21.565411+0000","last_unstale":"2023-10-18T10:44:23.229729+0000","last_undegraded":"2023-10-18T10:44:23.229729+0000","last_fullsized":"2023-10-18T10:44:23.229729+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:38.817680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229141+0000","last_change":"2023-10-18T10:44:21.571929+0000","last_active":"2023-10-18T10:44:23.229141+0000","last_peered":"2023-10-18T10:44:23.229141+0000","last_clean":"2023-10-18T10:44:23.229141+0000","last_became_active":"2023-10-18T10:44:21.571674+0000","last_became_peered":"2023-10-18T10:44:21.571674+0000","last_unstale":"2023-10-18T10:44:23.229141+0000","last_undegraded":"2023-10-18T10:44:23.229141+0000","last_fullsized":"2023-10-18T10:44:23.229141+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:15.415236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230333+0000","last_change":"2023-10-18T10:44:21.563434+0000","last_active":"2023-10-18T10:44:23.230333+0000","last_peered":"2023-10-18T10:44:23.230333+0000","last_clean":"2023-10-18T10:44:23.230333+0000","last_became_active":"2023-10-18T10:44:21.563140+0000","last_became_peered":"2023-10-18T10:44:21.563140+0000","last_unstale":"2023-10-18T10:44:23.230333+0000","last_undegraded":"2023-10-18T10:44:23.230333+0000","last_fullsized":"2023-10-18T10:44:23.230333+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:37.968875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564486+0000","last_change":"2023-10-18T10:44:21.567529+0000","last_active":"2023-10-18T10:44:22.564486+0000","last_peered":"2023-10-18T10:44:22.564486+0000","last_clean":"2023-10-18T10:44:22.564486+0000","last_became_active":"2023-10-18T10:44:21.567226+0000","last_became_peered":"2023-10-18T10:44:21.567226+0000","last_unstale":"2023-10-18T10:44:22.564486+0000","last_undegraded":"2023-10-18T10:44:22.564486+0000","last_fullsized":"2023-10-18T10:44:22.564486+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:50.772585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229682+0000","last_change":"2023-10-18T10:44:21.571205+0000","last_active":"2023-10-18T10:44:23.229682+0000","last_peered":"2023-10-18T10:44:23.229682+0000","last_clean":"2023-10-18T10:44:23.229682+0000","last_became_active":"2023-10-18T10:44:21.570639+0000","last_became_peered":"2023-10-18T10:44:21.570639+0000","last_unstale":"2023-10-18T10:44:23.229682+0000","last_undegraded":"2023-10-18T10:44:23.229682+0000","last_fullsized":"2023-10-18T10:44:23.229682+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:28.045604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564020+0000","last_change":"2023-10-18T10:44:21.576465+0000","last_active":"2023-10-18T10:44:22.564020+0000","last_peered":"2023-10-18T10:44:22.564020+0000","last_clean":"2023-10-18T10:44:22.564020+0000","last_became_active":"2023-10-18T10:44:21.576113+0000","last_became_peered":"2023-10-18T10:44:21.576113+0000","last_unstale":"2023-10-18T10:44:22.564020+0000","last_undegraded":"2023-10-18T10:44:22.564020+0000","last_fullsized":"2023-10-18T10:44:22.564020+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:08.901111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564621+0000","last_change":"2023-10-18T10:44:21.573810+0000","last_active":"2023-10-18T10:44:22.564621+0000","last_peered":"2023-10-18T10:44:22.564621+0000","last_clean":"2023-10-18T10:44:22.564621+0000","last_became_active":"2023-10-18T10:44:21.571738+0000","last_became_peered":"2023-10-18T10:44:21.571738+0000","last_unstale":"2023-10-18T10:44:22.564621+0000","last_undegraded":"2023-10-18T10:44:22.564621+0000","last_fullsized":"2023-10-18T10:44:22.564621+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:25.282137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229144+0000","last_change":"2023-10-18T10:44:21.564526+0000","last_active":"2023-10-18T10:44:23.229144+0000","last_peered":"2023-10-18T10:44:23.229144+0000","last_clean":"2023-10-18T10:44:23.229144+0000","last_became_active":"2023-10-18T10:44:21.564253+0000","last_became_peered":"2023-10-18T10:44:21.564253+0000","last_unstale":"2023-10-18T10:44:23.229144+0000","last_undegraded":"2023-10-18T10:44:23.229144+0000","last_fullsized":"2023-10-18T10:44:23.229144+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:41.493559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229842+0000","last_change":"2023-10-18T10:44:21.566265+0000","last_active":"2023-10-18T10:44:23.229842+0000","last_peered":"2023-10-18T10:44:23.229842+0000","last_clean":"2023-10-18T10:44:23.229842+0000","last_became_active":"2023-10-18T10:44:21.566015+0000","last_became_peered":"2023-10-18T10:44:21.566015+0000","last_unstale":"2023-10-18T10:44:23.229842+0000","last_undegraded":"2023-10-18T10:44:23.229842+0000","last_fullsized":"2023-10-18T10:44:23.229842+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:45.613040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230026+0000","last_change":"2023-10-18T10:44:21.573779+0000","last_active":"2023-10-18T10:44:23.230026+0000","last_peered":"2023-10-18T10:44:23.230026+0000","last_clean":"2023-10-18T10:44:23.230026+0000","last_became_active":"2023-10-18T10:44:21.573438+0000","last_became_peered":"2023-10-18T10:44:21.573438+0000","last_unstale":"2023-10-18T10:44:23.230026+0000","last_undegraded":"2023-10-18T10:44:23.230026+0000","last_fullsized":"2023-10-18T10:44:23.230026+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:02.736471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229096+0000","last_change":"2023-10-18T10:44:21.560905+0000","last_active":"2023-10-18T10:44:23.229096+0000","last_peered":"2023-10-18T10:44:23.229096+0000","last_clean":"2023-10-18T10:44:23.229096+0000","last_became_active":"2023-10-18T10:44:21.560646+0000","last_became_peered":"2023-10-18T10:44:21.560646+0000","last_unstale":"2023-10-18T10:44:23.229096+0000","last_undegraded":"2023-10-18T10:44:23.229096+0000","last_fullsized":"2023-10-18T10:44:23.229096+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:33.058253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578375+0000","last_change":"2023-10-18T10:44:21.562213+0000","last_active":"2023-10-18T10:44:22.578375+0000","last_peered":"2023-10-18T10:44:22.578375+0000","last_clean":"2023-10-18T10:44:22.578375+0000","last_became_active":"2023-10-18T10:44:21.561968+0000","last_became_peered":"2023-10-18T10:44:21.561968+0000","last_unstale":"2023-10-18T10:44:22.578375+0000","last_undegraded":"2023-10-18T10:44:22.578375+0000","last_fullsized":"2023-10-18T10:44:22.578375+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:51:43.685746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.565002+0000","last_change":"2023-10-18T10:44:21.571801+0000","last_active":"2023-10-18T10:44:22.565002+0000","last_peered":"2023-10-18T10:44:22.565002+0000","last_clean":"2023-10-18T10:44:22.565002+0000","last_became_active":"2023-10-18T10:44:21.571493+0000","last_became_peered":"2023-10-18T10:44:21.571493+0000","last_unstale":"2023-10-18T10:44:22.565002+0000","last_undegraded":"2023-10-18T10:44:22.565002+0000","last_fullsized":"2023-10-18T10:44:22.565002+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:20.385907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564100+0000","last_change":"2023-10-18T10:44:21.576395+0000","last_active":"2023-10-18T10:44:22.564100+0000","last_peered":"2023-10-18T10:44:22.564100+0000","last_clean":"2023-10-18T10:44:22.564100+0000","last_became_active":"2023-10-18T10:44:21.576070+0000","last_became_peered":"2023-10-18T10:44:21.576070+0000","last_unstale":"2023-10-18T10:44:22.564100+0000","last_undegraded":"2023-10-18T10:44:22.564100+0000","last_fullsized":"2023-10-18T10:44:22.564100+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:43.691084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229598+0000","last_change":"2023-10-18T10:44:21.574452+0000","last_active":"2023-10-18T10:44:23.229598+0000","last_peered":"2023-10-18T10:44:23.229598+0000","last_clean":"2023-10-18T10:44:23.229598+0000","last_became_active":"2023-10-18T10:44:21.574176+0000","last_became_peered":"2023-10-18T10:44:21.574176+0000","last_unstale":"2023-10-18T10:44:23.229598+0000","last_undegraded":"2023-10-18T10:44:23.229598+0000","last_fullsized":"2023-10-18T10:44:23.229598+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:28.198131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"142'4","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229601+0000","last_change":"2023-10-18T10:44:20.554401+0000","last_active":"2023-10-18T10:44:23.229601+0000","last_peered":"2023-10-18T10:44:23.229601+0000","last_clean":"2023-10-18T10:44:23.229601+0000","last_became_active":"2023-10-18T10:44:20.554147+0000","last_became_peered":"2023-10-18T10:44:20.554147+0000","last_unstale":"2023-10-18T10:44:23.229601+0000","last_undegraded":"2023-10-18T10:44:23.229601+0000","last_fullsized":"2023-10-18T10:44:23.229601+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:40.125919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230036+0000","last_change":"2023-10-18T10:44:21.571376+0000","last_active":"2023-10-18T10:44:23.230036+0000","last_peered":"2023-10-18T10:44:23.230036+0000","last_clean":"2023-10-18T10:44:23.230036+0000","last_became_active":"2023-10-18T10:44:21.570991+0000","last_became_peered":"2023-10-18T10:44:21.570991+0000","last_unstale":"2023-10-18T10:44:23.230036+0000","last_undegraded":"2023-10-18T10:44:23.230036+0000","last_fullsized":"2023-10-18T10:44:23.230036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:51:03.439091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229510+0000","last_change":"2023-10-18T10:44:20.549263+0000","last_active":"2023-10-18T10:44:23.229510+0000","last_peered":"2023-10-18T10:44:23.229510+0000","last_clean":"2023-10-18T10:44:23.229510+0000","last_became_active":"2023-10-18T10:44:20.549001+0000","last_became_peered":"2023-10-18T10:44:20.549001+0000","last_unstale":"2023-10-18T10:44:23.229510+0000","last_undegraded":"2023-10-18T10:44:23.229510+0000","last_fullsized":"2023-10-18T10:44:23.229510+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:27.133011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564566+0000","last_change":"2023-10-18T10:44:21.576865+0000","last_active":"2023-10-18T10:44:22.564566+0000","last_peered":"2023-10-18T10:44:22.564566+0000","last_clean":"2023-10-18T10:44:22.564566+0000","last_became_active":"2023-10-18T10:44:21.576238+0000","last_became_peered":"2023-10-18T10:44:21.576238+0000","last_unstale":"2023-10-18T10:44:22.564566+0000","last_undegraded":"2023-10-18T10:44:22.564566+0000","last_fullsized":"2023-10-18T10:44:22.564566+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:22:49.158268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230246+0000","last_change":"2023-10-18T10:44:20.552971+0000","last_active":"2023-10-18T10:44:23.230246+0000","last_peered":"2023-10-18T10:44:23.230246+0000","last_clean":"2023-10-18T10:44:23.230246+0000","last_became_active":"2023-10-18T10:44:20.551870+0000","last_became_peered":"2023-10-18T10:44:20.551870+0000","last_unstale":"2023-10-18T10:44:23.230246+0000","last_undegraded":"2023-10-18T10:44:23.230246+0000","last_fullsized":"2023-10-18T10:44:23.230246+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:09:24.277243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230436+0000","last_change":"2023-10-18T10:44:21.563435+0000","last_active":"2023-10-18T10:44:23.230436+0000","last_peered":"2023-10-18T10:44:23.230436+0000","last_clean":"2023-10-18T10:44:23.230436+0000","last_became_active":"2023-10-18T10:44:21.563155+0000","last_became_peered":"2023-10-18T10:44:21.563155+0000","last_unstale":"2023-10-18T10:44:23.230436+0000","last_undegraded":"2023-10-18T10:44:23.230436+0000","last_fullsized":"2023-10-18T10:44:23.230436+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:15.656446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229620+0000","last_change":"2023-10-18T10:44:20.554735+0000","last_active":"2023-10-18T10:44:23.229620+0000","last_peered":"2023-10-18T10:44:23.229620+0000","last_clean":"2023-10-18T10:44:23.229620+0000","last_became_active":"2023-10-18T10:44:20.554456+0000","last_became_peered":"2023-10-18T10:44:20.554456+0000","last_unstale":"2023-10-18T10:44:23.229620+0000","last_undegraded":"2023-10-18T10:44:23.229620+0000","last_fullsized":"2023-10-18T10:44:23.229620+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:55.061568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229349+0000","last_change":"2023-10-18T10:44:21.564071+0000","last_active":"2023-10-18T10:44:23.229349+0000","last_peered":"2023-10-18T10:44:23.229349+0000","last_clean":"2023-10-18T10:44:23.229349+0000","last_became_active":"2023-10-18T10:44:21.563810+0000","last_became_peered":"2023-10-18T10:44:21.563810+0000","last_unstale":"2023-10-18T10:44:23.229349+0000","last_undegraded":"2023-10-18T10:44:23.229349+0000","last_fullsized":"2023-10-18T10:44:23.229349+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:32.562901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564409+0000","last_change":"2023-10-18T10:44:20.558619+0000","last_active":"2023-10-18T10:44:22.564409+0000","last_peered":"2023-10-18T10:44:22.564409+0000","last_clean":"2023-10-18T10:44:22.564409+0000","last_became_active":"2023-10-18T10:44:20.557889+0000","last_became_peered":"2023-10-18T10:44:20.557889+0000","last_unstale":"2023-10-18T10:44:22.564409+0000","last_undegraded":"2023-10-18T10:44:22.564409+0000","last_fullsized":"2023-10-18T10:44:22.564409+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:47.550038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564220+0000","last_change":"2023-10-18T10:44:21.568245+0000","last_active":"2023-10-18T10:44:22.564220+0000","last_peered":"2023-10-18T10:44:22.564220+0000","last_clean":"2023-10-18T10:44:22.564220+0000","last_became_active":"2023-10-18T10:44:21.567982+0000","last_became_peered":"2023-10-18T10:44:21.567982+0000","last_unstale":"2023-10-18T10:44:22.564220+0000","last_undegraded":"2023-10-18T10:44:22.564220+0000","last_fullsized":"2023-10-18T10:44:22.564220+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:07.009131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229791+0000","last_change":"2023-10-18T10:44:20.556161+0000","last_active":"2023-10-18T10:44:23.229791+0000","last_peered":"2023-10-18T10:44:23.229791+0000","last_clean":"2023-10-18T10:44:23.229791+0000","last_became_active":"2023-10-18T10:44:20.555910+0000","last_became_peered":"2023-10-18T10:44:20.555910+0000","last_unstale":"2023-10-18T10:44:23.229791+0000","last_undegraded":"2023-10-18T10:44:23.229791+0000","last_fullsized":"2023-10-18T10:44:23.229791+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:08.315986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578409+0000","last_change":"2023-10-18T10:44:21.559359+0000","last_active":"2023-10-18T10:44:22.578409+0000","last_peered":"2023-10-18T10:44:22.578409+0000","last_clean":"2023-10-18T10:44:22.578409+0000","last_became_active":"2023-10-18T10:44:21.559109+0000","last_became_peered":"2023-10-18T10:44:21.559109+0000","last_unstale":"2023-10-18T10:44:22.578409+0000","last_undegraded":"2023-10-18T10:44:22.578409+0000","last_fullsized":"2023-10-18T10:44:22.578409+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:20.603381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564906+0000","last_change":"2023-10-18T10:44:20.555390+0000","last_active":"2023-10-18T10:44:22.564906+0000","last_peered":"2023-10-18T10:44:22.564906+0000","last_clean":"2023-10-18T10:44:22.564906+0000","last_became_active":"2023-10-18T10:44:20.555150+0000","last_became_peered":"2023-10-18T10:44:20.555150+0000","last_unstale":"2023-10-18T10:44:22.564906+0000","last_undegraded":"2023-10-18T10:44:22.564906+0000","last_fullsized":"2023-10-18T10:44:22.564906+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:35:41.665405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229705+0000","last_change":"2023-10-18T10:44:21.567559+0000","last_active":"2023-10-18T10:44:23.229705+0000","last_peered":"2023-10-18T10:44:23.229705+0000","last_clean":"2023-10-18T10:44:23.229705+0000","last_became_active":"2023-10-18T10:44:21.567301+0000","last_became_peered":"2023-10-18T10:44:21.567301+0000","last_unstale":"2023-10-18T10:44:23.229705+0000","last_undegraded":"2023-10-18T10:44:23.229705+0000","last_fullsized":"2023-10-18T10:44:23.229705+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:04.527071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229782+0000","last_change":"2023-10-18T10:44:20.561064+0000","last_active":"2023-10-18T10:44:23.229782+0000","last_peered":"2023-10-18T10:44:23.229782+0000","last_clean":"2023-10-18T10:44:23.229782+0000","last_became_active":"2023-10-18T10:44:20.560807+0000","last_became_peered":"2023-10-18T10:44:20.560807+0000","last_unstale":"2023-10-18T10:44:23.229782+0000","last_undegraded":"2023-10-18T10:44:23.229782+0000","last_fullsized":"2023-10-18T10:44:23.229782+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:24:47.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229953+0000","last_change":"2023-10-18T10:44:21.560982+0000","last_active":"2023-10-18T10:44:23.229953+0000","last_peered":"2023-10-18T10:44:23.229953+0000","last_clean":"2023-10-18T10:44:23.229953+0000","last_became_active":"2023-10-18T10:44:21.560705+0000","last_became_peered":"2023-10-18T10:44:21.560705+0000","last_unstale":"2023-10-18T10:44:23.229953+0000","last_undegraded":"2023-10-18T10:44:23.229953+0000","last_fullsized":"2023-10-18T10:44:23.229953+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:07.009131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229691+0000","last_change":"2023-10-18T10:44:20.558136+0000","last_active":"2023-10-18T10:44:23.229691+0000","last_peered":"2023-10-18T10:44:23.229691+0000","last_clean":"2023-10-18T10:44:23.229691+0000","last_became_active":"2023-10-18T10:44:20.557868+0000","last_became_peered":"2023-10-18T10:44:20.557868+0000","last_unstale":"2023-10-18T10:44:23.229691+0000","last_undegraded":"2023-10-18T10:44:23.229691+0000","last_fullsized":"2023-10-18T10:44:23.229691+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:13.933951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564849+0000","last_change":"2023-10-18T10:44:21.572721+0000","last_active":"2023-10-18T10:44:22.564849+0000","last_peered":"2023-10-18T10:44:22.564849+0000","last_clean":"2023-10-18T10:44:22.564849+0000","last_became_active":"2023-10-18T10:44:21.572452+0000","last_became_peered":"2023-10-18T10:44:21.572452+0000","last_unstale":"2023-10-18T10:44:22.564849+0000","last_undegraded":"2023-10-18T10:44:22.564849+0000","last_fullsized":"2023-10-18T10:44:22.564849+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:11.608888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564656+0000","last_change":"2023-10-18T10:44:20.554698+0000","last_active":"2023-10-18T10:44:22.564656+0000","last_peered":"2023-10-18T10:44:22.564656+0000","last_clean":"2023-10-18T10:44:22.564656+0000","last_became_active":"2023-10-18T10:44:20.554438+0000","last_became_peered":"2023-10-18T10:44:20.554438+0000","last_unstale":"2023-10-18T10:44:22.564656+0000","last_undegraded":"2023-10-18T10:44:22.564656+0000","last_fullsized":"2023-10-18T10:44:22.564656+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:46.314630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578684+0000","last_change":"2023-10-18T10:44:21.561055+0000","last_active":"2023-10-18T10:44:22.578684+0000","last_peered":"2023-10-18T10:44:22.578684+0000","last_clean":"2023-10-18T10:44:22.578684+0000","last_became_active":"2023-10-18T10:44:21.560278+0000","last_became_peered":"2023-10-18T10:44:21.560278+0000","last_unstale":"2023-10-18T10:44:22.578684+0000","last_undegraded":"2023-10-18T10:44:22.578684+0000","last_fullsized":"2023-10-18T10:44:22.578684+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:53:01.261584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"142'3","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564965+0000","last_change":"2023-10-18T10:44:20.553723+0000","last_active":"2023-10-18T10:44:22.564965+0000","last_peered":"2023-10-18T10:44:22.564965+0000","last_clean":"2023-10-18T10:44:22.564965+0000","last_became_active":"2023-10-18T10:44:20.553398+0000","last_became_peered":"2023-10-18T10:44:20.553398+0000","last_unstale":"2023-10-18T10:44:22.564965+0000","last_undegraded":"2023-10-18T10:44:22.564965+0000","last_fullsized":"2023-10-18T10:44:22.564965+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:11.270346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564776+0000","last_change":"2023-10-18T10:44:21.573464+0000","last_active":"2023-10-18T10:44:22.564776+0000","last_peered":"2023-10-18T10:44:22.564776+0000","last_clean":"2023-10-18T10:44:22.564776+0000","last_became_active":"2023-10-18T10:44:21.572310+0000","last_became_peered":"2023-10-18T10:44:21.572310+0000","last_unstale":"2023-10-18T10:44:22.564776+0000","last_undegraded":"2023-10-18T10:44:22.564776+0000","last_fullsized":"2023-10-18T10:44:22.564776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:11.670089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"142'6","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578498+0000","last_change":"2023-10-18T10:44:20.560661+0000","last_active":"2023-10-18T10:44:22.578498+0000","last_peered":"2023-10-18T10:44:22.578498+0000","last_clean":"2023-10-18T10:44:22.578498+0000","last_became_active":"2023-10-18T10:44:20.560167+0000","last_became_peered":"2023-10-18T10:44:20.560167+0000","last_unstale":"2023-10-18T10:44:22.578498+0000","last_undegraded":"2023-10-18T10:44:22.578498+0000","last_fullsized":"2023-10-18T10:44:22.578498+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:42.686189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229978+0000","last_change":"2023-10-18T10:44:21.558514+0000","last_active":"2023-10-18T10:44:23.229978+0000","last_peered":"2023-10-18T10:44:23.229978+0000","last_clean":"2023-10-18T10:44:23.229978+0000","last_became_active":"2023-10-18T10:44:21.558237+0000","last_became_peered":"2023-10-18T10:44:21.558237+0000","last_unstale":"2023-10-18T10:44:23.229978+0000","last_undegraded":"2023-10-18T10:44:23.229978+0000","last_fullsized":"2023-10-18T10:44:23.229978+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:46.001986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578398+0000","last_change":"2023-10-18T10:44:20.552670+0000","last_active":"2023-10-18T10:44:22.578398+0000","last_peered":"2023-10-18T10:44:22.578398+0000","last_clean":"2023-10-18T10:44:22.578398+0000","last_became_active":"2023-10-18T10:44:20.552408+0000","last_became_peered":"2023-10-18T10:44:20.552408+0000","last_unstale":"2023-10-18T10:44:22.578398+0000","last_undegraded":"2023-10-18T10:44:22.578398+0000","last_fullsized":"2023-10-18T10:44:22.578398+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:10.665194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229402+0000","last_change":"2023-10-18T10:44:21.566860+0000","last_active":"2023-10-18T10:44:23.229402+0000","last_peered":"2023-10-18T10:44:23.229402+0000","last_clean":"2023-10-18T10:44:23.229402+0000","last_became_active":"2023-10-18T10:44:21.566602+0000","last_became_peered":"2023-10-18T10:44:21.566602+0000","last_unstale":"2023-10-18T10:44:23.229402+0000","last_undegraded":"2023-10-18T10:44:23.229402+0000","last_fullsized":"2023-10-18T10:44:23.229402+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:29.254273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578516+0000","last_change":"2023-10-18T10:44:20.557841+0000","last_active":"2023-10-18T10:44:22.578516+0000","last_peered":"2023-10-18T10:44:22.578516+0000","last_clean":"2023-10-18T10:44:22.578516+0000","last_became_active":"2023-10-18T10:44:20.557579+0000","last_became_peered":"2023-10-18T10:44:20.557579+0000","last_unstale":"2023-10-18T10:44:22.578516+0000","last_undegraded":"2023-10-18T10:44:22.578516+0000","last_fullsized":"2023-10-18T10:44:22.578516+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:27.040709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.228997+0000","last_change":"2023-10-18T10:44:21.561345+0000","last_active":"2023-10-18T10:44:23.228997+0000","last_peered":"2023-10-18T10:44:23.228997+0000","last_clean":"2023-10-18T10:44:23.228997+0000","last_became_active":"2023-10-18T10:44:21.561090+0000","last_became_peered":"2023-10-18T10:44:21.561090+0000","last_unstale":"2023-10-18T10:44:23.228997+0000","last_undegraded":"2023-10-18T10:44:23.228997+0000","last_fullsized":"2023-10-18T10:44:23.228997+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:53.624058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578287+0000","last_change":"2023-10-18T10:44:20.560994+0000","last_active":"2023-10-18T10:44:22.578287+0000","last_peered":"2023-10-18T10:44:22.578287+0000","last_clean":"2023-10-18T10:44:22.578287+0000","last_became_active":"2023-10-18T10:44:20.560402+0000","last_became_peered":"2023-10-18T10:44:20.560402+0000","last_unstale":"2023-10-18T10:44:22.578287+0000","last_undegraded":"2023-10-18T10:44:22.578287+0000","last_fullsized":"2023-10-18T10:44:22.578287+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:36.301690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564425+0000","last_change":"2023-10-18T10:44:21.573842+0000","last_active":"2023-10-18T10:44:22.564425+0000","last_peered":"2023-10-18T10:44:22.564425+0000","last_clean":"2023-10-18T10:44:22.564425+0000","last_became_active":"2023-10-18T10:44:21.573572+0000","last_became_peered":"2023-10-18T10:44:21.573572+0000","last_unstale":"2023-10-18T10:44:22.564425+0000","last_undegraded":"2023-10-18T10:44:22.564425+0000","last_fullsized":"2023-10-18T10:44:22.564425+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:36.142060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229650+0000","last_change":"2023-10-18T10:44:20.558709+0000","last_active":"2023-10-18T10:44:23.229650+0000","last_peered":"2023-10-18T10:44:23.229650+0000","last_clean":"2023-10-18T10:44:23.229650+0000","last_became_active":"2023-10-18T10:44:20.558454+0000","last_became_peered":"2023-10-18T10:44:20.558454+0000","last_unstale":"2023-10-18T10:44:23.229650+0000","last_undegraded":"2023-10-18T10:44:23.229650+0000","last_fullsized":"2023-10-18T10:44:23.229650+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:35:25.000743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564329+0000","last_change":"2023-10-18T10:44:21.571099+0000","last_active":"2023-10-18T10:44:22.564329+0000","last_peered":"2023-10-18T10:44:22.564329+0000","last_clean":"2023-10-18T10:44:22.564329+0000","last_became_active":"2023-10-18T10:44:21.570843+0000","last_became_peered":"2023-10-18T10:44:21.570843+0000","last_unstale":"2023-10-18T10:44:22.564329+0000","last_undegraded":"2023-10-18T10:44:22.564329+0000","last_fullsized":"2023-10-18T10:44:22.564329+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:46:36.531300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229275+0000","last_change":"2023-10-18T10:44:20.557537+0000","last_active":"2023-10-18T10:44:23.229275+0000","last_peered":"2023-10-18T10:44:23.229275+0000","last_clean":"2023-10-18T10:44:23.229275+0000","last_became_active":"2023-10-18T10:44:20.557279+0000","last_became_peered":"2023-10-18T10:44:20.557279+0000","last_unstale":"2023-10-18T10:44:23.229275+0000","last_undegraded":"2023-10-18T10:44:23.229275+0000","last_fullsized":"2023-10-18T10:44:23.229275+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:31:17.119852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230128+0000","last_change":"2023-10-18T10:44:21.569266+0000","last_active":"2023-10-18T10:44:23.230128+0000","last_peered":"2023-10-18T10:44:23.230128+0000","last_clean":"2023-10-18T10:44:23.230128+0000","last_became_active":"2023-10-18T10:44:21.568991+0000","last_became_peered":"2023-10-18T10:44:21.568991+0000","last_unstale":"2023-10-18T10:44:23.230128+0000","last_undegraded":"2023-10-18T10:44:23.230128+0000","last_fullsized":"2023-10-18T10:44:23.230128+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:14.757285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"142'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230308+0000","last_change":"2023-10-18T10:44:20.556381+0000","last_active":"2023-10-18T10:44:23.230308+0000","last_peered":"2023-10-18T10:44:23.230308+0000","last_clean":"2023-10-18T10:44:23.230308+0000","last_became_active":"2023-10-18T10:44:20.555808+0000","last_became_peered":"2023-10-18T10:44:20.555808+0000","last_unstale":"2023-10-18T10:44:23.230308+0000","last_undegraded":"2023-10-18T10:44:23.230308+0000","last_fullsized":"2023-10-18T10:44:23.230308+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:10.665194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564870+0000","last_change":"2023-10-18T10:44:21.572780+0000","last_active":"2023-10-18T10:44:22.564870+0000","last_peered":"2023-10-18T10:44:22.564870+0000","last_clean":"2023-10-18T10:44:22.564870+0000","last_became_active":"2023-10-18T10:44:21.571305+0000","last_became_peered":"2023-10-18T10:44:21.571305+0000","last_unstale":"2023-10-18T10:44:22.564870+0000","last_undegraded":"2023-10-18T10:44:22.564870+0000","last_fullsized":"2023-10-18T10:44:22.564870+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:37.306584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230003+0000","last_change":"2023-10-18T10:44:20.560300+0000","last_active":"2023-10-18T10:44:23.230003+0000","last_peered":"2023-10-18T10:44:23.230003+0000","last_clean":"2023-10-18T10:44:23.230003+0000","last_became_active":"2023-10-18T10:44:20.560024+0000","last_became_peered":"2023-10-18T10:44:20.560024+0000","last_unstale":"2023-10-18T10:44:23.230003+0000","last_undegraded":"2023-10-18T10:44:23.230003+0000","last_fullsized":"2023-10-18T10:44:23.230003+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:54:41.781127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229707+0000","last_change":"2023-10-18T10:44:21.569228+0000","last_active":"2023-10-18T10:44:23.229707+0000","last_peered":"2023-10-18T10:44:23.229707+0000","last_clean":"2023-10-18T10:44:23.229707+0000","last_became_active":"2023-10-18T10:44:21.568808+0000","last_became_peered":"2023-10-18T10:44:21.568808+0000","last_unstale":"2023-10-18T10:44:23.229707+0000","last_undegraded":"2023-10-18T10:44:23.229707+0000","last_fullsized":"2023-10-18T10:44:23.229707+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:57:44.839359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229830+0000","last_change":"2023-10-18T10:44:20.560128+0000","last_active":"2023-10-18T10:44:23.229830+0000","last_peered":"2023-10-18T10:44:23.229830+0000","last_clean":"2023-10-18T10:44:23.229830+0000","last_became_active":"2023-10-18T10:44:20.559860+0000","last_became_peered":"2023-10-18T10:44:20.559860+0000","last_unstale":"2023-10-18T10:44:23.229830+0000","last_undegraded":"2023-10-18T10:44:23.229830+0000","last_fullsized":"2023-10-18T10:44:23.229830+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:22.637653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578197+0000","last_change":"2023-10-18T10:44:21.577386+0000","last_active":"2023-10-18T10:44:22.578197+0000","last_peered":"2023-10-18T10:44:22.578197+0000","last_clean":"2023-10-18T10:44:22.578197+0000","last_became_active":"2023-10-18T10:44:21.577122+0000","last_became_peered":"2023-10-18T10:44:21.577122+0000","last_unstale":"2023-10-18T10:44:22.578197+0000","last_undegraded":"2023-10-18T10:44:22.578197+0000","last_fullsized":"2023-10-18T10:44:22.578197+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:37.968875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229944+0000","last_change":"2023-10-18T10:44:20.557928+0000","last_active":"2023-10-18T10:44:23.229944+0000","last_peered":"2023-10-18T10:44:23.229944+0000","last_clean":"2023-10-18T10:44:23.229944+0000","last_became_active":"2023-10-18T10:44:20.557670+0000","last_became_peered":"2023-10-18T10:44:20.557670+0000","last_unstale":"2023-10-18T10:44:23.229944+0000","last_undegraded":"2023-10-18T10:44:23.229944+0000","last_fullsized":"2023-10-18T10:44:23.229944+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:19.381013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578543+0000","last_change":"2023-10-18T10:44:21.566860+0000","last_active":"2023-10-18T10:44:22.578543+0000","last_peered":"2023-10-18T10:44:22.578543+0000","last_clean":"2023-10-18T10:44:22.578543+0000","last_became_active":"2023-10-18T10:44:21.566409+0000","last_became_peered":"2023-10-18T10:44:21.566409+0000","last_unstale":"2023-10-18T10:44:22.578543+0000","last_undegraded":"2023-10-18T10:44:22.578543+0000","last_fullsized":"2023-10-18T10:44:22.578543+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:12.324740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578307+0000","last_change":"2023-10-18T10:44:20.558099+0000","last_active":"2023-10-18T10:44:22.578307+0000","last_peered":"2023-10-18T10:44:22.578307+0000","last_clean":"2023-10-18T10:44:22.578307+0000","last_became_active":"2023-10-18T10:44:20.557837+0000","last_became_peered":"2023-10-18T10:44:20.557837+0000","last_unstale":"2023-10-18T10:44:22.578307+0000","last_undegraded":"2023-10-18T10:44:22.578307+0000","last_fullsized":"2023-10-18T10:44:22.578307+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:56.009593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564355+0000","last_change":"2023-10-18T10:44:21.572333+0000","last_active":"2023-10-18T10:44:22.564355+0000","last_peered":"2023-10-18T10:44:22.564355+0000","last_clean":"2023-10-18T10:44:22.564355+0000","last_became_active":"2023-10-18T10:44:21.572035+0000","last_became_peered":"2023-10-18T10:44:21.572035+0000","last_unstale":"2023-10-18T10:44:22.564355+0000","last_undegraded":"2023-10-18T10:44:22.564355+0000","last_fullsized":"2023-10-18T10:44:22.564355+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:17.682666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229348+0000","last_change":"2023-10-18T10:44:20.555014+0000","last_active":"2023-10-18T10:44:23.229348+0000","last_peered":"2023-10-18T10:44:23.229348+0000","last_clean":"2023-10-18T10:44:23.229348+0000","last_became_active":"2023-10-18T10:44:20.554187+0000","last_became_peered":"2023-10-18T10:44:20.554187+0000","last_unstale":"2023-10-18T10:44:23.229348+0000","last_undegraded":"2023-10-18T10:44:23.229348+0000","last_fullsized":"2023-10-18T10:44:23.229348+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:07.896216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230157+0000","last_change":"2023-10-18T10:44:21.573151+0000","last_active":"2023-10-18T10:44:23.230157+0000","last_peered":"2023-10-18T10:44:23.230157+0000","last_clean":"2023-10-18T10:44:23.230157+0000","last_became_active":"2023-10-18T10:44:21.572419+0000","last_became_peered":"2023-10-18T10:44:21.572419+0000","last_unstale":"2023-10-18T10:44:23.230157+0000","last_undegraded":"2023-10-18T10:44:23.230157+0000","last_fullsized":"2023-10-18T10:44:23.230157+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:51:53.373910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"142'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230113+0000","last_change":"2023-10-18T10:44:20.560435+0000","last_active":"2023-10-18T10:44:23.230113+0000","last_peered":"2023-10-18T10:44:23.230113+0000","last_clean":"2023-10-18T10:44:23.230113+0000","last_became_active":"2023-10-18T10:44:20.560180+0000","last_became_peered":"2023-10-18T10:44:20.560180+0000","last_unstale":"2023-10-18T10:44:23.230113+0000","last_undegraded":"2023-10-18T10:44:23.230113+0000","last_fullsized":"2023-10-18T10:44:23.230113+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:02.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229141+0000","last_change":"2023-10-18T10:44:21.563539+0000","last_active":"2023-10-18T10:44:23.229141+0000","last_peered":"2023-10-18T10:44:23.229141+0000","last_clean":"2023-10-18T10:44:23.229141+0000","last_became_active":"2023-10-18T10:44:21.563281+0000","last_became_peered":"2023-10-18T10:44:21.563281+0000","last_unstale":"2023-10-18T10:44:23.229141+0000","last_undegraded":"2023-10-18T10:44:23.229141+0000","last_fullsized":"2023-10-18T10:44:23.229141+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:29:42.989075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229801+0000","last_change":"2023-10-18T10:44:20.558728+0000","last_active":"2023-10-18T10:44:23.229801+0000","last_peered":"2023-10-18T10:44:23.229801+0000","last_clean":"2023-10-18T10:44:23.229801+0000","last_became_active":"2023-10-18T10:44:20.558466+0000","last_became_peered":"2023-10-18T10:44:20.558466+0000","last_unstale":"2023-10-18T10:44:23.229801+0000","last_undegraded":"2023-10-18T10:44:23.229801+0000","last_fullsized":"2023-10-18T10:44:23.229801+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:15.327159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230308+0000","last_change":"2023-10-18T10:44:21.565365+0000","last_active":"2023-10-18T10:44:23.230308+0000","last_peered":"2023-10-18T10:44:23.230308+0000","last_clean":"2023-10-18T10:44:23.230308+0000","last_became_active":"2023-10-18T10:44:21.565087+0000","last_became_peered":"2023-10-18T10:44:21.565087+0000","last_unstale":"2023-10-18T10:44:23.230308+0000","last_undegraded":"2023-10-18T10:44:23.230308+0000","last_fullsized":"2023-10-18T10:44:23.230308+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:37.267378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229858+0000","last_change":"2023-10-18T10:44:20.556980+0000","last_active":"2023-10-18T10:44:23.229858+0000","last_peered":"2023-10-18T10:44:23.229858+0000","last_clean":"2023-10-18T10:44:23.229858+0000","last_became_active":"2023-10-18T10:44:20.556696+0000","last_became_peered":"2023-10-18T10:44:20.556696+0000","last_unstale":"2023-10-18T10:44:23.229858+0000","last_undegraded":"2023-10-18T10:44:23.229858+0000","last_fullsized":"2023-10-18T10:44:23.229858+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:37:25.602393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229211+0000","last_change":"2023-10-18T10:44:21.572221+0000","last_active":"2023-10-18T10:44:23.229211+0000","last_peered":"2023-10-18T10:44:23.229211+0000","last_clean":"2023-10-18T10:44:23.229211+0000","last_became_active":"2023-10-18T10:44:21.571952+0000","last_became_peered":"2023-10-18T10:44:21.571952+0000","last_unstale":"2023-10-18T10:44:23.229211+0000","last_undegraded":"2023-10-18T10:44:23.229211+0000","last_fullsized":"2023-10-18T10:44:23.229211+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:14:14.754025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230218+0000","last_change":"2023-10-18T10:44:20.562360+0000","last_active":"2023-10-18T10:44:23.230218+0000","last_peered":"2023-10-18T10:44:23.230218+0000","last_clean":"2023-10-18T10:44:23.230218+0000","last_became_active":"2023-10-18T10:44:20.562101+0000","last_became_peered":"2023-10-18T10:44:20.562101+0000","last_unstale":"2023-10-18T10:44:23.230218+0000","last_undegraded":"2023-10-18T10:44:23.230218+0000","last_fullsized":"2023-10-18T10:44:23.230218+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:54.222737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229767+0000","last_change":"2023-10-18T10:44:21.569070+0000","last_active":"2023-10-18T10:44:23.229767+0000","last_peered":"2023-10-18T10:44:23.229767+0000","last_clean":"2023-10-18T10:44:23.229767+0000","last_became_active":"2023-10-18T10:44:21.568704+0000","last_became_peered":"2023-10-18T10:44:21.568704+0000","last_unstale":"2023-10-18T10:44:23.229767+0000","last_undegraded":"2023-10-18T10:44:23.229767+0000","last_fullsized":"2023-10-18T10:44:23.229767+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:06.144512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564507+0000","last_change":"2023-10-18T10:44:20.559585+0000","last_active":"2023-10-18T10:44:22.564507+0000","last_peered":"2023-10-18T10:44:22.564507+0000","last_clean":"2023-10-18T10:44:22.564507+0000","last_became_active":"2023-10-18T10:44:20.559293+0000","last_became_peered":"2023-10-18T10:44:20.559293+0000","last_unstale":"2023-10-18T10:44:22.564507+0000","last_undegraded":"2023-10-18T10:44:22.564507+0000","last_fullsized":"2023-10-18T10:44:22.564507+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:21.797094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578233+0000","last_change":"2023-10-18T10:44:21.567212+0000","last_active":"2023-10-18T10:44:22.578233+0000","last_peered":"2023-10-18T10:44:22.578233+0000","last_clean":"2023-10-18T10:44:22.578233+0000","last_became_active":"2023-10-18T10:44:21.566945+0000","last_became_peered":"2023-10-18T10:44:21.566945+0000","last_unstale":"2023-10-18T10:44:22.578233+0000","last_undegraded":"2023-10-18T10:44:22.578233+0000","last_fullsized":"2023-10-18T10:44:22.578233+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:01.830388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.564631+0000","last_change":"2023-10-18T10:44:20.560139+0000","last_active":"2023-10-18T10:44:22.564631+0000","last_peered":"2023-10-18T10:44:22.564631+0000","last_clean":"2023-10-18T10:44:22.564631+0000","last_became_active":"2023-10-18T10:44:20.559766+0000","last_became_peered":"2023-10-18T10:44:20.559766+0000","last_unstale":"2023-10-18T10:44:22.564631+0000","last_undegraded":"2023-10-18T10:44:22.564631+0000","last_fullsized":"2023-10-18T10:44:22.564631+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:37:25.602393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229550+0000","last_change":"2023-10-18T10:44:21.570144+0000","last_active":"2023-10-18T10:44:23.229550+0000","last_peered":"2023-10-18T10:44:23.229550+0000","last_clean":"2023-10-18T10:44:23.229550+0000","last_became_active":"2023-10-18T10:44:21.569829+0000","last_became_peered":"2023-10-18T10:44:21.569829+0000","last_unstale":"2023-10-18T10:44:23.229550+0000","last_undegraded":"2023-10-18T10:44:23.229550+0000","last_fullsized":"2023-10-18T10:44:23.229550+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:53.278742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229571+0000","last_change":"2023-10-18T10:44:20.561863+0000","last_active":"2023-10-18T10:44:23.229571+0000","last_peered":"2023-10-18T10:44:23.229571+0000","last_clean":"2023-10-18T10:44:23.229571+0000","last_became_active":"2023-10-18T10:44:20.561608+0000","last_became_peered":"2023-10-18T10:44:20.561608+0000","last_unstale":"2023-10-18T10:44:23.229571+0000","last_undegraded":"2023-10-18T10:44:23.229571+0000","last_fullsized":"2023-10-18T10:44:23.229571+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:46.320265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229764+0000","last_change":"2023-10-18T10:44:21.566242+0000","last_active":"2023-10-18T10:44:23.229764+0000","last_peered":"2023-10-18T10:44:23.229764+0000","last_clean":"2023-10-18T10:44:23.229764+0000","last_became_active":"2023-10-18T10:44:21.566000+0000","last_became_peered":"2023-10-18T10:44:21.566000+0000","last_unstale":"2023-10-18T10:44:23.229764+0000","last_undegraded":"2023-10-18T10:44:23.229764+0000","last_fullsized":"2023-10-18T10:44:23.229764+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:30.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.229345+0000","last_change":"2023-10-18T10:44:20.550433+0000","last_active":"2023-10-18T10:44:23.229345+0000","last_peered":"2023-10-18T10:44:23.229345+0000","last_clean":"2023-10-18T10:44:23.229345+0000","last_became_active":"2023-10-18T10:44:20.550162+0000","last_became_peered":"2023-10-18T10:44:20.550162+0000","last_unstale":"2023-10-18T10:44:23.229345+0000","last_undegraded":"2023-10-18T10:44:23.229345+0000","last_fullsized":"2023-10-18T10:44:23.229345+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:16.677771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:23.230436+0000","last_change":"2023-10-18T10:44:21.564597+0000","last_active":"2023-10-18T10:44:23.230436+0000","last_peered":"2023-10-18T10:44:23.230436+0000","last_clean":"2023-10-18T10:44:23.230436+0000","last_became_active":"2023-10-18T10:44:21.564330+0000","last_became_peered":"2023-10-18T10:44:21.564330+0000","last_unstale":"2023-10-18T10:44:23.230436+0000","last_undegraded":"2023-10-18T10:44:23.230436+0000","last_fullsized":"2023-10-18T10:44:23.230436+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:20.538434+0000","last_clean_scrub_stamp":"2023-10-18T10:44:20.538434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:51:43.685746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"142'8","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T10:44:22.578191+0000","last_change":"2023-10-18T10:44:20.552906+0000","last_active":"2023-10-18T10:44:22.578191+0000","last_peered":"2023-10-18T10:44:22.578191+0000","last_clean":"2023-10-18T10:44:22.578191+0000","last_became_active":"2023-10-18T10:44:20.552648+0000","last_became_peered":"2023-10-18T10:44:20.552648+0000","last_unstale":"2023-10-18T10:44:22.578191+0000","last_undegraded":"2023-10-18T10:44:22.578191+0000","last_fullsized":"2023-10-18T10:44:22.578191+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:19.533540+0000","last_clean_scrub_stamp":"2023-10-18T10:44:19.533540+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:46.314630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"131'236","reported_seq":701,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T10:44:21.553098+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:44:21.553098+0000","last_peered":"2023-10-18T10:44:21.553098+0000","last_clean":"2023-10-18T10:44:21.553098+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:44:21.553098+0000","last_undegraded":"2023-10-18T10:44:21.553098+0000","last_fullsized":"2023-10-18T10:44:21.553098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1154,"num_read_kb":2351,"num_write":375,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575082,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2375680,"data_stored":1482697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.217,"15min":0.217},"max":{"1min":0.77900000000000003,"5min":0.876,"15min":0.88300000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.434},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.90700000000000003},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77900000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.505},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60099999999999998},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63400000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.86199999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575082,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1769472,"data_stored":892135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.81200000000000006,"15min":0.82499999999999996},"last":0.502},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72099999999999997,"5min":0.78500000000000003,"15min":0.83399999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34100000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.74199999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.748,"5min":0.80300000000000005,"15min":0.80400000000000005},"last":0.53200000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.75700000000000001,"5min":0.77600000000000002,"15min":0.81399999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.313},"max":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.54200000000000004}]},{"osd":4,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.66200000000000003,"5min":0.872,"15min":0.872},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.51800000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.30199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40500000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.39600000000000002,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.87},"last":0.57199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575082,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50976,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320864,"statfs":{"total":96636764160,"available":96584564736,"internally_reserved":0,"allocated":1785856,"data_stored":898361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.38800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56499999999999995,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.71499999999999997,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.81299999999999994}]},{"osd":1,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.93600000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.66500000000000004,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.52300000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.84199999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.378},"max":{"1min":0.75600000000000001,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.69999999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60899999999999999,"5min":0.72299999999999998,"15min":0.80100000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.505,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.83299999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.83099999999999996}]},{"osd":7,"last update":"Wed Oct 18 10:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.31},"max":{"1min":0.71099999999999997,"5min":0.84299999999999997,"15min":0.89300000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.85299999999999998,"15min":0.871},"last":0.85899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575082,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37092,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334748,"statfs":{"total":96636764160,"available":96598781952,"internally_reserved":0,"allocated":1789952,"data_stored":899813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.76200000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.877},"last":0.57699999999999996}]},{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83199999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.89300000000000002},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81999999999999995,"15min":0.85899999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.46500000000000002,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.873,"15min":0.873},"last":0.629}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79800000000000004,"5min":0.81299999999999994,"15min":0.86099999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74199999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.505,"15min":0.498},"min":{"1min":0.377,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35999999999999999}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51100000000000001,"15min":0.501},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":1.446},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81599999999999995,"15min":0.84399999999999997},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":1.0469999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":0.874,"5min":0.88600000000000001,"15min":1.0880000000000001},"last":0.64200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575082,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":899847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.36499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.62},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.59299999999999997},"last":0.29499999999999998}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20799999999999999},"max":{"1min":0.504,"5min":0.624,"15min":0.69099999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.59299999999999997,"15min":0.67000000000000004},"last":0.318}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.98299999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.59599999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86599999999999999,"15min":1.002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.88100000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.53400000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38,"15min":0.37},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.018},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.504}]},{"osd":7,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.879,"15min":0.96799999999999997},"last":0.442},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":0.98899999999999999},"last":0.51900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575082,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55120,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316720,"statfs":{"total":96636764160,"available":96580321280,"internally_reserved":0,"allocated":1765376,"data_stored":891553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.375,"15min":0.375},"min":{"1min":0.223,"5min":0.223,"15min":0.17599999999999999},"max":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.192},"max":{"1min":0.54100000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.311}]},{"osd":1,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.40300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.503,"5min":0.64800000000000002,"15min":0.67000000000000004},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.33500000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.372},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.58899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.30399999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.75700000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.95699999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.87,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.42499999999999999,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.89100000000000001,"5min":0.91400000000000003,"15min":0.93600000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.625}]},{"osd":7,"last update":"Wed Oct 18 10:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.46000000000000002,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.88900000000000001,"15min":0.91000000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.441,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1769472,"data_stored":891575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.22600000000000001,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.59199999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20100000000000001},"max":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.70799999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51600000000000001,"5min":0.627,"15min":0.64800000000000002},"last":0.39400000000000002}]},{"osd":3,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56999999999999995,"5min":0.71499999999999997,"15min":0.72899999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40999999999999998,"15min":0.41199999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.59499999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.442}]},{"osd":4,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38,"5min":0.377,"15min":0.374},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.875},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.32400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Wed Oct 18 10:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.63200000000000001},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.77800000000000002,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.72499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575081,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37656,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334184,"statfs":{"total":96636764160,"available":96598204416,"internally_reserved":0,"allocated":2363392,"data_stored":1481943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36799999999999999,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.46600000000000003,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.37,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.214,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.62,"15min":0.76900000000000002},"last":0.495}]},{"osd":2,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39700000000000002,"15min":0.40699999999999997},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.56399999999999995,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39500000000000002,"15min":0.41499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.54900000000000004}]},{"osd":3,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.56000000000000005,"5min":0.629,"15min":0.69699999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.59199999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53200000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.438,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.89200000000000002,"5min":0.996,"15min":0.996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.92400000000000004,"5min":1.038,"15min":1.038},"last":0.69899999999999995}]},{"osd":5,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.85799999999999998,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63,"15min":0.64000000000000001},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94099999999999995,"5min":1.026,"15min":1.3700000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94099999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.81399999999999995}]},{"osd":7,"last update":"Wed Oct 18 10:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.64300000000000002},"min":{"1min":0.46999999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97499999999999998,"5min":1.05,"15min":1.339},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":1.01,"15min":1.01},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:44:29.226 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:44:29.227 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:44:29.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.368+0000 7fb620f3c640 1 -- 172.21.15.125:0/1185375553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb61c159bd0 msgr2=0x7fb61c15c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.368+0000 7fb620f3c640 1 --2- 172.21.15.125:0/1185375553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb61c159bd0 0x7fb61c15c770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb604005fd0 tx=0x7fb60402bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:29.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.368+0000 7fb620f3c640 1 -- 172.21.15.125:0/1185375553 shutdown_connections 2023-10-18T10:44:29.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.368+0000 7fb620f3c640 1 --2- 172.21.15.125:0/1185375553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb61c159bd0 0x7fb61c15c770 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.368+0000 7fb620f3c640 1 --2- 172.21.15.125:0/1185375553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb61c06b030 0x7fb61c06b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.368+0000 7fb620f3c640 1 -- 172.21.15.125:0/1185375553 >> 172.21.15.125:0/1185375553 conn(0x7fb61c069930 msgr2=0x7fb61c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.368+0000 7fb620f3c640 1 -- 172.21.15.125:0/1185375553 shutdown_connections 2023-10-18T10:44:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.368+0000 7fb620f3c640 1 -- 172.21.15.125:0/1185375553 wait complete. 2023-10-18T10:44:29.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.369+0000 7fb620f3c640 1 Processor -- start 2023-10-18T10:44:29.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.369+0000 7fb620f3c640 1 -- start start 2023-10-18T10:44:29.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb620f3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb61c06b030 0x7fb61c12d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb620f3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c159bd0 0x7fb61c0aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb620f3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb61c15ccb0 0x7fb61c12c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb620f3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c12bdc0 con 0x7fb61c159bd0 2023-10-18T10:44:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb620f3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c12dba0 con 0x7fb61c15ccb0 2023-10-18T10:44:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb620f3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c12b1b0 con 0x7fb61c06b030 2023-10-18T10:44:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c159bd0 0x7fb61c0aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c159bd0 0x7fb61c0aa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33960/0 (socket says 172.21.15.125:33960) 2023-10-18T10:44:29.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb61a59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb61c06b030 0x7fb61c12d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 -- 172.21.15.125:0/2293955295 learned_addr learned my addr 172.21.15.125:0/2293955295 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:29.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb61ad9d640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb61c15ccb0 0x7fb61c12c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 -- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb61c06b030 msgr2=0x7fb61c12d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb61c06b030 0x7fb61c12d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 -- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb61c15ccb0 msgr2=0x7fb61c12c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb61c15ccb0 0x7fb61c12c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 -- 172.21.15.125:0/2293955295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb604009560 con 0x7fb61c159bd0 2023-10-18T10:44:29.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb613fff640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c159bd0 0x7fb61c0aa2f0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb604036040 tx=0x7fb60402da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb612ffd640 1 -- 172.21.15.125:0/2293955295 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60402de60 con 0x7fb61c159bd0 2023-10-18T10:44:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb612ffd640 1 -- 172.21.15.125:0/2293955295 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb6040036e0 con 0x7fb61c159bd0 2023-10-18T10:44:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c12b490 con 0x7fb61c159bd0 2023-10-18T10:44:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.370+0000 7fb612ffd640 1 -- 172.21.15.125:0/2293955295 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604003af0 con 0x7fb61c159bd0 2023-10-18T10:44:29.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.371+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c12ba50 con 0x7fb61c159bd0 2023-10-18T10:44:29.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.371+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c06c570 con 0x7fb61c159bd0 2023-10-18T10:44:29.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.372+0000 7fb612ffd640 1 -- 172.21.15.125:0/2293955295 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb604003c50 con 0x7fb61c159bd0 2023-10-18T10:44:29.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.372+0000 7fb612ffd640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb5fc06af60 0x7fb5fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.372+0000 7fb612ffd640 1 -- 172.21.15.125:0/2293955295 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fb6040ac2b0 con 0x7fb61c159bd0 2023-10-18T10:44:29.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.373+0000 7fb61a59c640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb5fc06af60 0x7fb5fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.373+0000 7fb61a59c640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb5fc06af60 0x7fb5fc06d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb61c1533b0 tx=0x7fb60c005f90 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:29.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.377+0000 7fb612ffd640 1 -- 172.21.15.125:0/2293955295 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6040771a0 con 0x7fb61c159bd0 2023-10-18T10:44:29.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.612+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb61c0690e0 con 0x7fb61c159bd0 2023-10-18T10:44:29.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.614+0000 7fb612ffd640 1 -- 172.21.15.125:0/2293955295 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb60407a7c0 con 0x7fb61c159bd0 2023-10-18T10:44:29.616 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:44:29.617 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:44:29.617 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T10:44:29.617 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T10:44:29.617 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:44:29.619 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T10:44:29.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb5fc06af60 msgr2=0x7fb5fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb5fc06af60 0x7fb5fc06d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb61c1533b0 tx=0x7fb60c005f90 comp rx=0 tx=0).stop 2023-10-18T10:44:29.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c159bd0 msgr2=0x7fb61c0aa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c159bd0 0x7fb61c0aa2f0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb604036040 tx=0x7fb60402da60 comp rx=0 tx=0).stop 2023-10-18T10:44:29.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 shutdown_connections 2023-10-18T10:44:29.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fb5fc06af60 0x7fb5fc06d420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb61c15ccb0 0x7fb61c12c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c159bd0 0x7fb61c0aa2f0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 --2- 172.21.15.125:0/2293955295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb61c06b030 0x7fb61c12d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 >> 172.21.15.125:0/2293955295 conn(0x7fb61c069930 msgr2=0x7fb61c15ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:29.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 shutdown_connections 2023-10-18T10:44:29.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.616+0000 7fb620f3c640 1 -- 172.21.15.125:0/2293955295 wait complete. 2023-10-18T10:44:29.629 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:29.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.729+0000 7ff9e0373640 1 -- 172.21.15.125:0/1436997408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80afa20 msgr2=0x7ff9d80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.729+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1436997408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80afa20 0x7ff9d80705a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009830 tx=0x7ff9c802bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:29.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.729+0000 7ff9e0373640 1 -- 172.21.15.125:0/1436997408 shutdown_connections 2023-10-18T10:44:29.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.729+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1436997408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80afa20 0x7ff9d80705a0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.729+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1436997408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9d80cbd20 0x7ff9d8158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.729+0000 7ff9e0373640 1 -- 172.21.15.125:0/1436997408 >> 172.21.15.125:0/1436997408 conn(0x7ff9d8069930 msgr2=0x7ff9d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:29.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.729+0000 7ff9e0373640 1 -- 172.21.15.125:0/1436997408 shutdown_connections 2023-10-18T10:44:29.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.729+0000 7ff9e0373640 1 -- 172.21.15.125:0/1436997408 wait complete. 2023-10-18T10:44:29.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.730+0000 7ff9e0373640 1 Processor -- start 2023-10-18T10:44:29.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.730+0000 7ff9e0373640 1 -- start start 2023-10-18T10:44:29.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9e0373640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9d80afa20 0x7ff9d813dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9e0373640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9d80b0530 0x7ff9d80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9e0373640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80cbd20 0x7ff9d8148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9e0373640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d813e6a0 con 0x7ff9d80cbd20 2023-10-18T10:44:29.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9e0373640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d813e980 con 0x7ff9d80b0530 2023-10-18T10:44:29.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9e0373640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8139760 con 0x7ff9d80afa20 2023-10-18T10:44:29.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80cbd20 0x7ff9d8148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80cbd20 0x7ff9d8148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33966/0 (socket says 172.21.15.125:33966) 2023-10-18T10:44:29.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 -- 172.21.15.125:0/1952523057 learned_addr learned my addr 172.21.15.125:0/1952523057 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:29.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de10f640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9d80afa20 0x7ff9d813dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 -- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9d80afa20 msgr2=0x7ff9d813dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9dd90e640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9d80b0530 0x7ff9d80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9d80afa20 0x7ff9d813dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 -- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9d80b0530 msgr2=0x7ff9d80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9d80b0530 0x7ff9d80caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 -- 172.21.15.125:0/1952523057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8009490 con 0x7ff9d80cbd20 2023-10-18T10:44:29.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9de910640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80cbd20 0x7ff9d8148220 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000cb40 tx=0x7ff9d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:29.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.731+0000 7ff9ceffd640 1 -- 172.21.15.125:0/1952523057 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000aca0 con 0x7ff9d80cbd20 2023-10-18T10:44:29.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.732+0000 7ff9ceffd640 1 -- 172.21.15.125:0/1952523057 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9d0012420 con 0x7ff9d80cbd20 2023-10-18T10:44:29.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.732+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d8070cf0 con 0x7ff9d80cbd20 2023-10-18T10:44:29.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.732+0000 7ff9ceffd640 1 -- 172.21.15.125:0/1952523057 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0012850 con 0x7ff9d80cbd20 2023-10-18T10:44:29.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.732+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d813b1c0 con 0x7ff9d80cbd20 2023-10-18T10:44:29.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.733+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d8139c80 con 0x7ff9d80cbd20 2023-10-18T10:44:29.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.733+0000 7ff9ceffd640 1 -- 172.21.15.125:0/1952523057 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff9d001a3b0 con 0x7ff9d80cbd20 2023-10-18T10:44:29.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.733+0000 7ff9ceffd640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff9bc06af60 0x7ff9bc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:29.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.733+0000 7ff9ceffd640 1 -- 172.21.15.125:0/1952523057 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7ff9d008a0c0 con 0x7ff9d80cbd20 2023-10-18T10:44:29.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.734+0000 7ff9de10f640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff9bc06af60 0x7ff9bc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:29.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.734+0000 7ff9de10f640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff9bc06af60 0x7ff9bc06d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40058c0 tx=0x7ff9d4005850 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:29.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.737+0000 7ff9ceffd640 1 -- 172.21.15.125:0/1952523057 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9d8139c80 con 0x7ff9d80cbd20 2023-10-18T10:44:29.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.936+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9d8139c80 con 0x7ff9d80cbd20 2023-10-18T10:44:29.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.936+0000 7ff9ceffd640 1 -- 172.21.15.125:0/1952523057 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4924 (secure 0 0 0) 0x7ff9d8139c80 con 0x7ff9d80cbd20 2023-10-18T10:44:29.939 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:29.939 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":53,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7112,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/585529921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":585529921},{"type":"v1","addr":"172.21.15.125:6835","nonce":585529921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1680315524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1680315524},{"type":"v1","addr":"172.21.15.139:6837","nonce":1680315524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1127901065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1127901065},{"type":"v1","addr":"172.21.15.139:6836","nonce":1127901065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:20.891510+0000","modified":"2023-10-18T10:44:22.895207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9159":{"gid":9159,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2651847004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2651847004},{"type":"v1","addr":"172.21.15.125:6834","nonce":2651847004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T10:44:29.940 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 53 2023-10-18T10:44:29.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff9bc06af60 msgr2=0x7ff9bc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff9bc06af60 0x7ff9bc06d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40058c0 tx=0x7ff9d4005850 comp rx=0 tx=0).stop 2023-10-18T10:44:29.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80cbd20 msgr2=0x7ff9d8148220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:29.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80cbd20 0x7ff9d8148220 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000cb40 tx=0x7ff9d000a060 comp rx=0 tx=0).stop 2023-10-18T10:44:29.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 shutdown_connections 2023-10-18T10:44:29.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7ff9bc06af60 0x7ff9bc06d420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9d80b0530 0x7ff9d80caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d80cbd20 0x7ff9d8148220 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 --2- 172.21.15.125:0/1952523057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9d80afa20 0x7ff9d813dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:29.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 >> 172.21.15.125:0/1952523057 conn(0x7ff9d8069930 msgr2=0x7ff9d80ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:29.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 shutdown_connections 2023-10-18T10:44:29.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:29.938+0000 7ff9e0373640 1 -- 172.21.15.125:0/1952523057 wait complete. 2023-10-18T10:44:30.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:44:20.891510+0000', 'modified': '2023-10-18T10:44:22.895207+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9159}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9159': {'gid': 9159, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6834/2651847004', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6832', 'nonce': 2651847004}, {'type': 'v1', 'addr': '172.21.15.125:6834', 'nonce': 2651847004}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:44:30.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:44:30.355 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:44:30.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.458+0000 7fa89538a640 1 -- 172.21.15.125:0/3826935272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8780039a0 msgr2=0x7fa878003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.458+0000 7fa89538a640 1 --2- 172.21.15.125:0/3826935272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8780039a0 0x7fa878003dc0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa880009e20 tx=0x7fa8780057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.458+0000 7fa88f7fe640 1 -- 172.21.15.125:0/3826935272 reap_dead start 2023-10-18T10:44:30.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.458+0000 7fa89538a640 1 -- 172.21.15.125:0/3826935272 shutdown_connections 2023-10-18T10:44:30.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.458+0000 7fa89538a640 1 -- 172.21.15.125:0/3826935272 >> 172.21.15.125:0/3826935272 conn(0x7fa89006fae0 msgr2=0x7fa89006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:30.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.458+0000 7fa89538a640 1 -- 172.21.15.125:0/3826935272 shutdown_connections 2023-10-18T10:44:30.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.458+0000 7fa89538a640 1 -- 172.21.15.125:0/3826935272 wait complete. 2023-10-18T10:44:30.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa89538a640 1 Processor -- start 2023-10-18T10:44:30.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa89538a640 1 -- start start 2023-10-18T10:44:30.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa89538a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8780039a0 0x7fa890147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa89538a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8901e13b0 0x7fa8901e1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa89538a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8901e35a0 0x7fa8901e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa89538a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901e3f60 con 0x7fa8901e35a0 2023-10-18T10:44:30.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa89538a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901e1f00 con 0x7fa8780039a0 2023-10-18T10:44:30.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa89538a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901e21e0 con 0x7fa8901e13b0 2023-10-18T10:44:30.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa88f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8901e35a0 0x7fa8901e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa88f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8901e35a0 0x7fa8901e3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33972/0 (socket says 172.21.15.125:33972) 2023-10-18T10:44:30.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.460+0000 7fa88f7fe640 1 -- 172.21.15.125:0/2473543080 learned_addr learned my addr 172.21.15.125:0/2473543080 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:30.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa88f7fe640 1 -- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8901e13b0 msgr2=0x7fa8901e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa88f7fe640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8901e13b0 0x7fa8901e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa88f7fe640 1 -- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8780039a0 msgr2=0x7fa890147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa88f7fe640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8780039a0 0x7fa890147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa88f7fe640 1 -- 172.21.15.125:0/2473543080 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8900b2330 con 0x7fa8901e35a0 2023-10-18T10:44:30.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa88f7fe640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8901e35a0 0x7fa8901e3a20 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa890065ed0 tx=0x7fa88800e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:30.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa8777fe640 1 -- 172.21.15.125:0/2473543080 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888011870 con 0x7fa8901e35a0 2023-10-18T10:44:30.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa8777fe640 1 -- 172.21.15.125:0/2473543080 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa888011ae0 con 0x7fa8901e35a0 2023-10-18T10:44:30.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa89538a640 1 -- 172.21.15.125:0/2473543080 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8901e24c0 con 0x7fa8901e35a0 2023-10-18T10:44:30.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa8777fe640 1 -- 172.21.15.125:0/2473543080 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8880103c0 con 0x7fa8901e35a0 2023-10-18T10:44:30.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.461+0000 7fa89538a640 1 -- 172.21.15.125:0/2473543080 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8901e2a80 con 0x7fa8901e35a0 2023-10-18T10:44:30.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.462+0000 7fa8777fe640 1 -- 172.21.15.125:0/2473543080 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa888010520 con 0x7fa8901e35a0 2023-10-18T10:44:30.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.463+0000 7fa8777fe640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa87c06af60 0x7fa87c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.463+0000 7fa8777fe640 1 -- 172.21.15.125:0/2473543080 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa888089f50 con 0x7fa8901e35a0 2023-10-18T10:44:30.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.463+0000 7fa88effd640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa87c06af60 0x7fa87c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.464+0000 7fa88effd640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa87c06af60 0x7fa87c06d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa880036000 tx=0x7fa880035040 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.464+0000 7fa897ff4740 1 Processor -- start 2023-10-18T10:44:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.464+0000 7fa897ff4740 1 -- start start 2023-10-18T10:44:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 0x5599f909cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f909d020 0x5599f90cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa897ff4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f8fae730 con 0x5599f90c9f20 2023-10-18T10:44:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa897ff4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f900f780 con 0x5599f90c9b40 2023-10-18T10:44:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa897ff4740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599f900ce70 con 0x5599f909d020 2023-10-18T10:44:30.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33974/0 (socket says 172.21.15.125:33974) 2023-10-18T10:44:30.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 -- 172.21.15.125:0/4007337263 learned_addr learned my addr 172.21.15.125:0/4007337263 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:30.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88effd640 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 0x5599f909cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88f7fe640 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f909d020 0x5599f90cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 -- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f909d020 msgr2=0x5599f90cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:44:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f909d020 0x5599f90cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 -- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 msgr2=0x5599f909cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:44:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 0x5599f909cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 -- 172.21.15.125:0/4007337263 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599f900d3d0 con 0x5599f90c9f20 2023-10-18T10:44:30.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88f7fe640 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f909d020 0x5599f90cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:30.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90cc8d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa884006310 tx=0x7fa8840049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec59363ff8b354f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:30.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa874ff9640 1 -- 172.21.15.125:0/4007337263 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884011660 con 0x5599f90c9f20 2023-10-18T10:44:30.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa874ff9640 1 -- 172.21.15.125:0/4007337263 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa8840118b0 con 0x5599f90c9f20 2023-10-18T10:44:30.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.465+0000 7fa874ff9640 1 -- 172.21.15.125:0/4007337263 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884011ca0 con 0x5599f90c9f20 2023-10-18T10:44:30.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 -- 172.21.15.125:0/4007337263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 msgr2=0x5599f90cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90cc8d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa884006310 tx=0x7fa8840049d0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 -- 172.21.15.125:0/4007337263 shutdown_connections 2023-10-18T10:44:30.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f909d020 0x5599f90cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90cc8d0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 --2- 172.21.15.125:0/4007337263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 0x5599f909cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 -- 172.21.15.125:0/4007337263 >> 172.21.15.125:0/4007337263 conn(0x5599f909ed50 msgr2=0x5599f90e78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:30.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 -- 172.21.15.125:0/4007337263 shutdown_connections 2023-10-18T10:44:30.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.466+0000 7fa897ff4740 1 -- 172.21.15.125:0/4007337263 wait complete. 2023-10-18T10:44:30.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.467+0000 7fa897ff4740 1 -- start start 2023-10-18T10:44:30.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.467+0000 7fa897ff4740 1 Processor -- start 2023-10-18T10:44:30.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.467+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 0x5599f90cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.467+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.467+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90e2650 0x5599f90e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.467+0000 7fa897ff4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f9008760 con 0x5599f90c9f20 2023-10-18T10:44:30.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.467+0000 7fa897ff4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f8fc38f0 con 0x5599f90c9b40 2023-10-18T10:44:30.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.467+0000 7fa897ff4740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599f8fc3590 con 0x5599f90e2650 2023-10-18T10:44:30.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa897ff4740 10 client.0 _ll_register_callbacks cb 0x5599f8fffee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:44:30.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90e2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33988/0 (socket says 172.21.15.125:33988) 2023-10-18T10:44:30.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 -- 172.21.15.125:0/3572537588 learned_addr learned my addr 172.21.15.125:0/3572537588 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:30.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88f7fe640 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90e2650 0x5599f90e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88effd640 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 0x5599f90cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 -- 172.21.15.125:0/3572537588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90e2650 msgr2=0x5599f90e6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:44:30.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90e2650 0x5599f90e6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 -- 172.21.15.125:0/3572537588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 msgr2=0x5599f90cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:44:30.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 0x5599f90cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 -- 172.21.15.125:0/3572537588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8900b13d0 con 0x5599f90c9f20 2023-10-18T10:44:30.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90e2110 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa884012000 tx=0x7fa884011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31204b8fba2cd59 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:30.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa86dffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:44:30.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa86dffb640 1 -- 172.21.15.125:0/3572537588 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884027440 con 0x5599f90c9f20 2023-10-18T10:44:30.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa897ff4740 1 -- 172.21.15.125:0/3572537588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599f9001740 con 0x5599f90c9f20 2023-10-18T10:44:30.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa86dffb640 1 -- 172.21.15.125:0/3572537588 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa884028db0 con 0x5599f90c9f20 2023-10-18T10:44:30.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa86dffb640 1 -- 172.21.15.125:0/3572537588 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa88401d420 con 0x5599f90c9f20 2023-10-18T10:44:30.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa897ff4740 10 client.9333 fetch_fsmap learned FSMap version 53 2023-10-18T10:44:30.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa897ff4740 1 -- 172.21.15.125:0/3572537588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5599f9006de0 con 0x5599f90c9f20 2023-10-18T10:44:30.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.468+0000 7fa86dffb640 1 -- 172.21.15.125:0/3572537588 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5599f9001740 con 0x5599f90c9f20 2023-10-18T10:44:30.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa86dffb640 1 -- 172.21.15.125:0/3572537588 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa8840275a0 con 0x5599f90c9f20 2023-10-18T10:44:30.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa897ff4740 10 client.9333 fetch_fsmap finished waiting for FSMap version 53 2023-10-18T10:44:30.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa897ff4740 10 client.9333 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:44:30.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa897ff4740 20 client.9333 populate_metadata read hostname 'smithi125' 2023-10-18T10:44:30.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] conn(0x5599f90faf80 0x5599f90fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa897ff4740 4 client.9333 mds_command: new command op to 9159 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:44:30.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa897ff4740 1 -- 172.21.15.125:0/3572537588 --> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599f9001460 con 0x5599f90faf80 2023-10-18T10:44:30.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa88effd640 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] conn(0x5599f90faf80 0x5599f90fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa88effd640 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] conn(0x5599f90faf80 0x5599f90fb360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe2a34769142acb server_cookie=4df249d803d436fd in_seq=0 out_seq=0 2023-10-18T10:44:30.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.469+0000 7fa86dffb640 10 client.9333 ms_handle_connect on v2:172.21.15.125:6832/2651847004 2023-10-18T10:44:30.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa86dffb640 1 -- 172.21.15.125:0/3572537588 <== mds.0 v2:172.21.15.125:6832/2651847004 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599f9001460 con 0x5599f90faf80 2023-10-18T10:44:30.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa86dffb640 10 client.9333 handle_command_reply: tid=0 2023-10-18T10:44:30.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 client.9333 shutdown 2023-10-18T10:44:30.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 -- 172.21.15.125:0/3572537588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 msgr2=0x5599f90e2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90e2110 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa884012000 tx=0x7fa884011690 comp rx=0 tx=0).stop 2023-10-18T10:44:30.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 -- 172.21.15.125:0/3572537588 shutdown_connections 2023-10-18T10:44:30.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] conn(0x5599f90faf80 0x5599f90fb360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90e2650 0x5599f90e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa86dffb640 0 client.9333 ms_handle_reset on v2:172.21.15.125:6832/2651847004 2023-10-18T10:44:30.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90e2110 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3572537588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90c9b40 0x5599f90cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 -- 172.21.15.125:0/3572537588 >> 172.21.15.125:0/3572537588 conn(0x5599f909d020 msgr2=0x5599f90a0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:30.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 -- 172.21.15.125:0/3572537588 shutdown_connections 2023-10-18T10:44:30.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 1 -- 172.21.15.125:0/3572537588 wait complete. 2023-10-18T10:44:30.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.470+0000 7fa897ff4740 20 client.9333 trim_cache size 0 max 16384 2023-10-18T10:44:30.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.485+0000 7fa897ff4740 1 Processor -- start 2023-10-18T10:44:30.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.485+0000 7fa897ff4740 1 -- start start 2023-10-18T10:44:30.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9b40 0x5599f90a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9f20 0x5599f90e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 0x5599f90eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa897ff4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f8f89270 con 0x5599f90c9b40 2023-10-18T10:44:30.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa897ff4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f90ceae0 con 0x5599f90e2650 2023-10-18T10:44:30.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa897ff4740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599f90e7360 con 0x5599f90c9f20 2023-10-18T10:44:30.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9b40 0x5599f90a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9b40 0x5599f90a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33992/0 (socket says 172.21.15.125:33992) 2023-10-18T10:44:30.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 -- 172.21.15.125:0/2030165805 learned_addr learned my addr 172.21.15.125:0/2030165805 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:30.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 -- 172.21.15.125:0/2030165805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9f20 msgr2=0x5599f90e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:44:30.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 --2- 172.21.15.125:0/2030165805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9f20 0x5599f90e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 -- 172.21.15.125:0/2030165805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 msgr2=0x5599f90eade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:44:30.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 --2- 172.21.15.125:0/2030165805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 0x5599f90eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 -- 172.21.15.125:0/2030165805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599f8fa5d00 con 0x5599f90c9b40 2023-10-18T10:44:30.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa88effd640 1 --2- 172.21.15.125:0/2030165805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9b40 0x5599f90a2c00 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa880037040 tx=0x7fa88003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268e3797eb8f84c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:30.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa8757fa640 1 -- 172.21.15.125:0/2030165805 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880033c60 con 0x5599f90c9b40 2023-10-18T10:44:30.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa8757fa640 1 -- 172.21.15.125:0/2030165805 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa880033dc0 con 0x5599f90c9b40 2023-10-18T10:44:30.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.486+0000 7fa8757fa640 1 -- 172.21.15.125:0/2030165805 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88003d640 con 0x5599f90c9b40 2023-10-18T10:44:30.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 -- 172.21.15.125:0/2030165805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9b40 msgr2=0x5599f90a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 --2- 172.21.15.125:0/2030165805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9b40 0x5599f90a2c00 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa880037040 tx=0x7fa88003d000 comp rx=0 tx=0).stop 2023-10-18T10:44:30.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 -- 172.21.15.125:0/2030165805 shutdown_connections 2023-10-18T10:44:30.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 --2- 172.21.15.125:0/2030165805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 0x5599f90eade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 --2- 172.21.15.125:0/2030165805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9f20 0x5599f90e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 --2- 172.21.15.125:0/2030165805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9b40 0x5599f90a2c00 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 -- 172.21.15.125:0/2030165805 >> 172.21.15.125:0/2030165805 conn(0x5599f909d020 msgr2=0x5599f90de950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:30.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 -- 172.21.15.125:0/2030165805 shutdown_connections 2023-10-18T10:44:30.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 -- 172.21.15.125:0/2030165805 wait complete. 2023-10-18T10:44:30.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 -- start start 2023-10-18T10:44:30.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.487+0000 7fa897ff4740 1 Processor -- start 2023-10-18T10:44:30.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9b40 0x5599f9102900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa897ff4740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 0x5599f90eb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa897ff4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f90ea750 con 0x5599f90c9f20 2023-10-18T10:44:30.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa897ff4740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599f90ea220 con 0x5599f90e2650 2023-10-18T10:44:30.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa897ff4740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599f90ea470 con 0x5599f90c9b40 2023-10-18T10:44:30.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa897ff4740 10 client.0 _ll_register_callbacks cb 0x7fa884006c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:44:30.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90eade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34004/0 (socket says 172.21.15.125:34004) 2023-10-18T10:44:30.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88e7fc640 1 -- 172.21.15.125:0/3064446470 learned_addr learned my addr 172.21.15.125:0/3064446470 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:30.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88f7fe640 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 0x5599f90eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88effd640 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9b40 0x5599f9102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88e7fc640 1 -- 172.21.15.125:0/3064446470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9b40 msgr2=0x5599f9102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:44:30.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9b40 0x5599f9102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88e7fc640 1 -- 172.21.15.125:0/3064446470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 msgr2=0x5599f90eb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:44:30.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 0x5599f90eb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.488+0000 7fa88e7fc640 1 -- 172.21.15.125:0/3064446470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa878003520 con 0x5599f90c9f20 2023-10-18T10:44:30.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa88e7fc640 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90eade0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa884027d10 tx=0x7fa884027d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc02d9b10e43208 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:30.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa86dffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:44:30.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa86dffb640 1 -- 172.21.15.125:0/3064446470 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884027640 con 0x5599f90c9f20 2023-10-18T10:44:30.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 1 -- 172.21.15.125:0/3064446470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8840118b0 con 0x5599f90c9f20 2023-10-18T10:44:30.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa86dffb640 1 -- 172.21.15.125:0/3064446470 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa884006180 con 0x5599f90c9f20 2023-10-18T10:44:30.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa86dffb640 1 -- 172.21.15.125:0/3064446470 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa88401d420 con 0x5599f90c9f20 2023-10-18T10:44:30.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 10 client.9339 fetch_fsmap learned FSMap version 53 2023-10-18T10:44:30.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 1 -- 172.21.15.125:0/3064446470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5599f8fcb4f0 con 0x5599f90c9f20 2023-10-18T10:44:30.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa86dffb640 1 -- 172.21.15.125:0/3064446470 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa8840118b0 con 0x5599f90c9f20 2023-10-18T10:44:30.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa86dffb640 1 -- 172.21.15.125:0/3064446470 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa88401d670 con 0x5599f90c9f20 2023-10-18T10:44:30.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 10 client.9339 fetch_fsmap finished waiting for FSMap version 53 2023-10-18T10:44:30.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 10 client.9339 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:44:30.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 20 client.9339 populate_metadata read hostname 'smithi125' 2023-10-18T10:44:30.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] conn(0x5599f91541e0 0x5599f91545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 4 client.9339 mds_command: new command op to 9159 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:44:30.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa897ff4740 1 -- 172.21.15.125:0/3064446470 --> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5599f8d0a6c0 con 0x5599f91541e0 2023-10-18T10:44:30.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa88effd640 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] conn(0x5599f91541e0 0x5599f91545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa86dffb640 10 client.9339 ms_handle_connect on v2:172.21.15.125:6832/2651847004 2023-10-18T10:44:30.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.489+0000 7fa88effd640 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] conn(0x5599f91541e0 0x5599f91545c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7768e9c433f4e308 server_cookie=e1baa2ce41463339 in_seq=0 out_seq=0 2023-10-18T10:44:30.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa86dffb640 1 -- 172.21.15.125:0/3064446470 <== mds.0 v2:172.21.15.125:6832/2651847004 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5599f8d0a6c0 con 0x5599f91541e0 2023-10-18T10:44:30.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa86dffb640 10 client.9339 handle_command_reply: tid=0 2023-10-18T10:44:30.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 client.9339 shutdown 2023-10-18T10:44:30.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 -- 172.21.15.125:0/3064446470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 msgr2=0x5599f90eade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90eade0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa884027d10 tx=0x7fa884027d40 comp rx=0 tx=0).stop 2023-10-18T10:44:30.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 -- 172.21.15.125:0/3064446470 shutdown_connections 2023-10-18T10:44:30.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.125:6832/2651847004,v1:172.21.15.125:6834/2651847004] conn(0x5599f91541e0 0x5599f91545c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5599f90e2650 0x5599f90eb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa86dffb640 0 client.9339 ms_handle_reset on v2:172.21.15.125:6832/2651847004 2023-10-18T10:44:30.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5599f90c9f20 0x5599f90eade0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 --2- 172.21.15.125:0/3064446470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5599f90c9b40 0x5599f9102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 -- 172.21.15.125:0/3064446470 >> 172.21.15.125:0/3064446470 conn(0x5599f909d020 msgr2=0x5599f90a08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:30.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 -- 172.21.15.125:0/3064446470 shutdown_connections 2023-10-18T10:44:30.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 1 -- 172.21.15.125:0/3064446470 wait complete. 2023-10-18T10:44:30.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa897ff4740 20 client.9339 trim_cache size 0 max 16384 2023-10-18T10:44:30.515 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:44:30.516 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:44:30.516 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:44:30.516 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 9159, 2023-10-18T10:44:30.516 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:44:30.516 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:44:30.517 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:44:30.517 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.9337044280000004, 2023-10-18T10:44:30.517 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 53, 2023-10-18T10:44:30.517 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 143, 2023-10-18T10:44:30.517 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 143, 2023-10-18T10:44:30.518 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 14.690875465 2023-10-18T10:44:30.518 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:44:30.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 -- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa87c06af60 msgr2=0x7fa87c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa87c06af60 0x7fa87c06d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa880036000 tx=0x7fa880035040 comp rx=0 tx=0).stop 2023-10-18T10:44:30.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 -- 172.21.15.125:0/2473543080 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8901e35a0 msgr2=0x7fa8901e3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8901e35a0 0x7fa8901e3a20 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa890065ed0 tx=0x7fa88800e980 comp rx=0 tx=0).stop 2023-10-18T10:44:30.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 -- 172.21.15.125:0/2473543080 shutdown_connections 2023-10-18T10:44:30.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa87c06af60 0x7fa87c06d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8901e35a0 0x7fa8901e3a20 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8901e13b0 0x7fa8901e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 --2- 172.21.15.125:0/2473543080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8780039a0 0x7fa890147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 -- 172.21.15.125:0/2473543080 >> 172.21.15.125:0/2473543080 conn(0x7fa89006fae0 msgr2=0x7fa89014a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:30.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 -- 172.21.15.125:0/2473543080 shutdown_connections 2023-10-18T10:44:30.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.490+0000 7fa89538a640 1 -- 172.21.15.125:0/2473543080 wait complete. 2023-10-18T10:44:30.522 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:44:30.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.625+0000 7f5c605b0640 1 -- 172.21.15.125:0/2338415472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 msgr2=0x7f5c44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.625+0000 7f5c605b0640 1 --2- 172.21.15.125:0/2338415472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 0x7f5c44004c40 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006fd0 tx=0x7f5c440057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.625+0000 7f5c605b0640 1 -- 172.21.15.125:0/2338415472 shutdown_connections 2023-10-18T10:44:30.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.625+0000 7f5c605b0640 1 --2- 172.21.15.125:0/2338415472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 0x7f5c44004c40 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.625+0000 7f5c605b0640 1 -- 172.21.15.125:0/2338415472 >> 172.21.15.125:0/2338415472 conn(0x7f5c580d0d60 msgr2=0x7f5c580d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:30.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.625+0000 7f5c605b0640 1 -- 172.21.15.125:0/2338415472 shutdown_connections 2023-10-18T10:44:30.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.625+0000 7f5c605b0640 1 -- 172.21.15.125:0/2338415472 wait complete. 2023-10-18T10:44:30.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.626+0000 7f5c605b0640 1 Processor -- start 2023-10-18T10:44:30.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.626+0000 7f5c605b0640 1 -- start start 2023-10-18T10:44:30.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c605b0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 0x7f5c581f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c605b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c581f4d70 0x7f5c5815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c605b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5814b310 0x7f5c5814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c605b0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b4d70 con 0x7f5c440047c0 2023-10-18T10:44:30.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c605b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581f2700 con 0x7f5c5814b310 2023-10-18T10:44:30.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c605b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581f5230 con 0x7f5c581f4d70 2023-10-18T10:44:30.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 0x7f5c581f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 0x7f5c581f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34020/0 (socket says 172.21.15.125:34020) 2023-10-18T10:44:30.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 -- 172.21.15.125:0/3367117333 learned_addr learned my addr 172.21.15.125:0/3367117333 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:30.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5db4b640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c581f4d70 0x7f5c5815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5eb4d640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5814b310 0x7f5c5814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 -- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c581f4d70 msgr2=0x7f5c5815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c581f4d70 0x7f5c5815b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 -- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5814b310 msgr2=0x7f5c5814b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5814b310 0x7f5c5814b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 -- 172.21.15.125:0/3367117333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c581f5510 con 0x7f5c440047c0 2023-10-18T10:44:30.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5eb4d640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5814b310 0x7f5c5814b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:30.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c5e34c640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 0x7f5c581f21c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f5c54011d40 tx=0x7f5c5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:30.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c4effd640 1 -- 172.21.15.125:0/3367117333 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54003d80 con 0x7f5c440047c0 2023-10-18T10:44:30.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c4effd640 1 -- 172.21.15.125:0/3367117333 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c54004b20 con 0x7f5c440047c0 2023-10-18T10:44:30.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c581f57f0 con 0x7f5c440047c0 2023-10-18T10:44:30.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c4effd640 1 -- 172.21.15.125:0/3367117333 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54004050 con 0x7f5c440047c0 2023-10-18T10:44:30.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.627+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5815bbc0 con 0x7f5c440047c0 2023-10-18T10:44:30.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.628+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5808acf0 con 0x7f5c440047c0 2023-10-18T10:44:30.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.629+0000 7f5c4effd640 1 -- 172.21.15.125:0/3367117333 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c540041b0 con 0x7f5c440047c0 2023-10-18T10:44:30.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.629+0000 7f5c4effd640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5c3c06af60 0x7f5c3c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:30.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.629+0000 7f5c4effd640 1 -- 172.21.15.125:0/3367117333 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f5c5408ad40 con 0x7f5c440047c0 2023-10-18T10:44:30.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.629+0000 7f5c5db4b640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5c3c06af60 0x7f5c3c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:30.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.630+0000 7f5c5db4b640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5c3c06af60 0x7f5c3c06d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c48034000 tx=0x7f5c4802e000 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:30.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.633+0000 7f5c4effd640 1 -- 172.21.15.125:0/3367117333 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c54055a80 con 0x7f5c440047c0 2023-10-18T10:44:30.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.835+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5c580690e0 con 0x7f5c440047c0 2023-10-18T10:44:30.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.835+0000 7f5c4effd640 1 -- 172.21.15.125:0/3367117333 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5c540590a0 con 0x7f5c440047c0 2023-10-18T10:44:30.838 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:44:30.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.837+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5c3c06af60 msgr2=0x7f5c3c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.837+0000 7f5c605b0640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5c3c06af60 0x7f5c3c06d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c48034000 tx=0x7f5c4802e000 comp rx=0 tx=0).stop 2023-10-18T10:44:30.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.837+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 msgr2=0x7f5c581f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:30.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.837+0000 7f5c605b0640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 0x7f5c581f21c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f5c54011d40 tx=0x7f5c5400f790 comp rx=0 tx=0).stop 2023-10-18T10:44:30.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.838+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 shutdown_connections 2023-10-18T10:44:30.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.838+0000 7f5c605b0640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f5c3c06af60 0x7f5c3c06d420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.838+0000 7f5c605b0640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5814b310 0x7f5c5814b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.838+0000 7f5c605b0640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c581f4d70 0x7f5c5815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.838+0000 7f5c605b0640 1 --2- 172.21.15.125:0/3367117333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c440047c0 0x7f5c581f21c0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:30.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.838+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 >> 172.21.15.125:0/3367117333 conn(0x7f5c580d0d60 msgr2=0x7f5c580ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:30.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.838+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 shutdown_connections 2023-10-18T10:44:30.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:30.838+0000 7f5c605b0640 1 -- 172.21.15.125:0/3367117333 wait complete. 2023-10-18T10:44:30.850 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:44:30.850 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:44:30.851 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:44:30.851 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:44:30.851 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:44:30.852 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:44:30.852 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:44:30.852 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:44:30.853 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:44:30.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:30.853 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:44:30.869 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:44:30.869 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:44:30.869 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:44:30.870 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:44:30.870 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:44:30.870 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:44:30.870 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:44:30.870 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:44:30.871 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:44:30.871 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:44:30.871 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:44:30.871 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:44:30.871 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:44:30.872 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:44:30.872 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:44:30.872 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:44:30.872 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:44:30.872 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 41553sec preferred_lft 41553sec 2023-10-18T10:44:30.873 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:44:30.873 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:44:30.873 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:44:30.873 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:44:30.873 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:44:30.873 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:44:30.874 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T10:44:30.874 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:44:30.874 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:44:30.874 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:44:30.875 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T10:44:30.875 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:44:30.875 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:44:30.875 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:44:30.875 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T10:44:30.876 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:44:30.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:30.876 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:44:30.935 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:44:30.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:30.937 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:44:30.992 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:44:30.992 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:44:30.992 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:44:30.993 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:44:30.993 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:44:30.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:30.994 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:44:30.994 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:44:30.994 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:44:30.995 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:44:30.995 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:44:30.996 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:44:31.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:31.087 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:44:31.102 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:44:31.102 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:44:31.103 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:44:31.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:31.104 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:44:31.169 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:44:31.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:44:31.170 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:44:31.224 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:44:31.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:44:31.225 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:44:31.279 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:44:31.345 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:44:31.401 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:44:31.401 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:44:31.401 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:44:31.402 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:44:31.402 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:44:31.402 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:44:31.402 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:44:31.402 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:44:31.403 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:44:31.403 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:44:31.403 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:44:31.403 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:44:31.403 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:44:31.404 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:44:31.404 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:44:31.404 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:44:31.404 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:44:31.404 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:44:31.405 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:44:31.405 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:44:31.405 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:44:31.405 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:44:31.405 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:44:31.405 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:44:31.406 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:44:31.406 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:44:31.406 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:44:31.406 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:44:31.406 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:44:31.407 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:44:31.407 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:44:31.407 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:44:31.407 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:44:31.407 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:44:31.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:31.408 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:44:31.456 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:44:31.456 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:44:31.499 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:44:31.525 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:44:31.581 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:44:31.581 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:44:31.581 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:44:31.582 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:44:31.582 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:44:31.582 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:44:31.582 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:44:31.582 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:44:31.583 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:44:31.583 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:44:31.583 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:44:31.583 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:44:31.583 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:44:31.584 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:44:31.584 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:44:31.584 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:44:31.584 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:44:31.584 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:44:31.585 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:44:31.585 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:44:31.585 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:44:31.585 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:44:31.585 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:44:31.585 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:44:31.586 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:44:31.586 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:44:31.586 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:44:31.586 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:44:31.586 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:44:31.587 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:44:31.587 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:44:31.587 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:44:31.587 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:44:31.587 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:44:31.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:31.588 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:44:31.589 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:44:31.581+0000 7f5377248480 -1 init, newargv = 0x55857ab659c0 newargc=15 2023-10-18T10:44:31.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[133056]: starting ceph client 2023-10-18T10:44:31.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[133056]: starting fuse 2023-10-18T10:44:31.603 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:44:31.604 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:44:31.604 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:44:31.605 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:44:31.605 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:44:31.605 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:44:31.606 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:44:31.606 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:44:31.606 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:44:31.607 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:44:31.607 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:44:31.608 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:44:31.608 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:44:31.608 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:44:31.608 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:44:31.609 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:44:31.609 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:44:31.610 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:44:31.610 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:44:31.610 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:44:31.611 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:44:31.611 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:44:31.611 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:44:31.612 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:44:31.612 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:44:31.612 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:44:31.613 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:44:31.613 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:44:31.613 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:44:31.722 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.133056.asok 2023-10-18T10:44:31.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:44:31.732 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.133056.asok 2023-10-18T10:44:31.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:31.733 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.133056.asok status 2023-10-18T10:44:31.866 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:44:31.866 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:44:31.866 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:44:31.867 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:44:31.867 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:44:31.867 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:44:31.867 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:44:31.867 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "133056", 2023-10-18T10:44:31.868 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:44:31.868 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:31.868 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:44:31.868 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:44:31.868 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 9354, 2023-10-18T10:44:31.868 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:44:31.869 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:44:31.869 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:44:31.869 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 9354 2023-10-18T10:44:31.869 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:31.869 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:44:31.870 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:44:31.870 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:44:31.870 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 4126745678 2023-10-18T10:44:31.870 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:31.870 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:31.871 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:44:31.871 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:44:31.871 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:44:31.871 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 4126745678 2023-10-18T10:44:31.871 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:31.872 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.9354 192.168.0.1:0/4126745678", 2023-10-18T10:44:31.872 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/4126745678", 2023-10-18T10:44:31.872 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:44:31.872 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 53, 2023-10-18T10:44:31.872 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 143, 2023-10-18T10:44:31.872 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:44:31.873 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:44:31.873 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:44:31.873 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:44:31.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:31.875 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:44:31.875 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:44:31.890 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:44:31.890 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:44:31.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:31.891 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:44:31.958 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T10:44:32.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 -- 172.21.15.125:0/2456485130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 msgr2=0x7efdb415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:32.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/2456485130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 0x7efdb415a9f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7efdb40cbd90 tx=0x7efdb002c180 comp rx=0 tx=0).stop 2023-10-18T10:44:32.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 -- 172.21.15.125:0/2456485130 shutdown_connections 2023-10-18T10:44:32.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/2456485130 >> v2:172.21.15.139:3300/0 conn(0x7efdb40c9d00 0x7efdb40ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/2456485130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb4140660 0x7efdb40c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/2456485130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 0x7efdb415a9f0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 -- 172.21.15.125:0/2456485130 >> 172.21.15.125:0/2456485130 conn(0x7efdb4069930 msgr2=0x7efdb4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:32.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 -- 172.21.15.125:0/2456485130 shutdown_connections 2023-10-18T10:44:32.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.099+0000 7efdbc1e2640 1 -- 172.21.15.125:0/2456485130 wait complete. 2023-10-18T10:44:32.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.100+0000 7efdbc1e2640 1 Processor -- start 2023-10-18T10:44:32.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.100+0000 7efdbc1e2640 1 -- start start 2023-10-18T10:44:32.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.100+0000 7efdbc1e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb40c9d00 0x7efdb4134db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:32.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.100+0000 7efdbc1e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 0x7efdb4156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:32.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.100+0000 7efdbc1e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb4140660 0x7efdb40aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:32.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.100+0000 7efdbc1e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40ad550 con 0x7efdb413d360 2023-10-18T10:44:32.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.100+0000 7efdbc1e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40ad830 con 0x7efdb4140660 2023-10-18T10:44:32.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.100+0000 7efdbc1e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40adb10 con 0x7efdb40c9d00 2023-10-18T10:44:32.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 0x7efdb4156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:32.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 0x7efdb4156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34048/0 (socket says 172.21.15.125:34048) 2023-10-18T10:44:32.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 -- 172.21.15.125:0/3456906307 learned_addr learned my addr 172.21.15.125:0/3456906307 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:32.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdba77f640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb4140660 0x7efdb40aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:32.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb9f7e640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb40c9d00 0x7efdb4134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:32.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 -- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb40c9d00 msgr2=0x7efdb4134db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:32.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb40c9d00 0x7efdb4134db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 -- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb4140660 msgr2=0x7efdb40aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:32.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb4140660 0x7efdb40aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 -- 172.21.15.125:0/3456906307 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb0009df0 con 0x7efdb413d360 2023-10-18T10:44:32.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdb977d640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 0x7efdb4156600 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7efda400c970 tx=0x7efda400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:32.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdaa7fc640 1 -- 172.21.15.125:0/3456906307 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4007bb0 con 0x7efdb413d360 2023-10-18T10:44:32.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdaa7fc640 1 -- 172.21.15.125:0/3456906307 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efda4007e00 con 0x7efdb413d360 2023-10-18T10:44:32.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb40b0780 con 0x7efdb413d360 2023-10-18T10:44:32.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdaa7fc640 1 -- 172.21.15.125:0/3456906307 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda400f6b0 con 0x7efdb413d360 2023-10-18T10:44:32.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.101+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb40c9050 con 0x7efdb413d360 2023-10-18T10:44:32.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.102+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb40b1fb0 con 0x7efdb413d360 2023-10-18T10:44:32.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.103+0000 7efdaa7fc640 1 -- 172.21.15.125:0/3456906307 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efda400f810 con 0x7efdb413d360 2023-10-18T10:44:32.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.103+0000 7efdaa7fc640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7efd9806af60 0x7efd9806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:32.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.103+0000 7efdaa7fc640 1 -- 172.21.15.125:0/3456906307 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7efda408a1f0 con 0x7efdb413d360 2023-10-18T10:44:32.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.103+0000 7efdb9f7e640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7efd9806af60 0x7efd9806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:32.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.104+0000 7efdb9f7e640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7efd9806af60 0x7efd9806d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efdb0034000 tx=0x7efdb0004730 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:32.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.107+0000 7efdaa7fc640 1 -- 172.21.15.125:0/3456906307 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efda4054fb0 con 0x7efdb413d360 2023-10-18T10:44:32.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.293+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 --> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efdb415b940 con 0x7efd9806af60 2023-10-18T10:44:32.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.296+0000 7efdaa7fc640 1 -- 172.21.15.125:0/3456906307 <== mgr.7786 v2:172.21.15.139:6832/4144373589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7efdb415b940 con 0x7efd9806af60 2023-10-18T10:44:32.297 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:44:32.297 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:32.297 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T10:44:32.297 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:32.298 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:44:32.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7efd9806af60 msgr2=0x7efd9806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:32.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7efd9806af60 0x7efd9806d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efdb0034000 tx=0x7efdb0004730 comp rx=0 tx=0).stop 2023-10-18T10:44:32.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 msgr2=0x7efdb4156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:32.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 0x7efdb4156600 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7efda400c970 tx=0x7efda400ce30 comp rx=0 tx=0).stop 2023-10-18T10:44:32.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 shutdown_connections 2023-10-18T10:44:32.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7efd9806af60 0x7efd9806d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb4140660 0x7efdb40aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb413d360 0x7efdb4156600 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 --2- 172.21.15.125:0/3456906307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efdb40c9d00 0x7efdb4134db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 >> 172.21.15.125:0/3456906307 conn(0x7efdb4069930 msgr2=0x7efdb4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:32.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 shutdown_connections 2023-10-18T10:44:32.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.297+0000 7efdbc1e2640 1 -- 172.21.15.125:0/3456906307 wait complete. 2023-10-18T10:44:32.309 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T10:44:32.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.410+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3207115362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee00cf340 msgr2=0x7f2ee00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:32.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.410+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3207115362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee00cf340 0x7f2ee00d1910 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009830 tx=0x7f2ed002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.410+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3207115362 shutdown_connections 2023-10-18T10:44:32.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.410+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3207115362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ee015b2a0 0x7f2ee00cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.410+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3207115362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee00cf340 0x7f2ee00d1910 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.410+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3207115362 >> 172.21.15.125:0/3207115362 conn(0x7f2ee00d0d60 msgr2=0x7f2ee00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:32.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.410+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3207115362 shutdown_connections 2023-10-18T10:44:32.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.410+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3207115362 wait complete. 2023-10-18T10:44:32.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.411+0000 7f2ee66d4640 1 Processor -- start 2023-10-18T10:44:32.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.411+0000 7f2ee66d4640 1 -- start start 2023-10-18T10:44:32.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee66d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00ceb40 0x7f2ee00c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:32.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee66d4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ee00cf340 0x7f2ee0157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:32.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee66d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee015b2a0 0x7f2ee00ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:32.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee66d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee00b4770 con 0x7f2ee015b2a0 2023-10-18T10:44:32.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee66d4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee00b38e0 con 0x7f2ee00ceb40 2023-10-18T10:44:32.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee66d4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee00c8530 con 0x7f2ee00cf340 2023-10-18T10:44:32.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee4c71640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee015b2a0 0x7f2ee00ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:32.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee4c71640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee015b2a0 0x7f2ee00ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34068/0 (socket says 172.21.15.125:34068) 2023-10-18T10:44:32.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.412+0000 7f2ee4c71640 1 -- 172.21.15.125:0/3185492370 learned_addr learned my addr 172.21.15.125:0/3185492370 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:32.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2edffff640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00ceb40 0x7f2ee00c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:32.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2edf7fe640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ee00cf340 0x7f2ee0157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:32.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2ee4c71640 1 -- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ee00cf340 msgr2=0x7f2ee0157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:32.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2ee4c71640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ee00cf340 0x7f2ee0157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2ee4c71640 1 -- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00ceb40 msgr2=0x7f2ee00c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:32.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2ee4c71640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00ceb40 0x7f2ee00c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:32.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2ee4c71640 1 -- 172.21.15.125:0/3185492370 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0009490 con 0x7f2ee015b2a0 2023-10-18T10:44:32.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2edffff640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00ceb40 0x7f2ee00c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:32.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2edf7fe640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ee00cf340 0x7f2ee0157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:32.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2ee4c71640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee015b2a0 0x7f2ee00ca850 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8001560 tx=0x7f2ed800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2edcff9640 1 -- 172.21.15.125:0/3185492370 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8003da0 con 0x7f2ee015b2a0 2023-10-18T10:44:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2edcff9640 1 -- 172.21.15.125:0/3185492370 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ed8005070 con 0x7f2ee015b2a0 2023-10-18T10:44:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2edcff9640 1 -- 172.21.15.125:0/3185492370 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8005460 con 0x7f2ee015b2a0 2023-10-18T10:44:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee00b3af0 con 0x7f2ee015b2a0 2023-10-18T10:44:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.413+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee00b40b0 con 0x7f2ee015b2a0 2023-10-18T10:44:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.414+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee00b48b0 con 0x7f2ee015b2a0 2023-10-18T10:44:32.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.414+0000 7f2edcff9640 1 -- 172.21.15.125:0/3185492370 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 71) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ed80055c0 con 0x7f2ee015b2a0 2023-10-18T10:44:32.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.415+0000 7f2edcff9640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2ecc06af60 0x7f2ecc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:32.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.415+0000 7f2edcff9640 1 -- 172.21.15.125:0/3185492370 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f2ed80894a0 con 0x7f2ee015b2a0 2023-10-18T10:44:32.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.415+0000 7f2edffff640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2ecc06af60 0x7f2ecc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:32.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.415+0000 7f2edffff640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2ecc06af60 0x7f2ecc06d420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0036000 tx=0x7f2ed0004730 comp rx=0 tx=0).ready entity=mgr.7786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:32.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.419+0000 7f2edcff9640 1 -- 172.21.15.125:0/3185492370 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ee00b48b0 con 0x7f2ee015b2a0 2023-10-18T10:44:32.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:32.631+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2ee00b48b0 con 0x7f2ee015b2a0 2023-10-18T10:44:33.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.600+0000 7f2edcff9640 1 -- 172.21.15.125:0/3185492370 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v72) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2ee00b48b0 con 0x7f2ee015b2a0 2023-10-18T10:44:33.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2ecc06af60 msgr2=0x7f2ecc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:33.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2ecc06af60 0x7f2ecc06d420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0036000 tx=0x7f2ed0004730 comp rx=0 tx=0).stop 2023-10-18T10:44:33.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee015b2a0 msgr2=0x7f2ee00ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:33.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee015b2a0 0x7f2ee00ca850 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8001560 tx=0x7f2ed800cc70 comp rx=0 tx=0).stop 2023-10-18T10:44:33.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 shutdown_connections 2023-10-18T10:44:33.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f2ecc06af60 0x7f2ecc06d420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ee015b2a0 0x7f2ee00ca850 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ee00cf340 0x7f2ee0157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 --2- 172.21.15.125:0/3185492370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ee00ceb40 0x7f2ee00c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.603+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 >> 172.21.15.125:0/3185492370 conn(0x7f2ee00d0d60 msgr2=0x7f2ee015a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:33.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.604+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 shutdown_connections 2023-10-18T10:44:33.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.604+0000 7f2ee66d4640 1 -- 172.21.15.125:0/3185492370 wait complete. 2023-10-18T10:44:33.618 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T10:44:33.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.727+0000 7f0872f53640 1 -- 172.21.15.125:0/1656499561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c15b2a0 msgr2=0x7f086c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:33.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.727+0000 7f0872f53640 1 --2- 172.21.15.125:0/1656499561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c15b2a0 0x7f086c0cf720 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0858009830 tx=0x7f085802bcd0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.727+0000 7f0872f53640 1 -- 172.21.15.125:0/1656499561 shutdown_connections 2023-10-18T10:44:33.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.727+0000 7f0872f53640 1 --2- 172.21.15.125:0/1656499561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c15b2a0 0x7f086c0cf720 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.727+0000 7f0872f53640 1 --2- 172.21.15.125:0/1656499561 >> v2:172.21.15.139:3300/0 conn(0x7f086c0cf340 0x7f086c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.727+0000 7f0872f53640 1 -- 172.21.15.125:0/1656499561 >> 172.21.15.125:0/1656499561 conn(0x7f086c0d0d60 msgr2=0x7f086c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:33.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.727+0000 7f0872f53640 1 -- 172.21.15.125:0/1656499561 shutdown_connections 2023-10-18T10:44:33.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.727+0000 7f0872f53640 1 -- 172.21.15.125:0/1656499561 wait complete. 2023-10-18T10:44:33.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.728+0000 7f0872f53640 1 Processor -- start 2023-10-18T10:44:33.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.728+0000 7f0872f53640 1 -- start start 2023-10-18T10:44:33.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f0872f53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f086c0ceb40 0x7f086c15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:33.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f0872f53640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c0cf340 0x7f086c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:33.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f0872f53640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f086c15b2a0 0x7f086c1e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:33.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f0872f53640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0b3ab0 con 0x7f086c15b2a0 2023-10-18T10:44:33.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f0872f53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c159b60 con 0x7f086c0ceb40 2023-10-18T10:44:33.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f0872f53640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c1e51f0 con 0x7f086c0cf340 2023-10-18T10:44:33.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f08714f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f086c15b2a0 0x7f086c1e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:33.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f086bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c0cf340 0x7f086c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:33.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f0870cef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f086c0ceb40 0x7f086c15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:33.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f086bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c0cf340 0x7f086c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:41396/0 (socket says 172.21.15.125:41396) 2023-10-18T10:44:33.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f08714f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f086c15b2a0 0x7f086c1e4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34094/0 (socket says 172.21.15.125:34094) 2023-10-18T10:44:33.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f086bfff640 1 -- 172.21.15.125:0/2086718813 learned_addr learned my addr 172.21.15.125:0/2086718813 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:33.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f086bfff640 1 -- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f086c0ceb40 msgr2=0x7f086c15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:33.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f086bfff640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f086c0ceb40 0x7f086c15d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f086bfff640 1 -- 172.21.15.125:0/2086718813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f086c15b2a0 msgr2=0x7f086c1e4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:33.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f086bfff640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f086c15b2a0 0x7f086c1e4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.729+0000 7f086bfff640 1 -- 172.21.15.125:0/2086718813 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858009490 con 0x7f086c0cf340 2023-10-18T10:44:33.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.730+0000 7f086bfff640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c0cf340 0x7f086c159620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0858009950 tx=0x7f0858003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:33.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.730+0000 7f08697fa640 1 -- 172.21.15.125:0/2086718813 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858003e20 con 0x7f086c0cf340 2023-10-18T10:44:33.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.730+0000 7f08697fa640 1 -- 172.21.15.125:0/2086718813 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0858034440 con 0x7f086c0cf340 2023-10-18T10:44:33.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.730+0000 7f08697fa640 1 -- 172.21.15.125:0/2086718813 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858034850 con 0x7f086c0cf340 2023-10-18T10:44:33.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.730+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c15dd40 con 0x7f086c0cf340 2023-10-18T10:44:33.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.730+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c15e300 con 0x7f086c0cf340 2023-10-18T10:44:33.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.731+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086c0b3c60 con 0x7f086c0cf340 2023-10-18T10:44:33.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.731+0000 7f08697fa640 1 -- 172.21.15.125:0/2086718813 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 72) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f08580349b0 con 0x7f086c0cf340 2023-10-18T10:44:33.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.732+0000 7f08697fa640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f085006afe0 0x7f085006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:33.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.732+0000 7f0870cef640 1 -- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f085006afe0 msgr2=0x7f085006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4144373589 2023-10-18T10:44:33.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.732+0000 7f08697fa640 1 -- 172.21.15.125:0/2086718813 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f08580ac040 con 0x7f086c0cf340 2023-10-18T10:44:33.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.732+0000 7f0870cef640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f085006afe0 0x7f085006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:33.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.736+0000 7f08697fa640 1 -- 172.21.15.125:0/2086718813 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f086c0b3c60 con 0x7f086c0cf340 2023-10-18T10:44:33.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.925+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f086c0b3c60 con 0x7f086c0cf340 2023-10-18T10:44:33.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.930+0000 7f08697fa640 1 -- 172.21.15.125:0/2086718813 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v56)=0 v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7f086c0b3c60 con 0x7f086c0cf340 2023-10-18T10:44:33.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f085006afe0 msgr2=0x7f085006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:33.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f085006afe0 0x7f085006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c0cf340 msgr2=0x7f086c159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:33.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c0cf340 0x7f086c159620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0858009950 tx=0x7f0858003820 comp rx=0 tx=0).stop 2023-10-18T10:44:33.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 shutdown_connections 2023-10-18T10:44:33.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f085006afe0 0x7f085006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f086c15b2a0 0x7f086c1e4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f086c0cf340 0x7f086c159620 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 --2- 172.21.15.125:0/2086718813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f086c0ceb40 0x7f086c15d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:33.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 >> 172.21.15.125:0/2086718813 conn(0x7f086c0d0d60 msgr2=0x7f086c15a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:33.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 shutdown_connections 2023-10-18T10:44:33.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:33.932+0000 7f0872f53640 1 -- 172.21.15.125:0/2086718813 wait complete. 2023-10-18T10:44:33.945 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T10:44:33.987 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:33.985+0000 7f54cb41a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:44:34.013 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.011+0000 7f8b6e63f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:44:34.031 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.030+0000 7f54cb41a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:44:34.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.051+0000 7fa032ce1640 1 -- 172.21.15.125:0/517918813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 msgr2=0x7fa02c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:34.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.051+0000 7fa032ce1640 1 --2- 172.21.15.125:0/517918813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 0x7fa02c0b0650 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fa024005fd0 tx=0x7fa02402c0e0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.051+0000 7fa032ce1640 1 -- 172.21.15.125:0/517918813 shutdown_connections 2023-10-18T10:44:34.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.051+0000 7fa032ce1640 1 --2- 172.21.15.125:0/517918813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 0x7fa02c0b0650 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.051+0000 7fa032ce1640 1 -- 172.21.15.125:0/517918813 >> 172.21.15.125:0/517918813 conn(0x7fa02c0d0d60 msgr2=0x7fa02c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:34.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.051+0000 7fa032ce1640 1 -- 172.21.15.125:0/517918813 shutdown_connections 2023-10-18T10:44:34.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.051+0000 7fa032ce1640 1 -- 172.21.15.125:0/517918813 wait complete. 2023-10-18T10:44:34.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.052+0000 7fa032ce1640 1 Processor -- start 2023-10-18T10:44:34.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.052+0000 7fa032ce1640 1 -- start start 2023-10-18T10:44:34.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa032ce1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02c0cf340 0x7fa02c147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:34.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa032ce1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 0x7fa02c14dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:34.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa032ce1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa02c15b2a0 0x7fa02c149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:34.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa032ce1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c14e220 con 0x7fa02c0d1910 2023-10-18T10:44:34.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa032ce1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c15bac0 con 0x7fa02c0cf340 2023-10-18T10:44:34.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa032ce1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c1e6670 con 0x7fa02c15b2a0 2023-10-18T10:44:34.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa02bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 0x7fa02c14dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:34.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa02bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 0x7fa02c14dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34110/0 (socket says 172.21.15.125:34110) 2023-10-18T10:44:34.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.053+0000 7fa02bfff640 1 -- 172.21.15.125:0/2124418929 learned_addr learned my addr 172.21.15.125:0/2124418929 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:34.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa030a7d640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02c0cf340 0x7fa02c147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:34.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa03127e640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa02c15b2a0 0x7fa02c149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:34.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa02bfff640 1 -- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa02c15b2a0 msgr2=0x7fa02c149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:34.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa02bfff640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa02c15b2a0 0x7fa02c149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa02bfff640 1 -- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02c0cf340 msgr2=0x7fa02c147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:34.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa02bfff640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02c0cf340 0x7fa02c147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa02bfff640 1 -- 172.21.15.125:0/2124418929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024009840 con 0x7fa02c0d1910 2023-10-18T10:44:34.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa02bfff640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 0x7fa02c14dce0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c3a0 tx=0x7fa01800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:34.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa0297fa640 1 -- 172.21.15.125:0/2124418929 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018011790 con 0x7fa02c0d1910 2023-10-18T10:44:34.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa0297fa640 1 -- 172.21.15.125:0/2124418929 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa018011a00 con 0x7fa02c0d1910 2023-10-18T10:44:34.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02c15bd10 con 0x7fa02c0d1910 2023-10-18T10:44:34.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa0297fa640 1 -- 172.21.15.125:0/2124418929 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018011e10 con 0x7fa02c0d1910 2023-10-18T10:44:34.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.054+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02c15c2d0 con 0x7fa02c0d1910 2023-10-18T10:44:34.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.055+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02c15c4f0 con 0x7fa02c0d1910 2023-10-18T10:44:34.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.056+0000 7fa0297fa640 1 -- 172.21.15.125:0/2124418929 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 72) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa01801c460 con 0x7fa02c0d1910 2023-10-18T10:44:34.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.056+0000 7fa0297fa640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa01006afb0 0x7fa01006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:34.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.056+0000 7fa030a7d640 1 -- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa01006afb0 msgr2=0x7fa01006d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4144373589 2023-10-18T10:44:34.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.056+0000 7fa0297fa640 1 -- 172.21.15.125:0/2124418929 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa018089820 con 0x7fa02c0d1910 2023-10-18T10:44:34.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.056+0000 7fa030a7d640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa01006afb0 0x7fa01006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:34.064 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.060+0000 7f8b6e63f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:44:34.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.060+0000 7fa0297fa640 1 -- 172.21.15.125:0/2124418929 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0180547a0 con 0x7fa02c0d1910 2023-10-18T10:44:34.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.236+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fa02c0690e0 con 0x7fa02c0d1910 2023-10-18T10:44:34.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.239+0000 7fa0297fa640 1 -- 172.21.15.125:0/2124418929 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v57)=0 v57) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa018057dc0 con 0x7fa02c0d1910 2023-10-18T10:44:34.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa01006afb0 msgr2=0x7fa01006d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:34.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa01006afb0 0x7fa01006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 msgr2=0x7fa02c14dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:34.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 0x7fa02c14dce0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c3a0 tx=0x7fa01800c860 comp rx=0 tx=0).stop 2023-10-18T10:44:34.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 shutdown_connections 2023-10-18T10:44:34.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7fa01006afb0 0x7fa01006d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa02c15b2a0 0x7fa02c149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02c0d1910 0x7fa02c14dce0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 --2- 172.21.15.125:0/2124418929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02c0cf340 0x7fa02c147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 >> 172.21.15.125:0/2124418929 conn(0x7fa02c0d0d60 msgr2=0x7fa02c15a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:34.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 shutdown_connections 2023-10-18T10:44:34.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.241+0000 7fa032ce1640 1 -- 172.21.15.125:0/2124418929 wait complete. 2023-10-18T10:44:34.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:34.254 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/minutes) 2023-10-18T10:44:34.276 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/minutes 1m 2023-10-18T10:44:34.421 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.419+0000 7f54cb41a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:44:34.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.422+0000 7f3878db6640 1 -- 172.21.15.125:0/3871313371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f387415b2a0 msgr2=0x7f38740d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:34.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.422+0000 7f3878db6640 1 --2- 172.21.15.125:0/3871313371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f387415b2a0 0x7f38740d1910 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3864009830 tx=0x7f386402bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:34.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.422+0000 7f3878db6640 1 -- 172.21.15.125:0/3871313371 shutdown_connections 2023-10-18T10:44:34.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.422+0000 7f3878db6640 1 --2- 172.21.15.125:0/3871313371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f387415b2a0 0x7f38740d1910 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.422+0000 7f3878db6640 1 -- 172.21.15.125:0/3871313371 >> 172.21.15.125:0/3871313371 conn(0x7f38740d0d60 msgr2=0x7f38740d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:34.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.422+0000 7f3878db6640 1 -- 172.21.15.125:0/3871313371 shutdown_connections 2023-10-18T10:44:34.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.422+0000 7f3878db6640 1 -- 172.21.15.125:0/3871313371 wait complete. 2023-10-18T10:44:34.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.423+0000 7f3878db6640 1 Processor -- start 2023-10-18T10:44:34.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3878db6640 1 -- start start 2023-10-18T10:44:34.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3878db6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38740ceb40 0x7f387415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:34.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3878db6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38740cf340 0x7f38740ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:34.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3878db6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f387415b2a0 0x7f38741f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:34.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3878db6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740cadc0 con 0x7f38740ceb40 2023-10-18T10:44:34.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3878db6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c8770 con 0x7f387415b2a0 2023-10-18T10:44:34.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3878db6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c8a50 con 0x7f38740cf340 2023-10-18T10:44:34.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f387259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38740ceb40 0x7f387415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:34.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f387259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38740ceb40 0x7f387415c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34116/0 (socket says 172.21.15.125:34116) 2023-10-18T10:44:34.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f387259c640 1 -- 172.21.15.125:0/1489050556 learned_addr learned my addr 172.21.15.125:0/1489050556 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:34.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3872d9d640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f387415b2a0 0x7f38741f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:34.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.424+0000 7f3871d9b640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38740cf340 0x7f38740ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:34.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f387259c640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38740cf340 msgr2=0x7f38740ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:34.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f387259c640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38740cf340 0x7f38740ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f387259c640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f387415b2a0 msgr2=0x7f38741f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:34.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f387259c640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f387415b2a0 0x7f38741f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:34.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f387259c640 1 -- 172.21.15.125:0/1489050556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3864009490 con 0x7f38740ceb40 2023-10-18T10:44:34.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f387259c640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38740ceb40 0x7f387415c0e0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f386800c400 tx=0x7f386800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:34.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38680052d0 con 0x7f38740ceb40 2023-10-18T10:44:34.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3868007020 con 0x7f38740ceb40 2023-10-18T10:44:34.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868007470 con 0x7f38740ceb40 2023-10-18T10:44:34.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38740c8d30 con 0x7f38740ceb40 2023-10-18T10:44:34.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.425+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38740c92f0 con 0x7f38740ceb40 2023-10-18T10:44:34.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.426+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38740c9480 con 0x7f38740ceb40 2023-10-18T10:44:34.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.427+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 72) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f38680067d0 con 0x7f38740ceb40 2023-10-18T10:44:34.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.427+0000 7f3862ffd640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 0x7f385406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:34.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.427+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f3868088ea0 con 0x7f38740ceb40 2023-10-18T10:44:34.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.427+0000 7f3871d9b640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 msgr2=0x7f385406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4144373589 2023-10-18T10:44:34.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.427+0000 7f3871d9b640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 0x7f385406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:34.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.432+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3868053e20 con 0x7f38740ceb40 2023-10-18T10:44:34.466 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.463+0000 7f54cb41a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:44:34.473 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.472+0000 7f8b6e63f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:44:34.506 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.505+0000 7f54cb41a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:44:34.519 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.517+0000 7f8b6e63f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:44:34.557 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.556+0000 7f54cb41a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:44:34.563 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.561+0000 7f8b6e63f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:44:34.598 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.597+0000 7f54cb41a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:44:34.617 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.615+0000 7f8b6e63f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:44:34.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.627+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 --> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f38740690e0 con 0x7f385406afb0 2023-10-18T10:44:34.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.628+0000 7f3871d9b640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 msgr2=0x7f385406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4144373589 2023-10-18T10:44:34.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:34.628+0000 7f3871d9b640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 0x7f385406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:44:34.659 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.657+0000 7f8b6e63f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:44:34.697 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.696+0000 7f54cb41a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:44:34.749 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.748+0000 7f54cb41a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:34.757 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.755+0000 7f8b6e63f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:44:34.790 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.789+0000 7f54cb41a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:44:34.809 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.808+0000 7f8b6e63f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:34.838 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.836+0000 7f54cb41a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:44:34.851 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.849+0000 7f8b6e63f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:44:34.880 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:34.878+0000 7f54cb41a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:44:34.898 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.897+0000 7f8b6e63f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:44:34.941 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:34.939+0000 7f8b6e63f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:44:35.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:35.029+0000 7f3871d9b640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 msgr2=0x7f385406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4144373589 2023-10-18T10:44:35.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:35.029+0000 7f3871d9b640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 0x7f385406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:44:35.192 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.190+0000 7f54cb41a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:44:35.257 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.256+0000 7f8b6e63f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:44:35.320 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.318+0000 7f54cb41a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:44:35.388 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.386+0000 7f8b6e63f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:44:35.410 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.408+0000 7f54cb41a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:44:35.452 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.451+0000 7f54cb41a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:44:35.479 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.477+0000 7f8b6e63f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:44:35.522 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.520+0000 7f8b6e63f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:44:35.546 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.544+0000 7f54cb41a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:35.617 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.615+0000 7f8b6e63f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:35.636 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.635+0000 7f54cb41a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:44:35.706 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.704+0000 7f8b6e63f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:44:35.771 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.770+0000 7f54cb41a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:35.813 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.811+0000 7f54cb41a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:35.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:35.830+0000 7f3871d9b640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 msgr2=0x7f385406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4144373589 2023-10-18T10:44:35.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:35.830+0000 7f3871d9b640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 0x7f385406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:44:35.841 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.839+0000 7f8b6e63f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:35.882 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.881+0000 7f8b6e63f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:35.917 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:35.915+0000 7f54cb41a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:44:35.985 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:35.984+0000 7f8b6e63f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:44:36.445 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:44:36.448 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:44:36.529 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:36.527+0000 7f54cb41a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:44:36.601 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:36.599+0000 7f8b6e63f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:44:36.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:36.789+0000 7f54cb41a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:44:36.839 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:36.837+0000 7f54cb41a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:44:36.861 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:36.859+0000 7f8b6e63f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:44:36.909 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:36.907+0000 7f8b6e63f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:44:36.937 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:36.936+0000 7f54cb41a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:44:36.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:36.955+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 73) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f38680541f0 con 0x7f38740ceb40 2023-10-18T10:44:36.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:36.956+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 msgr2=0x7f385406d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:36.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:36.956+0000 7f3862ffd640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/4144373589,v1:172.21.15.139:6833/4144373589] conn(0x7f385406afb0 0x7f385406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:37.013 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:37.012+0000 7f8b6e63f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:44:37.051 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f544ac02640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.051 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f544ac02640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.051 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f5447bbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.051 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f5447bbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.052 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f5447bbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.052 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f5447bbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.052 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f5447bbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.052 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f544ac02640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.052 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f544ac02640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.053 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:37.049+0000 7f544ac02640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:37.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:37.960+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 74) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f38680075d0 con 0x7f38740ceb40 2023-10-18T10:44:37.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:37.961+0000 7f3862ffd640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f3854035850 0x7f38540753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:37.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:37.961+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f38740690e0 con 0x7f3854035850 2023-10-18T10:44:37.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:37.963+0000 7f3871d9b640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f3854035850 0x7f38540753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:37.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:37.966+0000 7f3871d9b640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f3854035850 0x7f38540753e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3864036000 tx=0x7f3864004730 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:38.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.019+0000 7f3862ffd640 1 -- 172.21.15.125:0/1489050556 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7f38740690e0 con 0x7f3854035850 2023-10-18T10:44:38.021 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1/minutes 2023-10-18T10:44:38.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.021+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f3854035850 msgr2=0x7f38540753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.021+0000 7f3878db6640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f3854035850 0x7f38540753e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3864036000 tx=0x7f3864004730 comp rx=0 tx=0).stop 2023-10-18T10:44:38.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.021+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38740ceb40 msgr2=0x7f387415c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.021+0000 7f3878db6640 1 --2- 172.21.15.125:0/1489050556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38740ceb40 0x7f387415c0e0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f386800c400 tx=0x7f386800c8c0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.021+0000 7f3872d9d640 1 -- 172.21.15.125:0/1489050556 reap_dead start 2023-10-18T10:44:38.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.021+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 shutdown_connections 2023-10-18T10:44:38.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.021+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 >> 172.21.15.125:0/1489050556 conn(0x7f38740d0d60 msgr2=0x7f38740ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:38.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.022+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 shutdown_connections 2023-10-18T10:44:38.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.022+0000 7f3878db6640 1 -- 172.21.15.125:0/1489050556 wait complete. 2023-10-18T10:44:38.035 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/minutes 2023-10-18T10:44:38.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:38.036 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/hourly) 2023-10-18T10:44:38.055 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/hourly 1h 2023-10-18T10:44:38.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.200+0000 7f0648094640 1 -- 172.21.15.125:0/658888975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 msgr2=0x7f06400c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.200+0000 7f0648094640 1 --2- 172.21.15.125:0/658888975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 0x7f06400c9d00 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f0638005fd0 tx=0x7f063802be10 comp rx=0 tx=0).stop 2023-10-18T10:44:38.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.200+0000 7f0648094640 1 -- 172.21.15.125:0/658888975 shutdown_connections 2023-10-18T10:44:38.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.200+0000 7f0648094640 1 --2- 172.21.15.125:0/658888975 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 0x7f06400c9d00 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.200+0000 7f0648094640 1 --2- 172.21.15.125:0/658888975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0640140660 0x7f06400c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.200+0000 7f0648094640 1 -- 172.21.15.125:0/658888975 >> 172.21.15.125:0/658888975 conn(0x7f0640069930 msgr2=0x7f0640069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:38.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.200+0000 7f0648094640 1 -- 172.21.15.125:0/658888975 shutdown_connections 2023-10-18T10:44:38.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.200+0000 7f0648094640 1 -- 172.21.15.125:0/658888975 wait complete. 2023-10-18T10:44:38.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.201+0000 7f0648094640 1 Processor -- start 2023-10-18T10:44:38.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.201+0000 7f0648094640 1 -- start start 2023-10-18T10:44:38.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0648094640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 0x7f06401eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:38.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0648094640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f064013d360 0x7f06401380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:38.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0648094640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0640140660 0x7f0640150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:38.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0648094640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401ef1b0 con 0x7f064006d170 2023-10-18T10:44:38.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0648094640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640139430 con 0x7f0640140660 2023-10-18T10:44:38.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0648094640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0640138b90 con 0x7f064013d360 2023-10-18T10:44:38.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 0x7f06401eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:38.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 0x7f06401eec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46102/0 (socket says 172.21.15.125:46102) 2023-10-18T10:44:38.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 -- 172.21.15.125:0/380492649 learned_addr learned my addr 172.21.15.125:0/380492649 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:38.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0646631640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0640140660 0x7f0640150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:38.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f064562f640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f064013d360 0x7f06401380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:38.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 -- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f064013d360 msgr2=0x7f06401380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f064013d360 0x7f06401380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 -- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0640140660 msgr2=0x7f0640150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0640140660 0x7f0640150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 -- 172.21.15.125:0/380492649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638009560 con 0x7f064006d170 2023-10-18T10:44:38.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0646631640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0640140660 0x7f0640150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:38.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.202+0000 7f0645e30640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 0x7f06401eec70 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c890 tx=0x7f063c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:38.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.203+0000 7f06367fc640 1 -- 172.21.15.125:0/380492649 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c003da0 con 0x7f064006d170 2023-10-18T10:44:38.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.203+0000 7f06367fc640 1 -- 172.21.15.125:0/380492649 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f063c005070 con 0x7f064006d170 2023-10-18T10:44:38.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.203+0000 7f06367fc640 1 -- 172.21.15.125:0/380492649 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c005460 con 0x7f064006d170 2023-10-18T10:44:38.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.203+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0640138e70 con 0x7f064006d170 2023-10-18T10:44:38.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.203+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0640139d20 con 0x7f064006d170 2023-10-18T10:44:38.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.204+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06400690e0 con 0x7f064006d170 2023-10-18T10:44:38.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.204+0000 7f06367fc640 1 -- 172.21.15.125:0/380492649 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 74) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f063c0055c0 con 0x7f064006d170 2023-10-18T10:44:38.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.204+0000 7f06367fc640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f062406ace0 0x7f062406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:38.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.205+0000 7f06367fc640 1 -- 172.21.15.125:0/380492649 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f063c089140 con 0x7f064006d170 2023-10-18T10:44:38.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.205+0000 7f064562f640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f062406ace0 0x7f062406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:38.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.205+0000 7f064562f640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f062406ace0 0x7f062406d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0630005fd0 tx=0x7f0630005850 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:38.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.209+0000 7f06367fc640 1 -- 172.21.15.125:0/380492649 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f063c054270 con 0x7f064006d170 2023-10-18T10:44:38.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.395+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/hourly", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f06400cb9f0 con 0x7f062406ace0 2023-10-18T10:44:38.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.446+0000 7f06367fc640 1 -- 172.21.15.125:0/380492649 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f06400cb9f0 con 0x7f062406ace0 2023-10-18T10:44:38.448 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1/hourly 2023-10-18T10:44:38.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f062406ace0 msgr2=0x7f062406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f062406ace0 0x7f062406d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0630005fd0 tx=0x7f0630005850 comp rx=0 tx=0).stop 2023-10-18T10:44:38.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 msgr2=0x7f06401eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 0x7f06401eec70 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c890 tx=0x7f063c00cd50 comp rx=0 tx=0).stop 2023-10-18T10:44:38.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 shutdown_connections 2023-10-18T10:44:38.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f062406ace0 0x7f062406d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0640140660 0x7f0640150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f064013d360 0x7f06401380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 --2- 172.21.15.125:0/380492649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f064006d170 0x7f06401eec70 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 >> 172.21.15.125:0/380492649 conn(0x7f0640069930 msgr2=0x7f064015a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:38.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 shutdown_connections 2023-10-18T10:44:38.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.448+0000 7f0648094640 1 -- 172.21.15.125:0/380492649 wait complete. 2023-10-18T10:44:38.462 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/hourly 2023-10-18T10:44:38.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:38.463 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/daily) 2023-10-18T10:44:38.479 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/daily 1d 2023-10-18T10:44:38.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.624+0000 7f1df2c75640 1 -- 172.21.15.125:0/1728303435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 msgr2=0x7f1ddc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.624+0000 7f1df2c75640 1 --2- 172.21.15.125:0/1728303435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 0x7f1ddc004c40 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8006fd0 tx=0x7f1ddc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.624+0000 7f1df1212640 1 -- 172.21.15.125:0/1728303435 reap_dead start 2023-10-18T10:44:38.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.624+0000 7f1df2c75640 1 -- 172.21.15.125:0/1728303435 shutdown_connections 2023-10-18T10:44:38.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.624+0000 7f1df2c75640 1 -- 172.21.15.125:0/1728303435 >> 172.21.15.125:0/1728303435 conn(0x7f1dec069930 msgr2=0x7f1dec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:38.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.624+0000 7f1df2c75640 1 -- 172.21.15.125:0/1728303435 shutdown_connections 2023-10-18T10:44:38.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.624+0000 7f1df2c75640 1 -- 172.21.15.125:0/1728303435 wait complete. 2023-10-18T10:44:38.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.625+0000 7f1df2c75640 1 Processor -- start 2023-10-18T10:44:38.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.625+0000 7f1df2c75640 1 -- start start 2023-10-18T10:44:38.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df2c75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 0x7f1dec146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:38.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df2c75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dec15eb90 0x7f1dec15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:38.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df2c75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dec15f010 0x7f1dec157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:38.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df2c75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec1449d0 con 0x7f1ddc0047c0 2023-10-18T10:44:38.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df2c75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec149ea0 con 0x7f1dec15eb90 2023-10-18T10:44:38.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df2c75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec143ae0 con 0x7f1dec15f010 2023-10-18T10:44:38.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df0a11640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 0x7f1dec146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:38.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df0a11640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 0x7f1dec146fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46130/0 (socket says 172.21.15.125:46130) 2023-10-18T10:44:38.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df0a11640 1 -- 172.21.15.125:0/1164578507 learned_addr learned my addr 172.21.15.125:0/1164578507 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df1212640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dec15f010 0x7f1dec157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1debfff640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dec15eb90 0x7f1dec15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df0a11640 1 -- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dec15f010 msgr2=0x7f1dec157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df0a11640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dec15f010 0x7f1dec157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df0a11640 1 -- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dec15eb90 msgr2=0x7f1dec15cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df0a11640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dec15eb90 0x7f1dec15cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.626+0000 7f1df0a11640 1 -- 172.21.15.125:0/1164578507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec143dc0 con 0x7f1ddc0047c0 2023-10-18T10:44:38.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.627+0000 7f1df0a11640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 0x7f1dec146fd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1de0011a80 tx=0x7f1de0011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:38.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.627+0000 7f1de97fa640 1 -- 172.21.15.125:0/1164578507 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de000fed0 con 0x7f1ddc0047c0 2023-10-18T10:44:38.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.627+0000 7f1de97fa640 1 -- 172.21.15.125:0/1164578507 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1de0012450 con 0x7f1ddc0047c0 2023-10-18T10:44:38.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.627+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dec1440a0 con 0x7f1ddc0047c0 2023-10-18T10:44:38.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.627+0000 7f1de97fa640 1 -- 172.21.15.125:0/1164578507 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0012800 con 0x7f1ddc0047c0 2023-10-18T10:44:38.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.627+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dec144660 con 0x7f1ddc0047c0 2023-10-18T10:44:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.628+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dec0690e0 con 0x7f1ddc0047c0 2023-10-18T10:44:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.628+0000 7f1de97fa640 1 -- 172.21.15.125:0/1164578507 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 74) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f1de0012960 con 0x7f1ddc0047c0 2023-10-18T10:44:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.628+0000 7f1de97fa640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1dd006ace0 0x7f1dd006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.629+0000 7f1de97fa640 1 -- 172.21.15.125:0/1164578507 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f1de008a5c0 con 0x7f1ddc0047c0 2023-10-18T10:44:38.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.629+0000 7f1debfff640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1dd006ace0 0x7f1dd006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:38.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.629+0000 7f1debfff640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1dd006ace0 0x7f1dd006d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8034000 tx=0x7f1dd8001a70 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:38.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.633+0000 7f1de97fa640 1 -- 172.21.15.125:0/1164578507 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1de00557a0 con 0x7f1ddc0047c0 2023-10-18T10:44:38.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.820+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/daily", "snap_schedule": "1d", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dec154af0 con 0x7f1dd006ace0 2023-10-18T10:44:38.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.873+0000 7f1de97fa640 1 -- 172.21.15.125:0/1164578507 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1dd8006050 con 0x7f1dd006ace0 2023-10-18T10:44:38.878 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1/daily 2023-10-18T10:44:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1dd006ace0 msgr2=0x7f1dd006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1dd006ace0 0x7f1dd006d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8034000 tx=0x7f1dd8001a70 comp rx=0 tx=0).stop 2023-10-18T10:44:38.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 msgr2=0x7f1dec146fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 0x7f1dec146fd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1de0011a80 tx=0x7f1de0011f40 comp rx=0 tx=0).stop 2023-10-18T10:44:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 shutdown_connections 2023-10-18T10:44:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1dd006ace0 0x7f1dd006d1a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dec15eb90 0x7f1dec15cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dec15f010 0x7f1dec157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 --2- 172.21.15.125:0/1164578507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ddc0047c0 0x7f1dec146fd0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 >> 172.21.15.125:0/1164578507 conn(0x7f1dec069930 msgr2=0x7f1dec0cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 shutdown_connections 2023-10-18T10:44:38.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:38.875+0000 7f1df2c75640 1 -- 172.21.15.125:0/1164578507 wait complete. 2023-10-18T10:44:38.888 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/daily 2023-10-18T10:44:38.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:38.889 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/weekly) 2023-10-18T10:44:38.905 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/weekly 1w 2023-10-18T10:44:39.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 -- 172.21.15.125:0/3095207292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 msgr2=0x7f38cc13db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3095207292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 0x7f38cc13db80 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f38cc06f390 tx=0x7f38bc02bda0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 -- 172.21.15.125:0/3095207292 shutdown_connections 2023-10-18T10:44:39.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3095207292 >> v2:172.21.15.139:3300/0 conn(0x7f38cc0c9750 0x7f38cc0c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3095207292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38cc13d350 0x7f38cc0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3095207292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 0x7f38cc13db80 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 -- 172.21.15.125:0/3095207292 >> 172.21.15.125:0/3095207292 conn(0x7f38cc069930 msgr2=0x7f38cc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:39.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 -- 172.21.15.125:0/3095207292 shutdown_connections 2023-10-18T10:44:39.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.050+0000 7f38d1f83640 1 -- 172.21.15.125:0/3095207292 wait complete. 2023-10-18T10:44:39.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.051+0000 7f38d1f83640 1 Processor -- start 2023-10-18T10:44:39.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.052+0000 7f38d1f83640 1 -- start start 2023-10-18T10:44:39.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.052+0000 7f38d1f83640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38cc0c9750 0x7f38cc1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.052+0000 7f38d1f83640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 0x7f38cc136260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.052+0000 7f38d1f83640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38cc0adfc0 0x7f38cc150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.052+0000 7f38d1f83640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1367a0 con 0x7f38cc0cb980 2023-10-18T10:44:39.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.052+0000 7f38d1f83640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1533f0 con 0x7f38cc0adfc0 2023-10-18T10:44:39.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.052+0000 7f38d1f83640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0ae840 con 0x7f38cc0c9750 2023-10-18T10:44:39.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.052+0000 7f38caffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 0x7f38cc136260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38caffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 0x7f38cc136260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46136/0 (socket says 172.21.15.125:46136) 2023-10-18T10:44:39.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38caffd640 1 -- 172.21.15.125:0/3957466696 learned_addr learned my addr 172.21.15.125:0/3957466696 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:39.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38cb7fe640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38cc0c9750 0x7f38cc1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38cbfff640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38cc0adfc0 0x7f38cc150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38caffd640 1 -- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38cc0c9750 msgr2=0x7f38cc1598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38caffd640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38cc0c9750 0x7f38cc1598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38caffd640 1 -- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38cc0adfc0 msgr2=0x7f38cc150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38caffd640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38cc0adfc0 0x7f38cc150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38caffd640 1 -- 172.21.15.125:0/3957466696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc009560 con 0x7f38cc0cb980 2023-10-18T10:44:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38caffd640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 0x7f38cc136260 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c970 tx=0x7f38c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38b3fff640 1 -- 172.21.15.125:0/3957466696 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0007bb0 con 0x7f38cc0cb980 2023-10-18T10:44:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38b3fff640 1 -- 172.21.15.125:0/3957466696 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f38c0007e20 con 0x7f38cc0cb980 2023-10-18T10:44:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc0aeb20 con 0x7f38cc0cb980 2023-10-18T10:44:39.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38b3fff640 1 -- 172.21.15.125:0/3957466696 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0005280 con 0x7f38cc0cb980 2023-10-18T10:44:39.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.053+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc0af0e0 con 0x7f38cc0cb980 2023-10-18T10:44:39.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.054+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc1535a0 con 0x7f38cc0cb980 2023-10-18T10:44:39.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.055+0000 7f38b3fff640 1 -- 172.21.15.125:0/3957466696 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 74) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f38c00053e0 con 0x7f38cc0cb980 2023-10-18T10:44:39.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.055+0000 7f38b3fff640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f38b806ad10 0x7f38b806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.055+0000 7f38b3fff640 1 -- 172.21.15.125:0/3957466696 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f38c0089a60 con 0x7f38cc0cb980 2023-10-18T10:44:39.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.055+0000 7f38cb7fe640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f38b806ad10 0x7f38b806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.056+0000 7f38cb7fe640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f38b806ad10 0x7f38b806d1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38bc035000 tx=0x7f38bc004730 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:39.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.059+0000 7f38b3fff640 1 -- 172.21.15.125:0/3957466696 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38cc1535a0 con 0x7f38cc0cb980 2023-10-18T10:44:39.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.243+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/weekly", "snap_schedule": "1w", "target": ["mon-mgr", ""]}) v1 -- 0x7f38cc15c1e0 con 0x7f38b806ad10 2023-10-18T10:44:39.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.295+0000 7f38b3fff640 1 -- 172.21.15.125:0/3957466696 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f38bc004380 con 0x7f38b806ad10 2023-10-18T10:44:39.298 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1/weekly 2023-10-18T10:44:39.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.297+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f38b806ad10 msgr2=0x7f38b806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.297+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f38b806ad10 0x7f38b806d1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38bc035000 tx=0x7f38bc004730 comp rx=0 tx=0).stop 2023-10-18T10:44:39.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.297+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 msgr2=0x7f38cc136260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.297+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 0x7f38cc136260 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c970 tx=0x7f38c000ce30 comp rx=0 tx=0).stop 2023-10-18T10:44:39.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.298+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 shutdown_connections 2023-10-18T10:44:39.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.298+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f38b806ad10 0x7f38b806d1d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.298+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38cc0adfc0 0x7f38cc150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.298+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38cc0cb980 0x7f38cc136260 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.298+0000 7f38d1f83640 1 --2- 172.21.15.125:0/3957466696 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38cc0c9750 0x7f38cc1598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.298+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 >> 172.21.15.125:0/3957466696 conn(0x7f38cc069930 msgr2=0x7f38cc156410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:39.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.298+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 shutdown_connections 2023-10-18T10:44:39.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.298+0000 7f38d1f83640 1 -- 172.21.15.125:0/3957466696 wait complete. 2023-10-18T10:44:39.310 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/weekly 2023-10-18T10:44:39.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:39.311 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/monthly) 2023-10-18T10:44:39.327 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/monthly 1M 2023-10-18T10:44:39.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.470+0000 7fead0ac9640 1 -- 172.21.15.125:0/3875668885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 msgr2=0x7feacc0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.470+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3875668885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 0x7feacc0c9b30 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7feac0009900 tx=0x7feac002be00 comp rx=0 tx=0).stop 2023-10-18T10:44:39.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.470+0000 7fead0ac9640 1 -- 172.21.15.125:0/3875668885 shutdown_connections 2023-10-18T10:44:39.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.470+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3875668885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 0x7feacc0c9b30 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.470+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3875668885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feacc06af00 0x7feacc06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.470+0000 7fead0ac9640 1 -- 172.21.15.125:0/3875668885 >> 172.21.15.125:0/3875668885 conn(0x7feacc069930 msgr2=0x7feacc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:39.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.470+0000 7fead0ac9640 1 -- 172.21.15.125:0/3875668885 shutdown_connections 2023-10-18T10:44:39.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.470+0000 7fead0ac9640 1 -- 172.21.15.125:0/3875668885 wait complete. 2023-10-18T10:44:39.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.472+0000 7fead0ac9640 1 Processor -- start 2023-10-18T10:44:39.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.472+0000 7fead0ac9640 1 -- start start 2023-10-18T10:44:39.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.472+0000 7fead0ac9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feacc06af00 0x7feacc13b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.472+0000 7fead0ac9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feacc0ab500 0x7feacc1f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.472+0000 7fead0ac9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 0x7feacc14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.472+0000 7fead0ac9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc1f2480 con 0x7feacc0c96b0 2023-10-18T10:44:39.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.472+0000 7fead0ac9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc070030 con 0x7feacc0ab500 2023-10-18T10:44:39.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.472+0000 7fead0ac9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc06f420 con 0x7feacc06af00 2023-10-18T10:44:39.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 0x7feacc14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 0x7feacc14f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46162/0 (socket says 172.21.15.125:46162) 2023-10-18T10:44:39.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 -- 172.21.15.125:0/3182581562 learned_addr learned my addr 172.21.15.125:0/3182581562 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:39.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feac9d9b640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feacc0ab500 0x7feacc1f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feaca59c640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feacc06af00 0x7feacc13b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 -- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feacc06af00 msgr2=0x7feacc13b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feacc06af00 0x7feacc13b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 -- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feacc0ab500 msgr2=0x7feacc1f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feacc0ab500 0x7feacc1f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 -- 172.21.15.125:0/3182581562 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0009560 con 0x7feacc0c96b0 2023-10-18T10:44:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feacad9d640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 0x7feacc14f9e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7feac400c710 tx=0x7feac400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feabaffd640 1 -- 172.21.15.125:0/3182581562 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4006d50 con 0x7feacc0c96b0 2023-10-18T10:44:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc06f700 con 0x7feacc0c96b0 2023-10-18T10:44:39.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feabaffd640 1 -- 172.21.15.125:0/3182581562 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feac4003970 con 0x7feacc0c96b0 2023-10-18T10:44:39.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7feabaffd640 1 -- 172.21.15.125:0/3182581562 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4003d80 con 0x7feacc0c96b0 2023-10-18T10:44:39.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.473+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc06fcc0 con 0x7feacc0c96b0 2023-10-18T10:44:39.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.474+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc0690e0 con 0x7feacc0c96b0 2023-10-18T10:44:39.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.475+0000 7feabaffd640 1 -- 172.21.15.125:0/3182581562 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 74) v1 ==== 89084+0+0 (secure 0 0 0) 0x7feac40050d0 con 0x7feacc0c96b0 2023-10-18T10:44:39.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.475+0000 7feabaffd640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7feaac06ad10 0x7feaac06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.475+0000 7feabaffd640 1 -- 172.21.15.125:0/3182581562 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7feac4089220 con 0x7feacc0c96b0 2023-10-18T10:44:39.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.475+0000 7feaca59c640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7feaac06ad10 0x7feaac06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.476+0000 7feaca59c640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7feaac06ad10 0x7feaac06d1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feabc009990 tx=0x7feabc009110 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:39.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.479+0000 7feabaffd640 1 -- 172.21.15.125:0/3182581562 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feac40542d0 con 0x7feacc0c96b0 2023-10-18T10:44:39.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.668+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/monthly", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7feacc159c50 con 0x7feaac06ad10 2023-10-18T10:44:39.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.722+0000 7feabaffd640 1 -- 172.21.15.125:0/3182581562 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7feabc0095e0 con 0x7feaac06ad10 2023-10-18T10:44:39.724 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1/monthly 2023-10-18T10:44:39.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7feaac06ad10 msgr2=0x7feaac06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7feaac06ad10 0x7feaac06d1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feabc009990 tx=0x7feabc009110 comp rx=0 tx=0).stop 2023-10-18T10:44:39.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 msgr2=0x7feacc14f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 0x7feacc14f9e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7feac400c710 tx=0x7feac400cb10 comp rx=0 tx=0).stop 2023-10-18T10:44:39.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 shutdown_connections 2023-10-18T10:44:39.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7feaac06ad10 0x7feaac06d1d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feacc0ab500 0x7feacc1f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feacc0c96b0 0x7feacc14f9e0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 --2- 172.21.15.125:0/3182581562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feacc06af00 0x7feacc13b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 >> 172.21.15.125:0/3182581562 conn(0x7feacc069930 msgr2=0x7feacc15a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:39.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 shutdown_connections 2023-10-18T10:44:39.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.724+0000 7fead0ac9640 1 -- 172.21.15.125:0/3182581562 wait complete. 2023-10-18T10:44:39.737 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/monthly 2023-10-18T10:44:39.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:39.738 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/yearly) 2023-10-18T10:44:39.754 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/yearly 1Y 2023-10-18T10:44:39.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.896+0000 7f9695810640 1 -- 172.21.15.125:0/1409939087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900cf340 msgr2=0x7f96900d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.896+0000 7f9695810640 1 --2- 172.21.15.125:0/1409939087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900cf340 0x7f96900d1910 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f969015b680 tx=0x7f968002c080 comp rx=0 tx=0).stop 2023-10-18T10:44:39.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.896+0000 7f9695810640 1 -- 172.21.15.125:0/1409939087 shutdown_connections 2023-10-18T10:44:39.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.896+0000 7f9695810640 1 --2- 172.21.15.125:0/1409939087 >> v2:172.21.15.139:3300/0 conn(0x7f96900ceb40 0x7f96900b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.896+0000 7f9695810640 1 --2- 172.21.15.125:0/1409939087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900cf340 0x7f96900d1910 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.896+0000 7f9695810640 1 -- 172.21.15.125:0/1409939087 >> 172.21.15.125:0/1409939087 conn(0x7f96900d0d60 msgr2=0x7f96900d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:39.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.896+0000 7f9695810640 1 -- 172.21.15.125:0/1409939087 shutdown_connections 2023-10-18T10:44:39.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.896+0000 7f9695810640 1 -- 172.21.15.125:0/1409939087 wait complete. 2023-10-18T10:44:39.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.897+0000 7f9695810640 1 Processor -- start 2023-10-18T10:44:39.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.897+0000 7f9695810640 1 -- start start 2023-10-18T10:44:39.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f9695810640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900ceb40 0x7f9690159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f9695810640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96900cf340 0x7f969015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f9695810640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f969015b2a0 0x7f969015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f9695810640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900b6730 con 0x7f96900ceb40 2023-10-18T10:44:39.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f9695810640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900b5ab0 con 0x7f969015b2a0 2023-10-18T10:44:39.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f9695810640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96900b5d90 con 0x7f96900cf340 2023-10-18T10:44:39.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900ceb40 0x7f9690159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900ceb40 0x7f9690159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46178/0 (socket says 172.21.15.125:46178) 2023-10-18T10:44:39.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968effd640 1 -- 172.21.15.125:0/1807504328 learned_addr learned my addr 172.21.15.125:0/1807504328 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:39.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968e7fc640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96900cf340 0x7f969015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968effd640 1 -- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96900cf340 msgr2=0x7f969015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968f7fe640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f969015b2a0 0x7f969015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968effd640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96900cf340 0x7f969015a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968effd640 1 -- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f969015b2a0 msgr2=0x7f969015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:39.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968effd640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f969015b2a0 0x7f969015a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:39.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.898+0000 7f968effd640 1 -- 172.21.15.125:0/1807504328 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680009840 con 0x7f96900ceb40 2023-10-18T10:44:39.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.899+0000 7f968effd640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900ceb40 0x7f9690159280 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9680036040 tx=0x7f9680004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:39.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.899+0000 7f9677fff640 1 -- 172.21.15.125:0/1807504328 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968003d430 con 0x7f96900ceb40 2023-10-18T10:44:39.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.899+0000 7f9677fff640 1 -- 172.21.15.125:0/1807504328 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f968003d680 con 0x7f96900ceb40 2023-10-18T10:44:39.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.899+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96900b6070 con 0x7f96900ceb40 2023-10-18T10:44:39.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.899+0000 7f9677fff640 1 -- 172.21.15.125:0/1807504328 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968003da70 con 0x7f96900ceb40 2023-10-18T10:44:39.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.899+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96900b3a70 con 0x7f96900ceb40 2023-10-18T10:44:39.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.900+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96900d04f0 con 0x7f96900ceb40 2023-10-18T10:44:39.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.900+0000 7f9677fff640 1 -- 172.21.15.125:0/1807504328 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 74) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f968003dbd0 con 0x7f96900ceb40 2023-10-18T10:44:39.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.901+0000 7f9677fff640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f967c06ace0 0x7f967c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:39.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.901+0000 7f9677fff640 1 -- 172.21.15.125:0/1807504328 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f96800ac520 con 0x7f96900ceb40 2023-10-18T10:44:39.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.901+0000 7f968e7fc640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f967c06ace0 0x7f967c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:39.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.901+0000 7f968e7fc640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f967c06ace0 0x7f967c06d1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f96901f4920 tx=0x7f9684009040 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:39.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.905+0000 7f9677fff640 1 -- 172.21.15.125:0/1807504328 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9680077650 con 0x7f96900ceb40 2023-10-18T10:44:39.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:39.967+0000 7f9677fff640 1 -- 172.21.15.125:0/1807504328 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 75) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f9680077a20 con 0x7f96900ceb40 2023-10-18T10:44:40.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.095+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/yearly", "snap_schedule": "1Y", "target": ["mon-mgr", ""]}) v1 -- 0x7f96900b0d00 con 0x7f967c06ace0 2023-10-18T10:44:40.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.149+0000 7f9677fff640 1 -- 172.21.15.125:0/1807504328 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f96900b0d00 con 0x7f967c06ace0 2023-10-18T10:44:40.152 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1/yearly 2023-10-18T10:44:40.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f967c06ace0 msgr2=0x7f967c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f967c06ace0 0x7f967c06d1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f96901f4920 tx=0x7f9684009040 comp rx=0 tx=0).stop 2023-10-18T10:44:40.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900ceb40 msgr2=0x7f9690159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900ceb40 0x7f9690159280 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9680036040 tx=0x7f9680004360 comp rx=0 tx=0).stop 2023-10-18T10:44:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 shutdown_connections 2023-10-18T10:44:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f967c06ace0 0x7f967c06d1a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f969015b2a0 0x7f969015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96900cf340 0x7f969015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 --2- 172.21.15.125:0/1807504328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96900ceb40 0x7f9690159280 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 >> 172.21.15.125:0/1807504328 conn(0x7f96900d0d60 msgr2=0x7f96900cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 shutdown_connections 2023-10-18T10:44:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.152+0000 7f9695810640 1 -- 172.21.15.125:0/1807504328 wait complete. 2023-10-18T10:44:40.164 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/yearly 2023-10-18T10:44:40.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:40.165 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/bad_period_spec) 2023-10-18T10:44:40.182 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1X 2023-10-18T10:44:40.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.324+0000 7f4144bd7640 1 -- 172.21.15.125:0/71973496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f414015ccb0 msgr2=0x7f41400c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.324+0000 7f4144bd7640 1 --2- 172.21.15.125:0/71973496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f414015ccb0 0x7f41400c9750 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f41400cc8a0 tx=0x7f413802bcf0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.324+0000 7f4144bd7640 1 -- 172.21.15.125:0/71973496 shutdown_connections 2023-10-18T10:44:40.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.324+0000 7f4144bd7640 1 --2- 172.21.15.125:0/71973496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f414015ccb0 0x7f41400c9750 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.324+0000 7f4144bd7640 1 --2- 172.21.15.125:0/71973496 >> v2:172.21.15.139:3300/0 conn(0x7f414006b410 0x7f414015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.324+0000 7f4144bd7640 1 -- 172.21.15.125:0/71973496 >> 172.21.15.125:0/71973496 conn(0x7f4140069930 msgr2=0x7f4140069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:40.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.324+0000 7f4144bd7640 1 -- 172.21.15.125:0/71973496 shutdown_connections 2023-10-18T10:44:40.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.324+0000 7f4144bd7640 1 -- 172.21.15.125:0/71973496 wait complete. 2023-10-18T10:44:40.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.325+0000 7f4144bd7640 1 Processor -- start 2023-10-18T10:44:40.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.325+0000 7f4144bd7640 1 -- start start 2023-10-18T10:44:40.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f4144bd7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f414006b030 0x7f414013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f4144bd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f414006b410 0x7f414006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f4144bd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f414015ccb0 0x7f41400aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f4144bd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4140071040 con 0x7f414015ccb0 2023-10-18T10:44:40.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f4144bd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400704c0 con 0x7f414006b410 2023-10-18T10:44:40.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f4144bd7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41400707a0 con 0x7f414006b030 2023-10-18T10:44:40.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f414015ccb0 0x7f41400aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f414015ccb0 0x7f41400aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46200/0 (socket says 172.21.15.125:46200) 2023-10-18T10:44:40.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f413ed9d640 1 -- 172.21.15.125:0/168641286 learned_addr learned my addr 172.21.15.125:0/168641286 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:40.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f413dd9b640 1 --2- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f414006b410 0x7f414006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.326+0000 7f413e59c640 1 --2- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f414006b030 0x7f414013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f413ed9d640 1 -- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f414006b030 msgr2=0x7f414013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f413ed9d640 1 --2- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f414006b030 0x7f414013a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f413ed9d640 1 -- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f414006b410 msgr2=0x7f414006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f413ed9d640 1 --2- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f414006b410 0x7f414006fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f413ed9d640 1 -- 172.21.15.125:0/168641286 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4138009490 con 0x7f414015ccb0 2023-10-18T10:44:40.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f413ed9d640 1 --2- 172.21.15.125:0/168641286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f414015ccb0 0x7f41400aa8c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f4138009920 tx=0x7f4138004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:40.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f412effd640 1 -- 172.21.15.125:0/168641286 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138003b00 con 0x7f414015ccb0 2023-10-18T10:44:40.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f412effd640 1 -- 172.21.15.125:0/168641286 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4138003d50 con 0x7f414015ccb0 2023-10-18T10:44:40.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f412effd640 1 -- 172.21.15.125:0/168641286 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138003ee0 con 0x7f414015ccb0 2023-10-18T10:44:40.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f4144bd7640 1 -- 172.21.15.125:0/168641286 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4140070a80 con 0x7f414015ccb0 2023-10-18T10:44:40.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.327+0000 7f4144bd7640 1 -- 172.21.15.125:0/168641286 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4140152990 con 0x7f414015ccb0 2023-10-18T10:44:40.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.328+0000 7f4144bd7640 1 -- 172.21.15.125:0/168641286 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41400ab4e0 con 0x7f414015ccb0 2023-10-18T10:44:40.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.328+0000 7f412effd640 1 -- 172.21.15.125:0/168641286 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 75) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f413803b040 con 0x7f414015ccb0 2023-10-18T10:44:40.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.329+0000 7f412effd640 1 --2- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f412006afb0 0x7f412006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.329+0000 7f412effd640 1 -- 172.21.15.125:0/168641286 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f41380abc80 con 0x7f414015ccb0 2023-10-18T10:44:40.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.329+0000 7f413e59c640 1 --2- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f412006afb0 0x7f412006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.329+0000 7f413e59c640 1 --2- 172.21.15.125:0/168641286 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f412006afb0 0x7f412006d470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4130004a30 tx=0x7f4130001ab0 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:40.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.334+0000 7f412effd640 1 -- 172.21.15.125:0/168641286 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41400ab4e0 con 0x7f414015ccb0 2023-10-18T10:44:40.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.520+0000 7f4144bd7640 1 -- 172.21.15.125:0/168641286 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1X", "target": ["mon-mgr", ""]}) v1 -- 0x7f4140152b20 con 0x7f412006afb0 2023-10-18T10:44:40.524 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:40.523+0000 7f5451c10640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-18T10:44:40.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.523+0000 7f412effd640 1 -- 172.21.15.125:0/168641286 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - multiplier should be one of h,d,w,M,Y) v1 ==== 74+0+0 (secure 0 0 0) 0x7f4140152b20 con 0x7f412006afb0 2023-10-18T10:44:40.525 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-18T10:44:40.529 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T10:44:40.531 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1MM 2023-10-18T10:44:40.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 -- 172.21.15.125:0/2673072837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b000c96b0 msgr2=0x7f1b000c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 --2- 172.21.15.125:0/2673072837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b000c96b0 0x7f1b000c9b60 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f1af0005fd0 tx=0x7f1af002bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:40.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 -- 172.21.15.125:0/2673072837 shutdown_connections 2023-10-18T10:44:40.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 --2- 172.21.15.125:0/2673072837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b000c96b0 0x7f1b000c9b60 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 --2- 172.21.15.125:0/2673072837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b000ab500 0x7f1b000ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 --2- 172.21.15.125:0/2673072837 >> v2:172.21.15.139:3300/0 conn(0x7f1b0006af00 0x7f1b0006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 -- 172.21.15.125:0/2673072837 >> 172.21.15.125:0/2673072837 conn(0x7f1b00069930 msgr2=0x7f1b00069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:40.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 -- 172.21.15.125:0/2673072837 shutdown_connections 2023-10-18T10:44:40.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.633+0000 7f1b07c39640 1 -- 172.21.15.125:0/2673072837 wait complete. 2023-10-18T10:44:40.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.634+0000 7f1b07c39640 1 Processor -- start 2023-10-18T10:44:40.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.634+0000 7f1b07c39640 1 -- start start 2023-10-18T10:44:40.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b07c39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b0006af00 0x7f1b00154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b07c39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b000ab500 0x7f1b001f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b07c39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b000c96b0 0x7f1b0013a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b07c39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00153f20 con 0x7f1b0006af00 2023-10-18T10:44:40.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b07c39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001f2370 con 0x7f1b000c96b0 2023-10-18T10:44:40.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b07c39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0013a900 con 0x7f1b000ab500 2023-10-18T10:44:40.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b059d5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b0006af00 0x7f1b00154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b059d5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b0006af00 0x7f1b00154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46222/0 (socket says 172.21.15.125:46222) 2023-10-18T10:44:40.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b059d5640 1 -- 172.21.15.125:0/2067896364 learned_addr learned my addr 172.21.15.125:0/2067896364 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:40.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b051d4640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b000ab500 0x7f1b001f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b059d5640 1 -- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b000ab500 msgr2=0x7f1b001f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b059d5640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b000ab500 0x7f1b001f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b059d5640 1 -- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b000c96b0 msgr2=0x7f1b0013a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b061d6640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b000c96b0 0x7f1b0013a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b059d5640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b000c96b0 0x7f1b0013a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b059d5640 1 -- 172.21.15.125:0/2067896364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af0009560 con 0x7f1b0006af00 2023-10-18T10:44:40.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b051d4640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b000ab500 0x7f1b001f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:40.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.635+0000 7f1b061d6640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b000c96b0 0x7f1b0013a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:40.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.636+0000 7f1b059d5640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b0006af00 0x7f1b00154c20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00cd10 tx=0x7f1afc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:40.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.636+0000 7f1af67fc640 1 -- 172.21.15.125:0/2067896364 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc004b20 con 0x7f1b0006af00 2023-10-18T10:44:40.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.636+0000 7f1af67fc640 1 -- 172.21.15.125:0/2067896364 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1afc004d70 con 0x7f1b0006af00 2023-10-18T10:44:40.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.636+0000 7f1b07c39640 1 -- 172.21.15.125:0/2067896364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b00155320 con 0x7f1b0006af00 2023-10-18T10:44:40.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.636+0000 7f1af67fc640 1 -- 172.21.15.125:0/2067896364 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc005280 con 0x7f1b0006af00 2023-10-18T10:44:40.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.636+0000 7f1b07c39640 1 -- 172.21.15.125:0/2067896364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b001538d0 con 0x7f1b0006af00 2023-10-18T10:44:40.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.637+0000 7f1b07c39640 1 -- 172.21.15.125:0/2067896364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b00155840 con 0x7f1b0006af00 2023-10-18T10:44:40.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.637+0000 7f1af67fc640 1 -- 172.21.15.125:0/2067896364 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 75) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f1afc0053e0 con 0x7f1b0006af00 2023-10-18T10:44:40.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.638+0000 7f1af67fc640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ae406afb0 0x7f1ae406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.638+0000 7f1af67fc640 1 -- 172.21.15.125:0/2067896364 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f1afc08a480 con 0x7f1b0006af00 2023-10-18T10:44:40.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.638+0000 7f1b051d4640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ae406afb0 0x7f1ae406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.638+0000 7f1b051d4640 1 --2- 172.21.15.125:0/2067896364 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ae406afb0 0x7f1ae406d470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1af0035000 tx=0x7f1af0004730 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:40.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.643+0000 7f1af67fc640 1 -- 172.21.15.125:0/2067896364 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1afc055050 con 0x7f1b0006af00 2023-10-18T10:44:40.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.831+0000 7f1b07c39640 1 -- 172.21.15.125:0/2067896364 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1MM", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b000abed0 con 0x7f1ae406afb0 2023-10-18T10:44:40.835 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:40.834+0000 7f5451c10640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-18T10:44:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.834+0000 7f1af67fc640 1 -- 172.21.15.125:0/2067896364 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1af0004330 con 0x7f1ae406afb0 2023-10-18T10:44:40.836 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-18T10:44:40.841 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T10:44:40.842 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1 2023-10-18T10:44:40.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.943+0000 7f571af24640 1 -- 172.21.15.125:0/1822396386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57140ceb40 msgr2=0x7f57140b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.943+0000 7f571af24640 1 --2- 172.21.15.125:0/1822396386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57140ceb40 0x7f57140b0650 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f571408a120 tx=0x7f570c02bcf0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.943+0000 7f571af24640 1 -- 172.21.15.125:0/1822396386 shutdown_connections 2023-10-18T10:44:40.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.943+0000 7f571af24640 1 --2- 172.21.15.125:0/1822396386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57140ceb40 0x7f57140b0650 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.943+0000 7f571af24640 1 --2- 172.21.15.125:0/1822396386 >> v2:172.21.15.139:3300/0 conn(0x7f571415b2a0 0x7f57140d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.943+0000 7f571af24640 1 -- 172.21.15.125:0/1822396386 >> 172.21.15.125:0/1822396386 conn(0x7f57140d0d60 msgr2=0x7f57140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:40.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.943+0000 7f571af24640 1 -- 172.21.15.125:0/1822396386 shutdown_connections 2023-10-18T10:44:40.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.943+0000 7f571af24640 1 -- 172.21.15.125:0/1822396386 wait complete. 2023-10-18T10:44:40.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.944+0000 7f571af24640 1 Processor -- start 2023-10-18T10:44:40.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.944+0000 7f571af24640 1 -- start start 2023-10-18T10:44:40.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f571af24640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57140ceb40 0x7f57140b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f571af24640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57140cf340 0x7f5714154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f571af24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f571415b2a0 0x7f571415df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f571af24640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57141f20b0 con 0x7f57140ceb40 2023-10-18T10:44:40.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f571af24640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140b4e70 con 0x7f571415b2a0 2023-10-18T10:44:40.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f571af24640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57141f14a0 con 0x7f57140cf340 2023-10-18T10:44:40.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f5718cc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57140ceb40 0x7f57140b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f5718cc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57140ceb40 0x7f57140b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46246/0 (socket says 172.21.15.125:46246) 2023-10-18T10:44:40.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.945+0000 7f5718cc0640 1 -- 172.21.15.125:0/3982596306 learned_addr learned my addr 172.21.15.125:0/3982596306 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:40.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f5713fff640 1 --2- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57140cf340 0x7f5714154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f57194c1640 1 --2- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f571415b2a0 0x7f571415df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f5718cc0640 1 -- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57140cf340 msgr2=0x7f5714154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f5718cc0640 1 --2- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57140cf340 0x7f5714154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f5718cc0640 1 -- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f571415b2a0 msgr2=0x7f571415df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:40.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f5718cc0640 1 --2- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f571415b2a0 0x7f571415df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:40.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f5718cc0640 1 -- 172.21.15.125:0/3982596306 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c009490 con 0x7f57140ceb40 2023-10-18T10:44:40.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f5718cc0640 1 --2- 172.21.15.125:0/3982596306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57140ceb40 0x7f57140b4930 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c430 tx=0x7f56fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:40.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f57117fa640 1 -- 172.21.15.125:0/3982596306 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc004230 con 0x7f57140ceb40 2023-10-18T10:44:40.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f57117fa640 1 -- 172.21.15.125:0/3982596306 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f56fc0044a0 con 0x7f57140ceb40 2023-10-18T10:44:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f571af24640 1 -- 172.21.15.125:0/3982596306 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57141f1780 con 0x7f57140ceb40 2023-10-18T10:44:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f57117fa640 1 -- 172.21.15.125:0/3982596306 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc003850 con 0x7f57140ceb40 2023-10-18T10:44:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.946+0000 7f571af24640 1 -- 172.21.15.125:0/3982596306 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57141f1d40 con 0x7f57140ceb40 2023-10-18T10:44:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.947+0000 7f571af24640 1 -- 172.21.15.125:0/3982596306 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57140690e0 con 0x7f57140ceb40 2023-10-18T10:44:40.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.947+0000 7f57117fa640 1 -- 172.21.15.125:0/3982596306 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 75) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f56fc003a30 con 0x7f57140ceb40 2023-10-18T10:44:40.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.948+0000 7f57117fa640 1 --2- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f56f406afb0 0x7f56f406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:40.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.948+0000 7f57117fa640 1 -- 172.21.15.125:0/3982596306 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f56fc0892a0 con 0x7f57140ceb40 2023-10-18T10:44:40.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.948+0000 7f5713fff640 1 --2- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f56f406afb0 0x7f56f406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:40.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.949+0000 7f5713fff640 1 --2- 172.21.15.125:0/3982596306 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f56f406afb0 0x7f56f406d470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f57040092f0 tx=0x7f5704009260 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:40.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:40.952+0000 7f57117fa640 1 -- 172.21.15.125:0/3982596306 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56fc057b60 con 0x7f57140ceb40 2023-10-18T10:44:41.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.139+0000 7f571af24640 1 -- 172.21.15.125:0/3982596306 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f57140cc170 con 0x7f56f406afb0 2023-10-18T10:44:41.143 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:41.141+0000 7f5451c10640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-18T10:44:41.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.141+0000 7f57117fa640 1 -- 172.21.15.125:0/3982596306 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5704009af0 con 0x7f56f406afb0 2023-10-18T10:44:41.143 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-18T10:44:41.148 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T10:44:41.148 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec M 2023-10-18T10:44:41.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.251+0000 7fd1f04a9640 1 -- 172.21.15.125:0/3449479645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1e815b2a0 msgr2=0x7fd1e80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.251+0000 7fd1f04a9640 1 --2- 172.21.15.125:0/3449479645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1e815b2a0 0x7fd1e80cf720 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fd1e808a810 tx=0x7fd1d802bcf0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.251+0000 7fd1f04a9640 1 -- 172.21.15.125:0/3449479645 shutdown_connections 2023-10-18T10:44:41.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.251+0000 7fd1f04a9640 1 --2- 172.21.15.125:0/3449479645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1e815b2a0 0x7fd1e80cf720 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.251+0000 7fd1f04a9640 1 --2- 172.21.15.125:0/3449479645 >> v2:172.21.15.139:3300/0 conn(0x7fd1e80cf340 0x7fd1e80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.251+0000 7fd1f04a9640 1 -- 172.21.15.125:0/3449479645 >> 172.21.15.125:0/3449479645 conn(0x7fd1e80d0d60 msgr2=0x7fd1e80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:41.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.251+0000 7fd1f04a9640 1 -- 172.21.15.125:0/3449479645 shutdown_connections 2023-10-18T10:44:41.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.251+0000 7fd1f04a9640 1 -- 172.21.15.125:0/3449479645 wait complete. 2023-10-18T10:44:41.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.252+0000 7fd1f04a9640 1 Processor -- start 2023-10-18T10:44:41.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.252+0000 7fd1f04a9640 1 -- start start 2023-10-18T10:44:41.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1f04a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80ceb40 0x7fd1e8156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1f04a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1e80cf340 0x7fd1e809e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1f04a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e815b2a0 0x7fd1e80b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1f04a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80ae330 con 0x7fd1e80cf340 2023-10-18T10:44:41.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1f04a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80ae580 con 0x7fd1e815b2a0 2023-10-18T10:44:41.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1f04a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80ae860 con 0x7fd1e80ceb40 2023-10-18T10:44:41.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1eda44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1e80cf340 0x7fd1e809e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1eda44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1e80cf340 0x7fd1e809e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46266/0 (socket says 172.21.15.125:46266) 2023-10-18T10:44:41.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1eda44640 1 -- 172.21.15.125:0/544978299 learned_addr learned my addr 172.21.15.125:0/544978299 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:41.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1eea46640 1 --2- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e815b2a0 0x7fd1e80b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1ee245640 1 --2- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80ceb40 0x7fd1e8156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.253+0000 7fd1eda44640 1 -- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80ceb40 msgr2=0x7fd1e8156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1eda44640 1 --2- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80ceb40 0x7fd1e8156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1eda44640 1 -- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e815b2a0 msgr2=0x7fd1e80b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1eda44640 1 --2- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e815b2a0 0x7fd1e80b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1eda44640 1 -- 172.21.15.125:0/544978299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d8009490 con 0x7fd1e80cf340 2023-10-18T10:44:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1eda44640 1 --2- 172.21.15.125:0/544978299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd1e80cf340 0x7fd1e809e2e0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009780 tx=0x7fd1d80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1deffd640 1 -- 172.21.15.125:0/544978299 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003ea0 con 0x7fd1e80cf340 2023-10-18T10:44:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1deffd640 1 -- 172.21.15.125:0/544978299 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd1d8005020 con 0x7fd1e80cf340 2023-10-18T10:44:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1f04a9640 1 -- 172.21.15.125:0/544978299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e80aeb40 con 0x7fd1e80cf340 2023-10-18T10:44:41.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1deffd640 1 -- 172.21.15.125:0/544978299 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d80053f0 con 0x7fd1e80cf340 2023-10-18T10:44:41.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.254+0000 7fd1f04a9640 1 -- 172.21.15.125:0/544978299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e81574e0 con 0x7fd1e80cf340 2023-10-18T10:44:41.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.255+0000 7fd1f04a9640 1 -- 172.21.15.125:0/544978299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e80690e0 con 0x7fd1e80cf340 2023-10-18T10:44:41.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.255+0000 7fd1deffd640 1 -- 172.21.15.125:0/544978299 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 75) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fd1d80047a0 con 0x7fd1e80cf340 2023-10-18T10:44:41.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.256+0000 7fd1deffd640 1 --2- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fd1cc06afb0 0x7fd1cc06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.256+0000 7fd1deffd640 1 -- 172.21.15.125:0/544978299 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fd1d80abf50 con 0x7fd1e80cf340 2023-10-18T10:44:41.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.256+0000 7fd1ee245640 1 --2- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fd1cc06afb0 0x7fd1cc06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.256+0000 7fd1ee245640 1 --2- 172.21.15.125:0/544978299 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fd1cc06afb0 0x7fd1cc06d470 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40058c0 tx=0x7fd1e400a4c0 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:41.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.260+0000 7fd1deffd640 1 -- 172.21.15.125:0/544978299 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1d8076c30 con 0x7fd1e80cf340 2023-10-18T10:44:41.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.440+0000 7fd1deffd640 1 -- 172.21.15.125:0/544978299 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fd1d80aa1b0 con 0x7fd1e80cf340 2023-10-18T10:44:41.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.448+0000 7fd1f04a9640 1 -- 172.21.15.125:0/544978299 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1e80cb980 con 0x7fd1cc06afb0 2023-10-18T10:44:41.452 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:41.450+0000 7f5451c10640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-18T10:44:41.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.451+0000 7fd1deffd640 1 -- 172.21.15.125:0/544978299 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd1e80cb980 con 0x7fd1cc06afb0 2023-10-18T10:44:41.453 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-18T10:44:41.457 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T10:44:41.458 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec -1m 2023-10-18T10:44:41.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.562+0000 7efd635f5640 1 -- 172.21.15.125:0/2311663234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd5c140660 msgr2=0x7efd5c0c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.562+0000 7efd635f5640 1 --2- 172.21.15.125:0/2311663234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd5c140660 0x7efd5c0c97c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7efd4c005fd0 tx=0x7efd4c02c610 comp rx=0 tx=0).stop 2023-10-18T10:44:41.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.562+0000 7efd635f5640 1 -- 172.21.15.125:0/2311663234 shutdown_connections 2023-10-18T10:44:41.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.562+0000 7efd635f5640 1 --2- 172.21.15.125:0/2311663234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd5c0c9d00 0x7efd5c06b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.562+0000 7efd635f5640 1 --2- 172.21.15.125:0/2311663234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd5c140660 0x7efd5c0c97c0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.562+0000 7efd635f5640 1 -- 172.21.15.125:0/2311663234 >> 172.21.15.125:0/2311663234 conn(0x7efd5c069930 msgr2=0x7efd5c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:41.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.562+0000 7efd635f5640 1 -- 172.21.15.125:0/2311663234 shutdown_connections 2023-10-18T10:44:41.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.562+0000 7efd635f5640 1 -- 172.21.15.125:0/2311663234 wait complete. 2023-10-18T10:44:41.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.563+0000 7efd635f5640 1 Processor -- start 2023-10-18T10:44:41.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.563+0000 7efd635f5640 1 -- start start 2023-10-18T10:44:41.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.563+0000 7efd635f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd5c0c9d00 0x7efd5c1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.563+0000 7efd635f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c13d360 0x7efd5c139820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.563+0000 7efd635f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd5c140660 0x7efd5c139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.563+0000 7efd635f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c1eefa0 con 0x7efd5c140660 2023-10-18T10:44:41.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.563+0000 7efd635f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c154c10 con 0x7efd5c13d360 2023-10-18T10:44:41.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.563+0000 7efd635f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c1383a0 con 0x7efd5c0c9d00 2023-10-18T10:44:41.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd5c140660 0x7efd5c139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd5c140660 0x7efd5c139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46288/0 (socket says 172.21.15.125:46288) 2023-10-18T10:44:41.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 -- 172.21.15.125:0/3448353096 learned_addr learned my addr 172.21.15.125:0/3448353096 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:41.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61391640 1 --2- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd5c0c9d00 0x7efd5c1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd60b90640 1 --2- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c13d360 0x7efd5c139820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 -- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd5c0c9d00 msgr2=0x7efd5c1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 --2- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd5c0c9d00 0x7efd5c1546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 -- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c13d360 msgr2=0x7efd5c139820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 --2- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c13d360 0x7efd5c139820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 -- 172.21.15.125:0/3448353096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c009df0 con 0x7efd5c140660 2023-10-18T10:44:41.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd61b92640 1 --2- 172.21.15.125:0/3448353096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd5c140660 0x7efd5c139d60 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7efd5400b970 tx=0x7efd5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:41.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd51ffb640 1 -- 172.21.15.125:0/3448353096 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400cb40 con 0x7efd5c140660 2023-10-18T10:44:41.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd51ffb640 1 -- 172.21.15.125:0/3448353096 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd5400cdb0 con 0x7efd5c140660 2023-10-18T10:44:41.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd635f5640 1 -- 172.21.15.125:0/3448353096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c137b00 con 0x7efd5c140660 2023-10-18T10:44:41.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd51ffb640 1 -- 172.21.15.125:0/3448353096 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd540146a0 con 0x7efd5c140660 2023-10-18T10:44:41.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.564+0000 7efd635f5640 1 -- 172.21.15.125:0/3448353096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c1380c0 con 0x7efd5c140660 2023-10-18T10:44:41.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.565+0000 7efd635f5640 1 -- 172.21.15.125:0/3448353096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c0690e0 con 0x7efd5c140660 2023-10-18T10:44:41.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.566+0000 7efd51ffb640 1 -- 172.21.15.125:0/3448353096 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7efd5401d3b0 con 0x7efd5c140660 2023-10-18T10:44:41.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.566+0000 7efd51ffb640 1 --2- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7efd4006b070 0x7efd4006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.566+0000 7efd51ffb640 1 -- 172.21.15.125:0/3448353096 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7efd54089d80 con 0x7efd5c140660 2023-10-18T10:44:41.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.566+0000 7efd61391640 1 --2- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7efd4006b070 0x7efd4006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.567+0000 7efd61391640 1 --2- 172.21.15.125:0/3448353096 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7efd4006b070 0x7efd4006d530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efd58007fd0 tx=0x7efd58009040 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:41.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.570+0000 7efd51ffb640 1 -- 172.21.15.125:0/3448353096 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd540548f0 con 0x7efd5c140660 2023-10-18T10:44:41.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.757+0000 7efd635f5640 1 -- 172.21.15.125:0/3448353096 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "-1m", "target": ["mon-mgr", ""]}) v1 -- 0x7efd5c06be50 con 0x7efd4006b070 2023-10-18T10:44:41.760 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:41.758+0000 7f5451c10640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-18T10:44:41.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.758+0000 7efd51ffb640 1 -- 172.21.15.125:0/3448353096 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc.) v1 ==== 96+0+0 (secure 0 0 0) 0x7efd58001c60 con 0x7efd4006b070 2023-10-18T10:44:41.761 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-18T10:44:41.765 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T10:44:41.766 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec '' 2023-10-18T10:44:41.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.869+0000 7efc36264640 1 -- 172.21.15.125:0/2046897402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc3013e520 msgr2=0x7efc3006bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.869+0000 7efc36264640 1 --2- 172.21.15.125:0/2046897402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc3013e520 0x7efc3006bb60 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7efc20005fd0 tx=0x7efc2002be20 comp rx=0 tx=0).stop 2023-10-18T10:44:41.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.869+0000 7efc36264640 1 -- 172.21.15.125:0/2046897402 shutdown_connections 2023-10-18T10:44:41.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.869+0000 7efc36264640 1 --2- 172.21.15.125:0/2046897402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc3013e520 0x7efc3006bb60 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.869+0000 7efc36264640 1 -- 172.21.15.125:0/2046897402 >> 172.21.15.125:0/2046897402 conn(0x7efc30069930 msgr2=0x7efc30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:41.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.869+0000 7efc36264640 1 -- 172.21.15.125:0/2046897402 shutdown_connections 2023-10-18T10:44:41.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.869+0000 7efc36264640 1 -- 172.21.15.125:0/2046897402 wait complete. 2023-10-18T10:44:41.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.870+0000 7efc36264640 1 Processor -- start 2023-10-18T10:44:41.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.870+0000 7efc36264640 1 -- start start 2023-10-18T10:44:41.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc36264640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc30070af0 0x7efc3013aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc36264640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc3013e520 0x7efc3013bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc36264640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc30141690 0x7efc3013d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc36264640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3013d1d0 con 0x7efc30141690 2023-10-18T10:44:41.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc36264640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3013c0d0 con 0x7efc3013e520 2023-10-18T10:44:41.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc36264640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3013c2e0 con 0x7efc30070af0 2023-10-18T10:44:41.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc34801640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc30141690 0x7efc3013d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc34801640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc30141690 0x7efc3013d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46312/0 (socket says 172.21.15.125:46312) 2023-10-18T10:44:41.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc34801640 1 -- 172.21.15.125:0/869203712 learned_addr learned my addr 172.21.15.125:0/869203712 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:41.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc2ffff640 1 --2- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc30070af0 0x7efc3013aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.871+0000 7efc2f7fe640 1 --2- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc3013e520 0x7efc3013bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc34801640 1 -- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc30070af0 msgr2=0x7efc3013aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc34801640 1 --2- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc30070af0 0x7efc3013aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc34801640 1 -- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc3013e520 msgr2=0x7efc3013bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:41.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc34801640 1 --2- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc3013e520 0x7efc3013bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:41.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc34801640 1 -- 172.21.15.125:0/869203712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20009560 con 0x7efc30141690 2023-10-18T10:44:41.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc34801640 1 --2- 172.21.15.125:0/869203712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc30141690 0x7efc3013d660 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7efc2800ea00 tx=0x7efc2800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:41.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc2cff9640 1 -- 172.21.15.125:0/869203712 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28003da0 con 0x7efc30141690 2023-10-18T10:44:41.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc2cff9640 1 -- 172.21.15.125:0/869203712 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc2800c970 con 0x7efc30141690 2023-10-18T10:44:41.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc36264640 1 -- 172.21.15.125:0/869203712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3013c5c0 con 0x7efc30141690 2023-10-18T10:44:41.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc2cff9640 1 -- 172.21.15.125:0/869203712 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800cd80 con 0x7efc30141690 2023-10-18T10:44:41.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.872+0000 7efc36264640 1 -- 172.21.15.125:0/869203712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3013cb80 con 0x7efc30141690 2023-10-18T10:44:41.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.873+0000 7efc36264640 1 -- 172.21.15.125:0/869203712 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc300b1bd0 con 0x7efc30141690 2023-10-18T10:44:41.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.873+0000 7efc2cff9640 1 -- 172.21.15.125:0/869203712 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7efc28005070 con 0x7efc30141690 2023-10-18T10:44:41.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.874+0000 7efc2cff9640 1 --2- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7efc1c06b040 0x7efc1c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:41.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.874+0000 7efc2cff9640 1 -- 172.21.15.125:0/869203712 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7efc2808c510 con 0x7efc30141690 2023-10-18T10:44:41.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.874+0000 7efc2ffff640 1 --2- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7efc1c06b040 0x7efc1c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:41.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.874+0000 7efc2ffff640 1 --2- 172.21.15.125:0/869203712 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7efc1c06b040 0x7efc1c06d500 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efc20035000 tx=0x7efc20004730 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:41.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:41.878+0000 7efc2cff9640 1 -- 172.21.15.125:0/869203712 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc28057160 con 0x7efc30141690 2023-10-18T10:44:42.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.063+0000 7efc36264640 1 -- 172.21.15.125:0/869203712 --> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "", "target": ["mon-mgr", ""]}) v1 -- 0x7efc300690e0 con 0x7efc1c06b040 2023-10-18T10:44:42.066 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:42.064+0000 7f5451c10640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-18T10:44:42.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.065+0000 7efc2cff9640 1 -- 172.21.15.125:0/869203712 <== mgr.7241 v2:172.21.15.139:6832/427428567 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7efc300690e0 con 0x7efc1c06b040 2023-10-18T10:44:42.067 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-18T10:44:42.071 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T10:44:42.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:42.072 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/minutes) 2023-10-18T10:44:42.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:42.090 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/hourly) 2023-10-18T10:44:42.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:42.148 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/daily) 2023-10-18T10:44:42.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:42.205 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/weekly) 2023-10-18T10:44:42.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:42.262 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/monthly) 2023-10-18T10:44:42.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:42.321 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/yearly) 2023-10-18T10:44:42.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:42.379 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/bad_period_spec) 2023-10-18T10:44:42.436 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T10:44:42.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.579+0000 7fce65525640 1 -- 172.21.15.125:0/2081484591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce6015b260 msgr2=0x7fce600cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:42.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.579+0000 7fce65525640 1 --2- 172.21.15.125:0/2081484591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce6015b260 0x7fce600cf310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce50009be0 tx=0x7fce5002c020 comp rx=0 tx=0).stop 2023-10-18T10:44:42.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.579+0000 7fce65525640 1 -- 172.21.15.125:0/2081484591 shutdown_connections 2023-10-18T10:44:42.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.579+0000 7fce65525640 1 --2- 172.21.15.125:0/2081484591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce6015b260 0x7fce600cf310 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:42.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.579+0000 7fce65525640 1 -- 172.21.15.125:0/2081484591 >> 172.21.15.125:0/2081484591 conn(0x7fce6009a030 msgr2=0x7fce600d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:42.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.579+0000 7fce65525640 1 -- 172.21.15.125:0/2081484591 shutdown_connections 2023-10-18T10:44:42.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.579+0000 7fce65525640 1 -- 172.21.15.125:0/2081484591 wait complete. 2023-10-18T10:44:42.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.580+0000 7fce65525640 1 Processor -- start 2023-10-18T10:44:42.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.580+0000 7fce65525640 1 -- start start 2023-10-18T10:44:42.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce65525640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce600cf850 0x7fce600c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:42.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce65525640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce6015b260 0x7fce600ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:42.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce65525640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6015b640 0x7fce6014d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:42.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce65525640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce600c97e0 con 0x7fce6015b260 2023-10-18T10:44:42.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce65525640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce600c8b60 con 0x7fce6015b640 2023-10-18T10:44:42.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce65525640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce600c8e40 con 0x7fce600cf850 2023-10-18T10:44:42.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce6015b260 0x7fce600ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:42.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce6015b260 0x7fce600ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46316/0 (socket says 172.21.15.125:46316) 2023-10-18T10:44:42.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5e7fc640 1 -- 172.21.15.125:0/1010388355 learned_addr learned my addr 172.21.15.125:0/1010388355 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:42.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5effd640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce600cf850 0x7fce600c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:42.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5f7fe640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6015b640 0x7fce6014d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:42.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5e7fc640 1 -- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce600cf850 msgr2=0x7fce600c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:42.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5e7fc640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce600cf850 0x7fce600c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:42.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5e7fc640 1 -- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6015b640 msgr2=0x7fce6014d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:42.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.581+0000 7fce5e7fc640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6015b640 0x7fce6014d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:42.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.582+0000 7fce5e7fc640 1 -- 172.21.15.125:0/1010388355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50009840 con 0x7fce6015b260 2023-10-18T10:44:42.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.582+0000 7fce5effd640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce600cf850 0x7fce600c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:42.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.582+0000 7fce5e7fc640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce6015b260 0x7fce600ca2f0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c700 tx=0x7fce5400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:42.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.582+0000 7fce477fe640 1 -- 172.21.15.125:0/1010388355 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5400a840 con 0x7fce6015b260 2023-10-18T10:44:42.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.582+0000 7fce477fe640 1 -- 172.21.15.125:0/1010388355 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce5400aa90 con 0x7fce6015b260 2023-10-18T10:44:42.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.582+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce600c9120 con 0x7fce6015b260 2023-10-18T10:44:42.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.582+0000 7fce477fe640 1 -- 172.21.15.125:0/1010388355 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5400aea0 con 0x7fce6015b260 2023-10-18T10:44:42.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.582+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce600c9c00 con 0x7fce6015b260 2023-10-18T10:44:42.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.583+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce600c9920 con 0x7fce6015b260 2023-10-18T10:44:42.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.583+0000 7fce477fe640 1 -- 172.21.15.125:0/1010388355 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 76) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fce5401c460 con 0x7fce6015b260 2023-10-18T10:44:42.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.584+0000 7fce477fe640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fce4c06b040 0x7fce4c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:42.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.584+0000 7fce477fe640 1 -- 172.21.15.125:0/1010388355 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fce54089f00 con 0x7fce6015b260 2023-10-18T10:44:42.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.584+0000 7fce5effd640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fce4c06b040 0x7fce4c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:42.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.584+0000 7fce5effd640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fce4c06b040 0x7fce4c06d500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fce50035000 tx=0x7fce50004ab0 comp rx=0 tx=0).ready entity=mgr.7241 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:42.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.588+0000 7fce477fe640 1 -- 172.21.15.125:0/1010388355 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce600c9920 con 0x7fce6015b260 2023-10-18T10:44:42.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:42.802+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fce600c9920 con 0x7fce6015b260 2023-10-18T10:44:43.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.444+0000 7fce477fe640 1 -- 172.21.15.125:0/1010388355 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v77) v1 ==== 93+0+0 (secure 0 0 0) 0x7fce600c9920 con 0x7fce6015b260 2023-10-18T10:44:43.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fce4c06b040 msgr2=0x7fce4c06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:43.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fce4c06b040 0x7fce4c06d500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fce50035000 tx=0x7fce50004ab0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce6015b260 msgr2=0x7fce600ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:43.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce477fe640 1 -- 172.21.15.125:0/1010388355 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fce54054ed0 con 0x7fce6015b260 2023-10-18T10:44:43.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce6015b260 0x7fce600ca2f0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c700 tx=0x7fce5400cbc0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 shutdown_connections 2023-10-18T10:44:43.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fce4c06b040 0x7fce4c06d500 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6015b640 0x7fce6014d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce6015b260 0x7fce600ca2f0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 --2- 172.21.15.125:0/1010388355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce600cf850 0x7fce600c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.448+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 >> 172.21.15.125:0/1010388355 conn(0x7fce6009a030 msgr2=0x7fce60159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:43.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.449+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 shutdown_connections 2023-10-18T10:44:43.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.449+0000 7fce65525640 1 -- 172.21.15.125:0/1010388355 wait complete. 2023-10-18T10:44:43.463 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:44:43.464 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:44:43.507 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:44:43.564 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:44:43.565 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:44:43.565 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:43.581 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.724+0000 7f229c6a9640 1 -- 172.21.15.125:0/1600203043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 msgr2=0x7f22940b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.724+0000 7f229c6a9640 1 --2- 172.21.15.125:0/1600203043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 0x7f22940b0650 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f228c005fd0 tx=0x7f228c02c240 comp rx=0 tx=0).stop 2023-10-18T10:44:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.725+0000 7f229c6a9640 1 -- 172.21.15.125:0/1600203043 shutdown_connections 2023-10-18T10:44:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.725+0000 7f229c6a9640 1 --2- 172.21.15.125:0/1600203043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 0x7f22940b0650 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.725+0000 7f229c6a9640 1 --2- 172.21.15.125:0/1600203043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22940cf340 0x7f22940d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.725+0000 7f229c6a9640 1 -- 172.21.15.125:0/1600203043 >> 172.21.15.125:0/1600203043 conn(0x7f22940d0d60 msgr2=0x7f22940d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.725+0000 7f229c6a9640 1 -- 172.21.15.125:0/1600203043 shutdown_connections 2023-10-18T10:44:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.725+0000 7f229c6a9640 1 -- 172.21.15.125:0/1600203043 wait complete. 2023-10-18T10:44:43.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.726+0000 7f229c6a9640 1 Processor -- start 2023-10-18T10:44:43.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.726+0000 7f229c6a9640 1 -- start start 2023-10-18T10:44:43.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.726+0000 7f229c6a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 0x7f2294158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:43.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.726+0000 7f229c6a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22940cf340 0x7f229409e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:43.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.726+0000 7f229c6a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f229415b2a0 0x7f22940b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:43.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.726+0000 7f229c6a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22940b49a0 con 0x7f229415b2a0 2023-10-18T10:44:43.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.726+0000 7f229c6a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22940b4c80 con 0x7f22940ceb40 2023-10-18T10:44:43.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.726+0000 7f229c6a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22940b4f60 con 0x7f22940cf340 2023-10-18T10:44:43.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f229415b2a0 0x7f22940b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:43.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f229415b2a0 0x7f22940b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46346/0 (socket says 172.21.15.125:46346) 2023-10-18T10:44:43.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f2299c44640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22940cf340 0x7f229409e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:43.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 -- 172.21.15.125:0/2373959432 learned_addr learned my addr 172.21.15.125:0/2373959432 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:43.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229a445640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 0x7f2294158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:43.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 -- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22940cf340 msgr2=0x7f229409e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:43.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22940cf340 0x7f229409e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 -- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 msgr2=0x7f2294158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:43.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 0x7f2294158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 -- 172.21.15.125:0/2373959432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290009490 con 0x7f229415b2a0 2023-10-18T10:44:43.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f2299c44640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22940cf340 0x7f229409e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:44:43.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229a445640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 0x7f2294158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:44:43.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229ac46640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f229415b2a0 0x7f22940b5d20 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f228c009bf0 tx=0x7f228c035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:43.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f228affd640 1 -- 172.21.15.125:0/2373959432 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c034440 con 0x7f229415b2a0 2023-10-18T10:44:43.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c009900 con 0x7f229415b2a0 2023-10-18T10:44:43.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f228affd640 1 -- 172.21.15.125:0/2373959432 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f228c003e60 con 0x7f229415b2a0 2023-10-18T10:44:43.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f228affd640 1 -- 172.21.15.125:0/2373959432 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c0044b0 con 0x7f229415b2a0 2023-10-18T10:44:43.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.727+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22940b5520 con 0x7f229415b2a0 2023-10-18T10:44:43.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.728+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22940690e0 con 0x7f229415b2a0 2023-10-18T10:44:43.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.729+0000 7f228affd640 1 -- 172.21.15.125:0/2373959432 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f228c00a040 con 0x7f229415b2a0 2023-10-18T10:44:43.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.729+0000 7f228affd640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f227806aff0 0x7f227806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:43.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.729+0000 7f229a445640 1 -- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f227806aff0 msgr2=0x7f227806d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:43.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.729+0000 7f228affd640 1 -- 172.21.15.125:0/2373959432 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f228c0ac940 con 0x7f229415b2a0 2023-10-18T10:44:43.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.729+0000 7f229a445640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f227806aff0 0x7f227806d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:43.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.734+0000 7f228affd640 1 -- 172.21.15.125:0/2373959432 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f228c0774e0 con 0x7f229415b2a0 2023-10-18T10:44:43.836 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:43.834+0000 7fa7014b0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:44:43.843 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:43.841+0000 7fa9a342d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:44:43.880 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:43.879+0000 7fa7014b0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:44:43.890 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:43.888+0000 7fa9a342d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:44:43.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.909+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22940d0b40 con 0x7f229415b2a0 2023-10-18T10:44:43.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.910+0000 7f228affd640 1 -- 172.21.15.125:0/2373959432 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+17383 (secure 0 0 0) 0x7f228c07ab00 con 0x7f229415b2a0 2023-10-18T10:44:43.913 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:43.913 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":144,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:36.950567+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:18.717169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:19.861005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3295054028":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1133703870":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/4009278606":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/494329198":"2023-10-19T10:44:36.950527+0000","172.21.15.139:6833/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:6832/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/337665570":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1274722331":"2023-10-19T10:44:36.950527+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:43.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f227806aff0 msgr2=0x7f227806d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:43.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f227806aff0 0x7f227806d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f229415b2a0 msgr2=0x7f22940b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:43.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f229415b2a0 0x7f22940b5d20 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f228c009bf0 tx=0x7f228c035e30 comp rx=0 tx=0).stop 2023-10-18T10:44:43.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 shutdown_connections 2023-10-18T10:44:43.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f227806aff0 0x7f227806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f229415b2a0 0x7f22940b5d20 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22940cf340 0x7f229409e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 --2- 172.21.15.125:0/2373959432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22940ceb40 0x7f2294158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:43.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 >> 172.21.15.125:0/2373959432 conn(0x7f22940d0d60 msgr2=0x7f229406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:43.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 shutdown_connections 2023-10-18T10:44:43.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:43.912+0000 7f229c6a9640 1 -- 172.21.15.125:0/2373959432 wait complete. 2023-10-18T10:44:43.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:44:18.717169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:44:19.861005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:44:43.926 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:44:44.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.033+0000 7fbc95875640 1 -- 172.21.15.125:0/4105885986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 msgr2=0x7fbc9006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.033+0000 7fbc95875640 1 --2- 172.21.15.125:0/4105885986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 0x7fbc9006d260 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fbc90071150 tx=0x7fbc7802bd30 comp rx=0 tx=0).stop 2023-10-18T10:44:44.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.033+0000 7fbc95875640 1 -- 172.21.15.125:0/4105885986 shutdown_connections 2023-10-18T10:44:44.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.033+0000 7fbc95875640 1 --2- 172.21.15.125:0/4105885986 >> v2:172.21.15.139:3300/0 conn(0x7fbc9015ccf0 0x7fbc9015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.033+0000 7fbc95875640 1 --2- 172.21.15.125:0/4105885986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 0x7fbc9006d260 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.033+0000 7fbc95875640 1 -- 172.21.15.125:0/4105885986 >> 172.21.15.125:0/4105885986 conn(0x7fbc900b0670 msgr2=0x7fbc900b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:44.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.033+0000 7fbc95875640 1 -- 172.21.15.125:0/4105885986 shutdown_connections 2023-10-18T10:44:44.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.033+0000 7fbc95875640 1 -- 172.21.15.125:0/4105885986 wait complete. 2023-10-18T10:44:44.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.034+0000 7fbc95875640 1 Processor -- start 2023-10-18T10:44:44.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.034+0000 7fbc95875640 1 -- start start 2023-10-18T10:44:44.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc95875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 0x7fbc900b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc95875640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc90159060 0x7fbc9014d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc95875640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9015ccf0 0x7fbc9006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc95875640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900b30a0 con 0x7fbc900ce240 2023-10-18T10:44:44.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc95875640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900b3380 con 0x7fbc90159060 2023-10-18T10:44:44.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc95875640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900b3660 con 0x7fbc9015ccf0 2023-10-18T10:44:44.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 0x7fbc900b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 0x7fbc900b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46362/0 (socket says 172.21.15.125:46362) 2023-10-18T10:44:44.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8effd640 1 -- 172.21.15.125:0/125336675 learned_addr learned my addr 172.21.15.125:0/125336675 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:44.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc87fff640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc90159060 0x7fbc9014d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8f7fe640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9015ccf0 0x7fbc9006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8effd640 1 -- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9015ccf0 msgr2=0x7fbc9006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8effd640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9015ccf0 0x7fbc9006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8effd640 1 -- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc90159060 msgr2=0x7fbc9014d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8effd640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc90159060 0x7fbc9014d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.035+0000 7fbc8effd640 1 -- 172.21.15.125:0/125336675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78009560 con 0x7fbc900ce240 2023-10-18T10:44:44.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.036+0000 7fbc8effd640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 0x7fbc900b2aa0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000c430 tx=0x7fbc8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:44.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.036+0000 7fbc8cff9640 1 -- 172.21.15.125:0/125336675 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80011790 con 0x7fbc900ce240 2023-10-18T10:44:44.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.036+0000 7fbc8cff9640 1 -- 172.21.15.125:0/125336675 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc800119e0 con 0x7fbc900ce240 2023-10-18T10:44:44.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.036+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc900b3940 con 0x7fbc900ce240 2023-10-18T10:44:44.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.036+0000 7fbc8cff9640 1 -- 172.21.15.125:0/125336675 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80011e30 con 0x7fbc900ce240 2023-10-18T10:44:44.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.036+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9006c1f0 con 0x7fbc900ce240 2023-10-18T10:44:44.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.037+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc900675f0 con 0x7fbc900ce240 2023-10-18T10:44:44.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.037+0000 7fbc8cff9640 1 -- 172.21.15.125:0/125336675 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fbc8001c460 con 0x7fbc900ce240 2023-10-18T10:44:44.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.038+0000 7fbc8cff9640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fbc7006b020 0x7fbc7006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.038+0000 7fbc87fff640 1 -- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fbc7006b020 msgr2=0x7fbc7006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:44.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.038+0000 7fbc8cff9640 1 -- 172.21.15.125:0/125336675 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fbc80089bf0 con 0x7fbc900ce240 2023-10-18T10:44:44.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.038+0000 7fbc87fff640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fbc7006b020 0x7fbc7006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:44.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.046+0000 7fbc8cff9640 1 -- 172.21.15.125:0/125336675 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc800548a0 con 0x7fbc900ce240 2023-10-18T10:44:44.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.218+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc900b4aa0 con 0x7fbc900ce240 2023-10-18T10:44:44.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.218+0000 7fbc8cff9640 1 -- 172.21.15.125:0/125336675 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc80057ec0 con 0x7fbc900ce240 2023-10-18T10:44:44.221 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:44:44.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fbc7006b020 msgr2=0x7fbc7006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:44.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fbc7006b020 0x7fbc7006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 msgr2=0x7fbc900b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 0x7fbc900b2aa0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000c430 tx=0x7fbc8000c8f0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 shutdown_connections 2023-10-18T10:44:44.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fbc7006b020 0x7fbc7006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc90159060 0x7fbc9014d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9015ccf0 0x7fbc9006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 --2- 172.21.15.125:0/125336675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc900ce240 0x7fbc900b2aa0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 >> 172.21.15.125:0/125336675 conn(0x7fbc900b0670 msgr2=0x7fbc900af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:44.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 shutdown_connections 2023-10-18T10:44:44.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.220+0000 7fbc95875640 1 -- 172.21.15.125:0/125336675 wait complete. 2023-10-18T10:44:44.233 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:44:44.270 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.269+0000 7fa7014b0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:44:44.304 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.302+0000 7fa9a342d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:44:44.314 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.312+0000 7fa7014b0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:44:44.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/3765273622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb406e0b0 msgr2=0x7f1bb406e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/3765273622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb406e0b0 0x7f1bb406e540 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40b1030 tx=0x7f1bac02c250 comp rx=0 tx=0).stop 2023-10-18T10:44:44.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/3765273622 shutdown_connections 2023-10-18T10:44:44.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/3765273622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb406e0b0 0x7f1bb406e540 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/3765273622 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bb4069950 0x7f1bb4069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/3765273622 >> v2:172.21.15.139:3300/0 conn(0x7f1bb4152120 0x7f1bb40b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/3765273622 >> 172.21.15.125:0/3765273622 conn(0x7f1bb40af730 msgr2=0x7f1bb40afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:44.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/3765273622 shutdown_connections 2023-10-18T10:44:44.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.339+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/3765273622 wait complete. 2023-10-18T10:44:44.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.340+0000 7f1bb9aa9640 1 Processor -- start 2023-10-18T10:44:44.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.340+0000 7f1bb9aa9640 1 -- start start 2023-10-18T10:44:44.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb9aa9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb4069950 0x7f1bb4152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb9aa9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bb406e0b0 0x7f1bb41f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb9aa9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb4152120 0x7f1bb41f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb9aa9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41f2990 con 0x7f1bb4069950 2023-10-18T10:44:44.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb9aa9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41f5450 con 0x7f1bb4152120 2023-10-18T10:44:44.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb9aa9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41f5730 con 0x7f1bb406e0b0 2023-10-18T10:44:44.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb4069950 0x7f1bb4152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb4069950 0x7f1bb4152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46380/0 (socket says 172.21.15.125:46380) 2023-10-18T10:44:44.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 -- 172.21.15.125:0/40956693 learned_addr learned my addr 172.21.15.125:0/40956693 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:44.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb3fff640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb4152120 0x7f1bb41f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb2ffd640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bb406e0b0 0x7f1bb41f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 -- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bb406e0b0 msgr2=0x7f1bb41f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bb406e0b0 0x7f1bb41f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 -- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb4152120 msgr2=0x7f1bb41f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb4152120 0x7f1bb41f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 -- 172.21.15.125:0/40956693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac009900 con 0x7f1bb4069950 2023-10-18T10:44:44.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb3fff640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb4152120 0x7f1bb41f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:44:44.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb37fe640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb4069950 0x7f1bb4152c10 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400e790 tx=0x7f1ba400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:44.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1b9bfff640 1 -- 172.21.15.125:0/40956693 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4009580 con 0x7f1bb4069950 2023-10-18T10:44:44.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1b9bfff640 1 -- 172.21.15.125:0/40956693 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1ba400cdb0 con 0x7f1bb4069950 2023-10-18T10:44:44.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb41f5a10 con 0x7f1bb4069950 2023-10-18T10:44:44.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1b9bfff640 1 -- 172.21.15.125:0/40956693 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4010690 con 0x7f1bb4069950 2023-10-18T10:44:44.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.341+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb414daf0 con 0x7f1bb4069950 2023-10-18T10:44:44.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.342+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb414dc80 con 0x7f1bb4069950 2023-10-18T10:44:44.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.343+0000 7f1b9bfff640 1 -- 172.21.15.125:0/40956693 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f1ba40107f0 con 0x7f1bb4069950 2023-10-18T10:44:44.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.343+0000 7f1b9bfff640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ba006aff0 0x7f1ba006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.343+0000 7f1bb2ffd640 1 -- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ba006aff0 msgr2=0x7f1ba006d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:44.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.343+0000 7f1b9bfff640 1 -- 172.21.15.125:0/40956693 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f1ba4014070 con 0x7f1bb4069950 2023-10-18T10:44:44.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.343+0000 7f1bb2ffd640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ba006aff0 0x7f1ba006d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:44.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.347+0000 7f1b9bfff640 1 -- 172.21.15.125:0/40956693 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ba4051ca0 con 0x7f1bb4069950 2023-10-18T10:44:44.352 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.350+0000 7fa9a342d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:44:44.357 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.355+0000 7fa7014b0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:44:44.396 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.394+0000 7fa9a342d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:44:44.409 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.407+0000 7fa7014b0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:44:44.449 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.448+0000 7fa9a342d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:44:44.452 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.450+0000 7fa7014b0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:44:44.493 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.491+0000 7fa9a342d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:44:44.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.521+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1bb40ad390 con 0x7f1bb4069950 2023-10-18T10:44:44.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.522+0000 7f1b9bfff640 1 -- 172.21.15.125:0/40956693 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ba4058c00 con 0x7f1bb4069950 2023-10-18T10:44:44.524 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:44:44.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ba006aff0 msgr2=0x7f1ba006d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:44.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ba006aff0 0x7f1ba006d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb4069950 msgr2=0x7f1bb4152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb4069950 0x7f1bb4152c10 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400e790 tx=0x7f1ba400ec50 comp rx=0 tx=0).stop 2023-10-18T10:44:44.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 shutdown_connections 2023-10-18T10:44:44.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f1ba006aff0 0x7f1ba006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb4152120 0x7f1bb41f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bb406e0b0 0x7f1bb41f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 --2- 172.21.15.125:0/40956693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bb4069950 0x7f1bb4152c10 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 >> 172.21.15.125:0/40956693 conn(0x7f1bb40af730 msgr2=0x7f1bb40afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:44.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 shutdown_connections 2023-10-18T10:44:44.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.524+0000 7f1bb9aa9640 1 -- 172.21.15.125:0/40956693 wait complete. 2023-10-18T10:44:44.537 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:44:44.549 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.548+0000 7fa7014b0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:44:44.598 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.596+0000 7fa9a342d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:44:44.604 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.603+0000 7fa7014b0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:44.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.644+0000 7f734d98a640 1 -- 172.21.15.125:0/3510141937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73480b1980 msgr2=0x7f7348070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.644+0000 7f734d98a640 1 --2- 172.21.15.125:0/3510141937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73480b1980 0x7f7348070080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7330009830 tx=0x7f733002bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.644+0000 7f734d98a640 1 -- 172.21.15.125:0/3510141937 shutdown_connections 2023-10-18T10:44:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.644+0000 7f734d98a640 1 --2- 172.21.15.125:0/3510141937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73480b1980 0x7f7348070080 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.644+0000 7f734d98a640 1 --2- 172.21.15.125:0/3510141937 >> v2:172.21.15.139:3300/0 conn(0x7f73481416f0 0x7f73480b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.644+0000 7f734d98a640 1 -- 172.21.15.125:0/3510141937 >> 172.21.15.125:0/3510141937 conn(0x7f73480af730 msgr2=0x7f73480afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.644+0000 7f734d98a640 1 -- 172.21.15.125:0/3510141937 shutdown_connections 2023-10-18T10:44:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.644+0000 7f734d98a640 1 -- 172.21.15.125:0/3510141937 wait complete. 2023-10-18T10:44:44.648 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.645+0000 7fa7014b0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:44:44.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.645+0000 7f734d98a640 1 Processor -- start 2023-10-18T10:44:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.645+0000 7f734d98a640 1 -- start start 2023-10-18T10:44:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f734d98a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73480705c0 0x7f73480b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f734d98a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73480b1980 0x7f73480b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f734d98a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73481416f0 0x7f73480ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f734d98a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7348159f80 con 0x7f73480705c0 2023-10-18T10:44:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f734d98a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734815ade0 con 0x7f73480b1980 2023-10-18T10:44:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f734d98a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734815a1d0 con 0x7f73481416f0 2023-10-18T10:44:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73480705c0 0x7f73480b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73480705c0 0x7f73480b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46382/0 (socket says 172.21.15.125:46382) 2023-10-18T10:44:44.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 -- 172.21.15.125:0/2152358961 learned_addr learned my addr 172.21.15.125:0/2152358961 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f73477fe640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73481416f0 0x7f73480ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f733ffff640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73480b1980 0x7f73480b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 -- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73481416f0 msgr2=0x7f73480ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73481416f0 0x7f73480ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 -- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73480b1980 msgr2=0x7f73480b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73480b1980 0x7f73480b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 -- 172.21.15.125:0/2152358961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7338009560 con 0x7f73480705c0 2023-10-18T10:44:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f733ffff640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73480b1980 0x7f73480b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:44:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7346ffd640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73480705c0 0x7f73480b0c30 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f733800e9c0 tx=0x7f733800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7344ff9640 1 -- 172.21.15.125:0/2152358961 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800cd80 con 0x7f73480705c0 2023-10-18T10:44:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7344ff9640 1 -- 172.21.15.125:0/2152358961 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7338004020 con 0x7f73480705c0 2023-10-18T10:44:44.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7330009490 con 0x7f73480705c0 2023-10-18T10:44:44.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f7344ff9640 1 -- 172.21.15.125:0/2152358961 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338004410 con 0x7f73480705c0 2023-10-18T10:44:44.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.646+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734815a790 con 0x7f73480705c0 2023-10-18T10:44:44.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.647+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73480675f0 con 0x7f73480705c0 2023-10-18T10:44:44.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.648+0000 7f7344ff9640 1 -- 172.21.15.125:0/2152358961 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f73380035f0 con 0x7f73480705c0 2023-10-18T10:44:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.648+0000 7f7344ff9640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f732806aff0 0x7f732806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.648+0000 7f733ffff640 1 -- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f732806aff0 msgr2=0x7f732806d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.649+0000 7f7344ff9640 1 -- 172.21.15.125:0/2152358961 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f7338014070 con 0x7f73480705c0 2023-10-18T10:44:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.649+0000 7f733ffff640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f732806aff0 0x7f732806d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:44.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.653+0000 7f7344ff9640 1 -- 172.21.15.125:0/2152358961 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7338055200 con 0x7f73480705c0 2023-10-18T10:44:44.657 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.653+0000 7fa9a342d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:44.694 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.693+0000 7fa7014b0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:44:44.699 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.697+0000 7fa9a342d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:44:44.736 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:44.734+0000 7fa7014b0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:44:44.749 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.747+0000 7fa9a342d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:44:44.793 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:44.791+0000 7fa9a342d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:44:44.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.826+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73480b2150 con 0x7f73480705c0 2023-10-18T10:44:44.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.827+0000 7f7344ff9640 1 -- 172.21.15.125:0/2152358961 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7338058820 con 0x7f73480705c0 2023-10-18T10:44:44.831 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:44:44.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f732806aff0 msgr2=0x7f732806d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:44.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f732806aff0 0x7f732806d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73480705c0 msgr2=0x7f73480b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73480705c0 0x7f73480b0c30 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f733800e9c0 tx=0x7f733800ee80 comp rx=0 tx=0).stop 2023-10-18T10:44:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 shutdown_connections 2023-10-18T10:44:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f732806aff0 0x7f732806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73480b1980 0x7f73480b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73481416f0 0x7f73480ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 --2- 172.21.15.125:0/2152358961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73480705c0 0x7f73480b0c30 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 >> 172.21.15.125:0/2152358961 conn(0x7f73480af730 msgr2=0x7f734815b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:44.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 shutdown_connections 2023-10-18T10:44:44.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.829+0000 7f734d98a640 1 -- 172.21.15.125:0/2152358961 wait complete. 2023-10-18T10:44:44.843 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T10:44:44.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.945+0000 7f5e5162b640 1 -- 172.21.15.125:0/4040832324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c0cb2f0 msgr2=0x7f5e4c070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.945+0000 7f5e5162b640 1 --2- 172.21.15.125:0/4040832324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c0cb2f0 0x7f5e4c070520 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c072460 tx=0x7f5e3402bd30 comp rx=0 tx=0).stop 2023-10-18T10:44:44.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.946+0000 7f5e5162b640 1 -- 172.21.15.125:0/4040832324 shutdown_connections 2023-10-18T10:44:44.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.946+0000 7f5e5162b640 1 --2- 172.21.15.125:0/4040832324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c0cb2f0 0x7f5e4c070520 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.946+0000 7f5e5162b640 1 --2- 172.21.15.125:0/4040832324 >> v2:172.21.15.139:3300/0 conn(0x7f5e4c157c00 0x7f5e4c0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.946+0000 7f5e5162b640 1 -- 172.21.15.125:0/4040832324 >> 172.21.15.125:0/4040832324 conn(0x7f5e4c0aec40 msgr2=0x7f5e4c0af050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:44.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.946+0000 7f5e5162b640 1 -- 172.21.15.125:0/4040832324 shutdown_connections 2023-10-18T10:44:44.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.946+0000 7f5e5162b640 1 -- 172.21.15.125:0/4040832324 wait complete. 2023-10-18T10:44:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.947+0000 7f5e5162b640 1 Processor -- start 2023-10-18T10:44:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.947+0000 7f5e5162b640 1 -- start start 2023-10-18T10:44:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e5162b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c070a60 0x7f5e4c13b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e5162b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e4c0cb2f0 0x7f5e4c14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e5162b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e4c157c00 0x7f5e4c1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e5162b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c14cff0 con 0x7f5e4c070a60 2023-10-18T10:44:44.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e5162b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c14c420 con 0x7f5e4c157c00 2023-10-18T10:44:44.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e5162b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c14c6c0 con 0x7f5e4c0cb2f0 2023-10-18T10:44:44.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c070a60 0x7f5e4c13b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c070a60 0x7f5e4c13b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46388/0 (socket says 172.21.15.125:46388) 2023-10-18T10:44:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 -- 172.21.15.125:0/192637044 learned_addr learned my addr 172.21.15.125:0/192637044 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4b7fe640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e4c157c00 0x7f5e4c1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e43fff640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e4c0cb2f0 0x7f5e4c14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 -- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e4c0cb2f0 msgr2=0x7f5e4c14d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e4c0cb2f0 0x7f5e4c14d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 -- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e4c157c00 msgr2=0x7f5e4c1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e4c157c00 0x7f5e4c1540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 -- 172.21.15.125:0/192637044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34009560 con 0x7f5e4c070a60 2023-10-18T10:44:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e4affd640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c070a60 0x7f5e4c13b3f0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00e7c0 tx=0x7f5e3c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e437fe640 1 -- 172.21.15.125:0/192637044 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0095a0 con 0x7f5e4c070a60 2023-10-18T10:44:44.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e437fe640 1 -- 172.21.15.125:0/192637044 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e3c00eed0 con 0x7f5e4c070a60 2023-10-18T10:44:44.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c14c9a0 con 0x7f5e4c070a60 2023-10-18T10:44:44.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.949+0000 7f5e437fe640 1 -- 172.21.15.125:0/192637044 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c005020 con 0x7f5e4c070a60 2023-10-18T10:44:44.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.950+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c13b930 con 0x7f5e4c070a60 2023-10-18T10:44:44.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.950+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c0690e0 con 0x7f5e4c070a60 2023-10-18T10:44:44.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.951+0000 7f5e437fe640 1 -- 172.21.15.125:0/192637044 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f5e3c005180 con 0x7f5e4c070a60 2023-10-18T10:44:44.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.951+0000 7f5e437fe640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f5e2c06aff0 0x7f5e2c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:44.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.951+0000 7f5e43fff640 1 -- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f5e2c06aff0 msgr2=0x7f5e2c06d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:44.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.952+0000 7f5e437fe640 1 -- 172.21.15.125:0/192637044 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f5e3c014070 con 0x7f5e4c070a60 2023-10-18T10:44:44.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.952+0000 7f5e43fff640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f5e2c06aff0 0x7f5e2c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:44.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:44.956+0000 7f5e437fe640 1 -- 172.21.15.125:0/192637044 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e3c055110 con 0x7f5e4c070a60 2023-10-18T10:44:45.051 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.049+0000 7fa7014b0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:44:45.131 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.129+0000 7fa9a342d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:44:45.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.150+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e4c071440 con 0x7f5e4c070a60 2023-10-18T10:44:45.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.150+0000 7f5e437fe640 1 -- 172.21.15.125:0/192637044 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v144) v1 ==== 80+0+476 (secure 0 0 0) 0x7f5e3c058730 con 0x7f5e4c070a60 2023-10-18T10:44:45.152 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3295054028 2023-10-19T10:44:36.950527+0000 2023-10-18T10:44:45.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1133703870 2023-10-19T10:44:36.950527+0000 2023-10-18T10:44:45.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/4009278606 2023-10-19T10:44:36.950527+0000 2023-10-18T10:44:45.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/494329198 2023-10-19T10:44:36.950527+0000 2023-10-18T10:44:45.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/4144373589 2023-10-19T10:44:36.950527+0000 2023-10-18T10:44:45.154 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/4144373589 2023-10-19T10:44:36.950527+0000 2023-10-18T10:44:45.154 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/337665570 2023-10-19T10:44:36.950527+0000 2023-10-18T10:44:45.154 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1274722331 2023-10-19T10:44:36.950527+0000 2023-10-18T10:44:45.155 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T10:44:45.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f5e2c06aff0 msgr2=0x7f5e2c06d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:45.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f5e2c06aff0 0x7f5e2c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c070a60 msgr2=0x7f5e4c13b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:45.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c070a60 0x7f5e4c13b3f0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00e7c0 tx=0x7f5e3c00ec80 comp rx=0 tx=0).stop 2023-10-18T10:44:45.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 shutdown_connections 2023-10-18T10:44:45.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f5e2c06aff0 0x7f5e2c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e4c157c00 0x7f5e4c1540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e4c0cb2f0 0x7f5e4c14d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 --2- 172.21.15.125:0/192637044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e4c070a60 0x7f5e4c13b3f0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 >> 172.21.15.125:0/192637044 conn(0x7f5e4c0aec40 msgr2=0x7f5e4c14a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:45.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 shutdown_connections 2023-10-18T10:44:45.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.152+0000 7f5e5162b640 1 -- 172.21.15.125:0/192637044 wait complete. 2023-10-18T10:44:45.165 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T10:44:45.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:45.166 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T10:44:45.180 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.178+0000 7fa7014b0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:44:45.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[133056]: fuse finished with error 0 and tester_r 0 2023-10-18T10:44:45.192 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:44:45.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:45.193 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T10:44:45.268 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.266+0000 7fa9a342d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:44:45.272 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.270+0000 7fa7014b0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:44:45.313 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.311+0000 7fa7014b0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:44:45.357 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:44:45.358 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:45.361 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.359+0000 7fa9a342d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:44:45.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:44:45.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:44:45.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:45.374 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T10:44:45.405 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.403+0000 7fa9a342d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:44:45.410 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.408+0000 7fa7014b0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:45.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T10:44:45.425 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:44:45.425 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:45.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:44:45.441 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T10:44:45.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:45.442 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T10:44:45.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:45.496 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T10:44:45.496 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:44:45.497 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:45.504 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.502+0000 7fa9a342d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:45.505 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.503+0000 7fa7014b0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:44:45.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:44:45.513 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T10:44:45.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:44:45.514 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T10:44:45.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T10:44:45.568 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T10:44:45.569 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:45.601 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.599+0000 7fa9a342d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:44:45.649 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.646+0000 7fa7014b0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:45.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.673+0000 7f033e6c2640 1 -- 172.21.15.125:0/3515912332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 msgr2=0x7f03380d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:45.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.673+0000 7f033e6c2640 1 --2- 172.21.15.125:0/3515912332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 0x7f03380d1910 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f032c009830 tx=0x7f032c02bd60 comp rx=0 tx=0).stop 2023-10-18T10:44:45.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.673+0000 7f033e6c2640 1 -- 172.21.15.125:0/3515912332 shutdown_connections 2023-10-18T10:44:45.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.673+0000 7f033e6c2640 1 --2- 172.21.15.125:0/3515912332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 0x7f03380d1910 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.673+0000 7f033e6c2640 1 -- 172.21.15.125:0/3515912332 >> 172.21.15.125:0/3515912332 conn(0x7f03380d0d60 msgr2=0x7f03380d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:45.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.673+0000 7f033e6c2640 1 -- 172.21.15.125:0/3515912332 shutdown_connections 2023-10-18T10:44:45.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.673+0000 7f033e6c2640 1 -- 172.21.15.125:0/3515912332 wait complete. 2023-10-18T10:44:45.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.674+0000 7f033e6c2640 1 Processor -- start 2023-10-18T10:44:45.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.674+0000 7f033e6c2640 1 -- start start 2023-10-18T10:44:45.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f033e6c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03380ceb40 0x7f0338158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:45.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f033e6c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 0x7f03381f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:45.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f033e6c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033815b2a0 0x7f033809df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:45.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f033e6c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b6780 con 0x7f03380cf340 2023-10-18T10:44:45.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f033e6c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b38e0 con 0x7f03380ceb40 2023-10-18T10:44:45.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f033e6c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b3ea0 con 0x7f033815b2a0 2023-10-18T10:44:45.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f03377fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 0x7f03381f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:45.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f03377fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 0x7f03381f1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46408/0 (socket says 172.21.15.125:46408) 2023-10-18T10:44:45.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f03377fe640 1 -- 172.21.15.125:0/351188027 learned_addr learned my addr 172.21.15.125:0/351188027 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:45.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f0337fff640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03380ceb40 0x7f0338158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:45.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f033cc5f640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033815b2a0 0x7f033809df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:45.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f03377fe640 1 -- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033815b2a0 msgr2=0x7f033809df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:45.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f03377fe640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033815b2a0 0x7f033809df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f03377fe640 1 -- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03380ceb40 msgr2=0x7f0338158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:45.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f03377fe640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03380ceb40 0x7f0338158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.675+0000 7f03377fe640 1 -- 172.21.15.125:0/351188027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009490 con 0x7f03380cf340 2023-10-18T10:44:45.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.676+0000 7f03377fe640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 0x7f03381f1970 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f032800c3f0 tx=0x7f032800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:45.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.676+0000 7f0334ff9640 1 -- 172.21.15.125:0/351188027 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328011790 con 0x7f03380cf340 2023-10-18T10:44:45.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.676+0000 7f0334ff9640 1 -- 172.21.15.125:0/351188027 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03280119e0 con 0x7f03380cf340 2023-10-18T10:44:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.676+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0338158ed0 con 0x7f03380cf340 2023-10-18T10:44:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.676+0000 7f0334ff9640 1 -- 172.21.15.125:0/351188027 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328011dd0 con 0x7f03380cf340 2023-10-18T10:44:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.676+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03381f2740 con 0x7f03380cf340 2023-10-18T10:44:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.677+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03380690e0 con 0x7f03380cf340 2023-10-18T10:44:45.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.677+0000 7f0334ff9640 1 -- 172.21.15.125:0/351188027 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f032801c460 con 0x7f03380cf340 2023-10-18T10:44:45.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.678+0000 7f0334ff9640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f032406aff0 0x7f032406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:45.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.678+0000 7f0337fff640 1 -- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f032406aff0 msgr2=0x7f032406d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:45.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.678+0000 7f0334ff9640 1 -- 172.21.15.125:0/351188027 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f0328089bb0 con 0x7f03380cf340 2023-10-18T10:44:45.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.678+0000 7f0337fff640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f032406aff0 0x7f032406d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:45.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.682+0000 7f0334ff9640 1 -- 172.21.15.125:0/351188027 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03280547d0 con 0x7f03380cf340 2023-10-18T10:44:45.690 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.688+0000 7fa7014b0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:45.744 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.743+0000 7fa9a342d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:44:45.788 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.786+0000 7fa9a342d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:44:45.794 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:45.792+0000 7fa7014b0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:44:45.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.879+0000 7f0337fff640 1 -- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f032406aff0 msgr2=0x7f032406d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:45.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.879+0000 7f0337fff640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f032406aff0 0x7f032406d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:44:45.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.892+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03381f28d0 con 0x7f03380cf340 2023-10-18T10:44:45.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.893+0000 7f0334ff9640 1 -- 172.21.15.125:0/351188027 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f0328057df0 con 0x7f03380cf340 2023-10-18T10:44:45.895 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:45.895 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":53,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7112,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/585529921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":585529921},{"type":"v1","addr":"172.21.15.125:6835","nonce":585529921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1680315524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1680315524},{"type":"v1","addr":"172.21.15.139:6837","nonce":1680315524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1127901065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1127901065},{"type":"v1","addr":"172.21.15.139:6836","nonce":1127901065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:20.891510+0000","modified":"2023-10-18T10:44:22.895207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9159":{"gid":9159,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2651847004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2651847004},{"type":"v1","addr":"172.21.15.125:6834","nonce":2651847004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T10:44:45.895 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 53 2023-10-18T10:44:45.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.894+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f032406aff0 msgr2=0x7f032406d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:45.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.894+0000 7f033e6c2640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f032406aff0 0x7f032406d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.894+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 msgr2=0x7f03381f1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:45.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.894+0000 7f033e6c2640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 0x7f03381f1970 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f032800c3f0 tx=0x7f032800c8b0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.894+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 shutdown_connections 2023-10-18T10:44:45.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.894+0000 7f033e6c2640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f032406aff0 0x7f032406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.894+0000 7f033e6c2640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033815b2a0 0x7f033809df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.895+0000 7f033e6c2640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03380cf340 0x7f03381f1970 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.895+0000 7f033e6c2640 1 --2- 172.21.15.125:0/351188027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03380ceb40 0x7f0338158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:45.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.895+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 >> 172.21.15.125:0/351188027 conn(0x7f03380d0d60 msgr2=0x7f03380b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:45.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.895+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 shutdown_connections 2023-10-18T10:44:45.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:45.895+0000 7f033e6c2640 1 -- 172.21.15.125:0/351188027 wait complete. 2023-10-18T10:44:45.899 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:45.895+0000 7fa9a342d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:44:45.907 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:46.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.050+0000 7f6581a10640 1 -- 172.21.15.125:0/2810530631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 msgr2=0x7f657c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.050+0000 7f6581a10640 1 --2- 172.21.15.125:0/2810530631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 0x7f657c0c9b30 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f6570009900 tx=0x7f657002be00 comp rx=0 tx=0).stop 2023-10-18T10:44:46.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.050+0000 7f6581a10640 1 -- 172.21.15.125:0/2810530631 shutdown_connections 2023-10-18T10:44:46.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.050+0000 7f6581a10640 1 --2- 172.21.15.125:0/2810530631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 0x7f657c0c9b30 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.050+0000 7f6581a10640 1 --2- 172.21.15.125:0/2810530631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f657c06af00 0x7f657c06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.050+0000 7f6581a10640 1 -- 172.21.15.125:0/2810530631 >> 172.21.15.125:0/2810530631 conn(0x7f657c069930 msgr2=0x7f657c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:46.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.050+0000 7f6581a10640 1 -- 172.21.15.125:0/2810530631 shutdown_connections 2023-10-18T10:44:46.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.050+0000 7f6581a10640 1 -- 172.21.15.125:0/2810530631 wait complete. 2023-10-18T10:44:46.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.051+0000 7f6581a10640 1 Processor -- start 2023-10-18T10:44:46.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f6581a10640 1 -- start start 2023-10-18T10:44:46.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f6581a10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c06af00 0x7f657c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f6581a10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f657c0ab500 0x7f657c0af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f6581a10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 0x7f657c1f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f6581a10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c15b000 con 0x7f657c0c96b0 2023-10-18T10:44:46.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f6581a10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0afec0 con 0x7f657c06af00 2023-10-18T10:44:46.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f6581a10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c15aa40 con 0x7f657c0ab500 2023-10-18T10:44:46.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f657b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 0x7f657c1f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f657b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 0x7f657c1f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46414/0 (socket says 172.21.15.125:46414) 2023-10-18T10:44:46.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.052+0000 7f657b7fe640 1 -- 172.21.15.125:0/3096957276 learned_addr learned my addr 172.21.15.125:0/3096957276 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:46.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f657a7fc640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f657c0ab500 0x7f657c0af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f657affd640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c06af00 0x7f657c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f657b7fe640 1 -- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f657c0ab500 msgr2=0x7f657c0af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f657b7fe640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f657c0ab500 0x7f657c0af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f657b7fe640 1 -- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c06af00 msgr2=0x7f657c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f657b7fe640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c06af00 0x7f657c15a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f657b7fe640 1 -- 172.21.15.125:0/3096957276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570009560 con 0x7f657c0c96b0 2023-10-18T10:44:46.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f657b7fe640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 0x7f657c1f2230 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f657400c710 tx=0x7f657400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:46.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f6563fff640 1 -- 172.21.15.125:0/3096957276 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574006d50 con 0x7f657c0c96b0 2023-10-18T10:44:46.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f6563fff640 1 -- 172.21.15.125:0/3096957276 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f657400ce20 con 0x7f657c0c96b0 2023-10-18T10:44:46.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f6563fff640 1 -- 172.21.15.125:0/3096957276 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574003bd0 con 0x7f657c0c96b0 2023-10-18T10:44:46.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c15ad20 con 0x7f657c0c96b0 2023-10-18T10:44:46.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.053+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c1f1d10 con 0x7f657c0c96b0 2023-10-18T10:44:46.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.054+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c1f1890 con 0x7f657c0c96b0 2023-10-18T10:44:46.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.055+0000 7f6563fff640 1 -- 172.21.15.125:0/3096957276 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f6574003d30 con 0x7f657c0c96b0 2023-10-18T10:44:46.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.055+0000 7f6563fff640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f656806aff0 0x7f656806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.055+0000 7f657affd640 1 -- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f656806aff0 msgr2=0x7f656806d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:46.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.055+0000 7f6563fff640 1 -- 172.21.15.125:0/3096957276 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f6574089880 con 0x7f657c0c96b0 2023-10-18T10:44:46.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.055+0000 7f657affd640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f656806aff0 0x7f656806d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:46.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.059+0000 7f6563fff640 1 -- 172.21.15.125:0/3096957276 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f657c1f1890 con 0x7f657c0c96b0 2023-10-18T10:44:46.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.229+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f657c1f1890 con 0x7f657c0c96b0 2023-10-18T10:44:46.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.231+0000 7f6563fff640 1 -- 172.21.15.125:0/3096957276 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+17383 (secure 0 0 0) 0x7f657c1f1890 con 0x7f657c0c96b0 2023-10-18T10:44:46.233 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:46.233 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":144,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:36.950567+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:18.717169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:19.861005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3295054028":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1133703870":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/4009278606":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/494329198":"2023-10-19T10:44:36.950527+0000","172.21.15.139:6833/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:6832/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/337665570":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1274722331":"2023-10-19T10:44:36.950527+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:46.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f656806aff0 msgr2=0x7f656806d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:46.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f656806aff0 0x7f656806d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 msgr2=0x7f657c1f2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 0x7f657c1f2230 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f657400c710 tx=0x7f657400cbd0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 shutdown_connections 2023-10-18T10:44:46.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7f656806aff0 0x7f656806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c0c96b0 0x7f657c1f2230 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f657c0ab500 0x7f657c0af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 --2- 172.21.15.125:0/3096957276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c06af00 0x7f657c15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 >> 172.21.15.125:0/3096957276 conn(0x7f657c069930 msgr2=0x7f657c0adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:46.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 shutdown_connections 2023-10-18T10:44:46.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.233+0000 7f6581a10640 1 -- 172.21.15.125:0/3096957276 wait complete. 2023-10-18T10:44:46.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:44:18.717169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:44:19.861005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:44:46.246 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:44:46.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.353+0000 7fca81f6c640 1 -- 172.21.15.125:0/29458850 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c0cc200 msgr2=0x7fca7c0cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.353+0000 7fca81f6c640 1 --2- 172.21.15.125:0/29458850 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c0cc200 0x7fca7c0cc690 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fca74005fd0 tx=0x7fca7402be10 comp rx=0 tx=0).stop 2023-10-18T10:44:46.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.353+0000 7fca81f6c640 1 -- 172.21.15.125:0/29458850 shutdown_connections 2023-10-18T10:44:46.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.353+0000 7fca81f6c640 1 --2- 172.21.15.125:0/29458850 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c0cc200 0x7fca7c0cc690 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.353+0000 7fca81f6c640 1 --2- 172.21.15.125:0/29458850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca7c070080 0x7fca7c0704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.353+0000 7fca81f6c640 1 -- 172.21.15.125:0/29458850 >> 172.21.15.125:0/29458850 conn(0x7fca7c0af730 msgr2=0x7fca7c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:46.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.353+0000 7fca81f6c640 1 -- 172.21.15.125:0/29458850 shutdown_connections 2023-10-18T10:44:46.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.353+0000 7fca81f6c640 1 -- 172.21.15.125:0/29458850 wait complete. 2023-10-18T10:44:46.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.354+0000 7fca81f6c640 1 Processor -- start 2023-10-18T10:44:46.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.355+0000 7fca81f6c640 1 -- start start 2023-10-18T10:44:46.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.355+0000 7fca81f6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca7c070080 0x7fca7c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.355+0000 7fca81f6c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0cc200 0x7fca7c0ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.355+0000 7fca81f6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c1416f0 0x7fca7c0ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.355+0000 7fca81f6c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c06ceb0 con 0x7fca7c1416f0 2023-10-18T10:44:46.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.355+0000 7fca81f6c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c06bfe0 con 0x7fca7c0cc200 2023-10-18T10:44:46.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.355+0000 7fca81f6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c06c230 con 0x7fca7c070080 2023-10-18T10:44:46.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.355+0000 7fca7bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c1416f0 0x7fca7c0ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c1416f0 0x7fca7c0ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46430/0 (socket says 172.21.15.125:46430) 2023-10-18T10:44:46.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7bfff640 1 -- 172.21.15.125:0/97275124 learned_addr learned my addr 172.21.15.125:0/97275124 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:46.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7b7fe640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca7c070080 0x7fca7c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7affd640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0cc200 0x7fca7c0ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7bfff640 1 -- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca7c070080 msgr2=0x7fca7c06da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7bfff640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca7c070080 0x7fca7c06da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7bfff640 1 -- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0cc200 msgr2=0x7fca7c0ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7bfff640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0cc200 0x7fca7c0ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7bfff640 1 -- 172.21.15.125:0/97275124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca74009560 con 0x7fca7c1416f0 2023-10-18T10:44:46.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca7bfff640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c1416f0 0x7fca7c0ac930 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fca74006010 tx=0x7fca7402db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:46.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca73fff640 1 -- 172.21.15.125:0/97275124 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca740035f0 con 0x7fca7c1416f0 2023-10-18T10:44:46.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca73fff640 1 -- 172.21.15.125:0/97275124 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fca74003840 con 0x7fca7c1416f0 2023-10-18T10:44:46.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c0b1cd0 con 0x7fca7c1416f0 2023-10-18T10:44:46.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca73fff640 1 -- 172.21.15.125:0/97275124 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca74003bf0 con 0x7fca7c1416f0 2023-10-18T10:44:46.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.356+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c06c7f0 con 0x7fca7c1416f0 2023-10-18T10:44:46.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.358+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c1e2640 con 0x7fca7c1416f0 2023-10-18T10:44:46.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.358+0000 7fca73fff640 1 -- 172.21.15.125:0/97275124 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fca74003d50 con 0x7fca7c1416f0 2023-10-18T10:44:46.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.358+0000 7fca73fff640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fca5c06aff0 0x7fca5c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.358+0000 7fca7b7fe640 1 -- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fca5c06aff0 msgr2=0x7fca5c06d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.358+0000 7fca73fff640 1 -- 172.21.15.125:0/97275124 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fca740ac530 con 0x7fca7c1416f0 2023-10-18T10:44:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.358+0000 7fca7b7fe640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fca5c06aff0 0x7fca5c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.362+0000 7fca73fff640 1 -- 172.21.15.125:0/97275124 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca74077270 con 0x7fca7c1416f0 2023-10-18T10:44:46.410 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.408+0000 7fa7014b0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:44:46.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.559+0000 7fca7b7fe640 1 -- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fca5c06aff0 msgr2=0x7fca5c06d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:46.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.559+0000 7fca7b7fe640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fca5c06aff0 0x7fca5c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:44:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.573+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca7c0675f0 con 0x7fca7c1416f0 2023-10-18T10:44:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.573+0000 7fca73fff640 1 -- 172.21.15.125:0/97275124 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca7407a890 con 0x7fca7c1416f0 2023-10-18T10:44:46.575 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:44:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fca5c06aff0 msgr2=0x7fca5c06d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fca5c06aff0 0x7fca5c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c1416f0 msgr2=0x7fca7c0ac930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c1416f0 0x7fca7c0ac930 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fca74006010 tx=0x7fca7402db30 comp rx=0 tx=0).stop 2023-10-18T10:44:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 shutdown_connections 2023-10-18T10:44:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fca5c06aff0 0x7fca5c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0cc200 0x7fca7c0ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fca7c1416f0 0x7fca7c0ac930 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 --2- 172.21.15.125:0/97275124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fca7c070080 0x7fca7c06da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 >> 172.21.15.125:0/97275124 conn(0x7fca7c0af730 msgr2=0x7fca7c0ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 shutdown_connections 2023-10-18T10:44:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.575+0000 7fca81f6c640 1 -- 172.21.15.125:0/97275124 wait complete. 2023-10-18T10:44:46.580 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:46.578+0000 7fa9a342d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:44:46.587 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T10:44:46.674 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.672+0000 7fa7014b0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:44:46.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.691+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2726380314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480cf340 msgr2=0x7fb5480d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.691+0000 7fb54f3e2640 1 --2- 172.21.15.125:0/2726380314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480cf340 0x7fb5480d1910 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fb544005fd0 tx=0x7fb54402bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:46.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.691+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2726380314 shutdown_connections 2023-10-18T10:44:46.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.691+0000 7fb54f3e2640 1 --2- 172.21.15.125:0/2726380314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb54815b2a0 0x7fb5480cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.691+0000 7fb54f3e2640 1 --2- 172.21.15.125:0/2726380314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480cf340 0x7fb5480d1910 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.691+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2726380314 >> 172.21.15.125:0/2726380314 conn(0x7fb5480d0d60 msgr2=0x7fb5480d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:46.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.691+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2726380314 shutdown_connections 2023-10-18T10:44:46.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.691+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2726380314 wait complete. 2023-10-18T10:44:46.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.692+0000 7fb54f3e2640 1 Processor -- start 2023-10-18T10:44:46.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54f3e2640 1 -- start start 2023-10-18T10:44:46.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54f3e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480ceb40 0x7fb54815db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54f3e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5480cf340 0x7fb54815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54f3e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb54815b2a0 0x7fb5480cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54f3e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548154da0 con 0x7fb5480ceb40 2023-10-18T10:44:46.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54f3e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815e200 con 0x7fb54815b2a0 2023-10-18T10:44:46.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54f3e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480b52d0 con 0x7fb5480cf340 2023-10-18T10:44:46.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54d17e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480ceb40 0x7fb54815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54d17e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480ceb40 0x7fb54815db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46444/0 (socket says 172.21.15.125:46444) 2023-10-18T10:44:46.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54d17e640 1 -- 172.21.15.125:0/2833282326 learned_addr learned my addr 172.21.15.125:0/2833282326 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:46.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54c97d640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5480cf340 0x7fb54815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.693+0000 7fb54d97f640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb54815b2a0 0x7fb5480cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:46.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb54d17e640 1 -- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5480cf340 msgr2=0x7fb54815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb54d17e640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5480cf340 0x7fb54815cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb54d17e640 1 -- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb54815b2a0 msgr2=0x7fb5480cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb54d17e640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb54815b2a0 0x7fb5480cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb54d17e640 1 -- 172.21.15.125:0/2833282326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544009560 con 0x7fb5480ceb40 2023-10-18T10:44:46.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb54d17e640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480ceb40 0x7fb54815db90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fb5440099f0 tx=0x7fb54402da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:46.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544003800 con 0x7fb5480ceb40 2023-10-18T10:44:46.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb544003a50 con 0x7fb5480ceb40 2023-10-18T10:44:46.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54815d4c0 con 0x7fb5480ceb40 2023-10-18T10:44:46.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544003e00 con 0x7fb5480ceb40 2023-10-18T10:44:46.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.694+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5480b5540 con 0x7fb5480ceb40 2023-10-18T10:44:46.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.695+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5480b5130 con 0x7fb5480ceb40 2023-10-18T10:44:46.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.696+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 77) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fb54403e450 con 0x7fb5480ceb40 2023-10-18T10:44:46.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.696+0000 7fb53dffb640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fb52c06aff0 0x7fb52c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:46.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.696+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fb5440ac680 con 0x7fb5480ceb40 2023-10-18T10:44:46.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.696+0000 7fb54c97d640 1 -- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fb52c06aff0 msgr2=0x7fb52c06d4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/427428567 2023-10-18T10:44:46.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.696+0000 7fb54c97d640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fb52c06aff0 0x7fb52c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:44:46.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.704+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5480b5130 con 0x7fb5480ceb40 2023-10-18T10:44:46.722 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.721+0000 7fa7014b0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:44:46.822 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.820+0000 7fa7014b0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:44:46.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.844+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 78) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fb5440775f0 con 0x7fb5480ceb40 2023-10-18T10:44:46.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.844+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fb52c06aff0 msgr2=0x7fb52c06d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:44:46.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.844+0000 7fb53dffb640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fb52c06aff0 0x7fb52c06d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.874 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:46.872+0000 7fa9a342d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:44:46.897 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa683f43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.897 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa683f43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.897 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa683f43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.897 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa683f43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.898 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa683f43640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.898 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa686f49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.898 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa686f49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.898 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa686f49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.898 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa686f49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.899 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:44:46.895+0000 7fa686f49640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:44:46.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.899+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5480b5130 con 0x7fb5480ceb40 2023-10-18T10:44:46.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.900+0000 7fb53dffb640 1 -- 172.21.15.125:0/2833282326 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5480b5130 con 0x7fb5480ceb40 2023-10-18T10:44:46.902 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T10:44:46.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480ceb40 msgr2=0x7fb54815db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:46.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480ceb40 0x7fb54815db90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fb5440099f0 tx=0x7fb54402da60 comp rx=0 tx=0).stop 2023-10-18T10:44:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 shutdown_connections 2023-10-18T10:44:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:6832/427428567,v1:172.21.15.139:6833/427428567] conn(0x7fb52c06aff0 0x7fb52c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb54815b2a0 0x7fb5480cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5480cf340 0x7fb54815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 --2- 172.21.15.125:0/2833282326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5480ceb40 0x7fb54815db90 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:46.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 >> 172.21.15.125:0/2833282326 conn(0x7fb5480d0d60 msgr2=0x7fb5480b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:46.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 shutdown_connections 2023-10-18T10:44:46.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:46.902+0000 7fb54f3e2640 1 -- 172.21.15.125:0/2833282326 wait complete. 2023-10-18T10:44:46.915 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T10:44:46.929 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:46.927+0000 7fa9a342d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:44:47.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.020+0000 7fddb4a56640 1 -- 172.21.15.125:0/4064634872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd9c003970 msgr2=0x7fdd9c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.020+0000 7fddb4a56640 1 --2- 172.21.15.125:0/4064634872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd9c003970 0x7fdd9c003d90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009d20 tx=0x7fdd9c0057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.020+0000 7fddb4a56640 1 -- 172.21.15.125:0/4064634872 shutdown_connections 2023-10-18T10:44:47.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.020+0000 7fddb4a56640 1 --2- 172.21.15.125:0/4064634872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd9c003970 0x7fdd9c003d90 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.020+0000 7fddb4a56640 1 -- 172.21.15.125:0/4064634872 >> 172.21.15.125:0/4064634872 conn(0x7fddb00d0d60 msgr2=0x7fddb00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:47.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.020+0000 7fddb4a56640 1 -- 172.21.15.125:0/4064634872 shutdown_connections 2023-10-18T10:44:47.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.020+0000 7fddb4a56640 1 -- 172.21.15.125:0/4064634872 wait complete. 2023-10-18T10:44:47.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fddb4a56640 1 Processor -- start 2023-10-18T10:44:47.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fddb4a56640 1 -- start start 2023-10-18T10:44:47.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fddb4a56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9c003970 0x7fddb00b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fddb4a56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb01f14d0 0x7fddb014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fddb4a56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb00c91d0 0x7fddb00c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fddb4a56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b4040 con 0x7fddb01f14d0 2023-10-18T10:44:47.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fddb4a56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00c8060 con 0x7fddb00c91d0 2023-10-18T10:44:47.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fddb4a56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb01f1c40 con 0x7fdd9c003970 2023-10-18T10:44:47.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fdda7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb01f14d0 0x7fddb014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fdda7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb01f14d0 0x7fddb014c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46464/0 (socket says 172.21.15.125:46464) 2023-10-18T10:44:47.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.022+0000 7fdda7fff640 1 -- 172.21.15.125:0/4285225826 learned_addr learned my addr 172.21.15.125:0/4285225826 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:47.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fddaed9d640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb00c91d0 0x7fddb00c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fddae59c640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9c003970 0x7fddb00b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda7fff640 1 -- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9c003970 msgr2=0x7fddb00b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda7fff640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9c003970 0x7fddb00b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda7fff640 1 -- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb00c91d0 msgr2=0x7fddb00c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda7fff640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb00c91d0 0x7fddb00c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda7fff640 1 -- 172.21.15.125:0/4285225826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb01f1f20 con 0x7fddb01f14d0 2023-10-18T10:44:47.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fddaed9d640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb00c91d0 0x7fddb00c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:47.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda7fff640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb01f14d0 0x7fddb014c080 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fdd98003e40 tx=0x7fdd98001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:47.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda6ffd640 1 -- 172.21.15.125:0/4285225826 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011640 con 0x7fddb01f14d0 2023-10-18T10:44:47.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda6ffd640 1 -- 172.21.15.125:0/4285225826 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd980117a0 con 0x7fddb01f14d0 2023-10-18T10:44:47.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fdda6ffd640 1 -- 172.21.15.125:0/4285225826 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011b90 con 0x7fddb01f14d0 2023-10-18T10:44:47.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb01f2200 con 0x7fddb01f14d0 2023-10-18T10:44:47.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.023+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb014c5c0 con 0x7fddb01f14d0 2023-10-18T10:44:47.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.024+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb00d0270 con 0x7fddb01f14d0 2023-10-18T10:44:47.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.025+0000 7fdda6ffd640 1 -- 172.21.15.125:0/4285225826 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 78) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fdd98011cf0 con 0x7fddb01f14d0 2023-10-18T10:44:47.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.025+0000 7fdda6ffd640 1 -- 172.21.15.125:0/4285225826 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6788+0+0 (secure 0 0 0) 0x7fdd9808a4e0 con 0x7fddb01f14d0 2023-10-18T10:44:47.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.029+0000 7fdda6ffd640 1 -- 172.21.15.125:0/4285225826 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd98055520 con 0x7fddb01f14d0 2023-10-18T10:44:47.036 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:44:47.034+0000 7fa9a342d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:44:47.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.195+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fddb00690e0 con 0x7fddb01f14d0 2023-10-18T10:44:47.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.195+0000 7fdda6ffd640 1 -- 172.21.15.125:0/4285225826 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd9801f0a0 con 0x7fddb01f14d0 2023-10-18T10:44:47.198 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T10:44:47.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb01f14d0 msgr2=0x7fddb014c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb01f14d0 0x7fddb014c080 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fdd98003e40 tx=0x7fdd98001630 comp rx=0 tx=0).stop 2023-10-18T10:44:47.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 shutdown_connections 2023-10-18T10:44:47.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb00c91d0 0x7fddb00c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb01f14d0 0x7fddb014c080 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 --2- 172.21.15.125:0/4285225826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd9c003970 0x7fddb00b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 >> 172.21.15.125:0/4285225826 conn(0x7fddb00d0d60 msgr2=0x7fddb006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:47.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 shutdown_connections 2023-10-18T10:44:47.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.197+0000 7fddb4a56640 1 -- 172.21.15.125:0/4285225826 wait complete. 2023-10-18T10:44:47.210 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:47.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.349+0000 7f3d75968640 1 -- 172.21.15.125:0/3481673600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d700cf340 msgr2=0x7f3d700d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.349+0000 7f3d75968640 1 --2- 172.21.15.125:0/3481673600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d700cf340 0x7f3d700d1910 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005fd0 tx=0x7f3d6002c0e0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.349+0000 7f3d75968640 1 -- 172.21.15.125:0/3481673600 shutdown_connections 2023-10-18T10:44:47.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.349+0000 7f3d75968640 1 --2- 172.21.15.125:0/3481673600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d700cf340 0x7f3d700d1910 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.349+0000 7f3d75968640 1 -- 172.21.15.125:0/3481673600 >> 172.21.15.125:0/3481673600 conn(0x7f3d700d0d60 msgr2=0x7f3d700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:47.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.349+0000 7f3d75968640 1 -- 172.21.15.125:0/3481673600 shutdown_connections 2023-10-18T10:44:47.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.349+0000 7f3d75968640 1 -- 172.21.15.125:0/3481673600 wait complete. 2023-10-18T10:44:47.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.350+0000 7f3d75968640 1 Processor -- start 2023-10-18T10:44:47.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.350+0000 7f3d75968640 1 -- start start 2023-10-18T10:44:47.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d75968640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700ceb40 0x7f3d701f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d75968640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d700cf340 0x7f3d701f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d75968640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d7015b2a0 0x7f3d700ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d75968640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700b38e0 con 0x7f3d7015b2a0 2023-10-18T10:44:47.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d75968640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d701f5a90 con 0x7f3d700ceb40 2023-10-18T10:44:47.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d75968640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700b3af0 con 0x7f3d700cf340 2023-10-18T10:44:47.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d6f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d7015b2a0 0x7f3d700ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d6f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d7015b2a0 0x7f3d700ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46478/0 (socket says 172.21.15.125:46478) 2023-10-18T10:44:47.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d6f7fe640 1 -- 172.21.15.125:0/35871536 learned_addr learned my addr 172.21.15.125:0/35871536 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:47.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d6effd640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700ceb40 0x7f3d701f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.351+0000 7f3d6e7fc640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d700cf340 0x7f3d701f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d6f7fe640 1 -- 172.21.15.125:0/35871536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d700cf340 msgr2=0x7f3d701f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d6f7fe640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d700cf340 0x7f3d701f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d6f7fe640 1 -- 172.21.15.125:0/35871536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700ceb40 msgr2=0x7f3d701f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d6f7fe640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700ceb40 0x7f3d701f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d6f7fe640 1 -- 172.21.15.125:0/35871536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d60009840 con 0x7f3d7015b2a0 2023-10-18T10:44:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d6f7fe640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d7015b2a0 0x7f3d700ae3c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800c3c0 tx=0x7f3d6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d57fff640 1 -- 172.21.15.125:0/35871536 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680052d0 con 0x7f3d7015b2a0 2023-10-18T10:44:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d57fff640 1 -- 172.21.15.125:0/35871536 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d68007020 con 0x7f3d7015b2a0 2023-10-18T10:44:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d700b3dd0 con 0x7f3d7015b2a0 2023-10-18T10:44:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d57fff640 1 -- 172.21.15.125:0/35871536 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68007430 con 0x7f3d7015b2a0 2023-10-18T10:44:47.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.352+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d700b4390 con 0x7f3d7015b2a0 2023-10-18T10:44:47.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.353+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d700690e0 con 0x7f3d7015b2a0 2023-10-18T10:44:47.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.353+0000 7f3d57fff640 1 -- 172.21.15.125:0/35871536 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 78) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f3d680067d0 con 0x7f3d7015b2a0 2023-10-18T10:44:47.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.354+0000 7f3d57fff640 1 -- 172.21.15.125:0/35871536 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6788+0+0 (secure 0 0 0) 0x7f3d68088a80 con 0x7f3d7015b2a0 2023-10-18T10:44:47.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.358+0000 7f3d57fff640 1 -- 172.21.15.125:0/35871536 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d68053b70 con 0x7f3d7015b2a0 2023-10-18T10:44:47.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.560+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d700d0b40 con 0x7f3d7015b2a0 2023-10-18T10:44:47.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.560+0000 7f3d57fff640 1 -- 172.21.15.125:0/35871536 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f3d68057190 con 0x7f3d7015b2a0 2023-10-18T10:44:47.563 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:47.563 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":53,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7112,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/585529921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":585529921},{"type":"v1","addr":"172.21.15.125:6835","nonce":585529921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1680315524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1680315524},{"type":"v1","addr":"172.21.15.139:6837","nonce":1680315524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1127901065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1127901065},{"type":"v1","addr":"172.21.15.139:6836","nonce":1127901065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:20.891510+0000","modified":"2023-10-18T10:44:22.895207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9159":{"gid":9159,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2651847004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2651847004},{"type":"v1","addr":"172.21.15.125:6834","nonce":2651847004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T10:44:47.563 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 53 2023-10-18T10:44:47.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d7015b2a0 msgr2=0x7f3d700ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d7015b2a0 0x7f3d700ae3c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800c3c0 tx=0x7f3d6800c880 comp rx=0 tx=0).stop 2023-10-18T10:44:47.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 shutdown_connections 2023-10-18T10:44:47.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d7015b2a0 0x7f3d700ae3c0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d700cf340 0x7f3d701f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 --2- 172.21.15.125:0/35871536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700ceb40 0x7f3d701f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 >> 172.21.15.125:0/35871536 conn(0x7f3d700d0d60 msgr2=0x7f3d700ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:47.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 shutdown_connections 2023-10-18T10:44:47.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.562+0000 7f3d75968640 1 -- 172.21.15.125:0/35871536 wait complete. 2023-10-18T10:44:47.575 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:47.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.674+0000 7fa2e2c90640 1 -- 172.21.15.125:0/835299796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc13e520 msgr2=0x7fa2dc06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.674+0000 7fa2e2c90640 1 --2- 172.21.15.125:0/835299796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc13e520 0x7fa2dc06bb60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8005fd0 tx=0x7fa2d802bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.674+0000 7fa2e2c90640 1 -- 172.21.15.125:0/835299796 shutdown_connections 2023-10-18T10:44:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.674+0000 7fa2e2c90640 1 --2- 172.21.15.125:0/835299796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc13e520 0x7fa2dc06bb60 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.674+0000 7fa2e2c90640 1 -- 172.21.15.125:0/835299796 >> 172.21.15.125:0/835299796 conn(0x7fa2dc069930 msgr2=0x7fa2dc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.674+0000 7fa2e2c90640 1 -- 172.21.15.125:0/835299796 shutdown_connections 2023-10-18T10:44:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.674+0000 7fa2e2c90640 1 -- 172.21.15.125:0/835299796 wait complete. 2023-10-18T10:44:47.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.675+0000 7fa2e2c90640 1 Processor -- start 2023-10-18T10:44:47.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.675+0000 7fa2e2c90640 1 -- start start 2023-10-18T10:44:47.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e2c90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc070af0 0x7fa2dc13b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e2c90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc13e520 0x7fa2dc1e2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e2c90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc141690 0x7fa2dc0aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e2c90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc1e34c0 con 0x7fa2dc141690 2023-10-18T10:44:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e2c90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc1e2f00 con 0x7fa2dc13e520 2023-10-18T10:44:47.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e2c90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc1e31e0 con 0x7fa2dc070af0 2023-10-18T10:44:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e122d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc141690 0x7fa2dc0aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e122d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc141690 0x7fa2dc0aa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37776/0 (socket says 172.21.15.125:37776) 2023-10-18T10:44:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.676+0000 7fa2e122d640 1 -- 172.21.15.125:0/2562165499 learned_addr learned my addr 172.21.15.125:0/2562165499 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:47.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2e122d640 1 -- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc070af0 msgr2=0x7fa2dc13b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2e122d640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc070af0 0x7fa2dc13b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2e122d640 1 -- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc13e520 msgr2=0x7fa2dc1e2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2e122d640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc13e520 0x7fa2dc1e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2e122d640 1 -- 172.21.15.125:0/2562165499 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8009560 con 0x7fa2dc141690 2023-10-18T10:44:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2e122d640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc141690 0x7fa2dc0aa5a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4001560 tx=0x7fa2d400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:47.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2d17fa640 1 -- 172.21.15.125:0/2562165499 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4003da0 con 0x7fa2dc141690 2023-10-18T10:44:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2d17fa640 1 -- 172.21.15.125:0/2562165499 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2d4005070 con 0x7fa2dc141690 2023-10-18T10:44:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc0a68a0 con 0x7fa2dc141690 2023-10-18T10:44:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2d17fa640 1 -- 172.21.15.125:0/2562165499 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4005480 con 0x7fa2dc141690 2023-10-18T10:44:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.677+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc13b6f0 con 0x7fa2dc141690 2023-10-18T10:44:47.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.678+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc0a6dc0 con 0x7fa2dc141690 2023-10-18T10:44:47.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.678+0000 7fa2d17fa640 1 -- 172.21.15.125:0/2562165499 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 78) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa2d40055e0 con 0x7fa2dc141690 2023-10-18T10:44:47.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.679+0000 7fa2d17fa640 1 -- 172.21.15.125:0/2562165499 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6788+0+0 (secure 0 0 0) 0x7fa2d40890e0 con 0x7fa2dc141690 2023-10-18T10:44:47.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.683+0000 7fa2d17fa640 1 -- 172.21.15.125:0/2562165499 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2dc0a6dc0 con 0x7fa2dc141690 2023-10-18T10:44:47.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.847+0000 7fa2d17fa640 1 -- 172.21.15.125:0/2562165499 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa2d4053bd0 con 0x7fa2dc141690 2023-10-18T10:44:47.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.847+0000 7fa2d17fa640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa2c8070b50 0x7fa2c8072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.852+0000 7fa2e0a2c640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa2c8070b50 0x7fa2c8072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.852+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2dc0a6dc0 con 0x7fa2dc141690 2023-10-18T10:44:47.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.853+0000 7fa2e0a2c640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa2c8070b50 0x7fa2c8072f40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8035000 tx=0x7fa2d80046c0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:47.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.854+0000 7fa2d17fa640 1 -- 172.21.15.125:0/2562165499 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17953 (secure 0 0 0) 0x7fa2dc0a6dc0 con 0x7fa2dc141690 2023-10-18T10:44:47.856 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:47.856 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":145,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:46.834541+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:18.717169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:19.861005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2424959739":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/848644581":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/1809262820":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6833/427428567":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6832/427428567":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/2263683535":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/3114574713":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/3295054028":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1133703870":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/4009278606":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/494329198":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/2843740551":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6833/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:6832/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/3297526571":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/337665570":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1274722331":"2023-10-19T10:44:36.950527+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:47.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.855+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa2c8070b50 msgr2=0x7fa2c8072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.855+0000 7fa2e2c90640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa2c8070b50 0x7fa2c8072f40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8035000 tx=0x7fa2d80046c0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc141690 msgr2=0x7fa2dc0aa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc141690 0x7fa2dc0aa5a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4001560 tx=0x7fa2d400cc70 comp rx=0 tx=0).stop 2023-10-18T10:44:47.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 shutdown_connections 2023-10-18T10:44:47.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa2c8070b50 0x7fa2c8072f40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc13e520 0x7fa2dc1e2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2dc141690 0x7fa2dc0aa5a0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 --2- 172.21.15.125:0/2562165499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc070af0 0x7fa2dc13b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 >> 172.21.15.125:0/2562165499 conn(0x7fa2dc069930 msgr2=0x7fa2dc1403a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:47.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 shutdown_connections 2023-10-18T10:44:47.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.856+0000 7fa2e2c90640 1 -- 172.21.15.125:0/2562165499 wait complete. 2023-10-18T10:44:47.868 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T10:44:47.869 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.970+0000 7fc939f97640 1 -- 172.21.15.125:0/4103708738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc93415b2a0 msgr2=0x7fc9340d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.970+0000 7fc939f97640 1 --2- 172.21.15.125:0/4103708738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc93415b2a0 0x7fc9340d1910 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fc93415b680 tx=0x7fc91c02bdb0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.970+0000 7fc939f97640 1 -- 172.21.15.125:0/4103708738 shutdown_connections 2023-10-18T10:44:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.970+0000 7fc939f97640 1 --2- 172.21.15.125:0/4103708738 >> v2:172.21.15.139:3300/0 conn(0x7fc9340ceb40 0x7fc9340b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.970+0000 7fc939f97640 1 --2- 172.21.15.125:0/4103708738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc93415b2a0 0x7fc9340d1910 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.970+0000 7fc939f97640 1 -- 172.21.15.125:0/4103708738 >> 172.21.15.125:0/4103708738 conn(0x7fc9340d0d60 msgr2=0x7fc9340d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.970+0000 7fc939f97640 1 -- 172.21.15.125:0/4103708738 shutdown_connections 2023-10-18T10:44:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.970+0000 7fc939f97640 1 -- 172.21.15.125:0/4103708738 wait complete. 2023-10-18T10:44:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.971+0000 7fc939f97640 1 Processor -- start 2023-10-18T10:44:47.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.971+0000 7fc939f97640 1 -- start start 2023-10-18T10:44:47.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc939f97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9340ceb40 0x7fc9341f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc939f97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9340cf340 0x7fc934145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc939f97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc93415b2a0 0x7fc93415dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc939f97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93415d910 con 0x7fc9340cf340 2023-10-18T10:44:47.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc939f97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93415cf70 con 0x7fc9340ceb40 2023-10-18T10:44:47.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc939f97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93415d250 con 0x7fc93415b2a0 2023-10-18T10:44:47.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc92affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9340cf340 0x7fc934145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc92affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9340cf340 0x7fc934145fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37790/0 (socket says 172.21.15.125:37790) 2023-10-18T10:44:47.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc92affd640 1 -- 172.21.15.125:0/1845557448 learned_addr learned my addr 172.21.15.125:0/1845557448 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:47.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc9337fe640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9340ceb40 0x7fc9341f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc933fff640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc93415b2a0 0x7fc93415dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc92affd640 1 -- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc93415b2a0 msgr2=0x7fc93415dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc92affd640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc93415b2a0 0x7fc93415dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc92affd640 1 -- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9340ceb40 msgr2=0x7fc9341f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:47.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.972+0000 7fc92affd640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9340ceb40 0x7fc9341f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:47.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.973+0000 7fc92affd640 1 -- 172.21.15.125:0/1845557448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c009560 con 0x7fc9340cf340 2023-10-18T10:44:47.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.973+0000 7fc933fff640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc93415b2a0 0x7fc93415dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:47.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.973+0000 7fc92affd640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9340cf340 0x7fc934145fe0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009850 tx=0x7fc91c02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:47.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.973+0000 7fc930ff9640 1 -- 172.21.15.125:0/1845557448 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c003800 con 0x7fc9340cf340 2023-10-18T10:44:47.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.973+0000 7fc930ff9640 1 -- 172.21.15.125:0/1845557448 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc91c003a50 con 0x7fc9340cf340 2023-10-18T10:44:47.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.973+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93415d530 con 0x7fc9340cf340 2023-10-18T10:44:47.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.973+0000 7fc930ff9640 1 -- 172.21.15.125:0/1845557448 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c003e20 con 0x7fc9340cf340 2023-10-18T10:44:47.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.973+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc934146520 con 0x7fc9340cf340 2023-10-18T10:44:47.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.974+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93415da50 con 0x7fc9340cf340 2023-10-18T10:44:47.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.974+0000 7fc930ff9640 1 -- 172.21.15.125:0/1845557448 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc91c03e450 con 0x7fc9340cf340 2023-10-18T10:44:47.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.975+0000 7fc930ff9640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc91406ac90 0x7fc91406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:47.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.975+0000 7fc930ff9640 1 -- 172.21.15.125:0/1845557448 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6788+0+0 (secure 0 0 0) 0x7fc91c0ac150 con 0x7fc9340cf340 2023-10-18T10:44:47.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.975+0000 7fc9337fe640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc91406ac90 0x7fc91406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:47.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.975+0000 7fc9337fe640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc91406ac90 0x7fc91406d150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc924000f60 tx=0x7fc924009040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:47.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:47.979+0000 7fc930ff9640 1 -- 172.21.15.125:0/1845557448 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc93415da50 con 0x7fc9340cf340 2023-10-18T10:44:48.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.179+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc93415da50 con 0x7fc9340cf340 2023-10-18T10:44:48.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.180+0000 7fc930ff9640 1 -- 172.21.15.125:0/1845557448 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fc93415da50 con 0x7fc9340cf340 2023-10-18T10:44:48.183 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:48.183 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":53,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7112,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/585529921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":585529921},{"type":"v1","addr":"172.21.15.125:6835","nonce":585529921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1680315524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1680315524},{"type":"v1","addr":"172.21.15.139:6837","nonce":1680315524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1127901065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1127901065},{"type":"v1","addr":"172.21.15.139:6836","nonce":1127901065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:20.891510+0000","modified":"2023-10-18T10:44:22.895207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9159":{"gid":9159,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2651847004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2651847004},{"type":"v1","addr":"172.21.15.125:6834","nonce":2651847004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T10:44:48.184 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 53 2023-10-18T10:44:48.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc91406ac90 msgr2=0x7fc91406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc91406ac90 0x7fc91406d150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc924000f60 tx=0x7fc924009040 comp rx=0 tx=0).stop 2023-10-18T10:44:48.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9340cf340 msgr2=0x7fc934145fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9340cf340 0x7fc934145fe0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009850 tx=0x7fc91c02dae0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 shutdown_connections 2023-10-18T10:44:48.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc91406ac90 0x7fc91406d150 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc93415b2a0 0x7fc93415dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9340cf340 0x7fc934145fe0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 --2- 172.21.15.125:0/1845557448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9340ceb40 0x7fc9341f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.181+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 >> 172.21.15.125:0/1845557448 conn(0x7fc9340d0d60 msgr2=0x7fc93406b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:48.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.182+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 shutdown_connections 2023-10-18T10:44:48.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.182+0000 7fc939f97640 1 -- 172.21.15.125:0/1845557448 wait complete. 2023-10-18T10:44:48.194 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:48.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.295+0000 7f44d2094640 1 -- 172.21.15.125:0/3424246564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc15b2a0 msgr2=0x7f44cc0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.295+0000 7f44d2094640 1 --2- 172.21.15.125:0/3424246564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc15b2a0 0x7f44cc0d1910 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f44c0005fd0 tx=0x7f44c002bcf0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.295+0000 7f44d2094640 1 -- 172.21.15.125:0/3424246564 shutdown_connections 2023-10-18T10:44:48.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.295+0000 7f44d2094640 1 --2- 172.21.15.125:0/3424246564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44cc0ceb40 0x7f44cc0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.295+0000 7f44d2094640 1 --2- 172.21.15.125:0/3424246564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc15b2a0 0x7f44cc0d1910 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.295+0000 7f44d2094640 1 -- 172.21.15.125:0/3424246564 >> 172.21.15.125:0/3424246564 conn(0x7f44cc0d0d60 msgr2=0x7f44cc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:48.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.295+0000 7f44d2094640 1 -- 172.21.15.125:0/3424246564 shutdown_connections 2023-10-18T10:44:48.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.295+0000 7f44d2094640 1 -- 172.21.15.125:0/3424246564 wait complete. 2023-10-18T10:44:48.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.296+0000 7f44d2094640 1 Processor -- start 2023-10-18T10:44:48.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.296+0000 7f44d2094640 1 -- start start 2023-10-18T10:44:48.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44d2094640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44cc0ceb40 0x7f44cc1f1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44d2094640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc0cf340 0x7f44cc15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44d2094640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44cc15b2a0 0x7f44cc0ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44d2094640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0b38e0 con 0x7f44cc0cf340 2023-10-18T10:44:48.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44d2094640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc14ecb0 con 0x7f44cc15b2a0 2023-10-18T10:44:48.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44d2094640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0b3b30 con 0x7f44cc0ceb40 2023-10-18T10:44:48.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44caffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc0cf340 0x7f44cc15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44caffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc0cf340 0x7f44cc15df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37808/0 (socket says 172.21.15.125:37808) 2023-10-18T10:44:48.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44caffd640 1 -- 172.21.15.125:0/427760713 learned_addr learned my addr 172.21.15.125:0/427760713 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:48.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44cbfff640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44cc15b2a0 0x7f44cc0ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.297+0000 7f44cb7fe640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44cc0ceb40 0x7f44cc1f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44caffd640 1 -- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44cc0ceb40 msgr2=0x7f44cc1f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44caffd640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44cc0ceb40 0x7f44cc1f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44caffd640 1 -- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44cc15b2a0 msgr2=0x7f44cc0ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44caffd640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44cc15b2a0 0x7f44cc0ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44caffd640 1 -- 172.21.15.125:0/427760713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c0009560 con 0x7f44cc0cf340 2023-10-18T10:44:48.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44caffd640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc0cf340 0x7f44cc15df50 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f44c00099f0 tx=0x7f44c002dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:48.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44b3fff640 1 -- 172.21.15.125:0/427760713 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0005650 con 0x7f44cc0cf340 2023-10-18T10:44:48.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44b3fff640 1 -- 172.21.15.125:0/427760713 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f44c00038f0 con 0x7f44cc0cf340 2023-10-18T10:44:48.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc0b0d50 con 0x7f44cc0cf340 2023-10-18T10:44:48.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44b3fff640 1 -- 172.21.15.125:0/427760713 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0003d00 con 0x7f44cc0cf340 2023-10-18T10:44:48.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.298+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc0b40f0 con 0x7f44cc0cf340 2023-10-18T10:44:48.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.299+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc0d04f0 con 0x7f44cc0cf340 2023-10-18T10:44:48.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.299+0000 7f44b3fff640 1 -- 172.21.15.125:0/427760713 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f44c003e450 con 0x7f44cc0cf340 2023-10-18T10:44:48.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.300+0000 7f44b3fff640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f44b806ac90 0x7f44b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.300+0000 7f44b3fff640 1 -- 172.21.15.125:0/427760713 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6788+0+0 (secure 0 0 0) 0x7f44c00abf30 con 0x7f44cc0cf340 2023-10-18T10:44:48.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.300+0000 7f44cb7fe640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f44b806ac90 0x7f44b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.300+0000 7f44cb7fe640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f44b806ac90 0x7f44b806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f44bc004f60 tx=0x7f44bc007e30 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:48.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.304+0000 7f44b3fff640 1 -- 172.21.15.125:0/427760713 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44c00770f0 con 0x7f44cc0cf340 2023-10-18T10:44:48.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.511+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44cc0690e0 con 0x7f44cc0cf340 2023-10-18T10:44:48.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.511+0000 7f44b3fff640 1 -- 172.21.15.125:0/427760713 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f44c0003e60 con 0x7f44cc0cf340 2023-10-18T10:44:48.514 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:48.515 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":53,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7112,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/585529921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":585529921},{"type":"v1","addr":"172.21.15.125:6835","nonce":585529921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1680315524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1680315524},{"type":"v1","addr":"172.21.15.139:6837","nonce":1680315524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1127901065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1127901065},{"type":"v1","addr":"172.21.15.139:6836","nonce":1127901065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:20.891510+0000","modified":"2023-10-18T10:44:22.895207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9159},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9159":{"gid":9159,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2651847004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2651847004},{"type":"v1","addr":"172.21.15.125:6834","nonce":2651847004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T10:44:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 53 2023-10-18T10:44:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f44b806ac90 msgr2=0x7f44b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f44b806ac90 0x7f44b806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f44bc004f60 tx=0x7f44bc007e30 comp rx=0 tx=0).stop 2023-10-18T10:44:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc0cf340 msgr2=0x7f44cc15df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc0cf340 0x7f44cc15df50 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f44c00099f0 tx=0x7f44c002dcc0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 shutdown_connections 2023-10-18T10:44:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f44b806ac90 0x7f44b806d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44cc15b2a0 0x7f44cc0ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44cc0cf340 0x7f44cc15df50 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 --2- 172.21.15.125:0/427760713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44cc0ceb40 0x7f44cc1f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 >> 172.21.15.125:0/427760713 conn(0x7f44cc0d0d60 msgr2=0x7f44cc06a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 shutdown_connections 2023-10-18T10:44:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.513+0000 7f44d2094640 1 -- 172.21.15.125:0/427760713 wait complete. 2023-10-18T10:44:48.526 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:48.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.626+0000 7ff6bb4af640 1 -- 172.21.15.125:0/497146495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b40ab500 msgr2=0x7ff6b40ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.626+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/497146495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b40ab500 0x7ff6b40ab9b0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac005fd0 tx=0x7ff6ac02c240 comp rx=0 tx=0).stop 2023-10-18T10:44:48.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.626+0000 7ff6bb4af640 1 -- 172.21.15.125:0/497146495 shutdown_connections 2023-10-18T10:44:48.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.626+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/497146495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b40ab500 0x7ff6b40ab9b0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.626+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/497146495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b40c9dc0 0x7ff6b406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.626+0000 7ff6bb4af640 1 -- 172.21.15.125:0/497146495 >> 172.21.15.125:0/497146495 conn(0x7ff6b4069930 msgr2=0x7ff6b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:48.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.627+0000 7ff6bb4af640 1 -- 172.21.15.125:0/497146495 shutdown_connections 2023-10-18T10:44:48.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.627+0000 7ff6bb4af640 1 -- 172.21.15.125:0/497146495 wait complete. 2023-10-18T10:44:48.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6bb4af640 1 Processor -- start 2023-10-18T10:44:48.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6bb4af640 1 -- start start 2023-10-18T10:44:48.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6bb4af640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b406af00 0x7ff6b40b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6bb4af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b40ab500 0x7ff6b414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6bb4af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b40c9dc0 0x7ff6b415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6bb4af640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40af5d0 con 0x7ff6b406af00 2023-10-18T10:44:48.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6bb4af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40ae700 con 0x7ff6b40c9dc0 2023-10-18T10:44:48.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6bb4af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40b1600 con 0x7ff6b40ab500 2023-10-18T10:44:48.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6b924b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b406af00 0x7ff6b40b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6b924b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b406af00 0x7ff6b40b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37832/0 (socket says 172.21.15.125:37832) 2023-10-18T10:44:48.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.628+0000 7ff6b924b640 1 -- 172.21.15.125:0/3816080925 learned_addr learned my addr 172.21.15.125:0/3816080925 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:48.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b9a4c640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b40c9dc0 0x7ff6b415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b8a4a640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b40ab500 0x7ff6b414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b924b640 1 -- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b40ab500 msgr2=0x7ff6b414ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b924b640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b40ab500 0x7ff6b414ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b924b640 1 -- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b40c9dc0 msgr2=0x7ff6b415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b924b640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b40c9dc0 0x7ff6b415a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b924b640 1 -- 172.21.15.125:0/3816080925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac009900 con 0x7ff6b406af00 2023-10-18T10:44:48.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b9a4c640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b40c9dc0 0x7ff6b415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:48.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b8a4a640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b40ab500 0x7ff6b414ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:48.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6b924b640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b406af00 0x7ff6b40b10c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c430 tx=0x7ff6b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:48.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6a9ffb640 1 -- 172.21.15.125:0/3816080925 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0011790 con 0x7ff6b406af00 2023-10-18T10:44:48.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6a9ffb640 1 -- 172.21.15.125:0/3816080925 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff6b00119e0 con 0x7ff6b406af00 2023-10-18T10:44:48.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b414f340 con 0x7ff6b406af00 2023-10-18T10:44:48.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6a9ffb640 1 -- 172.21.15.125:0/3816080925 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0011e30 con 0x7ff6b406af00 2023-10-18T10:44:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.629+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b40aec30 con 0x7ff6b406af00 2023-10-18T10:44:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.630+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b40690e0 con 0x7ff6b406af00 2023-10-18T10:44:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.631+0000 7ff6a9ffb640 1 -- 172.21.15.125:0/3816080925 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff6b001c460 con 0x7ff6b406af00 2023-10-18T10:44:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.631+0000 7ff6a9ffb640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff69806acc0 0x7ff69806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.631+0000 7ff6a9ffb640 1 -- 172.21.15.125:0/3816080925 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6788+0+0 (secure 0 0 0) 0x7ff6b00898b0 con 0x7ff6b406af00 2023-10-18T10:44:48.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.631+0000 7ff6b8a4a640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff69806acc0 0x7ff69806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.632+0000 7ff6b8a4a640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff69806acc0 0x7ff69806d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005fd0 tx=0x7ff6a4005ea0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:48.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.635+0000 7ff6a9ffb640 1 -- 172.21.15.125:0/3816080925 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6b0054860 con 0x7ff6b406af00 2023-10-18T10:44:48.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.800+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6b40cc420 con 0x7ff6b406af00 2023-10-18T10:44:48.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.801+0000 7ff6a9ffb640 1 -- 172.21.15.125:0/3816080925 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17953 (secure 0 0 0) 0x7ff6b0057e80 con 0x7ff6b406af00 2023-10-18T10:44:48.803 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:48.804 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":145,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:46.834541+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:18.717169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:19.861005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2424959739":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/848644581":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/1809262820":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6833/427428567":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6832/427428567":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/2263683535":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/3114574713":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/3295054028":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1133703870":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/4009278606":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/494329198":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/2843740551":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6833/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:6832/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/3297526571":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/337665570":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1274722331":"2023-10-19T10:44:36.950527+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:48.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff69806acc0 msgr2=0x7ff69806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff69806acc0 0x7ff69806d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005fd0 tx=0x7ff6a4005ea0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b406af00 msgr2=0x7ff6b40b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b406af00 0x7ff6b40b10c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c430 tx=0x7ff6b000c8f0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 shutdown_connections 2023-10-18T10:44:48.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff69806acc0 0x7ff69806d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6b40c9dc0 0x7ff6b415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6b40ab500 0x7ff6b414ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 --2- 172.21.15.125:0/3816080925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6b406af00 0x7ff6b40b10c0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 >> 172.21.15.125:0/3816080925 conn(0x7ff6b4069930 msgr2=0x7ff6b40cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:48.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 shutdown_connections 2023-10-18T10:44:48.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.803+0000 7ff6bb4af640 1 -- 172.21.15.125:0/3816080925 wait complete. 2023-10-18T10:44:48.816 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T10:44:48.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.923+0000 7f1efee87640 1 -- 172.21.15.125:0/4138973293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 msgr2=0x7f1ee0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.923+0000 7f1efee87640 1 --2- 172.21.15.125:0/4138973293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 0x7f1ee0003dc0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8006fd0 tx=0x7f1ee00057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.923+0000 7f1efd424640 1 -- 172.21.15.125:0/4138973293 reap_dead start 2023-10-18T10:44:48.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.923+0000 7f1efee87640 1 -- 172.21.15.125:0/4138973293 shutdown_connections 2023-10-18T10:44:48.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.923+0000 7f1efee87640 1 -- 172.21.15.125:0/4138973293 >> 172.21.15.125:0/4138973293 conn(0x7f1ef80d0d60 msgr2=0x7f1ef80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:48.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.923+0000 7f1efee87640 1 -- 172.21.15.125:0/4138973293 shutdown_connections 2023-10-18T10:44:48.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.923+0000 7f1efee87640 1 -- 172.21.15.125:0/4138973293 wait complete. 2023-10-18T10:44:48.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.924+0000 7f1efee87640 1 Processor -- start 2023-10-18T10:44:48.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.924+0000 7f1efee87640 1 -- start start 2023-10-18T10:44:48.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efee87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 0x7f1ef8156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efee87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef81f1680 0x7f1ef80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efee87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ef815e140 0x7f1ef806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efee87640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef815e5f0 con 0x7f1ee00039a0 2023-10-18T10:44:48.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efee87640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef80caac0 con 0x7f1ef81f1680 2023-10-18T10:44:48.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efee87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef815b900 con 0x7f1ef815e140 2023-10-18T10:44:48.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 0x7f1ef8156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 0x7f1ef8156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37844/0 (socket says 172.21.15.125:37844) 2023-10-18T10:44:48.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 -- 172.21.15.125:0/3321381543 learned_addr learned my addr 172.21.15.125:0/3321381543 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:48.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1ef7fff640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef81f1680 0x7f1ef80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efd424640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ef815e140 0x7f1ef806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 -- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ef815e140 msgr2=0x7f1ef806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ef815e140 0x7f1ef806b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 -- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef81f1680 msgr2=0x7f1ef80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:48.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef81f1680 0x7f1ef80ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:48.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 -- 172.21.15.125:0/3321381543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef815bbe0 con 0x7f1ee00039a0 2023-10-18T10:44:48.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1ef7fff640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef81f1680 0x7f1ef80ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:48.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efd424640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ef815e140 0x7f1ef806b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:48.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efcc23640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 0x7f1ef8156670 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1ee802c900 tx=0x7f1ee80016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:48.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1ef57fa640 1 -- 172.21.15.125:0/3321381543 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8006230 con 0x7f1ee00039a0 2023-10-18T10:44:48.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1ef57fa640 1 -- 172.21.15.125:0/3321381543 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1ee8006390 con 0x7f1ee00039a0 2023-10-18T10:44:48.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef815bec0 con 0x7f1ee00039a0 2023-10-18T10:44:48.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1ef57fa640 1 -- 172.21.15.125:0/3321381543 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80035f0 con 0x7f1ee00039a0 2023-10-18T10:44:48.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.925+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef806b710 con 0x7f1ee00039a0 2023-10-18T10:44:48.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.927+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef80690e0 con 0x7f1ee00039a0 2023-10-18T10:44:48.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.927+0000 7f1ef57fa640 1 -- 172.21.15.125:0/3321381543 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f1ee8031030 con 0x7f1ee00039a0 2023-10-18T10:44:48.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.927+0000 7f1ef57fa640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1ee406acc0 0x7f1ee406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:48.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.927+0000 7f1ef57fa640 1 -- 172.21.15.125:0/3321381543 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6788+0+0 (secure 0 0 0) 0x7f1ee80aa620 con 0x7f1ee00039a0 2023-10-18T10:44:48.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.928+0000 7f1ef7fff640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1ee406acc0 0x7f1ee406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:48.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.928+0000 7f1ef7fff640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1ee406acc0 0x7f1ee406d180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80c7760 tx=0x7f1eec00b260 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:48.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:48.931+0000 7f1ef57fa640 1 -- 172.21.15.125:0/3321381543 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ee8075550 con 0x7f1ee00039a0 2023-10-18T10:44:49.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.132+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1ef80d0b40 con 0x7f1ee00039a0 2023-10-18T10:44:49.135 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T10:44:49.133+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T10:44:49.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.134+0000 7f1ef57fa640 1 -- 172.21.15.125:0/3321381543 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1ee8078b70 con 0x7f1ee00039a0 2023-10-18T10:44:49.136 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T10:44:49.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1ee406acc0 msgr2=0x7f1ee406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:49.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1ee406acc0 0x7f1ee406d180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80c7760 tx=0x7f1eec00b260 comp rx=0 tx=0).stop 2023-10-18T10:44:49.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 msgr2=0x7f1ef8156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:49.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 0x7f1ef8156670 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1ee802c900 tx=0x7f1ee80016e0 comp rx=0 tx=0).stop 2023-10-18T10:44:49.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 shutdown_connections 2023-10-18T10:44:49.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1ee406acc0 0x7f1ee406d180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:49.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ef81f1680 0x7f1ef80ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:49.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ef815e140 0x7f1ef806b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:49.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 --2- 172.21.15.125:0/3321381543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ee00039a0 0x7f1ef8156670 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:49.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 >> 172.21.15.125:0/3321381543 conn(0x7f1ef80d0d60 msgr2=0x7f1ef815a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:49.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 shutdown_connections 2023-10-18T10:44:49.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.136+0000 7f1efee87640 1 -- 172.21.15.125:0/3321381543 wait complete. 2023-10-18T10:44:49.142 INFO:tasks.ceph.mds.a.smithi125.stderr: -8718> 2023-10-18T10:44:26.525+0000 7fc383161640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:44:49.150 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T10:44:49.174 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:44:49.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.253+0000 7f4910325640 1 -- 172.21.15.125:0/3735690864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 msgr2=0x7f48f4004eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:49.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.253+0000 7f4910325640 1 --2- 172.21.15.125:0/3735690864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 0x7f48f4004eb0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c6c0 tx=0x7f48f4006900 comp rx=0 tx=0).stop 2023-10-18T10:44:49.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.253+0000 7f490e8c2640 1 -- 172.21.15.125:0/3735690864 reap_dead start 2023-10-18T10:44:49.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.253+0000 7f4910325640 1 -- 172.21.15.125:0/3735690864 shutdown_connections 2023-10-18T10:44:49.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.253+0000 7f4910325640 1 -- 172.21.15.125:0/3735690864 >> 172.21.15.125:0/3735690864 conn(0x7f4908069930 msgr2=0x7f4908069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:49.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.253+0000 7f4910325640 1 -- 172.21.15.125:0/3735690864 shutdown_connections 2023-10-18T10:44:49.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.253+0000 7f4910325640 1 -- 172.21.15.125:0/3735690864 wait complete. 2023-10-18T10:44:49.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.254+0000 7f4910325640 1 Processor -- start 2023-10-18T10:44:49.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.254+0000 7f4910325640 1 -- start start 2023-10-18T10:44:49.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f4910325640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 0x7f4908146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:49.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f4910325640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f490814a810 0x7f490814ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:49.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f4910325640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490815ae00 0x7f490815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:49.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f4910325640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490815b7f0 con 0x7f48f4004ab0 2023-10-18T10:44:49.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f4910325640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490814d980 con 0x7f490815ae00 2023-10-18T10:44:49.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f4910325640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4908146ac0 con 0x7f490814a810 2023-10-18T10:44:49.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f490e0c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 0x7f4908146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:49.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f490e0c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 0x7f4908146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37890/0 (socket says 172.21.15.125:37890) 2023-10-18T10:44:49.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f490e0c1640 1 -- 172.21.15.125:0/2963806874 learned_addr learned my addr 172.21.15.125:0/2963806874 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:49.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f490e0c1640 1 -- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f490814a810 msgr2=0x7f490814ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:49.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f490e0c1640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f490814a810 0x7f490814ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:49.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f490e0c1640 1 -- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490815ae00 msgr2=0x7f490815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:49.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.255+0000 7f490e0c1640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490815ae00 0x7f490815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:49.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.256+0000 7f490e0c1640 1 -- 172.21.15.125:0/2963806874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490814d3c0 con 0x7f48f4004ab0 2023-10-18T10:44:49.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.256+0000 7f490e0c1640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 0x7f4908146580 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f48fc031040 tx=0x7f48fc006190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:49.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.256+0000 7f48faffd640 1 -- 172.21.15.125:0/2963806874 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc03d830 con 0x7f48f4004ab0 2023-10-18T10:44:49.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.256+0000 7f48faffd640 1 -- 172.21.15.125:0/2963806874 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f48fc03da80 con 0x7f48f4004ab0 2023-10-18T10:44:49.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.256+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490814d6a0 con 0x7f48f4004ab0 2023-10-18T10:44:49.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.256+0000 7f48faffd640 1 -- 172.21.15.125:0/2963806874 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc03de90 con 0x7f48f4004ab0 2023-10-18T10:44:49.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.256+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4908147a20 con 0x7f48f4004ab0 2023-10-18T10:44:49.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.257+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49081475a0 con 0x7f48f4004ab0 2023-10-18T10:44:49.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.257+0000 7f48faffd640 1 -- 172.21.15.125:0/2963806874 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 79) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f48fc043070 con 0x7f48f4004ab0 2023-10-18T10:44:49.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.257+0000 7f48faffd640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f48ec06acc0 0x7f48ec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:49.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.258+0000 7f48faffd640 1 -- 172.21.15.125:0/2963806874 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6874+0+0 (secure 0 0 0) 0x7f48fc035070 con 0x7f48f4004ab0 2023-10-18T10:44:49.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.258+0000 7f490d8c0640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f48ec06acc0 0x7f48ec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:49.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.258+0000 7f490d8c0640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f48ec06acc0 0x7f48ec06d180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4904001eb0 tx=0x7f4904001150 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:49.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.262+0000 7f48faffd640 1 -- 172.21.15.125:0/2963806874 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49081475a0 con 0x7f48f4004ab0 2023-10-18T10:44:49.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.466+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f49081475a0 con 0x7f48f4004ab0 2023-10-18T10:44:49.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:49.855+0000 7f48faffd640 1 -- 172.21.15.125:0/2963806874 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 80) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f48fc037070 con 0x7f48f4004ab0 2023-10-18T10:44:50.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.138+0000 7f48faffd640 1 -- 172.21.15.125:0/2963806874 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f48fc092640 con 0x7f48f4004ab0 2023-10-18T10:44:50.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f48ec06acc0 msgr2=0x7f48ec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:50.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f48ec06acc0 0x7f48ec06d180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4904001eb0 tx=0x7f4904001150 comp rx=0 tx=0).stop 2023-10-18T10:44:50.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 msgr2=0x7f4908146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:50.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 0x7f4908146580 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f48fc031040 tx=0x7f48fc006190 comp rx=0 tx=0).stop 2023-10-18T10:44:50.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 shutdown_connections 2023-10-18T10:44:50.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f48ec06acc0 0x7f48ec06d180 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:50.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490815ae00 0x7f490815b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:50.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f490814a810 0x7f490814ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:50.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 --2- 172.21.15.125:0/2963806874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f48f4004ab0 0x7f4908146580 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:50.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 >> 172.21.15.125:0/2963806874 conn(0x7f4908069930 msgr2=0x7f490806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:50.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 shutdown_connections 2023-10-18T10:44:50.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.141+0000 7f4910325640 1 -- 172.21.15.125:0/2963806874 wait complete. 2023-10-18T10:44:50.156 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T10:44:50.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.307+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2530633553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73a406b030 msgr2=0x7f73a406b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:50.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.307+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2530633553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73a406b030 0x7f73a406b410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005fd0 tx=0x7f73a002c240 comp rx=0 tx=0).stop 2023-10-18T10:44:50.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.307+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2530633553 shutdown_connections 2023-10-18T10:44:50.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.307+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2530633553 >> v2:172.21.15.139:3300/0 conn(0x7f73a40c9750 0x7f73a40c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:50.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.307+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2530633553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73a406b030 0x7f73a406b410 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:50.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.307+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2530633553 >> 172.21.15.125:0/2530633553 conn(0x7f73a4069930 msgr2=0x7f73a4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:50.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.307+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2530633553 shutdown_connections 2023-10-18T10:44:50.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.307+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2530633553 wait complete. 2023-10-18T10:44:50.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73ac3e9640 1 Processor -- start 2023-10-18T10:44:50.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73ac3e9640 1 -- start start 2023-10-18T10:44:50.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73ac3e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73a406b030 0x7f73a41365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:50.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73ac3e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73a40c9750 0x7f73a4138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:50.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73ac3e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a4159bd0 0x7f73a4138770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:50.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73ac3e9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40709f0 con 0x7f73a4159bd0 2023-10-18T10:44:50.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73ac3e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a406fe70 con 0x7f73a406b030 2023-10-18T10:44:50.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73ac3e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a4070150 con 0x7f73a40c9750 2023-10-18T10:44:50.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73aa986640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a4159bd0 0x7f73a4138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:50.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73aa986640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a4159bd0 0x7f73a4138770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37906/0 (socket says 172.21.15.125:37906) 2023-10-18T10:44:50.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73aa986640 1 -- 172.21.15.125:0/2420824222 learned_addr learned my addr 172.21.15.125:0/2420824222 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:50.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.309+0000 7f73aa185640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73a406b030 0x7f73a41365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:50.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73a9984640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73a40c9750 0x7f73a4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:50.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73aa986640 1 -- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73a40c9750 msgr2=0x7f73a4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:50.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73aa986640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73a40c9750 0x7f73a4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:50.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73aa986640 1 -- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73a406b030 msgr2=0x7f73a41365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:50.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73aa986640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73a406b030 0x7f73a41365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:50.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73aa986640 1 -- 172.21.15.125:0/2420824222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c009490 con 0x7f73a4159bd0 2023-10-18T10:44:50.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73aa986640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a4159bd0 0x7f73a4138770 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f739c00e9b0 tx=0x7f739c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:50.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f739affd640 1 -- 172.21.15.125:0/2420824222 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00cd80 con 0x7f73a4159bd0 2023-10-18T10:44:50.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f739affd640 1 -- 172.21.15.125:0/2420824222 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f739c004020 con 0x7f73a4159bd0 2023-10-18T10:44:50.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a0009900 con 0x7f73a4159bd0 2023-10-18T10:44:50.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f739affd640 1 -- 172.21.15.125:0/2420824222 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00cee0 con 0x7f73a4159bd0 2023-10-18T10:44:50.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.310+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a4070710 con 0x7f73a4159bd0 2023-10-18T10:44:50.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.311+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a4150d00 con 0x7f73a4159bd0 2023-10-18T10:44:50.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.312+0000 7f739affd640 1 -- 172.21.15.125:0/2420824222 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 80) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f739c0035f0 con 0x7f73a4159bd0 2023-10-18T10:44:50.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.312+0000 7f739affd640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f738806aed0 0x7f738806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:50.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.312+0000 7f739affd640 1 -- 172.21.15.125:0/2420824222 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6874+0+0 (secure 0 0 0) 0x7f739c014070 con 0x7f73a4159bd0 2023-10-18T10:44:50.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.312+0000 7f73aa185640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f738806aed0 0x7f738806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:50.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.313+0000 7f73aa185640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f738806aed0 0x7f738806d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f73a0036000 tx=0x7f73a0004ab0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:50.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.316+0000 7f739affd640 1 -- 172.21.15.125:0/2420824222 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f739c058ec0 con 0x7f73a4159bd0 2023-10-18T10:44:50.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:50.484+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73a406c290 con 0x7f73a4159bd0 2023-10-18T10:44:51.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.472+0000 7f739affd640 1 -- 172.21.15.125:0/2420824222 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7f739c058a50 con 0x7f73a4159bd0 2023-10-18T10:44:51.475 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T10:44:51.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f738806aed0 msgr2=0x7f738806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:51.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f738806aed0 0x7f738806d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f73a0036000 tx=0x7f73a0004ab0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a4159bd0 msgr2=0x7f73a4138770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a4159bd0 0x7f73a4138770 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f739c00e9b0 tx=0x7f739c00ee70 comp rx=0 tx=0).stop 2023-10-18T10:44:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 shutdown_connections 2023-10-18T10:44:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f738806aed0 0x7f738806d390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a4159bd0 0x7f73a4138770 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73a40c9750 0x7f73a4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 --2- 172.21.15.125:0/2420824222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73a406b030 0x7f73a41365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 >> 172.21.15.125:0/2420824222 conn(0x7f73a4069930 msgr2=0x7f73a414fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:51.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 shutdown_connections 2023-10-18T10:44:51.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.475+0000 7f73ac3e9640 1 -- 172.21.15.125:0/2420824222 wait complete. 2023-10-18T10:44:51.492 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T10:44:51.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.598+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/112892322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140cbd20 msgr2=0x7f5d14158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.598+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/112892322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140cbd20 0x7f5d14158300 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f5d140b10f0 tx=0x7f5d0802be40 comp rx=0 tx=0).stop 2023-10-18T10:44:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.598+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/112892322 shutdown_connections 2023-10-18T10:44:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.598+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/112892322 >> v2:172.21.15.139:3300/0 conn(0x7f5d140b0530 0x7f5d140b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.598+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/112892322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140cbd20 0x7f5d14158300 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.598+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/112892322 >> 172.21.15.125:0/112892322 conn(0x7f5d14069930 msgr2=0x7f5d14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:51.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.598+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/112892322 shutdown_connections 2023-10-18T10:44:51.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.598+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/112892322 wait complete. 2023-10-18T10:44:51.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.599+0000 7f5d1a3a9640 1 Processor -- start 2023-10-18T10:44:51.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d1a3a9640 1 -- start start 2023-10-18T10:44:51.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d1a3a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140afa20 0x7f5d14156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:51.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d1a3a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5d140b0530 0x7f5d14145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:51.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d1a3a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d140cbd20 0x7f5d141438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:51.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d1a3a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14149b00 con 0x7f5d140afa20 2023-10-18T10:44:51.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d1a3a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14144c20 con 0x7f5d140cbd20 2023-10-18T10:44:51.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d1a3a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14144f00 con 0x7f5d140b0530 2023-10-18T10:44:51.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d13fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140afa20 0x7f5d14156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:51.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d13fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140afa20 0x7f5d14156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37920/0 (socket says 172.21.15.125:37920) 2023-10-18T10:44:51.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.600+0000 7f5d13fff640 1 -- 172.21.15.125:0/3778886029 learned_addr learned my addr 172.21.15.125:0/3778886029 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:51.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d13fff640 1 -- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5d140b0530 msgr2=0x7f5d14145700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:51.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d13fff640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5d140b0530 0x7f5d14145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d18946640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d140cbd20 0x7f5d141438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:51.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d13fff640 1 -- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d140cbd20 msgr2=0x7f5d141438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:51.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d13fff640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d140cbd20 0x7f5d141438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:51.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d13fff640 1 -- 172.21.15.125:0/3778886029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08009560 con 0x7f5d140afa20 2023-10-18T10:44:51.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d13fff640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140afa20 0x7f5d14156530 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f5d08036040 tx=0x7f5d0802de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:51.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d10ff9640 1 -- 172.21.15.125:0/3778886029 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08005d50 con 0x7f5d140afa20 2023-10-18T10:44:51.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d10ff9640 1 -- 172.21.15.125:0/3778886029 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5d08003910 con 0x7f5d140afa20 2023-10-18T10:44:51.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d10ff9640 1 -- 172.21.15.125:0/3778886029 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08003d20 con 0x7f5d140afa20 2023-10-18T10:44:51.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d141451e0 con 0x7f5d140afa20 2023-10-18T10:44:51.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.601+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d14156c50 con 0x7f5d140afa20 2023-10-18T10:44:51.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.602+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d14144a80 con 0x7f5d140afa20 2023-10-18T10:44:51.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.603+0000 7f5d10ff9640 1 -- 172.21.15.125:0/3778886029 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5d0803e450 con 0x7f5d140afa20 2023-10-18T10:44:51.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.603+0000 7f5d10ff9640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f5d0006af90 0x7f5d0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:51.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.603+0000 7f5d10ff9640 1 -- 172.21.15.125:0/3778886029 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6436+0+0 (secure 0 0 0) 0x7f5d080ac290 con 0x7f5d140afa20 2023-10-18T10:44:51.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.603+0000 7f5d137fe640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f5d0006af90 0x7f5d0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:51.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.604+0000 7f5d137fe640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f5d0006af90 0x7f5d0006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5d04000f60 tx=0x7f5d040079d0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:51.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.607+0000 7f5d10ff9640 1 -- 172.21.15.125:0/3778886029 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d14144a80 con 0x7f5d140afa20 2023-10-18T10:44:51.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:51.782+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d14156c50 con 0x7f5d140afa20 2023-10-18T10:44:52.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.474+0000 7f5d10ff9640 1 -- 172.21.15.125:0/3778886029 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5d14156c50 con 0x7f5d140afa20 2023-10-18T10:44:52.476 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T10:44:52.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f5d0006af90 msgr2=0x7f5d0006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f5d0006af90 0x7f5d0006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5d04000f60 tx=0x7f5d040079d0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140afa20 msgr2=0x7f5d14156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140afa20 0x7f5d14156530 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f5d08036040 tx=0x7f5d0802de10 comp rx=0 tx=0).stop 2023-10-18T10:44:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 shutdown_connections 2023-10-18T10:44:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f5d0006af90 0x7f5d0006d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d140cbd20 0x7f5d141438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5d140b0530 0x7f5d14145700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 --2- 172.21.15.125:0/3778886029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d140afa20 0x7f5d14156530 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.476+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 >> 172.21.15.125:0/3778886029 conn(0x7f5d14069930 msgr2=0x7f5d14069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:52.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.477+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 shutdown_connections 2023-10-18T10:44:52.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.477+0000 7f5d1a3a9640 1 -- 172.21.15.125:0/3778886029 wait complete. 2023-10-18T10:44:52.491 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T10:44:52.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.605+0000 7f51e18da640 1 -- 172.21.15.125:0/3349207502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0cf8b0 msgr2=0x7f51dc0b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.605+0000 7f51e18da640 1 --2- 172.21.15.125:0/3349207502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0cf8b0 0x7f51dc0b42d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f51d4005fd0 tx=0x7f51d402c2e0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.606+0000 7f51e18da640 1 -- 172.21.15.125:0/3349207502 shutdown_connections 2023-10-18T10:44:52.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.606+0000 7f51e18da640 1 --2- 172.21.15.125:0/3349207502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0cf8b0 0x7f51dc0b42d0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.606+0000 7f51e18da640 1 --2- 172.21.15.125:0/3349207502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51dc15e5e0 0x7f51dc0ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.606+0000 7f51e18da640 1 -- 172.21.15.125:0/3349207502 >> 172.21.15.125:0/3349207502 conn(0x7f51dc0d0c50 msgr2=0x7f51dc0d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:52.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.606+0000 7f51e18da640 1 -- 172.21.15.125:0/3349207502 shutdown_connections 2023-10-18T10:44:52.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.606+0000 7f51e18da640 1 -- 172.21.15.125:0/3349207502 wait complete. 2023-10-18T10:44:52.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.607+0000 7f51e18da640 1 Processor -- start 2023-10-18T10:44:52.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.607+0000 7f51e18da640 1 -- start start 2023-10-18T10:44:52.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.607+0000 7f51e18da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0b1ca0 0x7f51dc14df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:52.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51e18da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51dc0cf8b0 0x7f51dc0ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:52.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51e18da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51dc15e5e0 0x7f51dc06b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:52.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51e18da640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0c8e60 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51e18da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc14e4d0 con 0x7f51dc0cf8b0 2023-10-18T10:44:52.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51e18da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0c8250 con 0x7f51dc15e5e0 2023-10-18T10:44:52.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0b1ca0 0x7f51dc14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:52.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0b1ca0 0x7f51dc14df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37928/0 (socket says 172.21.15.125:37928) 2023-10-18T10:44:52.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 -- 172.21.15.125:0/815864313 learned_addr learned my addr 172.21.15.125:0/815864313 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:52.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51da7fc640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51dc0cf8b0 0x7f51dc0ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:52.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51db7fe640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51dc15e5e0 0x7f51dc06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:52.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 -- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51dc15e5e0 msgr2=0x7f51dc06b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51dc15e5e0 0x7f51dc06b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 -- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51dc0cf8b0 msgr2=0x7f51dc0ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51dc0cf8b0 0x7f51dc0ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 -- 172.21.15.125:0/815864313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4009900 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51daffd640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0b1ca0 0x7f51dc14df90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00d740 tx=0x7f51cc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:52.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51c3fff640 1 -- 172.21.15.125:0/815864313 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc009c30 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51c3fff640 1 -- 172.21.15.125:0/815864313 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f51cc009e80 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc0c8530 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51c3fff640 1 -- 172.21.15.125:0/815864313 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0106c0 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.608+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc0c8af0 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.609+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc0690e0 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.610+0000 7f51c3fff640 1 -- 172.21.15.125:0/815864313 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f51cc010820 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.610+0000 7f51c3fff640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f51c806af90 0x7f51c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:52.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.610+0000 7f51c3fff640 1 -- 172.21.15.125:0/815864313 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7f51cc089f10 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.610+0000 7f51da7fc640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f51c806af90 0x7f51c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:52.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.611+0000 7f51da7fc640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f51c806af90 0x7f51c806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51dc1f4710 tx=0x7f51d0009040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:52.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.614+0000 7f51c3fff640 1 -- 172.21.15.125:0/815864313 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51cc054c00 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.781+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7f51dc069ca0 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.785+0000 7f51c3fff640 1 -- 172.21.15.125:0/815864313 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v58) v1 ==== 71+0+0 (secure 0 0 0) 0x7f51cc017080 con 0x7f51dc0b1ca0 2023-10-18T10:44:52.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.786+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f51c806af90 msgr2=0x7f51c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.786+0000 7f51e18da640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f51c806af90 0x7f51c806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51dc1f4710 tx=0x7f51d0009040 comp rx=0 tx=0).stop 2023-10-18T10:44:52.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.786+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0b1ca0 msgr2=0x7f51dc14df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.786+0000 7f51e18da640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0b1ca0 0x7f51dc14df90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00d740 tx=0x7f51cc00dc00 comp rx=0 tx=0).stop 2023-10-18T10:44:52.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.787+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 shutdown_connections 2023-10-18T10:44:52.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.787+0000 7f51e18da640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f51c806af90 0x7f51c806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.787+0000 7f51e18da640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51dc0cf8b0 0x7f51dc0ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.787+0000 7f51e18da640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51dc15e5e0 0x7f51dc06b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.787+0000 7f51e18da640 1 --2- 172.21.15.125:0/815864313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51dc0b1ca0 0x7f51dc14df90 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.787+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 >> 172.21.15.125:0/815864313 conn(0x7f51dc0d0c50 msgr2=0x7f51dc15af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:52.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.787+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 shutdown_connections 2023-10-18T10:44:52.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.787+0000 7f51e18da640 1 -- 172.21.15.125:0/815864313 wait complete. 2023-10-18T10:44:52.799 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-18T10:44:52.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.901+0000 7efdbccc2640 1 -- 172.21.15.125:0/2614354555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda4003970 msgr2=0x7efda4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.901+0000 7efdbccc2640 1 --2- 172.21.15.125:0/2614354555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda4003970 0x7efda4003d90 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7efda8001680 tx=0x7efda40057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.901+0000 7efdbccc2640 1 -- 172.21.15.125:0/2614354555 shutdown_connections 2023-10-18T10:44:52.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.901+0000 7efdbccc2640 1 --2- 172.21.15.125:0/2614354555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda4003970 0x7efda4003d90 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.901+0000 7efdbccc2640 1 -- 172.21.15.125:0/2614354555 >> 172.21.15.125:0/2614354555 conn(0x7efdb8069930 msgr2=0x7efdb8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:52.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.901+0000 7efdbccc2640 1 -- 172.21.15.125:0/2614354555 shutdown_connections 2023-10-18T10:44:52.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.901+0000 7efdbccc2640 1 -- 172.21.15.125:0/2614354555 wait complete. 2023-10-18T10:44:52.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.902+0000 7efdbccc2640 1 Processor -- start 2023-10-18T10:44:52.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.902+0000 7efdbccc2640 1 -- start start 2023-10-18T10:44:52.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdbccc2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda4003970 0x7efdb8138d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:52.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdbccc2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8150330 0x7efdb8157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:52.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdbccc2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb81507b0 0x7efdb8158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:52.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdbccc2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb8151920 con 0x7efdb81507b0 2023-10-18T10:44:52.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdbccc2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb8150d10 con 0x7efdb8150330 2023-10-18T10:44:52.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdbccc2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb8150ff0 con 0x7efda4003970 2023-10-18T10:44:52.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb81507b0 0x7efdb8158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:52.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdb6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb81507b0 0x7efdb8158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37944/0 (socket says 172.21.15.125:37944) 2023-10-18T10:44:52.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdb6d9d640 1 -- 172.21.15.125:0/1756609714 learned_addr learned my addr 172.21.15.125:0/1756609714 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:52.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdaffff640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8150330 0x7efdb8157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:52.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.903+0000 7efdb659c640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda4003970 0x7efdb8138d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:52.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdb6d9d640 1 -- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda4003970 msgr2=0x7efdb8138d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdb6d9d640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda4003970 0x7efdb8138d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdb6d9d640 1 -- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8150330 msgr2=0x7efdb8157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:52.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdb6d9d640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8150330 0x7efdb8157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:52.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdb6d9d640 1 -- 172.21.15.125:0/1756609714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb81512d0 con 0x7efdb81507b0 2023-10-18T10:44:52.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdaffff640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8150330 0x7efdb8157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:52.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdb6d9d640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb81507b0 0x7efdb8158220 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7efdb80cc150 tx=0x7efdb0011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:52.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdaeffd640 1 -- 172.21.15.125:0/1756609714 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0004b20 con 0x7efdb81507b0 2023-10-18T10:44:52.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdaeffd640 1 -- 172.21.15.125:0/1756609714 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efdb0004c80 con 0x7efdb81507b0 2023-10-18T10:44:52.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb81515b0 con 0x7efdb81507b0 2023-10-18T10:44:52.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdaeffd640 1 -- 172.21.15.125:0/1756609714 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0003bd0 con 0x7efdb81507b0 2023-10-18T10:44:52.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.904+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb8070e90 con 0x7efdb81507b0 2023-10-18T10:44:52.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.905+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb813b7f0 con 0x7efdb81507b0 2023-10-18T10:44:52.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.905+0000 7efdaeffd640 1 -- 172.21.15.125:0/1756609714 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efdb0003d30 con 0x7efdb81507b0 2023-10-18T10:44:52.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.906+0000 7efdaeffd640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7efd9806af90 0x7efd9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:52.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.906+0000 7efdaeffd640 1 -- 172.21.15.125:0/1756609714 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7efdb008a320 con 0x7efdb81507b0 2023-10-18T10:44:52.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.906+0000 7efdb659c640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7efd9806af90 0x7efd9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:52.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.907+0000 7efdb659c640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7efd9806af90 0x7efd9806d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efda8034000 tx=0x7efda802e040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:52.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:52.910+0000 7efdaeffd640 1 -- 172.21.15.125:0/1756609714 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdb0055070 con 0x7efdb81507b0 2023-10-18T10:44:53.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.140+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7efdb80690e0 con 0x7efdb81507b0 2023-10-18T10:44:53.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.486+0000 7efdaeffd640 1 -- 172.21.15.125:0/1756609714 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v759) v1 ==== 160+0+0 (secure 0 0 0) 0x7efdb0058690 con 0x7efdb81507b0 2023-10-18T10:44:53.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7efd9806af90 msgr2=0x7efd9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7efd9806af90 0x7efd9806d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efda8034000 tx=0x7efda802e040 comp rx=0 tx=0).stop 2023-10-18T10:44:53.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb81507b0 msgr2=0x7efdb8158220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb81507b0 0x7efdb8158220 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7efdb80cc150 tx=0x7efdb0011f40 comp rx=0 tx=0).stop 2023-10-18T10:44:53.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 shutdown_connections 2023-10-18T10:44:53.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7efd9806af90 0x7efd9806d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8150330 0x7efdb8157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efdb81507b0 0x7efdb8158220 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 --2- 172.21.15.125:0/1756609714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda4003970 0x7efdb8138d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 >> 172.21.15.125:0/1756609714 conn(0x7efdb8069930 msgr2=0x7efdb815b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:53.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.488+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 shutdown_connections 2023-10-18T10:44:53.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.489+0000 7efdbccc2640 1 -- 172.21.15.125:0/1756609714 wait complete. 2023-10-18T10:44:53.502 INFO:tasks.cephfs_test_runner:test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) ... ok 2023-10-18T10:44:53.502 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-10-18T10:44:53.503 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T10:44:53.503 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:53.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.607+0000 7f4c32010640 1 -- 172.21.15.125:0/3084139969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c18004d00 msgr2=0x7f4c18005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.607+0000 7f4c32010640 1 --2- 172.21.15.125:0/3084139969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c18004d00 0x7f4c18005180 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c640 tx=0x7f4c180057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.607+0000 7f4c2bfff640 1 -- 172.21.15.125:0/3084139969 reap_dead start 2023-10-18T10:44:53.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.607+0000 7f4c32010640 1 -- 172.21.15.125:0/3084139969 shutdown_connections 2023-10-18T10:44:53.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.607+0000 7f4c32010640 1 -- 172.21.15.125:0/3084139969 >> 172.21.15.125:0/3084139969 conn(0x7f4c2c069930 msgr2=0x7f4c2c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:53.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.607+0000 7f4c32010640 1 -- 172.21.15.125:0/3084139969 shutdown_connections 2023-10-18T10:44:53.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.607+0000 7f4c32010640 1 -- 172.21.15.125:0/3084139969 wait complete. 2023-10-18T10:44:53.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.608+0000 7f4c32010640 1 Processor -- start 2023-10-18T10:44:53.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.608+0000 7f4c32010640 1 -- start start 2023-10-18T10:44:53.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c32010640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1f0d70 0x7f4c2c13e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:53.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c32010640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c2c1f1150 0x7f4c2c0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:53.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c32010640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c2c0aae00 0x7f4c2c157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:53.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c32010640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c154580 con 0x7f4c2c0aae00 2023-10-18T10:44:53.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c32010640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0ab290 con 0x7f4c2c1f0d70 2023-10-18T10:44:53.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c32010640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c133320 con 0x7f4c2c1f1150 2023-10-18T10:44:53.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c2c0aae00 0x7f4c2c157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:53.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c2c0aae00 0x7f4c2c157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37958/0 (socket says 172.21.15.125:37958) 2023-10-18T10:44:53.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2bfff640 1 -- 172.21.15.125:0/842565821 learned_addr learned my addr 172.21.15.125:0/842565821 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:53.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2affd640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c2c1f1150 0x7f4c2c0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:53.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2b7fe640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1f0d70 0x7f4c2c13e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:53.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2bfff640 1 -- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c2c1f1150 msgr2=0x7f4c2c0aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2bfff640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c2c1f1150 0x7f4c2c0aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2bfff640 1 -- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1f0d70 msgr2=0x7f4c2c13e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2bfff640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1f0d70 0x7f4c2c13e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2bfff640 1 -- 172.21.15.125:0/842565821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c13dad0 con 0x7f4c2c0aae00 2023-10-18T10:44:53.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.609+0000 7f4c2affd640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c2c1f1150 0x7f4c2c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:53.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.610+0000 7f4c2bfff640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c2c0aae00 0x7f4c2c157a70 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f4c2402ccc0 tx=0x7f4c24002060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:53.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.610+0000 7f4c23fff640 1 -- 172.21.15.125:0/842565821 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240019b0 con 0x7f4c2c0aae00 2023-10-18T10:44:53.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.610+0000 7f4c23fff640 1 -- 172.21.15.125:0/842565821 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4c24003a50 con 0x7f4c2c0aae00 2023-10-18T10:44:53.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.610+0000 7f4c23fff640 1 -- 172.21.15.125:0/842565821 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24003e00 con 0x7f4c2c0aae00 2023-10-18T10:44:53.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.610+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c13ddb0 con 0x7f4c2c0aae00 2023-10-18T10:44:53.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.610+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c13e370 con 0x7f4c2c0aae00 2023-10-18T10:44:53.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.611+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c0cc010 con 0x7f4c2c0aae00 2023-10-18T10:44:53.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.611+0000 7f4c23fff640 1 -- 172.21.15.125:0/842565821 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4c24031030 con 0x7f4c2c0aae00 2023-10-18T10:44:53.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.611+0000 7f4c23fff640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f4c0c06af60 0x7f4c0c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:53.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.612+0000 7f4c23fff640 1 -- 172.21.15.125:0/842565821 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7f4c240aa4f0 con 0x7f4c2c0aae00 2023-10-18T10:44:53.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.612+0000 7f4c2b7fe640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f4c0c06af60 0x7f4c0c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:53.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.612+0000 7f4c2b7fe640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f4c0c06af60 0x7f4c0c06d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c005d00 tx=0x7f4c1c005c70 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:53.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.616+0000 7f4c23fff640 1 -- 172.21.15.125:0/842565821 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c24075240 con 0x7f4c2c0aae00 2023-10-18T10:44:53.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.784+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c2c159a00 con 0x7f4c2c0aae00 2023-10-18T10:44:53.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.785+0000 7f4c23fff640 1 -- 172.21.15.125:0/842565821 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+14373 (secure 0 0 0) 0x7f4c24078860 con 0x7f4c2c0aae00 2023-10-18T10:44:53.787 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:53.787 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":148,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:52.473520+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6834/2651847004":"2023-10-19T10:44:49.133947+0000","172.21.15.139:0/2424959739":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/848644581":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/1809262820":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6833/427428567":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6832/427428567":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/2263683535":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/3114574713":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/3295054028":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/1133703870":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/4009278606":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/494329198":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/2843740551":"2023-10-19T10:44:46.834486+0000","172.21.15.139:6833/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:6832/4144373589":"2023-10-19T10:44:36.950527+0000","172.21.15.139:0/3297526571":"2023-10-19T10:44:46.834486+0000","172.21.15.139:0/337665570":"2023-10-19T10:44:36.950527+0000","172.21.15.125:6832/2651847004":"2023-10-19T10:44:49.133947+0000","172.21.15.139:0/1274722331":"2023-10-19T10:44:36.950527+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:53.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f4c0c06af60 msgr2=0x7f4c0c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f4c0c06af60 0x7f4c0c06d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c005d00 tx=0x7f4c1c005c70 comp rx=0 tx=0).stop 2023-10-18T10:44:53.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c2c0aae00 msgr2=0x7f4c2c157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c2c0aae00 0x7f4c2c157a70 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f4c2402ccc0 tx=0x7f4c24002060 comp rx=0 tx=0).stop 2023-10-18T10:44:53.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 shutdown_connections 2023-10-18T10:44:53.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f4c0c06af60 0x7f4c0c06d420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c2c0aae00 0x7f4c2c157a70 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c2c1f1150 0x7f4c2c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 --2- 172.21.15.125:0/842565821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1f0d70 0x7f4c2c13e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 >> 172.21.15.125:0/842565821 conn(0x7f4c2c069930 msgr2=0x7f4c2c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:53.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 shutdown_connections 2023-10-18T10:44:53.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.787+0000 7f4c32010640 1 -- 172.21.15.125:0/842565821 wait complete. 2023-10-18T10:44:53.799 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:44:53.800 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:44:53.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.903+0000 7fad8234c640 1 -- 172.21.15.125:0/2075249274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c0cf340 msgr2=0x7fad7c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.903+0000 7fad8234c640 1 --2- 172.21.15.125:0/2075249274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c0cf340 0x7fad7c0d1910 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fad70005fd0 tx=0x7fad7002bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:53.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.903+0000 7fad8234c640 1 -- 172.21.15.125:0/2075249274 shutdown_connections 2023-10-18T10:44:53.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.903+0000 7fad8234c640 1 --2- 172.21.15.125:0/2075249274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c0cf340 0x7fad7c0d1910 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.903+0000 7fad8234c640 1 -- 172.21.15.125:0/2075249274 >> 172.21.15.125:0/2075249274 conn(0x7fad7c0d0d60 msgr2=0x7fad7c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:53.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.903+0000 7fad8234c640 1 -- 172.21.15.125:0/2075249274 shutdown_connections 2023-10-18T10:44:53.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.903+0000 7fad8234c640 1 -- 172.21.15.125:0/2075249274 wait complete. 2023-10-18T10:44:53.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.904+0000 7fad8234c640 1 Processor -- start 2023-10-18T10:44:53.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.904+0000 7fad8234c640 1 -- start start 2023-10-18T10:44:53.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad8234c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad7c0ceb40 0x7fad7c1f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:53.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad8234c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad7c0cf340 0x7fad7c1f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:53.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad8234c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c15b2a0 0x7fad7c0b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:53.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad8234c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c0c7fb0 con 0x7fad7c15b2a0 2023-10-18T10:44:53.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad8234c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c1f21b0 con 0x7fad7c0ceb40 2023-10-18T10:44:53.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad8234c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c0c8270 con 0x7fad7c0cf340 2023-10-18T10:44:53.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad808e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c15b2a0 0x7fad7c0b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:53.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad808e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c15b2a0 0x7fad7c0b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37976/0 (socket says 172.21.15.125:37976) 2023-10-18T10:44:53.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.905+0000 7fad808e9640 1 -- 172.21.15.125:0/3875280588 learned_addr learned my addr 172.21.15.125:0/3875280588 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:53.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad808e9640 1 -- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad7c0cf340 msgr2=0x7fad7c1f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad7b7fe640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad7c0cf340 0x7fad7c1f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:53.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad7bfff640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad7c0ceb40 0x7fad7c1f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:53.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad808e9640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad7c0cf340 0x7fad7c1f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad808e9640 1 -- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad7c0ceb40 msgr2=0x7fad7c1f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:53.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad808e9640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad7c0ceb40 0x7fad7c1f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:53.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad808e9640 1 -- 172.21.15.125:0/3875280588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70009560 con 0x7fad7c15b2a0 2023-10-18T10:44:53.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad7bfff640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad7c0ceb40 0x7fad7c1f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:53.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad808e9640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c15b2a0 0x7fad7c0b5270 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c780 tx=0x7fad7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:53.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad78ff9640 1 -- 172.21.15.125:0/3875280588 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74003d80 con 0x7fad7c15b2a0 2023-10-18T10:44:53.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad78ff9640 1 -- 172.21.15.125:0/3875280588 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad7400ce90 con 0x7fad7c15b2a0 2023-10-18T10:44:53.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c0c8550 con 0x7fad7c15b2a0 2023-10-18T10:44:53.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad78ff9640 1 -- 172.21.15.125:0/3875280588 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740052d0 con 0x7fad7c15b2a0 2023-10-18T10:44:53.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.906+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c0b57b0 con 0x7fad7c15b2a0 2023-10-18T10:44:53.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.907+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c0690e0 con 0x7fad7c15b2a0 2023-10-18T10:44:53.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.907+0000 7fad78ff9640 1 -- 172.21.15.125:0/3875280588 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fad74005430 con 0x7fad7c15b2a0 2023-10-18T10:44:53.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.908+0000 7fad78ff9640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fad6806af60 0x7fad6806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:53.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.908+0000 7fad78ff9640 1 -- 172.21.15.125:0/3875280588 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7fad74089300 con 0x7fad7c15b2a0 2023-10-18T10:44:53.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.908+0000 7fad7bfff640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fad6806af60 0x7fad6806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:53.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.908+0000 7fad7bfff640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fad6806af60 0x7fad6806d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fad70035000 tx=0x7fad70004730 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:53.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:53.912+0000 7fad78ff9640 1 -- 172.21.15.125:0/3875280588 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad740541f0 con 0x7fad7c15b2a0 2023-10-18T10:44:54.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.078+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad7c0d0b40 con 0x7fad7c15b2a0 2023-10-18T10:44:54.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.078+0000 7fad78ff9640 1 -- 172.21.15.125:0/3875280588 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad74017020 con 0x7fad7c15b2a0 2023-10-18T10:44:54.081 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:44:54.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.080+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fad6806af60 msgr2=0x7fad6806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.080+0000 7fad8234c640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fad6806af60 0x7fad6806d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fad70035000 tx=0x7fad70004730 comp rx=0 tx=0).stop 2023-10-18T10:44:54.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.080+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c15b2a0 msgr2=0x7fad7c0b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.080+0000 7fad8234c640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c15b2a0 0x7fad7c0b5270 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c780 tx=0x7fad7400cc40 comp rx=0 tx=0).stop 2023-10-18T10:44:54.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.081+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 shutdown_connections 2023-10-18T10:44:54.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.081+0000 7fad8234c640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fad6806af60 0x7fad6806d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.081+0000 7fad8234c640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad7c15b2a0 0x7fad7c0b5270 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.081+0000 7fad8234c640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad7c0cf340 0x7fad7c1f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.081+0000 7fad8234c640 1 --2- 172.21.15.125:0/3875280588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad7c0ceb40 0x7fad7c1f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.081+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 >> 172.21.15.125:0/3875280588 conn(0x7fad7c0d0d60 msgr2=0x7fad7c0b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:54.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.081+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 shutdown_connections 2023-10-18T10:44:54.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.081+0000 7fad8234c640 1 -- 172.21.15.125:0/3875280588 wait complete. 2023-10-18T10:44:54.093 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-18T10:44:54.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.194+0000 7fb10fa56640 1 -- 172.21.15.125:0/633250042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0f40047c0 msgr2=0x7fb0f4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.194+0000 7fb10fa56640 1 --2- 172.21.15.125:0/633250042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0f40047c0 0x7fb0f4004c40 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009b50 tx=0x7fb0f40057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.194+0000 7fb10fa56640 1 -- 172.21.15.125:0/633250042 shutdown_connections 2023-10-18T10:44:54.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.194+0000 7fb10fa56640 1 --2- 172.21.15.125:0/633250042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0f40047c0 0x7fb0f4004c40 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.194+0000 7fb10fa56640 1 -- 172.21.15.125:0/633250042 >> 172.21.15.125:0/633250042 conn(0x7fb108069930 msgr2=0x7fb108069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:54.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.194+0000 7fb10fa56640 1 -- 172.21.15.125:0/633250042 shutdown_connections 2023-10-18T10:44:54.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.194+0000 7fb10fa56640 1 -- 172.21.15.125:0/633250042 wait complete. 2023-10-18T10:44:54.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10fa56640 1 Processor -- start 2023-10-18T10:44:54.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10fa56640 1 -- start start 2023-10-18T10:44:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10fa56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0f40047c0 0x7fb10813aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10fa56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb108159340 0x7fb1081597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10fa56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1080b09a0 0x7fb1080b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10fa56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080b1360 con 0x7fb108159340 2023-10-18T10:44:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10fa56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10813cc20 con 0x7fb0f40047c0 2023-10-18T10:44:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10fa56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb108135a90 con 0x7fb1080b09a0 2023-10-18T10:44:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10cff1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb108159340 0x7fb1081597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10cff1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb108159340 0x7fb1081597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37996/0 (socket says 172.21.15.125:37996) 2023-10-18T10:44:54.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.196+0000 7fb10cff1640 1 -- 172.21.15.125:0/1034649880 learned_addr learned my addr 172.21.15.125:0/1034649880 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:54.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10dff3640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1080b09a0 0x7fb1080b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10d7f2640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0f40047c0 0x7fb10813aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10cff1640 1 -- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1080b09a0 msgr2=0x7fb1080b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10cff1640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1080b09a0 0x7fb1080b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10cff1640 1 -- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0f40047c0 msgr2=0x7fb10813aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10cff1640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0f40047c0 0x7fb10813aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10cff1640 1 -- 172.21.15.125:0/1034649880 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10813bb80 con 0x7fb108159340 2023-10-18T10:44:54.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10dff3640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1080b09a0 0x7fb1080b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:54.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10cff1640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb108159340 0x7fb1081597a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009b50 tx=0x7fb0f8033bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:54.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb0fdffb640 1 -- 172.21.15.125:0/1034649880 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8004230 con 0x7fb108159340 2023-10-18T10:44:54.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb0fdffb640 1 -- 172.21.15.125:0/1034649880 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb0f8004390 con 0x7fb108159340 2023-10-18T10:44:54.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10813be60 con 0x7fb108159340 2023-10-18T10:44:54.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb0fdffb640 1 -- 172.21.15.125:0/1034649880 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f803d880 con 0x7fb108159340 2023-10-18T10:44:54.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.197+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10813c420 con 0x7fb108159340 2023-10-18T10:44:54.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.198+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1080690e0 con 0x7fb108159340 2023-10-18T10:44:54.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.199+0000 7fb0fdffb640 1 -- 172.21.15.125:0/1034649880 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb0f803d9e0 con 0x7fb108159340 2023-10-18T10:44:54.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.199+0000 7fb0fdffb640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0ec06af60 0x7fb0ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.199+0000 7fb0fdffb640 1 -- 172.21.15.125:0/1034649880 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7fb0f80ac8b0 con 0x7fb108159340 2023-10-18T10:44:54.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.199+0000 7fb10d7f2640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0ec06af60 0x7fb0ec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.200+0000 7fb10d7f2640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0ec06af60 0x7fb0ec06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb104001b20 tx=0x7fb104002190 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:54.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.203+0000 7fb0fdffb640 1 -- 172.21.15.125:0/1034649880 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0f8077570 con 0x7fb108159340 2023-10-18T10:44:54.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.433+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fb108159fe0 con 0x7fb108159340 2023-10-18T10:44:54.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.487+0000 7fb0fdffb640 1 -- 172.21.15.125:0/1034649880 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v760) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb0f8041030 con 0x7fb108159340 2023-10-18T10:44:54.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0ec06af60 msgr2=0x7fb0ec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0ec06af60 0x7fb0ec06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb104001b20 tx=0x7fb104002190 comp rx=0 tx=0).stop 2023-10-18T10:44:54.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb108159340 msgr2=0x7fb1081597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb108159340 0x7fb1081597a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009b50 tx=0x7fb0f8033bd0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 shutdown_connections 2023-10-18T10:44:54.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0ec06af60 0x7fb0ec06d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1080b09a0 0x7fb1080b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb108159340 0x7fb1081597a0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 --2- 172.21.15.125:0/1034649880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0f40047c0 0x7fb10813aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.489+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 >> 172.21.15.125:0/1034649880 conn(0x7fb108069930 msgr2=0x7fb108162180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:54.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.490+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 shutdown_connections 2023-10-18T10:44:54.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.490+0000 7fb10fa56640 1 -- 172.21.15.125:0/1034649880 wait complete. 2023-10-18T10:44:54.502 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T10:44:54.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.603+0000 7f8b75e56640 1 -- 172.21.15.125:0/2914323353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b58004d00 msgr2=0x7f8b58005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.603+0000 7f8b75e56640 1 --2- 172.21.15.125:0/2914323353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b58004d00 0x7f8b58005180 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f8b68001680 tx=0x7f8b580057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.603+0000 7f8b6ffff640 1 -- 172.21.15.125:0/2914323353 reap_dead start 2023-10-18T10:44:54.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.603+0000 7f8b75e56640 1 -- 172.21.15.125:0/2914323353 shutdown_connections 2023-10-18T10:44:54.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.603+0000 7f8b75e56640 1 -- 172.21.15.125:0/2914323353 >> 172.21.15.125:0/2914323353 conn(0x7f8b700d0d60 msgr2=0x7f8b700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:54.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.603+0000 7f8b75e56640 1 -- 172.21.15.125:0/2914323353 shutdown_connections 2023-10-18T10:44:54.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.603+0000 7f8b75e56640 1 -- 172.21.15.125:0/2914323353 wait complete. 2023-10-18T10:44:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.604+0000 7f8b75e56640 1 Processor -- start 2023-10-18T10:44:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.604+0000 7f8b75e56640 1 -- start start 2023-10-18T10:44:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b75e56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b7014c2b0 0x7f8b7014cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b75e56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b700b6810 0x7f8b700b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b75e56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b700c9e40 0x7f8b700ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b75e56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700ca800 con 0x7f8b7014c2b0 2023-10-18T10:44:54.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b75e56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7015d3c0 con 0x7f8b700c9e40 2023-10-18T10:44:54.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b75e56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7014c690 con 0x7f8b700b6810 2023-10-18T10:44:54.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b7014c2b0 0x7f8b7014cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b7014c2b0 0x7f8b7014cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38022/0 (socket says 172.21.15.125:38022) 2023-10-18T10:44:54.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 -- 172.21.15.125:0/3319150829 learned_addr learned my addr 172.21.15.125:0/3319150829 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:54.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6effd640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b700b6810 0x7f8b700b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 -- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b700b6810 msgr2=0x7f8b700b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6ffff640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b700c9e40 0x7f8b700ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b700b6810 0x7f8b700b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 -- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b700c9e40 msgr2=0x7f8b700ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b700c9e40 0x7f8b700ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 -- 172.21.15.125:0/3319150829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700b4f00 con 0x7f8b7014c2b0 2023-10-18T10:44:54.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6effd640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b700b6810 0x7f8b700b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6ffff640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b700c9e40 0x7f8b700ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.605+0000 7f8b6f7fe640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b7014c2b0 0x7f8b7014cc00 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f8b60006400 tx=0x7f8b600068c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.606+0000 7f8b57fff640 1 -- 172.21.15.125:0/3319150829 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000fda0 con 0x7f8b7014c2b0 2023-10-18T10:44:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.606+0000 7f8b57fff640 1 -- 172.21.15.125:0/3319150829 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b60011430 con 0x7f8b7014c2b0 2023-10-18T10:44:54.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.606+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700b51e0 con 0x7f8b7014c2b0 2023-10-18T10:44:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.606+0000 7f8b57fff640 1 -- 172.21.15.125:0/3319150829 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60011840 con 0x7f8b7014c2b0 2023-10-18T10:44:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.606+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7015d050 con 0x7f8b7014c2b0 2023-10-18T10:44:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.607+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7015cbd0 con 0x7f8b7014c2b0 2023-10-18T10:44:54.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.607+0000 7f8b57fff640 1 -- 172.21.15.125:0/3319150829 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8b600119a0 con 0x7f8b7014c2b0 2023-10-18T10:44:54.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.607+0000 7f8b57fff640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f8b5c06af60 0x7f8b5c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.608+0000 7f8b57fff640 1 -- 172.21.15.125:0/3319150829 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7f8b6008abc0 con 0x7f8b7014c2b0 2023-10-18T10:44:54.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.608+0000 7f8b6effd640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f8b5c06af60 0x7f8b5c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.608+0000 7f8b6effd640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f8b5c06af60 0x7f8b5c06d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8b7009ec30 tx=0x7f8b64005f90 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:54.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.612+0000 7f8b57fff640 1 -- 172.21.15.125:0/3319150829 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b7015cbd0 con 0x7f8b7014c2b0 2023-10-18T10:44:54.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.779+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f8b7015cbd0 con 0x7f8b7014c2b0 2023-10-18T10:44:54.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.779+0000 7f8b57fff640 1 -- 172.21.15.125:0/3319150829 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v58) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f8b7015cbd0 con 0x7f8b7014c2b0 2023-10-18T10:44:54.782 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T10:44:54.782 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T10:44:54.782 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T10:44:54.782 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T10:44:54.782 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T10:44:54.783 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T10:44:54.783 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T10:44:54.783 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T10:44:54.783 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T10:44:54.784 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T10:44:54.784 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T10:44:54.784 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T10:44:54.784 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T10:44:54.784 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T10:44:54.785 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T10:44:54.785 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T10:44:54.785 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T10:44:54.785 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T10:44:54.785 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T10:44:54.785 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T10:44:54.786 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T10:44:54.786 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T10:44:54.786 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T10:44:54.786 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T10:44:54.786 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T10:44:54.787 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T10:44:54.787 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T10:44:54.787 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T10:44:54.787 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T10:44:54.787 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T10:44:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f8b5c06af60 msgr2=0x7f8b5c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f8b5c06af60 0x7f8b5c06d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8b7009ec30 tx=0x7f8b64005f90 comp rx=0 tx=0).stop 2023-10-18T10:44:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b7014c2b0 msgr2=0x7f8b7014cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b7014c2b0 0x7f8b7014cc00 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f8b60006400 tx=0x7f8b600068c0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 shutdown_connections 2023-10-18T10:44:54.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f8b5c06af60 0x7f8b5c06d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b700c9e40 0x7f8b700ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b700b6810 0x7f8b700b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 --2- 172.21.15.125:0/3319150829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8b7014c2b0 0x7f8b7014cc00 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 >> 172.21.15.125:0/3319150829 conn(0x7f8b700d0d60 msgr2=0x7f8b700ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:54.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 shutdown_connections 2023-10-18T10:44:54.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.781+0000 7f8b75e56640 1 -- 172.21.15.125:0/3319150829 wait complete. 2023-10-18T10:44:54.794 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T10:44:54.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.897+0000 7fb4326e5640 1 -- 172.21.15.125:0/345350251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 msgr2=0x7fb4140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.897+0000 7fb4326e5640 1 --2- 172.21.15.125:0/345350251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 0x7fb4140047c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fb420009b80 tx=0x7fb4140057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.897+0000 7fb430c82640 1 -- 172.21.15.125:0/345350251 reap_dead start 2023-10-18T10:44:54.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.897+0000 7fb4326e5640 1 -- 172.21.15.125:0/345350251 shutdown_connections 2023-10-18T10:44:54.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.897+0000 7fb4326e5640 1 -- 172.21.15.125:0/345350251 >> 172.21.15.125:0/345350251 conn(0x7fb42c0d0d60 msgr2=0x7fb42c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:54.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.897+0000 7fb4326e5640 1 -- 172.21.15.125:0/345350251 shutdown_connections 2023-10-18T10:44:54.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.897+0000 7fb4326e5640 1 -- 172.21.15.125:0/345350251 wait complete. 2023-10-18T10:44:54.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.899+0000 7fb4326e5640 1 Processor -- start 2023-10-18T10:44:54.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.899+0000 7fb4326e5640 1 -- start start 2023-10-18T10:44:54.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.899+0000 7fb4326e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 0x7fb42c156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.899+0000 7fb4326e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c157ef0 0x7fb42c15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.899+0000 7fb4326e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c1583b0 0x7fb42c0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.899+0000 7fb4326e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1f5480 con 0x7fb414003ef0 2023-10-18T10:44:54.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.899+0000 7fb4326e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c156c70 con 0x7fb42c1583b0 2023-10-18T10:44:54.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.899+0000 7fb4326e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c09e6b0 con 0x7fb42c157ef0 2023-10-18T10:44:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 0x7fb42c156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 0x7fb42c156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38040/0 (socket says 172.21.15.125:38040) 2023-10-18T10:44:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 -- 172.21.15.125:0/1978659810 learned_addr learned my addr 172.21.15.125:0/1978659810 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:54.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb430c82640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c1583b0 0x7fb42c0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42b7fe640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c157ef0 0x7fb42c15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 -- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c157ef0 msgr2=0x7fb42c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c157ef0 0x7fb42c15b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 -- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c1583b0 msgr2=0x7fb42c0b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:54.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c1583b0 0x7fb42c0b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:54.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 -- 172.21.15.125:0/1978659810 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c0b4400 con 0x7fb414003ef0 2023-10-18T10:44:54.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb42bfff640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 0x7fb42c156730 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00e650 tx=0x7fb41c00eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:54.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb428ff9640 1 -- 172.21.15.125:0/1978659810 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c004440 con 0x7fb414003ef0 2023-10-18T10:44:54.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb428ff9640 1 -- 172.21.15.125:0/1978659810 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb41c00c840 con 0x7fb414003ef0 2023-10-18T10:44:54.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb428ff9640 1 -- 172.21.15.125:0/1978659810 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00cc50 con 0x7fb414003ef0 2023-10-18T10:44:54.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c0b46e0 con 0x7fb414003ef0 2023-10-18T10:44:54.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.900+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c15bbc0 con 0x7fb414003ef0 2023-10-18T10:44:54.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.901+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c0b48d0 con 0x7fb414003ef0 2023-10-18T10:44:54.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.902+0000 7fb428ff9640 1 -- 172.21.15.125:0/1978659810 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb41c0035f0 con 0x7fb414003ef0 2023-10-18T10:44:54.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.902+0000 7fb428ff9640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb41806af90 0x7fb41806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:54.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.902+0000 7fb428ff9640 1 -- 172.21.15.125:0/1978659810 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7fb41c08a420 con 0x7fb414003ef0 2023-10-18T10:44:54.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.902+0000 7fb42b7fe640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb41806af90 0x7fb41806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:54.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.903+0000 7fb42b7fe640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb41806af90 0x7fb41806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb420036000 tx=0x7fb420035040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:54.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:54.906+0000 7fb428ff9640 1 -- 172.21.15.125:0/1978659810 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb41c055170 con 0x7fb414003ef0 2023-10-18T10:44:55.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.073+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb42c0690e0 con 0x7fb414003ef0 2023-10-18T10:44:55.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.076+0000 7fb428ff9640 1 -- 172.21.15.125:0/1978659810 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v59)=0 v59) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb41c00cdb0 con 0x7fb414003ef0 2023-10-18T10:44:55.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb41806af90 msgr2=0x7fb41806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb41806af90 0x7fb41806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb420036000 tx=0x7fb420035040 comp rx=0 tx=0).stop 2023-10-18T10:44:55.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 msgr2=0x7fb42c156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 0x7fb42c156730 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00e650 tx=0x7fb41c00eb10 comp rx=0 tx=0).stop 2023-10-18T10:44:55.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 shutdown_connections 2023-10-18T10:44:55.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb41806af90 0x7fb41806d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c1583b0 0x7fb42c0b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c157ef0 0x7fb42c15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 --2- 172.21.15.125:0/1978659810 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb414003ef0 0x7fb42c156730 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.078+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 >> 172.21.15.125:0/1978659810 conn(0x7fb42c0d0d60 msgr2=0x7fb42c0b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:55.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.079+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 shutdown_connections 2023-10-18T10:44:55.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.079+0000 7fb4326e5640 1 -- 172.21.15.125:0/1978659810 wait complete. 2023-10-18T10:44:55.091 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T10:44:55.092 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:44:55.137 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T10:44:55.193 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T10:44:55.194 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:44:55.194 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:55.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:44:55.209 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:44:55.210 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:55.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:44:55.226 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T10:44:55.227 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:55.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T10:44:55.264 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T10:44:55.265 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T10:44:55.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T10:44:55.283 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T10:44:55.284 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:44:55.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 -- 172.21.15.125:0/1015536493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc40ceb40 msgr2=0x7f0dc40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/1015536493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc40ceb40 0x7f0dc40b0650 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f0dc408a810 tx=0x7f0dbc02c1d0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 -- 172.21.15.125:0/1015536493 shutdown_connections 2023-10-18T10:44:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/1015536493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc40ceb40 0x7f0dc40b0650 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/1015536493 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dc415b2a0 0x7f0dc40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/1015536493 >> v2:172.21.15.139:3300/0 conn(0x7f0dc40cf340 0x7f0dc40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 -- 172.21.15.125:0/1015536493 >> 172.21.15.125:0/1015536493 conn(0x7f0dc40d0d60 msgr2=0x7f0dc40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 -- 172.21.15.125:0/1015536493 shutdown_connections 2023-10-18T10:44:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.404+0000 7f0dca2e5640 1 -- 172.21.15.125:0/1015536493 wait complete. 2023-10-18T10:44:55.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.405+0000 7f0dca2e5640 1 Processor -- start 2023-10-18T10:44:55.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dca2e5640 1 -- start start 2023-10-18T10:44:55.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dca2e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dc40ceb40 0x7f0dc4159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:55.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dca2e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dc40cf340 0x7f0dc41e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:55.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dca2e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc415b2a0 0x7f0dc40cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:55.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dca2e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4159950 con 0x7f0dc415b2a0 2023-10-18T10:44:55.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dca2e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc40b5270 con 0x7f0dc40ceb40 2023-10-18T10:44:55.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dca2e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc41f5490 con 0x7f0dc40cf340 2023-10-18T10:44:55.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dc8882640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc415b2a0 0x7f0dc40cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:55.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dc8882640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc415b2a0 0x7f0dc40cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38056/0 (socket says 172.21.15.125:38056) 2023-10-18T10:44:55.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dc8882640 1 -- 172.21.15.125:0/2301989933 learned_addr learned my addr 172.21.15.125:0/2301989933 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:55.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dc3fff640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dc40ceb40 0x7f0dc4159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:55.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.406+0000 7f0dc37fe640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dc40cf340 0x7f0dc41e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:55.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc8882640 1 -- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dc40cf340 msgr2=0x7f0dc41e5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc8882640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dc40cf340 0x7f0dc41e5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc8882640 1 -- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dc40ceb40 msgr2=0x7f0dc4159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc8882640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dc40ceb40 0x7f0dc4159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc8882640 1 -- 172.21.15.125:0/2301989933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc009900 con 0x7f0dc415b2a0 2023-10-18T10:44:55.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc8882640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc415b2a0 0x7f0dc40cc630 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009d60 tx=0x7f0dbc035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:55.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc0ff9640 1 -- 172.21.15.125:0/2301989933 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc034440 con 0x7f0dc415b2a0 2023-10-18T10:44:55.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc0ff9640 1 -- 172.21.15.125:0/2301989933 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0dbc004540 con 0x7f0dc415b2a0 2023-10-18T10:44:55.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc41f5770 con 0x7f0dc415b2a0 2023-10-18T10:44:55.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dc0ff9640 1 -- 172.21.15.125:0/2301989933 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc040450 con 0x7f0dc415b2a0 2023-10-18T10:44:55.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.407+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc41f5d30 con 0x7f0dc415b2a0 2023-10-18T10:44:55.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.408+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc40b5470 con 0x7f0dc415b2a0 2023-10-18T10:44:55.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.408+0000 7f0dc0ff9640 1 -- 172.21.15.125:0/2301989933 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0dbc00a040 con 0x7f0dc415b2a0 2023-10-18T10:44:55.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.409+0000 7f0dc0ff9640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0db006af60 0x7f0db006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:55.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.409+0000 7f0dc0ff9640 1 -- 172.21.15.125:0/2301989933 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7f0dbc0acbd0 con 0x7f0dc415b2a0 2023-10-18T10:44:55.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.409+0000 7f0dc3fff640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0db006af60 0x7f0db006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:55.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.410+0000 7f0dc3fff640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0db006af60 0x7f0db006d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0db40058c0 tx=0x7f0db400a2d0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:55.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.413+0000 7f0dc0ff9640 1 -- 172.21.15.125:0/2301989933 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0dc40b5470 con 0x7f0dc415b2a0 2023-10-18T10:44:55.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.616+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dc40b5430 con 0x7f0dc415b2a0 2023-10-18T10:44:55.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.616+0000 7f0dc0ff9640 1 -- 172.21.15.125:0/2301989933 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f0dbc039090 con 0x7f0dc415b2a0 2023-10-18T10:44:55.619 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:55.619 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7112,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/585529921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":585529921},{"type":"v1","addr":"172.21.15.125:6835","nonce":585529921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7118,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1680315524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1680315524},{"type":"v1","addr":"172.21.15.139:6837","nonce":1680315524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1127901065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1127901065},{"type":"v1","addr":"172.21.15.139:6836","nonce":1127901065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":9690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/403689041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":403689041},{"type":"v1","addr":"172.21.15.125:6834","nonce":403689041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[]} 2023-10-18T10:44:55.621 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 55 2023-10-18T10:44:55.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0db006af60 msgr2=0x7f0db006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0db006af60 0x7f0db006d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0db40058c0 tx=0x7f0db400a2d0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc415b2a0 msgr2=0x7f0dc40cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc415b2a0 0x7f0dc40cc630 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009d60 tx=0x7f0dbc035e30 comp rx=0 tx=0).stop 2023-10-18T10:44:55.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 shutdown_connections 2023-10-18T10:44:55.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0db006af60 0x7f0db006d420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dc415b2a0 0x7f0dc40cc630 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0dc40cf340 0x7f0dc41e5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 --2- 172.21.15.125:0/2301989933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dc40ceb40 0x7f0dc4159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 >> 172.21.15.125:0/2301989933 conn(0x7f0dc40d0d60 msgr2=0x7f0dc415a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:55.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 shutdown_connections 2023-10-18T10:44:55.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.618+0000 7f0dca2e5640 1 -- 172.21.15.125:0/2301989933 wait complete. 2023-10-18T10:44:55.631 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T10:44:55.631 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T10:44:55.631 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T10:44:55.632 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:44:55.632 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T10:44:55.632 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T10:44:55.633 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T10:44:55.633 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:44:55.633 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T10:44:55.634 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T10:44:55.634 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T10:44:55.634 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:44:55.635 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T10:44:55.635 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T10:44:55.635 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T10:44:55.635 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T10:44:55.640 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T10:44:55.641 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T10:44:55.642 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T10:44:55.642 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T10:44:55.645 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T10:44:55.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T10:44:55.647 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T10:44:55.647 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T10:44:55.684 INFO:tasks.ceph.mds.a:Started 2023-10-18T10:44:55.685 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:55.686 INFO:tasks.ceph.mds.c:Started 2023-10-18T10:44:55.689 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:55.691 INFO:tasks.ceph.mds.b:Started 2023-10-18T10:44:55.692 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:55.692 INFO:tasks.ceph.mds.d:Started 2023-10-18T10:44:55.693 DEBUG:teuthology.parallel:result is None 2023-10-18T10:44:55.694 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T10:44:55.759 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T10:44:55.759 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T10:44:55.773 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T10:44:55.774 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T10:44:55.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.800+0000 7f64e0ff5640 1 -- 172.21.15.125:0/605784130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64c4005180 msgr2=0x7f64c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.800+0000 7f64e0ff5640 1 --2- 172.21.15.125:0/605784130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64c4005180 0x7f64c400b980 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c640 tx=0x7f64c40057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.800+0000 7f64dad9d640 1 -- 172.21.15.125:0/605784130 reap_dead start 2023-10-18T10:44:55.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.800+0000 7f64e0ff5640 1 -- 172.21.15.125:0/605784130 shutdown_connections 2023-10-18T10:44:55.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.800+0000 7f64e0ff5640 1 -- 172.21.15.125:0/605784130 >> 172.21.15.125:0/605784130 conn(0x7f64dc0d0d60 msgr2=0x7f64dc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:55.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.800+0000 7f64e0ff5640 1 -- 172.21.15.125:0/605784130 shutdown_connections 2023-10-18T10:44:55.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.800+0000 7f64e0ff5640 1 -- 172.21.15.125:0/605784130 wait complete. 2023-10-18T10:44:55.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.801+0000 7f64e0ff5640 1 Processor -- start 2023-10-18T10:44:55.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.801+0000 7f64e0ff5640 1 -- start start 2023-10-18T10:44:55.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64e0ff5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c4005180 0x7f64dc0b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:55.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64e0ff5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc1f4170 0x7f64dc0aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:55.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64e0ff5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64dc1f45d0 0x7f64dc0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:55.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64e0ff5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc06bb60 con 0x7f64dc1f45d0 2023-10-18T10:44:55.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64e0ff5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0c8210 con 0x7f64c4005180 2023-10-18T10:44:55.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64e0ff5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0c84f0 con 0x7f64dc1f4170 2023-10-18T10:44:55.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64dc1f45d0 0x7f64dc0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:55.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64dc1f45d0 0x7f64dc0af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38102/0 (socket says 172.21.15.125:38102) 2023-10-18T10:44:55.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 -- 172.21.15.125:0/2042818056 learned_addr learned my addr 172.21.15.125:0/2042818056 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:55.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64da59c640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c4005180 0x7f64dc0b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:55.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64d9d9b640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc1f4170 0x7f64dc0aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:55.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 -- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc1f4170 msgr2=0x7f64dc0aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc1f4170 0x7f64dc0aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 -- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c4005180 msgr2=0x7f64dc0b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:55.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c4005180 0x7f64dc0b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:55.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 -- 172.21.15.125:0/2042818056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0c87d0 con 0x7f64dc1f45d0 2023-10-18T10:44:55.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.802+0000 7f64dad9d640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64dc1f45d0 0x7f64dc0af440 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c760 tx=0x7f64d4001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:55.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.803+0000 7f64caffd640 1 -- 172.21.15.125:0/2042818056 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d40017f0 con 0x7f64dc1f45d0 2023-10-18T10:44:55.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.803+0000 7f64caffd640 1 -- 172.21.15.125:0/2042818056 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64d4001970 con 0x7f64dc1f45d0 2023-10-18T10:44:55.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.803+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc0c8ab0 con 0x7f64dc1f45d0 2023-10-18T10:44:55.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.803+0000 7f64caffd640 1 -- 172.21.15.125:0/2042818056 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4003bd0 con 0x7f64dc1f45d0 2023-10-18T10:44:55.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.803+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc0b5850 con 0x7f64dc1f45d0 2023-10-18T10:44:55.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.804+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc0c8fd0 con 0x7f64dc1f45d0 2023-10-18T10:44:55.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.804+0000 7f64caffd640 1 -- 172.21.15.125:0/2042818056 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f64d4031030 con 0x7f64dc1f45d0 2023-10-18T10:44:55.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.805+0000 7f64caffd640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f64bc06af60 0x7f64bc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:55.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.805+0000 7f64caffd640 1 -- 172.21.15.125:0/2042818056 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6022+0+0 (secure 0 0 0) 0x7f64d40aaa70 con 0x7f64dc1f45d0 2023-10-18T10:44:55.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.805+0000 7f64da59c640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f64bc06af60 0x7f64bc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:55.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.806+0000 7f64da59c640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f64bc06af60 0x7f64bc06d420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0036b0 tx=0x7f64cc006060 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:55.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.809+0000 7f64caffd640 1 -- 172.21.15.125:0/2042818056 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64dc0c8fd0 con 0x7f64dc1f45d0 2023-10-18T10:44:55.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:55.987+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f64dc0c8fd0 con 0x7f64dc1f45d0 2023-10-18T10:44:56.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.506+0000 7f64caffd640 1 -- 172.21.15.125:0/2042818056 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) v1 ==== 97+0+0 (secure 0 0 0) 0x7f64dc0c8fd0 con 0x7f64dc1f45d0 2023-10-18T10:44:56.509 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T10:44:56.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f64bc06af60 msgr2=0x7f64bc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:56.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f64bc06af60 0x7f64bc06d420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0036b0 tx=0x7f64cc006060 comp rx=0 tx=0).stop 2023-10-18T10:44:56.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64dc1f45d0 msgr2=0x7f64dc0af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:56.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64dc1f45d0 0x7f64dc0af440 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c760 tx=0x7f64d4001410 comp rx=0 tx=0).stop 2023-10-18T10:44:56.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 shutdown_connections 2023-10-18T10:44:56.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f64bc06af60 0x7f64bc06d420 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64dc1f45d0 0x7f64dc0af440 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc1f4170 0x7f64dc0aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 --2- 172.21.15.125:0/2042818056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c4005180 0x7f64dc0b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 >> 172.21.15.125:0/2042818056 conn(0x7f64dc0d0d60 msgr2=0x7f64dc0cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:56.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 shutdown_connections 2023-10-18T10:44:56.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.510+0000 7f64e0ff5640 1 -- 172.21.15.125:0/2042818056 wait complete. 2023-10-18T10:44:56.526 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T10:44:56.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.629+0000 7ff318e32640 1 -- 172.21.15.125:0/818159402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff31413e520 msgr2=0x7ff31406bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:56.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.629+0000 7ff318e32640 1 --2- 172.21.15.125:0/818159402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff31413e520 0x7ff31406bb60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff3140cbdc0 tx=0x7ff30402bd30 comp rx=0 tx=0).stop 2023-10-18T10:44:56.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.629+0000 7ff318e32640 1 -- 172.21.15.125:0/818159402 shutdown_connections 2023-10-18T10:44:56.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.629+0000 7ff318e32640 1 --2- 172.21.15.125:0/818159402 >> v2:172.21.15.139:3300/0 conn(0x7ff314141690 0x7ff3140703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.629+0000 7ff318e32640 1 --2- 172.21.15.125:0/818159402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff31413e520 0x7ff31406bb60 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.629+0000 7ff318e32640 1 -- 172.21.15.125:0/818159402 >> 172.21.15.125:0/818159402 conn(0x7ff314069930 msgr2=0x7ff314069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:56.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.629+0000 7ff318e32640 1 -- 172.21.15.125:0/818159402 shutdown_connections 2023-10-18T10:44:56.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.630+0000 7ff318e32640 1 -- 172.21.15.125:0/818159402 wait complete. 2023-10-18T10:44:56.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.631+0000 7ff318e32640 1 Processor -- start 2023-10-18T10:44:56.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.631+0000 7ff318e32640 1 -- start start 2023-10-18T10:44:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.631+0000 7ff318e32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff31406c0a0 0x7ff314152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.631+0000 7ff318e32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff31413e520 0x7ff314153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.631+0000 7ff318e32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff314141690 0x7ff31413ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.631+0000 7ff318e32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314154030 con 0x7ff314141690 2023-10-18T10:44:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.631+0000 7ff318e32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314157680 con 0x7ff31413e520 2023-10-18T10:44:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.631+0000 7ff318e32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff314156a70 con 0x7ff31406c0a0 2023-10-18T10:44:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff314141690 0x7ff31413ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff314141690 0x7ff31413ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38120/0 (socket says 172.21.15.125:38120) 2023-10-18T10:44:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 -- 172.21.15.125:0/1508599137 learned_addr learned my addr 172.21.15.125:0/1508599137 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff30bfff640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff31413e520 0x7ff314153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff31259c640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff31406c0a0 0x7ff314152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 -- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff31406c0a0 msgr2=0x7ff314152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff31406c0a0 0x7ff314152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 -- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff31413e520 msgr2=0x7ff314153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff31413e520 0x7ff314153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 -- 172.21.15.125:0/1508599137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304009560 con 0x7ff314141690 2023-10-18T10:44:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff312d9d640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff314141690 0x7ff31413ac10 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c7b0 tx=0x7ff30c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff30affd640 1 -- 172.21.15.125:0/1508599137 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c003d80 con 0x7ff314141690 2023-10-18T10:44:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff30affd640 1 -- 172.21.15.125:0/1508599137 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff30c005070 con 0x7ff314141690 2023-10-18T10:44:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff314156d50 con 0x7ff314141690 2023-10-18T10:44:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff30affd640 1 -- 172.21.15.125:0/1508599137 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0054a0 con 0x7ff314141690 2023-10-18T10:44:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.632+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff314157310 con 0x7ff314141690 2023-10-18T10:44:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.633+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31413e0d0 con 0x7ff314141690 2023-10-18T10:44:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.634+0000 7ff30affd640 1 -- 172.21.15.125:0/1508599137 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff30c005600 con 0x7ff314141690 2023-10-18T10:44:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.634+0000 7ff30affd640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff2f406af60 0x7ff2f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.634+0000 7ff30affd640 1 -- 172.21.15.125:0/1508599137 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5205+0+0 (secure 0 0 0) 0x7ff30c088c20 con 0x7ff314141690 2023-10-18T10:44:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.634+0000 7ff31259c640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff2f406af60 0x7ff2f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.635+0000 7ff31259c640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff2f406af60 0x7ff2f406d420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff304035000 tx=0x7ff304004730 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.638+0000 7ff30affd640 1 -- 172.21.15.125:0/1508599137 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff30c053db0 con 0x7ff314141690 2023-10-18T10:44:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.872+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff3140690e0 con 0x7ff314141690 2023-10-18T10:44:56.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.872+0000 7ff30affd640 1 -- 172.21.15.125:0/1508599137 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff30c017090 con 0x7ff314141690 2023-10-18T10:44:56.874 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:56.875 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:44:56.875 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T10:44:56.875 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.875 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T10:44:56.875 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T10:44:56.876 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.876 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:56.876 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:56.876 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:56.876 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.877 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.877 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.877 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.877 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T10:44:56.877 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T10:44:56.878 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.878 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:56.878 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:56.878 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:56.878 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.879 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.879 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.879 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.879 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T10:44:56.879 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T10:44:56.879 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.880 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:56.880 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:56.880 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:56.880 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.880 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.881 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.881 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.881 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T10:44:56.881 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T10:44:56.881 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.882 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:56.882 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:56.882 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:56.882 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.882 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.883 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.883 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.883 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T10:44:56.883 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T10:44:56.883 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.883 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:56.884 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:56.884 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.884 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.884 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.884 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.885 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T10:44:56.885 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T10:44:56.885 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.885 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:56.885 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:56.886 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.886 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.886 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.886 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.886 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T10:44:56.887 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T10:44:56.887 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.887 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:56.887 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:56.887 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.887 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.888 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.888 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.888 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T10:44:56.888 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T10:44:56.888 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.889 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:56.889 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:56.889 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.889 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.889 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.890 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.890 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T10:44:56.890 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T10:44:56.890 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.890 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:56.890 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:56.891 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.891 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.891 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.891 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.891 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T10:44:56.892 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T10:44:56.892 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.892 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:56.892 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:56.892 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.893 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.893 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.893 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.893 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T10:44:56.893 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T10:44:56.893 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.894 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:56.894 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:56.894 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.894 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.895 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.895 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.895 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T10:44:56.895 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T10:44:56.895 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.896 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T10:44:56.896 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T10:44:56.896 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.896 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.896 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.896 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.897 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T10:44:56.897 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T10:44:56.897 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.897 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:56.897 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T10:44:56.898 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T10:44:56.898 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.898 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.898 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.898 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T10:44:56.899 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T10:44:56.899 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.899 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:56.899 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:44:56.899 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:44:56.900 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:44:56.900 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.900 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.900 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.900 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T10:44:56.901 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T10:44:56.901 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.901 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:56.901 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:44:56.901 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:44:56.901 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:44:56.902 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.902 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.902 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.902 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T10:44:56.902 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T10:44:56.903 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.903 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T10:44:56.903 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T10:44:56.903 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T10:44:56.903 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T10:44:56.904 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.904 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.904 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.904 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T10:44:56.904 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T10:44:56.904 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.905 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:44:56.905 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T10:44:56.905 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T10:44:56.905 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.905 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.906 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.906 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.906 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T10:44:56.906 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T10:44:56.906 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.907 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:44:56.907 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:44:56.907 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.907 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.907 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:44:56.907 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:56.908 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T10:44:56.908 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T10:44:56.908 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T10:44:56.908 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T10:44:56.908 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T10:44:56.909 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T10:44:56.909 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.909 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:56.909 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T10:44:56.909 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:44:56.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff2f406af60 msgr2=0x7ff2f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:56.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff2f406af60 0x7ff2f406d420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff304035000 tx=0x7ff304004730 comp rx=0 tx=0).stop 2023-10-18T10:44:56.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff314141690 msgr2=0x7ff31413ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff314141690 0x7ff31413ac10 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c7b0 tx=0x7ff30c00cc70 comp rx=0 tx=0).stop 2023-10-18T10:44:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 shutdown_connections 2023-10-18T10:44:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff2f406af60 0x7ff2f406d420 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff31413e520 0x7ff314153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff314141690 0x7ff31413ac10 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 --2- 172.21.15.125:0/1508599137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff31406c0a0 0x7ff314152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:56.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 >> 172.21.15.125:0/1508599137 conn(0x7ff314069930 msgr2=0x7ff3140c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:56.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 shutdown_connections 2023-10-18T10:44:56.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:56.874+0000 7ff318e32640 1 -- 172.21.15.125:0/1508599137 wait complete. 2023-10-18T10:44:56.914 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:44:57.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.015+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1698472853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b80003970 msgr2=0x7f7b80003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.015+0000 7f7b9cf68640 1 --2- 172.21.15.125:0/1698472853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b80003970 0x7f7b80003d90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009b50 tx=0x7f7b800057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.015+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1698472853 shutdown_connections 2023-10-18T10:44:57.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.015+0000 7f7b9cf68640 1 --2- 172.21.15.125:0/1698472853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b80003970 0x7f7b80003d90 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.015+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1698472853 >> 172.21.15.125:0/1698472853 conn(0x7f7b98069930 msgr2=0x7f7b98069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:57.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.015+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1698472853 shutdown_connections 2023-10-18T10:44:57.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.015+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1698472853 wait complete. 2023-10-18T10:44:57.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.016+0000 7f7b9cf68640 1 Processor -- start 2023-10-18T10:44:57.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.016+0000 7f7b9cf68640 1 -- start start 2023-10-18T10:44:57.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b9cf68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b80003970 0x7f7b9813b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b9cf68640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b980acde0 0x7f7b980ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b9cf68640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b980c1ae0 0x7f7b980c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b9cf68640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981f2170 con 0x7f7b980c1ae0 2023-10-18T10:44:57.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b9cf68640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980ad780 con 0x7f7b80003970 2023-10-18T10:44:57.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b9cf68640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9813b780 con 0x7f7b980acde0 2023-10-18T10:44:57.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b96d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b980c1ae0 0x7f7b980c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b96d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b980c1ae0 0x7f7b980c1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38140/0 (socket says 172.21.15.125:38140) 2023-10-18T10:44:57.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b96d9d640 1 -- 172.21.15.125:0/1107611882 learned_addr learned my addr 172.21.15.125:0/1107611882 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:57.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b9659c640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b80003970 0x7f7b9813b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b95d9b640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b980acde0 0x7f7b980ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b96d9d640 1 -- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b980acde0 msgr2=0x7f7b980ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b96d9d640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b980acde0 0x7f7b980ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b96d9d640 1 -- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b80003970 msgr2=0x7f7b9813b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b96d9d640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b80003970 0x7f7b9813b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b96d9d640 1 -- 172.21.15.125:0/1107611882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b981f14f0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.017+0000 7f7b9659c640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b80003970 0x7f7b9813b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:57.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.018+0000 7f7b96d9d640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b980c1ae0 0x7f7b980c1f60 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7b9806d570 tx=0x7f7b90001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:57.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.018+0000 7f7b86ffd640 1 -- 172.21.15.125:0/1107611882 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90011840 con 0x7f7b980c1ae0 2023-10-18T10:44:57.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.018+0000 7f7b86ffd640 1 -- 172.21.15.125:0/1107611882 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b900119a0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.018+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b981f17d0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.018+0000 7f7b86ffd640 1 -- 172.21.15.125:0/1107611882 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90011d90 con 0x7f7b980c1ae0 2023-10-18T10:44:57.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.018+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b981f1d90 con 0x7f7b980c1ae0 2023-10-18T10:44:57.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.019+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b981f22b0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.019+0000 7f7b86ffd640 1 -- 172.21.15.125:0/1107611882 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7b900103c0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.019+0000 7f7b86ffd640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7b7806af90 0x7f7b7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.020+0000 7f7b86ffd640 1 -- 172.21.15.125:0/1107611882 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f7b9008a5e0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.020+0000 7f7b9659c640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7b7806af90 0x7f7b7806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.020+0000 7f7b9659c640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7b7806af90 0x7f7b7806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c036000 tx=0x7f7b8c035040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:57.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.024+0000 7f7b86ffd640 1 -- 172.21.15.125:0/1107611882 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b981f22b0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.191+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b981f22b0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.192+0000 7f7b86ffd640 1 -- 172.21.15.125:0/1107611882 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+13164 (secure 0 0 0) 0x7f7b981f22b0 con 0x7f7b980c1ae0 2023-10-18T10:44:57.194 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:57.194 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":152,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:56.516286+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:44:57.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.193+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7b7806af90 msgr2=0x7f7b7806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.193+0000 7f7b9cf68640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7b7806af90 0x7f7b7806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c036000 tx=0x7f7b8c035040 comp rx=0 tx=0).stop 2023-10-18T10:44:57.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.193+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b980c1ae0 msgr2=0x7f7b980c1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.193+0000 7f7b9cf68640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b980c1ae0 0x7f7b980c1f60 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f7b9806d570 tx=0x7f7b90001c30 comp rx=0 tx=0).stop 2023-10-18T10:44:57.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.194+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 shutdown_connections 2023-10-18T10:44:57.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.194+0000 7f7b9cf68640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7b7806af90 0x7f7b7806d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.194+0000 7f7b9cf68640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b980c1ae0 0x7f7b980c1f60 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.194+0000 7f7b9cf68640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b980acde0 0x7f7b980ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.194+0000 7f7b9cf68640 1 --2- 172.21.15.125:0/1107611882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b80003970 0x7f7b9813b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.194+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 >> 172.21.15.125:0/1107611882 conn(0x7f7b98069930 msgr2=0x7f7b980ac670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:57.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.194+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 shutdown_connections 2023-10-18T10:44:57.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.194+0000 7f7b9cf68640 1 -- 172.21.15.125:0/1107611882 wait complete. 2023-10-18T10:44:57.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T10:44:57.207 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T10:44:57.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.310+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1183127137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 msgr2=0x7f1d540cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.310+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1183127137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 0x7f1d540cf720 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f1d5408a120 tx=0x7f1d3c02bd10 comp rx=0 tx=0).stop 2023-10-18T10:44:57.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.310+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1183127137 shutdown_connections 2023-10-18T10:44:57.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.310+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1183127137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 0x7f1d540cf720 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.310+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1183127137 >> v2:172.21.15.139:3300/0 conn(0x7f1d540cf340 0x7f1d540d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.310+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1183127137 >> 172.21.15.125:0/1183127137 conn(0x7f1d540d0d60 msgr2=0x7f1d540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:57.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.310+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1183127137 shutdown_connections 2023-10-18T10:44:57.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.310+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1183127137 wait complete. 2023-10-18T10:44:57.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.311+0000 7f1d58d1d640 1 Processor -- start 2023-10-18T10:44:57.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.311+0000 7f1d58d1d640 1 -- start start 2023-10-18T10:44:57.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d58d1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d540ceb40 0x7f1d54156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d58d1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d540cf340 0x7f1d54157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d58d1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 0x7f1d540b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d58d1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d541f4620 con 0x7f1d5415b2a0 2023-10-18T10:44:57.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d58d1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d54157120 con 0x7f1d540ceb40 2023-10-18T10:44:57.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d58d1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5406ab00 con 0x7f1d540cf340 2023-10-18T10:44:57.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d52d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 0x7f1d540b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d52d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 0x7f1d540b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38170/0 (socket says 172.21.15.125:38170) 2023-10-18T10:44:57.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.312+0000 7f1d52d9d640 1 -- 172.21.15.125:0/1337931443 learned_addr learned my addr 172.21.15.125:0/1337931443 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:57.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d5259c640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d540ceb40 0x7f1d54156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d4bfff640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d540cf340 0x7f1d54157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d52d9d640 1 -- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d540cf340 msgr2=0x7f1d54157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d52d9d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d540cf340 0x7f1d54157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d52d9d640 1 -- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d540ceb40 msgr2=0x7f1d54156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d52d9d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d540ceb40 0x7f1d54156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d52d9d640 1 -- 172.21.15.125:0/1337931443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c009560 con 0x7f1d5415b2a0 2023-10-18T10:44:57.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d5259c640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d540ceb40 0x7f1d54156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:57.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d4bfff640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d540cf340 0x7f1d54157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:57.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d52d9d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 0x7f1d540b40d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c7b0 tx=0x7f1d4c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:57.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d4affd640 1 -- 172.21.15.125:0/1337931443 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c003d80 con 0x7f1d5415b2a0 2023-10-18T10:44:57.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d4affd640 1 -- 172.21.15.125:0/1337931443 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d4c005070 con 0x7f1d5415b2a0 2023-10-18T10:44:57.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5406ade0 con 0x7f1d5415b2a0 2023-10-18T10:44:57.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d4affd640 1 -- 172.21.15.125:0/1337931443 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0054a0 con 0x7f1d5415b2a0 2023-10-18T10:44:57.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.313+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5406b3a0 con 0x7f1d5415b2a0 2023-10-18T10:44:57.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.314+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d540690e0 con 0x7f1d5415b2a0 2023-10-18T10:44:57.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.314+0000 7f1d4affd640 1 -- 172.21.15.125:0/1337931443 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1d4c005600 con 0x7f1d5415b2a0 2023-10-18T10:44:57.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.315+0000 7f1d4affd640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1d3406af90 0x7f1d3406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.315+0000 7f1d4affd640 1 -- 172.21.15.125:0/1337931443 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f1d4c088bd0 con 0x7f1d5415b2a0 2023-10-18T10:44:57.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.315+0000 7f1d5259c640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1d3406af90 0x7f1d3406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.315+0000 7f1d5259c640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1d3406af90 0x7f1d3406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d44006fd0 tx=0x7f1d44008040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:57.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.319+0000 7f1d4affd640 1 -- 172.21.15.125:0/1337931443 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d4c053d70 con 0x7f1d5415b2a0 2023-10-18T10:44:57.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.484+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d540cb9d0 con 0x7f1d5415b2a0 2023-10-18T10:44:57.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.485+0000 7f1d4affd640 1 -- 172.21.15.125:0/1337931443 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d4c057390 con 0x7f1d5415b2a0 2023-10-18T10:44:57.487 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T10:44:57.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1d3406af90 msgr2=0x7f1d3406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1d3406af90 0x7f1d3406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d44006fd0 tx=0x7f1d44008040 comp rx=0 tx=0).stop 2023-10-18T10:44:57.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 msgr2=0x7f1d540b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 0x7f1d540b40d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c7b0 tx=0x7f1d4c00cc70 comp rx=0 tx=0).stop 2023-10-18T10:44:57.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 shutdown_connections 2023-10-18T10:44:57.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f1d3406af90 0x7f1d3406d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d5415b2a0 0x7f1d540b40d0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d540cf340 0x7f1d54157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 --2- 172.21.15.125:0/1337931443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d540ceb40 0x7f1d54156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 >> 172.21.15.125:0/1337931443 conn(0x7f1d540d0d60 msgr2=0x7f1d5415a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:57.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 shutdown_connections 2023-10-18T10:44:57.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.487+0000 7f1d58d1d640 1 -- 172.21.15.125:0/1337931443 wait complete. 2023-10-18T10:44:57.499 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T10:44:57.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.600+0000 7fdc01f4c640 1 -- 172.21.15.125:0/1458143738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 msgr2=0x7fdbfc06df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.600+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/1458143738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 0x7fdbfc06df60 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4005fd0 tx=0x7fdbf402c1d0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.600+0000 7fdc01f4c640 1 -- 172.21.15.125:0/1458143738 shutdown_connections 2023-10-18T10:44:57.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.600+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/1458143738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 0x7fdbfc06df60 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.600+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/1458143738 >> v2:172.21.15.139:3300/0 conn(0x7fdbfc0b3070 0x7fdbfc06da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.600+0000 7fdc01f4c640 1 -- 172.21.15.125:0/1458143738 >> 172.21.15.125:0/1458143738 conn(0x7fdbfc0b3f00 msgr2=0x7fdbfc0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:57.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.600+0000 7fdc01f4c640 1 -- 172.21.15.125:0/1458143738 shutdown_connections 2023-10-18T10:44:57.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.600+0000 7fdc01f4c640 1 -- 172.21.15.125:0/1458143738 wait complete. 2023-10-18T10:44:57.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.601+0000 7fdc01f4c640 1 Processor -- start 2023-10-18T10:44:57.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.601+0000 7fdc01f4c640 1 -- start start 2023-10-18T10:44:57.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdc01f4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 0x7fdbfc144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdc01f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc0b24d0 0x7fdbfc0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdc01f4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbfc0b3070 0x7fdbfc1475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdc01f4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0c7d60 con 0x7fdbfc0acd40 2023-10-18T10:44:57.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdc01f4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0c9cc0 con 0x7fdbfc0b24d0 2023-10-18T10:44:57.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdc01f4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1480c0 con 0x7fdbfc0b3070 2023-10-18T10:44:57.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 0x7fdbfc144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 0x7fdbfc144ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36660/0 (socket says 172.21.15.125:36660) 2023-10-18T10:44:57.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 -- 172.21.15.125:0/338684412 learned_addr learned my addr 172.21.15.125:0/338684412 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:57.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 -- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbfc0b3070 msgr2=0x7fdbfc1475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbfc0b3070 0x7fdbfc1475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 -- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc0b24d0 msgr2=0x7fdbfc0c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfaffd640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc0b24d0 0x7fdbfc0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc0b24d0 0x7fdbfc0c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 -- 172.21.15.125:0/338684412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf4009900 con 0x7fdbfc0acd40 2023-10-18T10:44:57.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbfb7fe640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 0x7fdbfc144ef0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c430 tx=0x7fdbec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:57.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbe3fff640 1 -- 172.21.15.125:0/338684412 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec011790 con 0x7fdbfc0acd40 2023-10-18T10:44:57.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbe3fff640 1 -- 172.21.15.125:0/338684412 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdbec0119e0 con 0x7fdbfc0acd40 2023-10-18T10:44:57.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc1483a0 con 0x7fdbfc0acd40 2023-10-18T10:44:57.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdbe3fff640 1 -- 172.21.15.125:0/338684412 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec011e30 con 0x7fdbfc0acd40 2023-10-18T10:44:57.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.602+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc0c8290 con 0x7fdbfc0acd40 2023-10-18T10:44:57.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.603+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbfc0ae190 con 0x7fdbfc0acd40 2023-10-18T10:44:57.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.604+0000 7fdbe3fff640 1 -- 172.21.15.125:0/338684412 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdbec01c460 con 0x7fdbfc0acd40 2023-10-18T10:44:57.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.604+0000 7fdbe3fff640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fdbe806af60 0x7fdbe806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.604+0000 7fdbe3fff640 1 -- 172.21.15.125:0/338684412 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5205+0+0 (secure 0 0 0) 0x7fdbec089770 con 0x7fdbfc0acd40 2023-10-18T10:44:57.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.604+0000 7fdbfaffd640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fdbe806af60 0x7fdbe806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.605+0000 7fdbfaffd640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fdbe806af60 0x7fdbe806d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00058c0 tx=0x7fdbf0005850 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:57.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.608+0000 7fdbe3fff640 1 -- 172.21.15.125:0/338684412 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdbec054990 con 0x7fdbfc0acd40 2023-10-18T10:44:57.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.774+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdbfc0690e0 con 0x7fdbfc0acd40 2023-10-18T10:44:57.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.775+0000 7fdbe3fff640 1 -- 172.21.15.125:0/338684412 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v152) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdbec057fb0 con 0x7fdbfc0acd40 2023-10-18T10:44:57.777 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:44:57.777 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:44:57.778 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:44:57.778 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T10:44:57.778 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T10:44:57.778 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:44:57.779 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:44:57.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fdbe806af60 msgr2=0x7fdbe806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fdbe806af60 0x7fdbe806d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00058c0 tx=0x7fdbf0005850 comp rx=0 tx=0).stop 2023-10-18T10:44:57.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 msgr2=0x7fdbfc144ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 0x7fdbfc144ef0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c430 tx=0x7fdbec00c8f0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 shutdown_connections 2023-10-18T10:44:57.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fdbe806af60 0x7fdbe806d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc0b24d0 0x7fdbfc0c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbfc0b3070 0x7fdbfc1475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 --2- 172.21.15.125:0/338684412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbfc0acd40 0x7fdbfc144ef0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 >> 172.21.15.125:0/338684412 conn(0x7fdbfc0b3f00 msgr2=0x7fdbfc15b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:57.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 shutdown_connections 2023-10-18T10:44:57.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.777+0000 7fdc01f4c640 1 -- 172.21.15.125:0/338684412 wait complete. 2023-10-18T10:44:57.789 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T10:44:57.790 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T10:44:57.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.894+0000 7f945fbee640 1 -- 172.21.15.125:0/449407165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f945806af00 msgr2=0x7f945806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.894+0000 7f945fbee640 1 --2- 172.21.15.125:0/449407165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f945806af00 0x7f945806b2e0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9454009830 tx=0x7f945402bcd0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.894+0000 7f945fbee640 1 -- 172.21.15.125:0/449407165 shutdown_connections 2023-10-18T10:44:57.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.894+0000 7f945fbee640 1 --2- 172.21.15.125:0/449407165 >> v2:172.21.15.139:3300/0 conn(0x7f94580abc10 0x7f94580c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.894+0000 7f945fbee640 1 --2- 172.21.15.125:0/449407165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f945806af00 0x7f945806b2e0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.894+0000 7f945fbee640 1 -- 172.21.15.125:0/449407165 >> 172.21.15.125:0/449407165 conn(0x7f9458069930 msgr2=0x7f9458069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:57.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.895+0000 7f945fbee640 1 -- 172.21.15.125:0/449407165 shutdown_connections 2023-10-18T10:44:57.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.895+0000 7f945fbee640 1 -- 172.21.15.125:0/449407165 wait complete. 2023-10-18T10:44:57.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945fbee640 1 Processor -- start 2023-10-18T10:44:57.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945fbee640 1 -- start start 2023-10-18T10:44:57.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945fbee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f945806af00 0x7f9458151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945fbee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94580abc10 0x7f94580af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945fbee640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94580c96b0 0x7f9458137d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945fbee640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580b1160 con 0x7f94580c96b0 2023-10-18T10:44:57.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945fbee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580b00c0 con 0x7f945806af00 2023-10-18T10:44:57.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945fbee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94580b03a0 con 0x7f94580abc10 2023-10-18T10:44:57.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945e18b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94580c96b0 0x7f9458137d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945e18b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94580c96b0 0x7f9458137d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36678/0 (socket says 172.21.15.125:36678) 2023-10-18T10:44:57.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.896+0000 7f945e18b640 1 -- 172.21.15.125:0/3960627331 learned_addr learned my addr 172.21.15.125:0/3960627331 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:57.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945d98a640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f945806af00 0x7f9458151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945d189640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94580abc10 0x7f94580af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945e18b640 1 -- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94580abc10 msgr2=0x7f94580af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945e18b640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94580abc10 0x7f94580af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945e18b640 1 -- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f945806af00 msgr2=0x7f9458151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:57.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945e18b640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f945806af00 0x7f9458151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:57.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945e18b640 1 -- 172.21.15.125:0/3960627331 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9454009490 con 0x7f94580c96b0 2023-10-18T10:44:57.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945d189640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94580abc10 0x7f94580af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:44:57.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945e18b640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94580c96b0 0x7f9458137d60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f945000eaa0 tx=0x7f945000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:57.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f944e7fc640 1 -- 172.21.15.125:0/3960627331 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000cdb0 con 0x7f94580c96b0 2023-10-18T10:44:57.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f944e7fc640 1 -- 172.21.15.125:0/3960627331 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9450004b20 con 0x7f94580c96b0 2023-10-18T10:44:57.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94580b0680 con 0x7f94580c96b0 2023-10-18T10:44:57.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f944e7fc640 1 -- 172.21.15.125:0/3960627331 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450005020 con 0x7f94580c96b0 2023-10-18T10:44:57.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.897+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94580b0c40 con 0x7f94580c96b0 2023-10-18T10:44:57.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.898+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94580690e0 con 0x7f94580c96b0 2023-10-18T10:44:57.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.898+0000 7f944e7fc640 1 -- 172.21.15.125:0/3960627331 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9450005180 con 0x7f94580c96b0 2023-10-18T10:44:57.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.899+0000 7f944e7fc640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f943c06af60 0x7f943c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:57.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.899+0000 7f944e7fc640 1 -- 172.21.15.125:0/3960627331 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f945008a040 con 0x7f94580c96b0 2023-10-18T10:44:57.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.899+0000 7f945d98a640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f943c06af60 0x7f943c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:57.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.900+0000 7f945d98a640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f943c06af60 0x7f943c06d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9454036000 tx=0x7f9454004730 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:57.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:57.903+0000 7f944e7fc640 1 -- 172.21.15.125:0/3960627331 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9450055150 con 0x7f94580c96b0 2023-10-18T10:44:58.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.068+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f94580cc420 con 0x7f94580c96b0 2023-10-18T10:44:58.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.523+0000 7f944e7fc640 1 -- 172.21.15.125:0/3960627331 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v153) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9450058770 con 0x7f94580c96b0 2023-10-18T10:44:58.525 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T10:44:58.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f943c06af60 msgr2=0x7f943c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:58.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f943c06af60 0x7f943c06d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9454036000 tx=0x7f9454004730 comp rx=0 tx=0).stop 2023-10-18T10:44:58.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94580c96b0 msgr2=0x7f9458137d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:58.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94580c96b0 0x7f9458137d60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f945000eaa0 tx=0x7f945000ef60 comp rx=0 tx=0).stop 2023-10-18T10:44:58.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 shutdown_connections 2023-10-18T10:44:58.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f943c06af60 0x7f943c06d420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:58.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94580c96b0 0x7f9458137d60 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:58.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94580abc10 0x7f94580af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:58.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 --2- 172.21.15.125:0/3960627331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f945806af00 0x7f9458151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:58.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 >> 172.21.15.125:0/3960627331 conn(0x7f9458069930 msgr2=0x7f94580cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:58.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 shutdown_connections 2023-10-18T10:44:58.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.525+0000 7f945fbee640 1 -- 172.21.15.125:0/3960627331 wait complete. 2023-10-18T10:44:58.542 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T10:44:58.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.653+0000 7f7834b3e640 1 -- 172.21.15.125:0/4142268075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78300ceb40 msgr2=0x7f78300b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:58.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.653+0000 7f7834b3e640 1 --2- 172.21.15.125:0/4142268075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78300ceb40 0x7f78300b0650 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7828005fd0 tx=0x7f782802bd90 comp rx=0 tx=0).stop 2023-10-18T10:44:58.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.653+0000 7f7834b3e640 1 -- 172.21.15.125:0/4142268075 shutdown_connections 2023-10-18T10:44:58.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.653+0000 7f7834b3e640 1 --2- 172.21.15.125:0/4142268075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78300ceb40 0x7f78300b0650 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:58.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.653+0000 7f7834b3e640 1 --2- 172.21.15.125:0/4142268075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f783015b2a0 0x7f78300d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:58.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.653+0000 7f7834b3e640 1 -- 172.21.15.125:0/4142268075 >> 172.21.15.125:0/4142268075 conn(0x7f78300d0d60 msgr2=0x7f78300d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:58.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.653+0000 7f7834b3e640 1 -- 172.21.15.125:0/4142268075 shutdown_connections 2023-10-18T10:44:58.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.653+0000 7f7834b3e640 1 -- 172.21.15.125:0/4142268075 wait complete. 2023-10-18T10:44:58.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.654+0000 7f7834b3e640 1 Processor -- start 2023-10-18T10:44:58.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.654+0000 7f7834b3e640 1 -- start start 2023-10-18T10:44:58.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f7834b3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78300ceb40 0x7f783014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:58.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f7834b3e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78300cf340 0x7f7830150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:58.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f7834b3e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f783015b2a0 0x7f78300c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:58.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f7834b3e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7830151850 con 0x7f78300cf340 2023-10-18T10:44:58.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f7834b3e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7830150970 con 0x7f78300ceb40 2023-10-18T10:44:58.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f7834b3e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7830150bd0 con 0x7f783015b2a0 2023-10-18T10:44:58.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78300cf340 0x7f7830150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:58.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78300cf340 0x7f7830150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36682/0 (socket says 172.21.15.125:36682) 2023-10-18T10:44:58.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 -- 172.21.15.125:0/202926958 learned_addr learned my addr 172.21.15.125:0/202926958 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782ed9d640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f783015b2a0 0x7f78300c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782e59c640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78300ceb40 0x7f783014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 -- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f783015b2a0 msgr2=0x7f78300c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:58.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f783015b2a0 0x7f78300c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 -- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78300ceb40 msgr2=0x7f783014fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78300ceb40 0x7f783014fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 -- 172.21.15.125:0/202926958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824009490 con 0x7f78300cf340 2023-10-18T10:44:58.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.655+0000 7f782dd9b640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78300cf340 0x7f7830150430 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f782400e9f0 tx=0x7f782400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.656+0000 7f781effd640 1 -- 172.21.15.125:0/202926958 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782400cd80 con 0x7f78300cf340 2023-10-18T10:44:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.656+0000 7f781effd640 1 -- 172.21.15.125:0/202926958 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7824004020 con 0x7f78300cf340 2023-10-18T10:44:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.656+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7828009560 con 0x7f78300cf340 2023-10-18T10:44:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.656+0000 7f781effd640 1 -- 172.21.15.125:0/202926958 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782400cee0 con 0x7f78300cf340 2023-10-18T10:44:58.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.656+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78300b0d50 con 0x7f78300cf340 2023-10-18T10:44:58.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.656+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78301e48c0 con 0x7f78300cf340 2023-10-18T10:44:58.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.657+0000 7f781effd640 1 -- 172.21.15.125:0/202926958 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f78240035f0 con 0x7f78300cf340 2023-10-18T10:44:58.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.657+0000 7f781effd640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f781006af60 0x7f781006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:58.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.658+0000 7f781effd640 1 -- 172.21.15.125:0/202926958 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5575+0+0 (secure 0 0 0) 0x7f7824014070 con 0x7f78300cf340 2023-10-18T10:44:58.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.658+0000 7f782e59c640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f781006af60 0x7f781006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:58.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.658+0000 7f782e59c640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f781006af60 0x7f781006d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7820001f10 tx=0x7f7820009040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:58.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.661+0000 7f781effd640 1 -- 172.21.15.125:0/202926958 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78240551e0 con 0x7f78300cf340 2023-10-18T10:44:58.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:58.827+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f78301e48c0 con 0x7f78300cf340 2023-10-18T10:44:59.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.530+0000 7f781effd640 1 -- 172.21.15.125:0/202926958 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v154) v1 ==== 170+0+0 (secure 0 0 0) 0x7f78301e48c0 con 0x7f78300cf340 2023-10-18T10:44:59.532 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T10:44:59.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f781006af60 msgr2=0x7f781006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:59.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f781006af60 0x7f781006d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7820001f10 tx=0x7f7820009040 comp rx=0 tx=0).stop 2023-10-18T10:44:59.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78300cf340 msgr2=0x7f7830150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:59.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78300cf340 0x7f7830150430 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f782400e9f0 tx=0x7f782400eeb0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 shutdown_connections 2023-10-18T10:44:59.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f781006af60 0x7f781006d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f783015b2a0 0x7f78300c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78300cf340 0x7f7830150430 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 --2- 172.21.15.125:0/202926958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78300ceb40 0x7f783014fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 >> 172.21.15.125:0/202926958 conn(0x7f78300d0d60 msgr2=0x7f783006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:59.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 shutdown_connections 2023-10-18T10:44:59.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.533+0000 7f7834b3e640 1 -- 172.21.15.125:0/202926958 wait complete. 2023-10-18T10:44:59.556 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T10:44:59.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.663+0000 7fd9d974c640 1 -- 172.21.15.125:0/3509845458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9bc003970 msgr2=0x7fd9bc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:59.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.663+0000 7fd9d974c640 1 --2- 172.21.15.125:0/3509845458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9bc003970 0x7fd9bc003d90 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009ed0 tx=0x7fd9bc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.663+0000 7fd9d974c640 1 -- 172.21.15.125:0/3509845458 shutdown_connections 2023-10-18T10:44:59.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.663+0000 7fd9d974c640 1 --2- 172.21.15.125:0/3509845458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9bc003970 0x7fd9bc003d90 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.663+0000 7fd9d974c640 1 -- 172.21.15.125:0/3509845458 >> 172.21.15.125:0/3509845458 conn(0x7fd9d40d0d90 msgr2=0x7fd9d40d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:44:59.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.663+0000 7fd9d974c640 1 -- 172.21.15.125:0/3509845458 shutdown_connections 2023-10-18T10:44:59.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.663+0000 7fd9d974c640 1 -- 172.21.15.125:0/3509845458 wait complete. 2023-10-18T10:44:59.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.664+0000 7fd9d974c640 1 Processor -- start 2023-10-18T10:44:59.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.664+0000 7fd9d974c640 1 -- start start 2023-10-18T10:44:59.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d974c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9bc003970 0x7fd9d40b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:59.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d974c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9d40b3ed0 0x7fd9d4146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:59.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d974c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4147400 0x7fd9d40c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:59.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d974c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40ca7f0 con 0x7fd9d4147400 2023-10-18T10:44:59.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d974c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40b4350 con 0x7fd9d40b3ed0 2023-10-18T10:44:59.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d974c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d41478b0 con 0x7fd9bc003970 2023-10-18T10:44:59.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4147400 0x7fd9d40c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:59.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4147400 0x7fd9d40c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36696/0 (socket says 172.21.15.125:36696) 2023-10-18T10:44:59.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 -- 172.21.15.125:0/4037372330 learned_addr learned my addr 172.21.15.125:0/4037372330 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:44:59.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d27fc640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9d40b3ed0 0x7fd9d4146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:59.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d2ffd640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9bc003970 0x7fd9d40b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:59.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 -- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9bc003970 msgr2=0x7fd9d40b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:59.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9bc003970 0x7fd9d40b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 -- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9d40b3ed0 msgr2=0x7fd9d4146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:44:59.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9d40b3ed0 0x7fd9d4146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:44:59.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 -- 172.21.15.125:0/4037372330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d415e660 con 0x7fd9d4147400 2023-10-18T10:44:59.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d2ffd640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9bc003970 0x7fd9d40b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:44:59.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9d37fe640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4147400 0x7fd9d40c8eb0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40cd6f0 tx=0x7fd9cc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:59.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9bb7fe640 1 -- 172.21.15.125:0/4037372330 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc011840 con 0x7fd9d4147400 2023-10-18T10:44:59.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9bb7fe640 1 -- 172.21.15.125:0/4037372330 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9cc0065b0 con 0x7fd9d4147400 2023-10-18T10:44:59.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.665+0000 7fd9bb7fe640 1 -- 172.21.15.125:0/4037372330 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc011c60 con 0x7fd9d4147400 2023-10-18T10:44:59.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.666+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d40c9b70 con 0x7fd9d4147400 2023-10-18T10:44:59.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.666+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d40ca130 con 0x7fd9d4147400 2023-10-18T10:44:59.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.667+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d4066960 con 0x7fd9d4147400 2023-10-18T10:44:59.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.667+0000 7fd9bb7fe640 1 -- 172.21.15.125:0/4037372330 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd9cc0103c0 con 0x7fd9d4147400 2023-10-18T10:44:59.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.667+0000 7fd9bb7fe640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fd9c006af60 0x7fd9c006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:44:59.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.667+0000 7fd9bb7fe640 1 -- 172.21.15.125:0/4037372330 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fd9cc08a620 con 0x7fd9d4147400 2023-10-18T10:44:59.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.668+0000 7fd9d2ffd640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fd9c006af60 0x7fd9c006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:44:59.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.668+0000 7fd9d2ffd640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fd9c006af60 0x7fd9c006d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4037000 tx=0x7fd9c4033040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:44:59.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.672+0000 7fd9bb7fe640 1 -- 172.21.15.125:0/4037372330 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9cc0554f0 con 0x7fd9d4147400 2023-10-18T10:44:59.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:44:59.875+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd9d40675f0 con 0x7fd9d4147400 2023-10-18T10:45:00.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.532+0000 7fd9bb7fe640 1 -- 172.21.15.125:0/4037372330 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd9cc01f0c0 con 0x7fd9d4147400 2023-10-18T10:45:00.534 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 19 and data pool 20 2023-10-18T10:45:00.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.535+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fd9c006af60 msgr2=0x7fd9c006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.535+0000 7fd9d974c640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fd9c006af60 0x7fd9c006d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4037000 tx=0x7fd9c4033040 comp rx=0 tx=0).stop 2023-10-18T10:45:00.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.535+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4147400 msgr2=0x7fd9d40c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.535+0000 7fd9d974c640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4147400 0x7fd9d40c8eb0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40cd6f0 tx=0x7fd9cc0039c0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.536+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 shutdown_connections 2023-10-18T10:45:00.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.536+0000 7fd9d974c640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fd9c006af60 0x7fd9c006d420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.536+0000 7fd9d974c640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9d40b3ed0 0x7fd9d4146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.536+0000 7fd9d974c640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d4147400 0x7fd9d40c8eb0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.536+0000 7fd9d974c640 1 --2- 172.21.15.125:0/4037372330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9bc003970 0x7fd9d40b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.536+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 >> 172.21.15.125:0/4037372330 conn(0x7fd9d40d0d90 msgr2=0x7fd9d40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:00.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.538+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 shutdown_connections 2023-10-18T10:45:00.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.540+0000 7fd9d974c640 1 -- 172.21.15.125:0/4037372330 wait complete. 2023-10-18T10:45:00.567 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:00.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.667+0000 7f45affb6640 1 -- 172.21.15.125:0/3530113315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a8070350 msgr2=0x7f45a80707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.667+0000 7f45affb6640 1 --2- 172.21.15.125:0/3530113315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a8070350 0x7f45a80707e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005fd0 tx=0x7f45a002c0e0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.667+0000 7f45affb6640 1 -- 172.21.15.125:0/3530113315 shutdown_connections 2023-10-18T10:45:00.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.667+0000 7f45affb6640 1 --2- 172.21.15.125:0/3530113315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a8070350 0x7f45a80707e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.667+0000 7f45affb6640 1 --2- 172.21.15.125:0/3530113315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45a80cb980 0x7f45a813db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.667+0000 7f45affb6640 1 -- 172.21.15.125:0/3530113315 >> 172.21.15.125:0/3530113315 conn(0x7f45a8069930 msgr2=0x7f45a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:00.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.667+0000 7f45affb6640 1 -- 172.21.15.125:0/3530113315 shutdown_connections 2023-10-18T10:45:00.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.667+0000 7f45affb6640 1 -- 172.21.15.125:0/3530113315 wait complete. 2023-10-18T10:45:00.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.668+0000 7f45affb6640 1 Processor -- start 2023-10-18T10:45:00.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.668+0000 7f45affb6640 1 -- start start 2023-10-18T10:45:00.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45affb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45a8070350 0x7f45a8151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:00.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45affb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a80cb980 0x7f45a81ef710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:00.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45affb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45a81f1b40 0x7f45a813bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:00.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45affb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81f2ae0 con 0x7f45a81f1b40 2023-10-18T10:45:00.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45affb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81f2520 con 0x7f45a80cb980 2023-10-18T10:45:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45affb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81f2800 con 0x7f45a8070350 2023-10-18T10:45:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45a81f1b40 0x7f45a813bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45a81f1b40 0x7f45a813bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36702/0 (socket says 172.21.15.125:36702) 2023-10-18T10:45:00.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 -- 172.21.15.125:0/3647045617 learned_addr learned my addr 172.21.15.125:0/3647045617 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45add52640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45a8070350 0x7f45a8151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ad551640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a80cb980 0x7f45a81ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 -- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45a8070350 msgr2=0x7f45a8151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45a8070350 0x7f45a8151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 -- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a80cb980 msgr2=0x7f45a81ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a80cb980 0x7f45a81ef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 -- 172.21.15.125:0/3647045617 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a4009560 con 0x7f45a81f1b40 2023-10-18T10:45:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45add52640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45a8070350 0x7f45a8151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ad551640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a80cb980 0x7f45a81ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T10:45:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.669+0000 7f45ae553640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45a81f1b40 0x7f45a813bfa0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f45a0036040 tx=0x7f45a0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:00.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.670+0000 7f459e7fc640 1 -- 172.21.15.125:0/3647045617 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a0004880 con 0x7f45a81f1b40 2023-10-18T10:45:00.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.670+0000 7f459e7fc640 1 -- 172.21.15.125:0/3647045617 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45a0040070 con 0x7f45a81f1b40 2023-10-18T10:45:00.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.670+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a0009840 con 0x7f45a81f1b40 2023-10-18T10:45:00.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.670+0000 7f459e7fc640 1 -- 172.21.15.125:0/3647045617 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a00347e0 con 0x7f45a81f1b40 2023-10-18T10:45:00.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.670+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a814f650 con 0x7f45a81f1b40 2023-10-18T10:45:00.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.671+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a814f1d0 con 0x7f45a81f1b40 2023-10-18T10:45:00.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.671+0000 7f459e7fc640 1 -- 172.21.15.125:0/3647045617 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f45a0034940 con 0x7f45a81f1b40 2023-10-18T10:45:00.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.671+0000 7f459e7fc640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f458c06af60 0x7f458c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:00.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.672+0000 7f459e7fc640 1 -- 172.21.15.125:0/3647045617 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f45a0042070 con 0x7f45a81f1b40 2023-10-18T10:45:00.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.672+0000 7f45add52640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f458c06af60 0x7f458c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:00.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.672+0000 7f45add52640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f458c06af60 0x7f458c06d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009530 tx=0x7f45a4009260 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:00.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.676+0000 7f459e7fc640 1 -- 172.21.15.125:0/3647045617 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45a814f1d0 con 0x7f45a81f1b40 2023-10-18T10:45:00.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.843+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45a806b460 con 0x7f45a81f1b40 2023-10-18T10:45:00.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.844+0000 7f459e7fc640 1 -- 172.21.15.125:0/3647045617 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f45a0079710 con 0x7f45a81f1b40 2023-10-18T10:45:00.846 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:00.846 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":155,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:59.877181+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:58.087653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:58.847874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:00.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f458c06af60 msgr2=0x7f458c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f458c06af60 0x7f458c06d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009530 tx=0x7f45a4009260 comp rx=0 tx=0).stop 2023-10-18T10:45:00.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45a81f1b40 msgr2=0x7f45a813bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45a81f1b40 0x7f45a813bfa0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f45a0036040 tx=0x7f45a0005c80 comp rx=0 tx=0).stop 2023-10-18T10:45:00.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 shutdown_connections 2023-10-18T10:45:00.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f458c06af60 0x7f458c06d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45a80cb980 0x7f45a81ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45a81f1b40 0x7f45a813bfa0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 --2- 172.21.15.125:0/3647045617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45a8070350 0x7f45a8151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 >> 172.21.15.125:0/3647045617 conn(0x7f45a8069930 msgr2=0x7f45a8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:00.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 shutdown_connections 2023-10-18T10:45:00.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.846+0000 7f45affb6640 1 -- 172.21.15.125:0/3647045617 wait complete. 2023-10-18T10:45:00.858 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:00.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.959+0000 7f06912e5640 1 -- 172.21.15.125:0/2937872686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06740047c0 msgr2=0x7f0674004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.959+0000 7f06912e5640 1 --2- 172.21.15.125:0/2937872686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06740047c0 0x7f0674004c40 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f068000baf0 tx=0x7f06740057f0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.959+0000 7f068b7fe640 1 -- 172.21.15.125:0/2937872686 reap_dead start 2023-10-18T10:45:00.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.959+0000 7f06912e5640 1 -- 172.21.15.125:0/2937872686 shutdown_connections 2023-10-18T10:45:00.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.959+0000 7f06912e5640 1 -- 172.21.15.125:0/2937872686 >> 172.21.15.125:0/2937872686 conn(0x7f068c069930 msgr2=0x7f068c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:00.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.959+0000 7f06912e5640 1 -- 172.21.15.125:0/2937872686 shutdown_connections 2023-10-18T10:45:00.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.959+0000 7f06912e5640 1 -- 172.21.15.125:0/2937872686 wait complete. 2023-10-18T10:45:00.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f06912e5640 1 Processor -- start 2023-10-18T10:45:00.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f06912e5640 1 -- start start 2023-10-18T10:45:00.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f06912e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06740047c0 0x7f068c14fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:00.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f06912e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f068c15b4b0 0x7f068c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:00.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f06912e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068c12cbb0 0x7f068c12d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:00.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f06912e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c15be70 con 0x7f068c15b4b0 2023-10-18T10:45:00.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f06912e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c12d960 con 0x7f06740047c0 2023-10-18T10:45:00.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f06912e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c154ad0 con 0x7f068c12cbb0 2023-10-18T10:45:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.961+0000 7f068a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f068c15b4b0 0x7f068c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f068c15b4b0 0x7f068c15b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36716/0 (socket says 172.21.15.125:36716) 2023-10-18T10:45:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068a7fc640 1 -- 172.21.15.125:0/2493999774 learned_addr learned my addr 172.21.15.125:0/2493999774 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068affd640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06740047c0 0x7f068c14fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:00.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068b7fe640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068c12cbb0 0x7f068c12d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068a7fc640 1 -- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068c12cbb0 msgr2=0x7f068c12d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068a7fc640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068c12cbb0 0x7f068c12d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068a7fc640 1 -- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06740047c0 msgr2=0x7f068c14fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068a7fc640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06740047c0 0x7f068c14fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:00.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068a7fc640 1 -- 172.21.15.125:0/2493999774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c154db0 con 0x7f068c15b4b0 2023-10-18T10:45:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f068a7fc640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f068c15b4b0 0x7f068c15b930 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f0680038040 tx=0x7f0680036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f06737fe640 1 -- 172.21.15.125:0/2493999774 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680036d20 con 0x7f068c15b4b0 2023-10-18T10:45:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f06737fe640 1 -- 172.21.15.125:0/2493999774 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0680004030 con 0x7f068c15b4b0 2023-10-18T10:45:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c155090 con 0x7f068c15b4b0 2023-10-18T10:45:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f06737fe640 1 -- 172.21.15.125:0/2493999774 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680004400 con 0x7f068c15b4b0 2023-10-18T10:45:00.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.962+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c14ffc0 con 0x7f068c15b4b0 2023-10-18T10:45:00.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.963+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c06c850 con 0x7f068c15b4b0 2023-10-18T10:45:00.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.964+0000 7f06737fe640 1 -- 172.21.15.125:0/2493999774 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f06800035f0 con 0x7f068c15b4b0 2023-10-18T10:45:00.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.964+0000 7f06737fe640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f067806af90 0x7f067806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:00.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.964+0000 7f06737fe640 1 -- 172.21.15.125:0/2493999774 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f06800ab8f0 con 0x7f068c15b4b0 2023-10-18T10:45:00.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.964+0000 7f068affd640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f067806af90 0x7f067806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:00.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.965+0000 7f068affd640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f067806af90 0x7f067806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f067c004620 tx=0x7f067c00b260 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:00.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:00.968+0000 7f06737fe640 1 -- 172.21.15.125:0/2493999774 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06800767e0 con 0x7f068c15b4b0 2023-10-18T10:45:01.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.135+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f068c06ae00 con 0x7f068c15b4b0 2023-10-18T10:45:01.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.136+0000 7f06737fe640 1 -- 172.21.15.125:0/2493999774 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f0680048090 con 0x7f068c15b4b0 2023-10-18T10:45:01.138 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:01.139 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":155,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:44:59.877181+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:58.087653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:58.847874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:01.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f067806af90 msgr2=0x7f067806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f067806af90 0x7f067806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f067c004620 tx=0x7f067c00b260 comp rx=0 tx=0).stop 2023-10-18T10:45:01.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f068c15b4b0 msgr2=0x7f068c15b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f068c15b4b0 0x7f068c15b930 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f0680038040 tx=0x7f0680036720 comp rx=0 tx=0).stop 2023-10-18T10:45:01.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 shutdown_connections 2023-10-18T10:45:01.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f067806af90 0x7f067806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068c12cbb0 0x7f068c12d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f068c15b4b0 0x7f068c15b930 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 --2- 172.21.15.125:0/2493999774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06740047c0 0x7f068c14fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 >> 172.21.15.125:0/2493999774 conn(0x7f068c069930 msgr2=0x7f068c15a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:01.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 shutdown_connections 2023-10-18T10:45:01.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.138+0000 7f06912e5640 1 -- 172.21.15.125:0/2493999774 wait complete. 2023-10-18T10:45:01.151 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T10:45:01.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.254+0000 7f57e9e1a640 1 -- 172.21.15.125:0/3856848718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57cc003b30 msgr2=0x7f57cc003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.254+0000 7f57e9e1a640 1 --2- 172.21.15.125:0/3856848718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57cc003b30 0x7f57cc003f30 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f57d4001800 tx=0x7f57cc0057f0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.254+0000 7f57e3fff640 1 -- 172.21.15.125:0/3856848718 reap_dead start 2023-10-18T10:45:01.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.254+0000 7f57e9e1a640 1 -- 172.21.15.125:0/3856848718 shutdown_connections 2023-10-18T10:45:01.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.254+0000 7f57e9e1a640 1 -- 172.21.15.125:0/3856848718 >> 172.21.15.125:0/3856848718 conn(0x7f57e4069930 msgr2=0x7f57e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:01.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.254+0000 7f57e9e1a640 1 -- 172.21.15.125:0/3856848718 shutdown_connections 2023-10-18T10:45:01.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.255+0000 7f57e9e1a640 1 -- 172.21.15.125:0/3856848718 wait complete. 2023-10-18T10:45:01.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.256+0000 7f57e9e1a640 1 Processor -- start 2023-10-18T10:45:01.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.256+0000 7f57e9e1a640 1 -- start start 2023-10-18T10:45:01.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.256+0000 7f57e9e1a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57cc003b30 0x7f57e4147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:01.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.256+0000 7f57e9e1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57e414fed0 0x7f57e4150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:01.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.256+0000 7f57e9e1a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57e414a050 0x7f57e414a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:01.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e9e1a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e4150890 con 0x7f57e414fed0 2023-10-18T10:45:01.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e9e1a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e414ada0 con 0x7f57cc003b30 2023-10-18T10:45:01.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e9e1a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e414b080 con 0x7f57e414a050 2023-10-18T10:45:01.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57e414fed0 0x7f57e4150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:01.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57e414fed0 0x7f57e4150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36722/0 (socket says 172.21.15.125:36722) 2023-10-18T10:45:01.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 -- 172.21.15.125:0/1906559652 learned_addr learned my addr 172.21.15.125:0/1906559652 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:01.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e37fe640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57cc003b30 0x7f57e4147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:01.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e3fff640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57e414a050 0x7f57e414a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:01.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 -- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57e414a050 msgr2=0x7f57e414a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57e414a050 0x7f57e414a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 -- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57cc003b30 msgr2=0x7f57e4147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57cc003b30 0x7f57e4147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 -- 172.21.15.125:0/1906559652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e414b360 con 0x7f57e414fed0 2023-10-18T10:45:01.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e37fe640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57cc003b30 0x7f57e4147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:01.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e3fff640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57e414a050 0x7f57e414a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:01.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e2ffd640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57e414fed0 0x7f57e4150350 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f57d8011a80 tx=0x7f57d8011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:01.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57cbfff640 1 -- 172.21.15.125:0/1906559652 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8010e80 con 0x7f57e414fed0 2023-10-18T10:45:01.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57cbfff640 1 -- 172.21.15.125:0/1906559652 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57d8004050 con 0x7f57e414fed0 2023-10-18T10:45:01.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e414b640 con 0x7f57e414fed0 2023-10-18T10:45:01.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57cbfff640 1 -- 172.21.15.125:0/1906559652 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8004440 con 0x7f57e414fed0 2023-10-18T10:45:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.257+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e414aa10 con 0x7f57e414fed0 2023-10-18T10:45:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.258+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e40690e0 con 0x7f57e414fed0 2023-10-18T10:45:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.259+0000 7f57cbfff640 1 -- 172.21.15.125:0/1906559652 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f57d8018070 con 0x7f57e414fed0 2023-10-18T10:45:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.259+0000 7f57cbfff640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f57d006af90 0x7f57d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:01.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.259+0000 7f57cbfff640 1 -- 172.21.15.125:0/1906559652 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f57d808ad00 con 0x7f57e414fed0 2023-10-18T10:45:01.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.259+0000 7f57e37fe640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f57d006af90 0x7f57d006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:01.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.260+0000 7f57e37fe640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f57d006af90 0x7f57d006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f57d4034000 tx=0x7f57d402e070 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:01.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.263+0000 7f57cbfff640 1 -- 172.21.15.125:0/1906559652 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57d8055c00 con 0x7f57e414fed0 2023-10-18T10:45:01.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.464+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f57e406f700 con 0x7f57e414fed0 2023-10-18T10:45:01.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.543+0000 7f57cbfff640 1 -- 172.21.15.125:0/1906559652 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) v1 ==== 118+0+0 (secure 0 0 0) 0x7f57d8059220 con 0x7f57e414fed0 2023-10-18T10:45:01.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f57d006af90 msgr2=0x7f57d006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f57d006af90 0x7f57d006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f57d4034000 tx=0x7f57d402e070 comp rx=0 tx=0).stop 2023-10-18T10:45:01.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57e414fed0 msgr2=0x7f57e4150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57e414fed0 0x7f57e4150350 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f57d8011a80 tx=0x7f57d8011f40 comp rx=0 tx=0).stop 2023-10-18T10:45:01.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 shutdown_connections 2023-10-18T10:45:01.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f57d006af90 0x7f57d006d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57e414a050 0x7f57e414a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57e414fed0 0x7f57e4150350 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 --2- 172.21.15.125:0/1906559652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57cc003b30 0x7f57e4147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 >> 172.21.15.125:0/1906559652 conn(0x7f57e4069930 msgr2=0x7f57e4070510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:01.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 shutdown_connections 2023-10-18T10:45:01.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.546+0000 7f57e9e1a640 1 -- 172.21.15.125:0/1906559652 wait complete. 2023-10-18T10:45:01.559 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T10:45:01.559 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T10:45:01.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.664+0000 7f2199e9b640 1 -- 172.21.15.125:0/2079461228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21800039a0 msgr2=0x7f2180003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.664+0000 7f2199e9b640 1 --2- 172.21.15.125:0/2079461228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21800039a0 0x7f2180003dc0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f218400bb20 tx=0x7f21800057f0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.664+0000 7f2193fff640 1 -- 172.21.15.125:0/2079461228 reap_dead start 2023-10-18T10:45:01.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.664+0000 7f2199e9b640 1 -- 172.21.15.125:0/2079461228 shutdown_connections 2023-10-18T10:45:01.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.664+0000 7f2199e9b640 1 -- 172.21.15.125:0/2079461228 >> 172.21.15.125:0/2079461228 conn(0x7f21940d0d60 msgr2=0x7f21940d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:01.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.664+0000 7f2199e9b640 1 -- 172.21.15.125:0/2079461228 shutdown_connections 2023-10-18T10:45:01.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.664+0000 7f2199e9b640 1 -- 172.21.15.125:0/2079461228 wait complete. 2023-10-18T10:45:01.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.665+0000 7f2199e9b640 1 Processor -- start 2023-10-18T10:45:01.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.665+0000 7f2199e9b640 1 -- start start 2023-10-18T10:45:01.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2199e9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21800039a0 0x7f219409aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:01.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2199e9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21940c99d0 0x7f219406ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:01.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2199e9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21940c9e50 0x7f2194157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:01.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2199e9b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21940b4390 con 0x7f21940c9e50 2023-10-18T10:45:01.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2199e9b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219406b700 con 0x7f21940c99d0 2023-10-18T10:45:01.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2199e9b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21940b39f0 con 0x7f21800039a0 2023-10-18T10:45:01.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21940c9e50 0x7f2194157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:01.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21940c9e50 0x7f2194157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36736/0 (socket says 172.21.15.125:36736) 2023-10-18T10:45:01.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 -- 172.21.15.125:0/1377369214 learned_addr learned my addr 172.21.15.125:0/1377369214 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:01.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f218affd640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21940c99d0 0x7f219406ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:01.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f21937fe640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21800039a0 0x7f219409aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:01.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 -- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21800039a0 msgr2=0x7f219409aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21800039a0 0x7f219409aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 -- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21940c99d0 msgr2=0x7f219406ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:01.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21940c99d0 0x7f219406ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:01.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 -- 172.21.15.125:0/1377369214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940b3cd0 con 0x7f21940c9e50 2023-10-18T10:45:01.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.666+0000 7f2193fff640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21940c9e50 0x7f2194157f70 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f218c000f80 tx=0x7f218c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:01.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.667+0000 7f2190ff9640 1 -- 172.21.15.125:0/1377369214 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c011870 con 0x7f21940c9e50 2023-10-18T10:45:01.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.667+0000 7f2190ff9640 1 -- 172.21.15.125:0/1377369214 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f218c011ae0 con 0x7f21940c9e50 2023-10-18T10:45:01.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.667+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21940b3fb0 con 0x7f21940c9e50 2023-10-18T10:45:01.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.667+0000 7f2190ff9640 1 -- 172.21.15.125:0/1377369214 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c0103c0 con 0x7f21940c9e50 2023-10-18T10:45:01.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.667+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219406b1a0 con 0x7f21940c9e50 2023-10-18T10:45:01.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.668+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21940b44d0 con 0x7f21940c9e50 2023-10-18T10:45:01.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.668+0000 7f2190ff9640 1 -- 172.21.15.125:0/1377369214 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f218c010520 con 0x7f21940c9e50 2023-10-18T10:45:01.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.668+0000 7f2190ff9640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f217406af60 0x7f217406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:01.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.668+0000 7f2190ff9640 1 -- 172.21.15.125:0/1377369214 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f218c08a3a0 con 0x7f21940c9e50 2023-10-18T10:45:01.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.669+0000 7f21937fe640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f217406af60 0x7f217406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:01.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.669+0000 7f21937fe640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f217406af60 0x7f217406d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2184037000 tx=0x7f218402e040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:01.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.673+0000 7f2190ff9640 1 -- 172.21.15.125:0/1377369214 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21940b44d0 con 0x7f21940c9e50 2023-10-18T10:45:01.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:01.874+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f21940b44d0 con 0x7f21940c9e50 2023-10-18T10:45:02.514 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T10:45:02.512+0000 7fe82811d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T10:45:02.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.549+0000 7f2190ff9640 1 -- 172.21.15.125:0/1377369214 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) v1 ==== 122+0+0 (secure 0 0 0) 0x7f21940b44d0 con 0x7f21940c9e50 2023-10-18T10:45:02.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f217406af60 msgr2=0x7f217406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:02.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f217406af60 0x7f217406d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2184037000 tx=0x7f218402e040 comp rx=0 tx=0).stop 2023-10-18T10:45:02.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21940c9e50 msgr2=0x7f2194157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:02.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21940c9e50 0x7f2194157f70 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f218c000f80 tx=0x7f218c0019b0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 shutdown_connections 2023-10-18T10:45:02.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f217406af60 0x7f217406d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21940c99d0 0x7f219406ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21940c9e50 0x7f2194157f70 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 --2- 172.21.15.125:0/1377369214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21800039a0 0x7f219409aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.551+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 >> 172.21.15.125:0/1377369214 conn(0x7f21940d0d60 msgr2=0x7f219415a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:02.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.552+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 shutdown_connections 2023-10-18T10:45:02.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.552+0000 7f2199e9b640 1 -- 172.21.15.125:0/1377369214 wait complete. 2023-10-18T10:45:02.565 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T10:45:02.566 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:45:02.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.670+0000 7f7065875640 1 -- 172.21.15.125:0/2223220934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 msgr2=0x7f70600cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:02.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.670+0000 7f7065875640 1 --2- 172.21.15.125:0/2223220934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 0x7f70600cf720 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f7060089ae0 tx=0x7f704802bd10 comp rx=0 tx=0).stop 2023-10-18T10:45:02.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.670+0000 7f7065875640 1 -- 172.21.15.125:0/2223220934 shutdown_connections 2023-10-18T10:45:02.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.670+0000 7f7065875640 1 --2- 172.21.15.125:0/2223220934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 0x7f70600cf720 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.670+0000 7f7065875640 1 --2- 172.21.15.125:0/2223220934 >> v2:172.21.15.139:3300/0 conn(0x7f70600ceb40 0x7f70600b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.671+0000 7f7065875640 1 --2- 172.21.15.125:0/2223220934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70600cf340 0x7f70600d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.671+0000 7f7065875640 1 -- 172.21.15.125:0/2223220934 >> 172.21.15.125:0/2223220934 conn(0x7f70600d0d60 msgr2=0x7f70600d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:02.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.671+0000 7f7065875640 1 -- 172.21.15.125:0/2223220934 shutdown_connections 2023-10-18T10:45:02.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.671+0000 7f7065875640 1 -- 172.21.15.125:0/2223220934 wait complete. 2023-10-18T10:45:02.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f7065875640 1 Processor -- start 2023-10-18T10:45:02.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f7065875640 1 -- start start 2023-10-18T10:45:02.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f7065875640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70600ceb40 0x7f70600b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:02.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f7065875640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70600cf340 0x7f7060150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:02.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f7065875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 0x7f706015cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:02.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f7065875640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7060150970 con 0x7f706015b2a0 2023-10-18T10:45:02.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f7065875640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706015d040 con 0x7f70600cf340 2023-10-18T10:45:02.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f7065875640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70600b5ac0 con 0x7f70600ceb40 2023-10-18T10:45:02.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.672+0000 7f705f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 0x7f706015cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:02.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 0x7f706015cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36754/0 (socket says 172.21.15.125:36754) 2023-10-18T10:45:02.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705f7fe640 1 -- 172.21.15.125:0/2829897460 learned_addr learned my addr 172.21.15.125:0/2829897460 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:02.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f7057fff640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70600cf340 0x7f7060150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:02.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705effd640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70600ceb40 0x7f70600b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:02.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705f7fe640 1 -- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70600ceb40 msgr2=0x7f70600b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:02.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705f7fe640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70600ceb40 0x7f70600b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705f7fe640 1 -- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70600cf340 msgr2=0x7f7060150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:02.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705f7fe640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70600cf340 0x7f7060150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705f7fe640 1 -- 172.21.15.125:0/2829897460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7048009560 con 0x7f706015b2a0 2023-10-18T10:45:02.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f7057fff640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70600cf340 0x7f7060150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:02.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705f7fe640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 0x7f706015cb00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f705800b970 tx=0x7f705800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:02.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705cff9640 1 -- 172.21.15.125:0/2829897460 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705800cb40 con 0x7f706015b2a0 2023-10-18T10:45:02.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705cff9640 1 -- 172.21.15.125:0/2829897460 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f705800cdb0 con 0x7f706015b2a0 2023-10-18T10:45:02.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70600b4a20 con 0x7f706015b2a0 2023-10-18T10:45:02.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f705cff9640 1 -- 172.21.15.125:0/2829897460 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70580146a0 con 0x7f706015b2a0 2023-10-18T10:45:02.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.673+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70600b4fe0 con 0x7f706015b2a0 2023-10-18T10:45:02.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.674+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70600690e0 con 0x7f706015b2a0 2023-10-18T10:45:02.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.675+0000 7f705cff9640 1 -- 172.21.15.125:0/2829897460 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f705801d3b0 con 0x7f706015b2a0 2023-10-18T10:45:02.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.675+0000 7f705cff9640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f704006af90 0x7f704006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:02.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.675+0000 7f705cff9640 1 -- 172.21.15.125:0/2829897460 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f7058089a90 con 0x7f706015b2a0 2023-10-18T10:45:02.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.675+0000 7f705effd640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f704006af90 0x7f704006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:02.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.676+0000 7f705effd640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f704006af90 0x7f704006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f705000a810 tx=0x7f7050008040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:02.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.679+0000 7f705cff9640 1 -- 172.21.15.125:0/2829897460 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70580548d0 con 0x7f706015b2a0 2023-10-18T10:45:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.879+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70600d0b40 con 0x7f706015b2a0 2023-10-18T10:45:02.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.880+0000 7f705cff9640 1 -- 172.21.15.125:0/2829897460 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f7058019020 con 0x7f706015b2a0 2023-10-18T10:45:02.882 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:02.882 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2692191151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2692191151},{"type":"v1","addr":"172.21.15.139:6836","nonce":2692191151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1995790917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1995790917},{"type":"v1","addr":"172.21.15.139:6837","nonce":1995790917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/55232611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":55232611},{"type":"v1","addr":"172.21.15.125:6834","nonce":55232611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:59.877487+0000","modified":"2023-10-18T10:45:01.875939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9765":{"gid":9765,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/631315131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":631315131},{"type":"v1","addr":"172.21.15.125:6835","nonce":631315131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T10:45:02.883 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 60 2023-10-18T10:45:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f704006af90 msgr2=0x7f704006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f704006af90 0x7f704006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f705000a810 tx=0x7f7050008040 comp rx=0 tx=0).stop 2023-10-18T10:45:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 msgr2=0x7f706015cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 0x7f706015cb00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f705800b970 tx=0x7f705800be30 comp rx=0 tx=0).stop 2023-10-18T10:45:02.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 shutdown_connections 2023-10-18T10:45:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f704006af90 0x7f704006d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70600cf340 0x7f7060150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f706015b2a0 0x7f706015cb00 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 --2- 172.21.15.125:0/2829897460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70600ceb40 0x7f70600b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.881+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 >> 172.21.15.125:0/2829897460 conn(0x7f70600d0d60 msgr2=0x7f70600b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:02.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.882+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 shutdown_connections 2023-10-18T10:45:02.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.882+0000 7f7065875640 1 -- 172.21.15.125:0/2829897460 wait complete. 2023-10-18T10:45:02.894 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T10:45:02.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.995+0000 7fc36d555640 1 -- 172.21.15.125:0/2832544372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 msgr2=0x7fc350004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:02.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.995+0000 7fc36d555640 1 --2- 172.21.15.125:0/2832544372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 0x7fc350004c40 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00bb20 tx=0x7fc3500057f0 comp rx=0 tx=0).stop 2023-10-18T10:45:02.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.995+0000 7fc3677fe640 1 -- 172.21.15.125:0/2832544372 reap_dead start 2023-10-18T10:45:02.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.995+0000 7fc36d555640 1 -- 172.21.15.125:0/2832544372 shutdown_connections 2023-10-18T10:45:02.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.995+0000 7fc36d555640 1 -- 172.21.15.125:0/2832544372 >> 172.21.15.125:0/2832544372 conn(0x7fc3680b3f00 msgr2=0x7fc3680b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:02.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.995+0000 7fc36d555640 1 -- 172.21.15.125:0/2832544372 shutdown_connections 2023-10-18T10:45:02.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.995+0000 7fc36d555640 1 -- 172.21.15.125:0/2832544372 wait complete. 2023-10-18T10:45:03.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc36d555640 1 Processor -- start 2023-10-18T10:45:03.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc36d555640 1 -- start start 2023-10-18T10:45:03.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc36d555640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 0x7fc3681f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc36d555640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc368141bc0 0x7fc3681ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc36d555640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368154840 0x7fc368154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc36d555640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368155230 con 0x7fc3500047c0 2023-10-18T10:45:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc36d555640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368098e00 con 0x7fc368154840 2023-10-18T10:45:03.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc36d555640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36813fb30 con 0x7fc368141bc0 2023-10-18T10:45:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc366ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 0x7fc3681f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc366ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 0x7fc3681f2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36768/0 (socket says 172.21.15.125:36768) 2023-10-18T10:45:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.997+0000 7fc366ffd640 1 -- 172.21.15.125:0/1175856392 learned_addr learned my addr 172.21.15.125:0/1175856392 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc3677fe640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368154840 0x7fc368154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc3667fc640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc368141bc0 0x7fc3681ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc366ffd640 1 -- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc368141bc0 msgr2=0x7fc3681ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc366ffd640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc368141bc0 0x7fc3681ee4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc366ffd640 1 -- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368154840 msgr2=0x7fc368154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc366ffd640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368154840 0x7fc368154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc366ffd640 1 -- 172.21.15.125:0/1175856392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3681f3120 con 0x7fc3500047c0 2023-10-18T10:45:03.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc3667fc640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc368141bc0 0x7fc3681ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:03.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc366ffd640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 0x7fc3681f2be0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fc35800e630 tx=0x7fc35800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:03.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc34f7fe640 1 -- 172.21.15.125:0/1175856392 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358004220 con 0x7fc3500047c0 2023-10-18T10:45:03.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc34f7fe640 1 -- 172.21.15.125:0/1175856392 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc358004470 con 0x7fc3500047c0 2023-10-18T10:45:03.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368142130 con 0x7fc3500047c0 2023-10-18T10:45:03.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc34f7fe640 1 -- 172.21.15.125:0/1175856392 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35800ca00 con 0x7fc3500047c0 2023-10-18T10:45:03.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.998+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681426f0 con 0x7fc3500047c0 2023-10-18T10:45:03.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.999+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36813fcf0 con 0x7fc3500047c0 2023-10-18T10:45:03.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:02.999+0000 7fc34f7fe640 1 -- 172.21.15.125:0/1175856392 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc35800cb60 con 0x7fc3500047c0 2023-10-18T10:45:03.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.000+0000 7fc34f7fe640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc35406af90 0x7fc35406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.000+0000 7fc34f7fe640 1 -- 172.21.15.125:0/1175856392 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fc358089ed0 con 0x7fc3500047c0 2023-10-18T10:45:03.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.000+0000 7fc3667fc640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc35406af90 0x7fc35406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.001+0000 7fc3667fc640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc35406af90 0x7fc35406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc35c037000 tx=0x7fc35c02e040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:03.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.004+0000 7fc34f7fe640 1 -- 172.21.15.125:0/1175856392 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc358058750 con 0x7fc3500047c0 2023-10-18T10:45:03.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.173+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3680690e0 con 0x7fc3500047c0 2023-10-18T10:45:03.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.174+0000 7fc34f7fe640 1 -- 172.21.15.125:0/1175856392 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+16824 (secure 0 0 0) 0x7fc358015020 con 0x7fc3500047c0 2023-10-18T10:45:03.176 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:03.176 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":156,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T10:45:01.540062+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:58.087653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:58.847874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T10:45:03.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc35406af90 msgr2=0x7fc35406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc35406af90 0x7fc35406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc35c037000 tx=0x7fc35c02e040 comp rx=0 tx=0).stop 2023-10-18T10:45:03.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 msgr2=0x7fc3681f2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 0x7fc3681f2be0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fc35800e630 tx=0x7fc35800eaf0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 shutdown_connections 2023-10-18T10:45:03.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fc35406af90 0x7fc35406d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368154840 0x7fc368154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc368141bc0 0x7fc3681ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 --2- 172.21.15.125:0/1175856392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3500047c0 0x7fc3681f2be0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 >> 172.21.15.125:0/1175856392 conn(0x7fc3680b3f00 msgr2=0x7fc368155920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:03.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 shutdown_connections 2023-10-18T10:45:03.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.176+0000 7fc36d555640 1 -- 172.21.15.125:0/1175856392 wait complete. 2023-10-18T10:45:03.188 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T10:45:03.189 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:45:03.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.289+0000 7fa6ae73c640 1 -- 172.21.15.125:0/1076539053 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa690004d00 msgr2=0x7fa690005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.289+0000 7fa6ae73c640 1 --2- 172.21.15.125:0/1076539053 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa690004d00 0x7fa690005180 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009ed0 tx=0x7fa6900057f0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.289+0000 7fa6accd9640 1 -- 172.21.15.125:0/1076539053 reap_dead start 2023-10-18T10:45:03.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.289+0000 7fa6ae73c640 1 -- 172.21.15.125:0/1076539053 shutdown_connections 2023-10-18T10:45:03.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.289+0000 7fa6ae73c640 1 -- 172.21.15.125:0/1076539053 >> 172.21.15.125:0/1076539053 conn(0x7fa6a8069930 msgr2=0x7fa6a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:03.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.289+0000 7fa6ae73c640 1 -- 172.21.15.125:0/1076539053 shutdown_connections 2023-10-18T10:45:03.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.289+0000 7fa6ae73c640 1 -- 172.21.15.125:0/1076539053 wait complete. 2023-10-18T10:45:03.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.290+0000 7fa6ae73c640 1 Processor -- start 2023-10-18T10:45:03.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.290+0000 7fa6ae73c640 1 -- start start 2023-10-18T10:45:03.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6ae73c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa690004d00 0x7fa6a80ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6ae73c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6a80aa400 0x7fa6a80a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6ae73c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6a8152960 0x7fa6a815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6ae73c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80a9f70 con 0x7fa6a80aa400 2023-10-18T10:45:03.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6ae73c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80a9360 con 0x7fa690004d00 2023-10-18T10:45:03.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6ae73c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80a9640 con 0x7fa6a8152960 2023-10-18T10:45:03.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6a80aa400 0x7fa6a80a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6a80aa400 0x7fa6a80a8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36782/0 (socket says 172.21.15.125:36782) 2023-10-18T10:45:03.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 -- 172.21.15.125:0/471532313 learned_addr learned my addr 172.21.15.125:0/471532313 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:03.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a7fff640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa690004d00 0x7fa6a80ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6accd9640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6a8152960 0x7fa6a815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 -- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6a8152960 msgr2=0x7fa6a815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6a8152960 0x7fa6a815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 -- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa690004d00 msgr2=0x7fa6a80ab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa690004d00 0x7fa6a80ab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 -- 172.21.15.125:0/471532313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80a9920 con 0x7fa6a80aa400 2023-10-18T10:45:03.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a77fe640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6a80aa400 0x7fa6a80a8b10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fa69c006650 tx=0x7fa69c001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a4ff9640 1 -- 172.21.15.125:0/471532313 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c006220 con 0x7fa6a80aa400 2023-10-18T10:45:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a4ff9640 1 -- 172.21.15.125:0/471532313 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa69c011700 con 0x7fa6a80aa400 2023-10-18T10:45:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6a4ff9640 1 -- 172.21.15.125:0/471532313 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c011b10 con 0x7fa6a80aa400 2023-10-18T10:45:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.291+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a80a9c00 con 0x7fa6a80aa400 2023-10-18T10:45:03.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.292+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a80a9050 con 0x7fa6a80aa400 2023-10-18T10:45:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.292+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a80aa0e0 con 0x7fa6a80aa400 2023-10-18T10:45:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.293+0000 7fa6a4ff9640 1 -- 172.21.15.125:0/471532313 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa69c011c70 con 0x7fa6a80aa400 2023-10-18T10:45:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.293+0000 7fa6a4ff9640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa69406af90 0x7fa69406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.293+0000 7fa6a4ff9640 1 -- 172.21.15.125:0/471532313 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa69c08a9e0 con 0x7fa6a80aa400 2023-10-18T10:45:03.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.293+0000 7fa6a7fff640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa69406af90 0x7fa69406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.294+0000 7fa6a7fff640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa69406af90 0x7fa69406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa6a81590f0 tx=0x7fa6980068a0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:03.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.297+0000 7fa6a4ff9640 1 -- 172.21.15.125:0/471532313 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa69c055710 con 0x7fa6a80aa400 2023-10-18T10:45:03.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.459+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 --> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6a815ce80 con 0x7fa69406af90 2023-10-18T10:45:03.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.467+0000 7fa6a4ff9640 1 -- 172.21.15.125:0/471532313 <== mgr.7340 v2:172.21.15.139:6832/3182280683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274380 (secure 0 0 0) 0x7fa698016070 con 0x7fa69406af90 2023-10-18T10:45:03.472 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:03.473 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:45:03.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa69406af90 msgr2=0x7fa69406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa69406af90 0x7fa69406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa6a81590f0 tx=0x7fa6980068a0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6a80aa400 msgr2=0x7fa6a80a8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6a80aa400 0x7fa6a80a8b10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fa69c006650 tx=0x7fa69c001790 comp rx=0 tx=0).stop 2023-10-18T10:45:03.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 shutdown_connections 2023-10-18T10:45:03.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa69406af90 0x7fa69406d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6a8152960 0x7fa6a815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6a80aa400 0x7fa6a80a8b10 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 --2- 172.21.15.125:0/471532313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa690004d00 0x7fa6a80ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.470+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 >> 172.21.15.125:0/471532313 conn(0x7fa6a8069930 msgr2=0x7fa6a8159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:03.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.471+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 shutdown_connections 2023-10-18T10:45:03.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.471+0000 7fa6ae73c640 1 -- 172.21.15.125:0/471532313 wait complete. 2023-10-18T10:45:03.483 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T10:45:02.852884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":344304,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754630416,"statfs":{"total":773094113280,"available":772741545984,"internally_reserved":0,"allocated":16101376,"data_stored":8711893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":336318008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.327554"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"145'258","reported_seq":784,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-18T10:44:51.720345+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:44:51.720345+0000","last_peered":"2023-10-18T10:44:51.720345+0000","last_clean":"2023-10-18T10:44:51.720345+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:44:51.720345+0000","last_undegraded":"2023-10-18T10:44:51.720345+0000","last_fullsized":"2023-10-18T10:44:51.720345+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575089,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59936,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311904,"statfs":{"total":96636764160,"available":96575389696,"internally_reserved":0,"allocated":2441216,"data_stored":1523775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.217,"15min":0.217},"max":{"1min":0.77900000000000003,"5min":0.876,"15min":0.88300000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.90700000000000003},"last":0.61199999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.443}]},{"osd":3,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77900000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60099999999999998},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63400000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.86199999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.55800000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37148,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":0.82499999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.75900000000000001,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.58499999999999996}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.36799999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.749,"5min":0.78900000000000003,"15min":0.82899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.70099999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.60699999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.442,"5min":0.372,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.498}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.65400000000000003,"5min":0.78400000000000003,"15min":0.872},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76100000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71499999999999997,"5min":0.82499999999999996,"15min":0.83699999999999997},"last":0.65500000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.313},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.85399999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84699999999999998,"15min":0.87},"last":0.623}]}]},{"osd":5,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51088,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320752,"statfs":{"total":96636764160,"available":96584450048,"internally_reserved":0,"allocated":1896448,"data_stored":956509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.39600000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.58599999999999997},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.58299999999999996},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93600000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76500000000000001,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.80100000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.52300000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.83299999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.83099999999999996,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.62}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.31},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.89300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.871},"last":0.79900000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37204,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":1896448,"data_stored":956509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.877},"last":0.55000000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.89300000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.33100000000000002},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.5,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.82899999999999996,"5min":0.873,"15min":0.873},"last":0.61899999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.66200000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.504,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":2.8519999999999999,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.60399999999999998,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.69399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32988,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.36499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.62},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.59299999999999997},"last":0.28100000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20799999999999999},"max":{"1min":0.504,"5min":0.624,"15min":0.69099999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.59299999999999997,"15min":0.67000000000000004},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.372},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.98299999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.79300000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86599999999999999,"15min":1.002},"last":0.754},{"interface":"front","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.88100000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38,"15min":0.37},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.018},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.879,"15min":0.96799999999999997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":0.98899999999999999},"last":0.58899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55228,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1884160,"data_stored":951104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.17599999999999999},"max":{"1min":0.54500000000000004,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.38700000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.192},"max":{"1min":0.505,"5min":0.55500000000000005,"15min":0.626},"last":0.27500000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.55500000000000005,"5min":0.57899999999999996,"15min":0.67000000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55700000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.378,"15min":0.39300000000000002},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.32700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.64000000000000001},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.627,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.84799999999999998,"5min":0.87,"15min":0.93100000000000005},"last":0.5}]},{"osd":6,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.89100000000000001,"15min":0.93600000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.40100000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.878,"15min":0.91000000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575089,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32988,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.374,"15min":0.38600000000000001},"min":{"1min":0.221,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51400000000000001,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36699999999999999,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.68899999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52000000000000002,"5min":0.627,"15min":0.64800000000000002},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.42899999999999999},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":0.70199999999999996,"15min":0.72899999999999998},"last":0.497},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.443}]},{"osd":4,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":0.82999999999999996,"5min":1.099,"15min":1.099},"last":0.875},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":0.88100000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.63300000000000001},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.373,"15min":1.373},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":0.83099999999999996,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.45600000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.86099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575088,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37724,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334116,"statfs":{"total":96636764160,"available":96598134784,"internally_reserved":0,"allocated":2441216,"data_stored":1523775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.37,"15min":0.379},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.56599999999999995,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.39000000000000001},"min":{"1min":0.26300000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.57999999999999996,"5min":0.62,"15min":0.76900000000000002},"last":0.26000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.53700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.22,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.309}]},{"osd":3,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.30299999999999999,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.69699999999999995},"last":0.371},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.57599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.28999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.996,"15min":0.996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.79200000000000004,"5min":1.038,"15min":1.038},"last":0.61199999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.44500000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90400000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63500000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.872,"5min":1.026,"15min":1.3700000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.339},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.48799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":1.01,"15min":1.01},"last":0.65600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:45:03.530 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:45:03.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.631+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/840211624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e80c9dc0 msgr2=0x7ff5e806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.631+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/840211624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e80c9dc0 0x7ff5e806b2e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806d590 tx=0x7ff5dc02be40 comp rx=0 tx=0).stop 2023-10-18T10:45:03.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.631+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/840211624 shutdown_connections 2023-10-18T10:45:03.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.631+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/840211624 >> v2:172.21.15.139:3300/0 conn(0x7ff5e80ab500 0x7ff5e80ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.631+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/840211624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e80c9dc0 0x7ff5e806b2e0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.631+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/840211624 >> 172.21.15.125:0/840211624 conn(0x7ff5e8069930 msgr2=0x7ff5e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:03.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.631+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/840211624 shutdown_connections 2023-10-18T10:45:03.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.631+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/840211624 wait complete. 2023-10-18T10:45:03.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.632+0000 7ff5ecbe9640 1 Processor -- start 2023-10-18T10:45:03.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.632+0000 7ff5ecbe9640 1 -- start start 2023-10-18T10:45:03.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5ecbe9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e806af00 0x7ff5e814fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5ecbe9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e80ab500 0x7ff5e80ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5ecbe9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e80c9dc0 0x7ff5e80adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5ecbe9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8153870 con 0x7ff5e806af00 2023-10-18T10:45:03.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5ecbe9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e81f1190 con 0x7ff5e80c9dc0 2023-10-18T10:45:03.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5ecbe9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e81f1fb0 con 0x7ff5e80ab500 2023-10-18T10:45:03.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e806af00 0x7ff5e814fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e806af00 0x7ff5e814fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36798/0 (socket says 172.21.15.125:36798) 2023-10-18T10:45:03.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 -- 172.21.15.125:0/2236954101 learned_addr learned my addr 172.21.15.125:0/2236954101 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:03.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 -- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e80ab500 msgr2=0x7ff5e80ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e80ab500 0x7ff5e80ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 -- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e80c9dc0 msgr2=0x7ff5e80adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e80c9dc0 0x7ff5e80adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 -- 172.21.15.125:0/2236954101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc009560 con 0x7ff5e806af00 2023-10-18T10:45:03.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.633+0000 7ff5e659c640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e806af00 0x7ff5e814fa50 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c4b0 tx=0x7ff5d800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:03.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.634+0000 7ff5d6ffd640 1 -- 172.21.15.125:0/2236954101 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8005480 con 0x7ff5e806af00 2023-10-18T10:45:03.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.634+0000 7ff5d6ffd640 1 -- 172.21.15.125:0/2236954101 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5d80036e0 con 0x7ff5e806af00 2023-10-18T10:45:03.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.634+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e81f13a0 con 0x7ff5e806af00 2023-10-18T10:45:03.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.634+0000 7ff5d6ffd640 1 -- 172.21.15.125:0/2236954101 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8003b30 con 0x7ff5e806af00 2023-10-18T10:45:03.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.634+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e81f1960 con 0x7ff5e806af00 2023-10-18T10:45:03.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.635+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e80690e0 con 0x7ff5e806af00 2023-10-18T10:45:03.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.635+0000 7ff5d6ffd640 1 -- 172.21.15.125:0/2236954101 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff5d8003c90 con 0x7ff5e806af00 2023-10-18T10:45:03.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.635+0000 7ff5d6ffd640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff5c806af90 0x7ff5c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.636+0000 7ff5d6ffd640 1 -- 172.21.15.125:0/2236954101 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7ff5d8089130 con 0x7ff5e806af00 2023-10-18T10:45:03.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.636+0000 7ff5e5d9b640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff5c806af90 0x7ff5c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.636+0000 7ff5e5d9b640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff5c806af90 0x7ff5c806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc035000 tx=0x7ff5dc0046c0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:03.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.640+0000 7ff5d6ffd640 1 -- 172.21.15.125:0/2236954101 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5d8053ef0 con 0x7ff5e806af00 2023-10-18T10:45:03.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.803+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 --> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5e80abed0 con 0x7ff5c806af90 2023-10-18T10:45:03.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.810+0000 7ff5d6ffd640 1 -- 172.21.15.125:0/2236954101 <== mgr.7340 v2:172.21.15.139:6832/3182280683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274380 (secure 0 0 0) 0x7ff5dc004210 con 0x7ff5c806af90 2023-10-18T10:45:03.814 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:03.816 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:45:03.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff5c806af90 msgr2=0x7ff5c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff5c806af90 0x7ff5c806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc035000 tx=0x7ff5dc0046c0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e806af00 msgr2=0x7ff5e814fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e806af00 0x7ff5e814fa50 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c4b0 tx=0x7ff5d800c970 comp rx=0 tx=0).stop 2023-10-18T10:45:03.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 shutdown_connections 2023-10-18T10:45:03.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7ff5c806af90 0x7ff5c806d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e80c9dc0 0x7ff5e80adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e80ab500 0x7ff5e80ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 --2- 172.21.15.125:0/2236954101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5e806af00 0x7ff5e814fa50 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 >> 172.21.15.125:0/2236954101 conn(0x7ff5e8069930 msgr2=0x7ff5e8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:03.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 shutdown_connections 2023-10-18T10:45:03.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.813+0000 7ff5ecbe9640 1 -- 172.21.15.125:0/2236954101 wait complete. 2023-10-18T10:45:03.825 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T10:45:02.852884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":344304,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754630416,"statfs":{"total":773094113280,"available":772741545984,"internally_reserved":0,"allocated":16101376,"data_stored":8711893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":336318008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.327554"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"145'258","reported_seq":784,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-18T10:44:51.720345+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:44:51.720345+0000","last_peered":"2023-10-18T10:44:51.720345+0000","last_clean":"2023-10-18T10:44:51.720345+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:44:51.720345+0000","last_undegraded":"2023-10-18T10:44:51.720345+0000","last_fullsized":"2023-10-18T10:44:51.720345+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575089,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59936,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311904,"statfs":{"total":96636764160,"available":96575389696,"internally_reserved":0,"allocated":2441216,"data_stored":1523775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.217,"15min":0.217},"max":{"1min":0.77900000000000003,"5min":0.876,"15min":0.88300000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.90700000000000003},"last":0.61199999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.443}]},{"osd":3,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77900000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60099999999999998},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63400000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.86199999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.55800000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37148,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":0.82499999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.75900000000000001,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.58499999999999996}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.36799999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.749,"5min":0.78900000000000003,"15min":0.82899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.70099999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.60699999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.442,"5min":0.372,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.498}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.65400000000000003,"5min":0.78400000000000003,"15min":0.872},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76100000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71499999999999997,"5min":0.82499999999999996,"15min":0.83699999999999997},"last":0.65500000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.313},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.85399999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84699999999999998,"15min":0.87},"last":0.623}]}]},{"osd":5,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51088,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320752,"statfs":{"total":96636764160,"available":96584450048,"internally_reserved":0,"allocated":1896448,"data_stored":956509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.39600000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.58599999999999997},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.58299999999999996},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93600000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76500000000000001,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.80100000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.52300000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.83299999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.83099999999999996,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.62}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.31},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.89300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.871},"last":0.79900000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37204,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":1896448,"data_stored":956509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.877},"last":0.55000000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.89300000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.33100000000000002},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.5,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.82899999999999996,"5min":0.873,"15min":0.873},"last":0.61899999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.66200000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.504,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":2.8519999999999999,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.60399999999999998,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.69399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32988,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.36499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.62},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.59299999999999997},"last":0.28100000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20799999999999999},"max":{"1min":0.504,"5min":0.624,"15min":0.69099999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.59299999999999997,"15min":0.67000000000000004},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.372},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.98299999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.79300000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86599999999999999,"15min":1.002},"last":0.754},{"interface":"front","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.88100000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38,"15min":0.37},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.018},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.879,"15min":0.96799999999999997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":0.98899999999999999},"last":0.58899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55228,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1884160,"data_stored":951104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.17599999999999999},"max":{"1min":0.54500000000000004,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.38700000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.192},"max":{"1min":0.505,"5min":0.55500000000000005,"15min":0.626},"last":0.27500000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.55500000000000005,"5min":0.57899999999999996,"15min":0.67000000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55700000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.378,"15min":0.39300000000000002},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.32700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.64000000000000001},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.627,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.84799999999999998,"5min":0.87,"15min":0.93100000000000005},"last":0.5}]},{"osd":6,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.89100000000000001,"15min":0.93600000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.40100000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.878,"15min":0.91000000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575089,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32988,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.374,"15min":0.38600000000000001},"min":{"1min":0.221,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51400000000000001,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36699999999999999,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.68899999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52000000000000002,"5min":0.627,"15min":0.64800000000000002},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.42899999999999999},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":0.70199999999999996,"15min":0.72899999999999998},"last":0.497},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.443}]},{"osd":4,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":0.82999999999999996,"5min":1.099,"15min":1.099},"last":0.875},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":0.88100000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.63300000000000001},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.373,"15min":1.373},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":0.83099999999999996,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.45600000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.86099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575088,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37724,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334116,"statfs":{"total":96636764160,"available":96598134784,"internally_reserved":0,"allocated":2441216,"data_stored":1523775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.37,"15min":0.379},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.56599999999999995,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.39000000000000001},"min":{"1min":0.26300000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.57999999999999996,"5min":0.62,"15min":0.76900000000000002},"last":0.26000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.53700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.22,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.309}]},{"osd":3,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.30299999999999999,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.69699999999999995},"last":0.371},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.57599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.28999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.996,"15min":0.996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.79200000000000004,"5min":1.038,"15min":1.038},"last":0.61199999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.44500000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90400000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63500000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.872,"5min":1.026,"15min":1.3700000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.339},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.48799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":1.01,"15min":1.01},"last":0.65600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:45:03.831 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:45:03.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.969+0000 7fa74e3f0640 1 -- 172.21.15.125:0/1306555161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74806af00 msgr2=0x7fa74806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.969+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/1306555161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74806af00 0x7fa74806b2e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fa738005fd0 tx=0x7fa73802bd10 comp rx=0 tx=0).stop 2023-10-18T10:45:03.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.969+0000 7fa74e3f0640 1 -- 172.21.15.125:0/1306555161 shutdown_connections 2023-10-18T10:45:03.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.969+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/1306555161 >> v2:172.21.15.139:3300/0 conn(0x7fa7480c96b0 0x7fa7480c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.969+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/1306555161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa74806af00 0x7fa74806b2e0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.969+0000 7fa74e3f0640 1 -- 172.21.15.125:0/1306555161 >> 172.21.15.125:0/1306555161 conn(0x7fa748069930 msgr2=0x7fa748069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:03.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.969+0000 7fa74e3f0640 1 -- 172.21.15.125:0/1306555161 shutdown_connections 2023-10-18T10:45:03.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.969+0000 7fa74e3f0640 1 -- 172.21.15.125:0/1306555161 wait complete. 2023-10-18T10:45:03.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.970+0000 7fa74e3f0640 1 Processor -- start 2023-10-18T10:45:03.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.970+0000 7fa74e3f0640 1 -- start start 2023-10-18T10:45:03.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa74e3f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa74806af00 0x7fa7480b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa74e3f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7480ab500 0x7fa7480ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa74e3f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7480c96b0 0x7fa7480adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa74e3f0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74813aae0 con 0x7fa7480ab500 2023-10-18T10:45:03.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa74e3f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480b0bd0 con 0x7fa7480c96b0 2023-10-18T10:45:03.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa74e3f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7481f1ad0 con 0x7fa74806af00 2023-10-18T10:45:03.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa7477fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7480ab500 0x7fa7480ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa7477fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7480ab500 0x7fa7480ad6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36812/0 (socket says 172.21.15.125:36812) 2023-10-18T10:45:03.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.971+0000 7fa7477fe640 1 -- 172.21.15.125:0/2370287780 learned_addr learned my addr 172.21.15.125:0/2370287780 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:03.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa7477fe640 1 -- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa74806af00 msgr2=0x7fa7480b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa7477fe640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa74806af00 0x7fa7480b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa7477fe640 1 -- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7480c96b0 msgr2=0x7fa7480adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:03.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa7477fe640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7480c96b0 0x7fa7480adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:03.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa7477fe640 1 -- 172.21.15.125:0/2370287780 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738009560 con 0x7fa7480ab500 2023-10-18T10:45:03.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa7477fe640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7480ab500 0x7fa7480ad6a0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b410 tx=0x7fa73c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:03.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa744ff9640 1 -- 172.21.15.125:0/2370287780 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c011830 con 0x7fa7480ab500 2023-10-18T10:45:03.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa744ff9640 1 -- 172.21.15.125:0/2370287780 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa73c011aa0 con 0x7fa7480ab500 2023-10-18T10:45:03.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7481f1db0 con 0x7fa7480ab500 2023-10-18T10:45:03.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa744ff9640 1 -- 172.21.15.125:0/2370287780 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c010380 con 0x7fa7480ab500 2023-10-18T10:45:03.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.972+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7481f2370 con 0x7fa7480ab500 2023-10-18T10:45:03.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.973+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7480690e0 con 0x7fa7480ab500 2023-10-18T10:45:03.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.973+0000 7fa744ff9640 1 -- 172.21.15.125:0/2370287780 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa73c0104e0 con 0x7fa7480ab500 2023-10-18T10:45:03.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.974+0000 7fa744ff9640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa73406af60 0x7fa73406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:03.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.974+0000 7fa744ff9640 1 -- 172.21.15.125:0/2370287780 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa73c089ca0 con 0x7fa7480ab500 2023-10-18T10:45:03.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.974+0000 7fa747fff640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa73406af60 0x7fa73406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:03.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.974+0000 7fa747fff640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa73406af60 0x7fa73406d420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa738035000 tx=0x7fa7380046c0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:03.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:03.978+0000 7fa744ff9640 1 -- 172.21.15.125:0/2370287780 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa73c054ae0 con 0x7fa7480ab500 2023-10-18T10:45:04.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.141+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 --> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa748159c50 con 0x7fa73406af60 2023-10-18T10:45:04.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.148+0000 7fa744ff9640 1 -- 172.21.15.125:0/2370287780 <== mgr.7340 v2:172.21.15.139:6832/3182280683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274380 (secure 0 0 0) 0x7fa7380041c0 con 0x7fa73406af60 2023-10-18T10:45:04.153 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:04.155 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:45:04.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa73406af60 msgr2=0x7fa73406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa73406af60 0x7fa73406d420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa738035000 tx=0x7fa7380046c0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7480ab500 msgr2=0x7fa7480ad6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7480ab500 0x7fa7480ad6a0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b410 tx=0x7fa73c00b8d0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 shutdown_connections 2023-10-18T10:45:04.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa73406af60 0x7fa73406d420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7480c96b0 0x7fa7480adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7480ab500 0x7fa7480ad6a0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 --2- 172.21.15.125:0/2370287780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa74806af00 0x7fa7480b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 >> 172.21.15.125:0/2370287780 conn(0x7fa748069930 msgr2=0x7fa748069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:04.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.151+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 shutdown_connections 2023-10-18T10:45:04.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.152+0000 7fa74e3f0640 1 -- 172.21.15.125:0/2370287780 wait complete. 2023-10-18T10:45:04.164 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T10:45:02.852884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":344304,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754630416,"statfs":{"total":773094113280,"available":772741545984,"internally_reserved":0,"allocated":16101376,"data_stored":8711893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":336318008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.327554"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:59.524757+0000","last_change":"2023-10-18T10:44:59.524757+0000","last_active":"2023-10-18T10:44:59.524757+0000","last_peered":"2023-10-18T10:44:59.524757+0000","last_clean":"2023-10-18T10:44:59.524757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:59.524757+0000","last_undegraded":"2023-10-18T10:44:59.524757+0000","last_fullsized":"2023-10-18T10:44:59.524757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T10:44:58.520998+0000","last_change":"2023-10-18T10:44:58.520998+0000","last_active":"2023-10-18T10:44:58.520998+0000","last_peered":"2023-10-18T10:44:58.520998+0000","last_clean":"2023-10-18T10:44:58.520998+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T10:44:58.520998+0000","last_undegraded":"2023-10-18T10:44:58.520998+0000","last_fullsized":"2023-10-18T10:44:58.520998+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"145'258","reported_seq":784,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-18T10:44:51.720345+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:44:51.720345+0000","last_peered":"2023-10-18T10:44:51.720345+0000","last_clean":"2023-10-18T10:44:51.720345+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:44:51.720345+0000","last_undegraded":"2023-10-18T10:44:51.720345+0000","last_fullsized":"2023-10-18T10:44:51.720345+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575089,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59936,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311904,"statfs":{"total":96636764160,"available":96575389696,"internally_reserved":0,"allocated":2441216,"data_stored":1523775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.217,"15min":0.217},"max":{"1min":0.77900000000000003,"5min":0.876,"15min":0.88300000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.90700000000000003},"last":0.61199999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":0.443}]},{"osd":3,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77900000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60099999999999998},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63400000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.86199999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.55800000000000005}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37148,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":0.82499999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.75900000000000001,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.58499999999999996}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.36799999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.749,"5min":0.78900000000000003,"15min":0.82899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.70099999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.60699999999999998}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.442,"5min":0.372,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.498}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.65400000000000003,"5min":0.78400000000000003,"15min":0.872},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76100000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71499999999999997,"5min":0.82499999999999996,"15min":0.83699999999999997},"last":0.65500000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.313},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.85399999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84699999999999998,"15min":0.87},"last":0.623}]}]},{"osd":5,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51088,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320752,"statfs":{"total":96636764160,"available":96584450048,"internally_reserved":0,"allocated":1896448,"data_stored":956509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.39600000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.58599999999999997},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.58299999999999996},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93600000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76500000000000001,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.80100000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.52300000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.83299999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.83099999999999996,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.62}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.31},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.89300000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.871},"last":0.79900000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37204,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":1896448,"data_stored":956509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.877},"last":0.55000000000000004}]},{"osd":1,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.89300000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.33100000000000002},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.5,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.82899999999999996,"5min":0.873,"15min":0.873},"last":0.61899999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.66200000000000003}]},{"osd":5,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.504,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":2.8519999999999999,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.60399999999999998,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.69399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32988,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.36499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.62},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.59299999999999997},"last":0.28100000000000003}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20799999999999999},"max":{"1min":0.504,"5min":0.624,"15min":0.69099999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.59299999999999997,"15min":0.67000000000000004},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.372},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.98299999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.79300000000000004}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86599999999999999,"15min":1.002},"last":0.754},{"interface":"front","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.88100000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38,"15min":0.37},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.018},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.879,"15min":0.96799999999999997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":0.98899999999999999},"last":0.58899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575089,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55228,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1884160,"data_stored":951104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.17599999999999999},"max":{"1min":0.54500000000000004,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.38700000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.192},"max":{"1min":0.505,"5min":0.55500000000000005,"15min":0.626},"last":0.27500000000000002}]},{"osd":1,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.55500000000000005,"5min":0.57899999999999996,"15min":0.67000000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55700000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.378,"15min":0.39300000000000002},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.32700000000000001}]},{"osd":4,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.64000000000000001},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.627,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.84799999999999998,"5min":0.87,"15min":0.93100000000000005},"last":0.5}]},{"osd":6,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.89100000000000001,"15min":0.93600000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.40100000000000002}]},{"osd":7,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.878,"15min":0.91000000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575089,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32988,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1847296,"data_stored":933407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.374,"15min":0.38600000000000001},"min":{"1min":0.221,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51400000000000001,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36699999999999999,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.68899999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52000000000000002,"5min":0.627,"15min":0.64800000000000002},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.42899999999999999},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":0.70199999999999996,"15min":0.72899999999999998},"last":0.497},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.443}]},{"osd":4,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":0.82999999999999996,"5min":1.099,"15min":1.099},"last":0.875},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":0.88100000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.63300000000000001},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.373,"15min":1.373},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":0.83099999999999996,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.45600000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.86099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575088,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37724,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334116,"statfs":{"total":96636764160,"available":96598134784,"internally_reserved":0,"allocated":2441216,"data_stored":1523775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.37,"15min":0.379},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.56599999999999995,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.39000000000000001},"min":{"1min":0.26300000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.57999999999999996,"5min":0.62,"15min":0.76900000000000002},"last":0.26000000000000001}]},{"osd":2,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.53700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.22,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.309}]},{"osd":3,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.30299999999999999,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.69699999999999995},"last":0.371},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.57599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.28999999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.996,"15min":0.996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.79200000000000004,"5min":1.038,"15min":1.038},"last":0.61199999999999999}]},{"osd":5,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.44500000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90400000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63500000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.872,"5min":1.026,"15min":1.3700000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.339},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.48799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":1.01,"15min":1.01},"last":0.65600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:45:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3f is not active+clean 2023-10-18T10:45:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3e is not active+clean 2023-10-18T10:45:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3d is not active+clean 2023-10-18T10:45:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3c is not active+clean 2023-10-18T10:45:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3b is not active+clean 2023-10-18T10:45:04.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3a is not active+clean 2023-10-18T10:45:04.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.39 is not active+clean 2023-10-18T10:45:04.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.38 is not active+clean 2023-10-18T10:45:04.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.37 is not active+clean 2023-10-18T10:45:04.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.36 is not active+clean 2023-10-18T10:45:04.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.35 is not active+clean 2023-10-18T10:45:04.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.34 is not active+clean 2023-10-18T10:45:04.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.33 is not active+clean 2023-10-18T10:45:04.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.32 is not active+clean 2023-10-18T10:45:04.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.31 is not active+clean 2023-10-18T10:45:04.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.30 is not active+clean 2023-10-18T10:45:04.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2f is not active+clean 2023-10-18T10:45:04.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2e is not active+clean 2023-10-18T10:45:04.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2d is not active+clean 2023-10-18T10:45:04.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2c is not active+clean 2023-10-18T10:45:04.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2b is not active+clean 2023-10-18T10:45:04.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2a is not active+clean 2023-10-18T10:45:04.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.29 is not active+clean 2023-10-18T10:45:04.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.28 is not active+clean 2023-10-18T10:45:04.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.27 is not active+clean 2023-10-18T10:45:04.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.26 is not active+clean 2023-10-18T10:45:04.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.25 is not active+clean 2023-10-18T10:45:04.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.24 is not active+clean 2023-10-18T10:45:04.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.23 is not active+clean 2023-10-18T10:45:04.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.22 is not active+clean 2023-10-18T10:45:04.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.21 is not active+clean 2023-10-18T10:45:04.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.20 is not active+clean 2023-10-18T10:45:04.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.18 is not active+clean 2023-10-18T10:45:04.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2023-10-18T10:45:04.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.19 is not active+clean 2023-10-18T10:45:04.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2023-10-18T10:45:04.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1a is not active+clean 2023-10-18T10:45:04.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2023-10-18T10:45:04.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1b is not active+clean 2023-10-18T10:45:04.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2023-10-18T10:45:04.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1c is not active+clean 2023-10-18T10:45:04.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2023-10-18T10:45:04.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1d is not active+clean 2023-10-18T10:45:04.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2023-10-18T10:45:04.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1e is not active+clean 2023-10-18T10:45:04.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.19 is not active+clean 2023-10-18T10:45:04.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1f is not active+clean 2023-10-18T10:45:04.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2023-10-18T10:45:04.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.10 is not active+clean 2023-10-18T10:45:04.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.17 is not active+clean 2023-10-18T10:45:04.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.11 is not active+clean 2023-10-18T10:45:04.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.16 is not active+clean 2023-10-18T10:45:04.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.12 is not active+clean 2023-10-18T10:45:04.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2023-10-18T10:45:04.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.13 is not active+clean 2023-10-18T10:45:04.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2023-10-18T10:45:04.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.14 is not active+clean 2023-10-18T10:45:04.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.13 is not active+clean 2023-10-18T10:45:04.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.16 is not active+clean 2023-10-18T10:45:04.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2023-10-18T10:45:04.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.17 is not active+clean 2023-10-18T10:45:04.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.10 is not active+clean 2023-10-18T10:45:04.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.8 is not active+clean 2023-10-18T10:45:04.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.f is not active+clean 2023-10-18T10:45:04.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.9 is not active+clean 2023-10-18T10:45:04.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2023-10-18T10:45:04.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.a is not active+clean 2023-10-18T10:45:04.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.d is not active+clean 2023-10-18T10:45:04.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.b is not active+clean 2023-10-18T10:45:04.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2023-10-18T10:45:04.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.c is not active+clean 2023-10-18T10:45:04.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2023-10-18T10:45:04.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.d is not active+clean 2023-10-18T10:45:04.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.a is not active+clean 2023-10-18T10:45:04.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.e is not active+clean 2023-10-18T10:45:04.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2023-10-18T10:45:04.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.f is not active+clean 2023-10-18T10:45:04.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.8 is not active+clean 2023-10-18T10:45:04.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.0 is not active+clean 2023-10-18T10:45:04.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.7 is not active+clean 2023-10-18T10:45:04.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1 is not active+clean 2023-10-18T10:45:04.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2023-10-18T10:45:04.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2 is not active+clean 2023-10-18T10:45:04.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2023-10-18T10:45:04.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3 is not active+clean 2023-10-18T10:45:04.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.4 is not active+clean 2023-10-18T10:45:04.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.4 is not active+clean 2023-10-18T10:45:04.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3 is not active+clean 2023-10-18T10:45:04.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.5 is not active+clean 2023-10-18T10:45:04.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2023-10-18T10:45:04.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.6 is not active+clean 2023-10-18T10:45:04.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1 is not active+clean 2023-10-18T10:45:04.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.7 is not active+clean 2023-10-18T10:45:04.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2023-10-18T10:45:04.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.15 is not active+clean 2023-10-18T10:45:04.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:59.524757+0000', 'last_change': '2023-10-18T10:44:59.524757+0000', 'last_active': '2023-10-18T10:44:59.524757+0000', 'last_peered': '2023-10-18T10:44:59.524757+0000', 'last_clean': '2023-10-18T10:44:59.524757+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:59.524757+0000', 'last_undegraded': '2023-10-18T10:44:59.524757+0000', 'last_fullsized': '2023-10-18T10:44:59.524757+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:59.524757+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2023-10-18T10:45:04.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T10:44:58.520998+0000', 'last_change': '2023-10-18T10:44:58.520998+0000', 'last_active': '2023-10-18T10:44:58.520998+0000', 'last_peered': '2023-10-18T10:44:58.520998+0000', 'last_clean': '2023-10-18T10:44:58.520998+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T10:44:58.520998+0000', 'last_undegraded': '2023-10-18T10:44:58.520998+0000', 'last_fullsized': '2023-10-18T10:44:58.520998+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'last_clean_scrub_stamp': '2023-10-18T10:44:58.520998+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T10:45:04.253 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T10:45:04.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.353+0000 7f260ba39640 1 -- 172.21.15.125:0/1403033598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f260415b2a0 msgr2=0x7f26040d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.353+0000 7f260ba39640 1 --2- 172.21.15.125:0/1403033598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f260415b2a0 0x7f26040d1910 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f2600009830 tx=0x7f260002bcd0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.353+0000 7f260ba39640 1 -- 172.21.15.125:0/1403033598 shutdown_connections 2023-10-18T10:45:04.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.353+0000 7f260ba39640 1 --2- 172.21.15.125:0/1403033598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f260415b2a0 0x7f26040d1910 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.353+0000 7f260ba39640 1 -- 172.21.15.125:0/1403033598 >> 172.21.15.125:0/1403033598 conn(0x7f26040d0d60 msgr2=0x7f26040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:04.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.353+0000 7f260ba39640 1 -- 172.21.15.125:0/1403033598 shutdown_connections 2023-10-18T10:45:04.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.353+0000 7f260ba39640 1 -- 172.21.15.125:0/1403033598 wait complete. 2023-10-18T10:45:04.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.354+0000 7f260ba39640 1 Processor -- start 2023-10-18T10:45:04.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.354+0000 7f260ba39640 1 -- start start 2023-10-18T10:45:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f260ba39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26040ceb40 0x7f26040b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f260ba39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26040cf340 0x7f26040b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f260ba39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f260415b2a0 0x7f26040b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:04.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f260ba39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2604153d90 con 0x7f26040ceb40 2023-10-18T10:45:04.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f260ba39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260415e5d0 con 0x7f260415b2a0 2023-10-18T10:45:04.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f260ba39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260415e2f0 con 0x7f26040cf340 2023-10-18T10:45:04.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f26097d5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26040ceb40 0x7f26040b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:04.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f26097d5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26040ceb40 0x7f26040b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36836/0 (socket says 172.21.15.125:36836) 2023-10-18T10:45:04.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f26097d5640 1 -- 172.21.15.125:0/239748091 learned_addr learned my addr 172.21.15.125:0/239748091 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:04.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f26097d5640 1 -- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26040cf340 msgr2=0x7f26040b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f26097d5640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26040cf340 0x7f26040b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f2609fd6640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f260415b2a0 0x7f26040b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:04.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.355+0000 7f26097d5640 1 -- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f260415b2a0 msgr2=0x7f26040b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f26097d5640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f260415b2a0 0x7f26040b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f26097d5640 1 -- 172.21.15.125:0/239748091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2600009490 con 0x7f26040ceb40 2023-10-18T10:45:04.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f2609fd6640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f260415b2a0 0x7f26040b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:04.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f26097d5640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26040ceb40 0x7f26040b4360 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c430 tx=0x7f25f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:04.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f25f9ffb640 1 -- 172.21.15.125:0/239748091 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4011790 con 0x7f26040ceb40 2023-10-18T10:45:04.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26040c9020 con 0x7f26040ceb40 2023-10-18T10:45:04.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f25f9ffb640 1 -- 172.21.15.125:0/239748091 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25f40119e0 con 0x7f26040ceb40 2023-10-18T10:45:04.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f25f9ffb640 1 -- 172.21.15.125:0/239748091 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4011e30 con 0x7f26040ceb40 2023-10-18T10:45:04.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.356+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26040c95e0 con 0x7f26040ceb40 2023-10-18T10:45:04.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.357+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26040c8e80 con 0x7f26040ceb40 2023-10-18T10:45:04.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.357+0000 7f25f9ffb640 1 -- 172.21.15.125:0/239748091 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f25f401c460 con 0x7f26040ceb40 2023-10-18T10:45:04.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.358+0000 7f25f9ffb640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f25e806af60 0x7f25e806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:04.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.358+0000 7f25f9ffb640 1 -- 172.21.15.125:0/239748091 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f25f4089b70 con 0x7f26040ceb40 2023-10-18T10:45:04.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.358+0000 7f2608fd4640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f25e806af60 0x7f25e806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:04.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.359+0000 7f2608fd4640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f25e806af60 0x7f25e806d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2600036000 tx=0x7f26000046c0 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:04.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.362+0000 7f25f9ffb640 1 -- 172.21.15.125:0/239748091 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26040c8e80 con 0x7f26040ceb40 2023-10-18T10:45:04.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.595+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f26040c8e80 con 0x7f26040ceb40 2023-10-18T10:45:04.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.595+0000 7f25f9ffb640 1 -- 172.21.15.125:0/239748091 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1024 (secure 0 0 0) 0x7f26040c8e80 con 0x7f26040ceb40 2023-10-18T10:45:04.597 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:04.598 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1226,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":156,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":352567296,"bytes_avail":772741545984,"bytes_total":773094113280,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":60,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9765}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-18T10:42:45.920324+0000","services":{}},"progress_events":{}} 2023-10-18T10:45:04.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f25e806af60 msgr2=0x7f25e806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f25e806af60 0x7f25e806d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2600036000 tx=0x7f26000046c0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26040ceb40 msgr2=0x7f26040b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26040ceb40 0x7f26040b4360 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c430 tx=0x7f25f400c8f0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 shutdown_connections 2023-10-18T10:45:04.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f25e806af60 0x7f25e806d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f260415b2a0 0x7f26040b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26040cf340 0x7f26040b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 --2- 172.21.15.125:0/239748091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26040ceb40 0x7f26040b4360 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 >> 172.21.15.125:0/239748091 conn(0x7f26040d0d60 msgr2=0x7f26040d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:04.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 shutdown_connections 2023-10-18T10:45:04.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.597+0000 7f260ba39640 1 -- 172.21.15.125:0/239748091 wait complete. 2023-10-18T10:45:04.609 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T10:45:04.610 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:45:04.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.711+0000 7f0d85b2c640 1 -- 172.21.15.125:0/624010641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 msgr2=0x7f0d800705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.711+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/624010641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 0x7f0d800705a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f0d74005fd0 tx=0x7f0d7402be10 comp rx=0 tx=0).stop 2023-10-18T10:45:04.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.711+0000 7f0d85b2c640 1 -- 172.21.15.125:0/624010641 shutdown_connections 2023-10-18T10:45:04.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.711+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/624010641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 0x7f0d800705a0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.711+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/624010641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800cbd20 0x7f0d80158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.711+0000 7f0d85b2c640 1 -- 172.21.15.125:0/624010641 >> 172.21.15.125:0/624010641 conn(0x7f0d80069930 msgr2=0x7f0d80069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:04.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.711+0000 7f0d85b2c640 1 -- 172.21.15.125:0/624010641 shutdown_connections 2023-10-18T10:45:04.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.711+0000 7f0d85b2c640 1 -- 172.21.15.125:0/624010641 wait complete. 2023-10-18T10:45:04.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.712+0000 7f0d85b2c640 1 Processor -- start 2023-10-18T10:45:04.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.712+0000 7f0d85b2c640 1 -- start start 2023-10-18T10:45:04.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d85b2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 0x7f0d8014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:04.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d85b2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d800b0530 0x7f0d8013d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:04.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d85b2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800cbd20 0x7f0d8015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:04.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d85b2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d801f21c0 con 0x7f0d800afa20 2023-10-18T10:45:04.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d85b2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d801f24a0 con 0x7f0d800b0530 2023-10-18T10:45:04.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d85b2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d801f2780 con 0x7f0d800cbd20 2023-10-18T10:45:04.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 0x7f0d8014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:04.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 0x7f0d8014cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36854/0 (socket says 172.21.15.125:36854) 2023-10-18T10:45:04.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7f7fe640 1 -- 172.21.15.125:0/1338021950 learned_addr learned my addr 172.21.15.125:0/1338021950 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:04.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7f7fe640 1 -- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800cbd20 msgr2=0x7f0d8015a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7f7fe640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800cbd20 0x7f0d8015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7f7fe640 1 -- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d800b0530 msgr2=0x7f0d8013d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7effd640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d800b0530 0x7f0d8013d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:04.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7f7fe640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d800b0530 0x7f0d8013d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.713+0000 7f0d7f7fe640 1 -- 172.21.15.125:0/1338021950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009560 con 0x7f0d800afa20 2023-10-18T10:45:04.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.714+0000 7f0d7f7fe640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 0x7f0d8014cd00 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b410 tx=0x7f0d7000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:04.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.714+0000 7f0d67fff640 1 -- 172.21.15.125:0/1338021950 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70011830 con 0x7f0d800afa20 2023-10-18T10:45:04.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.714+0000 7f0d67fff640 1 -- 172.21.15.125:0/1338021950 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d70011aa0 con 0x7f0d800afa20 2023-10-18T10:45:04.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.714+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d80070cf0 con 0x7f0d800afa20 2023-10-18T10:45:04.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.714+0000 7f0d67fff640 1 -- 172.21.15.125:0/1338021950 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70010380 con 0x7f0d800afa20 2023-10-18T10:45:04.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.714+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d801f2d40 con 0x7f0d800afa20 2023-10-18T10:45:04.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.715+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d801f2f30 con 0x7f0d800afa20 2023-10-18T10:45:04.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.715+0000 7f0d67fff640 1 -- 172.21.15.125:0/1338021950 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0d700104e0 con 0x7f0d800afa20 2023-10-18T10:45:04.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.716+0000 7f0d67fff640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0d6c06af60 0x7f0d6c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:04.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.716+0000 7f0d67fff640 1 -- 172.21.15.125:0/1338021950 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f0d70089ca0 con 0x7f0d800afa20 2023-10-18T10:45:04.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.716+0000 7f0d7effd640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0d6c06af60 0x7f0d6c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:04.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.716+0000 7f0d7effd640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0d6c06af60 0x7f0d6c06d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d74035000 tx=0x7f0d74004730 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:04.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.720+0000 7f0d67fff640 1 -- 172.21.15.125:0/1338021950 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d70054ae0 con 0x7f0d800afa20 2023-10-18T10:45:04.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.885+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 --> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d800cc1e0 con 0x7f0d6c06af60 2023-10-18T10:45:04.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.893+0000 7f0d67fff640 1 -- 172.21.15.125:0/1338021950 <== mgr.7340 v2:172.21.15.139:6832/3182280683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289628 (secure 0 0 0) 0x7f0d800cc1e0 con 0x7f0d6c06af60 2023-10-18T10:45:04.897 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:04.899 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:45:04.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0d6c06af60 msgr2=0x7f0d6c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0d6c06af60 0x7f0d6c06d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d74035000 tx=0x7f0d74004730 comp rx=0 tx=0).stop 2023-10-18T10:45:04.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 msgr2=0x7f0d8014cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:04.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 0x7f0d8014cd00 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b410 tx=0x7f0d7000b8d0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 shutdown_connections 2023-10-18T10:45:04.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f0d6c06af60 0x7f0d6c06d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d800b0530 0x7f0d8013d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d800cbd20 0x7f0d8015a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 --2- 172.21.15.125:0/1338021950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d800afa20 0x7f0d8014cd00 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:04.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 >> 172.21.15.125:0/1338021950 conn(0x7f0d80069930 msgr2=0x7f0d80158980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:04.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 shutdown_connections 2023-10-18T10:45:04.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:04.896+0000 7f0d85b2c640 1 -- 172.21.15.125:0/1338021950 wait complete. 2023-10-18T10:45:04.908 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T10:45:04.853362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":459,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344976,"kb_used_data":16400,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754629744,"statfs":{"total":773094113280,"available":772740857856,"internally_reserved":0,"allocated":16793600,"data_stored":9036740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":336318008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001539"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325812+0000","last_change":"2023-10-18T10:45:00.546090+0000","last_active":"2023-10-18T10:45:02.325812+0000","last_peered":"2023-10-18T10:45:02.325812+0000","last_clean":"2023-10-18T10:45:02.325812+0000","last_became_active":"2023-10-18T10:45:00.545758+0000","last_became_peered":"2023-10-18T10:45:00.545758+0000","last_unstale":"2023-10-18T10:45:02.325812+0000","last_undegraded":"2023-10-18T10:45:02.325812+0000","last_fullsized":"2023-10-18T10:45:02.325812+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:16.508214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326876+0000","last_change":"2023-10-18T10:45:00.546074+0000","last_active":"2023-10-18T10:45:02.326876+0000","last_peered":"2023-10-18T10:45:02.326876+0000","last_clean":"2023-10-18T10:45:02.326876+0000","last_became_active":"2023-10-18T10:45:00.545822+0000","last_became_peered":"2023-10-18T10:45:00.545822+0000","last_unstale":"2023-10-18T10:45:02.326876+0000","last_undegraded":"2023-10-18T10:45:02.326876+0000","last_fullsized":"2023-10-18T10:45:02.326876+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:50.162176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326810+0000","last_change":"2023-10-18T10:45:00.544896+0000","last_active":"2023-10-18T10:45:02.326810+0000","last_peered":"2023-10-18T10:45:02.326810+0000","last_clean":"2023-10-18T10:45:02.326810+0000","last_became_active":"2023-10-18T10:45:00.544647+0000","last_became_peered":"2023-10-18T10:45:00.544647+0000","last_unstale":"2023-10-18T10:45:02.326810+0000","last_undegraded":"2023-10-18T10:45:02.326810+0000","last_fullsized":"2023-10-18T10:45:02.326810+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:26.305847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326344+0000","last_change":"2023-10-18T10:45:00.543943+0000","last_active":"2023-10-18T10:45:02.326344+0000","last_peered":"2023-10-18T10:45:02.326344+0000","last_clean":"2023-10-18T10:45:02.326344+0000","last_became_active":"2023-10-18T10:45:00.543664+0000","last_became_peered":"2023-10-18T10:45:00.543664+0000","last_unstale":"2023-10-18T10:45:02.326344+0000","last_undegraded":"2023-10-18T10:45:02.326344+0000","last_fullsized":"2023-10-18T10:45:02.326344+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:44:47.375502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546238+0000","last_change":"2023-10-18T10:45:00.550989+0000","last_active":"2023-10-18T10:45:01.546238+0000","last_peered":"2023-10-18T10:45:01.546238+0000","last_clean":"2023-10-18T10:45:01.546238+0000","last_became_active":"2023-10-18T10:45:00.549311+0000","last_became_peered":"2023-10-18T10:45:00.549311+0000","last_unstale":"2023-10-18T10:45:01.546238+0000","last_undegraded":"2023-10-18T10:45:01.546238+0000","last_fullsized":"2023-10-18T10:45:01.546238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:07.713568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824940+0000","last_change":"2023-10-18T10:45:00.558326+0000","last_active":"2023-10-18T10:45:01.824940+0000","last_peered":"2023-10-18T10:45:01.824940+0000","last_clean":"2023-10-18T10:45:01.824940+0000","last_became_active":"2023-10-18T10:45:00.558088+0000","last_became_peered":"2023-10-18T10:45:00.558088+0000","last_unstale":"2023-10-18T10:45:01.824940+0000","last_undegraded":"2023-10-18T10:45:01.824940+0000","last_fullsized":"2023-10-18T10:45:01.824940+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:53.463565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547504+0000","last_change":"2023-10-18T10:45:00.559455+0000","last_active":"2023-10-18T10:45:01.547504+0000","last_peered":"2023-10-18T10:45:01.547504+0000","last_clean":"2023-10-18T10:45:01.547504+0000","last_became_active":"2023-10-18T10:45:00.559188+0000","last_became_peered":"2023-10-18T10:45:00.559188+0000","last_unstale":"2023-10-18T10:45:01.547504+0000","last_undegraded":"2023-10-18T10:45:01.547504+0000","last_fullsized":"2023-10-18T10:45:01.547504+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:07.713568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326010+0000","last_change":"2023-10-18T10:45:00.553452+0000","last_active":"2023-10-18T10:45:02.326010+0000","last_peered":"2023-10-18T10:45:02.326010+0000","last_clean":"2023-10-18T10:45:02.326010+0000","last_became_active":"2023-10-18T10:45:00.553135+0000","last_became_peered":"2023-10-18T10:45:00.553135+0000","last_unstale":"2023-10-18T10:45:02.326010+0000","last_undegraded":"2023-10-18T10:45:02.326010+0000","last_fullsized":"2023-10-18T10:45:02.326010+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:46:46.309099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825235+0000","last_change":"2023-10-18T10:45:00.555455+0000","last_active":"2023-10-18T10:45:01.825235+0000","last_peered":"2023-10-18T10:45:01.825235+0000","last_clean":"2023-10-18T10:45:01.825235+0000","last_became_active":"2023-10-18T10:45:00.553752+0000","last_became_peered":"2023-10-18T10:45:00.553752+0000","last_unstale":"2023-10-18T10:45:01.825235+0000","last_undegraded":"2023-10-18T10:45:01.825235+0000","last_fullsized":"2023-10-18T10:45:01.825235+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:05.808964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325837+0000","last_change":"2023-10-18T10:45:00.551287+0000","last_active":"2023-10-18T10:45:02.325837+0000","last_peered":"2023-10-18T10:45:02.325837+0000","last_clean":"2023-10-18T10:45:02.325837+0000","last_became_active":"2023-10-18T10:45:00.551048+0000","last_became_peered":"2023-10-18T10:45:00.551048+0000","last_unstale":"2023-10-18T10:45:02.325837+0000","last_undegraded":"2023-10-18T10:45:02.325837+0000","last_fullsized":"2023-10-18T10:45:02.325837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:42.148152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326207+0000","last_change":"2023-10-18T10:45:00.544992+0000","last_active":"2023-10-18T10:45:02.326207+0000","last_peered":"2023-10-18T10:45:02.326207+0000","last_clean":"2023-10-18T10:45:02.326207+0000","last_became_active":"2023-10-18T10:45:00.544409+0000","last_became_peered":"2023-10-18T10:45:00.544409+0000","last_unstale":"2023-10-18T10:45:02.326207+0000","last_undegraded":"2023-10-18T10:45:02.326207+0000","last_fullsized":"2023-10-18T10:45:02.326207+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:56.819335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325595+0000","last_change":"2023-10-18T10:45:00.544308+0000","last_active":"2023-10-18T10:45:02.325595+0000","last_peered":"2023-10-18T10:45:02.325595+0000","last_clean":"2023-10-18T10:45:02.325595+0000","last_became_active":"2023-10-18T10:45:00.544058+0000","last_became_peered":"2023-10-18T10:45:00.544058+0000","last_unstale":"2023-10-18T10:45:02.325595+0000","last_undegraded":"2023-10-18T10:45:02.325595+0000","last_fullsized":"2023-10-18T10:45:02.325595+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:06:01.716823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825170+0000","last_change":"2023-10-18T10:45:00.558519+0000","last_active":"2023-10-18T10:45:01.825170+0000","last_peered":"2023-10-18T10:45:01.825170+0000","last_clean":"2023-10-18T10:45:01.825170+0000","last_became_active":"2023-10-18T10:45:00.558257+0000","last_became_peered":"2023-10-18T10:45:00.558257+0000","last_unstale":"2023-10-18T10:45:01.825170+0000","last_undegraded":"2023-10-18T10:45:01.825170+0000","last_fullsized":"2023-10-18T10:45:01.825170+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:53:40.247907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325444+0000","last_change":"2023-10-18T10:45:00.556436+0000","last_active":"2023-10-18T10:45:02.325444+0000","last_peered":"2023-10-18T10:45:02.325444+0000","last_clean":"2023-10-18T10:45:02.325444+0000","last_became_active":"2023-10-18T10:45:00.556175+0000","last_became_peered":"2023-10-18T10:45:00.556175+0000","last_unstale":"2023-10-18T10:45:02.325444+0000","last_undegraded":"2023-10-18T10:45:02.325444+0000","last_fullsized":"2023-10-18T10:45:02.325444+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:04.086878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325231+0000","last_change":"2023-10-18T10:45:00.554061+0000","last_active":"2023-10-18T10:45:02.325231+0000","last_peered":"2023-10-18T10:45:02.325231+0000","last_clean":"2023-10-18T10:45:02.325231+0000","last_became_active":"2023-10-18T10:45:00.553781+0000","last_became_peered":"2023-10-18T10:45:00.553781+0000","last_unstale":"2023-10-18T10:45:02.325231+0000","last_undegraded":"2023-10-18T10:45:02.325231+0000","last_fullsized":"2023-10-18T10:45:02.325231+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:53:44.483745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824813+0000","last_change":"2023-10-18T10:45:00.547932+0000","last_active":"2023-10-18T10:45:01.824813+0000","last_peered":"2023-10-18T10:45:01.824813+0000","last_clean":"2023-10-18T10:45:01.824813+0000","last_became_active":"2023-10-18T10:45:00.547610+0000","last_became_peered":"2023-10-18T10:45:00.547610+0000","last_unstale":"2023-10-18T10:45:01.824813+0000","last_undegraded":"2023-10-18T10:45:01.824813+0000","last_fullsized":"2023-10-18T10:45:01.824813+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:22.672068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825262+0000","last_change":"2023-10-18T10:45:00.555957+0000","last_active":"2023-10-18T10:45:01.825262+0000","last_peered":"2023-10-18T10:45:01.825262+0000","last_clean":"2023-10-18T10:45:01.825262+0000","last_became_active":"2023-10-18T10:45:00.555715+0000","last_became_peered":"2023-10-18T10:45:00.555715+0000","last_unstale":"2023-10-18T10:45:01.825262+0000","last_undegraded":"2023-10-18T10:45:01.825262+0000","last_fullsized":"2023-10-18T10:45:01.825262+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:07.028997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546229+0000","last_change":"2023-10-18T10:45:00.549570+0000","last_active":"2023-10-18T10:45:01.546229+0000","last_peered":"2023-10-18T10:45:01.546229+0000","last_clean":"2023-10-18T10:45:01.546229+0000","last_became_active":"2023-10-18T10:45:00.549320+0000","last_became_peered":"2023-10-18T10:45:00.549320+0000","last_unstale":"2023-10-18T10:45:01.546229+0000","last_undegraded":"2023-10-18T10:45:01.546229+0000","last_fullsized":"2023-10-18T10:45:01.546229+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:19.566195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547943+0000","last_change":"2023-10-18T10:45:00.561776+0000","last_active":"2023-10-18T10:45:01.547943+0000","last_peered":"2023-10-18T10:45:01.547943+0000","last_clean":"2023-10-18T10:45:01.547943+0000","last_became_active":"2023-10-18T10:45:00.560779+0000","last_became_peered":"2023-10-18T10:45:00.560779+0000","last_unstale":"2023-10-18T10:45:01.547943+0000","last_undegraded":"2023-10-18T10:45:01.547943+0000","last_fullsized":"2023-10-18T10:45:01.547943+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:27:26.311482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546287+0000","last_change":"2023-10-18T10:45:00.549814+0000","last_active":"2023-10-18T10:45:01.546287+0000","last_peered":"2023-10-18T10:45:01.546287+0000","last_clean":"2023-10-18T10:45:01.546287+0000","last_became_active":"2023-10-18T10:45:00.549559+0000","last_became_peered":"2023-10-18T10:45:00.549559+0000","last_unstale":"2023-10-18T10:45:01.546287+0000","last_undegraded":"2023-10-18T10:45:01.546287+0000","last_fullsized":"2023-10-18T10:45:01.546287+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:48.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325802+0000","last_change":"2023-10-18T10:45:00.549693+0000","last_active":"2023-10-18T10:45:02.325802+0000","last_peered":"2023-10-18T10:45:02.325802+0000","last_clean":"2023-10-18T10:45:02.325802+0000","last_became_active":"2023-10-18T10:45:00.549444+0000","last_became_peered":"2023-10-18T10:45:00.549444+0000","last_unstale":"2023-10-18T10:45:02.325802+0000","last_undegraded":"2023-10-18T10:45:02.325802+0000","last_fullsized":"2023-10-18T10:45:02.325802+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:56.971883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325319+0000","last_change":"2023-10-18T10:45:00.547143+0000","last_active":"2023-10-18T10:45:02.325319+0000","last_peered":"2023-10-18T10:45:02.325319+0000","last_clean":"2023-10-18T10:45:02.325319+0000","last_became_active":"2023-10-18T10:45:00.546868+0000","last_became_peered":"2023-10-18T10:45:00.546868+0000","last_unstale":"2023-10-18T10:45:02.325319+0000","last_undegraded":"2023-10-18T10:45:02.325319+0000","last_fullsized":"2023-10-18T10:45:02.325319+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:30.548288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325857+0000","last_change":"2023-10-18T10:45:00.554470+0000","last_active":"2023-10-18T10:45:02.325857+0000","last_peered":"2023-10-18T10:45:02.325857+0000","last_clean":"2023-10-18T10:45:02.325857+0000","last_became_active":"2023-10-18T10:45:00.554221+0000","last_became_peered":"2023-10-18T10:45:00.554221+0000","last_unstale":"2023-10-18T10:45:02.325857+0000","last_undegraded":"2023-10-18T10:45:02.325857+0000","last_fullsized":"2023-10-18T10:45:02.325857+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:47:17.421888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824845+0000","last_change":"2023-10-18T10:45:00.556840+0000","last_active":"2023-10-18T10:45:01.824845+0000","last_peered":"2023-10-18T10:45:01.824845+0000","last_clean":"2023-10-18T10:45:01.824845+0000","last_became_active":"2023-10-18T10:45:00.556338+0000","last_became_peered":"2023-10-18T10:45:00.556338+0000","last_unstale":"2023-10-18T10:45:01.824845+0000","last_undegraded":"2023-10-18T10:45:01.824845+0000","last_fullsized":"2023-10-18T10:45:01.824845+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:59.589703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327115+0000","last_change":"2023-10-18T10:45:00.546465+0000","last_active":"2023-10-18T10:45:02.327115+0000","last_peered":"2023-10-18T10:45:02.327115+0000","last_clean":"2023-10-18T10:45:02.327115+0000","last_became_active":"2023-10-18T10:45:00.544998+0000","last_became_peered":"2023-10-18T10:45:00.544998+0000","last_unstale":"2023-10-18T10:45:02.327115+0000","last_undegraded":"2023-10-18T10:45:02.327115+0000","last_fullsized":"2023-10-18T10:45:02.327115+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:47.887433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326834+0000","last_change":"2023-10-18T10:45:00.555836+0000","last_active":"2023-10-18T10:45:02.326834+0000","last_peered":"2023-10-18T10:45:02.326834+0000","last_clean":"2023-10-18T10:45:02.326834+0000","last_became_active":"2023-10-18T10:45:00.555567+0000","last_became_peered":"2023-10-18T10:45:00.555567+0000","last_unstale":"2023-10-18T10:45:02.326834+0000","last_undegraded":"2023-10-18T10:45:02.326834+0000","last_fullsized":"2023-10-18T10:45:02.326834+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:16.195593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547906+0000","last_change":"2023-10-18T10:45:00.563377+0000","last_active":"2023-10-18T10:45:01.547906+0000","last_peered":"2023-10-18T10:45:01.547906+0000","last_clean":"2023-10-18T10:45:01.547906+0000","last_became_active":"2023-10-18T10:45:00.563093+0000","last_became_peered":"2023-10-18T10:45:00.563093+0000","last_unstale":"2023-10-18T10:45:01.547906+0000","last_undegraded":"2023-10-18T10:45:01.547906+0000","last_fullsized":"2023-10-18T10:45:01.547906+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:50.656392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325533+0000","last_change":"2023-10-18T10:45:00.550887+0000","last_active":"2023-10-18T10:45:02.325533+0000","last_peered":"2023-10-18T10:45:02.325533+0000","last_clean":"2023-10-18T10:45:02.325533+0000","last_became_active":"2023-10-18T10:45:00.550614+0000","last_became_peered":"2023-10-18T10:45:00.550614+0000","last_unstale":"2023-10-18T10:45:02.325533+0000","last_undegraded":"2023-10-18T10:45:02.325533+0000","last_fullsized":"2023-10-18T10:45:02.325533+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:31:59.143400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326344+0000","last_change":"2023-10-18T10:45:00.550674+0000","last_active":"2023-10-18T10:45:02.326344+0000","last_peered":"2023-10-18T10:45:02.326344+0000","last_clean":"2023-10-18T10:45:02.326344+0000","last_became_active":"2023-10-18T10:45:00.550416+0000","last_became_peered":"2023-10-18T10:45:00.550416+0000","last_unstale":"2023-10-18T10:45:02.326344+0000","last_undegraded":"2023-10-18T10:45:02.326344+0000","last_fullsized":"2023-10-18T10:45:02.326344+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:36:21.656622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326521+0000","last_change":"2023-10-18T10:45:00.546279+0000","last_active":"2023-10-18T10:45:02.326521+0000","last_peered":"2023-10-18T10:45:02.326521+0000","last_clean":"2023-10-18T10:45:02.326521+0000","last_became_active":"2023-10-18T10:45:00.545095+0000","last_became_peered":"2023-10-18T10:45:00.545095+0000","last_unstale":"2023-10-18T10:45:02.326521+0000","last_undegraded":"2023-10-18T10:45:02.326521+0000","last_fullsized":"2023-10-18T10:45:02.326521+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:46:46.309099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325952+0000","last_change":"2023-10-18T10:45:00.549973+0000","last_active":"2023-10-18T10:45:02.325952+0000","last_peered":"2023-10-18T10:45:02.325952+0000","last_clean":"2023-10-18T10:45:02.325952+0000","last_became_active":"2023-10-18T10:45:00.549717+0000","last_became_peered":"2023-10-18T10:45:00.549717+0000","last_unstale":"2023-10-18T10:45:02.325952+0000","last_undegraded":"2023-10-18T10:45:02.325952+0000","last_fullsized":"2023-10-18T10:45:02.325952+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:36:21.656622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325604+0000","last_change":"2023-10-18T10:45:00.548793+0000","last_active":"2023-10-18T10:45:02.325604+0000","last_peered":"2023-10-18T10:45:02.325604+0000","last_clean":"2023-10-18T10:45:02.325604+0000","last_became_active":"2023-10-18T10:45:00.548548+0000","last_became_peered":"2023-10-18T10:45:00.548548+0000","last_unstale":"2023-10-18T10:45:02.325604+0000","last_undegraded":"2023-10-18T10:45:02.325604+0000","last_fullsized":"2023-10-18T10:45:02.325604+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:21.744298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325864+0000","last_change":"2023-10-18T10:45:00.548611+0000","last_active":"2023-10-18T10:45:02.325864+0000","last_peered":"2023-10-18T10:45:02.325864+0000","last_clean":"2023-10-18T10:45:02.325864+0000","last_became_active":"2023-10-18T10:45:00.548351+0000","last_became_peered":"2023-10-18T10:45:00.548351+0000","last_unstale":"2023-10-18T10:45:02.325864+0000","last_undegraded":"2023-10-18T10:45:02.325864+0000","last_fullsized":"2023-10-18T10:45:02.325864+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:04.768446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"155'4","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327197+0000","last_change":"2023-10-18T10:44:59.546005+0000","last_active":"2023-10-18T10:45:02.327197+0000","last_peered":"2023-10-18T10:45:02.327197+0000","last_clean":"2023-10-18T10:45:02.327197+0000","last_became_active":"2023-10-18T10:44:59.545741+0000","last_became_peered":"2023-10-18T10:44:59.545741+0000","last_unstale":"2023-10-18T10:45:02.327197+0000","last_undegraded":"2023-10-18T10:45:02.327197+0000","last_fullsized":"2023-10-18T10:45:02.327197+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:17.293990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325326+0000","last_change":"2023-10-18T10:45:00.546398+0000","last_active":"2023-10-18T10:45:02.325326+0000","last_peered":"2023-10-18T10:45:02.325326+0000","last_clean":"2023-10-18T10:45:02.325326+0000","last_became_active":"2023-10-18T10:45:00.546122+0000","last_became_peered":"2023-10-18T10:45:00.546122+0000","last_unstale":"2023-10-18T10:45:02.325326+0000","last_undegraded":"2023-10-18T10:45:02.325326+0000","last_fullsized":"2023-10-18T10:45:02.325326+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:45:39.976465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546472+0000","last_change":"2023-10-18T10:44:59.549443+0000","last_active":"2023-10-18T10:45:01.546472+0000","last_peered":"2023-10-18T10:45:01.546472+0000","last_clean":"2023-10-18T10:45:01.546472+0000","last_became_active":"2023-10-18T10:44:59.549114+0000","last_became_peered":"2023-10-18T10:44:59.549114+0000","last_unstale":"2023-10-18T10:45:01.546472+0000","last_undegraded":"2023-10-18T10:45:01.546472+0000","last_fullsized":"2023-10-18T10:45:01.546472+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:52.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325840+0000","last_change":"2023-10-18T10:45:00.553749+0000","last_active":"2023-10-18T10:45:02.325840+0000","last_peered":"2023-10-18T10:45:02.325840+0000","last_clean":"2023-10-18T10:45:02.325840+0000","last_became_active":"2023-10-18T10:45:00.553477+0000","last_became_peered":"2023-10-18T10:45:00.553477+0000","last_unstale":"2023-10-18T10:45:02.325840+0000","last_undegraded":"2023-10-18T10:45:02.325840+0000","last_fullsized":"2023-10-18T10:45:02.325840+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:10.899448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.548016+0000","last_change":"2023-10-18T10:44:59.551861+0000","last_active":"2023-10-18T10:45:01.548016+0000","last_peered":"2023-10-18T10:45:01.548016+0000","last_clean":"2023-10-18T10:45:01.548016+0000","last_became_active":"2023-10-18T10:44:59.551597+0000","last_became_peered":"2023-10-18T10:44:59.551597+0000","last_unstale":"2023-10-18T10:45:01.548016+0000","last_undegraded":"2023-10-18T10:45:01.548016+0000","last_fullsized":"2023-10-18T10:45:01.548016+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:52.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325397+0000","last_change":"2023-10-18T10:45:00.554016+0000","last_active":"2023-10-18T10:45:02.325397+0000","last_peered":"2023-10-18T10:45:02.325397+0000","last_clean":"2023-10-18T10:45:02.325397+0000","last_became_active":"2023-10-18T10:45:00.553751+0000","last_became_peered":"2023-10-18T10:45:00.553751+0000","last_unstale":"2023-10-18T10:45:02.325397+0000","last_undegraded":"2023-10-18T10:45:02.325397+0000","last_fullsized":"2023-10-18T10:45:02.325397+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:09:55.073866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547756+0000","last_change":"2023-10-18T10:44:59.551206+0000","last_active":"2023-10-18T10:45:01.547756+0000","last_peered":"2023-10-18T10:45:01.547756+0000","last_clean":"2023-10-18T10:45:01.547756+0000","last_became_active":"2023-10-18T10:44:59.550813+0000","last_became_peered":"2023-10-18T10:44:59.550813+0000","last_unstale":"2023-10-18T10:45:01.547756+0000","last_undegraded":"2023-10-18T10:45:01.547756+0000","last_fullsized":"2023-10-18T10:45:01.547756+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:09.895689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327152+0000","last_change":"2023-10-18T10:45:00.551266+0000","last_active":"2023-10-18T10:45:02.327152+0000","last_peered":"2023-10-18T10:45:02.327152+0000","last_clean":"2023-10-18T10:45:02.327152+0000","last_became_active":"2023-10-18T10:45:00.551011+0000","last_became_peered":"2023-10-18T10:45:00.551011+0000","last_unstale":"2023-10-18T10:45:02.327152+0000","last_undegraded":"2023-10-18T10:45:02.327152+0000","last_fullsized":"2023-10-18T10:45:02.327152+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:51.890773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326018+0000","last_change":"2023-10-18T10:44:59.544948+0000","last_active":"2023-10-18T10:45:02.326018+0000","last_peered":"2023-10-18T10:45:02.326018+0000","last_clean":"2023-10-18T10:45:02.326018+0000","last_became_active":"2023-10-18T10:44:59.544685+0000","last_became_peered":"2023-10-18T10:44:59.544685+0000","last_unstale":"2023-10-18T10:45:02.326018+0000","last_undegraded":"2023-10-18T10:45:02.326018+0000","last_fullsized":"2023-10-18T10:45:02.326018+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:37:08.444269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326696+0000","last_change":"2023-10-18T10:45:00.557517+0000","last_active":"2023-10-18T10:45:02.326696+0000","last_peered":"2023-10-18T10:45:02.326696+0000","last_clean":"2023-10-18T10:45:02.326696+0000","last_became_active":"2023-10-18T10:45:00.557258+0000","last_became_peered":"2023-10-18T10:45:00.557258+0000","last_unstale":"2023-10-18T10:45:02.326696+0000","last_undegraded":"2023-10-18T10:45:02.326696+0000","last_fullsized":"2023-10-18T10:45:02.326696+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:39.894454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546090+0000","last_change":"2023-10-18T10:44:59.545459+0000","last_active":"2023-10-18T10:45:01.546090+0000","last_peered":"2023-10-18T10:45:01.546090+0000","last_clean":"2023-10-18T10:45:01.546090+0000","last_became_active":"2023-10-18T10:44:59.545188+0000","last_became_peered":"2023-10-18T10:44:59.545188+0000","last_unstale":"2023-10-18T10:45:01.546090+0000","last_undegraded":"2023-10-18T10:45:01.546090+0000","last_fullsized":"2023-10-18T10:45:01.546090+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:43.900519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546376+0000","last_change":"2023-10-18T10:45:00.558273+0000","last_active":"2023-10-18T10:45:01.546376+0000","last_peered":"2023-10-18T10:45:01.546376+0000","last_clean":"2023-10-18T10:45:01.546376+0000","last_became_active":"2023-10-18T10:45:00.557995+0000","last_became_peered":"2023-10-18T10:45:00.557995+0000","last_unstale":"2023-10-18T10:45:01.546376+0000","last_undegraded":"2023-10-18T10:45:01.546376+0000","last_fullsized":"2023-10-18T10:45:01.546376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:27.070435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824797+0000","last_change":"2023-10-18T10:44:59.551285+0000","last_active":"2023-10-18T10:45:01.824797+0000","last_peered":"2023-10-18T10:45:01.824797+0000","last_clean":"2023-10-18T10:45:01.824797+0000","last_became_active":"2023-10-18T10:44:59.550971+0000","last_became_peered":"2023-10-18T10:44:59.550971+0000","last_unstale":"2023-10-18T10:45:01.824797+0000","last_undegraded":"2023-10-18T10:45:01.824797+0000","last_fullsized":"2023-10-18T10:45:01.824797+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:34.997051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547576+0000","last_change":"2023-10-18T10:45:00.561634+0000","last_active":"2023-10-18T10:45:01.547576+0000","last_peered":"2023-10-18T10:45:01.547576+0000","last_clean":"2023-10-18T10:45:01.547576+0000","last_became_active":"2023-10-18T10:45:00.561382+0000","last_became_peered":"2023-10-18T10:45:00.561382+0000","last_unstale":"2023-10-18T10:45:01.547576+0000","last_undegraded":"2023-10-18T10:45:01.547576+0000","last_fullsized":"2023-10-18T10:45:01.547576+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:19.566195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326639+0000","last_change":"2023-10-18T10:44:59.554031+0000","last_active":"2023-10-18T10:45:02.326639+0000","last_peered":"2023-10-18T10:45:02.326639+0000","last_clean":"2023-10-18T10:45:02.326639+0000","last_became_active":"2023-10-18T10:44:59.553823+0000","last_became_peered":"2023-10-18T10:44:59.553823+0000","last_unstale":"2023-10-18T10:45:02.326639+0000","last_undegraded":"2023-10-18T10:45:02.326639+0000","last_fullsized":"2023-10-18T10:45:02.326639+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:29.312474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325230+0000","last_change":"2023-10-18T10:45:00.547832+0000","last_active":"2023-10-18T10:45:02.325230+0000","last_peered":"2023-10-18T10:45:02.325230+0000","last_clean":"2023-10-18T10:45:02.325230+0000","last_became_active":"2023-10-18T10:45:00.547509+0000","last_became_peered":"2023-10-18T10:45:00.547509+0000","last_unstale":"2023-10-18T10:45:02.325230+0000","last_undegraded":"2023-10-18T10:45:02.325230+0000","last_fullsized":"2023-10-18T10:45:02.325230+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:54.915478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327037+0000","last_change":"2023-10-18T10:44:59.548871+0000","last_active":"2023-10-18T10:45:02.327037+0000","last_peered":"2023-10-18T10:45:02.327037+0000","last_clean":"2023-10-18T10:45:02.327037+0000","last_became_active":"2023-10-18T10:44:59.548587+0000","last_became_peered":"2023-10-18T10:44:59.548587+0000","last_unstale":"2023-10-18T10:45:02.327037+0000","last_undegraded":"2023-10-18T10:45:02.327037+0000","last_fullsized":"2023-10-18T10:45:02.327037+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:50.205402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824921+0000","last_change":"2023-10-18T10:45:00.553629+0000","last_active":"2023-10-18T10:45:01.824921+0000","last_peered":"2023-10-18T10:45:01.824921+0000","last_clean":"2023-10-18T10:45:01.824921+0000","last_became_active":"2023-10-18T10:45:00.553350+0000","last_became_peered":"2023-10-18T10:45:00.553350+0000","last_unstale":"2023-10-18T10:45:01.824921+0000","last_undegraded":"2023-10-18T10:45:01.824921+0000","last_fullsized":"2023-10-18T10:45:01.824921+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:51.311062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325703+0000","last_change":"2023-10-18T10:44:59.546697+0000","last_active":"2023-10-18T10:45:02.325703+0000","last_peered":"2023-10-18T10:45:02.325703+0000","last_clean":"2023-10-18T10:45:02.325703+0000","last_became_active":"2023-10-18T10:44:59.546384+0000","last_became_peered":"2023-10-18T10:44:59.546384+0000","last_unstale":"2023-10-18T10:45:02.325703+0000","last_undegraded":"2023-10-18T10:45:02.325703+0000","last_fullsized":"2023-10-18T10:45:02.325703+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:27.802004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326462+0000","last_change":"2023-10-18T10:45:00.540375+0000","last_active":"2023-10-18T10:45:02.326462+0000","last_peered":"2023-10-18T10:45:02.326462+0000","last_clean":"2023-10-18T10:45:02.326462+0000","last_became_active":"2023-10-18T10:45:00.540112+0000","last_became_peered":"2023-10-18T10:45:00.540112+0000","last_unstale":"2023-10-18T10:45:02.326462+0000","last_undegraded":"2023-10-18T10:45:02.326462+0000","last_fullsized":"2023-10-18T10:45:02.326462+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:39.894454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"155'3","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.548042+0000","last_change":"2023-10-18T10:44:59.548751+0000","last_active":"2023-10-18T10:45:01.548042+0000","last_peered":"2023-10-18T10:45:01.548042+0000","last_clean":"2023-10-18T10:45:01.548042+0000","last_became_active":"2023-10-18T10:44:59.548474+0000","last_became_peered":"2023-10-18T10:44:59.548474+0000","last_unstale":"2023-10-18T10:45:01.548042+0000","last_undegraded":"2023-10-18T10:45:01.548042+0000","last_fullsized":"2023-10-18T10:45:01.548042+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:37.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327074+0000","last_change":"2023-10-18T10:45:00.549775+0000","last_active":"2023-10-18T10:45:02.327074+0000","last_peered":"2023-10-18T10:45:02.327074+0000","last_clean":"2023-10-18T10:45:02.327074+0000","last_became_active":"2023-10-18T10:45:00.549331+0000","last_became_peered":"2023-10-18T10:45:00.549331+0000","last_unstale":"2023-10-18T10:45:02.327074+0000","last_undegraded":"2023-10-18T10:45:02.327074+0000","last_fullsized":"2023-10-18T10:45:02.327074+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:51.261564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"155'6","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547724+0000","last_change":"2023-10-18T10:44:59.553324+0000","last_active":"2023-10-18T10:45:01.547724+0000","last_peered":"2023-10-18T10:45:01.547724+0000","last_clean":"2023-10-18T10:45:01.547724+0000","last_became_active":"2023-10-18T10:44:59.553025+0000","last_became_peered":"2023-10-18T10:44:59.553025+0000","last_unstale":"2023-10-18T10:45:01.547724+0000","last_undegraded":"2023-10-18T10:45:01.547724+0000","last_fullsized":"2023-10-18T10:45:01.547724+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:47:16.418129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325662+0000","last_change":"2023-10-18T10:45:00.545940+0000","last_active":"2023-10-18T10:45:02.325662+0000","last_peered":"2023-10-18T10:45:02.325662+0000","last_clean":"2023-10-18T10:45:02.325662+0000","last_became_active":"2023-10-18T10:45:00.545680+0000","last_became_peered":"2023-10-18T10:45:00.545680+0000","last_unstale":"2023-10-18T10:45:02.325662+0000","last_undegraded":"2023-10-18T10:45:02.325662+0000","last_fullsized":"2023-10-18T10:45:02.325662+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:48.726330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325494+0000","last_change":"2023-10-18T10:44:59.543508+0000","last_active":"2023-10-18T10:45:02.325494+0000","last_peered":"2023-10-18T10:45:02.325494+0000","last_clean":"2023-10-18T10:45:02.325494+0000","last_became_active":"2023-10-18T10:44:59.543237+0000","last_became_peered":"2023-10-18T10:44:59.543237+0000","last_unstale":"2023-10-18T10:45:02.325494+0000","last_undegraded":"2023-10-18T10:45:02.325494+0000","last_fullsized":"2023-10-18T10:45:02.325494+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:59.696128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547856+0000","last_change":"2023-10-18T10:45:00.556708+0000","last_active":"2023-10-18T10:45:01.547856+0000","last_peered":"2023-10-18T10:45:01.547856+0000","last_clean":"2023-10-18T10:45:01.547856+0000","last_became_active":"2023-10-18T10:45:00.556432+0000","last_became_peered":"2023-10-18T10:45:00.556432+0000","last_unstale":"2023-10-18T10:45:01.547856+0000","last_undegraded":"2023-10-18T10:45:01.547856+0000","last_fullsized":"2023-10-18T10:45:01.547856+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:05:13.997775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326393+0000","last_change":"2023-10-18T10:44:59.548866+0000","last_active":"2023-10-18T10:45:02.326393+0000","last_peered":"2023-10-18T10:45:02.326393+0000","last_clean":"2023-10-18T10:45:02.326393+0000","last_became_active":"2023-10-18T10:44:59.548228+0000","last_became_peered":"2023-10-18T10:44:59.548228+0000","last_unstale":"2023-10-18T10:45:02.326393+0000","last_undegraded":"2023-10-18T10:45:02.326393+0000","last_fullsized":"2023-10-18T10:45:02.326393+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:06:11.225137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546589+0000","last_change":"2023-10-18T10:45:00.548775+0000","last_active":"2023-10-18T10:45:01.546589+0000","last_peered":"2023-10-18T10:45:01.546589+0000","last_clean":"2023-10-18T10:45:01.546589+0000","last_became_active":"2023-10-18T10:45:00.548532+0000","last_became_peered":"2023-10-18T10:45:00.548532+0000","last_unstale":"2023-10-18T10:45:01.546589+0000","last_undegraded":"2023-10-18T10:45:01.546589+0000","last_fullsized":"2023-10-18T10:45:01.546589+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:42.529530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"155'8","reported_seq":20,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824912+0000","last_change":"2023-10-18T10:44:59.547619+0000","last_active":"2023-10-18T10:45:01.824912+0000","last_peered":"2023-10-18T10:45:01.824912+0000","last_clean":"2023-10-18T10:45:01.824912+0000","last_became_active":"2023-10-18T10:44:59.547301+0000","last_became_peered":"2023-10-18T10:44:59.547301+0000","last_unstale":"2023-10-18T10:45:01.824912+0000","last_undegraded":"2023-10-18T10:45:01.824912+0000","last_fullsized":"2023-10-18T10:45:01.824912+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:03.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326562+0000","last_change":"2023-10-18T10:45:00.546039+0000","last_active":"2023-10-18T10:45:02.326562+0000","last_peered":"2023-10-18T10:45:02.326562+0000","last_clean":"2023-10-18T10:45:02.326562+0000","last_became_active":"2023-10-18T10:45:00.545794+0000","last_became_peered":"2023-10-18T10:45:00.545794+0000","last_unstale":"2023-10-18T10:45:02.326562+0000","last_undegraded":"2023-10-18T10:45:02.326562+0000","last_fullsized":"2023-10-18T10:45:02.326562+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:51.890773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327161+0000","last_change":"2023-10-18T10:44:59.547983+0000","last_active":"2023-10-18T10:45:02.327161+0000","last_peered":"2023-10-18T10:45:02.327161+0000","last_clean":"2023-10-18T10:45:02.327161+0000","last_became_active":"2023-10-18T10:44:59.547638+0000","last_became_peered":"2023-10-18T10:44:59.547638+0000","last_unstale":"2023-10-18T10:45:02.327161+0000","last_undegraded":"2023-10-18T10:45:02.327161+0000","last_fullsized":"2023-10-18T10:45:02.327161+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:41:04.836469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547926+0000","last_change":"2023-10-18T10:45:00.558723+0000","last_active":"2023-10-18T10:45:01.547926+0000","last_peered":"2023-10-18T10:45:01.547926+0000","last_clean":"2023-10-18T10:45:01.547926+0000","last_became_active":"2023-10-18T10:45:00.558459+0000","last_became_peered":"2023-10-18T10:45:00.558459+0000","last_unstale":"2023-10-18T10:45:01.547926+0000","last_undegraded":"2023-10-18T10:45:01.547926+0000","last_fullsized":"2023-10-18T10:45:01.547926+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:42.529530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326919+0000","last_change":"2023-10-18T10:44:59.546867+0000","last_active":"2023-10-18T10:45:02.326919+0000","last_peered":"2023-10-18T10:45:02.326919+0000","last_clean":"2023-10-18T10:45:02.326919+0000","last_became_active":"2023-10-18T10:44:59.546598+0000","last_became_peered":"2023-10-18T10:44:59.546598+0000","last_unstale":"2023-10-18T10:45:02.326919+0000","last_undegraded":"2023-10-18T10:45:02.326919+0000","last_fullsized":"2023-10-18T10:45:02.326919+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:55.189311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547885+0000","last_change":"2023-10-18T10:45:00.562225+0000","last_active":"2023-10-18T10:45:01.547885+0000","last_peered":"2023-10-18T10:45:01.547885+0000","last_clean":"2023-10-18T10:45:01.547885+0000","last_became_active":"2023-10-18T10:45:00.561976+0000","last_became_peered":"2023-10-18T10:45:00.561976+0000","last_unstale":"2023-10-18T10:45:01.547885+0000","last_undegraded":"2023-10-18T10:45:01.547885+0000","last_fullsized":"2023-10-18T10:45:01.547885+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:27.070435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825094+0000","last_change":"2023-10-18T10:44:59.549433+0000","last_active":"2023-10-18T10:45:01.825094+0000","last_peered":"2023-10-18T10:45:01.825094+0000","last_clean":"2023-10-18T10:45:01.825094+0000","last_became_active":"2023-10-18T10:44:59.549180+0000","last_became_peered":"2023-10-18T10:44:59.549180+0000","last_unstale":"2023-10-18T10:45:01.825094+0000","last_undegraded":"2023-10-18T10:45:01.825094+0000","last_fullsized":"2023-10-18T10:45:01.825094+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:06.028167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325844+0000","last_change":"2023-10-18T10:45:00.547164+0000","last_active":"2023-10-18T10:45:02.325844+0000","last_peered":"2023-10-18T10:45:02.325844+0000","last_clean":"2023-10-18T10:45:02.325844+0000","last_became_active":"2023-10-18T10:45:00.546926+0000","last_became_peered":"2023-10-18T10:45:00.546926+0000","last_unstale":"2023-10-18T10:45:02.325844+0000","last_undegraded":"2023-10-18T10:45:02.325844+0000","last_fullsized":"2023-10-18T10:45:02.325844+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:48.531003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546363+0000","last_change":"2023-10-18T10:44:59.552659+0000","last_active":"2023-10-18T10:45:01.546363+0000","last_peered":"2023-10-18T10:45:01.546363+0000","last_clean":"2023-10-18T10:45:01.546363+0000","last_became_active":"2023-10-18T10:44:59.552403+0000","last_became_peered":"2023-10-18T10:44:59.552403+0000","last_unstale":"2023-10-18T10:45:01.546363+0000","last_undegraded":"2023-10-18T10:45:01.546363+0000","last_fullsized":"2023-10-18T10:45:01.546363+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:56.316476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546365+0000","last_change":"2023-10-18T10:45:00.542789+0000","last_active":"2023-10-18T10:45:01.546365+0000","last_peered":"2023-10-18T10:45:01.546365+0000","last_clean":"2023-10-18T10:45:01.546365+0000","last_became_active":"2023-10-18T10:45:00.542558+0000","last_became_peered":"2023-10-18T10:45:00.542558+0000","last_unstale":"2023-10-18T10:45:01.546365+0000","last_undegraded":"2023-10-18T10:45:01.546365+0000","last_fullsized":"2023-10-18T10:45:01.546365+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:05:13.997775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326479+0000","last_change":"2023-10-18T10:44:59.544421+0000","last_active":"2023-10-18T10:45:02.326479+0000","last_peered":"2023-10-18T10:45:02.326479+0000","last_clean":"2023-10-18T10:45:02.326479+0000","last_became_active":"2023-10-18T10:44:59.544159+0000","last_became_peered":"2023-10-18T10:44:59.544159+0000","last_unstale":"2023-10-18T10:45:02.326479+0000","last_undegraded":"2023-10-18T10:45:02.326479+0000","last_fullsized":"2023-10-18T10:45:02.326479+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:29.312474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325739+0000","last_change":"2023-10-18T10:45:00.556582+0000","last_active":"2023-10-18T10:45:02.325739+0000","last_peered":"2023-10-18T10:45:02.325739+0000","last_clean":"2023-10-18T10:45:02.325739+0000","last_became_active":"2023-10-18T10:45:00.556335+0000","last_became_peered":"2023-10-18T10:45:00.556335+0000","last_unstale":"2023-10-18T10:45:02.325739+0000","last_undegraded":"2023-10-18T10:45:02.325739+0000","last_fullsized":"2023-10-18T10:45:02.325739+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:07.811816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325537+0000","last_change":"2023-10-18T10:44:59.540315+0000","last_active":"2023-10-18T10:45:02.325537+0000","last_peered":"2023-10-18T10:45:02.325537+0000","last_clean":"2023-10-18T10:45:02.325537+0000","last_became_active":"2023-10-18T10:44:59.540036+0000","last_became_peered":"2023-10-18T10:44:59.540036+0000","last_unstale":"2023-10-18T10:45:02.325537+0000","last_undegraded":"2023-10-18T10:45:02.325537+0000","last_fullsized":"2023-10-18T10:45:02.325537+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:29:49.439257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546500+0000","last_change":"2023-10-18T10:45:00.548145+0000","last_active":"2023-10-18T10:45:01.546500+0000","last_peered":"2023-10-18T10:45:01.546500+0000","last_clean":"2023-10-18T10:45:01.546500+0000","last_became_active":"2023-10-18T10:45:00.547873+0000","last_became_peered":"2023-10-18T10:45:00.547873+0000","last_unstale":"2023-10-18T10:45:01.546500+0000","last_undegraded":"2023-10-18T10:45:01.546500+0000","last_fullsized":"2023-10-18T10:45:01.546500+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:50.656392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547694+0000","last_change":"2023-10-18T10:44:59.548633+0000","last_active":"2023-10-18T10:45:01.547694+0000","last_peered":"2023-10-18T10:45:01.547694+0000","last_clean":"2023-10-18T10:45:01.547694+0000","last_became_active":"2023-10-18T10:44:59.548345+0000","last_became_peered":"2023-10-18T10:44:59.548345+0000","last_unstale":"2023-10-18T10:45:01.547694+0000","last_undegraded":"2023-10-18T10:45:01.547694+0000","last_fullsized":"2023-10-18T10:45:01.547694+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:42:01.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326896+0000","last_change":"2023-10-18T10:45:00.545565+0000","last_active":"2023-10-18T10:45:02.326896+0000","last_peered":"2023-10-18T10:45:02.326896+0000","last_clean":"2023-10-18T10:45:02.326896+0000","last_became_active":"2023-10-18T10:45:00.545316+0000","last_became_peered":"2023-10-18T10:45:00.545316+0000","last_unstale":"2023-10-18T10:45:02.326896+0000","last_undegraded":"2023-10-18T10:45:02.326896+0000","last_fullsized":"2023-10-18T10:45:02.326896+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:04.326525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825124+0000","last_change":"2023-10-18T10:44:59.552577+0000","last_active":"2023-10-18T10:45:01.825124+0000","last_peered":"2023-10-18T10:45:01.825124+0000","last_clean":"2023-10-18T10:45:01.825124+0000","last_became_active":"2023-10-18T10:44:59.552329+0000","last_became_peered":"2023-10-18T10:44:59.552329+0000","last_unstale":"2023-10-18T10:45:01.825124+0000","last_undegraded":"2023-10-18T10:45:01.825124+0000","last_fullsized":"2023-10-18T10:45:01.825124+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:49.591451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546161+0000","last_change":"2023-10-18T10:45:00.548502+0000","last_active":"2023-10-18T10:45:01.546161+0000","last_peered":"2023-10-18T10:45:01.546161+0000","last_clean":"2023-10-18T10:45:01.546161+0000","last_became_active":"2023-10-18T10:45:00.548256+0000","last_became_peered":"2023-10-18T10:45:00.548256+0000","last_unstale":"2023-10-18T10:45:01.546161+0000","last_undegraded":"2023-10-18T10:45:01.546161+0000","last_fullsized":"2023-10-18T10:45:01.546161+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:02.628870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325868+0000","last_change":"2023-10-18T10:44:59.547011+0000","last_active":"2023-10-18T10:45:02.325868+0000","last_peered":"2023-10-18T10:45:02.325868+0000","last_clean":"2023-10-18T10:45:02.325868+0000","last_became_active":"2023-10-18T10:44:59.546769+0000","last_became_peered":"2023-10-18T10:44:59.546769+0000","last_unstale":"2023-10-18T10:45:02.325868+0000","last_undegraded":"2023-10-18T10:45:02.325868+0000","last_fullsized":"2023-10-18T10:45:02.325868+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:22.399832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326305+0000","last_change":"2023-10-18T10:45:00.542241+0000","last_active":"2023-10-18T10:45:02.326305+0000","last_peered":"2023-10-18T10:45:02.326305+0000","last_clean":"2023-10-18T10:45:02.326305+0000","last_became_active":"2023-10-18T10:45:00.541962+0000","last_became_peered":"2023-10-18T10:45:00.541962+0000","last_unstale":"2023-10-18T10:45:02.326305+0000","last_undegraded":"2023-10-18T10:45:02.326305+0000","last_fullsized":"2023-10-18T10:45:02.326305+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:04.768446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327275+0000","last_change":"2023-10-18T10:44:59.542014+0000","last_active":"2023-10-18T10:45:02.327275+0000","last_peered":"2023-10-18T10:45:02.327275+0000","last_clean":"2023-10-18T10:45:02.327275+0000","last_became_active":"2023-10-18T10:44:59.541731+0000","last_became_peered":"2023-10-18T10:44:59.541731+0000","last_unstale":"2023-10-18T10:45:02.327275+0000","last_undegraded":"2023-10-18T10:45:02.327275+0000","last_fullsized":"2023-10-18T10:45:02.327275+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:55.189311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547400+0000","last_change":"2023-10-18T10:45:00.558921+0000","last_active":"2023-10-18T10:45:01.547400+0000","last_peered":"2023-10-18T10:45:01.547400+0000","last_clean":"2023-10-18T10:45:01.547400+0000","last_became_active":"2023-10-18T10:45:00.558631+0000","last_became_peered":"2023-10-18T10:45:00.558631+0000","last_unstale":"2023-10-18T10:45:01.547400+0000","last_undegraded":"2023-10-18T10:45:01.547400+0000","last_fullsized":"2023-10-18T10:45:01.547400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:02.628870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326782+0000","last_change":"2023-10-18T10:44:59.541045+0000","last_active":"2023-10-18T10:45:02.326782+0000","last_peered":"2023-10-18T10:45:02.326782+0000","last_clean":"2023-10-18T10:45:02.326782+0000","last_became_active":"2023-10-18T10:44:59.540792+0000","last_became_peered":"2023-10-18T10:44:59.540792+0000","last_unstale":"2023-10-18T10:45:02.326782+0000","last_undegraded":"2023-10-18T10:45:02.326782+0000","last_fullsized":"2023-10-18T10:45:02.326782+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:04.237888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325937+0000","last_change":"2023-10-18T10:45:00.549757+0000","last_active":"2023-10-18T10:45:02.325937+0000","last_peered":"2023-10-18T10:45:02.325937+0000","last_clean":"2023-10-18T10:45:02.325937+0000","last_became_active":"2023-10-18T10:45:00.548892+0000","last_became_peered":"2023-10-18T10:45:00.548892+0000","last_unstale":"2023-10-18T10:45:02.325937+0000","last_undegraded":"2023-10-18T10:45:02.325937+0000","last_fullsized":"2023-10-18T10:45:02.325937+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:28:30.569831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825052+0000","last_change":"2023-10-18T10:44:59.551117+0000","last_active":"2023-10-18T10:45:01.825052+0000","last_peered":"2023-10-18T10:45:01.825052+0000","last_clean":"2023-10-18T10:45:01.825052+0000","last_became_active":"2023-10-18T10:44:59.550799+0000","last_became_peered":"2023-10-18T10:44:59.550799+0000","last_unstale":"2023-10-18T10:45:01.825052+0000","last_undegraded":"2023-10-18T10:45:01.825052+0000","last_fullsized":"2023-10-18T10:45:01.825052+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:39.812951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825083+0000","last_change":"2023-10-18T10:45:00.551092+0000","last_active":"2023-10-18T10:45:01.825083+0000","last_peered":"2023-10-18T10:45:01.825083+0000","last_clean":"2023-10-18T10:45:01.825083+0000","last_became_active":"2023-10-18T10:45:00.550827+0000","last_became_peered":"2023-10-18T10:45:00.550827+0000","last_unstale":"2023-10-18T10:45:01.825083+0000","last_undegraded":"2023-10-18T10:45:01.825083+0000","last_fullsized":"2023-10-18T10:45:01.825083+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:04.833412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326306+0000","last_change":"2023-10-18T10:44:59.544415+0000","last_active":"2023-10-18T10:45:02.326306+0000","last_peered":"2023-10-18T10:45:02.326306+0000","last_clean":"2023-10-18T10:45:02.326306+0000","last_became_active":"2023-10-18T10:44:59.543968+0000","last_became_peered":"2023-10-18T10:44:59.543968+0000","last_unstale":"2023-10-18T10:45:02.326306+0000","last_undegraded":"2023-10-18T10:45:02.326306+0000","last_fullsized":"2023-10-18T10:45:02.326306+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:00.870187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825088+0000","last_change":"2023-10-18T10:45:00.550486+0000","last_active":"2023-10-18T10:45:01.825088+0000","last_peered":"2023-10-18T10:45:01.825088+0000","last_clean":"2023-10-18T10:45:01.825088+0000","last_became_active":"2023-10-18T10:45:00.550239+0000","last_became_peered":"2023-10-18T10:45:00.550239+0000","last_unstale":"2023-10-18T10:45:01.825088+0000","last_undegraded":"2023-10-18T10:45:01.825088+0000","last_fullsized":"2023-10-18T10:45:01.825088+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:13.442928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547672+0000","last_change":"2023-10-18T10:44:59.551190+0000","last_active":"2023-10-18T10:45:01.547672+0000","last_peered":"2023-10-18T10:45:01.547672+0000","last_clean":"2023-10-18T10:45:01.547672+0000","last_became_active":"2023-10-18T10:44:59.550893+0000","last_became_peered":"2023-10-18T10:44:59.550893+0000","last_unstale":"2023-10-18T10:45:01.547672+0000","last_undegraded":"2023-10-18T10:45:01.547672+0000","last_fullsized":"2023-10-18T10:45:01.547672+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:56.251510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825179+0000","last_change":"2023-10-18T10:45:00.554546+0000","last_active":"2023-10-18T10:45:01.825179+0000","last_peered":"2023-10-18T10:45:01.825179+0000","last_clean":"2023-10-18T10:45:01.825179+0000","last_became_active":"2023-10-18T10:45:00.554296+0000","last_became_peered":"2023-10-18T10:45:00.554296+0000","last_unstale":"2023-10-18T10:45:01.825179+0000","last_undegraded":"2023-10-18T10:45:01.825179+0000","last_fullsized":"2023-10-18T10:45:01.825179+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:45.203109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824916+0000","last_change":"2023-10-18T10:44:59.551559+0000","last_active":"2023-10-18T10:45:01.824916+0000","last_peered":"2023-10-18T10:45:01.824916+0000","last_clean":"2023-10-18T10:45:01.824916+0000","last_became_active":"2023-10-18T10:44:59.551296+0000","last_became_peered":"2023-10-18T10:44:59.551296+0000","last_unstale":"2023-10-18T10:45:01.824916+0000","last_undegraded":"2023-10-18T10:45:01.824916+0000","last_fullsized":"2023-10-18T10:45:01.824916+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:34:31.261305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547848+0000","last_change":"2023-10-18T10:45:00.556380+0000","last_active":"2023-10-18T10:45:01.547848+0000","last_peered":"2023-10-18T10:45:01.547848+0000","last_clean":"2023-10-18T10:45:01.547848+0000","last_became_active":"2023-10-18T10:45:00.556121+0000","last_became_peered":"2023-10-18T10:45:00.556121+0000","last_unstale":"2023-10-18T10:45:01.547848+0000","last_undegraded":"2023-10-18T10:45:01.547848+0000","last_fullsized":"2023-10-18T10:45:01.547848+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:48.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546246+0000","last_change":"2023-10-18T10:44:59.552570+0000","last_active":"2023-10-18T10:45:01.546246+0000","last_peered":"2023-10-18T10:45:01.546246+0000","last_clean":"2023-10-18T10:45:01.546246+0000","last_became_active":"2023-10-18T10:44:59.552313+0000","last_became_peered":"2023-10-18T10:44:59.552313+0000","last_unstale":"2023-10-18T10:45:01.546246+0000","last_undegraded":"2023-10-18T10:45:01.546246+0000","last_fullsized":"2023-10-18T10:45:01.546246+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:37.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326963+0000","last_change":"2023-10-18T10:45:00.543609+0000","last_active":"2023-10-18T10:45:02.326963+0000","last_peered":"2023-10-18T10:45:02.326963+0000","last_clean":"2023-10-18T10:45:02.326963+0000","last_became_active":"2023-10-18T10:45:00.542794+0000","last_became_peered":"2023-10-18T10:45:00.542794+0000","last_unstale":"2023-10-18T10:45:02.326963+0000","last_undegraded":"2023-10-18T10:45:02.326963+0000","last_fullsized":"2023-10-18T10:45:02.326963+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:55:21.772344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547503+0000","last_change":"2023-10-18T10:44:59.548609+0000","last_active":"2023-10-18T10:45:01.547503+0000","last_peered":"2023-10-18T10:45:01.547503+0000","last_clean":"2023-10-18T10:45:01.547503+0000","last_became_active":"2023-10-18T10:44:59.548246+0000","last_became_peered":"2023-10-18T10:44:59.548246+0000","last_unstale":"2023-10-18T10:45:01.547503+0000","last_undegraded":"2023-10-18T10:45:01.547503+0000","last_fullsized":"2023-10-18T10:45:01.547503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:44:14.959576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"145'258","reported_seq":788,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-18T10:45:00.546249+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:45:00.546249+0000","last_peered":"2023-10-18T10:45:00.546249+0000","last_clean":"2023-10-18T10:45:00.546249+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:45:00.546249+0000","last_undegraded":"2023-10-18T10:45:00.546249+0000","last_fullsized":"2023-10-18T10:45:00.546249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575090,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60052,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311788,"statfs":{"total":96636764160,"available":96575270912,"internally_reserved":0,"allocated":2555904,"data_stored":1575130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.217,"15min":0.217},"max":{"1min":0.77900000000000003,"5min":0.876,"15min":0.88300000000000001},"last":0.878},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.96199999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":2.899},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.90700000000000003},"last":1.036}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":1.012}]},{"osd":3,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77900000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60099999999999998},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":2.8199999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63400000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":1.8839999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.86199999999999999},"last":4.8019999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":2.7210000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":1.6200000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":1.397},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":1.671}]}]},{"osd":6,"up_from":13,"seq":55834575090,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1941504,"data_stored":977084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":0.82499999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.75900000000000001,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.61299999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.36799999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.749,"5min":0.78900000000000003,"15min":0.82899999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.70099999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.442,"5min":0.372,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.70099999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.65400000000000003,"5min":0.78400000000000003,"15min":0.872},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.72599999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76100000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.745},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71499999999999997,"5min":0.82499999999999996,"15min":0.83699999999999997},"last":0.59799999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.313},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.85399999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84699999999999998,"15min":0.87},"last":0.79300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575090,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51132,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320708,"statfs":{"total":96636764160,"available":96584404992,"internally_reserved":0,"allocated":1949696,"data_stored":983082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.497}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.58599999999999997},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.58299999999999996},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76500000000000001,"15min":0.84199999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.80100000000000005},"last":0.376},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.41499999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.83299999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.83099999999999996,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.31},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.89300000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.871},"last":0.44500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575090,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1945600,"data_stored":983060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.877},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.89300000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.33100000000000002},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.5,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.82899999999999996,"5min":0.873,"15min":0.873},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.504,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.434}]},{"osd":6,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":2.8519999999999999,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.48199999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.60399999999999998,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.46100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575090,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1961984,"data_stored":984742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.36499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.62},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.59299999999999997},"last":0.26500000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20799999999999999},"max":{"1min":0.504,"5min":0.624,"15min":0.69099999999999995},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.59299999999999997,"15min":0.67000000000000004},"last":0.28599999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.33600000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.98299999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.72999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86599999999999999,"15min":1.002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.88100000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.70099999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38,"15min":0.37},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.018},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.879,"15min":0.96799999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":0.98899999999999999},"last":0.81999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575090,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55292,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316548,"statfs":{"total":96636764160,"available":96580145152,"internally_reserved":0,"allocated":1941504,"data_stored":977028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.17599999999999999},"max":{"1min":0.54500000000000004,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.38700000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.192},"max":{"1min":0.505,"5min":0.55500000000000005,"15min":0.626},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.55500000000000005,"5min":0.57899999999999996,"15min":0.67000000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55700000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.378,"15min":0.39300000000000002},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.40899999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.64000000000000001},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.627,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.84499999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.84799999999999998,"5min":0.87,"15min":0.93100000000000005},"last":0.79600000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.89100000000000001,"15min":0.93600000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.85199999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.878,"15min":0.91000000000000003},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":983130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.253},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.374,"15min":0.38600000000000001},"min":{"1min":0.221,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51400000000000001,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.44}]},{"osd":2,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36699999999999999,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.68899999999999995},"last":0.314},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52000000000000002,"5min":0.627,"15min":0.64800000000000002},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.42899999999999999},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":0.70199999999999996,"15min":0.72899999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":0.82999999999999996,"5min":1.099,"15min":1.099},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":0.88100000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.63300000000000001},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.373,"15min":1.373},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":0.83099999999999996,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.45600000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.86099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.51300000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575089,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37828,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334012,"statfs":{"total":96636764160,"available":96598028288,"internally_reserved":0,"allocated":2547712,"data_stored":1573484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.37,"15min":0.379},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.56599999999999995,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.39000000000000001},"min":{"1min":0.26300000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.57999999999999996,"5min":0.62,"15min":0.76900000000000002},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.53700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.22,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.30299999999999999,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.69699999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.57599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39300000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.996,"15min":0.996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.79200000000000004,"5min":1.038,"15min":1.038},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.44500000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90400000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63500000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.872,"5min":1.026,"15min":1.3700000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.339},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.48799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":1.01,"15min":1.01},"last":0.57899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:45:06.509 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:45:06.512 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:45:07.915 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T10:45:08.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.033+0000 7f2a0bea9640 1 -- 172.21.15.125:0/1667254541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a040c9d00 msgr2=0x7f2a0406b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.033+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/1667254541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a040c9d00 0x7f2a0406b910 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005fd0 tx=0x7f29fc02c0e0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.033+0000 7f2a0bea9640 1 -- 172.21.15.125:0/1667254541 shutdown_connections 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.033+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/1667254541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a040c9d00 0x7f2a0406b910 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.033+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/1667254541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a04140660 0x7f2a040c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.033+0000 7f2a0bea9640 1 -- 172.21.15.125:0/1667254541 >> 172.21.15.125:0/1667254541 conn(0x7f2a04069930 msgr2=0x7f2a04069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:08.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.033+0000 7f2a0bea9640 1 -- 172.21.15.125:0/1667254541 shutdown_connections 2023-10-18T10:45:08.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.033+0000 7f2a0bea9640 1 -- 172.21.15.125:0/1667254541 wait complete. 2023-10-18T10:45:08.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0bea9640 1 Processor -- start 2023-10-18T10:45:08.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0bea9640 1 -- start start 2023-10-18T10:45:08.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0bea9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a040c9d00 0x7f2a04136fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0bea9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0413d360 0x7f2a041513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0bea9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a04140660 0x7f2a04138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0bea9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04150fd0 con 0x7f2a04140660 2023-10-18T10:45:08.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0bea9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04139df0 con 0x7f2a040c9d00 2023-10-18T10:45:08.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0bea9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04139830 con 0x7f2a0413d360 2023-10-18T10:45:08.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0a446640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a04140660 0x7f2a04138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0a446640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a04140660 0x7f2a04138ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40784/0 (socket says 172.21.15.125:40784) 2023-10-18T10:45:08.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.035+0000 7f2a0a446640 1 -- 172.21.15.125:0/2265515203 learned_addr learned my addr 172.21.15.125:0/2265515203 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:08.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a09444640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0413d360 0x7f2a041513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a09c45640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a040c9d00 0x7f2a04136fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a0a446640 1 -- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0413d360 msgr2=0x7f2a041513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a0a446640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0413d360 0x7f2a041513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a0a446640 1 -- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a040c9d00 msgr2=0x7f2a04136fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a0a446640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a040c9d00 0x7f2a04136fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a0a446640 1 -- 172.21.15.125:0/2265515203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40071c0 con 0x7f2a04140660 2023-10-18T10:45:08.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a0a446640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a04140660 0x7f2a04138ec0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f29fc036040 tx=0x7f29fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:08.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f29fa7fc640 1 -- 172.21.15.125:0/2265515203 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc004670 con 0x7f2a04140660 2023-10-18T10:45:08.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f29fa7fc640 1 -- 172.21.15.125:0/2265515203 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29fc0345a0 con 0x7f2a04140660 2023-10-18T10:45:08.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f29fa7fc640 1 -- 172.21.15.125:0/2265515203 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc034970 con 0x7f2a04140660 2023-10-18T10:45:08.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc009840 con 0x7f2a04140660 2023-10-18T10:45:08.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.036+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a04150bf0 con 0x7f2a04140660 2023-10-18T10:45:08.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.037+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0406ccd0 con 0x7f2a04140660 2023-10-18T10:45:08.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.038+0000 7f29fa7fc640 1 -- 172.21.15.125:0/2265515203 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f29fc034ad0 con 0x7f2a04140660 2023-10-18T10:45:08.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.038+0000 7f29fa7fc640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f29e806af90 0x7f29e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.038+0000 7f29fa7fc640 1 -- 172.21.15.125:0/2265515203 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f29fc0ac8c0 con 0x7f2a04140660 2023-10-18T10:45:08.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.038+0000 7f2a09c45640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f29e806af90 0x7f29e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.039+0000 7f2a09c45640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f29e806af90 0x7f29e806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a00000f60 tx=0x7f2a00009040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:08.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.043+0000 7f29fa7fc640 1 -- 172.21.15.125:0/2265515203 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29fc0777b0 con 0x7f2a04140660 2023-10-18T10:45:08.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.206+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 --> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a0406be50 con 0x7f29e806af90 2023-10-18T10:45:08.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.214+0000 7f29fa7fc640 1 -- 172.21.15.125:0/2265515203 <== mgr.7340 v2:172.21.15.139:6832/3182280683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289628 (secure 0 0 0) 0x7f2a000014a0 con 0x7f29e806af90 2023-10-18T10:45:08.219 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:08.220 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T10:45:08.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f29e806af90 msgr2=0x7f29e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f29e806af90 0x7f29e806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a00000f60 tx=0x7f2a00009040 comp rx=0 tx=0).stop 2023-10-18T10:45:08.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a04140660 msgr2=0x7f2a04138ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a04140660 0x7f2a04138ec0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f29fc036040 tx=0x7f29fc004270 comp rx=0 tx=0).stop 2023-10-18T10:45:08.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 shutdown_connections 2023-10-18T10:45:08.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f29e806af90 0x7f29e806d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a04140660 0x7f2a04138ec0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0413d360 0x7f2a041513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 --2- 172.21.15.125:0/2265515203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a040c9d00 0x7f2a04136fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 >> 172.21.15.125:0/2265515203 conn(0x7f2a04069930 msgr2=0x7f2a041565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:08.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 shutdown_connections 2023-10-18T10:45:08.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.217+0000 7f2a0bea9640 1 -- 172.21.15.125:0/2265515203 wait complete. 2023-10-18T10:45:08.229 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T10:45:06.853898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":459,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344976,"kb_used_data":16400,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754629744,"statfs":{"total":773094113280,"available":772740857856,"internally_reserved":0,"allocated":16793600,"data_stored":9036740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":336318008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.319940"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325812+0000","last_change":"2023-10-18T10:45:00.546090+0000","last_active":"2023-10-18T10:45:02.325812+0000","last_peered":"2023-10-18T10:45:02.325812+0000","last_clean":"2023-10-18T10:45:02.325812+0000","last_became_active":"2023-10-18T10:45:00.545758+0000","last_became_peered":"2023-10-18T10:45:00.545758+0000","last_unstale":"2023-10-18T10:45:02.325812+0000","last_undegraded":"2023-10-18T10:45:02.325812+0000","last_fullsized":"2023-10-18T10:45:02.325812+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:16.508214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326876+0000","last_change":"2023-10-18T10:45:00.546074+0000","last_active":"2023-10-18T10:45:02.326876+0000","last_peered":"2023-10-18T10:45:02.326876+0000","last_clean":"2023-10-18T10:45:02.326876+0000","last_became_active":"2023-10-18T10:45:00.545822+0000","last_became_peered":"2023-10-18T10:45:00.545822+0000","last_unstale":"2023-10-18T10:45:02.326876+0000","last_undegraded":"2023-10-18T10:45:02.326876+0000","last_fullsized":"2023-10-18T10:45:02.326876+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:50.162176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326810+0000","last_change":"2023-10-18T10:45:00.544896+0000","last_active":"2023-10-18T10:45:02.326810+0000","last_peered":"2023-10-18T10:45:02.326810+0000","last_clean":"2023-10-18T10:45:02.326810+0000","last_became_active":"2023-10-18T10:45:00.544647+0000","last_became_peered":"2023-10-18T10:45:00.544647+0000","last_unstale":"2023-10-18T10:45:02.326810+0000","last_undegraded":"2023-10-18T10:45:02.326810+0000","last_fullsized":"2023-10-18T10:45:02.326810+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:26.305847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326344+0000","last_change":"2023-10-18T10:45:00.543943+0000","last_active":"2023-10-18T10:45:02.326344+0000","last_peered":"2023-10-18T10:45:02.326344+0000","last_clean":"2023-10-18T10:45:02.326344+0000","last_became_active":"2023-10-18T10:45:00.543664+0000","last_became_peered":"2023-10-18T10:45:00.543664+0000","last_unstale":"2023-10-18T10:45:02.326344+0000","last_undegraded":"2023-10-18T10:45:02.326344+0000","last_fullsized":"2023-10-18T10:45:02.326344+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:44:47.375502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546238+0000","last_change":"2023-10-18T10:45:00.550989+0000","last_active":"2023-10-18T10:45:01.546238+0000","last_peered":"2023-10-18T10:45:01.546238+0000","last_clean":"2023-10-18T10:45:01.546238+0000","last_became_active":"2023-10-18T10:45:00.549311+0000","last_became_peered":"2023-10-18T10:45:00.549311+0000","last_unstale":"2023-10-18T10:45:01.546238+0000","last_undegraded":"2023-10-18T10:45:01.546238+0000","last_fullsized":"2023-10-18T10:45:01.546238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:07.713568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824940+0000","last_change":"2023-10-18T10:45:00.558326+0000","last_active":"2023-10-18T10:45:01.824940+0000","last_peered":"2023-10-18T10:45:01.824940+0000","last_clean":"2023-10-18T10:45:01.824940+0000","last_became_active":"2023-10-18T10:45:00.558088+0000","last_became_peered":"2023-10-18T10:45:00.558088+0000","last_unstale":"2023-10-18T10:45:01.824940+0000","last_undegraded":"2023-10-18T10:45:01.824940+0000","last_fullsized":"2023-10-18T10:45:01.824940+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:53.463565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547504+0000","last_change":"2023-10-18T10:45:00.559455+0000","last_active":"2023-10-18T10:45:01.547504+0000","last_peered":"2023-10-18T10:45:01.547504+0000","last_clean":"2023-10-18T10:45:01.547504+0000","last_became_active":"2023-10-18T10:45:00.559188+0000","last_became_peered":"2023-10-18T10:45:00.559188+0000","last_unstale":"2023-10-18T10:45:01.547504+0000","last_undegraded":"2023-10-18T10:45:01.547504+0000","last_fullsized":"2023-10-18T10:45:01.547504+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:07.713568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326010+0000","last_change":"2023-10-18T10:45:00.553452+0000","last_active":"2023-10-18T10:45:02.326010+0000","last_peered":"2023-10-18T10:45:02.326010+0000","last_clean":"2023-10-18T10:45:02.326010+0000","last_became_active":"2023-10-18T10:45:00.553135+0000","last_became_peered":"2023-10-18T10:45:00.553135+0000","last_unstale":"2023-10-18T10:45:02.326010+0000","last_undegraded":"2023-10-18T10:45:02.326010+0000","last_fullsized":"2023-10-18T10:45:02.326010+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:46:46.309099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825235+0000","last_change":"2023-10-18T10:45:00.555455+0000","last_active":"2023-10-18T10:45:01.825235+0000","last_peered":"2023-10-18T10:45:01.825235+0000","last_clean":"2023-10-18T10:45:01.825235+0000","last_became_active":"2023-10-18T10:45:00.553752+0000","last_became_peered":"2023-10-18T10:45:00.553752+0000","last_unstale":"2023-10-18T10:45:01.825235+0000","last_undegraded":"2023-10-18T10:45:01.825235+0000","last_fullsized":"2023-10-18T10:45:01.825235+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:05.808964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325837+0000","last_change":"2023-10-18T10:45:00.551287+0000","last_active":"2023-10-18T10:45:02.325837+0000","last_peered":"2023-10-18T10:45:02.325837+0000","last_clean":"2023-10-18T10:45:02.325837+0000","last_became_active":"2023-10-18T10:45:00.551048+0000","last_became_peered":"2023-10-18T10:45:00.551048+0000","last_unstale":"2023-10-18T10:45:02.325837+0000","last_undegraded":"2023-10-18T10:45:02.325837+0000","last_fullsized":"2023-10-18T10:45:02.325837+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:48:42.148152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326207+0000","last_change":"2023-10-18T10:45:00.544992+0000","last_active":"2023-10-18T10:45:02.326207+0000","last_peered":"2023-10-18T10:45:02.326207+0000","last_clean":"2023-10-18T10:45:02.326207+0000","last_became_active":"2023-10-18T10:45:00.544409+0000","last_became_peered":"2023-10-18T10:45:00.544409+0000","last_unstale":"2023-10-18T10:45:02.326207+0000","last_undegraded":"2023-10-18T10:45:02.326207+0000","last_fullsized":"2023-10-18T10:45:02.326207+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:56.819335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325595+0000","last_change":"2023-10-18T10:45:00.544308+0000","last_active":"2023-10-18T10:45:02.325595+0000","last_peered":"2023-10-18T10:45:02.325595+0000","last_clean":"2023-10-18T10:45:02.325595+0000","last_became_active":"2023-10-18T10:45:00.544058+0000","last_became_peered":"2023-10-18T10:45:00.544058+0000","last_unstale":"2023-10-18T10:45:02.325595+0000","last_undegraded":"2023-10-18T10:45:02.325595+0000","last_fullsized":"2023-10-18T10:45:02.325595+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:06:01.716823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825170+0000","last_change":"2023-10-18T10:45:00.558519+0000","last_active":"2023-10-18T10:45:01.825170+0000","last_peered":"2023-10-18T10:45:01.825170+0000","last_clean":"2023-10-18T10:45:01.825170+0000","last_became_active":"2023-10-18T10:45:00.558257+0000","last_became_peered":"2023-10-18T10:45:00.558257+0000","last_unstale":"2023-10-18T10:45:01.825170+0000","last_undegraded":"2023-10-18T10:45:01.825170+0000","last_fullsized":"2023-10-18T10:45:01.825170+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:53:40.247907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325444+0000","last_change":"2023-10-18T10:45:00.556436+0000","last_active":"2023-10-18T10:45:02.325444+0000","last_peered":"2023-10-18T10:45:02.325444+0000","last_clean":"2023-10-18T10:45:02.325444+0000","last_became_active":"2023-10-18T10:45:00.556175+0000","last_became_peered":"2023-10-18T10:45:00.556175+0000","last_unstale":"2023-10-18T10:45:02.325444+0000","last_undegraded":"2023-10-18T10:45:02.325444+0000","last_fullsized":"2023-10-18T10:45:02.325444+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:04.086878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325231+0000","last_change":"2023-10-18T10:45:00.554061+0000","last_active":"2023-10-18T10:45:02.325231+0000","last_peered":"2023-10-18T10:45:02.325231+0000","last_clean":"2023-10-18T10:45:02.325231+0000","last_became_active":"2023-10-18T10:45:00.553781+0000","last_became_peered":"2023-10-18T10:45:00.553781+0000","last_unstale":"2023-10-18T10:45:02.325231+0000","last_undegraded":"2023-10-18T10:45:02.325231+0000","last_fullsized":"2023-10-18T10:45:02.325231+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:53:44.483745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824813+0000","last_change":"2023-10-18T10:45:00.547932+0000","last_active":"2023-10-18T10:45:01.824813+0000","last_peered":"2023-10-18T10:45:01.824813+0000","last_clean":"2023-10-18T10:45:01.824813+0000","last_became_active":"2023-10-18T10:45:00.547610+0000","last_became_peered":"2023-10-18T10:45:00.547610+0000","last_unstale":"2023-10-18T10:45:01.824813+0000","last_undegraded":"2023-10-18T10:45:01.824813+0000","last_fullsized":"2023-10-18T10:45:01.824813+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:22.672068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825262+0000","last_change":"2023-10-18T10:45:00.555957+0000","last_active":"2023-10-18T10:45:01.825262+0000","last_peered":"2023-10-18T10:45:01.825262+0000","last_clean":"2023-10-18T10:45:01.825262+0000","last_became_active":"2023-10-18T10:45:00.555715+0000","last_became_peered":"2023-10-18T10:45:00.555715+0000","last_unstale":"2023-10-18T10:45:01.825262+0000","last_undegraded":"2023-10-18T10:45:01.825262+0000","last_fullsized":"2023-10-18T10:45:01.825262+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:07.028997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546229+0000","last_change":"2023-10-18T10:45:00.549570+0000","last_active":"2023-10-18T10:45:01.546229+0000","last_peered":"2023-10-18T10:45:01.546229+0000","last_clean":"2023-10-18T10:45:01.546229+0000","last_became_active":"2023-10-18T10:45:00.549320+0000","last_became_peered":"2023-10-18T10:45:00.549320+0000","last_unstale":"2023-10-18T10:45:01.546229+0000","last_undegraded":"2023-10-18T10:45:01.546229+0000","last_fullsized":"2023-10-18T10:45:01.546229+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:19.566195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547943+0000","last_change":"2023-10-18T10:45:00.561776+0000","last_active":"2023-10-18T10:45:01.547943+0000","last_peered":"2023-10-18T10:45:01.547943+0000","last_clean":"2023-10-18T10:45:01.547943+0000","last_became_active":"2023-10-18T10:45:00.560779+0000","last_became_peered":"2023-10-18T10:45:00.560779+0000","last_unstale":"2023-10-18T10:45:01.547943+0000","last_undegraded":"2023-10-18T10:45:01.547943+0000","last_fullsized":"2023-10-18T10:45:01.547943+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:27:26.311482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546287+0000","last_change":"2023-10-18T10:45:00.549814+0000","last_active":"2023-10-18T10:45:01.546287+0000","last_peered":"2023-10-18T10:45:01.546287+0000","last_clean":"2023-10-18T10:45:01.546287+0000","last_became_active":"2023-10-18T10:45:00.549559+0000","last_became_peered":"2023-10-18T10:45:00.549559+0000","last_unstale":"2023-10-18T10:45:01.546287+0000","last_undegraded":"2023-10-18T10:45:01.546287+0000","last_fullsized":"2023-10-18T10:45:01.546287+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:48.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325802+0000","last_change":"2023-10-18T10:45:00.549693+0000","last_active":"2023-10-18T10:45:02.325802+0000","last_peered":"2023-10-18T10:45:02.325802+0000","last_clean":"2023-10-18T10:45:02.325802+0000","last_became_active":"2023-10-18T10:45:00.549444+0000","last_became_peered":"2023-10-18T10:45:00.549444+0000","last_unstale":"2023-10-18T10:45:02.325802+0000","last_undegraded":"2023-10-18T10:45:02.325802+0000","last_fullsized":"2023-10-18T10:45:02.325802+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:56.971883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325319+0000","last_change":"2023-10-18T10:45:00.547143+0000","last_active":"2023-10-18T10:45:02.325319+0000","last_peered":"2023-10-18T10:45:02.325319+0000","last_clean":"2023-10-18T10:45:02.325319+0000","last_became_active":"2023-10-18T10:45:00.546868+0000","last_became_peered":"2023-10-18T10:45:00.546868+0000","last_unstale":"2023-10-18T10:45:02.325319+0000","last_undegraded":"2023-10-18T10:45:02.325319+0000","last_fullsized":"2023-10-18T10:45:02.325319+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:30.548288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325857+0000","last_change":"2023-10-18T10:45:00.554470+0000","last_active":"2023-10-18T10:45:02.325857+0000","last_peered":"2023-10-18T10:45:02.325857+0000","last_clean":"2023-10-18T10:45:02.325857+0000","last_became_active":"2023-10-18T10:45:00.554221+0000","last_became_peered":"2023-10-18T10:45:00.554221+0000","last_unstale":"2023-10-18T10:45:02.325857+0000","last_undegraded":"2023-10-18T10:45:02.325857+0000","last_fullsized":"2023-10-18T10:45:02.325857+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:47:17.421888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824845+0000","last_change":"2023-10-18T10:45:00.556840+0000","last_active":"2023-10-18T10:45:01.824845+0000","last_peered":"2023-10-18T10:45:01.824845+0000","last_clean":"2023-10-18T10:45:01.824845+0000","last_became_active":"2023-10-18T10:45:00.556338+0000","last_became_peered":"2023-10-18T10:45:00.556338+0000","last_unstale":"2023-10-18T10:45:01.824845+0000","last_undegraded":"2023-10-18T10:45:01.824845+0000","last_fullsized":"2023-10-18T10:45:01.824845+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:59.589703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327115+0000","last_change":"2023-10-18T10:45:00.546465+0000","last_active":"2023-10-18T10:45:02.327115+0000","last_peered":"2023-10-18T10:45:02.327115+0000","last_clean":"2023-10-18T10:45:02.327115+0000","last_became_active":"2023-10-18T10:45:00.544998+0000","last_became_peered":"2023-10-18T10:45:00.544998+0000","last_unstale":"2023-10-18T10:45:02.327115+0000","last_undegraded":"2023-10-18T10:45:02.327115+0000","last_fullsized":"2023-10-18T10:45:02.327115+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:47.887433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326834+0000","last_change":"2023-10-18T10:45:00.555836+0000","last_active":"2023-10-18T10:45:02.326834+0000","last_peered":"2023-10-18T10:45:02.326834+0000","last_clean":"2023-10-18T10:45:02.326834+0000","last_became_active":"2023-10-18T10:45:00.555567+0000","last_became_peered":"2023-10-18T10:45:00.555567+0000","last_unstale":"2023-10-18T10:45:02.326834+0000","last_undegraded":"2023-10-18T10:45:02.326834+0000","last_fullsized":"2023-10-18T10:45:02.326834+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:16.195593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547906+0000","last_change":"2023-10-18T10:45:00.563377+0000","last_active":"2023-10-18T10:45:01.547906+0000","last_peered":"2023-10-18T10:45:01.547906+0000","last_clean":"2023-10-18T10:45:01.547906+0000","last_became_active":"2023-10-18T10:45:00.563093+0000","last_became_peered":"2023-10-18T10:45:00.563093+0000","last_unstale":"2023-10-18T10:45:01.547906+0000","last_undegraded":"2023-10-18T10:45:01.547906+0000","last_fullsized":"2023-10-18T10:45:01.547906+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:50.656392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325533+0000","last_change":"2023-10-18T10:45:00.550887+0000","last_active":"2023-10-18T10:45:02.325533+0000","last_peered":"2023-10-18T10:45:02.325533+0000","last_clean":"2023-10-18T10:45:02.325533+0000","last_became_active":"2023-10-18T10:45:00.550614+0000","last_became_peered":"2023-10-18T10:45:00.550614+0000","last_unstale":"2023-10-18T10:45:02.325533+0000","last_undegraded":"2023-10-18T10:45:02.325533+0000","last_fullsized":"2023-10-18T10:45:02.325533+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:31:59.143400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326344+0000","last_change":"2023-10-18T10:45:00.550674+0000","last_active":"2023-10-18T10:45:02.326344+0000","last_peered":"2023-10-18T10:45:02.326344+0000","last_clean":"2023-10-18T10:45:02.326344+0000","last_became_active":"2023-10-18T10:45:00.550416+0000","last_became_peered":"2023-10-18T10:45:00.550416+0000","last_unstale":"2023-10-18T10:45:02.326344+0000","last_undegraded":"2023-10-18T10:45:02.326344+0000","last_fullsized":"2023-10-18T10:45:02.326344+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:36:21.656622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326521+0000","last_change":"2023-10-18T10:45:00.546279+0000","last_active":"2023-10-18T10:45:02.326521+0000","last_peered":"2023-10-18T10:45:02.326521+0000","last_clean":"2023-10-18T10:45:02.326521+0000","last_became_active":"2023-10-18T10:45:00.545095+0000","last_became_peered":"2023-10-18T10:45:00.545095+0000","last_unstale":"2023-10-18T10:45:02.326521+0000","last_undegraded":"2023-10-18T10:45:02.326521+0000","last_fullsized":"2023-10-18T10:45:02.326521+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:46:46.309099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325952+0000","last_change":"2023-10-18T10:45:00.549973+0000","last_active":"2023-10-18T10:45:02.325952+0000","last_peered":"2023-10-18T10:45:02.325952+0000","last_clean":"2023-10-18T10:45:02.325952+0000","last_became_active":"2023-10-18T10:45:00.549717+0000","last_became_peered":"2023-10-18T10:45:00.549717+0000","last_unstale":"2023-10-18T10:45:02.325952+0000","last_undegraded":"2023-10-18T10:45:02.325952+0000","last_fullsized":"2023-10-18T10:45:02.325952+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:36:21.656622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325604+0000","last_change":"2023-10-18T10:45:00.548793+0000","last_active":"2023-10-18T10:45:02.325604+0000","last_peered":"2023-10-18T10:45:02.325604+0000","last_clean":"2023-10-18T10:45:02.325604+0000","last_became_active":"2023-10-18T10:45:00.548548+0000","last_became_peered":"2023-10-18T10:45:00.548548+0000","last_unstale":"2023-10-18T10:45:02.325604+0000","last_undegraded":"2023-10-18T10:45:02.325604+0000","last_fullsized":"2023-10-18T10:45:02.325604+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:21.744298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325864+0000","last_change":"2023-10-18T10:45:00.548611+0000","last_active":"2023-10-18T10:45:02.325864+0000","last_peered":"2023-10-18T10:45:02.325864+0000","last_clean":"2023-10-18T10:45:02.325864+0000","last_became_active":"2023-10-18T10:45:00.548351+0000","last_became_peered":"2023-10-18T10:45:00.548351+0000","last_unstale":"2023-10-18T10:45:02.325864+0000","last_undegraded":"2023-10-18T10:45:02.325864+0000","last_fullsized":"2023-10-18T10:45:02.325864+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:04.768446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"155'4","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327197+0000","last_change":"2023-10-18T10:44:59.546005+0000","last_active":"2023-10-18T10:45:02.327197+0000","last_peered":"2023-10-18T10:45:02.327197+0000","last_clean":"2023-10-18T10:45:02.327197+0000","last_became_active":"2023-10-18T10:44:59.545741+0000","last_became_peered":"2023-10-18T10:44:59.545741+0000","last_unstale":"2023-10-18T10:45:02.327197+0000","last_undegraded":"2023-10-18T10:45:02.327197+0000","last_fullsized":"2023-10-18T10:45:02.327197+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:59:17.293990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325326+0000","last_change":"2023-10-18T10:45:00.546398+0000","last_active":"2023-10-18T10:45:02.325326+0000","last_peered":"2023-10-18T10:45:02.325326+0000","last_clean":"2023-10-18T10:45:02.325326+0000","last_became_active":"2023-10-18T10:45:00.546122+0000","last_became_peered":"2023-10-18T10:45:00.546122+0000","last_unstale":"2023-10-18T10:45:02.325326+0000","last_undegraded":"2023-10-18T10:45:02.325326+0000","last_fullsized":"2023-10-18T10:45:02.325326+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:45:39.976465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546472+0000","last_change":"2023-10-18T10:44:59.549443+0000","last_active":"2023-10-18T10:45:01.546472+0000","last_peered":"2023-10-18T10:45:01.546472+0000","last_clean":"2023-10-18T10:45:01.546472+0000","last_became_active":"2023-10-18T10:44:59.549114+0000","last_became_peered":"2023-10-18T10:44:59.549114+0000","last_unstale":"2023-10-18T10:45:01.546472+0000","last_undegraded":"2023-10-18T10:45:01.546472+0000","last_fullsized":"2023-10-18T10:45:01.546472+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:52.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325840+0000","last_change":"2023-10-18T10:45:00.553749+0000","last_active":"2023-10-18T10:45:02.325840+0000","last_peered":"2023-10-18T10:45:02.325840+0000","last_clean":"2023-10-18T10:45:02.325840+0000","last_became_active":"2023-10-18T10:45:00.553477+0000","last_became_peered":"2023-10-18T10:45:00.553477+0000","last_unstale":"2023-10-18T10:45:02.325840+0000","last_undegraded":"2023-10-18T10:45:02.325840+0000","last_fullsized":"2023-10-18T10:45:02.325840+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:10.899448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.548016+0000","last_change":"2023-10-18T10:44:59.551861+0000","last_active":"2023-10-18T10:45:01.548016+0000","last_peered":"2023-10-18T10:45:01.548016+0000","last_clean":"2023-10-18T10:45:01.548016+0000","last_became_active":"2023-10-18T10:44:59.551597+0000","last_became_peered":"2023-10-18T10:44:59.551597+0000","last_unstale":"2023-10-18T10:45:01.548016+0000","last_undegraded":"2023-10-18T10:45:01.548016+0000","last_fullsized":"2023-10-18T10:45:01.548016+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:52.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325397+0000","last_change":"2023-10-18T10:45:00.554016+0000","last_active":"2023-10-18T10:45:02.325397+0000","last_peered":"2023-10-18T10:45:02.325397+0000","last_clean":"2023-10-18T10:45:02.325397+0000","last_became_active":"2023-10-18T10:45:00.553751+0000","last_became_peered":"2023-10-18T10:45:00.553751+0000","last_unstale":"2023-10-18T10:45:02.325397+0000","last_undegraded":"2023-10-18T10:45:02.325397+0000","last_fullsized":"2023-10-18T10:45:02.325397+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:09:55.073866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547756+0000","last_change":"2023-10-18T10:44:59.551206+0000","last_active":"2023-10-18T10:45:01.547756+0000","last_peered":"2023-10-18T10:45:01.547756+0000","last_clean":"2023-10-18T10:45:01.547756+0000","last_became_active":"2023-10-18T10:44:59.550813+0000","last_became_peered":"2023-10-18T10:44:59.550813+0000","last_unstale":"2023-10-18T10:45:01.547756+0000","last_undegraded":"2023-10-18T10:45:01.547756+0000","last_fullsized":"2023-10-18T10:45:01.547756+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:09.895689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327152+0000","last_change":"2023-10-18T10:45:00.551266+0000","last_active":"2023-10-18T10:45:02.327152+0000","last_peered":"2023-10-18T10:45:02.327152+0000","last_clean":"2023-10-18T10:45:02.327152+0000","last_became_active":"2023-10-18T10:45:00.551011+0000","last_became_peered":"2023-10-18T10:45:00.551011+0000","last_unstale":"2023-10-18T10:45:02.327152+0000","last_undegraded":"2023-10-18T10:45:02.327152+0000","last_fullsized":"2023-10-18T10:45:02.327152+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:51.890773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326018+0000","last_change":"2023-10-18T10:44:59.544948+0000","last_active":"2023-10-18T10:45:02.326018+0000","last_peered":"2023-10-18T10:45:02.326018+0000","last_clean":"2023-10-18T10:45:02.326018+0000","last_became_active":"2023-10-18T10:44:59.544685+0000","last_became_peered":"2023-10-18T10:44:59.544685+0000","last_unstale":"2023-10-18T10:45:02.326018+0000","last_undegraded":"2023-10-18T10:45:02.326018+0000","last_fullsized":"2023-10-18T10:45:02.326018+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:37:08.444269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326696+0000","last_change":"2023-10-18T10:45:00.557517+0000","last_active":"2023-10-18T10:45:02.326696+0000","last_peered":"2023-10-18T10:45:02.326696+0000","last_clean":"2023-10-18T10:45:02.326696+0000","last_became_active":"2023-10-18T10:45:00.557258+0000","last_became_peered":"2023-10-18T10:45:00.557258+0000","last_unstale":"2023-10-18T10:45:02.326696+0000","last_undegraded":"2023-10-18T10:45:02.326696+0000","last_fullsized":"2023-10-18T10:45:02.326696+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:39.894454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546090+0000","last_change":"2023-10-18T10:44:59.545459+0000","last_active":"2023-10-18T10:45:01.546090+0000","last_peered":"2023-10-18T10:45:01.546090+0000","last_clean":"2023-10-18T10:45:01.546090+0000","last_became_active":"2023-10-18T10:44:59.545188+0000","last_became_peered":"2023-10-18T10:44:59.545188+0000","last_unstale":"2023-10-18T10:45:01.546090+0000","last_undegraded":"2023-10-18T10:45:01.546090+0000","last_fullsized":"2023-10-18T10:45:01.546090+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:43.900519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546376+0000","last_change":"2023-10-18T10:45:00.558273+0000","last_active":"2023-10-18T10:45:01.546376+0000","last_peered":"2023-10-18T10:45:01.546376+0000","last_clean":"2023-10-18T10:45:01.546376+0000","last_became_active":"2023-10-18T10:45:00.557995+0000","last_became_peered":"2023-10-18T10:45:00.557995+0000","last_unstale":"2023-10-18T10:45:01.546376+0000","last_undegraded":"2023-10-18T10:45:01.546376+0000","last_fullsized":"2023-10-18T10:45:01.546376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:27.070435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824797+0000","last_change":"2023-10-18T10:44:59.551285+0000","last_active":"2023-10-18T10:45:01.824797+0000","last_peered":"2023-10-18T10:45:01.824797+0000","last_clean":"2023-10-18T10:45:01.824797+0000","last_became_active":"2023-10-18T10:44:59.550971+0000","last_became_peered":"2023-10-18T10:44:59.550971+0000","last_unstale":"2023-10-18T10:45:01.824797+0000","last_undegraded":"2023-10-18T10:45:01.824797+0000","last_fullsized":"2023-10-18T10:45:01.824797+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:34.997051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547576+0000","last_change":"2023-10-18T10:45:00.561634+0000","last_active":"2023-10-18T10:45:01.547576+0000","last_peered":"2023-10-18T10:45:01.547576+0000","last_clean":"2023-10-18T10:45:01.547576+0000","last_became_active":"2023-10-18T10:45:00.561382+0000","last_became_peered":"2023-10-18T10:45:00.561382+0000","last_unstale":"2023-10-18T10:45:01.547576+0000","last_undegraded":"2023-10-18T10:45:01.547576+0000","last_fullsized":"2023-10-18T10:45:01.547576+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:19.566195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326639+0000","last_change":"2023-10-18T10:44:59.554031+0000","last_active":"2023-10-18T10:45:02.326639+0000","last_peered":"2023-10-18T10:45:02.326639+0000","last_clean":"2023-10-18T10:45:02.326639+0000","last_became_active":"2023-10-18T10:44:59.553823+0000","last_became_peered":"2023-10-18T10:44:59.553823+0000","last_unstale":"2023-10-18T10:45:02.326639+0000","last_undegraded":"2023-10-18T10:45:02.326639+0000","last_fullsized":"2023-10-18T10:45:02.326639+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:29.312474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325230+0000","last_change":"2023-10-18T10:45:00.547832+0000","last_active":"2023-10-18T10:45:02.325230+0000","last_peered":"2023-10-18T10:45:02.325230+0000","last_clean":"2023-10-18T10:45:02.325230+0000","last_became_active":"2023-10-18T10:45:00.547509+0000","last_became_peered":"2023-10-18T10:45:00.547509+0000","last_unstale":"2023-10-18T10:45:02.325230+0000","last_undegraded":"2023-10-18T10:45:02.325230+0000","last_fullsized":"2023-10-18T10:45:02.325230+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:54.915478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327037+0000","last_change":"2023-10-18T10:44:59.548871+0000","last_active":"2023-10-18T10:45:02.327037+0000","last_peered":"2023-10-18T10:45:02.327037+0000","last_clean":"2023-10-18T10:45:02.327037+0000","last_became_active":"2023-10-18T10:44:59.548587+0000","last_became_peered":"2023-10-18T10:44:59.548587+0000","last_unstale":"2023-10-18T10:45:02.327037+0000","last_undegraded":"2023-10-18T10:45:02.327037+0000","last_fullsized":"2023-10-18T10:45:02.327037+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:50.205402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824921+0000","last_change":"2023-10-18T10:45:00.553629+0000","last_active":"2023-10-18T10:45:01.824921+0000","last_peered":"2023-10-18T10:45:01.824921+0000","last_clean":"2023-10-18T10:45:01.824921+0000","last_became_active":"2023-10-18T10:45:00.553350+0000","last_became_peered":"2023-10-18T10:45:00.553350+0000","last_unstale":"2023-10-18T10:45:01.824921+0000","last_undegraded":"2023-10-18T10:45:01.824921+0000","last_fullsized":"2023-10-18T10:45:01.824921+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:51.311062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325703+0000","last_change":"2023-10-18T10:44:59.546697+0000","last_active":"2023-10-18T10:45:02.325703+0000","last_peered":"2023-10-18T10:45:02.325703+0000","last_clean":"2023-10-18T10:45:02.325703+0000","last_became_active":"2023-10-18T10:44:59.546384+0000","last_became_peered":"2023-10-18T10:44:59.546384+0000","last_unstale":"2023-10-18T10:45:02.325703+0000","last_undegraded":"2023-10-18T10:45:02.325703+0000","last_fullsized":"2023-10-18T10:45:02.325703+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:27.802004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326462+0000","last_change":"2023-10-18T10:45:00.540375+0000","last_active":"2023-10-18T10:45:02.326462+0000","last_peered":"2023-10-18T10:45:02.326462+0000","last_clean":"2023-10-18T10:45:02.326462+0000","last_became_active":"2023-10-18T10:45:00.540112+0000","last_became_peered":"2023-10-18T10:45:00.540112+0000","last_unstale":"2023-10-18T10:45:02.326462+0000","last_undegraded":"2023-10-18T10:45:02.326462+0000","last_fullsized":"2023-10-18T10:45:02.326462+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:39.894454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"155'3","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.548042+0000","last_change":"2023-10-18T10:44:59.548751+0000","last_active":"2023-10-18T10:45:01.548042+0000","last_peered":"2023-10-18T10:45:01.548042+0000","last_clean":"2023-10-18T10:45:01.548042+0000","last_became_active":"2023-10-18T10:44:59.548474+0000","last_became_peered":"2023-10-18T10:44:59.548474+0000","last_unstale":"2023-10-18T10:45:01.548042+0000","last_undegraded":"2023-10-18T10:45:01.548042+0000","last_fullsized":"2023-10-18T10:45:01.548042+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:37.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327074+0000","last_change":"2023-10-18T10:45:00.549775+0000","last_active":"2023-10-18T10:45:02.327074+0000","last_peered":"2023-10-18T10:45:02.327074+0000","last_clean":"2023-10-18T10:45:02.327074+0000","last_became_active":"2023-10-18T10:45:00.549331+0000","last_became_peered":"2023-10-18T10:45:00.549331+0000","last_unstale":"2023-10-18T10:45:02.327074+0000","last_undegraded":"2023-10-18T10:45:02.327074+0000","last_fullsized":"2023-10-18T10:45:02.327074+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:51.261564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"155'6","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547724+0000","last_change":"2023-10-18T10:44:59.553324+0000","last_active":"2023-10-18T10:45:01.547724+0000","last_peered":"2023-10-18T10:45:01.547724+0000","last_clean":"2023-10-18T10:45:01.547724+0000","last_became_active":"2023-10-18T10:44:59.553025+0000","last_became_peered":"2023-10-18T10:44:59.553025+0000","last_unstale":"2023-10-18T10:45:01.547724+0000","last_undegraded":"2023-10-18T10:45:01.547724+0000","last_fullsized":"2023-10-18T10:45:01.547724+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:47:16.418129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325662+0000","last_change":"2023-10-18T10:45:00.545940+0000","last_active":"2023-10-18T10:45:02.325662+0000","last_peered":"2023-10-18T10:45:02.325662+0000","last_clean":"2023-10-18T10:45:02.325662+0000","last_became_active":"2023-10-18T10:45:00.545680+0000","last_became_peered":"2023-10-18T10:45:00.545680+0000","last_unstale":"2023-10-18T10:45:02.325662+0000","last_undegraded":"2023-10-18T10:45:02.325662+0000","last_fullsized":"2023-10-18T10:45:02.325662+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:48.726330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325494+0000","last_change":"2023-10-18T10:44:59.543508+0000","last_active":"2023-10-18T10:45:02.325494+0000","last_peered":"2023-10-18T10:45:02.325494+0000","last_clean":"2023-10-18T10:45:02.325494+0000","last_became_active":"2023-10-18T10:44:59.543237+0000","last_became_peered":"2023-10-18T10:44:59.543237+0000","last_unstale":"2023-10-18T10:45:02.325494+0000","last_undegraded":"2023-10-18T10:45:02.325494+0000","last_fullsized":"2023-10-18T10:45:02.325494+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:59.696128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547856+0000","last_change":"2023-10-18T10:45:00.556708+0000","last_active":"2023-10-18T10:45:01.547856+0000","last_peered":"2023-10-18T10:45:01.547856+0000","last_clean":"2023-10-18T10:45:01.547856+0000","last_became_active":"2023-10-18T10:45:00.556432+0000","last_became_peered":"2023-10-18T10:45:00.556432+0000","last_unstale":"2023-10-18T10:45:01.547856+0000","last_undegraded":"2023-10-18T10:45:01.547856+0000","last_fullsized":"2023-10-18T10:45:01.547856+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:05:13.997775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326393+0000","last_change":"2023-10-18T10:44:59.548866+0000","last_active":"2023-10-18T10:45:02.326393+0000","last_peered":"2023-10-18T10:45:02.326393+0000","last_clean":"2023-10-18T10:45:02.326393+0000","last_became_active":"2023-10-18T10:44:59.548228+0000","last_became_peered":"2023-10-18T10:44:59.548228+0000","last_unstale":"2023-10-18T10:45:02.326393+0000","last_undegraded":"2023-10-18T10:45:02.326393+0000","last_fullsized":"2023-10-18T10:45:02.326393+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:06:11.225137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546589+0000","last_change":"2023-10-18T10:45:00.548775+0000","last_active":"2023-10-18T10:45:01.546589+0000","last_peered":"2023-10-18T10:45:01.546589+0000","last_clean":"2023-10-18T10:45:01.546589+0000","last_became_active":"2023-10-18T10:45:00.548532+0000","last_became_peered":"2023-10-18T10:45:00.548532+0000","last_unstale":"2023-10-18T10:45:01.546589+0000","last_undegraded":"2023-10-18T10:45:01.546589+0000","last_fullsized":"2023-10-18T10:45:01.546589+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:42.529530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"155'8","reported_seq":20,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824912+0000","last_change":"2023-10-18T10:44:59.547619+0000","last_active":"2023-10-18T10:45:01.824912+0000","last_peered":"2023-10-18T10:45:01.824912+0000","last_clean":"2023-10-18T10:45:01.824912+0000","last_became_active":"2023-10-18T10:44:59.547301+0000","last_became_peered":"2023-10-18T10:44:59.547301+0000","last_unstale":"2023-10-18T10:45:01.824912+0000","last_undegraded":"2023-10-18T10:45:01.824912+0000","last_fullsized":"2023-10-18T10:45:01.824912+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:03.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326562+0000","last_change":"2023-10-18T10:45:00.546039+0000","last_active":"2023-10-18T10:45:02.326562+0000","last_peered":"2023-10-18T10:45:02.326562+0000","last_clean":"2023-10-18T10:45:02.326562+0000","last_became_active":"2023-10-18T10:45:00.545794+0000","last_became_peered":"2023-10-18T10:45:00.545794+0000","last_unstale":"2023-10-18T10:45:02.326562+0000","last_undegraded":"2023-10-18T10:45:02.326562+0000","last_fullsized":"2023-10-18T10:45:02.326562+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:51.890773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327161+0000","last_change":"2023-10-18T10:44:59.547983+0000","last_active":"2023-10-18T10:45:02.327161+0000","last_peered":"2023-10-18T10:45:02.327161+0000","last_clean":"2023-10-18T10:45:02.327161+0000","last_became_active":"2023-10-18T10:44:59.547638+0000","last_became_peered":"2023-10-18T10:44:59.547638+0000","last_unstale":"2023-10-18T10:45:02.327161+0000","last_undegraded":"2023-10-18T10:45:02.327161+0000","last_fullsized":"2023-10-18T10:45:02.327161+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:41:04.836469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547926+0000","last_change":"2023-10-18T10:45:00.558723+0000","last_active":"2023-10-18T10:45:01.547926+0000","last_peered":"2023-10-18T10:45:01.547926+0000","last_clean":"2023-10-18T10:45:01.547926+0000","last_became_active":"2023-10-18T10:45:00.558459+0000","last_became_peered":"2023-10-18T10:45:00.558459+0000","last_unstale":"2023-10-18T10:45:01.547926+0000","last_undegraded":"2023-10-18T10:45:01.547926+0000","last_fullsized":"2023-10-18T10:45:01.547926+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:42.529530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326919+0000","last_change":"2023-10-18T10:44:59.546867+0000","last_active":"2023-10-18T10:45:02.326919+0000","last_peered":"2023-10-18T10:45:02.326919+0000","last_clean":"2023-10-18T10:45:02.326919+0000","last_became_active":"2023-10-18T10:44:59.546598+0000","last_became_peered":"2023-10-18T10:44:59.546598+0000","last_unstale":"2023-10-18T10:45:02.326919+0000","last_undegraded":"2023-10-18T10:45:02.326919+0000","last_fullsized":"2023-10-18T10:45:02.326919+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:55.189311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547885+0000","last_change":"2023-10-18T10:45:00.562225+0000","last_active":"2023-10-18T10:45:01.547885+0000","last_peered":"2023-10-18T10:45:01.547885+0000","last_clean":"2023-10-18T10:45:01.547885+0000","last_became_active":"2023-10-18T10:45:00.561976+0000","last_became_peered":"2023-10-18T10:45:00.561976+0000","last_unstale":"2023-10-18T10:45:01.547885+0000","last_undegraded":"2023-10-18T10:45:01.547885+0000","last_fullsized":"2023-10-18T10:45:01.547885+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:27.070435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"155'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825094+0000","last_change":"2023-10-18T10:44:59.549433+0000","last_active":"2023-10-18T10:45:01.825094+0000","last_peered":"2023-10-18T10:45:01.825094+0000","last_clean":"2023-10-18T10:45:01.825094+0000","last_became_active":"2023-10-18T10:44:59.549180+0000","last_became_peered":"2023-10-18T10:44:59.549180+0000","last_unstale":"2023-10-18T10:45:01.825094+0000","last_undegraded":"2023-10-18T10:45:01.825094+0000","last_fullsized":"2023-10-18T10:45:01.825094+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:06.028167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325844+0000","last_change":"2023-10-18T10:45:00.547164+0000","last_active":"2023-10-18T10:45:02.325844+0000","last_peered":"2023-10-18T10:45:02.325844+0000","last_clean":"2023-10-18T10:45:02.325844+0000","last_became_active":"2023-10-18T10:45:00.546926+0000","last_became_peered":"2023-10-18T10:45:00.546926+0000","last_unstale":"2023-10-18T10:45:02.325844+0000","last_undegraded":"2023-10-18T10:45:02.325844+0000","last_fullsized":"2023-10-18T10:45:02.325844+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:48.531003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546363+0000","last_change":"2023-10-18T10:44:59.552659+0000","last_active":"2023-10-18T10:45:01.546363+0000","last_peered":"2023-10-18T10:45:01.546363+0000","last_clean":"2023-10-18T10:45:01.546363+0000","last_became_active":"2023-10-18T10:44:59.552403+0000","last_became_peered":"2023-10-18T10:44:59.552403+0000","last_unstale":"2023-10-18T10:45:01.546363+0000","last_undegraded":"2023-10-18T10:45:01.546363+0000","last_fullsized":"2023-10-18T10:45:01.546363+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:56.316476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546365+0000","last_change":"2023-10-18T10:45:00.542789+0000","last_active":"2023-10-18T10:45:01.546365+0000","last_peered":"2023-10-18T10:45:01.546365+0000","last_clean":"2023-10-18T10:45:01.546365+0000","last_became_active":"2023-10-18T10:45:00.542558+0000","last_became_peered":"2023-10-18T10:45:00.542558+0000","last_unstale":"2023-10-18T10:45:01.546365+0000","last_undegraded":"2023-10-18T10:45:01.546365+0000","last_fullsized":"2023-10-18T10:45:01.546365+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:05:13.997775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326479+0000","last_change":"2023-10-18T10:44:59.544421+0000","last_active":"2023-10-18T10:45:02.326479+0000","last_peered":"2023-10-18T10:45:02.326479+0000","last_clean":"2023-10-18T10:45:02.326479+0000","last_became_active":"2023-10-18T10:44:59.544159+0000","last_became_peered":"2023-10-18T10:44:59.544159+0000","last_unstale":"2023-10-18T10:45:02.326479+0000","last_undegraded":"2023-10-18T10:45:02.326479+0000","last_fullsized":"2023-10-18T10:45:02.326479+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:44:29.312474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325739+0000","last_change":"2023-10-18T10:45:00.556582+0000","last_active":"2023-10-18T10:45:02.325739+0000","last_peered":"2023-10-18T10:45:02.325739+0000","last_clean":"2023-10-18T10:45:02.325739+0000","last_became_active":"2023-10-18T10:45:00.556335+0000","last_became_peered":"2023-10-18T10:45:00.556335+0000","last_unstale":"2023-10-18T10:45:02.325739+0000","last_undegraded":"2023-10-18T10:45:02.325739+0000","last_fullsized":"2023-10-18T10:45:02.325739+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:07.811816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325537+0000","last_change":"2023-10-18T10:44:59.540315+0000","last_active":"2023-10-18T10:45:02.325537+0000","last_peered":"2023-10-18T10:45:02.325537+0000","last_clean":"2023-10-18T10:45:02.325537+0000","last_became_active":"2023-10-18T10:44:59.540036+0000","last_became_peered":"2023-10-18T10:44:59.540036+0000","last_unstale":"2023-10-18T10:45:02.325537+0000","last_undegraded":"2023-10-18T10:45:02.325537+0000","last_fullsized":"2023-10-18T10:45:02.325537+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:29:49.439257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546500+0000","last_change":"2023-10-18T10:45:00.548145+0000","last_active":"2023-10-18T10:45:01.546500+0000","last_peered":"2023-10-18T10:45:01.546500+0000","last_clean":"2023-10-18T10:45:01.546500+0000","last_became_active":"2023-10-18T10:45:00.547873+0000","last_became_peered":"2023-10-18T10:45:00.547873+0000","last_unstale":"2023-10-18T10:45:01.546500+0000","last_undegraded":"2023-10-18T10:45:01.546500+0000","last_fullsized":"2023-10-18T10:45:01.546500+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:51:50.656392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547694+0000","last_change":"2023-10-18T10:44:59.548633+0000","last_active":"2023-10-18T10:45:01.547694+0000","last_peered":"2023-10-18T10:45:01.547694+0000","last_clean":"2023-10-18T10:45:01.547694+0000","last_became_active":"2023-10-18T10:44:59.548345+0000","last_became_peered":"2023-10-18T10:44:59.548345+0000","last_unstale":"2023-10-18T10:45:01.547694+0000","last_undegraded":"2023-10-18T10:45:01.547694+0000","last_fullsized":"2023-10-18T10:45:01.547694+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:42:01.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326896+0000","last_change":"2023-10-18T10:45:00.545565+0000","last_active":"2023-10-18T10:45:02.326896+0000","last_peered":"2023-10-18T10:45:02.326896+0000","last_clean":"2023-10-18T10:45:02.326896+0000","last_became_active":"2023-10-18T10:45:00.545316+0000","last_became_peered":"2023-10-18T10:45:00.545316+0000","last_unstale":"2023-10-18T10:45:02.326896+0000","last_undegraded":"2023-10-18T10:45:02.326896+0000","last_fullsized":"2023-10-18T10:45:02.326896+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:04.326525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825124+0000","last_change":"2023-10-18T10:44:59.552577+0000","last_active":"2023-10-18T10:45:01.825124+0000","last_peered":"2023-10-18T10:45:01.825124+0000","last_clean":"2023-10-18T10:45:01.825124+0000","last_became_active":"2023-10-18T10:44:59.552329+0000","last_became_peered":"2023-10-18T10:44:59.552329+0000","last_unstale":"2023-10-18T10:45:01.825124+0000","last_undegraded":"2023-10-18T10:45:01.825124+0000","last_fullsized":"2023-10-18T10:45:01.825124+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:49.591451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546161+0000","last_change":"2023-10-18T10:45:00.548502+0000","last_active":"2023-10-18T10:45:01.546161+0000","last_peered":"2023-10-18T10:45:01.546161+0000","last_clean":"2023-10-18T10:45:01.546161+0000","last_became_active":"2023-10-18T10:45:00.548256+0000","last_became_peered":"2023-10-18T10:45:00.548256+0000","last_unstale":"2023-10-18T10:45:01.546161+0000","last_undegraded":"2023-10-18T10:45:01.546161+0000","last_fullsized":"2023-10-18T10:45:01.546161+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:02.628870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"155'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325868+0000","last_change":"2023-10-18T10:44:59.547011+0000","last_active":"2023-10-18T10:45:02.325868+0000","last_peered":"2023-10-18T10:45:02.325868+0000","last_clean":"2023-10-18T10:45:02.325868+0000","last_became_active":"2023-10-18T10:44:59.546769+0000","last_became_peered":"2023-10-18T10:44:59.546769+0000","last_unstale":"2023-10-18T10:45:02.325868+0000","last_undegraded":"2023-10-18T10:45:02.325868+0000","last_fullsized":"2023-10-18T10:45:02.325868+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:22.399832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326305+0000","last_change":"2023-10-18T10:45:00.542241+0000","last_active":"2023-10-18T10:45:02.326305+0000","last_peered":"2023-10-18T10:45:02.326305+0000","last_clean":"2023-10-18T10:45:02.326305+0000","last_became_active":"2023-10-18T10:45:00.541962+0000","last_became_peered":"2023-10-18T10:45:00.541962+0000","last_unstale":"2023-10-18T10:45:02.326305+0000","last_undegraded":"2023-10-18T10:45:02.326305+0000","last_fullsized":"2023-10-18T10:45:02.326305+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:04.768446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.327275+0000","last_change":"2023-10-18T10:44:59.542014+0000","last_active":"2023-10-18T10:45:02.327275+0000","last_peered":"2023-10-18T10:45:02.327275+0000","last_clean":"2023-10-18T10:45:02.327275+0000","last_became_active":"2023-10-18T10:44:59.541731+0000","last_became_peered":"2023-10-18T10:44:59.541731+0000","last_unstale":"2023-10-18T10:45:02.327275+0000","last_undegraded":"2023-10-18T10:45:02.327275+0000","last_fullsized":"2023-10-18T10:45:02.327275+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:55.189311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547400+0000","last_change":"2023-10-18T10:45:00.558921+0000","last_active":"2023-10-18T10:45:01.547400+0000","last_peered":"2023-10-18T10:45:01.547400+0000","last_clean":"2023-10-18T10:45:01.547400+0000","last_became_active":"2023-10-18T10:45:00.558631+0000","last_became_peered":"2023-10-18T10:45:00.558631+0000","last_unstale":"2023-10-18T10:45:01.547400+0000","last_undegraded":"2023-10-18T10:45:01.547400+0000","last_fullsized":"2023-10-18T10:45:01.547400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:02.628870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326782+0000","last_change":"2023-10-18T10:44:59.541045+0000","last_active":"2023-10-18T10:45:02.326782+0000","last_peered":"2023-10-18T10:45:02.326782+0000","last_clean":"2023-10-18T10:45:02.326782+0000","last_became_active":"2023-10-18T10:44:59.540792+0000","last_became_peered":"2023-10-18T10:44:59.540792+0000","last_unstale":"2023-10-18T10:45:02.326782+0000","last_undegraded":"2023-10-18T10:45:02.326782+0000","last_fullsized":"2023-10-18T10:45:02.326782+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:04.237888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.325937+0000","last_change":"2023-10-18T10:45:00.549757+0000","last_active":"2023-10-18T10:45:02.325937+0000","last_peered":"2023-10-18T10:45:02.325937+0000","last_clean":"2023-10-18T10:45:02.325937+0000","last_became_active":"2023-10-18T10:45:00.548892+0000","last_became_peered":"2023-10-18T10:45:00.548892+0000","last_unstale":"2023-10-18T10:45:02.325937+0000","last_undegraded":"2023-10-18T10:45:02.325937+0000","last_fullsized":"2023-10-18T10:45:02.325937+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:28:30.569831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825052+0000","last_change":"2023-10-18T10:44:59.551117+0000","last_active":"2023-10-18T10:45:01.825052+0000","last_peered":"2023-10-18T10:45:01.825052+0000","last_clean":"2023-10-18T10:45:01.825052+0000","last_became_active":"2023-10-18T10:44:59.550799+0000","last_became_peered":"2023-10-18T10:44:59.550799+0000","last_unstale":"2023-10-18T10:45:01.825052+0000","last_undegraded":"2023-10-18T10:45:01.825052+0000","last_fullsized":"2023-10-18T10:45:01.825052+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:39.812951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825083+0000","last_change":"2023-10-18T10:45:00.551092+0000","last_active":"2023-10-18T10:45:01.825083+0000","last_peered":"2023-10-18T10:45:01.825083+0000","last_clean":"2023-10-18T10:45:01.825083+0000","last_became_active":"2023-10-18T10:45:00.550827+0000","last_became_peered":"2023-10-18T10:45:00.550827+0000","last_unstale":"2023-10-18T10:45:01.825083+0000","last_undegraded":"2023-10-18T10:45:01.825083+0000","last_fullsized":"2023-10-18T10:45:01.825083+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:04.833412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326306+0000","last_change":"2023-10-18T10:44:59.544415+0000","last_active":"2023-10-18T10:45:02.326306+0000","last_peered":"2023-10-18T10:45:02.326306+0000","last_clean":"2023-10-18T10:45:02.326306+0000","last_became_active":"2023-10-18T10:44:59.543968+0000","last_became_peered":"2023-10-18T10:44:59.543968+0000","last_unstale":"2023-10-18T10:45:02.326306+0000","last_undegraded":"2023-10-18T10:45:02.326306+0000","last_fullsized":"2023-10-18T10:45:02.326306+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:00.870187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825088+0000","last_change":"2023-10-18T10:45:00.550486+0000","last_active":"2023-10-18T10:45:01.825088+0000","last_peered":"2023-10-18T10:45:01.825088+0000","last_clean":"2023-10-18T10:45:01.825088+0000","last_became_active":"2023-10-18T10:45:00.550239+0000","last_became_peered":"2023-10-18T10:45:00.550239+0000","last_unstale":"2023-10-18T10:45:01.825088+0000","last_undegraded":"2023-10-18T10:45:01.825088+0000","last_fullsized":"2023-10-18T10:45:01.825088+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:13.442928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547672+0000","last_change":"2023-10-18T10:44:59.551190+0000","last_active":"2023-10-18T10:45:01.547672+0000","last_peered":"2023-10-18T10:45:01.547672+0000","last_clean":"2023-10-18T10:45:01.547672+0000","last_became_active":"2023-10-18T10:44:59.550893+0000","last_became_peered":"2023-10-18T10:44:59.550893+0000","last_unstale":"2023-10-18T10:45:01.547672+0000","last_undegraded":"2023-10-18T10:45:01.547672+0000","last_fullsized":"2023-10-18T10:45:01.547672+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:56.251510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.825179+0000","last_change":"2023-10-18T10:45:00.554546+0000","last_active":"2023-10-18T10:45:01.825179+0000","last_peered":"2023-10-18T10:45:01.825179+0000","last_clean":"2023-10-18T10:45:01.825179+0000","last_became_active":"2023-10-18T10:45:00.554296+0000","last_became_peered":"2023-10-18T10:45:00.554296+0000","last_unstale":"2023-10-18T10:45:01.825179+0000","last_undegraded":"2023-10-18T10:45:01.825179+0000","last_fullsized":"2023-10-18T10:45:01.825179+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:45.203109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.824916+0000","last_change":"2023-10-18T10:44:59.551559+0000","last_active":"2023-10-18T10:45:01.824916+0000","last_peered":"2023-10-18T10:45:01.824916+0000","last_clean":"2023-10-18T10:45:01.824916+0000","last_became_active":"2023-10-18T10:44:59.551296+0000","last_became_peered":"2023-10-18T10:44:59.551296+0000","last_unstale":"2023-10-18T10:45:01.824916+0000","last_undegraded":"2023-10-18T10:45:01.824916+0000","last_fullsized":"2023-10-18T10:45:01.824916+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:34:31.261305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547848+0000","last_change":"2023-10-18T10:45:00.556380+0000","last_active":"2023-10-18T10:45:01.547848+0000","last_peered":"2023-10-18T10:45:01.547848+0000","last_clean":"2023-10-18T10:45:01.547848+0000","last_became_active":"2023-10-18T10:45:00.556121+0000","last_became_peered":"2023-10-18T10:45:00.556121+0000","last_unstale":"2023-10-18T10:45:01.547848+0000","last_undegraded":"2023-10-18T10:45:01.547848+0000","last_fullsized":"2023-10-18T10:45:01.547848+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:48.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.546246+0000","last_change":"2023-10-18T10:44:59.552570+0000","last_active":"2023-10-18T10:45:01.546246+0000","last_peered":"2023-10-18T10:45:01.546246+0000","last_clean":"2023-10-18T10:45:01.546246+0000","last_became_active":"2023-10-18T10:44:59.552313+0000","last_became_peered":"2023-10-18T10:44:59.552313+0000","last_unstale":"2023-10-18T10:45:01.546246+0000","last_undegraded":"2023-10-18T10:45:01.546246+0000","last_fullsized":"2023-10-18T10:45:01.546246+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:37.839306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:02.326963+0000","last_change":"2023-10-18T10:45:00.543609+0000","last_active":"2023-10-18T10:45:02.326963+0000","last_peered":"2023-10-18T10:45:02.326963+0000","last_clean":"2023-10-18T10:45:02.326963+0000","last_became_active":"2023-10-18T10:45:00.542794+0000","last_became_peered":"2023-10-18T10:45:00.542794+0000","last_unstale":"2023-10-18T10:45:02.326963+0000","last_undegraded":"2023-10-18T10:45:02.326963+0000","last_fullsized":"2023-10-18T10:45:02.326963+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:59.524757+0000","last_clean_scrub_stamp":"2023-10-18T10:44:59.524757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:55:21.772344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-18T10:45:01.547503+0000","last_change":"2023-10-18T10:44:59.548609+0000","last_active":"2023-10-18T10:45:01.547503+0000","last_peered":"2023-10-18T10:45:01.547503+0000","last_clean":"2023-10-18T10:45:01.547503+0000","last_became_active":"2023-10-18T10:44:59.548246+0000","last_became_peered":"2023-10-18T10:44:59.548246+0000","last_unstale":"2023-10-18T10:45:01.547503+0000","last_undegraded":"2023-10-18T10:45:01.547503+0000","last_fullsized":"2023-10-18T10:45:01.547503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:44:58.520998+0000","last_clean_scrub_stamp":"2023-10-18T10:44:58.520998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:44:14.959576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"145'258","reported_seq":788,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-18T10:45:00.546249+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T10:45:00.546249+0000","last_peered":"2023-10-18T10:45:00.546249+0000","last_clean":"2023-10-18T10:45:00.546249+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T10:45:00.546249+0000","last_undegraded":"2023-10-18T10:45:00.546249+0000","last_fullsized":"2023-10-18T10:45:00.546249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1346,"num_read_kb":2763,"num_write":415,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575090,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60052,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311788,"statfs":{"total":96636764160,"available":96575270912,"internally_reserved":0,"allocated":2555904,"data_stored":1575130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.217,"15min":0.217},"max":{"1min":0.77900000000000003,"5min":0.876,"15min":0.88300000000000001},"last":0.878},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.96199999999999997}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.69199999999999995,"5min":0.77000000000000002,"15min":0.82199999999999995},"last":2.899},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.58099999999999996},"min":{"1min":0.43099999999999999,"5min":0.312,"15min":0.28799999999999998},"max":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.90700000000000003},"last":1.036}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.79400000000000004,"15min":0.84899999999999998},"last":1.012}]},{"osd":3,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29699999999999999},"max":{"1min":0.77900000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60099999999999998},"min":{"1min":0.442,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.85799999999999998},"last":2.8199999999999998}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63400000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":1.8839999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"min":{"1min":0.38900000000000001,"5min":0.28599999999999998,"15min":0.26300000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.86199999999999999},"last":4.8019999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.30499999999999999,"5min":0.28399999999999997,"15min":0.25900000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.81000000000000005},"last":2.7210000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":1.6200000000000001}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.41899999999999998,"5min":0.38,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":1.397},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":1.671}]}]},{"osd":6,"up_from":13,"seq":55834575090,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1941504,"data_stored":977084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.29699999999999999},"max":{"1min":0.71899999999999997,"5min":0.76900000000000002,"15min":0.82499999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.27700000000000002},"max":{"1min":0.75900000000000001,"5min":0.81999999999999995,"15min":0.82599999999999996},"last":0.61299999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.36799999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.749,"5min":0.78900000000000003,"15min":0.82899999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.37,"15min":0.33000000000000002},"max":{"1min":0.70099999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.44700000000000001,"5min":0.35999999999999999,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.59699999999999998},"min":{"1min":0.442,"5min":0.372,"15min":0.313},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.84399999999999997},"last":0.70099999999999996}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.65400000000000003,"5min":0.78400000000000003,"15min":0.872},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.35099999999999998},"max":{"1min":0.77800000000000002,"5min":0.81399999999999995,"15min":0.84999999999999998},"last":0.72599999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76100000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.745},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71499999999999997,"5min":0.82499999999999996,"15min":0.83699999999999997},"last":0.59799999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.313},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.85399999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84699999999999998,"15min":0.87},"last":0.79300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575090,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51132,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320708,"statfs":{"total":96636764160,"available":96584404992,"internally_reserved":0,"allocated":1949696,"data_stored":983082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.97499999999999998},"last":0.497}]},{"osd":1,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.58599999999999997},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":0.81999999999999995,"15min":0.92000000000000004},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.58299999999999996},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.93600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.82799999999999996},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.70399999999999996,"5min":0.76500000000000001,"15min":0.84199999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83399999999999996,"15min":0.90000000000000002},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.80100000000000005},"last":0.376},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":1.0109999999999999},"last":0.41499999999999998}]},{"osd":6,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.83299999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.28599999999999998},"max":{"1min":0.83099999999999996,"5min":0.89300000000000002,"15min":0.96099999999999997},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed Oct 18 10:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.31},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.89300000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.307},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.871},"last":0.44500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575090,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1945600,"data_stored":983060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47199999999999998,"5min":0.379,"15min":0.26000000000000001},"max":{"1min":0.74299999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55900000000000005,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.877},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.89300000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.33100000000000002},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.5,"5min":0.371,"15min":0.29899999999999999},"max":{"1min":0.82899999999999996,"5min":0.873,"15min":0.873},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.313},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.504,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52500000000000002,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.29199999999999998,"15min":0.248},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.434}]},{"osd":6,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30099999999999999},"max":{"1min":2.8519999999999999,"5min":2.8519999999999999,"15min":2.8519999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.48199999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.29299999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.60399999999999998,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.28499999999999998,"15min":0.27100000000000002},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.46100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575090,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1961984,"data_stored":984742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.36499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.62},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.20399999999999999},"max":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.59299999999999997},"last":0.26500000000000001}]},{"osd":1,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20799999999999999},"max":{"1min":0.504,"5min":0.624,"15min":0.69099999999999995},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.311,"5min":0.221,"15min":0.221},"max":{"1min":0.52400000000000002,"5min":0.59299999999999997,"15min":0.67000000000000004},"last":0.28599999999999998}]},{"osd":2,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.42999999999999999},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.25900000000000001},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.70099999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23300000000000001},"max":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.71299999999999997},"last":0.33600000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.98299999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":0.91200000000000003,"15min":0.94699999999999995},"last":0.72999999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.86599999999999999,"15min":1.002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.371},"max":{"1min":0.88100000000000001,"5min":0.93899999999999995,"15min":0.97499999999999998},"last":0.70099999999999996}]},{"osd":6,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38,"15min":0.37},"max":{"1min":0.871,"5min":0.91500000000000004,"15min":1.018},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":0.879,"15min":0.96799999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":0.98899999999999999},"last":0.81999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575090,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55292,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316548,"statfs":{"total":96636764160,"available":96580145152,"internally_reserved":0,"allocated":1941504,"data_stored":977028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35999999999999999,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.17599999999999999},"max":{"1min":0.54500000000000004,"5min":0.65600000000000003,"15min":0.65800000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.38700000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.192},"max":{"1min":0.505,"5min":0.55500000000000005,"15min":0.626},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.55500000000000005,"5min":0.57899999999999996,"15min":0.67000000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.26400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55700000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.378,"15min":0.39300000000000002},"min":{"1min":0.28399999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.61599999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.40899999999999997}]},{"osd":4,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.64000000000000001},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.254},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.627,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.42599999999999999,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.84499999999999997}]},{"osd":5,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.61499999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.39800000000000002,"15min":0.27000000000000002},"max":{"1min":0.84799999999999998,"5min":0.87,"15min":0.93100000000000005},"last":0.79600000000000004}]},{"osd":6,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.89100000000000001,"15min":0.93600000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.42699999999999999,"15min":0.40300000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.93700000000000006},"last":0.85199999999999998}]},{"osd":7,"last update":"Wed Oct 18 10:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.48299999999999998,"5min":0.38500000000000001,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.878,"15min":0.91000000000000003},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39600000000000002},"max":{"1min":0.90900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":983130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.24099999999999999,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.64800000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.253},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.374,"15min":0.38600000000000001},"min":{"1min":0.221,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51400000000000001,"5min":0.58899999999999997,"15min":0.63700000000000001},"last":0.44}]},{"osd":2,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36699999999999999,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":0.55500000000000005,"15min":0.68899999999999995},"last":0.314},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40100000000000002,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52000000000000002,"5min":0.627,"15min":0.64800000000000002},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.42899999999999999},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63100000000000001,"5min":0.70199999999999996,"15min":0.72899999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.316},"max":{"1min":0.82999999999999996,"5min":1.099,"15min":1.099},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.308},"max":{"1min":0.88100000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.63300000000000001},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.373,"15min":1.373},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":0.83099999999999996,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61699999999999999,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.45600000000000002,"5min":0.42599999999999999,"15min":0.40100000000000002},"max":{"1min":0.86099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.51300000000000001}]},{"osd":7,"last update":"Wed Oct 18 10:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.91700000000000004,"15min":0.93500000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575089,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37828,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334012,"statfs":{"total":96636764160,"available":96598028288,"internally_reserved":0,"allocated":2547712,"data_stored":1573484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.37,"15min":0.379},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":0.56599999999999995,"5min":0.66100000000000003,"15min":0.71099999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.39000000000000001},"min":{"1min":0.26300000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.57999999999999996,"5min":0.62,"15min":0.76900000000000002},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.19600000000000001,"15min":0.187},"max":{"1min":0.53700000000000003,"5min":0.81100000000000005,"15min":0.96099999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.22,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.93000000000000005},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.30299999999999999,"5min":0.25900000000000001,"15min":0.24199999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.69699999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.33700000000000002,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.57599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39300000000000002}]},{"osd":4,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.996,"15min":0.996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.30499999999999999},"max":{"1min":0.79200000000000004,"5min":1.038,"15min":1.038},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.625},"min":{"1min":0.44500000000000001,"5min":0.376,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":1.0649999999999999,"15min":1.3859999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.36399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90400000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63500000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.872,"5min":1.026,"15min":1.3700000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":1.0920000000000001,"15min":1.4099999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Oct 18 10:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.339},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.48799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":1.01,"15min":1.01},"last":0.57899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T10:45:08.235 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T10:45:08.236 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T10:45:08.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.376+0000 7fa9728f2640 1 -- 172.21.15.125:0/3049090907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954006290 msgr2=0x7fa95400ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.376+0000 7fa9728f2640 1 --2- 172.21.15.125:0/3049090907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954006290 0x7fa95400ca90 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c610 tx=0x7fa954006900 comp rx=0 tx=0).stop 2023-10-18T10:45:08.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.376+0000 7fa9728f2640 1 -- 172.21.15.125:0/3049090907 shutdown_connections 2023-10-18T10:45:08.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.376+0000 7fa9728f2640 1 --2- 172.21.15.125:0/3049090907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954006290 0x7fa95400ca90 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.376+0000 7fa9728f2640 1 -- 172.21.15.125:0/3049090907 >> 172.21.15.125:0/3049090907 conn(0x7fa96c069930 msgr2=0x7fa96c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:08.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.376+0000 7fa9728f2640 1 -- 172.21.15.125:0/3049090907 shutdown_connections 2023-10-18T10:45:08.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.376+0000 7fa9728f2640 1 -- 172.21.15.125:0/3049090907 wait complete. 2023-10-18T10:45:08.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.377+0000 7fa9728f2640 1 Processor -- start 2023-10-18T10:45:08.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.377+0000 7fa9728f2640 1 -- start start 2023-10-18T10:45:08.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa9728f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa954006290 0x7fa96c14af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa9728f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa96c14b5b0 0x7fa96c14cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa9728f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa96c14f9b0 0x7fa96c14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa9728f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c150380 con 0x7fa96c14f9b0 2023-10-18T10:45:08.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa9728f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c14e2b0 con 0x7fa954006290 2023-10-18T10:45:08.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa9728f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c14d3c0 con 0x7fa96c14b5b0 2023-10-18T10:45:08.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa970e8f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa96c14f9b0 0x7fa96c14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa970e8f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa96c14f9b0 0x7fa96c14fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40794/0 (socket says 172.21.15.125:40794) 2023-10-18T10:45:08.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa970e8f640 1 -- 172.21.15.125:0/4024432783 learned_addr learned my addr 172.21.15.125:0/4024432783 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:08.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa96bfff640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa954006290 0x7fa96c14af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa96b7fe640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa96c14b5b0 0x7fa96c14cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa970e8f640 1 -- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa96c14b5b0 msgr2=0x7fa96c14cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa970e8f640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa96c14b5b0 0x7fa96c14cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa970e8f640 1 -- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa954006290 msgr2=0x7fa96c14af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa970e8f640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa954006290 0x7fa96c14af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.378+0000 7fa970e8f640 1 -- 172.21.15.125:0/4024432783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c14d6a0 con 0x7fa96c14f9b0 2023-10-18T10:45:08.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.379+0000 7fa970e8f640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa96c14f9b0 0x7fa96c14fe40 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa96402d010 tx=0x7fa964001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:08.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.379+0000 7fa968ff9640 1 -- 172.21.15.125:0/4024432783 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964001b80 con 0x7fa96c14f9b0 2023-10-18T10:45:08.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.379+0000 7fa968ff9640 1 -- 172.21.15.125:0/4024432783 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa964003b70 con 0x7fa96c14f9b0 2023-10-18T10:45:08.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.379+0000 7fa968ff9640 1 -- 172.21.15.125:0/4024432783 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640060d0 con 0x7fa96c14f9b0 2023-10-18T10:45:08.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.379+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c14d980 con 0x7fa96c14f9b0 2023-10-18T10:45:08.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.379+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c14df40 con 0x7fa96c14f9b0 2023-10-18T10:45:08.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.380+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c14e0d0 con 0x7fa96c14f9b0 2023-10-18T10:45:08.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.380+0000 7fa968ff9640 1 -- 172.21.15.125:0/4024432783 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa964031030 con 0x7fa96c14f9b0 2023-10-18T10:45:08.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.381+0000 7fa968ff9640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa95806af90 0x7fa95806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.381+0000 7fa968ff9640 1 -- 172.21.15.125:0/4024432783 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa9640aa450 con 0x7fa96c14f9b0 2023-10-18T10:45:08.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.381+0000 7fa96bfff640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa95806af90 0x7fa95806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.381+0000 7fa96bfff640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa95806af90 0x7fa95806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0a6720 tx=0x7fa95c005f90 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:08.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.385+0000 7fa968ff9640 1 -- 172.21.15.125:0/4024432783 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa964075230 con 0x7fa96c14f9b0 2023-10-18T10:45:08.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.620+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa96c0690e0 con 0x7fa96c14f9b0 2023-10-18T10:45:08.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.622+0000 7fa968ff9640 1 -- 172.21.15.125:0/4024432783 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa964078850 con 0x7fa96c14f9b0 2023-10-18T10:45:08.624 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T10:45:08.624 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T10:45:08.624 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T10:45:08.625 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T10:45:08.625 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T10:45:08.626 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T10:45:08.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa95806af90 msgr2=0x7fa95806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa95806af90 0x7fa95806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0a6720 tx=0x7fa95c005f90 comp rx=0 tx=0).stop 2023-10-18T10:45:08.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa96c14f9b0 msgr2=0x7fa96c14fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa96c14f9b0 0x7fa96c14fe40 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa96402d010 tx=0x7fa964001990 comp rx=0 tx=0).stop 2023-10-18T10:45:08.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 shutdown_connections 2023-10-18T10:45:08.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fa95806af90 0x7fa95806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa96c14f9b0 0x7fa96c14fe40 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa96c14b5b0 0x7fa96c14cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 --2- 172.21.15.125:0/4024432783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa954006290 0x7fa96c14af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 >> 172.21.15.125:0/4024432783 conn(0x7fa96c069930 msgr2=0x7fa96c0af010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:08.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 shutdown_connections 2023-10-18T10:45:08.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.624+0000 7fa9728f2640 1 -- 172.21.15.125:0/4024432783 wait complete. 2023-10-18T10:45:08.636 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T10:45:08.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.737+0000 7fb121a4b640 1 -- 172.21.15.125:0/3864859302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 msgr2=0x7fb11c158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.737+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3864859302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 0x7fb11c158cf0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb10c005fd0 tx=0x7fb10c02be10 comp rx=0 tx=0).stop 2023-10-18T10:45:08.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.737+0000 7fb121a4b640 1 -- 172.21.15.125:0/3864859302 shutdown_connections 2023-10-18T10:45:08.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.737+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3864859302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb11c0cd1c0 0x7fb11c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.737+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3864859302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 0x7fb11c158cf0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.737+0000 7fb121a4b640 1 -- 172.21.15.125:0/3864859302 >> 172.21.15.125:0/3864859302 conn(0x7fb11c0af7a0 msgr2=0x7fb11c0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:08.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.737+0000 7fb121a4b640 1 -- 172.21.15.125:0/3864859302 shutdown_connections 2023-10-18T10:45:08.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.737+0000 7fb121a4b640 1 -- 172.21.15.125:0/3864859302 wait complete. 2023-10-18T10:45:08.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.738+0000 7fb121a4b640 1 Processor -- start 2023-10-18T10:45:08.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.738+0000 7fb121a4b640 1 -- start start 2023-10-18T10:45:08.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb121a4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cd1c0 0x7fb11c14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb121a4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb11c0cd9e0 0x7fb11c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb121a4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 0x7fb11c0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb121a4b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0c7360 con 0x7fb11c158910 2023-10-18T10:45:08.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb121a4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c157db0 con 0x7fb11c0cd1c0 2023-10-18T10:45:08.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb121a4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0c7570 con 0x7fb11c0cd9e0 2023-10-18T10:45:08.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 0x7fb11c0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 0x7fb11c0c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40812/0 (socket says 172.21.15.125:40812) 2023-10-18T10:45:08.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 -- 172.21.15.125:0/3992956496 learned_addr learned my addr 172.21.15.125:0/3992956496 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:08.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11affd640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cd1c0 0x7fb11c14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb113fff640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb11c0cd9e0 0x7fb11c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 -- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb11c0cd9e0 msgr2=0x7fb11c0c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb11c0cd9e0 0x7fb11c0c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 -- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cd1c0 msgr2=0x7fb11c14a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cd1c0 0x7fb11c14a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 -- 172.21.15.125:0/3992956496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c009560 con 0x7fb11c158910 2023-10-18T10:45:08.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb11b7fe640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 0x7fb11c0c8610 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c7d0 tx=0x7fb11400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:08.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb118ff9640 1 -- 172.21.15.125:0/3992956496 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114003b70 con 0x7fb11c158910 2023-10-18T10:45:08.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb118ff9640 1 -- 172.21.15.125:0/3992956496 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb114003cf0 con 0x7fb11c158910 2023-10-18T10:45:08.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb118ff9640 1 -- 172.21.15.125:0/3992956496 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114003ea0 con 0x7fb11c158910 2023-10-18T10:45:08.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c0c7850 con 0x7fb11c158910 2023-10-18T10:45:08.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.739+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c0c7e10 con 0x7fb11c158910 2023-10-18T10:45:08.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.740+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c0690e0 con 0x7fb11c158910 2023-10-18T10:45:08.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.741+0000 7fb118ff9640 1 -- 172.21.15.125:0/3992956496 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb11400aab0 con 0x7fb11c158910 2023-10-18T10:45:08.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.741+0000 7fb118ff9640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0fc06af60 0x7fb0fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:08.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.741+0000 7fb118ff9640 1 -- 172.21.15.125:0/3992956496 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fb114089780 con 0x7fb11c158910 2023-10-18T10:45:08.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.741+0000 7fb11affd640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0fc06af60 0x7fb0fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:08.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.742+0000 7fb11affd640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0fc06af60 0x7fb0fc06d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb10c035000 tx=0x7fb10c004730 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:08.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.746+0000 7fb118ff9640 1 -- 172.21.15.125:0/3992956496 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb114054670 con 0x7fb11c158910 2023-10-18T10:45:08.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.944+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb11c0ce5e0 con 0x7fb11c158910 2023-10-18T10:45:08.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.945+0000 7fb118ff9640 1 -- 172.21.15.125:0/3992956496 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fb114057c90 con 0x7fb11c158910 2023-10-18T10:45:08.947 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:08.947 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2692191151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2692191151},{"type":"v1","addr":"172.21.15.139:6836","nonce":2692191151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1995790917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1995790917},{"type":"v1","addr":"172.21.15.139:6837","nonce":1995790917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/55232611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":55232611},{"type":"v1","addr":"172.21.15.125:6834","nonce":55232611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:59.877487+0000","modified":"2023-10-18T10:45:01.875939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9765":{"gid":9765,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/631315131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":631315131},{"type":"v1","addr":"172.21.15.125:6835","nonce":631315131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T10:45:08.948 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 60 2023-10-18T10:45:08.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0fc06af60 msgr2=0x7fb0fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0fc06af60 0x7fb0fc06d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb10c035000 tx=0x7fb10c004730 comp rx=0 tx=0).stop 2023-10-18T10:45:08.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 msgr2=0x7fb11c0c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:08.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 0x7fb11c0c8610 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c7d0 tx=0x7fb11400cc90 comp rx=0 tx=0).stop 2023-10-18T10:45:08.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 shutdown_connections 2023-10-18T10:45:08.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fb0fc06af60 0x7fb0fc06d420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb11c158910 0x7fb11c0c8610 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb11c0cd9e0 0x7fb11c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 --2- 172.21.15.125:0/3992956496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cd1c0 0x7fb11c14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:08.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.946+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 >> 172.21.15.125:0/3992956496 conn(0x7fb11c0af7a0 msgr2=0x7fb11c14af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:08.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.947+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 shutdown_connections 2023-10-18T10:45:08.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:08.947+0000 7fb121a4b640 1 -- 172.21.15.125:0/3992956496 wait complete. 2023-10-18T10:45:09.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T10:44:59.877487+0000', 'modified': '2023-10-18T10:45:01.875939+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9765}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9765': {'gid': 9765, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/631315131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 631315131}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 631315131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T10:45:09.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T10:45:09.362 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T10:45:09.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.467+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1350648433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf015b2a0 msgr2=0x7f7cf00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.467+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1350648433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf015b2a0 0x7f7cf00d1910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4005fd0 tx=0x7f7ce402bd90 comp rx=0 tx=0).stop 2023-10-18T10:45:09.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.467+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1350648433 shutdown_connections 2023-10-18T10:45:09.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.467+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1350648433 >> v2:172.21.15.139:3300/0 conn(0x7f7cf00ceb40 0x7f7cf00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.467+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1350648433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf015b2a0 0x7f7cf00d1910 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.467+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1350648433 >> 172.21.15.125:0/1350648433 conn(0x7f7cf00d0d60 msgr2=0x7f7cf00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:09.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.467+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1350648433 shutdown_connections 2023-10-18T10:45:09.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.467+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1350648433 wait complete. 2023-10-18T10:45:09.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.468+0000 7f7cf7c64640 1 Processor -- start 2023-10-18T10:45:09.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.468+0000 7f7cf7c64640 1 -- start start 2023-10-18T10:45:09.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf7c64640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf00ceb40 0x7f7cf014f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf7c64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf00cf340 0x7f7cf01e3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf7c64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cf015b2a0 0x7f7cf01e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf7c64640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf01e4f90 con 0x7f7cf015b2a0 2023-10-18T10:45:09.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf7c64640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf01e53b0 con 0x7f7cf00cf340 2023-10-18T10:45:09.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf7c64640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf01e4310 con 0x7f7cf00ceb40 2023-10-18T10:45:09.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cf015b2a0 0x7f7cf01e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cf015b2a0 0x7f7cf01e59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40826/0 (socket says 172.21.15.125:40826) 2023-10-18T10:45:09.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 -- 172.21.15.125:0/1837279244 learned_addr learned my addr 172.21.15.125:0/1837279244 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:09.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf00ceb40 0x7f7cf014f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf00cf340 0x7f7cf01e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 -- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf00ceb40 msgr2=0x7f7cf014f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf00ceb40 0x7f7cf014f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 -- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf00cf340 msgr2=0x7f7cf01e3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf00cf340 0x7f7cf01e3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 -- 172.21.15.125:0/1837279244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce4009560 con 0x7f7cf015b2a0 2023-10-18T10:45:09.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf00ceb40 0x7f7cf014f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:09.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf00cf340 0x7f7cf01e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:09.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.469+0000 7f7cf6201640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cf015b2a0 0x7f7cf01e59d0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e970 tx=0x7f7ce800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:09.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.470+0000 7f7cee7fc640 1 -- 172.21.15.125:0/1837279244 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8009720 con 0x7f7cf015b2a0 2023-10-18T10:45:09.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.470+0000 7f7cee7fc640 1 -- 172.21.15.125:0/1837279244 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ce800ccc0 con 0x7f7cf015b2a0 2023-10-18T10:45:09.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.470+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1837279244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf00b0d50 con 0x7f7cf015b2a0 2023-10-18T10:45:09.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.470+0000 7f7cee7fc640 1 -- 172.21.15.125:0/1837279244 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8005280 con 0x7f7cf015b2a0 2023-10-18T10:45:09.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.470+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1837279244 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf01e48d0 con 0x7f7cf015b2a0 2023-10-18T10:45:09.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.471+0000 7f7cee7fc640 1 -- 172.21.15.125:0/1837279244 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7ce80053e0 con 0x7f7cf015b2a0 2023-10-18T10:45:09.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.471+0000 7f7cee7fc640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7cdc06af60 0x7f7cdc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.471+0000 7f7cee7fc640 1 -- 172.21.15.125:0/1837279244 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f7ce8014070 con 0x7f7cf015b2a0 2023-10-18T10:45:09.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.472+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7cdc06af60 0x7f7cdc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.472+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7cdc06af60 0x7f7cdc06d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0001d70 tx=0x7f7ce0009040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:09.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cfa7f9740 1 Processor -- start 2023-10-18T10:45:09.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cfa7f9740 1 -- start start 2023-10-18T10:45:09.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515b40 0x562c324e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515f20 0x562c325188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c324e9020 0x562c32518e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c323fa730 con 0x562c324e9020 2023-10-18T10:45:09.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3245b780 con 0x562c32515f20 2023-10-18T10:45:09.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c32458e70 con 0x562c32515b40 2023-10-18T10:45:09.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cf6201640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c324e9020 0x562c32518e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cf6201640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c324e9020 0x562c32518e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40828/0 (socket says 172.21.15.125:40828) 2023-10-18T10:45:09.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.473+0000 7f7cf6201640 1 -- 172.21.15.125:0/2293137451 learned_addr learned my addr 172.21.15.125:0/2293137451 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:09.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515f20 0x562c325188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515b40 0x562c324e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cf6201640 1 -- 172.21.15.125:0/2293137451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515b40 msgr2=0x562c324e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:09.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cf6201640 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515b40 0x562c324e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cf6201640 1 -- 172.21.15.125:0/2293137451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515f20 msgr2=0x562c325188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:09.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cf6201640 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515f20 0x562c325188d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cf6201640 1 -- 172.21.15.125:0/2293137451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c324593d0 con 0x562c324e9020 2023-10-18T10:45:09.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cf6201640 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c324e9020 0x562c32518e10 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8058e70 tx=0x7f7ce8058fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ed85f62e38f492 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:09.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cd77fe640 1 -- 172.21.15.125:0/2293137451 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800ce50 con 0x562c324e9020 2023-10-18T10:45:09.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cd77fe640 1 -- 172.21.15.125:0/2293137451 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ce8057020 con 0x562c324e9020 2023-10-18T10:45:09.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.474+0000 7f7cd77fe640 1 -- 172.21.15.125:0/2293137451 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80888a0 con 0x562c324e9020 2023-10-18T10:45:09.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2293137451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c324e9020 msgr2=0x562c32518e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c324e9020 0x562c32518e10 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8058e70 tx=0x7f7ce8058fa0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2293137451 shutdown_connections 2023-10-18T10:45:09.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515f20 0x562c325188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c324e9020 0x562c32518e10 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2293137451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515b40 0x562c324e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2293137451 >> 172.21.15.125:0/2293137451 conn(0x562c324ead50 msgr2=0x562c325338d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:09.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2293137451 shutdown_connections 2023-10-18T10:45:09.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2293137451 wait complete. 2023-10-18T10:45:09.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 -- start start 2023-10-18T10:45:09.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.475+0000 7f7cfa7f9740 1 Processor -- start 2023-10-18T10:45:09.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 0x562c3252a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c32517da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 0x562c324ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c32454760 con 0x562c32515f20 2023-10-18T10:45:09.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3240f8f0 con 0x562c32515b40 2023-10-18T10:45:09.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c3240f590 con 0x562c32547710 2023-10-18T10:45:09.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cfa7f9740 10 client.0 _ll_register_callbacks cb 0x562c3244bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:45:09.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cf51ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c32517da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cf51ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c32517da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40842/0 (socket says 172.21.15.125:40842) 2023-10-18T10:45:09.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cf51ff640 1 -- 172.21.15.125:0/2504550893 learned_addr learned my addr 172.21.15.125:0/2504550893 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:09.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.476+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 0x562c3252a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cf6201640 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 0x562c324ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cf51ff640 1 -- 172.21.15.125:0/2504550893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 msgr2=0x562c324ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:09.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 0x562c324ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cf51ff640 1 -- 172.21.15.125:0/2504550893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 msgr2=0x562c3252a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:09.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 0x562c3252a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cf51ff640 1 -- 172.21.15.125:0/2504550893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf01e48d0 con 0x562c32515f20 2023-10-18T10:45:09.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c32517da0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4036040 tx=0x7f7ce402c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e995bbf02477e51c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:09.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:45:09.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/2504550893 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40037f0 con 0x562c32515f20 2023-10-18T10:45:09.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2504550893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c3244d740 con 0x562c32515f20 2023-10-18T10:45:09.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/2504550893 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f7ce4003950 con 0x562c32515f20 2023-10-18T10:45:09.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/2504550893 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ce4003b10 con 0x562c32515f20 2023-10-18T10:45:09.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cfa7f9740 10 client.9927 fetch_fsmap learned FSMap version 60 2023-10-18T10:45:09.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2504550893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x562c32452de0 con 0x562c32515f20 2023-10-18T10:45:09.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/2504550893 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x562c3244d740 con 0x562c32515f20 2023-10-18T10:45:09.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/2504550893 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7ce4045020 con 0x562c32515f20 2023-10-18T10:45:09.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.477+0000 7f7cfa7f9740 10 client.9927 fetch_fsmap finished waiting for FSMap version 60 2023-10-18T10:45:09.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cfa7f9740 10 client.9927 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:45:09.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cfa7f9740 20 client.9927 populate_metadata read hostname 'smithi125' 2023-10-18T10:45:09.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] conn(0x562c32501d80 0x562c32502160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cfa7f9740 4 client.9927 mds_command: new command op to 9765 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T10:45:09.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2504550893 --> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c3244d460 con 0x562c32501d80 2023-10-18T10:45:09.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] conn(0x562c32501d80 0x562c32502160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cd4ff9640 10 client.9927 ms_handle_connect on v2:172.21.15.125:6833/631315131 2023-10-18T10:45:09.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] conn(0x562c32501d80 0x562c32502160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5153147705b403a server_cookie=7c3261caae9ced52 in_seq=0 out_seq=0 2023-10-18T10:45:09.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/2504550893 <== mds.0 v2:172.21.15.125:6833/631315131 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562c3244d460 con 0x562c32501d80 2023-10-18T10:45:09.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cd4ff9640 10 client.9927 handle_command_reply: tid=0 2023-10-18T10:45:09.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.478+0000 7f7cfa7f9740 1 client.9927 shutdown 2023-10-18T10:45:09.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2504550893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 msgr2=0x562c32517da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c32517da0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4036040 tx=0x7f7ce402c730 comp rx=0 tx=0).stop 2023-10-18T10:45:09.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2504550893 shutdown_connections 2023-10-18T10:45:09.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] conn(0x562c32501d80 0x562c32502160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 0x562c324ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c32517da0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cd4ff9640 0 client.9927 ms_handle_reset on v2:172.21.15.125:6833/631315131 2023-10-18T10:45:09.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/2504550893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 0x562c3252a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2504550893 >> 172.21.15.125:0/2504550893 conn(0x562c324e9020 msgr2=0x562c324ec9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:09.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2504550893 shutdown_connections 2023-10-18T10:45:09.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/2504550893 wait complete. 2023-10-18T10:45:09.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.479+0000 7f7cfa7f9740 20 client.9927 trim_cache size 0 max 16384 2023-10-18T10:45:09.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cfa7f9740 1 Processor -- start 2023-10-18T10:45:09.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cfa7f9740 1 -- start start 2023-10-18T10:45:09.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515b40 0x562c324eec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515f20 0x562c324e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32547710 0x562c324e4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c323d5270 con 0x562c32515b40 2023-10-18T10:45:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c324ef150 con 0x562c32547710 2023-10-18T10:45:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c324eaa10 con 0x562c32515f20 2023-10-18T10:45:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cf5a00640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515b40 0x562c324eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cf5a00640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515b40 0x562c324eec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40848/0 (socket says 172.21.15.125:40848) 2023-10-18T10:45:09.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.494+0000 7f7cf5a00640 1 -- 172.21.15.125:0/494528698 learned_addr learned my addr 172.21.15.125:0/494528698 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf6201640 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32547710 0x562c324e4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515f20 0x562c324e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf5a00640 1 -- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515f20 msgr2=0x562c324e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515f20 0x562c324e4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf5a00640 1 -- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32547710 msgr2=0x562c324e4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:09.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32547710 0x562c324e4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf5a00640 1 -- 172.21.15.125:0/494528698 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c323f1d00 con 0x562c32515b40 2023-10-18T10:45:09.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf6201640 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32547710 0x562c324e4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:09.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515f20 0x562c324e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:09.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515b40 0x562c324eec10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000dfb0 tx=0x7f7ce0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fcb4bcc98ab9fb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:09.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cd7fff640 1 -- 172.21.15.125:0/494528698 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000da30 con 0x562c32515b40 2023-10-18T10:45:09.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cd7fff640 1 -- 172.21.15.125:0/494528698 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ce0025040 con 0x562c32515b40 2023-10-18T10:45:09.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.495+0000 7f7cd7fff640 1 -- 172.21.15.125:0/494528698 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0025410 con 0x562c32515b40 2023-10-18T10:45:09.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/494528698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515b40 msgr2=0x562c324eec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515b40 0x562c324eec10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000dfb0 tx=0x7f7ce0013000 comp rx=0 tx=0).stop 2023-10-18T10:45:09.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/494528698 shutdown_connections 2023-10-18T10:45:09.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32547710 0x562c324e4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32515f20 0x562c324e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/494528698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515b40 0x562c324eec10 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/494528698 >> 172.21.15.125:0/494528698 conn(0x562c324e9020 msgr2=0x562c32555e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:09.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/494528698 shutdown_connections 2023-10-18T10:45:09.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/494528698 wait complete. 2023-10-18T10:45:09.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 -- start start 2023-10-18T10:45:09.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.496+0000 7f7cfa7f9740 1 Processor -- start 2023-10-18T10:45:09.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 0x562c324ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c324edd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cfa7f9740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 0x562c325a4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c32500a80 con 0x562c32515f20 2023-10-18T10:45:09.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c325a5270 con 0x562c32515b40 2023-10-18T10:45:09.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cfa7f9740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c324e4a50 con 0x562c32547710 2023-10-18T10:45:09.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cfa7f9740 10 client.0 _ll_register_callbacks cb 0x7f7ce8058a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T10:45:09.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cf51ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c324edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cf51ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c324edd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40856/0 (socket says 172.21.15.125:40856) 2023-10-18T10:45:09.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cf51ff640 1 -- 172.21.15.125:0/3598687900 learned_addr learned my addr 172.21.15.125:0/3598687900 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:09.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 0x562c324ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.497+0000 7f7cf6201640 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 0x562c325a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cf51ff640 1 -- 172.21.15.125:0/3598687900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 msgr2=0x562c325a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:09.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 0x562c325a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cf51ff640 1 -- 172.21.15.125:0/3598687900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 msgr2=0x562c324ed7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T10:45:09.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 0x562c324ed7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cf51ff640 1 -- 172.21.15.125:0/3598687900 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce001c040 con 0x562c32515f20 2023-10-18T10:45:09.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cf51ff640 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c324edd20 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006010 tx=0x7f7ce4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f02a2b6902323e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:09.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T10:45:09.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/3598687900 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4034d40 con 0x562c32515f20 2023-10-18T10:45:09.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/3598687900 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f7ce4000e40 con 0x562c32515f20 2023-10-18T10:45:09.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/3598687900 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ce8057020 con 0x562c32515f20 2023-10-18T10:45:09.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/3598687900 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ce4034ea0 con 0x562c32515f20 2023-10-18T10:45:09.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 10 client.9933 fetch_fsmap learned FSMap version 60 2023-10-18T10:45:09.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/3598687900 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x562c324174f0 con 0x562c32515f20 2023-10-18T10:45:09.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/3598687900 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ce8057020 con 0x562c32515f20 2023-10-18T10:45:09.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/3598687900 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7ce4045030 con 0x562c32515f20 2023-10-18T10:45:09.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 10 client.9933 fetch_fsmap finished waiting for FSMap version 60 2023-10-18T10:45:09.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 10 client.9933 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T10:45:09.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 20 client.9933 populate_metadata read hostname 'smithi125' 2023-10-18T10:45:09.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] conn(0x562c32500ea0 0x562c32501280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 4 client.9933 mds_command: new command op to 9765 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T10:45:09.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/3598687900 --> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562c321566c0 con 0x562c32500ea0 2023-10-18T10:45:09.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.498+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] conn(0x562c32500ea0 0x562c32501280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cd4ff9640 10 client.9933 ms_handle_connect on v2:172.21.15.125:6833/631315131 2023-10-18T10:45:09.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cf5a00640 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] conn(0x562c32500ea0 0x562c32501280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=530ac996867cbb server_cookie=e6001221054fde99 in_seq=0 out_seq=0 2023-10-18T10:45:09.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cd4ff9640 1 -- 172.21.15.125:0/3598687900 <== mds.0 v2:172.21.15.125:6833/631315131 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x562c321566c0 con 0x562c32500ea0 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cd4ff9640 10 client.9933 handle_command_reply: tid=0 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 client.9933 shutdown 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/3598687900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 msgr2=0x562c324edd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c324edd20 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006010 tx=0x7f7ce4009780 comp rx=0 tx=0).stop 2023-10-18T10:45:09.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/3598687900 shutdown_connections 2023-10-18T10:45:09.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.125:6833/631315131,v1:172.21.15.125:6835/631315131] conn(0x562c32500ea0 0x562c32501280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c32547710 0x562c325a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562c32515f20 0x562c324edd20 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 --2- 172.21.15.125:0/3598687900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c32515b40 0x562c324ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cd4ff9640 0 client.9933 ms_handle_reset on v2:172.21.15.125:6833/631315131 2023-10-18T10:45:09.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/3598687900 >> 172.21.15.125:0/3598687900 conn(0x562c324e9020 msgr2=0x562c32516d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:09.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/3598687900 shutdown_connections 2023-10-18T10:45:09.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 1 -- 172.21.15.125:0/3598687900 wait complete. 2023-10-18T10:45:09.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.499+0000 7f7cfa7f9740 20 client.9933 trim_cache size 0 max 16384 2023-10-18T10:45:09.526 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:45:09.526 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T10:45:09.526 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T10:45:09.526 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 9765, 2023-10-18T10:45:09.526 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T10:45:09.527 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T10:45:09.527 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T10:45:09.527 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.9666849590000002, 2023-10-18T10:45:09.527 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 60, 2023-10-18T10:45:09.527 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 156, 2023-10-18T10:45:09.528 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 156, 2023-10-18T10:45:09.528 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 13.740131051000001 2023-10-18T10:45:09.528 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:45:09.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7cdc06af60 msgr2=0x7f7cdc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7cdc06af60 0x7f7cdc06d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0001d70 tx=0x7f7ce0009040 comp rx=0 tx=0).stop 2023-10-18T10:45:09.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1837279244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cf015b2a0 msgr2=0x7f7cf01e59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cf015b2a0 0x7f7cf01e59d0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e970 tx=0x7f7ce800ed70 comp rx=0 tx=0).stop 2023-10-18T10:45:09.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1837279244 shutdown_connections 2023-10-18T10:45:09.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f7cdc06af60 0x7f7cdc06d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cf00cf340 0x7f7cf01e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cf015b2a0 0x7f7cf01e59d0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 --2- 172.21.15.125:0/1837279244 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cf00ceb40 0x7f7cf014f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1837279244 >> 172.21.15.125:0/1837279244 conn(0x7f7cf00d0d60 msgr2=0x7f7cf006b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:09.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1837279244 shutdown_connections 2023-10-18T10:45:09.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.500+0000 7f7cf7c64640 1 -- 172.21.15.125:0/1837279244 wait complete. 2023-10-18T10:45:09.533 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T10:45:09.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.633+0000 7f6cf6a56640 1 -- 172.21.15.125:0/3507744707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 msgr2=0x7f6cf0162e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.633+0000 7f6cf6a56640 1 --2- 172.21.15.125:0/3507744707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 0x7f6cf0162e10 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8005fd0 tx=0x7f6ce802c170 comp rx=0 tx=0).stop 2023-10-18T10:45:09.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.633+0000 7f6cf6a56640 1 -- 172.21.15.125:0/3507744707 shutdown_connections 2023-10-18T10:45:09.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.633+0000 7f6cf6a56640 1 --2- 172.21.15.125:0/3507744707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 0x7f6cf0162e10 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.633+0000 7f6cf6a56640 1 -- 172.21.15.125:0/3507744707 >> 172.21.15.125:0/3507744707 conn(0x7f6cf0069930 msgr2=0x7f6cf0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:09.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.633+0000 7f6cf6a56640 1 -- 172.21.15.125:0/3507744707 shutdown_connections 2023-10-18T10:45:09.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.633+0000 7f6cf6a56640 1 -- 172.21.15.125:0/3507744707 wait complete. 2023-10-18T10:45:09.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.634+0000 7f6cf6a56640 1 Processor -- start 2023-10-18T10:45:09.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.634+0000 7f6cf6a56640 1 -- start start 2023-10-18T10:45:09.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf6a56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6cf00cb980 0x7f6cf0132890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf6a56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 0x7f6cf01330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf6a56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cf0152770 0x7f6cf0162520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf6a56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0139150 con 0x7f6cf0160a20 2023-10-18T10:45:09.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf6a56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf01384d0 con 0x7f6cf0152770 2023-10-18T10:45:09.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf6a56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf01387b0 con 0x7f6cf00cb980 2023-10-18T10:45:09.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 0x7f6cf01330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 0x7f6cf01330b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40878/0 (socket says 172.21.15.125:40878) 2023-10-18T10:45:09.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 -- 172.21.15.125:0/427346843 learned_addr learned my addr 172.21.15.125:0/427346843 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:09.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf4ff3640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cf0152770 0x7f6cf0162520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6ceffff640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6cf00cb980 0x7f6cf0132890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 -- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6cf00cb980 msgr2=0x7f6cf0132890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6cf00cb980 0x7f6cf0132890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 -- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cf0152770 msgr2=0x7f6cf0162520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cf0152770 0x7f6cf0162520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 -- 172.21.15.125:0/427346843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce8009840 con 0x7f6cf0160a20 2023-10-18T10:45:09.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf4ff3640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cf0152770 0x7f6cf0162520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:45:09.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cef7fe640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 0x7f6cf01330b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c450 tx=0x7f6ce400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:09.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cecff9640 1 -- 172.21.15.125:0/427346843 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40052d0 con 0x7f6cf0160a20 2023-10-18T10:45:09.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cecff9640 1 -- 172.21.15.125:0/427346843 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ce4005430 con 0x7f6cf0160a20 2023-10-18T10:45:09.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cecff9640 1 -- 172.21.15.125:0/427346843 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4012890 con 0x7f6cf0160a20 2023-10-18T10:45:09.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf0138a90 con 0x7f6cf0160a20 2023-10-18T10:45:09.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.635+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf0132dd0 con 0x7f6cf0160a20 2023-10-18T10:45:09.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.636+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf0139290 con 0x7f6cf0160a20 2023-10-18T10:45:09.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.637+0000 7f6cecff9640 1 -- 172.21.15.125:0/427346843 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6ce40035f0 con 0x7f6cf0160a20 2023-10-18T10:45:09.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.637+0000 7f6cecff9640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f6cdc06af60 0x7f6cdc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:09.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.637+0000 7f6cecff9640 1 -- 172.21.15.125:0/427346843 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f6ce4089570 con 0x7f6cf0160a20 2023-10-18T10:45:09.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.638+0000 7f6ceffff640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f6cdc06af60 0x7f6cdc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:09.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.638+0000 7f6ceffff640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f6cdc06af60 0x7f6cdc06d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00046b0 tx=0x7f6ce0009040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:09.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.642+0000 7f6cecff9640 1 -- 172.21.15.125:0/427346843 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6cf0139290 con 0x7f6cf0160a20 2023-10-18T10:45:09.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.841+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6cf0139290 con 0x7f6cf0160a20 2023-10-18T10:45:09.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.841+0000 7f6cecff9640 1 -- 172.21.15.125:0/427346843 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6cf0139290 con 0x7f6cf0160a20 2023-10-18T10:45:09.843 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T10:45:09.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f6cdc06af60 msgr2=0x7f6cdc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f6cdc06af60 0x7f6cdc06d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00046b0 tx=0x7f6ce0009040 comp rx=0 tx=0).stop 2023-10-18T10:45:09.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 msgr2=0x7f6cf01330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:09.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 0x7f6cf01330b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c450 tx=0x7f6ce400c910 comp rx=0 tx=0).stop 2023-10-18T10:45:09.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 shutdown_connections 2023-10-18T10:45:09.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f6cdc06af60 0x7f6cdc06d420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6cf0152770 0x7f6cf0162520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cf0160a20 0x7f6cf01330b0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 --2- 172.21.15.125:0/427346843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6cf00cb980 0x7f6cf0132890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:09.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 >> 172.21.15.125:0/427346843 conn(0x7f6cf0069930 msgr2=0x7f6cf00c8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:09.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 shutdown_connections 2023-10-18T10:45:09.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:09.843+0000 7f6cf6a56640 1 -- 172.21.15.125:0/427346843 wait complete. 2023-10-18T10:45:09.855 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T10:45:09.855 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T10:45:09.856 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T10:45:09.856 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T10:45:09.856 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T10:45:09.857 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T10:45:09.857 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T10:45:09.857 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T10:45:09.858 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:45:09.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:09.858 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T10:45:09.874 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T10:45:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T10:45:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T10:45:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:45:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T10:45:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:45:09.875 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:45:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:45:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T10:45:09.875 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:45:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:45:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T10:45:09.876 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T10:45:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:45:09.876 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T10:45:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:45:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T10:45:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 41514sec preferred_lft 41514sec 2023-10-18T10:45:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T10:45:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:45:09.878 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T10:45:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T10:45:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T10:45:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:45:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T10:45:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:45:09.879 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:45:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:45:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T10:45:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:45:09.880 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T10:45:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:45:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T10:45:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T10:45:09.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:09.881 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T10:45:09.939 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T10:45:09.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:09.941 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T10:45:09.996 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T10:45:09.996 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T10:45:09.996 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T10:45:09.996 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T10:45:09.997 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T10:45:09.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:09.997 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T10:45:09.998 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T10:45:09.998 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T10:45:09.999 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T10:45:09.999 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T10:45:09.999 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T10:45:10.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:10.092 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T10:45:10.107 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T10:45:10.107 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T10:45:10.107 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:45:10.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:10.108 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T10:45:10.173 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T10:45:10.173 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:45:10.173 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T10:45:10.227 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T10:45:10.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T10:45:10.228 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:45:10.282 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:45:10.348 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:45:10.406 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:45:10.407 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:45:10.407 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:45:10.407 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:45:10.407 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:45:10.407 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:45:10.408 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:45:10.408 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:45:10.408 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:45:10.408 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:45:10.409 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:45:10.409 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:45:10.409 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:45:10.409 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:45:10.409 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:45:10.410 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:45:10.410 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:45:10.410 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:45:10.410 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:45:10.410 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:45:10.411 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:45:10.411 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:45:10.411 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:45:10.411 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:45:10.411 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:45:10.412 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:45:10.412 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:45:10.412 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:45:10.412 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:45:10.412 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:45:10.413 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:45:10.413 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:45:10.413 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:45:10.413 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:45:10.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:10.414 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:45:10.461 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T10:45:10.461 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T10:45:10.504 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T10:45:10.530 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T10:45:10.585 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T10:45:10.585 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T10:45:10.586 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T10:45:10.586 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T10:45:10.586 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T10:45:10.586 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T10:45:10.587 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T10:45:10.587 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T10:45:10.587 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T10:45:10.587 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T10:45:10.587 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T10:45:10.588 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T10:45:10.588 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T10:45:10.588 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T10:45:10.588 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T10:45:10.588 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T10:45:10.589 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T10:45:10.589 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T10:45:10.589 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T10:45:10.589 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T10:45:10.589 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T10:45:10.590 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T10:45:10.590 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T10:45:10.590 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T10:45:10.590 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T10:45:10.590 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T10:45:10.591 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T10:45:10.591 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T10:45:10.591 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T10:45:10.591 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T10:45:10.591 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:45:10.592 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T10:45:10.592 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:45:10.592 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T10:45:10.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:10.593 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T10:45:10.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T10:45:10.586+0000 7f396303a480 -1 init, newargv = 0x55a96e68d1f0 newargc=15 2023-10-18T10:45:10.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[136921]: starting ceph client 2023-10-18T10:45:10.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[136921]: starting fuse 2023-10-18T10:45:10.610 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T10:45:10.610 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T10:45:10.611 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T10:45:10.611 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T10:45:10.611 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T10:45:10.612 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T10:45:10.612 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T10:45:10.612 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:45:10.613 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T10:45:10.613 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T10:45:10.613 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T10:45:10.614 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T10:45:10.614 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:45:10.615 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T10:45:10.615 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T10:45:10.615 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T10:45:10.616 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T10:45:10.616 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:45:10.617 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T10:45:10.617 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T10:45:10.617 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T10:45:10.618 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T10:45:10.618 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T10:45:10.618 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T10:45:10.619 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T10:45:10.619 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T10:45:10.619 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T10:45:10.620 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T10:45:10.620 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T10:45:10.730 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.136921.asok 2023-10-18T10:45:10.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T10:45:10.740 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.136921.asok 2023-10-18T10:45:10.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:10.741 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.136921.asok status 2023-10-18T10:45:10.874 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T10:45:10.874 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T10:45:10.874 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T10:45:10.875 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T10:45:10.875 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T10:45:10.875 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T10:45:10.875 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T10:45:10.875 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "136921", 2023-10-18T10:45:10.876 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T10:45:10.876 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:45:10.876 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T10:45:10.876 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T10:45:10.876 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 9948, 2023-10-18T10:45:10.877 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T10:45:10.877 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T10:45:10.877 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T10:45:10.877 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 9948 2023-10-18T10:45:10.877 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:45:10.878 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:45:10.878 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:45:10.878 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:45:10.878 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 900792781 2023-10-18T10:45:10.879 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:45:10.879 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:45:10.879 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T10:45:10.879 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T10:45:10.879 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T10:45:10.880 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 900792781 2023-10-18T10:45:10.880 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T10:45:10.880 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.9948 192.168.0.1:0/900792781", 2023-10-18T10:45:10.880 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/900792781", 2023-10-18T10:45:10.880 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T10:45:10.881 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 60, 2023-10-18T10:45:10.881 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 156, 2023-10-18T10:45:10.881 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T10:45:10.881 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T10:45:10.881 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T10:45:10.882 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T10:45:10.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:10.883 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T10:45:10.883 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T10:45:10.899 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T10:45:10.899 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T10:45:10.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:10.900 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T10:45:10.967 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T10:45:11.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.107+0000 7fbbedda9640 1 -- 172.21.15.125:0/4172545062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe80cf9b0 msgr2=0x7fbbe80b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:11.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.107+0000 7fbbedda9640 1 --2- 172.21.15.125:0/4172545062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe80cf9b0 0x7fbbe80b00f0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0005fd0 tx=0x7fbbe002c170 comp rx=0 tx=0).stop 2023-10-18T10:45:11.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.107+0000 7fbbedda9640 1 -- 172.21.15.125:0/4172545062 shutdown_connections 2023-10-18T10:45:11.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.107+0000 7fbbedda9640 1 --2- 172.21.15.125:0/4172545062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe80cf9b0 0x7fbbe80b00f0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.107+0000 7fbbedda9640 1 -- 172.21.15.125:0/4172545062 >> 172.21.15.125:0/4172545062 conn(0x7fbbe808a650 msgr2=0x7fbbe8099f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:11.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.107+0000 7fbbedda9640 1 -- 172.21.15.125:0/4172545062 shutdown_connections 2023-10-18T10:45:11.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.107+0000 7fbbedda9640 1 -- 172.21.15.125:0/4172545062 wait complete. 2023-10-18T10:45:11.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.108+0000 7fbbedda9640 1 Processor -- start 2023-10-18T10:45:11.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.108+0000 7fbbedda9640 1 -- start start 2023-10-18T10:45:11.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbedda9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbe80cf9b0 0x7fbbe80c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:11.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbedda9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe815b200 0x7fbbe81ef070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:11.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbedda9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbbe815eba0 0x7fbbe81f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:11.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbedda9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe81f2ea0 con 0x7fbbe815b200 2023-10-18T10:45:11.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbedda9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c47b0 con 0x7fbbe80cf9b0 2023-10-18T10:45:11.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbedda9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe815d6d0 con 0x7fbbe815eba0 2023-10-18T10:45:11.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe815b200 0x7fbbe81ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:11.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe815b200 0x7fbbe81ef070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40894/0 (socket says 172.21.15.125:40894) 2023-10-18T10:45:11.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbdeffd640 1 -- 172.21.15.125:0/1934801849 learned_addr learned my addr 172.21.15.125:0/1934801849 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:11.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbdeffd640 1 -- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbbe815eba0 msgr2=0x7fbbe81f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:11.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.109+0000 7fbbdeffd640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbbe815eba0 0x7fbbe81f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbdeffd640 1 -- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbe80cf9b0 msgr2=0x7fbbe80c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:11.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbdeffd640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbe80cf9b0 0x7fbbe80c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbdeffd640 1 -- 172.21.15.125:0/1934801849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0009840 con 0x7fbbe815b200 2023-10-18T10:45:11.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbdeffd640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe815b200 0x7fbbe81ef070 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c3e0 tx=0x7fbbd000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:11.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbe4ff9640 1 -- 172.21.15.125:0/1934801849 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00035f0 con 0x7fbbe815b200 2023-10-18T10:45:11.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbe4ff9640 1 -- 172.21.15.125:0/1934801849 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbbd0003750 con 0x7fbbe815b200 2023-10-18T10:45:11.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe815d9b0 con 0x7fbbe815b200 2023-10-18T10:45:11.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbe4ff9640 1 -- 172.21.15.125:0/1934801849 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0003b80 con 0x7fbbe815b200 2023-10-18T10:45:11.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.110+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe80c4d50 con 0x7fbbe815b200 2023-10-18T10:45:11.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.111+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe8066930 con 0x7fbbe815b200 2023-10-18T10:45:11.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.111+0000 7fbbe4ff9640 1 -- 172.21.15.125:0/1934801849 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbbd0003ce0 con 0x7fbbe815b200 2023-10-18T10:45:11.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.111+0000 7fbbe4ff9640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fbbc806af60 0x7fbbc806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:11.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.112+0000 7fbbe4ff9640 1 -- 172.21.15.125:0/1934801849 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fbbd00890c0 con 0x7fbbe815b200 2023-10-18T10:45:11.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.112+0000 7fbbe77fe640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fbbc806af60 0x7fbbc806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:11.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.112+0000 7fbbe77fe640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fbbc806af60 0x7fbbc806d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8001560 tx=0x7fbbd8007990 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:11.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.116+0000 7fbbe4ff9640 1 -- 172.21.15.125:0/1934801849 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbd0053f00 con 0x7fbbe815b200 2023-10-18T10:45:11.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.303+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 --> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbe80cf7d0 con 0x7fbbc806af60 2023-10-18T10:45:11.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.305+0000 7fbbe4ff9640 1 -- 172.21.15.125:0/1934801849 <== mgr.7340 v2:172.21.15.139:6832/3182280683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbbe80cf7d0 con 0x7fbbc806af60 2023-10-18T10:45:11.307 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T10:45:11.307 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T10:45:11.307 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T10:45:11.308 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T10:45:11.308 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T10:45:11.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fbbc806af60 msgr2=0x7fbbc806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:11.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fbbc806af60 0x7fbbc806d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8001560 tx=0x7fbbd8007990 comp rx=0 tx=0).stop 2023-10-18T10:45:11.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe815b200 msgr2=0x7fbbe81ef070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:11.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe815b200 0x7fbbe81ef070 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c3e0 tx=0x7fbbd000c8a0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 shutdown_connections 2023-10-18T10:45:11.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fbbc806af60 0x7fbbc806d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbbe815eba0 0x7fbbe81f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbe815b200 0x7fbbe81ef070 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 --2- 172.21.15.125:0/1934801849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbe80cf9b0 0x7fbbe80c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 >> 172.21.15.125:0/1934801849 conn(0x7fbbe808a650 msgr2=0x7fbbe80cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:11.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 shutdown_connections 2023-10-18T10:45:11.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.307+0000 7fbbedda9640 1 -- 172.21.15.125:0/1934801849 wait complete. 2023-10-18T10:45:11.319 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T10:45:11.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2506906994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0cf340 msgr2=0x7f60dc0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:11.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2506906994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0cf340 0x7f60dc0d1910 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f60dc06c850 tx=0x7f60cc02c1d0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2506906994 shutdown_connections 2023-10-18T10:45:11.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2506906994 >> v2:172.21.15.139:3300/0 conn(0x7f60dc0ceb40 0x7f60dc0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2506906994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60dc15b2a0 0x7f60dc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2506906994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0cf340 0x7f60dc0d1910 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2506906994 >> 172.21.15.125:0/2506906994 conn(0x7f60dc0d0d60 msgr2=0x7f60dc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:11.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2506906994 shutdown_connections 2023-10-18T10:45:11.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.421+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2506906994 wait complete. 2023-10-18T10:45:11.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.422+0000 7f60e1a1a640 1 Processor -- start 2023-10-18T10:45:11.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.422+0000 7f60e1a1a640 1 -- start start 2023-10-18T10:45:11.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60e1a1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0ceb40 0x7f60dc0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:11.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60e1a1a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60dc0cf340 0x7f60dc1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:11.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60e1a1a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60dc15b2a0 0x7f60dc15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:11.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60e1a1a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc0b47b0 con 0x7f60dc0ceb40 2023-10-18T10:45:11.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60e1a1a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc159930 con 0x7f60dc0cf340 2023-10-18T10:45:11.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60e1a1a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc15cc60 con 0x7f60dc15b2a0 2023-10-18T10:45:11.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60daffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0ceb40 0x7f60dc0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:11.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60daffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0ceb40 0x7f60dc0c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40908/0 (socket says 172.21.15.125:40908) 2023-10-18T10:45:11.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60daffd640 1 -- 172.21.15.125:0/2823840298 learned_addr learned my addr 172.21.15.125:0/2823840298 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:11.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60da7fc640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60dc0cf340 0x7f60dc1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:11.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60db7fe640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60dc15b2a0 0x7f60dc15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:11.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60daffd640 1 -- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60dc15b2a0 msgr2=0x7f60dc15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:11.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60daffd640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60dc15b2a0 0x7f60dc15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60daffd640 1 -- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60dc0cf340 msgr2=0x7f60dc1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:11.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60daffd640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60dc0cf340 0x7f60dc1593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:11.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60daffd640 1 -- 172.21.15.125:0/2823840298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc009900 con 0x7f60dc0ceb40 2023-10-18T10:45:11.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60db7fe640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60dc15b2a0 0x7f60dc15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:11.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.423+0000 7f60da7fc640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60dc0cf340 0x7f60dc1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T10:45:11.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.424+0000 7f60daffd640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0ceb40 0x7f60dc0c7e30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009d60 tx=0x7f60cc035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:11.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.424+0000 7f60c3fff640 1 -- 172.21.15.125:0/2823840298 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc034440 con 0x7f60dc0ceb40 2023-10-18T10:45:11.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.424+0000 7f60c3fff640 1 -- 172.21.15.125:0/2823840298 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f60cc004560 con 0x7f60dc0ceb40 2023-10-18T10:45:11.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.424+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc0b6c00 con 0x7f60dc0ceb40 2023-10-18T10:45:11.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.424+0000 7f60c3fff640 1 -- 172.21.15.125:0/2823840298 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc040450 con 0x7f60dc0ceb40 2023-10-18T10:45:11.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.424+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc0b40f0 con 0x7f60dc0ceb40 2023-10-18T10:45:11.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.425+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60dc0b48f0 con 0x7f60dc0ceb40 2023-10-18T10:45:11.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.425+0000 7f60c3fff640 1 -- 172.21.15.125:0/2823840298 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 81) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f60cc00a040 con 0x7f60dc0ceb40 2023-10-18T10:45:11.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.426+0000 7f60c3fff640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f60c806af60 0x7f60c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:11.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.426+0000 7f60c3fff640 1 -- 172.21.15.125:0/2823840298 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f60cc0ac7d0 con 0x7f60dc0ceb40 2023-10-18T10:45:11.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.426+0000 7f60da7fc640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f60c806af60 0x7f60c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:11.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.426+0000 7f60da7fc640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f60c806af60 0x7f60c806d420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60d000a870 tx=0x7f60d0008040 comp rx=0 tx=0).ready entity=mgr.7340 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:11.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.430+0000 7f60c3fff640 1 -- 172.21.15.125:0/2823840298 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60dc0b48f0 con 0x7f60dc0ceb40 2023-10-18T10:45:11.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:11.641+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f60dc0b48f0 con 0x7f60dc0ceb40 2023-10-18T10:45:12.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.434+0000 7f60c3fff640 1 -- 172.21.15.125:0/2823840298 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v82) v1 ==== 92+0+0 (secure 0 0 0) 0x7f60dc0b48f0 con 0x7f60dc0ceb40 2023-10-18T10:45:12.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f60c806af60 msgr2=0x7f60c806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f60c806af60 0x7f60c806d420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60d000a870 tx=0x7f60d0008040 comp rx=0 tx=0).stop 2023-10-18T10:45:12.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0ceb40 msgr2=0x7f60dc0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0ceb40 0x7f60dc0c7e30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009d60 tx=0x7f60cc035db0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 shutdown_connections 2023-10-18T10:45:12.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f60c806af60 0x7f60c806d420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60dc0cf340 0x7f60dc1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f60dc15b2a0 0x7f60dc15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 --2- 172.21.15.125:0/2823840298 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f60dc0ceb40 0x7f60dc0c7e30 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.437+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 >> 172.21.15.125:0/2823840298 conn(0x7f60dc0d0d60 msgr2=0x7f60dc0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:12.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.438+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 shutdown_connections 2023-10-18T10:45:12.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.438+0000 7f60e1a1a640 1 -- 172.21.15.125:0/2823840298 wait complete. 2023-10-18T10:45:12.454 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T10:45:12.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.566+0000 7f300c11d640 1 -- 172.21.15.125:0/2464629165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ff0004d00 msgr2=0x7f2ff0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.566+0000 7f300c11d640 1 --2- 172.21.15.125:0/2464629165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ff0004d00 0x7f2ff0005180 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009ed0 tx=0x7f2ff00057f0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.566+0000 7f300a6ba640 1 -- 172.21.15.125:0/2464629165 reap_dead start 2023-10-18T10:45:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.566+0000 7f300c11d640 1 -- 172.21.15.125:0/2464629165 shutdown_connections 2023-10-18T10:45:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.566+0000 7f300c11d640 1 -- 172.21.15.125:0/2464629165 >> 172.21.15.125:0/2464629165 conn(0x7f300409a030 msgr2=0x7f30040d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.566+0000 7f300c11d640 1 -- 172.21.15.125:0/2464629165 shutdown_connections 2023-10-18T10:45:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.566+0000 7f300c11d640 1 -- 172.21.15.125:0/2464629165 wait complete. 2023-10-18T10:45:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f300c11d640 1 Processor -- start 2023-10-18T10:45:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f300c11d640 1 -- start start 2023-10-18T10:45:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f300c11d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff0004d00 0x7f30040caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f300c11d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30041490b0 0x7f300406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f300c11d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f300406b600 0x7f30040c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f300c11d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300406ba80 con 0x7f30041490b0 2023-10-18T10:45:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f300c11d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30040c8400 con 0x7f2ff0004d00 2023-10-18T10:45:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f300c11d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30040c7df0 con 0x7f300406b600 2023-10-18T10:45:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f30096b8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30041490b0 0x7f300406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f30096b8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30041490b0 0x7f300406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40940/0 (socket says 172.21.15.125:40940) 2023-10-18T10:45:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.568+0000 7f30096b8640 1 -- 172.21.15.125:0/2198529479 learned_addr learned my addr 172.21.15.125:0/2198529479 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f3009eb9640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff0004d00 0x7f30040caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f300a6ba640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f300406b600 0x7f30040c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f30096b8640 1 -- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f300406b600 msgr2=0x7f30040c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f30096b8640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f300406b600 0x7f30040c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f30096b8640 1 -- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff0004d00 msgr2=0x7f30040caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f30096b8640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff0004d00 0x7f30040caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f30096b8640 1 -- 172.21.15.125:0/2198529479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30040b38e0 con 0x7f30041490b0 2023-10-18T10:45:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f30096b8640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30041490b0 0x7f300406b0c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4003e70 tx=0x7f2ff4001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f2ffa7fc640 1 -- 172.21.15.125:0/2198529479 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4011840 con 0x7f30041490b0 2023-10-18T10:45:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f2ffa7fc640 1 -- 172.21.15.125:0/2198529479 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ff40119a0 con 0x7f30041490b0 2023-10-18T10:45:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f2ffa7fc640 1 -- 172.21.15.125:0/2198529479 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4011db0 con 0x7f30041490b0 2023-10-18T10:45:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30040b3bc0 con 0x7f30041490b0 2023-10-18T10:45:12.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.569+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30040c8090 con 0x7f30041490b0 2023-10-18T10:45:12.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.570+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30040675f0 con 0x7f30041490b0 2023-10-18T10:45:12.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.571+0000 7f2ffa7fc640 1 -- 172.21.15.125:0/2198529479 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 82) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f2ff40103c0 con 0x7f30041490b0 2023-10-18T10:45:12.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.571+0000 7f2ffa7fc640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f2fe806afb0 0x7f2fe806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:12.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.571+0000 7f2ffa7fc640 1 -- 172.21.15.125:0/2198529479 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f2ff408a970 con 0x7f30041490b0 2023-10-18T10:45:12.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.571+0000 7f3009eb9640 1 -- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f2fe806afb0 msgr2=0x7f2fe806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3182280683 2023-10-18T10:45:12.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.571+0000 7f3009eb9640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f2fe806afb0 0x7f2fe806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:45:12.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.575+0000 7f2ffa7fc640 1 -- 172.21.15.125:0/2198529479 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ff4055750 con 0x7f30041490b0 2023-10-18T10:45:12.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.749+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f30040b2eb0 con 0x7f30041490b0 2023-10-18T10:45:12.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.753+0000 7f2ffa7fc640 1 -- 172.21.15.125:0/2198529479 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v60)=0 v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2ff4015070 con 0x7f30041490b0 2023-10-18T10:45:12.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f2fe806afb0 msgr2=0x7f2fe806d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:45:12.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f2fe806afb0 0x7f2fe806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30041490b0 msgr2=0x7f300406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30041490b0 0x7f300406b0c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4003e70 tx=0x7f2ff4001770 comp rx=0 tx=0).stop 2023-10-18T10:45:12.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 shutdown_connections 2023-10-18T10:45:12.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7f2fe806afb0 0x7f2fe806d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f300406b600 0x7f30040c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30041490b0 0x7f300406b0c0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 --2- 172.21.15.125:0/2198529479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff0004d00 0x7f30040caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 >> 172.21.15.125:0/2198529479 conn(0x7f300409a030 msgr2=0x7f300406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:12.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 shutdown_connections 2023-10-18T10:45:12.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.755+0000 7f300c11d640 1 -- 172.21.15.125:0/2198529479 wait complete. 2023-10-18T10:45:12.768 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T10:45:12.823 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:12.821+0000 7fa5d6637040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:45:12.835 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:12.834+0000 7fb466e25040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T10:45:12.867 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:12.866+0000 7fa5d6637040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:45:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.873+0000 7fef64f4c640 1 -- 172.21.15.125:0/2422733641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 msgr2=0x7fef600d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.873+0000 7fef64f4c640 1 --2- 172.21.15.125:0/2422733641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 0x7fef600d1910 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fef48009830 tx=0x7fef4802bd60 comp rx=0 tx=0).stop 2023-10-18T10:45:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.873+0000 7fef64f4c640 1 -- 172.21.15.125:0/2422733641 shutdown_connections 2023-10-18T10:45:12.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.873+0000 7fef64f4c640 1 --2- 172.21.15.125:0/2422733641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 0x7fef600d1910 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.873+0000 7fef64f4c640 1 -- 172.21.15.125:0/2422733641 >> 172.21.15.125:0/2422733641 conn(0x7fef600d0d60 msgr2=0x7fef600d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.873+0000 7fef64f4c640 1 -- 172.21.15.125:0/2422733641 shutdown_connections 2023-10-18T10:45:12.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.873+0000 7fef64f4c640 1 -- 172.21.15.125:0/2422733641 wait complete. 2023-10-18T10:45:12.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.874+0000 7fef64f4c640 1 Processor -- start 2023-10-18T10:45:12.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef64f4c640 1 -- start start 2023-10-18T10:45:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef64f4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef600ceb40 0x7fef600c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef64f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef600cf340 0x7fef600b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef64f4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 0x7fef600b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef64f4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef601f1580 con 0x7fef6015b2a0 2023-10-18T10:45:12.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef64f4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6015ba70 con 0x7fef600cf340 2023-10-18T10:45:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef64f4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef600b6820 con 0x7fef600ceb40 2023-10-18T10:45:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef5ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 0x7fef600b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef5ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 0x7fef600b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40948/0 (socket says 172.21.15.125:40948) 2023-10-18T10:45:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef5ed9d640 1 -- 172.21.15.125:0/4057327444 learned_addr learned my addr 172.21.15.125:0/4057327444 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:12.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.875+0000 7fef55d9b640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef600cf340 0x7fef600b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef5e59c640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef600ceb40 0x7fef600c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef5ed9d640 1 -- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef600ceb40 msgr2=0x7fef600c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef5ed9d640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef600ceb40 0x7fef600c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef5ed9d640 1 -- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef600cf340 msgr2=0x7fef600b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:12.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef5ed9d640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef600cf340 0x7fef600b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef5ed9d640 1 -- 172.21.15.125:0/4057327444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48009490 con 0x7fef6015b2a0 2023-10-18T10:45:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef5ed9d640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 0x7fef600b3fe0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fef5800c7a0 tx=0x7fef5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef577fe640 1 -- 172.21.15.125:0/4057327444 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58003b70 con 0x7fef6015b2a0 2023-10-18T10:45:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef577fe640 1 -- 172.21.15.125:0/4057327444 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef58003cf0 con 0x7fef6015b2a0 2023-10-18T10:45:12.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef600b6b00 con 0x7fef6015b2a0 2023-10-18T10:45:12.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef577fe640 1 -- 172.21.15.125:0/4057327444 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58003ea0 con 0x7fef6015b2a0 2023-10-18T10:45:12.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.876+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef600c9330 con 0x7fef6015b2a0 2023-10-18T10:45:12.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.877+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef600690e0 con 0x7fef6015b2a0 2023-10-18T10:45:12.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.878+0000 7fef577fe640 1 -- 172.21.15.125:0/4057327444 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 82) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fef5800aab0 con 0x7fef6015b2a0 2023-10-18T10:45:12.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.878+0000 7fef577fe640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fef4006afb0 0x7fef4006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:12.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.878+0000 7fef577fe640 1 -- 172.21.15.125:0/4057327444 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fef58089860 con 0x7fef6015b2a0 2023-10-18T10:45:12.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.878+0000 7fef5e59c640 1 -- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fef4006afb0 msgr2=0x7fef4006d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3182280683 2023-10-18T10:45:12.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.878+0000 7fef5e59c640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fef4006afb0 0x7fef4006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:45:12.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:12.882+0000 7fef577fe640 1 -- 172.21.15.125:0/4057327444 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef580546d0 con 0x7fef6015b2a0 2023-10-18T10:45:12.885 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:12.882+0000 7fb466e25040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T10:45:13.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.055+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fef600b0b40 con 0x7fef6015b2a0 2023-10-18T10:45:13.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.058+0000 7fef577fe640 1 -- 172.21.15.125:0/4057327444 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v61)=0 v61) v1 ==== 133+0+0 (secure 0 0 0) 0x7fef5805e020 con 0x7fef6015b2a0 2023-10-18T10:45:13.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fef4006afb0 msgr2=0x7fef4006d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:45:13.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fef4006afb0 0x7fef4006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:13.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 msgr2=0x7fef600b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:13.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 0x7fef600b3fe0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fef5800c7a0 tx=0x7fef5800cc60 comp rx=0 tx=0).stop 2023-10-18T10:45:13.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 shutdown_connections 2023-10-18T10:45:13.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fef4006afb0 0x7fef4006d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:13.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef600cf340 0x7fef600b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:13.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef6015b2a0 0x7fef600b3fe0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:13.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 --2- 172.21.15.125:0/4057327444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef600ceb40 0x7fef600c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:13.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 >> 172.21.15.125:0/4057327444 conn(0x7fef600d0d60 msgr2=0x7fef60150f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:13.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 shutdown_connections 2023-10-18T10:45:13.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.060+0000 7fef64f4c640 1 -- 172.21.15.125:0/4057327444 wait complete. 2023-10-18T10:45:13.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T10:45:13.074 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T10:45:13.094 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1m 2023-10-18T10:45:13.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.239+0000 7fed97f3c640 1 -- 172.21.15.125:0/4191869838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900ce420 msgr2=0x7fed900cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.239+0000 7fed97f3c640 1 --2- 172.21.15.125:0/4191869838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900ce420 0x7fed900cd6b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009830 tx=0x7fed8c02be80 comp rx=0 tx=0).stop 2023-10-18T10:45:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.239+0000 7fed97f3c640 1 -- 172.21.15.125:0/4191869838 shutdown_connections 2023-10-18T10:45:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.239+0000 7fed97f3c640 1 --2- 172.21.15.125:0/4191869838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900ce420 0x7fed900cd6b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.239+0000 7fed97f3c640 1 -- 172.21.15.125:0/4191869838 >> 172.21.15.125:0/4191869838 conn(0x7fed90099ff0 msgr2=0x7fed900d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.239+0000 7fed97f3c640 1 -- 172.21.15.125:0/4191869838 shutdown_connections 2023-10-18T10:45:13.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.239+0000 7fed97f3c640 1 -- 172.21.15.125:0/4191869838 wait complete. 2023-10-18T10:45:13.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.240+0000 7fed97f3c640 1 Processor -- start 2023-10-18T10:45:13.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.240+0000 7fed97f3c640 1 -- start start 2023-10-18T10:45:13.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed97f3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900cdbf0 0x7fed9014d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:13.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed97f3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed900ce420 0x7fed901f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:13.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed97f3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed9015b180 0x7fed900b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:13.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed97f3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9014ce80 con 0x7fed900cdbf0 2023-10-18T10:45:13.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed97f3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed901f57b0 con 0x7fed9015b180 2023-10-18T10:45:13.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed97f3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed901f51f0 con 0x7fed900ce420 2023-10-18T10:45:13.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed95cd8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900cdbf0 0x7fed9014d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:13.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed95cd8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900cdbf0 0x7fed9014d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40974/0 (socket says 172.21.15.125:40974) 2023-10-18T10:45:13.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed95cd8640 1 -- 172.21.15.125:0/3262479605 learned_addr learned my addr 172.21.15.125:0/3262479605 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:13.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed964d9640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed9015b180 0x7fed900b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:13.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.241+0000 7fed954d7640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed900ce420 0x7fed901f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:13.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed95cd8640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed900ce420 msgr2=0x7fed901f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:13.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed95cd8640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed900ce420 0x7fed901f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:13.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed95cd8640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed9015b180 msgr2=0x7fed900b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:13.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed95cd8640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed9015b180 0x7fed900b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed95cd8640 1 -- 172.21.15.125:0/3262479605 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c009490 con 0x7fed900cdbf0 2023-10-18T10:45:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed954d7640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed900ce420 0x7fed901f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T10:45:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed95cd8640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900cdbf0 0x7fed9014d2a0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fed8000c400 tx=0x7fed8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800052d0 con 0x7fed900cdbf0 2023-10-18T10:45:13.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed800035f0 con 0x7fed900cdbf0 2023-10-18T10:45:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed901f54d0 con 0x7fed900cdbf0 2023-10-18T10:45:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80003a00 con 0x7fed900cdbf0 2023-10-18T10:45:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.242+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9014d7e0 con 0x7fed900cdbf0 2023-10-18T10:45:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.243+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed90066960 con 0x7fed900cdbf0 2023-10-18T10:45:13.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.243+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 82) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fed80003b60 con 0x7fed900cdbf0 2023-10-18T10:45:13.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.244+0000 7fed867fc640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 0x7fed7406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:13.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.244+0000 7fed954d7640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 msgr2=0x7fed7406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3182280683 2023-10-18T10:45:13.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.244+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fed80089090 con 0x7fed900cdbf0 2023-10-18T10:45:13.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.244+0000 7fed954d7640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 0x7fed7406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T10:45:13.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.248+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed80053f90 con 0x7fed900cdbf0 2023-10-18T10:45:13.258 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.257+0000 7fa5d6637040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:45:13.293 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.291+0000 7fb466e25040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T10:45:13.302 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.300+0000 7fa5d6637040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:45:13.338 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.336+0000 7fb466e25040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T10:45:13.344 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.343+0000 7fa5d6637040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:45:13.380 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.378+0000 7fb466e25040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T10:45:13.396 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.394+0000 7fa5d6637040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:45:13.431 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.429+0000 7fb466e25040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T10:45:13.439 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.437+0000 7fa5d6637040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:45:13.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.438+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 --> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fed9014c920 con 0x7fed7406afb0 2023-10-18T10:45:13.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.445+0000 7fed954d7640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 msgr2=0x7fed7406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3182280683 2023-10-18T10:45:13.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.445+0000 7fed954d7640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 0x7fed7406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T10:45:13.474 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.472+0000 7fb466e25040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T10:45:13.537 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.536+0000 7fa5d6637040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:45:13.572 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.570+0000 7fb466e25040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T10:45:13.591 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.589+0000 7fa5d6637040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:45:13.626 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.624+0000 7fb466e25040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T10:45:13.633 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.632+0000 7fa5d6637040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:45:13.668 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.666+0000 7fb466e25040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T10:45:13.682 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.680+0000 7fa5d6637040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:45:13.715 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.713+0000 7fb466e25040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T10:45:13.725 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:13.723+0000 7fa5d6637040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:45:13.756 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:13.755+0000 7fb466e25040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T10:45:13.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.846+0000 7fed954d7640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 msgr2=0x7fed7406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3182280683 2023-10-18T10:45:13.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:13.846+0000 7fed954d7640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 0x7fed7406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T10:45:14.040 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.038+0000 7fa5d6637040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:45:14.075 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.073+0000 7fb466e25040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T10:45:14.171 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.170+0000 7fa5d6637040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:45:14.205 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.204+0000 7fb466e25040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T10:45:14.264 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.263+0000 7fa5d6637040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:45:14.298 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.296+0000 7fb466e25040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T10:45:14.307 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.306+0000 7fa5d6637040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:45:14.340 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.338+0000 7fb466e25040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T10:45:14.404 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.402+0000 7fa5d6637040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:45:14.434 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.432+0000 7fb466e25040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:45:14.494 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.493+0000 7fa5d6637040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:45:14.524 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.522+0000 7fb466e25040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T10:45:14.631 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.630+0000 7fa5d6637040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:45:14.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:14.647+0000 7fed954d7640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 msgr2=0x7fed7406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3182280683 2023-10-18T10:45:14.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:14.647+0000 7fed954d7640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 0x7fed7406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T10:45:14.661 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.659+0000 7fb466e25040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T10:45:14.673 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.672+0000 7fa5d6637040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:45:14.703 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.701+0000 7fb466e25040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T10:45:14.778 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:14.776+0000 7fa5d6637040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:45:14.806 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:14.805+0000 7fb466e25040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T10:45:15.388 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.386+0000 7fa5d6637040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:45:15.428 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:15.426+0000 7fb466e25040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T10:45:15.652 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.651+0000 7fa5d6637040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:45:15.690 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:15.689+0000 7fb466e25040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T10:45:15.701 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.700+0000 7fa5d6637040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:45:15.738 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:15.737+0000 7fb466e25040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T10:45:15.802 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.800+0000 7fa5d6637040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:45:15.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:15.821+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 83) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fed80054360 con 0x7fed900cdbf0 2023-10-18T10:45:15.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:15.822+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 msgr2=0x7fed7406d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T10:45:15.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:15.822+0000 7fed867fc640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/3182280683,v1:172.21.15.139:6833/3182280683] conn(0x7fed7406afb0 0x7fed7406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:15.842 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T10:45:15.840+0000 7fb466e25040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T10:45:15.922 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.918+0000 7fa553116640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.922 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.918+0000 7fa553116640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.922 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.918+0000 7fa553116640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.923 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.918+0000 7fa553116640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.923 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.918+0000 7fa553116640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.923 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.919+0000 7fa55615c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.924 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.919+0000 7fa55615c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.924 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.919+0000 7fa55615c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.924 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.919+0000 7fa55615c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:15.924 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:45:15.919+0000 7fa55615c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:45:16.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.827+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 84) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fed80087420 con 0x7fed900cdbf0 2023-10-18T10:45:16.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.827+0000 7fed867fc640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fed740358b0 0x7fed740753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:16.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.827+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 --> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fed74035c90 con 0x7fed740358b0 2023-10-18T10:45:16.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.829+0000 7fed954d7640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fed740358b0 0x7fed740753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:16.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.831+0000 7fed954d7640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fed740358b0 0x7fed740753e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed8c036000 tx=0x7fed8c004730 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:16.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.885+0000 7fed867fc640 1 -- 172.21.15.125:0/3262479605 <== mgr.8557 v2:172.21.15.139:6832/1032201494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fed74035c90 con 0x7fed740358b0 2023-10-18T10:45:16.887 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1 2023-10-18T10:45:16.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fed740358b0 msgr2=0x7fed740753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:16.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed97f3c640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fed740358b0 0x7fed740753e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed8c036000 tx=0x7fed8c004730 comp rx=0 tx=0).stop 2023-10-18T10:45:16.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900cdbf0 msgr2=0x7fed9014d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:16.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed97f3c640 1 --2- 172.21.15.125:0/3262479605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed900cdbf0 0x7fed9014d2a0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fed8000c400 tx=0x7fed8000c8c0 comp rx=0 tx=0).stop 2023-10-18T10:45:16.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed964d9640 1 -- 172.21.15.125:0/3262479605 reap_dead start 2023-10-18T10:45:16.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 shutdown_connections 2023-10-18T10:45:16.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 >> 172.21.15.125:0/3262479605 conn(0x7fed90099ff0 msgr2=0x7fed9015c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:16.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 shutdown_connections 2023-10-18T10:45:16.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:16.888+0000 7fed97f3c640 1 -- 172.21.15.125:0/3262479605 wait complete. 2023-10-18T10:45:16.902 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-10-18T10:45:16.903 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2023-10-18T10:45:17.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.010+0000 7ffae2a44640 1 -- 172.21.15.125:0/121176978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 msgr2=0x7ffadc0affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.010+0000 7ffae2a44640 1 --2- 172.21.15.125:0/121176978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 0x7ffadc0affe0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0cbdc0 tx=0x7ffad402c260 comp rx=0 tx=0).stop 2023-10-18T10:45:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.010+0000 7ffae2a44640 1 -- 172.21.15.125:0/121176978 shutdown_connections 2023-10-18T10:45:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.010+0000 7ffae2a44640 1 --2- 172.21.15.125:0/121176978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 0x7ffadc0affe0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.010+0000 7ffae2a44640 1 --2- 172.21.15.125:0/121176978 >> v2:172.21.15.139:3300/0 conn(0x7ffadc141690 0x7ffadc0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.010+0000 7ffae2a44640 1 -- 172.21.15.125:0/121176978 >> 172.21.15.125:0/121176978 conn(0x7ffadc069930 msgr2=0x7ffadc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:17.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.010+0000 7ffae2a44640 1 -- 172.21.15.125:0/121176978 shutdown_connections 2023-10-18T10:45:17.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.010+0000 7ffae2a44640 1 -- 172.21.15.125:0/121176978 wait complete. 2023-10-18T10:45:17.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.011+0000 7ffae2a44640 1 Processor -- start 2023-10-18T10:45:17.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffae2a44640 1 -- start start 2023-10-18T10:45:17.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffae2a44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 0x7ffadc1e2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:17.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffae2a44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffadc13e520 0x7ffadc0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:17.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffae2a44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffadc141690 0x7ffadc0a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:17.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffae2a44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0a7220 con 0x7ffadc070920 2023-10-18T10:45:17.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffae2a44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0a6650 con 0x7ffadc13e520 2023-10-18T10:45:17.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffae2a44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0a68f0 con 0x7ffadc141690 2023-10-18T10:45:17.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffadbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 0x7ffadc1e2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:17.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffadbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 0x7ffadc1e2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40988/0 (socket says 172.21.15.125:40988) 2023-10-18T10:45:17.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffadbfff640 1 -- 172.21.15.125:0/26006178 learned_addr learned my addr 172.21.15.125:0/26006178 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:17.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.012+0000 7ffadb7fe640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffadc13e520 0x7ffadc0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:17.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffae0fe1640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffadc141690 0x7ffadc0a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:17.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffadbfff640 1 -- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffadc141690 msgr2=0x7ffadc0a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffadbfff640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffadc141690 0x7ffadc0a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffadbfff640 1 -- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffadc13e520 msgr2=0x7ffadc0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffadbfff640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffadc13e520 0x7ffadc0c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffadbfff640 1 -- 172.21.15.125:0/26006178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4009900 con 0x7ffadc070920 2023-10-18T10:45:17.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffadbfff640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 0x7ffadc1e2640 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00c450 tx=0x7ffacc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:17.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffad8ff9640 1 -- 172.21.15.125:0/26006178 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0052d0 con 0x7ffadc070920 2023-10-18T10:45:17.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffad8ff9640 1 -- 172.21.15.125:0/26006178 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffacc005430 con 0x7ffadc070920 2023-10-18T10:45:17.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc0a6bd0 con 0x7ffadc070920 2023-10-18T10:45:17.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffad8ff9640 1 -- 172.21.15.125:0/26006178 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc012890 con 0x7ffadc070920 2023-10-18T10:45:17.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.013+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc0c5fa0 con 0x7ffadc070920 2023-10-18T10:45:17.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.014+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc0aef90 con 0x7ffadc070920 2023-10-18T10:45:17.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.014+0000 7ffad8ff9640 1 -- 172.21.15.125:0/26006178 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 84) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ffacc0035f0 con 0x7ffadc070920 2023-10-18T10:45:17.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.015+0000 7ffad8ff9640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ffac806ace0 0x7ffac806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:17.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.015+0000 7ffad8ff9640 1 -- 172.21.15.125:0/26006178 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6401+0+0 (secure 0 0 0) 0x7ffacc089200 con 0x7ffadc070920 2023-10-18T10:45:17.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.015+0000 7ffadb7fe640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ffac806ace0 0x7ffac806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:17.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.015+0000 7ffadb7fe640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ffac806ace0 0x7ffac806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffad0005d70 tx=0x7ffad0005d00 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:17.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.019+0000 7ffad8ff9640 1 -- 172.21.15.125:0/26006178 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffacc054330 con 0x7ffadc070920 2023-10-18T10:45:17.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.204+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 --> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7ffadc0a7360 con 0x7ffac806ace0 2023-10-18T10:45:17.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.255+0000 7ffad8ff9640 1 -- 172.21.15.125:0/26006178 <== mgr.8557 v2:172.21.15.139:6832/1032201494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7ffad000ea60 con 0x7ffac806ace0 2023-10-18T10:45:17.257 INFO:teuthology.orchestra.run.smithi125.stdout:Retention added to path /snap_test_dir1 2023-10-18T10:45:17.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ffac806ace0 msgr2=0x7ffac806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ffac806ace0 0x7ffac806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffad0005d70 tx=0x7ffad0005d00 comp rx=0 tx=0).stop 2023-10-18T10:45:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 msgr2=0x7ffadc1e2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 0x7ffadc1e2640 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00c450 tx=0x7ffacc00c910 comp rx=0 tx=0).stop 2023-10-18T10:45:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 shutdown_connections 2023-10-18T10:45:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ffac806ace0 0x7ffac806d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffadc13e520 0x7ffadc0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffadc141690 0x7ffadc0a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 --2- 172.21.15.125:0/26006178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffadc070920 0x7ffadc1e2640 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 >> 172.21.15.125:0/26006178 conn(0x7ffadc069930 msgr2=0x7ffadc13d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:17.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 shutdown_connections 2023-10-18T10:45:17.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.257+0000 7ffae2a44640 1 -- 172.21.15.125:0/26006178 wait complete. 2023-10-18T10:45:17.269 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-10-18T10:45:17.270 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-18T10:45:17.270 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2023-10-18T10:45:17.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.374+0000 7f4b78ac2640 1 -- 172.21.15.125:0/4268670265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b60004d00 msgr2=0x7f4b60005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.374+0000 7f4b78ac2640 1 --2- 172.21.15.125:0/4268670265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b60004d00 0x7f4b60005180 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c001d80 tx=0x7f4b600057f0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.374+0000 7f4b72d9d640 1 -- 172.21.15.125:0/4268670265 reap_dead start 2023-10-18T10:45:17.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.374+0000 7f4b78ac2640 1 -- 172.21.15.125:0/4268670265 shutdown_connections 2023-10-18T10:45:17.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.374+0000 7f4b78ac2640 1 -- 172.21.15.125:0/4268670265 >> 172.21.15.125:0/4268670265 conn(0x7f4b740d0d60 msgr2=0x7f4b740d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:17.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.374+0000 7f4b78ac2640 1 -- 172.21.15.125:0/4268670265 shutdown_connections 2023-10-18T10:45:17.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.374+0000 7f4b78ac2640 1 -- 172.21.15.125:0/4268670265 wait complete. 2023-10-18T10:45:17.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.375+0000 7f4b78ac2640 1 Processor -- start 2023-10-18T10:45:17.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.375+0000 7f4b78ac2640 1 -- start start 2023-10-18T10:45:17.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b78ac2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b60004d00 0x7f4b7415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:17.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b78ac2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b740ca780 0x7f4b740aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:17.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b78ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b740c9ba0 0x7f4b740b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:17.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b78ac2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7415c1a0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b78ac2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740cac80 con 0x7f4b60004d00 2023-10-18T10:45:17.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b78ac2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7415cae0 con 0x7f4b740ca780 2023-10-18T10:45:17.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b740c9ba0 0x7f4b740b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:17.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b740c9ba0 0x7f4b740b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41002/0 (socket says 172.21.15.125:41002) 2023-10-18T10:45:17.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 -- 172.21.15.125:0/3426531431 learned_addr learned my addr 172.21.15.125:0/3426531431 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T10:45:17.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b6bfff640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b740ca780 0x7f4b740aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:17.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b7259c640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b60004d00 0x7f4b7415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:17.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 -- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b740ca780 msgr2=0x7f4b740aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b740ca780 0x7f4b740aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 -- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b60004d00 msgr2=0x7f4b7415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b60004d00 0x7f4b7415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 -- 172.21.15.125:0/3426531431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7415b800 con 0x7f4b740c9ba0 2023-10-18T10:45:17.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b72d9d640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b740c9ba0 0x7f4b740b4010 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c02c530 tx=0x7f4b6c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:17.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b6affd640 1 -- 172.21.15.125:0/3426531431 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c02ccd0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.376+0000 7f4b6affd640 1 -- 172.21.15.125:0/3426531431 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b6c0060b0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.377+0000 7f4b6affd640 1 -- 172.21.15.125:0/3426531431 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c006480 con 0x7f4b740c9ba0 2023-10-18T10:45:17.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.377+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7415bae0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.377+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b740af0f0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.377+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b740690e0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.378+0000 7f4b6affd640 1 -- 172.21.15.125:0/3426531431 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 84) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f4b6c0065e0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.378+0000 7f4b6affd640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f4b5406ace0 0x7f4b5406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T10:45:17.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.378+0000 7f4b6affd640 1 -- 172.21.15.125:0/3426531431 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f4b6c0aa6b0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.378+0000 7f4b7259c640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f4b5406ace0 0x7f4b5406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T10:45:17.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.379+0000 7f4b7259c640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f4b5406ace0 0x7f4b5406d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009b90 tx=0x7f4b64005f90 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T10:45:17.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.383+0000 7f4b6affd640 1 -- 172.21.15.125:0/3426531431 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b6c0757e0 con 0x7f4b740c9ba0 2023-10-18T10:45:17.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.566+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 --> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b740d0b40 con 0x7f4b5406ace0 2023-10-18T10:45:17.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.573+0000 7f4b6affd640 1 -- 172.21.15.125:0/3426531431 <== mgr.8557 v2:172.21.15.139:6832/1032201494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f4b740d0b40 con 0x7f4b5406ace0 2023-10-18T10:45:17.575 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T10:45:17.575 INFO:teuthology.orchestra.run.smithi125.stdout:{"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-18T10:45:17.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f4b5406ace0 msgr2=0x7f4b5406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f4b5406ace0 0x7f4b5406d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009b90 tx=0x7f4b64005f90 comp rx=0 tx=0).stop 2023-10-18T10:45:17.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b740c9ba0 msgr2=0x7f4b740b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T10:45:17.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b740c9ba0 0x7f4b740b4010 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c02c530 tx=0x7f4b6c003ba0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 shutdown_connections 2023-10-18T10:45:17.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b740c9ba0 0x7f4b740b4010 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b740ca780 0x7f4b740aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f4b5406ace0 0x7f4b5406d1a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 --2- 172.21.15.125:0/3426531431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b60004d00 0x7f4b7415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T10:45:17.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 >> 172.21.15.125:0/3426531431 conn(0x7f4b740d0d60 msgr2=0x7f4b7415a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T10:45:17.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 shutdown_connections 2023-10-18T10:45:17.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T10:45:17.575+0000 7f4b78ac2640 1 -- 172.21.15.125:0/3426531431 wait complete. 2023-10-18T10:45:17.588 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-18T10:45:17.588 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1m'], 'retention': [{'n': 55}]} 2023-10-18T10:45:36.847 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:45:36.851 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:46:00.266 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:46:00.264+0000 7fa5450fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:46:00.266 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:46:00.264+0000 7fa5450fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:46:00.267 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:46:00.264+0000 7fa5450fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:46:00.267 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:46:00.264+0000 7fa5450fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:46:00.267 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T10:46:00.264+0000 7fa5450fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T10:46:06.957 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:46:06.960 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:46:37.088 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:46:37.091 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:47:07.158 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:47:07.162 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:47:37.283 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:47:37.287 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:48:07.512 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:48:07.516 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:48:37.615 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:48:37.618 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:49:07.737 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:49:07.741 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:49:37.884 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:49:37.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:08.014 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:08.018 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:38.084 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:50:38.088 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:51:08.225 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:51:08.229 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:51:38.347 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:51:38.350 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:52:08.473 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:52:08.477 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:52:38.536 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:52:38.539 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:53:08.640 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:53:08.644 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:53:38.748 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:53:38.752 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:54:08.879 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:54:08.883 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:54:39.221 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:54:39.225 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:55:09.392 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:55:09.395 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:55:39.501 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:55:39.505 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:56:09.601 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:56:09.604 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:56:39.739 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:56:39.743 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:57:09.810 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:57:09.814 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:57:39.974 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:57:39.991 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:58:10.069 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:58:10.072 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:58:40.198 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:58:40.202 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:59:10.270 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:59:10.273 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:59:40.500 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T10:59:40.504 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:10.563 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:10.566 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:40.700 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:00:40.704 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:01:10.905 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:01:10.909 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:01:41.043 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:01:41.047 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:11.164 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:11.169 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:41.422 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:02:41.426 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:03:11.485 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:03:11.489 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:03:41.627 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:03:41.631 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:11.791 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:11.795 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:41.853 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:04:41.857 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:05:11.957 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:05:11.961 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:05:42.244 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:05:42.247 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:06:12.390 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:06:12.393 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:06:42.454 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:06:42.458 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:07:12.641 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:07:12.644 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:07:42.780 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:07:42.783 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:08:12.859 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:08:12.862 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:08:43.040 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:08:43.044 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:09:13.178 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:09:13.182 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:09:43.315 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:09:43.318 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:10:13.396 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:10:13.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:10:43.557 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:10:43.561 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:11:13.650 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:11:13.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:11:43.870 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:11:43.873 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:12:13.930 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:12:13.933 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:12:44.071 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:12:44.074 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:13:14.323 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:13:14.326 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:13:44.410 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:13:44.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:14:14.548 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:14:14.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:14:44.692 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:14:44.696 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:15:14.824 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:15:14.827 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:15:44.911 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:15:44.914 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:16:15.007 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:16:15.011 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:16:45.244 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:16:45.248 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:17:15.391 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:17:15.395 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:17:45.538 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:17:45.541 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:18:15.666 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:18:15.669 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:18:45.730 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:18:45.734 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:19:15.885 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:19:15.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:19:46.022 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:19:46.025 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:20:16.134 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:20:16.138 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:20:46.266 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:20:46.270 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:21:16.396 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:21:16.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:21:46.532 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:21:46.536 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:22:16.619 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:22:16.623 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:22:46.749 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:22:46.752 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:23:16.811 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:23:16.815 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:23:46.941 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:23:46.944 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:24:17.044 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:24:17.047 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:24:47.108 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:24:47.112 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:25:17.246 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:25:17.250 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:25:47.368 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:25:47.372 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:26:17.706 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:26:17.710 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:26:47.852 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:26:47.855 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:27:17.915 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:27:17.919 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:27:48.045 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:27:48.048 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:28:18.124 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:28:18.127 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:28:48.262 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:28:48.265 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:29:18.349 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:29:18.353 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:29:48.465 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:29:48.469 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:30:18.596 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:30:18.599 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:30:48.673 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:30:48.677 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:31:18.822 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:31:18.826 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:31:48.952 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:31:48.956 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:32:19.077 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:32:19.081 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:32:49.382 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:32:49.385 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:33:19.503 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:33:19.508 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:33:49.855 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:33:49.858 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:34:20.029 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:34:20.032 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:34:50.116 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:34:50.119 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:35:20.428 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:35:20.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:35:50.492 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:35:50.495 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:36:20.563 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:36:20.566 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:36:50.902 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:36:50.905 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:37:20.965 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:37:20.968 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:37:51.138 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:37:51.141 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:38:21.199 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:38:21.212 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:38:51.351 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:38:51.354 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:39:21.449 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:39:21.453 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:39:51.730 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:39:51.734 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:40:21.801 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:40:21.805 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:40:51.904 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:40:51.907 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:41:15.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:15.591 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:41:15.620 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_47_00_UTC 2023-10-18T11:41:15.620 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_48_00_UTC 2023-10-18T11:41:15.620 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_49_00_UTC 2023-10-18T11:41:15.621 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_50_00_UTC 2023-10-18T11:41:15.621 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_51_00_UTC 2023-10-18T11:41:15.621 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_52_00_UTC 2023-10-18T11:41:15.621 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_53_00_UTC 2023-10-18T11:41:15.621 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_54_00_UTC 2023-10-18T11:41:15.622 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_55_00_UTC 2023-10-18T11:41:15.622 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_56_00_UTC 2023-10-18T11:41:15.622 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_57_00_UTC 2023-10-18T11:41:15.622 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_58_00_UTC 2023-10-18T11:41:15.622 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_59_00_UTC 2023-10-18T11:41:15.623 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_00_00_UTC 2023-10-18T11:41:15.623 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_01_00_UTC 2023-10-18T11:41:15.623 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_02_00_UTC 2023-10-18T11:41:15.623 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_03_00_UTC 2023-10-18T11:41:15.623 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_04_00_UTC 2023-10-18T11:41:15.624 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_05_00_UTC 2023-10-18T11:41:15.624 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_06_00_UTC 2023-10-18T11:41:15.624 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_07_00_UTC 2023-10-18T11:41:15.624 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_08_00_UTC 2023-10-18T11:41:15.624 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_09_00_UTC 2023-10-18T11:41:15.625 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_10_00_UTC 2023-10-18T11:41:15.625 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_11_00_UTC 2023-10-18T11:41:15.625 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_12_00_UTC 2023-10-18T11:41:15.625 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_13_00_UTC 2023-10-18T11:41:15.625 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_14_00_UTC 2023-10-18T11:41:15.626 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_15_00_UTC 2023-10-18T11:41:15.626 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_16_00_UTC 2023-10-18T11:41:15.626 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_17_00_UTC 2023-10-18T11:41:15.626 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_18_00_UTC 2023-10-18T11:41:15.626 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_19_00_UTC 2023-10-18T11:41:15.627 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_20_00_UTC 2023-10-18T11:41:15.627 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_21_00_UTC 2023-10-18T11:41:15.627 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_22_00_UTC 2023-10-18T11:41:15.627 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_23_00_UTC 2023-10-18T11:41:15.627 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_24_00_UTC 2023-10-18T11:41:15.627 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_25_00_UTC 2023-10-18T11:41:15.628 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_26_00_UTC 2023-10-18T11:41:15.628 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_27_00_UTC 2023-10-18T11:41:15.628 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_28_00_UTC 2023-10-18T11:41:15.628 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_29_00_UTC 2023-10-18T11:41:15.628 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_30_00_UTC 2023-10-18T11:41:15.629 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_31_00_UTC 2023-10-18T11:41:15.629 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_32_00_UTC 2023-10-18T11:41:15.629 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_33_00_UTC 2023-10-18T11:41:15.629 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_34_00_UTC 2023-10-18T11:41:15.629 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_35_00_UTC 2023-10-18T11:41:15.630 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_36_00_UTC 2023-10-18T11:41:15.630 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_37_00_UTC 2023-10-18T11:41:15.630 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_38_00_UTC 2023-10-18T11:41:15.630 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_39_00_UTC 2023-10-18T11:41:15.630 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_40_00_UTC 2023-10-18T11:41:15.631 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_41_00_UTC 2023-10-18T11:41:15.631 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-10_47_00_UTC', 'scheduled-2023-10-18-10_48_00_UTC', 'scheduled-2023-10-18-10_49_00_UTC', 'scheduled-2023-10-18-10_50_00_UTC', 'scheduled-2023-10-18-10_51_00_UTC', 'scheduled-2023-10-18-10_52_00_UTC', 'scheduled-2023-10-18-10_53_00_UTC', 'scheduled-2023-10-18-10_54_00_UTC', 'scheduled-2023-10-18-10_55_00_UTC', 'scheduled-2023-10-18-10_56_00_UTC', 'scheduled-2023-10-18-10_57_00_UTC', 'scheduled-2023-10-18-10_58_00_UTC', 'scheduled-2023-10-18-10_59_00_UTC', 'scheduled-2023-10-18-11_00_00_UTC', 'scheduled-2023-10-18-11_01_00_UTC', 'scheduled-2023-10-18-11_02_00_UTC', 'scheduled-2023-10-18-11_03_00_UTC', 'scheduled-2023-10-18-11_04_00_UTC', 'scheduled-2023-10-18-11_05_00_UTC', 'scheduled-2023-10-18-11_06_00_UTC', 'scheduled-2023-10-18-11_07_00_UTC', 'scheduled-2023-10-18-11_08_00_UTC', 'scheduled-2023-10-18-11_09_00_UTC', 'scheduled-2023-10-18-11_10_00_UTC', 'scheduled-2023-10-18-11_11_00_UTC', 'scheduled-2023-10-18-11_12_00_UTC', 'scheduled-2023-10-18-11_13_00_UTC', 'scheduled-2023-10-18-11_14_00_UTC', 'scheduled-2023-10-18-11_15_00_UTC', 'scheduled-2023-10-18-11_16_00_UTC', 'scheduled-2023-10-18-11_17_00_UTC', 'scheduled-2023-10-18-11_18_00_UTC', 'scheduled-2023-10-18-11_19_00_UTC', 'scheduled-2023-10-18-11_20_00_UTC', 'scheduled-2023-10-18-11_21_00_UTC', 'scheduled-2023-10-18-11_22_00_UTC', 'scheduled-2023-10-18-11_23_00_UTC', 'scheduled-2023-10-18-11_24_00_UTC', 'scheduled-2023-10-18-11_25_00_UTC', 'scheduled-2023-10-18-11_26_00_UTC', 'scheduled-2023-10-18-11_27_00_UTC', 'scheduled-2023-10-18-11_28_00_UTC', 'scheduled-2023-10-18-11_29_00_UTC', 'scheduled-2023-10-18-11_30_00_UTC', 'scheduled-2023-10-18-11_31_00_UTC', 'scheduled-2023-10-18-11_32_00_UTC', 'scheduled-2023-10-18-11_33_00_UTC', 'scheduled-2023-10-18-11_34_00_UTC', 'scheduled-2023-10-18-11_35_00_UTC', 'scheduled-2023-10-18-11_36_00_UTC', 'scheduled-2023-10-18-11_37_00_UTC', 'scheduled-2023-10-18-11_38_00_UTC', 'scheduled-2023-10-18-11_39_00_UTC', 'scheduled-2023-10-18-11_40_00_UTC', 'scheduled-2023-10-18-11_41_00_UTC'] 2023-10-18T11:41:15.631 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2023-10-18T11:41:15.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.779+0000 7f53b798a640 1 -- 172.21.15.125:0/1357429906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 msgr2=0x7f53b006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:15.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.779+0000 7f53b798a640 1 --2- 172.21.15.125:0/1357429906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 0x7f53b006b2e0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f53b013c240 tx=0x7f53ac02bd90 comp rx=0 tx=0).stop 2023-10-18T11:41:15.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.780+0000 7f53b798a640 1 -- 172.21.15.125:0/1357429906 shutdown_connections 2023-10-18T11:41:15.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.780+0000 7f53b798a640 1 --2- 172.21.15.125:0/1357429906 >> v2:172.21.15.139:3300/0 conn(0x7f53b00abc10 0x7f53b00c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.780+0000 7f53b798a640 1 --2- 172.21.15.125:0/1357429906 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 0x7f53b006b2e0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.780+0000 7f53b798a640 1 -- 172.21.15.125:0/1357429906 >> 172.21.15.125:0/1357429906 conn(0x7f53b0069930 msgr2=0x7f53b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:15.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.780+0000 7f53b798a640 1 -- 172.21.15.125:0/1357429906 shutdown_connections 2023-10-18T11:41:15.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.780+0000 7f53b798a640 1 -- 172.21.15.125:0/1357429906 wait complete. 2023-10-18T11:41:15.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.781+0000 7f53b798a640 1 Processor -- start 2023-10-18T11:41:15.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.781+0000 7f53b798a640 1 -- start start 2023-10-18T11:41:15.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b798a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 0x7f53b014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:15.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b798a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53b00abc10 0x7f53b00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:15.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b798a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53b00c96b0 0x7f53b015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:15.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b798a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0129190 con 0x7f53b006af00 2023-10-18T11:41:15.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b798a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00b1240 con 0x7f53b00c96b0 2023-10-18T11:41:15.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b798a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00b1520 con 0x7f53b00abc10 2023-10-18T11:41:15.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 0x7f53b014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:15.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 0x7f53b014cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43714/0 (socket says 172.21.15.125:43714) 2023-10-18T11:41:15.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 -- 172.21.15.125:0/4132296372 learned_addr learned my addr 172.21.15.125:0/4132296372 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:15.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5f27640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53b00c96b0 0x7f53b015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:15.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 -- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53b00abc10 msgr2=0x7f53b00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:15.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53b00abc10 0x7f53b00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 -- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53b00c96b0 msgr2=0x7f53b015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:15.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53b00c96b0 0x7f53b015ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 -- 172.21.15.125:0/4132296372 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac009560 con 0x7f53b006af00 2023-10-18T11:41:15.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b5726640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 0x7f53b014cda0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f53ac036040 tx=0x7f53ac02dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:15.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53a5ffb640 1 -- 172.21.15.125:0/4132296372 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac005650 con 0x7f53b006af00 2023-10-18T11:41:15.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53a5ffb640 1 -- 172.21.15.125:0/4132296372 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f53ac0038f0 con 0x7f53b006af00 2023-10-18T11:41:15.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b0159cc0 con 0x7f53b006af00 2023-10-18T11:41:15.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.782+0000 7f53a5ffb640 1 -- 172.21.15.125:0/4132296372 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac003ca0 con 0x7f53b006af00 2023-10-18T11:41:15.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.783+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b0128db0 con 0x7f53b006af00 2023-10-18T11:41:15.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.784+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b014dee0 con 0x7f53b006af00 2023-10-18T11:41:15.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.784+0000 7f53a5ffb640 1 -- 172.21.15.125:0/4132296372 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 88) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f53ac03e450 con 0x7f53b006af00 2023-10-18T11:41:15.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.785+0000 7f53a5ffb640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f539406afb0 0x7f539406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:15.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.785+0000 7f53a5ffb640 1 -- 172.21.15.125:0/4132296372 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f53ac0ac830 con 0x7f53b006af00 2023-10-18T11:41:15.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.785+0000 7f53b4f25640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f539406afb0 0x7f539406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:15.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.785+0000 7f53b4f25640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f539406afb0 0x7f539406d470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53b00abbe0 tx=0x7f53a0006ba0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:15.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.789+0000 7f53a5ffb640 1 -- 172.21.15.125:0/4132296372 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53ac0775a0 con 0x7f53b006af00 2023-10-18T11:41:15.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.976+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 --> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53b00b1660 con 0x7f539406afb0 2023-10-18T11:41:15.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.985+0000 7f53a5ffb640 1 -- 172.21.15.125:0/4132296372 <== mgr.8557 v2:172.21.15.139:6832/1032201494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f53b00b1660 con 0x7f539406afb0 2023-10-18T11:41:15.988 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:15.988 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:45:16", "first": "2023-10-18T10:46:00", "last": "2023-10-18T11:41:00", "last_pruned": "2023-10-18T11:41:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-18T11:41:15.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f539406afb0 msgr2=0x7f539406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:15.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f539406afb0 0x7f539406d470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53b00abbe0 tx=0x7f53a0006ba0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 msgr2=0x7f53b014cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 0x7f53b014cda0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f53ac036040 tx=0x7f53ac02dd60 comp rx=0 tx=0).stop 2023-10-18T11:41:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 shutdown_connections 2023-10-18T11:41:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f539406afb0 0x7f539406d470 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53b00c96b0 0x7f53b015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53b00abc10 0x7f53b00b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 --2- 172.21.15.125:0/4132296372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53b006af00 0x7f53b014cda0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 >> 172.21.15.125:0/4132296372 conn(0x7f53b0069930 msgr2=0x7f53b013b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 shutdown_connections 2023-10-18T11:41:15.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:15.987+0000 7f53b798a640 1 -- 172.21.15.125:0/4132296372 wait complete. 2023-10-18T11:41:16.000 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T10:45:16", "first": "2023-10-18T10:46:00", "last": "2023-10-18T11:41:00", "last_pruned": "2023-10-18T11:41:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-18T11:41:16.000 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1m', 'retention': {'n': 55}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T10:45:16', 'first': '2023-10-18T10:46:00', 'last': '2023-10-18T11:41:00', 'last_pruned': '2023-10-18T11:41:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-10-18T11:41:16.001 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-10-18T11:41:16.001 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-10-18T11:41:16.001 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2023-10-18T11:41:16.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.103+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/3959244056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 msgr2=0x7f8a700051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.103+0000 7f8a8c3a5640 1 --2- 172.21.15.125:0/3959244056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 0x7f8a700051f0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00c6c0 tx=0x7f8a700057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.103+0000 7f8a8a942640 1 -- 172.21.15.125:0/3959244056 reap_dead start 2023-10-18T11:41:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.103+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/3959244056 shutdown_connections 2023-10-18T11:41:16.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.103+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/3959244056 >> 172.21.15.125:0/3959244056 conn(0x7f8a84069930 msgr2=0x7f8a84069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:16.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.103+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/3959244056 shutdown_connections 2023-10-18T11:41:16.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.103+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/3959244056 wait complete. 2023-10-18T11:41:16.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.104+0000 7f8a8c3a5640 1 Processor -- start 2023-10-18T11:41:16.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.104+0000 7f8a8c3a5640 1 -- start start 2023-10-18T11:41:16.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8c3a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 0x7f8a840ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:16.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8c3a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a840ad1f0 0x7f8a84152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:16.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8c3a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a84134d40 0x7f8a841351c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:16.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8c3a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84135700 con 0x7f8a70004d70 2023-10-18T11:41:16.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8c3a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a840aec20 con 0x7f8a840ad1f0 2023-10-18T11:41:16.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8c3a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a841340f0 con 0x7f8a84134d40 2023-10-18T11:41:16.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 0x7f8a840ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:16.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 0x7f8a840ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43728/0 (socket says 172.21.15.125:43728) 2023-10-18T11:41:16.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 -- 172.21.15.125:0/8967186 learned_addr learned my addr 172.21.15.125:0/8967186 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:16.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a89940640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a840ad1f0 0x7f8a84152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:16.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a942640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a84134d40 0x7f8a841351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:16.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 -- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a84134d40 msgr2=0x7f8a841351c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:16.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a84134d40 0x7f8a841351c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:16.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 -- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a840ad1f0 msgr2=0x7f8a84152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:16.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a840ad1f0 0x7f8a84152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:16.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 -- 172.21.15.125:0/8967186 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840ae940 con 0x7f8a70004d70 2023-10-18T11:41:16.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a942640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a84134d40 0x7f8a841351c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:16.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a8a141640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 0x7f8a840ae0e0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000fa60 tx=0x7f8a8000ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:16.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.105+0000 7f8a7affd640 1 -- 172.21.15.125:0/8967186 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000de50 con 0x7f8a70004d70 2023-10-18T11:41:16.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.106+0000 7f8a7affd640 1 -- 172.21.15.125:0/8967186 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a80018440 con 0x7f8a70004d70 2023-10-18T11:41:16.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.106+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8413c8d0 con 0x7f8a70004d70 2023-10-18T11:41:16.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.106+0000 7f8a7affd640 1 -- 172.21.15.125:0/8967186 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80018850 con 0x7f8a70004d70 2023-10-18T11:41:16.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.106+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8413ce90 con 0x7f8a70004d70 2023-10-18T11:41:16.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.106+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a840690e0 con 0x7f8a70004d70 2023-10-18T11:41:16.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.107+0000 7f8a7affd640 1 -- 172.21.15.125:0/8967186 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 88) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f8a800189b0 con 0x7f8a70004d70 2023-10-18T11:41:16.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.107+0000 7f8a7affd640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f8a6806afe0 0x7f8a6806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:16.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.107+0000 7f8a7affd640 1 -- 172.21.15.125:0/8967186 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f8a8008a1e0 con 0x7f8a70004d70 2023-10-18T11:41:16.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.107+0000 7f8a89940640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f8a6806afe0 0x7f8a6806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:16.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.108+0000 7f8a89940640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f8a6806afe0 0x7f8a6806d4a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a840ac400 tx=0x7f8a74006bc0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:16.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.111+0000 7f8a7affd640 1 -- 172.21.15.125:0/8967186 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a80054f30 con 0x7f8a70004d70 2023-10-18T11:41:16.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.299+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 --> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a8415a030 con 0x7f8a6806afe0 2023-10-18T11:41:16.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.364+0000 7f8a7affd640 1 -- 172.21.15.125:0/8967186 <== mgr.8557 v2:172.21.15.139:6832/1032201494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f8a74003d80 con 0x7f8a6806afe0 2023-10-18T11:41:16.367 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1 2023-10-18T11:41:16.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f8a6806afe0 msgr2=0x7f8a6806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:16.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f8a6806afe0 0x7f8a6806d4a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a840ac400 tx=0x7f8a74006bc0 comp rx=0 tx=0).stop 2023-10-18T11:41:16.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 msgr2=0x7f8a840ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:16.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 0x7f8a840ae0e0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000fa60 tx=0x7f8a8000ff20 comp rx=0 tx=0).stop 2023-10-18T11:41:16.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 shutdown_connections 2023-10-18T11:41:16.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f8a6806afe0 0x7f8a6806d4a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:16.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a840ad1f0 0x7f8a84152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:16.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a84134d40 0x7f8a841351c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:16.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 --2- 172.21.15.125:0/8967186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a70004d70 0x7f8a840ae0e0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:16.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 >> 172.21.15.125:0/8967186 conn(0x7f8a84069930 msgr2=0x7f8a8413aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:16.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 shutdown_connections 2023-10-18T11:41:16.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:16.366+0000 7f8a8c3a5640 1 -- 172.21.15.125:0/8967186 wait complete. 2023-10-18T11:41:16.379 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-10-18T11:41:16.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.380 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:41:16.400 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_47_00_UTC 2023-10-18T11:41:16.400 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_48_00_UTC 2023-10-18T11:41:16.401 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_49_00_UTC 2023-10-18T11:41:16.401 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_50_00_UTC 2023-10-18T11:41:16.401 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_51_00_UTC 2023-10-18T11:41:16.401 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_52_00_UTC 2023-10-18T11:41:16.402 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_53_00_UTC 2023-10-18T11:41:16.402 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_54_00_UTC 2023-10-18T11:41:16.402 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_55_00_UTC 2023-10-18T11:41:16.402 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_56_00_UTC 2023-10-18T11:41:16.403 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_57_00_UTC 2023-10-18T11:41:16.403 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_58_00_UTC 2023-10-18T11:41:16.403 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-10_59_00_UTC 2023-10-18T11:41:16.403 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_00_00_UTC 2023-10-18T11:41:16.403 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_01_00_UTC 2023-10-18T11:41:16.403 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_02_00_UTC 2023-10-18T11:41:16.404 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_03_00_UTC 2023-10-18T11:41:16.404 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_04_00_UTC 2023-10-18T11:41:16.404 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_05_00_UTC 2023-10-18T11:41:16.404 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_06_00_UTC 2023-10-18T11:41:16.404 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_07_00_UTC 2023-10-18T11:41:16.405 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_08_00_UTC 2023-10-18T11:41:16.405 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_09_00_UTC 2023-10-18T11:41:16.405 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_10_00_UTC 2023-10-18T11:41:16.405 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_11_00_UTC 2023-10-18T11:41:16.405 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_12_00_UTC 2023-10-18T11:41:16.406 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_13_00_UTC 2023-10-18T11:41:16.406 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_14_00_UTC 2023-10-18T11:41:16.406 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_15_00_UTC 2023-10-18T11:41:16.406 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_16_00_UTC 2023-10-18T11:41:16.406 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_17_00_UTC 2023-10-18T11:41:16.406 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_18_00_UTC 2023-10-18T11:41:16.407 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_19_00_UTC 2023-10-18T11:41:16.407 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_20_00_UTC 2023-10-18T11:41:16.407 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_21_00_UTC 2023-10-18T11:41:16.407 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_22_00_UTC 2023-10-18T11:41:16.407 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_23_00_UTC 2023-10-18T11:41:16.408 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_24_00_UTC 2023-10-18T11:41:16.408 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_25_00_UTC 2023-10-18T11:41:16.408 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_26_00_UTC 2023-10-18T11:41:16.408 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_27_00_UTC 2023-10-18T11:41:16.408 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_28_00_UTC 2023-10-18T11:41:16.409 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_29_00_UTC 2023-10-18T11:41:16.409 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_30_00_UTC 2023-10-18T11:41:16.409 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_31_00_UTC 2023-10-18T11:41:16.409 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_32_00_UTC 2023-10-18T11:41:16.409 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_33_00_UTC 2023-10-18T11:41:16.410 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_34_00_UTC 2023-10-18T11:41:16.410 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_35_00_UTC 2023-10-18T11:41:16.410 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_36_00_UTC 2023-10-18T11:41:16.410 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_37_00_UTC 2023-10-18T11:41:16.411 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_38_00_UTC 2023-10-18T11:41:16.411 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_39_00_UTC 2023-10-18T11:41:16.411 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_40_00_UTC 2023-10-18T11:41:16.411 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_41_00_UTC 2023-10-18T11:41:16.411 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_47_00_UTC 2023-10-18T11:41:16.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.412 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_47_00_UTC) 2023-10-18T11:41:16.462 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_48_00_UTC 2023-10-18T11:41:16.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.463 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_48_00_UTC) 2023-10-18T11:41:16.524 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_49_00_UTC 2023-10-18T11:41:16.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.525 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_49_00_UTC) 2023-10-18T11:41:16.586 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_50_00_UTC 2023-10-18T11:41:16.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.587 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_50_00_UTC) 2023-10-18T11:41:16.649 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_51_00_UTC 2023-10-18T11:41:16.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.650 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_51_00_UTC) 2023-10-18T11:41:16.711 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_52_00_UTC 2023-10-18T11:41:16.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.712 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_52_00_UTC) 2023-10-18T11:41:16.772 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_53_00_UTC 2023-10-18T11:41:16.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.773 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_53_00_UTC) 2023-10-18T11:41:16.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_54_00_UTC 2023-10-18T11:41:16.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.836 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_54_00_UTC) 2023-10-18T11:41:16.896 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_55_00_UTC 2023-10-18T11:41:16.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.897 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_55_00_UTC) 2023-10-18T11:41:16.958 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_56_00_UTC 2023-10-18T11:41:16.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:16.959 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_56_00_UTC) 2023-10-18T11:41:17.021 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_57_00_UTC 2023-10-18T11:41:17.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.022 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_57_00_UTC) 2023-10-18T11:41:17.083 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_58_00_UTC 2023-10-18T11:41:17.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.085 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_58_00_UTC) 2023-10-18T11:41:17.145 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-10_59_00_UTC 2023-10-18T11:41:17.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.146 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-10_59_00_UTC) 2023-10-18T11:41:17.208 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_00_00_UTC 2023-10-18T11:41:17.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.209 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_00_00_UTC) 2023-10-18T11:41:17.271 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_01_00_UTC 2023-10-18T11:41:17.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.272 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_01_00_UTC) 2023-10-18T11:41:17.334 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_02_00_UTC 2023-10-18T11:41:17.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.335 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_02_00_UTC) 2023-10-18T11:41:17.396 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_03_00_UTC 2023-10-18T11:41:17.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.397 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_03_00_UTC) 2023-10-18T11:41:17.458 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_04_00_UTC 2023-10-18T11:41:17.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.459 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_04_00_UTC) 2023-10-18T11:41:17.520 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_05_00_UTC 2023-10-18T11:41:17.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.521 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_05_00_UTC) 2023-10-18T11:41:17.581 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_06_00_UTC 2023-10-18T11:41:17.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.581 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_06_00_UTC) 2023-10-18T11:41:17.642 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_07_00_UTC 2023-10-18T11:41:17.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.643 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_07_00_UTC) 2023-10-18T11:41:17.704 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_08_00_UTC 2023-10-18T11:41:17.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.705 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_08_00_UTC) 2023-10-18T11:41:17.766 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_09_00_UTC 2023-10-18T11:41:17.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.767 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_09_00_UTC) 2023-10-18T11:41:17.827 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_10_00_UTC 2023-10-18T11:41:17.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.828 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_10_00_UTC) 2023-10-18T11:41:17.888 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_11_00_UTC 2023-10-18T11:41:17.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.889 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_11_00_UTC) 2023-10-18T11:41:17.949 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_12_00_UTC 2023-10-18T11:41:17.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:17.950 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_12_00_UTC) 2023-10-18T11:41:18.011 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_13_00_UTC 2023-10-18T11:41:18.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.012 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_13_00_UTC) 2023-10-18T11:41:18.073 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_14_00_UTC 2023-10-18T11:41:18.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.074 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_14_00_UTC) 2023-10-18T11:41:18.135 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_15_00_UTC 2023-10-18T11:41:18.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.136 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_15_00_UTC) 2023-10-18T11:41:18.197 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_16_00_UTC 2023-10-18T11:41:18.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.197 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_16_00_UTC) 2023-10-18T11:41:18.258 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_17_00_UTC 2023-10-18T11:41:18.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.259 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_17_00_UTC) 2023-10-18T11:41:18.320 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_18_00_UTC 2023-10-18T11:41:18.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.321 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_18_00_UTC) 2023-10-18T11:41:18.382 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_19_00_UTC 2023-10-18T11:41:18.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.383 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_19_00_UTC) 2023-10-18T11:41:18.443 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_20_00_UTC 2023-10-18T11:41:18.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.444 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_20_00_UTC) 2023-10-18T11:41:18.504 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_21_00_UTC 2023-10-18T11:41:18.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.505 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_21_00_UTC) 2023-10-18T11:41:18.565 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_22_00_UTC 2023-10-18T11:41:18.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.566 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_22_00_UTC) 2023-10-18T11:41:18.626 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_23_00_UTC 2023-10-18T11:41:18.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.627 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_23_00_UTC) 2023-10-18T11:41:18.687 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_24_00_UTC 2023-10-18T11:41:18.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.688 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_24_00_UTC) 2023-10-18T11:41:18.748 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_25_00_UTC 2023-10-18T11:41:18.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.749 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_25_00_UTC) 2023-10-18T11:41:18.810 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_26_00_UTC 2023-10-18T11:41:18.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.811 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_26_00_UTC) 2023-10-18T11:41:18.872 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_27_00_UTC 2023-10-18T11:41:18.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.872 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_27_00_UTC) 2023-10-18T11:41:18.933 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_28_00_UTC 2023-10-18T11:41:18.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.934 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_28_00_UTC) 2023-10-18T11:41:18.994 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_29_00_UTC 2023-10-18T11:41:18.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:18.995 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_29_00_UTC) 2023-10-18T11:41:19.056 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_30_00_UTC 2023-10-18T11:41:19.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.057 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_30_00_UTC) 2023-10-18T11:41:19.117 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_31_00_UTC 2023-10-18T11:41:19.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.118 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_31_00_UTC) 2023-10-18T11:41:19.178 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_32_00_UTC 2023-10-18T11:41:19.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.180 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_32_00_UTC) 2023-10-18T11:41:19.241 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_33_00_UTC 2023-10-18T11:41:19.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.242 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_33_00_UTC) 2023-10-18T11:41:19.302 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_34_00_UTC 2023-10-18T11:41:19.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.304 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_34_00_UTC) 2023-10-18T11:41:19.363 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_35_00_UTC 2023-10-18T11:41:19.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.364 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_35_00_UTC) 2023-10-18T11:41:19.424 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_36_00_UTC 2023-10-18T11:41:19.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.426 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_36_00_UTC) 2023-10-18T11:41:19.485 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_37_00_UTC 2023-10-18T11:41:19.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.486 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_37_00_UTC) 2023-10-18T11:41:19.546 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_38_00_UTC 2023-10-18T11:41:19.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.547 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_38_00_UTC) 2023-10-18T11:41:19.607 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_39_00_UTC 2023-10-18T11:41:19.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.608 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_39_00_UTC) 2023-10-18T11:41:19.669 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_40_00_UTC 2023-10-18T11:41:19.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.670 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_40_00_UTC) 2023-10-18T11:41:19.730 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-11_41_00_UTC 2023-10-18T11:41:19.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.731 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-11_41_00_UTC) 2023-10-18T11:41:19.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:19.792 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-18T11:41:19.849 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T11:41:19.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.991+0000 7f8084675640 1 -- 172.21.15.125:0/4166435882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 msgr2=0x7f8068003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:19.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.991+0000 7f8084675640 1 --2- 172.21.15.125:0/4166435882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 0x7f8068003dc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8074001d80 tx=0x7f80680057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:19.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.991+0000 7f8082c12640 1 -- 172.21.15.125:0/4166435882 reap_dead start 2023-10-18T11:41:19.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.991+0000 7f8084675640 1 -- 172.21.15.125:0/4166435882 shutdown_connections 2023-10-18T11:41:19.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.991+0000 7f8084675640 1 -- 172.21.15.125:0/4166435882 >> 172.21.15.125:0/4166435882 conn(0x7f807c069930 msgr2=0x7f807c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:19.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.991+0000 7f8084675640 1 -- 172.21.15.125:0/4166435882 shutdown_connections 2023-10-18T11:41:19.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.991+0000 7f8084675640 1 -- 172.21.15.125:0/4166435882 wait complete. 2023-10-18T11:41:19.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.992+0000 7f8084675640 1 Processor -- start 2023-10-18T11:41:19.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.992+0000 7f8084675640 1 -- start start 2023-10-18T11:41:19.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8084675640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 0x7f807c1d9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:19.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8084675640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f807c1d7070 0x7f807c0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:19.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8084675640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c1d9a80 0x7f807c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:19.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8084675640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c1d8180 con 0x7f80680039a0 2023-10-18T11:41:19.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8084675640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c1d7500 con 0x7f807c1d9a80 2023-10-18T11:41:19.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8084675640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c1d77e0 con 0x7f807c1d7070 2023-10-18T11:41:19.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8082411640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 0x7f807c1d9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:20.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8082411640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 0x7f807c1d9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39756/0 (socket says 172.21.15.125:39756) 2023-10-18T11:41:20.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8082411640 1 -- 172.21.15.125:0/4249257825 learned_addr learned my addr 172.21.15.125:0/4249257825 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:20.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8082411640 1 -- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f807c1d7070 msgr2=0x7f807c0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:20.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.993+0000 7f8082411640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f807c1d7070 0x7f807c0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:20.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8082411640 1 -- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c1d9a80 msgr2=0x7f807c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:20.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8082411640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c1d9a80 0x7f807c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:20.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8082411640 1 -- 172.21.15.125:0/4249257825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c1d7ac0 con 0x7f80680039a0 2023-10-18T11:41:20.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8082411640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 0x7f807c1d9540 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f8074001ea0 tx=0x7f807402c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:20.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8072ffd640 1 -- 172.21.15.125:0/4249257825 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807402ce40 con 0x7f80680039a0 2023-10-18T11:41:20.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8072ffd640 1 -- 172.21.15.125:0/4249257825 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8074006050 con 0x7f80680039a0 2023-10-18T11:41:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c1d7da0 con 0x7f80680039a0 2023-10-18T11:41:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8072ffd640 1 -- 172.21.15.125:0/4249257825 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074006440 con 0x7f80680039a0 2023-10-18T11:41:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.994+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c13b5b0 con 0x7f80680039a0 2023-10-18T11:41:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.995+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c0690e0 con 0x7f80680039a0 2023-10-18T11:41:20.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.995+0000 7f8072ffd640 1 -- 172.21.15.125:0/4249257825 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 88) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f80740065a0 con 0x7f80680039a0 2023-10-18T11:41:20.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.996+0000 7f8072ffd640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f806006afb0 0x7f806006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:20.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.996+0000 7f8072ffd640 1 -- 172.21.15.125:0/4249257825 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f80740ab480 con 0x7f80680039a0 2023-10-18T11:41:20.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.996+0000 7f8081c10640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f806006afb0 0x7f806006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:20.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:19.996+0000 7f8081c10640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f806006afb0 0x7f806006d470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f807c0cb910 tx=0x7f806c005ab0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:20.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.000+0000 7f8072ffd640 1 -- 172.21.15.125:0/4249257825 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80740761d0 con 0x7f80680039a0 2023-10-18T11:41:20.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.211+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f807c1e4410 con 0x7f80680039a0 2023-10-18T11:41:20.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.749+0000 7f8072ffd640 1 -- 172.21.15.125:0/4249257825 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v89) v1 ==== 93+0+0 (secure 0 0 0) 0x7f80740797f0 con 0x7f80680039a0 2023-10-18T11:41:20.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f806006afb0 msgr2=0x7f806006d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:20.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f806006afb0 0x7f806006d470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f807c0cb910 tx=0x7f806c005ab0 comp rx=0 tx=0).stop 2023-10-18T11:41:20.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 msgr2=0x7f807c1d9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:20.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 0x7f807c1d9540 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f8074001ea0 tx=0x7f807402c8c0 comp rx=0 tx=0).stop 2023-10-18T11:41:20.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 shutdown_connections 2023-10-18T11:41:20.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f806006afb0 0x7f806006d470 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:20.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c1d9a80 0x7f807c0cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:20.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f807c1d7070 0x7f807c0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:20.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 --2- 172.21.15.125:0/4249257825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80680039a0 0x7f807c1d9540 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:20.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.751+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 >> 172.21.15.125:0/4249257825 conn(0x7f807c069930 msgr2=0x7f807c1e3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:20.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.752+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 shutdown_connections 2023-10-18T11:41:20.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:20.752+0000 7f8084675640 1 -- 172.21.15.125:0/4249257825 wait complete. 2023-10-18T11:41:20.766 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T11:41:20.767 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:41:20.808 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:41:20.849 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T11:41:20.849 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:41:20.850 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:20.866 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:21.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.010+0000 7fd3d35c0640 1 -- 172.21.15.125:0/2478846755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b8003b30 msgr2=0x7fd3b8003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.010+0000 7fd3d35c0640 1 --2- 172.21.15.125:0/2478846755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b8003b30 0x7fd3b8003f30 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc006f80 tx=0x7fd3b80057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.010+0000 7fd3d1b5d640 1 -- 172.21.15.125:0/2478846755 reap_dead start 2023-10-18T11:41:21.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.010+0000 7fd3d35c0640 1 -- 172.21.15.125:0/2478846755 shutdown_connections 2023-10-18T11:41:21.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.010+0000 7fd3d35c0640 1 -- 172.21.15.125:0/2478846755 >> 172.21.15.125:0/2478846755 conn(0x7fd3cc0d0d60 msgr2=0x7fd3cc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:21.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.010+0000 7fd3d35c0640 1 -- 172.21.15.125:0/2478846755 shutdown_connections 2023-10-18T11:41:21.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.010+0000 7fd3d35c0640 1 -- 172.21.15.125:0/2478846755 wait complete. 2023-10-18T11:41:21.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.011+0000 7fd3d35c0640 1 Processor -- start 2023-10-18T11:41:21.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.011+0000 7fd3d35c0640 1 -- start start 2023-10-18T11:41:21.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d35c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3b8003b30 0x7fd3cc0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d35c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3cc1f16f0 0x7fd3cc14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d35c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc0ae260 0x7fd3cc0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d35c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1f2730 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d35c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0aec20 con 0x7fd3cc0ae260 2023-10-18T11:41:21.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d35c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0c9740 con 0x7fd3b8003b30 2023-10-18T11:41:21.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3cc1f16f0 0x7fd3cc14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3cc1f16f0 0x7fd3cc14f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39794/0 (socket says 172.21.15.125:39794) 2023-10-18T11:41:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 -- 172.21.15.125:0/870617978 learned_addr learned my addr 172.21.15.125:0/870617978 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d135c640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3b8003b30 0x7fd3cc0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d1b5d640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc0ae260 0x7fd3cc0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 -- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3b8003b30 msgr2=0x7fd3cc0c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3b8003b30 0x7fd3cc0c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 -- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc0ae260 msgr2=0x7fd3cc0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc0ae260 0x7fd3cc0ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 -- 172.21.15.125:0/870617978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc14f700 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d0b5b640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3cc1f16f0 0x7fd3cc14f1c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8011a30 tx=0x7fd3c8011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3c1ffb640 1 -- 172.21.15.125:0/870617978 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800fe70 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3c1ffb640 1 -- 172.21.15.125:0/870617978 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd3c8012450 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc1f1e90 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3c1ffb640 1 -- 172.21.15.125:0/870617978 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8012800 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.012+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc1f2450 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.013+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc0690e0 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.014+0000 7fd3c1ffb640 1 -- 172.21.15.125:0/870617978 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd3c8012960 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.014+0000 7fd3c1ffb640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fd3b006af90 0x7fd3b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.015+0000 7fd3d135c640 1 -- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fd3b006af90 msgr2=0x7fd3b006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.015+0000 7fd3c1ffb640 1 -- 172.21.15.125:0/870617978 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fd3c808a9f0 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.015+0000 7fd3d135c640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fd3b006af90 0x7fd3b006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:21.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.019+0000 7fd3c1ffb640 1 -- 172.21.15.125:0/870617978 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3c8055760 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.144 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:21.142+0000 7f7890c45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:41:21.149 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.147+0000 7fbaf344a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:41:21.189 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:21.187+0000 7f7890c45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:41:21.195 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.193+0000 7fbaf344a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:41:21.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.194+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3cc0d0b40 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.195+0000 7fd3c1ffb640 1 -- 172.21.15.125:0/870617978 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+17424 (secure 0 0 0) 0x7fd3c801f020 con 0x7fd3cc1f16f0 2023-10-18T11:41:21.197 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:21.198 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":159,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:09.132915+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:58.087653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:58.847874+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":58,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/555070972":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1072662486":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6833/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1079295697":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/2400687467":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6832/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1927085609":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1424238360":"2023-10-19T10:45:15.816055+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":20,"snaps":[{"begin":2,"length":1},{"begin":58,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:21.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fd3b006af90 msgr2=0x7fd3b006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:21.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fd3b006af90 0x7fd3b006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3cc1f16f0 msgr2=0x7fd3cc14f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3cc1f16f0 0x7fd3cc14f1c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8011a30 tx=0x7fd3c8011ef0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 shutdown_connections 2023-10-18T11:41:21.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fd3b006af90 0x7fd3b006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3cc0ae260 0x7fd3cc0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3cc1f16f0 0x7fd3cc14f1c0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 --2- 172.21.15.125:0/870617978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3b8003b30 0x7fd3cc0c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 >> 172.21.15.125:0/870617978 conn(0x7fd3cc0d0d60 msgr2=0x7fd3cc06b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:21.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 shutdown_connections 2023-10-18T11:41:21.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.197+0000 7fd3d35c0640 1 -- 172.21.15.125:0/870617978 wait complete. 2023-10-18T11:41:21.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:44:58.087653+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:44:58.847874+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 58, 'snap_epoch': 158, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:41:21.211 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:41:21.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.316+0000 7f1d18e56640 1 -- 172.21.15.125:0/2314402067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d140ceb40 msgr2=0x7f1d140b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.316+0000 7f1d18e56640 1 --2- 172.21.15.125:0/2314402067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d140ceb40 0x7f1d140b0650 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c005fd0 tx=0x7f1d0c02c240 comp rx=0 tx=0).stop 2023-10-18T11:41:21.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.316+0000 7f1d18e56640 1 -- 172.21.15.125:0/2314402067 shutdown_connections 2023-10-18T11:41:21.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.316+0000 7f1d18e56640 1 --2- 172.21.15.125:0/2314402067 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d140ceb40 0x7f1d140b0650 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.316+0000 7f1d18e56640 1 --2- 172.21.15.125:0/2314402067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d1415b2a0 0x7f1d140d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.316+0000 7f1d18e56640 1 -- 172.21.15.125:0/2314402067 >> 172.21.15.125:0/2314402067 conn(0x7f1d140d0d60 msgr2=0x7f1d140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:21.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.316+0000 7f1d18e56640 1 -- 172.21.15.125:0/2314402067 shutdown_connections 2023-10-18T11:41:21.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.316+0000 7f1d18e56640 1 -- 172.21.15.125:0/2314402067 wait complete. 2023-10-18T11:41:21.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.317+0000 7f1d18e56640 1 Processor -- start 2023-10-18T11:41:21.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.317+0000 7f1d18e56640 1 -- start start 2023-10-18T11:41:21.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d18e56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d140ceb40 0x7f1d14158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d18e56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d140cf340 0x7f1d1415d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d18e56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d1415b2a0 0x7f1d1415dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d18e56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1415b800 con 0x7f1d1415b2a0 2023-10-18T11:41:21.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d18e56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d141f2560 con 0x7f1d140ceb40 2023-10-18T11:41:21.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d18e56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140c9d40 con 0x7f1d140cf340 2023-10-18T11:41:21.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d1415b2a0 0x7f1d1415dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d1415b2a0 0x7f1d1415dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39806/0 (socket says 172.21.15.125:39806) 2023-10-18T11:41:21.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 -- 172.21.15.125:0/408679125 learned_addr learned my addr 172.21.15.125:0/408679125 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:21.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d1259c640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d140ceb40 0x7f1d14158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d0bfff640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d140cf340 0x7f1d1415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 -- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d140cf340 msgr2=0x7f1d1415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d140cf340 0x7f1d1415d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 -- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d140ceb40 msgr2=0x7f1d14158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d140ceb40 0x7f1d14158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 -- 172.21.15.125:0/408679125 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c009900 con 0x7f1d1415b2a0 2023-10-18T11:41:21.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.318+0000 7f1d12d9d640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d1415b2a0 0x7f1d1415dc40 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c037040 tx=0x7f1d0c035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:21.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.319+0000 7f1d0affd640 1 -- 172.21.15.125:0/408679125 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c034440 con 0x7f1d1415b2a0 2023-10-18T11:41:21.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.319+0000 7f1d0affd640 1 -- 172.21.15.125:0/408679125 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d0c004330 con 0x7f1d1415b2a0 2023-10-18T11:41:21.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.319+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d140ca020 con 0x7f1d1415b2a0 2023-10-18T11:41:21.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.319+0000 7f1d0affd640 1 -- 172.21.15.125:0/408679125 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c0046c0 con 0x7f1d1415b2a0 2023-10-18T11:41:21.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.319+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d140ca300 con 0x7f1d1415b2a0 2023-10-18T11:41:21.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.320+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d140690e0 con 0x7f1d1415b2a0 2023-10-18T11:41:21.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.320+0000 7f1d0affd640 1 -- 172.21.15.125:0/408679125 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f1d0c03b040 con 0x7f1d1415b2a0 2023-10-18T11:41:21.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.320+0000 7f1d0affd640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f1cf406af60 0x7f1cf406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.321+0000 7f1d1259c640 1 -- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f1cf406af60 msgr2=0x7f1cf406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:21.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.321+0000 7f1d0affd640 1 -- 172.21.15.125:0/408679125 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f1d0c0ac500 con 0x7f1d1415b2a0 2023-10-18T11:41:21.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.321+0000 7f1d1259c640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f1cf406af60 0x7f1cf406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:21.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.325+0000 7f1d0affd640 1 -- 172.21.15.125:0/408679125 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d0c077260 con 0x7f1d1415b2a0 2023-10-18T11:41:21.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.498+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d140cb980 con 0x7f1d1415b2a0 2023-10-18T11:41:21.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.498+0000 7f1d0affd640 1 -- 172.21.15.125:0/408679125 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d0c07a880 con 0x7f1d1415b2a0 2023-10-18T11:41:21.501 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:41:21.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.500+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f1cf406af60 msgr2=0x7f1cf406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:21.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.500+0000 7f1d18e56640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f1cf406af60 0x7f1cf406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.500+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d1415b2a0 msgr2=0x7f1d1415dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.500+0000 7f1d18e56640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d1415b2a0 0x7f1d1415dc40 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c037040 tx=0x7f1d0c035a40 comp rx=0 tx=0).stop 2023-10-18T11:41:21.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.501+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 shutdown_connections 2023-10-18T11:41:21.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.501+0000 7f1d18e56640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d1415b2a0 0x7f1d1415dc40 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.501+0000 7f1d18e56640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d140cf340 0x7f1d1415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.501+0000 7f1d18e56640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f1cf406af60 0x7f1cf406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.501+0000 7f1d18e56640 1 --2- 172.21.15.125:0/408679125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d140ceb40 0x7f1d14158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.501+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 >> 172.21.15.125:0/408679125 conn(0x7f1d140d0d60 msgr2=0x7f1d1415a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.501+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 shutdown_connections 2023-10-18T11:41:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.501+0000 7f1d18e56640 1 -- 172.21.15.125:0/408679125 wait complete. 2023-10-18T11:41:21.514 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:41:21.578 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:21.576+0000 7f7890c45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:41:21.609 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.607+0000 7fbaf344a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:41:21.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 -- 172.21.15.125:0/910567717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 msgr2=0x7fa028070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 --2- 172.21.15.125:0/910567717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 0x7fa028070920 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fa02806a780 tx=0x7fa02002c140 comp rx=0 tx=0).stop 2023-10-18T11:41:21.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 -- 172.21.15.125:0/910567717 shutdown_connections 2023-10-18T11:41:21.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 --2- 172.21.15.125:0/910567717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 0x7fa028070920 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 --2- 172.21.15.125:0/910567717 >> v2:172.21.15.139:3300/0 conn(0x7fa028141690 0x7fa0280703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 --2- 172.21.15.125:0/910567717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa02813e520 0x7fa02806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 -- 172.21.15.125:0/910567717 >> 172.21.15.125:0/910567717 conn(0x7fa028069930 msgr2=0x7fa028069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:21.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 -- 172.21.15.125:0/910567717 shutdown_connections 2023-10-18T11:41:21.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.619+0000 7fa02d56e640 1 -- 172.21.15.125:0/910567717 wait complete. 2023-10-18T11:41:21.624 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:21.620+0000 7f7890c45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:41:21.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa02d56e640 1 Processor -- start 2023-10-18T11:41:21.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa02d56e640 1 -- start start 2023-10-18T11:41:21.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa02d56e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 0x7fa0281d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa02d56e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02813e520 0x7fa02812c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa02d56e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa028141690 0x7fa028140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa02d56e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0281dead0 con 0x7fa02806c0a0 2023-10-18T11:41:21.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa02d56e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0281ddec0 con 0x7fa02813e520 2023-10-18T11:41:21.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa02d56e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0281de1a0 con 0x7fa028141690 2023-10-18T11:41:21.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.621+0000 7fa026ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 0x7fa0281d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa026ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 0x7fa0281d93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39820/0 (socket says 172.21.15.125:39820) 2023-10-18T11:41:21.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa026ffd640 1 -- 172.21.15.125:0/3592676140 learned_addr learned my addr 172.21.15.125:0/3592676140 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:21.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa0267fc640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02813e520 0x7fa02812c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa0277fe640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa028141690 0x7fa028140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa026ffd640 1 -- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa028141690 msgr2=0x7fa028140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa026ffd640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa028141690 0x7fa028140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa026ffd640 1 -- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02813e520 msgr2=0x7fa02812c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa026ffd640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02813e520 0x7fa02812c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa026ffd640 1 -- 172.21.15.125:0/3592676140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa020009900 con 0x7fa02806c0a0 2023-10-18T11:41:21.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa026ffd640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 0x7fa0281d93a0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fa01800d720 tx=0x7fa01800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:21.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa00f7fe640 1 -- 172.21.15.125:0/3592676140 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018009c30 con 0x7fa02806c0a0 2023-10-18T11:41:21.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa00f7fe640 1 -- 172.21.15.125:0/3592676140 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa018009e80 con 0x7fa02806c0a0 2023-10-18T11:41:21.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0281de480 con 0x7fa02806c0a0 2023-10-18T11:41:21.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa00f7fe640 1 -- 172.21.15.125:0/3592676140 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018004260 con 0x7fa02806c0a0 2023-10-18T11:41:21.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.622+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0281d98e0 con 0x7fa02806c0a0 2023-10-18T11:41:21.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.623+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0280690e0 con 0x7fa02806c0a0 2023-10-18T11:41:21.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.624+0000 7fa00f7fe640 1 -- 172.21.15.125:0/3592676140 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fa018005020 con 0x7fa02806c0a0 2023-10-18T11:41:21.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.625+0000 7fa00f7fe640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fa01406af60 0x7fa01406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.625+0000 7fa0267fc640 1 -- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fa01406af60 msgr2=0x7fa01406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:21.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.625+0000 7fa00f7fe640 1 -- 172.21.15.125:0/3592676140 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fa018089f00 con 0x7fa02806c0a0 2023-10-18T11:41:21.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.625+0000 7fa0267fc640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fa01406af60 0x7fa01406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:21.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.629+0000 7fa00f7fe640 1 -- 172.21.15.125:0/3592676140 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa018054b60 con 0x7fa02806c0a0 2023-10-18T11:41:21.656 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.654+0000 7fbaf344a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:41:21.665 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:21.662+0000 7f7890c45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:41:21.700 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.698+0000 7fbaf344a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:41:21.716 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:21.714+0000 7f7890c45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:41:21.754 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.752+0000 7fbaf344a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:41:21.762 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:21.760+0000 7f7890c45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:41:21.805 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.803+0000 7fbaf344a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:41:21.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.811+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0280cc1b0 con 0x7fa02806c0a0 2023-10-18T11:41:21.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.811+0000 7fa00f7fe640 1 -- 172.21.15.125:0/3592676140 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa018017020 con 0x7fa02806c0a0 2023-10-18T11:41:21.814 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T11:41:21.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.814+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fa01406af60 msgr2=0x7fa01406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:21.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.814+0000 7fa02d56e640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fa01406af60 0x7fa01406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.814+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 msgr2=0x7fa0281d93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.814+0000 7fa02d56e640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 0x7fa0281d93a0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fa01800d720 tx=0x7fa01800dbe0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.815+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 shutdown_connections 2023-10-18T11:41:21.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.815+0000 7fa02d56e640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fa01406af60 0x7fa01406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.818 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.815+0000 7fa02d56e640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa02813e520 0x7fa02812c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.815+0000 7fa02d56e640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa028141690 0x7fa028140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.815+0000 7fa02d56e640 1 --2- 172.21.15.125:0/3592676140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa02806c0a0 0x7fa0281d93a0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.815+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 >> 172.21.15.125:0/3592676140 conn(0x7fa028069930 msgr2=0x7fa0280addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:21.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.815+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 shutdown_connections 2023-10-18T11:41:21.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.815+0000 7fa02d56e640 1 -- 172.21.15.125:0/3592676140 wait complete. 2023-10-18T11:41:21.833 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:41:21.930 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.928+0000 7fbaf344a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:41:21.950 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:21.947+0000 7f7890c45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:41:21.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.953+0000 7fe79c9e2640 1 -- 172.21.15.125:0/1362604680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79813bd70 msgr2=0x7fe798070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.953+0000 7fe79c9e2640 1 --2- 172.21.15.125:0/1362604680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79813bd70 0x7fe798070ce0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fe788009830 tx=0x7fe78802bdf0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.953+0000 7fe79c9e2640 1 -- 172.21.15.125:0/1362604680 shutdown_connections 2023-10-18T11:41:21.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.953+0000 7fe79c9e2640 1 --2- 172.21.15.125:0/1362604680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe79813bd70 0x7fe798070ce0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.953+0000 7fe79c9e2640 1 -- 172.21.15.125:0/1362604680 >> 172.21.15.125:0/1362604680 conn(0x7fe7980af730 msgr2=0x7fe7980afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:21.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.953+0000 7fe79c9e2640 1 -- 172.21.15.125:0/1362604680 shutdown_connections 2023-10-18T11:41:21.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.953+0000 7fe79c9e2640 1 -- 172.21.15.125:0/1362604680 wait complete. 2023-10-18T11:41:21.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.954+0000 7fe79c9e2640 1 Processor -- start 2023-10-18T11:41:21.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79c9e2640 1 -- start start 2023-10-18T11:41:21.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79c9e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe798069720 0x7fe79814d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79c9e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe79813bd70 0x7fe798159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79c9e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe79813ebe0 0x7fe79815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79c9e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798135330 con 0x7fe798069720 2023-10-18T11:41:21.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79c9e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79814d590 con 0x7fe79813ebe0 2023-10-18T11:41:21.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79c9e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe798159f20 con 0x7fe79813bd70 2023-10-18T11:41:21.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe798069720 0x7fe79814d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe798069720 0x7fe79814d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39844/0 (socket says 172.21.15.125:39844) 2023-10-18T11:41:21.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe79659c640 1 -- 172.21.15.125:0/2027657507 learned_addr learned my addr 172.21.15.125:0/2027657507 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:21.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.955+0000 7fe796d9d640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe79813ebe0 0x7fe79815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe795d9b640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe79813bd70 0x7fe798159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:21.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe79659c640 1 -- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe79813bd70 msgr2=0x7fe798159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe79659c640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe79813bd70 0x7fe798159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe79659c640 1 -- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe79813ebe0 msgr2=0x7fe79815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:21.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe79659c640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe79813ebe0 0x7fe79815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:21.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe79659c640 1 -- 172.21.15.125:0/2027657507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788009490 con 0x7fe798069720 2023-10-18T11:41:21.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe79659c640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe798069720 0x7fe79814d050 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe788009950 tx=0x7fe788003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:21.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe786ffd640 1 -- 172.21.15.125:0/2027657507 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788003e20 con 0x7fe798069720 2023-10-18T11:41:21.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe786ffd640 1 -- 172.21.15.125:0/2027657507 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe788005110 con 0x7fe798069720 2023-10-18T11:41:21.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe786ffd640 1 -- 172.21.15.125:0/2027657507 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788005520 con 0x7fe798069720 2023-10-18T11:41:21.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79815a200 con 0x7fe798069720 2023-10-18T11:41:21.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.956+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe798134fc0 con 0x7fe798069720 2023-10-18T11:41:21.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.957+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe798134b40 con 0x7fe798069720 2023-10-18T11:41:21.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.958+0000 7fe786ffd640 1 -- 172.21.15.125:0/2027657507 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe7880047a0 con 0x7fe798069720 2023-10-18T11:41:21.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.958+0000 7fe786ffd640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fe77806af60 0x7fe77806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:21.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.959+0000 7fe786ffd640 1 -- 172.21.15.125:0/2027657507 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fe7880ac340 con 0x7fe798069720 2023-10-18T11:41:21.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.959+0000 7fe795d9b640 1 -- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fe77806af60 msgr2=0x7fe77806d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:21.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.959+0000 7fe795d9b640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fe77806af60 0x7fe77806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:21.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:21.967+0000 7fe786ffd640 1 -- 172.21.15.125:0/2027657507 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe798134b40 con 0x7fe798069720 2023-10-18T11:41:21.998 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:21.996+0000 7fbaf344a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:22.037 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:41:22.039 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:41:22.043 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.041+0000 7f7890c45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:22.054 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:22.052+0000 7fbaf344a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:41:22.110 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.108+0000 7f7890c45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:41:22.139 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:22.137+0000 7fbaf344a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:41:22.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.160+0000 7fe795d9b640 1 -- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fe77806af60 msgr2=0x7fe77806d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:22.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.160+0000 7fe795d9b640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fe77806af60 0x7fe77806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:41:22.170 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.168+0000 7f7890c45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:41:22.214 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:22.212+0000 7fbaf344a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:41:22.222 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.220+0000 7f7890c45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:41:22.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.242+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe798134b40 con 0x7fe798069720 2023-10-18T11:41:22.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.243+0000 7fe786ffd640 1 -- 172.21.15.125:0/2027657507 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe798134b40 con 0x7fe798069720 2023-10-18T11:41:22.245 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T11:41:22.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fe77806af60 msgr2=0x7fe77806d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:22.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fe77806af60 0x7fe77806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe798069720 msgr2=0x7fe79814d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:22.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe798069720 0x7fe79814d050 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe788009950 tx=0x7fe788003820 comp rx=0 tx=0).stop 2023-10-18T11:41:22.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 shutdown_connections 2023-10-18T11:41:22.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7fe77806af60 0x7fe77806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe79813ebe0 0x7fe79815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe79813bd70 0x7fe798159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 --2- 172.21.15.125:0/2027657507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe798069720 0x7fe79814d050 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 >> 172.21.15.125:0/2027657507 conn(0x7fe7980af730 msgr2=0x7fe79806c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:22.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 shutdown_connections 2023-10-18T11:41:22.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.246+0000 7fe79c9e2640 1 -- 172.21.15.125:0/2027657507 wait complete. 2023-10-18T11:41:22.265 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T11:41:22.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.460+0000 7f536f76e640 1 -- 172.21.15.125:0/3772133563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cd740 msgr2=0x7f53680cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:22.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.460+0000 7f536f76e640 1 --2- 172.21.15.125:0/3772133563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cd740 0x7f53680cdba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5358009830 tx=0x7f535802bcd0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.460+0000 7f536f76e640 1 -- 172.21.15.125:0/3772133563 shutdown_connections 2023-10-18T11:41:22.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.460+0000 7f536f76e640 1 --2- 172.21.15.125:0/3772133563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cd740 0x7f53680cdba0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.460+0000 7f536f76e640 1 --2- 172.21.15.125:0/3772133563 >> v2:172.21.15.139:3300/0 conn(0x7f536815aff0 0x7f53680d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.460+0000 7f536f76e640 1 -- 172.21.15.125:0/3772133563 >> 172.21.15.125:0/3772133563 conn(0x7f536809a010 msgr2=0x7f53680d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:22.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.460+0000 7f536f76e640 1 -- 172.21.15.125:0/3772133563 shutdown_connections 2023-10-18T11:41:22.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.460+0000 7f536f76e640 1 -- 172.21.15.125:0/3772133563 wait complete. 2023-10-18T11:41:22.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.461+0000 7f536f76e640 1 Processor -- start 2023-10-18T11:41:22.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.461+0000 7f536f76e640 1 -- start start 2023-10-18T11:41:22.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.461+0000 7f536f76e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cb980 0x7f53681f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:22.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536f76e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53680cd740 0x7f536815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:22.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536f76e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536815aff0 0x7f536814f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:22.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536f76e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53681e4780 con 0x7f53680cd740 2023-10-18T11:41:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536f76e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536815c4c0 con 0x7f536815aff0 2023-10-18T11:41:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536f76e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536815c7a0 con 0x7f53680cb980 2023-10-18T11:41:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536cd09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53680cd740 0x7f536815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536d50a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cb980 0x7f53681f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536d50a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cb980 0x7f53681f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:58660/0 (socket says 172.21.15.125:58660) 2023-10-18T11:41:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536d50a640 1 -- 172.21.15.125:0/4174872309 learned_addr learned my addr 172.21.15.125:0/4174872309 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536cd09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53680cd740 0x7f536815df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39858/0 (socket says 172.21.15.125:39858) 2023-10-18T11:41:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536dd0b640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536815aff0 0x7f536814f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536cd09640 1 -- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cb980 msgr2=0x7f53681f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536cd09640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cb980 0x7f53681f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536cd09640 1 -- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536815aff0 msgr2=0x7f536814f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:22.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536cd09640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536815aff0 0x7f536814f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536cd09640 1 -- 172.21.15.125:0/4174872309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5358009490 con 0x7f53680cd740 2023-10-18T11:41:22.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536d50a640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cb980 0x7f53681f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:41:22.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f536cd09640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53680cd740 0x7f536815df50 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5358033000 tx=0x7f53580038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:22.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f535dffb640 1 -- 172.21.15.125:0/4174872309 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358003ea0 con 0x7f53680cd740 2023-10-18T11:41:22.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f535dffb640 1 -- 172.21.15.125:0/4174872309 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5358034440 con 0x7f53680cd740 2023-10-18T11:41:22.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.462+0000 7f535dffb640 1 -- 172.21.15.125:0/4174872309 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358034850 con 0x7f53680cd740 2023-10-18T11:41:22.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.463+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53680d1d50 con 0x7f53680cd740 2023-10-18T11:41:22.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.463+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53680c5dc0 con 0x7f53680cd740 2023-10-18T11:41:22.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.464+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53680ce190 con 0x7f53680cd740 2023-10-18T11:41:22.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.465+0000 7f535dffb640 1 -- 172.21.15.125:0/4174872309 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f535803b040 con 0x7f53680cd740 2023-10-18T11:41:22.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.465+0000 7f535dffb640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f534c06af60 0x7f534c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:22.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.465+0000 7f535dffb640 1 -- 172.21.15.125:0/4174872309 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f53580abcc0 con 0x7f53680cd740 2023-10-18T11:41:22.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.465+0000 7f536d50a640 1 -- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f534c06af60 msgr2=0x7f534c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:22.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.466+0000 7f536d50a640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f534c06af60 0x7f534c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:22.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.470+0000 7f535dffb640 1 -- 172.21.15.125:0/4174872309 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5358076a90 con 0x7f53680cd740 2023-10-18T11:41:22.593 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.591+0000 7f7890c45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:41:22.632 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:22.629+0000 7fbaf344a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:41:22.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.656+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f53680690e0 con 0x7f53680cd740 2023-10-18T11:41:22.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.656+0000 7f535dffb640 1 -- 172.21.15.125:0/4174872309 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v160) v1 ==== 80+0+477 (secure 0 0 0) 0x7f5358039080 con 0x7f53680cd740 2023-10-18T11:41:22.659 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/555070972 2023-10-19T10:45:15.816055+0000 2023-10-18T11:41:22.660 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1072662486 2023-10-19T10:45:15.816055+0000 2023-10-18T11:41:22.660 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/3182280683 2023-10-19T10:45:15.816055+0000 2023-10-18T11:41:22.660 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1079295697 2023-10-19T10:45:15.816055+0000 2023-10-18T11:41:22.660 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2400687467 2023-10-19T10:45:15.816055+0000 2023-10-18T11:41:22.661 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/3182280683 2023-10-19T10:45:15.816055+0000 2023-10-18T11:41:22.661 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1927085609 2023-10-19T10:45:15.816055+0000 2023-10-18T11:41:22.661 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1424238360 2023-10-19T10:45:15.816055+0000 2023-10-18T11:41:22.662 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T11:41:22.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f534c06af60 msgr2=0x7f534c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:22.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f534c06af60 0x7f534c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53680cd740 msgr2=0x7f536815df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:22.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53680cd740 0x7f536815df50 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5358033000 tx=0x7f53580038a0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 shutdown_connections 2023-10-18T11:41:22.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f534c06af60 0x7f534c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536815aff0 0x7f536814f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f53680cd740 0x7f536815df50 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.658+0000 7f536f76e640 1 --2- 172.21.15.125:0/4174872309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53680cb980 0x7f53681f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:22.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.659+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 >> 172.21.15.125:0/4174872309 conn(0x7f536809a010 msgr2=0x7f536806b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:22.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.659+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 shutdown_connections 2023-10-18T11:41:22.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:22.659+0000 7f536f76e640 1 -- 172.21.15.125:0/4174872309 wait complete. 2023-10-18T11:41:22.671 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T11:41:22.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:22.672 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T11:41:22.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[136921]: fuse finished with error 0 and tester_r 0 2023-10-18T11:41:22.699 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T11:41:22.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:22.700 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T11:41:22.727 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.724+0000 7f7890c45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:41:22.810 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:22.808+0000 7fbaf344a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:41:22.846 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.844+0000 7f7890c45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:41:22.872 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:41:22.873 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:22.889 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:41:22.889 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T11:41:22.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:22.890 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T11:41:22.891 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.887+0000 7f7890c45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:41:22.915 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:22.913+0000 7fbaf344a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:41:22.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:41:22.944 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T11:41:22.945 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:41:22.945 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:22.958 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:22.956+0000 7fbaf344a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:41:22.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:41:22.960 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T11:41:22.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:22.961 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T11:41:22.987 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:22.985+0000 7f7890c45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:41:23.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:41:23.015 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T11:41:23.016 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:41:23.016 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:23.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:41:23.032 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T11:41:23.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:23.033 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T11:41:23.058 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:23.056+0000 7fbaf344a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:41:23.081 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:23.078+0000 7f7890c45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:41:23.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:41:23.087 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T11:41:23.087 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:41:23.156 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:23.154+0000 7fbaf344a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:41:23.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.192+0000 7ff40910c640 1 -- 172.21.15.125:0/2086631015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff40415b2a0 msgr2=0x7ff4040d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.192+0000 7ff40910c640 1 --2- 172.21.15.125:0/2086631015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff40415b2a0 0x7ff4040d1910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009830 tx=0x7ff3ec02bd60 comp rx=0 tx=0).stop 2023-10-18T11:41:23.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.192+0000 7ff40910c640 1 -- 172.21.15.125:0/2086631015 shutdown_connections 2023-10-18T11:41:23.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.192+0000 7ff40910c640 1 --2- 172.21.15.125:0/2086631015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff40415b2a0 0x7ff4040d1910 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.192+0000 7ff40910c640 1 -- 172.21.15.125:0/2086631015 >> 172.21.15.125:0/2086631015 conn(0x7ff4040d0d60 msgr2=0x7ff4040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:23.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.193+0000 7ff40910c640 1 -- 172.21.15.125:0/2086631015 shutdown_connections 2023-10-18T11:41:23.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.193+0000 7ff40910c640 1 -- 172.21.15.125:0/2086631015 wait complete. 2023-10-18T11:41:23.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.194+0000 7ff40910c640 1 Processor -- start 2023-10-18T11:41:23.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.194+0000 7ff40910c640 1 -- start start 2023-10-18T11:41:23.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40910c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4040ceb40 0x7ff4041511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40910c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4040cf340 0x7ff4040b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40910c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff40415b2a0 0x7ff4040b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40910c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40409bd80 con 0x7ff40415b2a0 2023-10-18T11:41:23.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40910c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40415d040 con 0x7ff4040cf340 2023-10-18T11:41:23.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40910c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40415c4c0 con 0x7ff4040ceb40 2023-10-18T11:41:23.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff40415b2a0 0x7ff4040b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff3fa59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4040cf340 0x7ff4040b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff40415b2a0 0x7ff4040b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39860/0 (socket says 172.21.15.125:39860) 2023-10-18T11:41:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40359e640 1 -- 172.21.15.125:0/2939029106 learned_addr learned my addr 172.21.15.125:0/2939029106 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff3fa59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4040cf340 0x7ff4040b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:52568/0 (socket says 172.21.15.125:52568) 2023-10-18T11:41:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff402d9d640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4040ceb40 0x7ff4041511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff3fa59c640 1 -- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4040ceb40 msgr2=0x7ff4041511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff3fa59c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4040ceb40 0x7ff4041511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff3fa59c640 1 -- 172.21.15.125:0/2939029106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff40415b2a0 msgr2=0x7ff4040b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff3fa59c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff40415b2a0 0x7ff4040b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff3fa59c640 1 -- 172.21.15.125:0/2939029106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec009490 con 0x7ff4040cf340 2023-10-18T11:41:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.195+0000 7ff40359e640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff40415b2a0 0x7ff4040b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:41:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.196+0000 7ff3fa59c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4040cf340 0x7ff4040b5b40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec033000 tx=0x7ff3ec0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.196+0000 7ff3fbfff640 1 -- 172.21.15.125:0/2939029106 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec003ea0 con 0x7ff4040cf340 2023-10-18T11:41:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.196+0000 7ff3fbfff640 1 -- 172.21.15.125:0/2939029106 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff3ec005c90 con 0x7ff4040cf340 2023-10-18T11:41:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.196+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40415c7a0 con 0x7ff4040cf340 2023-10-18T11:41:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.196+0000 7ff3fbfff640 1 -- 172.21.15.125:0/2939029106 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec005260 con 0x7ff4040cf340 2023-10-18T11:41:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.196+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40415cd60 con 0x7ff4040cf340 2023-10-18T11:41:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.197+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40409bef0 con 0x7ff4040cf340 2023-10-18T11:41:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.197+0000 7ff3fbfff640 1 -- 172.21.15.125:0/2939029106 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff3ec0053c0 con 0x7ff4040cf340 2023-10-18T11:41:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.198+0000 7ff3fbfff640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ff3e406af60 0x7ff3e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.198+0000 7ff402d9d640 1 -- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ff3e406af60 msgr2=0x7ff3e406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.198+0000 7ff3fbfff640 1 -- 172.21.15.125:0/2939029106 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6489+0+0 (secure 0 0 0) 0x7ff3ec0ac5a0 con 0x7ff4040cf340 2023-10-18T11:41:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.198+0000 7ff402d9d640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ff3e406af60 0x7ff3e406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.202+0000 7ff3fbfff640 1 -- 172.21.15.125:0/2939029106 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff40409bef0 con 0x7ff4040cf340 2023-10-18T11:41:23.223 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:23.220+0000 7f7890c45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:41:23.265 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:23.262+0000 7f7890c45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:23.301 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:23.299+0000 7fbaf344a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:41:23.344 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:23.342+0000 7fbaf344a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:23.368 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:23.366+0000 7f7890c45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:41:23.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.399+0000 7ff402d9d640 1 -- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ff3e406af60 msgr2=0x7ff3e406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:23.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.399+0000 7ff402d9d640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ff3e406af60 0x7ff3e406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:41:23.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.411+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff40409bef0 con 0x7ff4040cf340 2023-10-18T11:41:23.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.412+0000 7ff3fbfff640 1 -- 172.21.15.125:0/2939029106 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7ff40409bef0 con 0x7ff4040cf340 2023-10-18T11:41:23.414 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:23.415 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2692191151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2692191151},{"type":"v1","addr":"172.21.15.139:6836","nonce":2692191151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1995790917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1995790917},{"type":"v1","addr":"172.21.15.139:6837","nonce":1995790917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/55232611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":55232611},{"type":"v1","addr":"172.21.15.125:6834","nonce":55232611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:59.877487+0000","modified":"2023-10-18T10:45:01.875939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9765":{"gid":9765,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/631315131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":631315131},{"type":"v1","addr":"172.21.15.125:6835","nonce":631315131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T11:41:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 60 2023-10-18T11:41:23.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ff3e406af60 msgr2=0x7ff3e406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ff3e406af60 0x7ff3e406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4040cf340 msgr2=0x7ff4040b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4040cf340 0x7ff4040b5b40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec033000 tx=0x7ff3ec0038a0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 shutdown_connections 2023-10-18T11:41:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7ff3e406af60 0x7ff3e406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4040cf340 0x7ff4040b5b40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff40415b2a0 0x7ff4040b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 --2- 172.21.15.125:0/2939029106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4040ceb40 0x7ff4041511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 >> 172.21.15.125:0/2939029106 conn(0x7ff4040d0d60 msgr2=0x7ff40406b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 shutdown_connections 2023-10-18T11:41:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.414+0000 7ff40910c640 1 -- 172.21.15.125:0/2939029106 wait complete. 2023-10-18T11:41:23.427 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:23.454 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:23.452+0000 7fbaf344a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:41:23.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 -- 172.21.15.125:0/4202652759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe015b2a0 msgr2=0x7f2fe00cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/4202652759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe015b2a0 0x7f2fe00cf720 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0089ae0 tx=0x7f2fd002bd10 comp rx=0 tx=0).stop 2023-10-18T11:41:23.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 -- 172.21.15.125:0/4202652759 shutdown_connections 2023-10-18T11:41:23.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/4202652759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe015b2a0 0x7f2fe00cf720 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/4202652759 >> v2:172.21.15.139:3300/0 conn(0x7f2fe00ceb40 0x7f2fe00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/4202652759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fe00cf340 0x7f2fe00d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 -- 172.21.15.125:0/4202652759 >> 172.21.15.125:0/4202652759 conn(0x7f2fe00d0d60 msgr2=0x7f2fe00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:23.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 -- 172.21.15.125:0/4202652759 shutdown_connections 2023-10-18T11:41:23.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.529+0000 7f2fe84a2640 1 -- 172.21.15.125:0/4202652759 wait complete. 2023-10-18T11:41:23.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.530+0000 7f2fe84a2640 1 Processor -- start 2023-10-18T11:41:23.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.530+0000 7f2fe84a2640 1 -- start start 2023-10-18T11:41:23.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe84a2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fe00ceb40 0x7f2fe00b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe84a2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe00cf340 0x7f2fe006a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe84a2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fe015b2a0 0x7f2fe0158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe84a2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe01f5330 con 0x7f2fe00cf340 2023-10-18T11:41:23.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe84a2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b5a60 con 0x7f2fe015b2a0 2023-10-18T11:41:23.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe84a2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe01f4ce0 con 0x7f2fe00ceb40 2023-10-18T11:41:23.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe5a3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe00cf340 0x7f2fe006a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe623e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fe00ceb40 0x7f2fe00b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe6a3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fe015b2a0 0x7f2fe0158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe623e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fe00ceb40 0x7f2fe00b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:58690/0 (socket says 172.21.15.125:58690) 2023-10-18T11:41:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe6a3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fe015b2a0 0x7f2fe0158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:52582/0 (socket says 172.21.15.125:52582) 2023-10-18T11:41:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe5a3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe00cf340 0x7f2fe006a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39874/0 (socket says 172.21.15.125:39874) 2023-10-18T11:41:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe623e640 1 -- 172.21.15.125:0/2909027272 learned_addr learned my addr 172.21.15.125:0/2909027272 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe5a3d640 1 -- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fe00ceb40 msgr2=0x7f2fe00b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe5a3d640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fe00ceb40 0x7f2fe00b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe5a3d640 1 -- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fe015b2a0 msgr2=0x7f2fe0158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe5a3d640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fe015b2a0 0x7f2fe0158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe5a3d640 1 -- 172.21.15.125:0/2909027272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009560 con 0x7f2fe00cf340 2023-10-18T11:41:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe6a3f640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fe015b2a0 0x7f2fe0158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:41:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe623e640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fe00ceb40 0x7f2fe00b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:41:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.531+0000 7f2fe5a3d640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe00cf340 0x7f2fe006a200 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0033000 tx=0x7f2fd002da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.532+0000 7f2fd6ffd640 1 -- 172.21.15.125:0/2909027272 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0003800 con 0x7f2fe00cf340 2023-10-18T11:41:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.532+0000 7f2fd6ffd640 1 -- 172.21.15.125:0/2909027272 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2fd0003a50 con 0x7f2fe00cf340 2023-10-18T11:41:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.532+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe01f4fc0 con 0x7f2fe00cf340 2023-10-18T11:41:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.532+0000 7f2fd6ffd640 1 -- 172.21.15.125:0/2909027272 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0003e00 con 0x7f2fe00cf340 2023-10-18T11:41:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.532+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe00b5750 con 0x7f2fe00cf340 2023-10-18T11:41:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.533+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe01f4e20 con 0x7f2fe00cf340 2023-10-18T11:41:23.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.533+0000 7f2fd6ffd640 1 -- 172.21.15.125:0/2909027272 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f2fd003e450 con 0x7f2fe00cf340 2023-10-18T11:41:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.534+0000 7f2fd6ffd640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f2fc406af90 0x7f2fc406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.534+0000 7f2fd6ffd640 1 -- 172.21.15.125:0/2909027272 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f2fd00ac420 con 0x7f2fe00cf340 2023-10-18T11:41:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.534+0000 7f2fe623e640 1 -- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f2fc406af90 msgr2=0x7f2fc406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.534+0000 7f2fe623e640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f2fc406af90 0x7f2fc406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.538+0000 7f2fd6ffd640 1 -- 172.21.15.125:0/2909027272 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fe01f4e20 con 0x7f2fe00cf340 2023-10-18T11:41:23.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.714+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fe01f4e20 con 0x7f2fe00cf340 2023-10-18T11:41:23.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.715+0000 7f2fd6ffd640 1 -- 172.21.15.125:0/2909027272 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+17498 (secure 0 0 0) 0x7f2fe01f4e20 con 0x7f2fe00cf340 2023-10-18T11:41:23.718 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:23.718 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":161,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:22.751209+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:58.087653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:58.847874+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":161,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/555070972":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1072662486":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6833/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1079295697":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/2400687467":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6832/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1927085609":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1424238360":"2023-10-19T10:45:15.816055+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:23.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f2fc406af90 msgr2=0x7f2fc406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:23.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f2fc406af90 0x7f2fc406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe00cf340 msgr2=0x7f2fe006a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe00cf340 0x7f2fe006a200 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0033000 tx=0x7f2fd002da00 comp rx=0 tx=0).stop 2023-10-18T11:41:23.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 shutdown_connections 2023-10-18T11:41:23.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f2fc406af90 0x7f2fc406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fe015b2a0 0x7f2fe0158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe00cf340 0x7f2fe006a200 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 --2- 172.21.15.125:0/2909027272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fe00ceb40 0x7f2fe00b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.717+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 >> 172.21.15.125:0/2909027272 conn(0x7f2fe00d0d60 msgr2=0x7f2fe015a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:23.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.718+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 shutdown_connections 2023-10-18T11:41:23.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.718+0000 7f2fe84a2640 1 -- 172.21.15.125:0/2909027272 wait complete. 2023-10-18T11:41:23.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T10:44:58.087653+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T10:44:58.847874+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 113, 'snap_epoch': 161, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:41:23.731 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:41:23.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.876+0000 7f737c5a9640 1 -- 172.21.15.125:0/2649592685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740c96b0 msgr2=0x7f73740c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.876+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2649592685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740c96b0 0x7f73740c9b60 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f7364005fd0 tx=0x7f736402bd90 comp rx=0 tx=0).stop 2023-10-18T11:41:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.877+0000 7f737c5a9640 1 -- 172.21.15.125:0/2649592685 shutdown_connections 2023-10-18T11:41:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.877+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2649592685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740c96b0 0x7f73740c9b60 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.877+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2649592685 >> v2:172.21.15.139:3300/0 conn(0x7f737406af00 0x7f737406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.877+0000 7f737c5a9640 1 -- 172.21.15.125:0/2649592685 >> 172.21.15.125:0/2649592685 conn(0x7f7374069930 msgr2=0x7f7374069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.877+0000 7f737c5a9640 1 -- 172.21.15.125:0/2649592685 shutdown_connections 2023-10-18T11:41:23.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.877+0000 7f737c5a9640 1 -- 172.21.15.125:0/2649592685 wait complete. 2023-10-18T11:41:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.878+0000 7f737c5a9640 1 Processor -- start 2023-10-18T11:41:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.878+0000 7f737c5a9640 1 -- start start 2023-10-18T11:41:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.878+0000 7f737c5a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f737406af00 0x7f7374139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.878+0000 7f737c5a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740ab500 0x7f73740a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.878+0000 7f737c5a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73740c96b0 0x7f73740a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.878+0000 7f737c5a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740af750 con 0x7f73740ab500 2023-10-18T11:41:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.878+0000 7f737c5a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740a7490 con 0x7f73740c96b0 2023-10-18T11:41:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.878+0000 7f737c5a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73740a7770 con 0x7f737406af00 2023-10-18T11:41:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740ab500 0x7f73740a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740ab500 0x7f73740a8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39888/0 (socket says 172.21.15.125:39888) 2023-10-18T11:41:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 -- 172.21.15.125:0/2460567293 learned_addr learned my addr 172.21.15.125:0/2460567293 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f737a345640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f737406af00 0x7f7374139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f737ab46640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73740c96b0 0x7f73740a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 -- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f737406af00 msgr2=0x7f7374139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f737406af00 0x7f7374139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 -- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73740c96b0 msgr2=0x7f73740a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73740c96b0 0x7f73740a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 -- 172.21.15.125:0/2460567293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7364009560 con 0x7f73740ab500 2023-10-18T11:41:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f7379b44640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740ab500 0x7f73740a8ae0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f7364006010 tx=0x7f736402db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f736affd640 1 -- 172.21.15.125:0/2460567293 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640035f0 con 0x7f73740ab500 2023-10-18T11:41:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f736affd640 1 -- 172.21.15.125:0/2460567293 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7364003840 con 0x7f73740ab500 2023-10-18T11:41:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73740a7a50 con 0x7f73740ab500 2023-10-18T11:41:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f736affd640 1 -- 172.21.15.125:0/2460567293 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364003c50 con 0x7f73740ab500 2023-10-18T11:41:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.879+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737413a270 con 0x7f73740ab500 2023-10-18T11:41:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.880+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73740af900 con 0x7f73740ab500 2023-10-18T11:41:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.881+0000 7f736affd640 1 -- 172.21.15.125:0/2460567293 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f736403e450 con 0x7f73740ab500 2023-10-18T11:41:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.881+0000 7f736affd640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f735806af60 0x7f735806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.881+0000 7f737a345640 1 -- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f735806af60 msgr2=0x7f735806d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.881+0000 7f736affd640 1 -- 172.21.15.125:0/2460567293 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f73640ac4f0 con 0x7f73740ab500 2023-10-18T11:41:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.881+0000 7f737a345640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f735806af60 0x7f735806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:23.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:23.888+0000 7f736affd640 1 -- 172.21.15.125:0/2460567293 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73740af900 con 0x7f73740ab500 2023-10-18T11:41:23.993 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:23.991+0000 7f7890c45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:41:24.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.061+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73740690e0 con 0x7f73740ab500 2023-10-18T11:41:24.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.062+0000 7f736affd640 1 -- 172.21.15.125:0/2460567293 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v161) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7364077230 con 0x7f73740ab500 2023-10-18T11:41:24.064 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:41:24.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.063+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f735806af60 msgr2=0x7f735806d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:24.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.063+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f735806af60 0x7f735806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740ab500 msgr2=0x7f73740a8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740ab500 0x7f73740a8ae0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f7364006010 tx=0x7f736402db30 comp rx=0 tx=0).stop 2023-10-18T11:41:24.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 shutdown_connections 2023-10-18T11:41:24.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f735806af60 0x7f735806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73740c96b0 0x7f73740a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73740ab500 0x7f73740a8ae0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 --2- 172.21.15.125:0/2460567293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f737406af00 0x7f7374139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 >> 172.21.15.125:0/2460567293 conn(0x7f7374069930 msgr2=0x7f737415a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:24.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 shutdown_connections 2023-10-18T11:41:24.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.064+0000 7f737c5a9640 1 -- 172.21.15.125:0/2460567293 wait complete. 2023-10-18T11:41:24.076 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:41:24.110 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:24.107+0000 7fbaf344a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:41:24.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.183+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/1525697831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 msgr2=0x7f0a780c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.183+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/1525697831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 0x7f0a780c5550 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005fd0 tx=0x7f0a7002c2e0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.183+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/1525697831 shutdown_connections 2023-10-18T11:41:24.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.183+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/1525697831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 0x7f0a780c5550 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.183+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/1525697831 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a780cda60 0x7f0a780c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.183+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/1525697831 >> 172.21.15.125:0/1525697831 conn(0x7f0a780d0c20 msgr2=0x7f0a780d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:24.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.183+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/1525697831 shutdown_connections 2023-10-18T11:41:24.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.183+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/1525697831 wait complete. 2023-10-18T11:41:24.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.184+0000 7f0a7d7a0640 1 Processor -- start 2023-10-18T11:41:24.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.184+0000 7f0a7d7a0640 1 -- start start 2023-10-18T11:41:24.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a7d7a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a780cda60 0x7f0a781f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a7d7a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 0x7f0a78145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a7d7a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a7815b2f0 0x7f0a780b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a7d7a0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78147120 con 0x7f0a780ce2f0 2023-10-18T11:41:24.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a7d7a0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781464a0 con 0x7f0a780cda60 2023-10-18T11:41:24.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a7d7a0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78146780 con 0x7f0a7815b2f0 2023-10-18T11:41:24.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 0x7f0a78145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 0x7f0a78145a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39920/0 (socket says 172.21.15.125:39920) 2023-10-18T11:41:24.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a777fe640 1 -- 172.21.15.125:0/3931910544 learned_addr learned my addr 172.21.15.125:0/3931910544 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:24.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a7cf9f640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a7815b2f0 0x7f0a780b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a77fff640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a780cda60 0x7f0a781f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a777fe640 1 -- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a7815b2f0 msgr2=0x7f0a780b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a777fe640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a7815b2f0 0x7f0a780b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a777fe640 1 -- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a780cda60 msgr2=0x7f0a781f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a777fe640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a780cda60 0x7f0a781f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.185+0000 7f0a777fe640 1 -- 172.21.15.125:0/3931910544 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70009900 con 0x7f0a780ce2f0 2023-10-18T11:41:24.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.186+0000 7f0a777fe640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 0x7f0a78145a50 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e790 tx=0x7f0a6800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:24.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.186+0000 7f0a74ff9640 1 -- 172.21.15.125:0/3931910544 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68009580 con 0x7f0a780ce2f0 2023-10-18T11:41:24.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.186+0000 7f0a74ff9640 1 -- 172.21.15.125:0/3931910544 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a6800cdb0 con 0x7f0a780ce2f0 2023-10-18T11:41:24.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.186+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a78146a60 con 0x7f0a780ce2f0 2023-10-18T11:41:24.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.186+0000 7f0a74ff9640 1 -- 172.21.15.125:0/3931910544 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68010690 con 0x7f0a780ce2f0 2023-10-18T11:41:24.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.186+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781f1060 con 0x7f0a780ce2f0 2023-10-18T11:41:24.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.187+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a78066960 con 0x7f0a780ce2f0 2023-10-18T11:41:24.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.188+0000 7f0a74ff9640 1 -- 172.21.15.125:0/3931910544 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 89) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f0a680107f0 con 0x7f0a780ce2f0 2023-10-18T11:41:24.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.188+0000 7f0a74ff9640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f0a6406af60 0x7f0a6406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.188+0000 7f0a77fff640 1 -- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f0a6406af60 msgr2=0x7f0a6406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1032201494 2023-10-18T11:41:24.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.188+0000 7f0a74ff9640 1 -- 172.21.15.125:0/3931910544 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f0a68014070 con 0x7f0a780ce2f0 2023-10-18T11:41:24.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.188+0000 7f0a77fff640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f0a6406af60 0x7f0a6406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:24.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.193+0000 7f0a74ff9640 1 -- 172.21.15.125:0/3931910544 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a68051ca0 con 0x7f0a780ce2f0 2023-10-18T11:41:24.256 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.254+0000 7f7890c45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:41:24.304 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.302+0000 7f7890c45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:41:24.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.365+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a780d16f0 con 0x7f0a780ce2f0 2023-10-18T11:41:24.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.365+0000 7f0a74ff9640 1 -- 172.21.15.125:0/3931910544 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v161) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a680551d0 con 0x7f0a780ce2f0 2023-10-18T11:41:24.368 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T11:41:24.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f0a6406af60 msgr2=0x7f0a6406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:24.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f0a6406af60 0x7f0a6406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 msgr2=0x7f0a78145a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 0x7f0a78145a50 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e790 tx=0x7f0a6800ec50 comp rx=0 tx=0).stop 2023-10-18T11:41:24.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 shutdown_connections 2023-10-18T11:41:24.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a7815b2f0 0x7f0a780b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a780ce2f0 0x7f0a78145a50 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:6832/1032201494,v1:172.21.15.139:6833/1032201494] conn(0x7f0a6406af60 0x7f0a6406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 --2- 172.21.15.125:0/3931910544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a780cda60 0x7f0a781f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 >> 172.21.15.125:0/3931910544 conn(0x7f0a780d0c20 msgr2=0x7f0a780c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:24.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 shutdown_connections 2023-10-18T11:41:24.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.367+0000 7f0a7d7a0640 1 -- 172.21.15.125:0/3931910544 wait complete. 2023-10-18T11:41:24.381 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:41:24.385 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:24.383+0000 7fbaf344a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:41:24.403 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.401+0000 7f7890c45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:41:24.440 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:24.438+0000 7fbaf344a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:41:24.478 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.475+0000 7f7812fa2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.478 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.475+0000 7f7812fa2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.479 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.475+0000 7f7812fa2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.479 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.475+0000 7f7812fa2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.479 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.475+0000 7f7812fa2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.479 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.476+0000 7f7815fa8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.479 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.476+0000 7f7815fa8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.480 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.476+0000 7f7815fa8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.480 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.476+0000 7f7815fa8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.480 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:24.476+0000 7f7815fa8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:24.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.492+0000 7f7591c7c640 1 -- 172.21.15.125:0/3007520463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0ceb40 msgr2=0x7f758c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.492+0000 7f7591c7c640 1 --2- 172.21.15.125:0/3007520463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0ceb40 0x7f758c0b0650 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f7584005fd0 tx=0x7f758402c240 comp rx=0 tx=0).stop 2023-10-18T11:41:24.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.492+0000 7f7591c7c640 1 -- 172.21.15.125:0/3007520463 shutdown_connections 2023-10-18T11:41:24.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.492+0000 7f7591c7c640 1 --2- 172.21.15.125:0/3007520463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0ceb40 0x7f758c0b0650 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.492+0000 7f7591c7c640 1 --2- 172.21.15.125:0/3007520463 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f758c0cf340 0x7f758c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.492+0000 7f7591c7c640 1 -- 172.21.15.125:0/3007520463 >> 172.21.15.125:0/3007520463 conn(0x7f758c0d0d60 msgr2=0x7f758c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:24.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.492+0000 7f7591c7c640 1 -- 172.21.15.125:0/3007520463 shutdown_connections 2023-10-18T11:41:24.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.492+0000 7f7591c7c640 1 -- 172.21.15.125:0/3007520463 wait complete. 2023-10-18T11:41:24.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.493+0000 7f7591c7c640 1 Processor -- start 2023-10-18T11:41:24.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.493+0000 7f7591c7c640 1 -- start start 2023-10-18T11:41:24.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f7591c7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f758c0ceb40 0x7f758c06a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f7591c7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0cf340 0x7f758c15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f7591c7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f758c15b2a0 0x7f758c0c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f7591c7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15d660 con 0x7f758c0cf340 2023-10-18T11:41:24.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f7591c7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15cae0 con 0x7f758c0ceb40 2023-10-18T11:41:24.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f7591c7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15cdc0 con 0x7f758c15b2a0 2023-10-18T11:41:24.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f758affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0cf340 0x7f758c15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f758affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0cf340 0x7f758c15e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39936/0 (socket says 172.21.15.125:39936) 2023-10-18T11:41:24.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f758affd640 1 -- 172.21.15.125:0/2499742388 learned_addr learned my addr 172.21.15.125:0/2499742388 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:24.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f758b7fe640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f758c0ceb40 0x7f758c06a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.494+0000 7f758bfff640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f758c15b2a0 0x7f758c0c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f758affd640 1 -- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f758c15b2a0 msgr2=0x7f758c0c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f758affd640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f758c15b2a0 0x7f758c0c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f758affd640 1 -- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f758c0ceb40 msgr2=0x7f758c06a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f758affd640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f758c0ceb40 0x7f758c06a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f758affd640 1 -- 172.21.15.125:0/2499742388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009900 con 0x7f758c0cf340 2023-10-18T11:41:24.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f758b7fe640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f758c0ceb40 0x7f758c06a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:24.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f758affd640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0cf340 0x7f758c15e1c0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f757c00c3c0 tx=0x7f757c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:24.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f7573fff640 1 -- 172.21.15.125:0/2499742388 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c0052d0 con 0x7f758c0cf340 2023-10-18T11:41:24.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c15d0a0 con 0x7f758c0cf340 2023-10-18T11:41:24.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f7573fff640 1 -- 172.21.15.125:0/2499742388 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f757c0035f0 con 0x7f758c0cf340 2023-10-18T11:41:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f7573fff640 1 -- 172.21.15.125:0/2499742388 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c003a00 con 0x7f758c0cf340 2023-10-18T11:41:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.495+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c15c720 con 0x7f758c0cf340 2023-10-18T11:41:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.496+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c15c8b0 con 0x7f758c0cf340 2023-10-18T11:41:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.497+0000 7f7573fff640 1 -- 172.21.15.125:0/2499742388 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 90) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f757c003b60 con 0x7f758c0cf340 2023-10-18T11:41:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.497+0000 7f7573fff640 1 -- 172.21.15.125:0/2499742388 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f757c088ca0 con 0x7f758c0cf340 2023-10-18T11:41:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.501+0000 7f7573fff640 1 -- 172.21.15.125:0/2499742388 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f757c08d070 con 0x7f758c0cf340 2023-10-18T11:41:24.549 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:24.547+0000 7fbaf344a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:41:24.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.670+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f758c0cba00 con 0x7f758c0cf340 2023-10-18T11:41:24.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.671+0000 7f7573fff640 1 -- 172.21.15.125:0/2499742388 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+11 (secure 0 0 0) 0x7f757c053ce0 con 0x7f758c0cf340 2023-10-18T11:41:24.673 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T11:41:24.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0cf340 msgr2=0x7f758c15e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0cf340 0x7f758c15e1c0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f757c00c3c0 tx=0x7f757c00c880 comp rx=0 tx=0).stop 2023-10-18T11:41:24.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 shutdown_connections 2023-10-18T11:41:24.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f758c15b2a0 0x7f758c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758c0cf340 0x7f758c15e1c0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 --2- 172.21.15.125:0/2499742388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f758c0ceb40 0x7f758c06a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 >> 172.21.15.125:0/2499742388 conn(0x7f758c0d0d60 msgr2=0x7f758c0d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:24.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 shutdown_connections 2023-10-18T11:41:24.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.673+0000 7f7591c7c640 1 -- 172.21.15.125:0/2499742388 wait complete. 2023-10-18T11:41:24.685 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:41:24.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.786+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4162115059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880cf8b0 msgr2=0x7fe7880b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.786+0000 7fe78d9f5640 1 --2- 172.21.15.125:0/4162115059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880cf8b0 0x7fe7880b42d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe780005fd0 tx=0x7fe78002c0e0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.786+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4162115059 shutdown_connections 2023-10-18T11:41:24.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.786+0000 7fe78d9f5640 1 --2- 172.21.15.125:0/4162115059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880cf8b0 0x7fe7880b42d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.786+0000 7fe78d9f5640 1 --2- 172.21.15.125:0/4162115059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe78815e5e0 0x7fe7880ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.786+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4162115059 >> 172.21.15.125:0/4162115059 conn(0x7fe7880d0c50 msgr2=0x7fe7880d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:24.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.786+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4162115059 shutdown_connections 2023-10-18T11:41:24.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.786+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4162115059 wait complete. 2023-10-18T11:41:24.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.787+0000 7fe78d9f5640 1 Processor -- start 2023-10-18T11:41:24.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.787+0000 7fe78d9f5640 1 -- start start 2023-10-18T11:41:24.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe78d9f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880b1ca0 0x7fe7880b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe78d9f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7880cf8b0 0x7fe788159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe78d9f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe78815e5e0 0x7fe78806a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:24.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe78d9f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881f5950 con 0x7fe7880cf8b0 2023-10-18T11:41:24.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe78d9f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880b03f0 con 0x7fe7880b1ca0 2023-10-18T11:41:24.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe78d9f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881f4cd0 con 0x7fe78815e5e0 2023-10-18T11:41:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe7867fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7880cf8b0 0x7fe788159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe7867fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7880cf8b0 0x7fe788159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39952/0 (socket says 172.21.15.125:39952) 2023-10-18T11:41:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe7867fc640 1 -- 172.21.15.125:0/4083824057 learned_addr learned my addr 172.21.15.125:0/4083824057 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe786ffd640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880b1ca0 0x7fe7880b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe7877fe640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe78815e5e0 0x7fe78806a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe7867fc640 1 -- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe78815e5e0 msgr2=0x7fe78806a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe7867fc640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe78815e5e0 0x7fe78806a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe7867fc640 1 -- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880b1ca0 msgr2=0x7fe7880b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.788+0000 7fe7867fc640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880b1ca0 0x7fe7880b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:24.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.789+0000 7fe7867fc640 1 -- 172.21.15.125:0/4083824057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe780009840 con 0x7fe7880cf8b0 2023-10-18T11:41:24.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.789+0000 7fe786ffd640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880b1ca0 0x7fe7880b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:41:24.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.789+0000 7fe7867fc640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7880cf8b0 0x7fe788159890 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00c490 tx=0x7fe77c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:24.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.789+0000 7fe76ffff640 1 -- 172.21.15.125:0/4083824057 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c012830 con 0x7fe7880cf8b0 2023-10-18T11:41:24.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.789+0000 7fe76ffff640 1 -- 172.21.15.125:0/4083824057 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe77c012a80 con 0x7fe7880cf8b0 2023-10-18T11:41:24.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.789+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7881f4fb0 con 0x7fe7880cf8b0 2023-10-18T11:41:24.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.789+0000 7fe76ffff640 1 -- 172.21.15.125:0/4083824057 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011380 con 0x7fe7880cf8b0 2023-10-18T11:41:24.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.789+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7881f5570 con 0x7fe7880cf8b0 2023-10-18T11:41:24.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.790+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7881f5a90 con 0x7fe7880cf8b0 2023-10-18T11:41:24.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.790+0000 7fe76ffff640 1 -- 172.21.15.125:0/4083824057 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 90) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe77c0114e0 con 0x7fe7880cf8b0 2023-10-18T11:41:24.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.791+0000 7fe76ffff640 1 -- 172.21.15.125:0/4083824057 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fe77c089290 con 0x7fe7880cf8b0 2023-10-18T11:41:24.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.795+0000 7fe76ffff640 1 -- 172.21.15.125:0/4083824057 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7881f5a90 con 0x7fe7880cf8b0 2023-10-18T11:41:25.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.998+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7881f5a90 con 0x7fe7880cf8b0 2023-10-18T11:41:25.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:24.999+0000 7fe76ffff640 1 -- 172.21.15.125:0/4083824057 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fe7881f5a90 con 0x7fe7880cf8b0 2023-10-18T11:41:25.002 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:25.002 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2692191151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2692191151},{"type":"v1","addr":"172.21.15.139:6836","nonce":2692191151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1995790917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1995790917},{"type":"v1","addr":"172.21.15.139:6837","nonce":1995790917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/55232611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":55232611},{"type":"v1","addr":"172.21.15.125:6834","nonce":55232611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:59.877487+0000","modified":"2023-10-18T10:45:01.875939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9765":{"gid":9765,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/631315131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":631315131},{"type":"v1","addr":"172.21.15.125:6835","nonce":631315131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T11:41:25.003 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 60 2023-10-18T11:41:25.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.000+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7880cf8b0 msgr2=0x7fe788159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.000+0000 7fe78d9f5640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7880cf8b0 0x7fe788159890 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00c490 tx=0x7fe77c00c950 comp rx=0 tx=0).stop 2023-10-18T11:41:25.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.000+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 shutdown_connections 2023-10-18T11:41:25.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.000+0000 7fe78d9f5640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe78815e5e0 0x7fe78806a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.000+0000 7fe78d9f5640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7880cf8b0 0x7fe788159890 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.000+0000 7fe78d9f5640 1 --2- 172.21.15.125:0/4083824057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880b1ca0 0x7fe7880b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.000+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 >> 172.21.15.125:0/4083824057 conn(0x7fe7880d0c50 msgr2=0x7fe78815bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:25.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.001+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 shutdown_connections 2023-10-18T11:41:25.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.001+0000 7fe78d9f5640 1 -- 172.21.15.125:0/4083824057 wait complete. 2023-10-18T11:41:25.013 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:25.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.113+0000 7f730b44b640 1 -- 172.21.15.125:0/740215077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 msgr2=0x7f730415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.113+0000 7f730b44b640 1 --2- 172.21.15.125:0/740215077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 0x7f730415a9f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7300005fd0 tx=0x7f730002c0e0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.113+0000 7f730b44b640 1 -- 172.21.15.125:0/740215077 shutdown_connections 2023-10-18T11:41:25.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.113+0000 7f730b44b640 1 --2- 172.21.15.125:0/740215077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7304140660 0x7f73040c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.113+0000 7f730b44b640 1 --2- 172.21.15.125:0/740215077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 0x7f730415a9f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.113+0000 7f730b44b640 1 -- 172.21.15.125:0/740215077 >> 172.21.15.125:0/740215077 conn(0x7f7304069930 msgr2=0x7f7304069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:25.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.113+0000 7f730b44b640 1 -- 172.21.15.125:0/740215077 shutdown_connections 2023-10-18T11:41:25.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.113+0000 7f730b44b640 1 -- 172.21.15.125:0/740215077 wait complete. 2023-10-18T11:41:25.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.114+0000 7f730b44b640 1 Processor -- start 2023-10-18T11:41:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.114+0000 7f730b44b640 1 -- start start 2023-10-18T11:41:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f730b44b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f730406d170 0x7f7304159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f730b44b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 0x7f7304132170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f730b44b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7304140660 0x7f73041326b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f730b44b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730412dba0 con 0x7f730406d170 2023-10-18T11:41:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f730b44b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730412ddf0 con 0x7f730413d360 2023-10-18T11:41:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f730b44b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730412e0d0 con 0x7f7304140660 2023-10-18T11:41:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f73091e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f730406d170 0x7f7304159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f73091e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f730406d170 0x7f7304159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39958/0 (socket says 172.21.15.125:39958) 2023-10-18T11:41:25.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.115+0000 7f73091e7640 1 -- 172.21.15.125:0/2660722986 learned_addr learned my addr 172.21.15.125:0/2660722986 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73089e6640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 0x7f7304132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73099e8640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7304140660 0x7f73041326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73091e7640 1 -- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7304140660 msgr2=0x7f73041326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73091e7640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7304140660 0x7f73041326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73091e7640 1 -- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 msgr2=0x7f7304132170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73091e7640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 0x7f7304132170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73091e7640 1 -- 172.21.15.125:0/2660722986 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7300009840 con 0x7f730406d170 2023-10-18T11:41:25.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73089e6640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 0x7f7304132170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:25.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f73091e7640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f730406d170 0x7f7304159390 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f7300036040 tx=0x7f73000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:25.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f72f9ffb640 1 -- 172.21.15.125:0/2660722986 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000046f0 con 0x7f730406d170 2023-10-18T11:41:25.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f72f9ffb640 1 -- 172.21.15.125:0/2660722986 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f73000345a0 con 0x7f730406d170 2023-10-18T11:41:25.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730412e3b0 con 0x7f730406d170 2023-10-18T11:41:25.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f72f9ffb640 1 -- 172.21.15.125:0/2660722986 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000349b0 con 0x7f730406d170 2023-10-18T11:41:25.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.116+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730412e970 con 0x7f730406d170 2023-10-18T11:41:25.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.117+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73040690e0 con 0x7f730406d170 2023-10-18T11:41:25.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.118+0000 7f72f9ffb640 1 -- 172.21.15.125:0/2660722986 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 90) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f7300034b10 con 0x7f730406d170 2023-10-18T11:41:25.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.118+0000 7f72f9ffb640 1 -- 172.21.15.125:0/2660722986 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f73000ac5f0 con 0x7f730406d170 2023-10-18T11:41:25.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.122+0000 7f72f9ffb640 1 -- 172.21.15.125:0/2660722986 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73000776b0 con 0x7f730406d170 2023-10-18T11:41:25.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.288+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f730415a100 con 0x7f730406d170 2023-10-18T11:41:25.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.289+0000 7f72f9ffb640 1 -- 172.21.15.125:0/2660722986 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+17935 (secure 0 0 0) 0x7f730007acd0 con 0x7f730406d170 2023-10-18T11:41:25.292 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:25.292 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":162,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:24.415723+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:58.087653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:58.847874+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":161,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2893867972":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/912903828":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/1276835180":"2023-10-19T11:41:24.415677+0000","172.21.15.139:6832/1032201494":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/300871790":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/4164237327":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/555070972":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/562742640":"2023-10-19T11:41:24.415677+0000","172.21.15.139:6833/1032201494":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/1072662486":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6833/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1079295697":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/2400687467":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6832/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1927085609":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1424238360":"2023-10-19T10:45:15.816055+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:25.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f730406d170 msgr2=0x7f7304159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f730406d170 0x7f7304159390 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f7300036040 tx=0x7f73000042f0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 shutdown_connections 2023-10-18T11:41:25.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f730413d360 0x7f7304132170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7304140660 0x7f73041326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 --2- 172.21.15.125:0/2660722986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f730406d170 0x7f7304159390 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 >> 172.21.15.125:0/2660722986 conn(0x7f7304069930 msgr2=0x7f7304157600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:25.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 shutdown_connections 2023-10-18T11:41:25.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.291+0000 7f730b44b640 1 -- 172.21.15.125:0/2660722986 wait complete. 2023-10-18T11:41:25.304 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T11:41:25.305 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:41:25.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.405+0000 7f0209ede640 1 -- 172.21.15.125:0/4235563035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 msgr2=0x7f01e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.405+0000 7f0209ede640 1 --2- 172.21.15.125:0/4235563035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 0x7f01e400b980 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009ed0 tx=0x7f01e40057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.405+0000 7f0209ede640 1 -- 172.21.15.125:0/4235563035 shutdown_connections 2023-10-18T11:41:25.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.405+0000 7f0209ede640 1 --2- 172.21.15.125:0/4235563035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 0x7f01e400b980 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.405+0000 7f0209ede640 1 -- 172.21.15.125:0/4235563035 >> 172.21.15.125:0/4235563035 conn(0x7f02040d0d60 msgr2=0x7f02040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:25.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.405+0000 7f0209ede640 1 -- 172.21.15.125:0/4235563035 shutdown_connections 2023-10-18T11:41:25.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.405+0000 7f0209ede640 1 -- 172.21.15.125:0/4235563035 wait complete. 2023-10-18T11:41:25.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.406+0000 7f0209ede640 1 Processor -- start 2023-10-18T11:41:25.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.406+0000 7f0209ede640 1 -- start start 2023-10-18T11:41:25.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f0209ede640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 0x7f02040b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f0209ede640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0204150a30 0x7f0204150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f0209ede640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f020415bf20 0x7f020406b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f0209ede640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02041508c0 con 0x7f01e40050e0 2023-10-18T11:41:25.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f0209ede640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204151330 con 0x7f020415bf20 2023-10-18T11:41:25.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f0209ede640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02040b55d0 con 0x7f0204150a30 2023-10-18T11:41:25.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 0x7f02040b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 0x7f02040b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39972/0 (socket says 172.21.15.125:39972) 2023-10-18T11:41:25.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 -- 172.21.15.125:0/3112370704 learned_addr learned my addr 172.21.15.125:0/3112370704 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:25.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f0203fff640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f020415bf20 0x7f020406b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f01faffd640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0204150a30 0x7f0204150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 -- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0204150a30 msgr2=0x7f0204150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0204150a30 0x7f0204150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 -- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f020415bf20 msgr2=0x7f020406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f020415bf20 0x7f020406b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 -- 172.21.15.125:0/3112370704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040b58b0 con 0x7f01e40050e0 2023-10-18T11:41:25.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.407+0000 7f02037fe640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 0x7f02040b63d0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006630 tx=0x7f01f40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.408+0000 7f0200ff9640 1 -- 172.21.15.125:0/3112370704 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011900 con 0x7f01e40050e0 2023-10-18T11:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.408+0000 7f0200ff9640 1 -- 172.21.15.125:0/3112370704 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f01f4011a80 con 0x7f01e40050e0 2023-10-18T11:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.408+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02040b5b90 con 0x7f01e40050e0 2023-10-18T11:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.408+0000 7f0200ff9640 1 -- 172.21.15.125:0/3112370704 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011e90 con 0x7f01e40050e0 2023-10-18T11:41:25.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.408+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020415ba00 con 0x7f01e40050e0 2023-10-18T11:41:25.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.409+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02040cd1d0 con 0x7f01e40050e0 2023-10-18T11:41:25.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.409+0000 7f0200ff9640 1 -- 172.21.15.125:0/3112370704 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 90) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f01f40103c0 con 0x7f01e40050e0 2023-10-18T11:41:25.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.409+0000 7f0200ff9640 1 -- 172.21.15.125:0/3112370704 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f01f408a5a0 con 0x7f01e40050e0 2023-10-18T11:41:25.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.414+0000 7f0200ff9640 1 -- 172.21.15.125:0/3112370704 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01f4055660 con 0x7f01e40050e0 2023-10-18T11:41:25.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.424+0000 7f0200ff9640 1 -- 172.21.15.125:0/3112370704 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f01f4055110 con 0x7f01e40050e0 2023-10-18T11:41:25.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.425+0000 7f0200ff9640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f01e8070be0 0x7f01e8072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.428+0000 7f01faffd640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f01e8070be0 0x7f01e8072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.430+0000 7f01faffd640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f01e8070be0 0x7f01e8072fd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01ec006830 tx=0x7f01ec0067c0 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:25.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.618+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02040690e0 con 0x7f01e40050e0 2023-10-18T11:41:25.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.618+0000 7f0200ff9640 1 -- 172.21.15.125:0/3112370704 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f01f401f090 con 0x7f01e40050e0 2023-10-18T11:41:25.621 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:25.622 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2692191151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2692191151},{"type":"v1","addr":"172.21.15.139:6836","nonce":2692191151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1995790917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1995790917},{"type":"v1","addr":"172.21.15.139:6837","nonce":1995790917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/55232611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":55232611},{"type":"v1","addr":"172.21.15.125:6834","nonce":55232611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:59.877487+0000","modified":"2023-10-18T10:45:01.875939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9765":{"gid":9765,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/631315131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":631315131},{"type":"v1","addr":"172.21.15.125:6835","nonce":631315131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T11:41:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 60 2023-10-18T11:41:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f01e8070be0 msgr2=0x7f01e8072fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f01e8070be0 0x7f01e8072fd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01ec006830 tx=0x7f01ec0067c0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 msgr2=0x7f02040b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 0x7f02040b63d0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006630 tx=0x7f01f40039c0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 shutdown_connections 2023-10-18T11:41:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f01e8070be0 0x7f01e8072fd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f020415bf20 0x7f020406b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0204150a30 0x7f0204150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 --2- 172.21.15.125:0/3112370704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01e40050e0 0x7f02040b63d0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 >> 172.21.15.125:0/3112370704 conn(0x7f02040d0d60 msgr2=0x7f020415a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 shutdown_connections 2023-10-18T11:41:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.620+0000 7f0209ede640 1 -- 172.21.15.125:0/3112370704 wait complete. 2023-10-18T11:41:25.633 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:41:25.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.733+0000 7fe06e955640 1 -- 172.21.15.125:0/2354417051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 msgr2=0x7fe06806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.733+0000 7fe06e955640 1 --2- 172.21.15.125:0/2354417051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 0x7fe06806bb60 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fe0680cbdc0 tx=0x7fe05802bdb0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.733+0000 7fe06e955640 1 -- 172.21.15.125:0/2354417051 shutdown_connections 2023-10-18T11:41:25.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.733+0000 7fe06e955640 1 --2- 172.21.15.125:0/2354417051 >> v2:172.21.15.139:3300/0 conn(0x7fe068141690 0x7fe0680703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.733+0000 7fe06e955640 1 --2- 172.21.15.125:0/2354417051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 0x7fe06806bb60 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.733+0000 7fe06e955640 1 -- 172.21.15.125:0/2354417051 >> 172.21.15.125:0/2354417051 conn(0x7fe068069930 msgr2=0x7fe068069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:25.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.733+0000 7fe06e955640 1 -- 172.21.15.125:0/2354417051 shutdown_connections 2023-10-18T11:41:25.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.733+0000 7fe06e955640 1 -- 172.21.15.125:0/2354417051 wait complete. 2023-10-18T11:41:25.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.734+0000 7fe06e955640 1 Processor -- start 2023-10-18T11:41:25.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.734+0000 7fe06e955640 1 -- start start 2023-10-18T11:41:25.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe06e955640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06806c0a0 0x7fe0681ed850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe06e955640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 0x7fe068151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe06e955640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe068141690 0x7fe06813f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe06e955640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068154360 con 0x7fe06813e520 2023-10-18T11:41:25.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe06e955640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068152320 con 0x7fe068141690 2023-10-18T11:41:25.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe06e955640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe068153a30 con 0x7fe06806c0a0 2023-10-18T11:41:25.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 0x7fe068151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 0x7fe068151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39998/0 (socket says 172.21.15.125:39998) 2023-10-18T11:41:25.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 -- 172.21.15.125:0/1817376242 learned_addr learned my addr 172.21.15.125:0/1817376242 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:25.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe067fff640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06806c0a0 0x7fe0681ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe06cef2640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe068141690 0x7fe06813f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 -- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06806c0a0 msgr2=0x7fe0681ed850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06806c0a0 0x7fe0681ed850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 -- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe068141690 msgr2=0x7fe06813f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe068141690 0x7fe06813f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 -- 172.21.15.125:0/1817376242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe058009560 con 0x7fe06813e520 2023-10-18T11:41:25.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe067fff640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06806c0a0 0x7fe0681ed850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:25.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe06cef2640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe068141690 0x7fe06813f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:25.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.735+0000 7fe0677fe640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 0x7fe068151830 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009830 tx=0x7fe05c009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:25.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.736+0000 7fe064ff9640 1 -- 172.21.15.125:0/1817376242 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c012070 con 0x7fe06813e520 2023-10-18T11:41:25.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.736+0000 7fe064ff9640 1 -- 172.21.15.125:0/1817376242 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe05c0077a0 con 0x7fe06813e520 2023-10-18T11:41:25.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.736+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe068153d10 con 0x7fe06813e520 2023-10-18T11:41:25.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.736+0000 7fe064ff9640 1 -- 172.21.15.125:0/1817376242 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c007bb0 con 0x7fe06813e520 2023-10-18T11:41:25.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.736+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06812f120 con 0x7fe06813e520 2023-10-18T11:41:25.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.737+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe068154510 con 0x7fe06813e520 2023-10-18T11:41:25.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.737+0000 7fe064ff9640 1 -- 172.21.15.125:0/1817376242 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe05c016070 con 0x7fe06813e520 2023-10-18T11:41:25.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.737+0000 7fe064ff9640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe05406ac90 0x7fe05406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:25.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.737+0000 7fe064ff9640 1 -- 172.21.15.125:0/1817376242 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fe05c08adc0 con 0x7fe06813e520 2023-10-18T11:41:25.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.738+0000 7fe067fff640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe05406ac90 0x7fe05406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:25.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.738+0000 7fe067fff640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe05406ac90 0x7fe05406d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe058035000 tx=0x7fe058004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:25.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.742+0000 7fe064ff9640 1 -- 172.21.15.125:0/1817376242 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe068154510 con 0x7fe06813e520 2023-10-18T11:41:25.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.945+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0680690e0 con 0x7fe06813e520 2023-10-18T11:41:25.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.945+0000 7fe064ff9640 1 -- 172.21.15.125:0/1817376242 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fe05c08c020 con 0x7fe06813e520 2023-10-18T11:41:25.948 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:25.949 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":60,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2692191151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2692191151},{"type":"v1","addr":"172.21.15.139:6836","nonce":2692191151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1995790917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1995790917},{"type":"v1","addr":"172.21.15.139:6837","nonce":1995790917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/55232611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":55232611},{"type":"v1","addr":"172.21.15.125:6834","nonce":55232611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T10:44:59.877487+0000","modified":"2023-10-18T10:45:01.875939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9765":{"gid":9765,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/631315131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":631315131},{"type":"v1","addr":"172.21.15.125:6835","nonce":631315131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T11:41:25.949 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 60 2023-10-18T11:41:25.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe05406ac90 msgr2=0x7fe05406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe05406ac90 0x7fe05406d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe058035000 tx=0x7fe058004730 comp rx=0 tx=0).stop 2023-10-18T11:41:25.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 msgr2=0x7fe068151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:25.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 0x7fe068151830 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009830 tx=0x7fe05c009cf0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 shutdown_connections 2023-10-18T11:41:25.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe05406ac90 0x7fe05406d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe068141690 0x7fe06813f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe06813e520 0x7fe068151830 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 --2- 172.21.15.125:0/1817376242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe06806c0a0 0x7fe0681ed850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:25.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 >> 172.21.15.125:0/1817376242 conn(0x7fe068069930 msgr2=0x7fe0681586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:25.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.947+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 shutdown_connections 2023-10-18T11:41:25.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:25.948+0000 7fe06e955640 1 -- 172.21.15.125:0/1817376242 wait complete. 2023-10-18T11:41:25.960 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:26.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.060+0000 7f79f3697640 1 -- 172.21.15.125:0/1453508081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0afa20 msgr2=0x7f79ec158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.060+0000 7f79f3697640 1 --2- 172.21.15.125:0/1453508081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0afa20 0x7f79ec158300 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005fd0 tx=0x7f79e802bed0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.060+0000 7f79f3697640 1 -- 172.21.15.125:0/1453508081 shutdown_connections 2023-10-18T11:41:26.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.060+0000 7f79f3697640 1 --2- 172.21.15.125:0/1453508081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0afa20 0x7f79ec158300 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.060+0000 7f79f3697640 1 -- 172.21.15.125:0/1453508081 >> 172.21.15.125:0/1453508081 conn(0x7f79ec069930 msgr2=0x7f79ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:26.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.060+0000 7f79f3697640 1 -- 172.21.15.125:0/1453508081 shutdown_connections 2023-10-18T11:41:26.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.061+0000 7f79f3697640 1 -- 172.21.15.125:0/1453508081 wait complete. 2023-10-18T11:41:26.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f3697640 1 Processor -- start 2023-10-18T11:41:26.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f3697640 1 -- start start 2023-10-18T11:41:26.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f3697640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79ec0afa20 0x7f79ec142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f3697640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79ec0b0530 0x7f79ec1352e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f3697640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0cbd20 0x7f79ec14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f3697640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec135820 con 0x7f79ec0cbd20 2023-10-18T11:41:26.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f3697640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec14c550 con 0x7f79ec0b0530 2023-10-18T11:41:26.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f3697640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec14c2d0 con 0x7f79ec0afa20 2023-10-18T11:41:26.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f1c34640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0cbd20 0x7f79ec14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f1c34640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0cbd20 0x7f79ec14bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40026/0 (socket says 172.21.15.125:40026) 2023-10-18T11:41:26.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.062+0000 7f79f1c34640 1 -- 172.21.15.125:0/3638192418 learned_addr learned my addr 172.21.15.125:0/3638192418 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:26.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f0c32640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79ec0b0530 0x7f79ec1352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f1433640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79ec0afa20 0x7f79ec142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f1c34640 1 -- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79ec0afa20 msgr2=0x7f79ec142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f1c34640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79ec0afa20 0x7f79ec142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f1c34640 1 -- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79ec0b0530 msgr2=0x7f79ec1352e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f1c34640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79ec0b0530 0x7f79ec1352e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f1c34640 1 -- 172.21.15.125:0/3638192418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e8009560 con 0x7f79ec0cbd20 2023-10-18T11:41:26.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f1c34640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0cbd20 0x7f79ec14bd90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c7a0 tx=0x7f79e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:26.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79e1ffb640 1 -- 172.21.15.125:0/3638192418 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4003b70 con 0x7f79ec0cbd20 2023-10-18T11:41:26.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79e1ffb640 1 -- 172.21.15.125:0/3638192418 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79e4003cf0 con 0x7f79ec0cbd20 2023-10-18T11:41:26.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec14d260 con 0x7f79ec0cbd20 2023-10-18T11:41:26.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79e1ffb640 1 -- 172.21.15.125:0/3638192418 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400aab0 con 0x7f79ec0cbd20 2023-10-18T11:41:26.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.063+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec14d820 con 0x7f79ec0cbd20 2023-10-18T11:41:26.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.064+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec158f50 con 0x7f79ec0cbd20 2023-10-18T11:41:26.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.064+0000 7f79e1ffb640 1 -- 172.21.15.125:0/3638192418 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f79e400ac10 con 0x7f79ec0cbd20 2023-10-18T11:41:26.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.065+0000 7f79e1ffb640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f79d006ac90 0x7f79d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.065+0000 7f79e1ffb640 1 -- 172.21.15.125:0/3638192418 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f79e4089690 con 0x7f79ec0cbd20 2023-10-18T11:41:26.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.065+0000 7f79f1433640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f79d006ac90 0x7f79d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.066+0000 7f79f1433640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f79d006ac90 0x7f79d006d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f79ec15ac10 tx=0x7f79dc009040 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:26.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.069+0000 7f79e1ffb640 1 -- 172.21.15.125:0/3638192418 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79e4054640 con 0x7f79ec0cbd20 2023-10-18T11:41:26.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.235+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79ec0690e0 con 0x7f79ec0cbd20 2023-10-18T11:41:26.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.236+0000 7f79e1ffb640 1 -- 172.21.15.125:0/3638192418 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+17935 (secure 0 0 0) 0x7f79e4057c60 con 0x7f79ec0cbd20 2023-10-18T11:41:26.239 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:26.239 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":162,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:24.415723+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-18T10:44:58.087653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-18T10:44:58.847874+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":161,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2893867972":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/912903828":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/1276835180":"2023-10-19T11:41:24.415677+0000","172.21.15.139:6832/1032201494":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/300871790":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/4164237327":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/555070972":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/562742640":"2023-10-19T11:41:24.415677+0000","172.21.15.139:6833/1032201494":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/1072662486":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6833/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1079295697":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/2400687467":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6832/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1927085609":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1424238360":"2023-10-19T10:45:15.816055+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:26.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f79d006ac90 msgr2=0x7f79d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f79d006ac90 0x7f79d006d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f79ec15ac10 tx=0x7f79dc009040 comp rx=0 tx=0).stop 2023-10-18T11:41:26.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0cbd20 msgr2=0x7f79ec14bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0cbd20 0x7f79ec14bd90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c7a0 tx=0x7f79e400cc60 comp rx=0 tx=0).stop 2023-10-18T11:41:26.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 shutdown_connections 2023-10-18T11:41:26.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f79d006ac90 0x7f79d006d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79ec0b0530 0x7f79ec1352e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79ec0cbd20 0x7f79ec14bd90 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 --2- 172.21.15.125:0/3638192418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79ec0afa20 0x7f79ec142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 >> 172.21.15.125:0/3638192418 conn(0x7f79ec069930 msgr2=0x7f79ec151100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:26.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 shutdown_connections 2023-10-18T11:41:26.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.238+0000 7f79f3697640 1 -- 172.21.15.125:0/3638192418 wait complete. 2023-10-18T11:41:26.252 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T11:41:26.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.354+0000 7f494964f640 1 -- 172.21.15.125:0/2620736459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 msgr2=0x7f49440ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.354+0000 7f494964f640 1 --2- 172.21.15.125:0/2620736459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 0x7f49440ce9a0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f492c005fd0 tx=0x7f492c02be20 comp rx=0 tx=0).stop 2023-10-18T11:41:26.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.354+0000 7f494964f640 1 -- 172.21.15.125:0/2620736459 shutdown_connections 2023-10-18T11:41:26.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.354+0000 7f494964f640 1 --2- 172.21.15.125:0/2620736459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 0x7f49440ce9a0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.354+0000 7f494964f640 1 -- 172.21.15.125:0/2620736459 >> 172.21.15.125:0/2620736459 conn(0x7f49440d0c50 msgr2=0x7f49440d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:26.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.354+0000 7f494964f640 1 -- 172.21.15.125:0/2620736459 shutdown_connections 2023-10-18T11:41:26.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.354+0000 7f494964f640 1 -- 172.21.15.125:0/2620736459 wait complete. 2023-10-18T11:41:26.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.355+0000 7f494964f640 1 Processor -- start 2023-10-18T11:41:26.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.355+0000 7f494964f640 1 -- start start 2023-10-18T11:41:26.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.355+0000 7f494964f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49440b1ca0 0x7f49440cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.355+0000 7f494964f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 0x7f49441591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.355+0000 7f494964f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494415e5e0 0x7f494409ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.355+0000 7f494964f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49441f4e10 con 0x7f49440cf8b0 2023-10-18T11:41:26.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f494964f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944159720 con 0x7f494415e5e0 2023-10-18T11:41:26.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f494964f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440c6b50 con 0x7f49440b1ca0 2023-10-18T11:41:26.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 0x7f49441591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 0x7f49441591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40034/0 (socket says 172.21.15.125:40034) 2023-10-18T11:41:26.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 -- 172.21.15.125:0/2532542378 learned_addr learned my addr 172.21.15.125:0/2532542378 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:26.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f4942ffd640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49440b1ca0 0x7f49440cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f49437fe640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494415e5e0 0x7f494409ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 -- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49440b1ca0 msgr2=0x7f49440cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49440b1ca0 0x7f49440cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 -- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494415e5e0 msgr2=0x7f494409ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494415e5e0 0x7f494409ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 -- 172.21.15.125:0/2532542378 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c009560 con 0x7f49440cf8b0 2023-10-18T11:41:26.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493bfff640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 0x7f49441591e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f492c006010 tx=0x7f492c02ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:26.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493b7fe640 1 -- 172.21.15.125:0/2532542378 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c005650 con 0x7f49440cf8b0 2023-10-18T11:41:26.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493b7fe640 1 -- 172.21.15.125:0/2532542378 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f492c0038f0 con 0x7f49440cf8b0 2023-10-18T11:41:26.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f493b7fe640 1 -- 172.21.15.125:0/2532542378 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c003ca0 con 0x7f49440cf8b0 2023-10-18T11:41:26.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49440c6e30 con 0x7f49440cf8b0 2023-10-18T11:41:26.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.356+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49440c73f0 con 0x7f49440cf8b0 2023-10-18T11:41:26.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.357+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49440c7580 con 0x7f49440cf8b0 2023-10-18T11:41:26.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.358+0000 7f493b7fe640 1 -- 172.21.15.125:0/2532542378 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f492c03e450 con 0x7f49440cf8b0 2023-10-18T11:41:26.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.358+0000 7f493b7fe640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f492406ac90 0x7f492406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.358+0000 7f493b7fe640 1 -- 172.21.15.125:0/2532542378 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f492c0ac350 con 0x7f49440cf8b0 2023-10-18T11:41:26.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.358+0000 7f4942ffd640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f492406ac90 0x7f492406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.359+0000 7f4942ffd640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f492406ac90 0x7f492406d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4934007fd0 tx=0x7f4934009040 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:26.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.362+0000 7f493b7fe640 1 -- 172.21.15.125:0/2532542378 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f492c0774e0 con 0x7f49440cf8b0 2023-10-18T11:41:26.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.568+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f49440690e0 con 0x7f49440cf8b0 2023-10-18T11:41:26.571 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T11:41:26.568+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T11:41:26.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.570+0000 7f493b7fe640 1 -- 172.21.15.125:0/2532542378 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7f492c043090 con 0x7f49440cf8b0 2023-10-18T11:41:26.572 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T11:41:26.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.572+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f492406ac90 msgr2=0x7f492406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.572+0000 7f494964f640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f492406ac90 0x7f492406d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4934007fd0 tx=0x7f4934009040 comp rx=0 tx=0).stop 2023-10-18T11:41:26.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 msgr2=0x7f49441591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 0x7f49441591e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f492c006010 tx=0x7f492c02ddf0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 shutdown_connections 2023-10-18T11:41:26.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f492406ac90 0x7f492406d150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494415e5e0 0x7f494409ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f49440cf8b0 0x7f49441591e0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 --2- 172.21.15.125:0/2532542378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49440b1ca0 0x7f49440cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 >> 172.21.15.125:0/2532542378 conn(0x7f49440d0c50 msgr2=0x7f49440b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:26.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 shutdown_connections 2023-10-18T11:41:26.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.573+0000 7f494964f640 1 -- 172.21.15.125:0/2532542378 wait complete. 2023-10-18T11:41:26.586 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T11:41:26.608 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T11:41:26.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.690+0000 7f6da0490640 1 -- 172.21.15.125:0/956406028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d80003b30 msgr2=0x7f6d80003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.690+0000 7f6da0490640 1 --2- 172.21.15.125:0/956406028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d80003b30 0x7f6d80003f30 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f6d90001800 tx=0x7f6d800057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.690+0000 7f6d9ea2d640 1 -- 172.21.15.125:0/956406028 reap_dead start 2023-10-18T11:41:26.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.690+0000 7f6da0490640 1 -- 172.21.15.125:0/956406028 shutdown_connections 2023-10-18T11:41:26.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.690+0000 7f6da0490640 1 -- 172.21.15.125:0/956406028 >> 172.21.15.125:0/956406028 conn(0x7f6d98069930 msgr2=0x7f6d98069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:26.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.690+0000 7f6da0490640 1 -- 172.21.15.125:0/956406028 shutdown_connections 2023-10-18T11:41:26.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.690+0000 7f6da0490640 1 -- 172.21.15.125:0/956406028 wait complete. 2023-10-18T11:41:26.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6da0490640 1 Processor -- start 2023-10-18T11:41:26.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6da0490640 1 -- start start 2023-10-18T11:41:26.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6da0490640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d80003b30 0x7f6d98159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6da0490640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d98134e70 0x7f6d981352f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6da0490640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d980af7a0 0x7f6d980afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6da0490640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d98135830 con 0x7f6d980af7a0 2023-10-18T11:41:26.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6da0490640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d98153fc0 con 0x7f6d80003b30 2023-10-18T11:41:26.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6da0490640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9815a500 con 0x7f6d98134e70 2023-10-18T11:41:26.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6d9ea2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d980af7a0 0x7f6d980afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6d9ea2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d980af7a0 0x7f6d980afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40076/0 (socket says 172.21.15.125:40076) 2023-10-18T11:41:26.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.692+0000 7f6d9ea2d640 1 -- 172.21.15.125:0/339700485 learned_addr learned my addr 172.21.15.125:0/339700485 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:26.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9e22c640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d80003b30 0x7f6d98159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9ea2d640 1 -- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d98134e70 msgr2=0x7f6d981352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9da2b640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d98134e70 0x7f6d981352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9ea2d640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d98134e70 0x7f6d981352f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9ea2d640 1 -- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d80003b30 msgr2=0x7f6d98159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:26.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9ea2d640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d80003b30 0x7f6d98159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:26.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9ea2d640 1 -- 172.21.15.125:0/339700485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d980cabe0 con 0x7f6d980af7a0 2023-10-18T11:41:26.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9e22c640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d80003b30 0x7f6d98159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:26.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d9ea2d640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d980af7a0 0x7f6d980afc20 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f6d9806d510 tx=0x7f6d8800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:26.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d96ffd640 1 -- 172.21.15.125:0/339700485 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003970 con 0x7f6d980af7a0 2023-10-18T11:41:26.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d96ffd640 1 -- 172.21.15.125:0/339700485 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d88003ad0 con 0x7f6d980af7a0 2023-10-18T11:41:26.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d980caec0 con 0x7f6d980af7a0 2023-10-18T11:41:26.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6d96ffd640 1 -- 172.21.15.125:0/339700485 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88004030 con 0x7f6d980af7a0 2023-10-18T11:41:26.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.693+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d980cb480 con 0x7f6d980af7a0 2023-10-18T11:41:26.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.694+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d98141630 con 0x7f6d980af7a0 2023-10-18T11:41:26.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.694+0000 7f6d96ffd640 1 -- 172.21.15.125:0/339700485 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 91) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6d88004190 con 0x7f6d980af7a0 2023-10-18T11:41:26.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.695+0000 7f6d96ffd640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f6d8406acc0 0x7f6d8406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:26.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.695+0000 7f6d96ffd640 1 -- 172.21.15.125:0/339700485 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6875+0+0 (secure 0 0 0) 0x7f6d88089f40 con 0x7f6d980af7a0 2023-10-18T11:41:26.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.695+0000 7f6d9e22c640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f6d8406acc0 0x7f6d8406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:26.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.695+0000 7f6d9e22c640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f6d8406acc0 0x7f6d8406d180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d90034000 tx=0x7f6d9002e070 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:26.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.699+0000 7f6d96ffd640 1 -- 172.21.15.125:0/339700485 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d88054ea0 con 0x7f6d980af7a0 2023-10-18T11:41:26.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:26.902+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6d980690e0 con 0x7f6d980af7a0 2023-10-18T11:41:27.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.430+0000 7f6d96ffd640 1 -- 172.21.15.125:0/339700485 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 92) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6d88055270 con 0x7f6d980af7a0 2023-10-18T11:41:27.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.571+0000 7f6d96ffd640 1 -- 172.21.15.125:0/339700485 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d880724a0 con 0x7f6d980af7a0 2023-10-18T11:41:27.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f6d8406acc0 msgr2=0x7f6d8406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:27.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f6d8406acc0 0x7f6d8406d180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d90034000 tx=0x7f6d9002e070 comp rx=0 tx=0).stop 2023-10-18T11:41:27.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d980af7a0 msgr2=0x7f6d980afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:27.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d980af7a0 0x7f6d980afc20 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f6d9806d510 tx=0x7f6d8800f3c0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 shutdown_connections 2023-10-18T11:41:27.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f6d8406acc0 0x7f6d8406d180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d980af7a0 0x7f6d980afc20 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d98134e70 0x7f6d981352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 --2- 172.21.15.125:0/339700485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d80003b30 0x7f6d98159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 >> 172.21.15.125:0/339700485 conn(0x7f6d98069930 msgr2=0x7f6d980c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:27.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 shutdown_connections 2023-10-18T11:41:27.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.574+0000 7f6da0490640 1 -- 172.21.15.125:0/339700485 wait complete. 2023-10-18T11:41:27.590 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T11:41:27.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 -- 172.21.15.125:0/3958295559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7798140660 msgr2=0x7f779815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:27.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 --2- 172.21.15.125:0/3958295559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7798140660 0x7f779815a9f0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f77980ca400 tx=0x7f778802bda0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 -- 172.21.15.125:0/3958295559 shutdown_connections 2023-10-18T11:41:27.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 --2- 172.21.15.125:0/3958295559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7798140660 0x7f779815a9f0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 --2- 172.21.15.125:0/3958295559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77980c97c0 0x7f77980c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 --2- 172.21.15.125:0/3958295559 >> v2:172.21.15.139:3300/0 conn(0x7f779806d170 0x7f779813d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 -- 172.21.15.125:0/3958295559 >> 172.21.15.125:0/3958295559 conn(0x7f7798069930 msgr2=0x7f7798069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:27.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 -- 172.21.15.125:0/3958295559 shutdown_connections 2023-10-18T11:41:27.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.741+0000 7f779fb25640 1 -- 172.21.15.125:0/3958295559 wait complete. 2023-10-18T11:41:27.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.742+0000 7f779fb25640 1 Processor -- start 2023-10-18T11:41:27.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.742+0000 7f779fb25640 1 -- start start 2023-10-18T11:41:27.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779fb25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f779806d170 0x7f7798156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:27.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779fb25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77980c97c0 0x7f77981559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:27.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779fb25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7798140660 0x7f77981e6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:27.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779fb25640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798159210 con 0x7f779806d170 2023-10-18T11:41:27.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779fb25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798158970 con 0x7f77980c97c0 2023-10-18T11:41:27.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779fb25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7798158c50 con 0x7f7798140660 2023-10-18T11:41:27.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f779806d170 0x7f7798156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:27.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f779806d170 0x7f7798156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52412/0 (socket says 172.21.15.125:52412) 2023-10-18T11:41:27.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 -- 172.21.15.125:0/2006926954 learned_addr learned my addr 172.21.15.125:0/2006926954 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:27.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d0c0640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77980c97c0 0x7f77981559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:27.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779e0c2640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7798140660 0x7f77981e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:27.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 -- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7798140660 msgr2=0x7f77981e6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:27.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7798140660 0x7f77981e6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 -- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77980c97c0 msgr2=0x7f77981559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:27.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77980c97c0 0x7f77981559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:27.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 -- 172.21.15.125:0/2006926954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788009560 con 0x7f779806d170 2023-10-18T11:41:27.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779e0c2640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7798140660 0x7f77981e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:27.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.743+0000 7f779d8c1640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f779806d170 0x7f7798156150 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f779400cce0 tx=0x7f779400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:27.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.744+0000 7f778e7fc640 1 -- 172.21.15.125:0/2006926954 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794004b20 con 0x7f779806d170 2023-10-18T11:41:27.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.744+0000 7f778e7fc640 1 -- 172.21.15.125:0/2006926954 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7794004d70 con 0x7f779806d170 2023-10-18T11:41:27.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.744+0000 7f778e7fc640 1 -- 172.21.15.125:0/2006926954 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794005280 con 0x7f779806d170 2023-10-18T11:41:27.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.744+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7798158f30 con 0x7f779806d170 2023-10-18T11:41:27.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.744+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7798155ee0 con 0x7f779806d170 2023-10-18T11:41:27.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.745+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7798135d30 con 0x7f779806d170 2023-10-18T11:41:27.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.745+0000 7f778e7fc640 1 -- 172.21.15.125:0/2006926954 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 92) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f77940053e0 con 0x7f779806d170 2023-10-18T11:41:27.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.746+0000 7f778e7fc640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f777c06aed0 0x7f777c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:27.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.746+0000 7f778e7fc640 1 -- 172.21.15.125:0/2006926954 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6875+0+0 (secure 0 0 0) 0x7f779408a3e0 con 0x7f779806d170 2023-10-18T11:41:27.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.746+0000 7f779d0c0640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f777c06aed0 0x7f777c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:27.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.746+0000 7f779d0c0640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f777c06aed0 0x7f777c06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7788035000 tx=0x7f7788004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:27.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.750+0000 7f778e7fc640 1 -- 172.21.15.125:0/2006926954 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7798135d30 con 0x7f779806d170 2023-10-18T11:41:27.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:27.917+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7798135d30 con 0x7f779806d170 2023-10-18T11:41:28.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.775+0000 7f778e7fc640 1 -- 172.21.15.125:0/2006926954 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7798135d30 con 0x7f779806d170 2023-10-18T11:41:28.777 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T11:41:28.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f777c06aed0 msgr2=0x7f777c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:28.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f777c06aed0 0x7f777c06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7788035000 tx=0x7f7788004730 comp rx=0 tx=0).stop 2023-10-18T11:41:28.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f779806d170 msgr2=0x7f7798156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:28.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f779806d170 0x7f7798156150 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f779400cce0 tx=0x7f779400a620 comp rx=0 tx=0).stop 2023-10-18T11:41:28.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 shutdown_connections 2023-10-18T11:41:28.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f777c06aed0 0x7f777c06d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:28.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77980c97c0 0x7f77981559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:28.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7798140660 0x7f77981e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:28.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 --2- 172.21.15.125:0/2006926954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f779806d170 0x7f7798156150 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:28.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 >> 172.21.15.125:0/2006926954 conn(0x7f7798069930 msgr2=0x7f7798069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:28.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 shutdown_connections 2023-10-18T11:41:28.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.777+0000 7f779fb25640 1 -- 172.21.15.125:0/2006926954 wait complete. 2023-10-18T11:41:28.796 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T11:41:28.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.907+0000 7fdfaff39640 1 -- 172.21.15.125:0/1601546185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa81416f0 msgr2=0x7fdfa80b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:28.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.907+0000 7fdfaff39640 1 --2- 172.21.15.125:0/1601546185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa81416f0 0x7fdfa80b1440 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80cea60 tx=0x7fdfa402be40 comp rx=0 tx=0).stop 2023-10-18T11:41:28.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.907+0000 7fdfaff39640 1 -- 172.21.15.125:0/1601546185 shutdown_connections 2023-10-18T11:41:28.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.907+0000 7fdfaff39640 1 --2- 172.21.15.125:0/1601546185 >> v2:172.21.15.139:3300/0 conn(0x7fdfa8070790 0x7fdfa80cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:28.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.907+0000 7fdfaff39640 1 --2- 172.21.15.125:0/1601546185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa81416f0 0x7fdfa80b1440 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:28.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.907+0000 7fdfaff39640 1 -- 172.21.15.125:0/1601546185 >> 172.21.15.125:0/1601546185 conn(0x7fdfa80af730 msgr2=0x7fdfa80afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:28.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.907+0000 7fdfaff39640 1 -- 172.21.15.125:0/1601546185 shutdown_connections 2023-10-18T11:41:28.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.907+0000 7fdfaff39640 1 -- 172.21.15.125:0/1601546185 wait complete. 2023-10-18T11:41:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.908+0000 7fdfaff39640 1 Processor -- start 2023-10-18T11:41:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.908+0000 7fdfaff39640 1 -- start start 2023-10-18T11:41:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfaff39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa8070790 0x7fdfa8152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfaff39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa80b1980 0x7fdfa81e2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfaff39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa81416f0 0x7fdfa815a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfaff39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81531c0 con 0x7fdfa80b1980 2023-10-18T11:41:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfaff39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81e3f70 con 0x7fdfa81416f0 2023-10-18T11:41:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfaff39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81e4250 con 0x7fdfa8070790 2023-10-18T11:41:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa80b1980 0x7fdfa81e2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa80b1980 0x7fdfa81e2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52432/0 (socket says 172.21.15.125:52432) 2023-10-18T11:41:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 -- 172.21.15.125:0/3373907419 learned_addr learned my addr 172.21.15.125:0/3373907419 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfae4d6640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa81416f0 0x7fdfa815a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfadcd5640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa8070790 0x7fdfa8152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 -- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa8070790 msgr2=0x7fdfa8152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:28.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa8070790 0x7fdfa8152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:28.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 -- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa81416f0 msgr2=0x7fdfa815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:28.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa81416f0 0x7fdfa815a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:28.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 -- 172.21.15.125:0/3373907419 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4009560 con 0x7fdfa80b1980 2023-10-18T11:41:28.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfad4d4640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa80b1980 0x7fdfa81e2d30 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800c450 tx=0x7fdf9800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:28.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdf9e7fc640 1 -- 172.21.15.125:0/3373907419 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf980052d0 con 0x7fdfa80b1980 2023-10-18T11:41:28.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdf9e7fc640 1 -- 172.21.15.125:0/3373907419 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf98005430 con 0x7fdfa80b1980 2023-10-18T11:41:28.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa81e4530 con 0x7fdfa80b1980 2023-10-18T11:41:28.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdf9e7fc640 1 -- 172.21.15.125:0/3373907419 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98012890 con 0x7fdfa80b1980 2023-10-18T11:41:28.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.909+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa81e2530 con 0x7fdfa80b1980 2023-10-18T11:41:28.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.910+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa80675f0 con 0x7fdfa80b1980 2023-10-18T11:41:28.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.911+0000 7fdf9e7fc640 1 -- 172.21.15.125:0/3373907419 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdf980035f0 con 0x7fdfa80b1980 2023-10-18T11:41:28.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.911+0000 7fdf9e7fc640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fdf8c06af90 0x7fdf8c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:28.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.911+0000 7fdf9e7fc640 1 -- 172.21.15.125:0/3373907419 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6437+0+0 (secure 0 0 0) 0x7fdf980896f0 con 0x7fdfa80b1980 2023-10-18T11:41:28.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.911+0000 7fdfadcd5640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fdf8c06af90 0x7fdf8c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:28.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.912+0000 7fdfadcd5640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fdf8c06af90 0x7fdf8c06d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4035000 tx=0x7fdfa4004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:28.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:28.915+0000 7fdf9e7fc640 1 -- 172.21.15.125:0/3373907419 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf980543b0 con 0x7fdfa80b1980 2023-10-18T11:41:29.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.081+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdfa80b2150 con 0x7fdfa80b1980 2023-10-18T11:41:29.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.137+0000 7fdf9e7fc640 1 -- 172.21.15.125:0/3373907419 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v165) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf980579d0 con 0x7fdfa80b1980 2023-10-18T11:41:29.140 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T11:41:29.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fdf8c06af90 msgr2=0x7fdf8c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fdf8c06af90 0x7fdf8c06d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4035000 tx=0x7fdfa4004730 comp rx=0 tx=0).stop 2023-10-18T11:41:29.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa80b1980 msgr2=0x7fdfa81e2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa80b1980 0x7fdfa81e2d30 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800c450 tx=0x7fdf9800c910 comp rx=0 tx=0).stop 2023-10-18T11:41:29.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 shutdown_connections 2023-10-18T11:41:29.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fdf8c06af90 0x7fdf8c06d450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa81416f0 0x7fdfa815a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfa80b1980 0x7fdfa81e2d30 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 --2- 172.21.15.125:0/3373907419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa8070790 0x7fdfa8152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 >> 172.21.15.125:0/3373907419 conn(0x7fdfa80af730 msgr2=0x7fdfa813dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:29.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 shutdown_connections 2023-10-18T11:41:29.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.139+0000 7fdfaff39640 1 -- 172.21.15.125:0/3373907419 wait complete. 2023-10-18T11:41:29.153 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T11:41:29.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.254+0000 7f51438c2640 1 -- 172.21.15.125:0/1270091620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 msgr2=0x7f5128004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.254+0000 7f51438c2640 1 --2- 172.21.15.125:0/1270091620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 0x7f5128004c40 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f5138006fd0 tx=0x7f51280057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.254+0000 7f51438c2640 1 -- 172.21.15.125:0/1270091620 shutdown_connections 2023-10-18T11:41:29.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.254+0000 7f51438c2640 1 --2- 172.21.15.125:0/1270091620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 0x7f5128004c40 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.254+0000 7f51438c2640 1 -- 172.21.15.125:0/1270091620 >> 172.21.15.125:0/1270091620 conn(0x7f513c069930 msgr2=0x7f513c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:29.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.254+0000 7f51438c2640 1 -- 172.21.15.125:0/1270091620 shutdown_connections 2023-10-18T11:41:29.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.254+0000 7f51438c2640 1 -- 172.21.15.125:0/1270091620 wait complete. 2023-10-18T11:41:29.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.255+0000 7f51438c2640 1 Processor -- start 2023-10-18T11:41:29.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.255+0000 7f51438c2640 1 -- start start 2023-10-18T11:41:29.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f51438c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 0x7f513c13b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:29.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f51438c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f513c1f53c0 0x7f513c13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:29.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f51438c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f513c13db10 0x7f513c0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:29.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f51438c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c13c120 con 0x7f51280047c0 2023-10-18T11:41:29.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f51438c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1f4fa0 con 0x7f513c13db10 2023-10-18T11:41:29.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f51438c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c13b780 con 0x7f513c1f53c0 2023-10-18T11:41:29.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f514165e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 0x7f513c13b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:29.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f514165e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 0x7f513c13b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52450/0 (socket says 172.21.15.125:52450) 2023-10-18T11:41:29.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f514165e640 1 -- 172.21.15.125:0/968822625 learned_addr learned my addr 172.21.15.125:0/968822625 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:29.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f5140e5d640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f513c1f53c0 0x7f513c13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:29.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f5141e5f640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f513c13db10 0x7f513c0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:29.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.256+0000 7f514165e640 1 -- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f513c1f53c0 msgr2=0x7f513c13d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f514165e640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f513c1f53c0 0x7f513c13d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f514165e640 1 -- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f513c13db10 msgr2=0x7f513c0aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f514165e640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f513c13db10 0x7f513c0aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f514165e640 1 -- 172.21.15.125:0/968822625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c13be40 con 0x7f51280047c0 2023-10-18T11:41:29.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f514165e640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 0x7f513c13b240 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f512c004df0 tx=0x7f512c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:29.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f5131ffb640 1 -- 172.21.15.125:0/968822625 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0105d0 con 0x7f51280047c0 2023-10-18T11:41:29.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f5131ffb640 1 -- 172.21.15.125:0/968822625 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f512c010820 con 0x7f51280047c0 2023-10-18T11:41:29.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f5131ffb640 1 -- 172.21.15.125:0/968822625 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c010c30 con 0x7f51280047c0 2023-10-18T11:41:29.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c1f48e0 con 0x7f51280047c0 2023-10-18T11:41:29.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.257+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c13b9d0 con 0x7f51280047c0 2023-10-18T11:41:29.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.258+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c1f5210 con 0x7f51280047c0 2023-10-18T11:41:29.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.258+0000 7f5131ffb640 1 -- 172.21.15.125:0/968822625 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f512c01b460 con 0x7f51280047c0 2023-10-18T11:41:29.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.259+0000 7f5131ffb640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f512006af60 0x7f512006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:29.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.259+0000 7f5131ffb640 1 -- 172.21.15.125:0/968822625 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7f512c08a440 con 0x7f51280047c0 2023-10-18T11:41:29.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.259+0000 7f5140e5d640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f512006af60 0x7f512006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:29.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.260+0000 7f5140e5d640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f512006af60 0x7f512006d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5138034000 tx=0x7f5138039040 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:29.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.263+0000 7f5131ffb640 1 -- 172.21.15.125:0/968822625 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f512c055190 con 0x7f51280047c0 2023-10-18T11:41:29.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.430+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7f513c1f4740 con 0x7f51280047c0 2023-10-18T11:41:29.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.436+0000 7f5131ffb640 1 -- 172.21.15.125:0/968822625 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v62) v1 ==== 71+0+0 (secure 0 0 0) 0x7f513c1f4740 con 0x7f51280047c0 2023-10-18T11:41:29.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f512006af60 msgr2=0x7f512006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f512006af60 0x7f512006d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5138034000 tx=0x7f5138039040 comp rx=0 tx=0).stop 2023-10-18T11:41:29.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 msgr2=0x7f513c13b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 0x7f513c13b240 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f512c004df0 tx=0x7f512c001630 comp rx=0 tx=0).stop 2023-10-18T11:41:29.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 shutdown_connections 2023-10-18T11:41:29.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f512006af60 0x7f512006d420 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f513c13db10 0x7f513c0aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f513c1f53c0 0x7f513c13d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 --2- 172.21.15.125:0/968822625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51280047c0 0x7f513c13b240 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 >> 172.21.15.125:0/968822625 conn(0x7f513c069930 msgr2=0x7f513c1e2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:29.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 shutdown_connections 2023-10-18T11:41:29.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.438+0000 7f51438c2640 1 -- 172.21.15.125:0/968822625 wait complete. 2023-10-18T11:41:29.451 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-18T11:41:29.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.551+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/307478794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f180ceb40 msgr2=0x7f8f180b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.551+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/307478794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f180ceb40 0x7f8f180b0650 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f8f10005fd0 tx=0x7f8f1002c050 comp rx=0 tx=0).stop 2023-10-18T11:41:29.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.551+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/307478794 shutdown_connections 2023-10-18T11:41:29.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.551+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/307478794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f180ceb40 0x7f8f180b0650 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.551+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/307478794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f180cf340 0x7f8f180d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.551+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/307478794 >> 172.21.15.125:0/307478794 conn(0x7f8f180d0d60 msgr2=0x7f8f180d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:29.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.551+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/307478794 shutdown_connections 2023-10-18T11:41:29.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.551+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/307478794 wait complete. 2023-10-18T11:41:29.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.552+0000 7f8f1e8f7640 1 Processor -- start 2023-10-18T11:41:29.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.552+0000 7f8f1e8f7640 1 -- start start 2023-10-18T11:41:29.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1e8f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f180ceb40 0x7f8f18151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:29.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1e8f7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f180cf340 0x7f8f180c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:29.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1e8f7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1815b2a0 0x7f8f18069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:29.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1e8f7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180caa70 con 0x7f8f1815b2a0 2023-10-18T11:41:29.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1e8f7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180c9df0 con 0x7f8f180ceb40 2023-10-18T11:41:29.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1e8f7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180ca0d0 con 0x7f8f180cf340 2023-10-18T11:41:29.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1815b2a0 0x7f8f18069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:29.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1815b2a0 0x7f8f18069f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52460/0 (socket says 172.21.15.125:52460) 2023-10-18T11:41:29.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 -- 172.21.15.125:0/3596773807 learned_addr learned my addr 172.21.15.125:0/3596773807 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:29.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f17fff640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f180ceb40 0x7f8f18151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:29.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 -- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f180cf340 msgr2=0x7f8f180c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f180cf340 0x7f8f180c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 -- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f180ceb40 msgr2=0x7f8f18151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:29.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f180ceb40 0x7f8f18151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:29.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 -- 172.21.15.125:0/3596773807 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f10009840 con 0x7f8f1815b2a0 2023-10-18T11:41:29.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f17fff640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f180ceb40 0x7f8f18151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:29.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.553+0000 7f8f1ce94640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1815b2a0 0x7f8f18069f90 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f8f10036040 tx=0x7f8f100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:29.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.554+0000 7f8f14ff9640 1 -- 172.21.15.125:0/3596773807 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1003e460 con 0x7f8f1815b2a0 2023-10-18T11:41:29.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.554+0000 7f8f14ff9640 1 -- 172.21.15.125:0/3596773807 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8f1003e6d0 con 0x7f8f1815b2a0 2023-10-18T11:41:29.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.554+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f180ca3b0 con 0x7f8f1815b2a0 2023-10-18T11:41:29.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.554+0000 7f8f14ff9640 1 -- 172.21.15.125:0/3596773807 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1003eae0 con 0x7f8f1815b2a0 2023-10-18T11:41:29.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.554+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f18151a90 con 0x7f8f1815b2a0 2023-10-18T11:41:29.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.555+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f180cabb0 con 0x7f8f1815b2a0 2023-10-18T11:41:29.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.555+0000 7f8f14ff9640 1 -- 172.21.15.125:0/3596773807 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8f1003ec40 con 0x7f8f1815b2a0 2023-10-18T11:41:29.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.555+0000 7f8f14ff9640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f8f0406af90 0x7f8f0406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:29.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.556+0000 7f8f14ff9640 1 -- 172.21.15.125:0/3596773807 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7f8f100ac8d0 con 0x7f8f1815b2a0 2023-10-18T11:41:29.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.556+0000 7f8f17fff640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f8f0406af90 0x7f8f0406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:29.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.556+0000 7f8f17fff640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f8f0406af90 0x7f8f0406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009b40 tx=0x7f8f08005e50 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:29.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.560+0000 7f8f14ff9640 1 -- 172.21.15.125:0/3596773807 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f180cabb0 con 0x7f8f1815b2a0 2023-10-18T11:41:29.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:29.800+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f8f180cabb0 con 0x7f8f1815b2a0 2023-10-18T11:41:30.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.785+0000 7f8f14ff9640 1 -- 172.21.15.125:0/3596773807 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2479) v1 ==== 178+0+0 (secure 0 0 0) 0x7f8f180cabb0 con 0x7f8f1815b2a0 2023-10-18T11:41:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f8f0406af90 msgr2=0x7f8f0406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f8f0406af90 0x7f8f0406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009b40 tx=0x7f8f08005e50 comp rx=0 tx=0).stop 2023-10-18T11:41:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1815b2a0 msgr2=0x7f8f18069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:30.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1815b2a0 0x7f8f18069f90 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f8f10036040 tx=0x7f8f100042f0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 shutdown_connections 2023-10-18T11:41:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f8f0406af90 0x7f8f0406d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f1815b2a0 0x7f8f18069f90 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f180cf340 0x7f8f180c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 --2- 172.21.15.125:0/3596773807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f180ceb40 0x7f8f18151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 >> 172.21.15.125:0/3596773807 conn(0x7f8f180d0d60 msgr2=0x7f8f180ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:30.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.787+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 shutdown_connections 2023-10-18T11:41:30.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.788+0000 7f8f1e8f7640 1 -- 172.21.15.125:0/3596773807 wait complete. 2023-10-18T11:41:30.803 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-10-18T11:41:30.803 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-10-18T11:41:30.804 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T11:41:30.804 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:30.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.914+0000 7fd152339640 1 -- 172.21.15.125:0/2105286885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c0c9750 msgr2=0x7fd14c0c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:30.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.914+0000 7fd152339640 1 --2- 172.21.15.125:0/2105286885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c0c9750 0x7fd14c0c9b70 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0cbd30 tx=0x7fd14402c080 comp rx=0 tx=0).stop 2023-10-18T11:41:30.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.914+0000 7fd152339640 1 -- 172.21.15.125:0/2105286885 shutdown_connections 2023-10-18T11:41:30.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.914+0000 7fd152339640 1 --2- 172.21.15.125:0/2105286885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c0c9750 0x7fd14c0c9b70 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.914+0000 7fd152339640 1 --2- 172.21.15.125:0/2105286885 >> v2:172.21.15.139:3300/0 conn(0x7fd14c06b030 0x7fd14c06b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.914+0000 7fd152339640 1 -- 172.21.15.125:0/2105286885 >> 172.21.15.125:0/2105286885 conn(0x7fd14c069930 msgr2=0x7fd14c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:30.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.914+0000 7fd152339640 1 -- 172.21.15.125:0/2105286885 shutdown_connections 2023-10-18T11:41:30.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.914+0000 7fd152339640 1 -- 172.21.15.125:0/2105286885 wait complete. 2023-10-18T11:41:30.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.915+0000 7fd152339640 1 Processor -- start 2023-10-18T11:41:30.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.915+0000 7fd152339640 1 -- start start 2023-10-18T11:41:30.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd152339640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c06b030 0x7fd14c152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:30.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd152339640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd14c0c9750 0x7fd14c1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:30.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd152339640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd14c159bd0 0x7fd14c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:30.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd152339640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c1499e0 con 0x7fd14c06b030 2023-10-18T11:41:30.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd152339640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c14f950 con 0x7fd14c0c9750 2023-10-18T11:41:30.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd152339640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c14fc30 con 0x7fd14c159bd0 2023-10-18T11:41:30.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c06b030 0x7fd14c152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:30.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c06b030 0x7fd14c152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52466/0 (socket says 172.21.15.125:52466) 2023-10-18T11:41:30.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 -- 172.21.15.125:0/523874978 learned_addr learned my addr 172.21.15.125:0/523874978 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:30.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14b7fe640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd14c0c9750 0x7fd14c1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:30.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 -- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd14c159bd0 msgr2=0x7fd14c151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:30.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd14c159bd0 0x7fd14c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 -- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd14c0c9750 msgr2=0x7fd14c1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:30.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd14c0c9750 0x7fd14c1511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:30.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 -- 172.21.15.125:0/523874978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144009840 con 0x7fd14c06b030 2023-10-18T11:41:30.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14b7fe640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd14c0c9750 0x7fd14c1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:30.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd14bfff640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c06b030 0x7fd14c152520 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00c920 tx=0x7fd13c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:30.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd148ff9640 1 -- 172.21.15.125:0/523874978 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c007c70 con 0x7fd14c06b030 2023-10-18T11:41:30.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd148ff9640 1 -- 172.21.15.125:0/523874978 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd13c00f450 con 0x7fd14c06b030 2023-10-18T11:41:30.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd148ff9640 1 -- 172.21.15.125:0/523874978 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00f860 con 0x7fd14c06b030 2023-10-18T11:41:30.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.916+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c149c30 con 0x7fd14c06b030 2023-10-18T11:41:30.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.917+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c152a60 con 0x7fd14c06b030 2023-10-18T11:41:30.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.917+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c14a150 con 0x7fd14c06b030 2023-10-18T11:41:30.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.918+0000 7fd148ff9640 1 -- 172.21.15.125:0/523874978 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd13c00f9c0 con 0x7fd14c06b030 2023-10-18T11:41:30.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.918+0000 7fd148ff9640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd13806af60 0x7fd13806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:30.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.918+0000 7fd148ff9640 1 -- 172.21.15.125:0/523874978 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7fd13c08a270 con 0x7fd14c06b030 2023-10-18T11:41:30.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.918+0000 7fd14b7fe640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd13806af60 0x7fd13806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:30.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.919+0000 7fd14b7fe640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd13806af60 0x7fd13806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd14c1f2620 tx=0x7fd140009040 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:30.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:30.923+0000 7fd148ff9640 1 -- 172.21.15.125:0/523874978 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd14c14a150 con 0x7fd14c06b030 2023-10-18T11:41:31.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.089+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd14c14a150 con 0x7fd14c06b030 2023-10-18T11:41:31.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.090+0000 7fd148ff9640 1 -- 172.21.15.125:0/523874978 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+14379 (secure 0 0 0) 0x7fd14c14a150 con 0x7fd14c06b030 2023-10-18T11:41:31.093 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:31.093 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":165,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:29.137309+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6835/631315131":"2023-10-19T11:41:26.569625+0000","172.21.15.125:6833/631315131":"2023-10-19T11:41:26.569625+0000","172.21.15.139:0/2893867972":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/912903828":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/1276835180":"2023-10-19T11:41:24.415677+0000","172.21.15.139:6832/1032201494":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/300871790":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/4164237327":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/555070972":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/562742640":"2023-10-19T11:41:24.415677+0000","172.21.15.139:6833/1032201494":"2023-10-19T11:41:24.415677+0000","172.21.15.139:0/1072662486":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6833/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1079295697":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/2400687467":"2023-10-19T10:45:15.816055+0000","172.21.15.139:6832/3182280683":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1927085609":"2023-10-19T10:45:15.816055+0000","172.21.15.139:0/1424238360":"2023-10-19T10:45:15.816055+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":20,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:31.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd13806af60 msgr2=0x7fd13806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd13806af60 0x7fd13806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd14c1f2620 tx=0x7fd140009040 comp rx=0 tx=0).stop 2023-10-18T11:41:31.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c06b030 msgr2=0x7fd14c152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c06b030 0x7fd14c152520 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00c920 tx=0x7fd13c00cde0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 shutdown_connections 2023-10-18T11:41:31.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd13806af60 0x7fd13806d420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd14c0c9750 0x7fd14c1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd14c159bd0 0x7fd14c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 --2- 172.21.15.125:0/523874978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd14c06b030 0x7fd14c152520 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 >> 172.21.15.125:0/523874978 conn(0x7fd14c069930 msgr2=0x7fd14c13b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:31.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 shutdown_connections 2023-10-18T11:41:31.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.092+0000 7fd152339640 1 -- 172.21.15.125:0/523874978 wait complete. 2023-10-18T11:41:31.105 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:41:31.106 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:41:31.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.208+0000 7f3982810640 1 -- 172.21.15.125:0/1613510960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c141690 msgr2=0x7f397c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.208+0000 7f3982810640 1 --2- 172.21.15.125:0/1613510960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c141690 0x7f397c0703e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f397c0cbd00 tx=0x7f397002bdb0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.208+0000 7f3982810640 1 -- 172.21.15.125:0/1613510960 shutdown_connections 2023-10-18T11:41:31.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.208+0000 7f3982810640 1 --2- 172.21.15.125:0/1613510960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c141690 0x7f397c0703e0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.208+0000 7f3982810640 1 --2- 172.21.15.125:0/1613510960 >> v2:172.21.15.139:3300/0 conn(0x7f397c13e520 0x7f397c06bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.208+0000 7f3982810640 1 -- 172.21.15.125:0/1613510960 >> 172.21.15.125:0/1613510960 conn(0x7f397c069930 msgr2=0x7f397c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:31.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.208+0000 7f3982810640 1 -- 172.21.15.125:0/1613510960 shutdown_connections 2023-10-18T11:41:31.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.208+0000 7f3982810640 1 -- 172.21.15.125:0/1613510960 wait complete. 2023-10-18T11:41:31.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.209+0000 7f3982810640 1 Processor -- start 2023-10-18T11:41:31.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f3982810640 1 -- start start 2023-10-18T11:41:31.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f3982810640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c06c0a0 0x7f397c1db600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f3982810640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397c13e520 0x7f397c13b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f3982810640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c141690 0x7f397c0ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f3982810640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1dbb40 con 0x7f397c06c0a0 2023-10-18T11:41:31.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f3982810640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1d9a10 con 0x7f397c141690 2023-10-18T11:41:31.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f3982810640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1d8e00 con 0x7f397c13e520 2023-10-18T11:41:31.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f397bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c06c0a0 0x7f397c1db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f397bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c06c0a0 0x7f397c1db600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52484/0 (socket says 172.21.15.125:52484) 2023-10-18T11:41:31.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.210+0000 7f397bfff640 1 -- 172.21.15.125:0/3139486968 learned_addr learned my addr 172.21.15.125:0/3139486968 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:31.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f3980dad640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c141690 0x7f397c0ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f397b7fe640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397c13e520 0x7f397c13b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f397bfff640 1 -- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397c13e520 msgr2=0x7f397c13b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f397bfff640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397c13e520 0x7f397c13b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f397bfff640 1 -- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c141690 msgr2=0x7f397c0ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f397bfff640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c141690 0x7f397c0ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f397bfff640 1 -- 172.21.15.125:0/3139486968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3970009560 con 0x7f397c06c0a0 2023-10-18T11:41:31.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f3980dad640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c141690 0x7f397c0ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:31.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f397b7fe640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397c13e520 0x7f397c13b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:31.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f397bfff640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c06c0a0 0x7f397c1db600 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f396c00d950 tx=0x7f396c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:31.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f3978ff9640 1 -- 172.21.15.125:0/3139486968 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c009c30 con 0x7f397c06c0a0 2023-10-18T11:41:31.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f3978ff9640 1 -- 172.21.15.125:0/3139486968 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f396c009e80 con 0x7f397c06c0a0 2023-10-18T11:41:31.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c1d90e0 con 0x7f397c06c0a0 2023-10-18T11:41:31.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f3978ff9640 1 -- 172.21.15.125:0/3139486968 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0106c0 con 0x7f397c06c0a0 2023-10-18T11:41:31.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.211+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c1d96a0 con 0x7f397c06c0a0 2023-10-18T11:41:31.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.212+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c0690e0 con 0x7f397c06c0a0 2023-10-18T11:41:31.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.212+0000 7f3978ff9640 1 -- 172.21.15.125:0/3139486968 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f396c010820 con 0x7f397c06c0a0 2023-10-18T11:41:31.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.213+0000 7f3978ff9640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396806af60 0x7f396806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.213+0000 7f3978ff9640 1 -- 172.21.15.125:0/3139486968 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7f396c089ed0 con 0x7f397c06c0a0 2023-10-18T11:41:31.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.213+0000 7f397b7fe640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396806af60 0x7f396806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.213+0000 7f397b7fe640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396806af60 0x7f396806d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3970035000 tx=0x7f3970004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:31.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.217+0000 7f3978ff9640 1 -- 172.21.15.125:0/3139486968 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f396c054c20 con 0x7f397c06c0a0 2023-10-18T11:41:31.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.381+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f397c069db0 con 0x7f397c06c0a0 2023-10-18T11:41:31.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.382+0000 7f3978ff9640 1 -- 172.21.15.125:0/3139486968 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7f396c0170a0 con 0x7f397c06c0a0 2023-10-18T11:41:31.386 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:41:31.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396806af60 msgr2=0x7f396806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396806af60 0x7f396806d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3970035000 tx=0x7f3970004730 comp rx=0 tx=0).stop 2023-10-18T11:41:31.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c06c0a0 msgr2=0x7f397c1db600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c06c0a0 0x7f397c1db600 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f396c00d950 tx=0x7f396c00de10 comp rx=0 tx=0).stop 2023-10-18T11:41:31.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 shutdown_connections 2023-10-18T11:41:31.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396806af60 0x7f396806d420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c141690 0x7f397c0ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397c13e520 0x7f397c13b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 --2- 172.21.15.125:0/3139486968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c06c0a0 0x7f397c1db600 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 >> 172.21.15.125:0/3139486968 conn(0x7f397c069930 msgr2=0x7f397c0c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:31.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 shutdown_connections 2023-10-18T11:41:31.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.383+0000 7f3982810640 1 -- 172.21.15.125:0/3139486968 wait complete. 2023-10-18T11:41:31.398 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-18T11:41:31.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.498+0000 7f087ecb4640 1 -- 172.21.15.125:0/506314999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 msgr2=0x7f086400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.498+0000 7f087ecb4640 1 --2- 172.21.15.125:0/506314999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 0x7f086400b980 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f0868001680 tx=0x7f08640057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.498+0000 7f087ecb4640 1 -- 172.21.15.125:0/506314999 shutdown_connections 2023-10-18T11:41:31.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.498+0000 7f087ecb4640 1 --2- 172.21.15.125:0/506314999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 0x7f086400b980 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.498+0000 7f087ecb4640 1 -- 172.21.15.125:0/506314999 >> 172.21.15.125:0/506314999 conn(0x7f08780d0d60 msgr2=0x7f08780d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:31.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.498+0000 7f087ecb4640 1 -- 172.21.15.125:0/506314999 shutdown_connections 2023-10-18T11:41:31.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.498+0000 7f087ecb4640 1 -- 172.21.15.125:0/506314999 wait complete. 2023-10-18T11:41:31.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.499+0000 7f087ecb4640 1 Processor -- start 2023-10-18T11:41:31.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.499+0000 7f087ecb4640 1 -- start start 2023-10-18T11:41:31.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ecb4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 0x7f08781f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ecb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08780b5a70 0x7f08780b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ecb4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08780c92e0 0x7f08780c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ecb4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781f2500 con 0x7f0864005180 2023-10-18T11:41:31.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ecb4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780b6430 con 0x7f08780b5a70 2023-10-18T11:41:31.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ecb4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08780c9ca0 con 0x7f08780c92e0 2023-10-18T11:41:31.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 0x7f08781f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 0x7f08781f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52514/0 (socket says 172.21.15.125:52514) 2023-10-18T11:41:31.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 -- 172.21.15.125:0/755057699 learned_addr learned my addr 172.21.15.125:0/755057699 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:31.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f0877fff640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08780b5a70 0x7f08780b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087d251640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08780c92e0 0x7f08780c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 -- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08780c92e0 msgr2=0x7f08780c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08780c92e0 0x7f08780c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 -- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08780b5a70 msgr2=0x7f08780b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08780b5a70 0x7f08780b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 -- 172.21.15.125:0/755057699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08781f1880 con 0x7f0864005180 2023-10-18T11:41:31.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ca50640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 0x7f08781f1200 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f0870004fd0 tx=0x7f087000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:31.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f08757fa640 1 -- 172.21.15.125:0/755057699 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000cbf0 con 0x7f0864005180 2023-10-18T11:41:31.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f08757fa640 1 -- 172.21.15.125:0/755057699 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f087000ce40 con 0x7f0864005180 2023-10-18T11:41:31.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08781f1b60 con 0x7f0864005180 2023-10-18T11:41:31.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f08757fa640 1 -- 172.21.15.125:0/755057699 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700116c0 con 0x7f0864005180 2023-10-18T11:41:31.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.500+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08781f2120 con 0x7f0864005180 2023-10-18T11:41:31.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.501+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08780690e0 con 0x7f0864005180 2023-10-18T11:41:31.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.502+0000 7f08757fa640 1 -- 172.21.15.125:0/755057699 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0870011820 con 0x7f0864005180 2023-10-18T11:41:31.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.502+0000 7f08757fa640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f085806af90 0x7f085806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.502+0000 7f08757fa640 1 -- 172.21.15.125:0/755057699 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7f087008a3e0 con 0x7f0864005180 2023-10-18T11:41:31.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.503+0000 7f0877fff640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f085806af90 0x7f085806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.503+0000 7f0877fff640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f085806af90 0x7f085806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f08780b6700 tx=0x7f0860005f90 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:31.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.508+0000 7f08757fa640 1 -- 172.21.15.125:0/755057699 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0870058d10 con 0x7f0864005180 2023-10-18T11:41:31.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.736+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f087809ea40 con 0x7f0864005180 2023-10-18T11:41:31.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.785+0000 7f08757fa640 1 -- 172.21.15.125:0/755057699 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2480) v1 ==== 177+0+0 (secure 0 0 0) 0x7f08700588a0 con 0x7f0864005180 2023-10-18T11:41:31.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f085806af90 msgr2=0x7f085806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f085806af90 0x7f085806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f08780b6700 tx=0x7f0860005f90 comp rx=0 tx=0).stop 2023-10-18T11:41:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 msgr2=0x7f08781f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 0x7f08781f1200 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f0870004fd0 tx=0x7f087000ed50 comp rx=0 tx=0).stop 2023-10-18T11:41:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 shutdown_connections 2023-10-18T11:41:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f085806af90 0x7f085806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08780b5a70 0x7f08780b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08780c92e0 0x7f08780c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 --2- 172.21.15.125:0/755057699 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0864005180 0x7f08781f1200 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 >> 172.21.15.125:0/755057699 conn(0x7f08780d0d60 msgr2=0x7f08781595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:31.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 shutdown_connections 2023-10-18T11:41:31.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.787+0000 7f087ecb4640 1 -- 172.21.15.125:0/755057699 wait complete. 2023-10-18T11:41:31.801 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T11:41:31.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.899+0000 7f5c8f47c640 1 -- 172.21.15.125:0/1943472940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880cbd20 msgr2=0x7f5c88158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.899+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/1943472940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880cbd20 0x7f5c88158300 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005fd0 tx=0x7f5c8402d1c0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.899+0000 7f5c8f47c640 1 -- 172.21.15.125:0/1943472940 shutdown_connections 2023-10-18T11:41:31.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.899+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/1943472940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c880afa20 0x7f5c880705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.899+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/1943472940 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880cbd20 0x7f5c88158300 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.899+0000 7f5c8f47c640 1 -- 172.21.15.125:0/1943472940 >> 172.21.15.125:0/1943472940 conn(0x7f5c88069930 msgr2=0x7f5c88069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:31.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.899+0000 7f5c8f47c640 1 -- 172.21.15.125:0/1943472940 shutdown_connections 2023-10-18T11:41:31.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.899+0000 7f5c8f47c640 1 -- 172.21.15.125:0/1943472940 wait complete. 2023-10-18T11:41:31.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.900+0000 7f5c8f47c640 1 Processor -- start 2023-10-18T11:41:31.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.900+0000 7f5c8f47c640 1 -- start start 2023-10-18T11:41:31.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8f47c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c880afa20 0x7f5c8814f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8f47c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880b0530 0x7f5c881487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8f47c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c880cbd20 0x7f5c880ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8f47c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881f5360 con 0x7f5c880b0530 2023-10-18T11:41:31.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8f47c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881f5640 con 0x7f5c880afa20 2023-10-18T11:41:31.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8f47c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815a340 con 0x7f5c880cbd20 2023-10-18T11:41:31.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8ca17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880b0530 0x7f5c881487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8ca17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880b0530 0x7f5c881487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52522/0 (socket says 172.21.15.125:52522) 2023-10-18T11:41:31.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8ca17640 1 -- 172.21.15.125:0/3065648818 learned_addr learned my addr 172.21.15.125:0/3065648818 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:31.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8ca17640 1 -- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c880cbd20 msgr2=0x7f5c880ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8ca17640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c880cbd20 0x7f5c880ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8ca17640 1 -- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c880afa20 msgr2=0x7f5c8814f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8d218640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c880afa20 0x7f5c8814f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8ca17640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c880afa20 0x7f5c8814f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:31.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.901+0000 7f5c8ca17640 1 -- 172.21.15.125:0/3065648818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8400aa10 con 0x7f5c880b0530 2023-10-18T11:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.902+0000 7f5c8ca17640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880b0530 0x7f5c881487f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c920 tx=0x7f5c7800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.902+0000 7f5c7dffb640 1 -- 172.21.15.125:0/3065648818 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78007c70 con 0x7f5c880b0530 2023-10-18T11:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.902+0000 7f5c7dffb640 1 -- 172.21.15.125:0/3065648818 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c7800f450 con 0x7f5c880b0530 2023-10-18T11:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.902+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8815a620 con 0x7f5c880b0530 2023-10-18T11:41:31.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.902+0000 7f5c7dffb640 1 -- 172.21.15.125:0/3065648818 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800f860 con 0x7f5c880b0530 2023-10-18T11:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.902+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c88148d30 con 0x7f5c880b0530 2023-10-18T11:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.903+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8815a1a0 con 0x7f5c880b0530 2023-10-18T11:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.903+0000 7f5c7dffb640 1 -- 172.21.15.125:0/3065648818 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c7800f9c0 con 0x7f5c880b0530 2023-10-18T11:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.904+0000 7f5c7dffb640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5c6c06af60 0x7f5c6c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:31.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.904+0000 7f5c7dffb640 1 -- 172.21.15.125:0/3065648818 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7f5c7808a270 con 0x7f5c880b0530 2023-10-18T11:41:31.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.904+0000 7f5c8d218640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5c6c06af60 0x7f5c6c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:31.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.904+0000 7f5c8d218640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5c6c06af60 0x7f5c6c06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5c84037000 tx=0x7f5c84004ab0 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:31.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:31.908+0000 7f5c7dffb640 1 -- 172.21.15.125:0/3065648818 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c8815a1a0 con 0x7f5c880b0530 2023-10-18T11:41:32.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.076+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f5c8815a1a0 con 0x7f5c880b0530 2023-10-18T11:41:32.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.077+0000 7f5c7dffb640 1 -- 172.21.15.125:0/3065648818 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v62) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f5c7801d8c0 con 0x7f5c880b0530 2023-10-18T11:41:32.080 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T11:41:32.081 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T11:41:32.081 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T11:41:32.081 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T11:41:32.081 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T11:41:32.081 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T11:41:32.082 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T11:41:32.082 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T11:41:32.082 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T11:41:32.082 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T11:41:32.082 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T11:41:32.083 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T11:41:32.083 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T11:41:32.083 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T11:41:32.083 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T11:41:32.083 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T11:41:32.084 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T11:41:32.084 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T11:41:32.084 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T11:41:32.084 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T11:41:32.084 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T11:41:32.085 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T11:41:32.085 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T11:41:32.085 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T11:41:32.085 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T11:41:32.085 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T11:41:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T11:41:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T11:41:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T11:41:32.086 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T11:41:32.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5c6c06af60 msgr2=0x7f5c6c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5c6c06af60 0x7f5c6c06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5c84037000 tx=0x7f5c84004ab0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880b0530 msgr2=0x7f5c881487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880b0530 0x7f5c881487f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c920 tx=0x7f5c7800cde0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 shutdown_connections 2023-10-18T11:41:32.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5c6c06af60 0x7f5c6c06d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c880cbd20 0x7f5c880ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c880b0530 0x7f5c881487f0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 --2- 172.21.15.125:0/3065648818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c880afa20 0x7f5c8814f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 >> 172.21.15.125:0/3065648818 conn(0x7f5c88069930 msgr2=0x7f5c880c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:32.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 shutdown_connections 2023-10-18T11:41:32.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.079+0000 7f5c8f47c640 1 -- 172.21.15.125:0/3065648818 wait complete. 2023-10-18T11:41:32.092 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T11:41:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.194+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/2215159152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 msgr2=0x7fd6c00cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.194+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/2215159152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 0x7fd6c00cf720 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4005fd0 tx=0x7fd6b402bd90 comp rx=0 tx=0).stop 2023-10-18T11:41:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.194+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/2215159152 shutdown_connections 2023-10-18T11:41:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.194+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/2215159152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 0x7fd6c00cf720 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.194+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/2215159152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c00cf340 0x7fd6c00d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.194+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/2215159152 >> 172.21.15.125:0/2215159152 conn(0x7fd6c00d0d60 msgr2=0x7fd6c00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:32.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.194+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/2215159152 shutdown_connections 2023-10-18T11:41:32.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.194+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/2215159152 wait complete. 2023-10-18T11:41:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.195+0000 7fd6c6c4b640 1 Processor -- start 2023-10-18T11:41:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.195+0000 7fd6c6c4b640 1 -- start start 2023-10-18T11:41:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c6c4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c00ceb40 0x7fd6c00c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c6c4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c00cf340 0x7fd6c00b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c6c4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 0x7fd6c00c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c6c4b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00b3b20 con 0x7fd6c015b2a0 2023-10-18T11:41:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c6c4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00b3e00 con 0x7fd6c00cf340 2023-10-18T11:41:32.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c6c4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00b40e0 con 0x7fd6c00ceb40 2023-10-18T11:41:32.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c51e8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 0x7fd6c00c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:32.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c51e8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 0x7fd6c00c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52532/0 (socket says 172.21.15.125:52532) 2023-10-18T11:41:32.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.196+0000 7fd6c51e8640 1 -- 172.21.15.125:0/3246245969 learned_addr learned my addr 172.21.15.125:0/3246245969 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:32.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c49e7640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c00ceb40 0x7fd6c00c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:32.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c51e8640 1 -- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c00ceb40 msgr2=0x7fd6c00c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c51e8640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c00ceb40 0x7fd6c00c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c51e8640 1 -- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c00cf340 msgr2=0x7fd6c00b4bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T11:41:32.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c51e8640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c00cf340 0x7fd6c00b4bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c51e8640 1 -- 172.21.15.125:0/3246245969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4009560 con 0x7fd6c015b2a0 2023-10-18T11:41:32.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c51e8640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 0x7fd6c00c9630 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c7a0 tx=0x7fd6b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:32.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6bd7fa640 1 -- 172.21.15.125:0/3246245969 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8003b70 con 0x7fd6c015b2a0 2023-10-18T11:41:32.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6bd7fa640 1 -- 172.21.15.125:0/3246245969 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd6b8003cf0 con 0x7fd6c015b2a0 2023-10-18T11:41:32.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c00b43c0 con 0x7fd6c015b2a0 2023-10-18T11:41:32.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6bd7fa640 1 -- 172.21.15.125:0/3246245969 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800aab0 con 0x7fd6c015b2a0 2023-10-18T11:41:32.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.197+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c00c8d80 con 0x7fd6c015b2a0 2023-10-18T11:41:32.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.198+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c00b48e0 con 0x7fd6c015b2a0 2023-10-18T11:41:32.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.198+0000 7fd6bd7fa640 1 -- 172.21.15.125:0/3246245969 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd6b800ac10 con 0x7fd6c015b2a0 2023-10-18T11:41:32.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.199+0000 7fd6bd7fa640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd6ac06af60 0x7fd6ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:32.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.199+0000 7fd6bd7fa640 1 -- 172.21.15.125:0/3246245969 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7fd6b8089820 con 0x7fd6c015b2a0 2023-10-18T11:41:32.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.200+0000 7fd6c49e7640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd6ac06af60 0x7fd6ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:32.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.201+0000 7fd6c49e7640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd6ac06af60 0x7fd6ac06d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009bd0 tx=0x7fd6b0005f90 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:32.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.203+0000 7fd6bd7fa640 1 -- 172.21.15.125:0/3246245969 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6c00b48e0 con 0x7fd6c015b2a0 2023-10-18T11:41:32.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.370+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd6c00b48e0 con 0x7fd6c015b2a0 2023-10-18T11:41:32.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.373+0000 7fd6bd7fa640 1 -- 172.21.15.125:0/3246245969 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v63)=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd6c00b48e0 con 0x7fd6c015b2a0 2023-10-18T11:41:32.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd6ac06af60 msgr2=0x7fd6ac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd6ac06af60 0x7fd6ac06d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009bd0 tx=0x7fd6b0005f90 comp rx=0 tx=0).stop 2023-10-18T11:41:32.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 msgr2=0x7fd6c00c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 0x7fd6c00c9630 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c7a0 tx=0x7fd6b800cc60 comp rx=0 tx=0).stop 2023-10-18T11:41:32.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 shutdown_connections 2023-10-18T11:41:32.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd6ac06af60 0x7fd6ac06d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c00cf340 0x7fd6c00b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6c015b2a0 0x7fd6c00c9630 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 --2- 172.21.15.125:0/3246245969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c00ceb40 0x7fd6c00c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 >> 172.21.15.125:0/3246245969 conn(0x7fd6c00d0d60 msgr2=0x7fd6c00cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:32.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 shutdown_connections 2023-10-18T11:41:32.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.375+0000 7fd6c6c4b640 1 -- 172.21.15.125:0/3246245969 wait complete. 2023-10-18T11:41:32.389 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T11:41:32.390 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:41:32.431 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:41:32.478 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T11:41:32.478 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:41:32.479 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:32.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:41:32.495 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:41:32.495 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:32.512 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:41:32.512 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:41:32.513 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:32.550 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:41:32.550 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:41:32.551 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:32.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:41:32.568 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T11:41:32.569 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:41:32.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.691+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/95835305 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 msgr2=0x7fe7c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.691+0000 7fe7dd7a0640 1 --2- 172.21.15.125:0/95835305 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 0x7fe7c0003d90 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c260 tx=0x7fe7c00057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.691+0000 7fe7dcf9f640 1 -- 172.21.15.125:0/95835305 reap_dead start 2023-10-18T11:41:32.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.691+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/95835305 shutdown_connections 2023-10-18T11:41:32.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.691+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/95835305 >> 172.21.15.125:0/95835305 conn(0x7fe7d80d0c20 msgr2=0x7fe7d80d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:32.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.691+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/95835305 shutdown_connections 2023-10-18T11:41:32.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.691+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/95835305 wait complete. 2023-10-18T11:41:32.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.692+0000 7fe7dd7a0640 1 Processor -- start 2023-10-18T11:41:32.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.692+0000 7fe7dd7a0640 1 -- start start 2023-10-18T11:41:32.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7dd7a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 0x7fe7d8153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:32.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7dd7a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d8157b00 0x7fe7d8146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:32.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7dd7a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d81f1610 0x7fe7d81f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:32.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7dd7a0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d8154790 con 0x7fe7c0003970 2023-10-18T11:41:32.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7dd7a0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80c96e0 con 0x7fe7d81f1610 2023-10-18T11:41:32.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7dd7a0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d8157f80 con 0x7fe7d8157b00 2023-10-18T11:41:32.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 0x7fe7d8153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:32.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 0x7fe7d8153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52558/0 (socket says 172.21.15.125:52558) 2023-10-18T11:41:32.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 -- 172.21.15.125:0/2090869 learned_addr learned my addr 172.21.15.125:0/2090869 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:32.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7dcf9f640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d81f1610 0x7fe7d81f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:32.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d77fe640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d8157b00 0x7fe7d8146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:32.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 -- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d8157b00 msgr2=0x7fe7d8146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d8157b00 0x7fe7d8146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 -- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d81f1610 msgr2=0x7fe7d81f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d81f1610 0x7fe7d81f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 -- 172.21.15.125:0/2090869 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8154bf0 con 0x7fe7c0003970 2023-10-18T11:41:32.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d7fff640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 0x7fe7d8153fd0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc000c00 tx=0x7fe7cc0015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:32.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.693+0000 7fe7d4ff9640 1 -- 172.21.15.125:0/2090869 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc006310 con 0x7fe7c0003970 2023-10-18T11:41:32.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.694+0000 7fe7d4ff9640 1 -- 172.21.15.125:0/2090869 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe7cc006470 con 0x7fe7c0003970 2023-10-18T11:41:32.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.694+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d8146550 con 0x7fe7c0003970 2023-10-18T11:41:32.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.694+0000 7fe7d4ff9640 1 -- 172.21.15.125:0/2090869 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc003850 con 0x7fe7c0003970 2023-10-18T11:41:32.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.694+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d80c9920 con 0x7fe7c0003970 2023-10-18T11:41:32.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.695+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d8066960 con 0x7fe7c0003970 2023-10-18T11:41:32.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.695+0000 7fe7d4ff9640 1 -- 172.21.15.125:0/2090869 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe7cc031030 con 0x7fe7c0003970 2023-10-18T11:41:32.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.695+0000 7fe7d4ff9640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe7c406af60 0x7fe7c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:32.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.696+0000 7fe7d4ff9640 1 -- 172.21.15.125:0/2090869 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7fe7cc0aa760 con 0x7fe7c0003970 2023-10-18T11:41:32.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.696+0000 7fe7d77fe640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe7c406af60 0x7fe7c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:32.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.696+0000 7fe7d77fe640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe7c406af60 0x7fe7c406d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009b10 tx=0x7fe7c8005f90 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:32.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.701+0000 7fe7d4ff9640 1 -- 172.21.15.125:0/2090869 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7cc078f40 con 0x7fe7c0003970 2023-10-18T11:41:32.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.900+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7d804da70 con 0x7fe7c0003970 2023-10-18T11:41:32.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.900+0000 7fe7d4ff9640 1 -- 172.21.15.125:0/2090869 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3706 (secure 0 0 0) 0x7fe7cc044e90 con 0x7fe7c0003970 2023-10-18T11:41:32.903 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:32.903 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":62,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2692191151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2692191151},{"type":"v1","addr":"172.21.15.139:6836","nonce":2692191151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1995790917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1995790917},{"type":"v1","addr":"172.21.15.139:6837","nonce":1995790917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/55232611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":55232611},{"type":"v1","addr":"172.21.15.125:6834","nonce":55232611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":10230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/4145296700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":4145296700},{"type":"v1","addr":"172.21.15.125:6835","nonce":4145296700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[]} 2023-10-18T11:41:32.904 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 62 2023-10-18T11:41:32.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe7c406af60 msgr2=0x7fe7c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe7c406af60 0x7fe7c406d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009b10 tx=0x7fe7c8005f90 comp rx=0 tx=0).stop 2023-10-18T11:41:32.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 msgr2=0x7fe7d8153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:32.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 0x7fe7d8153fd0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc000c00 tx=0x7fe7cc0015f0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 shutdown_connections 2023-10-18T11:41:32.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe7c406af60 0x7fe7c406d420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d81f1610 0x7fe7d81f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d8157b00 0x7fe7d8146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 --2- 172.21.15.125:0/2090869 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c0003970 0x7fe7d8153fd0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:32.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 >> 172.21.15.125:0/2090869 conn(0x7fe7d80d0c20 msgr2=0x7fe7d806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:32.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 shutdown_connections 2023-10-18T11:41:32.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:32.902+0000 7fe7dd7a0640 1 -- 172.21.15.125:0/2090869 wait complete. 2023-10-18T11:41:32.916 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T11:41:32.916 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T11:41:32.917 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T11:41:32.917 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:41:32.918 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T11:41:32.918 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T11:41:32.918 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T11:41:32.919 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:41:32.919 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T11:41:32.919 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T11:41:32.920 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T11:41:32.920 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:41:32.921 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T11:41:32.921 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T11:41:32.922 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T11:41:32.922 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:41:32.926 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T11:41:32.926 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T11:41:32.928 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T11:41:32.928 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T11:41:32.932 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T11:41:32.933 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T11:41:32.934 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T11:41:32.934 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T11:41:32.971 INFO:tasks.ceph.mds.a:Started 2023-10-18T11:41:32.971 DEBUG:teuthology.parallel:result is None 2023-10-18T11:41:32.972 INFO:tasks.ceph.mds.c:Started 2023-10-18T11:41:32.972 DEBUG:teuthology.parallel:result is None 2023-10-18T11:41:32.976 INFO:tasks.ceph.mds.b:Started 2023-10-18T11:41:32.976 DEBUG:teuthology.parallel:result is None 2023-10-18T11:41:32.977 INFO:tasks.ceph.mds.d:Started 2023-10-18T11:41:32.978 DEBUG:teuthology.parallel:result is None 2023-10-18T11:41:32.978 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T11:41:33.045 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T11:41:33.045 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T11:41:33.059 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T11:41:33.060 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T11:41:33.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.084+0000 7fe862444640 1 -- 172.21.15.125:0/1008278569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 msgr2=0x7fe85c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:33.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.084+0000 7fe862444640 1 --2- 172.21.15.125:0/1008278569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 0x7fe85c0d1910 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fe84c005fd0 tx=0x7fe84c02bd90 comp rx=0 tx=0).stop 2023-10-18T11:41:33.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.084+0000 7fe862444640 1 -- 172.21.15.125:0/1008278569 shutdown_connections 2023-10-18T11:41:33.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.084+0000 7fe862444640 1 --2- 172.21.15.125:0/1008278569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe85c15b2a0 0x7fe85c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:33.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.084+0000 7fe862444640 1 --2- 172.21.15.125:0/1008278569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 0x7fe85c0d1910 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:33.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.084+0000 7fe862444640 1 -- 172.21.15.125:0/1008278569 >> 172.21.15.125:0/1008278569 conn(0x7fe85c0d0d60 msgr2=0x7fe85c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:33.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.084+0000 7fe862444640 1 -- 172.21.15.125:0/1008278569 shutdown_connections 2023-10-18T11:41:33.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.084+0000 7fe862444640 1 -- 172.21.15.125:0/1008278569 wait complete. 2023-10-18T11:41:33.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.085+0000 7fe862444640 1 Processor -- start 2023-10-18T11:41:33.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.085+0000 7fe862444640 1 -- start start 2023-10-18T11:41:33.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe862444640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe85c0ceb40 0x7fe85c1f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:33.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe862444640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 0x7fe85c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:33.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe862444640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe85c15b2a0 0x7fe85c1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:33.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe862444640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0ca290 con 0x7fe85c0cf340 2023-10-18T11:41:33.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe862444640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15b870 con 0x7fe85c0ceb40 2023-10-18T11:41:33.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe862444640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15bb50 con 0x7fe85c15b2a0 2023-10-18T11:41:33.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 0x7fe85c0c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:33.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 0x7fe85c0c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52596/0 (socket says 172.21.15.125:52596) 2023-10-18T11:41:33.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 -- 172.21.15.125:0/3082796150 learned_addr learned my addr 172.21.15.125:0/3082796150 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:33.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe8609e1640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe85c15b2a0 0x7fe85c1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:33.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85bfff640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe85c0ceb40 0x7fe85c1f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:33.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 -- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe85c15b2a0 msgr2=0x7fe85c1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:33.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe85c15b2a0 0x7fe85c1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:33.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 -- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe85c0ceb40 msgr2=0x7fe85c1f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:33.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe85c0ceb40 0x7fe85c1f15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:33.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 -- 172.21.15.125:0/3082796150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c009560 con 0x7fe85c0cf340 2023-10-18T11:41:33.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe8609e1640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe85c15b2a0 0x7fe85c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:33.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.086+0000 7fe85b7fe640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 0x7fe85c0c9d50 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fe85000d740 tx=0x7fe85000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:33.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.087+0000 7fe858ff9640 1 -- 172.21.15.125:0/3082796150 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850009c30 con 0x7fe85c0cf340 2023-10-18T11:41:33.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.087+0000 7fe858ff9640 1 -- 172.21.15.125:0/3082796150 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe850009e80 con 0x7fe85c0cf340 2023-10-18T11:41:33.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.087+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c15be30 con 0x7fe85c0cf340 2023-10-18T11:41:33.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.087+0000 7fe858ff9640 1 -- 172.21.15.125:0/3082796150 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8500106c0 con 0x7fe85c0cf340 2023-10-18T11:41:33.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.087+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c15c3f0 con 0x7fe85c0cf340 2023-10-18T11:41:33.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.088+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c15c580 con 0x7fe85c0cf340 2023-10-18T11:41:33.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.088+0000 7fe858ff9640 1 -- 172.21.15.125:0/3082796150 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe850010820 con 0x7fe85c0cf340 2023-10-18T11:41:33.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.088+0000 7fe858ff9640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe84806af60 0x7fe84806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:33.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.088+0000 7fe858ff9640 1 -- 172.21.15.125:0/3082796150 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6023+0+0 (secure 0 0 0) 0x7fe850089e80 con 0x7fe85c0cf340 2023-10-18T11:41:33.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.089+0000 7fe85bfff640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe84806af60 0x7fe84806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:33.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.089+0000 7fe85bfff640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe84806af60 0x7fe84806d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe84c035000 tx=0x7fe84c004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:33.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.093+0000 7fe858ff9640 1 -- 172.21.15.125:0/3082796150 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe850054c60 con 0x7fe85c0cf340 2023-10-18T11:41:33.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:33.270+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe85c0690e0 con 0x7fe85c0cf340 2023-10-18T11:41:34.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.387+0000 7fe858ff9640 1 -- 172.21.15.125:0/3082796150 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v170) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe850021080 con 0x7fe85c0cf340 2023-10-18T11:41:34.389 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T11:41:34.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe84806af60 msgr2=0x7fe84806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe84806af60 0x7fe84806d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe84c035000 tx=0x7fe84c004730 comp rx=0 tx=0).stop 2023-10-18T11:41:34.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 msgr2=0x7fe85c0c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 0x7fe85c0c9d50 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fe85000d740 tx=0x7fe85000dc00 comp rx=0 tx=0).stop 2023-10-18T11:41:34.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 shutdown_connections 2023-10-18T11:41:34.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe84806af60 0x7fe84806d420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe85c15b2a0 0x7fe85c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe85c0cf340 0x7fe85c0c9d50 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 --2- 172.21.15.125:0/3082796150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe85c0ceb40 0x7fe85c1f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.389+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 >> 172.21.15.125:0/3082796150 conn(0x7fe85c0d0d60 msgr2=0x7fe85c15a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:34.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.390+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 shutdown_connections 2023-10-18T11:41:34.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.390+0000 7fe862444640 1 -- 172.21.15.125:0/3082796150 wait complete. 2023-10-18T11:41:34.405 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T11:41:34.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.511+0000 7fc86164f640 1 -- 172.21.15.125:0/3974568225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc844003970 msgr2=0x7fc844003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.511+0000 7fc86164f640 1 --2- 172.21.15.125:0/3974568225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc844003970 0x7fc844003d90 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009f20 tx=0x7fc8440057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.511+0000 7fc85b7fe640 1 -- 172.21.15.125:0/3974568225 reap_dead start 2023-10-18T11:41:34.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.511+0000 7fc86164f640 1 -- 172.21.15.125:0/3974568225 shutdown_connections 2023-10-18T11:41:34.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.511+0000 7fc86164f640 1 -- 172.21.15.125:0/3974568225 >> 172.21.15.125:0/3974568225 conn(0x7fc85c069930 msgr2=0x7fc85c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:34.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.511+0000 7fc86164f640 1 -- 172.21.15.125:0/3974568225 shutdown_connections 2023-10-18T11:41:34.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.511+0000 7fc86164f640 1 -- 172.21.15.125:0/3974568225 wait complete. 2023-10-18T11:41:34.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.513+0000 7fc86164f640 1 Processor -- start 2023-10-18T11:41:34.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.513+0000 7fc86164f640 1 -- start start 2023-10-18T11:41:34.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.513+0000 7fc86164f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844003970 0x7fc85c1eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:34.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.513+0000 7fc86164f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc85c14db10 0x7fc85c14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:34.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.513+0000 7fc86164f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14cdd0 0x7fc85c14ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:34.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc86164f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c14c940 con 0x7fc85c14db10 2023-10-18T11:41:34.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc86164f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c14bd30 con 0x7fc844003970 2023-10-18T11:41:34.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc86164f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c14c010 con 0x7fc85c14cdd0 2023-10-18T11:41:34.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc85c14db10 0x7fc85c14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:34.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc85c14db10 0x7fc85c14b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52606/0 (socket says 172.21.15.125:52606) 2023-10-18T11:41:34.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 -- 172.21.15.125:0/2608859839 learned_addr learned my addr 172.21.15.125:0/2608859839 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:34.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85b7fe640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14cdd0 0x7fc85c14ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:34.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85affd640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844003970 0x7fc85c1eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:34.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 -- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14cdd0 msgr2=0x7fc85c14ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14cdd0 0x7fc85c14ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 -- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844003970 msgr2=0x7fc85c1eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844003970 0x7fc85c1eed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 -- 172.21.15.125:0/2608859839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c14c2f0 con 0x7fc85c14db10 2023-10-18T11:41:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85b7fe640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14cdd0 0x7fc85c14ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc85a7fc640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc85c14db10 0x7fc85c14b3c0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fc850000fb0 tx=0x7fc85000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc8437fe640 1 -- 172.21.15.125:0/2608859839 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000c910 con 0x7fc85c14db10 2023-10-18T11:41:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc8437fe640 1 -- 172.21.15.125:0/2608859839 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc85000cb60 con 0x7fc85c14db10 2023-10-18T11:41:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c14c5d0 con 0x7fc85c14db10 2023-10-18T11:41:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc8437fe640 1 -- 172.21.15.125:0/2608859839 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850011460 con 0x7fc85c14db10 2023-10-18T11:41:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.514+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c14b900 con 0x7fc85c14db10 2023-10-18T11:41:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.515+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c14cab0 con 0x7fc85c14db10 2023-10-18T11:41:34.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.516+0000 7fc8437fe640 1 -- 172.21.15.125:0/2608859839 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc8500115c0 con 0x7fc85c14db10 2023-10-18T11:41:34.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.516+0000 7fc8437fe640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc84806af90 0x7fc84806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:34.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.516+0000 7fc8437fe640 1 -- 172.21.15.125:0/2608859839 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5205+0+0 (secure 0 0 0) 0x7fc850089f10 con 0x7fc85c14db10 2023-10-18T11:41:34.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.516+0000 7fc85affd640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc84806af90 0x7fc84806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.517+0000 7fc85affd640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc84806af90 0x7fc84806d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc84c037000 tx=0x7fc84c004a40 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.520+0000 7fc8437fe640 1 -- 172.21.15.125:0/2608859839 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc850054f90 con 0x7fc85c14db10 2023-10-18T11:41:34.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.749+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc85c0690e0 con 0x7fc85c14db10 2023-10-18T11:41:34.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.750+0000 7fc8437fe640 1 -- 172.21.15.125:0/2608859839 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v47) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc85001f020 con 0x7fc85c14db10 2023-10-18T11:41:34.752 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:34.752 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:41:34.753 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T11:41:34.753 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.753 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T11:41:34.753 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T11:41:34.753 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.754 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:41:34.754 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:41:34.754 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:41:34.754 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.754 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.755 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.755 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.755 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T11:41:34.755 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T11:41:34.755 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.756 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:41:34.756 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:41:34.756 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:41:34.756 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.756 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.757 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.757 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.757 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T11:41:34.757 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T11:41:34.757 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.758 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:41:34.758 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:41:34.758 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:41:34.758 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.758 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.759 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.759 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.759 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T11:41:34.759 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T11:41:34.759 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.760 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:41:34.760 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:41:34.760 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:41:34.760 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.760 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.760 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.761 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.761 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T11:41:34.761 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T11:41:34.761 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.761 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:41:34.762 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:41:34.762 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.762 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.762 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.762 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.763 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T11:41:34.763 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T11:41:34.763 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.763 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:41:34.763 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:41:34.764 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.764 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.764 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.764 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.764 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T11:41:34.764 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T11:41:34.765 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.765 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:41:34.765 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:41:34.765 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.765 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.766 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.766 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.766 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T11:41:34.766 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T11:41:34.766 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.767 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:41:34.767 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:41:34.767 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.767 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.767 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.768 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.768 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T11:41:34.768 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T11:41:34.768 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.768 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:41:34.769 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:41:34.769 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.769 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.769 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.769 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.770 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T11:41:34.770 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T11:41:34.770 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.770 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:41:34.770 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:41:34.771 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.771 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.771 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.771 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.771 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T11:41:34.772 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T11:41:34.772 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.772 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:41:34.772 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:41:34.772 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.773 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.773 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.773 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.773 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T11:41:34.773 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T11:41:34.773 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.774 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:41:34.774 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:41:34.774 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.774 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.774 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.775 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.775 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T11:41:34.775 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T11:41:34.775 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.775 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:41:34.776 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T11:41:34.776 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T11:41:34.776 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.776 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.776 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.777 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T11:41:34.777 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T11:41:34.777 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.777 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:41:34.777 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:41:34.778 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:41:34.778 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:41:34.778 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.778 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.778 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.779 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T11:41:34.779 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T11:41:34.779 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.779 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:41:34.779 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:41:34.780 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:41:34.780 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:41:34.780 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.780 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.780 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.781 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T11:41:34.781 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T11:41:34.781 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.781 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:41:34.781 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:41:34.781 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:41:34.782 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:41:34.782 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.782 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.782 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.782 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T11:41:34.783 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T11:41:34.783 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.783 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:41:34.783 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:41:34.783 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:41:34.784 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.784 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.784 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.784 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.784 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T11:41:34.785 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T11:41:34.785 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.785 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:41:34.785 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T11:41:34.785 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.786 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.786 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:34.786 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:34.786 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T11:41:34.786 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T11:41:34.787 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:41:34.787 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:41:34.787 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T11:41:34.787 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:41:34.787 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.788 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:34.788 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T11:41:34.788 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:41:34.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc84806af90 msgr2=0x7fc84806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc84806af90 0x7fc84806d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc84c037000 tx=0x7fc84c004a40 comp rx=0 tx=0).stop 2023-10-18T11:41:34.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc85c14db10 msgr2=0x7fc85c14b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc85c14db10 0x7fc85c14b3c0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fc850000fb0 tx=0x7fc85000ea90 comp rx=0 tx=0).stop 2023-10-18T11:41:34.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 shutdown_connections 2023-10-18T11:41:34.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc84806af90 0x7fc84806d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14cdd0 0x7fc85c14ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc85c14db10 0x7fc85c14b3c0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 --2- 172.21.15.125:0/2608859839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844003970 0x7fc85c1eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.751+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 >> 172.21.15.125:0/2608859839 conn(0x7fc85c069930 msgr2=0x7fc85c152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:34.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.752+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 shutdown_connections 2023-10-18T11:41:34.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.752+0000 7fc86164f640 1 -- 172.21.15.125:0/2608859839 wait complete. 2023-10-18T11:41:34.793 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:34.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.892+0000 7f4fd3a75640 1 -- 172.21.15.125:0/2970342748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fb8004d00 msgr2=0x7f4fb8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.892+0000 7f4fd3a75640 1 --2- 172.21.15.125:0/2970342748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fb8004d00 0x7f4fb8005180 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c290 tx=0x7f4fb80057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.892+0000 7f4fd2012640 1 -- 172.21.15.125:0/2970342748 reap_dead start 2023-10-18T11:41:34.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.893+0000 7f4fd3a75640 1 -- 172.21.15.125:0/2970342748 shutdown_connections 2023-10-18T11:41:34.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.893+0000 7f4fd3a75640 1 -- 172.21.15.125:0/2970342748 >> 172.21.15.125:0/2970342748 conn(0x7f4fcc0b3f00 msgr2=0x7f4fcc0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:34.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.893+0000 7f4fd3a75640 1 -- 172.21.15.125:0/2970342748 shutdown_connections 2023-10-18T11:41:34.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.893+0000 7f4fd3a75640 1 -- 172.21.15.125:0/2970342748 wait complete. 2023-10-18T11:41:34.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.894+0000 7f4fd3a75640 1 Processor -- start 2023-10-18T11:41:34.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.894+0000 7f4fd3a75640 1 -- start start 2023-10-18T11:41:34.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.894+0000 7f4fd3a75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fcc1427c0 0x7f4fcc142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:34.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.894+0000 7f4fd3a75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4fcc143170 0x7f4fcc0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:34.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.894+0000 7f4fd3a75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fcc13fc30 0x7f4fcc06b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:34.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd3a75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc150260 con 0x7f4fcc13fc30 2023-10-18T11:41:34.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd3a75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc1435b0 con 0x7f4fcc1427c0 2023-10-18T11:41:34.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd3a75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc14d1e0 con 0x7f4fcc143170 2023-10-18T11:41:34.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fcc13fc30 0x7f4fcc06b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:34.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fcc13fc30 0x7f4fcc06b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52616/0 (socket says 172.21.15.125:52616) 2023-10-18T11:41:34.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 -- 172.21.15.125:0/1713993792 learned_addr learned my addr 172.21.15.125:0/1713993792 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:34.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd1010640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4fcc143170 0x7f4fcc0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:34.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd1811640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fcc1427c0 0x7f4fcc142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:34.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 -- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4fcc143170 msgr2=0x7f4fcc0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4fcc143170 0x7f4fcc0c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 -- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fcc1427c0 msgr2=0x7f4fcc142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:34.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fcc1427c0 0x7f4fcc142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:34.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 -- 172.21.15.125:0/1713993792 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0c82a0 con 0x7f4fcc13fc30 2023-10-18T11:41:34.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd1811640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fcc1427c0 0x7f4fcc142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:34.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd2012640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fcc13fc30 0x7f4fcc06b240 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c290 tx=0x7f4fc40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:34.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fc27fc640 1 -- 172.21.15.125:0/1713993792 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40062d0 con 0x7f4fcc13fc30 2023-10-18T11:41:34.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fc27fc640 1 -- 172.21.15.125:0/1713993792 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4fc4006430 con 0x7f4fcc13fc30 2023-10-18T11:41:34.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fc27fc640 1 -- 172.21.15.125:0/1713993792 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4001610 con 0x7f4fcc13fc30 2023-10-18T11:41:34.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc14d430 con 0x7f4fcc13fc30 2023-10-18T11:41:34.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.895+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc140540 con 0x7f4fcc13fc30 2023-10-18T11:41:34.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.896+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc1406d0 con 0x7f4fcc13fc30 2023-10-18T11:41:34.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.897+0000 7f4fc27fc640 1 -- 172.21.15.125:0/1713993792 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4fc4031030 con 0x7f4fcc13fc30 2023-10-18T11:41:34.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.897+0000 7f4fc27fc640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4fb006af60 0x7f4fb006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:34.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.897+0000 7f4fc27fc640 1 -- 172.21.15.125:0/1713993792 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f4fc40aa870 con 0x7f4fcc13fc30 2023-10-18T11:41:34.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.897+0000 7f4fd1811640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4fb006af60 0x7f4fb006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:34.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.898+0000 7f4fd1811640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4fb006af60 0x7f4fb006d420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8005fd0 tx=0x7f4fc8009500 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:34.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:34.901+0000 7f4fc27fc640 1 -- 172.21.15.125:0/1713993792 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fcc1406d0 con 0x7f4fcc13fc30 2023-10-18T11:41:35.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.069+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fcc14d950 con 0x7f4fcc13fc30 2023-10-18T11:41:35.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.070+0000 7f4fc27fc640 1 -- 172.21.15.125:0/1713993792 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+13164 (secure 0 0 0) 0x7f4fcc14d950 con 0x7f4fcc13fc30 2023-10-18T11:41:35.072 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:35.072 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":170,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:34.386271+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:35.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4fb006af60 msgr2=0x7f4fb006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4fb006af60 0x7f4fb006d420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8005fd0 tx=0x7f4fc8009500 comp rx=0 tx=0).stop 2023-10-18T11:41:35.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fcc13fc30 msgr2=0x7f4fcc06b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fcc13fc30 0x7f4fcc06b240 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c290 tx=0x7f4fc40014a0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 shutdown_connections 2023-10-18T11:41:35.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4fb006af60 0x7f4fb006d420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fcc13fc30 0x7f4fcc06b240 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4fcc143170 0x7f4fcc0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 --2- 172.21.15.125:0/1713993792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fcc1427c0 0x7f4fcc142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 >> 172.21.15.125:0/1713993792 conn(0x7f4fcc0b3f00 msgr2=0x7f4fcc155740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:35.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 shutdown_connections 2023-10-18T11:41:35.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.072+0000 7f4fd3a75640 1 -- 172.21.15.125:0/1713993792 wait complete. 2023-10-18T11:41:35.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:41:35.086 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:41:35.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.188+0000 7f3b863c5640 1 -- 172.21.15.125:0/104162271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8015b2a0 msgr2=0x7f3b800d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.188+0000 7f3b863c5640 1 --2- 172.21.15.125:0/104162271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8015b2a0 0x7f3b800d1910 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009830 tx=0x7f3b7402bd60 comp rx=0 tx=0).stop 2023-10-18T11:41:35.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.188+0000 7f3b863c5640 1 -- 172.21.15.125:0/104162271 shutdown_connections 2023-10-18T11:41:35.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.188+0000 7f3b863c5640 1 --2- 172.21.15.125:0/104162271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8015b2a0 0x7f3b800d1910 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.188+0000 7f3b863c5640 1 -- 172.21.15.125:0/104162271 >> 172.21.15.125:0/104162271 conn(0x7f3b800d0d60 msgr2=0x7f3b800d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:35.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.188+0000 7f3b863c5640 1 -- 172.21.15.125:0/104162271 shutdown_connections 2023-10-18T11:41:35.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.188+0000 7f3b863c5640 1 -- 172.21.15.125:0/104162271 wait complete. 2023-10-18T11:41:35.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.189+0000 7f3b863c5640 1 Processor -- start 2023-10-18T11:41:35.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b863c5640 1 -- start start 2023-10-18T11:41:35.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b863c5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800ceb40 0x7f3b801f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b863c5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800cf340 0x7f3b800b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b863c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8015b2a0 0x7f3b800c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b863c5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b801f1580 con 0x7f3b800ceb40 2023-10-18T11:41:35.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b863c5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8015d710 con 0x7f3b8015b2a0 2023-10-18T11:41:35.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b863c5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80158bb0 con 0x7f3b800cf340 2023-10-18T11:41:35.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b7ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800ceb40 0x7f3b801f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b7ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800ceb40 0x7f3b801f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52620/0 (socket says 172.21.15.125:52620) 2023-10-18T11:41:35.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b7ffff640 1 -- 172.21.15.125:0/1778783732 learned_addr learned my addr 172.21.15.125:0/1778783732 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:35.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.190+0000 7f3b7f7fe640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800cf340 0x7f3b800b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b84962640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8015b2a0 0x7f3b800c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7ffff640 1 -- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800cf340 msgr2=0x7f3b800b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7ffff640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800cf340 0x7f3b800b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7ffff640 1 -- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8015b2a0 msgr2=0x7f3b800c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7ffff640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8015b2a0 0x7f3b800c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7ffff640 1 -- 172.21.15.125:0/1778783732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b74009490 con 0x7f3b800ceb40 2023-10-18T11:41:35.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7f7fe640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800cf340 0x7f3b800b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:35.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7ffff640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800ceb40 0x7f3b801f1040 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000c430 tx=0x7f3b7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:35.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7cff9640 1 -- 172.21.15.125:0/1778783732 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b700052d0 con 0x7f3b800ceb40 2023-10-18T11:41:35.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7cff9640 1 -- 172.21.15.125:0/1778783732 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b70007020 con 0x7f3b800ceb40 2023-10-18T11:41:35.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b800b5c80 con 0x7f3b800ceb40 2023-10-18T11:41:35.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b7cff9640 1 -- 172.21.15.125:0/1778783732 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70007430 con 0x7f3b800ceb40 2023-10-18T11:41:35.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.191+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8015cb90 con 0x7f3b800ceb40 2023-10-18T11:41:35.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.192+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b801f16c0 con 0x7f3b800ceb40 2023-10-18T11:41:35.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.192+0000 7f3b7cff9640 1 -- 172.21.15.125:0/1778783732 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3b700067d0 con 0x7f3b800ceb40 2023-10-18T11:41:35.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.193+0000 7f3b7cff9640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3b6c06af90 0x7f3b6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.193+0000 7f3b7cff9640 1 -- 172.21.15.125:0/1778783732 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f3b70088c20 con 0x7f3b800ceb40 2023-10-18T11:41:35.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.193+0000 7f3b7f7fe640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3b6c06af90 0x7f3b6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.193+0000 7f3b7f7fe640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3b6c06af90 0x7f3b6c06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b74036000 tx=0x7f3b74004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:35.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.197+0000 7f3b7cff9640 1 -- 172.21.15.125:0/1778783732 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b801f16c0 con 0x7f3b800ceb40 2023-10-18T11:41:35.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.361+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b801f16c0 con 0x7f3b800ceb40 2023-10-18T11:41:35.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.361+0000 7f3b7cff9640 1 -- 172.21.15.125:0/1778783732 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b801f16c0 con 0x7f3b800ceb40 2023-10-18T11:41:35.364 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:41:35.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3b6c06af90 msgr2=0x7f3b6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3b6c06af90 0x7f3b6c06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b74036000 tx=0x7f3b74004730 comp rx=0 tx=0).stop 2023-10-18T11:41:35.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800ceb40 msgr2=0x7f3b801f1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800ceb40 0x7f3b801f1040 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000c430 tx=0x7f3b7000c8f0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 shutdown_connections 2023-10-18T11:41:35.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3b6c06af90 0x7f3b6c06d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b8015b2a0 0x7f3b800c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b800cf340 0x7f3b800b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 --2- 172.21.15.125:0/1778783732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b800ceb40 0x7f3b801f1040 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 >> 172.21.15.125:0/1778783732 conn(0x7f3b800d0d60 msgr2=0x7f3b8006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:35.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 shutdown_connections 2023-10-18T11:41:35.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.363+0000 7f3b863c5640 1 -- 172.21.15.125:0/1778783732 wait complete. 2023-10-18T11:41:35.377 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T11:41:35.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 -- 172.21.15.125:0/2081535746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 msgr2=0x7f3c440cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/2081535746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 0x7f3c440cf720 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f3c44089ae0 tx=0x7f3c2c02bd10 comp rx=0 tx=0).stop 2023-10-18T11:41:35.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 -- 172.21.15.125:0/2081535746 shutdown_connections 2023-10-18T11:41:35.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/2081535746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 0x7f3c440cf720 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/2081535746 >> v2:172.21.15.139:3300/0 conn(0x7f3c440ceb40 0x7f3c440b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/2081535746 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c440cf340 0x7f3c440d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 -- 172.21.15.125:0/2081535746 >> 172.21.15.125:0/2081535746 conn(0x7f3c440d0d60 msgr2=0x7f3c440d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:35.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 -- 172.21.15.125:0/2081535746 shutdown_connections 2023-10-18T11:41:35.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.478+0000 7f3c49fc9640 1 -- 172.21.15.125:0/2081535746 wait complete. 2023-10-18T11:41:35.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.479+0000 7f3c49fc9640 1 Processor -- start 2023-10-18T11:41:35.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c49fc9640 1 -- start start 2023-10-18T11:41:35.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c49fc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c440ceb40 0x7f3c44158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c49fc9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c440cf340 0x7f3c4406afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c49fc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 0x7f3c4406b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c49fc9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c440c9de0 con 0x7f3c4415b2a0 2023-10-18T11:41:35.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c49fc9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c440c7e30 con 0x7f3c440ceb40 2023-10-18T11:41:35.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c49fc9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c44158940 con 0x7f3c440cf340 2023-10-18T11:41:35.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c43fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 0x7f3c4406b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c43fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 0x7f3c4406b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52636/0 (socket says 172.21.15.125:52636) 2023-10-18T11:41:35.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.480+0000 7f3c43fff640 1 -- 172.21.15.125:0/1037614526 learned_addr learned my addr 172.21.15.125:0/1037614526 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:35.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c3affd640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c440cf340 0x7f3c4406afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c437fe640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c440ceb40 0x7f3c44158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c43fff640 1 -- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c440cf340 msgr2=0x7f3c4406afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c43fff640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c440cf340 0x7f3c4406afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c43fff640 1 -- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c440ceb40 msgr2=0x7f3c44158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c43fff640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c440ceb40 0x7f3c44158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c43fff640 1 -- 172.21.15.125:0/1037614526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c009560 con 0x7f3c4415b2a0 2023-10-18T11:41:35.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c3affd640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c440cf340 0x7f3c4406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:35.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c43fff640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 0x7f3c4406b520 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00da90 tx=0x7f3c3c00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:35.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c40ff9640 1 -- 172.21.15.125:0/1037614526 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c00bdd0 con 0x7f3c4415b2a0 2023-10-18T11:41:35.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c40ff9640 1 -- 172.21.15.125:0/1037614526 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c3c003970 con 0x7f3c4415b2a0 2023-10-18T11:41:35.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c440c8040 con 0x7f3c4415b2a0 2023-10-18T11:41:35.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c40ff9640 1 -- 172.21.15.125:0/1037614526 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c003d40 con 0x7f3c4415b2a0 2023-10-18T11:41:35.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.481+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c440c8600 con 0x7f3c4415b2a0 2023-10-18T11:41:35.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.482+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c44159c80 con 0x7f3c4415b2a0 2023-10-18T11:41:35.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.482+0000 7f3c40ff9640 1 -- 172.21.15.125:0/1037614526 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3c3c003ea0 con 0x7f3c4415b2a0 2023-10-18T11:41:35.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.483+0000 7f3c40ff9640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3c2406af60 0x7f3c2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.483+0000 7f3c40ff9640 1 -- 172.21.15.125:0/1037614526 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f3c3c089b20 con 0x7f3c4415b2a0 2023-10-18T11:41:35.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.483+0000 7f3c437fe640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3c2406af60 0x7f3c2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.483+0000 7f3c437fe640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3c2406af60 0x7f3c2406d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c440b0000 tx=0x7f3c34005f90 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:35.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.487+0000 7f3c40ff9640 1 -- 172.21.15.125:0/1037614526 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c3c054c30 con 0x7f3c4415b2a0 2023-10-18T11:41:35.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.654+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3c440690e0 con 0x7f3c4415b2a0 2023-10-18T11:41:35.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.654+0000 7f3c40ff9640 1 -- 172.21.15.125:0/1037614526 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v170) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3c3c017090 con 0x7f3c4415b2a0 2023-10-18T11:41:35.659 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:35.659 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T11:41:35.659 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:35.659 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T11:41:35.660 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T11:41:35.660 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:35.660 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T11:41:35.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3c2406af60 msgr2=0x7f3c2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3c2406af60 0x7f3c2406d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c440b0000 tx=0x7f3c34005f90 comp rx=0 tx=0).stop 2023-10-18T11:41:35.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 msgr2=0x7f3c4406b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 0x7f3c4406b520 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00da90 tx=0x7f3c3c00df50 comp rx=0 tx=0).stop 2023-10-18T11:41:35.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 shutdown_connections 2023-10-18T11:41:35.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f3c2406af60 0x7f3c2406d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4415b2a0 0x7f3c4406b520 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c440cf340 0x7f3c4406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 --2- 172.21.15.125:0/1037614526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c440ceb40 0x7f3c44158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 >> 172.21.15.125:0/1037614526 conn(0x7f3c440d0d60 msgr2=0x7f3c4415a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:35.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 shutdown_connections 2023-10-18T11:41:35.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.656+0000 7f3c49fc9640 1 -- 172.21.15.125:0/1037614526 wait complete. 2023-10-18T11:41:35.669 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T11:41:35.669 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T11:41:35.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.774+0000 7fd78f14c640 1 -- 172.21.15.125:0/3260517698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 msgr2=0x7fd7880c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.774+0000 7fd78f14c640 1 --2- 172.21.15.125:0/3260517698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 0x7fd7880c9b60 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005fd0 tx=0x7fd77c02be20 comp rx=0 tx=0).stop 2023-10-18T11:41:35.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.774+0000 7fd78f14c640 1 -- 172.21.15.125:0/3260517698 shutdown_connections 2023-10-18T11:41:35.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.774+0000 7fd78f14c640 1 --2- 172.21.15.125:0/3260517698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 0x7fd7880c9b60 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.774+0000 7fd78f14c640 1 -- 172.21.15.125:0/3260517698 >> 172.21.15.125:0/3260517698 conn(0x7fd788069930 msgr2=0x7fd788069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:35.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.774+0000 7fd78f14c640 1 -- 172.21.15.125:0/3260517698 shutdown_connections 2023-10-18T11:41:35.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.774+0000 7fd78f14c640 1 -- 172.21.15.125:0/3260517698 wait complete. 2023-10-18T11:41:35.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.775+0000 7fd78f14c640 1 Processor -- start 2023-10-18T11:41:35.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.775+0000 7fd78f14c640 1 -- start start 2023-10-18T11:41:35.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78f14c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7880b1920 0x7fd788156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78f14c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 0x7fd7881565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78f14c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd788154af0 0x7fd7880ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78f14c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd788158330 con 0x7fd7880c96b0 2023-10-18T11:41:35.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78f14c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880ad070 con 0x7fd7880b1920 2023-10-18T11:41:35.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78f14c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880ad350 con 0x7fd788154af0 2023-10-18T11:41:35.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 0x7fd7881565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 0x7fd7881565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52662/0 (socket says 172.21.15.125:52662) 2023-10-18T11:41:35.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 -- 172.21.15.125:0/82042773 learned_addr learned my addr 172.21.15.125:0/82042773 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:35.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78cee8640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7880b1920 0x7fd788156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78d6e9640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd788154af0 0x7fd7880ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 -- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd788154af0 msgr2=0x7fd7880ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd788154af0 0x7fd7880ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 -- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7880b1920 msgr2=0x7fd788156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:35.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7880b1920 0x7fd788156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:35.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 -- 172.21.15.125:0/82042773 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c009560 con 0x7fd7880c96b0 2023-10-18T11:41:35.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd78d6e9640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd788154af0 0x7fd7880ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:35.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.776+0000 7fd787fff640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 0x7fd7881565b0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd77c006010 tx=0x7fd77c02ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:35.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.777+0000 7fd7857fa640 1 -- 172.21.15.125:0/82042773 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c005650 con 0x7fd7880c96b0 2023-10-18T11:41:35.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.777+0000 7fd7857fa640 1 -- 172.21.15.125:0/82042773 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd77c0038f0 con 0x7fd7880c96b0 2023-10-18T11:41:35.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.777+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd788070300 con 0x7fd7880c96b0 2023-10-18T11:41:35.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.777+0000 7fd7857fa640 1 -- 172.21.15.125:0/82042773 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c003ca0 con 0x7fd7880c96b0 2023-10-18T11:41:35.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.777+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78815c3f0 con 0x7fd7880c96b0 2023-10-18T11:41:35.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.777+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7880710a0 con 0x7fd7880c96b0 2023-10-18T11:41:35.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.778+0000 7fd7857fa640 1 -- 172.21.15.125:0/82042773 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd77c03e450 con 0x7fd7880c96b0 2023-10-18T11:41:35.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.778+0000 7fd7857fa640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd77406af60 0x7fd77406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:35.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.779+0000 7fd7857fa640 1 -- 172.21.15.125:0/82042773 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5205+0+0 (secure 0 0 0) 0x7fd77c0ac220 con 0x7fd7880c96b0 2023-10-18T11:41:35.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.779+0000 7fd78cee8640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd77406af60 0x7fd77406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:35.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.779+0000 7fd78cee8640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd77406af60 0x7fd77406d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7780017f0 tx=0x7fd778009040 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:35.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.783+0000 7fd7857fa640 1 -- 172.21.15.125:0/82042773 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd77c077590 con 0x7fd7880c96b0 2023-10-18T11:41:35.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:35.949+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd7880690e0 con 0x7fd7880c96b0 2023-10-18T11:41:36.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.800+0000 7fd7857fa640 1 -- 172.21.15.125:0/82042773 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v171) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd77c07abb0 con 0x7fd7880c96b0 2023-10-18T11:41:36.803 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T11:41:36.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd77406af60 msgr2=0x7fd77406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:36.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd77406af60 0x7fd77406d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7780017f0 tx=0x7fd778009040 comp rx=0 tx=0).stop 2023-10-18T11:41:36.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 msgr2=0x7fd7881565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:36.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 0x7fd7881565b0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd77c006010 tx=0x7fd77c02ddf0 comp rx=0 tx=0).stop 2023-10-18T11:41:36.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 shutdown_connections 2023-10-18T11:41:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fd77406af60 0x7fd77406d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd788154af0 0x7fd7880ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7880c96b0 0x7fd7881565b0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 --2- 172.21.15.125:0/82042773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7880b1920 0x7fd788156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:36.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 >> 172.21.15.125:0/82042773 conn(0x7fd788069930 msgr2=0x7fd788154380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:36.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 shutdown_connections 2023-10-18T11:41:36.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.803+0000 7fd78f14c640 1 -- 172.21.15.125:0/82042773 wait complete. 2023-10-18T11:41:36.821 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T11:41:36.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.934+0000 7f397d555640 1 -- 172.21.15.125:0/3251713645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39600039a0 msgr2=0x7f3960003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:36.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.934+0000 7f397d555640 1 --2- 172.21.15.125:0/3251713645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39600039a0 0x7f3960003dc0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f3968001d50 tx=0x7f39600057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:36.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.934+0000 7f39777fe640 1 -- 172.21.15.125:0/3251713645 reap_dead start 2023-10-18T11:41:36.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.934+0000 7f397d555640 1 -- 172.21.15.125:0/3251713645 shutdown_connections 2023-10-18T11:41:36.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.934+0000 7f397d555640 1 -- 172.21.15.125:0/3251713645 >> 172.21.15.125:0/3251713645 conn(0x7f3978069930 msgr2=0x7f3978069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:36.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.934+0000 7f397d555640 1 -- 172.21.15.125:0/3251713645 shutdown_connections 2023-10-18T11:41:36.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.934+0000 7f397d555640 1 -- 172.21.15.125:0/3251713645 wait complete. 2023-10-18T11:41:36.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.935+0000 7f397d555640 1 Processor -- start 2023-10-18T11:41:36.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.935+0000 7f397d555640 1 -- start start 2023-10-18T11:41:36.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f397d555640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39600039a0 0x7f3978150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:36.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f397d555640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39780b1880 0x7f397812ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:36.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f397d555640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3978157cc0 0x7f3978158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:36.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f397d555640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3978158680 con 0x7f39780b1880 2023-10-18T11:41:36.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f397d555640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39780b1d00 con 0x7f39600039a0 2023-10-18T11:41:36.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f397d555640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3978150910 con 0x7f3978157cc0 2023-10-18T11:41:36.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39780b1880 0x7f397812ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:36.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39780b1880 0x7f397812ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:52690/0 (socket says 172.21.15.125:52690) 2023-10-18T11:41:36.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 -- 172.21.15.125:0/4090956904 learned_addr learned my addr 172.21.15.125:0/4090956904 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:36.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f3976ffd640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39600039a0 0x7f3978150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:36.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39777fe640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3978157cc0 0x7f3978158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:36.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 -- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3978157cc0 msgr2=0x7f3978158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:36.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3978157cc0 0x7f3978158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:36.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 -- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39600039a0 msgr2=0x7f3978150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:36.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39600039a0 0x7f3978150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:36.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 -- 172.21.15.125:0/4090956904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3978070990 con 0x7f39780b1880 2023-10-18T11:41:36.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f39767fc640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39780b1880 0x7f397812ef40 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f396c006660 tx=0x7f396c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:36.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.936+0000 7f395f7fe640 1 -- 172.21.15.125:0/4090956904 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c011900 con 0x7f39780b1880 2023-10-18T11:41:36.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.937+0000 7f395f7fe640 1 -- 172.21.15.125:0/4090956904 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f396c011a80 con 0x7f39780b1880 2023-10-18T11:41:36.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.937+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397812f480 con 0x7f39780b1880 2023-10-18T11:41:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.937+0000 7f395f7fe640 1 -- 172.21.15.125:0/4090956904 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c011e90 con 0x7f39780b1880 2023-10-18T11:41:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.937+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397812f950 con 0x7f39780b1880 2023-10-18T11:41:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.938+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3978070050 con 0x7f39780b1880 2023-10-18T11:41:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.938+0000 7f395f7fe640 1 -- 172.21.15.125:0/4090956904 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f396c00f430 con 0x7f39780b1880 2023-10-18T11:41:36.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.938+0000 7f395f7fe640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396406af90 0x7f396406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.939+0000 7f395f7fe640 1 -- 172.21.15.125:0/4090956904 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5575+0+0 (secure 0 0 0) 0x7f396c08a5c0 con 0x7f39780b1880 2023-10-18T11:41:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.939+0000 7f3976ffd640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396406af90 0x7f396406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.939+0000 7f3976ffd640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396406af90 0x7f396406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3968034000 tx=0x7f3968001510 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:36.943+0000 7f395f7fe640 1 -- 172.21.15.125:0/4090956904 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3978070050 con 0x7f39780b1880 2023-10-18T11:41:37.112 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.110+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3978070050 con 0x7f39780b1880 2023-10-18T11:41:37.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.803+0000 7f395f7fe640 1 -- 172.21.15.125:0/4090956904 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v172) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3978070050 con 0x7f39780b1880 2023-10-18T11:41:37.806 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T11:41:37.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396406af90 msgr2=0x7f396406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:37.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396406af90 0x7f396406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3968034000 tx=0x7f3968001510 comp rx=0 tx=0).stop 2023-10-18T11:41:37.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39780b1880 msgr2=0x7f397812ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:37.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39780b1880 0x7f397812ef40 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f396c006660 tx=0x7f396c003a80 comp rx=0 tx=0).stop 2023-10-18T11:41:37.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 shutdown_connections 2023-10-18T11:41:37.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f396406af90 0x7f396406d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:37.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3978157cc0 0x7f3978158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:37.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39780b1880 0x7f397812ef40 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:37.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 --2- 172.21.15.125:0/4090956904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39600039a0 0x7f3978150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:37.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 >> 172.21.15.125:0/4090956904 conn(0x7f3978069930 msgr2=0x7f397813b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:37.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 shutdown_connections 2023-10-18T11:41:37.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.805+0000 7f397d555640 1 -- 172.21.15.125:0/4090956904 wait complete. 2023-10-18T11:41:37.834 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T11:41:37.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.938+0000 7f983f916640 1 -- 172.21.15.125:0/2981573797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 msgr2=0x7f983806b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:37.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.938+0000 7f983f916640 1 --2- 172.21.15.125:0/2981573797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 0x7f983806b980 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9838071340 tx=0x7f983002be40 comp rx=0 tx=0).stop 2023-10-18T11:41:37.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.938+0000 7f983f916640 1 -- 172.21.15.125:0/2981573797 shutdown_connections 2023-10-18T11:41:37.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.938+0000 7f983f916640 1 --2- 172.21.15.125:0/2981573797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 0x7f983806b980 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:37.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.938+0000 7f983f916640 1 --2- 172.21.15.125:0/2981573797 >> v2:172.21.15.139:3300/0 conn(0x7f98380acc70 0x7f983806a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:37.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.938+0000 7f983f916640 1 -- 172.21.15.125:0/2981573797 >> 172.21.15.125:0/2981573797 conn(0x7f983806fae0 msgr2=0x7f983806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:37.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.938+0000 7f983f916640 1 -- 172.21.15.125:0/2981573797 shutdown_connections 2023-10-18T11:41:37.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.938+0000 7f983f916640 1 -- 172.21.15.125:0/2981573797 wait complete. 2023-10-18T11:41:37.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.939+0000 7f983f916640 1 Processor -- start 2023-10-18T11:41:37.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.939+0000 7f983f916640 1 -- start start 2023-10-18T11:41:37.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983f916640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 0x7f983814a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:37.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983f916640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98380acc70 0x7f98381593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:37.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983f916640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815ad70 0x7f98381598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:37.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983f916640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983814e040 con 0x7f983806ac70 2023-10-18T11:41:37.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983f916640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98381f14e0 con 0x7f983815ad70 2023-10-18T11:41:37.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983f916640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983814d430 con 0x7f98380acc70 2023-10-18T11:41:37.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 0x7f983814a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:37.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 0x7f983814a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56482/0 (socket says 172.21.15.125:56482) 2023-10-18T11:41:37.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 -- 172.21.15.125:0/1375526561 learned_addr learned my addr 172.21.15.125:0/1375526561 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:37.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983deb3640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815ad70 0x7f98381598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:37.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983ceb1640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98380acc70 0x7f98381593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:37.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 -- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98380acc70 msgr2=0x7f98381593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:37.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98380acc70 0x7f98381593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:37.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 -- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815ad70 msgr2=0x7f98381598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:37.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815ad70 0x7f98381598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:37.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 -- 172.21.15.125:0/1375526561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830009560 con 0x7f983806ac70 2023-10-18T11:41:37.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983deb3640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815ad70 0x7f98381598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:37.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.940+0000 7f983d6b2640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 0x7f983814a570 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f983400c7f0 tx=0x7f983400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:37.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.941+0000 7f982dffb640 1 -- 172.21.15.125:0/1375526561 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834003da0 con 0x7f983806ac70 2023-10-18T11:41:37.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.941+0000 7f982dffb640 1 -- 172.21.15.125:0/1375526561 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9834005070 con 0x7f983806ac70 2023-10-18T11:41:37.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.941+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983814d710 con 0x7f983806ac70 2023-10-18T11:41:37.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.941+0000 7f982dffb640 1 -- 172.21.15.125:0/1375526561 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834005460 con 0x7f983806ac70 2023-10-18T11:41:37.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.941+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983814dcd0 con 0x7f983806ac70 2023-10-18T11:41:37.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.942+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98380690e0 con 0x7f983806ac70 2023-10-18T11:41:37.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.942+0000 7f982dffb640 1 -- 172.21.15.125:0/1375526561 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f98340055c0 con 0x7f983806ac70 2023-10-18T11:41:37.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.943+0000 7f982dffb640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f981c06af60 0x7f981c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:37.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.943+0000 7f982dffb640 1 -- 172.21.15.125:0/1375526561 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f9834089300 con 0x7f983806ac70 2023-10-18T11:41:37.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.943+0000 7f983ceb1640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f981c06af60 0x7f981c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:37.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.943+0000 7f983ceb1640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f981c06af60 0x7f981c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9828005fd0 tx=0x7f9828005850 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:37.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:37.947+0000 7f982dffb640 1 -- 172.21.15.125:0/1375526561 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9834054230 con 0x7f983806ac70 2023-10-18T11:41:38.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.147+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f98380b0940 con 0x7f983806ac70 2023-10-18T11:41:38.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.821+0000 7f982dffb640 1 -- 172.21.15.125:0/1375526561 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9834057850 con 0x7f983806ac70 2023-10-18T11:41:38.824 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 21 and data pool 22 2023-10-18T11:41:38.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.826+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f981c06af60 msgr2=0x7f981c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:38.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.826+0000 7f983f916640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f981c06af60 0x7f981c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9828005fd0 tx=0x7f9828005850 comp rx=0 tx=0).stop 2023-10-18T11:41:38.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.826+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 msgr2=0x7f983814a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:38.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.826+0000 7f983f916640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 0x7f983814a570 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f983400c7f0 tx=0x7f983400ccb0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.829+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 shutdown_connections 2023-10-18T11:41:38.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.829+0000 7f983f916640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f981c06af60 0x7f981c06d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.829+0000 7f983f916640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815ad70 0x7f98381598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.829+0000 7f983f916640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98380acc70 0x7f98381593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.829+0000 7f983f916640 1 --2- 172.21.15.125:0/1375526561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f983806ac70 0x7f983814a570 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.829+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 >> 172.21.15.125:0/1375526561 conn(0x7f983806fae0 msgr2=0x7f9838065490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:38.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.831+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 shutdown_connections 2023-10-18T11:41:38.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.831+0000 7f983f916640 1 -- 172.21.15.125:0/1375526561 wait complete. 2023-10-18T11:41:38.856 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:38.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.960+0000 7fc990f4c640 1 -- 172.21.15.125:0/2661892544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98c0b0530 msgr2=0x7fc98c0b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.960+0000 7fc990f4c640 1 --2- 172.21.15.125:0/2661892544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98c0b0530 0x7fc98c0b09c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc984005fd0 tx=0x7fc98402c0e0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.960+0000 7fc990f4c640 1 -- 172.21.15.125:0/2661892544 shutdown_connections 2023-10-18T11:41:38.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.960+0000 7fc990f4c640 1 --2- 172.21.15.125:0/2661892544 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98c0b0530 0x7fc98c0b09c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.960+0000 7fc990f4c640 1 --2- 172.21.15.125:0/2661892544 >> v2:172.21.15.139:3300/0 conn(0x7fc98c0afa20 0x7fc98c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.960+0000 7fc990f4c640 1 -- 172.21.15.125:0/2661892544 >> 172.21.15.125:0/2661892544 conn(0x7fc98c069930 msgr2=0x7fc98c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:38.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.960+0000 7fc990f4c640 1 -- 172.21.15.125:0/2661892544 shutdown_connections 2023-10-18T11:41:38.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.960+0000 7fc990f4c640 1 -- 172.21.15.125:0/2661892544 wait complete. 2023-10-18T11:41:38.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.961+0000 7fc990f4c640 1 Processor -- start 2023-10-18T11:41:38.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.961+0000 7fc990f4c640 1 -- start start 2023-10-18T11:41:38.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc990f4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98c0afa20 0x7fc98c147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:38.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc990f4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98c0b0530 0x7fc98c146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:38.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc990f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc98c0cbd20 0x7fc98c145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:38.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc990f4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c147320 con 0x7fc98c0b0530 2023-10-18T11:41:38.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc990f4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c146230 con 0x7fc98c0cbd20 2023-10-18T11:41:38.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc990f4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c147c30 con 0x7fc98c0afa20 2023-10-18T11:41:38.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98c0b0530 0x7fc98c146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:38.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98c0b0530 0x7fc98c146de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56484/0 (socket says 172.21.15.125:56484) 2023-10-18T11:41:38.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 -- 172.21.15.125:0/50673808 learned_addr learned my addr 172.21.15.125:0/50673808 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:38.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc98a59c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98c0afa20 0x7fc98c147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:38.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc98ad9d640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc98c0cbd20 0x7fc98c145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:38.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 -- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98c0afa20 msgr2=0x7fc98c147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:38.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98c0afa20 0x7fc98c147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 -- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc98c0cbd20 msgr2=0x7fc98c145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:38.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc98c0cbd20 0x7fc98c145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 -- 172.21.15.125:0/50673808 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97400b6a0 con 0x7fc98c0b0530 2023-10-18T11:41:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.962+0000 7fc983fff640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98c0b0530 0x7fc98c146de0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fc974010b80 tx=0x7fc97400e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.963+0000 7fc982ffd640 1 -- 172.21.15.125:0/50673808 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400eb40 con 0x7fc98c0b0530 2023-10-18T11:41:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.963+0000 7fc982ffd640 1 -- 172.21.15.125:0/50673808 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc974011040 con 0x7fc98c0b0530 2023-10-18T11:41:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.963+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc984009840 con 0x7fc98c0b0530 2023-10-18T11:41:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.963+0000 7fc982ffd640 1 -- 172.21.15.125:0/50673808 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974004020 con 0x7fc98c0b0530 2023-10-18T11:41:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.963+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98c14b7f0 con 0x7fc98c0b0530 2023-10-18T11:41:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.964+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98c0690e0 con 0x7fc98c0b0530 2023-10-18T11:41:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.964+0000 7fc982ffd640 1 -- 172.21.15.125:0/50673808 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc974004180 con 0x7fc98c0b0530 2023-10-18T11:41:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.965+0000 7fc982ffd640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc96c06af60 0x7fc96c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.965+0000 7fc982ffd640 1 -- 172.21.15.125:0/50673808 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fc97408c350 con 0x7fc98c0b0530 2023-10-18T11:41:38.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.965+0000 7fc98a59c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc96c06af60 0x7fc96c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:38.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.965+0000 7fc98a59c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc96c06af60 0x7fc96c06d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0056a0 tx=0x7fc97c007e50 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:38.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:38.969+0000 7fc982ffd640 1 -- 172.21.15.125:0/50673808 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9740571c0 con 0x7fc98c0b0530 2023-10-18T11:41:39.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.135+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc98c06f700 con 0x7fc98c0b0530 2023-10-18T11:41:39.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.136+0000 7fc982ffd640 1 -- 172.21.15.125:0/50673808 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+16829 (secure 0 0 0) 0x7fc97405a7e0 con 0x7fc98c0b0530 2023-10-18T11:41:39.139 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:39.139 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":173,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:38.148917+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:41:35.970906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-18T11:41:37.131199+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:39.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc96c06af60 msgr2=0x7fc96c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc96c06af60 0x7fc96c06d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0056a0 tx=0x7fc97c007e50 comp rx=0 tx=0).stop 2023-10-18T11:41:39.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98c0b0530 msgr2=0x7fc98c146de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98c0b0530 0x7fc98c146de0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fc974010b80 tx=0x7fc97400e490 comp rx=0 tx=0).stop 2023-10-18T11:41:39.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 shutdown_connections 2023-10-18T11:41:39.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc96c06af60 0x7fc96c06d420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc98c0cbd20 0x7fc98c145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc98c0b0530 0x7fc98c146de0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 --2- 172.21.15.125:0/50673808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc98c0afa20 0x7fc98c147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 >> 172.21.15.125:0/50673808 conn(0x7fc98c069930 msgr2=0x7fc98c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:39.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 shutdown_connections 2023-10-18T11:41:39.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.139+0000 7fc990f4c640 1 -- 172.21.15.125:0/50673808 wait complete. 2023-10-18T11:41:39.153 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:39.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.253+0000 7f076fba9640 1 -- 172.21.15.125:0/267661490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07540047c0 msgr2=0x7f0754004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.253+0000 7f076fba9640 1 --2- 172.21.15.125:0/267661490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07540047c0 0x7f0754004c40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f075800c310 tx=0x7f07540057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.253+0000 7f076e146640 1 -- 172.21.15.125:0/267661490 reap_dead start 2023-10-18T11:41:39.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.253+0000 7f076fba9640 1 -- 172.21.15.125:0/267661490 shutdown_connections 2023-10-18T11:41:39.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.253+0000 7f076fba9640 1 -- 172.21.15.125:0/267661490 >> 172.21.15.125:0/267661490 conn(0x7f076806fae0 msgr2=0x7f076806fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:39.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.254+0000 7f076fba9640 1 -- 172.21.15.125:0/267661490 shutdown_connections 2023-10-18T11:41:39.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.254+0000 7f076fba9640 1 -- 172.21.15.125:0/267661490 wait complete. 2023-10-18T11:41:39.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076fba9640 1 Processor -- start 2023-10-18T11:41:39.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076fba9640 1 -- start start 2023-10-18T11:41:39.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076fba9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07540047c0 0x7f07681502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:39.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076fba9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f076815b650 0x7f076815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076fba9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0768152c00 0x7f0768153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076fba9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07681535c0 con 0x7f0768152c00 2023-10-18T11:41:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076fba9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076815bff0 con 0x7f076815b650 2023-10-18T11:41:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076fba9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0768153da0 con 0x7f07540047c0 2023-10-18T11:41:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076e146640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0768152c00 0x7f0768153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.255+0000 7f076e146640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0768152c00 0x7f0768153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56498/0 (socket says 172.21.15.125:56498) 2023-10-18T11:41:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076e146640 1 -- 172.21.15.125:0/2268081443 learned_addr learned my addr 172.21.15.125:0/2268081443 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076e146640 1 -- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07540047c0 msgr2=0x7f07681502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076e146640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07540047c0 0x7f07681502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076e146640 1 -- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f076815b650 msgr2=0x7f076815bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076d144640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f076815b650 0x7f076815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:39.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076e146640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f076815b650 0x7f076815bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076e146640 1 -- 172.21.15.125:0/2268081443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07681507e0 con 0x7f0768152c00 2023-10-18T11:41:39.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076e146640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0768152c00 0x7f0768153080 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f076806db70 tx=0x7f076000d6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f075e7fc640 1 -- 172.21.15.125:0/2268081443 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760010650 con 0x7f0768152c00 2023-10-18T11:41:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f075e7fc640 1 -- 172.21.15.125:0/2268081443 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07600108a0 con 0x7f0768152c00 2023-10-18T11:41:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f075e7fc640 1 -- 172.21.15.125:0/2268081443 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760010cb0 con 0x7f0768152c00 2023-10-18T11:41:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07681537e0 con 0x7f0768152c00 2023-10-18T11:41:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.256+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07681e26a0 con 0x7f0768152c00 2023-10-18T11:41:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.257+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07681e2500 con 0x7f0768152c00 2023-10-18T11:41:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.258+0000 7f075e7fc640 1 -- 172.21.15.125:0/2268081443 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f076001b460 con 0x7f0768152c00 2023-10-18T11:41:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.258+0000 7f075e7fc640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f074c06af90 0x7f074c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.258+0000 7f075e7fc640 1 -- 172.21.15.125:0/2268081443 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f0760089a10 con 0x7f0768152c00 2023-10-18T11:41:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.258+0000 7f076d945640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f074c06af90 0x7f074c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:39.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.259+0000 7f076d945640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f074c06af90 0x7f074c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0764005fd0 tx=0x7f076400a680 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:39.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.262+0000 7f075e7fc640 1 -- 172.21.15.125:0/2268081443 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07681e2500 con 0x7f0768152c00 2023-10-18T11:41:39.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.428+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07681e2500 con 0x7f0768152c00 2023-10-18T11:41:39.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.429+0000 7f075e7fc640 1 -- 172.21.15.125:0/2268081443 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+16829 (secure 0 0 0) 0x7f07681e2500 con 0x7f0768152c00 2023-10-18T11:41:39.432 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:39.432 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":173,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:38.148917+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:41:35.970906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-18T11:41:37.131199+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:39.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f074c06af90 msgr2=0x7f074c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f074c06af90 0x7f074c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0764005fd0 tx=0x7f076400a680 comp rx=0 tx=0).stop 2023-10-18T11:41:39.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0768152c00 msgr2=0x7f0768153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0768152c00 0x7f0768153080 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f076806db70 tx=0x7f076000d6a0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 shutdown_connections 2023-10-18T11:41:39.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f074c06af90 0x7f074c06d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f076815b650 0x7f076815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0768152c00 0x7f0768153080 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 --2- 172.21.15.125:0/2268081443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07540047c0 0x7f07681502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 >> 172.21.15.125:0/2268081443 conn(0x7f076806fae0 msgr2=0x7f07680ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:39.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 shutdown_connections 2023-10-18T11:41:39.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.431+0000 7f076fba9640 1 -- 172.21.15.125:0/2268081443 wait complete. 2023-10-18T11:41:39.445 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T11:41:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.548+0000 7f36cfc10640 1 -- 172.21.15.125:0/3182515432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36b4003970 msgr2=0x7f36b4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.548+0000 7f36cfc10640 1 --2- 172.21.15.125:0/3182515432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36b4003970 0x7f36b4003d90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f36c400cc30 tx=0x7f36b40057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.548+0000 7f36cfc10640 1 -- 172.21.15.125:0/3182515432 shutdown_connections 2023-10-18T11:41:39.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.548+0000 7f36cfc10640 1 --2- 172.21.15.125:0/3182515432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36b4003970 0x7f36b4003d90 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.548+0000 7f36cfc10640 1 -- 172.21.15.125:0/3182515432 >> 172.21.15.125:0/3182515432 conn(0x7f36c8069930 msgr2=0x7f36c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.548+0000 7f36cfc10640 1 -- 172.21.15.125:0/3182515432 shutdown_connections 2023-10-18T11:41:39.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.548+0000 7f36cfc10640 1 -- 172.21.15.125:0/3182515432 wait complete. 2023-10-18T11:41:39.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.549+0000 7f36cfc10640 1 Processor -- start 2023-10-18T11:41:39.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.549+0000 7f36cfc10640 1 -- start start 2023-10-18T11:41:39.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36cfc10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4003970 0x7f36c814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:39.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36cfc10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36c814b580 0x7f36c81e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:39.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36cfc10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36c8134fe0 0x7f36c8135460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:39.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36cfc10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8132d20 con 0x7f36c8134fe0 2023-10-18T11:41:39.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36cfc10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c81359a0 con 0x7f36b4003970 2023-10-18T11:41:39.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36cfc10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c81443e0 con 0x7f36c814b580 2023-10-18T11:41:39.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36c8134fe0 0x7f36c8135460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:39.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36c8134fe0 0x7f36c8135460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56516/0 (socket says 172.21.15.125:56516) 2023-10-18T11:41:39.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 -- 172.21.15.125:0/2048668631 learned_addr learned my addr 172.21.15.125:0/2048668631 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:39.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36cd9ac640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4003970 0x7f36c814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:39.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36cd1ab640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36c814b580 0x7f36c81e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:39.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 -- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36c814b580 msgr2=0x7f36c81e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36c814b580 0x7f36c81e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 -- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4003970 msgr2=0x7f36c814cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:39.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4003970 0x7f36c814cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:39.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 -- 172.21.15.125:0/2048668631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c814bad0 con 0x7f36c8134fe0 2023-10-18T11:41:39.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36ce1ad640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36c8134fe0 0x7f36c8135460 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f36c806f6d0 tx=0x7f36c00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:39.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.550+0000 7f36be7fc640 1 -- 172.21.15.125:0/2048668631 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0011870 con 0x7f36c8134fe0 2023-10-18T11:41:39.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.551+0000 7f36be7fc640 1 -- 172.21.15.125:0/2048668631 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f36c0011ac0 con 0x7f36c8134fe0 2023-10-18T11:41:39.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.551+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c814bdb0 con 0x7f36c8134fe0 2023-10-18T11:41:39.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.551+0000 7f36be7fc640 1 -- 172.21.15.125:0/2048668631 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0002090 con 0x7f36c8134fe0 2023-10-18T11:41:39.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.551+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c814c370 con 0x7f36c8134fe0 2023-10-18T11:41:39.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.552+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c80690e0 con 0x7f36c8134fe0 2023-10-18T11:41:39.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.552+0000 7f36be7fc640 1 -- 172.21.15.125:0/2048668631 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f36c001c460 con 0x7f36c8134fe0 2023-10-18T11:41:39.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.552+0000 7f36be7fc640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f36ac06af60 0x7f36ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:39.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.552+0000 7f36be7fc640 1 -- 172.21.15.125:0/2048668631 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f36c008a400 con 0x7f36c8134fe0 2023-10-18T11:41:39.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.553+0000 7f36cd9ac640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f36ac06af60 0x7f36ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:39.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.553+0000 7f36cd9ac640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f36ac06af60 0x7f36ac06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36c4038000 tx=0x7f36c402f040 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:39.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.557+0000 7f36be7fc640 1 -- 172.21.15.125:0/2048668631 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36c0055130 con 0x7f36c8134fe0 2023-10-18T11:41:39.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:39.758+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f36c806f700 con 0x7f36c8134fe0 2023-10-18T11:41:40.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.148+0000 7f36be7fc640 1 -- 172.21.15.125:0/2048668631 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7f36c0058750 con 0x7f36c8134fe0 2023-10-18T11:41:40.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f36ac06af60 msgr2=0x7f36ac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:40.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f36ac06af60 0x7f36ac06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36c4038000 tx=0x7f36c402f040 comp rx=0 tx=0).stop 2023-10-18T11:41:40.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36c8134fe0 msgr2=0x7f36c8135460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:40.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36c8134fe0 0x7f36c8135460 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f36c806f6d0 tx=0x7f36c00019b0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 shutdown_connections 2023-10-18T11:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f36ac06af60 0x7f36ac06d420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36c8134fe0 0x7f36c8135460 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36c814b580 0x7f36c81e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 --2- 172.21.15.125:0/2048668631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36b4003970 0x7f36c814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 >> 172.21.15.125:0/2048668631 conn(0x7f36c8069930 msgr2=0x7f36c81505f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 shutdown_connections 2023-10-18T11:41:40.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.151+0000 7f36cfc10640 1 -- 172.21.15.125:0/2048668631 wait complete. 2023-10-18T11:41:40.164 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T11:41:40.165 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T11:41:40.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.268+0000 7fe458e61640 1 -- 172.21.15.125:0/1369467879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4540c9dc0 msgr2=0x7fe45406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.268+0000 7fe458e61640 1 --2- 172.21.15.125:0/1369467879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4540c9dc0 0x7fe45406b2e0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fe45406d590 tx=0x7fe43c02bdb0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.268+0000 7fe458e61640 1 -- 172.21.15.125:0/1369467879 shutdown_connections 2023-10-18T11:41:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.268+0000 7fe458e61640 1 --2- 172.21.15.125:0/1369467879 >> v2:172.21.15.139:3300/0 conn(0x7fe4540ab500 0x7fe4540ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.268+0000 7fe458e61640 1 --2- 172.21.15.125:0/1369467879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4540c9dc0 0x7fe45406b2e0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.268+0000 7fe458e61640 1 -- 172.21.15.125:0/1369467879 >> 172.21.15.125:0/1369467879 conn(0x7fe454069930 msgr2=0x7fe454069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:40.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.268+0000 7fe458e61640 1 -- 172.21.15.125:0/1369467879 shutdown_connections 2023-10-18T11:41:40.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.268+0000 7fe458e61640 1 -- 172.21.15.125:0/1369467879 wait complete. 2023-10-18T11:41:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.270+0000 7fe458e61640 1 Processor -- start 2023-10-18T11:41:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.270+0000 7fe458e61640 1 -- start start 2023-10-18T11:41:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.270+0000 7fe458e61640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe45406af00 0x7fe4541f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.270+0000 7fe458e61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4540ab500 0x7fe4540aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.270+0000 7fe458e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4540c9dc0 0x7fe4540aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.270+0000 7fe458e61640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45413d1b0 con 0x7fe45406af00 2023-10-18T11:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.270+0000 7fe458e61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45413d5d0 con 0x7fe4540c9dc0 2023-10-18T11:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.270+0000 7fe458e61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45413c530 con 0x7fe4540ab500 2023-10-18T11:41:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe45406af00 0x7fe4541f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe45406af00 0x7fe4541f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56530/0 (socket says 172.21.15.125:56530) 2023-10-18T11:41:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 -- 172.21.15.125:0/412184705 learned_addr learned my addr 172.21.15.125:0/412184705 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 -- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4540ab500 msgr2=0x7fe4540aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:40.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4540ab500 0x7fe4540aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 -- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4540c9dc0 msgr2=0x7fe4540aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:40.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe452d9d640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4540c9dc0 0x7fe4540aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:40.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4540c9dc0 0x7fe4540aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:40.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 -- 172.21.15.125:0/412184705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c009560 con 0x7fe45406af00 2023-10-18T11:41:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe45259c640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe45406af00 0x7fe4541f47d0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fe44400c410 tx=0x7fe44400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe44affd640 1 -- 172.21.15.125:0/412184705 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4440035f0 con 0x7fe45406af00 2023-10-18T11:41:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe44affd640 1 -- 172.21.15.125:0/412184705 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe444003750 con 0x7fe45406af00 2023-10-18T11:41:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45413c810 con 0x7fe45406af00 2023-10-18T11:41:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe44affd640 1 -- 172.21.15.125:0/412184705 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444003ba0 con 0x7fe45406af00 2023-10-18T11:41:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.271+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45413cdd0 con 0x7fe45406af00 2023-10-18T11:41:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.272+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45413d2f0 con 0x7fe45406af00 2023-10-18T11:41:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.273+0000 7fe44affd640 1 -- 172.21.15.125:0/412184705 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe444003d00 con 0x7fe45406af00 2023-10-18T11:41:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.273+0000 7fe44affd640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe43406af60 0x7fe43406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.273+0000 7fe44affd640 1 -- 172.21.15.125:0/412184705 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fe4440890c0 con 0x7fe45406af00 2023-10-18T11:41:40.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.273+0000 7fe44bfff640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe43406af60 0x7fe43406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:40.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.274+0000 7fe44bfff640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe43406af60 0x7fe43406d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe43c035000 tx=0x7fe43c0046c0 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:40.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.277+0000 7fe44affd640 1 -- 172.21.15.125:0/412184705 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe45413d2f0 con 0x7fe45406af00 2023-10-18T11:41:40.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:40.477+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe45406c500 con 0x7fe45406af00 2023-10-18T11:41:41.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.150+0000 7fe44affd640 1 -- 172.21.15.125:0/412184705 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4440170c0 con 0x7fe45406af00 2023-10-18T11:41:41.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.152+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe43406af60 msgr2=0x7fe43406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.152+0000 7fe458e61640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe43406af60 0x7fe43406d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe43c035000 tx=0x7fe43c0046c0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.152+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe45406af00 msgr2=0x7fe4541f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.152+0000 7fe458e61640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe45406af00 0x7fe4541f47d0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fe44400c410 tx=0x7fe44400c8d0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.152+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 shutdown_connections 2023-10-18T11:41:41.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.152+0000 7fe458e61640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fe43406af60 0x7fe43406d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.153+0000 7fe458e61640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4540c9dc0 0x7fe4540aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.153+0000 7fe458e61640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4540ab500 0x7fe4540aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.153+0000 7fe458e61640 1 --2- 172.21.15.125:0/412184705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe45406af00 0x7fe4541f47d0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.153+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 >> 172.21.15.125:0/412184705 conn(0x7fe454069930 msgr2=0x7fe45415a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:41.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.153+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 shutdown_connections 2023-10-18T11:41:41.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.153+0000 7fe458e61640 1 -- 172.21.15.125:0/412184705 wait complete. 2023-10-18T11:41:41.168 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T11:41:41.169 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:41:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/2470414581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac015b2a0 msgr2=0x7f2ac00cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/2470414581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac015b2a0 0x7f2ac00cf720 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0089ae0 tx=0x7f2ab402bd10 comp rx=0 tx=0).stop 2023-10-18T11:41:41.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/2470414581 shutdown_connections 2023-10-18T11:41:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/2470414581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac015b2a0 0x7f2ac00cf720 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/2470414581 >> v2:172.21.15.139:3300/0 conn(0x7f2ac00ceb40 0x7f2ac00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/2470414581 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ac00cf340 0x7f2ac00d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/2470414581 >> 172.21.15.125:0/2470414581 conn(0x7f2ac00d0d60 msgr2=0x7f2ac00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:41.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/2470414581 shutdown_connections 2023-10-18T11:41:41.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.274+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/2470414581 wait complete. 2023-10-18T11:41:41.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.275+0000 7f2ac6b4c640 1 Processor -- start 2023-10-18T11:41:41.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.275+0000 7f2ac6b4c640 1 -- start start 2023-10-18T11:41:41.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2ac6b4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ac00ceb40 0x7f2ac00b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2ac6b4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac00cf340 0x7f2ac0159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2ac6b4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ac015b2a0 0x7f2ac009e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2ac6b4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00b4c20 con 0x7f2ac00cf340 2023-10-18T11:41:41.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2ac6b4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac0159c80 con 0x7f2ac00ceb40 2023-10-18T11:41:41.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2ac6b4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac009ea10 con 0x7f2ac015b2a0 2023-10-18T11:41:41.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2abffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac00cf340 0x7f2ac0159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2abffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac00cf340 0x7f2ac0159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56548/0 (socket says 172.21.15.125:56548) 2023-10-18T11:41:41.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2abffff640 1 -- 172.21.15.125:0/258029757 learned_addr learned my addr 172.21.15.125:0/258029757 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:41.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2ac50e9640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ac015b2a0 0x7f2ac009e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.276+0000 7f2ac48e8640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ac00ceb40 0x7f2ac00b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abffff640 1 -- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ac015b2a0 msgr2=0x7f2ac009e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abffff640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ac015b2a0 0x7f2ac009e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abffff640 1 -- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ac00ceb40 msgr2=0x7f2ac00b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abffff640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ac00ceb40 0x7f2ac00b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abffff640 1 -- 172.21.15.125:0/258029757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4009560 con 0x7f2ac00cf340 2023-10-18T11:41:41.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abffff640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac00cf340 0x7f2ac0159740 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4036040 tx=0x7f2ab402dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:41.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abd7fa640 1 -- 172.21.15.125:0/258029757 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4005650 con 0x7f2ac00cf340 2023-10-18T11:41:41.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abd7fa640 1 -- 172.21.15.125:0/258029757 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ab40038f0 con 0x7f2ac00cf340 2023-10-18T11:41:41.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac00b4010 con 0x7f2ac00cf340 2023-10-18T11:41:41.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2abd7fa640 1 -- 172.21.15.125:0/258029757 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4003ce0 con 0x7f2ac00cf340 2023-10-18T11:41:41.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.277+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac00b45d0 con 0x7f2ac00cf340 2023-10-18T11:41:41.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.278+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac00690e0 con 0x7f2ac00cf340 2023-10-18T11:41:41.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.278+0000 7f2abd7fa640 1 -- 172.21.15.125:0/258029757 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ab403e450 con 0x7f2ac00cf340 2023-10-18T11:41:41.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.279+0000 7f2abd7fa640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f2aac06af60 0x7f2aac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.279+0000 7f2abd7fa640 1 -- 172.21.15.125:0/258029757 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f2ab40ac2d0 con 0x7f2ac00cf340 2023-10-18T11:41:41.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.279+0000 7f2ac48e8640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f2aac06af60 0x7f2aac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.279+0000 7f2ac48e8640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f2aac06af60 0x7f2aac06d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0005fd0 tx=0x7f2ab0005e20 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:41.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.283+0000 7f2abd7fa640 1 -- 172.21.15.125:0/258029757 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ab4077240 con 0x7f2ac00cf340 2023-10-18T11:41:41.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.483+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ac00d0b40 con 0x7f2ac00cf340 2023-10-18T11:41:41.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.483+0000 7f2abd7fa640 1 -- 172.21.15.125:0/258029757 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f2ab407a860 con 0x7f2ac00cf340 2023-10-18T11:41:41.486 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:41.487 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":67,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3883749389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3883749389},{"type":"v1","addr":"172.21.15.139:6836","nonce":3883749389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2038008432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2038008432},{"type":"v1","addr":"172.21.15.139:6837","nonce":2038008432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":10317,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1543508506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1543508506},{"type":"v1","addr":"172.21.15.125:6834","nonce":1543508506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:41:38.149238+0000","modified":"2023-10-18T11:41:40.479014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10320":{"gid":10320,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2949284679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2949284679},{"type":"v1","addr":"172.21.15.125:6835","nonce":2949284679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T11:41:41.488 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 67 2023-10-18T11:41:41.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f2aac06af60 msgr2=0x7f2aac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f2aac06af60 0x7f2aac06d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0005fd0 tx=0x7f2ab0005e20 comp rx=0 tx=0).stop 2023-10-18T11:41:41.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac00cf340 msgr2=0x7f2ac0159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac00cf340 0x7f2ac0159740 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4036040 tx=0x7f2ab402dce0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 shutdown_connections 2023-10-18T11:41:41.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f2aac06af60 0x7f2aac06d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ac015b2a0 0x7f2ac009e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ac00cf340 0x7f2ac0159740 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 --2- 172.21.15.125:0/258029757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ac00ceb40 0x7f2ac00b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 >> 172.21.15.125:0/258029757 conn(0x7f2ac00d0d60 msgr2=0x7f2ac0159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:41.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 shutdown_connections 2023-10-18T11:41:41.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.485+0000 7f2ac6b4c640 1 -- 172.21.15.125:0/258029757 wait complete. 2023-10-18T11:41:41.498 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:41.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.598+0000 7f35bc700640 1 -- 172.21.15.125:0/502533547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b40cf340 msgr2=0x7f35b40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.598+0000 7f35bc700640 1 --2- 172.21.15.125:0/502533547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b40cf340 0x7f35b40d1910 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f35b415b680 tx=0x7f35b002bd30 comp rx=0 tx=0).stop 2023-10-18T11:41:41.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.598+0000 7f35bc700640 1 -- 172.21.15.125:0/502533547 shutdown_connections 2023-10-18T11:41:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.598+0000 7f35bc700640 1 --2- 172.21.15.125:0/502533547 >> v2:172.21.15.139:3300/0 conn(0x7f35b40ceb40 0x7f35b40b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.598+0000 7f35bc700640 1 --2- 172.21.15.125:0/502533547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b40cf340 0x7f35b40d1910 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.598+0000 7f35bc700640 1 -- 172.21.15.125:0/502533547 >> 172.21.15.125:0/502533547 conn(0x7f35b40d0d60 msgr2=0x7f35b40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.598+0000 7f35bc700640 1 -- 172.21.15.125:0/502533547 shutdown_connections 2023-10-18T11:41:41.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.598+0000 7f35bc700640 1 -- 172.21.15.125:0/502533547 wait complete. 2023-10-18T11:41:41.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bc700640 1 Processor -- start 2023-10-18T11:41:41.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bc700640 1 -- start start 2023-10-18T11:41:41.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bc700640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35b40ceb40 0x7f35b4151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bc700640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35b40cf340 0x7f35b40c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bc700640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b415b2a0 0x7f35b40b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bc700640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40c7ed0 con 0x7f35b415b2a0 2023-10-18T11:41:41.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bc700640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40c8170 con 0x7f35b40ceb40 2023-10-18T11:41:41.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bc700640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40c8450 con 0x7f35b40cf340 2023-10-18T11:41:41.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bac9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b415b2a0 0x7f35b40b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bac9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b415b2a0 0x7f35b40b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56570/0 (socket says 172.21.15.125:56570) 2023-10-18T11:41:41.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.600+0000 7f35bac9d640 1 -- 172.21.15.125:0/345993854 learned_addr learned my addr 172.21.15.125:0/345993854 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:41.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35ba49c640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35b40ceb40 0x7f35b4151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35b9c9b640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35b40cf340 0x7f35b40c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35bac9d640 1 -- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35b40cf340 msgr2=0x7f35b40c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35bac9d640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35b40cf340 0x7f35b40c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35bac9d640 1 -- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35b40ceb40 msgr2=0x7f35b4151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35bac9d640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35b40ceb40 0x7f35b4151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35bac9d640 1 -- 172.21.15.125:0/345993854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0009560 con 0x7f35b415b2a0 2023-10-18T11:41:41.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35ba49c640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35b40ceb40 0x7f35b4151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:41.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35bac9d640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b415b2a0 0x7f35b40b0ee0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b970 tx=0x7f35ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:41.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35aaffd640 1 -- 172.21.15.125:0/345993854 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00cb40 con 0x7f35b415b2a0 2023-10-18T11:41:41.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35aaffd640 1 -- 172.21.15.125:0/345993854 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f35ac00cdb0 con 0x7f35b415b2a0 2023-10-18T11:41:41.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35aaffd640 1 -- 172.21.15.125:0/345993854 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0146a0 con 0x7f35b415b2a0 2023-10-18T11:41:41.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b40c8730 con 0x7f35b415b2a0 2023-10-18T11:41:41.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.601+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b41518d0 con 0x7f35b415b2a0 2023-10-18T11:41:41.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.602+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b40c8c50 con 0x7f35b415b2a0 2023-10-18T11:41:41.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.602+0000 7f35aaffd640 1 -- 172.21.15.125:0/345993854 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f35ac01d3b0 con 0x7f35b415b2a0 2023-10-18T11:41:41.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.603+0000 7f35aaffd640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f359806af60 0x7f359806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.603+0000 7f35aaffd640 1 -- 172.21.15.125:0/345993854 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f35ac089b00 con 0x7f35b415b2a0 2023-10-18T11:41:41.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.603+0000 7f35ba49c640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f359806af60 0x7f359806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.604+0000 7f35ba49c640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f359806af60 0x7f359806d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35b0035000 tx=0x7f35b0004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:41.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.607+0000 7f35aaffd640 1 -- 172.21.15.125:0/345993854 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35b40c8c50 con 0x7f35b415b2a0 2023-10-18T11:41:41.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.772+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35b40c8c50 con 0x7f35b415b2a0 2023-10-18T11:41:41.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.773+0000 7f35aaffd640 1 -- 172.21.15.125:0/345993854 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+16816 (secure 0 0 0) 0x7f35b40c8c50 con 0x7f35b415b2a0 2023-10-18T11:41:41.776 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:41.776 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":174,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:39.819661+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:41:35.970906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-18T11:41:37.131199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:41.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f359806af60 msgr2=0x7f359806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f359806af60 0x7f359806d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35b0035000 tx=0x7f35b0004730 comp rx=0 tx=0).stop 2023-10-18T11:41:41.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b415b2a0 msgr2=0x7f35b40b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b415b2a0 0x7f35b40b0ee0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b970 tx=0x7f35ac00be30 comp rx=0 tx=0).stop 2023-10-18T11:41:41.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 shutdown_connections 2023-10-18T11:41:41.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f359806af60 0x7f359806d420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b415b2a0 0x7f35b40b0ee0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35b40cf340 0x7f35b40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 --2- 172.21.15.125:0/345993854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35b40ceb40 0x7f35b4151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 >> 172.21.15.125:0/345993854 conn(0x7f35b40d0d60 msgr2=0x7f35b40cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:41.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 shutdown_connections 2023-10-18T11:41:41.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.775+0000 7f35bc700640 1 -- 172.21.15.125:0/345993854 wait complete. 2023-10-18T11:41:41.788 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T11:41:41.789 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:41:41.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.890+0000 7f5691997640 1 -- 172.21.15.125:0/2747429116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 msgr2=0x7f5674004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.890+0000 7f5691997640 1 --2- 172.21.15.125:0/2747429116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 0x7f5674004860 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f568c15b6a0 tx=0x7f56740057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.890+0000 7f568b7fe640 1 -- 172.21.15.125:0/2747429116 reap_dead start 2023-10-18T11:41:41.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.891+0000 7f5691997640 1 -- 172.21.15.125:0/2747429116 shutdown_connections 2023-10-18T11:41:41.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.891+0000 7f5691997640 1 -- 172.21.15.125:0/2747429116 >> 172.21.15.125:0/2747429116 conn(0x7f568c0d0d60 msgr2=0x7f568c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:41.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.891+0000 7f5691997640 1 -- 172.21.15.125:0/2747429116 shutdown_connections 2023-10-18T11:41:41.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.891+0000 7f5691997640 1 -- 172.21.15.125:0/2747429116 wait complete. 2023-10-18T11:41:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.892+0000 7f5691997640 1 Processor -- start 2023-10-18T11:41:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.892+0000 7f5691997640 1 -- start start 2023-10-18T11:41:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.892+0000 7f5691997640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 0x7f568c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.892+0000 7f5691997640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568c0b5150 0x7f568c06a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.892+0000 7f5691997640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f568c156810 0x7f568c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.892+0000 7f5691997640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c15b7c0 con 0x7f5674004390 2023-10-18T11:41:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f5691997640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0c91b0 con 0x7f568c0b5150 2023-10-18T11:41:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f5691997640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0c9490 con 0x7f568c156810 2023-10-18T11:41:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 0x7f568c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 0x7f568c0c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56584/0 (socket says 172.21.15.125:56584) 2023-10-18T11:41:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 -- 172.21.15.125:0/769556662 learned_addr learned my addr 172.21.15.125:0/769556662 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568b7fe640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f568c156810 0x7f568c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568a7fc640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568c0b5150 0x7f568c06a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 -- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f568c156810 msgr2=0x7f568c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f568c156810 0x7f568c06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 -- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568c0b5150 msgr2=0x7f568c06a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:41.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568c0b5150 0x7f568c06a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:41.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 -- 172.21.15.125:0/769556662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0c9770 con 0x7f5674004390 2023-10-18T11:41:41.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568b7fe640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f568c156810 0x7f568c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:41.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568a7fc640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568c0b5150 0x7f568c06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:41.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f568affd640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 0x7f568c0c8840 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f567c010bf0 tx=0x7f567c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:41.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f5673fff640 1 -- 172.21.15.125:0/769556662 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c003b70 con 0x7f5674004390 2023-10-18T11:41:41.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f5673fff640 1 -- 172.21.15.125:0/769556662 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f567c015070 con 0x7f5674004390 2023-10-18T11:41:41.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f5673fff640 1 -- 172.21.15.125:0/769556662 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0042b0 con 0x7f5674004390 2023-10-18T11:41:41.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c15b9b0 con 0x7f5674004390 2023-10-18T11:41:41.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.893+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c0c8d80 con 0x7f5674004390 2023-10-18T11:41:41.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.894+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c0690e0 con 0x7f5674004390 2023-10-18T11:41:41.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.895+0000 7f5673fff640 1 -- 172.21.15.125:0/769556662 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f567c004410 con 0x7f5674004390 2023-10-18T11:41:41.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.895+0000 7f5673fff640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f567806af90 0x7f567806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:41.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.895+0000 7f5673fff640 1 -- 172.21.15.125:0/769556662 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f567c017070 con 0x7f5674004390 2023-10-18T11:41:41.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.895+0000 7f568a7fc640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f567806af90 0x7f567806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:41.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.896+0000 7f568a7fc640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f567806af90 0x7f567806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5680034000 tx=0x7f5680037040 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:41.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:41.899+0000 7f5673fff640 1 -- 172.21.15.125:0/769556662 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f567c0557e0 con 0x7f5674004390 2023-10-18T11:41:42.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.061+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 --> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f568c0b57e0 con 0x7f567806af90 2023-10-18T11:41:42.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.069+0000 7f5673fff640 1 -- 172.21.15.125:0/769556662 <== mgr.8644 v2:172.21.15.139:6832/4236255547 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278480 (secure 0 0 0) 0x7f568c0b57e0 con 0x7f567806af90 2023-10-18T11:41:42.074 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:42.076 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:41:42.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f567806af90 msgr2=0x7f567806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f567806af90 0x7f567806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5680034000 tx=0x7f5680037040 comp rx=0 tx=0).stop 2023-10-18T11:41:42.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 msgr2=0x7f568c0c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 0x7f568c0c8840 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f567c010bf0 tx=0x7f567c00e720 comp rx=0 tx=0).stop 2023-10-18T11:41:42.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 shutdown_connections 2023-10-18T11:41:42.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f567806af90 0x7f567806d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568c0b5150 0x7f568c06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f568c156810 0x7f568c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 --2- 172.21.15.125:0/769556662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5674004390 0x7f568c0c8840 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 >> 172.21.15.125:0/769556662 conn(0x7f568c0d0d60 msgr2=0x7f568c0aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:42.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 shutdown_connections 2023-10-18T11:41:42.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.072+0000 7f5691997640 1 -- 172.21.15.125:0/769556662 wait complete. 2023-10-18T11:41:42.086 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T11:41:40.431885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":474,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":385256,"kb_used_data":17828,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754589464,"statfs":{"total":773094113280,"available":772699611136,"internally_reserved":0,"allocated":18255872,"data_stored":9821435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000903"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829496+0000","last_change":"2023-10-18T11:41:38.825716+0000","last_active":"2023-10-18T11:41:38.829496+0000","last_peered":"2023-10-18T11:41:38.829496+0000","last_clean":"2023-10-18T11:41:38.829496+0000","last_became_active":"2023-10-18T11:41:38.825425+0000","last_became_peered":"2023-10-18T11:41:38.825425+0000","last_unstale":"2023-10-18T11:41:38.829496+0000","last_undegraded":"2023-10-18T11:41:38.829496+0000","last_fullsized":"2023-10-18T11:41:38.829496+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:39:43.232981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.833625+0000","last_change":"2023-10-18T11:41:38.826683+0000","last_active":"2023-10-18T11:41:38.833625+0000","last_peered":"2023-10-18T11:41:38.833625+0000","last_clean":"2023-10-18T11:41:38.833625+0000","last_became_active":"2023-10-18T11:41:38.826277+0000","last_became_peered":"2023-10-18T11:41:38.826277+0000","last_unstale":"2023-10-18T11:41:38.833625+0000","last_undegraded":"2023-10-18T11:41:38.833625+0000","last_fullsized":"2023-10-18T11:41:38.833625+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:09.384565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.840839+0000","last_change":"2023-10-18T11:41:38.837755+0000","last_active":"2023-10-18T11:41:38.840839+0000","last_peered":"2023-10-18T11:41:38.840839+0000","last_clean":"2023-10-18T11:41:38.840839+0000","last_became_active":"2023-10-18T11:41:38.837504+0000","last_became_peered":"2023-10-18T11:41:38.837504+0000","last_unstale":"2023-10-18T11:41:38.840839+0000","last_undegraded":"2023-10-18T11:41:38.840839+0000","last_fullsized":"2023-10-18T11:41:38.840839+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:15:30.360143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831198+0000","last_change":"2023-10-18T11:41:38.826514+0000","last_active":"2023-10-18T11:41:38.831198+0000","last_peered":"2023-10-18T11:41:38.831198+0000","last_clean":"2023-10-18T11:41:38.831198+0000","last_became_active":"2023-10-18T11:41:38.826186+0000","last_became_peered":"2023-10-18T11:41:38.826186+0000","last_unstale":"2023-10-18T11:41:38.831198+0000","last_undegraded":"2023-10-18T11:41:38.831198+0000","last_fullsized":"2023-10-18T11:41:38.831198+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:10.683364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.825812+0000","last_change":"2023-10-18T11:41:38.820452+0000","last_active":"2023-10-18T11:41:38.825812+0000","last_peered":"2023-10-18T11:41:38.825812+0000","last_clean":"2023-10-18T11:41:38.825812+0000","last_became_active":"2023-10-18T11:41:38.820159+0000","last_became_peered":"2023-10-18T11:41:38.820159+0000","last_unstale":"2023-10-18T11:41:38.825812+0000","last_undegraded":"2023-10-18T11:41:38.825812+0000","last_fullsized":"2023-10-18T11:41:38.825812+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:30.211327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829649+0000","last_change":"2023-10-18T11:41:38.823794+0000","last_active":"2023-10-18T11:41:38.829649+0000","last_peered":"2023-10-18T11:41:38.829649+0000","last_clean":"2023-10-18T11:41:38.829649+0000","last_became_active":"2023-10-18T11:41:38.823494+0000","last_became_peered":"2023-10-18T11:41:38.823494+0000","last_unstale":"2023-10-18T11:41:38.829649+0000","last_undegraded":"2023-10-18T11:41:38.829649+0000","last_fullsized":"2023-10-18T11:41:38.829649+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:52.835599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.832555+0000","last_change":"2023-10-18T11:41:38.824262+0000","last_active":"2023-10-18T11:41:38.832555+0000","last_peered":"2023-10-18T11:41:38.832555+0000","last_clean":"2023-10-18T11:41:38.832555+0000","last_became_active":"2023-10-18T11:41:38.824005+0000","last_became_peered":"2023-10-18T11:41:38.824005+0000","last_unstale":"2023-10-18T11:41:38.832555+0000","last_undegraded":"2023-10-18T11:41:38.832555+0000","last_fullsized":"2023-10-18T11:41:38.832555+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:34.843533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.815029+0000","last_change":"2023-10-18T11:41:37.820269+0000","last_active":"2023-10-18T11:41:38.815029+0000","last_peered":"2023-10-18T11:41:38.815029+0000","last_clean":"2023-10-18T11:41:38.815029+0000","last_became_active":"2023-10-18T11:41:37.819988+0000","last_became_peered":"2023-10-18T11:41:37.819988+0000","last_unstale":"2023-10-18T11:41:38.815029+0000","last_undegraded":"2023-10-18T11:41:38.815029+0000","last_fullsized":"2023-10-18T11:41:38.815029+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:53.297750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829428+0000","last_change":"2023-10-18T11:41:38.820999+0000","last_active":"2023-10-18T11:41:38.829428+0000","last_peered":"2023-10-18T11:41:38.829428+0000","last_clean":"2023-10-18T11:41:38.829428+0000","last_became_active":"2023-10-18T11:41:38.820731+0000","last_became_peered":"2023-10-18T11:41:38.820731+0000","last_unstale":"2023-10-18T11:41:38.829428+0000","last_undegraded":"2023-10-18T11:41:38.829428+0000","last_fullsized":"2023-10-18T11:41:38.829428+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:12.937570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.816460+0000","last_change":"2023-10-18T11:41:37.814357+0000","last_active":"2023-10-18T11:41:38.816460+0000","last_peered":"2023-10-18T11:41:38.816460+0000","last_clean":"2023-10-18T11:41:38.816460+0000","last_became_active":"2023-10-18T11:41:37.814082+0000","last_became_peered":"2023-10-18T11:41:37.814082+0000","last_unstale":"2023-10-18T11:41:38.816460+0000","last_undegraded":"2023-10-18T11:41:38.816460+0000","last_fullsized":"2023-10-18T11:41:38.816460+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:41.460284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"173'8","reported_seq":19,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844653+0000","last_change":"2023-10-18T11:41:37.822141+0000","last_active":"2023-10-18T11:41:38.844653+0000","last_peered":"2023-10-18T11:41:38.844653+0000","last_clean":"2023-10-18T11:41:38.844653+0000","last_became_active":"2023-10-18T11:41:37.821819+0000","last_became_peered":"2023-10-18T11:41:37.821819+0000","last_unstale":"2023-10-18T11:41:38.844653+0000","last_undegraded":"2023-10-18T11:41:38.844653+0000","last_fullsized":"2023-10-18T11:41:38.844653+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:19.314971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829630+0000","last_change":"2023-10-18T11:41:38.822398+0000","last_active":"2023-10-18T11:41:38.829630+0000","last_peered":"2023-10-18T11:41:38.829630+0000","last_clean":"2023-10-18T11:41:38.829630+0000","last_became_active":"2023-10-18T11:41:38.822156+0000","last_became_peered":"2023-10-18T11:41:38.822156+0000","last_unstale":"2023-10-18T11:41:38.829630+0000","last_undegraded":"2023-10-18T11:41:38.829630+0000","last_fullsized":"2023-10-18T11:41:38.829630+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:17.259402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835456+0000","last_change":"2023-10-18T11:41:38.833975+0000","last_active":"2023-10-18T11:41:38.835456+0000","last_peered":"2023-10-18T11:41:38.835456+0000","last_clean":"2023-10-18T11:41:38.835456+0000","last_became_active":"2023-10-18T11:41:38.833708+0000","last_became_peered":"2023-10-18T11:41:38.833708+0000","last_unstale":"2023-10-18T11:41:38.835456+0000","last_undegraded":"2023-10-18T11:41:38.835456+0000","last_fullsized":"2023-10-18T11:41:38.835456+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:13.225614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.846834+0000","last_change":"2023-10-18T11:41:37.822283+0000","last_active":"2023-10-18T11:41:38.846834+0000","last_peered":"2023-10-18T11:41:38.846834+0000","last_clean":"2023-10-18T11:41:38.846834+0000","last_became_active":"2023-10-18T11:41:37.822019+0000","last_became_peered":"2023-10-18T11:41:37.822019+0000","last_unstale":"2023-10-18T11:41:38.846834+0000","last_undegraded":"2023-10-18T11:41:38.846834+0000","last_fullsized":"2023-10-18T11:41:38.846834+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:27.346874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.817051+0000","last_change":"2023-10-18T11:41:37.819830+0000","last_active":"2023-10-18T11:41:38.817051+0000","last_peered":"2023-10-18T11:41:38.817051+0000","last_clean":"2023-10-18T11:41:38.817051+0000","last_became_active":"2023-10-18T11:41:37.819576+0000","last_became_peered":"2023-10-18T11:41:37.819576+0000","last_unstale":"2023-10-18T11:41:38.817051+0000","last_undegraded":"2023-10-18T11:41:38.817051+0000","last_fullsized":"2023-10-18T11:41:38.817051+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:54.501911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.816866+0000","last_change":"2023-10-18T11:41:37.823322+0000","last_active":"2023-10-18T11:41:38.816866+0000","last_peered":"2023-10-18T11:41:38.816866+0000","last_clean":"2023-10-18T11:41:38.816866+0000","last_became_active":"2023-10-18T11:41:37.822856+0000","last_became_peered":"2023-10-18T11:41:37.822856+0000","last_unstale":"2023-10-18T11:41:38.816866+0000","last_undegraded":"2023-10-18T11:41:38.816866+0000","last_fullsized":"2023-10-18T11:41:38.816866+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:52.515374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831189+0000","last_change":"2023-10-18T11:41:38.827748+0000","last_active":"2023-10-18T11:41:38.831189+0000","last_peered":"2023-10-18T11:41:38.831189+0000","last_clean":"2023-10-18T11:41:38.831189+0000","last_became_active":"2023-10-18T11:41:38.827469+0000","last_became_peered":"2023-10-18T11:41:38.827469+0000","last_unstale":"2023-10-18T11:41:38.831189+0000","last_undegraded":"2023-10-18T11:41:38.831189+0000","last_fullsized":"2023-10-18T11:41:38.831189+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:09.333393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.814926+0000","last_change":"2023-10-18T11:41:37.824985+0000","last_active":"2023-10-18T11:41:38.814926+0000","last_peered":"2023-10-18T11:41:38.814926+0000","last_clean":"2023-10-18T11:41:38.814926+0000","last_became_active":"2023-10-18T11:41:37.824481+0000","last_became_peered":"2023-10-18T11:41:37.824481+0000","last_unstale":"2023-10-18T11:41:38.814926+0000","last_undegraded":"2023-10-18T11:41:38.814926+0000","last_fullsized":"2023-10-18T11:41:38.814926+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:39.668889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.815291+0000","last_change":"2023-10-18T11:41:37.814973+0000","last_active":"2023-10-18T11:41:38.815291+0000","last_peered":"2023-10-18T11:41:38.815291+0000","last_clean":"2023-10-18T11:41:38.815291+0000","last_became_active":"2023-10-18T11:41:37.814699+0000","last_became_peered":"2023-10-18T11:41:37.814699+0000","last_unstale":"2023-10-18T11:41:38.815291+0000","last_undegraded":"2023-10-18T11:41:38.815291+0000","last_fullsized":"2023-10-18T11:41:38.815291+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:19.194946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"173'6","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.848425+0000","last_change":"2023-10-18T11:41:37.820513+0000","last_active":"2023-10-18T11:41:38.848425+0000","last_peered":"2023-10-18T11:41:38.848425+0000","last_clean":"2023-10-18T11:41:38.848425+0000","last_became_active":"2023-10-18T11:41:37.820194+0000","last_became_peered":"2023-10-18T11:41:37.820194+0000","last_unstale":"2023-10-18T11:41:38.848425+0000","last_undegraded":"2023-10-18T11:41:38.848425+0000","last_fullsized":"2023-10-18T11:41:38.848425+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:58.848022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"162'280","reported_seq":908,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-18T11:41:33.756629+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:41:33.756629+0000","last_peered":"2023-10-18T11:41:33.756629+0000","last_clean":"2023-10-18T11:41:33.756629+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:41:33.756629+0000","last_undegraded":"2023-10-18T11:41:33.756629+0000","last_fullsized":"2023-10-18T11:41:33.756629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":2},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":18,"acting":18,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575767,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51880,"kb_used_data":2644,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319960,"statfs":{"total":96636764160,"available":96583639040,"internally_reserved":0,"allocated":2707456,"data_stored":1661416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.442},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.55200000000000005}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66700000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.65500000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37412,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334428,"statfs":{"total":96636764160,"available":96598454272,"internally_reserved":0,"allocated":2113536,"data_stored":1071048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.499}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.39600000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.65400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575768,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59660,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312180,"statfs":{"total":96636764160,"available":96575672320,"internally_reserved":0,"allocated":2162688,"data_stored":1096994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.54200000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.86199999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.84599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41584,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330256,"statfs":{"total":96636764160,"available":96594182144,"internally_reserved":0,"allocated":2125824,"data_stored":1077083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.27800000000000002},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56599999999999995}]},{"osd":1,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.626,"15min":0.623},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.86399999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.626},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84599999999999997,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.376}]},{"osd":6,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.30499999999999999},"max":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.33000000000000002},"max":{"1min":0.77000000000000002,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.42099999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29999999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58899999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.36199999999999999},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.58999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59612,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312228,"statfs":{"total":96636764160,"available":96575721472,"internally_reserved":0,"allocated":2113536,"data_stored":1071048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.25600000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.317},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.33200000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.436}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.48899999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.39700000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.374},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59624,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312216,"statfs":{"total":96636764160,"available":96575709184,"internally_reserved":0,"allocated":2125824,"data_stored":1077083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.311},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40100000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.28899999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.625}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.56299999999999994}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37428,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2125824,"data_stored":1077083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.374}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.313}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.56399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575767,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2781184,"data_stored":1689680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:41:42.093 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:41:42.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.225+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3774327696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc02c003b30 msgr2=0x7fc02c003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.225+0000 7fc04a5b6640 1 --2- 172.21.15.125:0/3774327696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc02c003b30 0x7fc02c003f30 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fc034001800 tx=0x7fc02c0057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.225+0000 7fc048b53640 1 -- 172.21.15.125:0/3774327696 reap_dead start 2023-10-18T11:41:42.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.226+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3774327696 shutdown_connections 2023-10-18T11:41:42.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.226+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3774327696 >> 172.21.15.125:0/3774327696 conn(0x7fc044069930 msgr2=0x7fc044069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:42.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.226+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3774327696 shutdown_connections 2023-10-18T11:41:42.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.226+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3774327696 wait complete. 2023-10-18T11:41:42.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.227+0000 7fc04a5b6640 1 Processor -- start 2023-10-18T11:41:42.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.227+0000 7fc04a5b6640 1 -- start start 2023-10-18T11:41:42.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc04a5b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc02c003b30 0x7fc0440cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc04a5b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc044135450 0x7fc0441294e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc04a5b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc044070a50 0x7fc04412c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc04a5b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440caf90 con 0x7fc044070a50 2023-10-18T11:41:42.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc04a5b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440ac380 con 0x7fc02c003b30 2023-10-18T11:41:42.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc04a5b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc044129a20 con 0x7fc044135450 2023-10-18T11:41:42.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc044070a50 0x7fc04412c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc044070a50 0x7fc04412c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56608/0 (socket says 172.21.15.125:56608) 2023-10-18T11:41:42.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 -- 172.21.15.125:0/3781449947 learned_addr learned my addr 172.21.15.125:0/3781449947 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:42.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc0437fe640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc044135450 0x7fc0441294e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc043fff640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc02c003b30 0x7fc0440cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 -- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc044135450 msgr2=0x7fc0441294e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc044135450 0x7fc0441294e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 -- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc02c003b30 msgr2=0x7fc0440cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc02c003b30 0x7fc0440cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 -- 172.21.15.125:0/3781449947 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440ca910 con 0x7fc044070a50 2023-10-18T11:41:42.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc048b53640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc044070a50 0x7fc04412c160 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fc0440cc510 tx=0x7fc03c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:42.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc040ff9640 1 -- 172.21.15.125:0/3781449947 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00cc20 con 0x7fc044070a50 2023-10-18T11:41:42.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc040ff9640 1 -- 172.21.15.125:0/3781449947 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc03c00ce70 con 0x7fc044070a50 2023-10-18T11:41:42.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc044135930 con 0x7fc044070a50 2023-10-18T11:41:42.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc040ff9640 1 -- 172.21.15.125:0/3781449947 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0116a0 con 0x7fc044070a50 2023-10-18T11:41:42.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.228+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc044135c10 con 0x7fc044070a50 2023-10-18T11:41:42.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.229+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0440cb0d0 con 0x7fc044070a50 2023-10-18T11:41:42.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.230+0000 7fc040ff9640 1 -- 172.21.15.125:0/3781449947 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc03c011800 con 0x7fc044070a50 2023-10-18T11:41:42.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.230+0000 7fc040ff9640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc03006af60 0x7fc03006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.230+0000 7fc040ff9640 1 -- 172.21.15.125:0/3781449947 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fc03c08a3d0 con 0x7fc044070a50 2023-10-18T11:41:42.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.230+0000 7fc043fff640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc03006af60 0x7fc03006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.231+0000 7fc043fff640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc03006af60 0x7fc03006d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc034034000 tx=0x7fc034039040 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:42.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.234+0000 7fc040ff9640 1 -- 172.21.15.125:0/3781449947 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0440cb0d0 con 0x7fc044070a50 2023-10-18T11:41:42.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.395+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 --> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0440cbd60 con 0x7fc03006af60 2023-10-18T11:41:42.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.403+0000 7fc040ff9640 1 -- 172.21.15.125:0/3781449947 <== mgr.8644 v2:172.21.15.139:6832/4236255547 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278480 (secure 0 0 0) 0x7fc034006160 con 0x7fc03006af60 2023-10-18T11:41:42.408 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:42.409 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:41:42.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc03006af60 msgr2=0x7fc03006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc03006af60 0x7fc03006d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc034034000 tx=0x7fc034039040 comp rx=0 tx=0).stop 2023-10-18T11:41:42.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc044070a50 msgr2=0x7fc04412c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc044070a50 0x7fc04412c160 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fc0440cc510 tx=0x7fc03c00ed30 comp rx=0 tx=0).stop 2023-10-18T11:41:42.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 shutdown_connections 2023-10-18T11:41:42.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc03006af60 0x7fc03006d420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc044070a50 0x7fc04412c160 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc044135450 0x7fc0441294e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 --2- 172.21.15.125:0/3781449947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc02c003b30 0x7fc0440cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 >> 172.21.15.125:0/3781449947 conn(0x7fc044069930 msgr2=0x7fc044159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:42.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 shutdown_connections 2023-10-18T11:41:42.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.406+0000 7fc04a5b6640 1 -- 172.21.15.125:0/3781449947 wait complete. 2023-10-18T11:41:42.419 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T11:41:40.431885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":474,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":385256,"kb_used_data":17828,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754589464,"statfs":{"total":773094113280,"available":772699611136,"internally_reserved":0,"allocated":18255872,"data_stored":9821435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000903"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829496+0000","last_change":"2023-10-18T11:41:38.825716+0000","last_active":"2023-10-18T11:41:38.829496+0000","last_peered":"2023-10-18T11:41:38.829496+0000","last_clean":"2023-10-18T11:41:38.829496+0000","last_became_active":"2023-10-18T11:41:38.825425+0000","last_became_peered":"2023-10-18T11:41:38.825425+0000","last_unstale":"2023-10-18T11:41:38.829496+0000","last_undegraded":"2023-10-18T11:41:38.829496+0000","last_fullsized":"2023-10-18T11:41:38.829496+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:39:43.232981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.833625+0000","last_change":"2023-10-18T11:41:38.826683+0000","last_active":"2023-10-18T11:41:38.833625+0000","last_peered":"2023-10-18T11:41:38.833625+0000","last_clean":"2023-10-18T11:41:38.833625+0000","last_became_active":"2023-10-18T11:41:38.826277+0000","last_became_peered":"2023-10-18T11:41:38.826277+0000","last_unstale":"2023-10-18T11:41:38.833625+0000","last_undegraded":"2023-10-18T11:41:38.833625+0000","last_fullsized":"2023-10-18T11:41:38.833625+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:09.384565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.840839+0000","last_change":"2023-10-18T11:41:38.837755+0000","last_active":"2023-10-18T11:41:38.840839+0000","last_peered":"2023-10-18T11:41:38.840839+0000","last_clean":"2023-10-18T11:41:38.840839+0000","last_became_active":"2023-10-18T11:41:38.837504+0000","last_became_peered":"2023-10-18T11:41:38.837504+0000","last_unstale":"2023-10-18T11:41:38.840839+0000","last_undegraded":"2023-10-18T11:41:38.840839+0000","last_fullsized":"2023-10-18T11:41:38.840839+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:15:30.360143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831198+0000","last_change":"2023-10-18T11:41:38.826514+0000","last_active":"2023-10-18T11:41:38.831198+0000","last_peered":"2023-10-18T11:41:38.831198+0000","last_clean":"2023-10-18T11:41:38.831198+0000","last_became_active":"2023-10-18T11:41:38.826186+0000","last_became_peered":"2023-10-18T11:41:38.826186+0000","last_unstale":"2023-10-18T11:41:38.831198+0000","last_undegraded":"2023-10-18T11:41:38.831198+0000","last_fullsized":"2023-10-18T11:41:38.831198+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:10.683364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.825812+0000","last_change":"2023-10-18T11:41:38.820452+0000","last_active":"2023-10-18T11:41:38.825812+0000","last_peered":"2023-10-18T11:41:38.825812+0000","last_clean":"2023-10-18T11:41:38.825812+0000","last_became_active":"2023-10-18T11:41:38.820159+0000","last_became_peered":"2023-10-18T11:41:38.820159+0000","last_unstale":"2023-10-18T11:41:38.825812+0000","last_undegraded":"2023-10-18T11:41:38.825812+0000","last_fullsized":"2023-10-18T11:41:38.825812+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:30.211327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829649+0000","last_change":"2023-10-18T11:41:38.823794+0000","last_active":"2023-10-18T11:41:38.829649+0000","last_peered":"2023-10-18T11:41:38.829649+0000","last_clean":"2023-10-18T11:41:38.829649+0000","last_became_active":"2023-10-18T11:41:38.823494+0000","last_became_peered":"2023-10-18T11:41:38.823494+0000","last_unstale":"2023-10-18T11:41:38.829649+0000","last_undegraded":"2023-10-18T11:41:38.829649+0000","last_fullsized":"2023-10-18T11:41:38.829649+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:52.835599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.832555+0000","last_change":"2023-10-18T11:41:38.824262+0000","last_active":"2023-10-18T11:41:38.832555+0000","last_peered":"2023-10-18T11:41:38.832555+0000","last_clean":"2023-10-18T11:41:38.832555+0000","last_became_active":"2023-10-18T11:41:38.824005+0000","last_became_peered":"2023-10-18T11:41:38.824005+0000","last_unstale":"2023-10-18T11:41:38.832555+0000","last_undegraded":"2023-10-18T11:41:38.832555+0000","last_fullsized":"2023-10-18T11:41:38.832555+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:34.843533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.815029+0000","last_change":"2023-10-18T11:41:37.820269+0000","last_active":"2023-10-18T11:41:38.815029+0000","last_peered":"2023-10-18T11:41:38.815029+0000","last_clean":"2023-10-18T11:41:38.815029+0000","last_became_active":"2023-10-18T11:41:37.819988+0000","last_became_peered":"2023-10-18T11:41:37.819988+0000","last_unstale":"2023-10-18T11:41:38.815029+0000","last_undegraded":"2023-10-18T11:41:38.815029+0000","last_fullsized":"2023-10-18T11:41:38.815029+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:53.297750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829428+0000","last_change":"2023-10-18T11:41:38.820999+0000","last_active":"2023-10-18T11:41:38.829428+0000","last_peered":"2023-10-18T11:41:38.829428+0000","last_clean":"2023-10-18T11:41:38.829428+0000","last_became_active":"2023-10-18T11:41:38.820731+0000","last_became_peered":"2023-10-18T11:41:38.820731+0000","last_unstale":"2023-10-18T11:41:38.829428+0000","last_undegraded":"2023-10-18T11:41:38.829428+0000","last_fullsized":"2023-10-18T11:41:38.829428+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:12.937570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.816460+0000","last_change":"2023-10-18T11:41:37.814357+0000","last_active":"2023-10-18T11:41:38.816460+0000","last_peered":"2023-10-18T11:41:38.816460+0000","last_clean":"2023-10-18T11:41:38.816460+0000","last_became_active":"2023-10-18T11:41:37.814082+0000","last_became_peered":"2023-10-18T11:41:37.814082+0000","last_unstale":"2023-10-18T11:41:38.816460+0000","last_undegraded":"2023-10-18T11:41:38.816460+0000","last_fullsized":"2023-10-18T11:41:38.816460+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:41.460284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"173'8","reported_seq":19,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844653+0000","last_change":"2023-10-18T11:41:37.822141+0000","last_active":"2023-10-18T11:41:38.844653+0000","last_peered":"2023-10-18T11:41:38.844653+0000","last_clean":"2023-10-18T11:41:38.844653+0000","last_became_active":"2023-10-18T11:41:37.821819+0000","last_became_peered":"2023-10-18T11:41:37.821819+0000","last_unstale":"2023-10-18T11:41:38.844653+0000","last_undegraded":"2023-10-18T11:41:38.844653+0000","last_fullsized":"2023-10-18T11:41:38.844653+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:19.314971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829630+0000","last_change":"2023-10-18T11:41:38.822398+0000","last_active":"2023-10-18T11:41:38.829630+0000","last_peered":"2023-10-18T11:41:38.829630+0000","last_clean":"2023-10-18T11:41:38.829630+0000","last_became_active":"2023-10-18T11:41:38.822156+0000","last_became_peered":"2023-10-18T11:41:38.822156+0000","last_unstale":"2023-10-18T11:41:38.829630+0000","last_undegraded":"2023-10-18T11:41:38.829630+0000","last_fullsized":"2023-10-18T11:41:38.829630+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:17.259402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835456+0000","last_change":"2023-10-18T11:41:38.833975+0000","last_active":"2023-10-18T11:41:38.835456+0000","last_peered":"2023-10-18T11:41:38.835456+0000","last_clean":"2023-10-18T11:41:38.835456+0000","last_became_active":"2023-10-18T11:41:38.833708+0000","last_became_peered":"2023-10-18T11:41:38.833708+0000","last_unstale":"2023-10-18T11:41:38.835456+0000","last_undegraded":"2023-10-18T11:41:38.835456+0000","last_fullsized":"2023-10-18T11:41:38.835456+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:13.225614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.846834+0000","last_change":"2023-10-18T11:41:37.822283+0000","last_active":"2023-10-18T11:41:38.846834+0000","last_peered":"2023-10-18T11:41:38.846834+0000","last_clean":"2023-10-18T11:41:38.846834+0000","last_became_active":"2023-10-18T11:41:37.822019+0000","last_became_peered":"2023-10-18T11:41:37.822019+0000","last_unstale":"2023-10-18T11:41:38.846834+0000","last_undegraded":"2023-10-18T11:41:38.846834+0000","last_fullsized":"2023-10-18T11:41:38.846834+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:27.346874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.817051+0000","last_change":"2023-10-18T11:41:37.819830+0000","last_active":"2023-10-18T11:41:38.817051+0000","last_peered":"2023-10-18T11:41:38.817051+0000","last_clean":"2023-10-18T11:41:38.817051+0000","last_became_active":"2023-10-18T11:41:37.819576+0000","last_became_peered":"2023-10-18T11:41:37.819576+0000","last_unstale":"2023-10-18T11:41:38.817051+0000","last_undegraded":"2023-10-18T11:41:38.817051+0000","last_fullsized":"2023-10-18T11:41:38.817051+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:54.501911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.816866+0000","last_change":"2023-10-18T11:41:37.823322+0000","last_active":"2023-10-18T11:41:38.816866+0000","last_peered":"2023-10-18T11:41:38.816866+0000","last_clean":"2023-10-18T11:41:38.816866+0000","last_became_active":"2023-10-18T11:41:37.822856+0000","last_became_peered":"2023-10-18T11:41:37.822856+0000","last_unstale":"2023-10-18T11:41:38.816866+0000","last_undegraded":"2023-10-18T11:41:38.816866+0000","last_fullsized":"2023-10-18T11:41:38.816866+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:52.515374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831189+0000","last_change":"2023-10-18T11:41:38.827748+0000","last_active":"2023-10-18T11:41:38.831189+0000","last_peered":"2023-10-18T11:41:38.831189+0000","last_clean":"2023-10-18T11:41:38.831189+0000","last_became_active":"2023-10-18T11:41:38.827469+0000","last_became_peered":"2023-10-18T11:41:38.827469+0000","last_unstale":"2023-10-18T11:41:38.831189+0000","last_undegraded":"2023-10-18T11:41:38.831189+0000","last_fullsized":"2023-10-18T11:41:38.831189+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:09.333393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.814926+0000","last_change":"2023-10-18T11:41:37.824985+0000","last_active":"2023-10-18T11:41:38.814926+0000","last_peered":"2023-10-18T11:41:38.814926+0000","last_clean":"2023-10-18T11:41:38.814926+0000","last_became_active":"2023-10-18T11:41:37.824481+0000","last_became_peered":"2023-10-18T11:41:37.824481+0000","last_unstale":"2023-10-18T11:41:38.814926+0000","last_undegraded":"2023-10-18T11:41:38.814926+0000","last_fullsized":"2023-10-18T11:41:38.814926+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:39.668889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.815291+0000","last_change":"2023-10-18T11:41:37.814973+0000","last_active":"2023-10-18T11:41:38.815291+0000","last_peered":"2023-10-18T11:41:38.815291+0000","last_clean":"2023-10-18T11:41:38.815291+0000","last_became_active":"2023-10-18T11:41:37.814699+0000","last_became_peered":"2023-10-18T11:41:37.814699+0000","last_unstale":"2023-10-18T11:41:38.815291+0000","last_undegraded":"2023-10-18T11:41:38.815291+0000","last_fullsized":"2023-10-18T11:41:38.815291+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:19.194946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:36.796826+0000","last_change":"2023-10-18T11:41:36.796826+0000","last_active":"2023-10-18T11:41:36.796826+0000","last_peered":"2023-10-18T11:41:36.796826+0000","last_clean":"2023-10-18T11:41:36.796826+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:36.796826+0000","last_undegraded":"2023-10-18T11:41:36.796826+0000","last_fullsized":"2023-10-18T11:41:36.796826+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:41:37.801652+0000","last_change":"2023-10-18T11:41:37.801652+0000","last_active":"2023-10-18T11:41:37.801652+0000","last_peered":"2023-10-18T11:41:37.801652+0000","last_clean":"2023-10-18T11:41:37.801652+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:41:37.801652+0000","last_undegraded":"2023-10-18T11:41:37.801652+0000","last_fullsized":"2023-10-18T11:41:37.801652+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"173'6","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.848425+0000","last_change":"2023-10-18T11:41:37.820513+0000","last_active":"2023-10-18T11:41:38.848425+0000","last_peered":"2023-10-18T11:41:38.848425+0000","last_clean":"2023-10-18T11:41:38.848425+0000","last_became_active":"2023-10-18T11:41:37.820194+0000","last_became_peered":"2023-10-18T11:41:37.820194+0000","last_unstale":"2023-10-18T11:41:38.848425+0000","last_undegraded":"2023-10-18T11:41:38.848425+0000","last_fullsized":"2023-10-18T11:41:38.848425+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:58.848022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"162'280","reported_seq":908,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-18T11:41:33.756629+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:41:33.756629+0000","last_peered":"2023-10-18T11:41:33.756629+0000","last_clean":"2023-10-18T11:41:33.756629+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:41:33.756629+0000","last_undegraded":"2023-10-18T11:41:33.756629+0000","last_fullsized":"2023-10-18T11:41:33.756629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":2},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":18,"acting":18,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575767,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51880,"kb_used_data":2644,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319960,"statfs":{"total":96636764160,"available":96583639040,"internally_reserved":0,"allocated":2707456,"data_stored":1661416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.442},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.55200000000000005}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66700000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.65500000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37412,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334428,"statfs":{"total":96636764160,"available":96598454272,"internally_reserved":0,"allocated":2113536,"data_stored":1071048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.499}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.39600000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.65400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575768,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59660,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312180,"statfs":{"total":96636764160,"available":96575672320,"internally_reserved":0,"allocated":2162688,"data_stored":1096994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.54200000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.86199999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.84599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41584,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330256,"statfs":{"total":96636764160,"available":96594182144,"internally_reserved":0,"allocated":2125824,"data_stored":1077083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.27800000000000002},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56599999999999995}]},{"osd":1,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.626,"15min":0.623},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.86399999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.626},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84599999999999997,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.376}]},{"osd":6,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.30499999999999999},"max":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.33000000000000002},"max":{"1min":0.77000000000000002,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.42099999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29999999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58899999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.36199999999999999},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.58999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59612,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312228,"statfs":{"total":96636764160,"available":96575721472,"internally_reserved":0,"allocated":2113536,"data_stored":1071048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.25600000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.317},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.33200000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.436}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.48899999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.39700000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.374},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59624,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312216,"statfs":{"total":96636764160,"available":96575709184,"internally_reserved":0,"allocated":2125824,"data_stored":1077083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.311},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40100000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.28899999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.625}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.56299999999999994}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37428,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2125824,"data_stored":1077083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.374}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.313}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.56399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575767,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2781184,"data_stored":1689680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:41:42.425 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:41:42.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.559+0000 7f2a1aa94640 1 -- 172.21.15.125:0/4279624404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a000047c0 msgr2=0x7f2a00004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.559+0000 7f2a1aa94640 1 --2- 172.21.15.125:0/4279624404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a000047c0 0x7f2a00004c40 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009b50 tx=0x7f2a000057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.559+0000 7f2a19031640 1 -- 172.21.15.125:0/4279624404 reap_dead start 2023-10-18T11:41:42.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.559+0000 7f2a1aa94640 1 -- 172.21.15.125:0/4279624404 shutdown_connections 2023-10-18T11:41:42.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.559+0000 7f2a1aa94640 1 -- 172.21.15.125:0/4279624404 >> 172.21.15.125:0/4279624404 conn(0x7f2a14069930 msgr2=0x7f2a14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:42.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.560+0000 7f2a1aa94640 1 -- 172.21.15.125:0/4279624404 shutdown_connections 2023-10-18T11:41:42.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.560+0000 7f2a1aa94640 1 -- 172.21.15.125:0/4279624404 wait complete. 2023-10-18T11:41:42.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.561+0000 7f2a1aa94640 1 Processor -- start 2023-10-18T11:41:42.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.561+0000 7f2a1aa94640 1 -- start start 2023-10-18T11:41:42.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a1aa94640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a000047c0 0x7f2a1413c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a1aa94640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a140b10d0 0x7f2a14135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a1aa94640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a140b1550 0x7f2a141364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a1aa94640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a141541c0 con 0x7f2a140b10d0 2023-10-18T11:41:42.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a1aa94640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a141545e0 con 0x7f2a140b1550 2023-10-18T11:41:42.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a1aa94640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a14153540 con 0x7f2a000047c0 2023-10-18T11:41:42.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a140b10d0 0x7f2a14135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a140b10d0 0x7f2a14135f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56620/0 (socket says 172.21.15.125:56620) 2023-10-18T11:41:42.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 -- 172.21.15.125:0/2987860753 learned_addr learned my addr 172.21.15.125:0/2987860753 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:42.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a18830640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a000047c0 0x7f2a1413c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a19031640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a140b1550 0x7f2a141364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 -- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a000047c0 msgr2=0x7f2a1413c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a000047c0 0x7f2a1413c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 -- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a140b1550 msgr2=0x7f2a141364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a140b1550 0x7f2a141364c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 -- 172.21.15.125:0/2987860753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14153820 con 0x7f2a140b10d0 2023-10-18T11:41:42.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a13fff640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a140b10d0 0x7f2a14135f80 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009c70 tx=0x7f29fc033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:42.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a117fa640 1 -- 172.21.15.125:0/2987860753 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc004440 con 0x7f2a140b10d0 2023-10-18T11:41:42.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a117fa640 1 -- 172.21.15.125:0/2987860753 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29fc033e60 con 0x7f2a140b10d0 2023-10-18T11:41:42.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a14153b00 con 0x7f2a140b10d0 2023-10-18T11:41:42.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a117fa640 1 -- 172.21.15.125:0/2987860753 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc003850 con 0x7f2a140b10d0 2023-10-18T11:41:42.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.562+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a14153160 con 0x7f2a140b10d0 2023-10-18T11:41:42.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.563+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a141532f0 con 0x7f2a140b10d0 2023-10-18T11:41:42.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.564+0000 7f2a117fa640 1 -- 172.21.15.125:0/2987860753 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f29fc0039b0 con 0x7f2a140b10d0 2023-10-18T11:41:42.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.564+0000 7f2a117fa640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f29f406af90 0x7f29f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.564+0000 7f2a117fa640 1 -- 172.21.15.125:0/2987860753 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f29fc0ac600 con 0x7f2a140b10d0 2023-10-18T11:41:42.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.564+0000 7f2a18830640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f29f406af90 0x7f29f406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.565+0000 7f2a18830640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f29f406af90 0x7f29f406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009b10 tx=0x7f2a04005f90 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:42.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.568+0000 7f2a117fa640 1 -- 172.21.15.125:0/2987860753 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29fc0773c0 con 0x7f2a140b10d0 2023-10-18T11:41:42.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.731+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 --> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a1415c1e0 con 0x7f29f406af90 2023-10-18T11:41:42.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.738+0000 7f2a117fa640 1 -- 172.21.15.125:0/2987860753 <== mgr.8644 v2:172.21.15.139:6832/4236255547 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290319 (secure 0 0 0) 0x7f2a04013bd0 con 0x7f29f406af90 2023-10-18T11:41:42.744 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:42.746 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:41:42.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f29f406af90 msgr2=0x7f29f406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f29f406af90 0x7f29f406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009b10 tx=0x7f2a04005f90 comp rx=0 tx=0).stop 2023-10-18T11:41:42.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a140b10d0 msgr2=0x7f2a14135f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a140b10d0 0x7f2a14135f80 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009c70 tx=0x7f29fc033c30 comp rx=0 tx=0).stop 2023-10-18T11:41:42.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 shutdown_connections 2023-10-18T11:41:42.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f29f406af90 0x7f29f406d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a140b1550 0x7f2a141364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a140b10d0 0x7f2a14135f80 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 --2- 172.21.15.125:0/2987860753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a000047c0 0x7f2a1413c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.741+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 >> 172.21.15.125:0/2987860753 conn(0x7f2a14069930 msgr2=0x7f2a1414e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:42.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.742+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 shutdown_connections 2023-10-18T11:41:42.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.742+0000 7f2a1aa94640 1 -- 172.21.15.125:0/2987860753 wait complete. 2023-10-18T11:41:42.754 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T11:41:42.432325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":499,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385560,"kb_used_data":18144,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754589160,"statfs":{"total":773094113280,"available":772699299840,"internally_reserved":0,"allocated":18579456,"data_stored":9974804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.629045"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829496+0000","last_change":"2023-10-18T11:41:38.825716+0000","last_active":"2023-10-18T11:41:38.829496+0000","last_peered":"2023-10-18T11:41:38.829496+0000","last_clean":"2023-10-18T11:41:38.829496+0000","last_became_active":"2023-10-18T11:41:38.825425+0000","last_became_peered":"2023-10-18T11:41:38.825425+0000","last_unstale":"2023-10-18T11:41:38.829496+0000","last_undegraded":"2023-10-18T11:41:38.829496+0000","last_fullsized":"2023-10-18T11:41:38.829496+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:39:43.232981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841612+0000","last_change":"2023-10-18T11:41:38.837347+0000","last_active":"2023-10-18T11:41:38.841612+0000","last_peered":"2023-10-18T11:41:38.841612+0000","last_clean":"2023-10-18T11:41:38.841612+0000","last_became_active":"2023-10-18T11:41:38.837097+0000","last_became_peered":"2023-10-18T11:41:38.837097+0000","last_unstale":"2023-10-18T11:41:38.841612+0000","last_undegraded":"2023-10-18T11:41:38.841612+0000","last_fullsized":"2023-10-18T11:41:38.841612+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:40.673714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.842484+0000","last_change":"2023-10-18T11:41:38.840032+0000","last_active":"2023-10-18T11:41:38.842484+0000","last_peered":"2023-10-18T11:41:38.842484+0000","last_clean":"2023-10-18T11:41:38.842484+0000","last_became_active":"2023-10-18T11:41:38.838657+0000","last_became_peered":"2023-10-18T11:41:38.838657+0000","last_unstale":"2023-10-18T11:41:38.842484+0000","last_undegraded":"2023-10-18T11:41:38.842484+0000","last_fullsized":"2023-10-18T11:41:38.842484+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:29.239361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.834357+0000","last_change":"2023-10-18T11:41:38.832579+0000","last_active":"2023-10-18T11:41:38.834357+0000","last_peered":"2023-10-18T11:41:38.834357+0000","last_clean":"2023-10-18T11:41:38.834357+0000","last_became_active":"2023-10-18T11:41:38.832320+0000","last_became_peered":"2023-10-18T11:41:38.832320+0000","last_unstale":"2023-10-18T11:41:38.834357+0000","last_undegraded":"2023-10-18T11:41:38.834357+0000","last_fullsized":"2023-10-18T11:41:38.834357+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:12:30.419426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.843919+0000","last_change":"2023-10-18T11:41:38.840547+0000","last_active":"2023-10-18T11:41:38.843919+0000","last_peered":"2023-10-18T11:41:38.843919+0000","last_clean":"2023-10-18T11:41:38.843919+0000","last_became_active":"2023-10-18T11:41:38.840290+0000","last_became_peered":"2023-10-18T11:41:38.840290+0000","last_unstale":"2023-10-18T11:41:38.843919+0000","last_undegraded":"2023-10-18T11:41:38.843919+0000","last_fullsized":"2023-10-18T11:41:38.843919+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:37.966595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841820+0000","last_change":"2023-10-18T11:41:38.839747+0000","last_active":"2023-10-18T11:41:38.841820+0000","last_peered":"2023-10-18T11:41:38.841820+0000","last_clean":"2023-10-18T11:41:38.841820+0000","last_became_active":"2023-10-18T11:41:38.839470+0000","last_became_peered":"2023-10-18T11:41:38.839470+0000","last_unstale":"2023-10-18T11:41:38.841820+0000","last_undegraded":"2023-10-18T11:41:38.841820+0000","last_fullsized":"2023-10-18T11:41:38.841820+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:25.523639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.833625+0000","last_change":"2023-10-18T11:41:38.826683+0000","last_active":"2023-10-18T11:41:38.833625+0000","last_peered":"2023-10-18T11:41:38.833625+0000","last_clean":"2023-10-18T11:41:38.833625+0000","last_became_active":"2023-10-18T11:41:38.826277+0000","last_became_peered":"2023-10-18T11:41:38.826277+0000","last_unstale":"2023-10-18T11:41:38.833625+0000","last_undegraded":"2023-10-18T11:41:38.833625+0000","last_fullsized":"2023-10-18T11:41:38.833625+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:09.384565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829657+0000","last_change":"2023-10-18T11:41:38.828519+0000","last_active":"2023-10-18T11:41:39.829657+0000","last_peered":"2023-10-18T11:41:39.829657+0000","last_clean":"2023-10-18T11:41:39.829657+0000","last_became_active":"2023-10-18T11:41:38.828256+0000","last_became_peered":"2023-10-18T11:41:38.828256+0000","last_unstale":"2023-10-18T11:41:39.829657+0000","last_undegraded":"2023-10-18T11:41:39.829657+0000","last_fullsized":"2023-10-18T11:41:39.829657+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:18.914354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841743+0000","last_change":"2023-10-18T11:41:38.836612+0000","last_active":"2023-10-18T11:41:38.841743+0000","last_peered":"2023-10-18T11:41:38.841743+0000","last_clean":"2023-10-18T11:41:38.841743+0000","last_became_active":"2023-10-18T11:41:38.836342+0000","last_became_peered":"2023-10-18T11:41:38.836342+0000","last_unstale":"2023-10-18T11:41:38.841743+0000","last_undegraded":"2023-10-18T11:41:38.841743+0000","last_fullsized":"2023-10-18T11:41:38.841743+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:36.452850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.825014+0000","last_change":"2023-10-18T11:41:38.822440+0000","last_active":"2023-10-18T11:41:38.825014+0000","last_peered":"2023-10-18T11:41:38.825014+0000","last_clean":"2023-10-18T11:41:38.825014+0000","last_became_active":"2023-10-18T11:41:38.822164+0000","last_became_peered":"2023-10-18T11:41:38.822164+0000","last_unstale":"2023-10-18T11:41:38.825014+0000","last_undegraded":"2023-10-18T11:41:38.825014+0000","last_fullsized":"2023-10-18T11:41:38.825014+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:43.202732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828721+0000","last_change":"2023-10-18T11:41:38.832961+0000","last_active":"2023-10-18T11:41:39.828721+0000","last_peered":"2023-10-18T11:41:39.828721+0000","last_clean":"2023-10-18T11:41:39.828721+0000","last_became_active":"2023-10-18T11:41:38.832373+0000","last_became_peered":"2023-10-18T11:41:38.832373+0000","last_unstale":"2023-10-18T11:41:39.828721+0000","last_undegraded":"2023-10-18T11:41:39.828721+0000","last_fullsized":"2023-10-18T11:41:39.828721+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:16.135065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.834340+0000","last_change":"2023-10-18T11:41:38.830562+0000","last_active":"2023-10-18T11:41:38.834340+0000","last_peered":"2023-10-18T11:41:38.834340+0000","last_clean":"2023-10-18T11:41:38.834340+0000","last_became_active":"2023-10-18T11:41:38.830272+0000","last_became_peered":"2023-10-18T11:41:38.830272+0000","last_unstale":"2023-10-18T11:41:38.834340+0000","last_undegraded":"2023-10-18T11:41:38.834340+0000","last_fullsized":"2023-10-18T11:41:38.834340+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:29.879841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835196+0000","last_change":"2023-10-18T11:41:38.831348+0000","last_active":"2023-10-18T11:41:38.835196+0000","last_peered":"2023-10-18T11:41:38.835196+0000","last_clean":"2023-10-18T11:41:38.835196+0000","last_became_active":"2023-10-18T11:41:38.826783+0000","last_became_peered":"2023-10-18T11:41:38.826783+0000","last_unstale":"2023-10-18T11:41:38.835196+0000","last_undegraded":"2023-10-18T11:41:38.835196+0000","last_fullsized":"2023-10-18T11:41:38.835196+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:15:13.102514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.834516+0000","last_change":"2023-10-18T11:41:38.830730+0000","last_active":"2023-10-18T11:41:38.834516+0000","last_peered":"2023-10-18T11:41:38.834516+0000","last_clean":"2023-10-18T11:41:38.834516+0000","last_became_active":"2023-10-18T11:41:38.830409+0000","last_became_peered":"2023-10-18T11:41:38.830409+0000","last_unstale":"2023-10-18T11:41:38.834516+0000","last_undegraded":"2023-10-18T11:41:38.834516+0000","last_fullsized":"2023-10-18T11:41:38.834516+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:01.324176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841760+0000","last_change":"2023-10-18T11:41:38.836640+0000","last_active":"2023-10-18T11:41:38.841760+0000","last_peered":"2023-10-18T11:41:38.841760+0000","last_clean":"2023-10-18T11:41:38.841760+0000","last_became_active":"2023-10-18T11:41:38.835870+0000","last_became_peered":"2023-10-18T11:41:38.835870+0000","last_unstale":"2023-10-18T11:41:38.841760+0000","last_undegraded":"2023-10-18T11:41:38.841760+0000","last_fullsized":"2023-10-18T11:41:38.841760+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:02.224387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835247+0000","last_change":"2023-10-18T11:41:38.831324+0000","last_active":"2023-10-18T11:41:38.835247+0000","last_peered":"2023-10-18T11:41:38.835247+0000","last_clean":"2023-10-18T11:41:38.835247+0000","last_became_active":"2023-10-18T11:41:38.831052+0000","last_became_peered":"2023-10-18T11:41:38.831052+0000","last_unstale":"2023-10-18T11:41:38.835247+0000","last_undegraded":"2023-10-18T11:41:38.835247+0000","last_fullsized":"2023-10-18T11:41:38.835247+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:10.646615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.836918+0000","last_change":"2023-10-18T11:41:38.834503+0000","last_active":"2023-10-18T11:41:38.836918+0000","last_peered":"2023-10-18T11:41:38.836918+0000","last_clean":"2023-10-18T11:41:38.836918+0000","last_became_active":"2023-10-18T11:41:38.834215+0000","last_became_peered":"2023-10-18T11:41:38.834215+0000","last_unstale":"2023-10-18T11:41:38.836918+0000","last_undegraded":"2023-10-18T11:41:38.836918+0000","last_fullsized":"2023-10-18T11:41:38.836918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:22:04.684733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828696+0000","last_change":"2023-10-18T11:41:38.827931+0000","last_active":"2023-10-18T11:41:39.828696+0000","last_peered":"2023-10-18T11:41:39.828696+0000","last_clean":"2023-10-18T11:41:39.828696+0000","last_became_active":"2023-10-18T11:41:38.827669+0000","last_became_peered":"2023-10-18T11:41:38.827669+0000","last_unstale":"2023-10-18T11:41:39.828696+0000","last_undegraded":"2023-10-18T11:41:39.828696+0000","last_fullsized":"2023-10-18T11:41:39.828696+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:03.238765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835680+0000","last_change":"2023-10-18T11:41:38.832584+0000","last_active":"2023-10-18T11:41:38.835680+0000","last_peered":"2023-10-18T11:41:38.835680+0000","last_clean":"2023-10-18T11:41:38.835680+0000","last_became_active":"2023-10-18T11:41:38.832073+0000","last_became_peered":"2023-10-18T11:41:38.832073+0000","last_unstale":"2023-10-18T11:41:38.835680+0000","last_undegraded":"2023-10-18T11:41:38.835680+0000","last_fullsized":"2023-10-18T11:41:38.835680+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:51.765003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.840839+0000","last_change":"2023-10-18T11:41:38.837755+0000","last_active":"2023-10-18T11:41:38.840839+0000","last_peered":"2023-10-18T11:41:38.840839+0000","last_clean":"2023-10-18T11:41:38.840839+0000","last_became_active":"2023-10-18T11:41:38.837504+0000","last_became_peered":"2023-10-18T11:41:38.837504+0000","last_unstale":"2023-10-18T11:41:38.840839+0000","last_undegraded":"2023-10-18T11:41:38.840839+0000","last_fullsized":"2023-10-18T11:41:38.840839+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:15:30.360143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835616+0000","last_change":"2023-10-18T11:41:38.832688+0000","last_active":"2023-10-18T11:41:38.835616+0000","last_peered":"2023-10-18T11:41:38.835616+0000","last_clean":"2023-10-18T11:41:38.835616+0000","last_became_active":"2023-10-18T11:41:38.832343+0000","last_became_peered":"2023-10-18T11:41:38.832343+0000","last_unstale":"2023-10-18T11:41:38.835616+0000","last_undegraded":"2023-10-18T11:41:38.835616+0000","last_fullsized":"2023-10-18T11:41:38.835616+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:27.983985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831198+0000","last_change":"2023-10-18T11:41:38.826514+0000","last_active":"2023-10-18T11:41:38.831198+0000","last_peered":"2023-10-18T11:41:38.831198+0000","last_clean":"2023-10-18T11:41:38.831198+0000","last_became_active":"2023-10-18T11:41:38.826186+0000","last_became_peered":"2023-10-18T11:41:38.826186+0000","last_unstale":"2023-10-18T11:41:38.831198+0000","last_undegraded":"2023-10-18T11:41:38.831198+0000","last_fullsized":"2023-10-18T11:41:38.831198+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:10.683364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829945+0000","last_change":"2023-10-18T11:41:38.826932+0000","last_active":"2023-10-18T11:41:39.829945+0000","last_peered":"2023-10-18T11:41:39.829945+0000","last_clean":"2023-10-18T11:41:39.829945+0000","last_became_active":"2023-10-18T11:41:38.826645+0000","last_became_peered":"2023-10-18T11:41:38.826645+0000","last_unstale":"2023-10-18T11:41:39.829945+0000","last_undegraded":"2023-10-18T11:41:39.829945+0000","last_fullsized":"2023-10-18T11:41:39.829945+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:54:15.505963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828515+0000","last_change":"2023-10-18T11:41:38.827801+0000","last_active":"2023-10-18T11:41:39.828515+0000","last_peered":"2023-10-18T11:41:39.828515+0000","last_clean":"2023-10-18T11:41:39.828515+0000","last_became_active":"2023-10-18T11:41:38.827514+0000","last_became_peered":"2023-10-18T11:41:38.827514+0000","last_unstale":"2023-10-18T11:41:39.828515+0000","last_undegraded":"2023-10-18T11:41:39.828515+0000","last_fullsized":"2023-10-18T11:41:39.828515+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:05:57.235594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.825812+0000","last_change":"2023-10-18T11:41:38.820452+0000","last_active":"2023-10-18T11:41:38.825812+0000","last_peered":"2023-10-18T11:41:38.825812+0000","last_clean":"2023-10-18T11:41:38.825812+0000","last_became_active":"2023-10-18T11:41:38.820159+0000","last_became_peered":"2023-10-18T11:41:38.820159+0000","last_unstale":"2023-10-18T11:41:38.825812+0000","last_undegraded":"2023-10-18T11:41:38.825812+0000","last_fullsized":"2023-10-18T11:41:38.825812+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:30.211327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829649+0000","last_change":"2023-10-18T11:41:38.823794+0000","last_active":"2023-10-18T11:41:38.829649+0000","last_peered":"2023-10-18T11:41:38.829649+0000","last_clean":"2023-10-18T11:41:38.829649+0000","last_became_active":"2023-10-18T11:41:38.823494+0000","last_became_peered":"2023-10-18T11:41:38.823494+0000","last_unstale":"2023-10-18T11:41:38.829649+0000","last_undegraded":"2023-10-18T11:41:38.829649+0000","last_fullsized":"2023-10-18T11:41:38.829649+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:52.835599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.830854+0000","last_change":"2023-10-18T11:41:38.826372+0000","last_active":"2023-10-18T11:41:38.830854+0000","last_peered":"2023-10-18T11:41:38.830854+0000","last_clean":"2023-10-18T11:41:38.830854+0000","last_became_active":"2023-10-18T11:41:38.826126+0000","last_became_peered":"2023-10-18T11:41:38.826126+0000","last_unstale":"2023-10-18T11:41:38.830854+0000","last_undegraded":"2023-10-18T11:41:38.830854+0000","last_fullsized":"2023-10-18T11:41:38.830854+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:49.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828583+0000","last_change":"2023-10-18T11:41:38.823012+0000","last_active":"2023-10-18T11:41:39.828583+0000","last_peered":"2023-10-18T11:41:39.828583+0000","last_clean":"2023-10-18T11:41:39.828583+0000","last_became_active":"2023-10-18T11:41:38.822261+0000","last_became_peered":"2023-10-18T11:41:38.822261+0000","last_unstale":"2023-10-18T11:41:39.828583+0000","last_undegraded":"2023-10-18T11:41:39.828583+0000","last_fullsized":"2023-10-18T11:41:39.828583+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:36.619175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829996+0000","last_change":"2023-10-18T11:41:38.827149+0000","last_active":"2023-10-18T11:41:39.829996+0000","last_peered":"2023-10-18T11:41:39.829996+0000","last_clean":"2023-10-18T11:41:39.829996+0000","last_became_active":"2023-10-18T11:41:38.825505+0000","last_became_peered":"2023-10-18T11:41:38.825505+0000","last_unstale":"2023-10-18T11:41:39.829996+0000","last_undegraded":"2023-10-18T11:41:39.829996+0000","last_fullsized":"2023-10-18T11:41:39.829996+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:06.291395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828541+0000","last_change":"2023-10-18T11:41:38.837766+0000","last_active":"2023-10-18T11:41:39.828541+0000","last_peered":"2023-10-18T11:41:39.828541+0000","last_clean":"2023-10-18T11:41:39.828541+0000","last_became_active":"2023-10-18T11:41:38.837482+0000","last_became_peered":"2023-10-18T11:41:38.837482+0000","last_unstale":"2023-10-18T11:41:39.828541+0000","last_undegraded":"2023-10-18T11:41:39.828541+0000","last_fullsized":"2023-10-18T11:41:39.828541+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:05.028643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835514+0000","last_change":"2023-10-18T11:41:38.831958+0000","last_active":"2023-10-18T11:41:38.835514+0000","last_peered":"2023-10-18T11:41:38.835514+0000","last_clean":"2023-10-18T11:41:38.835514+0000","last_became_active":"2023-10-18T11:41:38.831705+0000","last_became_peered":"2023-10-18T11:41:38.831705+0000","last_unstale":"2023-10-18T11:41:38.835514+0000","last_undegraded":"2023-10-18T11:41:38.835514+0000","last_fullsized":"2023-10-18T11:41:38.835514+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:42.346674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829267+0000","last_change":"2023-10-18T11:41:38.827986+0000","last_active":"2023-10-18T11:41:39.829267+0000","last_peered":"2023-10-18T11:41:39.829267+0000","last_clean":"2023-10-18T11:41:39.829267+0000","last_became_active":"2023-10-18T11:41:38.827661+0000","last_became_peered":"2023-10-18T11:41:38.827661+0000","last_unstale":"2023-10-18T11:41:39.829267+0000","last_undegraded":"2023-10-18T11:41:39.829267+0000","last_fullsized":"2023-10-18T11:41:39.829267+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:51.388492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841809+0000","last_change":"2023-10-18T11:41:38.837107+0000","last_active":"2023-10-18T11:41:38.841809+0000","last_peered":"2023-10-18T11:41:38.841809+0000","last_clean":"2023-10-18T11:41:38.841809+0000","last_became_active":"2023-10-18T11:41:38.836829+0000","last_became_peered":"2023-10-18T11:41:38.836829+0000","last_unstale":"2023-10-18T11:41:38.841809+0000","last_undegraded":"2023-10-18T11:41:38.841809+0000","last_fullsized":"2023-10-18T11:41:38.841809+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:21:03.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"173'4","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.845429+0000","last_change":"2023-10-18T11:41:37.820263+0000","last_active":"2023-10-18T11:41:38.845429+0000","last_peered":"2023-10-18T11:41:38.845429+0000","last_clean":"2023-10-18T11:41:38.845429+0000","last_became_active":"2023-10-18T11:41:37.819933+0000","last_became_peered":"2023-10-18T11:41:37.819933+0000","last_unstale":"2023-10-18T11:41:38.845429+0000","last_undegraded":"2023-10-18T11:41:38.845429+0000","last_fullsized":"2023-10-18T11:41:38.845429+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:41:57.852127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.842464+0000","last_change":"2023-10-18T11:41:38.839828+0000","last_active":"2023-10-18T11:41:38.842464+0000","last_peered":"2023-10-18T11:41:38.842464+0000","last_clean":"2023-10-18T11:41:38.842464+0000","last_became_active":"2023-10-18T11:41:38.838892+0000","last_became_peered":"2023-10-18T11:41:38.838892+0000","last_unstale":"2023-10-18T11:41:38.842464+0000","last_undegraded":"2023-10-18T11:41:38.842464+0000","last_fullsized":"2023-10-18T11:41:38.842464+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:35.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"173'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844405+0000","last_change":"2023-10-18T11:41:37.821129+0000","last_active":"2023-10-18T11:41:38.844405+0000","last_peered":"2023-10-18T11:41:38.844405+0000","last_clean":"2023-10-18T11:41:38.844405+0000","last_became_active":"2023-10-18T11:41:37.820850+0000","last_became_peered":"2023-10-18T11:41:37.820850+0000","last_unstale":"2023-10-18T11:41:38.844405+0000","last_undegraded":"2023-10-18T11:41:38.844405+0000","last_fullsized":"2023-10-18T11:41:38.844405+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:09.641790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829709+0000","last_change":"2023-10-18T11:41:38.831858+0000","last_active":"2023-10-18T11:41:39.829709+0000","last_peered":"2023-10-18T11:41:39.829709+0000","last_clean":"2023-10-18T11:41:39.829709+0000","last_became_active":"2023-10-18T11:41:38.831590+0000","last_became_peered":"2023-10-18T11:41:38.831590+0000","last_unstale":"2023-10-18T11:41:39.829709+0000","last_undegraded":"2023-10-18T11:41:39.829709+0000","last_fullsized":"2023-10-18T11:41:39.829709+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:42.177687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844505+0000","last_change":"2023-10-18T11:41:37.821038+0000","last_active":"2023-10-18T11:41:38.844505+0000","last_peered":"2023-10-18T11:41:38.844505+0000","last_clean":"2023-10-18T11:41:38.844505+0000","last_became_active":"2023-10-18T11:41:37.820759+0000","last_became_peered":"2023-10-18T11:41:37.820759+0000","last_unstale":"2023-10-18T11:41:38.844505+0000","last_undegraded":"2023-10-18T11:41:38.844505+0000","last_fullsized":"2023-10-18T11:41:38.844505+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:48.941548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.833804+0000","last_change":"2023-10-18T11:41:38.830270+0000","last_active":"2023-10-18T11:41:38.833804+0000","last_peered":"2023-10-18T11:41:38.833804+0000","last_clean":"2023-10-18T11:41:38.833804+0000","last_became_active":"2023-10-18T11:41:38.830001+0000","last_became_peered":"2023-10-18T11:41:38.830001+0000","last_unstale":"2023-10-18T11:41:38.833804+0000","last_undegraded":"2023-10-18T11:41:38.833804+0000","last_fullsized":"2023-10-18T11:41:38.833804+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:43.202732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.839014+0000","last_change":"2023-10-18T11:41:37.819635+0000","last_active":"2023-10-18T11:41:38.839014+0000","last_peered":"2023-10-18T11:41:38.839014+0000","last_clean":"2023-10-18T11:41:38.839014+0000","last_became_active":"2023-10-18T11:41:37.819375+0000","last_became_peered":"2023-10-18T11:41:37.819375+0000","last_unstale":"2023-10-18T11:41:38.839014+0000","last_undegraded":"2023-10-18T11:41:38.839014+0000","last_fullsized":"2023-10-18T11:41:38.839014+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:51.830773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.832555+0000","last_change":"2023-10-18T11:41:38.824262+0000","last_active":"2023-10-18T11:41:38.832555+0000","last_peered":"2023-10-18T11:41:38.832555+0000","last_clean":"2023-10-18T11:41:38.832555+0000","last_became_active":"2023-10-18T11:41:38.824005+0000","last_became_peered":"2023-10-18T11:41:38.824005+0000","last_unstale":"2023-10-18T11:41:38.832555+0000","last_undegraded":"2023-10-18T11:41:38.832555+0000","last_fullsized":"2023-10-18T11:41:38.832555+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:34.843533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"173'1","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.830055+0000","last_change":"2023-10-18T11:41:37.826154+0000","last_active":"2023-10-18T11:41:39.830055+0000","last_peered":"2023-10-18T11:41:39.830055+0000","last_clean":"2023-10-18T11:41:39.830055+0000","last_became_active":"2023-10-18T11:41:37.825607+0000","last_became_peered":"2023-10-18T11:41:37.825607+0000","last_unstale":"2023-10-18T11:41:39.830055+0000","last_undegraded":"2023-10-18T11:41:39.830055+0000","last_fullsized":"2023-10-18T11:41:39.830055+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:24.888182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.836951+0000","last_change":"2023-10-18T11:41:38.834243+0000","last_active":"2023-10-18T11:41:38.836951+0000","last_peered":"2023-10-18T11:41:38.836951+0000","last_clean":"2023-10-18T11:41:38.836951+0000","last_became_active":"2023-10-18T11:41:38.833958+0000","last_became_peered":"2023-10-18T11:41:38.833958+0000","last_unstale":"2023-10-18T11:41:38.836951+0000","last_undegraded":"2023-10-18T11:41:38.836951+0000","last_fullsized":"2023-10-18T11:41:38.836951+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:49.235985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"173'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.846614+0000","last_change":"2023-10-18T11:41:37.824548+0000","last_active":"2023-10-18T11:41:38.846614+0000","last_peered":"2023-10-18T11:41:38.846614+0000","last_clean":"2023-10-18T11:41:38.846614+0000","last_became_active":"2023-10-18T11:41:37.824278+0000","last_became_peered":"2023-10-18T11:41:37.824278+0000","last_unstale":"2023-10-18T11:41:38.846614+0000","last_undegraded":"2023-10-18T11:41:38.846614+0000","last_fullsized":"2023-10-18T11:41:38.846614+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:05.442472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.842326+0000","last_change":"2023-10-18T11:41:38.838537+0000","last_active":"2023-10-18T11:41:38.842326+0000","last_peered":"2023-10-18T11:41:38.842326+0000","last_clean":"2023-10-18T11:41:38.842326+0000","last_became_active":"2023-10-18T11:41:38.835109+0000","last_became_peered":"2023-10-18T11:41:38.835109+0000","last_unstale":"2023-10-18T11:41:38.842326+0000","last_undegraded":"2023-10-18T11:41:38.842326+0000","last_fullsized":"2023-10-18T11:41:38.842326+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:12.937570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.815029+0000","last_change":"2023-10-18T11:41:37.820269+0000","last_active":"2023-10-18T11:41:38.815029+0000","last_peered":"2023-10-18T11:41:38.815029+0000","last_clean":"2023-10-18T11:41:38.815029+0000","last_became_active":"2023-10-18T11:41:37.819988+0000","last_became_peered":"2023-10-18T11:41:37.819988+0000","last_unstale":"2023-10-18T11:41:38.815029+0000","last_undegraded":"2023-10-18T11:41:38.815029+0000","last_fullsized":"2023-10-18T11:41:38.815029+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:53.297750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829428+0000","last_change":"2023-10-18T11:41:38.820999+0000","last_active":"2023-10-18T11:41:38.829428+0000","last_peered":"2023-10-18T11:41:38.829428+0000","last_clean":"2023-10-18T11:41:38.829428+0000","last_became_active":"2023-10-18T11:41:38.820731+0000","last_became_peered":"2023-10-18T11:41:38.820731+0000","last_unstale":"2023-10-18T11:41:38.829428+0000","last_undegraded":"2023-10-18T11:41:38.829428+0000","last_fullsized":"2023-10-18T11:41:38.829428+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:12.937570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.818927+0000","last_change":"2023-10-18T11:41:37.824809+0000","last_active":"2023-10-18T11:41:38.818927+0000","last_peered":"2023-10-18T11:41:38.818927+0000","last_clean":"2023-10-18T11:41:38.818927+0000","last_became_active":"2023-10-18T11:41:37.824509+0000","last_became_peered":"2023-10-18T11:41:37.824509+0000","last_unstale":"2023-10-18T11:41:38.818927+0000","last_undegraded":"2023-10-18T11:41:38.818927+0000","last_fullsized":"2023-10-18T11:41:38.818927+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:50.760177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829761+0000","last_change":"2023-10-18T11:41:38.827847+0000","last_active":"2023-10-18T11:41:39.829761+0000","last_peered":"2023-10-18T11:41:39.829761+0000","last_clean":"2023-10-18T11:41:39.829761+0000","last_became_active":"2023-10-18T11:41:38.827553+0000","last_became_peered":"2023-10-18T11:41:38.827553+0000","last_unstale":"2023-10-18T11:41:39.829761+0000","last_undegraded":"2023-10-18T11:41:39.829761+0000","last_fullsized":"2023-10-18T11:41:39.829761+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:54:44.886016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.822854+0000","last_change":"2023-10-18T11:41:37.825094+0000","last_active":"2023-10-18T11:41:38.822854+0000","last_peered":"2023-10-18T11:41:38.822854+0000","last_clean":"2023-10-18T11:41:38.822854+0000","last_became_active":"2023-10-18T11:41:37.824553+0000","last_became_peered":"2023-10-18T11:41:37.824553+0000","last_unstale":"2023-10-18T11:41:38.822854+0000","last_undegraded":"2023-10-18T11:41:38.822854+0000","last_fullsized":"2023-10-18T11:41:38.822854+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:36.961770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828711+0000","last_change":"2023-10-18T11:41:38.833135+0000","last_active":"2023-10-18T11:41:39.828711+0000","last_peered":"2023-10-18T11:41:39.828711+0000","last_clean":"2023-10-18T11:41:39.828711+0000","last_became_active":"2023-10-18T11:41:38.832708+0000","last_became_peered":"2023-10-18T11:41:38.832708+0000","last_unstale":"2023-10-18T11:41:39.828711+0000","last_undegraded":"2023-10-18T11:41:39.828711+0000","last_fullsized":"2023-10-18T11:41:39.828711+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:39.928349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.816460+0000","last_change":"2023-10-18T11:41:37.814357+0000","last_active":"2023-10-18T11:41:38.816460+0000","last_peered":"2023-10-18T11:41:38.816460+0000","last_clean":"2023-10-18T11:41:38.816460+0000","last_became_active":"2023-10-18T11:41:37.814082+0000","last_became_peered":"2023-10-18T11:41:37.814082+0000","last_unstale":"2023-10-18T11:41:38.816460+0000","last_undegraded":"2023-10-18T11:41:38.816460+0000","last_fullsized":"2023-10-18T11:41:38.816460+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:41.460284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828728+0000","last_change":"2023-10-18T11:41:38.827430+0000","last_active":"2023-10-18T11:41:39.828728+0000","last_peered":"2023-10-18T11:41:39.828728+0000","last_clean":"2023-10-18T11:41:39.828728+0000","last_became_active":"2023-10-18T11:41:38.827151+0000","last_became_peered":"2023-10-18T11:41:38.827151+0000","last_unstale":"2023-10-18T11:41:39.828728+0000","last_undegraded":"2023-10-18T11:41:39.828728+0000","last_fullsized":"2023-10-18T11:41:39.828728+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:57.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"173'3","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828816+0000","last_change":"2023-10-18T11:41:37.816938+0000","last_active":"2023-10-18T11:41:39.828816+0000","last_peered":"2023-10-18T11:41:39.828816+0000","last_clean":"2023-10-18T11:41:39.828816+0000","last_became_active":"2023-10-18T11:41:37.816710+0000","last_became_peered":"2023-10-18T11:41:37.816710+0000","last_unstale":"2023-10-18T11:41:39.828816+0000","last_undegraded":"2023-10-18T11:41:39.828816+0000","last_fullsized":"2023-10-18T11:41:39.828816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:21:55.432322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829426+0000","last_change":"2023-10-18T11:41:38.827972+0000","last_active":"2023-10-18T11:41:39.829426+0000","last_peered":"2023-10-18T11:41:39.829426+0000","last_clean":"2023-10-18T11:41:39.829426+0000","last_became_active":"2023-10-18T11:41:38.827700+0000","last_became_peered":"2023-10-18T11:41:38.827700+0000","last_unstale":"2023-10-18T11:41:39.829426+0000","last_undegraded":"2023-10-18T11:41:39.829426+0000","last_fullsized":"2023-10-18T11:41:39.829426+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:29.998896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.845165+0000","last_change":"2023-10-18T11:41:37.819453+0000","last_active":"2023-10-18T11:41:38.845165+0000","last_peered":"2023-10-18T11:41:38.845165+0000","last_clean":"2023-10-18T11:41:38.845165+0000","last_became_active":"2023-10-18T11:41:37.819216+0000","last_became_peered":"2023-10-18T11:41:37.819216+0000","last_unstale":"2023-10-18T11:41:38.845165+0000","last_undegraded":"2023-10-18T11:41:38.845165+0000","last_fullsized":"2023-10-18T11:41:38.845165+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:00.103896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841777+0000","last_change":"2023-10-18T11:41:38.836803+0000","last_active":"2023-10-18T11:41:38.841777+0000","last_peered":"2023-10-18T11:41:38.841777+0000","last_clean":"2023-10-18T11:41:38.841777+0000","last_became_active":"2023-10-18T11:41:38.836239+0000","last_became_peered":"2023-10-18T11:41:38.836239+0000","last_unstale":"2023-10-18T11:41:38.841777+0000","last_undegraded":"2023-10-18T11:41:38.841777+0000","last_fullsized":"2023-10-18T11:41:38.841777+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:54.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.823799+0000","last_change":"2023-10-18T11:41:37.823427+0000","last_active":"2023-10-18T11:41:38.823799+0000","last_peered":"2023-10-18T11:41:38.823799+0000","last_clean":"2023-10-18T11:41:38.823799+0000","last_became_active":"2023-10-18T11:41:37.823126+0000","last_became_peered":"2023-10-18T11:41:37.823126+0000","last_unstale":"2023-10-18T11:41:38.823799+0000","last_undegraded":"2023-10-18T11:41:38.823799+0000","last_fullsized":"2023-10-18T11:41:38.823799+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:48.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.828098+0000","last_change":"2023-10-18T11:41:38.825410+0000","last_active":"2023-10-18T11:41:38.828098+0000","last_peered":"2023-10-18T11:41:38.828098+0000","last_clean":"2023-10-18T11:41:38.828098+0000","last_became_active":"2023-10-18T11:41:38.825132+0000","last_became_peered":"2023-10-18T11:41:38.825132+0000","last_unstale":"2023-10-18T11:41:38.828098+0000","last_undegraded":"2023-10-18T11:41:38.828098+0000","last_fullsized":"2023-10-18T11:41:38.828098+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:46.074774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.824065+0000","last_change":"2023-10-18T11:41:37.823660+0000","last_active":"2023-10-18T11:41:38.824065+0000","last_peered":"2023-10-18T11:41:38.824065+0000","last_clean":"2023-10-18T11:41:38.824065+0000","last_became_active":"2023-10-18T11:41:37.823201+0000","last_became_peered":"2023-10-18T11:41:37.823201+0000","last_unstale":"2023-10-18T11:41:38.824065+0000","last_undegraded":"2023-10-18T11:41:38.824065+0000","last_fullsized":"2023-10-18T11:41:38.824065+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:12.705578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.846070+0000","last_change":"2023-10-18T11:41:38.844785+0000","last_active":"2023-10-18T11:41:38.846070+0000","last_peered":"2023-10-18T11:41:38.846070+0000","last_clean":"2023-10-18T11:41:38.846070+0000","last_became_active":"2023-10-18T11:41:38.844502+0000","last_became_peered":"2023-10-18T11:41:38.844502+0000","last_unstale":"2023-10-18T11:41:38.846070+0000","last_undegraded":"2023-10-18T11:41:38.846070+0000","last_fullsized":"2023-10-18T11:41:38.846070+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:20.319797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"173'8","reported_seq":19,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844653+0000","last_change":"2023-10-18T11:41:37.822141+0000","last_active":"2023-10-18T11:41:38.844653+0000","last_peered":"2023-10-18T11:41:38.844653+0000","last_clean":"2023-10-18T11:41:38.844653+0000","last_became_active":"2023-10-18T11:41:37.821819+0000","last_became_peered":"2023-10-18T11:41:37.821819+0000","last_unstale":"2023-10-18T11:41:38.844653+0000","last_undegraded":"2023-10-18T11:41:38.844653+0000","last_fullsized":"2023-10-18T11:41:38.844653+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:19.314971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.840706+0000","last_change":"2023-10-18T11:41:38.837476+0000","last_active":"2023-10-18T11:41:38.840706+0000","last_peered":"2023-10-18T11:41:38.840706+0000","last_clean":"2023-10-18T11:41:38.840706+0000","last_became_active":"2023-10-18T11:41:38.836917+0000","last_became_peered":"2023-10-18T11:41:38.836917+0000","last_unstale":"2023-10-18T11:41:38.840706+0000","last_undegraded":"2023-10-18T11:41:38.840706+0000","last_fullsized":"2023-10-18T11:41:38.840706+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:27.983985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"173'1","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828619+0000","last_change":"2023-10-18T11:41:37.821090+0000","last_active":"2023-10-18T11:41:39.828619+0000","last_peered":"2023-10-18T11:41:39.828619+0000","last_clean":"2023-10-18T11:41:39.828619+0000","last_became_active":"2023-10-18T11:41:37.820845+0000","last_became_peered":"2023-10-18T11:41:37.820845+0000","last_unstale":"2023-10-18T11:41:39.828619+0000","last_undegraded":"2023-10-18T11:41:39.828619+0000","last_fullsized":"2023-10-18T11:41:39.828619+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:15.610937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835699+0000","last_change":"2023-10-18T11:41:38.832709+0000","last_active":"2023-10-18T11:41:38.835699+0000","last_peered":"2023-10-18T11:41:38.835699+0000","last_clean":"2023-10-18T11:41:38.835699+0000","last_became_active":"2023-10-18T11:41:38.832436+0000","last_became_peered":"2023-10-18T11:41:38.832436+0000","last_unstale":"2023-10-18T11:41:38.835699+0000","last_undegraded":"2023-10-18T11:41:38.835699+0000","last_fullsized":"2023-10-18T11:41:38.835699+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:55.448471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.845756+0000","last_change":"2023-10-18T11:41:37.823846+0000","last_active":"2023-10-18T11:41:38.845756+0000","last_peered":"2023-10-18T11:41:38.845756+0000","last_clean":"2023-10-18T11:41:38.845756+0000","last_became_active":"2023-10-18T11:41:37.822544+0000","last_became_peered":"2023-10-18T11:41:37.822544+0000","last_unstale":"2023-10-18T11:41:38.845756+0000","last_undegraded":"2023-10-18T11:41:38.845756+0000","last_fullsized":"2023-10-18T11:41:38.845756+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:01.806142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829630+0000","last_change":"2023-10-18T11:41:38.822398+0000","last_active":"2023-10-18T11:41:38.829630+0000","last_peered":"2023-10-18T11:41:38.829630+0000","last_clean":"2023-10-18T11:41:38.829630+0000","last_became_active":"2023-10-18T11:41:38.822156+0000","last_became_peered":"2023-10-18T11:41:38.822156+0000","last_unstale":"2023-10-18T11:41:38.829630+0000","last_undegraded":"2023-10-18T11:41:38.829630+0000","last_fullsized":"2023-10-18T11:41:38.829630+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:17.259402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"173'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.839691+0000","last_change":"2023-10-18T11:41:37.824435+0000","last_active":"2023-10-18T11:41:38.839691+0000","last_peered":"2023-10-18T11:41:38.839691+0000","last_clean":"2023-10-18T11:41:38.839691+0000","last_became_active":"2023-10-18T11:41:37.823886+0000","last_became_peered":"2023-10-18T11:41:37.823886+0000","last_unstale":"2023-10-18T11:41:38.839691+0000","last_undegraded":"2023-10-18T11:41:38.839691+0000","last_fullsized":"2023-10-18T11:41:38.839691+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:19.194946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835456+0000","last_change":"2023-10-18T11:41:38.833975+0000","last_active":"2023-10-18T11:41:38.835456+0000","last_peered":"2023-10-18T11:41:38.835456+0000","last_clean":"2023-10-18T11:41:38.835456+0000","last_became_active":"2023-10-18T11:41:38.833708+0000","last_became_peered":"2023-10-18T11:41:38.833708+0000","last_unstale":"2023-10-18T11:41:38.835456+0000","last_undegraded":"2023-10-18T11:41:38.835456+0000","last_fullsized":"2023-10-18T11:41:38.835456+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:13.225614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.824069+0000","last_change":"2023-10-18T11:41:37.823230+0000","last_active":"2023-10-18T11:41:38.824069+0000","last_peered":"2023-10-18T11:41:38.824069+0000","last_clean":"2023-10-18T11:41:38.824069+0000","last_became_active":"2023-10-18T11:41:37.822954+0000","last_became_peered":"2023-10-18T11:41:37.822954+0000","last_unstale":"2023-10-18T11:41:38.824069+0000","last_undegraded":"2023-10-18T11:41:38.824069+0000","last_fullsized":"2023-10-18T11:41:38.824069+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:44.059048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835337+0000","last_change":"2023-10-18T11:41:38.832536+0000","last_active":"2023-10-18T11:41:38.835337+0000","last_peered":"2023-10-18T11:41:38.835337+0000","last_clean":"2023-10-18T11:41:38.835337+0000","last_became_active":"2023-10-18T11:41:38.832268+0000","last_became_peered":"2023-10-18T11:41:38.832268+0000","last_unstale":"2023-10-18T11:41:38.835337+0000","last_undegraded":"2023-10-18T11:41:38.835337+0000","last_fullsized":"2023-10-18T11:41:38.835337+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:36.452850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.846834+0000","last_change":"2023-10-18T11:41:37.822283+0000","last_active":"2023-10-18T11:41:38.846834+0000","last_peered":"2023-10-18T11:41:38.846834+0000","last_clean":"2023-10-18T11:41:38.846834+0000","last_became_active":"2023-10-18T11:41:37.822019+0000","last_became_peered":"2023-10-18T11:41:37.822019+0000","last_unstale":"2023-10-18T11:41:38.846834+0000","last_undegraded":"2023-10-18T11:41:38.846834+0000","last_fullsized":"2023-10-18T11:41:38.846834+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:27.346874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841742+0000","last_change":"2023-10-18T11:41:38.835474+0000","last_active":"2023-10-18T11:41:38.841742+0000","last_peered":"2023-10-18T11:41:38.841742+0000","last_clean":"2023-10-18T11:41:38.841742+0000","last_became_active":"2023-10-18T11:41:38.835205+0000","last_became_peered":"2023-10-18T11:41:38.835205+0000","last_unstale":"2023-10-18T11:41:38.841742+0000","last_undegraded":"2023-10-18T11:41:38.841742+0000","last_fullsized":"2023-10-18T11:41:38.841742+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:11:32.696938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829423+0000","last_change":"2023-10-18T11:41:37.819210+0000","last_active":"2023-10-18T11:41:39.829423+0000","last_peered":"2023-10-18T11:41:39.829423+0000","last_clean":"2023-10-18T11:41:39.829423+0000","last_became_active":"2023-10-18T11:41:37.818926+0000","last_became_peered":"2023-10-18T11:41:37.818926+0000","last_unstale":"2023-10-18T11:41:39.829423+0000","last_undegraded":"2023-10-18T11:41:39.829423+0000","last_fullsized":"2023-10-18T11:41:39.829423+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:15.032324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.833763+0000","last_change":"2023-10-18T11:41:38.829408+0000","last_active":"2023-10-18T11:41:38.833763+0000","last_peered":"2023-10-18T11:41:38.833763+0000","last_clean":"2023-10-18T11:41:38.833763+0000","last_became_active":"2023-10-18T11:41:38.828917+0000","last_became_peered":"2023-10-18T11:41:38.828917+0000","last_unstale":"2023-10-18T11:41:38.833763+0000","last_undegraded":"2023-10-18T11:41:38.833763+0000","last_fullsized":"2023-10-18T11:41:38.833763+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:46.074774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.817051+0000","last_change":"2023-10-18T11:41:37.819830+0000","last_active":"2023-10-18T11:41:38.817051+0000","last_peered":"2023-10-18T11:41:38.817051+0000","last_clean":"2023-10-18T11:41:38.817051+0000","last_became_active":"2023-10-18T11:41:37.819576+0000","last_became_peered":"2023-10-18T11:41:37.819576+0000","last_unstale":"2023-10-18T11:41:38.817051+0000","last_undegraded":"2023-10-18T11:41:38.817051+0000","last_fullsized":"2023-10-18T11:41:38.817051+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:54.501911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841674+0000","last_change":"2023-10-18T11:41:38.837848+0000","last_active":"2023-10-18T11:41:38.841674+0000","last_peered":"2023-10-18T11:41:38.841674+0000","last_clean":"2023-10-18T11:41:38.841674+0000","last_became_active":"2023-10-18T11:41:38.837607+0000","last_became_peered":"2023-10-18T11:41:38.837607+0000","last_unstale":"2023-10-18T11:41:38.841674+0000","last_undegraded":"2023-10-18T11:41:38.841674+0000","last_fullsized":"2023-10-18T11:41:38.841674+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:13:56.797253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829767+0000","last_change":"2023-10-18T11:41:37.818588+0000","last_active":"2023-10-18T11:41:39.829767+0000","last_peered":"2023-10-18T11:41:39.829767+0000","last_clean":"2023-10-18T11:41:39.829767+0000","last_became_active":"2023-10-18T11:41:37.817798+0000","last_became_peered":"2023-10-18T11:41:37.817798+0000","last_unstale":"2023-10-18T11:41:39.829767+0000","last_undegraded":"2023-10-18T11:41:39.829767+0000","last_fullsized":"2023-10-18T11:41:39.829767+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:50:52.183260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.843869+0000","last_change":"2023-10-18T11:41:38.842544+0000","last_active":"2023-10-18T11:41:38.843869+0000","last_peered":"2023-10-18T11:41:38.843869+0000","last_clean":"2023-10-18T11:41:38.843869+0000","last_became_active":"2023-10-18T11:41:38.841060+0000","last_became_peered":"2023-10-18T11:41:38.841060+0000","last_unstale":"2023-10-18T11:41:38.843869+0000","last_undegraded":"2023-10-18T11:41:38.843869+0000","last_fullsized":"2023-10-18T11:41:38.843869+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:33.088719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844520+0000","last_change":"2023-10-18T11:41:37.823910+0000","last_active":"2023-10-18T11:41:38.844520+0000","last_peered":"2023-10-18T11:41:38.844520+0000","last_clean":"2023-10-18T11:41:38.844520+0000","last_became_active":"2023-10-18T11:41:37.823671+0000","last_became_peered":"2023-10-18T11:41:37.823671+0000","last_unstale":"2023-10-18T11:41:38.844520+0000","last_undegraded":"2023-10-18T11:41:38.844520+0000","last_fullsized":"2023-10-18T11:41:38.844520+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:15.970084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.836949+0000","last_change":"2023-10-18T11:41:38.834278+0000","last_active":"2023-10-18T11:41:38.836949+0000","last_peered":"2023-10-18T11:41:38.836949+0000","last_clean":"2023-10-18T11:41:38.836949+0000","last_became_active":"2023-10-18T11:41:38.833972+0000","last_became_peered":"2023-10-18T11:41:38.833972+0000","last_unstale":"2023-10-18T11:41:38.836949+0000","last_undegraded":"2023-10-18T11:41:38.836949+0000","last_fullsized":"2023-10-18T11:41:38.836949+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:28.199054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.816866+0000","last_change":"2023-10-18T11:41:37.823322+0000","last_active":"2023-10-18T11:41:38.816866+0000","last_peered":"2023-10-18T11:41:38.816866+0000","last_clean":"2023-10-18T11:41:38.816866+0000","last_became_active":"2023-10-18T11:41:37.822856+0000","last_became_peered":"2023-10-18T11:41:37.822856+0000","last_unstale":"2023-10-18T11:41:38.816866+0000","last_undegraded":"2023-10-18T11:41:38.816866+0000","last_fullsized":"2023-10-18T11:41:38.816866+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:52.515374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831189+0000","last_change":"2023-10-18T11:41:38.827748+0000","last_active":"2023-10-18T11:41:38.831189+0000","last_peered":"2023-10-18T11:41:38.831189+0000","last_clean":"2023-10-18T11:41:38.831189+0000","last_became_active":"2023-10-18T11:41:38.827469+0000","last_became_peered":"2023-10-18T11:41:38.827469+0000","last_unstale":"2023-10-18T11:41:38.831189+0000","last_undegraded":"2023-10-18T11:41:38.831189+0000","last_fullsized":"2023-10-18T11:41:38.831189+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:09.333393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831053+0000","last_change":"2023-10-18T11:41:37.821788+0000","last_active":"2023-10-18T11:41:38.831053+0000","last_peered":"2023-10-18T11:41:38.831053+0000","last_clean":"2023-10-18T11:41:38.831053+0000","last_became_active":"2023-10-18T11:41:37.821313+0000","last_became_peered":"2023-10-18T11:41:37.821313+0000","last_unstale":"2023-10-18T11:41:38.831053+0000","last_undegraded":"2023-10-18T11:41:38.831053+0000","last_fullsized":"2023-10-18T11:41:38.831053+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:29.206502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841722+0000","last_change":"2023-10-18T11:41:38.835302+0000","last_active":"2023-10-18T11:41:38.841722+0000","last_peered":"2023-10-18T11:41:38.841722+0000","last_clean":"2023-10-18T11:41:38.841722+0000","last_became_active":"2023-10-18T11:41:38.835040+0000","last_became_peered":"2023-10-18T11:41:38.835040+0000","last_unstale":"2023-10-18T11:41:38.841722+0000","last_undegraded":"2023-10-18T11:41:38.841722+0000","last_fullsized":"2023-10-18T11:41:38.841722+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.810967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.819109+0000","last_change":"2023-10-18T11:41:37.822147+0000","last_active":"2023-10-18T11:41:38.819109+0000","last_peered":"2023-10-18T11:41:38.819109+0000","last_clean":"2023-10-18T11:41:38.819109+0000","last_became_active":"2023-10-18T11:41:37.821839+0000","last_became_peered":"2023-10-18T11:41:37.821839+0000","last_unstale":"2023-10-18T11:41:38.819109+0000","last_undegraded":"2023-10-18T11:41:38.819109+0000","last_fullsized":"2023-10-18T11:41:38.819109+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:54.443645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829996+0000","last_change":"2023-10-18T11:41:38.821087+0000","last_active":"2023-10-18T11:41:39.829996+0000","last_peered":"2023-10-18T11:41:39.829996+0000","last_clean":"2023-10-18T11:41:39.829996+0000","last_became_active":"2023-10-18T11:41:38.820806+0000","last_became_peered":"2023-10-18T11:41:38.820806+0000","last_unstale":"2023-10-18T11:41:39.829996+0000","last_undegraded":"2023-10-18T11:41:39.829996+0000","last_fullsized":"2023-10-18T11:41:39.829996+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:39.471952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.814926+0000","last_change":"2023-10-18T11:41:37.824985+0000","last_active":"2023-10-18T11:41:38.814926+0000","last_peered":"2023-10-18T11:41:38.814926+0000","last_clean":"2023-10-18T11:41:38.814926+0000","last_became_active":"2023-10-18T11:41:37.824481+0000","last_became_peered":"2023-10-18T11:41:37.824481+0000","last_unstale":"2023-10-18T11:41:38.814926+0000","last_undegraded":"2023-10-18T11:41:38.814926+0000","last_fullsized":"2023-10-18T11:41:38.814926+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:39.668889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.834364+0000","last_change":"2023-10-18T11:41:38.830152+0000","last_active":"2023-10-18T11:41:38.834364+0000","last_peered":"2023-10-18T11:41:38.834364+0000","last_clean":"2023-10-18T11:41:38.834364+0000","last_became_active":"2023-10-18T11:41:38.829828+0000","last_became_peered":"2023-10-18T11:41:38.829828+0000","last_unstale":"2023-10-18T11:41:38.834364+0000","last_undegraded":"2023-10-18T11:41:38.834364+0000","last_fullsized":"2023-10-18T11:41:38.834364+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:01:40.285325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828507+0000","last_change":"2023-10-18T11:41:37.820903+0000","last_active":"2023-10-18T11:41:39.828507+0000","last_peered":"2023-10-18T11:41:39.828507+0000","last_clean":"2023-10-18T11:41:39.828507+0000","last_became_active":"2023-10-18T11:41:37.820650+0000","last_became_peered":"2023-10-18T11:41:37.820650+0000","last_unstale":"2023-10-18T11:41:39.828507+0000","last_undegraded":"2023-10-18T11:41:39.828507+0000","last_fullsized":"2023-10-18T11:41:39.828507+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:11.753576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841714+0000","last_change":"2023-10-18T11:41:38.838162+0000","last_active":"2023-10-18T11:41:38.841714+0000","last_peered":"2023-10-18T11:41:38.841714+0000","last_clean":"2023-10-18T11:41:38.841714+0000","last_became_active":"2023-10-18T11:41:38.836514+0000","last_became_peered":"2023-10-18T11:41:38.836514+0000","last_unstale":"2023-10-18T11:41:38.841714+0000","last_undegraded":"2023-10-18T11:41:38.841714+0000","last_fullsized":"2023-10-18T11:41:38.841714+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:26.755708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.815291+0000","last_change":"2023-10-18T11:41:37.814973+0000","last_active":"2023-10-18T11:41:38.815291+0000","last_peered":"2023-10-18T11:41:38.815291+0000","last_clean":"2023-10-18T11:41:38.815291+0000","last_became_active":"2023-10-18T11:41:37.814699+0000","last_became_peered":"2023-10-18T11:41:37.814699+0000","last_unstale":"2023-10-18T11:41:38.815291+0000","last_undegraded":"2023-10-18T11:41:38.815291+0000","last_fullsized":"2023-10-18T11:41:38.815291+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:19.194946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829747+0000","last_change":"2023-10-18T11:41:38.826161+0000","last_active":"2023-10-18T11:41:39.829747+0000","last_peered":"2023-10-18T11:41:39.829747+0000","last_clean":"2023-10-18T11:41:39.829747+0000","last_became_active":"2023-10-18T11:41:38.825615+0000","last_became_peered":"2023-10-18T11:41:38.825615+0000","last_unstale":"2023-10-18T11:41:39.829747+0000","last_undegraded":"2023-10-18T11:41:39.829747+0000","last_fullsized":"2023-10-18T11:41:39.829747+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:47.816689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.822205+0000","last_change":"2023-10-18T11:41:37.822249+0000","last_active":"2023-10-18T11:41:38.822205+0000","last_peered":"2023-10-18T11:41:38.822205+0000","last_clean":"2023-10-18T11:41:38.822205+0000","last_became_active":"2023-10-18T11:41:37.821714+0000","last_became_peered":"2023-10-18T11:41:37.821714+0000","last_unstale":"2023-10-18T11:41:38.822205+0000","last_undegraded":"2023-10-18T11:41:38.822205+0000","last_fullsized":"2023-10-18T11:41:38.822205+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:24.518814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829803+0000","last_change":"2023-10-18T11:41:38.836422+0000","last_active":"2023-10-18T11:41:39.829803+0000","last_peered":"2023-10-18T11:41:39.829803+0000","last_clean":"2023-10-18T11:41:39.829803+0000","last_became_active":"2023-10-18T11:41:38.835474+0000","last_became_peered":"2023-10-18T11:41:38.835474+0000","last_unstale":"2023-10-18T11:41:39.829803+0000","last_undegraded":"2023-10-18T11:41:39.829803+0000","last_fullsized":"2023-10-18T11:41:39.829803+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:38.697917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"173'6","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.848425+0000","last_change":"2023-10-18T11:41:37.820513+0000","last_active":"2023-10-18T11:41:38.848425+0000","last_peered":"2023-10-18T11:41:38.848425+0000","last_clean":"2023-10-18T11:41:38.848425+0000","last_became_active":"2023-10-18T11:41:37.820194+0000","last_became_peered":"2023-10-18T11:41:37.820194+0000","last_unstale":"2023-10-18T11:41:38.848425+0000","last_undegraded":"2023-10-18T11:41:38.848425+0000","last_fullsized":"2023-10-18T11:41:38.848425+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:58.848022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"162'280","reported_seq":911,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-18T11:41:37.815051+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:41:37.815051+0000","last_peered":"2023-10-18T11:41:37.815051+0000","last_clean":"2023-10-18T11:41:37.815051+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:41:37.815051+0000","last_undegraded":"2023-10-18T11:41:37.815051+0000","last_fullsized":"2023-10-18T11:41:37.815051+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575768,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51928,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319912,"statfs":{"total":96636764160,"available":96583589888,"internally_reserved":0,"allocated":2760704,"data_stored":1687384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.626},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.47699999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.64600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575768,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37460,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334380,"statfs":{"total":96636764160,"available":96598405120,"internally_reserved":0,"allocated":2166784,"data_stored":1097064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.375}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.442},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49399999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56000000000000005}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575768,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59660,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312180,"statfs":{"total":96636764160,"available":96575672320,"internally_reserved":0,"allocated":2162688,"data_stored":1096994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.54200000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.86199999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.84599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575768,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41640,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330200,"statfs":{"total":96636764160,"available":96594124800,"internally_reserved":0,"allocated":2187264,"data_stored":1105254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.27800000000000002},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.626,"15min":0.623},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.86399999999999999},"last":0.47399999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84599999999999997,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.30499999999999999},"max":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.33000000000000002},"max":{"1min":0.77000000000000002,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29999999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58899999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.36199999999999999},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.80800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575768,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":2170880,"data_stored":1097610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.31},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.34799999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.49099999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.45900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575768,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59684,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312156,"statfs":{"total":96636764160,"available":96575647744,"internally_reserved":0,"allocated":2183168,"data_stored":1103754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.28999999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.32700000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.74299999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.622}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575768,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2166784,"data_stored":1097064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.24299999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.222},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.29699999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.318},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.27600000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.73499999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.439},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.5}]}]},{"osd":0,"up_from":13,"seq":55834575767,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2781184,"data_stored":1689680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:41:42.761 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T11:41:42.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 -- 172.21.15.125:0/3448208287 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f0159bd0 msgr2=0x7f31f015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 --2- 172.21.15.125:0/3448208287 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f0159bd0 0x7f31f015c770 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f31f00cbd30 tx=0x7f31e002bd10 comp rx=0 tx=0).stop 2023-10-18T11:41:42.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 -- 172.21.15.125:0/3448208287 shutdown_connections 2023-10-18T11:41:42.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 --2- 172.21.15.125:0/3448208287 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f0159bd0 0x7f31f015c770 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 --2- 172.21.15.125:0/3448208287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31f015ccb0 0x7f31f00c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 --2- 172.21.15.125:0/3448208287 >> v2:172.21.15.139:3300/0 conn(0x7f31f006b030 0x7f31f006b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 -- 172.21.15.125:0/3448208287 >> 172.21.15.125:0/3448208287 conn(0x7f31f0069930 msgr2=0x7f31f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 -- 172.21.15.125:0/3448208287 shutdown_connections 2023-10-18T11:41:42.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.895+0000 7f31f78c9640 1 -- 172.21.15.125:0/3448208287 wait complete. 2023-10-18T11:41:42.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.896+0000 7f31f78c9640 1 Processor -- start 2023-10-18T11:41:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.896+0000 7f31f78c9640 1 -- start start 2023-10-18T11:41:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f78c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f006b030 0x7f31f0070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f78c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31f0159bd0 0x7f31f00caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f78c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31f015ccb0 0x7f31f013ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f78c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01521b0 con 0x7f31f006b030 2023-10-18T11:41:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f78c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00ab2e0 con 0x7f31f0159bd0 2023-10-18T11:41:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f78c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f01505d0 con 0x7f31f015ccb0 2023-10-18T11:41:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f006b030 0x7f31f0070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f006b030 0x7f31f0070940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56630/0 (socket says 172.21.15.125:56630) 2023-10-18T11:41:42.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 -- 172.21.15.125:0/869690826 learned_addr learned my addr 172.21.15.125:0/869690826 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f4e64640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31f0159bd0 0x7f31f00caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5e66640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31f015ccb0 0x7f31f013ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 -- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31f015ccb0 msgr2=0x7f31f013ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31f015ccb0 0x7f31f013ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 -- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31f0159bd0 msgr2=0x7f31f00caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31f0159bd0 0x7f31f00caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 -- 172.21.15.125:0/869690826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0009560 con 0x7f31f006b030 2023-10-18T11:41:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f4e64640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31f0159bd0 0x7f31f00caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f5665640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f006b030 0x7f31f0070940 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c890 tx=0x7f31ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:42.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31e5ffb640 1 -- 172.21.15.125:0/869690826 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec004220 con 0x7f31f006b030 2023-10-18T11:41:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31e5ffb640 1 -- 172.21.15.125:0/869690826 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31ec004470 con 0x7f31f006b030 2023-10-18T11:41:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31e5ffb640 1 -- 172.21.15.125:0/869690826 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007c10 con 0x7f31f006b030 2023-10-18T11:41:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.897+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f014f720 con 0x7f31f006b030 2023-10-18T11:41:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.898+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f014fa70 con 0x7f31f006b030 2023-10-18T11:41:42.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.899+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f00690e0 con 0x7f31f006b030 2023-10-18T11:41:42.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.899+0000 7f31e5ffb640 1 -- 172.21.15.125:0/869690826 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f31ec005020 con 0x7f31f006b030 2023-10-18T11:41:42.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.899+0000 7f31e5ffb640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f31d406af90 0x7f31d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:42.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.899+0000 7f31e5ffb640 1 -- 172.21.15.125:0/869690826 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f31ec089e90 con 0x7f31f006b030 2023-10-18T11:41:42.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.899+0000 7f31f4e64640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f31d406af90 0x7f31d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:42.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.900+0000 7f31f4e64640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f31d406af90 0x7f31d406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31e0035000 tx=0x7f31e0004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:42.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:42.903+0000 7f31e5ffb640 1 -- 172.21.15.125:0/869690826 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31ec054c50 con 0x7f31f006b030 2023-10-18T11:41:43.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.132+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f31f013f100 con 0x7f31f006b030 2023-10-18T11:41:43.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.133+0000 7f31e5ffb640 1 -- 172.21.15.125:0/869690826 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1018 (secure 0 0 0) 0x7f31ec01d020 con 0x7f31f006b030 2023-10-18T11:41:43.136 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:43.136 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4624,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":174,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":394813440,"bytes_avail":772699299840,"bytes_total":773094113280,"write_bytes_sec":2546,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":67,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":10320}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-18T11:31:18.534143+0000","services":{}},"progress_events":{}} 2023-10-18T11:41:43.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f31d406af90 msgr2=0x7f31d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:43.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f31d406af90 0x7f31d406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31e0035000 tx=0x7f31e0004730 comp rx=0 tx=0).stop 2023-10-18T11:41:43.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f006b030 msgr2=0x7f31f0070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:43.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f006b030 0x7f31f0070940 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c890 tx=0x7f31ec00cd50 comp rx=0 tx=0).stop 2023-10-18T11:41:43.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 shutdown_connections 2023-10-18T11:41:43.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f31d406af90 0x7f31d406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31f0159bd0 0x7f31f00caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31f015ccb0 0x7f31f013ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 --2- 172.21.15.125:0/869690826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31f006b030 0x7f31f0070940 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 >> 172.21.15.125:0/869690826 conn(0x7f31f0069930 msgr2=0x7f31f015bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:43.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 shutdown_connections 2023-10-18T11:41:43.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.135+0000 7f31f78c9640 1 -- 172.21.15.125:0/869690826 wait complete. 2023-10-18T11:41:43.148 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T11:41:43.148 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:41:43.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.249+0000 7f4440021640 1 -- 172.21.15.125:0/927549549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4438141690 msgr2=0x7f44380703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.249+0000 7f4440021640 1 --2- 172.21.15.125:0/927549549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4438141690 0x7f44380703e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4428009830 tx=0x7f442802bcd0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.249+0000 7f4440021640 1 -- 172.21.15.125:0/927549549 shutdown_connections 2023-10-18T11:41:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.249+0000 7f4440021640 1 --2- 172.21.15.125:0/927549549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4438070920 0x7f44380affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.249+0000 7f4440021640 1 --2- 172.21.15.125:0/927549549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4438141690 0x7f44380703e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.249+0000 7f4440021640 1 -- 172.21.15.125:0/927549549 >> 172.21.15.125:0/927549549 conn(0x7f4438069930 msgr2=0x7f4438069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:43.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.249+0000 7f4440021640 1 -- 172.21.15.125:0/927549549 shutdown_connections 2023-10-18T11:41:43.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.249+0000 7f4440021640 1 -- 172.21.15.125:0/927549549 wait complete. 2023-10-18T11:41:43.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.250+0000 7f4440021640 1 Processor -- start 2023-10-18T11:41:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.250+0000 7f4440021640 1 -- start start 2023-10-18T11:41:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f4440021640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4438070920 0x7f44381db700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f4440021640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f443813e520 0x7f44381d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:43.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f4440021640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4438141690 0x7f443813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f4440021640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44381db2e0 con 0x7f4438070920 2023-10-18T11:41:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f4440021640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44381e3240 con 0x7f443813e520 2023-10-18T11:41:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f4440021640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44381da940 con 0x7f4438141690 2023-10-18T11:41:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4438070920 0x7f44381db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:43.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4438070920 0x7f44381db700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56638/0 (socket says 172.21.15.125:56638) 2023-10-18T11:41:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 -- 172.21.15.125:0/4130027411 learned_addr learned my addr 172.21.15.125:0/4130027411 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443d5bc640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f443813e520 0x7f44381d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443e5be640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4438141690 0x7f443813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:43.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 -- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4438141690 msgr2=0x7f443813ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:43.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4438141690 0x7f443813ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 -- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f443813e520 msgr2=0x7f44381d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:43.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f443813e520 0x7f44381d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 -- 172.21.15.125:0/4130027411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430009900 con 0x7f4438070920 2023-10-18T11:41:43.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443d5bc640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f443813e520 0x7f44381d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:43.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f443ddbd640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4438070920 0x7f44381db700 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f443400c400 tx=0x7f443400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:43.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f442e7fc640 1 -- 172.21.15.125:0/4130027411 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44340052d0 con 0x7f4438070920 2023-10-18T11:41:43.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f442e7fc640 1 -- 172.21.15.125:0/4130027411 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4434007020 con 0x7f4438070920 2023-10-18T11:41:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4428009490 con 0x7f4438070920 2023-10-18T11:41:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f442e7fc640 1 -- 172.21.15.125:0/4130027411 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434007430 con 0x7f4438070920 2023-10-18T11:41:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.251+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44381daf00 con 0x7f4438070920 2023-10-18T11:41:43.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.252+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44381db420 con 0x7f4438070920 2023-10-18T11:41:43.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.253+0000 7f442e7fc640 1 -- 172.21.15.125:0/4130027411 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4434007590 con 0x7f4438070920 2023-10-18T11:41:43.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.253+0000 7f442e7fc640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f441c06af60 0x7f441c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:43.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.253+0000 7f442e7fc640 1 -- 172.21.15.125:0/4130027411 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f4434088ef0 con 0x7f4438070920 2023-10-18T11:41:43.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.253+0000 7f443d5bc640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f441c06af60 0x7f441c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:43.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.254+0000 7f443d5bc640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f441c06af60 0x7f441c06d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4428036000 tx=0x7f4428004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:43.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.257+0000 7f442e7fc640 1 -- 172.21.15.125:0/4130027411 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4434053dc0 con 0x7f4438070920 2023-10-18T11:41:43.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.420+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 --> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44380b1bd0 con 0x7f441c06af60 2023-10-18T11:41:43.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.428+0000 7f442e7fc640 1 -- 172.21.15.125:0/4130027411 <== mgr.8644 v2:172.21.15.139:6832/4236255547 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290319 (secure 0 0 0) 0x7f44380b1bd0 con 0x7f441c06af60 2023-10-18T11:41:43.434 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:43.435 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:41:43.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f441c06af60 msgr2=0x7f441c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:43.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f441c06af60 0x7f441c06d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4428036000 tx=0x7f4428004730 comp rx=0 tx=0).stop 2023-10-18T11:41:43.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4438070920 msgr2=0x7f44381db700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:43.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4438070920 0x7f44381db700 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f443400c400 tx=0x7f443400c8c0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 shutdown_connections 2023-10-18T11:41:43.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f441c06af60 0x7f441c06d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f443813e520 0x7f44381d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4438141690 0x7f443813ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 --2- 172.21.15.125:0/4130027411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4438070920 0x7f44381db700 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:43.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 >> 172.21.15.125:0/4130027411 conn(0x7f4438069930 msgr2=0x7f44381df840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:43.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 shutdown_connections 2023-10-18T11:41:43.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:43.432+0000 7f4440021640 1 -- 172.21.15.125:0/4130027411 wait complete. 2023-10-18T11:41:43.445 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T11:41:42.432325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":499,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385560,"kb_used_data":18144,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754589160,"statfs":{"total":773094113280,"available":772699299840,"internally_reserved":0,"allocated":18579456,"data_stored":9974804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.629045"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829496+0000","last_change":"2023-10-18T11:41:38.825716+0000","last_active":"2023-10-18T11:41:38.829496+0000","last_peered":"2023-10-18T11:41:38.829496+0000","last_clean":"2023-10-18T11:41:38.829496+0000","last_became_active":"2023-10-18T11:41:38.825425+0000","last_became_peered":"2023-10-18T11:41:38.825425+0000","last_unstale":"2023-10-18T11:41:38.829496+0000","last_undegraded":"2023-10-18T11:41:38.829496+0000","last_fullsized":"2023-10-18T11:41:38.829496+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:39:43.232981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841612+0000","last_change":"2023-10-18T11:41:38.837347+0000","last_active":"2023-10-18T11:41:38.841612+0000","last_peered":"2023-10-18T11:41:38.841612+0000","last_clean":"2023-10-18T11:41:38.841612+0000","last_became_active":"2023-10-18T11:41:38.837097+0000","last_became_peered":"2023-10-18T11:41:38.837097+0000","last_unstale":"2023-10-18T11:41:38.841612+0000","last_undegraded":"2023-10-18T11:41:38.841612+0000","last_fullsized":"2023-10-18T11:41:38.841612+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:40.673714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.842484+0000","last_change":"2023-10-18T11:41:38.840032+0000","last_active":"2023-10-18T11:41:38.842484+0000","last_peered":"2023-10-18T11:41:38.842484+0000","last_clean":"2023-10-18T11:41:38.842484+0000","last_became_active":"2023-10-18T11:41:38.838657+0000","last_became_peered":"2023-10-18T11:41:38.838657+0000","last_unstale":"2023-10-18T11:41:38.842484+0000","last_undegraded":"2023-10-18T11:41:38.842484+0000","last_fullsized":"2023-10-18T11:41:38.842484+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:29.239361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.834357+0000","last_change":"2023-10-18T11:41:38.832579+0000","last_active":"2023-10-18T11:41:38.834357+0000","last_peered":"2023-10-18T11:41:38.834357+0000","last_clean":"2023-10-18T11:41:38.834357+0000","last_became_active":"2023-10-18T11:41:38.832320+0000","last_became_peered":"2023-10-18T11:41:38.832320+0000","last_unstale":"2023-10-18T11:41:38.834357+0000","last_undegraded":"2023-10-18T11:41:38.834357+0000","last_fullsized":"2023-10-18T11:41:38.834357+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:12:30.419426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.843919+0000","last_change":"2023-10-18T11:41:38.840547+0000","last_active":"2023-10-18T11:41:38.843919+0000","last_peered":"2023-10-18T11:41:38.843919+0000","last_clean":"2023-10-18T11:41:38.843919+0000","last_became_active":"2023-10-18T11:41:38.840290+0000","last_became_peered":"2023-10-18T11:41:38.840290+0000","last_unstale":"2023-10-18T11:41:38.843919+0000","last_undegraded":"2023-10-18T11:41:38.843919+0000","last_fullsized":"2023-10-18T11:41:38.843919+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:37.966595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841820+0000","last_change":"2023-10-18T11:41:38.839747+0000","last_active":"2023-10-18T11:41:38.841820+0000","last_peered":"2023-10-18T11:41:38.841820+0000","last_clean":"2023-10-18T11:41:38.841820+0000","last_became_active":"2023-10-18T11:41:38.839470+0000","last_became_peered":"2023-10-18T11:41:38.839470+0000","last_unstale":"2023-10-18T11:41:38.841820+0000","last_undegraded":"2023-10-18T11:41:38.841820+0000","last_fullsized":"2023-10-18T11:41:38.841820+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:25.523639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.833625+0000","last_change":"2023-10-18T11:41:38.826683+0000","last_active":"2023-10-18T11:41:38.833625+0000","last_peered":"2023-10-18T11:41:38.833625+0000","last_clean":"2023-10-18T11:41:38.833625+0000","last_became_active":"2023-10-18T11:41:38.826277+0000","last_became_peered":"2023-10-18T11:41:38.826277+0000","last_unstale":"2023-10-18T11:41:38.833625+0000","last_undegraded":"2023-10-18T11:41:38.833625+0000","last_fullsized":"2023-10-18T11:41:38.833625+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:09.384565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829657+0000","last_change":"2023-10-18T11:41:38.828519+0000","last_active":"2023-10-18T11:41:39.829657+0000","last_peered":"2023-10-18T11:41:39.829657+0000","last_clean":"2023-10-18T11:41:39.829657+0000","last_became_active":"2023-10-18T11:41:38.828256+0000","last_became_peered":"2023-10-18T11:41:38.828256+0000","last_unstale":"2023-10-18T11:41:39.829657+0000","last_undegraded":"2023-10-18T11:41:39.829657+0000","last_fullsized":"2023-10-18T11:41:39.829657+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:18.914354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841743+0000","last_change":"2023-10-18T11:41:38.836612+0000","last_active":"2023-10-18T11:41:38.841743+0000","last_peered":"2023-10-18T11:41:38.841743+0000","last_clean":"2023-10-18T11:41:38.841743+0000","last_became_active":"2023-10-18T11:41:38.836342+0000","last_became_peered":"2023-10-18T11:41:38.836342+0000","last_unstale":"2023-10-18T11:41:38.841743+0000","last_undegraded":"2023-10-18T11:41:38.841743+0000","last_fullsized":"2023-10-18T11:41:38.841743+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:36.452850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.825014+0000","last_change":"2023-10-18T11:41:38.822440+0000","last_active":"2023-10-18T11:41:38.825014+0000","last_peered":"2023-10-18T11:41:38.825014+0000","last_clean":"2023-10-18T11:41:38.825014+0000","last_became_active":"2023-10-18T11:41:38.822164+0000","last_became_peered":"2023-10-18T11:41:38.822164+0000","last_unstale":"2023-10-18T11:41:38.825014+0000","last_undegraded":"2023-10-18T11:41:38.825014+0000","last_fullsized":"2023-10-18T11:41:38.825014+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:43.202732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828721+0000","last_change":"2023-10-18T11:41:38.832961+0000","last_active":"2023-10-18T11:41:39.828721+0000","last_peered":"2023-10-18T11:41:39.828721+0000","last_clean":"2023-10-18T11:41:39.828721+0000","last_became_active":"2023-10-18T11:41:38.832373+0000","last_became_peered":"2023-10-18T11:41:38.832373+0000","last_unstale":"2023-10-18T11:41:39.828721+0000","last_undegraded":"2023-10-18T11:41:39.828721+0000","last_fullsized":"2023-10-18T11:41:39.828721+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:16.135065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.834340+0000","last_change":"2023-10-18T11:41:38.830562+0000","last_active":"2023-10-18T11:41:38.834340+0000","last_peered":"2023-10-18T11:41:38.834340+0000","last_clean":"2023-10-18T11:41:38.834340+0000","last_became_active":"2023-10-18T11:41:38.830272+0000","last_became_peered":"2023-10-18T11:41:38.830272+0000","last_unstale":"2023-10-18T11:41:38.834340+0000","last_undegraded":"2023-10-18T11:41:38.834340+0000","last_fullsized":"2023-10-18T11:41:38.834340+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:29.879841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835196+0000","last_change":"2023-10-18T11:41:38.831348+0000","last_active":"2023-10-18T11:41:38.835196+0000","last_peered":"2023-10-18T11:41:38.835196+0000","last_clean":"2023-10-18T11:41:38.835196+0000","last_became_active":"2023-10-18T11:41:38.826783+0000","last_became_peered":"2023-10-18T11:41:38.826783+0000","last_unstale":"2023-10-18T11:41:38.835196+0000","last_undegraded":"2023-10-18T11:41:38.835196+0000","last_fullsized":"2023-10-18T11:41:38.835196+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:15:13.102514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.834516+0000","last_change":"2023-10-18T11:41:38.830730+0000","last_active":"2023-10-18T11:41:38.834516+0000","last_peered":"2023-10-18T11:41:38.834516+0000","last_clean":"2023-10-18T11:41:38.834516+0000","last_became_active":"2023-10-18T11:41:38.830409+0000","last_became_peered":"2023-10-18T11:41:38.830409+0000","last_unstale":"2023-10-18T11:41:38.834516+0000","last_undegraded":"2023-10-18T11:41:38.834516+0000","last_fullsized":"2023-10-18T11:41:38.834516+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:01.324176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841760+0000","last_change":"2023-10-18T11:41:38.836640+0000","last_active":"2023-10-18T11:41:38.841760+0000","last_peered":"2023-10-18T11:41:38.841760+0000","last_clean":"2023-10-18T11:41:38.841760+0000","last_became_active":"2023-10-18T11:41:38.835870+0000","last_became_peered":"2023-10-18T11:41:38.835870+0000","last_unstale":"2023-10-18T11:41:38.841760+0000","last_undegraded":"2023-10-18T11:41:38.841760+0000","last_fullsized":"2023-10-18T11:41:38.841760+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:02.224387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835247+0000","last_change":"2023-10-18T11:41:38.831324+0000","last_active":"2023-10-18T11:41:38.835247+0000","last_peered":"2023-10-18T11:41:38.835247+0000","last_clean":"2023-10-18T11:41:38.835247+0000","last_became_active":"2023-10-18T11:41:38.831052+0000","last_became_peered":"2023-10-18T11:41:38.831052+0000","last_unstale":"2023-10-18T11:41:38.835247+0000","last_undegraded":"2023-10-18T11:41:38.835247+0000","last_fullsized":"2023-10-18T11:41:38.835247+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:10.646615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.836918+0000","last_change":"2023-10-18T11:41:38.834503+0000","last_active":"2023-10-18T11:41:38.836918+0000","last_peered":"2023-10-18T11:41:38.836918+0000","last_clean":"2023-10-18T11:41:38.836918+0000","last_became_active":"2023-10-18T11:41:38.834215+0000","last_became_peered":"2023-10-18T11:41:38.834215+0000","last_unstale":"2023-10-18T11:41:38.836918+0000","last_undegraded":"2023-10-18T11:41:38.836918+0000","last_fullsized":"2023-10-18T11:41:38.836918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:22:04.684733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828696+0000","last_change":"2023-10-18T11:41:38.827931+0000","last_active":"2023-10-18T11:41:39.828696+0000","last_peered":"2023-10-18T11:41:39.828696+0000","last_clean":"2023-10-18T11:41:39.828696+0000","last_became_active":"2023-10-18T11:41:38.827669+0000","last_became_peered":"2023-10-18T11:41:38.827669+0000","last_unstale":"2023-10-18T11:41:39.828696+0000","last_undegraded":"2023-10-18T11:41:39.828696+0000","last_fullsized":"2023-10-18T11:41:39.828696+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:03.238765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835680+0000","last_change":"2023-10-18T11:41:38.832584+0000","last_active":"2023-10-18T11:41:38.835680+0000","last_peered":"2023-10-18T11:41:38.835680+0000","last_clean":"2023-10-18T11:41:38.835680+0000","last_became_active":"2023-10-18T11:41:38.832073+0000","last_became_peered":"2023-10-18T11:41:38.832073+0000","last_unstale":"2023-10-18T11:41:38.835680+0000","last_undegraded":"2023-10-18T11:41:38.835680+0000","last_fullsized":"2023-10-18T11:41:38.835680+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:51.765003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.840839+0000","last_change":"2023-10-18T11:41:38.837755+0000","last_active":"2023-10-18T11:41:38.840839+0000","last_peered":"2023-10-18T11:41:38.840839+0000","last_clean":"2023-10-18T11:41:38.840839+0000","last_became_active":"2023-10-18T11:41:38.837504+0000","last_became_peered":"2023-10-18T11:41:38.837504+0000","last_unstale":"2023-10-18T11:41:38.840839+0000","last_undegraded":"2023-10-18T11:41:38.840839+0000","last_fullsized":"2023-10-18T11:41:38.840839+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:15:30.360143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835616+0000","last_change":"2023-10-18T11:41:38.832688+0000","last_active":"2023-10-18T11:41:38.835616+0000","last_peered":"2023-10-18T11:41:38.835616+0000","last_clean":"2023-10-18T11:41:38.835616+0000","last_became_active":"2023-10-18T11:41:38.832343+0000","last_became_peered":"2023-10-18T11:41:38.832343+0000","last_unstale":"2023-10-18T11:41:38.835616+0000","last_undegraded":"2023-10-18T11:41:38.835616+0000","last_fullsized":"2023-10-18T11:41:38.835616+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:27.983985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831198+0000","last_change":"2023-10-18T11:41:38.826514+0000","last_active":"2023-10-18T11:41:38.831198+0000","last_peered":"2023-10-18T11:41:38.831198+0000","last_clean":"2023-10-18T11:41:38.831198+0000","last_became_active":"2023-10-18T11:41:38.826186+0000","last_became_peered":"2023-10-18T11:41:38.826186+0000","last_unstale":"2023-10-18T11:41:38.831198+0000","last_undegraded":"2023-10-18T11:41:38.831198+0000","last_fullsized":"2023-10-18T11:41:38.831198+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:10.683364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829945+0000","last_change":"2023-10-18T11:41:38.826932+0000","last_active":"2023-10-18T11:41:39.829945+0000","last_peered":"2023-10-18T11:41:39.829945+0000","last_clean":"2023-10-18T11:41:39.829945+0000","last_became_active":"2023-10-18T11:41:38.826645+0000","last_became_peered":"2023-10-18T11:41:38.826645+0000","last_unstale":"2023-10-18T11:41:39.829945+0000","last_undegraded":"2023-10-18T11:41:39.829945+0000","last_fullsized":"2023-10-18T11:41:39.829945+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:54:15.505963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828515+0000","last_change":"2023-10-18T11:41:38.827801+0000","last_active":"2023-10-18T11:41:39.828515+0000","last_peered":"2023-10-18T11:41:39.828515+0000","last_clean":"2023-10-18T11:41:39.828515+0000","last_became_active":"2023-10-18T11:41:38.827514+0000","last_became_peered":"2023-10-18T11:41:38.827514+0000","last_unstale":"2023-10-18T11:41:39.828515+0000","last_undegraded":"2023-10-18T11:41:39.828515+0000","last_fullsized":"2023-10-18T11:41:39.828515+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:05:57.235594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.825812+0000","last_change":"2023-10-18T11:41:38.820452+0000","last_active":"2023-10-18T11:41:38.825812+0000","last_peered":"2023-10-18T11:41:38.825812+0000","last_clean":"2023-10-18T11:41:38.825812+0000","last_became_active":"2023-10-18T11:41:38.820159+0000","last_became_peered":"2023-10-18T11:41:38.820159+0000","last_unstale":"2023-10-18T11:41:38.825812+0000","last_undegraded":"2023-10-18T11:41:38.825812+0000","last_fullsized":"2023-10-18T11:41:38.825812+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:30.211327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829649+0000","last_change":"2023-10-18T11:41:38.823794+0000","last_active":"2023-10-18T11:41:38.829649+0000","last_peered":"2023-10-18T11:41:38.829649+0000","last_clean":"2023-10-18T11:41:38.829649+0000","last_became_active":"2023-10-18T11:41:38.823494+0000","last_became_peered":"2023-10-18T11:41:38.823494+0000","last_unstale":"2023-10-18T11:41:38.829649+0000","last_undegraded":"2023-10-18T11:41:38.829649+0000","last_fullsized":"2023-10-18T11:41:38.829649+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:52.835599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.830854+0000","last_change":"2023-10-18T11:41:38.826372+0000","last_active":"2023-10-18T11:41:38.830854+0000","last_peered":"2023-10-18T11:41:38.830854+0000","last_clean":"2023-10-18T11:41:38.830854+0000","last_became_active":"2023-10-18T11:41:38.826126+0000","last_became_peered":"2023-10-18T11:41:38.826126+0000","last_unstale":"2023-10-18T11:41:38.830854+0000","last_undegraded":"2023-10-18T11:41:38.830854+0000","last_fullsized":"2023-10-18T11:41:38.830854+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:49.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828583+0000","last_change":"2023-10-18T11:41:38.823012+0000","last_active":"2023-10-18T11:41:39.828583+0000","last_peered":"2023-10-18T11:41:39.828583+0000","last_clean":"2023-10-18T11:41:39.828583+0000","last_became_active":"2023-10-18T11:41:38.822261+0000","last_became_peered":"2023-10-18T11:41:38.822261+0000","last_unstale":"2023-10-18T11:41:39.828583+0000","last_undegraded":"2023-10-18T11:41:39.828583+0000","last_fullsized":"2023-10-18T11:41:39.828583+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:36.619175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829996+0000","last_change":"2023-10-18T11:41:38.827149+0000","last_active":"2023-10-18T11:41:39.829996+0000","last_peered":"2023-10-18T11:41:39.829996+0000","last_clean":"2023-10-18T11:41:39.829996+0000","last_became_active":"2023-10-18T11:41:38.825505+0000","last_became_peered":"2023-10-18T11:41:38.825505+0000","last_unstale":"2023-10-18T11:41:39.829996+0000","last_undegraded":"2023-10-18T11:41:39.829996+0000","last_fullsized":"2023-10-18T11:41:39.829996+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:06.291395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828541+0000","last_change":"2023-10-18T11:41:38.837766+0000","last_active":"2023-10-18T11:41:39.828541+0000","last_peered":"2023-10-18T11:41:39.828541+0000","last_clean":"2023-10-18T11:41:39.828541+0000","last_became_active":"2023-10-18T11:41:38.837482+0000","last_became_peered":"2023-10-18T11:41:38.837482+0000","last_unstale":"2023-10-18T11:41:39.828541+0000","last_undegraded":"2023-10-18T11:41:39.828541+0000","last_fullsized":"2023-10-18T11:41:39.828541+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:05.028643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835514+0000","last_change":"2023-10-18T11:41:38.831958+0000","last_active":"2023-10-18T11:41:38.835514+0000","last_peered":"2023-10-18T11:41:38.835514+0000","last_clean":"2023-10-18T11:41:38.835514+0000","last_became_active":"2023-10-18T11:41:38.831705+0000","last_became_peered":"2023-10-18T11:41:38.831705+0000","last_unstale":"2023-10-18T11:41:38.835514+0000","last_undegraded":"2023-10-18T11:41:38.835514+0000","last_fullsized":"2023-10-18T11:41:38.835514+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:42.346674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829267+0000","last_change":"2023-10-18T11:41:38.827986+0000","last_active":"2023-10-18T11:41:39.829267+0000","last_peered":"2023-10-18T11:41:39.829267+0000","last_clean":"2023-10-18T11:41:39.829267+0000","last_became_active":"2023-10-18T11:41:38.827661+0000","last_became_peered":"2023-10-18T11:41:38.827661+0000","last_unstale":"2023-10-18T11:41:39.829267+0000","last_undegraded":"2023-10-18T11:41:39.829267+0000","last_fullsized":"2023-10-18T11:41:39.829267+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:51.388492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841809+0000","last_change":"2023-10-18T11:41:38.837107+0000","last_active":"2023-10-18T11:41:38.841809+0000","last_peered":"2023-10-18T11:41:38.841809+0000","last_clean":"2023-10-18T11:41:38.841809+0000","last_became_active":"2023-10-18T11:41:38.836829+0000","last_became_peered":"2023-10-18T11:41:38.836829+0000","last_unstale":"2023-10-18T11:41:38.841809+0000","last_undegraded":"2023-10-18T11:41:38.841809+0000","last_fullsized":"2023-10-18T11:41:38.841809+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:21:03.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"173'4","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.845429+0000","last_change":"2023-10-18T11:41:37.820263+0000","last_active":"2023-10-18T11:41:38.845429+0000","last_peered":"2023-10-18T11:41:38.845429+0000","last_clean":"2023-10-18T11:41:38.845429+0000","last_became_active":"2023-10-18T11:41:37.819933+0000","last_became_peered":"2023-10-18T11:41:37.819933+0000","last_unstale":"2023-10-18T11:41:38.845429+0000","last_undegraded":"2023-10-18T11:41:38.845429+0000","last_fullsized":"2023-10-18T11:41:38.845429+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:41:57.852127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.842464+0000","last_change":"2023-10-18T11:41:38.839828+0000","last_active":"2023-10-18T11:41:38.842464+0000","last_peered":"2023-10-18T11:41:38.842464+0000","last_clean":"2023-10-18T11:41:38.842464+0000","last_became_active":"2023-10-18T11:41:38.838892+0000","last_became_peered":"2023-10-18T11:41:38.838892+0000","last_unstale":"2023-10-18T11:41:38.842464+0000","last_undegraded":"2023-10-18T11:41:38.842464+0000","last_fullsized":"2023-10-18T11:41:38.842464+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:35.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"173'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844405+0000","last_change":"2023-10-18T11:41:37.821129+0000","last_active":"2023-10-18T11:41:38.844405+0000","last_peered":"2023-10-18T11:41:38.844405+0000","last_clean":"2023-10-18T11:41:38.844405+0000","last_became_active":"2023-10-18T11:41:37.820850+0000","last_became_peered":"2023-10-18T11:41:37.820850+0000","last_unstale":"2023-10-18T11:41:38.844405+0000","last_undegraded":"2023-10-18T11:41:38.844405+0000","last_fullsized":"2023-10-18T11:41:38.844405+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:09.641790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829709+0000","last_change":"2023-10-18T11:41:38.831858+0000","last_active":"2023-10-18T11:41:39.829709+0000","last_peered":"2023-10-18T11:41:39.829709+0000","last_clean":"2023-10-18T11:41:39.829709+0000","last_became_active":"2023-10-18T11:41:38.831590+0000","last_became_peered":"2023-10-18T11:41:38.831590+0000","last_unstale":"2023-10-18T11:41:39.829709+0000","last_undegraded":"2023-10-18T11:41:39.829709+0000","last_fullsized":"2023-10-18T11:41:39.829709+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:42.177687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844505+0000","last_change":"2023-10-18T11:41:37.821038+0000","last_active":"2023-10-18T11:41:38.844505+0000","last_peered":"2023-10-18T11:41:38.844505+0000","last_clean":"2023-10-18T11:41:38.844505+0000","last_became_active":"2023-10-18T11:41:37.820759+0000","last_became_peered":"2023-10-18T11:41:37.820759+0000","last_unstale":"2023-10-18T11:41:38.844505+0000","last_undegraded":"2023-10-18T11:41:38.844505+0000","last_fullsized":"2023-10-18T11:41:38.844505+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:48.941548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.833804+0000","last_change":"2023-10-18T11:41:38.830270+0000","last_active":"2023-10-18T11:41:38.833804+0000","last_peered":"2023-10-18T11:41:38.833804+0000","last_clean":"2023-10-18T11:41:38.833804+0000","last_became_active":"2023-10-18T11:41:38.830001+0000","last_became_peered":"2023-10-18T11:41:38.830001+0000","last_unstale":"2023-10-18T11:41:38.833804+0000","last_undegraded":"2023-10-18T11:41:38.833804+0000","last_fullsized":"2023-10-18T11:41:38.833804+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:43.202732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.839014+0000","last_change":"2023-10-18T11:41:37.819635+0000","last_active":"2023-10-18T11:41:38.839014+0000","last_peered":"2023-10-18T11:41:38.839014+0000","last_clean":"2023-10-18T11:41:38.839014+0000","last_became_active":"2023-10-18T11:41:37.819375+0000","last_became_peered":"2023-10-18T11:41:37.819375+0000","last_unstale":"2023-10-18T11:41:38.839014+0000","last_undegraded":"2023-10-18T11:41:38.839014+0000","last_fullsized":"2023-10-18T11:41:38.839014+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:51.830773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.832555+0000","last_change":"2023-10-18T11:41:38.824262+0000","last_active":"2023-10-18T11:41:38.832555+0000","last_peered":"2023-10-18T11:41:38.832555+0000","last_clean":"2023-10-18T11:41:38.832555+0000","last_became_active":"2023-10-18T11:41:38.824005+0000","last_became_peered":"2023-10-18T11:41:38.824005+0000","last_unstale":"2023-10-18T11:41:38.832555+0000","last_undegraded":"2023-10-18T11:41:38.832555+0000","last_fullsized":"2023-10-18T11:41:38.832555+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:34.843533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"173'1","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.830055+0000","last_change":"2023-10-18T11:41:37.826154+0000","last_active":"2023-10-18T11:41:39.830055+0000","last_peered":"2023-10-18T11:41:39.830055+0000","last_clean":"2023-10-18T11:41:39.830055+0000","last_became_active":"2023-10-18T11:41:37.825607+0000","last_became_peered":"2023-10-18T11:41:37.825607+0000","last_unstale":"2023-10-18T11:41:39.830055+0000","last_undegraded":"2023-10-18T11:41:39.830055+0000","last_fullsized":"2023-10-18T11:41:39.830055+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:24.888182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.836951+0000","last_change":"2023-10-18T11:41:38.834243+0000","last_active":"2023-10-18T11:41:38.836951+0000","last_peered":"2023-10-18T11:41:38.836951+0000","last_clean":"2023-10-18T11:41:38.836951+0000","last_became_active":"2023-10-18T11:41:38.833958+0000","last_became_peered":"2023-10-18T11:41:38.833958+0000","last_unstale":"2023-10-18T11:41:38.836951+0000","last_undegraded":"2023-10-18T11:41:38.836951+0000","last_fullsized":"2023-10-18T11:41:38.836951+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:49.235985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"173'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.846614+0000","last_change":"2023-10-18T11:41:37.824548+0000","last_active":"2023-10-18T11:41:38.846614+0000","last_peered":"2023-10-18T11:41:38.846614+0000","last_clean":"2023-10-18T11:41:38.846614+0000","last_became_active":"2023-10-18T11:41:37.824278+0000","last_became_peered":"2023-10-18T11:41:37.824278+0000","last_unstale":"2023-10-18T11:41:38.846614+0000","last_undegraded":"2023-10-18T11:41:38.846614+0000","last_fullsized":"2023-10-18T11:41:38.846614+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:05.442472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.842326+0000","last_change":"2023-10-18T11:41:38.838537+0000","last_active":"2023-10-18T11:41:38.842326+0000","last_peered":"2023-10-18T11:41:38.842326+0000","last_clean":"2023-10-18T11:41:38.842326+0000","last_became_active":"2023-10-18T11:41:38.835109+0000","last_became_peered":"2023-10-18T11:41:38.835109+0000","last_unstale":"2023-10-18T11:41:38.842326+0000","last_undegraded":"2023-10-18T11:41:38.842326+0000","last_fullsized":"2023-10-18T11:41:38.842326+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:12.937570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.815029+0000","last_change":"2023-10-18T11:41:37.820269+0000","last_active":"2023-10-18T11:41:38.815029+0000","last_peered":"2023-10-18T11:41:38.815029+0000","last_clean":"2023-10-18T11:41:38.815029+0000","last_became_active":"2023-10-18T11:41:37.819988+0000","last_became_peered":"2023-10-18T11:41:37.819988+0000","last_unstale":"2023-10-18T11:41:38.815029+0000","last_undegraded":"2023-10-18T11:41:38.815029+0000","last_fullsized":"2023-10-18T11:41:38.815029+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:53.297750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829428+0000","last_change":"2023-10-18T11:41:38.820999+0000","last_active":"2023-10-18T11:41:38.829428+0000","last_peered":"2023-10-18T11:41:38.829428+0000","last_clean":"2023-10-18T11:41:38.829428+0000","last_became_active":"2023-10-18T11:41:38.820731+0000","last_became_peered":"2023-10-18T11:41:38.820731+0000","last_unstale":"2023-10-18T11:41:38.829428+0000","last_undegraded":"2023-10-18T11:41:38.829428+0000","last_fullsized":"2023-10-18T11:41:38.829428+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:12.937570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.818927+0000","last_change":"2023-10-18T11:41:37.824809+0000","last_active":"2023-10-18T11:41:38.818927+0000","last_peered":"2023-10-18T11:41:38.818927+0000","last_clean":"2023-10-18T11:41:38.818927+0000","last_became_active":"2023-10-18T11:41:37.824509+0000","last_became_peered":"2023-10-18T11:41:37.824509+0000","last_unstale":"2023-10-18T11:41:38.818927+0000","last_undegraded":"2023-10-18T11:41:38.818927+0000","last_fullsized":"2023-10-18T11:41:38.818927+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:50.760177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829761+0000","last_change":"2023-10-18T11:41:38.827847+0000","last_active":"2023-10-18T11:41:39.829761+0000","last_peered":"2023-10-18T11:41:39.829761+0000","last_clean":"2023-10-18T11:41:39.829761+0000","last_became_active":"2023-10-18T11:41:38.827553+0000","last_became_peered":"2023-10-18T11:41:38.827553+0000","last_unstale":"2023-10-18T11:41:39.829761+0000","last_undegraded":"2023-10-18T11:41:39.829761+0000","last_fullsized":"2023-10-18T11:41:39.829761+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:54:44.886016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.822854+0000","last_change":"2023-10-18T11:41:37.825094+0000","last_active":"2023-10-18T11:41:38.822854+0000","last_peered":"2023-10-18T11:41:38.822854+0000","last_clean":"2023-10-18T11:41:38.822854+0000","last_became_active":"2023-10-18T11:41:37.824553+0000","last_became_peered":"2023-10-18T11:41:37.824553+0000","last_unstale":"2023-10-18T11:41:38.822854+0000","last_undegraded":"2023-10-18T11:41:38.822854+0000","last_fullsized":"2023-10-18T11:41:38.822854+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:36.961770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828711+0000","last_change":"2023-10-18T11:41:38.833135+0000","last_active":"2023-10-18T11:41:39.828711+0000","last_peered":"2023-10-18T11:41:39.828711+0000","last_clean":"2023-10-18T11:41:39.828711+0000","last_became_active":"2023-10-18T11:41:38.832708+0000","last_became_peered":"2023-10-18T11:41:38.832708+0000","last_unstale":"2023-10-18T11:41:39.828711+0000","last_undegraded":"2023-10-18T11:41:39.828711+0000","last_fullsized":"2023-10-18T11:41:39.828711+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:39.928349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.816460+0000","last_change":"2023-10-18T11:41:37.814357+0000","last_active":"2023-10-18T11:41:38.816460+0000","last_peered":"2023-10-18T11:41:38.816460+0000","last_clean":"2023-10-18T11:41:38.816460+0000","last_became_active":"2023-10-18T11:41:37.814082+0000","last_became_peered":"2023-10-18T11:41:37.814082+0000","last_unstale":"2023-10-18T11:41:38.816460+0000","last_undegraded":"2023-10-18T11:41:38.816460+0000","last_fullsized":"2023-10-18T11:41:38.816460+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:41.460284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828728+0000","last_change":"2023-10-18T11:41:38.827430+0000","last_active":"2023-10-18T11:41:39.828728+0000","last_peered":"2023-10-18T11:41:39.828728+0000","last_clean":"2023-10-18T11:41:39.828728+0000","last_became_active":"2023-10-18T11:41:38.827151+0000","last_became_peered":"2023-10-18T11:41:38.827151+0000","last_unstale":"2023-10-18T11:41:39.828728+0000","last_undegraded":"2023-10-18T11:41:39.828728+0000","last_fullsized":"2023-10-18T11:41:39.828728+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:57.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"173'3","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828816+0000","last_change":"2023-10-18T11:41:37.816938+0000","last_active":"2023-10-18T11:41:39.828816+0000","last_peered":"2023-10-18T11:41:39.828816+0000","last_clean":"2023-10-18T11:41:39.828816+0000","last_became_active":"2023-10-18T11:41:37.816710+0000","last_became_peered":"2023-10-18T11:41:37.816710+0000","last_unstale":"2023-10-18T11:41:39.828816+0000","last_undegraded":"2023-10-18T11:41:39.828816+0000","last_fullsized":"2023-10-18T11:41:39.828816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:21:55.432322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829426+0000","last_change":"2023-10-18T11:41:38.827972+0000","last_active":"2023-10-18T11:41:39.829426+0000","last_peered":"2023-10-18T11:41:39.829426+0000","last_clean":"2023-10-18T11:41:39.829426+0000","last_became_active":"2023-10-18T11:41:38.827700+0000","last_became_peered":"2023-10-18T11:41:38.827700+0000","last_unstale":"2023-10-18T11:41:39.829426+0000","last_undegraded":"2023-10-18T11:41:39.829426+0000","last_fullsized":"2023-10-18T11:41:39.829426+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:29.998896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.845165+0000","last_change":"2023-10-18T11:41:37.819453+0000","last_active":"2023-10-18T11:41:38.845165+0000","last_peered":"2023-10-18T11:41:38.845165+0000","last_clean":"2023-10-18T11:41:38.845165+0000","last_became_active":"2023-10-18T11:41:37.819216+0000","last_became_peered":"2023-10-18T11:41:37.819216+0000","last_unstale":"2023-10-18T11:41:38.845165+0000","last_undegraded":"2023-10-18T11:41:38.845165+0000","last_fullsized":"2023-10-18T11:41:38.845165+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:00.103896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841777+0000","last_change":"2023-10-18T11:41:38.836803+0000","last_active":"2023-10-18T11:41:38.841777+0000","last_peered":"2023-10-18T11:41:38.841777+0000","last_clean":"2023-10-18T11:41:38.841777+0000","last_became_active":"2023-10-18T11:41:38.836239+0000","last_became_peered":"2023-10-18T11:41:38.836239+0000","last_unstale":"2023-10-18T11:41:38.841777+0000","last_undegraded":"2023-10-18T11:41:38.841777+0000","last_fullsized":"2023-10-18T11:41:38.841777+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:54.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.823799+0000","last_change":"2023-10-18T11:41:37.823427+0000","last_active":"2023-10-18T11:41:38.823799+0000","last_peered":"2023-10-18T11:41:38.823799+0000","last_clean":"2023-10-18T11:41:38.823799+0000","last_became_active":"2023-10-18T11:41:37.823126+0000","last_became_peered":"2023-10-18T11:41:37.823126+0000","last_unstale":"2023-10-18T11:41:38.823799+0000","last_undegraded":"2023-10-18T11:41:38.823799+0000","last_fullsized":"2023-10-18T11:41:38.823799+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:48.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.828098+0000","last_change":"2023-10-18T11:41:38.825410+0000","last_active":"2023-10-18T11:41:38.828098+0000","last_peered":"2023-10-18T11:41:38.828098+0000","last_clean":"2023-10-18T11:41:38.828098+0000","last_became_active":"2023-10-18T11:41:38.825132+0000","last_became_peered":"2023-10-18T11:41:38.825132+0000","last_unstale":"2023-10-18T11:41:38.828098+0000","last_undegraded":"2023-10-18T11:41:38.828098+0000","last_fullsized":"2023-10-18T11:41:38.828098+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:46.074774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.824065+0000","last_change":"2023-10-18T11:41:37.823660+0000","last_active":"2023-10-18T11:41:38.824065+0000","last_peered":"2023-10-18T11:41:38.824065+0000","last_clean":"2023-10-18T11:41:38.824065+0000","last_became_active":"2023-10-18T11:41:37.823201+0000","last_became_peered":"2023-10-18T11:41:37.823201+0000","last_unstale":"2023-10-18T11:41:38.824065+0000","last_undegraded":"2023-10-18T11:41:38.824065+0000","last_fullsized":"2023-10-18T11:41:38.824065+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:12.705578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.846070+0000","last_change":"2023-10-18T11:41:38.844785+0000","last_active":"2023-10-18T11:41:38.846070+0000","last_peered":"2023-10-18T11:41:38.846070+0000","last_clean":"2023-10-18T11:41:38.846070+0000","last_became_active":"2023-10-18T11:41:38.844502+0000","last_became_peered":"2023-10-18T11:41:38.844502+0000","last_unstale":"2023-10-18T11:41:38.846070+0000","last_undegraded":"2023-10-18T11:41:38.846070+0000","last_fullsized":"2023-10-18T11:41:38.846070+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:20.319797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"173'8","reported_seq":19,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844653+0000","last_change":"2023-10-18T11:41:37.822141+0000","last_active":"2023-10-18T11:41:38.844653+0000","last_peered":"2023-10-18T11:41:38.844653+0000","last_clean":"2023-10-18T11:41:38.844653+0000","last_became_active":"2023-10-18T11:41:37.821819+0000","last_became_peered":"2023-10-18T11:41:37.821819+0000","last_unstale":"2023-10-18T11:41:38.844653+0000","last_undegraded":"2023-10-18T11:41:38.844653+0000","last_fullsized":"2023-10-18T11:41:38.844653+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:19.314971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.840706+0000","last_change":"2023-10-18T11:41:38.837476+0000","last_active":"2023-10-18T11:41:38.840706+0000","last_peered":"2023-10-18T11:41:38.840706+0000","last_clean":"2023-10-18T11:41:38.840706+0000","last_became_active":"2023-10-18T11:41:38.836917+0000","last_became_peered":"2023-10-18T11:41:38.836917+0000","last_unstale":"2023-10-18T11:41:38.840706+0000","last_undegraded":"2023-10-18T11:41:38.840706+0000","last_fullsized":"2023-10-18T11:41:38.840706+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:27.983985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"173'1","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828619+0000","last_change":"2023-10-18T11:41:37.821090+0000","last_active":"2023-10-18T11:41:39.828619+0000","last_peered":"2023-10-18T11:41:39.828619+0000","last_clean":"2023-10-18T11:41:39.828619+0000","last_became_active":"2023-10-18T11:41:37.820845+0000","last_became_peered":"2023-10-18T11:41:37.820845+0000","last_unstale":"2023-10-18T11:41:39.828619+0000","last_undegraded":"2023-10-18T11:41:39.828619+0000","last_fullsized":"2023-10-18T11:41:39.828619+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:15.610937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835699+0000","last_change":"2023-10-18T11:41:38.832709+0000","last_active":"2023-10-18T11:41:38.835699+0000","last_peered":"2023-10-18T11:41:38.835699+0000","last_clean":"2023-10-18T11:41:38.835699+0000","last_became_active":"2023-10-18T11:41:38.832436+0000","last_became_peered":"2023-10-18T11:41:38.832436+0000","last_unstale":"2023-10-18T11:41:38.835699+0000","last_undegraded":"2023-10-18T11:41:38.835699+0000","last_fullsized":"2023-10-18T11:41:38.835699+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:55.448471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.845756+0000","last_change":"2023-10-18T11:41:37.823846+0000","last_active":"2023-10-18T11:41:38.845756+0000","last_peered":"2023-10-18T11:41:38.845756+0000","last_clean":"2023-10-18T11:41:38.845756+0000","last_became_active":"2023-10-18T11:41:37.822544+0000","last_became_peered":"2023-10-18T11:41:37.822544+0000","last_unstale":"2023-10-18T11:41:38.845756+0000","last_undegraded":"2023-10-18T11:41:38.845756+0000","last_fullsized":"2023-10-18T11:41:38.845756+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:01.806142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.829630+0000","last_change":"2023-10-18T11:41:38.822398+0000","last_active":"2023-10-18T11:41:38.829630+0000","last_peered":"2023-10-18T11:41:38.829630+0000","last_clean":"2023-10-18T11:41:38.829630+0000","last_became_active":"2023-10-18T11:41:38.822156+0000","last_became_peered":"2023-10-18T11:41:38.822156+0000","last_unstale":"2023-10-18T11:41:38.829630+0000","last_undegraded":"2023-10-18T11:41:38.829630+0000","last_fullsized":"2023-10-18T11:41:38.829630+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:17.259402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"173'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.839691+0000","last_change":"2023-10-18T11:41:37.824435+0000","last_active":"2023-10-18T11:41:38.839691+0000","last_peered":"2023-10-18T11:41:38.839691+0000","last_clean":"2023-10-18T11:41:38.839691+0000","last_became_active":"2023-10-18T11:41:37.823886+0000","last_became_peered":"2023-10-18T11:41:37.823886+0000","last_unstale":"2023-10-18T11:41:38.839691+0000","last_undegraded":"2023-10-18T11:41:38.839691+0000","last_fullsized":"2023-10-18T11:41:38.839691+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:19.194946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835456+0000","last_change":"2023-10-18T11:41:38.833975+0000","last_active":"2023-10-18T11:41:38.835456+0000","last_peered":"2023-10-18T11:41:38.835456+0000","last_clean":"2023-10-18T11:41:38.835456+0000","last_became_active":"2023-10-18T11:41:38.833708+0000","last_became_peered":"2023-10-18T11:41:38.833708+0000","last_unstale":"2023-10-18T11:41:38.835456+0000","last_undegraded":"2023-10-18T11:41:38.835456+0000","last_fullsized":"2023-10-18T11:41:38.835456+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:13.225614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.824069+0000","last_change":"2023-10-18T11:41:37.823230+0000","last_active":"2023-10-18T11:41:38.824069+0000","last_peered":"2023-10-18T11:41:38.824069+0000","last_clean":"2023-10-18T11:41:38.824069+0000","last_became_active":"2023-10-18T11:41:37.822954+0000","last_became_peered":"2023-10-18T11:41:37.822954+0000","last_unstale":"2023-10-18T11:41:38.824069+0000","last_undegraded":"2023-10-18T11:41:38.824069+0000","last_fullsized":"2023-10-18T11:41:38.824069+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:44.059048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.835337+0000","last_change":"2023-10-18T11:41:38.832536+0000","last_active":"2023-10-18T11:41:38.835337+0000","last_peered":"2023-10-18T11:41:38.835337+0000","last_clean":"2023-10-18T11:41:38.835337+0000","last_became_active":"2023-10-18T11:41:38.832268+0000","last_became_peered":"2023-10-18T11:41:38.832268+0000","last_unstale":"2023-10-18T11:41:38.835337+0000","last_undegraded":"2023-10-18T11:41:38.835337+0000","last_fullsized":"2023-10-18T11:41:38.835337+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:36.452850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.846834+0000","last_change":"2023-10-18T11:41:37.822283+0000","last_active":"2023-10-18T11:41:38.846834+0000","last_peered":"2023-10-18T11:41:38.846834+0000","last_clean":"2023-10-18T11:41:38.846834+0000","last_became_active":"2023-10-18T11:41:37.822019+0000","last_became_peered":"2023-10-18T11:41:37.822019+0000","last_unstale":"2023-10-18T11:41:38.846834+0000","last_undegraded":"2023-10-18T11:41:38.846834+0000","last_fullsized":"2023-10-18T11:41:38.846834+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:27.346874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841742+0000","last_change":"2023-10-18T11:41:38.835474+0000","last_active":"2023-10-18T11:41:38.841742+0000","last_peered":"2023-10-18T11:41:38.841742+0000","last_clean":"2023-10-18T11:41:38.841742+0000","last_became_active":"2023-10-18T11:41:38.835205+0000","last_became_peered":"2023-10-18T11:41:38.835205+0000","last_unstale":"2023-10-18T11:41:38.841742+0000","last_undegraded":"2023-10-18T11:41:38.841742+0000","last_fullsized":"2023-10-18T11:41:38.841742+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:11:32.696938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829423+0000","last_change":"2023-10-18T11:41:37.819210+0000","last_active":"2023-10-18T11:41:39.829423+0000","last_peered":"2023-10-18T11:41:39.829423+0000","last_clean":"2023-10-18T11:41:39.829423+0000","last_became_active":"2023-10-18T11:41:37.818926+0000","last_became_peered":"2023-10-18T11:41:37.818926+0000","last_unstale":"2023-10-18T11:41:39.829423+0000","last_undegraded":"2023-10-18T11:41:39.829423+0000","last_fullsized":"2023-10-18T11:41:39.829423+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:15.032324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.833763+0000","last_change":"2023-10-18T11:41:38.829408+0000","last_active":"2023-10-18T11:41:38.833763+0000","last_peered":"2023-10-18T11:41:38.833763+0000","last_clean":"2023-10-18T11:41:38.833763+0000","last_became_active":"2023-10-18T11:41:38.828917+0000","last_became_peered":"2023-10-18T11:41:38.828917+0000","last_unstale":"2023-10-18T11:41:38.833763+0000","last_undegraded":"2023-10-18T11:41:38.833763+0000","last_fullsized":"2023-10-18T11:41:38.833763+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:46.074774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.817051+0000","last_change":"2023-10-18T11:41:37.819830+0000","last_active":"2023-10-18T11:41:38.817051+0000","last_peered":"2023-10-18T11:41:38.817051+0000","last_clean":"2023-10-18T11:41:38.817051+0000","last_became_active":"2023-10-18T11:41:37.819576+0000","last_became_peered":"2023-10-18T11:41:37.819576+0000","last_unstale":"2023-10-18T11:41:38.817051+0000","last_undegraded":"2023-10-18T11:41:38.817051+0000","last_fullsized":"2023-10-18T11:41:38.817051+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:54.501911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841674+0000","last_change":"2023-10-18T11:41:38.837848+0000","last_active":"2023-10-18T11:41:38.841674+0000","last_peered":"2023-10-18T11:41:38.841674+0000","last_clean":"2023-10-18T11:41:38.841674+0000","last_became_active":"2023-10-18T11:41:38.837607+0000","last_became_peered":"2023-10-18T11:41:38.837607+0000","last_unstale":"2023-10-18T11:41:38.841674+0000","last_undegraded":"2023-10-18T11:41:38.841674+0000","last_fullsized":"2023-10-18T11:41:38.841674+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:13:56.797253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829767+0000","last_change":"2023-10-18T11:41:37.818588+0000","last_active":"2023-10-18T11:41:39.829767+0000","last_peered":"2023-10-18T11:41:39.829767+0000","last_clean":"2023-10-18T11:41:39.829767+0000","last_became_active":"2023-10-18T11:41:37.817798+0000","last_became_peered":"2023-10-18T11:41:37.817798+0000","last_unstale":"2023-10-18T11:41:39.829767+0000","last_undegraded":"2023-10-18T11:41:39.829767+0000","last_fullsized":"2023-10-18T11:41:39.829767+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:50:52.183260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.843869+0000","last_change":"2023-10-18T11:41:38.842544+0000","last_active":"2023-10-18T11:41:38.843869+0000","last_peered":"2023-10-18T11:41:38.843869+0000","last_clean":"2023-10-18T11:41:38.843869+0000","last_became_active":"2023-10-18T11:41:38.841060+0000","last_became_peered":"2023-10-18T11:41:38.841060+0000","last_unstale":"2023-10-18T11:41:38.843869+0000","last_undegraded":"2023-10-18T11:41:38.843869+0000","last_fullsized":"2023-10-18T11:41:38.843869+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:33.088719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"173'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.844520+0000","last_change":"2023-10-18T11:41:37.823910+0000","last_active":"2023-10-18T11:41:38.844520+0000","last_peered":"2023-10-18T11:41:38.844520+0000","last_clean":"2023-10-18T11:41:38.844520+0000","last_became_active":"2023-10-18T11:41:37.823671+0000","last_became_peered":"2023-10-18T11:41:37.823671+0000","last_unstale":"2023-10-18T11:41:38.844520+0000","last_undegraded":"2023-10-18T11:41:38.844520+0000","last_fullsized":"2023-10-18T11:41:38.844520+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:15.970084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.836949+0000","last_change":"2023-10-18T11:41:38.834278+0000","last_active":"2023-10-18T11:41:38.836949+0000","last_peered":"2023-10-18T11:41:38.836949+0000","last_clean":"2023-10-18T11:41:38.836949+0000","last_became_active":"2023-10-18T11:41:38.833972+0000","last_became_peered":"2023-10-18T11:41:38.833972+0000","last_unstale":"2023-10-18T11:41:38.836949+0000","last_undegraded":"2023-10-18T11:41:38.836949+0000","last_fullsized":"2023-10-18T11:41:38.836949+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:28.199054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.816866+0000","last_change":"2023-10-18T11:41:37.823322+0000","last_active":"2023-10-18T11:41:38.816866+0000","last_peered":"2023-10-18T11:41:38.816866+0000","last_clean":"2023-10-18T11:41:38.816866+0000","last_became_active":"2023-10-18T11:41:37.822856+0000","last_became_peered":"2023-10-18T11:41:37.822856+0000","last_unstale":"2023-10-18T11:41:38.816866+0000","last_undegraded":"2023-10-18T11:41:38.816866+0000","last_fullsized":"2023-10-18T11:41:38.816866+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:52.515374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831189+0000","last_change":"2023-10-18T11:41:38.827748+0000","last_active":"2023-10-18T11:41:38.831189+0000","last_peered":"2023-10-18T11:41:38.831189+0000","last_clean":"2023-10-18T11:41:38.831189+0000","last_became_active":"2023-10-18T11:41:38.827469+0000","last_became_peered":"2023-10-18T11:41:38.827469+0000","last_unstale":"2023-10-18T11:41:38.831189+0000","last_undegraded":"2023-10-18T11:41:38.831189+0000","last_fullsized":"2023-10-18T11:41:38.831189+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:09.333393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.831053+0000","last_change":"2023-10-18T11:41:37.821788+0000","last_active":"2023-10-18T11:41:38.831053+0000","last_peered":"2023-10-18T11:41:38.831053+0000","last_clean":"2023-10-18T11:41:38.831053+0000","last_became_active":"2023-10-18T11:41:37.821313+0000","last_became_peered":"2023-10-18T11:41:37.821313+0000","last_unstale":"2023-10-18T11:41:38.831053+0000","last_undegraded":"2023-10-18T11:41:38.831053+0000","last_fullsized":"2023-10-18T11:41:38.831053+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:29.206502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841722+0000","last_change":"2023-10-18T11:41:38.835302+0000","last_active":"2023-10-18T11:41:38.841722+0000","last_peered":"2023-10-18T11:41:38.841722+0000","last_clean":"2023-10-18T11:41:38.841722+0000","last_became_active":"2023-10-18T11:41:38.835040+0000","last_became_peered":"2023-10-18T11:41:38.835040+0000","last_unstale":"2023-10-18T11:41:38.841722+0000","last_undegraded":"2023-10-18T11:41:38.841722+0000","last_fullsized":"2023-10-18T11:41:38.841722+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.810967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.819109+0000","last_change":"2023-10-18T11:41:37.822147+0000","last_active":"2023-10-18T11:41:38.819109+0000","last_peered":"2023-10-18T11:41:38.819109+0000","last_clean":"2023-10-18T11:41:38.819109+0000","last_became_active":"2023-10-18T11:41:37.821839+0000","last_became_peered":"2023-10-18T11:41:37.821839+0000","last_unstale":"2023-10-18T11:41:38.819109+0000","last_undegraded":"2023-10-18T11:41:38.819109+0000","last_fullsized":"2023-10-18T11:41:38.819109+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:54.443645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829996+0000","last_change":"2023-10-18T11:41:38.821087+0000","last_active":"2023-10-18T11:41:39.829996+0000","last_peered":"2023-10-18T11:41:39.829996+0000","last_clean":"2023-10-18T11:41:39.829996+0000","last_became_active":"2023-10-18T11:41:38.820806+0000","last_became_peered":"2023-10-18T11:41:38.820806+0000","last_unstale":"2023-10-18T11:41:39.829996+0000","last_undegraded":"2023-10-18T11:41:39.829996+0000","last_fullsized":"2023-10-18T11:41:39.829996+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:39.471952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.814926+0000","last_change":"2023-10-18T11:41:37.824985+0000","last_active":"2023-10-18T11:41:38.814926+0000","last_peered":"2023-10-18T11:41:38.814926+0000","last_clean":"2023-10-18T11:41:38.814926+0000","last_became_active":"2023-10-18T11:41:37.824481+0000","last_became_peered":"2023-10-18T11:41:37.824481+0000","last_unstale":"2023-10-18T11:41:38.814926+0000","last_undegraded":"2023-10-18T11:41:38.814926+0000","last_fullsized":"2023-10-18T11:41:38.814926+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:39.668889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.834364+0000","last_change":"2023-10-18T11:41:38.830152+0000","last_active":"2023-10-18T11:41:38.834364+0000","last_peered":"2023-10-18T11:41:38.834364+0000","last_clean":"2023-10-18T11:41:38.834364+0000","last_became_active":"2023-10-18T11:41:38.829828+0000","last_became_peered":"2023-10-18T11:41:38.829828+0000","last_unstale":"2023-10-18T11:41:38.834364+0000","last_undegraded":"2023-10-18T11:41:38.834364+0000","last_fullsized":"2023-10-18T11:41:38.834364+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:01:40.285325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828507+0000","last_change":"2023-10-18T11:41:37.820903+0000","last_active":"2023-10-18T11:41:39.828507+0000","last_peered":"2023-10-18T11:41:39.828507+0000","last_clean":"2023-10-18T11:41:39.828507+0000","last_became_active":"2023-10-18T11:41:37.820650+0000","last_became_peered":"2023-10-18T11:41:37.820650+0000","last_unstale":"2023-10-18T11:41:39.828507+0000","last_undegraded":"2023-10-18T11:41:39.828507+0000","last_fullsized":"2023-10-18T11:41:39.828507+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:11.753576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.841714+0000","last_change":"2023-10-18T11:41:38.838162+0000","last_active":"2023-10-18T11:41:38.841714+0000","last_peered":"2023-10-18T11:41:38.841714+0000","last_clean":"2023-10-18T11:41:38.841714+0000","last_became_active":"2023-10-18T11:41:38.836514+0000","last_became_peered":"2023-10-18T11:41:38.836514+0000","last_unstale":"2023-10-18T11:41:38.841714+0000","last_undegraded":"2023-10-18T11:41:38.841714+0000","last_fullsized":"2023-10-18T11:41:38.841714+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:27:26.755708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.815291+0000","last_change":"2023-10-18T11:41:37.814973+0000","last_active":"2023-10-18T11:41:38.815291+0000","last_peered":"2023-10-18T11:41:38.815291+0000","last_clean":"2023-10-18T11:41:38.815291+0000","last_became_active":"2023-10-18T11:41:37.814699+0000","last_became_peered":"2023-10-18T11:41:37.814699+0000","last_unstale":"2023-10-18T11:41:38.815291+0000","last_undegraded":"2023-10-18T11:41:38.815291+0000","last_fullsized":"2023-10-18T11:41:38.815291+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:19.194946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829747+0000","last_change":"2023-10-18T11:41:38.826161+0000","last_active":"2023-10-18T11:41:39.829747+0000","last_peered":"2023-10-18T11:41:39.829747+0000","last_clean":"2023-10-18T11:41:39.829747+0000","last_became_active":"2023-10-18T11:41:38.825615+0000","last_became_peered":"2023-10-18T11:41:38.825615+0000","last_unstale":"2023-10-18T11:41:39.829747+0000","last_undegraded":"2023-10-18T11:41:39.829747+0000","last_fullsized":"2023-10-18T11:41:39.829747+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:47.816689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.822205+0000","last_change":"2023-10-18T11:41:37.822249+0000","last_active":"2023-10-18T11:41:38.822205+0000","last_peered":"2023-10-18T11:41:38.822205+0000","last_clean":"2023-10-18T11:41:38.822205+0000","last_became_active":"2023-10-18T11:41:37.821714+0000","last_became_peered":"2023-10-18T11:41:37.821714+0000","last_unstale":"2023-10-18T11:41:38.822205+0000","last_undegraded":"2023-10-18T11:41:38.822205+0000","last_fullsized":"2023-10-18T11:41:38.822205+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:24.518814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829803+0000","last_change":"2023-10-18T11:41:38.836422+0000","last_active":"2023-10-18T11:41:39.829803+0000","last_peered":"2023-10-18T11:41:39.829803+0000","last_clean":"2023-10-18T11:41:39.829803+0000","last_became_active":"2023-10-18T11:41:38.835474+0000","last_became_peered":"2023-10-18T11:41:38.835474+0000","last_unstale":"2023-10-18T11:41:39.829803+0000","last_undegraded":"2023-10-18T11:41:39.829803+0000","last_fullsized":"2023-10-18T11:41:39.829803+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:38.697917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"173'6","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.848425+0000","last_change":"2023-10-18T11:41:37.820513+0000","last_active":"2023-10-18T11:41:38.848425+0000","last_peered":"2023-10-18T11:41:38.848425+0000","last_clean":"2023-10-18T11:41:38.848425+0000","last_became_active":"2023-10-18T11:41:37.820194+0000","last_became_peered":"2023-10-18T11:41:37.820194+0000","last_unstale":"2023-10-18T11:41:38.848425+0000","last_undegraded":"2023-10-18T11:41:38.848425+0000","last_fullsized":"2023-10-18T11:41:38.848425+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:58.848022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"162'280","reported_seq":911,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-18T11:41:37.815051+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:41:37.815051+0000","last_peered":"2023-10-18T11:41:37.815051+0000","last_clean":"2023-10-18T11:41:37.815051+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:41:37.815051+0000","last_undegraded":"2023-10-18T11:41:37.815051+0000","last_fullsized":"2023-10-18T11:41:37.815051+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575768,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51928,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319912,"statfs":{"total":96636764160,"available":96583589888,"internally_reserved":0,"allocated":2760704,"data_stored":1687384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.626},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.47699999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.64600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575768,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37460,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334380,"statfs":{"total":96636764160,"available":96598405120,"internally_reserved":0,"allocated":2166784,"data_stored":1097064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.375}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53300000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.442},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49399999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56000000000000005}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575768,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59660,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312180,"statfs":{"total":96636764160,"available":96575672320,"internally_reserved":0,"allocated":2162688,"data_stored":1096994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.54200000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.86199999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.84599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575768,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41640,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330200,"statfs":{"total":96636764160,"available":96594124800,"internally_reserved":0,"allocated":2187264,"data_stored":1105254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.27800000000000002},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.626,"15min":0.623},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.86399999999999999},"last":0.47399999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84599999999999997,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.30499999999999999},"max":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.33000000000000002},"max":{"1min":0.77000000000000002,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29999999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58899999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.36199999999999999},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.80800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575768,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":2170880,"data_stored":1097610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.31},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.34799999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.49099999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.45900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575768,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59684,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312156,"statfs":{"total":96636764160,"available":96575647744,"internally_reserved":0,"allocated":2183168,"data_stored":1103754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.28999999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.32700000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.74299999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.622}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575768,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2166784,"data_stored":1097064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.24299999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.222},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.29699999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.318},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.27600000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.73499999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.439},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.5}]}]},{"osd":0,"up_from":13,"seq":55834575767,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2781184,"data_stored":1689680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:41:43.797 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T11:41:43.794+0000 7f5b41f3a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T11:41:46.452 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:41:46.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.567+0000 7f123dc7c640 1 -- 172.21.15.125:0/2330136317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 msgr2=0x7f1224003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:46.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.567+0000 7f123dc7c640 1 --2- 172.21.15.125:0/2330136317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 0x7f1224003d90 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f1228001680 tx=0x7f12240057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:46.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.567+0000 7f123dc7c640 1 -- 172.21.15.125:0/2330136317 shutdown_connections 2023-10-18T11:41:46.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.567+0000 7f123dc7c640 1 --2- 172.21.15.125:0/2330136317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 0x7f1224003d90 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:46.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.567+0000 7f123dc7c640 1 -- 172.21.15.125:0/2330136317 >> 172.21.15.125:0/2330136317 conn(0x7f1238069930 msgr2=0x7f1238069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:46.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.567+0000 7f123dc7c640 1 -- 172.21.15.125:0/2330136317 shutdown_connections 2023-10-18T11:41:46.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.567+0000 7f123dc7c640 1 -- 172.21.15.125:0/2330136317 wait complete. 2023-10-18T11:41:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.568+0000 7f123dc7c640 1 Processor -- start 2023-10-18T11:41:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f123dc7c640 1 -- start start 2023-10-18T11:41:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f123dc7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 0x7f123813ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f123dc7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1238129c20 0x7f12381462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f123dc7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1238146810 0x7f1238140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f123dc7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123813f480 con 0x7f1224003970 2023-10-18T11:41:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f123dc7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1238127ac0 con 0x7f1238146810 2023-10-18T11:41:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f123dc7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1238127da0 con 0x7f1238129c20 2023-10-18T11:41:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f12377fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 0x7f123813ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f12377fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 0x7f123813ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56654/0 (socket says 172.21.15.125:56654) 2023-10-18T11:41:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.569+0000 7f12377fe640 1 -- 172.21.15.125:0/496183083 learned_addr learned my addr 172.21.15.125:0/496183083 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f12377fe640 1 -- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1238129c20 msgr2=0x7f12381462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f12377fe640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1238129c20 0x7f12381462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f12377fe640 1 -- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1238146810 msgr2=0x7f1238140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f1237fff640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1238146810 0x7f1238140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f12377fe640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1238146810 0x7f1238140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f12377fe640 1 -- 172.21.15.125:0/496183083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238128080 con 0x7f1224003970 2023-10-18T11:41:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f12377fe640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 0x7f123813ef40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f122802c0d0 tx=0x7f1228006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f1234ff9640 1 -- 172.21.15.125:0/496183083 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228006540 con 0x7f1224003970 2023-10-18T11:41:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f1234ff9640 1 -- 172.21.15.125:0/496183083 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1228003c70 con 0x7f1224003970 2023-10-18T11:41:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1238128360 con 0x7f1224003970 2023-10-18T11:41:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f1234ff9640 1 -- 172.21.15.125:0/496183083 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122802c790 con 0x7f1224003970 2023-10-18T11:41:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.570+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123812a6b0 con 0x7f1224003970 2023-10-18T11:41:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.571+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123812a840 con 0x7f1224003970 2023-10-18T11:41:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.571+0000 7f1234ff9640 1 -- 172.21.15.125:0/496183083 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f122802c8f0 con 0x7f1224003970 2023-10-18T11:41:46.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.572+0000 7f1234ff9640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f121806af60 0x7f121806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:46.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.572+0000 7f1234ff9640 1 -- 172.21.15.125:0/496183083 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f12280aaa10 con 0x7f1224003970 2023-10-18T11:41:46.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.572+0000 7f122effd640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f121806af60 0x7f121806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:46.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.573+0000 7f122effd640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f121806af60 0x7f121806d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1238066af0 tx=0x7f1220009530 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:46.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.576+0000 7f1234ff9640 1 -- 172.21.15.125:0/496183083 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1228075850 con 0x7f1224003970 2023-10-18T11:41:46.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.742+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 --> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f123813f480 con 0x7f121806af60 2023-10-18T11:41:46.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.749+0000 7f1234ff9640 1 -- 172.21.15.125:0/496183083 <== mgr.8644 v2:172.21.15.139:6832/4236255547 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290209 (secure 0 0 0) 0x7f123813f480 con 0x7f121806af60 2023-10-18T11:41:47.243 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:47.247 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:41:47.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f121806af60 msgr2=0x7f121806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f121806af60 0x7f121806d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1238066af0 tx=0x7f1220009530 comp rx=0 tx=0).stop 2023-10-18T11:41:47.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 msgr2=0x7f123813ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 0x7f123813ef40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f122802c0d0 tx=0x7f1228006360 comp rx=0 tx=0).stop 2023-10-18T11:41:47.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 shutdown_connections 2023-10-18T11:41:47.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f121806af60 0x7f121806d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1238146810 0x7f1238140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1238129c20 0x7f12381462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 --2- 172.21.15.125:0/496183083 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1224003970 0x7f123813ef40 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 >> 172.21.15.125:0/496183083 conn(0x7f1238069930 msgr2=0x7f123806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:47.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 shutdown_connections 2023-10-18T11:41:47.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:46.752+0000 7f123dc7c640 1 -- 172.21.15.125:0/496183083 wait complete. 2023-10-18T11:41:47.260 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-18T11:41:46.433408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":500,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":319,"ondisk_log_size":319,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385628,"kb_used_data":18216,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754589092,"statfs":{"total":773094113280,"available":772699230208,"internally_reserved":0,"allocated":18653184,"data_stored":10014692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002035"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632447+0000","last_change":"2023-10-18T11:41:38.825716+0000","last_active":"2023-10-18T11:41:40.632447+0000","last_peered":"2023-10-18T11:41:40.632447+0000","last_clean":"2023-10-18T11:41:40.632447+0000","last_became_active":"2023-10-18T11:41:38.825425+0000","last_became_peered":"2023-10-18T11:41:38.825425+0000","last_unstale":"2023-10-18T11:41:40.632447+0000","last_undegraded":"2023-10-18T11:41:40.632447+0000","last_fullsized":"2023-10-18T11:41:40.632447+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:29.459089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632331+0000","last_change":"2023-10-18T11:41:38.837347+0000","last_active":"2023-10-18T11:41:40.632331+0000","last_peered":"2023-10-18T11:41:40.632331+0000","last_clean":"2023-10-18T11:41:40.632331+0000","last_became_active":"2023-10-18T11:41:38.837097+0000","last_became_peered":"2023-10-18T11:41:38.837097+0000","last_unstale":"2023-10-18T11:41:40.632331+0000","last_undegraded":"2023-10-18T11:41:40.632331+0000","last_fullsized":"2023-10-18T11:41:40.632331+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:17.259402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632267+0000","last_change":"2023-10-18T11:41:38.840032+0000","last_active":"2023-10-18T11:41:40.632267+0000","last_peered":"2023-10-18T11:41:40.632267+0000","last_clean":"2023-10-18T11:41:40.632267+0000","last_became_active":"2023-10-18T11:41:38.838657+0000","last_became_peered":"2023-10-18T11:41:38.838657+0000","last_unstale":"2023-10-18T11:41:40.632267+0000","last_undegraded":"2023-10-18T11:41:40.632267+0000","last_fullsized":"2023-10-18T11:41:40.632267+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:10.683364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633493+0000","last_change":"2023-10-18T11:41:38.832579+0000","last_active":"2023-10-18T11:41:40.633493+0000","last_peered":"2023-10-18T11:41:40.633493+0000","last_clean":"2023-10-18T11:41:40.633493+0000","last_became_active":"2023-10-18T11:41:38.832320+0000","last_became_peered":"2023-10-18T11:41:38.832320+0000","last_unstale":"2023-10-18T11:41:40.633493+0000","last_undegraded":"2023-10-18T11:41:40.633493+0000","last_fullsized":"2023-10-18T11:41:40.633493+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:20.638468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632490+0000","last_change":"2023-10-18T11:41:38.840547+0000","last_active":"2023-10-18T11:41:40.632490+0000","last_peered":"2023-10-18T11:41:40.632490+0000","last_clean":"2023-10-18T11:41:40.632490+0000","last_became_active":"2023-10-18T11:41:38.840290+0000","last_became_peered":"2023-10-18T11:41:38.840290+0000","last_unstale":"2023-10-18T11:41:40.632490+0000","last_undegraded":"2023-10-18T11:41:40.632490+0000","last_fullsized":"2023-10-18T11:41:40.632490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:45.321557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632289+0000","last_change":"2023-10-18T11:41:38.839747+0000","last_active":"2023-10-18T11:41:40.632289+0000","last_peered":"2023-10-18T11:41:40.632289+0000","last_clean":"2023-10-18T11:41:40.632289+0000","last_became_active":"2023-10-18T11:41:38.839470+0000","last_became_peered":"2023-10-18T11:41:38.839470+0000","last_unstale":"2023-10-18T11:41:40.632289+0000","last_undegraded":"2023-10-18T11:41:40.632289+0000","last_fullsized":"2023-10-18T11:41:40.632289+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:35:39.421743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632884+0000","last_change":"2023-10-18T11:41:38.826683+0000","last_active":"2023-10-18T11:41:40.632884+0000","last_peered":"2023-10-18T11:41:40.632884+0000","last_clean":"2023-10-18T11:41:40.632884+0000","last_became_active":"2023-10-18T11:41:38.826277+0000","last_became_peered":"2023-10-18T11:41:38.826277+0000","last_unstale":"2023-10-18T11:41:40.632884+0000","last_undegraded":"2023-10-18T11:41:40.632884+0000","last_fullsized":"2023-10-18T11:41:40.632884+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:33.088719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829657+0000","last_change":"2023-10-18T11:41:38.828519+0000","last_active":"2023-10-18T11:41:39.829657+0000","last_peered":"2023-10-18T11:41:39.829657+0000","last_clean":"2023-10-18T11:41:39.829657+0000","last_became_active":"2023-10-18T11:41:38.828256+0000","last_became_peered":"2023-10-18T11:41:38.828256+0000","last_unstale":"2023-10-18T11:41:39.829657+0000","last_undegraded":"2023-10-18T11:41:39.829657+0000","last_fullsized":"2023-10-18T11:41:39.829657+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:25:18.914354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632603+0000","last_change":"2023-10-18T11:41:38.836612+0000","last_active":"2023-10-18T11:41:40.632603+0000","last_peered":"2023-10-18T11:41:40.632603+0000","last_clean":"2023-10-18T11:41:40.632603+0000","last_became_active":"2023-10-18T11:41:38.836342+0000","last_became_peered":"2023-10-18T11:41:38.836342+0000","last_unstale":"2023-10-18T11:41:40.632603+0000","last_undegraded":"2023-10-18T11:41:40.632603+0000","last_fullsized":"2023-10-18T11:41:40.632603+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:56.976963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632427+0000","last_change":"2023-10-18T11:41:38.822440+0000","last_active":"2023-10-18T11:41:40.632427+0000","last_peered":"2023-10-18T11:41:40.632427+0000","last_clean":"2023-10-18T11:41:40.632427+0000","last_became_active":"2023-10-18T11:41:38.822164+0000","last_became_peered":"2023-10-18T11:41:38.822164+0000","last_unstale":"2023-10-18T11:41:40.632427+0000","last_undegraded":"2023-10-18T11:41:40.632427+0000","last_fullsized":"2023-10-18T11:41:40.632427+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:55.140291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828721+0000","last_change":"2023-10-18T11:41:38.832961+0000","last_active":"2023-10-18T11:41:39.828721+0000","last_peered":"2023-10-18T11:41:39.828721+0000","last_clean":"2023-10-18T11:41:39.828721+0000","last_became_active":"2023-10-18T11:41:38.832373+0000","last_became_peered":"2023-10-18T11:41:38.832373+0000","last_unstale":"2023-10-18T11:41:39.828721+0000","last_undegraded":"2023-10-18T11:41:39.828721+0000","last_fullsized":"2023-10-18T11:41:39.828721+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:16.135065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633503+0000","last_change":"2023-10-18T11:41:38.830562+0000","last_active":"2023-10-18T11:41:40.633503+0000","last_peered":"2023-10-18T11:41:40.633503+0000","last_clean":"2023-10-18T11:41:40.633503+0000","last_became_active":"2023-10-18T11:41:38.830272+0000","last_became_peered":"2023-10-18T11:41:38.830272+0000","last_unstale":"2023-10-18T11:41:40.633503+0000","last_undegraded":"2023-10-18T11:41:40.633503+0000","last_fullsized":"2023-10-18T11:41:40.633503+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:58.419347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632267+0000","last_change":"2023-10-18T11:41:38.831348+0000","last_active":"2023-10-18T11:41:40.632267+0000","last_peered":"2023-10-18T11:41:40.632267+0000","last_clean":"2023-10-18T11:41:40.632267+0000","last_became_active":"2023-10-18T11:41:38.826783+0000","last_became_peered":"2023-10-18T11:41:38.826783+0000","last_unstale":"2023-10-18T11:41:40.632267+0000","last_undegraded":"2023-10-18T11:41:40.632267+0000","last_fullsized":"2023-10-18T11:41:40.632267+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:12:27.032354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633501+0000","last_change":"2023-10-18T11:41:38.830730+0000","last_active":"2023-10-18T11:41:40.633501+0000","last_peered":"2023-10-18T11:41:40.633501+0000","last_clean":"2023-10-18T11:41:40.633501+0000","last_became_active":"2023-10-18T11:41:38.830409+0000","last_became_peered":"2023-10-18T11:41:38.830409+0000","last_unstale":"2023-10-18T11:41:40.633501+0000","last_undegraded":"2023-10-18T11:41:40.633501+0000","last_fullsized":"2023-10-18T11:41:40.633501+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:33.767844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632453+0000","last_change":"2023-10-18T11:41:38.836640+0000","last_active":"2023-10-18T11:41:40.632453+0000","last_peered":"2023-10-18T11:41:40.632453+0000","last_clean":"2023-10-18T11:41:40.632453+0000","last_became_active":"2023-10-18T11:41:38.835870+0000","last_became_peered":"2023-10-18T11:41:38.835870+0000","last_unstale":"2023-10-18T11:41:40.632453+0000","last_undegraded":"2023-10-18T11:41:40.632453+0000","last_fullsized":"2023-10-18T11:41:40.632453+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:53.520200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.631912+0000","last_change":"2023-10-18T11:41:38.831324+0000","last_active":"2023-10-18T11:41:40.631912+0000","last_peered":"2023-10-18T11:41:40.631912+0000","last_clean":"2023-10-18T11:41:40.631912+0000","last_became_active":"2023-10-18T11:41:38.831052+0000","last_became_peered":"2023-10-18T11:41:38.831052+0000","last_unstale":"2023-10-18T11:41:40.631912+0000","last_undegraded":"2023-10-18T11:41:40.631912+0000","last_fullsized":"2023-10-18T11:41:40.631912+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:55.500613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633555+0000","last_change":"2023-10-18T11:41:38.834503+0000","last_active":"2023-10-18T11:41:40.633555+0000","last_peered":"2023-10-18T11:41:40.633555+0000","last_clean":"2023-10-18T11:41:40.633555+0000","last_became_active":"2023-10-18T11:41:38.834215+0000","last_became_peered":"2023-10-18T11:41:38.834215+0000","last_unstale":"2023-10-18T11:41:40.633555+0000","last_undegraded":"2023-10-18T11:41:40.633555+0000","last_fullsized":"2023-10-18T11:41:40.633555+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:13.740561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828696+0000","last_change":"2023-10-18T11:41:38.827931+0000","last_active":"2023-10-18T11:41:39.828696+0000","last_peered":"2023-10-18T11:41:39.828696+0000","last_clean":"2023-10-18T11:41:39.828696+0000","last_became_active":"2023-10-18T11:41:38.827669+0000","last_became_peered":"2023-10-18T11:41:38.827669+0000","last_unstale":"2023-10-18T11:41:39.828696+0000","last_undegraded":"2023-10-18T11:41:39.828696+0000","last_fullsized":"2023-10-18T11:41:39.828696+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:03.238765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633202+0000","last_change":"2023-10-18T11:41:38.832584+0000","last_active":"2023-10-18T11:41:40.633202+0000","last_peered":"2023-10-18T11:41:40.633202+0000","last_clean":"2023-10-18T11:41:40.633202+0000","last_became_active":"2023-10-18T11:41:38.832073+0000","last_became_peered":"2023-10-18T11:41:38.832073+0000","last_unstale":"2023-10-18T11:41:40.633202+0000","last_undegraded":"2023-10-18T11:41:40.633202+0000","last_fullsized":"2023-10-18T11:41:40.633202+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:06.540031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632925+0000","last_change":"2023-10-18T11:41:38.837755+0000","last_active":"2023-10-18T11:41:40.632925+0000","last_peered":"2023-10-18T11:41:40.632925+0000","last_clean":"2023-10-18T11:41:40.632925+0000","last_became_active":"2023-10-18T11:41:38.837504+0000","last_became_peered":"2023-10-18T11:41:38.837504+0000","last_unstale":"2023-10-18T11:41:40.632925+0000","last_undegraded":"2023-10-18T11:41:40.632925+0000","last_fullsized":"2023-10-18T11:41:40.632925+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:33:27.002897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632382+0000","last_change":"2023-10-18T11:41:38.832688+0000","last_active":"2023-10-18T11:41:40.632382+0000","last_peered":"2023-10-18T11:41:40.632382+0000","last_clean":"2023-10-18T11:41:40.632382+0000","last_became_active":"2023-10-18T11:41:38.832343+0000","last_became_peered":"2023-10-18T11:41:38.832343+0000","last_unstale":"2023-10-18T11:41:40.632382+0000","last_undegraded":"2023-10-18T11:41:40.632382+0000","last_fullsized":"2023-10-18T11:41:40.632382+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:09.897615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632319+0000","last_change":"2023-10-18T11:41:38.826514+0000","last_active":"2023-10-18T11:41:40.632319+0000","last_peered":"2023-10-18T11:41:40.632319+0000","last_clean":"2023-10-18T11:41:40.632319+0000","last_became_active":"2023-10-18T11:41:38.826186+0000","last_became_peered":"2023-10-18T11:41:38.826186+0000","last_unstale":"2023-10-18T11:41:40.632319+0000","last_undegraded":"2023-10-18T11:41:40.632319+0000","last_fullsized":"2023-10-18T11:41:40.632319+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:03.082825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829945+0000","last_change":"2023-10-18T11:41:38.826932+0000","last_active":"2023-10-18T11:41:39.829945+0000","last_peered":"2023-10-18T11:41:39.829945+0000","last_clean":"2023-10-18T11:41:39.829945+0000","last_became_active":"2023-10-18T11:41:38.826645+0000","last_became_peered":"2023-10-18T11:41:38.826645+0000","last_unstale":"2023-10-18T11:41:39.829945+0000","last_undegraded":"2023-10-18T11:41:39.829945+0000","last_fullsized":"2023-10-18T11:41:39.829945+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:54:15.505963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828515+0000","last_change":"2023-10-18T11:41:38.827801+0000","last_active":"2023-10-18T11:41:39.828515+0000","last_peered":"2023-10-18T11:41:39.828515+0000","last_clean":"2023-10-18T11:41:39.828515+0000","last_became_active":"2023-10-18T11:41:38.827514+0000","last_became_peered":"2023-10-18T11:41:38.827514+0000","last_unstale":"2023-10-18T11:41:39.828515+0000","last_undegraded":"2023-10-18T11:41:39.828515+0000","last_fullsized":"2023-10-18T11:41:39.828515+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:05:57.235594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632687+0000","last_change":"2023-10-18T11:41:38.820452+0000","last_active":"2023-10-18T11:41:40.632687+0000","last_peered":"2023-10-18T11:41:40.632687+0000","last_clean":"2023-10-18T11:41:40.632687+0000","last_became_active":"2023-10-18T11:41:38.820159+0000","last_became_peered":"2023-10-18T11:41:38.820159+0000","last_unstale":"2023-10-18T11:41:40.632687+0000","last_undegraded":"2023-10-18T11:41:40.632687+0000","last_fullsized":"2023-10-18T11:41:40.632687+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:42:46.510681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632604+0000","last_change":"2023-10-18T11:41:38.823794+0000","last_active":"2023-10-18T11:41:40.632604+0000","last_peered":"2023-10-18T11:41:40.632604+0000","last_clean":"2023-10-18T11:41:40.632604+0000","last_became_active":"2023-10-18T11:41:38.823494+0000","last_became_peered":"2023-10-18T11:41:38.823494+0000","last_unstale":"2023-10-18T11:41:40.632604+0000","last_undegraded":"2023-10-18T11:41:40.632604+0000","last_fullsized":"2023-10-18T11:41:40.632604+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:06:15.897744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632209+0000","last_change":"2023-10-18T11:41:38.826372+0000","last_active":"2023-10-18T11:41:40.632209+0000","last_peered":"2023-10-18T11:41:40.632209+0000","last_clean":"2023-10-18T11:41:40.632209+0000","last_became_active":"2023-10-18T11:41:38.826126+0000","last_became_peered":"2023-10-18T11:41:38.826126+0000","last_unstale":"2023-10-18T11:41:40.632209+0000","last_undegraded":"2023-10-18T11:41:40.632209+0000","last_fullsized":"2023-10-18T11:41:40.632209+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:30.439866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828583+0000","last_change":"2023-10-18T11:41:38.823012+0000","last_active":"2023-10-18T11:41:39.828583+0000","last_peered":"2023-10-18T11:41:39.828583+0000","last_clean":"2023-10-18T11:41:39.828583+0000","last_became_active":"2023-10-18T11:41:38.822261+0000","last_became_peered":"2023-10-18T11:41:38.822261+0000","last_unstale":"2023-10-18T11:41:39.828583+0000","last_undegraded":"2023-10-18T11:41:39.828583+0000","last_fullsized":"2023-10-18T11:41:39.828583+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:36.619175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829996+0000","last_change":"2023-10-18T11:41:38.827149+0000","last_active":"2023-10-18T11:41:39.829996+0000","last_peered":"2023-10-18T11:41:39.829996+0000","last_clean":"2023-10-18T11:41:39.829996+0000","last_became_active":"2023-10-18T11:41:38.825505+0000","last_became_peered":"2023-10-18T11:41:38.825505+0000","last_unstale":"2023-10-18T11:41:39.829996+0000","last_undegraded":"2023-10-18T11:41:39.829996+0000","last_fullsized":"2023-10-18T11:41:39.829996+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:06.291395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828541+0000","last_change":"2023-10-18T11:41:38.837766+0000","last_active":"2023-10-18T11:41:39.828541+0000","last_peered":"2023-10-18T11:41:39.828541+0000","last_clean":"2023-10-18T11:41:39.828541+0000","last_became_active":"2023-10-18T11:41:38.837482+0000","last_became_peered":"2023-10-18T11:41:38.837482+0000","last_unstale":"2023-10-18T11:41:39.828541+0000","last_undegraded":"2023-10-18T11:41:39.828541+0000","last_fullsized":"2023-10-18T11:41:39.828541+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:05.028643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632097+0000","last_change":"2023-10-18T11:41:38.831958+0000","last_active":"2023-10-18T11:41:40.632097+0000","last_peered":"2023-10-18T11:41:40.632097+0000","last_clean":"2023-10-18T11:41:40.632097+0000","last_became_active":"2023-10-18T11:41:38.831705+0000","last_became_peered":"2023-10-18T11:41:38.831705+0000","last_unstale":"2023-10-18T11:41:40.632097+0000","last_undegraded":"2023-10-18T11:41:40.632097+0000","last_fullsized":"2023-10-18T11:41:40.632097+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:06.540031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829267+0000","last_change":"2023-10-18T11:41:38.827986+0000","last_active":"2023-10-18T11:41:39.829267+0000","last_peered":"2023-10-18T11:41:39.829267+0000","last_clean":"2023-10-18T11:41:39.829267+0000","last_became_active":"2023-10-18T11:41:38.827661+0000","last_became_peered":"2023-10-18T11:41:38.827661+0000","last_unstale":"2023-10-18T11:41:39.829267+0000","last_undegraded":"2023-10-18T11:41:39.829267+0000","last_fullsized":"2023-10-18T11:41:39.829267+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:51.388492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632077+0000","last_change":"2023-10-18T11:41:38.837107+0000","last_active":"2023-10-18T11:41:40.632077+0000","last_peered":"2023-10-18T11:41:40.632077+0000","last_clean":"2023-10-18T11:41:40.632077+0000","last_became_active":"2023-10-18T11:41:38.836829+0000","last_became_peered":"2023-10-18T11:41:38.836829+0000","last_unstale":"2023-10-18T11:41:40.632077+0000","last_undegraded":"2023-10-18T11:41:40.632077+0000","last_fullsized":"2023-10-18T11:41:40.632077+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:09.384565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"173'4","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.433225+0000","last_change":"2023-10-18T11:41:37.820263+0000","last_active":"2023-10-18T11:41:40.433225+0000","last_peered":"2023-10-18T11:41:40.433225+0000","last_clean":"2023-10-18T11:41:40.433225+0000","last_became_active":"2023-10-18T11:41:37.819933+0000","last_became_peered":"2023-10-18T11:41:37.819933+0000","last_unstale":"2023-10-18T11:41:40.433225+0000","last_undegraded":"2023-10-18T11:41:40.433225+0000","last_fullsized":"2023-10-18T11:41:40.433225+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:41:57.852127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632388+0000","last_change":"2023-10-18T11:41:38.839828+0000","last_active":"2023-10-18T11:41:40.632388+0000","last_peered":"2023-10-18T11:41:40.632388+0000","last_clean":"2023-10-18T11:41:40.632388+0000","last_became_active":"2023-10-18T11:41:38.838892+0000","last_became_peered":"2023-10-18T11:41:38.838892+0000","last_unstale":"2023-10-18T11:41:40.632388+0000","last_undegraded":"2023-10-18T11:41:40.632388+0000","last_fullsized":"2023-10-18T11:41:40.632388+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:42.465109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"173'1","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.435044+0000","last_change":"2023-10-18T11:41:37.821129+0000","last_active":"2023-10-18T11:41:40.435044+0000","last_peered":"2023-10-18T11:41:40.435044+0000","last_clean":"2023-10-18T11:41:40.435044+0000","last_became_active":"2023-10-18T11:41:37.820850+0000","last_became_peered":"2023-10-18T11:41:37.820850+0000","last_unstale":"2023-10-18T11:41:40.435044+0000","last_undegraded":"2023-10-18T11:41:40.435044+0000","last_fullsized":"2023-10-18T11:41:40.435044+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:09.641790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829709+0000","last_change":"2023-10-18T11:41:38.831858+0000","last_active":"2023-10-18T11:41:39.829709+0000","last_peered":"2023-10-18T11:41:39.829709+0000","last_clean":"2023-10-18T11:41:39.829709+0000","last_became_active":"2023-10-18T11:41:38.831590+0000","last_became_peered":"2023-10-18T11:41:38.831590+0000","last_unstale":"2023-10-18T11:41:39.829709+0000","last_undegraded":"2023-10-18T11:41:39.829709+0000","last_fullsized":"2023-10-18T11:41:39.829709+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:35:42.177687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"173'2","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.435020+0000","last_change":"2023-10-18T11:41:37.821038+0000","last_active":"2023-10-18T11:41:40.435020+0000","last_peered":"2023-10-18T11:41:40.435020+0000","last_clean":"2023-10-18T11:41:40.435020+0000","last_became_active":"2023-10-18T11:41:37.820759+0000","last_became_peered":"2023-10-18T11:41:37.820759+0000","last_unstale":"2023-10-18T11:41:40.435020+0000","last_undegraded":"2023-10-18T11:41:40.435020+0000","last_fullsized":"2023-10-18T11:41:40.435020+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:48.941548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632940+0000","last_change":"2023-10-18T11:41:38.830270+0000","last_active":"2023-10-18T11:41:40.632940+0000","last_peered":"2023-10-18T11:41:40.632940+0000","last_clean":"2023-10-18T11:41:40.632940+0000","last_became_active":"2023-10-18T11:41:38.830001+0000","last_became_peered":"2023-10-18T11:41:38.830001+0000","last_unstale":"2023-10-18T11:41:40.632940+0000","last_undegraded":"2023-10-18T11:41:40.632940+0000","last_fullsized":"2023-10-18T11:41:40.632940+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:55.500613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"173'2","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.433144+0000","last_change":"2023-10-18T11:41:37.819635+0000","last_active":"2023-10-18T11:41:40.433144+0000","last_peered":"2023-10-18T11:41:40.433144+0000","last_clean":"2023-10-18T11:41:40.433144+0000","last_became_active":"2023-10-18T11:41:37.819375+0000","last_became_peered":"2023-10-18T11:41:37.819375+0000","last_unstale":"2023-10-18T11:41:40.433144+0000","last_undegraded":"2023-10-18T11:41:40.433144+0000","last_fullsized":"2023-10-18T11:41:40.433144+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:51.830773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632149+0000","last_change":"2023-10-18T11:41:38.824262+0000","last_active":"2023-10-18T11:41:40.632149+0000","last_peered":"2023-10-18T11:41:40.632149+0000","last_clean":"2023-10-18T11:41:40.632149+0000","last_became_active":"2023-10-18T11:41:38.824005+0000","last_became_peered":"2023-10-18T11:41:38.824005+0000","last_unstale":"2023-10-18T11:41:40.632149+0000","last_undegraded":"2023-10-18T11:41:40.632149+0000","last_fullsized":"2023-10-18T11:41:40.632149+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:27:20.542456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"173'1","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.830055+0000","last_change":"2023-10-18T11:41:37.826154+0000","last_active":"2023-10-18T11:41:39.830055+0000","last_peered":"2023-10-18T11:41:39.830055+0000","last_clean":"2023-10-18T11:41:39.830055+0000","last_became_active":"2023-10-18T11:41:37.825607+0000","last_became_peered":"2023-10-18T11:41:37.825607+0000","last_unstale":"2023-10-18T11:41:39.830055+0000","last_undegraded":"2023-10-18T11:41:39.830055+0000","last_fullsized":"2023-10-18T11:41:39.830055+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:24.888182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633088+0000","last_change":"2023-10-18T11:41:38.834243+0000","last_active":"2023-10-18T11:41:40.633088+0000","last_peered":"2023-10-18T11:41:40.633088+0000","last_clean":"2023-10-18T11:41:40.633088+0000","last_became_active":"2023-10-18T11:41:38.833958+0000","last_became_peered":"2023-10-18T11:41:38.833958+0000","last_unstale":"2023-10-18T11:41:40.633088+0000","last_undegraded":"2023-10-18T11:41:40.633088+0000","last_fullsized":"2023-10-18T11:41:40.633088+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:45.676489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"173'1","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.433042+0000","last_change":"2023-10-18T11:41:37.824548+0000","last_active":"2023-10-18T11:41:40.433042+0000","last_peered":"2023-10-18T11:41:40.433042+0000","last_clean":"2023-10-18T11:41:40.433042+0000","last_became_active":"2023-10-18T11:41:37.824278+0000","last_became_peered":"2023-10-18T11:41:37.824278+0000","last_unstale":"2023-10-18T11:41:40.433042+0000","last_undegraded":"2023-10-18T11:41:40.433042+0000","last_fullsized":"2023-10-18T11:41:40.433042+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:05.442472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632067+0000","last_change":"2023-10-18T11:41:38.838537+0000","last_active":"2023-10-18T11:41:40.632067+0000","last_peered":"2023-10-18T11:41:40.632067+0000","last_clean":"2023-10-18T11:41:40.632067+0000","last_became_active":"2023-10-18T11:41:38.835109+0000","last_became_peered":"2023-10-18T11:41:38.835109+0000","last_unstale":"2023-10-18T11:41:40.632067+0000","last_undegraded":"2023-10-18T11:41:40.632067+0000","last_fullsized":"2023-10-18T11:41:40.632067+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:39:43.232981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.431190+0000","last_change":"2023-10-18T11:41:37.820269+0000","last_active":"2023-10-18T11:41:40.431190+0000","last_peered":"2023-10-18T11:41:40.431190+0000","last_clean":"2023-10-18T11:41:40.431190+0000","last_became_active":"2023-10-18T11:41:37.819988+0000","last_became_peered":"2023-10-18T11:41:37.819988+0000","last_unstale":"2023-10-18T11:41:40.431190+0000","last_undegraded":"2023-10-18T11:41:40.431190+0000","last_fullsized":"2023-10-18T11:41:40.431190+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:48:53.297750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632535+0000","last_change":"2023-10-18T11:41:38.820999+0000","last_active":"2023-10-18T11:41:40.632535+0000","last_peered":"2023-10-18T11:41:40.632535+0000","last_clean":"2023-10-18T11:41:40.632535+0000","last_became_active":"2023-10-18T11:41:38.820731+0000","last_became_peered":"2023-10-18T11:41:38.820731+0000","last_unstale":"2023-10-18T11:41:40.632535+0000","last_undegraded":"2023-10-18T11:41:40.632535+0000","last_fullsized":"2023-10-18T11:41:40.632535+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:00:22.778626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.432968+0000","last_change":"2023-10-18T11:41:37.824809+0000","last_active":"2023-10-18T11:41:40.432968+0000","last_peered":"2023-10-18T11:41:40.432968+0000","last_clean":"2023-10-18T11:41:40.432968+0000","last_became_active":"2023-10-18T11:41:37.824509+0000","last_became_peered":"2023-10-18T11:41:37.824509+0000","last_unstale":"2023-10-18T11:41:40.432968+0000","last_undegraded":"2023-10-18T11:41:40.432968+0000","last_fullsized":"2023-10-18T11:41:40.432968+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:50.760177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829761+0000","last_change":"2023-10-18T11:41:38.827847+0000","last_active":"2023-10-18T11:41:39.829761+0000","last_peered":"2023-10-18T11:41:39.829761+0000","last_clean":"2023-10-18T11:41:39.829761+0000","last_became_active":"2023-10-18T11:41:38.827553+0000","last_became_peered":"2023-10-18T11:41:38.827553+0000","last_unstale":"2023-10-18T11:41:39.829761+0000","last_undegraded":"2023-10-18T11:41:39.829761+0000","last_fullsized":"2023-10-18T11:41:39.829761+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:54:44.886016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.435084+0000","last_change":"2023-10-18T11:41:37.825094+0000","last_active":"2023-10-18T11:41:40.435084+0000","last_peered":"2023-10-18T11:41:40.435084+0000","last_clean":"2023-10-18T11:41:40.435084+0000","last_became_active":"2023-10-18T11:41:37.824553+0000","last_became_peered":"2023-10-18T11:41:37.824553+0000","last_unstale":"2023-10-18T11:41:40.435084+0000","last_undegraded":"2023-10-18T11:41:40.435084+0000","last_fullsized":"2023-10-18T11:41:40.435084+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:36.961770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828711+0000","last_change":"2023-10-18T11:41:38.833135+0000","last_active":"2023-10-18T11:41:39.828711+0000","last_peered":"2023-10-18T11:41:39.828711+0000","last_clean":"2023-10-18T11:41:39.828711+0000","last_became_active":"2023-10-18T11:41:38.832708+0000","last_became_peered":"2023-10-18T11:41:38.832708+0000","last_unstale":"2023-10-18T11:41:39.828711+0000","last_undegraded":"2023-10-18T11:41:39.828711+0000","last_fullsized":"2023-10-18T11:41:39.828711+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:39.928349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.431338+0000","last_change":"2023-10-18T11:41:37.814357+0000","last_active":"2023-10-18T11:41:40.431338+0000","last_peered":"2023-10-18T11:41:40.431338+0000","last_clean":"2023-10-18T11:41:40.431338+0000","last_became_active":"2023-10-18T11:41:37.814082+0000","last_became_peered":"2023-10-18T11:41:37.814082+0000","last_unstale":"2023-10-18T11:41:40.431338+0000","last_undegraded":"2023-10-18T11:41:40.431338+0000","last_fullsized":"2023-10-18T11:41:40.431338+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:41.460284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828728+0000","last_change":"2023-10-18T11:41:38.827430+0000","last_active":"2023-10-18T11:41:39.828728+0000","last_peered":"2023-10-18T11:41:39.828728+0000","last_clean":"2023-10-18T11:41:39.828728+0000","last_became_active":"2023-10-18T11:41:38.827151+0000","last_became_peered":"2023-10-18T11:41:38.827151+0000","last_unstale":"2023-10-18T11:41:39.828728+0000","last_undegraded":"2023-10-18T11:41:39.828728+0000","last_fullsized":"2023-10-18T11:41:39.828728+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:57.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"173'3","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828816+0000","last_change":"2023-10-18T11:41:37.816938+0000","last_active":"2023-10-18T11:41:39.828816+0000","last_peered":"2023-10-18T11:41:39.828816+0000","last_clean":"2023-10-18T11:41:39.828816+0000","last_became_active":"2023-10-18T11:41:37.816710+0000","last_became_peered":"2023-10-18T11:41:37.816710+0000","last_unstale":"2023-10-18T11:41:39.828816+0000","last_undegraded":"2023-10-18T11:41:39.828816+0000","last_fullsized":"2023-10-18T11:41:39.828816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:21:55.432322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829426+0000","last_change":"2023-10-18T11:41:38.827972+0000","last_active":"2023-10-18T11:41:39.829426+0000","last_peered":"2023-10-18T11:41:39.829426+0000","last_clean":"2023-10-18T11:41:39.829426+0000","last_became_active":"2023-10-18T11:41:38.827700+0000","last_became_peered":"2023-10-18T11:41:38.827700+0000","last_unstale":"2023-10-18T11:41:39.829426+0000","last_undegraded":"2023-10-18T11:41:39.829426+0000","last_fullsized":"2023-10-18T11:41:39.829426+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:29.998896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"173'2","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.433106+0000","last_change":"2023-10-18T11:41:37.819453+0000","last_active":"2023-10-18T11:41:40.433106+0000","last_peered":"2023-10-18T11:41:40.433106+0000","last_clean":"2023-10-18T11:41:40.433106+0000","last_became_active":"2023-10-18T11:41:37.819216+0000","last_became_peered":"2023-10-18T11:41:37.819216+0000","last_unstale":"2023-10-18T11:41:40.433106+0000","last_undegraded":"2023-10-18T11:41:40.433106+0000","last_fullsized":"2023-10-18T11:41:40.433106+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:00.103896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632566+0000","last_change":"2023-10-18T11:41:38.836803+0000","last_active":"2023-10-18T11:41:40.632566+0000","last_peered":"2023-10-18T11:41:40.632566+0000","last_clean":"2023-10-18T11:41:40.632566+0000","last_became_active":"2023-10-18T11:41:38.836239+0000","last_became_peered":"2023-10-18T11:41:38.836239+0000","last_unstale":"2023-10-18T11:41:40.632566+0000","last_undegraded":"2023-10-18T11:41:40.632566+0000","last_fullsized":"2023-10-18T11:41:40.632566+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:13.225614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.434733+0000","last_change":"2023-10-18T11:41:37.823427+0000","last_active":"2023-10-18T11:41:40.434733+0000","last_peered":"2023-10-18T11:41:40.434733+0000","last_clean":"2023-10-18T11:41:40.434733+0000","last_became_active":"2023-10-18T11:41:37.823126+0000","last_became_peered":"2023-10-18T11:41:37.823126+0000","last_unstale":"2023-10-18T11:41:40.434733+0000","last_undegraded":"2023-10-18T11:41:40.434733+0000","last_fullsized":"2023-10-18T11:41:40.434733+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:48.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632392+0000","last_change":"2023-10-18T11:41:38.825410+0000","last_active":"2023-10-18T11:41:40.632392+0000","last_peered":"2023-10-18T11:41:40.632392+0000","last_clean":"2023-10-18T11:41:40.632392+0000","last_became_active":"2023-10-18T11:41:38.825132+0000","last_became_peered":"2023-10-18T11:41:38.825132+0000","last_unstale":"2023-10-18T11:41:40.632392+0000","last_undegraded":"2023-10-18T11:41:40.632392+0000","last_fullsized":"2023-10-18T11:41:40.632392+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:07:31.695811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.434706+0000","last_change":"2023-10-18T11:41:37.823660+0000","last_active":"2023-10-18T11:41:40.434706+0000","last_peered":"2023-10-18T11:41:40.434706+0000","last_clean":"2023-10-18T11:41:40.434706+0000","last_became_active":"2023-10-18T11:41:37.823201+0000","last_became_peered":"2023-10-18T11:41:37.823201+0000","last_unstale":"2023-10-18T11:41:40.434706+0000","last_undegraded":"2023-10-18T11:41:40.434706+0000","last_fullsized":"2023-10-18T11:41:40.434706+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:12.705578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632597+0000","last_change":"2023-10-18T11:41:38.844785+0000","last_active":"2023-10-18T11:41:40.632597+0000","last_peered":"2023-10-18T11:41:40.632597+0000","last_clean":"2023-10-18T11:41:40.632597+0000","last_became_active":"2023-10-18T11:41:38.844502+0000","last_became_peered":"2023-10-18T11:41:38.844502+0000","last_unstale":"2023-10-18T11:41:40.632597+0000","last_undegraded":"2023-10-18T11:41:40.632597+0000","last_fullsized":"2023-10-18T11:41:40.632597+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:03.082825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"173'8","reported_seq":20,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.431118+0000","last_change":"2023-10-18T11:41:37.822141+0000","last_active":"2023-10-18T11:41:40.431118+0000","last_peered":"2023-10-18T11:41:40.431118+0000","last_clean":"2023-10-18T11:41:40.431118+0000","last_became_active":"2023-10-18T11:41:37.821819+0000","last_became_peered":"2023-10-18T11:41:37.821819+0000","last_unstale":"2023-10-18T11:41:40.431118+0000","last_undegraded":"2023-10-18T11:41:40.431118+0000","last_fullsized":"2023-10-18T11:41:40.431118+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:19.314971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632707+0000","last_change":"2023-10-18T11:41:38.837476+0000","last_active":"2023-10-18T11:41:40.632707+0000","last_peered":"2023-10-18T11:41:40.632707+0000","last_clean":"2023-10-18T11:41:40.632707+0000","last_became_active":"2023-10-18T11:41:38.836917+0000","last_became_peered":"2023-10-18T11:41:38.836917+0000","last_unstale":"2023-10-18T11:41:40.632707+0000","last_undegraded":"2023-10-18T11:41:40.632707+0000","last_fullsized":"2023-10-18T11:41:40.632707+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:41:58.856953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"173'1","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828619+0000","last_change":"2023-10-18T11:41:37.821090+0000","last_active":"2023-10-18T11:41:39.828619+0000","last_peered":"2023-10-18T11:41:39.828619+0000","last_clean":"2023-10-18T11:41:39.828619+0000","last_became_active":"2023-10-18T11:41:37.820845+0000","last_became_peered":"2023-10-18T11:41:37.820845+0000","last_unstale":"2023-10-18T11:41:39.828619+0000","last_undegraded":"2023-10-18T11:41:39.828619+0000","last_fullsized":"2023-10-18T11:41:39.828619+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:15.610937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633328+0000","last_change":"2023-10-18T11:41:38.832709+0000","last_active":"2023-10-18T11:41:40.633328+0000","last_peered":"2023-10-18T11:41:40.633328+0000","last_clean":"2023-10-18T11:41:40.633328+0000","last_became_active":"2023-10-18T11:41:38.832436+0000","last_became_peered":"2023-10-18T11:41:38.832436+0000","last_unstale":"2023-10-18T11:41:40.633328+0000","last_undegraded":"2023-10-18T11:41:40.633328+0000","last_fullsized":"2023-10-18T11:41:40.633328+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:30.439866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"173'2","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.435062+0000","last_change":"2023-10-18T11:41:37.823846+0000","last_active":"2023-10-18T11:41:40.435062+0000","last_peered":"2023-10-18T11:41:40.435062+0000","last_clean":"2023-10-18T11:41:40.435062+0000","last_became_active":"2023-10-18T11:41:37.822544+0000","last_became_peered":"2023-10-18T11:41:37.822544+0000","last_unstale":"2023-10-18T11:41:40.435062+0000","last_undegraded":"2023-10-18T11:41:40.435062+0000","last_fullsized":"2023-10-18T11:41:40.435062+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:01.806142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632849+0000","last_change":"2023-10-18T11:41:38.822398+0000","last_active":"2023-10-18T11:41:40.632849+0000","last_peered":"2023-10-18T11:41:40.632849+0000","last_clean":"2023-10-18T11:41:40.632849+0000","last_became_active":"2023-10-18T11:41:38.822156+0000","last_became_peered":"2023-10-18T11:41:38.822156+0000","last_unstale":"2023-10-18T11:41:40.632849+0000","last_undegraded":"2023-10-18T11:41:40.632849+0000","last_fullsized":"2023-10-18T11:41:40.632849+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:53:10.646615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"173'1","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.433201+0000","last_change":"2023-10-18T11:41:37.824435+0000","last_active":"2023-10-18T11:41:40.433201+0000","last_peered":"2023-10-18T11:41:40.433201+0000","last_clean":"2023-10-18T11:41:40.433201+0000","last_became_active":"2023-10-18T11:41:37.823886+0000","last_became_peered":"2023-10-18T11:41:37.823886+0000","last_unstale":"2023-10-18T11:41:40.433201+0000","last_undegraded":"2023-10-18T11:41:40.433201+0000","last_fullsized":"2023-10-18T11:41:40.433201+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:19.194946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632799+0000","last_change":"2023-10-18T11:41:38.833975+0000","last_active":"2023-10-18T11:41:40.632799+0000","last_peered":"2023-10-18T11:41:40.632799+0000","last_clean":"2023-10-18T11:41:40.632799+0000","last_became_active":"2023-10-18T11:41:38.833708+0000","last_became_peered":"2023-10-18T11:41:38.833708+0000","last_unstale":"2023-10-18T11:41:40.632799+0000","last_undegraded":"2023-10-18T11:41:40.632799+0000","last_fullsized":"2023-10-18T11:41:40.632799+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:21:42.346674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.434765+0000","last_change":"2023-10-18T11:41:37.823230+0000","last_active":"2023-10-18T11:41:40.434765+0000","last_peered":"2023-10-18T11:41:40.434765+0000","last_clean":"2023-10-18T11:41:40.434765+0000","last_became_active":"2023-10-18T11:41:37.822954+0000","last_became_peered":"2023-10-18T11:41:37.822954+0000","last_unstale":"2023-10-18T11:41:40.434765+0000","last_undegraded":"2023-10-18T11:41:40.434765+0000","last_fullsized":"2023-10-18T11:41:40.434765+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:44.059048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632374+0000","last_change":"2023-10-18T11:41:38.832536+0000","last_active":"2023-10-18T11:41:40.632374+0000","last_peered":"2023-10-18T11:41:40.632374+0000","last_clean":"2023-10-18T11:41:40.632374+0000","last_became_active":"2023-10-18T11:41:38.832268+0000","last_became_peered":"2023-10-18T11:41:38.832268+0000","last_unstale":"2023-10-18T11:41:40.632374+0000","last_undegraded":"2023-10-18T11:41:40.632374+0000","last_fullsized":"2023-10-18T11:41:40.632374+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:01.108721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"173'2","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.430533+0000","last_change":"2023-10-18T11:41:37.822283+0000","last_active":"2023-10-18T11:41:40.430533+0000","last_peered":"2023-10-18T11:41:40.430533+0000","last_clean":"2023-10-18T11:41:40.430533+0000","last_became_active":"2023-10-18T11:41:37.822019+0000","last_became_peered":"2023-10-18T11:41:37.822019+0000","last_unstale":"2023-10-18T11:41:40.430533+0000","last_undegraded":"2023-10-18T11:41:40.430533+0000","last_fullsized":"2023-10-18T11:41:40.430533+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:27.346874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632682+0000","last_change":"2023-10-18T11:41:38.835474+0000","last_active":"2023-10-18T11:41:40.632682+0000","last_peered":"2023-10-18T11:41:40.632682+0000","last_clean":"2023-10-18T11:41:40.632682+0000","last_became_active":"2023-10-18T11:41:38.835205+0000","last_became_peered":"2023-10-18T11:41:38.835205+0000","last_unstale":"2023-10-18T11:41:40.632682+0000","last_undegraded":"2023-10-18T11:41:40.632682+0000","last_fullsized":"2023-10-18T11:41:40.632682+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:29.459089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829423+0000","last_change":"2023-10-18T11:41:37.819210+0000","last_active":"2023-10-18T11:41:39.829423+0000","last_peered":"2023-10-18T11:41:39.829423+0000","last_clean":"2023-10-18T11:41:39.829423+0000","last_became_active":"2023-10-18T11:41:37.818926+0000","last_became_peered":"2023-10-18T11:41:37.818926+0000","last_unstale":"2023-10-18T11:41:39.829423+0000","last_undegraded":"2023-10-18T11:41:39.829423+0000","last_fullsized":"2023-10-18T11:41:39.829423+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:15.032324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632551+0000","last_change":"2023-10-18T11:41:38.829408+0000","last_active":"2023-10-18T11:41:40.632551+0000","last_peered":"2023-10-18T11:41:40.632551+0000","last_clean":"2023-10-18T11:41:40.632551+0000","last_became_active":"2023-10-18T11:41:38.828917+0000","last_became_peered":"2023-10-18T11:41:38.828917+0000","last_unstale":"2023-10-18T11:41:40.632551+0000","last_undegraded":"2023-10-18T11:41:40.632551+0000","last_fullsized":"2023-10-18T11:41:40.632551+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:15:13.102514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.430515+0000","last_change":"2023-10-18T11:41:37.819830+0000","last_active":"2023-10-18T11:41:40.430515+0000","last_peered":"2023-10-18T11:41:40.430515+0000","last_clean":"2023-10-18T11:41:40.430515+0000","last_became_active":"2023-10-18T11:41:37.819576+0000","last_became_peered":"2023-10-18T11:41:37.819576+0000","last_unstale":"2023-10-18T11:41:40.430515+0000","last_undegraded":"2023-10-18T11:41:40.430515+0000","last_fullsized":"2023-10-18T11:41:40.430515+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:54.501911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632700+0000","last_change":"2023-10-18T11:41:38.837848+0000","last_active":"2023-10-18T11:41:40.632700+0000","last_peered":"2023-10-18T11:41:40.632700+0000","last_clean":"2023-10-18T11:41:40.632700+0000","last_became_active":"2023-10-18T11:41:38.837607+0000","last_became_peered":"2023-10-18T11:41:38.837607+0000","last_unstale":"2023-10-18T11:41:40.632700+0000","last_undegraded":"2023-10-18T11:41:40.632700+0000","last_fullsized":"2023-10-18T11:41:40.632700+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:00:22.778626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829767+0000","last_change":"2023-10-18T11:41:37.818588+0000","last_active":"2023-10-18T11:41:39.829767+0000","last_peered":"2023-10-18T11:41:39.829767+0000","last_clean":"2023-10-18T11:41:39.829767+0000","last_became_active":"2023-10-18T11:41:37.817798+0000","last_became_peered":"2023-10-18T11:41:37.817798+0000","last_unstale":"2023-10-18T11:41:39.829767+0000","last_undegraded":"2023-10-18T11:41:39.829767+0000","last_fullsized":"2023-10-18T11:41:39.829767+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:50:52.183260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632922+0000","last_change":"2023-10-18T11:41:38.842544+0000","last_active":"2023-10-18T11:41:40.632922+0000","last_peered":"2023-10-18T11:41:40.632922+0000","last_clean":"2023-10-18T11:41:40.632922+0000","last_became_active":"2023-10-18T11:41:38.841060+0000","last_became_peered":"2023-10-18T11:41:38.841060+0000","last_unstale":"2023-10-18T11:41:40.632922+0000","last_undegraded":"2023-10-18T11:41:40.632922+0000","last_fullsized":"2023-10-18T11:41:40.632922+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:34:30.298531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"173'2","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.434801+0000","last_change":"2023-10-18T11:41:37.823910+0000","last_active":"2023-10-18T11:41:40.434801+0000","last_peered":"2023-10-18T11:41:40.434801+0000","last_clean":"2023-10-18T11:41:40.434801+0000","last_became_active":"2023-10-18T11:41:37.823671+0000","last_became_peered":"2023-10-18T11:41:37.823671+0000","last_unstale":"2023-10-18T11:41:40.434801+0000","last_undegraded":"2023-10-18T11:41:40.434801+0000","last_fullsized":"2023-10-18T11:41:40.434801+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:15.970084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632967+0000","last_change":"2023-10-18T11:41:38.834278+0000","last_active":"2023-10-18T11:41:40.632967+0000","last_peered":"2023-10-18T11:41:40.632967+0000","last_clean":"2023-10-18T11:41:40.632967+0000","last_became_active":"2023-10-18T11:41:38.833972+0000","last_became_peered":"2023-10-18T11:41:38.833972+0000","last_unstale":"2023-10-18T11:41:40.632967+0000","last_undegraded":"2023-10-18T11:41:40.632967+0000","last_fullsized":"2023-10-18T11:41:40.632967+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:33:05.252055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"174'1","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633322+0000","last_change":"2023-10-18T11:41:37.823322+0000","last_active":"2023-10-18T11:41:40.633322+0000","last_peered":"2023-10-18T11:41:40.633322+0000","last_clean":"2023-10-18T11:41:40.633322+0000","last_became_active":"2023-10-18T11:41:37.822856+0000","last_became_peered":"2023-10-18T11:41:37.822856+0000","last_unstale":"2023-10-18T11:41:40.633322+0000","last_undegraded":"2023-10-18T11:41:40.633322+0000","last_fullsized":"2023-10-18T11:41:40.633322+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:52.515374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633117+0000","last_change":"2023-10-18T11:41:38.827748+0000","last_active":"2023-10-18T11:41:40.633117+0000","last_peered":"2023-10-18T11:41:40.633117+0000","last_clean":"2023-10-18T11:41:40.633117+0000","last_became_active":"2023-10-18T11:41:38.827469+0000","last_became_peered":"2023-10-18T11:41:38.827469+0000","last_unstale":"2023-10-18T11:41:40.633117+0000","last_undegraded":"2023-10-18T11:41:40.633117+0000","last_fullsized":"2023-10-18T11:41:40.633117+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:02.810967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.433219+0000","last_change":"2023-10-18T11:41:37.821788+0000","last_active":"2023-10-18T11:41:40.433219+0000","last_peered":"2023-10-18T11:41:40.433219+0000","last_clean":"2023-10-18T11:41:40.433219+0000","last_became_active":"2023-10-18T11:41:37.821313+0000","last_became_peered":"2023-10-18T11:41:37.821313+0000","last_unstale":"2023-10-18T11:41:40.433219+0000","last_undegraded":"2023-10-18T11:41:40.433219+0000","last_fullsized":"2023-10-18T11:41:40.433219+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:29.206502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632715+0000","last_change":"2023-10-18T11:41:38.835302+0000","last_active":"2023-10-18T11:41:40.632715+0000","last_peered":"2023-10-18T11:41:40.632715+0000","last_clean":"2023-10-18T11:41:40.632715+0000","last_became_active":"2023-10-18T11:41:38.835040+0000","last_became_peered":"2023-10-18T11:41:38.835040+0000","last_unstale":"2023-10-18T11:41:40.632715+0000","last_undegraded":"2023-10-18T11:41:40.632715+0000","last_fullsized":"2023-10-18T11:41:40.632715+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:46.797708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.433139+0000","last_change":"2023-10-18T11:41:37.822147+0000","last_active":"2023-10-18T11:41:40.433139+0000","last_peered":"2023-10-18T11:41:40.433139+0000","last_clean":"2023-10-18T11:41:40.433139+0000","last_became_active":"2023-10-18T11:41:37.821839+0000","last_became_peered":"2023-10-18T11:41:37.821839+0000","last_unstale":"2023-10-18T11:41:40.433139+0000","last_undegraded":"2023-10-18T11:41:40.433139+0000","last_fullsized":"2023-10-18T11:41:40.433139+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:54.443645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829996+0000","last_change":"2023-10-18T11:41:38.821087+0000","last_active":"2023-10-18T11:41:39.829996+0000","last_peered":"2023-10-18T11:41:39.829996+0000","last_clean":"2023-10-18T11:41:39.829996+0000","last_became_active":"2023-10-18T11:41:38.820806+0000","last_became_peered":"2023-10-18T11:41:38.820806+0000","last_unstale":"2023-10-18T11:41:39.829996+0000","last_undegraded":"2023-10-18T11:41:39.829996+0000","last_fullsized":"2023-10-18T11:41:39.829996+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:39.471952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.431263+0000","last_change":"2023-10-18T11:41:37.824985+0000","last_active":"2023-10-18T11:41:40.431263+0000","last_peered":"2023-10-18T11:41:40.431263+0000","last_clean":"2023-10-18T11:41:40.431263+0000","last_became_active":"2023-10-18T11:41:37.824481+0000","last_became_peered":"2023-10-18T11:41:37.824481+0000","last_unstale":"2023-10-18T11:41:40.431263+0000","last_undegraded":"2023-10-18T11:41:40.431263+0000","last_fullsized":"2023-10-18T11:41:40.431263+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:59:39.668889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.633240+0000","last_change":"2023-10-18T11:41:38.830152+0000","last_active":"2023-10-18T11:41:40.633240+0000","last_peered":"2023-10-18T11:41:40.633240+0000","last_clean":"2023-10-18T11:41:40.633240+0000","last_became_active":"2023-10-18T11:41:38.829828+0000","last_became_peered":"2023-10-18T11:41:38.829828+0000","last_unstale":"2023-10-18T11:41:40.633240+0000","last_undegraded":"2023-10-18T11:41:40.633240+0000","last_fullsized":"2023-10-18T11:41:40.633240+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:35:10.933389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.828507+0000","last_change":"2023-10-18T11:41:37.820903+0000","last_active":"2023-10-18T11:41:39.828507+0000","last_peered":"2023-10-18T11:41:39.828507+0000","last_clean":"2023-10-18T11:41:39.828507+0000","last_became_active":"2023-10-18T11:41:37.820650+0000","last_became_peered":"2023-10-18T11:41:37.820650+0000","last_unstale":"2023-10-18T11:41:39.828507+0000","last_undegraded":"2023-10-18T11:41:39.828507+0000","last_fullsized":"2023-10-18T11:41:39.828507+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:45:11.753576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.632242+0000","last_change":"2023-10-18T11:41:38.838162+0000","last_active":"2023-10-18T11:41:40.632242+0000","last_peered":"2023-10-18T11:41:40.632242+0000","last_clean":"2023-10-18T11:41:40.632242+0000","last_became_active":"2023-10-18T11:41:38.836514+0000","last_became_peered":"2023-10-18T11:41:38.836514+0000","last_unstale":"2023-10-18T11:41:40.632242+0000","last_undegraded":"2023-10-18T11:41:40.632242+0000","last_fullsized":"2023-10-18T11:41:40.632242+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:09.333393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.431208+0000","last_change":"2023-10-18T11:41:37.814973+0000","last_active":"2023-10-18T11:41:40.431208+0000","last_peered":"2023-10-18T11:41:40.431208+0000","last_clean":"2023-10-18T11:41:40.431208+0000","last_became_active":"2023-10-18T11:41:37.814699+0000","last_became_peered":"2023-10-18T11:41:37.814699+0000","last_unstale":"2023-10-18T11:41:40.431208+0000","last_undegraded":"2023-10-18T11:41:40.431208+0000","last_fullsized":"2023-10-18T11:41:40.431208+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:19.194946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829747+0000","last_change":"2023-10-18T11:41:38.826161+0000","last_active":"2023-10-18T11:41:39.829747+0000","last_peered":"2023-10-18T11:41:39.829747+0000","last_clean":"2023-10-18T11:41:39.829747+0000","last_became_active":"2023-10-18T11:41:38.825615+0000","last_became_peered":"2023-10-18T11:41:38.825615+0000","last_unstale":"2023-10-18T11:41:39.829747+0000","last_undegraded":"2023-10-18T11:41:39.829747+0000","last_fullsized":"2023-10-18T11:41:39.829747+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:32:47.816689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.434926+0000","last_change":"2023-10-18T11:41:37.822249+0000","last_active":"2023-10-18T11:41:40.434926+0000","last_peered":"2023-10-18T11:41:40.434926+0000","last_clean":"2023-10-18T11:41:40.434926+0000","last_became_active":"2023-10-18T11:41:37.821714+0000","last_became_peered":"2023-10-18T11:41:37.821714+0000","last_unstale":"2023-10-18T11:41:40.434926+0000","last_undegraded":"2023-10-18T11:41:40.434926+0000","last_fullsized":"2023-10-18T11:41:40.434926+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:24.518814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-10-18T11:41:39.829803+0000","last_change":"2023-10-18T11:41:38.836422+0000","last_active":"2023-10-18T11:41:39.829803+0000","last_peered":"2023-10-18T11:41:39.829803+0000","last_clean":"2023-10-18T11:41:39.829803+0000","last_became_active":"2023-10-18T11:41:38.835474+0000","last_became_peered":"2023-10-18T11:41:38.835474+0000","last_unstale":"2023-10-18T11:41:39.829803+0000","last_undegraded":"2023-10-18T11:41:39.829803+0000","last_fullsized":"2023-10-18T11:41:39.829803+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:37.801652+0000","last_clean_scrub_stamp":"2023-10-18T11:41:37.801652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:38.697917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"173'6","reported_seq":18,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:40.430562+0000","last_change":"2023-10-18T11:41:37.820513+0000","last_active":"2023-10-18T11:41:40.430562+0000","last_peered":"2023-10-18T11:41:40.430562+0000","last_clean":"2023-10-18T11:41:40.430562+0000","last_became_active":"2023-10-18T11:41:37.820194+0000","last_became_peered":"2023-10-18T11:41:37.820194+0000","last_unstale":"2023-10-18T11:41:40.430562+0000","last_undegraded":"2023-10-18T11:41:40.430562+0000","last_fullsized":"2023-10-18T11:41:40.430562+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:41:36.796826+0000","last_clean_scrub_stamp":"2023-10-18T11:41:36.796826+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:58.848022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"162'280","reported_seq":912,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-18T11:41:38.822307+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:41:38.822307+0000","last_peered":"2023-10-18T11:41:38.822307+0000","last_clean":"2023-10-18T11:41:38.822307+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:41:38.822307+0000","last_undegraded":"2023-10-18T11:41:38.822307+0000","last_fullsized":"2023-10-18T11:41:38.822307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1538,"num_read_kb":3175,"num_write":455,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575769,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51936,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319904,"statfs":{"total":96636764160,"available":96583581696,"internally_reserved":0,"allocated":2772992,"data_stored":1694032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.629}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.56399999999999995}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.54500000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.37},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.44700000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.41899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575769,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37468,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334372,"statfs":{"total":96636764160,"available":96598396928,"internally_reserved":0,"allocated":2179072,"data_stored":1103712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52600000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72999999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.441}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.39800000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.371}]}]},{"osd":5,"up_from":13,"seq":55834575769,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59676,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312164,"statfs":{"total":96636764160,"available":96575655936,"internally_reserved":0,"allocated":2174976,"data_stored":1103642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.50700000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.628},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60599999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.76000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575769,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41640,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330200,"statfs":{"total":96636764160,"available":96594124800,"internally_reserved":0,"allocated":2187264,"data_stored":1105254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.61199999999999999},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.27800000000000002},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.499},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.626,"15min":0.623},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.52700000000000002,"5min":0.45600000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.84499999999999997,"15min":0.86399999999999999},"last":0.71999999999999997}]},{"osd":2,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.81399999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84599999999999997,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.45200000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.30499999999999999},"max":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.33000000000000002},"max":{"1min":0.77000000000000002,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29999999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58899999999999997,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.36199999999999999},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.69899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575769,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59680,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312160,"statfs":{"total":96636764160,"available":96575651840,"internally_reserved":0,"allocated":2183168,"data_stored":1104258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.499},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.621},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.63700000000000001}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.752}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.70299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575769,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59684,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312156,"statfs":{"total":96636764160,"available":96575647744,"internally_reserved":0,"allocated":2183168,"data_stored":1103754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40500000000000003}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.48799999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.434},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.65800000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.61599999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.751}]}]},{"osd":1,"up_from":13,"seq":55834575769,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37472,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334368,"statfs":{"total":96636764160,"available":96598392832,"internally_reserved":0,"allocated":2179072,"data_stored":1103712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.435},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.40899999999999997}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.91000000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.70899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575768,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38072,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333768,"statfs":{"total":96636764160,"available":96597778432,"internally_reserved":0,"allocated":2793472,"data_stored":1696328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.26100000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.441},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.36899999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.39800000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:41:47.269 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T11:41:47.270 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T11:41:47.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.380+0000 7faa1615e640 1 -- 172.21.15.125:0/2162297282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa1006ae40 msgr2=0x7faa100b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.380+0000 7faa1615e640 1 --2- 172.21.15.125:0/2162297282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa1006ae40 0x7faa100b0e90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faa04005fd0 tx=0x7faa0402be10 comp rx=0 tx=0).stop 2023-10-18T11:41:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.380+0000 7faa1615e640 1 -- 172.21.15.125:0/2162297282 shutdown_connections 2023-10-18T11:41:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.380+0000 7faa1615e640 1 --2- 172.21.15.125:0/2162297282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa1015c1f0 0x7faa10157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.380+0000 7faa1615e640 1 --2- 172.21.15.125:0/2162297282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa1006ae40 0x7faa100b0e90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.380+0000 7faa1615e640 1 -- 172.21.15.125:0/2162297282 >> 172.21.15.125:0/2162297282 conn(0x7faa1006fae0 msgr2=0x7faa1006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:47.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.380+0000 7faa1615e640 1 -- 172.21.15.125:0/2162297282 shutdown_connections 2023-10-18T11:41:47.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.380+0000 7faa1615e640 1 -- 172.21.15.125:0/2162297282 wait complete. 2023-10-18T11:41:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.381+0000 7faa1615e640 1 Processor -- start 2023-10-18T11:41:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.381+0000 7faa1615e640 1 -- start start 2023-10-18T11:41:47.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa1615e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa1006ae40 0x7faa101455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa1615e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa101579f0 0x7faa100ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa1615e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa1015c1f0 0x7faa101e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa1615e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100ae690 con 0x7faa1006ae40 2023-10-18T11:41:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa1615e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100ada80 con 0x7faa101579f0 2023-10-18T11:41:47.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa1615e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa100add60 con 0x7faa1015c1f0 2023-10-18T11:41:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa1006ae40 0x7faa101455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa1006ae40 0x7faa101455a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56670/0 (socket says 172.21.15.125:56670) 2023-10-18T11:41:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 -- 172.21.15.125:0/4060894589 learned_addr learned my addr 172.21.15.125:0/4060894589 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0effd640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa101579f0 0x7faa100ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:47.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0ffff640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa1015c1f0 0x7faa101e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:47.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 -- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa1015c1f0 msgr2=0x7faa101e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa1015c1f0 0x7faa101e44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 -- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa101579f0 msgr2=0x7faa100ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa101579f0 0x7faa100ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 -- 172.21.15.125:0/4060894589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa04009560 con 0x7faa1006ae40 2023-10-18T11:41:47.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0effd640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa101579f0 0x7faa100ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:47.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa0f7fe640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa1006ae40 0x7faa101455a0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7faa0000c400 tx=0x7faa0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:47.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7fa9f7fff640 1 -- 172.21.15.125:0/4060894589 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa000052d0 con 0x7faa1006ae40 2023-10-18T11:41:47.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa100ae040 con 0x7faa1006ae40 2023-10-18T11:41:47.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.382+0000 7fa9f7fff640 1 -- 172.21.15.125:0/4060894589 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa00007020 con 0x7faa1006ae40 2023-10-18T11:41:47.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.383+0000 7fa9f7fff640 1 -- 172.21.15.125:0/4060894589 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00007470 con 0x7faa1006ae40 2023-10-18T11:41:47.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.383+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa100ad790 con 0x7faa1006ae40 2023-10-18T11:41:47.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.384+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa100690e0 con 0x7faa1006ae40 2023-10-18T11:41:47.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.384+0000 7fa9f7fff640 1 -- 172.21.15.125:0/4060894589 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faa000067d0 con 0x7faa1006ae40 2023-10-18T11:41:47.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.384+0000 7fa9f7fff640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fa9fc06af60 0x7fa9fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:47.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.384+0000 7fa9f7fff640 1 -- 172.21.15.125:0/4060894589 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7faa00089000 con 0x7faa1006ae40 2023-10-18T11:41:47.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.385+0000 7faa0effd640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fa9fc06af60 0x7fa9fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:47.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.385+0000 7faa0effd640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fa9fc06af60 0x7fa9fc06d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa04035000 tx=0x7faa04004730 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:47.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.389+0000 7fa9f7fff640 1 -- 172.21.15.125:0/4060894589 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa00053dc0 con 0x7faa1006ae40 2023-10-18T11:41:47.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.621+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faa1014ae50 con 0x7faa1006ae40 2023-10-18T11:41:47.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.623+0000 7fa9f7fff640 1 -- 172.21.15.125:0/4060894589 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v49) v1 ==== 190+0+147 (secure 0 0 0) 0x7faa000573e0 con 0x7faa1006ae40 2023-10-18T11:41:47.626 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T11:41:47.626 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T11:41:47.626 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T11:41:47.626 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T11:41:47.627 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T11:41:47.627 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T11:41:47.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.625+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fa9fc06af60 msgr2=0x7fa9fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.625+0000 7faa1615e640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fa9fc06af60 0x7fa9fc06d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa04035000 tx=0x7faa04004730 comp rx=0 tx=0).stop 2023-10-18T11:41:47.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.625+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa1006ae40 msgr2=0x7faa101455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.625+0000 7faa1615e640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa1006ae40 0x7faa101455a0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7faa0000c400 tx=0x7faa0000c8c0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.626+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 shutdown_connections 2023-10-18T11:41:47.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.626+0000 7faa1615e640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fa9fc06af60 0x7fa9fc06d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.626+0000 7faa1615e640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa101579f0 0x7faa100ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.626+0000 7faa1615e640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa1015c1f0 0x7faa101e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.626+0000 7faa1615e640 1 --2- 172.21.15.125:0/4060894589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa1006ae40 0x7faa101455a0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.626+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 >> 172.21.15.125:0/4060894589 conn(0x7faa1006fae0 msgr2=0x7faa10069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:47.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.626+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 shutdown_connections 2023-10-18T11:41:47.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.626+0000 7faa1615e640 1 -- 172.21.15.125:0/4060894589 wait complete. 2023-10-18T11:41:47.639 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:41:47.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.739+0000 7f09724f2640 1 -- 172.21.15.125:0/2070087090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 msgr2=0x7f0954004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.739+0000 7f09724f2640 1 --2- 172.21.15.125:0/2070087090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 0x7f0954004c40 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f096000d7f0 tx=0x7f09540057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.739+0000 7f09724f2640 1 -- 172.21.15.125:0/2070087090 shutdown_connections 2023-10-18T11:41:47.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.739+0000 7f09724f2640 1 --2- 172.21.15.125:0/2070087090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 0x7f0954004c40 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.739+0000 7f09724f2640 1 -- 172.21.15.125:0/2070087090 >> 172.21.15.125:0/2070087090 conn(0x7f096c069930 msgr2=0x7f096c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:47.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.739+0000 7f09724f2640 1 -- 172.21.15.125:0/2070087090 shutdown_connections 2023-10-18T11:41:47.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.739+0000 7f09724f2640 1 -- 172.21.15.125:0/2070087090 wait complete. 2023-10-18T11:41:47.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.740+0000 7f09724f2640 1 Processor -- start 2023-10-18T11:41:47.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.741+0000 7f09724f2640 1 -- start start 2023-10-18T11:41:47.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.741+0000 7f09724f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 0x7f096c13d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:47.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.741+0000 7f09724f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c151f20 0x7f096c06ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:47.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.741+0000 7f09724f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c1e50a0 0x7f096c1e5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:47.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.741+0000 7f09724f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1590e0 con 0x7f09540047c0 2023-10-18T11:41:47.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.741+0000 7f09724f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1e5a60 con 0x7f096c151f20 2023-10-18T11:41:47.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.741+0000 7f09724f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c06c630 con 0x7f096c1e50a0 2023-10-18T11:41:47.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.741+0000 7f096bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 0x7f096c13d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:47.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 0x7f096c13d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49230/0 (socket says 172.21.15.125:49230) 2023-10-18T11:41:47.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096bfff640 1 -- 172.21.15.125:0/643322111 learned_addr learned my addr 172.21.15.125:0/643322111 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:47.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f0970a8f640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c1e50a0 0x7f096c1e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:47.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096b7fe640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c151f20 0x7f096c06ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:47.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096bfff640 1 -- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c1e50a0 msgr2=0x7f096c1e5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096bfff640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c1e50a0 0x7f096c1e5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096bfff640 1 -- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c151f20 msgr2=0x7f096c06ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096bfff640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c151f20 0x7f096c06ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096bfff640 1 -- 172.21.15.125:0/643322111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c06c350 con 0x7f09540047c0 2023-10-18T11:41:47.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096b7fe640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c151f20 0x7f096c06ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:41:47.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f096bfff640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 0x7f096c13d1f0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f095c006610 tx=0x7f095c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:47.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f0968ff9640 1 -- 172.21.15.125:0/643322111 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00f960 con 0x7f09540047c0 2023-10-18T11:41:47.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f0968ff9640 1 -- 172.21.15.125:0/643322111 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f095c00fac0 con 0x7f09540047c0 2023-10-18T11:41:47.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c158a20 con 0x7f09540047c0 2023-10-18T11:41:47.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f0968ff9640 1 -- 172.21.15.125:0/643322111 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00feb0 con 0x7f09540047c0 2023-10-18T11:41:47.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.742+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c13d730 con 0x7f09540047c0 2023-10-18T11:41:47.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.743+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c0690e0 con 0x7f09540047c0 2023-10-18T11:41:47.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.743+0000 7f0968ff9640 1 -- 172.21.15.125:0/643322111 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f095c011430 con 0x7f09540047c0 2023-10-18T11:41:47.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.744+0000 7f0968ff9640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:47.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.744+0000 7f0968ff9640 1 -- 172.21.15.125:0/643322111 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f095c08a800 con 0x7f09540047c0 2023-10-18T11:41:47.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.744+0000 7f096b7fe640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:47.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.745+0000 7f096b7fe640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f095806af60 0x7f095806d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0960035000 tx=0x7f0960003760 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:47.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.748+0000 7f0968ff9640 1 -- 172.21.15.125:0/643322111 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f095c0555c0 con 0x7f09540047c0 2023-10-18T11:41:47.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.949+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f096c06f700 con 0x7f09540047c0 2023-10-18T11:41:47.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.950+0000 7f0968ff9640 1 -- 172.21.15.125:0/643322111 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f095c020020 con 0x7f09540047c0 2023-10-18T11:41:47.953 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:47.953 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":67,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3883749389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3883749389},{"type":"v1","addr":"172.21.15.139:6836","nonce":3883749389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2038008432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2038008432},{"type":"v1","addr":"172.21.15.139:6837","nonce":2038008432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":10317,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1543508506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1543508506},{"type":"v1","addr":"172.21.15.125:6834","nonce":1543508506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:41:38.149238+0000","modified":"2023-10-18T11:41:40.479014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10320":{"gid":10320,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2949284679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2949284679},{"type":"v1","addr":"172.21.15.125:6835","nonce":2949284679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T11:41:47.953 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 67 2023-10-18T11:41:47.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f095806af60 msgr2=0x7f095806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f095806af60 0x7f095806d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0960035000 tx=0x7f0960003760 comp rx=0 tx=0).stop 2023-10-18T11:41:47.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 msgr2=0x7f096c13d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:47.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 0x7f096c13d1f0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f095c006610 tx=0x7f095c0039c0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 shutdown_connections 2023-10-18T11:41:47.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f095806af60 0x7f095806d420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f096c151f20 0x7f096c06ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f096c1e50a0 0x7f096c1e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 --2- 172.21.15.125:0/643322111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09540047c0 0x7f096c13d1f0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:47.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 >> 172.21.15.125:0/643322111 conn(0x7f096c069930 msgr2=0x7f096c14fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:47.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 shutdown_connections 2023-10-18T11:41:47.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:47.952+0000 7f09724f2640 1 -- 172.21.15.125:0/643322111 wait complete. 2023-10-18T11:41:48.367 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T11:41:38.149238+0000', 'modified': '2023-10-18T11:41:40.479014+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10320}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10320': {'gid': 10320, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/2949284679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 2949284679}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 2949284679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T11:41:48.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T11:41:48.368 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T11:41:48.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.473+0000 7f1bfc2da640 1 -- 172.21.15.125:0/2274531090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf415b2a0 msgr2=0x7f1bf40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.473+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/2274531090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf415b2a0 0x7f1bf40cf720 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f1be4005fd0 tx=0x7f1be402bcf0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.473+0000 7f1bfc2da640 1 -- 172.21.15.125:0/2274531090 shutdown_connections 2023-10-18T11:41:48.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.473+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/2274531090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf415b2a0 0x7f1bf40cf720 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.473+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/2274531090 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bf40cf340 0x7f1bf40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.473+0000 7f1bfc2da640 1 -- 172.21.15.125:0/2274531090 >> 172.21.15.125:0/2274531090 conn(0x7f1bf40d0d60 msgr2=0x7f1bf40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:48.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.473+0000 7f1bfc2da640 1 -- 172.21.15.125:0/2274531090 shutdown_connections 2023-10-18T11:41:48.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.473+0000 7f1bfc2da640 1 -- 172.21.15.125:0/2274531090 wait complete. 2023-10-18T11:41:48.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.474+0000 7f1bfc2da640 1 Processor -- start 2023-10-18T11:41:48.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.474+0000 7f1bfc2da640 1 -- start start 2023-10-18T11:41:48.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfc2da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bf40ceb40 0x7f1bf40b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfc2da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf40cf340 0x7f1bf41f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfc2da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bf415b2a0 0x7f1bf40c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfc2da640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf41f57d0 con 0x7f1bf40cf340 2023-10-18T11:41:48.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfc2da640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf40af3b0 con 0x7f1bf40ceb40 2023-10-18T11:41:48.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfc2da640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf40af600 con 0x7f1bf415b2a0 2023-10-18T11:41:48.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf40cf340 0x7f1bf41f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf40cf340 0x7f1bf41f5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49258/0 (socket says 172.21.15.125:49258) 2023-10-18T11:41:48.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 -- 172.21.15.125:0/1220711721 learned_addr learned my addr 172.21.15.125:0/1220711721 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:48.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfa076640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bf40ceb40 0x7f1bf40b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfa877640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bf415b2a0 0x7f1bf40c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 -- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bf415b2a0 msgr2=0x7f1bf40c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bf415b2a0 0x7f1bf40c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 -- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bf40ceb40 msgr2=0x7f1bf40b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bf40ceb40 0x7f1bf40b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 -- 172.21.15.125:0/1220711721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be4009560 con 0x7f1bf40cf340 2023-10-18T11:41:48.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bf9875640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf40cf340 0x7f1bf41f5290 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f1be4036040 tx=0x7f1be402da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:48.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1beaffd640 1 -- 172.21.15.125:0/1220711721 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4003800 con 0x7f1bf40cf340 2023-10-18T11:41:48.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1beaffd640 1 -- 172.21.15.125:0/1220711721 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1be4003a50 con 0x7f1bf40cf340 2023-10-18T11:41:48.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.475+0000 7f1bfc2da640 1 -- 172.21.15.125:0/1220711721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf40af8e0 con 0x7f1bf40cf340 2023-10-18T11:41:48.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.476+0000 7f1beaffd640 1 -- 172.21.15.125:0/1220711721 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4003e00 con 0x7f1bf40cf340 2023-10-18T11:41:48.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.476+0000 7f1bfc2da640 1 -- 172.21.15.125:0/1220711721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf4157140 con 0x7f1bf40cf340 2023-10-18T11:41:48.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.477+0000 7f1beaffd640 1 -- 172.21.15.125:0/1220711721 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1be403e450 con 0x7f1bf40cf340 2023-10-18T11:41:48.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.477+0000 7f1beaffd640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f1bd806af60 0x7f1bd806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.477+0000 7f1beaffd640 1 -- 172.21.15.125:0/1220711721 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f1be40ac500 con 0x7f1bf40cf340 2023-10-18T11:41:48.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.477+0000 7f1bfa076640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f1bd806af60 0x7f1bd806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.478+0000 7f1bfa076640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f1bd806af60 0x7f1bd806d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0007350 tx=0x7f1bf00072e0 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:48.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.478+0000 7f1bfee6d740 1 Processor -- start 2023-10-18T11:41:48.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.478+0000 7f1bfee6d740 1 -- start start 2023-10-18T11:41:48.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac1dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac4af20 0x562daac4d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac1e020 0x562daac4de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daab2f730 con 0x562daac4ab40 2023-10-18T11:41:48.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daab90780 con 0x562daac1e020 2023-10-18T11:41:48.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562daab8de70 con 0x562daac4af20 2023-10-18T11:41:48.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfa076640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac1dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfa076640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac1dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49272/0 (socket says 172.21.15.125:49272) 2023-10-18T11:41:48.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfa076640 1 -- 172.21.15.125:0/3123992308 learned_addr learned my addr 172.21.15.125:0/3123992308 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:48.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfa076640 1 -- 172.21.15.125:0/3123992308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac4af20 msgr2=0x562daac4d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:41:48.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfa076640 1 --2- 172.21.15.125:0/3123992308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac4af20 0x562daac4d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfa076640 1 -- 172.21.15.125:0/3123992308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac1e020 msgr2=0x562daac4de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:41:48.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfa076640 1 --2- 172.21.15.125:0/3123992308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac1e020 0x562daac4de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.479+0000 7f1bfa076640 1 -- 172.21.15.125:0/3123992308 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562daab8e3d0 con 0x562daac4ab40 2023-10-18T11:41:48.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bfa076640 1 --2- 172.21.15.125:0/3123992308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac1dae0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0010570 tx=0x7f1bf0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcb5527955b7a52 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:48.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bd7fff640 1 -- 172.21.15.125:0/3123992308 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf001d860 con 0x562daac4ab40 2023-10-18T11:41:48.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bd7fff640 1 -- 172.21.15.125:0/3123992308 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bf001dab0 con 0x562daac4ab40 2023-10-18T11:41:48.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bd7fff640 1 -- 172.21.15.125:0/3123992308 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf001de80 con 0x562daac4ab40 2023-10-18T11:41:48.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bfee6d740 1 -- 172.21.15.125:0/3123992308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 msgr2=0x562daac1dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/3123992308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac1dae0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0010570 tx=0x7f1bf0011000 comp rx=0 tx=0).stop 2023-10-18T11:41:48.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bfee6d740 1 -- 172.21.15.125:0/3123992308 shutdown_connections 2023-10-18T11:41:48.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/3123992308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac1e020 0x562daac4de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/3123992308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac4af20 0x562daac4d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/3123992308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac1dae0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.480+0000 7f1bfee6d740 1 -- 172.21.15.125:0/3123992308 >> 172.21.15.125:0/3123992308 conn(0x562daac1fd50 msgr2=0x562daac688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:48.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.481+0000 7f1bfee6d740 1 -- 172.21.15.125:0/3123992308 shutdown_connections 2023-10-18T11:41:48.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.481+0000 7f1bfee6d740 1 -- 172.21.15.125:0/3123992308 wait complete. 2023-10-18T11:41:48.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.481+0000 7f1bfee6d740 1 -- start start 2023-10-18T11:41:48.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.481+0000 7f1bfee6d740 1 Processor -- start 2023-10-18T11:41:48.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daac4d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4af20 0x562daac43a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac321d0 0x562daac313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daab89760 con 0x562daac4af20 2023-10-18T11:41:48.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daab448f0 con 0x562daac4ab40 2023-10-18T11:41:48.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562daab44590 con 0x562daac321d0 2023-10-18T11:41:48.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfee6d740 10 client.0 _ll_register_callbacks cb 0x562daab80ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T11:41:48.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bf9875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4af20 0x562daac43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bf9875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4af20 0x562daac43a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49286/0 (socket says 172.21.15.125:49286) 2023-10-18T11:41:48.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bf9875640 1 -- 172.21.15.125:0/331420102 learned_addr learned my addr 172.21.15.125:0/331420102 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:48.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfa877640 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac321d0 0x562daac313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfa076640 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daac4d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bf9875640 1 -- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac321d0 msgr2=0x562daac313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:41:48.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bf9875640 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac321d0 0x562daac313e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bf9875640 1 -- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 msgr2=0x562daac4d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:41:48.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bf9875640 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daac4d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bf9875640 1 -- 172.21.15.125:0/331420102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf0007e20 con 0x562daac4af20 2023-10-18T11:41:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.482+0000 7f1bfa076640 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daac4d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bf9875640 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4af20 0x562daac43a20 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f1be407afa0 tx=0x7f1be407b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5b804265187cd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T11:41:48.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bd57fa640 1 -- 172.21.15.125:0/331420102 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be407fea0 con 0x562daac4af20 2023-10-18T11:41:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bd57fa640 1 -- 172.21.15.125:0/331420102 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f1be40ac0e0 con 0x562daac4af20 2023-10-18T11:41:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 1 -- 172.21.15.125:0/331420102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562daab82740 con 0x562daac4af20 2023-10-18T11:41:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bd57fa640 1 -- 172.21.15.125:0/331420102 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be40812b0 con 0x562daac4af20 2023-10-18T11:41:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 10 client.10479 fetch_fsmap learned FSMap version 67 2023-10-18T11:41:48.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 1 -- 172.21.15.125:0/331420102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x562daab87de0 con 0x562daac4af20 2023-10-18T11:41:48.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bd57fa640 1 -- 172.21.15.125:0/331420102 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x562daab82740 con 0x562daac4af20 2023-10-18T11:41:48.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bd57fa640 1 -- 172.21.15.125:0/331420102 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1be404e020 con 0x562daac4af20 2023-10-18T11:41:48.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 10 client.10479 fetch_fsmap finished waiting for FSMap version 67 2023-10-18T11:41:48.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 10 client.10479 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T11:41:48.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 20 client.10479 populate_metadata read hostname 'smithi125' 2023-10-18T11:41:48.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] conn(0x562daac45a20 0x562daac45e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 4 client.10479 mds_command: new command op to 10320 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T11:41:48.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfee6d740 1 -- 172.21.15.125:0/331420102 --> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562daab82460 con 0x562daac45a20 2023-10-18T11:41:48.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.483+0000 7f1bfa076640 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] conn(0x562daac45a20 0x562daac45e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bd57fa640 10 client.10479 ms_handle_connect on v2:172.21.15.125:6833/2949284679 2023-10-18T11:41:48.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfa076640 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] conn(0x562daac45a20 0x562daac45e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f5a1202c754ef1 server_cookie=28109a7193d565b4 in_seq=0 out_seq=0 2023-10-18T11:41:48.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bd57fa640 1 -- 172.21.15.125:0/331420102 <== mds.0 v2:172.21.15.125:6833/2949284679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562daab82460 con 0x562daac45a20 2023-10-18T11:41:48.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bd57fa640 10 client.10479 handle_command_reply: tid=0 2023-10-18T11:41:48.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 client.10479 shutdown 2023-10-18T11:41:48.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 -- 172.21.15.125:0/331420102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4af20 msgr2=0x562daac43a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4af20 0x562daac43a20 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f1be407afa0 tx=0x7f1be407b850 comp rx=0 tx=0).stop 2023-10-18T11:41:48.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 -- 172.21.15.125:0/331420102 shutdown_connections 2023-10-18T11:41:48.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] conn(0x562daac45a20 0x562daac45e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac321d0 0x562daac313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4af20 0x562daac43a20 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bd57fa640 0 client.10479 ms_handle_reset on v2:172.21.15.125:6833/2949284679 2023-10-18T11:41:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/331420102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daac4d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 -- 172.21.15.125:0/331420102 >> 172.21.15.125:0/331420102 conn(0x562daac1e020 msgr2=0x562daac219f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 -- 172.21.15.125:0/331420102 shutdown_connections 2023-10-18T11:41:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.484+0000 7f1bfee6d740 1 -- 172.21.15.125:0/331420102 wait complete. 2023-10-18T11:41:48.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.485+0000 7f1bfee6d740 20 client.10479 trim_cache size 0 max 16384 2023-10-18T11:41:48.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfee6d740 1 Processor -- start 2023-10-18T11:41:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfee6d740 1 -- start start 2023-10-18T11:41:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac321d0 0x562daac45e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac21c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daab0a270 con 0x562daac4ab40 2023-10-18T11:41:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daac4cbb0 con 0x562daac321d0 2023-10-18T11:41:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562daac4a2b0 con 0x562daac45a20 2023-10-18T11:41:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.500+0000 7f1bfa877640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac21c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa877640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac21c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49288/0 (socket says 172.21.15.125:49288) 2023-10-18T11:41:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa877640 1 -- 172.21.15.125:0/2572718611 learned_addr learned my addr 172.21.15.125:0/2572718611 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bf9875640 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa076640 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac321d0 0x562daac45e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa877640 1 -- 172.21.15.125:0/2572718611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 msgr2=0x562daac216e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:41:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa877640 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac216e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa877640 1 -- 172.21.15.125:0/2572718611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac321d0 msgr2=0x562daac45e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:41:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa877640 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac321d0 0x562daac45e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa877640 1 -- 172.21.15.125:0/2572718611 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562daab26d00 con 0x562daac4ab40 2023-10-18T11:41:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1bfa877640 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac21c20 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f1bec007830 tx=0x7f1bec0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e9a8a892dab959 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1be8e00640 1 -- 172.21.15.125:0/2572718611 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0106e0 con 0x562daac4ab40 2023-10-18T11:41:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1be8e00640 1 -- 172.21.15.125:0/2572718611 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bec010950 con 0x562daac4ab40 2023-10-18T11:41:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.501+0000 7f1be8e00640 1 -- 172.21.15.125:0/2572718611 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec010d60 con 0x562daac4ab40 2023-10-18T11:41:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 -- 172.21.15.125:0/2572718611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 msgr2=0x562daac21c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac21c20 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f1bec007830 tx=0x7f1bec0050d0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 -- 172.21.15.125:0/2572718611 shutdown_connections 2023-10-18T11:41:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac4ab40 0x562daac21c20 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/2572718611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac321d0 0x562daac45e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 -- 172.21.15.125:0/2572718611 >> 172.21.15.125:0/2572718611 conn(0x562daac1e020 msgr2=0x562daac43830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 -- 172.21.15.125:0/2572718611 shutdown_connections 2023-10-18T11:41:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 -- 172.21.15.125:0/2572718611 wait complete. 2023-10-18T11:41:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 -- start start 2023-10-18T11:41:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.502+0000 7f1bfee6d740 1 Processor -- start 2023-10-18T11:41:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac321d0 0x562daac216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac21c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfee6d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daacd4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daac33100 con 0x562daac321d0 2023-10-18T11:41:48.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daac46680 con 0x562daac4ab40 2023-10-18T11:41:48.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfee6d740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562daac463a0 con 0x562daac45a20 2023-10-18T11:41:48.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfee6d740 10 client.0 _ll_register_callbacks cb 0x7f1bf0014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T11:41:48.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfa076640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac321d0 0x562daac216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfa076640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac321d0 0x562daac216e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49296/0 (socket says 172.21.15.125:49296) 2023-10-18T11:41:48.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfa076640 1 -- 172.21.15.125:0/1390997150 learned_addr learned my addr 172.21.15.125:0/1390997150 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:48.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfa877640 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daacd4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bf9875640 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac21c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.503+0000 7f1bfa076640 1 -- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 msgr2=0x562daac21c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:41:48.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfa076640 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac21c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfa076640 1 -- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 msgr2=0x562daacd4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:41:48.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfa076640 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daacd4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfa076640 1 -- 172.21.15.125:0/1390997150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec00f040 con 0x562daac321d0 2023-10-18T11:41:48.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bf9875640 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac21c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:48.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfa076640 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac321d0 0x562daac216e0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0007be0 tx=0x7f1bf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13031eb817d05a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:48.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T11:41:48.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bd57fa640 1 -- 172.21.15.125:0/1390997150 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf002d040 con 0x562daac321d0 2023-10-18T11:41:48.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bd57fa640 1 -- 172.21.15.125:0/1390997150 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f1bf0032900 con 0x562daac321d0 2023-10-18T11:41:48.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 1 -- 172.21.15.125:0/1390997150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bf001dab0 con 0x562daac321d0 2023-10-18T11:41:48.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bd57fa640 1 -- 172.21.15.125:0/1390997150 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf0032b00 con 0x562daac321d0 2023-10-18T11:41:48.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 10 client.10485 fetch_fsmap learned FSMap version 67 2023-10-18T11:41:48.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 1 -- 172.21.15.125:0/1390997150 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x562daab4c4f0 con 0x562daac321d0 2023-10-18T11:41:48.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bd57fa640 1 -- 172.21.15.125:0/1390997150 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bf001dab0 con 0x562daac321d0 2023-10-18T11:41:48.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bd57fa640 1 -- 172.21.15.125:0/1390997150 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bf0038020 con 0x562daac321d0 2023-10-18T11:41:48.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 10 client.10485 fetch_fsmap finished waiting for FSMap version 67 2023-10-18T11:41:48.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 10 client.10485 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T11:41:48.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 20 client.10485 populate_metadata read hostname 'smithi125' 2023-10-18T11:41:48.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] conn(0x562daac4af20 0x562daacd0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 4 client.10485 mds_command: new command op to 10320 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T11:41:48.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bfee6d740 1 -- 172.21.15.125:0/1390997150 --> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562daa88b6c0 con 0x562daac4af20 2023-10-18T11:41:48.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.504+0000 7f1bf9875640 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] conn(0x562daac4af20 0x562daacd0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bd57fa640 10 client.10485 ms_handle_connect on v2:172.21.15.125:6833/2949284679 2023-10-18T11:41:48.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bf9875640 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] conn(0x562daac4af20 0x562daacd0460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d146259263843b00 server_cookie=13f74abad0b9a130 in_seq=0 out_seq=0 2023-10-18T11:41:48.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bd57fa640 1 -- 172.21.15.125:0/1390997150 <== mds.0 v2:172.21.15.125:6833/2949284679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x562daa88b6c0 con 0x562daac4af20 2023-10-18T11:41:48.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bd57fa640 10 client.10485 handle_command_reply: tid=0 2023-10-18T11:41:48.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 client.10485 shutdown 2023-10-18T11:41:48.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 -- 172.21.15.125:0/1390997150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac321d0 msgr2=0x562daac216e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac321d0 0x562daac216e0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0007be0 tx=0x7f1bf0004a40 comp rx=0 tx=0).stop 2023-10-18T11:41:48.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 -- 172.21.15.125:0/1390997150 shutdown_connections 2023-10-18T11:41:48.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.125:6833/2949284679,v1:172.21.15.125:6835/2949284679] conn(0x562daac4af20 0x562daacd0460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562daac4ab40 0x562daacd4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562daac45a20 0x562daac21c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bd57fa640 0 client.10485 ms_handle_reset on v2:172.21.15.125:6833/2949284679 2023-10-18T11:41:48.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 --2- 172.21.15.125:0/1390997150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562daac321d0 0x562daac216e0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 -- 172.21.15.125:0/1390997150 >> 172.21.15.125:0/1390997150 conn(0x562daac1e020 msgr2=0x562daac44f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:48.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 -- 172.21.15.125:0/1390997150 shutdown_connections 2023-10-18T11:41:48.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 1 -- 172.21.15.125:0/1390997150 wait complete. 2023-10-18T11:41:48.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.505+0000 7f1bfee6d740 20 client.10485 trim_cache size 0 max 16384 2023-10-18T11:41:48.533 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:41:48.534 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T11:41:48.534 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T11:41:48.534 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 10320, 2023-10-18T11:41:48.534 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T11:41:48.535 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T11:41:48.535 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T11:41:48.535 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 9.6838446690000008, 2023-10-18T11:41:48.535 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 67, 2023-10-18T11:41:48.535 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 174, 2023-10-18T11:41:48.536 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 174, 2023-10-18T11:41:48.536 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 15.460452755 2023-10-18T11:41:48.536 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:41:48.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 -- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f1bd806af60 msgr2=0x7f1bd806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f1bd806af60 0x7f1bd806d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0007350 tx=0x7f1bf00072e0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 -- 172.21.15.125:0/1220711721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf40cf340 msgr2=0x7f1bf41f5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf40cf340 0x7f1bf41f5290 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f1be4036040 tx=0x7f1be402da70 comp rx=0 tx=0).stop 2023-10-18T11:41:48.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 -- 172.21.15.125:0/1220711721 shutdown_connections 2023-10-18T11:41:48.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f1bd806af60 0x7f1bd806d420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1bf415b2a0 0x7f1bf40c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1bf40cf340 0x7f1bf41f5290 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 --2- 172.21.15.125:0/1220711721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bf40ceb40 0x7f1bf40b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 -- 172.21.15.125:0/1220711721 >> 172.21.15.125:0/1220711721 conn(0x7f1bf40d0d60 msgr2=0x7f1bf406b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:48.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 -- 172.21.15.125:0/1220711721 shutdown_connections 2023-10-18T11:41:48.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.506+0000 7f1bfc2da640 1 -- 172.21.15.125:0/1220711721 wait complete. 2023-10-18T11:41:48.541 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T11:41:48.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.642+0000 7f86dac7c640 1 -- 172.21.15.125:0/2307331986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d40abc10 msgr2=0x7f86d40c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.642+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2307331986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d40abc10 0x7f86d40c9b30 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009ca0 tx=0x7f86cc02c230 comp rx=0 tx=0).stop 2023-10-18T11:41:48.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.642+0000 7f86dac7c640 1 -- 172.21.15.125:0/2307331986 shutdown_connections 2023-10-18T11:41:48.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.642+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2307331986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d40abc10 0x7f86d40c9b30 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.642+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2307331986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86d406af00 0x7f86d406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.642+0000 7f86dac7c640 1 -- 172.21.15.125:0/2307331986 >> 172.21.15.125:0/2307331986 conn(0x7f86d4069930 msgr2=0x7f86d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:48.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.642+0000 7f86dac7c640 1 -- 172.21.15.125:0/2307331986 shutdown_connections 2023-10-18T11:41:48.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.642+0000 7f86dac7c640 1 -- 172.21.15.125:0/2307331986 wait complete. 2023-10-18T11:41:48.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.643+0000 7f86dac7c640 1 Processor -- start 2023-10-18T11:41:48.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.644+0000 7f86dac7c640 1 -- start start 2023-10-18T11:41:48.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.644+0000 7f86dac7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d406af00 0x7f86d4138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.644+0000 7f86dac7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86d40abc10 0x7f86d41517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.644+0000 7f86dac7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d40c96b0 0x7f86d413d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.644+0000 7f86dac7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d4138e70 con 0x7f86d406af00 2023-10-18T11:41:48.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.644+0000 7f86dac7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41f3840 con 0x7f86d40c96b0 2023-10-18T11:41:48.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.644+0000 7f86dac7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40a7840 con 0x7f86d40abc10 2023-10-18T11:41:48.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.644+0000 7f86d8a18640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d406af00 0x7f86d4138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d8a18640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d406af00 0x7f86d4138930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49312/0 (socket says 172.21.15.125:49312) 2023-10-18T11:41:48.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d8a18640 1 -- 172.21.15.125:0/2488512569 learned_addr learned my addr 172.21.15.125:0/2488512569 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:48.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d3fff640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86d40abc10 0x7f86d41517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d9219640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d40c96b0 0x7f86d413d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d8a18640 1 -- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86d40abc10 msgr2=0x7f86d41517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d8a18640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86d40abc10 0x7f86d41517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d8a18640 1 -- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d40c96b0 msgr2=0x7f86d413d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d8a18640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d40c96b0 0x7f86d413d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d8a18640 1 -- 172.21.15.125:0/2488512569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc009900 con 0x7f86d406af00 2023-10-18T11:41:48.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d8a18640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d406af00 0x7f86d4138930 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f86c800c890 tx=0x7f86c800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:48.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d17fa640 1 -- 172.21.15.125:0/2488512569 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8004220 con 0x7f86d406af00 2023-10-18T11:41:48.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d17fa640 1 -- 172.21.15.125:0/2488512569 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f86c8004470 con 0x7f86d406af00 2023-10-18T11:41:48.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d41f2c30 con 0x7f86d406af00 2023-10-18T11:41:48.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86d17fa640 1 -- 172.21.15.125:0/2488512569 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8007c10 con 0x7f86d406af00 2023-10-18T11:41:48.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.645+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d41f31f0 con 0x7f86d406af00 2023-10-18T11:41:48.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.646+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d40690e0 con 0x7f86d406af00 2023-10-18T11:41:48.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.647+0000 7f86d17fa640 1 -- 172.21.15.125:0/2488512569 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f86c8005020 con 0x7f86d406af00 2023-10-18T11:41:48.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.647+0000 7f86d17fa640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f86b806af60 0x7f86b806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:48.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.647+0000 7f86d17fa640 1 -- 172.21.15.125:0/2488512569 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f86c8089ea0 con 0x7f86d406af00 2023-10-18T11:41:48.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.647+0000 7f86d3fff640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f86b806af60 0x7f86b806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:48.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.648+0000 7f86d3fff640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f86b806af60 0x7f86b806d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f86d40ac4d0 tx=0x7f86c0009060 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:48.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.651+0000 7f86d17fa640 1 -- 172.21.15.125:0/2488512569 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86c8054c80 con 0x7f86d406af00 2023-10-18T11:41:48.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.853+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f86d40cc420 con 0x7f86d406af00 2023-10-18T11:41:48.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.854+0000 7f86d17fa640 1 -- 172.21.15.125:0/2488512569 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f86c801d070 con 0x7f86d406af00 2023-10-18T11:41:48.857 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T11:41:48.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.855+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f86b806af60 msgr2=0x7f86b806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.855+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f86b806af60 0x7f86b806d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f86d40ac4d0 tx=0x7f86c0009060 comp rx=0 tx=0).stop 2023-10-18T11:41:48.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.855+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d406af00 msgr2=0x7f86d4138930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:48.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.855+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d406af00 0x7f86d4138930 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f86c800c890 tx=0x7f86c800cd50 comp rx=0 tx=0).stop 2023-10-18T11:41:48.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.856+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 shutdown_connections 2023-10-18T11:41:48.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.856+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f86b806af60 0x7f86b806d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.856+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d40c96b0 0x7f86d413d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.856+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86d40abc10 0x7f86d41517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.856+0000 7f86dac7c640 1 --2- 172.21.15.125:0/2488512569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86d406af00 0x7f86d4138930 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:48.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.856+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 >> 172.21.15.125:0/2488512569 conn(0x7f86d4069930 msgr2=0x7f86d4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:48.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.856+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 shutdown_connections 2023-10-18T11:41:48.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:48.856+0000 7f86dac7c640 1 -- 172.21.15.125:0/2488512569 wait complete. 2023-10-18T11:41:48.869 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T11:41:48.869 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T11:41:48.869 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T11:41:48.870 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T11:41:48.870 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T11:41:48.870 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T11:41:48.871 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T11:41:48.871 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T11:41:48.871 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:41:48.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:48.872 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T11:41:48.887 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T11:41:48.888 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T11:41:48.888 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T11:41:48.888 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:41:48.888 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T11:41:48.888 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:41:48.889 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:41:48.889 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:41:48.889 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T11:41:48.889 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:41:48.889 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:41:48.890 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T11:41:48.890 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:41:48.890 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:41:48.890 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T11:41:48.890 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:41:48.891 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T11:41:48.891 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 38115sec preferred_lft 38115sec 2023-10-18T11:41:48.891 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T11:41:48.891 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:41:48.891 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T11:41:48.892 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:41:48.892 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T11:41:48.892 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:41:48.892 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T11:41:48.892 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:41:48.893 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T11:41:48.893 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:41:48.893 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T11:41:48.893 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:41:48.893 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T11:41:48.894 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:41:48.894 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T11:41:48.894 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:41:48.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:48.895 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T11:41:48.954 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T11:41:48.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:48.956 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T11:41:49.013 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T11:41:49.013 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T11:41:49.014 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T11:41:49.014 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T11:41:49.014 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T11:41:49.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:49.015 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T11:41:49.016 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T11:41:49.016 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T11:41:49.017 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T11:41:49.017 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T11:41:49.018 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T11:41:49.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:49.108 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T11:41:49.123 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T11:41:49.123 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T11:41:49.124 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:41:49.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:49.125 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T11:41:49.191 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T11:41:49.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T11:41:49.192 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T11:41:49.246 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:41:49.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T11:41:49.247 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T11:41:49.300 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T11:41:49.367 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T11:41:49.423 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T11:41:49.423 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T11:41:49.424 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T11:41:49.424 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T11:41:49.424 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T11:41:49.424 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T11:41:49.425 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T11:41:49.425 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T11:41:49.425 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T11:41:49.425 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T11:41:49.426 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T11:41:49.426 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T11:41:49.426 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T11:41:49.426 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T11:41:49.427 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T11:41:49.427 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T11:41:49.427 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T11:41:49.427 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T11:41:49.428 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T11:41:49.428 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T11:41:49.428 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T11:41:49.428 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T11:41:49.428 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T11:41:49.429 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T11:41:49.429 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T11:41:49.429 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T11:41:49.429 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T11:41:49.429 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T11:41:49.429 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T11:41:49.430 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T11:41:49.430 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:41:49.430 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:41:49.430 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:41:49.430 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:41:49.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:49.431 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T11:41:49.478 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T11:41:49.479 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T11:41:49.520 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T11:41:49.547 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T11:41:49.602 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T11:41:49.602 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T11:41:49.602 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T11:41:49.603 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T11:41:49.603 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T11:41:49.603 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T11:41:49.603 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T11:41:49.603 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T11:41:49.603 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T11:41:49.604 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T11:41:49.604 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T11:41:49.604 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T11:41:49.604 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T11:41:49.604 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T11:41:49.605 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T11:41:49.605 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T11:41:49.605 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T11:41:49.605 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T11:41:49.605 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T11:41:49.605 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T11:41:49.606 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T11:41:49.606 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T11:41:49.606 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T11:41:49.606 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T11:41:49.606 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T11:41:49.607 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T11:41:49.607 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T11:41:49.607 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T11:41:49.607 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T11:41:49.608 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T11:41:49.608 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:41:49.608 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:41:49.608 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:41:49.609 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:41:49.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:49.610 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T11:41:49.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T11:41:49.603+0000 7f57d0579480 -1 init, newargv = 0x5617e5573c10 newargc=15 2023-10-18T11:41:49.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[143307]: starting ceph client 2023-10-18T11:41:49.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[143307]: starting fuse 2023-10-18T11:41:49.665 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T11:41:49.665 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T11:41:49.665 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T11:41:49.666 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T11:41:49.666 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T11:41:49.667 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T11:41:49.667 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T11:41:49.667 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:41:49.668 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T11:41:49.668 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T11:41:49.669 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T11:41:49.669 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T11:41:49.670 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:41:49.670 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T11:41:49.670 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T11:41:49.671 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T11:41:49.671 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T11:41:49.672 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:41:49.672 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T11:41:49.672 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T11:41:49.673 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T11:41:49.673 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T11:41:49.673 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T11:41:49.674 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T11:41:49.674 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T11:41:49.674 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T11:41:49.675 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:41:49.675 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T11:41:49.676 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T11:41:49.784 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.143307.asok 2023-10-18T11:41:49.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T11:41:49.795 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.143307.asok 2023-10-18T11:41:49.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:49.796 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.143307.asok status 2023-10-18T11:41:49.929 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:41:49.929 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T11:41:49.929 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T11:41:49.929 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T11:41:49.930 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T11:41:49.930 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T11:41:49.930 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T11:41:49.930 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "143307", 2023-10-18T11:41:49.930 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T11:41:49.931 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:49.931 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T11:41:49.931 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T11:41:49.931 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 10497, 2023-10-18T11:41:49.931 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T11:41:49.932 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T11:41:49.932 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T11:41:49.932 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 10497 2023-10-18T11:41:49.932 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:49.932 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T11:41:49.933 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T11:41:49.933 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T11:41:49.933 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 1802347388 2023-10-18T11:41:49.933 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:49.933 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:49.933 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T11:41:49.934 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T11:41:49.934 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T11:41:49.934 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 1802347388 2023-10-18T11:41:49.934 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:41:49.934 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.10497 192.168.0.1:0/1802347388", 2023-10-18T11:41:49.935 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/1802347388", 2023-10-18T11:41:49.935 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T11:41:49.935 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 67, 2023-10-18T11:41:49.935 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 174, 2023-10-18T11:41:49.935 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T11:41:49.936 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T11:41:49.936 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T11:41:49.936 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:41:49.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:49.938 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T11:41:49.938 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T11:41:49.953 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T11:41:49.953 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T11:41:49.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:49.954 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T11:41:50.020 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T11:41:50.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.161+0000 7f4cce133640 1 -- 172.21.15.125:0/1244386613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 msgr2=0x7f4cc8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:50.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.161+0000 7f4cce133640 1 --2- 172.21.15.125:0/1244386613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 0x7f4cc8158300 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009830 tx=0x7f4cb802bd60 comp rx=0 tx=0).stop 2023-10-18T11:41:50.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.161+0000 7f4cce133640 1 -- 172.21.15.125:0/1244386613 shutdown_connections 2023-10-18T11:41:50.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.161+0000 7f4cce133640 1 --2- 172.21.15.125:0/1244386613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cc80afa20 0x7f4cc80705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.161+0000 7f4cce133640 1 --2- 172.21.15.125:0/1244386613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 0x7f4cc8158300 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.161+0000 7f4cce133640 1 -- 172.21.15.125:0/1244386613 >> 172.21.15.125:0/1244386613 conn(0x7f4cc8069930 msgr2=0x7f4cc8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:50.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.161+0000 7f4cce133640 1 -- 172.21.15.125:0/1244386613 shutdown_connections 2023-10-18T11:41:50.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.161+0000 7f4cce133640 1 -- 172.21.15.125:0/1244386613 wait complete. 2023-10-18T11:41:50.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.162+0000 7f4cce133640 1 Processor -- start 2023-10-18T11:41:50.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.162+0000 7f4cce133640 1 -- start start 2023-10-18T11:41:50.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cce133640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc80afa20 0x7f4cc813d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:50.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cce133640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cc80b0530 0x7f4cc8159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:50.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cce133640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 0x7f4cc81596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:50.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cce133640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc813dac0 con 0x7f4cc80cbd20 2023-10-18T11:41:50.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cce133640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc813fbc0 con 0x7f4cc80afa20 2023-10-18T11:41:50.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cce133640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc813eb20 con 0x7f4cc80b0530 2023-10-18T11:41:50.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 0x7f4cc81596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:50.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 0x7f4cc81596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49330/0 (socket says 172.21.15.125:49330) 2023-10-18T11:41:50.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 -- 172.21.15.125:0/2479746745 learned_addr learned my addr 172.21.15.125:0/2479746745 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:50.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 -- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cc80b0530 msgr2=0x7f4cc8159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:50.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cc80b0530 0x7f4cc8159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 -- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc80afa20 msgr2=0x7f4cc813d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:50.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc80afa20 0x7f4cc813d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 -- 172.21.15.125:0/2479746745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb8009490 con 0x7f4cc80cbd20 2023-10-18T11:41:50.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.163+0000 7f4cc7fff640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 0x7f4cc81596c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000cb40 tx=0x7f4cc000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:50.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.164+0000 7f4caffff640 1 -- 172.21.15.125:0/2479746745 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc000aca0 con 0x7f4cc80cbd20 2023-10-18T11:41:50.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.164+0000 7f4caffff640 1 -- 172.21.15.125:0/2479746745 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4cc0012420 con 0x7f4cc80cbd20 2023-10-18T11:41:50.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.164+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc8070cf0 con 0x7f4cc80cbd20 2023-10-18T11:41:50.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.164+0000 7f4caffff640 1 -- 172.21.15.125:0/2479746745 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0012810 con 0x7f4cc80cbd20 2023-10-18T11:41:50.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.164+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc813f0e0 con 0x7f4cc80cbd20 2023-10-18T11:41:50.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.164+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc80690e0 con 0x7f4cc80cbd20 2023-10-18T11:41:50.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.165+0000 7f4caffff640 1 -- 172.21.15.125:0/2479746745 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4cc001b400 con 0x7f4cc80cbd20 2023-10-18T11:41:50.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.165+0000 7f4caffff640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4cb406af60 0x7f4cb406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:50.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.166+0000 7f4caffff640 1 -- 172.21.15.125:0/2479746745 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f4cc008a2b0 con 0x7f4cc80cbd20 2023-10-18T11:41:50.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.166+0000 7f4cc77fe640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4cb406af60 0x7f4cb406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:50.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.166+0000 7f4cc77fe640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4cb406af60 0x7f4cb406d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8036000 tx=0x7f4cb80046c0 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:50.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.170+0000 7f4caffff640 1 -- 172.21.15.125:0/2479746745 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4cc0055180 con 0x7f4cc80cbd20 2023-10-18T11:41:50.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.359+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 --> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cc80af460 con 0x7f4cb406af60 2023-10-18T11:41:50.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.361+0000 7f4caffff640 1 -- 172.21.15.125:0/2479746745 <== mgr.8644 v2:172.21.15.139:6832/4236255547 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4cc80af460 con 0x7f4cb406af60 2023-10-18T11:41:50.363 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T11:41:50.363 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:41:50.364 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T11:41:50.364 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:41:50.364 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T11:41:50.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4cb406af60 msgr2=0x7f4cb406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:50.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4cb406af60 0x7f4cb406d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8036000 tx=0x7f4cb80046c0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 msgr2=0x7f4cc81596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:50.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 0x7f4cc81596c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000cb40 tx=0x7f4cc000a060 comp rx=0 tx=0).stop 2023-10-18T11:41:50.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 shutdown_connections 2023-10-18T11:41:50.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f4cb406af60 0x7f4cb406d420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc80cbd20 0x7f4cc81596c0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cc80b0530 0x7f4cc8159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 --2- 172.21.15.125:0/2479746745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc80afa20 0x7f4cc813d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 >> 172.21.15.125:0/2479746745 conn(0x7f4cc8069930 msgr2=0x7f4cc80ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:50.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 shutdown_connections 2023-10-18T11:41:50.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.363+0000 7f4cce133640 1 -- 172.21.15.125:0/2479746745 wait complete. 2023-10-18T11:41:50.375 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T11:41:50.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.478+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/3189772398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b700047c0 msgr2=0x7f5b70004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:50.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.478+0000 7f5b8b4f2640 1 --2- 172.21.15.125:0/3189772398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b700047c0 0x7f5b70004c40 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009b20 tx=0x7f5b700057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.478+0000 7f5b89a8f640 1 -- 172.21.15.125:0/3189772398 reap_dead start 2023-10-18T11:41:50.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.478+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/3189772398 shutdown_connections 2023-10-18T11:41:50.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.478+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/3189772398 >> 172.21.15.125:0/3189772398 conn(0x7f5b84069930 msgr2=0x7f5b84069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:50.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.479+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/3189772398 shutdown_connections 2023-10-18T11:41:50.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.479+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/3189772398 wait complete. 2023-10-18T11:41:50.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b8b4f2640 1 Processor -- start 2023-10-18T11:41:50.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b8b4f2640 1 -- start start 2023-10-18T11:41:50.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b8b4f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b700047c0 0x7f5b84139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:50.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b8b4f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b8413bd00 0x7f5b8413c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:50.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b8b4f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b841e4750 0x7f5b841e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:50.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b8b4f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8413c6c0 con 0x7f5b8413bd00 2023-10-18T11:41:50.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b8b4f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b841480a0 con 0x7f5b841e4750 2023-10-18T11:41:50.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b8b4f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84148380 con 0x7f5b700047c0 2023-10-18T11:41:50.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b88a8d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b8413bd00 0x7f5b8413c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:50.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b88a8d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b8413bd00 0x7f5b8413c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49344/0 (socket says 172.21.15.125:49344) 2023-10-18T11:41:50.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.480+0000 7f5b88a8d640 1 -- 172.21.15.125:0/2069230511 learned_addr learned my addr 172.21.15.125:0/2069230511 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:50.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b88a8d640 1 -- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b700047c0 msgr2=0x7f5b84139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:50.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b88a8d640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b700047c0 0x7f5b84139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b88a8d640 1 -- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b841e4750 msgr2=0x7f5b841e4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:50.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b88a8d640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b841e4750 0x7f5b841e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:50.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b88a8d640 1 -- 172.21.15.125:0/2069230511 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8406cac0 con 0x7f5b8413bd00 2023-10-18T11:41:50.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b88a8d640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b8413bd00 0x7f5b8413c180 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f5b7402cb20 tx=0x7f5b74033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:50.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b79ffb640 1 -- 172.21.15.125:0/2069230511 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74004220 con 0x7f5b8413bd00 2023-10-18T11:41:50.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b79ffb640 1 -- 172.21.15.125:0/2069230511 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5b74004380 con 0x7f5b8413bd00 2023-10-18T11:41:50.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b79ffb640 1 -- 172.21.15.125:0/2069230511 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7403d940 con 0x7f5b8413bd00 2023-10-18T11:41:50.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b841e54f0 con 0x7f5b8413bd00 2023-10-18T11:41:50.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.481+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b841e5ab0 con 0x7f5b8413bd00 2023-10-18T11:41:50.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.482+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b841e5cd0 con 0x7f5b8413bd00 2023-10-18T11:41:50.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.483+0000 7f5b79ffb640 1 -- 172.21.15.125:0/2069230511 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 93) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5b7403daa0 con 0x7f5b8413bd00 2023-10-18T11:41:50.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.483+0000 7f5b79ffb640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5b6806af90 0x7f5b6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:50.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.483+0000 7f5b79ffb640 1 -- 172.21.15.125:0/2069230511 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f5b74033e40 con 0x7f5b8413bd00 2023-10-18T11:41:50.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.483+0000 7f5b8928e640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5b6806af90 0x7f5b6806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:50.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.484+0000 7f5b8928e640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5b6806af90 0x7f5b6806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b80006fd0 tx=0x7f5b80006e00 comp rx=0 tx=0).ready entity=mgr.8644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:50.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.487+0000 7f5b79ffb640 1 -- 172.21.15.125:0/2069230511 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5b74077350 con 0x7f5b8413bd00 2023-10-18T11:41:50.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:50.699+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f5b8413c6c0 con 0x7f5b8413bd00 2023-10-18T11:41:51.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.633+0000 7f5b79ffb640 1 -- 172.21.15.125:0/2069230511 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v94) v1 ==== 92+0+0 (secure 0 0 0) 0x7f5b8413c6c0 con 0x7f5b8413bd00 2023-10-18T11:41:51.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5b6806af90 msgr2=0x7f5b6806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:51.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5b6806af90 0x7f5b6806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b80006fd0 tx=0x7f5b80006e00 comp rx=0 tx=0).stop 2023-10-18T11:41:51.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b8413bd00 msgr2=0x7f5b8413c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:51.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b8413bd00 0x7f5b8413c180 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f5b7402cb20 tx=0x7f5b74033c50 comp rx=0 tx=0).stop 2023-10-18T11:41:51.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 shutdown_connections 2023-10-18T11:41:51.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f5b6806af90 0x7f5b6806d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b841e4750 0x7f5b841e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5b8413bd00 0x7f5b8413c180 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 --2- 172.21.15.125:0/2069230511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5b700047c0 0x7f5b84139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 >> 172.21.15.125:0/2069230511 conn(0x7f5b84069930 msgr2=0x7f5b84159580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:51.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 shutdown_connections 2023-10-18T11:41:51.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.637+0000 7f5b8b4f2640 1 -- 172.21.15.125:0/2069230511 wait complete. 2023-10-18T11:41:51.657 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T11:41:51.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.768+0000 7f068edd7640 1 -- 172.21.15.125:0/2616122081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068815b2a0 msgr2=0x7f06880cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:51.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.768+0000 7f068edd7640 1 --2- 172.21.15.125:0/2616122081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068815b2a0 0x7f06880cf720 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0670009830 tx=0x7f067002bcd0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.769+0000 7f068edd7640 1 -- 172.21.15.125:0/2616122081 shutdown_connections 2023-10-18T11:41:51.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.769+0000 7f068edd7640 1 --2- 172.21.15.125:0/2616122081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068815b2a0 0x7f06880cf720 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.769+0000 7f068edd7640 1 --2- 172.21.15.125:0/2616122081 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06880cf340 0x7f06880d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.769+0000 7f068edd7640 1 -- 172.21.15.125:0/2616122081 >> 172.21.15.125:0/2616122081 conn(0x7f06880d0d60 msgr2=0x7f06880d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:51.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.769+0000 7f068edd7640 1 -- 172.21.15.125:0/2616122081 shutdown_connections 2023-10-18T11:41:51.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.769+0000 7f068edd7640 1 -- 172.21.15.125:0/2616122081 wait complete. 2023-10-18T11:41:51.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.770+0000 7f068edd7640 1 Processor -- start 2023-10-18T11:41:51.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.770+0000 7f068edd7640 1 -- start start 2023-10-18T11:41:51.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068edd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06880ceb40 0x7f06881f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:51.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068edd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06880cf340 0x7f06881f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:51.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068edd7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068815b2a0 0x7f06880ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:51.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068edd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068814da90 con 0x7f06880cf340 2023-10-18T11:41:51.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068edd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06881f5b00 con 0x7f06880ceb40 2023-10-18T11:41:51.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068edd7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068814ce10 con 0x7f068815b2a0 2023-10-18T11:41:51.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f0687fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06880cf340 0x7f06881f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:51.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06880ceb40 0x7f06881f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:51.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06880ceb40 0x7f06881f55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:56026/0 (socket says 172.21.15.125:56026) 2023-10-18T11:41:51.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 -- 172.21.15.125:0/1248688692 learned_addr learned my addr 172.21.15.125:0/1248688692 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:51.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068d374640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068815b2a0 0x7f06880ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:51.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 -- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068815b2a0 msgr2=0x7f06880ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:51.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068815b2a0 0x7f06880ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 -- 172.21.15.125:0/1248688692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06880cf340 msgr2=0x7f06881f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:51.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06880cf340 0x7f06881f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 -- 172.21.15.125:0/1248688692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0670009490 con 0x7f06880ceb40 2023-10-18T11:41:51.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f0687fff640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06880cf340 0x7f06881f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:41:51.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068cb73640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06880ceb40 0x7f06881f55c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f068000ebc0 tx=0x7f068000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:51.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f06857fa640 1 -- 172.21.15.125:0/1248688692 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06800099b0 con 0x7f06880ceb40 2023-10-18T11:41:51.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f06857fa640 1 -- 172.21.15.125:0/1248688692 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0680010720 con 0x7f06880ceb40 2023-10-18T11:41:51.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f06857fa640 1 -- 172.21.15.125:0/1248688692 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680010b30 con 0x7f06880ceb40 2023-10-18T11:41:51.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.771+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068814d0f0 con 0x7f06880ceb40 2023-10-18T11:41:51.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.772+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068814d6b0 con 0x7f06880ceb40 2023-10-18T11:41:51.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.772+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06880b3ba0 con 0x7f06880ceb40 2023-10-18T11:41:51.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.773+0000 7f06857fa640 1 -- 172.21.15.125:0/1248688692 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 94) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f0680010c90 con 0x7f06880ceb40 2023-10-18T11:41:51.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.773+0000 7f06857fa640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f066806afb0 0x7f066806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:51.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.773+0000 7f0687fff640 1 -- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f066806afb0 msgr2=0x7f066806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4236255547 2023-10-18T11:41:51.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.773+0000 7f06857fa640 1 -- 172.21.15.125:0/1248688692 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f0680014070 con 0x7f06880ceb40 2023-10-18T11:41:51.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.773+0000 7f0687fff640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f066806afb0 0x7f066806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:51.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.778+0000 7f06857fa640 1 -- 172.21.15.125:0/1248688692 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0680051ca0 con 0x7f06880ceb40 2023-10-18T11:41:51.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.952+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f068814d840 con 0x7f06880ceb40 2023-10-18T11:41:51.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.958+0000 7f06857fa640 1 -- 172.21.15.125:0/1248688692 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v64)=0 v64) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0680055690 con 0x7f06880ceb40 2023-10-18T11:41:51.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f066806afb0 msgr2=0x7f066806d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:51.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f066806afb0 0x7f066806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06880ceb40 msgr2=0x7f06881f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:51.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06880ceb40 0x7f06881f55c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f068000ebc0 tx=0x7f068000c6a0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 shutdown_connections 2023-10-18T11:41:51.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f066806afb0 0x7f066806d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f068815b2a0 0x7f06880ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06880cf340 0x7f06881f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 --2- 172.21.15.125:0/1248688692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06880ceb40 0x7f06881f55c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:51.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 >> 172.21.15.125:0/1248688692 conn(0x7f06880d0d60 msgr2=0x7f06880adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:51.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 shutdown_connections 2023-10-18T11:41:51.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:51.960+0000 7f068edd7640 1 -- 172.21.15.125:0/1248688692 wait complete. 2023-10-18T11:41:51.973 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T11:41:52.022 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.020+0000 7f7e4584c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:41:52.040 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.037+0000 7f8c8b632040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:41:52.066 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.064+0000 7f7e4584c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:41:52.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.078+0000 7f9c950c2640 1 -- 172.21.15.125:0/4197442572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c900cf340 msgr2=0x7f9c900d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:52.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.078+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4197442572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c900cf340 0x7f9c900d1910 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f9c9015b680 tx=0x7f9c8002bdb0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.078+0000 7f9c950c2640 1 -- 172.21.15.125:0/4197442572 shutdown_connections 2023-10-18T11:41:52.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.078+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4197442572 >> v2:172.21.15.139:3300/0 conn(0x7f9c900ceb40 0x7f9c900b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.078+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4197442572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c900cf340 0x7f9c900d1910 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.078+0000 7f9c950c2640 1 -- 172.21.15.125:0/4197442572 >> 172.21.15.125:0/4197442572 conn(0x7f9c900d0d60 msgr2=0x7f9c900d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:52.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.078+0000 7f9c950c2640 1 -- 172.21.15.125:0/4197442572 shutdown_connections 2023-10-18T11:41:52.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.078+0000 7f9c950c2640 1 -- 172.21.15.125:0/4197442572 wait complete. 2023-10-18T11:41:52.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.080+0000 7f9c950c2640 1 Processor -- start 2023-10-18T11:41:52.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.081+0000 7f9c950c2640 1 -- start start 2023-10-18T11:41:52.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c950c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c900ceb40 0x7f9c90157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:52.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c950c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c900cf340 0x7f9c900b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:52.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c950c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9015b2a0 0x7f9c900cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:52.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c950c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9015bd20 con 0x7f9c9015b2a0 2023-10-18T11:41:52.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c950c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90158360 con 0x7f9c900cf340 2023-10-18T11:41:52.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c950c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c901f1470 con 0x7f9c900ceb40 2023-10-18T11:41:52.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9015b2a0 0x7f9c900cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:52.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9015b2a0 0x7f9c900cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49398/0 (socket says 172.21.15.125:49398) 2023-10-18T11:41:52.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 -- 172.21.15.125:0/4126440949 learned_addr learned my addr 172.21.15.125:0/4126440949 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:52.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c87fff640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c900cf340 0x7f9c900b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:52.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8ed9d640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c900ceb40 0x7f9c90157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:52.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 -- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c900ceb40 msgr2=0x7f9c90157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:52.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c900ceb40 0x7f9c90157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 -- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c900cf340 msgr2=0x7f9c900b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:52.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c900cf340 0x7f9c900b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 -- 172.21.15.125:0/4126440949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c80009560 con 0x7f9c9015b2a0 2023-10-18T11:41:52.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c8f59e640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9015b2a0 0x7f9c900cc790 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800eb10 tx=0x7f9c8800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:52.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.082+0000 7f9c877fe640 1 -- 172.21.15.125:0/4126440949 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88004b20 con 0x7f9c9015b2a0 2023-10-18T11:41:52.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.083+0000 7f9c877fe640 1 -- 172.21.15.125:0/4126440949 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9c88004ca0 con 0x7f9c9015b2a0 2023-10-18T11:41:52.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.083+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c901587e0 con 0x7f9c9015b2a0 2023-10-18T11:41:52.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.083+0000 7f9c877fe640 1 -- 172.21.15.125:0/4126440949 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88005280 con 0x7f9c9015b2a0 2023-10-18T11:41:52.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.083+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c900b5b60 con 0x7f9c9015b2a0 2023-10-18T11:41:52.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.084+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9015be90 con 0x7f9c9015b2a0 2023-10-18T11:41:52.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.084+0000 7f9c877fe640 1 -- 172.21.15.125:0/4126440949 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 94) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9c880053e0 con 0x7f9c9015b2a0 2023-10-18T11:41:52.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.085+0000 7f9c877fe640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f9c7006afb0 0x7f9c7006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:52.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.085+0000 7f9c8ed9d640 1 -- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f9c7006afb0 msgr2=0x7f9c7006d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4236255547 2023-10-18T11:41:52.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.085+0000 7f9c877fe640 1 -- 172.21.15.125:0/4126440949 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f9c88014070 con 0x7f9c9015b2a0 2023-10-18T11:41:52.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.085+0000 7f9c8ed9d640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f9c7006afb0 0x7f9c7006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:52.093 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.087+0000 7f8c8b632040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:41:52.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.089+0000 7f9c877fe640 1 -- 172.21.15.125:0/4126440949 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c88055490 con 0x7f9c9015b2a0 2023-10-18T11:41:52.105 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:41:52.107 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:41:52.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.264+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f9c900b0d00 con 0x7f9c9015b2a0 2023-10-18T11:41:52.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.268+0000 7f9c877fe640 1 -- 172.21.15.125:0/4126440949 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v65)=0 v65) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9c88058ab0 con 0x7f9c9015b2a0 2023-10-18T11:41:52.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f9c7006afb0 msgr2=0x7f9c7006d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:52.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f9c7006afb0 0x7f9c7006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9015b2a0 msgr2=0x7f9c900cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:52.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9015b2a0 0x7f9c900cc790 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800eb10 tx=0x7f9c8800c2d0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 shutdown_connections 2023-10-18T11:41:52.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7f9c7006afb0 0x7f9c7006d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c900cf340 0x7f9c900b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9015b2a0 0x7f9c900cc790 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 --2- 172.21.15.125:0/4126440949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9c900ceb40 0x7f9c90157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 >> 172.21.15.125:0/4126440949 conn(0x7f9c900d0d60 msgr2=0x7f9c9015a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:52.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 shutdown_connections 2023-10-18T11:41:52.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.270+0000 7f9c950c2640 1 -- 172.21.15.125:0/4126440949 wait complete. 2023-10-18T11:41:52.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:52.283 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T11:41:52.303 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2023-10-18T11:41:52.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.449+0000 7fc216ea9640 1 -- 172.21.15.125:0/715268867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 msgr2=0x7fc2100cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:52.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.449+0000 7fc216ea9640 1 --2- 172.21.15.125:0/715268867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 0x7fc2100cf350 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fc204009830 tx=0x7fc20402bdf0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.449+0000 7fc216ea9640 1 -- 172.21.15.125:0/715268867 shutdown_connections 2023-10-18T11:41:52.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.449+0000 7fc216ea9640 1 --2- 172.21.15.125:0/715268867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2100cddc0 0x7fc2100ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.449+0000 7fc216ea9640 1 --2- 172.21.15.125:0/715268867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 0x7fc2100cf350 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.449+0000 7fc216ea9640 1 -- 172.21.15.125:0/715268867 >> 172.21.15.125:0/715268867 conn(0x7fc210099ff0 msgr2=0x7fc2100d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:52.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.449+0000 7fc216ea9640 1 -- 172.21.15.125:0/715268867 shutdown_connections 2023-10-18T11:41:52.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.449+0000 7fc216ea9640 1 -- 172.21.15.125:0/715268867 wait complete. 2023-10-18T11:41:52.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.450+0000 7fc216ea9640 1 Processor -- start 2023-10-18T11:41:52.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.450+0000 7fc216ea9640 1 -- start start 2023-10-18T11:41:52.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc216ea9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2100cddc0 0x7fc210157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:52.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc216ea9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2100ce420 0x7fc2100c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:52.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc216ea9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 0x7fc2100b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:52.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc216ea9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100b5e30 con 0x7fc21015b180 2023-10-18T11:41:52.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc216ea9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc210158350 con 0x7fc2100cddc0 2023-10-18T11:41:52.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc216ea9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100b5500 con 0x7fc2100ce420 2023-10-18T11:41:52.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc215446640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 0x7fc2100b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:52.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc215446640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 0x7fc2100b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49408/0 (socket says 172.21.15.125:49408) 2023-10-18T11:41:52.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc215446640 1 -- 172.21.15.125:0/2516806684 learned_addr learned my addr 172.21.15.125:0/2516806684 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:52.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc214c45640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2100cddc0 0x7fc210157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:52.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc20ffff640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2100ce420 0x7fc2100c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:52.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc215446640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2100ce420 msgr2=0x7fc2100c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:52.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc215446640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2100ce420 0x7fc2100c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc215446640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2100cddc0 msgr2=0x7fc210157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:52.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc215446640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2100cddc0 0x7fc210157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:52.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.451+0000 7fc215446640 1 -- 172.21.15.125:0/2516806684 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc204009490 con 0x7fc21015b180 2023-10-18T11:41:52.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.452+0000 7fc215446640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 0x7fc2100b62c0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fc20800ead0 tx=0x7fc20800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:52.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.452+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800cde0 con 0x7fc21015b180 2023-10-18T11:41:52.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.452+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2100b57e0 con 0x7fc21015b180 2023-10-18T11:41:52.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.452+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc208004c10 con 0x7fc21015b180 2023-10-18T11:41:52.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.452+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208005020 con 0x7fc21015b180 2023-10-18T11:41:52.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.452+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc210159800 con 0x7fc21015b180 2023-10-18T11:41:52.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.453+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc210066960 con 0x7fc21015b180 2023-10-18T11:41:52.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.453+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 94) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc208005180 con 0x7fc21015b180 2023-10-18T11:41:52.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.453+0000 7fc20d7fa640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 0x7fc1f406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:52.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.454+0000 7fc214c45640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 msgr2=0x7fc1f406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4236255547 2023-10-18T11:41:52.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.454+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fc20808a380 con 0x7fc21015b180 2023-10-18T11:41:52.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.454+0000 7fc214c45640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 0x7fc1f406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:52.463 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.460+0000 7f7e4584c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:41:52.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.458+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc208055170 con 0x7fc21015b180 2023-10-18T11:41:52.500 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.498+0000 7f8c8b632040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:41:52.506 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.504+0000 7f7e4584c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:41:52.544 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.542+0000 7f8c8b632040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:41:52.549 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.547+0000 7f7e4584c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:41:52.587 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.585+0000 7f8c8b632040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:41:52.600 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.598+0000 7f7e4584c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:41:52.641 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.639+0000 7f8c8b632040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:41:52.643 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.640+0000 7f7e4584c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:41:52.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.649+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 --> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2100b5670 con 0x7fc1f406afe0 2023-10-18T11:41:52.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.655+0000 7fc214c45640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 msgr2=0x7fc1f406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4236255547 2023-10-18T11:41:52.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:52.655+0000 7fc214c45640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 0x7fc1f406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:41:52.684 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.681+0000 7f8c8b632040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:41:52.741 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.738+0000 7f7e4584c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:41:52.781 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.779+0000 7f8c8b632040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:41:52.794 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.791+0000 7f7e4584c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:52.833 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.831+0000 7f8c8b632040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:52.836 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.833+0000 7f7e4584c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:41:52.874 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.872+0000 7f8c8b632040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:41:52.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.881+0000 7f7e4584c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:41:52.922 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.920+0000 7f8c8b632040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:41:52.926 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:52.924+0000 7f7e4584c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:41:52.965 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:52.963+0000 7f8c8b632040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:41:53.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:53.056+0000 7fc214c45640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 msgr2=0x7fc1f406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4236255547 2023-10-18T11:41:53.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:53.056+0000 7fc214c45640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 0x7fc1f406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T11:41:53.241 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.239+0000 7f7e4584c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:41:53.278 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:53.276+0000 7f8c8b632040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:41:53.371 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.369+0000 7f7e4584c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:41:53.408 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:53.405+0000 7f8c8b632040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:41:53.462 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.459+0000 7f7e4584c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:41:53.499 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:53.497+0000 7f8c8b632040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:41:53.504 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.502+0000 7f7e4584c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:41:53.542 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:53.540+0000 7f8c8b632040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:41:53.598 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.596+0000 7f7e4584c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:41:53.636 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:53.634+0000 7f8c8b632040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:41:53.689 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.686+0000 7f7e4584c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:41:53.725 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:53.723+0000 7f8c8b632040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:41:53.826 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.824+0000 7f7e4584c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:41:53.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:53.857+0000 7fc214c45640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 msgr2=0x7fc1f406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/4236255547 2023-10-18T11:41:53.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:53.857+0000 7fc214c45640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 0x7fc1f406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T11:41:53.863 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:53.861+0000 7f8c8b632040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:41:53.868 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.866+0000 7f7e4584c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:53.905 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:53.903+0000 7f8c8b632040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:53.973 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:53.971+0000 7f7e4584c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:41:54.011 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:54.009+0000 7f8c8b632040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:41:54.589 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:54.587+0000 7f7e4584c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:41:54.622 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:54.619+0000 7f8c8b632040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:41:54.852 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:54.850+0000 7f7e4584c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:41:54.884 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:54.882+0000 7f8c8b632040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:41:54.902 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:54.900+0000 7f7e4584c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:41:54.933 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:54.931+0000 7f8c8b632040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:41:55.002 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.000+0000 7f7e4584c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:41:55.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:55.024+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 95) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fc208088740 con 0x7fc21015b180 2023-10-18T11:41:55.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:55.024+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 msgr2=0x7fc1f406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:55.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:55.024+0000 7fc20d7fa640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/4236255547,v1:172.21.15.139:6833/4236255547] conn(0x7fc1f406afe0 0x7fc1f406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:55.037 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:55.034+0000 7f8c8b632040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:41:55.131 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.128+0000 7f7dc2339640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.132 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.128+0000 7f7dc2339640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.132 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.128+0000 7f7dc2339640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.132 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.128+0000 7f7dc2339640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.132 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.128+0000 7f7dc2339640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.133 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.129+0000 7f7dc537f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.133 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.129+0000 7f7dc537f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.133 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.129+0000 7f7dc537f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.133 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.129+0000 7f7dc537f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:55.134 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:55.129+0000 7f7dc537f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:41:56.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.026+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 96) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc208055540 con 0x7fc21015b180 2023-10-18T11:41:56.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.027+0000 7fc20d7fa640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fc1f4035880 0x7fc1f4075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.027+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 --> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2100b5670 con 0x7fc1f4035880 2023-10-18T11:41:56.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.028+0000 7fc214c45640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fc1f4035880 0x7fc1f4075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.029+0000 7fc214c45640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fc1f4035880 0x7fc1f4075410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc204036000 tx=0x7fc204004730 comp rx=0 tx=0).ready entity=mgr.7925 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:56.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.080+0000 7fc20d7fa640 1 -- 172.21.15.125:0/2516806684 <== mgr.7925 v2:172.21.15.139:6832/1922997989 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc2100b5670 con 0x7fc1f4035880 2023-10-18T11:41:56.082 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1 2023-10-18T11:41:56.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fc1f4035880 msgr2=0x7fc1f4075410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc216ea9640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fc1f4035880 0x7fc1f4075410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc204036000 tx=0x7fc204004730 comp rx=0 tx=0).stop 2023-10-18T11:41:56.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 msgr2=0x7fc2100b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc216ea9640 1 --2- 172.21.15.125:0/2516806684 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc21015b180 0x7fc2100b62c0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fc20800ead0 tx=0x7fc20800ef90 comp rx=0 tx=0).stop 2023-10-18T11:41:56.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc215446640 1 -- 172.21.15.125:0/2516806684 reap_dead start 2023-10-18T11:41:56.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 shutdown_connections 2023-10-18T11:41:56.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 >> 172.21.15.125:0/2516806684 conn(0x7fc210099ff0 msgr2=0x7fc21015e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:56.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 shutdown_connections 2023-10-18T11:41:56.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.082+0000 7fc216ea9640 1 -- 172.21.15.125:0/2516806684 wait complete. 2023-10-18T11:41:56.097 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-18T11:41:56.097 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-18T11:41:56.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.202+0000 7f30d9ca9640 1 -- 172.21.15.125:0/3255813282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d415b2a0 msgr2=0x7f30d40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.202+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/3255813282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d415b2a0 0x7f30d40cf720 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f30d408a120 tx=0x7f30bc02bcf0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.202+0000 7f30d9ca9640 1 -- 172.21.15.125:0/3255813282 shutdown_connections 2023-10-18T11:41:56.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.202+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/3255813282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d415b2a0 0x7f30d40cf720 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.202+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/3255813282 >> v2:172.21.15.139:3300/0 conn(0x7f30d40cf340 0x7f30d40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.202+0000 7f30d9ca9640 1 -- 172.21.15.125:0/3255813282 >> 172.21.15.125:0/3255813282 conn(0x7f30d40d0d60 msgr2=0x7f30d40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:56.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.202+0000 7f30d9ca9640 1 -- 172.21.15.125:0/3255813282 shutdown_connections 2023-10-18T11:41:56.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.202+0000 7f30d9ca9640 1 -- 172.21.15.125:0/3255813282 wait complete. 2023-10-18T11:41:56.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.203+0000 7f30d9ca9640 1 Processor -- start 2023-10-18T11:41:56.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.203+0000 7f30d9ca9640 1 -- start start 2023-10-18T11:41:56.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30d9ca9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40ceb40 0x7f30d41f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30d9ca9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d40cf340 0x7f30d41f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30d9ca9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d415b2a0 0x7f30d40b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30d9ca9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d415c110 con 0x7f30d40cf340 2023-10-18T11:41:56.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30d9ca9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d415dfa0 con 0x7f30d40ceb40 2023-10-18T11:41:56.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30d9ca9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d41f2790 con 0x7f30d415b2a0 2023-10-18T11:41:56.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30caffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d40cf340 0x7f30d41f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30caffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d40cf340 0x7f30d41f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49414/0 (socket says 172.21.15.125:49414) 2023-10-18T11:41:56.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30caffd640 1 -- 172.21.15.125:0/803167616 learned_addr learned my addr 172.21.15.125:0/803167616 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:56.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30d37fe640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40ceb40 0x7f30d41f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30d3fff640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d415b2a0 0x7f30d40b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30caffd640 1 -- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d415b2a0 msgr2=0x7f30d40b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30caffd640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d415b2a0 0x7f30d40b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30caffd640 1 -- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40ceb40 msgr2=0x7f30d41f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30caffd640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40ceb40 0x7f30d41f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.204+0000 7f30caffd640 1 -- 172.21.15.125:0/803167616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc009560 con 0x7f30d40cf340 2023-10-18T11:41:56.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.205+0000 7f30caffd640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d40cf340 0x7f30d41f2990 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f30bc036040 tx=0x7f30bc02da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:56.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.205+0000 7f30d0ff9640 1 -- 172.21.15.125:0/803167616 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc003800 con 0x7f30d40cf340 2023-10-18T11:41:56.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.205+0000 7f30d0ff9640 1 -- 172.21.15.125:0/803167616 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f30bc003a50 con 0x7f30d40cf340 2023-10-18T11:41:56.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.205+0000 7f30d0ff9640 1 -- 172.21.15.125:0/803167616 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc003e20 con 0x7f30d40cf340 2023-10-18T11:41:56.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.205+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d415b7e0 con 0x7f30d40cf340 2023-10-18T11:41:56.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.205+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d415bda0 con 0x7f30d40cf340 2023-10-18T11:41:56.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.206+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d415c2c0 con 0x7f30d40cf340 2023-10-18T11:41:56.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.206+0000 7f30d0ff9640 1 -- 172.21.15.125:0/803167616 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 96) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f30bc03e450 con 0x7f30d40cf340 2023-10-18T11:41:56.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.206+0000 7f30d0ff9640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f30b406ace0 0x7f30b406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.207+0000 7f30d0ff9640 1 -- 172.21.15.125:0/803167616 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f30bc0abf00 con 0x7f30d40cf340 2023-10-18T11:41:56.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.207+0000 7f30d37fe640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f30b406ace0 0x7f30b406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.207+0000 7f30d37fe640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f30b406ace0 0x7f30b406d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005fd0 tx=0x7f30c400b390 comp rx=0 tx=0).ready entity=mgr.7925 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:56.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.211+0000 7f30d0ff9640 1 -- 172.21.15.125:0/803167616 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30d415c2c0 con 0x7f30d40cf340 2023-10-18T11:41:56.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.396+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 --> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f30d40cb9d0 con 0x7f30b406ace0 2023-10-18T11:41:56.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.451+0000 7f30d0ff9640 1 -- 172.21.15.125:0/803167616 <== mgr.7925 v2:172.21.15.139:6832/1922997989 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f30d40cb9d0 con 0x7f30b406ace0 2023-10-18T11:41:56.453 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path snap_test_dir1 2023-10-18T11:41:56.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f30b406ace0 msgr2=0x7f30b406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f30b406ace0 0x7f30b406d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005fd0 tx=0x7f30c400b390 comp rx=0 tx=0).stop 2023-10-18T11:41:56.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d40cf340 msgr2=0x7f30d41f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d40cf340 0x7f30d41f2990 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f30bc036040 tx=0x7f30bc02da20 comp rx=0 tx=0).stop 2023-10-18T11:41:56.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 shutdown_connections 2023-10-18T11:41:56.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f30b406ace0 0x7f30b406d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30d415b2a0 0x7f30d40b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30d40cf340 0x7f30d41f2990 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 --2- 172.21.15.125:0/803167616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40ceb40 0x7f30d41f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 >> 172.21.15.125:0/803167616 conn(0x7f30d40d0d60 msgr2=0x7f30d406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:56.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 shutdown_connections 2023-10-18T11:41:56.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.453+0000 7f30d9ca9640 1 -- 172.21.15.125:0/803167616 wait complete. 2023-10-18T11:41:56.467 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-18T11:41:56.467 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-18T11:41:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.570+0000 7fd72ccf2640 1 -- 172.21.15.125:0/2762697430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 msgr2=0x7fd72806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.570+0000 7fd72ccf2640 1 --2- 172.21.15.125:0/2762697430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 0x7fd72806b2e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd718009860 tx=0x7fd71802bd60 comp rx=0 tx=0).stop 2023-10-18T11:41:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.570+0000 7fd72ccf2640 1 -- 172.21.15.125:0/2762697430 shutdown_connections 2023-10-18T11:41:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.570+0000 7fd72ccf2640 1 --2- 172.21.15.125:0/2762697430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7280c96b0 0x7fd7280c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.570+0000 7fd72ccf2640 1 --2- 172.21.15.125:0/2762697430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 0x7fd72806b2e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.570+0000 7fd72ccf2640 1 -- 172.21.15.125:0/2762697430 >> 172.21.15.125:0/2762697430 conn(0x7fd728069930 msgr2=0x7fd728069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.570+0000 7fd72ccf2640 1 -- 172.21.15.125:0/2762697430 shutdown_connections 2023-10-18T11:41:56.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.570+0000 7fd72ccf2640 1 -- 172.21.15.125:0/2762697430 wait complete. 2023-10-18T11:41:56.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.571+0000 7fd72ccf2640 1 Processor -- start 2023-10-18T11:41:56.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.571+0000 7fd72ccf2640 1 -- start start 2023-10-18T11:41:56.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72ccf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 0x7fd72814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72ccf2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7280ab500 0x7fd7280a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72ccf2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7280c96b0 0x7fd7280afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72ccf2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72814f220 con 0x7fd7280c96b0 2023-10-18T11:41:56.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72ccf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280a7370 con 0x7fd72806af00 2023-10-18T11:41:56.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72ccf2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280a7650 con 0x7fd7280ab500 2023-10-18T11:41:56.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 0x7fd72814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd726d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7280c96b0 0x7fd7280afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 0x7fd72814ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:56070/0 (socket says 172.21.15.125:56070) 2023-10-18T11:41:56.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72659c640 1 -- 172.21.15.125:0/357528188 learned_addr learned my addr 172.21.15.125:0/357528188 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:56.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd71ffff640 1 --2- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7280ab500 0x7fd7280a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd726d9d640 1 -- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7280ab500 msgr2=0x7fd7280a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd726d9d640 1 --2- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7280ab500 0x7fd7280a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd726d9d640 1 -- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 msgr2=0x7fd72814ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd726d9d640 1 --2- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 0x7fd72814ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd726d9d640 1 -- 172.21.15.125:0/357528188 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710009490 con 0x7fd7280c96b0 2023-10-18T11:41:56.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd72659c640 1 --2- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd72806af00 0x7fd72814ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:41:56.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd726d9d640 1 --2- 172.21.15.125:0/357528188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7280c96b0 0x7fd7280afff0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fd72000ca20 tx=0x7fd72000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:56.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd71effd640 1 -- 172.21.15.125:0/357528188 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72000abd0 con 0x7fd7280c96b0 2023-10-18T11:41:56.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.572+0000 7fd71effd640 1 -- 172.21.15.125:0/357528188 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd72000ae20 con 0x7fd7280c96b0 2023-10-18T11:41:56.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.573+0000 7fd72ccf2640 1 -- 172.21.15.125:0/357528188 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd718009490 con 0x7fd7280c96b0 2023-10-18T11:41:56.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.573+0000 7fd71effd640 1 -- 172.21.15.125:0/357528188 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720003bd0 con 0x7fd7280c96b0 2023-10-18T11:41:56.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.573+0000 7fd72ccf2640 1 -- 172.21.15.125:0/357528188 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7280afad0 con 0x7fd7280c96b0 2023-10-18T11:41:56.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.574+0000 7fd72ccf2640 1 -- 172.21.15.125:0/357528188 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7280afcf0 con 0x7fd7280c96b0 2023-10-18T11:41:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.574+0000 7fd71effd640 1 -- 172.21.15.125:0/357528188 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 96) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd720004030 con 0x7fd7280c96b0 2023-10-18T11:41:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.574+0000 7fd71effd640 1 --2- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd70806ace0 0x7fd70806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.574+0000 7fd71effd640 1 -- 172.21.15.125:0/357528188 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fd720089e40 con 0x7fd7280c96b0 2023-10-18T11:41:56.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.575+0000 7fd72659c640 1 --2- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd70806ace0 0x7fd70806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.575+0000 7fd72659c640 1 --2- 172.21.15.125:0/357528188 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd70806ace0 0x7fd70806d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd718009860 tx=0x7fd718004730 comp rx=0 tx=0).ready entity=mgr.7925 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:56.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.579+0000 7fd71effd640 1 -- 172.21.15.125:0/357528188 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd720054fa0 con 0x7fd7280c96b0 2023-10-18T11:41:56.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.763+0000 7fd72ccf2640 1 -- 172.21.15.125:0/357528188 --> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd72815a110 con 0x7fd70806ace0 2023-10-18T11:41:56.772 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:56.770+0000 7f7dcc38d640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-18T11:41:56.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.770+0000 7fd71effd640 1 -- 172.21.15.125:0/357528188 <== mgr.7925 v2:172.21.15.139:6832/1922997989 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fd71802c9b0 con 0x7fd70806ace0 2023-10-18T11:41:56.773 INFO:teuthology.orchestra.run.smithi125.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-18T11:41:56.777 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-18T11:41:56.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:56.779 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-18T11:41:56.797 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T11:41:56.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.938+0000 7f16a9e4b640 1 -- 172.21.15.125:0/300206970 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 msgr2=0x7f16a40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.938+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/300206970 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 0x7f16a40b0650 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f16a408a120 tx=0x7f169c02c260 comp rx=0 tx=0).stop 2023-10-18T11:41:56.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.938+0000 7f16a9e4b640 1 -- 172.21.15.125:0/300206970 shutdown_connections 2023-10-18T11:41:56.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.938+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/300206970 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 0x7f16a40b0650 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.938+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/300206970 >> v2:172.21.15.139:3300/0 conn(0x7f16a415b2a0 0x7f16a40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.938+0000 7f16a9e4b640 1 -- 172.21.15.125:0/300206970 >> 172.21.15.125:0/300206970 conn(0x7f16a40d0d60 msgr2=0x7f16a40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:56.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.938+0000 7f16a9e4b640 1 -- 172.21.15.125:0/300206970 shutdown_connections 2023-10-18T11:41:56.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.938+0000 7f16a9e4b640 1 -- 172.21.15.125:0/300206970 wait complete. 2023-10-18T11:41:56.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.939+0000 7f16a9e4b640 1 Processor -- start 2023-10-18T11:41:56.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a9e4b640 1 -- start start 2023-10-18T11:41:56.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a9e4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 0x7f16a414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a9e4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16a40cf340 0x7f16a41f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a9e4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16a415b2a0 0x7f16a4155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a9e4b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4158080 con 0x7f16a40ceb40 2023-10-18T11:41:56.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a9e4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415ca40 con 0x7f16a415b2a0 2023-10-18T11:41:56.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a9e4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415cd20 con 0x7f16a40cf340 2023-10-18T11:41:56.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 0x7f16a414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 0x7f16a414f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49438/0 (socket says 172.21.15.125:49438) 2023-10-18T11:41:56.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.940+0000 7f16a37fe640 1 -- 172.21.15.125:0/1007546685 learned_addr learned my addr 172.21.15.125:0/1007546685 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:56.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a3fff640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16a415b2a0 0x7f16a4155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a2ffd640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16a40cf340 0x7f16a41f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a37fe640 1 -- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16a40cf340 msgr2=0x7f16a41f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a37fe640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16a40cf340 0x7f16a41f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a37fe640 1 -- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16a415b2a0 msgr2=0x7f16a4155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:56.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a37fe640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16a415b2a0 0x7f16a4155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:56.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a37fe640 1 -- 172.21.15.125:0/1007546685 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c009900 con 0x7f16a40ceb40 2023-10-18T11:41:56.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a2ffd640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16a40cf340 0x7f16a41f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:56.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a3fff640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16a415b2a0 0x7f16a4155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:41:56.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a37fe640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 0x7f16a414f0d0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f169400c400 tx=0x7f169400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:56.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f168bfff640 1 -- 172.21.15.125:0/1007546685 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940052d0 con 0x7f16a40ceb40 2023-10-18T11:41:56.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f168bfff640 1 -- 172.21.15.125:0/1007546685 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1694007020 con 0x7f16a40ceb40 2023-10-18T11:41:56.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f168bfff640 1 -- 172.21.15.125:0/1007546685 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694007470 con 0x7f16a40ceb40 2023-10-18T11:41:56.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a415d000 con 0x7f16a40ceb40 2023-10-18T11:41:56.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.941+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a415d5c0 con 0x7f16a40ceb40 2023-10-18T11:41:56.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.942+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a40afdc0 con 0x7f16a40ceb40 2023-10-18T11:41:56.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.942+0000 7f168bfff640 1 -- 172.21.15.125:0/1007546685 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 96) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f16940067d0 con 0x7f16a40ceb40 2023-10-18T11:41:56.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.943+0000 7f168bfff640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f169006ace0 0x7f169006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:56.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.943+0000 7f168bfff640 1 -- 172.21.15.125:0/1007546685 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f1694088c60 con 0x7f16a40ceb40 2023-10-18T11:41:56.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.943+0000 7f16a2ffd640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f169006ace0 0x7f169006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:56.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.944+0000 7f16a2ffd640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f169006ace0 0x7f169006d1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f16a40b07e0 tx=0x7f1698005f90 comp rx=0 tx=0).ready entity=mgr.7925 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:56.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:56.947+0000 7f168bfff640 1 -- 172.21.15.125:0/1007546685 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1694053d10 con 0x7f16a40ceb40 2023-10-18T11:41:57.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:57.166+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f16a40690e0 con 0x7f16a40ceb40 2023-10-18T11:41:58.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.031+0000 7f168bfff640 1 -- 172.21.15.125:0/1007546685 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v97) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1694057330 con 0x7f16a40ceb40 2023-10-18T11:41:58.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f169006ace0 msgr2=0x7f169006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f169006ace0 0x7f169006d1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f16a40b07e0 tx=0x7f1698005f90 comp rx=0 tx=0).stop 2023-10-18T11:41:58.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 msgr2=0x7f16a414f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 0x7f16a414f0d0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f169400c400 tx=0x7f169400c8c0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 shutdown_connections 2023-10-18T11:41:58.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f169006ace0 0x7f169006d1a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16a415b2a0 0x7f16a4155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16a40cf340 0x7f16a41f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 --2- 172.21.15.125:0/1007546685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16a40ceb40 0x7f16a414f0d0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.035+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 >> 172.21.15.125:0/1007546685 conn(0x7f16a40d0d60 msgr2=0x7f16a40b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:58.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.036+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 shutdown_connections 2023-10-18T11:41:58.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.036+0000 7f16a9e4b640 1 -- 172.21.15.125:0/1007546685 wait complete. 2023-10-18T11:41:58.053 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T11:41:58.053 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:41:58.095 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:41:58.146 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T11:41:58.146 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:41:58.147 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:58.163 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:41:58.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.305+0000 7f6294832640 1 -- 172.21.15.125:0/1316057247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6278003970 msgr2=0x7f6278003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.305+0000 7f6294832640 1 --2- 172.21.15.125:0/1316057247 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6278003970 0x7f6278003d90 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f6280009b50 tx=0x7f62780057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.305+0000 7f628ed9d640 1 -- 172.21.15.125:0/1316057247 reap_dead start 2023-10-18T11:41:58.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.305+0000 7f6294832640 1 -- 172.21.15.125:0/1316057247 shutdown_connections 2023-10-18T11:41:58.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.305+0000 7f6294832640 1 -- 172.21.15.125:0/1316057247 >> 172.21.15.125:0/1316057247 conn(0x7f62900d0d60 msgr2=0x7f62900d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:58.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.305+0000 7f6294832640 1 -- 172.21.15.125:0/1316057247 shutdown_connections 2023-10-18T11:41:58.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.305+0000 7f6294832640 1 -- 172.21.15.125:0/1316057247 wait complete. 2023-10-18T11:41:58.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.306+0000 7f6294832640 1 Processor -- start 2023-10-18T11:41:58.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.306+0000 7f6294832640 1 -- start start 2023-10-18T11:41:58.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f6294832640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6278003970 0x7f6290154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f6294832640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f629006afe0 0x7f629006b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f6294832640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f629015ccd0 0x7f629015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f6294832640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629006b9a0 con 0x7f629015ccd0 2023-10-18T11:41:58.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f6294832640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62900c95d0 con 0x7f629006afe0 2023-10-18T11:41:58.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f6294832640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62900c8a50 con 0x7f6278003970 2023-10-18T11:41:58.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f629015ccd0 0x7f629015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f629015ccd0 0x7f629015d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:34998/0 (socket says 172.21.15.125:34998) 2023-10-18T11:41:58.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628ed9d640 1 -- 172.21.15.125:0/2938867495 learned_addr learned my addr 172.21.15.125:0/2938867495 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:58.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628e59c640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6278003970 0x7f6290154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628dd9b640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f629006afe0 0x7f629006b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628ed9d640 1 -- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6278003970 msgr2=0x7f6290154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628ed9d640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6278003970 0x7f6290154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628ed9d640 1 -- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f629006afe0 msgr2=0x7f629006b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628ed9d640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f629006afe0 0x7f629006b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.307+0000 7f628ed9d640 1 -- 172.21.15.125:0/2938867495 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900c8d30 con 0x7f629015ccd0 2023-10-18T11:41:58.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.308+0000 7f628ed9d640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f629015ccd0 0x7f629015d150 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f62900b6e10 tx=0x7f6288010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:58.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.308+0000 7f627effd640 1 -- 172.21.15.125:0/2938867495 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800ee70 con 0x7f629015ccd0 2023-10-18T11:41:58.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.308+0000 7f627effd640 1 -- 172.21.15.125:0/2938867495 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6288003a80 con 0x7f629015ccd0 2023-10-18T11:41:58.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.308+0000 7f627effd640 1 -- 172.21.15.125:0/2938867495 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288003e90 con 0x7f629015ccd0 2023-10-18T11:41:58.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.308+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62900c9010 con 0x7f629015ccd0 2023-10-18T11:41:58.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.308+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62900c86e0 con 0x7f629015ccd0 2023-10-18T11:41:58.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.309+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629015b6b0 con 0x7f629015ccd0 2023-10-18T11:41:58.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.309+0000 7f627effd640 1 -- 172.21.15.125:0/2938867495 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f6288019030 con 0x7f629015ccd0 2023-10-18T11:41:58.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.310+0000 7f627effd640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.310+0000 7f628e59c640 1 -- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f627006af90 msgr2=0x7f627006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:41:58.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.310+0000 7f627effd640 1 -- 172.21.15.125:0/2938867495 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f6288017070 con 0x7f629015ccd0 2023-10-18T11:41:58.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.310+0000 7f628e59c640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:58.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.314+0000 7f627effd640 1 -- 172.21.15.125:0/2938867495 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62880551d0 con 0x7f629015ccd0 2023-10-18T11:41:58.425 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:58.422+0000 7fb56aa3e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:41:58.432 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:58.430+0000 7f2176213040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:41:58.469 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:58.467+0000 7fb56aa3e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:41:58.479 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:58.477+0000 7f2176213040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:41:58.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.489+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62900690e0 con 0x7f629015ccd0 2023-10-18T11:41:58.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.490+0000 7f627effd640 1 -- 172.21.15.125:0/2938867495 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+17323 (secure 0 0 0) 0x7f62880587f0 con 0x7f629015ccd0 2023-10-18T11:41:58.493 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:41:58.493 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":175,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:55.015079+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:41:35.970906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-18T11:41:37.131199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3094960806":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/139213704":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4234510722":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4113808248":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6833/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/312743033":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6832/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/1471732770":"2023-10-19T11:41:55.015024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:41:58.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.492+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f627006af90 msgr2=0x7f627006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:58.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.492+0000 7f6294832640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.492+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f629015ccd0 msgr2=0x7f629015d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.492+0000 7f6294832640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f629015ccd0 0x7f629015d150 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f62900b6e10 tx=0x7f6288010f40 comp rx=0 tx=0).stop 2023-10-18T11:41:58.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.493+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 shutdown_connections 2023-10-18T11:41:58.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.493+0000 7f6294832640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.493+0000 7f6294832640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f629006afe0 0x7f629006b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.493+0000 7f6294832640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f629015ccd0 0x7f629015d150 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.493+0000 7f6294832640 1 --2- 172.21.15.125:0/2938867495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6278003970 0x7f6290154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.493+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 >> 172.21.15.125:0/2938867495 conn(0x7f62900d0d60 msgr2=0x7f6290158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:58.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.493+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 shutdown_connections 2023-10-18T11:41:58.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.493+0000 7f6294832640 1 -- 172.21.15.125:0/2938867495 wait complete. 2023-10-18T11:41:58.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T11:41:35.970906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '173', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T11:41:37.131199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:41:58.507 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:41:58.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.611+0000 7f699e064640 1 -- 172.21.15.125:0/3751217008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69800047c0 msgr2=0x7f6980004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.611+0000 7f699e064640 1 --2- 172.21.15.125:0/3751217008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69800047c0 0x7f6980004c40 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f698c009b50 tx=0x7f69800057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.611+0000 7f699e064640 1 -- 172.21.15.125:0/3751217008 shutdown_connections 2023-10-18T11:41:58.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.611+0000 7f699e064640 1 --2- 172.21.15.125:0/3751217008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69800047c0 0x7f6980004c40 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.611+0000 7f699e064640 1 -- 172.21.15.125:0/3751217008 >> 172.21.15.125:0/3751217008 conn(0x7f69980d0d60 msgr2=0x7f69980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:58.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.611+0000 7f699e064640 1 -- 172.21.15.125:0/3751217008 shutdown_connections 2023-10-18T11:41:58.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.611+0000 7f699e064640 1 -- 172.21.15.125:0/3751217008 wait complete. 2023-10-18T11:41:58.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.612+0000 7f699e064640 1 Processor -- start 2023-10-18T11:41:58.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f699e064640 1 -- start start 2023-10-18T11:41:58.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f699e064640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69800047c0 0x7f69980c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f699e064640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6998157b20 0x7f69980b6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f699e064640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69980b38e0 0x7f69980ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f699e064640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6998153030 con 0x7f6998157b20 2023-10-18T11:41:58.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f699e064640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69980c94a0 con 0x7f69800047c0 2023-10-18T11:41:58.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f699e064640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980b4320 con 0x7f69980b38e0 2023-10-18T11:41:58.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f6996ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6998157b20 0x7f69980b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f6996ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6998157b20 0x7f69980b6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35014/0 (socket says 172.21.15.125:35014) 2023-10-18T11:41:58.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.613+0000 7f6996ffd640 1 -- 172.21.15.125:0/3621758165 learned_addr learned my addr 172.21.15.125:0/3621758165 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:58.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f6997fff640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69980b38e0 0x7f69980ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f69977fe640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69800047c0 0x7f69980c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f6996ffd640 1 -- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69980b38e0 msgr2=0x7f69980ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f6996ffd640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69980b38e0 0x7f69980ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f6996ffd640 1 -- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69800047c0 msgr2=0x7f69980c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f6996ffd640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69800047c0 0x7f69980c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f6996ffd640 1 -- 172.21.15.125:0/3621758165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980b4600 con 0x7f6998157b20 2023-10-18T11:41:58.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f6996ffd640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6998157b20 0x7f69980b6600 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f698c009b50 tx=0x7f698c033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:58.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f697ffff640 1 -- 172.21.15.125:0/3621758165 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c033ea0 con 0x7f6998157b20 2023-10-18T11:41:58.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f697ffff640 1 -- 172.21.15.125:0/3621758165 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f698c004440 con 0x7f6998157b20 2023-10-18T11:41:58.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69980b48e0 con 0x7f6998157b20 2023-10-18T11:41:58.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f697ffff640 1 -- 172.21.15.125:0/3621758165 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c003850 con 0x7f6998157b20 2023-10-18T11:41:58.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.614+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69980b6b40 con 0x7f6998157b20 2023-10-18T11:41:58.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.615+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69980690e0 con 0x7f6998157b20 2023-10-18T11:41:58.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.616+0000 7f697ffff640 1 -- 172.21.15.125:0/3621758165 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f698c0039b0 con 0x7f6998157b20 2023-10-18T11:41:58.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.616+0000 7f697ffff640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f698406af90 0x7f698406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.616+0000 7f697ffff640 1 -- 172.21.15.125:0/3621758165 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f698c0ac6c0 con 0x7f6998157b20 2023-10-18T11:41:58.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.616+0000 7f69977fe640 1 -- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f698406af90 msgr2=0x7f698406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:41:58.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.616+0000 7f69977fe640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f698406af90 0x7f698406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:58.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.620+0000 7f697ffff640 1 -- 172.21.15.125:0/3621758165 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f698c077340 con 0x7f6998157b20 2023-10-18T11:41:58.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.788+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6998153030 con 0x7f6998157b20 2023-10-18T11:41:58.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.788+0000 7f697ffff640 1 -- 172.21.15.125:0/3621758165 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) v1 ==== 93+0+10 (secure 0 0 0) 0x7f698c041020 con 0x7f6998157b20 2023-10-18T11:41:58.791 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:41:58.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f698406af90 msgr2=0x7f698406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:58.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f698406af90 0x7f698406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6998157b20 msgr2=0x7f69980b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6998157b20 0x7f69980b6600 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f698c009b50 tx=0x7f698c033bf0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 shutdown_connections 2023-10-18T11:41:58.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f698406af90 0x7f698406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69980b38e0 0x7f69980ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6998157b20 0x7f69980b6600 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 --2- 172.21.15.125:0/3621758165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69800047c0 0x7f69980c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 >> 172.21.15.125:0/3621758165 conn(0x7f69980d0d60 msgr2=0x7f69980ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:58.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 shutdown_connections 2023-10-18T11:41:58.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.790+0000 7f699e064640 1 -- 172.21.15.125:0/3621758165 wait complete. 2023-10-18T11:41:58.804 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:41:58.861 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:58.859+0000 7fb56aa3e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:41:58.887 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:58.884+0000 7f2176213040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:41:58.905 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:58.903+0000 7fb56aa3e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:41:58.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.910+0000 7f089ed4c640 1 -- 172.21.15.125:0/1305363745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0888004d00 msgr2=0x7f0888005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.910+0000 7f089ed4c640 1 --2- 172.21.15.125:0/1305363745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0888004d00 0x7f0888005180 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f089000c640 tx=0x7f08880057f0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.910+0000 7f089d2e9640 1 -- 172.21.15.125:0/1305363745 reap_dead start 2023-10-18T11:41:58.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.910+0000 7f089ed4c640 1 -- 172.21.15.125:0/1305363745 shutdown_connections 2023-10-18T11:41:58.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.910+0000 7f089ed4c640 1 -- 172.21.15.125:0/1305363745 >> 172.21.15.125:0/1305363745 conn(0x7f08980d0d60 msgr2=0x7f08980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:58.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.910+0000 7f089ed4c640 1 -- 172.21.15.125:0/1305363745 shutdown_connections 2023-10-18T11:41:58.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.910+0000 7f089ed4c640 1 -- 172.21.15.125:0/1305363745 wait complete. 2023-10-18T11:41:58.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.911+0000 7f089ed4c640 1 Processor -- start 2023-10-18T11:41:58.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.911+0000 7f089ed4c640 1 -- start start 2023-10-18T11:41:58.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089ed4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0888004d00 0x7f0898158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089ed4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08980b4740 0x7f08980b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089ed4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08980b4bc0 0x7f08980b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089ed4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089815def0 con 0x7f08980b4bc0 2023-10-18T11:41:58.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089ed4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08980b3980 con 0x7f08980b4740 2023-10-18T11:41:58.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089ed4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08980b3c60 con 0x7f0888004d00 2023-10-18T11:41:58.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08980b4bc0 0x7f08980b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089cae8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0888004d00 0x7f0898158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08980b4bc0 0x7f08980b5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35030/0 (socket says 172.21.15.125:35030) 2023-10-18T11:41:58.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089cae8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0888004d00 0x7f0898158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53860/0 (socket says 172.21.15.125:53860) 2023-10-18T11:41:58.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 -- 172.21.15.125:0/3315641063 learned_addr learned my addr 172.21.15.125:0/3315641063 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:58.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f0897fff640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08980b4740 0x7f08980b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:58.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 -- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0888004d00 msgr2=0x7f0898158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0888004d00 0x7f0898158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 -- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08980b4740 msgr2=0x7f08980b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:58.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08980b4740 0x7f08980b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:58.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 -- 172.21.15.125:0/3315641063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08980b3f40 con 0x7f08980b4bc0 2023-10-18T11:41:58.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.912+0000 7f089d2e9640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08980b4bc0 0x7f08980b5b00 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f089000c760 tx=0x7f0890001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:58.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.913+0000 7f08957fa640 1 -- 172.21.15.125:0/3315641063 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900017f0 con 0x7f08980b4bc0 2023-10-18T11:41:58.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.913+0000 7f08957fa640 1 -- 172.21.15.125:0/3315641063 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0890001970 con 0x7f08980b4bc0 2023-10-18T11:41:58.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.913+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08980b4220 con 0x7f08980b4bc0 2023-10-18T11:41:58.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.913+0000 7f08957fa640 1 -- 172.21.15.125:0/3315641063 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890003bd0 con 0x7f08980b4bc0 2023-10-18T11:41:58.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.913+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0898158b60 con 0x7f08980b4bc0 2023-10-18T11:41:58.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.914+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08980690e0 con 0x7f08980b4bc0 2023-10-18T11:41:58.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.915+0000 7f08957fa640 1 -- 172.21.15.125:0/3315641063 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f0890031030 con 0x7f08980b4bc0 2023-10-18T11:41:58.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.915+0000 7f08957fa640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f087c06af60 0x7f087c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:58.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.915+0000 7f08957fa640 1 -- 172.21.15.125:0/3315641063 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f08900aaa30 con 0x7f08980b4bc0 2023-10-18T11:41:58.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.915+0000 7f089cae8640 1 -- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f087c06af60 msgr2=0x7f087c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:41:58.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.915+0000 7f089cae8640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f087c06af60 0x7f087c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:58.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:58.919+0000 7f08957fa640 1 -- 172.21.15.125:0/3315641063 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0890075860 con 0x7f08980b4bc0 2023-10-18T11:41:58.935 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:58.933+0000 7f2176213040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:41:58.948 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:58.946+0000 7fb56aa3e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:41:58.980 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:58.978+0000 7f2176213040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:41:59.000 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:58.998+0000 7fb56aa3e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:41:59.034 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.032+0000 7f2176213040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:41:59.044 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.042+0000 7fb56aa3e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:41:59.077 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.075+0000 7f2176213040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:41:59.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.091+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08980d0b40 con 0x7f08980b4bc0 2023-10-18T11:41:59.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.091+0000 7f08957fa640 1 -- 172.21.15.125:0/3315641063 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v175) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0890043020 con 0x7f08980b4bc0 2023-10-18T11:41:59.095 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T11:41:59.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f087c06af60 msgr2=0x7f087c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:59.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f087c06af60 0x7f087c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08980b4bc0 msgr2=0x7f08980b5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08980b4bc0 0x7f08980b5b00 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f089000c760 tx=0x7f0890001410 comp rx=0 tx=0).stop 2023-10-18T11:41:59.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 shutdown_connections 2023-10-18T11:41:59.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f087c06af60 0x7f087c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08980b4740 0x7f08980b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08980b4bc0 0x7f08980b5b00 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 --2- 172.21.15.125:0/3315641063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0888004d00 0x7f0898158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 >> 172.21.15.125:0/3315641063 conn(0x7f08980d0d60 msgr2=0x7f08980b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:59.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 shutdown_connections 2023-10-18T11:41:59.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.094+0000 7f089ed4c640 1 -- 172.21.15.125:0/3315641063 wait complete. 2023-10-18T11:41:59.107 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:41:59.142 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.140+0000 7fb56aa3e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:41:59.182 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.179+0000 7f2176213040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:41:59.197 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.195+0000 7fb56aa3e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:59.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 -- 172.21.15.125:0/3454658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0ceb40 msgr2=0x7f0a9c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/3454658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0ceb40 0x7f0a9c0b0650 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c06c510 tx=0x7f0a9402c1d0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 -- 172.21.15.125:0/3454658 shutdown_connections 2023-10-18T11:41:59.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/3454658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0ceb40 0x7f0a9c0b0650 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/3454658 >> v2:172.21.15.139:3300/0 conn(0x7f0a9c15b2a0 0x7f0a9c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/3454658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a9c0cf340 0x7f0a9c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 -- 172.21.15.125:0/3454658 >> 172.21.15.125:0/3454658 conn(0x7f0a9c0d0d60 msgr2=0x7f0a9c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:59.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 -- 172.21.15.125:0/3454658 shutdown_connections 2023-10-18T11:41:59.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.213+0000 7f0aa3f68640 1 -- 172.21.15.125:0/3454658 wait complete. 2023-10-18T11:41:59.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.214+0000 7f0aa3f68640 1 Processor -- start 2023-10-18T11:41:59.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.214+0000 7f0aa3f68640 1 -- start start 2023-10-18T11:41:59.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa3f68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a9c0ceb40 0x7f0a9c149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:59.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa3f68640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0cf340 0x7f0a9c1499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:59.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa3f68640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a9c15b2a0 0x7f0a9c0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:59.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa3f68640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0c7e30 con 0x7f0a9c0cf340 2023-10-18T11:41:59.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa3f68640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0c80d0 con 0x7f0a9c0ceb40 2023-10-18T11:41:59.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa3f68640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0c83b0 con 0x7f0a9c15b2a0 2023-10-18T11:41:59.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1503640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0cf340 0x7f0a9c1499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:59.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1503640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0cf340 0x7f0a9c1499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35046/0 (socket says 172.21.15.125:35046) 2023-10-18T11:41:59.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1503640 1 -- 172.21.15.125:0/1445019777 learned_addr learned my addr 172.21.15.125:0/1445019777 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:59.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1d04640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a9c0ceb40 0x7f0a9c149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:59.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa2505640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a9c15b2a0 0x7f0a9c0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:59.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1503640 1 -- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a9c15b2a0 msgr2=0x7f0a9c0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1503640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a9c15b2a0 0x7f0a9c0c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1503640 1 -- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a9c0ceb40 msgr2=0x7f0a9c149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1503640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a9c0ceb40 0x7f0a9c149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.215+0000 7f0aa1503640 1 -- 172.21.15.125:0/1445019777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94009900 con 0x7f0a9c0cf340 2023-10-18T11:41:59.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.216+0000 7f0aa1503640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0cf340 0x7f0a9c1499a0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009fd0 tx=0x7f0a9800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:59.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.216+0000 7f0a927fc640 1 -- 172.21.15.125:0/1445019777 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98009580 con 0x7f0a9c0cf340 2023-10-18T11:41:59.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.216+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c0c8690 con 0x7f0a9c0cf340 2023-10-18T11:41:59.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.216+0000 7f0a927fc640 1 -- 172.21.15.125:0/1445019777 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a9800ccc0 con 0x7f0a9c0cf340 2023-10-18T11:41:59.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.216+0000 7f0a927fc640 1 -- 172.21.15.125:0/1445019777 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98005280 con 0x7f0a9c0cf340 2023-10-18T11:41:59.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.216+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c0c8c50 con 0x7f0a9c0cf340 2023-10-18T11:41:59.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.217+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c0690e0 con 0x7f0a9c0cf340 2023-10-18T11:41:59.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.217+0000 7f0a927fc640 1 -- 172.21.15.125:0/1445019777 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f0a980053e0 con 0x7f0a9c0cf340 2023-10-18T11:41:59.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.217+0000 7f0a927fc640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f0a8006af90 0x7f0a8006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:59.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.218+0000 7f0a927fc640 1 -- 172.21.15.125:0/1445019777 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f0a98014070 con 0x7f0a9c0cf340 2023-10-18T11:41:59.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.218+0000 7f0aa1d04640 1 -- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f0a8006af90 msgr2=0x7f0a8006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:41:59.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.218+0000 7f0aa1d04640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f0a8006af90 0x7f0a8006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:59.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.222+0000 7f0a927fc640 1 -- 172.21.15.125:0/1445019777 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a98055020 con 0x7f0a9c0cf340 2023-10-18T11:41:59.238 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.236+0000 7f2176213040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:41:59.240 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.238+0000 7fb56aa3e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:41:59.281 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.279+0000 7f2176213040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:41:59.289 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.286+0000 7fb56aa3e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:41:59.330 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.328+0000 7f2176213040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:41:59.332 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.330+0000 7fb56aa3e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:41:59.372 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.370+0000 7f2176213040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:41:59.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.390+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a9c0d0b40 con 0x7f0a9c0cf340 2023-10-18T11:41:59.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.390+0000 7f0a927fc640 1 -- 172.21.15.125:0/1445019777 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v175) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a98058640 con 0x7f0a9c0cf340 2023-10-18T11:41:59.393 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T11:41:59.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f0a8006af90 msgr2=0x7f0a8006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:59.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f0a8006af90 0x7f0a8006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0cf340 msgr2=0x7f0a9c1499a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0cf340 0x7f0a9c1499a0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009fd0 tx=0x7f0a9800ec20 comp rx=0 tx=0).stop 2023-10-18T11:41:59.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 shutdown_connections 2023-10-18T11:41:59.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f0a8006af90 0x7f0a8006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a9c15b2a0 0x7f0a9c0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0a9c0cf340 0x7f0a9c1499a0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 --2- 172.21.15.125:0/1445019777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a9c0ceb40 0x7f0a9c149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 >> 172.21.15.125:0/1445019777 conn(0x7f0a9c0d0d60 msgr2=0x7f0a9c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:59.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 shutdown_connections 2023-10-18T11:41:59.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.392+0000 7f0aa3f68640 1 -- 172.21.15.125:0/1445019777 wait complete. 2023-10-18T11:41:59.405 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T11:41:59.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 -- 172.21.15.125:0/1902948524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2580cf340 msgr2=0x7fd2580d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 --2- 172.21.15.125:0/1902948524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2580cf340 0x7fd2580d1910 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fd25808a120 tx=0x7fd24c02bd10 comp rx=0 tx=0).stop 2023-10-18T11:41:59.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 -- 172.21.15.125:0/1902948524 shutdown_connections 2023-10-18T11:41:59.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 --2- 172.21.15.125:0/1902948524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2580ceb40 0x7fd2580b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 --2- 172.21.15.125:0/1902948524 >> v2:172.21.15.139:3300/0 conn(0x7fd25815b2a0 0x7fd2580cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 --2- 172.21.15.125:0/1902948524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2580cf340 0x7fd2580d1910 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 -- 172.21.15.125:0/1902948524 >> 172.21.15.125:0/1902948524 conn(0x7fd2580d0d60 msgr2=0x7fd2580d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:59.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 -- 172.21.15.125:0/1902948524 shutdown_connections 2023-10-18T11:41:59.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.508+0000 7fd25e38a640 1 -- 172.21.15.125:0/1902948524 wait complete. 2023-10-18T11:41:59.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.509+0000 7fd25e38a640 1 Processor -- start 2023-10-18T11:41:59.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.509+0000 7fd25e38a640 1 -- start start 2023-10-18T11:41:59.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25e38a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2580ceb40 0x7fd25814a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:59.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25e38a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2580cf340 0x7fd25814e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:59.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25e38a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd25815b2a0 0x7fd2580ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:59.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25e38a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25814a960 con 0x7fd25815b2a0 2023-10-18T11:41:59.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25e38a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580c7e30 con 0x7fd2580cf340 2023-10-18T11:41:59.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25e38a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25814ea80 con 0x7fd2580ceb40 2023-10-18T11:41:59.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd25815b2a0 0x7fd2580ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:59.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd257fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2580ceb40 0x7fd25814a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:59.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd25815b2a0 0x7fd2580ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35072/0 (socket says 172.21.15.125:35072) 2023-10-18T11:41:59.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 -- 172.21.15.125:0/2284817359 learned_addr learned my addr 172.21.15.125:0/2284817359 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:41:59.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd2577fe640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2580cf340 0x7fd25814e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:41:59.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 -- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2580ceb40 msgr2=0x7fd25814a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2580ceb40 0x7fd25814a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 -- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2580cf340 msgr2=0x7fd25814e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2580cf340 0x7fd25814e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 -- 172.21.15.125:0/2284817359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c009560 con 0x7fd25815b2a0 2023-10-18T11:41:59.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd257fff640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2580ceb40 0x7fd25814a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:41:59.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.510+0000 7fd25c927640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd25815b2a0 0x7fd2580ca160 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd25000eaa0 tx=0x7fd25000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:41:59.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.511+0000 7fd254ff9640 1 -- 172.21.15.125:0/2284817359 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25000ce70 con 0x7fd25815b2a0 2023-10-18T11:41:59.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.511+0000 7fd254ff9640 1 -- 172.21.15.125:0/2284817359 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd250004c10 con 0x7fd25815b2a0 2023-10-18T11:41:59.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.511+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25814ed60 con 0x7fd25815b2a0 2023-10-18T11:41:59.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.511+0000 7fd254ff9640 1 -- 172.21.15.125:0/2284817359 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250005020 con 0x7fd25815b2a0 2023-10-18T11:41:59.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.511+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25814f320 con 0x7fd25815b2a0 2023-10-18T11:41:59.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.512+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2580c8050 con 0x7fd25815b2a0 2023-10-18T11:41:59.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.512+0000 7fd254ff9640 1 -- 172.21.15.125:0/2284817359 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd2500051a0 con 0x7fd25815b2a0 2023-10-18T11:41:59.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.513+0000 7fd254ff9640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd24406af60 0x7fd24406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:41:59.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.513+0000 7fd257fff640 1 -- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd24406af60 msgr2=0x7fd24406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:41:59.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.513+0000 7fd254ff9640 1 -- 172.21.15.125:0/2284817359 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fd250014070 con 0x7fd25815b2a0 2023-10-18T11:41:59.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.513+0000 7fd257fff640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd24406af60 0x7fd24406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:41:59.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.517+0000 7fd254ff9640 1 -- 172.21.15.125:0/2284817359 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2580c8050 con 0x7fd25815b2a0 2023-10-18T11:41:59.650 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.647+0000 7fb56aa3e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:41:59.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.695+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd2580d07d0 con 0x7fd25815b2a0 2023-10-18T11:41:59.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.696+0000 7fd254ff9640 1 -- 172.21.15.125:0/2284817359 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v175) v1 ==== 80+0+476 (secure 0 0 0) 0x7fd250055590 con 0x7fd25815b2a0 2023-10-18T11:41:59.698 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3094960806 2023-10-19T11:41:55.015024+0000 2023-10-18T11:41:59.699 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/139213704 2023-10-19T11:41:55.015024+0000 2023-10-18T11:41:59.699 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/4234510722 2023-10-19T11:41:55.015024+0000 2023-10-18T11:41:59.699 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/4113808248 2023-10-19T11:41:55.015024+0000 2023-10-18T11:41:59.699 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/4236255547 2023-10-19T11:41:55.015024+0000 2023-10-18T11:41:59.699 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/312743033 2023-10-19T11:41:55.015024+0000 2023-10-18T11:41:59.700 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/4236255547 2023-10-19T11:41:55.015024+0000 2023-10-18T11:41:59.700 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1471732770 2023-10-19T11:41:55.015024+0000 2023-10-18T11:41:59.700 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T11:41:59.701 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.697+0000 7f2176213040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:41:59.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd24406af60 msgr2=0x7fd24406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:41:59.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd24406af60 0x7fd24406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd25815b2a0 msgr2=0x7fd2580ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:41:59.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd25815b2a0 0x7fd2580ca160 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd25000eaa0 tx=0x7fd25000ef60 comp rx=0 tx=0).stop 2023-10-18T11:41:59.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 shutdown_connections 2023-10-18T11:41:59.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fd24406af60 0x7fd24406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2580cf340 0x7fd25814e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd25815b2a0 0x7fd2580ca160 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 --2- 172.21.15.125:0/2284817359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2580ceb40 0x7fd25814a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:41:59.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 >> 172.21.15.125:0/2284817359 conn(0x7fd2580d0d60 msgr2=0x7fd2580cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:41:59.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 shutdown_connections 2023-10-18T11:41:59.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:41:59.698+0000 7fd25e38a640 1 -- 172.21.15.125:0/2284817359 wait complete. 2023-10-18T11:41:59.711 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T11:41:59.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:59.712 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T11:41:59.736 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[143307]: fuse finished with error 0 and tester_r 0 2023-10-18T11:41:59.738 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T11:41:59.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:59.739 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T11:41:59.782 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.780+0000 7fb56aa3e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:41:59.806 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:41:59.806 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:59.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:41:59.822 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T11:41:59.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:59.823 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T11:41:59.838 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.836+0000 7f2176213040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:41:59.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:41:59.877 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T11:41:59.878 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:41:59.878 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:59.879 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.876+0000 7fb56aa3e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:41:59.893 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:41:59.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T11:41:59.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:59.894 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T11:41:59.922 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:41:59.920+0000 7fb56aa3e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:41:59.932 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.930+0000 7f2176213040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:41:59.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:41:59.946 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T11:41:59.947 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:41:59.947 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:41:59.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:41:59.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T11:41:59.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:41:59.963 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T11:41:59.976 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:41:59.974+0000 7f2176213040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:42:00.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:42:00.017 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T11:42:00.018 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:42:00.020 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:00.018+0000 7fb56aa3e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:42:00.074 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:00.072+0000 7f2176213040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:42:00.114 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:00.112+0000 7fb56aa3e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:42:00.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.119+0000 7f15365b6640 1 -- 172.21.15.125:0/3965968369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300cf340 msgr2=0x7f15300d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.119+0000 7f15365b6640 1 --2- 172.21.15.125:0/3965968369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300cf340 0x7f15300d1910 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f153008a120 tx=0x7f152002bdb0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.119+0000 7f15365b6640 1 -- 172.21.15.125:0/3965968369 shutdown_connections 2023-10-18T11:42:00.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.119+0000 7f15365b6640 1 --2- 172.21.15.125:0/3965968369 >> v2:172.21.15.139:3300/0 conn(0x7f153015b2a0 0x7f15300cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.119+0000 7f15365b6640 1 --2- 172.21.15.125:0/3965968369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300cf340 0x7f15300d1910 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.119+0000 7f15365b6640 1 -- 172.21.15.125:0/3965968369 >> 172.21.15.125:0/3965968369 conn(0x7f15300d0d60 msgr2=0x7f15300d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:00.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.119+0000 7f15365b6640 1 -- 172.21.15.125:0/3965968369 shutdown_connections 2023-10-18T11:42:00.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.119+0000 7f15365b6640 1 -- 172.21.15.125:0/3965968369 wait complete. 2023-10-18T11:42:00.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.120+0000 7f15365b6640 1 Processor -- start 2023-10-18T11:42:00.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f15365b6640 1 -- start start 2023-10-18T11:42:00.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f15365b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300ceb40 0x7f153006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f15365b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15300cf340 0x7f15300ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f15365b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153015b2a0 0x7f15300ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f15365b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15300b4810 con 0x7f15300ceb40 2023-10-18T11:42:00.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f15365b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15300c7f60 con 0x7f153015b2a0 2023-10-18T11:42:00.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f15365b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15300c8240 con 0x7f15300cf340 2023-10-18T11:42:00.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f152ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300ceb40 0x7f153006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f152ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300ceb40 0x7f153006b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35088/0 (socket says 172.21.15.125:35088) 2023-10-18T11:42:00.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.121+0000 7f152ffff640 1 -- 172.21.15.125:0/3736457255 learned_addr learned my addr 172.21.15.125:0/3736457255 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:00.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152f7fe640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15300cf340 0x7f15300ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f1534b53640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153015b2a0 0x7f15300ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152ffff640 1 -- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15300cf340 msgr2=0x7f15300ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152ffff640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15300cf340 0x7f15300ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152ffff640 1 -- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153015b2a0 msgr2=0x7f15300ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152ffff640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153015b2a0 0x7f15300ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152ffff640 1 -- 172.21.15.125:0/3736457255 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520009560 con 0x7f15300ceb40 2023-10-18T11:42:00.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152ffff640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300ceb40 0x7f153006b5b0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f1520000c00 tx=0x7f152002da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:00.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152cff9640 1 -- 172.21.15.125:0/3736457255 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520003800 con 0x7f15300ceb40 2023-10-18T11:42:00.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152cff9640 1 -- 172.21.15.125:0/3736457255 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1520003a50 con 0x7f15300ceb40 2023-10-18T11:42:00.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f152cff9640 1 -- 172.21.15.125:0/3736457255 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520003e60 con 0x7f15300ceb40 2023-10-18T11:42:00.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15300c8520 con 0x7f15300ceb40 2023-10-18T11:42:00.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.122+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15300caba0 con 0x7f15300ceb40 2023-10-18T11:42:00.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.123+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15300690e0 con 0x7f15300ceb40 2023-10-18T11:42:00.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.124+0000 7f152cff9640 1 -- 172.21.15.125:0/3736457255 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f152003e450 con 0x7f15300ceb40 2023-10-18T11:42:00.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.124+0000 7f152cff9640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f151c06af60 0x7f151c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.124+0000 7f152f7fe640 1 -- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f151c06af60 msgr2=0x7f151c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:42:00.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.124+0000 7f152cff9640 1 -- 172.21.15.125:0/3736457255 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f15200ac3d0 con 0x7f15300ceb40 2023-10-18T11:42:00.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.124+0000 7f152f7fe640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f151c06af60 0x7f151c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:42:00.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.128+0000 7f152cff9640 1 -- 172.21.15.125:0/3736457255 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15200771f0 con 0x7f15300ceb40 2023-10-18T11:42:00.170 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:00.167+0000 7f2176213040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:42:00.253 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:00.250+0000 7fb56aa3e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:42:00.294 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:00.291+0000 7fb56aa3e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:42:00.308 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:00.306+0000 7f2176213040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:42:00.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.325+0000 7f152f7fe640 1 -- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f151c06af60 msgr2=0x7f151c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:42:00.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.325+0000 7f152f7fe640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f151c06af60 0x7f151c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:42:00.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.331+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15300cb9d0 con 0x7f15300ceb40 2023-10-18T11:42:00.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.332+0000 7f152cff9640 1 -- 172.21.15.125:0/3736457255 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f152007a810 con 0x7f15300ceb40 2023-10-18T11:42:00.335 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:00.335 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":67,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3883749389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3883749389},{"type":"v1","addr":"172.21.15.139:6836","nonce":3883749389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2038008432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2038008432},{"type":"v1","addr":"172.21.15.139:6837","nonce":2038008432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":10317,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1543508506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1543508506},{"type":"v1","addr":"172.21.15.125:6834","nonce":1543508506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:41:38.149238+0000","modified":"2023-10-18T11:41:40.479014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10320":{"gid":10320,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2949284679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2949284679},{"type":"v1","addr":"172.21.15.125:6835","nonce":2949284679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T11:42:00.335 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 67 2023-10-18T11:42:00.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f151c06af60 msgr2=0x7f151c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:42:00.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f151c06af60 0x7f151c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300ceb40 msgr2=0x7f153006b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300ceb40 0x7f153006b5b0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f1520000c00 tx=0x7f152002da80 comp rx=0 tx=0).stop 2023-10-18T11:42:00.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 shutdown_connections 2023-10-18T11:42:00.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f151c06af60 0x7f151c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153015b2a0 0x7f15300ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15300cf340 0x7f15300ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 --2- 172.21.15.125:0/3736457255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15300ceb40 0x7f153006b5b0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 >> 172.21.15.125:0/3736457255 conn(0x7f15300d0d60 msgr2=0x7f153015a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:00.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 shutdown_connections 2023-10-18T11:42:00.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.334+0000 7f15365b6640 1 -- 172.21.15.125:0/3736457255 wait complete. 2023-10-18T11:42:00.347 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:42:00.352 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:00.350+0000 7f2176213040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:42:00.398 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:00.396+0000 7fb56aa3e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:42:00.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.449+0000 7febf4010640 1 -- 172.21.15.125:0/3930315183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febd8003970 msgr2=0x7febd8003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.449+0000 7febf4010640 1 --2- 172.21.15.125:0/3930315183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febd8003970 0x7febd8003d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7febe800c260 tx=0x7febd80057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.449+0000 7febf25ad640 1 -- 172.21.15.125:0/3930315183 reap_dead start 2023-10-18T11:42:00.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.449+0000 7febf4010640 1 -- 172.21.15.125:0/3930315183 shutdown_connections 2023-10-18T11:42:00.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.449+0000 7febf4010640 1 -- 172.21.15.125:0/3930315183 >> 172.21.15.125:0/3930315183 conn(0x7febec069930 msgr2=0x7febec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:00.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.449+0000 7febf4010640 1 -- 172.21.15.125:0/3930315183 shutdown_connections 2023-10-18T11:42:00.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.449+0000 7febf4010640 1 -- 172.21.15.125:0/3930315183 wait complete. 2023-10-18T11:42:00.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.450+0000 7febf4010640 1 Processor -- start 2023-10-18T11:42:00.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.450+0000 7febf4010640 1 -- start start 2023-10-18T11:42:00.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf4010640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febd8003970 0x7febec1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf4010640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febec070cd0 0x7febec15a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf4010640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febec15aeb0 0x7febec1393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf4010640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec071240 con 0x7febec070cd0 2023-10-18T11:42:00.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf4010640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0af510 con 0x7febd8003970 2023-10-18T11:42:00.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf4010640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec158e10 con 0x7febec15aeb0 2023-10-18T11:42:00.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febec070cd0 0x7febec15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febec070cd0 0x7febec15a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35110/0 (socket says 172.21.15.125:35110) 2023-10-18T11:42:00.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 -- 172.21.15.125:0/1862056104 learned_addr learned my addr 172.21.15.125:0/1862056104 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:00.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf1dac640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febd8003970 0x7febec1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf25ad640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febec15aeb0 0x7febec1393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 -- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febec15aeb0 msgr2=0x7febec1393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febec15aeb0 0x7febec1393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 -- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febd8003970 msgr2=0x7febec1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febd8003970 0x7febec1588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 -- 172.21.15.125:0/1862056104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0aef50 con 0x7febec070cd0 2023-10-18T11:42:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.451+0000 7febf15ab640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febec070cd0 0x7febec15a970 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7febdc011ba0 tx=0x7febdc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.452+0000 7febe27fc640 1 -- 172.21.15.125:0/1862056104 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc00e7c0 con 0x7febec070cd0 2023-10-18T11:42:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.452+0000 7febe27fc640 1 -- 172.21.15.125:0/1862056104 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7febdc00e920 con 0x7febec070cd0 2023-10-18T11:42:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.452+0000 7febe27fc640 1 -- 172.21.15.125:0/1862056104 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc00ecf0 con 0x7febec070cd0 2023-10-18T11:42:00.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.452+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec0af230 con 0x7febec070cd0 2023-10-18T11:42:00.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.452+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec154ea0 con 0x7febec070cd0 2023-10-18T11:42:00.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.453+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febec1525c0 con 0x7febec070cd0 2023-10-18T11:42:00.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.453+0000 7febe27fc640 1 -- 172.21.15.125:0/1862056104 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7febdc019430 con 0x7febec070cd0 2023-10-18T11:42:00.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.454+0000 7febe27fc640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7febd006af60 0x7febd006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.454+0000 7febf1dac640 1 -- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7febd006af60 msgr2=0x7febd006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:42:00.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.454+0000 7febe27fc640 1 -- 172.21.15.125:0/1862056104 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7febdc08af10 con 0x7febec070cd0 2023-10-18T11:42:00.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.454+0000 7febf1dac640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7febd006af60 0x7febd006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:42:00.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.458+0000 7febe27fc640 1 -- 172.21.15.125:0/1862056104 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7febdc055af0 con 0x7febec070cd0 2023-10-18T11:42:00.465 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:00.462+0000 7f2176213040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:42:00.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.627+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febec133cb0 con 0x7febec070cd0 2023-10-18T11:42:00.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.628+0000 7febe27fc640 1 -- 172.21.15.125:0/1862056104 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+17323 (secure 0 0 0) 0x7febdc01e020 con 0x7febec070cd0 2023-10-18T11:42:00.631 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:00.631 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":175,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:41:55.015079+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:41:35.970906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-18T11:41:37.131199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3094960806":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/139213704":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4234510722":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4113808248":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6833/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/312743033":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6832/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/1471732770":"2023-10-19T11:41:55.015024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:42:00.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7febd006af60 msgr2=0x7febd006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:42:00.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7febd006af60 0x7febd006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febec070cd0 msgr2=0x7febec15a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febec070cd0 0x7febec15a970 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7febdc011ba0 tx=0x7febdc00f790 comp rx=0 tx=0).stop 2023-10-18T11:42:00.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 shutdown_connections 2023-10-18T11:42:00.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7febd006af60 0x7febd006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7febec15aeb0 0x7febec1393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febec070cd0 0x7febec15a970 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 --2- 172.21.15.125:0/1862056104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febd8003970 0x7febec1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 >> 172.21.15.125:0/1862056104 conn(0x7febec069930 msgr2=0x7febec13b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:00.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 shutdown_connections 2023-10-18T11:42:00.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.630+0000 7febf4010640 1 -- 172.21.15.125:0/1862056104 wait complete. 2023-10-18T11:42:00.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T11:41:35.970906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '173', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T11:41:37.131199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:42:00.644 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:42:00.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.786+0000 7f21976b4640 1 -- 172.21.15.125:0/1343925858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21900cf340 msgr2=0x7f21900d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.786+0000 7f21976b4640 1 --2- 172.21.15.125:0/1343925858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21900cf340 0x7f21900d1910 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f218c005fd0 tx=0x7f218c02bd90 comp rx=0 tx=0).stop 2023-10-18T11:42:00.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.786+0000 7f21976b4640 1 -- 172.21.15.125:0/1343925858 shutdown_connections 2023-10-18T11:42:00.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.786+0000 7f21976b4640 1 --2- 172.21.15.125:0/1343925858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f219015b2a0 0x7f21900cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.786+0000 7f21976b4640 1 --2- 172.21.15.125:0/1343925858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21900cf340 0x7f21900d1910 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.786+0000 7f21976b4640 1 -- 172.21.15.125:0/1343925858 >> 172.21.15.125:0/1343925858 conn(0x7f21900d0d60 msgr2=0x7f21900d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:00.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.786+0000 7f21976b4640 1 -- 172.21.15.125:0/1343925858 shutdown_connections 2023-10-18T11:42:00.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.786+0000 7f21976b4640 1 -- 172.21.15.125:0/1343925858 wait complete. 2023-10-18T11:42:00.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.787+0000 7f21976b4640 1 Processor -- start 2023-10-18T11:42:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f21976b4640 1 -- start start 2023-10-18T11:42:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f21976b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900ceb40 0x7f21900c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f21976b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21900cf340 0x7f2190158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f21976b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f219015b2a0 0x7f21900b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f21976b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21900c9820 con 0x7f219015b2a0 2023-10-18T11:42:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f21976b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21900c9390 con 0x7f21900ceb40 2023-10-18T11:42:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f21976b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21900c8780 con 0x7f21900cf340 2023-10-18T11:42:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f2195c51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f219015b2a0 0x7f21900b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f2195c51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f219015b2a0 0x7f21900b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35130/0 (socket says 172.21.15.125:35130) 2023-10-18T11:42:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f2195c51640 1 -- 172.21.15.125:0/1446352806 learned_addr learned my addr 172.21.15.125:0/1446352806 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f2194c4f640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21900cf340 0x7f2190158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.788+0000 7f2195450640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900ceb40 0x7f21900c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2195c51640 1 -- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21900cf340 msgr2=0x7f2190158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2195c51640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21900cf340 0x7f2190158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2195c51640 1 -- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900ceb40 msgr2=0x7f21900c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2195c51640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900ceb40 0x7f21900c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2195c51640 1 -- 172.21.15.125:0/1446352806 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c009560 con 0x7f219015b2a0 2023-10-18T11:42:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2195c51640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f219015b2a0 0x7f21900b5200 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f218800c710 tx=0x7f218800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:00.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2185ffb640 1 -- 172.21.15.125:0/1446352806 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188006d50 con 0x7f219015b2a0 2023-10-18T11:42:00.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21900c8a60 con 0x7f219015b2a0 2023-10-18T11:42:00.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2185ffb640 1 -- 172.21.15.125:0/1446352806 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f218800ce20 con 0x7f219015b2a0 2023-10-18T11:42:00.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f2185ffb640 1 -- 172.21.15.125:0/1446352806 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188003bd0 con 0x7f219015b2a0 2023-10-18T11:42:00.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.789+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21900c9020 con 0x7f219015b2a0 2023-10-18T11:42:00.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.790+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21900cb450 con 0x7f219015b2a0 2023-10-18T11:42:00.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.791+0000 7f2185ffb640 1 -- 172.21.15.125:0/1446352806 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2188003d30 con 0x7f219015b2a0 2023-10-18T11:42:00.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.791+0000 7f2185ffb640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f217406af60 0x7f217406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:00.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.791+0000 7f2195450640 1 -- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f217406af60 msgr2=0x7f217406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:42:00.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.791+0000 7f2185ffb640 1 -- 172.21.15.125:0/1446352806 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f2188089710 con 0x7f219015b2a0 2023-10-18T11:42:00.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.791+0000 7f2195450640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f217406af60 0x7f217406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:42:00.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.796+0000 7f2185ffb640 1 -- 172.21.15.125:0/1446352806 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21880544b0 con 0x7f219015b2a0 2023-10-18T11:42:00.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.967+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21900690e0 con 0x7f219015b2a0 2023-10-18T11:42:00.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.968+0000 7f2185ffb640 1 -- 172.21.15.125:0/1446352806 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2188057ad0 con 0x7f219015b2a0 2023-10-18T11:42:00.971 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:42:00.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f217406af60 msgr2=0x7f217406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:42:00.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f217406af60 0x7f217406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f219015b2a0 msgr2=0x7f21900b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:00.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f219015b2a0 0x7f21900b5200 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f218800c710 tx=0x7f218800cbd0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 shutdown_connections 2023-10-18T11:42:00.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f217406af60 0x7f217406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f219015b2a0 0x7f21900b5200 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21900cf340 0x7f2190158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 --2- 172.21.15.125:0/1446352806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900ceb40 0x7f21900c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:00.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 >> 172.21.15.125:0/1446352806 conn(0x7f21900d0d60 msgr2=0x7f219006afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:00.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 shutdown_connections 2023-10-18T11:42:00.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:00.970+0000 7f21976b4640 1 -- 172.21.15.125:0/1446352806 wait complete. 2023-10-18T11:42:00.983 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:42:01.015 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.013+0000 7fb56aa3e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:42:01.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.089+0000 7f7ecd464640 1 -- 172.21.15.125:0/2168176925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 msgr2=0x7f7ec806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.089+0000 7f7ecd464640 1 --2- 172.21.15.125:0/2168176925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 0x7f7ec806d6b0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc005fd0 tx=0x7f7ebc02be10 comp rx=0 tx=0).stop 2023-10-18T11:42:01.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.089+0000 7f7ecd464640 1 -- 172.21.15.125:0/2168176925 shutdown_connections 2023-10-18T11:42:01.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.089+0000 7f7ecd464640 1 --2- 172.21.15.125:0/2168176925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ec80ad370 0x7f7ec80ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.089+0000 7f7ecd464640 1 --2- 172.21.15.125:0/2168176925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 0x7f7ec806d6b0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.089+0000 7f7ecd464640 1 -- 172.21.15.125:0/2168176925 >> 172.21.15.125:0/2168176925 conn(0x7f7ec80b0670 msgr2=0x7f7ec80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:01.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.089+0000 7f7ecd464640 1 -- 172.21.15.125:0/2168176925 shutdown_connections 2023-10-18T11:42:01.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.089+0000 7f7ecd464640 1 -- 172.21.15.125:0/2168176925 wait complete. 2023-10-18T11:42:01.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.090+0000 7f7ecd464640 1 Processor -- start 2023-10-18T11:42:01.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.090+0000 7f7ecd464640 1 -- start start 2023-10-18T11:42:01.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ecd464640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 0x7f7ec80cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ecd464640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ec806f0a0 0x7f7ec81f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ecd464640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ec80ad370 0x7f7ec81f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ecd464640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80c81d0 con 0x7f7ec806d2d0 2023-10-18T11:42:01.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ecd464640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80c84b0 con 0x7f7ec806f0a0 2023-10-18T11:42:01.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ecd464640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80c8790 con 0x7f7ec80ad370 2023-10-18T11:42:01.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 0x7f7ec80cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 0x7f7ec80cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35152/0 (socket says 172.21.15.125:35152) 2023-10-18T11:42:01.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 -- 172.21.15.125:0/884735101 learned_addr learned my addr 172.21.15.125:0/884735101 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:01.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec67fc640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ec806f0a0 0x7f7ec81f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec77fe640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ec80ad370 0x7f7ec81f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 -- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ec80ad370 msgr2=0x7f7ec81f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ec80ad370 0x7f7ec81f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 -- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ec806f0a0 msgr2=0x7f7ec81f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ec806f0a0 0x7f7ec81f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 -- 172.21.15.125:0/884735101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc009560 con 0x7f7ec806d2d0 2023-10-18T11:42:01.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec67fc640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ec806f0a0 0x7f7ec81f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:42:01.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7ec6ffd640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 0x7f7ec80cbe10 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009850 tx=0x7f7ebc02da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:01.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.091+0000 7f7eaf7fe640 1 -- 172.21.15.125:0/884735101 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc02de50 con 0x7f7ec806d2d0 2023-10-18T11:42:01.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.092+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec80c8a70 con 0x7f7ec806d2d0 2023-10-18T11:42:01.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.092+0000 7f7eaf7fe640 1 -- 172.21.15.125:0/884735101 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ebc0036e0 con 0x7f7ec806d2d0 2023-10-18T11:42:01.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.092+0000 7f7eaf7fe640 1 -- 172.21.15.125:0/884735101 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc003a90 con 0x7f7ec806d2d0 2023-10-18T11:42:01.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.092+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec80c7e80 con 0x7f7ec806d2d0 2023-10-18T11:42:01.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.093+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec80ce1d0 con 0x7f7ec806d2d0 2023-10-18T11:42:01.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.093+0000 7f7eaf7fe640 1 -- 172.21.15.125:0/884735101 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f7ebc003bf0 con 0x7f7ec806d2d0 2023-10-18T11:42:01.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.093+0000 7f7eaf7fe640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f7eb406af60 0x7f7eb406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.093+0000 7f7ec67fc640 1 -- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f7eb406af60 msgr2=0x7f7eb406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:42:01.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.094+0000 7f7eaf7fe640 1 -- 172.21.15.125:0/884735101 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f7ebc0ac4f0 con 0x7f7ec806d2d0 2023-10-18T11:42:01.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.094+0000 7f7ec67fc640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f7eb406af60 0x7f7eb406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:42:01.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.098+0000 7f7eaf7fe640 1 -- 172.21.15.125:0/884735101 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ebc077320 con 0x7f7ec806d2d0 2023-10-18T11:42:01.112 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:01.110+0000 7f2176213040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:42:01.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.269+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ec80690e0 con 0x7f7ec806d2d0 2023-10-18T11:42:01.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.269+0000 7f7eaf7fe640 1 -- 172.21.15.125:0/884735101 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v175) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ebc07a940 con 0x7f7ec806d2d0 2023-10-18T11:42:01.272 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T11:42:01.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f7eb406af60 msgr2=0x7f7eb406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:42:01.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f7eb406af60 0x7f7eb406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 msgr2=0x7f7ec80cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 0x7f7ec80cbe10 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009850 tx=0x7f7ebc02da50 comp rx=0 tx=0).stop 2023-10-18T11:42:01.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 shutdown_connections 2023-10-18T11:42:01.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7f7eb406af60 0x7f7eb406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ec806f0a0 0x7f7ec81f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ec80ad370 0x7f7ec81f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 --2- 172.21.15.125:0/884735101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec806d2d0 0x7f7ec80cbe10 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 >> 172.21.15.125:0/884735101 conn(0x7f7ec80b0670 msgr2=0x7f7ec80ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:01.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 shutdown_connections 2023-10-18T11:42:01.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.271+0000 7f7ecd464640 1 -- 172.21.15.125:0/884735101 wait complete. 2023-10-18T11:42:01.279 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.277+0000 7fb56aa3e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:42:01.284 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:42:01.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.325+0000 7fb56aa3e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:42:01.383 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:01.380+0000 7f2176213040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:42:01.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.387+0000 7fddcf368640 1 -- 172.21.15.125:0/4156044930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc80ce240 msgr2=0x7fddc806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.387+0000 7fddcf368640 1 --2- 172.21.15.125:0/4156044930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc80ce240 0x7fddc806d260 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fddb8005fd0 tx=0x7fddb802bd90 comp rx=0 tx=0).stop 2023-10-18T11:42:01.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.387+0000 7fddcf368640 1 -- 172.21.15.125:0/4156044930 shutdown_connections 2023-10-18T11:42:01.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.387+0000 7fddcf368640 1 --2- 172.21.15.125:0/4156044930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc815ccf0 0x7fddc815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.388+0000 7fddcf368640 1 --2- 172.21.15.125:0/4156044930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc80ce240 0x7fddc806d260 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.388+0000 7fddcf368640 1 -- 172.21.15.125:0/4156044930 >> 172.21.15.125:0/4156044930 conn(0x7fddc80b0670 msgr2=0x7fddc80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:01.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.388+0000 7fddcf368640 1 -- 172.21.15.125:0/4156044930 shutdown_connections 2023-10-18T11:42:01.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.388+0000 7fddcf368640 1 -- 172.21.15.125:0/4156044930 wait complete. 2023-10-18T11:42:01.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.389+0000 7fddcf368640 1 Processor -- start 2023-10-18T11:42:01.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.389+0000 7fddcf368640 1 -- start start 2023-10-18T11:42:01.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.389+0000 7fddcf368640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc80ce240 0x7fddc80c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.389+0000 7fddcf368640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc8159060 0x7fddc80b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.389+0000 7fddcf368640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc815ccf0 0x7fddc80b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.389+0000 7fddcf368640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80c3e20 con 0x7fddc8159060 2023-10-18T11:42:01.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.389+0000 7fddcf368640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80c4100 con 0x7fddc80ce240 2023-10-18T11:42:01.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.389+0000 7fddcf368640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80c43e0 con 0x7fddc815ccf0 2023-10-18T11:42:01.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc8159060 0x7fddc80b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc8159060 0x7fddc80b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35164/0 (socket says 172.21.15.125:35164) 2023-10-18T11:42:01.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 -- 172.21.15.125:0/263857778 learned_addr learned my addr 172.21.15.125:0/263857778 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:01.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcd905640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc815ccf0 0x7fddc80b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcd104640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc80ce240 0x7fddc80c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 -- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc815ccf0 msgr2=0x7fddc80b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc815ccf0 0x7fddc80b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 -- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc80ce240 msgr2=0x7fddc80c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc80ce240 0x7fddc80c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 -- 172.21.15.125:0/263857778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8009560 con 0x7fddc8159060 2023-10-18T11:42:01.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcc903640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc8159060 0x7fddc80b3770 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009850 tx=0x7fddb802da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:01.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb802de60 con 0x7fddc8159060 2023-10-18T11:42:01.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fddb80036e0 con 0x7fddc8159060 2023-10-18T11:42:01.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc80c46c0 con 0x7fddc8159060 2023-10-18T11:42:01.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.390+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8003a90 con 0x7fddc8159060 2023-10-18T11:42:01.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.391+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc8157f20 con 0x7fddc8159060 2023-10-18T11:42:01.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.392+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc815b150 con 0x7fddc8159060 2023-10-18T11:42:01.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.392+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 97) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fddb8003bf0 con 0x7fddc8159060 2023-10-18T11:42:01.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.392+0000 7fddbdffb640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fddac06af60 0x7fddac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.393+0000 7fddcd104640 1 -- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fddac06af60 msgr2=0x7fddac06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1922997989 2023-10-18T11:42:01.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.393+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fddb80ac530 con 0x7fddc8159060 2023-10-18T11:42:01.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.393+0000 7fddcd104640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fddac06af60 0x7fddac06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:42:01.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.396+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fddc815b150 con 0x7fddc8159060 2023-10-18T11:42:01.427 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.425+0000 7fb56aa3e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:42:01.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.445+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 98) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fddb8077820 con 0x7fddc8159060 2023-10-18T11:42:01.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.446+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fddac06af60 msgr2=0x7fddac06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:42:01.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.446+0000 7fddbdffb640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fddac06af60 0x7fddac06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.459 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:01.457+0000 7f2176213040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:42:01.502 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.500+0000 7fb4efd71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.503 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.500+0000 7fb4efd71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.503 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.500+0000 7fb4efd71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.503 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.500+0000 7fb4efd71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.503 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.500+0000 7fb4efd71640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.504 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.501+0000 7fb4ecd6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.505 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.501+0000 7fb4ecd6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.505 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.501+0000 7fb4ecd6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.505 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.501+0000 7fb4ecd6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.505 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:01.501+0000 7fb4ecd6b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:01.580 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:01.578+0000 7f2176213040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:42:01.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.626+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fddc815b150 con 0x7fddc8159060 2023-10-18T11:42:01.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.627+0000 7fddbdffb640 1 -- 172.21.15.125:0/263857778 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v176) v1 ==== 100+0+11 (secure 0 0 0) 0x7fddc815b150 con 0x7fddc8159060 2023-10-18T11:42:01.629 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T11:42:01.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc8159060 msgr2=0x7fddc80b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc8159060 0x7fddc80b3770 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009850 tx=0x7fddb802da60 comp rx=0 tx=0).stop 2023-10-18T11:42:01.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 shutdown_connections 2023-10-18T11:42:01.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:6832/1922997989,v1:172.21.15.139:6833/1922997989] conn(0x7fddac06af60 0x7fddac06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc815ccf0 0x7fddc80b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc8159060 0x7fddc80b3770 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 --2- 172.21.15.125:0/263857778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc80ce240 0x7fddc80c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 >> 172.21.15.125:0/263857778 conn(0x7fddc80b0670 msgr2=0x7fddc80cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:01.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 shutdown_connections 2023-10-18T11:42:01.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.629+0000 7fddcf368640 1 -- 172.21.15.125:0/263857778 wait complete. 2023-10-18T11:42:01.642 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:42:01.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 -- 172.21.15.125:0/1523590184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d806c0a0 msgr2=0x7f06d8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 --2- 172.21.15.125:0/1523590184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d806c0a0 0x7f06d8070920 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f06d806a780 tx=0x7f06d002c140 comp rx=0 tx=0).stop 2023-10-18T11:42:01.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 -- 172.21.15.125:0/1523590184 shutdown_connections 2023-10-18T11:42:01.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 --2- 172.21.15.125:0/1523590184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d806c0a0 0x7f06d8070920 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 --2- 172.21.15.125:0/1523590184 >> v2:172.21.15.139:3300/0 conn(0x7f06d8141690 0x7f06d80703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 --2- 172.21.15.125:0/1523590184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06d813e520 0x7f06d806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 -- 172.21.15.125:0/1523590184 >> 172.21.15.125:0/1523590184 conn(0x7f06d8069930 msgr2=0x7f06d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:01.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 -- 172.21.15.125:0/1523590184 shutdown_connections 2023-10-18T11:42:01.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.742+0000 7f06dea32640 1 -- 172.21.15.125:0/1523590184 wait complete. 2023-10-18T11:42:01.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.743+0000 7f06dea32640 1 Processor -- start 2023-10-18T11:42:01.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.743+0000 7f06dea32640 1 -- start start 2023-10-18T11:42:01.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dea32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d806c0a0 0x7f06d8139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dea32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06d813e520 0x7f06d813a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dea32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d8141690 0x7f06d813cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:01.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dea32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8157f20 con 0x7f06d8141690 2023-10-18T11:42:01.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dea32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d81628b0 con 0x7f06d806c0a0 2023-10-18T11:42:01.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dea32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d813a420 con 0x7f06d813e520 2023-10-18T11:42:01.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d8141690 0x7f06d813cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d8141690 0x7f06d813cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35188/0 (socket says 172.21.15.125:35188) 2023-10-18T11:42:01.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 -- 172.21.15.125:0/3356197866 learned_addr learned my addr 172.21.15.125:0/3356197866 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:01.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06d7fff640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d806c0a0 0x7f06d8139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06d77fe640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06d813e520 0x7f06d813a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:01.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 -- 172.21.15.125:0/3356197866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06d813e520 msgr2=0x7f06d813a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06d813e520 0x7f06d813a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 -- 172.21.15.125:0/3356197866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d806c0a0 msgr2=0x7f06d8139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d806c0a0 0x7f06d8139ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 -- 172.21.15.125:0/3356197866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0009900 con 0x7f06d8141690 2023-10-18T11:42:01.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dcfcf640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d8141690 0x7f06d813cb20 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009d60 tx=0x7f06d0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:01.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06d4ff9640 1 -- 172.21.15.125:0/3356197866 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0034440 con 0x7f06d8141690 2023-10-18T11:42:01.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06d4ff9640 1 -- 172.21.15.125:0/3356197866 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f06d0004540 con 0x7f06d8141690 2023-10-18T11:42:01.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d813b030 con 0x7f06d8141690 2023-10-18T11:42:01.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06d4ff9640 1 -- 172.21.15.125:0/3356197866 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0003e10 con 0x7f06d8141690 2023-10-18T11:42:01.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.744+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d813b5f0 con 0x7f06d8141690 2023-10-18T11:42:01.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.745+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d8070f10 con 0x7f06d8141690 2023-10-18T11:42:01.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.746+0000 7f06d4ff9640 1 -- 172.21.15.125:0/3356197866 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 98) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f06d000a040 con 0x7f06d8141690 2023-10-18T11:42:01.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.746+0000 7f06d4ff9640 1 -- 172.21.15.125:0/3356197866 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f06d00ac1e0 con 0x7f06d8141690 2023-10-18T11:42:01.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.750+0000 7f06d4ff9640 1 -- 172.21.15.125:0/3356197866 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06d0077380 con 0x7f06d8141690 2023-10-18T11:42:01.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.953+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06d80690e0 con 0x7f06d8141690 2023-10-18T11:42:01.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.954+0000 7f06d4ff9640 1 -- 172.21.15.125:0/3356197866 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f06d007a9a0 con 0x7f06d8141690 2023-10-18T11:42:01.957 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:01.957 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":67,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3883749389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3883749389},{"type":"v1","addr":"172.21.15.139:6836","nonce":3883749389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2038008432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2038008432},{"type":"v1","addr":"172.21.15.139:6837","nonce":2038008432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":10317,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1543508506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1543508506},{"type":"v1","addr":"172.21.15.125:6834","nonce":1543508506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:41:38.149238+0000","modified":"2023-10-18T11:41:40.479014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10320":{"gid":10320,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2949284679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2949284679},{"type":"v1","addr":"172.21.15.125:6835","nonce":2949284679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T11:42:01.957 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 67 2023-10-18T11:42:01.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d8141690 msgr2=0x7f06d813cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:01.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d8141690 0x7f06d813cb20 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009d60 tx=0x7f06d0003c20 comp rx=0 tx=0).stop 2023-10-18T11:42:01.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 shutdown_connections 2023-10-18T11:42:01.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06d8141690 0x7f06d813cb20 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06d813e520 0x7f06d813a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 --2- 172.21.15.125:0/3356197866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d806c0a0 0x7f06d8139ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:01.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 >> 172.21.15.125:0/3356197866 conn(0x7f06d8069930 msgr2=0x7f06d8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:01.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 shutdown_connections 2023-10-18T11:42:01.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:01.956+0000 7f06dea32640 1 -- 172.21.15.125:0/3356197866 wait complete. 2023-10-18T11:42:01.969 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:42:02.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.069+0000 7f9fee7e7640 1 -- 172.21.15.125:0/2678318802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 msgr2=0x7f9fe80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.069+0000 7f9fee7e7640 1 --2- 172.21.15.125:0/2678318802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 0x7f9fe80705a0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8005fd0 tx=0x7f9fd802be10 comp rx=0 tx=0).stop 2023-10-18T11:42:02.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.069+0000 7f9fee7e7640 1 -- 172.21.15.125:0/2678318802 shutdown_connections 2023-10-18T11:42:02.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.069+0000 7f9fee7e7640 1 --2- 172.21.15.125:0/2678318802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 0x7f9fe80705a0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.069+0000 7f9fee7e7640 1 --2- 172.21.15.125:0/2678318802 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe80cbd20 0x7f9fe8158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.069+0000 7f9fee7e7640 1 -- 172.21.15.125:0/2678318802 >> 172.21.15.125:0/2678318802 conn(0x7f9fe8069930 msgr2=0x7f9fe8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:02.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.070+0000 7f9fee7e7640 1 -- 172.21.15.125:0/2678318802 shutdown_connections 2023-10-18T11:42:02.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.070+0000 7f9fee7e7640 1 -- 172.21.15.125:0/2678318802 wait complete. 2023-10-18T11:42:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.071+0000 7f9fee7e7640 1 Processor -- start 2023-10-18T11:42:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.071+0000 7f9fee7e7640 1 -- start start 2023-10-18T11:42:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fee7e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 0x7f9fe80abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fee7e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe80b0530 0x7f9fe81e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fee7e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fe80cbd20 0x7f9fe80adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fee7e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80ac320 con 0x7f9fe80afa20 2023-10-18T11:42:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fee7e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8158a20 con 0x7f9fe80cbd20 2023-10-18T11:42:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fee7e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8158d00 con 0x7f9fe80b0530 2023-10-18T11:42:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 0x7f9fe80abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 0x7f9fe80abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35204/0 (socket says 172.21.15.125:35204) 2023-10-18T11:42:02.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 -- 172.21.15.125:0/1750230878 learned_addr learned my addr 172.21.15.125:0/1750230878 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe77fe640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe80b0530 0x7f9fe81e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fecd84640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fe80cbd20 0x7f9fe80adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 -- 172.21.15.125:0/1750230878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe80b0530 msgr2=0x7f9fe81e5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe80b0530 0x7f9fe81e5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 -- 172.21.15.125:0/1750230878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fe80cbd20 msgr2=0x7f9fe80adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fe80cbd20 0x7f9fe80adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 -- 172.21.15.125:0/1750230878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8009560 con 0x7f9fe80afa20 2023-10-18T11:42:02.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe7fff640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 0x7f9fe80abde0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00d750 tx=0x7f9fdc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:02.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe4ff9640 1 -- 172.21.15.125:0/1750230878 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc004220 con 0x7f9fe80afa20 2023-10-18T11:42:02.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe4ff9640 1 -- 172.21.15.125:0/1750230878 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9fdc004470 con 0x7f9fe80afa20 2023-10-18T11:42:02.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe8070cf0 con 0x7f9fe80afa20 2023-10-18T11:42:02.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fe4ff9640 1 -- 172.21.15.125:0/1750230878 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc009c30 con 0x7f9fe80afa20 2023-10-18T11:42:02.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.072+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe81592c0 con 0x7f9fe80afa20 2023-10-18T11:42:02.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.073+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe80690e0 con 0x7f9fe80afa20 2023-10-18T11:42:02.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.074+0000 7f9fe4ff9640 1 -- 172.21.15.125:0/1750230878 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 98) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f9fdc005020 con 0x7f9fe80afa20 2023-10-18T11:42:02.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.074+0000 7f9fe4ff9640 1 -- 172.21.15.125:0/1750230878 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f9fdc089730 con 0x7f9fe80afa20 2023-10-18T11:42:02.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.078+0000 7f9fe4ff9640 1 -- 172.21.15.125:0/1750230878 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9fdc0548d0 con 0x7f9fe80afa20 2023-10-18T11:42:02.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.247+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fe80cc1e0 con 0x7f9fe80afa20 2023-10-18T11:42:02.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.248+0000 7f9fe4ff9640 1 -- 172.21.15.125:0/1750230878 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+17831 (secure 0 0 0) 0x7f9fdc017020 con 0x7f9fe80afa20 2023-10-18T11:42:02.251 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:02.251 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":176,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:42:01.439741+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:41:35.970906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-18T11:41:37.131199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6832/1922997989":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2170035400":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/153806259":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2991312240":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/767536410":"2023-10-19T11:42:01.439709+0000","172.21.15.139:6833/1922997989":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2776413208":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/3094960806":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/139213704":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4234510722":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4113808248":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6833/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/312743033":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6832/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/1067360436":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/1471732770":"2023-10-19T11:41:55.015024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:42:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 msgr2=0x7f9fe80abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 0x7f9fe80abde0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00d750 tx=0x7f9fdc00dc10 comp rx=0 tx=0).stop 2023-10-18T11:42:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 shutdown_connections 2023-10-18T11:42:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9fe80cbd20 0x7f9fe80adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe80b0530 0x7f9fe81e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 --2- 172.21.15.125:0/1750230878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe80afa20 0x7f9fe80abde0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 >> 172.21.15.125:0/1750230878 conn(0x7f9fe8069930 msgr2=0x7f9fe80ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:02.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 shutdown_connections 2023-10-18T11:42:02.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.250+0000 7f9fee7e7640 1 -- 172.21.15.125:0/1750230878 wait complete. 2023-10-18T11:42:02.263 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T11:42:02.264 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:42:02.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.364+0000 7fe0220e0640 1 -- 172.21.15.125:0/3372331774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe004004d00 msgr2=0x7fe004005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.364+0000 7fe0220e0640 1 --2- 172.21.15.125:0/3372331774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe004004d00 0x7fe004005180 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fe014005fd0 tx=0x7fe0040057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.364+0000 7fe01bfff640 1 -- 172.21.15.125:0/3372331774 reap_dead start 2023-10-18T11:42:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.364+0000 7fe0220e0640 1 -- 172.21.15.125:0/3372331774 shutdown_connections 2023-10-18T11:42:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.364+0000 7fe0220e0640 1 -- 172.21.15.125:0/3372331774 >> 172.21.15.125:0/3372331774 conn(0x7fe01c069930 msgr2=0x7fe01c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.364+0000 7fe0220e0640 1 -- 172.21.15.125:0/3372331774 shutdown_connections 2023-10-18T11:42:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.364+0000 7fe0220e0640 1 -- 172.21.15.125:0/3372331774 wait complete. 2023-10-18T11:42:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.366+0000 7fe0220e0640 1 Processor -- start 2023-10-18T11:42:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.366+0000 7fe0220e0640 1 -- start start 2023-10-18T11:42:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.366+0000 7fe0220e0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe01c139bb0 0x7fe01c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.366+0000 7fe0220e0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c1541e0 0x7fe01c155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.366+0000 7fe0220e0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15a220 0x7fe01c15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.366+0000 7fe0220e0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c15abe0 con 0x7fe01c1541e0 2023-10-18T11:42:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.366+0000 7fe0220e0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c13a4e0 con 0x7fe01c15a220 2023-10-18T11:42:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.366+0000 7fe0220e0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c158c20 con 0x7fe01c139bb0 2023-10-18T11:42:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c1541e0 0x7fe01c155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c1541e0 0x7fe01c155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35224/0 (socket says 172.21.15.125:35224) 2023-10-18T11:42:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 -- 172.21.15.125:0/2952794342 learned_addr learned my addr 172.21.15.125:0/2952794342 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01bfff640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15a220 0x7fe01c15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01b7fe640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe01c139bb0 0x7fe01c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 -- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe01c139bb0 msgr2=0x7fe01c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe01c139bb0 0x7fe01c1586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 -- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15a220 msgr2=0x7fe01c15a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15a220 0x7fe01c15a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 -- 172.21.15.125:0/2952794342 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c139f90 con 0x7fe01c1541e0 2023-10-18T11:42:02.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe01affd640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c1541e0 0x7fe01c155d40 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fe010011a80 tx=0x7fe010011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:02.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe003fff640 1 -- 172.21.15.125:0/2952794342 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000fcf0 con 0x7fe01c1541e0 2023-10-18T11:42:02.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe003fff640 1 -- 172.21.15.125:0/2952794342 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe010012450 con 0x7fe01c1541e0 2023-10-18T11:42:02.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c154730 con 0x7fe01c1541e0 2023-10-18T11:42:02.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe003fff640 1 -- 172.21.15.125:0/2952794342 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010012800 con 0x7fe01c1541e0 2023-10-18T11:42:02.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.367+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c0a9f00 con 0x7fe01c1541e0 2023-10-18T11:42:02.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.368+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c0cb1f0 con 0x7fe01c1541e0 2023-10-18T11:42:02.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.369+0000 7fe003fff640 1 -- 172.21.15.125:0/2952794342 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 98) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe010012960 con 0x7fe01c1541e0 2023-10-18T11:42:02.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.369+0000 7fe003fff640 1 -- 172.21.15.125:0/2952794342 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6745+0+0 (secure 0 0 0) 0x7fe01008a310 con 0x7fe01c1541e0 2023-10-18T11:42:02.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.373+0000 7fe003fff640 1 -- 172.21.15.125:0/2952794342 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe010055400 con 0x7fe01c1541e0 2023-10-18T11:42:02.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.449+0000 7fe003fff640 1 -- 172.21.15.125:0/2952794342 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 99) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe010054eb0 con 0x7fe01c1541e0 2023-10-18T11:42:02.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.449+0000 7fe003fff640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe008070b80 0x7fe008072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.452+0000 7fe01b7fe640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe008070b80 0x7fe008072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.453+0000 7fe01b7fe640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe008070b80 0x7fe008072f70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe00c001050 tx=0x7fe00c0035f0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:02.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.578+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe01c06c570 con 0x7fe01c1541e0 2023-10-18T11:42:02.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.578+0000 7fe003fff640 1 -- 172.21.15.125:0/2952794342 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4933 (secure 0 0 0) 0x7fe010086fb0 con 0x7fe01c1541e0 2023-10-18T11:42:02.581 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:02.581 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":67,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3883749389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3883749389},{"type":"v1","addr":"172.21.15.139:6836","nonce":3883749389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2038008432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2038008432},{"type":"v1","addr":"172.21.15.139:6837","nonce":2038008432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":10317,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1543508506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1543508506},{"type":"v1","addr":"172.21.15.125:6834","nonce":1543508506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:41:38.149238+0000","modified":"2023-10-18T11:41:40.479014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10320":{"gid":10320,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2949284679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2949284679},{"type":"v1","addr":"172.21.15.125:6835","nonce":2949284679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T11:42:02.583 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 67 2023-10-18T11:42:02.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe008070b80 msgr2=0x7fe008072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe008070b80 0x7fe008072f70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe00c001050 tx=0x7fe00c0035f0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c1541e0 msgr2=0x7fe01c155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c1541e0 0x7fe01c155d40 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fe010011a80 tx=0x7fe010011f40 comp rx=0 tx=0).stop 2023-10-18T11:42:02.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 shutdown_connections 2023-10-18T11:42:02.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe008070b80 0x7fe008072f70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15a220 0x7fe01c15a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe01c1541e0 0x7fe01c155d40 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 --2- 172.21.15.125:0/2952794342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe01c139bb0 0x7fe01c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 >> 172.21.15.125:0/2952794342 conn(0x7fe01c069930 msgr2=0x7fe01c0ab9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:02.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 shutdown_connections 2023-10-18T11:42:02.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.580+0000 7fe0220e0640 1 -- 172.21.15.125:0/2952794342 wait complete. 2023-10-18T11:42:02.593 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:42:02.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.692+0000 7f67eef55640 1 -- 172.21.15.125:0/1401029256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e806b030 msgr2=0x7f67e806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.692+0000 7f67eef55640 1 --2- 172.21.15.125:0/1401029256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e806b030 0x7f67e806b410 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f67e80cb9b0 tx=0x7f67d802bdb0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.692+0000 7f67eef55640 1 -- 172.21.15.125:0/1401029256 shutdown_connections 2023-10-18T11:42:02.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.692+0000 7f67eef55640 1 --2- 172.21.15.125:0/1401029256 >> v2:172.21.15.139:3300/0 conn(0x7f67e8159bd0 0x7f67e815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.692+0000 7f67eef55640 1 --2- 172.21.15.125:0/1401029256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e806b030 0x7f67e806b410 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.692+0000 7f67eef55640 1 -- 172.21.15.125:0/1401029256 >> 172.21.15.125:0/1401029256 conn(0x7f67e8069930 msgr2=0x7f67e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:02.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.692+0000 7f67eef55640 1 -- 172.21.15.125:0/1401029256 shutdown_connections 2023-10-18T11:42:02.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.692+0000 7f67eef55640 1 -- 172.21.15.125:0/1401029256 wait complete. 2023-10-18T11:42:02.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.693+0000 7f67eef55640 1 Processor -- start 2023-10-18T11:42:02.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.693+0000 7f67eef55640 1 -- start start 2023-10-18T11:42:02.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67eef55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e806b030 0x7f67e80aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67eef55640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e8159bd0 0x7f67e81f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67eef55640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e815ccb0 0x7f67e8157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67eef55640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e80ab170 con 0x7f67e815ccb0 2023-10-18T11:42:02.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67eef55640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81f1750 con 0x7f67e806b030 2023-10-18T11:42:02.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67eef55640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81f1a30 con 0x7f67e8159bd0 2023-10-18T11:42:02.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67ed4f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e815ccb0 0x7f67e8157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67ed4f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e815ccb0 0x7f67e8157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35244/0 (socket says 172.21.15.125:35244) 2023-10-18T11:42:02.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67ed4f2640 1 -- 172.21.15.125:0/1302494110 learned_addr learned my addr 172.21.15.125:0/1302494110 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:02.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67ed4f2640 1 -- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e8159bd0 msgr2=0x7f67e81f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67e7fff640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e8159bd0 0x7f67e81f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67ed4f2640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e8159bd0 0x7f67e81f0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67ed4f2640 1 -- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e806b030 msgr2=0x7f67e80aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67eccf1640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e806b030 0x7f67e80aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67ed4f2640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e806b030 0x7f67e80aaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.694+0000 7f67ed4f2640 1 -- 172.21.15.125:0/1302494110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8009560 con 0x7f67e815ccb0 2023-10-18T11:42:02.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.695+0000 7f67ed4f2640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e815ccb0 0x7f67e8157c50 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f67e000c730 tx=0x7f67e000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:02.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.695+0000 7f67e57fa640 1 -- 172.21.15.125:0/1302494110 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0006d50 con 0x7f67e815ccb0 2023-10-18T11:42:02.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.695+0000 7f67e57fa640 1 -- 172.21.15.125:0/1302494110 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f67e0003970 con 0x7f67e815ccb0 2023-10-18T11:42:02.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.695+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e81f1d10 con 0x7f67e815ccb0 2023-10-18T11:42:02.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.695+0000 7f67e57fa640 1 -- 172.21.15.125:0/1302494110 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0003d80 con 0x7f67e815ccb0 2023-10-18T11:42:02.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.695+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e81f1300 con 0x7f67e815ccb0 2023-10-18T11:42:02.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.696+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e81f1580 con 0x7f67e815ccb0 2023-10-18T11:42:02.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.696+0000 7f67e57fa640 1 -- 172.21.15.125:0/1302494110 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 99) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f67e00050d0 con 0x7f67e815ccb0 2023-10-18T11:42:02.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.697+0000 7f67e57fa640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f67d406ac90 0x7f67d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:02.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.697+0000 7f67e57fa640 1 -- 172.21.15.125:0/1302494110 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f67e00893f0 con 0x7f67e815ccb0 2023-10-18T11:42:02.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.697+0000 7f67eccf1640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f67d406ac90 0x7f67d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:02.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.697+0000 7f67eccf1640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f67d406ac90 0x7f67d406d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67d8035000 tx=0x7f67d8004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:02.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.701+0000 7f67e57fa640 1 -- 172.21.15.125:0/1302494110 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67e00543d0 con 0x7f67e815ccb0 2023-10-18T11:42:02.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.904+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67e8069f80 con 0x7f67e815ccb0 2023-10-18T11:42:02.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.905+0000 7f67e57fa640 1 -- 172.21.15.125:0/1302494110 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f67e00579f0 con 0x7f67e815ccb0 2023-10-18T11:42:02.907 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:02.908 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":67,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3883749389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3883749389},{"type":"v1","addr":"172.21.15.139:6836","nonce":3883749389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2038008432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2038008432},{"type":"v1","addr":"172.21.15.139:6837","nonce":2038008432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":10317,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1543508506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1543508506},{"type":"v1","addr":"172.21.15.125:6834","nonce":1543508506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:41:38.149238+0000","modified":"2023-10-18T11:41:40.479014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10320":{"gid":10320,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/2949284679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2949284679},{"type":"v1","addr":"172.21.15.125:6835","nonce":2949284679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T11:42:02.908 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 67 2023-10-18T11:42:02.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.906+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f67d406ac90 msgr2=0x7f67d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.906+0000 7f67eef55640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f67d406ac90 0x7f67d406d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67d8035000 tx=0x7f67d8004730 comp rx=0 tx=0).stop 2023-10-18T11:42:02.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.906+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e815ccb0 msgr2=0x7f67e8157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:02.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.906+0000 7f67eef55640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e815ccb0 0x7f67e8157c50 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f67e000c730 tx=0x7f67e000cb30 comp rx=0 tx=0).stop 2023-10-18T11:42:02.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.907+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 shutdown_connections 2023-10-18T11:42:02.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.907+0000 7f67eef55640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f67d406ac90 0x7f67d406d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.907+0000 7f67eef55640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e815ccb0 0x7f67e8157c50 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.907+0000 7f67eef55640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67e8159bd0 0x7f67e81f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.907+0000 7f67eef55640 1 --2- 172.21.15.125:0/1302494110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67e806b030 0x7f67e80aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.907+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 >> 172.21.15.125:0/1302494110 conn(0x7f67e8069930 msgr2=0x7f67e815c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.907+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 shutdown_connections 2023-10-18T11:42:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:02.907+0000 7f67eef55640 1 -- 172.21.15.125:0/1302494110 wait complete. 2023-10-18T11:42:02.919 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:42:03.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.019+0000 7f22f17c5640 1 -- 172.21.15.125:0/790659156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec156790 msgr2=0x7f22ec0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.019+0000 7f22f17c5640 1 --2- 172.21.15.125:0/790659156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec156790 0x7f22ec0cc990 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f22ec061510 tx=0x7f22dc02be20 comp rx=0 tx=0).stop 2023-10-18T11:42:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.019+0000 7f22f17c5640 1 -- 172.21.15.125:0/790659156 shutdown_connections 2023-10-18T11:42:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.019+0000 7f22f17c5640 1 --2- 172.21.15.125:0/790659156 >> v2:172.21.15.139:3300/0 conn(0x7f22ec0ac870 0x7f22ec06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.019+0000 7f22f17c5640 1 --2- 172.21.15.125:0/790659156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec156790 0x7f22ec0cc990 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.019+0000 7f22f17c5640 1 -- 172.21.15.125:0/790659156 >> 172.21.15.125:0/790659156 conn(0x7f22ec06fae0 msgr2=0x7f22ec06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:03.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.019+0000 7f22f17c5640 1 -- 172.21.15.125:0/790659156 shutdown_connections 2023-10-18T11:42:03.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.019+0000 7f22f17c5640 1 -- 172.21.15.125:0/790659156 wait complete. 2023-10-18T11:42:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.020+0000 7f22f17c5640 1 Processor -- start 2023-10-18T11:42:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.020+0000 7f22f17c5640 1 -- start start 2023-10-18T11:42:03.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22f17c5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06bec0 0x7f22ec0a7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22f17c5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec0ac870 0x7f22ec06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22f17c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec156790 0x7f22ec146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22f17c5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec15cee0 con 0x7f22ec0ac870 2023-10-18T11:42:03.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22f17c5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec15c2d0 con 0x7f22ec156790 2023-10-18T11:42:03.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22f17c5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec15c5b0 con 0x7f22ec06bec0 2023-10-18T11:42:03.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec0ac870 0x7f22ec06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec0ac870 0x7f22ec06c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35268/0 (socket says 172.21.15.125:35268) 2023-10-18T11:42:03.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 -- 172.21.15.125:0/3255858431 learned_addr learned my addr 172.21.15.125:0/3255858431 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:03.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22eaffd640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06bec0 0x7f22ec0a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 -- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06bec0 msgr2=0x7f22ec0a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06bec0 0x7f22ec0a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22eb7fe640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec156790 0x7f22ec146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 -- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec156790 msgr2=0x7f22ec146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec156790 0x7f22ec146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 -- 172.21.15.125:0/3255858431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc009560 con 0x7f22ec0ac870 2023-10-18T11:42:03.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22eaffd640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06bec0 0x7f22ec0a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:03.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22eb7fe640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec156790 0x7f22ec146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:03.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.021+0000 7f22ea7fc640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec0ac870 0x7f22ec06c560 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c890 tx=0x7f22e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:03.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.022+0000 7f22d37fe640 1 -- 172.21.15.125:0/3255858431 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0004220 con 0x7f22ec0ac870 2023-10-18T11:42:03.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.022+0000 7f22d37fe640 1 -- 172.21.15.125:0/3255858431 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f22e0004470 con 0x7f22ec0ac870 2023-10-18T11:42:03.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.022+0000 7f22d37fe640 1 -- 172.21.15.125:0/3255858431 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0007c10 con 0x7f22ec0ac870 2023-10-18T11:42:03.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.022+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec15c890 con 0x7f22ec0ac870 2023-10-18T11:42:03.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.022+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec06caa0 con 0x7f22ec0ac870 2023-10-18T11:42:03.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.023+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec0690e0 con 0x7f22ec0ac870 2023-10-18T11:42:03.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.023+0000 7f22d37fe640 1 -- 172.21.15.125:0/3255858431 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 99) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f22e0005020 con 0x7f22ec0ac870 2023-10-18T11:42:03.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.023+0000 7f22d37fe640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f22d806ac90 0x7f22d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.024+0000 7f22d37fe640 1 -- 172.21.15.125:0/3255858431 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f22e0089c60 con 0x7f22ec0ac870 2023-10-18T11:42:03.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.024+0000 7f22eaffd640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f22d806ac90 0x7f22d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.024+0000 7f22eaffd640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f22d806ac90 0x7f22d806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f22dc035000 tx=0x7f22dc004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:03.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.028+0000 7f22d37fe640 1 -- 172.21.15.125:0/3255858431 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22e0054bb0 con 0x7f22ec0ac870 2023-10-18T11:42:03.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.195+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22ec156320 con 0x7f22ec0ac870 2023-10-18T11:42:03.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.196+0000 7f22d37fe640 1 -- 172.21.15.125:0/3255858431 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+17831 (secure 0 0 0) 0x7f22e001d030 con 0x7f22ec0ac870 2023-10-18T11:42:03.199 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:03.199 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":176,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:42:01.439741+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:41:35.970906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-18T11:41:37.131199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6832/1922997989":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2170035400":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/153806259":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2991312240":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/767536410":"2023-10-19T11:42:01.439709+0000","172.21.15.139:6833/1922997989":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2776413208":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/3094960806":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/139213704":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4234510722":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4113808248":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6833/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/312743033":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6832/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/1067360436":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/1471732770":"2023-10-19T11:41:55.015024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:42:03.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f22d806ac90 msgr2=0x7f22d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f22d806ac90 0x7f22d806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f22dc035000 tx=0x7f22dc004730 comp rx=0 tx=0).stop 2023-10-18T11:42:03.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec0ac870 msgr2=0x7f22ec06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec0ac870 0x7f22ec06c560 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c890 tx=0x7f22e000cd50 comp rx=0 tx=0).stop 2023-10-18T11:42:03.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 shutdown_connections 2023-10-18T11:42:03.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f22d806ac90 0x7f22d806d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22ec156790 0x7f22ec146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22ec0ac870 0x7f22ec06c560 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 --2- 172.21.15.125:0/3255858431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22ec06bec0 0x7f22ec0a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 >> 172.21.15.125:0/3255858431 conn(0x7f22ec06fae0 msgr2=0x7f22ec154c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:03.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 shutdown_connections 2023-10-18T11:42:03.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.198+0000 7f22f17c5640 1 -- 172.21.15.125:0/3255858431 wait complete. 2023-10-18T11:42:03.211 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T11:42:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.311+0000 7f299e505640 1 -- 172.21.15.125:0/4251555003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f299815b2a0 msgr2=0x7f29980d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.311+0000 7f299e505640 1 --2- 172.21.15.125:0/4251555003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f299815b2a0 0x7f29980d1910 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f298c005fd0 tx=0x7f298c02bd90 comp rx=0 tx=0).stop 2023-10-18T11:42:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.311+0000 7f299e505640 1 -- 172.21.15.125:0/4251555003 shutdown_connections 2023-10-18T11:42:03.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.311+0000 7f299e505640 1 --2- 172.21.15.125:0/4251555003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29980ceb40 0x7f29980b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.311+0000 7f299e505640 1 --2- 172.21.15.125:0/4251555003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f299815b2a0 0x7f29980d1910 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.311+0000 7f299e505640 1 -- 172.21.15.125:0/4251555003 >> 172.21.15.125:0/4251555003 conn(0x7f29980d0d60 msgr2=0x7f29980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.311+0000 7f299e505640 1 -- 172.21.15.125:0/4251555003 shutdown_connections 2023-10-18T11:42:03.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.311+0000 7f299e505640 1 -- 172.21.15.125:0/4251555003 wait complete. 2023-10-18T11:42:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.312+0000 7f299e505640 1 Processor -- start 2023-10-18T11:42:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.312+0000 7f299e505640 1 -- start start 2023-10-18T11:42:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f299e505640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29980ceb40 0x7f29981f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f299e505640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29980cf340 0x7f299815e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f299e505640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299815b2a0 0x7f29980b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f299e505640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29980b3ca0 con 0x7f29980ceb40 2023-10-18T11:42:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f299e505640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29980b3f80 con 0x7f299815b2a0 2023-10-18T11:42:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f299e505640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29981f4d90 con 0x7f29980cf340 2023-10-18T11:42:03.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f2997fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29980ceb40 0x7f29981f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f2997fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29980ceb40 0x7f29981f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35278/0 (socket says 172.21.15.125:35278) 2023-10-18T11:42:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f2997fff640 1 -- 172.21.15.125:0/3302058102 learned_addr learned my addr 172.21.15.125:0/3302058102 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.313+0000 7f2997fff640 1 -- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29980cf340 msgr2=0x7f299815e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f2997fff640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29980cf340 0x7f299815e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f2997fff640 1 -- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299815b2a0 msgr2=0x7f29980b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f2997fff640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299815b2a0 0x7f29980b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f2997fff640 1 -- 172.21.15.125:0/3302058102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c009560 con 0x7f29980ceb40 2023-10-18T11:42:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f2997fff640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29980ceb40 0x7f29981f5870 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f298800c400 tx=0x7f298800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f2994ff9640 1 -- 172.21.15.125:0/3302058102 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29880052d0 con 0x7f29980ceb40 2023-10-18T11:42:03.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f2994ff9640 1 -- 172.21.15.125:0/3302058102 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2988007020 con 0x7f29980ceb40 2023-10-18T11:42:03.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29980b0d50 con 0x7f29980ceb40 2023-10-18T11:42:03.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f2994ff9640 1 -- 172.21.15.125:0/3302058102 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988007470 con 0x7f29980ceb40 2023-10-18T11:42:03.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.314+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29981f5350 con 0x7f29980ceb40 2023-10-18T11:42:03.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.315+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29980690e0 con 0x7f29980ceb40 2023-10-18T11:42:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.315+0000 7f2994ff9640 1 -- 172.21.15.125:0/3302058102 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 99) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f29880067d0 con 0x7f29980ceb40 2023-10-18T11:42:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.316+0000 7f2994ff9640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f298406acc0 0x7f298406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.316+0000 7f2994ff9640 1 -- 172.21.15.125:0/3302058102 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f2988088d40 con 0x7f29980ceb40 2023-10-18T11:42:03.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.316+0000 7f29977fe640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f298406acc0 0x7f298406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.316+0000 7f29977fe640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f298406acc0 0x7f298406d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f298c035000 tx=0x7f298c0046c0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:03.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.320+0000 7f2994ff9640 1 -- 172.21.15.125:0/3302058102 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2988053ce0 con 0x7f29980ceb40 2023-10-18T11:42:03.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.523+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f29980d0b40 con 0x7f29980ceb40 2023-10-18T11:42:03.525 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T11:42:03.523+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T11:42:03.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.524+0000 7f2994ff9640 1 -- 172.21.15.125:0/3302058102 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2988057300 con 0x7f29980ceb40 2023-10-18T11:42:03.527 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T11:42:03.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f298406acc0 msgr2=0x7f298406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f298406acc0 0x7f298406d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f298c035000 tx=0x7f298c0046c0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29980ceb40 msgr2=0x7f29981f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29980ceb40 0x7f29981f5870 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f298800c400 tx=0x7f298800c8c0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 shutdown_connections 2023-10-18T11:42:03.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f298406acc0 0x7f298406d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f299815b2a0 0x7f29980b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29980cf340 0x7f299815e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 --2- 172.21.15.125:0/3302058102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29980ceb40 0x7f29981f5870 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 >> 172.21.15.125:0/3302058102 conn(0x7f29980d0d60 msgr2=0x7f29980ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:03.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 shutdown_connections 2023-10-18T11:42:03.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.527+0000 7f299e505640 1 -- 172.21.15.125:0/3302058102 wait complete. 2023-10-18T11:42:03.532 INFO:tasks.ceph.mds.a.smithi125.stderr: -3017> 2023-10-18T11:41:43.794+0000 7f5b41f3a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T11:42:03.543 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T11:42:03.561 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T11:42:03.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 -- 172.21.15.125:0/3328503372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a80ceb40 msgr2=0x7f77a80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 --2- 172.21.15.125:0/3328503372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a80ceb40 0x7f77a80b0650 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f77a806c510 tx=0x7f77a002c1d0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 -- 172.21.15.125:0/3328503372 shutdown_connections 2023-10-18T11:42:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 --2- 172.21.15.125:0/3328503372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a80ceb40 0x7f77a80b0650 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 --2- 172.21.15.125:0/3328503372 >> v2:172.21.15.139:3300/0 conn(0x7f77a815b2a0 0x7f77a80cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 --2- 172.21.15.125:0/3328503372 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77a80cf340 0x7f77a80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 -- 172.21.15.125:0/3328503372 >> 172.21.15.125:0/3328503372 conn(0x7f77a80d0d60 msgr2=0x7f77a80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 -- 172.21.15.125:0/3328503372 shutdown_connections 2023-10-18T11:42:03.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.646+0000 7f77add25640 1 -- 172.21.15.125:0/3328503372 wait complete. 2023-10-18T11:42:03.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.647+0000 7f77add25640 1 Processor -- start 2023-10-18T11:42:03.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.647+0000 7f77add25640 1 -- start start 2023-10-18T11:42:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77add25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77a80ceb40 0x7f77a8157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77add25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77a80cf340 0x7f77a81e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77add25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a815b2a0 0x7f77a80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77add25640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8158480 con 0x7f77a815b2a0 2023-10-18T11:42:03.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77add25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80c7e30 con 0x7f77a80ceb40 2023-10-18T11:42:03.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77add25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80c8110 con 0x7f77a80cf340 2023-10-18T11:42:03.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77a7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a815b2a0 0x7f77a80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77a7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a815b2a0 0x7f77a80b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35318/0 (socket says 172.21.15.125:35318) 2023-10-18T11:42:03.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77a7fff640 1 -- 172.21.15.125:0/1048313779 learned_addr learned my addr 172.21.15.125:0/1048313779 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:03.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f77a77fe640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77a80ceb40 0x7f77a8157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.648+0000 7f779effd640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77a80cf340 0x7f77a81e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a7fff640 1 -- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77a80cf340 msgr2=0x7f77a81e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a7fff640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77a80cf340 0x7f77a81e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a7fff640 1 -- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77a80ceb40 msgr2=0x7f77a8157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:03.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a7fff640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77a80ceb40 0x7f77a8157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:03.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a7fff640 1 -- 172.21.15.125:0/1048313779 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0009900 con 0x7f77a815b2a0 2023-10-18T11:42:03.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a7fff640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a815b2a0 0x7f77a80b3950 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009d60 tx=0x7f77a0035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:03.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a4ff9640 1 -- 172.21.15.125:0/1048313779 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0034440 con 0x7f77a815b2a0 2023-10-18T11:42:03.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a4ff9640 1 -- 172.21.15.125:0/1048313779 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f77a0004540 con 0x7f77a815b2a0 2023-10-18T11:42:03.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a80c83f0 con 0x7f77a815b2a0 2023-10-18T11:42:03.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77a4ff9640 1 -- 172.21.15.125:0/1048313779 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0040450 con 0x7f77a815b2a0 2023-10-18T11:42:03.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.649+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a80c89b0 con 0x7f77a815b2a0 2023-10-18T11:42:03.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.650+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a80ad950 con 0x7f77a815b2a0 2023-10-18T11:42:03.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.651+0000 7f77a4ff9640 1 -- 172.21.15.125:0/1048313779 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 99) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f77a000a040 con 0x7f77a815b2a0 2023-10-18T11:42:03.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.651+0000 7f77a4ff9640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f778806ac90 0x7f778806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:03.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.651+0000 7f77a4ff9640 1 -- 172.21.15.125:0/1048313779 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f77a00aca00 con 0x7f77a815b2a0 2023-10-18T11:42:03.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.651+0000 7f77a77fe640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f778806ac90 0x7f778806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:03.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.652+0000 7f77a77fe640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f778806ac90 0x7f778806d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7798005910 tx=0x7f77980058a0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:03.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.655+0000 7f77a4ff9640 1 -- 172.21.15.125:0/1048313779 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77a0077a40 con 0x7f77a815b2a0 2023-10-18T11:42:03.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:03.857+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f77a80690e0 con 0x7f77a815b2a0 2023-10-18T11:42:04.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.458+0000 7f77a4ff9640 1 -- 172.21.15.125:0/1048313779 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 100) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f77a0077e10 con 0x7f77a815b2a0 2023-10-18T11:42:04.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.526+0000 7f77a4ff9640 1 -- 172.21.15.125:0/1048313779 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7f77a008f3a0 con 0x7f77a815b2a0 2023-10-18T11:42:04.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f778806ac90 msgr2=0x7f778806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:04.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f778806ac90 0x7f778806d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7798005910 tx=0x7f77980058a0 comp rx=0 tx=0).stop 2023-10-18T11:42:04.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a815b2a0 msgr2=0x7f77a80b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:04.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a815b2a0 0x7f77a80b3950 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009d60 tx=0x7f77a0035e30 comp rx=0 tx=0).stop 2023-10-18T11:42:04.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 shutdown_connections 2023-10-18T11:42:04.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f778806ac90 0x7f778806d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:04.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77a815b2a0 0x7f77a80b3950 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:04.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77a80cf340 0x7f77a81e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:04.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 --2- 172.21.15.125:0/1048313779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77a80ceb40 0x7f77a8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:04.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 >> 172.21.15.125:0/1048313779 conn(0x7f77a80d0d60 msgr2=0x7f77a815a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:04.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 shutdown_connections 2023-10-18T11:42:04.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.528+0000 7f77add25640 1 -- 172.21.15.125:0/1048313779 wait complete. 2023-10-18T11:42:04.542 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T11:42:04.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.689+0000 7fcb6fb05640 1 -- 172.21.15.125:0/381712661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb54004d00 msgr2=0x7fcb54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:04.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.689+0000 7fcb6fb05640 1 --2- 172.21.15.125:0/381712661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb54004d00 0x7fcb54005180 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000c290 tx=0x7fcb540057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:04.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.689+0000 7fcb6e0a2640 1 -- 172.21.15.125:0/381712661 reap_dead start 2023-10-18T11:42:04.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.689+0000 7fcb6fb05640 1 -- 172.21.15.125:0/381712661 shutdown_connections 2023-10-18T11:42:04.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.689+0000 7fcb6fb05640 1 -- 172.21.15.125:0/381712661 >> 172.21.15.125:0/381712661 conn(0x7fcb680d0d60 msgr2=0x7fcb680d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:04.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.689+0000 7fcb6fb05640 1 -- 172.21.15.125:0/381712661 shutdown_connections 2023-10-18T11:42:04.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.689+0000 7fcb6fb05640 1 -- 172.21.15.125:0/381712661 wait complete. 2023-10-18T11:42:04.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.690+0000 7fcb6fb05640 1 Processor -- start 2023-10-18T11:42:04.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.690+0000 7fcb6fb05640 1 -- start start 2023-10-18T11:42:04.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6fb05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb6806ae00 0x7fcb6815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:04.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6fb05640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb680c8840 0x7fcb68158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:04.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6fb05640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb6806b1e0 0x7fcb68158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:04.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6fb05640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680c99c0 con 0x7fcb680c8840 2023-10-18T11:42:04.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6fb05640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680c8d40 con 0x7fcb6806ae00 2023-10-18T11:42:04.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6fb05640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680c9020 con 0x7fcb6806b1e0 2023-10-18T11:42:04.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb680c8840 0x7fcb68158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:04.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb680c8840 0x7fcb68158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35344/0 (socket says 172.21.15.125:35344) 2023-10-18T11:42:04.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 -- 172.21.15.125:0/3483978657 learned_addr learned my addr 172.21.15.125:0/3483978657 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:04.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d8a1640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb6806ae00 0x7fcb6815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:04.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6e0a2640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb6806b1e0 0x7fcb68158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:04.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 -- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb6806b1e0 msgr2=0x7fcb68158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:04.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb6806b1e0 0x7fcb68158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:04.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 -- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb6806ae00 msgr2=0x7fcb6815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:04.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb6806ae00 0x7fcb6815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:04.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 -- 172.21.15.125:0/3483978657 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680c9300 con 0x7fcb680c8840 2023-10-18T11:42:04.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6d0a0640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb680c8840 0x7fcb68158220 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fcb58011ae0 tx=0x7fcb58009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:04.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb5e7fc640 1 -- 172.21.15.125:0/3483978657 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800fe70 con 0x7fcb680c8840 2023-10-18T11:42:04.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb5e7fc640 1 -- 172.21.15.125:0/3483978657 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb58004050 con 0x7fcb680c8840 2023-10-18T11:42:04.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb680c95e0 con 0x7fcb680c8840 2023-10-18T11:42:04.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb5e7fc640 1 -- 172.21.15.125:0/3483978657 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58004460 con 0x7fcb680c8840 2023-10-18T11:42:04.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.691+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb680c9de0 con 0x7fcb680c8840 2023-10-18T11:42:04.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.692+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb680690e0 con 0x7fcb680c8840 2023-10-18T11:42:04.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.693+0000 7fcb5e7fc640 1 -- 172.21.15.125:0/3483978657 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 100) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fcb580035f0 con 0x7fcb680c8840 2023-10-18T11:42:04.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.693+0000 7fcb5e7fc640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fcb4c06aed0 0x7fcb4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:04.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.693+0000 7fcb5e7fc640 1 -- 172.21.15.125:0/3483978657 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fcb5808a8c0 con 0x7fcb680c8840 2023-10-18T11:42:04.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.693+0000 7fcb6d8a1640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fcb4c06aed0 0x7fcb4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:04.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.694+0000 7fcb6d8a1640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fcb4c06aed0 0x7fcb4c06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb64005fd0 tx=0x7fcb64009500 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:04.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.698+0000 7fcb5e7fc640 1 -- 172.21.15.125:0/3483978657 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb58055610 con 0x7fcb680c8840 2023-10-18T11:42:04.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:04.864+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcb680cb980 con 0x7fcb680c8840 2023-10-18T11:42:05.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.527+0000 7fcb5e7fc640 1 -- 172.21.15.125:0/3483978657 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v178) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcb5801e0d0 con 0x7fcb680c8840 2023-10-18T11:42:05.530 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T11:42:05.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fcb4c06aed0 msgr2=0x7fcb4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:05.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fcb4c06aed0 0x7fcb4c06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb64005fd0 tx=0x7fcb64009500 comp rx=0 tx=0).stop 2023-10-18T11:42:05.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb680c8840 msgr2=0x7fcb68158220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:05.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb680c8840 0x7fcb68158220 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fcb58011ae0 tx=0x7fcb58009e90 comp rx=0 tx=0).stop 2023-10-18T11:42:05.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 shutdown_connections 2023-10-18T11:42:05.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fcb4c06aed0 0x7fcb4c06d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb6806b1e0 0x7fcb68158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb680c8840 0x7fcb68158220 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 --2- 172.21.15.125:0/3483978657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb6806ae00 0x7fcb6815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 >> 172.21.15.125:0/3483978657 conn(0x7fcb680d0d60 msgr2=0x7fcb680aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:05.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 shutdown_connections 2023-10-18T11:42:05.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.531+0000 7fcb6fb05640 1 -- 172.21.15.125:0/3483978657 wait complete. 2023-10-18T11:42:05.547 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T11:42:05.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 -- 172.21.15.125:0/349733352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af815b2a0 msgr2=0x7f2af80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:05.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 --2- 172.21.15.125:0/349733352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af815b2a0 0x7f2af80cf720 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f2af8089ae0 tx=0x7f2ae802bd10 comp rx=0 tx=0).stop 2023-10-18T11:42:05.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 -- 172.21.15.125:0/349733352 shutdown_connections 2023-10-18T11:42:05.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 --2- 172.21.15.125:0/349733352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af815b2a0 0x7f2af80cf720 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 --2- 172.21.15.125:0/349733352 >> v2:172.21.15.139:3300/0 conn(0x7f2af80ceb40 0x7f2af80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 --2- 172.21.15.125:0/349733352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2af80cf340 0x7f2af80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 -- 172.21.15.125:0/349733352 >> 172.21.15.125:0/349733352 conn(0x7f2af80d0d60 msgr2=0x7f2af80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:05.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 -- 172.21.15.125:0/349733352 shutdown_connections 2023-10-18T11:42:05.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.655+0000 7f2afff25640 1 -- 172.21.15.125:0/349733352 wait complete. 2023-10-18T11:42:05.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afff25640 1 Processor -- start 2023-10-18T11:42:05.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afff25640 1 -- start start 2023-10-18T11:42:05.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afff25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2af80ceb40 0x7f2af809e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:05.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afff25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af80cf340 0x7f2af81f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:05.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afff25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2af815b2a0 0x7f2af81f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:05.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afff25640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af814e930 con 0x7f2af80cf340 2023-10-18T11:42:05.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afff25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af814e000 con 0x7f2af80ceb40 2023-10-18T11:42:05.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afff25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af814e270 con 0x7f2af815b2a0 2023-10-18T11:42:05.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afd4c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af80cf340 0x7f2af81f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:05.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afd4c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af80cf340 0x7f2af81f0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35352/0 (socket says 172.21.15.125:35352) 2023-10-18T11:42:05.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.657+0000 7f2afd4c0640 1 -- 172.21.15.125:0/1308028704 learned_addr learned my addr 172.21.15.125:0/1308028704 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:05.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afd4c0640 1 -- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2af815b2a0 msgr2=0x7f2af81f0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:05.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afd4c0640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2af815b2a0 0x7f2af81f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afd4c0640 1 -- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2af80ceb40 msgr2=0x7f2af809e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:05.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afdcc1640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2af80ceb40 0x7f2af809e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:05.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afd4c0640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2af80ceb40 0x7f2af809e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:05.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afd4c0640 1 -- 172.21.15.125:0/1308028704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae8009560 con 0x7f2af80cf340 2023-10-18T11:42:05.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afdcc1640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2af80ceb40 0x7f2af809e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:05.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afd4c0640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af80cf340 0x7f2af81f0940 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae802dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:05.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2aee7fc640 1 -- 172.21.15.125:0/1308028704 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8005670 con 0x7f2af80cf340 2023-10-18T11:42:05.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2aee7fc640 1 -- 172.21.15.125:0/1308028704 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ae80038f0 con 0x7f2af80cf340 2023-10-18T11:42:05.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2aee7fc640 1 -- 172.21.15.125:0/1308028704 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8003ce0 con 0x7f2af80cf340 2023-10-18T11:42:05.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af814e550 con 0x7f2af80cf340 2023-10-18T11:42:05.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.658+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af80c83a0 con 0x7f2af80cf340 2023-10-18T11:42:05.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.659+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af80690e0 con 0x7f2af80cf340 2023-10-18T11:42:05.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.660+0000 7f2aee7fc640 1 -- 172.21.15.125:0/1308028704 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 100) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2ae803e450 con 0x7f2af80cf340 2023-10-18T11:42:05.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.660+0000 7f2aee7fc640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2adc06aed0 0x7f2adc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:05.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.660+0000 7f2aee7fc640 1 -- 172.21.15.125:0/1308028704 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6393+0+0 (secure 0 0 0) 0x7f2ae80ac250 con 0x7f2af80cf340 2023-10-18T11:42:05.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.660+0000 7f2afdcc1640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2adc06aed0 0x7f2adc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:05.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.661+0000 7f2afdcc1640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2adc06aed0 0x7f2adc06d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2af0007eb0 tx=0x7f2af00073c0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:05.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.664+0000 7f2aee7fc640 1 -- 172.21.15.125:0/1308028704 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ae8077210 con 0x7f2af80cf340 2023-10-18T11:42:05.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:05.830+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2af80d0b40 con 0x7f2af80cf340 2023-10-18T11:42:06.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.051+0000 7f2aee7fc640 1 -- 172.21.15.125:0/1308028704 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ae80775e0 con 0x7f2af80cf340 2023-10-18T11:42:06.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.536+0000 7f2aee7fc640 1 -- 172.21.15.125:0/1308028704 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v179) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2ae8090f90 con 0x7f2af80cf340 2023-10-18T11:42:06.539 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T11:42:06.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2adc06aed0 msgr2=0x7f2adc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2adc06aed0 0x7f2adc06d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2af0007eb0 tx=0x7f2af00073c0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af80cf340 msgr2=0x7f2af81f0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af80cf340 0x7f2af81f0940 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae802dce0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 shutdown_connections 2023-10-18T11:42:06.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2adc06aed0 0x7f2adc06d390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2af815b2a0 0x7f2af81f0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2af80cf340 0x7f2af81f0940 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 --2- 172.21.15.125:0/1308028704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2af80ceb40 0x7f2af809e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 >> 172.21.15.125:0/1308028704 conn(0x7f2af80d0d60 msgr2=0x7f2af80b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:06.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 shutdown_connections 2023-10-18T11:42:06.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.539+0000 7f2afff25640 1 -- 172.21.15.125:0/1308028704 wait complete. 2023-10-18T11:42:06.553 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T11:42:06.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.657+0000 7fe36674c640 1 -- 172.21.15.125:0/3800862960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3600cf340 msgr2=0x7fe3600d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.657+0000 7fe36674c640 1 --2- 172.21.15.125:0/3800862960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3600cf340 0x7fe3600d1910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe350009830 tx=0x7fe35002bcd0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.657+0000 7fe36674c640 1 -- 172.21.15.125:0/3800862960 shutdown_connections 2023-10-18T11:42:06.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.657+0000 7fe36674c640 1 --2- 172.21.15.125:0/3800862960 >> v2:172.21.15.139:3300/0 conn(0x7fe36015b2a0 0x7fe3600cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.657+0000 7fe36674c640 1 --2- 172.21.15.125:0/3800862960 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3600cf340 0x7fe3600d1910 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.657+0000 7fe36674c640 1 -- 172.21.15.125:0/3800862960 >> 172.21.15.125:0/3800862960 conn(0x7fe3600d0d60 msgr2=0x7fe3600d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:06.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.657+0000 7fe36674c640 1 -- 172.21.15.125:0/3800862960 shutdown_connections 2023-10-18T11:42:06.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.657+0000 7fe36674c640 1 -- 172.21.15.125:0/3800862960 wait complete. 2023-10-18T11:42:06.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.658+0000 7fe36674c640 1 Processor -- start 2023-10-18T11:42:06.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.658+0000 7fe36674c640 1 -- start start 2023-10-18T11:42:06.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe36674c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3600ceb40 0x7fe3600c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:06.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe36674c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3600cf340 0x7fe360157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:06.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe36674c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36015b2a0 0x7fe360158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:06.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe36674c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600ca140 con 0x7fe36015b2a0 2023-10-18T11:42:06.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe36674c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36015c680 con 0x7fe3600ceb40 2023-10-18T11:42:06.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe36674c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36015c960 con 0x7fe3600cf340 2023-10-18T11:42:06.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36015b2a0 0x7fe360158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:06.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe35ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3600ceb40 0x7fe3600c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:06.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36015b2a0 0x7fe360158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35360/0 (socket says 172.21.15.125:35360) 2023-10-18T11:42:06.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 -- 172.21.15.125:0/522164362 learned_addr learned my addr 172.21.15.125:0/522164362 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:06.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe35f7fe640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3600cf340 0x7fe360157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:06.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 -- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3600cf340 msgr2=0x7fe360157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3600cf340 0x7fe360157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 -- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3600ceb40 msgr2=0x7fe3600c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3600ceb40 0x7fe3600c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 -- 172.21.15.125:0/522164362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe354009560 con 0x7fe36015b2a0 2023-10-18T11:42:06.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe364ce9640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36015b2a0 0x7fe360158b90 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fe35800c7a0 tx=0x7fe35800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:06.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe35cff9640 1 -- 172.21.15.125:0/522164362 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358003b70 con 0x7fe36015b2a0 2023-10-18T11:42:06.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.659+0000 7fe35cff9640 1 -- 172.21.15.125:0/522164362 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe358003cf0 con 0x7fe36015b2a0 2023-10-18T11:42:06.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.660+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe350009490 con 0x7fe36015b2a0 2023-10-18T11:42:06.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.660+0000 7fe35cff9640 1 -- 172.21.15.125:0/522164362 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800aab0 con 0x7fe36015b2a0 2023-10-18T11:42:06.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.660+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36015ccf0 con 0x7fe36015b2a0 2023-10-18T11:42:06.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.660+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3600690e0 con 0x7fe36015b2a0 2023-10-18T11:42:06.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.661+0000 7fe35cff9640 1 -- 172.21.15.125:0/522164362 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe35800ac10 con 0x7fe36015b2a0 2023-10-18T11:42:06.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.661+0000 7fe35cff9640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe34c06af60 0x7fe34c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:06.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.661+0000 7fe35cff9640 1 -- 172.21.15.125:0/522164362 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7fe3580896b0 con 0x7fe36015b2a0 2023-10-18T11:42:06.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.662+0000 7fe35ffff640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe34c06af60 0x7fe34c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:06.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.662+0000 7fe35ffff640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe34c06af60 0x7fe34c06d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe350036000 tx=0x7fe350004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:06.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.666+0000 7fe35cff9640 1 -- 172.21.15.125:0/522164362 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe358054680 con 0x7fe36015b2a0 2023-10-18T11:42:06.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.846+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7fe3600d0b40 con 0x7fe36015b2a0 2023-10-18T11:42:06.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.850+0000 7fe35cff9640 1 -- 172.21.15.125:0/522164362 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v66) v1 ==== 71+0+0 (secure 0 0 0) 0x7fe358057ca0 con 0x7fe36015b2a0 2023-10-18T11:42:06.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.851+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe34c06af60 msgr2=0x7fe34c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.851+0000 7fe36674c640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe34c06af60 0x7fe34c06d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe350036000 tx=0x7fe350004730 comp rx=0 tx=0).stop 2023-10-18T11:42:06.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36015b2a0 msgr2=0x7fe360158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36015b2a0 0x7fe360158b90 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fe35800c7a0 tx=0x7fe35800cc60 comp rx=0 tx=0).stop 2023-10-18T11:42:06.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 shutdown_connections 2023-10-18T11:42:06.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe34c06af60 0x7fe34c06d420 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36015b2a0 0x7fe360158b90 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3600cf340 0x7fe360157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 --2- 172.21.15.125:0/522164362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3600ceb40 0x7fe3600c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 >> 172.21.15.125:0/522164362 conn(0x7fe3600d0d60 msgr2=0x7fe3600b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:06.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 shutdown_connections 2023-10-18T11:42:06.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.852+0000 7fe36674c640 1 -- 172.21.15.125:0/522164362 wait complete. 2023-10-18T11:42:06.864 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-18T11:42:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.965+0000 7f34b047c640 1 -- 172.21.15.125:0/15114535 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 msgr2=0x7f3494004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.965+0000 7f34b047c640 1 --2- 172.21.15.125:0/15114535 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 0x7f3494004c40 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f349c009b50 tx=0x7f34940057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.965+0000 7f34aea19640 1 -- 172.21.15.125:0/15114535 reap_dead start 2023-10-18T11:42:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.965+0000 7f34b047c640 1 -- 172.21.15.125:0/15114535 shutdown_connections 2023-10-18T11:42:06.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.965+0000 7f34b047c640 1 -- 172.21.15.125:0/15114535 >> 172.21.15.125:0/15114535 conn(0x7f34a8069930 msgr2=0x7f34a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:06.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.965+0000 7f34b047c640 1 -- 172.21.15.125:0/15114535 shutdown_connections 2023-10-18T11:42:06.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.965+0000 7f34b047c640 1 -- 172.21.15.125:0/15114535 wait complete. 2023-10-18T11:42:06.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.966+0000 7f34b047c640 1 Processor -- start 2023-10-18T11:42:06.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.966+0000 7f34b047c640 1 -- start start 2023-10-18T11:42:06.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34b047c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 0x7f34a81eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:06.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34b047c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34a80c8ca0 0x7f34a80b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:06.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34b047c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34a813c320 0x7f34a80b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:06.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34b047c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a80ac9b0 con 0x7f34940047c0 2023-10-18T11:42:06.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34b047c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a813beb0 con 0x7f34a813c320 2023-10-18T11:42:06.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34b047c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a813b0f0 con 0x7f34a80c8ca0 2023-10-18T11:42:06.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 0x7f34a81eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:06.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 0x7f34a81eddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35384/0 (socket says 172.21.15.125:35384) 2023-10-18T11:42:06.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 -- 172.21.15.125:0/666743200 learned_addr learned my addr 172.21.15.125:0/666743200 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:06.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34aea19640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34a813c320 0x7f34a80b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:06.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ada17640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34a80c8ca0 0x7f34a80b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:06.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 -- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34a80c8ca0 msgr2=0x7f34a80b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34a80c8ca0 0x7f34a80b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 -- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34a813c320 msgr2=0x7f34a80b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:06.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34a813c320 0x7f34a80b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:06.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 -- 172.21.15.125:0/666743200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a813b3d0 con 0x7f34940047c0 2023-10-18T11:42:06.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ada17640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34a80c8ca0 0x7f34a80b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:42:06.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.967+0000 7f34ae218640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 0x7f34a81eddf0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f34a4006600 tx=0x7f34a40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:06.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.968+0000 7f349affd640 1 -- 172.21.15.125:0/666743200 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4011840 con 0x7f34940047c0 2023-10-18T11:42:06.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.968+0000 7f349affd640 1 -- 172.21.15.125:0/666743200 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f34a40119a0 con 0x7f34940047c0 2023-10-18T11:42:06.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.968+0000 7f349affd640 1 -- 172.21.15.125:0/666743200 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4011db0 con 0x7f34940047c0 2023-10-18T11:42:06.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.968+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a813b6b0 con 0x7f34940047c0 2023-10-18T11:42:06.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.968+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a81ee330 con 0x7f34940047c0 2023-10-18T11:42:06.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.969+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a806b740 con 0x7f34940047c0 2023-10-18T11:42:06.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.969+0000 7f349affd640 1 -- 172.21.15.125:0/666743200 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f34a40103c0 con 0x7f34940047c0 2023-10-18T11:42:06.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.969+0000 7f349affd640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f348c06af90 0x7f348c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:06.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.970+0000 7f349affd640 1 -- 172.21.15.125:0/666743200 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7f34a408aa40 con 0x7f34940047c0 2023-10-18T11:42:06.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.970+0000 7f34ada17640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f348c06af90 0x7f348c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:06.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.970+0000 7f34ada17640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f348c06af90 0x7f348c06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f349c009b50 tx=0x7f349c0046c0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:06.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:06.974+0000 7f349affd640 1 -- 172.21.15.125:0/666743200 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34a408ce60 con 0x7f34940047c0 2023-10-18T11:42:07.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.203+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f34a80690e0 con 0x7f34940047c0 2023-10-18T11:42:07.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.538+0000 7f349affd640 1 -- 172.21.15.125:0/666743200 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2510) v1 ==== 174+0+0 (secure 0 0 0) 0x7f34a4015020 con 0x7f34940047c0 2023-10-18T11:42:07.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f348c06af90 msgr2=0x7f348c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f348c06af90 0x7f348c06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f349c009b50 tx=0x7f349c0046c0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 msgr2=0x7f34a81eddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 0x7f34a81eddf0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f34a4006600 tx=0x7f34a40039c0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 shutdown_connections 2023-10-18T11:42:07.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f348c06af90 0x7f348c06d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34a813c320 0x7f34a80b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34a80c8ca0 0x7f34a80b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 --2- 172.21.15.125:0/666743200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34940047c0 0x7f34a81eddf0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 >> 172.21.15.125:0/666743200 conn(0x7f34a8069930 msgr2=0x7f34a80b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:07.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 shutdown_connections 2023-10-18T11:42:07.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.540+0000 7f34b047c640 1 -- 172.21.15.125:0/666743200 wait complete. 2023-10-18T11:42:07.554 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-10-18T11:42:07.555 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-10-18T11:42:07.555 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T11:42:07.555 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:42:07.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.657+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1827471304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80ceb40 msgr2=0x7f55f80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.657+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1827471304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80ceb40 0x7f55f80b0650 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f55f808a120 tx=0x7f55f002c1d0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.657+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1827471304 shutdown_connections 2023-10-18T11:42:07.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.657+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1827471304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80ceb40 0x7f55f80b0650 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.657+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1827471304 >> v2:172.21.15.139:3300/0 conn(0x7f55f815b2a0 0x7f55f80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.657+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1827471304 >> 172.21.15.125:0/1827471304 conn(0x7f55f80d0d60 msgr2=0x7f55f80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:07.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.657+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1827471304 shutdown_connections 2023-10-18T11:42:07.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.657+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1827471304 wait complete. 2023-10-18T11:42:07.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.658+0000 7f55fdf1d640 1 Processor -- start 2023-10-18T11:42:07.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.658+0000 7f55fdf1d640 1 -- start start 2023-10-18T11:42:07.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55fdf1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f80ceb40 0x7f55f81f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:07.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55fdf1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80cf340 0x7f55f8156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:07.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55fdf1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f815b2a0 0x7f55f81f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:07.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55fdf1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f809e8e0 con 0x7f55f80cf340 2023-10-18T11:42:07.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55fdf1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8156d30 con 0x7f55f80ceb40 2023-10-18T11:42:07.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55fdf1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f809ac00 con 0x7f55f815b2a0 2023-10-18T11:42:07.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80cf340 0x7f55f8156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:07.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80cf340 0x7f55f8156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38474/0 (socket says 172.21.15.125:38474) 2023-10-18T11:42:07.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55f6ffd640 1 -- 172.21.15.125:0/1895418472 learned_addr learned my addr 172.21.15.125:0/1895418472 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:07.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55f6ffd640 1 -- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f815b2a0 msgr2=0x7f55f81f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55f6ffd640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f815b2a0 0x7f55f81f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55f6ffd640 1 -- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f80ceb40 msgr2=0x7f55f81f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55f77fe640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f80ceb40 0x7f55f81f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:07.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.659+0000 7f55f6ffd640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f80ceb40 0x7f55f81f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.660+0000 7f55f6ffd640 1 -- 172.21.15.125:0/1895418472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0009900 con 0x7f55f80cf340 2023-10-18T11:42:07.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.660+0000 7f55f6ffd640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80cf340 0x7f55f8156ec0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f55e800b410 tx=0x7f55e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:07.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.660+0000 7f55effff640 1 -- 172.21.15.125:0/1895418472 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011830 con 0x7f55f80cf340 2023-10-18T11:42:07.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.660+0000 7f55effff640 1 -- 172.21.15.125:0/1895418472 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55e8011aa0 con 0x7f55f80cf340 2023-10-18T11:42:07.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.660+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f80b0d50 con 0x7f55f80cf340 2023-10-18T11:42:07.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.660+0000 7f55effff640 1 -- 172.21.15.125:0/1895418472 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8010380 con 0x7f55f80cf340 2023-10-18T11:42:07.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.660+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f8157400 con 0x7f55f80cf340 2023-10-18T11:42:07.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.661+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f80690e0 con 0x7f55f80cf340 2023-10-18T11:42:07.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.661+0000 7f55effff640 1 -- 172.21.15.125:0/1895418472 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f55e80104e0 con 0x7f55f80cf340 2023-10-18T11:42:07.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.662+0000 7f55effff640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f55d806af60 0x7f55d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:07.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.662+0000 7f55effff640 1 -- 172.21.15.125:0/1895418472 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7f55e8089cf0 con 0x7f55f80cf340 2023-10-18T11:42:07.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.662+0000 7f55f77fe640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f55d806af60 0x7f55d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:07.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.662+0000 7f55f77fe640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f55d806af60 0x7f55d806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f55e00015d0 tx=0x7f55e0009040 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:07.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.666+0000 7f55effff640 1 -- 172.21.15.125:0/1895418472 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55e808e070 con 0x7f55f80cf340 2023-10-18T11:42:07.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.834+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55f80d0b40 con 0x7f55f80cf340 2023-10-18T11:42:07.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.835+0000 7f55effff640 1 -- 172.21.15.125:0/1895418472 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+14311 (secure 0 0 0) 0x7f55e8015020 con 0x7f55f80cf340 2023-10-18T11:42:07.839 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:07.839 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":179,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:42:06.529138+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6833/2949284679":"2023-10-19T11:42:03.524333+0000","172.21.15.139:6832/1922997989":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2170035400":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/153806259":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2991312240":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/767536410":"2023-10-19T11:42:01.439709+0000","172.21.15.139:6833/1922997989":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/2776413208":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/3094960806":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/139213704":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4234510722":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/4113808248":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6833/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.139:0/312743033":"2023-10-19T11:41:55.015024+0000","172.21.15.139:6832/4236255547":"2023-10-19T11:41:55.015024+0000","172.21.15.125:6835/2949284679":"2023-10-19T11:42:03.524333+0000","172.21.15.139:0/1067360436":"2023-10-19T11:42:01.439709+0000","172.21.15.139:0/1471732770":"2023-10-19T11:41:55.015024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:42:07.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f55d806af60 msgr2=0x7f55d806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f55d806af60 0x7f55d806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f55e00015d0 tx=0x7f55e0009040 comp rx=0 tx=0).stop 2023-10-18T11:42:07.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80cf340 msgr2=0x7f55f8156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80cf340 0x7f55f8156ec0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f55e800b410 tx=0x7f55e800b8d0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 shutdown_connections 2023-10-18T11:42:07.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f55d806af60 0x7f55d806d420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f815b2a0 0x7f55f81f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f80cf340 0x7f55f8156ec0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 --2- 172.21.15.125:0/1895418472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f80ceb40 0x7f55f81f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 >> 172.21.15.125:0/1895418472 conn(0x7f55f80d0d60 msgr2=0x7f55f80cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:07.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 shutdown_connections 2023-10-18T11:42:07.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.837+0000 7f55fdf1d640 1 -- 172.21.15.125:0/1895418472 wait complete. 2023-10-18T11:42:07.851 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:42:07.851 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:42:07.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.953+0000 7f61358f2640 1 -- 172.21.15.125:0/3451232763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 msgr2=0x7f6118004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.953+0000 7f61358f2640 1 --2- 172.21.15.125:0/3451232763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 0x7f6118004ed0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f6120001d50 tx=0x7f6118006900 comp rx=0 tx=0).stop 2023-10-18T11:42:07.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.953+0000 7f612f7fe640 1 -- 172.21.15.125:0/3451232763 reap_dead start 2023-10-18T11:42:07.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.953+0000 7f61358f2640 1 -- 172.21.15.125:0/3451232763 shutdown_connections 2023-10-18T11:42:07.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.953+0000 7f61358f2640 1 -- 172.21.15.125:0/3451232763 >> 172.21.15.125:0/3451232763 conn(0x7f6130069930 msgr2=0x7f6130069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:07.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.953+0000 7f61358f2640 1 -- 172.21.15.125:0/3451232763 shutdown_connections 2023-10-18T11:42:07.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.953+0000 7f61358f2640 1 -- 172.21.15.125:0/3451232763 wait complete. 2023-10-18T11:42:07.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.954+0000 7f61358f2640 1 Processor -- start 2023-10-18T11:42:07.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f61358f2640 1 -- start start 2023-10-18T11:42:07.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f61358f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 0x7f61300af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:07.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f61358f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61300acfa0 0x7f6130156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:07.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f61358f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130149420 0x7f61301498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:07.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f61358f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6130149de0 con 0x7f6118004ab0 2023-10-18T11:42:07.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f61358f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61300ad880 con 0x7f6130149420 2023-10-18T11:42:07.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f61358f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613014b3b0 con 0x7f61300acfa0 2023-10-18T11:42:07.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f612effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 0x7f61300af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:07.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f612effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 0x7f61300af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38490/0 (socket says 172.21.15.125:38490) 2023-10-18T11:42:07.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.955+0000 7f612effd640 1 -- 172.21.15.125:0/3317886305 learned_addr learned my addr 172.21.15.125:0/3317886305 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:07.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f612effd640 1 -- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61300acfa0 msgr2=0x7f6130156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f612effd640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61300acfa0 0x7f6130156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f612effd640 1 -- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130149420 msgr2=0x7f61301498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:07.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f612f7fe640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130149420 0x7f61301498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:07.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f612effd640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130149420 0x7f61301498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:07.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f612effd640 1 -- 172.21.15.125:0/3317886305 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613014b690 con 0x7f6118004ab0 2023-10-18T11:42:07.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f612f7fe640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130149420 0x7f61301498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:07.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f612effd640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 0x7f61300af020 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f6120001e70 tx=0x7f612002c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:07.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f6117fff640 1 -- 172.21.15.125:0/3317886305 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612002ce80 con 0x7f6118004ab0 2023-10-18T11:42:07.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f6117fff640 1 -- 172.21.15.125:0/3317886305 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6120006050 con 0x7f6118004ab0 2023-10-18T11:42:07.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613014b970 con 0x7f6118004ab0 2023-10-18T11:42:07.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f6117fff640 1 -- 172.21.15.125:0/3317886305 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120006420 con 0x7f6118004ab0 2023-10-18T11:42:07.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.956+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6130156a70 con 0x7f6118004ab0 2023-10-18T11:42:07.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.957+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6130156cc0 con 0x7f6118004ab0 2023-10-18T11:42:07.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.957+0000 7f6117fff640 1 -- 172.21.15.125:0/3317886305 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6120006580 con 0x7f6118004ab0 2023-10-18T11:42:07.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.958+0000 7f6117fff640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f611c06af90 0x7f611c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:07.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.958+0000 7f6117fff640 1 -- 172.21.15.125:0/3317886305 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7f61200ab300 con 0x7f6118004ab0 2023-10-18T11:42:07.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.958+0000 7f612e7fc640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f611c06af90 0x7f611c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:07.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.959+0000 7f612e7fc640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f611c06af90 0x7f611c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f61300c9ac0 tx=0x7f6124005af0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:07.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:07.962+0000 7f6117fff640 1 -- 172.21.15.125:0/3317886305 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6120076190 con 0x7f6118004ab0 2023-10-18T11:42:08.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.127+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f613014b210 con 0x7f6118004ab0 2023-10-18T11:42:08.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.127+0000 7f6117fff640 1 -- 172.21.15.125:0/3317886305 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v179) v1 ==== 93+0+10 (secure 0 0 0) 0x7f613014b210 con 0x7f6118004ab0 2023-10-18T11:42:08.130 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:42:08.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f611c06af90 msgr2=0x7f611c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f611c06af90 0x7f611c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f61300c9ac0 tx=0x7f6124005af0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 msgr2=0x7f61300af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 0x7f61300af020 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f6120001e70 tx=0x7f612002c880 comp rx=0 tx=0).stop 2023-10-18T11:42:08.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 shutdown_connections 2023-10-18T11:42:08.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f611c06af90 0x7f611c06d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6130149420 0x7f61301498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61300acfa0 0x7f6130156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 --2- 172.21.15.125:0/3317886305 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6118004ab0 0x7f61300af020 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 >> 172.21.15.125:0/3317886305 conn(0x7f6130069930 msgr2=0x7f6130070510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 shutdown_connections 2023-10-18T11:42:08.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.129+0000 7f61358f2640 1 -- 172.21.15.125:0/3317886305 wait complete. 2023-10-18T11:42:08.142 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-18T11:42:08.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.243+0000 7fb295dc9640 1 -- 172.21.15.125:0/2073740953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2780047c0 msgr2=0x7fb278004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.243+0000 7fb295dc9640 1 --2- 172.21.15.125:0/2073740953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2780047c0 0x7fb278004c40 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fb284006e40 tx=0x7fb2780057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.243+0000 7fb28ffff640 1 -- 172.21.15.125:0/2073740953 reap_dead start 2023-10-18T11:42:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.243+0000 7fb295dc9640 1 -- 172.21.15.125:0/2073740953 shutdown_connections 2023-10-18T11:42:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.243+0000 7fb295dc9640 1 -- 172.21.15.125:0/2073740953 >> 172.21.15.125:0/2073740953 conn(0x7fb290069930 msgr2=0x7fb290069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:08.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.243+0000 7fb295dc9640 1 -- 172.21.15.125:0/2073740953 shutdown_connections 2023-10-18T11:42:08.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.243+0000 7fb295dc9640 1 -- 172.21.15.125:0/2073740953 wait complete. 2023-10-18T11:42:08.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.244+0000 7fb295dc9640 1 Processor -- start 2023-10-18T11:42:08.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.244+0000 7fb295dc9640 1 -- start start 2023-10-18T11:42:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb295dc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2780047c0 0x7fb290158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb295dc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29013c210 0x7fb2901ec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb295dc9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb29013c690 0x7fb2901ecac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb295dc9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29014f1d0 con 0x7fb29013c210 2023-10-18T11:42:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb295dc9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29014f470 con 0x7fb2780047c0 2023-10-18T11:42:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb295dc9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29014f750 con 0x7fb29013c690 2023-10-18T11:42:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29013c210 0x7fb2901ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29013c210 0x7fb2901ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38514/0 (socket says 172.21.15.125:38514) 2023-10-18T11:42:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28effd640 1 -- 172.21.15.125:0/3069590824 learned_addr learned my addr 172.21.15.125:0/3069590824 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28ffff640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb29013c690 0x7fb2901ecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28f7fe640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2780047c0 0x7fb290158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28effd640 1 -- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb29013c690 msgr2=0x7fb2901ecac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28effd640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb29013c690 0x7fb2901ecac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28effd640 1 -- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2780047c0 msgr2=0x7fb290158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28effd640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2780047c0 0x7fb290158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28effd640 1 -- 172.21.15.125:0/3069590824 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29014fa30 con 0x7fb29013c210 2023-10-18T11:42:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.245+0000 7fb28f7fe640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2780047c0 0x7fb290158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.246+0000 7fb28effd640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29013c210 0x7fb2901ec580 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fb28402c740 tx=0x7fb2840061e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.246+0000 7fb277fff640 1 -- 172.21.15.125:0/3069590824 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28403c040 con 0x7fb29013c210 2023-10-18T11:42:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.246+0000 7fb277fff640 1 -- 172.21.15.125:0/3069590824 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb284003810 con 0x7fb29013c210 2023-10-18T11:42:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.246+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29014fd10 con 0x7fb29013c210 2023-10-18T11:42:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.246+0000 7fb277fff640 1 -- 172.21.15.125:0/3069590824 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284003c20 con 0x7fb29013c210 2023-10-18T11:42:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.246+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb290136570 con 0x7fb29013c210 2023-10-18T11:42:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.247+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2900690e0 con 0x7fb29013c210 2023-10-18T11:42:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.247+0000 7fb277fff640 1 -- 172.21.15.125:0/3069590824 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb284032070 con 0x7fb29013c210 2023-10-18T11:42:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.247+0000 7fb277fff640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fb27c06af60 0x7fb27c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.248+0000 7fb277fff640 1 -- 172.21.15.125:0/3069590824 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7fb2840aaf50 con 0x7fb29013c210 2023-10-18T11:42:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.248+0000 7fb28f7fe640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fb27c06af60 0x7fb27c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.248+0000 7fb28f7fe640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fb27c06af60 0x7fb27c06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb280009a70 tx=0x7fb280005f90 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:08.257 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.252+0000 7fb277fff640 1 -- 172.21.15.125:0/3069590824 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb284075de0 con 0x7fb29013c210 2023-10-18T11:42:08.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.480+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fb29006ff80 con 0x7fb29013c210 2023-10-18T11:42:08.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.539+0000 7fb277fff640 1 -- 172.21.15.125:0/3069590824 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2511) v1 ==== 168+0+0 (secure 0 0 0) 0x7fb284046020 con 0x7fb29013c210 2023-10-18T11:42:08.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fb27c06af60 msgr2=0x7fb27c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fb27c06af60 0x7fb27c06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb280009a70 tx=0x7fb280005f90 comp rx=0 tx=0).stop 2023-10-18T11:42:08.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29013c210 msgr2=0x7fb2901ec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29013c210 0x7fb2901ec580 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fb28402c740 tx=0x7fb2840061e0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 shutdown_connections 2023-10-18T11:42:08.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fb27c06af60 0x7fb27c06d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb29013c690 0x7fb2901ecac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29013c210 0x7fb2901ec580 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 --2- 172.21.15.125:0/3069590824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2780047c0 0x7fb290158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 >> 172.21.15.125:0/3069590824 conn(0x7fb290069930 msgr2=0x7fb29006be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:08.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 shutdown_connections 2023-10-18T11:42:08.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.541+0000 7fb295dc9640 1 -- 172.21.15.125:0/3069590824 wait complete. 2023-10-18T11:42:08.554 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T11:42:08.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.654+0000 7f59567ce640 1 -- 172.21.15.125:0/2134781512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 msgr2=0x7f59500b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.654+0000 7f59567ce640 1 --2- 172.21.15.125:0/2134781512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 0x7f59500b0650 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f5948005fd0 tx=0x7f594802c050 comp rx=0 tx=0).stop 2023-10-18T11:42:08.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.654+0000 7f59567ce640 1 -- 172.21.15.125:0/2134781512 shutdown_connections 2023-10-18T11:42:08.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.654+0000 7f59567ce640 1 --2- 172.21.15.125:0/2134781512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 0x7f59500b0650 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.654+0000 7f59567ce640 1 --2- 172.21.15.125:0/2134781512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f595015b2a0 0x7f59500d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.654+0000 7f59567ce640 1 -- 172.21.15.125:0/2134781512 >> 172.21.15.125:0/2134781512 conn(0x7f59500d0d60 msgr2=0x7f59500d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:08.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.654+0000 7f59567ce640 1 -- 172.21.15.125:0/2134781512 shutdown_connections 2023-10-18T11:42:08.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.654+0000 7f59567ce640 1 -- 172.21.15.125:0/2134781512 wait complete. 2023-10-18T11:42:08.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.655+0000 7f59567ce640 1 Processor -- start 2023-10-18T11:42:08.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.655+0000 7f59567ce640 1 -- start start 2023-10-18T11:42:08.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f59567ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 0x7f59501e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f59567ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59500cf340 0x7f595014f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f59567ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f595015b2a0 0x7f59500ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f59567ce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59501e4820 con 0x7f59500ceb40 2023-10-18T11:42:08.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f59567ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59501e3c10 con 0x7f595015b2a0 2023-10-18T11:42:08.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f59567ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59501e3ef0 con 0x7f59500cf340 2023-10-18T11:42:08.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f594ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 0x7f59501e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f594ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 0x7f59501e4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38538/0 (socket says 172.21.15.125:38538) 2023-10-18T11:42:08.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f594ffff640 1 -- 172.21.15.125:0/1936005765 learned_addr learned my addr 172.21.15.125:0/1936005765 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:08.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f594ffff640 1 -- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59500cf340 msgr2=0x7f595014f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f594ffff640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59500cf340 0x7f595014f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f594ffff640 1 -- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f595015b2a0 msgr2=0x7f59500ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.656+0000 7f594ffff640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f595015b2a0 0x7f59500ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.657+0000 7f594ffff640 1 -- 172.21.15.125:0/1936005765 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948009840 con 0x7f59500ceb40 2023-10-18T11:42:08.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.657+0000 7f594ffff640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 0x7f59501e4cb0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f594000c4b0 tx=0x7f594000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:08.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.657+0000 7f594cff9640 1 -- 172.21.15.125:0/1936005765 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940005480 con 0x7f59500ceb40 2023-10-18T11:42:08.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.657+0000 7f594cff9640 1 -- 172.21.15.125:0/1936005765 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f59400036e0 con 0x7f59500ceb40 2023-10-18T11:42:08.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.657+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59501e41d0 con 0x7f59500ceb40 2023-10-18T11:42:08.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.657+0000 7f594cff9640 1 -- 172.21.15.125:0/1936005765 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940003b30 con 0x7f59500ceb40 2023-10-18T11:42:08.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.657+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59500cace0 con 0x7f59500ceb40 2023-10-18T11:42:08.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.658+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59500690e0 con 0x7f59500ceb40 2023-10-18T11:42:08.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.658+0000 7f594cff9640 1 -- 172.21.15.125:0/1936005765 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5940003c90 con 0x7f59500ceb40 2023-10-18T11:42:08.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.659+0000 7f594cff9640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f593c06af60 0x7f593c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.659+0000 7f594cff9640 1 -- 172.21.15.125:0/1936005765 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7f59400890f0 con 0x7f59500ceb40 2023-10-18T11:42:08.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.659+0000 7f594f7fe640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f593c06af60 0x7f593c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.659+0000 7f594f7fe640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f593c06af60 0x7f593c06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59500c7200 tx=0x7f5944005b50 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:08.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.663+0000 7f594cff9640 1 -- 172.21.15.125:0/1936005765 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5940053f80 con 0x7f59500ceb40 2023-10-18T11:42:08.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.829+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f59500d0b40 con 0x7f59500ceb40 2023-10-18T11:42:08.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.830+0000 7f594cff9640 1 -- 172.21.15.125:0/1936005765 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v66) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f5940017080 con 0x7f59500ceb40 2023-10-18T11:42:08.833 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T11:42:08.833 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T11:42:08.833 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T11:42:08.833 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T11:42:08.834 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T11:42:08.834 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T11:42:08.834 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T11:42:08.834 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T11:42:08.834 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T11:42:08.835 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T11:42:08.835 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T11:42:08.835 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T11:42:08.835 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T11:42:08.835 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T11:42:08.836 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T11:42:08.836 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T11:42:08.836 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T11:42:08.836 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T11:42:08.836 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T11:42:08.836 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T11:42:08.837 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T11:42:08.837 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T11:42:08.837 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T11:42:08.837 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T11:42:08.837 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T11:42:08.838 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T11:42:08.838 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T11:42:08.838 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T11:42:08.838 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T11:42:08.838 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T11:42:08.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f593c06af60 msgr2=0x7f593c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f593c06af60 0x7f593c06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59500c7200 tx=0x7f5944005b50 comp rx=0 tx=0).stop 2023-10-18T11:42:08.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 msgr2=0x7f59501e4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 0x7f59501e4cb0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f594000c4b0 tx=0x7f594000c970 comp rx=0 tx=0).stop 2023-10-18T11:42:08.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 shutdown_connections 2023-10-18T11:42:08.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f593c06af60 0x7f593c06d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f595015b2a0 0x7f59500ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59500cf340 0x7f595014f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 --2- 172.21.15.125:0/1936005765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59500ceb40 0x7f59501e4cb0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 >> 172.21.15.125:0/1936005765 conn(0x7f59500d0d60 msgr2=0x7f5950151110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:08.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 shutdown_connections 2023-10-18T11:42:08.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.832+0000 7f59567ce640 1 -- 172.21.15.125:0/1936005765 wait complete. 2023-10-18T11:42:08.845 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T11:42:08.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.948+0000 7fe737ee5640 1 -- 172.21.15.125:0/4120616036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe71c004d00 msgr2=0x7fe71c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.948+0000 7fe737ee5640 1 --2- 172.21.15.125:0/4120616036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe71c004d00 0x7fe71c005180 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7fe72800baf0 tx=0x7fe71c0057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.948+0000 7fe736482640 1 -- 172.21.15.125:0/4120616036 reap_dead start 2023-10-18T11:42:08.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.948+0000 7fe737ee5640 1 -- 172.21.15.125:0/4120616036 shutdown_connections 2023-10-18T11:42:08.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.948+0000 7fe737ee5640 1 -- 172.21.15.125:0/4120616036 >> 172.21.15.125:0/4120616036 conn(0x7fe730069930 msgr2=0x7fe730069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:08.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.949+0000 7fe737ee5640 1 -- 172.21.15.125:0/4120616036 shutdown_connections 2023-10-18T11:42:08.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.949+0000 7fe737ee5640 1 -- 172.21.15.125:0/4120616036 wait complete. 2023-10-18T11:42:08.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe737ee5640 1 Processor -- start 2023-10-18T11:42:08.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe737ee5640 1 -- start start 2023-10-18T11:42:08.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe737ee5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe730140200 0x7fe7300ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe737ee5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe730148970 0x7fe7300ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe737ee5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7300acad0 0x7fe73014c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe737ee5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730150790 con 0x7fe730140200 2023-10-18T11:42:08.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe737ee5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300acf50 con 0x7fe730148970 2023-10-18T11:42:08.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe737ee5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73014f070 con 0x7fe7300acad0 2023-10-18T11:42:08.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe735c81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe730140200 0x7fe7300ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe735c81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe730140200 0x7fe7300ad360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38570/0 (socket says 172.21.15.125:38570) 2023-10-18T11:42:08.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.950+0000 7fe735c81640 1 -- 172.21.15.125:0/2699354024 learned_addr learned my addr 172.21.15.125:0/2699354024 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:08.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe736482640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7300acad0 0x7fe73014c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe735480640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe730148970 0x7fe7300ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe735c81640 1 -- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7300acad0 msgr2=0x7fe73014c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe735c81640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7300acad0 0x7fe73014c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe735c81640 1 -- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe730148970 msgr2=0x7fe7300ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:08.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe735c81640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe730148970 0x7fe7300ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:08.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe735c81640 1 -- 172.21.15.125:0/2699354024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730147e90 con 0x7fe730140200 2023-10-18T11:42:08.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe736482640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7300acad0 0x7fe73014c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:08.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe735480640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe730148970 0x7fe7300ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:08.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe735c81640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe730140200 0x7fe7300ad360 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00edf0 tx=0x7fe72c00b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:08.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe7267fc640 1 -- 172.21.15.125:0/2699354024 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c003db0 con 0x7fe730140200 2023-10-18T11:42:08.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe7267fc640 1 -- 172.21.15.125:0/2699354024 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe72c00ce20 con 0x7fe730140200 2023-10-18T11:42:08.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe730148170 con 0x7fe730140200 2023-10-18T11:42:08.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe7267fc640 1 -- 172.21.15.125:0/2699354024 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0062d0 con 0x7fe730140200 2023-10-18T11:42:08.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.951+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe730148df0 con 0x7fe730140200 2023-10-18T11:42:08.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.952+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe730148690 con 0x7fe730140200 2023-10-18T11:42:08.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.952+0000 7fe7267fc640 1 -- 172.21.15.125:0/2699354024 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe72c006430 con 0x7fe730140200 2023-10-18T11:42:08.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.953+0000 7fe7267fc640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe71406af90 0x7fe71406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:08.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.953+0000 7fe7267fc640 1 -- 172.21.15.125:0/2699354024 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7fe72c08a110 con 0x7fe730140200 2023-10-18T11:42:08.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.953+0000 7fe735480640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe71406af90 0x7fe71406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:08.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.954+0000 7fe735480640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe71406af90 0x7fe71406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe720001d00 tx=0x7fe720001bb0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:08.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:08.957+0000 7fe7267fc640 1 -- 172.21.15.125:0/2699354024 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe730148690 con 0x7fe730140200 2023-10-18T11:42:09.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.124+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe730148690 con 0x7fe730140200 2023-10-18T11:42:09.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.127+0000 7fe7267fc640 1 -- 172.21.15.125:0/2699354024 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v67)=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe730148690 con 0x7fe730140200 2023-10-18T11:42:09.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe71406af90 msgr2=0x7fe71406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe71406af90 0x7fe71406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe720001d00 tx=0x7fe720001bb0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe730140200 msgr2=0x7fe7300ad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe730140200 0x7fe7300ad360 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00edf0 tx=0x7fe72c00b620 comp rx=0 tx=0).stop 2023-10-18T11:42:09.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 shutdown_connections 2023-10-18T11:42:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fe71406af90 0x7fe71406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe730148970 0x7fe7300ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7300acad0 0x7fe73014c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 --2- 172.21.15.125:0/2699354024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe730140200 0x7fe7300ad360 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 >> 172.21.15.125:0/2699354024 conn(0x7fe730069930 msgr2=0x7fe730156de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:09.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 shutdown_connections 2023-10-18T11:42:09.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.129+0000 7fe737ee5640 1 -- 172.21.15.125:0/2699354024 wait complete. 2023-10-18T11:42:09.142 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T11:42:09.143 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:42:09.182 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:42:09.235 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T11:42:09.235 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:42:09.236 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:42:09.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:42:09.252 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:42:09.252 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:42:09.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:42:09.269 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:42:09.269 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:42:09.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:42:09.306 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:42:09.307 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:42:09.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:42:09.325 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T11:42:09.325 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:42:09.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.446+0000 7f993f8b4640 1 -- 172.21.15.125:0/46381020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 msgr2=0x7f99380affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.446+0000 7f993f8b4640 1 --2- 172.21.15.125:0/46381020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 0x7f99380affe0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9930009830 tx=0x7f993002bcd0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.446+0000 7f993f8b4640 1 -- 172.21.15.125:0/46381020 shutdown_connections 2023-10-18T11:42:09.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.446+0000 7f993f8b4640 1 --2- 172.21.15.125:0/46381020 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 0x7f99380affe0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.446+0000 7f993f8b4640 1 --2- 172.21.15.125:0/46381020 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993813e520 0x7f993806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.446+0000 7f993f8b4640 1 -- 172.21.15.125:0/46381020 >> 172.21.15.125:0/46381020 conn(0x7f9938069930 msgr2=0x7f9938069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:09.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.446+0000 7f993f8b4640 1 -- 172.21.15.125:0/46381020 shutdown_connections 2023-10-18T11:42:09.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.446+0000 7f993f8b4640 1 -- 172.21.15.125:0/46381020 wait complete. 2023-10-18T11:42:09.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.447+0000 7f993f8b4640 1 Processor -- start 2023-10-18T11:42:09.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.447+0000 7f993f8b4640 1 -- start start 2023-10-18T11:42:09.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993f8b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 0x7f99380abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:09.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993f8b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993813e520 0x7f99381400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:09.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993f8b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938141690 0x7f993813cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:09.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993f8b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993813bed0 con 0x7f9938141690 2023-10-18T11:42:09.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993f8b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993813c1b0 con 0x7f9938070af0 2023-10-18T11:42:09.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993f8b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993813c490 con 0x7f993813e520 2023-10-18T11:42:09.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993de51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938141690 0x7f993813cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:09.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993de51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938141690 0x7f993813cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38582/0 (socket says 172.21.15.125:38582) 2023-10-18T11:42:09.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993de51640 1 -- 172.21.15.125:0/2157750897 learned_addr learned my addr 172.21.15.125:0/2157750897 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:09.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993d650640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 0x7f99380abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:09.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.448+0000 7f993ce4f640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993813e520 0x7f99381400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:09.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993de51640 1 -- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993813e520 msgr2=0x7f99381400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993de51640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993813e520 0x7f99381400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993de51640 1 -- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 msgr2=0x7f99380abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993de51640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 0x7f99380abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993de51640 1 -- 172.21.15.125:0/2157750897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934009840 con 0x7f9938141690 2023-10-18T11:42:09.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993ce4f640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993813e520 0x7f99381400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:42:09.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993d650640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 0x7f99380abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:42:09.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993de51640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938141690 0x7f993813cf70 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f9930009950 tx=0x7f99300038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:09.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f992dffb640 1 -- 172.21.15.125:0/2157750897 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930003ea0 con 0x7f9938141690 2023-10-18T11:42:09.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9930009490 con 0x7f9938141690 2023-10-18T11:42:09.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f992dffb640 1 -- 172.21.15.125:0/2157750897 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9930005020 con 0x7f9938141690 2023-10-18T11:42:09.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f992dffb640 1 -- 172.21.15.125:0/2157750897 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930005410 con 0x7f9938141690 2023-10-18T11:42:09.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.449+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993813ca50 con 0x7f9938141690 2023-10-18T11:42:09.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.450+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99380b1bd0 con 0x7f9938141690 2023-10-18T11:42:09.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.451+0000 7f992dffb640 1 -- 172.21.15.125:0/2157750897 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f99300047a0 con 0x7f9938141690 2023-10-18T11:42:09.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.451+0000 7f992dffb640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f991c06af60 0x7f991c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:09.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.451+0000 7f992dffb640 1 -- 172.21.15.125:0/2157750897 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7f99300abcf0 con 0x7f9938141690 2023-10-18T11:42:09.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.451+0000 7f993d650640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f991c06af60 0x7f991c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:09.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.452+0000 7f993d650640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f991c06af60 0x7f991c06d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9934005fd0 tx=0x7f9934009500 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:09.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.455+0000 7f992dffb640 1 -- 172.21.15.125:0/2157750897 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99300b0070 con 0x7f9938141690 2023-10-18T11:42:09.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.658+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99380690e0 con 0x7f9938141690 2023-10-18T11:42:09.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.658+0000 7f992dffb640 1 -- 172.21.15.125:0/2157750897 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+3713 (secure 0 0 0) 0x7f9930076b00 con 0x7f9938141690 2023-10-18T11:42:09.661 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:09.661 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3883749389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3883749389},{"type":"v1","addr":"172.21.15.139:6836","nonce":3883749389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8827,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2038008432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2038008432},{"type":"v1","addr":"172.21.15.139:6837","nonce":2038008432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":10317,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1543508506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1543508506},{"type":"v1","addr":"172.21.15.125:6834","nonce":1543508506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":10764,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/3142355508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":3142355508},{"type":"v1","addr":"172.21.15.125:6835","nonce":3142355508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[]} 2023-10-18T11:42:09.662 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 69 2023-10-18T11:42:09.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f991c06af60 msgr2=0x7f991c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f991c06af60 0x7f991c06d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9934005fd0 tx=0x7f9934009500 comp rx=0 tx=0).stop 2023-10-18T11:42:09.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938141690 msgr2=0x7f993813cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938141690 0x7f993813cf70 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f9930009950 tx=0x7f99300038a0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 shutdown_connections 2023-10-18T11:42:09.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f991c06af60 0x7f991c06d420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9938141690 0x7f993813cf70 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993813e520 0x7f99381400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 --2- 172.21.15.125:0/2157750897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9938070af0 0x7f99380abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 >> 172.21.15.125:0/2157750897 conn(0x7f9938069930 msgr2=0x7f99380703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:09.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 shutdown_connections 2023-10-18T11:42:09.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.660+0000 7f993f8b4640 1 -- 172.21.15.125:0/2157750897 wait complete. 2023-10-18T11:42:09.673 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T11:42:09.674 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T11:42:09.675 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T11:42:09.675 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:42:09.675 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T11:42:09.676 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T11:42:09.676 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T11:42:09.676 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:42:09.677 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T11:42:09.677 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T11:42:09.678 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T11:42:09.678 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:42:09.679 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T11:42:09.679 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T11:42:09.679 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T11:42:09.679 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:42:09.684 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T11:42:09.684 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T11:42:09.686 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T11:42:09.686 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T11:42:09.691 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T11:42:09.692 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T11:42:09.693 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T11:42:09.693 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T11:42:09.728 INFO:tasks.ceph.mds.a:Started 2023-10-18T11:42:09.728 DEBUG:teuthology.parallel:result is None 2023-10-18T11:42:09.729 INFO:tasks.ceph.mds.c:Started 2023-10-18T11:42:09.730 DEBUG:teuthology.parallel:result is None 2023-10-18T11:42:09.734 INFO:tasks.ceph.mds.d:Started 2023-10-18T11:42:09.735 DEBUG:teuthology.parallel:result is None 2023-10-18T11:42:09.735 INFO:tasks.ceph.mds.b:Started 2023-10-18T11:42:09.736 DEBUG:teuthology.parallel:result is None 2023-10-18T11:42:09.736 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T11:42:09.802 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T11:42:09.803 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T11:42:09.816 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T11:42:09.817 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T11:42:09.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.840+0000 7fbc569a9640 1 -- 172.21.15.125:0/1623814284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc5015b000 msgr2=0x7fbc500d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.840+0000 7fbc569a9640 1 --2- 172.21.15.125:0/1623814284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc5015b000 0x7fbc500d1810 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009830 tx=0x7fbc4002bdf0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.840+0000 7fbc569a9640 1 -- 172.21.15.125:0/1623814284 shutdown_connections 2023-10-18T11:42:09.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.840+0000 7fbc569a9640 1 --2- 172.21.15.125:0/1623814284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc5015b000 0x7fbc500d1810 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.840+0000 7fbc569a9640 1 -- 172.21.15.125:0/1623814284 >> 172.21.15.125:0/1623814284 conn(0x7fbc500d0cf0 msgr2=0x7fbc500d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:09.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.840+0000 7fbc569a9640 1 -- 172.21.15.125:0/1623814284 shutdown_connections 2023-10-18T11:42:09.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.840+0000 7fbc569a9640 1 -- 172.21.15.125:0/1623814284 wait complete. 2023-10-18T11:42:09.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.841+0000 7fbc569a9640 1 Processor -- start 2023-10-18T11:42:09.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.841+0000 7fbc569a9640 1 -- start start 2023-10-18T11:42:09.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc569a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc500cf9e0 0x7fbc501f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:09.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc569a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc5015b000 0x7fbc501524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:09.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc569a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc5015b3e0 0x7fbc501e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:09.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc569a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500b4820 con 0x7fbc500cf9e0 2023-10-18T11:42:09.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc569a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc501529e0 con 0x7fbc5015b3e0 2023-10-18T11:42:09.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc569a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc501e3380 con 0x7fbc5015b000 2023-10-18T11:42:09.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc4ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc500cf9e0 0x7fbc501f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:09.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc4ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc500cf9e0 0x7fbc501f5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38620/0 (socket says 172.21.15.125:38620) 2023-10-18T11:42:09.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc4ffff640 1 -- 172.21.15.125:0/1959873811 learned_addr learned my addr 172.21.15.125:0/1959873811 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:09.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc4ffff640 1 -- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc5015b000 msgr2=0x7fbc501524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc4ffff640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc5015b000 0x7fbc501524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc4ffff640 1 -- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc5015b3e0 msgr2=0x7fbc501e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:09.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.842+0000 7fbc54f46640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc5015b3e0 0x7fbc501e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:09.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc4ffff640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc5015b3e0 0x7fbc501e2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:09.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc4ffff640 1 -- 172.21.15.125:0/1959873811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc40009490 con 0x7fbc500cf9e0 2023-10-18T11:42:09.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc54f46640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc5015b3e0 0x7fbc501e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:09.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc4ffff640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc500cf9e0 0x7fbc501f5020 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009780 tx=0x7fbc400038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:09.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc4cff9640 1 -- 172.21.15.125:0/1959873811 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40003ea0 con 0x7fbc500cf9e0 2023-10-18T11:42:09.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc4cff9640 1 -- 172.21.15.125:0/1959873811 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc40034550 con 0x7fbc500cf9e0 2023-10-18T11:42:09.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc501f5f50 con 0x7fbc500cf9e0 2023-10-18T11:42:09.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc4cff9640 1 -- 172.21.15.125:0/1959873811 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40034960 con 0x7fbc500cf9e0 2023-10-18T11:42:09.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.843+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc501f5990 con 0x7fbc500cf9e0 2023-10-18T11:42:09.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.844+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc500afb80 con 0x7fbc500cf9e0 2023-10-18T11:42:09.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.844+0000 7fbc4cff9640 1 -- 172.21.15.125:0/1959873811 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbc4003b040 con 0x7fbc500cf9e0 2023-10-18T11:42:09.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.845+0000 7fbc4cff9640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fbc3c06af60 0x7fbc3c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:09.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.845+0000 7fbc4cff9640 1 -- 172.21.15.125:0/1959873811 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5979+0+0 (secure 0 0 0) 0x7fbc400abef0 con 0x7fbc500cf9e0 2023-10-18T11:42:09.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.845+0000 7fbc4f7fe640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fbc3c06af60 0x7fbc3c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:09.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.845+0000 7fbc4f7fe640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fbc3c06af60 0x7fbc3c06d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc44000f60 tx=0x7fbc44007a30 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:09.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:09.849+0000 7fbc4cff9640 1 -- 172.21.15.125:0/1959873811 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc40076d80 con 0x7fbc500cf9e0 2023-10-18T11:42:10.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:10.025+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbc500690e0 con 0x7fbc500cf9e0 2023-10-18T11:42:11.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.176+0000 7fbc4cff9640 1 -- 172.21.15.125:0/1959873811 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v184) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbc40039080 con 0x7fbc500cf9e0 2023-10-18T11:42:11.179 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T11:42:11.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fbc3c06af60 msgr2=0x7fbc3c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fbc3c06af60 0x7fbc3c06d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc44000f60 tx=0x7fbc44007a30 comp rx=0 tx=0).stop 2023-10-18T11:42:11.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc500cf9e0 msgr2=0x7fbc501f5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc500cf9e0 0x7fbc501f5020 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009780 tx=0x7fbc400038a0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 shutdown_connections 2023-10-18T11:42:11.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fbc3c06af60 0x7fbc3c06d420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc5015b3e0 0x7fbc501e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc5015b000 0x7fbc501524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 --2- 172.21.15.125:0/1959873811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc500cf9e0 0x7fbc501f5020 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 >> 172.21.15.125:0/1959873811 conn(0x7fbc500d0cf0 msgr2=0x7fbc500cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:11.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 shutdown_connections 2023-10-18T11:42:11.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.179+0000 7fbc569a9640 1 -- 172.21.15.125:0/1959873811 wait complete. 2023-10-18T11:42:11.194 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T11:42:11.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.297+0000 7f8d03e90640 1 -- 172.21.15.125:0/3656994281 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc13d360 msgr2=0x7f8cfc15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.297+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3656994281 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc13d360 0x7f8cfc15a9f0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800afd0 tx=0x7f8cf802cd80 comp rx=0 tx=0).stop 2023-10-18T11:42:11.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.297+0000 7f8d03e90640 1 -- 172.21.15.125:0/3656994281 shutdown_connections 2023-10-18T11:42:11.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.297+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3656994281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8cfc140660 0x7f8cfc0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.297+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3656994281 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc13d360 0x7f8cfc15a9f0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.297+0000 7f8d03e90640 1 -- 172.21.15.125:0/3656994281 >> 172.21.15.125:0/3656994281 conn(0x7f8cfc069930 msgr2=0x7f8cfc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:11.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.297+0000 7f8d03e90640 1 -- 172.21.15.125:0/3656994281 shutdown_connections 2023-10-18T11:42:11.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.297+0000 7f8d03e90640 1 -- 172.21.15.125:0/3656994281 wait complete. 2023-10-18T11:42:11.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.298+0000 7f8d03e90640 1 Processor -- start 2023-10-18T11:42:11.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.298+0000 7f8d03e90640 1 -- start start 2023-10-18T11:42:11.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d03e90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8cfc0c9d00 0x7f8cfc0ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d03e90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cfc13d360 0x7f8cfc0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d03e90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc140660 0x7f8cfc0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d03e90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1f1300 con 0x7f8cfc140660 2023-10-18T11:42:11.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d03e90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0b1380 con 0x7f8cfc13d360 2023-10-18T11:42:11.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d03e90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0ac270 con 0x7f8cfc0c9d00 2023-10-18T11:42:11.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc140660 0x7f8cfc0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc140660 0x7f8cfc0ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38638/0 (socket says 172.21.15.125:38638) 2023-10-18T11:42:11.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 -- 172.21.15.125:0/3588792272 learned_addr learned my addr 172.21.15.125:0/3588792272 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:11.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0142b640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cfc13d360 0x7f8cfc0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d01c2c640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8cfc0c9d00 0x7f8cfc0ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 -- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8cfc0c9d00 msgr2=0x7f8cfc0ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8cfc0c9d00 0x7f8cfc0ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 -- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cfc13d360 msgr2=0x7f8cfc0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cfc13d360 0x7f8cfc0b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 -- 172.21.15.125:0/3588792272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf8007490 con 0x7f8cfc140660 2023-10-18T11:42:11.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d01c2c640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8cfc0c9d00 0x7f8cfc0ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:11.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8d0242d640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc140660 0x7f8cfc0ad9b0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400c440 tx=0x7f8cf400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:11.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8cf27fc640 1 -- 172.21.15.125:0/3588792272 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4011790 con 0x7f8cfc140660 2023-10-18T11:42:11.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.299+0000 7f8cf27fc640 1 -- 172.21.15.125:0/3588792272 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8cf40119e0 con 0x7f8cfc140660 2023-10-18T11:42:11.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.300+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc0ab9d0 con 0x7f8cfc140660 2023-10-18T11:42:11.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.300+0000 7f8cf27fc640 1 -- 172.21.15.125:0/3588792272 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4011df0 con 0x7f8cfc140660 2023-10-18T11:42:11.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.300+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc0abf90 con 0x7f8cfc140660 2023-10-18T11:42:11.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.301+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc0690e0 con 0x7f8cfc140660 2023-10-18T11:42:11.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.301+0000 7f8cf27fc640 1 -- 172.21.15.125:0/3588792272 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8cf401c460 con 0x7f8cfc140660 2023-10-18T11:42:11.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.301+0000 7f8cf27fc640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f8ce006af60 0x7f8ce006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.301+0000 7f8cf27fc640 1 -- 172.21.15.125:0/3588792272 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f8cf4089660 con 0x7f8cfc140660 2023-10-18T11:42:11.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.302+0000 7f8d01c2c640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f8ce006af60 0x7f8ce006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.302+0000 7f8d01c2c640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f8ce006af60 0x7f8ce006d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8036000 tx=0x7f8cf8004ab0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:11.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.306+0000 7f8cf27fc640 1 -- 172.21.15.125:0/3588792272 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8cf4054880 con 0x7f8cfc140660 2023-10-18T11:42:11.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.533+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8cfc15b970 con 0x7f8cfc140660 2023-10-18T11:42:11.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.533+0000 7f8cf27fc640 1 -- 172.21.15.125:0/3588792272 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v49) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8cf4057ea0 con 0x7f8cfc140660 2023-10-18T11:42:11.536 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:11.537 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:42:11.537 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T11:42:11.537 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.537 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T11:42:11.537 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T11:42:11.538 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.538 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:42:11.538 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:42:11.538 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:42:11.538 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.539 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.539 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.539 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.539 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T11:42:11.539 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T11:42:11.540 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.540 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:42:11.540 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:42:11.540 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:42:11.541 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.541 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.541 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.541 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.541 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T11:42:11.542 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T11:42:11.542 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.542 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:42:11.542 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:42:11.543 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:42:11.543 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.543 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.543 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.544 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.544 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T11:42:11.544 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T11:42:11.544 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.544 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:42:11.545 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:42:11.545 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:42:11.545 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.545 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.545 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.546 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.546 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T11:42:11.546 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T11:42:11.546 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.546 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:42:11.547 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:42:11.547 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.547 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.547 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.547 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.548 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T11:42:11.548 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T11:42:11.548 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.548 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:42:11.548 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:42:11.548 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.549 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.549 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.549 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.549 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T11:42:11.549 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T11:42:11.550 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.550 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:42:11.550 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:42:11.550 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.550 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.551 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.551 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.551 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T11:42:11.551 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T11:42:11.551 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.552 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:42:11.552 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:42:11.552 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.552 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.552 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.553 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.553 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T11:42:11.553 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T11:42:11.553 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.553 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:42:11.554 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:42:11.554 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.554 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.554 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.554 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.555 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T11:42:11.555 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T11:42:11.555 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.555 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:42:11.555 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:42:11.556 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.556 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.556 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.556 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.556 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T11:42:11.556 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T11:42:11.557 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.557 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:42:11.557 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:42:11.557 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.557 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.557 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.558 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.558 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T11:42:11.558 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T11:42:11.558 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.558 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:42:11.559 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:42:11.559 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.559 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.559 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.559 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.560 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T11:42:11.560 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T11:42:11.560 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.560 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:42:11.560 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T11:42:11.561 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T11:42:11.561 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.561 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.561 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.561 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T11:42:11.562 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T11:42:11.562 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.562 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:42:11.562 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:42:11.562 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:42:11.563 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:42:11.563 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.563 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.563 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.563 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T11:42:11.564 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T11:42:11.564 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.564 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:42:11.564 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:42:11.564 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:42:11.565 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:42:11.565 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.565 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.565 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.565 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T11:42:11.565 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T11:42:11.566 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.566 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:42:11.566 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:42:11.566 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:42:11.566 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:42:11.567 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.567 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.567 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.567 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T11:42:11.567 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T11:42:11.568 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.568 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:42:11.568 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:42:11.568 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:42:11.568 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.569 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.569 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.569 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.569 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T11:42:11.569 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T11:42:11.570 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.570 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:42:11.570 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T11:42:11.570 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.570 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.571 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:11.571 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:11.571 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T11:42:11.571 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T11:42:11.571 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:42:11.572 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:42:11.572 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T11:42:11.572 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:42:11.572 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.572 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:11.572 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T11:42:11.573 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:42:11.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f8ce006af60 msgr2=0x7f8ce006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f8ce006af60 0x7f8ce006d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8036000 tx=0x7f8cf8004ab0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc140660 msgr2=0x7f8cfc0ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc140660 0x7f8cfc0ad9b0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400c440 tx=0x7f8cf400c900 comp rx=0 tx=0).stop 2023-10-18T11:42:11.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 shutdown_connections 2023-10-18T11:42:11.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f8ce006af60 0x7f8ce006d420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cfc13d360 0x7f8cfc0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8cfc140660 0x7f8cfc0ad9b0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 --2- 172.21.15.125:0/3588792272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8cfc0c9d00 0x7f8cfc0ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 >> 172.21.15.125:0/3588792272 conn(0x7f8cfc069930 msgr2=0x7f8cfc159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:11.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 shutdown_connections 2023-10-18T11:42:11.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.535+0000 7f8d03e90640 1 -- 172.21.15.125:0/3588792272 wait complete. 2023-10-18T11:42:11.577 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:42:11.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.679+0000 7fac923a9640 1 -- 172.21.15.125:0/3140268148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 msgr2=0x7fac74004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.679+0000 7fac923a9640 1 --2- 172.21.15.125:0/3140268148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 0x7fac74004c40 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fac8000baf0 tx=0x7fac740057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.679+0000 7fac923a9640 1 -- 172.21.15.125:0/3140268148 shutdown_connections 2023-10-18T11:42:11.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.679+0000 7fac923a9640 1 --2- 172.21.15.125:0/3140268148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 0x7fac74004c40 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.679+0000 7fac923a9640 1 -- 172.21.15.125:0/3140268148 >> 172.21.15.125:0/3140268148 conn(0x7fac8c0d0c50 msgr2=0x7fac8c0d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:11.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.679+0000 7fac923a9640 1 -- 172.21.15.125:0/3140268148 shutdown_connections 2023-10-18T11:42:11.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.679+0000 7fac923a9640 1 -- 172.21.15.125:0/3140268148 wait complete. 2023-10-18T11:42:11.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.680+0000 7fac923a9640 1 Processor -- start 2023-10-18T11:42:11.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.680+0000 7fac923a9640 1 -- start start 2023-10-18T11:42:11.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac923a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 0x7fac8c1f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac923a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac8c0c7260 0x7fac8c0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac923a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c0b26d0 0x7fac8c0b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac923a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c147460 con 0x7fac740047c0 2023-10-18T11:42:11.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac923a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c0b3090 con 0x7fac8c0b26d0 2023-10-18T11:42:11.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac923a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c1467e0 con 0x7fac8c0c7260 2023-10-18T11:42:11.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 0x7fac8c1f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 0x7fac8c1f0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38664/0 (socket says 172.21.15.125:38664) 2023-10-18T11:42:11.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 -- 172.21.15.125:0/1462600487 learned_addr learned my addr 172.21.15.125:0/1462600487 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:11.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac90946640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c0b26d0 0x7fac8c0b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8b7fe640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac8c0c7260 0x7fac8c0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 -- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac8c0c7260 msgr2=0x7fac8c0c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac8c0c7260 0x7fac8c0c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 -- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c0b26d0 msgr2=0x7fac8c0b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c0b26d0 0x7fac8c0b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 -- 172.21.15.125:0/1462600487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c146ac0 con 0x7fac740047c0 2023-10-18T11:42:11.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac8bfff640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 0x7fac8c1f0d90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fac7c001eb0 tx=0x7fac7c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:11.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac88ff9640 1 -- 172.21.15.125:0/1462600487 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c004460 con 0x7fac740047c0 2023-10-18T11:42:11.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac88ff9640 1 -- 172.21.15.125:0/1462600487 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac7c00ee90 con 0x7fac740047c0 2023-10-18T11:42:11.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac88ff9640 1 -- 172.21.15.125:0/1462600487 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00caa0 con 0x7fac740047c0 2023-10-18T11:42:11.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8c146da0 con 0x7fac740047c0 2023-10-18T11:42:11.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.681+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8c0c6d70 con 0x7fac740047c0 2023-10-18T11:42:11.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.682+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8c0c6f00 con 0x7fac740047c0 2023-10-18T11:42:11.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.683+0000 7fac88ff9640 1 -- 172.21.15.125:0/1462600487 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fac7c00cc00 con 0x7fac740047c0 2023-10-18T11:42:11.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.683+0000 7fac88ff9640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fac7806af90 0x7fac7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.683+0000 7fac88ff9640 1 -- 172.21.15.125:0/1462600487 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5205+0+0 (secure 0 0 0) 0x7fac7c08a110 con 0x7fac740047c0 2023-10-18T11:42:11.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.684+0000 7fac8b7fe640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fac7806af90 0x7fac7806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.684+0000 7fac8b7fe640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fac7806af90 0x7fac7806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac80037000 tx=0x7fac8002e040 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:11.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.688+0000 7fac88ff9640 1 -- 172.21.15.125:0/1462600487 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac7c055190 con 0x7fac740047c0 2023-10-18T11:42:11.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.854+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac8c0d0440 con 0x7fac740047c0 2023-10-18T11:42:11.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.855+0000 7fac88ff9640 1 -- 172.21.15.125:0/1462600487 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+13164 (secure 0 0 0) 0x7fac7c0587b0 con 0x7fac740047c0 2023-10-18T11:42:11.857 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:11.857 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":184,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:42:11.175997+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:42:11.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fac7806af90 msgr2=0x7fac7806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fac7806af90 0x7fac7806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac80037000 tx=0x7fac8002e040 comp rx=0 tx=0).stop 2023-10-18T11:42:11.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 msgr2=0x7fac8c1f0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 0x7fac8c1f0d90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fac7c001eb0 tx=0x7fac7c00eb80 comp rx=0 tx=0).stop 2023-10-18T11:42:11.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 shutdown_connections 2023-10-18T11:42:11.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fac7806af90 0x7fac7806d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac8c0b26d0 0x7fac8c0b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac8c0c7260 0x7fac8c0c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 --2- 172.21.15.125:0/1462600487 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac740047c0 0x7fac8c1f0d90 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 >> 172.21.15.125:0/1462600487 conn(0x7fac8c0d0c50 msgr2=0x7fac8c15d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:11.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 shutdown_connections 2023-10-18T11:42:11.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.857+0000 7fac923a9640 1 -- 172.21.15.125:0/1462600487 wait complete. 2023-10-18T11:42:11.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:42:11.871 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:42:11.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.973+0000 7f94ca8b4640 1 -- 172.21.15.125:0/3636190526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 msgr2=0x7f94c40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.973+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/3636190526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 0x7f94c40d1910 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f94c415b680 tx=0x7f94b802bd30 comp rx=0 tx=0).stop 2023-10-18T11:42:11.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.973+0000 7f94ca8b4640 1 -- 172.21.15.125:0/3636190526 shutdown_connections 2023-10-18T11:42:11.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.973+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/3636190526 >> v2:172.21.15.139:3300/0 conn(0x7f94c40ceb40 0x7f94c40b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.973+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/3636190526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 0x7f94c40d1910 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.973+0000 7f94ca8b4640 1 -- 172.21.15.125:0/3636190526 >> 172.21.15.125:0/3636190526 conn(0x7f94c40d0d60 msgr2=0x7f94c40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:11.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.973+0000 7f94ca8b4640 1 -- 172.21.15.125:0/3636190526 shutdown_connections 2023-10-18T11:42:11.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.973+0000 7f94ca8b4640 1 -- 172.21.15.125:0/3636190526 wait complete. 2023-10-18T11:42:11.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.974+0000 7f94ca8b4640 1 Processor -- start 2023-10-18T11:42:11.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94ca8b4640 1 -- start start 2023-10-18T11:42:11.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94ca8b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40ceb40 0x7f94c41f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94ca8b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40cf340 0x7f94c41f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94ca8b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 0x7f94c4151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94ca8b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40c8360 con 0x7f94c415b2a0 2023-10-18T11:42:11.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94ca8b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41f1eb0 con 0x7f94c40ceb40 2023-10-18T11:42:11.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94ca8b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41f2190 con 0x7f94c40cf340 2023-10-18T11:42:11.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94c8e51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 0x7f94c4151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94c8e51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 0x7f94c4151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38680/0 (socket says 172.21.15.125:38680) 2023-10-18T11:42:11.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.975+0000 7f94c8e51640 1 -- 172.21.15.125:0/820946149 learned_addr learned my addr 172.21.15.125:0/820946149 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:11.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c3fff640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40ceb40 0x7f94c41f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c37fe640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40cf340 0x7f94c41f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c8e51640 1 -- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40cf340 msgr2=0x7f94c41f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c8e51640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40cf340 0x7f94c41f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c8e51640 1 -- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40ceb40 msgr2=0x7f94c41f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:11.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c8e51640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40ceb40 0x7f94c41f57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:11.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c8e51640 1 -- 172.21.15.125:0/820946149 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009560 con 0x7f94c415b2a0 2023-10-18T11:42:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c8e51640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 0x7f94c4151200 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00eaa0 tx=0x7f94bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c0ff9640 1 -- 172.21.15.125:0/820946149 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00cdb0 con 0x7f94c415b2a0 2023-10-18T11:42:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c0ff9640 1 -- 172.21.15.125:0/820946149 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94bc004b20 con 0x7f94c415b2a0 2023-10-18T11:42:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c41f2470 con 0x7f94c415b2a0 2023-10-18T11:42:11.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94c0ff9640 1 -- 172.21.15.125:0/820946149 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc005020 con 0x7f94c415b2a0 2023-10-18T11:42:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.976+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c41f1b70 con 0x7f94c415b2a0 2023-10-18T11:42:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.977+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c40c87f0 con 0x7f94c415b2a0 2023-10-18T11:42:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.977+0000 7f94c0ff9640 1 -- 172.21.15.125:0/820946149 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f94bc005180 con 0x7f94c415b2a0 2023-10-18T11:42:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.978+0000 7f94c0ff9640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f94b006af90 0x7f94b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:11.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.978+0000 7f94c0ff9640 1 -- 172.21.15.125:0/820946149 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f94bc08a020 con 0x7f94c415b2a0 2023-10-18T11:42:11.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.978+0000 7f94c3fff640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f94b006af90 0x7f94b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:11.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.979+0000 7f94c3fff640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f94b006af90 0x7f94b006d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94b4003d90 tx=0x7f94b4009040 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:11.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:11.982+0000 7f94c0ff9640 1 -- 172.21.15.125:0/820946149 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94c40c87f0 con 0x7f94c415b2a0 2023-10-18T11:42:12.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.148+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94c40c87f0 con 0x7f94c415b2a0 2023-10-18T11:42:12.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.149+0000 7f94c0ff9640 1 -- 172.21.15.125:0/820946149 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v184) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94c40c87f0 con 0x7f94c415b2a0 2023-10-18T11:42:12.151 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:42:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f94b006af90 msgr2=0x7f94b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f94b006af90 0x7f94b006d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94b4003d90 tx=0x7f94b4009040 comp rx=0 tx=0).stop 2023-10-18T11:42:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 msgr2=0x7f94c4151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 0x7f94c4151200 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00eaa0 tx=0x7f94bc00ef60 comp rx=0 tx=0).stop 2023-10-18T11:42:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 shutdown_connections 2023-10-18T11:42:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f94b006af90 0x7f94b006d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94c415b2a0 0x7f94c4151200 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94c40cf340 0x7f94c41f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 --2- 172.21.15.125:0/820946149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94c40ceb40 0x7f94c41f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 >> 172.21.15.125:0/820946149 conn(0x7f94c40d0d60 msgr2=0x7f94c406b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 shutdown_connections 2023-10-18T11:42:12.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.151+0000 7f94ca8b4640 1 -- 172.21.15.125:0/820946149 wait complete. 2023-10-18T11:42:12.164 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T11:42:12.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.264+0000 7f876e333640 1 -- 172.21.15.125:0/3857426973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 msgr2=0x7f87500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.264+0000 7f876e333640 1 --2- 172.21.15.125:0/3857426973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 0x7f87500047c0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c260 tx=0x7f87500057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.264+0000 7f876c8d0640 1 -- 172.21.15.125:0/3857426973 reap_dead start 2023-10-18T11:42:12.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.264+0000 7f876e333640 1 -- 172.21.15.125:0/3857426973 shutdown_connections 2023-10-18T11:42:12.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.264+0000 7f876e333640 1 -- 172.21.15.125:0/3857426973 >> 172.21.15.125:0/3857426973 conn(0x7f8768069930 msgr2=0x7f8768069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:12.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.265+0000 7f876e333640 1 -- 172.21.15.125:0/3857426973 shutdown_connections 2023-10-18T11:42:12.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.265+0000 7f876e333640 1 -- 172.21.15.125:0/3857426973 wait complete. 2023-10-18T11:42:12.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.266+0000 7f876e333640 1 Processor -- start 2023-10-18T11:42:12.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.266+0000 7f876e333640 1 -- start start 2023-10-18T11:42:12.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.266+0000 7f876e333640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 0x7f87681578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:12.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.266+0000 7f876e333640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876814b3e0 0x7f8768157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:12.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.266+0000 7f876e333640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8768148ef0 0x7f8768149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:12.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.266+0000 7f876e333640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8768143d00 con 0x7f8750003ef0 2023-10-18T11:42:12.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f876e333640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8768141b70 con 0x7f876814b3e0 2023-10-18T11:42:12.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f876e333640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8768141e50 con 0x7f8768148ef0 2023-10-18T11:42:12.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 0x7f87681578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:12.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 0x7f87681578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38706/0 (socket says 172.21.15.125:38706) 2023-10-18T11:42:12.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 -- 172.21.15.125:0/1433144576 learned_addr learned my addr 172.21.15.125:0/1433144576 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:12.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f87677fe640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876814b3e0 0x7f8768157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:12.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f876c8d0640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8768148ef0 0x7f8768149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:12.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 -- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8768148ef0 msgr2=0x7f8768149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8768148ef0 0x7f8768149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 -- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876814b3e0 msgr2=0x7f8768157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876814b3e0 0x7f8768157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 -- 172.21.15.125:0/1433144576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8768142130 con 0x7f8750003ef0 2023-10-18T11:42:12.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f87677fe640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876814b3e0 0x7f8768157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:42:12.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8767fff640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 0x7f87681578d0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f8758011a50 tx=0x7f8758011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:12.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8764ff9640 1 -- 172.21.15.125:0/1433144576 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800fdf0 con 0x7f8750003ef0 2023-10-18T11:42:12.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8764ff9640 1 -- 172.21.15.125:0/1433144576 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8758012450 con 0x7f8750003ef0 2023-10-18T11:42:12.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8768142410 con 0x7f8750003ef0 2023-10-18T11:42:12.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f8764ff9640 1 -- 172.21.15.125:0/1433144576 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758012840 con 0x7f8750003ef0 2023-10-18T11:42:12.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.267+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8768149a90 con 0x7f8750003ef0 2023-10-18T11:42:12.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.268+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8768149d70 con 0x7f8750003ef0 2023-10-18T11:42:12.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.269+0000 7f8764ff9640 1 -- 172.21.15.125:0/1433144576 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f87580129a0 con 0x7f8750003ef0 2023-10-18T11:42:12.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.269+0000 7f8764ff9640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f875406af60 0x7f875406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:12.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.269+0000 7f8764ff9640 1 -- 172.21.15.125:0/1433144576 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f875808a5a0 con 0x7f8750003ef0 2023-10-18T11:42:12.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.269+0000 7f87677fe640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f875406af60 0x7f875406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:12.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.270+0000 7f87677fe640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f875406af60 0x7f875406d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f875c034000 tx=0x7f875c001a70 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:12.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.273+0000 7f8764ff9640 1 -- 172.21.15.125:0/1433144576 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87580557e0 con 0x7f8750003ef0 2023-10-18T11:42:12.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.441+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f87680690e0 con 0x7f8750003ef0 2023-10-18T11:42:12.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.441+0000 7f8764ff9640 1 -- 172.21.15.125:0/1433144576 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v184) v1 ==== 84+0+65 (secure 0 0 0) 0x7f875801f090 con 0x7f8750003ef0 2023-10-18T11:42:12.444 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:12.444 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T11:42:12.444 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:12.444 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T11:42:12.445 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T11:42:12.445 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:12.445 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T11:42:12.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f875406af60 msgr2=0x7f875406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f875406af60 0x7f875406d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f875c034000 tx=0x7f875c001a70 comp rx=0 tx=0).stop 2023-10-18T11:42:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 msgr2=0x7f87681578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 0x7f87681578d0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f8758011a50 tx=0x7f8758011f10 comp rx=0 tx=0).stop 2023-10-18T11:42:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 shutdown_connections 2023-10-18T11:42:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f875406af60 0x7f875406d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876814b3e0 0x7f8768157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8768148ef0 0x7f8768149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 --2- 172.21.15.125:0/1433144576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750003ef0 0x7f87681578d0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 >> 172.21.15.125:0/1433144576 conn(0x7f8768069930 msgr2=0x7f87680ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:12.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 shutdown_connections 2023-10-18T11:42:12.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.443+0000 7f876e333640 1 -- 172.21.15.125:0/1433144576 wait complete. 2023-10-18T11:42:12.456 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T11:42:12.456 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T11:42:12.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.563+0000 7f2d55ae5640 1 -- 172.21.15.125:0/1135212066 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d500ad370 msgr2=0x7f2d500ad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.563+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/1135212066 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d500ad370 0x7f2d500ad800 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005fd0 tx=0x7f2d4802c2e0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.563+0000 7f2d55ae5640 1 -- 172.21.15.125:0/1135212066 shutdown_connections 2023-10-18T11:42:12.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.563+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/1135212066 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d500ad370 0x7f2d500ad800 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.563+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/1135212066 >> v2:172.21.15.139:3300/0 conn(0x7f2d5006f0a0 0x7f2d5006ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.563+0000 7f2d55ae5640 1 -- 172.21.15.125:0/1135212066 >> 172.21.15.125:0/1135212066 conn(0x7f2d500b0670 msgr2=0x7f2d500b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:12.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.563+0000 7f2d55ae5640 1 -- 172.21.15.125:0/1135212066 shutdown_connections 2023-10-18T11:42:12.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.563+0000 7f2d55ae5640 1 -- 172.21.15.125:0/1135212066 wait complete. 2023-10-18T11:42:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.564+0000 7f2d55ae5640 1 Processor -- start 2023-10-18T11:42:12.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.564+0000 7f2d55ae5640 1 -- start start 2023-10-18T11:42:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.564+0000 7f2d55ae5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d5006d2d0 0x7f2d501512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.564+0000 7f2d55ae5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d5006f0a0 0x7f2d500caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.564+0000 7f2d55ae5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d500ad370 0x7f2d50158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:12.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.564+0000 7f2d55ae5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500caf70 con 0x7f2d500ad370 2023-10-18T11:42:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.564+0000 7f2d55ae5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500c6080 con 0x7f2d5006d2d0 2023-10-18T11:42:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.564+0000 7f2d55ae5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d501f2d00 con 0x7f2d5006f0a0 2023-10-18T11:42:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d500ad370 0x7f2d50158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d500ad370 0x7f2d50158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38712/0 (socket says 172.21.15.125:38712) 2023-10-18T11:42:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 -- 172.21.15.125:0/2863811405 learned_addr learned my addr 172.21.15.125:0/2863811405 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:12.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4f7fe640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d5006d2d0 0x7f2d501512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d47fff640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d5006f0a0 0x7f2d500caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 -- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d5006f0a0 msgr2=0x7f2d500caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d5006f0a0 0x7f2d500caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 -- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d5006d2d0 msgr2=0x7f2d501512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:12.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d5006d2d0 0x7f2d501512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 -- 172.21.15.125:0/2863811405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d48009900 con 0x7f2d500ad370 2023-10-18T11:42:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4f7fe640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d5006d2d0 0x7f2d501512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:42:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4ffff640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d500ad370 0x7f2d50158c40 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009bf0 tx=0x7f2d48035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4cff9640 1 -- 172.21.15.125:0/2863811405 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48004240 con 0x7f2d500ad370 2023-10-18T11:42:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4cff9640 1 -- 172.21.15.125:0/2863811405 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2d48004490 con 0x7f2d500ad370 2023-10-18T11:42:12.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d500cd310 con 0x7f2d500ad370 2023-10-18T11:42:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d4cff9640 1 -- 172.21.15.125:0/2863811405 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48004860 con 0x7f2d500ad370 2023-10-18T11:42:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.565+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d501f32c0 con 0x7f2d500ad370 2023-10-18T11:42:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.566+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d500ce1d0 con 0x7f2d500ad370 2023-10-18T11:42:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.567+0000 7f2d4cff9640 1 -- 172.21.15.125:0/2863811405 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2d4803b050 con 0x7f2d500ad370 2023-10-18T11:42:12.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.567+0000 7f2d4cff9640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2d3006af60 0x7f2d3006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.567+0000 7f2d4cff9640 1 -- 172.21.15.125:0/2863811405 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5205+0+0 (secure 0 0 0) 0x7f2d480ac160 con 0x7f2d500ad370 2023-10-18T11:42:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.567+0000 7f2d4f7fe640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2d3006af60 0x7f2d3006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:12.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.568+0000 7f2d4f7fe640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2d3006af60 0x7f2d3006d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2d40000f60 tx=0x7f2d40009040 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:12.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.571+0000 7f2d4cff9640 1 -- 172.21.15.125:0/2863811405 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d48077430 con 0x7f2d500ad370 2023-10-18T11:42:12.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:12.737+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2d500c61f0 con 0x7f2d500ad370 2023-10-18T11:42:13.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.558+0000 7f2d4cff9640 1 -- 172.21.15.125:0/2863811405 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v185) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2d48003e60 con 0x7f2d500ad370 2023-10-18T11:42:13.561 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T11:42:13.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2d3006af60 msgr2=0x7f2d3006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:13.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2d3006af60 0x7f2d3006d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2d40000f60 tx=0x7f2d40009040 comp rx=0 tx=0).stop 2023-10-18T11:42:13.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d500ad370 msgr2=0x7f2d50158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d500ad370 0x7f2d50158c40 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009bf0 tx=0x7f2d48035b50 comp rx=0 tx=0).stop 2023-10-18T11:42:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 shutdown_connections 2023-10-18T11:42:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f2d3006af60 0x7f2d3006d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d500ad370 0x7f2d50158c40 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:13.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2d5006f0a0 0x7f2d500caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:13.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 --2- 172.21.15.125:0/2863811405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d5006d2d0 0x7f2d501512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:13.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.561+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 >> 172.21.15.125:0/2863811405 conn(0x7f2d500b0670 msgr2=0x7f2d500ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:13.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.562+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 shutdown_connections 2023-10-18T11:42:13.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.562+0000 7f2d55ae5640 1 -- 172.21.15.125:0/2863811405 wait complete. 2023-10-18T11:42:13.582 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T11:42:13.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.696+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/1961551276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc004d00 msgr2=0x7f5fcc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.696+0000 7f5fe4ae7640 1 --2- 172.21.15.125:0/1961551276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fcc004d00 0x7f5fcc005180 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800c640 tx=0x7f5fcc0057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.696+0000 7f5fded9d640 1 -- 172.21.15.125:0/1961551276 reap_dead start 2023-10-18T11:42:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.696+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/1961551276 shutdown_connections 2023-10-18T11:42:13.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.696+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/1961551276 >> 172.21.15.125:0/1961551276 conn(0x7f5fe00d0d60 msgr2=0x7f5fe00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:13.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.696+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/1961551276 shutdown_connections 2023-10-18T11:42:13.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.696+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/1961551276 wait complete. 2023-10-18T11:42:13.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.697+0000 7f5fe4ae7640 1 Processor -- start 2023-10-18T11:42:13.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.697+0000 7f5fe4ae7640 1 -- start start 2023-10-18T11:42:13.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fe4ae7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc004d00 0x7f5fe00c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:13.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fe4ae7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fe0159840 0x7f5fe00b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:13.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fe4ae7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fe00b5e30 0x7f5fe015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:13.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fe4ae7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe01e4060 con 0x7f5fe0159840 2023-10-18T11:42:13.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fe4ae7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00b62b0 con 0x7f5fe00b5e30 2023-10-18T11:42:13.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fe4ae7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe015d880 con 0x7f5fcc004d00 2023-10-18T11:42:13.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fe0159840 0x7f5fe00b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:13.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fe0159840 0x7f5fe00b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38730/0 (socket says 172.21.15.125:38730) 2023-10-18T11:42:13.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fd7fff640 1 -- 172.21.15.125:0/2210110935 learned_addr learned my addr 172.21.15.125:0/2210110935 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:13.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fded9d640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fe00b5e30 0x7f5fe015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:13.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fde59c640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc004d00 0x7f5fe00c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:13.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fd7fff640 1 -- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc004d00 msgr2=0x7f5fe00c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:13.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fd7fff640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc004d00 0x7f5fe00c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:13.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fd7fff640 1 -- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fe00b5e30 msgr2=0x7f5fe015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:13.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fd7fff640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fe00b5e30 0x7f5fe015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:13.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fd7fff640 1 -- 172.21.15.125:0/2210110935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00c9650 con 0x7f5fe0159840 2023-10-18T11:42:13.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.698+0000 7f5fded9d640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fe00b5e30 0x7f5fe015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:13.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.699+0000 7f5fd7fff640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fe0159840 0x7f5fe00b58f0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8003ea0 tx=0x7f5fc8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:13.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.699+0000 7f5fd6ffd640 1 -- 172.21.15.125:0/2210110935 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8006110 con 0x7f5fe0159840 2023-10-18T11:42:13.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.699+0000 7f5fd6ffd640 1 -- 172.21.15.125:0/2210110935 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5fc8011840 con 0x7f5fe0159840 2023-10-18T11:42:13.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.699+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe015d5a0 con 0x7f5fe0159840 2023-10-18T11:42:13.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.699+0000 7f5fd6ffd640 1 -- 172.21.15.125:0/2210110935 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8011c30 con 0x7f5fe0159840 2023-10-18T11:42:13.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.699+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe01e3c80 con 0x7f5fe0159840 2023-10-18T11:42:13.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.700+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe00690e0 con 0x7f5fe0159840 2023-10-18T11:42:13.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.700+0000 7f5fd6ffd640 1 -- 172.21.15.125:0/2210110935 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5fc80103c0 con 0x7f5fe0159840 2023-10-18T11:42:13.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.701+0000 7f5fd6ffd640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f5fc006af60 0x7f5fc006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:13.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.701+0000 7f5fd6ffd640 1 -- 172.21.15.125:0/2210110935 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5575+0+0 (secure 0 0 0) 0x7f5fc808a740 con 0x7f5fe0159840 2023-10-18T11:42:13.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.701+0000 7f5fde59c640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f5fc006af60 0x7f5fc006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:13.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.701+0000 7f5fde59c640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f5fc006af60 0x7f5fc006d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009a90 tx=0x7f5fd0005f90 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:13.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.705+0000 7f5fd6ffd640 1 -- 172.21.15.125:0/2210110935 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fc80556e0 con 0x7f5fe0159840 2023-10-18T11:42:13.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:13.872+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5fe00d0b40 con 0x7f5fe0159840 2023-10-18T11:42:14.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.560+0000 7f5fd6ffd640 1 -- 172.21.15.125:0/2210110935 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v186) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5fc801f020 con 0x7f5fe0159840 2023-10-18T11:42:14.563 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T11:42:14.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f5fc006af60 msgr2=0x7f5fc006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:14.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f5fc006af60 0x7f5fc006d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009a90 tx=0x7f5fd0005f90 comp rx=0 tx=0).stop 2023-10-18T11:42:14.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fe0159840 msgr2=0x7f5fe00b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:14.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fe0159840 0x7f5fe00b58f0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8003ea0 tx=0x7f5fc8003f90 comp rx=0 tx=0).stop 2023-10-18T11:42:14.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 shutdown_connections 2023-10-18T11:42:14.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f5fc006af60 0x7f5fc006d420 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009a90 tx=0x7f5fd0005f90 comp rx=0 tx=0).stop 2023-10-18T11:42:14.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fe00b5e30 0x7f5fe015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:14.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fe0159840 0x7f5fe00b58f0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:14.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 --2- 172.21.15.125:0/2210110935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fcc004d00 0x7f5fe00c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:14.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.564+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 >> 172.21.15.125:0/2210110935 conn(0x7f5fe00d0d60 msgr2=0x7f5fe015a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:14.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.569+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 shutdown_connections 2023-10-18T11:42:14.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.572+0000 7f5fe4ae7640 1 -- 172.21.15.125:0/2210110935 wait complete. 2023-10-18T11:42:14.595 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T11:42:14.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.702+0000 7ff5f6810640 1 -- 172.21.15.125:0/881164890 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 msgr2=0x7ff5f00b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:14.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.702+0000 7ff5f6810640 1 --2- 172.21.15.125:0/881164890 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 0x7ff5f00b0530 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4005fd0 tx=0x7ff5e402d0b0 comp rx=0 tx=0).stop 2023-10-18T11:42:14.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.702+0000 7ff5f6810640 1 -- 172.21.15.125:0/881164890 shutdown_connections 2023-10-18T11:42:14.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.702+0000 7ff5f6810640 1 --2- 172.21.15.125:0/881164890 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 0x7ff5f00b0530 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:14.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.702+0000 7ff5f6810640 1 --2- 172.21.15.125:0/881164890 >> v2:172.21.15.139:3300/0 conn(0x7ff5f0157220 0x7ff5f00ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:14.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.702+0000 7ff5f6810640 1 -- 172.21.15.125:0/881164890 >> 172.21.15.125:0/881164890 conn(0x7ff5f0069930 msgr2=0x7ff5f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:14.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.702+0000 7ff5f6810640 1 -- 172.21.15.125:0/881164890 shutdown_connections 2023-10-18T11:42:14.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.702+0000 7ff5f6810640 1 -- 172.21.15.125:0/881164890 wait complete. 2023-10-18T11:42:14.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.703+0000 7ff5f6810640 1 Processor -- start 2023-10-18T11:42:14.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.704+0000 7ff5f6810640 1 -- start start 2023-10-18T11:42:14.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.704+0000 7ff5f6810640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 0x7ff5f01e04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:14.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.704+0000 7ff5f6810640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5f00afea0 0x7ff5f0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:14.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.704+0000 7ff5f6810640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5f0157220 0x7ff5f015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:14.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.704+0000 7ff5f6810640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f01e1a20 con 0x7ff5f00afea0 2023-10-18T11:42:14.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.704+0000 7ff5f6810640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f01e0da0 con 0x7ff5f0157220 2023-10-18T11:42:14.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.704+0000 7ff5f6810640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f01e1080 con 0x7ff5f00afa20 2023-10-18T11:42:14.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5f00afea0 0x7ff5f0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:14.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5f00afea0 0x7ff5f0159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38742/0 (socket says 172.21.15.125:38742) 2023-10-18T11:42:14.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 -- 172.21.15.125:0/1701949133 learned_addr learned my addr 172.21.15.125:0/1701949133 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:14.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5f4dad640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5f0157220 0x7ff5f015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:14.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5effff640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 0x7ff5f01e04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:14.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 -- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 msgr2=0x7ff5f01e04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:14.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 0x7ff5f01e04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:14.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 -- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5f0157220 msgr2=0x7ff5f015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:14.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5f0157220 0x7ff5f015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:14.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 -- 172.21.15.125:0/1701949133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0009490 con 0x7ff5f00afea0 2023-10-18T11:42:14.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5effff640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 0x7ff5f01e04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:42:14.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ef7fe640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5f00afea0 0x7ff5f0159d70 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4037040 tx=0x7ff5e40045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:14.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ecff9640 1 -- 172.21.15.125:0/1701949133 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4004d30 con 0x7ff5f00afea0 2023-10-18T11:42:14.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ecff9640 1 -- 172.21.15.125:0/1701949133 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5e40035f0 con 0x7ff5f00afea0 2023-10-18T11:42:14.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e400a670 con 0x7ff5f00afea0 2023-10-18T11:42:14.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5ecff9640 1 -- 172.21.15.125:0/1701949133 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40039a0 con 0x7ff5f00afea0 2023-10-18T11:42:14.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.705+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f01e1640 con 0x7ff5f00afea0 2023-10-18T11:42:14.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.706+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f00690e0 con 0x7ff5f00afea0 2023-10-18T11:42:14.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.707+0000 7ff5ecff9640 1 -- 172.21.15.125:0/1701949133 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff5e4003b00 con 0x7ff5f00afea0 2023-10-18T11:42:14.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.707+0000 7ff5ecff9640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff5dc06af60 0x7ff5dc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:14.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.707+0000 7ff5ecff9640 1 -- 172.21.15.125:0/1701949133 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5957+0+0 (secure 0 0 0) 0x7ff5e40ac9c0 con 0x7ff5f00afea0 2023-10-18T11:42:14.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.708+0000 7ff5effff640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff5dc06af60 0x7ff5dc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:14.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.708+0000 7ff5effff640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff5dc06af60 0x7ff5dc06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009db0 tx=0x7ff5e0005f90 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:14.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.712+0000 7ff5ecff9640 1 -- 172.21.15.125:0/1701949133 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5e4077870 con 0x7ff5f00afea0 2023-10-18T11:42:14.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:14.913+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff5f01e11c0 con 0x7ff5f00afea0 2023-10-18T11:42:15.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.579+0000 7ff5ecff9640 1 -- 172.21.15.125:0/1701949133 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v71) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff5f01e11c0 con 0x7ff5f00afea0 2023-10-18T11:42:15.582 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 23 and data pool 24 2023-10-18T11:42:15.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff5dc06af60 msgr2=0x7ff5dc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:15.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff5dc06af60 0x7ff5dc06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009db0 tx=0x7ff5e0005f90 comp rx=0 tx=0).stop 2023-10-18T11:42:15.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5f00afea0 msgr2=0x7ff5f0159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:15.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5f00afea0 0x7ff5f0159d70 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4037040 tx=0x7ff5e40045d0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 shutdown_connections 2023-10-18T11:42:15.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff5dc06af60 0x7ff5dc06d420 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009db0 tx=0x7ff5e0005f90 comp rx=0 tx=0).stop 2023-10-18T11:42:15.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5f0157220 0x7ff5f015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5f00afea0 0x7ff5f0159d70 secure :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4037040 tx=0x7ff5e40045d0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 --2- 172.21.15.125:0/1701949133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5f00afa20 0x7ff5f01e04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.582+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 >> 172.21.15.125:0/1701949133 conn(0x7ff5f0069930 msgr2=0x7ff5f00ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:15.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.583+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 shutdown_connections 2023-10-18T11:42:15.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.584+0000 7ff5f6810640 1 -- 172.21.15.125:0/1701949133 wait complete. 2023-10-18T11:42:15.612 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:42:15.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.715+0000 7f58a6997640 1 -- 172.21.15.125:0/3971054353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58880047c0 msgr2=0x7f5888004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:15.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.715+0000 7f58a6997640 1 --2- 172.21.15.125:0/3971054353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58880047c0 0x7f5888004c40 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f5894009b50 tx=0x7f58880057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.715+0000 7f58a6997640 1 -- 172.21.15.125:0/3971054353 shutdown_connections 2023-10-18T11:42:15.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.715+0000 7f58a6997640 1 --2- 172.21.15.125:0/3971054353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58880047c0 0x7f5888004c40 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.715+0000 7f58a6997640 1 -- 172.21.15.125:0/3971054353 >> 172.21.15.125:0/3971054353 conn(0x7f58a00d0d60 msgr2=0x7f58a00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:15.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.715+0000 7f58a6997640 1 -- 172.21.15.125:0/3971054353 shutdown_connections 2023-10-18T11:42:15.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.715+0000 7f58a6997640 1 -- 172.21.15.125:0/3971054353 wait complete. 2023-10-18T11:42:15.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.716+0000 7f58a6997640 1 Processor -- start 2023-10-18T11:42:15.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.716+0000 7f58a6997640 1 -- start start 2023-10-18T11:42:15.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f58a6997640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58880047c0 0x7f58a00c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:15.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f58a6997640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58a015c860 0x7f58a014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:15.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f58a6997640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58a006a790 0x7f58a006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:15.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f58a6997640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a006b150 con 0x7f58a015c860 2023-10-18T11:42:15.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f58a6997640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01f4e60 con 0x7f58880047c0 2023-10-18T11:42:15.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f58a6997640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01f5140 con 0x7f58a006a790 2023-10-18T11:42:15.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58a015c860 0x7f58a014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:15.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58a015c860 0x7f58a014c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38764/0 (socket says 172.21.15.125:38764) 2023-10-18T11:42:15.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 -- 172.21.15.125:0/1686398178 learned_addr learned my addr 172.21.15.125:0/1686398178 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:15.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589ffff640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58880047c0 0x7f58a00c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:15.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f58a4f34640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58a006a790 0x7f58a006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:15.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 -- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58a006a790 msgr2=0x7f58a006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:15.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58a006a790 0x7f58a006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 -- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58880047c0 msgr2=0x7f58a00c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:15.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58880047c0 0x7f58a00c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 -- 172.21.15.125:0/1686398178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a01f5420 con 0x7f58a015c860 2023-10-18T11:42:15.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589f7fe640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58a015c860 0x7f58a014c080 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f589402cb70 tx=0x7f5894033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:15.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589cff9640 1 -- 172.21.15.125:0/1686398178 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894004440 con 0x7f58a015c860 2023-10-18T11:42:15.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589cff9640 1 -- 172.21.15.125:0/1686398178 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f58940035f0 con 0x7f58a015c860 2023-10-18T11:42:15.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a01f5700 con 0x7f58a015c860 2023-10-18T11:42:15.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.717+0000 7f589cff9640 1 -- 172.21.15.125:0/1686398178 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58940039e0 con 0x7f58a015c860 2023-10-18T11:42:15.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.718+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a014c5c0 con 0x7f58a015c860 2023-10-18T11:42:15.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.719+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a00690e0 con 0x7f58a015c860 2023-10-18T11:42:15.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.719+0000 7f589cff9640 1 -- 172.21.15.125:0/1686398178 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5894003b40 con 0x7f58a015c860 2023-10-18T11:42:15.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.719+0000 7f589cff9640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f588c06af90 0x7f588c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:15.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.719+0000 7f589cff9640 1 -- 172.21.15.125:0/1686398178 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f58940ac600 con 0x7f58a015c860 2023-10-18T11:42:15.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.720+0000 7f589ffff640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f588c06af90 0x7f588c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:15.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.720+0000 7f589ffff640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f588c06af90 0x7f588c06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890006c60 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:15.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.724+0000 7f589cff9640 1 -- 172.21.15.125:0/1686398178 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5894077470 con 0x7f58a015c860 2023-10-18T11:42:15.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.890+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58a00d0b40 con 0x7f58a015c860 2023-10-18T11:42:15.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.891+0000 7f589cff9640 1 -- 172.21.15.125:0/1686398178 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f589407aa90 con 0x7f58a015c860 2023-10-18T11:42:15.894 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:15.894 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":187,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:42:14.915216+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:42:12.758545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-18T11:42:13.894048+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:42:15.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f588c06af90 msgr2=0x7f588c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:15.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f588c06af90 0x7f588c06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890006c60 comp rx=0 tx=0).stop 2023-10-18T11:42:15.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58a015c860 msgr2=0x7f58a014c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58a015c860 0x7f58a014c080 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f589402cb70 tx=0x7f5894033c10 comp rx=0 tx=0).stop 2023-10-18T11:42:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 shutdown_connections 2023-10-18T11:42:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f588c06af90 0x7f588c06d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58a006a790 0x7f58a006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58a015c860 0x7f58a014c080 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 --2- 172.21.15.125:0/1686398178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58880047c0 0x7f58a00c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.893+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 >> 172.21.15.125:0/1686398178 conn(0x7f58a00d0d60 msgr2=0x7f58a00ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.894+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 shutdown_connections 2023-10-18T11:42:15.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:15.894+0000 7f58a6997640 1 -- 172.21.15.125:0/1686398178 wait complete. 2023-10-18T11:42:15.907 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:42:16.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.007+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/3215473747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40c9dc0 msgr2=0x7fc1c406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.007+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/3215473747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40c9dc0 0x7fc1c406b2e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009830 tx=0x7fc1b802bd60 comp rx=0 tx=0).stop 2023-10-18T11:42:16.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.007+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/3215473747 shutdown_connections 2023-10-18T11:42:16.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.007+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/3215473747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1c40ab500 0x7fc1c40ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.007+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/3215473747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40c9dc0 0x7fc1c406b2e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.007+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/3215473747 >> 172.21.15.125:0/3215473747 conn(0x7fc1c4069930 msgr2=0x7fc1c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:16.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.007+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/3215473747 shutdown_connections 2023-10-18T11:42:16.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.007+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/3215473747 wait complete. 2023-10-18T11:42:16.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.008+0000 7fc1ca7dc640 1 Processor -- start 2023-10-18T11:42:16.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.008+0000 7fc1ca7dc640 1 -- start start 2023-10-18T11:42:16.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1ca7dc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1c406af00 0x7fc1c40a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1ca7dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40ab500 0x7fc1c40adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1ca7dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1c40c9dc0 0x7fc1c40a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1ca7dc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40a90e0 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1ca7dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c41514a0 con 0x7fc1c40ab500 2023-10-18T11:42:16.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1ca7dc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4151740 con 0x7fc1c406af00 2023-10-18T11:42:16.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1c8d79640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1c40c9dc0 0x7fc1c40a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1c8d79640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1c40c9dc0 0x7fc1c40a6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38774/0 (socket says 172.21.15.125:38774) 2023-10-18T11:42:16.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1c8d79640 1 -- 172.21.15.125:0/692281093 learned_addr learned my addr 172.21.15.125:0/692281093 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:16.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1c37fe640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40ab500 0x7fc1c40adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.009+0000 7fc1c3fff640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1c406af00 0x7fc1c40a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c8d79640 1 -- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1c406af00 msgr2=0x7fc1c40a8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c8d79640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1c406af00 0x7fc1c40a8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c8d79640 1 -- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40ab500 msgr2=0x7fc1c40adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c8d79640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40ab500 0x7fc1c40adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c8d79640 1 -- 172.21.15.125:0/692281093 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc009840 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c37fe640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40ab500 0x7fc1c40adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:42:16.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c8d79640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1c40c9dc0 0x7fc1c40a6b10 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00efd0 tx=0x7fc1bc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:16.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c0ff9640 1 -- 172.21.15.125:0/692281093 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc009b40 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c0ff9640 1 -- 172.21.15.125:0/692281093 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc1bc003c60 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b8009490 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1c0ff9640 1 -- 172.21.15.125:0/692281093 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc004020 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.010+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c4151d00 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.011+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c40690e0 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.012+0000 7fc1c0ff9640 1 -- 172.21.15.125:0/692281093 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc1bc004180 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.012+0000 7fc1c0ff9640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fc1b006af60 0x7fc1b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.012+0000 7fc1c0ff9640 1 -- 172.21.15.125:0/692281093 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fc1bc014070 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.012+0000 7fc1c3fff640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fc1b006af60 0x7fc1b006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.013+0000 7fc1c3fff640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fc1b006af60 0x7fc1b006d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40b1620 tx=0x7fc1b4007e50 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:16.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.016+0000 7fc1c0ff9640 1 -- 172.21.15.125:0/692281093 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1bc055730 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.185+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c40cc420 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.186+0000 7fc1c0ff9640 1 -- 172.21.15.125:0/692281093 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+16821 (secure 0 0 0) 0x7fc1bc01d090 con 0x7fc1c40c9dc0 2023-10-18T11:42:16.189 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:16.189 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":187,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:42:14.915216+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:42:12.758545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-18T11:42:13.894048+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:42:16.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fc1b006af60 msgr2=0x7fc1b006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fc1b006af60 0x7fc1b006d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40b1620 tx=0x7fc1b4007e50 comp rx=0 tx=0).stop 2023-10-18T11:42:16.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1c40c9dc0 msgr2=0x7fc1c40a6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1c40c9dc0 0x7fc1c40a6b10 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00efd0 tx=0x7fc1bc00c6a0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 shutdown_connections 2023-10-18T11:42:16.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fc1b006af60 0x7fc1b006d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1c40ab500 0x7fc1c40adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc1c40c9dc0 0x7fc1c40a6b10 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 --2- 172.21.15.125:0/692281093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1c406af00 0x7fc1c40a8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 >> 172.21.15.125:0/692281093 conn(0x7fc1c4069930 msgr2=0x7fc1c4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:16.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 shutdown_connections 2023-10-18T11:42:16.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.188+0000 7fc1ca7dc640 1 -- 172.21.15.125:0/692281093 wait complete. 2023-10-18T11:42:16.202 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T11:42:16.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.306+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/1925848585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380cf340 msgr2=0x7f7d380d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.306+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/1925848585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380cf340 0x7f7d380d1910 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009830 tx=0x7f7d3402bcd0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.306+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/1925848585 shutdown_connections 2023-10-18T11:42:16.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.306+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/1925848585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d380ceb40 0x7f7d380b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.306+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/1925848585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380cf340 0x7f7d380d1910 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.306+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/1925848585 >> 172.21.15.125:0/1925848585 conn(0x7f7d380d0d60 msgr2=0x7f7d380d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:16.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.306+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/1925848585 shutdown_connections 2023-10-18T11:42:16.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.306+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/1925848585 wait complete. 2023-10-18T11:42:16.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.307+0000 7f7d3fd5e640 1 Processor -- start 2023-10-18T11:42:16.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.307+0000 7f7d3fd5e640 1 -- start start 2023-10-18T11:42:16.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.307+0000 7f7d3fd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380ceb40 0x7f7d3814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.307+0000 7f7d3fd5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d380cf340 0x7f7d381e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.307+0000 7f7d3fd5e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3815b2a0 0x7f7d380b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3fd5e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d381e5830 con 0x7f7d380cf340 2023-10-18T11:42:16.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3fd5e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d381e4bb0 con 0x7f7d380ceb40 2023-10-18T11:42:16.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3fd5e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d381e4e90 con 0x7f7d3815b2a0 2023-10-18T11:42:16.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d380cf340 0x7f7d381e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d380cf340 0x7f7d381e5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38788/0 (socket says 172.21.15.125:38788) 2023-10-18T11:42:16.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 -- 172.21.15.125:0/2268952175 learned_addr learned my addr 172.21.15.125:0/2268952175 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:16.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3dafa640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380ceb40 0x7f7d3814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3e2fb640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3815b2a0 0x7f7d380b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 -- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3815b2a0 msgr2=0x7f7d380b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3815b2a0 0x7f7d380b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 -- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380ceb40 msgr2=0x7f7d3814c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380ceb40 0x7f7d3814c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 -- 172.21.15.125:0/2268952175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34009490 con 0x7f7d380cf340 2023-10-18T11:42:16.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3d2f9640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d380cf340 0x7f7d381e5c50 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c430 tx=0x7f7d2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:16.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d2e7fc640 1 -- 172.21.15.125:0/2268952175 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28011790 con 0x7f7d380cf340 2023-10-18T11:42:16.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d2e7fc640 1 -- 172.21.15.125:0/2268952175 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7d28011a00 con 0x7f7d380cf340 2023-10-18T11:42:16.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d381e5170 con 0x7f7d380cf340 2023-10-18T11:42:16.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d2e7fc640 1 -- 172.21.15.125:0/2268952175 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28011e10 con 0x7f7d380cf340 2023-10-18T11:42:16.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.308+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3814c6f0 con 0x7f7d380cf340 2023-10-18T11:42:16.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.309+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d381e5970 con 0x7f7d380cf340 2023-10-18T11:42:16.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.310+0000 7f7d2e7fc640 1 -- 172.21.15.125:0/2268952175 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7d2801c460 con 0x7f7d380cf340 2023-10-18T11:42:16.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.310+0000 7f7d2e7fc640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7d1c06af60 0x7f7d1c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.310+0000 7f7d2e7fc640 1 -- 172.21.15.125:0/2268952175 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f7d28089950 con 0x7f7d380cf340 2023-10-18T11:42:16.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.310+0000 7f7d3dafa640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7d1c06af60 0x7f7d1c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.311+0000 7f7d3dafa640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7d1c06af60 0x7f7d1c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d34036000 tx=0x7f7d34004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:16.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.314+0000 7f7d2e7fc640 1 -- 172.21.15.125:0/2268952175 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d381e5970 con 0x7f7d380cf340 2023-10-18T11:42:16.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.514+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7d381e5970 con 0x7f7d380cf340 2023-10-18T11:42:16.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.605+0000 7f7d2e7fc640 1 -- 172.21.15.125:0/2268952175 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7d381e5970 con 0x7f7d380cf340 2023-10-18T11:42:16.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.607+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7d1c06af60 msgr2=0x7f7d1c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.607+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7d1c06af60 0x7f7d1c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d34036000 tx=0x7f7d34004730 comp rx=0 tx=0).stop 2023-10-18T11:42:16.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.607+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d380cf340 msgr2=0x7f7d381e5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.607+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d380cf340 0x7f7d381e5c50 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c430 tx=0x7f7d2800c8f0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.608+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 shutdown_connections 2023-10-18T11:42:16.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.608+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7d1c06af60 0x7f7d1c06d420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.608+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3815b2a0 0x7f7d380b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.608+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d380cf340 0x7f7d381e5c50 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.608+0000 7f7d3fd5e640 1 --2- 172.21.15.125:0/2268952175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380ceb40 0x7f7d3814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.608+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 >> 172.21.15.125:0/2268952175 conn(0x7f7d380d0d60 msgr2=0x7f7d3815a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.608+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 shutdown_connections 2023-10-18T11:42:16.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.608+0000 7f7d3fd5e640 1 -- 172.21.15.125:0/2268952175 wait complete. 2023-10-18T11:42:16.621 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T11:42:16.621 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T11:42:16.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.724+0000 7f96acb2c640 1 -- 172.21.15.125:0/2197620818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a8070920 msgr2=0x7f96a80affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.724+0000 7f96acb2c640 1 --2- 172.21.15.125:0/2197620818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a8070920 0x7f96a80affe0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f96a80cbdc0 tx=0x7f969802bcf0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.724+0000 7f96acb2c640 1 -- 172.21.15.125:0/2197620818 shutdown_connections 2023-10-18T11:42:16.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.724+0000 7f96acb2c640 1 --2- 172.21.15.125:0/2197620818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a8070920 0x7f96a80affe0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.724+0000 7f96acb2c640 1 --2- 172.21.15.125:0/2197620818 >> v2:172.21.15.139:3300/0 conn(0x7f96a8141690 0x7f96a80703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.724+0000 7f96acb2c640 1 -- 172.21.15.125:0/2197620818 >> 172.21.15.125:0/2197620818 conn(0x7f96a8069930 msgr2=0x7f96a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:16.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.725+0000 7f96acb2c640 1 -- 172.21.15.125:0/2197620818 shutdown_connections 2023-10-18T11:42:16.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.725+0000 7f96acb2c640 1 -- 172.21.15.125:0/2197620818 wait complete. 2023-10-18T11:42:16.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.726+0000 7f96acb2c640 1 Processor -- start 2023-10-18T11:42:16.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.726+0000 7f96acb2c640 1 -- start start 2023-10-18T11:42:16.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96acb2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96a8070920 0x7f96a813b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96acb2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a813e520 0x7f96a80a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96acb2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96a8141690 0x7f96a80ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96acb2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80aaf60 con 0x7f96a813e520 2023-10-18T11:42:16.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96acb2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80aa2e0 con 0x7f96a8141690 2023-10-18T11:42:16.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96acb2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80aa5c0 con 0x7f96a8070920 2023-10-18T11:42:16.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a813e520 0x7f96a80a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a813e520 0x7f96a80a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38792/0 (socket says 172.21.15.125:38792) 2023-10-18T11:42:16.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 -- 172.21.15.125:0/3583995529 learned_addr learned my addr 172.21.15.125:0/3583995529 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:16.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a6d9d640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96a8141690 0x7f96a80ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 -- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96a8070920 msgr2=0x7f96a813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96a8070920 0x7f96a813b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 -- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96a8141690 msgr2=0x7f96a80ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:16.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96a8141690 0x7f96a80ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:16.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 -- 172.21.15.125:0/3583995529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698009490 con 0x7f96a813e520 2023-10-18T11:42:16.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96a5d9b640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a813e520 0x7f96a80a9840 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f969c00ea20 tx=0x7f969c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:16.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f9696ffd640 1 -- 172.21.15.125:0/3583995529 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00cde0 con 0x7f96a813e520 2023-10-18T11:42:16.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f9696ffd640 1 -- 172.21.15.125:0/3583995529 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f969c011560 con 0x7f96a813e520 2023-10-18T11:42:16.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a80aa8a0 con 0x7f96a813e520 2023-10-18T11:42:16.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f9696ffd640 1 -- 172.21.15.125:0/3583995529 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c011950 con 0x7f96a813e520 2023-10-18T11:42:16.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.727+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a80a7020 con 0x7f96a813e520 2023-10-18T11:42:16.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.728+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a80b1900 con 0x7f96a813e520 2023-10-18T11:42:16.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.729+0000 7f9696ffd640 1 -- 172.21.15.125:0/3583995529 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f969c011ab0 con 0x7f96a813e520 2023-10-18T11:42:16.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.729+0000 7f9696ffd640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f968806af60 0x7f968806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:16.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.729+0000 7f9696ffd640 1 -- 172.21.15.125:0/3583995529 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f969c08a6d0 con 0x7f96a813e520 2023-10-18T11:42:16.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.729+0000 7f96a659c640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f968806af60 0x7f968806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:16.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.730+0000 7f96a659c640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f968806af60 0x7f968806d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001600 tx=0x7f96a0007910 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:16.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.734+0000 7f9696ffd640 1 -- 172.21.15.125:0/3583995529 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f969c055400 con 0x7f96a813e520 2023-10-18T11:42:16.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:16.933+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f96a80b1bd0 con 0x7f96a813e520 2023-10-18T11:42:17.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.610+0000 7f9696ffd640 1 -- 172.21.15.125:0/3583995529 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) v1 ==== 122+0+0 (secure 0 0 0) 0x7f969c058a20 con 0x7f96a813e520 2023-10-18T11:42:17.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f968806af60 msgr2=0x7f968806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:17.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f968806af60 0x7f968806d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001600 tx=0x7f96a0007910 comp rx=0 tx=0).stop 2023-10-18T11:42:17.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a813e520 msgr2=0x7f96a80a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:17.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a813e520 0x7f96a80a9840 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f969c00ea20 tx=0x7f969c00eee0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 shutdown_connections 2023-10-18T11:42:17.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f968806af60 0x7f968806d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96a8141690 0x7f96a80ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96a813e520 0x7f96a80a9840 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 --2- 172.21.15.125:0/3583995529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96a8070920 0x7f96a813b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 >> 172.21.15.125:0/3583995529 conn(0x7f96a8069930 msgr2=0x7f96a80ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:17.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 shutdown_connections 2023-10-18T11:42:17.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.613+0000 7f96acb2c640 1 -- 172.21.15.125:0/3583995529 wait complete. 2023-10-18T11:42:17.628 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T11:42:17.629 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:42:17.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.737+0000 7f743d3e9640 1 -- 172.21.15.125:0/574140331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74380cd6d0 msgr2=0x7f74380cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:17.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.737+0000 7f743d3e9640 1 --2- 172.21.15.125:0/574140331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74380cd6d0 0x7f74380cdb30 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f742c005fd0 tx=0x7f742c02bd90 comp rx=0 tx=0).stop 2023-10-18T11:42:17.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.737+0000 7f743d3e9640 1 -- 172.21.15.125:0/574140331 shutdown_connections 2023-10-18T11:42:17.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.737+0000 7f743d3e9640 1 --2- 172.21.15.125:0/574140331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74380cd6d0 0x7f74380cdb30 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.737+0000 7f743d3e9640 1 --2- 172.21.15.125:0/574140331 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7438142850 0x7f7438159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.737+0000 7f743d3e9640 1 -- 172.21.15.125:0/574140331 >> 172.21.15.125:0/574140331 conn(0x7f74380af7a0 msgr2=0x7f74380afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:17.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.737+0000 7f743d3e9640 1 -- 172.21.15.125:0/574140331 shutdown_connections 2023-10-18T11:42:17.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.737+0000 7f743d3e9640 1 -- 172.21.15.125:0/574140331 wait complete. 2023-10-18T11:42:17.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.738+0000 7f743d3e9640 1 Processor -- start 2023-10-18T11:42:17.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.738+0000 7f743d3e9640 1 -- start start 2023-10-18T11:42:17.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f743d3e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7438069720 0x7f743813e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:17.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f743d3e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74380cd6d0 0x7f74380aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:17.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f743d3e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7438142850 0x7f74380c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:17.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f743d3e9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743813ede0 con 0x7f7438069720 2023-10-18T11:42:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f743d3e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380aaf10 con 0x7f7438142850 2023-10-18T11:42:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f743d3e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380ab1f0 con 0x7f74380cd6d0 2023-10-18T11:42:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7438069720 0x7f743813e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7438069720 0x7f743813e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57194/0 (socket says 172.21.15.125:57194) 2023-10-18T11:42:17.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 -- 172.21.15.125:0/2174094014 learned_addr learned my addr 172.21.15.125:0/2174094014 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f74367fc640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74380cd6d0 0x7f74380aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 -- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74380cd6d0 msgr2=0x7f74380aa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74380cd6d0 0x7f74380aa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 -- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7438142850 msgr2=0x7f74380c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:17.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f74377fe640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7438142850 0x7f74380c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7438142850 0x7f74380c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 -- 172.21.15.125:0/2174094014 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c009560 con 0x7f7438069720 2023-10-18T11:42:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f7436ffd640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7438069720 0x7f743813e8a0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f742800ba50 tx=0x7f742800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f741f7fe640 1 -- 172.21.15.125:0/2174094014 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742800cbe0 con 0x7f7438069720 2023-10-18T11:42:17.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f741f7fe640 1 -- 172.21.15.125:0/2174094014 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f742800ce50 con 0x7f7438069720 2023-10-18T11:42:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74380ab4d0 con 0x7f7438069720 2023-10-18T11:42:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f741f7fe640 1 -- 172.21.15.125:0/2174094014 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428014680 con 0x7f7438069720 2023-10-18T11:42:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.739+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74380abb20 con 0x7f7438069720 2023-10-18T11:42:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.740+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74380690e0 con 0x7f7438069720 2023-10-18T11:42:17.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.741+0000 7f741f7fe640 1 -- 172.21.15.125:0/2174094014 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f742801d3b0 con 0x7f7438069720 2023-10-18T11:42:17.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.741+0000 7f741f7fe640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f742406af60 0x7f742406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:17.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.742+0000 7f741f7fe640 1 -- 172.21.15.125:0/2174094014 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f7428089a90 con 0x7f7438069720 2023-10-18T11:42:17.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.742+0000 7f74367fc640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f742406af60 0x7f742406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:17.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.742+0000 7f74367fc640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f742406af60 0x7f742406d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f742c035000 tx=0x7f742c004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:17.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.746+0000 7f741f7fe640 1 -- 172.21.15.125:0/2174094014 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7428054960 con 0x7f7438069720 2023-10-18T11:42:17.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.945+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7438159a60 con 0x7f7438069720 2023-10-18T11:42:17.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.945+0000 7f741f7fe640 1 -- 172.21.15.125:0/2174094014 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f7428019020 con 0x7f7438069720 2023-10-18T11:42:17.948 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:17.948 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":73,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2139029617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2139029617},{"type":"v1","addr":"172.21.15.125:6835","nonce":2139029617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2438879012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2438879012},{"type":"v1","addr":"172.21.15.139:6837","nonce":2438879012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":10830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2643526616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2643526616},{"type":"v1","addr":"172.21.15.125:6834","nonce":2643526616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:42:14.915536+0000","modified":"2023-10-18T11:42:16.934466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6836/3571180970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3571180970},{"type":"v1","addr":"172.21.15.139:6836","nonce":3571180970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T11:42:17.949 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 73 2023-10-18T11:42:17.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f742406af60 msgr2=0x7f742406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:17.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f742406af60 0x7f742406d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f742c035000 tx=0x7f742c004730 comp rx=0 tx=0).stop 2023-10-18T11:42:17.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7438069720 msgr2=0x7f743813e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:17.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7438069720 0x7f743813e8a0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f742800ba50 tx=0x7f742800bf10 comp rx=0 tx=0).stop 2023-10-18T11:42:17.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 shutdown_connections 2023-10-18T11:42:17.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f742406af60 0x7f742406d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7438142850 0x7f74380c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74380cd6d0 0x7f74380aa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 --2- 172.21.15.125:0/2174094014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7438069720 0x7f743813e8a0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:17.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 >> 172.21.15.125:0/2174094014 conn(0x7f74380af7a0 msgr2=0x7f743813f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:17.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 shutdown_connections 2023-10-18T11:42:17.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:17.947+0000 7f743d3e9640 1 -- 172.21.15.125:0/2174094014 wait complete. 2023-10-18T11:42:17.960 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:42:18.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 -- 172.21.15.125:0/1199677939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 msgr2=0x7ff0400ab930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 --2- 172.21.15.125:0/1199677939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 0x7ff0400ab930 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7ff040071340 tx=0x7ff03802c250 comp rx=0 tx=0).stop 2023-10-18T11:42:18.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 -- 172.21.15.125:0/1199677939 shutdown_connections 2023-10-18T11:42:18.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 --2- 172.21.15.125:0/1199677939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 0x7ff0400ab930 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 --2- 172.21.15.125:0/1199677939 >> v2:172.21.15.139:3300/0 conn(0x7ff0400c9dc0 0x7ff0401531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 --2- 172.21.15.125:0/1199677939 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0400c99e0 0x7ff040152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 -- 172.21.15.125:0/1199677939 >> 172.21.15.125:0/1199677939 conn(0x7ff040069930 msgr2=0x7ff040069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:18.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 -- 172.21.15.125:0/1199677939 shutdown_connections 2023-10-18T11:42:18.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.061+0000 7ff046294640 1 -- 172.21.15.125:0/1199677939 wait complete. 2023-10-18T11:42:18.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.063+0000 7ff046294640 1 Processor -- start 2023-10-18T11:42:18.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.063+0000 7ff046294640 1 -- start start 2023-10-18T11:42:18.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.063+0000 7ff046294640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 0x7ff0400ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.063+0000 7ff046294640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0400c99e0 0x7ff0400ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.063+0000 7ff046294640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0400c9dc0 0x7ff0401367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.063+0000 7ff046294640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040154780 con 0x7ff0400ab500 2023-10-18T11:42:18.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.063+0000 7ff046294640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040139b40 con 0x7ff0400c9dc0 2023-10-18T11:42:18.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.063+0000 7ff046294640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400addd0 con 0x7ff0400c99e0 2023-10-18T11:42:18.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 0x7ff0400ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 0x7ff0400ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57214/0 (socket says 172.21.15.125:57214) 2023-10-18T11:42:18.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 -- 172.21.15.125:0/2350928710 learned_addr learned my addr 172.21.15.125:0/2350928710 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:18.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03f7fe640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0400c99e0 0x7ff0400ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff044831640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0400c9dc0 0x7ff0401367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 -- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0400c99e0 msgr2=0x7ff0400ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0400c99e0 0x7ff0400ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 -- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0400c9dc0 msgr2=0x7ff0401367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0400c9dc0 0x7ff0401367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 -- 172.21.15.125:0/2350928710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038009900 con 0x7ff0400ab500 2023-10-18T11:42:18.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03ffff640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 0x7ff0400ad230 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7ff03000d770 tx=0x7ff03000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:18.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03cff9640 1 -- 172.21.15.125:0/2350928710 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030009c30 con 0x7ff0400ab500 2023-10-18T11:42:18.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03cff9640 1 -- 172.21.15.125:0/2350928710 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff03000de80 con 0x7ff0400ab500 2023-10-18T11:42:18.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0400abff0 con 0x7ff0400ab500 2023-10-18T11:42:18.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff03cff9640 1 -- 172.21.15.125:0/2350928710 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030005280 con 0x7ff0400ab500 2023-10-18T11:42:18.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.064+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff040154cd0 con 0x7ff0400ab500 2023-10-18T11:42:18.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.065+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0400690e0 con 0x7ff0400ab500 2023-10-18T11:42:18.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.066+0000 7ff03cff9640 1 -- 172.21.15.125:0/2350928710 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff0300053e0 con 0x7ff0400ab500 2023-10-18T11:42:18.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.066+0000 7ff03cff9640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff02c06af60 0x7ff02c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.066+0000 7ff03cff9640 1 -- 172.21.15.125:0/2350928710 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7ff030089c60 con 0x7ff0400ab500 2023-10-18T11:42:18.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.066+0000 7ff03f7fe640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff02c06af60 0x7ff02c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.067+0000 7ff03f7fe640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff02c06af60 0x7ff02c06d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff034005fd0 tx=0x7ff034005850 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:18.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.070+0000 7ff03cff9640 1 -- 172.21.15.125:0/2350928710 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff030054900 con 0x7ff0400ab500 2023-10-18T11:42:18.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.235+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0400cc420 con 0x7ff0400ab500 2023-10-18T11:42:18.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.236+0000 7ff03cff9640 1 -- 172.21.15.125:0/2350928710 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+16808 (secure 0 0 0) 0x7ff030017030 con 0x7ff0400ab500 2023-10-18T11:42:18.239 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:18.239 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":188,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:42:16.580430+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:42:12.758545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-18T11:42:13.894048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:42:18.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff02c06af60 msgr2=0x7ff02c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff02c06af60 0x7ff02c06d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff034005fd0 tx=0x7ff034005850 comp rx=0 tx=0).stop 2023-10-18T11:42:18.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 msgr2=0x7ff0400ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 0x7ff0400ad230 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7ff03000d770 tx=0x7ff03000dc30 comp rx=0 tx=0).stop 2023-10-18T11:42:18.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 shutdown_connections 2023-10-18T11:42:18.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff02c06af60 0x7ff02c06d420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0400c9dc0 0x7ff0401367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0400c99e0 0x7ff0400ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 --2- 172.21.15.125:0/2350928710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0400ab500 0x7ff0400ad230 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 >> 172.21.15.125:0/2350928710 conn(0x7ff040069930 msgr2=0x7ff040152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:18.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 shutdown_connections 2023-10-18T11:42:18.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.238+0000 7ff046294640 1 -- 172.21.15.125:0/2350928710 wait complete. 2023-10-18T11:42:18.252 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T11:42:18.253 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:42:18.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.354+0000 7ff87b98e640 1 -- 172.21.15.125:0/2062397801 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 msgr2=0x7ff8740c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.354+0000 7ff87b98e640 1 --2- 172.21.15.125:0/2062397801 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 0x7ff8740c97c0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7ff87406f010 tx=0x7ff86402be40 comp rx=0 tx=0).stop 2023-10-18T11:42:18.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.354+0000 7ff87b98e640 1 -- 172.21.15.125:0/2062397801 shutdown_connections 2023-10-18T11:42:18.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.354+0000 7ff87b98e640 1 --2- 172.21.15.125:0/2062397801 >> v2:172.21.15.139:3300/0 conn(0x7ff87406d170 0x7ff8740c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.354+0000 7ff87b98e640 1 --2- 172.21.15.125:0/2062397801 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 0x7ff8740c97c0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.354+0000 7ff87b98e640 1 -- 172.21.15.125:0/2062397801 >> 172.21.15.125:0/2062397801 conn(0x7ff874069930 msgr2=0x7ff874069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:18.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.354+0000 7ff87b98e640 1 -- 172.21.15.125:0/2062397801 shutdown_connections 2023-10-18T11:42:18.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.354+0000 7ff87b98e640 1 -- 172.21.15.125:0/2062397801 wait complete. 2023-10-18T11:42:18.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.355+0000 7ff87b98e640 1 Processor -- start 2023-10-18T11:42:18.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.355+0000 7ff87b98e640 1 -- start start 2023-10-18T11:42:18.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff87b98e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87406d170 0x7ff874156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff87b98e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87413d360 0x7ff874129f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff87b98e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 0x7ff8741553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff87b98e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740b0870 con 0x7ff874140660 2023-10-18T11:42:18.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff87b98e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740b0b50 con 0x7ff87413d360 2023-10-18T11:42:18.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff87b98e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740b0e30 con 0x7ff87406d170 2023-10-18T11:42:18.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 0x7ff8741553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 0x7ff8741553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57228/0 (socket says 172.21.15.125:57228) 2023-10-18T11:42:18.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 -- 172.21.15.125:0/3904486605 learned_addr learned my addr 172.21.15.125:0/3904486605 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:18.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff87972a640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87406d170 0x7ff874156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff878f29640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87413d360 0x7ff874129f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 -- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87406d170 msgr2=0x7ff874156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87406d170 0x7ff874156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 -- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87413d360 msgr2=0x7ff874129f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87413d360 0x7ff874129f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 -- 172.21.15.125:0/3904486605 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff864009560 con 0x7ff874140660 2023-10-18T11:42:18.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff879f2b640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 0x7ff8741553b0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d950 tx=0x7ff86c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:18.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.356+0000 7ff869ffb640 1 -- 172.21.15.125:0/3904486605 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c009a30 con 0x7ff874140660 2023-10-18T11:42:18.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.357+0000 7ff869ffb640 1 -- 172.21.15.125:0/3904486605 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff86c009c80 con 0x7ff874140660 2023-10-18T11:42:18.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.357+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8740b1110 con 0x7ff874140660 2023-10-18T11:42:18.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.357+0000 7ff869ffb640 1 -- 172.21.15.125:0/3904486605 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c004020 con 0x7ff874140660 2023-10-18T11:42:18.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.357+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8740b04d0 con 0x7ff874140660 2023-10-18T11:42:18.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.358+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8740b09b0 con 0x7ff874140660 2023-10-18T11:42:18.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.358+0000 7ff869ffb640 1 -- 172.21.15.125:0/3904486605 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff86c004180 con 0x7ff874140660 2023-10-18T11:42:18.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.358+0000 7ff869ffb640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff85806af60 0x7ff85806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.359+0000 7ff869ffb640 1 -- 172.21.15.125:0/3904486605 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7ff86c089d70 con 0x7ff874140660 2023-10-18T11:42:18.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.359+0000 7ff87972a640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff85806af60 0x7ff85806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.359+0000 7ff87972a640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff85806af60 0x7ff85806d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff870007fd0 tx=0x7ff870009040 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:18.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.363+0000 7ff869ffb640 1 -- 172.21.15.125:0/3904486605 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8740b09b0 con 0x7ff874140660 2023-10-18T11:42:18.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.526+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 --> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8740b09b0 con 0x7ff85806af60 2023-10-18T11:42:18.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.534+0000 7ff869ffb640 1 -- 172.21.15.125:0/3904486605 <== mgr.8003 v2:172.21.15.139:6832/3367593104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276106 (secure 0 0 0) 0x7ff8740b09b0 con 0x7ff85806af60 2023-10-18T11:42:18.539 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:18.540 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:42:18.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff85806af60 msgr2=0x7ff85806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff85806af60 0x7ff85806d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff870007fd0 tx=0x7ff870009040 comp rx=0 tx=0).stop 2023-10-18T11:42:18.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 msgr2=0x7ff8741553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 0x7ff8741553b0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d950 tx=0x7ff86c00de10 comp rx=0 tx=0).stop 2023-10-18T11:42:18.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 shutdown_connections 2023-10-18T11:42:18.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7ff85806af60 0x7ff85806d420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87413d360 0x7ff874129f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874140660 0x7ff8741553b0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 --2- 172.21.15.125:0/3904486605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87406d170 0x7ff874156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 >> 172.21.15.125:0/3904486605 conn(0x7ff874069930 msgr2=0x7ff87413f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:18.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 shutdown_connections 2023-10-18T11:42:18.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.536+0000 7ff87b98e640 1 -- 172.21.15.125:0/3904486605 wait complete. 2023-10-18T11:42:18.549 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T11:42:17.456039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":386576,"kb_used_data":19152,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754588144,"statfs":{"total":773094113280,"available":772698259456,"internally_reserved":0,"allocated":19611648,"data_stored":10525536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000872"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.613794+0000","last_change":"2023-10-18T11:42:15.609772+0000","last_active":"2023-10-18T11:42:15.613794+0000","last_peered":"2023-10-18T11:42:15.613794+0000","last_clean":"2023-10-18T11:42:15.613794+0000","last_became_active":"2023-10-18T11:42:15.609451+0000","last_became_peered":"2023-10-18T11:42:15.609451+0000","last_unstale":"2023-10-18T11:42:15.613794+0000","last_undegraded":"2023-10-18T11:42:15.613794+0000","last_fullsized":"2023-10-18T11:42:15.613794+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:49:31.060237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605940+0000","last_change":"2023-10-18T11:42:15.598367+0000","last_active":"2023-10-18T11:42:15.605940+0000","last_peered":"2023-10-18T11:42:15.605940+0000","last_clean":"2023-10-18T11:42:15.605940+0000","last_became_active":"2023-10-18T11:42:15.598104+0000","last_became_peered":"2023-10-18T11:42:15.598104+0000","last_unstale":"2023-10-18T11:42:15.605940+0000","last_undegraded":"2023-10-18T11:42:15.605940+0000","last_fullsized":"2023-10-18T11:42:15.605940+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:00:17.431376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605906+0000","last_change":"2023-10-18T11:42:15.597629+0000","last_active":"2023-10-18T11:42:15.605906+0000","last_peered":"2023-10-18T11:42:15.605906+0000","last_clean":"2023-10-18T11:42:15.605906+0000","last_became_active":"2023-10-18T11:42:15.597349+0000","last_became_peered":"2023-10-18T11:42:15.597349+0000","last_unstale":"2023-10-18T11:42:15.605906+0000","last_undegraded":"2023-10-18T11:42:15.605906+0000","last_fullsized":"2023-10-18T11:42:15.605906+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:05.997023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604060+0000","last_change":"2023-10-18T11:42:15.596831+0000","last_active":"2023-10-18T11:42:15.604060+0000","last_peered":"2023-10-18T11:42:15.604060+0000","last_clean":"2023-10-18T11:42:15.604060+0000","last_became_active":"2023-10-18T11:42:15.596580+0000","last_became_peered":"2023-10-18T11:42:15.596580+0000","last_unstale":"2023-10-18T11:42:15.604060+0000","last_undegraded":"2023-10-18T11:42:15.604060+0000","last_fullsized":"2023-10-18T11:42:15.604060+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:38.982049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608663+0000","last_change":"2023-10-18T11:42:15.605780+0000","last_active":"2023-10-18T11:42:15.608663+0000","last_peered":"2023-10-18T11:42:15.608663+0000","last_clean":"2023-10-18T11:42:15.608663+0000","last_became_active":"2023-10-18T11:42:15.600458+0000","last_became_peered":"2023-10-18T11:42:15.600458+0000","last_unstale":"2023-10-18T11:42:15.608663+0000","last_undegraded":"2023-10-18T11:42:15.608663+0000","last_fullsized":"2023-10-18T11:42:15.608663+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:57.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"176'302","reported_seq":995,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T11:42:06.818833+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:42:06.818833+0000","last_peered":"2023-10-18T11:42:06.818833+0000","last_clean":"2023-10-18T11:42:06.818833+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:42:06.818833+0000","last_undegraded":"2023-10-18T11:42:06.818833+0000","last_fullsized":"2023-10-18T11:42:06.818833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575774,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52048,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319792,"statfs":{"total":96636764160,"available":96583467008,"internally_reserved":0,"allocated":2887680,"data_stored":1753717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.52500000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.501},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.58199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575775,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2342912,"data_stored":1189287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.44}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59808,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2306048,"data_stored":1169384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.621}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.72699999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.75}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.32100000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.505},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.53100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41768,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330072,"statfs":{"total":96636764160,"available":96593993728,"internally_reserved":0,"allocated":2306048,"data_stored":1169384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.622,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.27800000000000002},"max":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.872,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"max":{"1min":0.80700000000000005,"5min":0.82999999999999996,"15min":0.86399999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.85799999999999998},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.504,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.438},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.501},"min":{"1min":0.318,"5min":0.313,"15min":0.313},"max":{"1min":0.73799999999999999,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75800000000000001,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.498},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.54200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59788,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312052,"statfs":{"total":96636764160,"available":96575541248,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.374}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41099999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.88600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59792,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.627},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.49299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37580,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334260,"statfs":{"total":96636764160,"available":96598282240,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27400000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.307},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.34100000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86799999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575773,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38164,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333676,"statfs":{"total":96636764160,"available":96597684224,"internally_reserved":0,"allocated":2887680,"data_stored":1753717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.377},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.33600000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.317},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.54000000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.52000000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:42:18.556 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:42:18.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.690+0000 7f7f54541640 1 -- 172.21.15.125:0/3375950391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 msgr2=0x7f7f4c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.690+0000 7f7f54541640 1 --2- 172.21.15.125:0/3375950391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 0x7f7f4c0d1910 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c15b680 tx=0x7f7f4802c080 comp rx=0 tx=0).stop 2023-10-18T11:42:18.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.690+0000 7f7f54541640 1 -- 172.21.15.125:0/3375950391 shutdown_connections 2023-10-18T11:42:18.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.690+0000 7f7f54541640 1 --2- 172.21.15.125:0/3375950391 >> v2:172.21.15.139:3300/0 conn(0x7f7f4c0ceb40 0x7f7f4c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.690+0000 7f7f54541640 1 --2- 172.21.15.125:0/3375950391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 0x7f7f4c0d1910 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.690+0000 7f7f54541640 1 -- 172.21.15.125:0/3375950391 >> 172.21.15.125:0/3375950391 conn(0x7f7f4c0d0d60 msgr2=0x7f7f4c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:18.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.690+0000 7f7f54541640 1 -- 172.21.15.125:0/3375950391 shutdown_connections 2023-10-18T11:42:18.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.690+0000 7f7f54541640 1 -- 172.21.15.125:0/3375950391 wait complete. 2023-10-18T11:42:18.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.691+0000 7f7f54541640 1 Processor -- start 2023-10-18T11:42:18.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.691+0000 7f7f54541640 1 -- start start 2023-10-18T11:42:18.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f54541640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f4c0ceb40 0x7f7f4c1f1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f54541640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 0x7f7f4c0cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f54541640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f4c15b2a0 0x7f7f4c1e3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f54541640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c0b3a80 con 0x7f7f4c0cf340 2023-10-18T11:42:18.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f54541640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1f1ac0 con 0x7f7f4c15b2a0 2023-10-18T11:42:18.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f54541640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1f1da0 con 0x7f7f4c0ceb40 2023-10-18T11:42:18.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f51adc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 0x7f7f4c0cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f51adc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 0x7f7f4c0cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57244/0 (socket says 172.21.15.125:57244) 2023-10-18T11:42:18.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f51adc640 1 -- 172.21.15.125:0/1897478141 learned_addr learned my addr 172.21.15.125:0/1897478141 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:18.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f522dd640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f4c0ceb40 0x7f7f4c1f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f52ade640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f4c15b2a0 0x7f7f4c1e3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f51adc640 1 -- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f4c0ceb40 msgr2=0x7f7f4c1f1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f51adc640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f4c0ceb40 0x7f7f4c1f1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f51adc640 1 -- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f4c15b2a0 msgr2=0x7f7f4c1e3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f51adc640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f4c15b2a0 0x7f7f4c1e3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.692+0000 7f7f51adc640 1 -- 172.21.15.125:0/1897478141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f48009840 con 0x7f7f4c0cf340 2023-10-18T11:42:18.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.693+0000 7f7f51adc640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 0x7f7f4c0cabd0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c3c0 tx=0x7f7f3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:18.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.693+0000 7f7f42ffd640 1 -- 172.21.15.125:0/1897478141 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c011790 con 0x7f7f4c0cf340 2023-10-18T11:42:18.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.693+0000 7f7f42ffd640 1 -- 172.21.15.125:0/1897478141 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7f3c0119e0 con 0x7f7f4c0cf340 2023-10-18T11:42:18.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.693+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c1f2080 con 0x7f7f4c0cf340 2023-10-18T11:42:18.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.693+0000 7f7f42ffd640 1 -- 172.21.15.125:0/1897478141 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c011e10 con 0x7f7f4c0cf340 2023-10-18T11:42:18.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.693+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c0cb110 con 0x7f7f4c0cf340 2023-10-18T11:42:18.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.694+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4c0b3c30 con 0x7f7f4c0cf340 2023-10-18T11:42:18.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.694+0000 7f7f42ffd640 1 -- 172.21.15.125:0/1897478141 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7f3c01c460 con 0x7f7f4c0cf340 2023-10-18T11:42:18.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.695+0000 7f7f42ffd640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7f3006af60 0x7f7f3006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:18.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.695+0000 7f7f42ffd640 1 -- 172.21.15.125:0/1897478141 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f7f3c089890 con 0x7f7f4c0cf340 2023-10-18T11:42:18.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.695+0000 7f7f522dd640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7f3006af60 0x7f7f3006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:18.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.695+0000 7f7f522dd640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7f3006af60 0x7f7f3006d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f48035000 tx=0x7f7f48004ab0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:18.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.699+0000 7f7f42ffd640 1 -- 172.21.15.125:0/1897478141 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f4c0b3c30 con 0x7f7f4c0cf340 2023-10-18T11:42:18.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.860+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 --> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f4c0b0d00 con 0x7f7f3006af60 2023-10-18T11:42:18.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.868+0000 7f7f42ffd640 1 -- 172.21.15.125:0/1897478141 <== mgr.8003 v2:172.21.15.139:6832/3367593104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276106 (secure 0 0 0) 0x7f7f4c0b0d00 con 0x7f7f3006af60 2023-10-18T11:42:18.873 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:18.874 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:42:18.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7f3006af60 msgr2=0x7f7f3006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7f3006af60 0x7f7f3006d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f48035000 tx=0x7f7f48004ab0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 msgr2=0x7f7f4c0cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:18.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 0x7f7f4c0cabd0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c3c0 tx=0x7f7f3c00c880 comp rx=0 tx=0).stop 2023-10-18T11:42:18.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 shutdown_connections 2023-10-18T11:42:18.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f7f3006af60 0x7f7f3006d420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f4c15b2a0 0x7f7f4c1e3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7f4c0cf340 0x7f7f4c0cabd0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 --2- 172.21.15.125:0/1897478141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7f4c0ceb40 0x7f7f4c1f1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:18.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 >> 172.21.15.125:0/1897478141 conn(0x7f7f4c0d0d60 msgr2=0x7f7f4c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:18.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 shutdown_connections 2023-10-18T11:42:18.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:18.871+0000 7f7f54541640 1 -- 172.21.15.125:0/1897478141 wait complete. 2023-10-18T11:42:18.883 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T11:42:17.456039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":386576,"kb_used_data":19152,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754588144,"statfs":{"total":773094113280,"available":772698259456,"internally_reserved":0,"allocated":19611648,"data_stored":10525536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000872"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.613794+0000","last_change":"2023-10-18T11:42:15.609772+0000","last_active":"2023-10-18T11:42:15.613794+0000","last_peered":"2023-10-18T11:42:15.613794+0000","last_clean":"2023-10-18T11:42:15.613794+0000","last_became_active":"2023-10-18T11:42:15.609451+0000","last_became_peered":"2023-10-18T11:42:15.609451+0000","last_unstale":"2023-10-18T11:42:15.613794+0000","last_undegraded":"2023-10-18T11:42:15.613794+0000","last_fullsized":"2023-10-18T11:42:15.613794+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:49:31.060237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605940+0000","last_change":"2023-10-18T11:42:15.598367+0000","last_active":"2023-10-18T11:42:15.605940+0000","last_peered":"2023-10-18T11:42:15.605940+0000","last_clean":"2023-10-18T11:42:15.605940+0000","last_became_active":"2023-10-18T11:42:15.598104+0000","last_became_peered":"2023-10-18T11:42:15.598104+0000","last_unstale":"2023-10-18T11:42:15.605940+0000","last_undegraded":"2023-10-18T11:42:15.605940+0000","last_fullsized":"2023-10-18T11:42:15.605940+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:00:17.431376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605906+0000","last_change":"2023-10-18T11:42:15.597629+0000","last_active":"2023-10-18T11:42:15.605906+0000","last_peered":"2023-10-18T11:42:15.605906+0000","last_clean":"2023-10-18T11:42:15.605906+0000","last_became_active":"2023-10-18T11:42:15.597349+0000","last_became_peered":"2023-10-18T11:42:15.597349+0000","last_unstale":"2023-10-18T11:42:15.605906+0000","last_undegraded":"2023-10-18T11:42:15.605906+0000","last_fullsized":"2023-10-18T11:42:15.605906+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:05.997023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604060+0000","last_change":"2023-10-18T11:42:15.596831+0000","last_active":"2023-10-18T11:42:15.604060+0000","last_peered":"2023-10-18T11:42:15.604060+0000","last_clean":"2023-10-18T11:42:15.604060+0000","last_became_active":"2023-10-18T11:42:15.596580+0000","last_became_peered":"2023-10-18T11:42:15.596580+0000","last_unstale":"2023-10-18T11:42:15.604060+0000","last_undegraded":"2023-10-18T11:42:15.604060+0000","last_fullsized":"2023-10-18T11:42:15.604060+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:38.982049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608663+0000","last_change":"2023-10-18T11:42:15.605780+0000","last_active":"2023-10-18T11:42:15.608663+0000","last_peered":"2023-10-18T11:42:15.608663+0000","last_clean":"2023-10-18T11:42:15.608663+0000","last_became_active":"2023-10-18T11:42:15.600458+0000","last_became_peered":"2023-10-18T11:42:15.600458+0000","last_unstale":"2023-10-18T11:42:15.608663+0000","last_undegraded":"2023-10-18T11:42:15.608663+0000","last_fullsized":"2023-10-18T11:42:15.608663+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:57.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"176'302","reported_seq":995,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T11:42:06.818833+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:42:06.818833+0000","last_peered":"2023-10-18T11:42:06.818833+0000","last_clean":"2023-10-18T11:42:06.818833+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:42:06.818833+0000","last_undegraded":"2023-10-18T11:42:06.818833+0000","last_fullsized":"2023-10-18T11:42:06.818833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575774,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52048,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319792,"statfs":{"total":96636764160,"available":96583467008,"internally_reserved":0,"allocated":2887680,"data_stored":1753717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.52500000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.501},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.58199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575775,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2342912,"data_stored":1189287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.44}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59808,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2306048,"data_stored":1169384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.621}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.72699999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.75}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.32100000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.505},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.53100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41768,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330072,"statfs":{"total":96636764160,"available":96593993728,"internally_reserved":0,"allocated":2306048,"data_stored":1169384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.622,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.27800000000000002},"max":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.872,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"max":{"1min":0.80700000000000005,"5min":0.82999999999999996,"15min":0.86399999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.85799999999999998},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.504,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.438},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.501},"min":{"1min":0.318,"5min":0.313,"15min":0.313},"max":{"1min":0.73799999999999999,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75800000000000001,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.498},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.54200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59788,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312052,"statfs":{"total":96636764160,"available":96575541248,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.374}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41099999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.88600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59792,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.627},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.49299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37580,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334260,"statfs":{"total":96636764160,"available":96598282240,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27400000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.307},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.34100000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86799999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575773,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38164,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333676,"statfs":{"total":96636764160,"available":96597684224,"internally_reserved":0,"allocated":2887680,"data_stored":1753717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.377},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.33600000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.317},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.54000000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.52000000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:42:18.889 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:42:19.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.025+0000 7f8248368640 1 -- 172.21.15.125:0/1038572398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400cbd20 msgr2=0x7f8240158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.025+0000 7f8248368640 1 --2- 172.21.15.125:0/1038572398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400cbd20 0x7f8240158300 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f82400cc490 tx=0x7f823c02be40 comp rx=0 tx=0).stop 2023-10-18T11:42:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.025+0000 7f8248368640 1 -- 172.21.15.125:0/1038572398 shutdown_connections 2023-10-18T11:42:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.025+0000 7f8248368640 1 --2- 172.21.15.125:0/1038572398 >> v2:172.21.15.139:3300/0 conn(0x7f82400afa20 0x7f82400705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.025+0000 7f8248368640 1 --2- 172.21.15.125:0/1038572398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400cbd20 0x7f8240158300 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.025+0000 7f8248368640 1 -- 172.21.15.125:0/1038572398 >> 172.21.15.125:0/1038572398 conn(0x7f8240069930 msgr2=0x7f8240069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:19.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.025+0000 7f8248368640 1 -- 172.21.15.125:0/1038572398 shutdown_connections 2023-10-18T11:42:19.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.025+0000 7f8248368640 1 -- 172.21.15.125:0/1038572398 wait complete. 2023-10-18T11:42:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.026+0000 7f8248368640 1 Processor -- start 2023-10-18T11:42:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.026+0000 7f8248368640 1 -- start start 2023-10-18T11:42:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.026+0000 7f8248368640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82400afa20 0x7f824014bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.026+0000 7f8248368640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400b0530 0x7f82400a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8248368640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82400cbd20 0x7f82400aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8248368640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824014dc30 con 0x7f82400b0530 2023-10-18T11:42:19.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8248368640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824014df10 con 0x7f82400afa20 2023-10-18T11:42:19.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8248368640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824014e1f0 con 0x7f82400cbd20 2023-10-18T11:42:19.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400b0530 0x7f82400a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400b0530 0x7f82400a65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57274/0 (socket says 172.21.15.125:57274) 2023-10-18T11:42:19.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 -- 172.21.15.125:0/4085475204 learned_addr learned my addr 172.21.15.125:0/4085475204 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:19.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8246104640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82400afa20 0x7f824014bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8246905640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82400cbd20 0x7f82400aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 -- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82400cbd20 msgr2=0x7f82400aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82400cbd20 0x7f82400aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 -- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82400afa20 msgr2=0x7f824014bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82400afa20 0x7f824014bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 -- 172.21.15.125:0/4085475204 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c009560 con 0x7f82400b0530 2023-10-18T11:42:19.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8246905640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82400cbd20 0x7f82400aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:19.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8246104640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82400afa20 0x7f824014bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:19.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8245903640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400b0530 0x7f82400a65d0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f823000ddc0 tx=0x7f823000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:19.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8236ffd640 1 -- 172.21.15.125:0/4085475204 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000e650 con 0x7f82400b0530 2023-10-18T11:42:19.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8236ffd640 1 -- 172.21.15.125:0/4085475204 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f823000e8a0 con 0x7f82400b0530 2023-10-18T11:42:19.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824014e4d0 con 0x7f82400b0530 2023-10-18T11:42:19.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8236ffd640 1 -- 172.21.15.125:0/4085475204 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000ecb0 con 0x7f82400b0530 2023-10-18T11:42:19.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.027+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82400a6b10 con 0x7f82400b0530 2023-10-18T11:42:19.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.028+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824014f480 con 0x7f82400b0530 2023-10-18T11:42:19.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.029+0000 7f8236ffd640 1 -- 172.21.15.125:0/4085475204 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8230016440 con 0x7f82400b0530 2023-10-18T11:42:19.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.029+0000 7f8236ffd640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f822406af90 0x7f822406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.029+0000 7f8236ffd640 1 -- 172.21.15.125:0/4085475204 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f823008a070 con 0x7f82400b0530 2023-10-18T11:42:19.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.029+0000 7f8246104640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f822406af90 0x7f822406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.030+0000 7f8246104640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f822406af90 0x7f822406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f823c035000 tx=0x7f823c004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:19.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.033+0000 7f8236ffd640 1 -- 172.21.15.125:0/4085475204 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f824014f480 con 0x7f82400b0530 2023-10-18T11:42:19.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.198+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 --> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f824006b910 con 0x7f822406af90 2023-10-18T11:42:19.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.205+0000 7f8236ffd640 1 -- 172.21.15.125:0/4085475204 <== mgr.8003 v2:172.21.15.139:6832/3367593104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276106 (secure 0 0 0) 0x7f823c004270 con 0x7f822406af90 2023-10-18T11:42:19.210 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:19.212 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:42:19.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f822406af90 msgr2=0x7f822406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f822406af90 0x7f822406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f823c035000 tx=0x7f823c004730 comp rx=0 tx=0).stop 2023-10-18T11:42:19.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400b0530 msgr2=0x7f82400a65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400b0530 0x7f82400a65d0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f823000ddc0 tx=0x7f823000a620 comp rx=0 tx=0).stop 2023-10-18T11:42:19.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 shutdown_connections 2023-10-18T11:42:19.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f822406af90 0x7f822406d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82400cbd20 0x7f82400aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82400b0530 0x7f82400a65d0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 --2- 172.21.15.125:0/4085475204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82400afa20 0x7f824014bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 >> 172.21.15.125:0/4085475204 conn(0x7f8240069930 msgr2=0x7f82400c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:19.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 shutdown_connections 2023-10-18T11:42:19.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.208+0000 7f8248368640 1 -- 172.21.15.125:0/4085475204 wait complete. 2023-10-18T11:42:19.221 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T11:42:17.456039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":386576,"kb_used_data":19152,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754588144,"statfs":{"total":773094113280,"available":772698259456,"internally_reserved":0,"allocated":19611648,"data_stored":10525536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000872"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.613794+0000","last_change":"2023-10-18T11:42:15.609772+0000","last_active":"2023-10-18T11:42:15.613794+0000","last_peered":"2023-10-18T11:42:15.613794+0000","last_clean":"2023-10-18T11:42:15.613794+0000","last_became_active":"2023-10-18T11:42:15.609451+0000","last_became_peered":"2023-10-18T11:42:15.609451+0000","last_unstale":"2023-10-18T11:42:15.613794+0000","last_undegraded":"2023-10-18T11:42:15.613794+0000","last_fullsized":"2023-10-18T11:42:15.613794+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:49:31.060237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605940+0000","last_change":"2023-10-18T11:42:15.598367+0000","last_active":"2023-10-18T11:42:15.605940+0000","last_peered":"2023-10-18T11:42:15.605940+0000","last_clean":"2023-10-18T11:42:15.605940+0000","last_became_active":"2023-10-18T11:42:15.598104+0000","last_became_peered":"2023-10-18T11:42:15.598104+0000","last_unstale":"2023-10-18T11:42:15.605940+0000","last_undegraded":"2023-10-18T11:42:15.605940+0000","last_fullsized":"2023-10-18T11:42:15.605940+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:00:17.431376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605906+0000","last_change":"2023-10-18T11:42:15.597629+0000","last_active":"2023-10-18T11:42:15.605906+0000","last_peered":"2023-10-18T11:42:15.605906+0000","last_clean":"2023-10-18T11:42:15.605906+0000","last_became_active":"2023-10-18T11:42:15.597349+0000","last_became_peered":"2023-10-18T11:42:15.597349+0000","last_unstale":"2023-10-18T11:42:15.605906+0000","last_undegraded":"2023-10-18T11:42:15.605906+0000","last_fullsized":"2023-10-18T11:42:15.605906+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:05.997023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604060+0000","last_change":"2023-10-18T11:42:15.596831+0000","last_active":"2023-10-18T11:42:15.604060+0000","last_peered":"2023-10-18T11:42:15.604060+0000","last_clean":"2023-10-18T11:42:15.604060+0000","last_became_active":"2023-10-18T11:42:15.596580+0000","last_became_peered":"2023-10-18T11:42:15.596580+0000","last_unstale":"2023-10-18T11:42:15.604060+0000","last_undegraded":"2023-10-18T11:42:15.604060+0000","last_fullsized":"2023-10-18T11:42:15.604060+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:38.982049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608663+0000","last_change":"2023-10-18T11:42:15.605780+0000","last_active":"2023-10-18T11:42:15.608663+0000","last_peered":"2023-10-18T11:42:15.608663+0000","last_clean":"2023-10-18T11:42:15.608663+0000","last_became_active":"2023-10-18T11:42:15.600458+0000","last_became_peered":"2023-10-18T11:42:15.600458+0000","last_unstale":"2023-10-18T11:42:15.608663+0000","last_undegraded":"2023-10-18T11:42:15.608663+0000","last_fullsized":"2023-10-18T11:42:15.608663+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:57.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:14.559314+0000","last_change":"2023-10-18T11:42:14.559314+0000","last_active":"2023-10-18T11:42:14.559314+0000","last_peered":"2023-10-18T11:42:14.559314+0000","last_clean":"2023-10-18T11:42:14.559314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:14.559314+0000","last_undegraded":"2023-10-18T11:42:14.559314+0000","last_fullsized":"2023-10-18T11:42:14.559314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:42:13.553835+0000","last_change":"2023-10-18T11:42:13.553835+0000","last_active":"2023-10-18T11:42:13.553835+0000","last_peered":"2023-10-18T11:42:13.553835+0000","last_clean":"2023-10-18T11:42:13.553835+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:42:13.553835+0000","last_undegraded":"2023-10-18T11:42:13.553835+0000","last_fullsized":"2023-10-18T11:42:13.553835+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"176'302","reported_seq":995,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T11:42:06.818833+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:42:06.818833+0000","last_peered":"2023-10-18T11:42:06.818833+0000","last_clean":"2023-10-18T11:42:06.818833+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:42:06.818833+0000","last_undegraded":"2023-10-18T11:42:06.818833+0000","last_fullsized":"2023-10-18T11:42:06.818833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575774,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52048,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319792,"statfs":{"total":96636764160,"available":96583467008,"internally_reserved":0,"allocated":2887680,"data_stored":1753717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.52500000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.501},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.58199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575775,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2342912,"data_stored":1189287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.44}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59808,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2306048,"data_stored":1169384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.621}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.72699999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.75}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.32100000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.505},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.53100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41768,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330072,"statfs":{"total":96636764160,"available":96593993728,"internally_reserved":0,"allocated":2306048,"data_stored":1169384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.622,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.27800000000000002},"max":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.872,"15min":0.872},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"max":{"1min":0.80700000000000005,"5min":0.82999999999999996,"15min":0.86399999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.85799999999999998},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.504,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.438},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.501},"min":{"1min":0.318,"5min":0.313,"15min":0.313},"max":{"1min":0.73799999999999999,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75800000000000001,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.498},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.54200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59788,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312052,"statfs":{"total":96636764160,"available":96575541248,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.374}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41099999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.88600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59792,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.627},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.49299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37580,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334260,"statfs":{"total":96636764160,"available":96598282240,"internally_reserved":0,"allocated":2293760,"data_stored":1163349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27400000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.307},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.34100000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86799999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575773,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38164,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333676,"statfs":{"total":96636764160,"available":96597684224,"internally_reserved":0,"allocated":2887680,"data_stored":1753717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.377},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.33600000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.317},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.54000000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.52000000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:42:19.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3f is not active+clean 2023-10-18T11:42:19.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3e is not active+clean 2023-10-18T11:42:19.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3d is not active+clean 2023-10-18T11:42:19.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3c is not active+clean 2023-10-18T11:42:19.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3b is not active+clean 2023-10-18T11:42:19.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3a is not active+clean 2023-10-18T11:42:19.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.39 is not active+clean 2023-10-18T11:42:19.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.38 is not active+clean 2023-10-18T11:42:19.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.37 is not active+clean 2023-10-18T11:42:19.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.36 is not active+clean 2023-10-18T11:42:19.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.35 is not active+clean 2023-10-18T11:42:19.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.34 is not active+clean 2023-10-18T11:42:19.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.33 is not active+clean 2023-10-18T11:42:19.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.32 is not active+clean 2023-10-18T11:42:19.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.31 is not active+clean 2023-10-18T11:42:19.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.30 is not active+clean 2023-10-18T11:42:19.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2f is not active+clean 2023-10-18T11:42:19.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2d is not active+clean 2023-10-18T11:42:19.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2c is not active+clean 2023-10-18T11:42:19.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2b is not active+clean 2023-10-18T11:42:19.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2a is not active+clean 2023-10-18T11:42:19.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.29 is not active+clean 2023-10-18T11:42:19.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.28 is not active+clean 2023-10-18T11:42:19.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.27 is not active+clean 2023-10-18T11:42:19.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.26 is not active+clean 2023-10-18T11:42:19.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.25 is not active+clean 2023-10-18T11:42:19.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.24 is not active+clean 2023-10-18T11:42:19.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.23 is not active+clean 2023-10-18T11:42:19.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.21 is not active+clean 2023-10-18T11:42:19.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.10 is not active+clean 2023-10-18T11:42:19.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1f is not active+clean 2023-10-18T11:42:19.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.11 is not active+clean 2023-10-18T11:42:19.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1e is not active+clean 2023-10-18T11:42:19.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.12 is not active+clean 2023-10-18T11:42:19.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1d is not active+clean 2023-10-18T11:42:19.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.13 is not active+clean 2023-10-18T11:42:19.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1c is not active+clean 2023-10-18T11:42:19.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.14 is not active+clean 2023-10-18T11:42:19.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1b is not active+clean 2023-10-18T11:42:19.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.15 is not active+clean 2023-10-18T11:42:19.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1a is not active+clean 2023-10-18T11:42:19.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.16 is not active+clean 2023-10-18T11:42:19.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.19 is not active+clean 2023-10-18T11:42:19.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.17 is not active+clean 2023-10-18T11:42:19.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.18 is not active+clean 2023-10-18T11:42:19.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.18 is not active+clean 2023-10-18T11:42:19.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.17 is not active+clean 2023-10-18T11:42:19.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1a is not active+clean 2023-10-18T11:42:19.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.15 is not active+clean 2023-10-18T11:42:19.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1b is not active+clean 2023-10-18T11:42:19.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.14 is not active+clean 2023-10-18T11:42:19.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.13 is not active+clean 2023-10-18T11:42:19.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1d is not active+clean 2023-10-18T11:42:19.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.12 is not active+clean 2023-10-18T11:42:19.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1e is not active+clean 2023-10-18T11:42:19.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.11 is not active+clean 2023-10-18T11:42:19.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1f is not active+clean 2023-10-18T11:42:19.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.10 is not active+clean 2023-10-18T11:42:19.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.0 is not active+clean 2023-10-18T11:42:19.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.f is not active+clean 2023-10-18T11:42:19.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.e is not active+clean 2023-10-18T11:42:19.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2 is not active+clean 2023-10-18T11:42:19.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.d is not active+clean 2023-10-18T11:42:19.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3 is not active+clean 2023-10-18T11:42:19.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.c is not active+clean 2023-10-18T11:42:19.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.4 is not active+clean 2023-10-18T11:42:19.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.b is not active+clean 2023-10-18T11:42:19.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.5 is not active+clean 2023-10-18T11:42:19.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.a is not active+clean 2023-10-18T11:42:19.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.6 is not active+clean 2023-10-18T11:42:19.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.9 is not active+clean 2023-10-18T11:42:19.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.7 is not active+clean 2023-10-18T11:42:19.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.8 is not active+clean 2023-10-18T11:42:19.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.8 is not active+clean 2023-10-18T11:42:19.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.7 is not active+clean 2023-10-18T11:42:19.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.9 is not active+clean 2023-10-18T11:42:19.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.6 is not active+clean 2023-10-18T11:42:19.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.a is not active+clean 2023-10-18T11:42:19.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.5 is not active+clean 2023-10-18T11:42:19.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.b is not active+clean 2023-10-18T11:42:19.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.4 is not active+clean 2023-10-18T11:42:19.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.c is not active+clean 2023-10-18T11:42:19.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3 is not active+clean 2023-10-18T11:42:19.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.d is not active+clean 2023-10-18T11:42:19.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2 is not active+clean 2023-10-18T11:42:19.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.e is not active+clean 2023-10-18T11:42:19.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1 is not active+clean 2023-10-18T11:42:19.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.f is not active+clean 2023-10-18T11:42:19.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.0 is not active+clean 2023-10-18T11:42:19.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.19 is not active+clean 2023-10-18T11:42:19.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:14.559314+0000', 'last_change': '2023-10-18T11:42:14.559314+0000', 'last_active': '2023-10-18T11:42:14.559314+0000', 'last_peered': '2023-10-18T11:42:14.559314+0000', 'last_clean': '2023-10-18T11:42:14.559314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:14.559314+0000', 'last_undegraded': '2023-10-18T11:42:14.559314+0000', 'last_fullsized': '2023-10-18T11:42:14.559314+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:14.559314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.16 is not active+clean 2023-10-18T11:42:19.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:42:13.553835+0000', 'last_change': '2023-10-18T11:42:13.553835+0000', 'last_active': '2023-10-18T11:42:13.553835+0000', 'last_peered': '2023-10-18T11:42:13.553835+0000', 'last_clean': '2023-10-18T11:42:13.553835+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:42:13.553835+0000', 'last_undegraded': '2023-10-18T11:42:13.553835+0000', 'last_fullsized': '2023-10-18T11:42:13.553835+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'last_clean_scrub_stamp': '2023-10-18T11:42:13.553835+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:42:19.298 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T11:42:19.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.397+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/2937569907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac15b2a0 msgr2=0x7f9cac0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.397+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/2937569907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac15b2a0 0x7f9cac0cf720 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f9cac08a120 tx=0x7f9ca002bcf0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.397+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/2937569907 shutdown_connections 2023-10-18T11:42:19.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.397+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/2937569907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac15b2a0 0x7f9cac0cf720 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.397+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/2937569907 >> v2:172.21.15.139:3300/0 conn(0x7f9cac0cf340 0x7f9cac0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.397+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/2937569907 >> 172.21.15.125:0/2937569907 conn(0x7f9cac0d0d60 msgr2=0x7f9cac0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:19.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.397+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/2937569907 shutdown_connections 2023-10-18T11:42:19.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.397+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/2937569907 wait complete. 2023-10-18T11:42:19.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.398+0000 7f9cb1aa2640 1 Processor -- start 2023-10-18T11:42:19.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.398+0000 7f9cb1aa2640 1 -- start start 2023-10-18T11:42:19.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cb1aa2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cac0ceb40 0x7f9cac1f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cb1aa2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac0cf340 0x7f9cac0c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cb1aa2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cac15b2a0 0x7f9cac15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cb1aa2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac158bb0 con 0x7f9cac0cf340 2023-10-18T11:42:19.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cb1aa2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1f48e0 con 0x7f9cac0ceb40 2023-10-18T11:42:19.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cb1aa2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0c95f0 con 0x7f9cac15b2a0 2023-10-18T11:42:19.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9caaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac0cf340 0x7f9cac0c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9caaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac0cf340 0x7f9cac0c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57286/0 (socket says 172.21.15.125:57286) 2023-10-18T11:42:19.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9caaffd640 1 -- 172.21.15.125:0/423211580 learned_addr learned my addr 172.21.15.125:0/423211580 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:19.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cab7fe640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cac0ceb40 0x7f9cac1f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cabfff640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cac15b2a0 0x7f9cac15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9caaffd640 1 -- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cac15b2a0 msgr2=0x7f9cac15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9caaffd640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cac15b2a0 0x7f9cac15cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9caaffd640 1 -- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cac0ceb40 msgr2=0x7f9cac1f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9caaffd640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cac0ceb40 0x7f9cac1f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9caaffd640 1 -- 172.21.15.125:0/423211580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca0009560 con 0x7f9cac0cf340 2023-10-18T11:42:19.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.399+0000 7f9cabfff640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cac15b2a0 0x7f9cac15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:19.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.400+0000 7f9caaffd640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac0cf340 0x7f9cac0c9de0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009850 tx=0x7f9ca002da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:19.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.400+0000 7f9c93fff640 1 -- 172.21.15.125:0/423211580 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0003800 con 0x7f9cac0cf340 2023-10-18T11:42:19.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.400+0000 7f9c93fff640 1 -- 172.21.15.125:0/423211580 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ca0003a50 con 0x7f9cac0cf340 2023-10-18T11:42:19.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.400+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac0c89e0 con 0x7f9cac0cf340 2023-10-18T11:42:19.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.400+0000 7f9c93fff640 1 -- 172.21.15.125:0/423211580 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0003e20 con 0x7f9cac0cf340 2023-10-18T11:42:19.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.400+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac0c8fa0 con 0x7f9cac0cf340 2023-10-18T11:42:19.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.401+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac0c9780 con 0x7f9cac0cf340 2023-10-18T11:42:19.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.401+0000 7f9c93fff640 1 -- 172.21.15.125:0/423211580 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9ca003e450 con 0x7f9cac0cf340 2023-10-18T11:42:19.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.402+0000 7f9c93fff640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9c9806af90 0x7f9c9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.402+0000 7f9c93fff640 1 -- 172.21.15.125:0/423211580 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f9ca00ac390 con 0x7f9cac0cf340 2023-10-18T11:42:19.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.402+0000 7f9cab7fe640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9c9806af90 0x7f9c9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.403+0000 7f9cab7fe640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9c9806af90 0x7f9c9806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c005fd0 tx=0x7f9c9c005850 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:19.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.406+0000 7f9c93fff640 1 -- 172.21.15.125:0/423211580 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ca0077240 con 0x7f9cac0cf340 2023-10-18T11:42:19.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.636+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9cac0cb9d0 con 0x7f9cac0cf340 2023-10-18T11:42:19.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.637+0000 7f9c93fff640 1 -- 172.21.15.125:0/423211580 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1028 (secure 0 0 0) 0x7f9ca007a860 con 0x7f9cac0cf340 2023-10-18T11:42:19.639 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:19.639 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4661,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":188,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":91},{"state_name":"active+clean","count":6}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":395853824,"bytes_avail":772698259456,"bytes_total":773094113280,"unknown_pgs_ratio":0.93814432621002197},"fsmap":{"epoch":73,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"d","status":"up:active","gid":10845}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-18T11:31:18.534143+0000","services":{}},"progress_events":{}} 2023-10-18T11:42:19.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9c9806af90 msgr2=0x7f9c9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9c9806af90 0x7f9c9806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c005fd0 tx=0x7f9c9c005850 comp rx=0 tx=0).stop 2023-10-18T11:42:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac0cf340 msgr2=0x7f9cac0c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac0cf340 0x7f9cac0c9de0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009850 tx=0x7f9ca002da20 comp rx=0 tx=0).stop 2023-10-18T11:42:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 shutdown_connections 2023-10-18T11:42:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9c9806af90 0x7f9c9806d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cac15b2a0 0x7f9cac15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cac0cf340 0x7f9cac0c9de0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 --2- 172.21.15.125:0/423211580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cac0ceb40 0x7f9cac1f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 >> 172.21.15.125:0/423211580 conn(0x7f9cac0d0d60 msgr2=0x7f9cac0d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:19.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 shutdown_connections 2023-10-18T11:42:19.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.639+0000 7f9cb1aa2640 1 -- 172.21.15.125:0/423211580 wait complete. 2023-10-18T11:42:19.652 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T11:42:19.652 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:42:19.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.753+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1315042593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e006b030 msgr2=0x7fa8e006b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.753+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1315042593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e006b030 0x7fa8e006b410 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009830 tx=0x7fa8dc02bcd0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.753+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1315042593 shutdown_connections 2023-10-18T11:42:19.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.753+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1315042593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8e0159bd0 0x7fa8e015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.753+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1315042593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e006b030 0x7fa8e006b410 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.753+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1315042593 >> 172.21.15.125:0/1315042593 conn(0x7fa8e0069930 msgr2=0x7fa8e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:19.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.753+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1315042593 shutdown_connections 2023-10-18T11:42:19.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.753+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1315042593 wait complete. 2023-10-18T11:42:19.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.754+0000 7fa8e7c02640 1 Processor -- start 2023-10-18T11:42:19.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.754+0000 7fa8e7c02640 1 -- start start 2023-10-18T11:42:19.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e7c02640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8e006b030 0x7fa8e00c1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e7c02640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8e0159bd0 0x7fa8e00caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e7c02640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e015ccb0 0x7fa8e00c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e7c02640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00c1ed0 con 0x7fa8e0159bd0 2023-10-18T11:42:19.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e7c02640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e01ef8b0 con 0x7fa8e015ccb0 2023-10-18T11:42:19.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e7c02640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e01efb90 con 0x7fa8e006b030 2023-10-18T11:42:19.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8e0159bd0 0x7fa8e00caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8e0159bd0 0x7fa8e00caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57294/0 (socket says 172.21.15.125:57294) 2023-10-18T11:42:19.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 -- 172.21.15.125:0/1396562334 learned_addr learned my addr 172.21.15.125:0/1396562334 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:19.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e619f640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e015ccb0 0x7fa8e00c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e599e640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8e006b030 0x7fa8e00c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 -- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8e006b030 msgr2=0x7fa8e00c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8e006b030 0x7fa8e00c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 -- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e015ccb0 msgr2=0x7fa8e00c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e015ccb0 0x7fa8e00c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 -- 172.21.15.125:0/1396562334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0009490 con 0x7fa8e0159bd0 2023-10-18T11:42:19.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e619f640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e015ccb0 0x7fa8e00c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T11:42:19.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e519d640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8e0159bd0 0x7fa8e00caae0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000ea60 tx=0x7fa8d000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:19.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8d67fc640 1 -- 172.21.15.125:0/1396562334 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000cde0 con 0x7fa8e0159bd0 2023-10-18T11:42:19.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8d67fc640 1 -- 172.21.15.125:0/1396562334 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa8d0004020 con 0x7fa8e0159bd0 2023-10-18T11:42:19.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.755+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc009490 con 0x7fa8e0159bd0 2023-10-18T11:42:19.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.756+0000 7fa8d67fc640 1 -- 172.21.15.125:0/1396562334 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004410 con 0x7fa8e0159bd0 2023-10-18T11:42:19.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.756+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e01f0150 con 0x7fa8e0159bd0 2023-10-18T11:42:19.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.757+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e00ab4a0 con 0x7fa8e0159bd0 2023-10-18T11:42:19.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.757+0000 7fa8d67fc640 1 -- 172.21.15.125:0/1396562334 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa8d00035f0 con 0x7fa8e0159bd0 2023-10-18T11:42:19.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.757+0000 7fa8d67fc640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa8c406af60 0x7fa8c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:19.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.757+0000 7fa8d67fc640 1 -- 172.21.15.125:0/1396562334 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa8d0014070 con 0x7fa8e0159bd0 2023-10-18T11:42:19.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.758+0000 7fa8e599e640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa8c406af60 0x7fa8c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:19.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.758+0000 7fa8e599e640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa8c406af60 0x7fa8c406d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc036000 tx=0x7fa8dc004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:19.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.761+0000 7fa8d67fc640 1 -- 172.21.15.125:0/1396562334 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8d0055300 con 0x7fa8e0159bd0 2023-10-18T11:42:19.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.926+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 --> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8e00690e0 con 0x7fa8c406af60 2023-10-18T11:42:19.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.933+0000 7fa8d67fc640 1 -- 172.21.15.125:0/1396562334 <== mgr.8003 v2:172.21.15.139:6832/3367593104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290170 (secure 0 0 0) 0x7fa8e00690e0 con 0x7fa8c406af60 2023-10-18T11:42:19.938 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:19.940 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:42:19.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa8c406af60 msgr2=0x7fa8c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa8c406af60 0x7fa8c406d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc036000 tx=0x7fa8dc004730 comp rx=0 tx=0).stop 2023-10-18T11:42:19.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8e0159bd0 msgr2=0x7fa8e00caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:19.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8e0159bd0 0x7fa8e00caae0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000ea60 tx=0x7fa8d000ef20 comp rx=0 tx=0).stop 2023-10-18T11:42:19.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 shutdown_connections 2023-10-18T11:42:19.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa8c406af60 0x7fa8c406d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8e015ccb0 0x7fa8e00c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8e0159bd0 0x7fa8e00caae0 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 --2- 172.21.15.125:0/1396562334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8e006b030 0x7fa8e00c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:19.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 >> 172.21.15.125:0/1396562334 conn(0x7fa8e0069930 msgr2=0x7fa8e014fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:19.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.936+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 shutdown_connections 2023-10-18T11:42:19.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:19.937+0000 7fa8e7c02640 1 -- 172.21.15.125:0/1396562334 wait complete. 2023-10-18T11:42:19.949 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T11:42:19.456520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":539,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386984,"kb_used_data":19552,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754587736,"statfs":{"total":773094113280,"available":772697841664,"internally_reserved":0,"allocated":20021248,"data_stored":10713148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.896508"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586840+0000","last_change":"2023-10-18T11:42:15.606818+0000","last_active":"2023-10-18T11:42:16.586840+0000","last_peered":"2023-10-18T11:42:16.586840+0000","last_clean":"2023-10-18T11:42:16.586840+0000","last_became_active":"2023-10-18T11:42:15.606386+0000","last_became_peered":"2023-10-18T11:42:15.606386+0000","last_unstale":"2023-10-18T11:42:16.586840+0000","last_undegraded":"2023-10-18T11:42:16.586840+0000","last_fullsized":"2023-10-18T11:42:16.586840+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:31.750269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.594719+0000","last_change":"2023-10-18T11:42:15.593452+0000","last_active":"2023-10-18T11:42:15.594719+0000","last_peered":"2023-10-18T11:42:15.594719+0000","last_clean":"2023-10-18T11:42:15.594719+0000","last_became_active":"2023-10-18T11:42:15.593117+0000","last_became_peered":"2023-10-18T11:42:15.593117+0000","last_unstale":"2023-10-18T11:42:15.594719+0000","last_undegraded":"2023-10-18T11:42:15.594719+0000","last_fullsized":"2023-10-18T11:42:15.594719+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:27:57.300118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586657+0000","last_change":"2023-10-18T11:42:15.599123+0000","last_active":"2023-10-18T11:42:16.586657+0000","last_peered":"2023-10-18T11:42:16.586657+0000","last_clean":"2023-10-18T11:42:16.586657+0000","last_became_active":"2023-10-18T11:42:15.598857+0000","last_became_peered":"2023-10-18T11:42:15.598857+0000","last_unstale":"2023-10-18T11:42:16.586657+0000","last_undegraded":"2023-10-18T11:42:16.586657+0000","last_fullsized":"2023-10-18T11:42:16.586657+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:03:34.127362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586870+0000","last_change":"2023-10-18T11:42:15.590534+0000","last_active":"2023-10-18T11:42:16.586870+0000","last_peered":"2023-10-18T11:42:16.586870+0000","last_clean":"2023-10-18T11:42:16.586870+0000","last_became_active":"2023-10-18T11:42:15.590273+0000","last_became_peered":"2023-10-18T11:42:15.590273+0000","last_unstale":"2023-10-18T11:42:16.586870+0000","last_undegraded":"2023-10-18T11:42:16.586870+0000","last_fullsized":"2023-10-18T11:42:16.586870+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:38.695269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.607237+0000","last_change":"2023-10-18T11:42:15.605135+0000","last_active":"2023-10-18T11:42:15.607237+0000","last_peered":"2023-10-18T11:42:15.607237+0000","last_clean":"2023-10-18T11:42:15.607237+0000","last_became_active":"2023-10-18T11:42:15.604835+0000","last_became_peered":"2023-10-18T11:42:15.604835+0000","last_unstale":"2023-10-18T11:42:15.607237+0000","last_undegraded":"2023-10-18T11:42:15.607237+0000","last_fullsized":"2023-10-18T11:42:15.607237+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:14.143639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601080+0000","last_change":"2023-10-18T11:42:15.599883+0000","last_active":"2023-10-18T11:42:15.601080+0000","last_peered":"2023-10-18T11:42:15.601080+0000","last_clean":"2023-10-18T11:42:15.601080+0000","last_became_active":"2023-10-18T11:42:15.599573+0000","last_became_peered":"2023-10-18T11:42:15.599573+0000","last_unstale":"2023-10-18T11:42:15.601080+0000","last_undegraded":"2023-10-18T11:42:15.601080+0000","last_fullsized":"2023-10-18T11:42:15.601080+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:34:03.760559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609673+0000","last_change":"2023-10-18T11:42:15.606477+0000","last_active":"2023-10-18T11:42:15.609673+0000","last_peered":"2023-10-18T11:42:15.609673+0000","last_clean":"2023-10-18T11:42:15.609673+0000","last_became_active":"2023-10-18T11:42:15.606179+0000","last_became_peered":"2023-10-18T11:42:15.606179+0000","last_unstale":"2023-10-18T11:42:15.609673+0000","last_undegraded":"2023-10-18T11:42:15.609673+0000","last_fullsized":"2023-10-18T11:42:15.609673+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:43.297692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609621+0000","last_change":"2023-10-18T11:42:15.606342+0000","last_active":"2023-10-18T11:42:15.609621+0000","last_peered":"2023-10-18T11:42:15.609621+0000","last_clean":"2023-10-18T11:42:15.609621+0000","last_became_active":"2023-10-18T11:42:15.605925+0000","last_became_peered":"2023-10-18T11:42:15.605925+0000","last_unstale":"2023-10-18T11:42:15.609621+0000","last_undegraded":"2023-10-18T11:42:15.609621+0000","last_fullsized":"2023-10-18T11:42:15.609621+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:43.204959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608851+0000","last_change":"2023-10-18T11:42:15.605555+0000","last_active":"2023-10-18T11:42:15.608851+0000","last_peered":"2023-10-18T11:42:15.608851+0000","last_clean":"2023-10-18T11:42:15.608851+0000","last_became_active":"2023-10-18T11:42:15.605309+0000","last_became_peered":"2023-10-18T11:42:15.605309+0000","last_unstale":"2023-10-18T11:42:15.608851+0000","last_undegraded":"2023-10-18T11:42:15.608851+0000","last_fullsized":"2023-10-18T11:42:15.608851+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:26:31.897953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609137+0000","last_change":"2023-10-18T11:42:15.603920+0000","last_active":"2023-10-18T11:42:15.609137+0000","last_peered":"2023-10-18T11:42:15.609137+0000","last_clean":"2023-10-18T11:42:15.609137+0000","last_became_active":"2023-10-18T11:42:15.603680+0000","last_became_peered":"2023-10-18T11:42:15.603680+0000","last_unstale":"2023-10-18T11:42:15.609137+0000","last_undegraded":"2023-10-18T11:42:15.609137+0000","last_fullsized":"2023-10-18T11:42:15.609137+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:07.197527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586100+0000","last_change":"2023-10-18T11:42:15.597015+0000","last_active":"2023-10-18T11:42:16.586100+0000","last_peered":"2023-10-18T11:42:16.586100+0000","last_clean":"2023-10-18T11:42:16.586100+0000","last_became_active":"2023-10-18T11:42:15.595860+0000","last_became_peered":"2023-10-18T11:42:15.595860+0000","last_unstale":"2023-10-18T11:42:16.586100+0000","last_undegraded":"2023-10-18T11:42:16.586100+0000","last_fullsized":"2023-10-18T11:42:16.586100+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:34:03.760559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604616+0000","last_change":"2023-10-18T11:42:15.600953+0000","last_active":"2023-10-18T11:42:15.604616+0000","last_peered":"2023-10-18T11:42:15.604616+0000","last_clean":"2023-10-18T11:42:15.604616+0000","last_became_active":"2023-10-18T11:42:15.600686+0000","last_became_peered":"2023-10-18T11:42:15.600686+0000","last_unstale":"2023-10-18T11:42:15.604616+0000","last_undegraded":"2023-10-18T11:42:15.604616+0000","last_fullsized":"2023-10-18T11:42:15.604616+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:50.573861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.602513+0000","last_change":"2023-10-18T11:42:15.595457+0000","last_active":"2023-10-18T11:42:15.602513+0000","last_peered":"2023-10-18T11:42:15.602513+0000","last_clean":"2023-10-18T11:42:15.602513+0000","last_became_active":"2023-10-18T11:42:15.595222+0000","last_became_peered":"2023-10-18T11:42:15.595222+0000","last_unstale":"2023-10-18T11:42:15.602513+0000","last_undegraded":"2023-10-18T11:42:15.602513+0000","last_fullsized":"2023-10-18T11:42:15.602513+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:46.867733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.600550+0000","last_change":"2023-10-18T11:42:15.596962+0000","last_active":"2023-10-18T11:42:15.600550+0000","last_peered":"2023-10-18T11:42:15.600550+0000","last_clean":"2023-10-18T11:42:15.600550+0000","last_became_active":"2023-10-18T11:42:15.596054+0000","last_became_peered":"2023-10-18T11:42:15.596054+0000","last_unstale":"2023-10-18T11:42:15.600550+0000","last_undegraded":"2023-10-18T11:42:15.600550+0000","last_fullsized":"2023-10-18T11:42:15.600550+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:41.953340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585654+0000","last_change":"2023-10-18T11:42:15.607300+0000","last_active":"2023-10-18T11:42:16.585654+0000","last_peered":"2023-10-18T11:42:16.585654+0000","last_clean":"2023-10-18T11:42:16.585654+0000","last_became_active":"2023-10-18T11:42:15.607018+0000","last_became_peered":"2023-10-18T11:42:15.607018+0000","last_unstale":"2023-10-18T11:42:16.585654+0000","last_undegraded":"2023-10-18T11:42:16.585654+0000","last_fullsized":"2023-10-18T11:42:16.585654+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:16:07.117805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586462+0000","last_change":"2023-10-18T11:42:15.588719+0000","last_active":"2023-10-18T11:42:16.586462+0000","last_peered":"2023-10-18T11:42:16.586462+0000","last_clean":"2023-10-18T11:42:16.586462+0000","last_became_active":"2023-10-18T11:42:15.588475+0000","last_became_peered":"2023-10-18T11:42:15.588475+0000","last_unstale":"2023-10-18T11:42:16.586462+0000","last_undegraded":"2023-10-18T11:42:16.586462+0000","last_fullsized":"2023-10-18T11:42:16.586462+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:05:27.374254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608637+0000","last_change":"2023-10-18T11:42:15.604311+0000","last_active":"2023-10-18T11:42:15.608637+0000","last_peered":"2023-10-18T11:42:15.608637+0000","last_clean":"2023-10-18T11:42:15.608637+0000","last_became_active":"2023-10-18T11:42:15.604071+0000","last_became_peered":"2023-10-18T11:42:15.604071+0000","last_unstale":"2023-10-18T11:42:15.608637+0000","last_undegraded":"2023-10-18T11:42:15.608637+0000","last_fullsized":"2023-10-18T11:42:15.608637+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:08.453473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.613794+0000","last_change":"2023-10-18T11:42:15.609772+0000","last_active":"2023-10-18T11:42:15.613794+0000","last_peered":"2023-10-18T11:42:15.613794+0000","last_clean":"2023-10-18T11:42:15.613794+0000","last_became_active":"2023-10-18T11:42:15.609451+0000","last_became_peered":"2023-10-18T11:42:15.609451+0000","last_unstale":"2023-10-18T11:42:15.613794+0000","last_undegraded":"2023-10-18T11:42:15.613794+0000","last_fullsized":"2023-10-18T11:42:15.613794+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:49:31.060237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586016+0000","last_change":"2023-10-18T11:42:15.611056+0000","last_active":"2023-10-18T11:42:16.586016+0000","last_peered":"2023-10-18T11:42:16.586016+0000","last_clean":"2023-10-18T11:42:16.586016+0000","last_became_active":"2023-10-18T11:42:15.610794+0000","last_became_peered":"2023-10-18T11:42:15.610794+0000","last_unstale":"2023-10-18T11:42:16.586016+0000","last_undegraded":"2023-10-18T11:42:16.586016+0000","last_fullsized":"2023-10-18T11:42:16.586016+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:27:57.300118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601059+0000","last_change":"2023-10-18T11:42:15.593279+0000","last_active":"2023-10-18T11:42:15.601059+0000","last_peered":"2023-10-18T11:42:15.601059+0000","last_clean":"2023-10-18T11:42:15.601059+0000","last_became_active":"2023-10-18T11:42:15.593005+0000","last_became_peered":"2023-10-18T11:42:15.593005+0000","last_unstale":"2023-10-18T11:42:15.601059+0000","last_undegraded":"2023-10-18T11:42:15.601059+0000","last_fullsized":"2023-10-18T11:42:15.601059+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:39.754401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.589319+0000","last_change":"2023-10-18T11:42:15.588164+0000","last_active":"2023-10-18T11:42:15.589319+0000","last_peered":"2023-10-18T11:42:15.589319+0000","last_clean":"2023-10-18T11:42:15.589319+0000","last_became_active":"2023-10-18T11:42:15.587874+0000","last_became_peered":"2023-10-18T11:42:15.587874+0000","last_unstale":"2023-10-18T11:42:15.589319+0000","last_undegraded":"2023-10-18T11:42:15.589319+0000","last_fullsized":"2023-10-18T11:42:15.589319+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:14.143639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.599979+0000","last_change":"2023-10-18T11:42:15.599123+0000","last_active":"2023-10-18T11:42:15.599979+0000","last_peered":"2023-10-18T11:42:15.599979+0000","last_clean":"2023-10-18T11:42:15.599979+0000","last_became_active":"2023-10-18T11:42:15.598845+0000","last_became_peered":"2023-10-18T11:42:15.598845+0000","last_unstale":"2023-10-18T11:42:15.599979+0000","last_undegraded":"2023-10-18T11:42:15.599979+0000","last_fullsized":"2023-10-18T11:42:15.599979+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:35:45.022105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601751+0000","last_change":"2023-10-18T11:42:15.597441+0000","last_active":"2023-10-18T11:42:15.601751+0000","last_peered":"2023-10-18T11:42:15.601751+0000","last_clean":"2023-10-18T11:42:15.601751+0000","last_became_active":"2023-10-18T11:42:15.597184+0000","last_became_peered":"2023-10-18T11:42:15.597184+0000","last_unstale":"2023-10-18T11:42:15.601751+0000","last_undegraded":"2023-10-18T11:42:15.601751+0000","last_fullsized":"2023-10-18T11:42:15.601751+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:17:24.248276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586662+0000","last_change":"2023-10-18T11:42:15.587854+0000","last_active":"2023-10-18T11:42:16.586662+0000","last_peered":"2023-10-18T11:42:16.586662+0000","last_clean":"2023-10-18T11:42:16.586662+0000","last_became_active":"2023-10-18T11:42:15.587595+0000","last_became_peered":"2023-10-18T11:42:15.587595+0000","last_unstale":"2023-10-18T11:42:16.586662+0000","last_undegraded":"2023-10-18T11:42:16.586662+0000","last_fullsized":"2023-10-18T11:42:16.586662+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:31.610905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.593503+0000","last_change":"2023-10-18T11:42:15.593015+0000","last_active":"2023-10-18T11:42:15.593503+0000","last_peered":"2023-10-18T11:42:15.593503+0000","last_clean":"2023-10-18T11:42:15.593503+0000","last_became_active":"2023-10-18T11:42:15.592720+0000","last_became_peered":"2023-10-18T11:42:15.592720+0000","last_unstale":"2023-10-18T11:42:15.593503+0000","last_undegraded":"2023-10-18T11:42:15.593503+0000","last_fullsized":"2023-10-18T11:42:15.593503+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:17:24.248276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.602487+0000","last_change":"2023-10-18T11:42:15.600658+0000","last_active":"2023-10-18T11:42:15.602487+0000","last_peered":"2023-10-18T11:42:15.602487+0000","last_clean":"2023-10-18T11:42:15.602487+0000","last_became_active":"2023-10-18T11:42:15.600378+0000","last_became_peered":"2023-10-18T11:42:15.600378+0000","last_unstale":"2023-10-18T11:42:15.602487+0000","last_undegraded":"2023-10-18T11:42:15.602487+0000","last_fullsized":"2023-10-18T11:42:15.602487+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:17.823820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604904+0000","last_change":"2023-10-18T11:42:15.601686+0000","last_active":"2023-10-18T11:42:15.604904+0000","last_peered":"2023-10-18T11:42:15.604904+0000","last_clean":"2023-10-18T11:42:15.604904+0000","last_became_active":"2023-10-18T11:42:15.601423+0000","last_became_peered":"2023-10-18T11:42:15.601423+0000","last_unstale":"2023-10-18T11:42:15.604904+0000","last_undegraded":"2023-10-18T11:42:15.604904+0000","last_fullsized":"2023-10-18T11:42:15.604904+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:21.407358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.607473+0000","last_change":"2023-10-18T11:42:15.603698+0000","last_active":"2023-10-18T11:42:15.607473+0000","last_peered":"2023-10-18T11:42:15.607473+0000","last_clean":"2023-10-18T11:42:15.607473+0000","last_became_active":"2023-10-18T11:42:15.603345+0000","last_became_peered":"2023-10-18T11:42:15.603345+0000","last_unstale":"2023-10-18T11:42:15.607473+0000","last_undegraded":"2023-10-18T11:42:15.607473+0000","last_fullsized":"2023-10-18T11:42:15.607473+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:41.953340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608514+0000","last_change":"2023-10-18T11:42:15.607669+0000","last_active":"2023-10-18T11:42:15.608514+0000","last_peered":"2023-10-18T11:42:15.608514+0000","last_clean":"2023-10-18T11:42:15.608514+0000","last_became_active":"2023-10-18T11:42:15.607431+0000","last_became_peered":"2023-10-18T11:42:15.607431+0000","last_unstale":"2023-10-18T11:42:15.608514+0000","last_undegraded":"2023-10-18T11:42:15.608514+0000","last_fullsized":"2023-10-18T11:42:15.608514+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:10:21.572281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605940+0000","last_change":"2023-10-18T11:42:15.598367+0000","last_active":"2023-10-18T11:42:15.605940+0000","last_peered":"2023-10-18T11:42:15.605940+0000","last_clean":"2023-10-18T11:42:15.605940+0000","last_became_active":"2023-10-18T11:42:15.598104+0000","last_became_peered":"2023-10-18T11:42:15.598104+0000","last_unstale":"2023-10-18T11:42:15.605940+0000","last_undegraded":"2023-10-18T11:42:15.605940+0000","last_fullsized":"2023-10-18T11:42:15.605940+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:00:17.431376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585593+0000","last_change":"2023-10-18T11:42:15.601677+0000","last_active":"2023-10-18T11:42:16.585593+0000","last_peered":"2023-10-18T11:42:16.585593+0000","last_clean":"2023-10-18T11:42:16.585593+0000","last_became_active":"2023-10-18T11:42:15.601193+0000","last_became_peered":"2023-10-18T11:42:15.601193+0000","last_unstale":"2023-10-18T11:42:16.585593+0000","last_undegraded":"2023-10-18T11:42:16.585593+0000","last_fullsized":"2023-10-18T11:42:16.585593+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:11.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605906+0000","last_change":"2023-10-18T11:42:15.597629+0000","last_active":"2023-10-18T11:42:15.605906+0000","last_peered":"2023-10-18T11:42:15.605906+0000","last_clean":"2023-10-18T11:42:15.605906+0000","last_became_active":"2023-10-18T11:42:15.597349+0000","last_became_peered":"2023-10-18T11:42:15.597349+0000","last_unstale":"2023-10-18T11:42:15.605906+0000","last_undegraded":"2023-10-18T11:42:15.605906+0000","last_fullsized":"2023-10-18T11:42:15.605906+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:05.997023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605694+0000","last_change":"2023-10-18T11:42:15.604291+0000","last_active":"2023-10-18T11:42:15.605694+0000","last_peered":"2023-10-18T11:42:15.605694+0000","last_clean":"2023-10-18T11:42:15.605694+0000","last_became_active":"2023-10-18T11:42:15.603821+0000","last_became_peered":"2023-10-18T11:42:15.603821+0000","last_unstale":"2023-10-18T11:42:15.605694+0000","last_undegraded":"2023-10-18T11:42:15.605694+0000","last_fullsized":"2023-10-18T11:42:15.605694+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:40:04.248063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"187'4","reported_seq":16,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586958+0000","last_change":"2023-10-18T11:42:14.582995+0000","last_active":"2023-10-18T11:42:16.586958+0000","last_peered":"2023-10-18T11:42:16.586958+0000","last_clean":"2023-10-18T11:42:16.586958+0000","last_became_active":"2023-10-18T11:42:14.582456+0000","last_became_peered":"2023-10-18T11:42:14.582456+0000","last_unstale":"2023-10-18T11:42:16.586958+0000","last_undegraded":"2023-10-18T11:42:16.586958+0000","last_fullsized":"2023-10-18T11:42:16.586958+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:23.106867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.600563+0000","last_change":"2023-10-18T11:42:15.598554+0000","last_active":"2023-10-18T11:42:15.600563+0000","last_peered":"2023-10-18T11:42:15.600563+0000","last_clean":"2023-10-18T11:42:15.600563+0000","last_became_active":"2023-10-18T11:42:15.598297+0000","last_became_peered":"2023-10-18T11:42:15.598297+0000","last_unstale":"2023-10-18T11:42:15.600563+0000","last_undegraded":"2023-10-18T11:42:15.600563+0000","last_fullsized":"2023-10-18T11:42:15.600563+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:11.601909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611376+0000","last_change":"2023-10-18T11:42:14.578398+0000","last_active":"2023-10-18T11:42:15.611376+0000","last_peered":"2023-10-18T11:42:15.611376+0000","last_clean":"2023-10-18T11:42:15.611376+0000","last_became_active":"2023-10-18T11:42:14.578133+0000","last_became_peered":"2023-10-18T11:42:14.578133+0000","last_unstale":"2023-10-18T11:42:15.611376+0000","last_undegraded":"2023-10-18T11:42:15.611376+0000","last_fullsized":"2023-10-18T11:42:15.611376+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:15.866757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.600530+0000","last_change":"2023-10-18T11:42:15.595762+0000","last_active":"2023-10-18T11:42:15.600530+0000","last_peered":"2023-10-18T11:42:15.600530+0000","last_clean":"2023-10-18T11:42:15.600530+0000","last_became_active":"2023-10-18T11:42:15.594219+0000","last_became_peered":"2023-10-18T11:42:15.594219+0000","last_unstale":"2023-10-18T11:42:15.600530+0000","last_undegraded":"2023-10-18T11:42:15.600530+0000","last_fullsized":"2023-10-18T11:42:15.600530+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:11.618673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586106+0000","last_change":"2023-10-18T11:42:14.579514+0000","last_active":"2023-10-18T11:42:16.586106+0000","last_peered":"2023-10-18T11:42:16.586106+0000","last_clean":"2023-10-18T11:42:16.586106+0000","last_became_active":"2023-10-18T11:42:14.579258+0000","last_became_peered":"2023-10-18T11:42:14.579258+0000","last_unstale":"2023-10-18T11:42:16.586106+0000","last_undegraded":"2023-10-18T11:42:16.586106+0000","last_fullsized":"2023-10-18T11:42:16.586106+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:06.050714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608101+0000","last_change":"2023-10-18T11:42:15.605505+0000","last_active":"2023-10-18T11:42:15.608101+0000","last_peered":"2023-10-18T11:42:15.608101+0000","last_clean":"2023-10-18T11:42:15.608101+0000","last_became_active":"2023-10-18T11:42:15.605229+0000","last_became_peered":"2023-10-18T11:42:15.605229+0000","last_unstale":"2023-10-18T11:42:15.608101+0000","last_undegraded":"2023-10-18T11:42:15.608101+0000","last_fullsized":"2023-10-18T11:42:15.608101+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:57.166807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601215+0000","last_change":"2023-10-18T11:42:14.578534+0000","last_active":"2023-10-18T11:42:15.601215+0000","last_peered":"2023-10-18T11:42:15.601215+0000","last_clean":"2023-10-18T11:42:15.601215+0000","last_became_active":"2023-10-18T11:42:14.577698+0000","last_became_peered":"2023-10-18T11:42:14.577698+0000","last_unstale":"2023-10-18T11:42:15.601215+0000","last_undegraded":"2023-10-18T11:42:15.601215+0000","last_fullsized":"2023-10-18T11:42:15.601215+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:14.281386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609586+0000","last_change":"2023-10-18T11:42:15.605839+0000","last_active":"2023-10-18T11:42:15.609586+0000","last_peered":"2023-10-18T11:42:15.609586+0000","last_clean":"2023-10-18T11:42:15.609586+0000","last_became_active":"2023-10-18T11:42:15.605597+0000","last_became_peered":"2023-10-18T11:42:15.605597+0000","last_unstale":"2023-10-18T11:42:15.609586+0000","last_undegraded":"2023-10-18T11:42:15.609586+0000","last_fullsized":"2023-10-18T11:42:15.609586+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:16.179404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611577+0000","last_change":"2023-10-18T11:42:14.582581+0000","last_active":"2023-10-18T11:42:15.611577+0000","last_peered":"2023-10-18T11:42:15.611577+0000","last_clean":"2023-10-18T11:42:15.611577+0000","last_became_active":"2023-10-18T11:42:14.582303+0000","last_became_peered":"2023-10-18T11:42:14.582303+0000","last_unstale":"2023-10-18T11:42:15.611577+0000","last_undegraded":"2023-10-18T11:42:15.611577+0000","last_fullsized":"2023-10-18T11:42:15.611577+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:45.649798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.602807+0000","last_change":"2023-10-18T11:42:15.600693+0000","last_active":"2023-10-18T11:42:15.602807+0000","last_peered":"2023-10-18T11:42:15.602807+0000","last_clean":"2023-10-18T11:42:15.602807+0000","last_became_active":"2023-10-18T11:42:15.600473+0000","last_became_peered":"2023-10-18T11:42:15.600473+0000","last_unstale":"2023-10-18T11:42:15.602807+0000","last_undegraded":"2023-10-18T11:42:15.602807+0000","last_fullsized":"2023-10-18T11:42:15.602807+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:51.612334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611258+0000","last_change":"2023-10-18T11:42:14.576075+0000","last_active":"2023-10-18T11:42:15.611258+0000","last_peered":"2023-10-18T11:42:15.611258+0000","last_clean":"2023-10-18T11:42:15.611258+0000","last_became_active":"2023-10-18T11:42:14.575803+0000","last_became_peered":"2023-10-18T11:42:14.575803+0000","last_unstale":"2023-10-18T11:42:15.611258+0000","last_undegraded":"2023-10-18T11:42:15.611258+0000","last_fullsized":"2023-10-18T11:42:15.611258+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:48.437435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605828+0000","last_change":"2023-10-18T11:42:15.605064+0000","last_active":"2023-10-18T11:42:15.605828+0000","last_peered":"2023-10-18T11:42:15.605828+0000","last_clean":"2023-10-18T11:42:15.605828+0000","last_became_active":"2023-10-18T11:42:15.604800+0000","last_became_peered":"2023-10-18T11:42:15.604800+0000","last_unstale":"2023-10-18T11:42:15.605828+0000","last_undegraded":"2023-10-18T11:42:15.605828+0000","last_fullsized":"2023-10-18T11:42:15.605828+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:52.794811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.586925+0000","last_change":"2023-10-18T11:42:14.581943+0000","last_active":"2023-10-18T11:42:15.586925+0000","last_peered":"2023-10-18T11:42:15.586925+0000","last_clean":"2023-10-18T11:42:15.586925+0000","last_became_active":"2023-10-18T11:42:14.581716+0000","last_became_peered":"2023-10-18T11:42:14.581716+0000","last_unstale":"2023-10-18T11:42:15.586925+0000","last_undegraded":"2023-10-18T11:42:15.586925+0000","last_fullsized":"2023-10-18T11:42:15.586925+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:57.384398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586925+0000","last_change":"2023-10-18T11:42:15.591620+0000","last_active":"2023-10-18T11:42:16.586925+0000","last_peered":"2023-10-18T11:42:16.586925+0000","last_clean":"2023-10-18T11:42:16.586925+0000","last_became_active":"2023-10-18T11:42:15.591255+0000","last_became_peered":"2023-10-18T11:42:15.591255+0000","last_unstale":"2023-10-18T11:42:16.586925+0000","last_undegraded":"2023-10-18T11:42:16.586925+0000","last_fullsized":"2023-10-18T11:42:16.586925+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:35.023607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.590169+0000","last_change":"2023-10-18T11:42:14.583849+0000","last_active":"2023-10-18T11:42:15.590169+0000","last_peered":"2023-10-18T11:42:15.590169+0000","last_clean":"2023-10-18T11:42:15.590169+0000","last_became_active":"2023-10-18T11:42:14.583604+0000","last_became_peered":"2023-10-18T11:42:14.583604+0000","last_unstale":"2023-10-18T11:42:15.590169+0000","last_undegraded":"2023-10-18T11:42:15.590169+0000","last_fullsized":"2023-10-18T11:42:15.590169+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:15:48.854696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605675+0000","last_change":"2023-10-18T11:42:15.604111+0000","last_active":"2023-10-18T11:42:15.605675+0000","last_peered":"2023-10-18T11:42:15.605675+0000","last_clean":"2023-10-18T11:42:15.605675+0000","last_became_active":"2023-10-18T11:42:15.603469+0000","last_became_peered":"2023-10-18T11:42:15.603469+0000","last_unstale":"2023-10-18T11:42:15.605675+0000","last_undegraded":"2023-10-18T11:42:15.605675+0000","last_fullsized":"2023-10-18T11:42:15.605675+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:35:55.974941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.582617+0000","last_change":"2023-10-18T11:42:14.577638+0000","last_active":"2023-10-18T11:42:15.582617+0000","last_peered":"2023-10-18T11:42:15.582617+0000","last_clean":"2023-10-18T11:42:15.582617+0000","last_became_active":"2023-10-18T11:42:14.577391+0000","last_became_peered":"2023-10-18T11:42:14.577391+0000","last_unstale":"2023-10-18T11:42:15.582617+0000","last_undegraded":"2023-10-18T11:42:15.582617+0000","last_fullsized":"2023-10-18T11:42:15.582617+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:50:55.288012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.610856+0000","last_change":"2023-10-18T11:42:15.608297+0000","last_active":"2023-10-18T11:42:15.610856+0000","last_peered":"2023-10-18T11:42:15.610856+0000","last_clean":"2023-10-18T11:42:15.610856+0000","last_became_active":"2023-10-18T11:42:15.608034+0000","last_became_peered":"2023-10-18T11:42:15.608034+0000","last_unstale":"2023-10-18T11:42:15.610856+0000","last_undegraded":"2023-10-18T11:42:15.610856+0000","last_fullsized":"2023-10-18T11:42:15.610856+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:14.572083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"187'3","reported_seq":15,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586740+0000","last_change":"2023-10-18T11:42:14.581885+0000","last_active":"2023-10-18T11:42:16.586740+0000","last_peered":"2023-10-18T11:42:16.586740+0000","last_clean":"2023-10-18T11:42:16.586740+0000","last_became_active":"2023-10-18T11:42:14.581610+0000","last_became_peered":"2023-10-18T11:42:14.581610+0000","last_unstale":"2023-10-18T11:42:16.586740+0000","last_undegraded":"2023-10-18T11:42:16.586740+0000","last_fullsized":"2023-10-18T11:42:16.586740+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:34.171530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601289+0000","last_change":"2023-10-18T11:42:15.594687+0000","last_active":"2023-10-18T11:42:15.601289+0000","last_peered":"2023-10-18T11:42:15.601289+0000","last_clean":"2023-10-18T11:42:15.601289+0000","last_became_active":"2023-10-18T11:42:15.593908+0000","last_became_peered":"2023-10-18T11:42:15.593908+0000","last_unstale":"2023-10-18T11:42:15.601289+0000","last_undegraded":"2023-10-18T11:42:15.601289+0000","last_fullsized":"2023-10-18T11:42:15.601289+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:44.892491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"187'6","reported_seq":17,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.614491+0000","last_change":"2023-10-18T11:42:14.572036+0000","last_active":"2023-10-18T11:42:15.614491+0000","last_peered":"2023-10-18T11:42:15.614491+0000","last_clean":"2023-10-18T11:42:15.614491+0000","last_became_active":"2023-10-18T11:42:14.571772+0000","last_became_peered":"2023-10-18T11:42:14.571772+0000","last_unstale":"2023-10-18T11:42:15.614491+0000","last_undegraded":"2023-10-18T11:42:15.614491+0000","last_fullsized":"2023-10-18T11:42:15.614491+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:07.655075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604060+0000","last_change":"2023-10-18T11:42:15.596831+0000","last_active":"2023-10-18T11:42:15.604060+0000","last_peered":"2023-10-18T11:42:15.604060+0000","last_clean":"2023-10-18T11:42:15.604060+0000","last_became_active":"2023-10-18T11:42:15.596580+0000","last_became_peered":"2023-10-18T11:42:15.596580+0000","last_unstale":"2023-10-18T11:42:15.604060+0000","last_undegraded":"2023-10-18T11:42:15.604060+0000","last_fullsized":"2023-10-18T11:42:15.604060+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:38.982049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605223+0000","last_change":"2023-10-18T11:42:14.577480+0000","last_active":"2023-10-18T11:42:15.605223+0000","last_peered":"2023-10-18T11:42:15.605223+0000","last_clean":"2023-10-18T11:42:15.605223+0000","last_became_active":"2023-10-18T11:42:14.577183+0000","last_became_peered":"2023-10-18T11:42:14.577183+0000","last_unstale":"2023-10-18T11:42:15.605223+0000","last_undegraded":"2023-10-18T11:42:15.605223+0000","last_fullsized":"2023-10-18T11:42:15.605223+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:05.696631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611472+0000","last_change":"2023-10-18T11:42:15.609248+0000","last_active":"2023-10-18T11:42:15.611472+0000","last_peered":"2023-10-18T11:42:15.611472+0000","last_clean":"2023-10-18T11:42:15.611472+0000","last_became_active":"2023-10-18T11:42:15.604621+0000","last_became_peered":"2023-10-18T11:42:15.604621+0000","last_unstale":"2023-10-18T11:42:15.611472+0000","last_undegraded":"2023-10-18T11:42:15.611472+0000","last_fullsized":"2023-10-18T11:42:15.611472+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:35.614614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.583043+0000","last_change":"2023-10-18T11:42:14.574522+0000","last_active":"2023-10-18T11:42:15.583043+0000","last_peered":"2023-10-18T11:42:15.583043+0000","last_clean":"2023-10-18T11:42:15.583043+0000","last_became_active":"2023-10-18T11:42:14.574248+0000","last_became_peered":"2023-10-18T11:42:14.574248+0000","last_unstale":"2023-10-18T11:42:15.583043+0000","last_undegraded":"2023-10-18T11:42:15.583043+0000","last_fullsized":"2023-10-18T11:42:15.583043+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:10.613194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585921+0000","last_change":"2023-10-18T11:42:15.594310+0000","last_active":"2023-10-18T11:42:16.585921+0000","last_peered":"2023-10-18T11:42:16.585921+0000","last_clean":"2023-10-18T11:42:16.585921+0000","last_became_active":"2023-10-18T11:42:15.594051+0000","last_became_peered":"2023-10-18T11:42:15.594051+0000","last_unstale":"2023-10-18T11:42:16.585921+0000","last_undegraded":"2023-10-18T11:42:16.585921+0000","last_fullsized":"2023-10-18T11:42:16.585921+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:36.610510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585693+0000","last_change":"2023-10-18T11:42:14.578913+0000","last_active":"2023-10-18T11:42:16.585693+0000","last_peered":"2023-10-18T11:42:16.585693+0000","last_clean":"2023-10-18T11:42:16.585693+0000","last_became_active":"2023-10-18T11:42:14.578638+0000","last_became_peered":"2023-10-18T11:42:14.578638+0000","last_unstale":"2023-10-18T11:42:16.585693+0000","last_undegraded":"2023-10-18T11:42:16.585693+0000","last_fullsized":"2023-10-18T11:42:16.585693+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:32.729146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604428+0000","last_change":"2023-10-18T11:42:15.602323+0000","last_active":"2023-10-18T11:42:15.604428+0000","last_peered":"2023-10-18T11:42:15.604428+0000","last_clean":"2023-10-18T11:42:15.604428+0000","last_became_active":"2023-10-18T11:42:15.602075+0000","last_became_peered":"2023-10-18T11:42:15.602075+0000","last_unstale":"2023-10-18T11:42:15.604428+0000","last_undegraded":"2023-10-18T11:42:15.604428+0000","last_fullsized":"2023-10-18T11:42:15.604428+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:55:43.654543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"187'8","reported_seq":19,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.610476+0000","last_change":"2023-10-18T11:42:14.578512+0000","last_active":"2023-10-18T11:42:15.610476+0000","last_peered":"2023-10-18T11:42:15.610476+0000","last_clean":"2023-10-18T11:42:15.610476+0000","last_became_active":"2023-10-18T11:42:14.578237+0000","last_became_peered":"2023-10-18T11:42:14.578237+0000","last_unstale":"2023-10-18T11:42:15.610476+0000","last_undegraded":"2023-10-18T11:42:15.610476+0000","last_fullsized":"2023-10-18T11:42:15.610476+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:36.860904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608880+0000","last_change":"2023-10-18T11:42:15.605677+0000","last_active":"2023-10-18T11:42:15.608880+0000","last_peered":"2023-10-18T11:42:15.608880+0000","last_clean":"2023-10-18T11:42:15.608880+0000","last_became_active":"2023-10-18T11:42:15.602902+0000","last_became_peered":"2023-10-18T11:42:15.602902+0000","last_unstale":"2023-10-18T11:42:15.608880+0000","last_undegraded":"2023-10-18T11:42:15.608880+0000","last_fullsized":"2023-10-18T11:42:15.608880+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:46:22.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.614092+0000","last_change":"2023-10-18T11:42:14.582338+0000","last_active":"2023-10-18T11:42:15.614092+0000","last_peered":"2023-10-18T11:42:15.614092+0000","last_clean":"2023-10-18T11:42:15.614092+0000","last_became_active":"2023-10-18T11:42:14.582001+0000","last_became_peered":"2023-10-18T11:42:14.582001+0000","last_unstale":"2023-10-18T11:42:15.614092+0000","last_undegraded":"2023-10-18T11:42:15.614092+0000","last_fullsized":"2023-10-18T11:42:15.614092+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:38.500447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608663+0000","last_change":"2023-10-18T11:42:15.605780+0000","last_active":"2023-10-18T11:42:15.608663+0000","last_peered":"2023-10-18T11:42:15.608663+0000","last_clean":"2023-10-18T11:42:15.608663+0000","last_became_active":"2023-10-18T11:42:15.600458+0000","last_became_peered":"2023-10-18T11:42:15.600458+0000","last_unstale":"2023-10-18T11:42:15.608663+0000","last_undegraded":"2023-10-18T11:42:15.608663+0000","last_fullsized":"2023-10-18T11:42:15.608663+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:57.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609089+0000","last_change":"2023-10-18T11:42:14.582657+0000","last_active":"2023-10-18T11:42:15.609089+0000","last_peered":"2023-10-18T11:42:15.609089+0000","last_clean":"2023-10-18T11:42:15.609089+0000","last_became_active":"2023-10-18T11:42:14.582386+0000","last_became_peered":"2023-10-18T11:42:14.582386+0000","last_unstale":"2023-10-18T11:42:15.609089+0000","last_undegraded":"2023-10-18T11:42:15.609089+0000","last_fullsized":"2023-10-18T11:42:15.609089+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:32.729146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585607+0000","last_change":"2023-10-18T11:42:15.596605+0000","last_active":"2023-10-18T11:42:16.585607+0000","last_peered":"2023-10-18T11:42:16.585607+0000","last_clean":"2023-10-18T11:42:16.585607+0000","last_became_active":"2023-10-18T11:42:15.596305+0000","last_became_peered":"2023-10-18T11:42:15.596305+0000","last_unstale":"2023-10-18T11:42:16.585607+0000","last_undegraded":"2023-10-18T11:42:16.585607+0000","last_fullsized":"2023-10-18T11:42:16.585607+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:32.264398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609703+0000","last_change":"2023-10-18T11:42:14.582207+0000","last_active":"2023-10-18T11:42:15.609703+0000","last_peered":"2023-10-18T11:42:15.609703+0000","last_clean":"2023-10-18T11:42:15.609703+0000","last_became_active":"2023-10-18T11:42:14.581949+0000","last_became_peered":"2023-10-18T11:42:14.581949+0000","last_unstale":"2023-10-18T11:42:15.609703+0000","last_undegraded":"2023-10-18T11:42:15.609703+0000","last_fullsized":"2023-10-18T11:42:15.609703+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:29.706251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585805+0000","last_change":"2023-10-18T11:42:15.596528+0000","last_active":"2023-10-18T11:42:16.585805+0000","last_peered":"2023-10-18T11:42:16.585805+0000","last_clean":"2023-10-18T11:42:16.585805+0000","last_became_active":"2023-10-18T11:42:15.595978+0000","last_became_peered":"2023-10-18T11:42:15.595978+0000","last_unstale":"2023-10-18T11:42:16.585805+0000","last_undegraded":"2023-10-18T11:42:16.585805+0000","last_fullsized":"2023-10-18T11:42:16.585805+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:05.109362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.594939+0000","last_change":"2023-10-18T11:42:14.583437+0000","last_active":"2023-10-18T11:42:15.594939+0000","last_peered":"2023-10-18T11:42:15.594939+0000","last_clean":"2023-10-18T11:42:15.594939+0000","last_became_active":"2023-10-18T11:42:14.583103+0000","last_became_peered":"2023-10-18T11:42:14.583103+0000","last_unstale":"2023-10-18T11:42:15.594939+0000","last_undegraded":"2023-10-18T11:42:15.594939+0000","last_fullsized":"2023-10-18T11:42:15.594939+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:16:21.073740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.607493+0000","last_change":"2023-10-18T11:42:15.603852+0000","last_active":"2023-10-18T11:42:15.607493+0000","last_peered":"2023-10-18T11:42:15.607493+0000","last_clean":"2023-10-18T11:42:15.607493+0000","last_became_active":"2023-10-18T11:42:15.602918+0000","last_became_peered":"2023-10-18T11:42:15.602918+0000","last_unstale":"2023-10-18T11:42:15.607493+0000","last_undegraded":"2023-10-18T11:42:15.607493+0000","last_fullsized":"2023-10-18T11:42:15.607493+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:19.121794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.606039+0000","last_change":"2023-10-18T11:42:14.578859+0000","last_active":"2023-10-18T11:42:15.606039+0000","last_peered":"2023-10-18T11:42:15.606039+0000","last_clean":"2023-10-18T11:42:15.606039+0000","last_became_active":"2023-10-18T11:42:14.577551+0000","last_became_peered":"2023-10-18T11:42:14.577551+0000","last_unstale":"2023-10-18T11:42:15.606039+0000","last_undegraded":"2023-10-18T11:42:15.606039+0000","last_fullsized":"2023-10-18T11:42:15.606039+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:31.497909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.612141+0000","last_change":"2023-10-18T11:42:15.611433+0000","last_active":"2023-10-18T11:42:15.612141+0000","last_peered":"2023-10-18T11:42:15.612141+0000","last_clean":"2023-10-18T11:42:15.612141+0000","last_became_active":"2023-10-18T11:42:15.611114+0000","last_became_peered":"2023-10-18T11:42:15.611114+0000","last_unstale":"2023-10-18T11:42:15.612141+0000","last_undegraded":"2023-10-18T11:42:15.612141+0000","last_fullsized":"2023-10-18T11:42:15.612141+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:08.660554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585535+0000","last_change":"2023-10-18T11:42:14.574506+0000","last_active":"2023-10-18T11:42:16.585535+0000","last_peered":"2023-10-18T11:42:16.585535+0000","last_clean":"2023-10-18T11:42:16.585535+0000","last_became_active":"2023-10-18T11:42:14.574184+0000","last_became_peered":"2023-10-18T11:42:14.574184+0000","last_unstale":"2023-10-18T11:42:16.585535+0000","last_undegraded":"2023-10-18T11:42:16.585535+0000","last_fullsized":"2023-10-18T11:42:16.585535+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:16:21.073740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.612510+0000","last_change":"2023-10-18T11:42:15.611523+0000","last_active":"2023-10-18T11:42:15.612510+0000","last_peered":"2023-10-18T11:42:15.612510+0000","last_clean":"2023-10-18T11:42:15.612510+0000","last_became_active":"2023-10-18T11:42:15.611179+0000","last_became_peered":"2023-10-18T11:42:15.611179+0000","last_unstale":"2023-10-18T11:42:15.612510+0000","last_undegraded":"2023-10-18T11:42:15.612510+0000","last_fullsized":"2023-10-18T11:42:15.612510+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:19.121794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.587545+0000","last_change":"2023-10-18T11:42:14.575953+0000","last_active":"2023-10-18T11:42:15.587545+0000","last_peered":"2023-10-18T11:42:15.587545+0000","last_clean":"2023-10-18T11:42:15.587545+0000","last_became_active":"2023-10-18T11:42:14.575689+0000","last_became_peered":"2023-10-18T11:42:14.575689+0000","last_unstale":"2023-10-18T11:42:15.587545+0000","last_undegraded":"2023-10-18T11:42:15.587545+0000","last_fullsized":"2023-10-18T11:42:15.587545+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:55.425254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.594046+0000","last_change":"2023-10-18T11:42:15.592858+0000","last_active":"2023-10-18T11:42:15.594046+0000","last_peered":"2023-10-18T11:42:15.594046+0000","last_clean":"2023-10-18T11:42:15.594046+0000","last_became_active":"2023-10-18T11:42:15.592595+0000","last_became_peered":"2023-10-18T11:42:15.592595+0000","last_unstale":"2023-10-18T11:42:15.594046+0000","last_undegraded":"2023-10-18T11:42:15.594046+0000","last_fullsized":"2023-10-18T11:42:15.594046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:36.610510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.591191+0000","last_change":"2023-10-18T11:42:14.581265+0000","last_active":"2023-10-18T11:42:15.591191+0000","last_peered":"2023-10-18T11:42:15.591191+0000","last_clean":"2023-10-18T11:42:15.591191+0000","last_became_active":"2023-10-18T11:42:14.580990+0000","last_became_peered":"2023-10-18T11:42:14.580990+0000","last_unstale":"2023-10-18T11:42:15.591191+0000","last_undegraded":"2023-10-18T11:42:15.591191+0000","last_fullsized":"2023-10-18T11:42:15.591191+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:24.806953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585607+0000","last_change":"2023-10-18T11:42:15.595645+0000","last_active":"2023-10-18T11:42:16.585607+0000","last_peered":"2023-10-18T11:42:16.585607+0000","last_clean":"2023-10-18T11:42:16.585607+0000","last_became_active":"2023-10-18T11:42:15.595343+0000","last_became_peered":"2023-10-18T11:42:15.595343+0000","last_unstale":"2023-10-18T11:42:16.585607+0000","last_undegraded":"2023-10-18T11:42:16.585607+0000","last_fullsized":"2023-10-18T11:42:16.585607+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:11.601195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.606353+0000","last_change":"2023-10-18T11:42:14.579758+0000","last_active":"2023-10-18T11:42:15.606353+0000","last_peered":"2023-10-18T11:42:15.606353+0000","last_clean":"2023-10-18T11:42:15.606353+0000","last_became_active":"2023-10-18T11:42:14.579505+0000","last_became_peered":"2023-10-18T11:42:14.579505+0000","last_unstale":"2023-10-18T11:42:15.606353+0000","last_undegraded":"2023-10-18T11:42:15.606353+0000","last_fullsized":"2023-10-18T11:42:15.606353+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:09:56.691445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611677+0000","last_change":"2023-10-18T11:42:15.608870+0000","last_active":"2023-10-18T11:42:15.611677+0000","last_peered":"2023-10-18T11:42:15.611677+0000","last_clean":"2023-10-18T11:42:15.611677+0000","last_became_active":"2023-10-18T11:42:15.605790+0000","last_became_peered":"2023-10-18T11:42:15.605790+0000","last_unstale":"2023-10-18T11:42:15.611677+0000","last_undegraded":"2023-10-18T11:42:15.611677+0000","last_fullsized":"2023-10-18T11:42:15.611677+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:32.258275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.583007+0000","last_change":"2023-10-18T11:42:14.576812+0000","last_active":"2023-10-18T11:42:15.583007+0000","last_peered":"2023-10-18T11:42:15.583007+0000","last_clean":"2023-10-18T11:42:15.583007+0000","last_became_active":"2023-10-18T11:42:14.576377+0000","last_became_peered":"2023-10-18T11:42:14.576377+0000","last_unstale":"2023-10-18T11:42:15.583007+0000","last_undegraded":"2023-10-18T11:42:15.583007+0000","last_fullsized":"2023-10-18T11:42:15.583007+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:15.866757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.602132+0000","last_change":"2023-10-18T11:42:15.599807+0000","last_active":"2023-10-18T11:42:15.602132+0000","last_peered":"2023-10-18T11:42:15.602132+0000","last_clean":"2023-10-18T11:42:15.602132+0000","last_became_active":"2023-10-18T11:42:15.599536+0000","last_became_peered":"2023-10-18T11:42:15.599536+0000","last_unstale":"2023-10-18T11:42:15.602132+0000","last_undegraded":"2023-10-18T11:42:15.602132+0000","last_fullsized":"2023-10-18T11:42:15.602132+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:38.798007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.582769+0000","last_change":"2023-10-18T11:42:14.581450+0000","last_active":"2023-10-18T11:42:15.582769+0000","last_peered":"2023-10-18T11:42:15.582769+0000","last_clean":"2023-10-18T11:42:15.582769+0000","last_became_active":"2023-10-18T11:42:14.581174+0000","last_became_peered":"2023-10-18T11:42:14.581174+0000","last_unstale":"2023-10-18T11:42:15.582769+0000","last_undegraded":"2023-10-18T11:42:15.582769+0000","last_fullsized":"2023-10-18T11:42:15.582769+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:44:02.797286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609730+0000","last_change":"2023-10-18T11:42:15.606580+0000","last_active":"2023-10-18T11:42:15.609730+0000","last_peered":"2023-10-18T11:42:15.609730+0000","last_clean":"2023-10-18T11:42:15.609730+0000","last_became_active":"2023-10-18T11:42:15.604639+0000","last_became_peered":"2023-10-18T11:42:15.604639+0000","last_unstale":"2023-10-18T11:42:15.609730+0000","last_undegraded":"2023-10-18T11:42:15.609730+0000","last_fullsized":"2023-10-18T11:42:15.609730+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:03.790016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.587063+0000","last_change":"2023-10-18T11:42:14.581315+0000","last_active":"2023-10-18T11:42:15.587063+0000","last_peered":"2023-10-18T11:42:15.587063+0000","last_clean":"2023-10-18T11:42:15.587063+0000","last_became_active":"2023-10-18T11:42:14.581077+0000","last_became_peered":"2023-10-18T11:42:14.581077+0000","last_unstale":"2023-10-18T11:42:15.587063+0000","last_undegraded":"2023-10-18T11:42:15.587063+0000","last_fullsized":"2023-10-18T11:42:15.587063+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.198283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586960+0000","last_change":"2023-10-18T11:42:15.596716+0000","last_active":"2023-10-18T11:42:16.586960+0000","last_peered":"2023-10-18T11:42:16.586960+0000","last_clean":"2023-10-18T11:42:16.586960+0000","last_became_active":"2023-10-18T11:42:15.596467+0000","last_became_peered":"2023-10-18T11:42:15.596467+0000","last_unstale":"2023-10-18T11:42:16.586960+0000","last_undegraded":"2023-10-18T11:42:16.586960+0000","last_fullsized":"2023-10-18T11:42:16.586960+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:09.454600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586526+0000","last_change":"2023-10-18T11:42:14.582780+0000","last_active":"2023-10-18T11:42:16.586526+0000","last_peered":"2023-10-18T11:42:16.586526+0000","last_clean":"2023-10-18T11:42:16.586526+0000","last_became_active":"2023-10-18T11:42:14.582378+0000","last_became_peered":"2023-10-18T11:42:14.582378+0000","last_unstale":"2023-10-18T11:42:16.586526+0000","last_undegraded":"2023-10-18T11:42:16.586526+0000","last_fullsized":"2023-10-18T11:42:16.586526+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:49.492744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.606761+0000","last_change":"2023-10-18T11:42:15.604831+0000","last_active":"2023-10-18T11:42:15.606761+0000","last_peered":"2023-10-18T11:42:15.606761+0000","last_clean":"2023-10-18T11:42:15.606761+0000","last_became_active":"2023-10-18T11:42:15.604581+0000","last_became_peered":"2023-10-18T11:42:15.604581+0000","last_unstale":"2023-10-18T11:42:15.606761+0000","last_undegraded":"2023-10-18T11:42:15.606761+0000","last_fullsized":"2023-10-18T11:42:15.606761+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:43:56.873122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.582874+0000","last_change":"2023-10-18T11:42:14.574027+0000","last_active":"2023-10-18T11:42:15.582874+0000","last_peered":"2023-10-18T11:42:15.582874+0000","last_clean":"2023-10-18T11:42:15.582874+0000","last_became_active":"2023-10-18T11:42:14.573748+0000","last_became_peered":"2023-10-18T11:42:14.573748+0000","last_unstale":"2023-10-18T11:42:15.582874+0000","last_undegraded":"2023-10-18T11:42:15.582874+0000","last_fullsized":"2023-10-18T11:42:15.582874+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:09:56.691445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.592646+0000","last_change":"2023-10-18T11:42:15.589028+0000","last_active":"2023-10-18T11:42:15.592646+0000","last_peered":"2023-10-18T11:42:15.592646+0000","last_clean":"2023-10-18T11:42:15.592646+0000","last_became_active":"2023-10-18T11:42:15.588776+0000","last_became_peered":"2023-10-18T11:42:15.588776+0000","last_unstale":"2023-10-18T11:42:15.592646+0000","last_undegraded":"2023-10-18T11:42:15.592646+0000","last_fullsized":"2023-10-18T11:42:15.592646+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:15.252200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585763+0000","last_change":"2023-10-18T11:42:14.578983+0000","last_active":"2023-10-18T11:42:16.585763+0000","last_peered":"2023-10-18T11:42:16.585763+0000","last_clean":"2023-10-18T11:42:16.585763+0000","last_became_active":"2023-10-18T11:42:14.578147+0000","last_became_peered":"2023-10-18T11:42:14.578147+0000","last_unstale":"2023-10-18T11:42:16.585763+0000","last_undegraded":"2023-10-18T11:42:16.585763+0000","last_fullsized":"2023-10-18T11:42:16.585763+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:22.549891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.598583+0000","last_change":"2023-10-18T11:42:15.592580+0000","last_active":"2023-10-18T11:42:15.598583+0000","last_peered":"2023-10-18T11:42:15.598583+0000","last_clean":"2023-10-18T11:42:15.598583+0000","last_became_active":"2023-10-18T11:42:15.592324+0000","last_became_peered":"2023-10-18T11:42:15.592324+0000","last_unstale":"2023-10-18T11:42:15.598583+0000","last_undegraded":"2023-10-18T11:42:15.598583+0000","last_fullsized":"2023-10-18T11:42:15.598583+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:23.509994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.587225+0000","last_change":"2023-10-18T11:42:14.580455+0000","last_active":"2023-10-18T11:42:15.587225+0000","last_peered":"2023-10-18T11:42:15.587225+0000","last_clean":"2023-10-18T11:42:15.587225+0000","last_became_active":"2023-10-18T11:42:14.580204+0000","last_became_peered":"2023-10-18T11:42:14.580204+0000","last_unstale":"2023-10-18T11:42:15.587225+0000","last_undegraded":"2023-10-18T11:42:15.587225+0000","last_fullsized":"2023-10-18T11:42:15.587225+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:14.246721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608526+0000","last_change":"2023-10-18T11:42:15.606583+0000","last_active":"2023-10-18T11:42:15.608526+0000","last_peered":"2023-10-18T11:42:15.608526+0000","last_clean":"2023-10-18T11:42:15.608526+0000","last_became_active":"2023-10-18T11:42:15.606318+0000","last_became_peered":"2023-10-18T11:42:15.606318+0000","last_unstale":"2023-10-18T11:42:15.608526+0000","last_undegraded":"2023-10-18T11:42:15.608526+0000","last_fullsized":"2023-10-18T11:42:15.608526+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:05.109362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.582828+0000","last_change":"2023-10-18T11:42:14.578933+0000","last_active":"2023-10-18T11:42:15.582828+0000","last_peered":"2023-10-18T11:42:15.582828+0000","last_clean":"2023-10-18T11:42:15.582828+0000","last_became_active":"2023-10-18T11:42:14.578128+0000","last_became_peered":"2023-10-18T11:42:14.578128+0000","last_unstale":"2023-10-18T11:42:15.582828+0000","last_undegraded":"2023-10-18T11:42:15.582828+0000","last_fullsized":"2023-10-18T11:42:15.582828+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:44:56.908102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"176'302","reported_seq":999,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-10-18T11:42:14.576038+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:42:14.576038+0000","last_peered":"2023-10-18T11:42:14.576038+0000","last_clean":"2023-10-18T11:42:14.576038+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:42:14.576038+0000","last_undegraded":"2023-10-18T11:42:14.576038+0000","last_fullsized":"2023-10-18T11:42:14.576038+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575775,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52104,"kb_used_data":2868,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319736,"statfs":{"total":96636764160,"available":96583409664,"internally_reserved":0,"allocated":2936832,"data_stored":1779655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64900000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.61299999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.47699999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.629}]}]},{"osd":6,"up_from":13,"seq":55834575775,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2342912,"data_stored":1189287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.44}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575775,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59860,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311980,"statfs":{"total":96636764160,"available":96575467520,"internally_reserved":0,"allocated":2363392,"data_stored":1196059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.56399999999999995}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.624}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.496},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.45600000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.43099999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.372},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.47499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41824,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330016,"statfs":{"total":96636764160,"available":96593936384,"internally_reserved":0,"allocated":2371584,"data_stored":1196699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.622,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.27800000000000002},"max":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.872,"15min":0.872},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"max":{"1min":0.80700000000000005,"5min":0.82999999999999996,"15min":0.86399999999999999},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.85799999999999998},"last":0.751}]},{"osd":3,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.626},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.504,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.5},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.501},"min":{"1min":0.318,"5min":0.313,"15min":0.313},"max":{"1min":0.73799999999999999,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.47299999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75800000000000001,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.436}]}]},{"osd":3,"up_from":13,"seq":55834575775,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2363392,"data_stored":1191503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.22800000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.251},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.30499999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.35999999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.44900000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.35099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575775,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59840,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312000,"statfs":{"total":96636764160,"available":96575488000,"internally_reserved":0,"allocated":2342912,"data_stored":1189287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.311}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.48599999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.503},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.42999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575775,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":2355200,"data_stored":1190899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.376}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.623}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.65400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575774,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38220,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333620,"statfs":{"total":96636764160,"available":96597626880,"internally_reserved":0,"allocated":2945024,"data_stored":1779759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.312},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.32900000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.374}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.41599999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.51700000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.53700000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.497}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:42:20.551 INFO:tasks.ceph.mds.d.smithi139.stderr:2023-10-18T11:42:20.549+0000 7faba3120640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T11:42:22.168 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:42:22.171 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:42:22.956 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:42:23.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.058+0000 7fae96eec640 1 -- 172.21.15.125:0/1089666213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 msgr2=0x7fae90158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.058+0000 7fae96eec640 1 --2- 172.21.15.125:0/1089666213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 0x7fae90158300 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fae8400a940 tx=0x7fae8402d090 comp rx=0 tx=0).stop 2023-10-18T11:42:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.058+0000 7fae96eec640 1 -- 172.21.15.125:0/1089666213 shutdown_connections 2023-10-18T11:42:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.058+0000 7fae96eec640 1 --2- 172.21.15.125:0/1089666213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae900afa20 0x7fae900705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.058+0000 7fae96eec640 1 --2- 172.21.15.125:0/1089666213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 0x7fae90158300 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.058+0000 7fae96eec640 1 -- 172.21.15.125:0/1089666213 >> 172.21.15.125:0/1089666213 conn(0x7fae90069930 msgr2=0x7fae90069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:23.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.058+0000 7fae96eec640 1 -- 172.21.15.125:0/1089666213 shutdown_connections 2023-10-18T11:42:23.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.058+0000 7fae96eec640 1 -- 172.21.15.125:0/1089666213 wait complete. 2023-10-18T11:42:23.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.059+0000 7fae96eec640 1 Processor -- start 2023-10-18T11:42:23.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.059+0000 7fae96eec640 1 -- start start 2023-10-18T11:42:23.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae96eec640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae900afa20 0x7fae901eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae96eec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae900b0530 0x7fae901e2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae96eec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 0x7fae901e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae96eec640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae901535d0 con 0x7fae900cbd20 2023-10-18T11:42:23.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae96eec640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae90152c50 con 0x7fae900b0530 2023-10-18T11:42:23.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae96eec640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae90153820 con 0x7fae900afa20 2023-10-18T11:42:23.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae95489640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 0x7fae901e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae95489640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 0x7fae901e2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57308/0 (socket says 172.21.15.125:57308) 2023-10-18T11:42:23.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae95489640 1 -- 172.21.15.125:0/2964455572 learned_addr learned my addr 172.21.15.125:0/2964455572 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:23.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae8ffff640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae900b0530 0x7fae901e2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae94c88640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae900afa20 0x7fae901eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae95489640 1 -- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae900afa20 msgr2=0x7fae901eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae95489640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae900afa20 0x7fae901eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae95489640 1 -- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae900b0530 msgr2=0x7fae901e2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae95489640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae900b0530 0x7fae901e2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.060+0000 7fae95489640 1 -- 172.21.15.125:0/2964455572 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8400a5a0 con 0x7fae900cbd20 2023-10-18T11:42:23.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.061+0000 7fae95489640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 0x7fae901e2a80 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fae8800cae0 tx=0x7fae8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:23.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.061+0000 7fae8d7fa640 1 -- 172.21.15.125:0/2964455572 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8800ac40 con 0x7fae900cbd20 2023-10-18T11:42:23.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.061+0000 7fae8d7fa640 1 -- 172.21.15.125:0/2964455572 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae8800ae90 con 0x7fae900cbd20 2023-10-18T11:42:23.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.061+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae90153b00 con 0x7fae900cbd20 2023-10-18T11:42:23.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.061+0000 7fae8d7fa640 1 -- 172.21.15.125:0/2964455572 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88003bd0 con 0x7fae900cbd20 2023-10-18T11:42:23.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.061+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae901ef230 con 0x7fae900cbd20 2023-10-18T11:42:23.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.062+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae90154020 con 0x7fae900cbd20 2023-10-18T11:42:23.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.062+0000 7fae8d7fa640 1 -- 172.21.15.125:0/2964455572 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fae88003d30 con 0x7fae900cbd20 2023-10-18T11:42:23.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.063+0000 7fae8d7fa640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fae7406af60 0x7fae7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.063+0000 7fae8d7fa640 1 -- 172.21.15.125:0/2964455572 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fae8808a220 con 0x7fae900cbd20 2023-10-18T11:42:23.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.063+0000 7fae94c88640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fae7406af60 0x7fae7406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.063+0000 7fae94c88640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fae7406af60 0x7fae7406d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae84037000 tx=0x7fae84004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:23.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.067+0000 7fae8d7fa640 1 -- 172.21.15.125:0/2964455572 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae90154020 con 0x7fae900cbd20 2023-10-18T11:42:23.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.237+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 --> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae9006b910 con 0x7fae7406af60 2023-10-18T11:42:23.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.244+0000 7fae8d7fa640 1 -- 172.21.15.125:0/2964455572 <== mgr.8003 v2:172.21.15.139:6832/3367593104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290170 (secure 0 0 0) 0x7fae84004410 con 0x7fae7406af60 2023-10-18T11:42:23.250 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:23.251 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:42:23.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fae7406af60 msgr2=0x7fae7406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fae7406af60 0x7fae7406d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae84037000 tx=0x7fae84004730 comp rx=0 tx=0).stop 2023-10-18T11:42:23.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 msgr2=0x7fae901e2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 0x7fae901e2a80 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fae8800cae0 tx=0x7fae8800a060 comp rx=0 tx=0).stop 2023-10-18T11:42:23.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 shutdown_connections 2023-10-18T11:42:23.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fae7406af60 0x7fae7406d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae900b0530 0x7fae901e2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae900cbd20 0x7fae901e2a80 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 --2- 172.21.15.125:0/2964455572 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae900afa20 0x7fae901eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 >> 172.21.15.125:0/2964455572 conn(0x7fae90069930 msgr2=0x7fae900ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:23.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 shutdown_connections 2023-10-18T11:42:23.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.247+0000 7fae96eec640 1 -- 172.21.15.125:0/2964455572 wait complete. 2023-10-18T11:42:23.260 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-18T11:42:21.457050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":539,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386984,"kb_used_data":19552,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754587736,"statfs":{"total":773094113280,"available":772697841664,"internally_reserved":0,"allocated":20021248,"data_stored":10713148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":376098360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.889926"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586840+0000","last_change":"2023-10-18T11:42:15.606818+0000","last_active":"2023-10-18T11:42:16.586840+0000","last_peered":"2023-10-18T11:42:16.586840+0000","last_clean":"2023-10-18T11:42:16.586840+0000","last_became_active":"2023-10-18T11:42:15.606386+0000","last_became_peered":"2023-10-18T11:42:15.606386+0000","last_unstale":"2023-10-18T11:42:16.586840+0000","last_undegraded":"2023-10-18T11:42:16.586840+0000","last_fullsized":"2023-10-18T11:42:16.586840+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:31.750269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.594719+0000","last_change":"2023-10-18T11:42:15.593452+0000","last_active":"2023-10-18T11:42:15.594719+0000","last_peered":"2023-10-18T11:42:15.594719+0000","last_clean":"2023-10-18T11:42:15.594719+0000","last_became_active":"2023-10-18T11:42:15.593117+0000","last_became_peered":"2023-10-18T11:42:15.593117+0000","last_unstale":"2023-10-18T11:42:15.594719+0000","last_undegraded":"2023-10-18T11:42:15.594719+0000","last_fullsized":"2023-10-18T11:42:15.594719+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:27:57.300118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586657+0000","last_change":"2023-10-18T11:42:15.599123+0000","last_active":"2023-10-18T11:42:16.586657+0000","last_peered":"2023-10-18T11:42:16.586657+0000","last_clean":"2023-10-18T11:42:16.586657+0000","last_became_active":"2023-10-18T11:42:15.598857+0000","last_became_peered":"2023-10-18T11:42:15.598857+0000","last_unstale":"2023-10-18T11:42:16.586657+0000","last_undegraded":"2023-10-18T11:42:16.586657+0000","last_fullsized":"2023-10-18T11:42:16.586657+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:03:34.127362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586870+0000","last_change":"2023-10-18T11:42:15.590534+0000","last_active":"2023-10-18T11:42:16.586870+0000","last_peered":"2023-10-18T11:42:16.586870+0000","last_clean":"2023-10-18T11:42:16.586870+0000","last_became_active":"2023-10-18T11:42:15.590273+0000","last_became_peered":"2023-10-18T11:42:15.590273+0000","last_unstale":"2023-10-18T11:42:16.586870+0000","last_undegraded":"2023-10-18T11:42:16.586870+0000","last_fullsized":"2023-10-18T11:42:16.586870+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:38.695269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.607237+0000","last_change":"2023-10-18T11:42:15.605135+0000","last_active":"2023-10-18T11:42:15.607237+0000","last_peered":"2023-10-18T11:42:15.607237+0000","last_clean":"2023-10-18T11:42:15.607237+0000","last_became_active":"2023-10-18T11:42:15.604835+0000","last_became_peered":"2023-10-18T11:42:15.604835+0000","last_unstale":"2023-10-18T11:42:15.607237+0000","last_undegraded":"2023-10-18T11:42:15.607237+0000","last_fullsized":"2023-10-18T11:42:15.607237+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:14.143639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601080+0000","last_change":"2023-10-18T11:42:15.599883+0000","last_active":"2023-10-18T11:42:15.601080+0000","last_peered":"2023-10-18T11:42:15.601080+0000","last_clean":"2023-10-18T11:42:15.601080+0000","last_became_active":"2023-10-18T11:42:15.599573+0000","last_became_peered":"2023-10-18T11:42:15.599573+0000","last_unstale":"2023-10-18T11:42:15.601080+0000","last_undegraded":"2023-10-18T11:42:15.601080+0000","last_fullsized":"2023-10-18T11:42:15.601080+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:34:03.760559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609673+0000","last_change":"2023-10-18T11:42:15.606477+0000","last_active":"2023-10-18T11:42:15.609673+0000","last_peered":"2023-10-18T11:42:15.609673+0000","last_clean":"2023-10-18T11:42:15.609673+0000","last_became_active":"2023-10-18T11:42:15.606179+0000","last_became_peered":"2023-10-18T11:42:15.606179+0000","last_unstale":"2023-10-18T11:42:15.609673+0000","last_undegraded":"2023-10-18T11:42:15.609673+0000","last_fullsized":"2023-10-18T11:42:15.609673+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:43.297692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609621+0000","last_change":"2023-10-18T11:42:15.606342+0000","last_active":"2023-10-18T11:42:15.609621+0000","last_peered":"2023-10-18T11:42:15.609621+0000","last_clean":"2023-10-18T11:42:15.609621+0000","last_became_active":"2023-10-18T11:42:15.605925+0000","last_became_peered":"2023-10-18T11:42:15.605925+0000","last_unstale":"2023-10-18T11:42:15.609621+0000","last_undegraded":"2023-10-18T11:42:15.609621+0000","last_fullsized":"2023-10-18T11:42:15.609621+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:33:43.204959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608851+0000","last_change":"2023-10-18T11:42:15.605555+0000","last_active":"2023-10-18T11:42:15.608851+0000","last_peered":"2023-10-18T11:42:15.608851+0000","last_clean":"2023-10-18T11:42:15.608851+0000","last_became_active":"2023-10-18T11:42:15.605309+0000","last_became_peered":"2023-10-18T11:42:15.605309+0000","last_unstale":"2023-10-18T11:42:15.608851+0000","last_undegraded":"2023-10-18T11:42:15.608851+0000","last_fullsized":"2023-10-18T11:42:15.608851+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:26:31.897953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609137+0000","last_change":"2023-10-18T11:42:15.603920+0000","last_active":"2023-10-18T11:42:15.609137+0000","last_peered":"2023-10-18T11:42:15.609137+0000","last_clean":"2023-10-18T11:42:15.609137+0000","last_became_active":"2023-10-18T11:42:15.603680+0000","last_became_peered":"2023-10-18T11:42:15.603680+0000","last_unstale":"2023-10-18T11:42:15.609137+0000","last_undegraded":"2023-10-18T11:42:15.609137+0000","last_fullsized":"2023-10-18T11:42:15.609137+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:07.197527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586100+0000","last_change":"2023-10-18T11:42:15.597015+0000","last_active":"2023-10-18T11:42:16.586100+0000","last_peered":"2023-10-18T11:42:16.586100+0000","last_clean":"2023-10-18T11:42:16.586100+0000","last_became_active":"2023-10-18T11:42:15.595860+0000","last_became_peered":"2023-10-18T11:42:15.595860+0000","last_unstale":"2023-10-18T11:42:16.586100+0000","last_undegraded":"2023-10-18T11:42:16.586100+0000","last_fullsized":"2023-10-18T11:42:16.586100+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:34:03.760559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604616+0000","last_change":"2023-10-18T11:42:15.600953+0000","last_active":"2023-10-18T11:42:15.604616+0000","last_peered":"2023-10-18T11:42:15.604616+0000","last_clean":"2023-10-18T11:42:15.604616+0000","last_became_active":"2023-10-18T11:42:15.600686+0000","last_became_peered":"2023-10-18T11:42:15.600686+0000","last_unstale":"2023-10-18T11:42:15.604616+0000","last_undegraded":"2023-10-18T11:42:15.604616+0000","last_fullsized":"2023-10-18T11:42:15.604616+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:50.573861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.602513+0000","last_change":"2023-10-18T11:42:15.595457+0000","last_active":"2023-10-18T11:42:15.602513+0000","last_peered":"2023-10-18T11:42:15.602513+0000","last_clean":"2023-10-18T11:42:15.602513+0000","last_became_active":"2023-10-18T11:42:15.595222+0000","last_became_peered":"2023-10-18T11:42:15.595222+0000","last_unstale":"2023-10-18T11:42:15.602513+0000","last_undegraded":"2023-10-18T11:42:15.602513+0000","last_fullsized":"2023-10-18T11:42:15.602513+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:46.867733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.600550+0000","last_change":"2023-10-18T11:42:15.596962+0000","last_active":"2023-10-18T11:42:15.600550+0000","last_peered":"2023-10-18T11:42:15.600550+0000","last_clean":"2023-10-18T11:42:15.600550+0000","last_became_active":"2023-10-18T11:42:15.596054+0000","last_became_peered":"2023-10-18T11:42:15.596054+0000","last_unstale":"2023-10-18T11:42:15.600550+0000","last_undegraded":"2023-10-18T11:42:15.600550+0000","last_fullsized":"2023-10-18T11:42:15.600550+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:41.953340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585654+0000","last_change":"2023-10-18T11:42:15.607300+0000","last_active":"2023-10-18T11:42:16.585654+0000","last_peered":"2023-10-18T11:42:16.585654+0000","last_clean":"2023-10-18T11:42:16.585654+0000","last_became_active":"2023-10-18T11:42:15.607018+0000","last_became_peered":"2023-10-18T11:42:15.607018+0000","last_unstale":"2023-10-18T11:42:16.585654+0000","last_undegraded":"2023-10-18T11:42:16.585654+0000","last_fullsized":"2023-10-18T11:42:16.585654+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:16:07.117805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586462+0000","last_change":"2023-10-18T11:42:15.588719+0000","last_active":"2023-10-18T11:42:16.586462+0000","last_peered":"2023-10-18T11:42:16.586462+0000","last_clean":"2023-10-18T11:42:16.586462+0000","last_became_active":"2023-10-18T11:42:15.588475+0000","last_became_peered":"2023-10-18T11:42:15.588475+0000","last_unstale":"2023-10-18T11:42:16.586462+0000","last_undegraded":"2023-10-18T11:42:16.586462+0000","last_fullsized":"2023-10-18T11:42:16.586462+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:05:27.374254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608637+0000","last_change":"2023-10-18T11:42:15.604311+0000","last_active":"2023-10-18T11:42:15.608637+0000","last_peered":"2023-10-18T11:42:15.608637+0000","last_clean":"2023-10-18T11:42:15.608637+0000","last_became_active":"2023-10-18T11:42:15.604071+0000","last_became_peered":"2023-10-18T11:42:15.604071+0000","last_unstale":"2023-10-18T11:42:15.608637+0000","last_undegraded":"2023-10-18T11:42:15.608637+0000","last_fullsized":"2023-10-18T11:42:15.608637+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:08.453473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.613794+0000","last_change":"2023-10-18T11:42:15.609772+0000","last_active":"2023-10-18T11:42:15.613794+0000","last_peered":"2023-10-18T11:42:15.613794+0000","last_clean":"2023-10-18T11:42:15.613794+0000","last_became_active":"2023-10-18T11:42:15.609451+0000","last_became_peered":"2023-10-18T11:42:15.609451+0000","last_unstale":"2023-10-18T11:42:15.613794+0000","last_undegraded":"2023-10-18T11:42:15.613794+0000","last_fullsized":"2023-10-18T11:42:15.613794+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:49:31.060237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586016+0000","last_change":"2023-10-18T11:42:15.611056+0000","last_active":"2023-10-18T11:42:16.586016+0000","last_peered":"2023-10-18T11:42:16.586016+0000","last_clean":"2023-10-18T11:42:16.586016+0000","last_became_active":"2023-10-18T11:42:15.610794+0000","last_became_peered":"2023-10-18T11:42:15.610794+0000","last_unstale":"2023-10-18T11:42:16.586016+0000","last_undegraded":"2023-10-18T11:42:16.586016+0000","last_fullsized":"2023-10-18T11:42:16.586016+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:27:57.300118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601059+0000","last_change":"2023-10-18T11:42:15.593279+0000","last_active":"2023-10-18T11:42:15.601059+0000","last_peered":"2023-10-18T11:42:15.601059+0000","last_clean":"2023-10-18T11:42:15.601059+0000","last_became_active":"2023-10-18T11:42:15.593005+0000","last_became_peered":"2023-10-18T11:42:15.593005+0000","last_unstale":"2023-10-18T11:42:15.601059+0000","last_undegraded":"2023-10-18T11:42:15.601059+0000","last_fullsized":"2023-10-18T11:42:15.601059+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:39.754401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.589319+0000","last_change":"2023-10-18T11:42:15.588164+0000","last_active":"2023-10-18T11:42:15.589319+0000","last_peered":"2023-10-18T11:42:15.589319+0000","last_clean":"2023-10-18T11:42:15.589319+0000","last_became_active":"2023-10-18T11:42:15.587874+0000","last_became_peered":"2023-10-18T11:42:15.587874+0000","last_unstale":"2023-10-18T11:42:15.589319+0000","last_undegraded":"2023-10-18T11:42:15.589319+0000","last_fullsized":"2023-10-18T11:42:15.589319+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:14.143639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.599979+0000","last_change":"2023-10-18T11:42:15.599123+0000","last_active":"2023-10-18T11:42:15.599979+0000","last_peered":"2023-10-18T11:42:15.599979+0000","last_clean":"2023-10-18T11:42:15.599979+0000","last_became_active":"2023-10-18T11:42:15.598845+0000","last_became_peered":"2023-10-18T11:42:15.598845+0000","last_unstale":"2023-10-18T11:42:15.599979+0000","last_undegraded":"2023-10-18T11:42:15.599979+0000","last_fullsized":"2023-10-18T11:42:15.599979+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:35:45.022105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601751+0000","last_change":"2023-10-18T11:42:15.597441+0000","last_active":"2023-10-18T11:42:15.601751+0000","last_peered":"2023-10-18T11:42:15.601751+0000","last_clean":"2023-10-18T11:42:15.601751+0000","last_became_active":"2023-10-18T11:42:15.597184+0000","last_became_peered":"2023-10-18T11:42:15.597184+0000","last_unstale":"2023-10-18T11:42:15.601751+0000","last_undegraded":"2023-10-18T11:42:15.601751+0000","last_fullsized":"2023-10-18T11:42:15.601751+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:17:24.248276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586662+0000","last_change":"2023-10-18T11:42:15.587854+0000","last_active":"2023-10-18T11:42:16.586662+0000","last_peered":"2023-10-18T11:42:16.586662+0000","last_clean":"2023-10-18T11:42:16.586662+0000","last_became_active":"2023-10-18T11:42:15.587595+0000","last_became_peered":"2023-10-18T11:42:15.587595+0000","last_unstale":"2023-10-18T11:42:16.586662+0000","last_undegraded":"2023-10-18T11:42:16.586662+0000","last_fullsized":"2023-10-18T11:42:16.586662+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:31.610905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.593503+0000","last_change":"2023-10-18T11:42:15.593015+0000","last_active":"2023-10-18T11:42:15.593503+0000","last_peered":"2023-10-18T11:42:15.593503+0000","last_clean":"2023-10-18T11:42:15.593503+0000","last_became_active":"2023-10-18T11:42:15.592720+0000","last_became_peered":"2023-10-18T11:42:15.592720+0000","last_unstale":"2023-10-18T11:42:15.593503+0000","last_undegraded":"2023-10-18T11:42:15.593503+0000","last_fullsized":"2023-10-18T11:42:15.593503+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:17:24.248276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.602487+0000","last_change":"2023-10-18T11:42:15.600658+0000","last_active":"2023-10-18T11:42:15.602487+0000","last_peered":"2023-10-18T11:42:15.602487+0000","last_clean":"2023-10-18T11:42:15.602487+0000","last_became_active":"2023-10-18T11:42:15.600378+0000","last_became_peered":"2023-10-18T11:42:15.600378+0000","last_unstale":"2023-10-18T11:42:15.602487+0000","last_undegraded":"2023-10-18T11:42:15.602487+0000","last_fullsized":"2023-10-18T11:42:15.602487+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:17.823820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604904+0000","last_change":"2023-10-18T11:42:15.601686+0000","last_active":"2023-10-18T11:42:15.604904+0000","last_peered":"2023-10-18T11:42:15.604904+0000","last_clean":"2023-10-18T11:42:15.604904+0000","last_became_active":"2023-10-18T11:42:15.601423+0000","last_became_peered":"2023-10-18T11:42:15.601423+0000","last_unstale":"2023-10-18T11:42:15.604904+0000","last_undegraded":"2023-10-18T11:42:15.604904+0000","last_fullsized":"2023-10-18T11:42:15.604904+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:21.407358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.607473+0000","last_change":"2023-10-18T11:42:15.603698+0000","last_active":"2023-10-18T11:42:15.607473+0000","last_peered":"2023-10-18T11:42:15.607473+0000","last_clean":"2023-10-18T11:42:15.607473+0000","last_became_active":"2023-10-18T11:42:15.603345+0000","last_became_peered":"2023-10-18T11:42:15.603345+0000","last_unstale":"2023-10-18T11:42:15.607473+0000","last_undegraded":"2023-10-18T11:42:15.607473+0000","last_fullsized":"2023-10-18T11:42:15.607473+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:41.953340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608514+0000","last_change":"2023-10-18T11:42:15.607669+0000","last_active":"2023-10-18T11:42:15.608514+0000","last_peered":"2023-10-18T11:42:15.608514+0000","last_clean":"2023-10-18T11:42:15.608514+0000","last_became_active":"2023-10-18T11:42:15.607431+0000","last_became_peered":"2023-10-18T11:42:15.607431+0000","last_unstale":"2023-10-18T11:42:15.608514+0000","last_undegraded":"2023-10-18T11:42:15.608514+0000","last_fullsized":"2023-10-18T11:42:15.608514+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:10:21.572281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605940+0000","last_change":"2023-10-18T11:42:15.598367+0000","last_active":"2023-10-18T11:42:15.605940+0000","last_peered":"2023-10-18T11:42:15.605940+0000","last_clean":"2023-10-18T11:42:15.605940+0000","last_became_active":"2023-10-18T11:42:15.598104+0000","last_became_peered":"2023-10-18T11:42:15.598104+0000","last_unstale":"2023-10-18T11:42:15.605940+0000","last_undegraded":"2023-10-18T11:42:15.605940+0000","last_fullsized":"2023-10-18T11:42:15.605940+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:00:17.431376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585593+0000","last_change":"2023-10-18T11:42:15.601677+0000","last_active":"2023-10-18T11:42:16.585593+0000","last_peered":"2023-10-18T11:42:16.585593+0000","last_clean":"2023-10-18T11:42:16.585593+0000","last_became_active":"2023-10-18T11:42:15.601193+0000","last_became_peered":"2023-10-18T11:42:15.601193+0000","last_unstale":"2023-10-18T11:42:16.585593+0000","last_undegraded":"2023-10-18T11:42:16.585593+0000","last_fullsized":"2023-10-18T11:42:16.585593+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:11.830738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605906+0000","last_change":"2023-10-18T11:42:15.597629+0000","last_active":"2023-10-18T11:42:15.605906+0000","last_peered":"2023-10-18T11:42:15.605906+0000","last_clean":"2023-10-18T11:42:15.605906+0000","last_became_active":"2023-10-18T11:42:15.597349+0000","last_became_peered":"2023-10-18T11:42:15.597349+0000","last_unstale":"2023-10-18T11:42:15.605906+0000","last_undegraded":"2023-10-18T11:42:15.605906+0000","last_fullsized":"2023-10-18T11:42:15.605906+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:05.997023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605694+0000","last_change":"2023-10-18T11:42:15.604291+0000","last_active":"2023-10-18T11:42:15.605694+0000","last_peered":"2023-10-18T11:42:15.605694+0000","last_clean":"2023-10-18T11:42:15.605694+0000","last_became_active":"2023-10-18T11:42:15.603821+0000","last_became_peered":"2023-10-18T11:42:15.603821+0000","last_unstale":"2023-10-18T11:42:15.605694+0000","last_undegraded":"2023-10-18T11:42:15.605694+0000","last_fullsized":"2023-10-18T11:42:15.605694+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:40:04.248063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"187'4","reported_seq":16,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586958+0000","last_change":"2023-10-18T11:42:14.582995+0000","last_active":"2023-10-18T11:42:16.586958+0000","last_peered":"2023-10-18T11:42:16.586958+0000","last_clean":"2023-10-18T11:42:16.586958+0000","last_became_active":"2023-10-18T11:42:14.582456+0000","last_became_peered":"2023-10-18T11:42:14.582456+0000","last_unstale":"2023-10-18T11:42:16.586958+0000","last_undegraded":"2023-10-18T11:42:16.586958+0000","last_fullsized":"2023-10-18T11:42:16.586958+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:23.106867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.600563+0000","last_change":"2023-10-18T11:42:15.598554+0000","last_active":"2023-10-18T11:42:15.600563+0000","last_peered":"2023-10-18T11:42:15.600563+0000","last_clean":"2023-10-18T11:42:15.600563+0000","last_became_active":"2023-10-18T11:42:15.598297+0000","last_became_peered":"2023-10-18T11:42:15.598297+0000","last_unstale":"2023-10-18T11:42:15.600563+0000","last_undegraded":"2023-10-18T11:42:15.600563+0000","last_fullsized":"2023-10-18T11:42:15.600563+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:03:11.601909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611376+0000","last_change":"2023-10-18T11:42:14.578398+0000","last_active":"2023-10-18T11:42:15.611376+0000","last_peered":"2023-10-18T11:42:15.611376+0000","last_clean":"2023-10-18T11:42:15.611376+0000","last_became_active":"2023-10-18T11:42:14.578133+0000","last_became_peered":"2023-10-18T11:42:14.578133+0000","last_unstale":"2023-10-18T11:42:15.611376+0000","last_undegraded":"2023-10-18T11:42:15.611376+0000","last_fullsized":"2023-10-18T11:42:15.611376+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:15.866757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.600530+0000","last_change":"2023-10-18T11:42:15.595762+0000","last_active":"2023-10-18T11:42:15.600530+0000","last_peered":"2023-10-18T11:42:15.600530+0000","last_clean":"2023-10-18T11:42:15.600530+0000","last_became_active":"2023-10-18T11:42:15.594219+0000","last_became_peered":"2023-10-18T11:42:15.594219+0000","last_unstale":"2023-10-18T11:42:15.600530+0000","last_undegraded":"2023-10-18T11:42:15.600530+0000","last_fullsized":"2023-10-18T11:42:15.600530+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:11.618673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586106+0000","last_change":"2023-10-18T11:42:14.579514+0000","last_active":"2023-10-18T11:42:16.586106+0000","last_peered":"2023-10-18T11:42:16.586106+0000","last_clean":"2023-10-18T11:42:16.586106+0000","last_became_active":"2023-10-18T11:42:14.579258+0000","last_became_peered":"2023-10-18T11:42:14.579258+0000","last_unstale":"2023-10-18T11:42:16.586106+0000","last_undegraded":"2023-10-18T11:42:16.586106+0000","last_fullsized":"2023-10-18T11:42:16.586106+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:06.050714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608101+0000","last_change":"2023-10-18T11:42:15.605505+0000","last_active":"2023-10-18T11:42:15.608101+0000","last_peered":"2023-10-18T11:42:15.608101+0000","last_clean":"2023-10-18T11:42:15.608101+0000","last_became_active":"2023-10-18T11:42:15.605229+0000","last_became_peered":"2023-10-18T11:42:15.605229+0000","last_unstale":"2023-10-18T11:42:15.608101+0000","last_undegraded":"2023-10-18T11:42:15.608101+0000","last_fullsized":"2023-10-18T11:42:15.608101+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:57.166807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601215+0000","last_change":"2023-10-18T11:42:14.578534+0000","last_active":"2023-10-18T11:42:15.601215+0000","last_peered":"2023-10-18T11:42:15.601215+0000","last_clean":"2023-10-18T11:42:15.601215+0000","last_became_active":"2023-10-18T11:42:14.577698+0000","last_became_peered":"2023-10-18T11:42:14.577698+0000","last_unstale":"2023-10-18T11:42:15.601215+0000","last_undegraded":"2023-10-18T11:42:15.601215+0000","last_fullsized":"2023-10-18T11:42:15.601215+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:14.281386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609586+0000","last_change":"2023-10-18T11:42:15.605839+0000","last_active":"2023-10-18T11:42:15.609586+0000","last_peered":"2023-10-18T11:42:15.609586+0000","last_clean":"2023-10-18T11:42:15.609586+0000","last_became_active":"2023-10-18T11:42:15.605597+0000","last_became_peered":"2023-10-18T11:42:15.605597+0000","last_unstale":"2023-10-18T11:42:15.609586+0000","last_undegraded":"2023-10-18T11:42:15.609586+0000","last_fullsized":"2023-10-18T11:42:15.609586+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:16.179404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611577+0000","last_change":"2023-10-18T11:42:14.582581+0000","last_active":"2023-10-18T11:42:15.611577+0000","last_peered":"2023-10-18T11:42:15.611577+0000","last_clean":"2023-10-18T11:42:15.611577+0000","last_became_active":"2023-10-18T11:42:14.582303+0000","last_became_peered":"2023-10-18T11:42:14.582303+0000","last_unstale":"2023-10-18T11:42:15.611577+0000","last_undegraded":"2023-10-18T11:42:15.611577+0000","last_fullsized":"2023-10-18T11:42:15.611577+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:45.649798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.602807+0000","last_change":"2023-10-18T11:42:15.600693+0000","last_active":"2023-10-18T11:42:15.602807+0000","last_peered":"2023-10-18T11:42:15.602807+0000","last_clean":"2023-10-18T11:42:15.602807+0000","last_became_active":"2023-10-18T11:42:15.600473+0000","last_became_peered":"2023-10-18T11:42:15.600473+0000","last_unstale":"2023-10-18T11:42:15.602807+0000","last_undegraded":"2023-10-18T11:42:15.602807+0000","last_fullsized":"2023-10-18T11:42:15.602807+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:51.612334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611258+0000","last_change":"2023-10-18T11:42:14.576075+0000","last_active":"2023-10-18T11:42:15.611258+0000","last_peered":"2023-10-18T11:42:15.611258+0000","last_clean":"2023-10-18T11:42:15.611258+0000","last_became_active":"2023-10-18T11:42:14.575803+0000","last_became_peered":"2023-10-18T11:42:14.575803+0000","last_unstale":"2023-10-18T11:42:15.611258+0000","last_undegraded":"2023-10-18T11:42:15.611258+0000","last_fullsized":"2023-10-18T11:42:15.611258+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:48.437435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605828+0000","last_change":"2023-10-18T11:42:15.605064+0000","last_active":"2023-10-18T11:42:15.605828+0000","last_peered":"2023-10-18T11:42:15.605828+0000","last_clean":"2023-10-18T11:42:15.605828+0000","last_became_active":"2023-10-18T11:42:15.604800+0000","last_became_peered":"2023-10-18T11:42:15.604800+0000","last_unstale":"2023-10-18T11:42:15.605828+0000","last_undegraded":"2023-10-18T11:42:15.605828+0000","last_fullsized":"2023-10-18T11:42:15.605828+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:52.794811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.586925+0000","last_change":"2023-10-18T11:42:14.581943+0000","last_active":"2023-10-18T11:42:15.586925+0000","last_peered":"2023-10-18T11:42:15.586925+0000","last_clean":"2023-10-18T11:42:15.586925+0000","last_became_active":"2023-10-18T11:42:14.581716+0000","last_became_peered":"2023-10-18T11:42:14.581716+0000","last_unstale":"2023-10-18T11:42:15.586925+0000","last_undegraded":"2023-10-18T11:42:15.586925+0000","last_fullsized":"2023-10-18T11:42:15.586925+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:57.384398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586925+0000","last_change":"2023-10-18T11:42:15.591620+0000","last_active":"2023-10-18T11:42:16.586925+0000","last_peered":"2023-10-18T11:42:16.586925+0000","last_clean":"2023-10-18T11:42:16.586925+0000","last_became_active":"2023-10-18T11:42:15.591255+0000","last_became_peered":"2023-10-18T11:42:15.591255+0000","last_unstale":"2023-10-18T11:42:16.586925+0000","last_undegraded":"2023-10-18T11:42:16.586925+0000","last_fullsized":"2023-10-18T11:42:16.586925+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:35.023607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.590169+0000","last_change":"2023-10-18T11:42:14.583849+0000","last_active":"2023-10-18T11:42:15.590169+0000","last_peered":"2023-10-18T11:42:15.590169+0000","last_clean":"2023-10-18T11:42:15.590169+0000","last_became_active":"2023-10-18T11:42:14.583604+0000","last_became_peered":"2023-10-18T11:42:14.583604+0000","last_unstale":"2023-10-18T11:42:15.590169+0000","last_undegraded":"2023-10-18T11:42:15.590169+0000","last_fullsized":"2023-10-18T11:42:15.590169+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:15:48.854696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605675+0000","last_change":"2023-10-18T11:42:15.604111+0000","last_active":"2023-10-18T11:42:15.605675+0000","last_peered":"2023-10-18T11:42:15.605675+0000","last_clean":"2023-10-18T11:42:15.605675+0000","last_became_active":"2023-10-18T11:42:15.603469+0000","last_became_peered":"2023-10-18T11:42:15.603469+0000","last_unstale":"2023-10-18T11:42:15.605675+0000","last_undegraded":"2023-10-18T11:42:15.605675+0000","last_fullsized":"2023-10-18T11:42:15.605675+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:35:55.974941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.582617+0000","last_change":"2023-10-18T11:42:14.577638+0000","last_active":"2023-10-18T11:42:15.582617+0000","last_peered":"2023-10-18T11:42:15.582617+0000","last_clean":"2023-10-18T11:42:15.582617+0000","last_became_active":"2023-10-18T11:42:14.577391+0000","last_became_peered":"2023-10-18T11:42:14.577391+0000","last_unstale":"2023-10-18T11:42:15.582617+0000","last_undegraded":"2023-10-18T11:42:15.582617+0000","last_fullsized":"2023-10-18T11:42:15.582617+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:50:55.288012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.610856+0000","last_change":"2023-10-18T11:42:15.608297+0000","last_active":"2023-10-18T11:42:15.610856+0000","last_peered":"2023-10-18T11:42:15.610856+0000","last_clean":"2023-10-18T11:42:15.610856+0000","last_became_active":"2023-10-18T11:42:15.608034+0000","last_became_peered":"2023-10-18T11:42:15.608034+0000","last_unstale":"2023-10-18T11:42:15.610856+0000","last_undegraded":"2023-10-18T11:42:15.610856+0000","last_fullsized":"2023-10-18T11:42:15.610856+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:14.572083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"187'3","reported_seq":15,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586740+0000","last_change":"2023-10-18T11:42:14.581885+0000","last_active":"2023-10-18T11:42:16.586740+0000","last_peered":"2023-10-18T11:42:16.586740+0000","last_clean":"2023-10-18T11:42:16.586740+0000","last_became_active":"2023-10-18T11:42:14.581610+0000","last_became_peered":"2023-10-18T11:42:14.581610+0000","last_unstale":"2023-10-18T11:42:16.586740+0000","last_undegraded":"2023-10-18T11:42:16.586740+0000","last_fullsized":"2023-10-18T11:42:16.586740+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:48:34.171530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.601289+0000","last_change":"2023-10-18T11:42:15.594687+0000","last_active":"2023-10-18T11:42:15.601289+0000","last_peered":"2023-10-18T11:42:15.601289+0000","last_clean":"2023-10-18T11:42:15.601289+0000","last_became_active":"2023-10-18T11:42:15.593908+0000","last_became_peered":"2023-10-18T11:42:15.593908+0000","last_unstale":"2023-10-18T11:42:15.601289+0000","last_undegraded":"2023-10-18T11:42:15.601289+0000","last_fullsized":"2023-10-18T11:42:15.601289+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:44.892491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"187'6","reported_seq":17,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.614491+0000","last_change":"2023-10-18T11:42:14.572036+0000","last_active":"2023-10-18T11:42:15.614491+0000","last_peered":"2023-10-18T11:42:15.614491+0000","last_clean":"2023-10-18T11:42:15.614491+0000","last_became_active":"2023-10-18T11:42:14.571772+0000","last_became_peered":"2023-10-18T11:42:14.571772+0000","last_unstale":"2023-10-18T11:42:15.614491+0000","last_undegraded":"2023-10-18T11:42:15.614491+0000","last_fullsized":"2023-10-18T11:42:15.614491+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:07.655075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604060+0000","last_change":"2023-10-18T11:42:15.596831+0000","last_active":"2023-10-18T11:42:15.604060+0000","last_peered":"2023-10-18T11:42:15.604060+0000","last_clean":"2023-10-18T11:42:15.604060+0000","last_became_active":"2023-10-18T11:42:15.596580+0000","last_became_peered":"2023-10-18T11:42:15.596580+0000","last_unstale":"2023-10-18T11:42:15.604060+0000","last_undegraded":"2023-10-18T11:42:15.604060+0000","last_fullsized":"2023-10-18T11:42:15.604060+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:18:38.982049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.605223+0000","last_change":"2023-10-18T11:42:14.577480+0000","last_active":"2023-10-18T11:42:15.605223+0000","last_peered":"2023-10-18T11:42:15.605223+0000","last_clean":"2023-10-18T11:42:15.605223+0000","last_became_active":"2023-10-18T11:42:14.577183+0000","last_became_peered":"2023-10-18T11:42:14.577183+0000","last_unstale":"2023-10-18T11:42:15.605223+0000","last_undegraded":"2023-10-18T11:42:15.605223+0000","last_fullsized":"2023-10-18T11:42:15.605223+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:05.696631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611472+0000","last_change":"2023-10-18T11:42:15.609248+0000","last_active":"2023-10-18T11:42:15.611472+0000","last_peered":"2023-10-18T11:42:15.611472+0000","last_clean":"2023-10-18T11:42:15.611472+0000","last_became_active":"2023-10-18T11:42:15.604621+0000","last_became_peered":"2023-10-18T11:42:15.604621+0000","last_unstale":"2023-10-18T11:42:15.611472+0000","last_undegraded":"2023-10-18T11:42:15.611472+0000","last_fullsized":"2023-10-18T11:42:15.611472+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:35.614614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.583043+0000","last_change":"2023-10-18T11:42:14.574522+0000","last_active":"2023-10-18T11:42:15.583043+0000","last_peered":"2023-10-18T11:42:15.583043+0000","last_clean":"2023-10-18T11:42:15.583043+0000","last_became_active":"2023-10-18T11:42:14.574248+0000","last_became_peered":"2023-10-18T11:42:14.574248+0000","last_unstale":"2023-10-18T11:42:15.583043+0000","last_undegraded":"2023-10-18T11:42:15.583043+0000","last_fullsized":"2023-10-18T11:42:15.583043+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:10.613194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585921+0000","last_change":"2023-10-18T11:42:15.594310+0000","last_active":"2023-10-18T11:42:16.585921+0000","last_peered":"2023-10-18T11:42:16.585921+0000","last_clean":"2023-10-18T11:42:16.585921+0000","last_became_active":"2023-10-18T11:42:15.594051+0000","last_became_peered":"2023-10-18T11:42:15.594051+0000","last_unstale":"2023-10-18T11:42:16.585921+0000","last_undegraded":"2023-10-18T11:42:16.585921+0000","last_fullsized":"2023-10-18T11:42:16.585921+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:36.610510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585693+0000","last_change":"2023-10-18T11:42:14.578913+0000","last_active":"2023-10-18T11:42:16.585693+0000","last_peered":"2023-10-18T11:42:16.585693+0000","last_clean":"2023-10-18T11:42:16.585693+0000","last_became_active":"2023-10-18T11:42:14.578638+0000","last_became_peered":"2023-10-18T11:42:14.578638+0000","last_unstale":"2023-10-18T11:42:16.585693+0000","last_undegraded":"2023-10-18T11:42:16.585693+0000","last_fullsized":"2023-10-18T11:42:16.585693+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:32.729146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.604428+0000","last_change":"2023-10-18T11:42:15.602323+0000","last_active":"2023-10-18T11:42:15.604428+0000","last_peered":"2023-10-18T11:42:15.604428+0000","last_clean":"2023-10-18T11:42:15.604428+0000","last_became_active":"2023-10-18T11:42:15.602075+0000","last_became_peered":"2023-10-18T11:42:15.602075+0000","last_unstale":"2023-10-18T11:42:15.604428+0000","last_undegraded":"2023-10-18T11:42:15.604428+0000","last_fullsized":"2023-10-18T11:42:15.604428+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:55:43.654543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"187'8","reported_seq":19,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.610476+0000","last_change":"2023-10-18T11:42:14.578512+0000","last_active":"2023-10-18T11:42:15.610476+0000","last_peered":"2023-10-18T11:42:15.610476+0000","last_clean":"2023-10-18T11:42:15.610476+0000","last_became_active":"2023-10-18T11:42:14.578237+0000","last_became_peered":"2023-10-18T11:42:14.578237+0000","last_unstale":"2023-10-18T11:42:15.610476+0000","last_undegraded":"2023-10-18T11:42:15.610476+0000","last_fullsized":"2023-10-18T11:42:15.610476+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:36.860904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608880+0000","last_change":"2023-10-18T11:42:15.605677+0000","last_active":"2023-10-18T11:42:15.608880+0000","last_peered":"2023-10-18T11:42:15.608880+0000","last_clean":"2023-10-18T11:42:15.608880+0000","last_became_active":"2023-10-18T11:42:15.602902+0000","last_became_peered":"2023-10-18T11:42:15.602902+0000","last_unstale":"2023-10-18T11:42:15.608880+0000","last_undegraded":"2023-10-18T11:42:15.608880+0000","last_fullsized":"2023-10-18T11:42:15.608880+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:46:22.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.614092+0000","last_change":"2023-10-18T11:42:14.582338+0000","last_active":"2023-10-18T11:42:15.614092+0000","last_peered":"2023-10-18T11:42:15.614092+0000","last_clean":"2023-10-18T11:42:15.614092+0000","last_became_active":"2023-10-18T11:42:14.582001+0000","last_became_peered":"2023-10-18T11:42:14.582001+0000","last_unstale":"2023-10-18T11:42:15.614092+0000","last_undegraded":"2023-10-18T11:42:15.614092+0000","last_fullsized":"2023-10-18T11:42:15.614092+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:38.500447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608663+0000","last_change":"2023-10-18T11:42:15.605780+0000","last_active":"2023-10-18T11:42:15.608663+0000","last_peered":"2023-10-18T11:42:15.608663+0000","last_clean":"2023-10-18T11:42:15.608663+0000","last_became_active":"2023-10-18T11:42:15.600458+0000","last_became_peered":"2023-10-18T11:42:15.600458+0000","last_unstale":"2023-10-18T11:42:15.608663+0000","last_undegraded":"2023-10-18T11:42:15.608663+0000","last_fullsized":"2023-10-18T11:42:15.608663+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:57.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609089+0000","last_change":"2023-10-18T11:42:14.582657+0000","last_active":"2023-10-18T11:42:15.609089+0000","last_peered":"2023-10-18T11:42:15.609089+0000","last_clean":"2023-10-18T11:42:15.609089+0000","last_became_active":"2023-10-18T11:42:14.582386+0000","last_became_peered":"2023-10-18T11:42:14.582386+0000","last_unstale":"2023-10-18T11:42:15.609089+0000","last_undegraded":"2023-10-18T11:42:15.609089+0000","last_fullsized":"2023-10-18T11:42:15.609089+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:32.729146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585607+0000","last_change":"2023-10-18T11:42:15.596605+0000","last_active":"2023-10-18T11:42:16.585607+0000","last_peered":"2023-10-18T11:42:16.585607+0000","last_clean":"2023-10-18T11:42:16.585607+0000","last_became_active":"2023-10-18T11:42:15.596305+0000","last_became_peered":"2023-10-18T11:42:15.596305+0000","last_unstale":"2023-10-18T11:42:16.585607+0000","last_undegraded":"2023-10-18T11:42:16.585607+0000","last_fullsized":"2023-10-18T11:42:16.585607+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:32.264398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609703+0000","last_change":"2023-10-18T11:42:14.582207+0000","last_active":"2023-10-18T11:42:15.609703+0000","last_peered":"2023-10-18T11:42:15.609703+0000","last_clean":"2023-10-18T11:42:15.609703+0000","last_became_active":"2023-10-18T11:42:14.581949+0000","last_became_peered":"2023-10-18T11:42:14.581949+0000","last_unstale":"2023-10-18T11:42:15.609703+0000","last_undegraded":"2023-10-18T11:42:15.609703+0000","last_fullsized":"2023-10-18T11:42:15.609703+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:29.706251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585805+0000","last_change":"2023-10-18T11:42:15.596528+0000","last_active":"2023-10-18T11:42:16.585805+0000","last_peered":"2023-10-18T11:42:16.585805+0000","last_clean":"2023-10-18T11:42:16.585805+0000","last_became_active":"2023-10-18T11:42:15.595978+0000","last_became_peered":"2023-10-18T11:42:15.595978+0000","last_unstale":"2023-10-18T11:42:16.585805+0000","last_undegraded":"2023-10-18T11:42:16.585805+0000","last_fullsized":"2023-10-18T11:42:16.585805+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:05.109362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.594939+0000","last_change":"2023-10-18T11:42:14.583437+0000","last_active":"2023-10-18T11:42:15.594939+0000","last_peered":"2023-10-18T11:42:15.594939+0000","last_clean":"2023-10-18T11:42:15.594939+0000","last_became_active":"2023-10-18T11:42:14.583103+0000","last_became_peered":"2023-10-18T11:42:14.583103+0000","last_unstale":"2023-10-18T11:42:15.594939+0000","last_undegraded":"2023-10-18T11:42:15.594939+0000","last_fullsized":"2023-10-18T11:42:15.594939+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:16:21.073740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.607493+0000","last_change":"2023-10-18T11:42:15.603852+0000","last_active":"2023-10-18T11:42:15.607493+0000","last_peered":"2023-10-18T11:42:15.607493+0000","last_clean":"2023-10-18T11:42:15.607493+0000","last_became_active":"2023-10-18T11:42:15.602918+0000","last_became_peered":"2023-10-18T11:42:15.602918+0000","last_unstale":"2023-10-18T11:42:15.607493+0000","last_undegraded":"2023-10-18T11:42:15.607493+0000","last_fullsized":"2023-10-18T11:42:15.607493+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:19.121794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.606039+0000","last_change":"2023-10-18T11:42:14.578859+0000","last_active":"2023-10-18T11:42:15.606039+0000","last_peered":"2023-10-18T11:42:15.606039+0000","last_clean":"2023-10-18T11:42:15.606039+0000","last_became_active":"2023-10-18T11:42:14.577551+0000","last_became_peered":"2023-10-18T11:42:14.577551+0000","last_unstale":"2023-10-18T11:42:15.606039+0000","last_undegraded":"2023-10-18T11:42:15.606039+0000","last_fullsized":"2023-10-18T11:42:15.606039+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:31.497909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.612141+0000","last_change":"2023-10-18T11:42:15.611433+0000","last_active":"2023-10-18T11:42:15.612141+0000","last_peered":"2023-10-18T11:42:15.612141+0000","last_clean":"2023-10-18T11:42:15.612141+0000","last_became_active":"2023-10-18T11:42:15.611114+0000","last_became_peered":"2023-10-18T11:42:15.611114+0000","last_unstale":"2023-10-18T11:42:15.612141+0000","last_undegraded":"2023-10-18T11:42:15.612141+0000","last_fullsized":"2023-10-18T11:42:15.612141+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:03:08.660554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585535+0000","last_change":"2023-10-18T11:42:14.574506+0000","last_active":"2023-10-18T11:42:16.585535+0000","last_peered":"2023-10-18T11:42:16.585535+0000","last_clean":"2023-10-18T11:42:16.585535+0000","last_became_active":"2023-10-18T11:42:14.574184+0000","last_became_peered":"2023-10-18T11:42:14.574184+0000","last_unstale":"2023-10-18T11:42:16.585535+0000","last_undegraded":"2023-10-18T11:42:16.585535+0000","last_fullsized":"2023-10-18T11:42:16.585535+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:16:21.073740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.612510+0000","last_change":"2023-10-18T11:42:15.611523+0000","last_active":"2023-10-18T11:42:15.612510+0000","last_peered":"2023-10-18T11:42:15.612510+0000","last_clean":"2023-10-18T11:42:15.612510+0000","last_became_active":"2023-10-18T11:42:15.611179+0000","last_became_peered":"2023-10-18T11:42:15.611179+0000","last_unstale":"2023-10-18T11:42:15.612510+0000","last_undegraded":"2023-10-18T11:42:15.612510+0000","last_fullsized":"2023-10-18T11:42:15.612510+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:19.121794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.587545+0000","last_change":"2023-10-18T11:42:14.575953+0000","last_active":"2023-10-18T11:42:15.587545+0000","last_peered":"2023-10-18T11:42:15.587545+0000","last_clean":"2023-10-18T11:42:15.587545+0000","last_became_active":"2023-10-18T11:42:14.575689+0000","last_became_peered":"2023-10-18T11:42:14.575689+0000","last_unstale":"2023-10-18T11:42:15.587545+0000","last_undegraded":"2023-10-18T11:42:15.587545+0000","last_fullsized":"2023-10-18T11:42:15.587545+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:12:55.425254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.594046+0000","last_change":"2023-10-18T11:42:15.592858+0000","last_active":"2023-10-18T11:42:15.594046+0000","last_peered":"2023-10-18T11:42:15.594046+0000","last_clean":"2023-10-18T11:42:15.594046+0000","last_became_active":"2023-10-18T11:42:15.592595+0000","last_became_peered":"2023-10-18T11:42:15.592595+0000","last_unstale":"2023-10-18T11:42:15.594046+0000","last_undegraded":"2023-10-18T11:42:15.594046+0000","last_fullsized":"2023-10-18T11:42:15.594046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:36.610510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.591191+0000","last_change":"2023-10-18T11:42:14.581265+0000","last_active":"2023-10-18T11:42:15.591191+0000","last_peered":"2023-10-18T11:42:15.591191+0000","last_clean":"2023-10-18T11:42:15.591191+0000","last_became_active":"2023-10-18T11:42:14.580990+0000","last_became_peered":"2023-10-18T11:42:14.580990+0000","last_unstale":"2023-10-18T11:42:15.591191+0000","last_undegraded":"2023-10-18T11:42:15.591191+0000","last_fullsized":"2023-10-18T11:42:15.591191+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:24.806953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585607+0000","last_change":"2023-10-18T11:42:15.595645+0000","last_active":"2023-10-18T11:42:16.585607+0000","last_peered":"2023-10-18T11:42:16.585607+0000","last_clean":"2023-10-18T11:42:16.585607+0000","last_became_active":"2023-10-18T11:42:15.595343+0000","last_became_peered":"2023-10-18T11:42:15.595343+0000","last_unstale":"2023-10-18T11:42:16.585607+0000","last_undegraded":"2023-10-18T11:42:16.585607+0000","last_fullsized":"2023-10-18T11:42:16.585607+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:11.601195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"187'2","reported_seq":13,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.606353+0000","last_change":"2023-10-18T11:42:14.579758+0000","last_active":"2023-10-18T11:42:15.606353+0000","last_peered":"2023-10-18T11:42:15.606353+0000","last_clean":"2023-10-18T11:42:15.606353+0000","last_became_active":"2023-10-18T11:42:14.579505+0000","last_became_peered":"2023-10-18T11:42:14.579505+0000","last_unstale":"2023-10-18T11:42:15.606353+0000","last_undegraded":"2023-10-18T11:42:15.606353+0000","last_fullsized":"2023-10-18T11:42:15.606353+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:09:56.691445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.611677+0000","last_change":"2023-10-18T11:42:15.608870+0000","last_active":"2023-10-18T11:42:15.611677+0000","last_peered":"2023-10-18T11:42:15.611677+0000","last_clean":"2023-10-18T11:42:15.611677+0000","last_became_active":"2023-10-18T11:42:15.605790+0000","last_became_peered":"2023-10-18T11:42:15.605790+0000","last_unstale":"2023-10-18T11:42:15.611677+0000","last_undegraded":"2023-10-18T11:42:15.611677+0000","last_fullsized":"2023-10-18T11:42:15.611677+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:32.258275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.583007+0000","last_change":"2023-10-18T11:42:14.576812+0000","last_active":"2023-10-18T11:42:15.583007+0000","last_peered":"2023-10-18T11:42:15.583007+0000","last_clean":"2023-10-18T11:42:15.583007+0000","last_became_active":"2023-10-18T11:42:14.576377+0000","last_became_peered":"2023-10-18T11:42:14.576377+0000","last_unstale":"2023-10-18T11:42:15.583007+0000","last_undegraded":"2023-10-18T11:42:15.583007+0000","last_fullsized":"2023-10-18T11:42:15.583007+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:15.866757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.602132+0000","last_change":"2023-10-18T11:42:15.599807+0000","last_active":"2023-10-18T11:42:15.602132+0000","last_peered":"2023-10-18T11:42:15.602132+0000","last_clean":"2023-10-18T11:42:15.602132+0000","last_became_active":"2023-10-18T11:42:15.599536+0000","last_became_peered":"2023-10-18T11:42:15.599536+0000","last_unstale":"2023-10-18T11:42:15.602132+0000","last_undegraded":"2023-10-18T11:42:15.602132+0000","last_fullsized":"2023-10-18T11:42:15.602132+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:38.798007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.582769+0000","last_change":"2023-10-18T11:42:14.581450+0000","last_active":"2023-10-18T11:42:15.582769+0000","last_peered":"2023-10-18T11:42:15.582769+0000","last_clean":"2023-10-18T11:42:15.582769+0000","last_became_active":"2023-10-18T11:42:14.581174+0000","last_became_peered":"2023-10-18T11:42:14.581174+0000","last_unstale":"2023-10-18T11:42:15.582769+0000","last_undegraded":"2023-10-18T11:42:15.582769+0000","last_fullsized":"2023-10-18T11:42:15.582769+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:44:02.797286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.609730+0000","last_change":"2023-10-18T11:42:15.606580+0000","last_active":"2023-10-18T11:42:15.609730+0000","last_peered":"2023-10-18T11:42:15.609730+0000","last_clean":"2023-10-18T11:42:15.609730+0000","last_became_active":"2023-10-18T11:42:15.604639+0000","last_became_peered":"2023-10-18T11:42:15.604639+0000","last_unstale":"2023-10-18T11:42:15.609730+0000","last_undegraded":"2023-10-18T11:42:15.609730+0000","last_fullsized":"2023-10-18T11:42:15.609730+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:03.790016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.587063+0000","last_change":"2023-10-18T11:42:14.581315+0000","last_active":"2023-10-18T11:42:15.587063+0000","last_peered":"2023-10-18T11:42:15.587063+0000","last_clean":"2023-10-18T11:42:15.587063+0000","last_became_active":"2023-10-18T11:42:14.581077+0000","last_became_peered":"2023-10-18T11:42:14.581077+0000","last_unstale":"2023-10-18T11:42:15.587063+0000","last_undegraded":"2023-10-18T11:42:15.587063+0000","last_fullsized":"2023-10-18T11:42:15.587063+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:46:28.198283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586960+0000","last_change":"2023-10-18T11:42:15.596716+0000","last_active":"2023-10-18T11:42:16.586960+0000","last_peered":"2023-10-18T11:42:16.586960+0000","last_clean":"2023-10-18T11:42:16.586960+0000","last_became_active":"2023-10-18T11:42:15.596467+0000","last_became_peered":"2023-10-18T11:42:15.596467+0000","last_unstale":"2023-10-18T11:42:16.586960+0000","last_undegraded":"2023-10-18T11:42:16.586960+0000","last_fullsized":"2023-10-18T11:42:16.586960+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:09.454600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.586526+0000","last_change":"2023-10-18T11:42:14.582780+0000","last_active":"2023-10-18T11:42:16.586526+0000","last_peered":"2023-10-18T11:42:16.586526+0000","last_clean":"2023-10-18T11:42:16.586526+0000","last_became_active":"2023-10-18T11:42:14.582378+0000","last_became_peered":"2023-10-18T11:42:14.582378+0000","last_unstale":"2023-10-18T11:42:16.586526+0000","last_undegraded":"2023-10-18T11:42:16.586526+0000","last_fullsized":"2023-10-18T11:42:16.586526+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:55:49.492744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.606761+0000","last_change":"2023-10-18T11:42:15.604831+0000","last_active":"2023-10-18T11:42:15.606761+0000","last_peered":"2023-10-18T11:42:15.606761+0000","last_clean":"2023-10-18T11:42:15.606761+0000","last_became_active":"2023-10-18T11:42:15.604581+0000","last_became_peered":"2023-10-18T11:42:15.604581+0000","last_unstale":"2023-10-18T11:42:15.606761+0000","last_undegraded":"2023-10-18T11:42:15.606761+0000","last_fullsized":"2023-10-18T11:42:15.606761+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:43:56.873122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.582874+0000","last_change":"2023-10-18T11:42:14.574027+0000","last_active":"2023-10-18T11:42:15.582874+0000","last_peered":"2023-10-18T11:42:15.582874+0000","last_clean":"2023-10-18T11:42:15.582874+0000","last_became_active":"2023-10-18T11:42:14.573748+0000","last_became_peered":"2023-10-18T11:42:14.573748+0000","last_unstale":"2023-10-18T11:42:15.582874+0000","last_undegraded":"2023-10-18T11:42:15.582874+0000","last_fullsized":"2023-10-18T11:42:15.582874+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:09:56.691445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.592646+0000","last_change":"2023-10-18T11:42:15.589028+0000","last_active":"2023-10-18T11:42:15.592646+0000","last_peered":"2023-10-18T11:42:15.592646+0000","last_clean":"2023-10-18T11:42:15.592646+0000","last_became_active":"2023-10-18T11:42:15.588776+0000","last_became_peered":"2023-10-18T11:42:15.588776+0000","last_unstale":"2023-10-18T11:42:15.592646+0000","last_undegraded":"2023-10-18T11:42:15.592646+0000","last_fullsized":"2023-10-18T11:42:15.592646+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:15.252200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-18T11:42:16.585763+0000","last_change":"2023-10-18T11:42:14.578983+0000","last_active":"2023-10-18T11:42:16.585763+0000","last_peered":"2023-10-18T11:42:16.585763+0000","last_clean":"2023-10-18T11:42:16.585763+0000","last_became_active":"2023-10-18T11:42:14.578147+0000","last_became_peered":"2023-10-18T11:42:14.578147+0000","last_unstale":"2023-10-18T11:42:16.585763+0000","last_undegraded":"2023-10-18T11:42:16.585763+0000","last_fullsized":"2023-10-18T11:42:16.585763+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:22.549891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.598583+0000","last_change":"2023-10-18T11:42:15.592580+0000","last_active":"2023-10-18T11:42:15.598583+0000","last_peered":"2023-10-18T11:42:15.598583+0000","last_clean":"2023-10-18T11:42:15.598583+0000","last_became_active":"2023-10-18T11:42:15.592324+0000","last_became_peered":"2023-10-18T11:42:15.592324+0000","last_unstale":"2023-10-18T11:42:15.598583+0000","last_undegraded":"2023-10-18T11:42:15.598583+0000","last_fullsized":"2023-10-18T11:42:15.598583+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:23.509994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.587225+0000","last_change":"2023-10-18T11:42:14.580455+0000","last_active":"2023-10-18T11:42:15.587225+0000","last_peered":"2023-10-18T11:42:15.587225+0000","last_clean":"2023-10-18T11:42:15.587225+0000","last_became_active":"2023-10-18T11:42:14.580204+0000","last_became_peered":"2023-10-18T11:42:14.580204+0000","last_unstale":"2023-10-18T11:42:15.587225+0000","last_undegraded":"2023-10-18T11:42:15.587225+0000","last_fullsized":"2023-10-18T11:42:15.587225+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:14.246721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.608526+0000","last_change":"2023-10-18T11:42:15.606583+0000","last_active":"2023-10-18T11:42:15.608526+0000","last_peered":"2023-10-18T11:42:15.608526+0000","last_clean":"2023-10-18T11:42:15.608526+0000","last_became_active":"2023-10-18T11:42:15.606318+0000","last_became_peered":"2023-10-18T11:42:15.606318+0000","last_unstale":"2023-10-18T11:42:15.608526+0000","last_undegraded":"2023-10-18T11:42:15.608526+0000","last_fullsized":"2023-10-18T11:42:15.608526+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:14.559314+0000","last_clean_scrub_stamp":"2023-10-18T11:42:14.559314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:05.109362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-18T11:42:15.582828+0000","last_change":"2023-10-18T11:42:14.578933+0000","last_active":"2023-10-18T11:42:15.582828+0000","last_peered":"2023-10-18T11:42:15.582828+0000","last_clean":"2023-10-18T11:42:15.582828+0000","last_became_active":"2023-10-18T11:42:14.578128+0000","last_became_peered":"2023-10-18T11:42:14.578128+0000","last_unstale":"2023-10-18T11:42:15.582828+0000","last_undegraded":"2023-10-18T11:42:15.582828+0000","last_fullsized":"2023-10-18T11:42:15.582828+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:42:13.553835+0000","last_clean_scrub_stamp":"2023-10-18T11:42:13.553835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:44:56.908102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"176'302","reported_seq":999,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-10-18T11:42:14.576038+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:42:14.576038+0000","last_peered":"2023-10-18T11:42:14.576038+0000","last_clean":"2023-10-18T11:42:14.576038+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:42:14.576038+0000","last_undegraded":"2023-10-18T11:42:14.576038+0000","last_fullsized":"2023-10-18T11:42:14.576038+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1730,"num_read_kb":3587,"num_write":495,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575775,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52104,"kb_used_data":2868,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319736,"statfs":{"total":96636764160,"available":96583409664,"internally_reserved":0,"allocated":2936832,"data_stored":1779655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.318},"max":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.378,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.78600000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64900000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.42599999999999999,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.85499999999999998,"15min":0.86899999999999999},"last":0.61299999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.74199999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.47699999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.28599999999999998},"max":{"1min":0.65900000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.30499999999999999},"max":{"1min":0.69499999999999995,"5min":0.749,"15min":0.76900000000000002},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29899999999999999},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.86899999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71899999999999997,"5min":0.83099999999999996,"15min":0.84499999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.83499999999999996},"last":0.629}]}]},{"osd":6,"up_from":13,"seq":55834575775,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2342912,"data_stored":1189287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"last":0.44}]},{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.36599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.61299999999999999},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41799999999999998},"max":{"1min":0.72399999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.35999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.437,"5min":0.436,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47799999999999998,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.41099999999999998,"5min":0.375,"15min":0.35299999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48799999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.85899999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.31900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575775,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59860,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311980,"statfs":{"total":96636764160,"available":96575467520,"internally_reserved":0,"allocated":2363392,"data_stored":1196059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28799999999999998},"max":{"1min":0.74099999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.877},"last":0.56399999999999995}]},{"osd":1,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36799999999999999},"max":{"1min":0.76600000000000001,"5min":0.84399999999999997,"15min":0.84499999999999997},"last":0.624}]},{"osd":2,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.90700000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.76900000000000002,"5min":0.872,"15min":0.872},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.496},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.45600000000000002}]},{"osd":6,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.43099999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.372},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.91300000000000003},"last":0.47499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41824,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330016,"statfs":{"total":96636764160,"available":96593936384,"internally_reserved":0,"allocated":2371584,"data_stored":1196699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.622,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.27800000000000002},"max":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.872,"15min":0.872},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"max":{"1min":0.80700000000000005,"5min":0.82999999999999996,"15min":0.86399999999999999},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.85799999999999998},"last":0.751}]},{"osd":3,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.626},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82099999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.504,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.307},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.5},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.501},"min":{"1min":0.318,"5min":0.313,"15min":0.313},"max":{"1min":0.73799999999999999,"5min":0.76700000000000002,"15min":0.82699999999999996},"last":0.47299999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":0.85899999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54900000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75800000000000001,"5min":0.78300000000000003,"15min":0.79500000000000004},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Oct 18 11:42:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.52900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.89900000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.82499999999999996,"15min":0.91100000000000003},"last":0.436}]}]},{"osd":3,"up_from":13,"seq":55834575775,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2363392,"data_stored":1191503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.218},"max":{"1min":0.48599999999999999,"5min":0.65300000000000002,"15min":0.71799999999999997},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.249},"max":{"1min":0.497,"5min":0.57099999999999995,"15min":0.66000000000000003},"last":0.22800000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.74099999999999999},"last":0.251},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54500000000000004,"5min":0.67200000000000004,"15min":0.82999999999999996},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59799999999999998,"5min":0.65300000000000002,"15min":0.66300000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26300000000000001},"max":{"1min":0.59799999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.30499999999999999}]},{"osd":4,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.92700000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.97599999999999998},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.006},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.97699999999999998,"15min":0.996},"last":0.35999999999999999}]},{"osd":6,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.92600000000000005,"15min":1.0549999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.083},"last":0.44900000000000001}]},{"osd":7,"last update":"Wed Oct 18 11:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.95499999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96699999999999997,"5min":0.999,"15min":1.0509999999999999},"last":0.35099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575775,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59840,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312000,"statfs":{"total":96636764160,"available":96575488000,"internally_reserved":0,"allocated":2342912,"data_stored":1189287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40500000000000003,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.311}]},{"osd":1,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.249},"max":{"1min":0.56499999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.68899999999999995,"15min":0.72699999999999998},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42199999999999999,"15min":0.42599999999999999},"min":{"1min":0.26900000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.75900000000000001},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.58199999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84099999999999997,"5min":0.93300000000000005,"15min":1.0249999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":1.046},"last":0.48599999999999999}]},{"osd":5,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.85199999999999998,"15min":0.97999999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99099999999999999},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.41699999999999998},"max":{"1min":0.80000000000000004,"5min":0.91800000000000004,"15min":0.996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.40600000000000003},"max":{"1min":0.81499999999999995,"5min":0.89700000000000002,"15min":1.028},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82599999999999996,"5min":0.98699999999999999,"15min":1.014},"last":0.503},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.41999999999999998},"max":{"1min":0.79400000000000004,"5min":0.89800000000000002,"15min":1.1299999999999999},"last":0.42999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575775,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":2355200,"data_stored":1190899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22900000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.71299999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.27600000000000002,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.376}]},{"osd":2,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.40500000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44400000000000001,"15min":0.433},"min":{"1min":0.315,"5min":0.307,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.63900000000000001,"15min":0.64000000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.63500000000000001,"15min":0.72799999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28000000000000003},"max":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66400000000000003},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66500000000000004,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81100000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88400000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.623}]},{"osd":7,"last update":"Wed Oct 18 11:41:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.872,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.65400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575774,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38220,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333620,"statfs":{"total":96636764160,"available":96597626880,"internally_reserved":0,"allocated":2945024,"data_stored":1779759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38200000000000001,"15min":0.39300000000000002},"min":{"1min":0.246,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.66500000000000004},"last":0.312},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.72599999999999998},"last":0.32900000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.28299999999999997,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.66000000000000003,"15min":0.67600000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41999999999999998,"15min":0.43099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.66400000000000003},"last":0.374}]},{"osd":3,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.69899999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28599999999999998},"max":{"1min":0.56399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.41599999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.40400000000000003},"max":{"1min":0.86399999999999999,"5min":0.90000000000000002,"15min":0.92200000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.41499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.94499999999999995},"last":0.51700000000000002}]},{"osd":5,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.48999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92800000000000005,"5min":0.92900000000000005,"15min":0.99299999999999999},"last":0.53700000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.97899999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":0.878,"15min":0.95899999999999996},"last":0.497}]},{"osd":7,"last update":"Wed Oct 18 11:41:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.43099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.93999999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:42:23.266 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T11:42:23.266 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T11:42:23.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.408+0000 7feb8c125640 1 -- 172.21.15.125:0/2610268247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb840ceb40 msgr2=0x7feb840b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.408+0000 7feb8c125640 1 --2- 172.21.15.125:0/2610268247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb840ceb40 0x7feb840b0650 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7feb7c005fd0 tx=0x7feb7c02c050 comp rx=0 tx=0).stop 2023-10-18T11:42:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.408+0000 7feb8c125640 1 -- 172.21.15.125:0/2610268247 shutdown_connections 2023-10-18T11:42:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.408+0000 7feb8c125640 1 --2- 172.21.15.125:0/2610268247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb840ceb40 0x7feb840b0650 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.408+0000 7feb8c125640 1 --2- 172.21.15.125:0/2610268247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb840cf340 0x7feb840d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.408+0000 7feb8c125640 1 -- 172.21.15.125:0/2610268247 >> 172.21.15.125:0/2610268247 conn(0x7feb840d0d60 msgr2=0x7feb840d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:23.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.409+0000 7feb8c125640 1 -- 172.21.15.125:0/2610268247 shutdown_connections 2023-10-18T11:42:23.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.409+0000 7feb8c125640 1 -- 172.21.15.125:0/2610268247 wait complete. 2023-10-18T11:42:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.410+0000 7feb8c125640 1 Processor -- start 2023-10-18T11:42:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.410+0000 7feb8c125640 1 -- start start 2023-10-18T11:42:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.410+0000 7feb8c125640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb840ceb40 0x7feb84158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.410+0000 7feb8c125640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb840cf340 0x7feb841e5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.410+0000 7feb8c125640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb8415b2a0 0x7feb841e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.410+0000 7feb8c125640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841f4bb0 con 0x7feb840ceb40 2023-10-18T11:42:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.410+0000 7feb8c125640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841f4e70 con 0x7feb840cf340 2023-10-18T11:42:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.410+0000 7feb8c125640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb841f5150 con 0x7feb8415b2a0 2023-10-18T11:42:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb840ceb40 0x7feb84158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb840ceb40 0x7feb84158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57314/0 (socket says 172.21.15.125:57314) 2023-10-18T11:42:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 -- 172.21.15.125:0/2144168279 learned_addr learned my addr 172.21.15.125:0/2144168279 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb896c0640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb840cf340 0x7feb841e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb8a6c2640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb8415b2a0 0x7feb841e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 -- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb8415b2a0 msgr2=0x7feb841e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb8415b2a0 0x7feb841e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 -- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb840cf340 msgr2=0x7feb841e5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb840cf340 0x7feb841e5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 -- 172.21.15.125:0/2144168279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb74009560 con 0x7feb840ceb40 2023-10-18T11:42:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb89ec1640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb840ceb40 0x7feb84158270 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7feb7400e9c0 tx=0x7feb7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb7a7fc640 1 -- 172.21.15.125:0/2144168279 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400cde0 con 0x7feb840ceb40 2023-10-18T11:42:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb7a7fc640 1 -- 172.21.15.125:0/2144168279 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb74004020 con 0x7feb840ceb40 2023-10-18T11:42:23.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c009840 con 0x7feb840ceb40 2023-10-18T11:42:23.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb7a7fc640 1 -- 172.21.15.125:0/2144168279 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74004410 con 0x7feb840ceb40 2023-10-18T11:42:23.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.411+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb841f5710 con 0x7feb840ceb40 2023-10-18T11:42:23.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.412+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb841f58a0 con 0x7feb840ceb40 2023-10-18T11:42:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.413+0000 7feb7a7fc640 1 -- 172.21.15.125:0/2144168279 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb740035f0 con 0x7feb840ceb40 2023-10-18T11:42:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.413+0000 7feb7a7fc640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7feb6806af60 0x7feb6806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.413+0000 7feb7a7fc640 1 -- 172.21.15.125:0/2144168279 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7feb74014070 con 0x7feb840ceb40 2023-10-18T11:42:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.414+0000 7feb896c0640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7feb6806af60 0x7feb6806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.414+0000 7feb896c0640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7feb6806af60 0x7feb6806d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb840b6aa0 tx=0x7feb80009040 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.418+0000 7feb7a7fc640 1 -- 172.21.15.125:0/2144168279 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb740552e0 con 0x7feb840ceb40 2023-10-18T11:42:23.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.661+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7feb840cb9b0 con 0x7feb840ceb40 2023-10-18T11:42:23.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.663+0000 7feb7a7fc640 1 -- 172.21.15.125:0/2144168279 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v50) v1 ==== 190+0+147 (secure 0 0 0) 0x7feb74058900 con 0x7feb840ceb40 2023-10-18T11:42:23.666 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T11:42:23.666 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T11:42:23.666 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T11:42:23.666 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T11:42:23.666 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T11:42:23.667 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T11:42:23.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7feb6806af60 msgr2=0x7feb6806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7feb6806af60 0x7feb6806d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb840b6aa0 tx=0x7feb80009040 comp rx=0 tx=0).stop 2023-10-18T11:42:23.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb840ceb40 msgr2=0x7feb84158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb840ceb40 0x7feb84158270 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7feb7400e9c0 tx=0x7feb7400ee80 comp rx=0 tx=0).stop 2023-10-18T11:42:23.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 shutdown_connections 2023-10-18T11:42:23.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7feb6806af60 0x7feb6806d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb840cf340 0x7feb841e5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb8415b2a0 0x7feb841e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 --2- 172.21.15.125:0/2144168279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb840ceb40 0x7feb84158270 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 >> 172.21.15.125:0/2144168279 conn(0x7feb840d0d60 msgr2=0x7feb840cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:23.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 shutdown_connections 2023-10-18T11:42:23.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.665+0000 7feb8c125640 1 -- 172.21.15.125:0/2144168279 wait complete. 2023-10-18T11:42:23.679 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:42:23.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.780+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/3053759658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe00047c0 msgr2=0x7f9fe0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.780+0000 7f9ffb6b4640 1 --2- 172.21.15.125:0/3053759658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fe00047c0 0x7f9fe0004c40 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400bb20 tx=0x7f9fe00057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.780+0000 7f9ff9c51640 1 -- 172.21.15.125:0/3053759658 reap_dead start 2023-10-18T11:42:23.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.780+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/3053759658 shutdown_connections 2023-10-18T11:42:23.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.780+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/3053759658 >> 172.21.15.125:0/3053759658 conn(0x7f9ff4069930 msgr2=0x7f9ff4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:23.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.780+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/3053759658 shutdown_connections 2023-10-18T11:42:23.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.780+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/3053759658 wait complete. 2023-10-18T11:42:23.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.781+0000 7f9ffb6b4640 1 Processor -- start 2023-10-18T11:42:23.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.781+0000 7f9ffb6b4640 1 -- start start 2023-10-18T11:42:23.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ffb6b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe00047c0 0x7f9ff41e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ffb6b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff41dd4e0 0x7f9ff41ded60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ffb6b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff41d8a40 0x7f9ff41d8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ffb6b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41de030 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ffb6b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41ddca0 con 0x7f9ff41d8a40 2023-10-18T11:42:23.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ffb6b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41de280 con 0x7f9fe00047c0 2023-10-18T11:42:23.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ff8c4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff41dd4e0 0x7f9ff41ded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ff8c4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff41dd4e0 0x7f9ff41ded60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57324/0 (socket says 172.21.15.125:57324) 2023-10-18T11:42:23.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ff8c4f640 1 -- 172.21.15.125:0/1391260626 learned_addr learned my addr 172.21.15.125:0/1391260626 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:23.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ff8c4f640 1 -- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe00047c0 msgr2=0x7f9ff41e2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ff8c4f640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe00047c0 0x7f9ff41e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ff8c4f640 1 -- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff41d8a40 msgr2=0x7f9ff41d8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ff8c4f640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff41d8a40 0x7f9ff41d8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.782+0000 7f9ff8c4f640 1 -- 172.21.15.125:0/1391260626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff41de560 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.783+0000 7f9ff8c4f640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff41dd4e0 0x7f9ff41ded60 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4038040 tx=0x7f9fe4036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:23.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.783+0000 7f9fe9ffb640 1 -- 172.21.15.125:0/1391260626 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4036de0 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.783+0000 7f9fe9ffb640 1 -- 172.21.15.125:0/1391260626 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9fe4004120 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.783+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff41de840 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.783+0000 7f9fe9ffb640 1 -- 172.21.15.125:0/1391260626 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40044f0 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.783+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff41df2a0 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.784+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff41de3c0 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.784+0000 7f9fe9ffb640 1 -- 172.21.15.125:0/1391260626 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9fe40035f0 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.785+0000 7f9fe9ffb640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9fd806af60 0x7f9fd806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:23.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.785+0000 7f9fe9ffb640 1 -- 172.21.15.125:0/1391260626 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f9fe40aba00 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.785+0000 7f9ff9450640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9fd806af60 0x7f9fd806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:23.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.786+0000 7f9ff9450640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9fd806af60 0x7f9fd806d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000d920 tx=0x7f9ff0005f90 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:23.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.789+0000 7f9fe9ffb640 1 -- 172.21.15.125:0/1391260626 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ff41de3c0 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.990+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ff41de3c0 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.990+0000 7f9fe9ffb640 1 -- 172.21.15.125:0/1391260626 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f9ff41de3c0 con 0x7f9ff41dd4e0 2023-10-18T11:42:23.993 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:42:23.994 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":73,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2139029617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2139029617},{"type":"v1","addr":"172.21.15.125:6835","nonce":2139029617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2438879012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2438879012},{"type":"v1","addr":"172.21.15.139:6837","nonce":2438879012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":10830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2643526616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2643526616},{"type":"v1","addr":"172.21.15.125:6834","nonce":2643526616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:42:14.915536+0000","modified":"2023-10-18T11:42:16.934466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6836/3571180970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3571180970},{"type":"v1","addr":"172.21.15.139:6836","nonce":3571180970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T11:42:23.994 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 73 2023-10-18T11:42:23.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.992+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9fd806af60 msgr2=0x7f9fd806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.992+0000 7f9ffb6b4640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9fd806af60 0x7f9fd806d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000d920 tx=0x7f9ff0005f90 comp rx=0 tx=0).stop 2023-10-18T11:42:23.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff41dd4e0 msgr2=0x7f9ff41ded60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff41dd4e0 0x7f9ff41ded60 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4038040 tx=0x7f9fe4036720 comp rx=0 tx=0).stop 2023-10-18T11:42:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 shutdown_connections 2023-10-18T11:42:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9fd806af60 0x7f9fd806d420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff41d8a40 0x7f9ff41d8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff41dd4e0 0x7f9ff41ded60 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 --2- 172.21.15.125:0/1391260626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9fe00047c0 0x7f9ff41e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 >> 172.21.15.125:0/1391260626 conn(0x7f9ff4069930 msgr2=0x7f9ff41403b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 shutdown_connections 2023-10-18T11:42:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:23.993+0000 7f9ffb6b4640 1 -- 172.21.15.125:0/1391260626 wait complete. 2023-10-18T11:42:24.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T11:42:14.915536+0000', 'modified': '2023-10-18T11:42:16.934466+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10845}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10845': {'gid': 10845, 'name': 'd', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6836/3571180970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3571180970}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 3571180970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T11:42:24.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T11:42:24.409 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-18T11:42:24.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.512+0000 7fa669dc0640 1 -- 172.21.15.125:0/84636074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa64c0050e0 msgr2=0x7fa64c0058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.512+0000 7fa669dc0640 1 --2- 172.21.15.125:0/84636074 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa64c0050e0 0x7fa64c0058d0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fa658006fd0 tx=0x7fa64c006900 comp rx=0 tx=0).stop 2023-10-18T11:42:24.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.512+0000 7fa663fff640 1 -- 172.21.15.125:0/84636074 reap_dead start 2023-10-18T11:42:24.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.512+0000 7fa669dc0640 1 -- 172.21.15.125:0/84636074 shutdown_connections 2023-10-18T11:42:24.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.512+0000 7fa669dc0640 1 -- 172.21.15.125:0/84636074 >> 172.21.15.125:0/84636074 conn(0x7fa664069930 msgr2=0x7fa664069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:24.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.513+0000 7fa669dc0640 1 -- 172.21.15.125:0/84636074 shutdown_connections 2023-10-18T11:42:24.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.513+0000 7fa669dc0640 1 -- 172.21.15.125:0/84636074 wait complete. 2023-10-18T11:42:24.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.514+0000 7fa669dc0640 1 Processor -- start 2023-10-18T11:42:24.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.514+0000 7fa669dc0640 1 -- start start 2023-10-18T11:42:24.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.514+0000 7fa669dc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa64c0050e0 0x7fa66414a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.514+0000 7fa669dc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa664152160 0x7fa6641525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.514+0000 7fa669dc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6641392d0 0x7fa664139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.514+0000 7fa669dc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66413ccd0 con 0x7fa6641392d0 2023-10-18T11:42:24.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.514+0000 7fa669dc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa664139c90 con 0x7fa64c0050e0 2023-10-18T11:42:24.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.514+0000 7fa669dc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66413c050 con 0x7fa664152160 2023-10-18T11:42:24.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6641392d0 0x7fa664139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6641392d0 0x7fa664139750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57342/0 (socket says 172.21.15.125:57342) 2023-10-18T11:42:24.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 -- 172.21.15.125:0/303275134 learned_addr learned my addr 172.21.15.125:0/303275134 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:24.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa6637fe640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa64c0050e0 0x7fa66414a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa662ffd640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa664152160 0x7fa6641525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 -- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa664152160 msgr2=0x7fa6641525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa664152160 0x7fa6641525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 -- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa64c0050e0 msgr2=0x7fa66414a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa64c0050e0 0x7fa66414a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 -- 172.21.15.125:0/303275134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66413c330 con 0x7fa6641392d0 2023-10-18T11:42:24.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa663fff640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6641392d0 0x7fa664139750 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fa6640b1110 tx=0x7fa65c010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:24.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa64bfff640 1 -- 172.21.15.125:0/303275134 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c003e40 con 0x7fa6641392d0 2023-10-18T11:42:24.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa64bfff640 1 -- 172.21.15.125:0/303275134 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa65c004b20 con 0x7fa6641392d0 2023-10-18T11:42:24.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa669dc0640 1 -- 172.21.15.125:0/303275134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66413c610 con 0x7fa6641392d0 2023-10-18T11:42:24.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa64bfff640 1 -- 172.21.15.125:0/303275134 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00e940 con 0x7fa6641392d0 2023-10-18T11:42:24.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.515+0000 7fa669dc0640 1 -- 172.21.15.125:0/303275134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66414be30 con 0x7fa6641392d0 2023-10-18T11:42:24.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.517+0000 7fa64bfff640 1 -- 172.21.15.125:0/303275134 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa65c00eaa0 con 0x7fa6641392d0 2023-10-18T11:42:24.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.517+0000 7fa64bfff640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa65006af60 0x7fa65006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.517+0000 7fa64bfff640 1 -- 172.21.15.125:0/303275134 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa65c08a450 con 0x7fa6641392d0 2023-10-18T11:42:24.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.517+0000 7fa6637fe640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa65006af60 0x7fa65006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.518+0000 7fa66ca2b740 1 Processor -- start 2023-10-18T11:42:24.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.518+0000 7fa66ca2b740 1 -- start start 2023-10-18T11:42:24.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.518+0000 7fa6637fe640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa65006af60 0x7fa65006d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa654006630 tx=0x7fa654006da0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:24.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286864ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x5622868948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286865020 0x562286894e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562286776730 con 0x562286865020 2023-10-18T11:42:24.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622867d7780 con 0x562286891b40 2023-10-18T11:42:24.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622867d4e70 con 0x562286891f20 2023-10-18T11:42:24.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286865020 0x562286894e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286865020 0x562286894e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57352/0 (socket says 172.21.15.125:57352) 2023-10-18T11:42:24.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 -- 172.21.15.125:0/4095507627 learned_addr learned my addr 172.21.15.125:0/4095507627 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:24.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 -- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 msgr2=0x5622868948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:42:24.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa662ffd640 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x5622868948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa6637fe640 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286864ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x5622868948d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 -- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 msgr2=0x562286864ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:42:24.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286864ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.533 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 -- 172.21.15.125:0/4095507627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622867d53d0 con 0x562286865020 2023-10-18T11:42:24.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa662ffd640 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x5622868948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:24.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa6637fe640 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286864ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:24.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa663fff640 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286865020 0x562286894e10 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005fd0 tx=0x7fa65c058bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27aee16ff626282 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:24.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa6497fa640 1 -- 172.21.15.125:0/4095507627 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c057320 con 0x562286865020 2023-10-18T11:42:24.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa6497fa640 1 -- 172.21.15.125:0/4095507627 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa65c00ed50 con 0x562286865020 2023-10-18T11:42:24.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.519+0000 7fa6497fa640 1 -- 172.21.15.125:0/4095507627 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c055690 con 0x562286865020 2023-10-18T11:42:24.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4095507627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286865020 msgr2=0x562286894e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286865020 0x562286894e10 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005fd0 tx=0x7fa65c058bc0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4095507627 shutdown_connections 2023-10-18T11:42:24.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286865020 0x562286894e10 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x5622868948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/4095507627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286864ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4095507627 >> 172.21.15.125:0/4095507627 conn(0x562286866d50 msgr2=0x5622868af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:24.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4095507627 shutdown_connections 2023-10-18T11:42:24.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.520+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4095507627 wait complete. 2023-10-18T11:42:24.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.521+0000 7fa66ca2b740 1 -- start start 2023-10-18T11:42:24.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.521+0000 7fa66ca2b740 1 Processor -- start 2023-10-18T11:42:24.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.521+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286897830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.521+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891f20 0x56228689bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.521+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286881ff0 0x56228689c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.521+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622867d0760 con 0x562286891f20 2023-10-18T11:42:24.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.521+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228678b8f0 con 0x562286891b40 2023-10-18T11:42:24.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.521+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56228678b590 con 0x562286881ff0 2023-10-18T11:42:24.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa66ca2b740 10 client.0 _ll_register_callbacks cb 0x5622867c7ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T11:42:24.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891f20 0x56228689bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891f20 0x56228689bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57364/0 (socket says 172.21.15.125:57364) 2023-10-18T11:42:24.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 -- 172.21.15.125:0/2113740451 learned_addr learned my addr 172.21.15.125:0/2113740451 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:24.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa6637fe640 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286897830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa663fff640 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286881ff0 0x56228689c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 -- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286881ff0 msgr2=0x56228689c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:42:24.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286881ff0 0x56228689c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 -- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 msgr2=0x562286897830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:42:24.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286897830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 -- 172.21.15.125:0/2113740451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66414be30 con 0x562286891f20 2023-10-18T11:42:24.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa662ffd640 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891f20 0x56228689bfd0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c350 tx=0x7fa6580014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbe9671d61538af server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:24.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa6427fc640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T11:42:24.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa6427fc640 1 -- 172.21.15.125:0/2113740451 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658046070 con 0x562286891f20 2023-10-18T11:42:24.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa6427fc640 1 -- 172.21.15.125:0/2113740451 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa658006030 con 0x562286891f20 2023-10-18T11:42:24.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2113740451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622867c9740 con 0x562286891f20 2023-10-18T11:42:24.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa6427fc640 1 -- 172.21.15.125:0/2113740451 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa658038070 con 0x562286891f20 2023-10-18T11:42:24.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa66ca2b740 10 client.10992 fetch_fsmap learned FSMap version 73 2023-10-18T11:42:24.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2113740451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5622867cede0 con 0x562286891f20 2023-10-18T11:42:24.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.522+0000 7fa6427fc640 1 -- 172.21.15.125:0/2113740451 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5622867c9740 con 0x562286891f20 2023-10-18T11:42:24.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa6427fc640 1 -- 172.21.15.125:0/2113740451 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa658049020 con 0x562286891f20 2023-10-18T11:42:24.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa66ca2b740 10 client.10992 fetch_fsmap finished waiting for FSMap version 73 2023-10-18T11:42:24.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa66ca2b740 10 client.10992 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-18T11:42:24.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa66ca2b740 20 client.10992 populate_metadata read hostname 'smithi125' 2023-10-18T11:42:24.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] conn(0x56228689a850 0x56228689ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa66ca2b740 4 client.10992 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T11:42:24.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2113740451 --> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622867c9460 con 0x56228689a850 2023-10-18T11:42:24.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa6637fe640 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] conn(0x56228689a850 0x56228689ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa6427fc640 10 client.10992 ms_handle_connect on v2:172.21.15.139:6834/3571180970 2023-10-18T11:42:24.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.523+0000 7fa6637fe640 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] conn(0x56228689a850 0x56228689ac30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4e24a25559218e server_cookie=4dc423c9d9061a0 in_seq=0 out_seq=0 2023-10-18T11:42:24.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa6427fc640 1 -- 172.21.15.125:0/2113740451 <== mds.0 v2:172.21.15.139:6834/3571180970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5622867c9460 con 0x56228689a850 2023-10-18T11:42:24.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa6427fc640 10 client.10992 handle_command_reply: tid=0 2023-10-18T11:42:24.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 client.10992 shutdown 2023-10-18T11:42:24.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2113740451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891f20 msgr2=0x56228689bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891f20 0x56228689bfd0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c350 tx=0x7fa6580014c0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2113740451 shutdown_connections 2023-10-18T11:42:24.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] conn(0x56228689a850 0x56228689ac30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286881ff0 0x56228689c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa6427fc640 0 client.10992 ms_handle_reset on v2:172.21.15.139:6834/3571180970 2023-10-18T11:42:24.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891f20 0x56228689bfd0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2113740451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891b40 0x562286897830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2113740451 >> 172.21.15.125:0/2113740451 conn(0x562286865020 msgr2=0x5622868689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:24.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2113740451 shutdown_connections 2023-10-18T11:42:24.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2113740451 wait complete. 2023-10-18T11:42:24.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.524+0000 7fa66ca2b740 20 client.10992 trim_cache size 0 max 16384 2023-10-18T11:42:24.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.539+0000 7fa66ca2b740 1 Processor -- start 2023-10-18T11:42:24.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.539+0000 7fa66ca2b740 1 -- start start 2023-10-18T11:42:24.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286881ff0 0x56228689be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891b40 0x56228689c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891f20 0x56228689ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562286751270 con 0x562286881ff0 2023-10-18T11:42:24.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562286869610 con 0x562286891f20 2023-10-18T11:42:24.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x562286860c50 con 0x562286891b40 2023-10-18T11:42:24.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286881ff0 0x56228689be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286881ff0 0x56228689be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57380/0 (socket says 172.21.15.125:57380) 2023-10-18T11:42:24.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa6637fe640 1 -- 172.21.15.125:0/4253945884 learned_addr learned my addr 172.21.15.125:0/4253945884 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:24.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa6637fe640 1 -- 172.21.15.125:0/4253945884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891b40 msgr2=0x56228689c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:42:24.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa6637fe640 1 --2- 172.21.15.125:0/4253945884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891b40 0x56228689c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa6637fe640 1 -- 172.21.15.125:0/4253945884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891f20 msgr2=0x56228689ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:42:24.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa6637fe640 1 --2- 172.21.15.125:0/4253945884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891f20 0x56228689ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.540+0000 7fa6637fe640 1 -- 172.21.15.125:0/4253945884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228676dd00 con 0x562286881ff0 2023-10-18T11:42:24.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa6637fe640 1 --2- 172.21.15.125:0/4253945884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286881ff0 0x56228689be60 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fa654004e60 tx=0x7fa654014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b245c71d3d262a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:24.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa649ffb640 1 -- 172.21.15.125:0/4253945884 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654023040 con 0x562286881ff0 2023-10-18T11:42:24.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa649ffb640 1 -- 172.21.15.125:0/4253945884 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa654016070 con 0x562286881ff0 2023-10-18T11:42:24.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa649ffb640 1 -- 172.21.15.125:0/4253945884 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654012900 con 0x562286881ff0 2023-10-18T11:42:24.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4253945884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286881ff0 msgr2=0x56228689be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/4253945884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286881ff0 0x56228689be60 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fa654004e60 tx=0x7fa654014000 comp rx=0 tx=0).stop 2023-10-18T11:42:24.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4253945884 shutdown_connections 2023-10-18T11:42:24.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/4253945884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286891f20 0x56228689ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/4253945884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891b40 0x56228689c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/4253945884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286881ff0 0x56228689be60 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4253945884 >> 172.21.15.125:0/4253945884 conn(0x562286865020 msgr2=0x5622868b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:24.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4253945884 shutdown_connections 2023-10-18T11:42:24.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.541+0000 7fa66ca2b740 1 -- 172.21.15.125:0/4253945884 wait complete. 2023-10-18T11:42:24.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.542+0000 7fa66ca2b740 1 -- start start 2023-10-18T11:42:24.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.542+0000 7fa66ca2b740 1 Processor -- start 2023-10-18T11:42:24.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286881ff0 0x562286894cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891b40 0x562286890a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x562286890f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228689bae0 con 0x562286891b40 2023-10-18T11:42:24.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622868c2500 con 0x562286881ff0 2023-10-18T11:42:24.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622868c19b0 con 0x562286891f20 2023-10-18T11:42:24.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 10 client.0 _ll_register_callbacks cb 0x7fa64c0047c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T11:42:24.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891b40 0x562286890a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891b40 0x562286890a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57392/0 (socket says 172.21.15.125:57392) 2023-10-18T11:42:24.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 -- 172.21.15.125:0/2031467161 learned_addr learned my addr 172.21.15.125:0/2031467161 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:24.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa6637fe640 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286881ff0 0x562286894cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa663fff640 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x562286890f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 -- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 msgr2=0x562286890f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:42:24.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x562286890f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 -- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286881ff0 msgr2=0x562286894cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:42:24.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286881ff0 0x562286894cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 -- 172.21.15.125:0/2031467161 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa664070cb0 con 0x562286891b40 2023-10-18T11:42:24.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa6637fe640 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286881ff0 0x562286894cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:24.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa662ffd640 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891b40 0x562286890a40 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fa658007010 tx=0x7fa658006ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0afaf9999c93d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:24.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa6427fc640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T11:42:24.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa6427fc640 1 -- 172.21.15.125:0/2031467161 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658046070 con 0x562286891b40 2023-10-18T11:42:24.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa6427fc640 1 -- 172.21.15.125:0/2031467161 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa658006540 con 0x562286891b40 2023-10-18T11:42:24.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2031467161 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa65c00ed50 con 0x562286891b40 2023-10-18T11:42:24.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa6427fc640 1 -- 172.21.15.125:0/2031467161 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6580060b0 con 0x562286891b40 2023-10-18T11:42:24.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 10 client.10998 fetch_fsmap learned FSMap version 73 2023-10-18T11:42:24.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2031467161 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5622867934f0 con 0x562286891b40 2023-10-18T11:42:24.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.543+0000 7fa6427fc640 1 -- 172.21.15.125:0/2031467161 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa658058020 con 0x562286891b40 2023-10-18T11:42:24.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa6427fc640 1 -- 172.21.15.125:0/2031467161 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa658049070 con 0x562286891b40 2023-10-18T11:42:24.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa66ca2b740 10 client.10998 fetch_fsmap finished waiting for FSMap version 73 2023-10-18T11:42:24.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa66ca2b740 10 client.10998 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-18T11:42:24.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa66ca2b740 20 client.10998 populate_metadata read hostname 'smithi125' 2023-10-18T11:42:24.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] conn(0x56228689ac30 0x56228689b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa66ca2b740 4 client.10998 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T11:42:24.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2031467161 --> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5622864d26c0 con 0x56228689ac30 2023-10-18T11:42:24.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa6637fe640 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] conn(0x56228689ac30 0x56228689b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa6427fc640 10 client.10998 ms_handle_connect on v2:172.21.15.139:6834/3571180970 2023-10-18T11:42:24.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa6637fe640 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] conn(0x56228689ac30 0x56228689b010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa3671c63011e7f server_cookie=d87c4cd2af419c8b in_seq=0 out_seq=0 2023-10-18T11:42:24.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa6427fc640 1 -- 172.21.15.125:0/2031467161 <== mds.0 v2:172.21.15.139:6834/3571180970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5622864d26c0 con 0x56228689ac30 2023-10-18T11:42:24.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa6427fc640 10 client.10998 handle_command_reply: tid=0 2023-10-18T11:42:24.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.544+0000 7fa66ca2b740 1 client.10998 shutdown 2023-10-18T11:42:24.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2031467161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891b40 msgr2=0x562286890a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891b40 0x562286890a40 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fa658007010 tx=0x7fa658006ea0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2031467161 shutdown_connections 2023-10-18T11:42:24.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:6834/3571180970,v1:172.21.15.139:6836/3571180970] conn(0x56228689ac30 0x56228689b010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562286891f20 0x562286890f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562286891b40 0x562286890a40 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa6427fc640 0 client.10998 ms_handle_reset on v2:172.21.15.139:6834/3571180970 2023-10-18T11:42:24.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 --2- 172.21.15.125:0/2031467161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562286881ff0 0x562286894cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2031467161 >> 172.21.15.125:0/2031467161 conn(0x562286865020 msgr2=0x56228689c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:24.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2031467161 shutdown_connections 2023-10-18T11:42:24.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 1 -- 172.21.15.125:0/2031467161 wait complete. 2023-10-18T11:42:24.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa66ca2b740 20 client.10998 trim_cache size 0 max 16384 2023-10-18T11:42:24.568 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:42:24.569 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T11:42:24.569 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T11:42:24.569 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 10845, 2023-10-18T11:42:24.569 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T11:42:24.569 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T11:42:24.569 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T11:42:24.570 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.9646849530000008, 2023-10-18T11:42:24.570 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 73, 2023-10-18T11:42:24.570 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 188, 2023-10-18T11:42:24.570 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 188, 2023-10-18T11:42:24.571 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 14.728826355000001 2023-10-18T11:42:24.571 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:42:24.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 -- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa65006af60 msgr2=0x7fa65006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa65006af60 0x7fa65006d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa654006630 tx=0x7fa654006da0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 -- 172.21.15.125:0/303275134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6641392d0 msgr2=0x7fa664139750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6641392d0 0x7fa664139750 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fa6640b1110 tx=0x7fa65c010ee0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 -- 172.21.15.125:0/303275134 shutdown_connections 2023-10-18T11:42:24.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa65006af60 0x7fa65006d420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6641392d0 0x7fa664139750 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa664152160 0x7fa6641525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 --2- 172.21.15.125:0/303275134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa64c0050e0 0x7fa66414a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 -- 172.21.15.125:0/303275134 >> 172.21.15.125:0/303275134 conn(0x7fa664069930 msgr2=0x7fa66406bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:24.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 -- 172.21.15.125:0/303275134 shutdown_connections 2023-10-18T11:42:24.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.545+0000 7fa669dc0640 1 -- 172.21.15.125:0/303275134 wait complete. 2023-10-18T11:42:24.575 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T11:42:24.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.674+0000 7fa023e1a640 1 -- 172.21.15.125:0/1496698465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01c0c9dc0 msgr2=0x7fa01c1531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.674+0000 7fa023e1a640 1 --2- 172.21.15.125:0/1496698465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01c0c9dc0 0x7fa01c1531d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009830 tx=0x7fa00c02bc20 comp rx=0 tx=0).stop 2023-10-18T11:42:24.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.674+0000 7fa023e1a640 1 -- 172.21.15.125:0/1496698465 shutdown_connections 2023-10-18T11:42:24.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.674+0000 7fa023e1a640 1 --2- 172.21.15.125:0/1496698465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01c0c9dc0 0x7fa01c1531d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.674+0000 7fa023e1a640 1 --2- 172.21.15.125:0/1496698465 >> v2:172.21.15.139:3300/0 conn(0x7fa01c0c99e0 0x7fa01c152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.674+0000 7fa023e1a640 1 -- 172.21.15.125:0/1496698465 >> 172.21.15.125:0/1496698465 conn(0x7fa01c069930 msgr2=0x7fa01c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:24.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.674+0000 7fa023e1a640 1 -- 172.21.15.125:0/1496698465 shutdown_connections 2023-10-18T11:42:24.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.674+0000 7fa023e1a640 1 -- 172.21.15.125:0/1496698465 wait complete. 2023-10-18T11:42:24.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.675+0000 7fa023e1a640 1 Processor -- start 2023-10-18T11:42:24.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.675+0000 7fa023e1a640 1 -- start start 2023-10-18T11:42:24.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa023e1a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01c0ab500 0x7fa01c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa023e1a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01c0c99e0 0x7fa01c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa023e1a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa01c0c9dc0 0x7fa01c152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa023e1a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0b1f50 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa023e1a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0b1c70 con 0x7fa01c0c99e0 2023-10-18T11:42:24.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa023e1a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c12d6e0 con 0x7fa01c0ab500 2023-10-18T11:42:24.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa01c0c9dc0 0x7fa01c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa01c0c9dc0 0x7fa01c152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57404/0 (socket says 172.21.15.125:57404) 2023-10-18T11:42:24.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 -- 172.21.15.125:0/733375830 learned_addr learned my addr 172.21.15.125:0/733375830 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:24.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa021bb6640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01c0ab500 0x7fa01c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0213b5640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01c0c99e0 0x7fa01c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 -- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01c0ab500 msgr2=0x7fa01c15b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01c0ab500 0x7fa01c15b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 -- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01c0c99e0 msgr2=0x7fa01c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01c0c99e0 0x7fa01c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 -- 172.21.15.125:0/733375830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018009490 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0223b7640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa01c0c9dc0 0x7fa01c152080 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fa014001870 tx=0x7fa01400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:24.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0127fc640 1 -- 172.21.15.125:0/733375830 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014012420 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c009490 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0127fc640 1 -- 172.21.15.125:0/733375830 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa014012690 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.676+0000 7fa0127fc640 1 -- 172.21.15.125:0/733375830 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014012aa0 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.677+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c12dca0 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.677+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c0c8d50 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.678+0000 7fa0127fc640 1 -- 172.21.15.125:0/733375830 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa01401b400 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.678+0000 7fa0127fc640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa00006af60 0x7fa00006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:24.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.679+0000 7fa0127fc640 1 -- 172.21.15.125:0/733375830 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fa01408a520 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.679+0000 7fa021bb6640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa00006af60 0x7fa00006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:24.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.679+0000 7fa021bb6640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa00006af60 0x7fa00006d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa018009d60 tx=0x7fa018005f90 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:24.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.683+0000 7fa0127fc640 1 -- 172.21.15.125:0/733375830 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa014055390 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.885+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa01c0690e0 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.885+0000 7fa0127fc640 1 -- 172.21.15.125:0/733375830 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa0140589b0 con 0x7fa01c0c9dc0 2023-10-18T11:42:24.888 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T11:42:24.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa00006af60 msgr2=0x7fa00006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa00006af60 0x7fa00006d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa018009d60 tx=0x7fa018005f90 comp rx=0 tx=0).stop 2023-10-18T11:42:24.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa01c0c9dc0 msgr2=0x7fa01c152080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:24.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa01c0c9dc0 0x7fa01c152080 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fa014001870 tx=0x7fa01400a430 comp rx=0 tx=0).stop 2023-10-18T11:42:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 shutdown_connections 2023-10-18T11:42:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fa00006af60 0x7fa00006d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01c0c99e0 0x7fa01c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa01c0c9dc0 0x7fa01c152080 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 --2- 172.21.15.125:0/733375830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01c0ab500 0x7fa01c15b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:24.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 >> 172.21.15.125:0/733375830 conn(0x7fa01c069930 msgr2=0x7fa01c065490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:24.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 shutdown_connections 2023-10-18T11:42:24.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:24.887+0000 7fa023e1a640 1 -- 172.21.15.125:0/733375830 wait complete. 2023-10-18T11:42:24.900 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T11:42:24.900 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T11:42:24.900 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T11:42:24.901 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T11:42:24.901 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T11:42:24.901 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T11:42:24.902 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T11:42:24.903 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T11:42:24.904 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:42:24.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:24.905 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T11:42:24.920 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T11:42:24.920 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T11:42:24.920 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T11:42:24.920 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:42:24.921 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T11:42:24.921 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:42:24.921 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:42:24.921 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:42:24.921 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T11:42:24.922 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:42:24.922 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:42:24.922 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T11:42:24.922 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:42:24.923 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:42:24.923 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T11:42:24.923 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:42:24.923 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T11:42:24.923 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 38079sec preferred_lft 38079sec 2023-10-18T11:42:24.923 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T11:42:24.924 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:42:24.924 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T11:42:24.924 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:42:24.924 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T11:42:24.925 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:42:24.925 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T11:42:24.925 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:42:24.925 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T11:42:24.925 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:42:24.926 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T11:42:24.926 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:42:24.926 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T11:42:24.926 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:42:24.926 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T11:42:24.927 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:42:24.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:24.927 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T11:42:24.985 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T11:42:24.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:24.987 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T11:42:25.043 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T11:42:25.043 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T11:42:25.043 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T11:42:25.043 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T11:42:25.044 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T11:42:25.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:25.044 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T11:42:25.045 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T11:42:25.045 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T11:42:25.046 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T11:42:25.046 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T11:42:25.046 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T11:42:25.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:25.138 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T11:42:25.153 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T11:42:25.153 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T11:42:25.154 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:42:25.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:25.154 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T11:42:25.219 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T11:42:25.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T11:42:25.220 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T11:42:25.275 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:42:25.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T11:42:25.276 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T11:42:25.329 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T11:42:25.396 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T11:42:25.451 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T11:42:25.452 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T11:42:25.452 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T11:42:25.452 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T11:42:25.453 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T11:42:25.453 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T11:42:25.453 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T11:42:25.453 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T11:42:25.453 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T11:42:25.453 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T11:42:25.454 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T11:42:25.454 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T11:42:25.454 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T11:42:25.454 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T11:42:25.455 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T11:42:25.455 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T11:42:25.455 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T11:42:25.455 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T11:42:25.455 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T11:42:25.455 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T11:42:25.456 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T11:42:25.456 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T11:42:25.456 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T11:42:25.456 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T11:42:25.456 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T11:42:25.457 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T11:42:25.457 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T11:42:25.457 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T11:42:25.457 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T11:42:25.457 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T11:42:25.458 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:42:25.458 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:42:25.458 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:42:25.458 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:42:25.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:25.459 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T11:42:25.506 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T11:42:25.506 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T11:42:25.548 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T11:42:25.574 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T11:42:25.631 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T11:42:25.631 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T11:42:25.632 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T11:42:25.632 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T11:42:25.632 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T11:42:25.632 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T11:42:25.632 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T11:42:25.633 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T11:42:25.633 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T11:42:25.633 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T11:42:25.633 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T11:42:25.633 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T11:42:25.634 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T11:42:25.634 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T11:42:25.634 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T11:42:25.634 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T11:42:25.635 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T11:42:25.635 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T11:42:25.635 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T11:42:25.635 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T11:42:25.636 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T11:42:25.636 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T11:42:25.636 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T11:42:25.636 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T11:42:25.636 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T11:42:25.637 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T11:42:25.637 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T11:42:25.637 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T11:42:25.637 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T11:42:25.637 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T11:42:25.638 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:42:25.638 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:42:25.638 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:42:25.638 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:42:25.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:25.639 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T11:42:25.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T11:42:25.633+0000 7fc13e03a480 -1 init, newargv = 0x557a72178bf0 newargc=15 2023-10-18T11:42:25.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[146596]: starting ceph client 2023-10-18T11:42:25.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[146596]: starting fuse 2023-10-18T11:42:25.654 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T11:42:25.655 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T11:42:25.655 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T11:42:25.656 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T11:42:25.656 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T11:42:25.656 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T11:42:25.657 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T11:42:25.657 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:42:25.658 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T11:42:25.658 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T11:42:25.658 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T11:42:25.659 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T11:42:25.659 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:42:25.659 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T11:42:25.660 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T11:42:25.660 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T11:42:25.661 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T11:42:25.661 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:42:25.661 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T11:42:25.662 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T11:42:25.662 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T11:42:25.663 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T11:42:25.663 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T11:42:25.663 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T11:42:25.664 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T11:42:25.664 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T11:42:25.665 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:42:25.665 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T11:42:25.665 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T11:42:25.774 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.146596.asok 2023-10-18T11:42:25.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T11:42:25.784 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.146596.asok 2023-10-18T11:42:25.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:25.785 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.146596.asok status 2023-10-18T11:42:25.918 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:42:25.919 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T11:42:25.919 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T11:42:25.919 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T11:42:25.919 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T11:42:25.919 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T11:42:25.920 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T11:42:25.920 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "146596", 2023-10-18T11:42:25.920 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T11:42:25.920 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:25.921 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T11:42:25.921 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T11:42:25.921 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 11007, 2023-10-18T11:42:25.921 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T11:42:25.921 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T11:42:25.922 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T11:42:25.922 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 11007 2023-10-18T11:42:25.922 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:25.922 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T11:42:25.922 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T11:42:25.923 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T11:42:25.923 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 3172518104 2023-10-18T11:42:25.923 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:25.923 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:25.924 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T11:42:25.924 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T11:42:25.924 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T11:42:25.924 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 3172518104 2023-10-18T11:42:25.924 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:42:25.925 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.11007 192.168.0.1:0/3172518104", 2023-10-18T11:42:25.925 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/3172518104", 2023-10-18T11:42:25.925 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T11:42:25.925 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 73, 2023-10-18T11:42:25.925 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 188, 2023-10-18T11:42:25.926 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T11:42:25.926 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T11:42:25.926 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T11:42:25.926 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:42:25.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:25.928 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T11:42:25.928 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T11:42:25.943 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T11:42:25.944 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T11:42:25.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:25.945 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T11:42:26.012 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T11:42:26.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.152+0000 7feaea6f2640 1 -- 172.21.15.125:0/4077870580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 msgr2=0x7feae40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:26.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.152+0000 7feaea6f2640 1 --2- 172.21.15.125:0/4077870580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 0x7feae40d1910 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fead4005fd0 tx=0x7fead402bd10 comp rx=0 tx=0).stop 2023-10-18T11:42:26.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.152+0000 7feaea6f2640 1 -- 172.21.15.125:0/4077870580 shutdown_connections 2023-10-18T11:42:26.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.152+0000 7feaea6f2640 1 --2- 172.21.15.125:0/4077870580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feae415b2a0 0x7feae40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.152+0000 7feaea6f2640 1 --2- 172.21.15.125:0/4077870580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 0x7feae40d1910 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.152+0000 7feaea6f2640 1 -- 172.21.15.125:0/4077870580 >> 172.21.15.125:0/4077870580 conn(0x7feae40d0d60 msgr2=0x7feae40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:26.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.152+0000 7feaea6f2640 1 -- 172.21.15.125:0/4077870580 shutdown_connections 2023-10-18T11:42:26.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.152+0000 7feaea6f2640 1 -- 172.21.15.125:0/4077870580 wait complete. 2023-10-18T11:42:26.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.153+0000 7feaea6f2640 1 Processor -- start 2023-10-18T11:42:26.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feaea6f2640 1 -- start start 2023-10-18T11:42:26.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feaea6f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feae40ceb40 0x7feae415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:26.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feaea6f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 0x7feae4157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:26.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feaea6f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feae415b2a0 0x7feae40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:26.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feaea6f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae415b870 con 0x7feae40cf340 2023-10-18T11:42:26.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feaea6f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae40b57c0 con 0x7feae40ceb40 2023-10-18T11:42:26.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feaea6f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae40b54e0 con 0x7feae415b2a0 2023-10-18T11:42:26.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feae37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 0x7feae4157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:26.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feae37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 0x7feae4157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57426/0 (socket says 172.21.15.125:57426) 2023-10-18T11:42:26.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.154+0000 7feae37fe640 1 -- 172.21.15.125:0/3109826337 learned_addr learned my addr 172.21.15.125:0/3109826337 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:26.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae37fe640 1 -- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feae415b2a0 msgr2=0x7feae40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:26.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae3fff640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feae40ceb40 0x7feae415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:26.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae37fe640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feae415b2a0 0x7feae40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae37fe640 1 -- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feae40ceb40 msgr2=0x7feae415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:26.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae37fe640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feae40ceb40 0x7feae415c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae37fe640 1 -- 172.21.15.125:0/3109826337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead4009560 con 0x7feae40cf340 2023-10-18T11:42:26.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae37fe640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 0x7feae4157e20 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fead800c970 tx=0x7fead800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:26.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae0ff9640 1 -- 172.21.15.125:0/3109826337 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8007bb0 con 0x7feae40cf340 2023-10-18T11:42:26.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae0ff9640 1 -- 172.21.15.125:0/3109826337 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fead8007e00 con 0x7feae40cf340 2023-10-18T11:42:26.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae415bb10 con 0x7feae40cf340 2023-10-18T11:42:26.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feae0ff9640 1 -- 172.21.15.125:0/3109826337 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead800f6b0 con 0x7feae40cf340 2023-10-18T11:42:26.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.155+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae4158360 con 0x7feae40cf340 2023-10-18T11:42:26.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.156+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae415b6d0 con 0x7feae40cf340 2023-10-18T11:42:26.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.156+0000 7feae0ff9640 1 -- 172.21.15.125:0/3109826337 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fead800f810 con 0x7feae40cf340 2023-10-18T11:42:26.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.157+0000 7feae0ff9640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fead006af60 0x7fead006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:26.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.157+0000 7feae0ff9640 1 -- 172.21.15.125:0/3109826337 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fead808a290 con 0x7feae40cf340 2023-10-18T11:42:26.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.157+0000 7feae3fff640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fead006af60 0x7fead006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:26.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.158+0000 7feae3fff640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fead006af60 0x7fead006d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fead4035000 tx=0x7fead4004730 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:26.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.161+0000 7feae0ff9640 1 -- 172.21.15.125:0/3109826337 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feae415b6d0 con 0x7feae40cf340 2023-10-18T11:42:26.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.346+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 --> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7feae40cc170 con 0x7fead006af60 2023-10-18T11:42:26.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.349+0000 7feae0ff9640 1 -- 172.21.15.125:0/3109826337 <== mgr.8003 v2:172.21.15.139:6832/3367593104 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fead4005680 con 0x7fead006af60 2023-10-18T11:42:26.351 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T11:42:26.351 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:42:26.351 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T11:42:26.351 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:42:26.352 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T11:42:26.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fead006af60 msgr2=0x7fead006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:26.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fead006af60 0x7fead006d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fead4035000 tx=0x7fead4004730 comp rx=0 tx=0).stop 2023-10-18T11:42:26.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 msgr2=0x7feae4157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:26.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 0x7feae4157e20 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fead800c970 tx=0x7fead800ce30 comp rx=0 tx=0).stop 2023-10-18T11:42:26.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 shutdown_connections 2023-10-18T11:42:26.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fead006af60 0x7fead006d420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feae415b2a0 0x7feae40b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae40cf340 0x7feae4157e20 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 --2- 172.21.15.125:0/3109826337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feae40ceb40 0x7feae415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 >> 172.21.15.125:0/3109826337 conn(0x7feae40d0d60 msgr2=0x7feae415a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:26.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 shutdown_connections 2023-10-18T11:42:26.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.350+0000 7feaea6f2640 1 -- 172.21.15.125:0/3109826337 wait complete. 2023-10-18T11:42:26.363 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T11:42:26.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.465+0000 7f3e025b7640 1 -- 172.21.15.125:0/2965439866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de4005180 msgr2=0x7f3de400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:26.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.465+0000 7f3e025b7640 1 --2- 172.21.15.125:0/2965439866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de4005180 0x7f3de400b980 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f3df40043b0 tx=0x7f3de40057f0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.465+0000 7f3e025b7640 1 -- 172.21.15.125:0/2965439866 shutdown_connections 2023-10-18T11:42:26.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.465+0000 7f3e025b7640 1 --2- 172.21.15.125:0/2965439866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de4005180 0x7f3de400b980 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.465+0000 7f3e025b7640 1 -- 172.21.15.125:0/2965439866 >> 172.21.15.125:0/2965439866 conn(0x7f3dfc069930 msgr2=0x7f3dfc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:26.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.465+0000 7f3e025b7640 1 -- 172.21.15.125:0/2965439866 shutdown_connections 2023-10-18T11:42:26.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.465+0000 7f3e025b7640 1 -- 172.21.15.125:0/2965439866 wait complete. 2023-10-18T11:42:26.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.466+0000 7f3e025b7640 1 Processor -- start 2023-10-18T11:42:26.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.466+0000 7f3e025b7640 1 -- start start 2023-10-18T11:42:26.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3e025b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de4005180 0x7f3dfc1f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:26.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3e025b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dfc1f26a0 0x7f3dfc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:26.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3e025b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dfc06f5c0 0x7f3dfc0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:26.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3e025b7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc1f2520 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3e025b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc152c00 con 0x7f3dfc06f5c0 2023-10-18T11:42:26.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3e025b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc06fad0 con 0x7f3de4005180 2023-10-18T11:42:26.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dfc1f26a0 0x7f3dfc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:26.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dfc1f26a0 0x7f3dfc06f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57446/0 (socket says 172.21.15.125:57446) 2023-10-18T11:42:26.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfb7fe640 1 -- 172.21.15.125:0/1858205215 learned_addr learned my addr 172.21.15.125:0/1858205215 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:26.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3e00b54640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dfc06f5c0 0x7f3dfc0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:26.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfbfff640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de4005180 0x7f3dfc1f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:26.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfb7fe640 1 -- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de4005180 msgr2=0x7f3dfc1f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:26.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfb7fe640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de4005180 0x7f3dfc1f1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfb7fe640 1 -- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dfc06f5c0 msgr2=0x7f3dfc0aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:26.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfb7fe640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dfc06f5c0 0x7f3dfc0aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:26.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.467+0000 7f3dfb7fe640 1 -- 172.21.15.125:0/1858205215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc06fdb0 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.468+0000 7f3dfb7fe640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dfc1f26a0 0x7f3dfc06f080 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e510 tx=0x7f3dec00e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:26.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.468+0000 7f3df8ff9640 1 -- 172.21.15.125:0/1858205215 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec010890 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.468+0000 7f3df8ff9640 1 -- 172.21.15.125:0/1858205215 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3dec010ae0 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.468+0000 7f3df8ff9640 1 -- 172.21.15.125:0/1858205215 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec011460 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.468+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc152890 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.468+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc1f2b20 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.469+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dfc152d70 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.469+0000 7f3df8ff9640 1 -- 172.21.15.125:0/1858205215 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 101) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3dec0115c0 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.470+0000 7f3df8ff9640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f3de806af60 0x7f3de806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:26.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.470+0000 7f3df8ff9640 1 -- 172.21.15.125:0/1858205215 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f3dec08a230 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.470+0000 7f3dfbfff640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f3de806af60 0x7f3de806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:26.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.470+0000 7f3dfbfff640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f3de806af60 0x7f3de806d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3df000fad0 tx=0x7f3df000f4b0 comp rx=0 tx=0).ready entity=mgr.8003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:26.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.474+0000 7f3df8ff9640 1 -- 172.21.15.125:0/1858205215 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dec054ec0 con 0x7f3dfc1f26a0 2023-10-18T11:42:26.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:26.687+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f3dfc0690e0 con 0x7f3dfc1f26a0 2023-10-18T11:42:27.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.632+0000 7f3df8ff9640 1 -- 172.21.15.125:0/1858205215 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v102) v1 ==== 92+0+0 (secure 0 0 0) 0x7f3dec0150a0 con 0x7f3dfc1f26a0 2023-10-18T11:42:27.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f3de806af60 msgr2=0x7f3de806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:27.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f3de806af60 0x7f3de806d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3df000fad0 tx=0x7f3df000f4b0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dfc1f26a0 msgr2=0x7f3dfc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:27.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dfc1f26a0 0x7f3dfc06f080 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e510 tx=0x7f3dec00e9d0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 shutdown_connections 2023-10-18T11:42:27.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f3de806af60 0x7f3de806d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dfc06f5c0 0x7f3dfc0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dfc1f26a0 0x7f3dfc06f080 secure :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e510 tx=0x7f3dec00e9d0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 --2- 172.21.15.125:0/1858205215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de4005180 0x7f3dfc1f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.636+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 >> 172.21.15.125:0/1858205215 conn(0x7f3dfc069930 msgr2=0x7f3dfc159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:27.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.637+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 shutdown_connections 2023-10-18T11:42:27.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.637+0000 7f3e025b7640 1 -- 172.21.15.125:0/1858205215 wait complete. 2023-10-18T11:42:27.651 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T11:42:27.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 -- 172.21.15.125:0/167329540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e80ceb40 msgr2=0x7f15e80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:27.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 --2- 172.21.15.125:0/167329540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e80ceb40 0x7f15e80b0650 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f15e806c510 tx=0x7f15e002bd10 comp rx=0 tx=0).stop 2023-10-18T11:42:27.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 -- 172.21.15.125:0/167329540 shutdown_connections 2023-10-18T11:42:27.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 --2- 172.21.15.125:0/167329540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e80ceb40 0x7f15e80b0650 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 --2- 172.21.15.125:0/167329540 >> v2:172.21.15.139:3300/0 conn(0x7f15e815b2a0 0x7f15e80cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 --2- 172.21.15.125:0/167329540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15e80cf340 0x7f15e80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 -- 172.21.15.125:0/167329540 >> 172.21.15.125:0/167329540 conn(0x7f15e80d0d60 msgr2=0x7f15e80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:27.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 -- 172.21.15.125:0/167329540 shutdown_connections 2023-10-18T11:42:27.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.759+0000 7f15edc39640 1 -- 172.21.15.125:0/167329540 wait complete. 2023-10-18T11:42:27.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.760+0000 7f15edc39640 1 Processor -- start 2023-10-18T11:42:27.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15edc39640 1 -- start start 2023-10-18T11:42:27.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15edc39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15e80ceb40 0x7f15e81597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:27.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15edc39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e80cf340 0x7f15e80b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:27.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15edc39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e815b2a0 0x7f15e80b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:27.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15edc39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80c8f70 con 0x7f15e815b2a0 2023-10-18T11:42:27.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15edc39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80caf10 con 0x7f15e80cf340 2023-10-18T11:42:27.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15edc39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80ca300 con 0x7f15e80ceb40 2023-10-18T11:42:27.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15e7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e815b2a0 0x7f15e80b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:27.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15e7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e815b2a0 0x7f15e80b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38558/0 (socket says 172.21.15.125:38558) 2023-10-18T11:42:27.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15e7fff640 1 -- 172.21.15.125:0/3163750354 learned_addr learned my addr 172.21.15.125:0/3163750354 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:27.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.761+0000 7f15e77fe640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15e80ceb40 0x7f15e81597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:27.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15e6ffd640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e80cf340 0x7f15e80b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:27.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15e7fff640 1 -- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15e80ceb40 msgr2=0x7f15e81597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:27.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15e7fff640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15e80ceb40 0x7f15e81597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15e7fff640 1 -- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e80cf340 msgr2=0x7f15e80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:27.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15e7fff640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e80cf340 0x7f15e80b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15e7fff640 1 -- 172.21.15.125:0/3163750354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e0009560 con 0x7f15e815b2a0 2023-10-18T11:42:27.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15e7fff640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e815b2a0 0x7f15e80b61d0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f15e00099c0 tx=0x7f15e002d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:27.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15dffff640 1 -- 172.21.15.125:0/3163750354 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e002dd70 con 0x7f15e815b2a0 2023-10-18T11:42:27.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15dffff640 1 -- 172.21.15.125:0/3163750354 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f15e00035f0 con 0x7f15e815b2a0 2023-10-18T11:42:27.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15dffff640 1 -- 172.21.15.125:0/3163750354 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0003a00 con 0x7f15e815b2a0 2023-10-18T11:42:27.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e80ca5e0 con 0x7f15e815b2a0 2023-10-18T11:42:27.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.762+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e80caba0 con 0x7f15e815b2a0 2023-10-18T11:42:27.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.763+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e81e3df0 con 0x7f15e815b2a0 2023-10-18T11:42:27.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.764+0000 7f15dffff640 1 -- 172.21.15.125:0/3163750354 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 102) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f15e0003b60 con 0x7f15e815b2a0 2023-10-18T11:42:27.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.764+0000 7f15dffff640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f15c806afb0 0x7f15c806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:27.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.764+0000 7f15dffff640 1 -- 172.21.15.125:0/3163750354 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f15e00ac170 con 0x7f15e815b2a0 2023-10-18T11:42:27.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.764+0000 7f15e77fe640 1 -- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f15c806afb0 msgr2=0x7f15c806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3367593104 2023-10-18T11:42:27.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.764+0000 7f15e77fe640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f15c806afb0 0x7f15c806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:42:27.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.768+0000 7f15dffff640 1 -- 172.21.15.125:0/3163750354 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15e81e3df0 con 0x7f15e815b2a0 2023-10-18T11:42:27.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.944+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f15e81e3df0 con 0x7f15e815b2a0 2023-10-18T11:42:27.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.947+0000 7f15dffff640 1 -- 172.21.15.125:0/3163750354 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v68)=0 v68) v1 ==== 138+0+0 (secure 0 0 0) 0x7f15e81e3df0 con 0x7f15e815b2a0 2023-10-18T11:42:27.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.949+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f15c806afb0 msgr2=0x7f15c806d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:42:27.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f15c806afb0 0x7f15c806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e815b2a0 msgr2=0x7f15e80b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:27.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e815b2a0 0x7f15e80b61d0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f15e00099c0 tx=0x7f15e002d970 comp rx=0 tx=0).stop 2023-10-18T11:42:27.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 shutdown_connections 2023-10-18T11:42:27.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f15c806afb0 0x7f15c806d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15e80cf340 0x7f15e80b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15e815b2a0 0x7f15e80b61d0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 --2- 172.21.15.125:0/3163750354 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f15e80ceb40 0x7f15e81597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:27.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 >> 172.21.15.125:0/3163750354 conn(0x7f15e80d0d60 msgr2=0x7f15e80aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:27.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 shutdown_connections 2023-10-18T11:42:27.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:27.950+0000 7f15edc39640 1 -- 172.21.15.125:0/3163750354 wait complete. 2023-10-18T11:42:27.963 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T11:42:28.024 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.022+0000 7f6f47232040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:42:28.032 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.030+0000 7f14824b4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:42:28.069 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.066+0000 7f6f47232040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:42:28.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.069+0000 7fdd44c07640 1 -- 172.21.15.125:0/370707390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 msgr2=0x7fdd400b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:28.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.069+0000 7fdd44c07640 1 --2- 172.21.15.125:0/370707390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd400b0650 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fdd30005fd0 tx=0x7fdd3002bd90 comp rx=0 tx=0).stop 2023-10-18T11:42:28.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.070+0000 7fdd44c07640 1 -- 172.21.15.125:0/370707390 shutdown_connections 2023-10-18T11:42:28.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.070+0000 7fdd44c07640 1 --2- 172.21.15.125:0/370707390 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd400b0650 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.070+0000 7fdd44c07640 1 --2- 172.21.15.125:0/370707390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd4015b2a0 0x7fdd400d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.070+0000 7fdd44c07640 1 -- 172.21.15.125:0/370707390 >> 172.21.15.125:0/370707390 conn(0x7fdd400d0d60 msgr2=0x7fdd400d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:28.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.070+0000 7fdd44c07640 1 -- 172.21.15.125:0/370707390 shutdown_connections 2023-10-18T11:42:28.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.070+0000 7fdd44c07640 1 -- 172.21.15.125:0/370707390 wait complete. 2023-10-18T11:42:28.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.071+0000 7fdd44c07640 1 Processor -- start 2023-10-18T11:42:28.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.071+0000 7fdd44c07640 1 -- start start 2023-10-18T11:42:28.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd44c07640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd400b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:28.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd44c07640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd400cf340 0x7fdd401f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:28.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd44c07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd4015b2a0 0x7fdd40150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:28.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd44c07640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd401f5ba0 con 0x7fdd400ceb40 2023-10-18T11:42:28.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd44c07640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd401f4c10 con 0x7fdd4015b2a0 2023-10-18T11:42:28.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd44c07640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd400b5a40 con 0x7fdd400cf340 2023-10-18T11:42:28.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd400b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:28.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd400b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38576/0 (socket says 172.21.15.125:38576) 2023-10-18T11:42:28.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 -- 172.21.15.125:0/3554516561 learned_addr learned my addr 172.21.15.125:0/3554516561 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:28.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3dd9b640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd400cf340 0x7fdd401f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:28.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3ed9d640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd4015b2a0 0x7fdd40150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:28.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 -- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd400cf340 msgr2=0x7fdd401f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:28.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd400cf340 0x7fdd401f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 -- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd4015b2a0 msgr2=0x7fdd40150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:28.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd4015b2a0 0x7fdd40150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 -- 172.21.15.125:0/3554516561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30009560 con 0x7fdd400ceb40 2023-10-18T11:42:28.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd3e59c640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd400b67a0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800c790 tx=0x7fdd3800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:28.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd36ffd640 1 -- 172.21.15.125:0/3554516561 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800cea0 con 0x7fdd400ceb40 2023-10-18T11:42:28.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd36ffd640 1 -- 172.21.15.125:0/3554516561 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd38006e40 con 0x7fdd400ceb40 2023-10-18T11:42:28.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd400b0d50 con 0x7fdd400ceb40 2023-10-18T11:42:28.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.072+0000 7fdd36ffd640 1 -- 172.21.15.125:0/3554516561 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38003bd0 con 0x7fdd400ceb40 2023-10-18T11:42:28.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.073+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd40155070 con 0x7fdd400ceb40 2023-10-18T11:42:28.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.074+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd400690e0 con 0x7fdd400ceb40 2023-10-18T11:42:28.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.074+0000 7fdd36ffd640 1 -- 172.21.15.125:0/3554516561 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 102) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fdd38003d30 con 0x7fdd400ceb40 2023-10-18T11:42:28.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.074+0000 7fdd36ffd640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fdd2006afb0 0x7fdd2006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:28.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.074+0000 7fdd36ffd640 1 -- 172.21.15.125:0/3554516561 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7fdd380896b0 con 0x7fdd400ceb40 2023-10-18T11:42:28.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.074+0000 7fdd3dd9b640 1 -- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fdd2006afb0 msgr2=0x7fdd2006d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3367593104 2023-10-18T11:42:28.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.074+0000 7fdd3dd9b640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fdd2006afb0 0x7fdd2006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:42:28.082 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.078+0000 7f14824b4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:42:28.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.079+0000 7fdd36ffd640 1 -- 172.21.15.125:0/3554516561 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd38058040 con 0x7fdd400ceb40 2023-10-18T11:42:28.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.254+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fdd400d0b40 con 0x7fdd400ceb40 2023-10-18T11:42:28.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.257+0000 7fdd36ffd640 1 -- 172.21.15.125:0/3554516561 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v69)=0 v69) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdd38057bd0 con 0x7fdd400ceb40 2023-10-18T11:42:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fdd2006afb0 msgr2=0x7fdd2006d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:42:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fdd2006afb0 0x7fdd2006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 msgr2=0x7fdd400b67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd400b67a0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800c790 tx=0x7fdd3800cc50 comp rx=0 tx=0).stop 2023-10-18T11:42:28.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 shutdown_connections 2023-10-18T11:42:28.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7fdd2006afb0 0x7fdd2006d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd4015b2a0 0x7fdd40150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd400cf340 0x7fdd401f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 --2- 172.21.15.125:0/3554516561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd400ceb40 0x7fdd400b67a0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 >> 172.21.15.125:0/3554516561 conn(0x7fdd400d0d60 msgr2=0x7fdd400ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:28.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 shutdown_connections 2023-10-18T11:42:28.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.259+0000 7fdd44c07640 1 -- 172.21.15.125:0/3554516561 wait complete. 2023-10-18T11:42:28.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:28.273 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T11:42:28.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:42:28.294 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-10-18T11:42:28.351 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1m 2023-10-18T11:42:28.463 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.460+0000 7f6f47232040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:42:28.492 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.490+0000 7f14824b4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:42:28.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.496+0000 7f9d79a61640 1 -- 172.21.15.125:0/1258782549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d741428d0 msgr2=0x7f9d740ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:28.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.496+0000 7f9d79a61640 1 --2- 172.21.15.125:0/1258782549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d741428d0 0x7f9d740ccd20 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c005fd0 tx=0x7f9d6c02c0e0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.496+0000 7f9d79a61640 1 -- 172.21.15.125:0/1258782549 shutdown_connections 2023-10-18T11:42:28.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.496+0000 7f9d79a61640 1 --2- 172.21.15.125:0/1258782549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d741428d0 0x7f9d740ccd20 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.496+0000 7f9d79a61640 1 -- 172.21.15.125:0/1258782549 >> 172.21.15.125:0/1258782549 conn(0x7f9d740af830 msgr2=0x7f9d740afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:28.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.496+0000 7f9d79a61640 1 -- 172.21.15.125:0/1258782549 shutdown_connections 2023-10-18T11:42:28.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.496+0000 7f9d79a61640 1 -- 172.21.15.125:0/1258782549 wait complete. 2023-10-18T11:42:28.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.497+0000 7f9d79a61640 1 Processor -- start 2023-10-18T11:42:28.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.498+0000 7f9d79a61640 1 -- start start 2023-10-18T11:42:28.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.498+0000 7f9d79a61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d74069e30 0x7f9d740b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:28.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.498+0000 7f9d79a61640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d7406da10 0x7f9d741f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:28.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.498+0000 7f9d79a61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d741428d0 0x7f9d7415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:28.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.498+0000 7f9d79a61640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d741354c0 con 0x7f9d7406da10 2023-10-18T11:42:28.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.498+0000 7f9d79a61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74154860 con 0x7f9d741428d0 2023-10-18T11:42:28.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.498+0000 7f9d79a61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74138ce0 con 0x7f9d74069e30 2023-10-18T11:42:28.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d7406da10 0x7f9d741f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:28.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d7406da10 0x7f9d741f0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38584/0 (socket says 172.21.15.125:38584) 2023-10-18T11:42:28.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 -- 172.21.15.125:0/379139037 learned_addr learned my addr 172.21.15.125:0/379139037 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:28.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d72ffd640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d74069e30 0x7f9d740b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:28.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d737fe640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d741428d0 0x7f9d7415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:28.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d74069e30 msgr2=0x7f9d740b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:28.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d74069e30 0x7f9d740b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d741428d0 msgr2=0x7f9d7415b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:28.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d741428d0 0x7f9d7415b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:28.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 -- 172.21.15.125:0/379139037 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c009840 con 0x7f9d7406da10 2023-10-18T11:42:28.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d727fc640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d7406da10 0x7f9d741f0b60 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c480 tx=0x7f9d5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:28.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0052d0 con 0x7f9d7406da10 2023-10-18T11:42:28.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9d5c005430 con 0x7f9d7406da10 2023-10-18T11:42:28.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c012890 con 0x7f9d7406da10 2023-10-18T11:42:28.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d740b2c60 con 0x7f9d7406da10 2023-10-18T11:42:28.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.499+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d74154580 con 0x7f9d7406da10 2023-10-18T11:42:28.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.501+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d740690e0 con 0x7f9d7406da10 2023-10-18T11:42:28.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.501+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 102) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9d5c0035f0 con 0x7f9d7406da10 2023-10-18T11:42:28.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.501+0000 7f9d6bfff640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 0x7f9d5406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:28.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.501+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6057+0+0 (secure 0 0 0) 0x7f9d5c0894b0 con 0x7f9d7406da10 2023-10-18T11:42:28.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.501+0000 7f9d72ffd640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 msgr2=0x7f9d5406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3367593104 2023-10-18T11:42:28.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.501+0000 7f9d72ffd640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 0x7f9d5406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:42:28.508 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.505+0000 7f6f47232040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:42:28.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.505+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d5c054430 con 0x7f9d7406da10 2023-10-18T11:42:28.539 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.537+0000 7f14824b4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:42:28.549 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.547+0000 7f6f47232040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:42:28.583 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.581+0000 7f14824b4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:42:28.600 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.598+0000 7f6f47232040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:42:28.636 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.634+0000 7f14824b4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:42:28.642 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.640+0000 7f6f47232040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:42:28.680 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.677+0000 7f14824b4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:42:28.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.700+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 --> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d74154400 con 0x7f9d5406afb0 2023-10-18T11:42:28.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.702+0000 7f9d72ffd640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 msgr2=0x7f9d5406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3367593104 2023-10-18T11:42:28.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:28.702+0000 7f9d72ffd640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 0x7f9d5406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:42:28.741 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.738+0000 7f6f47232040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:42:28.779 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.777+0000 7f14824b4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:42:28.792 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.790+0000 7f6f47232040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:42:28.833 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.830+0000 7f14824b4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:42:28.833 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.831+0000 7f6f47232040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:42:28.874 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.871+0000 7f14824b4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:42:28.881 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.879+0000 7f6f47232040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:42:28.921 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.919+0000 7f14824b4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:42:28.923 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:28.921+0000 7f6f47232040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:42:28.963 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:28.960+0000 7f14824b4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:42:29.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:29.103+0000 7f9d72ffd640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 msgr2=0x7f9d5406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3367593104 2023-10-18T11:42:29.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:29.103+0000 7f9d72ffd640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 0x7f9d5406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T11:42:29.239 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.237+0000 7f6f47232040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:42:29.277 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:29.275+0000 7f14824b4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:42:29.369 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.367+0000 7f6f47232040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:42:29.408 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:29.406+0000 7f14824b4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:42:29.461 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.459+0000 7f6f47232040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:42:29.498 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:29.496+0000 7f14824b4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:42:29.503 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.501+0000 7f6f47232040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:42:29.540 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:29.537+0000 7f14824b4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:42:29.597 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.595+0000 7f6f47232040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:42:29.634 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:29.631+0000 7f14824b4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:42:29.688 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.685+0000 7f6f47232040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:42:29.723 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:29.721+0000 7f14824b4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:42:29.825 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.823+0000 7f6f47232040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:42:29.860 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:29.858+0000 7f14824b4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:42:29.867 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.865+0000 7f6f47232040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:42:29.901 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:29.899+0000 7f14824b4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:42:29.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:29.904+0000 7f9d72ffd640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 msgr2=0x7f9d5406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3367593104 2023-10-18T11:42:29.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:29.904+0000 7f9d72ffd640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 0x7f9d5406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T11:42:29.974 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:29.970+0000 7f6f47232040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:42:30.005 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:30.003+0000 7f14824b4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:42:30.589 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:30.587+0000 7f6f47232040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:42:30.619 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:30.617+0000 7f14824b4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:42:30.849 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:30.846+0000 7f6f47232040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:42:30.878 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:30.876+0000 7f14824b4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:42:30.897 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:30.895+0000 7f6f47232040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:42:30.925 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:30.923+0000 7f14824b4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:42:30.997 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:30.993+0000 7f6f47232040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:42:31.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:31.018+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 103) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f9d5c054800 con 0x7f9d7406da10 2023-10-18T11:42:31.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:31.018+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 msgr2=0x7f9d5406d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:42:31.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:31.018+0000 7f9d6bfff640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/3367593104,v1:172.21.15.139:6833/3367593104] conn(0x7f9d5406afb0 0x7f9d5406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:31.027 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:42:31.025+0000 7f14824b4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:42:31.122 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec6664640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.122 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec6664640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.122 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec6664640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.123 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec6664640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.123 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec6664640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.123 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec361e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.123 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec361e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.124 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec361e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.124 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec361e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:31.124 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:42:31.118+0000 7f6ec361e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:42:32.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.019+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 104) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f9d5c05e0f0 con 0x7f9d7406da10 2023-10-18T11:42:32.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.019+0000 7f9d6bfff640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f9d54035850 0x7f9d540753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:32.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.019+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 --> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d54035c30 con 0x7f9d54035850 2023-10-18T11:42:32.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.020+0000 7f9d72ffd640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f9d54035850 0x7f9d540753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:32.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.023+0000 7f9d72ffd640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f9d54035850 0x7f9d540753e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c035000 tx=0x7f9d6c004ab0 comp rx=0 tx=0).ready entity=mgr.9370 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:32.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.074+0000 7f9d6bfff640 1 -- 172.21.15.125:0/379139037 <== mgr.9370 v2:172.21.15.139:6832/2153770477 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9d54035c30 con 0x7f9d54035850 2023-10-18T11:42:32.076 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-10-18T11:42:32.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f9d54035850 msgr2=0x7f9d540753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:32.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d79a61640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f9d54035850 0x7f9d540753e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c035000 tx=0x7f9d6c004ab0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d7406da10 msgr2=0x7f9d741f0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:32.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d79a61640 1 --2- 172.21.15.125:0/379139037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d7406da10 0x7f9d741f0b60 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c480 tx=0x7f9d5c00c940 comp rx=0 tx=0).stop 2023-10-18T11:42:32.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d737fe640 1 -- 172.21.15.125:0/379139037 reap_dead start 2023-10-18T11:42:32.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 shutdown_connections 2023-10-18T11:42:32.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 >> 172.21.15.125:0/379139037 conn(0x7f9d740af830 msgr2=0x7f9d740afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:32.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 shutdown_connections 2023-10-18T11:42:32.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.076+0000 7f9d79a61640 1 -- 172.21.15.125:0/379139037 wait complete. 2023-10-18T11:42:32.090 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-10-18T11:42:32.090 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1m 2023-10-18T11:42:32.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.194+0000 7f4692100640 1 -- 172.21.15.125:0/1879422523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c13e520 msgr2=0x7f468c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.194+0000 7f4692100640 1 --2- 172.21.15.125:0/1879422523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c13e520 0x7f468c06bb60 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f468c0b20f0 tx=0x7f467c02bcf0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.194+0000 7f4692100640 1 -- 172.21.15.125:0/1879422523 shutdown_connections 2023-10-18T11:42:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.194+0000 7f4692100640 1 --2- 172.21.15.125:0/1879422523 >> v2:172.21.15.139:3300/0 conn(0x7f468c070af0 0x7f468c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.194+0000 7f4692100640 1 --2- 172.21.15.125:0/1879422523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c13e520 0x7f468c06bb60 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.194+0000 7f4692100640 1 -- 172.21.15.125:0/1879422523 >> 172.21.15.125:0/1879422523 conn(0x7f468c069930 msgr2=0x7f468c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.194+0000 7f4692100640 1 -- 172.21.15.125:0/1879422523 shutdown_connections 2023-10-18T11:42:32.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.194+0000 7f4692100640 1 -- 172.21.15.125:0/1879422523 wait complete. 2023-10-18T11:42:32.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.195+0000 7f4692100640 1 Processor -- start 2023-10-18T11:42:32.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.195+0000 7f4692100640 1 -- start start 2023-10-18T11:42:32.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4692100640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f468c070af0 0x7f468c13d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:32.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4692100640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f468c13e520 0x7f468c158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:32.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4692100640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c141690 0x7f468c159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:32.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4692100640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c13b010 con 0x7f468c141690 2023-10-18T11:42:32.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4692100640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c1f1140 con 0x7f468c13e520 2023-10-18T11:42:32.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4692100640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c13dd30 con 0x7f468c070af0 2023-10-18T11:42:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c141690 0x7f468c159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c141690 0x7f468c159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:38600/0 (socket says 172.21.15.125:38600) 2023-10-18T11:42:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 -- 172.21.15.125:0/3826612817 learned_addr learned my addr 172.21.15.125:0/3826612817 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:42:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468b7fe640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f468c070af0 0x7f468c13d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:32.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468affd640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f468c13e520 0x7f468c158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 -- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f468c070af0 msgr2=0x7f468c13d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f468c070af0 0x7f468c13d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 -- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f468c13e520 msgr2=0x7f468c158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f468c13e520 0x7f468c158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 -- 172.21.15.125:0/3826612817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c009490 con 0x7f468c141690 2023-10-18T11:42:32.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468affd640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f468c13e520 0x7f468c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:42:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f468bfff640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c141690 0x7f468c159080 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f468400b970 tx=0x7f468400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4673fff640 1 -- 172.21.15.125:0/3826612817 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468400cb40 con 0x7f468c141690 2023-10-18T11:42:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4673fff640 1 -- 172.21.15.125:0/3826612817 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f468400cdb0 con 0x7f468c141690 2023-10-18T11:42:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.196+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c156820 con 0x7f468c141690 2023-10-18T11:42:32.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.197+0000 7f4673fff640 1 -- 172.21.15.125:0/3826612817 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840146a0 con 0x7f468c141690 2023-10-18T11:42:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.197+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c156de0 con 0x7f468c141690 2023-10-18T11:42:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.198+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468c0690e0 con 0x7f468c141690 2023-10-18T11:42:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.198+0000 7f4673fff640 1 -- 172.21.15.125:0/3826612817 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 104) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f468401d3b0 con 0x7f468c141690 2023-10-18T11:42:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.198+0000 7f4673fff640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f467806ace0 0x7f467806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:42:32.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.198+0000 7f4673fff640 1 -- 172.21.15.125:0/3826612817 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f4684089760 con 0x7f468c141690 2023-10-18T11:42:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.198+0000 7f468b7fe640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f467806ace0 0x7f467806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:42:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.199+0000 7f468b7fe640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f467806ace0 0x7f467806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f467c036000 tx=0x7f467c004730 comp rx=0 tx=0).ready entity=mgr.9370 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:42:32.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.203+0000 7f4673fff640 1 -- 172.21.15.125:0/3826612817 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4684054890 con 0x7f468c141690 2023-10-18T11:42:32.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.390+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 --> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f468c06d160 con 0x7f467806ace0 2023-10-18T11:42:32.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.432+0000 7f4673fff640 1 -- 172.21.15.125:0/3826612817 <== mgr.9370 v2:172.21.15.139:6832/2153770477 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f468c06d160 con 0x7f467806ace0 2023-10-18T11:42:32.436 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-10-18T11:42:32.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f467806ace0 msgr2=0x7f467806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:32.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f467806ace0 0x7f467806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f467c036000 tx=0x7f467c004730 comp rx=0 tx=0).stop 2023-10-18T11:42:32.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c141690 msgr2=0x7f468c159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:42:32.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c141690 0x7f468c159080 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f468400b970 tx=0x7f468400be30 comp rx=0 tx=0).stop 2023-10-18T11:42:32.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 shutdown_connections 2023-10-18T11:42:32.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f467806ace0 0x7f467806d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f468c13e520 0x7f468c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f468c141690 0x7f468c159080 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 --2- 172.21.15.125:0/3826612817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f468c070af0 0x7f468c13d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:42:32.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.434+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 >> 172.21.15.125:0/3826612817 conn(0x7f468c069930 msgr2=0x7f468c0c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:42:32.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.435+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 shutdown_connections 2023-10-18T11:42:32.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:42:32.435+0000 7f4692100640 1 -- 172.21.15.125:0/3826612817 wait complete. 2023-10-18T11:42:32.448 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-10-18T11:42:52.345 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:42:52.349 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:43:00.446 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:43:00.443+0000 7f6eb5602640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:43:00.446 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:43:00.443+0000 7f6eb5602640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:43:00.447 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:43:00.443+0000 7f6eb5602640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:43:00.447 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:43:00.443+0000 7f6eb5602640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:43:00.447 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:43:00.443+0000 7f6eb5602640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:43:22.420 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:43:22.423 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:43:52.493 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:43:52.496 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:44:22.631 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:44:22.634 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:44:52.802 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:44:52.805 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:45:22.949 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:45:22.952 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:45:53.111 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:45:53.115 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:46:23.250 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:46:23.254 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:46:53.455 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:46:53.458 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:47:23.585 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:47:23.588 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:47:53.665 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:47:53.669 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:48:23.787 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:48:23.791 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:48:53.858 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:48:53.862 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:49:23.930 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:49:23.933 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:49:54.177 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:49:54.181 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:50:24.298 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:50:24.302 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:50:54.369 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:50:54.373 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:51:24.558 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:51:24.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:51:54.661 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:51:54.665 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:52:24.789 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:52:24.793 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:52:54.928 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:52:54.931 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:53:15.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:53:15.451 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-18T11:53:15.477 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_43_00_UTC 2023-10-18T11:53:15.477 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_44_00_UTC 2023-10-18T11:53:15.477 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_45_00_UTC 2023-10-18T11:53:15.477 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_46_00_UTC 2023-10-18T11:53:15.478 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_47_00_UTC 2023-10-18T11:53:15.478 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_48_00_UTC 2023-10-18T11:53:15.478 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_49_00_UTC 2023-10-18T11:53:15.478 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_50_00_UTC 2023-10-18T11:53:15.478 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_51_00_UTC 2023-10-18T11:53:15.479 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_52_00_UTC 2023-10-18T11:53:15.479 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_53_00_UTC 2023-10-18T11:53:15.479 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-11_43_00_UTC', 'scheduled-2023-10-18-11_44_00_UTC', 'scheduled-2023-10-18-11_45_00_UTC', 'scheduled-2023-10-18-11_46_00_UTC', 'scheduled-2023-10-18-11_47_00_UTC', 'scheduled-2023-10-18-11_48_00_UTC', 'scheduled-2023-10-18-11_49_00_UTC', 'scheduled-2023-10-18-11_50_00_UTC', 'scheduled-2023-10-18-11_51_00_UTC', 'scheduled-2023-10-18-11_52_00_UTC', 'scheduled-2023-10-18-11_53_00_UTC'] 2023-10-18T11:53:15.480 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-18T11:53:15.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.626+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/301601794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 msgr2=0x7f8bb4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.626+0000 7f8bd1ca9640 1 --2- 172.21.15.125:0/301601794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 0x7f8bb4003d90 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009b50 tx=0x7f8bb40057f0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.626+0000 7f8bcbfff640 1 -- 172.21.15.125:0/301601794 reap_dead start 2023-10-18T11:53:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.626+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/301601794 shutdown_connections 2023-10-18T11:53:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.626+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/301601794 >> 172.21.15.125:0/301601794 conn(0x7f8bcc069930 msgr2=0x7f8bcc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:53:15.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.626+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/301601794 shutdown_connections 2023-10-18T11:53:15.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.626+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/301601794 wait complete. 2023-10-18T11:53:15.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.628+0000 7f8bd1ca9640 1 Processor -- start 2023-10-18T11:53:15.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.628+0000 7f8bd1ca9640 1 -- start start 2023-10-18T11:53:15.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bd1ca9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 0x7f8bcc13d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:53:15.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bd1ca9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc1465b0 0x7f8bcc06bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:53:15.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bd1ca9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bcc15a460 0x7f8bcc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:53:15.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bd1ca9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc13d780 con 0x7f8bb4003970 2023-10-18T11:53:15.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bd1ca9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc14a860 con 0x7f8bcc1465b0 2023-10-18T11:53:15.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bd1ca9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc14ab40 con 0x7f8bcc15a460 2023-10-18T11:53:15.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 0x7f8bcc13d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:53:15.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 0x7f8bcc13d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42330/0 (socket says 172.21.15.125:42330) 2023-10-18T11:53:15.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/4095467894 learned_addr learned my addr 172.21.15.125:0/4095467894 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:53:15.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcaffd640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc1465b0 0x7f8bcc06bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:53:15.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcbfff640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bcc15a460 0x7f8bcc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:53:15.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bcc15a460 msgr2=0x7f8bcc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:15.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bcc15a460 0x7f8bcc06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc1465b0 msgr2=0x7f8bcc06bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:15.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc1465b0 0x7f8bcc06bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 -- 172.21.15.125:0/4095467894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc14ae20 con 0x7f8bb4003970 2023-10-18T11:53:15.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcbfff640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bcc15a460 0x7f8bcc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:53:15.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bcb7fe640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 0x7f8bcc13d240 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009b50 tx=0x7f8bbc033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:53:15.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bb3fff640 1 -- 172.21.15.125:0/4095467894 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc033ea0 con 0x7f8bb4003970 2023-10-18T11:53:15.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bb3fff640 1 -- 172.21.15.125:0/4095467894 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bbc004460 con 0x7f8bb4003970 2023-10-18T11:53:15.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc14b100 con 0x7f8bb4003970 2023-10-18T11:53:15.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bb3fff640 1 -- 172.21.15.125:0/4095467894 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc003850 con 0x7f8bb4003970 2023-10-18T11:53:15.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.629+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc14a4f0 con 0x7f8bb4003970 2023-10-18T11:53:15.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.631+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc0690e0 con 0x7f8bb4003970 2023-10-18T11:53:15.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.631+0000 7f8bb3fff640 1 -- 172.21.15.125:0/4095467894 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 108) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f8bbc0039b0 con 0x7f8bb4003970 2023-10-18T11:53:15.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.631+0000 7f8bb3fff640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f8bb806afe0 0x7f8bb806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:53:15.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.632+0000 7f8bb3fff640 1 -- 172.21.15.125:0/4095467894 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f8bbc0ac810 con 0x7f8bb4003970 2023-10-18T11:53:15.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.632+0000 7f8bcaffd640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f8bb806afe0 0x7f8bb806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:53:15.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.632+0000 7f8bcaffd640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f8bb806afe0 0x7f8bb806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000c00 tx=0x7f8bc00067a0 comp rx=0 tx=0).ready entity=mgr.9370 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:53:15.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.636+0000 7f8bb3fff640 1 -- 172.21.15.125:0/4095467894 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bbc077400 con 0x7f8bb4003970 2023-10-18T11:53:15.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.821+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 --> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bcc13d780 con 0x7f8bb806afe0 2023-10-18T11:53:15.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.829+0000 7f8bb3fff640 1 -- 172.21.15.125:0/4095467894 <== mgr.9370 v2:172.21.15.139:6832/2153770477 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f8bcc13d780 con 0x7f8bb806afe0 2023-10-18T11:53:15.832 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:53:15.832 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T11:42:32", "first": "2023-10-18T11:43:00", "last": "2023-10-18T11:53:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-18T11:53:15.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f8bb806afe0 msgr2=0x7f8bb806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:15.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f8bb806afe0 0x7f8bb806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000c00 tx=0x7f8bc00067a0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 msgr2=0x7f8bcc13d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:15.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 0x7f8bcc13d240 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009b50 tx=0x7f8bbc033c70 comp rx=0 tx=0).stop 2023-10-18T11:53:15.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 shutdown_connections 2023-10-18T11:53:15.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7f8bb806afe0 0x7f8bb806d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bcc1465b0 0x7f8bcc06bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bcc15a460 0x7f8bcc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 --2- 172.21.15.125:0/4095467894 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bb4003970 0x7f8bcc13d240 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 >> 172.21.15.125:0/4095467894 conn(0x7f8bcc069930 msgr2=0x7f8bcc150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:53:15.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 shutdown_connections 2023-10-18T11:53:15.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.831+0000 7f8bd1ca9640 1 -- 172.21.15.125:0/4095467894 wait complete. 2023-10-18T11:53:15.843 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T11:42:32", "first": "2023-10-18T11:43:00", "last": "2023-10-18T11:53:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-18T11:53:15.844 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T11:42:32', 'first': '2023-10-18T11:43:00', 'last': '2023-10-18T11:53:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-10-18T11:53:15.844 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-18T11:53:15.845 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-18T11:53:15.845 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-18T11:53:15.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.945+0000 7fda22664640 1 -- 172.21.15.125:0/1228302064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 msgr2=0x7fda040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:15.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.945+0000 7fda22664640 1 --2- 172.21.15.125:0/1228302064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 0x7fda040047c0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fda10001800 tx=0x7fda040057f0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.945+0000 7fda20c01640 1 -- 172.21.15.125:0/1228302064 reap_dead start 2023-10-18T11:53:15.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.945+0000 7fda22664640 1 -- 172.21.15.125:0/1228302064 shutdown_connections 2023-10-18T11:53:15.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.945+0000 7fda22664640 1 -- 172.21.15.125:0/1228302064 >> 172.21.15.125:0/1228302064 conn(0x7fda1c069930 msgr2=0x7fda1c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:53:15.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.945+0000 7fda22664640 1 -- 172.21.15.125:0/1228302064 shutdown_connections 2023-10-18T11:53:15.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.945+0000 7fda22664640 1 -- 172.21.15.125:0/1228302064 wait complete. 2023-10-18T11:53:15.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.946+0000 7fda22664640 1 Processor -- start 2023-10-18T11:53:15.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda22664640 1 -- start start 2023-10-18T11:53:15.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda22664640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 0x7fda1c139730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:53:15.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda22664640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda1c154b70 0x7fda1c153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:53:15.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda22664640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda1c136150 0x7fda1c136600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:53:15.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda22664640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c136b40 con 0x7fda04004090 2023-10-18T11:53:15.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda22664640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c1416c0 con 0x7fda1c154b70 2023-10-18T11:53:15.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda22664640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c139c70 con 0x7fda1c136150 2023-10-18T11:53:15.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda1bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 0x7fda1c139730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:53:15.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda1bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 0x7fda1c139730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:42342/0 (socket says 172.21.15.125:42342) 2023-10-18T11:53:15.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda1bfff640 1 -- 172.21.15.125:0/2289086580 learned_addr learned my addr 172.21.15.125:0/2289086580 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:53:15.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda20c01640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda1c136150 0x7fda1c136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:53:15.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.947+0000 7fda1b7fe640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda1c154b70 0x7fda1c153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:53:15.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda1bfff640 1 -- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda1c136150 msgr2=0x7fda1c136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:15.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda1bfff640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda1c136150 0x7fda1c136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda1bfff640 1 -- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda1c154b70 msgr2=0x7fda1c153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:15.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda1bfff640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda1c154b70 0x7fda1c153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:15.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda1bfff640 1 -- 172.21.15.125:0/2289086580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c153ad0 con 0x7fda04004090 2023-10-18T11:53:15.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda20c01640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda1c136150 0x7fda1c136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:53:15.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda1bfff640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 0x7fda1c139730 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fda0c006670 tx=0x7fda0c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:53:15.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda18ff9640 1 -- 172.21.15.125:0/2289086580 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00f7e0 con 0x7fda04004090 2023-10-18T11:53:15.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda18ff9640 1 -- 172.21.15.125:0/2289086580 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda0c00f940 con 0x7fda04004090 2023-10-18T11:53:15.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1c153db0 con 0x7fda04004090 2023-10-18T11:53:15.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda18ff9640 1 -- 172.21.15.125:0/2289086580 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00fd50 con 0x7fda04004090 2023-10-18T11:53:15.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.948+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c154370 con 0x7fda04004090 2023-10-18T11:53:15.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.949+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1c0690e0 con 0x7fda04004090 2023-10-18T11:53:15.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.949+0000 7fda18ff9640 1 -- 172.21.15.125:0/2289086580 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 108) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fda0c011430 con 0x7fda04004090 2023-10-18T11:53:15.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.950+0000 7fda18ff9640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7fda0806afe0 0x7fda0806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:53:15.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.950+0000 7fda18ff9640 1 -- 172.21.15.125:0/2289086580 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fda0c08aa00 con 0x7fda04004090 2023-10-18T11:53:15.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.950+0000 7fda1b7fe640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7fda0806afe0 0x7fda0806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:53:15.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.950+0000 7fda1b7fe640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7fda0806afe0 0x7fda0806d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda10034000 tx=0x7fda10039040 comp rx=0 tx=0).ready entity=mgr.9370 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:53:15.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:15.954+0000 7fda18ff9640 1 -- 172.21.15.125:0/2289086580 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda0c055540 con 0x7fda04004090 2023-10-18T11:53:16.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.166+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fda1c155560 con 0x7fda04004090 2023-10-18T11:53:16.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.169+0000 7fda18ff9640 1 -- 172.21.15.125:0/2289086580 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v108) v1 ==== 74+0+171246 (secure 0 0 0) 0x7fda0c015030 con 0x7fda04004090 2023-10-18T11:53:16.173 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:53:16.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7fda0806afe0 msgr2=0x7fda0806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:16.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7fda0806afe0 0x7fda0806d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda10034000 tx=0x7fda10039040 comp rx=0 tx=0).stop 2023-10-18T11:53:16.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 msgr2=0x7fda1c139730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:53:16.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 0x7fda1c139730 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fda0c006670 tx=0x7fda0c003a80 comp rx=0 tx=0).stop 2023-10-18T11:53:16.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 shutdown_connections 2023-10-18T11:53:16.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:6832/2153770477,v1:172.21.15.139:6833/2153770477] conn(0x7fda0806afe0 0x7fda0806d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:16.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda1c154b70 0x7fda1c153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:16.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda1c136150 0x7fda1c136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:16.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 --2- 172.21.15.125:0/2289086580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fda04004090 0x7fda1c139730 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:53:16.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 >> 172.21.15.125:0/2289086580 conn(0x7fda1c069930 msgr2=0x7fda1c137200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:53:16.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 shutdown_connections 2023-10-18T11:53:16.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:53:16.171+0000 7fda22664640 1 -- 172.21.15.125:0/2289086580 wait complete. 2023-10-18T11:53:16.184 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":108,"active_gid":9370,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2153770477},{"type":"v1","addr":"172.21.15.139:6833","nonce":2153770477}]},"active_addr":"172.21.15.139:6833/2153770477","active_change":"2023-10-18T11:42:31.010116+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":11025,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":189,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1293875531}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1126366928}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1872946902}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1253211552}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":608662592}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":141746223}]}]} 2023-10-18T11:53:16.190 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-10-18T11:53:16.191 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-18T11:53:16.191 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-10-18T11:53:16.192 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-18T11:53:16.192 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:53:16.226 INFO:tasks.ceph.mgr.x:Stopped 2023-10-18T11:53:16.226 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-18T11:53:16.269 INFO:tasks.ceph.mgr.x:Started 2023-10-18T11:53:16.686 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:16.683+0000 7f5d87353040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:53:16.731 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:16.728+0000 7f5d87353040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:53:17.117 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.114+0000 7f5d87353040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:53:17.160 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.157+0000 7f5d87353040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:53:17.202 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.199+0000 7f5d87353040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:53:17.252 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.249+0000 7f5d87353040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:53:17.293 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.290+0000 7f5d87353040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:53:17.390 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.387+0000 7f5d87353040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:53:17.442 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.440+0000 7f5d87353040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:53:17.484 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.481+0000 7f5d87353040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:53:17.531 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.528+0000 7f5d87353040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:53:17.573 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.570+0000 7f5d87353040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:53:17.883 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:17.881+0000 7f5d87353040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:53:18.014 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:18.011+0000 7f5d87353040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:53:18.103 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:18.101+0000 7f5d87353040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:53:18.146 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:18.143+0000 7f5d87353040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:53:18.239 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:18.237+0000 7f5d87353040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:53:18.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:18.327+0000 7f5d87353040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:53:18.464 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:18.462+0000 7f5d87353040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:53:18.505 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:18.503+0000 7f5d87353040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:53:18.610 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:18.608+0000 7f5d87353040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:53:19.229 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.227+0000 7f5d87353040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:53:19.491 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.489+0000 7f5d87353040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:53:19.539 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.537+0000 7f5d87353040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:53:19.641 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.638+0000 7f5d87353040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:53:19.790 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.788+0000 7f5d03d8f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.788+0000 7f5d03d8f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.788+0000 7f5d03d8f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.788+0000 7f5d03d8f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.788+0000 7f5d03d8f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.792 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.789+0000 7f5d06dd5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.792 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.789+0000 7f5d06dd5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.792 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.789+0000 7f5d06dd5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.793 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.789+0000 7f5d06dd5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:19.793 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:53:19.789+0000 7f5d06dd5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:53:25.085 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:53:25.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:53:55.213 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:53:55.217 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:54:00.790 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:54:00.788+0000 7f5d0ede5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:54:00.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:54:00.788+0000 7f5d0ede5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:54:00.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:54:00.788+0000 7f5d0ede5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:54:00.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:54:00.788+0000 7f5d0ede5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:54:00.791 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:54:00.788+0000 7f5d0ede5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:54:25.359 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:54:25.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:54:55.534 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:54:55.538 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:55:25.617 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:55:25.620 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:55:55.714 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:55:55.718 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:56:25.791 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:56:25.795 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:56:55.863 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:56:55.867 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:57:25.986 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:57:25.989 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:57:56.199 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:57:56.202 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:58:16.270 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1m 2023-10-18T11:58:16.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.385+0000 7f8ace700640 1 -- 172.21.15.125:0/315027565 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 msgr2=0x7f8ab00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:16.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.385+0000 7f8ace700640 1 --2- 172.21.15.125:0/315027565 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 0x7f8ab00047c0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f8abc006f40 tx=0x7f8ab00057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.385+0000 7f8accc9d640 1 -- 172.21.15.125:0/315027565 reap_dead start 2023-10-18T11:58:16.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.386+0000 7f8ace700640 1 -- 172.21.15.125:0/315027565 shutdown_connections 2023-10-18T11:58:16.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.386+0000 7f8ace700640 1 -- 172.21.15.125:0/315027565 >> 172.21.15.125:0/315027565 conn(0x7f8ac8069930 msgr2=0x7f8ac8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:16.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.386+0000 7f8ace700640 1 -- 172.21.15.125:0/315027565 shutdown_connections 2023-10-18T11:58:16.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.386+0000 7f8ace700640 1 -- 172.21.15.125:0/315027565 wait complete. 2023-10-18T11:58:16.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.387+0000 7f8ace700640 1 Processor -- start 2023-10-18T11:58:16.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.387+0000 7f8ace700640 1 -- start start 2023-10-18T11:58:16.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ace700640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 0x7f8ac8150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:16.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ace700640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac8148940 0x7f8ac8159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:16.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ace700640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac814b0f0 0x7f8ac8148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:16.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ace700640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac814ffe0 con 0x7f8ab0003f40 2023-10-18T11:58:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ace700640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac814f6b0 con 0x7f8ac8148940 2023-10-18T11:58:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ace700640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac814f990 con 0x7f8ac814b0f0 2023-10-18T11:58:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 0x7f8ac8150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 0x7f8ac8150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57844/0 (socket says 172.21.15.125:57844) 2023-10-18T11:58:16.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 -- 172.21.15.125:0/4143081328 learned_addr learned my addr 172.21.15.125:0/4143081328 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:16.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8accc9d640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac814b0f0 0x7f8ac8148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:16.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac77fe640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac8148940 0x7f8ac8159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:16.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 -- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac814b0f0 msgr2=0x7f8ac8148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:16.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac814b0f0 0x7f8ac8148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 -- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac8148940 msgr2=0x7f8ac8159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:16.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac8148940 0x7f8ac8159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 -- 172.21.15.125:0/4143081328 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac814fc70 con 0x7f8ab0003f40 2023-10-18T11:58:16.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac7fff640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 0x7f8ac8150470 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8011a50 tx=0x7f8ab8011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac4ff9640 1 -- 172.21.15.125:0/4143081328 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800fd90 con 0x7f8ab0003f40 2023-10-18T11:58:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ac4ff9640 1 -- 172.21.15.125:0/4143081328 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ab8012450 con 0x7f8ab0003f40 2023-10-18T11:58:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.388+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac81598c0 con 0x7f8ab0003f40 2023-10-18T11:58:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.389+0000 7f8ac4ff9640 1 -- 172.21.15.125:0/4143081328 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8012800 con 0x7f8ab0003f40 2023-10-18T11:58:16.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.389+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac814b750 con 0x7f8ab0003f40 2023-10-18T11:58:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.389+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac80690e0 con 0x7f8ab0003f40 2023-10-18T11:58:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.390+0000 7f8ac4ff9640 1 -- 172.21.15.125:0/4143081328 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 114) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f8ab8012960 con 0x7f8ab0003f40 2023-10-18T11:58:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.390+0000 7f8ac4ff9640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f8ab406afe0 0x7f8ab406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.390+0000 7f8ac4ff9640 1 -- 172.21.15.125:0/4143081328 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f8ab808aa50 con 0x7f8ab0003f40 2023-10-18T11:58:16.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.391+0000 7f8ac77fe640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f8ab406afe0 0x7f8ab406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:16.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.391+0000 7f8ac77fe640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f8ab406afe0 0x7f8ab406d4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8abc034000 tx=0x7f8abc001930 comp rx=0 tx=0).ready entity=mgr.9451 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:16.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.395+0000 7f8ac4ff9640 1 -- 172.21.15.125:0/4143081328 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ab8055690 con 0x7f8ab0003f40 2023-10-18T11:58:16.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.586+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 --> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ac80cc100 con 0x7f8ab406afe0 2023-10-18T11:58:16.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.645+0000 7f8ac4ff9640 1 -- 172.21.15.125:0/4143081328 <== mgr.9451 v2:172.21.15.139:6832/833207676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f8abc0016d0 con 0x7f8ab406afe0 2023-10-18T11:58:16.648 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-18T11:58:16.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f8ab406afe0 msgr2=0x7f8ab406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:16.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f8ab406afe0 0x7f8ab406d4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8abc034000 tx=0x7f8abc001930 comp rx=0 tx=0).stop 2023-10-18T11:58:16.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 msgr2=0x7f8ac8150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:16.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 0x7f8ac8150470 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8011a50 tx=0x7f8ab8011f10 comp rx=0 tx=0).stop 2023-10-18T11:58:16.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 shutdown_connections 2023-10-18T11:58:16.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac8148940 0x7f8ac8159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f8ab406afe0 0x7f8ab406d4a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac814b0f0 0x7f8ac8148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 --2- 172.21.15.125:0/4143081328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ab0003f40 0x7f8ac8150470 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 >> 172.21.15.125:0/4143081328 conn(0x7f8ac8069930 msgr2=0x7f8ac80ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:16.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 shutdown_connections 2023-10-18T11:58:16.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.647+0000 7f8ace700640 1 -- 172.21.15.125:0/4143081328 wait complete. 2023-10-18T11:58:16.660 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-18T11:58:16.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:16.661 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-18T11:58:16.680 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_43_00_UTC 2023-10-18T11:58:16.680 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_44_00_UTC 2023-10-18T11:58:16.681 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_45_00_UTC 2023-10-18T11:58:16.681 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_46_00_UTC 2023-10-18T11:58:16.681 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_47_00_UTC 2023-10-18T11:58:16.681 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_48_00_UTC 2023-10-18T11:58:16.681 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_49_00_UTC 2023-10-18T11:58:16.682 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_50_00_UTC 2023-10-18T11:58:16.682 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_51_00_UTC 2023-10-18T11:58:16.682 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_52_00_UTC 2023-10-18T11:58:16.682 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_53_00_UTC 2023-10-18T11:58:16.682 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_54_00_UTC 2023-10-18T11:58:16.683 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_55_00_UTC 2023-10-18T11:58:16.683 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_56_00_UTC 2023-10-18T11:58:16.683 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_57_00_UTC 2023-10-18T11:58:16.683 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_58_00_UTC 2023-10-18T11:58:16.683 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-18-11_43_00_UTC', 'scheduled-2023-10-18-11_44_00_UTC', 'scheduled-2023-10-18-11_45_00_UTC', 'scheduled-2023-10-18-11_46_00_UTC', 'scheduled-2023-10-18-11_47_00_UTC', 'scheduled-2023-10-18-11_48_00_UTC', 'scheduled-2023-10-18-11_49_00_UTC', 'scheduled-2023-10-18-11_50_00_UTC', 'scheduled-2023-10-18-11_51_00_UTC', 'scheduled-2023-10-18-11_52_00_UTC', 'scheduled-2023-10-18-11_53_00_UTC', 'scheduled-2023-10-18-11_54_00_UTC', 'scheduled-2023-10-18-11_55_00_UTC', 'scheduled-2023-10-18-11_56_00_UTC', 'scheduled-2023-10-18-11_57_00_UTC', 'scheduled-2023-10-18-11_58_00_UTC'] 2023-10-18T11:58:16.684 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-18T11:58:16.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.822+0000 7f6772410640 1 -- 172.21.15.125:0/600576206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c0cf340 msgr2=0x7f676c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:16.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.822+0000 7f6772410640 1 --2- 172.21.15.125:0/600576206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c0cf340 0x7f676c0d1910 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f6754005fd0 tx=0x7f675402bcf0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.822+0000 7f6772410640 1 -- 172.21.15.125:0/600576206 shutdown_connections 2023-10-18T11:58:16.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.822+0000 7f6772410640 1 --2- 172.21.15.125:0/600576206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676c15b2a0 0x7f676c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.822+0000 7f6772410640 1 --2- 172.21.15.125:0/600576206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c0cf340 0x7f676c0d1910 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.822+0000 7f6772410640 1 -- 172.21.15.125:0/600576206 >> 172.21.15.125:0/600576206 conn(0x7f676c0d0d60 msgr2=0x7f676c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:16.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.822+0000 7f6772410640 1 -- 172.21.15.125:0/600576206 shutdown_connections 2023-10-18T11:58:16.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.822+0000 7f6772410640 1 -- 172.21.15.125:0/600576206 wait complete. 2023-10-18T11:58:16.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.823+0000 7f6772410640 1 Processor -- start 2023-10-18T11:58:16.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.823+0000 7f6772410640 1 -- start start 2023-10-18T11:58:16.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f6772410640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676c0ceb40 0x7f676c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:16.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f6772410640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f676c0cf340 0x7f676c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:16.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f6772410640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c15b2a0 0x7f676c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:16.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f6772410640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c157bc0 con 0x7f676c15b2a0 2023-10-18T11:58:16.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f6772410640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c15dfa0 con 0x7f676c0cf340 2023-10-18T11:58:16.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f6772410640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c15d670 con 0x7f676c0ceb40 2023-10-18T11:58:16.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f67709ad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c15b2a0 0x7f676c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:16.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f67709ad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c15b2a0 0x7f676c157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57872/0 (socket says 172.21.15.125:57872) 2023-10-18T11:58:16.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.824+0000 7f67709ad640 1 -- 172.21.15.125:0/1996082676 learned_addr learned my addr 172.21.15.125:0/1996082676 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:16.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f67709ad640 1 -- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676c0ceb40 msgr2=0x7f676c15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:16.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f67709ad640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676c0ceb40 0x7f676c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f67709ad640 1 -- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f676c0cf340 msgr2=0x7f676c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:16.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f67709ad640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f676c0cf340 0x7f676c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:16.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f67709ad640 1 -- 172.21.15.125:0/1996082676 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754009560 con 0x7f676c15b2a0 2023-10-18T11:58:16.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f67709ad640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c15b2a0 0x7f676c157680 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f676400c780 tx=0x7f676400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:16.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f6768ff9640 1 -- 172.21.15.125:0/1996082676 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764003d80 con 0x7f676c15b2a0 2023-10-18T11:58:16.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f6768ff9640 1 -- 172.21.15.125:0/1996082676 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f676400ce90 con 0x7f676c15b2a0 2023-10-18T11:58:16.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f6768ff9640 1 -- 172.21.15.125:0/1996082676 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67640052d0 con 0x7f676c15b2a0 2023-10-18T11:58:16.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c15d950 con 0x7f676c15b2a0 2023-10-18T11:58:16.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.825+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c15cb40 con 0x7f676c15b2a0 2023-10-18T11:58:16.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.826+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c0690e0 con 0x7f676c15b2a0 2023-10-18T11:58:16.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.826+0000 7f6768ff9640 1 -- 172.21.15.125:0/1996082676 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 114) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f6764005430 con 0x7f676c15b2a0 2023-10-18T11:58:16.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.827+0000 7f6768ff9640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f674c06afb0 0x7f674c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:16.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.827+0000 7f6768ff9640 1 -- 172.21.15.125:0/1996082676 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f67640892b0 con 0x7f676c15b2a0 2023-10-18T11:58:16.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.827+0000 7f676bfff640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f674c06afb0 0x7f674c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:16.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.827+0000 7f676bfff640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f674c06afb0 0x7f674c06d470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6754035000 tx=0x7f67540046c0 comp rx=0 tx=0).ready entity=mgr.9451 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:16.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:16.832+0000 7f6768ff9640 1 -- 172.21.15.125:0/1996082676 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6764053ed0 con 0x7f676c15b2a0 2023-10-18T11:58:17.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.016+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 --> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f676c15e110 con 0x7f674c06afb0 2023-10-18T11:58:17.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.024+0000 7f6768ff9640 1 -- 172.21.15.125:0/1996082676 <== mgr.9451 v2:172.21.15.139:6832/833207676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f676c15e110 con 0x7f674c06afb0 2023-10-18T11:58:17.027 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:17.027 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T11:42:32", "first": "2023-10-18T11:43:00", "last": "2023-10-18T11:58:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-18T11:58:17.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f674c06afb0 msgr2=0x7f674c06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:17.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f674c06afb0 0x7f674c06d470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6754035000 tx=0x7f67540046c0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c15b2a0 msgr2=0x7f676c157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:17.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c15b2a0 0x7f676c157680 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f676400c780 tx=0x7f676400cc40 comp rx=0 tx=0).stop 2023-10-18T11:58:17.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 shutdown_connections 2023-10-18T11:58:17.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f676c0cf340 0x7f676c0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f674c06afb0 0x7f674c06d470 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f676c15b2a0 0x7f676c157680 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 --2- 172.21.15.125:0/1996082676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676c0ceb40 0x7f676c15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 >> 172.21.15.125:0/1996082676 conn(0x7f676c0d0d60 msgr2=0x7f676c0cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:17.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 shutdown_connections 2023-10-18T11:58:17.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.026+0000 7f6772410640 1 -- 172.21.15.125:0/1996082676 wait complete. 2023-10-18T11:58:17.039 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T11:42:32", "first": "2023-10-18T11:43:00", "last": "2023-10-18T11:58:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-18T11:58:17.039 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-18T00:00:00', 'created': '2023-10-18T11:42:32', 'first': '2023-10-18T11:43:00', 'last': '2023-10-18T11:58:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-10-18T11:58:17.039 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-10-18T11:58:17.040 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-10-18T11:58:17.040 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1m 2023-10-18T11:58:17.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.143+0000 7f749e621640 1 -- 172.21.15.125:0/3156735957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7480003970 msgr2=0x7f7480003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:17.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.143+0000 7f749e621640 1 --2- 172.21.15.125:0/3156735957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7480003970 0x7f7480003d90 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f7488009b50 tx=0x7f74800057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.144+0000 7f749e621640 1 -- 172.21.15.125:0/3156735957 shutdown_connections 2023-10-18T11:58:17.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.144+0000 7f749e621640 1 --2- 172.21.15.125:0/3156735957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7480003970 0x7f7480003d90 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.144+0000 7f749e621640 1 -- 172.21.15.125:0/3156735957 >> 172.21.15.125:0/3156735957 conn(0x7f7498069930 msgr2=0x7f7498069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:17.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.144+0000 7f749e621640 1 -- 172.21.15.125:0/3156735957 shutdown_connections 2023-10-18T11:58:17.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.144+0000 7f749e621640 1 -- 172.21.15.125:0/3156735957 wait complete. 2023-10-18T11:58:17.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.145+0000 7f749e621640 1 Processor -- start 2023-10-18T11:58:17.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.145+0000 7f749e621640 1 -- start start 2023-10-18T11:58:17.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749e621640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7480003970 0x7f749814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:17.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749e621640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f749813b270 0x7f74980aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:17.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749e621640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7498133ca0 0x7f7498134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:17.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749e621640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749814ddc0 con 0x7f749813b270 2023-10-18T11:58:17.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749e621640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7498134660 con 0x7f7480003970 2023-10-18T11:58:17.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749e621640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749814d520 con 0x7f7498133ca0 2023-10-18T11:58:17.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f749813b270 0x7f74980aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:17.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f749813b270 0x7f74980aab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57888/0 (socket says 172.21.15.125:57888) 2023-10-18T11:58:17.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 -- 172.21.15.125:0/1206827860 learned_addr learned my addr 172.21.15.125:0/1206827860 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:17.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f7497fff640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7480003970 0x7f749814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:17.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749cbbe640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7498133ca0 0x7f7498134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:17.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 -- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7498133ca0 msgr2=0x7f7498134120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:17.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7498133ca0 0x7f7498134120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 -- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7480003970 msgr2=0x7f749814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:17.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7480003970 0x7f749814cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 -- 172.21.15.125:0/1206827860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749814d800 con 0x7f749813b270 2023-10-18T11:58:17.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f74977fe640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f749813b270 0x7f74980aab30 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f748c004d70 tx=0x7f748c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:17.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f7494ff9640 1 -- 172.21.15.125:0/1206827860 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0105d0 con 0x7f749813b270 2023-10-18T11:58:17.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f7494ff9640 1 -- 172.21.15.125:0/1206827860 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f748c010820 con 0x7f749813b270 2023-10-18T11:58:17.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f7494ff9640 1 -- 172.21.15.125:0/1206827860 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c010c30 con 0x7f749813b270 2023-10-18T11:58:17.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749814dae0 con 0x7f749813b270 2023-10-18T11:58:17.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.146+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749814d110 con 0x7f749813b270 2023-10-18T11:58:17.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.147+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749815c0d0 con 0x7f749813b270 2023-10-18T11:58:17.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.148+0000 7f7494ff9640 1 -- 172.21.15.125:0/1206827860 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 114) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f748c01b460 con 0x7f749813b270 2023-10-18T11:58:17.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.148+0000 7f7494ff9640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f748406afb0 0x7f748406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:17.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.148+0000 7f7494ff9640 1 -- 172.21.15.125:0/1206827860 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f748c08a590 con 0x7f749813b270 2023-10-18T11:58:17.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.149+0000 7f7497fff640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f748406afb0 0x7f748406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:17.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.149+0000 7f7497fff640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f748406afb0 0x7f748406d470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7488036000 tx=0x7f7488035040 comp rx=0 tx=0).ready entity=mgr.9451 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:17.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.152+0000 7f7494ff9640 1 -- 172.21.15.125:0/1206827860 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f748c0550a0 con 0x7f749813b270 2023-10-18T11:58:17.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.337+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 --> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f749814d2a0 con 0x7f748406afb0 2023-10-18T11:58:17.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.404+0000 7f7494ff9640 1 -- 172.21.15.125:0/1206827860 <== mgr.9451 v2:172.21.15.139:6832/833207676 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f749814d2a0 con 0x7f748406afb0 2023-10-18T11:58:17.407 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-10-18T11:58:17.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f748406afb0 msgr2=0x7f748406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:17.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f748406afb0 0x7f748406d470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7488036000 tx=0x7f7488035040 comp rx=0 tx=0).stop 2023-10-18T11:58:17.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f749813b270 msgr2=0x7f74980aab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:17.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f749813b270 0x7f74980aab30 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f748c004d70 tx=0x7f748c001630 comp rx=0 tx=0).stop 2023-10-18T11:58:17.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 shutdown_connections 2023-10-18T11:58:17.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f748406afb0 0x7f748406d470 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7498133ca0 0x7f7498134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f749813b270 0x7f74980aab30 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 --2- 172.21.15.125:0/1206827860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7480003970 0x7f749814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:17.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 >> 172.21.15.125:0/1206827860 conn(0x7f7498069930 msgr2=0x7f74980ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:17.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 shutdown_connections 2023-10-18T11:58:17.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:17.406+0000 7f749e621640 1 -- 172.21.15.125:0/1206827860 wait complete. 2023-10-18T11:58:17.418 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-10-18T11:58:17.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.419 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-18T11:58:17.439 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_43_00_UTC 2023-10-18T11:58:17.439 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_44_00_UTC 2023-10-18T11:58:17.439 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_45_00_UTC 2023-10-18T11:58:17.439 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_46_00_UTC 2023-10-18T11:58:17.440 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_47_00_UTC 2023-10-18T11:58:17.440 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_48_00_UTC 2023-10-18T11:58:17.440 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_49_00_UTC 2023-10-18T11:58:17.440 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_50_00_UTC 2023-10-18T11:58:17.440 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_51_00_UTC 2023-10-18T11:58:17.440 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_52_00_UTC 2023-10-18T11:58:17.441 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_53_00_UTC 2023-10-18T11:58:17.441 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_54_00_UTC 2023-10-18T11:58:17.441 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_55_00_UTC 2023-10-18T11:58:17.441 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_56_00_UTC 2023-10-18T11:58:17.441 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_57_00_UTC 2023-10-18T11:58:17.442 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_58_00_UTC 2023-10-18T11:58:17.442 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_43_00_UTC 2023-10-18T11:58:17.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.443 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_43_00_UTC) 2023-10-18T11:58:17.501 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_44_00_UTC 2023-10-18T11:58:17.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.502 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_44_00_UTC) 2023-10-18T11:58:17.562 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_45_00_UTC 2023-10-18T11:58:17.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.563 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_45_00_UTC) 2023-10-18T11:58:17.625 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_46_00_UTC 2023-10-18T11:58:17.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.626 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_46_00_UTC) 2023-10-18T11:58:17.687 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_47_00_UTC 2023-10-18T11:58:17.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.688 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_47_00_UTC) 2023-10-18T11:58:17.749 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_48_00_UTC 2023-10-18T11:58:17.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.750 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_48_00_UTC) 2023-10-18T11:58:17.810 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_49_00_UTC 2023-10-18T11:58:17.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.811 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_49_00_UTC) 2023-10-18T11:58:17.871 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_50_00_UTC 2023-10-18T11:58:17.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.872 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_50_00_UTC) 2023-10-18T11:58:17.932 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_51_00_UTC 2023-10-18T11:58:17.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.933 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_51_00_UTC) 2023-10-18T11:58:17.994 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_52_00_UTC 2023-10-18T11:58:17.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:17.995 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_52_00_UTC) 2023-10-18T11:58:18.055 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_53_00_UTC 2023-10-18T11:58:18.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:18.056 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_53_00_UTC) 2023-10-18T11:58:18.116 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_54_00_UTC 2023-10-18T11:58:18.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:18.117 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_54_00_UTC) 2023-10-18T11:58:18.179 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_55_00_UTC 2023-10-18T11:58:18.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:18.180 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_55_00_UTC) 2023-10-18T11:58:18.240 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_56_00_UTC 2023-10-18T11:58:18.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:18.241 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_56_00_UTC) 2023-10-18T11:58:18.302 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_57_00_UTC 2023-10-18T11:58:18.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:18.303 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_57_00_UTC) 2023-10-18T11:58:18.363 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_58_00_UTC 2023-10-18T11:58:18.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:18.364 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-18-11_58_00_UTC) 2023-10-18T11:58:18.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:18.425 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-10-18T11:58:18.483 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T11:58:18.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 -- 172.21.15.125:0/300135293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380cbd20 msgr2=0x7fd338158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:18.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 --2- 172.21.15.125:0/300135293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380cbd20 0x7fd338158300 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fd3380cc490 tx=0x7fd33402c250 comp rx=0 tx=0).stop 2023-10-18T11:58:18.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 -- 172.21.15.125:0/300135293 shutdown_connections 2023-10-18T11:58:18.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 --2- 172.21.15.125:0/300135293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3380b0530 0x7fd3380b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:18.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 --2- 172.21.15.125:0/300135293 >> v2:172.21.15.139:3300/0 conn(0x7fd3380afa20 0x7fd3380705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:18.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 --2- 172.21.15.125:0/300135293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380cbd20 0x7fd338158300 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:18.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 -- 172.21.15.125:0/300135293 >> 172.21.15.125:0/300135293 conn(0x7fd338069930 msgr2=0x7fd338069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:18.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 -- 172.21.15.125:0/300135293 shutdown_connections 2023-10-18T11:58:18.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.625+0000 7fd340573640 1 -- 172.21.15.125:0/300135293 wait complete. 2023-10-18T11:58:18.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.626+0000 7fd340573640 1 Processor -- start 2023-10-18T11:58:18.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.626+0000 7fd340573640 1 -- start start 2023-10-18T11:58:18.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd340573640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3380afa20 0x7fd338147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:18.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd340573640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380b0530 0x7fd338144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:18.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd340573640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3380cbd20 0x7fd338165c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:18.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd340573640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd338146590 con 0x7fd3380b0530 2023-10-18T11:58:18.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd340573640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3381467a0 con 0x7fd3380afa20 2023-10-18T11:58:18.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd340573640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd338146a80 con 0x7fd3380cbd20 2023-10-18T11:58:18.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380b0530 0x7fd338144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:18.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380b0530 0x7fd338144230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39740/0 (socket says 172.21.15.125:39740) 2023-10-18T11:58:18.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 -- 172.21.15.125:0/2653718065 learned_addr learned my addr 172.21.15.125:0/2653718065 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:18.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33e30f640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3380afa20 0x7fd338147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:18.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33eb10640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3380cbd20 0x7fd338165c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:18.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 -- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3380cbd20 msgr2=0x7fd338165c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:18.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3380cbd20 0x7fd338165c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:18.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 -- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3380afa20 msgr2=0x7fd338147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:18.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3380afa20 0x7fd338147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:18.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 -- 172.21.15.125:0/2653718065 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd334009900 con 0x7fd3380b0530 2023-10-18T11:58:18.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33e30f640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3380afa20 0x7fd338147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd33db0e640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380b0530 0x7fd338144230 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fd32800d770 tx=0x7fd32800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd32effd640 1 -- 172.21.15.125:0/2653718065 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328009c30 con 0x7fd3380b0530 2023-10-18T11:58:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.627+0000 7fd32effd640 1 -- 172.21.15.125:0/2653718065 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd328009e80 con 0x7fd3380b0530 2023-10-18T11:58:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.628+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd338146d60 con 0x7fd3380b0530 2023-10-18T11:58:18.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.628+0000 7fd32effd640 1 -- 172.21.15.125:0/2653718065 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280106a0 con 0x7fd3380b0530 2023-10-18T11:58:18.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.628+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd338166170 con 0x7fd3380b0530 2023-10-18T11:58:18.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.628+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3380c96b0 con 0x7fd3380b0530 2023-10-18T11:58:18.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.629+0000 7fd32effd640 1 -- 172.21.15.125:0/2653718065 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 114) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fd328010800 con 0x7fd3380b0530 2023-10-18T11:58:18.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.629+0000 7fd32effd640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fd31c06afe0 0x7fd31c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:18.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.629+0000 7fd32effd640 1 -- 172.21.15.125:0/2653718065 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6745+0+0 (secure 0 0 0) 0x7fd32808a160 con 0x7fd3380b0530 2023-10-18T11:58:18.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.630+0000 7fd33e30f640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fd31c06afe0 0x7fd31c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:18.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.630+0000 7fd33e30f640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fd31c06afe0 0x7fd31c06d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd334036000 tx=0x7fd334004ab0 comp rx=0 tx=0).ready entity=mgr.9451 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:18.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.633+0000 7fd32effd640 1 -- 172.21.15.125:0/2653718065 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd328058700 con 0x7fd3380b0530 2023-10-18T11:58:18.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:18.843+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd3380690e0 con 0x7fd3380b0530 2023-10-18T11:58:19.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.419+0000 7fd32effd640 1 -- 172.21.15.125:0/2653718065 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v115) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd328058290 con 0x7fd3380b0530 2023-10-18T11:58:19.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.421+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fd31c06afe0 msgr2=0x7fd31c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:19.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.421+0000 7fd340573640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fd31c06afe0 0x7fd31c06d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd334036000 tx=0x7fd334004ab0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.421+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380b0530 msgr2=0x7fd338144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:19.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.421+0000 7fd340573640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380b0530 0x7fd338144230 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fd32800d770 tx=0x7fd32800dc30 comp rx=0 tx=0).stop 2023-10-18T11:58:19.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.422+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 shutdown_connections 2023-10-18T11:58:19.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.422+0000 7fd340573640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fd31c06afe0 0x7fd31c06d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.422+0000 7fd340573640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3380cbd20 0x7fd338165c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.422+0000 7fd340573640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3380b0530 0x7fd338144230 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.422+0000 7fd340573640 1 --2- 172.21.15.125:0/2653718065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3380afa20 0x7fd338147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.422+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 >> 172.21.15.125:0/2653718065 conn(0x7fd338069930 msgr2=0x7fd3380ac600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:19.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.422+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 shutdown_connections 2023-10-18T11:58:19.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.422+0000 7fd340573640 1 -- 172.21.15.125:0/2653718065 wait complete. 2023-10-18T11:58:19.439 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T11:58:19.440 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:58:19.491 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:58:19.531 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T11:58:19.532 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:58:19.532 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:58:19.548 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:19.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.690+0000 7f44f9dd7640 1 -- 172.21.15.125:0/1804372780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 msgr2=0x7f44f40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:19.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.690+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/1804372780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 0x7f44f40b0650 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f44f408a120 tx=0x7f44ec02c260 comp rx=0 tx=0).stop 2023-10-18T11:58:19.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.690+0000 7f44f9dd7640 1 -- 172.21.15.125:0/1804372780 shutdown_connections 2023-10-18T11:58:19.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.690+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/1804372780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 0x7f44f40b0650 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.690+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/1804372780 >> v2:172.21.15.139:3300/0 conn(0x7f44f415b2a0 0x7f44f40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.690+0000 7f44f9dd7640 1 -- 172.21.15.125:0/1804372780 >> 172.21.15.125:0/1804372780 conn(0x7f44f40d0d60 msgr2=0x7f44f40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:19.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.690+0000 7f44f9dd7640 1 -- 172.21.15.125:0/1804372780 shutdown_connections 2023-10-18T11:58:19.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.690+0000 7f44f9dd7640 1 -- 172.21.15.125:0/1804372780 wait complete. 2023-10-18T11:58:19.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.691+0000 7f44f9dd7640 1 Processor -- start 2023-10-18T11:58:19.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f9dd7640 1 -- start start 2023-10-18T11:58:19.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f9dd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 0x7f44f41e58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:19.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f9dd7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44f40cf340 0x7f44f40ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:19.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f9dd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44f415b2a0 0x7f44f40ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:19.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f9dd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f415b6f0 con 0x7f44f40ceb40 2023-10-18T11:58:19.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f9dd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40caca0 con 0x7f44f415b2a0 2023-10-18T11:58:19.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f9dd7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f41e6500 con 0x7f44f40cf340 2023-10-18T11:58:19.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 0x7f44f41e58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:19.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 0x7f44f41e58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39766/0 (socket says 172.21.15.125:39766) 2023-10-18T11:58:19.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f37fe640 1 -- 172.21.15.125:0/2007131446 learned_addr learned my addr 172.21.15.125:0/2007131446 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:19.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44eaffd640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44f40cf340 0x7f44f40ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:19.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.692+0000 7f44f3fff640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44f415b2a0 0x7f44f40ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:19.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f37fe640 1 -- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44f40cf340 msgr2=0x7f44f40ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:19.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f37fe640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44f40cf340 0x7f44f40ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f37fe640 1 -- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44f415b2a0 msgr2=0x7f44f40ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:19.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f37fe640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44f415b2a0 0x7f44f40ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f37fe640 1 -- 172.21.15.125:0/2007131446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec009900 con 0x7f44f40ceb40 2023-10-18T11:58:19.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f37fe640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 0x7f44f41e58f0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c430 tx=0x7f44e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:19.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f0ff9640 1 -- 172.21.15.125:0/2007131446 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40052d0 con 0x7f44f40ceb40 2023-10-18T11:58:19.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f0ff9640 1 -- 172.21.15.125:0/2007131446 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f44e4007020 con 0x7f44f40ceb40 2023-10-18T11:58:19.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f41e4280 con 0x7f44f40ceb40 2023-10-18T11:58:19.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f0ff9640 1 -- 172.21.15.125:0/2007131446 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4007430 con 0x7f44f40ceb40 2023-10-18T11:58:19.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.693+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f41e4840 con 0x7f44f40ceb40 2023-10-18T11:58:19.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.694+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f415b830 con 0x7f44f40ceb40 2023-10-18T11:58:19.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.695+0000 7f44f0ff9640 1 -- 172.21.15.125:0/2007131446 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f44e40067d0 con 0x7f44f40ceb40 2023-10-18T11:58:19.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.695+0000 7f44f0ff9640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f44d406af60 0x7f44d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:19.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.695+0000 7f44eaffd640 1 -- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f44d406af60 msgr2=0x7f44d406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:19.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.695+0000 7f44f0ff9640 1 -- 172.21.15.125:0/2007131446 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f44e40890f0 con 0x7f44f40ceb40 2023-10-18T11:58:19.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.695+0000 7f44eaffd640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f44d406af60 0x7f44d406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:19.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.700+0000 7f44f0ff9640 1 -- 172.21.15.125:0/2007131446 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44f415b830 con 0x7f44f40ceb40 2023-10-18T11:58:19.812 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:19.810+0000 7fb587212040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:58:19.818 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:19.815+0000 7ff111a3a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:58:19.857 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:19.855+0000 7fb587212040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:58:19.865 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:19.863+0000 7ff111a3a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:58:19.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.876+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44f415b830 con 0x7f44f40ceb40 2023-10-18T11:58:19.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.877+0000 7f44f0ff9640 1 -- 172.21.15.125:0/2007131446 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+17822 (secure 0 0 0) 0x7f44f415b830 con 0x7f44f40ceb40 2023-10-18T11:58:19.880 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:19.880 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":190,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:53:19.652742+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:42:12.758545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-18T11:42:13.894048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6833/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:6832/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1872946902":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1126366928":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1293875531":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/608662592":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/141746223":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1253211552":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/3751642323":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/402965407":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6833/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6832/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/4091087182":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/697097533":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/4016867169":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/878133557":"2023-10-19T11:42:31.009948+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:19.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f44d406af60 msgr2=0x7f44d406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:19.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f44d406af60 0x7f44d406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 msgr2=0x7f44f41e58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:19.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 0x7f44f41e58f0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c430 tx=0x7f44e400c8f0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 shutdown_connections 2023-10-18T11:58:19.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44f415b2a0 0x7f44f40ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f44d406af60 0x7f44d406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44f40cf340 0x7f44f40ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 --2- 172.21.15.125:0/2007131446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44f40ceb40 0x7f44f41e58f0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:19.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 >> 172.21.15.125:0/2007131446 conn(0x7f44f40d0d60 msgr2=0x7f44f406b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:19.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 shutdown_connections 2023-10-18T11:58:19.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.879+0000 7f44f9dd7640 1 -- 172.21.15.125:0/2007131446 wait complete. 2023-10-18T11:58:19.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T11:42:12.758545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T11:42:13.894048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:58:19.894 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:58:20.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.998+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3265564459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac15ccf0 msgr2=0x7f32ac15d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.998+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3265564459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac15ccf0 0x7f32ac15d160 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f32a4005fd0 tx=0x7f32a402c050 comp rx=0 tx=0).stop 2023-10-18T11:58:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.998+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3265564459 shutdown_connections 2023-10-18T11:58:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.998+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3265564459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac15ccf0 0x7f32ac15d160 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.998+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3265564459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32ac0ce240 0x7f32ac06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.998+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3265564459 >> 172.21.15.125:0/3265564459 conn(0x7f32ac0b0670 msgr2=0x7f32ac0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:20.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.998+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3265564459 shutdown_connections 2023-10-18T11:58:20.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.998+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3265564459 wait complete. 2023-10-18T11:58:20.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.999+0000 7f32b3cb0640 1 Processor -- start 2023-10-18T11:58:20.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:19.999+0000 7f32b3cb0640 1 -- start start 2023-10-18T11:58:20.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b3cb0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32ac0ce240 0x7f32ac0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b3cb0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac159060 0x7f32ac14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b3cb0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32ac15ccf0 0x7f32ac1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b3cb0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0c7da0 con 0x7f32ac159060 2023-10-18T11:58:20.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b3cb0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0b1c70 con 0x7f32ac15ccf0 2023-10-18T11:58:20.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b3cb0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0c7190 con 0x7f32ac0ce240 2023-10-18T11:58:20.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b124b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac159060 0x7f32ac14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b124b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac159060 0x7f32ac14d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39790/0 (socket says 172.21.15.125:39790) 2023-10-18T11:58:20.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b124b640 1 -- 172.21.15.125:0/3896594540 learned_addr learned my addr 172.21.15.125:0/3896594540 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:20.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b1a4c640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32ac0ce240 0x7f32ac0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b224d640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32ac15ccf0 0x7f32ac1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.000+0000 7f32b124b640 1 -- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32ac0ce240 msgr2=0x7f32ac0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f32b124b640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32ac0ce240 0x7f32ac0b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f32b124b640 1 -- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32ac15ccf0 msgr2=0x7f32ac1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f32b124b640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32ac15ccf0 0x7f32ac1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f32b124b640 1 -- 172.21.15.125:0/3896594540 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4009840 con 0x7f32ac159060 2023-10-18T11:58:20.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f32b124b640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac159060 0x7f32ac14d130 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d950 tx=0x7f32a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:20.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f329e7fc640 1 -- 172.21.15.125:0/3896594540 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8009c30 con 0x7f32ac159060 2023-10-18T11:58:20.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f329e7fc640 1 -- 172.21.15.125:0/3896594540 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f32a8009e80 con 0x7f32ac159060 2023-10-18T11:58:20.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f329e7fc640 1 -- 172.21.15.125:0/3896594540 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80106c0 con 0x7f32ac159060 2023-10-18T11:58:20.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac0c7470 con 0x7f32ac159060 2023-10-18T11:58:20.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.001+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac0c7a30 con 0x7f32ac159060 2023-10-18T11:58:20.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.002+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac0c7f50 con 0x7f32ac159060 2023-10-18T11:58:20.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.003+0000 7f329e7fc640 1 -- 172.21.15.125:0/3896594540 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f32a8010820 con 0x7f32ac159060 2023-10-18T11:58:20.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.003+0000 7f329e7fc640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f329006af90 0x7f329006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.003+0000 7f32b1a4c640 1 -- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f329006af90 msgr2=0x7f329006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:20.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.003+0000 7f329e7fc640 1 -- 172.21.15.125:0/3896594540 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f32a808a140 con 0x7f32ac159060 2023-10-18T11:58:20.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.003+0000 7f32b1a4c640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f329006af90 0x7f329006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:20.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.008+0000 7f329e7fc640 1 -- 172.21.15.125:0/3896594540 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32ac0c7f50 con 0x7f32ac159060 2023-10-18T11:58:20.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.181+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32ac0675f0 con 0x7f32ac159060 2023-10-18T11:58:20.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.181+0000 7f329e7fc640 1 -- 172.21.15.125:0/3896594540 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32a8054bf0 con 0x7f32ac159060 2023-10-18T11:58:20.184 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:58:20.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f329006af90 msgr2=0x7f329006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:20.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f329006af90 0x7f329006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac159060 msgr2=0x7f32ac14d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac159060 0x7f32ac14d130 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d950 tx=0x7f32a800de10 comp rx=0 tx=0).stop 2023-10-18T11:58:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 shutdown_connections 2023-10-18T11:58:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32ac15ccf0 0x7f32ac1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f329006af90 0x7f329006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f32ac159060 0x7f32ac14d130 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 --2- 172.21.15.125:0/3896594540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32ac0ce240 0x7f32ac0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 >> 172.21.15.125:0/3896594540 conn(0x7f32ac0b0670 msgr2=0x7f32ac0ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:20.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 shutdown_connections 2023-10-18T11:58:20.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.183+0000 7f32b3cb0640 1 -- 172.21.15.125:0/3896594540 wait complete. 2023-10-18T11:58:20.197 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:58:20.245 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.242+0000 7fb587212040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:58:20.275 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.273+0000 7ff111a3a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:58:20.288 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.286+0000 7fb587212040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:58:20.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.302+0000 7feec087c640 1 -- 172.21.15.125:0/3257425503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0ceb40 msgr2=0x7feebc0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.302+0000 7feec087c640 1 --2- 172.21.15.125:0/3257425503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0ceb40 0x7feebc0b0650 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7feebc08a120 tx=0x7feeb402c260 comp rx=0 tx=0).stop 2023-10-18T11:58:20.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.302+0000 7feec087c640 1 -- 172.21.15.125:0/3257425503 shutdown_connections 2023-10-18T11:58:20.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.302+0000 7feec087c640 1 --2- 172.21.15.125:0/3257425503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0ceb40 0x7feebc0b0650 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.302+0000 7feec087c640 1 --2- 172.21.15.125:0/3257425503 >> v2:172.21.15.139:3300/0 conn(0x7feebc15b2a0 0x7feebc0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.302+0000 7feec087c640 1 -- 172.21.15.125:0/3257425503 >> 172.21.15.125:0/3257425503 conn(0x7feebc0d0d60 msgr2=0x7feebc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:20.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.302+0000 7feec087c640 1 -- 172.21.15.125:0/3257425503 shutdown_connections 2023-10-18T11:58:20.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.302+0000 7feec087c640 1 -- 172.21.15.125:0/3257425503 wait complete. 2023-10-18T11:58:20.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feec087c640 1 Processor -- start 2023-10-18T11:58:20.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feec087c640 1 -- start start 2023-10-18T11:58:20.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feec087c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0ceb40 0x7feebc0cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feec087c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0cf340 0x7feebc1f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feec087c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc15b2a0 0x7feebc0b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feec087c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b4d70 con 0x7feebc0cf340 2023-10-18T11:58:20.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feec087c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b4830 con 0x7feebc15b2a0 2023-10-18T11:58:20.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feec087c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b4a90 con 0x7feebc0ceb40 2023-10-18T11:58:20.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0cf340 0x7feebc1f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0cf340 0x7feebc1f19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39802/0 (socket says 172.21.15.125:39802) 2023-10-18T11:58:20.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feeb3fff640 1 -- 172.21.15.125:0/2452767059 learned_addr learned my addr 172.21.15.125:0/2452767059 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:20.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.304+0000 7feebad9d640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc15b2a0 0x7feebc0b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeba59c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0ceb40 0x7feebc0cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb3fff640 1 -- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0ceb40 msgr2=0x7feebc0cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb3fff640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0ceb40 0x7feebc0cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb3fff640 1 -- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc15b2a0 msgr2=0x7feebc0b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb3fff640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc15b2a0 0x7feebc0b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb3fff640 1 -- 172.21.15.125:0/2452767059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb4009900 con 0x7feebc0cf340 2023-10-18T11:58:20.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb3fff640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0cf340 0x7feebc1f19c0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7feea400d740 tx=0x7feea400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:20.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb2ffd640 1 -- 172.21.15.125:0/2452767059 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4009c30 con 0x7feebc0cf340 2023-10-18T11:58:20.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb2ffd640 1 -- 172.21.15.125:0/2452767059 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feea4009e80 con 0x7feebc0cf340 2023-10-18T11:58:20.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feeb2ffd640 1 -- 172.21.15.125:0/2452767059 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea40106c0 con 0x7feebc0cf340 2023-10-18T11:58:20.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc0c7f60 con 0x7feebc0cf340 2023-10-18T11:58:20.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.305+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc0c8520 con 0x7feebc0cf340 2023-10-18T11:58:20.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.306+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc0c86e0 con 0x7feebc0cf340 2023-10-18T11:58:20.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.306+0000 7feeb2ffd640 1 -- 172.21.15.125:0/2452767059 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7feea4010820 con 0x7feebc0cf340 2023-10-18T11:58:20.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.307+0000 7feeb2ffd640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fee9c06af60 0x7fee9c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.307+0000 7feeba59c640 1 -- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fee9c06af60 msgr2=0x7fee9c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:20.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.307+0000 7feeb2ffd640 1 -- 172.21.15.125:0/2452767059 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6745+0+0 (secure 0 0 0) 0x7feea408a070 con 0x7feebc0cf340 2023-10-18T11:58:20.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.307+0000 7feeba59c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fee9c06af60 0x7fee9c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:20.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.311+0000 7feeb2ffd640 1 -- 172.21.15.125:0/2452767059 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feea4054c30 con 0x7feebc0cf340 2023-10-18T11:58:20.329 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.327+0000 7ff111a3a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:58:20.331 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.329+0000 7fb587212040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:58:20.382 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.379+0000 7fb587212040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:58:20.386 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.384+0000 7ff111a3a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:58:20.423 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.421+0000 7fb587212040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:58:20.442 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.440+0000 7ff111a3a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:58:20.510 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.507+0000 7ff111a3a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:58:20.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.508+0000 7feeba59c640 1 -- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fee9c06af60 msgr2=0x7fee9c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:20.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.508+0000 7feeba59c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fee9c06af60 0x7fee9c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:58:20.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.542+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feebc0b0d00 con 0x7feebc0cf340 2023-10-18T11:58:20.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.543+0000 7feeb2ffd640 1 -- 172.21.15.125:0/2452767059 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v191) v1 ==== 104+0+11 (secure 0 0 0) 0x7feea4017020 con 0x7feebc0cf340 2023-10-18T11:58:20.546 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T11:58:20.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.545+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fee9c06af60 msgr2=0x7fee9c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:20.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.545+0000 7feec087c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fee9c06af60 0x7fee9c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.545+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0cf340 msgr2=0x7feebc1f19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.545+0000 7feec087c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0cf340 0x7feebc1f19c0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7feea400d740 tx=0x7feea400dc00 comp rx=0 tx=0).stop 2023-10-18T11:58:20.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.545+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 shutdown_connections 2023-10-18T11:58:20.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.546+0000 7feec087c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc15b2a0 0x7feebc0b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.546+0000 7feec087c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fee9c06af60 0x7fee9c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.546+0000 7feec087c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feebc0cf340 0x7feebc1f19c0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.546+0000 7feec087c640 1 --2- 172.21.15.125:0/2452767059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc0ceb40 0x7feebc0cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.546+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 >> 172.21.15.125:0/2452767059 conn(0x7feebc0d0d60 msgr2=0x7feebc06a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:20.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.546+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 shutdown_connections 2023-10-18T11:58:20.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.546+0000 7feec087c640 1 -- 172.21.15.125:0/2452767059 wait complete. 2023-10-18T11:58:20.566 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:58:20.574 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.571+0000 7fb587212040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:58:20.626 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.624+0000 7fb587212040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:58:20.630 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.628+0000 7ff111a3a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:58:20.667 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.665+0000 7fb587212040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:58:20.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.672+0000 7fbffef76640 1 -- 172.21.15.125:0/2467056518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff815b2a0 msgr2=0x7fbff80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.672+0000 7fbffef76640 1 --2- 172.21.15.125:0/2467056518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff815b2a0 0x7fbff80d1910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009860 tx=0x7fbfe402bc20 comp rx=0 tx=0).stop 2023-10-18T11:58:20.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.672+0000 7fbffef76640 1 -- 172.21.15.125:0/2467056518 shutdown_connections 2023-10-18T11:58:20.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.672+0000 7fbffef76640 1 --2- 172.21.15.125:0/2467056518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff815b2a0 0x7fbff80d1910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.672+0000 7fbffef76640 1 -- 172.21.15.125:0/2467056518 >> 172.21.15.125:0/2467056518 conn(0x7fbff80d0d60 msgr2=0x7fbff80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:20.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.672+0000 7fbffef76640 1 -- 172.21.15.125:0/2467056518 shutdown_connections 2023-10-18T11:58:20.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.672+0000 7fbffef76640 1 -- 172.21.15.125:0/2467056518 wait complete. 2023-10-18T11:58:20.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.673+0000 7fbffef76640 1 Processor -- start 2023-10-18T11:58:20.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.673+0000 7fbffef76640 1 -- start start 2023-10-18T11:58:20.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbffef76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff80ceb40 0x7fbff814eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbffef76640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff80cf340 0x7fbff815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbffef76640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff815b2a0 0x7fbff815d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbffef76640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff815d0f0 con 0x7fbff80cf340 2023-10-18T11:58:20.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbffef76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff814f0a0 con 0x7fbff80ceb40 2023-10-18T11:58:20.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbffef76640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff80c89e0 con 0x7fbff815b2a0 2023-10-18T11:58:20.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbff7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff80cf340 0x7fbff815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbff7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff80cf340 0x7fbff815c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39812/0 (socket says 172.21.15.125:39812) 2023-10-18T11:58:20.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbff7fff640 1 -- 172.21.15.125:0/3437463273 learned_addr learned my addr 172.21.15.125:0/3437463273 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:20.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.674+0000 7fbffcd12640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff80ceb40 0x7fbff814eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbffd513640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff815b2a0 0x7fbff815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:20.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff7fff640 1 -- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff815b2a0 msgr2=0x7fbff815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff7fff640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff815b2a0 0x7fbff815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff7fff640 1 -- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff80ceb40 msgr2=0x7fbff814eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff7fff640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff80ceb40 0x7fbff814eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff7fff640 1 -- 172.21.15.125:0/3437463273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe4009490 con 0x7fbff80cf340 2023-10-18T11:58:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff7fff640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff80cf340 0x7fbff815c070 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009980 tx=0x7fbfe402dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff57fa640 1 -- 172.21.15.125:0/3437463273 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4005c90 con 0x7fbff80cf340 2023-10-18T11:58:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff57fa640 1 -- 172.21.15.125:0/3437463273 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfe4005020 con 0x7fbff80cf340 2023-10-18T11:58:20.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff80c83d0 con 0x7fbff80cf340 2023-10-18T11:58:20.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbff57fa640 1 -- 172.21.15.125:0/3437463273 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4005430 con 0x7fbff80cf340 2023-10-18T11:58:20.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.675+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff815ce10 con 0x7fbff80cf340 2023-10-18T11:58:20.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.676+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff80c8b90 con 0x7fbff80cf340 2023-10-18T11:58:20.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.677+0000 7fbff57fa640 1 -- 172.21.15.125:0/3437463273 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fbfe4003800 con 0x7fbff80cf340 2023-10-18T11:58:20.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.677+0000 7fbff57fa640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fbfdc06af60 0x7fbfdc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:20.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.677+0000 7fbffcd12640 1 -- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fbfdc06af60 msgr2=0x7fbfdc06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:20.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.677+0000 7fbff57fa640 1 -- 172.21.15.125:0/3437463273 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fbfe40abe10 con 0x7fbff80cf340 2023-10-18T11:58:20.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.677+0000 7fbffcd12640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fbfdc06af60 0x7fbfdc06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:20.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.682+0000 7fbff57fa640 1 -- 172.21.15.125:0/3437463273 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff80c8b90 con 0x7fbff80cf340 2023-10-18T11:58:20.688 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.685+0000 7ff111a3a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:58:20.716 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.714+0000 7fb587212040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:58:20.731 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.729+0000 7ff111a3a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:58:20.761 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:20.759+0000 7fb587212040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:58:20.811 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.809+0000 7ff111a3a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:58:20.857 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:20.854+0000 7ff111a3a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:58:20.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.878+0000 7fbffcd12640 1 -- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fbfdc06af60 msgr2=0x7fbfdc06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:20.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.878+0000 7fbffcd12640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fbfdc06af60 0x7fbfdc06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:58:20.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.892+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbff80c8b50 con 0x7fbff80cf340 2023-10-18T11:58:20.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.892+0000 7fbff57fa640 1 -- 172.21.15.125:0/3437463273 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v191) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfe40390a0 con 0x7fbff80cf340 2023-10-18T11:58:20.895 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T11:58:20.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fbfdc06af60 msgr2=0x7fbfdc06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:20.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fbfdc06af60 0x7fbfdc06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff80cf340 msgr2=0x7fbff815c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:20.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff80cf340 0x7fbff815c070 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009980 tx=0x7fbfe402dd20 comp rx=0 tx=0).stop 2023-10-18T11:58:20.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 shutdown_connections 2023-10-18T11:58:20.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fbfdc06af60 0x7fbfdc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff815b2a0 0x7fbff815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff80cf340 0x7fbff815c070 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 --2- 172.21.15.125:0/3437463273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff80ceb40 0x7fbff814eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:20.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 >> 172.21.15.125:0/3437463273 conn(0x7fbff80d0d60 msgr2=0x7fbff806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:20.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 shutdown_connections 2023-10-18T11:58:20.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:20.894+0000 7fbffef76640 1 -- 172.21.15.125:0/3437463273 wait complete. 2023-10-18T11:58:20.908 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T11:58:21.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.010+0000 7f290fcb4640 1 -- 172.21.15.125:0/1120813021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 msgr2=0x7f28f4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:21.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.010+0000 7f290fcb4640 1 --2- 172.21.15.125:0/1120813021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 0x7f28f4004c40 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f28fc006ed0 tx=0x7f28f40057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.010+0000 7f290e251640 1 -- 172.21.15.125:0/1120813021 reap_dead start 2023-10-18T11:58:21.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.010+0000 7f290fcb4640 1 -- 172.21.15.125:0/1120813021 shutdown_connections 2023-10-18T11:58:21.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.010+0000 7f290fcb4640 1 -- 172.21.15.125:0/1120813021 >> 172.21.15.125:0/1120813021 conn(0x7f29080d0d60 msgr2=0x7f29080d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:21.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.011+0000 7f290fcb4640 1 -- 172.21.15.125:0/1120813021 shutdown_connections 2023-10-18T11:58:21.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.011+0000 7f290fcb4640 1 -- 172.21.15.125:0/1120813021 wait complete. 2023-10-18T11:58:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.012+0000 7f290fcb4640 1 Processor -- start 2023-10-18T11:58:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.012+0000 7f290fcb4640 1 -- start start 2023-10-18T11:58:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.012+0000 7f290fcb4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 0x7f29080c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.012+0000 7f290fcb4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f290815ca40 0x7f290815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:21.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.012+0000 7f290fcb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29080cc3a0 0x7f29080cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.012+0000 7f290fcb4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29080ccd60 con 0x7f28f40047c0 2023-10-18T11:58:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.012+0000 7f290fcb4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081e4230 con 0x7f29080cc3a0 2023-10-18T11:58:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.012+0000 7f290fcb4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29081e3620 con 0x7f290815ca40 2023-10-18T11:58:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 0x7f29080c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:21.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 0x7f29080c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39838/0 (socket says 172.21.15.125:39838) 2023-10-18T11:58:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 -- 172.21.15.125:0/113055666 learned_addr learned my addr 172.21.15.125:0/113055666 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290d24f640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f290815ca40 0x7f290815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290e251640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29080cc3a0 0x7f29080cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 -- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f290815ca40 msgr2=0x7f290815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f290815ca40 0x7f290815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 -- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29080cc3a0 msgr2=0x7f29080cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:21.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29080cc3a0 0x7f29080cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 -- 172.21.15.125:0/113055666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29081e3900 con 0x7f28f40047c0 2023-10-18T11:58:21.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290e251640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29080cc3a0 0x7f29080cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:21.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290da50640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 0x7f29080c69e0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f2904003ed0 tx=0x7f2904001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f28fa7fc640 1 -- 172.21.15.125:0/113055666 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904006180 con 0x7f28f40047c0 2023-10-18T11:58:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f28fa7fc640 1 -- 172.21.15.125:0/113055666 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2904011900 con 0x7f28f40047c0 2023-10-18T11:58:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f28fa7fc640 1 -- 172.21.15.125:0/113055666 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904011d10 con 0x7f28f40047c0 2023-10-18T11:58:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29081e3be0 con 0x7f28f40047c0 2023-10-18T11:58:21.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.013+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290814cc90 con 0x7f28f40047c0 2023-10-18T11:58:21.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.014+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29080690e0 con 0x7f28f40047c0 2023-10-18T11:58:21.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.015+0000 7f28fa7fc640 1 -- 172.21.15.125:0/113055666 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f29040103c0 con 0x7f28f40047c0 2023-10-18T11:58:21.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.015+0000 7f28fa7fc640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f28ec06af60 0x7f28ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:21.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.015+0000 7f290d24f640 1 -- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f28ec06af60 msgr2=0x7f28ec06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:21.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.015+0000 7f28fa7fc640 1 -- 172.21.15.125:0/113055666 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f290408ae70 con 0x7f28f40047c0 2023-10-18T11:58:21.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.015+0000 7f290d24f640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f28ec06af60 0x7f28ec06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:21.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.019+0000 7f28fa7fc640 1 -- 172.21.15.125:0/113055666 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2904055810 con 0x7f28f40047c0 2023-10-18T11:58:21.092 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.089+0000 7fb587212040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:58:21.183 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.181+0000 7ff111a3a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:58:21.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.199+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f29080d0b40 con 0x7f28f40047c0 2023-10-18T11:58:21.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.199+0000 7f28fa7fc640 1 -- 172.21.15.125:0/113055666 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v191) v1 ==== 81+0+951 (secure 0 0 0) 0x7f2904015080 con 0x7f28f40047c0 2023-10-18T11:58:21.203 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/2153770477 2023-10-19T11:53:19.652678+0000 2023-10-18T11:58:21.203 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/2153770477 2023-10-19T11:53:19.652678+0000 2023-10-18T11:58:21.203 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1872946902 2023-10-19T11:53:19.652678+0000 2023-10-18T11:58:21.203 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1126366928 2023-10-19T11:53:19.652678+0000 2023-10-18T11:58:21.203 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1293875531 2023-10-19T11:53:19.652678+0000 2023-10-18T11:58:21.204 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/608662592 2023-10-19T11:53:19.652678+0000 2023-10-18T11:58:21.204 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/141746223 2023-10-19T11:53:19.652678+0000 2023-10-18T11:58:21.204 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1253211552 2023-10-19T11:53:19.652678+0000 2023-10-18T11:58:21.204 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3751642323 2023-10-19T11:42:31.009948+0000 2023-10-18T11:58:21.205 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/402965407 2023-10-19T11:42:31.009948+0000 2023-10-18T11:58:21.205 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/3367593104 2023-10-19T11:42:31.009948+0000 2023-10-18T11:58:21.205 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/3367593104 2023-10-19T11:42:31.009948+0000 2023-10-18T11:58:21.205 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/4091087182 2023-10-19T11:42:31.009948+0000 2023-10-18T11:58:21.205 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/697097533 2023-10-19T11:42:31.009948+0000 2023-10-18T11:58:21.206 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/4016867169 2023-10-19T11:42:31.009948+0000 2023-10-18T11:58:21.206 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/878133557 2023-10-19T11:42:31.009948+0000 2023-10-18T11:58:21.207 INFO:teuthology.orchestra.run.smithi125.stderr:listed 16 entries 2023-10-18T11:58:21.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f28ec06af60 msgr2=0x7f28ec06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f28ec06af60 0x7f28ec06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 msgr2=0x7f29080c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 0x7f29080c69e0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f2904003ed0 tx=0x7f2904001770 comp rx=0 tx=0).stop 2023-10-18T11:58:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 shutdown_connections 2023-10-18T11:58:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29080cc3a0 0x7f29080cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f28ec06af60 0x7f28ec06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f290815ca40 0x7f290815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 --2- 172.21.15.125:0/113055666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28f40047c0 0x7f29080c69e0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.201+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 >> 172.21.15.125:0/113055666 conn(0x7f29080d0d60 msgr2=0x7f290815a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.202+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 shutdown_connections 2023-10-18T11:58:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.202+0000 7f290fcb4640 1 -- 172.21.15.125:0/113055666 wait complete. 2023-10-18T11:58:21.215 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T11:58:21.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:21.216 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T11:58:21.220 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.217+0000 7fb587212040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:58:21.241 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[146596]: fuse finished with error 0 and tester_r 0 2023-10-18T11:58:21.243 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T11:58:21.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:21.244 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T11:58:21.311 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.309+0000 7fb587212040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:58:21.322 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.320+0000 7ff111a3a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:58:21.354 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.351+0000 7fb587212040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:58:21.355 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:58:21.356 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:58:21.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:58:21.371 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T11:58:21.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:21.372 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T11:58:21.416 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.414+0000 7ff111a3a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:58:21.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:58:21.427 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T11:58:21.428 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:58:21.428 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:58:21.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:58:21.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T11:58:21.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:21.445 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T11:58:21.458 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.455+0000 7fb587212040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:58:21.463 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.461+0000 7ff111a3a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:58:21.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:58:21.479 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T11:58:21.479 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:58:21.479 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:58:21.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:58:21.494 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T11:58:21.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:21.495 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T11:58:21.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T11:58:21.550 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T11:58:21.551 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:58:21.555 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.553+0000 7fb587212040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:58:21.564 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.562+0000 7ff111a3a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:58:21.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.653+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2441309625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a806c630 msgr2=0x7fc2a8070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:21.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.653+0000 7fc2aea7c640 1 --2- 172.21.15.125:0/2441309625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a806c630 0x7fc2a8070080 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0005fd0 tx=0x7fc2a002c2e0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.653+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2441309625 shutdown_connections 2023-10-18T11:58:21.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.653+0000 7fc2aea7c640 1 --2- 172.21.15.125:0/2441309625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a806c630 0x7fc2a8070080 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.653+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2441309625 >> 172.21.15.125:0/2441309625 conn(0x7fc2a80af730 msgr2=0x7fc2a80afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:21.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.653+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2441309625 shutdown_connections 2023-10-18T11:58:21.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.653+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2441309625 wait complete. 2023-10-18T11:58:21.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.654+0000 7fc2aea7c640 1 Processor -- start 2023-10-18T11:58:21.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.654+0000 7fc2aea7c640 1 -- start start 2023-10-18T11:58:21.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2aea7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a8069e30 0x7fc2a81384b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:21.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2aea7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2a806c630 0x7fc2a813abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:21.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2aea7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2a81424c0 0x7fc2a8138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:21.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2aea7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a813a730 con 0x7fc2a8069e30 2023-10-18T11:58:21.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2aea7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8139b20 con 0x7fc2a806c630 2023-10-18T11:58:21.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2aea7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8139e00 con 0x7fc2a81424c0 2023-10-18T11:58:21.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a8069e30 0x7fc2a81384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:21.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a8069e30 0x7fc2a81384b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39854/0 (socket says 172.21.15.125:39854) 2023-10-18T11:58:21.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2a806c630 0x7fc2a813abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:21.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 -- 172.21.15.125:0/2667880822 learned_addr learned my addr 172.21.15.125:0/2667880822 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:21.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ad019640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2a81424c0 0x7fc2a8138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:21.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 -- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2a81424c0 msgr2=0x7fc2a8138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:21.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2a81424c0 0x7fc2a8138fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 -- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2a806c630 msgr2=0x7fc2a813abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:21.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2a806c630 0x7fc2a813abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 -- 172.21.15.125:0/2667880822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0009900 con 0x7fc2a8069e30 2023-10-18T11:58:21.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2ac818640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a8069e30 0x7fc2a81384b0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c3c0 tx=0x7fc29c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:21.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.655+0000 7fc2a57fa640 1 -- 172.21.15.125:0/2667880822 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0052d0 con 0x7fc2a8069e30 2023-10-18T11:58:21.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.656+0000 7fc2a57fa640 1 -- 172.21.15.125:0/2667880822 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc29c007020 con 0x7fc2a8069e30 2023-10-18T11:58:21.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.656+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a813a0e0 con 0x7fc2a8069e30 2023-10-18T11:58:21.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.656+0000 7fc2a57fa640 1 -- 172.21.15.125:0/2667880822 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c007470 con 0x7fc2a8069e30 2023-10-18T11:58:21.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.656+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a81561a0 con 0x7fc2a8069e30 2023-10-18T11:58:21.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.657+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a80675f0 con 0x7fc2a8069e30 2023-10-18T11:58:21.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.657+0000 7fc2a57fa640 1 -- 172.21.15.125:0/2667880822 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fc29c0067d0 con 0x7fc2a8069e30 2023-10-18T11:58:21.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.658+0000 7fc2a57fa640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fc28c06af90 0x7fc28c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:21.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.658+0000 7fc2a7fff640 1 -- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fc28c06af90 msgr2=0x7fc28c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:21.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.658+0000 7fc2a57fa640 1 -- 172.21.15.125:0/2667880822 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6817+0+0 (secure 0 0 0) 0x7fc29c089100 con 0x7fc2a8069e30 2023-10-18T11:58:21.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.658+0000 7fc2a7fff640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fc28c06af90 0x7fc28c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:21.668 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.659+0000 7ff111a3a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:58:21.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.662+0000 7fc2a57fa640 1 -- 172.21.15.125:0/2667880822 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc29c053cf0 con 0x7fc2a8069e30 2023-10-18T11:58:21.692 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.689+0000 7fb587212040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:58:21.733 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.730+0000 7fb587212040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:58:21.804 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.801+0000 7ff111a3a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:58:21.835 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:21.833+0000 7fb587212040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:58:21.847 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.845+0000 7ff111a3a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:58:21.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.859+0000 7fc2a7fff640 1 -- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fc28c06af90 msgr2=0x7fc28c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:21.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.859+0000 7fc2a7fff640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fc28c06af90 0x7fc28c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:58:21.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.870+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2a806bf00 con 0x7fc2a8069e30 2023-10-18T11:58:21.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.870+0000 7fc2a57fa640 1 -- 172.21.15.125:0/2667880822 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4933 (secure 0 0 0) 0x7fc29c057310 con 0x7fc2a8069e30 2023-10-18T11:58:21.873 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:21.873 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":73,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2139029617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2139029617},{"type":"v1","addr":"172.21.15.125:6835","nonce":2139029617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2438879012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2438879012},{"type":"v1","addr":"172.21.15.139:6837","nonce":2438879012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":10830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2643526616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2643526616},{"type":"v1","addr":"172.21.15.125:6834","nonce":2643526616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:42:14.915536+0000","modified":"2023-10-18T11:42:16.934466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6836/3571180970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3571180970},{"type":"v1","addr":"172.21.15.139:6836","nonce":3571180970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T11:58:21.874 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 73 2023-10-18T11:58:21.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fc28c06af90 msgr2=0x7fc28c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:21.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fc28c06af90 0x7fc28c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a8069e30 msgr2=0x7fc2a81384b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:21.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a8069e30 0x7fc2a81384b0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c3c0 tx=0x7fc29c00c880 comp rx=0 tx=0).stop 2023-10-18T11:58:21.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 shutdown_connections 2023-10-18T11:58:21.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2a806c630 0x7fc2a813abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fc28c06af90 0x7fc28c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2a81424c0 0x7fc2a8138fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 --2- 172.21.15.125:0/2667880822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2a8069e30 0x7fc2a81384b0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:21.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 >> 172.21.15.125:0/2667880822 conn(0x7fc2a80af730 msgr2=0x7fc2a80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:21.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.872+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 shutdown_connections 2023-10-18T11:58:21.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:21.873+0000 7fc2aea7c640 1 -- 172.21.15.125:0/2667880822 wait complete. 2023-10-18T11:58:21.886 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:21.958 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:21.955+0000 7ff111a3a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:58:22.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.031+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/3343252080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1e40cf340 msgr2=0x7fb1e40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.031+0000 7fb1ec4e7640 1 --2- 172.21.15.125:0/3343252080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1e40cf340 0x7fb1e40d1910 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009830 tx=0x7fb1e002bd60 comp rx=0 tx=0).stop 2023-10-18T11:58:22.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.031+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/3343252080 shutdown_connections 2023-10-18T11:58:22.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.031+0000 7fb1ec4e7640 1 --2- 172.21.15.125:0/3343252080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1e40cf340 0x7fb1e40d1910 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.031+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/3343252080 >> 172.21.15.125:0/3343252080 conn(0x7fb1e40d0d60 msgr2=0x7fb1e40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:22.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.031+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/3343252080 shutdown_connections 2023-10-18T11:58:22.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.031+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/3343252080 wait complete. 2023-10-18T11:58:22.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.032+0000 7fb1ec4e7640 1 Processor -- start 2023-10-18T11:58:22.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.032+0000 7fb1ec4e7640 1 -- start start 2023-10-18T11:58:22.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ec4e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1e40ceb40 0x7fb1e40ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ec4e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1e40cf340 0x7fb1e40cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ec4e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1e415b2a0 0x7fb1e4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ec4e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c8040 con 0x7fb1e40ceb40 2023-10-18T11:58:22.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ec4e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c8ea0 con 0x7fb1e415b2a0 2023-10-18T11:58:22.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ec4e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c8290 con 0x7fb1e40cf340 2023-10-18T11:58:22.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1e40ceb40 0x7fb1e40ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1e40ceb40 0x7fb1e40ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39856/0 (socket says 172.21.15.125:39856) 2023-10-18T11:58:22.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 -- 172.21.15.125:0/1666353812 learned_addr learned my addr 172.21.15.125:0/1666353812 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:22.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1eaa84640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1e415b2a0 0x7fb1e4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1e9a82640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1e40cf340 0x7fb1e40cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 -- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1e40cf340 msgr2=0x7fb1e40cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1e40cf340 0x7fb1e40cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 -- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1e415b2a0 msgr2=0x7fb1e4158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1e415b2a0 0x7fb1e4158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 -- 172.21.15.125:0/1666353812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0009490 con 0x7fb1e40ceb40 2023-10-18T11:58:22.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ea283640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1e40ceb40 0x7fb1e40ca820 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fb1e002c370 tx=0x7fb1e00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:22.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1daffd640 1 -- 172.21.15.125:0/1666353812 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0003ea0 con 0x7fb1e40ceb40 2023-10-18T11:58:22.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1daffd640 1 -- 172.21.15.125:0/1666353812 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb1e0005c90 con 0x7fb1e40ceb40 2023-10-18T11:58:22.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e40c8570 con 0x7fb1e40ceb40 2023-10-18T11:58:22.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1daffd640 1 -- 172.21.15.125:0/1666353812 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0005280 con 0x7fb1e40ceb40 2023-10-18T11:58:22.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.033+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40c8b30 con 0x7fb1e40ceb40 2023-10-18T11:58:22.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.034+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e40690e0 con 0x7fb1e40ceb40 2023-10-18T11:58:22.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.035+0000 7fb1daffd640 1 -- 172.21.15.125:0/1666353812 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fb1e00053e0 con 0x7fb1e40ceb40 2023-10-18T11:58:22.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.035+0000 7fb1daffd640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fb1c806af60 0x7fb1c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.035+0000 7fb1e9a82640 1 -- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fb1c806af60 msgr2=0x7fb1c806d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:22.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.035+0000 7fb1daffd640 1 -- 172.21.15.125:0/1666353812 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6817+0+0 (secure 0 0 0) 0x7fb1e00ac6c0 con 0x7fb1e40ceb40 2023-10-18T11:58:22.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.035+0000 7fb1e9a82640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fb1c806af60 0x7fb1c806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:22.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.039+0000 7fb1daffd640 1 -- 172.21.15.125:0/1666353812 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1e00772b0 con 0x7fb1e40ceb40 2023-10-18T11:58:22.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.214+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1e4157860 con 0x7fb1e40ceb40 2023-10-18T11:58:22.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.215+0000 7fb1daffd640 1 -- 172.21.15.125:0/1666353812 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+17961 (secure 0 0 0) 0x7fb1e007a8d0 con 0x7fb1e40ceb40 2023-10-18T11:58:22.218 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:22.218 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":192,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:58:21.425702+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:42:12.758545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-18T11:42:13.894048+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":192,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6833/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:6832/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1872946902":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1126366928":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1293875531":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/608662592":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/141746223":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1253211552":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/3751642323":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/402965407":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6833/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6832/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/4091087182":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/697097533":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/4016867169":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/878133557":"2023-10-19T11:42:31.009948+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:22.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fb1c806af60 msgr2=0x7fb1c806d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:22.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fb1c806af60 0x7fb1c806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1e40ceb40 msgr2=0x7fb1e40ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1e40ceb40 0x7fb1e40ca820 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fb1e002c370 tx=0x7fb1e00038a0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 shutdown_connections 2023-10-18T11:58:22.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1e415b2a0 0x7fb1e4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fb1c806af60 0x7fb1c806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1e40cf340 0x7fb1e40cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 --2- 172.21.15.125:0/1666353812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1e40ceb40 0x7fb1e40ca820 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 >> 172.21.15.125:0/1666353812 conn(0x7fb1e40d0d60 msgr2=0x7fb1e406b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:22.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 shutdown_connections 2023-10-18T11:58:22.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.217+0000 7fb1ec4e7640 1 -- 172.21.15.125:0/1666353812 wait complete. 2023-10-18T11:58:22.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T11:42:12.758545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T11:42:13.894048+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 192, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:58:22.231 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:58:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.376+0000 7f1a3458a640 1 -- 172.21.15.125:0/4243375894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0704e0 msgr2=0x7f1a2c0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.376+0000 7f1a3458a640 1 --2- 172.21.15.125:0/4243375894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0704e0 0x7f1a2c0cc910 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1a24005fd0 tx=0x7f1a2402c170 comp rx=0 tx=0).stop 2023-10-18T11:58:22.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.376+0000 7f1a3458a640 1 -- 172.21.15.125:0/4243375894 shutdown_connections 2023-10-18T11:58:22.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.376+0000 7f1a3458a640 1 --2- 172.21.15.125:0/4243375894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0704e0 0x7f1a2c0cc910 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.376+0000 7f1a3458a640 1 -- 172.21.15.125:0/4243375894 >> 172.21.15.125:0/4243375894 conn(0x7f1a2c0af730 msgr2=0x7f1a2c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:22.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.376+0000 7f1a3458a640 1 -- 172.21.15.125:0/4243375894 shutdown_connections 2023-10-18T11:58:22.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.376+0000 7f1a3458a640 1 -- 172.21.15.125:0/4243375894 wait complete. 2023-10-18T11:58:22.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.377+0000 7f1a3458a640 1 Processor -- start 2023-10-18T11:58:22.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.377+0000 7f1a3458a640 1 -- start start 2023-10-18T11:58:22.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a3458a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c070080 0x7f1a2c134c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a3458a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0704e0 0x7f1a2c1f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a3458a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c1416f0 0x7f1a2c1f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a3458a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c1567e0 con 0x7f1a2c070080 2023-10-18T11:58:22.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a3458a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c1f18f0 con 0x7f1a2c0704e0 2023-10-18T11:58:22.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a3458a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c140fa0 con 0x7f1a2c1416f0 2023-10-18T11:58:22.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32326640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c070080 0x7f1a2c134c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32326640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c070080 0x7f1a2c134c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39858/0 (socket says 172.21.15.125:39858) 2023-10-18T11:58:22.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32326640 1 -- 172.21.15.125:0/3201515885 learned_addr learned my addr 172.21.15.125:0/3201515885 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:22.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a31b25640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0704e0 0x7f1a2c1f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32b27640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c1416f0 0x7f1a2c1f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32326640 1 -- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c1416f0 msgr2=0x7f1a2c1f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32326640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c1416f0 0x7f1a2c1f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32326640 1 -- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0704e0 msgr2=0x7f1a2c1f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32326640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0704e0 0x7f1a2c1f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.378+0000 7f1a32326640 1 -- 172.21.15.125:0/3201515885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009840 con 0x7f1a2c070080 2023-10-18T11:58:22.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.379+0000 7f1a32326640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c070080 0x7f1a2c134c60 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c3c0 tx=0x7f1a1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:22.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.379+0000 7f1a22ffd640 1 -- 172.21.15.125:0/3201515885 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c004290 con 0x7f1a2c070080 2023-10-18T11:58:22.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.379+0000 7f1a22ffd640 1 -- 172.21.15.125:0/3201515885 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1a1c0035f0 con 0x7f1a2c070080 2023-10-18T11:58:22.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.379+0000 7f1a22ffd640 1 -- 172.21.15.125:0/3201515885 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c004500 con 0x7f1a2c070080 2023-10-18T11:58:22.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.379+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c0b1cd0 con 0x7f1a2c070080 2023-10-18T11:58:22.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.379+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c156350 con 0x7f1a2c070080 2023-10-18T11:58:22.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.380+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c1351a0 con 0x7f1a2c070080 2023-10-18T11:58:22.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.380+0000 7f1a22ffd640 1 -- 172.21.15.125:0/3201515885 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f1a1c003a40 con 0x7f1a2c070080 2023-10-18T11:58:22.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.380+0000 7f1a22ffd640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f1a1006af60 0x7f1a1006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.380+0000 7f1a31b25640 1 -- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f1a1006af60 msgr2=0x7f1a1006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:22.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.381+0000 7f1a22ffd640 1 -- 172.21.15.125:0/3201515885 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6817+0+0 (secure 0 0 0) 0x7f1a1c089340 con 0x7f1a2c070080 2023-10-18T11:58:22.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.381+0000 7f1a31b25640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f1a1006af60 0x7f1a1006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:22.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.385+0000 7f1a22ffd640 1 -- 172.21.15.125:0/3201515885 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a1c053f70 con 0x7f1a2c070080 2023-10-18T11:58:22.455 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.453+0000 7fb587212040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:58:22.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.559+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a2c156180 con 0x7f1a2c070080 2023-10-18T11:58:22.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.560+0000 7f1a22ffd640 1 -- 172.21.15.125:0/3201515885 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v192) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a1c057590 con 0x7f1a2c070080 2023-10-18T11:58:22.563 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:58:22.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f1a1006af60 msgr2=0x7f1a1006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:22.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f1a1006af60 0x7f1a1006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c070080 msgr2=0x7f1a2c134c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c070080 0x7f1a2c134c60 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c3c0 tx=0x7f1a1c00c880 comp rx=0 tx=0).stop 2023-10-18T11:58:22.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 shutdown_connections 2023-10-18T11:58:22.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a2c0704e0 0x7f1a2c1f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7f1a1006af60 0x7f1a1006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a2c1416f0 0x7f1a2c1f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 --2- 172.21.15.125:0/3201515885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c070080 0x7f1a2c134c60 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 >> 172.21.15.125:0/3201515885 conn(0x7f1a2c0af730 msgr2=0x7f1a2c13e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:22.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 shutdown_connections 2023-10-18T11:58:22.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.562+0000 7f1a3458a640 1 -- 172.21.15.125:0/3201515885 wait complete. 2023-10-18T11:58:22.575 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T11:58:22.620 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:22.617+0000 7ff111a3a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:58:22.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.680+0000 7fe193733640 1 -- 172.21.15.125:0/546495192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c15b2a0 msgr2=0x7fe18c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.680+0000 7fe193733640 1 --2- 172.21.15.125:0/546495192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c15b2a0 0x7fe18c0d1910 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fe18c15b680 tx=0x7fe18802bdb0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.680+0000 7fe193733640 1 -- 172.21.15.125:0/546495192 shutdown_connections 2023-10-18T11:58:22.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.680+0000 7fe193733640 1 --2- 172.21.15.125:0/546495192 >> v2:172.21.15.139:3300/0 conn(0x7fe18c0ceb40 0x7fe18c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.680+0000 7fe193733640 1 --2- 172.21.15.125:0/546495192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c15b2a0 0x7fe18c0d1910 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.680+0000 7fe193733640 1 -- 172.21.15.125:0/546495192 >> 172.21.15.125:0/546495192 conn(0x7fe18c0d0d60 msgr2=0x7fe18c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:22.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.680+0000 7fe193733640 1 -- 172.21.15.125:0/546495192 shutdown_connections 2023-10-18T11:58:22.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.680+0000 7fe193733640 1 -- 172.21.15.125:0/546495192 wait complete. 2023-10-18T11:58:22.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.681+0000 7fe193733640 1 Processor -- start 2023-10-18T11:58:22.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.681+0000 7fe193733640 1 -- start start 2023-10-18T11:58:22.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe193733640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0ceb40 0x7fe18c150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe193733640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c0cf340 0x7fe18c0b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe193733640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe18c15b2a0 0x7fe18c0ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe193733640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c1514a0 con 0x7fe18c0cf340 2023-10-18T11:58:22.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe193733640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0c93f0 con 0x7fe18c0ceb40 2023-10-18T11:58:22.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe193733640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0c96d0 con 0x7fe18c15b2a0 2023-10-18T11:58:22.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe190cce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c0cf340 0x7fe18c0b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe190cce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c0cf340 0x7fe18c0b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39884/0 (socket says 172.21.15.125:39884) 2023-10-18T11:58:22.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe190cce640 1 -- 172.21.15.125:0/108924740 learned_addr learned my addr 172.21.15.125:0/108924740 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:22.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe1914cf640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0ceb40 0x7fe18c150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.682+0000 7fe191cd0640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe18c15b2a0 0x7fe18c0ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe190cce640 1 -- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe18c15b2a0 msgr2=0x7fe18c0ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe190cce640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe18c15b2a0 0x7fe18c0ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe190cce640 1 -- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0ceb40 msgr2=0x7fe18c150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe190cce640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0ceb40 0x7fe18c150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe190cce640 1 -- 172.21.15.125:0/108924740 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188009560 con 0x7fe18c0cf340 2023-10-18T11:58:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe191cd0640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe18c15b2a0 0x7fe18c0ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe190cce640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c0cf340 0x7fe18c0b6470 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe18802da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188003800 con 0x7fe18c0cf340 2023-10-18T11:58:22.694 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe188003a50 con 0x7fe18c0cf340 2023-10-18T11:58:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c0c99b0 con 0x7fe18c0cf340 2023-10-18T11:58:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188003e60 con 0x7fe18c0cf340 2023-10-18T11:58:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.683+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c0c9f70 con 0x7fe18c0cf340 2023-10-18T11:58:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.684+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 115) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe18803e450 con 0x7fe18c0cf340 2023-10-18T11:58:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.684+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c0690e0 con 0x7fe18c0cf340 2023-10-18T11:58:22.695 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.685+0000 7fe181ffb640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fe17006af90 0x7fe17006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.685+0000 7fe1914cf640 1 -- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fe17006af90 msgr2=0x7fe17006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/833207676 2023-10-18T11:58:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.685+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6817+0+0 (secure 0 0 0) 0x7fe1880ac650 con 0x7fe18c0cf340 2023-10-18T11:58:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.685+0000 7fe1914cf640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fe17006af90 0x7fe17006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:22.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.689+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1880772b0 con 0x7fe18c0cf340 2023-10-18T11:58:22.718 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.716+0000 7fb587212040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:58:22.766 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.764+0000 7fb587212040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:58:22.864 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.862+0000 7fb587212040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:58:22.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.883+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 116) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe1880aa710 con 0x7fe18c0cf340 2023-10-18T11:58:22.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.883+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fe17006af90 msgr2=0x7fe17006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:22.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.883+0000 7fe181ffb640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fe17006af90 0x7fe17006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.903+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe18c0b0d00 con 0x7fe18c0cf340 2023-10-18T11:58:22.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.903+0000 7fe181ffb640 1 -- 172.21.15.125:0/108924740 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v193) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1880a56c0 con 0x7fe18c0cf340 2023-10-18T11:58:22.906 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T11:58:22.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c0cf340 msgr2=0x7fe18c0b6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:22.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c0cf340 0x7fe18c0b6470 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe18802da80 comp rx=0 tx=0).stop 2023-10-18T11:58:22.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 shutdown_connections 2023-10-18T11:58:22.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:6832/833207676,v1:172.21.15.139:6833/833207676] conn(0x7fe17006af90 0x7fe17006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe18c15b2a0 0x7fe18c0ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe18c0cf340 0x7fe18c0b6470 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 --2- 172.21.15.125:0/108924740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0ceb40 0x7fe18c150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:22.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 >> 172.21.15.125:0/108924740 conn(0x7fe18c0d0d60 msgr2=0x7fe18c1588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:22.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 shutdown_connections 2023-10-18T11:58:22.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:22.906+0000 7fe193733640 1 -- 172.21.15.125:0/108924740 wait complete. 2023-10-18T11:58:22.923 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T11:58:22.939 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:22.937+0000 7ff111a3a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:58:22.940 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.937+0000 7fb50c0ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.940 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.937+0000 7fb50c0ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.940 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.937+0000 7fb50c0ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.940 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.937+0000 7fb50c0ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.940 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.937+0000 7fb50c0ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.941 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.938+0000 7fb5088b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.941 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.938+0000 7fb5088b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.942 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.938+0000 7fb5088b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.942 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.938+0000 7fb5088b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.942 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:22.938+0000 7fb5088b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:22.991 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:22.989+0000 7ff111a3a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:58:23.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.029+0000 7f060d4cf640 1 -- 172.21.15.125:0/1512355773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05f40039a0 msgr2=0x7f05f4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.029+0000 7f060d4cf640 1 --2- 172.21.15.125:0/1512355773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05f40039a0 0x7f05f4003da0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f05f800c710 tx=0x7f05f40057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.029+0000 7f06077fe640 1 -- 172.21.15.125:0/1512355773 reap_dead start 2023-10-18T11:58:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.029+0000 7f060d4cf640 1 -- 172.21.15.125:0/1512355773 shutdown_connections 2023-10-18T11:58:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.029+0000 7f060d4cf640 1 -- 172.21.15.125:0/1512355773 >> 172.21.15.125:0/1512355773 conn(0x7f060809a030 msgr2=0x7f06080d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.029+0000 7f060d4cf640 1 -- 172.21.15.125:0/1512355773 shutdown_connections 2023-10-18T11:58:23.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.029+0000 7f060d4cf640 1 -- 172.21.15.125:0/1512355773 wait complete. 2023-10-18T11:58:23.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.030+0000 7f060d4cf640 1 Processor -- start 2023-10-18T11:58:23.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.030+0000 7f060d4cf640 1 -- start start 2023-10-18T11:58:23.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f060d4cf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05f40039a0 0x7f060814b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f060d4cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060815c7e0 0x7f06080b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f060d4cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06080b5740 0x7f06080cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f060d4cf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060809acb0 con 0x7f06080b5740 2023-10-18T11:58:23.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f060d4cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060815c350 con 0x7f060815c7e0 2023-10-18T11:58:23.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f060d4cf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06080b5be0 con 0x7f05f40039a0 2023-10-18T11:58:23.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06080b5740 0x7f06080cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06080b5740 0x7f06080cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39910/0 (socket says 172.21.15.125:39910) 2023-10-18T11:58:23.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 -- 172.21.15.125:0/1235082949 learned_addr learned my addr 172.21.15.125:0/1235082949 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:23.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f0606ffd640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05f40039a0 0x7f060814b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f05fffff640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060815c7e0 0x7f06080b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 -- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05f40039a0 msgr2=0x7f060814b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05f40039a0 0x7f060814b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 -- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060815c7e0 msgr2=0x7f06080b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060815c7e0 0x7f06080b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 -- 172.21.15.125:0/1235082949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060815ba20 con 0x7f06080b5740 2023-10-18T11:58:23.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f05fffff640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060815c7e0 0x7f06080b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:23.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f0606ffd640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05f40039a0 0x7f060814b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:23.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f06077fe640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06080b5740 0x7f06080cb9a0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f06080cfef0 tx=0x7f0600010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:23.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.031+0000 7f05ff7fe640 1 -- 172.21.15.125:0/1235082949 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600019070 con 0x7f06080b5740 2023-10-18T11:58:23.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.032+0000 7f05ff7fe640 1 -- 172.21.15.125:0/1235082949 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f060000ec70 con 0x7f06080b5740 2023-10-18T11:58:23.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.032+0000 7f05ff7fe640 1 -- 172.21.15.125:0/1235082949 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600004b20 con 0x7f06080b5740 2023-10-18T11:58:23.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.032+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060815bd00 con 0x7f06080b5740 2023-10-18T11:58:23.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.032+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0608159170 con 0x7f06080b5740 2023-10-18T11:58:23.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.033+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0608159330 con 0x7f06080b5740 2023-10-18T11:58:23.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.033+0000 7f05ff7fe640 1 -- 172.21.15.125:0/1235082949 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 116) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f0600003970 con 0x7f06080b5740 2023-10-18T11:58:23.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.033+0000 7f05ff7fe640 1 -- 172.21.15.125:0/1235082949 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7117+0+0 (secure 0 0 0) 0x7f0600089a10 con 0x7f06080b5740 2023-10-18T11:58:23.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.037+0000 7f05ff7fe640 1 -- 172.21.15.125:0/1235082949 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0600054990 con 0x7f06080b5740 2023-10-18T11:58:23.099 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:23.096+0000 7ff111a3a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:58:23.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.209+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f060815c4c0 con 0x7f06080b5740 2023-10-18T11:58:23.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.209+0000 7f05ff7fe640 1 -- 172.21.15.125:0/1235082949 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v193) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0600057fb0 con 0x7f06080b5740 2023-10-18T11:58:23.212 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T11:58:23.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.211+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06080b5740 msgr2=0x7f06080cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.211+0000 7f060d4cf640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06080b5740 0x7f06080cb9a0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f06080cfef0 tx=0x7f0600010f00 comp rx=0 tx=0).stop 2023-10-18T11:58:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.211+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 shutdown_connections 2023-10-18T11:58:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.211+0000 7f060d4cf640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f060815c7e0 0x7f06080b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.211+0000 7f060d4cf640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06080b5740 0x7f06080cb9a0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.211+0000 7f060d4cf640 1 --2- 172.21.15.125:0/1235082949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f05f40039a0 0x7f060814b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.211+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 >> 172.21.15.125:0/1235082949 conn(0x7f060809a030 msgr2=0x7f060806a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:23.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.212+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 shutdown_connections 2023-10-18T11:58:23.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.212+0000 7f060d4cf640 1 -- 172.21.15.125:0/1235082949 wait complete. 2023-10-18T11:58:23.224 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:58:23.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 -- 172.21.15.125:0/3015137316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 msgr2=0x7f1d6c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 --2- 172.21.15.125:0/3015137316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 0x7f1d6c0705a0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0b1450 tx=0x7f1d5c02d0d0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 -- 172.21.15.125:0/3015137316 shutdown_connections 2023-10-18T11:58:23.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 --2- 172.21.15.125:0/3015137316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 0x7f1d6c0705a0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 --2- 172.21.15.125:0/3015137316 >> v2:172.21.15.139:3300/0 conn(0x7f1d6c0b0530 0x7f1d6c0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 --2- 172.21.15.125:0/3015137316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d6c0cbd20 0x7f1d6c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 -- 172.21.15.125:0/3015137316 >> 172.21.15.125:0/3015137316 conn(0x7f1d6c069930 msgr2=0x7f1d6c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:23.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 -- 172.21.15.125:0/3015137316 shutdown_connections 2023-10-18T11:58:23.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.363+0000 7f1d7233c640 1 -- 172.21.15.125:0/3015137316 wait complete. 2023-10-18T11:58:23.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.364+0000 7f1d7233c640 1 Processor -- start 2023-10-18T11:58:23.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.364+0000 7f1d7233c640 1 -- start start 2023-10-18T11:58:23.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d7233c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 0x7f1d6c1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d7233c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d6c0b0530 0x7f1d6c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d7233c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d6c0cbd20 0x7f1d6c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d7233c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c145de0 con 0x7f1d6c0afa20 2023-10-18T11:58:23.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d7233c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c14c280 con 0x7f1d6c0cbd20 2023-10-18T11:58:23.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d7233c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1523f0 con 0x7f1d6c0b0530 2023-10-18T11:58:23.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 0x7f1d6c1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 0x7f1d6c1458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39922/0 (socket says 172.21.15.125:39922) 2023-10-18T11:58:23.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6bfff640 1 -- 172.21.15.125:0/2010023910 learned_addr learned my addr 172.21.15.125:0/2010023910 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:23.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d708d9640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d6c0cbd20 0x7f1d6c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6b7fe640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d6c0b0530 0x7f1d6c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6bfff640 1 -- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d6c0b0530 msgr2=0x7f1d6c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6bfff640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d6c0b0530 0x7f1d6c151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6bfff640 1 -- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d6c0cbd20 msgr2=0x7f1d6c1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6bfff640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d6c0cbd20 0x7f1d6c1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6bfff640 1 -- 172.21.15.125:0/2010023910 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c00a670 con 0x7f1d6c0afa20 2023-10-18T11:58:23.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d6b7fe640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d6c0b0530 0x7f1d6c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:23.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.365+0000 7f1d708d9640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d6c0cbd20 0x7f1d6c1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:23.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.366+0000 7f1d6bfff640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 0x7f1d6c1458a0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009a90 tx=0x7f1d60009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:23.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.366+0000 7f1d68ff9640 1 -- 172.21.15.125:0/2010023910 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60013070 con 0x7f1d6c0afa20 2023-10-18T11:58:23.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.366+0000 7f1d68ff9640 1 -- 172.21.15.125:0/2010023910 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d6000bb70 con 0x7f1d6c0afa20 2023-10-18T11:58:23.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.366+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c14b670 con 0x7f1d6c0afa20 2023-10-18T11:58:23.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.366+0000 7f1d68ff9640 1 -- 172.21.15.125:0/2010023910 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000e670 con 0x7f1d6c0afa20 2023-10-18T11:58:23.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.366+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c14bc30 con 0x7f1d6c0afa20 2023-10-18T11:58:23.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.367+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c0c96b0 con 0x7f1d6c0afa20 2023-10-18T11:58:23.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.367+0000 7f1d68ff9640 1 -- 172.21.15.125:0/2010023910 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 116) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f1d60017410 con 0x7f1d6c0afa20 2023-10-18T11:58:23.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.368+0000 7f1d68ff9640 1 -- 172.21.15.125:0/2010023910 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7117+0+0 (secure 0 0 0) 0x7f1d6008b450 con 0x7f1d6c0afa20 2023-10-18T11:58:23.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.372+0000 7f1d68ff9640 1 -- 172.21.15.125:0/2010023910 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d60056200 con 0x7f1d6c0afa20 2023-10-18T11:58:23.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.577+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d6c0690e0 con 0x7f1d6c0afa20 2023-10-18T11:58:23.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.577+0000 7f1d68ff9640 1 -- 172.21.15.125:0/2010023910 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f1d600599f0 con 0x7f1d6c0afa20 2023-10-18T11:58:23.580 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:23.581 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":73,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2139029617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2139029617},{"type":"v1","addr":"172.21.15.125:6835","nonce":2139029617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2438879012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2438879012},{"type":"v1","addr":"172.21.15.139:6837","nonce":2438879012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":10830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2643526616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2643526616},{"type":"v1","addr":"172.21.15.125:6834","nonce":2643526616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:42:14.915536+0000","modified":"2023-10-18T11:42:16.934466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6836/3571180970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3571180970},{"type":"v1","addr":"172.21.15.139:6836","nonce":3571180970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T11:58:23.581 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 73 2023-10-18T11:58:23.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 msgr2=0x7f1d6c1458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 0x7f1d6c1458a0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009a90 tx=0x7f1d60009f50 comp rx=0 tx=0).stop 2023-10-18T11:58:23.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 shutdown_connections 2023-10-18T11:58:23.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d6c0cbd20 0x7f1d6c1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d6c0b0530 0x7f1d6c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 --2- 172.21.15.125:0/2010023910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d6c0afa20 0x7f1d6c1458a0 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 >> 172.21.15.125:0/2010023910 conn(0x7f1d6c069930 msgr2=0x7f1d6c0cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:23.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 shutdown_connections 2023-10-18T11:58:23.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.579+0000 7f1d7233c640 1 -- 172.21.15.125:0/2010023910 wait complete. 2023-10-18T11:58:23.592 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:23.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.694+0000 7efc1a301640 1 -- 172.21.15.125:0/3972462186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efbfc0051d0 msgr2=0x7efbfc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.694+0000 7efc1a301640 1 --2- 172.21.15.125:0/3972462186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efbfc0051d0 0x7efbfc00b980 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c4a0 tx=0x7efbfc0057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.694+0000 7efc1a301640 1 -- 172.21.15.125:0/3972462186 shutdown_connections 2023-10-18T11:58:23.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.694+0000 7efc1a301640 1 --2- 172.21.15.125:0/3972462186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efbfc0051d0 0x7efbfc00b980 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.694+0000 7efc1a301640 1 -- 172.21.15.125:0/3972462186 >> 172.21.15.125:0/3972462186 conn(0x7efc140d0d60 msgr2=0x7efc140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:23.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.694+0000 7efc1a301640 1 -- 172.21.15.125:0/3972462186 shutdown_connections 2023-10-18T11:58:23.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.694+0000 7efc1a301640 1 -- 172.21.15.125:0/3972462186 wait complete. 2023-10-18T11:58:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.695+0000 7efc1a301640 1 Processor -- start 2023-10-18T11:58:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.695+0000 7efc1a301640 1 -- start start 2023-10-18T11:58:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1a301640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efbfc0051d0 0x7efc140b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1a301640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc14158630 0x7efc140c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1a301640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc14158ab0 0x7efc140ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1a301640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc14151ce0 con 0x7efc14158ab0 2023-10-18T11:58:23.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1a301640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b4c40 con 0x7efc14158630 2023-10-18T11:58:23.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1a301640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b3950 con 0x7efbfc0051d0 2023-10-18T11:58:23.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc14158ab0 0x7efc140ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc14158ab0 0x7efc140ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39936/0 (socket says 172.21.15.125:39936) 2023-10-18T11:58:23.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 -- 172.21.15.125:0/2013684094 learned_addr learned my addr 172.21.15.125:0/2013684094 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:23.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc137fe640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc14158630 0x7efc140c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc13fff640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efbfc0051d0 0x7efc140b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 -- 172.21.15.125:0/2013684094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efbfc0051d0 msgr2=0x7efc140b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efbfc0051d0 0x7efc140b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 -- 172.21.15.125:0/2013684094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc14158630 msgr2=0x7efc140c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc14158630 0x7efc140c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 -- 172.21.15.125:0/2013684094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140b47b0 con 0x7efc14158ab0 2023-10-18T11:58:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.696+0000 7efc1889e640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc14158ab0 0x7efc140ca380 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7efc0c02cc00 tx=0x7efc0c001870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.697+0000 7efc10ff9640 1 -- 172.21.15.125:0/2013684094 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c001a60 con 0x7efc14158ab0 2023-10-18T11:58:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.697+0000 7efc10ff9640 1 -- 172.21.15.125:0/2013684094 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc0c003970 con 0x7efc14158ab0 2023-10-18T11:58:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.697+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc140b5ff0 con 0x7efc14158ab0 2023-10-18T11:58:23.705 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.697+0000 7efc10ff9640 1 -- 172.21.15.125:0/2013684094 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c003d80 con 0x7efc14158ab0 2023-10-18T11:58:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.697+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc140b5ff0 con 0x7efc14158ab0 2023-10-18T11:58:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.698+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc140b5ff0 con 0x7efc14158ab0 2023-10-18T11:58:23.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.698+0000 7efc10ff9640 1 -- 172.21.15.125:0/2013684094 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 116) v1 ==== 88940+0+0 (secure 0 0 0) 0x7efc0c031030 con 0x7efc14158ab0 2023-10-18T11:58:23.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.699+0000 7efc10ff9640 1 -- 172.21.15.125:0/2013684094 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7117+0+0 (secure 0 0 0) 0x7efc0c0aa6a0 con 0x7efc14158ab0 2023-10-18T11:58:23.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.703+0000 7efc10ff9640 1 -- 172.21.15.125:0/2013684094 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc0c075620 con 0x7efc14158ab0 2023-10-18T11:58:23.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.868+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc140b4940 con 0x7efc14158ab0 2023-10-18T11:58:23.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.870+0000 7efc10ff9640 1 -- 172.21.15.125:0/2013684094 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+18400 (secure 0 0 0) 0x7efc0c07f020 con 0x7efc14158ab0 2023-10-18T11:58:23.873 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:23.873 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":193,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:58:22.877022+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:42:12.758545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-18T11:42:13.894048+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":192,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3199187177":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/2966502376":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6833/833207676":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6832/833207676":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/2823163906":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6833/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1424135438":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6832/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1872946902":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1126366928":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1293875531":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/608662592":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/141746223":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1253211552":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/3751642323":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/402965407":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6833/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6832/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/4091087182":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/1330912710":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/697097533":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/1787628043":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/4016867169":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/878133557":"2023-10-19T11:42:31.009948+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:23.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.871+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc14158ab0 msgr2=0x7efc140ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.871+0000 7efc1a301640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc14158ab0 0x7efc140ca380 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7efc0c02cc00 tx=0x7efc0c001870 comp rx=0 tx=0).stop 2023-10-18T11:58:23.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.871+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 shutdown_connections 2023-10-18T11:58:23.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.871+0000 7efc1a301640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc14158630 0x7efc140c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.871+0000 7efc1a301640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efc14158ab0 0x7efc140ca380 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.871+0000 7efc1a301640 1 --2- 172.21.15.125:0/2013684094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efbfc0051d0 0x7efc140b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.871+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 >> 172.21.15.125:0/2013684094 conn(0x7efc140d0d60 msgr2=0x7efc140b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:23.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.872+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 shutdown_connections 2023-10-18T11:58:23.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.872+0000 7efc1a301640 1 -- 172.21.15.125:0/2013684094 wait complete. 2023-10-18T11:58:23.885 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T11:58:23.885 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:58:23.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.986+0000 7f778ea75640 1 -- 172.21.15.125:0/3898360837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 msgr2=0x7f7778004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.986+0000 7f778ea75640 1 --2- 172.21.15.125:0/3898360837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 0x7f7778004c40 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f7774006f60 tx=0x7f77780057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.986+0000 7f778ea75640 1 -- 172.21.15.125:0/3898360837 shutdown_connections 2023-10-18T11:58:23.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.986+0000 7f778ea75640 1 --2- 172.21.15.125:0/3898360837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 0x7f7778004c40 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.986+0000 7f778ea75640 1 -- 172.21.15.125:0/3898360837 >> 172.21.15.125:0/3898360837 conn(0x7f7788069930 msgr2=0x7f7788069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:23.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.986+0000 7f778ea75640 1 -- 172.21.15.125:0/3898360837 shutdown_connections 2023-10-18T11:58:23.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.986+0000 7f778ea75640 1 -- 172.21.15.125:0/3898360837 wait complete. 2023-10-18T11:58:23.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.987+0000 7f778ea75640 1 Processor -- start 2023-10-18T11:58:23.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.987+0000 7f778ea75640 1 -- start start 2023-10-18T11:58:23.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778ea75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 0x7f778813e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778ea75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778814f2c0 0x7f7788151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778ea75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77881523d0 0x7f77881e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778ea75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77881534f0 con 0x7f77780047c0 2023-10-18T11:58:23.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778ea75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7788152870 con 0x7f778814f2c0 2023-10-18T11:58:23.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778ea75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7788152b50 con 0x7f77881523d0 2023-10-18T11:58:23.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778c811640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 0x7f778813e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778c811640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 0x7f778813e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39952/0 (socket says 172.21.15.125:39952) 2023-10-18T11:58:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778c811640 1 -- 172.21.15.125:0/2543048543 learned_addr learned my addr 172.21.15.125:0/2543048543 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f7787fff640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778814f2c0 0x7f7788151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778d012640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77881523d0 0x7f77881e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778c811640 1 -- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77881523d0 msgr2=0x7f77881e51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778c811640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77881523d0 0x7f77881e51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778c811640 1 -- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778814f2c0 msgr2=0x7f7788151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778c811640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778814f2c0 0x7f7788151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.988+0000 7f778c811640 1 -- 172.21.15.125:0/2543048543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788152e30 con 0x7f77780047c0 2023-10-18T11:58:23.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.989+0000 7f778c811640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 0x7f778813e490 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f777c003dd0 tx=0x7f777c001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:23.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.989+0000 7f77857fa640 1 -- 172.21.15.125:0/2543048543 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0060d0 con 0x7f77780047c0 2023-10-18T11:58:23.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.989+0000 7f77857fa640 1 -- 172.21.15.125:0/2543048543 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f777c011840 con 0x7f77780047c0 2023-10-18T11:58:23.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.989+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7788153110 con 0x7f77780047c0 2023-10-18T11:58:23.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.989+0000 7f77857fa640 1 -- 172.21.15.125:0/2543048543 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c011c50 con 0x7f77780047c0 2023-10-18T11:58:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.989+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77881e5710 con 0x7f77780047c0 2023-10-18T11:58:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.990+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7788153630 con 0x7f77780047c0 2023-10-18T11:58:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.990+0000 7f77857fa640 1 -- 172.21.15.125:0/2543048543 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 117) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f777c0103c0 con 0x7f77780047c0 2023-10-18T11:58:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.991+0000 7f77857fa640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f776c06ac90 0x7f776c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.991+0000 7f77857fa640 1 -- 172.21.15.125:0/2543048543 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7117+0+0 (secure 0 0 0) 0x7f777c08a640 con 0x7f77780047c0 2023-10-18T11:58:24.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.991+0000 7f7787fff640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f776c06ac90 0x7f776c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.991+0000 7f7787fff640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f776c06ac90 0x7f776c06d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7774034000 tx=0x7f77740019f0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:24.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:23.995+0000 7f77857fa640 1 -- 172.21.15.125:0/2543048543 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f777c055560 con 0x7f77780047c0 2023-10-18T11:58:24.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.194+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77880690e0 con 0x7f77780047c0 2023-10-18T11:58:24.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.195+0000 7f77857fa640 1 -- 172.21.15.125:0/2543048543 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f777c015020 con 0x7f77780047c0 2023-10-18T11:58:24.198 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:24.198 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":73,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2139029617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2139029617},{"type":"v1","addr":"172.21.15.125:6835","nonce":2139029617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2438879012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2438879012},{"type":"v1","addr":"172.21.15.139:6837","nonce":2438879012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":10830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2643526616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2643526616},{"type":"v1","addr":"172.21.15.125:6834","nonce":2643526616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:42:14.915536+0000","modified":"2023-10-18T11:42:16.934466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6836/3571180970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3571180970},{"type":"v1","addr":"172.21.15.139:6836","nonce":3571180970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T11:58:24.200 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 73 2023-10-18T11:58:24.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f776c06ac90 msgr2=0x7f776c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f776c06ac90 0x7f776c06d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7774034000 tx=0x7f77740019f0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 msgr2=0x7f778813e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 0x7f778813e490 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f777c003dd0 tx=0x7f777c001770 comp rx=0 tx=0).stop 2023-10-18T11:58:24.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 shutdown_connections 2023-10-18T11:58:24.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f776c06ac90 0x7f776c06d150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f778814f2c0 0x7f7788151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f77881523d0 0x7f77881e51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 --2- 172.21.15.125:0/2543048543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77780047c0 0x7f778813e490 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 >> 172.21.15.125:0/2543048543 conn(0x7f7788069930 msgr2=0x7f7788069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:24.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 shutdown_connections 2023-10-18T11:58:24.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.197+0000 7f778ea75640 1 -- 172.21.15.125:0/2543048543 wait complete. 2023-10-18T11:58:24.210 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:58:24.312 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.309+0000 7f6170044640 1 -- 172.21.15.125:0/2154303878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 msgr2=0x7f616806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.309+0000 7f6170044640 1 --2- 172.21.15.125:0/2154303878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 0x7f616806b2e0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f6164005fd0 tx=0x7f616402bd10 comp rx=0 tx=0).stop 2023-10-18T11:58:24.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.309+0000 7f6170044640 1 -- 172.21.15.125:0/2154303878 shutdown_connections 2023-10-18T11:58:24.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.309+0000 7f6170044640 1 --2- 172.21.15.125:0/2154303878 >> v2:172.21.15.139:3300/0 conn(0x7f61680abc10 0x7f61680c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.309+0000 7f6170044640 1 --2- 172.21.15.125:0/2154303878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 0x7f616806b2e0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.309+0000 7f6170044640 1 -- 172.21.15.125:0/2154303878 >> 172.21.15.125:0/2154303878 conn(0x7f6168069930 msgr2=0x7f6168069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:24.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.310+0000 7f6170044640 1 -- 172.21.15.125:0/2154303878 shutdown_connections 2023-10-18T11:58:24.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.310+0000 7f6170044640 1 -- 172.21.15.125:0/2154303878 wait complete. 2023-10-18T11:58:24.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f6170044640 1 Processor -- start 2023-10-18T11:58:24.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f6170044640 1 -- start start 2023-10-18T11:58:24.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f6170044640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 0x7f6168151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f6170044640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61680abc10 0x7f6168136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f6170044640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61680c96b0 0x7f6168138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f6170044640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61681ef2e0 con 0x7f616806af00 2023-10-18T11:58:24.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f6170044640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61681365c0 con 0x7f61680c96b0 2023-10-18T11:58:24.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f6170044640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61680b0300 con 0x7f61680abc10 2023-10-18T11:58:24.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.311+0000 7f616dde0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 0x7f6168151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616dde0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 0x7f6168151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39968/0 (socket says 172.21.15.125:39968) 2023-10-18T11:58:24.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616dde0640 1 -- 172.21.15.125:0/3515160748 learned_addr learned my addr 172.21.15.125:0/3515160748 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:24.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616e5e1640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61680c96b0 0x7f6168138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616d5df640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61680abc10 0x7f6168136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616dde0640 1 -- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61680abc10 msgr2=0x7f6168136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616dde0640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61680abc10 0x7f6168136760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616dde0640 1 -- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61680c96b0 msgr2=0x7f6168138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616dde0640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61680c96b0 0x7f6168138fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616dde0640 1 -- 172.21.15.125:0/3515160748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164009560 con 0x7f616806af00 2023-10-18T11:58:24.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616e5e1640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61680c96b0 0x7f6168138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:24.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616d5df640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61680abc10 0x7f6168136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:24.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f616dde0640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 0x7f6168151990 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f6164009850 tx=0x7f616402da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:24.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f615e7fc640 1 -- 172.21.15.125:0/3515160748 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616402de90 con 0x7f616806af00 2023-10-18T11:58:24.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f615e7fc640 1 -- 172.21.15.125:0/3515160748 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f61640036e0 con 0x7f616806af00 2023-10-18T11:58:24.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f615e7fc640 1 -- 172.21.15.125:0/3515160748 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164003a90 con 0x7f616806af00 2023-10-18T11:58:24.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61681362e0 con 0x7f616806af00 2023-10-18T11:58:24.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.312+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61680b0880 con 0x7f616806af00 2023-10-18T11:58:24.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.313+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61680b0160 con 0x7f616806af00 2023-10-18T11:58:24.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.314+0000 7f615e7fc640 1 -- 172.21.15.125:0/3515160748 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 117) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6164003bf0 con 0x7f616806af00 2023-10-18T11:58:24.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.314+0000 7f615e7fc640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f614c06ac90 0x7f614c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.314+0000 7f615e7fc640 1 -- 172.21.15.125:0/3515160748 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7117+0+0 (secure 0 0 0) 0x7f61640ac4b0 con 0x7f616806af00 2023-10-18T11:58:24.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.314+0000 7f616d5df640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f614c06ac90 0x7f614c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.315+0000 7f616d5df640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f614c06ac90 0x7f614c06d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f616806a290 tx=0x7f6158008040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:24.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.318+0000 7f615e7fc640 1 -- 172.21.15.125:0/3515160748 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61680b0160 con 0x7f616806af00 2023-10-18T11:58:24.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.521+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61680b0160 con 0x7f616806af00 2023-10-18T11:58:24.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.521+0000 7f615e7fc640 1 -- 172.21.15.125:0/3515160748 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f61680b0160 con 0x7f616806af00 2023-10-18T11:58:24.524 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:24.525 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":73,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2139029617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2139029617},{"type":"v1","addr":"172.21.15.125:6835","nonce":2139029617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2438879012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2438879012},{"type":"v1","addr":"172.21.15.139:6837","nonce":2438879012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":10830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2643526616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2643526616},{"type":"v1","addr":"172.21.15.125:6834","nonce":2643526616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:42:14.915536+0000","modified":"2023-10-18T11:42:16.934466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6836/3571180970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3571180970},{"type":"v1","addr":"172.21.15.139:6836","nonce":3571180970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T11:58:24.526 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 73 2023-10-18T11:58:24.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f614c06ac90 msgr2=0x7f614c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f614c06ac90 0x7f614c06d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f616806a290 tx=0x7f6158008040 comp rx=0 tx=0).stop 2023-10-18T11:58:24.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 msgr2=0x7f6168151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 0x7f6168151990 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f6164009850 tx=0x7f616402da90 comp rx=0 tx=0).stop 2023-10-18T11:58:24.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 shutdown_connections 2023-10-18T11:58:24.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f614c06ac90 0x7f614c06d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61680c96b0 0x7f6168138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61680abc10 0x7f6168136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 --2- 172.21.15.125:0/3515160748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f616806af00 0x7f6168151990 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 >> 172.21.15.125:0/3515160748 conn(0x7f6168069930 msgr2=0x7f616815b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:24.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 shutdown_connections 2023-10-18T11:58:24.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.523+0000 7f6170044640 1 -- 172.21.15.125:0/3515160748 wait complete. 2023-10-18T11:58:24.536 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:24.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.634+0000 7fa6001f5640 1 -- 172.21.15.125:0/1647587910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80afa20 msgr2=0x7fa5f8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.634+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1647587910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80afa20 0x7fa5f8158300 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80b10f0 tx=0x7fa5e802bd70 comp rx=0 tx=0).stop 2023-10-18T11:58:24.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.634+0000 7fa6001f5640 1 -- 172.21.15.125:0/1647587910 shutdown_connections 2023-10-18T11:58:24.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.634+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1647587910 >> v2:172.21.15.139:3300/0 conn(0x7fa5f80b0530 0x7fa5f80b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.634+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1647587910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80afa20 0x7fa5f8158300 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.634+0000 7fa6001f5640 1 -- 172.21.15.125:0/1647587910 >> 172.21.15.125:0/1647587910 conn(0x7fa5f8069930 msgr2=0x7fa5f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:24.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.635+0000 7fa6001f5640 1 -- 172.21.15.125:0/1647587910 shutdown_connections 2023-10-18T11:58:24.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.635+0000 7fa6001f5640 1 -- 172.21.15.125:0/1647587910 wait complete. 2023-10-18T11:58:24.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa6001f5640 1 Processor -- start 2023-10-18T11:58:24.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa6001f5640 1 -- start start 2023-10-18T11:58:24.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa6001f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80afa20 0x7fa5f813c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa6001f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5f80b0530 0x7fa5f80ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa6001f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80cbd20 0x7fa5f80abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa6001f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f813dae0 con 0x7fa5f80cbd20 2023-10-18T11:58:24.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa6001f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f813d1d0 con 0x7fa5f80afa20 2023-10-18T11:58:24.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa6001f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f813d490 con 0x7fa5f80b0530 2023-10-18T11:58:24.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.636+0000 7fa5fe792640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80cbd20 0x7fa5f80abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fe792640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80cbd20 0x7fa5f80abe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39998/0 (socket says 172.21.15.125:39998) 2023-10-18T11:58:24.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fe792640 1 -- 172.21.15.125:0/1626337091 learned_addr learned my addr 172.21.15.125:0/1626337091 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:24.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fdf91640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80afa20 0x7fa5f813c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fd790640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5f80b0530 0x7fa5f80ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fe792640 1 -- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5f80b0530 msgr2=0x7fa5f80ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fe792640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5f80b0530 0x7fa5f80ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fe792640 1 -- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80afa20 msgr2=0x7fa5f813c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fe792640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80afa20 0x7fa5f813c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fe792640 1 -- 172.21.15.125:0/1626337091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e8009490 con 0x7fa5f80cbd20 2023-10-18T11:58:24.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5fe792640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80cbd20 0x7fa5f80abe70 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000cad0 tx=0x7fa5f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:24.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5ee7fc640 1 -- 172.21.15.125:0/1626337091 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000ae20 con 0x7fa5f80cbd20 2023-10-18T11:58:24.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5ee7fc640 1 -- 172.21.15.125:0/1626337091 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa5f0004b20 con 0x7fa5f80cbd20 2023-10-18T11:58:24.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f813d770 con 0x7fa5f80cbd20 2023-10-18T11:58:24.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa5ee7fc640 1 -- 172.21.15.125:0/1626337091 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0005020 con 0x7fa5f80cbd20 2023-10-18T11:58:24.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.637+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f80a8f80 con 0x7fa5f80cbd20 2023-10-18T11:58:24.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.638+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f813dc50 con 0x7fa5f80cbd20 2023-10-18T11:58:24.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.639+0000 7fa5ee7fc640 1 -- 172.21.15.125:0/1626337091 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 117) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa5f0005180 con 0x7fa5f80cbd20 2023-10-18T11:58:24.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.639+0000 7fa5ee7fc640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fa5dc06ac90 0x7fa5dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.639+0000 7fa5ee7fc640 1 -- 172.21.15.125:0/1626337091 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7117+0+0 (secure 0 0 0) 0x7fa5f008a010 con 0x7fa5f80cbd20 2023-10-18T11:58:24.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.639+0000 7fa5fdf91640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fa5dc06ac90 0x7fa5dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.640+0000 7fa5fdf91640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fa5dc06ac90 0x7fa5dc06d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4000f60 tx=0x7fa5f4009040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:24.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.643+0000 7fa5ee7fc640 1 -- 172.21.15.125:0/1626337091 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5f0054ea0 con 0x7fa5f80cbd20 2023-10-18T11:58:24.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.808+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5f80690e0 con 0x7fa5f80cbd20 2023-10-18T11:58:24.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.809+0000 7fa5ee7fc640 1 -- 172.21.15.125:0/1626337091 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+18400 (secure 0 0 0) 0x7fa5f001d030 con 0x7fa5f80cbd20 2023-10-18T11:58:24.813 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:24.813 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":193,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:58:22.877022+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:42:12.758545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-18T11:42:13.894048+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":192,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3199187177":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/2966502376":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6833/833207676":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6832/833207676":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/2823163906":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6833/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1424135438":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6832/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1872946902":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1126366928":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1293875531":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/608662592":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/141746223":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1253211552":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/3751642323":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/402965407":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6833/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6832/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/4091087182":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/1330912710":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/697097533":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/1787628043":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/4016867169":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/878133557":"2023-10-19T11:42:31.009948+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:24.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fa5dc06ac90 msgr2=0x7fa5dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fa5dc06ac90 0x7fa5dc06d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4000f60 tx=0x7fa5f4009040 comp rx=0 tx=0).stop 2023-10-18T11:58:24.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80cbd20 msgr2=0x7fa5f80abe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80cbd20 0x7fa5f80abe70 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000cad0 tx=0x7fa5f000cf90 comp rx=0 tx=0).stop 2023-10-18T11:58:24.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 shutdown_connections 2023-10-18T11:58:24.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fa5dc06ac90 0x7fa5dc06d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f80cbd20 0x7fa5f80abe70 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5f80b0530 0x7fa5f80ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 --2- 172.21.15.125:0/1626337091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80afa20 0x7fa5f813c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 >> 172.21.15.125:0/1626337091 conn(0x7fa5f8069930 msgr2=0x7fa5f8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:24.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 shutdown_connections 2023-10-18T11:58:24.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.811+0000 7fa6001f5640 1 -- 172.21.15.125:0/1626337091 wait complete. 2023-10-18T11:58:24.824 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T11:58:24.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.927+0000 7f357f0e5640 1 -- 172.21.15.125:0/1072798093 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 msgr2=0x7f35680047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.927+0000 7f357f0e5640 1 --2- 172.21.15.125:0/1072798093 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 0x7f35680047c0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f3564001800 tx=0x7f35680057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.927+0000 7f357d682640 1 -- 172.21.15.125:0/1072798093 reap_dead start 2023-10-18T11:58:24.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.927+0000 7f357f0e5640 1 -- 172.21.15.125:0/1072798093 shutdown_connections 2023-10-18T11:58:24.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.927+0000 7f357f0e5640 1 -- 172.21.15.125:0/1072798093 >> 172.21.15.125:0/1072798093 conn(0x7f3578069930 msgr2=0x7f3578069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:24.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.927+0000 7f357f0e5640 1 -- 172.21.15.125:0/1072798093 shutdown_connections 2023-10-18T11:58:24.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.928+0000 7f357f0e5640 1 -- 172.21.15.125:0/1072798093 wait complete. 2023-10-18T11:58:24.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.929+0000 7f357f0e5640 1 Processor -- start 2023-10-18T11:58:24.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.929+0000 7f357f0e5640 1 -- start start 2023-10-18T11:58:24.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.929+0000 7f357f0e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 0x7f3578136280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.929+0000 7f357f0e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3578155f70 0x7f35781542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.929+0000 7f357f0e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3578154820 0x7f35780af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.929+0000 7f357f0e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3578154cb0 con 0x7f35680040e0 2023-10-18T11:58:24.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.929+0000 7f357f0e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35781367c0 con 0x7f3578154820 2023-10-18T11:58:24.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.929+0000 7f357f0e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35780ac060 con 0x7f3578155f70 2023-10-18T11:58:24.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 0x7f3578136280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 0x7f3578136280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40014/0 (socket says 172.21.15.125:40014) 2023-10-18T11:58:24.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 -- 172.21.15.125:0/1956431315 learned_addr learned my addr 172.21.15.125:0/1956431315 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:24.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357d682640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3578154820 0x7f35780af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f3577fff640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3578155f70 0x7f35781542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 -- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3578155f70 msgr2=0x7f35781542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3578155f70 0x7f35781542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 -- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3578154820 msgr2=0x7f35780af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:24.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3578154820 0x7f35780af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:24.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 -- 172.21.15.125:0/1956431315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780ab450 con 0x7f35680040e0 2023-10-18T11:58:24.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357d682640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3578154820 0x7f35780af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:24.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357ce81640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 0x7f3578136280 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f356c011a50 tx=0x7f356c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:24.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f35757fa640 1 -- 172.21.15.125:0/1956431315 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00fcd0 con 0x7f35680040e0 2023-10-18T11:58:24.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f35757fa640 1 -- 172.21.15.125:0/1956431315 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f356c012450 con 0x7f35680040e0 2023-10-18T11:58:24.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35780ab730 con 0x7f35680040e0 2023-10-18T11:58:24.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f35757fa640 1 -- 172.21.15.125:0/1956431315 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c012840 con 0x7f35680040e0 2023-10-18T11:58:24.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.930+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35780abcf0 con 0x7f35680040e0 2023-10-18T11:58:24.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.931+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357814f930 con 0x7f35680040e0 2023-10-18T11:58:24.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.932+0000 7f35757fa640 1 -- 172.21.15.125:0/1956431315 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 117) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f356c0129a0 con 0x7f35680040e0 2023-10-18T11:58:24.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.932+0000 7f35757fa640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f355c06ac90 0x7f355c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:24.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.932+0000 7f35757fa640 1 -- 172.21.15.125:0/1956431315 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7117+0+0 (secure 0 0 0) 0x7f356c08a550 con 0x7f35680040e0 2023-10-18T11:58:24.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.932+0000 7f3577fff640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f355c06ac90 0x7f355c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:24.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.933+0000 7f3577fff640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f355c06ac90 0x7f355c06d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3564034000 tx=0x7f3564039040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:24.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:24.936+0000 7f35757fa640 1 -- 172.21.15.125:0/1956431315 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f356c055470 con 0x7f35680040e0 2023-10-18T11:58:25.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.139+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f35780690e0 con 0x7f35680040e0 2023-10-18T11:58:25.142 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T11:58:25.139+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T11:58:25.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.141+0000 7f35757fa640 1 -- 172.21.15.125:0/1956431315 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7f356c01f090 con 0x7f35680040e0 2023-10-18T11:58:25.144 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T11:58:25.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f355c06ac90 msgr2=0x7f355c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:25.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f355c06ac90 0x7f355c06d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3564034000 tx=0x7f3564039040 comp rx=0 tx=0).stop 2023-10-18T11:58:25.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 msgr2=0x7f3578136280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:25.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 0x7f3578136280 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f356c011a50 tx=0x7f356c011f10 comp rx=0 tx=0).stop 2023-10-18T11:58:25.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 shutdown_connections 2023-10-18T11:58:25.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f355c06ac90 0x7f355c06d150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3578154820 0x7f35780af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3578155f70 0x7f35781542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 --2- 172.21.15.125:0/1956431315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35680040e0 0x7f3578136280 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 >> 172.21.15.125:0/1956431315 conn(0x7f3578069930 msgr2=0x7f35780ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:25.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 shutdown_connections 2023-10-18T11:58:25.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.143+0000 7f357f0e5640 1 -- 172.21.15.125:0/1956431315 wait complete. 2023-10-18T11:58:25.156 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T11:58:25.184 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T11:58:25.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 -- 172.21.15.125:0/725399200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840cbd20 msgr2=0x7f3584158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:25.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 --2- 172.21.15.125:0/725399200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840cbd20 0x7f3584158300 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f35840b1450 tx=0x7f357402bda0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 -- 172.21.15.125:0/725399200 shutdown_connections 2023-10-18T11:58:25.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 --2- 172.21.15.125:0/725399200 >> v2:172.21.15.139:3300/0 conn(0x7f35840b0530 0x7f35840b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 --2- 172.21.15.125:0/725399200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35840afa20 0x7f35840705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 --2- 172.21.15.125:0/725399200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840cbd20 0x7f3584158300 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 -- 172.21.15.125:0/725399200 >> 172.21.15.125:0/725399200 conn(0x7f3584069930 msgr2=0x7f3584069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:25.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 -- 172.21.15.125:0/725399200 shutdown_connections 2023-10-18T11:58:25.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.258+0000 7f358a407640 1 -- 172.21.15.125:0/725399200 wait complete. 2023-10-18T11:58:25.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.259+0000 7f358a407640 1 Processor -- start 2023-10-18T11:58:25.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.259+0000 7f358a407640 1 -- start start 2023-10-18T11:58:25.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f358a407640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840afa20 0x7f358413d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:25.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f358a407640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35840b0530 0x7f3584141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:25.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f358a407640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35840cbd20 0x7f358413f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:25.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f358a407640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358413efd0 con 0x7f35840afa20 2023-10-18T11:58:25.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f358a407640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3584158b20 con 0x7f35840b0530 2023-10-18T11:58:25.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f358a407640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358413e730 con 0x7f35840cbd20 2023-10-18T11:58:25.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840afa20 0x7f358413d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:25.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840afa20 0x7f358413d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40030/0 (socket says 172.21.15.125:40030) 2023-10-18T11:58:25.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 -- 172.21.15.125:0/3131015569 learned_addr learned my addr 172.21.15.125:0/3131015569 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:25.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f35889a4640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35840cbd20 0x7f358413f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:25.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f35837fe640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35840b0530 0x7f3584141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:25.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 -- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35840cbd20 msgr2=0x7f358413f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:25.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35840cbd20 0x7f358413f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 -- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35840b0530 msgr2=0x7f3584141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:25.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35840b0530 0x7f3584141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:25.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 -- 172.21.15.125:0/3131015569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574009560 con 0x7f35840afa20 2023-10-18T11:58:25.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3583fff640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840afa20 0x7f358413d3f0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f3574000c00 tx=0x7f357402db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:25.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3580ff9640 1 -- 172.21.15.125:0/3131015569 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574003800 con 0x7f35840afa20 2023-10-18T11:58:25.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3580ff9640 1 -- 172.21.15.125:0/3131015569 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3574003a50 con 0x7f35840afa20 2023-10-18T11:58:25.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358413ea10 con 0x7f35840afa20 2023-10-18T11:58:25.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f3580ff9640 1 -- 172.21.15.125:0/3131015569 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574003e60 con 0x7f35840afa20 2023-10-18T11:58:25.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.260+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35841477f0 con 0x7f35840afa20 2023-10-18T11:58:25.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.261+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3584158980 con 0x7f35840afa20 2023-10-18T11:58:25.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.262+0000 7f3580ff9640 1 -- 172.21.15.125:0/3131015569 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 117) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f357403e450 con 0x7f35840afa20 2023-10-18T11:58:25.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.262+0000 7f3580ff9640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357006ac90 0x7f357006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:25.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.263+0000 7f3580ff9640 1 -- 172.21.15.125:0/3131015569 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7203+0+0 (secure 0 0 0) 0x7f35740ac1b0 con 0x7f35840afa20 2023-10-18T11:58:25.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.263+0000 7f35837fe640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357006ac90 0x7f357006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:25.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.263+0000 7f35837fe640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357006ac90 0x7f357006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35841415a0 tx=0x7f357800b2e0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:25.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.267+0000 7f3580ff9640 1 -- 172.21.15.125:0/3131015569 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3584158980 con 0x7f35840afa20 2023-10-18T11:58:25.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.470+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3584158980 con 0x7f35840afa20 2023-10-18T11:58:25.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:25.893+0000 7f3580ff9640 1 -- 172.21.15.125:0/3131015569 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 118) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3574077440 con 0x7f35840afa20 2023-10-18T11:58:26.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.144+0000 7f3580ff9640 1 -- 172.21.15.125:0/3131015569 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f35740a50b0 con 0x7f35840afa20 2023-10-18T11:58:26.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357006ac90 msgr2=0x7f357006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:26.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357006ac90 0x7f357006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35841415a0 tx=0x7f357800b2e0 comp rx=0 tx=0).stop 2023-10-18T11:58:26.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840afa20 msgr2=0x7f358413d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:26.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840afa20 0x7f358413d3f0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f3574000c00 tx=0x7f357402db40 comp rx=0 tx=0).stop 2023-10-18T11:58:26.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 shutdown_connections 2023-10-18T11:58:26.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357006ac90 0x7f357006d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:26.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35840b0530 0x7f3584141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:26.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35840cbd20 0x7f358413f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:26.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 --2- 172.21.15.125:0/3131015569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35840afa20 0x7f358413d3f0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:26.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 >> 172.21.15.125:0/3131015569 conn(0x7f3584069930 msgr2=0x7f3584069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:26.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 shutdown_connections 2023-10-18T11:58:26.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.147+0000 7f358a407640 1 -- 172.21.15.125:0/3131015569 wait complete. 2023-10-18T11:58:26.164 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T11:58:26.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.272+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/897536290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 msgr2=0x7fd188005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:26.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.272+0000 7fd1a5cf2640 1 --2- 172.21.15.125:0/897536290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 0x7fd188005180 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c640 tx=0x7fd1880057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:26.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.272+0000 7fd19ffff640 1 -- 172.21.15.125:0/897536290 reap_dead start 2023-10-18T11:58:26.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.272+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/897536290 shutdown_connections 2023-10-18T11:58:26.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.272+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/897536290 >> 172.21.15.125:0/897536290 conn(0x7fd1a0069930 msgr2=0x7fd1a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:26.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.272+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/897536290 shutdown_connections 2023-10-18T11:58:26.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.272+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/897536290 wait complete. 2023-10-18T11:58:26.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.273+0000 7fd1a5cf2640 1 Processor -- start 2023-10-18T11:58:26.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd1a5cf2640 1 -- start start 2023-10-18T11:58:26.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd1a5cf2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 0x7fd1a013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:26.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd1a5cf2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1a0140550 0x7fd1a01409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:26.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd1a5cf2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1a0142510 0x7fd1a0142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:26.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd1a5cf2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0140f10 con 0x7fd188004d00 2023-10-18T11:58:26.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd1a5cf2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a013fca0 con 0x7fd1a0142510 2023-10-18T11:58:26.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd1a5cf2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0142ed0 con 0x7fd1a0140550 2023-10-18T11:58:26.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 0x7fd1a013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:26.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd19f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 0x7fd1a013f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40042/0 (socket says 172.21.15.125:40042) 2023-10-18T11:58:26.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.274+0000 7fd19f7fe640 1 -- 172.21.15.125:0/1641355238 learned_addr learned my addr 172.21.15.125:0/1641355238 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:26.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd19f7fe640 1 -- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1a0140550 msgr2=0x7fd1a01409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:26.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd19f7fe640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1a0140550 0x7fd1a01409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:26.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd19f7fe640 1 -- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1a0142510 msgr2=0x7fd1a0142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:26.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd19f7fe640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1a0142510 0x7fd1a0142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:26.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd19f7fe640 1 -- 172.21.15.125:0/1641355238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a0143180 con 0x7fd188004d00 2023-10-18T11:58:26.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd19f7fe640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 0x7fd1a013f760 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fd19000f9d0 tx=0x7fd19000fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:26.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd187fff640 1 -- 172.21.15.125:0/1641355238 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19000bc90 con 0x7fd188004d00 2023-10-18T11:58:26.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd187fff640 1 -- 172.21.15.125:0/1641355238 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd190004020 con 0x7fd188004d00 2023-10-18T11:58:26.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd187fff640 1 -- 172.21.15.125:0/1641355238 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1900043d0 con 0x7fd188004d00 2023-10-18T11:58:26.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a0144430 con 0x7fd188004d00 2023-10-18T11:58:26.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.275+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a01449f0 con 0x7fd188004d00 2023-10-18T11:58:26.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.276+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a00690e0 con 0x7fd188004d00 2023-10-18T11:58:26.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.276+0000 7fd187fff640 1 -- 172.21.15.125:0/1641355238 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 118) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd1900035f0 con 0x7fd188004d00 2023-10-18T11:58:26.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.277+0000 7fd187fff640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd18c06aed0 0x7fd18c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:26.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.277+0000 7fd187fff640 1 -- 172.21.15.125:0/1641355238 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7203+0+0 (secure 0 0 0) 0x7fd19008a970 con 0x7fd188004d00 2023-10-18T11:58:26.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.277+0000 7fd19effd640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd18c06aed0 0x7fd18c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:26.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.277+0000 7fd19effd640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd18c06aed0 0x7fd18c06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0148950 tx=0x7fd1940040a0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:26.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.281+0000 7fd187fff640 1 -- 172.21.15.125:0/1641355238 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd190055340 con 0x7fd188004d00 2023-10-18T11:58:26.361 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:58:26.364 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:58:26.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:26.450+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd1a006f700 con 0x7fd188004d00 2023-10-18T11:58:27.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.445+0000 7fd187fff640 1 -- 172.21.15.125:0/1641355238 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v195) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd190058960 con 0x7fd188004d00 2023-10-18T11:58:27.447 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T11:58:27.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.447+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd18c06aed0 msgr2=0x7fd18c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:27.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.447+0000 7fd1a5cf2640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd18c06aed0 0x7fd18c06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0148950 tx=0x7fd1940040a0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.447+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 msgr2=0x7fd1a013f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.447+0000 7fd1a5cf2640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 0x7fd1a013f760 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fd19000f9d0 tx=0x7fd19000fe90 comp rx=0 tx=0).stop 2023-10-18T11:58:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.448+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 shutdown_connections 2023-10-18T11:58:27.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.448+0000 7fd1a5cf2640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd18c06aed0 0x7fd18c06d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.448+0000 7fd1a5cf2640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1a0142510 0x7fd1a0142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.448+0000 7fd1a5cf2640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1a0140550 0x7fd1a01409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.448+0000 7fd1a5cf2640 1 --2- 172.21.15.125:0/1641355238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd188004d00 0x7fd1a013f760 secure :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fd19000f9d0 tx=0x7fd19000fe90 comp rx=0 tx=0).stop 2023-10-18T11:58:27.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.448+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 >> 172.21.15.125:0/1641355238 conn(0x7fd1a0069930 msgr2=0x7fd1a00adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:27.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.448+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 shutdown_connections 2023-10-18T11:58:27.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.449+0000 7fd1a5cf2640 1 -- 172.21.15.125:0/1641355238 wait complete. 2023-10-18T11:58:27.463 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T11:58:27.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 -- 172.21.15.125:0/121348319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 msgr2=0x7faa540cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:27.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 --2- 172.21.15.125:0/121348319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 0x7faa540cf720 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7faa5408a810 tx=0x7faa3c02bd10 comp rx=0 tx=0).stop 2023-10-18T11:58:27.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 -- 172.21.15.125:0/121348319 shutdown_connections 2023-10-18T11:58:27.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 --2- 172.21.15.125:0/121348319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 0x7faa540cf720 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 --2- 172.21.15.125:0/121348319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa540ceb40 0x7faa540b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 --2- 172.21.15.125:0/121348319 >> v2:172.21.15.139:3300/0 conn(0x7faa540cf340 0x7faa540d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 -- 172.21.15.125:0/121348319 >> 172.21.15.125:0/121348319 conn(0x7faa540d0d60 msgr2=0x7faa540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:27.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 -- 172.21.15.125:0/121348319 shutdown_connections 2023-10-18T11:58:27.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.610+0000 7faa59524640 1 -- 172.21.15.125:0/121348319 wait complete. 2023-10-18T11:58:27.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.611+0000 7faa59524640 1 Processor -- start 2023-10-18T11:58:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.611+0000 7faa59524640 1 -- start start 2023-10-18T11:58:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa59524640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa540ceb40 0x7faa54158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa59524640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa540cf340 0x7faa540b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa59524640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 0x7faa5414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa59524640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa540ca360 con 0x7faa5415b2a0 2023-10-18T11:58:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa59524640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa540b5220 con 0x7faa540ceb40 2023-10-18T11:58:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa59524640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa540c9750 con 0x7faa540cf340 2023-10-18T11:58:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 0x7faa5414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 0x7faa5414f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46204/0 (socket says 172.21.15.125:46204) 2023-10-18T11:58:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 -- 172.21.15.125:0/1927998079 learned_addr learned my addr 172.21.15.125:0/1927998079 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa52ffd640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa540ceb40 0x7faa54158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:27.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa4bfff640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa540cf340 0x7faa540b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:27.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 -- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa540cf340 msgr2=0x7faa540b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:27.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa540cf340 0x7faa540b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 -- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa540ceb40 msgr2=0x7faa54158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:27.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa540ceb40 0x7faa54158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:27.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 -- 172.21.15.125:0/1927998079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c009560 con 0x7faa5415b2a0 2023-10-18T11:58:27.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa537fe640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 0x7faa5414f170 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00b970 tx=0x7faa4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:27.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa4b7fe640 1 -- 172.21.15.125:0/1927998079 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00cb40 con 0x7faa5415b2a0 2023-10-18T11:58:27.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa4b7fe640 1 -- 172.21.15.125:0/1927998079 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa4c00cdb0 con 0x7faa5415b2a0 2023-10-18T11:58:27.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa540c9a30 con 0x7faa5415b2a0 2023-10-18T11:58:27.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa4b7fe640 1 -- 172.21.15.125:0/1927998079 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0146a0 con 0x7faa5415b2a0 2023-10-18T11:58:27.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.612+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa540c9ff0 con 0x7faa5415b2a0 2023-10-18T11:58:27.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.613+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa540ca4d0 con 0x7faa5415b2a0 2023-10-18T11:58:27.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.614+0000 7faa4b7fe640 1 -- 172.21.15.125:0/1927998079 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faa4c01d3b0 con 0x7faa5415b2a0 2023-10-18T11:58:27.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.614+0000 7faa4b7fe640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa3406af60 0x7faa3406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:27.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.614+0000 7faa4b7fe640 1 -- 172.21.15.125:0/1927998079 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6765+0+0 (secure 0 0 0) 0x7faa4c089dc0 con 0x7faa5415b2a0 2023-10-18T11:58:27.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.614+0000 7faa52ffd640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa3406af60 0x7faa3406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:27.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.615+0000 7faa52ffd640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa3406af60 0x7faa3406d420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa44005fd0 tx=0x7faa44005e40 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:27.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.618+0000 7faa4b7fe640 1 -- 172.21.15.125:0/1927998079 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa4c054940 con 0x7faa5415b2a0 2023-10-18T11:58:27.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:27.783+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faa540690e0 con 0x7faa5415b2a0 2023-10-18T11:58:28.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.449+0000 7faa4b7fe640 1 -- 172.21.15.125:0/1927998079 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v196) v1 ==== 169+0+0 (secure 0 0 0) 0x7faa4c019020 con 0x7faa5415b2a0 2023-10-18T11:58:28.453 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T11:58:28.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa3406af60 msgr2=0x7faa3406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa3406af60 0x7faa3406d420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa44005fd0 tx=0x7faa44005e40 comp rx=0 tx=0).stop 2023-10-18T11:58:28.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 msgr2=0x7faa5414f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 0x7faa5414f170 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00b970 tx=0x7faa4c00be30 comp rx=0 tx=0).stop 2023-10-18T11:58:28.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 shutdown_connections 2023-10-18T11:58:28.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa3406af60 0x7faa3406d420 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa5415b2a0 0x7faa5414f170 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa540cf340 0x7faa540b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 --2- 172.21.15.125:0/1927998079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa540ceb40 0x7faa54158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 >> 172.21.15.125:0/1927998079 conn(0x7faa540d0d60 msgr2=0x7faa540ae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:28.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 shutdown_connections 2023-10-18T11:58:28.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.452+0000 7faa59524640 1 -- 172.21.15.125:0/1927998079 wait complete. 2023-10-18T11:58:28.466 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T11:58:28.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.569+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/2585437969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 msgr2=0x7f3fc0003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.569+0000 7f3fd9bd0640 1 --2- 172.21.15.125:0/2585437969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 0x7f3fc0003d80 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4001790 tx=0x7f3fc00057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.569+0000 7f3fd3fff640 1 -- 172.21.15.125:0/2585437969 reap_dead start 2023-10-18T11:58:28.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.569+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/2585437969 shutdown_connections 2023-10-18T11:58:28.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.569+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/2585437969 >> 172.21.15.125:0/2585437969 conn(0x7f3fd4069930 msgr2=0x7f3fd4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:28.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.569+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/2585437969 shutdown_connections 2023-10-18T11:58:28.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.569+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/2585437969 wait complete. 2023-10-18T11:58:28.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.570+0000 7f3fd9bd0640 1 Processor -- start 2023-10-18T11:58:28.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.570+0000 7f3fd9bd0640 1 -- start start 2023-10-18T11:58:28.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd9bd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 0x7f3fd413c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:28.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd9bd0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fd41e0b00 0x7f3fd41defe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:28.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd9bd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fd41df520 0x7f3fd41dd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:28.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd9bd0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41dfa60 con 0x7f3fc00039a0 2023-10-18T11:58:28.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd9bd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41dfd40 con 0x7f3fd41df520 2023-10-18T11:58:28.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd9bd0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41e0020 con 0x7f3fd41e0b00 2023-10-18T11:58:28.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 0x7f3fd413c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:28.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 0x7f3fd413c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46214/0 (socket says 172.21.15.125:46214) 2023-10-18T11:58:28.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 -- 172.21.15.125:0/54683767 learned_addr learned my addr 172.21.15.125:0/54683767 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:28.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 -- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fd41e0b00 msgr2=0x7f3fd41defe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fcaffd640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fd41e0b00 0x7f3fd41defe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:28.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd3fff640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fd41df520 0x7f3fd41dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:28.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fd41e0b00 0x7f3fd41defe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 -- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fd41df520 msgr2=0x7f3fd41dd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fd41df520 0x7f3fd41dd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 -- 172.21.15.125:0/54683767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd41e0300 con 0x7f3fc00039a0 2023-10-18T11:58:28.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fcaffd640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fd41e0b00 0x7f3fd41defe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:28.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd3fff640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fd41df520 0x7f3fd41dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:28.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.571+0000 7f3fd37fe640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 0x7f3fd413c9d0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f3fc402c1e0 tx=0x7f3fc402cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:28.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.572+0000 7f3fd0ff9640 1 -- 172.21.15.125:0/54683767 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40037f0 con 0x7f3fc00039a0 2023-10-18T11:58:28.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.572+0000 7f3fd0ff9640 1 -- 172.21.15.125:0/54683767 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3fc4003a40 con 0x7f3fc00039a0 2023-10-18T11:58:28.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.572+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd41e05e0 con 0x7f3fc00039a0 2023-10-18T11:58:28.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.572+0000 7f3fd0ff9640 1 -- 172.21.15.125:0/54683767 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4003df0 con 0x7f3fc00039a0 2023-10-18T11:58:28.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.572+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd413cf10 con 0x7f3fc00039a0 2023-10-18T11:58:28.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.572+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd40b1bd0 con 0x7f3fc00039a0 2023-10-18T11:58:28.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.573+0000 7f3fd0ff9640 1 -- 172.21.15.125:0/54683767 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3fc4032070 con 0x7f3fc00039a0 2023-10-18T11:58:28.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.573+0000 7f3fd0ff9640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3fb406af90 0x7f3fb406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:28.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.573+0000 7f3fd0ff9640 1 -- 172.21.15.125:0/54683767 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7f3fc403c070 con 0x7f3fc00039a0 2023-10-18T11:58:28.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.573+0000 7f3fcaffd640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3fb406af90 0x7f3fb406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:28.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.574+0000 7f3fcaffd640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3fb406af90 0x7f3fb406d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0017a0 tx=0x7f3fbc000f30 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:28.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.577+0000 7f3fd0ff9640 1 -- 172.21.15.125:0/54683767 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fc40763c0 con 0x7f3fc00039a0 2023-10-18T11:58:28.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.747+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7f3fd40690e0 con 0x7f3fc00039a0 2023-10-18T11:58:28.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.750+0000 7f3fd0ff9640 1 -- 172.21.15.125:0/54683767 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v70) v1 ==== 71+0+0 (secure 0 0 0) 0x7f3fc407ee60 con 0x7f3fc00039a0 2023-10-18T11:58:28.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3fb406af90 msgr2=0x7f3fb406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3fb406af90 0x7f3fb406d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0017a0 tx=0x7f3fbc000f30 comp rx=0 tx=0).stop 2023-10-18T11:58:28.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 msgr2=0x7f3fd413c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 0x7f3fd413c9d0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f3fc402c1e0 tx=0x7f3fc402cda0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 shutdown_connections 2023-10-18T11:58:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3fb406af90 0x7f3fb406d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3fd41df520 0x7f3fd41dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3fd41e0b00 0x7f3fd41defe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 --2- 172.21.15.125:0/54683767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3fc00039a0 0x7f3fd413c9d0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 >> 172.21.15.125:0/54683767 conn(0x7f3fd4069930 msgr2=0x7f3fd40af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:28.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 shutdown_connections 2023-10-18T11:58:28.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.753+0000 7f3fd9bd0640 1 -- 172.21.15.125:0/54683767 wait complete. 2023-10-18T11:58:28.766 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-18T11:58:28.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.865+0000 7f855021a640 1 -- 172.21.15.125:0/3683470946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85480afa20 msgr2=0x7f85480705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.865+0000 7f855021a640 1 --2- 172.21.15.125:0/3683470946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85480afa20 0x7f85480705a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f853800a940 tx=0x7f853802d090 comp rx=0 tx=0).stop 2023-10-18T11:58:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.865+0000 7f855021a640 1 -- 172.21.15.125:0/3683470946 shutdown_connections 2023-10-18T11:58:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.865+0000 7f855021a640 1 --2- 172.21.15.125:0/3683470946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85480afa20 0x7f85480705a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.865+0000 7f855021a640 1 --2- 172.21.15.125:0/3683470946 >> v2:172.21.15.139:3300/0 conn(0x7f85480cbd20 0x7f8548158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.865+0000 7f855021a640 1 -- 172.21.15.125:0/3683470946 >> 172.21.15.125:0/3683470946 conn(0x7f8548069930 msgr2=0x7f8548069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:28.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.865+0000 7f855021a640 1 -- 172.21.15.125:0/3683470946 shutdown_connections 2023-10-18T11:58:28.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.865+0000 7f855021a640 1 -- 172.21.15.125:0/3683470946 wait complete. 2023-10-18T11:58:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f855021a640 1 Processor -- start 2023-10-18T11:58:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f855021a640 1 -- start start 2023-10-18T11:58:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f855021a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85480afa20 0x7f854813d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f855021a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85480b0530 0x7f854814d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:28.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f855021a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85480cbd20 0x7f854814d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f855021a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854813d7a0 con 0x7f85480cbd20 2023-10-18T11:58:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f855021a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8548153d80 con 0x7f85480afa20 2023-10-18T11:58:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f855021a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8548154060 con 0x7f85480b0530 2023-10-18T11:58:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f854e7b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85480cbd20 0x7f854814d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:28.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f854e7b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85480cbd20 0x7f854814d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46222/0 (socket says 172.21.15.125:46222) 2023-10-18T11:58:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f854e7b7640 1 -- 172.21.15.125:0/3038135120 learned_addr learned my addr 172.21.15.125:0/3038135120 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.867+0000 7f854dfb6640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85480afa20 0x7f854813d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f854d7b5640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85480b0530 0x7f854814d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f854e7b7640 1 -- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85480b0530 msgr2=0x7f854814d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f854e7b7640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85480b0530 0x7f854814d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f854e7b7640 1 -- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85480afa20 msgr2=0x7f854813d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f854e7b7640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85480afa20 0x7f854813d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f854e7b7640 1 -- 172.21.15.125:0/3038135120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853800a5a0 con 0x7f85480cbd20 2023-10-18T11:58:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f854e7b7640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85480cbd20 0x7f854814d640 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f854000cbc0 tx=0x7f854000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:28.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f853e7fc640 1 -- 172.21.15.125:0/3038135120 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854000ad40 con 0x7f85480cbd20 2023-10-18T11:58:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f853e7fc640 1 -- 172.21.15.125:0/3038135120 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8540012420 con 0x7f85480cbd20 2023-10-18T11:58:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8548070cf0 con 0x7f85480cbd20 2023-10-18T11:58:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f853e7fc640 1 -- 172.21.15.125:0/3038135120 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540012850 con 0x7f85480cbd20 2023-10-18T11:58:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.868+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8548154620 con 0x7f85480cbd20 2023-10-18T11:58:28.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.869+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85480690e0 con 0x7f85480cbd20 2023-10-18T11:58:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.870+0000 7f853e7fc640 1 -- 172.21.15.125:0/3038135120 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f854001a3b0 con 0x7f85480cbd20 2023-10-18T11:58:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.870+0000 7f853e7fc640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f852c06af90 0x7f852c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.870+0000 7f853e7fc640 1 -- 172.21.15.125:0/3038135120 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7f854008a150 con 0x7f85480cbd20 2023-10-18T11:58:28.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.870+0000 7f854dfb6640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f852c06af90 0x7f852c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:28.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.871+0000 7f854dfb6640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f852c06af90 0x7f852c06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8544009b30 tx=0x7f8544005f90 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:28.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:28.874+0000 7f853e7fc640 1 -- 172.21.15.125:0/3038135120 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8540054f20 con 0x7f85480cbd20 2023-10-18T11:58:29.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.106+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f85480cc1e0 con 0x7f85480cbd20 2023-10-18T11:58:29.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.451+0000 7f853e7fc640 1 -- 172.21.15.125:0/3038135120 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v3018) v1 ==== 165+0+0 (secure 0 0 0) 0x7f8540058540 con 0x7f85480cbd20 2023-10-18T11:58:29.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f852c06af90 msgr2=0x7f852c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f852c06af90 0x7f852c06d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8544009b30 tx=0x7f8544005f90 comp rx=0 tx=0).stop 2023-10-18T11:58:29.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85480cbd20 msgr2=0x7f854814d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85480cbd20 0x7f854814d640 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f854000cbc0 tx=0x7f854000a430 comp rx=0 tx=0).stop 2023-10-18T11:58:29.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 shutdown_connections 2023-10-18T11:58:29.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f852c06af90 0x7f852c06d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85480cbd20 0x7f854814d640 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85480b0530 0x7f854814d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 --2- 172.21.15.125:0/3038135120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85480afa20 0x7f854813d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 >> 172.21.15.125:0/3038135120 conn(0x7f8548069930 msgr2=0x7f85480ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:29.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 shutdown_connections 2023-10-18T11:58:29.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.453+0000 7f855021a640 1 -- 172.21.15.125:0/3038135120 wait complete. 2023-10-18T11:58:29.467 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-10-18T11:58:29.468 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-10-18T11:58:29.469 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-18T11:58:29.470 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:29.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.573+0000 7f628692c640 1 -- 172.21.15.125:0/2610522983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f628015b2a0 msgr2=0x7f62800d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.573+0000 7f628692c640 1 --2- 172.21.15.125:0/2610522983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f628015b2a0 0x7f62800d1910 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f628015b680 tx=0x7f627402bcf0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.573+0000 7f628692c640 1 -- 172.21.15.125:0/2610522983 shutdown_connections 2023-10-18T11:58:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.573+0000 7f628692c640 1 --2- 172.21.15.125:0/2610522983 >> v2:172.21.15.139:3300/0 conn(0x7f62800ceb40 0x7f62800b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.573+0000 7f628692c640 1 --2- 172.21.15.125:0/2610522983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f628015b2a0 0x7f62800d1910 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.573+0000 7f628692c640 1 -- 172.21.15.125:0/2610522983 >> 172.21.15.125:0/2610522983 conn(0x7f62800d0d60 msgr2=0x7f62800d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:29.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.573+0000 7f628692c640 1 -- 172.21.15.125:0/2610522983 shutdown_connections 2023-10-18T11:58:29.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.573+0000 7f628692c640 1 -- 172.21.15.125:0/2610522983 wait complete. 2023-10-18T11:58:29.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.574+0000 7f628692c640 1 Processor -- start 2023-10-18T11:58:29.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.574+0000 7f628692c640 1 -- start start 2023-10-18T11:58:29.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f628692c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62800ceb40 0x7f62801e6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:29.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f628692c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62800cf340 0x7f62800c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:29.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f628692c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628015b2a0 0x7f628015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:29.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f628692c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801e3eb0 con 0x7f62800cf340 2023-10-18T11:58:29.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f628692c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628015c620 con 0x7f628015b2a0 2023-10-18T11:58:29.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f628692c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628015c8c0 con 0x7f62800ceb40 2023-10-18T11:58:29.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f627f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62800cf340 0x7f62800c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:29.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f627f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62800cf340 0x7f62800c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46232/0 (socket says 172.21.15.125:46232) 2023-10-18T11:58:29.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.575+0000 7f627f7fe640 1 -- 172.21.15.125:0/286951737 learned_addr learned my addr 172.21.15.125:0/286951737 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:29.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f6284ec9640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628015b2a0 0x7f628015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627ffff640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62800ceb40 0x7f62801e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627f7fe640 1 -- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62800ceb40 msgr2=0x7f62801e6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627f7fe640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62800ceb40 0x7f62801e6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627f7fe640 1 -- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628015b2a0 msgr2=0x7f628015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627f7fe640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628015b2a0 0x7f628015c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627f7fe640 1 -- 172.21.15.125:0/286951737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009490 con 0x7f62800cf340 2023-10-18T11:58:29.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f6284ec9640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628015b2a0 0x7f628015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:29.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627ffff640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62800ceb40 0x7f62801e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:29.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627f7fe640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62800cf340 0x7f62800c8e20 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f627402c300 tx=0x7f62740038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:29.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627cff9640 1 -- 172.21.15.125:0/286951737 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274003ea0 con 0x7f62800cf340 2023-10-18T11:58:29.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627cff9640 1 -- 172.21.15.125:0/286951737 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6274034440 con 0x7f62800cf340 2023-10-18T11:58:29.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62801e3580 con 0x7f62800cf340 2023-10-18T11:58:29.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f627cff9640 1 -- 172.21.15.125:0/286951737 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274034810 con 0x7f62800cf340 2023-10-18T11:58:29.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.576+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62801e3b40 con 0x7f62800cf340 2023-10-18T11:58:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.577+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62801e4060 con 0x7f62800cf340 2023-10-18T11:58:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.577+0000 7f627cff9640 1 -- 172.21.15.125:0/286951737 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6274034970 con 0x7f62800cf340 2023-10-18T11:58:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.578+0000 7f627cff9640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f626c06af90 0x7f626c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.578+0000 7f627cff9640 1 -- 172.21.15.125:0/286951737 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7f62740aca70 con 0x7f62800cf340 2023-10-18T11:58:29.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.578+0000 7f627ffff640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f626c06af90 0x7f626c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:29.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.579+0000 7f627ffff640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f626c06af90 0x7f626c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6270000f60 tx=0x7f6270009040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:29.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.582+0000 7f627cff9640 1 -- 172.21.15.125:0/286951737 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62801e4060 con 0x7f62800cf340 2023-10-18T11:58:29.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.750+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62800690e0 con 0x7f62800cf340 2023-10-18T11:58:29.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.752+0000 7f627cff9640 1 -- 172.21.15.125:0/286951737 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+14864 (secure 0 0 0) 0x7f627404c090 con 0x7f62800cf340 2023-10-18T11:58:29.755 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:29.755 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":196,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:58:28.446130+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6834/3571180970":"2023-10-19T11:58:25.140585+0000","172.21.15.139:0/3199187177":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/2966502376":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6833/833207676":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6832/833207676":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/2823163906":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6833/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1424135438":"2023-10-19T11:58:22.876968+0000","172.21.15.139:6832/2153770477":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1872946902":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1126366928":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/1293875531":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/608662592":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/141746223":"2023-10-19T11:53:19.652678+0000","172.21.15.139:6836/3571180970":"2023-10-19T11:58:25.140585+0000","172.21.15.139:0/1253211552":"2023-10-19T11:53:19.652678+0000","172.21.15.139:0/3751642323":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/402965407":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6833/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:6832/3367593104":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/4091087182":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/1330912710":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/697097533":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/1787628043":"2023-10-19T11:58:22.876968+0000","172.21.15.139:0/4016867169":"2023-10-19T11:42:31.009948+0000","172.21.15.139:0/878133557":"2023-10-19T11:42:31.009948+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:29.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f626c06af90 msgr2=0x7f626c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f626c06af90 0x7f626c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6270000f60 tx=0x7f6270009040 comp rx=0 tx=0).stop 2023-10-18T11:58:29.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62800cf340 msgr2=0x7f62800c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62800cf340 0x7f62800c8e20 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f627402c300 tx=0x7f62740038a0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 shutdown_connections 2023-10-18T11:58:29.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f626c06af90 0x7f626c06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628015b2a0 0x7f628015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62800cf340 0x7f62800c8e20 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 --2- 172.21.15.125:0/286951737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62800ceb40 0x7f62801e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 >> 172.21.15.125:0/286951737 conn(0x7f62800d0d60 msgr2=0x7f628015a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:29.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 shutdown_connections 2023-10-18T11:58:29.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.754+0000 7f628692c640 1 -- 172.21.15.125:0/286951737 wait complete. 2023-10-18T11:58:29.768 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:58:29.768 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:58:29.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.870+0000 7fb97a1a5640 1 -- 172.21.15.125:0/857657021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9740abc10 msgr2=0x7fb9740c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.870+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/857657021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9740abc10 0x7fb9740c9b30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009be0 tx=0x7fb96c02c0c0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.870+0000 7fb97a1a5640 1 -- 172.21.15.125:0/857657021 shutdown_connections 2023-10-18T11:58:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.870+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/857657021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9740abc10 0x7fb9740c9b30 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.870+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/857657021 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97406af00 0x7fb97406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.870+0000 7fb97a1a5640 1 -- 172.21.15.125:0/857657021 >> 172.21.15.125:0/857657021 conn(0x7fb974069930 msgr2=0x7fb974069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:29.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.870+0000 7fb97a1a5640 1 -- 172.21.15.125:0/857657021 shutdown_connections 2023-10-18T11:58:29.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.870+0000 7fb97a1a5640 1 -- 172.21.15.125:0/857657021 wait complete. 2023-10-18T11:58:29.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.871+0000 7fb97a1a5640 1 Processor -- start 2023-10-18T11:58:29.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.872+0000 7fb97a1a5640 1 -- start start 2023-10-18T11:58:29.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.872+0000 7fb97a1a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97406af00 0x7fb974137b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:29.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.872+0000 7fb97a1a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9740abc10 0x7fb974138290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:29.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.872+0000 7fb97a1a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9740c96b0 0x7fb9740ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:29.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.872+0000 7fb97a1a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97413a680 con 0x7fb9740c96b0 2023-10-18T11:58:29.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.872+0000 7fb97a1a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741380d0 con 0x7fb9740abc10 2023-10-18T11:58:29.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.872+0000 7fb97a1a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9740afdc0 con 0x7fb97406af00 2023-10-18T11:58:29.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9740c96b0 0x7fb9740ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:29.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9740c96b0 0x7fb9740ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46240/0 (socket says 172.21.15.125:46240) 2023-10-18T11:58:29.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 -- 172.21.15.125:0/326214463 learned_addr learned my addr 172.21.15.125:0/326214463 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:29.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb972ffd640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9740abc10 0x7fb974138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:29.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb9737fe640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97406af00 0x7fb974137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:29.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 -- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97406af00 msgr2=0x7fb974137b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97406af00 0x7fb974137b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 -- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9740abc10 msgr2=0x7fb974138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:29.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9740abc10 0x7fb974138290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:29.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 -- 172.21.15.125:0/326214463 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c009840 con 0x7fb9740c96b0 2023-10-18T11:58:29.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb973fff640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9740c96b0 0x7fb9740ae700 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036040 tx=0x7fb96c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:29.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb95bfff640 1 -- 172.21.15.125:0/326214463 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c003e60 con 0x7fb9740c96b0 2023-10-18T11:58:29.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb95bfff640 1 -- 172.21.15.125:0/326214463 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb96c004790 con 0x7fb9740c96b0 2023-10-18T11:58:29.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9741387d0 con 0x7fb9740c96b0 2023-10-18T11:58:29.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb95bfff640 1 -- 172.21.15.125:0/326214463 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c034800 con 0x7fb9740c96b0 2023-10-18T11:58:29.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.873+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9740af2e0 con 0x7fb9740c96b0 2023-10-18T11:58:29.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.874+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97406bf40 con 0x7fb9740c96b0 2023-10-18T11:58:29.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.875+0000 7fb95bfff640 1 -- 172.21.15.125:0/326214463 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb96c034960 con 0x7fb9740c96b0 2023-10-18T11:58:29.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.875+0000 7fb95bfff640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb96006af90 0x7fb96006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:29.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.875+0000 7fb95bfff640 1 -- 172.21.15.125:0/326214463 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fb96c0acae0 con 0x7fb9740c96b0 2023-10-18T11:58:29.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.875+0000 7fb9737fe640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb96006af90 0x7fb96006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:29.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.876+0000 7fb9737fe640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb96006af90 0x7fb96006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb964009bd0 tx=0x7fb964005f90 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:29.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:29.879+0000 7fb95bfff640 1 -- 172.21.15.125:0/326214463 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb96c077830 con 0x7fb9740c96b0 2023-10-18T11:58:30.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.053+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9740690e0 con 0x7fb9740c96b0 2023-10-18T11:58:30.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.053+0000 7fb95bfff640 1 -- 172.21.15.125:0/326214463 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v196) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb96c07ae50 con 0x7fb9740c96b0 2023-10-18T11:58:30.057 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:58:30.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb96006af90 msgr2=0x7fb96006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb96006af90 0x7fb96006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb964009bd0 tx=0x7fb964005f90 comp rx=0 tx=0).stop 2023-10-18T11:58:30.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9740c96b0 msgr2=0x7fb9740ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9740c96b0 0x7fb9740ae700 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036040 tx=0x7fb96c005c80 comp rx=0 tx=0).stop 2023-10-18T11:58:30.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 shutdown_connections 2023-10-18T11:58:30.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb96006af90 0x7fb96006d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9740abc10 0x7fb974138290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9740c96b0 0x7fb9740ae700 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 --2- 172.21.15.125:0/326214463 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97406af00 0x7fb974137b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 >> 172.21.15.125:0/326214463 conn(0x7fb974069930 msgr2=0x7fb974069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:30.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 shutdown_connections 2023-10-18T11:58:30.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.055+0000 7fb97a1a5640 1 -- 172.21.15.125:0/326214463 wait complete. 2023-10-18T11:58:30.068 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-18T11:58:30.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.168+0000 7fb538226640 1 -- 172.21.15.125:0/1139765050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5300cf340 msgr2=0x7fb5300d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.168+0000 7fb538226640 1 --2- 172.21.15.125:0/1139765050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5300cf340 0x7fb5300d1910 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb52c005fd0 tx=0x7fb52c02c050 comp rx=0 tx=0).stop 2023-10-18T11:58:30.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.168+0000 7fb538226640 1 -- 172.21.15.125:0/1139765050 shutdown_connections 2023-10-18T11:58:30.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.168+0000 7fb538226640 1 --2- 172.21.15.125:0/1139765050 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb53015b2a0 0x7fb5300cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.168+0000 7fb538226640 1 --2- 172.21.15.125:0/1139765050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5300cf340 0x7fb5300d1910 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.168+0000 7fb538226640 1 -- 172.21.15.125:0/1139765050 >> 172.21.15.125:0/1139765050 conn(0x7fb5300d0d60 msgr2=0x7fb5300d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:30.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.169+0000 7fb538226640 1 -- 172.21.15.125:0/1139765050 shutdown_connections 2023-10-18T11:58:30.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.169+0000 7fb538226640 1 -- 172.21.15.125:0/1139765050 wait complete. 2023-10-18T11:58:30.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.170+0000 7fb538226640 1 Processor -- start 2023-10-18T11:58:30.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.170+0000 7fb538226640 1 -- start start 2023-10-18T11:58:30.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.170+0000 7fb538226640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5300ceb40 0x7fb5301e3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.170+0000 7fb538226640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5300cf340 0x7fb5301e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.170+0000 7fb538226640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53015b2a0 0x7fb53006b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.170+0000 7fb538226640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5301e4c40 con 0x7fb5300cf340 2023-10-18T11:58:30.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.170+0000 7fb538226640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5301e3db0 con 0x7fb53015b2a0 2023-10-18T11:58:30.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.170+0000 7fb538226640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5301e4030 con 0x7fb5300ceb40 2023-10-18T11:58:30.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5300cf340 0x7fb5301e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5300cf340 0x7fb5301e50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46242/0 (socket says 172.21.15.125:46242) 2023-10-18T11:58:30.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 -- 172.21.15.125:0/544201669 learned_addr learned my addr 172.21.15.125:0/544201669 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:30.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5367c3640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53015b2a0 0x7fb53006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb535fc2640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5300ceb40 0x7fb5301e3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 -- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5300ceb40 msgr2=0x7fb5301e3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5300ceb40 0x7fb5301e3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 -- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53015b2a0 msgr2=0x7fb53006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53015b2a0 0x7fb53006b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 -- 172.21.15.125:0/544201669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c009840 con 0x7fb5300cf340 2023-10-18T11:58:30.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5357c1640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5300cf340 0x7fb5301e50d0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e7c0 tx=0x7fb52000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:30.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5267fc640 1 -- 172.21.15.125:0/544201669 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520009580 con 0x7fb5300cf340 2023-10-18T11:58:30.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5267fc640 1 -- 172.21.15.125:0/544201669 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb52000eed0 con 0x7fb5300cf340 2023-10-18T11:58:30.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5301e4310 con 0x7fb5300cf340 2023-10-18T11:58:30.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb5267fc640 1 -- 172.21.15.125:0/544201669 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520010430 con 0x7fb5300cf340 2023-10-18T11:58:30.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.171+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5301e48d0 con 0x7fb5300cf340 2023-10-18T11:58:30.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.172+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5301e4df0 con 0x7fb5300cf340 2023-10-18T11:58:30.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.173+0000 7fb5267fc640 1 -- 172.21.15.125:0/544201669 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb5200105b0 con 0x7fb5300cf340 2023-10-18T11:58:30.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.173+0000 7fb5267fc640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb51406af60 0x7fb51406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.173+0000 7fb5267fc640 1 -- 172.21.15.125:0/544201669 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fb520014070 con 0x7fb5300cf340 2023-10-18T11:58:30.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.173+0000 7fb535fc2640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb51406af60 0x7fb51406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.174+0000 7fb535fc2640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb51406af60 0x7fb51406d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb52c035000 tx=0x7fb52c004ab0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:30.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.177+0000 7fb5267fc640 1 -- 172.21.15.125:0/544201669 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb520051ca0 con 0x7fb5300cf340 2023-10-18T11:58:30.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.407+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fb5300690e0 con 0x7fb5300cf340 2023-10-18T11:58:30.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.452+0000 7fb5267fc640 1 -- 172.21.15.125:0/544201669 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3019) v1 ==== 166+0+0 (secure 0 0 0) 0x7fb5301e4df0 con 0x7fb5300cf340 2023-10-18T11:58:30.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb51406af60 msgr2=0x7fb51406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb51406af60 0x7fb51406d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb52c035000 tx=0x7fb52c004ab0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5300cf340 msgr2=0x7fb5301e50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5300cf340 0x7fb5301e50d0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e7c0 tx=0x7fb52000ec80 comp rx=0 tx=0).stop 2023-10-18T11:58:30.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 shutdown_connections 2023-10-18T11:58:30.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb51406af60 0x7fb51406d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53015b2a0 0x7fb53006b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb5300cf340 0x7fb5301e50d0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 --2- 172.21.15.125:0/544201669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5300ceb40 0x7fb5301e3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 >> 172.21.15.125:0/544201669 conn(0x7fb5300d0d60 msgr2=0x7fb53015a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:30.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 shutdown_connections 2023-10-18T11:58:30.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.454+0000 7fb538226640 1 -- 172.21.15.125:0/544201669 wait complete. 2023-10-18T11:58:30.467 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T11:58:30.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.566+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/835141508 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb9815eba0 msgr2=0x7fcb980b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.566+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/835141508 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb9815eba0 0x7fcb980b00f0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fcb88005fd0 tx=0x7fcb8802bea0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.566+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/835141508 shutdown_connections 2023-10-18T11:58:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.566+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/835141508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb980cf2a0 0x7fcb980cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.566+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/835141508 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb9815eba0 0x7fcb980b00f0 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.566+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/835141508 >> 172.21.15.125:0/835141508 conn(0x7fcb9808a650 msgr2=0x7fcb98099f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:30.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.566+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/835141508 shutdown_connections 2023-10-18T11:58:30.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.566+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/835141508 wait complete. 2023-10-18T11:58:30.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.567+0000 7fcb9f6c2640 1 Processor -- start 2023-10-18T11:58:30.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.567+0000 7fcb9f6c2640 1 -- start start 2023-10-18T11:58:30.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9f6c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb980cf2a0 0x7fcb9815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9f6c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb9815b200 0x7fcb9815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9f6c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb9815eba0 0x7fcb980c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9f6c2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9815d9f0 con 0x7fcb980cf2a0 2023-10-18T11:58:30.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9f6c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb980af8f0 con 0x7fcb9815eba0 2023-10-18T11:58:30.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9f6c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb980af050 con 0x7fcb9815b200 2023-10-18T11:58:30.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9d45e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb980cf2a0 0x7fcb9815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9d45e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb980cf2a0 0x7fcb9815cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46268/0 (socket says 172.21.15.125:46268) 2023-10-18T11:58:30.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9d45e640 1 -- 172.21.15.125:0/2021960373 learned_addr learned my addr 172.21.15.125:0/2021960373 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:30.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9cc5d640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb9815b200 0x7fcb9815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.568+0000 7fcb9dc5f640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb9815eba0 0x7fcb980c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9d45e640 1 -- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb9815b200 msgr2=0x7fcb9815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9d45e640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb9815b200 0x7fcb9815d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9d45e640 1 -- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb9815eba0 msgr2=0x7fcb980c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9d45e640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb9815eba0 0x7fcb980c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9d45e640 1 -- 172.21.15.125:0/2021960373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb88009560 con 0x7fcb980cf2a0 2023-10-18T11:58:30.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9cc5d640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb9815b200 0x7fcb9815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:30.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9d45e640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb980cf2a0 0x7fcb9815cf70 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c870 tx=0x7fcb9000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:30.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb8dffb640 1 -- 172.21.15.125:0/2021960373 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90003b70 con 0x7fcb980cf2a0 2023-10-18T11:58:30.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb8dffb640 1 -- 172.21.15.125:0/2021960373 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb90003cf0 con 0x7fcb980cf2a0 2023-10-18T11:58:30.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb8dffb640 1 -- 172.21.15.125:0/2021960373 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90003ea0 con 0x7fcb980cf2a0 2023-10-18T11:58:30.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb980af330 con 0x7fcb980cf2a0 2023-10-18T11:58:30.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.569+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb980c4a30 con 0x7fcb980cf2a0 2023-10-18T11:58:30.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.570+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb98066930 con 0x7fcb980cf2a0 2023-10-18T11:58:30.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.571+0000 7fcb8dffb640 1 -- 172.21.15.125:0/2021960373 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcb9000aab0 con 0x7fcb980cf2a0 2023-10-18T11:58:30.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.571+0000 7fcb8dffb640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcb7c06af60 0x7fcb7c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.571+0000 7fcb8dffb640 1 -- 172.21.15.125:0/2021960373 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fcb900898f0 con 0x7fcb980cf2a0 2023-10-18T11:58:30.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.571+0000 7fcb9cc5d640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcb7c06af60 0x7fcb7c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.572+0000 7fcb9cc5d640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcb7c06af60 0x7fcb7c06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb88035000 tx=0x7fcb8802e040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:30.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.575+0000 7fcb8dffb640 1 -- 172.21.15.125:0/2021960373 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb900546a0 con 0x7fcb980cf2a0 2023-10-18T11:58:30.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.741+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fcb9804dae0 con 0x7fcb980cf2a0 2023-10-18T11:58:30.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.742+0000 7fcb8dffb640 1 -- 172.21.15.125:0/2021960373 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v70) v1 ==== 59+0+2220 (secure 0 0 0) 0x7fcb90057cc0 con 0x7fcb980cf2a0 2023-10-18T11:58:30.745 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T11:58:30.745 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T11:58:30.745 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T11:58:30.746 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T11:58:30.746 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T11:58:30.746 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T11:58:30.746 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T11:58:30.747 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T11:58:30.747 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T11:58:30.747 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T11:58:30.747 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T11:58:30.747 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T11:58:30.747 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T11:58:30.748 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T11:58:30.748 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T11:58:30.748 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T11:58:30.748 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T11:58:30.748 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T11:58:30.748 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T11:58:30.749 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T11:58:30.749 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T11:58:30.749 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T11:58:30.749 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T11:58:30.749 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T11:58:30.749 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T11:58:30.750 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T11:58:30.750 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T11:58:30.750 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T11:58:30.750 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T11:58:30.750 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T11:58:30.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcb7c06af60 msgr2=0x7fcb7c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcb7c06af60 0x7fcb7c06d420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb88035000 tx=0x7fcb8802e040 comp rx=0 tx=0).stop 2023-10-18T11:58:30.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb980cf2a0 msgr2=0x7fcb9815cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb980cf2a0 0x7fcb9815cf70 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c870 tx=0x7fcb9000cd30 comp rx=0 tx=0).stop 2023-10-18T11:58:30.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 shutdown_connections 2023-10-18T11:58:30.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcb7c06af60 0x7fcb7c06d420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb9815eba0 0x7fcb980c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb9815b200 0x7fcb9815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 --2- 172.21.15.125:0/2021960373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb980cf2a0 0x7fcb9815cf70 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 >> 172.21.15.125:0/2021960373 conn(0x7fcb9808a650 msgr2=0x7fcb980b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 shutdown_connections 2023-10-18T11:58:30.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.744+0000 7fcb9f6c2640 1 -- 172.21.15.125:0/2021960373 wait complete. 2023-10-18T11:58:30.757 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T11:58:30.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.859+0000 7f2e4907c640 1 -- 172.21.15.125:0/1023189965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 msgr2=0x7f2e440affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.859+0000 7f2e4907c640 1 --2- 172.21.15.125:0/1023189965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 0x7f2e440affe0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f2e34005fd0 tx=0x7f2e3402bd90 comp rx=0 tx=0).stop 2023-10-18T11:58:30.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.859+0000 7f2e4907c640 1 -- 172.21.15.125:0/1023189965 shutdown_connections 2023-10-18T11:58:30.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.859+0000 7f2e4907c640 1 --2- 172.21.15.125:0/1023189965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 0x7f2e440affe0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.859+0000 7f2e4907c640 1 --2- 172.21.15.125:0/1023189965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e4413e520 0x7f2e4406bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.859+0000 7f2e4907c640 1 -- 172.21.15.125:0/1023189965 >> 172.21.15.125:0/1023189965 conn(0x7f2e44069930 msgr2=0x7f2e44069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:30.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.859+0000 7f2e4907c640 1 -- 172.21.15.125:0/1023189965 shutdown_connections 2023-10-18T11:58:30.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.859+0000 7f2e4907c640 1 -- 172.21.15.125:0/1023189965 wait complete. 2023-10-18T11:58:30.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.861+0000 7f2e4907c640 1 Processor -- start 2023-10-18T11:58:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.861+0000 7f2e4907c640 1 -- start start 2023-10-18T11:58:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.861+0000 7f2e4907c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 0x7f2e4413b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.861+0000 7f2e4907c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e4413e520 0x7f2e441d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.861+0000 7f2e4907c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e44141690 0x7f2e441d74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.861+0000 7f2e4907c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e441d8c90 con 0x7f2e44070af0 2023-10-18T11:58:30.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.861+0000 7f2e4907c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4413b7f0 con 0x7f2e4413e520 2023-10-18T11:58:30.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.861+0000 7f2e4907c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e441d8010 con 0x7f2e44141690 2023-10-18T11:58:30.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 0x7f2e4413b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 0x7f2e4413b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46274/0 (socket says 172.21.15.125:46274) 2023-10-18T11:58:30.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 -- 172.21.15.125:0/442379440 learned_addr learned my addr 172.21.15.125:0/442379440 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:30.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e4259c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e4413e520 0x7f2e441d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e4359e640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e44141690 0x7f2e441d74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 -- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e44141690 msgr2=0x7f2e441d74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e44141690 0x7f2e441d74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 -- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e4413e520 msgr2=0x7f2e441d90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:30.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e4413e520 0x7f2e441d90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:30.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 -- 172.21.15.125:0/442379440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34009560 con 0x7f2e44070af0 2023-10-18T11:58:30.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e42d9d640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 0x7f2e4413b2b0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009970 tx=0x7f2e3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:30.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e3b7fe640 1 -- 172.21.15.125:0/442379440 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c012070 con 0x7f2e44070af0 2023-10-18T11:58:30.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e3b7fe640 1 -- 172.21.15.125:0/442379440 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2e3c00d020 con 0x7f2e44070af0 2023-10-18T11:58:30.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e441d82f0 con 0x7f2e44070af0 2023-10-18T11:58:30.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e3b7fe640 1 -- 172.21.15.125:0/442379440 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c007860 con 0x7f2e44070af0 2023-10-18T11:58:30.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.862+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e441d88b0 con 0x7f2e44070af0 2023-10-18T11:58:30.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.863+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e440690e0 con 0x7f2e44070af0 2023-10-18T11:58:30.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.864+0000 7f2e3b7fe640 1 -- 172.21.15.125:0/442379440 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2e3c016070 con 0x7f2e44070af0 2023-10-18T11:58:30.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.864+0000 7f2e3b7fe640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f2e2406af60 0x7f2e2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:30.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.864+0000 7f2e3b7fe640 1 -- 172.21.15.125:0/442379440 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7f2e3c08b220 con 0x7f2e44070af0 2023-10-18T11:58:30.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.864+0000 7f2e4259c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f2e2406af60 0x7f2e2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:30.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.865+0000 7f2e4259c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f2e2406af60 0x7f2e2406d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e441f16e0 tx=0x7f2e2c009040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:30.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:30.868+0000 7f2e3b7fe640 1 -- 172.21.15.125:0/442379440 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e3c059960 con 0x7f2e44070af0 2023-10-18T11:58:31.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.035+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2e440ca260 con 0x7f2e44070af0 2023-10-18T11:58:31.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.038+0000 7f2e3b7fe640 1 -- 172.21.15.125:0/442379440 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v71)=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2e3c0594f0 con 0x7f2e44070af0 2023-10-18T11:58:31.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f2e2406af60 msgr2=0x7f2e2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f2e2406af60 0x7f2e2406d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e441f16e0 tx=0x7f2e2c009040 comp rx=0 tx=0).stop 2023-10-18T11:58:31.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 msgr2=0x7f2e4413b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 0x7f2e4413b2b0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009970 tx=0x7f2e3c009e30 comp rx=0 tx=0).stop 2023-10-18T11:58:31.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 shutdown_connections 2023-10-18T11:58:31.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f2e2406af60 0x7f2e2406d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e4413e520 0x7f2e441d90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e44141690 0x7f2e441d74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 --2- 172.21.15.125:0/442379440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e44070af0 0x7f2e4413b2b0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 >> 172.21.15.125:0/442379440 conn(0x7f2e44069930 msgr2=0x7f2e440b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:31.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 shutdown_connections 2023-10-18T11:58:31.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.041+0000 7f2e4907c640 1 -- 172.21.15.125:0/442379440 wait complete. 2023-10-18T11:58:31.054 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T11:58:31.054 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:58:31.106 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T11:58:31.158 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T11:58:31.159 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:58:31.159 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:58:31.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:58:31.175 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:58:31.175 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:58:31.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:58:31.193 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T11:58:31.193 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:58:31.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T11:58:31.229 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T11:58:31.230 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T11:58:31.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T11:58:31.249 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T11:58:31.250 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:58:31.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.370+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/1190325357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1c40c9750 msgr2=0x7fb1c40c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.370+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/1190325357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1c40c9750 0x7fb1c40c9b70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009830 tx=0x7fb1bc02bcd0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.370+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/1190325357 shutdown_connections 2023-10-18T11:58:31.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.370+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/1190325357 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1c40c9750 0x7fb1c40c9b70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.370+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/1190325357 >> v2:172.21.15.139:3300/0 conn(0x7fb1c406b030 0x7fb1c406b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.370+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/1190325357 >> 172.21.15.125:0/1190325357 conn(0x7fb1c4069930 msgr2=0x7fb1c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:31.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.371+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/1190325357 shutdown_connections 2023-10-18T11:58:31.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.371+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/1190325357 wait complete. 2023-10-18T11:58:31.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.372+0000 7fb1c8fa4640 1 Processor -- start 2023-10-18T11:58:31.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.372+0000 7fb1c8fa4640 1 -- start start 2023-10-18T11:58:31.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c8fa4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1c406b030 0x7fb1c4152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:31.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c8fa4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c40c9750 0x7fb1c415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:31.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c8fa4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1c4159bd0 0x7fb1c415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:31.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c8fa4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c412ccf0 con 0x7fb1c406b030 2023-10-18T11:58:31.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c8fa4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c412a460 con 0x7fb1c40c9750 2023-10-18T11:58:31.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c8fa4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c41532f0 con 0x7fb1c4159bd0 2023-10-18T11:58:31.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1c406b030 0x7fb1c4152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:31.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1c406b030 0x7fb1c4152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46278/0 (socket says 172.21.15.125:46278) 2023-10-18T11:58:31.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 -- 172.21.15.125:0/169727506 learned_addr learned my addr 172.21.15.125:0/169727506 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:31.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c1d9b640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c40c9750 0x7fb1c415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:31.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c2d9d640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1c4159bd0 0x7fb1c415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:31.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 -- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1c4159bd0 msgr2=0x7fb1c415ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1c4159bd0 0x7fb1c415ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 -- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c40c9750 msgr2=0x7fb1c415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c40c9750 0x7fb1c415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 -- 172.21.15.125:0/169727506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc009490 con 0x7fb1c406b030 2023-10-18T11:58:31.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c259c640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1c406b030 0x7fb1c4152db0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400ebc0 tx=0x7fb1b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:31.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1b2ffd640 1 -- 172.21.15.125:0/169727506 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40099b0 con 0x7fb1c406b030 2023-10-18T11:58:31.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1b2ffd640 1 -- 172.21.15.125:0/169727506 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb1b4010720 con 0x7fb1c406b030 2023-10-18T11:58:31.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c412ca10 con 0x7fb1c406b030 2023-10-18T11:58:31.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1b2ffd640 1 -- 172.21.15.125:0/169727506 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4010b30 con 0x7fb1c406b030 2023-10-18T11:58:31.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.373+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c412a0f0 con 0x7fb1c406b030 2023-10-18T11:58:31.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.374+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c40c91f0 con 0x7fb1c406b030 2023-10-18T11:58:31.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.375+0000 7fb1b2ffd640 1 -- 172.21.15.125:0/169727506 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb1b4010c90 con 0x7fb1c406b030 2023-10-18T11:58:31.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.375+0000 7fb1b2ffd640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb1a406af60 0x7fb1a406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:31.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.375+0000 7fb1b2ffd640 1 -- 172.21.15.125:0/169727506 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fb1b4014070 con 0x7fb1c406b030 2023-10-18T11:58:31.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.375+0000 7fb1c1d9b640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb1a406af60 0x7fb1a406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:31.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.376+0000 7fb1c1d9b640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb1a406af60 0x7fb1a406d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1c41f29e0 tx=0x7fb1b8007cc0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:31.386 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.379+0000 7fb1b2ffd640 1 -- 172.21.15.125:0/169727506 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1b4051ca0 con 0x7fb1c406b030 2023-10-18T11:58:31.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.581+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1c40ab4b0 con 0x7fb1c406b030 2023-10-18T11:58:31.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.581+0000 7fb1b2ffd640 1 -- 172.21.15.125:0/169727506 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3712 (secure 0 0 0) 0x7fb1b4055630 con 0x7fb1c406b030 2023-10-18T11:58:31.585 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:31.585 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2139029617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2139029617},{"type":"v1","addr":"172.21.15.125:6835","nonce":2139029617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2438879012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2438879012},{"type":"v1","addr":"172.21.15.139:6837","nonce":2438879012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2883042650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2883042650},{"type":"v1","addr":"172.21.15.139:6836","nonce":2883042650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":10830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/2643526616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2643526616},{"type":"v1","addr":"172.21.15.125:6834","nonce":2643526616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[]} 2023-10-18T11:58:31.585 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 75 2023-10-18T11:58:31.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.583+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb1a406af60 msgr2=0x7fb1a406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.583+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb1a406af60 0x7fb1a406d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1c41f29e0 tx=0x7fb1b8007cc0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.583+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1c406b030 msgr2=0x7fb1c4152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.583+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1c406b030 0x7fb1c4152db0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400ebc0 tx=0x7fb1b400c6a0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.584+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 shutdown_connections 2023-10-18T11:58:31.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.584+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb1a406af60 0x7fb1a406d420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.584+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1c40c9750 0x7fb1c415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.584+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1c4159bd0 0x7fb1c415ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.584+0000 7fb1c8fa4640 1 --2- 172.21.15.125:0/169727506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1c406b030 0x7fb1c4152db0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.584+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 >> 172.21.15.125:0/169727506 conn(0x7fb1c4069930 msgr2=0x7fb1c40adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:31.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.584+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 shutdown_connections 2023-10-18T11:58:31.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.584+0000 7fb1c8fa4640 1 -- 172.21.15.125:0/169727506 wait complete. 2023-10-18T11:58:31.597 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T11:58:31.597 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T11:58:31.597 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T11:58:31.598 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:58:31.598 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T11:58:31.598 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T11:58:31.599 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T11:58:31.599 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:58:31.599 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T11:58:31.600 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T11:58:31.600 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T11:58:31.600 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:58:31.600 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T11:58:31.601 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T11:58:31.601 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T11:58:31.601 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T11:58:31.607 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T11:58:31.607 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T11:58:31.609 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T11:58:31.609 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T11:58:31.612 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T11:58:31.612 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T11:58:31.613 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T11:58:31.613 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T11:58:31.651 INFO:tasks.ceph.mds.a:Started 2023-10-18T11:58:31.652 DEBUG:teuthology.parallel:result is None 2023-10-18T11:58:31.653 INFO:tasks.ceph.mds.c:Started 2023-10-18T11:58:31.653 DEBUG:teuthology.parallel:result is None 2023-10-18T11:58:31.655 INFO:tasks.ceph.mds.b:Started 2023-10-18T11:58:31.656 DEBUG:teuthology.parallel:result is None 2023-10-18T11:58:31.656 INFO:tasks.ceph.mds.d:Started 2023-10-18T11:58:31.657 DEBUG:teuthology.parallel:result is None 2023-10-18T11:58:31.657 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T11:58:31.727 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T11:58:31.727 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T11:58:31.736 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T11:58:31.737 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T11:58:31.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.763+0000 7fe5f58a9640 1 -- 172.21.15.125:0/2036594549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 msgr2=0x7fe5f00b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.763+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/2036594549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 0x7fe5f00b0650 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8005fd0 tx=0x7fe5e802c1b0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.763+0000 7fe5f58a9640 1 -- 172.21.15.125:0/2036594549 shutdown_connections 2023-10-18T11:58:31.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.763+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/2036594549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 0x7fe5f00b0650 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.763+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/2036594549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5f015b2a0 0x7fe5f00d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.763+0000 7fe5f58a9640 1 -- 172.21.15.125:0/2036594549 >> 172.21.15.125:0/2036594549 conn(0x7fe5f00d0d60 msgr2=0x7fe5f00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:31.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.763+0000 7fe5f58a9640 1 -- 172.21.15.125:0/2036594549 shutdown_connections 2023-10-18T11:58:31.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.763+0000 7fe5f58a9640 1 -- 172.21.15.125:0/2036594549 wait complete. 2023-10-18T11:58:31.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.764+0000 7fe5f58a9640 1 Processor -- start 2023-10-18T11:58:31.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.765+0000 7fe5f58a9640 1 -- start start 2023-10-18T11:58:31.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.765+0000 7fe5f58a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 0x7fe5f00cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:31.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.765+0000 7fe5f58a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5f00cf340 0x7fe5f00ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:31.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.765+0000 7fe5f58a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f015b2a0 0x7fe5f014cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:31.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.765+0000 7fe5f58a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f009c040 con 0x7fe5f00ceb40 2023-10-18T11:58:31.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.765+0000 7fe5f58a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f01f1670 con 0x7fe5f015b2a0 2023-10-18T11:58:31.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.765+0000 7fe5f58a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00b3b90 con 0x7fe5f00cf340 2023-10-18T11:58:31.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.765+0000 7fe5eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 0x7fe5f00cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:31.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 0x7fe5f00cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46334/0 (socket says 172.21.15.125:46334) 2023-10-18T11:58:31.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5eeffd640 1 -- 172.21.15.125:0/142023492 learned_addr learned my addr 172.21.15.125:0/142023492 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:31.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5ee7fc640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5f00cf340 0x7fe5f00ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:31.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5ef7fe640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f015b2a0 0x7fe5f014cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:31.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5eeffd640 1 -- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5f00cf340 msgr2=0x7fe5f00ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5eeffd640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5f00cf340 0x7fe5f00ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5eeffd640 1 -- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f015b2a0 msgr2=0x7fe5f014cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:31.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5eeffd640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f015b2a0 0x7fe5f014cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:31.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5eeffd640 1 -- 172.21.15.125:0/142023492 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8009900 con 0x7fe5f00ceb40 2023-10-18T11:58:31.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5eeffd640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 0x7fe5f00cc8a0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c460 tx=0x7fe5e000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:31.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5e7fff640 1 -- 172.21.15.125:0/142023492 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0012830 con 0x7fe5f00ceb40 2023-10-18T11:58:31.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f00b4110 con 0x7fe5f00ceb40 2023-10-18T11:58:31.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5e7fff640 1 -- 172.21.15.125:0/142023492 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe5e0012a80 con 0x7fe5f00ceb40 2023-10-18T11:58:31.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5e7fff640 1 -- 172.21.15.125:0/142023492 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0012830 con 0x7fe5f00ceb40 2023-10-18T11:58:31.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.766+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f00b46d0 con 0x7fe5f00ceb40 2023-10-18T11:58:31.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.767+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f00690e0 con 0x7fe5f00ceb40 2023-10-18T11:58:31.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.768+0000 7fe5e7fff640 1 -- 172.21.15.125:0/142023492 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe5e001b400 con 0x7fe5f00ceb40 2023-10-18T11:58:31.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.768+0000 7fe5e7fff640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe5d006afc0 0x7fe5d006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:31.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.768+0000 7fe5e7fff640 1 -- 172.21.15.125:0/142023492 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fe5e0089780 con 0x7fe5f00ceb40 2023-10-18T11:58:31.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.768+0000 7fe5ee7fc640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe5d006afc0 0x7fe5d006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:31.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.769+0000 7fe5ee7fc640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe5d006afc0 0x7fe5d006d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0159a70 tx=0x7fe5d800b2e0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:31.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.773+0000 7fe5e7fff640 1 -- 172.21.15.125:0/142023492 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5e00544d0 con 0x7fe5f00ceb40 2023-10-18T11:58:31.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:31.954+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe5f00cb980 con 0x7fe5f00ceb40 2023-10-18T11:58:32.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.478+0000 7fe5e7fff640 1 -- 172.21.15.125:0/142023492 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v198) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe5e0057af0 con 0x7fe5f00ceb40 2023-10-18T11:58:32.480 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T11:58:32.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe5d006afc0 msgr2=0x7fe5d006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe5d006afc0 0x7fe5d006d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0159a70 tx=0x7fe5d800b2e0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 msgr2=0x7fe5f00cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 0x7fe5f00cc8a0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c460 tx=0x7fe5e000c920 comp rx=0 tx=0).stop 2023-10-18T11:58:32.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 shutdown_connections 2023-10-18T11:58:32.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe5d006afc0 0x7fe5d006d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f015b2a0 0x7fe5f014cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5f00cf340 0x7fe5f00ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 --2- 172.21.15.125:0/142023492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5f00ceb40 0x7fe5f00cc8a0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 >> 172.21.15.125:0/142023492 conn(0x7fe5f00d0d60 msgr2=0x7fe5f006b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:32.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 shutdown_connections 2023-10-18T11:58:32.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.480+0000 7fe5f58a9640 1 -- 172.21.15.125:0/142023492 wait complete. 2023-10-18T11:58:32.495 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T11:58:32.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.597+0000 7f172d0b0640 1 -- 172.21.15.125:0/3032359857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1728070af0 msgr2=0x7f17280affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.597+0000 7f172d0b0640 1 --2- 172.21.15.125:0/3032359857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1728070af0 0x7f17280affe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1720005fd0 tx=0x7f172002c050 comp rx=0 tx=0).stop 2023-10-18T11:58:32.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.597+0000 7f172d0b0640 1 -- 172.21.15.125:0/3032359857 shutdown_connections 2023-10-18T11:58:32.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.597+0000 7f172d0b0640 1 --2- 172.21.15.125:0/3032359857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1728070af0 0x7f17280affe0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.597+0000 7f172d0b0640 1 --2- 172.21.15.125:0/3032359857 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f172813e520 0x7f172806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.597+0000 7f172d0b0640 1 -- 172.21.15.125:0/3032359857 >> 172.21.15.125:0/3032359857 conn(0x7f1728069930 msgr2=0x7f1728069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:32.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.597+0000 7f172d0b0640 1 -- 172.21.15.125:0/3032359857 shutdown_connections 2023-10-18T11:58:32.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.597+0000 7f172d0b0640 1 -- 172.21.15.125:0/3032359857 wait complete. 2023-10-18T11:58:32.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.598+0000 7f172d0b0640 1 Processor -- start 2023-10-18T11:58:32.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.598+0000 7f172d0b0640 1 -- start start 2023-10-18T11:58:32.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172d0b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1728070af0 0x7f17281d8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:32.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172d0b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172813e520 0x7f17280ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:32.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172d0b0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1728141690 0x7f17280cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:32.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172d0b0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728132d90 con 0x7f1728141690 2023-10-18T11:58:32.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172d0b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281da690 con 0x7f172813e520 2023-10-18T11:58:32.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172d0b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17281f1550 con 0x7f1728070af0 2023-10-18T11:58:32.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1728141690 0x7f17280cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:32.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1728141690 0x7f17280cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46336/0 (socket says 172.21.15.125:46336) 2023-10-18T11:58:32.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172759e640 1 -- 172.21.15.125:0/1058610480 learned_addr learned my addr 172.21.15.125:0/1058610480 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:32.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f1726d9d640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1728070af0 0x7f17281d8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:32.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f171ffff640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172813e520 0x7f17280ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:32.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172759e640 1 -- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1728070af0 msgr2=0x7f17281d8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172759e640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1728070af0 0x7f17281d8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172759e640 1 -- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172813e520 msgr2=0x7f17280ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172759e640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172813e520 0x7f17280ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f172759e640 1 -- 172.21.15.125:0/1058610480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718009490 con 0x7f1728141690 2023-10-18T11:58:32.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.599+0000 7f171ffff640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172813e520 0x7f17280ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:32.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.600+0000 7f172759e640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1728141690 0x7f17280cadb0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f1720009cd0 tx=0x7f17200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:32.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.600+0000 7f171f7fe640 1 -- 172.21.15.125:0/1058610480 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172003e460 con 0x7f1728141690 2023-10-18T11:58:32.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.600+0000 7f171f7fe640 1 -- 172.21.15.125:0/1058610480 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f172003e6b0 con 0x7f1728141690 2023-10-18T11:58:32.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.600+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1720009840 con 0x7f1728141690 2023-10-18T11:58:32.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.600+0000 7f171f7fe640 1 -- 172.21.15.125:0/1058610480 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172003ea80 con 0x7f1728141690 2023-10-18T11:58:32.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.600+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17281f1b10 con 0x7f1728141690 2023-10-18T11:58:32.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.601+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17280b1f00 con 0x7f1728141690 2023-10-18T11:58:32.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.601+0000 7f171f7fe640 1 -- 172.21.15.125:0/1058610480 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f172003ebe0 con 0x7f1728141690 2023-10-18T11:58:32.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.602+0000 7f171f7fe640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f170806af90 0x7f170806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:32.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.602+0000 7f171f7fe640 1 -- 172.21.15.125:0/1058610480 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f17200ac680 con 0x7f1728141690 2023-10-18T11:58:32.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.602+0000 7f1726d9d640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f170806af90 0x7f170806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:32.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.602+0000 7f1726d9d640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f170806af90 0x7f170806d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f17280ac4b0 tx=0x7f1718005f90 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:32.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.606+0000 7f171f7fe640 1 -- 172.21.15.125:0/1058610480 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1720077770 con 0x7f1728141690 2023-10-18T11:58:32.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.835+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f17280ca240 con 0x7f1728141690 2023-10-18T11:58:32.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.835+0000 7f171f7fe640 1 -- 172.21.15.125:0/1058610480 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v55) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f172007ad90 con 0x7f1728141690 2023-10-18T11:58:32.839 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:32.839 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:58:32.839 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T11:58:32.839 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.839 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T11:58:32.840 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T11:58:32.840 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.840 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:58:32.840 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:58:32.840 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:58:32.841 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.841 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.841 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.841 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.842 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T11:58:32.842 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T11:58:32.842 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.842 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:58:32.842 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:58:32.843 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:58:32.843 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.843 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.843 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.843 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.844 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T11:58:32.844 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T11:58:32.844 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.844 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:58:32.844 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:58:32.844 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:58:32.845 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.845 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.845 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.845 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.845 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T11:58:32.846 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T11:58:32.846 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.846 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:58:32.846 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:58:32.846 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:58:32.847 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.847 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.847 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.847 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.847 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T11:58:32.848 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T11:58:32.848 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.848 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:58:32.848 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:58:32.848 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.849 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.849 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.849 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.849 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T11:58:32.849 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T11:58:32.849 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.850 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:58:32.850 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:58:32.850 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.850 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.851 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.851 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.851 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T11:58:32.851 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T11:58:32.851 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.852 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:58:32.852 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:58:32.852 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.852 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.852 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.852 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.853 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T11:58:32.853 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T11:58:32.853 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.853 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:58:32.853 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:58:32.854 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.854 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.854 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.854 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.854 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T11:58:32.855 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T11:58:32.855 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.855 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:58:32.855 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:58:32.856 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.856 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.856 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.856 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.857 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T11:58:32.857 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T11:58:32.857 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.857 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:58:32.857 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:58:32.858 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.858 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.858 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.858 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.858 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T11:58:32.859 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T11:58:32.859 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.859 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:58:32.859 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:58:32.859 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.860 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.860 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.860 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.860 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T11:58:32.860 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T11:58:32.861 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.861 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T11:58:32.861 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T11:58:32.861 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.861 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.862 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.862 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.862 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T11:58:32.862 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T11:58:32.862 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.863 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:58:32.863 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T11:58:32.863 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T11:58:32.863 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.863 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.863 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.864 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T11:58:32.864 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T11:58:32.864 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.864 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:58:32.864 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:58:32.865 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:58:32.865 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:58:32.865 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.865 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.865 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.866 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T11:58:32.866 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T11:58:32.866 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.866 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:58:32.866 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:58:32.867 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:58:32.867 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:58:32.867 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.867 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.867 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.867 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T11:58:32.868 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T11:58:32.868 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.868 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T11:58:32.868 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T11:58:32.868 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T11:58:32.869 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T11:58:32.869 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.869 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.869 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.869 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T11:58:32.870 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T11:58:32.870 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:58:32.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T11:58:32.870 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T11:58:32.871 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.871 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.871 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.871 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.871 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T11:58:32.872 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T11:58:32.872 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.872 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:58:32.872 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T11:58:32.872 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.873 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.873 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:32.873 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:32.873 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T11:58:32.873 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T11:58:32.873 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T11:58:32.874 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T11:58:32.874 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T11:58:32.874 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T11:58:32.874 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.874 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:32.875 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T11:58:32.875 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:58:32.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f170806af90 msgr2=0x7f170806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f170806af90 0x7f170806d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f17280ac4b0 tx=0x7f1718005f90 comp rx=0 tx=0).stop 2023-10-18T11:58:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1728141690 msgr2=0x7f17280cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1728141690 0x7f17280cadb0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f1720009cd0 tx=0x7f17200042f0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 shutdown_connections 2023-10-18T11:58:32.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f170806af90 0x7f170806d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172813e520 0x7f17280ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1728141690 0x7f17280cadb0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 --2- 172.21.15.125:0/1058610480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1728070af0 0x7f17281d8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.837+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 >> 172.21.15.125:0/1058610480 conn(0x7f1728069930 msgr2=0x7f172813c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:32.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.838+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 shutdown_connections 2023-10-18T11:58:32.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.838+0000 7f172d0b0640 1 -- 172.21.15.125:0/1058610480 wait complete. 2023-10-18T11:58:32.879 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:32.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.979+0000 7fadd8155640 1 -- 172.21.15.125:0/3092791300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadd013d360 msgr2=0x7fadd015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.979+0000 7fadd8155640 1 --2- 172.21.15.125:0/3092791300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadd013d360 0x7fadd015a9f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fadcc005fd0 tx=0x7fadcc02c0e0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.979+0000 7fadd8155640 1 -- 172.21.15.125:0/3092791300 shutdown_connections 2023-10-18T11:58:32.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.979+0000 7fadd8155640 1 --2- 172.21.15.125:0/3092791300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fadd00c9ed0 0x7fadd006b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.979+0000 7fadd8155640 1 --2- 172.21.15.125:0/3092791300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadd013d360 0x7fadd015a9f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.979+0000 7fadd8155640 1 -- 172.21.15.125:0/3092791300 >> 172.21.15.125:0/3092791300 conn(0x7fadd0069930 msgr2=0x7fadd0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:32.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.979+0000 7fadd8155640 1 -- 172.21.15.125:0/3092791300 shutdown_connections 2023-10-18T11:58:32.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.979+0000 7fadd8155640 1 -- 172.21.15.125:0/3092791300 wait complete. 2023-10-18T11:58:32.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.980+0000 7fadd8155640 1 Processor -- start 2023-10-18T11:58:32.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd8155640 1 -- start start 2023-10-18T11:58:32.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd8155640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadd00c9ed0 0x7fadd01558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:32.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd8155640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd013d360 0x7fadd01586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:32.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd8155640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fadd0140660 0x7fadd0157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:32.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd8155640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0155e00 con 0x7fadd013d360 2023-10-18T11:58:32.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd8155640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd013a1e0 con 0x7fadd00c9ed0 2023-10-18T11:58:32.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd8155640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd013a4c0 con 0x7fadd0140660 2023-10-18T11:58:32.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd56f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd013d360 0x7fadd01586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:32.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd56f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd013d360 0x7fadd01586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46340/0 (socket says 172.21.15.125:46340) 2023-10-18T11:58:32.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd56f0640 1 -- 172.21.15.125:0/3117458423 learned_addr learned my addr 172.21.15.125:0/3117458423 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:32.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.981+0000 7fadd5ef1640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadd00c9ed0 0x7fadd01558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:32.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd66f2640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fadd0140660 0x7fadd0157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:32.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd56f0640 1 -- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fadd0140660 msgr2=0x7fadd0157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd56f0640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fadd0140660 0x7fadd0157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd56f0640 1 -- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadd00c9ed0 msgr2=0x7fadd01558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:32.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd56f0640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadd00c9ed0 0x7fadd01558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:32.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd56f0640 1 -- 172.21.15.125:0/3117458423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc8009490 con 0x7fadd013d360 2023-10-18T11:58:32.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd56f0640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd013d360 0x7fadd01586c0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c430 tx=0x7fadc000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:32.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadc67fc640 1 -- 172.21.15.125:0/3117458423 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0011790 con 0x7fadd013d360 2023-10-18T11:58:32.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadc67fc640 1 -- 172.21.15.125:0/3117458423 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fadc0011a00 con 0x7fadd013d360 2023-10-18T11:58:32.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc009840 con 0x7fadd013d360 2023-10-18T11:58:32.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadc67fc640 1 -- 172.21.15.125:0/3117458423 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0011e10 con 0x7fadd013d360 2023-10-18T11:58:32.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.982+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd013aa80 con 0x7fadd013d360 2023-10-18T11:58:32.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.983+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd00690e0 con 0x7fadd013d360 2023-10-18T11:58:32.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.983+0000 7fadc67fc640 1 -- 172.21.15.125:0/3117458423 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fadc001c460 con 0x7fadd013d360 2023-10-18T11:58:32.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.984+0000 7fadc67fc640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fadb406af60 0x7fadb406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:32.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.984+0000 7fadc67fc640 1 -- 172.21.15.125:0/3117458423 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fadc00895d0 con 0x7fadd013d360 2023-10-18T11:58:32.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.984+0000 7fadd5ef1640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fadb406af60 0x7fadb406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:32.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.985+0000 7fadd5ef1640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fadb406af60 0x7fadb406d420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fadcc035000 tx=0x7fadcc004ab0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:32.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:32.988+0000 7fadc67fc640 1 -- 172.21.15.125:0/3117458423 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadc0054750 con 0x7fadd013d360 2023-10-18T11:58:33.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.153+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadd013b850 con 0x7fadd013d360 2023-10-18T11:58:33.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.154+0000 7fadc67fc640 1 -- 172.21.15.125:0/3117458423 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+13210 (secure 0 0 0) 0x7fadc0057d70 con 0x7fadd013d360 2023-10-18T11:58:33.159 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:33.159 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":200,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:58:32.491665+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:33.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fadb406af60 msgr2=0x7fadb406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fadb406af60 0x7fadb406d420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fadcc035000 tx=0x7fadcc004ab0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd013d360 msgr2=0x7fadd01586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd013d360 0x7fadd01586c0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c430 tx=0x7fadc000c830 comp rx=0 tx=0).stop 2023-10-18T11:58:33.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 shutdown_connections 2023-10-18T11:58:33.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fadb406af60 0x7fadb406d420 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fadd0140660 0x7fadd0157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd013d360 0x7fadd01586c0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 --2- 172.21.15.125:0/3117458423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadd00c9ed0 0x7fadd01558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 >> 172.21.15.125:0/3117458423 conn(0x7fadd0069930 msgr2=0x7fadd0159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:33.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 shutdown_connections 2023-10-18T11:58:33.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.156+0000 7fadd8155640 1 -- 172.21.15.125:0/3117458423 wait complete. 2023-10-18T11:58:33.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T11:58:33.171 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T11:58:33.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.275+0000 7f18b8105640 1 -- 172.21.15.125:0/630018790 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189c004d00 msgr2=0x7f189c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.275+0000 7f18b8105640 1 --2- 172.21.15.125:0/630018790 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f189c004d00 0x7f189c005180 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005fd0 tx=0x7f189c0057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.275+0000 7f18b66a2640 1 -- 172.21.15.125:0/630018790 reap_dead start 2023-10-18T11:58:33.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.275+0000 7f18b8105640 1 -- 172.21.15.125:0/630018790 shutdown_connections 2023-10-18T11:58:33.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.275+0000 7f18b8105640 1 -- 172.21.15.125:0/630018790 >> 172.21.15.125:0/630018790 conn(0x7f18b0069930 msgr2=0x7f18b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:33.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.275+0000 7f18b8105640 1 -- 172.21.15.125:0/630018790 shutdown_connections 2023-10-18T11:58:33.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.275+0000 7f18b8105640 1 -- 172.21.15.125:0/630018790 wait complete. 2023-10-18T11:58:33.285 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.276+0000 7f18b8105640 1 Processor -- start 2023-10-18T11:58:33.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.276+0000 7f18b8105640 1 -- start start 2023-10-18T11:58:33.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b8105640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b014e660 0x7f18b01360d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b8105640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18b00b1370 0x7f18b0136610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.286 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b8105640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b00ca870 0x7f18b00cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b8105640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00cb230 con 0x7f18b00b1370 2023-10-18T11:58:33.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b8105640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00b0900 con 0x7f18b014e660 2023-10-18T11:58:33.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b8105640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0136c00 con 0x7f18b00ca870 2023-10-18T11:58:33.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18b00b1370 0x7f18b0136610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18b00b1370 0x7f18b0136610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46352/0 (socket says 172.21.15.125:46352) 2023-10-18T11:58:33.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 -- 172.21.15.125:0/2296635333 learned_addr learned my addr 172.21.15.125:0/2296635333 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:33.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b66a2640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b00ca870 0x7f18b00cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b5ea1640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b014e660 0x7f18b01360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 -- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b00ca870 msgr2=0x7f18b00cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b00ca870 0x7f18b00cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 -- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b014e660 msgr2=0x7f18b01360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b014e660 0x7f18b01360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 -- 172.21.15.125:0/2296635333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00b0b70 con 0x7f18b00b1370 2023-10-18T11:58:33.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b5ea1640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b014e660 0x7f18b01360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:33.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b66a2640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b00ca870 0x7f18b00cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:33.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.277+0000 7f18b56a0640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18b00b1370 0x7f18b0136610 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006680 tx=0x7f18a0001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:33.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.278+0000 7f18a67fc640 1 -- 172.21.15.125:0/2296635333 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0006220 con 0x7f18b00b1370 2023-10-18T11:58:33.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.278+0000 7f18a67fc640 1 -- 172.21.15.125:0/2296635333 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18a0011700 con 0x7f18b00b1370 2023-10-18T11:58:33.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.278+0000 7f18a67fc640 1 -- 172.21.15.125:0/2296635333 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0011b10 con 0x7f18b00b1370 2023-10-18T11:58:33.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.278+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b00b0e50 con 0x7f18b00b1370 2023-10-18T11:58:33.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.278+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b0151ad0 con 0x7f18b00b1370 2023-10-18T11:58:33.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.279+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b00b1090 con 0x7f18b00b1370 2023-10-18T11:58:33.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.279+0000 7f18a67fc640 1 -- 172.21.15.125:0/2296635333 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f18a0011c70 con 0x7f18b00b1370 2023-10-18T11:58:33.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.280+0000 7f18a67fc640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f189406af90 0x7f189406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.280+0000 7f18a67fc640 1 -- 172.21.15.125:0/2296635333 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f18a008a6e0 con 0x7f18b00b1370 2023-10-18T11:58:33.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.280+0000 7f18b5ea1640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f189406af90 0x7f189406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.280+0000 7f18b5ea1640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f189406af90 0x7f189406d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f18ac001050 tx=0x7f18ac001c90 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:33.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.284+0000 7f18a67fc640 1 -- 172.21.15.125:0/2296635333 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18b00b1090 con 0x7f18b00b1370 2023-10-18T11:58:33.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.449+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18b00b1090 con 0x7f18b00b1370 2023-10-18T11:58:33.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.449+0000 7f18a67fc640 1 -- 172.21.15.125:0/2296635333 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v200) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18b00b1090 con 0x7f18b00b1370 2023-10-18T11:58:33.452 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T11:58:33.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f189406af90 msgr2=0x7f189406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f189406af90 0x7f189406d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f18ac001050 tx=0x7f18ac001c90 comp rx=0 tx=0).stop 2023-10-18T11:58:33.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18b00b1370 msgr2=0x7f18b0136610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18b00b1370 0x7f18b0136610 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006680 tx=0x7f18a0001790 comp rx=0 tx=0).stop 2023-10-18T11:58:33.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 shutdown_connections 2023-10-18T11:58:33.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f189406af90 0x7f189406d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b00ca870 0x7f18b00cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18b00b1370 0x7f18b0136610 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 --2- 172.21.15.125:0/2296635333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18b014e660 0x7f18b01360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 >> 172.21.15.125:0/2296635333 conn(0x7f18b0069930 msgr2=0x7f18b01589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:33.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 shutdown_connections 2023-10-18T11:58:33.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.451+0000 7f18b8105640 1 -- 172.21.15.125:0/2296635333 wait complete. 2023-10-18T11:58:33.464 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T11:58:33.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.565+0000 7f4e3d026640 1 -- 172.21.15.125:0/1721779953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e20004d00 msgr2=0x7f4e20005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.565+0000 7f4e3d026640 1 --2- 172.21.15.125:0/1721779953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e20004d00 0x7f4e20005180 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009b80 tx=0x7f4e200057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.565+0000 7f4e3759e640 1 -- 172.21.15.125:0/1721779953 reap_dead start 2023-10-18T11:58:33.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.565+0000 7f4e3d026640 1 -- 172.21.15.125:0/1721779953 shutdown_connections 2023-10-18T11:58:33.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.565+0000 7f4e3d026640 1 -- 172.21.15.125:0/1721779953 >> 172.21.15.125:0/1721779953 conn(0x7f4e38069930 msgr2=0x7f4e38069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:33.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.565+0000 7f4e3d026640 1 -- 172.21.15.125:0/1721779953 shutdown_connections 2023-10-18T11:58:33.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.565+0000 7f4e3d026640 1 -- 172.21.15.125:0/1721779953 wait complete. 2023-10-18T11:58:33.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.566+0000 7f4e3d026640 1 Processor -- start 2023-10-18T11:58:33.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.566+0000 7f4e3d026640 1 -- start start 2023-10-18T11:58:33.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3d026640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3813a450 0x7f4e38151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3d026640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e3813a830 0x7f4e380a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3d026640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e380b0920 0x7f4e380b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3d026640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38151e00 con 0x7f4e380b0920 2023-10-18T11:58:33.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3d026640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e381517d0 con 0x7f4e3813a830 2023-10-18T11:58:33.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3d026640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380b12f0 con 0x7f4e3813a450 2023-10-18T11:58:33.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e380b0920 0x7f4e380b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e380b0920 0x7f4e380b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46366/0 (socket says 172.21.15.125:46366) 2023-10-18T11:58:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 -- 172.21.15.125:0/1340681615 learned_addr learned my addr 172.21.15.125:0/1340681615 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3659c640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e3813a830 0x7f4e380a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e36d9d640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3813a450 0x7f4e38151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 -- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3813a450 msgr2=0x7f4e38151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3813a450 0x7f4e38151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 -- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e3813a830 msgr2=0x7f4e380a6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e3813a830 0x7f4e380a6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 -- 172.21.15.125:0/1340681615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38159a20 con 0x7f4e380b0920 2023-10-18T11:58:33.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3659c640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e3813a830 0x7f4e380a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:33.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.567+0000 7f4e3759e640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e380b0920 0x7f4e380b0db0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009b80 tx=0x7f4e28033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:33.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.568+0000 7f4e1f7fe640 1 -- 172.21.15.125:0/1340681615 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28004440 con 0x7f4e380b0920 2023-10-18T11:58:33.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.568+0000 7f4e1f7fe640 1 -- 172.21.15.125:0/1340681615 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e280035f0 con 0x7f4e380b0920 2023-10-18T11:58:33.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.568+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e38151a60 con 0x7f4e380b0920 2023-10-18T11:58:33.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.568+0000 7f4e1f7fe640 1 -- 172.21.15.125:0/1340681615 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e280039c0 con 0x7f4e380b0920 2023-10-18T11:58:33.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.568+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e380a7540 con 0x7f4e380b0920 2023-10-18T11:58:33.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.569+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e38151f40 con 0x7f4e380b0920 2023-10-18T11:58:33.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.569+0000 7f4e1f7fe640 1 -- 172.21.15.125:0/1340681615 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4e28003b20 con 0x7f4e380b0920 2023-10-18T11:58:33.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.569+0000 7f4e1f7fe640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4e2406af60 0x7f4e2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.570+0000 7f4e1f7fe640 1 -- 172.21.15.125:0/1340681615 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f4e280ac220 con 0x7f4e380b0920 2023-10-18T11:58:33.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.570+0000 7f4e36d9d640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4e2406af60 0x7f4e2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.570+0000 7f4e36d9d640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4e2406af60 0x7f4e2406d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e30000fb0 tx=0x7f4e300035f0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:33.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.574+0000 7f4e1f7fe640 1 -- 172.21.15.125:0/1340681615 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e38151f40 con 0x7f4e380b0920 2023-10-18T11:58:33.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.743+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4e38151f40 con 0x7f4e380b0920 2023-10-18T11:58:33.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.743+0000 7f4e1f7fe640 1 -- 172.21.15.125:0/1340681615 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v200) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4e38151f40 con 0x7f4e380b0920 2023-10-18T11:58:33.746 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:33.747 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T11:58:33.747 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:33.747 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T11:58:33.747 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T11:58:33.748 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:33.748 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T11:58:33.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4e2406af60 msgr2=0x7f4e2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4e2406af60 0x7f4e2406d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e30000fb0 tx=0x7f4e300035f0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e380b0920 msgr2=0x7f4e380b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e380b0920 0x7f4e380b0db0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009b80 tx=0x7f4e28033bf0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 shutdown_connections 2023-10-18T11:58:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4e2406af60 0x7f4e2406d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e3813a830 0x7f4e380a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e380b0920 0x7f4e380b0db0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 --2- 172.21.15.125:0/1340681615 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3813a450 0x7f4e38151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.745+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 >> 172.21.15.125:0/1340681615 conn(0x7f4e38069930 msgr2=0x7f4e380c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:33.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.746+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 shutdown_connections 2023-10-18T11:58:33.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.746+0000 7f4e3d026640 1 -- 172.21.15.125:0/1340681615 wait complete. 2023-10-18T11:58:33.759 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T11:58:33.759 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T11:58:33.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.863+0000 7fb08f25d640 1 -- 172.21.15.125:0/2840111739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb088141690 msgr2=0x7fb0880703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.863+0000 7fb08f25d640 1 --2- 172.21.15.125:0/2840111739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb088141690 0x7fb0880703e0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb0880cbd00 tx=0x7fb07402bdb0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.863+0000 7fb08f25d640 1 -- 172.21.15.125:0/2840111739 shutdown_connections 2023-10-18T11:58:33.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.863+0000 7fb08f25d640 1 --2- 172.21.15.125:0/2840111739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb088141690 0x7fb0880703e0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.863+0000 7fb08f25d640 1 --2- 172.21.15.125:0/2840111739 >> v2:172.21.15.139:3300/0 conn(0x7fb08813e520 0x7fb08806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.863+0000 7fb08f25d640 1 -- 172.21.15.125:0/2840111739 >> 172.21.15.125:0/2840111739 conn(0x7fb088069930 msgr2=0x7fb088069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:33.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.863+0000 7fb08f25d640 1 -- 172.21.15.125:0/2840111739 shutdown_connections 2023-10-18T11:58:33.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.863+0000 7fb08f25d640 1 -- 172.21.15.125:0/2840111739 wait complete. 2023-10-18T11:58:33.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.864+0000 7fb08f25d640 1 Processor -- start 2023-10-18T11:58:33.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.864+0000 7fb08f25d640 1 -- start start 2023-10-18T11:58:33.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb08f25d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08806c0a0 0x7fb0880a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb08f25d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb08813e520 0x7fb08813b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb08f25d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb088141690 0x7fb0881d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb08f25d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0881e21a0 con 0x7fb08813e520 2023-10-18T11:58:33.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb08f25d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08813b780 con 0x7fb08806c0a0 2023-10-18T11:58:33.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb08f25d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880a8da0 con 0x7fb088141690 2023-10-18T11:58:33.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb08813e520 0x7fb08813b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb08813e520 0x7fb08813b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46384/0 (socket says 172.21.15.125:46384) 2023-10-18T11:58:33.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 -- 172.21.15.125:0/607547363 learned_addr learned my addr 172.21.15.125:0/607547363 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:33.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 -- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb088141690 msgr2=0x7fb0881d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb088141690 0x7fb0881d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 -- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08806c0a0 msgr2=0x7fb0880a86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:33.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08806c0a0 0x7fb0880a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:33.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 -- 172.21.15.125:0/607547363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074009560 con 0x7fb08813e520 2023-10-18T11:58:33.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.865+0000 7fb087fff640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb08813e520 0x7fb08813b240 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fb074009850 tx=0x7fb07402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:33.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.866+0000 7fb0857fa640 1 -- 172.21.15.125:0/607547363 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074003800 con 0x7fb08813e520 2023-10-18T11:58:33.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.866+0000 7fb0857fa640 1 -- 172.21.15.125:0/607547363 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb074003a50 con 0x7fb08813e520 2023-10-18T11:58:33.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.866+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0880a9080 con 0x7fb08813e520 2023-10-18T11:58:33.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.866+0000 7fb0857fa640 1 -- 172.21.15.125:0/607547363 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074003e20 con 0x7fb08813e520 2023-10-18T11:58:33.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.866+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0880a9640 con 0x7fb08813e520 2023-10-18T11:58:33.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.867+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0880690e0 con 0x7fb08813e520 2023-10-18T11:58:33.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.867+0000 7fb0857fa640 1 -- 172.21.15.125:0/607547363 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb07403e450 con 0x7fb08813e520 2023-10-18T11:58:33.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.868+0000 7fb0857fa640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb06c06af60 0x7fb06c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:33.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.868+0000 7fb0857fa640 1 -- 172.21.15.125:0/607547363 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fb0740ac170 con 0x7fb08813e520 2023-10-18T11:58:33.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.868+0000 7fb08cff9640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb06c06af60 0x7fb06c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:33.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.868+0000 7fb08cff9640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb06c06af60 0x7fb06c06d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005fd0 tx=0x7fb07c005850 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:33.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:33.872+0000 7fb0857fa640 1 -- 172.21.15.125:0/607547363 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb074077250 con 0x7fb08813e520 2023-10-18T11:58:34.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.038+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb088072890 con 0x7fb08813e520 2023-10-18T11:58:34.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.498+0000 7fb0857fa640 1 -- 172.21.15.125:0/607547363 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v201) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb07407a870 con 0x7fb08813e520 2023-10-18T11:58:34.501 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T11:58:34.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.500+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb06c06af60 msgr2=0x7fb06c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:34.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.500+0000 7fb08f25d640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb06c06af60 0x7fb06c06d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005fd0 tx=0x7fb07c005850 comp rx=0 tx=0).stop 2023-10-18T11:58:34.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.500+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb08813e520 msgr2=0x7fb08813b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:34.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.500+0000 7fb08f25d640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb08813e520 0x7fb08813b240 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fb074009850 tx=0x7fb07402dae0 comp rx=0 tx=0).stop 2023-10-18T11:58:34.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.501+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 shutdown_connections 2023-10-18T11:58:34.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.501+0000 7fb08f25d640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb06c06af60 0x7fb06c06d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:34.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.501+0000 7fb08f25d640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb088141690 0x7fb0881d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:34.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.501+0000 7fb08f25d640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb08813e520 0x7fb08813b240 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:34.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.501+0000 7fb08f25d640 1 --2- 172.21.15.125:0/607547363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08806c0a0 0x7fb0880a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:34.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.501+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 >> 172.21.15.125:0/607547363 conn(0x7fb088069930 msgr2=0x7fb088069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:34.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.501+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 shutdown_connections 2023-10-18T11:58:34.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.501+0000 7fb08f25d640 1 -- 172.21.15.125:0/607547363 wait complete. 2023-10-18T11:58:34.519 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T11:58:34.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.626+0000 7f883ee32640 1 -- 172.21.15.125:0/4032848013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88380c9dc0 msgr2=0x7f883806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:34.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.626+0000 7f883ee32640 1 --2- 172.21.15.125:0/4032848013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88380c9dc0 0x7f883806b2e0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f8824009830 tx=0x7f882402bd60 comp rx=0 tx=0).stop 2023-10-18T11:58:34.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.626+0000 7f883ee32640 1 -- 172.21.15.125:0/4032848013 shutdown_connections 2023-10-18T11:58:34.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.626+0000 7f883ee32640 1 --2- 172.21.15.125:0/4032848013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88380c9dc0 0x7f883806b2e0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:34.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.626+0000 7f883ee32640 1 -- 172.21.15.125:0/4032848013 >> 172.21.15.125:0/4032848013 conn(0x7f8838069930 msgr2=0x7f8838069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:34.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.626+0000 7f883ee32640 1 -- 172.21.15.125:0/4032848013 shutdown_connections 2023-10-18T11:58:34.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.626+0000 7f883ee32640 1 -- 172.21.15.125:0/4032848013 wait complete. 2023-10-18T11:58:34.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.627+0000 7f883ee32640 1 Processor -- start 2023-10-18T11:58:34.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.627+0000 7f883ee32640 1 -- start start 2023-10-18T11:58:34.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.628+0000 7f883ee32640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f883806af00 0x7f8838151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:34.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.628+0000 7f883ee32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88380ab500 0x7f8838152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:34.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.628+0000 7f883ee32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88380c9dc0 0x7f8838152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:34.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.628+0000 7f883ee32640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838152440 con 0x7f883806af00 2023-10-18T11:58:34.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.628+0000 7f883ee32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88380b0160 con 0x7f88380c9dc0 2023-10-18T11:58:34.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.628+0000 7f883ee32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8838139660 con 0x7f88380ab500 2023-10-18T11:58:34.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.628+0000 7f883cbce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f883806af00 0x7f8838151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:34.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.628+0000 7f883cbce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f883806af00 0x7f8838151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46394/0 (socket says 172.21.15.125:46394) 2023-10-18T11:58:34.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883cbce640 1 -- 172.21.15.125:0/4063865243 learned_addr learned my addr 172.21.15.125:0/4063865243 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:34.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883d3cf640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88380c9dc0 0x7f8838152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:34.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f8837fff640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88380ab500 0x7f8838152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:34.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883cbce640 1 -- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88380ab500 msgr2=0x7f8838152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:34.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883cbce640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88380ab500 0x7f8838152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:34.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883cbce640 1 -- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88380c9dc0 msgr2=0x7f8838152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:34.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883cbce640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88380c9dc0 0x7f8838152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:34.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883cbce640 1 -- 172.21.15.125:0/4063865243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824009490 con 0x7f883806af00 2023-10-18T11:58:34.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883cbce640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f883806af00 0x7f8838151f00 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c500 tx=0x7f882c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:34.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f88357fa640 1 -- 172.21.15.125:0/4063865243 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c012830 con 0x7f883806af00 2023-10-18T11:58:34.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f88357fa640 1 -- 172.21.15.125:0/4063865243 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f882c012a80 con 0x7f883806af00 2023-10-18T11:58:34.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88380b03b0 con 0x7f883806af00 2023-10-18T11:58:34.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f88357fa640 1 -- 172.21.15.125:0/4063865243 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c012e90 con 0x7f883806af00 2023-10-18T11:58:34.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.629+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88380b0970 con 0x7f883806af00 2023-10-18T11:58:34.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.630+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88380690e0 con 0x7f883806af00 2023-10-18T11:58:34.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.631+0000 7f88357fa640 1 -- 172.21.15.125:0/4063865243 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f882c024420 con 0x7f883806af00 2023-10-18T11:58:34.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.631+0000 7f88357fa640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f881c06af90 0x7f881c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:34.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.631+0000 7f88357fa640 1 -- 172.21.15.125:0/4063865243 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f882c089d90 con 0x7f883806af00 2023-10-18T11:58:34.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.631+0000 7f8837fff640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f881c06af90 0x7f881c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:34.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.632+0000 7f8837fff640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f881c06af90 0x7f881c06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8824036000 tx=0x7f8824004730 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:34.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.635+0000 7f88357fa640 1 -- 172.21.15.125:0/4063865243 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f882c058830 con 0x7f883806af00 2023-10-18T11:58:34.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:34.801+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f88380cc420 con 0x7f883806af00 2023-10-18T11:58:35.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.503+0000 7f88357fa640 1 -- 172.21.15.125:0/4063865243 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v202) v1 ==== 170+0+0 (secure 0 0 0) 0x7f882c0011a0 con 0x7f883806af00 2023-10-18T11:58:35.506 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T11:58:35.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f881c06af90 msgr2=0x7f881c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:35.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f881c06af90 0x7f881c06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8824036000 tx=0x7f8824004730 comp rx=0 tx=0).stop 2023-10-18T11:58:35.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f883806af00 msgr2=0x7f8838151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:35.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f883806af00 0x7f8838151f00 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c500 tx=0x7f882c00c9c0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 shutdown_connections 2023-10-18T11:58:35.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f881c06af90 0x7f881c06d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88380c9dc0 0x7f8838152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88380ab500 0x7f8838152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 --2- 172.21.15.125:0/4063865243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f883806af00 0x7f8838151f00 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 >> 172.21.15.125:0/4063865243 conn(0x7f8838069930 msgr2=0x7f88380c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:35.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 shutdown_connections 2023-10-18T11:58:35.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.505+0000 7f883ee32640 1 -- 172.21.15.125:0/4063865243 wait complete. 2023-10-18T11:58:35.532 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T11:58:35.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.637+0000 7f15a178a640 1 -- 172.21.15.125:0/4165653094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c15b180 msgr2=0x7f159c0cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:35.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.637+0000 7f15a178a640 1 --2- 172.21.15.125:0/4165653094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c15b180 0x7f159c0cf350 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f159c0cb6a0 tx=0x7f158c02c370 comp rx=0 tx=0).stop 2023-10-18T11:58:35.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.637+0000 7f15a178a640 1 -- 172.21.15.125:0/4165653094 shutdown_connections 2023-10-18T11:58:35.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.637+0000 7f15a178a640 1 --2- 172.21.15.125:0/4165653094 >> v2:172.21.15.139:3300/0 conn(0x7f159c0ce420 0x7f159c0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.637+0000 7f15a178a640 1 --2- 172.21.15.125:0/4165653094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c15b180 0x7f159c0cf350 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.637+0000 7f15a178a640 1 -- 172.21.15.125:0/4165653094 >> 172.21.15.125:0/4165653094 conn(0x7f159c099ff0 msgr2=0x7f159c0d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:35.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.637+0000 7f15a178a640 1 -- 172.21.15.125:0/4165653094 shutdown_connections 2023-10-18T11:58:35.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.637+0000 7f15a178a640 1 -- 172.21.15.125:0/4165653094 wait complete. 2023-10-18T11:58:35.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.638+0000 7f15a178a640 1 Processor -- start 2023-10-18T11:58:35.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.638+0000 7f15a178a640 1 -- start start 2023-10-18T11:58:35.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f15a178a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f159c0cdbf0 0x7f159c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:35.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f15a178a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c0ce420 0x7f159c1e3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:35.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f15a178a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f159c15b180 0x7f159c1e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:35.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f15a178a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c1e3dc0 con 0x7f159c0ce420 2023-10-18T11:58:35.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f15a178a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c1e3ff0 con 0x7f159c15b180 2023-10-18T11:58:35.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f15a178a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c1e42d0 con 0x7f159c0cdbf0 2023-10-18T11:58:35.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c0ce420 0x7f159c1e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:35.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c0ce420 0x7f159c1e3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46418/0 (socket says 172.21.15.125:46418) 2023-10-18T11:58:35.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159a7fc640 1 -- 172.21.15.125:0/2003491628 learned_addr learned my addr 172.21.15.125:0/2003491628 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:35.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159b7fe640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f159c15b180 0x7f159c1e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:35.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159affd640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f159c0cdbf0 0x7f159c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:35.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159a7fc640 1 -- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f159c0cdbf0 msgr2=0x7f159c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:35.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159a7fc640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f159c0cdbf0 0x7f159c0b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159a7fc640 1 -- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f159c15b180 msgr2=0x7f159c1e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:35.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159a7fc640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f159c15b180 0x7f159c1e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:35.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.639+0000 7f159a7fc640 1 -- 172.21.15.125:0/2003491628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c009900 con 0x7f159c0ce420 2023-10-18T11:58:35.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.640+0000 7f159a7fc640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c0ce420 0x7f159c1e3880 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f159000b4a0 tx=0x7f159000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:35.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.640+0000 7f15837fe640 1 -- 172.21.15.125:0/2003491628 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590010610 con 0x7f159c0ce420 2023-10-18T11:58:35.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.640+0000 7f15837fe640 1 -- 172.21.15.125:0/2003491628 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1590010860 con 0x7f159c0ce420 2023-10-18T11:58:35.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.640+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c1e45b0 con 0x7f159c0ce420 2023-10-18T11:58:35.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.640+0000 7f15837fe640 1 -- 172.21.15.125:0/2003491628 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590010c70 con 0x7f159c0ce420 2023-10-18T11:58:35.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.640+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c1e4b70 con 0x7f159c0ce420 2023-10-18T11:58:35.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.641+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c066960 con 0x7f159c0ce420 2023-10-18T11:58:35.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.641+0000 7f15837fe640 1 -- 172.21.15.125:0/2003491628 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1590019450 con 0x7f159c0ce420 2023-10-18T11:58:35.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.642+0000 7f15837fe640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f158806af60 0x7f158806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:35.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.642+0000 7f15837fe640 1 -- 172.21.15.125:0/2003491628 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5985+0+0 (secure 0 0 0) 0x7f1590089cd0 con 0x7f159c0ce420 2023-10-18T11:58:35.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.642+0000 7f159affd640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f158806af60 0x7f158806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:35.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.642+0000 7f159affd640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f158806af60 0x7f158806d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f158c036000 tx=0x7f158c004ab0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:35.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.646+0000 7f15837fe640 1 -- 172.21.15.125:0/2003491628 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1590054ce0 con 0x7f159c0ce420 2023-10-18T11:58:35.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:35.846+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f159c1e4d90 con 0x7f159c0ce420 2023-10-18T11:58:36.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.504+0000 7f15837fe640 1 -- 172.21.15.125:0/2003491628 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7f15900140c0 con 0x7f159c0ce420 2023-10-18T11:58:36.507 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 25 and data pool 26 2023-10-18T11:58:36.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.508+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f158806af60 msgr2=0x7f158806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.508+0000 7f15a178a640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f158806af60 0x7f158806d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f158c036000 tx=0x7f158c004ab0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.508+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c0ce420 msgr2=0x7f159c1e3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.508+0000 7f15a178a640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c0ce420 0x7f159c1e3880 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f159000b4a0 tx=0x7f159000b960 comp rx=0 tx=0).stop 2023-10-18T11:58:36.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.509+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 shutdown_connections 2023-10-18T11:58:36.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.509+0000 7f15a178a640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f158806af60 0x7f158806d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.509+0000 7f15a178a640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f159c15b180 0x7f159c1e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.509+0000 7f15a178a640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f159c0ce420 0x7f159c1e3880 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.509+0000 7f15a178a640 1 --2- 172.21.15.125:0/2003491628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f159c0cdbf0 0x7f159c0b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.509+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 >> 172.21.15.125:0/2003491628 conn(0x7f159c099ff0 msgr2=0x7f159c151100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:36.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.509+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 shutdown_connections 2023-10-18T11:58:36.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.510+0000 7f15a178a640 1 -- 172.21.15.125:0/2003491628 wait complete. 2023-10-18T11:58:36.540 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:36.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.642+0000 7fd0effdc640 1 -- 172.21.15.125:0/2388842605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0d4004de0 msgr2=0x7fd0d40071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.642+0000 7fd0effdc640 1 --2- 172.21.15.125:0/2388842605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0d4004de0 0x7fd0d40071d0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006f60 tx=0x7fd0d40037d0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.642+0000 7fd0ee579640 1 -- 172.21.15.125:0/2388842605 reap_dead start 2023-10-18T11:58:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.642+0000 7fd0effdc640 1 -- 172.21.15.125:0/2388842605 shutdown_connections 2023-10-18T11:58:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.642+0000 7fd0effdc640 1 -- 172.21.15.125:0/2388842605 >> 172.21.15.125:0/2388842605 conn(0x7fd0e8069930 msgr2=0x7fd0e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.642+0000 7fd0effdc640 1 -- 172.21.15.125:0/2388842605 shutdown_connections 2023-10-18T11:58:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.642+0000 7fd0effdc640 1 -- 172.21.15.125:0/2388842605 wait complete. 2023-10-18T11:58:36.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0effdc640 1 Processor -- start 2023-10-18T11:58:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0effdc640 1 -- start start 2023-10-18T11:58:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0effdc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d4004de0 0x7fd0e8136e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0effdc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0e8136580 0x7fd0e813eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0effdc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e8136a00 0x7fd0e813f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0effdc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8159ae0 con 0x7fd0e8136a00 2023-10-18T11:58:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0effdc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8159d30 con 0x7fd0d4004de0 2023-10-18T11:58:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0effdc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80af840 con 0x7fd0e8136580 2023-10-18T11:58:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0ee579640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e8136a00 0x7fd0e813f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0ee579640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e8136a00 0x7fd0e813f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46444/0 (socket says 172.21.15.125:46444) 2023-10-18T11:58:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.644+0000 7fd0ee579640 1 -- 172.21.15.125:0/1386786931 learned_addr learned my addr 172.21.15.125:0/1386786931 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0ed577640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0e8136580 0x7fd0e813eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0edd78640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d4004de0 0x7fd0e8136e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0ee579640 1 -- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0e8136580 msgr2=0x7fd0e813eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0ee579640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0e8136580 0x7fd0e813eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0ee579640 1 -- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d4004de0 msgr2=0x7fd0e8136e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0ee579640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d4004de0 0x7fd0e8136e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0ee579640 1 -- 172.21.15.125:0/1386786931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e80afb20 con 0x7fd0e8136a00 2023-10-18T11:58:36.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0edd78640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d4004de0 0x7fd0e8136e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:36.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0ee579640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e8136a00 0x7fd0e813f070 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fd0e815a950 tx=0x7fd0e0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:36.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0de7fc640 1 -- 172.21.15.125:0/1386786931 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0011870 con 0x7fd0e8136a00 2023-10-18T11:58:36.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0de7fc640 1 -- 172.21.15.125:0/1386786931 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0e0011ae0 con 0x7fd0e8136a00 2023-10-18T11:58:36.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e815a010 con 0x7fd0e8136a00 2023-10-18T11:58:36.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0de7fc640 1 -- 172.21.15.125:0/1386786931 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00103c0 con 0x7fd0e8136a00 2023-10-18T11:58:36.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.645+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e80b0040 con 0x7fd0e8136a00 2023-10-18T11:58:36.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.646+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e80afd10 con 0x7fd0e8136a00 2023-10-18T11:58:36.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.647+0000 7fd0de7fc640 1 -- 172.21.15.125:0/1386786931 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd0e0010520 con 0x7fd0e8136a00 2023-10-18T11:58:36.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.647+0000 7fd0de7fc640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd0cc06af60 0x7fd0cc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:36.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.647+0000 7fd0de7fc640 1 -- 172.21.15.125:0/1386786931 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fd0e008a460 con 0x7fd0e8136a00 2023-10-18T11:58:36.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.647+0000 7fd0edd78640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd0cc06af60 0x7fd0cc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:36.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.648+0000 7fd0edd78640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd0cc06af60 0x7fd0cc06d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4034000 tx=0x7fd0e402e070 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:36.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.651+0000 7fd0de7fc640 1 -- 172.21.15.125:0/1386786931 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0e0055200 con 0x7fd0e8136a00 2023-10-18T11:58:36.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.818+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0e80c9790 con 0x7fd0e8136a00 2023-10-18T11:58:36.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.819+0000 7fd0de7fc640 1 -- 172.21.15.125:0/1386786931 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+16871 (secure 0 0 0) 0x7fd0e0058820 con 0x7fd0e8136a00 2023-10-18T11:58:36.823 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:36.823 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":203,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:58:35.847758+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:58:34.060453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-18T11:58:34.823261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:36.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd0cc06af60 msgr2=0x7fd0cc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd0cc06af60 0x7fd0cc06d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4034000 tx=0x7fd0e402e070 comp rx=0 tx=0).stop 2023-10-18T11:58:36.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e8136a00 msgr2=0x7fd0e813f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e8136a00 0x7fd0e813f070 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fd0e815a950 tx=0x7fd0e0001ae0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 shutdown_connections 2023-10-18T11:58:36.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fd0cc06af60 0x7fd0cc06d420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e8136a00 0x7fd0e813f070 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0e8136580 0x7fd0e813eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 --2- 172.21.15.125:0/1386786931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d4004de0 0x7fd0e8136e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 >> 172.21.15.125:0/1386786931 conn(0x7fd0e8069930 msgr2=0x7fd0e813bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:36.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 shutdown_connections 2023-10-18T11:58:36.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.822+0000 7fd0effdc640 1 -- 172.21.15.125:0/1386786931 wait complete. 2023-10-18T11:58:36.836 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:36.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.936+0000 7f359cc90640 1 -- 172.21.15.125:0/2183309166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 msgr2=0x7f3584005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.936+0000 7f359cc90640 1 --2- 172.21.15.125:0/2183309166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 0x7f3584005180 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f3590005fd0 tx=0x7f35840057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.936+0000 7f3596d9d640 1 -- 172.21.15.125:0/2183309166 reap_dead start 2023-10-18T11:58:36.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.936+0000 7f359cc90640 1 -- 172.21.15.125:0/2183309166 shutdown_connections 2023-10-18T11:58:36.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.936+0000 7f359cc90640 1 -- 172.21.15.125:0/2183309166 >> 172.21.15.125:0/2183309166 conn(0x7f3598069930 msgr2=0x7f3598069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:36.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.936+0000 7f359cc90640 1 -- 172.21.15.125:0/2183309166 shutdown_connections 2023-10-18T11:58:36.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.936+0000 7f359cc90640 1 -- 172.21.15.125:0/2183309166 wait complete. 2023-10-18T11:58:36.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.937+0000 7f359cc90640 1 Processor -- start 2023-10-18T11:58:36.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.937+0000 7f359cc90640 1 -- start start 2023-10-18T11:58:36.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359cc90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 0x7f3598137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:36.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359cc90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35981522e0 0x7f35981505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:36.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359cc90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3598150b30 0x7f3598157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:36.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359cc90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981527a0 con 0x7f3584004d00 2023-10-18T11:58:36.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359cc90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3598152a80 con 0x7f35981522e0 2023-10-18T11:58:36.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359cc90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3598152d60 con 0x7f3598150b30 2023-10-18T11:58:36.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 0x7f3598137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:36.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 0x7f3598137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46454/0 (socket says 172.21.15.125:46454) 2023-10-18T11:58:36.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359659c640 1 -- 172.21.15.125:0/1058106971 learned_addr learned my addr 172.21.15.125:0/1058106971 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:36.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f3596d9d640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3598150b30 0x7f3598157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:36.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f358ffff640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35981522e0 0x7f35981505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359659c640 1 -- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3598150b30 msgr2=0x7f3598157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359659c640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3598150b30 0x7f3598157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359659c640 1 -- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35981522e0 msgr2=0x7f35981505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359659c640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35981522e0 0x7f35981505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:36.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f359659c640 1 -- 172.21.15.125:0/1058106971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598153040 con 0x7f3584004d00 2023-10-18T11:58:36.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.938+0000 7f358ffff640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35981522e0 0x7f35981505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:36.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.939+0000 7f359659c640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 0x7f3598137920 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f3588004da0 tx=0x7f3588001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:36.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.939+0000 7f358effd640 1 -- 172.21.15.125:0/1058106971 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35880107d0 con 0x7f3584004d00 2023-10-18T11:58:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.939+0000 7f358effd640 1 -- 172.21.15.125:0/1058106971 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3588010a40 con 0x7f3584004d00 2023-10-18T11:58:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.939+0000 7f358effd640 1 -- 172.21.15.125:0/1058106971 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588010e50 con 0x7f3584004d00 2023-10-18T11:58:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.939+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359806f080 con 0x7f3584004d00 2023-10-18T11:58:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.939+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3598138040 con 0x7f3584004d00 2023-10-18T11:58:36.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.940+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35980690e0 con 0x7f3584004d00 2023-10-18T11:58:36.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.940+0000 7f358effd640 1 -- 172.21.15.125:0/1058106971 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f358801b460 con 0x7f3584004d00 2023-10-18T11:58:36.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.940+0000 7f358effd640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357806af60 0x7f357806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:36.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.941+0000 7f358effd640 1 -- 172.21.15.125:0/1058106971 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f358808a450 con 0x7f3584004d00 2023-10-18T11:58:36.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.941+0000 7f358ffff640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357806af60 0x7f357806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:36.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.941+0000 7f358ffff640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357806af60 0x7f357806d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35981e5400 tx=0x7f3580000f10 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:36.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:36.945+0000 7f358effd640 1 -- 172.21.15.125:0/1058106971 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35880550e0 con 0x7f3584004d00 2023-10-18T11:58:37.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.112+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35980cc420 con 0x7f3584004d00 2023-10-18T11:58:37.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.113+0000 7f358effd640 1 -- 172.21.15.125:0/1058106971 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+16871 (secure 0 0 0) 0x7f3588058700 con 0x7f3584004d00 2023-10-18T11:58:37.116 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:37.116 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":203,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:58:35.847758+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:58:34.060453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-18T11:58:34.823261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:37.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357806af60 msgr2=0x7f357806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357806af60 0x7f357806d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35981e5400 tx=0x7f3580000f10 comp rx=0 tx=0).stop 2023-10-18T11:58:37.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 msgr2=0x7f3598137920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 0x7f3598137920 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f3588004da0 tx=0x7f3588001630 comp rx=0 tx=0).stop 2023-10-18T11:58:37.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 shutdown_connections 2023-10-18T11:58:37.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f357806af60 0x7f357806d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35981522e0 0x7f35981505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3598150b30 0x7f3598157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 --2- 172.21.15.125:0/1058106971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3584004d00 0x7f3598137920 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 >> 172.21.15.125:0/1058106971 conn(0x7f3598069930 msgr2=0x7f3598069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:37.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 shutdown_connections 2023-10-18T11:58:37.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.115+0000 7f359cc90640 1 -- 172.21.15.125:0/1058106971 wait complete. 2023-10-18T11:58:37.129 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T11:58:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.232+0000 7faaa5221640 1 -- 172.21.15.125:0/868214405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa88004d00 msgr2=0x7faa88005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.232+0000 7faaa5221640 1 --2- 172.21.15.125:0/868214405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa88004d00 0x7faa88005180 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7faa98009b20 tx=0x7faa880057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.232+0000 7faa9f59e640 1 -- 172.21.15.125:0/868214405 reap_dead start 2023-10-18T11:58:37.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.232+0000 7faaa5221640 1 -- 172.21.15.125:0/868214405 shutdown_connections 2023-10-18T11:58:37.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.232+0000 7faaa5221640 1 -- 172.21.15.125:0/868214405 >> 172.21.15.125:0/868214405 conn(0x7faaa0069930 msgr2=0x7faaa0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:37.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.232+0000 7faaa5221640 1 -- 172.21.15.125:0/868214405 shutdown_connections 2023-10-18T11:58:37.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.232+0000 7faaa5221640 1 -- 172.21.15.125:0/868214405 wait complete. 2023-10-18T11:58:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.234+0000 7faaa5221640 1 Processor -- start 2023-10-18T11:58:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.234+0000 7faaa5221640 1 -- start start 2023-10-18T11:58:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.234+0000 7faaa5221640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa015cac0 0x7faaa00b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.234+0000 7faaa5221640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa00b06f0 0x7faaa01f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.234+0000 7faaa5221640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa01f0990 0x7faaa013c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.234+0000 7faaa5221640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa00afd90 con 0x7faaa00b06f0 2023-10-18T11:58:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.234+0000 7faaa5221640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa00af760 con 0x7faaa01f0990 2023-10-18T11:58:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.234+0000 7faaa5221640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa01efd10 con 0x7faaa015cac0 2023-10-18T11:58:37.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa00b06f0 0x7faaa01f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa00b06f0 0x7faaa01f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:46470/0 (socket says 172.21.15.125:46470) 2023-10-18T11:58:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 -- 172.21.15.125:0/1387101688 learned_addr learned my addr 172.21.15.125:0/1387101688 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9ed9d640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa015cac0 0x7faaa00b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9f59e640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa01f0990 0x7faaa013c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:37.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 -- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa015cac0 msgr2=0x7faaa00b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa015cac0 0x7faaa00b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 -- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa01f0990 msgr2=0x7faaa013c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa01f0990 0x7faaa013c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 -- 172.21.15.125:0/1387101688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa01efff0 con 0x7faaa00b06f0 2023-10-18T11:58:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9f59e640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa01f0990 0x7faaa013c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa9e59c640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa00b06f0 0x7faaa01f0450 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7faa94004e50 tx=0x7faa94001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa877fe640 1 -- 172.21.15.125:0/1387101688 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa940107d0 con 0x7faaa00b06f0 2023-10-18T11:58:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa877fe640 1 -- 172.21.15.125:0/1387101688 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa94010a40 con 0x7faaa00b06f0 2023-10-18T11:58:37.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faa877fe640 1 -- 172.21.15.125:0/1387101688 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94010e50 con 0x7faaa00b06f0 2023-10-18T11:58:37.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa00af9b0 con 0x7faaa00b06f0 2023-10-18T11:58:37.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.235+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa015d080 con 0x7faaa00b06f0 2023-10-18T11:58:37.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.236+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa01f0d70 con 0x7faaa00b06f0 2023-10-18T11:58:37.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.237+0000 7faa877fe640 1 -- 172.21.15.125:0/1387101688 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faa9401b460 con 0x7faaa00b06f0 2023-10-18T11:58:37.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.237+0000 7faa877fe640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa8c06af60 0x7faa8c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:37.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.237+0000 7faa877fe640 1 -- 172.21.15.125:0/1387101688 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6085+0+0 (secure 0 0 0) 0x7faa9408a310 con 0x7faaa00b06f0 2023-10-18T11:58:37.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.237+0000 7faa9ed9d640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa8c06af60 0x7faa8c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:37.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.238+0000 7faa9ed9d640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa8c06af60 0x7faa8c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa90004e40 tx=0x7faa90006040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:37.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.241+0000 7faa877fe640 1 -- 172.21.15.125:0/1387101688 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa940550b0 con 0x7faaa00b06f0 2023-10-18T11:58:37.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.442+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faaa00690e0 con 0x7faaa00b06f0 2023-10-18T11:58:37.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.523+0000 7faa877fe640 1 -- 172.21.15.125:0/1387101688 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7faa94001d40 con 0x7faaa00b06f0 2023-10-18T11:58:37.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa8c06af60 msgr2=0x7faa8c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa8c06af60 0x7faa8c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa90004e40 tx=0x7faa90006040 comp rx=0 tx=0).stop 2023-10-18T11:58:37.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa00b06f0 msgr2=0x7faaa01f0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa00b06f0 0x7faaa01f0450 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7faa94004e50 tx=0x7faa94001630 comp rx=0 tx=0).stop 2023-10-18T11:58:37.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 shutdown_connections 2023-10-18T11:58:37.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7faa8c06af60 0x7faa8c06d420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faaa01f0990 0x7faaa013c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faaa00b06f0 0x7faaa01f0450 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 --2- 172.21.15.125:0/1387101688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faaa015cac0 0x7faaa00b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 >> 172.21.15.125:0/1387101688 conn(0x7faaa0069930 msgr2=0x7faaa0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:37.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 shutdown_connections 2023-10-18T11:58:37.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.526+0000 7faaa5221640 1 -- 172.21.15.125:0/1387101688 wait complete. 2023-10-18T11:58:37.539 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T11:58:37.539 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T11:58:37.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.642+0000 7f90058e5640 1 -- 172.21.15.125:0/1676831710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900013e520 msgr2=0x7f900006bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.642+0000 7f90058e5640 1 --2- 172.21.15.125:0/1676831710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900013e520 0x7f900006bb60 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005fd0 tx=0x7f8ff002c120 comp rx=0 tx=0).stop 2023-10-18T11:58:37.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.642+0000 7f90058e5640 1 -- 172.21.15.125:0/1676831710 shutdown_connections 2023-10-18T11:58:37.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.642+0000 7f90058e5640 1 --2- 172.21.15.125:0/1676831710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9000141690 0x7f90000703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.642+0000 7f90058e5640 1 --2- 172.21.15.125:0/1676831710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900013e520 0x7f900006bb60 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.642+0000 7f90058e5640 1 -- 172.21.15.125:0/1676831710 >> 172.21.15.125:0/1676831710 conn(0x7f9000069930 msgr2=0x7f9000069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:37.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.642+0000 7f90058e5640 1 -- 172.21.15.125:0/1676831710 shutdown_connections 2023-10-18T11:58:37.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.642+0000 7f90058e5640 1 -- 172.21.15.125:0/1676831710 wait complete. 2023-10-18T11:58:37.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.643+0000 7f90058e5640 1 Processor -- start 2023-10-18T11:58:37.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.643+0000 7f90058e5640 1 -- start start 2023-10-18T11:58:37.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f90058e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900006c0a0 0x7f90000caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:37.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f90058e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900013e520 0x7f900013bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:37.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f90058e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9000141690 0x7f90001da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:37.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f90058e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000cac30 con 0x7f900006c0a0 2023-10-18T11:58:37.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f90058e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000cb3e0 con 0x7f9000141690 2023-10-18T11:58:37.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f90058e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900012cba0 con 0x7f900013e520 2023-10-18T11:58:37.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900006c0a0 0x7f90000caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:37.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900006c0a0 0x7f90000caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49944/0 (socket says 172.21.15.125:49944) 2023-10-18T11:58:37.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 -- 172.21.15.125:0/2455694747 learned_addr learned my addr 172.21.15.125:0/2455694747 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:37.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8fff7fe640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9000141690 0x7f90001da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:37.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffe7fc640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900013e520 0x7f900013bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:37.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 -- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900013e520 msgr2=0x7f900013bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900013e520 0x7f900013bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 -- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9000141690 msgr2=0x7f90001da1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:37.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9000141690 0x7f90001da1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:37.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 -- 172.21.15.125:0/2455694747 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0009900 con 0x7f900006c0a0 2023-10-18T11:58:37.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8ffeffd640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900006c0a0 0x7f90000caea0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009bf0 tx=0x7f8ff0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:37.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8fe7fff640 1 -- 172.21.15.125:0/2455694747 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0004450 con 0x7f900006c0a0 2023-10-18T11:58:37.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8fe7fff640 1 -- 172.21.15.125:0/2455694747 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ff00046a0 con 0x7f900006c0a0 2023-10-18T11:58:37.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f8fe7fff640 1 -- 172.21.15.125:0/2455694747 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff003f680 con 0x7f900006c0a0 2023-10-18T11:58:37.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.644+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90000b1320 con 0x7f900006c0a0 2023-10-18T11:58:37.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.645+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900013fbb0 con 0x7f900006c0a0 2023-10-18T11:58:37.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.646+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90000b1bd0 con 0x7f900006c0a0 2023-10-18T11:58:37.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.646+0000 7f8fe7fff640 1 -- 172.21.15.125:0/2455694747 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8ff003b040 con 0x7f900006c0a0 2023-10-18T11:58:37.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.646+0000 7f8fe7fff640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f8fec06af60 0x7f8fec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:37.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.646+0000 7f8fe7fff640 1 -- 172.21.15.125:0/2455694747 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f8ff00ac8a0 con 0x7f900006c0a0 2023-10-18T11:58:37.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.647+0000 7f8ffe7fc640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f8fec06af60 0x7f8fec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:37.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.647+0000 7f8ffe7fc640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f8fec06af60 0x7f8fec06d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f90001f2a10 tx=0x7f8ff4008040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:37.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.651+0000 7f8fe7fff640 1 -- 172.21.15.125:0/2455694747 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ff00776c0 con 0x7f900006c0a0 2023-10-18T11:58:37.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:37.855+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f90000690e0 con 0x7f900006c0a0 2023-10-18T11:58:38.484 INFO:tasks.ceph.mds.d.smithi139.stderr:2023-10-18T11:58:38.481+0000 7f8a6d6fc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T11:58:38.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.524+0000 7f8fe7fff640 1 -- 172.21.15.125:0/2455694747 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ff00430c0 con 0x7f900006c0a0 2023-10-18T11:58:38.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f8fec06af60 msgr2=0x7f8fec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f8fec06af60 0x7f8fec06d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f90001f2a10 tx=0x7f8ff4008040 comp rx=0 tx=0).stop 2023-10-18T11:58:38.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900006c0a0 msgr2=0x7f90000caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900006c0a0 0x7f90000caea0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009bf0 tx=0x7f8ff0003c20 comp rx=0 tx=0).stop 2023-10-18T11:58:38.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 shutdown_connections 2023-10-18T11:58:38.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f8fec06af60 0x7f8fec06d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9000141690 0x7f90001da1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f900013e520 0x7f900013bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 --2- 172.21.15.125:0/2455694747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f900006c0a0 0x7f90000caea0 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 >> 172.21.15.125:0/2455694747 conn(0x7f9000069930 msgr2=0x7f90001df890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:38.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 shutdown_connections 2023-10-18T11:58:38.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.527+0000 7f90058e5640 1 -- 172.21.15.125:0/2455694747 wait complete. 2023-10-18T11:58:38.542 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T11:58:38.542 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:58:38.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.646+0000 7f039d900640 1 -- 172.21.15.125:0/2979294916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 msgr2=0x7f0384003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.646+0000 7f039d900640 1 --2- 172.21.15.125:0/2979294916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 0x7f0384003dc0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f0390005fd0 tx=0x7f03840057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.646+0000 7f03977fe640 1 -- 172.21.15.125:0/2979294916 reap_dead start 2023-10-18T11:58:38.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.646+0000 7f039d900640 1 -- 172.21.15.125:0/2979294916 shutdown_connections 2023-10-18T11:58:38.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.646+0000 7f039d900640 1 -- 172.21.15.125:0/2979294916 >> 172.21.15.125:0/2979294916 conn(0x7f03980d0c50 msgr2=0x7f03980d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:38.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.646+0000 7f039d900640 1 -- 172.21.15.125:0/2979294916 shutdown_connections 2023-10-18T11:58:38.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.646+0000 7f039d900640 1 -- 172.21.15.125:0/2979294916 wait complete. 2023-10-18T11:58:38.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.647+0000 7f039d900640 1 Processor -- start 2023-10-18T11:58:38.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.647+0000 7f039d900640 1 -- start start 2023-10-18T11:58:38.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f039d900640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 0x7f0398151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:38.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f039d900640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03980c7c90 0x7f03980c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:38.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f039d900640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981e56f0 0x7f03981e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:38.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f039d900640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03980c9480 con 0x7f03840039a0 2023-10-18T11:58:38.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f039d900640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03981e60e0 con 0x7f03981e56f0 2023-10-18T11:58:38.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f039d900640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03980c8800 con 0x7f03980c7c90 2023-10-18T11:58:38.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 0x7f0398151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:38.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 0x7f0398151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49952/0 (socket says 172.21.15.125:49952) 2023-10-18T11:58:38.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 -- 172.21.15.125:0/3974864659 learned_addr learned my addr 172.21.15.125:0/3974864659 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:38.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 -- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03980c7c90 msgr2=0x7f03980c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03980c7c90 0x7f03980c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 -- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981e56f0 msgr2=0x7f03981e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981e56f0 0x7f03981e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 -- 172.21.15.125:0/3974864659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039815ebb0 con 0x7f03840039a0 2023-10-18T11:58:38.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f0396ffd640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 0x7f0398151160 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f0390005fd0 tx=0x7f0390031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:38.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f038ffff640 1 -- 172.21.15.125:0/3974864659 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039003f070 con 0x7f03840039a0 2023-10-18T11:58:38.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f038ffff640 1 -- 172.21.15.125:0/3974864659 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0390003970 con 0x7f03840039a0 2023-10-18T11:58:38.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03980c8ae0 con 0x7f03840039a0 2023-10-18T11:58:38.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f038ffff640 1 -- 172.21.15.125:0/3974864659 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390003d40 con 0x7f03840039a0 2023-10-18T11:58:38.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.648+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03980c90a0 con 0x7f03840039a0 2023-10-18T11:58:38.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.650+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03980c9230 con 0x7f03840039a0 2023-10-18T11:58:38.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.650+0000 7f038ffff640 1 -- 172.21.15.125:0/3974864659 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f039003a720 con 0x7f03840039a0 2023-10-18T11:58:38.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.650+0000 7f038ffff640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f037806af90 0x7f037806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:38.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.650+0000 7f038ffff640 1 -- 172.21.15.125:0/3974864659 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f03900ac390 con 0x7f03840039a0 2023-10-18T11:58:38.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.651+0000 7f03967fc640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f037806af90 0x7f037806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:38.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.651+0000 7f03967fc640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f037806af90 0x7f037806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0380001eb0 tx=0x7f0380001d80 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:38.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.655+0000 7f038ffff640 1 -- 172.21.15.125:0/3974864659 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0390077130 con 0x7f03840039a0 2023-10-18T11:58:38.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.852+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03980690e0 con 0x7f03840039a0 2023-10-18T11:58:38.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.852+0000 7f038ffff640 1 -- 172.21.15.125:0/3974864659 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f039007a750 con 0x7f03840039a0 2023-10-18T11:58:38.855 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:38.855 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":80,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2853562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2853562772},{"type":"v1","addr":"172.21.15.139:6836","nonce":2853562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":11421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3995754374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3995754374},{"type":"v1","addr":"172.21.15.125:6834","nonce":3995754374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":11424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2110336778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2110336778},{"type":"v1","addr":"172.21.15.125:6835","nonce":2110336778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:58:35.848069+0000","modified":"2023-10-18T11:58:37.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11439":{"gid":11439,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/1491435386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1491435386},{"type":"v1","addr":"172.21.15.139:6837","nonce":1491435386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-18T11:58:38.857 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 80 2023-10-18T11:58:38.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f037806af90 msgr2=0x7f037806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f037806af90 0x7f037806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0380001eb0 tx=0x7f0380001d80 comp rx=0 tx=0).stop 2023-10-18T11:58:38.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 msgr2=0x7f0398151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 0x7f0398151160 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f0390005fd0 tx=0x7f0390031720 comp rx=0 tx=0).stop 2023-10-18T11:58:38.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 shutdown_connections 2023-10-18T11:58:38.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f037806af90 0x7f037806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03981e56f0 0x7f03981e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03980c7c90 0x7f03980c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 --2- 172.21.15.125:0/3974864659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03840039a0 0x7f0398151160 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 >> 172.21.15.125:0/3974864659 conn(0x7f03980d0c50 msgr2=0x7f03980b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:38.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 shutdown_connections 2023-10-18T11:58:38.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.854+0000 7f039d900640 1 -- 172.21.15.125:0/3974864659 wait complete. 2023-10-18T11:58:38.868 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T11:58:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.968+0000 7f445a861640 1 -- 172.21.15.125:0/1598085370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 msgr2=0x7f4454069720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.968+0000 7f445a861640 1 --2- 172.21.15.125:0/1598085370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 0x7f4454069720 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f4448009830 tx=0x7f444802bc30 comp rx=0 tx=0).stop 2023-10-18T11:58:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.968+0000 7f445a861640 1 -- 172.21.15.125:0/1598085370 shutdown_connections 2023-10-18T11:58:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.968+0000 7f445a861640 1 --2- 172.21.15.125:0/1598085370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f445406cb00 0x7f4454069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.968+0000 7f445a861640 1 --2- 172.21.15.125:0/1598085370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 0x7f4454069720 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.968+0000 7f445a861640 1 -- 172.21.15.125:0/1598085370 >> 172.21.15.125:0/1598085370 conn(0x7f44540af830 msgr2=0x7f44540afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:38.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.968+0000 7f445a861640 1 -- 172.21.15.125:0/1598085370 shutdown_connections 2023-10-18T11:58:38.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.968+0000 7f445a861640 1 -- 172.21.15.125:0/1598085370 wait complete. 2023-10-18T11:58:38.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.969+0000 7f445a861640 1 Processor -- start 2023-10-18T11:58:38.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.969+0000 7f445a861640 1 -- start start 2023-10-18T11:58:38.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f445a861640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406cb00 0x7f4454139850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:38.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f445a861640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 0x7f4454137160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:38.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f445a861640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44541428d0 0x7f44541376a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:38.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f445a861640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44541e2080 con 0x7f445406da10 2023-10-18T11:58:38.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f445a861640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44541e2360 con 0x7f445406cb00 2023-10-18T11:58:38.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f445a861640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44541e2640 con 0x7f44541428d0 2023-10-18T11:58:38.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f44537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 0x7f4454137160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:38.976 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f44537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 0x7f4454137160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49970/0 (socket says 172.21.15.125:49970) 2023-10-18T11:58:38.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f44537fe640 1 -- 172.21.15.125:0/773925103 learned_addr learned my addr 172.21.15.125:0/773925103 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:38.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.970+0000 7f4453fff640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406cb00 0x7f4454139850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:38.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f44537fe640 1 -- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44541428d0 msgr2=0x7f44541376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f44537fe640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44541428d0 0x7f44541376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f44537fe640 1 -- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406cb00 msgr2=0x7f4454139850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:38.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f44537fe640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406cb00 0x7f4454139850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:38.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f44537fe640 1 -- 172.21.15.125:0/773925103 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4448009490 con 0x7f445406da10 2023-10-18T11:58:38.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f44537fe640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 0x7f4454137160 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f4448009950 tx=0x7f4448004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:38.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f4450ff9640 1 -- 172.21.15.125:0/773925103 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448005220 con 0x7f445406da10 2023-10-18T11:58:38.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44540cc8b0 con 0x7f445406da10 2023-10-18T11:58:38.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f4450ff9640 1 -- 172.21.15.125:0/773925103 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4448005470 con 0x7f445406da10 2023-10-18T11:58:38.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f4450ff9640 1 -- 172.21.15.125:0/773925103 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448003800 con 0x7f445406da10 2023-10-18T11:58:38.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.971+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44541e2c00 con 0x7f445406da10 2023-10-18T11:58:38.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.972+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44541e2e20 con 0x7f445406da10 2023-10-18T11:58:38.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.972+0000 7f4450ff9640 1 -- 172.21.15.125:0/773925103 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4448003960 con 0x7f445406da10 2023-10-18T11:58:38.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.973+0000 7f4450ff9640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f444006af60 0x7f444006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:38.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.973+0000 7f4450ff9640 1 -- 172.21.15.125:0/773925103 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f44480abf00 con 0x7f445406da10 2023-10-18T11:58:38.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.973+0000 7f4453fff640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f444006af60 0x7f444006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:38.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.974+0000 7f4453fff640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f444006af60 0x7f444006d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4444007fd0 tx=0x7f4444009040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:38.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:38.977+0000 7f4450ff9640 1 -- 172.21.15.125:0/773925103 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4448076d30 con 0x7f445406da10 2023-10-18T11:58:39.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.143+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44540690e0 con 0x7f445406da10 2023-10-18T11:58:39.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.144+0000 7f4450ff9640 1 -- 172.21.15.125:0/773925103 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v204) v1 ==== 74+0+16858 (secure 0 0 0) 0x7f444807a350 con 0x7f445406da10 2023-10-18T11:58:39.147 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:39.147 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":204,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:58:37.505501+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:58:34.060453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-18T11:58:34.823261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T11:58:39.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f444006af60 msgr2=0x7f444006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f444006af60 0x7f444006d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4444007fd0 tx=0x7f4444009040 comp rx=0 tx=0).stop 2023-10-18T11:58:39.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 msgr2=0x7f4454137160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 0x7f4454137160 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f4448009950 tx=0x7f4448004430 comp rx=0 tx=0).stop 2023-10-18T11:58:39.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 shutdown_connections 2023-10-18T11:58:39.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f444006af60 0x7f444006d420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44541428d0 0x7f44541376a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f445406da10 0x7f4454137160 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 --2- 172.21.15.125:0/773925103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406cb00 0x7f4454139850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 >> 172.21.15.125:0/773925103 conn(0x7f44540af830 msgr2=0x7f4454136050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:39.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 shutdown_connections 2023-10-18T11:58:39.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.146+0000 7f445a861640 1 -- 172.21.15.125:0/773925103 wait complete. 2023-10-18T11:58:39.159 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T11:58:39.160 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:58:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.259+0000 7f3bb31dc640 1 -- 172.21.15.125:0/2217737178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 msgr2=0x7f3bac06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.259+0000 7f3bb31dc640 1 --2- 172.21.15.125:0/2217737178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 0x7f3bac06b2e0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009860 tx=0x7f3b9c02bcd0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.259+0000 7f3bb31dc640 1 -- 172.21.15.125:0/2217737178 shutdown_connections 2023-10-18T11:58:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.259+0000 7f3bb31dc640 1 --2- 172.21.15.125:0/2217737178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 0x7f3bac06b2e0 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.259+0000 7f3bb31dc640 1 -- 172.21.15.125:0/2217737178 >> 172.21.15.125:0/2217737178 conn(0x7f3bac069930 msgr2=0x7f3bac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.259+0000 7f3bb31dc640 1 -- 172.21.15.125:0/2217737178 shutdown_connections 2023-10-18T11:58:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.259+0000 7f3bb31dc640 1 -- 172.21.15.125:0/2217737178 wait complete. 2023-10-18T11:58:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.260+0000 7f3bb31dc640 1 Processor -- start 2023-10-18T11:58:39.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.260+0000 7f3bb31dc640 1 -- start start 2023-10-18T11:58:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb31dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 0x7f3bac13d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb31dc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bac0abc10 0x7f3bac0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb31dc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bac0c96b0 0x7f3bac0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb31dc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac13cde0 con 0x7f3bac06af00 2023-10-18T11:58:39.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb31dc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac13d740 con 0x7f3bac0abc10 2023-10-18T11:58:39.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb31dc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac1f0170 con 0x7f3bac0c96b0 2023-10-18T11:58:39.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 0x7f3bac13d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 0x7f3bac13d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49994/0 (socket says 172.21.15.125:49994) 2023-10-18T11:58:39.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 -- 172.21.15.125:0/4080110553 learned_addr learned my addr 172.21.15.125:0/4080110553 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:39.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 -- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bac0c96b0 msgr2=0x7f3bac0adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bac0c96b0 0x7f3bac0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 -- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bac0abc10 msgr2=0x7f3bac0ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bac0abc10 0x7f3bac0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 -- 172.21.15.125:0/4080110553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009490 con 0x7f3bac06af00 2023-10-18T11:58:39.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb0f78640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 0x7f3bac13d200 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009780 tx=0x7f3b9c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:39.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3ba97fa640 1 -- 172.21.15.125:0/4080110553 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c003ea0 con 0x7f3bac06af00 2023-10-18T11:58:39.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3ba97fa640 1 -- 172.21.15.125:0/4080110553 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b9c005110 con 0x7f3bac06af00 2023-10-18T11:58:39.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3ba97fa640 1 -- 172.21.15.125:0/4080110553 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c005520 con 0x7f3bac06af00 2023-10-18T11:58:39.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bac1f0450 con 0x7f3bac06af00 2023-10-18T11:58:39.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.261+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac13ca00 con 0x7f3bac06af00 2023-10-18T11:58:39.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.262+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bac067930 con 0x7f3bac06af00 2023-10-18T11:58:39.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.263+0000 7f3ba97fa640 1 -- 172.21.15.125:0/4080110553 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3b9c0047a0 con 0x7f3bac06af00 2023-10-18T11:58:39.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.263+0000 7f3ba97fa640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3b9806af60 0x7f3b9806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.263+0000 7f3ba97fa640 1 -- 172.21.15.125:0/4080110553 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f3b9c0abea0 con 0x7f3bac06af00 2023-10-18T11:58:39.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.263+0000 7f3babfff640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3b9806af60 0x7f3b9806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.264+0000 7f3babfff640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3b9806af60 0x7f3b9806d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0ab4d0 tx=0x7f3ba0006ba0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:39.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.268+0000 7f3ba97fa640 1 -- 172.21.15.125:0/4080110553 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b9c076cc0 con 0x7f3bac06af00 2023-10-18T11:58:39.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.430+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 --> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bac159c50 con 0x7f3b9806af60 2023-10-18T11:58:39.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.438+0000 7f3ba97fa640 1 -- 172.21.15.125:0/4080110553 <== mgr.9529 v2:172.21.15.139:6832/1565260611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288588 (secure 0 0 0) 0x7f3ba0012390 con 0x7f3b9806af60 2023-10-18T11:58:39.444 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:39.445 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:58:39.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3b9806af60 msgr2=0x7f3b9806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3b9806af60 0x7f3b9806d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0ab4d0 tx=0x7f3ba0006ba0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 msgr2=0x7f3bac13d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 0x7f3bac13d200 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009780 tx=0x7f3b9c0038a0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 shutdown_connections 2023-10-18T11:58:39.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3b9806af60 0x7f3b9806d420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bac0abc10 0x7f3bac0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bac0c96b0 0x7f3bac0adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 --2- 172.21.15.125:0/4080110553 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bac06af00 0x7f3bac13d200 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 >> 172.21.15.125:0/4080110553 conn(0x7f3bac069930 msgr2=0x7f3bac15bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:39.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 shutdown_connections 2023-10-18T11:58:39.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.441+0000 7f3bb31dc640 1 -- 172.21.15.125:0/4080110553 wait complete. 2023-10-18T11:58:39.454 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T11:58:38.893982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":598,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":372,"ondisk_log_size":372,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":402452,"kb_used_data":21148,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754572268,"statfs":{"total":773094113280,"available":772682002432,"internally_reserved":0,"allocated":21655552,"data_stored":11588381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":390319672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.394160"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802702+0000","last_change":"2023-10-18T11:58:36.522565+0000","last_active":"2023-10-18T11:58:37.802702+0000","last_peered":"2023-10-18T11:58:37.802702+0000","last_clean":"2023-10-18T11:58:37.802702+0000","last_became_active":"2023-10-18T11:58:36.521736+0000","last_became_peered":"2023-10-18T11:58:36.521736+0000","last_unstale":"2023-10-18T11:58:37.802702+0000","last_undegraded":"2023-10-18T11:58:37.802702+0000","last_fullsized":"2023-10-18T11:58:37.802702+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:46:36.674788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802006+0000","last_change":"2023-10-18T11:58:36.527651+0000","last_active":"2023-10-18T11:58:37.802006+0000","last_peered":"2023-10-18T11:58:37.802006+0000","last_clean":"2023-10-18T11:58:37.802006+0000","last_became_active":"2023-10-18T11:58:36.527375+0000","last_became_peered":"2023-10-18T11:58:36.527375+0000","last_unstale":"2023-10-18T11:58:37.802006+0000","last_undegraded":"2023-10-18T11:58:37.802006+0000","last_fullsized":"2023-10-18T11:58:37.802006+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:55:29.683103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.534333+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.534066+0000","last_became_peered":"2023-10-18T11:58:36.534066+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:29.879395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508872+0000","last_change":"2023-10-18T11:58:36.529615+0000","last_active":"2023-10-18T11:58:37.508872+0000","last_peered":"2023-10-18T11:58:37.508872+0000","last_clean":"2023-10-18T11:58:37.508872+0000","last_became_active":"2023-10-18T11:58:36.529380+0000","last_became_peered":"2023-10-18T11:58:36.529380+0000","last_unstale":"2023-10-18T11:58:37.508872+0000","last_undegraded":"2023-10-18T11:58:37.508872+0000","last_fullsized":"2023-10-18T11:58:37.508872+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:44.701593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508825+0000","last_change":"2023-10-18T11:58:36.528591+0000","last_active":"2023-10-18T11:58:37.508825+0000","last_peered":"2023-10-18T11:58:37.508825+0000","last_clean":"2023-10-18T11:58:37.508825+0000","last_became_active":"2023-10-18T11:58:36.528332+0000","last_became_peered":"2023-10-18T11:58:36.528332+0000","last_unstale":"2023-10-18T11:58:37.508825+0000","last_undegraded":"2023-10-18T11:58:37.508825+0000","last_fullsized":"2023-10-18T11:58:37.508825+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:04.968822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802608+0000","last_change":"2023-10-18T11:58:36.526452+0000","last_active":"2023-10-18T11:58:37.802608+0000","last_peered":"2023-10-18T11:58:37.802608+0000","last_clean":"2023-10-18T11:58:37.802608+0000","last_became_active":"2023-10-18T11:58:36.526144+0000","last_became_peered":"2023-10-18T11:58:36.526144+0000","last_unstale":"2023-10-18T11:58:37.802608+0000","last_undegraded":"2023-10-18T11:58:37.802608+0000","last_fullsized":"2023-10-18T11:58:37.802608+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:15.609770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802428+0000","last_change":"2023-10-18T11:58:36.526263+0000","last_active":"2023-10-18T11:58:37.802428+0000","last_peered":"2023-10-18T11:58:37.802428+0000","last_clean":"2023-10-18T11:58:37.802428+0000","last_became_active":"2023-10-18T11:58:36.525882+0000","last_became_peered":"2023-10-18T11:58:36.525882+0000","last_unstale":"2023-10-18T11:58:37.802428+0000","last_undegraded":"2023-10-18T11:58:37.802428+0000","last_fullsized":"2023-10-18T11:58:37.802428+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:05.910088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802812+0000","last_change":"2023-10-18T11:58:36.520303+0000","last_active":"2023-10-18T11:58:37.802812+0000","last_peered":"2023-10-18T11:58:37.802812+0000","last_clean":"2023-10-18T11:58:37.802812+0000","last_became_active":"2023-10-18T11:58:36.520065+0000","last_became_peered":"2023-10-18T11:58:36.520065+0000","last_unstale":"2023-10-18T11:58:37.802812+0000","last_undegraded":"2023-10-18T11:58:37.802812+0000","last_fullsized":"2023-10-18T11:58:37.802812+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:40.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509204+0000","last_change":"2023-10-18T11:58:36.534318+0000","last_active":"2023-10-18T11:58:37.509204+0000","last_peered":"2023-10-18T11:58:37.509204+0000","last_clean":"2023-10-18T11:58:37.509204+0000","last_became_active":"2023-10-18T11:58:36.534071+0000","last_became_peered":"2023-10-18T11:58:36.534071+0000","last_unstale":"2023-10-18T11:58:37.509204+0000","last_undegraded":"2023-10-18T11:58:37.509204+0000","last_fullsized":"2023-10-18T11:58:37.509204+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:55.491621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802310+0000","last_change":"2023-10-18T11:58:36.520313+0000","last_active":"2023-10-18T11:58:37.802310+0000","last_peered":"2023-10-18T11:58:37.802310+0000","last_clean":"2023-10-18T11:58:37.802310+0000","last_became_active":"2023-10-18T11:58:36.520042+0000","last_became_peered":"2023-10-18T11:58:36.520042+0000","last_unstale":"2023-10-18T11:58:37.802310+0000","last_undegraded":"2023-10-18T11:58:37.802310+0000","last_fullsized":"2023-10-18T11:58:37.802310+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:57.623658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802462+0000","last_change":"2023-10-18T11:58:36.526211+0000","last_active":"2023-10-18T11:58:37.802462+0000","last_peered":"2023-10-18T11:58:37.802462+0000","last_clean":"2023-10-18T11:58:37.802462+0000","last_became_active":"2023-10-18T11:58:36.525812+0000","last_became_peered":"2023-10-18T11:58:36.525812+0000","last_unstale":"2023-10-18T11:58:37.802462+0000","last_undegraded":"2023-10-18T11:58:37.802462+0000","last_fullsized":"2023-10-18T11:58:37.802462+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:39.938700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512711+0000","last_change":"2023-10-18T11:58:36.529732+0000","last_active":"2023-10-18T11:58:37.512711+0000","last_peered":"2023-10-18T11:58:37.512711+0000","last_clean":"2023-10-18T11:58:37.512711+0000","last_became_active":"2023-10-18T11:58:36.529455+0000","last_became_peered":"2023-10-18T11:58:36.529455+0000","last_unstale":"2023-10-18T11:58:37.512711+0000","last_undegraded":"2023-10-18T11:58:37.512711+0000","last_fullsized":"2023-10-18T11:58:37.512711+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:38.419107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802037+0000","last_change":"2023-10-18T11:58:36.534249+0000","last_active":"2023-10-18T11:58:37.802037+0000","last_peered":"2023-10-18T11:58:37.802037+0000","last_clean":"2023-10-18T11:58:37.802037+0000","last_became_active":"2023-10-18T11:58:36.533956+0000","last_became_peered":"2023-10-18T11:58:36.533956+0000","last_unstale":"2023-10-18T11:58:37.802037+0000","last_undegraded":"2023-10-18T11:58:37.802037+0000","last_fullsized":"2023-10-18T11:58:37.802037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:29.001140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802132+0000","last_change":"2023-10-18T11:58:36.530863+0000","last_active":"2023-10-18T11:58:37.802132+0000","last_peered":"2023-10-18T11:58:37.802132+0000","last_clean":"2023-10-18T11:58:37.802132+0000","last_became_active":"2023-10-18T11:58:36.530625+0000","last_became_peered":"2023-10-18T11:58:36.530625+0000","last_unstale":"2023-10-18T11:58:37.802132+0000","last_undegraded":"2023-10-18T11:58:37.802132+0000","last_fullsized":"2023-10-18T11:58:37.802132+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:03.443134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508994+0000","last_change":"2023-10-18T11:58:36.531552+0000","last_active":"2023-10-18T11:58:37.508994+0000","last_peered":"2023-10-18T11:58:37.508994+0000","last_clean":"2023-10-18T11:58:37.508994+0000","last_became_active":"2023-10-18T11:58:36.531163+0000","last_became_peered":"2023-10-18T11:58:36.531163+0000","last_unstale":"2023-10-18T11:58:37.508994+0000","last_undegraded":"2023-10-18T11:58:37.508994+0000","last_fullsized":"2023-10-18T11:58:37.508994+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:49.241771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509268+0000","last_change":"2023-10-18T11:58:36.530026+0000","last_active":"2023-10-18T11:58:37.509268+0000","last_peered":"2023-10-18T11:58:37.509268+0000","last_clean":"2023-10-18T11:58:37.509268+0000","last_became_active":"2023-10-18T11:58:36.529058+0000","last_became_peered":"2023-10-18T11:58:36.529058+0000","last_unstale":"2023-10-18T11:58:37.509268+0000","last_undegraded":"2023-10-18T11:58:37.509268+0000","last_fullsized":"2023-10-18T11:58:37.509268+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:41.289044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513034+0000","last_change":"2023-10-18T11:58:36.532114+0000","last_active":"2023-10-18T11:58:37.513034+0000","last_peered":"2023-10-18T11:58:37.513034+0000","last_clean":"2023-10-18T11:58:37.513034+0000","last_became_active":"2023-10-18T11:58:36.531526+0000","last_became_peered":"2023-10-18T11:58:36.531526+0000","last_unstale":"2023-10-18T11:58:37.513034+0000","last_undegraded":"2023-10-18T11:58:37.513034+0000","last_fullsized":"2023-10-18T11:58:37.513034+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:38.904126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512884+0000","last_change":"2023-10-18T11:58:36.534033+0000","last_active":"2023-10-18T11:58:37.512884+0000","last_peered":"2023-10-18T11:58:37.512884+0000","last_clean":"2023-10-18T11:58:37.512884+0000","last_became_active":"2023-10-18T11:58:36.533772+0000","last_became_peered":"2023-10-18T11:58:36.533772+0000","last_unstale":"2023-10-18T11:58:37.512884+0000","last_undegraded":"2023-10-18T11:58:37.512884+0000","last_fullsized":"2023-10-18T11:58:37.512884+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:04.963404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508988+0000","last_change":"2023-10-18T11:58:36.529660+0000","last_active":"2023-10-18T11:58:37.508988+0000","last_peered":"2023-10-18T11:58:37.508988+0000","last_clean":"2023-10-18T11:58:37.508988+0000","last_became_active":"2023-10-18T11:58:36.529125+0000","last_became_peered":"2023-10-18T11:58:36.529125+0000","last_unstale":"2023-10-18T11:58:37.508988+0000","last_undegraded":"2023-10-18T11:58:37.508988+0000","last_fullsized":"2023-10-18T11:58:37.508988+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:01.717669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802201+0000","last_change":"2023-10-18T11:58:36.519499+0000","last_active":"2023-10-18T11:58:37.802201+0000","last_peered":"2023-10-18T11:58:37.802201+0000","last_clean":"2023-10-18T11:58:37.802201+0000","last_became_active":"2023-10-18T11:58:36.519246+0000","last_became_peered":"2023-10-18T11:58:36.519246+0000","last_unstale":"2023-10-18T11:58:37.802201+0000","last_undegraded":"2023-10-18T11:58:37.802201+0000","last_fullsized":"2023-10-18T11:58:37.802201+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:48.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802474+0000","last_change":"2023-10-18T11:58:36.514398+0000","last_active":"2023-10-18T11:58:37.802474+0000","last_peered":"2023-10-18T11:58:37.802474+0000","last_clean":"2023-10-18T11:58:37.802474+0000","last_became_active":"2023-10-18T11:58:36.514141+0000","last_became_peered":"2023-10-18T11:58:36.514141+0000","last_unstale":"2023-10-18T11:58:37.802474+0000","last_undegraded":"2023-10-18T11:58:37.802474+0000","last_fullsized":"2023-10-18T11:58:37.802474+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:43.247625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802305+0000","last_change":"2023-10-18T11:58:36.527748+0000","last_active":"2023-10-18T11:58:37.802305+0000","last_peered":"2023-10-18T11:58:37.802305+0000","last_clean":"2023-10-18T11:58:37.802305+0000","last_became_active":"2023-10-18T11:58:36.527498+0000","last_became_peered":"2023-10-18T11:58:36.527498+0000","last_unstale":"2023-10-18T11:58:37.802305+0000","last_undegraded":"2023-10-18T11:58:37.802305+0000","last_fullsized":"2023-10-18T11:58:37.802305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:35.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801779+0000","last_change":"2023-10-18T11:58:36.521950+0000","last_active":"2023-10-18T11:58:37.801779+0000","last_peered":"2023-10-18T11:58:37.801779+0000","last_clean":"2023-10-18T11:58:37.801779+0000","last_became_active":"2023-10-18T11:58:36.521681+0000","last_became_peered":"2023-10-18T11:58:36.521681+0000","last_unstale":"2023-10-18T11:58:37.801779+0000","last_undegraded":"2023-10-18T11:58:37.801779+0000","last_fullsized":"2023-10-18T11:58:37.801779+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:43.982016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802739+0000","last_change":"2023-10-18T11:58:36.518867+0000","last_active":"2023-10-18T11:58:37.802739+0000","last_peered":"2023-10-18T11:58:37.802739+0000","last_clean":"2023-10-18T11:58:37.802739+0000","last_became_active":"2023-10-18T11:58:36.518625+0000","last_became_peered":"2023-10-18T11:58:36.518625+0000","last_unstale":"2023-10-18T11:58:37.802739+0000","last_undegraded":"2023-10-18T11:58:37.802739+0000","last_fullsized":"2023-10-18T11:58:37.802739+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:20.411248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802849+0000","last_change":"2023-10-18T11:58:36.528722+0000","last_active":"2023-10-18T11:58:37.802849+0000","last_peered":"2023-10-18T11:58:37.802849+0000","last_clean":"2023-10-18T11:58:37.802849+0000","last_became_active":"2023-10-18T11:58:36.528462+0000","last_became_peered":"2023-10-18T11:58:36.528462+0000","last_unstale":"2023-10-18T11:58:37.802849+0000","last_undegraded":"2023-10-18T11:58:37.802849+0000","last_fullsized":"2023-10-18T11:58:37.802849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:58:27.045894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803175+0000","last_change":"2023-10-18T11:58:36.527594+0000","last_active":"2023-10-18T11:58:37.803175+0000","last_peered":"2023-10-18T11:58:37.803175+0000","last_clean":"2023-10-18T11:58:37.803175+0000","last_became_active":"2023-10-18T11:58:36.526226+0000","last_became_peered":"2023-10-18T11:58:36.526226+0000","last_unstale":"2023-10-18T11:58:37.803175+0000","last_undegraded":"2023-10-18T11:58:37.803175+0000","last_fullsized":"2023-10-18T11:58:37.803175+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:28.446100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509068+0000","last_change":"2023-10-18T11:58:36.529803+0000","last_active":"2023-10-18T11:58:37.509068+0000","last_peered":"2023-10-18T11:58:37.509068+0000","last_clean":"2023-10-18T11:58:37.509068+0000","last_became_active":"2023-10-18T11:58:36.529446+0000","last_became_peered":"2023-10-18T11:58:36.529446+0000","last_unstale":"2023-10-18T11:58:37.509068+0000","last_undegraded":"2023-10-18T11:58:37.509068+0000","last_fullsized":"2023-10-18T11:58:37.509068+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:19.031434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802519+0000","last_change":"2023-10-18T11:58:36.526866+0000","last_active":"2023-10-18T11:58:37.802519+0000","last_peered":"2023-10-18T11:58:37.802519+0000","last_clean":"2023-10-18T11:58:37.802519+0000","last_became_active":"2023-10-18T11:58:36.526609+0000","last_became_peered":"2023-10-18T11:58:36.526609+0000","last_unstale":"2023-10-18T11:58:37.802519+0000","last_undegraded":"2023-10-18T11:58:37.802519+0000","last_fullsized":"2023-10-18T11:58:37.802519+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:04.860806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802258+0000","last_change":"2023-10-18T11:58:36.515793+0000","last_active":"2023-10-18T11:58:37.802258+0000","last_peered":"2023-10-18T11:58:37.802258+0000","last_clean":"2023-10-18T11:58:37.802258+0000","last_became_active":"2023-10-18T11:58:36.515523+0000","last_became_peered":"2023-10-18T11:58:36.515523+0000","last_unstale":"2023-10-18T11:58:37.802258+0000","last_undegraded":"2023-10-18T11:58:37.802258+0000","last_fullsized":"2023-10-18T11:58:37.802258+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802828+0000","last_change":"2023-10-18T11:58:36.515930+0000","last_active":"2023-10-18T11:58:37.802828+0000","last_peered":"2023-10-18T11:58:37.802828+0000","last_clean":"2023-10-18T11:58:37.802828+0000","last_became_active":"2023-10-18T11:58:36.515681+0000","last_became_peered":"2023-10-18T11:58:36.515681+0000","last_unstale":"2023-10-18T11:58:37.802828+0000","last_undegraded":"2023-10-18T11:58:37.802828+0000","last_fullsized":"2023-10-18T11:58:37.802828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:48:47.733918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508862+0000","last_change":"2023-10-18T11:58:36.530777+0000","last_active":"2023-10-18T11:58:37.508862+0000","last_peered":"2023-10-18T11:58:37.508862+0000","last_clean":"2023-10-18T11:58:37.508862+0000","last_became_active":"2023-10-18T11:58:36.530454+0000","last_became_peered":"2023-10-18T11:58:36.530454+0000","last_unstale":"2023-10-18T11:58:37.508862+0000","last_undegraded":"2023-10-18T11:58:37.508862+0000","last_fullsized":"2023-10-18T11:58:37.508862+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:00.542319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803195+0000","last_change":"2023-10-18T11:58:36.525494+0000","last_active":"2023-10-18T11:58:37.803195+0000","last_peered":"2023-10-18T11:58:37.803195+0000","last_clean":"2023-10-18T11:58:37.803195+0000","last_became_active":"2023-10-18T11:58:36.525200+0000","last_became_peered":"2023-10-18T11:58:36.525200+0000","last_unstale":"2023-10-18T11:58:37.803195+0000","last_undegraded":"2023-10-18T11:58:37.803195+0000","last_fullsized":"2023-10-18T11:58:37.803195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"203'4","reported_seq":16,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803001+0000","last_change":"2023-10-18T11:58:35.527076+0000","last_active":"2023-10-18T11:58:37.803001+0000","last_peered":"2023-10-18T11:58:37.803001+0000","last_clean":"2023-10-18T11:58:37.803001+0000","last_became_active":"2023-10-18T11:58:35.526809+0000","last_became_peered":"2023-10-18T11:58:35.526809+0000","last_unstale":"2023-10-18T11:58:37.803001+0000","last_undegraded":"2023-10-18T11:58:37.803001+0000","last_fullsized":"2023-10-18T11:58:37.803001+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802906+0000","last_change":"2023-10-18T11:58:36.521457+0000","last_active":"2023-10-18T11:58:37.802906+0000","last_peered":"2023-10-18T11:58:37.802906+0000","last_clean":"2023-10-18T11:58:37.802906+0000","last_became_active":"2023-10-18T11:58:36.521209+0000","last_became_peered":"2023-10-18T11:58:36.521209+0000","last_unstale":"2023-10-18T11:58:37.802906+0000","last_undegraded":"2023-10-18T11:58:37.802906+0000","last_fullsized":"2023-10-18T11:58:37.802906+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:40.559400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509267+0000","last_change":"2023-10-18T11:58:35.521828+0000","last_active":"2023-10-18T11:58:37.509267+0000","last_peered":"2023-10-18T11:58:37.509267+0000","last_clean":"2023-10-18T11:58:37.509267+0000","last_became_active":"2023-10-18T11:58:35.521574+0000","last_became_peered":"2023-10-18T11:58:35.521574+0000","last_unstale":"2023-10-18T11:58:37.509267+0000","last_undegraded":"2023-10-18T11:58:37.509267+0000","last_fullsized":"2023-10-18T11:58:37.509267+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:47.472064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803025+0000","last_change":"2023-10-18T11:58:36.522588+0000","last_active":"2023-10-18T11:58:37.803025+0000","last_peered":"2023-10-18T11:58:37.803025+0000","last_clean":"2023-10-18T11:58:37.803025+0000","last_became_active":"2023-10-18T11:58:36.522182+0000","last_became_peered":"2023-10-18T11:58:36.522182+0000","last_unstale":"2023-10-18T11:58:37.803025+0000","last_undegraded":"2023-10-18T11:58:37.803025+0000","last_fullsized":"2023-10-18T11:58:37.803025+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.326674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802389+0000","last_change":"2023-10-18T11:58:35.516307+0000","last_active":"2023-10-18T11:58:37.802389+0000","last_peered":"2023-10-18T11:58:37.802389+0000","last_clean":"2023-10-18T11:58:37.802389+0000","last_became_active":"2023-10-18T11:58:35.515995+0000","last_became_peered":"2023-10-18T11:58:35.515995+0000","last_unstale":"2023-10-18T11:58:37.802389+0000","last_undegraded":"2023-10-18T11:58:37.802389+0000","last_fullsized":"2023-10-18T11:58:37.802389+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:43.302735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512924+0000","last_change":"2023-10-18T11:58:35.526193+0000","last_active":"2023-10-18T11:58:37.512924+0000","last_peered":"2023-10-18T11:58:37.512924+0000","last_clean":"2023-10-18T11:58:37.512924+0000","last_became_active":"2023-10-18T11:58:35.525920+0000","last_became_peered":"2023-10-18T11:58:35.525920+0000","last_unstale":"2023-10-18T11:58:37.512924+0000","last_undegraded":"2023-10-18T11:58:37.512924+0000","last_fullsized":"2023-10-18T11:58:37.512924+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:17.858763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512416+0000","last_change":"2023-10-18T11:58:36.529406+0000","last_active":"2023-10-18T11:58:37.512416+0000","last_peered":"2023-10-18T11:58:37.512416+0000","last_clean":"2023-10-18T11:58:37.512416+0000","last_became_active":"2023-10-18T11:58:36.528795+0000","last_became_peered":"2023-10-18T11:58:36.528795+0000","last_unstale":"2023-10-18T11:58:37.512416+0000","last_undegraded":"2023-10-18T11:58:37.512416+0000","last_fullsized":"2023-10-18T11:58:37.512416+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:26.184608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509045+0000","last_change":"2023-10-18T11:58:35.523280+0000","last_active":"2023-10-18T11:58:37.509045+0000","last_peered":"2023-10-18T11:58:37.509045+0000","last_clean":"2023-10-18T11:58:37.509045+0000","last_became_active":"2023-10-18T11:58:35.522621+0000","last_became_peered":"2023-10-18T11:58:35.522621+0000","last_unstale":"2023-10-18T11:58:37.509045+0000","last_undegraded":"2023-10-18T11:58:37.509045+0000","last_fullsized":"2023-10-18T11:58:37.509045+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:27.443746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802321+0000","last_change":"2023-10-18T11:58:36.516159+0000","last_active":"2023-10-18T11:58:37.802321+0000","last_peered":"2023-10-18T11:58:37.802321+0000","last_clean":"2023-10-18T11:58:37.802321+0000","last_became_active":"2023-10-18T11:58:36.515894+0000","last_became_peered":"2023-10-18T11:58:36.515894+0000","last_unstale":"2023-10-18T11:58:37.802321+0000","last_undegraded":"2023-10-18T11:58:37.802321+0000","last_fullsized":"2023-10-18T11:58:37.802321+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:42.645151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802627+0000","last_change":"2023-10-18T11:58:35.521126+0000","last_active":"2023-10-18T11:58:37.802627+0000","last_peered":"2023-10-18T11:58:37.802627+0000","last_clean":"2023-10-18T11:58:37.802627+0000","last_became_active":"2023-10-18T11:58:35.520861+0000","last_became_peered":"2023-10-18T11:58:35.520861+0000","last_unstale":"2023-10-18T11:58:37.802627+0000","last_undegraded":"2023-10-18T11:58:37.802627+0000","last_fullsized":"2023-10-18T11:58:37.802627+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802501+0000","last_change":"2023-10-18T11:58:36.531084+0000","last_active":"2023-10-18T11:58:37.802501+0000","last_peered":"2023-10-18T11:58:37.802501+0000","last_clean":"2023-10-18T11:58:37.802501+0000","last_became_active":"2023-10-18T11:58:36.530792+0000","last_became_peered":"2023-10-18T11:58:36.530792+0000","last_unstale":"2023-10-18T11:58:37.802501+0000","last_undegraded":"2023-10-18T11:58:37.802501+0000","last_fullsized":"2023-10-18T11:58:37.802501+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:37.483860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512533+0000","last_change":"2023-10-18T11:58:35.518246+0000","last_active":"2023-10-18T11:58:37.512533+0000","last_peered":"2023-10-18T11:58:37.512533+0000","last_clean":"2023-10-18T11:58:37.512533+0000","last_became_active":"2023-10-18T11:58:35.517986+0000","last_became_peered":"2023-10-18T11:58:35.517986+0000","last_unstale":"2023-10-18T11:58:37.512533+0000","last_undegraded":"2023-10-18T11:58:37.512533+0000","last_fullsized":"2023-10-18T11:58:37.512533+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:02:15.620821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803081+0000","last_change":"2023-10-18T11:58:36.533670+0000","last_active":"2023-10-18T11:58:37.803081+0000","last_peered":"2023-10-18T11:58:37.803081+0000","last_clean":"2023-10-18T11:58:37.803081+0000","last_became_active":"2023-10-18T11:58:36.533394+0000","last_became_peered":"2023-10-18T11:58:36.533394+0000","last_unstale":"2023-10-18T11:58:37.803081+0000","last_undegraded":"2023-10-18T11:58:37.803081+0000","last_fullsized":"2023-10-18T11:58:37.803081+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:21.396797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509229+0000","last_change":"2023-10-18T11:58:35.524709+0000","last_active":"2023-10-18T11:58:37.509229+0000","last_peered":"2023-10-18T11:58:37.509229+0000","last_clean":"2023-10-18T11:58:37.509229+0000","last_became_active":"2023-10-18T11:58:35.524471+0000","last_became_peered":"2023-10-18T11:58:35.524471+0000","last_unstale":"2023-10-18T11:58:37.509229+0000","last_undegraded":"2023-10-18T11:58:37.509229+0000","last_fullsized":"2023-10-18T11:58:37.509229+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:53.288380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509250+0000","last_change":"2023-10-18T11:58:36.523093+0000","last_active":"2023-10-18T11:58:37.509250+0000","last_peered":"2023-10-18T11:58:37.509250+0000","last_clean":"2023-10-18T11:58:37.509250+0000","last_became_active":"2023-10-18T11:58:36.522551+0000","last_became_peered":"2023-10-18T11:58:36.522551+0000","last_unstale":"2023-10-18T11:58:37.509250+0000","last_undegraded":"2023-10-18T11:58:37.509250+0000","last_fullsized":"2023-10-18T11:58:37.509250+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:22.715669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802085+0000","last_change":"2023-10-18T11:58:35.522093+0000","last_active":"2023-10-18T11:58:37.802085+0000","last_peered":"2023-10-18T11:58:37.802085+0000","last_clean":"2023-10-18T11:58:37.802085+0000","last_became_active":"2023-10-18T11:58:35.521805+0000","last_became_peered":"2023-10-18T11:58:35.521805+0000","last_unstale":"2023-10-18T11:58:37.802085+0000","last_undegraded":"2023-10-18T11:58:37.802085+0000","last_fullsized":"2023-10-18T11:58:37.802085+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:08.791931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513049+0000","last_change":"2023-10-18T11:58:36.531254+0000","last_active":"2023-10-18T11:58:37.513049+0000","last_peered":"2023-10-18T11:58:37.513049+0000","last_clean":"2023-10-18T11:58:37.513049+0000","last_became_active":"2023-10-18T11:58:36.530967+0000","last_became_peered":"2023-10-18T11:58:36.530967+0000","last_unstale":"2023-10-18T11:58:37.513049+0000","last_undegraded":"2023-10-18T11:58:37.513049+0000","last_fullsized":"2023-10-18T11:58:37.513049+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:01.297812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"203'3","reported_seq":15,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802710+0000","last_change":"2023-10-18T11:58:35.525207+0000","last_active":"2023-10-18T11:58:37.802710+0000","last_peered":"2023-10-18T11:58:37.802710+0000","last_clean":"2023-10-18T11:58:37.802710+0000","last_became_active":"2023-10-18T11:58:35.524946+0000","last_became_peered":"2023-10-18T11:58:35.524946+0000","last_unstale":"2023-10-18T11:58:37.802710+0000","last_undegraded":"2023-10-18T11:58:37.802710+0000","last_fullsized":"2023-10-18T11:58:37.802710+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802920+0000","last_change":"2023-10-18T11:58:36.516878+0000","last_active":"2023-10-18T11:58:37.802920+0000","last_peered":"2023-10-18T11:58:37.802920+0000","last_clean":"2023-10-18T11:58:37.802920+0000","last_became_active":"2023-10-18T11:58:36.516625+0000","last_became_peered":"2023-10-18T11:58:36.516625+0000","last_unstale":"2023-10-18T11:58:37.802920+0000","last_undegraded":"2023-10-18T11:58:37.802920+0000","last_fullsized":"2023-10-18T11:58:37.802920+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:59.525275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"203'6","reported_seq":18,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513026+0000","last_change":"2023-10-18T11:58:35.518030+0000","last_active":"2023-10-18T11:58:37.513026+0000","last_peered":"2023-10-18T11:58:37.513026+0000","last_clean":"2023-10-18T11:58:37.513026+0000","last_became_active":"2023-10-18T11:58:35.517747+0000","last_became_peered":"2023-10-18T11:58:35.517747+0000","last_unstale":"2023-10-18T11:58:37.513026+0000","last_undegraded":"2023-10-18T11:58:37.513026+0000","last_fullsized":"2023-10-18T11:58:37.513026+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.335660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801895+0000","last_change":"2023-10-18T11:58:36.524788+0000","last_active":"2023-10-18T11:58:37.801895+0000","last_peered":"2023-10-18T11:58:37.801895+0000","last_clean":"2023-10-18T11:58:37.801895+0000","last_became_active":"2023-10-18T11:58:36.524547+0000","last_became_peered":"2023-10-18T11:58:36.524547+0000","last_unstale":"2023-10-18T11:58:37.801895+0000","last_undegraded":"2023-10-18T11:58:37.801895+0000","last_fullsized":"2023-10-18T11:58:37.801895+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:02.661591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512833+0000","last_change":"2023-10-18T11:58:35.521655+0000","last_active":"2023-10-18T11:58:37.512833+0000","last_peered":"2023-10-18T11:58:37.512833+0000","last_clean":"2023-10-18T11:58:37.512833+0000","last_became_active":"2023-10-18T11:58:35.521430+0000","last_became_peered":"2023-10-18T11:58:35.521430+0000","last_unstale":"2023-10-18T11:58:37.512833+0000","last_undegraded":"2023-10-18T11:58:37.512833+0000","last_fullsized":"2023-10-18T11:58:37.512833+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:21.628472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508795+0000","last_change":"2023-10-18T11:58:36.523743+0000","last_active":"2023-10-18T11:58:37.508795+0000","last_peered":"2023-10-18T11:58:37.508795+0000","last_clean":"2023-10-18T11:58:37.508795+0000","last_became_active":"2023-10-18T11:58:36.523492+0000","last_became_peered":"2023-10-18T11:58:36.523492+0000","last_unstale":"2023-10-18T11:58:37.508795+0000","last_undegraded":"2023-10-18T11:58:37.508795+0000","last_fullsized":"2023-10-18T11:58:37.508795+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:42.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802579+0000","last_change":"2023-10-18T11:58:35.522746+0000","last_active":"2023-10-18T11:58:37.802579+0000","last_peered":"2023-10-18T11:58:37.802579+0000","last_clean":"2023-10-18T11:58:37.802579+0000","last_became_active":"2023-10-18T11:58:35.522486+0000","last_became_peered":"2023-10-18T11:58:35.522486+0000","last_unstale":"2023-10-18T11:58:37.802579+0000","last_undegraded":"2023-10-18T11:58:37.802579+0000","last_fullsized":"2023-10-18T11:58:37.802579+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:07.062822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512576+0000","last_change":"2023-10-18T11:58:36.533848+0000","last_active":"2023-10-18T11:58:37.512576+0000","last_peered":"2023-10-18T11:58:37.512576+0000","last_clean":"2023-10-18T11:58:37.512576+0000","last_became_active":"2023-10-18T11:58:36.533565+0000","last_became_peered":"2023-10-18T11:58:36.533565+0000","last_unstale":"2023-10-18T11:58:37.512576+0000","last_undegraded":"2023-10-18T11:58:37.512576+0000","last_fullsized":"2023-10-18T11:58:37.512576+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:48.249824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802427+0000","last_change":"2023-10-18T11:58:35.517587+0000","last_active":"2023-10-18T11:58:37.802427+0000","last_peered":"2023-10-18T11:58:37.802427+0000","last_clean":"2023-10-18T11:58:37.802427+0000","last_became_active":"2023-10-18T11:58:35.517322+0000","last_became_peered":"2023-10-18T11:58:35.517322+0000","last_unstale":"2023-10-18T11:58:37.802427+0000","last_undegraded":"2023-10-18T11:58:37.802427+0000","last_fullsized":"2023-10-18T11:58:37.802427+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:34.567188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512733+0000","last_change":"2023-10-18T11:58:36.531687+0000","last_active":"2023-10-18T11:58:37.512733+0000","last_peered":"2023-10-18T11:58:37.512733+0000","last_clean":"2023-10-18T11:58:37.512733+0000","last_became_active":"2023-10-18T11:58:36.531411+0000","last_became_peered":"2023-10-18T11:58:36.531411+0000","last_unstale":"2023-10-18T11:58:37.512733+0000","last_undegraded":"2023-10-18T11:58:37.512733+0000","last_fullsized":"2023-10-18T11:58:37.512733+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:04.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"203'8","reported_seq":20,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802771+0000","last_change":"2023-10-18T11:58:35.515506+0000","last_active":"2023-10-18T11:58:37.802771+0000","last_peered":"2023-10-18T11:58:37.802771+0000","last_clean":"2023-10-18T11:58:37.802771+0000","last_became_active":"2023-10-18T11:58:35.515198+0000","last_became_peered":"2023-10-18T11:58:35.515198+0000","last_unstale":"2023-10-18T11:58:37.802771+0000","last_undegraded":"2023-10-18T11:58:37.802771+0000","last_fullsized":"2023-10-18T11:58:37.802771+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.509931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802367+0000","last_change":"2023-10-18T11:58:36.515935+0000","last_active":"2023-10-18T11:58:37.802367+0000","last_peered":"2023-10-18T11:58:37.802367+0000","last_clean":"2023-10-18T11:58:37.802367+0000","last_became_active":"2023-10-18T11:58:36.515722+0000","last_became_peered":"2023-10-18T11:58:36.515722+0000","last_unstale":"2023-10-18T11:58:37.802367+0000","last_undegraded":"2023-10-18T11:58:37.802367+0000","last_fullsized":"2023-10-18T11:58:37.802367+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:58.998126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:35.515482+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:35.515185+0000","last_became_peered":"2023-10-18T11:58:35.515185+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.149129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509119+0000","last_change":"2023-10-18T11:58:36.534430+0000","last_active":"2023-10-18T11:58:37.509119+0000","last_peered":"2023-10-18T11:58:37.509119+0000","last_clean":"2023-10-18T11:58:37.509119+0000","last_became_active":"2023-10-18T11:58:36.534167+0000","last_became_peered":"2023-10-18T11:58:36.534167+0000","last_unstale":"2023-10-18T11:58:37.509119+0000","last_undegraded":"2023-10-18T11:58:37.509119+0000","last_fullsized":"2023-10-18T11:58:37.509119+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:44.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802606+0000","last_change":"2023-10-18T11:58:35.522154+0000","last_active":"2023-10-18T11:58:37.802606+0000","last_peered":"2023-10-18T11:58:37.802606+0000","last_clean":"2023-10-18T11:58:37.802606+0000","last_became_active":"2023-10-18T11:58:35.521902+0000","last_became_peered":"2023-10-18T11:58:35.521902+0000","last_unstale":"2023-10-18T11:58:37.802606+0000","last_undegraded":"2023-10-18T11:58:37.802606+0000","last_fullsized":"2023-10-18T11:58:37.802606+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:27.447038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803092+0000","last_change":"2023-10-18T11:58:36.529258+0000","last_active":"2023-10-18T11:58:37.803092+0000","last_peered":"2023-10-18T11:58:37.803092+0000","last_clean":"2023-10-18T11:58:37.803092+0000","last_became_active":"2023-10-18T11:58:36.528468+0000","last_became_peered":"2023-10-18T11:58:36.528468+0000","last_unstale":"2023-10-18T11:58:37.803092+0000","last_undegraded":"2023-10-18T11:58:37.803092+0000","last_fullsized":"2023-10-18T11:58:37.803092+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:58.035451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803037+0000","last_change":"2023-10-18T11:58:36.521908+0000","last_active":"2023-10-18T11:58:37.803037+0000","last_peered":"2023-10-18T11:58:37.803037+0000","last_clean":"2023-10-18T11:58:37.803037+0000","last_became_active":"2023-10-18T11:58:36.521381+0000","last_became_peered":"2023-10-18T11:58:36.521381+0000","last_unstale":"2023-10-18T11:58:37.803037+0000","last_undegraded":"2023-10-18T11:58:37.803037+0000","last_fullsized":"2023-10-18T11:58:37.803037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:18.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803110+0000","last_change":"2023-10-18T11:58:35.515862+0000","last_active":"2023-10-18T11:58:37.803110+0000","last_peered":"2023-10-18T11:58:37.803110+0000","last_clean":"2023-10-18T11:58:37.803110+0000","last_became_active":"2023-10-18T11:58:35.515521+0000","last_became_peered":"2023-10-18T11:58:35.515521+0000","last_unstale":"2023-10-18T11:58:37.803110+0000","last_undegraded":"2023-10-18T11:58:37.803110+0000","last_fullsized":"2023-10-18T11:58:37.803110+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:03.199289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801885+0000","last_change":"2023-10-18T11:58:36.520962+0000","last_active":"2023-10-18T11:58:37.801885+0000","last_peered":"2023-10-18T11:58:37.801885+0000","last_clean":"2023-10-18T11:58:37.801885+0000","last_became_active":"2023-10-18T11:58:36.520712+0000","last_became_peered":"2023-10-18T11:58:36.520712+0000","last_unstale":"2023-10-18T11:58:37.801885+0000","last_undegraded":"2023-10-18T11:58:37.801885+0000","last_fullsized":"2023-10-18T11:58:37.801885+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:46.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512865+0000","last_change":"2023-10-18T11:58:35.524483+0000","last_active":"2023-10-18T11:58:37.512865+0000","last_peered":"2023-10-18T11:58:37.512865+0000","last_clean":"2023-10-18T11:58:37.512865+0000","last_became_active":"2023-10-18T11:58:35.524231+0000","last_became_peered":"2023-10-18T11:58:35.524231+0000","last_unstale":"2023-10-18T11:58:37.512865+0000","last_undegraded":"2023-10-18T11:58:37.512865+0000","last_fullsized":"2023-10-18T11:58:37.512865+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:29:21.906133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.529745+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.529497+0000","last_became_peered":"2023-10-18T11:58:36.529497+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802708+0000","last_change":"2023-10-18T11:58:35.526709+0000","last_active":"2023-10-18T11:58:37.802708+0000","last_peered":"2023-10-18T11:58:37.802708+0000","last_clean":"2023-10-18T11:58:37.802708+0000","last_became_active":"2023-10-18T11:58:35.526448+0000","last_became_peered":"2023-10-18T11:58:35.526448+0000","last_unstale":"2023-10-18T11:58:37.802708+0000","last_undegraded":"2023-10-18T11:58:37.802708+0000","last_fullsized":"2023-10-18T11:58:37.802708+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:20.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802541+0000","last_change":"2023-10-18T11:58:36.528368+0000","last_active":"2023-10-18T11:58:37.802541+0000","last_peered":"2023-10-18T11:58:37.802541+0000","last_clean":"2023-10-18T11:58:37.802541+0000","last_became_active":"2023-10-18T11:58:36.528124+0000","last_became_peered":"2023-10-18T11:58:36.528124+0000","last_unstale":"2023-10-18T11:58:37.802541+0000","last_undegraded":"2023-10-18T11:58:37.802541+0000","last_fullsized":"2023-10-18T11:58:37.802541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:48.474419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802873+0000","last_change":"2023-10-18T11:58:35.525781+0000","last_active":"2023-10-18T11:58:37.802873+0000","last_peered":"2023-10-18T11:58:37.802873+0000","last_clean":"2023-10-18T11:58:37.802873+0000","last_became_active":"2023-10-18T11:58:35.525519+0000","last_became_peered":"2023-10-18T11:58:35.525519+0000","last_unstale":"2023-10-18T11:58:37.802873+0000","last_undegraded":"2023-10-18T11:58:37.802873+0000","last_fullsized":"2023-10-18T11:58:37.802873+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:47:37.957986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508969+0000","last_change":"2023-10-18T11:58:36.518933+0000","last_active":"2023-10-18T11:58:37.508969+0000","last_peered":"2023-10-18T11:58:37.508969+0000","last_clean":"2023-10-18T11:58:37.508969+0000","last_became_active":"2023-10-18T11:58:36.518691+0000","last_became_peered":"2023-10-18T11:58:36.518691+0000","last_unstale":"2023-10-18T11:58:37.508969+0000","last_undegraded":"2023-10-18T11:58:37.508969+0000","last_fullsized":"2023-10-18T11:58:37.508969+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:55.005558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802476+0000","last_change":"2023-10-18T11:58:36.522037+0000","last_active":"2023-10-18T11:58:37.802476+0000","last_peered":"2023-10-18T11:58:37.802476+0000","last_clean":"2023-10-18T11:58:37.802476+0000","last_became_active":"2023-10-18T11:58:36.521783+0000","last_became_peered":"2023-10-18T11:58:36.521783+0000","last_unstale":"2023-10-18T11:58:37.802476+0000","last_undegraded":"2023-10-18T11:58:37.802476+0000","last_fullsized":"2023-10-18T11:58:37.802476+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:37:11.613406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803083+0000","last_change":"2023-10-18T11:58:35.518886+0000","last_active":"2023-10-18T11:58:37.803083+0000","last_peered":"2023-10-18T11:58:37.803083+0000","last_clean":"2023-10-18T11:58:37.803083+0000","last_became_active":"2023-10-18T11:58:35.518591+0000","last_became_peered":"2023-10-18T11:58:35.518591+0000","last_unstale":"2023-10-18T11:58:37.803083+0000","last_undegraded":"2023-10-18T11:58:37.803083+0000","last_fullsized":"2023-10-18T11:58:37.803083+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:49.481190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802223+0000","last_change":"2023-10-18T11:58:36.519762+0000","last_active":"2023-10-18T11:58:37.802223+0000","last_peered":"2023-10-18T11:58:37.802223+0000","last_clean":"2023-10-18T11:58:37.802223+0000","last_became_active":"2023-10-18T11:58:36.517560+0000","last_became_peered":"2023-10-18T11:58:36.517560+0000","last_unstale":"2023-10-18T11:58:37.802223+0000","last_undegraded":"2023-10-18T11:58:37.802223+0000","last_fullsized":"2023-10-18T11:58:37.802223+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:29.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802645+0000","last_change":"2023-10-18T11:58:35.522798+0000","last_active":"2023-10-18T11:58:37.802645+0000","last_peered":"2023-10-18T11:58:37.802645+0000","last_clean":"2023-10-18T11:58:37.802645+0000","last_became_active":"2023-10-18T11:58:35.522532+0000","last_became_peered":"2023-10-18T11:58:35.522532+0000","last_unstale":"2023-10-18T11:58:37.802645+0000","last_undegraded":"2023-10-18T11:58:37.802645+0000","last_fullsized":"2023-10-18T11:58:37.802645+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:40:51.705544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:36.530354+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:36.530080+0000","last_became_peered":"2023-10-18T11:58:36.530080+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:19.995868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509241+0000","last_change":"2023-10-18T11:58:35.523100+0000","last_active":"2023-10-18T11:58:37.509241+0000","last_peered":"2023-10-18T11:58:37.509241+0000","last_clean":"2023-10-18T11:58:37.509241+0000","last_became_active":"2023-10-18T11:58:35.522802+0000","last_became_peered":"2023-10-18T11:58:35.522802+0000","last_unstale":"2023-10-18T11:58:37.509241+0000","last_undegraded":"2023-10-18T11:58:37.509241+0000","last_fullsized":"2023-10-18T11:58:37.509241+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802916+0000","last_change":"2023-10-18T11:58:36.522886+0000","last_active":"2023-10-18T11:58:37.802916+0000","last_peered":"2023-10-18T11:58:37.802916+0000","last_clean":"2023-10-18T11:58:37.802916+0000","last_became_active":"2023-10-18T11:58:36.522614+0000","last_became_peered":"2023-10-18T11:58:36.522614+0000","last_unstale":"2023-10-18T11:58:37.802916+0000","last_undegraded":"2023-10-18T11:58:37.802916+0000","last_fullsized":"2023-10-18T11:58:37.802916+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:52.212999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509088+0000","last_change":"2023-10-18T11:58:35.523409+0000","last_active":"2023-10-18T11:58:37.509088+0000","last_peered":"2023-10-18T11:58:37.509088+0000","last_clean":"2023-10-18T11:58:37.509088+0000","last_became_active":"2023-10-18T11:58:35.523146+0000","last_became_peered":"2023-10-18T11:58:35.523146+0000","last_unstale":"2023-10-18T11:58:37.509088+0000","last_undegraded":"2023-10-18T11:58:37.509088+0000","last_fullsized":"2023-10-18T11:58:37.509088+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:51.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802959+0000","last_change":"2023-10-18T11:58:35.522529+0000","last_active":"2023-10-18T11:58:37.802959+0000","last_peered":"2023-10-18T11:58:37.802959+0000","last_clean":"2023-10-18T11:58:37.802959+0000","last_became_active":"2023-10-18T11:58:35.522261+0000","last_became_peered":"2023-10-18T11:58:35.522261+0000","last_unstale":"2023-10-18T11:58:37.802959+0000","last_undegraded":"2023-10-18T11:58:37.802959+0000","last_fullsized":"2023-10-18T11:58:37.802959+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512534+0000","last_change":"2023-10-18T11:58:36.530248+0000","last_active":"2023-10-18T11:58:37.512534+0000","last_peered":"2023-10-18T11:58:37.512534+0000","last_clean":"2023-10-18T11:58:37.512534+0000","last_became_active":"2023-10-18T11:58:36.529997+0000","last_became_peered":"2023-10-18T11:58:36.529997+0000","last_unstale":"2023-10-18T11:58:37.512534+0000","last_undegraded":"2023-10-18T11:58:37.512534+0000","last_fullsized":"2023-10-18T11:58:37.512534+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:58.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802154+0000","last_change":"2023-10-18T11:58:35.527290+0000","last_active":"2023-10-18T11:58:37.802154+0000","last_peered":"2023-10-18T11:58:37.802154+0000","last_clean":"2023-10-18T11:58:37.802154+0000","last_became_active":"2023-10-18T11:58:35.527052+0000","last_became_peered":"2023-10-18T11:58:35.527052+0000","last_unstale":"2023-10-18T11:58:37.802154+0000","last_undegraded":"2023-10-18T11:58:37.802154+0000","last_fullsized":"2023-10-18T11:58:37.802154+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802594+0000","last_change":"2023-10-18T11:58:36.520551+0000","last_active":"2023-10-18T11:58:37.802594+0000","last_peered":"2023-10-18T11:58:37.802594+0000","last_clean":"2023-10-18T11:58:37.802594+0000","last_became_active":"2023-10-18T11:58:36.520297+0000","last_became_peered":"2023-10-18T11:58:36.520297+0000","last_unstale":"2023-10-18T11:58:37.802594+0000","last_undegraded":"2023-10-18T11:58:37.802594+0000","last_fullsized":"2023-10-18T11:58:37.802594+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:25.536175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802945+0000","last_change":"2023-10-18T11:58:35.526011+0000","last_active":"2023-10-18T11:58:37.802945+0000","last_peered":"2023-10-18T11:58:37.802945+0000","last_clean":"2023-10-18T11:58:37.802945+0000","last_became_active":"2023-10-18T11:58:35.525734+0000","last_became_peered":"2023-10-18T11:58:35.525734+0000","last_unstale":"2023-10-18T11:58:37.802945+0000","last_undegraded":"2023-10-18T11:58:37.802945+0000","last_fullsized":"2023-10-18T11:58:37.802945+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:19.352633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802222+0000","last_change":"2023-10-18T11:58:36.528280+0000","last_active":"2023-10-18T11:58:37.802222+0000","last_peered":"2023-10-18T11:58:37.802222+0000","last_clean":"2023-10-18T11:58:37.802222+0000","last_became_active":"2023-10-18T11:58:36.528012+0000","last_became_peered":"2023-10-18T11:58:36.528012+0000","last_unstale":"2023-10-18T11:58:37.802222+0000","last_undegraded":"2023-10-18T11:58:37.802222+0000","last_fullsized":"2023-10-18T11:58:37.802222+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:28.449393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801952+0000","last_change":"2023-10-18T11:58:35.513693+0000","last_active":"2023-10-18T11:58:37.801952+0000","last_peered":"2023-10-18T11:58:37.801952+0000","last_clean":"2023-10-18T11:58:37.801952+0000","last_became_active":"2023-10-18T11:58:35.513429+0000","last_became_peered":"2023-10-18T11:58:35.513429+0000","last_unstale":"2023-10-18T11:58:37.801952+0000","last_undegraded":"2023-10-18T11:58:37.801952+0000","last_fullsized":"2023-10-18T11:58:37.801952+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:12.442948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"193'335","reported_seq":1140,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-10-18T11:58:36.518008+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:58:36.518008+0000","last_peered":"2023-10-18T11:58:36.518008+0000","last_clean":"2023-10-18T11:58:36.518008+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:58:36.518008+0000","last_undegraded":"2023-10-18T11:58:36.518008+0000","last_fullsized":"2023-10-18T11:58:36.518008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":120,"acting":120,"num_store_stats":7},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52308,"kb_used_data":3080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319532,"statfs":{"total":96636764160,"available":96583200768,"internally_reserved":0,"allocated":3153920,"data_stored":1895422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.363},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74399999999999999,"5min":0.77200000000000002,"15min":0.84099999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.39700000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":1.298},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.82699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.35699999999999998},"max":{"1min":0.80200000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45200000000000001,"15min":0.44900000000000001},"min":{"1min":0.34000000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.80200000000000005,"15min":1.5249999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.27700000000000002},"max":{"1min":0.63500000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.78900000000000003,"15min":0.90800000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.375,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":0.82699999999999996,"15min":1.4179999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":1.478},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.874},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37768,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334072,"statfs":{"total":96636764160,"available":96598089728,"internally_reserved":0,"allocated":2486272,"data_stored":1267517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.50700000000000001,"5min":0.41899999999999998,"15min":0.29899999999999999},"max":{"1min":0.78100000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.27900000000000003},"max":{"1min":0.76700000000000002,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.87},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.625,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.25600000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.82599999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.626},"min":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47299999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.76900000000000002},"last":0.37},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.71999999999999997,"15min":0.82999999999999996},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.81200000000000006},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72699999999999998,"5min":0.80600000000000005,"15min":0.84599999999999997},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575970,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60068,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311772,"statfs":{"total":96636764160,"available":96575254528,"internally_reserved":0,"allocated":2576384,"data_stored":1306688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.249},"max":{"1min":0.84099999999999997,"5min":0.90300000000000002,"15min":0.90800000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.50700000000000001,"5min":0.38500000000000001,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.95899999999999996,"15min":0.96099999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.24299999999999999},"max":{"1min":1.081,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82199999999999995,"15min":0.754},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.28399999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.98699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.84999999999999998,"15min":0.98399999999999999},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.53200000000000003,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.0349999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.501,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":1,"15min":1},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.30299999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.98499999999999999,"15min":1.01},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.93899999999999995,"15min":0.97999999999999998},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83699999999999997,"5min":1.101,"15min":1.101},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.34000000000000002},"max":{"1min":0.83399999999999996,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575970,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29544,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342296,"statfs":{"total":96636764160,"available":96606511104,"internally_reserved":0,"allocated":2576384,"data_stored":1306700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.62},"min":{"1min":0.45400000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80700000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.80200000000000005,"15min":0.85399999999999998},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69499999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.372,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.752,"5min":1.123,"15min":1.123},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.92100000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.93600000000000005},"last":0.629}]},{"osd":3,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.312},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.34599999999999997,"15min":0.31900000000000001},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":2568192,"data_stored":1305214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.221},"max":{"1min":0.57099999999999995,"5min":0.61599999999999999,"15min":0.63300000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.28100000000000003,"15min":0.23400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.374}]},{"osd":1,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26400000000000001},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.56000000000000005,"5min":0.60599999999999998,"15min":0.65000000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.73399999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.052},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47499999999999998,"15min":0.40999999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.026},"last":0.625}]},{"osd":6,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.437},"max":{"1min":0.82299999999999995,"5min":0.93100000000000005,"15min":0.94499999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.68700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.45400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.55600000000000005,"5min":0.46400000000000002,"15min":0.442},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.58099999999999996,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.67800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60060,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311780,"statfs":{"total":96636764160,"available":96575262720,"internally_reserved":0,"allocated":2564096,"data_stored":1305636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.252,"5min":0.222,"15min":0.216},"max":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.67200000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.27900000000000003,"5min":0.26200000000000001,"15min":0.25800000000000001},"max":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.746,"15min":0.746},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.435,"15min":0.432},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.374},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35399999999999998,"5min":0.29899999999999999,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.33600000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.499,"5min":0.442,"15min":0.437},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.47799999999999998,"15min":0.41699999999999998},"max":{"1min":0.80900000000000005,"5min":1.087,"15min":1.087},"last":0.435}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.54500000000000004,"5min":0.46500000000000002,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66600000000000004,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.45200000000000001,"15min":0.40999999999999998},"max":{"1min":0.85799999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68500000000000005,"15min":0.68200000000000005},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.45100000000000001},"max":{"1min":0.84099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.443,"15min":0.42499999999999999},"max":{"1min":0.86799999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.53900000000000003,"5min":0.443,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.01},"last":0.499}]}]},{"osd":1,"up_from":13,"seq":55834575970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60048,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311792,"statfs":{"total":96636764160,"available":96575275008,"internally_reserved":0,"allocated":2560000,"data_stored":1305054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.182},"max":{"1min":0.73899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65800000000000003,"5min":0.81000000000000005,"15min":0.85299999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.23000000000000001},"max":{"1min":0.496,"5min":0.877,"15min":0.877},"last":0.28000000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.69799999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.377},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51400000000000001},"min":{"1min":0.28899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.74399999999999999,"5min":0.79800000000000004,"15min":0.91000000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.74099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.65600000000000003},"min":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.29599999999999999},"max":{"1min":0.80300000000000005,"5min":0.86599999999999999,"15min":1.2390000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.316},"max":{"1min":0.70299999999999996,"5min":1.05,"15min":1.212},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":1.0209999999999999,"15min":1.0589999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77600000000000002,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.47399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82499999999999996,"5min":0.88400000000000001,"15min":1.0620000000000001},"last":0.69899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42600,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329240,"statfs":{"total":96636764160,"available":96593141760,"internally_reserved":0,"allocated":3170304,"data_stored":1896150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.57199999999999995,"5min":0.64400000000000002,"15min":0.66200000000000003},"last":0.247}]},{"osd":2,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.33700000000000002,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.433,"15min":0.42799999999999999},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.25700000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.36699999999999999},"max":{"1min":0.876,"5min":0.96999999999999997,"15min":0.997},"last":0.495},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.50900000000000001,"5min":0.45400000000000001,"15min":0.39100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":0.93999999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.504,"5min":0.439,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.95799999999999996,"15min":0.996},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.44700000000000001},"max":{"1min":0.89300000000000002,"5min":1.0089999999999999,"15min":1.038},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.48599999999999999,"5min":0.44900000000000001,"15min":0.44800000000000001},"max":{"1min":0.91100000000000003,"5min":0.95099999999999996,"15min":1.0609999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":1.0109999999999999,"15min":1.0189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.42999999999999999},"max":{"1min":0.89400000000000002,"5min":0.997,"15min":0.997},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:58:39.461 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:58:39.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.595+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/2707010711 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fa00c96b0 msgr2=0x7f6fa00c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.595+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/2707010711 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fa00c96b0 0x7f6fa00c9b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009830 tx=0x7f6f9002bd60 comp rx=0 tx=0).stop 2023-10-18T11:58:39.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.595+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/2707010711 shutdown_connections 2023-10-18T11:58:39.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.595+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/2707010711 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fa00c96b0 0x7f6fa00c9b30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.595+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/2707010711 >> v2:172.21.15.139:3300/0 conn(0x7f6fa006af00 0x7f6fa006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.595+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/2707010711 >> 172.21.15.125:0/2707010711 conn(0x7f6fa0069930 msgr2=0x7f6fa0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.595+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/2707010711 shutdown_connections 2023-10-18T11:58:39.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.595+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/2707010711 wait complete. 2023-10-18T11:58:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.596+0000 7f6fa7a7c640 1 Processor -- start 2023-10-18T11:58:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.596+0000 7f6fa7a7c640 1 -- start start 2023-10-18T11:58:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa7a7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fa006af00 0x7f6fa00a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa7a7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fa00ab500 0x7f6fa00aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa7a7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fa00c96b0 0x7f6fa0138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa7a7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00a97b0 con 0x7f6fa006af00 2023-10-18T11:58:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa7a7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00a94d0 con 0x7f6fa00c96b0 2023-10-18T11:58:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa7a7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00a6630 con 0x7f6fa00ab500 2023-10-18T11:58:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fa006af00 0x7f6fa00a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fa006af00 0x7f6fa00a8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50000/0 (socket says 172.21.15.125:50000) 2023-10-18T11:58:39.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 -- 172.21.15.125:0/3965037301 learned_addr learned my addr 172.21.15.125:0/3965037301 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:39.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5017640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fa00ab500 0x7f6fa00aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 -- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fa00ab500 msgr2=0x7f6fa00aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fa00ab500 0x7f6fa00aad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa6019640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fa00c96b0 0x7f6fa0138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 -- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fa00c96b0 msgr2=0x7f6fa0138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fa00c96b0 0x7f6fa0138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 -- 172.21.15.125:0/3965037301 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c009490 con 0x7f6fa006af00 2023-10-18T11:58:39.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa6019640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fa00c96b0 0x7f6fa0138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:39.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6fa5818640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fa006af00 0x7f6fa00a8ec0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00ea60 tx=0x7f6f9c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:39.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.597+0000 7f6f967fc640 1 -- 172.21.15.125:0/3965037301 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00cde0 con 0x7f6fa006af00 2023-10-18T11:58:39.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.598+0000 7f6f967fc640 1 -- 172.21.15.125:0/3965037301 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6f9c004020 con 0x7f6fa006af00 2023-10-18T11:58:39.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.598+0000 7f6f967fc640 1 -- 172.21.15.125:0/3965037301 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c004410 con 0x7f6fa006af00 2023-10-18T11:58:39.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.598+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f90009490 con 0x7f6fa006af00 2023-10-18T11:58:39.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.598+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa00a6bf0 con 0x7f6fa006af00 2023-10-18T11:58:39.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.599+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa00690e0 con 0x7f6fa006af00 2023-10-18T11:58:39.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.599+0000 7f6f967fc640 1 -- 172.21.15.125:0/3965037301 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6f9c0035f0 con 0x7f6fa006af00 2023-10-18T11:58:39.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.599+0000 7f6f967fc640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f6f8406af60 0x7f6f8406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.599+0000 7f6f967fc640 1 -- 172.21.15.125:0/3965037301 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f6f9c014070 con 0x7f6fa006af00 2023-10-18T11:58:39.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.600+0000 7f6fa5017640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f6f8406af60 0x7f6f8406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.600+0000 7f6fa5017640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f6f8406af60 0x7f6f8406d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009830 tx=0x7f6f90004730 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:39.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.604+0000 7f6f967fc640 1 -- 172.21.15.125:0/3965037301 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f9c055370 con 0x7f6fa006af00 2023-10-18T11:58:39.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.768+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 --> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6fa0159c50 con 0x7f6f8406af60 2023-10-18T11:58:39.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.776+0000 7f6f967fc640 1 -- 172.21.15.125:0/3965037301 <== mgr.9529 v2:172.21.15.139:6832/1565260611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288588 (secure 0 0 0) 0x7f6f9002c700 con 0x7f6f8406af60 2023-10-18T11:58:39.781 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:39.783 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:58:39.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f6f8406af60 msgr2=0x7f6f8406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f6f8406af60 0x7f6f8406d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009830 tx=0x7f6f90004730 comp rx=0 tx=0).stop 2023-10-18T11:58:39.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fa006af00 msgr2=0x7f6fa00a8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fa006af00 0x7f6fa00a8ec0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00ea60 tx=0x7f6f9c00ef20 comp rx=0 tx=0).stop 2023-10-18T11:58:39.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 shutdown_connections 2023-10-18T11:58:39.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f6f8406af60 0x7f6f8406d420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fa00c96b0 0x7f6fa0138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6fa00ab500 0x7f6fa00aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 --2- 172.21.15.125:0/3965037301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6fa006af00 0x7f6fa00a8ec0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 >> 172.21.15.125:0/3965037301 conn(0x7f6fa0069930 msgr2=0x7f6fa013b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:39.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 shutdown_connections 2023-10-18T11:58:39.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.779+0000 7f6fa7a7c640 1 -- 172.21.15.125:0/3965037301 wait complete. 2023-10-18T11:58:39.792 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T11:58:38.893982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":598,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":372,"ondisk_log_size":372,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":402452,"kb_used_data":21148,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754572268,"statfs":{"total":773094113280,"available":772682002432,"internally_reserved":0,"allocated":21655552,"data_stored":11588381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":390319672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.394160"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802702+0000","last_change":"2023-10-18T11:58:36.522565+0000","last_active":"2023-10-18T11:58:37.802702+0000","last_peered":"2023-10-18T11:58:37.802702+0000","last_clean":"2023-10-18T11:58:37.802702+0000","last_became_active":"2023-10-18T11:58:36.521736+0000","last_became_peered":"2023-10-18T11:58:36.521736+0000","last_unstale":"2023-10-18T11:58:37.802702+0000","last_undegraded":"2023-10-18T11:58:37.802702+0000","last_fullsized":"2023-10-18T11:58:37.802702+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:46:36.674788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802006+0000","last_change":"2023-10-18T11:58:36.527651+0000","last_active":"2023-10-18T11:58:37.802006+0000","last_peered":"2023-10-18T11:58:37.802006+0000","last_clean":"2023-10-18T11:58:37.802006+0000","last_became_active":"2023-10-18T11:58:36.527375+0000","last_became_peered":"2023-10-18T11:58:36.527375+0000","last_unstale":"2023-10-18T11:58:37.802006+0000","last_undegraded":"2023-10-18T11:58:37.802006+0000","last_fullsized":"2023-10-18T11:58:37.802006+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:55:29.683103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.534333+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.534066+0000","last_became_peered":"2023-10-18T11:58:36.534066+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:29.879395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508872+0000","last_change":"2023-10-18T11:58:36.529615+0000","last_active":"2023-10-18T11:58:37.508872+0000","last_peered":"2023-10-18T11:58:37.508872+0000","last_clean":"2023-10-18T11:58:37.508872+0000","last_became_active":"2023-10-18T11:58:36.529380+0000","last_became_peered":"2023-10-18T11:58:36.529380+0000","last_unstale":"2023-10-18T11:58:37.508872+0000","last_undegraded":"2023-10-18T11:58:37.508872+0000","last_fullsized":"2023-10-18T11:58:37.508872+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:44.701593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508825+0000","last_change":"2023-10-18T11:58:36.528591+0000","last_active":"2023-10-18T11:58:37.508825+0000","last_peered":"2023-10-18T11:58:37.508825+0000","last_clean":"2023-10-18T11:58:37.508825+0000","last_became_active":"2023-10-18T11:58:36.528332+0000","last_became_peered":"2023-10-18T11:58:36.528332+0000","last_unstale":"2023-10-18T11:58:37.508825+0000","last_undegraded":"2023-10-18T11:58:37.508825+0000","last_fullsized":"2023-10-18T11:58:37.508825+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:04.968822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802608+0000","last_change":"2023-10-18T11:58:36.526452+0000","last_active":"2023-10-18T11:58:37.802608+0000","last_peered":"2023-10-18T11:58:37.802608+0000","last_clean":"2023-10-18T11:58:37.802608+0000","last_became_active":"2023-10-18T11:58:36.526144+0000","last_became_peered":"2023-10-18T11:58:36.526144+0000","last_unstale":"2023-10-18T11:58:37.802608+0000","last_undegraded":"2023-10-18T11:58:37.802608+0000","last_fullsized":"2023-10-18T11:58:37.802608+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:15.609770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802428+0000","last_change":"2023-10-18T11:58:36.526263+0000","last_active":"2023-10-18T11:58:37.802428+0000","last_peered":"2023-10-18T11:58:37.802428+0000","last_clean":"2023-10-18T11:58:37.802428+0000","last_became_active":"2023-10-18T11:58:36.525882+0000","last_became_peered":"2023-10-18T11:58:36.525882+0000","last_unstale":"2023-10-18T11:58:37.802428+0000","last_undegraded":"2023-10-18T11:58:37.802428+0000","last_fullsized":"2023-10-18T11:58:37.802428+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:05.910088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802812+0000","last_change":"2023-10-18T11:58:36.520303+0000","last_active":"2023-10-18T11:58:37.802812+0000","last_peered":"2023-10-18T11:58:37.802812+0000","last_clean":"2023-10-18T11:58:37.802812+0000","last_became_active":"2023-10-18T11:58:36.520065+0000","last_became_peered":"2023-10-18T11:58:36.520065+0000","last_unstale":"2023-10-18T11:58:37.802812+0000","last_undegraded":"2023-10-18T11:58:37.802812+0000","last_fullsized":"2023-10-18T11:58:37.802812+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:40.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509204+0000","last_change":"2023-10-18T11:58:36.534318+0000","last_active":"2023-10-18T11:58:37.509204+0000","last_peered":"2023-10-18T11:58:37.509204+0000","last_clean":"2023-10-18T11:58:37.509204+0000","last_became_active":"2023-10-18T11:58:36.534071+0000","last_became_peered":"2023-10-18T11:58:36.534071+0000","last_unstale":"2023-10-18T11:58:37.509204+0000","last_undegraded":"2023-10-18T11:58:37.509204+0000","last_fullsized":"2023-10-18T11:58:37.509204+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:55.491621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802310+0000","last_change":"2023-10-18T11:58:36.520313+0000","last_active":"2023-10-18T11:58:37.802310+0000","last_peered":"2023-10-18T11:58:37.802310+0000","last_clean":"2023-10-18T11:58:37.802310+0000","last_became_active":"2023-10-18T11:58:36.520042+0000","last_became_peered":"2023-10-18T11:58:36.520042+0000","last_unstale":"2023-10-18T11:58:37.802310+0000","last_undegraded":"2023-10-18T11:58:37.802310+0000","last_fullsized":"2023-10-18T11:58:37.802310+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:57.623658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802462+0000","last_change":"2023-10-18T11:58:36.526211+0000","last_active":"2023-10-18T11:58:37.802462+0000","last_peered":"2023-10-18T11:58:37.802462+0000","last_clean":"2023-10-18T11:58:37.802462+0000","last_became_active":"2023-10-18T11:58:36.525812+0000","last_became_peered":"2023-10-18T11:58:36.525812+0000","last_unstale":"2023-10-18T11:58:37.802462+0000","last_undegraded":"2023-10-18T11:58:37.802462+0000","last_fullsized":"2023-10-18T11:58:37.802462+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:39.938700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512711+0000","last_change":"2023-10-18T11:58:36.529732+0000","last_active":"2023-10-18T11:58:37.512711+0000","last_peered":"2023-10-18T11:58:37.512711+0000","last_clean":"2023-10-18T11:58:37.512711+0000","last_became_active":"2023-10-18T11:58:36.529455+0000","last_became_peered":"2023-10-18T11:58:36.529455+0000","last_unstale":"2023-10-18T11:58:37.512711+0000","last_undegraded":"2023-10-18T11:58:37.512711+0000","last_fullsized":"2023-10-18T11:58:37.512711+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:38.419107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802037+0000","last_change":"2023-10-18T11:58:36.534249+0000","last_active":"2023-10-18T11:58:37.802037+0000","last_peered":"2023-10-18T11:58:37.802037+0000","last_clean":"2023-10-18T11:58:37.802037+0000","last_became_active":"2023-10-18T11:58:36.533956+0000","last_became_peered":"2023-10-18T11:58:36.533956+0000","last_unstale":"2023-10-18T11:58:37.802037+0000","last_undegraded":"2023-10-18T11:58:37.802037+0000","last_fullsized":"2023-10-18T11:58:37.802037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:29.001140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802132+0000","last_change":"2023-10-18T11:58:36.530863+0000","last_active":"2023-10-18T11:58:37.802132+0000","last_peered":"2023-10-18T11:58:37.802132+0000","last_clean":"2023-10-18T11:58:37.802132+0000","last_became_active":"2023-10-18T11:58:36.530625+0000","last_became_peered":"2023-10-18T11:58:36.530625+0000","last_unstale":"2023-10-18T11:58:37.802132+0000","last_undegraded":"2023-10-18T11:58:37.802132+0000","last_fullsized":"2023-10-18T11:58:37.802132+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:03.443134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508994+0000","last_change":"2023-10-18T11:58:36.531552+0000","last_active":"2023-10-18T11:58:37.508994+0000","last_peered":"2023-10-18T11:58:37.508994+0000","last_clean":"2023-10-18T11:58:37.508994+0000","last_became_active":"2023-10-18T11:58:36.531163+0000","last_became_peered":"2023-10-18T11:58:36.531163+0000","last_unstale":"2023-10-18T11:58:37.508994+0000","last_undegraded":"2023-10-18T11:58:37.508994+0000","last_fullsized":"2023-10-18T11:58:37.508994+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:49.241771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509268+0000","last_change":"2023-10-18T11:58:36.530026+0000","last_active":"2023-10-18T11:58:37.509268+0000","last_peered":"2023-10-18T11:58:37.509268+0000","last_clean":"2023-10-18T11:58:37.509268+0000","last_became_active":"2023-10-18T11:58:36.529058+0000","last_became_peered":"2023-10-18T11:58:36.529058+0000","last_unstale":"2023-10-18T11:58:37.509268+0000","last_undegraded":"2023-10-18T11:58:37.509268+0000","last_fullsized":"2023-10-18T11:58:37.509268+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:41.289044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513034+0000","last_change":"2023-10-18T11:58:36.532114+0000","last_active":"2023-10-18T11:58:37.513034+0000","last_peered":"2023-10-18T11:58:37.513034+0000","last_clean":"2023-10-18T11:58:37.513034+0000","last_became_active":"2023-10-18T11:58:36.531526+0000","last_became_peered":"2023-10-18T11:58:36.531526+0000","last_unstale":"2023-10-18T11:58:37.513034+0000","last_undegraded":"2023-10-18T11:58:37.513034+0000","last_fullsized":"2023-10-18T11:58:37.513034+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:38.904126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512884+0000","last_change":"2023-10-18T11:58:36.534033+0000","last_active":"2023-10-18T11:58:37.512884+0000","last_peered":"2023-10-18T11:58:37.512884+0000","last_clean":"2023-10-18T11:58:37.512884+0000","last_became_active":"2023-10-18T11:58:36.533772+0000","last_became_peered":"2023-10-18T11:58:36.533772+0000","last_unstale":"2023-10-18T11:58:37.512884+0000","last_undegraded":"2023-10-18T11:58:37.512884+0000","last_fullsized":"2023-10-18T11:58:37.512884+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:04.963404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508988+0000","last_change":"2023-10-18T11:58:36.529660+0000","last_active":"2023-10-18T11:58:37.508988+0000","last_peered":"2023-10-18T11:58:37.508988+0000","last_clean":"2023-10-18T11:58:37.508988+0000","last_became_active":"2023-10-18T11:58:36.529125+0000","last_became_peered":"2023-10-18T11:58:36.529125+0000","last_unstale":"2023-10-18T11:58:37.508988+0000","last_undegraded":"2023-10-18T11:58:37.508988+0000","last_fullsized":"2023-10-18T11:58:37.508988+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:01.717669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802201+0000","last_change":"2023-10-18T11:58:36.519499+0000","last_active":"2023-10-18T11:58:37.802201+0000","last_peered":"2023-10-18T11:58:37.802201+0000","last_clean":"2023-10-18T11:58:37.802201+0000","last_became_active":"2023-10-18T11:58:36.519246+0000","last_became_peered":"2023-10-18T11:58:36.519246+0000","last_unstale":"2023-10-18T11:58:37.802201+0000","last_undegraded":"2023-10-18T11:58:37.802201+0000","last_fullsized":"2023-10-18T11:58:37.802201+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:48.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802474+0000","last_change":"2023-10-18T11:58:36.514398+0000","last_active":"2023-10-18T11:58:37.802474+0000","last_peered":"2023-10-18T11:58:37.802474+0000","last_clean":"2023-10-18T11:58:37.802474+0000","last_became_active":"2023-10-18T11:58:36.514141+0000","last_became_peered":"2023-10-18T11:58:36.514141+0000","last_unstale":"2023-10-18T11:58:37.802474+0000","last_undegraded":"2023-10-18T11:58:37.802474+0000","last_fullsized":"2023-10-18T11:58:37.802474+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:43.247625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802305+0000","last_change":"2023-10-18T11:58:36.527748+0000","last_active":"2023-10-18T11:58:37.802305+0000","last_peered":"2023-10-18T11:58:37.802305+0000","last_clean":"2023-10-18T11:58:37.802305+0000","last_became_active":"2023-10-18T11:58:36.527498+0000","last_became_peered":"2023-10-18T11:58:36.527498+0000","last_unstale":"2023-10-18T11:58:37.802305+0000","last_undegraded":"2023-10-18T11:58:37.802305+0000","last_fullsized":"2023-10-18T11:58:37.802305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:35.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801779+0000","last_change":"2023-10-18T11:58:36.521950+0000","last_active":"2023-10-18T11:58:37.801779+0000","last_peered":"2023-10-18T11:58:37.801779+0000","last_clean":"2023-10-18T11:58:37.801779+0000","last_became_active":"2023-10-18T11:58:36.521681+0000","last_became_peered":"2023-10-18T11:58:36.521681+0000","last_unstale":"2023-10-18T11:58:37.801779+0000","last_undegraded":"2023-10-18T11:58:37.801779+0000","last_fullsized":"2023-10-18T11:58:37.801779+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:43.982016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802739+0000","last_change":"2023-10-18T11:58:36.518867+0000","last_active":"2023-10-18T11:58:37.802739+0000","last_peered":"2023-10-18T11:58:37.802739+0000","last_clean":"2023-10-18T11:58:37.802739+0000","last_became_active":"2023-10-18T11:58:36.518625+0000","last_became_peered":"2023-10-18T11:58:36.518625+0000","last_unstale":"2023-10-18T11:58:37.802739+0000","last_undegraded":"2023-10-18T11:58:37.802739+0000","last_fullsized":"2023-10-18T11:58:37.802739+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:20.411248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802849+0000","last_change":"2023-10-18T11:58:36.528722+0000","last_active":"2023-10-18T11:58:37.802849+0000","last_peered":"2023-10-18T11:58:37.802849+0000","last_clean":"2023-10-18T11:58:37.802849+0000","last_became_active":"2023-10-18T11:58:36.528462+0000","last_became_peered":"2023-10-18T11:58:36.528462+0000","last_unstale":"2023-10-18T11:58:37.802849+0000","last_undegraded":"2023-10-18T11:58:37.802849+0000","last_fullsized":"2023-10-18T11:58:37.802849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:58:27.045894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803175+0000","last_change":"2023-10-18T11:58:36.527594+0000","last_active":"2023-10-18T11:58:37.803175+0000","last_peered":"2023-10-18T11:58:37.803175+0000","last_clean":"2023-10-18T11:58:37.803175+0000","last_became_active":"2023-10-18T11:58:36.526226+0000","last_became_peered":"2023-10-18T11:58:36.526226+0000","last_unstale":"2023-10-18T11:58:37.803175+0000","last_undegraded":"2023-10-18T11:58:37.803175+0000","last_fullsized":"2023-10-18T11:58:37.803175+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:28.446100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509068+0000","last_change":"2023-10-18T11:58:36.529803+0000","last_active":"2023-10-18T11:58:37.509068+0000","last_peered":"2023-10-18T11:58:37.509068+0000","last_clean":"2023-10-18T11:58:37.509068+0000","last_became_active":"2023-10-18T11:58:36.529446+0000","last_became_peered":"2023-10-18T11:58:36.529446+0000","last_unstale":"2023-10-18T11:58:37.509068+0000","last_undegraded":"2023-10-18T11:58:37.509068+0000","last_fullsized":"2023-10-18T11:58:37.509068+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:19.031434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802519+0000","last_change":"2023-10-18T11:58:36.526866+0000","last_active":"2023-10-18T11:58:37.802519+0000","last_peered":"2023-10-18T11:58:37.802519+0000","last_clean":"2023-10-18T11:58:37.802519+0000","last_became_active":"2023-10-18T11:58:36.526609+0000","last_became_peered":"2023-10-18T11:58:36.526609+0000","last_unstale":"2023-10-18T11:58:37.802519+0000","last_undegraded":"2023-10-18T11:58:37.802519+0000","last_fullsized":"2023-10-18T11:58:37.802519+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:04.860806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802258+0000","last_change":"2023-10-18T11:58:36.515793+0000","last_active":"2023-10-18T11:58:37.802258+0000","last_peered":"2023-10-18T11:58:37.802258+0000","last_clean":"2023-10-18T11:58:37.802258+0000","last_became_active":"2023-10-18T11:58:36.515523+0000","last_became_peered":"2023-10-18T11:58:36.515523+0000","last_unstale":"2023-10-18T11:58:37.802258+0000","last_undegraded":"2023-10-18T11:58:37.802258+0000","last_fullsized":"2023-10-18T11:58:37.802258+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802828+0000","last_change":"2023-10-18T11:58:36.515930+0000","last_active":"2023-10-18T11:58:37.802828+0000","last_peered":"2023-10-18T11:58:37.802828+0000","last_clean":"2023-10-18T11:58:37.802828+0000","last_became_active":"2023-10-18T11:58:36.515681+0000","last_became_peered":"2023-10-18T11:58:36.515681+0000","last_unstale":"2023-10-18T11:58:37.802828+0000","last_undegraded":"2023-10-18T11:58:37.802828+0000","last_fullsized":"2023-10-18T11:58:37.802828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:48:47.733918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508862+0000","last_change":"2023-10-18T11:58:36.530777+0000","last_active":"2023-10-18T11:58:37.508862+0000","last_peered":"2023-10-18T11:58:37.508862+0000","last_clean":"2023-10-18T11:58:37.508862+0000","last_became_active":"2023-10-18T11:58:36.530454+0000","last_became_peered":"2023-10-18T11:58:36.530454+0000","last_unstale":"2023-10-18T11:58:37.508862+0000","last_undegraded":"2023-10-18T11:58:37.508862+0000","last_fullsized":"2023-10-18T11:58:37.508862+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:00.542319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803195+0000","last_change":"2023-10-18T11:58:36.525494+0000","last_active":"2023-10-18T11:58:37.803195+0000","last_peered":"2023-10-18T11:58:37.803195+0000","last_clean":"2023-10-18T11:58:37.803195+0000","last_became_active":"2023-10-18T11:58:36.525200+0000","last_became_peered":"2023-10-18T11:58:36.525200+0000","last_unstale":"2023-10-18T11:58:37.803195+0000","last_undegraded":"2023-10-18T11:58:37.803195+0000","last_fullsized":"2023-10-18T11:58:37.803195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"203'4","reported_seq":16,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803001+0000","last_change":"2023-10-18T11:58:35.527076+0000","last_active":"2023-10-18T11:58:37.803001+0000","last_peered":"2023-10-18T11:58:37.803001+0000","last_clean":"2023-10-18T11:58:37.803001+0000","last_became_active":"2023-10-18T11:58:35.526809+0000","last_became_peered":"2023-10-18T11:58:35.526809+0000","last_unstale":"2023-10-18T11:58:37.803001+0000","last_undegraded":"2023-10-18T11:58:37.803001+0000","last_fullsized":"2023-10-18T11:58:37.803001+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802906+0000","last_change":"2023-10-18T11:58:36.521457+0000","last_active":"2023-10-18T11:58:37.802906+0000","last_peered":"2023-10-18T11:58:37.802906+0000","last_clean":"2023-10-18T11:58:37.802906+0000","last_became_active":"2023-10-18T11:58:36.521209+0000","last_became_peered":"2023-10-18T11:58:36.521209+0000","last_unstale":"2023-10-18T11:58:37.802906+0000","last_undegraded":"2023-10-18T11:58:37.802906+0000","last_fullsized":"2023-10-18T11:58:37.802906+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:40.559400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509267+0000","last_change":"2023-10-18T11:58:35.521828+0000","last_active":"2023-10-18T11:58:37.509267+0000","last_peered":"2023-10-18T11:58:37.509267+0000","last_clean":"2023-10-18T11:58:37.509267+0000","last_became_active":"2023-10-18T11:58:35.521574+0000","last_became_peered":"2023-10-18T11:58:35.521574+0000","last_unstale":"2023-10-18T11:58:37.509267+0000","last_undegraded":"2023-10-18T11:58:37.509267+0000","last_fullsized":"2023-10-18T11:58:37.509267+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:47.472064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803025+0000","last_change":"2023-10-18T11:58:36.522588+0000","last_active":"2023-10-18T11:58:37.803025+0000","last_peered":"2023-10-18T11:58:37.803025+0000","last_clean":"2023-10-18T11:58:37.803025+0000","last_became_active":"2023-10-18T11:58:36.522182+0000","last_became_peered":"2023-10-18T11:58:36.522182+0000","last_unstale":"2023-10-18T11:58:37.803025+0000","last_undegraded":"2023-10-18T11:58:37.803025+0000","last_fullsized":"2023-10-18T11:58:37.803025+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.326674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802389+0000","last_change":"2023-10-18T11:58:35.516307+0000","last_active":"2023-10-18T11:58:37.802389+0000","last_peered":"2023-10-18T11:58:37.802389+0000","last_clean":"2023-10-18T11:58:37.802389+0000","last_became_active":"2023-10-18T11:58:35.515995+0000","last_became_peered":"2023-10-18T11:58:35.515995+0000","last_unstale":"2023-10-18T11:58:37.802389+0000","last_undegraded":"2023-10-18T11:58:37.802389+0000","last_fullsized":"2023-10-18T11:58:37.802389+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:43.302735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512924+0000","last_change":"2023-10-18T11:58:35.526193+0000","last_active":"2023-10-18T11:58:37.512924+0000","last_peered":"2023-10-18T11:58:37.512924+0000","last_clean":"2023-10-18T11:58:37.512924+0000","last_became_active":"2023-10-18T11:58:35.525920+0000","last_became_peered":"2023-10-18T11:58:35.525920+0000","last_unstale":"2023-10-18T11:58:37.512924+0000","last_undegraded":"2023-10-18T11:58:37.512924+0000","last_fullsized":"2023-10-18T11:58:37.512924+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:17.858763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512416+0000","last_change":"2023-10-18T11:58:36.529406+0000","last_active":"2023-10-18T11:58:37.512416+0000","last_peered":"2023-10-18T11:58:37.512416+0000","last_clean":"2023-10-18T11:58:37.512416+0000","last_became_active":"2023-10-18T11:58:36.528795+0000","last_became_peered":"2023-10-18T11:58:36.528795+0000","last_unstale":"2023-10-18T11:58:37.512416+0000","last_undegraded":"2023-10-18T11:58:37.512416+0000","last_fullsized":"2023-10-18T11:58:37.512416+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:26.184608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509045+0000","last_change":"2023-10-18T11:58:35.523280+0000","last_active":"2023-10-18T11:58:37.509045+0000","last_peered":"2023-10-18T11:58:37.509045+0000","last_clean":"2023-10-18T11:58:37.509045+0000","last_became_active":"2023-10-18T11:58:35.522621+0000","last_became_peered":"2023-10-18T11:58:35.522621+0000","last_unstale":"2023-10-18T11:58:37.509045+0000","last_undegraded":"2023-10-18T11:58:37.509045+0000","last_fullsized":"2023-10-18T11:58:37.509045+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:27.443746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802321+0000","last_change":"2023-10-18T11:58:36.516159+0000","last_active":"2023-10-18T11:58:37.802321+0000","last_peered":"2023-10-18T11:58:37.802321+0000","last_clean":"2023-10-18T11:58:37.802321+0000","last_became_active":"2023-10-18T11:58:36.515894+0000","last_became_peered":"2023-10-18T11:58:36.515894+0000","last_unstale":"2023-10-18T11:58:37.802321+0000","last_undegraded":"2023-10-18T11:58:37.802321+0000","last_fullsized":"2023-10-18T11:58:37.802321+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:42.645151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802627+0000","last_change":"2023-10-18T11:58:35.521126+0000","last_active":"2023-10-18T11:58:37.802627+0000","last_peered":"2023-10-18T11:58:37.802627+0000","last_clean":"2023-10-18T11:58:37.802627+0000","last_became_active":"2023-10-18T11:58:35.520861+0000","last_became_peered":"2023-10-18T11:58:35.520861+0000","last_unstale":"2023-10-18T11:58:37.802627+0000","last_undegraded":"2023-10-18T11:58:37.802627+0000","last_fullsized":"2023-10-18T11:58:37.802627+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802501+0000","last_change":"2023-10-18T11:58:36.531084+0000","last_active":"2023-10-18T11:58:37.802501+0000","last_peered":"2023-10-18T11:58:37.802501+0000","last_clean":"2023-10-18T11:58:37.802501+0000","last_became_active":"2023-10-18T11:58:36.530792+0000","last_became_peered":"2023-10-18T11:58:36.530792+0000","last_unstale":"2023-10-18T11:58:37.802501+0000","last_undegraded":"2023-10-18T11:58:37.802501+0000","last_fullsized":"2023-10-18T11:58:37.802501+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:37.483860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512533+0000","last_change":"2023-10-18T11:58:35.518246+0000","last_active":"2023-10-18T11:58:37.512533+0000","last_peered":"2023-10-18T11:58:37.512533+0000","last_clean":"2023-10-18T11:58:37.512533+0000","last_became_active":"2023-10-18T11:58:35.517986+0000","last_became_peered":"2023-10-18T11:58:35.517986+0000","last_unstale":"2023-10-18T11:58:37.512533+0000","last_undegraded":"2023-10-18T11:58:37.512533+0000","last_fullsized":"2023-10-18T11:58:37.512533+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:02:15.620821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803081+0000","last_change":"2023-10-18T11:58:36.533670+0000","last_active":"2023-10-18T11:58:37.803081+0000","last_peered":"2023-10-18T11:58:37.803081+0000","last_clean":"2023-10-18T11:58:37.803081+0000","last_became_active":"2023-10-18T11:58:36.533394+0000","last_became_peered":"2023-10-18T11:58:36.533394+0000","last_unstale":"2023-10-18T11:58:37.803081+0000","last_undegraded":"2023-10-18T11:58:37.803081+0000","last_fullsized":"2023-10-18T11:58:37.803081+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:21.396797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509229+0000","last_change":"2023-10-18T11:58:35.524709+0000","last_active":"2023-10-18T11:58:37.509229+0000","last_peered":"2023-10-18T11:58:37.509229+0000","last_clean":"2023-10-18T11:58:37.509229+0000","last_became_active":"2023-10-18T11:58:35.524471+0000","last_became_peered":"2023-10-18T11:58:35.524471+0000","last_unstale":"2023-10-18T11:58:37.509229+0000","last_undegraded":"2023-10-18T11:58:37.509229+0000","last_fullsized":"2023-10-18T11:58:37.509229+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:53.288380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509250+0000","last_change":"2023-10-18T11:58:36.523093+0000","last_active":"2023-10-18T11:58:37.509250+0000","last_peered":"2023-10-18T11:58:37.509250+0000","last_clean":"2023-10-18T11:58:37.509250+0000","last_became_active":"2023-10-18T11:58:36.522551+0000","last_became_peered":"2023-10-18T11:58:36.522551+0000","last_unstale":"2023-10-18T11:58:37.509250+0000","last_undegraded":"2023-10-18T11:58:37.509250+0000","last_fullsized":"2023-10-18T11:58:37.509250+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:22.715669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802085+0000","last_change":"2023-10-18T11:58:35.522093+0000","last_active":"2023-10-18T11:58:37.802085+0000","last_peered":"2023-10-18T11:58:37.802085+0000","last_clean":"2023-10-18T11:58:37.802085+0000","last_became_active":"2023-10-18T11:58:35.521805+0000","last_became_peered":"2023-10-18T11:58:35.521805+0000","last_unstale":"2023-10-18T11:58:37.802085+0000","last_undegraded":"2023-10-18T11:58:37.802085+0000","last_fullsized":"2023-10-18T11:58:37.802085+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:08.791931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513049+0000","last_change":"2023-10-18T11:58:36.531254+0000","last_active":"2023-10-18T11:58:37.513049+0000","last_peered":"2023-10-18T11:58:37.513049+0000","last_clean":"2023-10-18T11:58:37.513049+0000","last_became_active":"2023-10-18T11:58:36.530967+0000","last_became_peered":"2023-10-18T11:58:36.530967+0000","last_unstale":"2023-10-18T11:58:37.513049+0000","last_undegraded":"2023-10-18T11:58:37.513049+0000","last_fullsized":"2023-10-18T11:58:37.513049+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:01.297812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"203'3","reported_seq":15,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802710+0000","last_change":"2023-10-18T11:58:35.525207+0000","last_active":"2023-10-18T11:58:37.802710+0000","last_peered":"2023-10-18T11:58:37.802710+0000","last_clean":"2023-10-18T11:58:37.802710+0000","last_became_active":"2023-10-18T11:58:35.524946+0000","last_became_peered":"2023-10-18T11:58:35.524946+0000","last_unstale":"2023-10-18T11:58:37.802710+0000","last_undegraded":"2023-10-18T11:58:37.802710+0000","last_fullsized":"2023-10-18T11:58:37.802710+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802920+0000","last_change":"2023-10-18T11:58:36.516878+0000","last_active":"2023-10-18T11:58:37.802920+0000","last_peered":"2023-10-18T11:58:37.802920+0000","last_clean":"2023-10-18T11:58:37.802920+0000","last_became_active":"2023-10-18T11:58:36.516625+0000","last_became_peered":"2023-10-18T11:58:36.516625+0000","last_unstale":"2023-10-18T11:58:37.802920+0000","last_undegraded":"2023-10-18T11:58:37.802920+0000","last_fullsized":"2023-10-18T11:58:37.802920+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:59.525275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"203'6","reported_seq":18,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513026+0000","last_change":"2023-10-18T11:58:35.518030+0000","last_active":"2023-10-18T11:58:37.513026+0000","last_peered":"2023-10-18T11:58:37.513026+0000","last_clean":"2023-10-18T11:58:37.513026+0000","last_became_active":"2023-10-18T11:58:35.517747+0000","last_became_peered":"2023-10-18T11:58:35.517747+0000","last_unstale":"2023-10-18T11:58:37.513026+0000","last_undegraded":"2023-10-18T11:58:37.513026+0000","last_fullsized":"2023-10-18T11:58:37.513026+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.335660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801895+0000","last_change":"2023-10-18T11:58:36.524788+0000","last_active":"2023-10-18T11:58:37.801895+0000","last_peered":"2023-10-18T11:58:37.801895+0000","last_clean":"2023-10-18T11:58:37.801895+0000","last_became_active":"2023-10-18T11:58:36.524547+0000","last_became_peered":"2023-10-18T11:58:36.524547+0000","last_unstale":"2023-10-18T11:58:37.801895+0000","last_undegraded":"2023-10-18T11:58:37.801895+0000","last_fullsized":"2023-10-18T11:58:37.801895+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:02.661591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512833+0000","last_change":"2023-10-18T11:58:35.521655+0000","last_active":"2023-10-18T11:58:37.512833+0000","last_peered":"2023-10-18T11:58:37.512833+0000","last_clean":"2023-10-18T11:58:37.512833+0000","last_became_active":"2023-10-18T11:58:35.521430+0000","last_became_peered":"2023-10-18T11:58:35.521430+0000","last_unstale":"2023-10-18T11:58:37.512833+0000","last_undegraded":"2023-10-18T11:58:37.512833+0000","last_fullsized":"2023-10-18T11:58:37.512833+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:21.628472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508795+0000","last_change":"2023-10-18T11:58:36.523743+0000","last_active":"2023-10-18T11:58:37.508795+0000","last_peered":"2023-10-18T11:58:37.508795+0000","last_clean":"2023-10-18T11:58:37.508795+0000","last_became_active":"2023-10-18T11:58:36.523492+0000","last_became_peered":"2023-10-18T11:58:36.523492+0000","last_unstale":"2023-10-18T11:58:37.508795+0000","last_undegraded":"2023-10-18T11:58:37.508795+0000","last_fullsized":"2023-10-18T11:58:37.508795+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:42.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802579+0000","last_change":"2023-10-18T11:58:35.522746+0000","last_active":"2023-10-18T11:58:37.802579+0000","last_peered":"2023-10-18T11:58:37.802579+0000","last_clean":"2023-10-18T11:58:37.802579+0000","last_became_active":"2023-10-18T11:58:35.522486+0000","last_became_peered":"2023-10-18T11:58:35.522486+0000","last_unstale":"2023-10-18T11:58:37.802579+0000","last_undegraded":"2023-10-18T11:58:37.802579+0000","last_fullsized":"2023-10-18T11:58:37.802579+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:07.062822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512576+0000","last_change":"2023-10-18T11:58:36.533848+0000","last_active":"2023-10-18T11:58:37.512576+0000","last_peered":"2023-10-18T11:58:37.512576+0000","last_clean":"2023-10-18T11:58:37.512576+0000","last_became_active":"2023-10-18T11:58:36.533565+0000","last_became_peered":"2023-10-18T11:58:36.533565+0000","last_unstale":"2023-10-18T11:58:37.512576+0000","last_undegraded":"2023-10-18T11:58:37.512576+0000","last_fullsized":"2023-10-18T11:58:37.512576+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:48.249824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802427+0000","last_change":"2023-10-18T11:58:35.517587+0000","last_active":"2023-10-18T11:58:37.802427+0000","last_peered":"2023-10-18T11:58:37.802427+0000","last_clean":"2023-10-18T11:58:37.802427+0000","last_became_active":"2023-10-18T11:58:35.517322+0000","last_became_peered":"2023-10-18T11:58:35.517322+0000","last_unstale":"2023-10-18T11:58:37.802427+0000","last_undegraded":"2023-10-18T11:58:37.802427+0000","last_fullsized":"2023-10-18T11:58:37.802427+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:34.567188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512733+0000","last_change":"2023-10-18T11:58:36.531687+0000","last_active":"2023-10-18T11:58:37.512733+0000","last_peered":"2023-10-18T11:58:37.512733+0000","last_clean":"2023-10-18T11:58:37.512733+0000","last_became_active":"2023-10-18T11:58:36.531411+0000","last_became_peered":"2023-10-18T11:58:36.531411+0000","last_unstale":"2023-10-18T11:58:37.512733+0000","last_undegraded":"2023-10-18T11:58:37.512733+0000","last_fullsized":"2023-10-18T11:58:37.512733+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:04.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"203'8","reported_seq":20,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802771+0000","last_change":"2023-10-18T11:58:35.515506+0000","last_active":"2023-10-18T11:58:37.802771+0000","last_peered":"2023-10-18T11:58:37.802771+0000","last_clean":"2023-10-18T11:58:37.802771+0000","last_became_active":"2023-10-18T11:58:35.515198+0000","last_became_peered":"2023-10-18T11:58:35.515198+0000","last_unstale":"2023-10-18T11:58:37.802771+0000","last_undegraded":"2023-10-18T11:58:37.802771+0000","last_fullsized":"2023-10-18T11:58:37.802771+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.509931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802367+0000","last_change":"2023-10-18T11:58:36.515935+0000","last_active":"2023-10-18T11:58:37.802367+0000","last_peered":"2023-10-18T11:58:37.802367+0000","last_clean":"2023-10-18T11:58:37.802367+0000","last_became_active":"2023-10-18T11:58:36.515722+0000","last_became_peered":"2023-10-18T11:58:36.515722+0000","last_unstale":"2023-10-18T11:58:37.802367+0000","last_undegraded":"2023-10-18T11:58:37.802367+0000","last_fullsized":"2023-10-18T11:58:37.802367+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:58.998126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:35.515482+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:35.515185+0000","last_became_peered":"2023-10-18T11:58:35.515185+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.149129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509119+0000","last_change":"2023-10-18T11:58:36.534430+0000","last_active":"2023-10-18T11:58:37.509119+0000","last_peered":"2023-10-18T11:58:37.509119+0000","last_clean":"2023-10-18T11:58:37.509119+0000","last_became_active":"2023-10-18T11:58:36.534167+0000","last_became_peered":"2023-10-18T11:58:36.534167+0000","last_unstale":"2023-10-18T11:58:37.509119+0000","last_undegraded":"2023-10-18T11:58:37.509119+0000","last_fullsized":"2023-10-18T11:58:37.509119+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:44.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802606+0000","last_change":"2023-10-18T11:58:35.522154+0000","last_active":"2023-10-18T11:58:37.802606+0000","last_peered":"2023-10-18T11:58:37.802606+0000","last_clean":"2023-10-18T11:58:37.802606+0000","last_became_active":"2023-10-18T11:58:35.521902+0000","last_became_peered":"2023-10-18T11:58:35.521902+0000","last_unstale":"2023-10-18T11:58:37.802606+0000","last_undegraded":"2023-10-18T11:58:37.802606+0000","last_fullsized":"2023-10-18T11:58:37.802606+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:27.447038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803092+0000","last_change":"2023-10-18T11:58:36.529258+0000","last_active":"2023-10-18T11:58:37.803092+0000","last_peered":"2023-10-18T11:58:37.803092+0000","last_clean":"2023-10-18T11:58:37.803092+0000","last_became_active":"2023-10-18T11:58:36.528468+0000","last_became_peered":"2023-10-18T11:58:36.528468+0000","last_unstale":"2023-10-18T11:58:37.803092+0000","last_undegraded":"2023-10-18T11:58:37.803092+0000","last_fullsized":"2023-10-18T11:58:37.803092+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:58.035451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803037+0000","last_change":"2023-10-18T11:58:36.521908+0000","last_active":"2023-10-18T11:58:37.803037+0000","last_peered":"2023-10-18T11:58:37.803037+0000","last_clean":"2023-10-18T11:58:37.803037+0000","last_became_active":"2023-10-18T11:58:36.521381+0000","last_became_peered":"2023-10-18T11:58:36.521381+0000","last_unstale":"2023-10-18T11:58:37.803037+0000","last_undegraded":"2023-10-18T11:58:37.803037+0000","last_fullsized":"2023-10-18T11:58:37.803037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:18.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803110+0000","last_change":"2023-10-18T11:58:35.515862+0000","last_active":"2023-10-18T11:58:37.803110+0000","last_peered":"2023-10-18T11:58:37.803110+0000","last_clean":"2023-10-18T11:58:37.803110+0000","last_became_active":"2023-10-18T11:58:35.515521+0000","last_became_peered":"2023-10-18T11:58:35.515521+0000","last_unstale":"2023-10-18T11:58:37.803110+0000","last_undegraded":"2023-10-18T11:58:37.803110+0000","last_fullsized":"2023-10-18T11:58:37.803110+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:03.199289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801885+0000","last_change":"2023-10-18T11:58:36.520962+0000","last_active":"2023-10-18T11:58:37.801885+0000","last_peered":"2023-10-18T11:58:37.801885+0000","last_clean":"2023-10-18T11:58:37.801885+0000","last_became_active":"2023-10-18T11:58:36.520712+0000","last_became_peered":"2023-10-18T11:58:36.520712+0000","last_unstale":"2023-10-18T11:58:37.801885+0000","last_undegraded":"2023-10-18T11:58:37.801885+0000","last_fullsized":"2023-10-18T11:58:37.801885+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:46.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512865+0000","last_change":"2023-10-18T11:58:35.524483+0000","last_active":"2023-10-18T11:58:37.512865+0000","last_peered":"2023-10-18T11:58:37.512865+0000","last_clean":"2023-10-18T11:58:37.512865+0000","last_became_active":"2023-10-18T11:58:35.524231+0000","last_became_peered":"2023-10-18T11:58:35.524231+0000","last_unstale":"2023-10-18T11:58:37.512865+0000","last_undegraded":"2023-10-18T11:58:37.512865+0000","last_fullsized":"2023-10-18T11:58:37.512865+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:29:21.906133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.529745+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.529497+0000","last_became_peered":"2023-10-18T11:58:36.529497+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802708+0000","last_change":"2023-10-18T11:58:35.526709+0000","last_active":"2023-10-18T11:58:37.802708+0000","last_peered":"2023-10-18T11:58:37.802708+0000","last_clean":"2023-10-18T11:58:37.802708+0000","last_became_active":"2023-10-18T11:58:35.526448+0000","last_became_peered":"2023-10-18T11:58:35.526448+0000","last_unstale":"2023-10-18T11:58:37.802708+0000","last_undegraded":"2023-10-18T11:58:37.802708+0000","last_fullsized":"2023-10-18T11:58:37.802708+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:20.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802541+0000","last_change":"2023-10-18T11:58:36.528368+0000","last_active":"2023-10-18T11:58:37.802541+0000","last_peered":"2023-10-18T11:58:37.802541+0000","last_clean":"2023-10-18T11:58:37.802541+0000","last_became_active":"2023-10-18T11:58:36.528124+0000","last_became_peered":"2023-10-18T11:58:36.528124+0000","last_unstale":"2023-10-18T11:58:37.802541+0000","last_undegraded":"2023-10-18T11:58:37.802541+0000","last_fullsized":"2023-10-18T11:58:37.802541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:48.474419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802873+0000","last_change":"2023-10-18T11:58:35.525781+0000","last_active":"2023-10-18T11:58:37.802873+0000","last_peered":"2023-10-18T11:58:37.802873+0000","last_clean":"2023-10-18T11:58:37.802873+0000","last_became_active":"2023-10-18T11:58:35.525519+0000","last_became_peered":"2023-10-18T11:58:35.525519+0000","last_unstale":"2023-10-18T11:58:37.802873+0000","last_undegraded":"2023-10-18T11:58:37.802873+0000","last_fullsized":"2023-10-18T11:58:37.802873+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:47:37.957986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508969+0000","last_change":"2023-10-18T11:58:36.518933+0000","last_active":"2023-10-18T11:58:37.508969+0000","last_peered":"2023-10-18T11:58:37.508969+0000","last_clean":"2023-10-18T11:58:37.508969+0000","last_became_active":"2023-10-18T11:58:36.518691+0000","last_became_peered":"2023-10-18T11:58:36.518691+0000","last_unstale":"2023-10-18T11:58:37.508969+0000","last_undegraded":"2023-10-18T11:58:37.508969+0000","last_fullsized":"2023-10-18T11:58:37.508969+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:55.005558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802476+0000","last_change":"2023-10-18T11:58:36.522037+0000","last_active":"2023-10-18T11:58:37.802476+0000","last_peered":"2023-10-18T11:58:37.802476+0000","last_clean":"2023-10-18T11:58:37.802476+0000","last_became_active":"2023-10-18T11:58:36.521783+0000","last_became_peered":"2023-10-18T11:58:36.521783+0000","last_unstale":"2023-10-18T11:58:37.802476+0000","last_undegraded":"2023-10-18T11:58:37.802476+0000","last_fullsized":"2023-10-18T11:58:37.802476+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:37:11.613406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803083+0000","last_change":"2023-10-18T11:58:35.518886+0000","last_active":"2023-10-18T11:58:37.803083+0000","last_peered":"2023-10-18T11:58:37.803083+0000","last_clean":"2023-10-18T11:58:37.803083+0000","last_became_active":"2023-10-18T11:58:35.518591+0000","last_became_peered":"2023-10-18T11:58:35.518591+0000","last_unstale":"2023-10-18T11:58:37.803083+0000","last_undegraded":"2023-10-18T11:58:37.803083+0000","last_fullsized":"2023-10-18T11:58:37.803083+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:49.481190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802223+0000","last_change":"2023-10-18T11:58:36.519762+0000","last_active":"2023-10-18T11:58:37.802223+0000","last_peered":"2023-10-18T11:58:37.802223+0000","last_clean":"2023-10-18T11:58:37.802223+0000","last_became_active":"2023-10-18T11:58:36.517560+0000","last_became_peered":"2023-10-18T11:58:36.517560+0000","last_unstale":"2023-10-18T11:58:37.802223+0000","last_undegraded":"2023-10-18T11:58:37.802223+0000","last_fullsized":"2023-10-18T11:58:37.802223+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:29.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802645+0000","last_change":"2023-10-18T11:58:35.522798+0000","last_active":"2023-10-18T11:58:37.802645+0000","last_peered":"2023-10-18T11:58:37.802645+0000","last_clean":"2023-10-18T11:58:37.802645+0000","last_became_active":"2023-10-18T11:58:35.522532+0000","last_became_peered":"2023-10-18T11:58:35.522532+0000","last_unstale":"2023-10-18T11:58:37.802645+0000","last_undegraded":"2023-10-18T11:58:37.802645+0000","last_fullsized":"2023-10-18T11:58:37.802645+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:40:51.705544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:36.530354+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:36.530080+0000","last_became_peered":"2023-10-18T11:58:36.530080+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:19.995868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509241+0000","last_change":"2023-10-18T11:58:35.523100+0000","last_active":"2023-10-18T11:58:37.509241+0000","last_peered":"2023-10-18T11:58:37.509241+0000","last_clean":"2023-10-18T11:58:37.509241+0000","last_became_active":"2023-10-18T11:58:35.522802+0000","last_became_peered":"2023-10-18T11:58:35.522802+0000","last_unstale":"2023-10-18T11:58:37.509241+0000","last_undegraded":"2023-10-18T11:58:37.509241+0000","last_fullsized":"2023-10-18T11:58:37.509241+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802916+0000","last_change":"2023-10-18T11:58:36.522886+0000","last_active":"2023-10-18T11:58:37.802916+0000","last_peered":"2023-10-18T11:58:37.802916+0000","last_clean":"2023-10-18T11:58:37.802916+0000","last_became_active":"2023-10-18T11:58:36.522614+0000","last_became_peered":"2023-10-18T11:58:36.522614+0000","last_unstale":"2023-10-18T11:58:37.802916+0000","last_undegraded":"2023-10-18T11:58:37.802916+0000","last_fullsized":"2023-10-18T11:58:37.802916+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:52.212999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509088+0000","last_change":"2023-10-18T11:58:35.523409+0000","last_active":"2023-10-18T11:58:37.509088+0000","last_peered":"2023-10-18T11:58:37.509088+0000","last_clean":"2023-10-18T11:58:37.509088+0000","last_became_active":"2023-10-18T11:58:35.523146+0000","last_became_peered":"2023-10-18T11:58:35.523146+0000","last_unstale":"2023-10-18T11:58:37.509088+0000","last_undegraded":"2023-10-18T11:58:37.509088+0000","last_fullsized":"2023-10-18T11:58:37.509088+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:51.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802959+0000","last_change":"2023-10-18T11:58:35.522529+0000","last_active":"2023-10-18T11:58:37.802959+0000","last_peered":"2023-10-18T11:58:37.802959+0000","last_clean":"2023-10-18T11:58:37.802959+0000","last_became_active":"2023-10-18T11:58:35.522261+0000","last_became_peered":"2023-10-18T11:58:35.522261+0000","last_unstale":"2023-10-18T11:58:37.802959+0000","last_undegraded":"2023-10-18T11:58:37.802959+0000","last_fullsized":"2023-10-18T11:58:37.802959+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512534+0000","last_change":"2023-10-18T11:58:36.530248+0000","last_active":"2023-10-18T11:58:37.512534+0000","last_peered":"2023-10-18T11:58:37.512534+0000","last_clean":"2023-10-18T11:58:37.512534+0000","last_became_active":"2023-10-18T11:58:36.529997+0000","last_became_peered":"2023-10-18T11:58:36.529997+0000","last_unstale":"2023-10-18T11:58:37.512534+0000","last_undegraded":"2023-10-18T11:58:37.512534+0000","last_fullsized":"2023-10-18T11:58:37.512534+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:58.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802154+0000","last_change":"2023-10-18T11:58:35.527290+0000","last_active":"2023-10-18T11:58:37.802154+0000","last_peered":"2023-10-18T11:58:37.802154+0000","last_clean":"2023-10-18T11:58:37.802154+0000","last_became_active":"2023-10-18T11:58:35.527052+0000","last_became_peered":"2023-10-18T11:58:35.527052+0000","last_unstale":"2023-10-18T11:58:37.802154+0000","last_undegraded":"2023-10-18T11:58:37.802154+0000","last_fullsized":"2023-10-18T11:58:37.802154+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802594+0000","last_change":"2023-10-18T11:58:36.520551+0000","last_active":"2023-10-18T11:58:37.802594+0000","last_peered":"2023-10-18T11:58:37.802594+0000","last_clean":"2023-10-18T11:58:37.802594+0000","last_became_active":"2023-10-18T11:58:36.520297+0000","last_became_peered":"2023-10-18T11:58:36.520297+0000","last_unstale":"2023-10-18T11:58:37.802594+0000","last_undegraded":"2023-10-18T11:58:37.802594+0000","last_fullsized":"2023-10-18T11:58:37.802594+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:25.536175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802945+0000","last_change":"2023-10-18T11:58:35.526011+0000","last_active":"2023-10-18T11:58:37.802945+0000","last_peered":"2023-10-18T11:58:37.802945+0000","last_clean":"2023-10-18T11:58:37.802945+0000","last_became_active":"2023-10-18T11:58:35.525734+0000","last_became_peered":"2023-10-18T11:58:35.525734+0000","last_unstale":"2023-10-18T11:58:37.802945+0000","last_undegraded":"2023-10-18T11:58:37.802945+0000","last_fullsized":"2023-10-18T11:58:37.802945+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:19.352633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802222+0000","last_change":"2023-10-18T11:58:36.528280+0000","last_active":"2023-10-18T11:58:37.802222+0000","last_peered":"2023-10-18T11:58:37.802222+0000","last_clean":"2023-10-18T11:58:37.802222+0000","last_became_active":"2023-10-18T11:58:36.528012+0000","last_became_peered":"2023-10-18T11:58:36.528012+0000","last_unstale":"2023-10-18T11:58:37.802222+0000","last_undegraded":"2023-10-18T11:58:37.802222+0000","last_fullsized":"2023-10-18T11:58:37.802222+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:28.449393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801952+0000","last_change":"2023-10-18T11:58:35.513693+0000","last_active":"2023-10-18T11:58:37.801952+0000","last_peered":"2023-10-18T11:58:37.801952+0000","last_clean":"2023-10-18T11:58:37.801952+0000","last_became_active":"2023-10-18T11:58:35.513429+0000","last_became_peered":"2023-10-18T11:58:35.513429+0000","last_unstale":"2023-10-18T11:58:37.801952+0000","last_undegraded":"2023-10-18T11:58:37.801952+0000","last_fullsized":"2023-10-18T11:58:37.801952+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:12.442948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"193'335","reported_seq":1140,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-10-18T11:58:36.518008+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:58:36.518008+0000","last_peered":"2023-10-18T11:58:36.518008+0000","last_clean":"2023-10-18T11:58:36.518008+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:58:36.518008+0000","last_undegraded":"2023-10-18T11:58:36.518008+0000","last_fullsized":"2023-10-18T11:58:36.518008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":120,"acting":120,"num_store_stats":7},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52308,"kb_used_data":3080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319532,"statfs":{"total":96636764160,"available":96583200768,"internally_reserved":0,"allocated":3153920,"data_stored":1895422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.363},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74399999999999999,"5min":0.77200000000000002,"15min":0.84099999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.39700000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":1.298},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.82699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.35699999999999998},"max":{"1min":0.80200000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45200000000000001,"15min":0.44900000000000001},"min":{"1min":0.34000000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.80200000000000005,"15min":1.5249999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.27700000000000002},"max":{"1min":0.63500000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.78900000000000003,"15min":0.90800000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.375,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":0.82699999999999996,"15min":1.4179999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":1.478},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.874},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37768,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334072,"statfs":{"total":96636764160,"available":96598089728,"internally_reserved":0,"allocated":2486272,"data_stored":1267517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.50700000000000001,"5min":0.41899999999999998,"15min":0.29899999999999999},"max":{"1min":0.78100000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.27900000000000003},"max":{"1min":0.76700000000000002,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.87},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.625,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.25600000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.82599999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.626},"min":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47299999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.76900000000000002},"last":0.37},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.71999999999999997,"15min":0.82999999999999996},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.81200000000000006},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72699999999999998,"5min":0.80600000000000005,"15min":0.84599999999999997},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575970,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60068,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311772,"statfs":{"total":96636764160,"available":96575254528,"internally_reserved":0,"allocated":2576384,"data_stored":1306688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.249},"max":{"1min":0.84099999999999997,"5min":0.90300000000000002,"15min":0.90800000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.50700000000000001,"5min":0.38500000000000001,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.95899999999999996,"15min":0.96099999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.24299999999999999},"max":{"1min":1.081,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82199999999999995,"15min":0.754},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.28399999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.98699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.84999999999999998,"15min":0.98399999999999999},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.53200000000000003,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.0349999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.501,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":1,"15min":1},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.30299999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.98499999999999999,"15min":1.01},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.93899999999999995,"15min":0.97999999999999998},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83699999999999997,"5min":1.101,"15min":1.101},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.34000000000000002},"max":{"1min":0.83399999999999996,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575970,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29544,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342296,"statfs":{"total":96636764160,"available":96606511104,"internally_reserved":0,"allocated":2576384,"data_stored":1306700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.62},"min":{"1min":0.45400000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80700000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.80200000000000005,"15min":0.85399999999999998},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69499999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.372,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.752,"5min":1.123,"15min":1.123},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.92100000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.93600000000000005},"last":0.629}]},{"osd":3,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.312},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.34599999999999997,"15min":0.31900000000000001},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":2568192,"data_stored":1305214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.221},"max":{"1min":0.57099999999999995,"5min":0.61599999999999999,"15min":0.63300000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.28100000000000003,"15min":0.23400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.374}]},{"osd":1,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26400000000000001},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.56000000000000005,"5min":0.60599999999999998,"15min":0.65000000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.73399999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.052},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47499999999999998,"15min":0.40999999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.026},"last":0.625}]},{"osd":6,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.437},"max":{"1min":0.82299999999999995,"5min":0.93100000000000005,"15min":0.94499999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.68700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.45400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.55600000000000005,"5min":0.46400000000000002,"15min":0.442},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.58099999999999996,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.67800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60060,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311780,"statfs":{"total":96636764160,"available":96575262720,"internally_reserved":0,"allocated":2564096,"data_stored":1305636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.252,"5min":0.222,"15min":0.216},"max":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.67200000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.27900000000000003,"5min":0.26200000000000001,"15min":0.25800000000000001},"max":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.746,"15min":0.746},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.435,"15min":0.432},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.374},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35399999999999998,"5min":0.29899999999999999,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.33600000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.499,"5min":0.442,"15min":0.437},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.47799999999999998,"15min":0.41699999999999998},"max":{"1min":0.80900000000000005,"5min":1.087,"15min":1.087},"last":0.435}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.54500000000000004,"5min":0.46500000000000002,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66600000000000004,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.45200000000000001,"15min":0.40999999999999998},"max":{"1min":0.85799999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68500000000000005,"15min":0.68200000000000005},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.45100000000000001},"max":{"1min":0.84099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.443,"15min":0.42499999999999999},"max":{"1min":0.86799999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.53900000000000003,"5min":0.443,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.01},"last":0.499}]}]},{"osd":1,"up_from":13,"seq":55834575970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60048,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311792,"statfs":{"total":96636764160,"available":96575275008,"internally_reserved":0,"allocated":2560000,"data_stored":1305054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.182},"max":{"1min":0.73899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65800000000000003,"5min":0.81000000000000005,"15min":0.85299999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.23000000000000001},"max":{"1min":0.496,"5min":0.877,"15min":0.877},"last":0.28000000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.69799999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.377},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51400000000000001},"min":{"1min":0.28899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.74399999999999999,"5min":0.79800000000000004,"15min":0.91000000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.74099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.65600000000000003},"min":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.29599999999999999},"max":{"1min":0.80300000000000005,"5min":0.86599999999999999,"15min":1.2390000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.316},"max":{"1min":0.70299999999999996,"5min":1.05,"15min":1.212},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":1.0209999999999999,"15min":1.0589999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77600000000000002,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.47399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82499999999999996,"5min":0.88400000000000001,"15min":1.0620000000000001},"last":0.69899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42600,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329240,"statfs":{"total":96636764160,"available":96593141760,"internally_reserved":0,"allocated":3170304,"data_stored":1896150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.57199999999999995,"5min":0.64400000000000002,"15min":0.66200000000000003},"last":0.247}]},{"osd":2,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.33700000000000002,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.433,"15min":0.42799999999999999},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.25700000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.36699999999999999},"max":{"1min":0.876,"5min":0.96999999999999997,"15min":0.997},"last":0.495},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.50900000000000001,"5min":0.45400000000000001,"15min":0.39100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":0.93999999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.504,"5min":0.439,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.95799999999999996,"15min":0.996},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.44700000000000001},"max":{"1min":0.89300000000000002,"5min":1.0089999999999999,"15min":1.038},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.48599999999999999,"5min":0.44900000000000001,"15min":0.44800000000000001},"max":{"1min":0.91100000000000003,"5min":0.95099999999999996,"15min":1.0609999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":1.0109999999999999,"15min":1.0189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.42999999999999999},"max":{"1min":0.89400000000000002,"5min":0.997,"15min":0.997},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:58:39.798 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:58:39.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.932+0000 7f675899e640 1 -- 172.21.15.125:0/1525210526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f675415b2a0 msgr2=0x7f67540d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.932+0000 7f675899e640 1 --2- 172.21.15.125:0/1525210526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f675415b2a0 0x7f67540d1910 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6748009830 tx=0x7f674802bcd0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.932+0000 7f675899e640 1 -- 172.21.15.125:0/1525210526 shutdown_connections 2023-10-18T11:58:39.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.932+0000 7f675899e640 1 --2- 172.21.15.125:0/1525210526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67540ceb40 0x7f67540b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.932+0000 7f675899e640 1 --2- 172.21.15.125:0/1525210526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f675415b2a0 0x7f67540d1910 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.932+0000 7f675899e640 1 -- 172.21.15.125:0/1525210526 >> 172.21.15.125:0/1525210526 conn(0x7f67540d0d60 msgr2=0x7f67540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:39.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.932+0000 7f675899e640 1 -- 172.21.15.125:0/1525210526 shutdown_connections 2023-10-18T11:58:39.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.932+0000 7f675899e640 1 -- 172.21.15.125:0/1525210526 wait complete. 2023-10-18T11:58:39.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.933+0000 7f675899e640 1 Processor -- start 2023-10-18T11:58:39.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.933+0000 7f675899e640 1 -- start start 2023-10-18T11:58:39.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f675899e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67540ceb40 0x7f675415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f675899e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67540cf340 0x7f675415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f675899e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f675415b2a0 0x7f675415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f675899e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c8290 con 0x7f675415b2a0 2023-10-18T11:58:39.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f675899e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675415c800 con 0x7f67540cf340 2023-10-18T11:58:39.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f675899e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675415e030 con 0x7f67540ceb40 2023-10-18T11:58:39.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f675415b2a0 0x7f675415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f675415b2a0 0x7f675415bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50002/0 (socket says 172.21.15.125:50002) 2023-10-18T11:58:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 -- 172.21.15.125:0/4029139446 learned_addr learned my addr 172.21.15.125:0/4029139446 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6751d9b640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67540cf340 0x7f675415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f675259c640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67540ceb40 0x7f675415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 -- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67540ceb40 msgr2=0x7f675415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67540ceb40 0x7f675415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 -- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67540cf340 msgr2=0x7f675415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:39.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67540cf340 0x7f675415b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:39.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 -- 172.21.15.125:0/4029139446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6748009490 con 0x7f675415b2a0 2023-10-18T11:58:39.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6752d9d640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f675415b2a0 0x7f675415bc10 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f674400eb10 tx=0x7f674400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.934+0000 7f6742ffd640 1 -- 172.21.15.125:0/4029139446 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744004b20 con 0x7f675415b2a0 2023-10-18T11:58:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.935+0000 7f6742ffd640 1 -- 172.21.15.125:0/4029139446 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6744004ca0 con 0x7f675415b2a0 2023-10-18T11:58:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.935+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67540c7f60 con 0x7f675415b2a0 2023-10-18T11:58:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.935+0000 7f6742ffd640 1 -- 172.21.15.125:0/4029139446 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744005280 con 0x7f675415b2a0 2023-10-18T11:58:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.935+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675415c240 con 0x7f675415b2a0 2023-10-18T11:58:39.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.936+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67540c8400 con 0x7f675415b2a0 2023-10-18T11:58:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.936+0000 7f6742ffd640 1 -- 172.21.15.125:0/4029139446 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f67440053e0 con 0x7f675415b2a0 2023-10-18T11:58:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.936+0000 7f6742ffd640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f673406af60 0x7f673406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.936+0000 7f6742ffd640 1 -- 172.21.15.125:0/4029139446 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f6744014070 con 0x7f675415b2a0 2023-10-18T11:58:39.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.936+0000 7f675259c640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f673406af60 0x7f673406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:39.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.937+0000 7f675259c640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f673406af60 0x7f673406d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f674c007fd0 tx=0x7f674c007cc0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:39.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:39.941+0000 7f6742ffd640 1 -- 172.21.15.125:0/4029139446 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6744055400 con 0x7f675415b2a0 2023-10-18T11:58:40.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.105+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 --> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67540c80f0 con 0x7f673406af60 2023-10-18T11:58:40.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.113+0000 7f6742ffd640 1 -- 172.21.15.125:0/4029139446 <== mgr.9529 v2:172.21.15.139:6832/1565260611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288588 (secure 0 0 0) 0x7f674c00f820 con 0x7f673406af60 2023-10-18T11:58:40.118 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:40.119 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:58:40.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f673406af60 msgr2=0x7f673406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f673406af60 0x7f673406d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f674c007fd0 tx=0x7f674c007cc0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f675415b2a0 msgr2=0x7f675415bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f675415b2a0 0x7f675415bc10 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f674400eb10 tx=0x7f674400c2d0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 shutdown_connections 2023-10-18T11:58:40.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f673406af60 0x7f673406d420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67540cf340 0x7f675415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f675415b2a0 0x7f675415bc10 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 --2- 172.21.15.125:0/4029139446 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67540ceb40 0x7f675415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 >> 172.21.15.125:0/4029139446 conn(0x7f67540d0d60 msgr2=0x7f67540cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:40.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 shutdown_connections 2023-10-18T11:58:40.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.116+0000 7f675899e640 1 -- 172.21.15.125:0/4029139446 wait complete. 2023-10-18T11:58:40.129 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T11:58:38.893982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":598,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":372,"ondisk_log_size":372,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":402452,"kb_used_data":21148,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754572268,"statfs":{"total":773094113280,"available":772682002432,"internally_reserved":0,"allocated":21655552,"data_stored":11588381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":390319672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.394160"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802702+0000","last_change":"2023-10-18T11:58:36.522565+0000","last_active":"2023-10-18T11:58:37.802702+0000","last_peered":"2023-10-18T11:58:37.802702+0000","last_clean":"2023-10-18T11:58:37.802702+0000","last_became_active":"2023-10-18T11:58:36.521736+0000","last_became_peered":"2023-10-18T11:58:36.521736+0000","last_unstale":"2023-10-18T11:58:37.802702+0000","last_undegraded":"2023-10-18T11:58:37.802702+0000","last_fullsized":"2023-10-18T11:58:37.802702+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:46:36.674788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802006+0000","last_change":"2023-10-18T11:58:36.527651+0000","last_active":"2023-10-18T11:58:37.802006+0000","last_peered":"2023-10-18T11:58:37.802006+0000","last_clean":"2023-10-18T11:58:37.802006+0000","last_became_active":"2023-10-18T11:58:36.527375+0000","last_became_peered":"2023-10-18T11:58:36.527375+0000","last_unstale":"2023-10-18T11:58:37.802006+0000","last_undegraded":"2023-10-18T11:58:37.802006+0000","last_fullsized":"2023-10-18T11:58:37.802006+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:55:29.683103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.534333+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.534066+0000","last_became_peered":"2023-10-18T11:58:36.534066+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:29.879395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508872+0000","last_change":"2023-10-18T11:58:36.529615+0000","last_active":"2023-10-18T11:58:37.508872+0000","last_peered":"2023-10-18T11:58:37.508872+0000","last_clean":"2023-10-18T11:58:37.508872+0000","last_became_active":"2023-10-18T11:58:36.529380+0000","last_became_peered":"2023-10-18T11:58:36.529380+0000","last_unstale":"2023-10-18T11:58:37.508872+0000","last_undegraded":"2023-10-18T11:58:37.508872+0000","last_fullsized":"2023-10-18T11:58:37.508872+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:44.701593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508825+0000","last_change":"2023-10-18T11:58:36.528591+0000","last_active":"2023-10-18T11:58:37.508825+0000","last_peered":"2023-10-18T11:58:37.508825+0000","last_clean":"2023-10-18T11:58:37.508825+0000","last_became_active":"2023-10-18T11:58:36.528332+0000","last_became_peered":"2023-10-18T11:58:36.528332+0000","last_unstale":"2023-10-18T11:58:37.508825+0000","last_undegraded":"2023-10-18T11:58:37.508825+0000","last_fullsized":"2023-10-18T11:58:37.508825+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:04.968822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802608+0000","last_change":"2023-10-18T11:58:36.526452+0000","last_active":"2023-10-18T11:58:37.802608+0000","last_peered":"2023-10-18T11:58:37.802608+0000","last_clean":"2023-10-18T11:58:37.802608+0000","last_became_active":"2023-10-18T11:58:36.526144+0000","last_became_peered":"2023-10-18T11:58:36.526144+0000","last_unstale":"2023-10-18T11:58:37.802608+0000","last_undegraded":"2023-10-18T11:58:37.802608+0000","last_fullsized":"2023-10-18T11:58:37.802608+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:15.609770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802428+0000","last_change":"2023-10-18T11:58:36.526263+0000","last_active":"2023-10-18T11:58:37.802428+0000","last_peered":"2023-10-18T11:58:37.802428+0000","last_clean":"2023-10-18T11:58:37.802428+0000","last_became_active":"2023-10-18T11:58:36.525882+0000","last_became_peered":"2023-10-18T11:58:36.525882+0000","last_unstale":"2023-10-18T11:58:37.802428+0000","last_undegraded":"2023-10-18T11:58:37.802428+0000","last_fullsized":"2023-10-18T11:58:37.802428+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:05.910088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802812+0000","last_change":"2023-10-18T11:58:36.520303+0000","last_active":"2023-10-18T11:58:37.802812+0000","last_peered":"2023-10-18T11:58:37.802812+0000","last_clean":"2023-10-18T11:58:37.802812+0000","last_became_active":"2023-10-18T11:58:36.520065+0000","last_became_peered":"2023-10-18T11:58:36.520065+0000","last_unstale":"2023-10-18T11:58:37.802812+0000","last_undegraded":"2023-10-18T11:58:37.802812+0000","last_fullsized":"2023-10-18T11:58:37.802812+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:40.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509204+0000","last_change":"2023-10-18T11:58:36.534318+0000","last_active":"2023-10-18T11:58:37.509204+0000","last_peered":"2023-10-18T11:58:37.509204+0000","last_clean":"2023-10-18T11:58:37.509204+0000","last_became_active":"2023-10-18T11:58:36.534071+0000","last_became_peered":"2023-10-18T11:58:36.534071+0000","last_unstale":"2023-10-18T11:58:37.509204+0000","last_undegraded":"2023-10-18T11:58:37.509204+0000","last_fullsized":"2023-10-18T11:58:37.509204+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:55.491621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802310+0000","last_change":"2023-10-18T11:58:36.520313+0000","last_active":"2023-10-18T11:58:37.802310+0000","last_peered":"2023-10-18T11:58:37.802310+0000","last_clean":"2023-10-18T11:58:37.802310+0000","last_became_active":"2023-10-18T11:58:36.520042+0000","last_became_peered":"2023-10-18T11:58:36.520042+0000","last_unstale":"2023-10-18T11:58:37.802310+0000","last_undegraded":"2023-10-18T11:58:37.802310+0000","last_fullsized":"2023-10-18T11:58:37.802310+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:57.623658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802462+0000","last_change":"2023-10-18T11:58:36.526211+0000","last_active":"2023-10-18T11:58:37.802462+0000","last_peered":"2023-10-18T11:58:37.802462+0000","last_clean":"2023-10-18T11:58:37.802462+0000","last_became_active":"2023-10-18T11:58:36.525812+0000","last_became_peered":"2023-10-18T11:58:36.525812+0000","last_unstale":"2023-10-18T11:58:37.802462+0000","last_undegraded":"2023-10-18T11:58:37.802462+0000","last_fullsized":"2023-10-18T11:58:37.802462+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:39.938700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512711+0000","last_change":"2023-10-18T11:58:36.529732+0000","last_active":"2023-10-18T11:58:37.512711+0000","last_peered":"2023-10-18T11:58:37.512711+0000","last_clean":"2023-10-18T11:58:37.512711+0000","last_became_active":"2023-10-18T11:58:36.529455+0000","last_became_peered":"2023-10-18T11:58:36.529455+0000","last_unstale":"2023-10-18T11:58:37.512711+0000","last_undegraded":"2023-10-18T11:58:37.512711+0000","last_fullsized":"2023-10-18T11:58:37.512711+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:38.419107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802037+0000","last_change":"2023-10-18T11:58:36.534249+0000","last_active":"2023-10-18T11:58:37.802037+0000","last_peered":"2023-10-18T11:58:37.802037+0000","last_clean":"2023-10-18T11:58:37.802037+0000","last_became_active":"2023-10-18T11:58:36.533956+0000","last_became_peered":"2023-10-18T11:58:36.533956+0000","last_unstale":"2023-10-18T11:58:37.802037+0000","last_undegraded":"2023-10-18T11:58:37.802037+0000","last_fullsized":"2023-10-18T11:58:37.802037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:29.001140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802132+0000","last_change":"2023-10-18T11:58:36.530863+0000","last_active":"2023-10-18T11:58:37.802132+0000","last_peered":"2023-10-18T11:58:37.802132+0000","last_clean":"2023-10-18T11:58:37.802132+0000","last_became_active":"2023-10-18T11:58:36.530625+0000","last_became_peered":"2023-10-18T11:58:36.530625+0000","last_unstale":"2023-10-18T11:58:37.802132+0000","last_undegraded":"2023-10-18T11:58:37.802132+0000","last_fullsized":"2023-10-18T11:58:37.802132+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:03.443134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508994+0000","last_change":"2023-10-18T11:58:36.531552+0000","last_active":"2023-10-18T11:58:37.508994+0000","last_peered":"2023-10-18T11:58:37.508994+0000","last_clean":"2023-10-18T11:58:37.508994+0000","last_became_active":"2023-10-18T11:58:36.531163+0000","last_became_peered":"2023-10-18T11:58:36.531163+0000","last_unstale":"2023-10-18T11:58:37.508994+0000","last_undegraded":"2023-10-18T11:58:37.508994+0000","last_fullsized":"2023-10-18T11:58:37.508994+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:49.241771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509268+0000","last_change":"2023-10-18T11:58:36.530026+0000","last_active":"2023-10-18T11:58:37.509268+0000","last_peered":"2023-10-18T11:58:37.509268+0000","last_clean":"2023-10-18T11:58:37.509268+0000","last_became_active":"2023-10-18T11:58:36.529058+0000","last_became_peered":"2023-10-18T11:58:36.529058+0000","last_unstale":"2023-10-18T11:58:37.509268+0000","last_undegraded":"2023-10-18T11:58:37.509268+0000","last_fullsized":"2023-10-18T11:58:37.509268+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:41.289044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513034+0000","last_change":"2023-10-18T11:58:36.532114+0000","last_active":"2023-10-18T11:58:37.513034+0000","last_peered":"2023-10-18T11:58:37.513034+0000","last_clean":"2023-10-18T11:58:37.513034+0000","last_became_active":"2023-10-18T11:58:36.531526+0000","last_became_peered":"2023-10-18T11:58:36.531526+0000","last_unstale":"2023-10-18T11:58:37.513034+0000","last_undegraded":"2023-10-18T11:58:37.513034+0000","last_fullsized":"2023-10-18T11:58:37.513034+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:38.904126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512884+0000","last_change":"2023-10-18T11:58:36.534033+0000","last_active":"2023-10-18T11:58:37.512884+0000","last_peered":"2023-10-18T11:58:37.512884+0000","last_clean":"2023-10-18T11:58:37.512884+0000","last_became_active":"2023-10-18T11:58:36.533772+0000","last_became_peered":"2023-10-18T11:58:36.533772+0000","last_unstale":"2023-10-18T11:58:37.512884+0000","last_undegraded":"2023-10-18T11:58:37.512884+0000","last_fullsized":"2023-10-18T11:58:37.512884+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:04.963404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508988+0000","last_change":"2023-10-18T11:58:36.529660+0000","last_active":"2023-10-18T11:58:37.508988+0000","last_peered":"2023-10-18T11:58:37.508988+0000","last_clean":"2023-10-18T11:58:37.508988+0000","last_became_active":"2023-10-18T11:58:36.529125+0000","last_became_peered":"2023-10-18T11:58:36.529125+0000","last_unstale":"2023-10-18T11:58:37.508988+0000","last_undegraded":"2023-10-18T11:58:37.508988+0000","last_fullsized":"2023-10-18T11:58:37.508988+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:01.717669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802201+0000","last_change":"2023-10-18T11:58:36.519499+0000","last_active":"2023-10-18T11:58:37.802201+0000","last_peered":"2023-10-18T11:58:37.802201+0000","last_clean":"2023-10-18T11:58:37.802201+0000","last_became_active":"2023-10-18T11:58:36.519246+0000","last_became_peered":"2023-10-18T11:58:36.519246+0000","last_unstale":"2023-10-18T11:58:37.802201+0000","last_undegraded":"2023-10-18T11:58:37.802201+0000","last_fullsized":"2023-10-18T11:58:37.802201+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:48.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802474+0000","last_change":"2023-10-18T11:58:36.514398+0000","last_active":"2023-10-18T11:58:37.802474+0000","last_peered":"2023-10-18T11:58:37.802474+0000","last_clean":"2023-10-18T11:58:37.802474+0000","last_became_active":"2023-10-18T11:58:36.514141+0000","last_became_peered":"2023-10-18T11:58:36.514141+0000","last_unstale":"2023-10-18T11:58:37.802474+0000","last_undegraded":"2023-10-18T11:58:37.802474+0000","last_fullsized":"2023-10-18T11:58:37.802474+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:43.247625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802305+0000","last_change":"2023-10-18T11:58:36.527748+0000","last_active":"2023-10-18T11:58:37.802305+0000","last_peered":"2023-10-18T11:58:37.802305+0000","last_clean":"2023-10-18T11:58:37.802305+0000","last_became_active":"2023-10-18T11:58:36.527498+0000","last_became_peered":"2023-10-18T11:58:36.527498+0000","last_unstale":"2023-10-18T11:58:37.802305+0000","last_undegraded":"2023-10-18T11:58:37.802305+0000","last_fullsized":"2023-10-18T11:58:37.802305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:35.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801779+0000","last_change":"2023-10-18T11:58:36.521950+0000","last_active":"2023-10-18T11:58:37.801779+0000","last_peered":"2023-10-18T11:58:37.801779+0000","last_clean":"2023-10-18T11:58:37.801779+0000","last_became_active":"2023-10-18T11:58:36.521681+0000","last_became_peered":"2023-10-18T11:58:36.521681+0000","last_unstale":"2023-10-18T11:58:37.801779+0000","last_undegraded":"2023-10-18T11:58:37.801779+0000","last_fullsized":"2023-10-18T11:58:37.801779+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:43.982016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802739+0000","last_change":"2023-10-18T11:58:36.518867+0000","last_active":"2023-10-18T11:58:37.802739+0000","last_peered":"2023-10-18T11:58:37.802739+0000","last_clean":"2023-10-18T11:58:37.802739+0000","last_became_active":"2023-10-18T11:58:36.518625+0000","last_became_peered":"2023-10-18T11:58:36.518625+0000","last_unstale":"2023-10-18T11:58:37.802739+0000","last_undegraded":"2023-10-18T11:58:37.802739+0000","last_fullsized":"2023-10-18T11:58:37.802739+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:20.411248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802849+0000","last_change":"2023-10-18T11:58:36.528722+0000","last_active":"2023-10-18T11:58:37.802849+0000","last_peered":"2023-10-18T11:58:37.802849+0000","last_clean":"2023-10-18T11:58:37.802849+0000","last_became_active":"2023-10-18T11:58:36.528462+0000","last_became_peered":"2023-10-18T11:58:36.528462+0000","last_unstale":"2023-10-18T11:58:37.802849+0000","last_undegraded":"2023-10-18T11:58:37.802849+0000","last_fullsized":"2023-10-18T11:58:37.802849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:58:27.045894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803175+0000","last_change":"2023-10-18T11:58:36.527594+0000","last_active":"2023-10-18T11:58:37.803175+0000","last_peered":"2023-10-18T11:58:37.803175+0000","last_clean":"2023-10-18T11:58:37.803175+0000","last_became_active":"2023-10-18T11:58:36.526226+0000","last_became_peered":"2023-10-18T11:58:36.526226+0000","last_unstale":"2023-10-18T11:58:37.803175+0000","last_undegraded":"2023-10-18T11:58:37.803175+0000","last_fullsized":"2023-10-18T11:58:37.803175+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:28.446100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509068+0000","last_change":"2023-10-18T11:58:36.529803+0000","last_active":"2023-10-18T11:58:37.509068+0000","last_peered":"2023-10-18T11:58:37.509068+0000","last_clean":"2023-10-18T11:58:37.509068+0000","last_became_active":"2023-10-18T11:58:36.529446+0000","last_became_peered":"2023-10-18T11:58:36.529446+0000","last_unstale":"2023-10-18T11:58:37.509068+0000","last_undegraded":"2023-10-18T11:58:37.509068+0000","last_fullsized":"2023-10-18T11:58:37.509068+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:19.031434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802519+0000","last_change":"2023-10-18T11:58:36.526866+0000","last_active":"2023-10-18T11:58:37.802519+0000","last_peered":"2023-10-18T11:58:37.802519+0000","last_clean":"2023-10-18T11:58:37.802519+0000","last_became_active":"2023-10-18T11:58:36.526609+0000","last_became_peered":"2023-10-18T11:58:36.526609+0000","last_unstale":"2023-10-18T11:58:37.802519+0000","last_undegraded":"2023-10-18T11:58:37.802519+0000","last_fullsized":"2023-10-18T11:58:37.802519+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:04.860806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802258+0000","last_change":"2023-10-18T11:58:36.515793+0000","last_active":"2023-10-18T11:58:37.802258+0000","last_peered":"2023-10-18T11:58:37.802258+0000","last_clean":"2023-10-18T11:58:37.802258+0000","last_became_active":"2023-10-18T11:58:36.515523+0000","last_became_peered":"2023-10-18T11:58:36.515523+0000","last_unstale":"2023-10-18T11:58:37.802258+0000","last_undegraded":"2023-10-18T11:58:37.802258+0000","last_fullsized":"2023-10-18T11:58:37.802258+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802828+0000","last_change":"2023-10-18T11:58:36.515930+0000","last_active":"2023-10-18T11:58:37.802828+0000","last_peered":"2023-10-18T11:58:37.802828+0000","last_clean":"2023-10-18T11:58:37.802828+0000","last_became_active":"2023-10-18T11:58:36.515681+0000","last_became_peered":"2023-10-18T11:58:36.515681+0000","last_unstale":"2023-10-18T11:58:37.802828+0000","last_undegraded":"2023-10-18T11:58:37.802828+0000","last_fullsized":"2023-10-18T11:58:37.802828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:48:47.733918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508862+0000","last_change":"2023-10-18T11:58:36.530777+0000","last_active":"2023-10-18T11:58:37.508862+0000","last_peered":"2023-10-18T11:58:37.508862+0000","last_clean":"2023-10-18T11:58:37.508862+0000","last_became_active":"2023-10-18T11:58:36.530454+0000","last_became_peered":"2023-10-18T11:58:36.530454+0000","last_unstale":"2023-10-18T11:58:37.508862+0000","last_undegraded":"2023-10-18T11:58:37.508862+0000","last_fullsized":"2023-10-18T11:58:37.508862+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:00.542319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803195+0000","last_change":"2023-10-18T11:58:36.525494+0000","last_active":"2023-10-18T11:58:37.803195+0000","last_peered":"2023-10-18T11:58:37.803195+0000","last_clean":"2023-10-18T11:58:37.803195+0000","last_became_active":"2023-10-18T11:58:36.525200+0000","last_became_peered":"2023-10-18T11:58:36.525200+0000","last_unstale":"2023-10-18T11:58:37.803195+0000","last_undegraded":"2023-10-18T11:58:37.803195+0000","last_fullsized":"2023-10-18T11:58:37.803195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"203'4","reported_seq":16,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803001+0000","last_change":"2023-10-18T11:58:35.527076+0000","last_active":"2023-10-18T11:58:37.803001+0000","last_peered":"2023-10-18T11:58:37.803001+0000","last_clean":"2023-10-18T11:58:37.803001+0000","last_became_active":"2023-10-18T11:58:35.526809+0000","last_became_peered":"2023-10-18T11:58:35.526809+0000","last_unstale":"2023-10-18T11:58:37.803001+0000","last_undegraded":"2023-10-18T11:58:37.803001+0000","last_fullsized":"2023-10-18T11:58:37.803001+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802906+0000","last_change":"2023-10-18T11:58:36.521457+0000","last_active":"2023-10-18T11:58:37.802906+0000","last_peered":"2023-10-18T11:58:37.802906+0000","last_clean":"2023-10-18T11:58:37.802906+0000","last_became_active":"2023-10-18T11:58:36.521209+0000","last_became_peered":"2023-10-18T11:58:36.521209+0000","last_unstale":"2023-10-18T11:58:37.802906+0000","last_undegraded":"2023-10-18T11:58:37.802906+0000","last_fullsized":"2023-10-18T11:58:37.802906+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:40.559400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509267+0000","last_change":"2023-10-18T11:58:35.521828+0000","last_active":"2023-10-18T11:58:37.509267+0000","last_peered":"2023-10-18T11:58:37.509267+0000","last_clean":"2023-10-18T11:58:37.509267+0000","last_became_active":"2023-10-18T11:58:35.521574+0000","last_became_peered":"2023-10-18T11:58:35.521574+0000","last_unstale":"2023-10-18T11:58:37.509267+0000","last_undegraded":"2023-10-18T11:58:37.509267+0000","last_fullsized":"2023-10-18T11:58:37.509267+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:47.472064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803025+0000","last_change":"2023-10-18T11:58:36.522588+0000","last_active":"2023-10-18T11:58:37.803025+0000","last_peered":"2023-10-18T11:58:37.803025+0000","last_clean":"2023-10-18T11:58:37.803025+0000","last_became_active":"2023-10-18T11:58:36.522182+0000","last_became_peered":"2023-10-18T11:58:36.522182+0000","last_unstale":"2023-10-18T11:58:37.803025+0000","last_undegraded":"2023-10-18T11:58:37.803025+0000","last_fullsized":"2023-10-18T11:58:37.803025+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.326674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802389+0000","last_change":"2023-10-18T11:58:35.516307+0000","last_active":"2023-10-18T11:58:37.802389+0000","last_peered":"2023-10-18T11:58:37.802389+0000","last_clean":"2023-10-18T11:58:37.802389+0000","last_became_active":"2023-10-18T11:58:35.515995+0000","last_became_peered":"2023-10-18T11:58:35.515995+0000","last_unstale":"2023-10-18T11:58:37.802389+0000","last_undegraded":"2023-10-18T11:58:37.802389+0000","last_fullsized":"2023-10-18T11:58:37.802389+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:43.302735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512924+0000","last_change":"2023-10-18T11:58:35.526193+0000","last_active":"2023-10-18T11:58:37.512924+0000","last_peered":"2023-10-18T11:58:37.512924+0000","last_clean":"2023-10-18T11:58:37.512924+0000","last_became_active":"2023-10-18T11:58:35.525920+0000","last_became_peered":"2023-10-18T11:58:35.525920+0000","last_unstale":"2023-10-18T11:58:37.512924+0000","last_undegraded":"2023-10-18T11:58:37.512924+0000","last_fullsized":"2023-10-18T11:58:37.512924+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:17.858763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512416+0000","last_change":"2023-10-18T11:58:36.529406+0000","last_active":"2023-10-18T11:58:37.512416+0000","last_peered":"2023-10-18T11:58:37.512416+0000","last_clean":"2023-10-18T11:58:37.512416+0000","last_became_active":"2023-10-18T11:58:36.528795+0000","last_became_peered":"2023-10-18T11:58:36.528795+0000","last_unstale":"2023-10-18T11:58:37.512416+0000","last_undegraded":"2023-10-18T11:58:37.512416+0000","last_fullsized":"2023-10-18T11:58:37.512416+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:26.184608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509045+0000","last_change":"2023-10-18T11:58:35.523280+0000","last_active":"2023-10-18T11:58:37.509045+0000","last_peered":"2023-10-18T11:58:37.509045+0000","last_clean":"2023-10-18T11:58:37.509045+0000","last_became_active":"2023-10-18T11:58:35.522621+0000","last_became_peered":"2023-10-18T11:58:35.522621+0000","last_unstale":"2023-10-18T11:58:37.509045+0000","last_undegraded":"2023-10-18T11:58:37.509045+0000","last_fullsized":"2023-10-18T11:58:37.509045+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:27.443746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802321+0000","last_change":"2023-10-18T11:58:36.516159+0000","last_active":"2023-10-18T11:58:37.802321+0000","last_peered":"2023-10-18T11:58:37.802321+0000","last_clean":"2023-10-18T11:58:37.802321+0000","last_became_active":"2023-10-18T11:58:36.515894+0000","last_became_peered":"2023-10-18T11:58:36.515894+0000","last_unstale":"2023-10-18T11:58:37.802321+0000","last_undegraded":"2023-10-18T11:58:37.802321+0000","last_fullsized":"2023-10-18T11:58:37.802321+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:42.645151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802627+0000","last_change":"2023-10-18T11:58:35.521126+0000","last_active":"2023-10-18T11:58:37.802627+0000","last_peered":"2023-10-18T11:58:37.802627+0000","last_clean":"2023-10-18T11:58:37.802627+0000","last_became_active":"2023-10-18T11:58:35.520861+0000","last_became_peered":"2023-10-18T11:58:35.520861+0000","last_unstale":"2023-10-18T11:58:37.802627+0000","last_undegraded":"2023-10-18T11:58:37.802627+0000","last_fullsized":"2023-10-18T11:58:37.802627+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802501+0000","last_change":"2023-10-18T11:58:36.531084+0000","last_active":"2023-10-18T11:58:37.802501+0000","last_peered":"2023-10-18T11:58:37.802501+0000","last_clean":"2023-10-18T11:58:37.802501+0000","last_became_active":"2023-10-18T11:58:36.530792+0000","last_became_peered":"2023-10-18T11:58:36.530792+0000","last_unstale":"2023-10-18T11:58:37.802501+0000","last_undegraded":"2023-10-18T11:58:37.802501+0000","last_fullsized":"2023-10-18T11:58:37.802501+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:37.483860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512533+0000","last_change":"2023-10-18T11:58:35.518246+0000","last_active":"2023-10-18T11:58:37.512533+0000","last_peered":"2023-10-18T11:58:37.512533+0000","last_clean":"2023-10-18T11:58:37.512533+0000","last_became_active":"2023-10-18T11:58:35.517986+0000","last_became_peered":"2023-10-18T11:58:35.517986+0000","last_unstale":"2023-10-18T11:58:37.512533+0000","last_undegraded":"2023-10-18T11:58:37.512533+0000","last_fullsized":"2023-10-18T11:58:37.512533+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:02:15.620821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803081+0000","last_change":"2023-10-18T11:58:36.533670+0000","last_active":"2023-10-18T11:58:37.803081+0000","last_peered":"2023-10-18T11:58:37.803081+0000","last_clean":"2023-10-18T11:58:37.803081+0000","last_became_active":"2023-10-18T11:58:36.533394+0000","last_became_peered":"2023-10-18T11:58:36.533394+0000","last_unstale":"2023-10-18T11:58:37.803081+0000","last_undegraded":"2023-10-18T11:58:37.803081+0000","last_fullsized":"2023-10-18T11:58:37.803081+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:21.396797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509229+0000","last_change":"2023-10-18T11:58:35.524709+0000","last_active":"2023-10-18T11:58:37.509229+0000","last_peered":"2023-10-18T11:58:37.509229+0000","last_clean":"2023-10-18T11:58:37.509229+0000","last_became_active":"2023-10-18T11:58:35.524471+0000","last_became_peered":"2023-10-18T11:58:35.524471+0000","last_unstale":"2023-10-18T11:58:37.509229+0000","last_undegraded":"2023-10-18T11:58:37.509229+0000","last_fullsized":"2023-10-18T11:58:37.509229+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:53.288380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509250+0000","last_change":"2023-10-18T11:58:36.523093+0000","last_active":"2023-10-18T11:58:37.509250+0000","last_peered":"2023-10-18T11:58:37.509250+0000","last_clean":"2023-10-18T11:58:37.509250+0000","last_became_active":"2023-10-18T11:58:36.522551+0000","last_became_peered":"2023-10-18T11:58:36.522551+0000","last_unstale":"2023-10-18T11:58:37.509250+0000","last_undegraded":"2023-10-18T11:58:37.509250+0000","last_fullsized":"2023-10-18T11:58:37.509250+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:22.715669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802085+0000","last_change":"2023-10-18T11:58:35.522093+0000","last_active":"2023-10-18T11:58:37.802085+0000","last_peered":"2023-10-18T11:58:37.802085+0000","last_clean":"2023-10-18T11:58:37.802085+0000","last_became_active":"2023-10-18T11:58:35.521805+0000","last_became_peered":"2023-10-18T11:58:35.521805+0000","last_unstale":"2023-10-18T11:58:37.802085+0000","last_undegraded":"2023-10-18T11:58:37.802085+0000","last_fullsized":"2023-10-18T11:58:37.802085+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:08.791931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513049+0000","last_change":"2023-10-18T11:58:36.531254+0000","last_active":"2023-10-18T11:58:37.513049+0000","last_peered":"2023-10-18T11:58:37.513049+0000","last_clean":"2023-10-18T11:58:37.513049+0000","last_became_active":"2023-10-18T11:58:36.530967+0000","last_became_peered":"2023-10-18T11:58:36.530967+0000","last_unstale":"2023-10-18T11:58:37.513049+0000","last_undegraded":"2023-10-18T11:58:37.513049+0000","last_fullsized":"2023-10-18T11:58:37.513049+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:01.297812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"203'3","reported_seq":15,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802710+0000","last_change":"2023-10-18T11:58:35.525207+0000","last_active":"2023-10-18T11:58:37.802710+0000","last_peered":"2023-10-18T11:58:37.802710+0000","last_clean":"2023-10-18T11:58:37.802710+0000","last_became_active":"2023-10-18T11:58:35.524946+0000","last_became_peered":"2023-10-18T11:58:35.524946+0000","last_unstale":"2023-10-18T11:58:37.802710+0000","last_undegraded":"2023-10-18T11:58:37.802710+0000","last_fullsized":"2023-10-18T11:58:37.802710+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802920+0000","last_change":"2023-10-18T11:58:36.516878+0000","last_active":"2023-10-18T11:58:37.802920+0000","last_peered":"2023-10-18T11:58:37.802920+0000","last_clean":"2023-10-18T11:58:37.802920+0000","last_became_active":"2023-10-18T11:58:36.516625+0000","last_became_peered":"2023-10-18T11:58:36.516625+0000","last_unstale":"2023-10-18T11:58:37.802920+0000","last_undegraded":"2023-10-18T11:58:37.802920+0000","last_fullsized":"2023-10-18T11:58:37.802920+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:59.525275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"203'6","reported_seq":18,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513026+0000","last_change":"2023-10-18T11:58:35.518030+0000","last_active":"2023-10-18T11:58:37.513026+0000","last_peered":"2023-10-18T11:58:37.513026+0000","last_clean":"2023-10-18T11:58:37.513026+0000","last_became_active":"2023-10-18T11:58:35.517747+0000","last_became_peered":"2023-10-18T11:58:35.517747+0000","last_unstale":"2023-10-18T11:58:37.513026+0000","last_undegraded":"2023-10-18T11:58:37.513026+0000","last_fullsized":"2023-10-18T11:58:37.513026+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.335660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801895+0000","last_change":"2023-10-18T11:58:36.524788+0000","last_active":"2023-10-18T11:58:37.801895+0000","last_peered":"2023-10-18T11:58:37.801895+0000","last_clean":"2023-10-18T11:58:37.801895+0000","last_became_active":"2023-10-18T11:58:36.524547+0000","last_became_peered":"2023-10-18T11:58:36.524547+0000","last_unstale":"2023-10-18T11:58:37.801895+0000","last_undegraded":"2023-10-18T11:58:37.801895+0000","last_fullsized":"2023-10-18T11:58:37.801895+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:02.661591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512833+0000","last_change":"2023-10-18T11:58:35.521655+0000","last_active":"2023-10-18T11:58:37.512833+0000","last_peered":"2023-10-18T11:58:37.512833+0000","last_clean":"2023-10-18T11:58:37.512833+0000","last_became_active":"2023-10-18T11:58:35.521430+0000","last_became_peered":"2023-10-18T11:58:35.521430+0000","last_unstale":"2023-10-18T11:58:37.512833+0000","last_undegraded":"2023-10-18T11:58:37.512833+0000","last_fullsized":"2023-10-18T11:58:37.512833+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:21.628472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508795+0000","last_change":"2023-10-18T11:58:36.523743+0000","last_active":"2023-10-18T11:58:37.508795+0000","last_peered":"2023-10-18T11:58:37.508795+0000","last_clean":"2023-10-18T11:58:37.508795+0000","last_became_active":"2023-10-18T11:58:36.523492+0000","last_became_peered":"2023-10-18T11:58:36.523492+0000","last_unstale":"2023-10-18T11:58:37.508795+0000","last_undegraded":"2023-10-18T11:58:37.508795+0000","last_fullsized":"2023-10-18T11:58:37.508795+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:42.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802579+0000","last_change":"2023-10-18T11:58:35.522746+0000","last_active":"2023-10-18T11:58:37.802579+0000","last_peered":"2023-10-18T11:58:37.802579+0000","last_clean":"2023-10-18T11:58:37.802579+0000","last_became_active":"2023-10-18T11:58:35.522486+0000","last_became_peered":"2023-10-18T11:58:35.522486+0000","last_unstale":"2023-10-18T11:58:37.802579+0000","last_undegraded":"2023-10-18T11:58:37.802579+0000","last_fullsized":"2023-10-18T11:58:37.802579+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:07.062822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512576+0000","last_change":"2023-10-18T11:58:36.533848+0000","last_active":"2023-10-18T11:58:37.512576+0000","last_peered":"2023-10-18T11:58:37.512576+0000","last_clean":"2023-10-18T11:58:37.512576+0000","last_became_active":"2023-10-18T11:58:36.533565+0000","last_became_peered":"2023-10-18T11:58:36.533565+0000","last_unstale":"2023-10-18T11:58:37.512576+0000","last_undegraded":"2023-10-18T11:58:37.512576+0000","last_fullsized":"2023-10-18T11:58:37.512576+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:48.249824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802427+0000","last_change":"2023-10-18T11:58:35.517587+0000","last_active":"2023-10-18T11:58:37.802427+0000","last_peered":"2023-10-18T11:58:37.802427+0000","last_clean":"2023-10-18T11:58:37.802427+0000","last_became_active":"2023-10-18T11:58:35.517322+0000","last_became_peered":"2023-10-18T11:58:35.517322+0000","last_unstale":"2023-10-18T11:58:37.802427+0000","last_undegraded":"2023-10-18T11:58:37.802427+0000","last_fullsized":"2023-10-18T11:58:37.802427+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:34.567188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512733+0000","last_change":"2023-10-18T11:58:36.531687+0000","last_active":"2023-10-18T11:58:37.512733+0000","last_peered":"2023-10-18T11:58:37.512733+0000","last_clean":"2023-10-18T11:58:37.512733+0000","last_became_active":"2023-10-18T11:58:36.531411+0000","last_became_peered":"2023-10-18T11:58:36.531411+0000","last_unstale":"2023-10-18T11:58:37.512733+0000","last_undegraded":"2023-10-18T11:58:37.512733+0000","last_fullsized":"2023-10-18T11:58:37.512733+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:04.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"203'8","reported_seq":20,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802771+0000","last_change":"2023-10-18T11:58:35.515506+0000","last_active":"2023-10-18T11:58:37.802771+0000","last_peered":"2023-10-18T11:58:37.802771+0000","last_clean":"2023-10-18T11:58:37.802771+0000","last_became_active":"2023-10-18T11:58:35.515198+0000","last_became_peered":"2023-10-18T11:58:35.515198+0000","last_unstale":"2023-10-18T11:58:37.802771+0000","last_undegraded":"2023-10-18T11:58:37.802771+0000","last_fullsized":"2023-10-18T11:58:37.802771+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.509931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802367+0000","last_change":"2023-10-18T11:58:36.515935+0000","last_active":"2023-10-18T11:58:37.802367+0000","last_peered":"2023-10-18T11:58:37.802367+0000","last_clean":"2023-10-18T11:58:37.802367+0000","last_became_active":"2023-10-18T11:58:36.515722+0000","last_became_peered":"2023-10-18T11:58:36.515722+0000","last_unstale":"2023-10-18T11:58:37.802367+0000","last_undegraded":"2023-10-18T11:58:37.802367+0000","last_fullsized":"2023-10-18T11:58:37.802367+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:58.998126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:35.515482+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:35.515185+0000","last_became_peered":"2023-10-18T11:58:35.515185+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.149129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509119+0000","last_change":"2023-10-18T11:58:36.534430+0000","last_active":"2023-10-18T11:58:37.509119+0000","last_peered":"2023-10-18T11:58:37.509119+0000","last_clean":"2023-10-18T11:58:37.509119+0000","last_became_active":"2023-10-18T11:58:36.534167+0000","last_became_peered":"2023-10-18T11:58:36.534167+0000","last_unstale":"2023-10-18T11:58:37.509119+0000","last_undegraded":"2023-10-18T11:58:37.509119+0000","last_fullsized":"2023-10-18T11:58:37.509119+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:44.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802606+0000","last_change":"2023-10-18T11:58:35.522154+0000","last_active":"2023-10-18T11:58:37.802606+0000","last_peered":"2023-10-18T11:58:37.802606+0000","last_clean":"2023-10-18T11:58:37.802606+0000","last_became_active":"2023-10-18T11:58:35.521902+0000","last_became_peered":"2023-10-18T11:58:35.521902+0000","last_unstale":"2023-10-18T11:58:37.802606+0000","last_undegraded":"2023-10-18T11:58:37.802606+0000","last_fullsized":"2023-10-18T11:58:37.802606+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:27.447038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803092+0000","last_change":"2023-10-18T11:58:36.529258+0000","last_active":"2023-10-18T11:58:37.803092+0000","last_peered":"2023-10-18T11:58:37.803092+0000","last_clean":"2023-10-18T11:58:37.803092+0000","last_became_active":"2023-10-18T11:58:36.528468+0000","last_became_peered":"2023-10-18T11:58:36.528468+0000","last_unstale":"2023-10-18T11:58:37.803092+0000","last_undegraded":"2023-10-18T11:58:37.803092+0000","last_fullsized":"2023-10-18T11:58:37.803092+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:58.035451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803037+0000","last_change":"2023-10-18T11:58:36.521908+0000","last_active":"2023-10-18T11:58:37.803037+0000","last_peered":"2023-10-18T11:58:37.803037+0000","last_clean":"2023-10-18T11:58:37.803037+0000","last_became_active":"2023-10-18T11:58:36.521381+0000","last_became_peered":"2023-10-18T11:58:36.521381+0000","last_unstale":"2023-10-18T11:58:37.803037+0000","last_undegraded":"2023-10-18T11:58:37.803037+0000","last_fullsized":"2023-10-18T11:58:37.803037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:18.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803110+0000","last_change":"2023-10-18T11:58:35.515862+0000","last_active":"2023-10-18T11:58:37.803110+0000","last_peered":"2023-10-18T11:58:37.803110+0000","last_clean":"2023-10-18T11:58:37.803110+0000","last_became_active":"2023-10-18T11:58:35.515521+0000","last_became_peered":"2023-10-18T11:58:35.515521+0000","last_unstale":"2023-10-18T11:58:37.803110+0000","last_undegraded":"2023-10-18T11:58:37.803110+0000","last_fullsized":"2023-10-18T11:58:37.803110+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:03.199289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801885+0000","last_change":"2023-10-18T11:58:36.520962+0000","last_active":"2023-10-18T11:58:37.801885+0000","last_peered":"2023-10-18T11:58:37.801885+0000","last_clean":"2023-10-18T11:58:37.801885+0000","last_became_active":"2023-10-18T11:58:36.520712+0000","last_became_peered":"2023-10-18T11:58:36.520712+0000","last_unstale":"2023-10-18T11:58:37.801885+0000","last_undegraded":"2023-10-18T11:58:37.801885+0000","last_fullsized":"2023-10-18T11:58:37.801885+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:46.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512865+0000","last_change":"2023-10-18T11:58:35.524483+0000","last_active":"2023-10-18T11:58:37.512865+0000","last_peered":"2023-10-18T11:58:37.512865+0000","last_clean":"2023-10-18T11:58:37.512865+0000","last_became_active":"2023-10-18T11:58:35.524231+0000","last_became_peered":"2023-10-18T11:58:35.524231+0000","last_unstale":"2023-10-18T11:58:37.512865+0000","last_undegraded":"2023-10-18T11:58:37.512865+0000","last_fullsized":"2023-10-18T11:58:37.512865+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:29:21.906133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.529745+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.529497+0000","last_became_peered":"2023-10-18T11:58:36.529497+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802708+0000","last_change":"2023-10-18T11:58:35.526709+0000","last_active":"2023-10-18T11:58:37.802708+0000","last_peered":"2023-10-18T11:58:37.802708+0000","last_clean":"2023-10-18T11:58:37.802708+0000","last_became_active":"2023-10-18T11:58:35.526448+0000","last_became_peered":"2023-10-18T11:58:35.526448+0000","last_unstale":"2023-10-18T11:58:37.802708+0000","last_undegraded":"2023-10-18T11:58:37.802708+0000","last_fullsized":"2023-10-18T11:58:37.802708+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:20.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802541+0000","last_change":"2023-10-18T11:58:36.528368+0000","last_active":"2023-10-18T11:58:37.802541+0000","last_peered":"2023-10-18T11:58:37.802541+0000","last_clean":"2023-10-18T11:58:37.802541+0000","last_became_active":"2023-10-18T11:58:36.528124+0000","last_became_peered":"2023-10-18T11:58:36.528124+0000","last_unstale":"2023-10-18T11:58:37.802541+0000","last_undegraded":"2023-10-18T11:58:37.802541+0000","last_fullsized":"2023-10-18T11:58:37.802541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:48.474419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802873+0000","last_change":"2023-10-18T11:58:35.525781+0000","last_active":"2023-10-18T11:58:37.802873+0000","last_peered":"2023-10-18T11:58:37.802873+0000","last_clean":"2023-10-18T11:58:37.802873+0000","last_became_active":"2023-10-18T11:58:35.525519+0000","last_became_peered":"2023-10-18T11:58:35.525519+0000","last_unstale":"2023-10-18T11:58:37.802873+0000","last_undegraded":"2023-10-18T11:58:37.802873+0000","last_fullsized":"2023-10-18T11:58:37.802873+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:47:37.957986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508969+0000","last_change":"2023-10-18T11:58:36.518933+0000","last_active":"2023-10-18T11:58:37.508969+0000","last_peered":"2023-10-18T11:58:37.508969+0000","last_clean":"2023-10-18T11:58:37.508969+0000","last_became_active":"2023-10-18T11:58:36.518691+0000","last_became_peered":"2023-10-18T11:58:36.518691+0000","last_unstale":"2023-10-18T11:58:37.508969+0000","last_undegraded":"2023-10-18T11:58:37.508969+0000","last_fullsized":"2023-10-18T11:58:37.508969+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:55.005558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802476+0000","last_change":"2023-10-18T11:58:36.522037+0000","last_active":"2023-10-18T11:58:37.802476+0000","last_peered":"2023-10-18T11:58:37.802476+0000","last_clean":"2023-10-18T11:58:37.802476+0000","last_became_active":"2023-10-18T11:58:36.521783+0000","last_became_peered":"2023-10-18T11:58:36.521783+0000","last_unstale":"2023-10-18T11:58:37.802476+0000","last_undegraded":"2023-10-18T11:58:37.802476+0000","last_fullsized":"2023-10-18T11:58:37.802476+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:37:11.613406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803083+0000","last_change":"2023-10-18T11:58:35.518886+0000","last_active":"2023-10-18T11:58:37.803083+0000","last_peered":"2023-10-18T11:58:37.803083+0000","last_clean":"2023-10-18T11:58:37.803083+0000","last_became_active":"2023-10-18T11:58:35.518591+0000","last_became_peered":"2023-10-18T11:58:35.518591+0000","last_unstale":"2023-10-18T11:58:37.803083+0000","last_undegraded":"2023-10-18T11:58:37.803083+0000","last_fullsized":"2023-10-18T11:58:37.803083+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:49.481190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802223+0000","last_change":"2023-10-18T11:58:36.519762+0000","last_active":"2023-10-18T11:58:37.802223+0000","last_peered":"2023-10-18T11:58:37.802223+0000","last_clean":"2023-10-18T11:58:37.802223+0000","last_became_active":"2023-10-18T11:58:36.517560+0000","last_became_peered":"2023-10-18T11:58:36.517560+0000","last_unstale":"2023-10-18T11:58:37.802223+0000","last_undegraded":"2023-10-18T11:58:37.802223+0000","last_fullsized":"2023-10-18T11:58:37.802223+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:29.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802645+0000","last_change":"2023-10-18T11:58:35.522798+0000","last_active":"2023-10-18T11:58:37.802645+0000","last_peered":"2023-10-18T11:58:37.802645+0000","last_clean":"2023-10-18T11:58:37.802645+0000","last_became_active":"2023-10-18T11:58:35.522532+0000","last_became_peered":"2023-10-18T11:58:35.522532+0000","last_unstale":"2023-10-18T11:58:37.802645+0000","last_undegraded":"2023-10-18T11:58:37.802645+0000","last_fullsized":"2023-10-18T11:58:37.802645+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:40:51.705544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:36.530354+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:36.530080+0000","last_became_peered":"2023-10-18T11:58:36.530080+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:19.995868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509241+0000","last_change":"2023-10-18T11:58:35.523100+0000","last_active":"2023-10-18T11:58:37.509241+0000","last_peered":"2023-10-18T11:58:37.509241+0000","last_clean":"2023-10-18T11:58:37.509241+0000","last_became_active":"2023-10-18T11:58:35.522802+0000","last_became_peered":"2023-10-18T11:58:35.522802+0000","last_unstale":"2023-10-18T11:58:37.509241+0000","last_undegraded":"2023-10-18T11:58:37.509241+0000","last_fullsized":"2023-10-18T11:58:37.509241+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802916+0000","last_change":"2023-10-18T11:58:36.522886+0000","last_active":"2023-10-18T11:58:37.802916+0000","last_peered":"2023-10-18T11:58:37.802916+0000","last_clean":"2023-10-18T11:58:37.802916+0000","last_became_active":"2023-10-18T11:58:36.522614+0000","last_became_peered":"2023-10-18T11:58:36.522614+0000","last_unstale":"2023-10-18T11:58:37.802916+0000","last_undegraded":"2023-10-18T11:58:37.802916+0000","last_fullsized":"2023-10-18T11:58:37.802916+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:52.212999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509088+0000","last_change":"2023-10-18T11:58:35.523409+0000","last_active":"2023-10-18T11:58:37.509088+0000","last_peered":"2023-10-18T11:58:37.509088+0000","last_clean":"2023-10-18T11:58:37.509088+0000","last_became_active":"2023-10-18T11:58:35.523146+0000","last_became_peered":"2023-10-18T11:58:35.523146+0000","last_unstale":"2023-10-18T11:58:37.509088+0000","last_undegraded":"2023-10-18T11:58:37.509088+0000","last_fullsized":"2023-10-18T11:58:37.509088+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:51.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802959+0000","last_change":"2023-10-18T11:58:35.522529+0000","last_active":"2023-10-18T11:58:37.802959+0000","last_peered":"2023-10-18T11:58:37.802959+0000","last_clean":"2023-10-18T11:58:37.802959+0000","last_became_active":"2023-10-18T11:58:35.522261+0000","last_became_peered":"2023-10-18T11:58:35.522261+0000","last_unstale":"2023-10-18T11:58:37.802959+0000","last_undegraded":"2023-10-18T11:58:37.802959+0000","last_fullsized":"2023-10-18T11:58:37.802959+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512534+0000","last_change":"2023-10-18T11:58:36.530248+0000","last_active":"2023-10-18T11:58:37.512534+0000","last_peered":"2023-10-18T11:58:37.512534+0000","last_clean":"2023-10-18T11:58:37.512534+0000","last_became_active":"2023-10-18T11:58:36.529997+0000","last_became_peered":"2023-10-18T11:58:36.529997+0000","last_unstale":"2023-10-18T11:58:37.512534+0000","last_undegraded":"2023-10-18T11:58:37.512534+0000","last_fullsized":"2023-10-18T11:58:37.512534+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:58.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802154+0000","last_change":"2023-10-18T11:58:35.527290+0000","last_active":"2023-10-18T11:58:37.802154+0000","last_peered":"2023-10-18T11:58:37.802154+0000","last_clean":"2023-10-18T11:58:37.802154+0000","last_became_active":"2023-10-18T11:58:35.527052+0000","last_became_peered":"2023-10-18T11:58:35.527052+0000","last_unstale":"2023-10-18T11:58:37.802154+0000","last_undegraded":"2023-10-18T11:58:37.802154+0000","last_fullsized":"2023-10-18T11:58:37.802154+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802594+0000","last_change":"2023-10-18T11:58:36.520551+0000","last_active":"2023-10-18T11:58:37.802594+0000","last_peered":"2023-10-18T11:58:37.802594+0000","last_clean":"2023-10-18T11:58:37.802594+0000","last_became_active":"2023-10-18T11:58:36.520297+0000","last_became_peered":"2023-10-18T11:58:36.520297+0000","last_unstale":"2023-10-18T11:58:37.802594+0000","last_undegraded":"2023-10-18T11:58:37.802594+0000","last_fullsized":"2023-10-18T11:58:37.802594+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:25.536175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802945+0000","last_change":"2023-10-18T11:58:35.526011+0000","last_active":"2023-10-18T11:58:37.802945+0000","last_peered":"2023-10-18T11:58:37.802945+0000","last_clean":"2023-10-18T11:58:37.802945+0000","last_became_active":"2023-10-18T11:58:35.525734+0000","last_became_peered":"2023-10-18T11:58:35.525734+0000","last_unstale":"2023-10-18T11:58:37.802945+0000","last_undegraded":"2023-10-18T11:58:37.802945+0000","last_fullsized":"2023-10-18T11:58:37.802945+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:19.352633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802222+0000","last_change":"2023-10-18T11:58:36.528280+0000","last_active":"2023-10-18T11:58:37.802222+0000","last_peered":"2023-10-18T11:58:37.802222+0000","last_clean":"2023-10-18T11:58:37.802222+0000","last_became_active":"2023-10-18T11:58:36.528012+0000","last_became_peered":"2023-10-18T11:58:36.528012+0000","last_unstale":"2023-10-18T11:58:37.802222+0000","last_undegraded":"2023-10-18T11:58:37.802222+0000","last_fullsized":"2023-10-18T11:58:37.802222+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:28.449393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801952+0000","last_change":"2023-10-18T11:58:35.513693+0000","last_active":"2023-10-18T11:58:37.801952+0000","last_peered":"2023-10-18T11:58:37.801952+0000","last_clean":"2023-10-18T11:58:37.801952+0000","last_became_active":"2023-10-18T11:58:35.513429+0000","last_became_peered":"2023-10-18T11:58:35.513429+0000","last_unstale":"2023-10-18T11:58:37.801952+0000","last_undegraded":"2023-10-18T11:58:37.801952+0000","last_fullsized":"2023-10-18T11:58:37.801952+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:12.442948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"193'335","reported_seq":1140,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-10-18T11:58:36.518008+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:58:36.518008+0000","last_peered":"2023-10-18T11:58:36.518008+0000","last_clean":"2023-10-18T11:58:36.518008+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:58:36.518008+0000","last_undegraded":"2023-10-18T11:58:36.518008+0000","last_fullsized":"2023-10-18T11:58:36.518008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":120,"acting":120,"num_store_stats":7},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52308,"kb_used_data":3080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319532,"statfs":{"total":96636764160,"available":96583200768,"internally_reserved":0,"allocated":3153920,"data_stored":1895422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.363},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74399999999999999,"5min":0.77200000000000002,"15min":0.84099999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.39700000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":1.298},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.82699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.35699999999999998},"max":{"1min":0.80200000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45200000000000001,"15min":0.44900000000000001},"min":{"1min":0.34000000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.80200000000000005,"15min":1.5249999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.27700000000000002},"max":{"1min":0.63500000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.78900000000000003,"15min":0.90800000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.375,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":0.82699999999999996,"15min":1.4179999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":1.478},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.874},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37768,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334072,"statfs":{"total":96636764160,"available":96598089728,"internally_reserved":0,"allocated":2486272,"data_stored":1267517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.50700000000000001,"5min":0.41899999999999998,"15min":0.29899999999999999},"max":{"1min":0.78100000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.27900000000000003},"max":{"1min":0.76700000000000002,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.87},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.625,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.25600000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.82599999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.626},"min":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47299999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.76900000000000002},"last":0.37},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.71999999999999997,"15min":0.82999999999999996},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.81200000000000006},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72699999999999998,"5min":0.80600000000000005,"15min":0.84599999999999997},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575970,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60068,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311772,"statfs":{"total":96636764160,"available":96575254528,"internally_reserved":0,"allocated":2576384,"data_stored":1306688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.249},"max":{"1min":0.84099999999999997,"5min":0.90300000000000002,"15min":0.90800000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.50700000000000001,"5min":0.38500000000000001,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.95899999999999996,"15min":0.96099999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.24299999999999999},"max":{"1min":1.081,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82199999999999995,"15min":0.754},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.28399999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.98699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.84999999999999998,"15min":0.98399999999999999},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.53200000000000003,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.0349999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.501,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":1,"15min":1},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.30299999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.98499999999999999,"15min":1.01},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.93899999999999995,"15min":0.97999999999999998},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83699999999999997,"5min":1.101,"15min":1.101},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.34000000000000002},"max":{"1min":0.83399999999999996,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575970,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29544,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342296,"statfs":{"total":96636764160,"available":96606511104,"internally_reserved":0,"allocated":2576384,"data_stored":1306700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.62},"min":{"1min":0.45400000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80700000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.80200000000000005,"15min":0.85399999999999998},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69499999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.372,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.752,"5min":1.123,"15min":1.123},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.92100000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.93600000000000005},"last":0.629}]},{"osd":3,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.312},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.34599999999999997,"15min":0.31900000000000001},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":2568192,"data_stored":1305214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.221},"max":{"1min":0.57099999999999995,"5min":0.61599999999999999,"15min":0.63300000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.28100000000000003,"15min":0.23400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.374}]},{"osd":1,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26400000000000001},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.56000000000000005,"5min":0.60599999999999998,"15min":0.65000000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.73399999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.052},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47499999999999998,"15min":0.40999999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.026},"last":0.625}]},{"osd":6,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.437},"max":{"1min":0.82299999999999995,"5min":0.93100000000000005,"15min":0.94499999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.68700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.45400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.55600000000000005,"5min":0.46400000000000002,"15min":0.442},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.58099999999999996,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.67800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60060,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311780,"statfs":{"total":96636764160,"available":96575262720,"internally_reserved":0,"allocated":2564096,"data_stored":1305636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.252,"5min":0.222,"15min":0.216},"max":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.67200000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.27900000000000003,"5min":0.26200000000000001,"15min":0.25800000000000001},"max":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.746,"15min":0.746},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.435,"15min":0.432},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.374},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35399999999999998,"5min":0.29899999999999999,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.33600000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.499,"5min":0.442,"15min":0.437},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.47799999999999998,"15min":0.41699999999999998},"max":{"1min":0.80900000000000005,"5min":1.087,"15min":1.087},"last":0.435}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.54500000000000004,"5min":0.46500000000000002,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66600000000000004,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.45200000000000001,"15min":0.40999999999999998},"max":{"1min":0.85799999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68500000000000005,"15min":0.68200000000000005},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.45100000000000001},"max":{"1min":0.84099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.443,"15min":0.42499999999999999},"max":{"1min":0.86799999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.53900000000000003,"5min":0.443,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.01},"last":0.499}]}]},{"osd":1,"up_from":13,"seq":55834575970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60048,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311792,"statfs":{"total":96636764160,"available":96575275008,"internally_reserved":0,"allocated":2560000,"data_stored":1305054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.182},"max":{"1min":0.73899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65800000000000003,"5min":0.81000000000000005,"15min":0.85299999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.23000000000000001},"max":{"1min":0.496,"5min":0.877,"15min":0.877},"last":0.28000000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.69799999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.377},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51400000000000001},"min":{"1min":0.28899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.74399999999999999,"5min":0.79800000000000004,"15min":0.91000000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.74099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.65600000000000003},"min":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.29599999999999999},"max":{"1min":0.80300000000000005,"5min":0.86599999999999999,"15min":1.2390000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.316},"max":{"1min":0.70299999999999996,"5min":1.05,"15min":1.212},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":1.0209999999999999,"15min":1.0589999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77600000000000002,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.47399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82499999999999996,"5min":0.88400000000000001,"15min":1.0620000000000001},"last":0.69899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42600,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329240,"statfs":{"total":96636764160,"available":96593141760,"internally_reserved":0,"allocated":3170304,"data_stored":1896150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.57199999999999995,"5min":0.64400000000000002,"15min":0.66200000000000003},"last":0.247}]},{"osd":2,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.33700000000000002,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.433,"15min":0.42799999999999999},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.25700000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.36699999999999999},"max":{"1min":0.876,"5min":0.96999999999999997,"15min":0.997},"last":0.495},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.50900000000000001,"5min":0.45400000000000001,"15min":0.39100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":0.93999999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.504,"5min":0.439,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.95799999999999996,"15min":0.996},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.44700000000000001},"max":{"1min":0.89300000000000002,"5min":1.0089999999999999,"15min":1.038},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.48599999999999999,"5min":0.44900000000000001,"15min":0.44800000000000001},"max":{"1min":0.91100000000000003,"5min":0.95099999999999996,"15min":1.0609999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":1.0109999999999999,"15min":1.0189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.42999999999999999},"max":{"1min":0.89400000000000002,"5min":0.997,"15min":0.997},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:58:40.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3d is not active+clean 2023-10-18T11:58:40.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:58:35.499419+0000', 'last_change': '2023-10-18T11:58:35.499419+0000', 'last_active': '2023-10-18T11:58:35.499419+0000', 'last_peered': '2023-10-18T11:58:35.499419+0000', 'last_clean': '2023-10-18T11:58:35.499419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:58:35.499419+0000', 'last_undegraded': '2023-10-18T11:58:35.499419+0000', 'last_fullsized': '2023-10-18T11:58:35.499419+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'last_clean_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:58:40.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1f is not active+clean 2023-10-18T11:58:40.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:58:35.499419+0000', 'last_change': '2023-10-18T11:58:35.499419+0000', 'last_active': '2023-10-18T11:58:35.499419+0000', 'last_peered': '2023-10-18T11:58:35.499419+0000', 'last_clean': '2023-10-18T11:58:35.499419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:58:35.499419+0000', 'last_undegraded': '2023-10-18T11:58:35.499419+0000', 'last_fullsized': '2023-10-18T11:58:35.499419+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'last_clean_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:58:40.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.d is not active+clean 2023-10-18T11:58:40.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:58:34.497064+0000', 'last_change': '2023-10-18T11:58:34.497064+0000', 'last_active': '2023-10-18T11:58:34.497064+0000', 'last_peered': '2023-10-18T11:58:34.497064+0000', 'last_clean': '2023-10-18T11:58:34.497064+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:58:34.497064+0000', 'last_undegraded': '2023-10-18T11:58:34.497064+0000', 'last_fullsized': '2023-10-18T11:58:34.497064+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'last_clean_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:58:40.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.8 is not active+clean 2023-10-18T11:58:40.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:58:34.497064+0000', 'last_change': '2023-10-18T11:58:34.497064+0000', 'last_active': '2023-10-18T11:58:34.497064+0000', 'last_peered': '2023-10-18T11:58:34.497064+0000', 'last_clean': '2023-10-18T11:58:34.497064+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:58:34.497064+0000', 'last_undegraded': '2023-10-18T11:58:34.497064+0000', 'last_fullsized': '2023-10-18T11:58:34.497064+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'last_clean_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:58:40.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.0 is not active+clean 2023-10-18T11:58:40.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:58:35.499419+0000', 'last_change': '2023-10-18T11:58:35.499419+0000', 'last_active': '2023-10-18T11:58:35.499419+0000', 'last_peered': '2023-10-18T11:58:35.499419+0000', 'last_clean': '2023-10-18T11:58:35.499419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:58:35.499419+0000', 'last_undegraded': '2023-10-18T11:58:35.499419+0000', 'last_fullsized': '2023-10-18T11:58:35.499419+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'last_clean_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:58:40.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.0 is not active+clean 2023-10-18T11:58:40.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:58:34.497064+0000', 'last_change': '2023-10-18T11:58:34.497064+0000', 'last_active': '2023-10-18T11:58:34.497064+0000', 'last_peered': '2023-10-18T11:58:34.497064+0000', 'last_clean': '2023-10-18T11:58:34.497064+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:58:34.497064+0000', 'last_undegraded': '2023-10-18T11:58:34.497064+0000', 'last_fullsized': '2023-10-18T11:58:34.497064+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'last_clean_scrub_stamp': '2023-10-18T11:58:34.497064+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:58:40.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1b is not active+clean 2023-10-18T11:58:40.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T11:58:35.499419+0000', 'last_change': '2023-10-18T11:58:35.499419+0000', 'last_active': '2023-10-18T11:58:35.499419+0000', 'last_peered': '2023-10-18T11:58:35.499419+0000', 'last_clean': '2023-10-18T11:58:35.499419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T11:58:35.499419+0000', 'last_undegraded': '2023-10-18T11:58:35.499419+0000', 'last_fullsized': '2023-10-18T11:58:35.499419+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'last_clean_scrub_stamp': '2023-10-18T11:58:35.499419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T11:58:40.141 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T11:58:40.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.268+0000 7ff96abc0640 1 -- 172.21.15.125:0/3181494665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff94c003970 msgr2=0x7ff94c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.268+0000 7ff96abc0640 1 --2- 172.21.15.125:0/3181494665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff94c003970 0x7ff94c003d90 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7ff95400baf0 tx=0x7ff94c0057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.268+0000 7ff96abc0640 1 -- 172.21.15.125:0/3181494665 shutdown_connections 2023-10-18T11:58:40.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.268+0000 7ff96abc0640 1 --2- 172.21.15.125:0/3181494665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff94c003970 0x7ff94c003d90 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.268+0000 7ff96abc0640 1 -- 172.21.15.125:0/3181494665 >> 172.21.15.125:0/3181494665 conn(0x7ff9640d0d60 msgr2=0x7ff9640d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.268+0000 7ff96abc0640 1 -- 172.21.15.125:0/3181494665 shutdown_connections 2023-10-18T11:58:40.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.268+0000 7ff96abc0640 1 -- 172.21.15.125:0/3181494665 wait complete. 2023-10-18T11:58:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.269+0000 7ff96abc0640 1 Processor -- start 2023-10-18T11:58:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.269+0000 7ff96abc0640 1 -- start start 2023-10-18T11:58:40.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96abc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff94c003970 0x7ff9641554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96abc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9640b53d0 0x7ff9640b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96abc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9640c9e40 0x7ff9640ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96abc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640ca800 con 0x7ff9640c9e40 2023-10-18T11:58:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96abc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96415ba50 con 0x7ff9640b53d0 2023-10-18T11:58:40.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96abc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96415b720 con 0x7ff94c003970 2023-10-18T11:58:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9640c9e40 0x7ff9640ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9640c9e40 0x7ff9640ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50034/0 (socket says 172.21.15.125:50034) 2023-10-18T11:58:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 -- 172.21.15.125:0/2929729753 learned_addr learned my addr 172.21.15.125:0/2929729753 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96895c640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff94c003970 0x7ff9641554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:40.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff963fff640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9640b53d0 0x7ff9640b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 -- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff94c003970 msgr2=0x7ff9641554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff94c003970 0x7ff9641554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 -- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9640b53d0 msgr2=0x7ff9640b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9640b53d0 0x7ff9640b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 -- 172.21.15.125:0/2929729753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9640b61b0 con 0x7ff9640c9e40 2023-10-18T11:58:40.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96895c640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff94c003970 0x7ff9641554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:40.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff963fff640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9640b53d0 0x7ff9640b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:40.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96915d640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9640c9e40 0x7ff9640ca2c0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7ff9640b6e10 tx=0x7ff95c0069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff9617fa640 1 -- 172.21.15.125:0/2929729753 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0107d0 con 0x7ff9640c9e40 2023-10-18T11:58:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff9617fa640 1 -- 172.21.15.125:0/2929729753 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff95c010950 con 0x7ff9640c9e40 2023-10-18T11:58:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9640b6490 con 0x7ff9640c9e40 2023-10-18T11:58:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff9617fa640 1 -- 172.21.15.125:0/2929729753 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c010d60 con 0x7ff9640c9e40 2023-10-18T11:58:40.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.270+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9640b6770 con 0x7ff9640c9e40 2023-10-18T11:58:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.272+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9640690e0 con 0x7ff9640c9e40 2023-10-18T11:58:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.272+0000 7ff9617fa640 1 -- 172.21.15.125:0/2929729753 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff95c01a400 con 0x7ff9640c9e40 2023-10-18T11:58:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.272+0000 7ff9617fa640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7ff95006af90 0x7ff95006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:40.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.272+0000 7ff9617fa640 1 -- 172.21.15.125:0/2929729753 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7ff95c089f30 con 0x7ff9640c9e40 2023-10-18T11:58:40.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.272+0000 7ff96895c640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7ff95006af90 0x7ff95006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:40.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.273+0000 7ff96895c640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7ff95006af90 0x7ff95006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff954037000 tx=0x7ff95402e040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:40.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.276+0000 7ff9617fa640 1 -- 172.21.15.125:0/2929729753 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff95c054cd0 con 0x7ff9640c9e40 2023-10-18T11:58:40.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.511+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff9640d0b40 con 0x7ff9640c9e40 2023-10-18T11:58:40.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.512+0000 7ff9617fa640 1 -- 172.21.15.125:0/2929729753 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1094 (secure 0 0 0) 0x7ff95c0582f0 con 0x7ff9640c9e40 2023-10-18T11:58:40.515 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:40.515 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5642,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":204,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":90},{"state_name":"unknown","count":7}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592708,"bytes_used":412110848,"bytes_avail":772682002432,"bytes_total":773094113280,"unknown_pgs_ratio":0.072164945304393768,"write_bytes_sec":3029,"read_op_per_sec":0,"write_op_per_sec":9},"fsmap":{"epoch":80,"id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"d","status":"up:active","gid":11439}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2023-10-18T11:54:21.685816+0000","services":{}},"progress_events":{}} 2023-10-18T11:58:40.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7ff95006af90 msgr2=0x7ff95006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7ff95006af90 0x7ff95006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff954037000 tx=0x7ff95402e040 comp rx=0 tx=0).stop 2023-10-18T11:58:40.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9640c9e40 msgr2=0x7ff9640ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9640c9e40 0x7ff9640ca2c0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7ff9640b6e10 tx=0x7ff95c0069b0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 shutdown_connections 2023-10-18T11:58:40.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7ff95006af90 0x7ff95006d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9640b53d0 0x7ff9640b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9640c9e40 0x7ff9640ca2c0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 --2- 172.21.15.125:0/2929729753 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff94c003970 0x7ff9641554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 >> 172.21.15.125:0/2929729753 conn(0x7ff9640d0d60 msgr2=0x7ff964069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:40.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 shutdown_connections 2023-10-18T11:58:40.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.514+0000 7ff96abc0640 1 -- 172.21.15.125:0/2929729753 wait complete. 2023-10-18T11:58:40.527 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T11:58:40.527 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:58:40.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.627+0000 7fcc9584f640 1 -- 172.21.15.125:0/1025811531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900b0530 msgr2=0x7fcc900b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.627+0000 7fcc9584f640 1 --2- 172.21.15.125:0/1025811531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900b0530 0x7fcc900b09c0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005fd0 tx=0x7fcc8802bd90 comp rx=0 tx=0).stop 2023-10-18T11:58:40.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.627+0000 7fcc9584f640 1 -- 172.21.15.125:0/1025811531 shutdown_connections 2023-10-18T11:58:40.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.627+0000 7fcc9584f640 1 --2- 172.21.15.125:0/1025811531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900b0530 0x7fcc900b09c0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.627+0000 7fcc9584f640 1 --2- 172.21.15.125:0/1025811531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900afa20 0x7fcc90158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.627+0000 7fcc9584f640 1 -- 172.21.15.125:0/1025811531 >> 172.21.15.125:0/1025811531 conn(0x7fcc90069930 msgr2=0x7fcc90069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:40.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.627+0000 7fcc9584f640 1 -- 172.21.15.125:0/1025811531 shutdown_connections 2023-10-18T11:58:40.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.627+0000 7fcc9584f640 1 -- 172.21.15.125:0/1025811531 wait complete. 2023-10-18T11:58:40.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.628+0000 7fcc9584f640 1 Processor -- start 2023-10-18T11:58:40.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.628+0000 7fcc9584f640 1 -- start start 2023-10-18T11:58:40.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc9584f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900afa20 0x7fcc90147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:40.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc9584f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900b0530 0x7fcc9014d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:40.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc9584f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900cbd20 0x7fcc901f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:40.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc9584f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90158d90 con 0x7fcc900afa20 2023-10-18T11:58:40.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc9584f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900abcc0 con 0x7fcc900cbd20 2023-10-18T11:58:40.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc9584f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900abfa0 con 0x7fcc900b0530 2023-10-18T11:58:40.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc8effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900afa20 0x7fcc90147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:40.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc8effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900afa20 0x7fcc90147ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50056/0 (socket says 172.21.15.125:50056) 2023-10-18T11:58:40.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc8effd640 1 -- 172.21.15.125:0/3329552506 learned_addr learned my addr 172.21.15.125:0/3329552506 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:40.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc8e7fc640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900b0530 0x7fcc9014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:40.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.629+0000 7fcc8f7fe640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900cbd20 0x7fcc901f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:40.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc8effd640 1 -- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900b0530 msgr2=0x7fcc9014d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc8effd640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900b0530 0x7fcc9014d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc8effd640 1 -- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900cbd20 msgr2=0x7fcc901f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc8effd640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900cbd20 0x7fcc901f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc8effd640 1 -- 172.21.15.125:0/3329552506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88009560 con 0x7fcc900afa20 2023-10-18T11:58:40.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc8effd640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900afa20 0x7fcc90147ca0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000ba50 tx=0x7fcc8000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:40.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc77fff640 1 -- 172.21.15.125:0/3329552506 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8000cbe0 con 0x7fcc900afa20 2023-10-18T11:58:40.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc77fff640 1 -- 172.21.15.125:0/3329552506 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcc8000ce70 con 0x7fcc900afa20 2023-10-18T11:58:40.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc900ac280 con 0x7fcc900afa20 2023-10-18T11:58:40.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc77fff640 1 -- 172.21.15.125:0/3329552506 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80014680 con 0x7fcc900afa20 2023-10-18T11:58:40.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.630+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc901483c0 con 0x7fcc900afa20 2023-10-18T11:58:40.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.631+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc90151780 con 0x7fcc900afa20 2023-10-18T11:58:40.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.631+0000 7fcc77fff640 1 -- 172.21.15.125:0/3329552506 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcc800147e0 con 0x7fcc900afa20 2023-10-18T11:58:40.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.632+0000 7fcc77fff640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcc7c06af60 0x7fcc7c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:40.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.632+0000 7fcc77fff640 1 -- 172.21.15.125:0/3329552506 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fcc80089a40 con 0x7fcc900afa20 2023-10-18T11:58:40.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.632+0000 7fcc8e7fc640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcc7c06af60 0x7fcc7c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:40.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.632+0000 7fcc8e7fc640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcc7c06af60 0x7fcc7c06d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcc90148a70 tx=0x7fcc8400b390 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:40.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.636+0000 7fcc77fff640 1 -- 172.21.15.125:0/3329552506 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc8008be60 con 0x7fcc900afa20 2023-10-18T11:58:40.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.798+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 --> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc900cc100 con 0x7fcc7c06af60 2023-10-18T11:58:40.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.806+0000 7fcc77fff640 1 -- 172.21.15.125:0/3329552506 <== mgr.9529 v2:172.21.15.139:6832/1565260611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288588 (secure 0 0 0) 0x7fcc84015070 con 0x7fcc7c06af60 2023-10-18T11:58:40.811 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:40.813 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:58:40.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcc7c06af60 msgr2=0x7fcc7c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcc7c06af60 0x7fcc7c06d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcc90148a70 tx=0x7fcc8400b390 comp rx=0 tx=0).stop 2023-10-18T11:58:40.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900afa20 msgr2=0x7fcc90147ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:40.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900afa20 0x7fcc90147ca0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000ba50 tx=0x7fcc8000bf10 comp rx=0 tx=0).stop 2023-10-18T11:58:40.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 shutdown_connections 2023-10-18T11:58:40.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fcc7c06af60 0x7fcc7c06d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900cbd20 0x7fcc901f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900b0530 0x7fcc9014d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 --2- 172.21.15.125:0/3329552506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcc900afa20 0x7fcc90147ca0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:40.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 >> 172.21.15.125:0/3329552506 conn(0x7fcc90069930 msgr2=0x7fcc90151070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:40.816 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 shutdown_connections 2023-10-18T11:58:40.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:40.809+0000 7fcc9584f640 1 -- 172.21.15.125:0/3329552506 wait complete. 2023-10-18T11:58:40.822 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T11:58:38.893982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":598,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":372,"ondisk_log_size":372,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":402452,"kb_used_data":21148,"kb_used_omap":12,"kb_used_meta":381171,"kb_avail":754572268,"statfs":{"total":773094113280,"available":772682002432,"internally_reserved":0,"allocated":21655552,"data_stored":11588381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":390319672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.394160"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802702+0000","last_change":"2023-10-18T11:58:36.522565+0000","last_active":"2023-10-18T11:58:37.802702+0000","last_peered":"2023-10-18T11:58:37.802702+0000","last_clean":"2023-10-18T11:58:37.802702+0000","last_became_active":"2023-10-18T11:58:36.521736+0000","last_became_peered":"2023-10-18T11:58:36.521736+0000","last_unstale":"2023-10-18T11:58:37.802702+0000","last_undegraded":"2023-10-18T11:58:37.802702+0000","last_fullsized":"2023-10-18T11:58:37.802702+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:46:36.674788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802006+0000","last_change":"2023-10-18T11:58:36.527651+0000","last_active":"2023-10-18T11:58:37.802006+0000","last_peered":"2023-10-18T11:58:37.802006+0000","last_clean":"2023-10-18T11:58:37.802006+0000","last_became_active":"2023-10-18T11:58:36.527375+0000","last_became_peered":"2023-10-18T11:58:36.527375+0000","last_unstale":"2023-10-18T11:58:37.802006+0000","last_undegraded":"2023-10-18T11:58:37.802006+0000","last_fullsized":"2023-10-18T11:58:37.802006+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:55:29.683103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.534333+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.534066+0000","last_became_peered":"2023-10-18T11:58:36.534066+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:29.879395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508872+0000","last_change":"2023-10-18T11:58:36.529615+0000","last_active":"2023-10-18T11:58:37.508872+0000","last_peered":"2023-10-18T11:58:37.508872+0000","last_clean":"2023-10-18T11:58:37.508872+0000","last_became_active":"2023-10-18T11:58:36.529380+0000","last_became_peered":"2023-10-18T11:58:36.529380+0000","last_unstale":"2023-10-18T11:58:37.508872+0000","last_undegraded":"2023-10-18T11:58:37.508872+0000","last_fullsized":"2023-10-18T11:58:37.508872+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:44.701593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508825+0000","last_change":"2023-10-18T11:58:36.528591+0000","last_active":"2023-10-18T11:58:37.508825+0000","last_peered":"2023-10-18T11:58:37.508825+0000","last_clean":"2023-10-18T11:58:37.508825+0000","last_became_active":"2023-10-18T11:58:36.528332+0000","last_became_peered":"2023-10-18T11:58:36.528332+0000","last_unstale":"2023-10-18T11:58:37.508825+0000","last_undegraded":"2023-10-18T11:58:37.508825+0000","last_fullsized":"2023-10-18T11:58:37.508825+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:04.968822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802608+0000","last_change":"2023-10-18T11:58:36.526452+0000","last_active":"2023-10-18T11:58:37.802608+0000","last_peered":"2023-10-18T11:58:37.802608+0000","last_clean":"2023-10-18T11:58:37.802608+0000","last_became_active":"2023-10-18T11:58:36.526144+0000","last_became_peered":"2023-10-18T11:58:36.526144+0000","last_unstale":"2023-10-18T11:58:37.802608+0000","last_undegraded":"2023-10-18T11:58:37.802608+0000","last_fullsized":"2023-10-18T11:58:37.802608+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:15.609770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802428+0000","last_change":"2023-10-18T11:58:36.526263+0000","last_active":"2023-10-18T11:58:37.802428+0000","last_peered":"2023-10-18T11:58:37.802428+0000","last_clean":"2023-10-18T11:58:37.802428+0000","last_became_active":"2023-10-18T11:58:36.525882+0000","last_became_peered":"2023-10-18T11:58:36.525882+0000","last_unstale":"2023-10-18T11:58:37.802428+0000","last_undegraded":"2023-10-18T11:58:37.802428+0000","last_fullsized":"2023-10-18T11:58:37.802428+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:05.910088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802812+0000","last_change":"2023-10-18T11:58:36.520303+0000","last_active":"2023-10-18T11:58:37.802812+0000","last_peered":"2023-10-18T11:58:37.802812+0000","last_clean":"2023-10-18T11:58:37.802812+0000","last_became_active":"2023-10-18T11:58:36.520065+0000","last_became_peered":"2023-10-18T11:58:36.520065+0000","last_unstale":"2023-10-18T11:58:37.802812+0000","last_undegraded":"2023-10-18T11:58:37.802812+0000","last_fullsized":"2023-10-18T11:58:37.802812+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:40.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509204+0000","last_change":"2023-10-18T11:58:36.534318+0000","last_active":"2023-10-18T11:58:37.509204+0000","last_peered":"2023-10-18T11:58:37.509204+0000","last_clean":"2023-10-18T11:58:37.509204+0000","last_became_active":"2023-10-18T11:58:36.534071+0000","last_became_peered":"2023-10-18T11:58:36.534071+0000","last_unstale":"2023-10-18T11:58:37.509204+0000","last_undegraded":"2023-10-18T11:58:37.509204+0000","last_fullsized":"2023-10-18T11:58:37.509204+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:55.491621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802310+0000","last_change":"2023-10-18T11:58:36.520313+0000","last_active":"2023-10-18T11:58:37.802310+0000","last_peered":"2023-10-18T11:58:37.802310+0000","last_clean":"2023-10-18T11:58:37.802310+0000","last_became_active":"2023-10-18T11:58:36.520042+0000","last_became_peered":"2023-10-18T11:58:36.520042+0000","last_unstale":"2023-10-18T11:58:37.802310+0000","last_undegraded":"2023-10-18T11:58:37.802310+0000","last_fullsized":"2023-10-18T11:58:37.802310+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:57.623658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802462+0000","last_change":"2023-10-18T11:58:36.526211+0000","last_active":"2023-10-18T11:58:37.802462+0000","last_peered":"2023-10-18T11:58:37.802462+0000","last_clean":"2023-10-18T11:58:37.802462+0000","last_became_active":"2023-10-18T11:58:36.525812+0000","last_became_peered":"2023-10-18T11:58:36.525812+0000","last_unstale":"2023-10-18T11:58:37.802462+0000","last_undegraded":"2023-10-18T11:58:37.802462+0000","last_fullsized":"2023-10-18T11:58:37.802462+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:39.938700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512711+0000","last_change":"2023-10-18T11:58:36.529732+0000","last_active":"2023-10-18T11:58:37.512711+0000","last_peered":"2023-10-18T11:58:37.512711+0000","last_clean":"2023-10-18T11:58:37.512711+0000","last_became_active":"2023-10-18T11:58:36.529455+0000","last_became_peered":"2023-10-18T11:58:36.529455+0000","last_unstale":"2023-10-18T11:58:37.512711+0000","last_undegraded":"2023-10-18T11:58:37.512711+0000","last_fullsized":"2023-10-18T11:58:37.512711+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:38.419107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802037+0000","last_change":"2023-10-18T11:58:36.534249+0000","last_active":"2023-10-18T11:58:37.802037+0000","last_peered":"2023-10-18T11:58:37.802037+0000","last_clean":"2023-10-18T11:58:37.802037+0000","last_became_active":"2023-10-18T11:58:36.533956+0000","last_became_peered":"2023-10-18T11:58:36.533956+0000","last_unstale":"2023-10-18T11:58:37.802037+0000","last_undegraded":"2023-10-18T11:58:37.802037+0000","last_fullsized":"2023-10-18T11:58:37.802037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:29.001140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802132+0000","last_change":"2023-10-18T11:58:36.530863+0000","last_active":"2023-10-18T11:58:37.802132+0000","last_peered":"2023-10-18T11:58:37.802132+0000","last_clean":"2023-10-18T11:58:37.802132+0000","last_became_active":"2023-10-18T11:58:36.530625+0000","last_became_peered":"2023-10-18T11:58:36.530625+0000","last_unstale":"2023-10-18T11:58:37.802132+0000","last_undegraded":"2023-10-18T11:58:37.802132+0000","last_fullsized":"2023-10-18T11:58:37.802132+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:03.443134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508994+0000","last_change":"2023-10-18T11:58:36.531552+0000","last_active":"2023-10-18T11:58:37.508994+0000","last_peered":"2023-10-18T11:58:37.508994+0000","last_clean":"2023-10-18T11:58:37.508994+0000","last_became_active":"2023-10-18T11:58:36.531163+0000","last_became_peered":"2023-10-18T11:58:36.531163+0000","last_unstale":"2023-10-18T11:58:37.508994+0000","last_undegraded":"2023-10-18T11:58:37.508994+0000","last_fullsized":"2023-10-18T11:58:37.508994+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:49.241771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509268+0000","last_change":"2023-10-18T11:58:36.530026+0000","last_active":"2023-10-18T11:58:37.509268+0000","last_peered":"2023-10-18T11:58:37.509268+0000","last_clean":"2023-10-18T11:58:37.509268+0000","last_became_active":"2023-10-18T11:58:36.529058+0000","last_became_peered":"2023-10-18T11:58:36.529058+0000","last_unstale":"2023-10-18T11:58:37.509268+0000","last_undegraded":"2023-10-18T11:58:37.509268+0000","last_fullsized":"2023-10-18T11:58:37.509268+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:41.289044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513034+0000","last_change":"2023-10-18T11:58:36.532114+0000","last_active":"2023-10-18T11:58:37.513034+0000","last_peered":"2023-10-18T11:58:37.513034+0000","last_clean":"2023-10-18T11:58:37.513034+0000","last_became_active":"2023-10-18T11:58:36.531526+0000","last_became_peered":"2023-10-18T11:58:36.531526+0000","last_unstale":"2023-10-18T11:58:37.513034+0000","last_undegraded":"2023-10-18T11:58:37.513034+0000","last_fullsized":"2023-10-18T11:58:37.513034+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:38.904126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512884+0000","last_change":"2023-10-18T11:58:36.534033+0000","last_active":"2023-10-18T11:58:37.512884+0000","last_peered":"2023-10-18T11:58:37.512884+0000","last_clean":"2023-10-18T11:58:37.512884+0000","last_became_active":"2023-10-18T11:58:36.533772+0000","last_became_peered":"2023-10-18T11:58:36.533772+0000","last_unstale":"2023-10-18T11:58:37.512884+0000","last_undegraded":"2023-10-18T11:58:37.512884+0000","last_fullsized":"2023-10-18T11:58:37.512884+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:04.963404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508988+0000","last_change":"2023-10-18T11:58:36.529660+0000","last_active":"2023-10-18T11:58:37.508988+0000","last_peered":"2023-10-18T11:58:37.508988+0000","last_clean":"2023-10-18T11:58:37.508988+0000","last_became_active":"2023-10-18T11:58:36.529125+0000","last_became_peered":"2023-10-18T11:58:36.529125+0000","last_unstale":"2023-10-18T11:58:37.508988+0000","last_undegraded":"2023-10-18T11:58:37.508988+0000","last_fullsized":"2023-10-18T11:58:37.508988+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:01.717669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802201+0000","last_change":"2023-10-18T11:58:36.519499+0000","last_active":"2023-10-18T11:58:37.802201+0000","last_peered":"2023-10-18T11:58:37.802201+0000","last_clean":"2023-10-18T11:58:37.802201+0000","last_became_active":"2023-10-18T11:58:36.519246+0000","last_became_peered":"2023-10-18T11:58:36.519246+0000","last_unstale":"2023-10-18T11:58:37.802201+0000","last_undegraded":"2023-10-18T11:58:37.802201+0000","last_fullsized":"2023-10-18T11:58:37.802201+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:48.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802474+0000","last_change":"2023-10-18T11:58:36.514398+0000","last_active":"2023-10-18T11:58:37.802474+0000","last_peered":"2023-10-18T11:58:37.802474+0000","last_clean":"2023-10-18T11:58:37.802474+0000","last_became_active":"2023-10-18T11:58:36.514141+0000","last_became_peered":"2023-10-18T11:58:36.514141+0000","last_unstale":"2023-10-18T11:58:37.802474+0000","last_undegraded":"2023-10-18T11:58:37.802474+0000","last_fullsized":"2023-10-18T11:58:37.802474+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:43.247625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802305+0000","last_change":"2023-10-18T11:58:36.527748+0000","last_active":"2023-10-18T11:58:37.802305+0000","last_peered":"2023-10-18T11:58:37.802305+0000","last_clean":"2023-10-18T11:58:37.802305+0000","last_became_active":"2023-10-18T11:58:36.527498+0000","last_became_peered":"2023-10-18T11:58:36.527498+0000","last_unstale":"2023-10-18T11:58:37.802305+0000","last_undegraded":"2023-10-18T11:58:37.802305+0000","last_fullsized":"2023-10-18T11:58:37.802305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:35.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801779+0000","last_change":"2023-10-18T11:58:36.521950+0000","last_active":"2023-10-18T11:58:37.801779+0000","last_peered":"2023-10-18T11:58:37.801779+0000","last_clean":"2023-10-18T11:58:37.801779+0000","last_became_active":"2023-10-18T11:58:36.521681+0000","last_became_peered":"2023-10-18T11:58:36.521681+0000","last_unstale":"2023-10-18T11:58:37.801779+0000","last_undegraded":"2023-10-18T11:58:37.801779+0000","last_fullsized":"2023-10-18T11:58:37.801779+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:43.982016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802739+0000","last_change":"2023-10-18T11:58:36.518867+0000","last_active":"2023-10-18T11:58:37.802739+0000","last_peered":"2023-10-18T11:58:37.802739+0000","last_clean":"2023-10-18T11:58:37.802739+0000","last_became_active":"2023-10-18T11:58:36.518625+0000","last_became_peered":"2023-10-18T11:58:36.518625+0000","last_unstale":"2023-10-18T11:58:37.802739+0000","last_undegraded":"2023-10-18T11:58:37.802739+0000","last_fullsized":"2023-10-18T11:58:37.802739+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:20.411248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802849+0000","last_change":"2023-10-18T11:58:36.528722+0000","last_active":"2023-10-18T11:58:37.802849+0000","last_peered":"2023-10-18T11:58:37.802849+0000","last_clean":"2023-10-18T11:58:37.802849+0000","last_became_active":"2023-10-18T11:58:36.528462+0000","last_became_peered":"2023-10-18T11:58:36.528462+0000","last_unstale":"2023-10-18T11:58:37.802849+0000","last_undegraded":"2023-10-18T11:58:37.802849+0000","last_fullsized":"2023-10-18T11:58:37.802849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:58:27.045894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803175+0000","last_change":"2023-10-18T11:58:36.527594+0000","last_active":"2023-10-18T11:58:37.803175+0000","last_peered":"2023-10-18T11:58:37.803175+0000","last_clean":"2023-10-18T11:58:37.803175+0000","last_became_active":"2023-10-18T11:58:36.526226+0000","last_became_peered":"2023-10-18T11:58:36.526226+0000","last_unstale":"2023-10-18T11:58:37.803175+0000","last_undegraded":"2023-10-18T11:58:37.803175+0000","last_fullsized":"2023-10-18T11:58:37.803175+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:28.446100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509068+0000","last_change":"2023-10-18T11:58:36.529803+0000","last_active":"2023-10-18T11:58:37.509068+0000","last_peered":"2023-10-18T11:58:37.509068+0000","last_clean":"2023-10-18T11:58:37.509068+0000","last_became_active":"2023-10-18T11:58:36.529446+0000","last_became_peered":"2023-10-18T11:58:36.529446+0000","last_unstale":"2023-10-18T11:58:37.509068+0000","last_undegraded":"2023-10-18T11:58:37.509068+0000","last_fullsized":"2023-10-18T11:58:37.509068+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:19.031434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802519+0000","last_change":"2023-10-18T11:58:36.526866+0000","last_active":"2023-10-18T11:58:37.802519+0000","last_peered":"2023-10-18T11:58:37.802519+0000","last_clean":"2023-10-18T11:58:37.802519+0000","last_became_active":"2023-10-18T11:58:36.526609+0000","last_became_peered":"2023-10-18T11:58:36.526609+0000","last_unstale":"2023-10-18T11:58:37.802519+0000","last_undegraded":"2023-10-18T11:58:37.802519+0000","last_fullsized":"2023-10-18T11:58:37.802519+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:04.860806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802258+0000","last_change":"2023-10-18T11:58:36.515793+0000","last_active":"2023-10-18T11:58:37.802258+0000","last_peered":"2023-10-18T11:58:37.802258+0000","last_clean":"2023-10-18T11:58:37.802258+0000","last_became_active":"2023-10-18T11:58:36.515523+0000","last_became_peered":"2023-10-18T11:58:36.515523+0000","last_unstale":"2023-10-18T11:58:37.802258+0000","last_undegraded":"2023-10-18T11:58:37.802258+0000","last_fullsized":"2023-10-18T11:58:37.802258+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802828+0000","last_change":"2023-10-18T11:58:36.515930+0000","last_active":"2023-10-18T11:58:37.802828+0000","last_peered":"2023-10-18T11:58:37.802828+0000","last_clean":"2023-10-18T11:58:37.802828+0000","last_became_active":"2023-10-18T11:58:36.515681+0000","last_became_peered":"2023-10-18T11:58:36.515681+0000","last_unstale":"2023-10-18T11:58:37.802828+0000","last_undegraded":"2023-10-18T11:58:37.802828+0000","last_fullsized":"2023-10-18T11:58:37.802828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:48:47.733918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508862+0000","last_change":"2023-10-18T11:58:36.530777+0000","last_active":"2023-10-18T11:58:37.508862+0000","last_peered":"2023-10-18T11:58:37.508862+0000","last_clean":"2023-10-18T11:58:37.508862+0000","last_became_active":"2023-10-18T11:58:36.530454+0000","last_became_peered":"2023-10-18T11:58:36.530454+0000","last_unstale":"2023-10-18T11:58:37.508862+0000","last_undegraded":"2023-10-18T11:58:37.508862+0000","last_fullsized":"2023-10-18T11:58:37.508862+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:00.542319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803195+0000","last_change":"2023-10-18T11:58:36.525494+0000","last_active":"2023-10-18T11:58:37.803195+0000","last_peered":"2023-10-18T11:58:37.803195+0000","last_clean":"2023-10-18T11:58:37.803195+0000","last_became_active":"2023-10-18T11:58:36.525200+0000","last_became_peered":"2023-10-18T11:58:36.525200+0000","last_unstale":"2023-10-18T11:58:37.803195+0000","last_undegraded":"2023-10-18T11:58:37.803195+0000","last_fullsized":"2023-10-18T11:58:37.803195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"203'4","reported_seq":16,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803001+0000","last_change":"2023-10-18T11:58:35.527076+0000","last_active":"2023-10-18T11:58:37.803001+0000","last_peered":"2023-10-18T11:58:37.803001+0000","last_clean":"2023-10-18T11:58:37.803001+0000","last_became_active":"2023-10-18T11:58:35.526809+0000","last_became_peered":"2023-10-18T11:58:35.526809+0000","last_unstale":"2023-10-18T11:58:37.803001+0000","last_undegraded":"2023-10-18T11:58:37.803001+0000","last_fullsized":"2023-10-18T11:58:37.803001+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802906+0000","last_change":"2023-10-18T11:58:36.521457+0000","last_active":"2023-10-18T11:58:37.802906+0000","last_peered":"2023-10-18T11:58:37.802906+0000","last_clean":"2023-10-18T11:58:37.802906+0000","last_became_active":"2023-10-18T11:58:36.521209+0000","last_became_peered":"2023-10-18T11:58:36.521209+0000","last_unstale":"2023-10-18T11:58:37.802906+0000","last_undegraded":"2023-10-18T11:58:37.802906+0000","last_fullsized":"2023-10-18T11:58:37.802906+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:40.559400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509267+0000","last_change":"2023-10-18T11:58:35.521828+0000","last_active":"2023-10-18T11:58:37.509267+0000","last_peered":"2023-10-18T11:58:37.509267+0000","last_clean":"2023-10-18T11:58:37.509267+0000","last_became_active":"2023-10-18T11:58:35.521574+0000","last_became_peered":"2023-10-18T11:58:35.521574+0000","last_unstale":"2023-10-18T11:58:37.509267+0000","last_undegraded":"2023-10-18T11:58:37.509267+0000","last_fullsized":"2023-10-18T11:58:37.509267+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:47.472064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803025+0000","last_change":"2023-10-18T11:58:36.522588+0000","last_active":"2023-10-18T11:58:37.803025+0000","last_peered":"2023-10-18T11:58:37.803025+0000","last_clean":"2023-10-18T11:58:37.803025+0000","last_became_active":"2023-10-18T11:58:36.522182+0000","last_became_peered":"2023-10-18T11:58:36.522182+0000","last_unstale":"2023-10-18T11:58:37.803025+0000","last_undegraded":"2023-10-18T11:58:37.803025+0000","last_fullsized":"2023-10-18T11:58:37.803025+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.326674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802389+0000","last_change":"2023-10-18T11:58:35.516307+0000","last_active":"2023-10-18T11:58:37.802389+0000","last_peered":"2023-10-18T11:58:37.802389+0000","last_clean":"2023-10-18T11:58:37.802389+0000","last_became_active":"2023-10-18T11:58:35.515995+0000","last_became_peered":"2023-10-18T11:58:35.515995+0000","last_unstale":"2023-10-18T11:58:37.802389+0000","last_undegraded":"2023-10-18T11:58:37.802389+0000","last_fullsized":"2023-10-18T11:58:37.802389+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:43.302735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512924+0000","last_change":"2023-10-18T11:58:35.526193+0000","last_active":"2023-10-18T11:58:37.512924+0000","last_peered":"2023-10-18T11:58:37.512924+0000","last_clean":"2023-10-18T11:58:37.512924+0000","last_became_active":"2023-10-18T11:58:35.525920+0000","last_became_peered":"2023-10-18T11:58:35.525920+0000","last_unstale":"2023-10-18T11:58:37.512924+0000","last_undegraded":"2023-10-18T11:58:37.512924+0000","last_fullsized":"2023-10-18T11:58:37.512924+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:17.858763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512416+0000","last_change":"2023-10-18T11:58:36.529406+0000","last_active":"2023-10-18T11:58:37.512416+0000","last_peered":"2023-10-18T11:58:37.512416+0000","last_clean":"2023-10-18T11:58:37.512416+0000","last_became_active":"2023-10-18T11:58:36.528795+0000","last_became_peered":"2023-10-18T11:58:36.528795+0000","last_unstale":"2023-10-18T11:58:37.512416+0000","last_undegraded":"2023-10-18T11:58:37.512416+0000","last_fullsized":"2023-10-18T11:58:37.512416+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:26.184608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509045+0000","last_change":"2023-10-18T11:58:35.523280+0000","last_active":"2023-10-18T11:58:37.509045+0000","last_peered":"2023-10-18T11:58:37.509045+0000","last_clean":"2023-10-18T11:58:37.509045+0000","last_became_active":"2023-10-18T11:58:35.522621+0000","last_became_peered":"2023-10-18T11:58:35.522621+0000","last_unstale":"2023-10-18T11:58:37.509045+0000","last_undegraded":"2023-10-18T11:58:37.509045+0000","last_fullsized":"2023-10-18T11:58:37.509045+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:27.443746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802321+0000","last_change":"2023-10-18T11:58:36.516159+0000","last_active":"2023-10-18T11:58:37.802321+0000","last_peered":"2023-10-18T11:58:37.802321+0000","last_clean":"2023-10-18T11:58:37.802321+0000","last_became_active":"2023-10-18T11:58:36.515894+0000","last_became_peered":"2023-10-18T11:58:36.515894+0000","last_unstale":"2023-10-18T11:58:37.802321+0000","last_undegraded":"2023-10-18T11:58:37.802321+0000","last_fullsized":"2023-10-18T11:58:37.802321+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:42.645151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802627+0000","last_change":"2023-10-18T11:58:35.521126+0000","last_active":"2023-10-18T11:58:37.802627+0000","last_peered":"2023-10-18T11:58:37.802627+0000","last_clean":"2023-10-18T11:58:37.802627+0000","last_became_active":"2023-10-18T11:58:35.520861+0000","last_became_peered":"2023-10-18T11:58:35.520861+0000","last_unstale":"2023-10-18T11:58:37.802627+0000","last_undegraded":"2023-10-18T11:58:37.802627+0000","last_fullsized":"2023-10-18T11:58:37.802627+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802501+0000","last_change":"2023-10-18T11:58:36.531084+0000","last_active":"2023-10-18T11:58:37.802501+0000","last_peered":"2023-10-18T11:58:37.802501+0000","last_clean":"2023-10-18T11:58:37.802501+0000","last_became_active":"2023-10-18T11:58:36.530792+0000","last_became_peered":"2023-10-18T11:58:36.530792+0000","last_unstale":"2023-10-18T11:58:37.802501+0000","last_undegraded":"2023-10-18T11:58:37.802501+0000","last_fullsized":"2023-10-18T11:58:37.802501+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:37.483860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512533+0000","last_change":"2023-10-18T11:58:35.518246+0000","last_active":"2023-10-18T11:58:37.512533+0000","last_peered":"2023-10-18T11:58:37.512533+0000","last_clean":"2023-10-18T11:58:37.512533+0000","last_became_active":"2023-10-18T11:58:35.517986+0000","last_became_peered":"2023-10-18T11:58:35.517986+0000","last_unstale":"2023-10-18T11:58:37.512533+0000","last_undegraded":"2023-10-18T11:58:37.512533+0000","last_fullsized":"2023-10-18T11:58:37.512533+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:02:15.620821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803081+0000","last_change":"2023-10-18T11:58:36.533670+0000","last_active":"2023-10-18T11:58:37.803081+0000","last_peered":"2023-10-18T11:58:37.803081+0000","last_clean":"2023-10-18T11:58:37.803081+0000","last_became_active":"2023-10-18T11:58:36.533394+0000","last_became_peered":"2023-10-18T11:58:36.533394+0000","last_unstale":"2023-10-18T11:58:37.803081+0000","last_undegraded":"2023-10-18T11:58:37.803081+0000","last_fullsized":"2023-10-18T11:58:37.803081+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:21.396797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509229+0000","last_change":"2023-10-18T11:58:35.524709+0000","last_active":"2023-10-18T11:58:37.509229+0000","last_peered":"2023-10-18T11:58:37.509229+0000","last_clean":"2023-10-18T11:58:37.509229+0000","last_became_active":"2023-10-18T11:58:35.524471+0000","last_became_peered":"2023-10-18T11:58:35.524471+0000","last_unstale":"2023-10-18T11:58:37.509229+0000","last_undegraded":"2023-10-18T11:58:37.509229+0000","last_fullsized":"2023-10-18T11:58:37.509229+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:53.288380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509250+0000","last_change":"2023-10-18T11:58:36.523093+0000","last_active":"2023-10-18T11:58:37.509250+0000","last_peered":"2023-10-18T11:58:37.509250+0000","last_clean":"2023-10-18T11:58:37.509250+0000","last_became_active":"2023-10-18T11:58:36.522551+0000","last_became_peered":"2023-10-18T11:58:36.522551+0000","last_unstale":"2023-10-18T11:58:37.509250+0000","last_undegraded":"2023-10-18T11:58:37.509250+0000","last_fullsized":"2023-10-18T11:58:37.509250+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:22.715669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802085+0000","last_change":"2023-10-18T11:58:35.522093+0000","last_active":"2023-10-18T11:58:37.802085+0000","last_peered":"2023-10-18T11:58:37.802085+0000","last_clean":"2023-10-18T11:58:37.802085+0000","last_became_active":"2023-10-18T11:58:35.521805+0000","last_became_peered":"2023-10-18T11:58:35.521805+0000","last_unstale":"2023-10-18T11:58:37.802085+0000","last_undegraded":"2023-10-18T11:58:37.802085+0000","last_fullsized":"2023-10-18T11:58:37.802085+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:08.791931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513049+0000","last_change":"2023-10-18T11:58:36.531254+0000","last_active":"2023-10-18T11:58:37.513049+0000","last_peered":"2023-10-18T11:58:37.513049+0000","last_clean":"2023-10-18T11:58:37.513049+0000","last_became_active":"2023-10-18T11:58:36.530967+0000","last_became_peered":"2023-10-18T11:58:36.530967+0000","last_unstale":"2023-10-18T11:58:37.513049+0000","last_undegraded":"2023-10-18T11:58:37.513049+0000","last_fullsized":"2023-10-18T11:58:37.513049+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:01.297812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"203'3","reported_seq":15,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802710+0000","last_change":"2023-10-18T11:58:35.525207+0000","last_active":"2023-10-18T11:58:37.802710+0000","last_peered":"2023-10-18T11:58:37.802710+0000","last_clean":"2023-10-18T11:58:37.802710+0000","last_became_active":"2023-10-18T11:58:35.524946+0000","last_became_peered":"2023-10-18T11:58:35.524946+0000","last_unstale":"2023-10-18T11:58:37.802710+0000","last_undegraded":"2023-10-18T11:58:37.802710+0000","last_fullsized":"2023-10-18T11:58:37.802710+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802920+0000","last_change":"2023-10-18T11:58:36.516878+0000","last_active":"2023-10-18T11:58:37.802920+0000","last_peered":"2023-10-18T11:58:37.802920+0000","last_clean":"2023-10-18T11:58:37.802920+0000","last_became_active":"2023-10-18T11:58:36.516625+0000","last_became_peered":"2023-10-18T11:58:36.516625+0000","last_unstale":"2023-10-18T11:58:37.802920+0000","last_undegraded":"2023-10-18T11:58:37.802920+0000","last_fullsized":"2023-10-18T11:58:37.802920+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:59.525275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"203'6","reported_seq":18,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513026+0000","last_change":"2023-10-18T11:58:35.518030+0000","last_active":"2023-10-18T11:58:37.513026+0000","last_peered":"2023-10-18T11:58:37.513026+0000","last_clean":"2023-10-18T11:58:37.513026+0000","last_became_active":"2023-10-18T11:58:35.517747+0000","last_became_peered":"2023-10-18T11:58:35.517747+0000","last_unstale":"2023-10-18T11:58:37.513026+0000","last_undegraded":"2023-10-18T11:58:37.513026+0000","last_fullsized":"2023-10-18T11:58:37.513026+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.335660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801895+0000","last_change":"2023-10-18T11:58:36.524788+0000","last_active":"2023-10-18T11:58:37.801895+0000","last_peered":"2023-10-18T11:58:37.801895+0000","last_clean":"2023-10-18T11:58:37.801895+0000","last_became_active":"2023-10-18T11:58:36.524547+0000","last_became_peered":"2023-10-18T11:58:36.524547+0000","last_unstale":"2023-10-18T11:58:37.801895+0000","last_undegraded":"2023-10-18T11:58:37.801895+0000","last_fullsized":"2023-10-18T11:58:37.801895+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:02.661591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512833+0000","last_change":"2023-10-18T11:58:35.521655+0000","last_active":"2023-10-18T11:58:37.512833+0000","last_peered":"2023-10-18T11:58:37.512833+0000","last_clean":"2023-10-18T11:58:37.512833+0000","last_became_active":"2023-10-18T11:58:35.521430+0000","last_became_peered":"2023-10-18T11:58:35.521430+0000","last_unstale":"2023-10-18T11:58:37.512833+0000","last_undegraded":"2023-10-18T11:58:37.512833+0000","last_fullsized":"2023-10-18T11:58:37.512833+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:21.628472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508795+0000","last_change":"2023-10-18T11:58:36.523743+0000","last_active":"2023-10-18T11:58:37.508795+0000","last_peered":"2023-10-18T11:58:37.508795+0000","last_clean":"2023-10-18T11:58:37.508795+0000","last_became_active":"2023-10-18T11:58:36.523492+0000","last_became_peered":"2023-10-18T11:58:36.523492+0000","last_unstale":"2023-10-18T11:58:37.508795+0000","last_undegraded":"2023-10-18T11:58:37.508795+0000","last_fullsized":"2023-10-18T11:58:37.508795+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:42.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802579+0000","last_change":"2023-10-18T11:58:35.522746+0000","last_active":"2023-10-18T11:58:37.802579+0000","last_peered":"2023-10-18T11:58:37.802579+0000","last_clean":"2023-10-18T11:58:37.802579+0000","last_became_active":"2023-10-18T11:58:35.522486+0000","last_became_peered":"2023-10-18T11:58:35.522486+0000","last_unstale":"2023-10-18T11:58:37.802579+0000","last_undegraded":"2023-10-18T11:58:37.802579+0000","last_fullsized":"2023-10-18T11:58:37.802579+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:07.062822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512576+0000","last_change":"2023-10-18T11:58:36.533848+0000","last_active":"2023-10-18T11:58:37.512576+0000","last_peered":"2023-10-18T11:58:37.512576+0000","last_clean":"2023-10-18T11:58:37.512576+0000","last_became_active":"2023-10-18T11:58:36.533565+0000","last_became_peered":"2023-10-18T11:58:36.533565+0000","last_unstale":"2023-10-18T11:58:37.512576+0000","last_undegraded":"2023-10-18T11:58:37.512576+0000","last_fullsized":"2023-10-18T11:58:37.512576+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:48.249824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802427+0000","last_change":"2023-10-18T11:58:35.517587+0000","last_active":"2023-10-18T11:58:37.802427+0000","last_peered":"2023-10-18T11:58:37.802427+0000","last_clean":"2023-10-18T11:58:37.802427+0000","last_became_active":"2023-10-18T11:58:35.517322+0000","last_became_peered":"2023-10-18T11:58:35.517322+0000","last_unstale":"2023-10-18T11:58:37.802427+0000","last_undegraded":"2023-10-18T11:58:37.802427+0000","last_fullsized":"2023-10-18T11:58:37.802427+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:34.567188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512733+0000","last_change":"2023-10-18T11:58:36.531687+0000","last_active":"2023-10-18T11:58:37.512733+0000","last_peered":"2023-10-18T11:58:37.512733+0000","last_clean":"2023-10-18T11:58:37.512733+0000","last_became_active":"2023-10-18T11:58:36.531411+0000","last_became_peered":"2023-10-18T11:58:36.531411+0000","last_unstale":"2023-10-18T11:58:37.512733+0000","last_undegraded":"2023-10-18T11:58:37.512733+0000","last_fullsized":"2023-10-18T11:58:37.512733+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:04.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"203'8","reported_seq":20,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802771+0000","last_change":"2023-10-18T11:58:35.515506+0000","last_active":"2023-10-18T11:58:37.802771+0000","last_peered":"2023-10-18T11:58:37.802771+0000","last_clean":"2023-10-18T11:58:37.802771+0000","last_became_active":"2023-10-18T11:58:35.515198+0000","last_became_peered":"2023-10-18T11:58:35.515198+0000","last_unstale":"2023-10-18T11:58:37.802771+0000","last_undegraded":"2023-10-18T11:58:37.802771+0000","last_fullsized":"2023-10-18T11:58:37.802771+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.509931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802367+0000","last_change":"2023-10-18T11:58:36.515935+0000","last_active":"2023-10-18T11:58:37.802367+0000","last_peered":"2023-10-18T11:58:37.802367+0000","last_clean":"2023-10-18T11:58:37.802367+0000","last_became_active":"2023-10-18T11:58:36.515722+0000","last_became_peered":"2023-10-18T11:58:36.515722+0000","last_unstale":"2023-10-18T11:58:37.802367+0000","last_undegraded":"2023-10-18T11:58:37.802367+0000","last_fullsized":"2023-10-18T11:58:37.802367+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:58.998126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:35.515482+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:35.515185+0000","last_became_peered":"2023-10-18T11:58:35.515185+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.149129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509119+0000","last_change":"2023-10-18T11:58:36.534430+0000","last_active":"2023-10-18T11:58:37.509119+0000","last_peered":"2023-10-18T11:58:37.509119+0000","last_clean":"2023-10-18T11:58:37.509119+0000","last_became_active":"2023-10-18T11:58:36.534167+0000","last_became_peered":"2023-10-18T11:58:36.534167+0000","last_unstale":"2023-10-18T11:58:37.509119+0000","last_undegraded":"2023-10-18T11:58:37.509119+0000","last_fullsized":"2023-10-18T11:58:37.509119+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:44.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802606+0000","last_change":"2023-10-18T11:58:35.522154+0000","last_active":"2023-10-18T11:58:37.802606+0000","last_peered":"2023-10-18T11:58:37.802606+0000","last_clean":"2023-10-18T11:58:37.802606+0000","last_became_active":"2023-10-18T11:58:35.521902+0000","last_became_peered":"2023-10-18T11:58:35.521902+0000","last_unstale":"2023-10-18T11:58:37.802606+0000","last_undegraded":"2023-10-18T11:58:37.802606+0000","last_fullsized":"2023-10-18T11:58:37.802606+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:27.447038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803092+0000","last_change":"2023-10-18T11:58:36.529258+0000","last_active":"2023-10-18T11:58:37.803092+0000","last_peered":"2023-10-18T11:58:37.803092+0000","last_clean":"2023-10-18T11:58:37.803092+0000","last_became_active":"2023-10-18T11:58:36.528468+0000","last_became_peered":"2023-10-18T11:58:36.528468+0000","last_unstale":"2023-10-18T11:58:37.803092+0000","last_undegraded":"2023-10-18T11:58:37.803092+0000","last_fullsized":"2023-10-18T11:58:37.803092+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:58.035451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803037+0000","last_change":"2023-10-18T11:58:36.521908+0000","last_active":"2023-10-18T11:58:37.803037+0000","last_peered":"2023-10-18T11:58:37.803037+0000","last_clean":"2023-10-18T11:58:37.803037+0000","last_became_active":"2023-10-18T11:58:36.521381+0000","last_became_peered":"2023-10-18T11:58:36.521381+0000","last_unstale":"2023-10-18T11:58:37.803037+0000","last_undegraded":"2023-10-18T11:58:37.803037+0000","last_fullsized":"2023-10-18T11:58:37.803037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:18.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803110+0000","last_change":"2023-10-18T11:58:35.515862+0000","last_active":"2023-10-18T11:58:37.803110+0000","last_peered":"2023-10-18T11:58:37.803110+0000","last_clean":"2023-10-18T11:58:37.803110+0000","last_became_active":"2023-10-18T11:58:35.515521+0000","last_became_peered":"2023-10-18T11:58:35.515521+0000","last_unstale":"2023-10-18T11:58:37.803110+0000","last_undegraded":"2023-10-18T11:58:37.803110+0000","last_fullsized":"2023-10-18T11:58:37.803110+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:03.199289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801885+0000","last_change":"2023-10-18T11:58:36.520962+0000","last_active":"2023-10-18T11:58:37.801885+0000","last_peered":"2023-10-18T11:58:37.801885+0000","last_clean":"2023-10-18T11:58:37.801885+0000","last_became_active":"2023-10-18T11:58:36.520712+0000","last_became_peered":"2023-10-18T11:58:36.520712+0000","last_unstale":"2023-10-18T11:58:37.801885+0000","last_undegraded":"2023-10-18T11:58:37.801885+0000","last_fullsized":"2023-10-18T11:58:37.801885+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:46.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512865+0000","last_change":"2023-10-18T11:58:35.524483+0000","last_active":"2023-10-18T11:58:37.512865+0000","last_peered":"2023-10-18T11:58:37.512865+0000","last_clean":"2023-10-18T11:58:37.512865+0000","last_became_active":"2023-10-18T11:58:35.524231+0000","last_became_peered":"2023-10-18T11:58:35.524231+0000","last_unstale":"2023-10-18T11:58:37.512865+0000","last_undegraded":"2023-10-18T11:58:37.512865+0000","last_fullsized":"2023-10-18T11:58:37.512865+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:29:21.906133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.529745+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.529497+0000","last_became_peered":"2023-10-18T11:58:36.529497+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802708+0000","last_change":"2023-10-18T11:58:35.526709+0000","last_active":"2023-10-18T11:58:37.802708+0000","last_peered":"2023-10-18T11:58:37.802708+0000","last_clean":"2023-10-18T11:58:37.802708+0000","last_became_active":"2023-10-18T11:58:35.526448+0000","last_became_peered":"2023-10-18T11:58:35.526448+0000","last_unstale":"2023-10-18T11:58:37.802708+0000","last_undegraded":"2023-10-18T11:58:37.802708+0000","last_fullsized":"2023-10-18T11:58:37.802708+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:20.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802541+0000","last_change":"2023-10-18T11:58:36.528368+0000","last_active":"2023-10-18T11:58:37.802541+0000","last_peered":"2023-10-18T11:58:37.802541+0000","last_clean":"2023-10-18T11:58:37.802541+0000","last_became_active":"2023-10-18T11:58:36.528124+0000","last_became_peered":"2023-10-18T11:58:36.528124+0000","last_unstale":"2023-10-18T11:58:37.802541+0000","last_undegraded":"2023-10-18T11:58:37.802541+0000","last_fullsized":"2023-10-18T11:58:37.802541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:48.474419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802873+0000","last_change":"2023-10-18T11:58:35.525781+0000","last_active":"2023-10-18T11:58:37.802873+0000","last_peered":"2023-10-18T11:58:37.802873+0000","last_clean":"2023-10-18T11:58:37.802873+0000","last_became_active":"2023-10-18T11:58:35.525519+0000","last_became_peered":"2023-10-18T11:58:35.525519+0000","last_unstale":"2023-10-18T11:58:37.802873+0000","last_undegraded":"2023-10-18T11:58:37.802873+0000","last_fullsized":"2023-10-18T11:58:37.802873+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:47:37.957986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508969+0000","last_change":"2023-10-18T11:58:36.518933+0000","last_active":"2023-10-18T11:58:37.508969+0000","last_peered":"2023-10-18T11:58:37.508969+0000","last_clean":"2023-10-18T11:58:37.508969+0000","last_became_active":"2023-10-18T11:58:36.518691+0000","last_became_peered":"2023-10-18T11:58:36.518691+0000","last_unstale":"2023-10-18T11:58:37.508969+0000","last_undegraded":"2023-10-18T11:58:37.508969+0000","last_fullsized":"2023-10-18T11:58:37.508969+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:55.005558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802476+0000","last_change":"2023-10-18T11:58:36.522037+0000","last_active":"2023-10-18T11:58:37.802476+0000","last_peered":"2023-10-18T11:58:37.802476+0000","last_clean":"2023-10-18T11:58:37.802476+0000","last_became_active":"2023-10-18T11:58:36.521783+0000","last_became_peered":"2023-10-18T11:58:36.521783+0000","last_unstale":"2023-10-18T11:58:37.802476+0000","last_undegraded":"2023-10-18T11:58:37.802476+0000","last_fullsized":"2023-10-18T11:58:37.802476+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:37:11.613406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803083+0000","last_change":"2023-10-18T11:58:35.518886+0000","last_active":"2023-10-18T11:58:37.803083+0000","last_peered":"2023-10-18T11:58:37.803083+0000","last_clean":"2023-10-18T11:58:37.803083+0000","last_became_active":"2023-10-18T11:58:35.518591+0000","last_became_peered":"2023-10-18T11:58:35.518591+0000","last_unstale":"2023-10-18T11:58:37.803083+0000","last_undegraded":"2023-10-18T11:58:37.803083+0000","last_fullsized":"2023-10-18T11:58:37.803083+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:49.481190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802223+0000","last_change":"2023-10-18T11:58:36.519762+0000","last_active":"2023-10-18T11:58:37.802223+0000","last_peered":"2023-10-18T11:58:37.802223+0000","last_clean":"2023-10-18T11:58:37.802223+0000","last_became_active":"2023-10-18T11:58:36.517560+0000","last_became_peered":"2023-10-18T11:58:36.517560+0000","last_unstale":"2023-10-18T11:58:37.802223+0000","last_undegraded":"2023-10-18T11:58:37.802223+0000","last_fullsized":"2023-10-18T11:58:37.802223+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:29.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802645+0000","last_change":"2023-10-18T11:58:35.522798+0000","last_active":"2023-10-18T11:58:37.802645+0000","last_peered":"2023-10-18T11:58:37.802645+0000","last_clean":"2023-10-18T11:58:37.802645+0000","last_became_active":"2023-10-18T11:58:35.522532+0000","last_became_peered":"2023-10-18T11:58:35.522532+0000","last_unstale":"2023-10-18T11:58:37.802645+0000","last_undegraded":"2023-10-18T11:58:37.802645+0000","last_fullsized":"2023-10-18T11:58:37.802645+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:40:51.705544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:36.530354+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:36.530080+0000","last_became_peered":"2023-10-18T11:58:36.530080+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:19.995868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509241+0000","last_change":"2023-10-18T11:58:35.523100+0000","last_active":"2023-10-18T11:58:37.509241+0000","last_peered":"2023-10-18T11:58:37.509241+0000","last_clean":"2023-10-18T11:58:37.509241+0000","last_became_active":"2023-10-18T11:58:35.522802+0000","last_became_peered":"2023-10-18T11:58:35.522802+0000","last_unstale":"2023-10-18T11:58:37.509241+0000","last_undegraded":"2023-10-18T11:58:37.509241+0000","last_fullsized":"2023-10-18T11:58:37.509241+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802916+0000","last_change":"2023-10-18T11:58:36.522886+0000","last_active":"2023-10-18T11:58:37.802916+0000","last_peered":"2023-10-18T11:58:37.802916+0000","last_clean":"2023-10-18T11:58:37.802916+0000","last_became_active":"2023-10-18T11:58:36.522614+0000","last_became_peered":"2023-10-18T11:58:36.522614+0000","last_unstale":"2023-10-18T11:58:37.802916+0000","last_undegraded":"2023-10-18T11:58:37.802916+0000","last_fullsized":"2023-10-18T11:58:37.802916+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:52.212999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509088+0000","last_change":"2023-10-18T11:58:35.523409+0000","last_active":"2023-10-18T11:58:37.509088+0000","last_peered":"2023-10-18T11:58:37.509088+0000","last_clean":"2023-10-18T11:58:37.509088+0000","last_became_active":"2023-10-18T11:58:35.523146+0000","last_became_peered":"2023-10-18T11:58:35.523146+0000","last_unstale":"2023-10-18T11:58:37.509088+0000","last_undegraded":"2023-10-18T11:58:37.509088+0000","last_fullsized":"2023-10-18T11:58:37.509088+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:51.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802959+0000","last_change":"2023-10-18T11:58:35.522529+0000","last_active":"2023-10-18T11:58:37.802959+0000","last_peered":"2023-10-18T11:58:37.802959+0000","last_clean":"2023-10-18T11:58:37.802959+0000","last_became_active":"2023-10-18T11:58:35.522261+0000","last_became_peered":"2023-10-18T11:58:35.522261+0000","last_unstale":"2023-10-18T11:58:37.802959+0000","last_undegraded":"2023-10-18T11:58:37.802959+0000","last_fullsized":"2023-10-18T11:58:37.802959+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512534+0000","last_change":"2023-10-18T11:58:36.530248+0000","last_active":"2023-10-18T11:58:37.512534+0000","last_peered":"2023-10-18T11:58:37.512534+0000","last_clean":"2023-10-18T11:58:37.512534+0000","last_became_active":"2023-10-18T11:58:36.529997+0000","last_became_peered":"2023-10-18T11:58:36.529997+0000","last_unstale":"2023-10-18T11:58:37.512534+0000","last_undegraded":"2023-10-18T11:58:37.512534+0000","last_fullsized":"2023-10-18T11:58:37.512534+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:58.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802154+0000","last_change":"2023-10-18T11:58:35.527290+0000","last_active":"2023-10-18T11:58:37.802154+0000","last_peered":"2023-10-18T11:58:37.802154+0000","last_clean":"2023-10-18T11:58:37.802154+0000","last_became_active":"2023-10-18T11:58:35.527052+0000","last_became_peered":"2023-10-18T11:58:35.527052+0000","last_unstale":"2023-10-18T11:58:37.802154+0000","last_undegraded":"2023-10-18T11:58:37.802154+0000","last_fullsized":"2023-10-18T11:58:37.802154+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802594+0000","last_change":"2023-10-18T11:58:36.520551+0000","last_active":"2023-10-18T11:58:37.802594+0000","last_peered":"2023-10-18T11:58:37.802594+0000","last_clean":"2023-10-18T11:58:37.802594+0000","last_became_active":"2023-10-18T11:58:36.520297+0000","last_became_peered":"2023-10-18T11:58:36.520297+0000","last_unstale":"2023-10-18T11:58:37.802594+0000","last_undegraded":"2023-10-18T11:58:37.802594+0000","last_fullsized":"2023-10-18T11:58:37.802594+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:25.536175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802945+0000","last_change":"2023-10-18T11:58:35.526011+0000","last_active":"2023-10-18T11:58:37.802945+0000","last_peered":"2023-10-18T11:58:37.802945+0000","last_clean":"2023-10-18T11:58:37.802945+0000","last_became_active":"2023-10-18T11:58:35.525734+0000","last_became_peered":"2023-10-18T11:58:35.525734+0000","last_unstale":"2023-10-18T11:58:37.802945+0000","last_undegraded":"2023-10-18T11:58:37.802945+0000","last_fullsized":"2023-10-18T11:58:37.802945+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:19.352633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802222+0000","last_change":"2023-10-18T11:58:36.528280+0000","last_active":"2023-10-18T11:58:37.802222+0000","last_peered":"2023-10-18T11:58:37.802222+0000","last_clean":"2023-10-18T11:58:37.802222+0000","last_became_active":"2023-10-18T11:58:36.528012+0000","last_became_peered":"2023-10-18T11:58:36.528012+0000","last_unstale":"2023-10-18T11:58:37.802222+0000","last_undegraded":"2023-10-18T11:58:37.802222+0000","last_fullsized":"2023-10-18T11:58:37.802222+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:28.449393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:34.497064+0000","last_change":"2023-10-18T11:58:34.497064+0000","last_active":"2023-10-18T11:58:34.497064+0000","last_peered":"2023-10-18T11:58:34.497064+0000","last_clean":"2023-10-18T11:58:34.497064+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:34.497064+0000","last_undegraded":"2023-10-18T11:58:34.497064+0000","last_fullsized":"2023-10-18T11:58:34.497064+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T11:58:35.499419+0000","last_change":"2023-10-18T11:58:35.499419+0000","last_active":"2023-10-18T11:58:35.499419+0000","last_peered":"2023-10-18T11:58:35.499419+0000","last_clean":"2023-10-18T11:58:35.499419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T11:58:35.499419+0000","last_undegraded":"2023-10-18T11:58:35.499419+0000","last_fullsized":"2023-10-18T11:58:35.499419+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801952+0000","last_change":"2023-10-18T11:58:35.513693+0000","last_active":"2023-10-18T11:58:37.801952+0000","last_peered":"2023-10-18T11:58:37.801952+0000","last_clean":"2023-10-18T11:58:37.801952+0000","last_became_active":"2023-10-18T11:58:35.513429+0000","last_became_peered":"2023-10-18T11:58:35.513429+0000","last_unstale":"2023-10-18T11:58:37.801952+0000","last_undegraded":"2023-10-18T11:58:37.801952+0000","last_fullsized":"2023-10-18T11:58:37.801952+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:12.442948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"193'335","reported_seq":1140,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-10-18T11:58:36.518008+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:58:36.518008+0000","last_peered":"2023-10-18T11:58:36.518008+0000","last_clean":"2023-10-18T11:58:36.518008+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:58:36.518008+0000","last_undegraded":"2023-10-18T11:58:36.518008+0000","last_fullsized":"2023-10-18T11:58:36.518008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":120,"acting":120,"num_store_stats":7},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52308,"kb_used_data":3080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319532,"statfs":{"total":96636764160,"available":96583200768,"internally_reserved":0,"allocated":3153920,"data_stored":1895422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.363},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74399999999999999,"5min":0.77200000000000002,"15min":0.84099999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.39700000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":1.298},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.82699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.35699999999999998},"max":{"1min":0.80200000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45200000000000001,"15min":0.44900000000000001},"min":{"1min":0.34000000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.80200000000000005,"15min":1.5249999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.27700000000000002},"max":{"1min":0.63500000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.78900000000000003,"15min":0.90800000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.375,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":0.82699999999999996,"15min":1.4179999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":1.478},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.874},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37768,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334072,"statfs":{"total":96636764160,"available":96598089728,"internally_reserved":0,"allocated":2486272,"data_stored":1267517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.50700000000000001,"5min":0.41899999999999998,"15min":0.29899999999999999},"max":{"1min":0.78100000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.27900000000000003},"max":{"1min":0.76700000000000002,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.87},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.625,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.25600000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.82599999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.626},"min":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47299999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.76900000000000002},"last":0.37},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.71999999999999997,"15min":0.82999999999999996},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.81200000000000006},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72699999999999998,"5min":0.80600000000000005,"15min":0.84599999999999997},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575970,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60068,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311772,"statfs":{"total":96636764160,"available":96575254528,"internally_reserved":0,"allocated":2576384,"data_stored":1306688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.249},"max":{"1min":0.84099999999999997,"5min":0.90300000000000002,"15min":0.90800000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.50700000000000001,"5min":0.38500000000000001,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.95899999999999996,"15min":0.96099999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.24299999999999999},"max":{"1min":1.081,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82199999999999995,"15min":0.754},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.28399999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.98699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.84999999999999998,"15min":0.98399999999999999},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.53200000000000003,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.0349999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.501,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":1,"15min":1},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.30299999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.98499999999999999,"15min":1.01},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.93899999999999995,"15min":0.97999999999999998},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83699999999999997,"5min":1.101,"15min":1.101},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.34000000000000002},"max":{"1min":0.83399999999999996,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575970,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29544,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342296,"statfs":{"total":96636764160,"available":96606511104,"internally_reserved":0,"allocated":2576384,"data_stored":1306700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.62},"min":{"1min":0.45400000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80700000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.80200000000000005,"15min":0.85399999999999998},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69499999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.372,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.752,"5min":1.123,"15min":1.123},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.92100000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.93600000000000005},"last":0.629}]},{"osd":3,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.312},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.34599999999999997,"15min":0.31900000000000001},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":2568192,"data_stored":1305214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.221},"max":{"1min":0.57099999999999995,"5min":0.61599999999999999,"15min":0.63300000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.28100000000000003,"15min":0.23400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.374}]},{"osd":1,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26400000000000001},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.56000000000000005,"5min":0.60599999999999998,"15min":0.65000000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.73399999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.052},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47499999999999998,"15min":0.40999999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.026},"last":0.625}]},{"osd":6,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.437},"max":{"1min":0.82299999999999995,"5min":0.93100000000000005,"15min":0.94499999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.68700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.45400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.55600000000000005,"5min":0.46400000000000002,"15min":0.442},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.58099999999999996,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.67800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60060,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311780,"statfs":{"total":96636764160,"available":96575262720,"internally_reserved":0,"allocated":2564096,"data_stored":1305636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.252,"5min":0.222,"15min":0.216},"max":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.67200000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.27900000000000003,"5min":0.26200000000000001,"15min":0.25800000000000001},"max":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.746,"15min":0.746},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.435,"15min":0.432},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.374},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35399999999999998,"5min":0.29899999999999999,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.33600000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.499,"5min":0.442,"15min":0.437},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.47799999999999998,"15min":0.41699999999999998},"max":{"1min":0.80900000000000005,"5min":1.087,"15min":1.087},"last":0.435}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.54500000000000004,"5min":0.46500000000000002,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66600000000000004,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.45200000000000001,"15min":0.40999999999999998},"max":{"1min":0.85799999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68500000000000005,"15min":0.68200000000000005},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.45100000000000001},"max":{"1min":0.84099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.443,"15min":0.42499999999999999},"max":{"1min":0.86799999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.53900000000000003,"5min":0.443,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.01},"last":0.499}]}]},{"osd":1,"up_from":13,"seq":55834575970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60048,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311792,"statfs":{"total":96636764160,"available":96575275008,"internally_reserved":0,"allocated":2560000,"data_stored":1305054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.182},"max":{"1min":0.73899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65800000000000003,"5min":0.81000000000000005,"15min":0.85299999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.23000000000000001},"max":{"1min":0.496,"5min":0.877,"15min":0.877},"last":0.28000000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.69799999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.377},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51400000000000001},"min":{"1min":0.28899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.74399999999999999,"5min":0.79800000000000004,"15min":0.91000000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.74099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.65600000000000003},"min":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.29599999999999999},"max":{"1min":0.80300000000000005,"5min":0.86599999999999999,"15min":1.2390000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.316},"max":{"1min":0.70299999999999996,"5min":1.05,"15min":1.212},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":1.0209999999999999,"15min":1.0589999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77600000000000002,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.47399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82499999999999996,"5min":0.88400000000000001,"15min":1.0620000000000001},"last":0.69899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42600,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329240,"statfs":{"total":96636764160,"available":96593141760,"internally_reserved":0,"allocated":3170304,"data_stored":1896150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.57199999999999995,"5min":0.64400000000000002,"15min":0.66200000000000003},"last":0.247}]},{"osd":2,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.33700000000000002,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.433,"15min":0.42799999999999999},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.25700000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.36699999999999999},"max":{"1min":0.876,"5min":0.96999999999999997,"15min":0.997},"last":0.495},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.50900000000000001,"5min":0.45400000000000001,"15min":0.39100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":0.93999999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.504,"5min":0.439,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.95799999999999996,"15min":0.996},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.44700000000000001},"max":{"1min":0.89300000000000002,"5min":1.0089999999999999,"15min":1.038},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.48599999999999999,"5min":0.44900000000000001,"15min":0.44800000000000001},"max":{"1min":0.91100000000000003,"5min":0.95099999999999996,"15min":1.0609999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":1.0109999999999999,"15min":1.0189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.42999999999999999},"max":{"1min":0.89400000000000002,"5min":0.997,"15min":0.997},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:58:43.828 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T11:58:43.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.947+0000 7f4dba15e640 1 -- 172.21.15.125:0/1868907983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db40ceb40 msgr2=0x7f4db40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:43.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.947+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1868907983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db40ceb40 0x7f4db40b0650 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005fd0 tx=0x7f4dac02c240 comp rx=0 tx=0).stop 2023-10-18T11:58:43.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.947+0000 7f4dba15e640 1 -- 172.21.15.125:0/1868907983 shutdown_connections 2023-10-18T11:58:43.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.947+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1868907983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db40ceb40 0x7f4db40b0650 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:43.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.947+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1868907983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4db415b2a0 0x7f4db40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:43.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.947+0000 7f4dba15e640 1 -- 172.21.15.125:0/1868907983 >> 172.21.15.125:0/1868907983 conn(0x7f4db40d0d60 msgr2=0x7f4db40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:43.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.947+0000 7f4dba15e640 1 -- 172.21.15.125:0/1868907983 shutdown_connections 2023-10-18T11:58:43.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.947+0000 7f4dba15e640 1 -- 172.21.15.125:0/1868907983 wait complete. 2023-10-18T11:58:43.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.948+0000 7f4dba15e640 1 Processor -- start 2023-10-18T11:58:43.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.948+0000 7f4dba15e640 1 -- start start 2023-10-18T11:58:43.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4dba15e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40ceb40 0x7f4db4150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:43.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4dba15e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4db40cf340 0x7f4db40c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:43.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4dba15e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db415b2a0 0x7f4db40b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:43.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4dba15e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40c8cb0 con 0x7f4db415b2a0 2023-10-18T11:58:43.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4dba15e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40c8f90 con 0x7f4db40ceb40 2023-10-18T11:58:43.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4dba15e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40c9270 con 0x7f4db40cf340 2023-10-18T11:58:43.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db415b2a0 0x7f4db40b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:43.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db415b2a0 0x7f4db40b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50066/0 (socket says 172.21.15.125:50066) 2023-10-18T11:58:43.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4db3fff640 1 -- 172.21.15.125:0/1035425576 learned_addr learned my addr 172.21.15.125:0/1035425576 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:43.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4db37fe640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40ceb40 0x7f4db4150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:43.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.949+0000 7f4db2ffd640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4db40cf340 0x7f4db40c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:43.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.950+0000 7f4db3fff640 1 -- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4db40cf340 msgr2=0x7f4db40c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:43.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.950+0000 7f4db3fff640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4db40cf340 0x7f4db40c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:43.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.950+0000 7f4db3fff640 1 -- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40ceb40 msgr2=0x7f4db4150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:43.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.950+0000 7f4db3fff640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40ceb40 0x7f4db4150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:43.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.950+0000 7f4db3fff640 1 -- 172.21.15.125:0/1035425576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac009900 con 0x7f4db415b2a0 2023-10-18T11:58:43.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.950+0000 7f4db3fff640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db415b2a0 0x7f4db40b58f0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009bf0 tx=0x7f4dac035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:43.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.951+0000 7f4d9bfff640 1 -- 172.21.15.125:0/1035425576 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac034440 con 0x7f4db415b2a0 2023-10-18T11:58:43.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.951+0000 7f4d9bfff640 1 -- 172.21.15.125:0/1035425576 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4dac004250 con 0x7f4db415b2a0 2023-10-18T11:58:43.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.951+0000 7f4d9bfff640 1 -- 172.21.15.125:0/1035425576 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac004600 con 0x7f4db415b2a0 2023-10-18T11:58:43.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.951+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db40b0d50 con 0x7f4db415b2a0 2023-10-18T11:58:43.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.951+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db41513f0 con 0x7f4db415b2a0 2023-10-18T11:58:43.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.952+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db40690e0 con 0x7f4db415b2a0 2023-10-18T11:58:43.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.953+0000 7f4d9bfff640 1 -- 172.21.15.125:0/1035425576 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4dac00a040 con 0x7f4db415b2a0 2023-10-18T11:58:43.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.953+0000 7f4d9bfff640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4da006af60 0x7f4da006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:43.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.953+0000 7f4d9bfff640 1 -- 172.21.15.125:0/1035425576 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f4dac0ac7a0 con 0x7f4db415b2a0 2023-10-18T11:58:43.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.953+0000 7f4db37fe640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4da006af60 0x7f4da006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:43.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.954+0000 7f4db37fe640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4da006af60 0x7f4da006d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4da4001780 tx=0x7f4da4009040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:43.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:43.957+0000 7f4d9bfff640 1 -- 172.21.15.125:0/1035425576 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4dac077670 con 0x7f4db415b2a0 2023-10-18T11:58:44.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.120+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 --> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4db40c9100 con 0x7f4da006af60 2023-10-18T11:58:44.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.129+0000 7f4d9bfff640 1 -- 172.21.15.125:0/1035425576 <== mgr.9529 v2:172.21.15.139:6832/1565260611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289888 (secure 0 0 0) 0x7f4db40c9100 con 0x7f4da006af60 2023-10-18T11:58:44.134 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:44.135 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T11:58:44.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4da006af60 msgr2=0x7f4da006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4da006af60 0x7f4da006d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4da4001780 tx=0x7f4da4009040 comp rx=0 tx=0).stop 2023-10-18T11:58:44.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db415b2a0 msgr2=0x7f4db40b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db415b2a0 0x7f4db40b58f0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009bf0 tx=0x7f4dac035e30 comp rx=0 tx=0).stop 2023-10-18T11:58:44.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 shutdown_connections 2023-10-18T11:58:44.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4da006af60 0x7f4da006d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4db415b2a0 0x7f4db40b58f0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4db40cf340 0x7f4db40c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 --2- 172.21.15.125:0/1035425576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40ceb40 0x7f4db4150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 >> 172.21.15.125:0/1035425576 conn(0x7f4db40d0d60 msgr2=0x7f4db415a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:44.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 shutdown_connections 2023-10-18T11:58:44.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.132+0000 7f4dba15e640 1 -- 172.21.15.125:0/1035425576 wait complete. 2023-10-18T11:58:44.145 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T11:58:42.895257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":599,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406692,"kb_used_data":21224,"kb_used_omap":12,"kb_used_meta":385331,"kb_avail":754568028,"statfs":{"total":773094113280,"available":772677660672,"internally_reserved":0,"allocated":21733376,"data_stored":11625988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":394579512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.389979"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802702+0000","last_change":"2023-10-18T11:58:36.522565+0000","last_active":"2023-10-18T11:58:37.802702+0000","last_peered":"2023-10-18T11:58:37.802702+0000","last_clean":"2023-10-18T11:58:37.802702+0000","last_became_active":"2023-10-18T11:58:36.521736+0000","last_became_peered":"2023-10-18T11:58:36.521736+0000","last_unstale":"2023-10-18T11:58:37.802702+0000","last_undegraded":"2023-10-18T11:58:37.802702+0000","last_fullsized":"2023-10-18T11:58:37.802702+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:46:36.674788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802006+0000","last_change":"2023-10-18T11:58:36.527651+0000","last_active":"2023-10-18T11:58:37.802006+0000","last_peered":"2023-10-18T11:58:37.802006+0000","last_clean":"2023-10-18T11:58:37.802006+0000","last_became_active":"2023-10-18T11:58:36.527375+0000","last_became_peered":"2023-10-18T11:58:36.527375+0000","last_unstale":"2023-10-18T11:58:37.802006+0000","last_undegraded":"2023-10-18T11:58:37.802006+0000","last_fullsized":"2023-10-18T11:58:37.802006+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:55:29.683103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802620+0000","last_change":"2023-10-18T11:58:36.517119+0000","last_active":"2023-10-18T11:58:37.802620+0000","last_peered":"2023-10-18T11:58:37.802620+0000","last_clean":"2023-10-18T11:58:37.802620+0000","last_became_active":"2023-10-18T11:58:36.516748+0000","last_became_peered":"2023-10-18T11:58:36.516748+0000","last_unstale":"2023-10-18T11:58:37.802620+0000","last_undegraded":"2023-10-18T11:58:37.802620+0000","last_fullsized":"2023-10-18T11:58:37.802620+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:29:46.401140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.534333+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.534066+0000","last_became_peered":"2023-10-18T11:58:36.534066+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:29.879395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508872+0000","last_change":"2023-10-18T11:58:36.529615+0000","last_active":"2023-10-18T11:58:37.508872+0000","last_peered":"2023-10-18T11:58:37.508872+0000","last_clean":"2023-10-18T11:58:37.508872+0000","last_became_active":"2023-10-18T11:58:36.529380+0000","last_became_peered":"2023-10-18T11:58:36.529380+0000","last_unstale":"2023-10-18T11:58:37.508872+0000","last_undegraded":"2023-10-18T11:58:37.508872+0000","last_fullsized":"2023-10-18T11:58:37.508872+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:44:44.701593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508825+0000","last_change":"2023-10-18T11:58:36.528591+0000","last_active":"2023-10-18T11:58:37.508825+0000","last_peered":"2023-10-18T11:58:37.508825+0000","last_clean":"2023-10-18T11:58:37.508825+0000","last_became_active":"2023-10-18T11:58:36.528332+0000","last_became_peered":"2023-10-18T11:58:36.528332+0000","last_unstale":"2023-10-18T11:58:37.508825+0000","last_undegraded":"2023-10-18T11:58:37.508825+0000","last_fullsized":"2023-10-18T11:58:37.508825+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:04.968822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802608+0000","last_change":"2023-10-18T11:58:36.526452+0000","last_active":"2023-10-18T11:58:37.802608+0000","last_peered":"2023-10-18T11:58:37.802608+0000","last_clean":"2023-10-18T11:58:37.802608+0000","last_became_active":"2023-10-18T11:58:36.526144+0000","last_became_peered":"2023-10-18T11:58:36.526144+0000","last_unstale":"2023-10-18T11:58:37.802608+0000","last_undegraded":"2023-10-18T11:58:37.802608+0000","last_fullsized":"2023-10-18T11:58:37.802608+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:15.609770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802428+0000","last_change":"2023-10-18T11:58:36.526263+0000","last_active":"2023-10-18T11:58:37.802428+0000","last_peered":"2023-10-18T11:58:37.802428+0000","last_clean":"2023-10-18T11:58:37.802428+0000","last_became_active":"2023-10-18T11:58:36.525882+0000","last_became_peered":"2023-10-18T11:58:36.525882+0000","last_unstale":"2023-10-18T11:58:37.802428+0000","last_undegraded":"2023-10-18T11:58:37.802428+0000","last_fullsized":"2023-10-18T11:58:37.802428+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:05.910088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802812+0000","last_change":"2023-10-18T11:58:36.520303+0000","last_active":"2023-10-18T11:58:37.802812+0000","last_peered":"2023-10-18T11:58:37.802812+0000","last_clean":"2023-10-18T11:58:37.802812+0000","last_became_active":"2023-10-18T11:58:36.520065+0000","last_became_peered":"2023-10-18T11:58:36.520065+0000","last_unstale":"2023-10-18T11:58:37.802812+0000","last_undegraded":"2023-10-18T11:58:37.802812+0000","last_fullsized":"2023-10-18T11:58:37.802812+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:40.039597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509204+0000","last_change":"2023-10-18T11:58:36.534318+0000","last_active":"2023-10-18T11:58:37.509204+0000","last_peered":"2023-10-18T11:58:37.509204+0000","last_clean":"2023-10-18T11:58:37.509204+0000","last_became_active":"2023-10-18T11:58:36.534071+0000","last_became_peered":"2023-10-18T11:58:36.534071+0000","last_unstale":"2023-10-18T11:58:37.509204+0000","last_undegraded":"2023-10-18T11:58:37.509204+0000","last_fullsized":"2023-10-18T11:58:37.509204+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:55.491621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802310+0000","last_change":"2023-10-18T11:58:36.520313+0000","last_active":"2023-10-18T11:58:37.802310+0000","last_peered":"2023-10-18T11:58:37.802310+0000","last_clean":"2023-10-18T11:58:37.802310+0000","last_became_active":"2023-10-18T11:58:36.520042+0000","last_became_peered":"2023-10-18T11:58:36.520042+0000","last_unstale":"2023-10-18T11:58:37.802310+0000","last_undegraded":"2023-10-18T11:58:37.802310+0000","last_fullsized":"2023-10-18T11:58:37.802310+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:57.623658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802462+0000","last_change":"2023-10-18T11:58:36.526211+0000","last_active":"2023-10-18T11:58:37.802462+0000","last_peered":"2023-10-18T11:58:37.802462+0000","last_clean":"2023-10-18T11:58:37.802462+0000","last_became_active":"2023-10-18T11:58:36.525812+0000","last_became_peered":"2023-10-18T11:58:36.525812+0000","last_unstale":"2023-10-18T11:58:37.802462+0000","last_undegraded":"2023-10-18T11:58:37.802462+0000","last_fullsized":"2023-10-18T11:58:37.802462+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:39.938700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512711+0000","last_change":"2023-10-18T11:58:36.529732+0000","last_active":"2023-10-18T11:58:37.512711+0000","last_peered":"2023-10-18T11:58:37.512711+0000","last_clean":"2023-10-18T11:58:37.512711+0000","last_became_active":"2023-10-18T11:58:36.529455+0000","last_became_peered":"2023-10-18T11:58:36.529455+0000","last_unstale":"2023-10-18T11:58:37.512711+0000","last_undegraded":"2023-10-18T11:58:37.512711+0000","last_fullsized":"2023-10-18T11:58:37.512711+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:38.419107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802037+0000","last_change":"2023-10-18T11:58:36.534249+0000","last_active":"2023-10-18T11:58:37.802037+0000","last_peered":"2023-10-18T11:58:37.802037+0000","last_clean":"2023-10-18T11:58:37.802037+0000","last_became_active":"2023-10-18T11:58:36.533956+0000","last_became_peered":"2023-10-18T11:58:36.533956+0000","last_unstale":"2023-10-18T11:58:37.802037+0000","last_undegraded":"2023-10-18T11:58:37.802037+0000","last_fullsized":"2023-10-18T11:58:37.802037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:29.001140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802132+0000","last_change":"2023-10-18T11:58:36.530863+0000","last_active":"2023-10-18T11:58:37.802132+0000","last_peered":"2023-10-18T11:58:37.802132+0000","last_clean":"2023-10-18T11:58:37.802132+0000","last_became_active":"2023-10-18T11:58:36.530625+0000","last_became_peered":"2023-10-18T11:58:36.530625+0000","last_unstale":"2023-10-18T11:58:37.802132+0000","last_undegraded":"2023-10-18T11:58:37.802132+0000","last_fullsized":"2023-10-18T11:58:37.802132+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:03.443134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508994+0000","last_change":"2023-10-18T11:58:36.531552+0000","last_active":"2023-10-18T11:58:37.508994+0000","last_peered":"2023-10-18T11:58:37.508994+0000","last_clean":"2023-10-18T11:58:37.508994+0000","last_became_active":"2023-10-18T11:58:36.531163+0000","last_became_peered":"2023-10-18T11:58:36.531163+0000","last_unstale":"2023-10-18T11:58:37.508994+0000","last_undegraded":"2023-10-18T11:58:37.508994+0000","last_fullsized":"2023-10-18T11:58:37.508994+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:04:49.241771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509268+0000","last_change":"2023-10-18T11:58:36.530026+0000","last_active":"2023-10-18T11:58:37.509268+0000","last_peered":"2023-10-18T11:58:37.509268+0000","last_clean":"2023-10-18T11:58:37.509268+0000","last_became_active":"2023-10-18T11:58:36.529058+0000","last_became_peered":"2023-10-18T11:58:36.529058+0000","last_unstale":"2023-10-18T11:58:37.509268+0000","last_undegraded":"2023-10-18T11:58:37.509268+0000","last_fullsized":"2023-10-18T11:58:37.509268+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:41.289044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513034+0000","last_change":"2023-10-18T11:58:36.532114+0000","last_active":"2023-10-18T11:58:37.513034+0000","last_peered":"2023-10-18T11:58:37.513034+0000","last_clean":"2023-10-18T11:58:37.513034+0000","last_became_active":"2023-10-18T11:58:36.531526+0000","last_became_peered":"2023-10-18T11:58:36.531526+0000","last_unstale":"2023-10-18T11:58:37.513034+0000","last_undegraded":"2023-10-18T11:58:37.513034+0000","last_fullsized":"2023-10-18T11:58:37.513034+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:38.904126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512884+0000","last_change":"2023-10-18T11:58:36.534033+0000","last_active":"2023-10-18T11:58:37.512884+0000","last_peered":"2023-10-18T11:58:37.512884+0000","last_clean":"2023-10-18T11:58:37.512884+0000","last_became_active":"2023-10-18T11:58:36.533772+0000","last_became_peered":"2023-10-18T11:58:36.533772+0000","last_unstale":"2023-10-18T11:58:37.512884+0000","last_undegraded":"2023-10-18T11:58:37.512884+0000","last_fullsized":"2023-10-18T11:58:37.512884+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:04.963404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508988+0000","last_change":"2023-10-18T11:58:36.529660+0000","last_active":"2023-10-18T11:58:37.508988+0000","last_peered":"2023-10-18T11:58:37.508988+0000","last_clean":"2023-10-18T11:58:37.508988+0000","last_became_active":"2023-10-18T11:58:36.529125+0000","last_became_peered":"2023-10-18T11:58:36.529125+0000","last_unstale":"2023-10-18T11:58:37.508988+0000","last_undegraded":"2023-10-18T11:58:37.508988+0000","last_fullsized":"2023-10-18T11:58:37.508988+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:01.717669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802201+0000","last_change":"2023-10-18T11:58:36.519499+0000","last_active":"2023-10-18T11:58:37.802201+0000","last_peered":"2023-10-18T11:58:37.802201+0000","last_clean":"2023-10-18T11:58:37.802201+0000","last_became_active":"2023-10-18T11:58:36.519246+0000","last_became_peered":"2023-10-18T11:58:36.519246+0000","last_unstale":"2023-10-18T11:58:37.802201+0000","last_undegraded":"2023-10-18T11:58:37.802201+0000","last_fullsized":"2023-10-18T11:58:37.802201+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:48.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802474+0000","last_change":"2023-10-18T11:58:36.514398+0000","last_active":"2023-10-18T11:58:37.802474+0000","last_peered":"2023-10-18T11:58:37.802474+0000","last_clean":"2023-10-18T11:58:37.802474+0000","last_became_active":"2023-10-18T11:58:36.514141+0000","last_became_peered":"2023-10-18T11:58:36.514141+0000","last_unstale":"2023-10-18T11:58:37.802474+0000","last_undegraded":"2023-10-18T11:58:37.802474+0000","last_fullsized":"2023-10-18T11:58:37.802474+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:43.247625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802305+0000","last_change":"2023-10-18T11:58:36.527748+0000","last_active":"2023-10-18T11:58:37.802305+0000","last_peered":"2023-10-18T11:58:37.802305+0000","last_clean":"2023-10-18T11:58:37.802305+0000","last_became_active":"2023-10-18T11:58:36.527498+0000","last_became_peered":"2023-10-18T11:58:36.527498+0000","last_unstale":"2023-10-18T11:58:37.802305+0000","last_undegraded":"2023-10-18T11:58:37.802305+0000","last_fullsized":"2023-10-18T11:58:37.802305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:35.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801779+0000","last_change":"2023-10-18T11:58:36.521950+0000","last_active":"2023-10-18T11:58:37.801779+0000","last_peered":"2023-10-18T11:58:37.801779+0000","last_clean":"2023-10-18T11:58:37.801779+0000","last_became_active":"2023-10-18T11:58:36.521681+0000","last_became_peered":"2023-10-18T11:58:36.521681+0000","last_unstale":"2023-10-18T11:58:37.801779+0000","last_undegraded":"2023-10-18T11:58:37.801779+0000","last_fullsized":"2023-10-18T11:58:37.801779+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:43.982016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802739+0000","last_change":"2023-10-18T11:58:36.518867+0000","last_active":"2023-10-18T11:58:37.802739+0000","last_peered":"2023-10-18T11:58:37.802739+0000","last_clean":"2023-10-18T11:58:37.802739+0000","last_became_active":"2023-10-18T11:58:36.518625+0000","last_became_peered":"2023-10-18T11:58:36.518625+0000","last_unstale":"2023-10-18T11:58:37.802739+0000","last_undegraded":"2023-10-18T11:58:37.802739+0000","last_fullsized":"2023-10-18T11:58:37.802739+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:20.411248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802849+0000","last_change":"2023-10-18T11:58:36.528722+0000","last_active":"2023-10-18T11:58:37.802849+0000","last_peered":"2023-10-18T11:58:37.802849+0000","last_clean":"2023-10-18T11:58:37.802849+0000","last_became_active":"2023-10-18T11:58:36.528462+0000","last_became_peered":"2023-10-18T11:58:36.528462+0000","last_unstale":"2023-10-18T11:58:37.802849+0000","last_undegraded":"2023-10-18T11:58:37.802849+0000","last_fullsized":"2023-10-18T11:58:37.802849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:58:27.045894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803175+0000","last_change":"2023-10-18T11:58:36.527594+0000","last_active":"2023-10-18T11:58:37.803175+0000","last_peered":"2023-10-18T11:58:37.803175+0000","last_clean":"2023-10-18T11:58:37.803175+0000","last_became_active":"2023-10-18T11:58:36.526226+0000","last_became_peered":"2023-10-18T11:58:36.526226+0000","last_unstale":"2023-10-18T11:58:37.803175+0000","last_undegraded":"2023-10-18T11:58:37.803175+0000","last_fullsized":"2023-10-18T11:58:37.803175+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:28.446100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509068+0000","last_change":"2023-10-18T11:58:36.529803+0000","last_active":"2023-10-18T11:58:37.509068+0000","last_peered":"2023-10-18T11:58:37.509068+0000","last_clean":"2023-10-18T11:58:37.509068+0000","last_became_active":"2023-10-18T11:58:36.529446+0000","last_became_peered":"2023-10-18T11:58:36.529446+0000","last_unstale":"2023-10-18T11:58:37.509068+0000","last_undegraded":"2023-10-18T11:58:37.509068+0000","last_fullsized":"2023-10-18T11:58:37.509068+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:19.031434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802519+0000","last_change":"2023-10-18T11:58:36.526866+0000","last_active":"2023-10-18T11:58:37.802519+0000","last_peered":"2023-10-18T11:58:37.802519+0000","last_clean":"2023-10-18T11:58:37.802519+0000","last_became_active":"2023-10-18T11:58:36.526609+0000","last_became_peered":"2023-10-18T11:58:36.526609+0000","last_unstale":"2023-10-18T11:58:37.802519+0000","last_undegraded":"2023-10-18T11:58:37.802519+0000","last_fullsized":"2023-10-18T11:58:37.802519+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:37:04.860806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802258+0000","last_change":"2023-10-18T11:58:36.515793+0000","last_active":"2023-10-18T11:58:37.802258+0000","last_peered":"2023-10-18T11:58:37.802258+0000","last_clean":"2023-10-18T11:58:37.802258+0000","last_became_active":"2023-10-18T11:58:36.515523+0000","last_became_peered":"2023-10-18T11:58:36.515523+0000","last_unstale":"2023-10-18T11:58:37.802258+0000","last_undegraded":"2023-10-18T11:58:37.802258+0000","last_fullsized":"2023-10-18T11:58:37.802258+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802828+0000","last_change":"2023-10-18T11:58:36.515930+0000","last_active":"2023-10-18T11:58:37.802828+0000","last_peered":"2023-10-18T11:58:37.802828+0000","last_clean":"2023-10-18T11:58:37.802828+0000","last_became_active":"2023-10-18T11:58:36.515681+0000","last_became_peered":"2023-10-18T11:58:36.515681+0000","last_unstale":"2023-10-18T11:58:37.802828+0000","last_undegraded":"2023-10-18T11:58:37.802828+0000","last_fullsized":"2023-10-18T11:58:37.802828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:48:47.733918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508862+0000","last_change":"2023-10-18T11:58:36.530777+0000","last_active":"2023-10-18T11:58:37.508862+0000","last_peered":"2023-10-18T11:58:37.508862+0000","last_clean":"2023-10-18T11:58:37.508862+0000","last_became_active":"2023-10-18T11:58:36.530454+0000","last_became_peered":"2023-10-18T11:58:36.530454+0000","last_unstale":"2023-10-18T11:58:37.508862+0000","last_undegraded":"2023-10-18T11:58:37.508862+0000","last_fullsized":"2023-10-18T11:58:37.508862+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:00.542319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803195+0000","last_change":"2023-10-18T11:58:36.525494+0000","last_active":"2023-10-18T11:58:37.803195+0000","last_peered":"2023-10-18T11:58:37.803195+0000","last_clean":"2023-10-18T11:58:37.803195+0000","last_became_active":"2023-10-18T11:58:36.525200+0000","last_became_peered":"2023-10-18T11:58:36.525200+0000","last_unstale":"2023-10-18T11:58:37.803195+0000","last_undegraded":"2023-10-18T11:58:37.803195+0000","last_fullsized":"2023-10-18T11:58:37.803195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"203'4","reported_seq":16,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803001+0000","last_change":"2023-10-18T11:58:35.527076+0000","last_active":"2023-10-18T11:58:37.803001+0000","last_peered":"2023-10-18T11:58:37.803001+0000","last_clean":"2023-10-18T11:58:37.803001+0000","last_became_active":"2023-10-18T11:58:35.526809+0000","last_became_peered":"2023-10-18T11:58:35.526809+0000","last_unstale":"2023-10-18T11:58:37.803001+0000","last_undegraded":"2023-10-18T11:58:37.803001+0000","last_fullsized":"2023-10-18T11:58:37.803001+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802906+0000","last_change":"2023-10-18T11:58:36.521457+0000","last_active":"2023-10-18T11:58:37.802906+0000","last_peered":"2023-10-18T11:58:37.802906+0000","last_clean":"2023-10-18T11:58:37.802906+0000","last_became_active":"2023-10-18T11:58:36.521209+0000","last_became_peered":"2023-10-18T11:58:36.521209+0000","last_unstale":"2023-10-18T11:58:37.802906+0000","last_undegraded":"2023-10-18T11:58:37.802906+0000","last_fullsized":"2023-10-18T11:58:37.802906+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:40.559400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509267+0000","last_change":"2023-10-18T11:58:35.521828+0000","last_active":"2023-10-18T11:58:37.509267+0000","last_peered":"2023-10-18T11:58:37.509267+0000","last_clean":"2023-10-18T11:58:37.509267+0000","last_became_active":"2023-10-18T11:58:35.521574+0000","last_became_peered":"2023-10-18T11:58:35.521574+0000","last_unstale":"2023-10-18T11:58:37.509267+0000","last_undegraded":"2023-10-18T11:58:37.509267+0000","last_fullsized":"2023-10-18T11:58:37.509267+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:47.472064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803025+0000","last_change":"2023-10-18T11:58:36.522588+0000","last_active":"2023-10-18T11:58:37.803025+0000","last_peered":"2023-10-18T11:58:37.803025+0000","last_clean":"2023-10-18T11:58:37.803025+0000","last_became_active":"2023-10-18T11:58:36.522182+0000","last_became_peered":"2023-10-18T11:58:36.522182+0000","last_unstale":"2023-10-18T11:58:37.803025+0000","last_undegraded":"2023-10-18T11:58:37.803025+0000","last_fullsized":"2023-10-18T11:58:37.803025+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:21.326674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802389+0000","last_change":"2023-10-18T11:58:35.516307+0000","last_active":"2023-10-18T11:58:37.802389+0000","last_peered":"2023-10-18T11:58:37.802389+0000","last_clean":"2023-10-18T11:58:37.802389+0000","last_became_active":"2023-10-18T11:58:35.515995+0000","last_became_peered":"2023-10-18T11:58:35.515995+0000","last_unstale":"2023-10-18T11:58:37.802389+0000","last_undegraded":"2023-10-18T11:58:37.802389+0000","last_fullsized":"2023-10-18T11:58:37.802389+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:43.302735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802494+0000","last_change":"2023-10-18T11:58:36.518717+0000","last_active":"2023-10-18T11:58:37.802494+0000","last_peered":"2023-10-18T11:58:37.802494+0000","last_clean":"2023-10-18T11:58:37.802494+0000","last_became_active":"2023-10-18T11:58:36.518455+0000","last_became_peered":"2023-10-18T11:58:36.518455+0000","last_unstale":"2023-10-18T11:58:37.802494+0000","last_undegraded":"2023-10-18T11:58:37.802494+0000","last_fullsized":"2023-10-18T11:58:37.802494+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:01.297812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512924+0000","last_change":"2023-10-18T11:58:35.526193+0000","last_active":"2023-10-18T11:58:37.512924+0000","last_peered":"2023-10-18T11:58:37.512924+0000","last_clean":"2023-10-18T11:58:37.512924+0000","last_became_active":"2023-10-18T11:58:35.525920+0000","last_became_peered":"2023-10-18T11:58:35.525920+0000","last_unstale":"2023-10-18T11:58:37.512924+0000","last_undegraded":"2023-10-18T11:58:37.512924+0000","last_fullsized":"2023-10-18T11:58:37.512924+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:17.858763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512416+0000","last_change":"2023-10-18T11:58:36.529406+0000","last_active":"2023-10-18T11:58:37.512416+0000","last_peered":"2023-10-18T11:58:37.512416+0000","last_clean":"2023-10-18T11:58:37.512416+0000","last_became_active":"2023-10-18T11:58:36.528795+0000","last_became_peered":"2023-10-18T11:58:36.528795+0000","last_unstale":"2023-10-18T11:58:37.512416+0000","last_undegraded":"2023-10-18T11:58:37.512416+0000","last_fullsized":"2023-10-18T11:58:37.512416+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:26.184608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509045+0000","last_change":"2023-10-18T11:58:35.523280+0000","last_active":"2023-10-18T11:58:37.509045+0000","last_peered":"2023-10-18T11:58:37.509045+0000","last_clean":"2023-10-18T11:58:37.509045+0000","last_became_active":"2023-10-18T11:58:35.522621+0000","last_became_peered":"2023-10-18T11:58:35.522621+0000","last_unstale":"2023-10-18T11:58:37.509045+0000","last_undegraded":"2023-10-18T11:58:37.509045+0000","last_fullsized":"2023-10-18T11:58:37.509045+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:26:27.443746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802321+0000","last_change":"2023-10-18T11:58:36.516159+0000","last_active":"2023-10-18T11:58:37.802321+0000","last_peered":"2023-10-18T11:58:37.802321+0000","last_clean":"2023-10-18T11:58:37.802321+0000","last_became_active":"2023-10-18T11:58:36.515894+0000","last_became_peered":"2023-10-18T11:58:36.515894+0000","last_unstale":"2023-10-18T11:58:37.802321+0000","last_undegraded":"2023-10-18T11:58:37.802321+0000","last_fullsized":"2023-10-18T11:58:37.802321+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:42.645151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802627+0000","last_change":"2023-10-18T11:58:35.521126+0000","last_active":"2023-10-18T11:58:37.802627+0000","last_peered":"2023-10-18T11:58:37.802627+0000","last_clean":"2023-10-18T11:58:37.802627+0000","last_became_active":"2023-10-18T11:58:35.520861+0000","last_became_peered":"2023-10-18T11:58:35.520861+0000","last_unstale":"2023-10-18T11:58:37.802627+0000","last_undegraded":"2023-10-18T11:58:37.802627+0000","last_fullsized":"2023-10-18T11:58:37.802627+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802501+0000","last_change":"2023-10-18T11:58:36.531084+0000","last_active":"2023-10-18T11:58:37.802501+0000","last_peered":"2023-10-18T11:58:37.802501+0000","last_clean":"2023-10-18T11:58:37.802501+0000","last_became_active":"2023-10-18T11:58:36.530792+0000","last_became_peered":"2023-10-18T11:58:36.530792+0000","last_unstale":"2023-10-18T11:58:37.802501+0000","last_undegraded":"2023-10-18T11:58:37.802501+0000","last_fullsized":"2023-10-18T11:58:37.802501+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:37.483860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512533+0000","last_change":"2023-10-18T11:58:35.518246+0000","last_active":"2023-10-18T11:58:37.512533+0000","last_peered":"2023-10-18T11:58:37.512533+0000","last_clean":"2023-10-18T11:58:37.512533+0000","last_became_active":"2023-10-18T11:58:35.517986+0000","last_became_peered":"2023-10-18T11:58:35.517986+0000","last_unstale":"2023-10-18T11:58:37.512533+0000","last_undegraded":"2023-10-18T11:58:37.512533+0000","last_fullsized":"2023-10-18T11:58:37.512533+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:02:15.620821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803081+0000","last_change":"2023-10-18T11:58:36.533670+0000","last_active":"2023-10-18T11:58:37.803081+0000","last_peered":"2023-10-18T11:58:37.803081+0000","last_clean":"2023-10-18T11:58:37.803081+0000","last_became_active":"2023-10-18T11:58:36.533394+0000","last_became_peered":"2023-10-18T11:58:36.533394+0000","last_unstale":"2023-10-18T11:58:37.803081+0000","last_undegraded":"2023-10-18T11:58:37.803081+0000","last_fullsized":"2023-10-18T11:58:37.803081+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:21.396797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509229+0000","last_change":"2023-10-18T11:58:35.524709+0000","last_active":"2023-10-18T11:58:37.509229+0000","last_peered":"2023-10-18T11:58:37.509229+0000","last_clean":"2023-10-18T11:58:37.509229+0000","last_became_active":"2023-10-18T11:58:35.524471+0000","last_became_peered":"2023-10-18T11:58:35.524471+0000","last_unstale":"2023-10-18T11:58:37.509229+0000","last_undegraded":"2023-10-18T11:58:37.509229+0000","last_fullsized":"2023-10-18T11:58:37.509229+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:53.288380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509250+0000","last_change":"2023-10-18T11:58:36.523093+0000","last_active":"2023-10-18T11:58:37.509250+0000","last_peered":"2023-10-18T11:58:37.509250+0000","last_clean":"2023-10-18T11:58:37.509250+0000","last_became_active":"2023-10-18T11:58:36.522551+0000","last_became_peered":"2023-10-18T11:58:36.522551+0000","last_unstale":"2023-10-18T11:58:37.509250+0000","last_undegraded":"2023-10-18T11:58:37.509250+0000","last_fullsized":"2023-10-18T11:58:37.509250+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:22.715669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802085+0000","last_change":"2023-10-18T11:58:35.522093+0000","last_active":"2023-10-18T11:58:37.802085+0000","last_peered":"2023-10-18T11:58:37.802085+0000","last_clean":"2023-10-18T11:58:37.802085+0000","last_became_active":"2023-10-18T11:58:35.521805+0000","last_became_peered":"2023-10-18T11:58:35.521805+0000","last_unstale":"2023-10-18T11:58:37.802085+0000","last_undegraded":"2023-10-18T11:58:37.802085+0000","last_fullsized":"2023-10-18T11:58:37.802085+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:08.791931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513049+0000","last_change":"2023-10-18T11:58:36.531254+0000","last_active":"2023-10-18T11:58:37.513049+0000","last_peered":"2023-10-18T11:58:37.513049+0000","last_clean":"2023-10-18T11:58:37.513049+0000","last_became_active":"2023-10-18T11:58:36.530967+0000","last_became_peered":"2023-10-18T11:58:36.530967+0000","last_unstale":"2023-10-18T11:58:37.513049+0000","last_undegraded":"2023-10-18T11:58:37.513049+0000","last_fullsized":"2023-10-18T11:58:37.513049+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:01.297812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"203'3","reported_seq":15,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802710+0000","last_change":"2023-10-18T11:58:35.525207+0000","last_active":"2023-10-18T11:58:37.802710+0000","last_peered":"2023-10-18T11:58:37.802710+0000","last_clean":"2023-10-18T11:58:37.802710+0000","last_became_active":"2023-10-18T11:58:35.524946+0000","last_became_peered":"2023-10-18T11:58:35.524946+0000","last_unstale":"2023-10-18T11:58:37.802710+0000","last_undegraded":"2023-10-18T11:58:37.802710+0000","last_fullsized":"2023-10-18T11:58:37.802710+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802920+0000","last_change":"2023-10-18T11:58:36.516878+0000","last_active":"2023-10-18T11:58:37.802920+0000","last_peered":"2023-10-18T11:58:37.802920+0000","last_clean":"2023-10-18T11:58:37.802920+0000","last_became_active":"2023-10-18T11:58:36.516625+0000","last_became_peered":"2023-10-18T11:58:36.516625+0000","last_unstale":"2023-10-18T11:58:37.802920+0000","last_undegraded":"2023-10-18T11:58:37.802920+0000","last_fullsized":"2023-10-18T11:58:37.802920+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:59.525275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"203'6","reported_seq":18,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.513026+0000","last_change":"2023-10-18T11:58:35.518030+0000","last_active":"2023-10-18T11:58:37.513026+0000","last_peered":"2023-10-18T11:58:37.513026+0000","last_clean":"2023-10-18T11:58:37.513026+0000","last_became_active":"2023-10-18T11:58:35.517747+0000","last_became_peered":"2023-10-18T11:58:35.517747+0000","last_unstale":"2023-10-18T11:58:37.513026+0000","last_undegraded":"2023-10-18T11:58:37.513026+0000","last_fullsized":"2023-10-18T11:58:37.513026+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:14:02.335660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801895+0000","last_change":"2023-10-18T11:58:36.524788+0000","last_active":"2023-10-18T11:58:37.801895+0000","last_peered":"2023-10-18T11:58:37.801895+0000","last_clean":"2023-10-18T11:58:37.801895+0000","last_became_active":"2023-10-18T11:58:36.524547+0000","last_became_peered":"2023-10-18T11:58:36.524547+0000","last_unstale":"2023-10-18T11:58:37.801895+0000","last_undegraded":"2023-10-18T11:58:37.801895+0000","last_fullsized":"2023-10-18T11:58:37.801895+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:02.661591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512833+0000","last_change":"2023-10-18T11:58:35.521655+0000","last_active":"2023-10-18T11:58:37.512833+0000","last_peered":"2023-10-18T11:58:37.512833+0000","last_clean":"2023-10-18T11:58:37.512833+0000","last_became_active":"2023-10-18T11:58:35.521430+0000","last_became_peered":"2023-10-18T11:58:35.521430+0000","last_unstale":"2023-10-18T11:58:37.512833+0000","last_undegraded":"2023-10-18T11:58:37.512833+0000","last_fullsized":"2023-10-18T11:58:37.512833+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:21.628472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508795+0000","last_change":"2023-10-18T11:58:36.523743+0000","last_active":"2023-10-18T11:58:37.508795+0000","last_peered":"2023-10-18T11:58:37.508795+0000","last_clean":"2023-10-18T11:58:37.508795+0000","last_became_active":"2023-10-18T11:58:36.523492+0000","last_became_peered":"2023-10-18T11:58:36.523492+0000","last_unstale":"2023-10-18T11:58:37.508795+0000","last_undegraded":"2023-10-18T11:58:37.508795+0000","last_fullsized":"2023-10-18T11:58:37.508795+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:47:42.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802579+0000","last_change":"2023-10-18T11:58:35.522746+0000","last_active":"2023-10-18T11:58:37.802579+0000","last_peered":"2023-10-18T11:58:37.802579+0000","last_clean":"2023-10-18T11:58:37.802579+0000","last_became_active":"2023-10-18T11:58:35.522486+0000","last_became_peered":"2023-10-18T11:58:35.522486+0000","last_unstale":"2023-10-18T11:58:37.802579+0000","last_undegraded":"2023-10-18T11:58:37.802579+0000","last_fullsized":"2023-10-18T11:58:37.802579+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:07.062822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512576+0000","last_change":"2023-10-18T11:58:36.533848+0000","last_active":"2023-10-18T11:58:37.512576+0000","last_peered":"2023-10-18T11:58:37.512576+0000","last_clean":"2023-10-18T11:58:37.512576+0000","last_became_active":"2023-10-18T11:58:36.533565+0000","last_became_peered":"2023-10-18T11:58:36.533565+0000","last_unstale":"2023-10-18T11:58:37.512576+0000","last_undegraded":"2023-10-18T11:58:37.512576+0000","last_fullsized":"2023-10-18T11:58:37.512576+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:48.249824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802427+0000","last_change":"2023-10-18T11:58:35.517587+0000","last_active":"2023-10-18T11:58:37.802427+0000","last_peered":"2023-10-18T11:58:37.802427+0000","last_clean":"2023-10-18T11:58:37.802427+0000","last_became_active":"2023-10-18T11:58:35.517322+0000","last_became_peered":"2023-10-18T11:58:35.517322+0000","last_unstale":"2023-10-18T11:58:37.802427+0000","last_undegraded":"2023-10-18T11:58:37.802427+0000","last_fullsized":"2023-10-18T11:58:37.802427+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:34.567188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512733+0000","last_change":"2023-10-18T11:58:36.531687+0000","last_active":"2023-10-18T11:58:37.512733+0000","last_peered":"2023-10-18T11:58:37.512733+0000","last_clean":"2023-10-18T11:58:37.512733+0000","last_became_active":"2023-10-18T11:58:36.531411+0000","last_became_peered":"2023-10-18T11:58:36.531411+0000","last_unstale":"2023-10-18T11:58:37.512733+0000","last_undegraded":"2023-10-18T11:58:37.512733+0000","last_fullsized":"2023-10-18T11:58:37.512733+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:04.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"203'8","reported_seq":20,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802771+0000","last_change":"2023-10-18T11:58:35.515506+0000","last_active":"2023-10-18T11:58:37.802771+0000","last_peered":"2023-10-18T11:58:37.802771+0000","last_clean":"2023-10-18T11:58:37.802771+0000","last_became_active":"2023-10-18T11:58:35.515198+0000","last_became_peered":"2023-10-18T11:58:35.515198+0000","last_unstale":"2023-10-18T11:58:37.802771+0000","last_undegraded":"2023-10-18T11:58:37.802771+0000","last_fullsized":"2023-10-18T11:58:37.802771+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:00:10.509931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802367+0000","last_change":"2023-10-18T11:58:36.515935+0000","last_active":"2023-10-18T11:58:37.802367+0000","last_peered":"2023-10-18T11:58:37.802367+0000","last_clean":"2023-10-18T11:58:37.802367+0000","last_became_active":"2023-10-18T11:58:36.515722+0000","last_became_peered":"2023-10-18T11:58:36.515722+0000","last_unstale":"2023-10-18T11:58:37.802367+0000","last_undegraded":"2023-10-18T11:58:37.802367+0000","last_fullsized":"2023-10-18T11:58:37.802367+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:58.998126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:35.515482+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:35.515185+0000","last_became_peered":"2023-10-18T11:58:35.515185+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.149129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509119+0000","last_change":"2023-10-18T11:58:36.534430+0000","last_active":"2023-10-18T11:58:37.509119+0000","last_peered":"2023-10-18T11:58:37.509119+0000","last_clean":"2023-10-18T11:58:37.509119+0000","last_became_active":"2023-10-18T11:58:36.534167+0000","last_became_peered":"2023-10-18T11:58:36.534167+0000","last_unstale":"2023-10-18T11:58:37.509119+0000","last_undegraded":"2023-10-18T11:58:37.509119+0000","last_fullsized":"2023-10-18T11:58:37.509119+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:12:44.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802606+0000","last_change":"2023-10-18T11:58:35.522154+0000","last_active":"2023-10-18T11:58:37.802606+0000","last_peered":"2023-10-18T11:58:37.802606+0000","last_clean":"2023-10-18T11:58:37.802606+0000","last_became_active":"2023-10-18T11:58:35.521902+0000","last_became_peered":"2023-10-18T11:58:35.521902+0000","last_unstale":"2023-10-18T11:58:37.802606+0000","last_undegraded":"2023-10-18T11:58:37.802606+0000","last_fullsized":"2023-10-18T11:58:37.802606+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:27.447038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803092+0000","last_change":"2023-10-18T11:58:36.529258+0000","last_active":"2023-10-18T11:58:37.803092+0000","last_peered":"2023-10-18T11:58:37.803092+0000","last_clean":"2023-10-18T11:58:37.803092+0000","last_became_active":"2023-10-18T11:58:36.528468+0000","last_became_peered":"2023-10-18T11:58:36.528468+0000","last_unstale":"2023-10-18T11:58:37.803092+0000","last_undegraded":"2023-10-18T11:58:37.803092+0000","last_fullsized":"2023-10-18T11:58:37.803092+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:58.035451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"203'1","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802652+0000","last_change":"2023-10-18T11:58:35.513195+0000","last_active":"2023-10-18T11:58:37.802652+0000","last_peered":"2023-10-18T11:58:37.802652+0000","last_clean":"2023-10-18T11:58:37.802652+0000","last_became_active":"2023-10-18T11:58:35.512858+0000","last_became_peered":"2023-10-18T11:58:35.512858+0000","last_unstale":"2023-10-18T11:58:37.802652+0000","last_undegraded":"2023-10-18T11:58:37.802652+0000","last_fullsized":"2023-10-18T11:58:37.802652+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:03.029076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803037+0000","last_change":"2023-10-18T11:58:36.521908+0000","last_active":"2023-10-18T11:58:37.803037+0000","last_peered":"2023-10-18T11:58:37.803037+0000","last_clean":"2023-10-18T11:58:37.803037+0000","last_became_active":"2023-10-18T11:58:36.521381+0000","last_became_peered":"2023-10-18T11:58:36.521381+0000","last_unstale":"2023-10-18T11:58:37.803037+0000","last_undegraded":"2023-10-18T11:58:37.803037+0000","last_fullsized":"2023-10-18T11:58:37.803037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:18.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803110+0000","last_change":"2023-10-18T11:58:35.515862+0000","last_active":"2023-10-18T11:58:37.803110+0000","last_peered":"2023-10-18T11:58:37.803110+0000","last_clean":"2023-10-18T11:58:37.803110+0000","last_became_active":"2023-10-18T11:58:35.515521+0000","last_became_peered":"2023-10-18T11:58:35.515521+0000","last_unstale":"2023-10-18T11:58:37.803110+0000","last_undegraded":"2023-10-18T11:58:37.803110+0000","last_fullsized":"2023-10-18T11:58:37.803110+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:03.199289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801885+0000","last_change":"2023-10-18T11:58:36.520962+0000","last_active":"2023-10-18T11:58:37.801885+0000","last_peered":"2023-10-18T11:58:37.801885+0000","last_clean":"2023-10-18T11:58:37.801885+0000","last_became_active":"2023-10-18T11:58:36.520712+0000","last_became_peered":"2023-10-18T11:58:36.520712+0000","last_unstale":"2023-10-18T11:58:37.801885+0000","last_undegraded":"2023-10-18T11:58:37.801885+0000","last_fullsized":"2023-10-18T11:58:37.801885+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:46.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512865+0000","last_change":"2023-10-18T11:58:35.524483+0000","last_active":"2023-10-18T11:58:37.512865+0000","last_peered":"2023-10-18T11:58:37.512865+0000","last_clean":"2023-10-18T11:58:37.512865+0000","last_became_active":"2023-10-18T11:58:35.524231+0000","last_became_peered":"2023-10-18T11:58:35.524231+0000","last_unstale":"2023-10-18T11:58:37.512865+0000","last_undegraded":"2023-10-18T11:58:37.512865+0000","last_fullsized":"2023-10-18T11:58:37.512865+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:29:21.906133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802353+0000","last_change":"2023-10-18T11:58:36.529745+0000","last_active":"2023-10-18T11:58:37.802353+0000","last_peered":"2023-10-18T11:58:37.802353+0000","last_clean":"2023-10-18T11:58:37.802353+0000","last_became_active":"2023-10-18T11:58:36.529497+0000","last_became_peered":"2023-10-18T11:58:36.529497+0000","last_unstale":"2023-10-18T11:58:37.802353+0000","last_undegraded":"2023-10-18T11:58:37.802353+0000","last_fullsized":"2023-10-18T11:58:37.802353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:54.290735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802708+0000","last_change":"2023-10-18T11:58:35.526709+0000","last_active":"2023-10-18T11:58:37.802708+0000","last_peered":"2023-10-18T11:58:37.802708+0000","last_clean":"2023-10-18T11:58:37.802708+0000","last_became_active":"2023-10-18T11:58:35.526448+0000","last_became_peered":"2023-10-18T11:58:35.526448+0000","last_unstale":"2023-10-18T11:58:37.802708+0000","last_undegraded":"2023-10-18T11:58:37.802708+0000","last_fullsized":"2023-10-18T11:58:37.802708+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:20.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802541+0000","last_change":"2023-10-18T11:58:36.528368+0000","last_active":"2023-10-18T11:58:37.802541+0000","last_peered":"2023-10-18T11:58:37.802541+0000","last_clean":"2023-10-18T11:58:37.802541+0000","last_became_active":"2023-10-18T11:58:36.528124+0000","last_became_peered":"2023-10-18T11:58:36.528124+0000","last_unstale":"2023-10-18T11:58:37.802541+0000","last_undegraded":"2023-10-18T11:58:37.802541+0000","last_fullsized":"2023-10-18T11:58:37.802541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:49:48.474419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802873+0000","last_change":"2023-10-18T11:58:35.525781+0000","last_active":"2023-10-18T11:58:37.802873+0000","last_peered":"2023-10-18T11:58:37.802873+0000","last_clean":"2023-10-18T11:58:37.802873+0000","last_became_active":"2023-10-18T11:58:35.525519+0000","last_became_peered":"2023-10-18T11:58:35.525519+0000","last_unstale":"2023-10-18T11:58:37.802873+0000","last_undegraded":"2023-10-18T11:58:37.802873+0000","last_fullsized":"2023-10-18T11:58:37.802873+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:47:37.957986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.508969+0000","last_change":"2023-10-18T11:58:36.518933+0000","last_active":"2023-10-18T11:58:37.508969+0000","last_peered":"2023-10-18T11:58:37.508969+0000","last_clean":"2023-10-18T11:58:37.508969+0000","last_became_active":"2023-10-18T11:58:36.518691+0000","last_became_peered":"2023-10-18T11:58:36.518691+0000","last_unstale":"2023-10-18T11:58:37.508969+0000","last_undegraded":"2023-10-18T11:58:37.508969+0000","last_fullsized":"2023-10-18T11:58:37.508969+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:55.005558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802210+0000","last_change":"2023-10-18T11:58:35.518136+0000","last_active":"2023-10-18T11:58:37.802210+0000","last_peered":"2023-10-18T11:58:37.802210+0000","last_clean":"2023-10-18T11:58:37.802210+0000","last_became_active":"2023-10-18T11:58:35.517842+0000","last_became_peered":"2023-10-18T11:58:35.517842+0000","last_unstale":"2023-10-18T11:58:37.802210+0000","last_undegraded":"2023-10-18T11:58:37.802210+0000","last_fullsized":"2023-10-18T11:58:37.802210+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:37.416753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802476+0000","last_change":"2023-10-18T11:58:36.522037+0000","last_active":"2023-10-18T11:58:37.802476+0000","last_peered":"2023-10-18T11:58:37.802476+0000","last_clean":"2023-10-18T11:58:37.802476+0000","last_became_active":"2023-10-18T11:58:36.521783+0000","last_became_peered":"2023-10-18T11:58:36.521783+0000","last_unstale":"2023-10-18T11:58:37.802476+0000","last_undegraded":"2023-10-18T11:58:37.802476+0000","last_fullsized":"2023-10-18T11:58:37.802476+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:37:11.613406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"203'2","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.803083+0000","last_change":"2023-10-18T11:58:35.518886+0000","last_active":"2023-10-18T11:58:37.803083+0000","last_peered":"2023-10-18T11:58:37.803083+0000","last_clean":"2023-10-18T11:58:37.803083+0000","last_became_active":"2023-10-18T11:58:35.518591+0000","last_became_peered":"2023-10-18T11:58:35.518591+0000","last_unstale":"2023-10-18T11:58:37.803083+0000","last_undegraded":"2023-10-18T11:58:37.803083+0000","last_fullsized":"2023-10-18T11:58:37.803083+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:49.481190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802223+0000","last_change":"2023-10-18T11:58:36.519762+0000","last_active":"2023-10-18T11:58:37.802223+0000","last_peered":"2023-10-18T11:58:37.802223+0000","last_clean":"2023-10-18T11:58:37.802223+0000","last_became_active":"2023-10-18T11:58:36.517560+0000","last_became_peered":"2023-10-18T11:58:36.517560+0000","last_unstale":"2023-10-18T11:58:37.802223+0000","last_undegraded":"2023-10-18T11:58:37.802223+0000","last_fullsized":"2023-10-18T11:58:37.802223+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:29.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802645+0000","last_change":"2023-10-18T11:58:35.522798+0000","last_active":"2023-10-18T11:58:37.802645+0000","last_peered":"2023-10-18T11:58:37.802645+0000","last_clean":"2023-10-18T11:58:37.802645+0000","last_became_active":"2023-10-18T11:58:35.522532+0000","last_became_peered":"2023-10-18T11:58:35.522532+0000","last_unstale":"2023-10-18T11:58:37.802645+0000","last_undegraded":"2023-10-18T11:58:37.802645+0000","last_fullsized":"2023-10-18T11:58:37.802645+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:40:51.705544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802663+0000","last_change":"2023-10-18T11:58:36.530354+0000","last_active":"2023-10-18T11:58:37.802663+0000","last_peered":"2023-10-18T11:58:37.802663+0000","last_clean":"2023-10-18T11:58:37.802663+0000","last_became_active":"2023-10-18T11:58:36.530080+0000","last_became_peered":"2023-10-18T11:58:36.530080+0000","last_unstale":"2023-10-18T11:58:37.802663+0000","last_undegraded":"2023-10-18T11:58:37.802663+0000","last_fullsized":"2023-10-18T11:58:37.802663+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:19.995868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509241+0000","last_change":"2023-10-18T11:58:35.523100+0000","last_active":"2023-10-18T11:58:37.509241+0000","last_peered":"2023-10-18T11:58:37.509241+0000","last_clean":"2023-10-18T11:58:37.509241+0000","last_became_active":"2023-10-18T11:58:35.522802+0000","last_became_peered":"2023-10-18T11:58:35.522802+0000","last_unstale":"2023-10-18T11:58:37.509241+0000","last_undegraded":"2023-10-18T11:58:37.509241+0000","last_fullsized":"2023-10-18T11:58:37.509241+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.993513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802916+0000","last_change":"2023-10-18T11:58:36.522886+0000","last_active":"2023-10-18T11:58:37.802916+0000","last_peered":"2023-10-18T11:58:37.802916+0000","last_clean":"2023-10-18T11:58:37.802916+0000","last_became_active":"2023-10-18T11:58:36.522614+0000","last_became_peered":"2023-10-18T11:58:36.522614+0000","last_unstale":"2023-10-18T11:58:37.802916+0000","last_undegraded":"2023-10-18T11:58:37.802916+0000","last_fullsized":"2023-10-18T11:58:37.802916+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:16:52.212999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.509088+0000","last_change":"2023-10-18T11:58:35.523409+0000","last_active":"2023-10-18T11:58:37.509088+0000","last_peered":"2023-10-18T11:58:37.509088+0000","last_clean":"2023-10-18T11:58:37.509088+0000","last_became_active":"2023-10-18T11:58:35.523146+0000","last_became_peered":"2023-10-18T11:58:35.523146+0000","last_unstale":"2023-10-18T11:58:37.509088+0000","last_undegraded":"2023-10-18T11:58:37.509088+0000","last_fullsized":"2023-10-18T11:58:37.509088+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:51.791758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802128+0000","last_change":"2023-10-18T11:58:36.518512+0000","last_active":"2023-10-18T11:58:37.802128+0000","last_peered":"2023-10-18T11:58:37.802128+0000","last_clean":"2023-10-18T11:58:37.802128+0000","last_became_active":"2023-10-18T11:58:36.518258+0000","last_became_peered":"2023-10-18T11:58:36.518258+0000","last_unstale":"2023-10-18T11:58:37.802128+0000","last_undegraded":"2023-10-18T11:58:37.802128+0000","last_fullsized":"2023-10-18T11:58:37.802128+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:38.904126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802959+0000","last_change":"2023-10-18T11:58:35.522529+0000","last_active":"2023-10-18T11:58:37.802959+0000","last_peered":"2023-10-18T11:58:37.802959+0000","last_clean":"2023-10-18T11:58:37.802959+0000","last_became_active":"2023-10-18T11:58:35.522261+0000","last_became_peered":"2023-10-18T11:58:35.522261+0000","last_unstale":"2023-10-18T11:58:37.802959+0000","last_undegraded":"2023-10-18T11:58:37.802959+0000","last_fullsized":"2023-10-18T11:58:37.802959+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:14.194166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.512534+0000","last_change":"2023-10-18T11:58:36.530248+0000","last_active":"2023-10-18T11:58:37.512534+0000","last_peered":"2023-10-18T11:58:37.512534+0000","last_clean":"2023-10-18T11:58:37.512534+0000","last_became_active":"2023-10-18T11:58:36.529997+0000","last_became_peered":"2023-10-18T11:58:36.529997+0000","last_unstale":"2023-10-18T11:58:37.512534+0000","last_undegraded":"2023-10-18T11:58:37.512534+0000","last_fullsized":"2023-10-18T11:58:37.512534+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:58.971449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802154+0000","last_change":"2023-10-18T11:58:35.527290+0000","last_active":"2023-10-18T11:58:37.802154+0000","last_peered":"2023-10-18T11:58:37.802154+0000","last_clean":"2023-10-18T11:58:37.802154+0000","last_became_active":"2023-10-18T11:58:35.527052+0000","last_became_peered":"2023-10-18T11:58:35.527052+0000","last_unstale":"2023-10-18T11:58:37.802154+0000","last_undegraded":"2023-10-18T11:58:37.802154+0000","last_fullsized":"2023-10-18T11:58:37.802154+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:17.894282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802594+0000","last_change":"2023-10-18T11:58:36.520551+0000","last_active":"2023-10-18T11:58:37.802594+0000","last_peered":"2023-10-18T11:58:37.802594+0000","last_clean":"2023-10-18T11:58:37.802594+0000","last_became_active":"2023-10-18T11:58:36.520297+0000","last_became_peered":"2023-10-18T11:58:36.520297+0000","last_unstale":"2023-10-18T11:58:37.802594+0000","last_undegraded":"2023-10-18T11:58:37.802594+0000","last_fullsized":"2023-10-18T11:58:37.802594+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:11:25.536175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802945+0000","last_change":"2023-10-18T11:58:35.526011+0000","last_active":"2023-10-18T11:58:37.802945+0000","last_peered":"2023-10-18T11:58:37.802945+0000","last_clean":"2023-10-18T11:58:37.802945+0000","last_became_active":"2023-10-18T11:58:35.525734+0000","last_became_peered":"2023-10-18T11:58:35.525734+0000","last_unstale":"2023-10-18T11:58:37.802945+0000","last_undegraded":"2023-10-18T11:58:37.802945+0000","last_fullsized":"2023-10-18T11:58:37.802945+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:19.352633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802222+0000","last_change":"2023-10-18T11:58:36.528280+0000","last_active":"2023-10-18T11:58:37.802222+0000","last_peered":"2023-10-18T11:58:37.802222+0000","last_clean":"2023-10-18T11:58:37.802222+0000","last_became_active":"2023-10-18T11:58:36.528012+0000","last_became_peered":"2023-10-18T11:58:36.528012+0000","last_unstale":"2023-10-18T11:58:37.802222+0000","last_undegraded":"2023-10-18T11:58:37.802222+0000","last_fullsized":"2023-10-18T11:58:37.802222+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:28.449393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802284+0000","last_change":"2023-10-18T11:58:35.520341+0000","last_active":"2023-10-18T11:58:37.802284+0000","last_peered":"2023-10-18T11:58:37.802284+0000","last_clean":"2023-10-18T11:58:37.802284+0000","last_became_active":"2023-10-18T11:58:35.520057+0000","last_became_peered":"2023-10-18T11:58:35.520057+0000","last_unstale":"2023-10-18T11:58:37.802284+0000","last_undegraded":"2023-10-18T11:58:37.802284+0000","last_fullsized":"2023-10-18T11:58:37.802284+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:03.961049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.802548+0000","last_change":"2023-10-18T11:58:36.521527+0000","last_active":"2023-10-18T11:58:37.802548+0000","last_peered":"2023-10-18T11:58:37.802548+0000","last_clean":"2023-10-18T11:58:37.802548+0000","last_became_active":"2023-10-18T11:58:36.521270+0000","last_became_peered":"2023-10-18T11:58:36.521270+0000","last_unstale":"2023-10-18T11:58:37.802548+0000","last_undegraded":"2023-10-18T11:58:37.802548+0000","last_fullsized":"2023-10-18T11:58:37.802548+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:35.499419+0000","last_clean_scrub_stamp":"2023-10-18T11:58:35.499419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:19:38.001818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-10-18T11:58:37.801952+0000","last_change":"2023-10-18T11:58:35.513693+0000","last_active":"2023-10-18T11:58:37.801952+0000","last_peered":"2023-10-18T11:58:37.801952+0000","last_clean":"2023-10-18T11:58:37.801952+0000","last_became_active":"2023-10-18T11:58:35.513429+0000","last_became_peered":"2023-10-18T11:58:35.513429+0000","last_unstale":"2023-10-18T11:58:37.801952+0000","last_undegraded":"2023-10-18T11:58:37.801952+0000","last_fullsized":"2023-10-18T11:58:37.801952+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":201,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T11:58:34.497064+0000","last_clean_scrub_stamp":"2023-10-18T11:58:34.497064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:12.442948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"193'335","reported_seq":1140,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-10-18T11:58:36.518008+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T11:58:36.518008+0000","last_peered":"2023-10-18T11:58:36.518008+0000","last_clean":"2023-10-18T11:58:36.518008+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T11:58:36.518008+0000","last_undegraded":"2023-10-18T11:58:36.518008+0000","last_fullsized":"2023-10-18T11:58:36.518008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2018,"num_read_kb":4205,"num_write":555,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52308,"kb_used_data":3080,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319532,"statfs":{"total":96636764160,"available":96583200768,"internally_reserved":0,"allocated":3153920,"data_stored":1895422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.363},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74399999999999999,"5min":0.77200000000000002,"15min":0.84099999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.39700000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":1.298},"last":0.38}]},{"osd":3,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.82699999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.62,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.35699999999999998},"max":{"1min":0.80200000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45200000000000001,"15min":0.44900000000000001},"min":{"1min":0.34000000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.61299999999999999,"5min":0.80200000000000005,"15min":1.5249999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.27700000000000002},"max":{"1min":0.63500000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.78900000000000003,"15min":0.90800000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.375,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":0.82699999999999996,"15min":1.4179999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":1.478},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.874},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42008,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329832,"statfs":{"total":96636764160,"available":96593747968,"internally_reserved":0,"allocated":2564096,"data_stored":1305124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.50700000000000001,"5min":0.41899999999999998,"15min":0.29899999999999999},"max":{"1min":0.78100000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.497}]},{"osd":1,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.27900000000000003},"max":{"1min":0.76700000000000002,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.87},"last":0.65400000000000003}]},{"osd":2,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.625,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.25600000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.82599999999999996},"last":0.61299999999999999}]},{"osd":3,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.624,"15min":0.626},"min":{"1min":0.47499999999999998,"5min":0.39800000000000002,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.57099999999999995}]},{"osd":4,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47299999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.76900000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.71999999999999997,"15min":0.82999999999999996},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Oct 18 11:58:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.81200000000000006},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72699999999999998,"5min":0.80600000000000005,"15min":0.84599999999999997},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575970,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60068,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311772,"statfs":{"total":96636764160,"available":96575254528,"internally_reserved":0,"allocated":2576384,"data_stored":1306688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.249},"max":{"1min":0.84099999999999997,"5min":0.90300000000000002,"15min":0.90800000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.50700000000000001,"5min":0.38500000000000001,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.95899999999999996,"15min":0.96099999999999997},"last":0.57899999999999996}]},{"osd":1,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.24299999999999999},"max":{"1min":1.081,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82199999999999995,"15min":0.754},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.28399999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.98699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":0.84999999999999998,"15min":0.98399999999999999},"last":0.439}]},{"osd":3,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.53200000000000003,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":1.0349999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.501,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81100000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":1,"15min":1},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.30299999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.98499999999999999,"15min":1.01},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.93899999999999995,"15min":0.97999999999999998},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83699999999999997,"5min":1.101,"15min":1.101},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.34000000000000002},"max":{"1min":0.83399999999999996,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575970,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29544,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342296,"statfs":{"total":96636764160,"available":96606511104,"internally_reserved":0,"allocated":2576384,"data_stored":1306700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.62},"min":{"1min":0.45400000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80700000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.80200000000000005,"15min":0.85399999999999998},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69499999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.372,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.752,"5min":1.123,"15min":1.123},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.92100000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.93600000000000005},"last":0.629}]},{"osd":3,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83499999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.312},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Oct 18 11:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.34599999999999997,"15min":0.31900000000000001},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":2568192,"data_stored":1305214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.221},"max":{"1min":0.57099999999999995,"5min":0.61599999999999999,"15min":0.63300000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.28100000000000003,"15min":0.23400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.374}]},{"osd":1,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26400000000000001},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.56000000000000005,"5min":0.60599999999999998,"15min":0.65000000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.73399999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.052},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47499999999999998,"15min":0.40999999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67800000000000005,"15min":0.66900000000000004},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":1.026},"last":0.625}]},{"osd":6,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.437},"max":{"1min":0.82299999999999995,"5min":0.93100000000000005,"15min":0.94499999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69799999999999995,"15min":0.68700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.45400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Oct 18 11:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.55600000000000005,"5min":0.46400000000000002,"15min":0.442},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.58099999999999996,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.875,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.67800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60060,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311780,"statfs":{"total":96636764160,"available":96575262720,"internally_reserved":0,"allocated":2564096,"data_stored":1305636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.252,"5min":0.222,"15min":0.216},"max":{"1min":0.64500000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.67200000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.27900000000000003,"5min":0.26200000000000001,"15min":0.25800000000000001},"max":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.746,"15min":0.746},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.435,"15min":0.432},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.374},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35399999999999998,"5min":0.29899999999999999,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.33600000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68100000000000005},"min":{"1min":0.499,"5min":0.442,"15min":0.437},"max":{"1min":0.80700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.47799999999999998,"15min":0.41699999999999998},"max":{"1min":0.80900000000000005,"5min":1.087,"15min":1.087},"last":0.435}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.54500000000000004,"5min":0.46500000000000002,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66600000000000004,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.45200000000000001,"15min":0.40999999999999998},"max":{"1min":0.85799999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.46200000000000002,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68500000000000005,"15min":0.68200000000000005},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.45100000000000001},"max":{"1min":0.84099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.443,"15min":0.42499999999999999},"max":{"1min":0.86799999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.53900000000000003,"5min":0.443,"15min":0.437},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":1.01},"last":0.499}]}]},{"osd":1,"up_from":13,"seq":55834575970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60048,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311792,"statfs":{"total":96636764160,"available":96575275008,"internally_reserved":0,"allocated":2560000,"data_stored":1305054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.182},"max":{"1min":0.73899999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.247,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65800000000000003,"5min":0.81000000000000005,"15min":0.85299999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.23000000000000001},"max":{"1min":0.496,"5min":0.877,"15min":0.877},"last":0.28000000000000003}]},{"osd":3,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.69799999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.377},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51400000000000001},"min":{"1min":0.28899999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.74399999999999999,"5min":0.79800000000000004,"15min":0.91000000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.74099999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.65600000000000003},"min":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.29599999999999999},"max":{"1min":0.80300000000000005,"5min":0.86599999999999999,"15min":1.2390000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.60899999999999999},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.316},"max":{"1min":0.70299999999999996,"5min":1.05,"15min":1.212},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.75600000000000001,"5min":1.0209999999999999,"15min":1.0589999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77600000000000002,"5min":0.90600000000000003,"15min":1.0389999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Oct 18 11:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":0.97499999999999998,"15min":1.0109999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.47399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82499999999999996,"5min":0.88400000000000001,"15min":1.0620000000000001},"last":0.69899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42600,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329240,"statfs":{"total":96636764160,"available":96593141760,"internally_reserved":0,"allocated":3170304,"data_stored":1896150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.26600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56000000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42199999999999999,"15min":0.42299999999999999},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.57199999999999995,"5min":0.64400000000000002,"15min":0.66200000000000003},"last":0.247}]},{"osd":2,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.33700000000000002,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.433,"15min":0.42799999999999999},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.25700000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66200000000000003},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.36699999999999999},"max":{"1min":0.876,"5min":0.96999999999999997,"15min":0.997},"last":0.495},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.50900000000000001,"5min":0.45400000000000001,"15min":0.39100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93300000000000005,"15min":0.93999999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.504,"5min":0.439,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.95799999999999996,"15min":0.996},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.44700000000000001},"max":{"1min":0.89300000000000002,"5min":1.0089999999999999,"15min":1.038},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.48599999999999999,"5min":0.44900000000000001,"15min":0.44800000000000001},"max":{"1min":0.91100000000000003,"5min":0.95099999999999996,"15min":1.0609999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:58:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85599999999999998,"5min":1.0109999999999999,"15min":1.0189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48799999999999999,"15min":0.42999999999999999},"max":{"1min":0.89400000000000002,"5min":0.997,"15min":0.997},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T11:58:44.151 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T11:58:44.151 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T11:58:44.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.292+0000 7f6533475640 1 -- 172.21.15.125:0/1527510460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0abc10 msgr2=0x7f652c0c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.292+0000 7f6533475640 1 --2- 172.21.15.125:0/1527510460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0abc10 0x7f652c0c9b60 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f6528009830 tx=0x7f652802bcd0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.292+0000 7f6533475640 1 -- 172.21.15.125:0/1527510460 shutdown_connections 2023-10-18T11:58:44.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.292+0000 7f6533475640 1 --2- 172.21.15.125:0/1527510460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0abc10 0x7f652c0c9b60 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.292+0000 7f6533475640 1 --2- 172.21.15.125:0/1527510460 >> v2:172.21.15.139:3300/0 conn(0x7f652c06af00 0x7f652c06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.292+0000 7f6533475640 1 -- 172.21.15.125:0/1527510460 >> 172.21.15.125:0/1527510460 conn(0x7f652c069930 msgr2=0x7f652c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:44.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.292+0000 7f6533475640 1 -- 172.21.15.125:0/1527510460 shutdown_connections 2023-10-18T11:58:44.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.292+0000 7f6533475640 1 -- 172.21.15.125:0/1527510460 wait complete. 2023-10-18T11:58:44.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.293+0000 7f6533475640 1 Processor -- start 2023-10-18T11:58:44.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.293+0000 7f6533475640 1 -- start start 2023-10-18T11:58:44.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6533475640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f652c06af00 0x7f652c0a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:44.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6533475640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652c0abc10 0x7f652c0a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6533475640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0c96b0 0x7f652c1f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6533475640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c0af4f0 con 0x7f652c0c96b0 2023-10-18T11:58:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6533475640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c0af7d0 con 0x7f652c06af00 2023-10-18T11:58:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6533475640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652c0afab0 con 0x7f652c0abc10 2023-10-18T11:58:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0c96b0 0x7f652c1f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:44.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0c96b0 0x7f652c1f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50086/0 (socket says 172.21.15.125:50086) 2023-10-18T11:58:44.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 -- 172.21.15.125:0/1087822462 learned_addr learned my addr 172.21.15.125:0/1087822462 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:44.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531211640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f652c06af00 0x7f652c0a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:44.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6530a10640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652c0abc10 0x7f652c0a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 -- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652c0abc10 msgr2=0x7f652c0a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652c0abc10 0x7f652c0a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 -- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f652c06af00 msgr2=0x7f652c0a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f652c06af00 0x7f652c0a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 -- 172.21.15.125:0/1087822462 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6528009490 con 0x7f652c0c96b0 2023-10-18T11:58:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6531a12640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0c96b0 0x7f652c1f2750 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f6528009950 tx=0x7f65280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6521ffb640 1 -- 172.21.15.125:0/1087822462 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528003ea0 con 0x7f652c0c96b0 2023-10-18T11:58:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6521ffb640 1 -- 172.21.15.125:0/1087822462 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6528005110 con 0x7f652c0c96b0 2023-10-18T11:58:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6521ffb640 1 -- 172.21.15.125:0/1087822462 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528005500 con 0x7f652c0c96b0 2023-10-18T11:58:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.294+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c0afd90 con 0x7f652c0c96b0 2023-10-18T11:58:44.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.295+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c1f2c90 con 0x7f652c0c96b0 2023-10-18T11:58:44.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.296+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652c159c10 con 0x7f652c0c96b0 2023-10-18T11:58:44.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.296+0000 7f6521ffb640 1 -- 172.21.15.125:0/1087822462 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f65280047a0 con 0x7f652c0c96b0 2023-10-18T11:58:44.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.296+0000 7f6521ffb640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f651006af60 0x7f651006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:44.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.296+0000 7f6521ffb640 1 -- 172.21.15.125:0/1087822462 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f65280abc40 con 0x7f652c0c96b0 2023-10-18T11:58:44.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.297+0000 7f6531211640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f651006af60 0x7f651006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:44.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.297+0000 7f6531211640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f651006af60 0x7f651006d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6524005fd0 tx=0x7f6524009680 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:44.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.301+0000 7f6521ffb640 1 -- 172.21.15.125:0/1087822462 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6528076ba0 con 0x7f652c0c96b0 2023-10-18T11:58:44.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.534+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f652c0690e0 con 0x7f652c0c96b0 2023-10-18T11:58:44.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.536+0000 7f6521ffb640 1 -- 172.21.15.125:0/1087822462 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v56) v1 ==== 190+0+147 (secure 0 0 0) 0x7f65280390e0 con 0x7f652c0c96b0 2023-10-18T11:58:44.538 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T11:58:44.539 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T11:58:44.539 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T11:58:44.539 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T11:58:44.539 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T11:58:44.540 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T11:58:44.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f651006af60 msgr2=0x7f651006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f651006af60 0x7f651006d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6524005fd0 tx=0x7f6524009680 comp rx=0 tx=0).stop 2023-10-18T11:58:44.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0c96b0 msgr2=0x7f652c1f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0c96b0 0x7f652c1f2750 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f6528009950 tx=0x7f65280038a0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 shutdown_connections 2023-10-18T11:58:44.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f651006af60 0x7f651006d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f652c0c96b0 0x7f652c1f2750 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652c0abc10 0x7f652c0a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 --2- 172.21.15.125:0/1087822462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f652c06af00 0x7f652c0a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 >> 172.21.15.125:0/1087822462 conn(0x7f652c069930 msgr2=0x7f652c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:44.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 shutdown_connections 2023-10-18T11:58:44.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.538+0000 7f6533475640 1 -- 172.21.15.125:0/1087822462 wait complete. 2023-10-18T11:58:44.551 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T11:58:44.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.652+0000 7f45f57d0640 1 -- 172.21.15.125:0/3956274206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 msgr2=0x7f45dc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.652+0000 7f45f57d0640 1 --2- 172.21.15.125:0/3956274206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 0x7f45dc003d90 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f45e0001050 tx=0x7f45dc0057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.652+0000 7f45ef7fe640 1 -- 172.21.15.125:0/3956274206 reap_dead start 2023-10-18T11:58:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.652+0000 7f45f57d0640 1 -- 172.21.15.125:0/3956274206 shutdown_connections 2023-10-18T11:58:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.652+0000 7f45f57d0640 1 -- 172.21.15.125:0/3956274206 >> 172.21.15.125:0/3956274206 conn(0x7f45f00b3f00 msgr2=0x7f45f00b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:44.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.652+0000 7f45f57d0640 1 -- 172.21.15.125:0/3956274206 shutdown_connections 2023-10-18T11:58:44.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.652+0000 7f45f57d0640 1 -- 172.21.15.125:0/3956274206 wait complete. 2023-10-18T11:58:44.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.653+0000 7f45f57d0640 1 Processor -- start 2023-10-18T11:58:44.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.653+0000 7f45f57d0640 1 -- start start 2023-10-18T11:58:44.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45f57d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 0x7f45f01f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:44.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45f57d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f01484c0 0x7f45f0148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:44.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45f57d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f01ef5b0 0x7f45f01efa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:44.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45f57d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f0147050 con 0x7f45dc003970 2023-10-18T11:58:44.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45f57d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f01464b0 con 0x7f45f01484c0 2023-10-18T11:58:44.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45f57d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f0148e60 con 0x7f45f01ef5b0 2023-10-18T11:58:44.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 0x7f45f01f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:44.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 0x7f45f01f04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50108/0 (socket says 172.21.15.125:50108) 2023-10-18T11:58:44.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45eeffd640 1 -- 172.21.15.125:0/3972229048 learned_addr learned my addr 172.21.15.125:0/3972229048 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:44.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45eeffd640 1 -- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f01ef5b0 msgr2=0x7f45f01efa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45eeffd640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f01ef5b0 0x7f45f01efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45eeffd640 1 -- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f01484c0 msgr2=0x7f45f0148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45eeffd640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f01484c0 0x7f45f0148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.654+0000 7f45eeffd640 1 -- 172.21.15.125:0/3972229048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0146720 con 0x7f45dc003970 2023-10-18T11:58:44.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.655+0000 7f45eeffd640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 0x7f45f01f04f0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f45e002c8c0 tx=0x7f45e0001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:44.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.655+0000 7f45e77fe640 1 -- 172.21.15.125:0/3972229048 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0006230 con 0x7f45dc003970 2023-10-18T11:58:44.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.655+0000 7f45e77fe640 1 -- 172.21.15.125:0/3972229048 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45e0006390 con 0x7f45dc003970 2023-10-18T11:58:44.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.655+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f0146a00 con 0x7f45dc003970 2023-10-18T11:58:44.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.655+0000 7f45e77fe640 1 -- 172.21.15.125:0/3972229048 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e00035f0 con 0x7f45dc003970 2023-10-18T11:58:44.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.655+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f01eff70 con 0x7f45dc003970 2023-10-18T11:58:44.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.656+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f00690e0 con 0x7f45dc003970 2023-10-18T11:58:44.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.656+0000 7f45e77fe640 1 -- 172.21.15.125:0/3972229048 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f45e0031030 con 0x7f45dc003970 2023-10-18T11:58:44.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.657+0000 7f45e77fe640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f45d006af90 0x7f45d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:44.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.657+0000 7f45e77fe640 1 -- 172.21.15.125:0/3972229048 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f45e00ab210 con 0x7f45dc003970 2023-10-18T11:58:44.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.657+0000 7f45e7fff640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f45d006af90 0x7f45d006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:44.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.657+0000 7f45e7fff640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f45d006af90 0x7f45d006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45f0142e60 tx=0x7f45d80040a0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:44.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.661+0000 7f45e77fe640 1 -- 172.21.15.125:0/3972229048 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45e0075fb0 con 0x7f45dc003970 2023-10-18T11:58:44.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.860+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45f00690e0 con 0x7f45dc003970 2023-10-18T11:58:44.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.860+0000 7f45e77fe640 1 -- 172.21.15.125:0/3972229048 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f45e00795d0 con 0x7f45dc003970 2023-10-18T11:58:44.863 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:44.864 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":80,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2853562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2853562772},{"type":"v1","addr":"172.21.15.139:6836","nonce":2853562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":11421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3995754374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3995754374},{"type":"v1","addr":"172.21.15.125:6834","nonce":3995754374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":11424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2110336778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2110336778},{"type":"v1","addr":"172.21.15.125:6835","nonce":2110336778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:58:35.848069+0000","modified":"2023-10-18T11:58:37.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11439":{"gid":11439,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/1491435386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1491435386},{"type":"v1","addr":"172.21.15.139:6837","nonce":1491435386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-18T11:58:44.864 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 80 2023-10-18T11:58:44.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f45d006af90 msgr2=0x7f45d006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f45d006af90 0x7f45d006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45f0142e60 tx=0x7f45d80040a0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 msgr2=0x7f45f01f04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:44.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 0x7f45f01f04f0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f45e002c8c0 tx=0x7f45e0001700 comp rx=0 tx=0).stop 2023-10-18T11:58:44.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 shutdown_connections 2023-10-18T11:58:44.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f45d006af90 0x7f45d006d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f01484c0 0x7f45f0148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f01ef5b0 0x7f45f01efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 --2- 172.21.15.125:0/3972229048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f45dc003970 0x7f45f01f04f0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:44.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 >> 172.21.15.125:0/3972229048 conn(0x7f45f00b3f00 msgr2=0x7f45f00b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:44.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 shutdown_connections 2023-10-18T11:58:44.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:44.862+0000 7f45f57d0640 1 -- 172.21.15.125:0/3972229048 wait complete. 2023-10-18T11:58:45.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T11:58:35.848069+0000', 'modified': '2023-10-18T11:58:37.856561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11439}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11439': {'gid': 11439, 'name': 'd', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6837/1491435386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6835', 'nonce': 1491435386}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 1491435386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T11:58:45.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T11:58:45.279 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-18T11:58:45.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.385+0000 7fb2e979e640 1 -- 172.21.15.125:0/355620504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 msgr2=0x7fb2e40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.385+0000 7fb2e979e640 1 --2- 172.21.15.125:0/355620504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 0x7fb2e40b0650 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc005fd0 tx=0x7fb2dc02c050 comp rx=0 tx=0).stop 2023-10-18T11:58:45.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.385+0000 7fb2e979e640 1 -- 172.21.15.125:0/355620504 shutdown_connections 2023-10-18T11:58:45.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.385+0000 7fb2e979e640 1 --2- 172.21.15.125:0/355620504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 0x7fb2e40b0650 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.385+0000 7fb2e979e640 1 --2- 172.21.15.125:0/355620504 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2e40cf340 0x7fb2e40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.386+0000 7fb2e979e640 1 -- 172.21.15.125:0/355620504 >> 172.21.15.125:0/355620504 conn(0x7fb2e40d0d60 msgr2=0x7fb2e40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:45.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.386+0000 7fb2e979e640 1 -- 172.21.15.125:0/355620504 shutdown_connections 2023-10-18T11:58:45.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.386+0000 7fb2e979e640 1 -- 172.21.15.125:0/355620504 wait complete. 2023-10-18T11:58:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.387+0000 7fb2e979e640 1 Processor -- start 2023-10-18T11:58:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.387+0000 7fb2e979e640 1 -- start start 2023-10-18T11:58:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.387+0000 7fb2e979e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 0x7fb2e40aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.387+0000 7fb2e979e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2e40cf340 0x7fb2e40b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.387+0000 7fb2e979e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e415b2a0 0x7fb2e4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.387+0000 7fb2e979e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b68a0 con 0x7fb2e40ceb40 2023-10-18T11:58:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.387+0000 7fb2e979e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4159330 con 0x7fb2e415b2a0 2023-10-18T11:58:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.387+0000 7fb2e979e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40af300 con 0x7fb2e40cf340 2023-10-18T11:58:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 0x7fb2e40aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 0x7fb2e40aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50136/0 (socket says 172.21.15.125:50136) 2023-10-18T11:58:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 -- 172.21.15.125:0/3706671654 learned_addr learned my addr 172.21.15.125:0/3706671654 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 -- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2e40cf340 msgr2=0x7fb2e40b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2e40cf340 0x7fb2e40b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e415b2a0 0x7fb2e4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2e40cf340 0x7fb2e40b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 -- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e415b2a0 msgr2=0x7fb2e4157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e415b2a0 0x7fb2e4157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 -- 172.21.15.125:0/3706671654 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc009840 con 0x7fb2e40ceb40 2023-10-18T11:58:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2e40cf340 0x7fb2e40b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:45.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e415b2a0 0x7fb2e4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:45.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 0x7fb2e40aed20 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400d980 tx=0x7fb2d400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:45.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2dbfff640 1 -- 172.21.15.125:0/3706671654 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4009c30 con 0x7fb2e40ceb40 2023-10-18T11:58:45.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2dbfff640 1 -- 172.21.15.125:0/3706671654 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2d4009e80 con 0x7fb2e40ceb40 2023-10-18T11:58:45.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e979e640 1 -- 172.21.15.125:0/3706671654 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e4158a00 con 0x7fb2e40ceb40 2023-10-18T11:58:45.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2dbfff640 1 -- 172.21.15.125:0/3706671654 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40106a0 con 0x7fb2e40ceb40 2023-10-18T11:58:45.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.388+0000 7fb2e979e640 1 -- 172.21.15.125:0/3706671654 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e4158fc0 con 0x7fb2e40ceb40 2023-10-18T11:58:45.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.390+0000 7fb2dbfff640 1 -- 172.21.15.125:0/3706671654 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb2d4010800 con 0x7fb2e40ceb40 2023-10-18T11:58:45.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.390+0000 7fb2dbfff640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb2c406af60 0x7fb2c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.390+0000 7fb2dbfff640 1 -- 172.21.15.125:0/3706671654 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fb2d4089ee0 con 0x7fb2e40ceb40 2023-10-18T11:58:45.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.390+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb2c406af60 0x7fb2c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.391+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb2c406af60 0x7fb2c406d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4159500 tx=0x7fb2cc009040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:45.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.391+0000 7fb2ec405740 1 Processor -- start 2023-10-18T11:58:45.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.391+0000 7fb2ec405740 1 -- start start 2023-10-18T11:58:45.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f5fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f62d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f5fe020 0x56185f62de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2ec405740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f50f730 con 0x56185f5fe020 2023-10-18T11:58:45.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2ec405740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f570780 con 0x56185f62af20 2023-10-18T11:58:45.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2ec405740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56185f56de70 con 0x56185f62ab40 2023-10-18T11:58:45.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f5fe020 0x56185f62de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f5fe020 0x56185f62de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50152/0 (socket says 172.21.15.125:50152) 2023-10-18T11:58:45.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 -- 172.21.15.125:0/1714742158 learned_addr learned my addr 172.21.15.125:0/1714742158 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:45.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f5fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f62d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 -- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 msgr2=0x56185f5fdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:58:45.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f5fdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 -- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 msgr2=0x56185f62d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:58:45.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f62d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 -- 172.21.15.125:0/1714742158 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56185f56e3d0 con 0x56185f5fe020 2023-10-18T11:58:45.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f62d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:45.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.392+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f5fe020 0x56185f62de10 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009cd0 tx=0x7fb2dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3c4e29cf718990 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:45.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2d8ff9640 1 -- 172.21.15.125:0/1714742158 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc02cce0 con 0x56185f5fe020 2023-10-18T11:58:45.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2d8ff9640 1 -- 172.21.15.125:0/1714742158 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2dc003c60 con 0x56185f5fe020 2023-10-18T11:58:45.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2d8ff9640 1 -- 172.21.15.125:0/1714742158 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc004040 con 0x56185f5fe020 2023-10-18T11:58:45.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 -- 172.21.15.125:0/1714742158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f5fe020 msgr2=0x56185f62de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f5fe020 0x56185f62de10 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009cd0 tx=0x7fb2dc004a40 comp rx=0 tx=0).stop 2023-10-18T11:58:45.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 -- 172.21.15.125:0/1714742158 shutdown_connections 2023-10-18T11:58:45.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f62d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f5fe020 0x56185f62de10 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 --2- 172.21.15.125:0/1714742158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f5fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 -- 172.21.15.125:0/1714742158 >> 172.21.15.125:0/1714742158 conn(0x56185f5ffd50 msgr2=0x56185f6488d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:45.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 -- 172.21.15.125:0/1714742158 shutdown_connections 2023-10-18T11:58:45.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.393+0000 7fb2ec405740 1 -- 172.21.15.125:0/1714742158 wait complete. 2023-10-18T11:58:45.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.394+0000 7fb2ec405740 1 -- start start 2023-10-18T11:58:45.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.394+0000 7fb2ec405740 1 Processor -- start 2023-10-18T11:58:45.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f63ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f658af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f648d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2ec405740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f569760 con 0x56185f62af20 2023-10-18T11:58:45.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2ec405740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f5248f0 con 0x56185f659030 2023-10-18T11:58:45.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2ec405740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56185f524590 con 0x56185f62ab40 2023-10-18T11:58:45.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2ec405740 10 client.0 _ll_register_callbacks cb 0x56185f560ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T11:58:45.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f658af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f658af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50162/0 (socket says 172.21.15.125:50162) 2023-10-18T11:58:45.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 -- 172.21.15.125:0/3194938819 learned_addr learned my addr 172.21.15.125:0/3194938819 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:45.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f63ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f648d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 -- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 msgr2=0x56185f63ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:58:45.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f63ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 -- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 msgr2=0x56185f648d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:58:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f648d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 -- 172.21.15.125:0/3194938819 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40cbc50 con 0x56185f62af20 2023-10-18T11:58:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f658af0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0046b0 tx=0x7fb2cc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4863a02de7f143b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T11:58:45.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/3194938819 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00dec0 con 0x56185f62af20 2023-10-18T11:58:45.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2ec405740 1 -- 172.21.15.125:0/3194938819 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56185f562740 con 0x56185f62af20 2023-10-18T11:58:45.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.395+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/3194938819 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fb2cc01f410 con 0x56185f62af20 2023-10-18T11:58:45.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/3194938819 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2cc01f5d0 con 0x56185f62af20 2023-10-18T11:58:45.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2ec405740 10 client.11595 fetch_fsmap learned FSMap version 80 2023-10-18T11:58:45.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2ec405740 1 -- 172.21.15.125:0/3194938819 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x56185f567de0 con 0x56185f62af20 2023-10-18T11:58:45.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/3194938819 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x56185f562740 con 0x56185f62af20 2023-10-18T11:58:45.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/3194938819 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb2cc01f730 con 0x56185f62af20 2023-10-18T11:58:45.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2ec405740 10 client.11595 fetch_fsmap finished waiting for FSMap version 80 2023-10-18T11:58:45.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2ec405740 10 client.11595 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-18T11:58:45.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2ec405740 20 client.11595 populate_metadata read hostname 'smithi125' 2023-10-18T11:58:45.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2ec405740 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] conn(0x56185f62ca70 0x56185f62ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2ec405740 4 client.11595 mds_command: new command op to 11439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T11:58:45.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2ec405740 1 -- 172.21.15.125:0/3194938819 --> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56185f562460 con 0x56185f62ca70 2023-10-18T11:58:45.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.396+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] conn(0x56185f62ca70 0x56185f62ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] conn(0x56185f62ca70 0x56185f62ce50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e8d634b0eb616c server_cookie=562dd8135ed3cf22 in_seq=0 out_seq=0 2023-10-18T11:58:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2c1ffb640 10 client.11595 ms_handle_connect on v2:172.21.15.139:6835/1491435386 2023-10-18T11:58:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/3194938819 <== mds.0 v2:172.21.15.139:6835/1491435386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56185f562460 con 0x56185f62ca70 2023-10-18T11:58:45.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2c1ffb640 10 client.11595 handle_command_reply: tid=0 2023-10-18T11:58:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 client.11595 shutdown 2023-10-18T11:58:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 -- 172.21.15.125:0/3194938819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 msgr2=0x56185f658af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f658af0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0046b0 tx=0x7fb2cc000ed0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 -- 172.21.15.125:0/3194938819 shutdown_connections 2023-10-18T11:58:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] conn(0x56185f62ca70 0x56185f62ce50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f648d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f658af0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2c1ffb640 0 client.11595 ms_handle_reset on v2:172.21.15.139:6835/1491435386 2023-10-18T11:58:45.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 --2- 172.21.15.125:0/3194938819 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f63ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 -- 172.21.15.125:0/3194938819 >> 172.21.15.125:0/3194938819 conn(0x56185f5fe020 msgr2=0x56185f603bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:45.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.397+0000 7fb2ec405740 1 -- 172.21.15.125:0/3194938819 shutdown_connections 2023-10-18T11:58:45.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.398+0000 7fb2ec405740 1 -- 172.21.15.125:0/3194938819 wait complete. 2023-10-18T11:58:45.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.398+0000 7fb2ec405740 20 client.11595 trim_cache size 0 max 16384 2023-10-18T11:58:45.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2ec405740 1 Processor -- start 2023-10-18T11:58:45.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2ec405740 1 -- start start 2023-10-18T11:58:45.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f631990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f5fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f659030 0x56185f640eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2ec405740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f4ea270 con 0x56185f659030 2023-10-18T11:58:45.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2ec405740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f631220 con 0x56185f62af20 2023-10-18T11:58:45.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2ec405740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56185f616dd0 con 0x56185f62ab40 2023-10-18T11:58:45.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.413+0000 7fb2e37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f659030 0x56185f640eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f659030 0x56185f640eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50168/0 (socket says 172.21.15.125:50168) 2023-10-18T11:58:45.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e37fe640 1 -- 172.21.15.125:0/2459812471 learned_addr learned my addr 172.21.15.125:0/2459812471 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:45.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f5fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f631990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e37fe640 1 -- 172.21.15.125:0/2459812471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 msgr2=0x56185f631990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:58:45.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f631990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e37fe640 1 -- 172.21.15.125:0/2459812471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 msgr2=0x56185f5fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:58:45.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f5fcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e37fe640 1 -- 172.21.15.125:0/2459812471 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56185f506d00 con 0x56185f659030 2023-10-18T11:58:45.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f659030 0x56185f640eb0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009cd0 tx=0x7fb2dc02c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d008edfbb45fe314 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:45.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2d97fa640 1 -- 172.21.15.125:0/2459812471 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc051650 con 0x56185f659030 2023-10-18T11:58:45.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2d97fa640 1 -- 172.21.15.125:0/2459812471 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2dc0518a0 con 0x56185f659030 2023-10-18T11:58:45.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2d97fa640 1 -- 172.21.15.125:0/2459812471 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc051c10 con 0x56185f659030 2023-10-18T11:58:45.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.414+0000 7fb2ec405740 1 -- 172.21.15.125:0/2459812471 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f659030 msgr2=0x56185f640eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f659030 0x56185f640eb0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009cd0 tx=0x7fb2dc02c050 comp rx=0 tx=0).stop 2023-10-18T11:58:45.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 -- 172.21.15.125:0/2459812471 shutdown_connections 2023-10-18T11:58:45.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f62af20 0x56185f5fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f659030 0x56185f640eb0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2459812471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f631990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 -- 172.21.15.125:0/2459812471 >> 172.21.15.125:0/2459812471 conn(0x56185f5fe020 msgr2=0x56185f601640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:45.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 -- 172.21.15.125:0/2459812471 shutdown_connections 2023-10-18T11:58:45.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 -- 172.21.15.125:0/2459812471 wait complete. 2023-10-18T11:58:45.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 -- start start 2023-10-18T11:58:45.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.415+0000 7fb2ec405740 1 Processor -- start 2023-10-18T11:58:45.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f699b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f6b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2ec405740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f63e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2ec405740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f6b1370 con 0x56185f62af20 2023-10-18T11:58:45.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2ec405740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f6b1790 con 0x56185f659030 2023-10-18T11:58:45.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2ec405740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56185f69a0f0 con 0x56185f62ab40 2023-10-18T11:58:45.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2ec405740 10 client.0 _ll_register_callbacks cb 0x7fb2dc009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T11:58:45.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f6b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f6b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50172/0 (socket says 172.21.15.125:50172) 2023-10-18T11:58:45.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 -- 172.21.15.125:0/2385513571 learned_addr learned my addr 172.21.15.125:0/2385513571 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:45.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f699b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f63e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 -- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 msgr2=0x56185f699b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:58:45.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f699b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 -- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 msgr2=0x56185f63e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T11:58:45.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f63e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 -- 172.21.15.125:0/2385513571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56185f56e3d0 con 0x56185f62af20 2023-10-18T11:58:45.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2e37fe640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f63e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:45.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f699b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:45.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2da7fc640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f6b0e30 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc001eb0 tx=0x7fb2cc01fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9208072651fd25f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:45.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T11:58:45.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/2385513571 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc014bc0 con 0x56185f62af20 2023-10-18T11:58:45.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.416+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/2385513571 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fb2cc006890 con 0x56185f62af20 2023-10-18T11:58:45.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 1 -- 172.21.15.125:0/2385513571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2dc003c60 con 0x56185f62af20 2023-10-18T11:58:45.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/2385513571 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2cc00d1f0 con 0x56185f62af20 2023-10-18T11:58:45.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 10 client.11601 fetch_fsmap learned FSMap version 80 2023-10-18T11:58:45.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 1 -- 172.21.15.125:0/2385513571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x56185f52c4f0 con 0x56185f62af20 2023-10-18T11:58:45.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/2385513571 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2dc003c60 con 0x56185f62af20 2023-10-18T11:58:45.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/2385513571 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 80) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb2cc006a90 con 0x56185f62af20 2023-10-18T11:58:45.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 10 client.11601 fetch_fsmap finished waiting for FSMap version 80 2023-10-18T11:58:45.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 10 client.11601 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-18T11:58:45.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 20 client.11601 populate_metadata read hostname 'smithi125' 2023-10-18T11:58:45.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] conn(0x56185f641060 0x56185f641440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 4 client.11601 mds_command: new command op to 11439 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T11:58:45.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2ec405740 1 -- 172.21.15.125:0/2385513571 --> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56185f26b6c0 con 0x56185f641060 2023-10-18T11:58:45.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.417+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] conn(0x56185f641060 0x56185f641440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2c1ffb640 10 client.11601 ms_handle_connect on v2:172.21.15.139:6835/1491435386 2023-10-18T11:58:45.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e2ffd640 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] conn(0x56185f641060 0x56185f641440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a954a71b43fdd9af server_cookie=22872833a4a4a01a in_seq=0 out_seq=0 2023-10-18T11:58:45.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2c1ffb640 1 -- 172.21.15.125:0/2385513571 <== mds.0 v2:172.21.15.139:6835/1491435386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x56185f26b6c0 con 0x56185f641060 2023-10-18T11:58:45.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2c1ffb640 10 client.11601 handle_command_reply: tid=0 2023-10-18T11:58:45.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 client.11601 shutdown 2023-10-18T11:58:45.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 -- 172.21.15.125:0/2385513571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 msgr2=0x56185f6b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f6b0e30 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc001eb0 tx=0x7fb2cc01fe20 comp rx=0 tx=0).stop 2023-10-18T11:58:45.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 -- 172.21.15.125:0/2385513571 shutdown_connections 2023-10-18T11:58:45.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:6835/1491435386,v1:172.21.15.139:6837/1491435386] conn(0x56185f641060 0x56185f641440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56185f659030 0x56185f63e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56185f62af20 0x56185f6b0e30 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 --2- 172.21.15.125:0/2385513571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56185f62ab40 0x56185f699b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2c1ffb640 0 client.11601 ms_handle_reset on v2:172.21.15.139:6835/1491435386 2023-10-18T11:58:45.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 -- 172.21.15.125:0/2385513571 >> 172.21.15.125:0/2385513571 conn(0x56185f5fe020 msgr2=0x56185f62b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:45.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 -- 172.21.15.125:0/2385513571 shutdown_connections 2023-10-18T11:58:45.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 1 -- 172.21.15.125:0/2385513571 wait complete. 2023-10-18T11:58:45.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2ec405740 20 client.11601 trim_cache size 0 max 16384 2023-10-18T11:58:45.451 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:58:45.452 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T11:58:45.452 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T11:58:45.452 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 11439, 2023-10-18T11:58:45.452 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T11:58:45.452 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T11:58:45.453 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T11:58:45.453 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.9130346130000007, 2023-10-18T11:58:45.453 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 80, 2023-10-18T11:58:45.453 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 204, 2023-10-18T11:58:45.453 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 204, 2023-10-18T11:58:45.454 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 13.682140473 2023-10-18T11:58:45.454 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:58:45.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 -- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb2c406af60 msgr2=0x7fb2c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb2c406af60 0x7fb2c406d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4159500 tx=0x7fb2cc009040 comp rx=0 tx=0).stop 2023-10-18T11:58:45.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 -- 172.21.15.125:0/3706671654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 msgr2=0x7fb2e40aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 0x7fb2e40aed20 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400d980 tx=0x7fb2d400de40 comp rx=0 tx=0).stop 2023-10-18T11:58:45.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 -- 172.21.15.125:0/3706671654 shutdown_connections 2023-10-18T11:58:45.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fb2c406af60 0x7fb2c406d420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2e415b2a0 0x7fb2e4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2e40cf340 0x7fb2e40b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 --2- 172.21.15.125:0/3706671654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb2e40ceb40 0x7fb2e40aed20 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.418+0000 7fb2e979e640 1 -- 172.21.15.125:0/3706671654 >> 172.21.15.125:0/3706671654 conn(0x7fb2e40d0d60 msgr2=0x7fb2e40c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:45.458 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.419+0000 7fb2e979e640 1 -- 172.21.15.125:0/3706671654 shutdown_connections 2023-10-18T11:58:45.459 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.419+0000 7fb2e979e640 1 -- 172.21.15.125:0/3706671654 wait complete. 2023-10-18T11:58:45.459 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T11:58:45.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.560+0000 7f84d075e640 1 -- 172.21.15.125:0/3856220032 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80ceb40 msgr2=0x7f84c80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.560+0000 7f84d075e640 1 --2- 172.21.15.125:0/3856220032 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80ceb40 0x7f84c80b0650 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f84c808a810 tx=0x7f84c002bd30 comp rx=0 tx=0).stop 2023-10-18T11:58:45.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.560+0000 7f84d075e640 1 -- 172.21.15.125:0/3856220032 shutdown_connections 2023-10-18T11:58:45.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.560+0000 7f84d075e640 1 --2- 172.21.15.125:0/3856220032 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80ceb40 0x7f84c80b0650 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.560+0000 7f84d075e640 1 --2- 172.21.15.125:0/3856220032 >> v2:172.21.15.139:3300/0 conn(0x7f84c80cf340 0x7f84c80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.560+0000 7f84d075e640 1 -- 172.21.15.125:0/3856220032 >> 172.21.15.125:0/3856220032 conn(0x7f84c80d0d60 msgr2=0x7f84c80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:45.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.560+0000 7f84d075e640 1 -- 172.21.15.125:0/3856220032 shutdown_connections 2023-10-18T11:58:45.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.560+0000 7f84d075e640 1 -- 172.21.15.125:0/3856220032 wait complete. 2023-10-18T11:58:45.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.561+0000 7f84d075e640 1 Processor -- start 2023-10-18T11:58:45.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.561+0000 7f84d075e640 1 -- start start 2023-10-18T11:58:45.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84d075e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84c80ceb40 0x7f84c81587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84d075e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80cf340 0x7f84c80c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84d075e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84c815b2a0 0x7f84c80ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84d075e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8158d00 con 0x7f84c80cf340 2023-10-18T11:58:45.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84d075e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81f1560 con 0x7f84c80ceb40 2023-10-18T11:58:45.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84d075e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81f2530 con 0x7f84c815b2a0 2023-10-18T11:58:45.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84cdcf9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80cf340 0x7f84c80c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84cdcf9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80cf340 0x7f84c80c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50180/0 (socket says 172.21.15.125:50180) 2023-10-18T11:58:45.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84cdcf9640 1 -- 172.21.15.125:0/948473048 learned_addr learned my addr 172.21.15.125:0/948473048 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:45.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84cecfb640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84c815b2a0 0x7f84c80ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.562+0000 7f84ce4fa640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84c80ceb40 0x7f84c81587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84cdcf9640 1 -- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84c815b2a0 msgr2=0x7f84c80ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84cdcf9640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84c815b2a0 0x7f84c80ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84cdcf9640 1 -- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84c80ceb40 msgr2=0x7f84c81587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84cdcf9640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84c80ceb40 0x7f84c81587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84cdcf9640 1 -- 172.21.15.125:0/948473048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c0009560 con 0x7f84c80cf340 2023-10-18T11:58:45.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84cdcf9640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80cf340 0x7f84c80c9e40 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b3e0 tx=0x7f84b800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:45.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84beffd640 1 -- 172.21.15.125:0/948473048 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8011830 con 0x7f84c80cf340 2023-10-18T11:58:45.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84beffd640 1 -- 172.21.15.125:0/948473048 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84b8011aa0 con 0x7f84c80cf340 2023-10-18T11:58:45.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c81f20a0 con 0x7f84c80cf340 2023-10-18T11:58:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84beffd640 1 -- 172.21.15.125:0/948473048 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8010380 con 0x7f84c80cf340 2023-10-18T11:58:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.563+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c81f1a50 con 0x7f84c80cf340 2023-10-18T11:58:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.564+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c81f2230 con 0x7f84c80cf340 2023-10-18T11:58:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.564+0000 7f84beffd640 1 -- 172.21.15.125:0/948473048 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f84b80104e0 con 0x7f84c80cf340 2023-10-18T11:58:45.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.565+0000 7f84beffd640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f84ac06af90 0x7f84ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:45.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.565+0000 7f84beffd640 1 -- 172.21.15.125:0/948473048 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f84b8089cd0 con 0x7f84c80cf340 2023-10-18T11:58:45.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.565+0000 7f84ce4fa640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f84ac06af90 0x7f84ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:45.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.565+0000 7f84ce4fa640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f84ac06af90 0x7f84ac06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84c4005c40 tx=0x7f84c400a390 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:45.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.569+0000 7f84beffd640 1 -- 172.21.15.125:0/948473048 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84b8054af0 con 0x7f84c80cf340 2023-10-18T11:58:45.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.772+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f84c80b0d00 con 0x7f84c80cf340 2023-10-18T11:58:45.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.772+0000 7f84beffd640 1 -- 172.21.15.125:0/948473048 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7f84b8015070 con 0x7f84c80cf340 2023-10-18T11:58:45.775 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T11:58:45.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f84ac06af90 msgr2=0x7f84ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f84ac06af90 0x7f84ac06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84c4005c40 tx=0x7f84c400a390 comp rx=0 tx=0).stop 2023-10-18T11:58:45.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80cf340 msgr2=0x7f84c80c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:45.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80cf340 0x7f84c80c9e40 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b3e0 tx=0x7f84b800b8a0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 shutdown_connections 2023-10-18T11:58:45.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f84ac06af90 0x7f84ac06d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84c815b2a0 0x7f84c80ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84c80cf340 0x7f84c80c9e40 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 --2- 172.21.15.125:0/948473048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84c80ceb40 0x7f84c81587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:45.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 >> 172.21.15.125:0/948473048 conn(0x7f84c80d0d60 msgr2=0x7f84c806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:45.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 shutdown_connections 2023-10-18T11:58:45.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:45.774+0000 7f84d075e640 1 -- 172.21.15.125:0/948473048 wait complete. 2023-10-18T11:58:45.787 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T11:58:45.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T11:58:45.788 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T11:58:45.788 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T11:58:45.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T11:58:45.789 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T11:58:45.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T11:58:45.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T11:58:45.790 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:58:45.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:45.791 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T11:58:45.806 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T11:58:45.807 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T11:58:45.807 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T11:58:45.807 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:58:45.807 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T11:58:45.808 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:58:45.808 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:58:45.808 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:58:45.808 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T11:58:45.808 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:58:45.809 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:58:45.809 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T11:58:45.809 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T11:58:45.809 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:58:45.809 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T11:58:45.810 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:58:45.810 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T11:58:45.810 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 37098sec preferred_lft 37098sec 2023-10-18T11:58:45.810 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T11:58:45.810 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:58:45.811 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T11:58:45.811 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T11:58:45.811 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T11:58:45.811 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:58:45.811 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T11:58:45.812 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:58:45.812 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T11:58:45.812 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:58:45.813 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T11:58:45.813 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:58:45.813 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T11:58:45.813 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:58:45.813 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T11:58:45.814 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T11:58:45.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:45.815 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T11:58:45.872 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T11:58:45.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:45.878 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T11:58:45.931 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T11:58:45.931 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T11:58:45.931 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T11:58:45.932 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T11:58:45.932 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T11:58:45.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:45.933 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T11:58:45.933 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T11:58:45.934 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T11:58:45.934 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T11:58:45.934 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T11:58:45.935 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T11:58:46.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:46.026 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T11:58:46.041 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T11:58:46.041 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T11:58:46.042 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:58:46.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:46.043 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T11:58:46.108 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T11:58:46.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T11:58:46.109 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T11:58:46.163 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T11:58:46.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T11:58:46.164 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T11:58:46.217 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T11:58:46.284 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T11:58:46.340 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T11:58:46.340 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T11:58:46.340 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T11:58:46.340 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T11:58:46.341 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T11:58:46.341 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T11:58:46.341 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T11:58:46.341 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T11:58:46.342 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T11:58:46.342 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T11:58:46.342 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T11:58:46.342 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T11:58:46.342 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T11:58:46.343 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T11:58:46.343 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T11:58:46.343 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T11:58:46.343 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T11:58:46.343 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T11:58:46.344 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T11:58:46.344 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T11:58:46.344 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T11:58:46.344 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T11:58:46.345 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T11:58:46.345 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T11:58:46.345 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T11:58:46.345 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T11:58:46.345 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T11:58:46.346 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T11:58:46.346 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T11:58:46.346 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T11:58:46.346 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:58:46.346 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:58:46.347 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:58:46.347 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:58:46.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:46.348 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T11:58:46.395 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T11:58:46.395 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T11:58:46.438 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T11:58:46.464 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T11:58:46.521 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T11:58:46.521 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T11:58:46.521 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T11:58:46.522 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T11:58:46.522 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T11:58:46.522 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T11:58:46.522 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T11:58:46.522 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T11:58:46.523 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T11:58:46.523 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T11:58:46.523 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T11:58:46.523 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T11:58:46.523 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T11:58:46.524 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T11:58:46.524 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T11:58:46.524 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T11:58:46.524 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T11:58:46.524 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T11:58:46.525 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T11:58:46.525 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T11:58:46.525 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T11:58:46.525 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T11:58:46.525 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T11:58:46.525 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T11:58:46.526 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T11:58:46.526 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T11:58:46.526 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T11:58:46.526 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T11:58:46.526 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T11:58:46.527 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T11:58:46.527 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:58:46.527 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T11:58:46.527 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:58:46.527 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T11:58:46.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:46.528 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T11:58:46.529 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T11:58:46.518+0000 7f525a23e480 -1 init, newargv = 0x55b06247cb60 newargc=15 2023-10-18T11:58:46.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[150913]: starting ceph client 2023-10-18T11:58:46.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[150913]: starting fuse 2023-10-18T11:58:46.544 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T11:58:46.544 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T11:58:46.545 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T11:58:46.545 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T11:58:46.546 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T11:58:46.546 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T11:58:46.546 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T11:58:46.547 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:58:46.547 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T11:58:46.548 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T11:58:46.548 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T11:58:46.548 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T11:58:46.549 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:58:46.549 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T11:58:46.549 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T11:58:46.550 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T11:58:46.550 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T11:58:46.550 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:58:46.551 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T11:58:46.551 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T11:58:46.552 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T11:58:46.552 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T11:58:46.552 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T11:58:46.553 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T11:58:46.553 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T11:58:46.554 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T11:58:46.554 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T11:58:46.554 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T11:58:46.555 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T11:58:46.662 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.150913.asok 2023-10-18T11:58:46.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T11:58:46.673 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.150913.asok 2023-10-18T11:58:46.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:46.674 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.150913.asok status 2023-10-18T11:58:46.807 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T11:58:46.807 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T11:58:46.807 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T11:58:46.808 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T11:58:46.808 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T11:58:46.808 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T11:58:46.808 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T11:58:46.808 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "150913", 2023-10-18T11:58:46.809 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T11:58:46.809 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:46.809 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T11:58:46.809 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T11:58:46.810 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 11613, 2023-10-18T11:58:46.810 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T11:58:46.810 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T11:58:46.810 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T11:58:46.810 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 11613 2023-10-18T11:58:46.811 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:46.811 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T11:58:46.811 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T11:58:46.811 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T11:58:46.811 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 3395434530 2023-10-18T11:58:46.812 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:46.812 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:46.812 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T11:58:46.812 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T11:58:46.812 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T11:58:46.813 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 3395434530 2023-10-18T11:58:46.813 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T11:58:46.813 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.11613 192.168.0.1:0/3395434530", 2023-10-18T11:58:46.813 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/3395434530", 2023-10-18T11:58:46.813 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T11:58:46.814 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 80, 2023-10-18T11:58:46.814 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 204, 2023-10-18T11:58:46.814 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T11:58:46.814 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T11:58:46.814 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T11:58:46.815 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T11:58:46.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:46.816 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T11:58:46.817 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T11:58:46.832 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T11:58:46.832 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T11:58:46.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:46.833 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T11:58:46.900 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T11:58:47.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.039+0000 7fe516f05640 1 -- 172.21.15.125:0/163663680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5100abc10 msgr2=0x7fe51006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:47.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.039+0000 7fe516f05640 1 --2- 172.21.15.125:0/163663680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5100abc10 0x7fe51006b2e0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fe508005fd0 tx=0x7fe50802c0e0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.040+0000 7fe516f05640 1 -- 172.21.15.125:0/163663680 shutdown_connections 2023-10-18T11:58:47.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.040+0000 7fe516f05640 1 --2- 172.21.15.125:0/163663680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5100abc10 0x7fe51006b2e0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.040+0000 7fe516f05640 1 -- 172.21.15.125:0/163663680 >> 172.21.15.125:0/163663680 conn(0x7fe510069930 msgr2=0x7fe510069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:47.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.040+0000 7fe516f05640 1 -- 172.21.15.125:0/163663680 shutdown_connections 2023-10-18T11:58:47.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.040+0000 7fe516f05640 1 -- 172.21.15.125:0/163663680 wait complete. 2023-10-18T11:58:47.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.041+0000 7fe516f05640 1 Processor -- start 2023-10-18T11:58:47.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.041+0000 7fe516f05640 1 -- start start 2023-10-18T11:58:47.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe516f05640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe51006af00 0x7fe510070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:47.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe516f05640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5100abc10 0x7fe510152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:47.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe516f05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5100c9dc0 0x7fe5100ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:47.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe516f05640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe510152910 con 0x7fe51006af00 2023-10-18T11:58:47.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe516f05640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5100af070 con 0x7fe5100c9dc0 2023-10-18T11:58:47.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe516f05640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5100af350 con 0x7fe5100abc10 2023-10-18T11:58:47.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe51006af00 0x7fe510070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:47.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe51006af00 0x7fe510070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50204/0 (socket says 172.21.15.125:50204) 2023-10-18T11:58:47.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 -- 172.21.15.125:0/170832984 learned_addr learned my addr 172.21.15.125:0/170832984 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:47.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 -- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5100abc10 msgr2=0x7fe510152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:47.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5100abc10 0x7fe510152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 -- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5100c9dc0 msgr2=0x7fe5100ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:47.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5100c9dc0 0x7fe5100ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 -- 172.21.15.125:0/170832984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe508009840 con 0x7fe51006af00 2023-10-18T11:58:47.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe514ca1640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe51006af00 0x7fe510070680 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fe50400c430 tx=0x7fe50400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:47.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe50d7fa640 1 -- 172.21.15.125:0/170832984 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504004230 con 0x7fe51006af00 2023-10-18T11:58:47.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe50d7fa640 1 -- 172.21.15.125:0/170832984 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe5040044a0 con 0x7fe51006af00 2023-10-18T11:58:47.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5101525a0 con 0x7fe51006af00 2023-10-18T11:58:47.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe50d7fa640 1 -- 172.21.15.125:0/170832984 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504003850 con 0x7fe51006af00 2023-10-18T11:58:47.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.042+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5100af630 con 0x7fe51006af00 2023-10-18T11:58:47.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.043+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5100690e0 con 0x7fe51006af00 2023-10-18T11:58:47.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.044+0000 7fe50d7fa640 1 -- 172.21.15.125:0/170832984 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe5040039b0 con 0x7fe51006af00 2023-10-18T11:58:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.044+0000 7fe50d7fa640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe4f406af60 0x7fe4f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.044+0000 7fe50d7fa640 1 -- 172.21.15.125:0/170832984 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fe5040892b0 con 0x7fe51006af00 2023-10-18T11:58:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.045+0000 7fe50ffff640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe4f406af60 0x7fe4f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.045+0000 7fe50ffff640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe4f406af60 0x7fe4f406d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc006fd0 tx=0x7fe4fc006bc0 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.049+0000 7fe50d7fa640 1 -- 172.21.15.125:0/170832984 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe504057ad0 con 0x7fe51006af00 2023-10-18T11:58:47.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.235+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 --> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe510152a50 con 0x7fe4f406af60 2023-10-18T11:58:47.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.238+0000 7fe50d7fa640 1 -- 172.21.15.125:0/170832984 <== mgr.9529 v2:172.21.15.139:6832/1565260611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe4fc013430 con 0x7fe4f406af60 2023-10-18T11:58:47.241 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T11:58:47.241 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T11:58:47.241 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T11:58:47.241 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T11:58:47.242 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T11:58:47.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe4f406af60 msgr2=0x7fe4f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:47.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe4f406af60 0x7fe4f406d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc006fd0 tx=0x7fe4fc006bc0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe51006af00 msgr2=0x7fe510070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:47.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe51006af00 0x7fe510070680 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fe50400c430 tx=0x7fe50400c8f0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 shutdown_connections 2023-10-18T11:58:47.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7fe4f406af60 0x7fe4f406d420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5100c9dc0 0x7fe5100ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5100abc10 0x7fe510152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 --2- 172.21.15.125:0/170832984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe51006af00 0x7fe510070680 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 >> 172.21.15.125:0/170832984 conn(0x7fe510069930 msgr2=0x7fe5100c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:47.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 shutdown_connections 2023-10-18T11:58:47.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.240+0000 7fe516f05640 1 -- 172.21.15.125:0/170832984 wait complete. 2023-10-18T11:58:47.253 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T11:58:47.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.355+0000 7f195097d640 1 -- 172.21.15.125:0/2712186904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0ceb40 msgr2=0x7f194c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:47.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.355+0000 7f195097d640 1 --2- 172.21.15.125:0/2712186904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0ceb40 0x7f194c0b0650 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f1944005fd0 tx=0x7f194402bcf0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.355+0000 7f195097d640 1 -- 172.21.15.125:0/2712186904 shutdown_connections 2023-10-18T11:58:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.355+0000 7f195097d640 1 --2- 172.21.15.125:0/2712186904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0ceb40 0x7f194c0b0650 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.355+0000 7f195097d640 1 --2- 172.21.15.125:0/2712186904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f194c15b2a0 0x7f194c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.355+0000 7f195097d640 1 -- 172.21.15.125:0/2712186904 >> 172.21.15.125:0/2712186904 conn(0x7f194c0d0d60 msgr2=0x7f194c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.355+0000 7f195097d640 1 -- 172.21.15.125:0/2712186904 shutdown_connections 2023-10-18T11:58:47.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.355+0000 7f195097d640 1 -- 172.21.15.125:0/2712186904 wait complete. 2023-10-18T11:58:47.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.356+0000 7f195097d640 1 Processor -- start 2023-10-18T11:58:47.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.356+0000 7f195097d640 1 -- start start 2023-10-18T11:58:47.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f195097d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f194c0ceb40 0x7f194c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:47.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f195097d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0cf340 0x7f194c0b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:47.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f195097d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c15b2a0 0x7f194c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:47.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f195097d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0c9200 con 0x7f194c0cf340 2023-10-18T11:58:47.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f195097d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b6b30 con 0x7f194c15b2a0 2023-10-18T11:58:47.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f195097d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0c8d60 con 0x7f194c0ceb40 2023-10-18T11:58:47.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f1949d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0cf340 0x7f194c0b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:47.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f1949d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0cf340 0x7f194c0b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:50222/0 (socket says 172.21.15.125:50222) 2023-10-18T11:58:47.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f1949d9b640 1 -- 172.21.15.125:0/393534295 learned_addr learned my addr 172.21.15.125:0/393534295 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:47.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f194a59c640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f194c0ceb40 0x7f194c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:47.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f194ad9d640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c15b2a0 0x7f194c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:47.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f1949d9b640 1 -- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f194c0ceb40 msgr2=0x7f194c0c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:47.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f1949d9b640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f194c0ceb40 0x7f194c0c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f1949d9b640 1 -- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c15b2a0 msgr2=0x7f194c157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:47.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.357+0000 7f1949d9b640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c15b2a0 0x7f194c157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:47.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f1949d9b640 1 -- 172.21.15.125:0/393534295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944009560 con 0x7f194c0cf340 2023-10-18T11:58:47.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f194a59c640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f194c0ceb40 0x7f194c0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:47.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f194ad9d640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c15b2a0 0x7f194c157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T11:58:47.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f1949d9b640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0cf340 0x7f194c0b65f0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f194000b440 tx=0x7f194000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:47.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f193affd640 1 -- 172.21.15.125:0/393534295 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940011830 con 0x7f194c0cf340 2023-10-18T11:58:47.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f193affd640 1 -- 172.21.15.125:0/393534295 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1940011a80 con 0x7f194c0cf340 2023-10-18T11:58:47.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f193affd640 1 -- 172.21.15.125:0/393534295 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940011e90 con 0x7f194c0cf340 2023-10-18T11:58:47.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c0c84c0 con 0x7f194c0cf340 2023-10-18T11:58:47.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.358+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c0c8a80 con 0x7f194c0cf340 2023-10-18T11:58:47.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.359+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c0c9340 con 0x7f194c0cf340 2023-10-18T11:58:47.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.359+0000 7f193affd640 1 -- 172.21.15.125:0/393534295 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 119) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f194001a400 con 0x7f194c0cf340 2023-10-18T11:58:47.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.360+0000 7f193affd640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f192c06af60 0x7f192c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:47.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.360+0000 7f193affd640 1 -- 172.21.15.125:0/393534295 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f1940089c60 con 0x7f194c0cf340 2023-10-18T11:58:47.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.360+0000 7f194a59c640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f192c06af60 0x7f192c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:47.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.361+0000 7f194a59c640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f192c06af60 0x7f192c06d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f193c001dc0 tx=0x7f193c009040 comp rx=0 tx=0).ready entity=mgr.9529 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:47.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.365+0000 7f193affd640 1 -- 172.21.15.125:0/393534295 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f194c0c9340 con 0x7f194c0cf340 2023-10-18T11:58:47.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:47.575+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f194c0c9340 con 0x7f194c0cf340 2023-10-18T11:58:48.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.362+0000 7f193affd640 1 -- 172.21.15.125:0/393534295 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v120) v1 ==== 92+0+0 (secure 0 0 0) 0x7f194c0c9340 con 0x7f194c0cf340 2023-10-18T11:58:48.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.365+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f192c06af60 msgr2=0x7f192c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.365+0000 7f195097d640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f192c06af60 0x7f192c06d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f193c001dc0 tx=0x7f193c009040 comp rx=0 tx=0).stop 2023-10-18T11:58:48.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.365+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0cf340 msgr2=0x7f194c0b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.365+0000 7f195097d640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0cf340 0x7f194c0b65f0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f194000b440 tx=0x7f194000b900 comp rx=0 tx=0).stop 2023-10-18T11:58:48.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.366+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 shutdown_connections 2023-10-18T11:58:48.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.366+0000 7f195097d640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f192c06af60 0x7f192c06d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.366+0000 7f195097d640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c15b2a0 0x7f194c157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.366+0000 7f195097d640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f194c0cf340 0x7f194c0b65f0 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.366+0000 7f195097d640 1 --2- 172.21.15.125:0/393534295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f194c0ceb40 0x7f194c0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.366+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 >> 172.21.15.125:0/393534295 conn(0x7f194c0d0d60 msgr2=0x7f194c15a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:48.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.366+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 shutdown_connections 2023-10-18T11:58:48.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.366+0000 7f195097d640 1 -- 172.21.15.125:0/393534295 wait complete. 2023-10-18T11:58:48.384 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T11:58:48.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.508+0000 7f27647d7640 1 -- 172.21.15.125:0/257994892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f275c159060 msgr2=0x7f275c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.508+0000 7f27647d7640 1 --2- 172.21.15.125:0/257994892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f275c159060 0x7f275c06d260 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2758009860 tx=0x7f275802bcb0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.508+0000 7f27647d7640 1 -- 172.21.15.125:0/257994892 shutdown_connections 2023-10-18T11:58:48.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.508+0000 7f27647d7640 1 --2- 172.21.15.125:0/257994892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f275c15d400 0x7f275c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.508+0000 7f27647d7640 1 --2- 172.21.15.125:0/257994892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f275c159060 0x7f275c06d260 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.508+0000 7f27647d7640 1 -- 172.21.15.125:0/257994892 >> 172.21.15.125:0/257994892 conn(0x7f275c0b0670 msgr2=0x7f275c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:48.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.508+0000 7f27647d7640 1 -- 172.21.15.125:0/257994892 shutdown_connections 2023-10-18T11:58:48.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.508+0000 7f27647d7640 1 -- 172.21.15.125:0/257994892 wait complete. 2023-10-18T11:58:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.509+0000 7f27647d7640 1 Processor -- start 2023-10-18T11:58:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.509+0000 7f27647d7640 1 -- start start 2023-10-18T11:58:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f27647d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f275c0cdb30 0x7f275c150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f27647d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f275c159060 0x7f275c0b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f27647d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f275c15d400 0x7f275c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f27647d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0c6e70 con 0x7f275c159060 2023-10-18T11:58:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f27647d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0b2220 con 0x7f275c15d400 2023-10-18T11:58:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f27647d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0c6820 con 0x7f275c0cdb30 2023-10-18T11:58:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f275c159060 0x7f275c0b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f275c159060 0x7f275c0b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55930/0 (socket says 172.21.15.125:55930) 2023-10-18T11:58:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 -- 172.21.15.125:0/1490106484 learned_addr learned my addr 172.21.15.125:0/1490106484 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2762573640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f275c0cdb30 0x7f275c150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2762d74640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f275c15d400 0x7f275c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 -- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f275c0cdb30 msgr2=0x7f275c150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f275c0cdb30 0x7f275c150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 -- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f275c15d400 msgr2=0x7f275c0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f275c15d400 0x7f275c0b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 -- 172.21.15.125:0/1490106484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2754009840 con 0x7f275c159060 2023-10-18T11:58:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2761d72640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f275c159060 0x7f275c0b1760 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f274c00c400 tx=0x7f274c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2752ffd640 1 -- 172.21.15.125:0/1490106484 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c004230 con 0x7f275c159060 2023-10-18T11:58:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2752ffd640 1 -- 172.21.15.125:0/1490106484 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f274c0044a0 con 0x7f275c159060 2023-10-18T11:58:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f2752ffd640 1 -- 172.21.15.125:0/1490106484 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c003850 con 0x7f275c159060 2023-10-18T11:58:48.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2758009490 con 0x7f275c159060 2023-10-18T11:58:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.510+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c0b2500 con 0x7f275c159060 2023-10-18T11:58:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.512+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c0c7020 con 0x7f275c159060 2023-10-18T11:58:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.512+0000 7f2752ffd640 1 -- 172.21.15.125:0/1490106484 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 120) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f274c0039b0 con 0x7f275c159060 2023-10-18T11:58:48.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.512+0000 7f2752ffd640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f274006afe0 0x7f274006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:48.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.512+0000 7f2752ffd640 1 -- 172.21.15.125:0/1490106484 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f274c088fd0 con 0x7f275c159060 2023-10-18T11:58:48.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.512+0000 7f2762573640 1 -- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f274006afe0 msgr2=0x7f274006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1565260611 2023-10-18T11:58:48.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.512+0000 7f2762573640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f274006afe0 0x7f274006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:48.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.517+0000 7f2752ffd640 1 -- 172.21.15.125:0/1490106484 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f275c0c7020 con 0x7f275c159060 2023-10-18T11:58:48.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.691+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f275c06ebc0 con 0x7f275c159060 2023-10-18T11:58:48.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.694+0000 7f2752ffd640 1 -- 172.21.15.125:0/1490106484 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v72)=0 v72) v1 ==== 138+0+0 (secure 0 0 0) 0x7f274c0170d0 con 0x7f275c159060 2023-10-18T11:58:48.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.696+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f274006afe0 msgr2=0x7f274006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:48.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f274006afe0 0x7f274006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f275c159060 msgr2=0x7f275c0b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f275c159060 0x7f275c0b1760 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f274c00c400 tx=0x7f274c00c800 comp rx=0 tx=0).stop 2023-10-18T11:58:48.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 shutdown_connections 2023-10-18T11:58:48.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f274006afe0 0x7f274006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f275c15d400 0x7f275c0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f275c159060 0x7f275c0b1760 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 --2- 172.21.15.125:0/1490106484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f275c0cdb30 0x7f275c150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 >> 172.21.15.125:0/1490106484 conn(0x7f275c0b0670 msgr2=0x7f275c0ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:48.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 shutdown_connections 2023-10-18T11:58:48.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.697+0000 7f27647d7640 1 -- 172.21.15.125:0/1490106484 wait complete. 2023-10-18T11:58:48.710 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T11:58:48.754 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:48.750+0000 7efe20c2d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:58:48.777 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:48.775+0000 7ff4b6e13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T11:58:48.799 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:48.795+0000 7efe20c2d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:58:48.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.816+0000 7f4a7dc39640 1 -- 172.21.15.125:0/2855991795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78141690 msgr2=0x7f4a780703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.816+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/2855991795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78141690 0x7f4a780703e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009830 tx=0x7f4a6c02bcd0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.816+0000 7f4a7dc39640 1 -- 172.21.15.125:0/2855991795 shutdown_connections 2023-10-18T11:58:48.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.816+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/2855991795 >> v2:172.21.15.139:3300/0 conn(0x7f4a78070920 0x7f4a780affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.816+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/2855991795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78141690 0x7f4a780703e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.816+0000 7f4a7dc39640 1 -- 172.21.15.125:0/2855991795 >> 172.21.15.125:0/2855991795 conn(0x7f4a78069930 msgr2=0x7f4a78069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:48.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.816+0000 7f4a7dc39640 1 -- 172.21.15.125:0/2855991795 shutdown_connections 2023-10-18T11:58:48.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.816+0000 7f4a7dc39640 1 -- 172.21.15.125:0/2855991795 wait complete. 2023-10-18T11:58:48.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.817+0000 7f4a7dc39640 1 Processor -- start 2023-10-18T11:58:48.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.817+0000 7f4a7dc39640 1 -- start start 2023-10-18T11:58:48.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a7dc39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a78070920 0x7f4a7813b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:48.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a7dc39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7813e520 0x7f4a780aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:48.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a7dc39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78141690 0x7f4a781d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:48.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a7dc39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781d20c0 con 0x7f4a78070920 2023-10-18T11:58:48.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a7dc39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7813b630 con 0x7f4a7813e520 2023-10-18T11:58:48.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a7dc39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781e2600 con 0x7f4a78141690 2023-10-18T11:58:48.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a78070920 0x7f4a7813b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:48.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a78070920 0x7f4a7813b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55932/0 (socket says 172.21.15.125:55932) 2023-10-18T11:58:48.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a777fe640 1 -- 172.21.15.125:0/291387217 learned_addr learned my addr 172.21.15.125:0/291387217 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:48.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a76ffd640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7813e520 0x7f4a780aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:48.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.818+0000 7f4a77fff640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78141690 0x7f4a781d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:48.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a777fe640 1 -- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78141690 msgr2=0x7f4a781d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a777fe640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78141690 0x7f4a781d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a777fe640 1 -- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7813e520 msgr2=0x7f4a780aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:48.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a777fe640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7813e520 0x7f4a780aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:48.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a777fe640 1 -- 172.21.15.125:0/291387217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a70009840 con 0x7f4a78070920 2023-10-18T11:58:48.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a777fe640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a78070920 0x7f4a7813b0f0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c430 tx=0x7f4a6800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:48.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a5ffff640 1 -- 172.21.15.125:0/291387217 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68004230 con 0x7f4a78070920 2023-10-18T11:58:48.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a5ffff640 1 -- 172.21.15.125:0/291387217 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a680044a0 con 0x7f4a78070920 2023-10-18T11:58:48.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c009490 con 0x7f4a78070920 2023-10-18T11:58:48.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a5ffff640 1 -- 172.21.15.125:0/291387217 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68003850 con 0x7f4a78070920 2023-10-18T11:58:48.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.819+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a781e2bc0 con 0x7f4a78070920 2023-10-18T11:58:48.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.820+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a780711c0 con 0x7f4a78070920 2023-10-18T11:58:48.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.820+0000 7f4a5ffff640 1 -- 172.21.15.125:0/291387217 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 120) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f4a680039d0 con 0x7f4a78070920 2023-10-18T11:58:48.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.821+0000 7f4a5ffff640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4a6406afb0 0x7f4a6406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:48.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.821+0000 7f4a76ffd640 1 -- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4a6406afb0 msgr2=0x7f4a6406d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1565260611 2023-10-18T11:58:48.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.821+0000 7f4a5ffff640 1 -- 172.21.15.125:0/291387217 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f4a68089080 con 0x7f4a78070920 2023-10-18T11:58:48.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.821+0000 7f4a76ffd640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4a6406afb0 0x7f4a6406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:48.830 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:48.823+0000 7ff4b6e13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T11:58:48.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.825+0000 7f4a5ffff640 1 -- 172.21.15.125:0/291387217 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a68054010 con 0x7f4a78070920 2023-10-18T11:58:48.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:48.996+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f4a780690e0 con 0x7f4a78070920 2023-10-18T11:58:49.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.001+0000 7f4a5ffff640 1 -- 172.21.15.125:0/291387217 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v73)=0 v73) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4a680170c0 con 0x7f4a78070920 2023-10-18T11:58:49.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4a6406afb0 msgr2=0x7f4a6406d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:49.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4a6406afb0 0x7f4a6406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a78070920 msgr2=0x7f4a7813b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:49.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a78070920 0x7f4a7813b0f0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c430 tx=0x7f4a6800c830 comp rx=0 tx=0).stop 2023-10-18T11:58:49.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 shutdown_connections 2023-10-18T11:58:49.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f4a6406afb0 0x7f4a6406d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7813e520 0x7f4a780aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78141690 0x7f4a781d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 --2- 172.21.15.125:0/291387217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a78070920 0x7f4a7813b0f0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 >> 172.21.15.125:0/291387217 conn(0x7f4a78069930 msgr2=0x7f4a780aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:49.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 shutdown_connections 2023-10-18T11:58:49.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.003+0000 7f4a7dc39640 1 -- 172.21.15.125:0/291387217 wait complete. 2023-10-18T11:58:49.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:49.017 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T11:58:49.038 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-18T11:58:49.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.184+0000 7f3cd270c640 1 -- 172.21.15.125:0/927597512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb4005180 msgr2=0x7f3cb400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:49.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.184+0000 7f3cd270c640 1 --2- 172.21.15.125:0/927597512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb4005180 0x7f3cb400b980 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4001d80 tx=0x7f3cb40057f0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.184+0000 7f3cd270c640 1 -- 172.21.15.125:0/927597512 shutdown_connections 2023-10-18T11:58:49.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.184+0000 7f3cd270c640 1 --2- 172.21.15.125:0/927597512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb4005180 0x7f3cb400b980 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.184+0000 7f3cd270c640 1 -- 172.21.15.125:0/927597512 >> 172.21.15.125:0/927597512 conn(0x7f3ccc099ff0 msgr2=0x7f3ccc0d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:49.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.184+0000 7f3cd270c640 1 -- 172.21.15.125:0/927597512 shutdown_connections 2023-10-18T11:58:49.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.184+0000 7f3cd270c640 1 -- 172.21.15.125:0/927597512 wait complete. 2023-10-18T11:58:49.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.185+0000 7f3cd270c640 1 Processor -- start 2023-10-18T11:58:49.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.185+0000 7f3cd270c640 1 -- start start 2023-10-18T11:58:49.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd270c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb4005180 0x7f3ccc14a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:49.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd270c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ccc14a770 0x7f3ccc0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:49.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd270c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ccc0b4fb0 0x7f3ccc0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:49.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd270c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0b6700 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd270c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0b2aa0 con 0x7f3ccc14a770 2023-10-18T11:58:49.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd270c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc14a5c0 con 0x7f3cb4005180 2023-10-18T11:58:49.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ccc0b4fb0 0x7f3ccc0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:49.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ccc0b4fb0 0x7f3ccc0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55948/0 (socket says 172.21.15.125:55948) 2023-10-18T11:58:49.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 -- 172.21.15.125:0/1407149905 learned_addr learned my addr 172.21.15.125:0/1407149905 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:49.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3ccbfff640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb4005180 0x7f3ccc14a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:49.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3ccb7fe640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ccc14a770 0x7f3ccc0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:49.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb4005180 msgr2=0x7f3ccc14a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:49.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb4005180 0x7f3ccc14a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ccc14a770 msgr2=0x7f3ccc0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:49.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ccc14a770 0x7f3ccc0c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:49.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 -- 172.21.15.125:0/1407149905 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0b5d60 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd0ca9640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ccc0b4fb0 0x7f3ccc0b5430 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4001ea0 tx=0x7f3cc402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:49.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4006230 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3cc40064a0 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.186+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc0b6040 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.187+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40068b0 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.187+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc0c7430 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.188+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc066960 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.188+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 120) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f3cc4034410 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.188+0000 7f3cc8ff9640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 0x7f3cb806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:49.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.188+0000 7f3ccbfff640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 msgr2=0x7f3cb806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1565260611 2023-10-18T11:58:49.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.188+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f3cc40aa8f0 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.189+0000 7f3ccbfff640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 0x7f3cb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T11:58:49.201 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.188+0000 7efe20c2d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:58:49.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.196+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3cc4075740 con 0x7f3ccc0b4fb0 2023-10-18T11:58:49.236 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.232+0000 7efe20c2d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:58:49.237 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.234+0000 7ff4b6e13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T11:58:49.278 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.274+0000 7efe20c2d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:58:49.282 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.280+0000 7ff4b6e13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T11:58:49.326 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.324+0000 7ff4b6e13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T11:58:49.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.326+0000 7efe20c2d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:58:49.371 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.368+0000 7efe20c2d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:58:49.380 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.378+0000 7ff4b6e13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T11:58:49.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.390+0000 7f3ccbfff640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 msgr2=0x7f3cb806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1565260611 2023-10-18T11:58:49.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.390+0000 7f3ccbfff640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 0x7f3cb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T11:58:49.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.393+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 --> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ccc0cf350 con 0x7f3cb806afe0 2023-10-18T11:58:49.423 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.421+0000 7ff4b6e13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T11:58:49.471 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.467+0000 7efe20c2d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:58:49.522 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.520+0000 7ff4b6e13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T11:58:49.523 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.520+0000 7efe20c2d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:58:49.565 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.561+0000 7efe20c2d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:58:49.575 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.573+0000 7ff4b6e13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T11:58:49.612 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.609+0000 7efe20c2d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:58:49.617 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.614+0000 7ff4b6e13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T11:58:49.654 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.651+0000 7efe20c2d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:58:49.664 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.662+0000 7ff4b6e13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T11:58:49.706 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:49.704+0000 7ff4b6e13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T11:58:49.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.791+0000 7f3ccbfff640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 msgr2=0x7f3cb806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1565260611 2023-10-18T11:58:49.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:49.791+0000 7f3ccbfff640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 0x7f3cb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T11:58:49.966 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:49.963+0000 7efe20c2d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:58:50.022 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.019+0000 7ff4b6e13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T11:58:50.095 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:50.091+0000 7efe20c2d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:58:50.152 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.150+0000 7ff4b6e13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T11:58:50.186 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:50.182+0000 7efe20c2d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:58:50.227 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:50.224+0000 7efe20c2d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:58:50.243 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.240+0000 7ff4b6e13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T11:58:50.340 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:50.318+0000 7efe20c2d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:58:50.340 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.283+0000 7ff4b6e13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T11:58:50.380 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.377+0000 7ff4b6e13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:58:50.410 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:50.407+0000 7efe20c2d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:58:50.470 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.468+0000 7ff4b6e13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T11:58:50.546 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:50.542+0000 7efe20c2d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:58:50.587 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:50.583+0000 7efe20c2d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:58:50.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:50.592+0000 7f3ccbfff640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 msgr2=0x7f3cb806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1565260611 2023-10-18T11:58:50.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:50.592+0000 7f3ccbfff640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 0x7f3cb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T11:58:50.608 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.605+0000 7ff4b6e13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T11:58:50.650 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.647+0000 7ff4b6e13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T11:58:50.690 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:50.686+0000 7efe20c2d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:58:50.754 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:50.751+0000 7ff4b6e13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T11:58:51.308 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.306+0000 7efe20c2d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:58:51.367 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:51.365+0000 7ff4b6e13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T11:58:51.570 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.568+0000 7efe20c2d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:58:51.618 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.616+0000 7efe20c2d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:58:51.629 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:51.627+0000 7ff4b6e13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T11:58:51.677 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:51.675+0000 7ff4b6e13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T11:58:51.717 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.714+0000 7efe20c2d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:58:51.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:51.739+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 121) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f3cc40a8c90 con 0x7f3ccc0b4fb0 2023-10-18T11:58:51.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:51.739+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 msgr2=0x7f3cb806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T11:58:51.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:51.739+0000 7f3cc8ff9640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/1565260611,v1:172.21.15.139:6833/1565260611] conn(0x7f3cb806afe0 0x7f3cb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:51.782 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T11:58:51.779+0000 7ff4b6e13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T11:58:51.847 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9fe6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.847 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9fe6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.847 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9fe6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.847 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9fe6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.848 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9fe6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.848 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9ce24640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.848 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9ce24640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.848 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9ce24640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.848 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9ce24640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:51.849 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:58:51.844+0000 7efd9ce24640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:58:52.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.740+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 122) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3cc4075b10 con 0x7f3ccc0b4fb0 2023-10-18T11:58:52.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.740+0000 7f3cc8ff9640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f3cb8035880 0x7f3cb8075470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:52.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.741+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 --> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cc40068b0 con 0x7f3cb8035880 2023-10-18T11:58:52.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.743+0000 7f3ccbfff640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f3cb8035880 0x7f3cb8075470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:52.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.749+0000 7f3ccbfff640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f3cb8035880 0x7f3cb8075470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009460 tx=0x7f3cc0005f90 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:52.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.803+0000 7f3cc8ff9640 1 -- 172.21.15.125:0/1407149905 <== mgr.9871 v2:172.21.15.139:6832/3820883843 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3cc40068b0 con 0x7f3cb8035880 2023-10-18T11:58:52.806 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1 2023-10-18T11:58:52.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f3cb8035880 msgr2=0x7f3cb8075470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:52.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd270c640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f3cb8035880 0x7f3cb8075470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009460 tx=0x7f3cc0005f90 comp rx=0 tx=0).stop 2023-10-18T11:58:52.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ccc0b4fb0 msgr2=0x7f3ccc0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:52.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd270c640 1 --2- 172.21.15.125:0/1407149905 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ccc0b4fb0 0x7f3ccc0b5430 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4001ea0 tx=0x7f3cc402f000 comp rx=0 tx=0).stop 2023-10-18T11:58:52.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd0ca9640 1 -- 172.21.15.125:0/1407149905 reap_dead start 2023-10-18T11:58:52.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 shutdown_connections 2023-10-18T11:58:52.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 >> 172.21.15.125:0/1407149905 conn(0x7f3ccc099ff0 msgr2=0x7f3ccc15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:52.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 shutdown_connections 2023-10-18T11:58:52.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.805+0000 7f3cd270c640 1 -- 172.21.15.125:0/1407149905 wait complete. 2023-10-18T11:58:52.819 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-18T11:58:52.820 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-18T11:58:52.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.923+0000 7f7371632640 1 -- 172.21.15.125:0/1815342977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c06d260 msgr2=0x7f736c15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:52.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.923+0000 7f7371632640 1 --2- 172.21.15.125:0/1815342977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c06d260 0x7f736c15d400 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f7364009830 tx=0x7f736402ba90 comp rx=0 tx=0).stop 2023-10-18T11:58:52.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.923+0000 7f7371632640 1 -- 172.21.15.125:0/1815342977 shutdown_connections 2023-10-18T11:58:52.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.923+0000 7f7371632640 1 --2- 172.21.15.125:0/1815342977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c06d260 0x7f736c15d400 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:52.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.923+0000 7f7371632640 1 -- 172.21.15.125:0/1815342977 >> 172.21.15.125:0/1815342977 conn(0x7f736c0b0670 msgr2=0x7f736c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:52.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.923+0000 7f7371632640 1 -- 172.21.15.125:0/1815342977 shutdown_connections 2023-10-18T11:58:52.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.923+0000 7f7371632640 1 -- 172.21.15.125:0/1815342977 wait complete. 2023-10-18T11:58:52.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.924+0000 7f7371632640 1 Processor -- start 2023-10-18T11:58:52.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.924+0000 7f7371632640 1 -- start start 2023-10-18T11:58:52.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f7371632640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736c06d260 0x7f736c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:52.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f7371632640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c0ce240 0x7f736c0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:52.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f7371632640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f736c159060 0x7f736c0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:52.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f7371632640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c159980 con 0x7f736c0ce240 2023-10-18T11:58:52.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f7371632640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c159c60 con 0x7f736c06d260 2023-10-18T11:58:52.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f7371632640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c157010 con 0x7f736c159060 2023-10-18T11:58:52.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f736a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c0ce240 0x7f736c0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:52.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f736a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c0ce240 0x7f736c0ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55954/0 (socket says 172.21.15.125:55954) 2023-10-18T11:58:52.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f736a7fc640 1 -- 172.21.15.125:0/590494596 learned_addr learned my addr 172.21.15.125:0/590494596 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:52.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f736affd640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736c06d260 0x7f736c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:52.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.925+0000 7f736b7fe640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f736c159060 0x7f736c0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:52.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f736a7fc640 1 -- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f736c159060 msgr2=0x7f736c0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:52.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f736a7fc640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f736c159060 0x7f736c0cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:52.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f736a7fc640 1 -- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736c06d260 msgr2=0x7f736c156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f736a7fc640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736c06d260 0x7f736c156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f736a7fc640 1 -- 172.21.15.125:0/590494596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7364009490 con 0x7f736c0ce240 2023-10-18T11:58:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f736affd640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736c06d260 0x7f736c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f736b7fe640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f736c159060 0x7f736c0cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f736a7fc640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c0ce240 0x7f736c0ceae0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f736000c780 tx=0x7f736000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f73537fe640 1 -- 172.21.15.125:0/590494596 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360005270 con 0x7f736c0ce240 2023-10-18T11:58:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f73537fe640 1 -- 172.21.15.125:0/590494596 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f736000a640 con 0x7f736c0ce240 2023-10-18T11:58:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c0c45f0 con 0x7f736c0ce240 2023-10-18T11:58:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f73537fe640 1 -- 172.21.15.125:0/590494596 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000aa50 con 0x7f736c0ce240 2023-10-18T11:58:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.926+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736c0c4bb0 con 0x7f736c0ce240 2023-10-18T11:58:52.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.927+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736c159dd0 con 0x7f736c0ce240 2023-10-18T11:58:52.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.927+0000 7f73537fe640 1 -- 172.21.15.125:0/590494596 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 122) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f736000abb0 con 0x7f736c0ce240 2023-10-18T11:58:52.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.928+0000 7f73537fe640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f735806ace0 0x7f735806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:52.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.928+0000 7f73537fe640 1 -- 172.21.15.125:0/590494596 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f7360088fd0 con 0x7f736c0ce240 2023-10-18T11:58:52.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.928+0000 7f736affd640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f735806ace0 0x7f735806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:52.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.929+0000 7f736affd640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f735806ace0 0x7f735806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f735c009fc0 tx=0x7f735c009550 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:52.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:52.932+0000 7f73537fe640 1 -- 172.21.15.125:0/590494596 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7360054070 con 0x7f736c0ce240 2023-10-18T11:58:53.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.118+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 --> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f736c159ac0 con 0x7f735806ace0 2023-10-18T11:58:53.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.169+0000 7f73537fe640 1 -- 172.21.15.125:0/590494596 <== mgr.9871 v2:172.21.15.139:6832/3820883843 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f735c009bb0 con 0x7f735806ace0 2023-10-18T11:58:53.172 INFO:teuthology.orchestra.run.smithi125.stdout:Retention added to path snap_test_dir1 2023-10-18T11:58:53.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f735806ace0 msgr2=0x7f735806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:53.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f735806ace0 0x7f735806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f735c009fc0 tx=0x7f735c009550 comp rx=0 tx=0).stop 2023-10-18T11:58:53.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c0ce240 msgr2=0x7f736c0ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:53.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c0ce240 0x7f736c0ceae0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f736000c780 tx=0x7f736000cc40 comp rx=0 tx=0).stop 2023-10-18T11:58:53.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 shutdown_connections 2023-10-18T11:58:53.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f735806ace0 0x7f735806d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f736c159060 0x7f736c0cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736c0ce240 0x7f736c0ceae0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 --2- 172.21.15.125:0/590494596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736c06d260 0x7f736c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 >> 172.21.15.125:0/590494596 conn(0x7f736c0b0670 msgr2=0x7f736c0ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:53.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 shutdown_connections 2023-10-18T11:58:53.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.171+0000 7f7371632640 1 -- 172.21.15.125:0/590494596 wait complete. 2023-10-18T11:58:53.185 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-18T11:58:53.185 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-18-11_59_00 in ~7s... 2023-10-18T11:58:53.186 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-18T11:58:53.186 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-18T11:58:53.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.288+0000 7fc0fefe2640 1 -- 172.21.15.125:0/972276968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0f80cf340 msgr2=0x7fc0f80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:53.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.288+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/972276968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0f80cf340 0x7fc0f80d1910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8005fd0 tx=0x7fc0e802bd90 comp rx=0 tx=0).stop 2023-10-18T11:58:53.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.288+0000 7fc0fefe2640 1 -- 172.21.15.125:0/972276968 shutdown_connections 2023-10-18T11:58:53.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.288+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/972276968 >> v2:172.21.15.139:3300/0 conn(0x7fc0f80ceb40 0x7fc0f80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.288+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/972276968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0f80cf340 0x7fc0f80d1910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.288+0000 7fc0fefe2640 1 -- 172.21.15.125:0/972276968 >> 172.21.15.125:0/972276968 conn(0x7fc0f80d0d60 msgr2=0x7fc0f80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:53.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.288+0000 7fc0fefe2640 1 -- 172.21.15.125:0/972276968 shutdown_connections 2023-10-18T11:58:53.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.289+0000 7fc0fefe2640 1 -- 172.21.15.125:0/972276968 wait complete. 2023-10-18T11:58:53.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.290+0000 7fc0fefe2640 1 Processor -- start 2023-10-18T11:58:53.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.290+0000 7fc0fefe2640 1 -- start start 2023-10-18T11:58:53.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.290+0000 7fc0fefe2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0f80ceb40 0x7fc0f8159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:53.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.290+0000 7fc0fefe2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0f80cf340 0x7fc0f81e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:53.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.290+0000 7fc0fefe2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f815b2a0 0x7fc0f81e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:53.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.290+0000 7fc0fefe2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f81e4310 con 0x7fc0f80cf340 2023-10-18T11:58:53.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.290+0000 7fc0fefe2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f81e45f0 con 0x7fc0f815b2a0 2023-10-18T11:58:53.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.290+0000 7fc0fefe2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f81e48d0 con 0x7fc0f80ceb40 2023-10-18T11:58:53.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0f80cf340 0x7fc0f81e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:53.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0f80cf340 0x7fc0f81e53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55968/0 (socket says 172.21.15.125:55968) 2023-10-18T11:58:53.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 -- 172.21.15.125:0/2388853924 learned_addr learned my addr 172.21.15.125:0/2388853924 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T11:58:53.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0fcd7e640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0f80ceb40 0x7fc0f8159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:53.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0fd57f640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f815b2a0 0x7fc0f81e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:53.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 -- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0f80ceb40 msgr2=0x7fc0f8159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:53.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0f80ceb40 0x7fc0f8159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 -- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f815b2a0 msgr2=0x7fc0f81e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:53.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f815b2a0 0x7fc0f81e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 -- 172.21.15.125:0/2388853924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0009840 con 0x7fc0f80cf340 2023-10-18T11:58:53.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0fd57f640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f815b2a0 0x7fc0f81e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T11:58:53.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f7fff640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0f80cf340 0x7fc0f81e53b0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000c440 tx=0x7fc0e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:53.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f57fa640 1 -- 172.21.15.125:0/2388853924 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00052d0 con 0x7fc0f80cf340 2023-10-18T11:58:53.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f57fa640 1 -- 172.21.15.125:0/2388853924 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc0e0007020 con 0x7fc0f80cf340 2023-10-18T11:58:53.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e8009560 con 0x7fc0f80cf340 2023-10-18T11:58:53.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0f57fa640 1 -- 172.21.15.125:0/2388853924 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0007470 con 0x7fc0f80cf340 2023-10-18T11:58:53.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.291+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f81e4e90 con 0x7fc0f80cf340 2023-10-18T11:58:53.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.292+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f80690e0 con 0x7fc0f80cf340 2023-10-18T11:58:53.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.293+0000 7fc0f57fa640 1 -- 172.21.15.125:0/2388853924 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 122) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc0e00067d0 con 0x7fc0f80cf340 2023-10-18T11:58:53.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.293+0000 7fc0f57fa640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fc0d806ace0 0x7fc0d806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T11:58:53.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.293+0000 7fc0f57fa640 1 -- 172.21.15.125:0/2388853924 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fc0e0088a80 con 0x7fc0f80cf340 2023-10-18T11:58:53.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.294+0000 7fc0fcd7e640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fc0d806ace0 0x7fc0d806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T11:58:53.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.294+0000 7fc0fcd7e640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fc0d806ace0 0x7fc0d806d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8035000 tx=0x7fc0e8004730 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T11:58:53.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.298+0000 7fc0f57fa640 1 -- 172.21.15.125:0/2388853924 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0e0053c50 con 0x7fc0f80cf340 2023-10-18T11:58:53.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.484+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 --> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0f80cb9b0 con 0x7fc0d806ace0 2023-10-18T11:58:53.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.491+0000 7fc0f57fa640 1 -- 172.21.15.125:0/2388853924 <== mgr.9871 v2:172.21.15.139:6832/3820883843 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fc0f80cb9b0 con 0x7fc0d806ace0 2023-10-18T11:58:53.494 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T11:58:53.494 INFO:teuthology.orchestra.run.smithi125.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-18T11:58:53.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fc0d806ace0 msgr2=0x7fc0d806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:53.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fc0d806ace0 0x7fc0d806d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8035000 tx=0x7fc0e8004730 comp rx=0 tx=0).stop 2023-10-18T11:58:53.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0f80cf340 msgr2=0x7fc0f81e53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T11:58:53.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0f80cf340 0x7fc0f81e53b0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000c440 tx=0x7fc0e000c900 comp rx=0 tx=0).stop 2023-10-18T11:58:53.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 shutdown_connections 2023-10-18T11:58:53.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fc0d806ace0 0x7fc0d806d1a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f815b2a0 0x7fc0f81e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0f80cf340 0x7fc0f81e53b0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 --2- 172.21.15.125:0/2388853924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0f80ceb40 0x7fc0f8159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T11:58:53.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 >> 172.21.15.125:0/2388853924 conn(0x7fc0f80d0d60 msgr2=0x7fc0f806b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T11:58:53.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 shutdown_connections 2023-10-18T11:58:53.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T11:58:53.493+0000 7fc0fefe2640 1 -- 172.21.15.125:0/2388853924 wait complete. 2023-10-18T11:58:53.507 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-18T11:58:53.507 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-18T11:58:53.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:53.508 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:58:53.527 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T11:58:53.527 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:58:53.527 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:58:54.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:54.529 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:58:54.550 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T11:58:54.551 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:58:54.552 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:58:55.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:55.553 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:58:55.575 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T11:58:55.576 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:58:55.576 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:58:56.484 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:58:56.487 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:58:56.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:56.578 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:58:56.600 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T11:58:56.600 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:58:56.601 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:58:57.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:57.602 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:58:57.625 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T11:58:57.625 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:58:57.626 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:58:58.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:58.627 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:58:58.650 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T11:58:58.651 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:58:58.651 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:58:59.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:58:59.652 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:58:59.675 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-18T11:58:59.676 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:58:59.676 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:00.183 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:59:00.180+0000 7efd8ee08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:59:00.183 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:59:00.180+0000 7efd8ee08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:59:00.183 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:59:00.180+0000 7efd8ee08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:59:00.184 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:59:00.180+0000 7efd8ee08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:59:00.184 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T11:59:00.180+0000 7efd8ee08640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T11:59:00.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:00.678 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:00.701 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:00.701 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:00.702 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:00.702 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:00.702 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-18-11_59_00_UTC'] 2023-10-18T11:59:00.703 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697630333.1857429, now = 1697630340.7030685, timo = 7 2023-10-18T11:59:01.704 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-10-18-11_59_00 in ~67s... 2023-10-18T11:59:01.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:01.705 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:01.728 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:01.728 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:01.729 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:01.729 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:02.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:02.730 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:02.753 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:02.754 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:02.754 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:02.755 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:03.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:03.757 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:03.779 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:03.780 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:03.780 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:03.780 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:04.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:04.782 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:04.805 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:04.805 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:04.805 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:04.806 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:05.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:05.808 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:05.831 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:05.831 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:05.832 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:05.832 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:06.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:06.833 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:06.856 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:06.857 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:06.857 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:06.857 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:07.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:07.859 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:07.884 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:07.884 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:07.885 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:07.886 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:08.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:08.888 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:08.911 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:08.911 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:08.912 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:08.912 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:09.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:09.914 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:09.936 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:09.937 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:09.937 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:09.938 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:10.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:10.940 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:10.963 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:10.964 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:10.964 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:10.964 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:11.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:11.966 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:11.990 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:11.991 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:11.991 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:11.992 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:12.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:12.993 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:13.016 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:13.016 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:13.017 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:13.017 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:14.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:14.019 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:14.044 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:14.045 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:14.045 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:14.045 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:15.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:15.047 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:15.070 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:15.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:15.071 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:15.071 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:16.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:16.073 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:16.097 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:16.097 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:16.098 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:16.098 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:17.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:17.100 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:17.123 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:17.124 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:17.124 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:17.124 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:18.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:18.126 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:18.147 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:18.148 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:18.149 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:18.149 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:19.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:19.151 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:19.176 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:19.176 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:19.177 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:19.177 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:20.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:20.179 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:20.202 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:20.203 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:20.203 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:20.203 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:21.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:21.205 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:21.231 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:21.232 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:21.232 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:21.232 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:22.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:22.234 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:22.256 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:22.257 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:22.258 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:22.258 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:23.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:23.259 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:23.282 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:23.282 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:23.282 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:23.283 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:24.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:24.284 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:24.306 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:24.307 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:24.307 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:24.308 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:25.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:25.310 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:25.333 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:25.334 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:25.334 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:25.335 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:26.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:26.336 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:26.360 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:26.361 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:26.361 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:26.361 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:26.629 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:59:26.632 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:59:27.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:27.363 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:27.386 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:27.386 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:27.387 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:27.387 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:28.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:28.389 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:28.412 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:28.413 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:28.413 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:28.413 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:29.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:29.416 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:29.438 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:29.439 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:29.440 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:29.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:30.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:30.442 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:30.465 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:30.466 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:30.466 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:30.466 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:31.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:31.468 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:31.492 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:31.493 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:31.493 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:31.493 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:32.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:32.495 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:32.518 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:32.519 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:32.520 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:32.520 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:33.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:33.521 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:33.543 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:33.544 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:33.545 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:33.545 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:34.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:34.547 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:34.570 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:34.571 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:34.571 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:34.571 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:35.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:35.574 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:35.597 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:35.597 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:35.598 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:35.598 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:36.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:36.599 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:36.623 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:36.623 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:36.624 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:36.624 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:37.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:37.625 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:37.648 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:37.649 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:37.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:37.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:38.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:38.651 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:38.674 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:38.675 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:38.675 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:38.675 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:39.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:39.677 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:39.700 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:39.701 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:39.701 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:39.702 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:40.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:40.703 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:40.727 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:40.727 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:40.727 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:40.728 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:41.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:41.729 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:41.751 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:41.752 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:41.752 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:41.752 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:42.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:42.755 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:42.777 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:42.778 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:42.778 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:42.779 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:43.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:43.780 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:43.803 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:43.804 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:43.804 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:43.804 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:44.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:44.806 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:44.828 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:44.829 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:44.829 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:44.830 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:45.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:45.832 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:45.855 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:45.856 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:45.856 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:45.856 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:46.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:46.858 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:46.881 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:46.881 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:46.882 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:46.882 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:47.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:47.884 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:47.906 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:47.907 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:47.907 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:47.907 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:48.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:48.910 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:48.933 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:48.933 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:48.934 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:48.934 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:49.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:49.936 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:49.958 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:49.959 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:49.959 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:49.959 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:50.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:50.962 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:50.984 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:50.985 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:50.985 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:50.986 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:51.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:51.988 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:52.010 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:52.011 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:52.011 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:52.011 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:53.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:53.013 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:53.035 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:53.036 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:53.037 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:53.037 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:54.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:54.038 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:54.061 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:54.061 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:54.062 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:54.062 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:55.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:55.064 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:55.086 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:55.087 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:55.087 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:55.087 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:56.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:56.089 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:56.111 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:56.112 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:56.112 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:56.112 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:56.760 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:59:56.763 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T11:59:57.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:57.115 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:57.137 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:57.138 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:57.139 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:57.139 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:58.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:58.141 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:58.164 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:58.165 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:58.165 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:58.165 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T11:59:59.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T11:59:59.167 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T11:59:59.190 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T11:59:59.190 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T11:59:59.191 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T11:59:59.191 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T12:00:00.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:00.193 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T12:00:00.216 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-11_59_00_UTC 2023-10-18T12:00:00.217 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T12:00:00.217 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-18T12:00:00.217 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-18T12:00:01.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:01.219 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T12:00:01.243 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-12_00_00_UTC 2023-10-18T12:00:01.244 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-18-12_00_00_UTC'} 2023-10-18T12:00:01.244 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-18-12_00_00_UTC'} 2023-10-18T12:00:01.244 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-10-18-11_59_00_UTC'} 2023-10-18T12:00:01.245 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-10-18-11_59_00_UTC'] 2023-10-18T12:00:01.245 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697630333.1857429, now = 1697630401.245423, timo = 67 2023-10-18T12:00:02.246 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-18T12:00:02.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.358+0000 7f4b912c9640 1 -- 172.21.15.125:0/3221221567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 msgr2=0x7f4b8c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.358+0000 7f4b912c9640 1 --2- 172.21.15.125:0/3221221567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 0x7f4b8c06bb60 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c005fd0 tx=0x7f4b7c02c050 comp rx=0 tx=0).stop 2023-10-18T12:00:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.358+0000 7f4b912c9640 1 -- 172.21.15.125:0/3221221567 shutdown_connections 2023-10-18T12:00:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.358+0000 7f4b912c9640 1 --2- 172.21.15.125:0/3221221567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 0x7f4b8c06bb60 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.358+0000 7f4b912c9640 1 -- 172.21.15.125:0/3221221567 >> 172.21.15.125:0/3221221567 conn(0x7f4b8c069930 msgr2=0x7f4b8c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:02.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.358+0000 7f4b912c9640 1 -- 172.21.15.125:0/3221221567 shutdown_connections 2023-10-18T12:00:02.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.358+0000 7f4b912c9640 1 -- 172.21.15.125:0/3221221567 wait complete. 2023-10-18T12:00:02.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.360+0000 7f4b912c9640 1 Processor -- start 2023-10-18T12:00:02.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.360+0000 7f4b912c9640 1 -- start start 2023-10-18T12:00:02.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.360+0000 7f4b912c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b8c070af0 0x7f4b8c1ef930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:02.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.360+0000 7f4b912c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 0x7f4b8c0a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:02.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.360+0000 7f4b912c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b8c141690 0x7f4b8c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:02.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.360+0000 7f4b912c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c1eff60 con 0x7f4b8c13e520 2023-10-18T12:00:02.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.360+0000 7f4b912c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c1f0240 con 0x7f4b8c141690 2023-10-18T12:00:02.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.360+0000 7f4b912c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c1f0520 con 0x7f4b8c070af0 2023-10-18T12:00:02.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 0x7f4b8c0a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 0x7f4b8c0a64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39840/0 (socket says 172.21.15.125:39840) 2023-10-18T12:00:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 -- 172.21.15.125:0/3746901666 learned_addr learned my addr 172.21.15.125:0/3746901666 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8b7fe640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b8c141690 0x7f4b8c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8affd640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b8c070af0 0x7f4b8c1ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:02.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 -- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b8c070af0 msgr2=0x7f4b8c1ef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:02.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b8c070af0 0x7f4b8c1ef930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 -- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b8c141690 msgr2=0x7f4b8c0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:02.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b8c141690 0x7f4b8c0caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 -- 172.21.15.125:0/3746901666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c009840 con 0x7f4b8c13e520 2023-10-18T12:00:02.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8b7fe640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b8c141690 0x7f4b8c0caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:02.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b8a7fc640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 0x7f4b8c0a64b0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000c390 tx=0x7f4b8000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:02.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b737fe640 1 -- 172.21.15.125:0/3746901666 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80004230 con 0x7f4b8c13e520 2023-10-18T12:00:02.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b737fe640 1 -- 172.21.15.125:0/3746901666 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b800044a0 con 0x7f4b8c13e520 2023-10-18T12:00:02.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c1f0800 con 0x7f4b8c13e520 2023-10-18T12:00:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b737fe640 1 -- 172.21.15.125:0/3746901666 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80003850 con 0x7f4b8c13e520 2023-10-18T12:00:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.361+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c0a69f0 con 0x7f4b8c13e520 2023-10-18T12:00:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.362+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c1f0d20 con 0x7f4b8c13e520 2023-10-18T12:00:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.363+0000 7f4b737fe640 1 -- 172.21.15.125:0/3746901666 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 125) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f4b800039f0 con 0x7f4b8c13e520 2023-10-18T12:00:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.363+0000 7f4b737fe640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f4b7806b0d0 0x7f4b7806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.363+0000 7f4b737fe640 1 -- 172.21.15.125:0/3746901666 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f4b800894c0 con 0x7f4b8c13e520 2023-10-18T12:00:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.363+0000 7f4b8affd640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f4b7806b0d0 0x7f4b7806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.364+0000 7f4b8affd640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f4b7806b0d0 0x7f4b7806d590 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c035000 tx=0x7f4b7c004ab0 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.367+0000 7f4b737fe640 1 -- 172.21.15.125:0/3746901666 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b8c1f0d20 con 0x7f4b8c13e520 2023-10-18T12:00:02.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.556+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 --> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b8c1f03b0 con 0x7f4b7806b0d0 2023-10-18T12:00:02.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.611+0000 7f4b737fe640 1 -- 172.21.15.125:0/3746901666 <== mgr.9871 v2:172.21.15.139:6832/3820883843 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4b8c1f03b0 con 0x7f4b7806b0d0 2023-10-18T12:00:02.614 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path snap_test_dir1 2023-10-18T12:00:02.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f4b7806b0d0 msgr2=0x7f4b7806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:02.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f4b7806b0d0 0x7f4b7806d590 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c035000 tx=0x7f4b7c004ab0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 msgr2=0x7f4b8c0a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 0x7f4b8c0a64b0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000c390 tx=0x7f4b8000c850 comp rx=0 tx=0).stop 2023-10-18T12:00:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 shutdown_connections 2023-10-18T12:00:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f4b7806b0d0 0x7f4b7806d590 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b8c141690 0x7f4b8c0caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b8c13e520 0x7f4b8c0a64b0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 --2- 172.21.15.125:0/3746901666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b8c070af0 0x7f4b8c1ef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 >> 172.21.15.125:0/3746901666 conn(0x7f4b8c069930 msgr2=0x7f4b8c0c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 shutdown_connections 2023-10-18T12:00:02.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.613+0000 7f4b912c9640 1 -- 172.21.15.125:0/3746901666 wait complete. 2023-10-18T12:00:02.626 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-18T12:00:02.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:02.627 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-18T12:00:02.646 INFO:teuthology.orchestra.run.smithi125.stdout:scheduled-2023-10-18-12_00_00_UTC 2023-10-18T12:00:02.646 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-18-12_00_00_UTC 2023-10-18T12:00:02.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:02.647 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-18-12_00_00_UTC) 2023-10-18T12:00:02.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:02.708 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-18T12:00:02.767 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T12:00:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.908+0000 7f18c1264640 1 -- 172.21.15.125:0/3231962350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 msgr2=0x7f18bc0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.908+0000 7f18c1264640 1 --2- 172.21.15.125:0/3231962350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 0x7f18bc0b0650 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009830 tx=0x7f18b402bcd0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.908+0000 7f18c1264640 1 -- 172.21.15.125:0/3231962350 shutdown_connections 2023-10-18T12:00:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.908+0000 7f18c1264640 1 --2- 172.21.15.125:0/3231962350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 0x7f18bc0b0650 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.908+0000 7f18c1264640 1 -- 172.21.15.125:0/3231962350 >> 172.21.15.125:0/3231962350 conn(0x7f18bc0d0d60 msgr2=0x7f18bc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.908+0000 7f18c1264640 1 -- 172.21.15.125:0/3231962350 shutdown_connections 2023-10-18T12:00:02.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.908+0000 7f18c1264640 1 -- 172.21.15.125:0/3231962350 wait complete. 2023-10-18T12:00:02.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.909+0000 7f18c1264640 1 Processor -- start 2023-10-18T12:00:02.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.909+0000 7f18c1264640 1 -- start start 2023-10-18T12:00:02.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18c1264640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc0cf340 0x7f18bc0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:02.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18c1264640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 0x7f18bc154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:02.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18c1264640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc15b2a0 0x7f18bc154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:02.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18c1264640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc157060 con 0x7f18bc0d1910 2023-10-18T12:00:02.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18c1264640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0b3c70 con 0x7f18bc15b2a0 2023-10-18T12:00:02.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18c1264640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0b3f50 con 0x7f18bc0cf340 2023-10-18T12:00:02.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18ba7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 0x7f18bc154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:02.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18ba7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 0x7f18bc154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39858/0 (socket says 172.21.15.125:39858) 2023-10-18T12:00:02.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18ba7fc640 1 -- 172.21.15.125:0/2610249711 learned_addr learned my addr 172.21.15.125:0/2610249711 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:02.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18bb7fe640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc15b2a0 0x7f18bc154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:02.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18ba7fc640 1 -- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc0cf340 msgr2=0x7f18bc0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:02.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18baffd640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc0cf340 0x7f18bc0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:02.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18ba7fc640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc0cf340 0x7f18bc0c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18ba7fc640 1 -- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc15b2a0 msgr2=0x7f18bc154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:02.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18ba7fc640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc15b2a0 0x7f18bc154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:02.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.910+0000 7f18ba7fc640 1 -- 172.21.15.125:0/2610249711 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b4009490 con 0x7f18bc0d1910 2023-10-18T12:00:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.911+0000 7f18ba7fc640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 0x7f18bc154150 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f18b000b3e0 tx=0x7f18b000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.911+0000 7f18a37fe640 1 -- 172.21.15.125:0/2610249711 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0011830 con 0x7f18bc0d1910 2023-10-18T12:00:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.911+0000 7f18a37fe640 1 -- 172.21.15.125:0/2610249711 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18b0011aa0 con 0x7f18bc0d1910 2023-10-18T12:00:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.911+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18bc0b4230 con 0x7f18bc0d1910 2023-10-18T12:00:02.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.911+0000 7f18a37fe640 1 -- 172.21.15.125:0/2610249711 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0010380 con 0x7f18bc0d1910 2023-10-18T12:00:02.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.911+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18bc0c93b0 con 0x7f18bc0d1910 2023-10-18T12:00:02.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.912+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18bc0c95d0 con 0x7f18bc0d1910 2023-10-18T12:00:02.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.912+0000 7f18a37fe640 1 -- 172.21.15.125:0/2610249711 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 125) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f18b00104e0 con 0x7f18bc0d1910 2023-10-18T12:00:02.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.912+0000 7f18a37fe640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f18a806b0d0 0x7f18a806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:02.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.913+0000 7f18a37fe640 1 -- 172.21.15.125:0/2610249711 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f18b008a240 con 0x7f18bc0d1910 2023-10-18T12:00:02.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.913+0000 7f18baffd640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f18a806b0d0 0x7f18a806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:02.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.913+0000 7f18baffd640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f18a806b0d0 0x7f18a806d590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18ac004a30 tx=0x7f18ac009040 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:02.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:02.917+0000 7f18a37fe640 1 -- 172.21.15.125:0/2610249711 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18b0054bb0 con 0x7f18bc0d1910 2023-10-18T12:00:03.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.126+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f18bc0b07f0 con 0x7f18bc0d1910 2023-10-18T12:00:03.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.383+0000 7f18a37fe640 1 -- 172.21.15.125:0/2610249711 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v126) v1 ==== 93+0+0 (secure 0 0 0) 0x7f18b00581d0 con 0x7f18bc0d1910 2023-10-18T12:00:03.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f18a806b0d0 msgr2=0x7f18a806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f18a806b0d0 0x7f18a806d590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18ac004a30 tx=0x7f18ac009040 comp rx=0 tx=0).stop 2023-10-18T12:00:03.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 msgr2=0x7f18bc154150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 0x7f18bc154150 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f18b000b3e0 tx=0x7f18b000b8a0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 shutdown_connections 2023-10-18T12:00:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f18a806b0d0 0x7f18a806d590 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18ac004a30 tx=0x7f18ac009040 comp rx=0 tx=0).stop 2023-10-18T12:00:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc15b2a0 0x7f18bc154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18bc0d1910 0x7f18bc154150 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 --2- 172.21.15.125:0/2610249711 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc0cf340 0x7f18bc0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 >> 172.21.15.125:0/2610249711 conn(0x7f18bc0d0d60 msgr2=0x7f18bc0b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:03.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 shutdown_connections 2023-10-18T12:00:03.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.386+0000 7f18c1264640 1 -- 172.21.15.125:0/2610249711 wait complete. 2023-10-18T12:00:03.401 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T12:00:03.402 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:00:03.441 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:00:03.483 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T12:00:03.484 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:00:03.484 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:03.500 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:03.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.640+0000 7fdacdbc7640 1 -- 172.21.15.125:0/231630462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 msgr2=0x7fdac80d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.640+0000 7fdacdbc7640 1 --2- 172.21.15.125:0/231630462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 0x7fdac80d1910 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009830 tx=0x7fdab802bbb0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.640+0000 7fdacdbc7640 1 -- 172.21.15.125:0/231630462 shutdown_connections 2023-10-18T12:00:03.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.640+0000 7fdacdbc7640 1 --2- 172.21.15.125:0/231630462 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 0x7fdac80d1910 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.640+0000 7fdacdbc7640 1 -- 172.21.15.125:0/231630462 >> 172.21.15.125:0/231630462 conn(0x7fdac80d0d60 msgr2=0x7fdac80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:03.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.640+0000 7fdacdbc7640 1 -- 172.21.15.125:0/231630462 shutdown_connections 2023-10-18T12:00:03.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.641+0000 7fdacdbc7640 1 -- 172.21.15.125:0/231630462 wait complete. 2023-10-18T12:00:03.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.642+0000 7fdacdbc7640 1 Processor -- start 2023-10-18T12:00:03.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.642+0000 7fdacdbc7640 1 -- start start 2023-10-18T12:00:03.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.642+0000 7fdacdbc7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdac80ceb40 0x7fdac814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:03.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.642+0000 7fdacdbc7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 0x7fdac81e5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:03.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.642+0000 7fdacdbc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac815b2a0 0x7fdac81e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:03.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.642+0000 7fdacdbc7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8151c40 con 0x7fdac80cf340 2023-10-18T12:00:03.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.642+0000 7fdacdbc7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8150470 con 0x7fdac815b2a0 2023-10-18T12:00:03.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.642+0000 7fdacdbc7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac815b850 con 0x7fdac80ceb40 2023-10-18T12:00:03.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 0x7fdac81e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:03.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac77fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdac80ceb40 0x7fdac814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:03.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 0x7fdac81e5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39888/0 (socket says 172.21.15.125:39888) 2023-10-18T12:00:03.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac815b2a0 0x7fdac81e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 -- 172.21.15.125:0/721688663 learned_addr learned my addr 172.21.15.125:0/721688663 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 -- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdac80ceb40 msgr2=0x7fdac814ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdac80ceb40 0x7fdac814ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 -- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac815b2a0 msgr2=0x7fdac81e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac815b2a0 0x7fdac81e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 -- 172.21.15.125:0/721688663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8009490 con 0x7fdac80cf340 2023-10-18T12:00:03.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac77fe640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdac80ceb40 0x7fdac814ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:03.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac7fff640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac815b2a0 0x7fdac81e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:03.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdac6ffd640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 0x7fdac81e5460 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00c410 tx=0x7fdabc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:03.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdaaffff640 1 -- 172.21.15.125:0/721688663 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0035f0 con 0x7fdac80cf340 2023-10-18T12:00:03.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdaaffff640 1 -- 172.21.15.125:0/721688663 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdabc003770 con 0x7fdac80cf340 2023-10-18T12:00:03.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac815bb30 con 0x7fdac80cf340 2023-10-18T12:00:03.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdaaffff640 1 -- 172.21.15.125:0/721688663 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc003b80 con 0x7fdac80cf340 2023-10-18T12:00:03.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.643+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac815c0f0 con 0x7fdac80cf340 2023-10-18T12:00:03.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.645+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac80690e0 con 0x7fdac80cf340 2023-10-18T12:00:03.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.645+0000 7fdaaffff640 1 -- 172.21.15.125:0/721688663 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fdabc003ce0 con 0x7fdac80cf340 2023-10-18T12:00:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.645+0000 7fdaaffff640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fdab406b080 0x7fdab406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.645+0000 7fdaaffff640 1 -- 172.21.15.125:0/721688663 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fdabc089590 con 0x7fdac80cf340 2023-10-18T12:00:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.645+0000 7fdac77fe640 1 -- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fdab406b080 msgr2=0x7fdab406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.645+0000 7fdac77fe640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fdab406b080 0x7fdab406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:03.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.650+0000 7fdaaffff640 1 -- 172.21.15.125:0/721688663 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdabc053fb0 con 0x7fdac80cf340 2023-10-18T12:00:03.777 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:03.775+0000 7f7fdac32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:00:03.779 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:03.777+0000 7f63c423b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:00:03.822 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:03.820+0000 7f7fdac32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:00:03.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.821+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdac80d0b40 con 0x7fdac80cf340 2023-10-18T12:00:03.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.822+0000 7fdaaffff640 1 -- 172.21.15.125:0/721688663 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+17658 (secure 0 0 0) 0x7fdabc0575d0 con 0x7fdac80cf340 2023-10-18T12:00:03.825 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:03.825 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":207,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:59:53.406049+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:58:34.060453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-18T11:58:34.823261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.13","mappings":[{"from":1,"to":6}]},{"pgid":"26.15","mappings":[{"from":3,"to":6}]},{"pgid":"26.19","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/870215635":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/3819578023":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/330535304":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2863286615":"2023-10-19T11:58:51.730016+0000","172.21.15.139:6833/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2575337659":"2023-10-19T11:58:51.730016+0000","172.21.15.139:6832/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/1682437":"2023-10-19T11:58:51.730016+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:03.826 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:03.823+0000 7f63c423b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:00:03.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fdab406b080 msgr2=0x7fdab406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:03.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fdab406b080 0x7fdab406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 msgr2=0x7fdac81e5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 0x7fdac81e5460 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00c410 tx=0x7fdabc00c8d0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 shutdown_connections 2023-10-18T12:00:03.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fdab406b080 0x7fdab406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac815b2a0 0x7fdac81e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdac80cf340 0x7fdac81e5460 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 --2- 172.21.15.125:0/721688663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdac80ceb40 0x7fdac814ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 >> 172.21.15.125:0/721688663 conn(0x7fdac80d0d60 msgr2=0x7fdac80cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:03.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 shutdown_connections 2023-10-18T12:00:03.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.824+0000 7fdacdbc7640 1 -- 172.21.15.125:0/721688663 wait complete. 2023-10-18T12:00:03.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T11:58:34.060453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '203', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T11:58:34.823261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '204', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:00:03.840 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:00:03.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.944+0000 7fe168b1d640 1 -- 172.21.15.125:0/2880460815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 msgr2=0x7fe1640b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.944+0000 7fe168b1d640 1 --2- 172.21.15.125:0/2880460815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 0x7fe1640b0650 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fe16408a810 tx=0x7fe15c02c260 comp rx=0 tx=0).stop 2023-10-18T12:00:03.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.944+0000 7fe168b1d640 1 -- 172.21.15.125:0/2880460815 shutdown_connections 2023-10-18T12:00:03.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.944+0000 7fe168b1d640 1 --2- 172.21.15.125:0/2880460815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 0x7fe1640b0650 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.944+0000 7fe168b1d640 1 --2- 172.21.15.125:0/2880460815 >> v2:172.21.15.139:3300/0 conn(0x7fe1640cf340 0x7fe1640d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.944+0000 7fe168b1d640 1 -- 172.21.15.125:0/2880460815 >> 172.21.15.125:0/2880460815 conn(0x7fe1640d0d60 msgr2=0x7fe1640d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:03.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.944+0000 7fe168b1d640 1 -- 172.21.15.125:0/2880460815 shutdown_connections 2023-10-18T12:00:03.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.944+0000 7fe168b1d640 1 -- 172.21.15.125:0/2880460815 wait complete. 2023-10-18T12:00:03.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.945+0000 7fe168b1d640 1 Processor -- start 2023-10-18T12:00:03.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.945+0000 7fe168b1d640 1 -- start start 2023-10-18T12:00:03.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe168b1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 0x7fe1640aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:03.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe168b1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1640cf340 0x7fe1640b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:03.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe168b1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe16415b2a0 0x7fe1640b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:03.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe168b1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640b6610 con 0x7fe1640ceb40 2023-10-18T12:00:03.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe168b1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640b5a00 con 0x7fe1640cf340 2023-10-18T12:00:03.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe168b1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640b5ce0 con 0x7fe16415b2a0 2023-10-18T12:00:03.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 0x7fe1640aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:03.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 0x7fe1640aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39906/0 (socket says 172.21.15.125:39906) 2023-10-18T12:00:03.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 -- 172.21.15.125:0/1453915919 learned_addr learned my addr 172.21.15.125:0/1453915919 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:03.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe162d9d640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe16415b2a0 0x7fe1640b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:03.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe15bfff640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1640cf340 0x7fe1640b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:03.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 -- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe16415b2a0 msgr2=0x7fe1640b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe16415b2a0 0x7fe1640b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 -- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1640cf340 msgr2=0x7fe1640b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:03.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1640cf340 0x7fe1640b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:03.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 -- 172.21.15.125:0/1453915919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c009900 con 0x7fe1640ceb40 2023-10-18T12:00:03.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe16259c640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 0x7fe1640aef30 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fe15400e9f0 tx=0x7fe15400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:03.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe15affd640 1 -- 172.21.15.125:0/1453915919 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154009700 con 0x7fe1640ceb40 2023-10-18T12:00:03.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe15affd640 1 -- 172.21.15.125:0/1453915919 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe15400ce70 con 0x7fe1640ceb40 2023-10-18T12:00:03.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.946+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1640b5fc0 con 0x7fe1640ceb40 2023-10-18T12:00:03.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.947+0000 7fe15affd640 1 -- 172.21.15.125:0/1453915919 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154010670 con 0x7fe1640ceb40 2023-10-18T12:00:03.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.947+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1640b1690 con 0x7fe1640ceb40 2023-10-18T12:00:03.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.947+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1640690e0 con 0x7fe1640ceb40 2023-10-18T12:00:03.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.948+0000 7fe15affd640 1 -- 172.21.15.125:0/1453915919 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fe1540107d0 con 0x7fe1640ceb40 2023-10-18T12:00:03.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.949+0000 7fe15affd640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe14406b080 0x7fe14406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:03.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.949+0000 7fe15affd640 1 -- 172.21.15.125:0/1453915919 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fe154014070 con 0x7fe1640ceb40 2023-10-18T12:00:03.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.949+0000 7fe15bfff640 1 -- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe14406b080 msgr2=0x7fe14406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:03.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.949+0000 7fe15bfff640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe14406b080 0x7fe14406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:03.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:03.953+0000 7fe15affd640 1 -- 172.21.15.125:0/1453915919 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe154051ca0 con 0x7fe1640ceb40 2023-10-18T12:00:04.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.122+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1640d0b40 con 0x7fe1640ceb40 2023-10-18T12:00:04.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.123+0000 7fe15affd640 1 -- 172.21.15.125:0/1453915919 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v207) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe154055490 con 0x7fe1640ceb40 2023-10-18T12:00:04.126 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:00:04.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.124+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe14406b080 msgr2=0x7fe14406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:04.127 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.124+0000 7fe168b1d640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe14406b080 0x7fe14406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.124+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 msgr2=0x7fe1640aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.124+0000 7fe168b1d640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 0x7fe1640aef30 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fe15400e9f0 tx=0x7fe15400eeb0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.124+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 shutdown_connections 2023-10-18T12:00:04.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.124+0000 7fe168b1d640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe14406b080 0x7fe14406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.124+0000 7fe168b1d640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1640cf340 0x7fe1640b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.125+0000 7fe168b1d640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe16415b2a0 0x7fe1640b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.125+0000 7fe168b1d640 1 --2- 172.21.15.125:0/1453915919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe1640ceb40 0x7fe1640aef30 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.125+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 >> 172.21.15.125:0/1453915919 conn(0x7fe1640d0d60 msgr2=0x7fe1640cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:04.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.125+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 shutdown_connections 2023-10-18T12:00:04.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.125+0000 7fe168b1d640 1 -- 172.21.15.125:0/1453915919 wait complete. 2023-10-18T12:00:04.138 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:00:04.216 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.214+0000 7f63c423b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:00:04.229 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.226+0000 7f7fdac32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:00:04.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.242+0000 7f381e9a9640 1 -- 172.21.15.125:0/3483488366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 msgr2=0x7f3800003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.242+0000 7f381e9a9640 1 --2- 172.21.15.125:0/3483488366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 0x7f3800003d90 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f380800c290 tx=0x7f38000057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.242+0000 7f381cf46640 1 -- 172.21.15.125:0/3483488366 reap_dead start 2023-10-18T12:00:04.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.242+0000 7f381e9a9640 1 -- 172.21.15.125:0/3483488366 shutdown_connections 2023-10-18T12:00:04.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.242+0000 7f381e9a9640 1 -- 172.21.15.125:0/3483488366 >> 172.21.15.125:0/3483488366 conn(0x7f38180d0d60 msgr2=0x7f38180d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:04.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.242+0000 7f381e9a9640 1 -- 172.21.15.125:0/3483488366 shutdown_connections 2023-10-18T12:00:04.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.242+0000 7f381e9a9640 1 -- 172.21.15.125:0/3483488366 wait complete. 2023-10-18T12:00:04.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.243+0000 7f381e9a9640 1 Processor -- start 2023-10-18T12:00:04.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.243+0000 7f381e9a9640 1 -- start start 2023-10-18T12:00:04.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f381e9a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 0x7f3818158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f381e9a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38180b59a0 0x7f38180b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f381e9a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38180b3a30 0x7f38180b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f381e9a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38181591e0 con 0x7f3800003970 2023-10-18T12:00:04.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f381e9a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38180b6360 con 0x7f38180b3a30 2023-10-18T12:00:04.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f381e9a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38180c7e30 con 0x7f38180b59a0 2023-10-18T12:00:04.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f3817fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 0x7f3818158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f3817fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 0x7f3818158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39930/0 (socket says 172.21.15.125:39930) 2023-10-18T12:00:04.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f381cf46640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38180b3a30 0x7f38180b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f3817fff640 1 -- 172.21.15.125:0/1809085937 learned_addr learned my addr 172.21.15.125:0/1809085937 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:04.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.244+0000 7f38177fe640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38180b59a0 0x7f38180b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f381cf46640 1 -- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38180b59a0 msgr2=0x7f38180b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f381cf46640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38180b59a0 0x7f38180b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f381cf46640 1 -- 172.21.15.125:0/1809085937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 msgr2=0x7f3818158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f381cf46640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 0x7f3818158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f381cf46640 1 -- 172.21.15.125:0/1809085937 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180b4760 con 0x7f38180b3a30 2023-10-18T12:00:04.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f3817fff640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 0x7f3818158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:04.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f381cf46640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38180b3a30 0x7f38180b3eb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f38180ce5d0 tx=0x7f3810011ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:04.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f3814ff9640 1 -- 172.21.15.125:0/1809085937 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810004b20 con 0x7f38180b3a30 2023-10-18T12:00:04.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f3814ff9640 1 -- 172.21.15.125:0/1809085937 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3810004c80 con 0x7f38180b3a30 2023-10-18T12:00:04.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38180b4a40 con 0x7f38180b3a30 2023-10-18T12:00:04.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f3814ff9640 1 -- 172.21.15.125:0/1809085937 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810003bd0 con 0x7f38180b3a30 2023-10-18T12:00:04.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.245+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38180b5000 con 0x7f38180b3a30 2023-10-18T12:00:04.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.246+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38180690e0 con 0x7f38180b3a30 2023-10-18T12:00:04.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.246+0000 7f3814ff9640 1 -- 172.21.15.125:0/1809085937 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f3810003d30 con 0x7f38180b3a30 2023-10-18T12:00:04.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.247+0000 7f3814ff9640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f380406b0b0 0x7f380406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.247+0000 7f3817fff640 1 -- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f380406b0b0 msgr2=0x7f380406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:04.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.247+0000 7f3814ff9640 1 -- 172.21.15.125:0/1809085937 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f381008aab0 con 0x7f38180b3a30 2023-10-18T12:00:04.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.247+0000 7f3817fff640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f380406b0b0 0x7f380406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:04.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.251+0000 7f3814ff9640 1 -- 172.21.15.125:0/1809085937 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3810055300 con 0x7f38180b3a30 2023-10-18T12:00:04.261 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.259+0000 7f63c423b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:00:04.276 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.274+0000 7f7fdac32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:00:04.304 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.302+0000 7f63c423b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:00:04.319 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.317+0000 7f7fdac32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:00:04.356 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.354+0000 7f63c423b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:00:04.371 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.369+0000 7f7fdac32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:00:04.399 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.396+0000 7f63c423b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:00:04.416 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.414+0000 7f7fdac32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:00:04.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.422+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38180c7f70 con 0x7f38180b3a30 2023-10-18T12:00:04.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.422+0000 7f3814ff9640 1 -- 172.21.15.125:0/1809085937 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v207) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3810058920 con 0x7f38180b3a30 2023-10-18T12:00:04.425 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:00:04.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f380406b0b0 msgr2=0x7f380406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:04.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f380406b0b0 0x7f380406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38180b3a30 msgr2=0x7f38180b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38180b3a30 0x7f38180b3eb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f38180ce5d0 tx=0x7f3810011ef0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 shutdown_connections 2023-10-18T12:00:04.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f380406b0b0 0x7f380406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38180b3a30 0x7f38180b3eb0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38180b59a0 0x7f38180b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 --2- 172.21.15.125:0/1809085937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3800003970 0x7f3818158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 >> 172.21.15.125:0/1809085937 conn(0x7f38180d0d60 msgr2=0x7f381806b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:04.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 shutdown_connections 2023-10-18T12:00:04.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.425+0000 7f381e9a9640 1 -- 172.21.15.125:0/1809085937 wait complete. 2023-10-18T12:00:04.439 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:00:04.498 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.495+0000 7f63c423b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:00:04.522 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.519+0000 7f7fdac32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:00:04.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.546+0000 7fe8e753c640 1 -- 172.21.15.125:0/2679143330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e00cf340 msgr2=0x7fe8e00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.546+0000 7fe8e753c640 1 --2- 172.21.15.125:0/2679143330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e00cf340 0x7fe8e00d1910 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0005fd0 tx=0x7fe8d002bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:04.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.546+0000 7fe8e753c640 1 -- 172.21.15.125:0/2679143330 shutdown_connections 2023-10-18T12:00:04.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.546+0000 7fe8e753c640 1 --2- 172.21.15.125:0/2679143330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e015b2a0 0x7fe8e00cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.546+0000 7fe8e753c640 1 --2- 172.21.15.125:0/2679143330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e00cf340 0x7fe8e00d1910 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.546+0000 7fe8e753c640 1 -- 172.21.15.125:0/2679143330 >> 172.21.15.125:0/2679143330 conn(0x7fe8e00d0d60 msgr2=0x7fe8e00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:04.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.546+0000 7fe8e753c640 1 -- 172.21.15.125:0/2679143330 shutdown_connections 2023-10-18T12:00:04.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.546+0000 7fe8e753c640 1 -- 172.21.15.125:0/2679143330 wait complete. 2023-10-18T12:00:04.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.547+0000 7fe8e753c640 1 Processor -- start 2023-10-18T12:00:04.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.547+0000 7fe8e753c640 1 -- start start 2023-10-18T12:00:04.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e753c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e00ceb40 0x7fe8e015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e753c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e00cf340 0x7fe8e015cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e753c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e015b2a0 0x7fe8e00cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e753c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0154da0 con 0x7fe8e015b2a0 2023-10-18T12:00:04.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e753c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e015e200 con 0x7fe8e00cf340 2023-10-18T12:00:04.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e753c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00b52d0 con 0x7fe8e00ceb40 2023-10-18T12:00:04.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e5ad9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e015b2a0 0x7fe8e00cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e5ad9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e015b2a0 0x7fe8e00cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39958/0 (socket says 172.21.15.125:39958) 2023-10-18T12:00:04.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e5ad9640 1 -- 172.21.15.125:0/485244494 learned_addr learned my addr 172.21.15.125:0/485244494 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:04.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e4ad7640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e00cf340 0x7fe8e015cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.548+0000 7fe8e52d8640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e00ceb40 0x7fe8e015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8e5ad9640 1 -- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e00ceb40 msgr2=0x7fe8e015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8e5ad9640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e00ceb40 0x7fe8e015db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8e5ad9640 1 -- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e00cf340 msgr2=0x7fe8e015cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8e5ad9640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e00cf340 0x7fe8e015cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8e5ad9640 1 -- 172.21.15.125:0/485244494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d0009560 con 0x7fe8e015b2a0 2023-10-18T12:00:04.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8e5ad9640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e015b2a0 0x7fe8e00cc660 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800c800 tx=0x7fe8d800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:04.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8d5ffb640 1 -- 172.21.15.125:0/485244494 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8003b70 con 0x7fe8e015b2a0 2023-10-18T12:00:04.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8d5ffb640 1 -- 172.21.15.125:0/485244494 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8d8003cf0 con 0x7fe8e015b2a0 2023-10-18T12:00:04.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e015d4c0 con 0x7fe8e015b2a0 2023-10-18T12:00:04.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8d5ffb640 1 -- 172.21.15.125:0/485244494 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8003ea0 con 0x7fe8e015b2a0 2023-10-18T12:00:04.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.549+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e00b5540 con 0x7fe8e015b2a0 2023-10-18T12:00:04.559 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.552+0000 7f63c423b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:04.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.550+0000 7fe8d5ffb640 1 -- 172.21.15.125:0/485244494 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fe8d800aab0 con 0x7fe8e015b2a0 2023-10-18T12:00:04.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.550+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e00b5130 con 0x7fe8e015b2a0 2023-10-18T12:00:04.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.551+0000 7fe8d5ffb640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe8c406b080 0x7fe8c406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.551+0000 7fe8e52d8640 1 -- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe8c406b080 msgr2=0x7fe8c406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:04.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.551+0000 7fe8d5ffb640 1 -- 172.21.15.125:0/485244494 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fe8d8089c20 con 0x7fe8e015b2a0 2023-10-18T12:00:04.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.551+0000 7fe8e52d8640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe8c406b080 0x7fe8c406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:04.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.555+0000 7fe8d5ffb640 1 -- 172.21.15.125:0/485244494 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8e00b5130 con 0x7fe8e015b2a0 2023-10-18T12:00:04.590 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.588+0000 7f7fdac32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:04.596 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.594+0000 7f63c423b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:00:04.632 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.630+0000 7f7fdac32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:00:04.644 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.641+0000 7f63c423b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:00:04.681 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.678+0000 7f7fdac32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:00:04.686 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.683+0000 7f63c423b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:00:04.724 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:04.721+0000 7f7fdac32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:00:04.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.737+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8e00b5130 con 0x7fe8e015b2a0 2023-10-18T12:00:04.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.737+0000 7fe8d5ffb640 1 -- 172.21.15.125:0/485244494 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v207) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8e00b5130 con 0x7fe8e015b2a0 2023-10-18T12:00:04.740 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:00:04.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.739+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe8c406b080 msgr2=0x7fe8c406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:04.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.739+0000 7fe8e753c640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe8c406b080 0x7fe8c406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.739+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e015b2a0 msgr2=0x7fe8e00cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.739+0000 7fe8e753c640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e015b2a0 0x7fe8e00cc660 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800c800 tx=0x7fe8d800ccc0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.739+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 shutdown_connections 2023-10-18T12:00:04.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.739+0000 7fe8e753c640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fe8c406b080 0x7fe8c406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.740+0000 7fe8e753c640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e00cf340 0x7fe8e015cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.740+0000 7fe8e753c640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8e015b2a0 0x7fe8e00cc660 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.740+0000 7fe8e753c640 1 --2- 172.21.15.125:0/485244494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e00ceb40 0x7fe8e015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.740+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 >> 172.21.15.125:0/485244494 conn(0x7fe8e00d0d60 msgr2=0x7fe8e00b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:04.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.740+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 shutdown_connections 2023-10-18T12:00:04.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.740+0000 7fe8e753c640 1 -- 172.21.15.125:0/485244494 wait complete. 2023-10-18T12:00:04.753 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T12:00:04.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.856+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3609226728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406b030 msgr2=0x7fa2f406b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.856+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3609226728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406b030 0x7fa2f406b410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009830 tx=0x7fa2e402bcd0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.856+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3609226728 shutdown_connections 2023-10-18T12:00:04.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.856+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3609226728 >> v2:172.21.15.139:3300/0 conn(0x7fa2f40c9750 0x7fa2f40c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.856+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3609226728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f406b030 0x7fa2f406b410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.856+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3609226728 >> 172.21.15.125:0/3609226728 conn(0x7fa2f4069930 msgr2=0x7fa2f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:04.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.856+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3609226728 shutdown_connections 2023-10-18T12:00:04.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.856+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3609226728 wait complete. 2023-10-18T12:00:04.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.857+0000 7fa2f92a9640 1 Processor -- start 2023-10-18T12:00:04.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.857+0000 7fa2f92a9640 1 -- start start 2023-10-18T12:00:04.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f92a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f406b030 0x7fa2f415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f92a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f40c9750 0x7fa2f40a7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f92a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f4159bd0 0x7fa2f40a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f92a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f4154070 con 0x7fa2f406b030 2023-10-18T12:00:04.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f92a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40a7fa0 con 0x7fa2f4159bd0 2023-10-18T12:00:04.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f92a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40a8270 con 0x7fa2f40c9750 2023-10-18T12:00:04.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f406b030 0x7fa2f415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f406b030 0x7fa2f415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39970/0 (socket says 172.21.15.125:39970) 2023-10-18T12:00:04.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 -- 172.21.15.125:0/3461703663 learned_addr learned my addr 172.21.15.125:0/3461703663 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:04.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f37fe640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f4159bd0 0x7fa2f40a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f27fc640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f40c9750 0x7fa2f40a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:04.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 -- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f40c9750 msgr2=0x7fa2f40a7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f40c9750 0x7fa2f40a7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 -- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f4159bd0 msgr2=0x7fa2f40a7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:04.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f4159bd0 0x7fa2f40a7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:04.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 -- 172.21.15.125:0/3461703663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec009840 con 0x7fa2f406b030 2023-10-18T12:00:04.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.858+0000 7fa2f2ffd640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f406b030 0x7fa2f415ab40 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c00 tx=0x7fa2e4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:04.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.859+0000 7fa2db7fe640 1 -- 172.21.15.125:0/3461703663 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4003e20 con 0x7fa2f406b030 2023-10-18T12:00:04.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.859+0000 7fa2db7fe640 1 -- 172.21.15.125:0/3461703663 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2e4034440 con 0x7fa2f406b030 2023-10-18T12:00:04.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.859+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e4009490 con 0x7fa2f406b030 2023-10-18T12:00:04.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.859+0000 7fa2db7fe640 1 -- 172.21.15.125:0/3461703663 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4034850 con 0x7fa2f406b030 2023-10-18T12:00:04.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.859+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f40a8830 con 0x7fa2f406b030 2023-10-18T12:00:04.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.860+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f40690e0 con 0x7fa2f406b030 2023-10-18T12:00:04.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.860+0000 7fa2db7fe640 1 -- 172.21.15.125:0/3461703663 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fa2e40349b0 con 0x7fa2f406b030 2023-10-18T12:00:04.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.861+0000 7fa2db7fe640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa2e006b080 0x7fa2e006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:04.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.861+0000 7fa2f27fc640 1 -- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa2e006b080 msgr2=0x7fa2e006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:04.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.861+0000 7fa2db7fe640 1 -- 172.21.15.125:0/3461703663 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fa2e40ac360 con 0x7fa2f406b030 2023-10-18T12:00:04.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.861+0000 7fa2f27fc640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa2e006b080 0x7fa2e006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:04.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:04.865+0000 7fa2db7fe640 1 -- 172.21.15.125:0/3461703663 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2e4076d60 con 0x7fa2f406b030 2023-10-18T12:00:05.001 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:04.999+0000 7f63c423b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:00:05.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.050+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa2f40cc460 con 0x7fa2f406b030 2023-10-18T12:00:05.053 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.050+0000 7f7fdac32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:00:05.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.050+0000 7fa2db7fe640 1 -- 172.21.15.125:0/3461703663 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v207) v1 ==== 80+0+473 (secure 0 0 0) 0x7fa2e407a380 con 0x7fa2f406b030 2023-10-18T12:00:05.054 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/870215635 2023-10-19T11:58:51.730016+0000 2023-10-18T12:00:05.054 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3819578023 2023-10-19T11:58:51.730016+0000 2023-10-18T12:00:05.054 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/330535304 2023-10-19T11:58:51.730016+0000 2023-10-18T12:00:05.054 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2863286615 2023-10-19T11:58:51.730016+0000 2023-10-18T12:00:05.055 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/1565260611 2023-10-19T11:58:51.730016+0000 2023-10-18T12:00:05.055 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2575337659 2023-10-19T11:58:51.730016+0000 2023-10-18T12:00:05.055 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/1565260611 2023-10-19T11:58:51.730016+0000 2023-10-18T12:00:05.055 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1682437 2023-10-19T11:58:51.730016+0000 2023-10-18T12:00:05.056 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T12:00:05.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa2e006b080 msgr2=0x7fa2e006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:05.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa2e006b080 0x7fa2e006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f406b030 msgr2=0x7fa2f415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:05.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f406b030 0x7fa2f415ab40 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c00 tx=0x7fa2e4003820 comp rx=0 tx=0).stop 2023-10-18T12:00:05.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 shutdown_connections 2023-10-18T12:00:05.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa2e006b080 0x7fa2e006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2f4159bd0 0x7fa2f40a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2f40c9750 0x7fa2f40a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 --2- 172.21.15.125:0/3461703663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2f406b030 0x7fa2f415ab40 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 >> 172.21.15.125:0/3461703663 conn(0x7fa2f4069930 msgr2=0x7fa2f415bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:05.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 shutdown_connections 2023-10-18T12:00:05.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.052+0000 7fa2f92a9640 1 -- 172.21.15.125:0/3461703663 wait complete. 2023-10-18T12:00:05.066 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T12:00:05.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:05.067 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T12:00:05.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[150913]: fuse finished with error 0 and tester_r 0 2023-10-18T12:00:05.093 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:00:05.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:05.094 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T12:00:05.131 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:05.128+0000 7f63c423b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:00:05.190 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.187+0000 7f7fdac32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:00:05.221 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:05.219+0000 7f63c423b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:00:05.259 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:00:05.260 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:05.263 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:05.261+0000 7f63c423b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:00:05.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:00:05.276 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:00:05.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:05.276 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T12:00:05.283 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.281+0000 7f7fdac32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:00:05.325 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.323+0000 7f7fdac32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:00:05.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:00:05.330 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T12:00:05.331 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:00:05.331 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:05.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:00:05.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:00:05.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:05.347 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T12:00:05.362 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:05.360+0000 7f63c423b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:05.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:00:05.402 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T12:00:05.403 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:00:05.403 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:05.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:00:05.421 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:00:05.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:05.422 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T12:00:05.425 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.423+0000 7f7fdac32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:05.456 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:05.454+0000 7f63c423b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:00:05.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:00:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T12:00:05.472 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:05.519 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.517+0000 7f7fdac32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:00:05.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.574+0000 7fa714a4b640 1 -- 172.21.15.125:0/3650910147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6fc003970 msgr2=0x7fa6fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:05.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.574+0000 7fa714a4b640 1 --2- 172.21.15.125:0/3650910147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6fc003970 0x7fa6fc003d90 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7fa700009b50 tx=0x7fa6fc0057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.574+0000 7fa70ed9d640 1 -- 172.21.15.125:0/3650910147 reap_dead start 2023-10-18T12:00:05.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.574+0000 7fa714a4b640 1 -- 172.21.15.125:0/3650910147 shutdown_connections 2023-10-18T12:00:05.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.574+0000 7fa714a4b640 1 -- 172.21.15.125:0/3650910147 >> 172.21.15.125:0/3650910147 conn(0x7fa7100d0d60 msgr2=0x7fa7100d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:05.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.574+0000 7fa714a4b640 1 -- 172.21.15.125:0/3650910147 shutdown_connections 2023-10-18T12:00:05.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.574+0000 7fa714a4b640 1 -- 172.21.15.125:0/3650910147 wait complete. 2023-10-18T12:00:05.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.576+0000 7fa714a4b640 1 Processor -- start 2023-10-18T12:00:05.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.576+0000 7fa714a4b640 1 -- start start 2023-10-18T12:00:05.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.576+0000 7fa714a4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc003970 0x7fa7100b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:05.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.576+0000 7fa714a4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7100b4f50 0x7fa710149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:05.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.576+0000 7fa714a4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7101e3ba0 0x7fa710069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:05.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.576+0000 7fa714a4b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100b6060 con 0x7fa7101e3ba0 2023-10-18T12:00:05.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.576+0000 7fa714a4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100b53e0 con 0x7fa7100b4f50 2023-10-18T12:00:05.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.576+0000 7fa714a4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100b56c0 con 0x7fa6fc003970 2023-10-18T12:00:05.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7101e3ba0 0x7fa710069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:05.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7101e3ba0 0x7fa710069f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39992/0 (socket says 172.21.15.125:39992) 2023-10-18T12:00:05.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 -- 172.21.15.125:0/3521473474 learned_addr learned my addr 172.21.15.125:0/3521473474 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:05.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70e59c640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc003970 0x7fa7100b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:05.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa707fff640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7100b4f50 0x7fa710149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:05.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 -- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc003970 msgr2=0x7fa7100b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:05.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc003970 0x7fa7100b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 -- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7100b4f50 msgr2=0x7fa710149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:05.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7100b4f50 0x7fa710149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 -- 172.21.15.125:0/3521473474 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7100b59a0 con 0x7fa7101e3ba0 2023-10-18T12:00:05.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa70ed9d640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7101e3ba0 0x7fa710069f90 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fa7100b6e10 tx=0x7fa708006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:05.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa706ffd640 1 -- 172.21.15.125:0/3521473474 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70800fa80 con 0x7fa7101e3ba0 2023-10-18T12:00:05.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa706ffd640 1 -- 172.21.15.125:0/3521473474 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa70800fbe0 con 0x7fa7101e3ba0 2023-10-18T12:00:05.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa706ffd640 1 -- 172.21.15.125:0/3521473474 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080069a0 con 0x7fa7101e3ba0 2023-10-18T12:00:05.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7100b5c80 con 0x7fa7101e3ba0 2023-10-18T12:00:05.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.577+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101e4030 con 0x7fa7101e3ba0 2023-10-18T12:00:05.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.578+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7100b61a0 con 0x7fa7101e3ba0 2023-10-18T12:00:05.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.579+0000 7fa706ffd640 1 -- 172.21.15.125:0/3521473474 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fa70801d460 con 0x7fa7101e3ba0 2023-10-18T12:00:05.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.579+0000 7fa706ffd640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa6f006b080 0x7fa6f006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:05.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.579+0000 7fa70e59c640 1 -- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa6f006b080 msgr2=0x7fa6f006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:05.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.579+0000 7fa706ffd640 1 -- 172.21.15.125:0/3521473474 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fa70808a760 con 0x7fa7101e3ba0 2023-10-18T12:00:05.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.579+0000 7fa70e59c640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa6f006b080 0x7fa6f006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:05.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.583+0000 7fa706ffd640 1 -- 172.21.15.125:0/3521473474 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7100b61a0 con 0x7fa7101e3ba0 2023-10-18T12:00:05.596 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:05.594+0000 7f63c423b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:05.638 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:05.635+0000 7f63c423b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:05.663 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.660+0000 7f7fdac32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:05.706 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.703+0000 7f7fdac32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:05.741 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:05.738+0000 7f63c423b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:00:05.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.780+0000 7fa70e59c640 1 -- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa6f006b080 msgr2=0x7fa6f006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:05.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.780+0000 7fa70e59c640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa6f006b080 0x7fa6f006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:00:05.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.791+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7100b61a0 con 0x7fa7101e3ba0 2023-10-18T12:00:05.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.792+0000 7fa706ffd640 1 -- 172.21.15.125:0/3521473474 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fa7100b61a0 con 0x7fa7101e3ba0 2023-10-18T12:00:05.795 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:05.795 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":80,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2853562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2853562772},{"type":"v1","addr":"172.21.15.139:6836","nonce":2853562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":11421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3995754374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3995754374},{"type":"v1","addr":"172.21.15.125:6834","nonce":3995754374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":11424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2110336778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2110336778},{"type":"v1","addr":"172.21.15.125:6835","nonce":2110336778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:58:35.848069+0000","modified":"2023-10-18T11:58:37.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11439":{"gid":11439,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/1491435386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1491435386},{"type":"v1","addr":"172.21.15.139:6837","nonce":1491435386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-18T12:00:05.796 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 80 2023-10-18T12:00:05.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.793+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa6f006b080 msgr2=0x7fa6f006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:05.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.793+0000 7fa714a4b640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa6f006b080 0x7fa6f006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.793+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7101e3ba0 msgr2=0x7fa710069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:05.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.793+0000 7fa714a4b640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7101e3ba0 0x7fa710069f90 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fa7100b6e10 tx=0x7fa708006660 comp rx=0 tx=0).stop 2023-10-18T12:00:05.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.794+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 shutdown_connections 2023-10-18T12:00:05.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.794+0000 7fa714a4b640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7fa6f006b080 0x7fa6f006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.794+0000 7fa714a4b640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7100b4f50 0x7fa710149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.794+0000 7fa714a4b640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7101e3ba0 0x7fa710069f90 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.794+0000 7fa714a4b640 1 --2- 172.21.15.125:0/3521473474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc003970 0x7fa7100b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.794+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 >> 172.21.15.125:0/3521473474 conn(0x7fa7100d0d60 msgr2=0x7fa71006b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:05.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.794+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 shutdown_connections 2023-10-18T12:00:05.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.794+0000 7fa714a4b640 1 -- 172.21.15.125:0/3521473474 wait complete. 2023-10-18T12:00:05.807 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:05.815 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:05.812+0000 7f7fdac32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:00:05.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.909+0000 7efeb2e51640 1 -- 172.21.15.125:0/2125518376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 msgr2=0x7efe9c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:05.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.909+0000 7efeb2e51640 1 --2- 172.21.15.125:0/2125518376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 0x7efe9c003d90 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7efea0009b50 tx=0x7efe9c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.909+0000 7efeb2e51640 1 -- 172.21.15.125:0/2125518376 shutdown_connections 2023-10-18T12:00:05.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.909+0000 7efeb2e51640 1 --2- 172.21.15.125:0/2125518376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 0x7efe9c003d90 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.909+0000 7efeb2e51640 1 -- 172.21.15.125:0/2125518376 >> 172.21.15.125:0/2125518376 conn(0x7efeac0d0d60 msgr2=0x7efeac0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:05.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.909+0000 7efeb2e51640 1 -- 172.21.15.125:0/2125518376 shutdown_connections 2023-10-18T12:00:05.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.909+0000 7efeb2e51640 1 -- 172.21.15.125:0/2125518376 wait complete. 2023-10-18T12:00:05.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.910+0000 7efeb2e51640 1 Processor -- start 2023-10-18T12:00:05.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.910+0000 7efeb2e51640 1 -- start start 2023-10-18T12:00:05.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb2e51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 0x7efeac0b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:05.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb2e51640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efeac14f460 0x7efeac14e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:05.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb2e51640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeac0ca540 0x7efeac0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:05.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb2e51640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0cb220 con 0x7efe9c003970 2023-10-18T12:00:05.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb2e51640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0b41b0 con 0x7efeac0ca540 2023-10-18T12:00:05.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb2e51640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0caf40 con 0x7efeac14f460 2023-10-18T12:00:05.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 0x7efeac0b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:05.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 0x7efeac0b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40010/0 (socket says 172.21.15.125:40010) 2023-10-18T12:00:05.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 -- 172.21.15.125:0/2184603415 learned_addr learned my addr 172.21.15.125:0/2184603415 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:05.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeabfff640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efeac14f460 0x7efeac14e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb13ee640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeac0ca540 0x7efeac0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 -- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efeac14f460 msgr2=0x7efeac14e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efeac14f460 0x7efeac14e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 -- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeac0ca540 msgr2=0x7efeac0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeac0ca540 0x7efeac0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 -- 172.21.15.125:0/2184603415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac0b3af0 con 0x7efe9c003970 2023-10-18T12:00:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efeb0bed640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 0x7efeac0b45d0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7efea0009a70 tx=0x7efea0033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.911+0000 7efea97fa640 1 -- 172.21.15.125:0/2184603415 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0004220 con 0x7efe9c003970 2023-10-18T12:00:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.912+0000 7efea97fa640 1 -- 172.21.15.125:0/2184603415 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efea0004380 con 0x7efe9c003970 2023-10-18T12:00:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.912+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac0b3dd0 con 0x7efe9c003970 2023-10-18T12:00:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.912+0000 7efea97fa640 1 -- 172.21.15.125:0/2184603415 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea003d940 con 0x7efe9c003970 2023-10-18T12:00:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.912+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac0ca9f0 con 0x7efe9c003970 2023-10-18T12:00:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.913+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac0b3950 con 0x7efe9c003970 2023-10-18T12:00:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.913+0000 7efea97fa640 1 -- 172.21.15.125:0/2184603415 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7efea003daa0 con 0x7efe9c003970 2023-10-18T12:00:05.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.913+0000 7efea97fa640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7efe9006b0b0 0x7efe9006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:05.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.914+0000 7efea97fa640 1 -- 172.21.15.125:0/2184603415 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7efea00ac950 con 0x7efe9c003970 2023-10-18T12:00:05.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.914+0000 7efeabfff640 1 -- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7efe9006b0b0 msgr2=0x7efe9006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:05.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.914+0000 7efeabfff640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7efe9006b0b0 0x7efe9006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:05.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:05.918+0000 7efea97fa640 1 -- 172.21.15.125:0/2184603415 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efeac0b3950 con 0x7efe9c003970 2023-10-18T12:00:06.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.091+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeac0b3950 con 0x7efe9c003970 2023-10-18T12:00:06.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.092+0000 7efea97fa640 1 -- 172.21.15.125:0/2184603415 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+17658 (secure 0 0 0) 0x7efeac0b3950 con 0x7efe9c003970 2023-10-18T12:00:06.095 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:06.096 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":207,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T11:59:53.406049+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:58:34.060453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-18T11:58:34.823261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.13","mappings":[{"from":1,"to":6}]},{"pgid":"26.15","mappings":[{"from":3,"to":6}]},{"pgid":"26.19","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/870215635":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/3819578023":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/330535304":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2863286615":"2023-10-19T11:58:51.730016+0000","172.21.15.139:6833/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2575337659":"2023-10-19T11:58:51.730016+0000","172.21.15.139:6832/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/1682437":"2023-10-19T11:58:51.730016+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:06.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7efe9006b0b0 msgr2=0x7efe9006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:06.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7efe9006b0b0 0x7efe9006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 msgr2=0x7efeac0b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 0x7efeac0b45d0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7efea0009a70 tx=0x7efea0033c90 comp rx=0 tx=0).stop 2023-10-18T12:00:06.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 shutdown_connections 2023-10-18T12:00:06.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7efe9006b0b0 0x7efe9006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efeac0ca540 0x7efeac0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efeac14f460 0x7efeac14e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 --2- 172.21.15.125:0/2184603415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe9c003970 0x7efeac0b45d0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 >> 172.21.15.125:0/2184603415 conn(0x7efeac0d0d60 msgr2=0x7efeac0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:06.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 shutdown_connections 2023-10-18T12:00:06.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.094+0000 7efeb2e51640 1 -- 172.21.15.125:0/2184603415 wait complete. 2023-10-18T12:00:06.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T11:58:34.060453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '203', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T11:58:34.823261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '204', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:00:06.109 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:00:06.258 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.255+0000 7f1a6398e640 1 -- 172.21.15.125:0/1945258089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a48003b50 msgr2=0x7f1a48003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.255+0000 7f1a6398e640 1 --2- 172.21.15.125:0/1945258089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a48003b50 0x7f1a48003f50 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f1a58006ec0 tx=0x7f1a480057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.255+0000 7f1a6398e640 1 -- 172.21.15.125:0/1945258089 shutdown_connections 2023-10-18T12:00:06.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.255+0000 7f1a6398e640 1 --2- 172.21.15.125:0/1945258089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a48003b50 0x7f1a48003f50 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.255+0000 7f1a6398e640 1 -- 172.21.15.125:0/1945258089 >> 172.21.15.125:0/1945258089 conn(0x7f1a5c0d0d60 msgr2=0x7f1a5c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:06.259 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.255+0000 7f1a6398e640 1 -- 172.21.15.125:0/1945258089 shutdown_connections 2023-10-18T12:00:06.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.255+0000 7f1a6398e640 1 -- 172.21.15.125:0/1945258089 wait complete. 2023-10-18T12:00:06.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.257+0000 7f1a6398e640 1 Processor -- start 2023-10-18T12:00:06.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.257+0000 7f1a6398e640 1 -- start start 2023-10-18T12:00:06.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a6398e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a48003b50 0x7f1a5c09e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a6398e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a5c1592e0 0x7f1a5c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a6398e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a5c0ca4c0 0x7f1a5c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a6398e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0caeb0 con 0x7f1a5c1592e0 2023-10-18T12:00:06.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a6398e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c14b560 con 0x7f1a5c0ca4c0 2023-10-18T12:00:06.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a6398e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c15c9e0 con 0x7f1a48003b50 2023-10-18T12:00:06.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a60f29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a5c1592e0 0x7f1a5c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a60f29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a5c1592e0 0x7f1a5c15c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40024/0 (socket says 172.21.15.125:40024) 2023-10-18T12:00:06.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a60f29640 1 -- 172.21.15.125:0/1441958575 learned_addr learned my addr 172.21.15.125:0/1441958575 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:06.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a61f2b640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a5c0ca4c0 0x7f1a5c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.258+0000 7f1a6172a640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a48003b50 0x7f1a5c09e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a60f29640 1 -- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a48003b50 msgr2=0x7f1a5c09e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a60f29640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a48003b50 0x7f1a5c09e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a60f29640 1 -- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a5c0ca4c0 msgr2=0x7f1a5c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a60f29640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a5c0ca4c0 0x7f1a5c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a60f29640 1 -- 172.21.15.125:0/1441958575 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c15ccc0 con 0x7f1a5c1592e0 2023-10-18T12:00:06.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a60f29640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a5c1592e0 0x7f1a5c15c090 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c011ab0 tx=0x7f1a4c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:06.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a51ffb640 1 -- 172.21.15.125:0/1441958575 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00fbf0 con 0x7f1a5c1592e0 2023-10-18T12:00:06.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a51ffb640 1 -- 172.21.15.125:0/1441958575 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1a4c00fd50 con 0x7f1a5c1592e0 2023-10-18T12:00:06.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c15cfa0 con 0x7f1a5c1592e0 2023-10-18T12:00:06.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a51ffb640 1 -- 172.21.15.125:0/1441958575 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c019670 con 0x7f1a5c1592e0 2023-10-18T12:00:06.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.259+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c09e9c0 con 0x7f1a5c1592e0 2023-10-18T12:00:06.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.260+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c14b710 con 0x7f1a5c1592e0 2023-10-18T12:00:06.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.261+0000 7f1a51ffb640 1 -- 172.21.15.125:0/1441958575 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f1a4c0197d0 con 0x7f1a5c1592e0 2023-10-18T12:00:06.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.261+0000 7f1a51ffb640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f1a4006b080 0x7f1a4006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.261+0000 7f1a6172a640 1 -- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f1a4006b080 msgr2=0x7f1a4006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:06.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.261+0000 7f1a51ffb640 1 -- 172.21.15.125:0/1441958575 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f1a4c08a9a0 con 0x7f1a5c1592e0 2023-10-18T12:00:06.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.261+0000 7f1a6172a640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f1a4006b080 0x7f1a4006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:06.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.266+0000 7f1a51ffb640 1 -- 172.21.15.125:0/1441958575 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a5c14b710 con 0x7f1a5c1592e0 2023-10-18T12:00:06.355 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.353+0000 7f63c423b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:00:06.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.441+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a5c14b6d0 con 0x7f1a5c1592e0 2023-10-18T12:00:06.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.442+0000 7f1a51ffb640 1 -- 172.21.15.125:0/1441958575 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v207) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a4c055330 con 0x7f1a5c1592e0 2023-10-18T12:00:06.445 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:00:06.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f1a4006b080 msgr2=0x7f1a4006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:06.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f1a4006b080 0x7f1a4006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a5c1592e0 msgr2=0x7f1a5c15c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a5c1592e0 0x7f1a5c15c090 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c011ab0 tx=0x7f1a4c011f70 comp rx=0 tx=0).stop 2023-10-18T12:00:06.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 shutdown_connections 2023-10-18T12:00:06.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f1a4006b080 0x7f1a4006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a5c0ca4c0 0x7f1a5c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a5c1592e0 0x7f1a5c15c090 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 --2- 172.21.15.125:0/1441958575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a48003b50 0x7f1a5c09e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 >> 172.21.15.125:0/1441958575 conn(0x7f1a5c0d0d60 msgr2=0x7f1a5c06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:06.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 shutdown_connections 2023-10-18T12:00:06.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.444+0000 7f1a6398e640 1 -- 172.21.15.125:0/1441958575 wait complete. 2023-10-18T12:00:06.457 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:00:06.474 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:06.471+0000 7f7fdac32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:00:06.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.560+0000 7f628993c640 1 -- 172.21.15.125:0/2874428492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62841579f0 msgr2=0x7f62840b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.560+0000 7f628993c640 1 --2- 172.21.15.125:0/2874428492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62841579f0 0x7f62840b0e90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6274009830 tx=0x7f627402bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:06.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.560+0000 7f628993c640 1 -- 172.21.15.125:0/2874428492 shutdown_connections 2023-10-18T12:00:06.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.560+0000 7f628993c640 1 --2- 172.21.15.125:0/2874428492 >> v2:172.21.15.139:3300/0 conn(0x7f628406ae40 0x7f6284157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.560+0000 7f628993c640 1 --2- 172.21.15.125:0/2874428492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62841579f0 0x7f62840b0e90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.560+0000 7f628993c640 1 -- 172.21.15.125:0/2874428492 >> 172.21.15.125:0/2874428492 conn(0x7f628406fae0 msgr2=0x7f628406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:06.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.561+0000 7f628993c640 1 -- 172.21.15.125:0/2874428492 shutdown_connections 2023-10-18T12:00:06.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.561+0000 7f628993c640 1 -- 172.21.15.125:0/2874428492 wait complete. 2023-10-18T12:00:06.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.562+0000 7f628993c640 1 Processor -- start 2023-10-18T12:00:06.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.562+0000 7f628993c640 1 -- start start 2023-10-18T12:00:06.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.562+0000 7f628993c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628406ae40 0x7f62841ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.562+0000 7f628993c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62841579f0 0x7f628414fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.562+0000 7f628993c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f628415c1f0 0x7f6284141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f628993c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284156f30 con 0x7f62841579f0 2023-10-18T12:00:06.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f628993c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62841ef7f0 con 0x7f628406ae40 2023-10-18T12:00:06.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f628993c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62841efad0 con 0x7f628415c1f0 2023-10-18T12:00:06.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62827fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62841579f0 0x7f628414fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62827fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62841579f0 0x7f628414fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40040/0 (socket says 172.21.15.125:40040) 2023-10-18T12:00:06.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62827fc640 1 -- 172.21.15.125:0/3869383726 learned_addr learned my addr 172.21.15.125:0/3869383726 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:06.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f6282ffd640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628406ae40 0x7f62841ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62837fe640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f628415c1f0 0x7f6284141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62837fe640 1 -- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628406ae40 msgr2=0x7f62841ef1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62837fe640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628406ae40 0x7f62841ef1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62837fe640 1 -- 172.21.15.125:0/3869383726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62841579f0 msgr2=0x7f628414fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62837fe640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62841579f0 0x7f628414fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62837fe640 1 -- 172.21.15.125:0/3869383726 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009490 con 0x7f628415c1f0 2023-10-18T12:00:06.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62827fc640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62841579f0 0x7f628414fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:06.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f62837fe640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f628415c1f0 0x7f6284141c70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f62840b0180 tx=0x7f627c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:06.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f626bfff640 1 -- 172.21.15.125:0/3869383726 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00cdf0 con 0x7f628415c1f0 2023-10-18T12:00:06.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f626bfff640 1 -- 172.21.15.125:0/3869383726 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f627c004c10 con 0x7f628415c1f0 2023-10-18T12:00:06.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62840b2330 con 0x7f628415c1f0 2023-10-18T12:00:06.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f626bfff640 1 -- 172.21.15.125:0/3869383726 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c005020 con 0x7f628415c1f0 2023-10-18T12:00:06.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.563+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62841f0090 con 0x7f628415c1f0 2023-10-18T12:00:06.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.564+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62840665e0 con 0x7f628415c1f0 2023-10-18T12:00:06.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.565+0000 7f626bfff640 1 -- 172.21.15.125:0/3869383726 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 126) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f627c0051a0 con 0x7f628415c1f0 2023-10-18T12:00:06.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.565+0000 7f626bfff640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f627006b0b0 0x7f627006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.565+0000 7f6282ffd640 1 -- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f627006b0b0 msgr2=0x7f627006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3820883843 2023-10-18T12:00:06.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.565+0000 7f626bfff640 1 -- 172.21.15.125:0/3869383726 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f627c014070 con 0x7f628415c1f0 2023-10-18T12:00:06.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.565+0000 7f6282ffd640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f627006b0b0 0x7f627006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:06.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.570+0000 7f626bfff640 1 -- 172.21.15.125:0/3869383726 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f627c055590 con 0x7f628415c1f0 2023-10-18T12:00:06.618 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.615+0000 7f63c423b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:00:06.666 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.663+0000 7f63c423b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:00:06.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.736+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62840b0b80 con 0x7f628415c1f0 2023-10-18T12:00:06.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.737+0000 7f626bfff640 1 -- 172.21.15.125:0/3869383726 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v207) v1 ==== 104+0+11 (secure 0 0 0) 0x7f627c058bb0 con 0x7f628415c1f0 2023-10-18T12:00:06.740 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:00:06.740 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:06.737+0000 7f7fdac32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:00:06.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f627006b0b0 msgr2=0x7f627006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:06.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f627006b0b0 0x7f627006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f628415c1f0 msgr2=0x7f6284141c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f628415c1f0 0x7f6284141c70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f62840b0180 tx=0x7f627c00ef10 comp rx=0 tx=0).stop 2023-10-18T12:00:06.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 shutdown_connections 2023-10-18T12:00:06.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:6832/3820883843,v1:172.21.15.139:6833/3820883843] conn(0x7f627006b0b0 0x7f627006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f628415c1f0 0x7f6284141c70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f62841579f0 0x7f628414fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 --2- 172.21.15.125:0/3869383726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f628406ae40 0x7f62841ef1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 >> 172.21.15.125:0/3869383726 conn(0x7f628406fae0 msgr2=0x7f6284152be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:06.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 shutdown_connections 2023-10-18T12:00:06.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.739+0000 7f628993c640 1 -- 172.21.15.125:0/3869383726 wait complete. 2023-10-18T12:00:06.752 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:00:06.766 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.763+0000 7f63c423b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:00:06.796 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:06.794+0000 7f7fdac32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:00:06.849 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6346573640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.850 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6346573640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.850 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6346573640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.850 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6346573640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.850 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6346573640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.851 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6349579640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.851 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6349579640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.851 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6349579640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.851 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6349579640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.851 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:06.846+0000 7f6349579640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:06.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.898+0000 7fed30925640 1 -- 172.21.15.125:0/316972166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c0cf340 msgr2=0x7fed2c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.898+0000 7fed30925640 1 --2- 172.21.15.125:0/316972166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c0cf340 0x7fed2c0d1910 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7fed2c15b680 tx=0x7fed1402bdb0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.898+0000 7fed30925640 1 -- 172.21.15.125:0/316972166 shutdown_connections 2023-10-18T12:00:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.898+0000 7fed30925640 1 --2- 172.21.15.125:0/316972166 >> v2:172.21.15.139:3300/0 conn(0x7fed2c0ceb40 0x7fed2c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.898+0000 7fed30925640 1 --2- 172.21.15.125:0/316972166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c0cf340 0x7fed2c0d1910 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.898+0000 7fed30925640 1 -- 172.21.15.125:0/316972166 >> 172.21.15.125:0/316972166 conn(0x7fed2c0d0d60 msgr2=0x7fed2c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:06.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.898+0000 7fed30925640 1 -- 172.21.15.125:0/316972166 shutdown_connections 2023-10-18T12:00:06.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.898+0000 7fed30925640 1 -- 172.21.15.125:0/316972166 wait complete. 2023-10-18T12:00:06.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.899+0000 7fed30925640 1 Processor -- start 2023-10-18T12:00:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed30925640 1 -- start start 2023-10-18T12:00:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed30925640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed2c0ceb40 0x7fed2c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed30925640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed2c0cf340 0x7fed2c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed30925640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c15b2a0 0x7fed2c0b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed30925640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c15e630 con 0x7fed2c15b2a0 2023-10-18T12:00:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed30925640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c15c4b0 con 0x7fed2c0cf340 2023-10-18T12:00:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed30925640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c15c790 con 0x7fed2c0ceb40 2023-10-18T12:00:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed2ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c15b2a0 0x7fed2c0b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed2ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c15b2a0 0x7fed2c0b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40060/0 (socket says 172.21.15.125:40060) 2023-10-18T12:00:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed2ad9d640 1 -- 172.21.15.125:0/227798444 learned_addr learned my addr 172.21.15.125:0/227798444 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.900+0000 7fed29d9b640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed2c0cf340 0x7fed2c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed2a59c640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed2c0ceb40 0x7fed2c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:06.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed2ad9d640 1 -- 172.21.15.125:0/227798444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed2c0ceb40 msgr2=0x7fed2c15e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed2ad9d640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed2c0ceb40 0x7fed2c15e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed2ad9d640 1 -- 172.21.15.125:0/227798444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed2c0cf340 msgr2=0x7fed2c0b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed2ad9d640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed2c0cf340 0x7fed2c0b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:06.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed2ad9d640 1 -- 172.21.15.125:0/227798444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed14009560 con 0x7fed2c15b2a0 2023-10-18T12:00:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed2ad9d640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c15b2a0 0x7fed2c0b5600 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fed2400eb10 tx=0x7fed2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed22ffd640 1 -- 172.21.15.125:0/227798444 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24004b20 con 0x7fed2c15b2a0 2023-10-18T12:00:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed22ffd640 1 -- 172.21.15.125:0/227798444 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed24004ca0 con 0x7fed2c15b2a0 2023-10-18T12:00:06.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c15ca70 con 0x7fed2c15b2a0 2023-10-18T12:00:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed22ffd640 1 -- 172.21.15.125:0/227798444 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24005280 con 0x7fed2c15b2a0 2023-10-18T12:00:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.901+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c15d030 con 0x7fed2c15b2a0 2023-10-18T12:00:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.902+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2c1f1500 con 0x7fed2c15b2a0 2023-10-18T12:00:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.902+0000 7fed22ffd640 1 -- 172.21.15.125:0/227798444 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 127) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fed240053e0 con 0x7fed2c15b2a0 2023-10-18T12:00:06.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.903+0000 7fed22ffd640 1 -- 172.21.15.125:0/227798444 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fed24014070 con 0x7fed2c15b2a0 2023-10-18T12:00:06.912 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:06.903+0000 7f7fdac32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:00:06.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:06.907+0000 7fed22ffd640 1 -- 172.21.15.125:0/227798444 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed2c1f1500 con 0x7fed2c15b2a0 2023-10-18T12:00:07.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.076+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed2c1f1500 con 0x7fed2c15b2a0 2023-10-18T12:00:07.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.076+0000 7fed22ffd640 1 -- 172.21.15.125:0/227798444 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v208) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed2c1f1500 con 0x7fed2c15b2a0 2023-10-18T12:00:07.079 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:00:07.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c15b2a0 msgr2=0x7fed2c0b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c15b2a0 0x7fed2c0b5600 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fed2400eb10 tx=0x7fed2400c2d0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 shutdown_connections 2023-10-18T12:00:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed2c0cf340 0x7fed2c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fed2c15b2a0 0x7fed2c0b5600 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 --2- 172.21.15.125:0/227798444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed2c0ceb40 0x7fed2c15e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 >> 172.21.15.125:0/227798444 conn(0x7fed2c0d0d60 msgr2=0x7fed2c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:07.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 shutdown_connections 2023-10-18T12:00:07.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.078+0000 7fed30925640 1 -- 172.21.15.125:0/227798444 wait complete. 2023-10-18T12:00:07.095 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:07.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.235+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/1724314126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7bc003970 msgr2=0x7fa7bc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.235+0000 7fa7d7aa2640 1 --2- 172.21.15.125:0/1724314126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7bc003970 0x7fa7bc003d90 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009d50 tx=0x7fa7bc0057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.235+0000 7fa7d603f640 1 -- 172.21.15.125:0/1724314126 reap_dead start 2023-10-18T12:00:07.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.235+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/1724314126 shutdown_connections 2023-10-18T12:00:07.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.235+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/1724314126 >> 172.21.15.125:0/1724314126 conn(0x7fa7d00d0d60 msgr2=0x7fa7d00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:07.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.235+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/1724314126 shutdown_connections 2023-10-18T12:00:07.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.235+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/1724314126 wait complete. 2023-10-18T12:00:07.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.236+0000 7fa7d7aa2640 1 Processor -- start 2023-10-18T12:00:07.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.236+0000 7fa7d7aa2640 1 -- start start 2023-10-18T12:00:07.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d7aa2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7bc003970 0x7fa7d015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d7aa2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7d00c8060 0x7fa7d00c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d7aa2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7d015b6d0 0x7fa7d015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d7aa2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c8dd0 con 0x7fa7d015b6d0 2023-10-18T12:00:07.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d7aa2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c90b0 con 0x7fa7bc003970 2023-10-18T12:00:07.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d7aa2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c9390 con 0x7fa7d00c8060 2023-10-18T12:00:07.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7d015b6d0 0x7fa7d015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7d015b6d0 0x7fa7d015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40074/0 (socket says 172.21.15.125:40074) 2023-10-18T12:00:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 -- 172.21.15.125:0/3167240217 learned_addr learned my addr 172.21.15.125:0/3167240217 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d583e640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7bc003970 0x7fa7d015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d503d640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7d00c8060 0x7fa7d00c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 -- 172.21.15.125:0/3167240217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7d00c8060 msgr2=0x7fa7d00c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7d00c8060 0x7fa7d00c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 -- 172.21.15.125:0/3167240217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7bc003970 msgr2=0x7fa7d015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7bc003970 0x7fa7d015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 -- 172.21.15.125:0/3167240217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00c9670 con 0x7fa7d015b6d0 2023-10-18T12:00:07.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d603f640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7d015b6d0 0x7fa7d015bb30 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00b6e10 tx=0x7fa7c800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:07.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7c67fc640 1 -- 172.21.15.125:0/3167240217 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8004b20 con 0x7fa7d015b6d0 2023-10-18T12:00:07.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7c67fc640 1 -- 172.21.15.125:0/3167240217 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa7c8004c80 con 0x7fa7d015b6d0 2023-10-18T12:00:07.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d00c8a20 con 0x7fa7d015b6d0 2023-10-18T12:00:07.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.237+0000 7fa7c67fc640 1 -- 172.21.15.125:0/3167240217 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8003bd0 con 0x7fa7d015b6d0 2023-10-18T12:00:07.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.238+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d015dbc0 con 0x7fa7d015b6d0 2023-10-18T12:00:07.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.239+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d00690e0 con 0x7fa7d015b6d0 2023-10-18T12:00:07.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.239+0000 7fa7c67fc640 1 -- 172.21.15.125:0/3167240217 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 127) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa7c8003d30 con 0x7fa7d015b6d0 2023-10-18T12:00:07.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.239+0000 7fa7c67fc640 1 -- 172.21.15.125:0/3167240217 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fa7c8089ce0 con 0x7fa7d015b6d0 2023-10-18T12:00:07.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.244+0000 7fa7c67fc640 1 -- 172.21.15.125:0/3167240217 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7c8054d40 con 0x7fa7d015b6d0 2023-10-18T12:00:07.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.446+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7d015dbc0 con 0x7fa7d015b6d0 2023-10-18T12:00:07.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.446+0000 7fa7c67fc640 1 -- 172.21.15.125:0/3167240217 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fa7d015dbc0 con 0x7fa7d015b6d0 2023-10-18T12:00:07.449 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:07.450 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":80,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2853562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2853562772},{"type":"v1","addr":"172.21.15.139:6836","nonce":2853562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":11421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3995754374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3995754374},{"type":"v1","addr":"172.21.15.125:6834","nonce":3995754374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":11424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2110336778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2110336778},{"type":"v1","addr":"172.21.15.125:6835","nonce":2110336778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:58:35.848069+0000","modified":"2023-10-18T11:58:37.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11439":{"gid":11439,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/1491435386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1491435386},{"type":"v1","addr":"172.21.15.139:6837","nonce":1491435386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-18T12:00:07.450 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 80 2023-10-18T12:00:07.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7d015b6d0 msgr2=0x7fa7d015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7d015b6d0 0x7fa7d015bb30 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00b6e10 tx=0x7fa7c800f3c0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 shutdown_connections 2023-10-18T12:00:07.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7d015b6d0 0x7fa7d015bb30 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7d00c8060 0x7fa7d00c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 --2- 172.21.15.125:0/3167240217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7bc003970 0x7fa7d015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 >> 172.21.15.125:0/3167240217 conn(0x7fa7d00d0d60 msgr2=0x7fa7d00b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:07.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 shutdown_connections 2023-10-18T12:00:07.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.448+0000 7fa7d7aa2640 1 -- 172.21.15.125:0/3167240217 wait complete. 2023-10-18T12:00:07.461 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:07.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.561+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/1580011983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 msgr2=0x7fe6b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.561+0000 7fe6d5e0e640 1 --2- 172.21.15.125:0/1580011983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000bb50 tx=0x7fe6b80057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.561+0000 7fe6cffff640 1 -- 172.21.15.125:0/1580011983 reap_dead start 2023-10-18T12:00:07.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.561+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/1580011983 shutdown_connections 2023-10-18T12:00:07.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.561+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/1580011983 >> 172.21.15.125:0/1580011983 conn(0x7fe6d0069930 msgr2=0x7fe6d0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:07.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.561+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/1580011983 shutdown_connections 2023-10-18T12:00:07.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.561+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/1580011983 wait complete. 2023-10-18T12:00:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.562+0000 7fe6d5e0e640 1 Processor -- start 2023-10-18T12:00:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.562+0000 7fe6d5e0e640 1 -- start start 2023-10-18T12:00:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6d5e0e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 0x7fe6d0153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6d5e0e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6d00b1200 0x7fe6d0070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6d5e0e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d015ad80 0x7fe6d015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6d5e0e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d012d710 con 0x7fe6b8003970 2023-10-18T12:00:07.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6d5e0e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d015b740 con 0x7fe6d015ad80 2023-10-18T12:00:07.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6d5e0e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00b1980 con 0x7fe6d00b1200 2023-10-18T12:00:07.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6cf7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 0x7fe6d0153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6cf7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 0x7fe6d0153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35104/0 (socket says 172.21.15.125:35104) 2023-10-18T12:00:07.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6cf7fe640 1 -- 172.21.15.125:0/3621259732 learned_addr learned my addr 172.21.15.125:0/3621259732 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:07.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.563+0000 7fe6cffff640 1 --2- 172.21.15.125:0/3621259732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d015ad80 0x7fe6d015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6cf7fe640 1 -- 172.21.15.125:0/3621259732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6d00b1200 msgr2=0x7fe6d0070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6cf7fe640 1 --2- 172.21.15.125:0/3621259732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6d00b1200 0x7fe6d0070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6cf7fe640 1 -- 172.21.15.125:0/3621259732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d015ad80 msgr2=0x7fe6d015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6cf7fe640 1 --2- 172.21.15.125:0/3621259732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d015ad80 0x7fe6d015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6cf7fe640 1 -- 172.21.15.125:0/3621259732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d012cd70 con 0x7fe6b8003970 2023-10-18T12:00:07.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6cf7fe640 1 --2- 172.21.15.125:0/3621259732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 0x7fe6d0153e30 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000baf0 tx=0x7fe6c0036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:07.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6b7fff640 1 -- 172.21.15.125:0/3621259732 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0036de0 con 0x7fe6b8003970 2023-10-18T12:00:07.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6b7fff640 1 -- 172.21.15.125:0/3621259732 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe6c0004120 con 0x7fe6b8003970 2023-10-18T12:00:07.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6b7fff640 1 -- 172.21.15.125:0/3621259732 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00044f0 con 0x7fe6b8003970 2023-10-18T12:00:07.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d012d050 con 0x7fe6b8003970 2023-10-18T12:00:07.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.564+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d00b1680 con 0x7fe6b8003970 2023-10-18T12:00:07.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.565+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d0159c10 con 0x7fe6b8003970 2023-10-18T12:00:07.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.565+0000 7fe6b7fff640 1 -- 172.21.15.125:0/3621259732 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 127) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe6c00035f0 con 0x7fe6b8003970 2023-10-18T12:00:07.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.566+0000 7fe6b7fff640 1 -- 172.21.15.125:0/3621259732 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fe6c00ab440 con 0x7fe6b8003970 2023-10-18T12:00:07.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.570+0000 7fe6b7fff640 1 -- 172.21.15.125:0/3621259732 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6c0076300 con 0x7fe6b8003970 2023-10-18T12:00:07.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.739+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6d00690e0 con 0x7fe6b8003970 2023-10-18T12:00:07.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.740+0000 7fe6b7fff640 1 -- 172.21.15.125:0/3621259732 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+18405 (secure 0 0 0) 0x7fe6c0079920 con 0x7fe6b8003970 2023-10-18T12:00:07.743 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:07.743 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":208,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:06.777619+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:58:34.060453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-18T11:58:34.823261+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":208,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.13","mappings":[{"from":1,"to":6}]},{"pgid":"26.15","mappings":[{"from":3,"to":6}]},{"pgid":"26.19","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3621824170":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6833/3820883843":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6832/3820883843":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2142329879":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/693299469":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/3012481544":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2446385430":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/870215635":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/3819578023":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/330535304":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2863286615":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/1415408500":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2323488592":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6833/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2575337659":"2023-10-19T11:58:51.730016+0000","172.21.15.139:6832/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2354007263":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/1682437":"2023-10-19T11:58:51.730016+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:07.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 msgr2=0x7fe6d0153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 --2- 172.21.15.125:0/3621259732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 0x7fe6d0153e30 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000baf0 tx=0x7fe6c0036720 comp rx=0 tx=0).stop 2023-10-18T12:00:07.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 shutdown_connections 2023-10-18T12:00:07.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 --2- 172.21.15.125:0/3621259732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6d015ad80 0x7fe6d015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 --2- 172.21.15.125:0/3621259732 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6d00b1200 0x7fe6d0070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 --2- 172.21.15.125:0/3621259732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6b8003970 0x7fe6d0153e30 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 >> 172.21.15.125:0/3621259732 conn(0x7fe6d0069930 msgr2=0x7fe6d015b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:07.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 shutdown_connections 2023-10-18T12:00:07.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.742+0000 7fe6d5e0e640 1 -- 172.21.15.125:0/3621259732 wait complete. 2023-10-18T12:00:07.755 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T12:00:07.755 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:07.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.857+0000 7fd970c9b640 1 -- 172.21.15.125:0/2643311101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b3e0 msgr2=0x7fd96c0b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.857+0000 7fd970c9b640 1 --2- 172.21.15.125:0/2643311101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b3e0 0x7fd96c0b0c10 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fd964005fd0 tx=0x7fd96402be10 comp rx=0 tx=0).stop 2023-10-18T12:00:07.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.857+0000 7fd970c9b640 1 -- 172.21.15.125:0/2643311101 shutdown_connections 2023-10-18T12:00:07.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.857+0000 7fd970c9b640 1 --2- 172.21.15.125:0/2643311101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b3e0 0x7fd96c0b0c10 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.857+0000 7fd970c9b640 1 --2- 172.21.15.125:0/2643311101 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd96c0cf9e0 0x7fd96c0d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.857+0000 7fd970c9b640 1 -- 172.21.15.125:0/2643311101 >> 172.21.15.125:0/2643311101 conn(0x7fd96c0d0cf0 msgr2=0x7fd96c0d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:07.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.857+0000 7fd970c9b640 1 -- 172.21.15.125:0/2643311101 shutdown_connections 2023-10-18T12:00:07.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.857+0000 7fd970c9b640 1 -- 172.21.15.125:0/2643311101 wait complete. 2023-10-18T12:00:07.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.858+0000 7fd970c9b640 1 Processor -- start 2023-10-18T12:00:07.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.858+0000 7fd970c9b640 1 -- start start 2023-10-18T12:00:07.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd970c9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd96c0cf9e0 0x7fd96c157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd970c9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b000 0x7fd96c1e0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd970c9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd96c15b3e0 0x7fd96c14a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd970c9b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c1e2c30 con 0x7fd96c15b000 2023-10-18T12:00:07.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd970c9b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c1e3400 con 0x7fd96c0cf9e0 2023-10-18T12:00:07.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd970c9b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c1e52e0 con 0x7fd96c15b3e0 2023-10-18T12:00:07.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd969d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b000 0x7fd96c1e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd969d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b000 0x7fd96c1e0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35114/0 (socket says 172.21.15.125:35114) 2023-10-18T12:00:07.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd969d9b640 1 -- 172.21.15.125:0/1334880944 learned_addr learned my addr 172.21.15.125:0/1334880944 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:07.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd96a59c640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd96c0cf9e0 0x7fd96c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.859+0000 7fd96ad9d640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd96c15b3e0 0x7fd96c14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd969d9b640 1 -- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd96c15b3e0 msgr2=0x7fd96c14a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd969d9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd96c15b3e0 0x7fd96c14a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd969d9b640 1 -- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd96c0cf9e0 msgr2=0x7fd96c157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:07.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd969d9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd96c0cf9e0 0x7fd96c157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:07.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd969d9b640 1 -- 172.21.15.125:0/1334880944 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd964009560 con 0x7fd96c15b000 2023-10-18T12:00:07.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd969d9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b000 0x7fd96c1e0500 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7fd96000d980 tx=0x7fd96000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:07.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd95affd640 1 -- 172.21.15.125:0/1334880944 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960009bd0 con 0x7fd96c15b000 2023-10-18T12:00:07.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd95affd640 1 -- 172.21.15.125:0/1334880944 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd960009e20 con 0x7fd96c15b000 2023-10-18T12:00:07.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd95affd640 1 -- 172.21.15.125:0/1334880944 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9600106c0 con 0x7fd96c15b000 2023-10-18T12:00:07.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c14a540 con 0x7fd96c15b000 2023-10-18T12:00:07.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.860+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c1e2e40 con 0x7fd96c15b000 2023-10-18T12:00:07.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.861+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96c0690e0 con 0x7fd96c15b000 2023-10-18T12:00:07.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.861+0000 7fd95affd640 1 -- 172.21.15.125:0/1334880944 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 128) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd960010820 con 0x7fd96c15b000 2023-10-18T12:00:07.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.862+0000 7fd95affd640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd94c06ac90 0x7fd94c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:07.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.862+0000 7fd95affd640 1 -- 172.21.15.125:0/1334880944 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fd960089ae0 con 0x7fd96c15b000 2023-10-18T12:00:07.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.862+0000 7fd96a59c640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd94c06ac90 0x7fd94c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:07.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.862+0000 7fd96a59c640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd94c06ac90 0x7fd94c06d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0012e0 tx=0x7fd95c009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:07.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:07.866+0000 7fd95affd640 1 -- 172.21.15.125:0/1334880944 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd960054a30 con 0x7fd96c15b000 2023-10-18T12:00:08.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.064+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd96c0cd290 con 0x7fd96c15b000 2023-10-18T12:00:08.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.064+0000 7fd95affd640 1 -- 172.21.15.125:0/1334880944 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fd960020090 con 0x7fd96c15b000 2023-10-18T12:00:08.067 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:08.068 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":80,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2853562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2853562772},{"type":"v1","addr":"172.21.15.139:6836","nonce":2853562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":11421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3995754374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3995754374},{"type":"v1","addr":"172.21.15.125:6834","nonce":3995754374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":11424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2110336778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2110336778},{"type":"v1","addr":"172.21.15.125:6835","nonce":2110336778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:58:35.848069+0000","modified":"2023-10-18T11:58:37.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11439":{"gid":11439,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/1491435386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1491435386},{"type":"v1","addr":"172.21.15.139:6837","nonce":1491435386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-18T12:00:08.068 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 80 2023-10-18T12:00:08.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd94c06ac90 msgr2=0x7fd94c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd94c06ac90 0x7fd94c06d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0012e0 tx=0x7fd95c009040 comp rx=0 tx=0).stop 2023-10-18T12:00:08.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b000 msgr2=0x7fd96c1e0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b000 0x7fd96c1e0500 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7fd96000d980 tx=0x7fd96000de40 comp rx=0 tx=0).stop 2023-10-18T12:00:08.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 shutdown_connections 2023-10-18T12:00:08.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd94c06ac90 0x7fd94c06d150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd96c15b3e0 0x7fd96c14a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd96c15b000 0x7fd96c1e0500 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 --2- 172.21.15.125:0/1334880944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd96c0cf9e0 0x7fd96c157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 >> 172.21.15.125:0/1334880944 conn(0x7fd96c0d0cf0 msgr2=0x7fd96c1e73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:08.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 shutdown_connections 2023-10-18T12:00:08.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.066+0000 7fd970c9b640 1 -- 172.21.15.125:0/1334880944 wait complete. 2023-10-18T12:00:08.079 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:08.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.179+0000 7f399a6c9640 1 -- 172.21.15.125:0/1287390006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c003ef0 msgr2=0x7f397c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.179+0000 7f399a6c9640 1 --2- 172.21.15.125:0/1287390006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397c003ef0 0x7f397c0047c0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f398800d3a0 tx=0x7f397c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.179+0000 7f3998c66640 1 -- 172.21.15.125:0/1287390006 reap_dead start 2023-10-18T12:00:08.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.179+0000 7f399a6c9640 1 -- 172.21.15.125:0/1287390006 shutdown_connections 2023-10-18T12:00:08.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.179+0000 7f399a6c9640 1 -- 172.21.15.125:0/1287390006 >> 172.21.15.125:0/1287390006 conn(0x7f3994069930 msgr2=0x7f3994069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:08.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.179+0000 7f399a6c9640 1 -- 172.21.15.125:0/1287390006 shutdown_connections 2023-10-18T12:00:08.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.179+0000 7f399a6c9640 1 -- 172.21.15.125:0/1287390006 wait complete. 2023-10-18T12:00:08.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.180+0000 7f399a6c9640 1 Processor -- start 2023-10-18T12:00:08.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f399a6c9640 1 -- start start 2023-10-18T12:00:08.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f399a6c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c003ef0 0x7f399414c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f399a6c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39940ad7f0 0x7f39940adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f399a6c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3994138010 0x7f3994138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f399a6c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39941389d0 con 0x7f3994138010 2023-10-18T12:00:08.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f399a6c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994149cf0 con 0x7f397c003ef0 2023-10-18T12:00:08.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f399a6c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3994149fd0 con 0x7f39940ad7f0 2023-10-18T12:00:08.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f3998c66640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3994138010 0x7f3994138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f3998c66640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3994138010 0x7f3994138490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35132/0 (socket says 172.21.15.125:35132) 2023-10-18T12:00:08.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f3998c66640 1 -- 172.21.15.125:0/1638712816 learned_addr learned my addr 172.21.15.125:0/1638712816 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:08.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.181+0000 7f3993fff640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c003ef0 0x7f399414c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f39937fe640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39940ad7f0 0x7f39940adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3998c66640 1 -- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39940ad7f0 msgr2=0x7f39940adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3998c66640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39940ad7f0 0x7f39940adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3998c66640 1 -- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c003ef0 msgr2=0x7f399414c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3998c66640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c003ef0 0x7f399414c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3998c66640 1 -- 172.21.15.125:0/1638712816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399414a2b0 con 0x7f3994138010 2023-10-18T12:00:08.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3993fff640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c003ef0 0x7f399414c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:08.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f39937fe640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39940ad7f0 0x7f39940adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:08.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3998c66640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3994138010 0x7f3994138490 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f399406f6d0 tx=0x7f398c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:08.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3990ff9640 1 -- 172.21.15.125:0/1638712816 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c00ee70 con 0x7f3994138010 2023-10-18T12:00:08.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3990ff9640 1 -- 172.21.15.125:0/1638712816 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f398c003a80 con 0x7f3994138010 2023-10-18T12:00:08.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399414a590 con 0x7f3994138010 2023-10-18T12:00:08.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f3990ff9640 1 -- 172.21.15.125:0/1638712816 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c003e90 con 0x7f3994138010 2023-10-18T12:00:08.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.182+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399414ab50 con 0x7f3994138010 2023-10-18T12:00:08.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.183+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399414ad70 con 0x7f3994138010 2023-10-18T12:00:08.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.183+0000 7f3990ff9640 1 -- 172.21.15.125:0/1638712816 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 128) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f398c019030 con 0x7f3994138010 2023-10-18T12:00:08.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.184+0000 7f3990ff9640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f398006ac90 0x7f398006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.184+0000 7f3990ff9640 1 -- 172.21.15.125:0/1638712816 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f398c017070 con 0x7f3994138010 2023-10-18T12:00:08.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.184+0000 7f3993fff640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f398006ac90 0x7f398006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.184+0000 7f3993fff640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f398006ac90 0x7f398006d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3984009bc0 tx=0x7f3984005f90 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:08.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.188+0000 7f3990ff9640 1 -- 172.21.15.125:0/1638712816 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f398c055070 con 0x7f3994138010 2023-10-18T12:00:08.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.388+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39940690e0 con 0x7f3994138010 2023-10-18T12:00:08.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.389+0000 7f3990ff9640 1 -- 172.21.15.125:0/1638712816 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f398c058690 con 0x7f3994138010 2023-10-18T12:00:08.392 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:08.392 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":80,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2853562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2853562772},{"type":"v1","addr":"172.21.15.139:6836","nonce":2853562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":11421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3995754374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3995754374},{"type":"v1","addr":"172.21.15.125:6834","nonce":3995754374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":11424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2110336778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2110336778},{"type":"v1","addr":"172.21.15.125:6835","nonce":2110336778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T11:58:35.848069+0000","modified":"2023-10-18T11:58:37.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11439":{"gid":11439,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/1491435386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1491435386},{"type":"v1","addr":"172.21.15.139:6837","nonce":1491435386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-18T12:00:08.392 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 80 2023-10-18T12:00:08.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f398006ac90 msgr2=0x7f398006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f398006ac90 0x7f398006d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3984009bc0 tx=0x7f3984005f90 comp rx=0 tx=0).stop 2023-10-18T12:00:08.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3994138010 msgr2=0x7f3994138490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3994138010 0x7f3994138490 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f399406f6d0 tx=0x7f398c010f40 comp rx=0 tx=0).stop 2023-10-18T12:00:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 shutdown_connections 2023-10-18T12:00:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f398006ac90 0x7f398006d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3994138010 0x7f3994138490 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39940ad7f0 0x7f39940adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.390+0000 7f399a6c9640 1 --2- 172.21.15.125:0/1638712816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397c003ef0 0x7f399414c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.391+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 >> 172.21.15.125:0/1638712816 conn(0x7f3994069930 msgr2=0x7f3994159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:08.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.391+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 shutdown_connections 2023-10-18T12:00:08.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.391+0000 7f399a6c9640 1 -- 172.21.15.125:0/1638712816 wait complete. 2023-10-18T12:00:08.404 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:08.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.503+0000 7feb0e71d640 1 -- 172.21.15.125:0/2133740728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb0806af00 msgr2=0x7feb0806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.503+0000 7feb0e71d640 1 --2- 172.21.15.125:0/2133740728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb0806af00 0x7feb0806b2e0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009830 tx=0x7feaf802bcd0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.503+0000 7feb0e71d640 1 -- 172.21.15.125:0/2133740728 shutdown_connections 2023-10-18T12:00:08.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.503+0000 7feb0e71d640 1 --2- 172.21.15.125:0/2133740728 >> v2:172.21.15.139:3300/0 conn(0x7feb080abc10 0x7feb080c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.503+0000 7feb0e71d640 1 --2- 172.21.15.125:0/2133740728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb0806af00 0x7feb0806b2e0 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.503+0000 7feb0e71d640 1 -- 172.21.15.125:0/2133740728 >> 172.21.15.125:0/2133740728 conn(0x7feb08069930 msgr2=0x7feb08069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:08.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.504+0000 7feb0e71d640 1 -- 172.21.15.125:0/2133740728 shutdown_connections 2023-10-18T12:00:08.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.504+0000 7feb0e71d640 1 -- 172.21.15.125:0/2133740728 wait complete. 2023-10-18T12:00:08.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.505+0000 7feb0e71d640 1 Processor -- start 2023-10-18T12:00:08.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.505+0000 7feb0e71d640 1 -- start start 2023-10-18T12:00:08.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0e71d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb0806af00 0x7feb081380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0e71d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb080abc10 0x7feb081f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0e71d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb080c96b0 0x7feb080b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0e71d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08139850 con 0x7feb080abc10 2023-10-18T12:00:08.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0e71d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb080b1450 con 0x7feb0806af00 2023-10-18T12:00:08.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0e71d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb081386d0 con 0x7feb080c96b0 2023-10-18T12:00:08.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb080abc10 0x7feb081f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb080abc10 0x7feb081f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35136/0 (socket says 172.21.15.125:35136) 2023-10-18T12:00:08.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 -- 172.21.15.125:0/1610385355 learned_addr learned my addr 172.21.15.125:0/1610385355 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:08.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb07fff640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb0806af00 0x7feb081380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0ccba640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb080c96b0 0x7feb080b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 -- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb080c96b0 msgr2=0x7feb080b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb080c96b0 0x7feb080b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 -- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb0806af00 msgr2=0x7feb081380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb0806af00 0x7feb081380c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 -- 172.21.15.125:0/1610385355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8009490 con 0x7feb080abc10 2023-10-18T12:00:08.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0ccba640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb080c96b0 0x7feb080b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:08.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb077fe640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb080abc10 0x7feb081f0ea0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c440 tx=0x7feafc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:08.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb04ff9640 1 -- 172.21.15.125:0/1610385355 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc005480 con 0x7feb080abc10 2023-10-18T12:00:08.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb04ff9640 1 -- 172.21.15.125:0/1610385355 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feafc0035f0 con 0x7feb080abc10 2023-10-18T12:00:08.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb04ff9640 1 -- 172.21.15.125:0/1610385355 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc003a00 con 0x7feb080abc10 2023-10-18T12:00:08.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb081389b0 con 0x7feb080abc10 2023-10-18T12:00:08.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.506+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb08138f70 con 0x7feb080abc10 2023-10-18T12:00:08.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.507+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0806f110 con 0x7feb080abc10 2023-10-18T12:00:08.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.508+0000 7feb04ff9640 1 -- 172.21.15.125:0/1610385355 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 128) v1 ==== 89067+0+0 (secure 0 0 0) 0x7feafc003b60 con 0x7feb080abc10 2023-10-18T12:00:08.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.508+0000 7feb04ff9640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7feaf406acc0 0x7feaf406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.508+0000 7feb04ff9640 1 -- 172.21.15.125:0/1610385355 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 7060+0+0 (secure 0 0 0) 0x7feafc088df0 con 0x7feb080abc10 2023-10-18T12:00:08.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.508+0000 7feb07fff640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7feaf406acc0 0x7feaf406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.509+0000 7feb07fff640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7feaf406acc0 0x7feaf406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7feaf8036000 tx=0x7feaf8004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:08.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.512+0000 7feb04ff9640 1 -- 172.21.15.125:0/1610385355 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb0806f110 con 0x7feb080abc10 2023-10-18T12:00:08.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.680+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb0806f110 con 0x7feb080abc10 2023-10-18T12:00:08.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.681+0000 7feb04ff9640 1 -- 172.21.15.125:0/1610385355 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+18405 (secure 0 0 0) 0x7feb0806f110 con 0x7feb080abc10 2023-10-18T12:00:08.684 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:08.685 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":208,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:06.777619+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-18T11:58:34.060453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-18T11:58:34.823261+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":208,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.f","mappings":[{"from":1,"to":0}]},{"pgid":"26.13","mappings":[{"from":1,"to":6}]},{"pgid":"26.15","mappings":[{"from":3,"to":6}]},{"pgid":"26.19","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3621824170":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6833/3820883843":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6832/3820883843":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2142329879":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/693299469":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/3012481544":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2446385430":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/870215635":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/3819578023":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/330535304":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2863286615":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/1415408500":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2323488592":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6833/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2575337659":"2023-10-19T11:58:51.730016+0000","172.21.15.139:6832/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2354007263":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/1682437":"2023-10-19T11:58:51.730016+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:08.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7feaf406acc0 msgr2=0x7feaf406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7feaf406acc0 0x7feaf406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7feaf8036000 tx=0x7feaf8004730 comp rx=0 tx=0).stop 2023-10-18T12:00:08.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb080abc10 msgr2=0x7feb081f0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb080abc10 0x7feb081f0ea0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c440 tx=0x7feafc00c900 comp rx=0 tx=0).stop 2023-10-18T12:00:08.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 shutdown_connections 2023-10-18T12:00:08.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7feaf406acc0 0x7feaf406d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb080c96b0 0x7feb080b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb080abc10 0x7feb081f0ea0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 --2- 172.21.15.125:0/1610385355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb0806af00 0x7feb081380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 >> 172.21.15.125:0/1610385355 conn(0x7feb08069930 msgr2=0x7feb080cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:08.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 shutdown_connections 2023-10-18T12:00:08.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.683+0000 7feb0e71d640 1 -- 172.21.15.125:0/1610385355 wait complete. 2023-10-18T12:00:08.697 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T12:00:08.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.797+0000 7f4d98f97640 1 -- 172.21.15.125:0/1726498079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d940cb980 msgr2=0x7f4d9413db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.797+0000 7f4d98f97640 1 --2- 172.21.15.125:0/1726498079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d940cb980 0x7f4d9413db80 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009830 tx=0x7f4d8402bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:08.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.798+0000 7f4d98f97640 1 -- 172.21.15.125:0/1726498079 shutdown_connections 2023-10-18T12:00:08.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.798+0000 7f4d98f97640 1 --2- 172.21.15.125:0/1726498079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d94070350 0x7f4d940707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.798+0000 7f4d98f97640 1 --2- 172.21.15.125:0/1726498079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d940cb980 0x7f4d9413db80 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.798+0000 7f4d98f97640 1 -- 172.21.15.125:0/1726498079 >> 172.21.15.125:0/1726498079 conn(0x7f4d94069930 msgr2=0x7f4d94069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:08.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.798+0000 7f4d98f97640 1 -- 172.21.15.125:0/1726498079 shutdown_connections 2023-10-18T12:00:08.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.798+0000 7f4d98f97640 1 -- 172.21.15.125:0/1726498079 wait complete. 2023-10-18T12:00:08.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.799+0000 7f4d98f97640 1 Processor -- start 2023-10-18T12:00:08.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.799+0000 7f4d98f97640 1 -- start start 2023-10-18T12:00:08.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d98f97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d94070350 0x7f4d941ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d98f97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d940cb980 0x7f4d941f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d98f97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d941f2cc0 0x7f4d941513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d98f97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d941f3cb0 con 0x7f4d941f2cc0 2023-10-18T12:00:08.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d98f97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d941ef6d0 con 0x7f4d940cb980 2023-10-18T12:00:08.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d98f97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d941efc50 con 0x7f4d94070350 2023-10-18T12:00:08.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d941f2cc0 0x7f4d941513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d941f2cc0 0x7f4d941513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35142/0 (socket says 172.21.15.125:35142) 2023-10-18T12:00:08.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 -- 172.21.15.125:0/2191019595 learned_addr learned my addr 172.21.15.125:0/2191019595 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:08.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d9259c640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d94070350 0x7f4d941ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d8bfff640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d940cb980 0x7f4d941f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 -- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d94070350 msgr2=0x7f4d941ecbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d94070350 0x7f4d941ecbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 -- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d940cb980 msgr2=0x7f4d941f00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:08.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d940cb980 0x7f4d941f00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:08.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 -- 172.21.15.125:0/2191019595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d84009490 con 0x7f4d941f2cc0 2023-10-18T12:00:08.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d8bfff640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d940cb980 0x7f4d941f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:08.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d92d9d640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d941f2cc0 0x7f4d941513e0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00eb10 tx=0x7f4d8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:08.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d8affd640 1 -- 172.21.15.125:0/2191019595 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c004b20 con 0x7f4d941f2cc0 2023-10-18T12:00:08.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d8affd640 1 -- 172.21.15.125:0/2191019595 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d8c004ca0 con 0x7f4d941f2cc0 2023-10-18T12:00:08.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d941ed0f0 con 0x7f4d941f2cc0 2023-10-18T12:00:08.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d8affd640 1 -- 172.21.15.125:0/2191019595 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c005280 con 0x7f4d941f2cc0 2023-10-18T12:00:08.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.800+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d941f39d0 con 0x7f4d941f2cc0 2023-10-18T12:00:08.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.801+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d940690e0 con 0x7f4d941f2cc0 2023-10-18T12:00:08.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.802+0000 7f4d8affd640 1 -- 172.21.15.125:0/2191019595 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 128) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4d8c0053e0 con 0x7f4d941f2cc0 2023-10-18T12:00:08.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.802+0000 7f4d8affd640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f4d7406acc0 0x7f4d7406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:08.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.802+0000 7f4d8affd640 1 -- 172.21.15.125:0/2191019595 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f4d8c014070 con 0x7f4d941f2cc0 2023-10-18T12:00:08.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.802+0000 7f4d9259c640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f4d7406acc0 0x7f4d7406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:08.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.803+0000 7f4d9259c640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f4d7406acc0 0x7f4d7406d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d84036000 tx=0x7f4d84004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:08.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:08.806+0000 7f4d8affd640 1 -- 172.21.15.125:0/2191019595 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d8c055310 con 0x7f4d941f2cc0 2023-10-18T12:00:09.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.010+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4d941efd90 con 0x7f4d941f2cc0 2023-10-18T12:00:09.013 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T12:00:09.010+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:00:09.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.012+0000 7f4d8affd640 1 -- 172.21.15.125:0/2191019595 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4d8c05de60 con 0x7f4d941f2cc0 2023-10-18T12:00:09.014 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T12:00:09.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f4d7406acc0 msgr2=0x7f4d7406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:09.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f4d7406acc0 0x7f4d7406d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d84036000 tx=0x7f4d84004730 comp rx=0 tx=0).stop 2023-10-18T12:00:09.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d941f2cc0 msgr2=0x7f4d941513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:09.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d941f2cc0 0x7f4d941513e0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00eb10 tx=0x7f4d8c00c2d0 comp rx=0 tx=0).stop 2023-10-18T12:00:09.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 shutdown_connections 2023-10-18T12:00:09.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f4d7406acc0 0x7f4d7406d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:09.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d940cb980 0x7f4d941f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:09.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d941f2cc0 0x7f4d941513e0 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:09.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 --2- 172.21.15.125:0/2191019595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d94070350 0x7f4d941ecbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:09.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 >> 172.21.15.125:0/2191019595 conn(0x7f4d94069930 msgr2=0x7f4d94155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:09.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 shutdown_connections 2023-10-18T12:00:09.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.014+0000 7f4d98f97640 1 -- 172.21.15.125:0/2191019595 wait complete. 2023-10-18T12:00:09.027 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T12:00:09.054 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T12:00:09.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.130+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/768331210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e4141690 msgr2=0x7fc7e40703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.130+0000 7fc7ea1b7640 1 --2- 172.21.15.125:0/768331210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e4141690 0x7fc7e40703e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009860 tx=0x7fc7cc02bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.130+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/768331210 shutdown_connections 2023-10-18T12:00:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.130+0000 7fc7ea1b7640 1 --2- 172.21.15.125:0/768331210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e4141690 0x7fc7e40703e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.130+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/768331210 >> 172.21.15.125:0/768331210 conn(0x7fc7e4069930 msgr2=0x7fc7e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:09.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.130+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/768331210 shutdown_connections 2023-10-18T12:00:09.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.130+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/768331210 wait complete. 2023-10-18T12:00:09.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.131+0000 7fc7ea1b7640 1 Processor -- start 2023-10-18T12:00:09.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.131+0000 7fc7ea1b7640 1 -- start start 2023-10-18T12:00:09.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7ea1b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e4070920 0x7fc7e41f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:09.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7ea1b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e413e520 0x7fc7e413a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:09.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7ea1b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7e4141690 0x7fc7e41d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:09.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7ea1b7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41d5cf0 con 0x7fc7e4141690 2023-10-18T12:00:09.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7ea1b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41f2310 con 0x7fc7e4070920 2023-10-18T12:00:09.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7ea1b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41d5f20 con 0x7fc7e413e520 2023-10-18T12:00:09.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7e4141690 0x7fc7e41d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:09.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7e4141690 0x7fc7e41d74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35148/0 (socket says 172.21.15.125:35148) 2023-10-18T12:00:09.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 -- 172.21.15.125:0/4141414737 learned_addr learned my addr 172.21.15.125:0/4141414737 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:09.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e2ffd640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e413e520 0x7fc7e413a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:09.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e37fe640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e4070920 0x7fc7e41f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:09.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 -- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e413e520 msgr2=0x7fc7e413a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:09.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e413e520 0x7fc7e413a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:09.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 -- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e4070920 msgr2=0x7fc7e41f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:09.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e4070920 0x7fc7e41f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:09.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 -- 172.21.15.125:0/4141414737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc009490 con 0x7fc7e4141690 2023-10-18T12:00:09.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7e3fff640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7e4141690 0x7fc7e41d74d0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc001eb0 tx=0x7fc7dc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:09.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7dbfff640 1 -- 172.21.15.125:0/4141414737 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc0052d0 con 0x7fc7e4141690 2023-10-18T12:00:09.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7dbfff640 1 -- 172.21.15.125:0/4141414737 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc7dc0035f0 con 0x7fc7e4141690 2023-10-18T12:00:09.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7dbfff640 1 -- 172.21.15.125:0/4141414737 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc0039e0 con 0x7fc7e4141690 2023-10-18T12:00:09.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e41d6200 con 0x7fc7e4141690 2023-10-18T12:00:09.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.132+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e41d67c0 con 0x7fc7e4141690 2023-10-18T12:00:09.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.133+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e40690e0 con 0x7fc7e4141690 2023-10-18T12:00:09.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.134+0000 7fc7dbfff640 1 -- 172.21.15.125:0/4141414737 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 128) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc7dc003b40 con 0x7fc7e4141690 2023-10-18T12:00:09.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.134+0000 7fc7dbfff640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc7c406ac90 0x7fc7c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:09.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.134+0000 7fc7dbfff640 1 -- 172.21.15.125:0/4141414737 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 7118+0+0 (secure 0 0 0) 0x7fc7dc088fc0 con 0x7fc7e4141690 2023-10-18T12:00:09.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.134+0000 7fc7e37fe640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc7c406ac90 0x7fc7c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:09.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.135+0000 7fc7e37fe640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc7c406ac90 0x7fc7c406d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4001560 tx=0x7fc7d4001ad0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:09.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.138+0000 7fc7dbfff640 1 -- 172.21.15.125:0/4141414737 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7dc053e60 con 0x7fc7e4141690 2023-10-18T12:00:09.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.336+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc7e40b1470 con 0x7fc7e4141690 2023-10-18T12:00:09.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:09.793+0000 7fc7dbfff640 1 -- 172.21.15.125:0/4141414737 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 129) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc7dc054230 con 0x7fc7e4141690 2023-10-18T12:00:10.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.013+0000 7fc7dbfff640 1 -- 172.21.15.125:0/4141414737 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc7dc083bd0 con 0x7fc7e4141690 2023-10-18T12:00:10.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc7c406ac90 msgr2=0x7fc7c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:10.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc7c406ac90 0x7fc7c406d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4001560 tx=0x7fc7d4001ad0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7e4141690 msgr2=0x7fc7e41d74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:10.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7e4141690 0x7fc7e41d74d0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc001eb0 tx=0x7fc7dc00c890 comp rx=0 tx=0).stop 2023-10-18T12:00:10.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 shutdown_connections 2023-10-18T12:00:10.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc7c406ac90 0x7fc7c406d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7e4141690 0x7fc7e41d74d0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7e413e520 0x7fc7e413a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 --2- 172.21.15.125:0/4141414737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e4070920 0x7fc7e41f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 >> 172.21.15.125:0/4141414737 conn(0x7fc7e4069930 msgr2=0x7fc7e40c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:10.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 shutdown_connections 2023-10-18T12:00:10.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.015+0000 7fc7ea1b7640 1 -- 172.21.15.125:0/4141414737 wait complete. 2023-10-18T12:00:10.029 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T12:00:10.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 -- 172.21.15.125:0/2686683725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 msgr2=0x7f73180d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:10.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 --2- 172.21.15.125:0/2686683725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 0x7f73180d1910 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f731806c850 tx=0x7f731402c1d0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 -- 172.21.15.125:0/2686683725 shutdown_connections 2023-10-18T12:00:10.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 --2- 172.21.15.125:0/2686683725 >> v2:172.21.15.139:3300/0 conn(0x7f73180ceb40 0x7f73180b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 --2- 172.21.15.125:0/2686683725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731815b2a0 0x7f73180cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 --2- 172.21.15.125:0/2686683725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 0x7f73180d1910 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 -- 172.21.15.125:0/2686683725 >> 172.21.15.125:0/2686683725 conn(0x7f73180d0d60 msgr2=0x7f73180d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:10.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 -- 172.21.15.125:0/2686683725 shutdown_connections 2023-10-18T12:00:10.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.133+0000 7f732044b640 1 -- 172.21.15.125:0/2686683725 wait complete. 2023-10-18T12:00:10.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.134+0000 7f732044b640 1 Processor -- start 2023-10-18T12:00:10.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.134+0000 7f732044b640 1 -- start start 2023-10-18T12:00:10.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f732044b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73180ceb40 0x7f73181591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:10.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f732044b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 0x7f731815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:10.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f732044b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f731815b2a0 0x7f731815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:10.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f732044b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73180cb030 con 0x7f73180cf340 2023-10-18T12:00:10.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f732044b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73180ca3b0 con 0x7f731815b2a0 2023-10-18T12:00:10.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f732044b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73180ca690 con 0x7f73180ceb40 2023-10-18T12:00:10.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731d9e6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 0x7f731815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:10.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731d9e6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 0x7f731815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35172/0 (socket says 172.21.15.125:35172) 2023-10-18T12:00:10.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731d9e6640 1 -- 172.21.15.125:0/2857650518 learned_addr learned my addr 172.21.15.125:0/2857650518 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:10.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731e1e7640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73180ceb40 0x7f73181591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:10.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731d9e6640 1 -- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73180ceb40 msgr2=0x7f73181591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:10.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731e9e8640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f731815b2a0 0x7f731815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:10.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731d9e6640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73180ceb40 0x7f73181591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731d9e6640 1 -- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f731815b2a0 msgr2=0x7f731815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:10.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731d9e6640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f731815b2a0 0x7f731815dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:10.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.135+0000 7f731d9e6640 1 -- 172.21.15.125:0/2857650518 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314009900 con 0x7f73180cf340 2023-10-18T12:00:10.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.136+0000 7f731e1e7640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73180ceb40 0x7f73181591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:10.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.136+0000 7f731d9e6640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 0x7f731815d8a0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f730800e7c0 tx=0x7f730800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:10.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.136+0000 7f730effd640 1 -- 172.21.15.125:0/2857650518 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080095a0 con 0x7f73180cf340 2023-10-18T12:00:10.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.136+0000 7f730effd640 1 -- 172.21.15.125:0/2857650518 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f730800eed0 con 0x7f73180cf340 2023-10-18T12:00:10.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.136+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73180ca970 con 0x7f73180cf340 2023-10-18T12:00:10.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.136+0000 7f730effd640 1 -- 172.21.15.125:0/2857650518 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308005020 con 0x7f73180cf340 2023-10-18T12:00:10.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.136+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7318159730 con 0x7f73180cf340 2023-10-18T12:00:10.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.137+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73180690e0 con 0x7f73180cf340 2023-10-18T12:00:10.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.137+0000 7f730effd640 1 -- 172.21.15.125:0/2857650518 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 129) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7308005180 con 0x7f73180cf340 2023-10-18T12:00:10.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.138+0000 7f730effd640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f72fc06aed0 0x7f72fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:10.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.138+0000 7f730effd640 1 -- 172.21.15.125:0/2857650518 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 7118+0+0 (secure 0 0 0) 0x7f7308014070 con 0x7f73180cf340 2023-10-18T12:00:10.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.138+0000 7f731e1e7640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f72fc06aed0 0x7f72fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:10.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.138+0000 7f731e1e7640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f72fc06aed0 0x7f72fc06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7314036000 tx=0x7f7314004ab0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:10.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.142+0000 7f730effd640 1 -- 172.21.15.125:0/2857650518 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7308058b60 con 0x7f73180cf340 2023-10-18T12:00:10.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:10.308+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73180d0b40 con 0x7f73180cf340 2023-10-18T12:00:11.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.014+0000 7f730effd640 1 -- 172.21.15.125:0/2857650518 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v211) v1 ==== 181+0+0 (secure 0 0 0) 0x7f73080586f0 con 0x7f73180cf340 2023-10-18T12:00:11.017 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T12:00:11.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f72fc06aed0 msgr2=0x7f72fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:11.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f72fc06aed0 0x7f72fc06d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7314036000 tx=0x7f7314004ab0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 msgr2=0x7f731815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:11.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 0x7f731815d8a0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f730800e7c0 tx=0x7f730800ec80 comp rx=0 tx=0).stop 2023-10-18T12:00:11.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 shutdown_connections 2023-10-18T12:00:11.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f72fc06aed0 0x7f72fc06d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f731815b2a0 0x7f731815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73180cf340 0x7f731815d8a0 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 --2- 172.21.15.125:0/2857650518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73180ceb40 0x7f73181591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.017+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 >> 172.21.15.125:0/2857650518 conn(0x7f73180d0d60 msgr2=0x7f73180b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:11.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.018+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 shutdown_connections 2023-10-18T12:00:11.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.018+0000 7f732044b640 1 -- 172.21.15.125:0/2857650518 wait complete. 2023-10-18T12:00:11.033 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T12:00:11.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.138+0000 7f57aae51640 1 -- 172.21.15.125:0/2548782718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a415b640 msgr2=0x7f57a40cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:11.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.138+0000 7f57aae51640 1 --2- 172.21.15.125:0/2548782718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a415b640 0x7f57a40cf850 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f5790005fd0 tx=0x7f579002be10 comp rx=0 tx=0).stop 2023-10-18T12:00:11.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.138+0000 7f57aae51640 1 -- 172.21.15.125:0/2548782718 shutdown_connections 2023-10-18T12:00:11.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.138+0000 7f57aae51640 1 --2- 172.21.15.125:0/2548782718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a415b640 0x7f57a40cf850 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.138+0000 7f57aae51640 1 --2- 172.21.15.125:0/2548782718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57a415b260 0x7f57a40cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.138+0000 7f57aae51640 1 -- 172.21.15.125:0/2548782718 >> 172.21.15.125:0/2548782718 conn(0x7f57a409a030 msgr2=0x7f57a40d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:11.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.138+0000 7f57aae51640 1 -- 172.21.15.125:0/2548782718 shutdown_connections 2023-10-18T12:00:11.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.138+0000 7f57aae51640 1 -- 172.21.15.125:0/2548782718 wait complete. 2023-10-18T12:00:11.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.139+0000 7f57aae51640 1 Processor -- start 2023-10-18T12:00:11.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.140+0000 7f57aae51640 1 -- start start 2023-10-18T12:00:11.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.140+0000 7f57aae51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a40b0b10 0x7f57a40c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:11.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.140+0000 7f57aae51640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57a415b260 0x7f57a41f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:11.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.140+0000 7f57aae51640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57a415b640 0x7f57a415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:11.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.140+0000 7f57aae51640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40c61a0 con 0x7f57a40b0b10 2023-10-18T12:00:11.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.140+0000 7f57aae51640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40c69b0 con 0x7f57a415b640 2023-10-18T12:00:11.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.140+0000 7f57aae51640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40c63f0 con 0x7f57a415b260 2023-10-18T12:00:11.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.140+0000 7f57a8bed640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a40b0b10 0x7f57a40c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:11.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a8bed640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a40b0b10 0x7f57a40c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35194/0 (socket says 172.21.15.125:35194) 2023-10-18T12:00:11.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a8bed640 1 -- 172.21.15.125:0/1133579259 learned_addr learned my addr 172.21.15.125:0/1133579259 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:11.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a8bed640 1 -- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57a415b260 msgr2=0x7f57a41f0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:11.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a3fff640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57a415b260 0x7f57a41f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:11.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a93ee640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57a415b640 0x7f57a415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:11.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a8bed640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57a415b260 0x7f57a41f0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a8bed640 1 -- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57a415b640 msgr2=0x7f57a415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:11.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a8bed640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57a415b640 0x7f57a415ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:11.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a8bed640 1 -- 172.21.15.125:0/1133579259 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5790009560 con 0x7f57a40b0b10 2023-10-18T12:00:11.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a3fff640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57a415b260 0x7f57a41f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:11.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a8bed640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a40b0b10 0x7f57a40c5260 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f579800d950 tx=0x7f579800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:11.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a17fa640 1 -- 172.21.15.125:0/1133579259 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798009c30 con 0x7f57a40b0b10 2023-10-18T12:00:11.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a17fa640 1 -- 172.21.15.125:0/1133579259 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5798009e80 con 0x7f57a40b0b10 2023-10-18T12:00:11.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57a17fa640 1 -- 172.21.15.125:0/1133579259 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980106c0 con 0x7f57a40b0b10 2023-10-18T12:00:11.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a40c66d0 con 0x7f57a40b0b10 2023-10-18T12:00:11.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.141+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a41512c0 con 0x7f57a40b0b10 2023-10-18T12:00:11.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.142+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a40675f0 con 0x7f57a40b0b10 2023-10-18T12:00:11.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.143+0000 7f57a17fa640 1 -- 172.21.15.125:0/1133579259 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 129) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5798010820 con 0x7f57a40b0b10 2023-10-18T12:00:11.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.143+0000 7f57a17fa640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f578806aed0 0x7f578806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:11.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.143+0000 7f57a17fa640 1 -- 172.21.15.125:0/1133579259 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f5798089f50 con 0x7f57a40b0b10 2023-10-18T12:00:11.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.144+0000 7f57a3fff640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f578806aed0 0x7f578806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:11.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.144+0000 7f57a3fff640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f578806aed0 0x7f578806d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5790035000 tx=0x7f5790004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:11.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.147+0000 7f57a17fa640 1 -- 172.21.15.125:0/1133579259 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57980bcca0 con 0x7f57a40b0b10 2023-10-18T12:00:11.315 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.313+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57a40ae880 con 0x7f57a40b0b10 2023-10-18T12:00:11.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:11.414+0000 7f57a17fa640 1 -- 172.21.15.125:0/1133579259 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5798054fb0 con 0x7f57a40b0b10 2023-10-18T12:00:12.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.021+0000 7f57a17fa640 1 -- 172.21.15.125:0/1133579259 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v212) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5798087cb0 con 0x7f57a40b0b10 2023-10-18T12:00:12.024 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T12:00:12.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f578806aed0 msgr2=0x7f578806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f578806aed0 0x7f578806d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5790035000 tx=0x7f5790004730 comp rx=0 tx=0).stop 2023-10-18T12:00:12.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a40b0b10 msgr2=0x7f57a40c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a40b0b10 0x7f57a40c5260 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f579800d950 tx=0x7f579800de10 comp rx=0 tx=0).stop 2023-10-18T12:00:12.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 shutdown_connections 2023-10-18T12:00:12.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f578806aed0 0x7f578806d390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57a415b640 0x7f57a415ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57a415b260 0x7f57a41f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 --2- 172.21.15.125:0/1133579259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57a40b0b10 0x7f57a40c5260 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 >> 172.21.15.125:0/1133579259 conn(0x7f57a409a030 msgr2=0x7f57a4159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:12.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 shutdown_connections 2023-10-18T12:00:12.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.023+0000 7f57aae51640 1 -- 172.21.15.125:0/1133579259 wait complete. 2023-10-18T12:00:12.038 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T12:00:12.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.139+0000 7fddc4f3c640 1 -- 172.21.15.125:0/1273432419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 msgr2=0x7fddc00b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.139+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/1273432419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 0x7fddc00b0650 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fddb8005fd0 tx=0x7fddb802c240 comp rx=0 tx=0).stop 2023-10-18T12:00:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.139+0000 7fddc4f3c640 1 -- 172.21.15.125:0/1273432419 shutdown_connections 2023-10-18T12:00:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.139+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/1273432419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 0x7fddc00b0650 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.139+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/1273432419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc00cf340 0x7fddc00d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.139+0000 7fddc4f3c640 1 -- 172.21.15.125:0/1273432419 >> 172.21.15.125:0/1273432419 conn(0x7fddc00d0d60 msgr2=0x7fddc00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:12.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.139+0000 7fddc4f3c640 1 -- 172.21.15.125:0/1273432419 shutdown_connections 2023-10-18T12:00:12.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.139+0000 7fddc4f3c640 1 -- 172.21.15.125:0/1273432419 wait complete. 2023-10-18T12:00:12.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddc4f3c640 1 Processor -- start 2023-10-18T12:00:12.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddc4f3c640 1 -- start start 2023-10-18T12:00:12.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddc4f3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 0x7fddc0157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:12.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddc4f3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc00cf340 0x7fddc01e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:12.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddc4f3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc015b2a0 0x7fddc0156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:12.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddc4f3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0158500 con 0x7fddc00ceb40 2023-10-18T12:00:12.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddc4f3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00b6850 con 0x7fddc015b2a0 2023-10-18T12:00:12.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddc4f3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01578f0 con 0x7fddc00cf340 2023-10-18T12:00:12.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 0x7fddc0157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:12.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 0x7fddc0157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35212/0 (socket says 172.21.15.125:35212) 2023-10-18T12:00:12.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.141+0000 7fddbe59c640 1 -- 172.21.15.125:0/3165538953 learned_addr learned my addr 172.21.15.125:0/3165538953 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:12.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddb7fff640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc00cf340 0x7fddc01e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:12.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddbed9d640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc015b2a0 0x7fddc0156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:12.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddbe59c640 1 -- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc00cf340 msgr2=0x7fddc01e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddbe59c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc00cf340 0x7fddc01e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddbe59c640 1 -- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc015b2a0 msgr2=0x7fddc0156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddbe59c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc015b2a0 0x7fddc0156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddbe59c640 1 -- 172.21.15.125:0/3165538953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8009900 con 0x7fddc00ceb40 2023-10-18T12:00:12.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddbe59c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 0x7fddc0157310 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fddb000d770 tx=0x7fddb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:12.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddb6ffd640 1 -- 172.21.15.125:0/3165538953 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0009c30 con 0x7fddc00ceb40 2023-10-18T12:00:12.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddb6ffd640 1 -- 172.21.15.125:0/3165538953 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fddb000de80 con 0x7fddc00ceb40 2023-10-18T12:00:12.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddb6ffd640 1 -- 172.21.15.125:0/3165538953 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0005280 con 0x7fddc00ceb40 2023-10-18T12:00:12.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc0157bd0 con 0x7fddc00ceb40 2023-10-18T12:00:12.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.142+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc0158190 con 0x7fddc00ceb40 2023-10-18T12:00:12.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.143+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc00690e0 con 0x7fddc00ceb40 2023-10-18T12:00:12.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.144+0000 7fddb6ffd640 1 -- 172.21.15.125:0/3165538953 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fddb00053e0 con 0x7fddc00ceb40 2023-10-18T12:00:12.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.144+0000 7fddb6ffd640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fdda006af60 0x7fdda006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.144+0000 7fddb6ffd640 1 -- 172.21.15.125:0/3165538953 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7fddb0089c80 con 0x7fddc00ceb40 2023-10-18T12:00:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.144+0000 7fddb7fff640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fdda006af60 0x7fdda006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:12.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.145+0000 7fddb7fff640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fdda006af60 0x7fdda006d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007fd0 tx=0x7fdda8009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:12.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.148+0000 7fddb6ffd640 1 -- 172.21.15.125:0/3165538953 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fddb0054990 con 0x7fddc00ceb40 2023-10-18T12:00:12.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.315+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7fddc00d0b40 con 0x7fddc00ceb40 2023-10-18T12:00:12.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.318+0000 7fddb6ffd640 1 -- 172.21.15.125:0/3165538953 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v74) v1 ==== 71+0+0 (secure 0 0 0) 0x7fddb0017020 con 0x7fddc00ceb40 2023-10-18T12:00:12.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.320+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fdda006af60 msgr2=0x7fdda006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.320+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fdda006af60 0x7fdda006d420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007fd0 tx=0x7fdda8009040 comp rx=0 tx=0).stop 2023-10-18T12:00:12.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.320+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 msgr2=0x7fddc0157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.320+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 0x7fddc0157310 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fddb000d770 tx=0x7fddb000dc30 comp rx=0 tx=0).stop 2023-10-18T12:00:12.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.321+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 shutdown_connections 2023-10-18T12:00:12.324 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.321+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fdda006af60 0x7fdda006d420 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.321+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddc015b2a0 0x7fddc0156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.321+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fddc00cf340 0x7fddc01e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.321+0000 7fddc4f3c640 1 --2- 172.21.15.125:0/3165538953 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddc00ceb40 0x7fddc0157310 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.321+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 >> 172.21.15.125:0/3165538953 conn(0x7fddc00d0d60 msgr2=0x7fddc015a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:12.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.321+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 shutdown_connections 2023-10-18T12:00:12.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.321+0000 7fddc4f3c640 1 -- 172.21.15.125:0/3165538953 wait complete. 2023-10-18T12:00:12.334 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-18T12:00:12.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.434+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1113094136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f980ceb40 msgr2=0x7f3f980b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.434+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1113094136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f980ceb40 0x7f3f980b0650 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f3f9808a120 tx=0x7f3f9002c1d0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.434+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1113094136 shutdown_connections 2023-10-18T12:00:12.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.434+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1113094136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f980ceb40 0x7f3f980b0650 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.434+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1113094136 >> v2:172.21.15.139:3300/0 conn(0x7f3f9815b2a0 0x7f3f980d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.434+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1113094136 >> 172.21.15.125:0/1113094136 conn(0x7f3f980d0d60 msgr2=0x7f3f980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:12.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.434+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1113094136 shutdown_connections 2023-10-18T12:00:12.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.434+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1113094136 wait complete. 2023-10-18T12:00:12.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.435+0000 7f3f9eec2640 1 Processor -- start 2023-10-18T12:00:12.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.435+0000 7f3f9eec2640 1 -- start start 2023-10-18T12:00:12.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9eec2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f980ceb40 0x7f3f98158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:12.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9eec2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f980cf340 0x7f3f9815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:12.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9eec2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f9815b2a0 0x7f3f9815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:12.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9eec2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f981e6770 con 0x7f3f9815b2a0 2023-10-18T12:00:12.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9eec2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f981e5b60 con 0x7f3f980cf340 2023-10-18T12:00:12.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9eec2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f981e5e40 con 0x7f3f980ceb40 2023-10-18T12:00:12.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9d45f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f9815b2a0 0x7f3f9815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:12.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9d45f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f9815b2a0 0x7f3f9815cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35234/0 (socket says 172.21.15.125:35234) 2023-10-18T12:00:12.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9d45f640 1 -- 172.21.15.125:0/1620215190 learned_addr learned my addr 172.21.15.125:0/1620215190 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:12.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f97fff640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f980cf340 0x7f3f9815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:12.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9cc5e640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f980ceb40 0x7f3f98158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:12.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9d45f640 1 -- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f980ceb40 msgr2=0x7f3f98158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9d45f640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f980ceb40 0x7f3f98158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9d45f640 1 -- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f980cf340 msgr2=0x7f3f9815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:12.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9d45f640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f980cf340 0x7f3f9815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:12.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.436+0000 7f3f9d45f640 1 -- 172.21.15.125:0/1620215190 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f90009900 con 0x7f3f9815b2a0 2023-10-18T12:00:12.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.437+0000 7f3f9d45f640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f9815b2a0 0x7f3f9815cc70 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009d60 tx=0x7f3f90035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:12.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.437+0000 7f3f957fa640 1 -- 172.21.15.125:0/1620215190 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90034440 con 0x7f3f9815b2a0 2023-10-18T12:00:12.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.437+0000 7f3f957fa640 1 -- 172.21.15.125:0/1620215190 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f90004450 con 0x7f3f9815b2a0 2023-10-18T12:00:12.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.437+0000 7f3f957fa640 1 -- 172.21.15.125:0/1620215190 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90004860 con 0x7f3f9815b2a0 2023-10-18T12:00:12.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.437+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f981e6120 con 0x7f3f9815b2a0 2023-10-18T12:00:12.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.437+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f981588b0 con 0x7f3f9815b2a0 2023-10-18T12:00:12.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.438+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f981f1190 con 0x7f3f9815b2a0 2023-10-18T12:00:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.438+0000 7f3f957fa640 1 -- 172.21.15.125:0/1620215190 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3f9000a040 con 0x7f3f9815b2a0 2023-10-18T12:00:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.439+0000 7f3f957fa640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3f7c06af60 0x7f3f7c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.439+0000 7f3f957fa640 1 -- 172.21.15.125:0/1620215190 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7f3f900ac8d0 con 0x7f3f9815b2a0 2023-10-18T12:00:12.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.439+0000 7f3f9cc5e640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3f7c06af60 0x7f3f7c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:12.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.439+0000 7f3f9cc5e640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3f7c06af60 0x7f3f7c06d420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c004f60 tx=0x7f3f8c007e30 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:12.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.443+0000 7f3f957fa640 1 -- 172.21.15.125:0/1620215190 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f981f1190 con 0x7f3f9815b2a0 2023-10-18T12:00:12.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:12.675+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f3f981f1190 con 0x7f3f9815b2a0 2023-10-18T12:00:13.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.022+0000 7f3f957fa640 1 -- 172.21.15.125:0/1620215190 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3085) v1 ==== 163+0+0 (secure 0 0 0) 0x7f3f981f1190 con 0x7f3f9815b2a0 2023-10-18T12:00:13.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3f7c06af60 msgr2=0x7f3f7c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3f7c06af60 0x7f3f7c06d420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c004f60 tx=0x7f3f8c007e30 comp rx=0 tx=0).stop 2023-10-18T12:00:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f9815b2a0 msgr2=0x7f3f9815cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f9815b2a0 0x7f3f9815cc70 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009d60 tx=0x7f3f90035e30 comp rx=0 tx=0).stop 2023-10-18T12:00:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 shutdown_connections 2023-10-18T12:00:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3f7c06af60 0x7f3f7c06d420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f980cf340 0x7f3f9815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3f9815b2a0 0x7f3f9815cc70 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 --2- 172.21.15.125:0/1620215190 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f980ceb40 0x7f3f98158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.025+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 >> 172.21.15.125:0/1620215190 conn(0x7f3f980d0d60 msgr2=0x7f3f9815a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.026+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 shutdown_connections 2023-10-18T12:00:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.026+0000 7f3f9eec2640 1 -- 172.21.15.125:0/1620215190 wait complete. 2023-10-18T12:00:13.039 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-10-18T12:00:13.040 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2023-10-18T12:00:13.040 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-18T12:00:13.041 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:13.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.141+0000 7f42225a9640 1 -- 172.21.15.125:0/4249849657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c141690 msgr2=0x7f421c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.141+0000 7f42225a9640 1 --2- 172.21.15.125:0/4249849657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c141690 0x7f421c0703e0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f4210009830 tx=0x7f421002bcd0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.141+0000 7f42225a9640 1 -- 172.21.15.125:0/4249849657 shutdown_connections 2023-10-18T12:00:13.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.141+0000 7f42225a9640 1 --2- 172.21.15.125:0/4249849657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c141690 0x7f421c0703e0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.141+0000 7f42225a9640 1 -- 172.21.15.125:0/4249849657 >> 172.21.15.125:0/4249849657 conn(0x7f421c069930 msgr2=0x7f421c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:13.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.141+0000 7f42225a9640 1 -- 172.21.15.125:0/4249849657 shutdown_connections 2023-10-18T12:00:13.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.141+0000 7f42225a9640 1 -- 172.21.15.125:0/4249849657 wait complete. 2023-10-18T12:00:13.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.142+0000 7f42225a9640 1 Processor -- start 2023-10-18T12:00:13.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.142+0000 7f42225a9640 1 -- start start 2023-10-18T12:00:13.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f42225a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f421c070920 0x7f421c0a8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f42225a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c13e520 0x7f421c0ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f42225a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f421c141690 0x7f421c1e2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f42225a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0a8d50 con 0x7f421c13e520 2023-10-18T12:00:13.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f42225a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0a8380 con 0x7f421c070920 2023-10-18T12:00:13.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f42225a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0a7a90 con 0x7f421c141690 2023-10-18T12:00:13.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c13e520 0x7f421c0ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:13.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c13e520 0x7f421c0ab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35254/0 (socket says 172.21.15.125:35254) 2023-10-18T12:00:13.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 -- 172.21.15.125:0/2723548925 learned_addr learned my addr 172.21.15.125:0/2723548925 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:13.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f4220b46640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f421c141690 0x7f421c1e2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:13.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421bfff640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f421c070920 0x7f421c0a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:13.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 -- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f421c141690 msgr2=0x7f421c1e2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f421c141690 0x7f421c1e2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 -- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f421c070920 msgr2=0x7f421c0a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f421c070920 0x7f421c0a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 -- 172.21.15.125:0/2723548925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210009490 con 0x7f421c13e520 2023-10-18T12:00:13.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421bfff640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f421c070920 0x7f421c0a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:13.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f421b7fe640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c13e520 0x7f421c0ab910 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f4210009780 tx=0x7f42100038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:13.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f4218ff9640 1 -- 172.21.15.125:0/2723548925 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210003ea0 con 0x7f421c13e520 2023-10-18T12:00:13.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f4218ff9640 1 -- 172.21.15.125:0/2723548925 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4210034550 con 0x7f421c13e520 2023-10-18T12:00:13.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f4218ff9640 1 -- 172.21.15.125:0/2723548925 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210034960 con 0x7f421c13e520 2023-10-18T12:00:13.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421c0cb560 con 0x7f421c13e520 2023-10-18T12:00:13.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.143+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421c0a8010 con 0x7f421c13e520 2023-10-18T12:00:13.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.144+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421c0a84f0 con 0x7f421c13e520 2023-10-18T12:00:13.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.145+0000 7f4218ff9640 1 -- 172.21.15.125:0/2723548925 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f421003b040 con 0x7f421c13e520 2023-10-18T12:00:13.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.145+0000 7f4218ff9640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f420806af60 0x7f420806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.145+0000 7f4218ff9640 1 -- 172.21.15.125:0/2723548925 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7f42100abf90 con 0x7f421c13e520 2023-10-18T12:00:13.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.146+0000 7f421bfff640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f420806af60 0x7f420806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:13.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.146+0000 7f421bfff640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f420806af60 0x7f420806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f420c0046b0 tx=0x7f420c009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:13.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.150+0000 7f4218ff9640 1 -- 172.21.15.125:0/2723548925 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42100deca0 con 0x7f421c13e520 2023-10-18T12:00:13.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.315+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f421c0a81a0 con 0x7f421c13e520 2023-10-18T12:00:13.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.316+0000 7f4218ff9640 1 -- 172.21.15.125:0/2723548925 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+14481 (secure 0 0 0) 0x7f4210039020 con 0x7f421c13e520 2023-10-18T12:00:13.319 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:13.319 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":212,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:12.015932+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6837/1491435386":"2023-10-19T12:00:09.011366+0000","172.21.15.139:6835/1491435386":"2023-10-19T12:00:09.011366+0000","172.21.15.139:0/3621824170":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6833/3820883843":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6832/3820883843":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2142329879":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/693299469":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/3012481544":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2446385430":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/870215635":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/3819578023":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/330535304":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2863286615":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/1415408500":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/2323488592":"2023-10-19T12:00:06.777556+0000","172.21.15.139:6833/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2575337659":"2023-10-19T11:58:51.730016+0000","172.21.15.139:6832/1565260611":"2023-10-19T11:58:51.730016+0000","172.21.15.139:0/2354007263":"2023-10-19T12:00:06.777556+0000","172.21.15.139:0/1682437":"2023-10-19T11:58:51.730016+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:13.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f420806af60 msgr2=0x7f420806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f420806af60 0x7f420806d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f420c0046b0 tx=0x7f420c009040 comp rx=0 tx=0).stop 2023-10-18T12:00:13.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c13e520 msgr2=0x7f421c0ab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c13e520 0x7f421c0ab910 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f4210009780 tx=0x7f42100038a0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 shutdown_connections 2023-10-18T12:00:13.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f420806af60 0x7f420806d420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f421c141690 0x7f421c1e2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.322 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f421c13e520 0x7f421c0ab910 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 --2- 172.21.15.125:0/2723548925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f421c070920 0x7f421c0a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 >> 172.21.15.125:0/2723548925 conn(0x7f421c069930 msgr2=0x7f421c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:13.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 shutdown_connections 2023-10-18T12:00:13.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.318+0000 7f42225a9640 1 -- 172.21.15.125:0/2723548925 wait complete. 2023-10-18T12:00:13.332 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:00:13.333 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:00:13.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 -- 172.21.15.125:0/3962871497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c15b2a0 msgr2=0x7f6d6c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/3962871497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c15b2a0 0x7f6d6c0cf720 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c08a810 tx=0x7f6d6802bd10 comp rx=0 tx=0).stop 2023-10-18T12:00:13.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 -- 172.21.15.125:0/3962871497 shutdown_connections 2023-10-18T12:00:13.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/3962871497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c15b2a0 0x7f6d6c0cf720 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/3962871497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d6c0ceb40 0x7f6d6c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/3962871497 >> v2:172.21.15.139:3300/0 conn(0x7f6d6c0cf340 0x7f6d6c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 -- 172.21.15.125:0/3962871497 >> 172.21.15.125:0/3962871497 conn(0x7f6d6c0d0d60 msgr2=0x7f6d6c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:13.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 -- 172.21.15.125:0/3962871497 shutdown_connections 2023-10-18T12:00:13.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.438+0000 7f6d73d8a640 1 -- 172.21.15.125:0/3962871497 wait complete. 2023-10-18T12:00:13.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.439+0000 7f6d73d8a640 1 Processor -- start 2023-10-18T12:00:13.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.439+0000 7f6d73d8a640 1 -- start start 2023-10-18T12:00:13.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d73d8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d6c0ceb40 0x7f6d6c1e5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d73d8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c0cf340 0x7f6d6c158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d73d8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d6c15b2a0 0x7f6d6c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d73d8a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c1e65e0 con 0x7f6d6c0cf340 2023-10-18T12:00:13.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d73d8a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c1e59d0 con 0x7f6d6c15b2a0 2023-10-18T12:00:13.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d73d8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c1e5cb0 con 0x7f6d6c0ceb40 2023-10-18T12:00:13.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c0cf340 0x7f6d6c158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:13.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c0cf340 0x7f6d6c158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35270/0 (socket says 172.21.15.125:35270) 2023-10-18T12:00:13.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 -- 172.21.15.125:0/1751654811 learned_addr learned my addr 172.21.15.125:0/1751654811 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:13.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 -- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d6c0ceb40 msgr2=0x7f6d6c1e5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d6c0ceb40 0x7f6d6c1e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 -- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d6c15b2a0 msgr2=0x7f6d6c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d6c15b2a0 0x7f6d6c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 -- 172.21.15.125:0/1751654811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d68009560 con 0x7f6d6c0cf340 2023-10-18T12:00:13.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.440+0000 7f6d71325640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c0cf340 0x7f6d6c158400 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f6d68000c00 tx=0x7f6d6802dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:13.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.441+0000 7f6d627fc640 1 -- 172.21.15.125:0/1751654811 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68003800 con 0x7f6d6c0cf340 2023-10-18T12:00:13.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.441+0000 7f6d627fc640 1 -- 172.21.15.125:0/1751654811 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d68003a50 con 0x7f6d6c0cf340 2023-10-18T12:00:13.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.441+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c1e5f90 con 0x7f6d6c0cf340 2023-10-18T12:00:13.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.441+0000 7f6d627fc640 1 -- 172.21.15.125:0/1751654811 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68003e60 con 0x7f6d6c0cf340 2023-10-18T12:00:13.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.441+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c158940 con 0x7f6d6c0cf340 2023-10-18T12:00:13.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.442+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c0d04f0 con 0x7f6d6c0cf340 2023-10-18T12:00:13.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.442+0000 7f6d627fc640 1 -- 172.21.15.125:0/1751654811 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6d6803e450 con 0x7f6d6c0cf340 2023-10-18T12:00:13.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.443+0000 7f6d627fc640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6d5006af60 0x7f6d5006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.443+0000 7f6d627fc640 1 -- 172.21.15.125:0/1751654811 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7f6d680ac3b0 con 0x7f6d6c0cf340 2023-10-18T12:00:13.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.443+0000 7f6d71b26640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6d5006af60 0x7f6d5006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:13.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.444+0000 7f6d71b26640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6d5006af60 0x7f6d5006d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c005fd0 tx=0x7f6d5c005850 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:13.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.447+0000 7f6d627fc640 1 -- 172.21.15.125:0/1751654811 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d68077260 con 0x7f6d6c0cf340 2023-10-18T12:00:13.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.613+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d6c0690e0 con 0x7f6d6c0cf340 2023-10-18T12:00:13.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.613+0000 7f6d627fc640 1 -- 172.21.15.125:0/1751654811 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v212) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d68042020 con 0x7f6d6c0cf340 2023-10-18T12:00:13.616 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:00:13.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6d5006af60 msgr2=0x7f6d5006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6d5006af60 0x7f6d5006d420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c005fd0 tx=0x7f6d5c005850 comp rx=0 tx=0).stop 2023-10-18T12:00:13.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c0cf340 msgr2=0x7f6d6c158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c0cf340 0x7f6d6c158400 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f6d68000c00 tx=0x7f6d6802dab0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 shutdown_connections 2023-10-18T12:00:13.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6d5006af60 0x7f6d5006d420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d6c15b2a0 0x7f6d6c15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6d6c0cf340 0x7f6d6c158400 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 --2- 172.21.15.125:0/1751654811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d6c0ceb40 0x7f6d6c1e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 >> 172.21.15.125:0/1751654811 conn(0x7f6d6c0d0d60 msgr2=0x7f6d6c15a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:13.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 shutdown_connections 2023-10-18T12:00:13.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.615+0000 7f6d73d8a640 1 -- 172.21.15.125:0/1751654811 wait complete. 2023-10-18T12:00:13.628 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-18T12:00:13.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.729+0000 7f7670765640 1 -- 172.21.15.125:0/346849203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 msgr2=0x7f76680c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.729+0000 7f7670765640 1 --2- 172.21.15.125:0/346849203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 0x7f76680c9b60 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f7658005fd0 tx=0x7f765802bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:13.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.729+0000 7f7670765640 1 -- 172.21.15.125:0/346849203 shutdown_connections 2023-10-18T12:00:13.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.729+0000 7f7670765640 1 --2- 172.21.15.125:0/346849203 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 0x7f76680c9b60 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.729+0000 7f7670765640 1 --2- 172.21.15.125:0/346849203 >> v2:172.21.15.139:3300/0 conn(0x7f766806af00 0x7f766806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.729+0000 7f7670765640 1 -- 172.21.15.125:0/346849203 >> 172.21.15.125:0/346849203 conn(0x7f7668069930 msgr2=0x7f7668069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:13.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.729+0000 7f7670765640 1 -- 172.21.15.125:0/346849203 shutdown_connections 2023-10-18T12:00:13.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.729+0000 7f7670765640 1 -- 172.21.15.125:0/346849203 wait complete. 2023-10-18T12:00:13.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.730+0000 7f7670765640 1 Processor -- start 2023-10-18T12:00:13.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.730+0000 7f7670765640 1 -- start start 2023-10-18T12:00:13.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f7670765640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f766806af00 0x7f76681f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f7670765640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76680ab500 0x7f76680af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f7670765640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 0x7f76680aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f7670765640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76680b06f0 con 0x7f76680c96b0 2023-10-18T12:00:13.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f7670765640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76681f1240 con 0x7f76680ab500 2023-10-18T12:00:13.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f7670765640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76680b0130 con 0x7f766806af00 2023-10-18T12:00:13.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 0x7f76680aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:13.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 0x7f76680aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35280/0 (socket says 172.21.15.125:35280) 2023-10-18T12:00:13.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 -- 172.21.15.125:0/3214070564 learned_addr learned my addr 172.21.15.125:0/3214070564 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:13.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 -- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f766806af00 msgr2=0x7f76681f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f766806af00 0x7f76681f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 -- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76680ab500 msgr2=0x7f76680af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:13.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76680ab500 0x7f76680af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:13.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 -- 172.21.15.125:0/3214070564 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658009560 con 0x7f76680c96b0 2023-10-18T12:00:13.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f766ed02640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 0x7f76680aa8c0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f7660001eb0 tx=0x7f766000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:13.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f765effd640 1 -- 172.21.15.125:0/3214070564 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600052d0 con 0x7f76680c96b0 2023-10-18T12:00:13.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f765effd640 1 -- 172.21.15.125:0/3214070564 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76600035f0 con 0x7f76680c96b0 2023-10-18T12:00:13.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76680b0410 con 0x7f76680c96b0 2023-10-18T12:00:13.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f765effd640 1 -- 172.21.15.125:0/3214070564 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600039e0 con 0x7f76680c96b0 2023-10-18T12:00:13.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.731+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766812d040 con 0x7f76680c96b0 2023-10-18T12:00:13.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.732+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76680690e0 con 0x7f76680c96b0 2023-10-18T12:00:13.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.733+0000 7f765effd640 1 -- 172.21.15.125:0/3214070564 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7660003b40 con 0x7f76680c96b0 2023-10-18T12:00:13.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.733+0000 7f765effd640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f764c06af60 0x7f764c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:13.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.733+0000 7f765effd640 1 -- 172.21.15.125:0/3214070564 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7f76600890a0 con 0x7f76680c96b0 2023-10-18T12:00:13.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.734+0000 7f766e501640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f764c06af60 0x7f764c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:13.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.734+0000 7f766e501640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f764c06af60 0x7f764c06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7664007be0 tx=0x7f7664005f70 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:13.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.737+0000 7f765effd640 1 -- 172.21.15.125:0/3214070564 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7660053f50 con 0x7f76680c96b0 2023-10-18T12:00:13.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:13.965+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f76680cc420 con 0x7f76680c96b0 2023-10-18T12:00:14.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.023+0000 7f765effd640 1 -- 172.21.15.125:0/3214070564 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3086) v1 ==== 189+0+0 (secure 0 0 0) 0x7f7660057570 con 0x7f76680c96b0 2023-10-18T12:00:14.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f764c06af60 msgr2=0x7f764c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f764c06af60 0x7f764c06d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7664007be0 tx=0x7f7664005f70 comp rx=0 tx=0).stop 2023-10-18T12:00:14.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 msgr2=0x7f76680aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 0x7f76680aa8c0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f7660001eb0 tx=0x7f766000c890 comp rx=0 tx=0).stop 2023-10-18T12:00:14.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 shutdown_connections 2023-10-18T12:00:14.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f764c06af60 0x7f764c06d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76680ab500 0x7f76680af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76680c96b0 0x7f76680aa8c0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 --2- 172.21.15.125:0/3214070564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f766806af00 0x7f76681f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 >> 172.21.15.125:0/3214070564 conn(0x7f7668069930 msgr2=0x7f766815a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:14.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 shutdown_connections 2023-10-18T12:00:14.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.025+0000 7f7670765640 1 -- 172.21.15.125:0/3214070564 wait complete. 2023-10-18T12:00:14.038 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T12:00:14.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.138+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1288288428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1a0004d00 msgr2=0x7fa1a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.138+0000 7fa1bb5b0640 1 --2- 172.21.15.125:0/1288288428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1a0004d00 0x7fa1a0005180 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0013d0 tx=0x7fa1a00057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.138+0000 7fa1b9b4d640 1 -- 172.21.15.125:0/1288288428 reap_dead start 2023-10-18T12:00:14.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.138+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1288288428 shutdown_connections 2023-10-18T12:00:14.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.138+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1288288428 >> 172.21.15.125:0/1288288428 conn(0x7fa1b4069930 msgr2=0x7fa1b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:14.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.138+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1288288428 shutdown_connections 2023-10-18T12:00:14.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.138+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1288288428 wait complete. 2023-10-18T12:00:14.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.139+0000 7fa1bb5b0640 1 Processor -- start 2023-10-18T12:00:14.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.139+0000 7fa1bb5b0640 1 -- start start 2023-10-18T12:00:14.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1bb5b0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1b413b060 0x7fa1b414d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1bb5b0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1b41443d0 0x7fa1b4144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1bb5b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1b4157890 0x7fa1b4157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1bb5b0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4144d90 con 0x7fa1b41443d0 2023-10-18T12:00:14.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1bb5b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4159a30 con 0x7fa1b4157890 2023-10-18T12:00:14.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1bb5b0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b413b440 con 0x7fa1b413b060 2023-10-18T12:00:14.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b8b4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1b41443d0 0x7fa1b4144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b8b4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1b41443d0 0x7fa1b4144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35300/0 (socket says 172.21.15.125:35300) 2023-10-18T12:00:14.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b8b4b640 1 -- 172.21.15.125:0/1683879978 learned_addr learned my addr 172.21.15.125:0/1683879978 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:14.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b9b4d640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1b4157890 0x7fa1b4157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b934c640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1b413b060 0x7fa1b414d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b8b4b640 1 -- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1b413b060 msgr2=0x7fa1b414d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b8b4b640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1b413b060 0x7fa1b414d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b8b4b640 1 -- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1b4157890 msgr2=0x7fa1b4157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b8b4b640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1b4157890 0x7fa1b4157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.140+0000 7fa1b8b4b640 1 -- 172.21.15.125:0/1683879978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4158e20 con 0x7fa1b41443d0 2023-10-18T12:00:14.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.141+0000 7fa1b8b4b640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1b41443d0 0x7fa1b4144850 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4003f10 tx=0x7fa1a400b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:14.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.141+0000 7fa1a9ffb640 1 -- 172.21.15.125:0/1683879978 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4005220 con 0x7fa1b41443d0 2023-10-18T12:00:14.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.141+0000 7fa1a9ffb640 1 -- 172.21.15.125:0/1683879978 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1a4015070 con 0x7fa1b41443d0 2023-10-18T12:00:14.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.141+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b4159100 con 0x7fa1b41443d0 2023-10-18T12:00:14.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.141+0000 7fa1a9ffb640 1 -- 172.21.15.125:0/1683879978 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4005600 con 0x7fa1b41443d0 2023-10-18T12:00:14.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.141+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b41596c0 con 0x7fa1b41443d0 2023-10-18T12:00:14.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.142+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b4159880 con 0x7fa1b41443d0 2023-10-18T12:00:14.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.142+0000 7fa1a9ffb640 1 -- 172.21.15.125:0/1683879978 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa1a4017030 con 0x7fa1b41443d0 2023-10-18T12:00:14.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.143+0000 7fa1a9ffb640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa19806af90 0x7fa19806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.143+0000 7fa1a9ffb640 1 -- 172.21.15.125:0/1683879978 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7fa1a4089fc0 con 0x7fa1b41443d0 2023-10-18T12:00:14.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.143+0000 7fa1b934c640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa19806af90 0x7fa19806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.143+0000 7fa1b934c640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa19806af90 0x7fa19806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00058c0 tx=0x7fa1b0005850 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:14.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.147+0000 7fa1a9ffb640 1 -- 172.21.15.125:0/1683879978 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1a40bcca0 con 0x7fa1b41443d0 2023-10-18T12:00:14.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.314+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fa1b4067930 con 0x7fa1b41443d0 2023-10-18T12:00:14.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.315+0000 7fa1a9ffb640 1 -- 172.21.15.125:0/1683879978 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v74) v1 ==== 59+0+2220 (secure 0 0 0) 0x7fa1a4054de0 con 0x7fa1b41443d0 2023-10-18T12:00:14.318 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T12:00:14.318 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T12:00:14.318 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T12:00:14.319 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T12:00:14.319 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T12:00:14.319 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T12:00:14.319 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T12:00:14.320 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T12:00:14.320 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T12:00:14.320 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T12:00:14.320 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T12:00:14.320 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T12:00:14.321 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T12:00:14.321 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T12:00:14.321 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T12:00:14.321 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T12:00:14.322 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T12:00:14.322 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T12:00:14.322 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T12:00:14.322 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T12:00:14.322 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T12:00:14.323 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T12:00:14.323 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T12:00:14.323 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T12:00:14.323 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T12:00:14.323 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T12:00:14.324 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T12:00:14.324 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T12:00:14.324 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T12:00:14.324 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T12:00:14.325 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa19806af90 msgr2=0x7fa19806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa19806af90 0x7fa19806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00058c0 tx=0x7fa1b0005850 comp rx=0 tx=0).stop 2023-10-18T12:00:14.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1b41443d0 msgr2=0x7fa1b4144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1b41443d0 0x7fa1b4144850 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4003f10 tx=0x7fa1a400b980 comp rx=0 tx=0).stop 2023-10-18T12:00:14.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 shutdown_connections 2023-10-18T12:00:14.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa19806af90 0x7fa19806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1b4157890 0x7fa1b4157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa1b41443d0 0x7fa1b4144850 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 --2- 172.21.15.125:0/1683879978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1b413b060 0x7fa1b414d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 >> 172.21.15.125:0/1683879978 conn(0x7fa1b4069930 msgr2=0x7fa1b41515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:14.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 shutdown_connections 2023-10-18T12:00:14.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.317+0000 7fa1bb5b0640 1 -- 172.21.15.125:0/1683879978 wait complete. 2023-10-18T12:00:14.330 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T12:00:14.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.434+0000 7f5c14e61640 1 -- 172.21.15.125:0/2876199380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c1013d350 msgr2=0x7f5c100cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.434+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2876199380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c1013d350 0x7f5c100cbd60 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009830 tx=0x7f5c0402bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:14.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.434+0000 7f5c14e61640 1 -- 172.21.15.125:0/2876199380 shutdown_connections 2023-10-18T12:00:14.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.434+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2876199380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c10070350 0x7f5c100707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.434+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2876199380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c1013d350 0x7f5c100cbd60 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.434+0000 7f5c14e61640 1 -- 172.21.15.125:0/2876199380 >> 172.21.15.125:0/2876199380 conn(0x7f5c10069930 msgr2=0x7f5c10069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:14.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.434+0000 7f5c14e61640 1 -- 172.21.15.125:0/2876199380 shutdown_connections 2023-10-18T12:00:14.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.434+0000 7f5c14e61640 1 -- 172.21.15.125:0/2876199380 wait complete. 2023-10-18T12:00:14.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.435+0000 7f5c14e61640 1 Processor -- start 2023-10-18T12:00:14.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.435+0000 7f5c14e61640 1 -- start start 2023-10-18T12:00:14.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c14e61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c10070350 0x7f5c10136280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c14e61640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c100cb980 0x7f5c101513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c14e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c100b16d0 0x7f5c100ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c14e61640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101367c0 con 0x7f5c100cb980 2023-10-18T12:00:14.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c14e61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101548c0 con 0x7f5c100b16d0 2023-10-18T12:00:14.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c14e61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10153820 con 0x7f5c10070350 2023-10-18T12:00:14.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c100cb980 0x7f5c101513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c100cb980 0x7f5c101513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35304/0 (socket says 172.21.15.125:35304) 2023-10-18T12:00:14.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0dd9b640 1 -- 172.21.15.125:0/2479894668 learned_addr learned my addr 172.21.15.125:0/2479894668 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:14.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0ed9d640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c100b16d0 0x7f5c100ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0e59c640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c10070350 0x7f5c10136280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0dd9b640 1 -- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c10070350 msgr2=0x7f5c10136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0dd9b640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c10070350 0x7f5c10136280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0dd9b640 1 -- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c100b16d0 msgr2=0x7f5c100ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0dd9b640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c100b16d0 0x7f5c100ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.436+0000 7f5c0dd9b640 1 -- 172.21.15.125:0/2479894668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009490 con 0x7f5c100cb980 2023-10-18T12:00:14.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.437+0000 7f5c0dd9b640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c100cb980 0x7f5c101513a0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000c00 tx=0x7f5c040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:14.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.437+0000 7f5bfeffd640 1 -- 172.21.15.125:0/2479894668 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04003ea0 con 0x7f5c100cb980 2023-10-18T12:00:14.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.437+0000 7f5bfeffd640 1 -- 172.21.15.125:0/2479894668 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c04034440 con 0x7f5c100cb980 2023-10-18T12:00:14.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.437+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c10153b00 con 0x7f5c100cb980 2023-10-18T12:00:14.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.437+0000 7f5bfeffd640 1 -- 172.21.15.125:0/2479894668 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04034850 con 0x7f5c100cb980 2023-10-18T12:00:14.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.437+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c101540c0 con 0x7f5c100cb980 2023-10-18T12:00:14.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.438+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c100690e0 con 0x7f5c100cb980 2023-10-18T12:00:14.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.438+0000 7f5bfeffd640 1 -- 172.21.15.125:0/2479894668 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c040349b0 con 0x7f5c100cb980 2023-10-18T12:00:14.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.439+0000 7f5bfeffd640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f5bf006af60 0x7f5bf006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.439+0000 7f5bfeffd640 1 -- 172.21.15.125:0/2479894668 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7f5c040ac000 con 0x7f5c100cb980 2023-10-18T12:00:14.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.439+0000 7f5c0e59c640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f5bf006af60 0x7f5bf006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.439+0000 7f5c0e59c640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f5bf006af60 0x7f5bf006d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c000056a0 tx=0x7f5c00007e50 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:14.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.443+0000 7f5bfeffd640 1 -- 172.21.15.125:0/2479894668 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c04076f60 con 0x7f5c100cb980 2023-10-18T12:00:14.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.608+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5c1006ff80 con 0x7f5c100cb980 2023-10-18T12:00:14.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.612+0000 7f5bfeffd640 1 -- 172.21.15.125:0/2479894668 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v75)=0 v75) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c0407a580 con 0x7f5c100cb980 2023-10-18T12:00:14.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f5bf006af60 msgr2=0x7f5bf006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f5bf006af60 0x7f5bf006d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c000056a0 tx=0x7f5c00007e50 comp rx=0 tx=0).stop 2023-10-18T12:00:14.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c100cb980 msgr2=0x7f5c101513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c100cb980 0x7f5c101513a0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000c00 tx=0x7f5c040038a0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 shutdown_connections 2023-10-18T12:00:14.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f5bf006af60 0x7f5bf006d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c100b16d0 0x7f5c100ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c100cb980 0x7f5c101513a0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 --2- 172.21.15.125:0/2479894668 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c10070350 0x7f5c10136280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 >> 172.21.15.125:0/2479894668 conn(0x7f5c10069930 msgr2=0x7f5c1006ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:14.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 shutdown_connections 2023-10-18T12:00:14.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.614+0000 7f5c14e61640 1 -- 172.21.15.125:0/2479894668 wait complete. 2023-10-18T12:00:14.627 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T12:00:14.627 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:00:14.673 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:00:14.721 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T12:00:14.722 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:00:14.722 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:14.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:00:14.737 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:00:14.738 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:14.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:00:14.756 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:00:14.756 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:14.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:00:14.793 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:00:14.794 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:14.812 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:00:14.812 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T12:00:14.812 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:14.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.931+0000 7fc70b597640 1 -- 172.21.15.125:0/573508472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7040cf340 msgr2=0x7fc7040d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.931+0000 7fc70b597640 1 --2- 172.21.15.125:0/573508472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7040cf340 0x7fc7040d1910 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fc700005fd0 tx=0x7fc70002c050 comp rx=0 tx=0).stop 2023-10-18T12:00:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.931+0000 7fc70b597640 1 -- 172.21.15.125:0/573508472 shutdown_connections 2023-10-18T12:00:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.931+0000 7fc70b597640 1 --2- 172.21.15.125:0/573508472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc70415b2a0 0x7fc7040cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.931+0000 7fc70b597640 1 --2- 172.21.15.125:0/573508472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc7040cf340 0x7fc7040d1910 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.931+0000 7fc70b597640 1 -- 172.21.15.125:0/573508472 >> 172.21.15.125:0/573508472 conn(0x7fc7040d0d60 msgr2=0x7fc7040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:14.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.931+0000 7fc70b597640 1 -- 172.21.15.125:0/573508472 shutdown_connections 2023-10-18T12:00:14.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.931+0000 7fc70b597640 1 -- 172.21.15.125:0/573508472 wait complete. 2023-10-18T12:00:14.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.932+0000 7fc70b597640 1 Processor -- start 2023-10-18T12:00:14.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.932+0000 7fc70b597640 1 -- start start 2023-10-18T12:00:14.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc70b597640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7040ceb40 0x7fc70415e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc70b597640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7040cf340 0x7fc7040ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc70b597640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc70415b2a0 0x7fc7040b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc70b597640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040caed0 con 0x7fc70415b2a0 2023-10-18T12:00:14.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc70b597640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040b3980 con 0x7fc7040cf340 2023-10-18T12:00:14.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc70b597640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040b3c20 con 0x7fc7040ceb40 2023-10-18T12:00:14.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc709b34640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc70415b2a0 0x7fc7040b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc709b34640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc70415b2a0 0x7fc7040b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35312/0 (socket says 172.21.15.125:35312) 2023-10-18T12:00:14.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.933+0000 7fc709b34640 1 -- 172.21.15.125:0/549606679 learned_addr learned my addr 172.21.15.125:0/549606679 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:14.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc709333640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7040ceb40 0x7fc70415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc709b34640 1 -- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7040ceb40 msgr2=0x7fc70415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc709b34640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7040ceb40 0x7fc70415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc709b34640 1 -- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7040cf340 msgr2=0x7fc7040ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:14.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc709b34640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7040cf340 0x7fc7040ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:14.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc709b34640 1 -- 172.21.15.125:0/549606679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc700009840 con 0x7fc70415b2a0 2023-10-18T12:00:14.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc709b34640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc70415b2a0 0x7fc7040b53e0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00c430 tx=0x7fc6fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:14.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc6f9ffb640 1 -- 172.21.15.125:0/549606679 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0035f0 con 0x7fc70415b2a0 2023-10-18T12:00:14.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc6f9ffb640 1 -- 172.21.15.125:0/549606679 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc6fc003770 con 0x7fc70415b2a0 2023-10-18T12:00:14.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc6f9ffb640 1 -- 172.21.15.125:0/549606679 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc003b80 con 0x7fc70415b2a0 2023-10-18T12:00:14.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7040b3f00 con 0x7fc70415b2a0 2023-10-18T12:00:14.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.934+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7040cb150 con 0x7fc70415b2a0 2023-10-18T12:00:14.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.935+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7040b4420 con 0x7fc70415b2a0 2023-10-18T12:00:14.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.935+0000 7fc6f9ffb640 1 -- 172.21.15.125:0/549606679 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc6fc003ce0 con 0x7fc70415b2a0 2023-10-18T12:00:14.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.936+0000 7fc6f9ffb640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc6e806af60 0x7fc6e806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:14.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.936+0000 7fc6f9ffb640 1 -- 172.21.15.125:0/549606679 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7fc6fc0890f0 con 0x7fc70415b2a0 2023-10-18T12:00:14.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.936+0000 7fc709333640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc6e806af60 0x7fc6e806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:14.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.936+0000 7fc709333640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc6e806af60 0x7fc6e806d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc700035000 tx=0x7fc700004ab0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:14.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:14.940+0000 7fc6f9ffb640 1 -- 172.21.15.125:0/549606679 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7040b4420 con 0x7fc70415b2a0 2023-10-18T12:00:15.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.145+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7040b4420 con 0x7fc70415b2a0 2023-10-18T12:00:15.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.145+0000 7fc6f9ffb640 1 -- 172.21.15.125:0/549606679 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3713 (secure 0 0 0) 0x7fc7040b4420 con 0x7fc70415b2a0 2023-10-18T12:00:15.148 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:15.149 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":82,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8900,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/4133875989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4133875989},{"type":"v1","addr":"172.21.15.139:6837","nonce":4133875989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":9751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2853562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2853562772},{"type":"v1","addr":"172.21.15.139:6836","nonce":2853562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":11421,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3995754374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3995754374},{"type":"v1","addr":"172.21.15.125:6834","nonce":3995754374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":11424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2110336778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2110336778},{"type":"v1","addr":"172.21.15.125:6835","nonce":2110336778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[]} 2023-10-18T12:00:15.149 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 82 2023-10-18T12:00:15.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc6e806af60 msgr2=0x7fc6e806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:15.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc6e806af60 0x7fc6e806d420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc700035000 tx=0x7fc700004ab0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc70415b2a0 msgr2=0x7fc7040b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:15.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc70415b2a0 0x7fc7040b53e0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00c430 tx=0x7fc6fc00c8f0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 shutdown_connections 2023-10-18T12:00:15.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc6e806af60 0x7fc6e806d420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7040cf340 0x7fc7040ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc70415b2a0 0x7fc7040b53e0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 --2- 172.21.15.125:0/549606679 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7040ceb40 0x7fc70415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 >> 172.21.15.125:0/549606679 conn(0x7fc7040d0d60 msgr2=0x7fc7040b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:15.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 shutdown_connections 2023-10-18T12:00:15.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.147+0000 7fc70b597640 1 -- 172.21.15.125:0/549606679 wait complete. 2023-10-18T12:00:15.161 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T12:00:15.161 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T12:00:15.161 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T12:00:15.161 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:00:15.162 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T12:00:15.162 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T12:00:15.162 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T12:00:15.163 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:00:15.163 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T12:00:15.163 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T12:00:15.163 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T12:00:15.164 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:00:15.164 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T12:00:15.164 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T12:00:15.164 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T12:00:15.165 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:00:15.170 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T12:00:15.171 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T12:00:15.172 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T12:00:15.172 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T12:00:15.175 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T12:00:15.176 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T12:00:15.177 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T12:00:15.177 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T12:00:15.214 INFO:tasks.ceph.mds.a:Started 2023-10-18T12:00:15.215 DEBUG:teuthology.parallel:result is None 2023-10-18T12:00:15.216 INFO:tasks.ceph.mds.c:Started 2023-10-18T12:00:15.216 DEBUG:teuthology.parallel:result is None 2023-10-18T12:00:15.219 INFO:tasks.ceph.mds.b:Started 2023-10-18T12:00:15.219 DEBUG:teuthology.parallel:result is None 2023-10-18T12:00:15.220 INFO:tasks.ceph.mds.d:Started 2023-10-18T12:00:15.220 DEBUG:teuthology.parallel:result is None 2023-10-18T12:00:15.221 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T12:00:15.289 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T12:00:15.290 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T12:00:15.300 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T12:00:15.301 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T12:00:15.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.325+0000 7f6889621640 1 -- 172.21.15.125:0/1471770852 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68840cf340 msgr2=0x7f68840d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:15.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.325+0000 7f6889621640 1 --2- 172.21.15.125:0/1471770852 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68840cf340 0x7f68840d1910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6874009860 tx=0x7f687402bb30 comp rx=0 tx=0).stop 2023-10-18T12:00:15.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.325+0000 7f6889621640 1 -- 172.21.15.125:0/1471770852 shutdown_connections 2023-10-18T12:00:15.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.325+0000 7f6889621640 1 --2- 172.21.15.125:0/1471770852 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68840cf340 0x7f68840d1910 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.325+0000 7f6889621640 1 -- 172.21.15.125:0/1471770852 >> 172.21.15.125:0/1471770852 conn(0x7f68840d0d60 msgr2=0x7f68840d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:15.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.325+0000 7f6889621640 1 -- 172.21.15.125:0/1471770852 shutdown_connections 2023-10-18T12:00:15.329 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.325+0000 7f6889621640 1 -- 172.21.15.125:0/1471770852 wait complete. 2023-10-18T12:00:15.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.326+0000 7f6889621640 1 Processor -- start 2023-10-18T12:00:15.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.326+0000 7f6889621640 1 -- start start 2023-10-18T12:00:15.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6889621640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68840ceb40 0x7f68841f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:15.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6889621640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68840cf340 0x7f68840ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:15.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6889621640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688415b2a0 0x7f68841f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:15.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6889621640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68840cae70 con 0x7f68840ceb40 2023-10-18T12:00:15.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6889621640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68840c7fd0 con 0x7f68840cf340 2023-10-18T12:00:15.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6889621640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68840cab90 con 0x7f688415b2a0 2023-10-18T12:00:15.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6882ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68840ceb40 0x7f68841f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:15.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6882ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68840ceb40 0x7f68841f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35342/0 (socket says 172.21.15.125:35342) 2023-10-18T12:00:15.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6882ffd640 1 -- 172.21.15.125:0/2671993766 learned_addr learned my addr 172.21.15.125:0/2671993766 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:15.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f687a7fc640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68840cf340 0x7f68840ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:15.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f68837fe640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688415b2a0 0x7f68841f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:15.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6882ffd640 1 -- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688415b2a0 msgr2=0x7f68841f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:15.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6882ffd640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688415b2a0 0x7f68841f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6882ffd640 1 -- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68840cf340 msgr2=0x7f68840ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:15.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6882ffd640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68840cf340 0x7f68840ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:15.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.327+0000 7f6882ffd640 1 -- 172.21.15.125:0/2671993766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6874009490 con 0x7f68840ceb40 2023-10-18T12:00:15.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.328+0000 7f6882ffd640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68840ceb40 0x7f68841f2990 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f6874009980 tx=0x7f687402dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:15.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.328+0000 7f687bfff640 1 -- 172.21.15.125:0/2671993766 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687403b040 con 0x7f68840ceb40 2023-10-18T12:00:15.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.328+0000 7f687bfff640 1 -- 172.21.15.125:0/2671993766 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6874005c90 con 0x7f68840ceb40 2023-10-18T12:00:15.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.328+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68840c8270 con 0x7f68840ceb40 2023-10-18T12:00:15.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.328+0000 7f687bfff640 1 -- 172.21.15.125:0/2671993766 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874005260 con 0x7f68840ceb40 2023-10-18T12:00:15.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.328+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68840ca630 con 0x7f68840ceb40 2023-10-18T12:00:15.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.329+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68840c7e10 con 0x7f68840ceb40 2023-10-18T12:00:15.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.329+0000 7f687bfff640 1 -- 172.21.15.125:0/2671993766 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f68740053c0 con 0x7f68840ceb40 2023-10-18T12:00:15.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.330+0000 7f687bfff640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f686406af60 0x7f686406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:15.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.330+0000 7f687bfff640 1 -- 172.21.15.125:0/2671993766 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6093+0+0 (secure 0 0 0) 0x7f68740ab7d0 con 0x7f68840ceb40 2023-10-18T12:00:15.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.330+0000 7f687a7fc640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f686406af60 0x7f686406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:15.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.330+0000 7f687a7fc640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f686406af60 0x7f686406d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68840c78b0 tx=0x7f686c001a40 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:15.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.334+0000 7f687bfff640 1 -- 172.21.15.125:0/2671993766 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6874076590 con 0x7f68840ceb40 2023-10-18T12:00:15.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:15.512+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f688415ade0 con 0x7f68840ceb40 2023-10-18T12:00:16.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.624+0000 7f687bfff640 1 -- 172.21.15.125:0/2671993766 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v217) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6874043080 con 0x7f68840ceb40 2023-10-18T12:00:16.627 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T12:00:16.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f686406af60 msgr2=0x7f686406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:16.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f686406af60 0x7f686406d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68840c78b0 tx=0x7f686c001a40 comp rx=0 tx=0).stop 2023-10-18T12:00:16.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68840ceb40 msgr2=0x7f68841f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:16.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68840ceb40 0x7f68841f2990 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f6874009980 tx=0x7f687402dce0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 shutdown_connections 2023-10-18T12:00:16.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f686406af60 0x7f686406d420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68840cf340 0x7f68840ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688415b2a0 0x7f68841f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 --2- 172.21.15.125:0/2671993766 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68840ceb40 0x7f68841f2990 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 >> 172.21.15.125:0/2671993766 conn(0x7f68840d0d60 msgr2=0x7f688415a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:16.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 shutdown_connections 2023-10-18T12:00:16.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.626+0000 7f6889621640 1 -- 172.21.15.125:0/2671993766 wait complete. 2023-10-18T12:00:16.641 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T12:00:16.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.743+0000 7f992c610640 1 -- 172.21.15.125:0/2540645835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240c96b0 msgr2=0x7f99240c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:16.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.743+0000 7f992c610640 1 --2- 172.21.15.125:0/2540645835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240c96b0 0x7f99240c9b30 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f9920009900 tx=0x7f992002bd70 comp rx=0 tx=0).stop 2023-10-18T12:00:16.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.743+0000 7f992c610640 1 -- 172.21.15.125:0/2540645835 shutdown_connections 2023-10-18T12:00:16.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.743+0000 7f992c610640 1 --2- 172.21.15.125:0/2540645835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240c96b0 0x7f99240c9b30 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.743+0000 7f992c610640 1 --2- 172.21.15.125:0/2540645835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f992406af00 0x7f992406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.743+0000 7f992c610640 1 -- 172.21.15.125:0/2540645835 >> 172.21.15.125:0/2540645835 conn(0x7f9924069930 msgr2=0x7f9924069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:16.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.743+0000 7f992c610640 1 -- 172.21.15.125:0/2540645835 shutdown_connections 2023-10-18T12:00:16.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.744+0000 7f992c610640 1 -- 172.21.15.125:0/2540645835 wait complete. 2023-10-18T12:00:16.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f992c610640 1 Processor -- start 2023-10-18T12:00:16.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f992c610640 1 -- start start 2023-10-18T12:00:16.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f992c610640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992406af00 0x7f992413a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:16.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f992c610640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240ab500 0x7f99240acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:16.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f992c610640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99240c96b0 0x7f99240ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:16.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f992c610640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992413a770 con 0x7f99240ab500 2023-10-18T12:00:16.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f992c610640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240aa5b0 con 0x7f992406af00 2023-10-18T12:00:16.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f992c610640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99240aa890 con 0x7f99240c96b0 2023-10-18T12:00:16.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f9929bab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240ab500 0x7f99240acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:16.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f9929bab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240ab500 0x7f99240acc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35356/0 (socket says 172.21.15.125:35356) 2023-10-18T12:00:16.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.745+0000 7f9929bab640 1 -- 172.21.15.125:0/2635893556 learned_addr learned my addr 172.21.15.125:0/2635893556 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:16.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f992abad640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99240c96b0 0x7f99240ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:16.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f9929bab640 1 -- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99240c96b0 msgr2=0x7f99240ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:16.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f992a3ac640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992406af00 0x7f992413a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:16.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f9929bab640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99240c96b0 0x7f99240ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f9929bab640 1 -- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992406af00 msgr2=0x7f992413a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:16.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f9929bab640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992406af00 0x7f992413a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:16.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f9929bab640 1 -- 172.21.15.125:0/2635893556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9920009560 con 0x7f99240ab500 2023-10-18T12:00:16.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f992abad640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99240c96b0 0x7f99240ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:16.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f992a3ac640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992406af00 0x7f992413a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:16.754 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f9929bab640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240ab500 0x7f99240acc30 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f9920036040 tx=0x7f992002daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:16.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f991affd640 1 -- 172.21.15.125:0/2635893556 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99200035f0 con 0x7f99240ab500 2023-10-18T12:00:16.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f991affd640 1 -- 172.21.15.125:0/2635893556 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9920003840 con 0x7f99240ab500 2023-10-18T12:00:16.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99240aab70 con 0x7f99240ab500 2023-10-18T12:00:16.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f991affd640 1 -- 172.21.15.125:0/2635893556 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920003bf0 con 0x7f99240ab500 2023-10-18T12:00:16.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.746+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99240ab130 con 0x7f99240ab500 2023-10-18T12:00:16.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.747+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99240aa9d0 con 0x7f99240ab500 2023-10-18T12:00:16.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.747+0000 7f991affd640 1 -- 172.21.15.125:0/2635893556 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9920003d50 con 0x7f99240ab500 2023-10-18T12:00:16.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.748+0000 7f991affd640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f990806af60 0x7f990806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:16.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.748+0000 7f991affd640 1 -- 172.21.15.125:0/2635893556 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f99200ac350 con 0x7f99240ab500 2023-10-18T12:00:16.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.748+0000 7f992a3ac640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f990806af60 0x7f990806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:16.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.749+0000 7f992a3ac640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f990806af60 0x7f990806d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99140058c0 tx=0x7f9914005d30 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:16.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.752+0000 7f991affd640 1 -- 172.21.15.125:0/2635893556 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99240aa9d0 con 0x7f99240ab500 2023-10-18T12:00:16.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.979+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f99240aa9d0 con 0x7f99240ab500 2023-10-18T12:00:16.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.980+0000 7f991affd640 1 -- 172.21.15.125:0/2635893556 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v56) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f99240aa9d0 con 0x7f99240ab500 2023-10-18T12:00:16.983 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:16.983 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:00:16.983 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T12:00:16.984 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.984 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T12:00:16.984 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T12:00:16.984 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.984 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:00:16.985 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:00:16.985 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:00:16.985 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:16.985 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:16.985 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:16.986 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.986 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T12:00:16.986 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T12:00:16.986 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.986 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:00:16.987 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:00:16.987 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:00:16.987 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:16.987 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:16.987 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:16.987 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.988 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T12:00:16.988 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T12:00:16.988 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.988 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:00:16.988 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:00:16.989 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:00:16.989 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:16.989 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:16.989 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:16.989 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.990 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T12:00:16.990 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T12:00:16.990 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.990 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:00:16.990 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:00:16.991 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:00:16.991 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:16.991 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:16.991 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:16.991 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.992 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T12:00:16.992 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T12:00:16.992 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.993 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:00:16.993 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:00:16.993 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:16.993 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:16.993 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:16.994 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.994 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T12:00:16.994 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T12:00:16.994 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.994 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:00:16.995 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:00:16.995 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:16.995 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:16.995 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:16.995 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.995 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T12:00:16.996 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T12:00:16.996 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.996 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:00:16.996 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:00:16.996 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:16.997 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:16.997 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:16.997 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.997 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T12:00:16.997 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T12:00:16.998 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.998 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:00:16.998 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:00:16.998 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:16.998 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:16.999 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:16.999 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:16.999 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T12:00:16.999 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T12:00:16.999 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:16.999 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:00:17.000 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:00:17.000 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:17.000 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.000 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.001 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.002 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T12:00:17.002 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T12:00:17.002 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.003 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:00:17.003 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:00:17.003 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:17.004 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.004 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.004 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.005 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T12:00:17.005 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T12:00:17.005 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.006 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:00:17.006 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:00:17.007 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:17.007 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.007 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.008 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.008 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T12:00:17.008 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T12:00:17.008 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.009 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:00:17.009 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:00:17.009 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:17.009 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.009 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.010 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.010 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T12:00:17.010 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T12:00:17.010 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.010 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:00:17.011 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T12:00:17.011 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T12:00:17.011 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.011 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.011 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.012 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T12:00:17.012 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T12:00:17.012 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.012 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:00:17.012 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:00:17.013 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:00:17.013 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:00:17.013 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.013 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.013 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.014 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T12:00:17.014 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T12:00:17.014 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.014 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:00:17.014 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:00:17.015 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:00:17.015 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:00:17.015 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.015 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.015 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.016 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T12:00:17.016 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T12:00:17.016 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.016 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:00:17.016 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:00:17.017 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:00:17.017 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:00:17.017 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.017 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.017 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.018 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T12:00:17.018 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T12:00:17.018 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.018 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:00:17.018 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:00:17.019 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:00:17.019 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:17.019 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.019 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.020 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.020 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T12:00:17.020 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T12:00:17.020 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.021 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:00:17.021 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T12:00:17.021 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:17.021 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.022 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:17.022 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.022 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T12:00:17.022 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T12:00:17.022 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:00:17.023 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:00:17.023 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T12:00:17.023 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:00:17.023 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.023 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.024 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T12:00:17.024 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:00:17.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f990806af60 msgr2=0x7f990806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f990806af60 0x7f990806d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99140058c0 tx=0x7f9914005d30 comp rx=0 tx=0).stop 2023-10-18T12:00:17.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240ab500 msgr2=0x7f99240acc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240ab500 0x7f99240acc30 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f9920036040 tx=0x7f992002daf0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 shutdown_connections 2023-10-18T12:00:17.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f990806af60 0x7f990806d420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99240c96b0 0x7f99240ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f99240ab500 0x7f99240acc30 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 --2- 172.21.15.125:0/2635893556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992406af00 0x7f992413a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 >> 172.21.15.125:0/2635893556 conn(0x7f9924069930 msgr2=0x7f9924069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:17.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 shutdown_connections 2023-10-18T12:00:17.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:16.982+0000 7f992c610640 1 -- 172.21.15.125:0/2635893556 wait complete. 2023-10-18T12:00:17.029 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:17.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.129+0000 7fb828639640 1 -- 172.21.15.125:0/4137371796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb80c0047c0 msgr2=0x7fb80c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.129+0000 7fb828639640 1 --2- 172.21.15.125:0/4137371796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb80c0047c0 0x7fb80c004c40 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fb81000bb20 tx=0x7fb80c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.129+0000 7fb826bd6640 1 -- 172.21.15.125:0/4137371796 reap_dead start 2023-10-18T12:00:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.130+0000 7fb828639640 1 -- 172.21.15.125:0/4137371796 shutdown_connections 2023-10-18T12:00:17.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.130+0000 7fb828639640 1 -- 172.21.15.125:0/4137371796 >> 172.21.15.125:0/4137371796 conn(0x7fb820069930 msgr2=0x7fb820069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:17.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.130+0000 7fb828639640 1 -- 172.21.15.125:0/4137371796 shutdown_connections 2023-10-18T12:00:17.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.130+0000 7fb828639640 1 -- 172.21.15.125:0/4137371796 wait complete. 2023-10-18T12:00:17.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.131+0000 7fb828639640 1 Processor -- start 2023-10-18T12:00:17.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.131+0000 7fb828639640 1 -- start start 2023-10-18T12:00:17.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.131+0000 7fb828639640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb80c0047c0 0x7fb8200a8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb828639640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8200aa750 0x7fb8201df9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb828639640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8201dff30 0x7fb82013ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb828639640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201e09a0 con 0x7fb8200aa750 2023-10-18T12:00:17.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb828639640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200a6650 con 0x7fb8201dff30 2023-10-18T12:00:17.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb828639640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201e03e0 con 0x7fb80c0047c0 2023-10-18T12:00:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8200aa750 0x7fb8201df9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8200aa750 0x7fb8201df9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35372/0 (socket says 172.21.15.125:35372) 2023-10-18T12:00:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 -- 172.21.15.125:0/4106806309 learned_addr learned my addr 172.21.15.125:0/4106806309 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb8263d5640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb80c0047c0 0x7fb8200a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb826bd6640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8201dff30 0x7fb82013ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 -- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb80c0047c0 msgr2=0x7fb8200a8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb80c0047c0 0x7fb8200a8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 -- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8201dff30 msgr2=0x7fb82013ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8201dff30 0x7fb82013ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 -- 172.21.15.125:0/4106806309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8201e06c0 con 0x7fb8200aa750 2023-10-18T12:00:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb826bd6640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8201dff30 0x7fb82013ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb825bd4640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8200aa750 0x7fb8201df9f0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fb81000bb20 tx=0x7fb810036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb816ffd640 1 -- 172.21.15.125:0/4106806309 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810036d20 con 0x7fb8200aa750 2023-10-18T12:00:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb816ffd640 1 -- 172.21.15.125:0/4106806309 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb810036e80 con 0x7fb8200aa750 2023-10-18T12:00:17.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb816ffd640 1 -- 172.21.15.125:0/4106806309 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810004290 con 0x7fb8200aa750 2023-10-18T12:00:17.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8200a68f0 con 0x7fb8200aa750 2023-10-18T12:00:17.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.132+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8200a8d70 con 0x7fb8200aa750 2023-10-18T12:00:17.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.133+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8200a8f60 con 0x7fb8200aa750 2023-10-18T12:00:17.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.134+0000 7fb816ffd640 1 -- 172.21.15.125:0/4106806309 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb8100035f0 con 0x7fb8200aa750 2023-10-18T12:00:17.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.134+0000 7fb816ffd640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb80406af90 0x7fb80406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.134+0000 7fb816ffd640 1 -- 172.21.15.125:0/4106806309 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fb8100ab490 con 0x7fb8200aa750 2023-10-18T12:00:17.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.134+0000 7fb8263d5640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb80406af90 0x7fb80406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.135+0000 7fb8263d5640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb80406af90 0x7fb80406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb81c003fd0 tx=0x7fb81c003d10 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:17.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.138+0000 7fb816ffd640 1 -- 172.21.15.125:0/4106806309 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8100766b0 con 0x7fb8200aa750 2023-10-18T12:00:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.306+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8200a64b0 con 0x7fb8200aa750 2023-10-18T12:00:17.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.307+0000 7fb816ffd640 1 -- 172.21.15.125:0/4106806309 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+13210 (secure 0 0 0) 0x7fb8200a64b0 con 0x7fb8200aa750 2023-10-18T12:00:17.310 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:17.311 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":217,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:16.623528+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:17.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb80406af90 msgr2=0x7fb80406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb80406af90 0x7fb80406d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb81c003fd0 tx=0x7fb81c003d10 comp rx=0 tx=0).stop 2023-10-18T12:00:17.313 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8200aa750 msgr2=0x7fb8201df9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8200aa750 0x7fb8201df9f0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fb81000bb20 tx=0x7fb810036720 comp rx=0 tx=0).stop 2023-10-18T12:00:17.314 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 shutdown_connections 2023-10-18T12:00:17.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb80406af90 0x7fb80406d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8201dff30 0x7fb82013ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8200aa750 0x7fb8201df9f0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.318 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 --2- 172.21.15.125:0/4106806309 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb80c0047c0 0x7fb8200a8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 >> 172.21.15.125:0/4106806309 conn(0x7fb820069930 msgr2=0x7fb82013be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:17.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 shutdown_connections 2023-10-18T12:00:17.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.309+0000 7fb828639640 1 -- 172.21.15.125:0/4106806309 wait complete. 2023-10-18T12:00:17.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:00:17.324 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:00:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.425+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/1754213818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9806af00 msgr2=0x7f0d9806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.425+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/1754213818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9806af00 0x7f0d9806b2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009860 tx=0x7f0d8802bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:17.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.425+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/1754213818 shutdown_connections 2023-10-18T12:00:17.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.425+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/1754213818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d980abc10 0x7f0d980c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.425+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/1754213818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9806af00 0x7f0d9806b2e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.425+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/1754213818 >> 172.21.15.125:0/1754213818 conn(0x7f0d98069930 msgr2=0x7f0d98069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.425+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/1754213818 shutdown_connections 2023-10-18T12:00:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.425+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/1754213818 wait complete. 2023-10-18T12:00:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.426+0000 7f0d9dfa9640 1 Processor -- start 2023-10-18T12:00:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d9dfa9640 1 -- start start 2023-10-18T12:00:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d9dfa9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d9806af00 0x7f0d980a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d9dfa9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d980abc10 0x7f0d98158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d9dfa9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d980c96b0 0x7f0d980aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d9dfa9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9812bda0 con 0x7f0d980abc10 2023-10-18T12:00:17.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d9dfa9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9812b190 con 0x7f0d980c96b0 2023-10-18T12:00:17.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d9dfa9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9812b470 con 0x7f0d9806af00 2023-10-18T12:00:17.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d96ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d980abc10 0x7f0d98158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d96ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d980abc10 0x7f0d98158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35384/0 (socket says 172.21.15.125:35384) 2023-10-18T12:00:17.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.427+0000 7f0d96ffd640 1 -- 172.21.15.125:0/3372397312 learned_addr learned my addr 172.21.15.125:0/3372397312 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:17.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d97fff640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d980c96b0 0x7f0d980aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d977fe640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d9806af00 0x7f0d980a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d96ffd640 1 -- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d9806af00 msgr2=0x7f0d980a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d96ffd640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d9806af00 0x7f0d980a7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d96ffd640 1 -- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d980c96b0 msgr2=0x7f0d980aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d96ffd640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d980c96b0 0x7f0d980aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d96ffd640 1 -- 172.21.15.125:0/3372397312 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88009490 con 0x7f0d980abc10 2023-10-18T12:00:17.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d96ffd640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d980abc10 0x7f0d98158bc0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00c3c0 tx=0x7f0d8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:17.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d7ffff640 1 -- 172.21.15.125:0/3372397312 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c011790 con 0x7f0d980abc10 2023-10-18T12:00:17.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d7ffff640 1 -- 172.21.15.125:0/3372397312 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d8c0119e0 con 0x7f0d980abc10 2023-10-18T12:00:17.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9812b750 con 0x7f0d980abc10 2023-10-18T12:00:17.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d7ffff640 1 -- 172.21.15.125:0/3372397312 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c011e10 con 0x7f0d980abc10 2023-10-18T12:00:17.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.428+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d981312e0 con 0x7f0d980abc10 2023-10-18T12:00:17.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.429+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d98131500 con 0x7f0d980abc10 2023-10-18T12:00:17.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.430+0000 7f0d7ffff640 1 -- 172.21.15.125:0/3372397312 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0d8c01c460 con 0x7f0d980abc10 2023-10-18T12:00:17.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.430+0000 7f0d7ffff640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f0d8406af90 0x7f0d8406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.430+0000 7f0d7ffff640 1 -- 172.21.15.125:0/3372397312 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f0d8c089670 con 0x7f0d980abc10 2023-10-18T12:00:17.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.430+0000 7f0d977fe640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f0d8406af90 0x7f0d8406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.431+0000 7f0d977fe640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f0d8406af90 0x7f0d8406d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0d88036000 tx=0x7f0d88004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:17.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.434+0000 7f0d7ffff640 1 -- 172.21.15.125:0/3372397312 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d8c0547e0 con 0x7f0d980abc10 2023-10-18T12:00:17.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.599+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d9806c220 con 0x7f0d980abc10 2023-10-18T12:00:17.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.599+0000 7f0d7ffff640 1 -- 172.21.15.125:0/3372397312 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v217) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d8c057e00 con 0x7f0d980abc10 2023-10-18T12:00:17.603 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:00:17.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f0d8406af90 msgr2=0x7f0d8406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f0d8406af90 0x7f0d8406d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0d88036000 tx=0x7f0d88004730 comp rx=0 tx=0).stop 2023-10-18T12:00:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d980abc10 msgr2=0x7f0d98158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d980abc10 0x7f0d98158bc0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00c3c0 tx=0x7f0d8c00c880 comp rx=0 tx=0).stop 2023-10-18T12:00:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 shutdown_connections 2023-10-18T12:00:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f0d8406af90 0x7f0d8406d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d980c96b0 0x7f0d980aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d980abc10 0x7f0d98158bc0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 --2- 172.21.15.125:0/3372397312 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d9806af00 0x7f0d980a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 >> 172.21.15.125:0/3372397312 conn(0x7f0d98069930 msgr2=0x7f0d980acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:17.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 shutdown_connections 2023-10-18T12:00:17.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.601+0000 7f0d9dfa9640 1 -- 172.21.15.125:0/3372397312 wait complete. 2023-10-18T12:00:17.614 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T12:00:17.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.714+0000 7fafb1d97640 1 -- 172.21.15.125:0/556871227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 msgr2=0x7fafac158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.714+0000 7fafb1d97640 1 --2- 172.21.15.125:0/556871227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 0x7fafac158300 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009830 tx=0x7faf9c02bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:17.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.714+0000 7fafb1d97640 1 -- 172.21.15.125:0/556871227 shutdown_connections 2023-10-18T12:00:17.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.714+0000 7fafb1d97640 1 --2- 172.21.15.125:0/556871227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac0b0530 0x7fafac0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.714+0000 7fafb1d97640 1 --2- 172.21.15.125:0/556871227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 0x7fafac158300 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.714+0000 7fafb1d97640 1 -- 172.21.15.125:0/556871227 >> 172.21.15.125:0/556871227 conn(0x7fafac069930 msgr2=0x7fafac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:17.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.714+0000 7fafb1d97640 1 -- 172.21.15.125:0/556871227 shutdown_connections 2023-10-18T12:00:17.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.714+0000 7fafb1d97640 1 -- 172.21.15.125:0/556871227 wait complete. 2023-10-18T12:00:17.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.715+0000 7fafb1d97640 1 Processor -- start 2023-10-18T12:00:17.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafb1d97640 1 -- start start 2023-10-18T12:00:17.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafb1d97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac0afa20 0x7fafac136630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafb1d97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac0b0530 0x7fafac1328d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafb1d97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 0x7fafac13dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafb1d97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac159500 con 0x7fafac0cbd20 2023-10-18T12:00:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafb1d97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac136ea0 con 0x7fafac0afa20 2023-10-18T12:00:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafb1d97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac137180 con 0x7fafac0b0530 2023-10-18T12:00:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafabfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 0x7fafac13dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafabfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 0x7fafac13dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43716/0 (socket says 172.21.15.125:43716) 2023-10-18T12:00:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.716+0000 7fafabfff640 1 -- 172.21.15.125:0/2742639187 learned_addr learned my addr 172.21.15.125:0/2742639187 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafab7fe640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac0afa20 0x7fafac136630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafa2ffd640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac0b0530 0x7fafac1328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafabfff640 1 -- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac0b0530 msgr2=0x7fafac1328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafabfff640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac0b0530 0x7fafac1328d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafabfff640 1 -- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac0afa20 msgr2=0x7fafac136630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafabfff640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac0afa20 0x7fafac136630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafabfff640 1 -- 172.21.15.125:0/2742639187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c009490 con 0x7fafac0cbd20 2023-10-18T12:00:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafab7fe640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac0afa20 0x7fafac136630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafa2ffd640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac0b0530 0x7fafac1328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafabfff640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 0x7fafac13dcc0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fafa400eaa0 tx=0x7fafa400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:17.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafa8ff9640 1 -- 172.21.15.125:0/2742639187 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa400cdb0 con 0x7fafac0cbd20 2023-10-18T12:00:17.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafa8ff9640 1 -- 172.21.15.125:0/2742639187 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fafa4004b20 con 0x7fafac0cbd20 2023-10-18T12:00:17.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac137460 con 0x7fafac0cbd20 2023-10-18T12:00:17.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafa8ff9640 1 -- 172.21.15.125:0/2742639187 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4005020 con 0x7fafac0cbd20 2023-10-18T12:00:17.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.717+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac136b70 con 0x7fafac0cbd20 2023-10-18T12:00:17.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.718+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac0690e0 con 0x7fafac0cbd20 2023-10-18T12:00:17.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.718+0000 7fafa8ff9640 1 -- 172.21.15.125:0/2742639187 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fafa4005180 con 0x7fafac0cbd20 2023-10-18T12:00:17.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.719+0000 7fafa8ff9640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf8c06af60 0x7faf8c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:17.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.719+0000 7fafa8ff9640 1 -- 172.21.15.125:0/2742639187 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fafa408a020 con 0x7fafac0cbd20 2023-10-18T12:00:17.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.719+0000 7fafab7fe640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf8c06af60 0x7faf8c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:17.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.720+0000 7fafab7fe640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf8c06af60 0x7faf8c06d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf9c036000 tx=0x7faf9c004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:17.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.723+0000 7fafa8ff9640 1 -- 172.21.15.125:0/2742639187 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafa4055110 con 0x7fafac0cbd20 2023-10-18T12:00:17.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.890+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fafac0700d0 con 0x7fafac0cbd20 2023-10-18T12:00:17.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.891+0000 7fafa8ff9640 1 -- 172.21.15.125:0/2742639187 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v217) v1 ==== 84+0+65 (secure 0 0 0) 0x7fafa401f090 con 0x7fafac0cbd20 2023-10-18T12:00:17.894 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:17.894 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T12:00:17.894 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:17.894 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T12:00:17.895 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T12:00:17.895 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:17.895 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T12:00:17.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.892+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf8c06af60 msgr2=0x7faf8c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.892+0000 7fafb1d97640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf8c06af60 0x7faf8c06d420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf9c036000 tx=0x7faf9c004730 comp rx=0 tx=0).stop 2023-10-18T12:00:17.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.892+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 msgr2=0x7fafac13dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:17.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.892+0000 7fafb1d97640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 0x7fafac13dcc0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fafa400eaa0 tx=0x7fafa400ef60 comp rx=0 tx=0).stop 2023-10-18T12:00:17.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.893+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 shutdown_connections 2023-10-18T12:00:17.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.893+0000 7fafb1d97640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf8c06af60 0x7faf8c06d420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.893+0000 7fafb1d97640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafac0cbd20 0x7fafac13dcc0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.893+0000 7fafb1d97640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac0b0530 0x7fafac1328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.893+0000 7fafb1d97640 1 --2- 172.21.15.125:0/2742639187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac0afa20 0x7fafac136630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.893+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 >> 172.21.15.125:0/2742639187 conn(0x7fafac069930 msgr2=0x7fafac06b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.893+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 shutdown_connections 2023-10-18T12:00:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:17.893+0000 7fafb1d97640 1 -- 172.21.15.125:0/2742639187 wait complete. 2023-10-18T12:00:17.906 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T12:00:17.906 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T12:00:18.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.009+0000 7fb677ce5640 1 -- 172.21.15.125:0/2772412117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb67015b2a0 msgr2=0x7fb6700d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:18.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.009+0000 7fb677ce5640 1 --2- 172.21.15.125:0/2772412117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb67015b2a0 0x7fb6700d1910 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fd0 tx=0x7fb66002bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:18.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.009+0000 7fb677ce5640 1 -- 172.21.15.125:0/2772412117 shutdown_connections 2023-10-18T12:00:18.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.009+0000 7fb677ce5640 1 --2- 172.21.15.125:0/2772412117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6700ceb40 0x7fb6700b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:18.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.009+0000 7fb677ce5640 1 --2- 172.21.15.125:0/2772412117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb67015b2a0 0x7fb6700d1910 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:18.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.009+0000 7fb677ce5640 1 -- 172.21.15.125:0/2772412117 >> 172.21.15.125:0/2772412117 conn(0x7fb6700d0d60 msgr2=0x7fb6700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:18.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.009+0000 7fb677ce5640 1 -- 172.21.15.125:0/2772412117 shutdown_connections 2023-10-18T12:00:18.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.009+0000 7fb677ce5640 1 -- 172.21.15.125:0/2772412117 wait complete. 2023-10-18T12:00:18.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.010+0000 7fb677ce5640 1 Processor -- start 2023-10-18T12:00:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.010+0000 7fb677ce5640 1 -- start start 2023-10-18T12:00:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb677ce5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6700ceb40 0x7fb670157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb677ce5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6700cf340 0x7fb67006afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb677ce5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb67015b2a0 0x7fb67006b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb677ce5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701f1560 con 0x7fb6700cf340 2023-10-18T12:00:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb677ce5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701f1750 con 0x7fb6700ceb40 2023-10-18T12:00:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb677ce5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701f19f0 con 0x7fb67015b2a0 2023-10-18T12:00:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb675280640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6700cf340 0x7fb67006afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb675280640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6700cf340 0x7fb67006afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43732/0 (socket says 172.21.15.125:43732) 2023-10-18T12:00:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb675280640 1 -- 172.21.15.125:0/3557055829 learned_addr learned my addr 172.21.15.125:0/3557055829 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb676282640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb67015b2a0 0x7fb67006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.011+0000 7fb675a81640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6700ceb40 0x7fb670157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb675280640 1 -- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb67015b2a0 msgr2=0x7fb67006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb675280640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb67015b2a0 0x7fb67006b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb675280640 1 -- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6700ceb40 msgr2=0x7fb670157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb675280640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6700ceb40 0x7fb670157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb675280640 1 -- 172.21.15.125:0/3557055829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660009560 con 0x7fb6700cf340 2023-10-18T12:00:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb675280640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6700cf340 0x7fb67006afe0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fb660009850 tx=0x7fb66002db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb6667fc640 1 -- 172.21.15.125:0/3557055829 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600035f0 con 0x7fb6700cf340 2023-10-18T12:00:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb6667fc640 1 -- 172.21.15.125:0/3557055829 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb660003750 con 0x7fb6700cf340 2023-10-18T12:00:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb6667fc640 1 -- 172.21.15.125:0/3557055829 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660003b00 con 0x7fb6700cf340 2023-10-18T12:00:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6700b0d50 con 0x7fb6700cf340 2023-10-18T12:00:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.012+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6701f1fb0 con 0x7fb6700cf340 2023-10-18T12:00:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.013+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6700677b0 con 0x7fb6700cf340 2023-10-18T12:00:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.013+0000 7fb6667fc640 1 -- 172.21.15.125:0/3557055829 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb660003c60 con 0x7fb6700cf340 2023-10-18T12:00:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.014+0000 7fb6667fc640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb65406af60 0x7fb65406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.014+0000 7fb6667fc640 1 -- 172.21.15.125:0/3557055829 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fb6600ac240 con 0x7fb6700cf340 2023-10-18T12:00:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.014+0000 7fb675a81640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb65406af60 0x7fb65406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.014+0000 7fb675a81640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb65406af60 0x7fb65406d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb66c007fd0 tx=0x7fb66c009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:18.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.018+0000 7fb6667fc640 1 -- 172.21.15.125:0/3557055829 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6600774d0 con 0x7fb6700cf340 2023-10-18T12:00:18.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:18.185+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb6700690e0 con 0x7fb6700cf340 2023-10-18T12:00:19.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.043+0000 7fb6667fc640 1 -- 172.21.15.125:0/3557055829 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v218) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb66007aaf0 con 0x7fb6700cf340 2023-10-18T12:00:19.046 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T12:00:19.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb65406af60 msgr2=0x7fb65406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:19.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb65406af60 0x7fb65406d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb66c007fd0 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2023-10-18T12:00:19.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6700cf340 msgr2=0x7fb67006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:19.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6700cf340 0x7fb67006afe0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fb660009850 tx=0x7fb66002db10 comp rx=0 tx=0).stop 2023-10-18T12:00:19.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 shutdown_connections 2023-10-18T12:00:19.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fb65406af60 0x7fb65406d420 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb66c007fd0 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2023-10-18T12:00:19.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb67015b2a0 0x7fb67006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:19.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6700cf340 0x7fb67006afe0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:19.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 --2- 172.21.15.125:0/3557055829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6700ceb40 0x7fb670157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:19.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 >> 172.21.15.125:0/3557055829 conn(0x7fb6700d0d60 msgr2=0x7fb6700b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:19.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 shutdown_connections 2023-10-18T12:00:19.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.046+0000 7fb677ce5640 1 -- 172.21.15.125:0/3557055829 wait complete. 2023-10-18T12:00:19.063 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T12:00:19.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.172+0000 7fad57fff640 1 -- 172.21.15.125:0/574646464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad5806c4f0 msgr2=0x7fad580b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:19.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.172+0000 7fad57fff640 1 --2- 172.21.15.125:0/574646464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad5806c4f0 0x7fad580b06f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fad50005fd0 tx=0x7fad5002c0e0 comp rx=0 tx=0).stop 2023-10-18T12:00:19.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.172+0000 7fad57fff640 1 -- 172.21.15.125:0/574646464 shutdown_connections 2023-10-18T12:00:19.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.172+0000 7fad57fff640 1 --2- 172.21.15.125:0/574646464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad5806c4f0 0x7fad580b06f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:19.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.172+0000 7fad57fff640 1 --2- 172.21.15.125:0/574646464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580cc2b0 0x7fad5806bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:19.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.172+0000 7fad57fff640 1 -- 172.21.15.125:0/574646464 >> 172.21.15.125:0/574646464 conn(0x7fad580ae5d0 msgr2=0x7fad580ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:19.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.172+0000 7fad57fff640 1 -- 172.21.15.125:0/574646464 shutdown_connections 2023-10-18T12:00:19.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.172+0000 7fad57fff640 1 -- 172.21.15.125:0/574646464 wait complete. 2023-10-18T12:00:19.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.173+0000 7fad57fff640 1 Processor -- start 2023-10-18T12:00:19.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.173+0000 7fad57fff640 1 -- start start 2023-10-18T12:00:19.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad57fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5806c4f0 0x7fad5813d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:19.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad57fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580cc2b0 0x7fad580c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:19.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad57fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58141c80 0x7fad580c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:19.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad57fff640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580c8610 con 0x7fad5806c4f0 2023-10-18T12:00:19.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad57fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5812d030 con 0x7fad58141c80 2023-10-18T12:00:19.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad57fff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5812d310 con 0x7fad580cc2b0 2023-10-18T12:00:19.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5806c4f0 0x7fad5813d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:19.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5806c4f0 0x7fad5813d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43736/0 (socket says 172.21.15.125:43736) 2023-10-18T12:00:19.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 -- 172.21.15.125:0/3836005337 learned_addr learned my addr 172.21.15.125:0/3836005337 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:19.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad4ffff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580cc2b0 0x7fad580c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:19.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad577fe640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58141c80 0x7fad580c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:19.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 -- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580cc2b0 msgr2=0x7fad580c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:19.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580cc2b0 0x7fad580c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:19.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 -- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58141c80 msgr2=0x7fad580c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:19.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58141c80 0x7fad580c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:19.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 -- 172.21.15.125:0/3836005337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40009560 con 0x7fad5806c4f0 2023-10-18T12:00:19.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad56ffd640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5806c4f0 0x7fad5813d3d0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fad4800c430 tx=0x7fad4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:19.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.174+0000 7fad4f7fe640 1 -- 172.21.15.125:0/3836005337 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480052d0 con 0x7fad5806c4f0 2023-10-18T12:00:19.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.175+0000 7fad4f7fe640 1 -- 172.21.15.125:0/3836005337 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad480035f0 con 0x7fad5806c4f0 2023-10-18T12:00:19.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.175+0000 7fad4f7fe640 1 -- 172.21.15.125:0/3836005337 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48003a00 con 0x7fad5806c4f0 2023-10-18T12:00:19.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.175+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad50009840 con 0x7fad5806c4f0 2023-10-18T12:00:19.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.175+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5812ccd0 con 0x7fad5806c4f0 2023-10-18T12:00:19.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.176+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5812ce90 con 0x7fad5806c4f0 2023-10-18T12:00:19.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.176+0000 7fad4f7fe640 1 -- 172.21.15.125:0/3836005337 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fad48003b60 con 0x7fad5806c4f0 2023-10-18T12:00:19.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.177+0000 7fad4f7fe640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fad3806af60 0x7fad3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:19.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.177+0000 7fad4f7fe640 1 -- 172.21.15.125:0/3836005337 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fad48088ee0 con 0x7fad5806c4f0 2023-10-18T12:00:19.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.177+0000 7fad4ffff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fad3806af60 0x7fad3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:19.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.177+0000 7fad4ffff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fad3806af60 0x7fad3806d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad580ae530 tx=0x7fad40009260 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:19.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.181+0000 7fad4f7fe640 1 -- 172.21.15.125:0/3836005337 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad5812ce90 con 0x7fad5806c4f0 2023-10-18T12:00:19.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:19.347+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fad5812d830 con 0x7fad5806c4f0 2023-10-18T12:00:20.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.045+0000 7fad4f7fe640 1 -- 172.21.15.125:0/3836005337 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v219) v1 ==== 170+0+0 (secure 0 0 0) 0x7fad5812d830 con 0x7fad5806c4f0 2023-10-18T12:00:20.048 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T12:00:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fad3806af60 msgr2=0x7fad3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fad3806af60 0x7fad3806d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad580ae530 tx=0x7fad40009260 comp rx=0 tx=0).stop 2023-10-18T12:00:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5806c4f0 msgr2=0x7fad5813d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:20.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5806c4f0 0x7fad5813d3d0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fad4800c430 tx=0x7fad4800c8f0 comp rx=0 tx=0).stop 2023-10-18T12:00:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 shutdown_connections 2023-10-18T12:00:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fad3806af60 0x7fad3806d420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad58141c80 0x7fad580c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:20.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580cc2b0 0x7fad580c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:20.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 --2- 172.21.15.125:0/3836005337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad5806c4f0 0x7fad5813d3d0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:20.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 >> 172.21.15.125:0/3836005337 conn(0x7fad580ae5d0 msgr2=0x7fad581405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:20.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 shutdown_connections 2023-10-18T12:00:20.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.047+0000 7fad57fff640 1 -- 172.21.15.125:0/3836005337 wait complete. 2023-10-18T12:00:20.072 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T12:00:20.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.176+0000 7f84512c9640 1 -- 172.21.15.125:0/3790684518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 msgr2=0x7f8438005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:20.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.176+0000 7f84512c9640 1 --2- 172.21.15.125:0/3790684518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 0x7f8438005180 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f8444009ed0 tx=0x7f84380057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:20.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.176+0000 7f844b7fe640 1 -- 172.21.15.125:0/3790684518 reap_dead start 2023-10-18T12:00:20.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.176+0000 7f84512c9640 1 -- 172.21.15.125:0/3790684518 shutdown_connections 2023-10-18T12:00:20.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.176+0000 7f84512c9640 1 -- 172.21.15.125:0/3790684518 >> 172.21.15.125:0/3790684518 conn(0x7f844c099ff0 msgr2=0x7f844c0d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:20.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.176+0000 7f84512c9640 1 -- 172.21.15.125:0/3790684518 shutdown_connections 2023-10-18T12:00:20.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.176+0000 7f84512c9640 1 -- 172.21.15.125:0/3790684518 wait complete. 2023-10-18T12:00:20.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.177+0000 7f84512c9640 1 Processor -- start 2023-10-18T12:00:20.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.177+0000 7f84512c9640 1 -- start start 2023-10-18T12:00:20.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f84512c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 0x7f844c1e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:20.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f84512c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f844c0b14a0 0x7f844c0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:20.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f84512c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f844c151930 0x7f844c151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:20.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f84512c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0b21d0 con 0x7f8438004d00 2023-10-18T12:00:20.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f84512c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0b24b0 con 0x7f844c151930 2023-10-18T12:00:20.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f84512c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0b2790 con 0x7f844c0b14a0 2023-10-18T12:00:20.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 0x7f844c1e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:20.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 0x7f844c1e51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43756/0 (socket says 172.21.15.125:43756) 2023-10-18T12:00:20.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844affd640 1 -- 172.21.15.125:0/63312754 learned_addr learned my addr 172.21.15.125:0/63312754 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:20.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f8443fff640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f844c0b14a0 0x7f844c0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:20.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844b7fe640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f844c151930 0x7f844c151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:20.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844affd640 1 -- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f844c0b14a0 msgr2=0x7f844c0b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:20.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844affd640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f844c0b14a0 0x7f844c0b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:20.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844affd640 1 -- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f844c151930 msgr2=0x7f844c151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:20.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844affd640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f844c151930 0x7f844c151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:20.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.178+0000 7f844affd640 1 -- 172.21.15.125:0/63312754 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c0b2a70 con 0x7f8438004d00 2023-10-18T12:00:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.179+0000 7f844affd640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 0x7f844c1e51d0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f843c001ce0 tx=0x7f843c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.179+0000 7f84437fe640 1 -- 172.21.15.125:0/63312754 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c011900 con 0x7f8438004d00 2023-10-18T12:00:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.179+0000 7f84437fe640 1 -- 172.21.15.125:0/63312754 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f843c011a80 con 0x7f8438004d00 2023-10-18T12:00:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.179+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844c0b2d50 con 0x7f8438004d00 2023-10-18T12:00:20.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.179+0000 7f84437fe640 1 -- 172.21.15.125:0/63312754 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c011e90 con 0x7f8438004d00 2023-10-18T12:00:20.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.179+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844c1522b0 con 0x7f8438004d00 2023-10-18T12:00:20.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.180+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844c066960 con 0x7f8438004d00 2023-10-18T12:00:20.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.180+0000 7f84437fe640 1 -- 172.21.15.125:0/63312754 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f843c010420 con 0x7f8438004d00 2023-10-18T12:00:20.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.181+0000 7f84437fe640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f842c06af60 0x7f842c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:20.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.181+0000 7f84437fe640 1 -- 172.21.15.125:0/63312754 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5985+0+0 (secure 0 0 0) 0x7f843c08a640 con 0x7f8438004d00 2023-10-18T12:00:20.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.181+0000 7f8443fff640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f842c06af60 0x7f842c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:20.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.182+0000 7f8443fff640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f842c06af60 0x7f842c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f844c0b1270 tx=0x7f8434005f90 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:20.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.186+0000 7f84437fe640 1 -- 172.21.15.125:0/63312754 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f843c0554f0 con 0x7f8438004d00 2023-10-18T12:00:20.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:20.386+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f844c0d18a0 con 0x7f8438004d00 2023-10-18T12:00:21.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.062+0000 7f84437fe640 1 -- 172.21.15.125:0/63312754 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v84) v1 ==== 174+0+0 (secure 0 0 0) 0x7f843c058b10 con 0x7f8438004d00 2023-10-18T12:00:21.065 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 27 and data pool 28 2023-10-18T12:00:21.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f842c06af60 msgr2=0x7f842c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f842c06af60 0x7f842c06d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f844c0b1270 tx=0x7f8434005f90 comp rx=0 tx=0).stop 2023-10-18T12:00:21.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 msgr2=0x7f844c1e51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 0x7f844c1e51d0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f843c001ce0 tx=0x7f843c0039c0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 shutdown_connections 2023-10-18T12:00:21.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f842c06af60 0x7f842c06d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f844c151930 0x7f844c151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f844c0b14a0 0x7f844c0b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 --2- 172.21.15.125:0/63312754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8438004d00 0x7f844c1e51d0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 >> 172.21.15.125:0/63312754 conn(0x7f844c099ff0 msgr2=0x7f844c0cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:21.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 shutdown_connections 2023-10-18T12:00:21.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.065+0000 7f84512c9640 1 -- 172.21.15.125:0/63312754 wait complete. 2023-10-18T12:00:21.098 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:21.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.199+0000 7f690c6e7640 1 -- 172.21.15.125:0/2660409308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f690415b2a0 msgr2=0x7f69040cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.199+0000 7f690c6e7640 1 --2- 172.21.15.125:0/2660409308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f690415b2a0 0x7f69040cf720 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f402bd10 comp rx=0 tx=0).stop 2023-10-18T12:00:21.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.199+0000 7f690c6e7640 1 -- 172.21.15.125:0/2660409308 shutdown_connections 2023-10-18T12:00:21.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.199+0000 7f690c6e7640 1 --2- 172.21.15.125:0/2660409308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f690415b2a0 0x7f69040cf720 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.199+0000 7f690c6e7640 1 --2- 172.21.15.125:0/2660409308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69040cf340 0x7f69040d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.199+0000 7f690c6e7640 1 -- 172.21.15.125:0/2660409308 >> 172.21.15.125:0/2660409308 conn(0x7f69040d0d60 msgr2=0x7f69040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:21.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.199+0000 7f690c6e7640 1 -- 172.21.15.125:0/2660409308 shutdown_connections 2023-10-18T12:00:21.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.199+0000 7f690c6e7640 1 -- 172.21.15.125:0/2660409308 wait complete. 2023-10-18T12:00:21.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.200+0000 7f690c6e7640 1 Processor -- start 2023-10-18T12:00:21.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.200+0000 7f690c6e7640 1 -- start start 2023-10-18T12:00:21.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f690c6e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69040ceb40 0x7f69040cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f690c6e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69040cf340 0x7f69041581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f690c6e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f690415b2a0 0x7f690415d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f690c6e7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040c8480 con 0x7f69040cf340 2023-10-18T12:00:21.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f690c6e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690415ded0 con 0x7f690415b2a0 2023-10-18T12:00:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f690c6e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690415e1b0 con 0x7f69040ceb40 2023-10-18T12:00:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f6909c82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69040cf340 0x7f69041581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f6909c82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69040cf340 0x7f69041581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43774/0 (socket says 172.21.15.125:43774) 2023-10-18T12:00:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.201+0000 7f6909c82640 1 -- 172.21.15.125:0/65169367 learned_addr learned my addr 172.21.15.125:0/65169367 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f690a483640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69040ceb40 0x7f69040cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f690ac84640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f690415b2a0 0x7f690415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f6909c82640 1 -- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69040ceb40 msgr2=0x7f69040cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f6909c82640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69040ceb40 0x7f69040cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f6909c82640 1 -- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f690415b2a0 msgr2=0x7f690415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f6909c82640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f690415b2a0 0x7f690415d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f6909c82640 1 -- 172.21.15.125:0/65169367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f4009560 con 0x7f69040cf340 2023-10-18T12:00:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f690ac84640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f690415b2a0 0x7f690415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f690a483640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69040ceb40 0x7f69040cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f6909c82640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69040cf340 0x7f69041581e0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f68f4006010 tx=0x7f68f402dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f68faffd640 1 -- 172.21.15.125:0/65169367 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4005650 con 0x7f69040cf340 2023-10-18T12:00:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f68faffd640 1 -- 172.21.15.125:0/65169367 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f68f4003800 con 0x7f69040cf340 2023-10-18T12:00:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69040c7e30 con 0x7f69040cf340 2023-10-18T12:00:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f68faffd640 1 -- 172.21.15.125:0/65169367 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4003c10 con 0x7f69040cf340 2023-10-18T12:00:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.202+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6904158720 con 0x7f69040cf340 2023-10-18T12:00:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.203+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69040c8630 con 0x7f69040cf340 2023-10-18T12:00:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.203+0000 7f68faffd640 1 -- 172.21.15.125:0/65169367 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f68f403e450 con 0x7f69040cf340 2023-10-18T12:00:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.204+0000 7f68faffd640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f68e806af90 0x7f68e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.204+0000 7f68faffd640 1 -- 172.21.15.125:0/65169367 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f68f40ac4f0 con 0x7f69040cf340 2023-10-18T12:00:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.204+0000 7f690a483640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f68e806af90 0x7f68e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.205+0000 7f690a483640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f68e806af90 0x7f68e806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f69041e7e30 tx=0x7f68fc005db0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:21.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.208+0000 7f68faffd640 1 -- 172.21.15.125:0/65169367 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69040c8630 con 0x7f69040cf340 2023-10-18T12:00:21.377 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.375+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69040690e0 con 0x7f69040cf340 2023-10-18T12:00:21.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.376+0000 7f68faffd640 1 -- 172.21.15.125:0/65169367 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+16935 (secure 0 0 0) 0x7f68f40773d0 con 0x7f69040cf340 2023-10-18T12:00:21.379 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:21.379 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":220,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:20.387992+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:21.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f68e806af90 msgr2=0x7f68e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f68e806af90 0x7f68e806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f69041e7e30 tx=0x7f68fc005db0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69040cf340 msgr2=0x7f69041581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69040cf340 0x7f69041581e0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f68f4006010 tx=0x7f68f402dce0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 shutdown_connections 2023-10-18T12:00:21.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f68e806af90 0x7f68e806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.382 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f690415b2a0 0x7f690415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69040cf340 0x7f69041581e0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 --2- 172.21.15.125:0/65169367 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f69040ceb40 0x7f69040cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 >> 172.21.15.125:0/65169367 conn(0x7f69040d0d60 msgr2=0x7f69040ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:21.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 shutdown_connections 2023-10-18T12:00:21.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.378+0000 7f690c6e7640 1 -- 172.21.15.125:0/65169367 wait complete. 2023-10-18T12:00:21.392 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:21.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.492+0000 7f0127440640 1 -- 172.21.15.125:0/1052449443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 msgr2=0x7f012006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.492+0000 7f0127440640 1 --2- 172.21.15.125:0/1052449443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 0x7f012006b2e0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f0110009ca0 tx=0x7f011002c230 comp rx=0 tx=0).stop 2023-10-18T12:00:21.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.492+0000 7f0127440640 1 -- 172.21.15.125:0/1052449443 shutdown_connections 2023-10-18T12:00:21.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.492+0000 7f0127440640 1 --2- 172.21.15.125:0/1052449443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01200c96b0 0x7f01200c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.492+0000 7f0127440640 1 --2- 172.21.15.125:0/1052449443 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 0x7f012006b2e0 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.492+0000 7f0127440640 1 -- 172.21.15.125:0/1052449443 >> 172.21.15.125:0/1052449443 conn(0x7f0120069930 msgr2=0x7f0120069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:21.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.492+0000 7f0127440640 1 -- 172.21.15.125:0/1052449443 shutdown_connections 2023-10-18T12:00:21.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.492+0000 7f0127440640 1 -- 172.21.15.125:0/1052449443 wait complete. 2023-10-18T12:00:21.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.493+0000 7f0127440640 1 Processor -- start 2023-10-18T12:00:21.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.493+0000 7f0127440640 1 -- start start 2023-10-18T12:00:21.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f0127440640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 0x7f0120158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f0127440640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01200ab500 0x7f0120138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f0127440640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01200c96b0 0x7f01200aa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f0127440640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201ef5e0 con 0x7f012006af00 2023-10-18T12:00:21.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f0127440640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201ef8c0 con 0x7f01200ab500 2023-10-18T12:00:21.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f0127440640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01201efba0 con 0x7f01200c96b0 2023-10-18T12:00:21.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 0x7f0120158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 0x7f0120158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43786/0 (socket says 172.21.15.125:43786) 2023-10-18T12:00:21.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 -- 172.21.15.125:0/4114460647 learned_addr learned my addr 172.21.15.125:0/4114460647 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01259dd640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01200c96b0 0x7f01200aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 -- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01200c96b0 msgr2=0x7f01200aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01249db640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01200ab500 0x7f0120138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01200c96b0 0x7f01200aa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 -- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01200ab500 msgr2=0x7f0120138fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01200ab500 0x7f0120138fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 -- 172.21.15.125:0/4114460647 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0110009900 con 0x7f012006af00 2023-10-18T12:00:21.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f01251dc640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 0x7f0120158950 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f0110037040 tx=0x7f0110035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:21.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f011dffb640 1 -- 172.21.15.125:0/4114460647 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110034440 con 0x7f012006af00 2023-10-18T12:00:21.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.494+0000 7f011dffb640 1 -- 172.21.15.125:0/4114460647 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0110004540 con 0x7f012006af00 2023-10-18T12:00:21.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.495+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01201efe80 con 0x7f012006af00 2023-10-18T12:00:21.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.495+0000 7f011dffb640 1 -- 172.21.15.125:0/4114460647 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011003f440 con 0x7f012006af00 2023-10-18T12:00:21.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.495+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0120158e90 con 0x7f012006af00 2023-10-18T12:00:21.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.496+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012006c7e0 con 0x7f012006af00 2023-10-18T12:00:21.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.496+0000 7f011dffb640 1 -- 172.21.15.125:0/4114460647 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f011003b040 con 0x7f012006af00 2023-10-18T12:00:21.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.497+0000 7f011dffb640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f010c06af60 0x7f010c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.497+0000 7f011dffb640 1 -- 172.21.15.125:0/4114460647 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f01100ac920 con 0x7f012006af00 2023-10-18T12:00:21.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.497+0000 7f01249db640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f010c06af60 0x7f010c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.497+0000 7f01249db640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f010c06af60 0x7f010c06d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0120071840 tx=0x7f0114008040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:21.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.501+0000 7f011dffb640 1 -- 172.21.15.125:0/4114460647 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0110077860 con 0x7f012006af00 2023-10-18T12:00:21.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.667+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01200690e0 con 0x7f012006af00 2023-10-18T12:00:21.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.668+0000 7f011dffb640 1 -- 172.21.15.125:0/4114460647 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+16935 (secure 0 0 0) 0x7f0110043090 con 0x7f012006af00 2023-10-18T12:00:21.671 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:21.672 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":220,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:20.387992+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:21.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f010c06af60 msgr2=0x7f010c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f010c06af60 0x7f010c06d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0120071840 tx=0x7f0114008040 comp rx=0 tx=0).stop 2023-10-18T12:00:21.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 msgr2=0x7f0120158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 0x7f0120158950 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f0110037040 tx=0x7f0110035e30 comp rx=0 tx=0).stop 2023-10-18T12:00:21.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 shutdown_connections 2023-10-18T12:00:21.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f010c06af60 0x7f010c06d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01200ab500 0x7f0120138fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01200c96b0 0x7f01200aa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 --2- 172.21.15.125:0/4114460647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f012006af00 0x7f0120158950 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 >> 172.21.15.125:0/4114460647 conn(0x7f0120069930 msgr2=0x7f0120069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:21.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 shutdown_connections 2023-10-18T12:00:21.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.670+0000 7f0127440640 1 -- 172.21.15.125:0/4114460647 wait complete. 2023-10-18T12:00:21.684 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T12:00:21.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.786+0000 7fc075f8e640 1 -- 172.21.15.125:0/2557537330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc070141690 msgr2=0x7fc0700703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.786+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2557537330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc070141690 0x7fc0700703e0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7fc064005fd0 tx=0x7fc06402bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:21.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.786+0000 7fc075f8e640 1 -- 172.21.15.125:0/2557537330 shutdown_connections 2023-10-18T12:00:21.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.786+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2557537330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc070070920 0x7fc0700affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.786+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2557537330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc070141690 0x7fc0700703e0 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.786+0000 7fc075f8e640 1 -- 172.21.15.125:0/2557537330 >> 172.21.15.125:0/2557537330 conn(0x7fc070069930 msgr2=0x7fc070069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:21.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.786+0000 7fc075f8e640 1 -- 172.21.15.125:0/2557537330 shutdown_connections 2023-10-18T12:00:21.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.786+0000 7fc075f8e640 1 -- 172.21.15.125:0/2557537330 wait complete. 2023-10-18T12:00:21.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.787+0000 7fc075f8e640 1 Processor -- start 2023-10-18T12:00:21.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.787+0000 7fc075f8e640 1 -- start start 2023-10-18T12:00:21.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc075f8e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc070070920 0x7fc0701d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc075f8e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc07013e520 0x7fc0701e1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc075f8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc070141690 0x7fc0701e2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc075f8e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0701d7a10 con 0x7fc07013e520 2023-10-18T12:00:21.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc075f8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0701d6e00 con 0x7fc070141690 2023-10-18T12:00:21.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc075f8e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0701d70e0 con 0x7fc070070920 2023-10-18T12:00:21.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc07013e520 0x7fc0701e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc07013e520 0x7fc0701e1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43814/0 (socket says 172.21.15.125:43814) 2023-10-18T12:00:21.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 -- 172.21.15.125:0/2476531230 learned_addr learned my addr 172.21.15.125:0/2476531230 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:21.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06ffff640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc070141690 0x7fc0701e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06f7fe640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc070070920 0x7fc0701d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 -- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc070070920 msgr2=0x7fc0701d9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc070070920 0x7fc0701d9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 -- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc070141690 msgr2=0x7fc0701e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:21.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc070141690 0x7fc0701e2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:21.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 -- 172.21.15.125:0/2476531230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc064009560 con 0x7fc07013e520 2023-10-18T12:00:21.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc06effd640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc07013e520 0x7fc0701e1590 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fc064006010 tx=0x7fc06402db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:21.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.788+0000 7fc057fff640 1 -- 172.21.15.125:0/2476531230 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0640035f0 con 0x7fc07013e520 2023-10-18T12:00:21.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.789+0000 7fc057fff640 1 -- 172.21.15.125:0/2476531230 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc064003840 con 0x7fc07013e520 2023-10-18T12:00:21.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.789+0000 7fc057fff640 1 -- 172.21.15.125:0/2476531230 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064003c50 con 0x7fc07013e520 2023-10-18T12:00:21.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.789+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0701d73c0 con 0x7fc07013e520 2023-10-18T12:00:21.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.789+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0701e1ad0 con 0x7fc07013e520 2023-10-18T12:00:21.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.790+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0701e1c60 con 0x7fc07013e520 2023-10-18T12:00:21.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.790+0000 7fc057fff640 1 -- 172.21.15.125:0/2476531230 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc06403e450 con 0x7fc07013e520 2023-10-18T12:00:21.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.790+0000 7fc057fff640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc05c06af60 0x7fc05c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:21.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.790+0000 7fc057fff640 1 -- 172.21.15.125:0/2476531230 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fc0640ac4b0 con 0x7fc07013e520 2023-10-18T12:00:21.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.791+0000 7fc06f7fe640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc05c06af60 0x7fc05c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:21.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.791+0000 7fc06f7fe640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc05c06af60 0x7fc05c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc060000f60 tx=0x7fc060009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:21.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.794+0000 7fc057fff640 1 -- 172.21.15.125:0/2476531230 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0701e1c60 con 0x7fc07013e520 2023-10-18T12:00:21.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:21.994+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc0701e1c60 con 0x7fc07013e520 2023-10-18T12:00:22.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.074+0000 7fc057fff640 1 -- 172.21.15.125:0/2476531230 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc0701e1c60 con 0x7fc07013e520 2023-10-18T12:00:22.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc05c06af60 msgr2=0x7fc05c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:22.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc05c06af60 0x7fc05c06d420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc060000f60 tx=0x7fc060009040 comp rx=0 tx=0).stop 2023-10-18T12:00:22.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc07013e520 msgr2=0x7fc0701e1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:22.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc07013e520 0x7fc0701e1590 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fc064006010 tx=0x7fc06402db30 comp rx=0 tx=0).stop 2023-10-18T12:00:22.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 shutdown_connections 2023-10-18T12:00:22.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fc05c06af60 0x7fc05c06d420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc070141690 0x7fc0701e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc07013e520 0x7fc0701e1590 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 --2- 172.21.15.125:0/2476531230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc070070920 0x7fc0701d9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.076+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 >> 172.21.15.125:0/2476531230 conn(0x7fc070069930 msgr2=0x7fc0700b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:22.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.077+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 shutdown_connections 2023-10-18T12:00:22.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.077+0000 7fc075f8e640 1 -- 172.21.15.125:0/2476531230 wait complete. 2023-10-18T12:00:22.090 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T12:00:22.091 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T12:00:22.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.196+0000 7fde0ed33640 1 -- 172.21.15.125:0/2584998861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0806af00 msgr2=0x7fde0806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:22.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.196+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2584998861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0806af00 0x7fde0806b2e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fde00009ca0 tx=0x7fde0002c2e0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.196+0000 7fde0ed33640 1 -- 172.21.15.125:0/2584998861 shutdown_connections 2023-10-18T12:00:22.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.196+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2584998861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde080c96b0 0x7fde080c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.196+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2584998861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0806af00 0x7fde0806b2e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.196+0000 7fde0ed33640 1 -- 172.21.15.125:0/2584998861 >> 172.21.15.125:0/2584998861 conn(0x7fde08069930 msgr2=0x7fde08069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:22.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.196+0000 7fde0ed33640 1 -- 172.21.15.125:0/2584998861 shutdown_connections 2023-10-18T12:00:22.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.196+0000 7fde0ed33640 1 -- 172.21.15.125:0/2584998861 wait complete. 2023-10-18T12:00:22.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.197+0000 7fde0ed33640 1 Processor -- start 2023-10-18T12:00:22.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.197+0000 7fde0ed33640 1 -- start start 2023-10-18T12:00:22.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde0ed33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0806af00 0x7fde081f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:22.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde0ed33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde080ab500 0x7fde08151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:22.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde0ed33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde080c96b0 0x7fde080a9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:22.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde0ed33640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080acaf0 con 0x7fde080ab500 2023-10-18T12:00:22.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde0ed33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde081f3cd0 con 0x7fde0806af00 2023-10-18T12:00:22.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde0ed33640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde081f3710 con 0x7fde080c96b0 2023-10-18T12:00:22.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde07fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde080ab500 0x7fde08151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:22.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde07fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde080ab500 0x7fde08151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43824/0 (socket says 172.21.15.125:43824) 2023-10-18T12:00:22.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde07fff640 1 -- 172.21.15.125:0/2207901841 learned_addr learned my addr 172.21.15.125:0/2207901841 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:22.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde0d2d0640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde080c96b0 0x7fde080a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:22.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.198+0000 7fde0cacf640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0806af00 0x7fde081f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:22.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde07fff640 1 -- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde080c96b0 msgr2=0x7fde080a9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:22.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde07fff640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde080c96b0 0x7fde080a9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde07fff640 1 -- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0806af00 msgr2=0x7fde081f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:22.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde07fff640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0806af00 0x7fde081f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:22.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde07fff640 1 -- 172.21.15.125:0/2207901841 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde00009900 con 0x7fde080ab500 2023-10-18T12:00:22.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde07fff640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde080ab500 0x7fde08151ea0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00e790 tx=0x7fddfc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:22.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde057fa640 1 -- 172.21.15.125:0/2207901841 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc009580 con 0x7fde080ab500 2023-10-18T12:00:22.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde057fa640 1 -- 172.21.15.125:0/2207901841 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fddfc00cdb0 con 0x7fde080ab500 2023-10-18T12:00:22.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde081f39f0 con 0x7fde080ab500 2023-10-18T12:00:22.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde057fa640 1 -- 172.21.15.125:0/2207901841 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc010690 con 0x7fde080ab500 2023-10-18T12:00:22.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.199+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0812a1e0 con 0x7fde080ab500 2023-10-18T12:00:22.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.200+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde080acc30 con 0x7fde080ab500 2023-10-18T12:00:22.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.200+0000 7fde057fa640 1 -- 172.21.15.125:0/2207901841 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fddfc0107f0 con 0x7fde080ab500 2023-10-18T12:00:22.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.201+0000 7fde057fa640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fddf406af60 0x7fddf406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:22.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.201+0000 7fde057fa640 1 -- 172.21.15.125:0/2207901841 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fddfc014070 con 0x7fde080ab500 2023-10-18T12:00:22.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.201+0000 7fde0cacf640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fddf406af60 0x7fddf406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:22.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.202+0000 7fde0cacf640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fddf406af60 0x7fddf406d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fde00036000 tx=0x7fde00004ab0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:22.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.205+0000 7fde057fa640 1 -- 172.21.15.125:0/2207901841 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fddfc051ca0 con 0x7fde080ab500 2023-10-18T12:00:22.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:22.404+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fde080ac320 con 0x7fde080ab500 2023-10-18T12:00:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.084+0000 7fde057fa640 1 -- 172.21.15.125:0/2207901841 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) v1 ==== 122+0+0 (secure 0 0 0) 0x7fddfc055180 con 0x7fde080ab500 2023-10-18T12:00:23.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fddf406af60 msgr2=0x7fddf406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fddf406af60 0x7fddf406d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fde00036000 tx=0x7fde00004ab0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde080ab500 msgr2=0x7fde08151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde080ab500 0x7fde08151ea0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00e790 tx=0x7fddfc00ec50 comp rx=0 tx=0).stop 2023-10-18T12:00:23.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 shutdown_connections 2023-10-18T12:00:23.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fddf406af60 0x7fddf406d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde080c96b0 0x7fde080a9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde080ab500 0x7fde08151ea0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 --2- 172.21.15.125:0/2207901841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde0806af00 0x7fde081f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 >> 172.21.15.125:0/2207901841 conn(0x7fde08069930 msgr2=0x7fde0815ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:23.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 shutdown_connections 2023-10-18T12:00:23.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.086+0000 7fde0ed33640 1 -- 172.21.15.125:0/2207901841 wait complete. 2023-10-18T12:00:23.101 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T12:00:23.102 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.203+0000 7fceb92af640 1 -- 172.21.15.125:0/378467721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40b24d0 msgr2=0x7fceb40b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.203+0000 7fceb92af640 1 --2- 172.21.15.125:0/378467721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40b24d0 0x7fceb40b28b0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fceb4070870 tx=0x7fcea402bd10 comp rx=0 tx=0).stop 2023-10-18T12:00:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.203+0000 7fceb92af640 1 -- 172.21.15.125:0/378467721 shutdown_connections 2023-10-18T12:00:23.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.203+0000 7fceb92af640 1 --2- 172.21.15.125:0/378467721 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceb40acd40 0x7fceb41595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.203+0000 7fceb92af640 1 --2- 172.21.15.125:0/378467721 >> v2:172.21.15.139:3300/0 conn(0x7fceb40b3070 0x7fceb41590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.203+0000 7fceb92af640 1 --2- 172.21.15.125:0/378467721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40b24d0 0x7fceb40b28b0 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.203+0000 7fceb92af640 1 -- 172.21.15.125:0/378467721 >> 172.21.15.125:0/378467721 conn(0x7fceb40b3f00 msgr2=0x7fceb40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:23.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.204+0000 7fceb92af640 1 -- 172.21.15.125:0/378467721 shutdown_connections 2023-10-18T12:00:23.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.204+0000 7fceb92af640 1 -- 172.21.15.125:0/378467721 wait complete. 2023-10-18T12:00:23.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb92af640 1 Processor -- start 2023-10-18T12:00:23.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb92af640 1 -- start start 2023-10-18T12:00:23.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb92af640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40acd40 0x7fceb41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb92af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceb40b24d0 0x7fceb40c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb92af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb40b3070 0x7fceb40ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb92af640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb406b770 con 0x7fceb40acd40 2023-10-18T12:00:23.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb92af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb406a830 con 0x7fceb40b3070 2023-10-18T12:00:23.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb92af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb406aaf0 con 0x7fceb40b24d0 2023-10-18T12:00:23.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40acd40 0x7fceb41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40acd40 0x7fceb41526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43848/0 (socket says 172.21.15.125:43848) 2023-10-18T12:00:23.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.205+0000 7fceb2ffd640 1 -- 172.21.15.125:0/3673424812 learned_addr learned my addr 172.21.15.125:0/3673424812 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:23.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb37fe640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb40b3070 0x7fceb40ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceabfff640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceb40b24d0 0x7fceb40c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb2ffd640 1 -- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceb40b24d0 msgr2=0x7fceb40c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb2ffd640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceb40b24d0 0x7fceb40c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb2ffd640 1 -- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb40b3070 msgr2=0x7fceb40ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb2ffd640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb40b3070 0x7fceb40ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb2ffd640 1 -- 172.21.15.125:0/3673424812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea4009560 con 0x7fceb40acd40 2023-10-18T12:00:23.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb2ffd640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40acd40 0x7fceb41526d0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fcea4036040 tx=0x7fcea402dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:23.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceab7fe640 1 -- 172.21.15.125:0/3673424812 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea4005650 con 0x7fceb40acd40 2023-10-18T12:00:23.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceab7fe640 1 -- 172.21.15.125:0/3673424812 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcea40038f0 con 0x7fceb40acd40 2023-10-18T12:00:23.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb406add0 con 0x7fceb40acd40 2023-10-18T12:00:23.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceab7fe640 1 -- 172.21.15.125:0/3673424812 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea4003d00 con 0x7fceb40acd40 2023-10-18T12:00:23.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.206+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb406b390 con 0x7fceb40acd40 2023-10-18T12:00:23.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.207+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb40690e0 con 0x7fceb40acd40 2023-10-18T12:00:23.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.207+0000 7fceab7fe640 1 -- 172.21.15.125:0/3673424812 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcea403e450 con 0x7fceb40acd40 2023-10-18T12:00:23.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.208+0000 7fceab7fe640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fce9406af60 0x7fce9406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.208+0000 7fceab7fe640 1 -- 172.21.15.125:0/3673424812 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fcea40ac300 con 0x7fceb40acd40 2023-10-18T12:00:23.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.208+0000 7fceabfff640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fce9406af60 0x7fce9406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.209+0000 7fceabfff640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fce9406af60 0x7fce9406d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fceb415ba20 tx=0x7fce9c008040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:23.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.212+0000 7fceab7fe640 1 -- 172.21.15.125:0/3673424812 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcea4077240 con 0x7fceb40acd40 2023-10-18T12:00:23.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.414+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fceb4157310 con 0x7fceb40acd40 2023-10-18T12:00:23.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.415+0000 7fceab7fe640 1 -- 172.21.15.125:0/3673424812 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4931 (secure 0 0 0) 0x7fcea4003e60 con 0x7fceb40acd40 2023-10-18T12:00:23.419 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:23.419 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":86,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10168,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:20.388305+0000","modified":"2023-10-18T12:00:22.406153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11976":{"gid":11976,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2975172452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2975172452},{"type":"v1","addr":"172.21.15.125:6834","nonce":2975172452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-18T12:00:23.421 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 86 2023-10-18T12:00:23.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fce9406af60 msgr2=0x7fce9406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fce9406af60 0x7fce9406d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fceb415ba20 tx=0x7fce9c008040 comp rx=0 tx=0).stop 2023-10-18T12:00:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40acd40 msgr2=0x7fceb41526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40acd40 0x7fceb41526d0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fcea4036040 tx=0x7fcea402dce0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 shutdown_connections 2023-10-18T12:00:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fce9406af60 0x7fce9406d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fceb40b3070 0x7fceb40ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceb40b24d0 0x7fceb40c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 --2- 172.21.15.125:0/3673424812 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fceb40acd40 0x7fceb41526d0 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 >> 172.21.15.125:0/3673424812 conn(0x7fceb40b3f00 msgr2=0x7fceb41590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:23.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 shutdown_connections 2023-10-18T12:00:23.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.417+0000 7fceb92af640 1 -- 172.21.15.125:0/3673424812 wait complete. 2023-10-18T12:00:23.430 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:23.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.531+0000 7f8e3830c640 1 -- 172.21.15.125:0/1396447223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300afa20 msgr2=0x7f8e30158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.531+0000 7f8e3830c640 1 --2- 172.21.15.125:0/1396447223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300afa20 0x7f8e30158300 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000a940 tx=0x7f8e2002d090 comp rx=0 tx=0).stop 2023-10-18T12:00:23.534 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.531+0000 7f8e3830c640 1 -- 172.21.15.125:0/1396447223 shutdown_connections 2023-10-18T12:00:23.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.531+0000 7f8e3830c640 1 --2- 172.21.15.125:0/1396447223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300afa20 0x7f8e30158300 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.531+0000 7f8e3830c640 1 -- 172.21.15.125:0/1396447223 >> 172.21.15.125:0/1396447223 conn(0x7f8e30069930 msgr2=0x7f8e30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:23.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.531+0000 7f8e3830c640 1 -- 172.21.15.125:0/1396447223 shutdown_connections 2023-10-18T12:00:23.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.531+0000 7f8e3830c640 1 -- 172.21.15.125:0/1396447223 wait complete. 2023-10-18T12:00:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.532+0000 7f8e3830c640 1 Processor -- start 2023-10-18T12:00:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.532+0000 7f8e3830c640 1 -- start start 2023-10-18T12:00:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e3830c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300afa20 0x7f8e30146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e3830c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e300b0530 0x7f8e301e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e3830c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300cbd20 0x7f8e301e3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e3830c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3014f710 con 0x7f8e300cbd20 2023-10-18T12:00:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e3830c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3014d890 con 0x7f8e300afa20 2023-10-18T12:00:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e3830c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e301e3830 con 0x7f8e300b0530 2023-10-18T12:00:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300cbd20 0x7f8e301e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300cbd20 0x7f8e301e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43874/0 (socket says 172.21.15.125:43874) 2023-10-18T12:00:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 -- 172.21.15.125:0/3283920360 learned_addr learned my addr 172.21.15.125:0/3283920360 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:23.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e360a8640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300afa20 0x7f8e30146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e358a7640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e300b0530 0x7f8e301e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 -- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e300b0530 msgr2=0x7f8e301e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e300b0530 0x7f8e301e2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 -- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300afa20 msgr2=0x7f8e30146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300afa20 0x7f8e30146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 -- 172.21.15.125:0/3283920360 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2000a5a0 con 0x7f8e300cbd20 2023-10-18T12:00:23.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e358a7640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e300b0530 0x7f8e301e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:23.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e368a9640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300cbd20 0x7f8e301e3200 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c430 tx=0x7f8e2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:23.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e26ffd640 1 -- 172.21.15.125:0/3283920360 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28011790 con 0x7f8e300cbd20 2023-10-18T12:00:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e26ffd640 1 -- 172.21.15.125:0/3283920360 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8e280119e0 con 0x7f8e300cbd20 2023-10-18T12:00:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3014db50 con 0x7f8e300cbd20 2023-10-18T12:00:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e26ffd640 1 -- 172.21.15.125:0/3283920360 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28011e10 con 0x7f8e300cbd20 2023-10-18T12:00:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.533+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3014e110 con 0x7f8e300cbd20 2023-10-18T12:00:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.534+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e300690e0 con 0x7f8e300cbd20 2023-10-18T12:00:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.535+0000 7f8e26ffd640 1 -- 172.21.15.125:0/3283920360 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8e2801c460 con 0x7f8e300cbd20 2023-10-18T12:00:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.535+0000 7f8e26ffd640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8e1406af60 0x7f8e1406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.535+0000 7f8e26ffd640 1 -- 172.21.15.125:0/3283920360 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f8e28089a80 con 0x7f8e300cbd20 2023-10-18T12:00:23.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.536+0000 7f8e360a8640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8e1406af60 0x7f8e1406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.536+0000 7f8e360a8640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8e1406af60 0x7f8e1406d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009f60 tx=0x7f8e2c0094f0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:23.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.540+0000 7f8e26ffd640 1 -- 172.21.15.125:0/3283920360 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e280548a0 con 0x7f8e300cbd20 2023-10-18T12:00:23.710 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.707+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e3006f700 con 0x7f8e300cbd20 2023-10-18T12:00:23.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.709+0000 7f8e26ffd640 1 -- 172.21.15.125:0/3283920360 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+16922 (secure 0 0 0) 0x7f8e28057ec0 con 0x7f8e300cbd20 2023-10-18T12:00:23.711 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:23.712 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":221,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:22.063337+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:23.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.710+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8e1406af60 msgr2=0x7f8e1406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.710+0000 7f8e3830c640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8e1406af60 0x7f8e1406d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009f60 tx=0x7f8e2c0094f0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.710+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300cbd20 msgr2=0x7f8e301e3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.710+0000 7f8e3830c640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300cbd20 0x7f8e301e3200 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c430 tx=0x7f8e2800c8f0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.714 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.711+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 shutdown_connections 2023-10-18T12:00:23.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.711+0000 7f8e3830c640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8e1406af60 0x7f8e1406d420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.711+0000 7f8e3830c640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e300cbd20 0x7f8e301e3200 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.711+0000 7f8e3830c640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e300b0530 0x7f8e301e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.711+0000 7f8e3830c640 1 --2- 172.21.15.125:0/3283920360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300afa20 0x7f8e30146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.711+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 >> 172.21.15.125:0/3283920360 conn(0x7f8e30069930 msgr2=0x7f8e300aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:23.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.711+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 shutdown_connections 2023-10-18T12:00:23.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.711+0000 7f8e3830c640 1 -- 172.21.15.125:0/3283920360 wait complete. 2023-10-18T12:00:23.724 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T12:00:23.725 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:00:23.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.824+0000 7f546d9c0640 1 -- 172.21.15.125:0/51136401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f546815aff0 msgr2=0x7f54680d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.824+0000 7f546d9c0640 1 --2- 172.21.15.125:0/51136401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f546815aff0 0x7f54680d1700 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f5458005fd0 tx=0x7f545802c240 comp rx=0 tx=0).stop 2023-10-18T12:00:23.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.824+0000 7f546d9c0640 1 -- 172.21.15.125:0/51136401 shutdown_connections 2023-10-18T12:00:23.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.824+0000 7f546d9c0640 1 --2- 172.21.15.125:0/51136401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f54680b1ec0 0x7f54680cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.824+0000 7f546d9c0640 1 --2- 172.21.15.125:0/51136401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f546815aff0 0x7f54680d1700 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.824+0000 7f546d9c0640 1 -- 172.21.15.125:0/51136401 >> 172.21.15.125:0/51136401 conn(0x7f546809a010 msgr2=0x7f54680d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:23.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.824+0000 7f546d9c0640 1 -- 172.21.15.125:0/51136401 shutdown_connections 2023-10-18T12:00:23.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.824+0000 7f546d9c0640 1 -- 172.21.15.125:0/51136401 wait complete. 2023-10-18T12:00:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.825+0000 7f546d9c0640 1 Processor -- start 2023-10-18T12:00:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.825+0000 7f546d9c0640 1 -- start start 2023-10-18T12:00:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f546d9c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54680b1ec0 0x7f54680cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f546d9c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54680cd740 0x7f54680b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f546d9c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f546815aff0 0x7f546814c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f546d9c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546815d350 con 0x7f54680b1ec0 2023-10-18T12:00:23.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f546d9c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546815d630 con 0x7f54680cd740 2023-10-18T12:00:23.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f546d9c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546815d910 con 0x7f546815aff0 2023-10-18T12:00:23.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54680b1ec0 0x7f54680cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54680b1ec0 0x7f54680cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43894/0 (socket says 172.21.15.125:43894) 2023-10-18T12:00:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 -- 172.21.15.125:0/520311195 learned_addr learned my addr 172.21.15.125:0/520311195 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f54677fe640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f546815aff0 0x7f546814c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f54667fc640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54680cd740 0x7f54680b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 -- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f546815aff0 msgr2=0x7f546814c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f546815aff0 0x7f546814c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 -- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54680cd740 msgr2=0x7f54680b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:23.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54680cd740 0x7f54680b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:23.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 -- 172.21.15.125:0/520311195 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5458009900 con 0x7f54680b1ec0 2023-10-18T12:00:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f54677fe640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f546815aff0 0x7f546814c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f54667fc640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54680cd740 0x7f54680b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f5466ffd640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54680b1ec0 0x7f54680cae30 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f5458006010 tx=0x7f5458035ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f544ffff640 1 -- 172.21.15.125:0/520311195 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458034440 con 0x7f54680b1ec0 2023-10-18T12:00:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f544ffff640 1 -- 172.21.15.125:0/520311195 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5458004330 con 0x7f54680b1ec0 2023-10-18T12:00:23.835 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546815dbf0 con 0x7f54680b1ec0 2023-10-18T12:00:23.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f544ffff640 1 -- 172.21.15.125:0/520311195 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580046e0 con 0x7f54680b1ec0 2023-10-18T12:00:23.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.826+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546815d000 con 0x7f54680b1ec0 2023-10-18T12:00:23.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.827+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546815da50 con 0x7f54680b1ec0 2023-10-18T12:00:23.836 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.828+0000 7f544ffff640 1 -- 172.21.15.125:0/520311195 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f545803b040 con 0x7f54680b1ec0 2023-10-18T12:00:23.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.828+0000 7f544ffff640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f545406af60 0x7f545406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:23.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.828+0000 7f544ffff640 1 -- 172.21.15.125:0/520311195 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f54580ac550 con 0x7f54680b1ec0 2023-10-18T12:00:23.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.828+0000 7f54667fc640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f545406af60 0x7f545406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:23.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.829+0000 7f54667fc640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f545406af60 0x7f545406d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f546808ad30 tx=0x7f545c009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:23.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.832+0000 7f544ffff640 1 -- 172.21.15.125:0/520311195 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f546815da50 con 0x7f54680b1ec0 2023-10-18T12:00:23.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:23.993+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 --> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f546815da50 con 0x7f545406af60 2023-10-18T12:00:24.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.002+0000 7f544ffff640 1 -- 172.21.15.125:0/520311195 <== mgr.8786 v2:172.21.15.139:6832/2362843834 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290123 (secure 0 0 0) 0x7f546815da50 con 0x7f545406af60 2023-10-18T12:00:24.008 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:24.010 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:00:24.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f545406af60 msgr2=0x7f545406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f545406af60 0x7f545406d420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f546808ad30 tx=0x7f545c009040 comp rx=0 tx=0).stop 2023-10-18T12:00:24.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54680b1ec0 msgr2=0x7f54680cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54680b1ec0 0x7f54680cae30 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f5458006010 tx=0x7f5458035ac0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 shutdown_connections 2023-10-18T12:00:24.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f545406af60 0x7f545406d420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54680cd740 0x7f54680b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f546815aff0 0x7f546814c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 --2- 172.21.15.125:0/520311195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f54680b1ec0 0x7f54680cae30 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 >> 172.21.15.125:0/520311195 conn(0x7f546809a010 msgr2=0x7f54680cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:24.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 shutdown_connections 2023-10-18T12:00:24.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.005+0000 7f546d9c0640 1 -- 172.21.15.125:0/520311195 wait complete. 2023-10-18T12:00:24.018 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T12:00:22.799446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":639,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413888,"kb_used_data":22860,"kb_used_omap":12,"kb_used_meta":390899,"kb_avail":754560832,"statfs":{"total":773094113280,"available":772670291968,"internally_reserved":0,"allocated":23408640,"data_stored":12523392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":400281144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000993"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067038+0000","last_change":"2023-10-18T12:00:21.094990+0000","last_active":"2023-10-18T12:00:22.067038+0000","last_peered":"2023-10-18T12:00:22.067038+0000","last_clean":"2023-10-18T12:00:22.067038+0000","last_became_active":"2023-10-18T12:00:21.094239+0000","last_became_peered":"2023-10-18T12:00:21.094239+0000","last_unstale":"2023-10-18T12:00:22.067038+0000","last_undegraded":"2023-10-18T12:00:22.067038+0000","last_fullsized":"2023-10-18T12:00:22.067038+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:22.792096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091251+0000","last_change":"2023-10-18T12:00:21.089403+0000","last_active":"2023-10-18T12:00:21.091251+0000","last_peered":"2023-10-18T12:00:21.091251+0000","last_clean":"2023-10-18T12:00:21.091251+0000","last_became_active":"2023-10-18T12:00:21.089138+0000","last_became_peered":"2023-10-18T12:00:21.089138+0000","last_unstale":"2023-10-18T12:00:21.091251+0000","last_undegraded":"2023-10-18T12:00:21.091251+0000","last_fullsized":"2023-10-18T12:00:21.091251+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:16:03.472142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167524+0000","last_change":"2023-10-18T12:00:21.084830+0000","last_active":"2023-10-18T12:00:22.167524+0000","last_peered":"2023-10-18T12:00:22.167524+0000","last_clean":"2023-10-18T12:00:22.167524+0000","last_became_active":"2023-10-18T12:00:21.084546+0000","last_became_peered":"2023-10-18T12:00:21.084546+0000","last_unstale":"2023-10-18T12:00:22.167524+0000","last_undegraded":"2023-10-18T12:00:22.167524+0000","last_fullsized":"2023-10-18T12:00:22.167524+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:18.371937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089002+0000","last_change":"2023-10-18T12:00:21.087099+0000","last_active":"2023-10-18T12:00:21.089002+0000","last_peered":"2023-10-18T12:00:21.089002+0000","last_clean":"2023-10-18T12:00:21.089002+0000","last_became_active":"2023-10-18T12:00:21.086806+0000","last_became_peered":"2023-10-18T12:00:21.086806+0000","last_unstale":"2023-10-18T12:00:21.089002+0000","last_undegraded":"2023-10-18T12:00:21.089002+0000","last_fullsized":"2023-10-18T12:00:21.089002+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:50.792000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487950+0000","last_change":"2023-10-18T12:00:21.081967+0000","last_active":"2023-10-18T12:00:22.487950+0000","last_peered":"2023-10-18T12:00:22.487950+0000","last_clean":"2023-10-18T12:00:22.487950+0000","last_became_active":"2023-10-18T12:00:21.081693+0000","last_became_peered":"2023-10-18T12:00:21.081693+0000","last_unstale":"2023-10-18T12:00:22.487950+0000","last_undegraded":"2023-10-18T12:00:22.487950+0000","last_fullsized":"2023-10-18T12:00:22.487950+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:47:47.036558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.087502+0000","last_change":"2023-10-18T12:00:21.083829+0000","last_active":"2023-10-18T12:00:21.087502+0000","last_peered":"2023-10-18T12:00:21.087502+0000","last_clean":"2023-10-18T12:00:21.087502+0000","last_became_active":"2023-10-18T12:00:21.083554+0000","last_became_peered":"2023-10-18T12:00:21.083554+0000","last_unstale":"2023-10-18T12:00:21.087502+0000","last_undegraded":"2023-10-18T12:00:21.087502+0000","last_fullsized":"2023-10-18T12:00:21.087502+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:01.308646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487844+0000","last_change":"2023-10-18T12:00:21.081320+0000","last_active":"2023-10-18T12:00:22.487844+0000","last_peered":"2023-10-18T12:00:22.487844+0000","last_clean":"2023-10-18T12:00:22.487844+0000","last_became_active":"2023-10-18T12:00:21.080987+0000","last_became_peered":"2023-10-18T12:00:21.080987+0000","last_unstale":"2023-10-18T12:00:22.487844+0000","last_undegraded":"2023-10-18T12:00:22.487844+0000","last_fullsized":"2023-10-18T12:00:22.487844+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:10:51.800092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461713+0000","last_change":"2023-10-18T12:00:21.085280+0000","last_active":"2023-10-18T12:00:22.461713+0000","last_peered":"2023-10-18T12:00:22.461713+0000","last_clean":"2023-10-18T12:00:22.461713+0000","last_became_active":"2023-10-18T12:00:21.085007+0000","last_became_peered":"2023-10-18T12:00:21.085007+0000","last_unstale":"2023-10-18T12:00:22.461713+0000","last_undegraded":"2023-10-18T12:00:22.461713+0000","last_fullsized":"2023-10-18T12:00:22.461713+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:00:11.590501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.085556+0000","last_change":"2023-10-18T12:00:21.084370+0000","last_active":"2023-10-18T12:00:21.085556+0000","last_peered":"2023-10-18T12:00:21.085556+0000","last_clean":"2023-10-18T12:00:21.085556+0000","last_became_active":"2023-10-18T12:00:21.082951+0000","last_became_peered":"2023-10-18T12:00:21.082951+0000","last_unstale":"2023-10-18T12:00:21.085556+0000","last_undegraded":"2023-10-18T12:00:21.085556+0000","last_fullsized":"2023-10-18T12:00:21.085556+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:13.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462333+0000","last_change":"2023-10-18T12:00:21.091511+0000","last_active":"2023-10-18T12:00:22.462333+0000","last_peered":"2023-10-18T12:00:22.462333+0000","last_clean":"2023-10-18T12:00:22.462333+0000","last_became_active":"2023-10-18T12:00:21.091236+0000","last_became_peered":"2023-10-18T12:00:21.091236+0000","last_unstale":"2023-10-18T12:00:22.462333+0000","last_undegraded":"2023-10-18T12:00:22.462333+0000","last_fullsized":"2023-10-18T12:00:22.462333+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:27.189758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167609+0000","last_change":"2023-10-18T12:00:21.097042+0000","last_active":"2023-10-18T12:00:22.167609+0000","last_peered":"2023-10-18T12:00:22.167609+0000","last_clean":"2023-10-18T12:00:22.167609+0000","last_became_active":"2023-10-18T12:00:21.096759+0000","last_became_peered":"2023-10-18T12:00:21.096759+0000","last_unstale":"2023-10-18T12:00:22.167609+0000","last_undegraded":"2023-10-18T12:00:22.167609+0000","last_fullsized":"2023-10-18T12:00:22.167609+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:41.300175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487996+0000","last_change":"2023-10-18T12:00:21.087044+0000","last_active":"2023-10-18T12:00:22.487996+0000","last_peered":"2023-10-18T12:00:22.487996+0000","last_clean":"2023-10-18T12:00:22.487996+0000","last_became_active":"2023-10-18T12:00:21.086784+0000","last_became_peered":"2023-10-18T12:00:21.086784+0000","last_unstale":"2023-10-18T12:00:22.487996+0000","last_undegraded":"2023-10-18T12:00:22.487996+0000","last_fullsized":"2023-10-18T12:00:22.487996+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:16.398494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.087531+0000","last_change":"2023-10-18T12:00:21.084368+0000","last_active":"2023-10-18T12:00:21.087531+0000","last_peered":"2023-10-18T12:00:21.087531+0000","last_clean":"2023-10-18T12:00:21.087531+0000","last_became_active":"2023-10-18T12:00:21.084087+0000","last_became_peered":"2023-10-18T12:00:21.084087+0000","last_unstale":"2023-10-18T12:00:21.087531+0000","last_undegraded":"2023-10-18T12:00:21.087531+0000","last_fullsized":"2023-10-18T12:00:21.087531+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:31.554580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461854+0000","last_change":"2023-10-18T12:00:21.091256+0000","last_active":"2023-10-18T12:00:22.461854+0000","last_peered":"2023-10-18T12:00:22.461854+0000","last_clean":"2023-10-18T12:00:22.461854+0000","last_became_active":"2023-10-18T12:00:21.090808+0000","last_became_peered":"2023-10-18T12:00:21.090808+0000","last_unstale":"2023-10-18T12:00:22.461854+0000","last_undegraded":"2023-10-18T12:00:22.461854+0000","last_fullsized":"2023-10-18T12:00:22.461854+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:53:42.580058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.068026+0000","last_change":"2023-10-18T12:00:21.085027+0000","last_active":"2023-10-18T12:00:22.068026+0000","last_peered":"2023-10-18T12:00:22.068026+0000","last_clean":"2023-10-18T12:00:22.068026+0000","last_became_active":"2023-10-18T12:00:21.084752+0000","last_became_peered":"2023-10-18T12:00:21.084752+0000","last_unstale":"2023-10-18T12:00:22.068026+0000","last_undegraded":"2023-10-18T12:00:22.068026+0000","last_fullsized":"2023-10-18T12:00:22.068026+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:55.331050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167449+0000","last_change":"2023-10-18T12:00:21.082735+0000","last_active":"2023-10-18T12:00:22.167449+0000","last_peered":"2023-10-18T12:00:22.167449+0000","last_clean":"2023-10-18T12:00:22.167449+0000","last_became_active":"2023-10-18T12:00:21.082501+0000","last_became_peered":"2023-10-18T12:00:21.082501+0000","last_unstale":"2023-10-18T12:00:22.167449+0000","last_undegraded":"2023-10-18T12:00:22.167449+0000","last_fullsized":"2023-10-18T12:00:22.167449+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:11.830792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.090994+0000","last_change":"2023-10-18T12:00:21.089110+0000","last_active":"2023-10-18T12:00:21.090994+0000","last_peered":"2023-10-18T12:00:21.090994+0000","last_clean":"2023-10-18T12:00:21.090994+0000","last_became_active":"2023-10-18T12:00:21.088828+0000","last_became_peered":"2023-10-18T12:00:21.088828+0000","last_unstale":"2023-10-18T12:00:21.090994+0000","last_undegraded":"2023-10-18T12:00:21.090994+0000","last_fullsized":"2023-10-18T12:00:21.090994+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:23.786569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.081032+0000","last_change":"2023-10-18T12:00:21.076021+0000","last_active":"2023-10-18T12:00:21.081032+0000","last_peered":"2023-10-18T12:00:21.081032+0000","last_clean":"2023-10-18T12:00:21.081032+0000","last_became_active":"2023-10-18T12:00:21.075761+0000","last_became_peered":"2023-10-18T12:00:21.075761+0000","last_unstale":"2023-10-18T12:00:21.081032+0000","last_undegraded":"2023-10-18T12:00:21.081032+0000","last_fullsized":"2023-10-18T12:00:21.081032+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:34.556666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462373+0000","last_change":"2023-10-18T12:00:21.092451+0000","last_active":"2023-10-18T12:00:22.462373+0000","last_peered":"2023-10-18T12:00:22.462373+0000","last_clean":"2023-10-18T12:00:22.462373+0000","last_became_active":"2023-10-18T12:00:21.091995+0000","last_became_peered":"2023-10-18T12:00:21.091995+0000","last_unstale":"2023-10-18T12:00:22.462373+0000","last_undegraded":"2023-10-18T12:00:22.462373+0000","last_fullsized":"2023-10-18T12:00:22.462373+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:27.792233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462175+0000","last_change":"2023-10-18T12:00:21.090556+0000","last_active":"2023-10-18T12:00:22.462175+0000","last_peered":"2023-10-18T12:00:22.462175+0000","last_clean":"2023-10-18T12:00:22.462175+0000","last_became_active":"2023-10-18T12:00:21.088740+0000","last_became_peered":"2023-10-18T12:00:21.088740+0000","last_unstale":"2023-10-18T12:00:22.462175+0000","last_undegraded":"2023-10-18T12:00:22.462175+0000","last_fullsized":"2023-10-18T12:00:22.462175+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:32.899439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461891+0000","last_change":"2023-10-18T12:00:21.093298+0000","last_active":"2023-10-18T12:00:22.461891+0000","last_peered":"2023-10-18T12:00:22.461891+0000","last_clean":"2023-10-18T12:00:22.461891+0000","last_became_active":"2023-10-18T12:00:21.092815+0000","last_became_peered":"2023-10-18T12:00:21.092815+0000","last_unstale":"2023-10-18T12:00:22.461891+0000","last_undegraded":"2023-10-18T12:00:22.461891+0000","last_fullsized":"2023-10-18T12:00:22.461891+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:28.526624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461867+0000","last_change":"2023-10-18T12:00:21.091079+0000","last_active":"2023-10-18T12:00:22.461867+0000","last_peered":"2023-10-18T12:00:22.461867+0000","last_clean":"2023-10-18T12:00:22.461867+0000","last_became_active":"2023-10-18T12:00:21.090440+0000","last_became_peered":"2023-10-18T12:00:21.090440+0000","last_unstale":"2023-10-18T12:00:22.461867+0000","last_undegraded":"2023-10-18T12:00:22.461867+0000","last_fullsized":"2023-10-18T12:00:22.461867+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:54.338893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.079346+0000","last_change":"2023-10-18T12:00:21.073828+0000","last_active":"2023-10-18T12:00:21.079346+0000","last_peered":"2023-10-18T12:00:21.079346+0000","last_clean":"2023-10-18T12:00:21.079346+0000","last_became_active":"2023-10-18T12:00:21.073594+0000","last_became_peered":"2023-10-18T12:00:21.073594+0000","last_unstale":"2023-10-18T12:00:21.079346+0000","last_undegraded":"2023-10-18T12:00:21.079346+0000","last_fullsized":"2023-10-18T12:00:21.079346+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:36.596128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487805+0000","last_change":"2023-10-18T12:00:21.082982+0000","last_active":"2023-10-18T12:00:22.487805+0000","last_peered":"2023-10-18T12:00:22.487805+0000","last_clean":"2023-10-18T12:00:22.487805+0000","last_became_active":"2023-10-18T12:00:21.082700+0000","last_became_peered":"2023-10-18T12:00:21.082700+0000","last_unstale":"2023-10-18T12:00:22.487805+0000","last_undegraded":"2023-10-18T12:00:22.487805+0000","last_fullsized":"2023-10-18T12:00:22.487805+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:25.713305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.081472+0000","last_change":"2023-10-18T12:00:21.075391+0000","last_active":"2023-10-18T12:00:21.081472+0000","last_peered":"2023-10-18T12:00:21.081472+0000","last_clean":"2023-10-18T12:00:21.081472+0000","last_became_active":"2023-10-18T12:00:21.075140+0000","last_became_peered":"2023-10-18T12:00:21.075140+0000","last_unstale":"2023-10-18T12:00:21.081472+0000","last_undegraded":"2023-10-18T12:00:21.081472+0000","last_fullsized":"2023-10-18T12:00:21.081472+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:15.346268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067253+0000","last_change":"2023-10-18T12:00:21.091634+0000","last_active":"2023-10-18T12:00:22.067253+0000","last_peered":"2023-10-18T12:00:22.067253+0000","last_clean":"2023-10-18T12:00:22.067253+0000","last_became_active":"2023-10-18T12:00:21.091327+0000","last_became_peered":"2023-10-18T12:00:21.091327+0000","last_unstale":"2023-10-18T12:00:22.067253+0000","last_undegraded":"2023-10-18T12:00:22.067253+0000","last_fullsized":"2023-10-18T12:00:22.067253+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:53:58.179224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167625+0000","last_change":"2023-10-18T12:00:21.093406+0000","last_active":"2023-10-18T12:00:22.167625+0000","last_peered":"2023-10-18T12:00:22.167625+0000","last_clean":"2023-10-18T12:00:22.167625+0000","last_became_active":"2023-10-18T12:00:21.093138+0000","last_became_peered":"2023-10-18T12:00:21.093138+0000","last_unstale":"2023-10-18T12:00:22.167625+0000","last_undegraded":"2023-10-18T12:00:22.167625+0000","last_fullsized":"2023-10-18T12:00:22.167625+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:09.568953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488124+0000","last_change":"2023-10-18T12:00:21.086100+0000","last_active":"2023-10-18T12:00:22.488124+0000","last_peered":"2023-10-18T12:00:22.488124+0000","last_clean":"2023-10-18T12:00:22.488124+0000","last_became_active":"2023-10-18T12:00:21.085820+0000","last_became_peered":"2023-10-18T12:00:21.085820+0000","last_unstale":"2023-10-18T12:00:22.488124+0000","last_undegraded":"2023-10-18T12:00:22.488124+0000","last_fullsized":"2023-10-18T12:00:22.488124+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:59.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.085463+0000","last_change":"2023-10-18T12:00:21.082792+0000","last_active":"2023-10-18T12:00:21.085463+0000","last_peered":"2023-10-18T12:00:21.085463+0000","last_clean":"2023-10-18T12:00:21.085463+0000","last_became_active":"2023-10-18T12:00:21.082529+0000","last_became_peered":"2023-10-18T12:00:21.082529+0000","last_unstale":"2023-10-18T12:00:21.085463+0000","last_undegraded":"2023-10-18T12:00:21.085463+0000","last_fullsized":"2023-10-18T12:00:21.085463+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:15.212784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067873+0000","last_change":"2023-10-18T12:00:21.087421+0000","last_active":"2023-10-18T12:00:22.067873+0000","last_peered":"2023-10-18T12:00:22.067873+0000","last_clean":"2023-10-18T12:00:22.067873+0000","last_became_active":"2023-10-18T12:00:21.087145+0000","last_became_peered":"2023-10-18T12:00:21.087145+0000","last_unstale":"2023-10-18T12:00:22.067873+0000","last_undegraded":"2023-10-18T12:00:22.067873+0000","last_fullsized":"2023-10-18T12:00:22.067873+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:28.489188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.093430+0000","last_change":"2023-10-18T12:00:21.092324+0000","last_active":"2023-10-18T12:00:21.093430+0000","last_peered":"2023-10-18T12:00:21.093430+0000","last_clean":"2023-10-18T12:00:21.093430+0000","last_became_active":"2023-10-18T12:00:21.092059+0000","last_became_peered":"2023-10-18T12:00:21.092059+0000","last_unstale":"2023-10-18T12:00:21.093430+0000","last_undegraded":"2023-10-18T12:00:21.093430+0000","last_fullsized":"2023-10-18T12:00:21.093430+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:58.467913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.094347+0000","last_change":"2023-10-18T12:00:21.093224+0000","last_active":"2023-10-18T12:00:21.094347+0000","last_peered":"2023-10-18T12:00:21.094347+0000","last_clean":"2023-10-18T12:00:21.094347+0000","last_became_active":"2023-10-18T12:00:21.092949+0000","last_became_peered":"2023-10-18T12:00:21.092949+0000","last_unstale":"2023-10-18T12:00:21.094347+0000","last_undegraded":"2023-10-18T12:00:21.094347+0000","last_fullsized":"2023-10-18T12:00:21.094347+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:40.263149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.082066+0000","last_change":"2023-10-18T12:00:21.073279+0000","last_active":"2023-10-18T12:00:21.082066+0000","last_peered":"2023-10-18T12:00:21.082066+0000","last_clean":"2023-10-18T12:00:21.082066+0000","last_became_active":"2023-10-18T12:00:21.073036+0000","last_became_peered":"2023-10-18T12:00:21.073036+0000","last_unstale":"2023-10-18T12:00:21.082066+0000","last_undegraded":"2023-10-18T12:00:21.082066+0000","last_fullsized":"2023-10-18T12:00:21.082066+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:03.802023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"220'4","reported_seq":16,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.068145+0000","last_change":"2023-10-18T12:00:20.063561+0000","last_active":"2023-10-18T12:00:22.068145+0000","last_peered":"2023-10-18T12:00:22.068145+0000","last_clean":"2023-10-18T12:00:22.068145+0000","last_became_active":"2023-10-18T12:00:20.063285+0000","last_became_peered":"2023-10-18T12:00:20.063285+0000","last_unstale":"2023-10-18T12:00:22.068145+0000","last_undegraded":"2023-10-18T12:00:22.068145+0000","last_fullsized":"2023-10-18T12:00:22.068145+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:55.051193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066637+0000","last_change":"2023-10-18T12:00:21.073385+0000","last_active":"2023-10-18T12:00:22.066637+0000","last_peered":"2023-10-18T12:00:22.066637+0000","last_clean":"2023-10-18T12:00:22.066637+0000","last_became_active":"2023-10-18T12:00:21.073048+0000","last_became_peered":"2023-10-18T12:00:21.073048+0000","last_unstale":"2023-10-18T12:00:22.066637+0000","last_undegraded":"2023-10-18T12:00:22.066637+0000","last_fullsized":"2023-10-18T12:00:22.066637+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:23:54.891459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"220'1","reported_seq":13,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067313+0000","last_change":"2023-10-18T12:00:20.057976+0000","last_active":"2023-10-18T12:00:22.067313+0000","last_peered":"2023-10-18T12:00:22.067313+0000","last_clean":"2023-10-18T12:00:22.067313+0000","last_became_active":"2023-10-18T12:00:20.057699+0000","last_became_peered":"2023-10-18T12:00:20.057699+0000","last_unstale":"2023-10-18T12:00:22.067313+0000","last_undegraded":"2023-10-18T12:00:22.067313+0000","last_fullsized":"2023-10-18T12:00:22.067313+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:45.951679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462070+0000","last_change":"2023-10-18T12:00:21.093491+0000","last_active":"2023-10-18T12:00:22.462070+0000","last_peered":"2023-10-18T12:00:22.462070+0000","last_clean":"2023-10-18T12:00:22.462070+0000","last_became_active":"2023-10-18T12:00:21.093199+0000","last_became_peered":"2023-10-18T12:00:21.093199+0000","last_unstale":"2023-10-18T12:00:22.462070+0000","last_undegraded":"2023-10-18T12:00:22.462070+0000","last_fullsized":"2023-10-18T12:00:22.462070+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:14.227710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"220'2","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.090216+0000","last_change":"2023-10-18T12:00:20.057262+0000","last_active":"2023-10-18T12:00:21.090216+0000","last_peered":"2023-10-18T12:00:21.090216+0000","last_clean":"2023-10-18T12:00:21.090216+0000","last_became_active":"2023-10-18T12:00:20.057010+0000","last_became_peered":"2023-10-18T12:00:20.057010+0000","last_unstale":"2023-10-18T12:00:21.090216+0000","last_undegraded":"2023-10-18T12:00:21.090216+0000","last_fullsized":"2023-10-18T12:00:21.090216+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:46.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487995+0000","last_change":"2023-10-18T12:00:21.089013+0000","last_active":"2023-10-18T12:00:22.487995+0000","last_peered":"2023-10-18T12:00:22.487995+0000","last_clean":"2023-10-18T12:00:22.487995+0000","last_became_active":"2023-10-18T12:00:21.088643+0000","last_became_peered":"2023-10-18T12:00:21.088643+0000","last_unstale":"2023-10-18T12:00:22.487995+0000","last_undegraded":"2023-10-18T12:00:22.487995+0000","last_fullsized":"2023-10-18T12:00:22.487995+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:15.272122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"220'2","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089377+0000","last_change":"2023-10-18T12:00:20.060797+0000","last_active":"2023-10-18T12:00:21.089377+0000","last_peered":"2023-10-18T12:00:21.089377+0000","last_clean":"2023-10-18T12:00:21.089377+0000","last_became_active":"2023-10-18T12:00:20.060568+0000","last_became_peered":"2023-10-18T12:00:20.060568+0000","last_unstale":"2023-10-18T12:00:21.089377+0000","last_undegraded":"2023-10-18T12:00:21.089377+0000","last_fullsized":"2023-10-18T12:00:21.089377+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:27:50.071689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.068083+0000","last_change":"2023-10-18T12:00:21.087738+0000","last_active":"2023-10-18T12:00:22.068083+0000","last_peered":"2023-10-18T12:00:22.068083+0000","last_clean":"2023-10-18T12:00:22.068083+0000","last_became_active":"2023-10-18T12:00:21.087104+0000","last_became_peered":"2023-10-18T12:00:21.087104+0000","last_unstale":"2023-10-18T12:00:22.068083+0000","last_undegraded":"2023-10-18T12:00:22.068083+0000","last_fullsized":"2023-10-18T12:00:22.068083+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:52.609784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"220'1","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089141+0000","last_change":"2023-10-18T12:00:20.070416+0000","last_active":"2023-10-18T12:00:21.089141+0000","last_peered":"2023-10-18T12:00:21.089141+0000","last_clean":"2023-10-18T12:00:21.089141+0000","last_became_active":"2023-10-18T12:00:20.070121+0000","last_became_peered":"2023-10-18T12:00:20.070121+0000","last_unstale":"2023-10-18T12:00:21.089141+0000","last_undegraded":"2023-10-18T12:00:21.089141+0000","last_fullsized":"2023-10-18T12:00:21.089141+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:33.588701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.068148+0000","last_change":"2023-10-18T12:00:21.081350+0000","last_active":"2023-10-18T12:00:22.068148+0000","last_peered":"2023-10-18T12:00:22.068148+0000","last_clean":"2023-10-18T12:00:22.068148+0000","last_became_active":"2023-10-18T12:00:21.080920+0000","last_became_peered":"2023-10-18T12:00:21.080920+0000","last_unstale":"2023-10-18T12:00:22.068148+0000","last_undegraded":"2023-10-18T12:00:22.068148+0000","last_fullsized":"2023-10-18T12:00:22.068148+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:37.696091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.077114+0000","last_change":"2023-10-18T12:00:20.070314+0000","last_active":"2023-10-18T12:00:21.077114+0000","last_peered":"2023-10-18T12:00:21.077114+0000","last_clean":"2023-10-18T12:00:21.077114+0000","last_became_active":"2023-10-18T12:00:20.069974+0000","last_became_peered":"2023-10-18T12:00:20.069974+0000","last_unstale":"2023-10-18T12:00:21.077114+0000","last_undegraded":"2023-10-18T12:00:21.077114+0000","last_fullsized":"2023-10-18T12:00:21.077114+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:39.359672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.081495+0000","last_change":"2023-10-18T12:00:21.076517+0000","last_active":"2023-10-18T12:00:21.081495+0000","last_peered":"2023-10-18T12:00:21.081495+0000","last_clean":"2023-10-18T12:00:21.081495+0000","last_became_active":"2023-10-18T12:00:21.076276+0000","last_became_peered":"2023-10-18T12:00:21.076276+0000","last_unstale":"2023-10-18T12:00:21.081495+0000","last_undegraded":"2023-10-18T12:00:21.081495+0000","last_fullsized":"2023-10-18T12:00:21.081495+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:26.534619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167494+0000","last_change":"2023-10-18T12:00:20.067947+0000","last_active":"2023-10-18T12:00:22.167494+0000","last_peered":"2023-10-18T12:00:22.167494+0000","last_clean":"2023-10-18T12:00:22.167494+0000","last_became_active":"2023-10-18T12:00:20.067433+0000","last_became_peered":"2023-10-18T12:00:20.067433+0000","last_unstale":"2023-10-18T12:00:22.167494+0000","last_undegraded":"2023-10-18T12:00:22.167494+0000","last_fullsized":"2023-10-18T12:00:22.167494+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:38.382188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067730+0000","last_change":"2023-10-18T12:00:21.079285+0000","last_active":"2023-10-18T12:00:22.067730+0000","last_peered":"2023-10-18T12:00:22.067730+0000","last_clean":"2023-10-18T12:00:22.067730+0000","last_became_active":"2023-10-18T12:00:21.079002+0000","last_became_peered":"2023-10-18T12:00:21.079002+0000","last_unstale":"2023-10-18T12:00:22.067730+0000","last_undegraded":"2023-10-18T12:00:22.067730+0000","last_fullsized":"2023-10-18T12:00:22.067730+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:49:28.849697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488078+0000","last_change":"2023-10-18T12:00:20.068018+0000","last_active":"2023-10-18T12:00:22.488078+0000","last_peered":"2023-10-18T12:00:22.488078+0000","last_clean":"2023-10-18T12:00:22.488078+0000","last_became_active":"2023-10-18T12:00:20.067744+0000","last_became_peered":"2023-10-18T12:00:20.067744+0000","last_unstale":"2023-10-18T12:00:22.488078+0000","last_undegraded":"2023-10-18T12:00:22.488078+0000","last_fullsized":"2023-10-18T12:00:22.488078+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:17.572716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167507+0000","last_change":"2023-10-18T12:00:21.078356+0000","last_active":"2023-10-18T12:00:22.167507+0000","last_peered":"2023-10-18T12:00:22.167507+0000","last_clean":"2023-10-18T12:00:22.167507+0000","last_became_active":"2023-10-18T12:00:21.078095+0000","last_became_peered":"2023-10-18T12:00:21.078095+0000","last_unstale":"2023-10-18T12:00:22.167507+0000","last_undegraded":"2023-10-18T12:00:22.167507+0000","last_fullsized":"2023-10-18T12:00:22.167507+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:44.526073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.078461+0000","last_change":"2023-10-18T12:00:20.070057+0000","last_active":"2023-10-18T12:00:21.078461+0000","last_peered":"2023-10-18T12:00:21.078461+0000","last_clean":"2023-10-18T12:00:21.078461+0000","last_became_active":"2023-10-18T12:00:20.069456+0000","last_became_peered":"2023-10-18T12:00:20.069456+0000","last_unstale":"2023-10-18T12:00:21.078461+0000","last_undegraded":"2023-10-18T12:00:21.078461+0000","last_fullsized":"2023-10-18T12:00:21.078461+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:14.207084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.077097+0000","last_change":"2023-10-18T12:00:21.074575+0000","last_active":"2023-10-18T12:00:21.077097+0000","last_peered":"2023-10-18T12:00:21.077097+0000","last_clean":"2023-10-18T12:00:21.077097+0000","last_became_active":"2023-10-18T12:00:21.074302+0000","last_became_peered":"2023-10-18T12:00:21.074302+0000","last_unstale":"2023-10-18T12:00:21.077097+0000","last_undegraded":"2023-10-18T12:00:21.077097+0000","last_fullsized":"2023-10-18T12:00:21.077097+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:58.467913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"220'3","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089992+0000","last_change":"2023-10-18T12:00:20.057436+0000","last_active":"2023-10-18T12:00:21.089992+0000","last_peered":"2023-10-18T12:00:21.089992+0000","last_clean":"2023-10-18T12:00:21.089992+0000","last_became_active":"2023-10-18T12:00:20.057117+0000","last_became_peered":"2023-10-18T12:00:20.057117+0000","last_unstale":"2023-10-18T12:00:21.089992+0000","last_undegraded":"2023-10-18T12:00:21.089992+0000","last_fullsized":"2023-10-18T12:00:21.089992+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:02.796323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488093+0000","last_change":"2023-10-18T12:00:21.092364+0000","last_active":"2023-10-18T12:00:22.488093+0000","last_peered":"2023-10-18T12:00:22.488093+0000","last_clean":"2023-10-18T12:00:22.488093+0000","last_became_active":"2023-10-18T12:00:21.092090+0000","last_became_peered":"2023-10-18T12:00:21.092090+0000","last_unstale":"2023-10-18T12:00:22.488093+0000","last_undegraded":"2023-10-18T12:00:22.488093+0000","last_fullsized":"2023-10-18T12:00:22.488093+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:19.318183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"220'6","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.097622+0000","last_change":"2023-10-18T12:00:20.070213+0000","last_active":"2023-10-18T12:00:21.097622+0000","last_peered":"2023-10-18T12:00:21.097622+0000","last_clean":"2023-10-18T12:00:21.097622+0000","last_became_active":"2023-10-18T12:00:20.069888+0000","last_became_peered":"2023-10-18T12:00:20.069888+0000","last_unstale":"2023-10-18T12:00:21.097622+0000","last_undegraded":"2023-10-18T12:00:21.097622+0000","last_fullsized":"2023-10-18T12:00:21.097622+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:55.247225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.080305+0000","last_change":"2023-10-18T12:00:21.078685+0000","last_active":"2023-10-18T12:00:21.080305+0000","last_peered":"2023-10-18T12:00:21.080305+0000","last_clean":"2023-10-18T12:00:21.080305+0000","last_became_active":"2023-10-18T12:00:21.078434+0000","last_became_peered":"2023-10-18T12:00:21.078434+0000","last_unstale":"2023-10-18T12:00:21.080305+0000","last_undegraded":"2023-10-18T12:00:21.080305+0000","last_fullsized":"2023-10-18T12:00:21.080305+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:42:38.614585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"220'2","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091496+0000","last_change":"2023-10-18T12:00:20.063126+0000","last_active":"2023-10-18T12:00:21.091496+0000","last_peered":"2023-10-18T12:00:21.091496+0000","last_clean":"2023-10-18T12:00:21.091496+0000","last_became_active":"2023-10-18T12:00:20.062833+0000","last_became_peered":"2023-10-18T12:00:20.062833+0000","last_unstale":"2023-10-18T12:00:21.091496+0000","last_undegraded":"2023-10-18T12:00:21.091496+0000","last_fullsized":"2023-10-18T12:00:21.091496+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:57.198338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167686+0000","last_change":"2023-10-18T12:00:21.075390+0000","last_active":"2023-10-18T12:00:22.167686+0000","last_peered":"2023-10-18T12:00:22.167686+0000","last_clean":"2023-10-18T12:00:22.167686+0000","last_became_active":"2023-10-18T12:00:21.075149+0000","last_became_peered":"2023-10-18T12:00:21.075149+0000","last_unstale":"2023-10-18T12:00:22.167686+0000","last_undegraded":"2023-10-18T12:00:22.167686+0000","last_fullsized":"2023-10-18T12:00:22.167686+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:00.806334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167467+0000","last_change":"2023-10-18T12:00:20.059890+0000","last_active":"2023-10-18T12:00:22.167467+0000","last_peered":"2023-10-18T12:00:22.167467+0000","last_clean":"2023-10-18T12:00:22.167467+0000","last_became_active":"2023-10-18T12:00:20.059647+0000","last_became_peered":"2023-10-18T12:00:20.059647+0000","last_unstale":"2023-10-18T12:00:22.167467+0000","last_undegraded":"2023-10-18T12:00:22.167467+0000","last_fullsized":"2023-10-18T12:00:22.167467+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:08.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.084681+0000","last_change":"2023-10-18T12:00:21.081617+0000","last_active":"2023-10-18T12:00:21.084681+0000","last_peered":"2023-10-18T12:00:21.084681+0000","last_clean":"2023-10-18T12:00:21.084681+0000","last_became_active":"2023-10-18T12:00:21.081232+0000","last_became_peered":"2023-10-18T12:00:21.081232+0000","last_unstale":"2023-10-18T12:00:21.084681+0000","last_undegraded":"2023-10-18T12:00:21.084681+0000","last_fullsized":"2023-10-18T12:00:21.084681+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:40.263149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.069951+0000","last_change":"2023-10-18T12:00:20.061573+0000","last_active":"2023-10-18T12:00:21.069951+0000","last_peered":"2023-10-18T12:00:21.069951+0000","last_clean":"2023-10-18T12:00:21.069951+0000","last_became_active":"2023-10-18T12:00:20.061363+0000","last_became_peered":"2023-10-18T12:00:20.061363+0000","last_unstale":"2023-10-18T12:00:21.069951+0000","last_undegraded":"2023-10-18T12:00:21.069951+0000","last_fullsized":"2023-10-18T12:00:21.069951+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:33.588701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.088949+0000","last_change":"2023-10-18T12:00:21.086868+0000","last_active":"2023-10-18T12:00:21.088949+0000","last_peered":"2023-10-18T12:00:21.088949+0000","last_clean":"2023-10-18T12:00:21.088949+0000","last_became_active":"2023-10-18T12:00:21.086595+0000","last_became_peered":"2023-10-18T12:00:21.086595+0000","last_unstale":"2023-10-18T12:00:21.088949+0000","last_undegraded":"2023-10-18T12:00:21.088949+0000","last_fullsized":"2023-10-18T12:00:21.088949+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:45.002374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"220'8","reported_seq":20,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487751+0000","last_change":"2023-10-18T12:00:20.066867+0000","last_active":"2023-10-18T12:00:22.487751+0000","last_peered":"2023-10-18T12:00:22.487751+0000","last_clean":"2023-10-18T12:00:22.487751+0000","last_became_active":"2023-10-18T12:00:20.066615+0000","last_became_peered":"2023-10-18T12:00:20.066615+0000","last_unstale":"2023-10-18T12:00:22.487751+0000","last_undegraded":"2023-10-18T12:00:22.487751+0000","last_fullsized":"2023-10-18T12:00:22.487751+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:31.444571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461760+0000","last_change":"2023-10-18T12:00:21.092578+0000","last_active":"2023-10-18T12:00:22.461760+0000","last_peered":"2023-10-18T12:00:22.461760+0000","last_clean":"2023-10-18T12:00:22.461760+0000","last_became_active":"2023-10-18T12:00:21.092290+0000","last_became_peered":"2023-10-18T12:00:21.092290+0000","last_unstale":"2023-10-18T12:00:22.461760+0000","last_undegraded":"2023-10-18T12:00:22.461760+0000","last_fullsized":"2023-10-18T12:00:22.461760+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:05.871282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"220'1","reported_seq":13,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067117+0000","last_change":"2023-10-18T12:00:20.058912+0000","last_active":"2023-10-18T12:00:22.067117+0000","last_peered":"2023-10-18T12:00:22.067117+0000","last_clean":"2023-10-18T12:00:22.067117+0000","last_became_active":"2023-10-18T12:00:20.058634+0000","last_became_peered":"2023-10-18T12:00:20.058634+0000","last_unstale":"2023-10-18T12:00:22.067117+0000","last_undegraded":"2023-10-18T12:00:22.067117+0000","last_fullsized":"2023-10-18T12:00:22.067117+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:06.254577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066788+0000","last_change":"2023-10-18T12:00:21.090572+0000","last_active":"2023-10-18T12:00:22.066788+0000","last_peered":"2023-10-18T12:00:22.066788+0000","last_clean":"2023-10-18T12:00:22.066788+0000","last_became_active":"2023-10-18T12:00:21.090312+0000","last_became_peered":"2023-10-18T12:00:21.090312+0000","last_unstale":"2023-10-18T12:00:22.066788+0000","last_undegraded":"2023-10-18T12:00:22.066788+0000","last_fullsized":"2023-10-18T12:00:22.066788+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:00:29.845435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"220'2","reported_seq":14,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067347+0000","last_change":"2023-10-18T12:00:20.067607+0000","last_active":"2023-10-18T12:00:22.067347+0000","last_peered":"2023-10-18T12:00:22.067347+0000","last_clean":"2023-10-18T12:00:22.067347+0000","last_became_active":"2023-10-18T12:00:20.067298+0000","last_became_peered":"2023-10-18T12:00:20.067298+0000","last_unstale":"2023-10-18T12:00:22.067347+0000","last_undegraded":"2023-10-18T12:00:22.067347+0000","last_fullsized":"2023-10-18T12:00:22.067347+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.982605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066968+0000","last_change":"2023-10-18T12:00:21.074593+0000","last_active":"2023-10-18T12:00:22.066968+0000","last_peered":"2023-10-18T12:00:22.066968+0000","last_clean":"2023-10-18T12:00:22.066968+0000","last_became_active":"2023-10-18T12:00:21.074328+0000","last_became_peered":"2023-10-18T12:00:21.074328+0000","last_unstale":"2023-10-18T12:00:22.066968+0000","last_undegraded":"2023-10-18T12:00:22.066968+0000","last_fullsized":"2023-10-18T12:00:22.066968+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:30:55.127165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"220'1","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091240+0000","last_change":"2023-10-18T12:00:20.067526+0000","last_active":"2023-10-18T12:00:21.091240+0000","last_peered":"2023-10-18T12:00:21.091240+0000","last_clean":"2023-10-18T12:00:21.091240+0000","last_became_active":"2023-10-18T12:00:20.067093+0000","last_became_peered":"2023-10-18T12:00:20.067093+0000","last_unstale":"2023-10-18T12:00:21.091240+0000","last_undegraded":"2023-10-18T12:00:21.091240+0000","last_fullsized":"2023-10-18T12:00:21.091240+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:08.654298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462106+0000","last_change":"2023-10-18T12:00:21.090430+0000","last_active":"2023-10-18T12:00:22.462106+0000","last_peered":"2023-10-18T12:00:22.462106+0000","last_clean":"2023-10-18T12:00:22.462106+0000","last_became_active":"2023-10-18T12:00:21.090144+0000","last_became_peered":"2023-10-18T12:00:21.090144+0000","last_unstale":"2023-10-18T12:00:22.462106+0000","last_undegraded":"2023-10-18T12:00:22.462106+0000","last_fullsized":"2023-10-18T12:00:22.462106+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:32:47.987741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488102+0000","last_change":"2023-10-18T12:00:20.063355+0000","last_active":"2023-10-18T12:00:22.488102+0000","last_peered":"2023-10-18T12:00:22.488102+0000","last_clean":"2023-10-18T12:00:22.488102+0000","last_became_active":"2023-10-18T12:00:20.063097+0000","last_became_peered":"2023-10-18T12:00:20.063097+0000","last_unstale":"2023-10-18T12:00:22.488102+0000","last_undegraded":"2023-10-18T12:00:22.488102+0000","last_fullsized":"2023-10-18T12:00:22.488102+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:13.852394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091189+0000","last_change":"2023-10-18T12:00:21.089358+0000","last_active":"2023-10-18T12:00:21.091189+0000","last_peered":"2023-10-18T12:00:21.091189+0000","last_clean":"2023-10-18T12:00:21.091189+0000","last_became_active":"2023-10-18T12:00:21.089109+0000","last_became_peered":"2023-10-18T12:00:21.089109+0000","last_unstale":"2023-10-18T12:00:21.091189+0000","last_undegraded":"2023-10-18T12:00:21.091189+0000","last_fullsized":"2023-10-18T12:00:21.091189+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:57.852277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"220'2","reported_seq":14,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488113+0000","last_change":"2023-10-18T12:00:20.058937+0000","last_active":"2023-10-18T12:00:22.488113+0000","last_peered":"2023-10-18T12:00:22.488113+0000","last_clean":"2023-10-18T12:00:22.488113+0000","last_became_active":"2023-10-18T12:00:20.058685+0000","last_became_peered":"2023-10-18T12:00:20.058685+0000","last_unstale":"2023-10-18T12:00:22.488113+0000","last_undegraded":"2023-10-18T12:00:22.488113+0000","last_fullsized":"2023-10-18T12:00:22.488113+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:07.432693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.094283+0000","last_change":"2023-10-18T12:00:21.092605+0000","last_active":"2023-10-18T12:00:21.094283+0000","last_peered":"2023-10-18T12:00:21.094283+0000","last_clean":"2023-10-18T12:00:21.094283+0000","last_became_active":"2023-10-18T12:00:21.092319+0000","last_became_peered":"2023-10-18T12:00:21.092319+0000","last_unstale":"2023-10-18T12:00:21.094283+0000","last_undegraded":"2023-10-18T12:00:21.094283+0000","last_fullsized":"2023-10-18T12:00:21.094283+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:01.961752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.068874+0000","last_change":"2023-10-18T12:00:20.062486+0000","last_active":"2023-10-18T12:00:21.068874+0000","last_peered":"2023-10-18T12:00:21.068874+0000","last_clean":"2023-10-18T12:00:21.068874+0000","last_became_active":"2023-10-18T12:00:20.062205+0000","last_became_peered":"2023-10-18T12:00:20.062205+0000","last_unstale":"2023-10-18T12:00:21.068874+0000","last_undegraded":"2023-10-18T12:00:21.068874+0000","last_fullsized":"2023-10-18T12:00:21.068874+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:01.124233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.090313+0000","last_change":"2023-10-18T12:00:21.087602+0000","last_active":"2023-10-18T12:00:21.090313+0000","last_peered":"2023-10-18T12:00:21.090313+0000","last_clean":"2023-10-18T12:00:21.090313+0000","last_became_active":"2023-10-18T12:00:21.087140+0000","last_became_peered":"2023-10-18T12:00:21.087140+0000","last_unstale":"2023-10-18T12:00:21.090313+0000","last_undegraded":"2023-10-18T12:00:21.090313+0000","last_fullsized":"2023-10-18T12:00:21.090313+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:40.365372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067886+0000","last_change":"2023-10-18T12:00:20.063488+0000","last_active":"2023-10-18T12:00:22.067886+0000","last_peered":"2023-10-18T12:00:22.067886+0000","last_clean":"2023-10-18T12:00:22.067886+0000","last_became_active":"2023-10-18T12:00:20.063220+0000","last_became_peered":"2023-10-18T12:00:20.063220+0000","last_unstale":"2023-10-18T12:00:22.067886+0000","last_undegraded":"2023-10-18T12:00:22.067886+0000","last_fullsized":"2023-10-18T12:00:22.067886+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:32.615922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462487+0000","last_change":"2023-10-18T12:00:21.094111+0000","last_active":"2023-10-18T12:00:22.462487+0000","last_peered":"2023-10-18T12:00:22.462487+0000","last_clean":"2023-10-18T12:00:22.462487+0000","last_became_active":"2023-10-18T12:00:21.093838+0000","last_became_peered":"2023-10-18T12:00:21.093838+0000","last_unstale":"2023-10-18T12:00:22.462487+0000","last_undegraded":"2023-10-18T12:00:22.462487+0000","last_fullsized":"2023-10-18T12:00:22.462487+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:57:22.028468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066571+0000","last_change":"2023-10-18T12:00:20.058023+0000","last_active":"2023-10-18T12:00:22.066571+0000","last_peered":"2023-10-18T12:00:22.066571+0000","last_clean":"2023-10-18T12:00:22.066571+0000","last_became_active":"2023-10-18T12:00:20.057731+0000","last_became_peered":"2023-10-18T12:00:20.057731+0000","last_unstale":"2023-10-18T12:00:22.066571+0000","last_undegraded":"2023-10-18T12:00:22.066571+0000","last_fullsized":"2023-10-18T12:00:22.066571+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:28.253477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461900+0000","last_change":"2023-10-18T12:00:21.092122+0000","last_active":"2023-10-18T12:00:22.461900+0000","last_peered":"2023-10-18T12:00:22.461900+0000","last_clean":"2023-10-18T12:00:22.461900+0000","last_became_active":"2023-10-18T12:00:21.091845+0000","last_became_peered":"2023-10-18T12:00:21.091845+0000","last_unstale":"2023-10-18T12:00:22.461900+0000","last_undegraded":"2023-10-18T12:00:22.461900+0000","last_fullsized":"2023-10-18T12:00:22.461900+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:31.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"220'2","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.095672+0000","last_change":"2023-10-18T12:00:20.063535+0000","last_active":"2023-10-18T12:00:21.095672+0000","last_peered":"2023-10-18T12:00:21.095672+0000","last_clean":"2023-10-18T12:00:21.095672+0000","last_became_active":"2023-10-18T12:00:20.063259+0000","last_became_peered":"2023-10-18T12:00:20.063259+0000","last_unstale":"2023-10-18T12:00:21.095672+0000","last_undegraded":"2023-10-18T12:00:21.095672+0000","last_fullsized":"2023-10-18T12:00:21.095672+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:54:11.686164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167436+0000","last_change":"2023-10-18T12:00:21.092403+0000","last_active":"2023-10-18T12:00:22.167436+0000","last_peered":"2023-10-18T12:00:22.167436+0000","last_clean":"2023-10-18T12:00:22.167436+0000","last_became_active":"2023-10-18T12:00:21.092167+0000","last_became_peered":"2023-10-18T12:00:21.092167+0000","last_unstale":"2023-10-18T12:00:22.167436+0000","last_undegraded":"2023-10-18T12:00:22.167436+0000","last_fullsized":"2023-10-18T12:00:22.167436+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:45:11.154204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462284+0000","last_change":"2023-10-18T12:00:20.063289+0000","last_active":"2023-10-18T12:00:22.462284+0000","last_peered":"2023-10-18T12:00:22.462284+0000","last_clean":"2023-10-18T12:00:22.462284+0000","last_became_active":"2023-10-18T12:00:20.063024+0000","last_became_peered":"2023-10-18T12:00:20.063024+0000","last_unstale":"2023-10-18T12:00:22.462284+0000","last_undegraded":"2023-10-18T12:00:22.462284+0000","last_fullsized":"2023-10-18T12:00:22.462284+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:54.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.080078+0000","last_change":"2023-10-18T12:00:21.076847+0000","last_active":"2023-10-18T12:00:21.080078+0000","last_peered":"2023-10-18T12:00:21.080078+0000","last_clean":"2023-10-18T12:00:21.080078+0000","last_became_active":"2023-10-18T12:00:21.076552+0000","last_became_peered":"2023-10-18T12:00:21.076552+0000","last_unstale":"2023-10-18T12:00:21.080078+0000","last_undegraded":"2023-10-18T12:00:21.080078+0000","last_fullsized":"2023-10-18T12:00:21.080078+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:01.961752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067860+0000","last_change":"2023-10-18T12:00:20.059978+0000","last_active":"2023-10-18T12:00:22.067860+0000","last_peered":"2023-10-18T12:00:22.067860+0000","last_clean":"2023-10-18T12:00:22.067860+0000","last_became_active":"2023-10-18T12:00:20.059686+0000","last_became_peered":"2023-10-18T12:00:20.059686+0000","last_unstale":"2023-10-18T12:00:22.067860+0000","last_undegraded":"2023-10-18T12:00:22.067860+0000","last_fullsized":"2023-10-18T12:00:22.067860+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:53.027447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066826+0000","last_change":"2023-10-18T12:00:21.086339+0000","last_active":"2023-10-18T12:00:22.066826+0000","last_peered":"2023-10-18T12:00:22.066826+0000","last_clean":"2023-10-18T12:00:22.066826+0000","last_became_active":"2023-10-18T12:00:21.086056+0000","last_became_peered":"2023-10-18T12:00:21.086056+0000","last_unstale":"2023-10-18T12:00:22.066826+0000","last_undegraded":"2023-10-18T12:00:22.066826+0000","last_fullsized":"2023-10-18T12:00:22.066826+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:39.387888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066981+0000","last_change":"2023-10-18T12:00:20.059094+0000","last_active":"2023-10-18T12:00:22.066981+0000","last_peered":"2023-10-18T12:00:22.066981+0000","last_clean":"2023-10-18T12:00:22.066981+0000","last_became_active":"2023-10-18T12:00:20.058785+0000","last_became_peered":"2023-10-18T12:00:20.058785+0000","last_unstale":"2023-10-18T12:00:22.066981+0000","last_undegraded":"2023-10-18T12:00:22.066981+0000","last_fullsized":"2023-10-18T12:00:22.066981+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:32.780678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462249+0000","last_change":"2023-10-18T12:00:21.090829+0000","last_active":"2023-10-18T12:00:22.462249+0000","last_peered":"2023-10-18T12:00:22.462249+0000","last_clean":"2023-10-18T12:00:22.462249+0000","last_became_active":"2023-10-18T12:00:21.090013+0000","last_became_peered":"2023-10-18T12:00:21.090013+0000","last_unstale":"2023-10-18T12:00:22.462249+0000","last_undegraded":"2023-10-18T12:00:22.462249+0000","last_fullsized":"2023-10-18T12:00:22.462249+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:14.286779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462049+0000","last_change":"2023-10-18T12:00:20.066439+0000","last_active":"2023-10-18T12:00:22.462049+0000","last_peered":"2023-10-18T12:00:22.462049+0000","last_clean":"2023-10-18T12:00:22.462049+0000","last_became_active":"2023-10-18T12:00:20.066204+0000","last_became_peered":"2023-10-18T12:00:20.066204+0000","last_unstale":"2023-10-18T12:00:22.462049+0000","last_undegraded":"2023-10-18T12:00:22.462049+0000","last_fullsized":"2023-10-18T12:00:22.462049+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:36.690391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089821+0000","last_change":"2023-10-18T12:00:21.088423+0000","last_active":"2023-10-18T12:00:21.089821+0000","last_peered":"2023-10-18T12:00:21.089821+0000","last_clean":"2023-10-18T12:00:21.089821+0000","last_became_active":"2023-10-18T12:00:21.087811+0000","last_became_peered":"2023-10-18T12:00:21.087811+0000","last_unstale":"2023-10-18T12:00:21.089821+0000","last_undegraded":"2023-10-18T12:00:21.089821+0000","last_fullsized":"2023-10-18T12:00:21.089821+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:27:51.077389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.071567+0000","last_change":"2023-10-18T12:00:20.060656+0000","last_active":"2023-10-18T12:00:21.071567+0000","last_peered":"2023-10-18T12:00:21.071567+0000","last_clean":"2023-10-18T12:00:21.071567+0000","last_became_active":"2023-10-18T12:00:20.060413+0000","last_became_peered":"2023-10-18T12:00:20.060413+0000","last_unstale":"2023-10-18T12:00:21.071567+0000","last_undegraded":"2023-10-18T12:00:21.071567+0000","last_fullsized":"2023-10-18T12:00:21.071567+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:32.716291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091923+0000","last_change":"2023-10-18T12:00:21.089842+0000","last_active":"2023-10-18T12:00:21.091923+0000","last_peered":"2023-10-18T12:00:21.091923+0000","last_clean":"2023-10-18T12:00:21.091923+0000","last_became_active":"2023-10-18T12:00:21.089584+0000","last_became_peered":"2023-10-18T12:00:21.089584+0000","last_unstale":"2023-10-18T12:00:21.091923+0000","last_undegraded":"2023-10-18T12:00:21.091923+0000","last_fullsized":"2023-10-18T12:00:21.091923+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:13.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066817+0000","last_change":"2023-10-18T12:00:20.065817+0000","last_active":"2023-10-18T12:00:22.066817+0000","last_peered":"2023-10-18T12:00:22.066817+0000","last_clean":"2023-10-18T12:00:22.066817+0000","last_became_active":"2023-10-18T12:00:20.065536+0000","last_became_peered":"2023-10-18T12:00:20.065536+0000","last_unstale":"2023-10-18T12:00:22.066817+0000","last_undegraded":"2023-10-18T12:00:22.066817+0000","last_fullsized":"2023-10-18T12:00:22.066817+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:57.379803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167642+0000","last_change":"2023-10-18T12:00:21.091679+0000","last_active":"2023-10-18T12:00:22.167642+0000","last_peered":"2023-10-18T12:00:22.167642+0000","last_clean":"2023-10-18T12:00:22.167642+0000","last_became_active":"2023-10-18T12:00:21.091412+0000","last_became_peered":"2023-10-18T12:00:21.091412+0000","last_unstale":"2023-10-18T12:00:22.167642+0000","last_undegraded":"2023-10-18T12:00:22.167642+0000","last_fullsized":"2023-10-18T12:00:22.167642+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:27.574111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487843+0000","last_change":"2023-10-18T12:00:20.066813+0000","last_active":"2023-10-18T12:00:22.487843+0000","last_peered":"2023-10-18T12:00:22.487843+0000","last_clean":"2023-10-18T12:00:22.487843+0000","last_became_active":"2023-10-18T12:00:20.066552+0000","last_became_peered":"2023-10-18T12:00:20.066552+0000","last_unstale":"2023-10-18T12:00:22.487843+0000","last_undegraded":"2023-10-18T12:00:22.487843+0000","last_fullsized":"2023-10-18T12:00:22.487843+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:22:59.208578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066682+0000","last_change":"2023-10-18T12:00:21.091052+0000","last_active":"2023-10-18T12:00:22.066682+0000","last_peered":"2023-10-18T12:00:22.066682+0000","last_clean":"2023-10-18T12:00:22.066682+0000","last_became_active":"2023-10-18T12:00:21.090785+0000","last_became_peered":"2023-10-18T12:00:21.090785+0000","last_unstale":"2023-10-18T12:00:22.066682+0000","last_undegraded":"2023-10-18T12:00:22.066682+0000","last_fullsized":"2023-10-18T12:00:22.066682+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:09.229786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"220'1","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.090136+0000","last_change":"2023-10-18T12:00:20.057601+0000","last_active":"2023-10-18T12:00:21.090136+0000","last_peered":"2023-10-18T12:00:21.090136+0000","last_clean":"2023-10-18T12:00:21.090136+0000","last_became_active":"2023-10-18T12:00:20.057294+0000","last_became_peered":"2023-10-18T12:00:20.057294+0000","last_unstale":"2023-10-18T12:00:21.090136+0000","last_undegraded":"2023-10-18T12:00:21.090136+0000","last_fullsized":"2023-10-18T12:00:21.090136+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:43.803662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"208'357","reported_seq":1232,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-18T12:00:20.062741+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:00:20.062741+0000","last_peered":"2023-10-18T12:00:20.062741+0000","last_clean":"2023-10-18T12:00:20.062741+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:00:20.062741+0000","last_undegraded":"2023-10-18T12:00:20.062741+0000","last_fullsized":"2023-10-18T12:00:20.062741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":595,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":595,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575990,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52528,"kb_used_data":3296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319312,"statfs":{"total":96636764160,"available":96582975488,"internally_reserved":0,"allocated":3375104,"data_stored":2006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.501},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.47199999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.377},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.48399999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.52900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42216,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329624,"statfs":{"total":96636764160,"available":96593534976,"internally_reserved":0,"allocated":2781184,"data_stored":1416920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.39600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45200000000000001,"5min":0.43099999999999999,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.87},"last":0.88800000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71199999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.82599999999999996},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81100000000000005,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.80400000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.78400000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.872,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70399999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.623},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85299999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.91100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575990,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60272,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311568,"statfs":{"total":96636764160,"available":96575045632,"internally_reserved":0,"allocated":2785280,"data_stored":1421458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72199999999999998,"5min":0.92400000000000004,"15min":0.96099999999999997},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.745,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.755,"5min":0.80500000000000005,"15min":1.0349999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.45000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":1},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52100000000000002,"15min":0.53800000000000003},"min":{"1min":0.377,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.01},"last":0.437},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.54000000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":1.101,"15min":1.101},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81499999999999995,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.502}]}]},{"osd":4,"up_from":13,"seq":55834575990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47788,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324052,"statfs":{"total":96636764160,"available":96587829248,"internally_reserved":0,"allocated":2781184,"data_stored":1421984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.22500000000000001},"max":{"1min":0.73899999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.33600000000000002,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.80200000000000005,"15min":0.81599999999999995},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.33600000000000002},"max":{"1min":0.86899999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":1.123,"15min":1.123},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.625,"15min":0.621},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.627},"min":{"1min":0.434,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.81999999999999995,"15min":0.83999999999999997},"last":0.59599999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.40300000000000002},"max":{"1min":0.84399999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66700000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.51100000000000001,"15min":0.498},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.443}]},{"osd":6,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.81499999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.68400000000000005,"5min":1.018,"15min":1.018},"last":0.39300000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.76100000000000001,"5min":1.004,"15min":1.05},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47792,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324048,"statfs":{"total":96636764160,"available":96587825152,"internally_reserved":0,"allocated":2777088,"data_stored":1414852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.23200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55000000000000004,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.41399999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23400000000000001},"max":{"1min":0.56699999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.30499999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41599999999999998,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.44500000000000001},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":1.899,"15min":1.899},"last":0.36599999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.29299999999999998,"15min":0.28399999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88400000000000001,"5min":1.034,"15min":1.052},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.40999999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.522,"15min":1.522},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.89600000000000002,"5min":0.91100000000000003,"15min":1.026},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":1.01,"15min":1.01},"last":0.47299999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67100000000000004,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81799999999999995,"5min":1.129,"15min":1.129},"last":0.442},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.376}]}]},{"osd":2,"up_from":13,"seq":55834575990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60252,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311588,"statfs":{"total":96636764160,"available":96575066112,"internally_reserved":0,"allocated":2768896,"data_stored":1414816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.216},"max":{"1min":0.56999999999999995,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.192},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.28599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.443},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.248}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65300000000000002,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.22600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.087,"15min":1.087},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.754},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.621}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66900000000000004,"15min":0.67400000000000004},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92000000000000004},"last":0.64000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60260,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311580,"statfs":{"total":96636764160,"available":96575057920,"internally_reserved":0,"allocated":2777088,"data_stored":1421402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.44600000000000001,"15min":0.435},"min":{"1min":0.25800000000000001,"5min":0.219,"15min":0.182},"max":{"1min":0.65200000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.379,"15min":0.38},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.23100000000000001},"max":{"1min":0.56699999999999995,"5min":0.74099999999999999,"15min":0.81000000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40699999999999997},"min":{"1min":0.34699999999999998,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.57699999999999996,"5min":0.68999999999999995,"15min":0.877},"last":0.30299999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.92000000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58799999999999997,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.46800000000000003,"5min":0.31900000000000001,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":0.82899999999999996,"15min":0.98599999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.62,"15min":0.58599999999999997},"min":{"1min":0.52500000000000002,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.77900000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.29599999999999999},"max":{"1min":0.83999999999999997,"5min":0.87,"15min":1.004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.51600000000000001,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.76100000000000001,"5min":0.91500000000000004,"15min":1.05},"last":0.627}]},{"osd":6,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.53400000000000003,"5min":0.379,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.39400000000000002},"max":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.54700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42780,"kb_used_data":3284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329060,"statfs":{"total":96636764160,"available":96592957440,"internally_reserved":0,"allocated":3362816,"data_stored":2005184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54700000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61899999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.432},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28899999999999998,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.66800000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51400000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.25700000000000001},"max":{"1min":0.62,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.56899999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.499}]},{"osd":4,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96999999999999997,"15min":0.997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.876,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88200000000000001,"5min":0.95799999999999996,"15min":0.96399999999999997},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66900000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":1.0089999999999999,"15min":1.03},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65800000000000003,"15min":0.68100000000000005},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.82699999999999996,"5min":0.997,"15min":0.997},"last":0.58399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:00:24.025 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:00:24.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.158+0000 7efda5c96640 1 -- 172.21.15.125:0/436451011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00b0530 msgr2=0x7efda00b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.158+0000 7efda5c96640 1 --2- 172.21.15.125:0/436451011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00b0530 0x7efda00b09c0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7efd98005fd0 tx=0x7efd9802c2e0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.158+0000 7efda5c96640 1 -- 172.21.15.125:0/436451011 shutdown_connections 2023-10-18T12:00:24.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.158+0000 7efda5c96640 1 --2- 172.21.15.125:0/436451011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00b0530 0x7efda00b09c0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.158+0000 7efda5c96640 1 --2- 172.21.15.125:0/436451011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda00afa20 0x7efda0158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.158+0000 7efda5c96640 1 -- 172.21.15.125:0/436451011 >> 172.21.15.125:0/436451011 conn(0x7efda0069930 msgr2=0x7efda0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:24.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.158+0000 7efda5c96640 1 -- 172.21.15.125:0/436451011 shutdown_connections 2023-10-18T12:00:24.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.158+0000 7efda5c96640 1 -- 172.21.15.125:0/436451011 wait complete. 2023-10-18T12:00:24.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.159+0000 7efda5c96640 1 Processor -- start 2023-10-18T12:00:24.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.159+0000 7efda5c96640 1 -- start start 2023-10-18T12:00:24.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efda5c96640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00afa20 0x7efda01423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efda5c96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efda00b0530 0x7efda00af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efda5c96640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda00cbd20 0x7efda01596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efda5c96640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda015a5b0 con 0x7efda00afa20 2023-10-18T12:00:24.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efda5c96640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda01428f0 con 0x7efda00b0530 2023-10-18T12:00:24.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efda5c96640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda00af8c0 con 0x7efda00cbd20 2023-10-18T12:00:24.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00afa20 0x7efda01423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00afa20 0x7efda01423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43910/0 (socket says 172.21.15.125:43910) 2023-10-18T12:00:24.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 -- 172.21.15.125:0/2074756556 learned_addr learned my addr 172.21.15.125:0/2074756556 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:24.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9effd640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efda00b0530 0x7efda00af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9ffff640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda00cbd20 0x7efda01596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 -- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda00cbd20 msgr2=0x7efda01596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda00cbd20 0x7efda01596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 -- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efda00b0530 msgr2=0x7efda00af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efda00b0530 0x7efda00af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 -- 172.21.15.125:0/2074756556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98009900 con 0x7efda00afa20 2023-10-18T12:00:24.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd9f7fe640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00afa20 0x7efda01423b0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7efd9000c400 tx=0x7efd9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:24.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd87fff640 1 -- 172.21.15.125:0/2074756556 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900052d0 con 0x7efda00afa20 2023-10-18T12:00:24.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd87fff640 1 -- 172.21.15.125:0/2074756556 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd900035f0 con 0x7efda00afa20 2023-10-18T12:00:24.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda014e8e0 con 0x7efda00afa20 2023-10-18T12:00:24.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.160+0000 7efd87fff640 1 -- 172.21.15.125:0/2074756556 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90003a00 con 0x7efda00afa20 2023-10-18T12:00:24.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.161+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda014cbc0 con 0x7efda00afa20 2023-10-18T12:00:24.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.162+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda00690e0 con 0x7efda00afa20 2023-10-18T12:00:24.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.162+0000 7efd87fff640 1 -- 172.21.15.125:0/2074756556 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efd90003b60 con 0x7efda00afa20 2023-10-18T12:00:24.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.162+0000 7efd87fff640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efd8c06af60 0x7efd8c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.162+0000 7efd87fff640 1 -- 172.21.15.125:0/2074756556 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7efd90089020 con 0x7efda00afa20 2023-10-18T12:00:24.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.163+0000 7efd9effd640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efd8c06af60 0x7efd8c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.163+0000 7efd9effd640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efd8c06af60 0x7efd8c06d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd94005fd0 tx=0x7efd94005850 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:24.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.167+0000 7efd87fff640 1 -- 172.21.15.125:0/2074756556 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd90053ed0 con 0x7efda00afa20 2023-10-18T12:00:24.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.329+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 --> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efda014ea70 con 0x7efd8c06af60 2023-10-18T12:00:24.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.336+0000 7efd87fff640 1 -- 172.21.15.125:0/2074756556 <== mgr.8786 v2:172.21.15.139:6832/2362843834 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290123 (secure 0 0 0) 0x7efd9400bca0 con 0x7efd8c06af60 2023-10-18T12:00:24.342 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:24.344 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:00:24.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efd8c06af60 msgr2=0x7efd8c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efd8c06af60 0x7efd8c06d420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd94005fd0 tx=0x7efd94005850 comp rx=0 tx=0).stop 2023-10-18T12:00:24.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00afa20 msgr2=0x7efda01423b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00afa20 0x7efda01423b0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7efd9000c400 tx=0x7efd9000c8c0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 shutdown_connections 2023-10-18T12:00:24.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efd8c06af60 0x7efd8c06d420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efda00b0530 0x7efda00af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efda00cbd20 0x7efda01596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 --2- 172.21.15.125:0/2074756556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efda00afa20 0x7efda01423b0 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 >> 172.21.15.125:0/2074756556 conn(0x7efda0069930 msgr2=0x7efda00ac930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:24.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 shutdown_connections 2023-10-18T12:00:24.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.340+0000 7efda5c96640 1 -- 172.21.15.125:0/2074756556 wait complete. 2023-10-18T12:00:24.353 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T12:00:22.799446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":639,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413888,"kb_used_data":22860,"kb_used_omap":12,"kb_used_meta":390899,"kb_avail":754560832,"statfs":{"total":773094113280,"available":772670291968,"internally_reserved":0,"allocated":23408640,"data_stored":12523392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":400281144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000993"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067038+0000","last_change":"2023-10-18T12:00:21.094990+0000","last_active":"2023-10-18T12:00:22.067038+0000","last_peered":"2023-10-18T12:00:22.067038+0000","last_clean":"2023-10-18T12:00:22.067038+0000","last_became_active":"2023-10-18T12:00:21.094239+0000","last_became_peered":"2023-10-18T12:00:21.094239+0000","last_unstale":"2023-10-18T12:00:22.067038+0000","last_undegraded":"2023-10-18T12:00:22.067038+0000","last_fullsized":"2023-10-18T12:00:22.067038+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:22.792096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091251+0000","last_change":"2023-10-18T12:00:21.089403+0000","last_active":"2023-10-18T12:00:21.091251+0000","last_peered":"2023-10-18T12:00:21.091251+0000","last_clean":"2023-10-18T12:00:21.091251+0000","last_became_active":"2023-10-18T12:00:21.089138+0000","last_became_peered":"2023-10-18T12:00:21.089138+0000","last_unstale":"2023-10-18T12:00:21.091251+0000","last_undegraded":"2023-10-18T12:00:21.091251+0000","last_fullsized":"2023-10-18T12:00:21.091251+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:16:03.472142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167524+0000","last_change":"2023-10-18T12:00:21.084830+0000","last_active":"2023-10-18T12:00:22.167524+0000","last_peered":"2023-10-18T12:00:22.167524+0000","last_clean":"2023-10-18T12:00:22.167524+0000","last_became_active":"2023-10-18T12:00:21.084546+0000","last_became_peered":"2023-10-18T12:00:21.084546+0000","last_unstale":"2023-10-18T12:00:22.167524+0000","last_undegraded":"2023-10-18T12:00:22.167524+0000","last_fullsized":"2023-10-18T12:00:22.167524+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:18.371937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089002+0000","last_change":"2023-10-18T12:00:21.087099+0000","last_active":"2023-10-18T12:00:21.089002+0000","last_peered":"2023-10-18T12:00:21.089002+0000","last_clean":"2023-10-18T12:00:21.089002+0000","last_became_active":"2023-10-18T12:00:21.086806+0000","last_became_peered":"2023-10-18T12:00:21.086806+0000","last_unstale":"2023-10-18T12:00:21.089002+0000","last_undegraded":"2023-10-18T12:00:21.089002+0000","last_fullsized":"2023-10-18T12:00:21.089002+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:50.792000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487950+0000","last_change":"2023-10-18T12:00:21.081967+0000","last_active":"2023-10-18T12:00:22.487950+0000","last_peered":"2023-10-18T12:00:22.487950+0000","last_clean":"2023-10-18T12:00:22.487950+0000","last_became_active":"2023-10-18T12:00:21.081693+0000","last_became_peered":"2023-10-18T12:00:21.081693+0000","last_unstale":"2023-10-18T12:00:22.487950+0000","last_undegraded":"2023-10-18T12:00:22.487950+0000","last_fullsized":"2023-10-18T12:00:22.487950+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:47:47.036558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.087502+0000","last_change":"2023-10-18T12:00:21.083829+0000","last_active":"2023-10-18T12:00:21.087502+0000","last_peered":"2023-10-18T12:00:21.087502+0000","last_clean":"2023-10-18T12:00:21.087502+0000","last_became_active":"2023-10-18T12:00:21.083554+0000","last_became_peered":"2023-10-18T12:00:21.083554+0000","last_unstale":"2023-10-18T12:00:21.087502+0000","last_undegraded":"2023-10-18T12:00:21.087502+0000","last_fullsized":"2023-10-18T12:00:21.087502+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:01.308646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487844+0000","last_change":"2023-10-18T12:00:21.081320+0000","last_active":"2023-10-18T12:00:22.487844+0000","last_peered":"2023-10-18T12:00:22.487844+0000","last_clean":"2023-10-18T12:00:22.487844+0000","last_became_active":"2023-10-18T12:00:21.080987+0000","last_became_peered":"2023-10-18T12:00:21.080987+0000","last_unstale":"2023-10-18T12:00:22.487844+0000","last_undegraded":"2023-10-18T12:00:22.487844+0000","last_fullsized":"2023-10-18T12:00:22.487844+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:10:51.800092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461713+0000","last_change":"2023-10-18T12:00:21.085280+0000","last_active":"2023-10-18T12:00:22.461713+0000","last_peered":"2023-10-18T12:00:22.461713+0000","last_clean":"2023-10-18T12:00:22.461713+0000","last_became_active":"2023-10-18T12:00:21.085007+0000","last_became_peered":"2023-10-18T12:00:21.085007+0000","last_unstale":"2023-10-18T12:00:22.461713+0000","last_undegraded":"2023-10-18T12:00:22.461713+0000","last_fullsized":"2023-10-18T12:00:22.461713+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:00:11.590501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.085556+0000","last_change":"2023-10-18T12:00:21.084370+0000","last_active":"2023-10-18T12:00:21.085556+0000","last_peered":"2023-10-18T12:00:21.085556+0000","last_clean":"2023-10-18T12:00:21.085556+0000","last_became_active":"2023-10-18T12:00:21.082951+0000","last_became_peered":"2023-10-18T12:00:21.082951+0000","last_unstale":"2023-10-18T12:00:21.085556+0000","last_undegraded":"2023-10-18T12:00:21.085556+0000","last_fullsized":"2023-10-18T12:00:21.085556+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:13.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462333+0000","last_change":"2023-10-18T12:00:21.091511+0000","last_active":"2023-10-18T12:00:22.462333+0000","last_peered":"2023-10-18T12:00:22.462333+0000","last_clean":"2023-10-18T12:00:22.462333+0000","last_became_active":"2023-10-18T12:00:21.091236+0000","last_became_peered":"2023-10-18T12:00:21.091236+0000","last_unstale":"2023-10-18T12:00:22.462333+0000","last_undegraded":"2023-10-18T12:00:22.462333+0000","last_fullsized":"2023-10-18T12:00:22.462333+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:27.189758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167609+0000","last_change":"2023-10-18T12:00:21.097042+0000","last_active":"2023-10-18T12:00:22.167609+0000","last_peered":"2023-10-18T12:00:22.167609+0000","last_clean":"2023-10-18T12:00:22.167609+0000","last_became_active":"2023-10-18T12:00:21.096759+0000","last_became_peered":"2023-10-18T12:00:21.096759+0000","last_unstale":"2023-10-18T12:00:22.167609+0000","last_undegraded":"2023-10-18T12:00:22.167609+0000","last_fullsized":"2023-10-18T12:00:22.167609+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:41.300175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487996+0000","last_change":"2023-10-18T12:00:21.087044+0000","last_active":"2023-10-18T12:00:22.487996+0000","last_peered":"2023-10-18T12:00:22.487996+0000","last_clean":"2023-10-18T12:00:22.487996+0000","last_became_active":"2023-10-18T12:00:21.086784+0000","last_became_peered":"2023-10-18T12:00:21.086784+0000","last_unstale":"2023-10-18T12:00:22.487996+0000","last_undegraded":"2023-10-18T12:00:22.487996+0000","last_fullsized":"2023-10-18T12:00:22.487996+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:16.398494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.087531+0000","last_change":"2023-10-18T12:00:21.084368+0000","last_active":"2023-10-18T12:00:21.087531+0000","last_peered":"2023-10-18T12:00:21.087531+0000","last_clean":"2023-10-18T12:00:21.087531+0000","last_became_active":"2023-10-18T12:00:21.084087+0000","last_became_peered":"2023-10-18T12:00:21.084087+0000","last_unstale":"2023-10-18T12:00:21.087531+0000","last_undegraded":"2023-10-18T12:00:21.087531+0000","last_fullsized":"2023-10-18T12:00:21.087531+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:31.554580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461854+0000","last_change":"2023-10-18T12:00:21.091256+0000","last_active":"2023-10-18T12:00:22.461854+0000","last_peered":"2023-10-18T12:00:22.461854+0000","last_clean":"2023-10-18T12:00:22.461854+0000","last_became_active":"2023-10-18T12:00:21.090808+0000","last_became_peered":"2023-10-18T12:00:21.090808+0000","last_unstale":"2023-10-18T12:00:22.461854+0000","last_undegraded":"2023-10-18T12:00:22.461854+0000","last_fullsized":"2023-10-18T12:00:22.461854+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:53:42.580058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.068026+0000","last_change":"2023-10-18T12:00:21.085027+0000","last_active":"2023-10-18T12:00:22.068026+0000","last_peered":"2023-10-18T12:00:22.068026+0000","last_clean":"2023-10-18T12:00:22.068026+0000","last_became_active":"2023-10-18T12:00:21.084752+0000","last_became_peered":"2023-10-18T12:00:21.084752+0000","last_unstale":"2023-10-18T12:00:22.068026+0000","last_undegraded":"2023-10-18T12:00:22.068026+0000","last_fullsized":"2023-10-18T12:00:22.068026+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:55.331050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167449+0000","last_change":"2023-10-18T12:00:21.082735+0000","last_active":"2023-10-18T12:00:22.167449+0000","last_peered":"2023-10-18T12:00:22.167449+0000","last_clean":"2023-10-18T12:00:22.167449+0000","last_became_active":"2023-10-18T12:00:21.082501+0000","last_became_peered":"2023-10-18T12:00:21.082501+0000","last_unstale":"2023-10-18T12:00:22.167449+0000","last_undegraded":"2023-10-18T12:00:22.167449+0000","last_fullsized":"2023-10-18T12:00:22.167449+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:11.830792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.090994+0000","last_change":"2023-10-18T12:00:21.089110+0000","last_active":"2023-10-18T12:00:21.090994+0000","last_peered":"2023-10-18T12:00:21.090994+0000","last_clean":"2023-10-18T12:00:21.090994+0000","last_became_active":"2023-10-18T12:00:21.088828+0000","last_became_peered":"2023-10-18T12:00:21.088828+0000","last_unstale":"2023-10-18T12:00:21.090994+0000","last_undegraded":"2023-10-18T12:00:21.090994+0000","last_fullsized":"2023-10-18T12:00:21.090994+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:23.786569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.081032+0000","last_change":"2023-10-18T12:00:21.076021+0000","last_active":"2023-10-18T12:00:21.081032+0000","last_peered":"2023-10-18T12:00:21.081032+0000","last_clean":"2023-10-18T12:00:21.081032+0000","last_became_active":"2023-10-18T12:00:21.075761+0000","last_became_peered":"2023-10-18T12:00:21.075761+0000","last_unstale":"2023-10-18T12:00:21.081032+0000","last_undegraded":"2023-10-18T12:00:21.081032+0000","last_fullsized":"2023-10-18T12:00:21.081032+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:34.556666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462373+0000","last_change":"2023-10-18T12:00:21.092451+0000","last_active":"2023-10-18T12:00:22.462373+0000","last_peered":"2023-10-18T12:00:22.462373+0000","last_clean":"2023-10-18T12:00:22.462373+0000","last_became_active":"2023-10-18T12:00:21.091995+0000","last_became_peered":"2023-10-18T12:00:21.091995+0000","last_unstale":"2023-10-18T12:00:22.462373+0000","last_undegraded":"2023-10-18T12:00:22.462373+0000","last_fullsized":"2023-10-18T12:00:22.462373+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:27.792233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462175+0000","last_change":"2023-10-18T12:00:21.090556+0000","last_active":"2023-10-18T12:00:22.462175+0000","last_peered":"2023-10-18T12:00:22.462175+0000","last_clean":"2023-10-18T12:00:22.462175+0000","last_became_active":"2023-10-18T12:00:21.088740+0000","last_became_peered":"2023-10-18T12:00:21.088740+0000","last_unstale":"2023-10-18T12:00:22.462175+0000","last_undegraded":"2023-10-18T12:00:22.462175+0000","last_fullsized":"2023-10-18T12:00:22.462175+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:32.899439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461891+0000","last_change":"2023-10-18T12:00:21.093298+0000","last_active":"2023-10-18T12:00:22.461891+0000","last_peered":"2023-10-18T12:00:22.461891+0000","last_clean":"2023-10-18T12:00:22.461891+0000","last_became_active":"2023-10-18T12:00:21.092815+0000","last_became_peered":"2023-10-18T12:00:21.092815+0000","last_unstale":"2023-10-18T12:00:22.461891+0000","last_undegraded":"2023-10-18T12:00:22.461891+0000","last_fullsized":"2023-10-18T12:00:22.461891+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:28.526624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461867+0000","last_change":"2023-10-18T12:00:21.091079+0000","last_active":"2023-10-18T12:00:22.461867+0000","last_peered":"2023-10-18T12:00:22.461867+0000","last_clean":"2023-10-18T12:00:22.461867+0000","last_became_active":"2023-10-18T12:00:21.090440+0000","last_became_peered":"2023-10-18T12:00:21.090440+0000","last_unstale":"2023-10-18T12:00:22.461867+0000","last_undegraded":"2023-10-18T12:00:22.461867+0000","last_fullsized":"2023-10-18T12:00:22.461867+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:54.338893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.079346+0000","last_change":"2023-10-18T12:00:21.073828+0000","last_active":"2023-10-18T12:00:21.079346+0000","last_peered":"2023-10-18T12:00:21.079346+0000","last_clean":"2023-10-18T12:00:21.079346+0000","last_became_active":"2023-10-18T12:00:21.073594+0000","last_became_peered":"2023-10-18T12:00:21.073594+0000","last_unstale":"2023-10-18T12:00:21.079346+0000","last_undegraded":"2023-10-18T12:00:21.079346+0000","last_fullsized":"2023-10-18T12:00:21.079346+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:36.596128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487805+0000","last_change":"2023-10-18T12:00:21.082982+0000","last_active":"2023-10-18T12:00:22.487805+0000","last_peered":"2023-10-18T12:00:22.487805+0000","last_clean":"2023-10-18T12:00:22.487805+0000","last_became_active":"2023-10-18T12:00:21.082700+0000","last_became_peered":"2023-10-18T12:00:21.082700+0000","last_unstale":"2023-10-18T12:00:22.487805+0000","last_undegraded":"2023-10-18T12:00:22.487805+0000","last_fullsized":"2023-10-18T12:00:22.487805+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:25.713305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.081472+0000","last_change":"2023-10-18T12:00:21.075391+0000","last_active":"2023-10-18T12:00:21.081472+0000","last_peered":"2023-10-18T12:00:21.081472+0000","last_clean":"2023-10-18T12:00:21.081472+0000","last_became_active":"2023-10-18T12:00:21.075140+0000","last_became_peered":"2023-10-18T12:00:21.075140+0000","last_unstale":"2023-10-18T12:00:21.081472+0000","last_undegraded":"2023-10-18T12:00:21.081472+0000","last_fullsized":"2023-10-18T12:00:21.081472+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:15.346268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067253+0000","last_change":"2023-10-18T12:00:21.091634+0000","last_active":"2023-10-18T12:00:22.067253+0000","last_peered":"2023-10-18T12:00:22.067253+0000","last_clean":"2023-10-18T12:00:22.067253+0000","last_became_active":"2023-10-18T12:00:21.091327+0000","last_became_peered":"2023-10-18T12:00:21.091327+0000","last_unstale":"2023-10-18T12:00:22.067253+0000","last_undegraded":"2023-10-18T12:00:22.067253+0000","last_fullsized":"2023-10-18T12:00:22.067253+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:53:58.179224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167625+0000","last_change":"2023-10-18T12:00:21.093406+0000","last_active":"2023-10-18T12:00:22.167625+0000","last_peered":"2023-10-18T12:00:22.167625+0000","last_clean":"2023-10-18T12:00:22.167625+0000","last_became_active":"2023-10-18T12:00:21.093138+0000","last_became_peered":"2023-10-18T12:00:21.093138+0000","last_unstale":"2023-10-18T12:00:22.167625+0000","last_undegraded":"2023-10-18T12:00:22.167625+0000","last_fullsized":"2023-10-18T12:00:22.167625+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:09.568953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488124+0000","last_change":"2023-10-18T12:00:21.086100+0000","last_active":"2023-10-18T12:00:22.488124+0000","last_peered":"2023-10-18T12:00:22.488124+0000","last_clean":"2023-10-18T12:00:22.488124+0000","last_became_active":"2023-10-18T12:00:21.085820+0000","last_became_peered":"2023-10-18T12:00:21.085820+0000","last_unstale":"2023-10-18T12:00:22.488124+0000","last_undegraded":"2023-10-18T12:00:22.488124+0000","last_fullsized":"2023-10-18T12:00:22.488124+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:59.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.085463+0000","last_change":"2023-10-18T12:00:21.082792+0000","last_active":"2023-10-18T12:00:21.085463+0000","last_peered":"2023-10-18T12:00:21.085463+0000","last_clean":"2023-10-18T12:00:21.085463+0000","last_became_active":"2023-10-18T12:00:21.082529+0000","last_became_peered":"2023-10-18T12:00:21.082529+0000","last_unstale":"2023-10-18T12:00:21.085463+0000","last_undegraded":"2023-10-18T12:00:21.085463+0000","last_fullsized":"2023-10-18T12:00:21.085463+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:15.212784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067873+0000","last_change":"2023-10-18T12:00:21.087421+0000","last_active":"2023-10-18T12:00:22.067873+0000","last_peered":"2023-10-18T12:00:22.067873+0000","last_clean":"2023-10-18T12:00:22.067873+0000","last_became_active":"2023-10-18T12:00:21.087145+0000","last_became_peered":"2023-10-18T12:00:21.087145+0000","last_unstale":"2023-10-18T12:00:22.067873+0000","last_undegraded":"2023-10-18T12:00:22.067873+0000","last_fullsized":"2023-10-18T12:00:22.067873+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:28.489188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.093430+0000","last_change":"2023-10-18T12:00:21.092324+0000","last_active":"2023-10-18T12:00:21.093430+0000","last_peered":"2023-10-18T12:00:21.093430+0000","last_clean":"2023-10-18T12:00:21.093430+0000","last_became_active":"2023-10-18T12:00:21.092059+0000","last_became_peered":"2023-10-18T12:00:21.092059+0000","last_unstale":"2023-10-18T12:00:21.093430+0000","last_undegraded":"2023-10-18T12:00:21.093430+0000","last_fullsized":"2023-10-18T12:00:21.093430+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:58.467913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.094347+0000","last_change":"2023-10-18T12:00:21.093224+0000","last_active":"2023-10-18T12:00:21.094347+0000","last_peered":"2023-10-18T12:00:21.094347+0000","last_clean":"2023-10-18T12:00:21.094347+0000","last_became_active":"2023-10-18T12:00:21.092949+0000","last_became_peered":"2023-10-18T12:00:21.092949+0000","last_unstale":"2023-10-18T12:00:21.094347+0000","last_undegraded":"2023-10-18T12:00:21.094347+0000","last_fullsized":"2023-10-18T12:00:21.094347+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:40.263149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.082066+0000","last_change":"2023-10-18T12:00:21.073279+0000","last_active":"2023-10-18T12:00:21.082066+0000","last_peered":"2023-10-18T12:00:21.082066+0000","last_clean":"2023-10-18T12:00:21.082066+0000","last_became_active":"2023-10-18T12:00:21.073036+0000","last_became_peered":"2023-10-18T12:00:21.073036+0000","last_unstale":"2023-10-18T12:00:21.082066+0000","last_undegraded":"2023-10-18T12:00:21.082066+0000","last_fullsized":"2023-10-18T12:00:21.082066+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:03.802023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"220'4","reported_seq":16,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.068145+0000","last_change":"2023-10-18T12:00:20.063561+0000","last_active":"2023-10-18T12:00:22.068145+0000","last_peered":"2023-10-18T12:00:22.068145+0000","last_clean":"2023-10-18T12:00:22.068145+0000","last_became_active":"2023-10-18T12:00:20.063285+0000","last_became_peered":"2023-10-18T12:00:20.063285+0000","last_unstale":"2023-10-18T12:00:22.068145+0000","last_undegraded":"2023-10-18T12:00:22.068145+0000","last_fullsized":"2023-10-18T12:00:22.068145+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:55.051193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066637+0000","last_change":"2023-10-18T12:00:21.073385+0000","last_active":"2023-10-18T12:00:22.066637+0000","last_peered":"2023-10-18T12:00:22.066637+0000","last_clean":"2023-10-18T12:00:22.066637+0000","last_became_active":"2023-10-18T12:00:21.073048+0000","last_became_peered":"2023-10-18T12:00:21.073048+0000","last_unstale":"2023-10-18T12:00:22.066637+0000","last_undegraded":"2023-10-18T12:00:22.066637+0000","last_fullsized":"2023-10-18T12:00:22.066637+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:23:54.891459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"220'1","reported_seq":13,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067313+0000","last_change":"2023-10-18T12:00:20.057976+0000","last_active":"2023-10-18T12:00:22.067313+0000","last_peered":"2023-10-18T12:00:22.067313+0000","last_clean":"2023-10-18T12:00:22.067313+0000","last_became_active":"2023-10-18T12:00:20.057699+0000","last_became_peered":"2023-10-18T12:00:20.057699+0000","last_unstale":"2023-10-18T12:00:22.067313+0000","last_undegraded":"2023-10-18T12:00:22.067313+0000","last_fullsized":"2023-10-18T12:00:22.067313+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:45.951679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462070+0000","last_change":"2023-10-18T12:00:21.093491+0000","last_active":"2023-10-18T12:00:22.462070+0000","last_peered":"2023-10-18T12:00:22.462070+0000","last_clean":"2023-10-18T12:00:22.462070+0000","last_became_active":"2023-10-18T12:00:21.093199+0000","last_became_peered":"2023-10-18T12:00:21.093199+0000","last_unstale":"2023-10-18T12:00:22.462070+0000","last_undegraded":"2023-10-18T12:00:22.462070+0000","last_fullsized":"2023-10-18T12:00:22.462070+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:14.227710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"220'2","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.090216+0000","last_change":"2023-10-18T12:00:20.057262+0000","last_active":"2023-10-18T12:00:21.090216+0000","last_peered":"2023-10-18T12:00:21.090216+0000","last_clean":"2023-10-18T12:00:21.090216+0000","last_became_active":"2023-10-18T12:00:20.057010+0000","last_became_peered":"2023-10-18T12:00:20.057010+0000","last_unstale":"2023-10-18T12:00:21.090216+0000","last_undegraded":"2023-10-18T12:00:21.090216+0000","last_fullsized":"2023-10-18T12:00:21.090216+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:46.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487995+0000","last_change":"2023-10-18T12:00:21.089013+0000","last_active":"2023-10-18T12:00:22.487995+0000","last_peered":"2023-10-18T12:00:22.487995+0000","last_clean":"2023-10-18T12:00:22.487995+0000","last_became_active":"2023-10-18T12:00:21.088643+0000","last_became_peered":"2023-10-18T12:00:21.088643+0000","last_unstale":"2023-10-18T12:00:22.487995+0000","last_undegraded":"2023-10-18T12:00:22.487995+0000","last_fullsized":"2023-10-18T12:00:22.487995+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:15.272122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"220'2","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089377+0000","last_change":"2023-10-18T12:00:20.060797+0000","last_active":"2023-10-18T12:00:21.089377+0000","last_peered":"2023-10-18T12:00:21.089377+0000","last_clean":"2023-10-18T12:00:21.089377+0000","last_became_active":"2023-10-18T12:00:20.060568+0000","last_became_peered":"2023-10-18T12:00:20.060568+0000","last_unstale":"2023-10-18T12:00:21.089377+0000","last_undegraded":"2023-10-18T12:00:21.089377+0000","last_fullsized":"2023-10-18T12:00:21.089377+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:27:50.071689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.068083+0000","last_change":"2023-10-18T12:00:21.087738+0000","last_active":"2023-10-18T12:00:22.068083+0000","last_peered":"2023-10-18T12:00:22.068083+0000","last_clean":"2023-10-18T12:00:22.068083+0000","last_became_active":"2023-10-18T12:00:21.087104+0000","last_became_peered":"2023-10-18T12:00:21.087104+0000","last_unstale":"2023-10-18T12:00:22.068083+0000","last_undegraded":"2023-10-18T12:00:22.068083+0000","last_fullsized":"2023-10-18T12:00:22.068083+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:52.609784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"220'1","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089141+0000","last_change":"2023-10-18T12:00:20.070416+0000","last_active":"2023-10-18T12:00:21.089141+0000","last_peered":"2023-10-18T12:00:21.089141+0000","last_clean":"2023-10-18T12:00:21.089141+0000","last_became_active":"2023-10-18T12:00:20.070121+0000","last_became_peered":"2023-10-18T12:00:20.070121+0000","last_unstale":"2023-10-18T12:00:21.089141+0000","last_undegraded":"2023-10-18T12:00:21.089141+0000","last_fullsized":"2023-10-18T12:00:21.089141+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:33.588701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.068148+0000","last_change":"2023-10-18T12:00:21.081350+0000","last_active":"2023-10-18T12:00:22.068148+0000","last_peered":"2023-10-18T12:00:22.068148+0000","last_clean":"2023-10-18T12:00:22.068148+0000","last_became_active":"2023-10-18T12:00:21.080920+0000","last_became_peered":"2023-10-18T12:00:21.080920+0000","last_unstale":"2023-10-18T12:00:22.068148+0000","last_undegraded":"2023-10-18T12:00:22.068148+0000","last_fullsized":"2023-10-18T12:00:22.068148+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:37.696091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.077114+0000","last_change":"2023-10-18T12:00:20.070314+0000","last_active":"2023-10-18T12:00:21.077114+0000","last_peered":"2023-10-18T12:00:21.077114+0000","last_clean":"2023-10-18T12:00:21.077114+0000","last_became_active":"2023-10-18T12:00:20.069974+0000","last_became_peered":"2023-10-18T12:00:20.069974+0000","last_unstale":"2023-10-18T12:00:21.077114+0000","last_undegraded":"2023-10-18T12:00:21.077114+0000","last_fullsized":"2023-10-18T12:00:21.077114+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:39.359672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.081495+0000","last_change":"2023-10-18T12:00:21.076517+0000","last_active":"2023-10-18T12:00:21.081495+0000","last_peered":"2023-10-18T12:00:21.081495+0000","last_clean":"2023-10-18T12:00:21.081495+0000","last_became_active":"2023-10-18T12:00:21.076276+0000","last_became_peered":"2023-10-18T12:00:21.076276+0000","last_unstale":"2023-10-18T12:00:21.081495+0000","last_undegraded":"2023-10-18T12:00:21.081495+0000","last_fullsized":"2023-10-18T12:00:21.081495+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:26.534619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167494+0000","last_change":"2023-10-18T12:00:20.067947+0000","last_active":"2023-10-18T12:00:22.167494+0000","last_peered":"2023-10-18T12:00:22.167494+0000","last_clean":"2023-10-18T12:00:22.167494+0000","last_became_active":"2023-10-18T12:00:20.067433+0000","last_became_peered":"2023-10-18T12:00:20.067433+0000","last_unstale":"2023-10-18T12:00:22.167494+0000","last_undegraded":"2023-10-18T12:00:22.167494+0000","last_fullsized":"2023-10-18T12:00:22.167494+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:38.382188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067730+0000","last_change":"2023-10-18T12:00:21.079285+0000","last_active":"2023-10-18T12:00:22.067730+0000","last_peered":"2023-10-18T12:00:22.067730+0000","last_clean":"2023-10-18T12:00:22.067730+0000","last_became_active":"2023-10-18T12:00:21.079002+0000","last_became_peered":"2023-10-18T12:00:21.079002+0000","last_unstale":"2023-10-18T12:00:22.067730+0000","last_undegraded":"2023-10-18T12:00:22.067730+0000","last_fullsized":"2023-10-18T12:00:22.067730+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:49:28.849697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488078+0000","last_change":"2023-10-18T12:00:20.068018+0000","last_active":"2023-10-18T12:00:22.488078+0000","last_peered":"2023-10-18T12:00:22.488078+0000","last_clean":"2023-10-18T12:00:22.488078+0000","last_became_active":"2023-10-18T12:00:20.067744+0000","last_became_peered":"2023-10-18T12:00:20.067744+0000","last_unstale":"2023-10-18T12:00:22.488078+0000","last_undegraded":"2023-10-18T12:00:22.488078+0000","last_fullsized":"2023-10-18T12:00:22.488078+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:17.572716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167507+0000","last_change":"2023-10-18T12:00:21.078356+0000","last_active":"2023-10-18T12:00:22.167507+0000","last_peered":"2023-10-18T12:00:22.167507+0000","last_clean":"2023-10-18T12:00:22.167507+0000","last_became_active":"2023-10-18T12:00:21.078095+0000","last_became_peered":"2023-10-18T12:00:21.078095+0000","last_unstale":"2023-10-18T12:00:22.167507+0000","last_undegraded":"2023-10-18T12:00:22.167507+0000","last_fullsized":"2023-10-18T12:00:22.167507+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:44.526073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.078461+0000","last_change":"2023-10-18T12:00:20.070057+0000","last_active":"2023-10-18T12:00:21.078461+0000","last_peered":"2023-10-18T12:00:21.078461+0000","last_clean":"2023-10-18T12:00:21.078461+0000","last_became_active":"2023-10-18T12:00:20.069456+0000","last_became_peered":"2023-10-18T12:00:20.069456+0000","last_unstale":"2023-10-18T12:00:21.078461+0000","last_undegraded":"2023-10-18T12:00:21.078461+0000","last_fullsized":"2023-10-18T12:00:21.078461+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:14.207084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.077097+0000","last_change":"2023-10-18T12:00:21.074575+0000","last_active":"2023-10-18T12:00:21.077097+0000","last_peered":"2023-10-18T12:00:21.077097+0000","last_clean":"2023-10-18T12:00:21.077097+0000","last_became_active":"2023-10-18T12:00:21.074302+0000","last_became_peered":"2023-10-18T12:00:21.074302+0000","last_unstale":"2023-10-18T12:00:21.077097+0000","last_undegraded":"2023-10-18T12:00:21.077097+0000","last_fullsized":"2023-10-18T12:00:21.077097+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:58.467913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"220'3","reported_seq":14,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089992+0000","last_change":"2023-10-18T12:00:20.057436+0000","last_active":"2023-10-18T12:00:21.089992+0000","last_peered":"2023-10-18T12:00:21.089992+0000","last_clean":"2023-10-18T12:00:21.089992+0000","last_became_active":"2023-10-18T12:00:20.057117+0000","last_became_peered":"2023-10-18T12:00:20.057117+0000","last_unstale":"2023-10-18T12:00:21.089992+0000","last_undegraded":"2023-10-18T12:00:21.089992+0000","last_fullsized":"2023-10-18T12:00:21.089992+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:02.796323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488093+0000","last_change":"2023-10-18T12:00:21.092364+0000","last_active":"2023-10-18T12:00:22.488093+0000","last_peered":"2023-10-18T12:00:22.488093+0000","last_clean":"2023-10-18T12:00:22.488093+0000","last_became_active":"2023-10-18T12:00:21.092090+0000","last_became_peered":"2023-10-18T12:00:21.092090+0000","last_unstale":"2023-10-18T12:00:22.488093+0000","last_undegraded":"2023-10-18T12:00:22.488093+0000","last_fullsized":"2023-10-18T12:00:22.488093+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:19.318183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"220'6","reported_seq":17,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.097622+0000","last_change":"2023-10-18T12:00:20.070213+0000","last_active":"2023-10-18T12:00:21.097622+0000","last_peered":"2023-10-18T12:00:21.097622+0000","last_clean":"2023-10-18T12:00:21.097622+0000","last_became_active":"2023-10-18T12:00:20.069888+0000","last_became_peered":"2023-10-18T12:00:20.069888+0000","last_unstale":"2023-10-18T12:00:21.097622+0000","last_undegraded":"2023-10-18T12:00:21.097622+0000","last_fullsized":"2023-10-18T12:00:21.097622+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:55.247225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.080305+0000","last_change":"2023-10-18T12:00:21.078685+0000","last_active":"2023-10-18T12:00:21.080305+0000","last_peered":"2023-10-18T12:00:21.080305+0000","last_clean":"2023-10-18T12:00:21.080305+0000","last_became_active":"2023-10-18T12:00:21.078434+0000","last_became_peered":"2023-10-18T12:00:21.078434+0000","last_unstale":"2023-10-18T12:00:21.080305+0000","last_undegraded":"2023-10-18T12:00:21.080305+0000","last_fullsized":"2023-10-18T12:00:21.080305+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:42:38.614585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"220'2","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091496+0000","last_change":"2023-10-18T12:00:20.063126+0000","last_active":"2023-10-18T12:00:21.091496+0000","last_peered":"2023-10-18T12:00:21.091496+0000","last_clean":"2023-10-18T12:00:21.091496+0000","last_became_active":"2023-10-18T12:00:20.062833+0000","last_became_peered":"2023-10-18T12:00:20.062833+0000","last_unstale":"2023-10-18T12:00:21.091496+0000","last_undegraded":"2023-10-18T12:00:21.091496+0000","last_fullsized":"2023-10-18T12:00:21.091496+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:57.198338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167686+0000","last_change":"2023-10-18T12:00:21.075390+0000","last_active":"2023-10-18T12:00:22.167686+0000","last_peered":"2023-10-18T12:00:22.167686+0000","last_clean":"2023-10-18T12:00:22.167686+0000","last_became_active":"2023-10-18T12:00:21.075149+0000","last_became_peered":"2023-10-18T12:00:21.075149+0000","last_unstale":"2023-10-18T12:00:22.167686+0000","last_undegraded":"2023-10-18T12:00:22.167686+0000","last_fullsized":"2023-10-18T12:00:22.167686+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:00.806334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167467+0000","last_change":"2023-10-18T12:00:20.059890+0000","last_active":"2023-10-18T12:00:22.167467+0000","last_peered":"2023-10-18T12:00:22.167467+0000","last_clean":"2023-10-18T12:00:22.167467+0000","last_became_active":"2023-10-18T12:00:20.059647+0000","last_became_peered":"2023-10-18T12:00:20.059647+0000","last_unstale":"2023-10-18T12:00:22.167467+0000","last_undegraded":"2023-10-18T12:00:22.167467+0000","last_fullsized":"2023-10-18T12:00:22.167467+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:08.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.084681+0000","last_change":"2023-10-18T12:00:21.081617+0000","last_active":"2023-10-18T12:00:21.084681+0000","last_peered":"2023-10-18T12:00:21.084681+0000","last_clean":"2023-10-18T12:00:21.084681+0000","last_became_active":"2023-10-18T12:00:21.081232+0000","last_became_peered":"2023-10-18T12:00:21.081232+0000","last_unstale":"2023-10-18T12:00:21.084681+0000","last_undegraded":"2023-10-18T12:00:21.084681+0000","last_fullsized":"2023-10-18T12:00:21.084681+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:40.263149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.069951+0000","last_change":"2023-10-18T12:00:20.061573+0000","last_active":"2023-10-18T12:00:21.069951+0000","last_peered":"2023-10-18T12:00:21.069951+0000","last_clean":"2023-10-18T12:00:21.069951+0000","last_became_active":"2023-10-18T12:00:20.061363+0000","last_became_peered":"2023-10-18T12:00:20.061363+0000","last_unstale":"2023-10-18T12:00:21.069951+0000","last_undegraded":"2023-10-18T12:00:21.069951+0000","last_fullsized":"2023-10-18T12:00:21.069951+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:33.588701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.088949+0000","last_change":"2023-10-18T12:00:21.086868+0000","last_active":"2023-10-18T12:00:21.088949+0000","last_peered":"2023-10-18T12:00:21.088949+0000","last_clean":"2023-10-18T12:00:21.088949+0000","last_became_active":"2023-10-18T12:00:21.086595+0000","last_became_peered":"2023-10-18T12:00:21.086595+0000","last_unstale":"2023-10-18T12:00:21.088949+0000","last_undegraded":"2023-10-18T12:00:21.088949+0000","last_fullsized":"2023-10-18T12:00:21.088949+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:45.002374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"220'8","reported_seq":20,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487751+0000","last_change":"2023-10-18T12:00:20.066867+0000","last_active":"2023-10-18T12:00:22.487751+0000","last_peered":"2023-10-18T12:00:22.487751+0000","last_clean":"2023-10-18T12:00:22.487751+0000","last_became_active":"2023-10-18T12:00:20.066615+0000","last_became_peered":"2023-10-18T12:00:20.066615+0000","last_unstale":"2023-10-18T12:00:22.487751+0000","last_undegraded":"2023-10-18T12:00:22.487751+0000","last_fullsized":"2023-10-18T12:00:22.487751+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:31.444571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461760+0000","last_change":"2023-10-18T12:00:21.092578+0000","last_active":"2023-10-18T12:00:22.461760+0000","last_peered":"2023-10-18T12:00:22.461760+0000","last_clean":"2023-10-18T12:00:22.461760+0000","last_became_active":"2023-10-18T12:00:21.092290+0000","last_became_peered":"2023-10-18T12:00:21.092290+0000","last_unstale":"2023-10-18T12:00:22.461760+0000","last_undegraded":"2023-10-18T12:00:22.461760+0000","last_fullsized":"2023-10-18T12:00:22.461760+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:05.871282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"220'1","reported_seq":13,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067117+0000","last_change":"2023-10-18T12:00:20.058912+0000","last_active":"2023-10-18T12:00:22.067117+0000","last_peered":"2023-10-18T12:00:22.067117+0000","last_clean":"2023-10-18T12:00:22.067117+0000","last_became_active":"2023-10-18T12:00:20.058634+0000","last_became_peered":"2023-10-18T12:00:20.058634+0000","last_unstale":"2023-10-18T12:00:22.067117+0000","last_undegraded":"2023-10-18T12:00:22.067117+0000","last_fullsized":"2023-10-18T12:00:22.067117+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:06.254577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066788+0000","last_change":"2023-10-18T12:00:21.090572+0000","last_active":"2023-10-18T12:00:22.066788+0000","last_peered":"2023-10-18T12:00:22.066788+0000","last_clean":"2023-10-18T12:00:22.066788+0000","last_became_active":"2023-10-18T12:00:21.090312+0000","last_became_peered":"2023-10-18T12:00:21.090312+0000","last_unstale":"2023-10-18T12:00:22.066788+0000","last_undegraded":"2023-10-18T12:00:22.066788+0000","last_fullsized":"2023-10-18T12:00:22.066788+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:00:29.845435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"220'2","reported_seq":14,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067347+0000","last_change":"2023-10-18T12:00:20.067607+0000","last_active":"2023-10-18T12:00:22.067347+0000","last_peered":"2023-10-18T12:00:22.067347+0000","last_clean":"2023-10-18T12:00:22.067347+0000","last_became_active":"2023-10-18T12:00:20.067298+0000","last_became_peered":"2023-10-18T12:00:20.067298+0000","last_unstale":"2023-10-18T12:00:22.067347+0000","last_undegraded":"2023-10-18T12:00:22.067347+0000","last_fullsized":"2023-10-18T12:00:22.067347+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.982605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066968+0000","last_change":"2023-10-18T12:00:21.074593+0000","last_active":"2023-10-18T12:00:22.066968+0000","last_peered":"2023-10-18T12:00:22.066968+0000","last_clean":"2023-10-18T12:00:22.066968+0000","last_became_active":"2023-10-18T12:00:21.074328+0000","last_became_peered":"2023-10-18T12:00:21.074328+0000","last_unstale":"2023-10-18T12:00:22.066968+0000","last_undegraded":"2023-10-18T12:00:22.066968+0000","last_fullsized":"2023-10-18T12:00:22.066968+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:30:55.127165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"220'1","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091240+0000","last_change":"2023-10-18T12:00:20.067526+0000","last_active":"2023-10-18T12:00:21.091240+0000","last_peered":"2023-10-18T12:00:21.091240+0000","last_clean":"2023-10-18T12:00:21.091240+0000","last_became_active":"2023-10-18T12:00:20.067093+0000","last_became_peered":"2023-10-18T12:00:20.067093+0000","last_unstale":"2023-10-18T12:00:21.091240+0000","last_undegraded":"2023-10-18T12:00:21.091240+0000","last_fullsized":"2023-10-18T12:00:21.091240+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:08.654298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462106+0000","last_change":"2023-10-18T12:00:21.090430+0000","last_active":"2023-10-18T12:00:22.462106+0000","last_peered":"2023-10-18T12:00:22.462106+0000","last_clean":"2023-10-18T12:00:22.462106+0000","last_became_active":"2023-10-18T12:00:21.090144+0000","last_became_peered":"2023-10-18T12:00:21.090144+0000","last_unstale":"2023-10-18T12:00:22.462106+0000","last_undegraded":"2023-10-18T12:00:22.462106+0000","last_fullsized":"2023-10-18T12:00:22.462106+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:32:47.987741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488102+0000","last_change":"2023-10-18T12:00:20.063355+0000","last_active":"2023-10-18T12:00:22.488102+0000","last_peered":"2023-10-18T12:00:22.488102+0000","last_clean":"2023-10-18T12:00:22.488102+0000","last_became_active":"2023-10-18T12:00:20.063097+0000","last_became_peered":"2023-10-18T12:00:20.063097+0000","last_unstale":"2023-10-18T12:00:22.488102+0000","last_undegraded":"2023-10-18T12:00:22.488102+0000","last_fullsized":"2023-10-18T12:00:22.488102+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:13.852394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091189+0000","last_change":"2023-10-18T12:00:21.089358+0000","last_active":"2023-10-18T12:00:21.091189+0000","last_peered":"2023-10-18T12:00:21.091189+0000","last_clean":"2023-10-18T12:00:21.091189+0000","last_became_active":"2023-10-18T12:00:21.089109+0000","last_became_peered":"2023-10-18T12:00:21.089109+0000","last_unstale":"2023-10-18T12:00:21.091189+0000","last_undegraded":"2023-10-18T12:00:21.091189+0000","last_fullsized":"2023-10-18T12:00:21.091189+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:57.852277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"220'2","reported_seq":14,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.488113+0000","last_change":"2023-10-18T12:00:20.058937+0000","last_active":"2023-10-18T12:00:22.488113+0000","last_peered":"2023-10-18T12:00:22.488113+0000","last_clean":"2023-10-18T12:00:22.488113+0000","last_became_active":"2023-10-18T12:00:20.058685+0000","last_became_peered":"2023-10-18T12:00:20.058685+0000","last_unstale":"2023-10-18T12:00:22.488113+0000","last_undegraded":"2023-10-18T12:00:22.488113+0000","last_fullsized":"2023-10-18T12:00:22.488113+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:07.432693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.094283+0000","last_change":"2023-10-18T12:00:21.092605+0000","last_active":"2023-10-18T12:00:21.094283+0000","last_peered":"2023-10-18T12:00:21.094283+0000","last_clean":"2023-10-18T12:00:21.094283+0000","last_became_active":"2023-10-18T12:00:21.092319+0000","last_became_peered":"2023-10-18T12:00:21.092319+0000","last_unstale":"2023-10-18T12:00:21.094283+0000","last_undegraded":"2023-10-18T12:00:21.094283+0000","last_fullsized":"2023-10-18T12:00:21.094283+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:01.961752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.068874+0000","last_change":"2023-10-18T12:00:20.062486+0000","last_active":"2023-10-18T12:00:21.068874+0000","last_peered":"2023-10-18T12:00:21.068874+0000","last_clean":"2023-10-18T12:00:21.068874+0000","last_became_active":"2023-10-18T12:00:20.062205+0000","last_became_peered":"2023-10-18T12:00:20.062205+0000","last_unstale":"2023-10-18T12:00:21.068874+0000","last_undegraded":"2023-10-18T12:00:21.068874+0000","last_fullsized":"2023-10-18T12:00:21.068874+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:01.124233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.090313+0000","last_change":"2023-10-18T12:00:21.087602+0000","last_active":"2023-10-18T12:00:21.090313+0000","last_peered":"2023-10-18T12:00:21.090313+0000","last_clean":"2023-10-18T12:00:21.090313+0000","last_became_active":"2023-10-18T12:00:21.087140+0000","last_became_peered":"2023-10-18T12:00:21.087140+0000","last_unstale":"2023-10-18T12:00:21.090313+0000","last_undegraded":"2023-10-18T12:00:21.090313+0000","last_fullsized":"2023-10-18T12:00:21.090313+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:40.365372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067886+0000","last_change":"2023-10-18T12:00:20.063488+0000","last_active":"2023-10-18T12:00:22.067886+0000","last_peered":"2023-10-18T12:00:22.067886+0000","last_clean":"2023-10-18T12:00:22.067886+0000","last_became_active":"2023-10-18T12:00:20.063220+0000","last_became_peered":"2023-10-18T12:00:20.063220+0000","last_unstale":"2023-10-18T12:00:22.067886+0000","last_undegraded":"2023-10-18T12:00:22.067886+0000","last_fullsized":"2023-10-18T12:00:22.067886+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:32.615922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462487+0000","last_change":"2023-10-18T12:00:21.094111+0000","last_active":"2023-10-18T12:00:22.462487+0000","last_peered":"2023-10-18T12:00:22.462487+0000","last_clean":"2023-10-18T12:00:22.462487+0000","last_became_active":"2023-10-18T12:00:21.093838+0000","last_became_peered":"2023-10-18T12:00:21.093838+0000","last_unstale":"2023-10-18T12:00:22.462487+0000","last_undegraded":"2023-10-18T12:00:22.462487+0000","last_fullsized":"2023-10-18T12:00:22.462487+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:57:22.028468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066571+0000","last_change":"2023-10-18T12:00:20.058023+0000","last_active":"2023-10-18T12:00:22.066571+0000","last_peered":"2023-10-18T12:00:22.066571+0000","last_clean":"2023-10-18T12:00:22.066571+0000","last_became_active":"2023-10-18T12:00:20.057731+0000","last_became_peered":"2023-10-18T12:00:20.057731+0000","last_unstale":"2023-10-18T12:00:22.066571+0000","last_undegraded":"2023-10-18T12:00:22.066571+0000","last_fullsized":"2023-10-18T12:00:22.066571+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:28.253477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.461900+0000","last_change":"2023-10-18T12:00:21.092122+0000","last_active":"2023-10-18T12:00:22.461900+0000","last_peered":"2023-10-18T12:00:22.461900+0000","last_clean":"2023-10-18T12:00:22.461900+0000","last_became_active":"2023-10-18T12:00:21.091845+0000","last_became_peered":"2023-10-18T12:00:21.091845+0000","last_unstale":"2023-10-18T12:00:22.461900+0000","last_undegraded":"2023-10-18T12:00:22.461900+0000","last_fullsized":"2023-10-18T12:00:22.461900+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:31.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"220'2","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.095672+0000","last_change":"2023-10-18T12:00:20.063535+0000","last_active":"2023-10-18T12:00:21.095672+0000","last_peered":"2023-10-18T12:00:21.095672+0000","last_clean":"2023-10-18T12:00:21.095672+0000","last_became_active":"2023-10-18T12:00:20.063259+0000","last_became_peered":"2023-10-18T12:00:20.063259+0000","last_unstale":"2023-10-18T12:00:21.095672+0000","last_undegraded":"2023-10-18T12:00:21.095672+0000","last_fullsized":"2023-10-18T12:00:21.095672+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:54:11.686164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167436+0000","last_change":"2023-10-18T12:00:21.092403+0000","last_active":"2023-10-18T12:00:22.167436+0000","last_peered":"2023-10-18T12:00:22.167436+0000","last_clean":"2023-10-18T12:00:22.167436+0000","last_became_active":"2023-10-18T12:00:21.092167+0000","last_became_peered":"2023-10-18T12:00:21.092167+0000","last_unstale":"2023-10-18T12:00:22.167436+0000","last_undegraded":"2023-10-18T12:00:22.167436+0000","last_fullsized":"2023-10-18T12:00:22.167436+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:45:11.154204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462284+0000","last_change":"2023-10-18T12:00:20.063289+0000","last_active":"2023-10-18T12:00:22.462284+0000","last_peered":"2023-10-18T12:00:22.462284+0000","last_clean":"2023-10-18T12:00:22.462284+0000","last_became_active":"2023-10-18T12:00:20.063024+0000","last_became_peered":"2023-10-18T12:00:20.063024+0000","last_unstale":"2023-10-18T12:00:22.462284+0000","last_undegraded":"2023-10-18T12:00:22.462284+0000","last_fullsized":"2023-10-18T12:00:22.462284+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:54.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.080078+0000","last_change":"2023-10-18T12:00:21.076847+0000","last_active":"2023-10-18T12:00:21.080078+0000","last_peered":"2023-10-18T12:00:21.080078+0000","last_clean":"2023-10-18T12:00:21.080078+0000","last_became_active":"2023-10-18T12:00:21.076552+0000","last_became_peered":"2023-10-18T12:00:21.076552+0000","last_unstale":"2023-10-18T12:00:21.080078+0000","last_undegraded":"2023-10-18T12:00:21.080078+0000","last_fullsized":"2023-10-18T12:00:21.080078+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:01.961752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.067860+0000","last_change":"2023-10-18T12:00:20.059978+0000","last_active":"2023-10-18T12:00:22.067860+0000","last_peered":"2023-10-18T12:00:22.067860+0000","last_clean":"2023-10-18T12:00:22.067860+0000","last_became_active":"2023-10-18T12:00:20.059686+0000","last_became_peered":"2023-10-18T12:00:20.059686+0000","last_unstale":"2023-10-18T12:00:22.067860+0000","last_undegraded":"2023-10-18T12:00:22.067860+0000","last_fullsized":"2023-10-18T12:00:22.067860+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:53.027447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066826+0000","last_change":"2023-10-18T12:00:21.086339+0000","last_active":"2023-10-18T12:00:22.066826+0000","last_peered":"2023-10-18T12:00:22.066826+0000","last_clean":"2023-10-18T12:00:22.066826+0000","last_became_active":"2023-10-18T12:00:21.086056+0000","last_became_peered":"2023-10-18T12:00:21.086056+0000","last_unstale":"2023-10-18T12:00:22.066826+0000","last_undegraded":"2023-10-18T12:00:22.066826+0000","last_fullsized":"2023-10-18T12:00:22.066826+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:39.387888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066981+0000","last_change":"2023-10-18T12:00:20.059094+0000","last_active":"2023-10-18T12:00:22.066981+0000","last_peered":"2023-10-18T12:00:22.066981+0000","last_clean":"2023-10-18T12:00:22.066981+0000","last_became_active":"2023-10-18T12:00:20.058785+0000","last_became_peered":"2023-10-18T12:00:20.058785+0000","last_unstale":"2023-10-18T12:00:22.066981+0000","last_undegraded":"2023-10-18T12:00:22.066981+0000","last_fullsized":"2023-10-18T12:00:22.066981+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:32.780678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462249+0000","last_change":"2023-10-18T12:00:21.090829+0000","last_active":"2023-10-18T12:00:22.462249+0000","last_peered":"2023-10-18T12:00:22.462249+0000","last_clean":"2023-10-18T12:00:22.462249+0000","last_became_active":"2023-10-18T12:00:21.090013+0000","last_became_peered":"2023-10-18T12:00:21.090013+0000","last_unstale":"2023-10-18T12:00:22.462249+0000","last_undegraded":"2023-10-18T12:00:22.462249+0000","last_fullsized":"2023-10-18T12:00:22.462249+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:14.286779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.462049+0000","last_change":"2023-10-18T12:00:20.066439+0000","last_active":"2023-10-18T12:00:22.462049+0000","last_peered":"2023-10-18T12:00:22.462049+0000","last_clean":"2023-10-18T12:00:22.462049+0000","last_became_active":"2023-10-18T12:00:20.066204+0000","last_became_peered":"2023-10-18T12:00:20.066204+0000","last_unstale":"2023-10-18T12:00:22.462049+0000","last_undegraded":"2023-10-18T12:00:22.462049+0000","last_fullsized":"2023-10-18T12:00:22.462049+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:36.690391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.089821+0000","last_change":"2023-10-18T12:00:21.088423+0000","last_active":"2023-10-18T12:00:21.089821+0000","last_peered":"2023-10-18T12:00:21.089821+0000","last_clean":"2023-10-18T12:00:21.089821+0000","last_became_active":"2023-10-18T12:00:21.087811+0000","last_became_peered":"2023-10-18T12:00:21.087811+0000","last_unstale":"2023-10-18T12:00:21.089821+0000","last_undegraded":"2023-10-18T12:00:21.089821+0000","last_fullsized":"2023-10-18T12:00:21.089821+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:27:51.077389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.071567+0000","last_change":"2023-10-18T12:00:20.060656+0000","last_active":"2023-10-18T12:00:21.071567+0000","last_peered":"2023-10-18T12:00:21.071567+0000","last_clean":"2023-10-18T12:00:21.071567+0000","last_became_active":"2023-10-18T12:00:20.060413+0000","last_became_peered":"2023-10-18T12:00:20.060413+0000","last_unstale":"2023-10-18T12:00:21.071567+0000","last_undegraded":"2023-10-18T12:00:21.071567+0000","last_fullsized":"2023-10-18T12:00:21.071567+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:32.716291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.091923+0000","last_change":"2023-10-18T12:00:21.089842+0000","last_active":"2023-10-18T12:00:21.091923+0000","last_peered":"2023-10-18T12:00:21.091923+0000","last_clean":"2023-10-18T12:00:21.091923+0000","last_became_active":"2023-10-18T12:00:21.089584+0000","last_became_peered":"2023-10-18T12:00:21.089584+0000","last_unstale":"2023-10-18T12:00:21.091923+0000","last_undegraded":"2023-10-18T12:00:21.091923+0000","last_fullsized":"2023-10-18T12:00:21.091923+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:13.985160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066817+0000","last_change":"2023-10-18T12:00:20.065817+0000","last_active":"2023-10-18T12:00:22.066817+0000","last_peered":"2023-10-18T12:00:22.066817+0000","last_clean":"2023-10-18T12:00:22.066817+0000","last_became_active":"2023-10-18T12:00:20.065536+0000","last_became_peered":"2023-10-18T12:00:20.065536+0000","last_unstale":"2023-10-18T12:00:22.066817+0000","last_undegraded":"2023-10-18T12:00:22.066817+0000","last_fullsized":"2023-10-18T12:00:22.066817+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:57.379803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.167642+0000","last_change":"2023-10-18T12:00:21.091679+0000","last_active":"2023-10-18T12:00:22.167642+0000","last_peered":"2023-10-18T12:00:22.167642+0000","last_clean":"2023-10-18T12:00:22.167642+0000","last_became_active":"2023-10-18T12:00:21.091412+0000","last_became_peered":"2023-10-18T12:00:21.091412+0000","last_unstale":"2023-10-18T12:00:22.167642+0000","last_undegraded":"2023-10-18T12:00:22.167642+0000","last_fullsized":"2023-10-18T12:00:22.167642+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:27.574111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.487843+0000","last_change":"2023-10-18T12:00:20.066813+0000","last_active":"2023-10-18T12:00:22.487843+0000","last_peered":"2023-10-18T12:00:22.487843+0000","last_clean":"2023-10-18T12:00:22.487843+0000","last_became_active":"2023-10-18T12:00:20.066552+0000","last_became_peered":"2023-10-18T12:00:20.066552+0000","last_unstale":"2023-10-18T12:00:22.487843+0000","last_undegraded":"2023-10-18T12:00:22.487843+0000","last_fullsized":"2023-10-18T12:00:22.487843+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:22:59.208578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":11,"reported_epoch":221,"state":"active+clean","last_fresh":"2023-10-18T12:00:22.066682+0000","last_change":"2023-10-18T12:00:21.091052+0000","last_active":"2023-10-18T12:00:22.066682+0000","last_peered":"2023-10-18T12:00:22.066682+0000","last_clean":"2023-10-18T12:00:22.066682+0000","last_became_active":"2023-10-18T12:00:21.090785+0000","last_became_peered":"2023-10-18T12:00:21.090785+0000","last_unstale":"2023-10-18T12:00:22.066682+0000","last_undegraded":"2023-10-18T12:00:22.066682+0000","last_fullsized":"2023-10-18T12:00:22.066682+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:09.229786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"220'1","reported_seq":12,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-18T12:00:21.090136+0000","last_change":"2023-10-18T12:00:20.057601+0000","last_active":"2023-10-18T12:00:21.090136+0000","last_peered":"2023-10-18T12:00:21.090136+0000","last_clean":"2023-10-18T12:00:21.090136+0000","last_became_active":"2023-10-18T12:00:20.057294+0000","last_became_peered":"2023-10-18T12:00:20.057294+0000","last_unstale":"2023-10-18T12:00:21.090136+0000","last_undegraded":"2023-10-18T12:00:21.090136+0000","last_fullsized":"2023-10-18T12:00:21.090136+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:43.803662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"208'357","reported_seq":1232,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-18T12:00:20.062741+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:00:20.062741+0000","last_peered":"2023-10-18T12:00:20.062741+0000","last_clean":"2023-10-18T12:00:20.062741+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:00:20.062741+0000","last_undegraded":"2023-10-18T12:00:20.062741+0000","last_fullsized":"2023-10-18T12:00:20.062741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":595,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":595,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575990,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52528,"kb_used_data":3296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319312,"statfs":{"total":96636764160,"available":96582975488,"internally_reserved":0,"allocated":3375104,"data_stored":2006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.501},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.47199999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.377},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.48399999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.52900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42216,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329624,"statfs":{"total":96636764160,"available":96593534976,"internally_reserved":0,"allocated":2781184,"data_stored":1416920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.39600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45200000000000001,"5min":0.43099999999999999,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.87},"last":0.88800000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71199999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.82599999999999996},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81100000000000005,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.80400000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.78400000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.872,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70399999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.623},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85299999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.91100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575990,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60272,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311568,"statfs":{"total":96636764160,"available":96575045632,"internally_reserved":0,"allocated":2785280,"data_stored":1421458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72199999999999998,"5min":0.92400000000000004,"15min":0.96099999999999997},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.745,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.755,"5min":0.80500000000000005,"15min":1.0349999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.45000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":1},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52100000000000002,"15min":0.53800000000000003},"min":{"1min":0.377,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.01},"last":0.437},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.54000000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":1.101,"15min":1.101},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81499999999999995,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.502}]}]},{"osd":4,"up_from":13,"seq":55834575990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47788,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324052,"statfs":{"total":96636764160,"available":96587829248,"internally_reserved":0,"allocated":2781184,"data_stored":1421984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.22500000000000001},"max":{"1min":0.73899999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.33600000000000002,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.80200000000000005,"15min":0.81599999999999995},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.33600000000000002},"max":{"1min":0.86899999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":1.123,"15min":1.123},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.625,"15min":0.621},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.627},"min":{"1min":0.434,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.81999999999999995,"15min":0.83999999999999997},"last":0.59599999999999997}]},{"osd":3,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.40300000000000002},"max":{"1min":0.84399999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66700000000000004}]},{"osd":5,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.51100000000000001,"15min":0.498},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.443}]},{"osd":6,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.81499999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.68400000000000005,"5min":1.018,"15min":1.018},"last":0.39300000000000002}]},{"osd":7,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.76100000000000001,"5min":1.004,"15min":1.05},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47792,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324048,"statfs":{"total":96636764160,"available":96587825152,"internally_reserved":0,"allocated":2777088,"data_stored":1414852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.23200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55000000000000004,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.41399999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23400000000000001},"max":{"1min":0.56699999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.30499999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41599999999999998,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.44500000000000001},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":1.899,"15min":1.899},"last":0.36599999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.29299999999999998,"15min":0.28399999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88400000000000001,"5min":1.034,"15min":1.052},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.40999999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.522,"15min":1.522},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.89600000000000002,"5min":0.91100000000000003,"15min":1.026},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":1.01,"15min":1.01},"last":0.47299999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67100000000000004,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81799999999999995,"5min":1.129,"15min":1.129},"last":0.442},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.376}]}]},{"osd":2,"up_from":13,"seq":55834575990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60252,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311588,"statfs":{"total":96636764160,"available":96575066112,"internally_reserved":0,"allocated":2768896,"data_stored":1414816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.216},"max":{"1min":0.56999999999999995,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.192},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.28599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.443},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.248}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65300000000000002,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.22600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.087,"15min":1.087},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.754},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.621}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66900000000000004,"15min":0.67400000000000004},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92000000000000004},"last":0.64000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60260,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311580,"statfs":{"total":96636764160,"available":96575057920,"internally_reserved":0,"allocated":2777088,"data_stored":1421402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.44600000000000001,"15min":0.435},"min":{"1min":0.25800000000000001,"5min":0.219,"15min":0.182},"max":{"1min":0.65200000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.379,"15min":0.38},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.23100000000000001},"max":{"1min":0.56699999999999995,"5min":0.74099999999999999,"15min":0.81000000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40699999999999997},"min":{"1min":0.34699999999999998,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.57699999999999996,"5min":0.68999999999999995,"15min":0.877},"last":0.30299999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.92000000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58799999999999997,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.46800000000000003,"5min":0.31900000000000001,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":0.82899999999999996,"15min":0.98599999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.62,"15min":0.58599999999999997},"min":{"1min":0.52500000000000002,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.77900000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.29599999999999999},"max":{"1min":0.83999999999999997,"5min":0.87,"15min":1.004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.51600000000000001,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.76100000000000001,"5min":0.91500000000000004,"15min":1.05},"last":0.627}]},{"osd":6,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.53400000000000003,"5min":0.379,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.39400000000000002},"max":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.54700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42780,"kb_used_data":3284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329060,"statfs":{"total":96636764160,"available":96592957440,"internally_reserved":0,"allocated":3362816,"data_stored":2005184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54700000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61899999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.432},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28899999999999998,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.66800000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51400000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.25700000000000001},"max":{"1min":0.62,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.56899999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.499}]},{"osd":4,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96999999999999997,"15min":0.997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.876,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88200000000000001,"5min":0.95799999999999996,"15min":0.96399999999999997},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66900000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":1.0089999999999999,"15min":1.03},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65800000000000003,"15min":0.68100000000000005},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.82699999999999996,"5min":0.997,"15min":0.997},"last":0.58399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:00:24.359 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T12:00:24.360 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T12:00:24.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.500+0000 7f758a555640 1 -- 172.21.15.125:0/1741335348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75840ab500 msgr2=0x7f75840ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.500+0000 7f758a555640 1 --2- 172.21.15.125:0/1741335348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75840ab500 0x7f75840ab9b0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f757c005fd0 tx=0x7f757c02c240 comp rx=0 tx=0).stop 2023-10-18T12:00:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.500+0000 7f758a555640 1 -- 172.21.15.125:0/1741335348 shutdown_connections 2023-10-18T12:00:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.500+0000 7f758a555640 1 --2- 172.21.15.125:0/1741335348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75840ab500 0x7f75840ab9b0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.500+0000 7f758a555640 1 --2- 172.21.15.125:0/1741335348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75840c9dc0 0x7f758406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.500+0000 7f758a555640 1 -- 172.21.15.125:0/1741335348 >> 172.21.15.125:0/1741335348 conn(0x7f7584069930 msgr2=0x7f7584069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.500+0000 7f758a555640 1 -- 172.21.15.125:0/1741335348 shutdown_connections 2023-10-18T12:00:24.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.500+0000 7f758a555640 1 -- 172.21.15.125:0/1741335348 wait complete. 2023-10-18T12:00:24.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.501+0000 7f758a555640 1 Processor -- start 2023-10-18T12:00:24.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f758a555640 1 -- start start 2023-10-18T12:00:24.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f758a555640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758406af00 0x7f758412df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f758a555640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75840ab500 0x7f75841f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f758a555640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75840c9dc0 0x7f75841eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f758a555640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75841f28e0 con 0x7f758406af00 2023-10-18T12:00:24.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f758a555640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75841ef330 con 0x7f75840ab500 2023-10-18T12:00:24.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f758a555640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758412ee90 con 0x7f75840c9dc0 2023-10-18T12:00:24.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f7583fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758406af00 0x7f758412df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f7583fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758406af00 0x7f758412df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43916/0 (socket says 172.21.15.125:43916) 2023-10-18T12:00:24.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.502+0000 7f7583fff640 1 -- 172.21.15.125:0/1878387646 learned_addr learned my addr 172.21.15.125:0/1878387646 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:24.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f75837fe640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75840ab500 0x7f75841f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7588af2640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75840c9dc0 0x7f75841eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7583fff640 1 -- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75840c9dc0 msgr2=0x7f75841eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7583fff640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75840c9dc0 0x7f75841eedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7583fff640 1 -- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75840ab500 msgr2=0x7f75841f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7583fff640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75840ab500 0x7f75841f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7583fff640 1 -- 172.21.15.125:0/1878387646 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c009900 con 0x7f758406af00 2023-10-18T12:00:24.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7588af2640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75840c9dc0 0x7f75841eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:24.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f75837fe640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75840ab500 0x7f75841f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:24.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7583fff640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758406af00 0x7f758412df60 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f757400c430 tx=0x7f757400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:24.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7580ff9640 1 -- 172.21.15.125:0/1878387646 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574011790 con 0x7f758406af00 2023-10-18T12:00:24.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7580ff9640 1 -- 172.21.15.125:0/1878387646 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75740119e0 con 0x7f758406af00 2023-10-18T12:00:24.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758412e5f0 con 0x7f758406af00 2023-10-18T12:00:24.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f7580ff9640 1 -- 172.21.15.125:0/1878387646 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574011e30 con 0x7f758406af00 2023-10-18T12:00:24.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.503+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758412ebb0 con 0x7f758406af00 2023-10-18T12:00:24.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.504+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75840690e0 con 0x7f758406af00 2023-10-18T12:00:24.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.505+0000 7f7580ff9640 1 -- 172.21.15.125:0/1878387646 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f757401c460 con 0x7f758406af00 2023-10-18T12:00:24.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.505+0000 7f7580ff9640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f757006af60 0x7f757006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.505+0000 7f7580ff9640 1 -- 172.21.15.125:0/1878387646 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f7574089bb0 con 0x7f758406af00 2023-10-18T12:00:24.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.505+0000 7f75837fe640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f757006af60 0x7f757006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.506+0000 7f75837fe640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f757006af60 0x7f757006d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7578005fd0 tx=0x7f757800f360 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:24.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.509+0000 7f7580ff9640 1 -- 172.21.15.125:0/1878387646 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7574054940 con 0x7f758406af00 2023-10-18T12:00:24.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.739+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f75841500d0 con 0x7f758406af00 2023-10-18T12:00:24.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.742+0000 7f7580ff9640 1 -- 172.21.15.125:0/1878387646 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v57) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7574057f60 con 0x7f758406af00 2023-10-18T12:00:24.744 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T12:00:24.745 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T12:00:24.745 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T12:00:24.745 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T12:00:24.745 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T12:00:24.746 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T12:00:24.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f757006af60 msgr2=0x7f757006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f757006af60 0x7f757006d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7578005fd0 tx=0x7f757800f360 comp rx=0 tx=0).stop 2023-10-18T12:00:24.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758406af00 msgr2=0x7f758412df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758406af00 0x7f758412df60 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f757400c430 tx=0x7f757400c8f0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 shutdown_connections 2023-10-18T12:00:24.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f757006af60 0x7f757006d420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75840ab500 0x7f75841f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75840c9dc0 0x7f75841eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 --2- 172.21.15.125:0/1878387646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f758406af00 0x7f758412df60 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 >> 172.21.15.125:0/1878387646 conn(0x7f7584069930 msgr2=0x7f7584069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:24.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 shutdown_connections 2023-10-18T12:00:24.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.744+0000 7f758a555640 1 -- 172.21.15.125:0/1878387646 wait complete. 2023-10-18T12:00:24.757 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:24.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.857+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2142440147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 msgr2=0x7f2fb0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.857+0000 7f2fc6ba5640 1 --2- 172.21.15.125:0/2142440147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 0x7f2fb0003dc0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4006fa0 tx=0x7f2fb00057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.857+0000 7f2fc5142640 1 -- 172.21.15.125:0/2142440147 reap_dead start 2023-10-18T12:00:24.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.857+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2142440147 shutdown_connections 2023-10-18T12:00:24.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.857+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2142440147 >> 172.21.15.125:0/2142440147 conn(0x7f2fc0069930 msgr2=0x7f2fc0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:24.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.857+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2142440147 shutdown_connections 2023-10-18T12:00:24.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.857+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2142440147 wait complete. 2023-10-18T12:00:24.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.858+0000 7f2fc6ba5640 1 Processor -- start 2023-10-18T12:00:24.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.858+0000 7f2fc6ba5640 1 -- start start 2023-10-18T12:00:24.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc6ba5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 0x7f2fc01f1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc6ba5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fc01586a0 0x7f2fc0158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc6ba5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc00acda0 0x7f2fc00ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc6ba5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0159060 con 0x7f2fb00039a0 2023-10-18T12:00:24.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc6ba5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00b1130 con 0x7f2fc00acda0 2023-10-18T12:00:24.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc6ba5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0141500 con 0x7f2fc01586a0 2023-10-18T12:00:24.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 0x7f2fc01f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 0x7f2fc01f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43934/0 (socket says 172.21.15.125:43934) 2023-10-18T12:00:24.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 -- 172.21.15.125:0/2675444236 learned_addr learned my addr 172.21.15.125:0/2675444236 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:24.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc5142640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc00acda0 0x7f2fc00ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 -- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fc01586a0 msgr2=0x7f2fc0158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fc01586a0 0x7f2fc0158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 -- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc00acda0 msgr2=0x7f2fc00ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:24.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc00acda0 0x7f2fc00ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:24.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 -- 172.21.15.125:0/2675444236 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00ac590 con 0x7f2fb00039a0 2023-10-18T12:00:24.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.859+0000 7f2fc4941640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 0x7f2fc01f1430 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c320 tx=0x7f2fb4001540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:24.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.860+0000 7f2fbd7fa640 1 -- 172.21.15.125:0/2675444236 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4006230 con 0x7f2fb00039a0 2023-10-18T12:00:24.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.860+0000 7f2fbd7fa640 1 -- 172.21.15.125:0/2675444236 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2fb4006390 con 0x7f2fb00039a0 2023-10-18T12:00:24.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.860+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc00b0ca0 con 0x7f2fb00039a0 2023-10-18T12:00:24.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.860+0000 7f2fbd7fa640 1 -- 172.21.15.125:0/2675444236 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4001730 con 0x7f2fb00039a0 2023-10-18T12:00:24.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.860+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc00ad760 con 0x7f2fb00039a0 2023-10-18T12:00:24.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.861+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc00ac840 con 0x7f2fb00039a0 2023-10-18T12:00:24.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.861+0000 7f2fbd7fa640 1 -- 172.21.15.125:0/2675444236 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2fb4031030 con 0x7f2fb00039a0 2023-10-18T12:00:24.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.861+0000 7f2fbd7fa640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2fa406af90 0x7f2fa406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:24.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.862+0000 7f2fbd7fa640 1 -- 172.21.15.125:0/2675444236 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f2fb40aa7c0 con 0x7f2fb00039a0 2023-10-18T12:00:24.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.862+0000 7f2fbffff640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2fa406af90 0x7f2fa406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:24.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.862+0000 7f2fbffff640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2fa406af90 0x7f2fa406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0157c50 tx=0x7f2fac006a00 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:24.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:24.866+0000 7f2fbd7fa640 1 -- 172.21.15.125:0/2675444236 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fb40754d0 con 0x7f2fb00039a0 2023-10-18T12:00:25.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.066+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fc0136fb0 con 0x7f2fb00039a0 2023-10-18T12:00:25.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.066+0000 7f2fbd7fa640 1 -- 172.21.15.125:0/2675444236 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4931 (secure 0 0 0) 0x7f2fb4044090 con 0x7f2fb00039a0 2023-10-18T12:00:25.070 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:25.070 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":86,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10168,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:20.388305+0000","modified":"2023-10-18T12:00:22.406153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11976":{"gid":11976,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2975172452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2975172452},{"type":"v1","addr":"172.21.15.125:6834","nonce":2975172452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-18T12:00:25.071 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 86 2023-10-18T12:00:25.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2fa406af90 msgr2=0x7f2fa406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2fa406af90 0x7f2fa406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0157c50 tx=0x7f2fac006a00 comp rx=0 tx=0).stop 2023-10-18T12:00:25.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 msgr2=0x7f2fc01f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 0x7f2fc01f1430 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c320 tx=0x7f2fb4001540 comp rx=0 tx=0).stop 2023-10-18T12:00:25.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 shutdown_connections 2023-10-18T12:00:25.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2fa406af90 0x7f2fa406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc00acda0 0x7f2fc00ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fc01586a0 0x7f2fc0158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 --2- 172.21.15.125:0/2675444236 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fb00039a0 0x7f2fc01f1430 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 >> 172.21.15.125:0/2675444236 conn(0x7f2fc0069930 msgr2=0x7f2fc01569e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 shutdown_connections 2023-10-18T12:00:25.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.068+0000 7f2fc6ba5640 1 -- 172.21.15.125:0/2675444236 wait complete. 2023-10-18T12:00:25.484 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T12:00:20.388305+0000', 'modified': '2023-10-18T12:00:22.406153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11976}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11976': {'gid': 11976, 'name': 'a', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6834/2975172452', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6832', 'nonce': 2975172452}, {'type': 'v1', 'addr': '172.21.15.125:6834', 'nonce': 2975172452}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T12:00:25.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T12:00:25.485 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T12:00:25.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.587+0000 7f24e84e5640 1 -- 172.21.15.125:0/3271091880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e0070920 msgr2=0x7f24e00affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.587+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3271091880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e0070920 0x7f24e00affe0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f24e00cbdc0 tx=0x7f24d802c1d0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.587+0000 7f24e84e5640 1 -- 172.21.15.125:0/3271091880 shutdown_connections 2023-10-18T12:00:25.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.587+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3271091880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e0070920 0x7f24e00affe0 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.587+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3271091880 >> v2:172.21.15.139:3300/0 conn(0x7f24e0141690 0x7f24e00703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.587+0000 7f24e84e5640 1 -- 172.21.15.125:0/3271091880 >> 172.21.15.125:0/3271091880 conn(0x7f24e0069930 msgr2=0x7f24e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.588+0000 7f24e84e5640 1 -- 172.21.15.125:0/3271091880 shutdown_connections 2023-10-18T12:00:25.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.588+0000 7f24e84e5640 1 -- 172.21.15.125:0/3271091880 wait complete. 2023-10-18T12:00:25.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e84e5640 1 Processor -- start 2023-10-18T12:00:25.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e84e5640 1 -- start start 2023-10-18T12:00:25.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e84e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24e0070920 0x7f24e00c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e84e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e013e520 0x7f24e00c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e84e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24e0141690 0x7f24e00ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e84e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0158ea0 con 0x7f24e013e520 2023-10-18T12:00:25.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e84e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01f4980 con 0x7f24e0141690 2023-10-18T12:00:25.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e84e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01f4c60 con 0x7f24e0070920 2023-10-18T12:00:25.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e5a80640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e013e520 0x7f24e00c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e5a80640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e013e520 0x7f24e00c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43954/0 (socket says 172.21.15.125:43954) 2023-10-18T12:00:25.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.589+0000 7f24e5a80640 1 -- 172.21.15.125:0/3495501464 learned_addr learned my addr 172.21.15.125:0/3495501464 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:25.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e6a82640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24e0141690 0x7f24e00ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e5a80640 1 -- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24e0070920 msgr2=0x7f24e00c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e5a80640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24e0070920 0x7f24e00c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e5a80640 1 -- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24e0141690 msgr2=0x7f24e00ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e5a80640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24e0141690 0x7f24e00ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e5a80640 1 -- 172.21.15.125:0/3495501464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8009900 con 0x7f24e013e520 2023-10-18T12:00:25.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e6a82640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24e0141690 0x7f24e00ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:25.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e5a80640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e013e520 0x7f24e00c5da0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f24d000c970 tx=0x7f24d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:25.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24d6ffd640 1 -- 172.21.15.125:0/3495501464 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0007bb0 con 0x7f24e013e520 2023-10-18T12:00:25.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24d6ffd640 1 -- 172.21.15.125:0/3495501464 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24d0007e00 con 0x7f24e013e520 2023-10-18T12:00:25.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e84e5640 1 -- 172.21.15.125:0/3495501464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e01f4f40 con 0x7f24e013e520 2023-10-18T12:00:25.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24d6ffd640 1 -- 172.21.15.125:0/3495501464 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d000f6b0 con 0x7f24e013e520 2023-10-18T12:00:25.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.590+0000 7f24e84e5640 1 -- 172.21.15.125:0/3495501464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e00c62e0 con 0x7f24e013e520 2023-10-18T12:00:25.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.591+0000 7f24d6ffd640 1 -- 172.21.15.125:0/3495501464 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f24d000f810 con 0x7f24e013e520 2023-10-18T12:00:25.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.592+0000 7f24d6ffd640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f24c406af60 0x7f24c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.592+0000 7f24d6ffd640 1 -- 172.21.15.125:0/3495501464 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f24d008a2b0 con 0x7f24e013e520 2023-10-18T12:00:25.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.592+0000 7f24e6281640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f24c406af60 0x7f24c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24e6281640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f24c406af60 0x7f24c406d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24dc003cc0 tx=0x7f24dc0079b0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:25.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24eb077740 1 Processor -- start 2023-10-18T12:00:25.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24eb077740 1 -- start start 2023-10-18T12:00:25.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24eb077740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af3d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24eb077740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3fff20 0x5623af4028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24eb077740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3d3020 0x5623af402e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24eb077740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623af2e4730 con 0x5623af3ffb40 2023-10-18T12:00:25.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24eb077740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623af345780 con 0x5623af3d3020 2023-10-18T12:00:25.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.593+0000 7f24eb077740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623af342e70 con 0x5623af3fff20 2023-10-18T12:00:25.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af3d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af3d2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43958/0 (socket says 172.21.15.125:43958) 2023-10-18T12:00:25.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 -- 172.21.15.125:0/240573720 learned_addr learned my addr 172.21.15.125:0/240573720 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:25.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e5a80640 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3fff20 0x5623af4028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6a82640 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3d3020 0x5623af402e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 -- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3fff20 msgr2=0x5623af4028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:25.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3fff20 0x5623af4028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 -- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3d3020 msgr2=0x5623af402e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:25.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3d3020 0x5623af402e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 -- 172.21.15.125:0/240573720 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623af3433d0 con 0x5623af3ffb40 2023-10-18T12:00:25.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e5a80640 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3fff20 0x5623af4028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:25.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6a82640 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3d3020 0x5623af402e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:25.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24e6281640 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af3d2ae0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0168f0 tx=0x7f24dc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d881cc8b8133f165 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:25.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24c3fff640 1 -- 172.21.15.125:0/240573720 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc01b070 con 0x5623af3ffb40 2023-10-18T12:00:25.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24c3fff640 1 -- 172.21.15.125:0/240573720 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24dc0162e0 con 0x5623af3ffb40 2023-10-18T12:00:25.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.594+0000 7f24c3fff640 1 -- 172.21.15.125:0/240573720 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc013af0 con 0x5623af3ffb40 2023-10-18T12:00:25.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 -- 172.21.15.125:0/240573720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 msgr2=0x5623af3d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af3d2ae0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0168f0 tx=0x7f24dc00f000 comp rx=0 tx=0).stop 2023-10-18T12:00:25.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 -- 172.21.15.125:0/240573720 shutdown_connections 2023-10-18T12:00:25.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3d3020 0x5623af402e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3fff20 0x5623af4028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 --2- 172.21.15.125:0/240573720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af3d2ae0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 -- 172.21.15.125:0/240573720 >> 172.21.15.125:0/240573720 conn(0x5623af3d4d50 msgr2=0x5623af41d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 -- 172.21.15.125:0/240573720 shutdown_connections 2023-10-18T12:00:25.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 -- 172.21.15.125:0/240573720 wait complete. 2023-10-18T12:00:25.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.595+0000 7f24eb077740 1 -- start start 2023-10-18T12:00:25.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.596+0000 7f24eb077740 1 Processor -- start 2023-10-18T12:00:25.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.596+0000 7f24eb077740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 0x5623af42e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.596+0000 7f24eb077740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af4147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.596+0000 7f24eb077740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3e9410 0x5623af3e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.596+0000 7f24eb077740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623af33e760 con 0x5623af3e9410 2023-10-18T12:00:25.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.596+0000 7f24eb077740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623af2f98f0 con 0x5623af3fff20 2023-10-18T12:00:25.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.596+0000 7f24eb077740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623af2f9590 con 0x5623af3ffb40 2023-10-18T12:00:25.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.596+0000 7f24eb077740 10 client.0 _ll_register_callbacks cb 0x5623af335ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:00:25.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3e9410 0x5623af3e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3e9410 0x5623af3e9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43972/0 (socket says 172.21.15.125:43972) 2023-10-18T12:00:25.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 -- 172.21.15.125:0/3604821414 learned_addr learned my addr 172.21.15.125:0/3604821414 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:25.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e5a80640 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af4147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6281640 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 0x5623af42e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 -- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 msgr2=0x5623af42e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:25.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 0x5623af42e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 -- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 msgr2=0x5623af4147f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:25.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af4147f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 -- 172.21.15.125:0/3604821414 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24dc0165a0 con 0x5623af3e9410 2023-10-18T12:00:25.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e5a80640 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af4147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:25.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24e6a82640 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3e9410 0x5623af3e9890 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f24d8005fd0 tx=0x7f24d8009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42059977f36286e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:25.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:00:25.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24c17fa640 1 -- 172.21.15.125:0/3604821414 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8034d50 con 0x5623af3e9410 2023-10-18T12:00:25.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24c17fa640 1 -- 172.21.15.125:0/3604821414 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f24d8004040 con 0x5623af3e9410 2023-10-18T12:00:25.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24eb077740 1 -- 172.21.15.125:0/3604821414 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623af337740 con 0x5623af3e9410 2023-10-18T12:00:25.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24c17fa640 1 -- 172.21.15.125:0/3604821414 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24d8004200 con 0x5623af3e9410 2023-10-18T12:00:25.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24eb077740 10 client.12123 fetch_fsmap learned FSMap version 86 2023-10-18T12:00:25.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24eb077740 1 -- 172.21.15.125:0/3604821414 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x5623af33cde0 con 0x5623af3e9410 2023-10-18T12:00:25.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.597+0000 7f24c17fa640 1 -- 172.21.15.125:0/3604821414 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5623af337740 con 0x5623af3e9410 2023-10-18T12:00:25.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24c17fa640 1 -- 172.21.15.125:0/3604821414 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24d8004360 con 0x5623af3e9410 2023-10-18T12:00:25.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24eb077740 10 client.12123 fetch_fsmap finished waiting for FSMap version 86 2023-10-18T12:00:25.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24eb077740 10 client.12123 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T12:00:25.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24eb077740 20 client.12123 populate_metadata read hostname 'smithi125' 2023-10-18T12:00:25.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24eb077740 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] conn(0x5623af401900 0x5623af401ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24eb077740 4 client.12123 mds_command: new command op to 11976 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:00:25.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24eb077740 1 -- 172.21.15.125:0/3604821414 --> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623af337460 con 0x5623af401900 2023-10-18T12:00:25.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24e6281640 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] conn(0x5623af401900 0x5623af401ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24c17fa640 10 client.12123 ms_handle_connect on v2:172.21.15.125:6832/2975172452 2023-10-18T12:00:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24e6281640 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] conn(0x5623af401900 0x5623af401ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a5b03d59147b5ad server_cookie=4c86f481ae224707 in_seq=0 out_seq=0 2023-10-18T12:00:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24c17fa640 1 -- 172.21.15.125:0/3604821414 <== mds.0 v2:172.21.15.125:6832/2975172452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5623af337460 con 0x5623af401900 2023-10-18T12:00:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.598+0000 7f24c17fa640 10 client.12123 handle_command_reply: tid=0 2023-10-18T12:00:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 client.12123 shutdown 2023-10-18T12:00:25.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 -- 172.21.15.125:0/3604821414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3e9410 msgr2=0x5623af3e9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3e9410 0x5623af3e9890 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f24d8005fd0 tx=0x7f24d8009880 comp rx=0 tx=0).stop 2023-10-18T12:00:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 -- 172.21.15.125:0/3604821414 shutdown_connections 2023-10-18T12:00:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] conn(0x5623af401900 0x5623af401ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af4147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24c17fa640 0 client.12123 ms_handle_reset on v2:172.21.15.125:6832/2975172452 2023-10-18T12:00:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3e9410 0x5623af3e9890 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 --2- 172.21.15.125:0/3604821414 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 0x5623af42e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 -- 172.21.15.125:0/3604821414 >> 172.21.15.125:0/3604821414 conn(0x5623af3d3020 msgr2=0x5623af420de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 -- 172.21.15.125:0/3604821414 shutdown_connections 2023-10-18T12:00:25.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 1 -- 172.21.15.125:0/3604821414 wait complete. 2023-10-18T12:00:25.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.599+0000 7f24eb077740 20 client.12123 trim_cache size 0 max 16384 2023-10-18T12:00:25.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.614+0000 7f24eb077740 1 Processor -- start 2023-10-18T12:00:25.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.614+0000 7f24eb077740 1 -- start start 2023-10-18T12:00:25.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24eb077740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3e9410 0x5623af3fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24eb077740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 0x5623af438060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24eb077740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3fff20 0x5623af4385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24eb077740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623af2bf270 con 0x5623af3fff20 2023-10-18T12:00:25.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24eb077740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623af403380 con 0x5623af3e9410 2023-10-18T12:00:25.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24eb077740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623af3faba0 con 0x5623af3ffb40 2023-10-18T12:00:25.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3fff20 0x5623af4385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3fff20 0x5623af4385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43978/0 (socket says 172.21.15.125:43978) 2023-10-18T12:00:25.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 -- 172.21.15.125:0/516620016 learned_addr learned my addr 172.21.15.125:0/516620016 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:25.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6281640 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3e9410 0x5623af3fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e5a80640 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 0x5623af438060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 -- 172.21.15.125:0/516620016 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 msgr2=0x5623af438060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:25.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 0x5623af438060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 -- 172.21.15.125:0/516620016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3e9410 msgr2=0x5623af3fa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:25.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3e9410 0x5623af3fa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 -- 172.21.15.125:0/516620016 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623af2dbd00 con 0x5623af3fff20 2023-10-18T12:00:25.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24e6a82640 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3fff20 0x5623af4385a0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f24d8002230 tx=0x7f24d8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d36ffaed6be3e45 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:25.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24d4ff9640 1 -- 172.21.15.125:0/516620016 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d80047d0 con 0x5623af3fff20 2023-10-18T12:00:25.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24d4ff9640 1 -- 172.21.15.125:0/516620016 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24d8034b50 con 0x5623af3fff20 2023-10-18T12:00:25.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.615+0000 7f24d4ff9640 1 -- 172.21.15.125:0/516620016 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8003e70 con 0x5623af3fff20 2023-10-18T12:00:25.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 -- 172.21.15.125:0/516620016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3fff20 msgr2=0x5623af4385a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3fff20 0x5623af4385a0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f24d8002230 tx=0x7f24d8009b60 comp rx=0 tx=0).stop 2023-10-18T12:00:25.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 -- 172.21.15.125:0/516620016 shutdown_connections 2023-10-18T12:00:25.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3fff20 0x5623af4385a0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3ffb40 0x5623af438060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 --2- 172.21.15.125:0/516620016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3e9410 0x5623af3fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 -- 172.21.15.125:0/516620016 >> 172.21.15.125:0/516620016 conn(0x5623af3d3020 msgr2=0x5623af3d78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 -- 172.21.15.125:0/516620016 shutdown_connections 2023-10-18T12:00:25.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 -- 172.21.15.125:0/516620016 wait complete. 2023-10-18T12:00:25.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 -- start start 2023-10-18T12:00:25.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.616+0000 7f24eb077740 1 Processor -- start 2023-10-18T12:00:25.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24eb077740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3e9410 0x5623af485d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24eb077740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af400cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24eb077740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af401230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24eb077740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623af487870 con 0x5623af3ffb40 2023-10-18T12:00:25.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24eb077740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623af3ec4e0 con 0x5623af3fff20 2023-10-18T12:00:25.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24eb077740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623af4026f0 con 0x5623af3e9410 2023-10-18T12:00:25.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24eb077740 10 client.0 _ll_register_callbacks cb 0x7f24dc0039f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:00:25.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24e5a80640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af400cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24e5a80640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af400cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43984/0 (socket says 172.21.15.125:43984) 2023-10-18T12:00:25.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24e5a80640 1 -- 172.21.15.125:0/2380381024 learned_addr learned my addr 172.21.15.125:0/2380381024 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:25.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24e6281640 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3e9410 0x5623af485d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.617+0000 7f24e6a82640 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af401230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24e5a80640 1 -- 172.21.15.125:0/2380381024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3e9410 msgr2=0x5623af485d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:25.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24e5a80640 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3e9410 0x5623af485d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24e5a80640 1 -- 172.21.15.125:0/2380381024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 msgr2=0x5623af401230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:25.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24e5a80640 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af401230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24e5a80640 1 -- 172.21.15.125:0/2380381024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623af33cde0 con 0x5623af3ffb40 2023-10-18T12:00:25.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24e5a80640 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af400cf0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f24d0058c10 tx=0x7f24d005cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f6026b3e5abf36 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:25.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:00:25.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24c17fa640 1 -- 172.21.15.125:0/2380381024 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0060af0 con 0x5623af3ffb40 2023-10-18T12:00:25.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24c17fa640 1 -- 172.21.15.125:0/2380381024 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f24d0060c50 con 0x5623af3ffb40 2023-10-18T12:00:25.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 1 -- 172.21.15.125:0/2380381024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24dc0162e0 con 0x5623af3ffb40 2023-10-18T12:00:25.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24c17fa640 1 -- 172.21.15.125:0/2380381024 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24d0016b60 con 0x5623af3ffb40 2023-10-18T12:00:25.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 10 client.12129 fetch_fsmap learned FSMap version 86 2023-10-18T12:00:25.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 1 -- 172.21.15.125:0/2380381024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x5623af3014f0 con 0x5623af3ffb40 2023-10-18T12:00:25.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24c17fa640 1 -- 172.21.15.125:0/2380381024 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24d0077020 con 0x5623af3ffb40 2023-10-18T12:00:25.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24c17fa640 1 -- 172.21.15.125:0/2380381024 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24d0016020 con 0x5623af3ffb40 2023-10-18T12:00:25.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 10 client.12129 fetch_fsmap finished waiting for FSMap version 86 2023-10-18T12:00:25.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 10 client.12129 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T12:00:25.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 20 client.12129 populate_metadata read hostname 'smithi125' 2023-10-18T12:00:25.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] conn(0x5623af3e8010 0x5623af3e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 4 client.12129 mds_command: new command op to 11976 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T12:00:25.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.618+0000 7f24eb077740 1 -- 172.21.15.125:0/2380381024 --> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5623af0406c0 con 0x5623af3e8010 2023-10-18T12:00:25.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24e6281640 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] conn(0x5623af3e8010 0x5623af3e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24c17fa640 10 client.12129 ms_handle_connect on v2:172.21.15.125:6832/2975172452 2023-10-18T12:00:25.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24e6281640 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] conn(0x5623af3e8010 0x5623af3e83f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7a2a7ef3130c77 server_cookie=1e8125226755c46a in_seq=0 out_seq=0 2023-10-18T12:00:25.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24c17fa640 1 -- 172.21.15.125:0/2380381024 <== mds.0 v2:172.21.15.125:6832/2975172452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5623af0406c0 con 0x5623af3e8010 2023-10-18T12:00:25.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24c17fa640 10 client.12129 handle_command_reply: tid=0 2023-10-18T12:00:25.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 client.12129 shutdown 2023-10-18T12:00:25.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 -- 172.21.15.125:0/2380381024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 msgr2=0x5623af400cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af400cf0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f24d0058c10 tx=0x7f24d005cdd0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 -- 172.21.15.125:0/2380381024 shutdown_connections 2023-10-18T12:00:25.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.125:6832/2975172452,v1:172.21.15.125:6834/2975172452] conn(0x5623af3e8010 0x5623af3e83f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5623af3fff20 0x5623af401230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5623af3ffb40 0x5623af400cf0 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24c17fa640 0 client.12129 ms_handle_reset on v2:172.21.15.125:6832/2975172452 2023-10-18T12:00:25.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 --2- 172.21.15.125:0/2380381024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5623af3e9410 0x5623af485d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 -- 172.21.15.125:0/2380381024 >> 172.21.15.125:0/2380381024 conn(0x5623af3d3020 msgr2=0x5623af3d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 -- 172.21.15.125:0/2380381024 shutdown_connections 2023-10-18T12:00:25.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 1 -- 172.21.15.125:0/2380381024 wait complete. 2023-10-18T12:00:25.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.619+0000 7f24eb077740 20 client.12129 trim_cache size 0 max 16384 2023-10-18T12:00:25.649 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:00:25.650 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T12:00:25.650 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T12:00:25.650 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 11976, 2023-10-18T12:00:25.650 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T12:00:25.651 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T12:00:25.651 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T12:00:25.651 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 4.5565013299999997, 2023-10-18T12:00:25.651 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 86, 2023-10-18T12:00:25.651 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 221, 2023-10-18T12:00:25.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 221, 2023-10-18T12:00:25.652 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 10.331198593 2023-10-18T12:00:25.652 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:00:25.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 -- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f24c406af60 msgr2=0x7f24c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f24c406af60 0x7f24c406d420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24dc003cc0 tx=0x7f24dc0079b0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 -- 172.21.15.125:0/3495501464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e013e520 msgr2=0x7f24e00c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e013e520 0x7f24e00c5da0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f24d000c970 tx=0x7f24d000ce30 comp rx=0 tx=0).stop 2023-10-18T12:00:25.654 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 -- 172.21.15.125:0/3495501464 shutdown_connections 2023-10-18T12:00:25.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f24c406af60 0x7f24c406d420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24e0141690 0x7f24e00ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e013e520 0x7f24e00c5da0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 --2- 172.21.15.125:0/3495501464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24e0070920 0x7f24e00c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 -- 172.21.15.125:0/3495501464 >> 172.21.15.125:0/3495501464 conn(0x7f24e0069930 msgr2=0x7f24e0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 -- 172.21.15.125:0/3495501464 shutdown_connections 2023-10-18T12:00:25.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.620+0000 7f24e84e5640 1 -- 172.21.15.125:0/3495501464 wait complete. 2023-10-18T12:00:25.656 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T12:00:25.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.757+0000 7efd11ee5640 1 -- 172.21.15.125:0/984087073 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c13e520 msgr2=0x7efd0c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.757+0000 7efd11ee5640 1 --2- 172.21.15.125:0/984087073 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c13e520 0x7efd0c06bb60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efcfc005fd0 tx=0x7efcfc02bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:25.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.757+0000 7efd11ee5640 1 -- 172.21.15.125:0/984087073 shutdown_connections 2023-10-18T12:00:25.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.757+0000 7efd11ee5640 1 --2- 172.21.15.125:0/984087073 >> v2:172.21.15.139:3300/0 conn(0x7efd0c070af0 0x7efd0c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.757+0000 7efd11ee5640 1 --2- 172.21.15.125:0/984087073 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c13e520 0x7efd0c06bb60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.757+0000 7efd11ee5640 1 -- 172.21.15.125:0/984087073 >> 172.21.15.125:0/984087073 conn(0x7efd0c069930 msgr2=0x7efd0c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.757+0000 7efd11ee5640 1 -- 172.21.15.125:0/984087073 shutdown_connections 2023-10-18T12:00:25.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.757+0000 7efd11ee5640 1 -- 172.21.15.125:0/984087073 wait complete. 2023-10-18T12:00:25.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.758+0000 7efd11ee5640 1 Processor -- start 2023-10-18T12:00:25.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.758+0000 7efd11ee5640 1 -- start start 2023-10-18T12:00:25.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd11ee5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c070af0 0x7efd0c0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd11ee5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd0c13e520 0x7efd0c0ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd11ee5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd0c141690 0x7efd0c13b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd11ee5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c13b550 con 0x7efd0c13e520 2023-10-18T12:00:25.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd11ee5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0ac2d0 con 0x7efd0c141690 2023-10-18T12:00:25.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd11ee5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c1da790 con 0x7efd0c070af0 2023-10-18T12:00:25.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd0c13e520 0x7efd0c0ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd0c13e520 0x7efd0c0ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:43996/0 (socket says 172.21.15.125:43996) 2023-10-18T12:00:25.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0affd640 1 -- 172.21.15.125:0/125916155 learned_addr learned my addr 172.21.15.125:0/125916155 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:25.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0bfff640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd0c141690 0x7efd0c13b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0b7fe640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c070af0 0x7efd0c0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0affd640 1 -- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c070af0 msgr2=0x7efd0c0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0affd640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c070af0 0x7efd0c0abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0affd640 1 -- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd0c141690 msgr2=0x7efd0c13b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0affd640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd0c141690 0x7efd0c13b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0affd640 1 -- 172.21.15.125:0/125916155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc009560 con 0x7efd0c13e520 2023-10-18T12:00:25.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0b7fe640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c070af0 0x7efd0c0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:25.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.759+0000 7efd0bfff640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd0c141690 0x7efd0c13b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:25.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.760+0000 7efd0affd640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd0c13e520 0x7efd0c0ad240 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7efd0000e6e0 tx=0x7efd0000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:25.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.760+0000 7efcf3fff640 1 -- 172.21.15.125:0/125916155 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd000052d0 con 0x7efd0c13e520 2023-10-18T12:00:25.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.760+0000 7efcf3fff640 1 -- 172.21.15.125:0/125916155 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd000016a0 con 0x7efd0c13e520 2023-10-18T12:00:25.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.760+0000 7efcf3fff640 1 -- 172.21.15.125:0/125916155 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00001ab0 con 0x7efd0c13e520 2023-10-18T12:00:25.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.760+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c1daa70 con 0x7efd0c13e520 2023-10-18T12:00:25.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.760+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c1db030 con 0x7efd0c13e520 2023-10-18T12:00:25.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.761+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c0690e0 con 0x7efd0c13e520 2023-10-18T12:00:25.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.761+0000 7efcf3fff640 1 -- 172.21.15.125:0/125916155 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efd00001c10 con 0x7efd0c13e520 2023-10-18T12:00:25.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.762+0000 7efcf3fff640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efcf806af60 0x7efcf806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:25.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.762+0000 7efcf3fff640 1 -- 172.21.15.125:0/125916155 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7efd0008be30 con 0x7efd0c13e520 2023-10-18T12:00:25.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.762+0000 7efd0b7fe640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efcf806af60 0x7efcf806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:25.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.762+0000 7efd0b7fe640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efcf806af60 0x7efcf806d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcfc035000 tx=0x7efcfc004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:25.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.766+0000 7efcf3fff640 1 -- 172.21.15.125:0/125916155 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd00056b40 con 0x7efd0c13e520 2023-10-18T12:00:25.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.967+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd0c0705f0 con 0x7efd0c13e520 2023-10-18T12:00:25.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.967+0000 7efcf3fff640 1 -- 172.21.15.125:0/125916155 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd0001f070 con 0x7efd0c13e520 2023-10-18T12:00:25.970 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T12:00:25.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efcf806af60 msgr2=0x7efcf806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efcf806af60 0x7efcf806d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcfc035000 tx=0x7efcfc004730 comp rx=0 tx=0).stop 2023-10-18T12:00:25.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd0c13e520 msgr2=0x7efd0c0ad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:25.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd0c13e520 0x7efd0c0ad240 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7efd0000e6e0 tx=0x7efd0000eba0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 shutdown_connections 2023-10-18T12:00:25.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7efcf806af60 0x7efcf806d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd0c141690 0x7efd0c13b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd0c13e520 0x7efd0c0ad240 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 --2- 172.21.15.125:0/125916155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efd0c070af0 0x7efd0c0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:25.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 >> 172.21.15.125:0/125916155 conn(0x7efd0c069930 msgr2=0x7efd0c0aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:25.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 shutdown_connections 2023-10-18T12:00:25.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:25.969+0000 7efd11ee5640 1 -- 172.21.15.125:0/125916155 wait complete. 2023-10-18T12:00:25.982 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T12:00:25.983 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T12:00:25.983 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T12:00:25.983 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T12:00:25.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T12:00:25.984 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T12:00:25.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T12:00:25.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T12:00:25.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:00:25.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:25.986 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T12:00:26.001 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T12:00:26.001 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T12:00:26.002 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T12:00:26.002 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:00:26.002 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T12:00:26.002 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:00:26.002 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:00:26.003 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:00:26.003 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T12:00:26.003 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:00:26.003 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:00:26.003 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T12:00:26.004 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:00:26.004 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:00:26.004 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T12:00:26.004 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:00:26.004 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T12:00:26.005 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 36998sec preferred_lft 36998sec 2023-10-18T12:00:26.005 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T12:00:26.005 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:00:26.005 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T12:00:26.005 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:00:26.006 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T12:00:26.006 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:00:26.006 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T12:00:26.006 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:00:26.006 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T12:00:26.007 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:00:26.007 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T12:00:26.007 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:00:26.007 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T12:00:26.007 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:00:26.008 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T12:00:26.008 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:00:26.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:26.009 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T12:00:26.039 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T12:00:26.037+0000 7f718de95640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:00:26.064 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T12:00:26.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:26.066 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T12:00:26.121 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T12:00:26.121 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T12:00:26.121 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T12:00:26.122 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T12:00:26.122 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T12:00:26.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:26.126 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T12:00:26.126 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T12:00:26.127 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T12:00:26.127 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T12:00:26.128 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T12:00:26.128 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T12:00:26.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:26.217 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T12:00:26.232 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T12:00:26.232 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T12:00:26.233 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:00:26.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:26.234 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T12:00:26.299 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T12:00:26.300 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:00:26.300 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T12:00:26.354 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:00:26.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:00:26.355 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T12:00:26.409 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T12:00:26.476 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:00:26.534 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T12:00:26.534 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T12:00:26.535 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T12:00:26.535 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T12:00:26.535 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T12:00:26.535 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T12:00:26.535 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T12:00:26.536 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T12:00:26.536 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T12:00:26.536 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T12:00:26.536 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T12:00:26.536 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T12:00:26.537 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:00:26.537 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T12:00:26.537 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T12:00:26.537 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T12:00:26.537 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T12:00:26.538 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T12:00:26.538 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T12:00:26.538 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T12:00:26.538 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T12:00:26.538 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T12:00:26.539 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:00:26.539 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T12:00:26.539 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T12:00:26.539 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T12:00:26.539 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T12:00:26.540 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T12:00:26.540 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T12:00:26.540 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T12:00:26.540 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:00:26.540 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:00:26.541 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:00:26.541 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:00:26.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:26.542 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T12:00:26.588 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T12:00:26.589 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T12:00:26.632 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T12:00:26.659 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:00:26.716 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T12:00:26.716 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T12:00:26.717 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T12:00:26.717 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T12:00:26.717 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T12:00:26.718 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T12:00:26.718 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T12:00:26.718 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T12:00:26.718 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T12:00:26.719 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T12:00:26.719 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T12:00:26.719 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T12:00:26.720 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:00:26.720 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T12:00:26.720 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T12:00:26.721 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T12:00:26.721 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T12:00:26.721 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T12:00:26.721 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T12:00:26.721 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T12:00:26.722 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T12:00:26.722 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T12:00:26.722 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:00:26.722 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T12:00:26.723 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T12:00:26.723 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T12:00:26.723 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T12:00:26.723 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T12:00:26.723 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T12:00:26.724 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T12:00:26.724 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:00:26.724 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:00:26.724 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:00:26.725 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:00:26.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T12:00:26.713+0000 7f5fc6533480 -1 init, newargv = 0x5627ee27e490 newargc=15 2023-10-18T12:00:26.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:26.727 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T12:00:26.727 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[155249]: starting ceph client 2023-10-18T12:00:26.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[155249]: starting fuse 2023-10-18T12:00:26.781 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T12:00:26.782 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T12:00:26.783 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T12:00:26.784 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T12:00:26.784 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T12:00:26.785 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T12:00:26.786 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T12:00:26.787 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:00:26.787 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T12:00:26.788 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T12:00:26.788 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T12:00:26.789 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T12:00:26.789 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:00:26.789 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T12:00:26.790 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T12:00:26.790 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T12:00:26.791 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T12:00:26.791 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:00:26.791 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T12:00:26.792 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T12:00:26.792 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T12:00:26.793 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T12:00:26.793 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T12:00:26.793 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T12:00:26.794 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T12:00:26.794 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T12:00:26.795 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:00:26.795 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T12:00:26.796 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T12:00:26.880 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:00:26.883 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:00:26.902 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.155249.asok 2023-10-18T12:00:26.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T12:00:26.913 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.155249.asok 2023-10-18T12:00:26.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:26.914 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.155249.asok status 2023-10-18T12:00:27.057 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:00:27.058 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T12:00:27.058 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T12:00:27.059 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T12:00:27.059 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T12:00:27.059 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T12:00:27.059 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T12:00:27.059 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "155249", 2023-10-18T12:00:27.060 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T12:00:27.060 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:27.060 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T12:00:27.060 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T12:00:27.061 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 12138, 2023-10-18T12:00:27.061 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T12:00:27.061 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T12:00:27.061 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T12:00:27.061 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 12138 2023-10-18T12:00:27.062 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:27.062 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T12:00:27.062 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T12:00:27.062 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T12:00:27.063 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 145513648 2023-10-18T12:00:27.063 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:27.063 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:27.063 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T12:00:27.064 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T12:00:27.064 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T12:00:27.064 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 145513648 2023-10-18T12:00:27.064 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:27.064 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.12138 192.168.0.1:0/145513648", 2023-10-18T12:00:27.065 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/145513648", 2023-10-18T12:00:27.065 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T12:00:27.065 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 86, 2023-10-18T12:00:27.065 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 221, 2023-10-18T12:00:27.066 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T12:00:27.066 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T12:00:27.066 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T12:00:27.066 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:00:27.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:27.068 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T12:00:27.068 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T12:00:27.083 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T12:00:27.084 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T12:00:27.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:27.085 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T12:00:27.151 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-18T12:00:27.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.294+0000 7f30cfa10640 1 -- 172.21.15.125:0/753836901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c81416f0 msgr2=0x7f30c80b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:27.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.294+0000 7f30cfa10640 1 --2- 172.21.15.125:0/753836901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c81416f0 0x7f30c80b1440 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009830 tx=0x7f30b802bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:27.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.294+0000 7f30cfa10640 1 -- 172.21.15.125:0/753836901 shutdown_connections 2023-10-18T12:00:27.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.294+0000 7f30cfa10640 1 --2- 172.21.15.125:0/753836901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c81416f0 0x7f30c80b1440 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:27.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.294+0000 7f30cfa10640 1 -- 172.21.15.125:0/753836901 >> 172.21.15.125:0/753836901 conn(0x7f30c80af730 msgr2=0x7f30c80afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:27.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.294+0000 7f30cfa10640 1 -- 172.21.15.125:0/753836901 shutdown_connections 2023-10-18T12:00:27.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.294+0000 7f30cfa10640 1 -- 172.21.15.125:0/753836901 wait complete. 2023-10-18T12:00:27.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.295+0000 7f30cfa10640 1 Processor -- start 2023-10-18T12:00:27.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30cfa10640 1 -- start start 2023-10-18T12:00:27.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30cfa10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c8070790 0x7f30c8137750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:27.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30cfa10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c80b1980 0x7f30c813c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:27.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30cfa10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c81416f0 0x7f30c813cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:27.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30cfa10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806c520 con 0x7f30c80b1980 2023-10-18T12:00:27.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30cfa10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806c800 con 0x7f30c81416f0 2023-10-18T12:00:27.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30cfa10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806cae0 con 0x7f30c8070790 2023-10-18T12:00:27.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30ccfab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c80b1980 0x7f30c813c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:27.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30ccfab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c80b1980 0x7f30c813c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:44012/0 (socket says 172.21.15.125:44012) 2023-10-18T12:00:27.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.296+0000 7f30ccfab640 1 -- 172.21.15.125:0/1602837958 learned_addr learned my addr 172.21.15.125:0/1602837958 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:27.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30cd7ac640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c8070790 0x7f30c8137750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:27.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30cdfad640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c81416f0 0x7f30c813cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:27.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30ccfab640 1 -- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c8070790 msgr2=0x7f30c8137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:27.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30ccfab640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c8070790 0x7f30c8137750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:27.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30ccfab640 1 -- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c81416f0 msgr2=0x7f30c813cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:27.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30ccfab640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c81416f0 0x7f30c813cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:27.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30ccfab640 1 -- 172.21.15.125:0/1602837958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8009490 con 0x7f30c80b1980 2023-10-18T12:00:27.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30cd7ac640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c8070790 0x7f30c8137750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:27.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30cdfad640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c81416f0 0x7f30c813cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:27.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30ccfab640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c80b1980 0x7f30c813c5f0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f30c400c410 tx=0x7f30c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:27.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30bdffb640 1 -- 172.21.15.125:0/1602837958 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c40035f0 con 0x7f30c80b1980 2023-10-18T12:00:27.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30bdffb640 1 -- 172.21.15.125:0/1602837958 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f30c4003750 con 0x7f30c80b1980 2023-10-18T12:00:27.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c806cdc0 con 0x7f30c80b1980 2023-10-18T12:00:27.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30bdffb640 1 -- 172.21.15.125:0/1602837958 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4003ba0 con 0x7f30c80b1980 2023-10-18T12:00:27.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.297+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8159270 con 0x7f30c80b1980 2023-10-18T12:00:27.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.298+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c81594f0 con 0x7f30c80b1980 2023-10-18T12:00:27.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.299+0000 7f30bdffb640 1 -- 172.21.15.125:0/1602837958 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f30c4003d00 con 0x7f30c80b1980 2023-10-18T12:00:27.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.299+0000 7f30bdffb640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f30ac06af60 0x7f30ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:27.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.299+0000 7f30bdffb640 1 -- 172.21.15.125:0/1602837958 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f30c4088fe0 con 0x7f30c80b1980 2023-10-18T12:00:27.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.299+0000 7f30cd7ac640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f30ac06af60 0x7f30ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:27.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.300+0000 7f30cd7ac640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f30ac06af60 0x7f30ac06d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036000 tx=0x7f30b8004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:27.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.303+0000 7f30bdffb640 1 -- 172.21.15.125:0/1602837958 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30c4053ef0 con 0x7f30c80b1980 2023-10-18T12:00:27.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:27.504+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f30c80cba60 con 0x7f30c80b1980 2023-10-18T12:00:28.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.382+0000 7f30bdffb640 1 -- 172.21.15.125:0/1602837958 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 136+0+0 (secure 0 0 0) 0x7f30c4057510 con 0x7f30c80b1980 2023-10-18T12:00:28.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.385+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f30ac06af60 msgr2=0x7f30ac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f30ac06af60 0x7f30ac06d420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036000 tx=0x7f30b8004730 comp rx=0 tx=0).stop 2023-10-18T12:00:28.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c80b1980 msgr2=0x7f30c813c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c80b1980 0x7f30c813c5f0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f30c400c410 tx=0x7f30c400c8d0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 shutdown_connections 2023-10-18T12:00:28.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f30ac06af60 0x7f30ac06d420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30c81416f0 0x7f30c813cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30c80b1980 0x7f30c813c5f0 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 --2- 172.21.15.125:0/1602837958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30c8070790 0x7f30c8137750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 >> 172.21.15.125:0/1602837958 conn(0x7f30c80af730 msgr2=0x7f30c80ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:28.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 shutdown_connections 2023-10-18T12:00:28.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.386+0000 7f30cfa10640 1 -- 172.21.15.125:0/1602837958 wait complete. 2023-10-18T12:00:28.401 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:28.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.506+0000 7faf526e1640 1 -- 172.21.15.125:0/2709098068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0cf340 msgr2=0x7faf4c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.506+0000 7faf526e1640 1 --2- 172.21.15.125:0/2709098068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0cf340 0x7faf4c0d1910 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009830 tx=0x7faf3c02bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:28.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.506+0000 7faf526e1640 1 -- 172.21.15.125:0/2709098068 shutdown_connections 2023-10-18T12:00:28.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.506+0000 7faf526e1640 1 --2- 172.21.15.125:0/2709098068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0cf340 0x7faf4c0d1910 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.506+0000 7faf526e1640 1 -- 172.21.15.125:0/2709098068 >> 172.21.15.125:0/2709098068 conn(0x7faf4c0d0d60 msgr2=0x7faf4c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:28.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.506+0000 7faf526e1640 1 -- 172.21.15.125:0/2709098068 shutdown_connections 2023-10-18T12:00:28.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.506+0000 7faf526e1640 1 -- 172.21.15.125:0/2709098068 wait complete. 2023-10-18T12:00:28.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.507+0000 7faf526e1640 1 Processor -- start 2023-10-18T12:00:28.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.507+0000 7faf526e1640 1 -- start start 2023-10-18T12:00:28.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf526e1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0ceb40 0x7faf4c0b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:28.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf526e1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf4c0cf340 0x7faf4c14eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:28.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf526e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4c15b2a0 0x7faf4c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:28.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf526e1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0b5310 con 0x7faf4c0ceb40 2023-10-18T12:00:28.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf526e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c15db70 con 0x7faf4c15b2a0 2023-10-18T12:00:28.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf526e1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0b6c50 con 0x7faf4c0cf340 2023-10-18T12:00:28.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf4bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0ceb40 0x7faf4c0b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:28.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf4bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0ceb40 0x7faf4c0b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60884/0 (socket says 172.21.15.125:60884) 2023-10-18T12:00:28.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf4bfff640 1 -- 172.21.15.125:0/38378520 learned_addr learned my addr 172.21.15.125:0/38378520 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:28.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf50c7e640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4c15b2a0 0x7faf4c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:28.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.508+0000 7faf4b7fe640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf4c0cf340 0x7faf4c14eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:28.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf4bfff640 1 -- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf4c0cf340 msgr2=0x7faf4c14eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf4bfff640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf4c0cf340 0x7faf4c14eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf4bfff640 1 -- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4c15b2a0 msgr2=0x7faf4c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf4bfff640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4c15b2a0 0x7faf4c15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf4bfff640 1 -- 172.21.15.125:0/38378520 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c009490 con 0x7faf4c0ceb40 2023-10-18T12:00:28.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf50c7e640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4c15b2a0 0x7faf4c15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:28.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf4bfff640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0ceb40 0x7faf4c0b6710 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009780 tx=0x7faf3c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:28.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf48ff9640 1 -- 172.21.15.125:0/38378520 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c003ea0 con 0x7faf4c0ceb40 2023-10-18T12:00:28.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf48ff9640 1 -- 172.21.15.125:0/38378520 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf3c034550 con 0x7faf4c0ceb40 2023-10-18T12:00:28.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf48ff9640 1 -- 172.21.15.125:0/38378520 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c034960 con 0x7faf4c0ceb40 2023-10-18T12:00:28.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c14f0a0 con 0x7faf4c0ceb40 2023-10-18T12:00:28.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.509+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c15d240 con 0x7faf4c0ceb40 2023-10-18T12:00:28.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.510+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c0690e0 con 0x7faf4c0ceb40 2023-10-18T12:00:28.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.510+0000 7faf48ff9640 1 -- 172.21.15.125:0/38378520 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faf3c03b040 con 0x7faf4c0ceb40 2023-10-18T12:00:28.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.511+0000 7faf48ff9640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf3806af60 0x7faf3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:28.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.511+0000 7faf48ff9640 1 -- 172.21.15.125:0/38378520 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7faf3c0ac070 con 0x7faf4c0ceb40 2023-10-18T12:00:28.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.511+0000 7faf4b7fe640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf3806af60 0x7faf3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:28.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.512+0000 7faf4b7fe640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf3806af60 0x7faf3806d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0b0cb0 tx=0x7faf40009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:28.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.515+0000 7faf48ff9640 1 -- 172.21.15.125:0/38378520 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf3c076d80 con 0x7faf4c0ceb40 2023-10-18T12:00:28.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.682+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf4c0d0b40 con 0x7faf4c0ceb40 2023-10-18T12:00:28.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.683+0000 7faf48ff9640 1 -- 172.21.15.125:0/38378520 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+16922 (secure 0 0 0) 0x7faf3c039090 con 0x7faf4c0ceb40 2023-10-18T12:00:28.686 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:28.687 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":221,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:22.063337+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:28.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.685+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf3806af60 msgr2=0x7faf3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.685+0000 7faf526e1640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf3806af60 0x7faf3806d420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0b0cb0 tx=0x7faf40009040 comp rx=0 tx=0).stop 2023-10-18T12:00:28.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.685+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0ceb40 msgr2=0x7faf4c0b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.685+0000 7faf526e1640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0ceb40 0x7faf4c0b6710 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009780 tx=0x7faf3c0038a0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.686+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 shutdown_connections 2023-10-18T12:00:28.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.686+0000 7faf526e1640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faf3806af60 0x7faf3806d420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.686+0000 7faf526e1640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4c15b2a0 0x7faf4c15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.686+0000 7faf526e1640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf4c0cf340 0x7faf4c14eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.686+0000 7faf526e1640 1 --2- 172.21.15.125:0/38378520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf4c0ceb40 0x7faf4c0b6710 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.686+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 >> 172.21.15.125:0/38378520 conn(0x7faf4c0d0d60 msgr2=0x7faf4c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:28.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.686+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 shutdown_connections 2023-10-18T12:00:28.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.686+0000 7faf526e1640 1 -- 172.21.15.125:0/38378520 wait complete. 2023-10-18T12:00:28.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:00:18.207564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:00:19.369758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '221', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:00:28.700 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:00:28.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.802+0000 7f42e70c9640 1 -- 172.21.15.125:0/417345941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42d0003970 msgr2=0x7f42d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.802+0000 7f42e70c9640 1 --2- 172.21.15.125:0/417345941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42d0003970 0x7f42d0003d90 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c260 tx=0x7f42d00057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.802+0000 7f42e5666640 1 -- 172.21.15.125:0/417345941 reap_dead start 2023-10-18T12:00:28.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.802+0000 7f42e70c9640 1 -- 172.21.15.125:0/417345941 shutdown_connections 2023-10-18T12:00:28.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.802+0000 7f42e70c9640 1 -- 172.21.15.125:0/417345941 >> 172.21.15.125:0/417345941 conn(0x7f42e0069930 msgr2=0x7f42e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:28.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.802+0000 7f42e70c9640 1 -- 172.21.15.125:0/417345941 shutdown_connections 2023-10-18T12:00:28.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.802+0000 7f42e70c9640 1 -- 172.21.15.125:0/417345941 wait complete. 2023-10-18T12:00:28.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.803+0000 7f42e70c9640 1 Processor -- start 2023-10-18T12:00:28.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.803+0000 7f42e70c9640 1 -- start start 2023-10-18T12:00:28.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42e70c9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42d0003970 0x7f42e013af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:28.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42e70c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42e013cd90 0x7f42e01f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:28.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42e70c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e00b07f0 0x7f42e00b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:28.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42e70c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00b11b0 con 0x7f42e013cd90 2023-10-18T12:00:28.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42e70c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01f4df0 con 0x7f42e00b07f0 2023-10-18T12:00:28.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42e70c9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e013b830 con 0x7f42d0003970 2023-10-18T12:00:28.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42dffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42e013cd90 0x7f42e01f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:28.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42dffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42e013cd90 0x7f42e01f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60904/0 (socket says 172.21.15.125:60904) 2023-10-18T12:00:28.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42dffff640 1 -- 172.21.15.125:0/3661209834 learned_addr learned my addr 172.21.15.125:0/3661209834 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:28.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42e4e65640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42d0003970 0x7f42e013af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:28.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.804+0000 7f42e5666640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e00b07f0 0x7f42e00b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:28.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dffff640 1 -- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42d0003970 msgr2=0x7f42e013af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dffff640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42d0003970 0x7f42e013af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dffff640 1 -- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e00b07f0 msgr2=0x7f42e00b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dffff640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e00b07f0 0x7f42e00b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dffff640 1 -- 172.21.15.125:0/3661209834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e013bb10 con 0x7f42e013cd90 2023-10-18T12:00:28.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dffff640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42e013cd90 0x7f42e01f5280 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f42cc011ae0 tx=0x7f42cc009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:28.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dd7fa640 1 -- 172.21.15.125:0/3661209834 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc004460 con 0x7f42e013cd90 2023-10-18T12:00:28.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dd7fa640 1 -- 172.21.15.125:0/3661209834 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f42cc00fa00 con 0x7f42e013cd90 2023-10-18T12:00:28.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e013bdf0 con 0x7f42e013cd90 2023-10-18T12:00:28.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42dd7fa640 1 -- 172.21.15.125:0/3661209834 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00fe10 con 0x7f42e013cd90 2023-10-18T12:00:28.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.805+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e013b490 con 0x7f42e013cd90 2023-10-18T12:00:28.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.806+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e00690e0 con 0x7f42e013cd90 2023-10-18T12:00:28.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.806+0000 7f42dd7fa640 1 -- 172.21.15.125:0/3661209834 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f42cc0035f0 con 0x7f42e013cd90 2023-10-18T12:00:28.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.807+0000 7f42dd7fa640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f42c406af60 0x7f42c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:28.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.807+0000 7f42dd7fa640 1 -- 172.21.15.125:0/3661209834 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f42cc08a7c0 con 0x7f42e013cd90 2023-10-18T12:00:28.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.807+0000 7f42e4e65640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f42c406af60 0x7f42c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:28.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.807+0000 7f42e4e65640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f42c406af60 0x7f42c406d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c260 tx=0x7f42d4001a70 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:28.817 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.811+0000 7f42dd7fa640 1 -- 172.21.15.125:0/3661209834 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42cc055500 con 0x7f42e013cd90 2023-10-18T12:00:28.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.977+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42e006ff80 con 0x7f42e013cd90 2023-10-18T12:00:28.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.977+0000 7f42dd7fa640 1 -- 172.21.15.125:0/3661209834 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v221) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42cc058b20 con 0x7f42e013cd90 2023-10-18T12:00:28.980 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:00:28.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f42c406af60 msgr2=0x7f42c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f42c406af60 0x7f42c406d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c260 tx=0x7f42d4001a70 comp rx=0 tx=0).stop 2023-10-18T12:00:28.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42e013cd90 msgr2=0x7f42e01f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:28.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42e013cd90 0x7f42e01f5280 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f42cc011ae0 tx=0x7f42cc009e40 comp rx=0 tx=0).stop 2023-10-18T12:00:28.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 shutdown_connections 2023-10-18T12:00:28.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f42c406af60 0x7f42c406d420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42e00b07f0 0x7f42e00b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42e013cd90 0x7f42e01f5280 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 --2- 172.21.15.125:0/3661209834 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42d0003970 0x7f42e013af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:28.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 >> 172.21.15.125:0/3661209834 conn(0x7f42e0069930 msgr2=0x7f42e0070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:28.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 shutdown_connections 2023-10-18T12:00:28.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:28.979+0000 7f42e70c9640 1 -- 172.21.15.125:0/3661209834 wait complete. 2023-10-18T12:00:28.992 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:00:29.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.097+0000 7f8040568640 1 -- 172.21.15.125:0/497224878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f803815b2a0 msgr2=0x7f80380d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.097+0000 7f8040568640 1 --2- 172.21.15.125:0/497224878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f803815b2a0 0x7f80380d1910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8028005fd0 tx=0x7f802802bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:29.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.097+0000 7f8040568640 1 -- 172.21.15.125:0/497224878 shutdown_connections 2023-10-18T12:00:29.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.097+0000 7f8040568640 1 --2- 172.21.15.125:0/497224878 >> v2:172.21.15.139:3300/0 conn(0x7f80380ceb40 0x7f80380b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.097+0000 7f8040568640 1 --2- 172.21.15.125:0/497224878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f803815b2a0 0x7f80380d1910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.097+0000 7f8040568640 1 -- 172.21.15.125:0/497224878 >> 172.21.15.125:0/497224878 conn(0x7f80380d0d60 msgr2=0x7f80380d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:29.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.097+0000 7f8040568640 1 -- 172.21.15.125:0/497224878 shutdown_connections 2023-10-18T12:00:29.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.097+0000 7f8040568640 1 -- 172.21.15.125:0/497224878 wait complete. 2023-10-18T12:00:29.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.098+0000 7f8040568640 1 Processor -- start 2023-10-18T12:00:29.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.098+0000 7f8040568640 1 -- start start 2023-10-18T12:00:29.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f8040568640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80380ceb40 0x7f80380b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f8040568640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80380cf340 0x7f80380c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f8040568640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803815b2a0 0x7f8038158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f8040568640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80380cadc0 con 0x7f803815b2a0 2023-10-18T12:00:29.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f8040568640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80381560c0 con 0x7f80380cf340 2023-10-18T12:00:29.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f8040568640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380ca800 con 0x7f80380ceb40 2023-10-18T12:00:29.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f803eb05640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803815b2a0 0x7f8038158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f803eb05640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803815b2a0 0x7f8038158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60920/0 (socket says 172.21.15.125:60920) 2023-10-18T12:00:29.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f803eb05640 1 -- 172.21.15.125:0/3348202749 learned_addr learned my addr 172.21.15.125:0/3348202749 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:29.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f803e304640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80380ceb40 0x7f80380b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.099+0000 7f803db03640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80380cf340 0x7f80380c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f803eb05640 1 -- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80380ceb40 msgr2=0x7f80380b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f803eb05640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80380ceb40 0x7f80380b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f803eb05640 1 -- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80380cf340 msgr2=0x7f80380c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f803eb05640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80380cf340 0x7f80380c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f803eb05640 1 -- 172.21.15.125:0/3348202749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009840 con 0x7f803815b2a0 2023-10-18T12:00:29.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f803eb05640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803815b2a0 0x7f8038158720 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f803000efd0 tx=0x7f803000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:29.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f802effd640 1 -- 172.21.15.125:0/3348202749 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030009b40 con 0x7f803815b2a0 2023-10-18T12:00:29.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f802effd640 1 -- 172.21.15.125:0/3348202749 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8030003c60 con 0x7f803815b2a0 2023-10-18T12:00:29.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8028009560 con 0x7f803815b2a0 2023-10-18T12:00:29.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f802effd640 1 -- 172.21.15.125:0/3348202749 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030004020 con 0x7f803815b2a0 2023-10-18T12:00:29.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.100+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80380b0d50 con 0x7f803815b2a0 2023-10-18T12:00:29.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.101+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80380690e0 con 0x7f803815b2a0 2023-10-18T12:00:29.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.101+0000 7f802effd640 1 -- 172.21.15.125:0/3348202749 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f80300041a0 con 0x7f803815b2a0 2023-10-18T12:00:29.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.102+0000 7f802effd640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f801c06af60 0x7f801c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.102+0000 7f802effd640 1 -- 172.21.15.125:0/3348202749 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f8030014070 con 0x7f803815b2a0 2023-10-18T12:00:29.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.102+0000 7f803e304640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f801c06af60 0x7f801c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.110 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.102+0000 7f803e304640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f801c06af60 0x7f801c06d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8034001780 tx=0x7f8034009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:29.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.106+0000 7f802effd640 1 -- 172.21.15.125:0/3348202749 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8030055890 con 0x7f803815b2a0 2023-10-18T12:00:29.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.271+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8038156200 con 0x7f803815b2a0 2023-10-18T12:00:29.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.272+0000 7f802effd640 1 -- 172.21.15.125:0/3348202749 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v221) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8030058eb0 con 0x7f803815b2a0 2023-10-18T12:00:29.275 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:00:29.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f801c06af60 msgr2=0x7f801c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f801c06af60 0x7f801c06d420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8034001780 tx=0x7f8034009040 comp rx=0 tx=0).stop 2023-10-18T12:00:29.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803815b2a0 msgr2=0x7f8038158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803815b2a0 0x7f8038158720 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f803000efd0 tx=0x7f803000c6a0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 shutdown_connections 2023-10-18T12:00:29.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f801c06af60 0x7f801c06d420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80380cf340 0x7f80380c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803815b2a0 0x7f8038158720 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 --2- 172.21.15.125:0/3348202749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80380ceb40 0x7f80380b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 >> 172.21.15.125:0/3348202749 conn(0x7f80380d0d60 msgr2=0x7f803806b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:29.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 shutdown_connections 2023-10-18T12:00:29.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.274+0000 7f8040568640 1 -- 172.21.15.125:0/3348202749 wait complete. 2023-10-18T12:00:29.287 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:00:29.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.389+0000 7f08ec955640 1 -- 172.21.15.125:0/2879639736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e8141690 msgr2=0x7f08e80703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.389+0000 7f08ec955640 1 --2- 172.21.15.125:0/2879639736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e8141690 0x7f08e80703e0 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f08e80b20f0 tx=0x7f08d002bd30 comp rx=0 tx=0).stop 2023-10-18T12:00:29.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.389+0000 7f08ec955640 1 -- 172.21.15.125:0/2879639736 shutdown_connections 2023-10-18T12:00:29.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.389+0000 7f08ec955640 1 --2- 172.21.15.125:0/2879639736 >> v2:172.21.15.139:3300/0 conn(0x7f08e8070920 0x7f08e80affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.389+0000 7f08ec955640 1 --2- 172.21.15.125:0/2879639736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e8141690 0x7f08e80703e0 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.389+0000 7f08ec955640 1 -- 172.21.15.125:0/2879639736 >> 172.21.15.125:0/2879639736 conn(0x7f08e8069930 msgr2=0x7f08e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:29.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.389+0000 7f08ec955640 1 -- 172.21.15.125:0/2879639736 shutdown_connections 2023-10-18T12:00:29.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.389+0000 7f08ec955640 1 -- 172.21.15.125:0/2879639736 wait complete. 2023-10-18T12:00:29.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.391+0000 7f08ec955640 1 Processor -- start 2023-10-18T12:00:29.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.391+0000 7f08ec955640 1 -- start start 2023-10-18T12:00:29.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.391+0000 7f08ec955640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08e8070920 0x7f08e81e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.391+0000 7f08ec955640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e813e520 0x7f08e813ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08ec955640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08e8141690 0x7f08e813b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08ec955640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80ab400 con 0x7f08e813e520 2023-10-18T12:00:29.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08ec955640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80ab6e0 con 0x7f08e8070920 2023-10-18T12:00:29.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08ec955640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80ab9c0 con 0x7f08e8141690 2023-10-18T12:00:29.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e813e520 0x7f08e813ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e813e520 0x7f08e813ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60932/0 (socket says 172.21.15.125:60932) 2023-10-18T12:00:29.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 -- 172.21.15.125:0/4278364006 learned_addr learned my addr 172.21.15.125:0/4278364006 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:29.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08e659c640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08e8070920 0x7f08e81e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08e6d9d640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08e8141690 0x7f08e813b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 -- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08e8141690 msgr2=0x7f08e813b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08e8141690 0x7f08e813b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 -- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08e8070920 msgr2=0x7f08e81e04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08e8070920 0x7f08e81e04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 -- 172.21.15.125:0/4278364006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d0009560 con 0x7f08e813e520 2023-10-18T12:00:29.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08dffff640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e813e520 0x7f08e813ae30 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009850 tx=0x7f08d002dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:29.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08deffd640 1 -- 172.21.15.125:0/4278364006 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0005650 con 0x7f08e813e520 2023-10-18T12:00:29.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08deffd640 1 -- 172.21.15.125:0/4278364006 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f08d00038f0 con 0x7f08e813e520 2023-10-18T12:00:29.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e80abca0 con 0x7f08e813e520 2023-10-18T12:00:29.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08deffd640 1 -- 172.21.15.125:0/4278364006 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0003ce0 con 0x7f08e813e520 2023-10-18T12:00:29.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.392+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e81e0a20 con 0x7f08e813e520 2023-10-18T12:00:29.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.393+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e80b1bd0 con 0x7f08e813e520 2023-10-18T12:00:29.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.394+0000 7f08deffd640 1 -- 172.21.15.125:0/4278364006 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f08d003e450 con 0x7f08e813e520 2023-10-18T12:00:29.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.394+0000 7f08deffd640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f08c806af60 0x7f08c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.394+0000 7f08deffd640 1 -- 172.21.15.125:0/4278364006 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f08d00ac160 con 0x7f08e813e520 2023-10-18T12:00:29.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.394+0000 7f08e659c640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f08c806af60 0x7f08c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.395+0000 7f08e659c640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f08c806af60 0x7f08c806d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08d80017f0 tx=0x7f08d8009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:29.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.399+0000 7f08deffd640 1 -- 172.21.15.125:0/4278364006 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08d0077040 con 0x7f08e813e520 2023-10-18T12:00:29.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.564+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08e813e0d0 con 0x7f08e813e520 2023-10-18T12:00:29.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.564+0000 7f08deffd640 1 -- 172.21.15.125:0/4278364006 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v221) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08d007a660 con 0x7f08e813e520 2023-10-18T12:00:29.567 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:00:29.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f08c806af60 msgr2=0x7f08c806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f08c806af60 0x7f08c806d420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08d80017f0 tx=0x7f08d8009040 comp rx=0 tx=0).stop 2023-10-18T12:00:29.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e813e520 msgr2=0x7f08e813ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e813e520 0x7f08e813ae30 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009850 tx=0x7f08d002dd00 comp rx=0 tx=0).stop 2023-10-18T12:00:29.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 shutdown_connections 2023-10-18T12:00:29.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f08c806af60 0x7f08c806d420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08e8141690 0x7f08e813b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08e813e520 0x7f08e813ae30 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 --2- 172.21.15.125:0/4278364006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08e8070920 0x7f08e81e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 >> 172.21.15.125:0/4278364006 conn(0x7f08e8069930 msgr2=0x7f08e81dfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:29.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 shutdown_connections 2023-10-18T12:00:29.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.566+0000 7f08ec955640 1 -- 172.21.15.125:0/4278364006 wait complete. 2023-10-18T12:00:29.579 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T12:00:29.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.677+0000 7f585a50f640 1 -- 172.21.15.125:0/1347844378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540afa20 msgr2=0x7f58540705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.677+0000 7f585a50f640 1 --2- 172.21.15.125:0/1347844378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540afa20 0x7f58540705a0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f5848009830 tx=0x7f584802bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:29.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.678+0000 7f585a50f640 1 -- 172.21.15.125:0/1347844378 shutdown_connections 2023-10-18T12:00:29.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.678+0000 7f585a50f640 1 --2- 172.21.15.125:0/1347844378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540afa20 0x7f58540705a0 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.678+0000 7f585a50f640 1 --2- 172.21.15.125:0/1347844378 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58540cbd20 0x7f5854158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.678+0000 7f585a50f640 1 -- 172.21.15.125:0/1347844378 >> 172.21.15.125:0/1347844378 conn(0x7f5854069930 msgr2=0x7f5854069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:29.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.678+0000 7f585a50f640 1 -- 172.21.15.125:0/1347844378 shutdown_connections 2023-10-18T12:00:29.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.678+0000 7f585a50f640 1 -- 172.21.15.125:0/1347844378 wait complete. 2023-10-18T12:00:29.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.679+0000 7f585a50f640 1 Processor -- start 2023-10-18T12:00:29.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.679+0000 7f585a50f640 1 -- start start 2023-10-18T12:00:29.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.679+0000 7f585a50f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58540afa20 0x7f5854140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f585a50f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58540b0530 0x7f585415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f585a50f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540cbd20 0x7f58540ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f585a50f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58541407d0 con 0x7f58540cbd20 2023-10-18T12:00:29.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f585a50f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5854154580 con 0x7f58540afa20 2023-10-18T12:00:29.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f585a50f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5854154860 con 0x7f58540b0530 2023-10-18T12:00:29.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540cbd20 0x7f58540ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540cbd20 0x7f58540ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60962/0 (socket says 172.21.15.125:60962) 2023-10-18T12:00:29.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 -- 172.21.15.125:0/420467882 learned_addr learned my addr 172.21.15.125:0/420467882 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:29.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f58537fe640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58540b0530 0x7f585415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5853fff640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58540afa20 0x7f5854140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 -- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58540b0530 msgr2=0x7f585415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58540b0530 0x7f585415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 -- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58540afa20 msgr2=0x7f5854140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58540afa20 0x7f5854140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 -- 172.21.15.125:0/420467882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5848009490 con 0x7f58540cbd20 2023-10-18T12:00:29.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5858aac640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540cbd20 0x7f58540ca870 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f584c00cae0 tx=0x7f584c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:29.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5850ff9640 1 -- 172.21.15.125:0/420467882 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c00ac40 con 0x7f58540cbd20 2023-10-18T12:00:29.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5850ff9640 1 -- 172.21.15.125:0/420467882 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f584c00ae90 con 0x7f58540cbd20 2023-10-18T12:00:29.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5854070cf0 con 0x7f58540cbd20 2023-10-18T12:00:29.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f5850ff9640 1 -- 172.21.15.125:0/420467882 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c003bd0 con 0x7f58540cbd20 2023-10-18T12:00:29.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.680+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5854154e20 con 0x7f58540cbd20 2023-10-18T12:00:29.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.681+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58540690e0 con 0x7f58540cbd20 2023-10-18T12:00:29.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.682+0000 7f5850ff9640 1 -- 172.21.15.125:0/420467882 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f584c003d30 con 0x7f58540cbd20 2023-10-18T12:00:29.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.682+0000 7f5850ff9640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f584006af60 0x7f584006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.682+0000 7f5850ff9640 1 -- 172.21.15.125:0/420467882 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f584c08a350 con 0x7f58540cbd20 2023-10-18T12:00:29.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.683+0000 7f5853fff640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f584006af60 0x7f584006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.683+0000 7f5853fff640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f584006af60 0x7f584006d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f584400bb50 tx=0x7f584400b2e0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:29.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.687+0000 7f5850ff9640 1 -- 172.21.15.125:0/420467882 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f584c055100 con 0x7f58540cbd20 2023-10-18T12:00:29.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.858+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f58540cc1e0 con 0x7f58540cbd20 2023-10-18T12:00:29.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.859+0000 7f5850ff9640 1 -- 172.21.15.125:0/420467882 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v221) v1 ==== 84+0+209 (secure 0 0 0) 0x7f584c058720 con 0x7f58540cbd20 2023-10-18T12:00:29.862 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:29.862 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T12:00:29.862 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:29.863 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T12:00:29.863 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T12:00:29.863 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:29.863 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:29.863 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 27, 2023-10-18T12:00:29.864 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": "cephfs_metadata" 2023-10-18T12:00:29.864 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:29.864 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:29.864 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 28, 2023-10-18T12:00:29.864 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": "cephfs_data" 2023-10-18T12:00:29.865 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:29.865 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T12:00:29.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.860+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f584006af60 msgr2=0x7f584006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.860+0000 7f585a50f640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f584006af60 0x7f584006d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f584400bb50 tx=0x7f584400b2e0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.860+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540cbd20 msgr2=0x7f58540ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.860+0000 7f585a50f640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540cbd20 0x7f58540ca870 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f584c00cae0 tx=0x7f584c00a060 comp rx=0 tx=0).stop 2023-10-18T12:00:29.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.861+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 shutdown_connections 2023-10-18T12:00:29.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.861+0000 7f585a50f640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f584006af60 0x7f584006d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.861+0000 7f585a50f640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58540cbd20 0x7f58540ca870 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.861+0000 7f585a50f640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58540b0530 0x7f585415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.861+0000 7f585a50f640 1 --2- 172.21.15.125:0/420467882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58540afa20 0x7f5854140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.861+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 >> 172.21.15.125:0/420467882 conn(0x7f5854069930 msgr2=0x7f58540ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:29.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.861+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 shutdown_connections 2023-10-18T12:00:29.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.861+0000 7f585a50f640 1 -- 172.21.15.125:0/420467882 wait complete. 2023-10-18T12:00:29.874 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-18T12:00:29.875 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-18T12:00:29.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.979+0000 7f2fa0505640 1 -- 172.21.15.125:0/151472288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 msgr2=0x7f2f980c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.979+0000 7f2fa0505640 1 --2- 172.21.15.125:0/151472288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 0x7f2f980c9b60 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f2f9806d570 tx=0x7f2f8802be40 comp rx=0 tx=0).stop 2023-10-18T12:00:29.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.979+0000 7f2fa0505640 1 -- 172.21.15.125:0/151472288 shutdown_connections 2023-10-18T12:00:29.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.979+0000 7f2fa0505640 1 --2- 172.21.15.125:0/151472288 >> v2:172.21.15.139:3300/0 conn(0x7f2f98154af0 0x7f2f98154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.979+0000 7f2fa0505640 1 --2- 172.21.15.125:0/151472288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 0x7f2f980c9b60 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.979+0000 7f2fa0505640 1 -- 172.21.15.125:0/151472288 >> 172.21.15.125:0/151472288 conn(0x7f2f98069930 msgr2=0x7f2f98069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:29.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.979+0000 7f2fa0505640 1 -- 172.21.15.125:0/151472288 shutdown_connections 2023-10-18T12:00:29.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.979+0000 7f2fa0505640 1 -- 172.21.15.125:0/151472288 wait complete. 2023-10-18T12:00:29.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.980+0000 7f2fa0505640 1 Processor -- start 2023-10-18T12:00:29.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2fa0505640 1 -- start start 2023-10-18T12:00:29.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2fa0505640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f980b1920 0x7f2f980acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2fa0505640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 0x7f2f9815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2fa0505640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f98154af0 0x7f2f98142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:29.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2fa0505640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f981f1880 con 0x7f2f980c96b0 2023-10-18T12:00:29.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2fa0505640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980ad0e0 con 0x7f2f980b1920 2023-10-18T12:00:29.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2fa0505640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98143020 con 0x7f2f98154af0 2023-10-18T12:00:29.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2f9daa0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 0x7f2f9815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2f9daa0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 0x7f2f9815d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:60984/0 (socket says 172.21.15.125:60984) 2023-10-18T12:00:29.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2f9daa0640 1 -- 172.21.15.125:0/905820736 learned_addr learned my addr 172.21.15.125:0/905820736 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:29.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2f9eaa2640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f98154af0 0x7f2f98142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.981+0000 7f2f9e2a1640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f980b1920 0x7f2f980acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:29.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f9daa0640 1 -- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f98154af0 msgr2=0x7f2f98142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f9daa0640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f98154af0 0x7f2f98142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f9daa0640 1 -- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f980b1920 msgr2=0x7f2f980acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:29.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f9daa0640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f980b1920 0x7f2f980acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:29.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f9daa0640 1 -- 172.21.15.125:0/905820736 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88009560 con 0x7f2f980c96b0 2023-10-18T12:00:29.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f9e2a1640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f980b1920 0x7f2f980acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:29.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f9eaa2640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f98154af0 0x7f2f98142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:29.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f9daa0640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 0x7f2f9815d6b0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f2f880099c0 tx=0x7f2f8802dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:29.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f8effd640 1 -- 172.21.15.125:0/905820736 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88003800 con 0x7f2f980c96b0 2023-10-18T12:00:29.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f8effd640 1 -- 172.21.15.125:0/905820736 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2f88003a50 con 0x7f2f980c96b0 2023-10-18T12:00:29.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f98070300 con 0x7f2f980c96b0 2023-10-18T12:00:29.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2f8effd640 1 -- 172.21.15.125:0/905820736 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88003e60 con 0x7f2f980c96b0 2023-10-18T12:00:30.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.982+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f981435e0 con 0x7f2f980c96b0 2023-10-18T12:00:30.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.983+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f981f1a30 con 0x7f2f980c96b0 2023-10-18T12:00:30.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.983+0000 7f2f8effd640 1 -- 172.21.15.125:0/905820736 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2f8803e450 con 0x7f2f980c96b0 2023-10-18T12:00:30.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.984+0000 7f2f8effd640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2f7c06af90 0x7f2f7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:30.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.984+0000 7f2f8effd640 1 -- 172.21.15.125:0/905820736 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f2f880ac3f0 con 0x7f2f980c96b0 2023-10-18T12:00:30.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.984+0000 7f2f9e2a1640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2f7c06af90 0x7f2f7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:30.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.984+0000 7f2f9e2a1640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2f7c06af90 0x7f2f7c06d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001e00 tx=0x7f2f90009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:30.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:29.988+0000 7f2f8effd640 1 -- 172.21.15.125:0/905820736 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f88077210 con 0x7f2f980c96b0 2023-10-18T12:00:30.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.154+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2f9815c370 con 0x7f2f980c96b0 2023-10-18T12:00:30.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.405+0000 7f2f8effd640 1 -- 172.21.15.125:0/905820736 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v222) v1 ==== 144+0+0 (secure 0 0 0) 0x7f2f8807a830 con 0x7f2f980c96b0 2023-10-18T12:00:30.408 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'backup_fs_metadata' created 2023-10-18T12:00:30.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2f7c06af90 msgr2=0x7f2f7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:30.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2f7c06af90 0x7f2f7c06d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001e00 tx=0x7f2f90009040 comp rx=0 tx=0).stop 2023-10-18T12:00:30.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 msgr2=0x7f2f9815d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:30.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 0x7f2f9815d6b0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f2f880099c0 tx=0x7f2f8802dbe0 comp rx=0 tx=0).stop 2023-10-18T12:00:30.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 shutdown_connections 2023-10-18T12:00:30.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f2f7c06af90 0x7f2f7c06d450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:30.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f98154af0 0x7f2f98142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:30.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f980c96b0 0x7f2f9815d6b0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:30.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 --2- 172.21.15.125:0/905820736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f980b1920 0x7f2f980acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:30.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 >> 172.21.15.125:0/905820736 conn(0x7f2f98069930 msgr2=0x7f2f981539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:30.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 shutdown_connections 2023-10-18T12:00:30.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.408+0000 7f2fa0505640 1 -- 172.21.15.125:0/905820736 wait complete. 2023-10-18T12:00:30.426 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T12:00:30.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.534+0000 7f50d654c640 1 -- 172.21.15.125:0/3969803007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 msgr2=0x7f50d00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:30.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.534+0000 7f50d654c640 1 --2- 172.21.15.125:0/3969803007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 0x7f50d00d1910 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009830 tx=0x7f50c402bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:30.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.534+0000 7f50d654c640 1 -- 172.21.15.125:0/3969803007 shutdown_connections 2023-10-18T12:00:30.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.534+0000 7f50d654c640 1 --2- 172.21.15.125:0/3969803007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 0x7f50d00d1910 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:30.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.534+0000 7f50d654c640 1 -- 172.21.15.125:0/3969803007 >> 172.21.15.125:0/3969803007 conn(0x7f50d00d0d60 msgr2=0x7f50d00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:30.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.534+0000 7f50d654c640 1 -- 172.21.15.125:0/3969803007 shutdown_connections 2023-10-18T12:00:30.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.534+0000 7f50d654c640 1 -- 172.21.15.125:0/3969803007 wait complete. 2023-10-18T12:00:30.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.535+0000 7f50d654c640 1 Processor -- start 2023-10-18T12:00:30.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d654c640 1 -- start start 2023-10-18T12:00:30.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d654c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d00ceb40 0x7f50d014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:30.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d654c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d00cf340 0x7f50d00c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:30.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d654c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 0x7f50d00ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:30.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d654c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d01496f0 con 0x7f50d015b2a0 2023-10-18T12:00:30.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d654c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d0148de0 con 0x7f50d00cf340 2023-10-18T12:00:30.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d654c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d014c2d0 con 0x7f50d00ceb40 2023-10-18T12:00:30.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d4ae9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 0x7f50d00ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:30.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d4ae9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 0x7f50d00ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32778/0 (socket says 172.21.15.125:32778) 2023-10-18T12:00:30.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.536+0000 7f50d4ae9640 1 -- 172.21.15.125:0/2384145300 learned_addr learned my addr 172.21.15.125:0/2384145300 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:30.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50cf7fe640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d00cf340 0x7f50d00c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:30.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50cffff640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d00ceb40 0x7f50d014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:30.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50d4ae9640 1 -- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d00ceb40 msgr2=0x7f50d014eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:30.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50d4ae9640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d00ceb40 0x7f50d014eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:30.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50d4ae9640 1 -- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d00cf340 msgr2=0x7f50d00c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:30.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50d4ae9640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d00cf340 0x7f50d00c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:30.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50d4ae9640 1 -- 172.21.15.125:0/2384145300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c4009490 con 0x7f50d015b2a0 2023-10-18T12:00:30.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50cffff640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d00ceb40 0x7f50d014eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:30.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50d4ae9640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 0x7f50d00ae3c0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f50c800c790 tx=0x7f50c800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50ccff9640 1 -- 172.21.15.125:0/2384145300 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c800cea0 con 0x7f50d015b2a0 2023-10-18T12:00:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50ccff9640 1 -- 172.21.15.125:0/2384145300 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f50c8003da0 con 0x7f50d015b2a0 2023-10-18T12:00:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50ccff9640 1 -- 172.21.15.125:0/2384145300 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c80052d0 con 0x7f50d015b2a0 2023-10-18T12:00:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d00c8ab0 con 0x7f50d015b2a0 2023-10-18T12:00:30.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.537+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d0149310 con 0x7f50d015b2a0 2023-10-18T12:00:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.538+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d0149830 con 0x7f50d015b2a0 2023-10-18T12:00:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.539+0000 7f50ccff9640 1 -- 172.21.15.125:0/2384145300 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f50c8005430 con 0x7f50d015b2a0 2023-10-18T12:00:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.539+0000 7f50ccff9640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f50bc06af60 0x7f50bc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.539+0000 7f50ccff9640 1 -- 172.21.15.125:0/2384145300 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f50c80899d0 con 0x7f50d015b2a0 2023-10-18T12:00:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.539+0000 7f50cffff640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f50bc06af60 0x7f50bc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.540+0000 7f50cffff640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f50bc06af60 0x7f50bc06d420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50d00b1b30 tx=0x7f50c0009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.544+0000 7f50ccff9640 1 -- 172.21.15.125:0/2384145300 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50c8054710 con 0x7f50d015b2a0 2023-10-18T12:00:30.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:30.708+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f50d00690e0 con 0x7f50d015b2a0 2023-10-18T12:00:31.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.412+0000 7f50ccff9640 1 -- 172.21.15.125:0/2384145300 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v223) v1 ==== 176+0+0 (secure 0 0 0) 0x7f50c8057d30 con 0x7f50d015b2a0 2023-10-18T12:00:31.415 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'backup_fs_data' created 2023-10-18T12:00:31.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f50bc06af60 msgr2=0x7f50bc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:31.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f50bc06af60 0x7f50bc06d420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50d00b1b30 tx=0x7f50c0009040 comp rx=0 tx=0).stop 2023-10-18T12:00:31.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 msgr2=0x7f50d00ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:31.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 0x7f50d00ae3c0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f50c800c790 tx=0x7f50c800cc50 comp rx=0 tx=0).stop 2023-10-18T12:00:31.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 shutdown_connections 2023-10-18T12:00:31.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f50bc06af60 0x7f50bc06d420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:31.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d00cf340 0x7f50d00c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:31.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f50d015b2a0 0x7f50d00ae3c0 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:31.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 --2- 172.21.15.125:0/2384145300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d00ceb40 0x7f50d014eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:31.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.414+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 >> 172.21.15.125:0/2384145300 conn(0x7f50d00d0d60 msgr2=0x7f50d00ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:31.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.415+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 shutdown_connections 2023-10-18T12:00:31.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.415+0000 7f50d654c640 1 -- 172.21.15.125:0/2384145300 wait complete. 2023-10-18T12:00:31.437 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-18T12:00:31.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.540+0000 7f3c9fca9640 1 -- 172.21.15.125:0/3637519979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c980cf340 msgr2=0x7f3c980d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:31.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.540+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/3637519979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c980cf340 0x7f3c980d1910 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3c88005fd0 tx=0x7f3c8802bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:31.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.541+0000 7f3c9fca9640 1 -- 172.21.15.125:0/3637519979 shutdown_connections 2023-10-18T12:00:31.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.541+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/3637519979 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c9815b2a0 0x7f3c980cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:31.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.541+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/3637519979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c980cf340 0x7f3c980d1910 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:31.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.541+0000 7f3c9fca9640 1 -- 172.21.15.125:0/3637519979 >> 172.21.15.125:0/3637519979 conn(0x7f3c980d0d60 msgr2=0x7f3c980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:31.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.541+0000 7f3c9fca9640 1 -- 172.21.15.125:0/3637519979 shutdown_connections 2023-10-18T12:00:31.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.541+0000 7f3c9fca9640 1 -- 172.21.15.125:0/3637519979 wait complete. 2023-10-18T12:00:31.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.542+0000 7f3c9fca9640 1 Processor -- start 2023-10-18T12:00:31.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.542+0000 7f3c9fca9640 1 -- start start 2023-10-18T12:00:31.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.542+0000 7f3c9fca9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c980ceb40 0x7f3c980c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:31.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.542+0000 7f3c9fca9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c980cf340 0x7f3c98158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:31.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.542+0000 7f3c9fca9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9815b2a0 0x7f3c980b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:31.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.542+0000 7f3c9fca9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980c9820 con 0x7f3c9815b2a0 2023-10-18T12:00:31.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.542+0000 7f3c9fca9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980c9390 con 0x7f3c980cf340 2023-10-18T12:00:31.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.542+0000 7f3c9fca9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980c8780 con 0x7f3c980ceb40 2023-10-18T12:00:31.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9815b2a0 0x7f3c980b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:31.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9815b2a0 0x7f3c980b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32786/0 (socket says 172.21.15.125:32786) 2023-10-18T12:00:31.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 -- 172.21.15.125:0/1504836745 learned_addr learned my addr 172.21.15.125:0/1504836745 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:31.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9d244640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c980cf340 0x7f3c98158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:31.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9da45640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c980ceb40 0x7f3c980c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:31.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 -- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c980ceb40 msgr2=0x7f3c980c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:31.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c980ceb40 0x7f3c980c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:31.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 -- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c980cf340 msgr2=0x7f3c98158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:31.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c980cf340 0x7f3c98158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:31.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 -- 172.21.15.125:0/1504836745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88009560 con 0x7f3c9815b2a0 2023-10-18T12:00:31.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9e246640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9815b2a0 0x7f3c980b5200 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000c710 tx=0x7f3c9000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:31.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c967fc640 1 -- 172.21.15.125:0/1504836745 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90006d50 con 0x7f3c9815b2a0 2023-10-18T12:00:31.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c967fc640 1 -- 172.21.15.125:0/1504836745 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c9000ce20 con 0x7f3c9815b2a0 2023-10-18T12:00:31.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c980c8a60 con 0x7f3c9815b2a0 2023-10-18T12:00:31.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c967fc640 1 -- 172.21.15.125:0/1504836745 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90003bd0 con 0x7f3c9815b2a0 2023-10-18T12:00:31.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.543+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c980c9020 con 0x7f3c9815b2a0 2023-10-18T12:00:31.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.544+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c980cb450 con 0x7f3c9815b2a0 2023-10-18T12:00:31.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.545+0000 7f3c967fc640 1 -- 172.21.15.125:0/1504836745 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3c90003d30 con 0x7f3c9815b2a0 2023-10-18T12:00:31.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.545+0000 7f3c967fc640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3c8406af60 0x7f3c8406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:31.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.545+0000 7f3c967fc640 1 -- 172.21.15.125:0/1504836745 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6843+0+0 (secure 0 0 0) 0x7f3c900898e0 con 0x7f3c9815b2a0 2023-10-18T12:00:31.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.545+0000 7f3c9da45640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3c8406af60 0x7f3c8406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:31.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.546+0000 7f3c9da45640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3c8406af60 0x7f3c8406d420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c88035000 tx=0x7f3c88004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:31.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.550+0000 7f3c967fc640 1 -- 172.21.15.125:0/1504836745 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c900544d0 con 0x7f3c9815b2a0 2023-10-18T12:00:31.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:31.749+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f3c980690e0 con 0x7f3c9815b2a0 2023-10-18T12:00:32.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.413+0000 7f3c967fc640 1 -- 172.21.15.125:0/1504836745 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 29 and data pool 30 v88) v1 ==== 183+0+0 (secure 0 0 0) 0x7f3c90057af0 con 0x7f3c9815b2a0 2023-10-18T12:00:32.417 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 29 and data pool 30 2023-10-18T12:00:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.417+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3c8406af60 msgr2=0x7f3c8406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.417+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3c8406af60 0x7f3c8406d420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c88035000 tx=0x7f3c88004730 comp rx=0 tx=0).stop 2023-10-18T12:00:32.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.417+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9815b2a0 msgr2=0x7f3c980b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.417+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9815b2a0 0x7f3c980b5200 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000c710 tx=0x7f3c9000cbd0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.418+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 shutdown_connections 2023-10-18T12:00:32.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.418+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3c8406af60 0x7f3c8406d420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.418+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c980cf340 0x7f3c98158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.418+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9815b2a0 0x7f3c980b5200 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.418+0000 7f3c9fca9640 1 --2- 172.21.15.125:0/1504836745 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c980ceb40 0x7f3c980c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.418+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 >> 172.21.15.125:0/1504836745 conn(0x7f3c980d0d60 msgr2=0x7f3c9806afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:32.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.418+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 shutdown_connections 2023-10-18T12:00:32.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.419+0000 7f3c9fca9640 1 -- 172.21.15.125:0/1504836745 wait complete. 2023-10-18T12:00:32.447 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:32.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.548+0000 7fd9e82e5640 1 -- 172.21.15.125:0/172566910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e015b2a0 msgr2=0x7fd9e00cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.548+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/172566910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e015b2a0 0x7fd9e00cf720 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009830 tx=0x7fd9d002bcd0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.548+0000 7fd9e82e5640 1 -- 172.21.15.125:0/172566910 shutdown_connections 2023-10-18T12:00:32.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.548+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/172566910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e015b2a0 0x7fd9e00cf720 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.548+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/172566910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e00cf340 0x7fd9e00d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.548+0000 7fd9e82e5640 1 -- 172.21.15.125:0/172566910 >> 172.21.15.125:0/172566910 conn(0x7fd9e00d0d60 msgr2=0x7fd9e00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:32.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.548+0000 7fd9e82e5640 1 -- 172.21.15.125:0/172566910 shutdown_connections 2023-10-18T12:00:32.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.548+0000 7fd9e82e5640 1 -- 172.21.15.125:0/172566910 wait complete. 2023-10-18T12:00:32.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.549+0000 7fd9e82e5640 1 Processor -- start 2023-10-18T12:00:32.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e82e5640 1 -- start start 2023-10-18T12:00:32.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e82e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e00ceb40 0x7fd9e01e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:32.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e82e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e00cf340 0x7fd9e0158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:32.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e82e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e015b2a0 0x7fd9e00b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:32.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e82e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e01e42d0 con 0x7fd9e00ceb40 2023-10-18T12:00:32.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e82e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e01e4520 con 0x7fd9e00cf340 2023-10-18T12:00:32.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e82e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e01e4800 con 0x7fd9e015b2a0 2023-10-18T12:00:32.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e6081640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e00ceb40 0x7fd9e01e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:32.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e6081640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e00ceb40 0x7fd9e01e55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32800/0 (socket says 172.21.15.125:32800) 2023-10-18T12:00:32.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e6081640 1 -- 172.21.15.125:0/1831212330 learned_addr learned my addr 172.21.15.125:0/1831212330 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:32.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.550+0000 7fd9e5880640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e00cf340 0x7fd9e0158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:32.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e6882640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e015b2a0 0x7fd9e00b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:32.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e6081640 1 -- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e015b2a0 msgr2=0x7fd9e00b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e6081640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e015b2a0 0x7fd9e00b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e6081640 1 -- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e00cf340 msgr2=0x7fd9e0158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e6081640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e00cf340 0x7fd9e0158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e6081640 1 -- 172.21.15.125:0/1831212330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc009900 con 0x7fd9e00ceb40 2023-10-18T12:00:32.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e5880640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e00cf340 0x7fd9e0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:32.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e6081640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e00ceb40 0x7fd9e01e55c0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00c420 tx=0x7fd9dc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:32.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9d6ffd640 1 -- 172.21.15.125:0/1831212330 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc003970 con 0x7fd9e00ceb40 2023-10-18T12:00:32.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9d6ffd640 1 -- 172.21.15.125:0/1831212330 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9dc003bc0 con 0x7fd9e00ceb40 2023-10-18T12:00:32.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d0009490 con 0x7fd9e00ceb40 2023-10-18T12:00:32.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9d6ffd640 1 -- 172.21.15.125:0/1831212330 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc004020 con 0x7fd9e00ceb40 2023-10-18T12:00:32.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.551+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e01e4b90 con 0x7fd9e00ceb40 2023-10-18T12:00:32.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.552+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e006bd40 con 0x7fd9e00ceb40 2023-10-18T12:00:32.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.553+0000 7fd9d6ffd640 1 -- 172.21.15.125:0/1831212330 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd9dc00a040 con 0x7fd9e00ceb40 2023-10-18T12:00:32.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.553+0000 7fd9d6ffd640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd9c406af60 0x7fd9c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:32.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.553+0000 7fd9d6ffd640 1 -- 172.21.15.125:0/1831212330 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fd9dc08a640 con 0x7fd9e00ceb40 2023-10-18T12:00:32.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.553+0000 7fd9e5880640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd9c406af60 0x7fd9c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:32.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.554+0000 7fd9e5880640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd9c406af60 0x7fd9c406d420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0036000 tx=0x7fd9d0004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:32.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.557+0000 7fd9d6ffd640 1 -- 172.21.15.125:0/1831212330 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9dc054ff0 con 0x7fd9e00ceb40 2023-10-18T12:00:32.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.725+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9e00d04f0 con 0x7fd9e00ceb40 2023-10-18T12:00:32.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.727+0000 7fd9d6ffd640 1 -- 172.21.15.125:0/1831212330 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+20591 (secure 0 0 0) 0x7fd9dc058610 con 0x7fd9e00ceb40 2023-10-18T12:00:32.729 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:32.730 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":224,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:31.750859+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:32.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd9c406af60 msgr2=0x7fd9c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd9c406af60 0x7fd9c406d420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0036000 tx=0x7fd9d0004730 comp rx=0 tx=0).stop 2023-10-18T12:00:32.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e00ceb40 msgr2=0x7fd9e01e55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e00ceb40 0x7fd9e01e55c0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00c420 tx=0x7fd9dc009f90 comp rx=0 tx=0).stop 2023-10-18T12:00:32.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 shutdown_connections 2023-10-18T12:00:32.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fd9c406af60 0x7fd9c406d420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e00cf340 0x7fd9e0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e015b2a0 0x7fd9e00b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 --2- 172.21.15.125:0/1831212330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9e00ceb40 0x7fd9e01e55c0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 >> 172.21.15.125:0/1831212330 conn(0x7fd9e00d0d60 msgr2=0x7fd9e006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:32.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 shutdown_connections 2023-10-18T12:00:32.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.729+0000 7fd9e82e5640 1 -- 172.21.15.125:0/1831212330 wait complete. 2023-10-18T12:00:32.742 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:32.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.843+0000 7fa55686f640 1 -- 172.21.15.125:0/152213989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5500ceb40 msgr2=0x7fa5500b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.843+0000 7fa55686f640 1 --2- 172.21.15.125:0/152213989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5500ceb40 0x7fa5500b0650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa548005fd0 tx=0x7fa54802c240 comp rx=0 tx=0).stop 2023-10-18T12:00:32.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.843+0000 7fa55686f640 1 -- 172.21.15.125:0/152213989 shutdown_connections 2023-10-18T12:00:32.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.843+0000 7fa55686f640 1 --2- 172.21.15.125:0/152213989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5500ceb40 0x7fa5500b0650 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.843+0000 7fa55686f640 1 --2- 172.21.15.125:0/152213989 >> v2:172.21.15.139:3300/0 conn(0x7fa55015b2a0 0x7fa5500d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.843+0000 7fa55686f640 1 -- 172.21.15.125:0/152213989 >> 172.21.15.125:0/152213989 conn(0x7fa5500d0d60 msgr2=0x7fa5500d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:32.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.843+0000 7fa55686f640 1 -- 172.21.15.125:0/152213989 shutdown_connections 2023-10-18T12:00:32.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.843+0000 7fa55686f640 1 -- 172.21.15.125:0/152213989 wait complete. 2023-10-18T12:00:32.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.844+0000 7fa55686f640 1 Processor -- start 2023-10-18T12:00:32.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.844+0000 7fa55686f640 1 -- start start 2023-10-18T12:00:32.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa55686f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5500ceb40 0x7fa55009d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:32.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa55686f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5500cf340 0x7fa55009e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:32.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa55686f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa55015b2a0 0x7fa55006b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:32.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa55686f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55009ea60 con 0x7fa5500ceb40 2023-10-18T12:00:32.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa55686f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500b3f30 con 0x7fa55015b2a0 2023-10-18T12:00:32.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa55686f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55009dd50 con 0x7fa5500cf340 2023-10-18T12:00:32.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5500ceb40 0x7fa55009d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:32.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5500ceb40 0x7fa55009d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32810/0 (socket says 172.21.15.125:32810) 2023-10-18T12:00:32.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 -- 172.21.15.125:0/3134806289 learned_addr learned my addr 172.21.15.125:0/3134806289 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:32.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa554e0c640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa55015b2a0 0x7fa55006b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:32.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54f7fe640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5500cf340 0x7fa55009e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:32.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 -- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5500cf340 msgr2=0x7fa55009e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5500cf340 0x7fa55009e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 -- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa55015b2a0 msgr2=0x7fa55006b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:32.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa55015b2a0 0x7fa55006b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:32.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 -- 172.21.15.125:0/3134806289 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548009900 con 0x7fa5500ceb40 2023-10-18T12:00:32.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.845+0000 7fa54ffff640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5500ceb40 0x7fa55009d6c0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c400 tx=0x7fa54000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:32.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.846+0000 7fa54cff9640 1 -- 172.21.15.125:0/3134806289 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5400052d0 con 0x7fa5500ceb40 2023-10-18T12:00:32.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.846+0000 7fa54cff9640 1 -- 172.21.15.125:0/3134806289 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa540007020 con 0x7fa5500ceb40 2023-10-18T12:00:32.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.846+0000 7fa54cff9640 1 -- 172.21.15.125:0/3134806289 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540007470 con 0x7fa5500ceb40 2023-10-18T12:00:32.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.846+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55009e030 con 0x7fa5500ceb40 2023-10-18T12:00:32.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.846+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5500b3bc0 con 0x7fa5500ceb40 2023-10-18T12:00:32.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.847+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5500690e0 con 0x7fa5500ceb40 2023-10-18T12:00:32.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.847+0000 7fa54cff9640 1 -- 172.21.15.125:0/3134806289 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa5400067d0 con 0x7fa5500ceb40 2023-10-18T12:00:32.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.847+0000 7fa54cff9640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa53c06af90 0x7fa53c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:32.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.848+0000 7fa54cff9640 1 -- 172.21.15.125:0/3134806289 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fa540089240 con 0x7fa5500ceb40 2023-10-18T12:00:32.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.848+0000 7fa54f7fe640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa53c06af90 0x7fa53c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:32.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.848+0000 7fa54f7fe640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa53c06af90 0x7fa53c06d450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa55009ab10 tx=0x7fa544005f90 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:32.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:32.852+0000 7fa54cff9640 1 -- 172.21.15.125:0/3134806289 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa540053d90 con 0x7fa5500ceb40 2023-10-18T12:00:33.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.017+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5500b0cd0 con 0x7fa5500ceb40 2023-10-18T12:00:33.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.019+0000 7fa54cff9640 1 -- 172.21.15.125:0/3134806289 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+20591 (secure 0 0 0) 0x7fa5400573b0 con 0x7fa5500ceb40 2023-10-18T12:00:33.022 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:33.022 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":224,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:31.750859+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:33.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa53c06af90 msgr2=0x7fa53c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa53c06af90 0x7fa53c06d450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa55009ab10 tx=0x7fa544005f90 comp rx=0 tx=0).stop 2023-10-18T12:00:33.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5500ceb40 msgr2=0x7fa55009d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5500ceb40 0x7fa55009d6c0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c400 tx=0x7fa54000c8c0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 shutdown_connections 2023-10-18T12:00:33.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fa53c06af90 0x7fa53c06d450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa55015b2a0 0x7fa55006b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa5500cf340 0x7fa55009e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 --2- 172.21.15.125:0/3134806289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5500ceb40 0x7fa55009d6c0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 >> 172.21.15.125:0/3134806289 conn(0x7fa5500d0d60 msgr2=0x7fa5500af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:33.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 shutdown_connections 2023-10-18T12:00:33.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.021+0000 7fa55686f640 1 -- 172.21.15.125:0/3134806289 wait complete. 2023-10-18T12:00:33.034 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-18T12:00:33.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.136+0000 7f6a729b6640 1 -- 172.21.15.125:0/3345252907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 msgr2=0x7f6a6c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.136+0000 7f6a729b6640 1 --2- 172.21.15.125:0/3345252907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 0x7f6a6c06b2e0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009860 tx=0x7f6a5c02bcd0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.136+0000 7f6a729b6640 1 -- 172.21.15.125:0/3345252907 shutdown_connections 2023-10-18T12:00:33.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.136+0000 7f6a729b6640 1 --2- 172.21.15.125:0/3345252907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 0x7f6a6c06b2e0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.136+0000 7f6a729b6640 1 -- 172.21.15.125:0/3345252907 >> 172.21.15.125:0/3345252907 conn(0x7f6a6c069930 msgr2=0x7f6a6c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:33.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.136+0000 7f6a729b6640 1 -- 172.21.15.125:0/3345252907 shutdown_connections 2023-10-18T12:00:33.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.136+0000 7f6a729b6640 1 -- 172.21.15.125:0/3345252907 wait complete. 2023-10-18T12:00:33.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.137+0000 7f6a729b6640 1 Processor -- start 2023-10-18T12:00:33.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.137+0000 7f6a729b6640 1 -- start start 2023-10-18T12:00:33.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a729b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 0x7f6a6c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:33.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a729b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a6c0abc10 0x7f6a6c070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:33.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a729b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a6c0c96b0 0x7f6a6c0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:33.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a729b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1588b0 con 0x7f6a6c06af00 2023-10-18T12:00:33.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a729b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c157a00 con 0x7f6a6c0c96b0 2023-10-18T12:00:33.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a729b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c157c30 con 0x7f6a6c0abc10 2023-10-18T12:00:33.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 0x7f6a6c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:33.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 0x7f6a6c155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32830/0 (socket says 172.21.15.125:32830) 2023-10-18T12:00:33.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6bfff640 1 -- 172.21.15.125:0/815498044 learned_addr learned my addr 172.21.15.125:0/815498044 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:33.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a70f53640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a6c0c96b0 0x7f6a6c0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:33.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6bfff640 1 -- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a6c0abc10 msgr2=0x7f6a6c070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6b7fe640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a6c0abc10 0x7f6a6c070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:33.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6bfff640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a6c0abc10 0x7f6a6c070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6bfff640 1 -- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a6c0c96b0 msgr2=0x7f6a6c0aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6bfff640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a6c0c96b0 0x7f6a6c0aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a6bfff640 1 -- 172.21.15.125:0/815498044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c009490 con 0x7f6a6c06af00 2023-10-18T12:00:33.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.138+0000 7f6a70f53640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a6c0c96b0 0x7f6a6c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:33.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.139+0000 7f6a6b7fe640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a6c0abc10 0x7f6a6c070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:33.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.139+0000 7f6a6bfff640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 0x7f6a6c155f40 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009780 tx=0x7f6a5c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:33.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.139+0000 7f6a68ff9640 1 -- 172.21.15.125:0/815498044 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c003ea0 con 0x7f6a6c06af00 2023-10-18T12:00:33.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.139+0000 7f6a68ff9640 1 -- 172.21.15.125:0/815498044 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6a5c034550 con 0x7f6a6c06af00 2023-10-18T12:00:33.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.139+0000 7f6a68ff9640 1 -- 172.21.15.125:0/815498044 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c034960 con 0x7f6a6c06af00 2023-10-18T12:00:33.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.139+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c159a20 con 0x7f6a6c06af00 2023-10-18T12:00:33.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.139+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c1581f0 con 0x7f6a6c06af00 2023-10-18T12:00:33.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.140+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c06bf40 con 0x7f6a6c06af00 2023-10-18T12:00:33.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.140+0000 7f6a68ff9640 1 -- 172.21.15.125:0/815498044 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6a5c03b040 con 0x7f6a6c06af00 2023-10-18T12:00:33.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.141+0000 7f6a68ff9640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6a5806af60 0x7f6a5806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:33.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.141+0000 7f6a68ff9640 1 -- 172.21.15.125:0/815498044 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f6a5c0ac2c0 con 0x7f6a6c06af00 2023-10-18T12:00:33.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.141+0000 7f6a6b7fe640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6a5806af60 0x7f6a5806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:33.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.142+0000 7f6a6b7fe640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6a5806af60 0x7f6a5806d420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c155690 tx=0x7f6a60009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:33.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.145+0000 7f6a68ff9640 1 -- 172.21.15.125:0/815498044 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a5c076d80 con 0x7f6a6c06af00 2023-10-18T12:00:33.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.344+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6a6c0690e0 con 0x7f6a6c06af00 2023-10-18T12:00:33.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.431+0000 7f6a68ff9640 1 -- 172.21.15.125:0/815498044 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6a5c0410c0 con 0x7f6a6c06af00 2023-10-18T12:00:33.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6a5806af60 msgr2=0x7f6a5806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6a5806af60 0x7f6a5806d420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c155690 tx=0x7f6a60009040 comp rx=0 tx=0).stop 2023-10-18T12:00:33.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 msgr2=0x7f6a6c155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 0x7f6a6c155f40 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009780 tx=0x7f6a5c0038a0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 shutdown_connections 2023-10-18T12:00:33.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f6a5806af60 0x7f6a5806d420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a6c0c96b0 0x7f6a6c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a6c0abc10 0x7f6a6c070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 --2- 172.21.15.125:0/815498044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6a6c06af00 0x7f6a6c155f40 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 >> 172.21.15.125:0/815498044 conn(0x7f6a6c069930 msgr2=0x7f6a6c15b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:33.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 shutdown_connections 2023-10-18T12:00:33.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.434+0000 7f6a729b6640 1 -- 172.21.15.125:0/815498044 wait complete. 2023-10-18T12:00:33.447 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T12:00:33.448 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-18T12:00:33.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 -- 172.21.15.125:0/3362831547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00980705a0 msgr2=0x7f00980b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 --2- 172.21.15.125:0/3362831547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00980705a0 0x7f00980b0530 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f00980cc7e0 tx=0x7f009002c250 comp rx=0 tx=0).stop 2023-10-18T12:00:33.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 -- 172.21.15.125:0/3362831547 shutdown_connections 2023-10-18T12:00:33.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 --2- 172.21.15.125:0/3362831547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00980705a0 0x7f00980b0530 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 --2- 172.21.15.125:0/3362831547 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0098158300 0x7f00980c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 --2- 172.21.15.125:0/3362831547 >> v2:172.21.15.139:3300/0 conn(0x7f00980afa20 0x7f00980cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 -- 172.21.15.125:0/3362831547 >> 172.21.15.125:0/3362831547 conn(0x7f0098069930 msgr2=0x7f0098069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:33.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 -- 172.21.15.125:0/3362831547 shutdown_connections 2023-10-18T12:00:33.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.551+0000 7f009dcd7640 1 -- 172.21.15.125:0/3362831547 wait complete. 2023-10-18T12:00:33.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.552+0000 7f009dcd7640 1 Processor -- start 2023-10-18T12:00:33.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.552+0000 7f009dcd7640 1 -- start start 2023-10-18T12:00:33.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f009dcd7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f00980705a0 0x7f00981f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:33.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f009dcd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980afa20 0x7f00981efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:33.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f009dcd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0098158300 0x7f009813d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:33.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f009dcd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00981f3650 con 0x7f0098158300 2023-10-18T12:00:33.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f009dcd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009813d9c0 con 0x7f00980afa20 2023-10-18T12:00:33.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f009dcd7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00981f2db0 con 0x7f00980705a0 2023-10-18T12:00:33.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f0097fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0098158300 0x7f009813d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:33.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f0097fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0098158300 0x7f009813d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32842/0 (socket says 172.21.15.125:32842) 2023-10-18T12:00:33.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.553+0000 7f0097fff640 1 -- 172.21.15.125:0/4252194818 learned_addr learned my addr 172.21.15.125:0/4252194818 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:33.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f0097fff640 1 -- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f00980705a0 msgr2=0x7f00981f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f0097fff640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f00980705a0 0x7f00981f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f0097fff640 1 -- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980afa20 msgr2=0x7f00981efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:33.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f0097fff640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980afa20 0x7f00981efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:33.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f0097fff640 1 -- 172.21.15.125:0/4252194818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0090009900 con 0x7f0098158300 2023-10-18T12:00:33.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f0097fff640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0098158300 0x7f009813d480 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f0090037040 tx=0x7f0090035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:33.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f007ffff640 1 -- 172.21.15.125:0/4252194818 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090034440 con 0x7f0098158300 2023-10-18T12:00:33.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f007ffff640 1 -- 172.21.15.125:0/4252194818 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0090004250 con 0x7f0098158300 2023-10-18T12:00:33.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f007ffff640 1 -- 172.21.15.125:0/4252194818 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090004640 con 0x7f0098158300 2023-10-18T12:00:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00981f3090 con 0x7f0098158300 2023-10-18T12:00:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.554+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00981f0400 con 0x7f0098158300 2023-10-18T12:00:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.555+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00981f0240 con 0x7f0098158300 2023-10-18T12:00:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.555+0000 7f007ffff640 1 -- 172.21.15.125:0/4252194818 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f009003b040 con 0x7f0098158300 2023-10-18T12:00:33.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.556+0000 7f007ffff640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f008406af60 0x7f008406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:33.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.556+0000 7f007ffff640 1 -- 172.21.15.125:0/4252194818 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f00900ac9c0 con 0x7f0098158300 2023-10-18T12:00:33.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.556+0000 7f00977fe640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f008406af60 0x7f008406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:33.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.556+0000 7f00977fe640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f008406af60 0x7f008406d420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0088009710 tx=0x7f0088006c90 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:33.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.560+0000 7f007ffff640 1 -- 172.21.15.125:0/4252194818 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0090077500 con 0x7f0098158300 2023-10-18T12:00:33.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:33.760+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f00980690e0 con 0x7f0098158300 2023-10-18T12:00:34.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.432+0000 7f007ffff640 1 -- 172.21.15.125:0/4252194818 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 125+0+0 (secure 0 0 0) 0x7f009007ab20 con 0x7f0098158300 2023-10-18T12:00:34.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f008406af60 msgr2=0x7f008406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f008406af60 0x7f008406d420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0088009710 tx=0x7f0088006c90 comp rx=0 tx=0).stop 2023-10-18T12:00:34.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0098158300 msgr2=0x7f009813d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0098158300 0x7f009813d480 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f0090037040 tx=0x7f0090035db0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 shutdown_connections 2023-10-18T12:00:34.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f008406af60 0x7f008406d420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980afa20 0x7f00981efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0098158300 0x7f009813d480 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 --2- 172.21.15.125:0/4252194818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f00980705a0 0x7f00981f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 >> 172.21.15.125:0/4252194818 conn(0x7f0098069930 msgr2=0x7f0098151020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:34.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 shutdown_connections 2023-10-18T12:00:34.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.435+0000 7f009dcd7640 1 -- 172.21.15.125:0/4252194818 wait complete. 2023-10-18T12:00:34.451 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-18T12:00:34.451 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:34.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.556+0000 7f7708c90640 1 -- 172.21.15.125:0/2341665619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0003970 msgr2=0x7f76f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.556+0000 7f7708c90640 1 --2- 172.21.15.125:0/2341665619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0003970 0x7f76f0003d90 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009b50 tx=0x7f76f00057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.556+0000 7f7702d9d640 1 -- 172.21.15.125:0/2341665619 reap_dead start 2023-10-18T12:00:34.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.556+0000 7f7708c90640 1 -- 172.21.15.125:0/2341665619 shutdown_connections 2023-10-18T12:00:34.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.556+0000 7f7708c90640 1 -- 172.21.15.125:0/2341665619 >> 172.21.15.125:0/2341665619 conn(0x7f7704069930 msgr2=0x7f7704069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:34.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.556+0000 7f7708c90640 1 -- 172.21.15.125:0/2341665619 shutdown_connections 2023-10-18T12:00:34.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.556+0000 7f7708c90640 1 -- 172.21.15.125:0/2341665619 wait complete. 2023-10-18T12:00:34.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.557+0000 7f7708c90640 1 Processor -- start 2023-10-18T12:00:34.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.557+0000 7f7708c90640 1 -- start start 2023-10-18T12:00:34.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f7708c90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f0003970 0x7f77041ef660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:34.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f7708c90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7704159970 0x7f770406bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:34.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f7708c90640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7704159df0 0x7f770406c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:34.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f7708c90640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704145fb0 con 0x7f7704159970 2023-10-18T12:00:34.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f7708c90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704146290 con 0x7f76f0003970 2023-10-18T12:00:34.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f7708c90640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7704146570 con 0x7f7704159df0 2023-10-18T12:00:34.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f76fbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7704159970 0x7f770406bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:34.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f76fbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7704159970 0x7f770406bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32852/0 (socket says 172.21.15.125:32852) 2023-10-18T12:00:34.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f76fbfff640 1 -- 172.21.15.125:0/2637731977 learned_addr learned my addr 172.21.15.125:0/2637731977 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:34.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f7702d9d640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7704159df0 0x7f770406c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:34.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f770259c640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f0003970 0x7f77041ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:34.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f76fbfff640 1 -- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7704159df0 msgr2=0x7f770406c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f76fbfff640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7704159df0 0x7f770406c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f76fbfff640 1 -- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f0003970 msgr2=0x7f77041ef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f76fbfff640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f0003970 0x7f77041ef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.558+0000 7f76fbfff640 1 -- 172.21.15.125:0/2637731977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704070cf0 con 0x7f7704159970 2023-10-18T12:00:34.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.559+0000 7f76fbfff640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7704159970 0x7f770406bef0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00fa00 tx=0x7f76ec00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:34.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.559+0000 7f76faffd640 1 -- 172.21.15.125:0/2637731977 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00bcb0 con 0x7f7704159970 2023-10-18T12:00:34.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.559+0000 7f76faffd640 1 -- 172.21.15.125:0/2637731977 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76ec005020 con 0x7f7704159970 2023-10-18T12:00:34.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.559+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7704146850 con 0x7f7704159970 2023-10-18T12:00:34.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.559+0000 7f76faffd640 1 -- 172.21.15.125:0/2637731977 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec005430 con 0x7f7704159970 2023-10-18T12:00:34.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.559+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77041efba0 con 0x7f7704159970 2023-10-18T12:00:34.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.560+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77040690e0 con 0x7f7704159970 2023-10-18T12:00:34.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.560+0000 7f76faffd640 1 -- 172.21.15.125:0/2637731977 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f76ec0047a0 con 0x7f7704159970 2023-10-18T12:00:34.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.561+0000 7f76faffd640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f76e406af60 0x7f76e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:34.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.561+0000 7f76faffd640 1 -- 172.21.15.125:0/2637731977 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f76ec08a690 con 0x7f7704159970 2023-10-18T12:00:34.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.561+0000 7f770259c640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f76e406af60 0x7f76e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:34.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.561+0000 7f770259c640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f76e406af60 0x7f76e406d420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036000 tx=0x7f76f40046c0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:34.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.565+0000 7f76faffd640 1 -- 172.21.15.125:0/2637731977 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76ec0550f0 con 0x7f7704159970 2023-10-18T12:00:34.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.767+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77040cc1b0 con 0x7f7704159970 2023-10-18T12:00:34.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.767+0000 7f76faffd640 1 -- 172.21.15.125:0/2637731977 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+6155 (secure 0 0 0) 0x7f76ec058710 con 0x7f7704159970 2023-10-18T12:00:34.770 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:34.770 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":90,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:20.388305+0000","modified":"2023-10-18T12:00:22.406153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11976":{"gid":11976,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2975172452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2975172452},{"type":"v1","addr":"172.21.15.125:6834","nonce":2975172452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:34.771 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 90 2023-10-18T12:00:34.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f76e406af60 msgr2=0x7f76e406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f76e406af60 0x7f76e406d420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036000 tx=0x7f76f40046c0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7704159970 msgr2=0x7f770406bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7704159970 0x7f770406bef0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00fa00 tx=0x7f76ec00fec0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 shutdown_connections 2023-10-18T12:00:34.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f76e406af60 0x7f76e406d420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7704159df0 0x7f770406c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7704159970 0x7f770406bef0 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 --2- 172.21.15.125:0/2637731977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f0003970 0x7f77041ef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 >> 172.21.15.125:0/2637731977 conn(0x7f7704069930 msgr2=0x7f770414bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:34.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 shutdown_connections 2023-10-18T12:00:34.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.769+0000 7f7708c90640 1 -- 172.21.15.125:0/2637731977 wait complete. 2023-10-18T12:00:34.782 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:34.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.882+0000 7f9dd3b65640 1 -- 172.21.15.125:0/65633378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc15eba0 msgr2=0x7f9dcc0b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.882+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/65633378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc15eba0 0x7f9dcc0b00f0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0ce750 tx=0x7f9dbc02be00 comp rx=0 tx=0).stop 2023-10-18T12:00:34.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.882+0000 7f9dd3b65640 1 -- 172.21.15.125:0/65633378 shutdown_connections 2023-10-18T12:00:34.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.882+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/65633378 >> v2:172.21.15.139:3300/0 conn(0x7f9dcc0cf2a0 0x7f9dcc0cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.882+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/65633378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc15eba0 0x7f9dcc0b00f0 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.882+0000 7f9dd3b65640 1 -- 172.21.15.125:0/65633378 >> 172.21.15.125:0/65633378 conn(0x7f9dcc08a650 msgr2=0x7f9dcc099f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:34.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.882+0000 7f9dd3b65640 1 -- 172.21.15.125:0/65633378 shutdown_connections 2023-10-18T12:00:34.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.882+0000 7f9dd3b65640 1 -- 172.21.15.125:0/65633378 wait complete. 2023-10-18T12:00:34.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.883+0000 7f9dd3b65640 1 Processor -- start 2023-10-18T12:00:34.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.883+0000 7f9dd3b65640 1 -- start start 2023-10-18T12:00:34.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd3b65640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc0cf2a0 0x7f9dcc0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:34.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd3b65640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9dcc15b200 0x7f9dcc1500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:34.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd3b65640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9dcc15eba0 0x7f9dcc15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:34.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd3b65640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc150600 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd3b65640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0c5610 con 0x7f9dcc15eba0 2023-10-18T12:00:34.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd3b65640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0c58f0 con 0x7f9dcc15b200 2023-10-18T12:00:34.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc0cf2a0 0x7f9dcc0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:34.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc0cf2a0 0x7f9dcc0c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32870/0 (socket says 172.21.15.125:32870) 2023-10-18T12:00:34.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 -- 172.21.15.125:0/1997139410 learned_addr learned my addr 172.21.15.125:0/1997139410 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:34.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 -- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9dcc15b200 msgr2=0x7f9dcc1500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9dcc15b200 0x7f9dcc1500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 -- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9dcc15eba0 msgr2=0x7f9dcc15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:34.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9dcc15eba0 0x7f9dcc15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:34.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 -- 172.21.15.125:0/1997139410 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc009490 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.884+0000 7f9dd1901640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc0cf2a0 0x7f9dcc0c7210 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800c890 tx=0x7f9dc800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:34.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.885+0000 7f9dc27fc640 1 -- 172.21.15.125:0/1997139410 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8003da0 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.885+0000 7f9dc27fc640 1 -- 172.21.15.125:0/1997139410 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9dc8005070 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.885+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc0c5bd0 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.885+0000 7f9dc27fc640 1 -- 172.21.15.125:0/1997139410 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8005480 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.885+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc0c6220 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.886+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dcc066930 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.886+0000 7f9dc27fc640 1 -- 172.21.15.125:0/1997139410 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9dc80055e0 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.886+0000 7f9dc27fc640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f9db006af90 0x7f9db006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:34.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.886+0000 7f9dc27fc640 1 -- 172.21.15.125:0/1997139410 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f9dc8089870 con 0x7f9dcc0cf2a0 2023-10-18T12:00:34.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.887+0000 7f9dd1100640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f9db006af90 0x7f9db006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:34.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.887+0000 7f9dd1100640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f9db006af90 0x7f9db006d450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc036000 tx=0x7f9dbc0046c0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:34.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:34.891+0000 7f9dc27fc640 1 -- 172.21.15.125:0/1997139410 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9dc8054330 con 0x7f9dcc0cf2a0 2023-10-18T12:00:35.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.059+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dcc04dae0 con 0x7f9dcc0cf2a0 2023-10-18T12:00:35.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.061+0000 7f9dc27fc640 1 -- 172.21.15.125:0/1997139410 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) v1 ==== 74+0+20578 (secure 0 0 0) 0x7f9dc8017090 con 0x7f9dcc0cf2a0 2023-10-18T12:00:35.064 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:35.064 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":225,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:33.414920+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:35.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.062+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f9db006af90 msgr2=0x7f9db006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.062+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f9db006af90 0x7f9db006d450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc036000 tx=0x7f9dbc0046c0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.062+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc0cf2a0 msgr2=0x7f9dcc0c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.062+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc0cf2a0 0x7f9dcc0c7210 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800c890 tx=0x7f9dc800cd50 comp rx=0 tx=0).stop 2023-10-18T12:00:35.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.062+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 shutdown_connections 2023-10-18T12:00:35.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.063+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f9db006af90 0x7f9db006d450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.063+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9dcc15eba0 0x7f9dcc15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.063+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9dcc15b200 0x7f9dcc1500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.063+0000 7f9dd3b65640 1 --2- 172.21.15.125:0/1997139410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9dcc0cf2a0 0x7f9dcc0c7210 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.063+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 >> 172.21.15.125:0/1997139410 conn(0x7f9dcc08a650 msgr2=0x7f9dcc06b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:35.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.063+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 shutdown_connections 2023-10-18T12:00:35.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.063+0000 7f9dd3b65640 1 -- 172.21.15.125:0/1997139410 wait complete. 2023-10-18T12:00:35.076 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T12:00:35.076 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:00:35.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.175+0000 7f7853916640 1 -- 172.21.15.125:0/3880047626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 msgr2=0x7f784c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.175+0000 7f7853916640 1 --2- 172.21.15.125:0/3880047626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 0x7f784c06b2e0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f7848009900 tx=0x7f784802be00 comp rx=0 tx=0).stop 2023-10-18T12:00:35.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.175+0000 7f7853916640 1 -- 172.21.15.125:0/3880047626 shutdown_connections 2023-10-18T12:00:35.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.175+0000 7f7853916640 1 --2- 172.21.15.125:0/3880047626 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f784c0c96b0 0x7f784c0c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.175+0000 7f7853916640 1 --2- 172.21.15.125:0/3880047626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 0x7f784c06b2e0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.175+0000 7f7853916640 1 -- 172.21.15.125:0/3880047626 >> 172.21.15.125:0/3880047626 conn(0x7f784c069930 msgr2=0x7f784c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:35.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.175+0000 7f7853916640 1 -- 172.21.15.125:0/3880047626 shutdown_connections 2023-10-18T12:00:35.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.176+0000 7f7853916640 1 -- 172.21.15.125:0/3880047626 wait complete. 2023-10-18T12:00:35.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f7853916640 1 Processor -- start 2023-10-18T12:00:35.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f7853916640 1 -- start start 2023-10-18T12:00:35.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f7853916640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 0x7f784c13d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f7853916640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784c0ab500 0x7f784c0a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f7853916640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f784c0c96b0 0x7f784c0a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f7853916640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c0aa520 con 0x7f784c06af00 2023-10-18T12:00:35.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f7853916640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c1f0240 con 0x7f784c0ab500 2023-10-18T12:00:35.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f7853916640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c1f0430 con 0x7f784c0c96b0 2023-10-18T12:00:35.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f78516b2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 0x7f784c13d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f78516b2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 0x7f784c13d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32894/0 (socket says 172.21.15.125:32894) 2023-10-18T12:00:35.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.177+0000 7f78516b2640 1 -- 172.21.15.125:0/2666581878 learned_addr learned my addr 172.21.15.125:0/2666581878 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:35.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7851eb3640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f784c0c96b0 0x7f784c0a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7850eb1640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784c0ab500 0x7f784c0a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f78516b2640 1 -- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f784c0c96b0 msgr2=0x7f784c0a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f78516b2640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f784c0c96b0 0x7f784c0a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f78516b2640 1 -- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784c0ab500 msgr2=0x7f784c0a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f78516b2640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784c0ab500 0x7f784c0a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f78516b2640 1 -- 172.21.15.125:0/2666581878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7848009560 con 0x7f784c06af00 2023-10-18T12:00:35.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7851eb3640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f784c0c96b0 0x7f784c0a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:35.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7850eb1640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784c0ab500 0x7f784c0a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:35.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f78516b2640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 0x7f784c13d670 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f7848036040 tx=0x7f784802ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:35.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7841ffb640 1 -- 172.21.15.125:0/2666581878 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848005650 con 0x7f784c06af00 2023-10-18T12:00:35.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7841ffb640 1 -- 172.21.15.125:0/2666581878 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78480038f0 con 0x7f784c06af00 2023-10-18T12:00:35.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c1f0680 con 0x7f784c06af00 2023-10-18T12:00:35.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7841ffb640 1 -- 172.21.15.125:0/2666581878 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848003d00 con 0x7f784c06af00 2023-10-18T12:00:35.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.178+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c1f0c40 con 0x7f784c06af00 2023-10-18T12:00:35.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.179+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c0690e0 con 0x7f784c06af00 2023-10-18T12:00:35.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.180+0000 7f7841ffb640 1 -- 172.21.15.125:0/2666581878 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f784803e450 con 0x7f784c06af00 2023-10-18T12:00:35.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.180+0000 7f7841ffb640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f783006af60 0x7f783006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.180+0000 7f7841ffb640 1 -- 172.21.15.125:0/2666581878 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f78480ac720 con 0x7f784c06af00 2023-10-18T12:00:35.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.180+0000 7f7850eb1640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f783006af60 0x7f783006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.181+0000 7f7850eb1640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f783006af60 0x7f783006d420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f783c00ba60 tx=0x7f783c005cf0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:35.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.184+0000 7f7841ffb640 1 -- 172.21.15.125:0/2666581878 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7848077300 con 0x7f784c06af00 2023-10-18T12:00:35.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.346+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 --> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f784c159c50 con 0x7f783006af60 2023-10-18T12:00:35.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.359+0000 7f7841ffb640 1 -- 172.21.15.125:0/2666581878 <== mgr.8786 v2:172.21.15.139:6832/2362843834 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529045 (secure 0 0 0) 0x7f783c00da80 con 0x7f783006af60 2023-10-18T12:00:35.368 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:35.372 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:00:35.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f783006af60 msgr2=0x7f783006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f783006af60 0x7f783006d420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f783c00ba60 tx=0x7f783c005cf0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 msgr2=0x7f784c13d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 0x7f784c13d670 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f7848036040 tx=0x7f784802ddd0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 shutdown_connections 2023-10-18T12:00:35.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f783006af60 0x7f783006d420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784c0ab500 0x7f784c0a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f784c0c96b0 0x7f784c0a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 --2- 172.21.15.125:0/2666581878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f784c06af00 0x7f784c13d670 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 >> 172.21.15.125:0/2666581878 conn(0x7f784c069930 msgr2=0x7f784c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:35.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 shutdown_connections 2023-10-18T12:00:35.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.364+0000 7f7853916640 1 -- 172.21.15.125:0/2666581878 wait complete. 2023-10-18T12:00:35.377 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-10-18T12:00:34.802697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596444,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":686,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":436,"ondisk_log_size":436,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":214,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":414220,"kb_used_data":23176,"kb_used_omap":12,"kb_used_meta":390899,"kb_avail":754560500,"statfs":{"total":773094113280,"available":772669952000,"internally_reserved":0,"allocated":23732224,"data_stored":12708656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":400281144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.395041"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.443713+0000","last_change":"2023-10-18T12:00:32.443239+0000","last_active":"2023-10-18T12:00:32.443713+0000","last_peered":"2023-10-18T12:00:32.443713+0000","last_clean":"2023-10-18T12:00:32.443713+0000","last_became_active":"2023-10-18T12:00:32.442955+0000","last_became_peered":"2023-10-18T12:00:32.442955+0000","last_unstale":"2023-10-18T12:00:32.443713+0000","last_undegraded":"2023-10-18T12:00:32.443713+0000","last_fullsized":"2023-10-18T12:00:32.443713+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:26.856452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"208'357","reported_seq":1236,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.425109+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:00:31.425109+0000","last_peered":"2023-10-18T12:00:31.425109+0000","last_clean":"2023-10-18T12:00:31.425109+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:00:31.425109+0000","last_undegraded":"2023-10-18T12:00:31.425109+0000","last_fullsized":"2023-10-18T12:00:31.425109+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":595,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417915+0000","last_change":"2023-10-18T12:00:20.057601+0000","last_active":"2023-10-18T12:00:31.417915+0000","last_peered":"2023-10-18T12:00:31.417915+0000","last_clean":"2023-10-18T12:00:31.417915+0000","last_became_active":"2023-10-18T12:00:20.057294+0000","last_became_peered":"2023-10-18T12:00:20.057294+0000","last_unstale":"2023-10-18T12:00:31.417915+0000","last_undegraded":"2023-10-18T12:00:31.417915+0000","last_fullsized":"2023-10-18T12:00:31.417915+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:43.803662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418996+0000","last_change":"2023-10-18T12:00:21.091052+0000","last_active":"2023-10-18T12:00:31.418996+0000","last_peered":"2023-10-18T12:00:31.418996+0000","last_clean":"2023-10-18T12:00:31.418996+0000","last_became_active":"2023-10-18T12:00:21.090785+0000","last_became_peered":"2023-10-18T12:00:21.090785+0000","last_unstale":"2023-10-18T12:00:31.418996+0000","last_undegraded":"2023-10-18T12:00:31.418996+0000","last_fullsized":"2023-10-18T12:00:31.418996+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:09.229786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445679+0000","last_change":"2023-10-18T12:00:31.433776+0000","last_active":"2023-10-18T12:00:32.445679+0000","last_peered":"2023-10-18T12:00:32.445679+0000","last_clean":"2023-10-18T12:00:32.445679+0000","last_became_active":"2023-10-18T12:00:31.433509+0000","last_became_peered":"2023-10-18T12:00:31.433509+0000","last_unstale":"2023-10-18T12:00:32.445679+0000","last_undegraded":"2023-10-18T12:00:32.445679+0000","last_fullsized":"2023-10-18T12:00:32.445679+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445783+0000","last_change":"2023-10-18T12:00:32.444780+0000","last_active":"2023-10-18T12:00:32.445783+0000","last_peered":"2023-10-18T12:00:32.445783+0000","last_clean":"2023-10-18T12:00:32.445783+0000","last_became_active":"2023-10-18T12:00:32.443183+0000","last_became_peered":"2023-10-18T12:00:32.443183+0000","last_unstale":"2023-10-18T12:00:32.445783+0000","last_undegraded":"2023-10-18T12:00:32.445783+0000","last_fullsized":"2023-10-18T12:00:32.445783+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:52.458791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426327+0000","last_change":"2023-10-18T12:00:20.066813+0000","last_active":"2023-10-18T12:00:31.426327+0000","last_peered":"2023-10-18T12:00:31.426327+0000","last_clean":"2023-10-18T12:00:31.426327+0000","last_became_active":"2023-10-18T12:00:20.066552+0000","last_became_peered":"2023-10-18T12:00:20.066552+0000","last_unstale":"2023-10-18T12:00:31.426327+0000","last_undegraded":"2023-10-18T12:00:31.426327+0000","last_fullsized":"2023-10-18T12:00:31.426327+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:22:59.208578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.420097+0000","last_change":"2023-10-18T12:00:21.091679+0000","last_active":"2023-10-18T12:00:31.420097+0000","last_peered":"2023-10-18T12:00:31.420097+0000","last_clean":"2023-10-18T12:00:31.420097+0000","last_became_active":"2023-10-18T12:00:21.091412+0000","last_became_peered":"2023-10-18T12:00:21.091412+0000","last_unstale":"2023-10-18T12:00:31.420097+0000","last_undegraded":"2023-10-18T12:00:31.420097+0000","last_fullsized":"2023-10-18T12:00:31.420097+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:27.574111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.421182+0000","last_change":"2023-10-18T12:00:31.425160+0000","last_active":"2023-10-18T12:00:32.421182+0000","last_peered":"2023-10-18T12:00:32.421182+0000","last_clean":"2023-10-18T12:00:32.421182+0000","last_became_active":"2023-10-18T12:00:31.424835+0000","last_became_peered":"2023-10-18T12:00:31.424835+0000","last_unstale":"2023-10-18T12:00:32.421182+0000","last_undegraded":"2023-10-18T12:00:32.421182+0000","last_fullsized":"2023-10-18T12:00:32.421182+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:12.318416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437163+0000","last_change":"2023-10-18T12:00:32.435087+0000","last_active":"2023-10-18T12:00:32.437163+0000","last_peered":"2023-10-18T12:00:32.437163+0000","last_clean":"2023-10-18T12:00:32.437163+0000","last_became_active":"2023-10-18T12:00:32.434844+0000","last_became_peered":"2023-10-18T12:00:32.434844+0000","last_unstale":"2023-10-18T12:00:32.437163+0000","last_undegraded":"2023-10-18T12:00:32.437163+0000","last_fullsized":"2023-10-18T12:00:32.437163+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:28:23.492154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419263+0000","last_change":"2023-10-18T12:00:20.065817+0000","last_active":"2023-10-18T12:00:31.419263+0000","last_peered":"2023-10-18T12:00:31.419263+0000","last_clean":"2023-10-18T12:00:31.419263+0000","last_became_active":"2023-10-18T12:00:20.065536+0000","last_became_peered":"2023-10-18T12:00:20.065536+0000","last_unstale":"2023-10-18T12:00:31.419263+0000","last_undegraded":"2023-10-18T12:00:31.419263+0000","last_fullsized":"2023-10-18T12:00:31.419263+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:57.379803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.423874+0000","last_change":"2023-10-18T12:00:21.089842+0000","last_active":"2023-10-18T12:00:31.423874+0000","last_peered":"2023-10-18T12:00:31.423874+0000","last_clean":"2023-10-18T12:00:31.423874+0000","last_became_active":"2023-10-18T12:00:21.089584+0000","last_became_peered":"2023-10-18T12:00:21.089584+0000","last_unstale":"2023-10-18T12:00:31.423874+0000","last_undegraded":"2023-10-18T12:00:31.423874+0000","last_fullsized":"2023-10-18T12:00:31.423874+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:36.596128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.451711+0000","last_change":"2023-10-18T12:00:31.436826+0000","last_active":"2023-10-18T12:00:32.451711+0000","last_peered":"2023-10-18T12:00:32.451711+0000","last_clean":"2023-10-18T12:00:32.451711+0000","last_became_active":"2023-10-18T12:00:31.436570+0000","last_became_peered":"2023-10-18T12:00:31.436570+0000","last_unstale":"2023-10-18T12:00:32.451711+0000","last_undegraded":"2023-10-18T12:00:32.451711+0000","last_fullsized":"2023-10-18T12:00:32.451711+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:07:55.443591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437028+0000","last_change":"2023-10-18T12:00:32.436481+0000","last_active":"2023-10-18T12:00:32.437028+0000","last_peered":"2023-10-18T12:00:32.437028+0000","last_clean":"2023-10-18T12:00:32.437028+0000","last_became_active":"2023-10-18T12:00:32.436213+0000","last_became_peered":"2023-10-18T12:00:32.436213+0000","last_unstale":"2023-10-18T12:00:32.437028+0000","last_undegraded":"2023-10-18T12:00:32.437028+0000","last_fullsized":"2023-10-18T12:00:32.437028+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:24:20.363215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418008+0000","last_change":"2023-10-18T12:00:20.060656+0000","last_active":"2023-10-18T12:00:31.418008+0000","last_peered":"2023-10-18T12:00:31.418008+0000","last_clean":"2023-10-18T12:00:31.418008+0000","last_became_active":"2023-10-18T12:00:20.060413+0000","last_became_peered":"2023-10-18T12:00:20.060413+0000","last_unstale":"2023-10-18T12:00:31.418008+0000","last_undegraded":"2023-10-18T12:00:31.418008+0000","last_fullsized":"2023-10-18T12:00:31.418008+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:32.716291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424128+0000","last_change":"2023-10-18T12:00:21.088423+0000","last_active":"2023-10-18T12:00:31.424128+0000","last_peered":"2023-10-18T12:00:31.424128+0000","last_clean":"2023-10-18T12:00:31.424128+0000","last_became_active":"2023-10-18T12:00:21.087811+0000","last_became_peered":"2023-10-18T12:00:21.087811+0000","last_unstale":"2023-10-18T12:00:31.424128+0000","last_undegraded":"2023-10-18T12:00:31.424128+0000","last_fullsized":"2023-10-18T12:00:31.424128+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:03.802023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445154+0000","last_change":"2023-10-18T12:00:31.434761+0000","last_active":"2023-10-18T12:00:32.445154+0000","last_peered":"2023-10-18T12:00:32.445154+0000","last_clean":"2023-10-18T12:00:32.445154+0000","last_became_active":"2023-10-18T12:00:31.434482+0000","last_became_peered":"2023-10-18T12:00:31.434482+0000","last_unstale":"2023-10-18T12:00:32.445154+0000","last_undegraded":"2023-10-18T12:00:32.445154+0000","last_fullsized":"2023-10-18T12:00:32.445154+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:49.906829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.452025+0000","last_change":"2023-10-18T12:00:32.451037+0000","last_active":"2023-10-18T12:00:32.452025+0000","last_peered":"2023-10-18T12:00:32.452025+0000","last_clean":"2023-10-18T12:00:32.452025+0000","last_became_active":"2023-10-18T12:00:32.450723+0000","last_became_peered":"2023-10-18T12:00:32.450723+0000","last_unstale":"2023-10-18T12:00:32.452025+0000","last_undegraded":"2023-10-18T12:00:32.452025+0000","last_fullsized":"2023-10-18T12:00:32.452025+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:56.364975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430868+0000","last_change":"2023-10-18T12:00:20.066439+0000","last_active":"2023-10-18T12:00:31.430868+0000","last_peered":"2023-10-18T12:00:31.430868+0000","last_clean":"2023-10-18T12:00:31.430868+0000","last_became_active":"2023-10-18T12:00:20.066204+0000","last_became_peered":"2023-10-18T12:00:20.066204+0000","last_unstale":"2023-10-18T12:00:31.430868+0000","last_undegraded":"2023-10-18T12:00:31.430868+0000","last_fullsized":"2023-10-18T12:00:31.430868+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:36.690391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.431486+0000","last_change":"2023-10-18T12:00:21.090829+0000","last_active":"2023-10-18T12:00:31.431486+0000","last_peered":"2023-10-18T12:00:31.431486+0000","last_clean":"2023-10-18T12:00:31.431486+0000","last_became_active":"2023-10-18T12:00:21.090013+0000","last_became_peered":"2023-10-18T12:00:21.090013+0000","last_unstale":"2023-10-18T12:00:31.431486+0000","last_undegraded":"2023-10-18T12:00:31.431486+0000","last_fullsized":"2023-10-18T12:00:31.431486+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:14.286779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.421622+0000","last_change":"2023-10-18T12:00:31.425068+0000","last_active":"2023-10-18T12:00:32.421622+0000","last_peered":"2023-10-18T12:00:32.421622+0000","last_clean":"2023-10-18T12:00:32.421622+0000","last_became_active":"2023-10-18T12:00:31.424804+0000","last_became_peered":"2023-10-18T12:00:31.424804+0000","last_unstale":"2023-10-18T12:00:32.421622+0000","last_undegraded":"2023-10-18T12:00:32.421622+0000","last_fullsized":"2023-10-18T12:00:32.421622+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:01.434053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430370+0000","last_change":"2023-10-18T12:00:32.428936+0000","last_active":"2023-10-18T12:00:32.430370+0000","last_peered":"2023-10-18T12:00:32.430370+0000","last_clean":"2023-10-18T12:00:32.430370+0000","last_became_active":"2023-10-18T12:00:32.428681+0000","last_became_peered":"2023-10-18T12:00:32.428681+0000","last_unstale":"2023-10-18T12:00:32.430370+0000","last_undegraded":"2023-10-18T12:00:32.430370+0000","last_fullsized":"2023-10-18T12:00:32.430370+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:19.695994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419333+0000","last_change":"2023-10-18T12:00:20.059094+0000","last_active":"2023-10-18T12:00:31.419333+0000","last_peered":"2023-10-18T12:00:31.419333+0000","last_clean":"2023-10-18T12:00:31.419333+0000","last_became_active":"2023-10-18T12:00:20.058785+0000","last_became_peered":"2023-10-18T12:00:20.058785+0000","last_unstale":"2023-10-18T12:00:31.419333+0000","last_undegraded":"2023-10-18T12:00:31.419333+0000","last_fullsized":"2023-10-18T12:00:31.419333+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:32.780678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418904+0000","last_change":"2023-10-18T12:00:21.086339+0000","last_active":"2023-10-18T12:00:31.418904+0000","last_peered":"2023-10-18T12:00:31.418904+0000","last_clean":"2023-10-18T12:00:31.418904+0000","last_became_active":"2023-10-18T12:00:21.086056+0000","last_became_peered":"2023-10-18T12:00:21.086056+0000","last_unstale":"2023-10-18T12:00:31.418904+0000","last_undegraded":"2023-10-18T12:00:31.418904+0000","last_fullsized":"2023-10-18T12:00:31.418904+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:39.387888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.425680+0000","last_change":"2023-10-18T12:00:31.420704+0000","last_active":"2023-10-18T12:00:32.425680+0000","last_peered":"2023-10-18T12:00:32.425680+0000","last_clean":"2023-10-18T12:00:32.425680+0000","last_became_active":"2023-10-18T12:00:31.420253+0000","last_became_peered":"2023-10-18T12:00:31.420253+0000","last_unstale":"2023-10-18T12:00:32.425680+0000","last_undegraded":"2023-10-18T12:00:32.425680+0000","last_fullsized":"2023-10-18T12:00:32.425680+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:38.148897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448971+0000","last_change":"2023-10-18T12:00:32.448482+0000","last_active":"2023-10-18T12:00:32.448971+0000","last_peered":"2023-10-18T12:00:32.448971+0000","last_clean":"2023-10-18T12:00:32.448971+0000","last_became_active":"2023-10-18T12:00:32.448207+0000","last_became_peered":"2023-10-18T12:00:32.448207+0000","last_unstale":"2023-10-18T12:00:32.448971+0000","last_undegraded":"2023-10-18T12:00:32.448971+0000","last_fullsized":"2023-10-18T12:00:32.448971+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:52.700336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418069+0000","last_change":"2023-10-18T12:00:20.059978+0000","last_active":"2023-10-18T12:00:31.418069+0000","last_peered":"2023-10-18T12:00:31.418069+0000","last_clean":"2023-10-18T12:00:31.418069+0000","last_became_active":"2023-10-18T12:00:20.059686+0000","last_became_peered":"2023-10-18T12:00:20.059686+0000","last_unstale":"2023-10-18T12:00:31.418069+0000","last_undegraded":"2023-10-18T12:00:31.418069+0000","last_fullsized":"2023-10-18T12:00:31.418069+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:53.027447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417735+0000","last_change":"2023-10-18T12:00:21.076847+0000","last_active":"2023-10-18T12:00:31.417735+0000","last_peered":"2023-10-18T12:00:31.417735+0000","last_clean":"2023-10-18T12:00:31.417735+0000","last_became_active":"2023-10-18T12:00:21.076552+0000","last_became_peered":"2023-10-18T12:00:21.076552+0000","last_unstale":"2023-10-18T12:00:31.417735+0000","last_undegraded":"2023-10-18T12:00:31.417735+0000","last_fullsized":"2023-10-18T12:00:31.417735+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:36.596128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.441498+0000","last_change":"2023-10-18T12:00:31.435045+0000","last_active":"2023-10-18T12:00:32.441498+0000","last_peered":"2023-10-18T12:00:32.441498+0000","last_clean":"2023-10-18T12:00:32.441498+0000","last_became_active":"2023-10-18T12:00:31.434797+0000","last_became_peered":"2023-10-18T12:00:31.434797+0000","last_unstale":"2023-10-18T12:00:32.441498+0000","last_undegraded":"2023-10-18T12:00:32.441498+0000","last_fullsized":"2023-10-18T12:00:32.441498+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:37.618423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.449206+0000","last_change":"2023-10-18T12:00:32.446096+0000","last_active":"2023-10-18T12:00:32.449206+0000","last_peered":"2023-10-18T12:00:32.449206+0000","last_clean":"2023-10-18T12:00:32.449206+0000","last_became_active":"2023-10-18T12:00:32.445755+0000","last_became_peered":"2023-10-18T12:00:32.445755+0000","last_unstale":"2023-10-18T12:00:32.449206+0000","last_undegraded":"2023-10-18T12:00:32.449206+0000","last_fullsized":"2023-10-18T12:00:32.449206+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:25.786603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"221'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.432460+0000","last_change":"2023-10-18T12:00:20.063289+0000","last_active":"2023-10-18T12:00:31.432460+0000","last_peered":"2023-10-18T12:00:31.432460+0000","last_clean":"2023-10-18T12:00:31.432460+0000","last_became_active":"2023-10-18T12:00:20.063024+0000","last_became_peered":"2023-10-18T12:00:20.063024+0000","last_unstale":"2023-10-18T12:00:31.432460+0000","last_undegraded":"2023-10-18T12:00:31.432460+0000","last_fullsized":"2023-10-18T12:00:31.432460+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:54.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418948+0000","last_change":"2023-10-18T12:00:21.092403+0000","last_active":"2023-10-18T12:00:31.418948+0000","last_peered":"2023-10-18T12:00:31.418948+0000","last_clean":"2023-10-18T12:00:31.418948+0000","last_became_active":"2023-10-18T12:00:21.092167+0000","last_became_peered":"2023-10-18T12:00:21.092167+0000","last_unstale":"2023-10-18T12:00:31.418948+0000","last_undegraded":"2023-10-18T12:00:31.418948+0000","last_fullsized":"2023-10-18T12:00:31.418948+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:45:11.154204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.438116+0000","last_change":"2023-10-18T12:00:31.438173+0000","last_active":"2023-10-18T12:00:32.438116+0000","last_peered":"2023-10-18T12:00:32.438116+0000","last_clean":"2023-10-18T12:00:32.438116+0000","last_became_active":"2023-10-18T12:00:31.437914+0000","last_became_peered":"2023-10-18T12:00:31.437914+0000","last_unstale":"2023-10-18T12:00:32.438116+0000","last_undegraded":"2023-10-18T12:00:32.438116+0000","last_fullsized":"2023-10-18T12:00:32.438116+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:51:43.375691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429327+0000","last_change":"2023-10-18T12:00:32.427052+0000","last_active":"2023-10-18T12:00:32.429327+0000","last_peered":"2023-10-18T12:00:32.429327+0000","last_clean":"2023-10-18T12:00:32.429327+0000","last_became_active":"2023-10-18T12:00:32.426788+0000","last_became_peered":"2023-10-18T12:00:32.426788+0000","last_unstale":"2023-10-18T12:00:32.429327+0000","last_undegraded":"2023-10-18T12:00:32.429327+0000","last_fullsized":"2023-10-18T12:00:32.429327+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:14.155039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418442+0000","last_change":"2023-10-18T12:00:20.063535+0000","last_active":"2023-10-18T12:00:31.418442+0000","last_peered":"2023-10-18T12:00:31.418442+0000","last_clean":"2023-10-18T12:00:31.418442+0000","last_became_active":"2023-10-18T12:00:20.063259+0000","last_became_peered":"2023-10-18T12:00:20.063259+0000","last_unstale":"2023-10-18T12:00:31.418442+0000","last_undegraded":"2023-10-18T12:00:31.418442+0000","last_fullsized":"2023-10-18T12:00:31.418442+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:54:11.686164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.429871+0000","last_change":"2023-10-18T12:00:21.092122+0000","last_active":"2023-10-18T12:00:31.429871+0000","last_peered":"2023-10-18T12:00:31.429871+0000","last_clean":"2023-10-18T12:00:31.429871+0000","last_became_active":"2023-10-18T12:00:21.091845+0000","last_became_peered":"2023-10-18T12:00:21.091845+0000","last_unstale":"2023-10-18T12:00:31.429871+0000","last_undegraded":"2023-10-18T12:00:31.429871+0000","last_fullsized":"2023-10-18T12:00:31.429871+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:31.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.428352+0000","last_change":"2023-10-18T12:00:31.430651+0000","last_active":"2023-10-18T12:00:32.428352+0000","last_peered":"2023-10-18T12:00:32.428352+0000","last_clean":"2023-10-18T12:00:32.428352+0000","last_became_active":"2023-10-18T12:00:31.430249+0000","last_became_peered":"2023-10-18T12:00:31.430249+0000","last_unstale":"2023-10-18T12:00:32.428352+0000","last_undegraded":"2023-10-18T12:00:32.428352+0000","last_fullsized":"2023-10-18T12:00:32.428352+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:47:24.776087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445801+0000","last_change":"2023-10-18T12:00:32.444989+0000","last_active":"2023-10-18T12:00:32.445801+0000","last_peered":"2023-10-18T12:00:32.445801+0000","last_clean":"2023-10-18T12:00:32.445801+0000","last_became_active":"2023-10-18T12:00:32.443013+0000","last_became_peered":"2023-10-18T12:00:32.443013+0000","last_unstale":"2023-10-18T12:00:32.445801+0000","last_undegraded":"2023-10-18T12:00:32.445801+0000","last_fullsized":"2023-10-18T12:00:32.445801+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:04.205911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418744+0000","last_change":"2023-10-18T12:00:20.058023+0000","last_active":"2023-10-18T12:00:31.418744+0000","last_peered":"2023-10-18T12:00:31.418744+0000","last_clean":"2023-10-18T12:00:31.418744+0000","last_became_active":"2023-10-18T12:00:20.057731+0000","last_became_peered":"2023-10-18T12:00:20.057731+0000","last_unstale":"2023-10-18T12:00:31.418744+0000","last_undegraded":"2023-10-18T12:00:31.418744+0000","last_fullsized":"2023-10-18T12:00:31.418744+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:28.253477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.433622+0000","last_change":"2023-10-18T12:00:21.094111+0000","last_active":"2023-10-18T12:00:31.433622+0000","last_peered":"2023-10-18T12:00:31.433622+0000","last_clean":"2023-10-18T12:00:31.433622+0000","last_became_active":"2023-10-18T12:00:21.093838+0000","last_became_peered":"2023-10-18T12:00:21.093838+0000","last_unstale":"2023-10-18T12:00:31.433622+0000","last_undegraded":"2023-10-18T12:00:31.433622+0000","last_fullsized":"2023-10-18T12:00:31.433622+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:57:22.028468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437844+0000","last_change":"2023-10-18T12:00:31.434488+0000","last_active":"2023-10-18T12:00:32.437844+0000","last_peered":"2023-10-18T12:00:32.437844+0000","last_clean":"2023-10-18T12:00:32.437844+0000","last_became_active":"2023-10-18T12:00:31.434270+0000","last_became_peered":"2023-10-18T12:00:31.434270+0000","last_unstale":"2023-10-18T12:00:32.437844+0000","last_undegraded":"2023-10-18T12:00:32.437844+0000","last_fullsized":"2023-10-18T12:00:32.437844+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:31.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.447246+0000","last_change":"2023-10-18T12:00:32.446941+0000","last_active":"2023-10-18T12:00:32.447246+0000","last_peered":"2023-10-18T12:00:32.447246+0000","last_clean":"2023-10-18T12:00:32.447246+0000","last_became_active":"2023-10-18T12:00:32.446705+0000","last_became_peered":"2023-10-18T12:00:32.446705+0000","last_unstale":"2023-10-18T12:00:32.447246+0000","last_undegraded":"2023-10-18T12:00:32.447246+0000","last_fullsized":"2023-10-18T12:00:32.447246+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:24.527236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418177+0000","last_change":"2023-10-18T12:00:20.063488+0000","last_active":"2023-10-18T12:00:31.418177+0000","last_peered":"2023-10-18T12:00:31.418177+0000","last_clean":"2023-10-18T12:00:31.418177+0000","last_became_active":"2023-10-18T12:00:20.063220+0000","last_became_peered":"2023-10-18T12:00:20.063220+0000","last_unstale":"2023-10-18T12:00:31.418177+0000","last_undegraded":"2023-10-18T12:00:31.418177+0000","last_fullsized":"2023-10-18T12:00:31.418177+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:32.615922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418181+0000","last_change":"2023-10-18T12:00:21.087602+0000","last_active":"2023-10-18T12:00:31.418181+0000","last_peered":"2023-10-18T12:00:31.418181+0000","last_clean":"2023-10-18T12:00:31.418181+0000","last_became_active":"2023-10-18T12:00:21.087140+0000","last_became_peered":"2023-10-18T12:00:21.087140+0000","last_unstale":"2023-10-18T12:00:31.418181+0000","last_undegraded":"2023-10-18T12:00:31.418181+0000","last_fullsized":"2023-10-18T12:00:31.418181+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:57.852277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.449467+0000","last_change":"2023-10-18T12:00:31.436777+0000","last_active":"2023-10-18T12:00:32.449467+0000","last_peered":"2023-10-18T12:00:32.449467+0000","last_clean":"2023-10-18T12:00:32.449467+0000","last_became_active":"2023-10-18T12:00:31.436495+0000","last_became_peered":"2023-10-18T12:00:31.436495+0000","last_unstale":"2023-10-18T12:00:32.449467+0000","last_undegraded":"2023-10-18T12:00:32.449467+0000","last_fullsized":"2023-10-18T12:00:32.449467+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:17:38.007561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448900+0000","last_change":"2023-10-18T12:00:32.447082+0000","last_active":"2023-10-18T12:00:32.448900+0000","last_peered":"2023-10-18T12:00:32.448900+0000","last_clean":"2023-10-18T12:00:32.448900+0000","last_became_active":"2023-10-18T12:00:32.442502+0000","last_became_peered":"2023-10-18T12:00:32.442502+0000","last_unstale":"2023-10-18T12:00:32.448900+0000","last_undegraded":"2023-10-18T12:00:32.448900+0000","last_fullsized":"2023-10-18T12:00:32.448900+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:15.903077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421491+0000","last_change":"2023-10-18T12:00:20.062486+0000","last_active":"2023-10-18T12:00:31.421491+0000","last_peered":"2023-10-18T12:00:31.421491+0000","last_clean":"2023-10-18T12:00:31.421491+0000","last_became_active":"2023-10-18T12:00:20.062205+0000","last_became_peered":"2023-10-18T12:00:20.062205+0000","last_unstale":"2023-10-18T12:00:31.421491+0000","last_undegraded":"2023-10-18T12:00:31.421491+0000","last_fullsized":"2023-10-18T12:00:31.421491+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:01.124233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424118+0000","last_change":"2023-10-18T12:00:21.092605+0000","last_active":"2023-10-18T12:00:31.424118+0000","last_peered":"2023-10-18T12:00:31.424118+0000","last_clean":"2023-10-18T12:00:31.424118+0000","last_became_active":"2023-10-18T12:00:21.092319+0000","last_became_peered":"2023-10-18T12:00:21.092319+0000","last_unstale":"2023-10-18T12:00:31.424118+0000","last_undegraded":"2023-10-18T12:00:31.424118+0000","last_fullsized":"2023-10-18T12:00:31.424118+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:44.809362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445018+0000","last_change":"2023-10-18T12:00:31.436132+0000","last_active":"2023-10-18T12:00:32.445018+0000","last_peered":"2023-10-18T12:00:32.445018+0000","last_clean":"2023-10-18T12:00:32.445018+0000","last_became_active":"2023-10-18T12:00:31.435398+0000","last_became_peered":"2023-10-18T12:00:31.435398+0000","last_unstale":"2023-10-18T12:00:32.445018+0000","last_undegraded":"2023-10-18T12:00:32.445018+0000","last_fullsized":"2023-10-18T12:00:32.445018+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:46:39.602864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.428503+0000","last_change":"2023-10-18T12:00:32.425636+0000","last_active":"2023-10-18T12:00:32.428503+0000","last_peered":"2023-10-18T12:00:32.428503+0000","last_clean":"2023-10-18T12:00:32.428503+0000","last_became_active":"2023-10-18T12:00:32.425355+0000","last_became_peered":"2023-10-18T12:00:32.425355+0000","last_unstale":"2023-10-18T12:00:32.428503+0000","last_undegraded":"2023-10-18T12:00:32.428503+0000","last_fullsized":"2023-10-18T12:00:32.428503+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:43:38.016035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427469+0000","last_change":"2023-10-18T12:00:20.058937+0000","last_active":"2023-10-18T12:00:31.427469+0000","last_peered":"2023-10-18T12:00:31.427469+0000","last_clean":"2023-10-18T12:00:31.427469+0000","last_became_active":"2023-10-18T12:00:20.058685+0000","last_became_peered":"2023-10-18T12:00:20.058685+0000","last_unstale":"2023-10-18T12:00:31.427469+0000","last_undegraded":"2023-10-18T12:00:31.427469+0000","last_fullsized":"2023-10-18T12:00:31.427469+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:07.432693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421695+0000","last_change":"2023-10-18T12:00:21.089358+0000","last_active":"2023-10-18T12:00:31.421695+0000","last_peered":"2023-10-18T12:00:31.421695+0000","last_clean":"2023-10-18T12:00:31.421695+0000","last_became_active":"2023-10-18T12:00:21.089109+0000","last_became_peered":"2023-10-18T12:00:21.089109+0000","last_unstale":"2023-10-18T12:00:31.421695+0000","last_undegraded":"2023-10-18T12:00:31.421695+0000","last_fullsized":"2023-10-18T12:00:31.421695+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:47:14.419423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454642+0000","last_change":"2023-10-18T12:00:31.431647+0000","last_active":"2023-10-18T12:00:32.454642+0000","last_peered":"2023-10-18T12:00:32.454642+0000","last_clean":"2023-10-18T12:00:32.454642+0000","last_became_active":"2023-10-18T12:00:31.431288+0000","last_became_peered":"2023-10-18T12:00:31.431288+0000","last_unstale":"2023-10-18T12:00:32.454642+0000","last_undegraded":"2023-10-18T12:00:32.454642+0000","last_fullsized":"2023-10-18T12:00:32.454642+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:56.618941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445494+0000","last_change":"2023-10-18T12:00:32.443504+0000","last_active":"2023-10-18T12:00:32.445494+0000","last_peered":"2023-10-18T12:00:32.445494+0000","last_clean":"2023-10-18T12:00:32.445494+0000","last_became_active":"2023-10-18T12:00:32.443191+0000","last_became_peered":"2023-10-18T12:00:32.443191+0000","last_unstale":"2023-10-18T12:00:32.445494+0000","last_undegraded":"2023-10-18T12:00:32.445494+0000","last_fullsized":"2023-10-18T12:00:32.445494+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:23.559656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427089+0000","last_change":"2023-10-18T12:00:20.063355+0000","last_active":"2023-10-18T12:00:31.427089+0000","last_peered":"2023-10-18T12:00:31.427089+0000","last_clean":"2023-10-18T12:00:31.427089+0000","last_became_active":"2023-10-18T12:00:20.063097+0000","last_became_peered":"2023-10-18T12:00:20.063097+0000","last_unstale":"2023-10-18T12:00:31.427089+0000","last_undegraded":"2023-10-18T12:00:31.427089+0000","last_fullsized":"2023-10-18T12:00:31.427089+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:13.852394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430839+0000","last_change":"2023-10-18T12:00:21.090430+0000","last_active":"2023-10-18T12:00:31.430839+0000","last_peered":"2023-10-18T12:00:31.430839+0000","last_clean":"2023-10-18T12:00:31.430839+0000","last_became_active":"2023-10-18T12:00:21.090144+0000","last_became_peered":"2023-10-18T12:00:21.090144+0000","last_unstale":"2023-10-18T12:00:31.430839+0000","last_undegraded":"2023-10-18T12:00:31.430839+0000","last_fullsized":"2023-10-18T12:00:31.430839+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:32:47.987741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.422216+0000","last_change":"2023-10-18T12:00:31.425713+0000","last_active":"2023-10-18T12:00:32.422216+0000","last_peered":"2023-10-18T12:00:32.422216+0000","last_clean":"2023-10-18T12:00:32.422216+0000","last_became_active":"2023-10-18T12:00:31.425465+0000","last_became_peered":"2023-10-18T12:00:31.425465+0000","last_unstale":"2023-10-18T12:00:32.422216+0000","last_undegraded":"2023-10-18T12:00:32.422216+0000","last_fullsized":"2023-10-18T12:00:32.422216+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:34.143233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.453304+0000","last_change":"2023-10-18T12:00:32.448429+0000","last_active":"2023-10-18T12:00:32.453304+0000","last_peered":"2023-10-18T12:00:32.453304+0000","last_clean":"2023-10-18T12:00:32.453304+0000","last_became_active":"2023-10-18T12:00:32.448128+0000","last_became_peered":"2023-10-18T12:00:32.448128+0000","last_unstale":"2023-10-18T12:00:32.453304+0000","last_undegraded":"2023-10-18T12:00:32.453304+0000","last_fullsized":"2023-10-18T12:00:32.453304+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:17.304006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421809+0000","last_change":"2023-10-18T12:00:20.067526+0000","last_active":"2023-10-18T12:00:31.421809+0000","last_peered":"2023-10-18T12:00:31.421809+0000","last_clean":"2023-10-18T12:00:31.421809+0000","last_became_active":"2023-10-18T12:00:20.067093+0000","last_became_peered":"2023-10-18T12:00:20.067093+0000","last_unstale":"2023-10-18T12:00:31.421809+0000","last_undegraded":"2023-10-18T12:00:31.421809+0000","last_fullsized":"2023-10-18T12:00:31.421809+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:08.654298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419376+0000","last_change":"2023-10-18T12:00:21.074593+0000","last_active":"2023-10-18T12:00:31.419376+0000","last_peered":"2023-10-18T12:00:31.419376+0000","last_clean":"2023-10-18T12:00:31.419376+0000","last_became_active":"2023-10-18T12:00:21.074328+0000","last_became_peered":"2023-10-18T12:00:21.074328+0000","last_unstale":"2023-10-18T12:00:31.419376+0000","last_undegraded":"2023-10-18T12:00:31.419376+0000","last_fullsized":"2023-10-18T12:00:31.419376+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:30:55.127165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454403+0000","last_change":"2023-10-18T12:00:31.432624+0000","last_active":"2023-10-18T12:00:32.454403+0000","last_peered":"2023-10-18T12:00:32.454403+0000","last_clean":"2023-10-18T12:00:32.454403+0000","last_became_active":"2023-10-18T12:00:31.432368+0000","last_became_peered":"2023-10-18T12:00:31.432368+0000","last_unstale":"2023-10-18T12:00:32.454403+0000","last_undegraded":"2023-10-18T12:00:32.454403+0000","last_fullsized":"2023-10-18T12:00:32.454403+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:47.695385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.449003+0000","last_change":"2023-10-18T12:00:32.448513+0000","last_active":"2023-10-18T12:00:32.449003+0000","last_peered":"2023-10-18T12:00:32.449003+0000","last_clean":"2023-10-18T12:00:32.449003+0000","last_became_active":"2023-10-18T12:00:32.448229+0000","last_became_peered":"2023-10-18T12:00:32.448229+0000","last_unstale":"2023-10-18T12:00:32.449003+0000","last_undegraded":"2023-10-18T12:00:32.449003+0000","last_fullsized":"2023-10-18T12:00:32.449003+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.778814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419240+0000","last_change":"2023-10-18T12:00:20.067607+0000","last_active":"2023-10-18T12:00:31.419240+0000","last_peered":"2023-10-18T12:00:31.419240+0000","last_clean":"2023-10-18T12:00:31.419240+0000","last_became_active":"2023-10-18T12:00:20.067298+0000","last_became_peered":"2023-10-18T12:00:20.067298+0000","last_unstale":"2023-10-18T12:00:31.419240+0000","last_undegraded":"2023-10-18T12:00:31.419240+0000","last_fullsized":"2023-10-18T12:00:31.419240+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.982605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419253+0000","last_change":"2023-10-18T12:00:21.090572+0000","last_active":"2023-10-18T12:00:31.419253+0000","last_peered":"2023-10-18T12:00:31.419253+0000","last_clean":"2023-10-18T12:00:31.419253+0000","last_became_active":"2023-10-18T12:00:21.090312+0000","last_became_peered":"2023-10-18T12:00:21.090312+0000","last_unstale":"2023-10-18T12:00:31.419253+0000","last_undegraded":"2023-10-18T12:00:31.419253+0000","last_fullsized":"2023-10-18T12:00:31.419253+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:00:29.845435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.420964+0000","last_change":"2023-10-18T12:00:31.434473+0000","last_active":"2023-10-18T12:00:32.420964+0000","last_peered":"2023-10-18T12:00:32.420964+0000","last_clean":"2023-10-18T12:00:32.420964+0000","last_became_active":"2023-10-18T12:00:31.434226+0000","last_became_peered":"2023-10-18T12:00:31.434226+0000","last_unstale":"2023-10-18T12:00:32.420964+0000","last_undegraded":"2023-10-18T12:00:32.420964+0000","last_fullsized":"2023-10-18T12:00:32.420964+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:43.981297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.439919+0000","last_change":"2023-10-18T12:00:32.439361+0000","last_active":"2023-10-18T12:00:32.439919+0000","last_peered":"2023-10-18T12:00:32.439919+0000","last_clean":"2023-10-18T12:00:32.439919+0000","last_became_active":"2023-10-18T12:00:32.439094+0000","last_became_peered":"2023-10-18T12:00:32.439094+0000","last_unstale":"2023-10-18T12:00:32.439919+0000","last_undegraded":"2023-10-18T12:00:32.439919+0000","last_fullsized":"2023-10-18T12:00:32.439919+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:52.458791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419689+0000","last_change":"2023-10-18T12:00:20.058912+0000","last_active":"2023-10-18T12:00:31.419689+0000","last_peered":"2023-10-18T12:00:31.419689+0000","last_clean":"2023-10-18T12:00:31.419689+0000","last_became_active":"2023-10-18T12:00:20.058634+0000","last_became_peered":"2023-10-18T12:00:20.058634+0000","last_unstale":"2023-10-18T12:00:31.419689+0000","last_undegraded":"2023-10-18T12:00:31.419689+0000","last_fullsized":"2023-10-18T12:00:31.419689+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:06.254577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.429785+0000","last_change":"2023-10-18T12:00:21.092578+0000","last_active":"2023-10-18T12:00:31.429785+0000","last_peered":"2023-10-18T12:00:31.429785+0000","last_clean":"2023-10-18T12:00:31.429785+0000","last_became_active":"2023-10-18T12:00:21.092290+0000","last_became_peered":"2023-10-18T12:00:21.092290+0000","last_unstale":"2023-10-18T12:00:31.429785+0000","last_undegraded":"2023-10-18T12:00:31.429785+0000","last_fullsized":"2023-10-18T12:00:31.429785+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:05.871282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.422216+0000","last_change":"2023-10-18T12:00:31.421864+0000","last_active":"2023-10-18T12:00:32.422216+0000","last_peered":"2023-10-18T12:00:32.422216+0000","last_clean":"2023-10-18T12:00:32.422216+0000","last_became_active":"2023-10-18T12:00:31.421615+0000","last_became_peered":"2023-10-18T12:00:31.421615+0000","last_unstale":"2023-10-18T12:00:32.422216+0000","last_undegraded":"2023-10-18T12:00:32.422216+0000","last_fullsized":"2023-10-18T12:00:32.422216+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:50.619813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.427230+0000","last_change":"2023-10-18T12:00:32.424481+0000","last_active":"2023-10-18T12:00:32.427230+0000","last_peered":"2023-10-18T12:00:32.427230+0000","last_clean":"2023-10-18T12:00:32.427230+0000","last_became_active":"2023-10-18T12:00:32.424225+0000","last_became_peered":"2023-10-18T12:00:32.424225+0000","last_unstale":"2023-10-18T12:00:32.427230+0000","last_undegraded":"2023-10-18T12:00:32.427230+0000","last_fullsized":"2023-10-18T12:00:32.427230+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:02.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"220'8","reported_seq":23,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426242+0000","last_change":"2023-10-18T12:00:20.066867+0000","last_active":"2023-10-18T12:00:31.426242+0000","last_peered":"2023-10-18T12:00:31.426242+0000","last_clean":"2023-10-18T12:00:31.426242+0000","last_became_active":"2023-10-18T12:00:20.066615+0000","last_became_peered":"2023-10-18T12:00:20.066615+0000","last_unstale":"2023-10-18T12:00:31.426242+0000","last_undegraded":"2023-10-18T12:00:31.426242+0000","last_fullsized":"2023-10-18T12:00:31.426242+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:31.444571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424209+0000","last_change":"2023-10-18T12:00:21.086868+0000","last_active":"2023-10-18T12:00:31.424209+0000","last_peered":"2023-10-18T12:00:31.424209+0000","last_clean":"2023-10-18T12:00:31.424209+0000","last_became_active":"2023-10-18T12:00:21.086595+0000","last_became_peered":"2023-10-18T12:00:21.086595+0000","last_unstale":"2023-10-18T12:00:31.424209+0000","last_undegraded":"2023-10-18T12:00:31.424209+0000","last_fullsized":"2023-10-18T12:00:31.424209+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:33.721991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.426079+0000","last_change":"2023-10-18T12:00:31.435014+0000","last_active":"2023-10-18T12:00:32.426079+0000","last_peered":"2023-10-18T12:00:32.426079+0000","last_clean":"2023-10-18T12:00:32.426079+0000","last_became_active":"2023-10-18T12:00:31.434304+0000","last_became_peered":"2023-10-18T12:00:31.434304+0000","last_unstale":"2023-10-18T12:00:32.426079+0000","last_undegraded":"2023-10-18T12:00:32.426079+0000","last_fullsized":"2023-10-18T12:00:32.426079+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:57:32.385132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429662+0000","last_change":"2023-10-18T12:00:32.428970+0000","last_active":"2023-10-18T12:00:32.429662+0000","last_peered":"2023-10-18T12:00:32.429662+0000","last_clean":"2023-10-18T12:00:32.429662+0000","last_became_active":"2023-10-18T12:00:32.428727+0000","last_became_peered":"2023-10-18T12:00:32.428727+0000","last_unstale":"2023-10-18T12:00:32.429662+0000","last_undegraded":"2023-10-18T12:00:32.429662+0000","last_fullsized":"2023-10-18T12:00:32.429662+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:07.615526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417258+0000","last_change":"2023-10-18T12:00:20.061573+0000","last_active":"2023-10-18T12:00:31.417258+0000","last_peered":"2023-10-18T12:00:31.417258+0000","last_clean":"2023-10-18T12:00:31.417258+0000","last_became_active":"2023-10-18T12:00:20.061363+0000","last_became_peered":"2023-10-18T12:00:20.061363+0000","last_unstale":"2023-10-18T12:00:31.417258+0000","last_undegraded":"2023-10-18T12:00:31.417258+0000","last_fullsized":"2023-10-18T12:00:31.417258+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:33.588701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418261+0000","last_change":"2023-10-18T12:00:21.081617+0000","last_active":"2023-10-18T12:00:31.418261+0000","last_peered":"2023-10-18T12:00:31.418261+0000","last_clean":"2023-10-18T12:00:31.418261+0000","last_became_active":"2023-10-18T12:00:21.081232+0000","last_became_peered":"2023-10-18T12:00:21.081232+0000","last_unstale":"2023-10-18T12:00:31.418261+0000","last_undegraded":"2023-10-18T12:00:31.418261+0000","last_fullsized":"2023-10-18T12:00:31.418261+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:59:30.932435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.433373+0000","last_change":"2023-10-18T12:00:31.428387+0000","last_active":"2023-10-18T12:00:32.433373+0000","last_peered":"2023-10-18T12:00:32.433373+0000","last_clean":"2023-10-18T12:00:32.433373+0000","last_became_active":"2023-10-18T12:00:31.428118+0000","last_became_peered":"2023-10-18T12:00:31.428118+0000","last_unstale":"2023-10-18T12:00:32.433373+0000","last_undegraded":"2023-10-18T12:00:32.433373+0000","last_fullsized":"2023-10-18T12:00:32.433373+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:42.705320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.433264+0000","last_change":"2023-10-18T12:00:32.432291+0000","last_active":"2023-10-18T12:00:32.433264+0000","last_peered":"2023-10-18T12:00:32.433264+0000","last_clean":"2023-10-18T12:00:32.433264+0000","last_became_active":"2023-10-18T12:00:32.432021+0000","last_became_peered":"2023-10-18T12:00:32.432021+0000","last_unstale":"2023-10-18T12:00:32.433264+0000","last_undegraded":"2023-10-18T12:00:32.433264+0000","last_fullsized":"2023-10-18T12:00:32.433264+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:38.552359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419133+0000","last_change":"2023-10-18T12:00:20.059890+0000","last_active":"2023-10-18T12:00:31.419133+0000","last_peered":"2023-10-18T12:00:31.419133+0000","last_clean":"2023-10-18T12:00:31.419133+0000","last_became_active":"2023-10-18T12:00:20.059647+0000","last_became_peered":"2023-10-18T12:00:20.059647+0000","last_unstale":"2023-10-18T12:00:31.419133+0000","last_undegraded":"2023-10-18T12:00:31.419133+0000","last_fullsized":"2023-10-18T12:00:31.419133+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:08.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419681+0000","last_change":"2023-10-18T12:00:21.075390+0000","last_active":"2023-10-18T12:00:31.419681+0000","last_peered":"2023-10-18T12:00:31.419681+0000","last_clean":"2023-10-18T12:00:31.419681+0000","last_became_active":"2023-10-18T12:00:21.075149+0000","last_became_peered":"2023-10-18T12:00:21.075149+0000","last_unstale":"2023-10-18T12:00:31.419681+0000","last_undegraded":"2023-10-18T12:00:31.419681+0000","last_fullsized":"2023-10-18T12:00:31.419681+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:00.806334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"224'6","reported_seq":17,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.456170+0000","last_change":"2023-10-18T12:00:31.434654+0000","last_active":"2023-10-18T12:00:32.456170+0000","last_peered":"2023-10-18T12:00:32.456170+0000","last_clean":"2023-10-18T12:00:32.456170+0000","last_became_active":"2023-10-18T12:00:31.434279+0000","last_became_peered":"2023-10-18T12:00:31.434279+0000","last_unstale":"2023-10-18T12:00:32.456170+0000","last_undegraded":"2023-10-18T12:00:32.456170+0000","last_fullsized":"2023-10-18T12:00:32.456170+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:41.911245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429689+0000","last_change":"2023-10-18T12:00:32.427768+0000","last_active":"2023-10-18T12:00:32.429689+0000","last_peered":"2023-10-18T12:00:32.429689+0000","last_clean":"2023-10-18T12:00:32.429689+0000","last_became_active":"2023-10-18T12:00:32.427530+0000","last_became_peered":"2023-10-18T12:00:32.427530+0000","last_unstale":"2023-10-18T12:00:32.429689+0000","last_undegraded":"2023-10-18T12:00:32.429689+0000","last_fullsized":"2023-10-18T12:00:32.429689+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:42.316328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421091+0000","last_change":"2023-10-18T12:00:20.063126+0000","last_active":"2023-10-18T12:00:31.421091+0000","last_peered":"2023-10-18T12:00:31.421091+0000","last_clean":"2023-10-18T12:00:31.421091+0000","last_became_active":"2023-10-18T12:00:20.062833+0000","last_became_peered":"2023-10-18T12:00:20.062833+0000","last_unstale":"2023-10-18T12:00:31.421091+0000","last_undegraded":"2023-10-18T12:00:31.421091+0000","last_fullsized":"2023-10-18T12:00:31.421091+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:57.198338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421619+0000","last_change":"2023-10-18T12:00:21.078685+0000","last_active":"2023-10-18T12:00:31.421619+0000","last_peered":"2023-10-18T12:00:31.421619+0000","last_clean":"2023-10-18T12:00:31.421619+0000","last_became_active":"2023-10-18T12:00:21.078434+0000","last_became_peered":"2023-10-18T12:00:21.078434+0000","last_unstale":"2023-10-18T12:00:31.421619+0000","last_undegraded":"2023-10-18T12:00:31.421619+0000","last_fullsized":"2023-10-18T12:00:31.421619+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:12.197055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"224'3","reported_seq":14,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.447244+0000","last_change":"2023-10-18T12:00:31.429037+0000","last_active":"2023-10-18T12:00:32.447244+0000","last_peered":"2023-10-18T12:00:32.447244+0000","last_clean":"2023-10-18T12:00:32.447244+0000","last_became_active":"2023-10-18T12:00:31.428766+0000","last_became_peered":"2023-10-18T12:00:31.428766+0000","last_unstale":"2023-10-18T12:00:32.447244+0000","last_undegraded":"2023-10-18T12:00:32.447244+0000","last_fullsized":"2023-10-18T12:00:32.447244+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:13.312973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454633+0000","last_change":"2023-10-18T12:00:32.453960+0000","last_active":"2023-10-18T12:00:32.454633+0000","last_peered":"2023-10-18T12:00:32.454633+0000","last_clean":"2023-10-18T12:00:32.454633+0000","last_became_active":"2023-10-18T12:00:32.453655+0000","last_became_peered":"2023-10-18T12:00:32.453655+0000","last_unstale":"2023-10-18T12:00:32.454633+0000","last_undegraded":"2023-10-18T12:00:32.454633+0000","last_fullsized":"2023-10-18T12:00:32.454633+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:31.476751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"221'8","reported_seq":23,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.425513+0000","last_change":"2023-10-18T12:00:20.070213+0000","last_active":"2023-10-18T12:00:31.425513+0000","last_peered":"2023-10-18T12:00:31.425513+0000","last_clean":"2023-10-18T12:00:31.425513+0000","last_became_active":"2023-10-18T12:00:20.069888+0000","last_became_peered":"2023-10-18T12:00:20.069888+0000","last_unstale":"2023-10-18T12:00:31.425513+0000","last_undegraded":"2023-10-18T12:00:31.425513+0000","last_fullsized":"2023-10-18T12:00:31.425513+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:55.247225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427378+0000","last_change":"2023-10-18T12:00:21.092364+0000","last_active":"2023-10-18T12:00:31.427378+0000","last_peered":"2023-10-18T12:00:31.427378+0000","last_clean":"2023-10-18T12:00:31.427378+0000","last_became_active":"2023-10-18T12:00:21.092090+0000","last_became_peered":"2023-10-18T12:00:21.092090+0000","last_unstale":"2023-10-18T12:00:31.427378+0000","last_undegraded":"2023-10-18T12:00:31.427378+0000","last_fullsized":"2023-10-18T12:00:31.427378+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:19.318183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.427693+0000","last_change":"2023-10-18T12:00:31.430567+0000","last_active":"2023-10-18T12:00:32.427693+0000","last_peered":"2023-10-18T12:00:32.427693+0000","last_clean":"2023-10-18T12:00:32.427693+0000","last_became_active":"2023-10-18T12:00:31.430271+0000","last_became_peered":"2023-10-18T12:00:31.430271+0000","last_unstale":"2023-10-18T12:00:32.427693+0000","last_undegraded":"2023-10-18T12:00:32.427693+0000","last_fullsized":"2023-10-18T12:00:32.427693+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:07.254107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.436812+0000","last_change":"2023-10-18T12:00:32.434053+0000","last_active":"2023-10-18T12:00:32.436812+0000","last_peered":"2023-10-18T12:00:32.436812+0000","last_clean":"2023-10-18T12:00:32.436812+0000","last_became_active":"2023-10-18T12:00:32.433800+0000","last_became_peered":"2023-10-18T12:00:32.433800+0000","last_unstale":"2023-10-18T12:00:32.436812+0000","last_undegraded":"2023-10-18T12:00:32.436812+0000","last_fullsized":"2023-10-18T12:00:32.436812+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:59.768535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"220'3","reported_seq":18,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418408+0000","last_change":"2023-10-18T12:00:20.057436+0000","last_active":"2023-10-18T12:00:31.418408+0000","last_peered":"2023-10-18T12:00:31.418408+0000","last_clean":"2023-10-18T12:00:31.418408+0000","last_became_active":"2023-10-18T12:00:20.057117+0000","last_became_peered":"2023-10-18T12:00:20.057117+0000","last_unstale":"2023-10-18T12:00:31.418408+0000","last_undegraded":"2023-10-18T12:00:31.418408+0000","last_fullsized":"2023-10-18T12:00:31.418408+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:02.796323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417821+0000","last_change":"2023-10-18T12:00:21.074575+0000","last_active":"2023-10-18T12:00:31.417821+0000","last_peered":"2023-10-18T12:00:31.417821+0000","last_clean":"2023-10-18T12:00:31.417821+0000","last_became_active":"2023-10-18T12:00:21.074302+0000","last_became_peered":"2023-10-18T12:00:21.074302+0000","last_unstale":"2023-10-18T12:00:31.417821+0000","last_undegraded":"2023-10-18T12:00:31.417821+0000","last_fullsized":"2023-10-18T12:00:31.417821+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:58.204038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454507+0000","last_change":"2023-10-18T12:00:31.437228+0000","last_active":"2023-10-18T12:00:32.454507+0000","last_peered":"2023-10-18T12:00:32.454507+0000","last_clean":"2023-10-18T12:00:32.454507+0000","last_became_active":"2023-10-18T12:00:31.436332+0000","last_became_peered":"2023-10-18T12:00:31.436332+0000","last_unstale":"2023-10-18T12:00:32.454507+0000","last_undegraded":"2023-10-18T12:00:32.454507+0000","last_fullsized":"2023-10-18T12:00:32.454507+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:56:52.724382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.435466+0000","last_change":"2023-10-18T12:00:32.432434+0000","last_active":"2023-10-18T12:00:32.435466+0000","last_peered":"2023-10-18T12:00:32.435466+0000","last_clean":"2023-10-18T12:00:32.435466+0000","last_became_active":"2023-10-18T12:00:32.432183+0000","last_became_peered":"2023-10-18T12:00:32.432183+0000","last_unstale":"2023-10-18T12:00:32.435466+0000","last_undegraded":"2023-10-18T12:00:32.435466+0000","last_fullsized":"2023-10-18T12:00:32.435466+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:09.830514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.423971+0000","last_change":"2023-10-18T12:00:20.070057+0000","last_active":"2023-10-18T12:00:31.423971+0000","last_peered":"2023-10-18T12:00:31.423971+0000","last_clean":"2023-10-18T12:00:31.423971+0000","last_became_active":"2023-10-18T12:00:20.069456+0000","last_became_peered":"2023-10-18T12:00:20.069456+0000","last_unstale":"2023-10-18T12:00:31.423971+0000","last_undegraded":"2023-10-18T12:00:31.423971+0000","last_fullsized":"2023-10-18T12:00:31.423971+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:14.207084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418867+0000","last_change":"2023-10-18T12:00:21.078356+0000","last_active":"2023-10-18T12:00:31.418867+0000","last_peered":"2023-10-18T12:00:31.418867+0000","last_clean":"2023-10-18T12:00:31.418867+0000","last_became_active":"2023-10-18T12:00:21.078095+0000","last_became_peered":"2023-10-18T12:00:21.078095+0000","last_unstale":"2023-10-18T12:00:31.418867+0000","last_undegraded":"2023-10-18T12:00:31.418867+0000","last_fullsized":"2023-10-18T12:00:31.418867+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:44.526073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"224'8","reported_seq":19,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454505+0000","last_change":"2023-10-18T12:00:31.437910+0000","last_active":"2023-10-18T12:00:32.454505+0000","last_peered":"2023-10-18T12:00:32.454505+0000","last_clean":"2023-10-18T12:00:32.454505+0000","last_became_active":"2023-10-18T12:00:31.437358+0000","last_became_peered":"2023-10-18T12:00:31.437358+0000","last_unstale":"2023-10-18T12:00:32.454505+0000","last_undegraded":"2023-10-18T12:00:32.454505+0000","last_fullsized":"2023-10-18T12:00:32.454505+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:54:49.192007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.444250+0000","last_change":"2023-10-18T12:00:32.443846+0000","last_active":"2023-10-18T12:00:32.444250+0000","last_peered":"2023-10-18T12:00:32.444250+0000","last_clean":"2023-10-18T12:00:32.444250+0000","last_became_active":"2023-10-18T12:00:32.443547+0000","last_became_peered":"2023-10-18T12:00:32.443547+0000","last_unstale":"2023-10-18T12:00:32.444250+0000","last_undegraded":"2023-10-18T12:00:32.444250+0000","last_fullsized":"2023-10-18T12:00:32.444250+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:43.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427667+0000","last_change":"2023-10-18T12:00:20.068018+0000","last_active":"2023-10-18T12:00:31.427667+0000","last_peered":"2023-10-18T12:00:31.427667+0000","last_clean":"2023-10-18T12:00:31.427667+0000","last_became_active":"2023-10-18T12:00:20.067744+0000","last_became_peered":"2023-10-18T12:00:20.067744+0000","last_unstale":"2023-10-18T12:00:31.427667+0000","last_undegraded":"2023-10-18T12:00:31.427667+0000","last_fullsized":"2023-10-18T12:00:31.427667+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:17.572716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417736+0000","last_change":"2023-10-18T12:00:21.079285+0000","last_active":"2023-10-18T12:00:31.417736+0000","last_peered":"2023-10-18T12:00:31.417736+0000","last_clean":"2023-10-18T12:00:31.417736+0000","last_became_active":"2023-10-18T12:00:21.079002+0000","last_became_peered":"2023-10-18T12:00:21.079002+0000","last_unstale":"2023-10-18T12:00:31.417736+0000","last_undegraded":"2023-10-18T12:00:31.417736+0000","last_fullsized":"2023-10-18T12:00:31.417736+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:49:28.849697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.421988+0000","last_change":"2023-10-18T12:00:31.427549+0000","last_active":"2023-10-18T12:00:32.421988+0000","last_peered":"2023-10-18T12:00:32.421988+0000","last_clean":"2023-10-18T12:00:32.421988+0000","last_became_active":"2023-10-18T12:00:31.427268+0000","last_became_peered":"2023-10-18T12:00:31.427268+0000","last_unstale":"2023-10-18T12:00:32.421988+0000","last_undegraded":"2023-10-18T12:00:32.421988+0000","last_fullsized":"2023-10-18T12:00:32.421988+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:28.728601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429571+0000","last_change":"2023-10-18T12:00:32.428371+0000","last_active":"2023-10-18T12:00:32.429571+0000","last_peered":"2023-10-18T12:00:32.429571+0000","last_clean":"2023-10-18T12:00:32.429571+0000","last_became_active":"2023-10-18T12:00:32.428129+0000","last_became_peered":"2023-10-18T12:00:32.428129+0000","last_unstale":"2023-10-18T12:00:32.429571+0000","last_undegraded":"2023-10-18T12:00:32.429571+0000","last_fullsized":"2023-10-18T12:00:32.429571+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:10.614449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454049+0000","last_change":"2023-10-18T12:00:31.434166+0000","last_active":"2023-10-18T12:00:32.454049+0000","last_peered":"2023-10-18T12:00:32.454049+0000","last_clean":"2023-10-18T12:00:32.454049+0000","last_became_active":"2023-10-18T12:00:31.433892+0000","last_became_peered":"2023-10-18T12:00:31.433892+0000","last_unstale":"2023-10-18T12:00:32.454049+0000","last_undegraded":"2023-10-18T12:00:32.454049+0000","last_fullsized":"2023-10-18T12:00:32.454049+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:59.870094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418892+0000","last_change":"2023-10-18T12:00:20.067947+0000","last_active":"2023-10-18T12:00:31.418892+0000","last_peered":"2023-10-18T12:00:31.418892+0000","last_clean":"2023-10-18T12:00:31.418892+0000","last_became_active":"2023-10-18T12:00:20.067433+0000","last_became_peered":"2023-10-18T12:00:20.067433+0000","last_unstale":"2023-10-18T12:00:31.418892+0000","last_undegraded":"2023-10-18T12:00:31.418892+0000","last_fullsized":"2023-10-18T12:00:31.418892+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:38.382188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418416+0000","last_change":"2023-10-18T12:00:21.076517+0000","last_active":"2023-10-18T12:00:31.418416+0000","last_peered":"2023-10-18T12:00:31.418416+0000","last_clean":"2023-10-18T12:00:31.418416+0000","last_became_active":"2023-10-18T12:00:21.076276+0000","last_became_peered":"2023-10-18T12:00:21.076276+0000","last_unstale":"2023-10-18T12:00:31.418416+0000","last_undegraded":"2023-10-18T12:00:31.418416+0000","last_fullsized":"2023-10-18T12:00:31.418416+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:02.908869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437343+0000","last_change":"2023-10-18T12:00:32.432675+0000","last_active":"2023-10-18T12:00:32.437343+0000","last_peered":"2023-10-18T12:00:32.437343+0000","last_clean":"2023-10-18T12:00:32.437343+0000","last_became_active":"2023-10-18T12:00:32.432408+0000","last_became_peered":"2023-10-18T12:00:32.432408+0000","last_unstale":"2023-10-18T12:00:32.437343+0000","last_undegraded":"2023-10-18T12:00:32.437343+0000","last_fullsized":"2023-10-18T12:00:32.437343+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:15:34.208634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"224'4","reported_seq":15,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.443955+0000","last_change":"2023-10-18T12:00:31.433811+0000","last_active":"2023-10-18T12:00:32.443955+0000","last_peered":"2023-10-18T12:00:32.443955+0000","last_clean":"2023-10-18T12:00:32.443955+0000","last_became_active":"2023-10-18T12:00:31.433567+0000","last_became_peered":"2023-10-18T12:00:31.433567+0000","last_unstale":"2023-10-18T12:00:32.443955+0000","last_undegraded":"2023-10-18T12:00:32.443955+0000","last_fullsized":"2023-10-18T12:00:32.443955+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:49.861828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424458+0000","last_change":"2023-10-18T12:00:20.070314+0000","last_active":"2023-10-18T12:00:31.424458+0000","last_peered":"2023-10-18T12:00:31.424458+0000","last_clean":"2023-10-18T12:00:31.424458+0000","last_became_active":"2023-10-18T12:00:20.069974+0000","last_became_peered":"2023-10-18T12:00:20.069974+0000","last_unstale":"2023-10-18T12:00:31.424458+0000","last_undegraded":"2023-10-18T12:00:31.424458+0000","last_fullsized":"2023-10-18T12:00:31.424458+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:39.359672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418039+0000","last_change":"2023-10-18T12:00:21.081350+0000","last_active":"2023-10-18T12:00:31.418039+0000","last_peered":"2023-10-18T12:00:31.418039+0000","last_clean":"2023-10-18T12:00:31.418039+0000","last_became_active":"2023-10-18T12:00:21.080920+0000","last_became_peered":"2023-10-18T12:00:21.080920+0000","last_unstale":"2023-10-18T12:00:31.418039+0000","last_undegraded":"2023-10-18T12:00:31.418039+0000","last_fullsized":"2023-10-18T12:00:31.418039+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:37.696091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.450381+0000","last_change":"2023-10-18T12:00:32.448948+0000","last_active":"2023-10-18T12:00:32.450381+0000","last_peered":"2023-10-18T12:00:32.450381+0000","last_clean":"2023-10-18T12:00:32.450381+0000","last_became_active":"2023-10-18T12:00:32.448247+0000","last_became_peered":"2023-10-18T12:00:32.448247+0000","last_unstale":"2023-10-18T12:00:32.450381+0000","last_undegraded":"2023-10-18T12:00:32.450381+0000","last_fullsized":"2023-10-18T12:00:32.450381+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:24.353309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424827+0000","last_change":"2023-10-18T12:00:20.070416+0000","last_active":"2023-10-18T12:00:31.424827+0000","last_peered":"2023-10-18T12:00:31.424827+0000","last_clean":"2023-10-18T12:00:31.424827+0000","last_became_active":"2023-10-18T12:00:20.070121+0000","last_became_peered":"2023-10-18T12:00:20.070121+0000","last_unstale":"2023-10-18T12:00:31.424827+0000","last_undegraded":"2023-10-18T12:00:31.424827+0000","last_fullsized":"2023-10-18T12:00:31.424827+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:33.588701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418223+0000","last_change":"2023-10-18T12:00:21.087738+0000","last_active":"2023-10-18T12:00:31.418223+0000","last_peered":"2023-10-18T12:00:31.418223+0000","last_clean":"2023-10-18T12:00:31.418223+0000","last_became_active":"2023-10-18T12:00:21.087104+0000","last_became_peered":"2023-10-18T12:00:21.087104+0000","last_unstale":"2023-10-18T12:00:31.418223+0000","last_undegraded":"2023-10-18T12:00:31.418223+0000","last_fullsized":"2023-10-18T12:00:31.418223+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:52.609784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.452420+0000","last_change":"2023-10-18T12:00:31.433384+0000","last_active":"2023-10-18T12:00:32.452420+0000","last_peered":"2023-10-18T12:00:32.452420+0000","last_clean":"2023-10-18T12:00:32.452420+0000","last_became_active":"2023-10-18T12:00:31.432880+0000","last_became_peered":"2023-10-18T12:00:31.432880+0000","last_unstale":"2023-10-18T12:00:32.452420+0000","last_undegraded":"2023-10-18T12:00:32.452420+0000","last_fullsized":"2023-10-18T12:00:32.452420+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:46.143888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.446550+0000","last_change":"2023-10-18T12:00:32.443728+0000","last_active":"2023-10-18T12:00:32.446550+0000","last_peered":"2023-10-18T12:00:32.446550+0000","last_clean":"2023-10-18T12:00:32.446550+0000","last_became_active":"2023-10-18T12:00:32.443454+0000","last_became_peered":"2023-10-18T12:00:32.443454+0000","last_unstale":"2023-10-18T12:00:32.446550+0000","last_undegraded":"2023-10-18T12:00:32.446550+0000","last_fullsized":"2023-10-18T12:00:32.446550+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:01.817297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418157+0000","last_change":"2023-10-18T12:00:20.060797+0000","last_active":"2023-10-18T12:00:31.418157+0000","last_peered":"2023-10-18T12:00:31.418157+0000","last_clean":"2023-10-18T12:00:31.418157+0000","last_became_active":"2023-10-18T12:00:20.060568+0000","last_became_peered":"2023-10-18T12:00:20.060568+0000","last_unstale":"2023-10-18T12:00:31.418157+0000","last_undegraded":"2023-10-18T12:00:31.418157+0000","last_fullsized":"2023-10-18T12:00:31.418157+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:27:50.071689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427310+0000","last_change":"2023-10-18T12:00:21.089013+0000","last_active":"2023-10-18T12:00:31.427310+0000","last_peered":"2023-10-18T12:00:31.427310+0000","last_clean":"2023-10-18T12:00:31.427310+0000","last_became_active":"2023-10-18T12:00:21.088643+0000","last_became_peered":"2023-10-18T12:00:21.088643+0000","last_unstale":"2023-10-18T12:00:31.427310+0000","last_undegraded":"2023-10-18T12:00:31.427310+0000","last_fullsized":"2023-10-18T12:00:31.427310+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:15.272122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448802+0000","last_change":"2023-10-18T12:00:31.436838+0000","last_active":"2023-10-18T12:00:32.448802+0000","last_peered":"2023-10-18T12:00:32.448802+0000","last_clean":"2023-10-18T12:00:32.448802+0000","last_became_active":"2023-10-18T12:00:31.436505+0000","last_became_peered":"2023-10-18T12:00:31.436505+0000","last_unstale":"2023-10-18T12:00:32.448802+0000","last_undegraded":"2023-10-18T12:00:32.448802+0000","last_fullsized":"2023-10-18T12:00:32.448802+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:18.690057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437629+0000","last_change":"2023-10-18T12:00:32.432921+0000","last_active":"2023-10-18T12:00:32.437629+0000","last_peered":"2023-10-18T12:00:32.437629+0000","last_clean":"2023-10-18T12:00:32.437629+0000","last_became_active":"2023-10-18T12:00:32.432594+0000","last_became_peered":"2023-10-18T12:00:32.432594+0000","last_unstale":"2023-10-18T12:00:32.437629+0000","last_undegraded":"2023-10-18T12:00:32.437629+0000","last_fullsized":"2023-10-18T12:00:32.437629+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:28:23.492154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421886+0000","last_change":"2023-10-18T12:00:20.057262+0000","last_active":"2023-10-18T12:00:31.421886+0000","last_peered":"2023-10-18T12:00:31.421886+0000","last_clean":"2023-10-18T12:00:31.421886+0000","last_became_active":"2023-10-18T12:00:20.057010+0000","last_became_peered":"2023-10-18T12:00:20.057010+0000","last_unstale":"2023-10-18T12:00:31.421886+0000","last_undegraded":"2023-10-18T12:00:31.421886+0000","last_fullsized":"2023-10-18T12:00:31.421886+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:46.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430746+0000","last_change":"2023-10-18T12:00:21.093491+0000","last_active":"2023-10-18T12:00:31.430746+0000","last_peered":"2023-10-18T12:00:31.430746+0000","last_clean":"2023-10-18T12:00:31.430746+0000","last_became_active":"2023-10-18T12:00:21.093199+0000","last_became_peered":"2023-10-18T12:00:21.093199+0000","last_unstale":"2023-10-18T12:00:31.430746+0000","last_undegraded":"2023-10-18T12:00:31.430746+0000","last_fullsized":"2023-10-18T12:00:31.430746+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:14.227710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454818+0000","last_change":"2023-10-18T12:00:31.432278+0000","last_active":"2023-10-18T12:00:32.454818+0000","last_peered":"2023-10-18T12:00:32.454818+0000","last_clean":"2023-10-18T12:00:32.454818+0000","last_became_active":"2023-10-18T12:00:31.432022+0000","last_became_peered":"2023-10-18T12:00:31.432022+0000","last_unstale":"2023-10-18T12:00:32.454818+0000","last_undegraded":"2023-10-18T12:00:32.454818+0000","last_fullsized":"2023-10-18T12:00:32.454818+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:50.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.432346+0000","last_change":"2023-10-18T12:00:32.429915+0000","last_active":"2023-10-18T12:00:32.432346+0000","last_peered":"2023-10-18T12:00:32.432346+0000","last_clean":"2023-10-18T12:00:32.432346+0000","last_became_active":"2023-10-18T12:00:32.429667+0000","last_became_peered":"2023-10-18T12:00:32.429667+0000","last_unstale":"2023-10-18T12:00:32.432346+0000","last_undegraded":"2023-10-18T12:00:32.432346+0000","last_fullsized":"2023-10-18T12:00:32.432346+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:54.061969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419184+0000","last_change":"2023-10-18T12:00:20.057976+0000","last_active":"2023-10-18T12:00:31.419184+0000","last_peered":"2023-10-18T12:00:31.419184+0000","last_clean":"2023-10-18T12:00:31.419184+0000","last_became_active":"2023-10-18T12:00:20.057699+0000","last_became_peered":"2023-10-18T12:00:20.057699+0000","last_unstale":"2023-10-18T12:00:31.419184+0000","last_undegraded":"2023-10-18T12:00:31.419184+0000","last_fullsized":"2023-10-18T12:00:31.419184+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:45.951679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418854+0000","last_change":"2023-10-18T12:00:21.073385+0000","last_active":"2023-10-18T12:00:31.418854+0000","last_peered":"2023-10-18T12:00:31.418854+0000","last_clean":"2023-10-18T12:00:31.418854+0000","last_became_active":"2023-10-18T12:00:21.073048+0000","last_became_peered":"2023-10-18T12:00:21.073048+0000","last_unstale":"2023-10-18T12:00:31.418854+0000","last_undegraded":"2023-10-18T12:00:31.418854+0000","last_fullsized":"2023-10-18T12:00:31.418854+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:23:54.891459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.418204+0000","last_change":"2023-10-18T12:00:31.432535+0000","last_active":"2023-10-18T12:00:32.418204+0000","last_peered":"2023-10-18T12:00:32.418204+0000","last_clean":"2023-10-18T12:00:32.418204+0000","last_became_active":"2023-10-18T12:00:31.432294+0000","last_became_peered":"2023-10-18T12:00:31.432294+0000","last_unstale":"2023-10-18T12:00:32.418204+0000","last_undegraded":"2023-10-18T12:00:32.418204+0000","last_fullsized":"2023-10-18T12:00:32.418204+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:31.165188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.439411+0000","last_change":"2023-10-18T12:00:32.438452+0000","last_active":"2023-10-18T12:00:32.439411+0000","last_peered":"2023-10-18T12:00:32.439411+0000","last_clean":"2023-10-18T12:00:32.439411+0000","last_became_active":"2023-10-18T12:00:32.438061+0000","last_became_peered":"2023-10-18T12:00:32.438061+0000","last_unstale":"2023-10-18T12:00:32.439411+0000","last_undegraded":"2023-10-18T12:00:32.439411+0000","last_fullsized":"2023-10-18T12:00:32.439411+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:45.957002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"220'4","reported_seq":19,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418193+0000","last_change":"2023-10-18T12:00:20.063561+0000","last_active":"2023-10-18T12:00:31.418193+0000","last_peered":"2023-10-18T12:00:31.418193+0000","last_clean":"2023-10-18T12:00:31.418193+0000","last_became_active":"2023-10-18T12:00:20.063285+0000","last_became_peered":"2023-10-18T12:00:20.063285+0000","last_unstale":"2023-10-18T12:00:31.418193+0000","last_undegraded":"2023-10-18T12:00:31.418193+0000","last_fullsized":"2023-10-18T12:00:31.418193+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:55.051193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421603+0000","last_change":"2023-10-18T12:00:21.073279+0000","last_active":"2023-10-18T12:00:31.421603+0000","last_peered":"2023-10-18T12:00:31.421603+0000","last_clean":"2023-10-18T12:00:31.421603+0000","last_became_active":"2023-10-18T12:00:21.073036+0000","last_became_peered":"2023-10-18T12:00:21.073036+0000","last_unstale":"2023-10-18T12:00:31.421603+0000","last_undegraded":"2023-10-18T12:00:31.421603+0000","last_fullsized":"2023-10-18T12:00:31.421603+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:28.430510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.421668+0000","last_change":"2023-10-18T12:00:31.433300+0000","last_active":"2023-10-18T12:00:32.421668+0000","last_peered":"2023-10-18T12:00:32.421668+0000","last_clean":"2023-10-18T12:00:32.421668+0000","last_became_active":"2023-10-18T12:00:31.432328+0000","last_became_peered":"2023-10-18T12:00:31.432328+0000","last_unstale":"2023-10-18T12:00:32.421668+0000","last_undegraded":"2023-10-18T12:00:32.421668+0000","last_fullsized":"2023-10-18T12:00:32.421668+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:43.256103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.452017+0000","last_change":"2023-10-18T12:00:32.451045+0000","last_active":"2023-10-18T12:00:32.452017+0000","last_peered":"2023-10-18T12:00:32.452017+0000","last_clean":"2023-10-18T12:00:32.452017+0000","last_became_active":"2023-10-18T12:00:32.450742+0000","last_became_peered":"2023-10-18T12:00:32.450742+0000","last_unstale":"2023-10-18T12:00:32.452017+0000","last_undegraded":"2023-10-18T12:00:32.452017+0000","last_fullsized":"2023-10-18T12:00:32.452017+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:26:52.679064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424960+0000","last_change":"2023-10-18T12:00:21.093224+0000","last_active":"2023-10-18T12:00:31.424960+0000","last_peered":"2023-10-18T12:00:31.424960+0000","last_clean":"2023-10-18T12:00:31.424960+0000","last_became_active":"2023-10-18T12:00:21.092949+0000","last_became_peered":"2023-10-18T12:00:21.092949+0000","last_unstale":"2023-10-18T12:00:31.424960+0000","last_undegraded":"2023-10-18T12:00:31.424960+0000","last_fullsized":"2023-10-18T12:00:31.424960+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:23.786569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.439268+0000","last_change":"2023-10-18T12:00:32.438297+0000","last_active":"2023-10-18T12:00:32.439268+0000","last_peered":"2023-10-18T12:00:32.439268+0000","last_clean":"2023-10-18T12:00:32.439268+0000","last_became_active":"2023-10-18T12:00:32.438050+0000","last_became_peered":"2023-10-18T12:00:32.438050+0000","last_unstale":"2023-10-18T12:00:32.439268+0000","last_undegraded":"2023-10-18T12:00:32.439268+0000","last_fullsized":"2023-10-18T12:00:32.439268+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:04.205911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424033+0000","last_change":"2023-10-18T12:00:21.092324+0000","last_active":"2023-10-18T12:00:31.424033+0000","last_peered":"2023-10-18T12:00:31.424033+0000","last_clean":"2023-10-18T12:00:31.424033+0000","last_became_active":"2023-10-18T12:00:21.092059+0000","last_became_peered":"2023-10-18T12:00:21.092059+0000","last_unstale":"2023-10-18T12:00:31.424033+0000","last_undegraded":"2023-10-18T12:00:31.424033+0000","last_fullsized":"2023-10-18T12:00:31.424033+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:15.346268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.452174+0000","last_change":"2023-10-18T12:00:32.451696+0000","last_active":"2023-10-18T12:00:32.452174+0000","last_peered":"2023-10-18T12:00:32.452174+0000","last_clean":"2023-10-18T12:00:32.452174+0000","last_became_active":"2023-10-18T12:00:32.451393+0000","last_became_peered":"2023-10-18T12:00:32.451393+0000","last_unstale":"2023-10-18T12:00:32.452174+0000","last_undegraded":"2023-10-18T12:00:32.452174+0000","last_fullsized":"2023-10-18T12:00:32.452174+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:20.931554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417811+0000","last_change":"2023-10-18T12:00:21.087421+0000","last_active":"2023-10-18T12:00:31.417811+0000","last_peered":"2023-10-18T12:00:31.417811+0000","last_clean":"2023-10-18T12:00:31.417811+0000","last_became_active":"2023-10-18T12:00:21.087145+0000","last_became_peered":"2023-10-18T12:00:21.087145+0000","last_unstale":"2023-10-18T12:00:31.417811+0000","last_undegraded":"2023-10-18T12:00:31.417811+0000","last_fullsized":"2023-10-18T12:00:31.417811+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:28.489188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430401+0000","last_change":"2023-10-18T12:00:32.429789+0000","last_active":"2023-10-18T12:00:32.430401+0000","last_peered":"2023-10-18T12:00:32.430401+0000","last_clean":"2023-10-18T12:00:32.430401+0000","last_became_active":"2023-10-18T12:00:32.429522+0000","last_became_peered":"2023-10-18T12:00:32.429522+0000","last_unstale":"2023-10-18T12:00:32.430401+0000","last_undegraded":"2023-10-18T12:00:32.430401+0000","last_fullsized":"2023-10-18T12:00:32.430401+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:55.109644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417772+0000","last_change":"2023-10-18T12:00:21.082792+0000","last_active":"2023-10-18T12:00:31.417772+0000","last_peered":"2023-10-18T12:00:31.417772+0000","last_clean":"2023-10-18T12:00:31.417772+0000","last_became_active":"2023-10-18T12:00:21.082529+0000","last_became_peered":"2023-10-18T12:00:21.082529+0000","last_unstale":"2023-10-18T12:00:31.417772+0000","last_undegraded":"2023-10-18T12:00:31.417772+0000","last_fullsized":"2023-10-18T12:00:31.417772+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:02.129934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.449548+0000","last_change":"2023-10-18T12:00:32.449052+0000","last_active":"2023-10-18T12:00:32.449548+0000","last_peered":"2023-10-18T12:00:32.449548+0000","last_clean":"2023-10-18T12:00:32.449548+0000","last_became_active":"2023-10-18T12:00:32.448768+0000","last_became_peered":"2023-10-18T12:00:32.448768+0000","last_unstale":"2023-10-18T12:00:32.449548+0000","last_undegraded":"2023-10-18T12:00:32.449548+0000","last_fullsized":"2023-10-18T12:00:32.449548+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:20.330916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427015+0000","last_change":"2023-10-18T12:00:21.086100+0000","last_active":"2023-10-18T12:00:31.427015+0000","last_peered":"2023-10-18T12:00:31.427015+0000","last_clean":"2023-10-18T12:00:31.427015+0000","last_became_active":"2023-10-18T12:00:21.085820+0000","last_became_peered":"2023-10-18T12:00:21.085820+0000","last_unstale":"2023-10-18T12:00:31.427015+0000","last_undegraded":"2023-10-18T12:00:31.427015+0000","last_fullsized":"2023-10-18T12:00:31.427015+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:59.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448820+0000","last_change":"2023-10-18T12:00:32.446349+0000","last_active":"2023-10-18T12:00:32.448820+0000","last_peered":"2023-10-18T12:00:32.448820+0000","last_clean":"2023-10-18T12:00:32.448820+0000","last_became_active":"2023-10-18T12:00:32.442822+0000","last_became_peered":"2023-10-18T12:00:32.442822+0000","last_unstale":"2023-10-18T12:00:32.448820+0000","last_undegraded":"2023-10-18T12:00:32.448820+0000","last_fullsized":"2023-10-18T12:00:32.448820+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:24.527236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419602+0000","last_change":"2023-10-18T12:00:21.093406+0000","last_active":"2023-10-18T12:00:31.419602+0000","last_peered":"2023-10-18T12:00:31.419602+0000","last_clean":"2023-10-18T12:00:31.419602+0000","last_became_active":"2023-10-18T12:00:21.093138+0000","last_became_peered":"2023-10-18T12:00:21.093138+0000","last_unstale":"2023-10-18T12:00:31.419602+0000","last_undegraded":"2023-10-18T12:00:31.419602+0000","last_fullsized":"2023-10-18T12:00:31.419602+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:09.568953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.447530+0000","last_change":"2023-10-18T12:00:32.445903+0000","last_active":"2023-10-18T12:00:32.447530+0000","last_peered":"2023-10-18T12:00:32.447530+0000","last_clean":"2023-10-18T12:00:32.447530+0000","last_became_active":"2023-10-18T12:00:32.442480+0000","last_became_peered":"2023-10-18T12:00:32.442480+0000","last_unstale":"2023-10-18T12:00:32.447530+0000","last_undegraded":"2023-10-18T12:00:32.447530+0000","last_fullsized":"2023-10-18T12:00:32.447530+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:09.352512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419081+0000","last_change":"2023-10-18T12:00:21.091634+0000","last_active":"2023-10-18T12:00:31.419081+0000","last_peered":"2023-10-18T12:00:31.419081+0000","last_clean":"2023-10-18T12:00:31.419081+0000","last_became_active":"2023-10-18T12:00:21.091327+0000","last_became_peered":"2023-10-18T12:00:21.091327+0000","last_unstale":"2023-10-18T12:00:31.419081+0000","last_undegraded":"2023-10-18T12:00:31.419081+0000","last_fullsized":"2023-10-18T12:00:31.419081+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:53:58.179224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.438726+0000","last_change":"2023-10-18T12:00:32.435574+0000","last_active":"2023-10-18T12:00:32.438726+0000","last_peered":"2023-10-18T12:00:32.438726+0000","last_clean":"2023-10-18T12:00:32.438726+0000","last_became_active":"2023-10-18T12:00:32.435277+0000","last_became_peered":"2023-10-18T12:00:32.435277+0000","last_unstale":"2023-10-18T12:00:32.438726+0000","last_undegraded":"2023-10-18T12:00:32.438726+0000","last_fullsized":"2023-10-18T12:00:32.438726+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:25.649380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418366+0000","last_change":"2023-10-18T12:00:21.075391+0000","last_active":"2023-10-18T12:00:31.418366+0000","last_peered":"2023-10-18T12:00:31.418366+0000","last_clean":"2023-10-18T12:00:31.418366+0000","last_became_active":"2023-10-18T12:00:21.075140+0000","last_became_peered":"2023-10-18T12:00:21.075140+0000","last_unstale":"2023-10-18T12:00:31.418366+0000","last_undegraded":"2023-10-18T12:00:31.418366+0000","last_fullsized":"2023-10-18T12:00:31.418366+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:16:03.472142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.439634+0000","last_change":"2023-10-18T12:00:32.438699+0000","last_active":"2023-10-18T12:00:32.439634+0000","last_peered":"2023-10-18T12:00:32.439634+0000","last_clean":"2023-10-18T12:00:32.439634+0000","last_became_active":"2023-10-18T12:00:32.438444+0000","last_became_peered":"2023-10-18T12:00:32.438444+0000","last_unstale":"2023-10-18T12:00:32.439634+0000","last_undegraded":"2023-10-18T12:00:32.439634+0000","last_fullsized":"2023-10-18T12:00:32.439634+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:25.347761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426145+0000","last_change":"2023-10-18T12:00:21.082982+0000","last_active":"2023-10-18T12:00:31.426145+0000","last_peered":"2023-10-18T12:00:31.426145+0000","last_clean":"2023-10-18T12:00:31.426145+0000","last_became_active":"2023-10-18T12:00:21.082700+0000","last_became_peered":"2023-10-18T12:00:21.082700+0000","last_unstale":"2023-10-18T12:00:31.426145+0000","last_undegraded":"2023-10-18T12:00:31.426145+0000","last_fullsized":"2023-10-18T12:00:31.426145+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:25.713305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430319+0000","last_change":"2023-10-18T12:00:32.429033+0000","last_active":"2023-10-18T12:00:32.430319+0000","last_peered":"2023-10-18T12:00:32.430319+0000","last_clean":"2023-10-18T12:00:32.430319+0000","last_became_active":"2023-10-18T12:00:32.428732+0000","last_became_peered":"2023-10-18T12:00:32.428732+0000","last_unstale":"2023-10-18T12:00:32.430319+0000","last_undegraded":"2023-10-18T12:00:32.430319+0000","last_fullsized":"2023-10-18T12:00:32.430319+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:12.002567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421025+0000","last_change":"2023-10-18T12:00:21.073828+0000","last_active":"2023-10-18T12:00:31.421025+0000","last_peered":"2023-10-18T12:00:31.421025+0000","last_clean":"2023-10-18T12:00:31.421025+0000","last_became_active":"2023-10-18T12:00:21.073594+0000","last_became_peered":"2023-10-18T12:00:21.073594+0000","last_unstale":"2023-10-18T12:00:31.421025+0000","last_undegraded":"2023-10-18T12:00:31.421025+0000","last_fullsized":"2023-10-18T12:00:31.421025+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:02.908869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445449+0000","last_change":"2023-10-18T12:00:32.441667+0000","last_active":"2023-10-18T12:00:32.445449+0000","last_peered":"2023-10-18T12:00:32.445449+0000","last_clean":"2023-10-18T12:00:32.445449+0000","last_became_active":"2023-10-18T12:00:32.441184+0000","last_became_peered":"2023-10-18T12:00:32.441184+0000","last_unstale":"2023-10-18T12:00:32.445449+0000","last_undegraded":"2023-10-18T12:00:32.445449+0000","last_fullsized":"2023-10-18T12:00:32.445449+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:39.793111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430664+0000","last_change":"2023-10-18T12:00:21.091079+0000","last_active":"2023-10-18T12:00:31.430664+0000","last_peered":"2023-10-18T12:00:31.430664+0000","last_clean":"2023-10-18T12:00:31.430664+0000","last_became_active":"2023-10-18T12:00:21.090440+0000","last_became_peered":"2023-10-18T12:00:21.090440+0000","last_unstale":"2023-10-18T12:00:31.430664+0000","last_undegraded":"2023-10-18T12:00:31.430664+0000","last_fullsized":"2023-10-18T12:00:31.430664+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:54.338893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429873+0000","last_change":"2023-10-18T12:00:32.428557+0000","last_active":"2023-10-18T12:00:32.429873+0000","last_peered":"2023-10-18T12:00:32.429873+0000","last_clean":"2023-10-18T12:00:32.429873+0000","last_became_active":"2023-10-18T12:00:32.428263+0000","last_became_peered":"2023-10-18T12:00:32.428263+0000","last_unstale":"2023-10-18T12:00:32.429873+0000","last_undegraded":"2023-10-18T12:00:32.429873+0000","last_fullsized":"2023-10-18T12:00:32.429873+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:24.581272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430792+0000","last_change":"2023-10-18T12:00:21.093298+0000","last_active":"2023-10-18T12:00:31.430792+0000","last_peered":"2023-10-18T12:00:31.430792+0000","last_clean":"2023-10-18T12:00:31.430792+0000","last_became_active":"2023-10-18T12:00:21.092815+0000","last_became_peered":"2023-10-18T12:00:21.092815+0000","last_unstale":"2023-10-18T12:00:31.430792+0000","last_undegraded":"2023-10-18T12:00:31.430792+0000","last_fullsized":"2023-10-18T12:00:31.430792+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:28.526624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430396+0000","last_change":"2023-10-18T12:00:32.429617+0000","last_active":"2023-10-18T12:00:32.430396+0000","last_peered":"2023-10-18T12:00:32.430396+0000","last_clean":"2023-10-18T12:00:32.430396+0000","last_became_active":"2023-10-18T12:00:32.429363+0000","last_became_peered":"2023-10-18T12:00:32.429363+0000","last_unstale":"2023-10-18T12:00:32.430396+0000","last_undegraded":"2023-10-18T12:00:32.430396+0000","last_fullsized":"2023-10-18T12:00:32.430396+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:18.493180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.431445+0000","last_change":"2023-10-18T12:00:21.090556+0000","last_active":"2023-10-18T12:00:31.431445+0000","last_peered":"2023-10-18T12:00:31.431445+0000","last_clean":"2023-10-18T12:00:31.431445+0000","last_became_active":"2023-10-18T12:00:21.088740+0000","last_became_peered":"2023-10-18T12:00:21.088740+0000","last_unstale":"2023-10-18T12:00:31.431445+0000","last_undegraded":"2023-10-18T12:00:31.431445+0000","last_fullsized":"2023-10-18T12:00:31.431445+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:32.899439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.451755+0000","last_change":"2023-10-18T12:00:32.449838+0000","last_active":"2023-10-18T12:00:32.451755+0000","last_peered":"2023-10-18T12:00:32.451755+0000","last_clean":"2023-10-18T12:00:32.451755+0000","last_became_active":"2023-10-18T12:00:32.449575+0000","last_became_peered":"2023-10-18T12:00:32.449575+0000","last_unstale":"2023-10-18T12:00:32.451755+0000","last_undegraded":"2023-10-18T12:00:32.451755+0000","last_fullsized":"2023-10-18T12:00:32.451755+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:35.845909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.432016+0000","last_change":"2023-10-18T12:00:21.092451+0000","last_active":"2023-10-18T12:00:31.432016+0000","last_peered":"2023-10-18T12:00:31.432016+0000","last_clean":"2023-10-18T12:00:31.432016+0000","last_became_active":"2023-10-18T12:00:21.091995+0000","last_became_peered":"2023-10-18T12:00:21.091995+0000","last_unstale":"2023-10-18T12:00:31.432016+0000","last_undegraded":"2023-10-18T12:00:31.432016+0000","last_fullsized":"2023-10-18T12:00:31.432016+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:27.792233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.432619+0000","last_change":"2023-10-18T12:00:32.430743+0000","last_active":"2023-10-18T12:00:32.432619+0000","last_peered":"2023-10-18T12:00:32.432619+0000","last_clean":"2023-10-18T12:00:32.432619+0000","last_became_active":"2023-10-18T12:00:32.430469+0000","last_became_peered":"2023-10-18T12:00:32.430469+0000","last_unstale":"2023-10-18T12:00:32.432619+0000","last_undegraded":"2023-10-18T12:00:32.432619+0000","last_fullsized":"2023-10-18T12:00:32.432619+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:12.671247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418175+0000","last_change":"2023-10-18T12:00:21.076021+0000","last_active":"2023-10-18T12:00:31.418175+0000","last_peered":"2023-10-18T12:00:31.418175+0000","last_clean":"2023-10-18T12:00:31.418175+0000","last_became_active":"2023-10-18T12:00:21.075761+0000","last_became_peered":"2023-10-18T12:00:21.075761+0000","last_unstale":"2023-10-18T12:00:31.418175+0000","last_undegraded":"2023-10-18T12:00:31.418175+0000","last_fullsized":"2023-10-18T12:00:31.418175+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:47.728965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.446244+0000","last_change":"2023-10-18T12:00:32.445150+0000","last_active":"2023-10-18T12:00:32.446244+0000","last_peered":"2023-10-18T12:00:32.446244+0000","last_clean":"2023-10-18T12:00:32.446244+0000","last_became_active":"2023-10-18T12:00:32.443531+0000","last_became_peered":"2023-10-18T12:00:32.443531+0000","last_unstale":"2023-10-18T12:00:32.446244+0000","last_undegraded":"2023-10-18T12:00:32.446244+0000","last_fullsized":"2023-10-18T12:00:32.446244+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:08.260044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418337+0000","last_change":"2023-10-18T12:00:21.089110+0000","last_active":"2023-10-18T12:00:31.418337+0000","last_peered":"2023-10-18T12:00:31.418337+0000","last_clean":"2023-10-18T12:00:31.418337+0000","last_became_active":"2023-10-18T12:00:21.088828+0000","last_became_peered":"2023-10-18T12:00:21.088828+0000","last_unstale":"2023-10-18T12:00:31.418337+0000","last_undegraded":"2023-10-18T12:00:31.418337+0000","last_fullsized":"2023-10-18T12:00:31.418337+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:09.659998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.446154+0000","last_change":"2023-10-18T12:00:32.444890+0000","last_active":"2023-10-18T12:00:32.446154+0000","last_peered":"2023-10-18T12:00:32.446154+0000","last_clean":"2023-10-18T12:00:32.446154+0000","last_became_active":"2023-10-18T12:00:32.443733+0000","last_became_peered":"2023-10-18T12:00:32.443733+0000","last_unstale":"2023-10-18T12:00:32.446154+0000","last_undegraded":"2023-10-18T12:00:32.446154+0000","last_fullsized":"2023-10-18T12:00:32.446154+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:24.356601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418828+0000","last_change":"2023-10-18T12:00:21.082735+0000","last_active":"2023-10-18T12:00:31.418828+0000","last_peered":"2023-10-18T12:00:31.418828+0000","last_clean":"2023-10-18T12:00:31.418828+0000","last_became_active":"2023-10-18T12:00:21.082501+0000","last_became_peered":"2023-10-18T12:00:21.082501+0000","last_unstale":"2023-10-18T12:00:31.418828+0000","last_undegraded":"2023-10-18T12:00:31.418828+0000","last_fullsized":"2023-10-18T12:00:31.418828+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:11.830792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.436512+0000","last_change":"2023-10-18T12:00:32.435390+0000","last_active":"2023-10-18T12:00:32.436512+0000","last_peered":"2023-10-18T12:00:32.436512+0000","last_clean":"2023-10-18T12:00:32.436512+0000","last_became_active":"2023-10-18T12:00:32.435108+0000","last_became_peered":"2023-10-18T12:00:32.435108+0000","last_unstale":"2023-10-18T12:00:32.436512+0000","last_undegraded":"2023-10-18T12:00:32.436512+0000","last_fullsized":"2023-10-18T12:00:32.436512+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:52.662776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418091+0000","last_change":"2023-10-18T12:00:21.085027+0000","last_active":"2023-10-18T12:00:31.418091+0000","last_peered":"2023-10-18T12:00:31.418091+0000","last_clean":"2023-10-18T12:00:31.418091+0000","last_became_active":"2023-10-18T12:00:21.084752+0000","last_became_peered":"2023-10-18T12:00:21.084752+0000","last_unstale":"2023-10-18T12:00:31.418091+0000","last_undegraded":"2023-10-18T12:00:31.418091+0000","last_fullsized":"2023-10-18T12:00:31.418091+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:55.331050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.435671+0000","last_change":"2023-10-18T12:00:32.434238+0000","last_active":"2023-10-18T12:00:32.435671+0000","last_peered":"2023-10-18T12:00:32.435671+0000","last_clean":"2023-10-18T12:00:32.435671+0000","last_became_active":"2023-10-18T12:00:32.433870+0000","last_became_peered":"2023-10-18T12:00:32.433870+0000","last_unstale":"2023-10-18T12:00:32.435671+0000","last_undegraded":"2023-10-18T12:00:32.435671+0000","last_fullsized":"2023-10-18T12:00:32.435671+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:12.168935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430586+0000","last_change":"2023-10-18T12:00:21.091256+0000","last_active":"2023-10-18T12:00:31.430586+0000","last_peered":"2023-10-18T12:00:31.430586+0000","last_clean":"2023-10-18T12:00:31.430586+0000","last_became_active":"2023-10-18T12:00:21.090808+0000","last_became_peered":"2023-10-18T12:00:21.090808+0000","last_unstale":"2023-10-18T12:00:31.430586+0000","last_undegraded":"2023-10-18T12:00:31.430586+0000","last_fullsized":"2023-10-18T12:00:31.430586+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:53:42.580058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.427205+0000","last_change":"2023-10-18T12:00:32.425449+0000","last_active":"2023-10-18T12:00:32.427205+0000","last_peered":"2023-10-18T12:00:32.427205+0000","last_clean":"2023-10-18T12:00:32.427205+0000","last_became_active":"2023-10-18T12:00:32.425207+0000","last_became_peered":"2023-10-18T12:00:32.425207+0000","last_unstale":"2023-10-18T12:00:32.427205+0000","last_undegraded":"2023-10-18T12:00:32.427205+0000","last_fullsized":"2023-10-18T12:00:32.427205+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:26.575385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424725+0000","last_change":"2023-10-18T12:00:21.084368+0000","last_active":"2023-10-18T12:00:31.424725+0000","last_peered":"2023-10-18T12:00:31.424725+0000","last_clean":"2023-10-18T12:00:31.424725+0000","last_became_active":"2023-10-18T12:00:21.084087+0000","last_became_peered":"2023-10-18T12:00:21.084087+0000","last_unstale":"2023-10-18T12:00:31.424725+0000","last_undegraded":"2023-10-18T12:00:31.424725+0000","last_fullsized":"2023-10-18T12:00:31.424725+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:26.534619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.432641+0000","last_change":"2023-10-18T12:00:32.430200+0000","last_active":"2023-10-18T12:00:32.432641+0000","last_peered":"2023-10-18T12:00:32.432641+0000","last_clean":"2023-10-18T12:00:32.432641+0000","last_became_active":"2023-10-18T12:00:32.429934+0000","last_became_peered":"2023-10-18T12:00:32.429934+0000","last_unstale":"2023-10-18T12:00:32.432641+0000","last_undegraded":"2023-10-18T12:00:32.432641+0000","last_fullsized":"2023-10-18T12:00:32.432641+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:40:58.139167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426934+0000","last_change":"2023-10-18T12:00:21.087044+0000","last_active":"2023-10-18T12:00:31.426934+0000","last_peered":"2023-10-18T12:00:31.426934+0000","last_clean":"2023-10-18T12:00:31.426934+0000","last_became_active":"2023-10-18T12:00:21.086784+0000","last_became_peered":"2023-10-18T12:00:21.086784+0000","last_unstale":"2023-10-18T12:00:31.426934+0000","last_undegraded":"2023-10-18T12:00:31.426934+0000","last_fullsized":"2023-10-18T12:00:31.426934+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:16.398494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430360+0000","last_change":"2023-10-18T12:00:32.427873+0000","last_active":"2023-10-18T12:00:32.430360+0000","last_peered":"2023-10-18T12:00:32.430360+0000","last_clean":"2023-10-18T12:00:32.430360+0000","last_became_active":"2023-10-18T12:00:32.427634+0000","last_became_peered":"2023-10-18T12:00:32.427634+0000","last_unstale":"2023-10-18T12:00:32.430360+0000","last_undegraded":"2023-10-18T12:00:32.430360+0000","last_fullsized":"2023-10-18T12:00:32.430360+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:38:51.295444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419521+0000","last_change":"2023-10-18T12:00:21.097042+0000","last_active":"2023-10-18T12:00:31.419521+0000","last_peered":"2023-10-18T12:00:31.419521+0000","last_clean":"2023-10-18T12:00:31.419521+0000","last_became_active":"2023-10-18T12:00:21.096759+0000","last_became_peered":"2023-10-18T12:00:21.096759+0000","last_unstale":"2023-10-18T12:00:31.419521+0000","last_undegraded":"2023-10-18T12:00:31.419521+0000","last_fullsized":"2023-10-18T12:00:31.419521+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:41.300175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454607+0000","last_change":"2023-10-18T12:00:32.453930+0000","last_active":"2023-10-18T12:00:32.454607+0000","last_peered":"2023-10-18T12:00:32.454607+0000","last_clean":"2023-10-18T12:00:32.454607+0000","last_became_active":"2023-10-18T12:00:32.453641+0000","last_became_peered":"2023-10-18T12:00:32.453641+0000","last_unstale":"2023-10-18T12:00:32.454607+0000","last_undegraded":"2023-10-18T12:00:32.454607+0000","last_fullsized":"2023-10-18T12:00:32.454607+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:32.455890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.432374+0000","last_change":"2023-10-18T12:00:21.091511+0000","last_active":"2023-10-18T12:00:31.432374+0000","last_peered":"2023-10-18T12:00:31.432374+0000","last_clean":"2023-10-18T12:00:31.432374+0000","last_became_active":"2023-10-18T12:00:21.091236+0000","last_became_peered":"2023-10-18T12:00:21.091236+0000","last_unstale":"2023-10-18T12:00:31.432374+0000","last_undegraded":"2023-10-18T12:00:31.432374+0000","last_fullsized":"2023-10-18T12:00:31.432374+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:27.189758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.433034+0000","last_change":"2023-10-18T12:00:32.431957+0000","last_active":"2023-10-18T12:00:32.433034+0000","last_peered":"2023-10-18T12:00:32.433034+0000","last_clean":"2023-10-18T12:00:32.433034+0000","last_became_active":"2023-10-18T12:00:32.431666+0000","last_became_peered":"2023-10-18T12:00:32.431666+0000","last_unstale":"2023-10-18T12:00:32.433034+0000","last_undegraded":"2023-10-18T12:00:32.433034+0000","last_fullsized":"2023-10-18T12:00:32.433034+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:32:59.350342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418257+0000","last_change":"2023-10-18T12:00:21.084370+0000","last_active":"2023-10-18T12:00:31.418257+0000","last_peered":"2023-10-18T12:00:31.418257+0000","last_clean":"2023-10-18T12:00:31.418257+0000","last_became_active":"2023-10-18T12:00:21.082951+0000","last_became_peered":"2023-10-18T12:00:21.082951+0000","last_unstale":"2023-10-18T12:00:31.418257+0000","last_undegraded":"2023-10-18T12:00:31.418257+0000","last_fullsized":"2023-10-18T12:00:31.418257+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:42:38.614585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.435958+0000","last_change":"2023-10-18T12:00:32.434925+0000","last_active":"2023-10-18T12:00:32.435958+0000","last_peered":"2023-10-18T12:00:32.435958+0000","last_clean":"2023-10-18T12:00:32.435958+0000","last_became_active":"2023-10-18T12:00:32.434647+0000","last_became_peered":"2023-10-18T12:00:32.434647+0000","last_unstale":"2023-10-18T12:00:32.435958+0000","last_undegraded":"2023-10-18T12:00:32.435958+0000","last_fullsized":"2023-10-18T12:00:32.435958+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:55.700931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.429699+0000","last_change":"2023-10-18T12:00:21.085280+0000","last_active":"2023-10-18T12:00:31.429699+0000","last_peered":"2023-10-18T12:00:31.429699+0000","last_clean":"2023-10-18T12:00:31.429699+0000","last_became_active":"2023-10-18T12:00:21.085007+0000","last_became_peered":"2023-10-18T12:00:21.085007+0000","last_unstale":"2023-10-18T12:00:31.429699+0000","last_undegraded":"2023-10-18T12:00:31.429699+0000","last_fullsized":"2023-10-18T12:00:31.429699+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:00:11.590501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445527+0000","last_change":"2023-10-18T12:00:32.441884+0000","last_active":"2023-10-18T12:00:32.445527+0000","last_peered":"2023-10-18T12:00:32.445527+0000","last_clean":"2023-10-18T12:00:32.445527+0000","last_became_active":"2023-10-18T12:00:32.441170+0000","last_became_peered":"2023-10-18T12:00:32.441170+0000","last_unstale":"2023-10-18T12:00:32.445527+0000","last_undegraded":"2023-10-18T12:00:32.445527+0000","last_fullsized":"2023-10-18T12:00:32.445527+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426467+0000","last_change":"2023-10-18T12:00:21.081320+0000","last_active":"2023-10-18T12:00:31.426467+0000","last_peered":"2023-10-18T12:00:31.426467+0000","last_clean":"2023-10-18T12:00:31.426467+0000","last_became_active":"2023-10-18T12:00:21.080987+0000","last_became_peered":"2023-10-18T12:00:21.080987+0000","last_unstale":"2023-10-18T12:00:31.426467+0000","last_undegraded":"2023-10-18T12:00:31.426467+0000","last_fullsized":"2023-10-18T12:00:31.426467+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:10:51.800092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.435922+0000","last_change":"2023-10-18T12:00:32.434522+0000","last_active":"2023-10-18T12:00:32.435922+0000","last_peered":"2023-10-18T12:00:32.435922+0000","last_clean":"2023-10-18T12:00:32.435922+0000","last_became_active":"2023-10-18T12:00:32.434236+0000","last_became_peered":"2023-10-18T12:00:32.434236+0000","last_unstale":"2023-10-18T12:00:32.435922+0000","last_undegraded":"2023-10-18T12:00:32.435922+0000","last_fullsized":"2023-10-18T12:00:32.435922+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:38.936712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.423907+0000","last_change":"2023-10-18T12:00:21.083829+0000","last_active":"2023-10-18T12:00:31.423907+0000","last_peered":"2023-10-18T12:00:31.423907+0000","last_clean":"2023-10-18T12:00:31.423907+0000","last_became_active":"2023-10-18T12:00:21.083554+0000","last_became_peered":"2023-10-18T12:00:21.083554+0000","last_unstale":"2023-10-18T12:00:31.423907+0000","last_undegraded":"2023-10-18T12:00:31.423907+0000","last_fullsized":"2023-10-18T12:00:31.423907+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:34.556666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437583+0000","last_change":"2023-10-18T12:00:32.434340+0000","last_active":"2023-10-18T12:00:32.437583+0000","last_peered":"2023-10-18T12:00:32.437583+0000","last_clean":"2023-10-18T12:00:32.437583+0000","last_became_active":"2023-10-18T12:00:32.434110+0000","last_became_peered":"2023-10-18T12:00:32.434110+0000","last_unstale":"2023-10-18T12:00:32.437583+0000","last_undegraded":"2023-10-18T12:00:32.437583+0000","last_fullsized":"2023-10-18T12:00:32.437583+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:08.260044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427238+0000","last_change":"2023-10-18T12:00:21.081967+0000","last_active":"2023-10-18T12:00:31.427238+0000","last_peered":"2023-10-18T12:00:31.427238+0000","last_clean":"2023-10-18T12:00:31.427238+0000","last_became_active":"2023-10-18T12:00:21.081693+0000","last_became_peered":"2023-10-18T12:00:21.081693+0000","last_unstale":"2023-10-18T12:00:31.427238+0000","last_undegraded":"2023-10-18T12:00:31.427238+0000","last_fullsized":"2023-10-18T12:00:31.427238+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:47:47.036558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.434495+0000","last_change":"2023-10-18T12:00:32.431303+0000","last_active":"2023-10-18T12:00:32.434495+0000","last_peered":"2023-10-18T12:00:32.434495+0000","last_clean":"2023-10-18T12:00:32.434495+0000","last_became_active":"2023-10-18T12:00:32.431062+0000","last_became_peered":"2023-10-18T12:00:32.431062+0000","last_unstale":"2023-10-18T12:00:32.434495+0000","last_undegraded":"2023-10-18T12:00:32.434495+0000","last_fullsized":"2023-10-18T12:00:32.434495+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:14.318910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.425425+0000","last_change":"2023-10-18T12:00:21.087099+0000","last_active":"2023-10-18T12:00:31.425425+0000","last_peered":"2023-10-18T12:00:31.425425+0000","last_clean":"2023-10-18T12:00:31.425425+0000","last_became_active":"2023-10-18T12:00:21.086806+0000","last_became_peered":"2023-10-18T12:00:21.086806+0000","last_unstale":"2023-10-18T12:00:31.425425+0000","last_undegraded":"2023-10-18T12:00:31.425425+0000","last_fullsized":"2023-10-18T12:00:31.425425+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:54:12.691864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448801+0000","last_change":"2023-10-18T12:00:32.446320+0000","last_active":"2023-10-18T12:00:32.448801+0000","last_peered":"2023-10-18T12:00:32.448801+0000","last_clean":"2023-10-18T12:00:32.448801+0000","last_became_active":"2023-10-18T12:00:32.443145+0000","last_became_peered":"2023-10-18T12:00:32.443145+0000","last_unstale":"2023-10-18T12:00:32.448801+0000","last_undegraded":"2023-10-18T12:00:32.448801+0000","last_fullsized":"2023-10-18T12:00:32.448801+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:28:23.492154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419438+0000","last_change":"2023-10-18T12:00:21.084830+0000","last_active":"2023-10-18T12:00:31.419438+0000","last_peered":"2023-10-18T12:00:31.419438+0000","last_clean":"2023-10-18T12:00:31.419438+0000","last_became_active":"2023-10-18T12:00:21.084546+0000","last_became_peered":"2023-10-18T12:00:21.084546+0000","last_unstale":"2023-10-18T12:00:31.419438+0000","last_undegraded":"2023-10-18T12:00:31.419438+0000","last_fullsized":"2023-10-18T12:00:31.419438+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:18.371937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.433081+0000","last_change":"2023-10-18T12:00:32.430679+0000","last_active":"2023-10-18T12:00:32.433081+0000","last_peered":"2023-10-18T12:00:32.433081+0000","last_clean":"2023-10-18T12:00:32.433081+0000","last_became_active":"2023-10-18T12:00:32.428990+0000","last_became_peered":"2023-10-18T12:00:32.428990+0000","last_unstale":"2023-10-18T12:00:32.433081+0000","last_undegraded":"2023-10-18T12:00:32.433081+0000","last_fullsized":"2023-10-18T12:00:32.433081+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:25.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.420926+0000","last_change":"2023-10-18T12:00:21.089403+0000","last_active":"2023-10-18T12:00:31.420926+0000","last_peered":"2023-10-18T12:00:31.420926+0000","last_clean":"2023-10-18T12:00:31.420926+0000","last_became_active":"2023-10-18T12:00:21.089138+0000","last_became_peered":"2023-10-18T12:00:21.089138+0000","last_unstale":"2023-10-18T12:00:31.420926+0000","last_undegraded":"2023-10-18T12:00:31.420926+0000","last_fullsized":"2023-10-18T12:00:31.420926+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:38:59.178484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.431239+0000","last_change":"2023-10-18T12:00:32.429453+0000","last_active":"2023-10-18T12:00:32.431239+0000","last_peered":"2023-10-18T12:00:32.431239+0000","last_clean":"2023-10-18T12:00:32.431239+0000","last_became_active":"2023-10-18T12:00:32.429193+0000","last_became_peered":"2023-10-18T12:00:32.429193+0000","last_unstale":"2023-10-18T12:00:32.431239+0000","last_undegraded":"2023-10-18T12:00:32.431239+0000","last_fullsized":"2023-10-18T12:00:32.431239+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:10.331116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419643+0000","last_change":"2023-10-18T12:00:21.094990+0000","last_active":"2023-10-18T12:00:31.419643+0000","last_peered":"2023-10-18T12:00:31.419643+0000","last_clean":"2023-10-18T12:00:31.419643+0000","last_became_active":"2023-10-18T12:00:21.094239+0000","last_became_peered":"2023-10-18T12:00:21.094239+0000","last_unstale":"2023-10-18T12:00:31.419643+0000","last_undegraded":"2023-10-18T12:00:31.419643+0000","last_fullsized":"2023-10-18T12:00:31.419643+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:22.792096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":595,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575992,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52580,"kb_used_data":3352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319260,"statfs":{"total":96636764160,"available":96582922240,"internally_reserved":0,"allocated":3432448,"data_stored":2037964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.40899999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.60499999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.373},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.46800000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.496},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.61799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575992,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42256,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329584,"statfs":{"total":96636764160,"available":96593494016,"internally_reserved":0,"allocated":2818048,"data_stored":1439415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.53800000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.39600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45200000000000001,"5min":0.43099999999999999,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.87},"last":0.64200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71199999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.82599999999999996},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81100000000000005,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.73999999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77300000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.872,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76100000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85299999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60296,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311544,"statfs":{"total":96636764160,"available":96575021056,"internally_reserved":0,"allocated":2809856,"data_stored":1436021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72199999999999998,"5min":0.92400000000000004,"15min":0.96099999999999997},"last":0.621}]},{"osd":1,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.745,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.52500000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.755,"5min":0.80500000000000005,"15min":1.0349999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.45000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.59299999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":1},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52100000000000002,"15min":0.53800000000000003},"min":{"1min":0.377,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.01},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":1.101,"15min":1.101},"last":0.746},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81499999999999995,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575992,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47832,"kb_used_data":2756,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324008,"statfs":{"total":96636764160,"available":96587784192,"internally_reserved":0,"allocated":2822144,"data_stored":1444658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.22500000000000001},"max":{"1min":0.73899999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.438},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.33600000000000002,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.80200000000000005,"15min":0.81599999999999995},"last":0.504}]},{"osd":1,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.33600000000000002},"max":{"1min":0.86899999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":1.123,"15min":1.123},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.625,"15min":0.621},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.627},"min":{"1min":0.434,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.81999999999999995,"15min":0.83999999999999997},"last":0.60799999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.40300000000000002},"max":{"1min":0.84399999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.51100000000000001,"15min":0.498},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.371},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.34100000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.81499999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.68400000000000005,"5min":1.018,"15min":1.018},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.76100000000000001,"5min":1.004,"15min":1.05},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575992,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47840,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324000,"statfs":{"total":96636764160,"available":96587776000,"internally_reserved":0,"allocated":2830336,"data_stored":1444134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.23200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55000000000000004,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.378},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.41399999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23400000000000001},"max":{"1min":0.56699999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.29499999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41599999999999998,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.44500000000000001},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":1.899,"15min":1.899},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.29299999999999998,"15min":0.28399999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88400000000000001,"5min":1.034,"15min":1.052},"last":0.625},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.522,"15min":1.522},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.89600000000000002,"5min":0.91100000000000003,"15min":1.026},"last":0.66100000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":1.01,"15min":1.01},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67100000000000004,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81799999999999995,"5min":1.129,"15min":1.129},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.59999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575992,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60292,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311548,"statfs":{"total":96636764160,"available":96575025152,"internally_reserved":0,"allocated":2805760,"data_stored":1436055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.216},"max":{"1min":0.56999999999999995,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.314}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.443},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.27700000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65300000000000002,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.087,"15min":1.087},"last":0.46899999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66900000000000004,"15min":0.67400000000000004},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92000000000000004},"last":0.52900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60300,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311540,"statfs":{"total":96636764160,"available":96575016960,"internally_reserved":0,"allocated":2813952,"data_stored":1443986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.44600000000000001,"15min":0.435},"min":{"1min":0.25800000000000001,"5min":0.219,"15min":0.182},"max":{"1min":0.65200000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.217},{"interface":"front","average":{"1min":0.441,"5min":0.379,"15min":0.38},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.29299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.23100000000000001},"max":{"1min":0.56699999999999995,"5min":0.74099999999999999,"15min":0.81000000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40699999999999997},"min":{"1min":0.34699999999999998,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.57699999999999996,"5min":0.68999999999999995,"15min":0.877},"last":0.251}]},{"osd":3,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.92000000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58799999999999997,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.46800000000000003,"5min":0.31900000000000001,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":0.82899999999999996,"15min":0.98599999999999999},"last":0.317},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.62,"15min":0.58599999999999997},"min":{"1min":0.52500000000000002,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.77900000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.44800000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.29599999999999999},"max":{"1min":0.83999999999999997,"5min":0.87,"15min":1.004},"last":0.495},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.51600000000000001,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.76100000000000001,"5min":0.91500000000000004,"15min":1.05},"last":0.376}]},{"osd":6,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.53400000000000003,"5min":0.379,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.46300000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.39400000000000002},"max":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575991,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42824,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329016,"statfs":{"total":96636764160,"available":96592912384,"internally_reserved":0,"allocated":3399680,"data_stored":2026423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54700000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.32300000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61899999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28899999999999998,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.66800000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.25700000000000001},"max":{"1min":0.62,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.56899999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39800000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96999999999999997,"15min":0.997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.876,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88200000000000001,"5min":0.95799999999999996,"15min":0.96399999999999997},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66900000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":1.0089999999999999,"15min":1.03},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.45800000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65800000000000003,"15min":0.68100000000000005},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.82699999999999996,"5min":0.997,"15min":0.997},"last":0.57999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:00:35.388 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:00:35.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 -- 172.21.15.125:0/2350621806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed015b640 msgr2=0x7faed00cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 --2- 172.21.15.125:0/2350621806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed015b640 0x7faed00cf850 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7faed00894b0 tx=0x7faec402bda0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 -- 172.21.15.125:0/2350621806 shutdown_connections 2023-10-18T12:00:35.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 --2- 172.21.15.125:0/2350621806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed015b640 0x7faed00cf850 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 --2- 172.21.15.125:0/2350621806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faed00b0b10 0x7faed00b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 --2- 172.21.15.125:0/2350621806 >> v2:172.21.15.139:3300/0 conn(0x7faed015b260 0x7faed00cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 -- 172.21.15.125:0/2350621806 >> 172.21.15.125:0/2350621806 conn(0x7faed009a030 msgr2=0x7faed00d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:35.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 -- 172.21.15.125:0/2350621806 shutdown_connections 2023-10-18T12:00:35.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.517+0000 7faed52bb640 1 -- 172.21.15.125:0/2350621806 wait complete. 2023-10-18T12:00:35.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.518+0000 7faed52bb640 1 Processor -- start 2023-10-18T12:00:35.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.518+0000 7faed52bb640 1 -- start start 2023-10-18T12:00:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faed52bb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed00b0b10 0x7faed015e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faed52bb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faed015b260 0x7faed014cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faed52bb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faed015b640 0x7faed014d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faed52bb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b4f60 con 0x7faed00b0b10 2023-10-18T12:00:35.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faed52bb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed014d540 con 0x7faed015b640 2023-10-18T12:00:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faed52bb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b4350 con 0x7faed015b260 2023-10-18T12:00:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faeceffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed00b0b10 0x7faed015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faeceffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed00b0b10 0x7faed015e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32904/0 (socket says 172.21.15.125:32904) 2023-10-18T12:00:35.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faeceffd640 1 -- 172.21.15.125:0/3800278003 learned_addr learned my addr 172.21.15.125:0/3800278003 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faeceffd640 1 -- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faed015b260 msgr2=0x7faed014cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faeceffd640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faed015b260 0x7faed014cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faecf7fe640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faed015b640 0x7faed014d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faeceffd640 1 -- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faed015b640 msgr2=0x7faed014d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faeceffd640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faed015b640 0x7faed014d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.519+0000 7faeceffd640 1 -- 172.21.15.125:0/3800278003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4009560 con 0x7faed00b0b10 2023-10-18T12:00:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.520+0000 7faeceffd640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed00b0b10 0x7faed015e010 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7faec000db00 tx=0x7faec000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.520+0000 7faeb77fe640 1 -- 172.21.15.125:0/3800278003 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec000bea0 con 0x7faed00b0b10 2023-10-18T12:00:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.520+0000 7faeb77fe640 1 -- 172.21.15.125:0/3800278003 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faec0003a60 con 0x7faed00b0b10 2023-10-18T12:00:35.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.520+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed00b4630 con 0x7faed00b0b10 2023-10-18T12:00:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.520+0000 7faeb77fe640 1 -- 172.21.15.125:0/3800278003 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0003e70 con 0x7faed00b0b10 2023-10-18T12:00:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.520+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed00b4bf0 con 0x7faed00b0b10 2023-10-18T12:00:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.521+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed00675f0 con 0x7faed00b0b10 2023-10-18T12:00:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.521+0000 7faeb77fe640 1 -- 172.21.15.125:0/3800278003 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faec0004020 con 0x7faed00b0b10 2023-10-18T12:00:35.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.522+0000 7faeb77fe640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faebc06af60 0x7faebc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.522+0000 7faeb77fe640 1 -- 172.21.15.125:0/3800278003 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7faec008a620 con 0x7faed00b0b10 2023-10-18T12:00:35.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.522+0000 7faece7fc640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faebc06af60 0x7faebc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.522+0000 7faece7fc640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faebc06af60 0x7faebc06d420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faec4035000 tx=0x7faec40046c0 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:35.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.526+0000 7faeb77fe640 1 -- 172.21.15.125:0/3800278003 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faec0054f50 con 0x7faed00b0b10 2023-10-18T12:00:35.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.688+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 --> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faed00ccb50 con 0x7faebc06af60 2023-10-18T12:00:35.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.701+0000 7faeb77fe640 1 -- 172.21.15.125:0/3800278003 <== mgr.8786 v2:172.21.15.139:6832/2362843834 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529045 (secure 0 0 0) 0x7faed00ccb50 con 0x7faebc06af60 2023-10-18T12:00:35.709 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:35.713 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:00:35.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faebc06af60 msgr2=0x7faebc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faebc06af60 0x7faebc06d420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faec4035000 tx=0x7faec40046c0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed00b0b10 msgr2=0x7faed015e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed00b0b10 0x7faed015e010 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7faec000db00 tx=0x7faec000b260 comp rx=0 tx=0).stop 2023-10-18T12:00:35.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 shutdown_connections 2023-10-18T12:00:35.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7faebc06af60 0x7faebc06d420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faed015b640 0x7faed014d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faed015b260 0x7faed014cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 --2- 172.21.15.125:0/3800278003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faed00b0b10 0x7faed015e010 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.704+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 >> 172.21.15.125:0/3800278003 conn(0x7faed009a030 msgr2=0x7faed00b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:35.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.705+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 shutdown_connections 2023-10-18T12:00:35.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.705+0000 7faed52bb640 1 -- 172.21.15.125:0/3800278003 wait complete. 2023-10-18T12:00:35.718 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-10-18T12:00:34.802697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596444,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":686,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":436,"ondisk_log_size":436,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":214,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":414220,"kb_used_data":23176,"kb_used_omap":12,"kb_used_meta":390899,"kb_avail":754560500,"statfs":{"total":773094113280,"available":772669952000,"internally_reserved":0,"allocated":23732224,"data_stored":12708656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":400281144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3225,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.395041"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.443713+0000","last_change":"2023-10-18T12:00:32.443239+0000","last_active":"2023-10-18T12:00:32.443713+0000","last_peered":"2023-10-18T12:00:32.443713+0000","last_clean":"2023-10-18T12:00:32.443713+0000","last_became_active":"2023-10-18T12:00:32.442955+0000","last_became_peered":"2023-10-18T12:00:32.442955+0000","last_unstale":"2023-10-18T12:00:32.443713+0000","last_undegraded":"2023-10-18T12:00:32.443713+0000","last_fullsized":"2023-10-18T12:00:32.443713+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:09:26.856452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"208'357","reported_seq":1236,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.425109+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:00:31.425109+0000","last_peered":"2023-10-18T12:00:31.425109+0000","last_clean":"2023-10-18T12:00:31.425109+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:00:31.425109+0000","last_undegraded":"2023-10-18T12:00:31.425109+0000","last_fullsized":"2023-10-18T12:00:31.425109+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":595,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417915+0000","last_change":"2023-10-18T12:00:20.057601+0000","last_active":"2023-10-18T12:00:31.417915+0000","last_peered":"2023-10-18T12:00:31.417915+0000","last_clean":"2023-10-18T12:00:31.417915+0000","last_became_active":"2023-10-18T12:00:20.057294+0000","last_became_peered":"2023-10-18T12:00:20.057294+0000","last_unstale":"2023-10-18T12:00:31.417915+0000","last_undegraded":"2023-10-18T12:00:31.417915+0000","last_fullsized":"2023-10-18T12:00:31.417915+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:43.803662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418996+0000","last_change":"2023-10-18T12:00:21.091052+0000","last_active":"2023-10-18T12:00:31.418996+0000","last_peered":"2023-10-18T12:00:31.418996+0000","last_clean":"2023-10-18T12:00:31.418996+0000","last_became_active":"2023-10-18T12:00:21.090785+0000","last_became_peered":"2023-10-18T12:00:21.090785+0000","last_unstale":"2023-10-18T12:00:31.418996+0000","last_undegraded":"2023-10-18T12:00:31.418996+0000","last_fullsized":"2023-10-18T12:00:31.418996+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:09.229786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445679+0000","last_change":"2023-10-18T12:00:31.433776+0000","last_active":"2023-10-18T12:00:32.445679+0000","last_peered":"2023-10-18T12:00:32.445679+0000","last_clean":"2023-10-18T12:00:32.445679+0000","last_became_active":"2023-10-18T12:00:31.433509+0000","last_became_peered":"2023-10-18T12:00:31.433509+0000","last_unstale":"2023-10-18T12:00:32.445679+0000","last_undegraded":"2023-10-18T12:00:32.445679+0000","last_fullsized":"2023-10-18T12:00:32.445679+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:58.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445783+0000","last_change":"2023-10-18T12:00:32.444780+0000","last_active":"2023-10-18T12:00:32.445783+0000","last_peered":"2023-10-18T12:00:32.445783+0000","last_clean":"2023-10-18T12:00:32.445783+0000","last_became_active":"2023-10-18T12:00:32.443183+0000","last_became_peered":"2023-10-18T12:00:32.443183+0000","last_unstale":"2023-10-18T12:00:32.445783+0000","last_undegraded":"2023-10-18T12:00:32.445783+0000","last_fullsized":"2023-10-18T12:00:32.445783+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:52.458791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426327+0000","last_change":"2023-10-18T12:00:20.066813+0000","last_active":"2023-10-18T12:00:31.426327+0000","last_peered":"2023-10-18T12:00:31.426327+0000","last_clean":"2023-10-18T12:00:31.426327+0000","last_became_active":"2023-10-18T12:00:20.066552+0000","last_became_peered":"2023-10-18T12:00:20.066552+0000","last_unstale":"2023-10-18T12:00:31.426327+0000","last_undegraded":"2023-10-18T12:00:31.426327+0000","last_fullsized":"2023-10-18T12:00:31.426327+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:22:59.208578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.420097+0000","last_change":"2023-10-18T12:00:21.091679+0000","last_active":"2023-10-18T12:00:31.420097+0000","last_peered":"2023-10-18T12:00:31.420097+0000","last_clean":"2023-10-18T12:00:31.420097+0000","last_became_active":"2023-10-18T12:00:21.091412+0000","last_became_peered":"2023-10-18T12:00:21.091412+0000","last_unstale":"2023-10-18T12:00:31.420097+0000","last_undegraded":"2023-10-18T12:00:31.420097+0000","last_fullsized":"2023-10-18T12:00:31.420097+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:27.574111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.421182+0000","last_change":"2023-10-18T12:00:31.425160+0000","last_active":"2023-10-18T12:00:32.421182+0000","last_peered":"2023-10-18T12:00:32.421182+0000","last_clean":"2023-10-18T12:00:32.421182+0000","last_became_active":"2023-10-18T12:00:31.424835+0000","last_became_peered":"2023-10-18T12:00:31.424835+0000","last_unstale":"2023-10-18T12:00:32.421182+0000","last_undegraded":"2023-10-18T12:00:32.421182+0000","last_fullsized":"2023-10-18T12:00:32.421182+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:12.318416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437163+0000","last_change":"2023-10-18T12:00:32.435087+0000","last_active":"2023-10-18T12:00:32.437163+0000","last_peered":"2023-10-18T12:00:32.437163+0000","last_clean":"2023-10-18T12:00:32.437163+0000","last_became_active":"2023-10-18T12:00:32.434844+0000","last_became_peered":"2023-10-18T12:00:32.434844+0000","last_unstale":"2023-10-18T12:00:32.437163+0000","last_undegraded":"2023-10-18T12:00:32.437163+0000","last_fullsized":"2023-10-18T12:00:32.437163+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:28:23.492154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419263+0000","last_change":"2023-10-18T12:00:20.065817+0000","last_active":"2023-10-18T12:00:31.419263+0000","last_peered":"2023-10-18T12:00:31.419263+0000","last_clean":"2023-10-18T12:00:31.419263+0000","last_became_active":"2023-10-18T12:00:20.065536+0000","last_became_peered":"2023-10-18T12:00:20.065536+0000","last_unstale":"2023-10-18T12:00:31.419263+0000","last_undegraded":"2023-10-18T12:00:31.419263+0000","last_fullsized":"2023-10-18T12:00:31.419263+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:57.379803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.423874+0000","last_change":"2023-10-18T12:00:21.089842+0000","last_active":"2023-10-18T12:00:31.423874+0000","last_peered":"2023-10-18T12:00:31.423874+0000","last_clean":"2023-10-18T12:00:31.423874+0000","last_became_active":"2023-10-18T12:00:21.089584+0000","last_became_peered":"2023-10-18T12:00:21.089584+0000","last_unstale":"2023-10-18T12:00:31.423874+0000","last_undegraded":"2023-10-18T12:00:31.423874+0000","last_fullsized":"2023-10-18T12:00:31.423874+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:36.596128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.451711+0000","last_change":"2023-10-18T12:00:31.436826+0000","last_active":"2023-10-18T12:00:32.451711+0000","last_peered":"2023-10-18T12:00:32.451711+0000","last_clean":"2023-10-18T12:00:32.451711+0000","last_became_active":"2023-10-18T12:00:31.436570+0000","last_became_peered":"2023-10-18T12:00:31.436570+0000","last_unstale":"2023-10-18T12:00:32.451711+0000","last_undegraded":"2023-10-18T12:00:32.451711+0000","last_fullsized":"2023-10-18T12:00:32.451711+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:07:55.443591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437028+0000","last_change":"2023-10-18T12:00:32.436481+0000","last_active":"2023-10-18T12:00:32.437028+0000","last_peered":"2023-10-18T12:00:32.437028+0000","last_clean":"2023-10-18T12:00:32.437028+0000","last_became_active":"2023-10-18T12:00:32.436213+0000","last_became_peered":"2023-10-18T12:00:32.436213+0000","last_unstale":"2023-10-18T12:00:32.437028+0000","last_undegraded":"2023-10-18T12:00:32.437028+0000","last_fullsized":"2023-10-18T12:00:32.437028+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:24:20.363215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418008+0000","last_change":"2023-10-18T12:00:20.060656+0000","last_active":"2023-10-18T12:00:31.418008+0000","last_peered":"2023-10-18T12:00:31.418008+0000","last_clean":"2023-10-18T12:00:31.418008+0000","last_became_active":"2023-10-18T12:00:20.060413+0000","last_became_peered":"2023-10-18T12:00:20.060413+0000","last_unstale":"2023-10-18T12:00:31.418008+0000","last_undegraded":"2023-10-18T12:00:31.418008+0000","last_fullsized":"2023-10-18T12:00:31.418008+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:32.716291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424128+0000","last_change":"2023-10-18T12:00:21.088423+0000","last_active":"2023-10-18T12:00:31.424128+0000","last_peered":"2023-10-18T12:00:31.424128+0000","last_clean":"2023-10-18T12:00:31.424128+0000","last_became_active":"2023-10-18T12:00:21.087811+0000","last_became_peered":"2023-10-18T12:00:21.087811+0000","last_unstale":"2023-10-18T12:00:31.424128+0000","last_undegraded":"2023-10-18T12:00:31.424128+0000","last_fullsized":"2023-10-18T12:00:31.424128+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:03.802023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445154+0000","last_change":"2023-10-18T12:00:31.434761+0000","last_active":"2023-10-18T12:00:32.445154+0000","last_peered":"2023-10-18T12:00:32.445154+0000","last_clean":"2023-10-18T12:00:32.445154+0000","last_became_active":"2023-10-18T12:00:31.434482+0000","last_became_peered":"2023-10-18T12:00:31.434482+0000","last_unstale":"2023-10-18T12:00:32.445154+0000","last_undegraded":"2023-10-18T12:00:32.445154+0000","last_fullsized":"2023-10-18T12:00:32.445154+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:49.906829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.452025+0000","last_change":"2023-10-18T12:00:32.451037+0000","last_active":"2023-10-18T12:00:32.452025+0000","last_peered":"2023-10-18T12:00:32.452025+0000","last_clean":"2023-10-18T12:00:32.452025+0000","last_became_active":"2023-10-18T12:00:32.450723+0000","last_became_peered":"2023-10-18T12:00:32.450723+0000","last_unstale":"2023-10-18T12:00:32.452025+0000","last_undegraded":"2023-10-18T12:00:32.452025+0000","last_fullsized":"2023-10-18T12:00:32.452025+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:56.364975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430868+0000","last_change":"2023-10-18T12:00:20.066439+0000","last_active":"2023-10-18T12:00:31.430868+0000","last_peered":"2023-10-18T12:00:31.430868+0000","last_clean":"2023-10-18T12:00:31.430868+0000","last_became_active":"2023-10-18T12:00:20.066204+0000","last_became_peered":"2023-10-18T12:00:20.066204+0000","last_unstale":"2023-10-18T12:00:31.430868+0000","last_undegraded":"2023-10-18T12:00:31.430868+0000","last_fullsized":"2023-10-18T12:00:31.430868+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:36.690391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.431486+0000","last_change":"2023-10-18T12:00:21.090829+0000","last_active":"2023-10-18T12:00:31.431486+0000","last_peered":"2023-10-18T12:00:31.431486+0000","last_clean":"2023-10-18T12:00:31.431486+0000","last_became_active":"2023-10-18T12:00:21.090013+0000","last_became_peered":"2023-10-18T12:00:21.090013+0000","last_unstale":"2023-10-18T12:00:31.431486+0000","last_undegraded":"2023-10-18T12:00:31.431486+0000","last_fullsized":"2023-10-18T12:00:31.431486+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:14.286779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.421622+0000","last_change":"2023-10-18T12:00:31.425068+0000","last_active":"2023-10-18T12:00:32.421622+0000","last_peered":"2023-10-18T12:00:32.421622+0000","last_clean":"2023-10-18T12:00:32.421622+0000","last_became_active":"2023-10-18T12:00:31.424804+0000","last_became_peered":"2023-10-18T12:00:31.424804+0000","last_unstale":"2023-10-18T12:00:32.421622+0000","last_undegraded":"2023-10-18T12:00:32.421622+0000","last_fullsized":"2023-10-18T12:00:32.421622+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:01.434053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430370+0000","last_change":"2023-10-18T12:00:32.428936+0000","last_active":"2023-10-18T12:00:32.430370+0000","last_peered":"2023-10-18T12:00:32.430370+0000","last_clean":"2023-10-18T12:00:32.430370+0000","last_became_active":"2023-10-18T12:00:32.428681+0000","last_became_peered":"2023-10-18T12:00:32.428681+0000","last_unstale":"2023-10-18T12:00:32.430370+0000","last_undegraded":"2023-10-18T12:00:32.430370+0000","last_fullsized":"2023-10-18T12:00:32.430370+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:19.695994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419333+0000","last_change":"2023-10-18T12:00:20.059094+0000","last_active":"2023-10-18T12:00:31.419333+0000","last_peered":"2023-10-18T12:00:31.419333+0000","last_clean":"2023-10-18T12:00:31.419333+0000","last_became_active":"2023-10-18T12:00:20.058785+0000","last_became_peered":"2023-10-18T12:00:20.058785+0000","last_unstale":"2023-10-18T12:00:31.419333+0000","last_undegraded":"2023-10-18T12:00:31.419333+0000","last_fullsized":"2023-10-18T12:00:31.419333+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:32.780678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418904+0000","last_change":"2023-10-18T12:00:21.086339+0000","last_active":"2023-10-18T12:00:31.418904+0000","last_peered":"2023-10-18T12:00:31.418904+0000","last_clean":"2023-10-18T12:00:31.418904+0000","last_became_active":"2023-10-18T12:00:21.086056+0000","last_became_peered":"2023-10-18T12:00:21.086056+0000","last_unstale":"2023-10-18T12:00:31.418904+0000","last_undegraded":"2023-10-18T12:00:31.418904+0000","last_fullsized":"2023-10-18T12:00:31.418904+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:39.387888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.425680+0000","last_change":"2023-10-18T12:00:31.420704+0000","last_active":"2023-10-18T12:00:32.425680+0000","last_peered":"2023-10-18T12:00:32.425680+0000","last_clean":"2023-10-18T12:00:32.425680+0000","last_became_active":"2023-10-18T12:00:31.420253+0000","last_became_peered":"2023-10-18T12:00:31.420253+0000","last_unstale":"2023-10-18T12:00:32.425680+0000","last_undegraded":"2023-10-18T12:00:32.425680+0000","last_fullsized":"2023-10-18T12:00:32.425680+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:38.148897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448971+0000","last_change":"2023-10-18T12:00:32.448482+0000","last_active":"2023-10-18T12:00:32.448971+0000","last_peered":"2023-10-18T12:00:32.448971+0000","last_clean":"2023-10-18T12:00:32.448971+0000","last_became_active":"2023-10-18T12:00:32.448207+0000","last_became_peered":"2023-10-18T12:00:32.448207+0000","last_unstale":"2023-10-18T12:00:32.448971+0000","last_undegraded":"2023-10-18T12:00:32.448971+0000","last_fullsized":"2023-10-18T12:00:32.448971+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:52.700336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418069+0000","last_change":"2023-10-18T12:00:20.059978+0000","last_active":"2023-10-18T12:00:31.418069+0000","last_peered":"2023-10-18T12:00:31.418069+0000","last_clean":"2023-10-18T12:00:31.418069+0000","last_became_active":"2023-10-18T12:00:20.059686+0000","last_became_peered":"2023-10-18T12:00:20.059686+0000","last_unstale":"2023-10-18T12:00:31.418069+0000","last_undegraded":"2023-10-18T12:00:31.418069+0000","last_fullsized":"2023-10-18T12:00:31.418069+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:53.027447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417735+0000","last_change":"2023-10-18T12:00:21.076847+0000","last_active":"2023-10-18T12:00:31.417735+0000","last_peered":"2023-10-18T12:00:31.417735+0000","last_clean":"2023-10-18T12:00:31.417735+0000","last_became_active":"2023-10-18T12:00:21.076552+0000","last_became_peered":"2023-10-18T12:00:21.076552+0000","last_unstale":"2023-10-18T12:00:31.417735+0000","last_undegraded":"2023-10-18T12:00:31.417735+0000","last_fullsized":"2023-10-18T12:00:31.417735+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:36.596128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.441498+0000","last_change":"2023-10-18T12:00:31.435045+0000","last_active":"2023-10-18T12:00:32.441498+0000","last_peered":"2023-10-18T12:00:32.441498+0000","last_clean":"2023-10-18T12:00:32.441498+0000","last_became_active":"2023-10-18T12:00:31.434797+0000","last_became_peered":"2023-10-18T12:00:31.434797+0000","last_unstale":"2023-10-18T12:00:32.441498+0000","last_undegraded":"2023-10-18T12:00:32.441498+0000","last_fullsized":"2023-10-18T12:00:32.441498+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:49:37.618423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.449206+0000","last_change":"2023-10-18T12:00:32.446096+0000","last_active":"2023-10-18T12:00:32.449206+0000","last_peered":"2023-10-18T12:00:32.449206+0000","last_clean":"2023-10-18T12:00:32.449206+0000","last_became_active":"2023-10-18T12:00:32.445755+0000","last_became_peered":"2023-10-18T12:00:32.445755+0000","last_unstale":"2023-10-18T12:00:32.449206+0000","last_undegraded":"2023-10-18T12:00:32.449206+0000","last_fullsized":"2023-10-18T12:00:32.449206+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:25.786603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"221'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.432460+0000","last_change":"2023-10-18T12:00:20.063289+0000","last_active":"2023-10-18T12:00:31.432460+0000","last_peered":"2023-10-18T12:00:31.432460+0000","last_clean":"2023-10-18T12:00:31.432460+0000","last_became_active":"2023-10-18T12:00:20.063024+0000","last_became_peered":"2023-10-18T12:00:20.063024+0000","last_unstale":"2023-10-18T12:00:31.432460+0000","last_undegraded":"2023-10-18T12:00:31.432460+0000","last_fullsized":"2023-10-18T12:00:31.432460+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:54.325350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418948+0000","last_change":"2023-10-18T12:00:21.092403+0000","last_active":"2023-10-18T12:00:31.418948+0000","last_peered":"2023-10-18T12:00:31.418948+0000","last_clean":"2023-10-18T12:00:31.418948+0000","last_became_active":"2023-10-18T12:00:21.092167+0000","last_became_peered":"2023-10-18T12:00:21.092167+0000","last_unstale":"2023-10-18T12:00:31.418948+0000","last_undegraded":"2023-10-18T12:00:31.418948+0000","last_fullsized":"2023-10-18T12:00:31.418948+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:45:11.154204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.438116+0000","last_change":"2023-10-18T12:00:31.438173+0000","last_active":"2023-10-18T12:00:32.438116+0000","last_peered":"2023-10-18T12:00:32.438116+0000","last_clean":"2023-10-18T12:00:32.438116+0000","last_became_active":"2023-10-18T12:00:31.437914+0000","last_became_peered":"2023-10-18T12:00:31.437914+0000","last_unstale":"2023-10-18T12:00:32.438116+0000","last_undegraded":"2023-10-18T12:00:32.438116+0000","last_fullsized":"2023-10-18T12:00:32.438116+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:51:43.375691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429327+0000","last_change":"2023-10-18T12:00:32.427052+0000","last_active":"2023-10-18T12:00:32.429327+0000","last_peered":"2023-10-18T12:00:32.429327+0000","last_clean":"2023-10-18T12:00:32.429327+0000","last_became_active":"2023-10-18T12:00:32.426788+0000","last_became_peered":"2023-10-18T12:00:32.426788+0000","last_unstale":"2023-10-18T12:00:32.429327+0000","last_undegraded":"2023-10-18T12:00:32.429327+0000","last_fullsized":"2023-10-18T12:00:32.429327+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:14.155039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418442+0000","last_change":"2023-10-18T12:00:20.063535+0000","last_active":"2023-10-18T12:00:31.418442+0000","last_peered":"2023-10-18T12:00:31.418442+0000","last_clean":"2023-10-18T12:00:31.418442+0000","last_became_active":"2023-10-18T12:00:20.063259+0000","last_became_peered":"2023-10-18T12:00:20.063259+0000","last_unstale":"2023-10-18T12:00:31.418442+0000","last_undegraded":"2023-10-18T12:00:31.418442+0000","last_fullsized":"2023-10-18T12:00:31.418442+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:54:11.686164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.429871+0000","last_change":"2023-10-18T12:00:21.092122+0000","last_active":"2023-10-18T12:00:31.429871+0000","last_peered":"2023-10-18T12:00:31.429871+0000","last_clean":"2023-10-18T12:00:31.429871+0000","last_became_active":"2023-10-18T12:00:21.091845+0000","last_became_peered":"2023-10-18T12:00:21.091845+0000","last_unstale":"2023-10-18T12:00:31.429871+0000","last_undegraded":"2023-10-18T12:00:31.429871+0000","last_fullsized":"2023-10-18T12:00:31.429871+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:31.536626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.428352+0000","last_change":"2023-10-18T12:00:31.430651+0000","last_active":"2023-10-18T12:00:32.428352+0000","last_peered":"2023-10-18T12:00:32.428352+0000","last_clean":"2023-10-18T12:00:32.428352+0000","last_became_active":"2023-10-18T12:00:31.430249+0000","last_became_peered":"2023-10-18T12:00:31.430249+0000","last_unstale":"2023-10-18T12:00:32.428352+0000","last_undegraded":"2023-10-18T12:00:32.428352+0000","last_fullsized":"2023-10-18T12:00:32.428352+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:47:24.776087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445801+0000","last_change":"2023-10-18T12:00:32.444989+0000","last_active":"2023-10-18T12:00:32.445801+0000","last_peered":"2023-10-18T12:00:32.445801+0000","last_clean":"2023-10-18T12:00:32.445801+0000","last_became_active":"2023-10-18T12:00:32.443013+0000","last_became_peered":"2023-10-18T12:00:32.443013+0000","last_unstale":"2023-10-18T12:00:32.445801+0000","last_undegraded":"2023-10-18T12:00:32.445801+0000","last_fullsized":"2023-10-18T12:00:32.445801+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:04.205911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418744+0000","last_change":"2023-10-18T12:00:20.058023+0000","last_active":"2023-10-18T12:00:31.418744+0000","last_peered":"2023-10-18T12:00:31.418744+0000","last_clean":"2023-10-18T12:00:31.418744+0000","last_became_active":"2023-10-18T12:00:20.057731+0000","last_became_peered":"2023-10-18T12:00:20.057731+0000","last_unstale":"2023-10-18T12:00:31.418744+0000","last_undegraded":"2023-10-18T12:00:31.418744+0000","last_fullsized":"2023-10-18T12:00:31.418744+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:28.253477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.433622+0000","last_change":"2023-10-18T12:00:21.094111+0000","last_active":"2023-10-18T12:00:31.433622+0000","last_peered":"2023-10-18T12:00:31.433622+0000","last_clean":"2023-10-18T12:00:31.433622+0000","last_became_active":"2023-10-18T12:00:21.093838+0000","last_became_peered":"2023-10-18T12:00:21.093838+0000","last_unstale":"2023-10-18T12:00:31.433622+0000","last_undegraded":"2023-10-18T12:00:31.433622+0000","last_fullsized":"2023-10-18T12:00:31.433622+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:57:22.028468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437844+0000","last_change":"2023-10-18T12:00:31.434488+0000","last_active":"2023-10-18T12:00:32.437844+0000","last_peered":"2023-10-18T12:00:32.437844+0000","last_clean":"2023-10-18T12:00:32.437844+0000","last_became_active":"2023-10-18T12:00:31.434270+0000","last_became_peered":"2023-10-18T12:00:31.434270+0000","last_unstale":"2023-10-18T12:00:32.437844+0000","last_undegraded":"2023-10-18T12:00:32.437844+0000","last_fullsized":"2023-10-18T12:00:32.437844+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:31.763503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.447246+0000","last_change":"2023-10-18T12:00:32.446941+0000","last_active":"2023-10-18T12:00:32.447246+0000","last_peered":"2023-10-18T12:00:32.447246+0000","last_clean":"2023-10-18T12:00:32.447246+0000","last_became_active":"2023-10-18T12:00:32.446705+0000","last_became_peered":"2023-10-18T12:00:32.446705+0000","last_unstale":"2023-10-18T12:00:32.447246+0000","last_undegraded":"2023-10-18T12:00:32.447246+0000","last_fullsized":"2023-10-18T12:00:32.447246+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:24.527236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418177+0000","last_change":"2023-10-18T12:00:20.063488+0000","last_active":"2023-10-18T12:00:31.418177+0000","last_peered":"2023-10-18T12:00:31.418177+0000","last_clean":"2023-10-18T12:00:31.418177+0000","last_became_active":"2023-10-18T12:00:20.063220+0000","last_became_peered":"2023-10-18T12:00:20.063220+0000","last_unstale":"2023-10-18T12:00:31.418177+0000","last_undegraded":"2023-10-18T12:00:31.418177+0000","last_fullsized":"2023-10-18T12:00:31.418177+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:41:32.615922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418181+0000","last_change":"2023-10-18T12:00:21.087602+0000","last_active":"2023-10-18T12:00:31.418181+0000","last_peered":"2023-10-18T12:00:31.418181+0000","last_clean":"2023-10-18T12:00:31.418181+0000","last_became_active":"2023-10-18T12:00:21.087140+0000","last_became_peered":"2023-10-18T12:00:21.087140+0000","last_unstale":"2023-10-18T12:00:31.418181+0000","last_undegraded":"2023-10-18T12:00:31.418181+0000","last_fullsized":"2023-10-18T12:00:31.418181+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:57.852277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.449467+0000","last_change":"2023-10-18T12:00:31.436777+0000","last_active":"2023-10-18T12:00:32.449467+0000","last_peered":"2023-10-18T12:00:32.449467+0000","last_clean":"2023-10-18T12:00:32.449467+0000","last_became_active":"2023-10-18T12:00:31.436495+0000","last_became_peered":"2023-10-18T12:00:31.436495+0000","last_unstale":"2023-10-18T12:00:32.449467+0000","last_undegraded":"2023-10-18T12:00:32.449467+0000","last_fullsized":"2023-10-18T12:00:32.449467+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:17:38.007561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448900+0000","last_change":"2023-10-18T12:00:32.447082+0000","last_active":"2023-10-18T12:00:32.448900+0000","last_peered":"2023-10-18T12:00:32.448900+0000","last_clean":"2023-10-18T12:00:32.448900+0000","last_became_active":"2023-10-18T12:00:32.442502+0000","last_became_peered":"2023-10-18T12:00:32.442502+0000","last_unstale":"2023-10-18T12:00:32.448900+0000","last_undegraded":"2023-10-18T12:00:32.448900+0000","last_fullsized":"2023-10-18T12:00:32.448900+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:15.903077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421491+0000","last_change":"2023-10-18T12:00:20.062486+0000","last_active":"2023-10-18T12:00:31.421491+0000","last_peered":"2023-10-18T12:00:31.421491+0000","last_clean":"2023-10-18T12:00:31.421491+0000","last_became_active":"2023-10-18T12:00:20.062205+0000","last_became_peered":"2023-10-18T12:00:20.062205+0000","last_unstale":"2023-10-18T12:00:31.421491+0000","last_undegraded":"2023-10-18T12:00:31.421491+0000","last_fullsized":"2023-10-18T12:00:31.421491+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:01.124233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424118+0000","last_change":"2023-10-18T12:00:21.092605+0000","last_active":"2023-10-18T12:00:31.424118+0000","last_peered":"2023-10-18T12:00:31.424118+0000","last_clean":"2023-10-18T12:00:31.424118+0000","last_became_active":"2023-10-18T12:00:21.092319+0000","last_became_peered":"2023-10-18T12:00:21.092319+0000","last_unstale":"2023-10-18T12:00:31.424118+0000","last_undegraded":"2023-10-18T12:00:31.424118+0000","last_fullsized":"2023-10-18T12:00:31.424118+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:44.809362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445018+0000","last_change":"2023-10-18T12:00:31.436132+0000","last_active":"2023-10-18T12:00:32.445018+0000","last_peered":"2023-10-18T12:00:32.445018+0000","last_clean":"2023-10-18T12:00:32.445018+0000","last_became_active":"2023-10-18T12:00:31.435398+0000","last_became_peered":"2023-10-18T12:00:31.435398+0000","last_unstale":"2023-10-18T12:00:32.445018+0000","last_undegraded":"2023-10-18T12:00:32.445018+0000","last_fullsized":"2023-10-18T12:00:32.445018+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:46:39.602864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.428503+0000","last_change":"2023-10-18T12:00:32.425636+0000","last_active":"2023-10-18T12:00:32.428503+0000","last_peered":"2023-10-18T12:00:32.428503+0000","last_clean":"2023-10-18T12:00:32.428503+0000","last_became_active":"2023-10-18T12:00:32.425355+0000","last_became_peered":"2023-10-18T12:00:32.425355+0000","last_unstale":"2023-10-18T12:00:32.428503+0000","last_undegraded":"2023-10-18T12:00:32.428503+0000","last_fullsized":"2023-10-18T12:00:32.428503+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:43:38.016035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427469+0000","last_change":"2023-10-18T12:00:20.058937+0000","last_active":"2023-10-18T12:00:31.427469+0000","last_peered":"2023-10-18T12:00:31.427469+0000","last_clean":"2023-10-18T12:00:31.427469+0000","last_became_active":"2023-10-18T12:00:20.058685+0000","last_became_peered":"2023-10-18T12:00:20.058685+0000","last_unstale":"2023-10-18T12:00:31.427469+0000","last_undegraded":"2023-10-18T12:00:31.427469+0000","last_fullsized":"2023-10-18T12:00:31.427469+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:07.432693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421695+0000","last_change":"2023-10-18T12:00:21.089358+0000","last_active":"2023-10-18T12:00:31.421695+0000","last_peered":"2023-10-18T12:00:31.421695+0000","last_clean":"2023-10-18T12:00:31.421695+0000","last_became_active":"2023-10-18T12:00:21.089109+0000","last_became_peered":"2023-10-18T12:00:21.089109+0000","last_unstale":"2023-10-18T12:00:31.421695+0000","last_undegraded":"2023-10-18T12:00:31.421695+0000","last_fullsized":"2023-10-18T12:00:31.421695+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:47:14.419423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454642+0000","last_change":"2023-10-18T12:00:31.431647+0000","last_active":"2023-10-18T12:00:32.454642+0000","last_peered":"2023-10-18T12:00:32.454642+0000","last_clean":"2023-10-18T12:00:32.454642+0000","last_became_active":"2023-10-18T12:00:31.431288+0000","last_became_peered":"2023-10-18T12:00:31.431288+0000","last_unstale":"2023-10-18T12:00:32.454642+0000","last_undegraded":"2023-10-18T12:00:32.454642+0000","last_fullsized":"2023-10-18T12:00:32.454642+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:56.618941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445494+0000","last_change":"2023-10-18T12:00:32.443504+0000","last_active":"2023-10-18T12:00:32.445494+0000","last_peered":"2023-10-18T12:00:32.445494+0000","last_clean":"2023-10-18T12:00:32.445494+0000","last_became_active":"2023-10-18T12:00:32.443191+0000","last_became_peered":"2023-10-18T12:00:32.443191+0000","last_unstale":"2023-10-18T12:00:32.445494+0000","last_undegraded":"2023-10-18T12:00:32.445494+0000","last_fullsized":"2023-10-18T12:00:32.445494+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:23.559656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427089+0000","last_change":"2023-10-18T12:00:20.063355+0000","last_active":"2023-10-18T12:00:31.427089+0000","last_peered":"2023-10-18T12:00:31.427089+0000","last_clean":"2023-10-18T12:00:31.427089+0000","last_became_active":"2023-10-18T12:00:20.063097+0000","last_became_peered":"2023-10-18T12:00:20.063097+0000","last_unstale":"2023-10-18T12:00:31.427089+0000","last_undegraded":"2023-10-18T12:00:31.427089+0000","last_fullsized":"2023-10-18T12:00:31.427089+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:13.852394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430839+0000","last_change":"2023-10-18T12:00:21.090430+0000","last_active":"2023-10-18T12:00:31.430839+0000","last_peered":"2023-10-18T12:00:31.430839+0000","last_clean":"2023-10-18T12:00:31.430839+0000","last_became_active":"2023-10-18T12:00:21.090144+0000","last_became_peered":"2023-10-18T12:00:21.090144+0000","last_unstale":"2023-10-18T12:00:31.430839+0000","last_undegraded":"2023-10-18T12:00:31.430839+0000","last_fullsized":"2023-10-18T12:00:31.430839+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:32:47.987741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.422216+0000","last_change":"2023-10-18T12:00:31.425713+0000","last_active":"2023-10-18T12:00:32.422216+0000","last_peered":"2023-10-18T12:00:32.422216+0000","last_clean":"2023-10-18T12:00:32.422216+0000","last_became_active":"2023-10-18T12:00:31.425465+0000","last_became_peered":"2023-10-18T12:00:31.425465+0000","last_unstale":"2023-10-18T12:00:32.422216+0000","last_undegraded":"2023-10-18T12:00:32.422216+0000","last_fullsized":"2023-10-18T12:00:32.422216+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:34.143233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.453304+0000","last_change":"2023-10-18T12:00:32.448429+0000","last_active":"2023-10-18T12:00:32.453304+0000","last_peered":"2023-10-18T12:00:32.453304+0000","last_clean":"2023-10-18T12:00:32.453304+0000","last_became_active":"2023-10-18T12:00:32.448128+0000","last_became_peered":"2023-10-18T12:00:32.448128+0000","last_unstale":"2023-10-18T12:00:32.453304+0000","last_undegraded":"2023-10-18T12:00:32.453304+0000","last_fullsized":"2023-10-18T12:00:32.453304+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:17.304006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421809+0000","last_change":"2023-10-18T12:00:20.067526+0000","last_active":"2023-10-18T12:00:31.421809+0000","last_peered":"2023-10-18T12:00:31.421809+0000","last_clean":"2023-10-18T12:00:31.421809+0000","last_became_active":"2023-10-18T12:00:20.067093+0000","last_became_peered":"2023-10-18T12:00:20.067093+0000","last_unstale":"2023-10-18T12:00:31.421809+0000","last_undegraded":"2023-10-18T12:00:31.421809+0000","last_fullsized":"2023-10-18T12:00:31.421809+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:08.654298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419376+0000","last_change":"2023-10-18T12:00:21.074593+0000","last_active":"2023-10-18T12:00:31.419376+0000","last_peered":"2023-10-18T12:00:31.419376+0000","last_clean":"2023-10-18T12:00:31.419376+0000","last_became_active":"2023-10-18T12:00:21.074328+0000","last_became_peered":"2023-10-18T12:00:21.074328+0000","last_unstale":"2023-10-18T12:00:31.419376+0000","last_undegraded":"2023-10-18T12:00:31.419376+0000","last_fullsized":"2023-10-18T12:00:31.419376+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:30:55.127165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454403+0000","last_change":"2023-10-18T12:00:31.432624+0000","last_active":"2023-10-18T12:00:32.454403+0000","last_peered":"2023-10-18T12:00:32.454403+0000","last_clean":"2023-10-18T12:00:32.454403+0000","last_became_active":"2023-10-18T12:00:31.432368+0000","last_became_peered":"2023-10-18T12:00:31.432368+0000","last_unstale":"2023-10-18T12:00:32.454403+0000","last_undegraded":"2023-10-18T12:00:32.454403+0000","last_fullsized":"2023-10-18T12:00:32.454403+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:47.695385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.449003+0000","last_change":"2023-10-18T12:00:32.448513+0000","last_active":"2023-10-18T12:00:32.449003+0000","last_peered":"2023-10-18T12:00:32.449003+0000","last_clean":"2023-10-18T12:00:32.449003+0000","last_became_active":"2023-10-18T12:00:32.448229+0000","last_became_peered":"2023-10-18T12:00:32.448229+0000","last_unstale":"2023-10-18T12:00:32.449003+0000","last_undegraded":"2023-10-18T12:00:32.449003+0000","last_fullsized":"2023-10-18T12:00:32.449003+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.778814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419240+0000","last_change":"2023-10-18T12:00:20.067607+0000","last_active":"2023-10-18T12:00:31.419240+0000","last_peered":"2023-10-18T12:00:31.419240+0000","last_clean":"2023-10-18T12:00:31.419240+0000","last_became_active":"2023-10-18T12:00:20.067298+0000","last_became_peered":"2023-10-18T12:00:20.067298+0000","last_unstale":"2023-10-18T12:00:31.419240+0000","last_undegraded":"2023-10-18T12:00:31.419240+0000","last_fullsized":"2023-10-18T12:00:31.419240+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:13.982605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419253+0000","last_change":"2023-10-18T12:00:21.090572+0000","last_active":"2023-10-18T12:00:31.419253+0000","last_peered":"2023-10-18T12:00:31.419253+0000","last_clean":"2023-10-18T12:00:31.419253+0000","last_became_active":"2023-10-18T12:00:21.090312+0000","last_became_peered":"2023-10-18T12:00:21.090312+0000","last_unstale":"2023-10-18T12:00:31.419253+0000","last_undegraded":"2023-10-18T12:00:31.419253+0000","last_fullsized":"2023-10-18T12:00:31.419253+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:00:29.845435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.420964+0000","last_change":"2023-10-18T12:00:31.434473+0000","last_active":"2023-10-18T12:00:32.420964+0000","last_peered":"2023-10-18T12:00:32.420964+0000","last_clean":"2023-10-18T12:00:32.420964+0000","last_became_active":"2023-10-18T12:00:31.434226+0000","last_became_peered":"2023-10-18T12:00:31.434226+0000","last_unstale":"2023-10-18T12:00:32.420964+0000","last_undegraded":"2023-10-18T12:00:32.420964+0000","last_fullsized":"2023-10-18T12:00:32.420964+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:43.981297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.439919+0000","last_change":"2023-10-18T12:00:32.439361+0000","last_active":"2023-10-18T12:00:32.439919+0000","last_peered":"2023-10-18T12:00:32.439919+0000","last_clean":"2023-10-18T12:00:32.439919+0000","last_became_active":"2023-10-18T12:00:32.439094+0000","last_became_peered":"2023-10-18T12:00:32.439094+0000","last_unstale":"2023-10-18T12:00:32.439919+0000","last_undegraded":"2023-10-18T12:00:32.439919+0000","last_fullsized":"2023-10-18T12:00:32.439919+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:52.458791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419689+0000","last_change":"2023-10-18T12:00:20.058912+0000","last_active":"2023-10-18T12:00:31.419689+0000","last_peered":"2023-10-18T12:00:31.419689+0000","last_clean":"2023-10-18T12:00:31.419689+0000","last_became_active":"2023-10-18T12:00:20.058634+0000","last_became_peered":"2023-10-18T12:00:20.058634+0000","last_unstale":"2023-10-18T12:00:31.419689+0000","last_undegraded":"2023-10-18T12:00:31.419689+0000","last_fullsized":"2023-10-18T12:00:31.419689+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:06.254577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.429785+0000","last_change":"2023-10-18T12:00:21.092578+0000","last_active":"2023-10-18T12:00:31.429785+0000","last_peered":"2023-10-18T12:00:31.429785+0000","last_clean":"2023-10-18T12:00:31.429785+0000","last_became_active":"2023-10-18T12:00:21.092290+0000","last_became_peered":"2023-10-18T12:00:21.092290+0000","last_unstale":"2023-10-18T12:00:31.429785+0000","last_undegraded":"2023-10-18T12:00:31.429785+0000","last_fullsized":"2023-10-18T12:00:31.429785+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:05.871282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.422216+0000","last_change":"2023-10-18T12:00:31.421864+0000","last_active":"2023-10-18T12:00:32.422216+0000","last_peered":"2023-10-18T12:00:32.422216+0000","last_clean":"2023-10-18T12:00:32.422216+0000","last_became_active":"2023-10-18T12:00:31.421615+0000","last_became_peered":"2023-10-18T12:00:31.421615+0000","last_unstale":"2023-10-18T12:00:32.422216+0000","last_undegraded":"2023-10-18T12:00:32.422216+0000","last_fullsized":"2023-10-18T12:00:32.422216+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:50.619813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.427230+0000","last_change":"2023-10-18T12:00:32.424481+0000","last_active":"2023-10-18T12:00:32.427230+0000","last_peered":"2023-10-18T12:00:32.427230+0000","last_clean":"2023-10-18T12:00:32.427230+0000","last_became_active":"2023-10-18T12:00:32.424225+0000","last_became_peered":"2023-10-18T12:00:32.424225+0000","last_unstale":"2023-10-18T12:00:32.427230+0000","last_undegraded":"2023-10-18T12:00:32.427230+0000","last_fullsized":"2023-10-18T12:00:32.427230+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:02.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"220'8","reported_seq":23,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426242+0000","last_change":"2023-10-18T12:00:20.066867+0000","last_active":"2023-10-18T12:00:31.426242+0000","last_peered":"2023-10-18T12:00:31.426242+0000","last_clean":"2023-10-18T12:00:31.426242+0000","last_became_active":"2023-10-18T12:00:20.066615+0000","last_became_peered":"2023-10-18T12:00:20.066615+0000","last_unstale":"2023-10-18T12:00:31.426242+0000","last_undegraded":"2023-10-18T12:00:31.426242+0000","last_fullsized":"2023-10-18T12:00:31.426242+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:08:31.444571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424209+0000","last_change":"2023-10-18T12:00:21.086868+0000","last_active":"2023-10-18T12:00:31.424209+0000","last_peered":"2023-10-18T12:00:31.424209+0000","last_clean":"2023-10-18T12:00:31.424209+0000","last_became_active":"2023-10-18T12:00:21.086595+0000","last_became_peered":"2023-10-18T12:00:21.086595+0000","last_unstale":"2023-10-18T12:00:31.424209+0000","last_undegraded":"2023-10-18T12:00:31.424209+0000","last_fullsized":"2023-10-18T12:00:31.424209+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:33.721991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.426079+0000","last_change":"2023-10-18T12:00:31.435014+0000","last_active":"2023-10-18T12:00:32.426079+0000","last_peered":"2023-10-18T12:00:32.426079+0000","last_clean":"2023-10-18T12:00:32.426079+0000","last_became_active":"2023-10-18T12:00:31.434304+0000","last_became_peered":"2023-10-18T12:00:31.434304+0000","last_unstale":"2023-10-18T12:00:32.426079+0000","last_undegraded":"2023-10-18T12:00:32.426079+0000","last_fullsized":"2023-10-18T12:00:32.426079+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:57:32.385132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429662+0000","last_change":"2023-10-18T12:00:32.428970+0000","last_active":"2023-10-18T12:00:32.429662+0000","last_peered":"2023-10-18T12:00:32.429662+0000","last_clean":"2023-10-18T12:00:32.429662+0000","last_became_active":"2023-10-18T12:00:32.428727+0000","last_became_peered":"2023-10-18T12:00:32.428727+0000","last_unstale":"2023-10-18T12:00:32.429662+0000","last_undegraded":"2023-10-18T12:00:32.429662+0000","last_fullsized":"2023-10-18T12:00:32.429662+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:07.615526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417258+0000","last_change":"2023-10-18T12:00:20.061573+0000","last_active":"2023-10-18T12:00:31.417258+0000","last_peered":"2023-10-18T12:00:31.417258+0000","last_clean":"2023-10-18T12:00:31.417258+0000","last_became_active":"2023-10-18T12:00:20.061363+0000","last_became_peered":"2023-10-18T12:00:20.061363+0000","last_unstale":"2023-10-18T12:00:31.417258+0000","last_undegraded":"2023-10-18T12:00:31.417258+0000","last_fullsized":"2023-10-18T12:00:31.417258+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:33.588701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418261+0000","last_change":"2023-10-18T12:00:21.081617+0000","last_active":"2023-10-18T12:00:31.418261+0000","last_peered":"2023-10-18T12:00:31.418261+0000","last_clean":"2023-10-18T12:00:31.418261+0000","last_became_active":"2023-10-18T12:00:21.081232+0000","last_became_peered":"2023-10-18T12:00:21.081232+0000","last_unstale":"2023-10-18T12:00:31.418261+0000","last_undegraded":"2023-10-18T12:00:31.418261+0000","last_fullsized":"2023-10-18T12:00:31.418261+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:59:30.932435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.433373+0000","last_change":"2023-10-18T12:00:31.428387+0000","last_active":"2023-10-18T12:00:32.433373+0000","last_peered":"2023-10-18T12:00:32.433373+0000","last_clean":"2023-10-18T12:00:32.433373+0000","last_became_active":"2023-10-18T12:00:31.428118+0000","last_became_peered":"2023-10-18T12:00:31.428118+0000","last_unstale":"2023-10-18T12:00:32.433373+0000","last_undegraded":"2023-10-18T12:00:32.433373+0000","last_fullsized":"2023-10-18T12:00:32.433373+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:42.705320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.433264+0000","last_change":"2023-10-18T12:00:32.432291+0000","last_active":"2023-10-18T12:00:32.433264+0000","last_peered":"2023-10-18T12:00:32.433264+0000","last_clean":"2023-10-18T12:00:32.433264+0000","last_became_active":"2023-10-18T12:00:32.432021+0000","last_became_peered":"2023-10-18T12:00:32.432021+0000","last_unstale":"2023-10-18T12:00:32.433264+0000","last_undegraded":"2023-10-18T12:00:32.433264+0000","last_fullsized":"2023-10-18T12:00:32.433264+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:38.552359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419133+0000","last_change":"2023-10-18T12:00:20.059890+0000","last_active":"2023-10-18T12:00:31.419133+0000","last_peered":"2023-10-18T12:00:31.419133+0000","last_clean":"2023-10-18T12:00:31.419133+0000","last_became_active":"2023-10-18T12:00:20.059647+0000","last_became_peered":"2023-10-18T12:00:20.059647+0000","last_unstale":"2023-10-18T12:00:31.419133+0000","last_undegraded":"2023-10-18T12:00:31.419133+0000","last_fullsized":"2023-10-18T12:00:31.419133+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:08.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419681+0000","last_change":"2023-10-18T12:00:21.075390+0000","last_active":"2023-10-18T12:00:31.419681+0000","last_peered":"2023-10-18T12:00:31.419681+0000","last_clean":"2023-10-18T12:00:31.419681+0000","last_became_active":"2023-10-18T12:00:21.075149+0000","last_became_peered":"2023-10-18T12:00:21.075149+0000","last_unstale":"2023-10-18T12:00:31.419681+0000","last_undegraded":"2023-10-18T12:00:31.419681+0000","last_fullsized":"2023-10-18T12:00:31.419681+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:00.806334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"224'6","reported_seq":17,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.456170+0000","last_change":"2023-10-18T12:00:31.434654+0000","last_active":"2023-10-18T12:00:32.456170+0000","last_peered":"2023-10-18T12:00:32.456170+0000","last_clean":"2023-10-18T12:00:32.456170+0000","last_became_active":"2023-10-18T12:00:31.434279+0000","last_became_peered":"2023-10-18T12:00:31.434279+0000","last_unstale":"2023-10-18T12:00:32.456170+0000","last_undegraded":"2023-10-18T12:00:32.456170+0000","last_fullsized":"2023-10-18T12:00:32.456170+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:17:41.911245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429689+0000","last_change":"2023-10-18T12:00:32.427768+0000","last_active":"2023-10-18T12:00:32.429689+0000","last_peered":"2023-10-18T12:00:32.429689+0000","last_clean":"2023-10-18T12:00:32.429689+0000","last_became_active":"2023-10-18T12:00:32.427530+0000","last_became_peered":"2023-10-18T12:00:32.427530+0000","last_unstale":"2023-10-18T12:00:32.429689+0000","last_undegraded":"2023-10-18T12:00:32.429689+0000","last_fullsized":"2023-10-18T12:00:32.429689+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:42.316328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421091+0000","last_change":"2023-10-18T12:00:20.063126+0000","last_active":"2023-10-18T12:00:31.421091+0000","last_peered":"2023-10-18T12:00:31.421091+0000","last_clean":"2023-10-18T12:00:31.421091+0000","last_became_active":"2023-10-18T12:00:20.062833+0000","last_became_peered":"2023-10-18T12:00:20.062833+0000","last_unstale":"2023-10-18T12:00:31.421091+0000","last_undegraded":"2023-10-18T12:00:31.421091+0000","last_fullsized":"2023-10-18T12:00:31.421091+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:57.198338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421619+0000","last_change":"2023-10-18T12:00:21.078685+0000","last_active":"2023-10-18T12:00:31.421619+0000","last_peered":"2023-10-18T12:00:31.421619+0000","last_clean":"2023-10-18T12:00:31.421619+0000","last_became_active":"2023-10-18T12:00:21.078434+0000","last_became_peered":"2023-10-18T12:00:21.078434+0000","last_unstale":"2023-10-18T12:00:31.421619+0000","last_undegraded":"2023-10-18T12:00:31.421619+0000","last_fullsized":"2023-10-18T12:00:31.421619+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:12.197055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"224'3","reported_seq":14,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.447244+0000","last_change":"2023-10-18T12:00:31.429037+0000","last_active":"2023-10-18T12:00:32.447244+0000","last_peered":"2023-10-18T12:00:32.447244+0000","last_clean":"2023-10-18T12:00:32.447244+0000","last_became_active":"2023-10-18T12:00:31.428766+0000","last_became_peered":"2023-10-18T12:00:31.428766+0000","last_unstale":"2023-10-18T12:00:32.447244+0000","last_undegraded":"2023-10-18T12:00:32.447244+0000","last_fullsized":"2023-10-18T12:00:32.447244+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:13.312973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454633+0000","last_change":"2023-10-18T12:00:32.453960+0000","last_active":"2023-10-18T12:00:32.454633+0000","last_peered":"2023-10-18T12:00:32.454633+0000","last_clean":"2023-10-18T12:00:32.454633+0000","last_became_active":"2023-10-18T12:00:32.453655+0000","last_became_peered":"2023-10-18T12:00:32.453655+0000","last_unstale":"2023-10-18T12:00:32.454633+0000","last_undegraded":"2023-10-18T12:00:32.454633+0000","last_fullsized":"2023-10-18T12:00:32.454633+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:31.476751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"221'8","reported_seq":23,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.425513+0000","last_change":"2023-10-18T12:00:20.070213+0000","last_active":"2023-10-18T12:00:31.425513+0000","last_peered":"2023-10-18T12:00:31.425513+0000","last_clean":"2023-10-18T12:00:31.425513+0000","last_became_active":"2023-10-18T12:00:20.069888+0000","last_became_peered":"2023-10-18T12:00:20.069888+0000","last_unstale":"2023-10-18T12:00:31.425513+0000","last_undegraded":"2023-10-18T12:00:31.425513+0000","last_fullsized":"2023-10-18T12:00:31.425513+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:06:55.247225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427378+0000","last_change":"2023-10-18T12:00:21.092364+0000","last_active":"2023-10-18T12:00:31.427378+0000","last_peered":"2023-10-18T12:00:31.427378+0000","last_clean":"2023-10-18T12:00:31.427378+0000","last_became_active":"2023-10-18T12:00:21.092090+0000","last_became_peered":"2023-10-18T12:00:21.092090+0000","last_unstale":"2023-10-18T12:00:31.427378+0000","last_undegraded":"2023-10-18T12:00:31.427378+0000","last_fullsized":"2023-10-18T12:00:31.427378+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:19.318183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.427693+0000","last_change":"2023-10-18T12:00:31.430567+0000","last_active":"2023-10-18T12:00:32.427693+0000","last_peered":"2023-10-18T12:00:32.427693+0000","last_clean":"2023-10-18T12:00:32.427693+0000","last_became_active":"2023-10-18T12:00:31.430271+0000","last_became_peered":"2023-10-18T12:00:31.430271+0000","last_unstale":"2023-10-18T12:00:32.427693+0000","last_undegraded":"2023-10-18T12:00:32.427693+0000","last_fullsized":"2023-10-18T12:00:32.427693+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:07.254107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.436812+0000","last_change":"2023-10-18T12:00:32.434053+0000","last_active":"2023-10-18T12:00:32.436812+0000","last_peered":"2023-10-18T12:00:32.436812+0000","last_clean":"2023-10-18T12:00:32.436812+0000","last_became_active":"2023-10-18T12:00:32.433800+0000","last_became_peered":"2023-10-18T12:00:32.433800+0000","last_unstale":"2023-10-18T12:00:32.436812+0000","last_undegraded":"2023-10-18T12:00:32.436812+0000","last_fullsized":"2023-10-18T12:00:32.436812+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:59.768535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"220'3","reported_seq":18,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418408+0000","last_change":"2023-10-18T12:00:20.057436+0000","last_active":"2023-10-18T12:00:31.418408+0000","last_peered":"2023-10-18T12:00:31.418408+0000","last_clean":"2023-10-18T12:00:31.418408+0000","last_became_active":"2023-10-18T12:00:20.057117+0000","last_became_peered":"2023-10-18T12:00:20.057117+0000","last_unstale":"2023-10-18T12:00:31.418408+0000","last_undegraded":"2023-10-18T12:00:31.418408+0000","last_fullsized":"2023-10-18T12:00:31.418408+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:02.796323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417821+0000","last_change":"2023-10-18T12:00:21.074575+0000","last_active":"2023-10-18T12:00:31.417821+0000","last_peered":"2023-10-18T12:00:31.417821+0000","last_clean":"2023-10-18T12:00:31.417821+0000","last_became_active":"2023-10-18T12:00:21.074302+0000","last_became_peered":"2023-10-18T12:00:21.074302+0000","last_unstale":"2023-10-18T12:00:31.417821+0000","last_undegraded":"2023-10-18T12:00:31.417821+0000","last_fullsized":"2023-10-18T12:00:31.417821+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:30:58.204038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454507+0000","last_change":"2023-10-18T12:00:31.437228+0000","last_active":"2023-10-18T12:00:32.454507+0000","last_peered":"2023-10-18T12:00:32.454507+0000","last_clean":"2023-10-18T12:00:32.454507+0000","last_became_active":"2023-10-18T12:00:31.436332+0000","last_became_peered":"2023-10-18T12:00:31.436332+0000","last_unstale":"2023-10-18T12:00:32.454507+0000","last_undegraded":"2023-10-18T12:00:32.454507+0000","last_fullsized":"2023-10-18T12:00:32.454507+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:56:52.724382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.435466+0000","last_change":"2023-10-18T12:00:32.432434+0000","last_active":"2023-10-18T12:00:32.435466+0000","last_peered":"2023-10-18T12:00:32.435466+0000","last_clean":"2023-10-18T12:00:32.435466+0000","last_became_active":"2023-10-18T12:00:32.432183+0000","last_became_peered":"2023-10-18T12:00:32.432183+0000","last_unstale":"2023-10-18T12:00:32.435466+0000","last_undegraded":"2023-10-18T12:00:32.435466+0000","last_fullsized":"2023-10-18T12:00:32.435466+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:09.830514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.423971+0000","last_change":"2023-10-18T12:00:20.070057+0000","last_active":"2023-10-18T12:00:31.423971+0000","last_peered":"2023-10-18T12:00:31.423971+0000","last_clean":"2023-10-18T12:00:31.423971+0000","last_became_active":"2023-10-18T12:00:20.069456+0000","last_became_peered":"2023-10-18T12:00:20.069456+0000","last_unstale":"2023-10-18T12:00:31.423971+0000","last_undegraded":"2023-10-18T12:00:31.423971+0000","last_fullsized":"2023-10-18T12:00:31.423971+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:14.207084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418867+0000","last_change":"2023-10-18T12:00:21.078356+0000","last_active":"2023-10-18T12:00:31.418867+0000","last_peered":"2023-10-18T12:00:31.418867+0000","last_clean":"2023-10-18T12:00:31.418867+0000","last_became_active":"2023-10-18T12:00:21.078095+0000","last_became_peered":"2023-10-18T12:00:21.078095+0000","last_unstale":"2023-10-18T12:00:31.418867+0000","last_undegraded":"2023-10-18T12:00:31.418867+0000","last_fullsized":"2023-10-18T12:00:31.418867+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:44.526073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"224'8","reported_seq":19,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454505+0000","last_change":"2023-10-18T12:00:31.437910+0000","last_active":"2023-10-18T12:00:32.454505+0000","last_peered":"2023-10-18T12:00:32.454505+0000","last_clean":"2023-10-18T12:00:32.454505+0000","last_became_active":"2023-10-18T12:00:31.437358+0000","last_became_peered":"2023-10-18T12:00:31.437358+0000","last_unstale":"2023-10-18T12:00:32.454505+0000","last_undegraded":"2023-10-18T12:00:32.454505+0000","last_fullsized":"2023-10-18T12:00:32.454505+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:54:49.192007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.444250+0000","last_change":"2023-10-18T12:00:32.443846+0000","last_active":"2023-10-18T12:00:32.444250+0000","last_peered":"2023-10-18T12:00:32.444250+0000","last_clean":"2023-10-18T12:00:32.444250+0000","last_became_active":"2023-10-18T12:00:32.443547+0000","last_became_peered":"2023-10-18T12:00:32.443547+0000","last_unstale":"2023-10-18T12:00:32.444250+0000","last_undegraded":"2023-10-18T12:00:32.444250+0000","last_fullsized":"2023-10-18T12:00:32.444250+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:43.711256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427667+0000","last_change":"2023-10-18T12:00:20.068018+0000","last_active":"2023-10-18T12:00:31.427667+0000","last_peered":"2023-10-18T12:00:31.427667+0000","last_clean":"2023-10-18T12:00:31.427667+0000","last_became_active":"2023-10-18T12:00:20.067744+0000","last_became_peered":"2023-10-18T12:00:20.067744+0000","last_unstale":"2023-10-18T12:00:31.427667+0000","last_undegraded":"2023-10-18T12:00:31.427667+0000","last_fullsized":"2023-10-18T12:00:31.427667+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:31:17.572716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417736+0000","last_change":"2023-10-18T12:00:21.079285+0000","last_active":"2023-10-18T12:00:31.417736+0000","last_peered":"2023-10-18T12:00:31.417736+0000","last_clean":"2023-10-18T12:00:31.417736+0000","last_became_active":"2023-10-18T12:00:21.079002+0000","last_became_peered":"2023-10-18T12:00:21.079002+0000","last_unstale":"2023-10-18T12:00:31.417736+0000","last_undegraded":"2023-10-18T12:00:31.417736+0000","last_fullsized":"2023-10-18T12:00:31.417736+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:49:28.849697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.421988+0000","last_change":"2023-10-18T12:00:31.427549+0000","last_active":"2023-10-18T12:00:32.421988+0000","last_peered":"2023-10-18T12:00:32.421988+0000","last_clean":"2023-10-18T12:00:32.421988+0000","last_became_active":"2023-10-18T12:00:31.427268+0000","last_became_peered":"2023-10-18T12:00:31.427268+0000","last_unstale":"2023-10-18T12:00:32.421988+0000","last_undegraded":"2023-10-18T12:00:32.421988+0000","last_fullsized":"2023-10-18T12:00:32.421988+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:28.728601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429571+0000","last_change":"2023-10-18T12:00:32.428371+0000","last_active":"2023-10-18T12:00:32.429571+0000","last_peered":"2023-10-18T12:00:32.429571+0000","last_clean":"2023-10-18T12:00:32.429571+0000","last_became_active":"2023-10-18T12:00:32.428129+0000","last_became_peered":"2023-10-18T12:00:32.428129+0000","last_unstale":"2023-10-18T12:00:32.429571+0000","last_undegraded":"2023-10-18T12:00:32.429571+0000","last_fullsized":"2023-10-18T12:00:32.429571+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:10.614449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454049+0000","last_change":"2023-10-18T12:00:31.434166+0000","last_active":"2023-10-18T12:00:32.454049+0000","last_peered":"2023-10-18T12:00:32.454049+0000","last_clean":"2023-10-18T12:00:32.454049+0000","last_became_active":"2023-10-18T12:00:31.433892+0000","last_became_peered":"2023-10-18T12:00:31.433892+0000","last_unstale":"2023-10-18T12:00:32.454049+0000","last_undegraded":"2023-10-18T12:00:32.454049+0000","last_fullsized":"2023-10-18T12:00:32.454049+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:59.870094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418892+0000","last_change":"2023-10-18T12:00:20.067947+0000","last_active":"2023-10-18T12:00:31.418892+0000","last_peered":"2023-10-18T12:00:31.418892+0000","last_clean":"2023-10-18T12:00:31.418892+0000","last_became_active":"2023-10-18T12:00:20.067433+0000","last_became_peered":"2023-10-18T12:00:20.067433+0000","last_unstale":"2023-10-18T12:00:31.418892+0000","last_undegraded":"2023-10-18T12:00:31.418892+0000","last_fullsized":"2023-10-18T12:00:31.418892+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:38.382188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418416+0000","last_change":"2023-10-18T12:00:21.076517+0000","last_active":"2023-10-18T12:00:31.418416+0000","last_peered":"2023-10-18T12:00:31.418416+0000","last_clean":"2023-10-18T12:00:31.418416+0000","last_became_active":"2023-10-18T12:00:21.076276+0000","last_became_peered":"2023-10-18T12:00:21.076276+0000","last_unstale":"2023-10-18T12:00:31.418416+0000","last_undegraded":"2023-10-18T12:00:31.418416+0000","last_fullsized":"2023-10-18T12:00:31.418416+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:02.908869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437343+0000","last_change":"2023-10-18T12:00:32.432675+0000","last_active":"2023-10-18T12:00:32.437343+0000","last_peered":"2023-10-18T12:00:32.437343+0000","last_clean":"2023-10-18T12:00:32.437343+0000","last_became_active":"2023-10-18T12:00:32.432408+0000","last_became_peered":"2023-10-18T12:00:32.432408+0000","last_unstale":"2023-10-18T12:00:32.437343+0000","last_undegraded":"2023-10-18T12:00:32.437343+0000","last_fullsized":"2023-10-18T12:00:32.437343+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:15:34.208634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"224'4","reported_seq":15,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.443955+0000","last_change":"2023-10-18T12:00:31.433811+0000","last_active":"2023-10-18T12:00:32.443955+0000","last_peered":"2023-10-18T12:00:32.443955+0000","last_clean":"2023-10-18T12:00:32.443955+0000","last_became_active":"2023-10-18T12:00:31.433567+0000","last_became_peered":"2023-10-18T12:00:31.433567+0000","last_unstale":"2023-10-18T12:00:32.443955+0000","last_undegraded":"2023-10-18T12:00:32.443955+0000","last_fullsized":"2023-10-18T12:00:32.443955+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:49.861828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424458+0000","last_change":"2023-10-18T12:00:20.070314+0000","last_active":"2023-10-18T12:00:31.424458+0000","last_peered":"2023-10-18T12:00:31.424458+0000","last_clean":"2023-10-18T12:00:31.424458+0000","last_became_active":"2023-10-18T12:00:20.069974+0000","last_became_peered":"2023-10-18T12:00:20.069974+0000","last_unstale":"2023-10-18T12:00:31.424458+0000","last_undegraded":"2023-10-18T12:00:31.424458+0000","last_fullsized":"2023-10-18T12:00:31.424458+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:39.359672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418039+0000","last_change":"2023-10-18T12:00:21.081350+0000","last_active":"2023-10-18T12:00:31.418039+0000","last_peered":"2023-10-18T12:00:31.418039+0000","last_clean":"2023-10-18T12:00:31.418039+0000","last_became_active":"2023-10-18T12:00:21.080920+0000","last_became_peered":"2023-10-18T12:00:21.080920+0000","last_unstale":"2023-10-18T12:00:31.418039+0000","last_undegraded":"2023-10-18T12:00:31.418039+0000","last_fullsized":"2023-10-18T12:00:31.418039+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:37.696091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.450381+0000","last_change":"2023-10-18T12:00:32.448948+0000","last_active":"2023-10-18T12:00:32.450381+0000","last_peered":"2023-10-18T12:00:32.450381+0000","last_clean":"2023-10-18T12:00:32.450381+0000","last_became_active":"2023-10-18T12:00:32.448247+0000","last_became_peered":"2023-10-18T12:00:32.448247+0000","last_unstale":"2023-10-18T12:00:32.450381+0000","last_undegraded":"2023-10-18T12:00:32.450381+0000","last_fullsized":"2023-10-18T12:00:32.450381+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:24.353309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424827+0000","last_change":"2023-10-18T12:00:20.070416+0000","last_active":"2023-10-18T12:00:31.424827+0000","last_peered":"2023-10-18T12:00:31.424827+0000","last_clean":"2023-10-18T12:00:31.424827+0000","last_became_active":"2023-10-18T12:00:20.070121+0000","last_became_peered":"2023-10-18T12:00:20.070121+0000","last_unstale":"2023-10-18T12:00:31.424827+0000","last_undegraded":"2023-10-18T12:00:31.424827+0000","last_fullsized":"2023-10-18T12:00:31.424827+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:33.588701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418223+0000","last_change":"2023-10-18T12:00:21.087738+0000","last_active":"2023-10-18T12:00:31.418223+0000","last_peered":"2023-10-18T12:00:31.418223+0000","last_clean":"2023-10-18T12:00:31.418223+0000","last_became_active":"2023-10-18T12:00:21.087104+0000","last_became_peered":"2023-10-18T12:00:21.087104+0000","last_unstale":"2023-10-18T12:00:31.418223+0000","last_undegraded":"2023-10-18T12:00:31.418223+0000","last_fullsized":"2023-10-18T12:00:31.418223+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:52.609784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"224'2","reported_seq":13,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.452420+0000","last_change":"2023-10-18T12:00:31.433384+0000","last_active":"2023-10-18T12:00:32.452420+0000","last_peered":"2023-10-18T12:00:32.452420+0000","last_clean":"2023-10-18T12:00:32.452420+0000","last_became_active":"2023-10-18T12:00:31.432880+0000","last_became_peered":"2023-10-18T12:00:31.432880+0000","last_unstale":"2023-10-18T12:00:32.452420+0000","last_undegraded":"2023-10-18T12:00:32.452420+0000","last_fullsized":"2023-10-18T12:00:32.452420+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:46.143888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.446550+0000","last_change":"2023-10-18T12:00:32.443728+0000","last_active":"2023-10-18T12:00:32.446550+0000","last_peered":"2023-10-18T12:00:32.446550+0000","last_clean":"2023-10-18T12:00:32.446550+0000","last_became_active":"2023-10-18T12:00:32.443454+0000","last_became_peered":"2023-10-18T12:00:32.443454+0000","last_unstale":"2023-10-18T12:00:32.446550+0000","last_undegraded":"2023-10-18T12:00:32.446550+0000","last_fullsized":"2023-10-18T12:00:32.446550+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:01.817297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418157+0000","last_change":"2023-10-18T12:00:20.060797+0000","last_active":"2023-10-18T12:00:31.418157+0000","last_peered":"2023-10-18T12:00:31.418157+0000","last_clean":"2023-10-18T12:00:31.418157+0000","last_became_active":"2023-10-18T12:00:20.060568+0000","last_became_peered":"2023-10-18T12:00:20.060568+0000","last_unstale":"2023-10-18T12:00:31.418157+0000","last_undegraded":"2023-10-18T12:00:31.418157+0000","last_fullsized":"2023-10-18T12:00:31.418157+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:27:50.071689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427310+0000","last_change":"2023-10-18T12:00:21.089013+0000","last_active":"2023-10-18T12:00:31.427310+0000","last_peered":"2023-10-18T12:00:31.427310+0000","last_clean":"2023-10-18T12:00:31.427310+0000","last_became_active":"2023-10-18T12:00:21.088643+0000","last_became_peered":"2023-10-18T12:00:21.088643+0000","last_unstale":"2023-10-18T12:00:31.427310+0000","last_undegraded":"2023-10-18T12:00:31.427310+0000","last_fullsized":"2023-10-18T12:00:31.427310+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:15.272122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448802+0000","last_change":"2023-10-18T12:00:31.436838+0000","last_active":"2023-10-18T12:00:32.448802+0000","last_peered":"2023-10-18T12:00:32.448802+0000","last_clean":"2023-10-18T12:00:32.448802+0000","last_became_active":"2023-10-18T12:00:31.436505+0000","last_became_peered":"2023-10-18T12:00:31.436505+0000","last_unstale":"2023-10-18T12:00:32.448802+0000","last_undegraded":"2023-10-18T12:00:32.448802+0000","last_fullsized":"2023-10-18T12:00:32.448802+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:18.690057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437629+0000","last_change":"2023-10-18T12:00:32.432921+0000","last_active":"2023-10-18T12:00:32.437629+0000","last_peered":"2023-10-18T12:00:32.437629+0000","last_clean":"2023-10-18T12:00:32.437629+0000","last_became_active":"2023-10-18T12:00:32.432594+0000","last_became_peered":"2023-10-18T12:00:32.432594+0000","last_unstale":"2023-10-18T12:00:32.437629+0000","last_undegraded":"2023-10-18T12:00:32.437629+0000","last_fullsized":"2023-10-18T12:00:32.437629+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:28:23.492154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"220'2","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421886+0000","last_change":"2023-10-18T12:00:20.057262+0000","last_active":"2023-10-18T12:00:31.421886+0000","last_peered":"2023-10-18T12:00:31.421886+0000","last_clean":"2023-10-18T12:00:31.421886+0000","last_became_active":"2023-10-18T12:00:20.057010+0000","last_became_peered":"2023-10-18T12:00:20.057010+0000","last_unstale":"2023-10-18T12:00:31.421886+0000","last_undegraded":"2023-10-18T12:00:31.421886+0000","last_fullsized":"2023-10-18T12:00:31.421886+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:46.723265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430746+0000","last_change":"2023-10-18T12:00:21.093491+0000","last_active":"2023-10-18T12:00:31.430746+0000","last_peered":"2023-10-18T12:00:31.430746+0000","last_clean":"2023-10-18T12:00:31.430746+0000","last_became_active":"2023-10-18T12:00:21.093199+0000","last_became_peered":"2023-10-18T12:00:21.093199+0000","last_unstale":"2023-10-18T12:00:31.430746+0000","last_undegraded":"2023-10-18T12:00:31.430746+0000","last_fullsized":"2023-10-18T12:00:31.430746+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:14.227710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"224'1","reported_seq":12,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454818+0000","last_change":"2023-10-18T12:00:31.432278+0000","last_active":"2023-10-18T12:00:32.454818+0000","last_peered":"2023-10-18T12:00:32.454818+0000","last_clean":"2023-10-18T12:00:32.454818+0000","last_became_active":"2023-10-18T12:00:31.432022+0000","last_became_peered":"2023-10-18T12:00:31.432022+0000","last_unstale":"2023-10-18T12:00:32.454818+0000","last_undegraded":"2023-10-18T12:00:32.454818+0000","last_fullsized":"2023-10-18T12:00:32.454818+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:50.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.432346+0000","last_change":"2023-10-18T12:00:32.429915+0000","last_active":"2023-10-18T12:00:32.432346+0000","last_peered":"2023-10-18T12:00:32.432346+0000","last_clean":"2023-10-18T12:00:32.432346+0000","last_became_active":"2023-10-18T12:00:32.429667+0000","last_became_peered":"2023-10-18T12:00:32.429667+0000","last_unstale":"2023-10-18T12:00:32.432346+0000","last_undegraded":"2023-10-18T12:00:32.432346+0000","last_fullsized":"2023-10-18T12:00:32.432346+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:54.061969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"220'1","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419184+0000","last_change":"2023-10-18T12:00:20.057976+0000","last_active":"2023-10-18T12:00:31.419184+0000","last_peered":"2023-10-18T12:00:31.419184+0000","last_clean":"2023-10-18T12:00:31.419184+0000","last_became_active":"2023-10-18T12:00:20.057699+0000","last_became_peered":"2023-10-18T12:00:20.057699+0000","last_unstale":"2023-10-18T12:00:31.419184+0000","last_undegraded":"2023-10-18T12:00:31.419184+0000","last_fullsized":"2023-10-18T12:00:31.419184+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:32:45.951679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418854+0000","last_change":"2023-10-18T12:00:21.073385+0000","last_active":"2023-10-18T12:00:31.418854+0000","last_peered":"2023-10-18T12:00:31.418854+0000","last_clean":"2023-10-18T12:00:31.418854+0000","last_became_active":"2023-10-18T12:00:21.073048+0000","last_became_peered":"2023-10-18T12:00:21.073048+0000","last_unstale":"2023-10-18T12:00:31.418854+0000","last_undegraded":"2023-10-18T12:00:31.418854+0000","last_fullsized":"2023-10-18T12:00:31.418854+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:23:54.891459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.418204+0000","last_change":"2023-10-18T12:00:31.432535+0000","last_active":"2023-10-18T12:00:32.418204+0000","last_peered":"2023-10-18T12:00:32.418204+0000","last_clean":"2023-10-18T12:00:32.418204+0000","last_became_active":"2023-10-18T12:00:31.432294+0000","last_became_peered":"2023-10-18T12:00:31.432294+0000","last_unstale":"2023-10-18T12:00:32.418204+0000","last_undegraded":"2023-10-18T12:00:32.418204+0000","last_fullsized":"2023-10-18T12:00:32.418204+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:31.165188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.439411+0000","last_change":"2023-10-18T12:00:32.438452+0000","last_active":"2023-10-18T12:00:32.439411+0000","last_peered":"2023-10-18T12:00:32.439411+0000","last_clean":"2023-10-18T12:00:32.439411+0000","last_became_active":"2023-10-18T12:00:32.438061+0000","last_became_peered":"2023-10-18T12:00:32.438061+0000","last_unstale":"2023-10-18T12:00:32.439411+0000","last_undegraded":"2023-10-18T12:00:32.439411+0000","last_fullsized":"2023-10-18T12:00:32.439411+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:45.957002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"220'4","reported_seq":19,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418193+0000","last_change":"2023-10-18T12:00:20.063561+0000","last_active":"2023-10-18T12:00:31.418193+0000","last_peered":"2023-10-18T12:00:31.418193+0000","last_clean":"2023-10-18T12:00:31.418193+0000","last_became_active":"2023-10-18T12:00:20.063285+0000","last_became_peered":"2023-10-18T12:00:20.063285+0000","last_unstale":"2023-10-18T12:00:31.418193+0000","last_undegraded":"2023-10-18T12:00:31.418193+0000","last_fullsized":"2023-10-18T12:00:31.418193+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:19.038326+0000","last_clean_scrub_stamp":"2023-10-18T12:00:19.038326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:55.051193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421603+0000","last_change":"2023-10-18T12:00:21.073279+0000","last_active":"2023-10-18T12:00:31.421603+0000","last_peered":"2023-10-18T12:00:31.421603+0000","last_clean":"2023-10-18T12:00:31.421603+0000","last_became_active":"2023-10-18T12:00:21.073036+0000","last_became_peered":"2023-10-18T12:00:21.073036+0000","last_unstale":"2023-10-18T12:00:31.421603+0000","last_undegraded":"2023-10-18T12:00:31.421603+0000","last_fullsized":"2023-10-18T12:00:31.421603+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:28.430510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.421668+0000","last_change":"2023-10-18T12:00:31.433300+0000","last_active":"2023-10-18T12:00:32.421668+0000","last_peered":"2023-10-18T12:00:32.421668+0000","last_clean":"2023-10-18T12:00:32.421668+0000","last_became_active":"2023-10-18T12:00:31.432328+0000","last_became_peered":"2023-10-18T12:00:31.432328+0000","last_unstale":"2023-10-18T12:00:32.421668+0000","last_undegraded":"2023-10-18T12:00:32.421668+0000","last_fullsized":"2023-10-18T12:00:32.421668+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":222,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:30.400690+0000","last_clean_scrub_stamp":"2023-10-18T12:00:30.400690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:43.256103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.452017+0000","last_change":"2023-10-18T12:00:32.451045+0000","last_active":"2023-10-18T12:00:32.452017+0000","last_peered":"2023-10-18T12:00:32.452017+0000","last_clean":"2023-10-18T12:00:32.452017+0000","last_became_active":"2023-10-18T12:00:32.450742+0000","last_became_peered":"2023-10-18T12:00:32.450742+0000","last_unstale":"2023-10-18T12:00:32.452017+0000","last_undegraded":"2023-10-18T12:00:32.452017+0000","last_fullsized":"2023-10-18T12:00:32.452017+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:26:52.679064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424960+0000","last_change":"2023-10-18T12:00:21.093224+0000","last_active":"2023-10-18T12:00:31.424960+0000","last_peered":"2023-10-18T12:00:31.424960+0000","last_clean":"2023-10-18T12:00:31.424960+0000","last_became_active":"2023-10-18T12:00:21.092949+0000","last_became_peered":"2023-10-18T12:00:21.092949+0000","last_unstale":"2023-10-18T12:00:31.424960+0000","last_undegraded":"2023-10-18T12:00:31.424960+0000","last_fullsized":"2023-10-18T12:00:31.424960+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:02:23.786569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.439268+0000","last_change":"2023-10-18T12:00:32.438297+0000","last_active":"2023-10-18T12:00:32.439268+0000","last_peered":"2023-10-18T12:00:32.439268+0000","last_clean":"2023-10-18T12:00:32.439268+0000","last_became_active":"2023-10-18T12:00:32.438050+0000","last_became_peered":"2023-10-18T12:00:32.438050+0000","last_unstale":"2023-10-18T12:00:32.439268+0000","last_undegraded":"2023-10-18T12:00:32.439268+0000","last_fullsized":"2023-10-18T12:00:32.439268+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:04.205911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424033+0000","last_change":"2023-10-18T12:00:21.092324+0000","last_active":"2023-10-18T12:00:31.424033+0000","last_peered":"2023-10-18T12:00:31.424033+0000","last_clean":"2023-10-18T12:00:31.424033+0000","last_became_active":"2023-10-18T12:00:21.092059+0000","last_became_peered":"2023-10-18T12:00:21.092059+0000","last_unstale":"2023-10-18T12:00:31.424033+0000","last_undegraded":"2023-10-18T12:00:31.424033+0000","last_fullsized":"2023-10-18T12:00:31.424033+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:15.346268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.452174+0000","last_change":"2023-10-18T12:00:32.451696+0000","last_active":"2023-10-18T12:00:32.452174+0000","last_peered":"2023-10-18T12:00:32.452174+0000","last_clean":"2023-10-18T12:00:32.452174+0000","last_became_active":"2023-10-18T12:00:32.451393+0000","last_became_peered":"2023-10-18T12:00:32.451393+0000","last_unstale":"2023-10-18T12:00:32.452174+0000","last_undegraded":"2023-10-18T12:00:32.452174+0000","last_fullsized":"2023-10-18T12:00:32.452174+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:20.931554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417811+0000","last_change":"2023-10-18T12:00:21.087421+0000","last_active":"2023-10-18T12:00:31.417811+0000","last_peered":"2023-10-18T12:00:31.417811+0000","last_clean":"2023-10-18T12:00:31.417811+0000","last_became_active":"2023-10-18T12:00:21.087145+0000","last_became_peered":"2023-10-18T12:00:21.087145+0000","last_unstale":"2023-10-18T12:00:31.417811+0000","last_undegraded":"2023-10-18T12:00:31.417811+0000","last_fullsized":"2023-10-18T12:00:31.417811+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:13:28.489188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430401+0000","last_change":"2023-10-18T12:00:32.429789+0000","last_active":"2023-10-18T12:00:32.430401+0000","last_peered":"2023-10-18T12:00:32.430401+0000","last_clean":"2023-10-18T12:00:32.430401+0000","last_became_active":"2023-10-18T12:00:32.429522+0000","last_became_peered":"2023-10-18T12:00:32.429522+0000","last_unstale":"2023-10-18T12:00:32.430401+0000","last_undegraded":"2023-10-18T12:00:32.430401+0000","last_fullsized":"2023-10-18T12:00:32.430401+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:55.109644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.417772+0000","last_change":"2023-10-18T12:00:21.082792+0000","last_active":"2023-10-18T12:00:31.417772+0000","last_peered":"2023-10-18T12:00:31.417772+0000","last_clean":"2023-10-18T12:00:31.417772+0000","last_became_active":"2023-10-18T12:00:21.082529+0000","last_became_peered":"2023-10-18T12:00:21.082529+0000","last_unstale":"2023-10-18T12:00:31.417772+0000","last_undegraded":"2023-10-18T12:00:31.417772+0000","last_fullsized":"2023-10-18T12:00:31.417772+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:02.129934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.449548+0000","last_change":"2023-10-18T12:00:32.449052+0000","last_active":"2023-10-18T12:00:32.449548+0000","last_peered":"2023-10-18T12:00:32.449548+0000","last_clean":"2023-10-18T12:00:32.449548+0000","last_became_active":"2023-10-18T12:00:32.448768+0000","last_became_peered":"2023-10-18T12:00:32.448768+0000","last_unstale":"2023-10-18T12:00:32.449548+0000","last_undegraded":"2023-10-18T12:00:32.449548+0000","last_fullsized":"2023-10-18T12:00:32.449548+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:20.330916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427015+0000","last_change":"2023-10-18T12:00:21.086100+0000","last_active":"2023-10-18T12:00:31.427015+0000","last_peered":"2023-10-18T12:00:31.427015+0000","last_clean":"2023-10-18T12:00:31.427015+0000","last_became_active":"2023-10-18T12:00:21.085820+0000","last_became_peered":"2023-10-18T12:00:21.085820+0000","last_unstale":"2023-10-18T12:00:31.427015+0000","last_undegraded":"2023-10-18T12:00:31.427015+0000","last_fullsized":"2023-10-18T12:00:31.427015+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:59.786963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448820+0000","last_change":"2023-10-18T12:00:32.446349+0000","last_active":"2023-10-18T12:00:32.448820+0000","last_peered":"2023-10-18T12:00:32.448820+0000","last_clean":"2023-10-18T12:00:32.448820+0000","last_became_active":"2023-10-18T12:00:32.442822+0000","last_became_peered":"2023-10-18T12:00:32.442822+0000","last_unstale":"2023-10-18T12:00:32.448820+0000","last_undegraded":"2023-10-18T12:00:32.448820+0000","last_fullsized":"2023-10-18T12:00:32.448820+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:24.527236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419602+0000","last_change":"2023-10-18T12:00:21.093406+0000","last_active":"2023-10-18T12:00:31.419602+0000","last_peered":"2023-10-18T12:00:31.419602+0000","last_clean":"2023-10-18T12:00:31.419602+0000","last_became_active":"2023-10-18T12:00:21.093138+0000","last_became_peered":"2023-10-18T12:00:21.093138+0000","last_unstale":"2023-10-18T12:00:31.419602+0000","last_undegraded":"2023-10-18T12:00:31.419602+0000","last_fullsized":"2023-10-18T12:00:31.419602+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:09.568953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.447530+0000","last_change":"2023-10-18T12:00:32.445903+0000","last_active":"2023-10-18T12:00:32.447530+0000","last_peered":"2023-10-18T12:00:32.447530+0000","last_clean":"2023-10-18T12:00:32.447530+0000","last_became_active":"2023-10-18T12:00:32.442480+0000","last_became_peered":"2023-10-18T12:00:32.442480+0000","last_unstale":"2023-10-18T12:00:32.447530+0000","last_undegraded":"2023-10-18T12:00:32.447530+0000","last_fullsized":"2023-10-18T12:00:32.447530+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:09.352512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419081+0000","last_change":"2023-10-18T12:00:21.091634+0000","last_active":"2023-10-18T12:00:31.419081+0000","last_peered":"2023-10-18T12:00:31.419081+0000","last_clean":"2023-10-18T12:00:31.419081+0000","last_became_active":"2023-10-18T12:00:21.091327+0000","last_became_peered":"2023-10-18T12:00:21.091327+0000","last_unstale":"2023-10-18T12:00:31.419081+0000","last_undegraded":"2023-10-18T12:00:31.419081+0000","last_fullsized":"2023-10-18T12:00:31.419081+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:53:58.179224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.438726+0000","last_change":"2023-10-18T12:00:32.435574+0000","last_active":"2023-10-18T12:00:32.438726+0000","last_peered":"2023-10-18T12:00:32.438726+0000","last_clean":"2023-10-18T12:00:32.438726+0000","last_became_active":"2023-10-18T12:00:32.435277+0000","last_became_peered":"2023-10-18T12:00:32.435277+0000","last_unstale":"2023-10-18T12:00:32.438726+0000","last_undegraded":"2023-10-18T12:00:32.438726+0000","last_fullsized":"2023-10-18T12:00:32.438726+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:58:25.649380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418366+0000","last_change":"2023-10-18T12:00:21.075391+0000","last_active":"2023-10-18T12:00:31.418366+0000","last_peered":"2023-10-18T12:00:31.418366+0000","last_clean":"2023-10-18T12:00:31.418366+0000","last_became_active":"2023-10-18T12:00:21.075140+0000","last_became_peered":"2023-10-18T12:00:21.075140+0000","last_unstale":"2023-10-18T12:00:31.418366+0000","last_undegraded":"2023-10-18T12:00:31.418366+0000","last_fullsized":"2023-10-18T12:00:31.418366+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:16:03.472142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.439634+0000","last_change":"2023-10-18T12:00:32.438699+0000","last_active":"2023-10-18T12:00:32.439634+0000","last_peered":"2023-10-18T12:00:32.439634+0000","last_clean":"2023-10-18T12:00:32.439634+0000","last_became_active":"2023-10-18T12:00:32.438444+0000","last_became_peered":"2023-10-18T12:00:32.438444+0000","last_unstale":"2023-10-18T12:00:32.439634+0000","last_undegraded":"2023-10-18T12:00:32.439634+0000","last_fullsized":"2023-10-18T12:00:32.439634+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:25.347761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426145+0000","last_change":"2023-10-18T12:00:21.082982+0000","last_active":"2023-10-18T12:00:31.426145+0000","last_peered":"2023-10-18T12:00:31.426145+0000","last_clean":"2023-10-18T12:00:31.426145+0000","last_became_active":"2023-10-18T12:00:21.082700+0000","last_became_peered":"2023-10-18T12:00:21.082700+0000","last_unstale":"2023-10-18T12:00:31.426145+0000","last_undegraded":"2023-10-18T12:00:31.426145+0000","last_fullsized":"2023-10-18T12:00:31.426145+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:25.713305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430319+0000","last_change":"2023-10-18T12:00:32.429033+0000","last_active":"2023-10-18T12:00:32.430319+0000","last_peered":"2023-10-18T12:00:32.430319+0000","last_clean":"2023-10-18T12:00:32.430319+0000","last_became_active":"2023-10-18T12:00:32.428732+0000","last_became_peered":"2023-10-18T12:00:32.428732+0000","last_unstale":"2023-10-18T12:00:32.430319+0000","last_undegraded":"2023-10-18T12:00:32.430319+0000","last_fullsized":"2023-10-18T12:00:32.430319+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:12.002567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.421025+0000","last_change":"2023-10-18T12:00:21.073828+0000","last_active":"2023-10-18T12:00:31.421025+0000","last_peered":"2023-10-18T12:00:31.421025+0000","last_clean":"2023-10-18T12:00:31.421025+0000","last_became_active":"2023-10-18T12:00:21.073594+0000","last_became_peered":"2023-10-18T12:00:21.073594+0000","last_unstale":"2023-10-18T12:00:31.421025+0000","last_undegraded":"2023-10-18T12:00:31.421025+0000","last_fullsized":"2023-10-18T12:00:31.421025+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:02.908869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445449+0000","last_change":"2023-10-18T12:00:32.441667+0000","last_active":"2023-10-18T12:00:32.445449+0000","last_peered":"2023-10-18T12:00:32.445449+0000","last_clean":"2023-10-18T12:00:32.445449+0000","last_became_active":"2023-10-18T12:00:32.441184+0000","last_became_peered":"2023-10-18T12:00:32.441184+0000","last_unstale":"2023-10-18T12:00:32.445449+0000","last_undegraded":"2023-10-18T12:00:32.445449+0000","last_fullsized":"2023-10-18T12:00:32.445449+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:39.793111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430664+0000","last_change":"2023-10-18T12:00:21.091079+0000","last_active":"2023-10-18T12:00:31.430664+0000","last_peered":"2023-10-18T12:00:31.430664+0000","last_clean":"2023-10-18T12:00:31.430664+0000","last_became_active":"2023-10-18T12:00:21.090440+0000","last_became_peered":"2023-10-18T12:00:21.090440+0000","last_unstale":"2023-10-18T12:00:31.430664+0000","last_undegraded":"2023-10-18T12:00:31.430664+0000","last_fullsized":"2023-10-18T12:00:31.430664+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:54.338893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.429873+0000","last_change":"2023-10-18T12:00:32.428557+0000","last_active":"2023-10-18T12:00:32.429873+0000","last_peered":"2023-10-18T12:00:32.429873+0000","last_clean":"2023-10-18T12:00:32.429873+0000","last_became_active":"2023-10-18T12:00:32.428263+0000","last_became_peered":"2023-10-18T12:00:32.428263+0000","last_unstale":"2023-10-18T12:00:32.429873+0000","last_undegraded":"2023-10-18T12:00:32.429873+0000","last_fullsized":"2023-10-18T12:00:32.429873+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:24.581272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430792+0000","last_change":"2023-10-18T12:00:21.093298+0000","last_active":"2023-10-18T12:00:31.430792+0000","last_peered":"2023-10-18T12:00:31.430792+0000","last_clean":"2023-10-18T12:00:31.430792+0000","last_became_active":"2023-10-18T12:00:21.092815+0000","last_became_peered":"2023-10-18T12:00:21.092815+0000","last_unstale":"2023-10-18T12:00:31.430792+0000","last_undegraded":"2023-10-18T12:00:31.430792+0000","last_fullsized":"2023-10-18T12:00:31.430792+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:28.526624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430396+0000","last_change":"2023-10-18T12:00:32.429617+0000","last_active":"2023-10-18T12:00:32.430396+0000","last_peered":"2023-10-18T12:00:32.430396+0000","last_clean":"2023-10-18T12:00:32.430396+0000","last_became_active":"2023-10-18T12:00:32.429363+0000","last_became_peered":"2023-10-18T12:00:32.429363+0000","last_unstale":"2023-10-18T12:00:32.430396+0000","last_undegraded":"2023-10-18T12:00:32.430396+0000","last_fullsized":"2023-10-18T12:00:32.430396+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:18.493180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.431445+0000","last_change":"2023-10-18T12:00:21.090556+0000","last_active":"2023-10-18T12:00:31.431445+0000","last_peered":"2023-10-18T12:00:31.431445+0000","last_clean":"2023-10-18T12:00:31.431445+0000","last_became_active":"2023-10-18T12:00:21.088740+0000","last_became_peered":"2023-10-18T12:00:21.088740+0000","last_unstale":"2023-10-18T12:00:31.431445+0000","last_undegraded":"2023-10-18T12:00:31.431445+0000","last_fullsized":"2023-10-18T12:00:31.431445+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:32.899439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.451755+0000","last_change":"2023-10-18T12:00:32.449838+0000","last_active":"2023-10-18T12:00:32.451755+0000","last_peered":"2023-10-18T12:00:32.451755+0000","last_clean":"2023-10-18T12:00:32.451755+0000","last_became_active":"2023-10-18T12:00:32.449575+0000","last_became_peered":"2023-10-18T12:00:32.449575+0000","last_unstale":"2023-10-18T12:00:32.451755+0000","last_undegraded":"2023-10-18T12:00:32.451755+0000","last_fullsized":"2023-10-18T12:00:32.451755+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:35.845909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.432016+0000","last_change":"2023-10-18T12:00:21.092451+0000","last_active":"2023-10-18T12:00:31.432016+0000","last_peered":"2023-10-18T12:00:31.432016+0000","last_clean":"2023-10-18T12:00:31.432016+0000","last_became_active":"2023-10-18T12:00:21.091995+0000","last_became_peered":"2023-10-18T12:00:21.091995+0000","last_unstale":"2023-10-18T12:00:31.432016+0000","last_undegraded":"2023-10-18T12:00:31.432016+0000","last_fullsized":"2023-10-18T12:00:31.432016+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:27.792233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.432619+0000","last_change":"2023-10-18T12:00:32.430743+0000","last_active":"2023-10-18T12:00:32.432619+0000","last_peered":"2023-10-18T12:00:32.432619+0000","last_clean":"2023-10-18T12:00:32.432619+0000","last_became_active":"2023-10-18T12:00:32.430469+0000","last_became_peered":"2023-10-18T12:00:32.430469+0000","last_unstale":"2023-10-18T12:00:32.432619+0000","last_undegraded":"2023-10-18T12:00:32.432619+0000","last_fullsized":"2023-10-18T12:00:32.432619+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:12.671247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418175+0000","last_change":"2023-10-18T12:00:21.076021+0000","last_active":"2023-10-18T12:00:31.418175+0000","last_peered":"2023-10-18T12:00:31.418175+0000","last_clean":"2023-10-18T12:00:31.418175+0000","last_became_active":"2023-10-18T12:00:21.075761+0000","last_became_peered":"2023-10-18T12:00:21.075761+0000","last_unstale":"2023-10-18T12:00:31.418175+0000","last_undegraded":"2023-10-18T12:00:31.418175+0000","last_fullsized":"2023-10-18T12:00:31.418175+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:25:47.728965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.446244+0000","last_change":"2023-10-18T12:00:32.445150+0000","last_active":"2023-10-18T12:00:32.446244+0000","last_peered":"2023-10-18T12:00:32.446244+0000","last_clean":"2023-10-18T12:00:32.446244+0000","last_became_active":"2023-10-18T12:00:32.443531+0000","last_became_peered":"2023-10-18T12:00:32.443531+0000","last_unstale":"2023-10-18T12:00:32.446244+0000","last_undegraded":"2023-10-18T12:00:32.446244+0000","last_fullsized":"2023-10-18T12:00:32.446244+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:08.260044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418337+0000","last_change":"2023-10-18T12:00:21.089110+0000","last_active":"2023-10-18T12:00:31.418337+0000","last_peered":"2023-10-18T12:00:31.418337+0000","last_clean":"2023-10-18T12:00:31.418337+0000","last_became_active":"2023-10-18T12:00:21.088828+0000","last_became_peered":"2023-10-18T12:00:21.088828+0000","last_unstale":"2023-10-18T12:00:31.418337+0000","last_undegraded":"2023-10-18T12:00:31.418337+0000","last_fullsized":"2023-10-18T12:00:31.418337+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:09.659998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.446154+0000","last_change":"2023-10-18T12:00:32.444890+0000","last_active":"2023-10-18T12:00:32.446154+0000","last_peered":"2023-10-18T12:00:32.446154+0000","last_clean":"2023-10-18T12:00:32.446154+0000","last_became_active":"2023-10-18T12:00:32.443733+0000","last_became_peered":"2023-10-18T12:00:32.443733+0000","last_unstale":"2023-10-18T12:00:32.446154+0000","last_undegraded":"2023-10-18T12:00:32.446154+0000","last_fullsized":"2023-10-18T12:00:32.446154+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:24.356601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418828+0000","last_change":"2023-10-18T12:00:21.082735+0000","last_active":"2023-10-18T12:00:31.418828+0000","last_peered":"2023-10-18T12:00:31.418828+0000","last_clean":"2023-10-18T12:00:31.418828+0000","last_became_active":"2023-10-18T12:00:21.082501+0000","last_became_peered":"2023-10-18T12:00:21.082501+0000","last_unstale":"2023-10-18T12:00:31.418828+0000","last_undegraded":"2023-10-18T12:00:31.418828+0000","last_fullsized":"2023-10-18T12:00:31.418828+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:11.830792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.436512+0000","last_change":"2023-10-18T12:00:32.435390+0000","last_active":"2023-10-18T12:00:32.436512+0000","last_peered":"2023-10-18T12:00:32.436512+0000","last_clean":"2023-10-18T12:00:32.436512+0000","last_became_active":"2023-10-18T12:00:32.435108+0000","last_became_peered":"2023-10-18T12:00:32.435108+0000","last_unstale":"2023-10-18T12:00:32.436512+0000","last_undegraded":"2023-10-18T12:00:32.436512+0000","last_fullsized":"2023-10-18T12:00:32.436512+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:52.662776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418091+0000","last_change":"2023-10-18T12:00:21.085027+0000","last_active":"2023-10-18T12:00:31.418091+0000","last_peered":"2023-10-18T12:00:31.418091+0000","last_clean":"2023-10-18T12:00:31.418091+0000","last_became_active":"2023-10-18T12:00:21.084752+0000","last_became_peered":"2023-10-18T12:00:21.084752+0000","last_unstale":"2023-10-18T12:00:31.418091+0000","last_undegraded":"2023-10-18T12:00:31.418091+0000","last_fullsized":"2023-10-18T12:00:31.418091+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:12:55.331050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.435671+0000","last_change":"2023-10-18T12:00:32.434238+0000","last_active":"2023-10-18T12:00:32.435671+0000","last_peered":"2023-10-18T12:00:32.435671+0000","last_clean":"2023-10-18T12:00:32.435671+0000","last_became_active":"2023-10-18T12:00:32.433870+0000","last_became_peered":"2023-10-18T12:00:32.433870+0000","last_unstale":"2023-10-18T12:00:32.435671+0000","last_undegraded":"2023-10-18T12:00:32.435671+0000","last_fullsized":"2023-10-18T12:00:32.435671+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:12.168935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.430586+0000","last_change":"2023-10-18T12:00:21.091256+0000","last_active":"2023-10-18T12:00:31.430586+0000","last_peered":"2023-10-18T12:00:31.430586+0000","last_clean":"2023-10-18T12:00:31.430586+0000","last_became_active":"2023-10-18T12:00:21.090808+0000","last_became_peered":"2023-10-18T12:00:21.090808+0000","last_unstale":"2023-10-18T12:00:31.430586+0000","last_undegraded":"2023-10-18T12:00:31.430586+0000","last_fullsized":"2023-10-18T12:00:31.430586+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:53:42.580058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.427205+0000","last_change":"2023-10-18T12:00:32.425449+0000","last_active":"2023-10-18T12:00:32.427205+0000","last_peered":"2023-10-18T12:00:32.427205+0000","last_clean":"2023-10-18T12:00:32.427205+0000","last_became_active":"2023-10-18T12:00:32.425207+0000","last_became_peered":"2023-10-18T12:00:32.425207+0000","last_unstale":"2023-10-18T12:00:32.427205+0000","last_undegraded":"2023-10-18T12:00:32.427205+0000","last_fullsized":"2023-10-18T12:00:32.427205+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:26.575385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.424725+0000","last_change":"2023-10-18T12:00:21.084368+0000","last_active":"2023-10-18T12:00:31.424725+0000","last_peered":"2023-10-18T12:00:31.424725+0000","last_clean":"2023-10-18T12:00:31.424725+0000","last_became_active":"2023-10-18T12:00:21.084087+0000","last_became_peered":"2023-10-18T12:00:21.084087+0000","last_unstale":"2023-10-18T12:00:31.424725+0000","last_undegraded":"2023-10-18T12:00:31.424725+0000","last_fullsized":"2023-10-18T12:00:31.424725+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:26.534619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.432641+0000","last_change":"2023-10-18T12:00:32.430200+0000","last_active":"2023-10-18T12:00:32.432641+0000","last_peered":"2023-10-18T12:00:32.432641+0000","last_clean":"2023-10-18T12:00:32.432641+0000","last_became_active":"2023-10-18T12:00:32.429934+0000","last_became_peered":"2023-10-18T12:00:32.429934+0000","last_unstale":"2023-10-18T12:00:32.432641+0000","last_undegraded":"2023-10-18T12:00:32.432641+0000","last_fullsized":"2023-10-18T12:00:32.432641+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:40:58.139167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426934+0000","last_change":"2023-10-18T12:00:21.087044+0000","last_active":"2023-10-18T12:00:31.426934+0000","last_peered":"2023-10-18T12:00:31.426934+0000","last_clean":"2023-10-18T12:00:31.426934+0000","last_became_active":"2023-10-18T12:00:21.086784+0000","last_became_peered":"2023-10-18T12:00:21.086784+0000","last_unstale":"2023-10-18T12:00:31.426934+0000","last_undegraded":"2023-10-18T12:00:31.426934+0000","last_fullsized":"2023-10-18T12:00:31.426934+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:16.398494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.430360+0000","last_change":"2023-10-18T12:00:32.427873+0000","last_active":"2023-10-18T12:00:32.430360+0000","last_peered":"2023-10-18T12:00:32.430360+0000","last_clean":"2023-10-18T12:00:32.430360+0000","last_became_active":"2023-10-18T12:00:32.427634+0000","last_became_peered":"2023-10-18T12:00:32.427634+0000","last_unstale":"2023-10-18T12:00:32.430360+0000","last_undegraded":"2023-10-18T12:00:32.430360+0000","last_fullsized":"2023-10-18T12:00:32.430360+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:38:51.295444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419521+0000","last_change":"2023-10-18T12:00:21.097042+0000","last_active":"2023-10-18T12:00:31.419521+0000","last_peered":"2023-10-18T12:00:31.419521+0000","last_clean":"2023-10-18T12:00:31.419521+0000","last_became_active":"2023-10-18T12:00:21.096759+0000","last_became_peered":"2023-10-18T12:00:21.096759+0000","last_unstale":"2023-10-18T12:00:31.419521+0000","last_undegraded":"2023-10-18T12:00:31.419521+0000","last_fullsized":"2023-10-18T12:00:31.419521+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:28:41.300175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.454607+0000","last_change":"2023-10-18T12:00:32.453930+0000","last_active":"2023-10-18T12:00:32.454607+0000","last_peered":"2023-10-18T12:00:32.454607+0000","last_clean":"2023-10-18T12:00:32.454607+0000","last_became_active":"2023-10-18T12:00:32.453641+0000","last_became_peered":"2023-10-18T12:00:32.453641+0000","last_unstale":"2023-10-18T12:00:32.454607+0000","last_undegraded":"2023-10-18T12:00:32.454607+0000","last_fullsized":"2023-10-18T12:00:32.454607+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:32.455890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.432374+0000","last_change":"2023-10-18T12:00:21.091511+0000","last_active":"2023-10-18T12:00:31.432374+0000","last_peered":"2023-10-18T12:00:31.432374+0000","last_clean":"2023-10-18T12:00:31.432374+0000","last_became_active":"2023-10-18T12:00:21.091236+0000","last_became_peered":"2023-10-18T12:00:21.091236+0000","last_unstale":"2023-10-18T12:00:31.432374+0000","last_undegraded":"2023-10-18T12:00:31.432374+0000","last_fullsized":"2023-10-18T12:00:31.432374+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:27.189758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.433034+0000","last_change":"2023-10-18T12:00:32.431957+0000","last_active":"2023-10-18T12:00:32.433034+0000","last_peered":"2023-10-18T12:00:32.433034+0000","last_clean":"2023-10-18T12:00:32.433034+0000","last_became_active":"2023-10-18T12:00:32.431666+0000","last_became_peered":"2023-10-18T12:00:32.431666+0000","last_unstale":"2023-10-18T12:00:32.433034+0000","last_undegraded":"2023-10-18T12:00:32.433034+0000","last_fullsized":"2023-10-18T12:00:32.433034+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:32:59.350342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.418257+0000","last_change":"2023-10-18T12:00:21.084370+0000","last_active":"2023-10-18T12:00:31.418257+0000","last_peered":"2023-10-18T12:00:31.418257+0000","last_clean":"2023-10-18T12:00:31.418257+0000","last_became_active":"2023-10-18T12:00:21.082951+0000","last_became_peered":"2023-10-18T12:00:21.082951+0000","last_unstale":"2023-10-18T12:00:31.418257+0000","last_undegraded":"2023-10-18T12:00:31.418257+0000","last_fullsized":"2023-10-18T12:00:31.418257+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:42:38.614585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.435958+0000","last_change":"2023-10-18T12:00:32.434925+0000","last_active":"2023-10-18T12:00:32.435958+0000","last_peered":"2023-10-18T12:00:32.435958+0000","last_clean":"2023-10-18T12:00:32.435958+0000","last_became_active":"2023-10-18T12:00:32.434647+0000","last_became_peered":"2023-10-18T12:00:32.434647+0000","last_unstale":"2023-10-18T12:00:32.435958+0000","last_undegraded":"2023-10-18T12:00:32.435958+0000","last_fullsized":"2023-10-18T12:00:32.435958+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:55.700931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.429699+0000","last_change":"2023-10-18T12:00:21.085280+0000","last_active":"2023-10-18T12:00:31.429699+0000","last_peered":"2023-10-18T12:00:31.429699+0000","last_clean":"2023-10-18T12:00:31.429699+0000","last_became_active":"2023-10-18T12:00:21.085007+0000","last_became_peered":"2023-10-18T12:00:21.085007+0000","last_unstale":"2023-10-18T12:00:31.429699+0000","last_undegraded":"2023-10-18T12:00:31.429699+0000","last_fullsized":"2023-10-18T12:00:31.429699+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:00:11.590501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.445527+0000","last_change":"2023-10-18T12:00:32.441884+0000","last_active":"2023-10-18T12:00:32.445527+0000","last_peered":"2023-10-18T12:00:32.445527+0000","last_clean":"2023-10-18T12:00:32.445527+0000","last_became_active":"2023-10-18T12:00:32.441170+0000","last_became_peered":"2023-10-18T12:00:32.441170+0000","last_unstale":"2023-10-18T12:00:32.445527+0000","last_undegraded":"2023-10-18T12:00:32.445527+0000","last_fullsized":"2023-10-18T12:00:32.445527+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.426467+0000","last_change":"2023-10-18T12:00:21.081320+0000","last_active":"2023-10-18T12:00:31.426467+0000","last_peered":"2023-10-18T12:00:31.426467+0000","last_clean":"2023-10-18T12:00:31.426467+0000","last_became_active":"2023-10-18T12:00:21.080987+0000","last_became_peered":"2023-10-18T12:00:21.080987+0000","last_unstale":"2023-10-18T12:00:31.426467+0000","last_undegraded":"2023-10-18T12:00:31.426467+0000","last_fullsized":"2023-10-18T12:00:31.426467+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:10:51.800092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.435922+0000","last_change":"2023-10-18T12:00:32.434522+0000","last_active":"2023-10-18T12:00:32.435922+0000","last_peered":"2023-10-18T12:00:32.435922+0000","last_clean":"2023-10-18T12:00:32.435922+0000","last_became_active":"2023-10-18T12:00:32.434236+0000","last_became_peered":"2023-10-18T12:00:32.434236+0000","last_unstale":"2023-10-18T12:00:32.435922+0000","last_undegraded":"2023-10-18T12:00:32.435922+0000","last_fullsized":"2023-10-18T12:00:32.435922+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:38.936712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.423907+0000","last_change":"2023-10-18T12:00:21.083829+0000","last_active":"2023-10-18T12:00:31.423907+0000","last_peered":"2023-10-18T12:00:31.423907+0000","last_clean":"2023-10-18T12:00:31.423907+0000","last_became_active":"2023-10-18T12:00:21.083554+0000","last_became_peered":"2023-10-18T12:00:21.083554+0000","last_unstale":"2023-10-18T12:00:31.423907+0000","last_undegraded":"2023-10-18T12:00:31.423907+0000","last_fullsized":"2023-10-18T12:00:31.423907+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:34.556666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.437583+0000","last_change":"2023-10-18T12:00:32.434340+0000","last_active":"2023-10-18T12:00:32.437583+0000","last_peered":"2023-10-18T12:00:32.437583+0000","last_clean":"2023-10-18T12:00:32.437583+0000","last_became_active":"2023-10-18T12:00:32.434110+0000","last_became_peered":"2023-10-18T12:00:32.434110+0000","last_unstale":"2023-10-18T12:00:32.437583+0000","last_undegraded":"2023-10-18T12:00:32.437583+0000","last_fullsized":"2023-10-18T12:00:32.437583+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:08.260044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.427238+0000","last_change":"2023-10-18T12:00:21.081967+0000","last_active":"2023-10-18T12:00:31.427238+0000","last_peered":"2023-10-18T12:00:31.427238+0000","last_clean":"2023-10-18T12:00:31.427238+0000","last_became_active":"2023-10-18T12:00:21.081693+0000","last_became_peered":"2023-10-18T12:00:21.081693+0000","last_unstale":"2023-10-18T12:00:31.427238+0000","last_undegraded":"2023-10-18T12:00:31.427238+0000","last_fullsized":"2023-10-18T12:00:31.427238+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:47:47.036558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.434495+0000","last_change":"2023-10-18T12:00:32.431303+0000","last_active":"2023-10-18T12:00:32.434495+0000","last_peered":"2023-10-18T12:00:32.434495+0000","last_clean":"2023-10-18T12:00:32.434495+0000","last_became_active":"2023-10-18T12:00:32.431062+0000","last_became_peered":"2023-10-18T12:00:32.431062+0000","last_unstale":"2023-10-18T12:00:32.434495+0000","last_undegraded":"2023-10-18T12:00:32.434495+0000","last_fullsized":"2023-10-18T12:00:32.434495+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:14.318910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.425425+0000","last_change":"2023-10-18T12:00:21.087099+0000","last_active":"2023-10-18T12:00:31.425425+0000","last_peered":"2023-10-18T12:00:31.425425+0000","last_clean":"2023-10-18T12:00:31.425425+0000","last_became_active":"2023-10-18T12:00:21.086806+0000","last_became_peered":"2023-10-18T12:00:21.086806+0000","last_unstale":"2023-10-18T12:00:31.425425+0000","last_undegraded":"2023-10-18T12:00:31.425425+0000","last_fullsized":"2023-10-18T12:00:31.425425+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:54:12.691864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.448801+0000","last_change":"2023-10-18T12:00:32.446320+0000","last_active":"2023-10-18T12:00:32.448801+0000","last_peered":"2023-10-18T12:00:32.448801+0000","last_clean":"2023-10-18T12:00:32.448801+0000","last_became_active":"2023-10-18T12:00:32.443145+0000","last_became_peered":"2023-10-18T12:00:32.443145+0000","last_unstale":"2023-10-18T12:00:32.448801+0000","last_undegraded":"2023-10-18T12:00:32.448801+0000","last_fullsized":"2023-10-18T12:00:32.448801+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:28:23.492154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419438+0000","last_change":"2023-10-18T12:00:21.084830+0000","last_active":"2023-10-18T12:00:31.419438+0000","last_peered":"2023-10-18T12:00:31.419438+0000","last_clean":"2023-10-18T12:00:31.419438+0000","last_became_active":"2023-10-18T12:00:21.084546+0000","last_became_peered":"2023-10-18T12:00:21.084546+0000","last_unstale":"2023-10-18T12:00:31.419438+0000","last_undegraded":"2023-10-18T12:00:31.419438+0000","last_fullsized":"2023-10-18T12:00:31.419438+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:18.371937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.433081+0000","last_change":"2023-10-18T12:00:32.430679+0000","last_active":"2023-10-18T12:00:32.433081+0000","last_peered":"2023-10-18T12:00:32.433081+0000","last_clean":"2023-10-18T12:00:32.433081+0000","last_became_active":"2023-10-18T12:00:32.428990+0000","last_became_peered":"2023-10-18T12:00:32.428990+0000","last_unstale":"2023-10-18T12:00:32.433081+0000","last_undegraded":"2023-10-18T12:00:32.433081+0000","last_fullsized":"2023-10-18T12:00:32.433081+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:25.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.420926+0000","last_change":"2023-10-18T12:00:21.089403+0000","last_active":"2023-10-18T12:00:31.420926+0000","last_peered":"2023-10-18T12:00:31.420926+0000","last_clean":"2023-10-18T12:00:31.420926+0000","last_became_active":"2023-10-18T12:00:21.089138+0000","last_became_peered":"2023-10-18T12:00:21.089138+0000","last_unstale":"2023-10-18T12:00:31.420926+0000","last_undegraded":"2023-10-18T12:00:31.420926+0000","last_fullsized":"2023-10-18T12:00:31.420926+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:38:59.178484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":224,"state":"active+clean","last_fresh":"2023-10-18T12:00:32.431239+0000","last_change":"2023-10-18T12:00:32.429453+0000","last_active":"2023-10-18T12:00:32.431239+0000","last_peered":"2023-10-18T12:00:32.431239+0000","last_clean":"2023-10-18T12:00:32.431239+0000","last_became_active":"2023-10-18T12:00:32.429193+0000","last_became_peered":"2023-10-18T12:00:32.429193+0000","last_unstale":"2023-10-18T12:00:32.431239+0000","last_undegraded":"2023-10-18T12:00:32.431239+0000","last_fullsized":"2023-10-18T12:00:32.431239+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":223,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:31.406627+0000","last_clean_scrub_stamp":"2023-10-18T12:00:31.406627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:10.331116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2023-10-18T12:00:31.419643+0000","last_change":"2023-10-18T12:00:21.094990+0000","last_active":"2023-10-18T12:00:31.419643+0000","last_peered":"2023-10-18T12:00:31.419643+0000","last_clean":"2023-10-18T12:00:31.419643+0000","last_became_active":"2023-10-18T12:00:21.094239+0000","last_became_peered":"2023-10-18T12:00:21.094239+0000","last_unstale":"2023-10-18T12:00:31.419643+0000","last_undegraded":"2023-10-18T12:00:31.419643+0000","last_fullsized":"2023-10-18T12:00:31.419643+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":219,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:00:20.044026+0000","last_clean_scrub_stamp":"2023-10-18T12:00:20.044026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:22.792096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2210,"num_read_kb":4617,"num_write":595,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575992,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52580,"kb_used_data":3352,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319260,"statfs":{"total":96636764160,"available":96582922240,"internally_reserved":0,"allocated":3432448,"data_stored":2037964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.40899999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.60499999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.373},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.46800000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.496},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.61799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575992,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42256,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329584,"statfs":{"total":96636764160,"available":96593494016,"internally_reserved":0,"allocated":2818048,"data_stored":1439415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.39100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.53800000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.39600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":0.81899999999999995,"15min":0.85699999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45200000000000001,"5min":0.43099999999999999,"15min":0.39200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.87},"last":0.64200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71199999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.82599999999999996},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.83999999999999997,"15min":0.85499999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81100000000000005,"5min":0.82499999999999996,"15min":0.84699999999999998},"last":0.73999999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77300000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.872,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76100000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85299999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60296,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311544,"statfs":{"total":96636764160,"available":96575021056,"internally_reserved":0,"allocated":2809856,"data_stored":1436021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72199999999999998,"5min":0.92400000000000004,"15min":0.96099999999999997},"last":0.621}]},{"osd":1,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.745,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.52500000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.755,"5min":0.80500000000000005,"15min":1.0349999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.45000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.59299999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":1},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52100000000000002,"15min":0.53800000000000003},"min":{"1min":0.377,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.01},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":1.101,"15min":1.101},"last":0.746},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81499999999999995,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575992,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47832,"kb_used_data":2756,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324008,"statfs":{"total":96636764160,"available":96587784192,"internally_reserved":0,"allocated":2822144,"data_stored":1444658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.629,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.22500000000000001},"max":{"1min":0.73899999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.438},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.33600000000000002,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.80200000000000005,"15min":0.81599999999999995},"last":0.504}]},{"osd":1,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.33600000000000002},"max":{"1min":0.86899999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.31900000000000001},"max":{"1min":0.76700000000000002,"5min":1.123,"15min":1.123},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.625,"15min":0.621},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.627},"min":{"1min":0.434,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.81999999999999995,"15min":0.83999999999999997},"last":0.60799999999999998}]},{"osd":3,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.746,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.40300000000000002},"max":{"1min":0.84399999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.51100000000000001,"15min":0.498},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66000000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.371},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.312},"max":{"1min":0.73199999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.34100000000000003}]},{"osd":6,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54300000000000004,"15min":0.52800000000000002},"min":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.81499999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.39000000000000001,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.68400000000000005,"5min":1.018,"15min":1.018},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Oct 18 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.76100000000000001,"5min":1.004,"15min":1.05},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575992,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47840,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324000,"statfs":{"total":96636764160,"available":96587776000,"internally_reserved":0,"allocated":2830336,"data_stored":1444134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.23200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55000000000000004,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.378},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.41399999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23400000000000001},"max":{"1min":0.56699999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.29499999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41599999999999998,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.44500000000000001},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":1.899,"15min":1.899},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.29299999999999998,"15min":0.28399999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88400000000000001,"5min":1.034,"15min":1.052},"last":0.625},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.522,"15min":1.522},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.89600000000000002,"5min":0.91100000000000003,"15min":1.026},"last":0.66100000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":1.01,"15min":1.01},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67100000000000004,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81799999999999995,"5min":1.129,"15min":1.129},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.59999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575992,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60292,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311548,"statfs":{"total":96636764160,"available":96575025152,"internally_reserved":0,"allocated":2805760,"data_stored":1436055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.216},"max":{"1min":0.56999999999999995,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.314}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.443},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.27700000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65300000000000002,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.087,"15min":1.087},"last":0.46899999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66900000000000004,"15min":0.67400000000000004},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92000000000000004},"last":0.52900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60300,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311540,"statfs":{"total":96636764160,"available":96575016960,"internally_reserved":0,"allocated":2813952,"data_stored":1443986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.44600000000000001,"15min":0.435},"min":{"1min":0.25800000000000001,"5min":0.219,"15min":0.182},"max":{"1min":0.65200000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.217},{"interface":"front","average":{"1min":0.441,"5min":0.379,"15min":0.38},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.82599999999999996},"last":0.29299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.23100000000000001},"max":{"1min":0.56699999999999995,"5min":0.74099999999999999,"15min":0.81000000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40699999999999997},"min":{"1min":0.34699999999999998,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.57699999999999996,"5min":0.68999999999999995,"15min":0.877},"last":0.251}]},{"osd":3,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.76700000000000002,"5min":0.80900000000000005,"15min":0.92000000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58799999999999997,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52200000000000002,"15min":0.505},"min":{"1min":0.46800000000000003,"5min":0.31900000000000001,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":0.82899999999999996,"15min":0.98599999999999999},"last":0.317},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.62,"15min":0.58599999999999997},"min":{"1min":0.52500000000000002,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.77900000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.44800000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.29599999999999999},"max":{"1min":0.83999999999999997,"5min":0.87,"15min":1.004},"last":0.495},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.51600000000000001,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.76100000000000001,"5min":0.91500000000000004,"15min":1.05},"last":0.376}]},{"osd":6,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.53400000000000003,"5min":0.379,"15min":0.376},"max":{"1min":0.83499999999999996,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.46300000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:00:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.39400000000000002},"max":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575991,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42824,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329016,"statfs":{"total":96636764160,"available":96592912384,"internally_reserved":0,"allocated":3399680,"data_stored":2026423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54700000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.32300000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61899999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28899999999999998,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.66800000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.25700000000000001},"max":{"1min":0.62,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.56899999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39800000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96999999999999997,"15min":0.997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.876,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88200000000000001,"5min":0.95799999999999996,"15min":0.96399999999999997},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66900000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":1.0089999999999999,"15min":1.03},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.45800000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65800000000000003,"15min":0.68100000000000005},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.82699999999999996,"5min":0.997,"15min":0.997},"last":0.57999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:00:35.729 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T12:00:35.729 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:35.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.857+0000 7f3b2276e640 1 -- 172.21.15.125:0/776165652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b1c0ceb40 msgr2=0x7f3b1c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.857+0000 7f3b2276e640 1 --2- 172.21.15.125:0/776165652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b1c0ceb40 0x7f3b1c0b0650 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005fd0 tx=0x7f3b1402c240 comp rx=0 tx=0).stop 2023-10-18T12:00:35.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.857+0000 7f3b2276e640 1 -- 172.21.15.125:0/776165652 shutdown_connections 2023-10-18T12:00:35.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.857+0000 7f3b2276e640 1 --2- 172.21.15.125:0/776165652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b1c0ceb40 0x7f3b1c0b0650 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.857+0000 7f3b2276e640 1 --2- 172.21.15.125:0/776165652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b1c15b2a0 0x7f3b1c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.857+0000 7f3b2276e640 1 -- 172.21.15.125:0/776165652 >> 172.21.15.125:0/776165652 conn(0x7f3b1c0d0d60 msgr2=0x7f3b1c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.858+0000 7f3b2276e640 1 -- 172.21.15.125:0/776165652 shutdown_connections 2023-10-18T12:00:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.858+0000 7f3b2276e640 1 -- 172.21.15.125:0/776165652 wait complete. 2023-10-18T12:00:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.859+0000 7f3b2276e640 1 Processor -- start 2023-10-18T12:00:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.859+0000 7f3b2276e640 1 -- start start 2023-10-18T12:00:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.859+0000 7f3b2276e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b1c0ceb40 0x7f3b1c0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.859+0000 7f3b2276e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b1c0cf340 0x7f3b1c157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.859+0000 7f3b2276e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b1c15b2a0 0x7f3b1c1e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.859+0000 7f3b2276e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c158480 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.859+0000 7f3b2276e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1e3fd0 con 0x7f3b1c0cf340 2023-10-18T12:00:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.859+0000 7f3b2276e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1e5120 con 0x7f3b1c0ceb40 2023-10-18T12:00:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b1c15b2a0 0x7f3b1c1e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b1c15b2a0 0x7f3b1c1e4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32912/0 (socket says 172.21.15.125:32912) 2023-10-18T12:00:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 -- 172.21.15.125:0/48417392 learned_addr learned my addr 172.21.15.125:0/48417392 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b1b7fe640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b1c0cf340 0x7f3b1c157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b1bfff640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b1c0ceb40 0x7f3b1c0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 -- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b1c0ceb40 msgr2=0x7f3b1c0ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b1c0ceb40 0x7f3b1c0ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 -- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b1c0cf340 msgr2=0x7f3b1c157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:35.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b1c0cf340 0x7f3b1c157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:35.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 -- 172.21.15.125:0/48417392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b14009900 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b20d0b640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b1c15b2a0 0x7f3b1c1e4be0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f3b14037040 tx=0x7f3b14035a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:35.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b18ff9640 1 -- 172.21.15.125:0/48417392 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14034440 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b18ff9640 1 -- 172.21.15.125:0/48417392 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b14004330 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b18ff9640 1 -- 172.21.15.125:0/48417392 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140046c0 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c1e5380 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.860+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c1e3980 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.861+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c1e4160 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.862+0000 7f3b18ff9640 1 -- 172.21.15.125:0/48417392 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3b1403b040 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.862+0000 7f3b18ff9640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3b0806af90 0x7f3b0806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:35.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.862+0000 7f3b18ff9640 1 -- 172.21.15.125:0/48417392 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f3b140ac600 con 0x7f3b1c15b2a0 2023-10-18T12:00:35.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.862+0000 7f3b1bfff640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3b0806af90 0x7f3b0806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:35.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.863+0000 7f3b1bfff640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3b0806af90 0x7f3b0806d450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0afb20 tx=0x7f3b0c009040 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:35.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:35.866+0000 7f3b18ff9640 1 -- 172.21.15.125:0/48417392 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b14077170 con 0x7f3b1c15b2a0 2023-10-18T12:00:36.040 INFO:tasks.ceph.mds.d.smithi139.stderr:2023-10-18T12:00:36.037+0000 7f329c96f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:00:36.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.067+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b1c0cb980 con 0x7f3b1c15b2a0 2023-10-18T12:00:36.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.067+0000 7f3b18ff9640 1 -- 172.21.15.125:0/48417392 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+6155 (secure 0 0 0) 0x7f3b1407a790 con 0x7f3b1c15b2a0 2023-10-18T12:00:36.070 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:36.071 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":90,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:20.388305+0000","modified":"2023-10-18T12:00:22.406153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11976":{"gid":11976,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2975172452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2975172452},{"type":"v1","addr":"172.21.15.125:6834","nonce":2975172452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:36.071 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 90 2023-10-18T12:00:36.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3b0806af90 msgr2=0x7f3b0806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3b0806af90 0x7f3b0806d450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0afb20 tx=0x7f3b0c009040 comp rx=0 tx=0).stop 2023-10-18T12:00:36.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b1c15b2a0 msgr2=0x7f3b1c1e4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b1c15b2a0 0x7f3b1c1e4be0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f3b14037040 tx=0x7f3b14035a20 comp rx=0 tx=0).stop 2023-10-18T12:00:36.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 shutdown_connections 2023-10-18T12:00:36.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f3b0806af90 0x7f3b0806d450 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b1c0cf340 0x7f3b1c157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b1c15b2a0 0x7f3b1c1e4be0 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 --2- 172.21.15.125:0/48417392 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b1c0ceb40 0x7f3b1c0ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 >> 172.21.15.125:0/48417392 conn(0x7f3b1c0d0d60 msgr2=0x7f3b1c0ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 shutdown_connections 2023-10-18T12:00:36.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.069+0000 7f3b2276e640 1 -- 172.21.15.125:0/48417392 wait complete. 2023-10-18T12:00:36.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T12:00:31.751249+0000', 'modified': '2023-10-18T12:00:33.761902+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10168}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10168': {'gid': 10168, 'name': 'd', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.139:6837/1825099907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6835', 'nonce': 1825099907}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 1825099907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T12:00:36.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T12:00:36.486 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-18T12:00:36.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.588+0000 7f64d47f0640 1 -- 172.21.15.125:0/2968668052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b8005180 msgr2=0x7f64b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.588+0000 7f64d47f0640 1 --2- 172.21.15.125:0/2968668052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b8005180 0x7f64b800b980 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f64c4001d80 tx=0x7f64b80057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.588+0000 7f64d47f0640 1 -- 172.21.15.125:0/2968668052 shutdown_connections 2023-10-18T12:00:36.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.588+0000 7f64d47f0640 1 --2- 172.21.15.125:0/2968668052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64b8005180 0x7f64b800b980 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.588+0000 7f64d47f0640 1 -- 172.21.15.125:0/2968668052 >> 172.21.15.125:0/2968668052 conn(0x7f64cc069930 msgr2=0x7f64cc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.588+0000 7f64d47f0640 1 -- 172.21.15.125:0/2968668052 shutdown_connections 2023-10-18T12:00:36.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.588+0000 7f64d47f0640 1 -- 172.21.15.125:0/2968668052 wait complete. 2023-10-18T12:00:36.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.589+0000 7f64d47f0640 1 Processor -- start 2023-10-18T12:00:36.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d47f0640 1 -- start start 2023-10-18T12:00:36.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d47f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64b8005180 0x7f64cc13b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d47f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64cc1dd7a0 0x7f64cc0a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d47f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64cc1e25f0 0x7f64cc13d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d47f0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc13b7f0 con 0x7f64cc1e25f0 2023-10-18T12:00:36.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d47f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1e12b0 con 0x7f64b8005180 2023-10-18T12:00:36.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d47f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1e1550 con 0x7f64cc1dd7a0 2023-10-18T12:00:36.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d2d8d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64cc1e25f0 0x7f64cc13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d2d8d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64cc1e25f0 0x7f64cc13d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32928/0 (socket says 172.21.15.125:32928) 2023-10-18T12:00:36.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d2d8d640 1 -- 172.21.15.125:0/2980556391 learned_addr learned my addr 172.21.15.125:0/2980556391 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:36.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d258c640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64b8005180 0x7f64cc13b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.590+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64cc1dd7a0 0x7f64cc0a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64d2d8d640 1 -- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64cc1dd7a0 msgr2=0x7f64cc0a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64d2d8d640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64cc1dd7a0 0x7f64cc0a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64d2d8d640 1 -- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64b8005180 msgr2=0x7f64cc13b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64d2d8d640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64b8005180 0x7f64cc13b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64d2d8d640 1 -- 172.21.15.125:0/2980556391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc1e1830 con 0x7f64cc1e25f0 2023-10-18T12:00:36.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64d2d8d640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64cc1e25f0 0x7f64cc13d040 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f64c402c530 tx=0x7f64c402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:36.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64c2ffd640 1 -- 172.21.15.125:0/2980556391 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40062d0 con 0x7f64cc1e25f0 2023-10-18T12:00:36.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64c2ffd640 1 -- 172.21.15.125:0/2980556391 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64c4006520 con 0x7f64cc1e25f0 2023-10-18T12:00:36.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64c2ffd640 1 -- 172.21.15.125:0/2980556391 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4034410 con 0x7f64cc1e25f0 2023-10-18T12:00:36.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64d47f0640 1 -- 172.21.15.125:0/2980556391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc1e1b10 con 0x7f64cc1e25f0 2023-10-18T12:00:36.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.591+0000 7f64d47f0640 1 -- 172.21.15.125:0/2980556391 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc1e20d0 con 0x7f64cc1e25f0 2023-10-18T12:00:36.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.592+0000 7f64c2ffd640 1 -- 172.21.15.125:0/2980556391 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f64c4034570 con 0x7f64cc1e25f0 2023-10-18T12:00:36.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.593+0000 7f64c2ffd640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f64b006af60 0x7f64b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.593+0000 7f64c2ffd640 1 -- 172.21.15.125:0/2980556391 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f64c40ab170 con 0x7f64cc1e25f0 2023-10-18T12:00:36.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.593+0000 7f64d258c640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f64b006af60 0x7f64b006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.594+0000 7f64d258c640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f64b006af60 0x7f64b006d420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009b10 tx=0x7f64c8005f90 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:36.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.594+0000 7f64d737c740 1 Processor -- start 2023-10-18T12:00:36.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.594+0000 7f64d737c740 1 -- start start 2023-10-18T12:00:36.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d737c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d71280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d737c740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d2abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d737c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d2b130 0x55c378d558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d737c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378c37730 con 0x55c378d736c0 2023-10-18T12:00:36.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d737c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378c98780 con 0x55c378d2b130 2023-10-18T12:00:36.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d737c740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c378c95e70 con 0x55c378d52f00 2023-10-18T12:00:36.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d71280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d71280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32942/0 (socket says 172.21.15.125:32942) 2023-10-18T12:00:36.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 -- 172.21.15.125:0/1989252968 learned_addr learned my addr 172.21.15.125:0/1989252968 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:36.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d2abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d2d8d640 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d2b130 0x55c378d558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 -- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 msgr2=0x55c378d2abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:36.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d2abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 -- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d2b130 msgr2=0x55c378d558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:36.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d2b130 0x55c378d558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 -- 172.21.15.125:0/1989252968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c378c963d0 con 0x55c378d736c0 2023-10-18T12:00:36.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d2d8d640 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d2b130 0x55c378d558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:36.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d2abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:36.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64d258c640 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d71280 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f64c8013010 tx=0x7f64c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f656420f58c3665 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:36.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64affff640 1 -- 172.21.15.125:0/1989252968 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8017070 con 0x55c378d736c0 2023-10-18T12:00:36.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64affff640 1 -- 172.21.15.125:0/1989252968 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64c8003b70 con 0x55c378d736c0 2023-10-18T12:00:36.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.595+0000 7f64affff640 1 -- 172.21.15.125:0/1989252968 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80097c0 con 0x55c378d736c0 2023-10-18T12:00:36.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 -- 172.21.15.125:0/1989252968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 msgr2=0x55c378d71280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d71280 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f64c8013010 tx=0x7f64c8017000 comp rx=0 tx=0).stop 2023-10-18T12:00:36.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 -- 172.21.15.125:0/1989252968 shutdown_connections 2023-10-18T12:00:36.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d2b130 0x55c378d558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d2abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 --2- 172.21.15.125:0/1989252968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d71280 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 -- 172.21.15.125:0/1989252968 >> 172.21.15.125:0/1989252968 conn(0x55c378d27d50 msgr2=0x55c378d708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 -- 172.21.15.125:0/1989252968 shutdown_connections 2023-10-18T12:00:36.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.596+0000 7f64d737c740 1 -- 172.21.15.125:0/1989252968 wait complete. 2023-10-18T12:00:36.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.597+0000 7f64d737c740 1 -- start start 2023-10-18T12:00:36.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.597+0000 7f64d737c740 1 Processor -- start 2023-10-18T12:00:36.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d6c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d58ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d6c820 0x55c378d59530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378c91760 con 0x55c378d736c0 2023-10-18T12:00:36.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378c4c8f0 con 0x55c378d6c820 2023-10-18T12:00:36.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c378c4c590 con 0x55c378d52f00 2023-10-18T12:00:36.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 10 client.0 _ll_register_callbacks cb 0x55c378c88ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:00:36.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d58ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d58ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32946/0 (socket says 172.21.15.125:32946) 2023-10-18T12:00:36.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 -- 172.21.15.125:0/3320546925 learned_addr learned my addr 172.21.15.125:0/3320546925 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:36.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d258c640 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d6c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d2d8d640 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d6c820 0x55c378d59530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 -- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 msgr2=0x55c378d6c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:36.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d6c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 -- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d6c820 msgr2=0x55c378d59530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:36.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d6c820 0x55c378d59530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 -- 172.21.15.125:0/3320546925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c8012c90 con 0x55c378d736c0 2023-10-18T12:00:36.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d258c640 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d6c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:36.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d58ff0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006380 tx=0x7f64bc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a58df8e1823368 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:36.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:00:36.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64ad7fa640 1 -- 172.21.15.125:0/3320546925 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc012850 con 0x55c378d736c0 2023-10-18T12:00:36.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64ad7fa640 1 -- 172.21.15.125:0/3320546925 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f64bc018d80 con 0x55c378d736c0 2023-10-18T12:00:36.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 1 -- 172.21.15.125:0/3320546925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c378c8a740 con 0x55c378d736c0 2023-10-18T12:00:36.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64ad7fa640 1 -- 172.21.15.125:0/3320546925 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64bc006590 con 0x55c378d736c0 2023-10-18T12:00:36.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 10 client.12264 fetch_fsmap learned FSMap version 90 2023-10-18T12:00:36.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.598+0000 7f64d737c740 1 -- 172.21.15.125:0/3320546925 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55c378c8fde0 con 0x55c378d736c0 2023-10-18T12:00:36.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64ad7fa640 1 -- 172.21.15.125:0/3320546925 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55c378c8a740 con 0x55c378d736c0 2023-10-18T12:00:36.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64ad7fa640 1 -- 172.21.15.125:0/3320546925 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 90) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f64bc01b070 con 0x55c378d736c0 2023-10-18T12:00:36.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64d737c740 10 client.12264 fetch_fsmap finished waiting for FSMap version 90 2023-10-18T12:00:36.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64d737c740 10 client.12264 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-18T12:00:36.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64d737c740 20 client.12264 populate_metadata read hostname 'smithi125' 2023-10-18T12:00:36.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64d737c740 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] conn(0x55c378d6f090 0x55c378d6f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64d737c740 4 client.12264 mds_command: new command op to 10168 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:00:36.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64d737c740 1 -- 172.21.15.125:0/3320546925 --> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c378c8a460 con 0x55c378d6f090 2023-10-18T12:00:36.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64d258c640 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] conn(0x55c378d6f090 0x55c378d6f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64d258c640 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] conn(0x55c378d6f090 0x55c378d6f470 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e3fad5d143c715 server_cookie=5a63d3a136283480 in_seq=0 out_seq=0 2023-10-18T12:00:36.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.599+0000 7f64ad7fa640 10 client.12264 ms_handle_connect on v2:172.21.15.139:6835/1825099907 2023-10-18T12:00:36.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64ad7fa640 1 -- 172.21.15.125:0/3320546925 <== mds.0 v2:172.21.15.139:6835/1825099907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c378c8a460 con 0x55c378d6f090 2023-10-18T12:00:36.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64ad7fa640 10 client.12264 handle_command_reply: tid=0 2023-10-18T12:00:36.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 client.12264 shutdown 2023-10-18T12:00:36.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 -- 172.21.15.125:0/3320546925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 msgr2=0x55c378d58ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d58ff0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006380 tx=0x7f64bc0039c0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 -- 172.21.15.125:0/3320546925 shutdown_connections 2023-10-18T12:00:36.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] conn(0x55c378d6f090 0x55c378d6f470 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d6c820 0x55c378d59530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d736c0 0x55c378d58ff0 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 --2- 172.21.15.125:0/3320546925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d52f00 0x55c378d6c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64ad7fa640 0 client.12264 ms_handle_reset on v2:172.21.15.139:6835/1825099907 2023-10-18T12:00:36.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 -- 172.21.15.125:0/3320546925 >> 172.21.15.125:0/3320546925 conn(0x55c378d2b130 msgr2=0x55c378d24ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 -- 172.21.15.125:0/3320546925 shutdown_connections 2023-10-18T12:00:36.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.600+0000 7f64d737c740 1 -- 172.21.15.125:0/3320546925 wait complete. 2023-10-18T12:00:36.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.601+0000 7f64d737c740 20 client.12264 trim_cache size 0 max 16384 2023-10-18T12:00:36.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d737c740 1 Processor -- start 2023-10-18T12:00:36.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d737c740 1 -- start start 2023-10-18T12:00:36.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d737c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 0x55c378d719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d737c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d72530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d737c740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 0x55c378d51fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d737c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378c12270 con 0x55c378d6c820 2023-10-18T12:00:36.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d737c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378c90900 con 0x55c378d52f00 2023-10-18T12:00:36.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d737c740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c378d6b810 con 0x55c378d736c0 2023-10-18T12:00:36.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d1d8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d72530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d1d8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d72530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32962/0 (socket says 172.21.15.125:32962) 2023-10-18T12:00:36.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.616+0000 7f64d1d8b640 1 -- 172.21.15.125:0/1325300621 learned_addr learned my addr 172.21.15.125:0/1325300621 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d1d8b640 1 -- 172.21.15.125:0/1325300621 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 msgr2=0x55c378d51fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/1325300621 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 0x55c378d51fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d1d8b640 1 -- 172.21.15.125:0/1325300621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 msgr2=0x55c378d719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/1325300621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 0x55c378d719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d1d8b640 1 -- 172.21.15.125:0/1325300621 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c378c2ed00 con 0x55c378d6c820 2023-10-18T12:00:36.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/1325300621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d72530 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006eb0 tx=0x7f64bc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb91ca5c72d022a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:36.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64c0ff9640 1 -- 172.21.15.125:0/1325300621 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc006590 con 0x55c378d6c820 2023-10-18T12:00:36.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64c0ff9640 1 -- 172.21.15.125:0/1325300621 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64bc012650 con 0x55c378d6c820 2023-10-18T12:00:36.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64c0ff9640 1 -- 172.21.15.125:0/1325300621 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc01d690 con 0x55c378d6c820 2023-10-18T12:00:36.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 -- 172.21.15.125:0/1325300621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 msgr2=0x55c378d72530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 --2- 172.21.15.125:0/1325300621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d72530 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006eb0 tx=0x7f64bc001a00 comp rx=0 tx=0).stop 2023-10-18T12:00:36.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 -- 172.21.15.125:0/1325300621 shutdown_connections 2023-10-18T12:00:36.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 --2- 172.21.15.125:0/1325300621 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 0x55c378d51fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 --2- 172.21.15.125:0/1325300621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d72530 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 --2- 172.21.15.125:0/1325300621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 0x55c378d719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 -- 172.21.15.125:0/1325300621 >> 172.21.15.125:0/1325300621 conn(0x55c378d2b130 msgr2=0x55c378d2ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 -- 172.21.15.125:0/1325300621 shutdown_connections 2023-10-18T12:00:36.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.617+0000 7f64d737c740 1 -- 172.21.15.125:0/1325300621 wait complete. 2023-10-18T12:00:36.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.618+0000 7f64d737c740 1 -- start start 2023-10-18T12:00:36.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.618+0000 7f64d737c740 1 Processor -- start 2023-10-18T12:00:36.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d737c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 0x55c378d24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d737c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d25020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d737c740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 0x55c378d533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d737c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378d52020 con 0x55c378d6c820 2023-10-18T12:00:36.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d737c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378d54ba0 con 0x55c378d52f00 2023-10-18T12:00:36.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d737c740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c378d4eba0 con 0x55c378d736c0 2023-10-18T12:00:36.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d737c740 10 client.0 _ll_register_callbacks cb 0x7f64c8012b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:00:36.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d25020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d25020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32966/0 (socket says 172.21.15.125:32966) 2023-10-18T12:00:36.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 -- 172.21.15.125:0/911229026 learned_addr learned my addr 172.21.15.125:0/911229026 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:36.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d258c640 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 0x55c378d24ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d2d8d640 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 0x55c378d533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 -- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 msgr2=0x55c378d533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:36.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 0x55c378d533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 -- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 msgr2=0x55c378d24ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:00:36.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 0x55c378d24ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 -- 172.21.15.125:0/911229026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c378c8fde0 con 0x55c378d6c820 2023-10-18T12:00:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64d1d8b640 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d25020 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006350 tx=0x7f64bc006640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8480cbe435c46d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.619+0000 7f64ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:00:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64ad7fa640 1 -- 172.21.15.125:0/911229026 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc030650 con 0x55c378d6c820 2023-10-18T12:00:36.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64ad7fa640 1 -- 172.21.15.125:0/911229026 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f64bc0307b0 con 0x55c378d6c820 2023-10-18T12:00:36.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 1 -- 172.21.15.125:0/911229026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64c8003b70 con 0x55c378d6c820 2023-10-18T12:00:36.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64ad7fa640 1 -- 172.21.15.125:0/911229026 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64bc030970 con 0x55c378d6c820 2023-10-18T12:00:36.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 10 client.12270 fetch_fsmap learned FSMap version 90 2023-10-18T12:00:36.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 1 -- 172.21.15.125:0/911229026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55c378c544f0 con 0x55c378d6c820 2023-10-18T12:00:36.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64ad7fa640 1 -- 172.21.15.125:0/911229026 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64c8003b70 con 0x55c378d6c820 2023-10-18T12:00:36.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64ad7fa640 1 -- 172.21.15.125:0/911229026 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 90) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f64bc01b050 con 0x55c378d6c820 2023-10-18T12:00:36.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 10 client.12270 fetch_fsmap finished waiting for FSMap version 90 2023-10-18T12:00:36.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 10 client.12270 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-18T12:00:36.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 20 client.12270 populate_metadata read hostname 'smithi125' 2023-10-18T12:00:36.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] conn(0x55c378d53920 0x55c378d53d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 4 client.12270 mds_command: new command op to 10168 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T12:00:36.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d737c740 1 -- 172.21.15.125:0/911229026 --> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c3789936c0 con 0x55c378d53920 2023-10-18T12:00:36.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.620+0000 7f64d258c640 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] conn(0x55c378d53920 0x55c378d53d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d258c640 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] conn(0x55c378d53920 0x55c378d53d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70ba0995c8dc1d2 server_cookie=8efd5315f6c0a493 in_seq=0 out_seq=0 2023-10-18T12:00:36.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64ad7fa640 10 client.12270 ms_handle_connect on v2:172.21.15.139:6835/1825099907 2023-10-18T12:00:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64ad7fa640 1 -- 172.21.15.125:0/911229026 <== mds.0 v2:172.21.15.139:6835/1825099907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+337 (crc 0 0 0) 0x55c3789936c0 con 0x55c378d53920 2023-10-18T12:00:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64ad7fa640 10 client.12270 handle_command_reply: tid=0 2023-10-18T12:00:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 client.12270 shutdown 2023-10-18T12:00:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 -- 172.21.15.125:0/911229026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 msgr2=0x55c378d25020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d25020 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006350 tx=0x7f64bc006640 comp rx=0 tx=0).stop 2023-10-18T12:00:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 -- 172.21.15.125:0/911229026 shutdown_connections 2023-10-18T12:00:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:6835/1825099907,v1:172.21.15.139:6837/1825099907] conn(0x55c378d53920 0x55c378d53d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c378d736c0 0x55c378d533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c378d6c820 0x55c378d25020 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 --2- 172.21.15.125:0/911229026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c378d52f00 0x55c378d24ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64ad7fa640 0 client.12270 ms_handle_reset on v2:172.21.15.139:6835/1825099907 2023-10-18T12:00:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 -- 172.21.15.125:0/911229026 >> 172.21.15.125:0/911229026 conn(0x55c378d2b130 msgr2=0x55c378d72530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 -- 172.21.15.125:0/911229026 shutdown_connections 2023-10-18T12:00:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 1 -- 172.21.15.125:0/911229026 wait complete. 2023-10-18T12:00:36.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.621+0000 7f64d737c740 20 client.12270 trim_cache size 0 max 16384 2023-10-18T12:00:36.653 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:00:36.653 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T12:00:36.653 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T12:00:36.653 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 10168, 2023-10-18T12:00:36.654 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T12:00:36.654 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T12:00:36.654 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "backup_fs", 2023-10-18T12:00:36.654 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 4.2035934790000002, 2023-10-18T12:00:36.655 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 90, 2023-10-18T12:00:36.655 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 225, 2023-10-18T12:00:36.655 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 225, 2023-10-18T12:00:36.655 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 21.321722191999999 2023-10-18T12:00:36.655 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:00:36.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 -- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f64b006af60 msgr2=0x7f64b006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f64b006af60 0x7f64b006d420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009b10 tx=0x7f64c8005f90 comp rx=0 tx=0).stop 2023-10-18T12:00:36.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 -- 172.21.15.125:0/2980556391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64cc1e25f0 msgr2=0x7f64cc13d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64cc1e25f0 0x7f64cc13d040 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f64c402c530 tx=0x7f64c402f000 comp rx=0 tx=0).stop 2023-10-18T12:00:36.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 -- 172.21.15.125:0/2980556391 shutdown_connections 2023-10-18T12:00:36.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f64b006af60 0x7f64b006d420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64cc1e25f0 0x7f64cc13d040 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64cc1dd7a0 0x7f64cc0a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 --2- 172.21.15.125:0/2980556391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64b8005180 0x7f64cc13b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 -- 172.21.15.125:0/2980556391 >> 172.21.15.125:0/2980556391 conn(0x7f64cc069930 msgr2=0x7f64cc13f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 -- 172.21.15.125:0/2980556391 shutdown_connections 2023-10-18T12:00:36.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.622+0000 7f64d47f0640 1 -- 172.21.15.125:0/2980556391 wait complete. 2023-10-18T12:00:36.663 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T12:00:36.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.764+0000 7f68457f5640 1 -- 172.21.15.125:0/3433147995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68400cd1c0 msgr2=0x7f684015c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.764+0000 7f68457f5640 1 --2- 172.21.15.125:0/3433147995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68400cd1c0 0x7f684015c0e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6834005fd0 tx=0x7f683402be10 comp rx=0 tx=0).stop 2023-10-18T12:00:36.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.764+0000 7f68457f5640 1 -- 172.21.15.125:0/3433147995 shutdown_connections 2023-10-18T12:00:36.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.764+0000 7f68457f5640 1 --2- 172.21.15.125:0/3433147995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68400cd1c0 0x7f684015c0e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.764+0000 7f68457f5640 1 --2- 172.21.15.125:0/3433147995 >> v2:172.21.15.139:3300/0 conn(0x7f6840158910 0x7f6840158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.764+0000 7f68457f5640 1 -- 172.21.15.125:0/3433147995 >> 172.21.15.125:0/3433147995 conn(0x7f68400af7a0 msgr2=0x7f68400afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.764+0000 7f68457f5640 1 -- 172.21.15.125:0/3433147995 shutdown_connections 2023-10-18T12:00:36.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.764+0000 7f68457f5640 1 -- 172.21.15.125:0/3433147995 wait complete. 2023-10-18T12:00:36.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.765+0000 7f68457f5640 1 Processor -- start 2023-10-18T12:00:36.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.765+0000 7f68457f5640 1 -- start start 2023-10-18T12:00:36.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f68457f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68400cd1c0 0x7f684014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f68457f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68400cd9e0 0x7f68401f0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f68457f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6840158910 0x7f68400c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f68457f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400c4260 con 0x7f6840158910 2023-10-18T12:00:36.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f68457f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400c3ca0 con 0x7f68400cd9e0 2023-10-18T12:00:36.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f68457f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68400c3f80 con 0x7f68400cd1c0 2023-10-18T12:00:36.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f683f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6840158910 0x7f68400c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f683f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6840158910 0x7f68400c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32982/0 (socket says 172.21.15.125:32982) 2023-10-18T12:00:36.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.766+0000 7f683f7fe640 1 -- 172.21.15.125:0/3422119958 learned_addr learned my addr 172.21.15.125:0/3422119958 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:36.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683e7fc640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68400cd9e0 0x7f68401f0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683effd640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68400cd1c0 0x7f684014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683f7fe640 1 -- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68400cd1c0 msgr2=0x7f684014f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683f7fe640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68400cd1c0 0x7f684014f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683f7fe640 1 -- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68400cd9e0 msgr2=0x7f68401f0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683f7fe640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68400cd9e0 0x7f68401f0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683f7fe640 1 -- 172.21.15.125:0/3422119958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834009560 con 0x7f6840158910 2023-10-18T12:00:36.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683e7fc640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68400cd9e0 0x7f68401f0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:36.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f683f7fe640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6840158910 0x7f68400c32e0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f683800c3c0 tx=0x7f683800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:36.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f68277fe640 1 -- 172.21.15.125:0/3422119958 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68380052d0 con 0x7f6840158910 2023-10-18T12:00:36.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f68277fe640 1 -- 172.21.15.125:0/3422119958 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6838007020 con 0x7f6840158910 2023-10-18T12:00:36.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6840159490 con 0x7f6840158910 2023-10-18T12:00:36.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f68277fe640 1 -- 172.21.15.125:0/3422119958 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838007430 con 0x7f6840158910 2023-10-18T12:00:36.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.767+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684014fa90 con 0x7f6840158910 2023-10-18T12:00:36.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.768+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68401599b0 con 0x7f6840158910 2023-10-18T12:00:36.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.768+0000 7f68277fe640 1 -- 172.21.15.125:0/3422119958 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f68380067d0 con 0x7f6840158910 2023-10-18T12:00:36.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.769+0000 7f68277fe640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f682c06af60 0x7f682c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:36.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.769+0000 7f68277fe640 1 -- 172.21.15.125:0/3422119958 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f6838089210 con 0x7f6840158910 2023-10-18T12:00:36.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.769+0000 7f683effd640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f682c06af60 0x7f682c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:36.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.769+0000 7f683effd640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f682c06af60 0x7f682c06d420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6830005fd0 tx=0x7f6830009500 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:36.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.773+0000 7f68277fe640 1 -- 172.21.15.125:0/3422119958 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68401599b0 con 0x7f6840158910 2023-10-18T12:00:36.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.959+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 --> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f68401599b0 con 0x7f682c06af60 2023-10-18T12:00:36.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.962+0000 7f68277fe640 1 -- 172.21.15.125:0/3422119958 <== mgr.8786 v2:172.21.15.139:6832/2362843834 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f68401599b0 con 0x7f682c06af60 2023-10-18T12:00:36.964 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T12:00:36.964 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:36.965 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T12:00:36.965 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:00:36.965 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:00:36.966 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "backup_fs" 2023-10-18T12:00:36.966 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:00:36.967 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T12:00:36.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f682c06af60 msgr2=0x7f682c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f682c06af60 0x7f682c06d420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6830005fd0 tx=0x7f6830009500 comp rx=0 tx=0).stop 2023-10-18T12:00:36.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6840158910 msgr2=0x7f68400c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:36.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6840158910 0x7f68400c32e0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f683800c3c0 tx=0x7f683800c880 comp rx=0 tx=0).stop 2023-10-18T12:00:36.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 shutdown_connections 2023-10-18T12:00:36.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f682c06af60 0x7f682c06d420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68400cd9e0 0x7f68401f0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6840158910 0x7f68400c32e0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 --2- 172.21.15.125:0/3422119958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f68400cd1c0 0x7f684014f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:36.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 >> 172.21.15.125:0/3422119958 conn(0x7f68400af7a0 msgr2=0x7f68400b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:36.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 shutdown_connections 2023-10-18T12:00:36.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:36.963+0000 7f68457f5640 1 -- 172.21.15.125:0/3422119958 wait complete. 2023-10-18T12:00:36.976 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T12:00:37.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.078+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3835326406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 msgr2=0x7f63f406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.078+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3835326406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 0x7f63f406ef80 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f63e4005fd0 tx=0x7f63e402bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:37.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.078+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3835326406 shutdown_connections 2023-10-18T12:00:37.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.078+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3835326406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63f40c9490 0x7f63f406e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.078+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3835326406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 0x7f63f406ef80 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.078+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3835326406 >> 172.21.15.125:0/3835326406 conn(0x7f63f40b06e0 msgr2=0x7f63f40b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:37.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.078+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3835326406 shutdown_connections 2023-10-18T12:00:37.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.078+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3835326406 wait complete. 2023-10-18T12:00:37.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.079+0000 7f63fc1f0640 1 Processor -- start 2023-10-18T12:00:37.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.079+0000 7f63fc1f0640 1 -- start start 2023-10-18T12:00:37.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63fc1f0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 0x7f63f41f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63fc1f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63f40c9490 0x7f63f40c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63fc1f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63f415c5b0 0x7f63f40ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63fc1f0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40c7e00 con 0x7f63f406d9c0 2023-10-18T12:00:37.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63fc1f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40c80e0 con 0x7f63f40c9490 2023-10-18T12:00:37.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63fc1f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40c83c0 con 0x7f63f415c5b0 2023-10-18T12:00:37.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 0x7f63f41f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 0x7f63f41f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33008/0 (socket says 172.21.15.125:33008) 2023-10-18T12:00:37.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 -- 172.21.15.125:0/3568011768 learned_addr learned my addr 172.21.15.125:0/3568011768 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:37.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63fa78d640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63f415c5b0 0x7f63f40ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f978b640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63f40c9490 0x7f63f40c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 -- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63f415c5b0 msgr2=0x7f63f40ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63f415c5b0 0x7f63f40ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 -- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63f40c9490 msgr2=0x7f63f40c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63f40c9490 0x7f63f40c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 -- 172.21.15.125:0/3568011768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4009560 con 0x7f63f406d9c0 2023-10-18T12:00:37.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63f9f8c640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 0x7f63f41f3a20 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c3c0 tx=0x7f63f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:37.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63ea7fc640 1 -- 172.21.15.125:0/3568011768 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00052d0 con 0x7f63f406d9c0 2023-10-18T12:00:37.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63ea7fc640 1 -- 172.21.15.125:0/3568011768 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f63f0007020 con 0x7f63f406d9c0 2023-10-18T12:00:37.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f40c86a0 con 0x7f63f406d9c0 2023-10-18T12:00:37.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.080+0000 7f63ea7fc640 1 -- 172.21.15.125:0/3568011768 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0007470 con 0x7f63f406d9c0 2023-10-18T12:00:37.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.081+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f40c7a60 con 0x7f63f406d9c0 2023-10-18T12:00:37.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.081+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f40cc510 con 0x7f63f406d9c0 2023-10-18T12:00:37.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.082+0000 7f63ea7fc640 1 -- 172.21.15.125:0/3568011768 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 130) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f63f00067d0 con 0x7f63f406d9c0 2023-10-18T12:00:37.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.082+0000 7f63ea7fc640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f63d806af60 0x7f63d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.082+0000 7f63ea7fc640 1 -- 172.21.15.125:0/3568011768 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f63f00891e0 con 0x7f63f406d9c0 2023-10-18T12:00:37.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.083+0000 7f63f978b640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f63d806af60 0x7f63d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.083+0000 7f63f978b640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f63d806af60 0x7f63d806d420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f63e4035000 tx=0x7f63e4004730 comp rx=0 tx=0).ready entity=mgr.8786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:37.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.087+0000 7f63ea7fc640 1 -- 172.21.15.125:0/3568011768 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63f0053d50 con 0x7f63f406d9c0 2023-10-18T12:00:37.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.302+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f63f40690e0 con 0x7f63f406d9c0 2023-10-18T12:00:37.457 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.454+0000 7f63ea7fc640 1 -- 172.21.15.125:0/3568011768 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v131) v1 ==== 92+0+0 (secure 0 0 0) 0x7f63f0057370 con 0x7f63f406d9c0 2023-10-18T12:00:37.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.456+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f63d806af60 msgr2=0x7f63d806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.456+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f63d806af60 0x7f63d806d420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f63e4035000 tx=0x7f63e4004730 comp rx=0 tx=0).stop 2023-10-18T12:00:37.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.456+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 msgr2=0x7f63f41f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.456+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 0x7f63f41f3a20 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c3c0 tx=0x7f63f000c880 comp rx=0 tx=0).stop 2023-10-18T12:00:37.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.457+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 shutdown_connections 2023-10-18T12:00:37.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.457+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f63d806af60 0x7f63d806d420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.457+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63f40c9490 0x7f63f40c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.457+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63f415c5b0 0x7f63f40ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.457+0000 7f63fc1f0640 1 --2- 172.21.15.125:0/3568011768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63f406d9c0 0x7f63f41f3a20 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.457+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 >> 172.21.15.125:0/3568011768 conn(0x7f63f40b06e0 msgr2=0x7f63f40ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:37.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.457+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 shutdown_connections 2023-10-18T12:00:37.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.457+0000 7f63fc1f0640 1 -- 172.21.15.125:0/3568011768 wait complete. 2023-10-18T12:00:37.471 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T12:00:37.617 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.614+0000 7f42b7565640 1 -- 172.21.15.125:0/225502889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b013d350 msgr2=0x7f42b00cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.614+0000 7f42b7565640 1 --2- 172.21.15.125:0/225502889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b013d350 0x7f42b00cbd60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009830 tx=0x7f42a002bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:37.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.614+0000 7f42b7565640 1 -- 172.21.15.125:0/225502889 shutdown_connections 2023-10-18T12:00:37.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.614+0000 7f42b7565640 1 --2- 172.21.15.125:0/225502889 >> v2:172.21.15.139:3300/0 conn(0x7f42b0070350 0x7f42b00707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.614+0000 7f42b7565640 1 --2- 172.21.15.125:0/225502889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b013d350 0x7f42b00cbd60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.614+0000 7f42b7565640 1 -- 172.21.15.125:0/225502889 >> 172.21.15.125:0/225502889 conn(0x7f42b0069930 msgr2=0x7f42b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:37.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.614+0000 7f42b7565640 1 -- 172.21.15.125:0/225502889 shutdown_connections 2023-10-18T12:00:37.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.614+0000 7f42b7565640 1 -- 172.21.15.125:0/225502889 wait complete. 2023-10-18T12:00:37.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b7565640 1 Processor -- start 2023-10-18T12:00:37.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b7565640 1 -- start start 2023-10-18T12:00:37.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b7565640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0070350 0x7f42b0140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b7565640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42b00cb980 0x7f42b01f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b7565640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b0158970 0x7f42b01572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b7565640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b01598d0 con 0x7f42b00cb980 2023-10-18T12:00:37.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b7565640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b01584c0 con 0x7f42b0158970 2023-10-18T12:00:37.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b7565640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b01355e0 con 0x7f42b0070350 2023-10-18T12:00:37.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b4b00640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42b00cb980 0x7f42b01f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b4b00640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42b00cb980 0x7f42b01f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56372/0 (socket says 172.21.15.125:56372) 2023-10-18T12:00:37.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b4b00640 1 -- 172.21.15.125:0/640611374 learned_addr learned my addr 172.21.15.125:0/640611374 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:37.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.616+0000 7f42b5301640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0070350 0x7f42b0140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b5b02640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b0158970 0x7f42b01572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b4b00640 1 -- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0070350 msgr2=0x7f42b0140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b4b00640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0070350 0x7f42b0140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b4b00640 1 -- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b0158970 msgr2=0x7f42b01572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b4b00640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b0158970 0x7f42b01572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b4b00640 1 -- 172.21.15.125:0/640611374 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009490 con 0x7f42b00cb980 2023-10-18T12:00:37.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b4b00640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42b00cb980 0x7f42b01f1f10 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009950 tx=0x7f42a0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:37.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42a5ffb640 1 -- 172.21.15.125:0/640611374 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0003e20 con 0x7f42b00cb980 2023-10-18T12:00:37.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42a5ffb640 1 -- 172.21.15.125:0/640611374 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f42a0034440 con 0x7f42b00cb980 2023-10-18T12:00:37.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42a5ffb640 1 -- 172.21.15.125:0/640611374 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0034850 con 0x7f42b00cb980 2023-10-18T12:00:37.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b01f2450 con 0x7f42b00cb980 2023-10-18T12:00:37.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.617+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b01404a0 con 0x7f42b00cb980 2023-10-18T12:00:37.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.619+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b00690e0 con 0x7f42b00cb980 2023-10-18T12:00:37.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.619+0000 7f42a5ffb640 1 -- 172.21.15.125:0/640611374 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 131) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f42a00349b0 con 0x7f42b00cb980 2023-10-18T12:00:37.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.619+0000 7f42a5ffb640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f429406afe0 0x7f429406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.619+0000 7f42b5301640 1 -- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f429406afe0 msgr2=0x7f429406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2362843834 2023-10-18T12:00:37.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.619+0000 7f42a5ffb640 1 -- 172.21.15.125:0/640611374 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f42a00ac240 con 0x7f42b00cb980 2023-10-18T12:00:37.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.619+0000 7f42b5301640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f429406afe0 0x7f429406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:37.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.623+0000 7f42a5ffb640 1 -- 172.21.15.125:0/640611374 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42a0076e40 con 0x7f42b00cb980 2023-10-18T12:00:37.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.797+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f42b006ff80 con 0x7f42b00cb980 2023-10-18T12:00:37.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.800+0000 7f42a5ffb640 1 -- 172.21.15.125:0/640611374 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v76)=0 v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f42a007a460 con 0x7f42b00cb980 2023-10-18T12:00:37.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f429406afe0 msgr2=0x7f429406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:37.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f429406afe0 0x7f429406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42b00cb980 msgr2=0x7f42b01f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42b00cb980 0x7f42b01f1f10 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009950 tx=0x7f42a0003820 comp rx=0 tx=0).stop 2023-10-18T12:00:37.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 shutdown_connections 2023-10-18T12:00:37.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f429406afe0 0x7f429406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b0158970 0x7f42b01572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42b00cb980 0x7f42b01f1f10 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 --2- 172.21.15.125:0/640611374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0070350 0x7f42b0140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 >> 172.21.15.125:0/640611374 conn(0x7f42b0069930 msgr2=0x7f42b0138670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:37.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 shutdown_connections 2023-10-18T12:00:37.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.803+0000 7f42b7565640 1 -- 172.21.15.125:0/640611374 wait complete. 2023-10-18T12:00:37.816 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T12:00:37.847 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:37.844+0000 7f6f57ab6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:00:37.885 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:37.882+0000 7f6e8ec45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:00:37.892 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:37.889+0000 7f6f57ab6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:00:37.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.921+0000 7f8af925d640 1 -- 172.21.15.125:0/1707533866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af415b2a0 msgr2=0x7f8af40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.921+0000 7f8af925d640 1 --2- 172.21.15.125:0/1707533866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af415b2a0 0x7f8af40d1910 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4005fd0 tx=0x7f8ae402be20 comp rx=0 tx=0).stop 2023-10-18T12:00:37.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.921+0000 7f8af925d640 1 -- 172.21.15.125:0/1707533866 shutdown_connections 2023-10-18T12:00:37.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.921+0000 7f8af925d640 1 --2- 172.21.15.125:0/1707533866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af415b2a0 0x7f8af40d1910 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.921+0000 7f8af925d640 1 -- 172.21.15.125:0/1707533866 >> 172.21.15.125:0/1707533866 conn(0x7f8af40d0d60 msgr2=0x7f8af40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:37.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.921+0000 7f8af925d640 1 -- 172.21.15.125:0/1707533866 shutdown_connections 2023-10-18T12:00:37.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.921+0000 7f8af925d640 1 -- 172.21.15.125:0/1707533866 wait complete. 2023-10-18T12:00:37.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.922+0000 7f8af925d640 1 Processor -- start 2023-10-18T12:00:37.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.922+0000 7f8af925d640 1 -- start start 2023-10-18T12:00:37.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af925d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af40ceb40 0x7f8af414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af925d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8af40cf340 0x7f8af415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af925d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8af415b2a0 0x7f8af41f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af925d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af414f8b0 con 0x7f8af40ceb40 2023-10-18T12:00:37.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af925d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40c93f0 con 0x7f8af40cf340 2023-10-18T12:00:37.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af925d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40c96d0 con 0x7f8af415b2a0 2023-10-18T12:00:37.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af40ceb40 0x7f8af414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af40ceb40 0x7f8af414f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56394/0 (socket says 172.21.15.125:56394) 2023-10-18T12:00:37.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8af40cf340 0x7f8af415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af2d9d640 1 -- 172.21.15.125:0/4248236843 learned_addr learned my addr 172.21.15.125:0/4248236843 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:37.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af359e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8af415b2a0 0x7f8af41f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:37.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af2d9d640 1 -- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8af415b2a0 msgr2=0x7f8af41f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af2d9d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8af415b2a0 0x7f8af41f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af2d9d640 1 -- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8af40cf340 msgr2=0x7f8af415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:37.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af2d9d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8af40cf340 0x7f8af415c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:37.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.923+0000 7f8af2d9d640 1 -- 172.21.15.125:0/4248236843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4009560 con 0x7f8af40ceb40 2023-10-18T12:00:37.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.924+0000 7f8af259c640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8af40cf340 0x7f8af415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:37.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.924+0000 7f8af2d9d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af40ceb40 0x7f8af414f370 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800c450 tx=0x7f8ae800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:37.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.924+0000 7f8adb7fe640 1 -- 172.21.15.125:0/4248236843 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80052d0 con 0x7f8af40ceb40 2023-10-18T12:00:37.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.924+0000 7f8adb7fe640 1 -- 172.21.15.125:0/4248236843 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ae8005430 con 0x7f8af40ceb40 2023-10-18T12:00:37.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.924+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af40c99b0 con 0x7f8af40ceb40 2023-10-18T12:00:37.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.924+0000 7f8adb7fe640 1 -- 172.21.15.125:0/4248236843 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8012890 con 0x7f8af40ceb40 2023-10-18T12:00:37.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.924+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af40c9f70 con 0x7f8af40ceb40 2023-10-18T12:00:37.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.925+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af40d07d0 con 0x7f8af40ceb40 2023-10-18T12:00:37.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.925+0000 7f8adb7fe640 1 -- 172.21.15.125:0/4248236843 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 131) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f8ae80129f0 con 0x7f8af40ceb40 2023-10-18T12:00:37.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.926+0000 7f8adb7fe640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8ae006afb0 0x7f8ae006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:37.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.926+0000 7f8adb7fe640 1 -- 172.21.15.125:0/4248236843 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f8ae8089840 con 0x7f8af40ceb40 2023-10-18T12:00:37.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.926+0000 7f8af259c640 1 -- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8ae006afb0 msgr2=0x7f8ae006d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2362843834 2023-10-18T12:00:37.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.926+0000 7f8af259c640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8ae006afb0 0x7f8ae006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:37.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:37.930+0000 7f8adb7fe640 1 -- 172.21.15.125:0/4248236843 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ae80543b0 con 0x7f8af40ceb40 2023-10-18T12:00:37.935 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:37.931+0000 7f6e8ec45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:00:38.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.105+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f8af40d04f0 con 0x7f8af40ceb40 2023-10-18T12:00:38.111 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.108+0000 7f8adb7fe640 1 -- 172.21.15.125:0/4248236843 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v77)=0 v77) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8ae80579d0 con 0x7f8af40ceb40 2023-10-18T12:00:38.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8ae006afb0 msgr2=0x7f8ae006d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:38.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8ae006afb0 0x7f8ae006d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af40ceb40 msgr2=0x7f8af414f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:38.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af40ceb40 0x7f8af414f370 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800c450 tx=0x7f8ae800c910 comp rx=0 tx=0).stop 2023-10-18T12:00:38.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 shutdown_connections 2023-10-18T12:00:38.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7f8ae006afb0 0x7f8ae006d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8af40cf340 0x7f8af415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8af415b2a0 0x7f8af41f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 --2- 172.21.15.125:0/4248236843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8af40ceb40 0x7f8af414f370 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 >> 172.21.15.125:0/4248236843 conn(0x7f8af40d0d60 msgr2=0x7f8af415a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:38.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 shutdown_connections 2023-10-18T12:00:38.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.111+0000 7f8af925d640 1 -- 172.21.15.125:0/4248236843 wait complete. 2023-10-18T12:00:38.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:38.125 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-18T12:00:38.143 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-18T12:00:38.285 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.282+0000 7f6f57ab6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:00:38.293 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.290+0000 7fcccf226640 1 -- 172.21.15.125:0/2100068846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 msgr2=0x7fccc80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:38.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.290+0000 7fcccf226640 1 --2- 172.21.15.125:0/2100068846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 0x7fccc80b0650 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fccc0005fd0 tx=0x7fccc002c240 comp rx=0 tx=0).stop 2023-10-18T12:00:38.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.290+0000 7fcccf226640 1 -- 172.21.15.125:0/2100068846 shutdown_connections 2023-10-18T12:00:38.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.290+0000 7fcccf226640 1 --2- 172.21.15.125:0/2100068846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 0x7fccc80b0650 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.290+0000 7fcccf226640 1 --2- 172.21.15.125:0/2100068846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccc815b2a0 0x7fccc80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.290+0000 7fcccf226640 1 -- 172.21.15.125:0/2100068846 >> 172.21.15.125:0/2100068846 conn(0x7fccc80d0d60 msgr2=0x7fccc80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:38.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.290+0000 7fcccf226640 1 -- 172.21.15.125:0/2100068846 shutdown_connections 2023-10-18T12:00:38.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.290+0000 7fcccf226640 1 -- 172.21.15.125:0/2100068846 wait complete. 2023-10-18T12:00:38.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.292+0000 7fcccf226640 1 Processor -- start 2023-10-18T12:00:38.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.292+0000 7fcccf226640 1 -- start start 2023-10-18T12:00:38.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fcccf226640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 0x7fccc80c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:38.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fcccf226640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccc80cf340 0x7fccc80c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:38.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fcccf226640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccc815b2a0 0x7fccc814f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:38.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fcccf226640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc81e5390 con 0x7fccc80ceb40 2023-10-18T12:00:38.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fcccf226640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc81e4780 con 0x7fccc815b2a0 2023-10-18T12:00:38.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fcccf226640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc81e4a60 con 0x7fccc80cf340 2023-10-18T12:00:38.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 0x7fccc80c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:38.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 0x7fccc80c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56412/0 (socket says 172.21.15.125:56412) 2023-10-18T12:00:38.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 -- 172.21.15.125:0/2710508380 learned_addr learned my addr 172.21.15.125:0/2710508380 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:38.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccc7fff640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccc80cf340 0x7fccc80c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:38.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fcccd7c3640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccc815b2a0 0x7fccc814f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:38.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccc80cf340 msgr2=0x7fccc80c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:38.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccc80cf340 0x7fccc80c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccc815b2a0 msgr2=0x7fccc814f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:38.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fccc815b2a0 0x7fccc814f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:38.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 -- 172.21.15.125:0/2710508380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0009900 con 0x7fccc80ceb40 2023-10-18T12:00:38.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccc7fff640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccc80cf340 0x7fccc80c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:38.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.294+0000 7fccccfc2640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 0x7fccc80c8a80 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c450 tx=0x7fccbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:38.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.295+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0052d0 con 0x7fccc80ceb40 2023-10-18T12:00:38.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.295+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fccbc005430 con 0x7fccc80ceb40 2023-10-18T12:00:38.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.295+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc81e4d40 con 0x7fccc80ceb40 2023-10-18T12:00:38.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.295+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc012890 con 0x7fccc80ceb40 2023-10-18T12:00:38.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.295+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc80c8fc0 con 0x7fccc80ceb40 2023-10-18T12:00:38.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.296+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc80690e0 con 0x7fccc80ceb40 2023-10-18T12:00:38.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.296+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 131) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fccbc0035f0 con 0x7fccc80ceb40 2023-10-18T12:00:38.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.297+0000 7fccc57fa640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 0x7fccac06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:38.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.297+0000 7fccc7fff640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 msgr2=0x7fccac06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2362843834 2023-10-18T12:00:38.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.297+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fccbc089830 con 0x7fccc80ceb40 2023-10-18T12:00:38.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.297+0000 7fccc7fff640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 0x7fccac06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:38.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.301+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fccbc0543a0 con 0x7fccc80ceb40 2023-10-18T12:00:38.330 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.327+0000 7f6f57ab6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:00:38.343 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.341+0000 7f6e8ec45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:00:38.372 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.369+0000 7f6f57ab6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:00:38.389 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.387+0000 7f6e8ec45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:00:38.423 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.421+0000 7f6f57ab6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:00:38.433 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.430+0000 7f6e8ec45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:00:38.467 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.465+0000 7f6f57ab6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:00:38.487 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.484+0000 7f6e8ec45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:00:38.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.496+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 --> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fccc81e54d0 con 0x7fccac06afb0 2023-10-18T12:00:38.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.498+0000 7fccc7fff640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 msgr2=0x7fccac06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2362843834 2023-10-18T12:00:38.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.498+0000 7fccc7fff640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 0x7fccac06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:00:38.529 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.526+0000 7f6e8ec45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:00:38.566 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.564+0000 7f6f57ab6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:00:38.619 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.616+0000 7f6f57ab6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:38.625 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.623+0000 7f6e8ec45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:00:38.660 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.658+0000 7f6f57ab6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:00:38.677 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.675+0000 7f6e8ec45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:38.707 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.705+0000 7f6f57ab6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:00:38.718 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.716+0000 7f6e8ec45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:00:38.749 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:38.747+0000 7f6f57ab6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:00:38.765 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.763+0000 7f6e8ec45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:00:38.807 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:38.805+0000 7f6e8ec45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:00:38.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.899+0000 7fccc7fff640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 msgr2=0x7fccac06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2362843834 2023-10-18T12:00:38.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:38.899+0000 7fccc7fff640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 0x7fccac06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T12:00:39.066 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.063+0000 7f6f57ab6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:00:39.120 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.117+0000 7f6e8ec45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:00:39.195 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.192+0000 7f6f57ab6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:00:39.248 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.245+0000 7f6e8ec45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:00:39.286 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.284+0000 7f6f57ab6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:00:39.329 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.327+0000 7f6f57ab6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:00:39.339 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.337+0000 7f6e8ec45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:00:39.381 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.379+0000 7f6e8ec45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:00:39.427 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.424+0000 7f6f57ab6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:39.479 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.476+0000 7f6e8ec45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:39.521 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.519+0000 7f6f57ab6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:00:39.569 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.567+0000 7f6e8ec45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:00:39.657 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.655+0000 7f6f57ab6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:39.698 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.696+0000 7f6f57ab6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:39.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:39.700+0000 7fccc7fff640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 msgr2=0x7fccac06d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2362843834 2023-10-18T12:00:39.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:39.700+0000 7fccc7fff640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 0x7fccac06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T12:00:39.703 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.701+0000 7f6e8ec45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:39.745 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.742+0000 7f6e8ec45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:39.802 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:39.799+0000 7f6f57ab6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:00:39.848 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:39.846+0000 7f6e8ec45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:00:40.420 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.417+0000 7f6f57ab6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:00:40.468 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:40.466+0000 7f6e8ec45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:00:40.683 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.681+0000 7f6f57ab6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:00:40.728 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:40.726+0000 7f6e8ec45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:00:40.731 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.729+0000 7f6f57ab6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:00:40.776 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:40.774+0000 7f6e8ec45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:00:40.830 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.827+0000 7f6f57ab6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:00:40.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:40.848+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 132) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fccbc087870 con 0x7fccc80ceb40 2023-10-18T12:00:40.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:40.848+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 msgr2=0x7fccac06d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:40.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:40.848+0000 7fccc57fa640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/2362843834,v1:172.21.15.139:6833/2362843834] conn(0x7fccac06afb0 0x7fccac06d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:40.880 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:40.877+0000 7f6e8ec45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:00:40.991 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.991 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.991 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.992 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.992 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.992 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.992 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.992 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.993 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.993 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.988+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:40.999 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.000 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.000 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.000 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.000 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed44f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.001 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.001 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.001 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.001 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.001 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:40.997+0000 7f6ed6cfc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:41.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.853+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 133) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fccbc05e0f0 con 0x7fccc80ceb40 2023-10-18T12:00:41.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.853+0000 7fccc57fa640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fccac035850 0x7fccac077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:41.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.853+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 --> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fccac035c30 con 0x7fccac035850 2023-10-18T12:00:41.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.854+0000 7fccc7fff640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fccac035850 0x7fccac077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:41.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.859+0000 7fccc7fff640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fccac035850 0x7fccac077b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fccc80b2310 tx=0x7fccb4005f90 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:41.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.913+0000 7fccc57fa640 1 -- 172.21.15.125:0/2710508380 <== mgr.10375 v2:172.21.15.139:6832/178515798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+53 (secure 0 0 0) 0x7fccac035c30 con 0x7fccac035850 2023-10-18T12:00:41.916 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path mandatory_fs_argument_test_dir 2023-10-18T12:00:41.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fccac035850 msgr2=0x7fccac077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:41.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccf226640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fccac035850 0x7fccac077b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fccc80b2310 tx=0x7fccb4005f90 comp rx=0 tx=0).stop 2023-10-18T12:00:41.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 msgr2=0x7fccc80c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:41.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccf226640 1 --2- 172.21.15.125:0/2710508380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccc80ceb40 0x7fccc80c8a80 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c450 tx=0x7fccbc00c910 comp rx=0 tx=0).stop 2023-10-18T12:00:41.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccd7c3640 1 -- 172.21.15.125:0/2710508380 reap_dead start 2023-10-18T12:00:41.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 shutdown_connections 2023-10-18T12:00:41.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 >> 172.21.15.125:0/2710508380 conn(0x7fccc80d0d60 msgr2=0x7fccc80ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:41.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 shutdown_connections 2023-10-18T12:00:41.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:41.916+0000 7fcccf226640 1 -- 172.21.15.125:0/2710508380 wait complete. 2023-10-18T12:00:41.930 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2023-10-18T12:00:41.931 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-18T12:00:42.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.037+0000 7fcb3161a640 1 -- 172.21.15.125:0/843194678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 msgr2=0x7fcb2c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.037+0000 7fcb3161a640 1 --2- 172.21.15.125:0/843194678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 0x7fcb2c06b2e0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009be0 tx=0x7fcb1c02c0e0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.037+0000 7fcb3161a640 1 -- 172.21.15.125:0/843194678 shutdown_connections 2023-10-18T12:00:42.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.037+0000 7fcb3161a640 1 --2- 172.21.15.125:0/843194678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 0x7fcb2c06b2e0 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.037+0000 7fcb3161a640 1 -- 172.21.15.125:0/843194678 >> 172.21.15.125:0/843194678 conn(0x7fcb2c069930 msgr2=0x7fcb2c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:42.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.037+0000 7fcb3161a640 1 -- 172.21.15.125:0/843194678 shutdown_connections 2023-10-18T12:00:42.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.037+0000 7fcb3161a640 1 -- 172.21.15.125:0/843194678 wait complete. 2023-10-18T12:00:42.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.038+0000 7fcb3161a640 1 Processor -- start 2023-10-18T12:00:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.038+0000 7fcb3161a640 1 -- start start 2023-10-18T12:00:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb3161a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 0x7fcb2c1ef610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb3161a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb2c0abc10 0x7fcb2c0ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb3161a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb2c0c96b0 0x7fcb2c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb3161a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0b1110 con 0x7fcb2c06af00 2023-10-18T12:00:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb3161a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c154720 con 0x7fcb2c0c96b0 2023-10-18T12:00:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb3161a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c154a00 con 0x7fcb2c0abc10 2023-10-18T12:00:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 0x7fcb2c1ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 0x7fcb2c1ef610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56424/0 (socket says 172.21.15.125:56424) 2023-10-18T12:00:42.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2affd640 1 -- 172.21.15.125:0/3736540502 learned_addr learned my addr 172.21.15.125:0/3736540502 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2affd640 1 -- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb2c0abc10 msgr2=0x7fcb2c0ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2affd640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb2c0abc10 0x7fcb2c0ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2affd640 1 -- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb2c0c96b0 msgr2=0x7fcb2c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2b7fe640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb2c0c96b0 0x7fcb2c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2affd640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb2c0c96b0 0x7fcb2c0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.039+0000 7fcb2affd640 1 -- 172.21.15.125:0/3736540502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c009840 con 0x7fcb2c06af00 2023-10-18T12:00:42.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.040+0000 7fcb2affd640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 0x7fcb2c1ef610 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c036040 tx=0x7fcb1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:42.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.040+0000 7fcb137fe640 1 -- 172.21.15.125:0/3736540502 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0046f0 con 0x7fcb2c06af00 2023-10-18T12:00:42.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.040+0000 7fcb137fe640 1 -- 172.21.15.125:0/3736540502 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb1c0345a0 con 0x7fcb2c06af00 2023-10-18T12:00:42.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.040+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c154ce0 con 0x7fcb2c06af00 2023-10-18T12:00:42.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.040+0000 7fcb137fe640 1 -- 172.21.15.125:0/3736540502 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0349b0 con 0x7fcb2c06af00 2023-10-18T12:00:42.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.040+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c0af9c0 con 0x7fcb2c06af00 2023-10-18T12:00:42.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.041+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c155200 con 0x7fcb2c06af00 2023-10-18T12:00:42.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.041+0000 7fcb137fe640 1 -- 172.21.15.125:0/3736540502 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 133) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fcb1c034b10 con 0x7fcb2c06af00 2023-10-18T12:00:42.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.042+0000 7fcb137fe640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fcb1806ace0 0x7fcb1806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.042+0000 7fcb137fe640 1 -- 172.21.15.125:0/3736540502 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7fcb1c0aca70 con 0x7fcb2c06af00 2023-10-18T12:00:42.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.042+0000 7fcb2a7fc640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fcb1806ace0 0x7fcb1806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.042+0000 7fcb2a7fc640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fcb1806ace0 0x7fcb1806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000f60 tx=0x7fcb200079d0 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:42.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.046+0000 7fcb137fe640 1 -- 172.21.15.125:0/3736540502 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb2c155200 con 0x7fcb2c06af00 2023-10-18T12:00:42.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.238+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 --> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb2c159c50 con 0x7fcb1806ace0 2023-10-18T12:00:42.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.276+0000 7fcb137fe640 1 -- 172.21.15.125:0/3736540502 <== mgr.10375 v2:172.21.15.139:6832/178515798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fcb20005080 con 0x7fcb1806ace0 2023-10-18T12:00:42.279 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2023-10-18T12:00:42.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fcb1806ace0 msgr2=0x7fcb1806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fcb1806ace0 0x7fcb1806d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000f60 tx=0x7fcb200079d0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 msgr2=0x7fcb2c1ef610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 0x7fcb2c1ef610 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c036040 tx=0x7fcb1c0042f0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 shutdown_connections 2023-10-18T12:00:42.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fcb1806ace0 0x7fcb1806d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb2c0c96b0 0x7fcb2c0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcb2c0abc10 0x7fcb2c0ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 --2- 172.21.15.125:0/3736540502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcb2c06af00 0x7fcb2c1ef610 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.278+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 >> 172.21.15.125:0/3736540502 conn(0x7fcb2c069930 msgr2=0x7fcb2c0adb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:42.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.279+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 shutdown_connections 2023-10-18T12:00:42.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.279+0000 7fcb3161a640 1 -- 172.21.15.125:0/3736540502 wait complete. 2023-10-18T12:00:42.292 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2023-10-18T12:00:42.292 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-18T12:00:42.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.398+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/1746614164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 msgr2=0x7fdcf80cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.398+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/1746614164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 0x7fdcf80cbd20 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fdce8005fd0 tx=0x7fdce802be10 comp rx=0 tx=0).stop 2023-10-18T12:00:42.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.398+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/1746614164 shutdown_connections 2023-10-18T12:00:42.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.398+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/1746614164 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcf8158300 0x7fdcf80c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.398+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/1746614164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 0x7fdcf80cbd20 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.398+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/1746614164 >> 172.21.15.125:0/1746614164 conn(0x7fdcf8069930 msgr2=0x7fdcf8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:42.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.398+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/1746614164 shutdown_connections 2023-10-18T12:00:42.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.398+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/1746614164 wait complete. 2023-10-18T12:00:42.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.399+0000 7fdcfe1b7640 1 Processor -- start 2023-10-18T12:00:42.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.399+0000 7fdcfe1b7640 1 -- start start 2023-10-18T12:00:42.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcfe1b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcf80705a0 0x7fdcf81ef160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcfe1b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 0x7fdcf8154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcfe1b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcf8158300 0x7fdcf81546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcfe1b7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf81e37f0 con 0x7fdcf80afa20 2023-10-18T12:00:42.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcfe1b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf81ef6a0 con 0x7fdcf80705a0 2023-10-18T12:00:42.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcfe1b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8156e00 con 0x7fdcf8158300 2023-10-18T12:00:42.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 0x7fdcf8154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 0x7fdcf8154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56438/0 (socket says 172.21.15.125:56438) 2023-10-18T12:00:42.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf6ffd640 1 -- 172.21.15.125:0/2924018024 learned_addr learned my addr 172.21.15.125:0/2924018024 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:42.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf77fe640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcf80705a0 0x7fdcf81ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf7fff640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcf8158300 0x7fdcf81546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf6ffd640 1 -- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcf8158300 msgr2=0x7fdcf81546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf6ffd640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcf8158300 0x7fdcf81546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf6ffd640 1 -- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcf80705a0 msgr2=0x7fdcf81ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf6ffd640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcf80705a0 0x7fdcf81ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.400+0000 7fdcf6ffd640 1 -- 172.21.15.125:0/2924018024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8009560 con 0x7fdcf80afa20 2023-10-18T12:00:42.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.401+0000 7fdcf6ffd640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 0x7fdcf8154160 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00d750 tx=0x7fdcec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:42.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.401+0000 7fdcdffff640 1 -- 172.21.15.125:0/2924018024 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec004220 con 0x7fdcf80afa20 2023-10-18T12:00:42.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.401+0000 7fdcdffff640 1 -- 172.21.15.125:0/2924018024 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdcec004470 con 0x7fdcf80afa20 2023-10-18T12:00:42.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.401+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf81e31a0 con 0x7fdcf80afa20 2023-10-18T12:00:42.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.401+0000 7fdcdffff640 1 -- 172.21.15.125:0/2924018024 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec009c30 con 0x7fdcf80afa20 2023-10-18T12:00:42.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.401+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf814fb50 con 0x7fdcf80afa20 2023-10-18T12:00:42.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.402+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf81e3000 con 0x7fdcf80afa20 2023-10-18T12:00:42.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.402+0000 7fdcdffff640 1 -- 172.21.15.125:0/2924018024 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 133) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fdcec005020 con 0x7fdcf80afa20 2023-10-18T12:00:42.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.403+0000 7fdcdffff640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fdce406ad10 0x7fdce406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.403+0000 7fdcdffff640 1 -- 172.21.15.125:0/2924018024 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7fdcec089bc0 con 0x7fdcf80afa20 2023-10-18T12:00:42.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.403+0000 7fdcf77fe640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fdce406ad10 0x7fdce406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.403+0000 7fdcf77fe640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fdce406ad10 0x7fdce406d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdce8035000 tx=0x7fdce8004730 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:42.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.407+0000 7fdcdffff640 1 -- 172.21.15.125:0/2924018024 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcf81e3000 con 0x7fdcf80afa20 2023-10-18T12:00:42.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.593+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 --> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcf81e3000 con 0x7fdce406ad10 2023-10-18T12:00:42.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.637+0000 7fdcdffff640 1 -- 172.21.15.125:0/2924018024 <== mgr.10375 v2:172.21.15.139:6832/178515798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fdcf81e3000 con 0x7fdce406ad10 2023-10-18T12:00:42.640 INFO:teuthology.orchestra.run.smithi125.stdout:Retention added to path mandatory_fs_argument_test_dir 2023-10-18T12:00:42.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fdce406ad10 msgr2=0x7fdce406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fdce406ad10 0x7fdce406d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdce8035000 tx=0x7fdce8004730 comp rx=0 tx=0).stop 2023-10-18T12:00:42.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 msgr2=0x7fdcf8154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 0x7fdcf8154160 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00d750 tx=0x7fdcec00dc10 comp rx=0 tx=0).stop 2023-10-18T12:00:42.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 shutdown_connections 2023-10-18T12:00:42.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fdce406ad10 0x7fdce406d1d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcf8158300 0x7fdcf81546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcf80afa20 0x7fdcf8154160 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 --2- 172.21.15.125:0/2924018024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcf80705a0 0x7fdcf81ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 >> 172.21.15.125:0/2924018024 conn(0x7fdcf8069930 msgr2=0x7fdcf815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:42.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 shutdown_connections 2023-10-18T12:00:42.645 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.639+0000 7fdcfe1b7640 1 -- 172.21.15.125:0/2924018024 wait complete. 2023-10-18T12:00:42.653 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2023-10-18T12:00:42.653 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-18T12:00:42.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.756+0000 7f3a64fc0640 1 -- 172.21.15.125:0/1411463912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 msgr2=0x7f3a600b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.756+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/1411463912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 0x7f3a600b0650 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f3a6008a810 tx=0x7f3a5802bdb0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.756+0000 7f3a64fc0640 1 -- 172.21.15.125:0/1411463912 shutdown_connections 2023-10-18T12:00:42.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.756+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/1411463912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 0x7f3a600b0650 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.756+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/1411463912 >> v2:172.21.15.139:3300/0 conn(0x7f3a600cf340 0x7f3a600d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.756+0000 7f3a64fc0640 1 -- 172.21.15.125:0/1411463912 >> 172.21.15.125:0/1411463912 conn(0x7f3a600d0d60 msgr2=0x7f3a600d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:42.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.756+0000 7f3a64fc0640 1 -- 172.21.15.125:0/1411463912 shutdown_connections 2023-10-18T12:00:42.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.756+0000 7f3a64fc0640 1 -- 172.21.15.125:0/1411463912 wait complete. 2023-10-18T12:00:42.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a64fc0640 1 Processor -- start 2023-10-18T12:00:42.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a64fc0640 1 -- start start 2023-10-18T12:00:42.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a64fc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 0x7f3a600b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a64fc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a600cf340 0x7f3a60157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a64fc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6015b2a0 0x7f3a601f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a64fc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60158480 con 0x7f3a600ceb40 2023-10-18T12:00:42.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a64fc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60158ec0 con 0x7f3a6015b2a0 2023-10-18T12:00:42.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a64fc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600c7e30 con 0x7f3a600cf340 2023-10-18T12:00:42.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a5e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 0x7f3a600b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a5e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 0x7f3a600b5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56456/0 (socket says 172.21.15.125:56456) 2023-10-18T12:00:42.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.758+0000 7f3a5e59c640 1 -- 172.21.15.125:0/939242364 learned_addr learned my addr 172.21.15.125:0/939242364 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:42.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5ed9d640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6015b2a0 0x7f3a601f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5dd9b640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a600cf340 0x7f3a60157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5e59c640 1 -- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a600cf340 msgr2=0x7f3a60157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5e59c640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a600cf340 0x7f3a60157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5e59c640 1 -- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6015b2a0 msgr2=0x7f3a601f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5e59c640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6015b2a0 0x7f3a601f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5e59c640 1 -- 172.21.15.125:0/939242364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a58009560 con 0x7f3a600ceb40 2023-10-18T12:00:42.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5ed9d640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6015b2a0 0x7f3a601f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:42.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5dd9b640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a600cf340 0x7f3a60157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:42.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a5e59c640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 0x7f3a600b5830 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000ebf0 tx=0x7f3a5000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:42.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a4effd640 1 -- 172.21.15.125:0/939242364 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50010630 con 0x7f3a600ceb40 2023-10-18T12:00:42.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a4effd640 1 -- 172.21.15.125:0/939242364 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a5000ced0 con 0x7f3a600ceb40 2023-10-18T12:00:42.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a600c89b0 con 0x7f3a600ceb40 2023-10-18T12:00:42.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a4effd640 1 -- 172.21.15.125:0/939242364 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50010b20 con 0x7f3a600ceb40 2023-10-18T12:00:42.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.759+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a600c8360 con 0x7f3a600ceb40 2023-10-18T12:00:42.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.760+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a600690e0 con 0x7f3a600ceb40 2023-10-18T12:00:42.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.760+0000 7f3a4effd640 1 -- 172.21.15.125:0/939242364 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 133) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3a50010c80 con 0x7f3a600ceb40 2023-10-18T12:00:42.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.761+0000 7f3a4effd640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f3a4006ace0 0x7f3a4006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:42.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.761+0000 7f3a4effd640 1 -- 172.21.15.125:0/939242364 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f3a50014070 con 0x7f3a600ceb40 2023-10-18T12:00:42.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.761+0000 7f3a5dd9b640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f3a4006ace0 0x7f3a4006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:42.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.762+0000 7f3a5dd9b640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f3a4006ace0 0x7f3a4006d1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3a54007fd0 tx=0x7f3a54009040 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:42.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.765+0000 7f3a4effd640 1 -- 172.21.15.125:0/939242364 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a50051ca0 con 0x7f3a600ceb40 2023-10-18T12:00:42.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.955+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 --> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a600b0d00 con 0x7f3a4006ace0 2023-10-18T12:00:42.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.961+0000 7f3a4effd640 1 -- 172.21.15.125:0/939242364 <== mgr.10375 v2:172.21.15.139:6832/178515798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+88 (secure 0 0 0) 0x7f3a600b0d00 con 0x7f3a4006ace0 2023-10-18T12:00:42.963 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:42.964 INFO:teuthology.orchestra.run.smithi125.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-18T12:00:42.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f3a4006ace0 msgr2=0x7f3a4006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f3a4006ace0 0x7f3a4006d1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3a54007fd0 tx=0x7f3a54009040 comp rx=0 tx=0).stop 2023-10-18T12:00:42.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 msgr2=0x7f3a600b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:42.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 0x7f3a600b5830 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000ebf0 tx=0x7f3a5000c6a0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 shutdown_connections 2023-10-18T12:00:42.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f3a4006ace0 0x7f3a4006d1a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6015b2a0 0x7f3a601f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a600cf340 0x7f3a60157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 --2- 172.21.15.125:0/939242364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a600ceb40 0x7f3a600b5830 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:42.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 >> 172.21.15.125:0/939242364 conn(0x7f3a600d0d60 msgr2=0x7f3a6006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:42.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 shutdown_connections 2023-10-18T12:00:42.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:42.963+0000 7f3a64fc0640 1 -- 172.21.15.125:0/939242364 wait complete. 2023-10-18T12:00:42.976 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-18T12:00:42.977 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-18T12:00:43.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.082+0000 7f7fc2097640 1 -- 172.21.15.125:0/1884378822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 msgr2=0x7f7fbc0ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.082+0000 7f7fc2097640 1 --2- 172.21.15.125:0/1884378822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 0x7f7fbc0ab990 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0ca260 tx=0x7f7fb402c110 comp rx=0 tx=0).stop 2023-10-18T12:00:43.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.082+0000 7f7fc2097640 1 -- 172.21.15.125:0/1884378822 shutdown_connections 2023-10-18T12:00:43.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.082+0000 7f7fc2097640 1 --2- 172.21.15.125:0/1884378822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 0x7f7fbc0ab990 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.082+0000 7f7fc2097640 1 --2- 172.21.15.125:0/1884378822 >> v2:172.21.15.139:3300/0 conn(0x7f7fbc0c9dc0 0x7f7fbc06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.082+0000 7f7fc2097640 1 -- 172.21.15.125:0/1884378822 >> 172.21.15.125:0/1884378822 conn(0x7f7fbc069930 msgr2=0x7f7fbc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:43.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.082+0000 7f7fc2097640 1 -- 172.21.15.125:0/1884378822 shutdown_connections 2023-10-18T12:00:43.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.082+0000 7f7fc2097640 1 -- 172.21.15.125:0/1884378822 wait complete. 2023-10-18T12:00:43.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.083+0000 7f7fc2097640 1 Processor -- start 2023-10-18T12:00:43.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.083+0000 7f7fc2097640 1 -- start start 2023-10-18T12:00:43.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fc2097640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fbc06af00 0x7f7fbc139a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fc2097640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 0x7f7fbc152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fc2097640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fbc0c9dc0 0x7f7fbc15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fc2097640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc137b40 con 0x7f7fbc0ab500 2023-10-18T12:00:43.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fc2097640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc0b1460 con 0x7f7fbc06af00 2023-10-18T12:00:43.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fc2097640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc138f60 con 0x7f7fbc0c9dc0 2023-10-18T12:00:43.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 0x7f7fbc152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 0x7f7fbc152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56470/0 (socket says 172.21.15.125:56470) 2023-10-18T12:00:43.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 -- 172.21.15.125:0/909666607 learned_addr learned my addr 172.21.15.125:0/909666607 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:43.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbb7fe640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fbc06af00 0x7f7fbc139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 -- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fbc0c9dc0 msgr2=0x7f7fbc15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fbc0c9dc0 0x7f7fbc15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 -- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fbc06af00 msgr2=0x7f7fbc139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fbc06af00 0x7f7fbc139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 -- 172.21.15.125:0/909666607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4009840 con 0x7f7fbc0ab500 2023-10-18T12:00:43.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbb7fe640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fbc06af00 0x7f7fbc139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:43.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.084+0000 7f7fbaffd640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 0x7f7fbc152b20 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e790 tx=0x7f7fb000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:43.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.085+0000 7f7fa3fff640 1 -- 172.21.15.125:0/909666607 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0009580 con 0x7f7fbc0ab500 2023-10-18T12:00:43.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.085+0000 7f7fa3fff640 1 -- 172.21.15.125:0/909666607 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7fb000cdb0 con 0x7f7fbc0ab500 2023-10-18T12:00:43.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.085+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc139240 con 0x7f7fbc0ab500 2023-10-18T12:00:43.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.085+0000 7f7fa3fff640 1 -- 172.21.15.125:0/909666607 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0010690 con 0x7f7fbc0ab500 2023-10-18T12:00:43.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.085+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc139f80 con 0x7f7fbc0ab500 2023-10-18T12:00:43.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.086+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fbc0690e0 con 0x7f7fbc0ab500 2023-10-18T12:00:43.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.086+0000 7f7fa3fff640 1 -- 172.21.15.125:0/909666607 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 133) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f7fb00107f0 con 0x7f7fbc0ab500 2023-10-18T12:00:43.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.086+0000 7f7fa3fff640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f7fa806ace0 0x7f7fa806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.086+0000 7f7fa3fff640 1 -- 172.21.15.125:0/909666607 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f7fb0014070 con 0x7f7fbc0ab500 2023-10-18T12:00:43.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.086+0000 7f7fbb7fe640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f7fa806ace0 0x7f7fa806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.087+0000 7f7fbb7fe640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f7fa806ace0 0x7f7fa806d1a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0016d0 tx=0x7f7fac009040 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:43.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.090+0000 7f7fa3fff640 1 -- 172.21.15.125:0/909666607 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7fb0051ca0 con 0x7f7fbc0ab500 2023-10-18T12:00:43.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.275+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 --> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fbc1390d0 con 0x7f7fa806ace0 2023-10-18T12:00:43.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.282+0000 7f7fa3fff640 1 -- 172.21.15.125:0/909666607 <== mgr.10375 v2:172.21.15.139:6832/178515798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+338 (secure 0 0 0) 0x7f7fbc1390d0 con 0x7f7fa806ace0 2023-10-18T12:00:43.285 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:43.285 INFO:teuthology.orchestra.run.smithi125.stdout:[{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T12:00:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-18T12:00:43.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f7fa806ace0 msgr2=0x7f7fa806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f7fa806ace0 0x7f7fa806d1a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0016d0 tx=0x7f7fac009040 comp rx=0 tx=0).stop 2023-10-18T12:00:43.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 msgr2=0x7f7fbc152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.287 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 0x7f7fbc152b20 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e790 tx=0x7f7fb000ec50 comp rx=0 tx=0).stop 2023-10-18T12:00:43.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 shutdown_connections 2023-10-18T12:00:43.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f7fa806ace0 0x7f7fa806d1a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fbc0c9dc0 0x7f7fbc15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.288 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fbc0ab500 0x7f7fbc152b20 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 --2- 172.21.15.125:0/909666607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fbc06af00 0x7f7fbc139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 >> 172.21.15.125:0/909666607 conn(0x7f7fbc069930 msgr2=0x7f7fbc069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:43.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 shutdown_connections 2023-10-18T12:00:43.289 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.284+0000 7f7fc2097640 1 -- 172.21.15.125:0/909666607 wait complete. 2023-10-18T12:00:43.297 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-18T00:00:00", "created": "2023-10-18T12:00:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-18T12:00:43.298 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-18T12:00:43.406 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.403+0000 7f0e7230f640 1 -- 172.21.15.125:0/3776565302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 msgr2=0x7f0e6c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.403+0000 7f0e7230f640 1 --2- 172.21.15.125:0/3776565302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 0x7f0e6c0b0650 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c06c510 tx=0x7f0e6402c1d0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.404+0000 7f0e7230f640 1 -- 172.21.15.125:0/3776565302 shutdown_connections 2023-10-18T12:00:43.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.404+0000 7f0e7230f640 1 --2- 172.21.15.125:0/3776565302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 0x7f0e6c0b0650 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.404+0000 7f0e7230f640 1 --2- 172.21.15.125:0/3776565302 >> v2:172.21.15.139:3300/0 conn(0x7f0e6c15b2a0 0x7f0e6c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.404+0000 7f0e7230f640 1 --2- 172.21.15.125:0/3776565302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6c0cf340 0x7f0e6c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.404+0000 7f0e7230f640 1 -- 172.21.15.125:0/3776565302 >> 172.21.15.125:0/3776565302 conn(0x7f0e6c0d0d60 msgr2=0x7f0e6c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:43.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.404+0000 7f0e7230f640 1 -- 172.21.15.125:0/3776565302 shutdown_connections 2023-10-18T12:00:43.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.404+0000 7f0e7230f640 1 -- 172.21.15.125:0/3776565302 wait complete. 2023-10-18T12:00:43.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.405+0000 7f0e7230f640 1 Processor -- start 2023-10-18T12:00:43.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.405+0000 7f0e7230f640 1 -- start start 2023-10-18T12:00:43.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e7230f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 0x7f0e6c0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e7230f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e6c0cf340 0x7f0e6c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e7230f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6c15b2a0 0x7f0e6c0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e7230f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c15c460 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e7230f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0b4480 con 0x7f0e6c0cf340 2023-10-18T12:00:43.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e7230f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c15b7e0 con 0x7f0e6c15b2a0 2023-10-18T12:00:43.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 0x7f0e6c0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 0x7f0e6c0b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56488/0 (socket says 172.21.15.125:56488) 2023-10-18T12:00:43.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 -- 172.21.15.125:0/1850456582 learned_addr learned my addr 172.21.15.125:0/1850456582 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:43.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6b7fe640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e6c0cf340 0x7f0e6c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e708ac640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6c15b2a0 0x7f0e6c0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 -- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6c15b2a0 msgr2=0x7f0e6c0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6c15b2a0 0x7f0e6c0b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 -- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e6c0cf340 msgr2=0x7f0e6c14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e6c0cf340 0x7f0e6c14afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 -- 172.21.15.125:0/1850456582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e64009900 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6b7fe640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e6c0cf340 0x7f0e6c14afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:43.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e6bfff640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 0x7f0e6c0b20e0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d750 tx=0x7f0e5c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:43.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e68ff9640 1 -- 172.21.15.125:0/1850456582 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c009bd0 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e68ff9640 1 -- 172.21.15.125:0/1850456582 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0e5c009e20 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c15bac0 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e68ff9640 1 -- 172.21.15.125:0/1850456582 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c005280 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.406+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c15c080 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.407+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c0690e0 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.408+0000 7f0e68ff9640 1 -- 172.21.15.125:0/1850456582 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 133) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f0e5c0053e0 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.408+0000 7f0e68ff9640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f0e5806ad10 0x7f0e5806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.408+0000 7f0e68ff9640 1 -- 172.21.15.125:0/1850456582 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f0e5c089b30 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.408+0000 7f0e6b7fe640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f0e5806ad10 0x7f0e5806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.409+0000 7f0e6b7fe640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f0e5806ad10 0x7f0e5806d1d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e60005fd0 tx=0x7f0e60005d40 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:43.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.412+0000 7f0e68ff9640 1 -- 172.21.15.125:0/1850456582 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e5c054820 con 0x7f0e6c0ceb40 2023-10-18T12:00:43.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.599+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 --> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e6c0cc170 con 0x7f0e5806ad10 2023-10-18T12:00:43.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.656+0000 7f0e68ff9640 1 -- 172.21.15.125:0/1850456582 <== mgr.10375 v2:172.21.15.139:6832/178515798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f0e600096d0 con 0x7f0e5806ad10 2023-10-18T12:00:43.659 INFO:teuthology.orchestra.run.smithi125.stdout:Retention removed from path mandatory_fs_argument_test_dir 2023-10-18T12:00:43.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f0e5806ad10 msgr2=0x7f0e5806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f0e5806ad10 0x7f0e5806d1d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e60005fd0 tx=0x7f0e60005d40 comp rx=0 tx=0).stop 2023-10-18T12:00:43.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 msgr2=0x7f0e6c0b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 0x7f0e6c0b20e0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d750 tx=0x7f0e5c00dc10 comp rx=0 tx=0).stop 2023-10-18T12:00:43.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 shutdown_connections 2023-10-18T12:00:43.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f0e5806ad10 0x7f0e5806d1d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e6c0cf340 0x7f0e6c14afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6c15b2a0 0x7f0e6c0b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 --2- 172.21.15.125:0/1850456582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0e6c0ceb40 0x7f0e6c0b20e0 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 >> 172.21.15.125:0/1850456582 conn(0x7f0e6c0d0d60 msgr2=0x7f0e6c15a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:43.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 shutdown_connections 2023-10-18T12:00:43.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.658+0000 7f0e7230f640 1 -- 172.21.15.125:0/1850456582 wait complete. 2023-10-18T12:00:43.672 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2023-10-18T12:00:43.672 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-18T12:00:43.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.777+0000 7effb34e5640 1 -- 172.21.15.125:0/4054772645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac0ceb40 msgr2=0x7effac0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.777+0000 7effb34e5640 1 --2- 172.21.15.125:0/4054772645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac0ceb40 0x7effac0b0650 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7effa4009830 tx=0x7effa402bcd0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.777+0000 7effb34e5640 1 -- 172.21.15.125:0/4054772645 shutdown_connections 2023-10-18T12:00:43.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.777+0000 7effb34e5640 1 --2- 172.21.15.125:0/4054772645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac0ceb40 0x7effac0b0650 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.777+0000 7effb34e5640 1 --2- 172.21.15.125:0/4054772645 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac15b2a0 0x7effac0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.777+0000 7effb34e5640 1 -- 172.21.15.125:0/4054772645 >> 172.21.15.125:0/4054772645 conn(0x7effac0d0d60 msgr2=0x7effac0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:43.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.777+0000 7effb34e5640 1 -- 172.21.15.125:0/4054772645 shutdown_connections 2023-10-18T12:00:43.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.777+0000 7effb34e5640 1 -- 172.21.15.125:0/4054772645 wait complete. 2023-10-18T12:00:43.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.778+0000 7effb34e5640 1 Processor -- start 2023-10-18T12:00:43.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.778+0000 7effb34e5640 1 -- start start 2023-10-18T12:00:43.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb34e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac0ceb40 0x7effac1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb34e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac0cf340 0x7effac15c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb34e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac15b2a0 0x7effac0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb34e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac15d0c0 con 0x7effac15b2a0 2023-10-18T12:00:43.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb34e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac15d3a0 con 0x7effac0cf340 2023-10-18T12:00:43.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb34e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac15d680 con 0x7effac0ceb40 2023-10-18T12:00:43.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb1a82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac15b2a0 0x7effac0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb1a82640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac15b2a0 0x7effac0c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56496/0 (socket says 172.21.15.125:56496) 2023-10-18T12:00:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb1a82640 1 -- 172.21.15.125:0/3882627916 learned_addr learned my addr 172.21.15.125:0/3882627916 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb1a82640 1 -- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac0ceb40 msgr2=0x7effac1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb0a80640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac0cf340 0x7effac15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb1a82640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac0ceb40 0x7effac1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb1a82640 1 -- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac0cf340 msgr2=0x7effac15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb1a82640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac0cf340 0x7effac15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.779+0000 7effb1a82640 1 -- 172.21.15.125:0/3882627916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4009490 con 0x7effac15b2a0 2023-10-18T12:00:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.780+0000 7effb1a82640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac15b2a0 0x7effac0c9e40 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7effa4009780 tx=0x7effa40038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.780+0000 7effa1ffb640 1 -- 172.21.15.125:0/3882627916 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4034440 con 0x7effac15b2a0 2023-10-18T12:00:43.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.780+0000 7effa1ffb640 1 -- 172.21.15.125:0/3882627916 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7effa4034690 con 0x7effac15b2a0 2023-10-18T12:00:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.780+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effac15d960 con 0x7effac15b2a0 2023-10-18T12:00:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.780+0000 7effa1ffb640 1 -- 172.21.15.125:0/3882627916 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4034aa0 con 0x7effac15b2a0 2023-10-18T12:00:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.780+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac157930 con 0x7effac15b2a0 2023-10-18T12:00:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.781+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effac15de80 con 0x7effac15b2a0 2023-10-18T12:00:43.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.781+0000 7effa1ffb640 1 -- 172.21.15.125:0/3882627916 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 133) v1 ==== 89084+0+0 (secure 0 0 0) 0x7effa403b040 con 0x7effac15b2a0 2023-10-18T12:00:43.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.782+0000 7effa1ffb640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7eff9006ad10 0x7eff9006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:43.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.782+0000 7effa1ffb640 1 -- 172.21.15.125:0/3882627916 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7effa40abf70 con 0x7effac15b2a0 2023-10-18T12:00:43.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.782+0000 7effb1281640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7eff9006ad10 0x7eff9006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:43.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.783+0000 7effb1281640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7eff9006ad10 0x7eff9006d1d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7effa80012e0 tx=0x7effa80016d0 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:43.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.786+0000 7effa1ffb640 1 -- 172.21.15.125:0/3882627916 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effa40deca0 con 0x7effac15b2a0 2023-10-18T12:00:43.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.858+0000 7effa1ffb640 1 -- 172.21.15.125:0/3882627916 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 134) v1 ==== 89451+0+0 (secure 0 0 0) 0x7effa4077080 con 0x7effac15b2a0 2023-10-18T12:00:43.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:43.980+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 --> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7effac15de80 con 0x7eff9006ad10 2023-10-18T12:00:44.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.028+0000 7effa1ffb640 1 -- 172.21.15.125:0/3882627916 <== mgr.10375 v2:172.21.15.139:6832/178515798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7effac15de80 con 0x7eff9006ad10 2023-10-18T12:00:44.031 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-18T12:00:44.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7eff9006ad10 msgr2=0x7eff9006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7eff9006ad10 0x7eff9006d1d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7effa80012e0 tx=0x7effa80016d0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac15b2a0 msgr2=0x7effac0c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac15b2a0 0x7effac0c9e40 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7effa4009780 tx=0x7effa40038f0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 shutdown_connections 2023-10-18T12:00:44.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7eff9006ad10 0x7eff9006d1d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac0cf340 0x7effac15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7effac15b2a0 0x7effac0c9e40 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 --2- 172.21.15.125:0/3882627916 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac0ceb40 0x7effac1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 >> 172.21.15.125:0/3882627916 conn(0x7effac0d0d60 msgr2=0x7effac158260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:44.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 shutdown_connections 2023-10-18T12:00:44.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.031+0000 7effb34e5640 1 -- 172.21.15.125:0/3882627916 wait complete. 2023-10-18T12:00:44.044 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-18T12:00:44.044 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-18T12:00:44.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.150+0000 7f27b516c640 1 -- 172.21.15.125:0/3451140165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b0141690 msgr2=0x7f27b00703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.150+0000 7f27b516c640 1 --2- 172.21.15.125:0/3451140165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b0141690 0x7f27b00703e0 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009830 tx=0x7f27a402bc30 comp rx=0 tx=0).stop 2023-10-18T12:00:44.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.150+0000 7f27b516c640 1 -- 172.21.15.125:0/3451140165 shutdown_connections 2023-10-18T12:00:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.150+0000 7f27b516c640 1 --2- 172.21.15.125:0/3451140165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b0141690 0x7f27b00703e0 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.150+0000 7f27b516c640 1 --2- 172.21.15.125:0/3451140165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27b013e520 0x7f27b006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.150+0000 7f27b516c640 1 -- 172.21.15.125:0/3451140165 >> 172.21.15.125:0/3451140165 conn(0x7f27b0069930 msgr2=0x7f27b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.150+0000 7f27b516c640 1 -- 172.21.15.125:0/3451140165 shutdown_connections 2023-10-18T12:00:44.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.150+0000 7f27b516c640 1 -- 172.21.15.125:0/3451140165 wait complete. 2023-10-18T12:00:44.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.151+0000 7f27b516c640 1 Processor -- start 2023-10-18T12:00:44.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.151+0000 7f27b516c640 1 -- start start 2023-10-18T12:00:44.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27b516c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27b006c0a0 0x7f27b013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:44.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27b516c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b013e520 0x7f27b0140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:44.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27b516c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27b0141690 0x7f27b00ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:44.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27b516c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b01591a0 con 0x7f27b013e520 2023-10-18T12:00:44.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27b516c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0159480 con 0x7f27b0141690 2023-10-18T12:00:44.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27b516c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0159760 con 0x7f27b006c0a0 2023-10-18T12:00:44.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b013e520 0x7f27b0140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:44.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b013e520 0x7f27b0140020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56520/0 (socket says 172.21.15.125:56520) 2023-10-18T12:00:44.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27ae59c640 1 -- 172.21.15.125:0/55421998 learned_addr learned my addr 172.21.15.125:0/55421998 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:44.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27aed9d640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27b006c0a0 0x7f27b013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:44.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27af59e640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27b0141690 0x7f27b00ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:44.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27ae59c640 1 -- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27b006c0a0 msgr2=0x7f27b013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27ae59c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27b006c0a0 0x7f27b013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27ae59c640 1 -- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27b0141690 msgr2=0x7f27b00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27ae59c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27b0141690 0x7f27b00ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27ae59c640 1 -- 172.21.15.125:0/55421998 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a4009490 con 0x7f27b013e520 2023-10-18T12:00:44.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.152+0000 7f27af59e640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27b0141690 0x7f27b00ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:44.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.153+0000 7f27ae59c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b013e520 0x7f27b0140020 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009780 tx=0x7f27a4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:44.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.153+0000 7f27977fe640 1 -- 172.21.15.125:0/55421998 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a4003a00 con 0x7f27b013e520 2023-10-18T12:00:44.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.153+0000 7f27977fe640 1 -- 172.21.15.125:0/55421998 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f27a4003c70 con 0x7f27b013e520 2023-10-18T12:00:44.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.153+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b00b1320 con 0x7f27b013e520 2023-10-18T12:00:44.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.153+0000 7f27977fe640 1 -- 172.21.15.125:0/55421998 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a4034440 con 0x7f27b013e520 2023-10-18T12:00:44.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.153+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b013bcb0 con 0x7f27b013e520 2023-10-18T12:00:44.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.154+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b013bea0 con 0x7f27b013e520 2023-10-18T12:00:44.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.154+0000 7f27977fe640 1 -- 172.21.15.125:0/55421998 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 134) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f27a403b040 con 0x7f27b013e520 2023-10-18T12:00:44.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.155+0000 7f27977fe640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f279c06b0d0 0x7f279c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:44.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.155+0000 7f27977fe640 1 -- 172.21.15.125:0/55421998 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f27a40ac2c0 con 0x7f27b013e520 2023-10-18T12:00:44.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.155+0000 7f27aed9d640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f279c06b0d0 0x7f279c06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:44.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.155+0000 7f27aed9d640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f279c06b0d0 0x7f279c06d590 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f27a0005fd0 tx=0x7f27a0009870 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:44.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.159+0000 7f27977fe640 1 -- 172.21.15.125:0/55421998 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27a4076960 con 0x7f27b013e520 2023-10-18T12:00:44.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.344+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 --> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f27b00690e0 con 0x7f279c06b0d0 2023-10-18T12:00:44.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.403+0000 7f27977fe640 1 -- 172.21.15.125:0/55421998 <== mgr.10375 v2:172.21.15.139:6832/178515798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f27b00690e0 con 0x7f279c06b0d0 2023-10-18T12:00:44.406 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2023-10-18T12:00:44.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f279c06b0d0 msgr2=0x7f279c06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f279c06b0d0 0x7f279c06d590 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f27a0005fd0 tx=0x7f27a0009870 comp rx=0 tx=0).stop 2023-10-18T12:00:44.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b013e520 msgr2=0x7f27b0140020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b013e520 0x7f27b0140020 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009780 tx=0x7f27a4004430 comp rx=0 tx=0).stop 2023-10-18T12:00:44.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 shutdown_connections 2023-10-18T12:00:44.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f279c06b0d0 0x7f279c06d590 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27b0141690 0x7f27b00ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27b013e520 0x7f27b0140020 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 --2- 172.21.15.125:0/55421998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27b006c0a0 0x7f27b013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 >> 172.21.15.125:0/55421998 conn(0x7f27b0069930 msgr2=0x7f27b006f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:44.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 shutdown_connections 2023-10-18T12:00:44.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.405+0000 7f27b516c640 1 -- 172.21.15.125:0/55421998 wait complete. 2023-10-18T12:00:44.418 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2023-10-18T12:00:44.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:44.419 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-18T12:00:44.437 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T12:00:44.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/2703087198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1806c0a0 msgr2=0x7f2a18070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/2703087198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1806c0a0 0x7f2a18070920 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f2a1806a780 tx=0x7f2a1002c140 comp rx=0 tx=0).stop 2023-10-18T12:00:44.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/2703087198 shutdown_connections 2023-10-18T12:00:44.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/2703087198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1806c0a0 0x7f2a18070920 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/2703087198 >> v2:172.21.15.139:3300/0 conn(0x7f2a18141690 0x7f2a180703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/2703087198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a1813e520 0x7f2a1806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/2703087198 >> 172.21.15.125:0/2703087198 conn(0x7f2a18069930 msgr2=0x7f2a18069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:44.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/2703087198 shutdown_connections 2023-10-18T12:00:44.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.579+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/2703087198 wait complete. 2023-10-18T12:00:44.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.580+0000 7f2a1e5b6640 1 Processor -- start 2023-10-18T12:00:44.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.580+0000 7f2a1e5b6640 1 -- start start 2023-10-18T12:00:44.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a1e5b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a1806c0a0 0x7f2a180cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:44.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a1e5b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1813e520 0x7f2a180ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:44.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a1e5b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a18141690 0x7f2a180ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:44.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a1e5b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180a8040 con 0x7f2a1813e520 2023-10-18T12:00:44.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a1e5b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180a7430 con 0x7f2a18141690 2023-10-18T12:00:44.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a1e5b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180a7710 con 0x7f2a1806c0a0 2023-10-18T12:00:44.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1813e520 0x7f2a180ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:44.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1813e520 0x7f2a180ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56538/0 (socket says 172.21.15.125:56538) 2023-10-18T12:00:44.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 -- 172.21.15.125:0/3982295894 learned_addr learned my addr 172.21.15.125:0/3982295894 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:44.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a17fff640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a1806c0a0 0x7f2a180cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:44.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a1cb53640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a18141690 0x7f2a180ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:44.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 -- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a1806c0a0 msgr2=0x7f2a180cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a1806c0a0 0x7f2a180cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 -- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a18141690 msgr2=0x7f2a180ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:44.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a18141690 0x7f2a180ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:44.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 -- 172.21.15.125:0/3982295894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a10009900 con 0x7f2a1813e520 2023-10-18T12:00:44.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a17fff640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a1806c0a0 0x7f2a180cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:44.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a177fe640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1813e520 0x7f2a180ad140 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00d740 tx=0x7f2a0c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:44.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a14ff9640 1 -- 172.21.15.125:0/3982295894 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c009c30 con 0x7f2a1813e520 2023-10-18T12:00:44.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a14ff9640 1 -- 172.21.15.125:0/3982295894 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a0c009e80 con 0x7f2a1813e520 2023-10-18T12:00:44.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a180a79f0 con 0x7f2a1813e520 2023-10-18T12:00:44.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.581+0000 7f2a14ff9640 1 -- 172.21.15.125:0/3982295894 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0106c0 con 0x7f2a1813e520 2023-10-18T12:00:44.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.582+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a180cb2e0 con 0x7f2a1813e520 2023-10-18T12:00:44.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.583+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a180a95c0 con 0x7f2a1813e520 2023-10-18T12:00:44.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.583+0000 7f2a14ff9640 1 -- 172.21.15.125:0/3982295894 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 134) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f2a0c010820 con 0x7f2a1813e520 2023-10-18T12:00:44.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.583+0000 7f2a14ff9640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2a0406b100 0x7f2a0406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:44.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.584+0000 7f2a14ff9640 1 -- 172.21.15.125:0/3982295894 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f2a0c08a730 con 0x7f2a1813e520 2023-10-18T12:00:44.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.584+0000 7f2a17fff640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2a0406b100 0x7f2a0406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:44.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.584+0000 7f2a17fff640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2a0406b100 0x7f2a0406d5c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a08005fd0 tx=0x7f2a080094d0 comp rx=0 tx=0).ready entity=mgr.10375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:44.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.588+0000 7f2a14ff9640 1 -- 172.21.15.125:0/3982295894 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a0c054cc0 con 0x7f2a1813e520 2023-10-18T12:00:44.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:44.799+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f2a180690e0 con 0x7f2a1813e520 2023-10-18T12:00:45.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.390+0000 7f2a14ff9640 1 -- 172.21.15.125:0/3982295894 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v135) v1 ==== 93+0+0 (secure 0 0 0) 0x7f2a0c0582e0 con 0x7f2a1813e520 2023-10-18T12:00:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2a0406b100 msgr2=0x7f2a0406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2a0406b100 0x7f2a0406d5c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a08005fd0 tx=0x7f2a080094d0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1813e520 msgr2=0x7f2a180ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1813e520 0x7f2a180ad140 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00d740 tx=0x7f2a0c00dc00 comp rx=0 tx=0).stop 2023-10-18T12:00:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 shutdown_connections 2023-10-18T12:00:45.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2a0406b100 0x7f2a0406d5c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a18141690 0x7f2a180ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a1813e520 0x7f2a180ad140 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 --2- 172.21.15.125:0/3982295894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a1806c0a0 0x7f2a180cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 >> 172.21.15.125:0/3982295894 conn(0x7f2a18069930 msgr2=0x7f2a180aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:45.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.392+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 shutdown_connections 2023-10-18T12:00:45.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.393+0000 7f2a1e5b6640 1 -- 172.21.15.125:0/3982295894 wait complete. 2023-10-18T12:00:45.408 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T12:00:45.409 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:00:45.454 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:00:45.495 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T12:00:45.496 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:00:45.496 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:45.512 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:45.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.653+0000 7fd850505640 1 -- 172.21.15.125:0/484843905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd84815b2a0 msgr2=0x7fd8480d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.653+0000 7fd850505640 1 --2- 172.21.15.125:0/484843905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd84815b2a0 0x7fd8480d1910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd844005fd0 tx=0x7fd84402bd90 comp rx=0 tx=0).stop 2023-10-18T12:00:45.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.653+0000 7fd850505640 1 -- 172.21.15.125:0/484843905 shutdown_connections 2023-10-18T12:00:45.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.653+0000 7fd850505640 1 --2- 172.21.15.125:0/484843905 >> v2:172.21.15.139:3300/0 conn(0x7fd8480ceb40 0x7fd8480b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.653+0000 7fd850505640 1 --2- 172.21.15.125:0/484843905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd84815b2a0 0x7fd8480d1910 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.653+0000 7fd850505640 1 -- 172.21.15.125:0/484843905 >> 172.21.15.125:0/484843905 conn(0x7fd8480d0d60 msgr2=0x7fd8480d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:45.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.653+0000 7fd850505640 1 -- 172.21.15.125:0/484843905 shutdown_connections 2023-10-18T12:00:45.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.653+0000 7fd850505640 1 -- 172.21.15.125:0/484843905 wait complete. 2023-10-18T12:00:45.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.654+0000 7fd850505640 1 Processor -- start 2023-10-18T12:00:45.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.654+0000 7fd850505640 1 -- start start 2023-10-18T12:00:45.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd850505640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8480ceb40 0x7fd8480c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:45.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd850505640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8480cf340 0x7fd84814f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:45.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd850505640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd84815b2a0 0x7fd84815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:45.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd850505640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480c9fb0 con 0x7fd84815b2a0 2023-10-18T12:00:45.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd850505640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481e3850 con 0x7fd8480ceb40 2023-10-18T12:00:45.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd850505640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd848150dc0 con 0x7fd8480cf340 2023-10-18T12:00:45.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd84daa0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8480cf340 0x7fd84814f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:45.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd84e2a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8480ceb40 0x7fd8480c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:45.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd84daa0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8480cf340 0x7fd84814f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:48180/0 (socket says 172.21.15.125:48180) 2023-10-18T12:00:45.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd84daa0640 1 -- 172.21.15.125:0/2783606455 learned_addr learned my addr 172.21.15.125:0/2783606455 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:45.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.655+0000 7fd84eaa2640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd84815b2a0 0x7fd84815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:45.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd84daa0640 1 -- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8480ceb40 msgr2=0x7fd8480c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd84daa0640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8480ceb40 0x7fd8480c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd84daa0640 1 -- 172.21.15.125:0/2783606455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd84815b2a0 msgr2=0x7fd84815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd84daa0640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd84815b2a0 0x7fd84815d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd84daa0640 1 -- 172.21.15.125:0/2783606455 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd840009840 con 0x7fd8480cf340 2023-10-18T12:00:45.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd84e2a1640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8480ceb40 0x7fd8480c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:45.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd84daa0640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8480cf340 0x7fd84814f900 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd844006010 tx=0x7fd84402d9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:45.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd83effd640 1 -- 172.21.15.125:0/2783606455 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84402ddf0 con 0x7fd8480cf340 2023-10-18T12:00:45.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd83effd640 1 -- 172.21.15.125:0/2783606455 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd8440036e0 con 0x7fd8480cf340 2023-10-18T12:00:45.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd83effd640 1 -- 172.21.15.125:0/2783606455 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844003af0 con 0x7fd8480cf340 2023-10-18T12:00:45.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd844009560 con 0x7fd8480cf340 2023-10-18T12:00:45.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.656+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd848151380 con 0x7fd8480cf340 2023-10-18T12:00:45.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.657+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8480690e0 con 0x7fd8480cf340 2023-10-18T12:00:45.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.657+0000 7fd83effd640 1 -- 172.21.15.125:0/2783606455 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fd844003c50 con 0x7fd8480cf340 2023-10-18T12:00:45.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.658+0000 7fd83effd640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fd82c06b110 0x7fd82c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:45.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.658+0000 7fd83effd640 1 -- 172.21.15.125:0/2783606455 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7fd8440ac950 con 0x7fd8480cf340 2023-10-18T12:00:45.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.658+0000 7fd84e2a1640 1 -- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fd82c06b110 msgr2=0x7fd82c06d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:45.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.658+0000 7fd84e2a1640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fd82c06b110 0x7fd82c06d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:45.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.662+0000 7fd83effd640 1 -- 172.21.15.125:0/2783606455 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd844077230 con 0x7fd8480cf340 2023-10-18T12:00:45.785 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:45.782+0000 7fae49cb6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:00:45.785 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:45.783+0000 7faf43c3b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:00:45.831 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:45.828+0000 7faf43c3b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:00:45.831 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:45.828+0000 7fae49cb6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:00:45.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.837+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8480d0b40 con 0x7fd8480cf340 2023-10-18T12:00:45.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.839+0000 7fd83effd640 1 -- 172.21.15.125:0/2783606455 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+21087 (secure 0 0 0) 0x7fd84407a850 con 0x7fd8480cf340 2023-10-18T12:00:45.842 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:45.842 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":226,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:40.842783+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/1417331710":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6833/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6832/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3836307336":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/2081746818":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1472046677":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3403101268":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1324329019":"2023-10-19T12:00:40.842747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:45.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fd82c06b110 msgr2=0x7fd82c06d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:45.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fd82c06b110 0x7fd82c06d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8480cf340 msgr2=0x7fd84814f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8480cf340 0x7fd84814f900 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd844006010 tx=0x7fd84402d9f0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 shutdown_connections 2023-10-18T12:00:45.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fd82c06b110 0x7fd82c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd84815b2a0 0x7fd84815d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8480cf340 0x7fd84814f900 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 --2- 172.21.15.125:0/2783606455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8480ceb40 0x7fd8480c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 >> 172.21.15.125:0/2783606455 conn(0x7fd8480d0d60 msgr2=0x7fd8480ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:45.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 shutdown_connections 2023-10-18T12:00:45.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.841+0000 7fd850505640 1 -- 172.21.15.125:0/2783606455 wait complete. 2023-10-18T12:00:45.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:00:18.207564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:00:19.369758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '221', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-18T12:00:30.176449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '224', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-18T12:00:30.730388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '225', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:00:45.856 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:00:45.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.960+0000 7f4177875640 1 -- 172.21.15.125:0/3538463430 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 msgr2=0x7f41700d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.960+0000 7f4177875640 1 --2- 172.21.15.125:0/3538463430 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 0x7f41700d1910 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f417008a120 tx=0x7f416002bdb0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.960+0000 7f4177875640 1 -- 172.21.15.125:0/3538463430 shutdown_connections 2023-10-18T12:00:45.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.960+0000 7f4177875640 1 --2- 172.21.15.125:0/3538463430 >> v2:172.21.15.139:3300/0 conn(0x7f417015b2a0 0x7f41700cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.960+0000 7f4177875640 1 --2- 172.21.15.125:0/3538463430 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 0x7f41700d1910 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.960+0000 7f4177875640 1 -- 172.21.15.125:0/3538463430 >> 172.21.15.125:0/3538463430 conn(0x7f41700d0d60 msgr2=0x7f41700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:45.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.960+0000 7f4177875640 1 -- 172.21.15.125:0/3538463430 shutdown_connections 2023-10-18T12:00:45.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.960+0000 7f4177875640 1 -- 172.21.15.125:0/3538463430 wait complete. 2023-10-18T12:00:45.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.961+0000 7f4177875640 1 Processor -- start 2023-10-18T12:00:45.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.961+0000 7f4177875640 1 -- start start 2023-10-18T12:00:45.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4177875640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41700ceb40 0x7f41700c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:45.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4177875640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 0x7f417006b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:45.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4177875640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417015b2a0 0x7f417014bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:45.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4177875640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41700ca0d0 con 0x7f41700cf340 2023-10-18T12:00:45.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4177875640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41700ca3b0 con 0x7f41700ceb40 2023-10-18T12:00:45.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4177875640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41700ca690 con 0x7f417015b2a0 2023-10-18T12:00:45.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4174e10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 0x7f417006b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:45.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4175611640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41700ceb40 0x7f41700c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:45.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4174e10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 0x7f417006b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56594/0 (socket says 172.21.15.125:56594) 2023-10-18T12:00:45.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4174e10640 1 -- 172.21.15.125:0/3937790165 learned_addr learned my addr 172.21.15.125:0/3937790165 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:45.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4175e12640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417015b2a0 0x7f417014bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:45.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4175611640 1 -- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417015b2a0 msgr2=0x7f417014bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4175611640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417015b2a0 0x7f417014bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4175611640 1 -- 172.21.15.125:0/3937790165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 msgr2=0x7f417006b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:45.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4175611640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 0x7f417006b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:45.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4175611640 1 -- 172.21.15.125:0/3937790165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4160009560 con 0x7f41700ceb40 2023-10-18T12:00:45.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.962+0000 7f4174e10640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 0x7f417006b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:45.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.963+0000 7f4175611640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41700ceb40 0x7f41700c87e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4160033000 tx=0x7f416002da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:45.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.963+0000 7f4165ffb640 1 -- 172.21.15.125:0/3937790165 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41600038c0 con 0x7f41700ceb40 2023-10-18T12:00:45.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.963+0000 7f4165ffb640 1 -- 172.21.15.125:0/3937790165 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4160003b30 con 0x7f41700ceb40 2023-10-18T12:00:45.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.963+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41700ca970 con 0x7f41700ceb40 2023-10-18T12:00:45.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.963+0000 7f4165ffb640 1 -- 172.21.15.125:0/3937790165 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416003e450 con 0x7f41700ceb40 2023-10-18T12:00:45.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.963+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417006b8e0 con 0x7f41700ceb40 2023-10-18T12:00:45.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.964+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41700b3950 con 0x7f41700ceb40 2023-10-18T12:00:45.972 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.964+0000 7f4165ffb640 1 -- 172.21.15.125:0/3937790165 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f416003e5b0 con 0x7f41700ceb40 2023-10-18T12:00:45.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.965+0000 7f4165ffb640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f415406b110 0x7f415406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:45.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.965+0000 7f4165ffb640 1 -- 172.21.15.125:0/3937790165 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f41600ac9a0 con 0x7f41700ceb40 2023-10-18T12:00:45.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.965+0000 7f4174e10640 1 -- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f415406b110 msgr2=0x7f415406d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:45.973 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.965+0000 7f4174e10640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f415406b110 0x7f415406d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:45.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:45.969+0000 7f4165ffb640 1 -- 172.21.15.125:0/3937790165 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41700b3950 con 0x7f41700ceb40 2023-10-18T12:00:46.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.142+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41700b3950 con 0x7f41700ceb40 2023-10-18T12:00:46.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.142+0000 7f4165ffb640 1 -- 172.21.15.125:0/3937790165 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v226) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41700b3950 con 0x7f41700ceb40 2023-10-18T12:00:46.145 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:00:46.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f415406b110 msgr2=0x7f415406d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:46.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f415406b110 0x7f415406d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41700ceb40 msgr2=0x7f41700c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41700ceb40 0x7f41700c87e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4160033000 tx=0x7f416002da80 comp rx=0 tx=0).stop 2023-10-18T12:00:46.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 shutdown_connections 2023-10-18T12:00:46.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f415406b110 0x7f415406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f417015b2a0 0x7f417014bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41700cf340 0x7f417006b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 --2- 172.21.15.125:0/3937790165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41700ceb40 0x7f41700c87e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 >> 172.21.15.125:0/3937790165 conn(0x7f41700d0d60 msgr2=0x7f41700aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:46.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 shutdown_connections 2023-10-18T12:00:46.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.144+0000 7f4177875640 1 -- 172.21.15.125:0/3937790165 wait complete. 2023-10-18T12:00:46.158 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:00:46.223 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.220+0000 7faf43c3b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:00:46.241 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.238+0000 7fae49cb6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:00:46.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.263+0000 7f706afd7640 1 -- 172.21.15.125:0/2814025190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 msgr2=0x7f706406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.263+0000 7f706afd7640 1 --2- 172.21.15.125:0/2814025190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 0x7f706406d260 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f7050009830 tx=0x7f705002bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:46.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.263+0000 7f706afd7640 1 -- 172.21.15.125:0/2814025190 shutdown_connections 2023-10-18T12:00:46.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.263+0000 7f706afd7640 1 --2- 172.21.15.125:0/2814025190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 0x7f706406d260 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.263+0000 7f706afd7640 1 -- 172.21.15.125:0/2814025190 >> 172.21.15.125:0/2814025190 conn(0x7f70640b0670 msgr2=0x7f70640b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:46.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.263+0000 7f706afd7640 1 -- 172.21.15.125:0/2814025190 shutdown_connections 2023-10-18T12:00:46.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.263+0000 7f706afd7640 1 -- 172.21.15.125:0/2814025190 wait complete. 2023-10-18T12:00:46.267 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.264+0000 7faf43c3b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:00:46.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.264+0000 7f706afd7640 1 Processor -- start 2023-10-18T12:00:46.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.264+0000 7f706afd7640 1 -- start start 2023-10-18T12:00:46.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f706afd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 0x7f7064159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f706afd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7064159060 0x7f70640b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f706afd7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f706415ccf0 0x7f706414e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f706afd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064159fd0 con 0x7f70640ce240 2023-10-18T12:00:46.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f706afd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70640b1ff0 con 0x7f7064159060 2023-10-18T12:00:46.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f706afd7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706415add0 con 0x7f706415ccf0 2023-10-18T12:00:46.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 0x7f7064159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 0x7f7064159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56608/0 (socket says 172.21.15.125:56608) 2023-10-18T12:00:46.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 -- 172.21.15.125:0/2813304560 learned_addr learned my addr 172.21.15.125:0/2813304560 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:46.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7063fff640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7064159060 0x7f70640b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7069574640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f706415ccf0 0x7f706414e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 -- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f706415ccf0 msgr2=0x7f706414e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f706415ccf0 0x7f706414e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 -- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7064159060 msgr2=0x7f70640b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7064159060 0x7f70640b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 -- 172.21.15.125:0/2813304560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7050009490 con 0x7f70640ce240 2023-10-18T12:00:46.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7069574640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f706415ccf0 0x7f706414e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:46.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f7068d73640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 0x7f7064159a90 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f705800c500 tx=0x7f705800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:46.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f70617fa640 1 -- 172.21.15.125:0/2813304560 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058012830 con 0x7f70640ce240 2023-10-18T12:00:46.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f70617fa640 1 -- 172.21.15.125:0/2813304560 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7058012a80 con 0x7f70640ce240 2023-10-18T12:00:46.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f70617fa640 1 -- 172.21.15.125:0/2813304560 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058012e90 con 0x7f70640ce240 2023-10-18T12:00:46.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706415a4a0 con 0x7f70640ce240 2023-10-18T12:00:46.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.265+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706415aa60 con 0x7f70640ce240 2023-10-18T12:00:46.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.267+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706415a220 con 0x7f70640ce240 2023-10-18T12:00:46.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.267+0000 7f70617fa640 1 -- 172.21.15.125:0/2813304560 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f705801a3b0 con 0x7f70640ce240 2023-10-18T12:00:46.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.267+0000 7f70617fa640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f704806b110 0x7f704806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.268+0000 7f7063fff640 1 -- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f704806b110 msgr2=0x7f704806d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:46.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.268+0000 7f70617fa640 1 -- 172.21.15.125:0/2813304560 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f7058089f30 con 0x7f70640ce240 2023-10-18T12:00:46.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.268+0000 7f7063fff640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f704806b110 0x7f704806d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:46.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.272+0000 7f70617fa640 1 -- 172.21.15.125:0/2813304560 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7058054670 con 0x7f70640ce240 2023-10-18T12:00:46.288 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.285+0000 7fae49cb6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:00:46.309 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.307+0000 7faf43c3b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:00:46.332 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.330+0000 7fae49cb6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:00:46.360 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.358+0000 7faf43c3b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:00:46.385 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.383+0000 7fae49cb6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:00:46.403 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.401+0000 7faf43c3b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:00:46.428 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.426+0000 7fae49cb6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:00:46.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.445+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70640675f0 con 0x7f70640ce240 2023-10-18T12:00:46.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.446+0000 7f70617fa640 1 -- 172.21.15.125:0/2813304560 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v226) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7058057c90 con 0x7f70640ce240 2023-10-18T12:00:46.449 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:00:46.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f704806b110 msgr2=0x7f704806d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:46.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f704806b110 0x7f704806d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 msgr2=0x7f7064159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 0x7f7064159a90 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f705800c500 tx=0x7f705800c9c0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 shutdown_connections 2023-10-18T12:00:46.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f704806b110 0x7f704806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7064159060 0x7f70640b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f706415ccf0 0x7f706414e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 --2- 172.21.15.125:0/2813304560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70640ce240 0x7f7064159a90 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 >> 172.21.15.125:0/2813304560 conn(0x7f70640b0670 msgr2=0x7f70640cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:46.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 shutdown_connections 2023-10-18T12:00:46.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.448+0000 7f706afd7640 1 -- 172.21.15.125:0/2813304560 wait complete. 2023-10-18T12:00:46.461 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:00:46.502 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.499+0000 7faf43c3b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:00:46.532 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.530+0000 7fae49cb6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:00:46.554 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.552+0000 7faf43c3b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:46.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.564+0000 7fb45393c640 1 -- 172.21.15.125:0/3009961907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c15b2a0 msgr2=0x7fb44c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.564+0000 7fb45393c640 1 --2- 172.21.15.125:0/3009961907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c15b2a0 0x7fb44c0d1910 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009830 tx=0x7fb43c02bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:46.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.565+0000 7fb45393c640 1 -- 172.21.15.125:0/3009961907 shutdown_connections 2023-10-18T12:00:46.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.565+0000 7fb45393c640 1 --2- 172.21.15.125:0/3009961907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c15b2a0 0x7fb44c0d1910 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.565+0000 7fb45393c640 1 -- 172.21.15.125:0/3009961907 >> 172.21.15.125:0/3009961907 conn(0x7fb44c0d0d60 msgr2=0x7fb44c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:46.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.565+0000 7fb45393c640 1 -- 172.21.15.125:0/3009961907 shutdown_connections 2023-10-18T12:00:46.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.565+0000 7fb45393c640 1 -- 172.21.15.125:0/3009961907 wait complete. 2023-10-18T12:00:46.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.566+0000 7fb45393c640 1 Processor -- start 2023-10-18T12:00:46.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.566+0000 7fb45393c640 1 -- start start 2023-10-18T12:00:46.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.566+0000 7fb45393c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c0ceb40 0x7fb44c0c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.566+0000 7fb45393c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb44c0cf340 0x7fb44c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.566+0000 7fb45393c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb44c15b2a0 0x7fb44c15c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.566+0000 7fb45393c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c15e670 con 0x7fb44c0ceb40 2023-10-18T12:00:46.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.566+0000 7fb45393c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c15de60 con 0x7fb44c0cf340 2023-10-18T12:00:46.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.566+0000 7fb45393c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c15e0b0 con 0x7fb44c15b2a0 2023-10-18T12:00:46.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c0ceb40 0x7fb44c0c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c0ceb40 0x7fb44c0c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56626/0 (socket says 172.21.15.125:56626) 2023-10-18T12:00:46.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 -- 172.21.15.125:0/3394274516 learned_addr learned my addr 172.21.15.125:0/3394274516 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:46.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb450ed7640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb44c0cf340 0x7fb44c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb451ed9640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb44c15b2a0 0x7fb44c15c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 -- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb44c15b2a0 msgr2=0x7fb44c15c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb44c15b2a0 0x7fb44c15c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 -- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb44c0cf340 msgr2=0x7fb44c1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb44c0cf340 0x7fb44c1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 -- 172.21.15.125:0/3394274516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c009490 con 0x7fb44c0ceb40 2023-10-18T12:00:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb451ed9640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb44c15b2a0 0x7fb44c15c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb4516d8640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c0ceb40 0x7fb44c0c87c0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fb44800c450 tx=0x7fb44800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb441ffb640 1 -- 172.21.15.125:0/3394274516 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480052d0 con 0x7fb44c0ceb40 2023-10-18T12:00:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb441ffb640 1 -- 172.21.15.125:0/3394274516 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb448005430 con 0x7fb44c0ceb40 2023-10-18T12:00:46.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c15e390 con 0x7fb44c0ceb40 2023-10-18T12:00:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb441ffb640 1 -- 172.21.15.125:0/3394274516 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448012890 con 0x7fb44c0ceb40 2023-10-18T12:00:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.567+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c0c8d00 con 0x7fb44c0ceb40 2023-10-18T12:00:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.568+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c15b6b0 con 0x7fb44c0ceb40 2023-10-18T12:00:46.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.569+0000 7fb441ffb640 1 -- 172.21.15.125:0/3394274516 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fb4480129f0 con 0x7fb44c0ceb40 2023-10-18T12:00:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.569+0000 7fb441ffb640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fb43006b110 0x7fb43006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.569+0000 7fb450ed7640 1 -- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fb43006b110 msgr2=0x7fb43006d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:46.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.569+0000 7fb441ffb640 1 -- 172.21.15.125:0/3394274516 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7fb448089cb0 con 0x7fb44c0ceb40 2023-10-18T12:00:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.569+0000 7fb450ed7640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fb43006b110 0x7fb43006d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:46.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.573+0000 7fb441ffb640 1 -- 172.21.15.125:0/3394274516 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb44c15b6b0 con 0x7fb44c0ceb40 2023-10-18T12:00:46.589 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.587+0000 7fae49cb6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:46.596 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.594+0000 7faf43c3b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:00:46.631 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.629+0000 7fae49cb6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:00:46.644 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.642+0000 7faf43c3b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:00:46.680 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.677+0000 7fae49cb6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:00:46.686 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:46.684+0000 7faf43c3b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:00:46.723 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:46.720+0000 7fae49cb6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:00:46.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.743+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb44c15b6b0 con 0x7fb44c0ceb40 2023-10-18T12:00:46.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.744+0000 7fb441ffb640 1 -- 172.21.15.125:0/3394274516 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v226) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb44c15b6b0 con 0x7fb44c0ceb40 2023-10-18T12:00:46.747 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:00:46.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fb43006b110 msgr2=0x7fb43006d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:46.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fb43006b110 0x7fb43006d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c0ceb40 msgr2=0x7fb44c0c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c0ceb40 0x7fb44c0c87c0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fb44800c450 tx=0x7fb44800c910 comp rx=0 tx=0).stop 2023-10-18T12:00:46.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 shutdown_connections 2023-10-18T12:00:46.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fb43006b110 0x7fb43006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb44c0cf340 0x7fb44c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb44c15b2a0 0x7fb44c15c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 --2- 172.21.15.125:0/3394274516 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb44c0ceb40 0x7fb44c0c87c0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 >> 172.21.15.125:0/3394274516 conn(0x7fb44c0d0d60 msgr2=0x7fb44c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:46.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 shutdown_connections 2023-10-18T12:00:46.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.746+0000 7fb45393c640 1 -- 172.21.15.125:0/3394274516 wait complete. 2023-10-18T12:00:46.759 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-18T12:00:46.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.864+0000 7f7158abb640 1 -- 172.21.15.125:0/1226477145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 msgr2=0x7f71540d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.864+0000 7f7158abb640 1 --2- 172.21.15.125:0/1226477145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 0x7f71540d1910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7144009860 tx=0x7f714402bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:46.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.865+0000 7f7158abb640 1 -- 172.21.15.125:0/1226477145 shutdown_connections 2023-10-18T12:00:46.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.865+0000 7f7158abb640 1 --2- 172.21.15.125:0/1226477145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 0x7f71540d1910 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.865+0000 7f7158abb640 1 -- 172.21.15.125:0/1226477145 >> 172.21.15.125:0/1226477145 conn(0x7f71540d0d60 msgr2=0x7f71540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:46.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.865+0000 7f7158abb640 1 -- 172.21.15.125:0/1226477145 shutdown_connections 2023-10-18T12:00:46.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.865+0000 7f7158abb640 1 -- 172.21.15.125:0/1226477145 wait complete. 2023-10-18T12:00:46.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.866+0000 7f7158abb640 1 Processor -- start 2023-10-18T12:00:46.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.866+0000 7f7158abb640 1 -- start start 2023-10-18T12:00:46.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f7158abb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71540ceb40 0x7f7154150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f7158abb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 0x7f71541e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f7158abb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f715415b2a0 0x7f715406a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f7158abb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71541e4830 con 0x7f71540ceb40 2023-10-18T12:00:46.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f7158abb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7154150710 con 0x7f715415b2a0 2023-10-18T12:00:46.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f7158abb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71541509f0 con 0x7f71540cf340 2023-10-18T12:00:46.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f715259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71540ceb40 0x7f7154150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f715259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71540ceb40 0x7f7154150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56638/0 (socket says 172.21.15.125:56638) 2023-10-18T12:00:46.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f715259c640 1 -- 172.21.15.125:0/1720905013 learned_addr learned my addr 172.21.15.125:0/1720905013 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:46.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714bfff640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 0x7f71541e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f7152d9d640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f715415b2a0 0x7f715406a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:46.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714bfff640 1 -- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f715415b2a0 msgr2=0x7f715406a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714bfff640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f715415b2a0 0x7f715406a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714bfff640 1 -- 172.21.15.125:0/1720905013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71540ceb40 msgr2=0x7f7154150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:46.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714bfff640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71540ceb40 0x7f7154150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:46.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714bfff640 1 -- 172.21.15.125:0/1720905013 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7144009490 con 0x7f71540cf340 2023-10-18T12:00:46.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f715259c640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71540ceb40 0x7f7154150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:46.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714bfff640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 0x7f71541e42f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f713c00c390 tx=0x7f713c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:46.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714affd640 1 -- 172.21.15.125:0/1720905013 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c011790 con 0x7f71540cf340 2023-10-18T12:00:46.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714affd640 1 -- 172.21.15.125:0/1720905013 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f713c011a00 con 0x7f71540cf340 2023-10-18T12:00:46.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f714affd640 1 -- 172.21.15.125:0/1720905013 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c011e90 con 0x7f71540cf340 2023-10-18T12:00:46.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.867+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7154150cd0 con 0x7f71540cf340 2023-10-18T12:00:46.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.868+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715406a740 con 0x7f71540cf340 2023-10-18T12:00:46.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.868+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71540690e0 con 0x7f71540cf340 2023-10-18T12:00:46.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.869+0000 7f714affd640 1 -- 172.21.15.125:0/1720905013 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f713c01c460 con 0x7f71540cf340 2023-10-18T12:00:46.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.869+0000 7f714affd640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f713406b110 0x7f713406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:46.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.869+0000 7f714affd640 1 -- 172.21.15.125:0/1720905013 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f713c089df0 con 0x7f71540cf340 2023-10-18T12:00:46.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.869+0000 7f715259c640 1 -- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f713406b110 msgr2=0x7f713406d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:46.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.869+0000 7f715259c640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f713406b110 0x7f713406d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:46.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:46.874+0000 7f714affd640 1 -- 172.21.15.125:0/1720905013 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f713c0545d0 con 0x7f71540cf340 2023-10-18T12:00:47.002 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.000+0000 7faf43c3b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:00:47.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.047+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71540d0b40 con 0x7f71540cf340 2023-10-18T12:00:47.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.048+0000 7f714affd640 1 -- 172.21.15.125:0/1720905013 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v226) v1 ==== 107+0+11 (secure 0 0 0) 0x7f713c057bf0 con 0x7f71540cf340 2023-10-18T12:00:47.051 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:00:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f713406b110 msgr2=0x7f713406d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f713406b110 0x7f713406d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 msgr2=0x7f71541e42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 0x7f71541e42f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f713c00c390 tx=0x7f713c00c850 comp rx=0 tx=0).stop 2023-10-18T12:00:47.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 shutdown_connections 2023-10-18T12:00:47.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f713406b110 0x7f713406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f715415b2a0 0x7f715406a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71540cf340 0x7f71541e42f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 --2- 172.21.15.125:0/1720905013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71540ceb40 0x7f7154150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 >> 172.21.15.125:0/1720905013 conn(0x7f71540d0d60 msgr2=0x7f71540b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:47.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 shutdown_connections 2023-10-18T12:00:47.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.050+0000 7f7158abb640 1 -- 172.21.15.125:0/1720905013 wait complete. 2023-10-18T12:00:47.055 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.051+0000 7fae49cb6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:00:47.063 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-18T12:00:47.134 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.132+0000 7faf43c3b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:00:47.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.169+0000 7f4e4ab65640 1 -- 172.21.15.125:0/4189569927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 msgr2=0x7f4e440b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.169+0000 7f4e4ab65640 1 --2- 172.21.15.125:0/4189569927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 0x7f4e440b0650 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c005fd0 tx=0x7f4e3c02c0e0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.169+0000 7f4e4ab65640 1 -- 172.21.15.125:0/4189569927 shutdown_connections 2023-10-18T12:00:47.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.169+0000 7f4e4ab65640 1 --2- 172.21.15.125:0/4189569927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 0x7f4e440b0650 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.169+0000 7f4e4ab65640 1 -- 172.21.15.125:0/4189569927 >> 172.21.15.125:0/4189569927 conn(0x7f4e440d0d60 msgr2=0x7f4e440d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:47.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.169+0000 7f4e4ab65640 1 -- 172.21.15.125:0/4189569927 shutdown_connections 2023-10-18T12:00:47.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.169+0000 7f4e4ab65640 1 -- 172.21.15.125:0/4189569927 wait complete. 2023-10-18T12:00:47.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.170+0000 7f4e4ab65640 1 Processor -- start 2023-10-18T12:00:47.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.170+0000 7f4e4ab65640 1 -- start start 2023-10-18T12:00:47.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e4ab65640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e440cf340 0x7f4e44157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:47.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e4ab65640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 0x7f4e440b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:47.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e4ab65640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e4415b2a0 0x7f4e4409ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:47.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e4ab65640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440b4f00 con 0x7f4e440cf340 2023-10-18T12:00:47.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e4ab65640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440b4280 con 0x7f4e440d1910 2023-10-18T12:00:47.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e4ab65640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440b4560 con 0x7f4e4415b2a0 2023-10-18T12:00:47.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e48901640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e440cf340 0x7f4e44157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:47.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 0x7f4e440b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:47.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e48901640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e440cf340 0x7f4e44157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56640/0 (socket says 172.21.15.125:56640) 2023-10-18T12:00:47.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e48901640 1 -- 172.21.15.125:0/3182985174 learned_addr learned my addr 172.21.15.125:0/3182985174 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:47.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e49102640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e4415b2a0 0x7f4e4409ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:47.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e43fff640 1 -- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e4415b2a0 msgr2=0x7f4e4409ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e43fff640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e4415b2a0 0x7f4e4409ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e43fff640 1 -- 172.21.15.125:0/3182985174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e440cf340 msgr2=0x7f4e44157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e43fff640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e440cf340 0x7f4e44157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.171+0000 7f4e43fff640 1 -- 172.21.15.125:0/3182985174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c009840 con 0x7f4e440d1910 2023-10-18T12:00:47.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.172+0000 7f4e43fff640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 0x7f4e440b3950 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4e440b00b0 tx=0x7f4e3000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:47.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.172+0000 7f4e417fa640 1 -- 172.21.15.125:0/3182985174 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30011850 con 0x7f4e440d1910 2023-10-18T12:00:47.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.172+0000 7f4e417fa640 1 -- 172.21.15.125:0/3182985174 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e30011aa0 con 0x7f4e440d1910 2023-10-18T12:00:47.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.172+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e440b4840 con 0x7f4e440d1910 2023-10-18T12:00:47.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.172+0000 7f4e417fa640 1 -- 172.21.15.125:0/3182985174 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30012420 con 0x7f4e440d1910 2023-10-18T12:00:47.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.172+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e441582b0 con 0x7f4e440d1910 2023-10-18T12:00:47.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.173+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e440b5040 con 0x7f4e440d1910 2023-10-18T12:00:47.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.173+0000 7f4e417fa640 1 -- 172.21.15.125:0/3182985174 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f4e30012580 con 0x7f4e440d1910 2023-10-18T12:00:47.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.174+0000 7f4e417fa640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f4e2806b110 0x7f4e2806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:47.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.174+0000 7f4e417fa640 1 -- 172.21.15.125:0/3182985174 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f4e3008a030 con 0x7f4e440d1910 2023-10-18T12:00:47.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.174+0000 7f4e48901640 1 -- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f4e2806b110 msgr2=0x7f4e2806d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:47.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.174+0000 7f4e48901640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f4e2806b110 0x7f4e2806d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:47.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.178+0000 7f4e417fa640 1 -- 172.21.15.125:0/3182985174 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e440b5040 con 0x7f4e440d1910 2023-10-18T12:00:47.194 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.192+0000 7fae49cb6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:00:47.228 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.225+0000 7faf43c3b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:00:47.270 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.267+0000 7faf43c3b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:00:47.290 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.287+0000 7fae49cb6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:00:47.334 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.332+0000 7fae49cb6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:00:47.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.353+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e440b5040 con 0x7f4e440d1910 2023-10-18T12:00:47.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.354+0000 7f4e417fa640 1 -- 172.21.15.125:0/3182985174 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v226) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4e440b5040 con 0x7f4e440d1910 2023-10-18T12:00:47.357 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:00:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f4e2806b110 msgr2=0x7f4e2806d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f4e2806b110 0x7f4e2806d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 msgr2=0x7f4e440b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 0x7f4e440b3950 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4e440b00b0 tx=0x7f4e3000c820 comp rx=0 tx=0).stop 2023-10-18T12:00:47.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 shutdown_connections 2023-10-18T12:00:47.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f4e2806b110 0x7f4e2806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e440d1910 0x7f4e440b3950 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e4415b2a0 0x7f4e4409ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 --2- 172.21.15.125:0/3182985174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e440cf340 0x7f4e44157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 >> 172.21.15.125:0/3182985174 conn(0x7f4e440d0d60 msgr2=0x7f4e440b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:47.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 shutdown_connections 2023-10-18T12:00:47.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.356+0000 7f4e4ab65640 1 -- 172.21.15.125:0/3182985174 wait complete. 2023-10-18T12:00:47.365 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.363+0000 7faf43c3b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:47.370 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T12:00:47.437 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.434+0000 7fae49cb6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:47.461 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.458+0000 7faf43c3b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:00:47.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.472+0000 7ff37ab97640 1 -- 172.21.15.125:0/2969792317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740ceb40 msgr2=0x7ff3740b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.472+0000 7ff37ab97640 1 --2- 172.21.15.125:0/2969792317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740ceb40 0x7ff3740b0650 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7ff37408a120 tx=0x7ff36c02c080 comp rx=0 tx=0).stop 2023-10-18T12:00:47.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.472+0000 7ff37ab97640 1 -- 172.21.15.125:0/2969792317 shutdown_connections 2023-10-18T12:00:47.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.472+0000 7ff37ab97640 1 --2- 172.21.15.125:0/2969792317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740ceb40 0x7ff3740b0650 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.472+0000 7ff37ab97640 1 --2- 172.21.15.125:0/2969792317 >> v2:172.21.15.139:3300/0 conn(0x7ff37415b2a0 0x7ff3740d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.472+0000 7ff37ab97640 1 -- 172.21.15.125:0/2969792317 >> 172.21.15.125:0/2969792317 conn(0x7ff3740d0d60 msgr2=0x7ff3740d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:47.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.472+0000 7ff37ab97640 1 -- 172.21.15.125:0/2969792317 shutdown_connections 2023-10-18T12:00:47.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.472+0000 7ff37ab97640 1 -- 172.21.15.125:0/2969792317 wait complete. 2023-10-18T12:00:47.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.473+0000 7ff37ab97640 1 Processor -- start 2023-10-18T12:00:47.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.473+0000 7ff37ab97640 1 -- start start 2023-10-18T12:00:47.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff37ab97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740ceb40 0x7ff3741524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:47.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff37ab97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740cf340 0x7ff374159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:47.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff37ab97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff37415b2a0 0x7ff374159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:47.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff37ab97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740b4e50 con 0x7ff3740cf340 2023-10-18T12:00:47.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff37ab97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740b3e20 con 0x7ff3740ceb40 2023-10-18T12:00:47.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff37ab97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740b4090 con 0x7ff37415b2a0 2023-10-18T12:00:47.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff373fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740cf340 0x7ff374159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:47.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff373fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740cf340 0x7ff374159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54838/0 (socket says 172.21.15.125:54838) 2023-10-18T12:00:47.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff378933640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740ceb40 0x7ff3741524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:47.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff373fff640 1 -- 172.21.15.125:0/3340487056 learned_addr learned my addr 172.21.15.125:0/3340487056 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:47.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff379134640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff37415b2a0 0x7ff374159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:47.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff378933640 1 -- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff37415b2a0 msgr2=0x7ff374159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff378933640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff37415b2a0 0x7ff374159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff378933640 1 -- 172.21.15.125:0/3340487056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740cf340 msgr2=0x7ff374159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff378933640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740cf340 0x7ff374159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.474+0000 7ff378933640 1 -- 172.21.15.125:0/3340487056 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c009840 con 0x7ff3740ceb40 2023-10-18T12:00:47.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.475+0000 7ff373fff640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740cf340 0x7ff374159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:00:47.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.475+0000 7ff378933640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740ceb40 0x7ff3741524a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff36800c450 tx=0x7ff36800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:47.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.475+0000 7ff3717fa640 1 -- 172.21.15.125:0/3340487056 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3680035f0 con 0x7ff3740ceb40 2023-10-18T12:00:47.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.475+0000 7ff3717fa640 1 -- 172.21.15.125:0/3340487056 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff368003750 con 0x7ff3740ceb40 2023-10-18T12:00:47.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.475+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3740b4370 con 0x7ff3740ceb40 2023-10-18T12:00:47.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.475+0000 7ff3717fa640 1 -- 172.21.15.125:0/3340487056 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368003b60 con 0x7ff3740ceb40 2023-10-18T12:00:47.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.475+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3740b4930 con 0x7ff3740ceb40 2023-10-18T12:00:47.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.476+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3740690e0 con 0x7ff3740ceb40 2023-10-18T12:00:47.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.476+0000 7ff3717fa640 1 -- 172.21.15.125:0/3340487056 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7ff368003cc0 con 0x7ff3740ceb40 2023-10-18T12:00:47.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.477+0000 7ff3717fa640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7ff35806b110 0x7ff35806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:47.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.477+0000 7ff373fff640 1 -- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7ff35806b110 msgr2=0x7ff35806d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:47.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.477+0000 7ff3717fa640 1 -- 172.21.15.125:0/3340487056 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7ff3680897b0 con 0x7ff3740ceb40 2023-10-18T12:00:47.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.477+0000 7ff373fff640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7ff35806b110 0x7ff35806d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:47.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.481+0000 7ff3717fa640 1 -- 172.21.15.125:0/3340487056 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff368053f90 con 0x7ff3740ceb40 2023-10-18T12:00:47.533 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.530+0000 7fae49cb6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:00:47.598 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.596+0000 7faf43c3b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:47.641 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.638+0000 7faf43c3b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:47.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.666+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff3740b0d00 con 0x7ff3740ceb40 2023-10-18T12:00:47.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.667+0000 7ff3717fa640 1 -- 172.21.15.125:0/3340487056 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v226) v1 ==== 80+0+478 (secure 0 0 0) 0x7ff368017080 con 0x7ff3740ceb40 2023-10-18T12:00:47.671 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1417331710 2023-10-19T12:00:40.842747+0000 2023-10-18T12:00:47.671 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/2362843834 2023-10-19T12:00:40.842747+0000 2023-10-18T12:00:47.671 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/2362843834 2023-10-19T12:00:40.842747+0000 2023-10-18T12:00:47.672 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3836307336 2023-10-19T12:00:40.842747+0000 2023-10-18T12:00:47.672 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2081746818 2023-10-19T12:00:40.842747+0000 2023-10-18T12:00:47.672 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1472046677 2023-10-19T12:00:40.842747+0000 2023-10-18T12:00:47.672 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3403101268 2023-10-19T12:00:40.842747+0000 2023-10-18T12:00:47.672 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1324329019 2023-10-19T12:00:40.842747+0000 2023-10-18T12:00:47.673 INFO:teuthology.orchestra.run.smithi125.stderr:listed 8 entries 2023-10-18T12:00:47.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7ff35806b110 msgr2=0x7ff35806d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:47.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7ff35806b110 0x7ff35806d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740ceb40 msgr2=0x7ff3741524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:47.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740ceb40 0x7ff3741524a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff36800c450 tx=0x7ff36800c910 comp rx=0 tx=0).stop 2023-10-18T12:00:47.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 shutdown_connections 2023-10-18T12:00:47.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7ff35806b110 0x7ff35806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff37415b2a0 0x7ff374159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3740cf340 0x7ff374159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 --2- 172.21.15.125:0/3340487056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740ceb40 0x7ff3741524a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:47.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 >> 172.21.15.125:0/3340487056 conn(0x7ff3740d0d60 msgr2=0x7ff37406b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 shutdown_connections 2023-10-18T12:00:47.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:47.669+0000 7ff37ab97640 1 -- 172.21.15.125:0/3340487056 wait complete. 2023-10-18T12:00:47.678 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.671+0000 7fae49cb6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:00:47.683 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T12:00:47.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:47.684 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T12:00:47.710 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[155249]: fuse finished with error 0 and tester_r 0 2023-10-18T12:00:47.714 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:00:47.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:47.715 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T12:00:47.721 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.717+0000 7fae49cb6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:00:47.735 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:00:47.735 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:47.749 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:47.747+0000 7faf43c3b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:00:47.751 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:00:47.752 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:00:47.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:47.753 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T12:00:47.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:00:47.806 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T12:00:47.807 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:00:47.807 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:47.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:00:47.823 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:00:47.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:47.824 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T12:00:47.832 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:47.829+0000 7fae49cb6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:00:47.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:00:47.846 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T12:00:47.847 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:00:47.847 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:00:47.862 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:00:47.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:00:47.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:00:47.864 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T12:00:47.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:00:47.918 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T12:00:47.918 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:48.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.019+0000 7fa8a64e5640 1 -- 172.21.15.125:0/155156442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 msgr2=0x7fa8880047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.019+0000 7fa8a64e5640 1 --2- 172.21.15.125:0/155156442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 0x7fa8880047c0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c290 tx=0x7fa8880057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.019+0000 7fa8a4a82640 1 -- 172.21.15.125:0/155156442 reap_dead start 2023-10-18T12:00:48.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.019+0000 7fa8a64e5640 1 -- 172.21.15.125:0/155156442 shutdown_connections 2023-10-18T12:00:48.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.019+0000 7fa8a64e5640 1 -- 172.21.15.125:0/155156442 >> 172.21.15.125:0/155156442 conn(0x7fa8a00d0d60 msgr2=0x7fa8a00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:48.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.019+0000 7fa8a64e5640 1 -- 172.21.15.125:0/155156442 shutdown_connections 2023-10-18T12:00:48.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.019+0000 7fa8a64e5640 1 -- 172.21.15.125:0/155156442 wait complete. 2023-10-18T12:00:48.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.020+0000 7fa8a64e5640 1 Processor -- start 2023-10-18T12:00:48.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.021+0000 7fa8a64e5640 1 -- start start 2023-10-18T12:00:48.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.021+0000 7fa8a64e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 0x7fa8a015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.021+0000 7fa8a64e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8a0157560 0x7fa8a00b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.021+0000 7fa8a64e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8a00b3e20 0x7fa8a01f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.021+0000 7fa8a64e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00b42a0 con 0x7fa888003ef0 2023-10-18T12:00:48.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.021+0000 7fa8a64e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a01f5a10 con 0x7fa8a00b3e20 2023-10-18T12:00:48.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.021+0000 7fa8a64e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a01f18f0 con 0x7fa8a0157560 2023-10-18T12:00:48.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.021+0000 7fa89ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 0x7fa8a015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 0x7fa8a015be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54850/0 (socket says 172.21.15.125:54850) 2023-10-18T12:00:48.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89ffff640 1 -- 172.21.15.125:0/2215928460 learned_addr learned my addr 172.21.15.125:0/2215928460 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:48.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa8a4a82640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8a00b3e20 0x7fa8a01f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89f7fe640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8a0157560 0x7fa8a00b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89ffff640 1 -- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8a0157560 msgr2=0x7fa8a00b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89ffff640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8a0157560 0x7fa8a00b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89ffff640 1 -- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8a00b3e20 msgr2=0x7fa8a01f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89ffff640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8a00b3e20 0x7fa8a01f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89ffff640 1 -- 172.21.15.125:0/2215928460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a01f5730 con 0x7fa888003ef0 2023-10-18T12:00:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89ffff640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 0x7fa8a015be30 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fa894011ae0 tx=0x7fa894009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89cff9640 1 -- 172.21.15.125:0/2215928460 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89400fc50 con 0x7fa888003ef0 2023-10-18T12:00:48.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89cff9640 1 -- 172.21.15.125:0/2215928460 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa89400fdb0 con 0x7fa888003ef0 2023-10-18T12:00:48.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a00c8fb0 con 0x7fa888003ef0 2023-10-18T12:00:48.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa89cff9640 1 -- 172.21.15.125:0/2215928460 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894004280 con 0x7fa888003ef0 2023-10-18T12:00:48.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.022+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a00c9570 con 0x7fa888003ef0 2023-10-18T12:00:48.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.023+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a0157940 con 0x7fa888003ef0 2023-10-18T12:00:48.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.024+0000 7fa89cff9640 1 -- 172.21.15.125:0/2215928460 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fa8940035f0 con 0x7fa888003ef0 2023-10-18T12:00:48.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.024+0000 7fa89cff9640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa88c06b110 0x7fa88c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.024+0000 7fa89f7fe640 1 -- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa88c06b110 msgr2=0x7fa88c06d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:48.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.024+0000 7fa89cff9640 1 -- 172.21.15.125:0/2215928460 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7fa89408aba0 con 0x7fa888003ef0 2023-10-18T12:00:48.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.024+0000 7fa89f7fe640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa88c06b110 0x7fa88c06d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:48.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.028+0000 7fa89cff9640 1 -- 172.21.15.125:0/2215928460 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa894055300 con 0x7fa888003ef0 2023-10-18T12:00:48.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.225+0000 7fa89f7fe640 1 -- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa88c06b110 msgr2=0x7fa88c06d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:48.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.225+0000 7fa89f7fe640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa88c06b110 0x7fa88c06d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:00:48.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.236+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8a00690e0 con 0x7fa888003ef0 2023-10-18T12:00:48.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.236+0000 7fa89cff9640 1 -- 172.21.15.125:0/2215928460 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+6155 (secure 0 0 0) 0x7fa894058920 con 0x7fa888003ef0 2023-10-18T12:00:48.240 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:48.240 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":90,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:20.388305+0000","modified":"2023-10-18T12:00:22.406153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11976":{"gid":11976,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2975172452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2975172452},{"type":"v1","addr":"172.21.15.125:6834","nonce":2975172452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:48.241 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 90 2023-10-18T12:00:48.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa88c06b110 msgr2=0x7fa88c06d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:48.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa88c06b110 0x7fa88c06d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 msgr2=0x7fa8a015be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 0x7fa8a015be30 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fa894011ae0 tx=0x7fa894009f10 comp rx=0 tx=0).stop 2023-10-18T12:00:48.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 shutdown_connections 2023-10-18T12:00:48.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa88c06b110 0x7fa88c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8a00b3e20 0x7fa8a01f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8a0157560 0x7fa8a00b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 --2- 172.21.15.125:0/2215928460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa888003ef0 0x7fa8a015be30 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 >> 172.21.15.125:0/2215928460 conn(0x7fa8a00d0d60 msgr2=0x7fa8a015a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:48.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 shutdown_connections 2023-10-18T12:00:48.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.239+0000 7fa8a64e5640 1 -- 172.21.15.125:0/2215928460 wait complete. 2023-10-18T12:00:48.254 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:48.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.355+0000 7fa3f9075640 1 -- 172.21.15.125:0/2722638000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40cf340 msgr2=0x7fa3f40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.355+0000 7fa3f9075640 1 --2- 172.21.15.125:0/2722638000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40cf340 0x7fa3f40d1910 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4005fd0 tx=0x7fa3e402be20 comp rx=0 tx=0).stop 2023-10-18T12:00:48.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.355+0000 7fa3f9075640 1 -- 172.21.15.125:0/2722638000 shutdown_connections 2023-10-18T12:00:48.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.355+0000 7fa3f9075640 1 --2- 172.21.15.125:0/2722638000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40cf340 0x7fa3f40d1910 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.355+0000 7fa3f9075640 1 -- 172.21.15.125:0/2722638000 >> 172.21.15.125:0/2722638000 conn(0x7fa3f40d0d60 msgr2=0x7fa3f40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:48.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.355+0000 7fa3f9075640 1 -- 172.21.15.125:0/2722638000 shutdown_connections 2023-10-18T12:00:48.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.355+0000 7fa3f9075640 1 -- 172.21.15.125:0/2722638000 wait complete. 2023-10-18T12:00:48.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f9075640 1 Processor -- start 2023-10-18T12:00:48.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f9075640 1 -- start start 2023-10-18T12:00:48.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f9075640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40ceb40 0x7fa3f40b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f9075640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3f40cf340 0x7fa3f41f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f9075640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3f415b2a0 0x7fa3f40b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f9075640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40c8900 con 0x7fa3f40ceb40 2023-10-18T12:00:48.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f9075640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f41f2d40 con 0x7fa3f415b2a0 2023-10-18T12:00:48.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f9075640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40c7fd0 con 0x7fa3f40cf340 2023-10-18T12:00:48.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40ceb40 0x7fa3f40b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40ceb40 0x7fa3f40b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54854/0 (socket says 172.21.15.125:54854) 2023-10-18T12:00:48.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.357+0000 7fa3f2d9d640 1 -- 172.21.15.125:0/1405286799 learned_addr learned my addr 172.21.15.125:0/1405286799 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:48.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f359e640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3f415b2a0 0x7fa3f40b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3ebfff640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3f40cf340 0x7fa3f41f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f2d9d640 1 -- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3f40cf340 msgr2=0x7fa3f41f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f2d9d640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3f40cf340 0x7fa3f41f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f2d9d640 1 -- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3f415b2a0 msgr2=0x7fa3f40b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f2d9d640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3f415b2a0 0x7fa3f40b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f2d9d640 1 -- 172.21.15.125:0/1405286799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4009560 con 0x7fa3f40ceb40 2023-10-18T12:00:48.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f2d9d640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40ceb40 0x7fa3f40b5bf0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40099f0 tx=0x7fa3e402ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:48.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3eb7fe640 1 -- 172.21.15.125:0/1405286799 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4005650 con 0x7fa3f40ceb40 2023-10-18T12:00:48.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3eb7fe640 1 -- 172.21.15.125:0/1405286799 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa3e40038f0 con 0x7fa3f40ceb40 2023-10-18T12:00:48.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f40c82b0 con 0x7fa3f40ceb40 2023-10-18T12:00:48.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3eb7fe640 1 -- 172.21.15.125:0/1405286799 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4003d00 con 0x7fa3f40ceb40 2023-10-18T12:00:48.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.358+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f40b4830 con 0x7fa3f40ceb40 2023-10-18T12:00:48.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.359+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f40690e0 con 0x7fa3f40ceb40 2023-10-18T12:00:48.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.360+0000 7fa3eb7fe640 1 -- 172.21.15.125:0/1405286799 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fa3e403e450 con 0x7fa3f40ceb40 2023-10-18T12:00:48.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.360+0000 7fa3eb7fe640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa3d406b110 0x7fa3d406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.360+0000 7fa3eb7fe640 1 -- 172.21.15.125:0/1405286799 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7fa3e40ad0b0 con 0x7fa3f40ceb40 2023-10-18T12:00:48.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.360+0000 7fa3ebfff640 1 -- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa3d406b110 msgr2=0x7fa3d406d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:48.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.360+0000 7fa3ebfff640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa3d406b110 0x7fa3d406d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:48.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.364+0000 7fa3eb7fe640 1 -- 172.21.15.125:0/1405286799 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3e4077a50 con 0x7fa3f40ceb40 2023-10-18T12:00:48.384 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.382+0000 7faf43c3b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:00:48.490 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:48.487+0000 7fae49cb6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:00:48.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.541+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3f40d0b40 con 0x7fa3f40ceb40 2023-10-18T12:00:48.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.543+0000 7fa3eb7fe640 1 -- 172.21.15.125:0/1405286799 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+21087 (secure 0 0 0) 0x7fa3e4003e60 con 0x7fa3f40ceb40 2023-10-18T12:00:48.547 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:48.547 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":226,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:40.842783+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/1417331710":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6833/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6832/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3836307336":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/2081746818":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1472046677":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3403101268":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1324329019":"2023-10-19T12:00:40.842747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:48.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa3d406b110 msgr2=0x7fa3d406d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:48.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa3d406b110 0x7fa3d406d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40ceb40 msgr2=0x7fa3f40b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40ceb40 0x7fa3f40b5bf0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40099f0 tx=0x7fa3e402ddf0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 shutdown_connections 2023-10-18T12:00:48.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7fa3d406b110 0x7fa3d406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3f415b2a0 0x7fa3f40b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3f40cf340 0x7fa3f41f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 --2- 172.21.15.125:0/1405286799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3f40ceb40 0x7fa3f40b5bf0 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 >> 172.21.15.125:0/1405286799 conn(0x7fa3f40d0d60 msgr2=0x7fa3f40b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:48.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 shutdown_connections 2023-10-18T12:00:48.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.545+0000 7fa3f9075640 1 -- 172.21.15.125:0/1405286799 wait complete. 2023-10-18T12:00:48.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:00:18.207564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:00:19.369758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '221', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-18T12:00:30.176449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '224', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-18T12:00:30.730388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '225', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:00:48.561 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:00:48.650 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.648+0000 7faf43c3b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:00:48.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.664+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3134767065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 msgr2=0x7f2afc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.664+0000 7f2b01b1d640 1 --2- 172.21.15.125:0/3134767065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 0x7f2afc06d260 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005fd0 tx=0x7f2aec02be20 comp rx=0 tx=0).stop 2023-10-18T12:00:48.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.664+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3134767065 shutdown_connections 2023-10-18T12:00:48.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.664+0000 7f2b01b1d640 1 --2- 172.21.15.125:0/3134767065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 0x7f2afc06d260 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.664+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3134767065 >> 172.21.15.125:0/3134767065 conn(0x7f2afc0b0670 msgr2=0x7f2afc0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:48.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.664+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3134767065 shutdown_connections 2023-10-18T12:00:48.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.664+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3134767065 wait complete. 2023-10-18T12:00:48.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.665+0000 7f2b01b1d640 1 Processor -- start 2023-10-18T12:00:48.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.665+0000 7f2b01b1d640 1 -- start start 2023-10-18T12:00:48.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2b01b1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 0x7f2afc157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2b01b1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2afc159060 0x7f2afc0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2b01b1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2afc15ccf0 0x7f2afc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2b01b1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc1578a0 con 0x7f2afc0ce240 2023-10-18T12:00:48.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2b01b1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15c860 con 0x7f2afc159060 2023-10-18T12:00:48.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2b01b1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15bce0 con 0x7f2afc15ccf0 2023-10-18T12:00:48.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 0x7f2afc157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 0x7f2afc157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54864/0 (socket says 172.21.15.125:54864) 2023-10-18T12:00:48.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 -- 172.21.15.125:0/3911907406 learned_addr learned my addr 172.21.15.125:0/3911907406 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:48.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afaffd640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2afc159060 0x7f2afc0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afbfff640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2afc15ccf0 0x7f2afc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 -- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2afc15ccf0 msgr2=0x7f2afc06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2afc15ccf0 0x7f2afc06d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 -- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2afc159060 msgr2=0x7f2afc0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2afc159060 0x7f2afc0b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 -- 172.21.15.125:0/3911907406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec009560 con 0x7f2afc0ce240 2023-10-18T12:00:48.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.666+0000 7f2afb7fe640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 0x7f2afc157360 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c450 tx=0x7f2af000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:48.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.667+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af00052d0 con 0x7f2afc0ce240 2023-10-18T12:00:48.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.667+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2af0005430 con 0x7f2afc0ce240 2023-10-18T12:00:48.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.667+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0012890 con 0x7f2afc0ce240 2023-10-18T12:00:48.675 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.667+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc15bfc0 con 0x7f2afc0ce240 2023-10-18T12:00:48.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.667+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc15c580 con 0x7f2afc0ce240 2023-10-18T12:00:48.676 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.668+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc0b1760 con 0x7f2afc0ce240 2023-10-18T12:00:48.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.668+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 135) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f2af00035f0 con 0x7f2afc0ce240 2023-10-18T12:00:48.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.669+0000 7f2ae3fff640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2ae806b110 0x7f2ae806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.669+0000 7f2afaffd640 1 -- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2ae806b110 msgr2=0x7f2ae806d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/178515798 2023-10-18T12:00:48.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.669+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7293+0+0 (secure 0 0 0) 0x7f2af0089c00 con 0x7f2afc0ce240 2023-10-18T12:00:48.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.669+0000 7f2afaffd640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2ae806b110 0x7f2ae806d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:00:48.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.673+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2af00543a0 con 0x7f2afc0ce240 2023-10-18T12:00:48.699 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.697+0000 7faf43c3b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:00:48.765 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:48.762+0000 7fae49cb6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:00:48.799 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.797+0000 7faf43c3b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:00:48.816 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:48.813+0000 7fae49cb6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:00:48.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.819+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 136) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f2af0054770 con 0x7f2afc0ce240 2023-10-18T12:00:48.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.820+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2ae806b110 msgr2=0x7f2ae806d5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:00:48.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.820+0000 7f2ae3fff640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2ae806b110 0x7f2ae806d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.850+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2afc0690e0 con 0x7f2afc0ce240 2023-10-18T12:00:48.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.850+0000 7f2ae3fff640 1 -- 172.21.15.125:0/3911907406 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v227) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2af0071fd0 con 0x7f2afc0ce240 2023-10-18T12:00:48.854 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:00:48.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 msgr2=0x7f2afc157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 0x7f2afc157360 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c450 tx=0x7f2af000c910 comp rx=0 tx=0).stop 2023-10-18T12:00:48.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 shutdown_connections 2023-10-18T12:00:48.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:6832/178515798,v1:172.21.15.139:6833/178515798] conn(0x7f2ae806b110 0x7f2ae806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2afc159060 0x7f2afc0b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2afc15ccf0 0x7f2afc06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 --2- 172.21.15.125:0/3911907406 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2afc0ce240 0x7f2afc157360 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 >> 172.21.15.125:0/3911907406 conn(0x7f2afc0b0670 msgr2=0x7f2afc157dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:48.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 shutdown_connections 2023-10-18T12:00:48.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.852+0000 7f2b01b1d640 1 -- 172.21.15.125:0/3911907406 wait complete. 2023-10-18T12:00:48.867 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:00:48.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.884 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.885 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.886 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.887 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.882+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.893 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.890+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.893 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.890+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.893 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.890+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.894 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.890+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.894 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.890+0000 7faec8f39640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.895 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.891+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.895 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.891+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.895 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.891+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.895 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.891+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.895 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:00:48.891+0000 7faec5f33640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:00:48.926 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:00:48.922+0000 7fae49cb6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:00:48.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.970+0000 7facb295e640 1 -- 172.21.15.125:0/1953369625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac94004d00 msgr2=0x7fac94005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.970+0000 7facb295e640 1 --2- 172.21.15.125:0/1953369625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fac94004d00 0x7fac94005180 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7faca4009ed0 tx=0x7fac940057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.970+0000 7facb0efb640 1 -- 172.21.15.125:0/1953369625 reap_dead start 2023-10-18T12:00:48.974 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.970+0000 7facb295e640 1 -- 172.21.15.125:0/1953369625 shutdown_connections 2023-10-18T12:00:48.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.970+0000 7facb295e640 1 -- 172.21.15.125:0/1953369625 >> 172.21.15.125:0/1953369625 conn(0x7facac0d0d60 msgr2=0x7facac0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:48.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.970+0000 7facb295e640 1 -- 172.21.15.125:0/1953369625 shutdown_connections 2023-10-18T12:00:48.975 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.970+0000 7facb295e640 1 -- 172.21.15.125:0/1953369625 wait complete. 2023-10-18T12:00:48.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.971+0000 7facb295e640 1 Processor -- start 2023-10-18T12:00:48.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.971+0000 7facb295e640 1 -- start start 2023-10-18T12:00:48.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb295e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac94004d00 0x7facac157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb295e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac0cc5a0 0x7facac157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb295e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7facac0cca00 0x7facac1576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:48.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb295e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac0c81c0 con 0x7facac0cca00 2023-10-18T12:00:48.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb295e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac157be0 con 0x7facac0cc5a0 2023-10-18T12:00:48.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb295e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac0b16e0 con 0x7fac94004d00 2023-10-18T12:00:48.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7facac0cca00 0x7facac1576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7facac0cca00 0x7facac1576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54882/0 (socket says 172.21.15.125:54882) 2023-10-18T12:00:48.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 -- 172.21.15.125:0/661560279 learned_addr learned my addr 172.21.15.125:0/661560279 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:48.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facabfff640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac94004d00 0x7facac157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facab7fe640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac0cc5a0 0x7facac157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:48.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 -- 172.21.15.125:0/661560279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac94004d00 msgr2=0x7facac157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac94004d00 0x7facac157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 -- 172.21.15.125:0/661560279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac0cc5a0 msgr2=0x7facac157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:48.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac0cc5a0 0x7facac157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:48.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 -- 172.21.15.125:0/661560279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac0b19c0 con 0x7facac0cca00 2023-10-18T12:00:48.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb0efb640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7facac0cca00 0x7facac1576a0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7faca4009ed0 tx=0x7faca4031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:48.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7faca8ff9640 1 -- 172.21.15.125:0/661560279 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca403f070 con 0x7facac0cca00 2023-10-18T12:00:48.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7faca8ff9640 1 -- 172.21.15.125:0/661560279 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faca4031c00 con 0x7facac0cca00 2023-10-18T12:00:48.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac0b1ca0 con 0x7facac0cca00 2023-10-18T12:00:48.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7faca8ff9640 1 -- 172.21.15.125:0/661560279 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca4003970 con 0x7facac0cca00 2023-10-18T12:00:48.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.973+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac0b2260 con 0x7facac0cca00 2023-10-18T12:00:48.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.974+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac0b2480 con 0x7facac0cca00 2023-10-18T12:00:48.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.974+0000 7faca8ff9640 1 -- 172.21.15.125:0/661560279 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 136) v1 ==== 88940+0+0 (secure 0 0 0) 0x7faca4003ad0 con 0x7facac0cca00 2023-10-18T12:00:48.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.974+0000 7faca8ff9640 1 -- 172.21.15.125:0/661560279 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7faca40ac020 con 0x7facac0cca00 2023-10-18T12:00:48.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:48.980+0000 7faca8ff9640 1 -- 172.21.15.125:0/661560279 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faca4076dc0 con 0x7facac0cca00 2023-10-18T12:00:49.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.146+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7facac0690e0 con 0x7facac0cca00 2023-10-18T12:00:49.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.147+0000 7faca8ff9640 1 -- 172.21.15.125:0/661560279 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v227) v1 ==== 104+0+11 (secure 0 0 0) 0x7faca40490a0 con 0x7facac0cca00 2023-10-18T12:00:49.150 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:00:49.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7facac0cca00 msgr2=0x7facac1576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7facac0cca00 0x7facac1576a0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7faca4009ed0 tx=0x7faca4031750 comp rx=0 tx=0).stop 2023-10-18T12:00:49.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 shutdown_connections 2023-10-18T12:00:49.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facac0cc5a0 0x7facac157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7facac0cca00 0x7facac1576a0 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 --2- 172.21.15.125:0/661560279 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac94004d00 0x7facac157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 >> 172.21.15.125:0/661560279 conn(0x7facac0d0d60 msgr2=0x7facac0cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 shutdown_connections 2023-10-18T12:00:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.149+0000 7facb295e640 1 -- 172.21.15.125:0/661560279 wait complete. 2023-10-18T12:00:49.162 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:00:49.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.265+0000 7fb00a173640 1 -- 172.21.15.125:0/3483223817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb00413d350 msgr2=0x7fb0040cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.265+0000 7fb00a173640 1 --2- 172.21.15.125:0/3483223817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb00413d350 0x7fb0040cbd60 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7faff8005fd0 tx=0x7faff802bed0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.265+0000 7fb00a173640 1 -- 172.21.15.125:0/3483223817 shutdown_connections 2023-10-18T12:00:49.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.265+0000 7fb00a173640 1 --2- 172.21.15.125:0/3483223817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb00413d350 0x7fb0040cbd60 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.265+0000 7fb00a173640 1 -- 172.21.15.125:0/3483223817 >> 172.21.15.125:0/3483223817 conn(0x7fb004069930 msgr2=0x7fb004069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:49.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.265+0000 7fb00a173640 1 -- 172.21.15.125:0/3483223817 shutdown_connections 2023-10-18T12:00:49.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.265+0000 7fb00a173640 1 -- 172.21.15.125:0/3483223817 wait complete. 2023-10-18T12:00:49.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.266+0000 7fb00a173640 1 Processor -- start 2023-10-18T12:00:49.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.266+0000 7fb00a173640 1 -- start start 2023-10-18T12:00:49.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb00a173640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb004070350 0x7fb004135370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb00a173640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0040cb980 0x7fb0040adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb00a173640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0040ae0d0 0x7fb004154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb00a173640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00413a390 con 0x7fb0040ae0d0 2023-10-18T12:00:49.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb00a173640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004139380 con 0x7fb004070350 2023-10-18T12:00:49.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb00a173640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb004139f00 con 0x7fb0040cb980 2023-10-18T12:00:49.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0040ae0d0 0x7fb004154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0040ae0d0 0x7fb004154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54900/0 (socket says 172.21.15.125:54900) 2023-10-18T12:00:49.274 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 -- 172.21.15.125:0/2694839627 learned_addr learned my addr 172.21.15.125:0/2694839627 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:49.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb0037fe640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb004070350 0x7fb004135370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb002ffd640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0040cb980 0x7fb0040adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 -- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0040cb980 msgr2=0x7fb0040adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0040cb980 0x7fb0040adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.275 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 -- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb004070350 msgr2=0x7fb004135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb004070350 0x7fb004135370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 -- 172.21.15.125:0/2694839627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff8009560 con 0x7fb0040ae0d0 2023-10-18T12:00:49.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb0037fe640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb004070350 0x7fb004135370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:49.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb002ffd640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0040cb980 0x7fb0040adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:49.276 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.267+0000 7fb003fff640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0040ae0d0 0x7fb004154980 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7faffc00c7a0 tx=0x7faffc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:49.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.268+0000 7fafebfff640 1 -- 172.21.15.125:0/2694839627 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc003b70 con 0x7fb0040ae0d0 2023-10-18T12:00:49.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.268+0000 7fafebfff640 1 -- 172.21.15.125:0/2694839627 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faffc003cf0 con 0x7fb0040ae0d0 2023-10-18T12:00:49.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.268+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0041395d0 con 0x7fb0040ae0d0 2023-10-18T12:00:49.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.268+0000 7fafebfff640 1 -- 172.21.15.125:0/2694839627 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00aab0 con 0x7fb0040ae0d0 2023-10-18T12:00:49.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.268+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb004139b90 con 0x7fb0040ae0d0 2023-10-18T12:00:49.277 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.269+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0040b1940 con 0x7fb0040ae0d0 2023-10-18T12:00:49.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.269+0000 7fafebfff640 1 -- 172.21.15.125:0/2694839627 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 136) v1 ==== 88940+0+0 (secure 0 0 0) 0x7faffc00ac10 con 0x7fb0040ae0d0 2023-10-18T12:00:49.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.270+0000 7fafebfff640 1 -- 172.21.15.125:0/2694839627 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7faffc0898f0 con 0x7fb0040ae0d0 2023-10-18T12:00:49.278 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.274+0000 7fafebfff640 1 -- 172.21.15.125:0/2694839627 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faffc0545e0 con 0x7fb0040ae0d0 2023-10-18T12:00:49.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.445+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb00406b460 con 0x7fb0040ae0d0 2023-10-18T12:00:49.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.445+0000 7fafebfff640 1 -- 172.21.15.125:0/2694839627 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v227) v1 ==== 100+0+11 (secure 0 0 0) 0x7faffc057c00 con 0x7fb0040ae0d0 2023-10-18T12:00:49.448 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:00:49.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0040ae0d0 msgr2=0x7fb004154980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0040ae0d0 0x7fb004154980 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7faffc00c7a0 tx=0x7faffc00cc60 comp rx=0 tx=0).stop 2023-10-18T12:00:49.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 shutdown_connections 2023-10-18T12:00:49.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0040ae0d0 0x7fb004154980 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0040cb980 0x7fb0040adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 --2- 172.21.15.125:0/2694839627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb004070350 0x7fb004135370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 >> 172.21.15.125:0/2694839627 conn(0x7fb004069930 msgr2=0x7fb004137200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:49.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 shutdown_connections 2023-10-18T12:00:49.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.447+0000 7fb00a173640 1 -- 172.21.15.125:0/2694839627 wait complete. 2023-10-18T12:00:49.460 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-18T12:00:49.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.563+0000 7f5eaf10f640 1 -- 172.21.15.125:0/4067990581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e90004a80 msgr2=0x7f5e90004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.563+0000 7f5eaf10f640 1 --2- 172.21.15.125:0/4067990581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e90004a80 0x7f5e90004ea0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009b50 tx=0x7f5e90006900 comp rx=0 tx=0).stop 2023-10-18T12:00:49.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.563+0000 7f5ead6ac640 1 -- 172.21.15.125:0/4067990581 reap_dead start 2023-10-18T12:00:49.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.563+0000 7f5eaf10f640 1 -- 172.21.15.125:0/4067990581 shutdown_connections 2023-10-18T12:00:49.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.563+0000 7f5eaf10f640 1 -- 172.21.15.125:0/4067990581 >> 172.21.15.125:0/4067990581 conn(0x7f5ea8069930 msgr2=0x7f5ea8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:49.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.563+0000 7f5eaf10f640 1 -- 172.21.15.125:0/4067990581 shutdown_connections 2023-10-18T12:00:49.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.564+0000 7f5eaf10f640 1 -- 172.21.15.125:0/4067990581 wait complete. 2023-10-18T12:00:49.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.565+0000 7f5eaf10f640 1 Processor -- start 2023-10-18T12:00:49.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.565+0000 7f5eaf10f640 1 -- start start 2023-10-18T12:00:49.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.565+0000 7f5eaf10f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90004a80 0x7f5ea806c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5eaf10f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea81473a0 0x7f5ea8141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5eaf10f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea814c0f0 0x7f5ea81599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5eaf10f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea814bcd0 con 0x7f5ea814c0f0 2023-10-18T12:00:49.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5eaf10f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8141750 con 0x7f5e90004a80 2023-10-18T12:00:49.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5eaf10f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8150b30 con 0x7f5ea81473a0 2023-10-18T12:00:49.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea814c0f0 0x7f5ea81599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea814c0f0 0x7f5ea81599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54920/0 (socket says 172.21.15.125:54920) 2023-10-18T12:00:49.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 -- 172.21.15.125:0/711261345 learned_addr learned my addr 172.21.15.125:0/711261345 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:49.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5eaceab640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90004a80 0x7f5ea806c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ea7fff640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea81473a0 0x7f5ea8141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 -- 172.21.15.125:0/711261345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea81473a0 msgr2=0x7f5ea8141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea81473a0 0x7f5ea8141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 -- 172.21.15.125:0/711261345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90004a80 msgr2=0x7f5ea806c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90004a80 0x7f5ea806c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 -- 172.21.15.125:0/711261345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8150e10 con 0x7f5ea814c0f0 2023-10-18T12:00:49.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ead6ac640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea814c0f0 0x7f5ea81599d0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005fd0 tx=0x7f5ea000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:49.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ea57fa640 1 -- 172.21.15.125:0/711261345 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0004b20 con 0x7f5ea814c0f0 2023-10-18T12:00:49.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ea57fa640 1 -- 172.21.15.125:0/711261345 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ea0004c80 con 0x7f5ea814c0f0 2023-10-18T12:00:49.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea814cc00 con 0x7f5ea814c0f0 2023-10-18T12:00:49.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5ea57fa640 1 -- 172.21.15.125:0/711261345 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0003bd0 con 0x7f5ea814c0f0 2023-10-18T12:00:49.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.566+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea814d1c0 con 0x7f5ea814c0f0 2023-10-18T12:00:49.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.567+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea80690e0 con 0x7f5ea814c0f0 2023-10-18T12:00:49.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.568+0000 7f5ea57fa640 1 -- 172.21.15.125:0/711261345 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 136) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f5ea0003d30 con 0x7f5ea814c0f0 2023-10-18T12:00:49.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.568+0000 7f5ea57fa640 1 -- 172.21.15.125:0/711261345 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7f5ea008a440 con 0x7f5ea814c0f0 2023-10-18T12:00:49.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.572+0000 7f5ea57fa640 1 -- 172.21.15.125:0/711261345 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ea00551e0 con 0x7f5ea814c0f0 2023-10-18T12:00:49.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.737+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ea80b0150 con 0x7f5ea814c0f0 2023-10-18T12:00:49.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.738+0000 7f5ea57fa640 1 -- 172.21.15.125:0/711261345 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v227) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5ea001e020 con 0x7f5ea814c0f0 2023-10-18T12:00:49.742 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:00:49.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea814c0f0 msgr2=0x7f5ea81599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea814c0f0 0x7f5ea81599d0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005fd0 tx=0x7f5ea000f3c0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 shutdown_connections 2023-10-18T12:00:49.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea814c0f0 0x7f5ea81599d0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea81473a0 0x7f5ea8141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 --2- 172.21.15.125:0/711261345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90004a80 0x7f5ea806c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 >> 172.21.15.125:0/711261345 conn(0x7f5ea8069930 msgr2=0x7f5ea80ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:49.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 shutdown_connections 2023-10-18T12:00:49.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.740+0000 7f5eaf10f640 1 -- 172.21.15.125:0/711261345 wait complete. 2023-10-18T12:00:49.753 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-18T12:00:49.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.858+0000 7f264387c640 1 -- 172.21.15.125:0/2787392858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 msgr2=0x7f263c13db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.858+0000 7f264387c640 1 --2- 172.21.15.125:0/2787392858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 0x7f263c13db80 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f2638005fd0 tx=0x7f263802be10 comp rx=0 tx=0).stop 2023-10-18T12:00:49.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.858+0000 7f264387c640 1 -- 172.21.15.125:0/2787392858 shutdown_connections 2023-10-18T12:00:49.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.858+0000 7f264387c640 1 --2- 172.21.15.125:0/2787392858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f263c13d350 0x7f263c0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.858+0000 7f264387c640 1 --2- 172.21.15.125:0/2787392858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 0x7f263c13db80 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.858+0000 7f264387c640 1 -- 172.21.15.125:0/2787392858 >> 172.21.15.125:0/2787392858 conn(0x7f263c069930 msgr2=0x7f263c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:49.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.859+0000 7f264387c640 1 -- 172.21.15.125:0/2787392858 shutdown_connections 2023-10-18T12:00:49.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.859+0000 7f264387c640 1 -- 172.21.15.125:0/2787392858 wait complete. 2023-10-18T12:00:49.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f264387c640 1 Processor -- start 2023-10-18T12:00:49.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f264387c640 1 -- start start 2023-10-18T12:00:49.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f264387c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263c0c96b0 0x7f263c1366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f264387c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 0x7f263c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f264387c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f263c1518e0 0x7f263c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f264387c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c151d90 con 0x7f263c0cb980 2023-10-18T12:00:49.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f264387c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c1f13f0 con 0x7f263c0c96b0 2023-10-18T12:00:49.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f264387c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c1f16d0 con 0x7f263c1518e0 2023-10-18T12:00:49.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.860+0000 7f2640e17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 0x7f263c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2640e17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 0x7f263c1513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54950/0 (socket says 172.21.15.125:54950) 2023-10-18T12:00:49.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2640e17640 1 -- 172.21.15.125:0/172506428 learned_addr learned my addr 172.21.15.125:0/172506428 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:49.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2641618640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263c0c96b0 0x7f263c1366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2641e19640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f263c1518e0 0x7f263c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2640e17640 1 -- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f263c1518e0 msgr2=0x7f263c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2640e17640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f263c1518e0 0x7f263c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2640e17640 1 -- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263c0c96b0 msgr2=0x7f263c1366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:49.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2640e17640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263c0c96b0 0x7f263c1366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:49.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2640e17640 1 -- 172.21.15.125:0/172506428 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2638009560 con 0x7f263c0cb980 2023-10-18T12:00:49.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2640e17640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 0x7f263c1513a0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f262c00d950 tx=0x7f262c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:49.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2631ffb640 1 -- 172.21.15.125:0/172506428 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c009c30 con 0x7f263c0cb980 2023-10-18T12:00:49.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2631ffb640 1 -- 172.21.15.125:0/172506428 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f262c009e80 con 0x7f263c0cb980 2023-10-18T12:00:49.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c1f19b0 con 0x7f263c0cb980 2023-10-18T12:00:49.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f2631ffb640 1 -- 172.21.15.125:0/172506428 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c004280 con 0x7f263c0cb980 2023-10-18T12:00:49.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.861+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c1f1f70 con 0x7f263c0cb980 2023-10-18T12:00:49.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.862+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c1f2100 con 0x7f263c0cb980 2023-10-18T12:00:49.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.863+0000 7f2631ffb640 1 -- 172.21.15.125:0/172506428 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f262c0035f0 con 0x7f263c0cb980 2023-10-18T12:00:49.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.863+0000 7f2631ffb640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f262006ac90 0x7f262006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:49.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.863+0000 7f2631ffb640 1 -- 172.21.15.125:0/172506428 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7f262c089c70 con 0x7f263c0cb980 2023-10-18T12:00:49.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.863+0000 7f2641618640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f262006ac90 0x7f262006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:49.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.864+0000 7f2641618640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f262006ac90 0x7f262006d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2638035000 tx=0x7f2638004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:49.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:49.867+0000 7f2631ffb640 1 -- 172.21.15.125:0/172506428 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f262c054850 con 0x7f263c0cb980 2023-10-18T12:00:50.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.031+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f263c0690e0 con 0x7f263c0cb980 2023-10-18T12:00:50.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.032+0000 7f2631ffb640 1 -- 172.21.15.125:0/172506428 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v227) v1 ==== 103+0+11 (secure 0 0 0) 0x7f262c020070 con 0x7f263c0cb980 2023-10-18T12:00:50.035 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:00:50.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f262006ac90 msgr2=0x7f262006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f262006ac90 0x7f262006d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2638035000 tx=0x7f2638004730 comp rx=0 tx=0).stop 2023-10-18T12:00:50.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 msgr2=0x7f263c1513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 0x7f263c1513a0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f262c00d950 tx=0x7f262c00de10 comp rx=0 tx=0).stop 2023-10-18T12:00:50.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 shutdown_connections 2023-10-18T12:00:50.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f262006ac90 0x7f262006d150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f263c1518e0 0x7f263c137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f263c0cb980 0x7f263c1513a0 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 --2- 172.21.15.125:0/172506428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263c0c96b0 0x7f263c1366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 >> 172.21.15.125:0/172506428 conn(0x7f263c069930 msgr2=0x7f263c06be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:50.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 shutdown_connections 2023-10-18T12:00:50.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.034+0000 7f264387c640 1 -- 172.21.15.125:0/172506428 wait complete. 2023-10-18T12:00:50.047 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:50.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.146+0000 7f48912a9640 1 -- 172.21.15.125:0/3530087527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f488c0c96b0 msgr2=0x7f488c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.146+0000 7f48912a9640 1 --2- 172.21.15.125:0/3530087527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f488c0c96b0 0x7f488c0c9b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4880005fd0 tx=0x7f488002be10 comp rx=0 tx=0).stop 2023-10-18T12:00:50.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.146+0000 7f48912a9640 1 -- 172.21.15.125:0/3530087527 shutdown_connections 2023-10-18T12:00:50.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.146+0000 7f48912a9640 1 --2- 172.21.15.125:0/3530087527 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f488c0c96b0 0x7f488c0c9b30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.146+0000 7f48912a9640 1 --2- 172.21.15.125:0/3530087527 >> v2:172.21.15.139:3300/0 conn(0x7f488c06af00 0x7f488c06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.146+0000 7f48912a9640 1 -- 172.21.15.125:0/3530087527 >> 172.21.15.125:0/3530087527 conn(0x7f488c069930 msgr2=0x7f488c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:50.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.146+0000 7f48912a9640 1 -- 172.21.15.125:0/3530087527 shutdown_connections 2023-10-18T12:00:50.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.146+0000 7f48912a9640 1 -- 172.21.15.125:0/3530087527 wait complete. 2023-10-18T12:00:50.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.147+0000 7f48912a9640 1 Processor -- start 2023-10-18T12:00:50.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.147+0000 7f48912a9640 1 -- start start 2023-10-18T12:00:50.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f48912a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f488c06af00 0x7f488c13a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f48912a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f488c0ab500 0x7f488c158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f48912a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f488c0c96b0 0x7f488c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f48912a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c13a700 con 0x7f488c0c96b0 2023-10-18T12:00:50.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f48912a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c158840 con 0x7f488c06af00 2023-10-18T12:00:50.154 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f48912a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c13d4a0 con 0x7f488c0ab500 2023-10-18T12:00:50.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f488c0c96b0 0x7f488c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f488c0c96b0 0x7f488c15b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54952/0 (socket says 172.21.15.125:54952) 2023-10-18T12:00:50.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488b7fe640 1 -- 172.21.15.125:0/2513974242 learned_addr learned my addr 172.21.15.125:0/2513974242 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:50.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488a7fc640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f488c0ab500 0x7f488c158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488affd640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f488c06af00 0x7f488c13a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488b7fe640 1 -- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f488c0ab500 msgr2=0x7f488c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488b7fe640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f488c0ab500 0x7f488c158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488b7fe640 1 -- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f488c06af00 msgr2=0x7f488c13a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.148+0000 7f488b7fe640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f488c06af00 0x7f488c13a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.149+0000 7f488b7fe640 1 -- 172.21.15.125:0/2513974242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c009490 con 0x7f488c0c96b0 2023-10-18T12:00:50.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.149+0000 7f488b7fe640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f488c0c96b0 0x7f488c15b450 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f488400c7a0 tx=0x7f488400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:50.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.149+0000 7f48737fe640 1 -- 172.21.15.125:0/2513974242 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884003b70 con 0x7f488c0c96b0 2023-10-18T12:00:50.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.149+0000 7f48737fe640 1 -- 172.21.15.125:0/2513974242 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4884003cf0 con 0x7f488c0c96b0 2023-10-18T12:00:50.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.149+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4880009560 con 0x7f488c0c96b0 2023-10-18T12:00:50.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.149+0000 7f48737fe640 1 -- 172.21.15.125:0/2513974242 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400aab0 con 0x7f488c0c96b0 2023-10-18T12:00:50.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.149+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c1f0290 con 0x7f488c0c96b0 2023-10-18T12:00:50.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.150+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c15d630 con 0x7f488c0c96b0 2023-10-18T12:00:50.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.150+0000 7f48737fe640 1 -- 172.21.15.125:0/2513974242 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f488400ac10 con 0x7f488c0c96b0 2023-10-18T12:00:50.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.151+0000 7f48737fe640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f487806acc0 0x7f487806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.151+0000 7f48737fe640 1 -- 172.21.15.125:0/2513974242 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7f4884089a20 con 0x7f488c0c96b0 2023-10-18T12:00:50.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.151+0000 7f488affd640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f487806acc0 0x7f487806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.152+0000 7f488affd640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f487806acc0 0x7f487806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f488c0c8bd0 tx=0x7f487c005f90 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:50.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.155+0000 7f48737fe640 1 -- 172.21.15.125:0/2513974242 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4884054600 con 0x7f488c0c96b0 2023-10-18T12:00:50.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.355+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f488c0690e0 con 0x7f488c0c96b0 2023-10-18T12:00:50.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.356+0000 7f48737fe640 1 -- 172.21.15.125:0/2513974242 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+6155 (secure 0 0 0) 0x7f4884057c20 con 0x7f488c0c96b0 2023-10-18T12:00:50.359 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:50.359 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":90,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:20.388305+0000","modified":"2023-10-18T12:00:22.406153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11976":{"gid":11976,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2975172452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2975172452},{"type":"v1","addr":"172.21.15.125:6834","nonce":2975172452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:50.361 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 90 2023-10-18T12:00:50.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f487806acc0 msgr2=0x7f487806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f487806acc0 0x7f487806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f488c0c8bd0 tx=0x7f487c005f90 comp rx=0 tx=0).stop 2023-10-18T12:00:50.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f488c0c96b0 msgr2=0x7f488c15b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f488c0c96b0 0x7f488c15b450 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f488400c7a0 tx=0x7f488400cc60 comp rx=0 tx=0).stop 2023-10-18T12:00:50.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 shutdown_connections 2023-10-18T12:00:50.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f487806acc0 0x7f487806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f488c0c96b0 0x7f488c15b450 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f488c0ab500 0x7f488c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 --2- 172.21.15.125:0/2513974242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f488c06af00 0x7f488c13a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 >> 172.21.15.125:0/2513974242 conn(0x7f488c069930 msgr2=0x7f488c0adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:50.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 shutdown_connections 2023-10-18T12:00:50.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.358+0000 7f48912a9640 1 -- 172.21.15.125:0/2513974242 wait complete. 2023-10-18T12:00:50.371 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:50.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 -- 172.21.15.125:0/3787520924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 msgr2=0x7f39700c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3787520924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 0x7f39700c9750 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f39700cbd30 tx=0x7f396802c1d0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 -- 172.21.15.125:0/3787520924 shutdown_connections 2023-10-18T12:00:50.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3787520924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 0x7f39700c9750 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3787520924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3970159bd0 0x7f397015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3787520924 >> v2:172.21.15.139:3300/0 conn(0x7f397006b030 0x7f397006b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 -- 172.21.15.125:0/3787520924 >> 172.21.15.125:0/3787520924 conn(0x7f3970069930 msgr2=0x7f3970069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:50.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 -- 172.21.15.125:0/3787520924 shutdown_connections 2023-10-18T12:00:50.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.471+0000 7f3976bb6640 1 -- 172.21.15.125:0/3787520924 wait complete. 2023-10-18T12:00:50.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.472+0000 7f3976bb6640 1 Processor -- start 2023-10-18T12:00:50.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.472+0000 7f3976bb6640 1 -- start start 2023-10-18T12:00:50.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3976bb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397006b030 0x7f397015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3976bb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3970159bd0 0x7f397014f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3976bb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 0x7f397014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3976bb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39700c1990 con 0x7f397015ccb0 2023-10-18T12:00:50.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3976bb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3970158bc0 con 0x7f3970159bd0 2023-10-18T12:00:50.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3976bb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39700c1c50 con 0x7f397006b030 2023-10-18T12:00:50.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 0x7f397014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 0x7f397014fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54980/0 (socket says 172.21.15.125:54980) 2023-10-18T12:00:50.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 -- 172.21.15.125:0/3935551541 learned_addr learned my addr 172.21.15.125:0/3935551541 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:50.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3974952640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397006b030 0x7f397015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3967fff640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3970159bd0 0x7f397014f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 -- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397006b030 msgr2=0x7f397015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397006b030 0x7f397015ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 -- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3970159bd0 msgr2=0x7f397014f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3970159bd0 0x7f397014f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 -- 172.21.15.125:0/3935551541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968009900 con 0x7f397015ccb0 2023-10-18T12:00:50.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3975153640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 0x7f397014fbf0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f3968009d60 tx=0x7f3968035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:50.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f39657fa640 1 -- 172.21.15.125:0/3935551541 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968034440 con 0x7f397015ccb0 2023-10-18T12:00:50.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f39657fa640 1 -- 172.21.15.125:0/3935551541 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3968004450 con 0x7f397015ccb0 2023-10-18T12:00:50.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39700c1f30 con 0x7f397015ccb0 2023-10-18T12:00:50.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f39657fa640 1 -- 172.21.15.125:0/3935551541 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968004860 con 0x7f397015ccb0 2023-10-18T12:00:50.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.473+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39701587e0 con 0x7f397015ccb0 2023-10-18T12:00:50.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.474+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39700c2450 con 0x7f397015ccb0 2023-10-18T12:00:50.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.475+0000 7f39657fa640 1 -- 172.21.15.125:0/3935551541 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f396800a040 con 0x7f397015ccb0 2023-10-18T12:00:50.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.475+0000 7f39657fa640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f395c06ac90 0x7f395c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.475+0000 7f39657fa640 1 -- 172.21.15.125:0/3935551541 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7f39680acb80 con 0x7f397015ccb0 2023-10-18T12:00:50.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.476+0000 7f3974952640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f395c06ac90 0x7f395c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.476+0000 7f3974952640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f395c06ac90 0x7f395c06d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f396c009a60 tx=0x7f396c005ed0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:50.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.479+0000 7f39657fa640 1 -- 172.21.15.125:0/3935551541 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39700c2450 con 0x7f397015ccb0 2023-10-18T12:00:50.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.644+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39700c2450 con 0x7f397015ccb0 2023-10-18T12:00:50.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.646+0000 7f39657fa640 1 -- 172.21.15.125:0/3935551541 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v227) v1 ==== 74+0+21780 (secure 0 0 0) 0x7f39700c2450 con 0x7f397015ccb0 2023-10-18T12:00:50.649 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:50.649 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":227,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:48.811749+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/1563782559":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/170280857":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6833/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6832/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/2119410248":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3691026234":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/557716760":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/395739661":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/225240329":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1417331710":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6833/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6832/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3774276243":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3836307336":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/2081746818":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3501758629":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1472046677":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3403101268":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1324329019":"2023-10-19T12:00:40.842747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:50.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f395c06ac90 msgr2=0x7f395c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f395c06ac90 0x7f395c06d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f396c009a60 tx=0x7f396c005ed0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 msgr2=0x7f397014fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 0x7f397014fbf0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f3968009d60 tx=0x7f3968035e30 comp rx=0 tx=0).stop 2023-10-18T12:00:50.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 shutdown_connections 2023-10-18T12:00:50.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f395c06ac90 0x7f395c06d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3970159bd0 0x7f397014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f397015ccb0 0x7f397014fbf0 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 --2- 172.21.15.125:0/3935551541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f397006b030 0x7f397015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 >> 172.21.15.125:0/3935551541 conn(0x7f3970069930 msgr2=0x7f397013b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:50.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 shutdown_connections 2023-10-18T12:00:50.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.648+0000 7f3976bb6640 1 -- 172.21.15.125:0/3935551541 wait complete. 2023-10-18T12:00:50.661 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T12:00:50.662 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.761+0000 7fbe33fd0640 1 -- 172.21.15.125:0/2818488197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c140660 msgr2=0x7fbe2c0c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.761+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/2818488197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c140660 0x7fbe2c0c97c0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c005fd0 tx=0x7fbe1c02be10 comp rx=0 tx=0).stop 2023-10-18T12:00:50.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.761+0000 7fbe33fd0640 1 -- 172.21.15.125:0/2818488197 shutdown_connections 2023-10-18T12:00:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.761+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/2818488197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe2c0c9d00 0x7fbe2c06b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.761+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/2818488197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c140660 0x7fbe2c0c97c0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.761+0000 7fbe33fd0640 1 -- 172.21.15.125:0/2818488197 >> 172.21.15.125:0/2818488197 conn(0x7fbe2c069930 msgr2=0x7fbe2c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.761+0000 7fbe33fd0640 1 -- 172.21.15.125:0/2818488197 shutdown_connections 2023-10-18T12:00:50.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.761+0000 7fbe33fd0640 1 -- 172.21.15.125:0/2818488197 wait complete. 2023-10-18T12:00:50.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.762+0000 7fbe33fd0640 1 Processor -- start 2023-10-18T12:00:50.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.762+0000 7fbe33fd0640 1 -- start start 2023-10-18T12:00:50.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe33fd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c0c9d00 0x7fbe2c135fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe33fd0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe2c13d360 0x7fbe2c155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe33fd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe2c140660 0x7fbe2c13eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe33fd0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c136500 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe33fd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c14bfb0 con 0x7fbe2c140660 2023-10-18T12:00:50.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe33fd0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c14c290 con 0x7fbe2c13d360 2023-10-18T12:00:50.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c0c9d00 0x7fbe2c135fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c0c9d00 0x7fbe2c135fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55004/0 (socket says 172.21.15.125:55004) 2023-10-18T12:00:50.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 -- 172.21.15.125:0/4097068159 learned_addr learned my addr 172.21.15.125:0/4097068159 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:50.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe3156b640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe2c13d360 0x7fbe2c155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe3256d640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe2c140660 0x7fbe2c13eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 -- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe2c13d360 msgr2=0x7fbe2c155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe2c13d360 0x7fbe2c155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 -- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe2c140660 msgr2=0x7fbe2c13eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe2c140660 0x7fbe2c13eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 -- 172.21.15.125:0/4097068159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c009560 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe3156b640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe2c13d360 0x7fbe2c155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:50.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe31d6c640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c0c9d00 0x7fbe2c135fc0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c870 tx=0x7fbe2800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:50.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe227fc640 1 -- 172.21.15.125:0/4097068159 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003b70 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe227fc640 1 -- 172.21.15.125:0/4097068159 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe28003cf0 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c14c570 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe227fc640 1 -- 172.21.15.125:0/4097068159 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003ea0 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.763+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c14cb30 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.764+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2c0690e0 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.765+0000 7fbe227fc640 1 -- 172.21.15.125:0/4097068159 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbe2800aab0 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.765+0000 7fbe227fc640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbe1006acc0 0x7fbe1006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:50.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.765+0000 7fbe227fc640 1 -- 172.21.15.125:0/4097068159 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7fbe28089a30 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.766+0000 7fbe3156b640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbe1006acc0 0x7fbe1006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:50.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.766+0000 7fbe3156b640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbe1006acc0 0x7fbe1006d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c035000 tx=0x7fbe1c004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:50.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.769+0000 7fbe227fc640 1 -- 172.21.15.125:0/4097068159 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe2808be60 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.977 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.974+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe2c0cc5c0 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.975+0000 7fbe227fc640 1 -- 172.21.15.125:0/4097068159 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+6155 (secure 0 0 0) 0x7fbe28054610 con 0x7fbe2c0c9d00 2023-10-18T12:00:50.978 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:50.978 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":90,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:20.388305+0000","modified":"2023-10-18T12:00:22.406153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11976":{"gid":11976,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2975172452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2975172452},{"type":"v1","addr":"172.21.15.125:6834","nonce":2975172452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:50.979 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 90 2023-10-18T12:00:50.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbe1006acc0 msgr2=0x7fbe1006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbe1006acc0 0x7fbe1006d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c035000 tx=0x7fbe1c004730 comp rx=0 tx=0).stop 2023-10-18T12:00:50.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c0c9d00 msgr2=0x7fbe2c135fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:50.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c0c9d00 0x7fbe2c135fc0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c870 tx=0x7fbe2800cd30 comp rx=0 tx=0).stop 2023-10-18T12:00:50.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 shutdown_connections 2023-10-18T12:00:50.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbe1006acc0 0x7fbe1006d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe2c140660 0x7fbe2c13eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe2c13d360 0x7fbe2c155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.981 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 --2- 172.21.15.125:0/4097068159 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe2c0c9d00 0x7fbe2c135fc0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:50.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 >> 172.21.15.125:0/4097068159 conn(0x7fbe2c069930 msgr2=0x7fbe2c157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:50.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 shutdown_connections 2023-10-18T12:00:50.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:50.977+0000 7fbe33fd0640 1 -- 172.21.15.125:0/4097068159 wait complete. 2023-10-18T12:00:50.990 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:51.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.090+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/1360287929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 msgr2=0x7fa3c806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.090+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/1360287929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 0x7fa3c806b2e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009830 tx=0x7fa3b802bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:51.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.090+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/1360287929 shutdown_connections 2023-10-18T12:00:51.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.090+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/1360287929 >> v2:172.21.15.139:3300/0 conn(0x7fa3c80abc10 0x7fa3c80c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.090+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/1360287929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 0x7fa3c806b2e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.090+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/1360287929 >> 172.21.15.125:0/1360287929 conn(0x7fa3c8069930 msgr2=0x7fa3c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.090+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/1360287929 shutdown_connections 2023-10-18T12:00:51.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.090+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/1360287929 wait complete. 2023-10-18T12:00:51.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.091+0000 7fa3ce6a9640 1 Processor -- start 2023-10-18T12:00:51.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.091+0000 7fa3ce6a9640 1 -- start start 2023-10-18T12:00:51.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3ce6a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 0x7fa3c80ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3ce6a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3c80abc10 0x7fa3c80adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3ce6a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3c80c96b0 0x7fa3c80b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3ce6a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80b1d20 con 0x7fa3c80abc10 2023-10-18T12:00:51.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3ce6a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80b11a0 con 0x7fa3c80c96b0 2023-10-18T12:00:51.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3ce6a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80b1480 con 0x7fa3c806af00 2023-10-18T12:00:51.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3c80abc10 0x7fa3c80adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3c80abc10 0x7fa3c80adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55006/0 (socket says 172.21.15.125:55006) 2023-10-18T12:00:51.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3c77fe640 1 -- 172.21.15.125:0/674323324 learned_addr learned my addr 172.21.15.125:0/674323324 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:51.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3ccc46640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3c80c96b0 0x7fa3c80b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.092+0000 7fa3c7fff640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 0x7fa3c80ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c77fe640 1 -- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 msgr2=0x7fa3c80ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c77fe640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 0x7fa3c80ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c77fe640 1 -- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3c80c96b0 msgr2=0x7fa3c80b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c77fe640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3c80c96b0 0x7fa3c80b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c77fe640 1 -- 172.21.15.125:0/674323324 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c0009840 con 0x7fa3c80abc10 2023-10-18T12:00:51.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c7fff640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 0x7fa3c80ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:51.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c77fe640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3c80abc10 0x7fa3c80adc00 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c3c0 tx=0x7fa3bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:51.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c4ff9640 1 -- 172.21.15.125:0/674323324 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc004230 con 0x7fa3c80abc10 2023-10-18T12:00:51.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b8009490 con 0x7fa3c80abc10 2023-10-18T12:00:51.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c4ff9640 1 -- 172.21.15.125:0/674323324 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa3bc0044a0 con 0x7fa3c80abc10 2023-10-18T12:00:51.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3c4ff9640 1 -- 172.21.15.125:0/674323324 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc003850 con 0x7fa3c80abc10 2023-10-18T12:00:51.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.093+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c80b1a40 con 0x7fa3c80abc10 2023-10-18T12:00:51.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.094+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c80690e0 con 0x7fa3c80abc10 2023-10-18T12:00:51.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.094+0000 7fa3c4ff9640 1 -- 172.21.15.125:0/674323324 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa3bc0039b0 con 0x7fa3c80abc10 2023-10-18T12:00:51.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.095+0000 7fa3c4ff9640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa3b406ac90 0x7fa3b406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.095+0000 7fa3c4ff9640 1 -- 172.21.15.125:0/674323324 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7fa3bc089260 con 0x7fa3c80abc10 2023-10-18T12:00:51.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.095+0000 7fa3c7fff640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa3b406ac90 0x7fa3b406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.095+0000 7fa3c7fff640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa3b406ac90 0x7fa3b406d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009830 tx=0x7fa3b8004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:51.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.099+0000 7fa3c4ff9640 1 -- 172.21.15.125:0/674323324 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3bc053dc0 con 0x7fa3c80abc10 2023-10-18T12:00:51.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.298+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3c80cc420 con 0x7fa3c80abc10 2023-10-18T12:00:51.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.299+0000 7fa3c4ff9640 1 -- 172.21.15.125:0/674323324 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+6155 (secure 0 0 0) 0x7fa3bc017020 con 0x7fa3c80abc10 2023-10-18T12:00:51.302 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:51.302 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":90,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:20.388305+0000","modified":"2023-10-18T12:00:22.406153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11976},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11976":{"gid":11976,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6834/2975172452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":2975172452},{"type":"v1","addr":"172.21.15.125:6834","nonce":2975172452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:51.303 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 90 2023-10-18T12:00:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa3b406ac90 msgr2=0x7fa3b406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa3b406ac90 0x7fa3b406d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009830 tx=0x7fa3b8004730 comp rx=0 tx=0).stop 2023-10-18T12:00:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3c80abc10 msgr2=0x7fa3c80adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3c80abc10 0x7fa3c80adc00 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c3c0 tx=0x7fa3bc00c880 comp rx=0 tx=0).stop 2023-10-18T12:00:51.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 shutdown_connections 2023-10-18T12:00:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa3b406ac90 0x7fa3b406d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3c80c96b0 0x7fa3c80b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3c80abc10 0x7fa3c80adc00 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 --2- 172.21.15.125:0/674323324 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3c806af00 0x7fa3c80ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 >> 172.21.15.125:0/674323324 conn(0x7fa3c8069930 msgr2=0x7fa3c8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:51.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 shutdown_connections 2023-10-18T12:00:51.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.301+0000 7fa3ce6a9640 1 -- 172.21.15.125:0/674323324 wait complete. 2023-10-18T12:00:51.314 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:51.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 -- 172.21.15.125:0/1911166944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 msgr2=0x7f4ee40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 --2- 172.21.15.125:0/1911166944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 0x7f4ee40b0650 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f4ee406c510 tx=0x7f4ee002bd10 comp rx=0 tx=0).stop 2023-10-18T12:00:51.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 -- 172.21.15.125:0/1911166944 shutdown_connections 2023-10-18T12:00:51.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 --2- 172.21.15.125:0/1911166944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 0x7f4ee40b0650 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 --2- 172.21.15.125:0/1911166944 >> v2:172.21.15.139:3300/0 conn(0x7f4ee415b2a0 0x7f4ee40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 --2- 172.21.15.125:0/1911166944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ee40cf340 0x7f4ee40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 -- 172.21.15.125:0/1911166944 >> 172.21.15.125:0/1911166944 conn(0x7f4ee40d0d60 msgr2=0x7f4ee40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 -- 172.21.15.125:0/1911166944 shutdown_connections 2023-10-18T12:00:51.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.414+0000 7f4eebd39640 1 -- 172.21.15.125:0/1911166944 wait complete. 2023-10-18T12:00:51.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.415+0000 7f4eebd39640 1 Processor -- start 2023-10-18T12:00:51.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.415+0000 7f4eebd39640 1 -- start start 2023-10-18T12:00:51.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4eebd39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 0x7f4ee409e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4eebd39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ee40cf340 0x7f4ee415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.423 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4eebd39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ee415b2a0 0x7f4ee41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4eebd39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40b39f0 con 0x7f4ee40ceb40 2023-10-18T12:00:51.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4eebd39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40b3c90 con 0x7f4ee415b2a0 2023-10-18T12:00:51.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4eebd39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40b3f70 con 0x7f4ee40cf340 2023-10-18T12:00:51.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee9ad5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 0x7f4ee409e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee9ad5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 0x7f4ee409e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55014/0 (socket says 172.21.15.125:55014) 2023-10-18T12:00:51.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee9ad5640 1 -- 172.21.15.125:0/478461643 learned_addr learned my addr 172.21.15.125:0/478461643 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:51.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4eea2d6640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ee415b2a0 0x7f4ee41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee92d4640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ee40cf340 0x7f4ee415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee9ad5640 1 -- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ee40cf340 msgr2=0x7f4ee415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee9ad5640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ee40cf340 0x7f4ee415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee9ad5640 1 -- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ee415b2a0 msgr2=0x7f4ee41590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee9ad5640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ee415b2a0 0x7f4ee41590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee9ad5640 1 -- 172.21.15.125:0/478461643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0009560 con 0x7f4ee40ceb40 2023-10-18T12:00:51.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.416+0000 7f4ee92d4640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ee40cf340 0x7f4ee415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:51.429 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.417+0000 7f4ee9ad5640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 0x7f4ee409e170 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00eaa0 tx=0x7f4edc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:51.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.417+0000 7f4eda7fc640 1 -- 172.21.15.125:0/478461643 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00cdb0 con 0x7f4ee40ceb40 2023-10-18T12:00:51.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.417+0000 7f4eda7fc640 1 -- 172.21.15.125:0/478461643 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4edc004b20 con 0x7f4ee40ceb40 2023-10-18T12:00:51.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.417+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee40b4250 con 0x7f4ee40ceb40 2023-10-18T12:00:51.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.417+0000 7f4eda7fc640 1 -- 172.21.15.125:0/478461643 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc005020 con 0x7f4ee40ceb40 2023-10-18T12:00:51.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.417+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee40b4a50 con 0x7f4ee40ceb40 2023-10-18T12:00:51.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.418+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee40690e0 con 0x7f4ee40ceb40 2023-10-18T12:00:51.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.418+0000 7f4eda7fc640 1 -- 172.21.15.125:0/478461643 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4edc005180 con 0x7f4ee40ceb40 2023-10-18T12:00:51.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.418+0000 7f4eda7fc640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4ec806ac90 0x7f4ec806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.419+0000 7f4eda7fc640 1 -- 172.21.15.125:0/478461643 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7f4edc08a440 con 0x7f4ee40ceb40 2023-10-18T12:00:51.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.419+0000 7f4ee92d4640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4ec806ac90 0x7f4ec806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.419+0000 7f4ee92d4640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4ec806ac90 0x7f4ec806d150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40b2420 tx=0x7f4ed4005db0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:51.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.423+0000 7f4eda7fc640 1 -- 172.21.15.125:0/478461643 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4edc055020 con 0x7f4ee40ceb40 2023-10-18T12:00:51.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.591+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ee40d0b40 con 0x7f4ee40ceb40 2023-10-18T12:00:51.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.592+0000 7f4eda7fc640 1 -- 172.21.15.125:0/478461643 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v227) v1 ==== 74+0+21780 (secure 0 0 0) 0x7f4edc01f090 con 0x7f4ee40ceb40 2023-10-18T12:00:51.596 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:51.596 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":227,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:48.811749+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:00:18.207564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-18T12:00:19.369758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/1563782559":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/170280857":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6833/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6832/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/2119410248":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3691026234":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/557716760":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/395739661":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/225240329":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1417331710":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6833/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6832/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3774276243":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3836307336":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/2081746818":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3501758629":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1472046677":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3403101268":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1324329019":"2023-10-19T12:00:40.842747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:51.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4ec806ac90 msgr2=0x7f4ec806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4ec806ac90 0x7f4ec806d150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40b2420 tx=0x7f4ed4005db0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 msgr2=0x7f4ee409e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 0x7f4ee409e170 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00eaa0 tx=0x7f4edc00ef60 comp rx=0 tx=0).stop 2023-10-18T12:00:51.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 shutdown_connections 2023-10-18T12:00:51.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4ec806ac90 0x7f4ec806d150 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ee415b2a0 0x7f4ee41590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ee40cf340 0x7f4ee415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 --2- 172.21.15.125:0/478461643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ee40ceb40 0x7f4ee409e170 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 >> 172.21.15.125:0/478461643 conn(0x7f4ee40d0d60 msgr2=0x7f4ee415a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 shutdown_connections 2023-10-18T12:00:51.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.594+0000 7f4eebd39640 1 -- 172.21.15.125:0/478461643 wait complete. 2023-10-18T12:00:51.608 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T12:00:51.711 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.708+0000 7f851ad89640 1 -- 172.21.15.125:0/2466264107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140cb980 msgr2=0x7f851413db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.708+0000 7f851ad89640 1 --2- 172.21.15.125:0/2466264107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140cb980 0x7f851413db80 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f851406f4d0 tx=0x7f850802be40 comp rx=0 tx=0).stop 2023-10-18T12:00:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.708+0000 7f851ad89640 1 -- 172.21.15.125:0/2466264107 shutdown_connections 2023-10-18T12:00:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.708+0000 7f851ad89640 1 --2- 172.21.15.125:0/2466264107 >> v2:172.21.15.139:3300/0 conn(0x7f851413d350 0x7f85140cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.712 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.708+0000 7f851ad89640 1 --2- 172.21.15.125:0/2466264107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140cb980 0x7f851413db80 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.708+0000 7f851ad89640 1 -- 172.21.15.125:0/2466264107 >> 172.21.15.125:0/2466264107 conn(0x7f8514069930 msgr2=0x7f8514069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:51.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.708+0000 7f851ad89640 1 -- 172.21.15.125:0/2466264107 shutdown_connections 2023-10-18T12:00:51.713 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.708+0000 7f851ad89640 1 -- 172.21.15.125:0/2466264107 wait complete. 2023-10-18T12:00:51.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.710+0000 7f851ad89640 1 Processor -- start 2023-10-18T12:00:51.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.710+0000 7f851ad89640 1 -- start start 2023-10-18T12:00:51.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.710+0000 7f851ad89640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140c96b0 0x7f85141ee940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.710+0000 7f851ad89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85140cb980 0x7f8514136580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.710+0000 7f851ad89640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85140ac590 0x7f85140aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.710+0000 7f851ad89640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140acf80 con 0x7f85140c96b0 2023-10-18T12:00:51.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.710+0000 7f851ad89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85141eee80 con 0x7f85140cb980 2023-10-18T12:00:51.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.710+0000 7f851ad89640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8514136ac0 con 0x7f85140ac590 2023-10-18T12:00:51.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140c96b0 0x7f85141ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140c96b0 0x7f85141ee940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55040/0 (socket says 172.21.15.125:55040) 2023-10-18T12:00:51.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 -- 172.21.15.125:0/613771994 learned_addr learned my addr 172.21.15.125:0/613771994 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:51.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8513fff640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85140cb980 0x7f8514136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8519326640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85140ac590 0x7f85140aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 -- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85140ac590 msgr2=0x7f85140aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85140ac590 0x7f85140aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 -- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85140cb980 msgr2=0x7f8514136580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85140cb980 0x7f8514136580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 -- 172.21.15.125:0/613771994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8508009560 con 0x7f85140c96b0 2023-10-18T12:00:51.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f8518b25640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140c96b0 0x7f85141ee940 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f8508036040 tx=0x7f850802de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:51.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f85117fa640 1 -- 172.21.15.125:0/613771994 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508005d50 con 0x7f85140c96b0 2023-10-18T12:00:51.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f85117fa640 1 -- 172.21.15.125:0/613771994 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8508003910 con 0x7f85140c96b0 2023-10-18T12:00:51.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8514137860 con 0x7f85140c96b0 2023-10-18T12:00:51.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f85117fa640 1 -- 172.21.15.125:0/613771994 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508003d20 con 0x7f85140c96b0 2023-10-18T12:00:51.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.711+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8514137e20 con 0x7f85140c96b0 2023-10-18T12:00:51.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.712+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85140af750 con 0x7f85140c96b0 2023-10-18T12:00:51.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.713+0000 7f85117fa640 1 -- 172.21.15.125:0/613771994 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f850803e450 con 0x7f85140c96b0 2023-10-18T12:00:51.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.713+0000 7f85117fa640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f84f806ac90 0x7f84f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:51.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.713+0000 7f85117fa640 1 -- 172.21.15.125:0/613771994 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7766+0+0 (secure 0 0 0) 0x7f85080ac560 con 0x7f85140c96b0 2023-10-18T12:00:51.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.713+0000 7f8513fff640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f84f806ac90 0x7f84f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:51.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.714+0000 7f8513fff640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f84f806ac90 0x7f84f806d150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f85000058c0 tx=0x7f8500005850 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:51.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.717+0000 7f85117fa640 1 -- 172.21.15.125:0/613771994 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85140af750 con 0x7f85140c96b0 2023-10-18T12:00:51.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.832+0000 7f85117fa640 1 -- 172.21.15.125:0/613771994 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 138) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f8508077490 con 0x7f85140c96b0 2023-10-18T12:00:51.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.922+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f85140af750 con 0x7f85140c96b0 2023-10-18T12:00:51.925 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T12:00:51.923+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:00:51.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.926+0000 7f85117fa640 1 -- 172.21.15.125:0/613771994 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7f85140af750 con 0x7f85140c96b0 2023-10-18T12:00:51.928 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T12:00:51.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.928+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f84f806ac90 msgr2=0x7f84f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.928+0000 7f851ad89640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f84f806ac90 0x7f84f806d150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f85000058c0 tx=0x7f8500005850 comp rx=0 tx=0).stop 2023-10-18T12:00:51.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.928+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140c96b0 msgr2=0x7f85141ee940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:51.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.928+0000 7f851ad89640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140c96b0 0x7f85141ee940 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f8508036040 tx=0x7f850802de10 comp rx=0 tx=0).stop 2023-10-18T12:00:51.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.929+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 shutdown_connections 2023-10-18T12:00:51.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.929+0000 7f851ad89640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f84f806ac90 0x7f84f806d150 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.929+0000 7f851ad89640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85140cb980 0x7f8514136580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.929+0000 7f851ad89640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85140ac590 0x7f85140aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.929+0000 7f851ad89640 1 --2- 172.21.15.125:0/613771994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85140c96b0 0x7f85141ee940 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:51.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.929+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 >> 172.21.15.125:0/613771994 conn(0x7f8514069930 msgr2=0x7f851413be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:51.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.929+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 shutdown_connections 2023-10-18T12:00:51.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:51.929+0000 7f851ad89640 1 -- 172.21.15.125:0/613771994 wait complete. 2023-10-18T12:00:51.934 INFO:tasks.ceph.mds.a.smithi125.stderr: -3212> 2023-10-18T12:00:26.037+0000 7f718de95640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:00:51.942 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T12:00:51.959 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T12:00:52.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.044+0000 7f736a133640 1 -- 172.21.15.125:0/1587303355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736415b2a0 msgr2=0x7f73640d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:52.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.044+0000 7f736a133640 1 --2- 172.21.15.125:0/1587303355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736415b2a0 0x7f73640d1910 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f734c009830 tx=0x7f734c02bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:52.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.044+0000 7f736a133640 1 -- 172.21.15.125:0/1587303355 shutdown_connections 2023-10-18T12:00:52.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.044+0000 7f736a133640 1 --2- 172.21.15.125:0/1587303355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736415b2a0 0x7f73640d1910 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:52.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.044+0000 7f736a133640 1 -- 172.21.15.125:0/1587303355 >> 172.21.15.125:0/1587303355 conn(0x7f73640d0d60 msgr2=0x7f73640d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:52.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.044+0000 7f736a133640 1 -- 172.21.15.125:0/1587303355 shutdown_connections 2023-10-18T12:00:52.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.044+0000 7f736a133640 1 -- 172.21.15.125:0/1587303355 wait complete. 2023-10-18T12:00:52.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.045+0000 7f736a133640 1 Processor -- start 2023-10-18T12:00:52.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f736a133640 1 -- start start 2023-10-18T12:00:52.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f736a133640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73640ceb40 0x7f73640c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:52.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f736a133640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73640cf340 0x7f736414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:52.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f736a133640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736415b2a0 0x7f73641553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:52.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f736a133640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736414ccd0 con 0x7f736415b2a0 2023-10-18T12:00:52.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f736a133640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736406b450 con 0x7f73640ceb40 2023-10-18T12:00:52.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f736a133640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73640c6e90 con 0x7f73640cf340 2023-10-18T12:00:52.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f7363fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736415b2a0 0x7f73641553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:52.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f7363fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736415b2a0 0x7f73641553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55050/0 (socket says 172.21.15.125:55050) 2023-10-18T12:00:52.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.046+0000 7f7363fff640 1 -- 172.21.15.125:0/3145680352 learned_addr learned my addr 172.21.15.125:0/3145680352 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:52.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f73637fe640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73640ceb40 0x7f73640c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:52.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f735affd640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73640cf340 0x7f736414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:52.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7363fff640 1 -- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73640cf340 msgr2=0x7f736414d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:52.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7363fff640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73640cf340 0x7f736414d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:52.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7363fff640 1 -- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73640ceb40 msgr2=0x7f73640c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:52.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7363fff640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73640ceb40 0x7f73640c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:52.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7363fff640 1 -- 172.21.15.125:0/3145680352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c009490 con 0x7f736415b2a0 2023-10-18T12:00:52.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f735affd640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73640cf340 0x7f736414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:52.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f73637fe640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73640ceb40 0x7f73640c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:52.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7363fff640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736415b2a0 0x7f73641553d0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f735c00cbf0 tx=0x7f735c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:52.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7360ff9640 1 -- 172.21.15.125:0/3145680352 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00ad50 con 0x7f736415b2a0 2023-10-18T12:00:52.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7360ff9640 1 -- 172.21.15.125:0/3145680352 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f735c012420 con 0x7f736415b2a0 2023-10-18T12:00:52.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73641f5920 con 0x7f736415b2a0 2023-10-18T12:00:52.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f7360ff9640 1 -- 172.21.15.125:0/3145680352 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c012830 con 0x7f736415b2a0 2023-10-18T12:00:52.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.047+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736414c610 con 0x7f736415b2a0 2023-10-18T12:00:52.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.048+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736414ce10 con 0x7f736415b2a0 2023-10-18T12:00:52.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.048+0000 7f7360ff9640 1 -- 172.21.15.125:0/3145680352 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 138) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f735c01a3b0 con 0x7f736415b2a0 2023-10-18T12:00:52.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.049+0000 7f7360ff9640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f734806aff0 0x7f734806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:52.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.049+0000 7f7360ff9640 1 -- 172.21.15.125:0/3145680352 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 7852+0+0 (secure 0 0 0) 0x7f735c08aa00 con 0x7f736415b2a0 2023-10-18T12:00:52.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.049+0000 7f73637fe640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f734806aff0 0x7f734806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:52.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.049+0000 7f73637fe640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f734806aff0 0x7f734806d4b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7354001700 tx=0x7f7354009040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:52.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.053+0000 7f7360ff9640 1 -- 172.21.15.125:0/3145680352 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f736414ce10 con 0x7f736415b2a0 2023-10-18T12:00:52.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.259+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f736414ce10 con 0x7f736415b2a0 2023-10-18T12:00:52.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.927+0000 7f7360ff9640 1 -- 172.21.15.125:0/3145680352 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7f736414ce10 con 0x7f736415b2a0 2023-10-18T12:00:52.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f734806aff0 msgr2=0x7f734806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f734806aff0 0x7f734806d4b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7354001700 tx=0x7f7354009040 comp rx=0 tx=0).stop 2023-10-18T12:00:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736415b2a0 msgr2=0x7f73641553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736415b2a0 0x7f73641553d0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f735c00cbf0 tx=0x7f735c00a430 comp rx=0 tx=0).stop 2023-10-18T12:00:52.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 shutdown_connections 2023-10-18T12:00:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f734806aff0 0x7f734806d4b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f736415b2a0 0x7f73641553d0 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73640cf340 0x7f736414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 --2- 172.21.15.125:0/3145680352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73640ceb40 0x7f73640c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:52.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 >> 172.21.15.125:0/3145680352 conn(0x7f73640d0d60 msgr2=0x7f736406b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:52.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 shutdown_connections 2023-10-18T12:00:52.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:52.930+0000 7f736a133640 1 -- 172.21.15.125:0/3145680352 wait complete. 2023-10-18T12:00:52.944 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T12:00:53.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.092+0000 7f7518d33640 1 -- 172.21.15.125:0/2987129140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74fc0047c0 msgr2=0x7f74fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:53.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.092+0000 7f7518d33640 1 --2- 172.21.15.125:0/2987129140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74fc0047c0 0x7f74fc004c40 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f7508006fc0 tx=0x7f74fc0057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:53.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.092+0000 7f7518d33640 1 -- 172.21.15.125:0/2987129140 shutdown_connections 2023-10-18T12:00:53.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.092+0000 7f7518d33640 1 --2- 172.21.15.125:0/2987129140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f74fc0047c0 0x7f74fc004c40 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:53.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.092+0000 7f7518d33640 1 -- 172.21.15.125:0/2987129140 >> 172.21.15.125:0/2987129140 conn(0x7f75140d0d60 msgr2=0x7f75140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:53.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.092+0000 7f7518d33640 1 -- 172.21.15.125:0/2987129140 shutdown_connections 2023-10-18T12:00:53.096 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.092+0000 7f7518d33640 1 -- 172.21.15.125:0/2987129140 wait complete. 2023-10-18T12:00:53.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.093+0000 7f7518d33640 1 Processor -- start 2023-10-18T12:00:53.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.093+0000 7f7518d33640 1 -- start start 2023-10-18T12:00:53.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7518d33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74fc0047c0 0x7f7514154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:53.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7518d33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75140c7e30 0x7f7514151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:53.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7518d33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75140c9360 0x7f75140c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:53.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7518d33640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514151ad0 con 0x7f75140c7e30 2023-10-18T12:00:53.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7518d33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75140c82c0 con 0x7f75140c9360 2023-10-18T12:00:53.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7518d33640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75140c85a0 con 0x7f74fc0047c0 2023-10-18T12:00:53.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75140c7e30 0x7f7514151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:53.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75140c7e30 0x7f7514151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55066/0 (socket says 172.21.15.125:55066) 2023-10-18T12:00:53.100 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 -- 172.21.15.125:0/1624918999 learned_addr learned my addr 172.21.15.125:0/1624918999 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:53.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f751259c640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74fc0047c0 0x7f7514154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:53.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7512d9d640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75140c9360 0x7f75140c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:53.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 -- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74fc0047c0 msgr2=0x7f7514154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:53.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74fc0047c0 0x7f7514154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:53.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 -- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75140c9360 msgr2=0x7f75140c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:53.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75140c9360 0x7f75140c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:53.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 -- 172.21.15.125:0/1624918999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75140c8880 con 0x7f75140c7e30 2023-10-18T12:00:53.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7512d9d640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75140c9360 0x7f75140c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:53.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.094+0000 7f7511d9b640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75140c7e30 0x7f7514151590 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f750802c920 tx=0x7f7508006570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:53.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.095+0000 7f7502ffd640 1 -- 172.21.15.125:0/1624918999 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750803c070 con 0x7f75140c7e30 2023-10-18T12:00:53.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.095+0000 7f7502ffd640 1 -- 172.21.15.125:0/1624918999 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75080016a0 con 0x7f75140c7e30 2023-10-18T12:00:53.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.095+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75140c8b60 con 0x7f75140c7e30 2023-10-18T12:00:53.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.095+0000 7f7502ffd640 1 -- 172.21.15.125:0/1624918999 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508003870 con 0x7f75140c7e30 2023-10-18T12:00:53.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.095+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7514154dd0 con 0x7f75140c7e30 2023-10-18T12:00:53.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.095+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7514154f60 con 0x7f75140c7e30 2023-10-18T12:00:53.104 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.096+0000 7f7502ffd640 1 -- 172.21.15.125:0/1624918999 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 138) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f7508032070 con 0x7f75140c7e30 2023-10-18T12:00:53.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.096+0000 7f7502ffd640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f74f406b020 0x7f74f406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:53.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.097+0000 7f7502ffd640 1 -- 172.21.15.125:0/1624918999 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 7852+0+0 (secure 0 0 0) 0x7f75080ab000 con 0x7f75140c7e30 2023-10-18T12:00:53.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.097+0000 7f751259c640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f74f406b020 0x7f74f406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:53.105 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.097+0000 7f751259c640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f74f406b020 0x7f74f406d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7504000c00 tx=0x7f7504006c60 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:53.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.101+0000 7f7502ffd640 1 -- 172.21.15.125:0/1624918999 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7508075680 con 0x7f75140c7e30 2023-10-18T12:00:53.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.268+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75140af9d0 con 0x7f75140c7e30 2023-10-18T12:00:53.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.412+0000 7f7502ffd640 1 -- 172.21.15.125:0/1624918999 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7508075a50 con 0x7f75140c7e30 2023-10-18T12:00:53.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.929+0000 7f7502ffd640 1 -- 172.21.15.125:0/1624918999 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v229) v1 ==== 181+0+0 (secure 0 0 0) 0x7f750808cfe0 con 0x7f75140c7e30 2023-10-18T12:00:53.932 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T12:00:53.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f74f406b020 msgr2=0x7f74f406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:53.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f74f406b020 0x7f74f406d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7504000c00 tx=0x7f7504006c60 comp rx=0 tx=0).stop 2023-10-18T12:00:53.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75140c7e30 msgr2=0x7f7514151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:53.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75140c7e30 0x7f7514151590 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f750802c920 tx=0x7f7508006570 comp rx=0 tx=0).stop 2023-10-18T12:00:53.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 shutdown_connections 2023-10-18T12:00:53.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f74f406b020 0x7f74f406d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:53.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75140c9360 0x7f75140c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:53.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75140c7e30 0x7f7514151590 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:53.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 --2- 172.21.15.125:0/1624918999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74fc0047c0 0x7f7514154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:53.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.932+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 >> 172.21.15.125:0/1624918999 conn(0x7f75140d0d60 msgr2=0x7f751406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:53.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.933+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 shutdown_connections 2023-10-18T12:00:53.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:53.933+0000 7f7518d33640 1 -- 172.21.15.125:0/1624918999 wait complete. 2023-10-18T12:00:53.948 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T12:00:54.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.055+0000 7f103631d640 1 -- 172.21.15.125:0/1036275129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f103013d350 msgr2=0x7f10300cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:54.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.055+0000 7f103631d640 1 --2- 172.21.15.125:0/1036275129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f103013d350 0x7f10300cbd60 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f1024009830 tx=0x7f102402bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:54.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.055+0000 7f103631d640 1 -- 172.21.15.125:0/1036275129 shutdown_connections 2023-10-18T12:00:54.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.055+0000 7f103631d640 1 --2- 172.21.15.125:0/1036275129 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1030070350 0x7f10300707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:54.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.055+0000 7f103631d640 1 --2- 172.21.15.125:0/1036275129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f103013d350 0x7f10300cbd60 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:54.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.055+0000 7f103631d640 1 -- 172.21.15.125:0/1036275129 >> 172.21.15.125:0/1036275129 conn(0x7f1030069930 msgr2=0x7f1030069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:54.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.055+0000 7f103631d640 1 -- 172.21.15.125:0/1036275129 shutdown_connections 2023-10-18T12:00:54.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.055+0000 7f103631d640 1 -- 172.21.15.125:0/1036275129 wait complete. 2023-10-18T12:00:54.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.056+0000 7f103631d640 1 Processor -- start 2023-10-18T12:00:54.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.056+0000 7f103631d640 1 -- start start 2023-10-18T12:00:54.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f103631d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1030070350 0x7f1030135b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:54.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f103631d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300cb980 0x7f10301edc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:54.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f103631d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1030136930 0x7f1030153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:54.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f103631d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10301ee180 con 0x7f1030070350 2023-10-18T12:00:54.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f103631d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1030136ea0 con 0x7f10300cb980 2023-10-18T12:00:54.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f103631d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1030134db0 con 0x7f1030136930 2023-10-18T12:00:54.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1030070350 0x7f1030135b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:54.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1030070350 0x7f1030135b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55084/0 (socket says 172.21.15.125:55084) 2023-10-18T12:00:54.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102ffff640 1 -- 172.21.15.125:0/2999314803 learned_addr learned my addr 172.21.15.125:0/2999314803 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:54.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f10348ba640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1030136930 0x7f1030153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:54.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102ffff640 1 -- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1030136930 msgr2=0x7f1030153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:54.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102ffff640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1030136930 0x7f1030153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:54.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102f7fe640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300cb980 0x7f10301edc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:54.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102ffff640 1 -- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300cb980 msgr2=0x7f10301edc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:54.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102ffff640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300cb980 0x7f10301edc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:54.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.057+0000 7f102ffff640 1 -- 172.21.15.125:0/2999314803 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1024009490 con 0x7f1030070350 2023-10-18T12:00:54.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.058+0000 7f102ffff640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1030070350 0x7f1030135b70 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f102000c4b0 tx=0x7f102000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:54.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.058+0000 7f102cff9640 1 -- 172.21.15.125:0/2999314803 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020005480 con 0x7f1030070350 2023-10-18T12:00:54.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.058+0000 7f102cff9640 1 -- 172.21.15.125:0/2999314803 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f10200036e0 con 0x7f1030070350 2023-10-18T12:00:54.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.058+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1030135090 con 0x7f1030070350 2023-10-18T12:00:54.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.058+0000 7f102cff9640 1 -- 172.21.15.125:0/2999314803 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020003b30 con 0x7f1030070350 2023-10-18T12:00:54.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.058+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1030135650 con 0x7f1030070350 2023-10-18T12:00:54.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.058+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103006fd70 con 0x7f1030070350 2023-10-18T12:00:54.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.059+0000 7f102cff9640 1 -- 172.21.15.125:0/2999314803 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1020003c90 con 0x7f1030070350 2023-10-18T12:00:54.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.059+0000 7f102cff9640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f101c06b0b0 0x7f101c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:54.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.060+0000 7f102cff9640 1 -- 172.21.15.125:0/2999314803 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 7414+0+0 (secure 0 0 0) 0x7f1020089850 con 0x7f1030070350 2023-10-18T12:00:54.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.060+0000 7f102f7fe640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f101c06b0b0 0x7f101c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:54.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.060+0000 7f102f7fe640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f101c06b0b0 0x7f101c06d570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1024036000 tx=0x7f1024004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:54.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.064+0000 7f102cff9640 1 -- 172.21.15.125:0/2999314803 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1020053f80 con 0x7f1030070350 2023-10-18T12:00:54.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.230+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1030135840 con 0x7f1030070350 2023-10-18T12:00:54.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.931+0000 7f102cff9640 1 -- 172.21.15.125:0/2999314803 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v230) v1 ==== 169+0+0 (secure 0 0 0) 0x7f10200170d0 con 0x7f1030070350 2023-10-18T12:00:54.933 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T12:00:54.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f101c06b0b0 msgr2=0x7f101c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:54.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f101c06b0b0 0x7f101c06d570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1024036000 tx=0x7f1024004730 comp rx=0 tx=0).stop 2023-10-18T12:00:54.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1030070350 msgr2=0x7f1030135b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:54.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1030070350 0x7f1030135b70 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f102000c4b0 tx=0x7f102000c970 comp rx=0 tx=0).stop 2023-10-18T12:00:54.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 shutdown_connections 2023-10-18T12:00:54.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f101c06b0b0 0x7f101c06d570 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:54.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300cb980 0x7f10301edc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:54.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1030136930 0x7f1030153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:54.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 --2- 172.21.15.125:0/2999314803 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1030070350 0x7f1030135b70 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:54.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 >> 172.21.15.125:0/2999314803 conn(0x7f1030069930 msgr2=0x7f1030137a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:54.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 shutdown_connections 2023-10-18T12:00:54.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:54.934+0000 7f103631d640 1 -- 172.21.15.125:0/2999314803 wait complete. 2023-10-18T12:00:54.950 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:55.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.051+0000 7f3a73ea9640 1 -- 172.21.15.125:0/2289204798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 msgr2=0x7f3a6c158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.051+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/2289204798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 0x7f3a6c158300 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009830 tx=0x7f3a6802bd60 comp rx=0 tx=0).stop 2023-10-18T12:00:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.051+0000 7f3a73ea9640 1 -- 172.21.15.125:0/2289204798 shutdown_connections 2023-10-18T12:00:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.051+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/2289204798 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a6c0afa20 0x7f3a6c0705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.051+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/2289204798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 0x7f3a6c158300 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.052+0000 7f3a73ea9640 1 -- 172.21.15.125:0/2289204798 >> 172.21.15.125:0/2289204798 conn(0x7f3a6c069930 msgr2=0x7f3a6c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:55.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.052+0000 7f3a73ea9640 1 -- 172.21.15.125:0/2289204798 shutdown_connections 2023-10-18T12:00:55.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.052+0000 7f3a73ea9640 1 -- 172.21.15.125:0/2289204798 wait complete. 2023-10-18T12:00:55.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.053+0000 7f3a73ea9640 1 Processor -- start 2023-10-18T12:00:55.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.053+0000 7f3a73ea9640 1 -- start start 2023-10-18T12:00:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.053+0000 7f3a73ea9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a6c0afa20 0x7f3a6c1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.053+0000 7f3a73ea9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6c0b0530 0x7f3a6c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.053+0000 7f3a73ea9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 0x7f3a6c13de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.053+0000 7f3a73ea9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c14d280 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.053+0000 7f3a73ea9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c14c3f0 con 0x7f3a6c0b0530 2023-10-18T12:00:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.053+0000 7f3a73ea9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c14c920 con 0x7f3a6c0afa20 2023-10-18T12:00:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 0x7f3a6c13de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 0x7f3a6c13de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55102/0 (socket says 172.21.15.125:55102) 2023-10-18T12:00:55.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 -- 172.21.15.125:0/3118136189 learned_addr learned my addr 172.21.15.125:0/3118136189 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a71c45640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a6c0afa20 0x7f3a6c1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a71444640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6c0b0530 0x7f3a6c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 -- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a6c0afa20 msgr2=0x7f3a6c1f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a6c0afa20 0x7f3a6c1f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 -- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6c0b0530 msgr2=0x7f3a6c0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6c0b0530 0x7f3a6c0caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 -- 172.21.15.125:0/3118136189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68009490 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a71444640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6c0b0530 0x7f3a6c0caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:55.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a72446640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 0x7f3a6c13de20 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400cbc0 tx=0x7f3a6400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:55.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a627fc640 1 -- 172.21.15.125:0/3118136189 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6400ad40 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a627fc640 1 -- 172.21.15.125:0/3118136189 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a64012420 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a627fc640 1 -- 172.21.15.125:0/3118136189 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64012850 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c14c640 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.054+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c1f4880 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.055+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c0690e0 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.056+0000 7f3a627fc640 1 -- 172.21.15.125:0/3118136189 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3a6401a3b0 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.056+0000 7f3a627fc640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3a5006b080 0x7f3a5006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.056+0000 7f3a627fc640 1 -- 172.21.15.125:0/3118136189 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7f3a6408a720 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.056+0000 7f3a71c45640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3a5006b080 0x7f3a5006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.057+0000 7f3a71c45640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3a5006b080 0x7f3a5006d540 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a68036000 tx=0x7f3a68004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:55.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.061+0000 7f3a627fc640 1 -- 172.21.15.125:0/3118136189 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a64054f60 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.229+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a6c06b8e0 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.230+0000 7f3a627fc640 1 -- 172.21.15.125:0/3118136189 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v230) v1 ==== 74+0+18200 (secure 0 0 0) 0x7f3a64058580 con 0x7f3a6c0cbd20 2023-10-18T12:00:55.233 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:55.233 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":230,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:54.930283+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6832/2975172452":"2023-10-19T12:00:51.923831+0000","172.21.15.139:0/1563782559":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/170280857":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6833/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6832/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/2119410248":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3691026234":"2023-10-19T12:00:48.811703+0000","172.21.15.125:6834/2975172452":"2023-10-19T12:00:51.923831+0000","172.21.15.139:0/557716760":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/395739661":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/225240329":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1417331710":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6833/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6832/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3774276243":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3836307336":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/2081746818":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3501758629":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1472046677":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3403101268":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1324329019":"2023-10-19T12:00:40.842747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:55.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3a5006b080 msgr2=0x7f3a5006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3a5006b080 0x7f3a5006d540 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a68036000 tx=0x7f3a68004730 comp rx=0 tx=0).stop 2023-10-18T12:00:55.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 msgr2=0x7f3a6c13de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 0x7f3a6c13de20 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400cbc0 tx=0x7f3a6400a430 comp rx=0 tx=0).stop 2023-10-18T12:00:55.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 shutdown_connections 2023-10-18T12:00:55.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3a5006b080 0x7f3a5006d540 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a6c0b0530 0x7f3a6c0caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3a6c0cbd20 0x7f3a6c13de20 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 --2- 172.21.15.125:0/3118136189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3a6c0afa20 0x7f3a6c1f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 >> 172.21.15.125:0/3118136189 conn(0x7f3a6c069930 msgr2=0x7f3a6c0c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:55.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 shutdown_connections 2023-10-18T12:00:55.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.232+0000 7f3a73ea9640 1 -- 172.21.15.125:0/3118136189 wait complete. 2023-10-18T12:00:55.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-18T12:00:30.176449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '224', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-18T12:00:30.730388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '225', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:00:55.246 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:00:55.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.356+0000 7f07fcdb6640 1 -- 172.21.15.125:0/850372062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 msgr2=0x7f07f80c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.356+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/850372062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 0x7f07f80c9b60 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f07f0005fd0 tx=0x7f07f002c250 comp rx=0 tx=0).stop 2023-10-18T12:00:55.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.356+0000 7f07fcdb6640 1 -- 172.21.15.125:0/850372062 shutdown_connections 2023-10-18T12:00:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.356+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/850372062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 0x7f07f80c9b60 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.356+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/850372062 >> v2:172.21.15.139:3300/0 conn(0x7f07f806af00 0x7f07f806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.356+0000 7f07fcdb6640 1 -- 172.21.15.125:0/850372062 >> 172.21.15.125:0/850372062 conn(0x7f07f8069930 msgr2=0x7f07f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:55.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.356+0000 7f07fcdb6640 1 -- 172.21.15.125:0/850372062 shutdown_connections 2023-10-18T12:00:55.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.356+0000 7f07fcdb6640 1 -- 172.21.15.125:0/850372062 wait complete. 2023-10-18T12:00:55.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07fcdb6640 1 Processor -- start 2023-10-18T12:00:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07fcdb6640 1 -- start start 2023-10-18T12:00:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07fcdb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f806af00 0x7f07f812db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07fcdb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 0x7f07f812b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07fcdb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80c96b0 0x7f07f81eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07fcdb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f812c170 con 0x7f07f80abc10 2023-10-18T12:00:55.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07fcdb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f812c450 con 0x7f07f806af00 2023-10-18T12:00:55.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07fcdb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f812c730 con 0x7f07f80c96b0 2023-10-18T12:00:55.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.358+0000 7f07edd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 0x7f07f812b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07edd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 0x7f07f812b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55126/0 (socket says 172.21.15.125:55126) 2023-10-18T12:00:55.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07edd9b640 1 -- 172.21.15.125:0/723941541 learned_addr learned my addr 172.21.15.125:0/723941541 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:55.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07edd9b640 1 -- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80c96b0 msgr2=0x7f07f81eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07edd9b640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80c96b0 0x7f07f81eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07edd9b640 1 -- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f806af00 msgr2=0x7f07f812db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07edd9b640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f806af00 0x7f07f812db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07edd9b640 1 -- 172.21.15.125:0/723941541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009900 con 0x7f07f80abc10 2023-10-18T12:00:55.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07edd9b640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 0x7f07f812b8f0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c400 tx=0x7f07e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:55.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07ef7fe640 1 -- 172.21.15.125:0/723941541 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00052d0 con 0x7f07f80abc10 2023-10-18T12:00:55.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07ef7fe640 1 -- 172.21.15.125:0/723941541 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07e00035f0 con 0x7f07f80abc10 2023-10-18T12:00:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07ef7fe640 1 -- 172.21.15.125:0/723941541 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0003a00 con 0x7f07f80abc10 2023-10-18T12:00:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f812ca10 con 0x7f07f80abc10 2023-10-18T12:00:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.359+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f812be30 con 0x7f07f80abc10 2023-10-18T12:00:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.360+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f812e400 con 0x7f07f80abc10 2023-10-18T12:00:55.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.361+0000 7f07ef7fe640 1 -- 172.21.15.125:0/723941541 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f07e0003b60 con 0x7f07f80abc10 2023-10-18T12:00:55.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.361+0000 7f07ef7fe640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f07d806b080 0x7f07d806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.361+0000 7f07ef7fe640 1 -- 172.21.15.125:0/723941541 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7f07e00895e0 con 0x7f07f80abc10 2023-10-18T12:00:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.361+0000 7f07f659c640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f07d806b080 0x7f07d806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.362+0000 7f07f659c640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f07d806b080 0x7f07d806d540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f07e8005fd0 tx=0x7f07e8005c70 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.365+0000 7f07ef7fe640 1 -- 172.21.15.125:0/723941541 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07f812e400 con 0x7f07f80abc10 2023-10-18T12:00:55.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.534+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07f812e400 con 0x7f07f80abc10 2023-10-18T12:00:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.534+0000 7f07ef7fe640 1 -- 172.21.15.125:0/723941541 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v230) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07f812e400 con 0x7f07f80abc10 2023-10-18T12:00:55.537 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:00:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f07d806b080 msgr2=0x7f07d806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f07d806b080 0x7f07d806d540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f07e8005fd0 tx=0x7f07e8005c70 comp rx=0 tx=0).stop 2023-10-18T12:00:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 msgr2=0x7f07f812b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 0x7f07f812b8f0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c400 tx=0x7f07e000c8c0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 shutdown_connections 2023-10-18T12:00:55.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f07d806b080 0x7f07d806d540 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07f80c96b0 0x7f07f81eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07f80abc10 0x7f07f812b8f0 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 --2- 172.21.15.125:0/723941541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f806af00 0x7f07f812db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 >> 172.21.15.125:0/723941541 conn(0x7f07f8069930 msgr2=0x7f07f8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:55.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 shutdown_connections 2023-10-18T12:00:55.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.536+0000 7f07fcdb6640 1 -- 172.21.15.125:0/723941541 wait complete. 2023-10-18T12:00:55.549 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-18T12:00:55.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.652+0000 7effb0c44640 1 -- 172.21.15.125:0/3408653631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 msgr2=0x7eff98003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.652+0000 7effb0c44640 1 --2- 172.21.15.125:0/3408653631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 0x7eff98003dc0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00bb20 tx=0x7eff980057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.652+0000 7effaad9d640 1 -- 172.21.15.125:0/3408653631 reap_dead start 2023-10-18T12:00:55.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.652+0000 7effb0c44640 1 -- 172.21.15.125:0/3408653631 shutdown_connections 2023-10-18T12:00:55.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.652+0000 7effb0c44640 1 -- 172.21.15.125:0/3408653631 >> 172.21.15.125:0/3408653631 conn(0x7effac0af7a0 msgr2=0x7effac0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:55.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.652+0000 7effb0c44640 1 -- 172.21.15.125:0/3408653631 shutdown_connections 2023-10-18T12:00:55.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.652+0000 7effb0c44640 1 -- 172.21.15.125:0/3408653631 wait complete. 2023-10-18T12:00:55.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.653+0000 7effb0c44640 1 Processor -- start 2023-10-18T12:00:55.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.653+0000 7effb0c44640 1 -- start start 2023-10-18T12:00:55.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effb0c44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 0x7effac1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effb0c44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac1e3870 0x7effac1e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effb0c44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac13a110 0x7effac13a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effb0c44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac15c800 con 0x7eff980039a0 2023-10-18T12:00:55.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effb0c44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac1e4210 con 0x7effac13a110 2023-10-18T12:00:55.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effb0c44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac13aad0 con 0x7effac1e3870 2023-10-18T12:00:55.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaa59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 0x7effac1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaa59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 0x7effac1578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55154/0 (socket says 172.21.15.125:55154) 2023-10-18T12:00:55.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaa59c640 1 -- 172.21.15.125:0/2272429240 learned_addr learned my addr 172.21.15.125:0/2272429240 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:55.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaad9d640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac13a110 0x7effac13a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effa1d9b640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac1e3870 0x7effac1e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaa59c640 1 -- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac1e3870 msgr2=0x7effac1e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaa59c640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac1e3870 0x7effac1e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaa59c640 1 -- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac13a110 msgr2=0x7effac13a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaa59c640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac13a110 0x7effac13a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaa59c640 1 -- 172.21.15.125:0/2272429240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac13ad60 con 0x7eff980039a0 2023-10-18T12:00:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.654+0000 7effaad9d640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac13a110 0x7effac13a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.655+0000 7effaa59c640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 0x7effac1578d0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00ba40 tx=0x7eff9c036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.655+0000 7effa37fe640 1 -- 172.21.15.125:0/2272429240 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c036d20 con 0x7eff980039a0 2023-10-18T12:00:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.655+0000 7effa37fe640 1 -- 172.21.15.125:0/2272429240 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff9c004020 con 0x7eff980039a0 2023-10-18T12:00:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.655+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effac13b040 con 0x7eff980039a0 2023-10-18T12:00:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.655+0000 7effa37fe640 1 -- 172.21.15.125:0/2272429240 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c004430 con 0x7eff980039a0 2023-10-18T12:00:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.655+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac1381b0 con 0x7eff980039a0 2023-10-18T12:00:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.656+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effac137d30 con 0x7eff980039a0 2023-10-18T12:00:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.656+0000 7effa37fe640 1 -- 172.21.15.125:0/2272429240 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7eff9c0035f0 con 0x7eff980039a0 2023-10-18T12:00:55.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.656+0000 7effa37fe640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7eff8c06b0b0 0x7eff8c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.657+0000 7effa37fe640 1 -- 172.21.15.125:0/2272429240 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7eff9c0abdd0 con 0x7eff980039a0 2023-10-18T12:00:55.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.657+0000 7effa1d9b640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7eff8c06b0b0 0x7eff8c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.657+0000 7effa1d9b640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7eff8c06b0b0 0x7eff8c06d570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7eff940017a0 tx=0x7eff94001e10 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:55.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.661+0000 7effa37fe640 1 -- 172.21.15.125:0/2272429240 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effac137d30 con 0x7eff980039a0 2023-10-18T12:00:55.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.825+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effac137d30 con 0x7eff980039a0 2023-10-18T12:00:55.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.825+0000 7effa37fe640 1 -- 172.21.15.125:0/2272429240 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v230) v1 ==== 107+0+11 (secure 0 0 0) 0x7effac137d30 con 0x7eff980039a0 2023-10-18T12:00:55.828 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:00:55.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7eff8c06b0b0 msgr2=0x7eff8c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7eff8c06b0b0 0x7eff8c06d570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7eff940017a0 tx=0x7eff94001e10 comp rx=0 tx=0).stop 2023-10-18T12:00:55.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 msgr2=0x7effac1578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 0x7effac1578d0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00ba40 tx=0x7eff9c036720 comp rx=0 tx=0).stop 2023-10-18T12:00:55.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 shutdown_connections 2023-10-18T12:00:55.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7eff8c06b0b0 0x7eff8c06d570 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effac13a110 0x7effac13a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effac1e3870 0x7effac1e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 --2- 172.21.15.125:0/2272429240 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7eff980039a0 0x7effac1578d0 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 >> 172.21.15.125:0/2272429240 conn(0x7effac0af7a0 msgr2=0x7effac0aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:55.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 shutdown_connections 2023-10-18T12:00:55.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.827+0000 7effb0c44640 1 -- 172.21.15.125:0/2272429240 wait complete. 2023-10-18T12:00:55.840 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-18T12:00:55.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.944+0000 7f1ba7733640 1 -- 172.21.15.125:0/4119147611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b8c003970 msgr2=0x7f1b8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.944+0000 7f1ba7733640 1 --2- 172.21.15.125:0/4119147611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1b8c003970 0x7f1b8c003d90 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009b80 tx=0x7f1b8c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.944+0000 7f1ba5cd0640 1 -- 172.21.15.125:0/4119147611 reap_dead start 2023-10-18T12:00:55.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.944+0000 7f1ba7733640 1 -- 172.21.15.125:0/4119147611 shutdown_connections 2023-10-18T12:00:55.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.944+0000 7f1ba7733640 1 -- 172.21.15.125:0/4119147611 >> 172.21.15.125:0/4119147611 conn(0x7f1ba00b0670 msgr2=0x7f1ba00b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:55.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.944+0000 7f1ba7733640 1 -- 172.21.15.125:0/4119147611 shutdown_connections 2023-10-18T12:00:55.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.944+0000 7f1ba7733640 1 -- 172.21.15.125:0/4119147611 wait complete. 2023-10-18T12:00:55.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.945+0000 7f1ba7733640 1 Processor -- start 2023-10-18T12:00:55.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.945+0000 7f1ba7733640 1 -- start start 2023-10-18T12:00:55.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba7733640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b8c003970 0x7f1ba01e4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba7733640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ba0150cd0 0x7f1ba0152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba7733640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ba00c3fd0 0x7f1ba00c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba7733640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00c4990 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba7733640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba014ec20 con 0x7f1b8c003970 2023-10-18T12:00:55.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba7733640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba0152750 con 0x7f1ba0150cd0 2023-10-18T12:00:55.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ba00c3fd0 0x7f1ba00c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ba00c3fd0 0x7f1ba00c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55176/0 (socket says 172.21.15.125:55176) 2023-10-18T12:00:55.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 -- 172.21.15.125:0/1002655502 learned_addr learned my addr 172.21.15.125:0/1002655502 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:55.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba54cf640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b8c003970 0x7f1ba01e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba4cce640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ba0150cd0 0x7f1ba0152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 -- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ba0150cd0 msgr2=0x7f1ba0152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ba0150cd0 0x7f1ba0152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 -- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b8c003970 msgr2=0x7f1ba01e4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:55.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b8c003970 0x7f1ba01e4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:55.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 -- 172.21.15.125:0/1002655502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba01529d0 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba4cce640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ba0150cd0 0x7f1ba0152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:55.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba54cf640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b8c003970 0x7f1ba01e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:55.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba5cd0640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ba00c3fd0 0x7f1ba00c4450 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f1ba015d0d0 tx=0x7f1b9800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:55.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1b91ffb640 1 -- 172.21.15.125:0/1002655502 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9800cba0 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1b91ffb640 1 -- 172.21.15.125:0/1002655502 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b9800cdf0 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba0152cb0 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1b91ffb640 1 -- 172.21.15.125:0/1002655502 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b980116c0 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.946+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba0153270 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.947+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba0153400 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.948+0000 7f1b91ffb640 1 -- 172.21.15.125:0/1002655502 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1b98011820 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.948+0000 7f1b91ffb640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1b8406b080 0x7f1b8406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:55.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.948+0000 7f1b91ffb640 1 -- 172.21.15.125:0/1002655502 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7f1b9808a830 con 0x7f1ba00c3fd0 2023-10-18T12:00:55.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.948+0000 7f1ba54cf640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1b8406b080 0x7f1b8406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:55.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.949+0000 7f1ba54cf640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1b8406b080 0x7f1b8406d540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c036000 tx=0x7f1b9c035040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:55.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:55.952+0000 7f1b91ffb640 1 -- 172.21.15.125:0/1002655502 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b98054ff0 con 0x7f1ba00c3fd0 2023-10-18T12:00:56.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.117+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ba00675f0 con 0x7f1ba00c3fd0 2023-10-18T12:00:56.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.117+0000 7f1b91ffb640 1 -- 172.21.15.125:0/1002655502 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v230) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1b980150a0 con 0x7f1ba00c3fd0 2023-10-18T12:00:56.120 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:00:56.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1b8406b080 msgr2=0x7f1b8406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1b8406b080 0x7f1b8406d540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c036000 tx=0x7f1b9c035040 comp rx=0 tx=0).stop 2023-10-18T12:00:56.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ba00c3fd0 msgr2=0x7f1ba00c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ba00c3fd0 0x7f1ba00c4450 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f1ba015d0d0 tx=0x7f1b9800ed10 comp rx=0 tx=0).stop 2023-10-18T12:00:56.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 shutdown_connections 2023-10-18T12:00:56.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1b8406b080 0x7f1b8406d540 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ba00c3fd0 0x7f1ba00c4450 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1ba0150cd0 0x7f1ba0152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 --2- 172.21.15.125:0/1002655502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b8c003970 0x7f1ba01e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 >> 172.21.15.125:0/1002655502 conn(0x7f1ba00b0670 msgr2=0x7f1ba00ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:56.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 shutdown_connections 2023-10-18T12:00:56.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.119+0000 7f1ba7733640 1 -- 172.21.15.125:0/1002655502 wait complete. 2023-10-18T12:00:56.132 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:56.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.231+0000 7f78c2c75640 1 -- 172.21.15.125:0/2357826329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 msgr2=0x7f78bc06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.231+0000 7f78c2c75640 1 --2- 172.21.15.125:0/2357826329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 0x7f78bc06b2e0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009900 tx=0x7f78ac02bd70 comp rx=0 tx=0).stop 2023-10-18T12:00:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.231+0000 7f78c2c75640 1 -- 172.21.15.125:0/2357826329 shutdown_connections 2023-10-18T12:00:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.231+0000 7f78c2c75640 1 --2- 172.21.15.125:0/2357826329 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78bc0abc10 0x7f78bc0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.231+0000 7f78c2c75640 1 --2- 172.21.15.125:0/2357826329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 0x7f78bc06b2e0 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.231+0000 7f78c2c75640 1 -- 172.21.15.125:0/2357826329 >> 172.21.15.125:0/2357826329 conn(0x7f78bc069930 msgr2=0x7f78bc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.231+0000 7f78c2c75640 1 -- 172.21.15.125:0/2357826329 shutdown_connections 2023-10-18T12:00:56.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.231+0000 7f78c2c75640 1 -- 172.21.15.125:0/2357826329 wait complete. 2023-10-18T12:00:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.233+0000 7f78c2c75640 1 Processor -- start 2023-10-18T12:00:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.233+0000 7f78c2c75640 1 -- start start 2023-10-18T12:00:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.233+0000 7f78c2c75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 0x7f78bc1f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.233+0000 7f78c2c75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc0abc10 0x7f78bc151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.233+0000 7f78c2c75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78bc0c96b0 0x7f78bc0700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.233+0000 7f78c2c75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc0b0580 con 0x7f78bc06af00 2023-10-18T12:00:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.233+0000 7f78c2c75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc0afc50 con 0x7f78bc0abc10 2023-10-18T12:00:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.233+0000 7f78c2c75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc0aff30 con 0x7f78bc0c96b0 2023-10-18T12:00:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 0x7f78bc1f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 0x7f78bc1f2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55196/0 (socket says 172.21.15.125:55196) 2023-10-18T12:00:56.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 -- 172.21.15.125:0/961909885 learned_addr learned my addr 172.21.15.125:0/961909885 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:56.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c1212640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78bc0c96b0 0x7f78bc0700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78bbfff640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc0abc10 0x7f78bc151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 -- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78bc0c96b0 msgr2=0x7f78bc0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78bc0c96b0 0x7f78bc0700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 -- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc0abc10 msgr2=0x7f78bc151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc0abc10 0x7f78bc151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 -- 172.21.15.125:0/961909885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac009560 con 0x7f78bc06af00 2023-10-18T12:00:56.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78bbfff640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc0abc10 0x7f78bc151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:56.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c0a11640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 0x7f78bc1f2a40 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c00 tx=0x7f78ac02db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:56.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78b97fa640 1 -- 172.21.15.125:0/961909885 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac003800 con 0x7f78bc06af00 2023-10-18T12:00:56.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78b97fa640 1 -- 172.21.15.125:0/961909885 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78ac003a50 con 0x7f78bc06af00 2023-10-18T12:00:56.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78bc0b0210 con 0x7f78bc06af00 2023-10-18T12:00:56.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78b97fa640 1 -- 172.21.15.125:0/961909885 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac003e20 con 0x7f78bc06af00 2023-10-18T12:00:56.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.234+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78bc1f2f80 con 0x7f78bc06af00 2023-10-18T12:00:56.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.235+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78bc0b03a0 con 0x7f78bc06af00 2023-10-18T12:00:56.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.236+0000 7f78b97fa640 1 -- 172.21.15.125:0/961909885 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f78ac03e450 con 0x7f78bc06af00 2023-10-18T12:00:56.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.236+0000 7f78b97fa640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78a806b080 0x7f78a806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.236+0000 7f78b97fa640 1 -- 172.21.15.125:0/961909885 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7f78ac0acab0 con 0x7f78bc06af00 2023-10-18T12:00:56.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.236+0000 7f78bbfff640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78a806b080 0x7f78a806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.237+0000 7f78bbfff640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78a806b080 0x7f78a806d540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0c8ee0 tx=0x7f78b0008040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:56.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.240+0000 7f78b97fa640 1 -- 172.21.15.125:0/961909885 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78ac077410 con 0x7f78bc06af00 2023-10-18T12:00:56.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.444+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78bc0690e0 con 0x7f78bc06af00 2023-10-18T12:00:56.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.444+0000 7f78b97fa640 1 -- 172.21.15.125:0/961909885 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f78ac07aa30 con 0x7f78bc06af00 2023-10-18T12:00:56.448 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:56.448 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":12609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1669677937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1669677937},{"type":"v1","addr":"172.21.15.125:6834","nonce":1669677937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:56.449 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 92 2023-10-18T12:00:56.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78a806b080 msgr2=0x7f78a806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78a806b080 0x7f78a806d540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0c8ee0 tx=0x7f78b0008040 comp rx=0 tx=0).stop 2023-10-18T12:00:56.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 msgr2=0x7f78bc1f2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 0x7f78bc1f2a40 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c00 tx=0x7f78ac02db10 comp rx=0 tx=0).stop 2023-10-18T12:00:56.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 shutdown_connections 2023-10-18T12:00:56.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78a806b080 0x7f78a806d540 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78bc0abc10 0x7f78bc151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78bc0c96b0 0x7f78bc0700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 --2- 172.21.15.125:0/961909885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78bc06af00 0x7f78bc1f2a40 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 >> 172.21.15.125:0/961909885 conn(0x7f78bc069930 msgr2=0x7f78bc0ac550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:56.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 shutdown_connections 2023-10-18T12:00:56.452 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.446+0000 7f78c2c75640 1 -- 172.21.15.125:0/961909885 wait complete. 2023-10-18T12:00:56.460 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:56.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3211977028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 msgr2=0x7f19f80cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3211977028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 0x7f19f80cf850 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f19f80cb6e0 tx=0x7f19e002bda0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3211977028 shutdown_connections 2023-10-18T12:00:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3211977028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 0x7f19f80cf850 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3211977028 >> v2:172.21.15.139:3300/0 conn(0x7f19f80b0b10 0x7f19f80b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3211977028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19f815b260 0x7f19f80cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3211977028 >> 172.21.15.125:0/3211977028 conn(0x7f19f809a030 msgr2=0x7f19f80d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3211977028 shutdown_connections 2023-10-18T12:00:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.559+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3211977028 wait complete. 2023-10-18T12:00:56.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.560+0000 7f19fdd8e640 1 Processor -- start 2023-10-18T12:00:56.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.560+0000 7f19fdd8e640 1 -- start start 2023-10-18T12:00:56.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19fdd8e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19f80b0b10 0x7f19f80b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19fdd8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f815b260 0x7f19f814d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19fdd8e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 0x7f19f81eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19fdd8e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80b6b30 con 0x7f19f815b640 2023-10-18T12:00:56.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19fdd8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f815cc30 con 0x7f19f815b260 2023-10-18T12:00:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19fdd8e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f814ddc0 con 0x7f19f80b0b10 2023-10-18T12:00:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 0x7f19f81eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 0x7f19f81eee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55214/0 (socket says 172.21.15.125:55214) 2023-10-18T12:00:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19f7fff640 1 -- 172.21.15.125:0/3369255909 learned_addr learned my addr 172.21.15.125:0/3369255909 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19f77fe640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19f80b0b10 0x7f19f80b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.561+0000 7f19f6ffd640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f815b260 0x7f19f814d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19f7fff640 1 -- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19f80b0b10 msgr2=0x7f19f80b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19f7fff640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19f80b0b10 0x7f19f80b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19f7fff640 1 -- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f815b260 msgr2=0x7f19f814d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19f7fff640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f815b260 0x7f19f814d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19f7fff640 1 -- 172.21.15.125:0/3369255909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0009560 con 0x7f19f815b640 2023-10-18T12:00:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19f7fff640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 0x7f19f81eee70 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f19e800d680 tx=0x7f19e800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19effff640 1 -- 172.21.15.125:0/3369255909 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8005220 con 0x7f19f815b640 2023-10-18T12:00:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19effff640 1 -- 172.21.15.125:0/3369255909 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f19e8005490 con 0x7f19f815b640 2023-10-18T12:00:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f81ef3b0 con 0x7f19f815b640 2023-10-18T12:00:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19effff640 1 -- 172.21.15.125:0/3369255909 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8009a80 con 0x7f19f815b640 2023-10-18T12:00:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.562+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f80b3cb0 con 0x7f19f815b640 2023-10-18T12:00:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.563+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f815cdc0 con 0x7f19f815b640 2023-10-18T12:00:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.563+0000 7f19effff640 1 -- 172.21.15.125:0/3369255909 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f19e8009be0 con 0x7f19f815b640 2023-10-18T12:00:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.564+0000 7f19effff640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f19d806b080 0x7f19d806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.564+0000 7f19effff640 1 -- 172.21.15.125:0/3369255909 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7f19e808a100 con 0x7f19f815b640 2023-10-18T12:00:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.564+0000 7f19f77fe640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f19d806b080 0x7f19d806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.564+0000 7f19f77fe640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f19d806b080 0x7f19d806d540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005fd0 tx=0x7f19f0009870 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:56.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.568+0000 7f19effff640 1 -- 172.21.15.125:0/3369255909 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19e80548c0 con 0x7f19f815b640 2023-10-18T12:00:56.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.734+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19f80675f0 con 0x7f19f815b640 2023-10-18T12:00:56.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.736+0000 7f19effff640 1 -- 172.21.15.125:0/3369255909 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v230) v1 ==== 74+0+18200 (secure 0 0 0) 0x7f19e8020090 con 0x7f19f815b640 2023-10-18T12:00:56.739 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:56.739 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":230,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:54.930283+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6832/2975172452":"2023-10-19T12:00:51.923831+0000","172.21.15.139:0/1563782559":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/170280857":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6833/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6832/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/2119410248":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3691026234":"2023-10-19T12:00:48.811703+0000","172.21.15.125:6834/2975172452":"2023-10-19T12:00:51.923831+0000","172.21.15.139:0/557716760":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/395739661":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/225240329":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1417331710":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6833/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6832/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3774276243":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3836307336":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/2081746818":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3501758629":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1472046677":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3403101268":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1324329019":"2023-10-19T12:00:40.842747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:56.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f19d806b080 msgr2=0x7f19d806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f19d806b080 0x7f19d806d540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005fd0 tx=0x7f19f0009870 comp rx=0 tx=0).stop 2023-10-18T12:00:56.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 msgr2=0x7f19f81eee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 0x7f19f81eee70 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f19e800d680 tx=0x7f19e800db40 comp rx=0 tx=0).stop 2023-10-18T12:00:56.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 shutdown_connections 2023-10-18T12:00:56.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f19d806b080 0x7f19d806d540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19f815b260 0x7f19f814d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f19f815b640 0x7f19f81eee70 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 --2- 172.21.15.125:0/3369255909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19f80b0b10 0x7f19f80b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 >> 172.21.15.125:0/3369255909 conn(0x7f19f809a030 msgr2=0x7f19f80aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:56.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 shutdown_connections 2023-10-18T12:00:56.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.738+0000 7f19fdd8e640 1 -- 172.21.15.125:0/3369255909 wait complete. 2023-10-18T12:00:56.751 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-18T12:00:56.752 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:56.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.852+0000 7fd4f1e44640 1 -- 172.21.15.125:0/1785566810 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4d8004ab0 msgr2=0x7fd4d8004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.852+0000 7fd4f1e44640 1 --2- 172.21.15.125:0/1785566810 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4d8004ab0 0x7fd4d8004ed0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc001650 tx=0x7fd4d8006900 comp rx=0 tx=0).stop 2023-10-18T12:00:56.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.852+0000 7fd4ebfff640 1 -- 172.21.15.125:0/1785566810 reap_dead start 2023-10-18T12:00:56.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.852+0000 7fd4f1e44640 1 -- 172.21.15.125:0/1785566810 shutdown_connections 2023-10-18T12:00:56.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.852+0000 7fd4f1e44640 1 -- 172.21.15.125:0/1785566810 >> 172.21.15.125:0/1785566810 conn(0x7fd4ec069930 msgr2=0x7fd4ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:56.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.852+0000 7fd4f1e44640 1 -- 172.21.15.125:0/1785566810 shutdown_connections 2023-10-18T12:00:56.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.852+0000 7fd4f1e44640 1 -- 172.21.15.125:0/1785566810 wait complete. 2023-10-18T12:00:56.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.853+0000 7fd4f1e44640 1 Processor -- start 2023-10-18T12:00:56.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.853+0000 7fd4f1e44640 1 -- start start 2023-10-18T12:00:56.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4f1e44640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4d8004ab0 0x7fd4ec13cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4f1e44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4ec15ae00 0x7fd4ec15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4f1e44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4ec13ded0 0x7fd4ec13e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4f1e44640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec15b7a0 con 0x7fd4ec13ded0 2023-10-18T12:00:56.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4f1e44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec13f250 con 0x7fd4ec15ae00 2023-10-18T12:00:56.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4f1e44640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec13d120 con 0x7fd4d8004ab0 2023-10-18T12:00:56.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4ec13ded0 0x7fd4ec13e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4ec13ded0 0x7fd4ec13e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55240/0 (socket says 172.21.15.125:55240) 2023-10-18T12:00:56.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 -- 172.21.15.125:0/461181688 learned_addr learned my addr 172.21.15.125:0/461181688 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:56.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4e2ffd640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4ec15ae00 0x7fd4ec15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 -- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4d8004ab0 msgr2=0x7fd4ec13cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4d8004ab0 0x7fd4ec13cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 -- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4ec15ae00 msgr2=0x7fd4ec15b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:56.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4ec15ae00 0x7fd4ec15b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:56.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 -- 172.21.15.125:0/461181688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec13ec90 con 0x7fd4ec13ded0 2023-10-18T12:00:56.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.854+0000 7fd4ebfff640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4ec13ded0 0x7fd4ec13e350 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4000f80 tx=0x7fd4e4006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:56.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.855+0000 7fd4e8ff9640 1 -- 172.21.15.125:0/461181688 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400fc90 con 0x7fd4ec13ded0 2023-10-18T12:00:56.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.855+0000 7fd4e8ff9640 1 -- 172.21.15.125:0/461181688 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd4e400fdf0 con 0x7fd4ec13ded0 2023-10-18T12:00:56.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.855+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec13ef70 con 0x7fd4ec13ded0 2023-10-18T12:00:56.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.855+0000 7fd4e8ff9640 1 -- 172.21.15.125:0/461181688 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4011690 con 0x7fd4ec13ded0 2023-10-18T12:00:56.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.855+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec140d30 con 0x7fd4ec13ded0 2023-10-18T12:00:56.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.856+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec0690e0 con 0x7fd4ec13ded0 2023-10-18T12:00:56.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.856+0000 7fd4e8ff9640 1 -- 172.21.15.125:0/461181688 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd4e40117f0 con 0x7fd4ec13ded0 2023-10-18T12:00:56.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.856+0000 7fd4e8ff9640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd4cc06b0b0 0x7fd4cc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:56.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.857+0000 7fd4e8ff9640 1 -- 172.21.15.125:0/461181688 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7fd4e408aec0 con 0x7fd4ec13ded0 2023-10-18T12:00:56.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.857+0000 7fd4eb7fe640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd4cc06b0b0 0x7fd4cc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:56.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.857+0000 7fd4eb7fe640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd4cc06b0b0 0x7fd4cc06d570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc034000 tx=0x7fd4dc001120 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:56.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:56.861+0000 7fd4e8ff9640 1 -- 172.21.15.125:0/461181688 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4e4059110 con 0x7fd4ec13ded0 2023-10-18T12:00:57.002 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:00:57.005 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:00:57.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.064+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4ec140ef0 con 0x7fd4ec13ded0 2023-10-18T12:00:57.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.065+0000 7fd4e8ff9640 1 -- 172.21.15.125:0/461181688 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fd4e4020020 con 0x7fd4ec13ded0 2023-10-18T12:00:57.068 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:57.068 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":12609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1669677937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1669677937},{"type":"v1","addr":"172.21.15.125:6834","nonce":1669677937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:57.069 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 92 2023-10-18T12:00:57.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.066+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd4cc06b0b0 msgr2=0x7fd4cc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.066+0000 7fd4f1e44640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd4cc06b0b0 0x7fd4cc06d570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc034000 tx=0x7fd4dc001120 comp rx=0 tx=0).stop 2023-10-18T12:00:57.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.066+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4ec13ded0 msgr2=0x7fd4ec13e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.066+0000 7fd4f1e44640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4ec13ded0 0x7fd4ec13e350 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4000f80 tx=0x7fd4e4006660 comp rx=0 tx=0).stop 2023-10-18T12:00:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.067+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 shutdown_connections 2023-10-18T12:00:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.067+0000 7fd4f1e44640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd4cc06b0b0 0x7fd4cc06d570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.067+0000 7fd4f1e44640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4ec15ae00 0x7fd4ec15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.067+0000 7fd4f1e44640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4ec13ded0 0x7fd4ec13e350 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.067+0000 7fd4f1e44640 1 --2- 172.21.15.125:0/461181688 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4d8004ab0 0x7fd4ec13cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.067+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 >> 172.21.15.125:0/461181688 conn(0x7fd4ec069930 msgr2=0x7fd4ec070510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:57.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.067+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 shutdown_connections 2023-10-18T12:00:57.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.067+0000 7fd4f1e44640 1 -- 172.21.15.125:0/461181688 wait complete. 2023-10-18T12:00:57.080 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:00:57.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.221+0000 7f78facda640 1 -- 172.21.15.125:0/2579898978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 msgr2=0x7f78f4157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.221+0000 7f78facda640 1 --2- 172.21.15.125:0/2579898978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 0x7f78f4157dd0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f78ec005fd0 tx=0x7f78ec02c240 comp rx=0 tx=0).stop 2023-10-18T12:00:57.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.221+0000 7f78facda640 1 -- 172.21.15.125:0/2579898978 shutdown_connections 2023-10-18T12:00:57.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.221+0000 7f78facda640 1 --2- 172.21.15.125:0/2579898978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 0x7f78f4157dd0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.221+0000 7f78facda640 1 --2- 172.21.15.125:0/2579898978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78f406ae40 0x7f78f40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.221+0000 7f78facda640 1 -- 172.21.15.125:0/2579898978 >> 172.21.15.125:0/2579898978 conn(0x7f78f406fae0 msgr2=0x7f78f406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:57.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.221+0000 7f78facda640 1 -- 172.21.15.125:0/2579898978 shutdown_connections 2023-10-18T12:00:57.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.221+0000 7f78facda640 1 -- 172.21.15.125:0/2579898978 wait complete. 2023-10-18T12:00:57.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.222+0000 7f78facda640 1 Processor -- start 2023-10-18T12:00:57.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.222+0000 7f78facda640 1 -- start start 2023-10-18T12:00:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78facda640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78f406ae40 0x7f78f40ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78facda640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78f41579f0 0x7f78f4140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78facda640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 0x7f78f4156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78facda640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40aeeb0 con 0x7f78f415c1f0 2023-10-18T12:00:57.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78facda640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40ae4e0 con 0x7f78f41579f0 2023-10-18T12:00:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78facda640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40ad5f0 con 0x7f78f406ae40 2023-10-18T12:00:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 0x7f78f4156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 0x7f78f4156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55258/0 (socket says 172.21.15.125:55258) 2023-10-18T12:00:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 -- 172.21.15.125:0/3679148497 learned_addr learned my addr 172.21.15.125:0/3679148497 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:57.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f3fff640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78f41579f0 0x7f78f4140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f8a76640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78f406ae40 0x7f78f40ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 -- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78f406ae40 msgr2=0x7f78f40ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78f406ae40 0x7f78f40ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 -- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78f41579f0 msgr2=0x7f78f4140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78f41579f0 0x7f78f4140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 -- 172.21.15.125:0/3679148497 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009900 con 0x7f78f415c1f0 2023-10-18T12:00:57.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f8a76640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78f406ae40 0x7f78f40ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:00:57.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f9277640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 0x7f78f4156420 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009d90 tx=0x7f78ec035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:57.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f17fa640 1 -- 172.21.15.125:0/3679148497 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec034440 con 0x7f78f415c1f0 2023-10-18T12:00:57.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f17fa640 1 -- 172.21.15.125:0/3679148497 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78ec004250 con 0x7f78f415c1f0 2023-10-18T12:00:57.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.223+0000 7f78f17fa640 1 -- 172.21.15.125:0/3679148497 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec004620 con 0x7f78f415c1f0 2023-10-18T12:00:57.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.224+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f40b2330 con 0x7f78f415c1f0 2023-10-18T12:00:57.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.224+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f40adbb0 con 0x7f78f415c1f0 2023-10-18T12:00:57.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.224+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f40ae670 con 0x7f78f415c1f0 2023-10-18T12:00:57.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.225+0000 7f78f17fa640 1 -- 172.21.15.125:0/3679148497 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f78ec00a040 con 0x7f78f415c1f0 2023-10-18T12:00:57.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.225+0000 7f78f17fa640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78d806b080 0x7f78d806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.225+0000 7f78f17fa640 1 -- 172.21.15.125:0/3679148497 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7f78ec0ad060 con 0x7f78f415c1f0 2023-10-18T12:00:57.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.225+0000 7f78f8a76640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78d806b080 0x7f78d806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.226+0000 7f78f8a76640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78d806b080 0x7f78d806d540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f78e8003eb0 tx=0x7f78e8009040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:57.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.229+0000 7f78f17fa640 1 -- 172.21.15.125:0/3679148497 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78ec0779c0 con 0x7f78f415c1f0 2023-10-18T12:00:57.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.428+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78f40690e0 con 0x7f78f415c1f0 2023-10-18T12:00:57.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.428+0000 7f78f17fa640 1 -- 172.21.15.125:0/3679148497 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f78ec044090 con 0x7f78f415c1f0 2023-10-18T12:00:57.431 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:57.432 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":12609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1669677937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1669677937},{"type":"v1","addr":"172.21.15.125:6834","nonce":1669677937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:00:31.751249+0000","modified":"2023-10-18T12:00:33.761902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10168},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10168":{"gid":10168,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":6,"addr":"172.21.15.139:6837/1825099907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1825099907},{"type":"v1","addr":"172.21.15.139:6837","nonce":1825099907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-18T12:00:57.432 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 92 2023-10-18T12:00:57.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78d806b080 msgr2=0x7f78d806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78d806b080 0x7f78d806d540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f78e8003eb0 tx=0x7f78e8009040 comp rx=0 tx=0).stop 2023-10-18T12:00:57.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 msgr2=0x7f78f4156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 0x7f78f4156420 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009d90 tx=0x7f78ec035e30 comp rx=0 tx=0).stop 2023-10-18T12:00:57.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 shutdown_connections 2023-10-18T12:00:57.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f78d806b080 0x7f78d806d540 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78f41579f0 0x7f78f4140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f415c1f0 0x7f78f4156420 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 --2- 172.21.15.125:0/3679148497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78f406ae40 0x7f78f40ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 >> 172.21.15.125:0/3679148497 conn(0x7f78f406fae0 msgr2=0x7f78f415a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:57.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 shutdown_connections 2023-10-18T12:00:57.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.430+0000 7f78facda640 1 -- 172.21.15.125:0/3679148497 wait complete. 2023-10-18T12:00:57.443 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:00:57.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.543+0000 7ff8f091d640 1 -- 172.21.15.125:0/1047513241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0cf340 msgr2=0x7ff8ec0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.543+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1047513241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0cf340 0x7ff8ec0d1910 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005fd0 tx=0x7ff8dc02bcf0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.543+0000 7ff8f091d640 1 -- 172.21.15.125:0/1047513241 shutdown_connections 2023-10-18T12:00:57.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.543+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1047513241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8ec0ceb40 0x7ff8ec0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.543+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1047513241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0cf340 0x7ff8ec0d1910 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.543+0000 7ff8f091d640 1 -- 172.21.15.125:0/1047513241 >> 172.21.15.125:0/1047513241 conn(0x7ff8ec0d0d60 msgr2=0x7ff8ec0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:57.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.543+0000 7ff8f091d640 1 -- 172.21.15.125:0/1047513241 shutdown_connections 2023-10-18T12:00:57.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.543+0000 7ff8f091d640 1 -- 172.21.15.125:0/1047513241 wait complete. 2023-10-18T12:00:57.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.544+0000 7ff8f091d640 1 Processor -- start 2023-10-18T12:00:57.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.544+0000 7ff8f091d640 1 -- start start 2023-10-18T12:00:57.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8f091d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0ceb40 0x7ff8ec155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8f091d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8ec0cf340 0x7ff8ec0b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8f091d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8ec15b2a0 0x7ff8ec1565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8f091d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec15c5a0 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8f091d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec15b6d0 con 0x7ff8ec0cf340 2023-10-18T12:00:57.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8f091d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec15b920 con 0x7ff8ec15b2a0 2023-10-18T12:00:57.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ea59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0ceb40 0x7ff8ec155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ea59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0ceb40 0x7ff8ec155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54078/0 (socket says 172.21.15.125:54078) 2023-10-18T12:00:57.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ea59c640 1 -- 172.21.15.125:0/1362071026 learned_addr learned my addr 172.21.15.125:0/1362071026 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:57.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ead9d640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8ec15b2a0 0x7ff8ec1565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8e3fff640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8ec0cf340 0x7ff8ec0b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ea59c640 1 -- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8ec15b2a0 msgr2=0x7ff8ec1565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ea59c640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8ec15b2a0 0x7ff8ec1565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ea59c640 1 -- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8ec0cf340 msgr2=0x7ff8ec0b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ea59c640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8ec0cf340 0x7ff8ec0b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ea59c640 1 -- 172.21.15.125:0/1362071026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc009560 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8e3fff640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8ec0cf340 0x7ff8ec0b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:57.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.545+0000 7ff8ead9d640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8ec15b2a0 0x7ff8ec1565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:00:57.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.546+0000 7ff8ea59c640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0ceb40 0x7ff8ec155240 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009850 tx=0x7ff8dc02d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:57.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.546+0000 7ff8e2ffd640 1 -- 172.21.15.125:0/1362071026 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc02ddc0 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.546+0000 7ff8e2ffd640 1 -- 172.21.15.125:0/1362071026 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff8dc0036e0 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.546+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec0b0d50 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.546+0000 7ff8e2ffd640 1 -- 172.21.15.125:0/1362071026 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc003a90 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.546+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec15bee0 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.547+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec0690e0 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.547+0000 7ff8e2ffd640 1 -- 172.21.15.125:0/1362071026 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff8dc003bf0 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.547+0000 7ff8e2ffd640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff8cc06b080 0x7ff8cc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.548+0000 7ff8e2ffd640 1 -- 172.21.15.125:0/1362071026 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7ff8dc0ac970 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.548+0000 7ff8e3fff640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff8cc06b080 0x7ff8cc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.548+0000 7ff8e3fff640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff8cc06b080 0x7ff8cc06d540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0afa80 tx=0x7ff8d4008040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:57.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.552+0000 7ff8e2ffd640 1 -- 172.21.15.125:0/1362071026 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8dc077240 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.721+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8ec15a1b0 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.722+0000 7ff8e2ffd640 1 -- 172.21.15.125:0/1362071026 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v230) v1 ==== 74+0+18200 (secure 0 0 0) 0x7ff8dc07a860 con 0x7ff8ec0ceb40 2023-10-18T12:00:57.725 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:00:57.725 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":230,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:00:54.930283+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:00:30.176449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"224","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:00:30.730388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"225","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6832/2975172452":"2023-10-19T12:00:51.923831+0000","172.21.15.139:0/1563782559":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/170280857":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6833/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6832/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/2119410248":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3691026234":"2023-10-19T12:00:48.811703+0000","172.21.15.125:6834/2975172452":"2023-10-19T12:00:51.923831+0000","172.21.15.139:0/557716760":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/395739661":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/225240329":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1417331710":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6833/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6832/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3774276243":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3836307336":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/2081746818":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3501758629":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1472046677":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3403101268":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1324329019":"2023-10-19T12:00:40.842747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:00:57.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff8cc06b080 msgr2=0x7ff8cc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff8cc06b080 0x7ff8cc06d540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0afa80 tx=0x7ff8d4008040 comp rx=0 tx=0).stop 2023-10-18T12:00:57.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0ceb40 msgr2=0x7ff8ec155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0ceb40 0x7ff8ec155240 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009850 tx=0x7ff8dc02d9c0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 shutdown_connections 2023-10-18T12:00:57.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff8cc06b080 0x7ff8cc06d540 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8ec0cf340 0x7ff8ec0b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff8ec15b2a0 0x7ff8ec1565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 --2- 172.21.15.125:0/1362071026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8ec0ceb40 0x7ff8ec155240 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 >> 172.21.15.125:0/1362071026 conn(0x7ff8ec0d0d60 msgr2=0x7ff8ec06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:57.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 shutdown_connections 2023-10-18T12:00:57.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.724+0000 7ff8f091d640 1 -- 172.21.15.125:0/1362071026 wait complete. 2023-10-18T12:00:57.738 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-18T12:00:57.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.839+0000 7ffa10461640 1 -- 172.21.15.125:0/186535671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 msgr2=0x7ff9f400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.839+0000 7ffa10461640 1 --2- 172.21.15.125:0/186535671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 0x7ff9f400b980 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009ed0 tx=0x7ff9f40057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.839+0000 7ffa10461640 1 -- 172.21.15.125:0/186535671 shutdown_connections 2023-10-18T12:00:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.839+0000 7ffa10461640 1 --2- 172.21.15.125:0/186535671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 0x7ff9f400b980 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.839+0000 7ffa10461640 1 -- 172.21.15.125:0/186535671 >> 172.21.15.125:0/186535671 conn(0x7ffa080b3f00 msgr2=0x7ffa080b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:57.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.839+0000 7ffa10461640 1 -- 172.21.15.125:0/186535671 shutdown_connections 2023-10-18T12:00:57.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.839+0000 7ffa10461640 1 -- 172.21.15.125:0/186535671 wait complete. 2023-10-18T12:00:57.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.840+0000 7ffa10461640 1 Processor -- start 2023-10-18T12:00:57.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa10461640 1 -- start start 2023-10-18T12:00:57.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa10461640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 0x7ffa08147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa10461640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa080cf3e0 0x7ffa080994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa10461640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa080ce6d0 0x7ffa080ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa10461640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08147b60 con 0x7ff9f4005180 2023-10-18T12:00:57.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa10461640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080cb4e0 con 0x7ffa080cf3e0 2023-10-18T12:00:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa10461640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08099d30 con 0x7ffa080ce6d0 2023-10-18T12:00:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa0e1fd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 0x7ffa08147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa0e1fd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 0x7ffa08147620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54092/0 (socket says 172.21.15.125:54092) 2023-10-18T12:00:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.841+0000 7ffa0e1fd640 1 -- 172.21.15.125:0/2739353924 learned_addr learned my addr 172.21.15.125:0/2739353924 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:57.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ffa0e1fd640 1 -- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa080ce6d0 msgr2=0x7ffa080ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ffa0e1fd640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa080ce6d0 0x7ffa080ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ffa0e1fd640 1 -- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa080cf3e0 msgr2=0x7ffa080994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ffa0e1fd640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa080cf3e0 0x7ffa080994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ffa0e1fd640 1 -- 172.21.15.125:0/2739353924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0806d5b0 con 0x7ff9f4005180 2023-10-18T12:00:57.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ffa0e1fd640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 0x7ffa08147620 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ffa040017a0 tx=0x7ffa0400ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:57.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ff9feffd640 1 -- 172.21.15.125:0/2739353924 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400c8f0 con 0x7ff9f4005180 2023-10-18T12:00:57.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ff9feffd640 1 -- 172.21.15.125:0/2739353924 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa0400cb40 con 0x7ff9f4005180 2023-10-18T12:00:57.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0809a010 con 0x7ff9f4005180 2023-10-18T12:00:57.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ff9feffd640 1 -- 172.21.15.125:0/2739353924 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04011460 con 0x7ff9f4005180 2023-10-18T12:00:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.842+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa080cb780 con 0x7ff9f4005180 2023-10-18T12:00:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.843+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa080690e0 con 0x7ff9f4005180 2023-10-18T12:00:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.843+0000 7ff9feffd640 1 -- 172.21.15.125:0/2739353924 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ffa040115c0 con 0x7ff9f4005180 2023-10-18T12:00:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.844+0000 7ff9feffd640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff9ec06b080 0x7ff9ec06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:57.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.844+0000 7ff9feffd640 1 -- 172.21.15.125:0/2739353924 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 7000+0+0 (secure 0 0 0) 0x7ffa0408a800 con 0x7ff9f4005180 2023-10-18T12:00:57.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.844+0000 7ffa0d9fc640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff9ec06b080 0x7ff9ec06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:57.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.844+0000 7ffa0d9fc640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff9ec06b080 0x7ff9ec06d540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8006fd0 tx=0x7ff9f8006890 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:57.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:57.848+0000 7ff9feffd640 1 -- 172.21.15.125:0/2739353924 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa04054f30 con 0x7ff9f4005180 2023-10-18T12:00:58.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.050+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7ffa080b2da0 con 0x7ff9f4005180 2023-10-18T12:00:58.053 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T12:00:58.050+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:00:58.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.052+0000 7ff9feffd640 1 -- 172.21.15.125:0/2739353924 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) v1 ==== 167+0+0 (secure 0 0 0) 0x7ffa0401f090 con 0x7ff9f4005180 2023-10-18T12:00:58.055 INFO:teuthology.orchestra.run.smithi125.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T12:00:58.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff9ec06b080 msgr2=0x7ff9ec06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:58.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff9ec06b080 0x7ff9ec06d540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8006fd0 tx=0x7ff9f8006890 comp rx=0 tx=0).stop 2023-10-18T12:00:58.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 msgr2=0x7ffa08147620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:58.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 0x7ffa08147620 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ffa040017a0 tx=0x7ffa0400ea00 comp rx=0 tx=0).stop 2023-10-18T12:00:58.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 shutdown_connections 2023-10-18T12:00:58.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff9ec06b080 0x7ff9ec06d540 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:58.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa080cf3e0 0x7ffa080994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:58.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa080ce6d0 0x7ffa080ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:58.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 --2- 172.21.15.125:0/2739353924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4005180 0x7ffa08147620 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:58.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 >> 172.21.15.125:0/2739353924 conn(0x7ffa080b3f00 msgr2=0x7ffa080b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:58.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 shutdown_connections 2023-10-18T12:00:58.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.054+0000 7ffa10461640 1 -- 172.21.15.125:0/2739353924 wait complete. 2023-10-18T12:00:58.060 INFO:tasks.ceph.mds.d.smithi139.stderr: -1558> 2023-10-18T12:00:36.037+0000 7f329c96f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:00:58.067 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-18T12:00:58.086 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T12:00:58.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.170+0000 7f53a36fc640 1 -- 172.21.15.125:0/284109705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5388005180 msgr2=0x7f538800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:58.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.170+0000 7f53a36fc640 1 --2- 172.21.15.125:0/284109705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5388005180 0x7f538800b980 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f5394001d80 tx=0x7f53880057f0 comp rx=0 tx=0).stop 2023-10-18T12:00:58.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.170+0000 7f53a36fc640 1 -- 172.21.15.125:0/284109705 shutdown_connections 2023-10-18T12:00:58.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.170+0000 7f53a36fc640 1 --2- 172.21.15.125:0/284109705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5388005180 0x7f538800b980 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:58.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.170+0000 7f53a36fc640 1 -- 172.21.15.125:0/284109705 >> 172.21.15.125:0/284109705 conn(0x7f539c069930 msgr2=0x7f539c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:58.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.170+0000 7f53a36fc640 1 -- 172.21.15.125:0/284109705 shutdown_connections 2023-10-18T12:00:58.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.170+0000 7f53a36fc640 1 -- 172.21.15.125:0/284109705 wait complete. 2023-10-18T12:00:58.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.171+0000 7f53a36fc640 1 Processor -- start 2023-10-18T12:00:58.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.171+0000 7f53a36fc640 1 -- start start 2023-10-18T12:00:58.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a36fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5388005180 0x7f539c13d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:58.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a36fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f539c0abea0 0x7f539c1efa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:58.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a36fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c13e6d0 0x7f539c1f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:58.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a36fc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c13e240 con 0x7f539c0abea0 2023-10-18T12:00:58.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a36fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c1eff80 con 0x7f539c13e6d0 2023-10-18T12:00:58.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a36fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c0ac320 con 0x7f5388005180 2023-10-18T12:00:58.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f539c0abea0 0x7f539c1efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:58.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f539c0abea0 0x7f539c1efa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54112/0 (socket says 172.21.15.125:54112) 2023-10-18T12:00:58.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 -- 172.21.15.125:0/1351213082 learned_addr learned my addr 172.21.15.125:0/1351213082 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:58.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 -- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5388005180 msgr2=0x7f539c13d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:58.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5388005180 0x7f539c13d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:58.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a1c99640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c13e6d0 0x7f539c1f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:58.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 -- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c13e6d0 msgr2=0x7f539c1f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:58.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c13e6d0 0x7f539c1f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:58.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 -- 172.21.15.125:0/1351213082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c13d630 con 0x7f539c0abea0 2023-10-18T12:00:58.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f53a0c97640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f539c0abea0 0x7f539c1efa40 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f538c006590 tx=0x7f538c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.172+0000 7f5391ffb640 1 -- 172.21.15.125:0/1351213082 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c011900 con 0x7f539c0abea0 2023-10-18T12:00:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.173+0000 7f5391ffb640 1 -- 172.21.15.125:0/1351213082 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f538c011a60 con 0x7f539c0abea0 2023-10-18T12:00:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.173+0000 7f5391ffb640 1 -- 172.21.15.125:0/1351213082 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c011e70 con 0x7f539c0abea0 2023-10-18T12:00:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.173+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c13d910 con 0x7f539c0abea0 2023-10-18T12:00:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.173+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c13ded0 con 0x7f539c0abea0 2023-10-18T12:00:58.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.174+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c13e060 con 0x7f539c0abea0 2023-10-18T12:00:58.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.174+0000 7f5391ffb640 1 -- 172.21.15.125:0/1351213082 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f538c010420 con 0x7f539c0abea0 2023-10-18T12:00:58.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.174+0000 7f5391ffb640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f538006b0b0 0x7f538006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:58.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.174+0000 7f5391ffb640 1 -- 172.21.15.125:0/1351213082 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7086+0+0 (secure 0 0 0) 0x7f538c08b1f0 con 0x7f539c0abea0 2023-10-18T12:00:58.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.175+0000 7f53a1498640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f538006b0b0 0x7f538006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:58.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.175+0000 7f53a1498640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f538006b0b0 0x7f538006d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f539c1f2510 tx=0x7f539800a480 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:58.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.178+0000 7f5391ffb640 1 -- 172.21.15.125:0/1351213082 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f538c0558e0 con 0x7f539c0abea0 2023-10-18T12:00:58.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:58.381+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f539c0690e0 con 0x7f539c0abea0 2023-10-18T12:00:59.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.056+0000 7f5391ffb640 1 -- 172.21.15.125:0/1351213082 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 107+0+0 (secure 0 0 0) 0x7f538c058f00 con 0x7f539c0abea0 2023-10-18T12:00:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f538006b0b0 msgr2=0x7f538006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f538006b0b0 0x7f538006d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f539c1f2510 tx=0x7f539800a480 comp rx=0 tx=0).stop 2023-10-18T12:00:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f539c0abea0 msgr2=0x7f539c1efa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f539c0abea0 0x7f539c1efa40 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f538c006590 tx=0x7f538c0039c0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 shutdown_connections 2023-10-18T12:00:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f538006b0b0 0x7f538006d570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f539c13e6d0 0x7f539c1f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f539c0abea0 0x7f539c1efa40 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 --2- 172.21.15.125:0/1351213082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5388005180 0x7f539c13d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 >> 172.21.15.125:0/1351213082 conn(0x7f539c069930 msgr2=0x7f539c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:59.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 shutdown_connections 2023-10-18T12:00:59.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.059+0000 7f53a36fc640 1 -- 172.21.15.125:0/1351213082 wait complete. 2023-10-18T12:00:59.074 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-18T12:00:59.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.183+0000 7f94dc261640 1 -- 172.21.15.125:0/591442421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406d2b0 msgr2=0x7f94d415bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:59.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.183+0000 7f94dc261640 1 --2- 172.21.15.125:0/591442421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406d2b0 0x7f94d415bb90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f94c4005fd0 tx=0x7f94c402be10 comp rx=0 tx=0).stop 2023-10-18T12:00:59.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.183+0000 7f94dc261640 1 -- 172.21.15.125:0/591442421 shutdown_connections 2023-10-18T12:00:59.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.183+0000 7f94dc261640 1 --2- 172.21.15.125:0/591442421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406d2b0 0x7f94d415bb90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.183+0000 7f94dc261640 1 --2- 172.21.15.125:0/591442421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d415b7b0 0x7f94d40b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.183+0000 7f94dc261640 1 -- 172.21.15.125:0/591442421 >> 172.21.15.125:0/591442421 conn(0x7f94d40b3f00 msgr2=0x7f94d40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:00:59.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.184+0000 7f94dc261640 1 -- 172.21.15.125:0/591442421 shutdown_connections 2023-10-18T12:00:59.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.184+0000 7f94dc261640 1 -- 172.21.15.125:0/591442421 wait complete. 2023-10-18T12:00:59.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.185+0000 7f94dc261640 1 Processor -- start 2023-10-18T12:00:59.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.185+0000 7f94dc261640 1 -- start start 2023-10-18T12:00:59.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.185+0000 7f94dc261640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406d2b0 0x7f94d414a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:59.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.185+0000 7f94dc261640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d415b7b0 0x7f94d4153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.185+0000 7f94dc261640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415c0d0 0x7f94d40b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.185+0000 7f94dc261640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40b0030 con 0x7f94d415c0d0 2023-10-18T12:00:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.185+0000 7f94dc261640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40b0310 con 0x7f94d406d2b0 2023-10-18T12:00:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.185+0000 7f94dc261640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40b05f0 con 0x7f94d415b7b0 2023-10-18T12:00:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415c0d0 0x7f94d40b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415c0d0 0x7f94d40b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54118/0 (socket says 172.21.15.125:54118) 2023-10-18T12:00:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 -- 172.21.15.125:0/3367909005 learned_addr learned my addr 172.21.15.125:0/3367909005 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:00:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94d9ffd640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406d2b0 0x7f94d414a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94d97fc640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d415b7b0 0x7f94d4153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 -- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d415b7b0 msgr2=0x7f94d4153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d415b7b0 0x7f94d4153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 -- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406d2b0 msgr2=0x7f94d414a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:00:59.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406d2b0 0x7f94d414a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:00:59.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 -- 172.21.15.125:0/3367909005 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0009840 con 0x7f94d415c0d0 2023-10-18T12:00:59.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94da7fe640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415c0d0 0x7f94d40b10d0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00c3c0 tx=0x7f94cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94ca7fc640 1 -- 172.21.15.125:0/3367909005 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0052d0 con 0x7f94d415c0d0 2023-10-18T12:00:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94ca7fc640 1 -- 172.21.15.125:0/3367909005 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94cc007020 con 0x7f94d415c0d0 2023-10-18T12:00:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94ca7fc640 1 -- 172.21.15.125:0/3367909005 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc007470 con 0x7f94d415c0d0 2023-10-18T12:00:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c4009560 con 0x7f94d415c0d0 2023-10-18T12:00:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.186+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d40b0bb0 con 0x7f94d415c0d0 2023-10-18T12:00:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.187+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d40690e0 con 0x7f94d415c0d0 2023-10-18T12:00:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.188+0000 7f94ca7fc640 1 -- 172.21.15.125:0/3367909005 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f94cc0067d0 con 0x7f94d415c0d0 2023-10-18T12:00:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.188+0000 7f94ca7fc640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f94b806b080 0x7f94b806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:00:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.188+0000 7f94ca7fc640 1 -- 172.21.15.125:0/3367909005 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 7086+0+0 (secure 0 0 0) 0x7f94cc0893f0 con 0x7f94d415c0d0 2023-10-18T12:00:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.189+0000 7f94d9ffd640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f94b806b080 0x7f94b806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:00:59.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.189+0000 7f94d9ffd640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f94b806b080 0x7f94b806d540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005fd0 tx=0x7f94d0009500 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:00:59.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.192+0000 7f94ca7fc640 1 -- 172.21.15.125:0/3367909005 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94cc053cb0 con 0x7f94d415c0d0 2023-10-18T12:00:59.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:00:59.362+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94d4159190 con 0x7f94d415c0d0 2023-10-18T12:01:00.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.059+0000 7f94ca7fc640 1 -- 172.21.15.125:0/3367909005 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v232) v1 ==== 190+0+0 (secure 0 0 0) 0x7f94cc0572d0 con 0x7f94d415c0d0 2023-10-18T12:01:00.063 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'backup_fs_metadata' removed 2023-10-18T12:01:00.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f94b806b080 msgr2=0x7f94b806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:00.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f94b806b080 0x7f94b806d540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005fd0 tx=0x7f94d0009500 comp rx=0 tx=0).stop 2023-10-18T12:01:00.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415c0d0 msgr2=0x7f94d40b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:00.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415c0d0 0x7f94d40b10d0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00c3c0 tx=0x7f94cc00c880 comp rx=0 tx=0).stop 2023-10-18T12:01:00.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 shutdown_connections 2023-10-18T12:01:00.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f94b806b080 0x7f94b806d540 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:00.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94d415c0d0 0x7f94d40b10d0 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:00.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94d415b7b0 0x7f94d4153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:00.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 --2- 172.21.15.125:0/3367909005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94d406d2b0 0x7f94d414a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:00.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 >> 172.21.15.125:0/3367909005 conn(0x7f94d40b3f00 msgr2=0x7f94d40b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:00.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 shutdown_connections 2023-10-18T12:01:00.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.062+0000 7f94dc261640 1 -- 172.21.15.125:0/3367909005 wait complete. 2023-10-18T12:01:00.078 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-18T12:01:00.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.185+0000 7f06eb5c5640 1 -- 172.21.15.125:0/4015590078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e413e520 msgr2=0x7f06e406bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:00.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.185+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/4015590078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e413e520 0x7f06e406bb60 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005fd0 tx=0x7f06e002bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:00.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.185+0000 7f06eb5c5640 1 -- 172.21.15.125:0/4015590078 shutdown_connections 2023-10-18T12:01:00.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.185+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/4015590078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06e4070af0 0x7f06e40affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:00.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.185+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/4015590078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e413e520 0x7f06e406bb60 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:00.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.185+0000 7f06eb5c5640 1 -- 172.21.15.125:0/4015590078 >> 172.21.15.125:0/4015590078 conn(0x7f06e4069930 msgr2=0x7f06e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:00.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.185+0000 7f06eb5c5640 1 -- 172.21.15.125:0/4015590078 shutdown_connections 2023-10-18T12:01:00.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.185+0000 7f06eb5c5640 1 -- 172.21.15.125:0/4015590078 wait complete. 2023-10-18T12:01:00.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.186+0000 7f06eb5c5640 1 Processor -- start 2023-10-18T12:01:00.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.186+0000 7f06eb5c5640 1 -- start start 2023-10-18T12:01:00.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06eb5c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e4070af0 0x7f06e4158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:00.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06eb5c5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06e413e520 0x7f06e413b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:00.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06eb5c5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06e4141690 0x7f06e4157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:00.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06eb5c5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e4158b50 con 0x7f06e4141690 2023-10-18T12:01:00.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06eb5c5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e413b780 con 0x7f06e4070af0 2023-10-18T12:01:00.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06eb5c5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e4157ed0 con 0x7f06e413e520 2023-10-18T12:01:00.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06e4141690 0x7f06e4157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:00.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06e4141690 0x7f06e4157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54130/0 (socket says 172.21.15.125:54130) 2023-10-18T12:01:00.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 -- 172.21.15.125:0/3437441546 learned_addr learned my addr 172.21.15.125:0/3437441546 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:00.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9361640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e4070af0 0x7f06e4158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:00.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e8b60640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06e413e520 0x7f06e413b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:00.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 -- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06e413e520 msgr2=0x7f06e413b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:00.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06e413e520 0x7f06e413b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:00.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 -- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e4070af0 msgr2=0x7f06e4158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:00.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e4070af0 0x7f06e4158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:00.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 -- 172.21.15.125:0/3437441546 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc00ba50 con 0x7f06e4141690 2023-10-18T12:01:00.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.187+0000 7f06e9b62640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06e4141690 0x7f06e4157680 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009e50 tx=0x7f06dc00e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:00.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.188+0000 7f06d9ffb640 1 -- 172.21.15.125:0/3437441546 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc010040 con 0x7f06e4141690 2023-10-18T12:01:00.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.188+0000 7f06d9ffb640 1 -- 172.21.15.125:0/3437441546 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f06dc00ee70 con 0x7f06e4141690 2023-10-18T12:01:00.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.188+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e0009560 con 0x7f06e4141690 2023-10-18T12:01:00.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.188+0000 7f06d9ffb640 1 -- 172.21.15.125:0/3437441546 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc014660 con 0x7f06e4141690 2023-10-18T12:01:00.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.188+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e4158490 con 0x7f06e4141690 2023-10-18T12:01:00.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.189+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e4158c90 con 0x7f06e4141690 2023-10-18T12:01:00.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.189+0000 7f06d9ffb640 1 -- 172.21.15.125:0/3437441546 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f06dc0147c0 con 0x7f06e4141690 2023-10-18T12:01:00.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.190+0000 7f06d9ffb640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f06c806b080 0x7f06c806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:00.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.190+0000 7f06d9ffb640 1 -- 172.21.15.125:0/3437441546 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f06dc08a430 con 0x7f06e4141690 2023-10-18T12:01:00.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.190+0000 7f06e9361640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f06c806b080 0x7f06c806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:00.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.190+0000 7f06e9361640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f06c806b080 0x7f06c806d540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06e0035000 tx=0x7f06e0004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:00.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.194+0000 7f06d9ffb640 1 -- 172.21.15.125:0/3437441546 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06dc054e00 con 0x7f06e4141690 2023-10-18T12:01:00.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:00.363+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06e40b0310 con 0x7f06e4141690 2023-10-18T12:01:01.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.068+0000 7f06d9ffb640 1 -- 172.21.15.125:0/3437441546 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v233) v1 ==== 178+0+0 (secure 0 0 0) 0x7f06dc058420 con 0x7f06e4141690 2023-10-18T12:01:01.071 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'backup_fs_data' removed 2023-10-18T12:01:01.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f06c806b080 msgr2=0x7f06c806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f06c806b080 0x7f06c806d540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06e0035000 tx=0x7f06e0004730 comp rx=0 tx=0).stop 2023-10-18T12:01:01.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06e4141690 msgr2=0x7f06e4157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06e4141690 0x7f06e4157680 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009e50 tx=0x7f06dc00e490 comp rx=0 tx=0).stop 2023-10-18T12:01:01.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 shutdown_connections 2023-10-18T12:01:01.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f06c806b080 0x7f06c806d540 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06e4141690 0x7f06e4157680 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06e413e520 0x7f06e413b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 --2- 172.21.15.125:0/3437441546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06e4070af0 0x7f06e4158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 >> 172.21.15.125:0/3437441546 conn(0x7f06e4069930 msgr2=0x7f06e415ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:01.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 shutdown_connections 2023-10-18T12:01:01.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.070+0000 7f06eb5c5640 1 -- 172.21.15.125:0/3437441546 wait complete. 2023-10-18T12:01:01.086 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T12:01:01.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.193+0000 7f109cb05640 1 -- 172.21.15.125:0/3875416219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 msgr2=0x7f1080003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.193+0000 7f109cb05640 1 --2- 172.21.15.125:0/3875416219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 0x7f1080003d90 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f108800c610 tx=0x7f10800057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.193+0000 7f109cb05640 1 -- 172.21.15.125:0/3875416219 shutdown_connections 2023-10-18T12:01:01.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.193+0000 7f109cb05640 1 --2- 172.21.15.125:0/3875416219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 0x7f1080003d90 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.193+0000 7f109cb05640 1 -- 172.21.15.125:0/3875416219 >> 172.21.15.125:0/3875416219 conn(0x7f10980d0d60 msgr2=0x7f10980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:01.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.193+0000 7f109cb05640 1 -- 172.21.15.125:0/3875416219 shutdown_connections 2023-10-18T12:01:01.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.193+0000 7f109cb05640 1 -- 172.21.15.125:0/3875416219 wait complete. 2023-10-18T12:01:01.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.194+0000 7f109cb05640 1 Processor -- start 2023-10-18T12:01:01.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.194+0000 7f109cb05640 1 -- start start 2023-10-18T12:01:01.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109cb05640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 0x7f10981f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:01.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109cb05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f109814fda0 0x7f10980b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:01.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109cb05640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10980b1f60 0x7f1098150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:01.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109cb05640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981f1d30 con 0x7f1080003970 2023-10-18T12:01:01.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109cb05640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980b23e0 con 0x7f109814fda0 2023-10-18T12:01:01.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109cb05640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10980c9950 con 0x7f10980b1f60 2023-10-18T12:01:01.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 0x7f10981f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:01.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 0x7f10981f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54154/0 (socket says 172.21.15.125:54154) 2023-10-18T12:01:01.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 -- 172.21.15.125:0/1616002648 learned_addr learned my addr 172.21.15.125:0/1616002648 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:01.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f1095d9b640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f109814fda0 0x7f10980b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:01.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f1096d9d640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10980b1f60 0x7f1098150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:01.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 -- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10980b1f60 msgr2=0x7f1098150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10980b1f60 0x7f1098150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 -- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f109814fda0 msgr2=0x7f10980b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f109814fda0 0x7f10980b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 -- 172.21.15.125:0/1616002648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098151390 con 0x7f1080003970 2023-10-18T12:01:01.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f1095d9b640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f109814fda0 0x7f10980b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:01.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109659c640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 0x7f10981f17f0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f108800c730 tx=0x7f10880019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:01.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f1086ffd640 1 -- 172.21.15.125:0/1616002648 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088003990 con 0x7f1080003970 2023-10-18T12:01:01.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f1086ffd640 1 -- 172.21.15.125:0/1616002648 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1088003af0 con 0x7f1080003970 2023-10-18T12:01:01.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1098151670 con 0x7f1080003970 2023-10-18T12:01:01.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.195+0000 7f1086ffd640 1 -- 172.21.15.125:0/1616002648 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088006070 con 0x7f1080003970 2023-10-18T12:01:01.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.196+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10980c95e0 con 0x7f1080003970 2023-10-18T12:01:01.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.197+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10980690e0 con 0x7f1080003970 2023-10-18T12:01:01.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.197+0000 7f1086ffd640 1 -- 172.21.15.125:0/1616002648 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1088031030 con 0x7f1080003970 2023-10-18T12:01:01.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.197+0000 7f1086ffd640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f107806b0b0 0x7f107806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:01.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.197+0000 7f1086ffd640 1 -- 172.21.15.125:0/1616002648 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6222+0+0 (secure 0 0 0) 0x7f10880aa830 con 0x7f1080003970 2023-10-18T12:01:01.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.198+0000 7f1095d9b640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f107806b0b0 0x7f107806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:01.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.198+0000 7f1095d9b640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f107806b0b0 0x7f107806d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f10981598c0 tx=0x7f108c006ba0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:01.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.202+0000 7f1086ffd640 1 -- 172.21.15.125:0/1616002648 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1088075340 con 0x7f1080003970 2023-10-18T12:01:01.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.368+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7f10980d0b40 con 0x7f1080003970 2023-10-18T12:01:01.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.374+0000 7f1086ffd640 1 -- 172.21.15.125:0/1616002648 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v78) v1 ==== 71+0+0 (secure 0 0 0) 0x7f1088043020 con 0x7f1080003970 2023-10-18T12:01:01.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f107806b0b0 msgr2=0x7f107806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f107806b0b0 0x7f107806d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f10981598c0 tx=0x7f108c006ba0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 msgr2=0x7f10981f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 0x7f10981f17f0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f108800c730 tx=0x7f10880019f0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 shutdown_connections 2023-10-18T12:01:01.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f107806b0b0 0x7f107806d570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f109814fda0 0x7f10980b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10980b1f60 0x7f1098150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 --2- 172.21.15.125:0/1616002648 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1080003970 0x7f10981f17f0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 >> 172.21.15.125:0/1616002648 conn(0x7f10980d0d60 msgr2=0x7f109806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:01.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 shutdown_connections 2023-10-18T12:01:01.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.376+0000 7f109cb05640 1 -- 172.21.15.125:0/1616002648 wait complete. 2023-10-18T12:01:01.389 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-18T12:01:01.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.490+0000 7fcd11b4c640 1 -- 172.21.15.125:0/4133717323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccf8003970 msgr2=0x7fccf8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.490+0000 7fcd11b4c640 1 --2- 172.21.15.125:0/4133717323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccf8003970 0x7fccf8003d90 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fccfc001680 tx=0x7fccf80057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.490+0000 7fcd11b4c640 1 -- 172.21.15.125:0/4133717323 shutdown_connections 2023-10-18T12:01:01.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.490+0000 7fcd11b4c640 1 --2- 172.21.15.125:0/4133717323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fccf8003970 0x7fccf8003d90 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.490+0000 7fcd11b4c640 1 -- 172.21.15.125:0/4133717323 >> 172.21.15.125:0/4133717323 conn(0x7fcd0c0d0d60 msgr2=0x7fcd0c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:01.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.490+0000 7fcd11b4c640 1 -- 172.21.15.125:0/4133717323 shutdown_connections 2023-10-18T12:01:01.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.490+0000 7fcd11b4c640 1 -- 172.21.15.125:0/4133717323 wait complete. 2023-10-18T12:01:01.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.491+0000 7fcd11b4c640 1 Processor -- start 2023-10-18T12:01:01.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.491+0000 7fcd11b4c640 1 -- start start 2023-10-18T12:01:01.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd11b4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccf8003970 0x7fcd0c1f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:01.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd11b4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd0c0b3bc0 0x7fcd0c1f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:01.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd11b4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0b4040 0x7fcd0c1f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:01.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd11b4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0af480 con 0x7fcd0c0b4040 2023-10-18T12:01:01.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd11b4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c1f30f0 con 0x7fcd0c0b3bc0 2023-10-18T12:01:01.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd11b4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0b6590 con 0x7fccf8003970 2023-10-18T12:01:01.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0b4040 0x7fcd0c1f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:01.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0b4040 0x7fcd0c1f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54172/0 (socket says 172.21.15.125:54172) 2023-10-18T12:01:01.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 -- 172.21.15.125:0/1377270117 learned_addr learned my addr 172.21.15.125:0/1377270117 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:01.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd02ffd640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd0c0b3bc0 0x7fcd0c1f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:01.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0b7fe640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccf8003970 0x7fcd0c1f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:01.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 -- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccf8003970 msgr2=0x7fcd0c1f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccf8003970 0x7fcd0c1f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 -- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd0c0b3bc0 msgr2=0x7fcd0c1f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:01.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd0c0b3bc0 0x7fcd0c1f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:01.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 -- 172.21.15.125:0/1377270117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0b6870 con 0x7fcd0c0b4040 2023-10-18T12:01:01.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd0bfff640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0b4040 0x7fcd0c1f2bb0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c06c850 tx=0x7fcd0400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:01.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd08ff9640 1 -- 172.21.15.125:0/1377270117 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400cba0 con 0x7fcd0c0b4040 2023-10-18T12:01:01.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd08ff9640 1 -- 172.21.15.125:0/1377270117 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd0400ce30 con 0x7fcd0c0b4040 2023-10-18T12:01:01.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd08ff9640 1 -- 172.21.15.125:0/1377270117 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040116c0 con 0x7fcd0c0b4040 2023-10-18T12:01:01.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c1f1550 con 0x7fcd0c0b4040 2023-10-18T12:01:01.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.492+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c1f1200 con 0x7fcd0c0b4040 2023-10-18T12:01:01.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.494+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0c1f1a70 con 0x7fcd0c0b4040 2023-10-18T12:01:01.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.494+0000 7fcd08ff9640 1 -- 172.21.15.125:0/1377270117 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcd04011820 con 0x7fcd0c0b4040 2023-10-18T12:01:01.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.494+0000 7fcd08ff9640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fccec06b080 0x7fccec06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:01.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.495+0000 7fcd08ff9640 1 -- 172.21.15.125:0/1377270117 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6222+0+0 (secure 0 0 0) 0x7fcd0408a310 con 0x7fcd0c0b4040 2023-10-18T12:01:01.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.495+0000 7fcd0b7fe640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fccec06b080 0x7fccec06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:01.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.495+0000 7fcd0b7fe640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fccec06b080 0x7fccec06d540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fccfc034000 tx=0x7fccfc001190 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:01.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.499+0000 7fcd08ff9640 1 -- 172.21.15.125:0/1377270117 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd0c1f1a70 con 0x7fcd0c0b4040 2023-10-18T12:01:01.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:01.731+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7fcd0c1f1a70 con 0x7fcd0c0b4040 2023-10-18T12:01:02.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.069+0000 7fcd08ff9640 1 -- 172.21.15.125:0/1377270117 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3130) v1 ==== 186+0+0 (secure 0 0 0) 0x7fcd0c1f1a70 con 0x7fcd0c0b4040 2023-10-18T12:01:02.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fccec06b080 msgr2=0x7fccec06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fccec06b080 0x7fccec06d540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fccfc034000 tx=0x7fccfc001190 comp rx=0 tx=0).stop 2023-10-18T12:01:02.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0b4040 msgr2=0x7fcd0c1f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0b4040 0x7fcd0c1f2bb0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c06c850 tx=0x7fcd0400ed10 comp rx=0 tx=0).stop 2023-10-18T12:01:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 shutdown_connections 2023-10-18T12:01:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fccec06b080 0x7fccec06d540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd0c0b3bc0 0x7fcd0c1f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd0c0b4040 0x7fcd0c1f2bb0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 --2- 172.21.15.125:0/1377270117 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fccf8003970 0x7fcd0c1f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 >> 172.21.15.125:0/1377270117 conn(0x7fcd0c0d0d60 msgr2=0x7fcd0c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 shutdown_connections 2023-10-18T12:01:02.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.071+0000 7fcd11b4c640 1 -- 172.21.15.125:0/1377270117 wait complete. 2023-10-18T12:01:02.086 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2023-10-18T12:01:02.087 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2023-10-18T12:01:02.087 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-18T12:01:02.088 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:02.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 -- 172.21.15.125:0/3272639259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500c96b0 msgr2=0x7f10500c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 --2- 172.21.15.125:0/3272639259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500c96b0 0x7f10500c9b20 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f10500cc560 tx=0x7f104802bda0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 -- 172.21.15.125:0/3272639259 shutdown_connections 2023-10-18T12:01:02.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 --2- 172.21.15.125:0/3272639259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500c96b0 0x7f10500c9b20 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 --2- 172.21.15.125:0/3272639259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f105013d350 0x7f10500cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 --2- 172.21.15.125:0/3272639259 >> v2:172.21.15.139:3300/0 conn(0x7f10500cb980 0x7f105013db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 -- 172.21.15.125:0/3272639259 >> 172.21.15.125:0/3272639259 conn(0x7f1050069930 msgr2=0x7f1050069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:02.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 -- 172.21.15.125:0/3272639259 shutdown_connections 2023-10-18T12:01:02.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.190+0000 7f1054b25640 1 -- 172.21.15.125:0/3272639259 wait complete. 2023-10-18T12:01:02.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.191+0000 7f1054b25640 1 Processor -- start 2023-10-18T12:01:02.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.191+0000 7f1054b25640 1 -- start start 2023-10-18T12:01:02.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f1054b25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10500c96b0 0x7f105013f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f1054b25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500cb980 0x7f1050157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f1054b25640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1050140a50 0x7f1050135a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f1054b25640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105014ea90 con 0x7f10500cb980 2023-10-18T12:01:02.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f1054b25640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10501577a0 con 0x7f10500c96b0 2023-10-18T12:01:02.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f1054b25640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1050141010 con 0x7f1050140a50 2023-10-18T12:01:02.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f104dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500cb980 0x7f1050157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f104dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500cb980 0x7f1050157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54184/0 (socket says 172.21.15.125:54184) 2023-10-18T12:01:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.192+0000 7f104dd9b640 1 -- 172.21.15.125:0/1460218488 learned_addr learned my addr 172.21.15.125:0/1460218488 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f104dd9b640 1 -- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1050140a50 msgr2=0x7f1050135a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f104dd9b640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1050140a50 0x7f1050135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f104dd9b640 1 -- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10500c96b0 msgr2=0x7f105013f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f104dd9b640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10500c96b0 0x7f105013f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f104dd9b640 1 -- 172.21.15.125:0/1460218488 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1048009560 con 0x7f10500cb980 2023-10-18T12:01:02.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f104dd9b640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500cb980 0x7f1050157260 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f104400b410 tx=0x7f104400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:02.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f103effd640 1 -- 172.21.15.125:0/1460218488 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044011830 con 0x7f10500cb980 2023-10-18T12:01:02.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f103effd640 1 -- 172.21.15.125:0/1460218488 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1044011aa0 con 0x7f10500cb980 2023-10-18T12:01:02.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105013feb0 con 0x7f10500cb980 2023-10-18T12:01:02.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f103effd640 1 -- 172.21.15.125:0/1460218488 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044010380 con 0x7f10500cb980 2023-10-18T12:01:02.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.193+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1050140470 con 0x7f10500cb980 2023-10-18T12:01:02.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.194+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1050152640 con 0x7f10500cb980 2023-10-18T12:01:02.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.194+0000 7f103effd640 1 -- 172.21.15.125:0/1460218488 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f10440104e0 con 0x7f10500cb980 2023-10-18T12:01:02.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.195+0000 7f103effd640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f103006b080 0x7f103006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.195+0000 7f103effd640 1 -- 172.21.15.125:0/1460218488 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6222+0+0 (secure 0 0 0) 0x7f104408a000 con 0x7f10500cb980 2023-10-18T12:01:02.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.195+0000 7f104e59c640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f103006b080 0x7f103006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.195+0000 7f104e59c640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f103006b080 0x7f103006d540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1040005fd0 tx=0x7f10400071c0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:02.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.199+0000 7f103effd640 1 -- 172.21.15.125:0/1460218488 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1050152640 con 0x7f10500cb980 2023-10-18T12:01:02.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.366+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1050152640 con 0x7f10500cb980 2023-10-18T12:01:02.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.367+0000 7f103effd640 1 -- 172.21.15.125:0/1460218488 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+14676 (secure 0 0 0) 0x7f1050152640 con 0x7f10500cb980 2023-10-18T12:01:02.369 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:02.370 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":233,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:01.061186+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6835/1825099907":"2023-10-19T12:00:58.051673+0000","172.21.15.125:6832/2975172452":"2023-10-19T12:00:51.923831+0000","172.21.15.139:0/1563782559":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/170280857":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6833/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:6832/178515798":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/2119410248":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3691026234":"2023-10-19T12:00:48.811703+0000","172.21.15.125:6834/2975172452":"2023-10-19T12:00:51.923831+0000","172.21.15.139:0/557716760":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/395739661":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/225240329":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1417331710":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6833/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6832/2362843834":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3774276243":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/3836307336":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/2081746818":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/3501758629":"2023-10-19T12:00:48.811703+0000","172.21.15.139:0/1472046677":"2023-10-19T12:00:40.842747+0000","172.21.15.139:6837/1825099907":"2023-10-19T12:00:58.051673+0000","172.21.15.139:0/3403101268":"2023-10-19T12:00:40.842747+0000","172.21.15.139:0/1324329019":"2023-10-19T12:00:40.842747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:02.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f103006b080 msgr2=0x7f103006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f103006b080 0x7f103006d540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1040005fd0 tx=0x7f10400071c0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500cb980 msgr2=0x7f1050157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500cb980 0x7f1050157260 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f104400b410 tx=0x7f104400b8d0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 shutdown_connections 2023-10-18T12:01:02.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f103006b080 0x7f103006d540 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1050140a50 0x7f1050135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10500cb980 0x7f1050157260 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 --2- 172.21.15.125:0/1460218488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10500c96b0 0x7f105013f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 >> 172.21.15.125:0/1460218488 conn(0x7f1050069930 msgr2=0x7f1050070350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:02.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 shutdown_connections 2023-10-18T12:01:02.374 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.369+0000 7f1054b25640 1 -- 172.21.15.125:0/1460218488 wait complete. 2023-10-18T12:01:02.383 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:01:02.384 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:01:02.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.488+0000 7f83652e1640 1 -- 172.21.15.125:0/2153863542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 msgr2=0x7f834c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.488+0000 7f83652e1640 1 --2- 172.21.15.125:0/2153863542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 0x7f834c0047c0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f8348009b80 tx=0x7f834c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.488+0000 7f835f7fe640 1 -- 172.21.15.125:0/2153863542 reap_dead start 2023-10-18T12:01:02.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.488+0000 7f83652e1640 1 -- 172.21.15.125:0/2153863542 shutdown_connections 2023-10-18T12:01:02.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.488+0000 7f83652e1640 1 -- 172.21.15.125:0/2153863542 >> 172.21.15.125:0/2153863542 conn(0x7f83600b3f00 msgr2=0x7f83600b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:02.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.488+0000 7f83652e1640 1 -- 172.21.15.125:0/2153863542 shutdown_connections 2023-10-18T12:01:02.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.488+0000 7f83652e1640 1 -- 172.21.15.125:0/2153863542 wait complete. 2023-10-18T12:01:02.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.489+0000 7f83652e1640 1 Processor -- start 2023-10-18T12:01:02.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.489+0000 7f83652e1640 1 -- start start 2023-10-18T12:01:02.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f83652e1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 0x7f836014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f83652e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83601505e0 0x7f8360150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f83652e1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8360150fa0 0x7f8360151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f83652e1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83600af520 con 0x7f834c003ef0 2023-10-18T12:01:02.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f83652e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836014c2b0 con 0x7f83601505e0 2023-10-18T12:01:02.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f83652e1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836014d550 con 0x7f8360150fa0 2023-10-18T12:01:02.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f835effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 0x7f836014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f835effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 0x7f836014b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54202/0 (socket says 172.21.15.125:54202) 2023-10-18T12:01:02.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f835effd640 1 -- 172.21.15.125:0/2374762231 learned_addr learned my addr 172.21.15.125:0/2374762231 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:02.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f835effd640 1 -- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8360150fa0 msgr2=0x7f8360151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f835effd640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8360150fa0 0x7f8360151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f8357fff640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83601505e0 0x7f8360150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f835effd640 1 -- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83601505e0 msgr2=0x7f8360150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f835effd640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83601505e0 0x7f8360150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.490+0000 7f835effd640 1 -- 172.21.15.125:0/2374762231 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836014d830 con 0x7f834c003ef0 2023-10-18T12:01:02.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.491+0000 7f835effd640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 0x7f836014b160 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f83500119d0 tx=0x7f8350011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:02.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.491+0000 7f83577fe640 1 -- 172.21.15.125:0/2374762231 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500095e0 con 0x7f834c003ef0 2023-10-18T12:01:02.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.491+0000 7f83577fe640 1 -- 172.21.15.125:0/2374762231 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f835000fde0 con 0x7f834c003ef0 2023-10-18T12:01:02.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.491+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836014b6a0 con 0x7f834c003ef0 2023-10-18T12:01:02.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.491+0000 7f83577fe640 1 -- 172.21.15.125:0/2374762231 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350012670 con 0x7f834c003ef0 2023-10-18T12:01:02.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.491+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836014c7a0 con 0x7f834c003ef0 2023-10-18T12:01:02.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.492+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83600690e0 con 0x7f834c003ef0 2023-10-18T12:01:02.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.492+0000 7f83577fe640 1 -- 172.21.15.125:0/2374762231 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f83500127d0 con 0x7f834c003ef0 2023-10-18T12:01:02.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.493+0000 7f83577fe640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f834006b080 0x7f834006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.493+0000 7f83577fe640 1 -- 172.21.15.125:0/2374762231 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6222+0+0 (secure 0 0 0) 0x7f835008a9a0 con 0x7f834c003ef0 2023-10-18T12:01:02.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.493+0000 7f8357fff640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f834006b080 0x7f834006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.494+0000 7f8357fff640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f834006b080 0x7f834006d540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8348036000 tx=0x7f8348035040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:02.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.497+0000 7f83577fe640 1 -- 172.21.15.125:0/2374762231 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83500555a0 con 0x7f834c003ef0 2023-10-18T12:01:02.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.662+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8360154740 con 0x7f834c003ef0 2023-10-18T12:01:02.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.662+0000 7f83577fe640 1 -- 172.21.15.125:0/2374762231 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v233) v1 ==== 93+0+10 (secure 0 0 0) 0x7f835001f070 con 0x7f834c003ef0 2023-10-18T12:01:02.666 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:01:02.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f834006b080 msgr2=0x7f834006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f834006b080 0x7f834006d540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8348036000 tx=0x7f8348035040 comp rx=0 tx=0).stop 2023-10-18T12:01:02.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 msgr2=0x7f836014b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 0x7f836014b160 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f83500119d0 tx=0x7f8350011e90 comp rx=0 tx=0).stop 2023-10-18T12:01:02.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 shutdown_connections 2023-10-18T12:01:02.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f834006b080 0x7f834006d540 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83601505e0 0x7f8360150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8360150fa0 0x7f8360151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 --2- 172.21.15.125:0/2374762231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f834c003ef0 0x7f836014b160 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 >> 172.21.15.125:0/2374762231 conn(0x7f83600b3f00 msgr2=0x7f836015af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:02.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 shutdown_connections 2023-10-18T12:01:02.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.664+0000 7f83652e1640 1 -- 172.21.15.125:0/2374762231 wait complete. 2023-10-18T12:01:02.678 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-18T12:01:02.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.778+0000 7fb81ceda640 1 -- 172.21.15.125:0/689997471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8180ceb40 msgr2=0x7fb8180b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.778+0000 7fb81ceda640 1 --2- 172.21.15.125:0/689997471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8180ceb40 0x7fb8180b0650 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb810005fd0 tx=0x7fb81002bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:02.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.778+0000 7fb81ceda640 1 -- 172.21.15.125:0/689997471 shutdown_connections 2023-10-18T12:01:02.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.778+0000 7fb81ceda640 1 --2- 172.21.15.125:0/689997471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8180ceb40 0x7fb8180b0650 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.778+0000 7fb81ceda640 1 --2- 172.21.15.125:0/689997471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb81815b2a0 0x7fb8180d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.778+0000 7fb81ceda640 1 -- 172.21.15.125:0/689997471 >> 172.21.15.125:0/689997471 conn(0x7fb8180d0d60 msgr2=0x7fb8180d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:02.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.778+0000 7fb81ceda640 1 -- 172.21.15.125:0/689997471 shutdown_connections 2023-10-18T12:01:02.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.778+0000 7fb81ceda640 1 -- 172.21.15.125:0/689997471 wait complete. 2023-10-18T12:01:02.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.779+0000 7fb81ceda640 1 Processor -- start 2023-10-18T12:01:02.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.779+0000 7fb81ceda640 1 -- start start 2023-10-18T12:01:02.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb81ceda640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8180ceb40 0x7fb818150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb81ceda640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8180cf340 0x7fb8180b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb81ceda640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb81815b2a0 0x7fb8180b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb81ceda640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818150a50 con 0x7fb8180cf340 2023-10-18T12:01:02.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb81ceda640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81815dba0 con 0x7fb81815b2a0 2023-10-18T12:01:02.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb81ceda640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81815de80 con 0x7fb8180ceb40 2023-10-18T12:01:02.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8180cf340 0x7fb8180b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8180cf340 0x7fb8180b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54206/0 (socket says 172.21.15.125:54206) 2023-10-18T12:01:02.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 -- 172.21.15.125:0/3362624200 learned_addr learned my addr 172.21.15.125:0/3362624200 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:02.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb81659c640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8180ceb40 0x7fb818150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb816d9d640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb81815b2a0 0x7fb8180b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 -- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8180ceb40 msgr2=0x7fb818150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8180ceb40 0x7fb818150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 -- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb81815b2a0 msgr2=0x7fb8180b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:02.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb81815b2a0 0x7fb8180b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:02.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 -- 172.21.15.125:0/3362624200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c009490 con 0x7fb8180cf340 2023-10-18T12:01:02.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb815d9b640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8180cf340 0x7fb8180b0ee0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e9f0 tx=0x7fb80c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:02.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.780+0000 7fb806ffd640 1 -- 172.21.15.125:0/3362624200 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00cd80 con 0x7fb8180cf340 2023-10-18T12:01:02.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.781+0000 7fb806ffd640 1 -- 172.21.15.125:0/3362624200 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb80c004020 con 0x7fb8180cf340 2023-10-18T12:01:02.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.781+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb810009560 con 0x7fb8180cf340 2023-10-18T12:01:02.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.781+0000 7fb806ffd640 1 -- 172.21.15.125:0/3362624200 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00cee0 con 0x7fb8180cf340 2023-10-18T12:01:02.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.781+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8180b0d50 con 0x7fb8180cf340 2023-10-18T12:01:02.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.782+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8180690e0 con 0x7fb8180cf340 2023-10-18T12:01:02.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.782+0000 7fb806ffd640 1 -- 172.21.15.125:0/3362624200 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb80c0035f0 con 0x7fb8180cf340 2023-10-18T12:01:02.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.782+0000 7fb806ffd640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb7f806b0b0 0x7fb7f806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:02.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.782+0000 7fb806ffd640 1 -- 172.21.15.125:0/3362624200 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6222+0+0 (secure 0 0 0) 0x7fb80c014070 con 0x7fb8180cf340 2023-10-18T12:01:02.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.782+0000 7fb81659c640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb7f806b0b0 0x7fb7f806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:02.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.783+0000 7fb81659c640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb7f806b0b0 0x7fb7f806d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb808001f10 tx=0x7fb808009040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:02.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:02.786+0000 7fb806ffd640 1 -- 172.21.15.125:0/3362624200 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb80c055240 con 0x7fb8180cf340 2023-10-18T12:01:03.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.026+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7fb8180c76c0 con 0x7fb8180cf340 2023-10-18T12:01:03.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.074+0000 7fb806ffd640 1 -- 172.21.15.125:0/3362624200 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3131) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb80c058860 con 0x7fb8180cf340 2023-10-18T12:01:03.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb7f806b0b0 msgr2=0x7fb7f806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb7f806b0b0 0x7fb7f806d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb808001f10 tx=0x7fb808009040 comp rx=0 tx=0).stop 2023-10-18T12:01:03.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8180cf340 msgr2=0x7fb8180b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8180cf340 0x7fb8180b0ee0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e9f0 tx=0x7fb80c00eeb0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 shutdown_connections 2023-10-18T12:01:03.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb7f806b0b0 0x7fb7f806d570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb81815b2a0 0x7fb8180b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb8180cf340 0x7fb8180b0ee0 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 --2- 172.21.15.125:0/3362624200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb8180ceb40 0x7fb818150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 >> 172.21.15.125:0/3362624200 conn(0x7fb8180d0d60 msgr2=0x7fb81815a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:03.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 shutdown_connections 2023-10-18T12:01:03.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.076+0000 7fb81ceda640 1 -- 172.21.15.125:0/3362624200 wait complete. 2023-10-18T12:01:03.089 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T12:01:03.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.189+0000 7fa33d524640 1 -- 172.21.15.125:0/3144407030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380ceb40 msgr2=0x7fa3380b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.189+0000 7fa33d524640 1 --2- 172.21.15.125:0/3144407030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380ceb40 0x7fa3380b0650 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fa330005fd0 tx=0x7fa33002c240 comp rx=0 tx=0).stop 2023-10-18T12:01:03.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.189+0000 7fa33d524640 1 -- 172.21.15.125:0/3144407030 shutdown_connections 2023-10-18T12:01:03.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.189+0000 7fa33d524640 1 --2- 172.21.15.125:0/3144407030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380ceb40 0x7fa3380b0650 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.189+0000 7fa33d524640 1 --2- 172.21.15.125:0/3144407030 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa33815b2a0 0x7fa3380d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.189+0000 7fa33d524640 1 -- 172.21.15.125:0/3144407030 >> 172.21.15.125:0/3144407030 conn(0x7fa3380d0d60 msgr2=0x7fa3380d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:03.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.189+0000 7fa33d524640 1 -- 172.21.15.125:0/3144407030 shutdown_connections 2023-10-18T12:01:03.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.189+0000 7fa33d524640 1 -- 172.21.15.125:0/3144407030 wait complete. 2023-10-18T12:01:03.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.190+0000 7fa33d524640 1 Processor -- start 2023-10-18T12:01:03.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.190+0000 7fa33d524640 1 -- start start 2023-10-18T12:01:03.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.190+0000 7fa33d524640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3380ceb40 0x7fa3380b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:03.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa33d524640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380cf340 0x7fa3381577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:03.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa33d524640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa33815b2a0 0x7fa3380c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:03.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa33d524640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa338158910 con 0x7fa3380cf340 2023-10-18T12:01:03.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa33d524640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3380b63a0 con 0x7fa3380ceb40 2023-10-18T12:01:03.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa33d524640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3380b3a80 con 0x7fa33815b2a0 2023-10-18T12:01:03.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380cf340 0x7fa3381577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:03.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380cf340 0x7fa3381577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54234/0 (socket says 172.21.15.125:54234) 2023-10-18T12:01:03.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 -- 172.21.15.125:0/4231906297 learned_addr learned my addr 172.21.15.125:0/4231906297 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:03.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa3377fe640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa33815b2a0 0x7fa3380c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:03.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa336ffd640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3380ceb40 0x7fa3380b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:03.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 -- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa33815b2a0 msgr2=0x7fa3380c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa33815b2a0 0x7fa3380c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 -- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3380ceb40 msgr2=0x7fa3380b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3380ceb40 0x7fa3380b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 -- 172.21.15.125:0/4231906297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa330009900 con 0x7fa3380cf340 2023-10-18T12:01:03.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32ffff640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380cf340 0x7fa3381577b0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fa32000b410 tx=0x7fa32000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:03.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32f7fe640 1 -- 172.21.15.125:0/4231906297 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320011830 con 0x7fa3380cf340 2023-10-18T12:01:03.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32f7fe640 1 -- 172.21.15.125:0/4231906297 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa320011aa0 con 0x7fa3380cf340 2023-10-18T12:01:03.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3380b3d60 con 0x7fa3380cf340 2023-10-18T12:01:03.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa32f7fe640 1 -- 172.21.15.125:0/4231906297 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320010380 con 0x7fa3380cf340 2023-10-18T12:01:03.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.191+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3380b4320 con 0x7fa3380cf340 2023-10-18T12:01:03.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.192+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3380690e0 con 0x7fa3380cf340 2023-10-18T12:01:03.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.193+0000 7fa32f7fe640 1 -- 172.21.15.125:0/4231906297 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa3200104e0 con 0x7fa3380cf340 2023-10-18T12:01:03.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.193+0000 7fa32f7fe640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa31806b080 0x7fa31806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:03.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.193+0000 7fa32f7fe640 1 -- 172.21.15.125:0/4231906297 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6222+0+0 (secure 0 0 0) 0x7fa32008a050 con 0x7fa3380cf340 2023-10-18T12:01:03.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.193+0000 7fa336ffd640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa31806b080 0x7fa31806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:03.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.194+0000 7fa336ffd640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa31806b080 0x7fa31806d540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa328000f60 tx=0x7fa328009040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:03.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.197+0000 7fa32f7fe640 1 -- 172.21.15.125:0/4231906297 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa320054ba0 con 0x7fa3380cf340 2023-10-18T12:01:03.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.364+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fa3380cb980 con 0x7fa3380cf340 2023-10-18T12:01:03.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.364+0000 7fa32f7fe640 1 -- 172.21.15.125:0/4231906297 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v78) v1 ==== 59+0+2220 (secure 0 0 0) 0x7fa3200158c0 con 0x7fa3380cf340 2023-10-18T12:01:03.403 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T12:01:03.404 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T12:01:03.404 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T12:01:03.404 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T12:01:03.404 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T12:01:03.404 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T12:01:03.405 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T12:01:03.405 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T12:01:03.405 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T12:01:03.405 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T12:01:03.405 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T12:01:03.405 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T12:01:03.406 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T12:01:03.406 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T12:01:03.406 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T12:01:03.406 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T12:01:03.407 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T12:01:03.407 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T12:01:03.407 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T12:01:03.407 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T12:01:03.407 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T12:01:03.407 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T12:01:03.408 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T12:01:03.408 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T12:01:03.408 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T12:01:03.408 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T12:01:03.408 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T12:01:03.409 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T12:01:03.409 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T12:01:03.409 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T12:01:03.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa31806b080 msgr2=0x7fa31806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa31806b080 0x7fa31806d540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa328000f60 tx=0x7fa328009040 comp rx=0 tx=0).stop 2023-10-18T12:01:03.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380cf340 msgr2=0x7fa3381577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380cf340 0x7fa3381577b0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fa32000b410 tx=0x7fa32000b8d0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 shutdown_connections 2023-10-18T12:01:03.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa31806b080 0x7fa31806d540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa33815b2a0 0x7fa3380c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3380cf340 0x7fa3381577b0 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 --2- 172.21.15.125:0/4231906297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3380ceb40 0x7fa3380b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 >> 172.21.15.125:0/4231906297 conn(0x7fa3380d0d60 msgr2=0x7fa33806a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:03.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 shutdown_connections 2023-10-18T12:01:03.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.366+0000 7fa33d524640 1 -- 172.21.15.125:0/4231906297 wait complete. 2023-10-18T12:01:03.413 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T12:01:03.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.516+0000 7f3cb86a9640 1 -- 172.21.15.125:0/4244412868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9c004a80 msgr2=0x7f3c9c004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.516+0000 7f3cb86a9640 1 --2- 172.21.15.125:0/4244412868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c9c004a80 0x7f3c9c004ea0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00bb50 tx=0x7f3c9c006900 comp rx=0 tx=0).stop 2023-10-18T12:01:03.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.516+0000 7f3cb6c46640 1 -- 172.21.15.125:0/4244412868 reap_dead start 2023-10-18T12:01:03.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.516+0000 7f3cb86a9640 1 -- 172.21.15.125:0/4244412868 shutdown_connections 2023-10-18T12:01:03.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.516+0000 7f3cb86a9640 1 -- 172.21.15.125:0/4244412868 >> 172.21.15.125:0/4244412868 conn(0x7f3cb0069930 msgr2=0x7f3cb0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:03.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.516+0000 7f3cb86a9640 1 -- 172.21.15.125:0/4244412868 shutdown_connections 2023-10-18T12:01:03.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.517+0000 7f3cb86a9640 1 -- 172.21.15.125:0/4244412868 wait complete. 2023-10-18T12:01:03.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb86a9640 1 Processor -- start 2023-10-18T12:01:03.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb86a9640 1 -- start start 2023-10-18T12:01:03.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb86a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9c004a80 0x7f3cb0141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:03.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb86a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0153b20 0x7f3cb0153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:03.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb86a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb0155b30 0x7f3cb0155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:03.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb86a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0157430 con 0x7f3cb0153b20 2023-10-18T12:01:03.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb86a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0153280 con 0x7f3c9c004a80 2023-10-18T12:01:03.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb86a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01544e0 con 0x7f3cb0155b30 2023-10-18T12:01:03.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb5c44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0153b20 0x7f3cb0153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:03.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb5c44640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0153b20 0x7f3cb0153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54254/0 (socket says 172.21.15.125:54254) 2023-10-18T12:01:03.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.518+0000 7f3cb5c44640 1 -- 172.21.15.125:0/2930558805 learned_addr learned my addr 172.21.15.125:0/2930558805 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:03.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb6c46640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb0155b30 0x7f3cb0155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:03.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb5c44640 1 -- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb0155b30 msgr2=0x7f3cb0155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb6445640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9c004a80 0x7f3cb0141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:03.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb5c44640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb0155b30 0x7f3cb0155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb5c44640 1 -- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9c004a80 msgr2=0x7f3cb0141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb5c44640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9c004a80 0x7f3cb0141210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb5c44640 1 -- 172.21.15.125:0/2930558805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb01568b0 con 0x7f3cb0153b20 2023-10-18T12:01:03.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb5c44640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0153b20 0x7f3cb0153fa0 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0006680 tx=0x7f3ca00066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:03.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3ca6ffd640 1 -- 172.21.15.125:0/2930558805 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011900 con 0x7f3cb0153b20 2023-10-18T12:01:03.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3ca6ffd640 1 -- 172.21.15.125:0/2930558805 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ca0011a80 con 0x7f3cb0153b20 2023-10-18T12:01:03.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb0156b90 con 0x7f3cb0153b20 2023-10-18T12:01:03.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3ca6ffd640 1 -- 172.21.15.125:0/2930558805 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011e90 con 0x7f3cb0153b20 2023-10-18T12:01:03.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.519+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb0157150 con 0x7f3cb0153b20 2023-10-18T12:01:03.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.520+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb0151780 con 0x7f3cb0153b20 2023-10-18T12:01:03.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.521+0000 7f3ca6ffd640 1 -- 172.21.15.125:0/2930558805 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3ca0010420 con 0x7f3cb0153b20 2023-10-18T12:01:03.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.521+0000 7f3ca6ffd640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3c9406b080 0x7f3c9406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:03.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.521+0000 7f3ca6ffd640 1 -- 172.21.15.125:0/2930558805 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6222+0+0 (secure 0 0 0) 0x7f3ca008aee0 con 0x7f3cb0153b20 2023-10-18T12:01:03.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.521+0000 7f3cb6445640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3c9406b080 0x7f3c9406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:03.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.522+0000 7f3cb6445640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3c9406b080 0x7f3c9406d540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3cac037000 tx=0x7f3cac02e040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:03.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.525+0000 7f3ca6ffd640 1 -- 172.21.15.125:0/2930558805 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ca0055920 con 0x7f3cb0153b20 2023-10-18T12:01:03.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.690+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3cb00690e0 con 0x7f3cb0153b20 2023-10-18T12:01:03.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.693+0000 7f3ca6ffd640 1 -- 172.21.15.125:0/2930558805 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v79)=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3ca0015070 con 0x7f3cb0153b20 2023-10-18T12:01:03.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3c9406b080 msgr2=0x7f3c9406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3c9406b080 0x7f3c9406d540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3cac037000 tx=0x7f3cac02e040 comp rx=0 tx=0).stop 2023-10-18T12:01:03.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0153b20 msgr2=0x7f3cb0153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:03.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0153b20 0x7f3cb0153fa0 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0006680 tx=0x7f3ca00066b0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 shutdown_connections 2023-10-18T12:01:03.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3c9406b080 0x7f3c9406d540 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cb0155b30 0x7f3cb0155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3cb0153b20 0x7f3cb0153fa0 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 --2- 172.21.15.125:0/2930558805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9c004a80 0x7f3cb0141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:03.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.695+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 >> 172.21.15.125:0/2930558805 conn(0x7f3cb0069930 msgr2=0x7f3cb006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:03.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.696+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 shutdown_connections 2023-10-18T12:01:03.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:03.696+0000 7f3cb86a9640 1 -- 172.21.15.125:0/2930558805 wait complete. 2023-10-18T12:01:03.708 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T12:01:03.709 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:01:03.755 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:01:03.800 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T12:01:03.801 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:01:03.801 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:03.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:01:03.816 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:01:03.817 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:03.834 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:01:03.834 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:01:03.835 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:03.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:01:03.872 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:01:03.872 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:03.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:01:03.891 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T12:01:03.892 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:04.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 -- 172.21.15.125:0/783100503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0780ab500 msgr2=0x7fb0780ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:04.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 --2- 172.21.15.125:0/783100503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0780ab500 0x7fb0780ab990 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7fb0780cbac0 tx=0x7fb07002c250 comp rx=0 tx=0).stop 2023-10-18T12:01:04.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 -- 172.21.15.125:0/783100503 shutdown_connections 2023-10-18T12:01:04.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 --2- 172.21.15.125:0/783100503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0780ab500 0x7fb0780ab990 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 --2- 172.21.15.125:0/783100503 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0780c96b0 0x7fb0780c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 --2- 172.21.15.125:0/783100503 >> v2:172.21.15.139:3300/0 conn(0x7fb07806af00 0x7fb07806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 -- 172.21.15.125:0/783100503 >> 172.21.15.125:0/783100503 conn(0x7fb078069930 msgr2=0x7fb078069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:04.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 -- 172.21.15.125:0/783100503 shutdown_connections 2023-10-18T12:01:04.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.011+0000 7fb080607640 1 -- 172.21.15.125:0/783100503 wait complete. 2023-10-18T12:01:04.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.012+0000 7fb080607640 1 Processor -- start 2023-10-18T12:01:04.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.012+0000 7fb080607640 1 -- start start 2023-10-18T12:01:04.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb080607640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb07806af00 0x7fb07812bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:04.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb080607640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0780ab500 0x7fb0781eee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:04.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb080607640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0780c96b0 0x7fb0780aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:04.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb080607640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0780ab2e0 con 0x7fb07806af00 2023-10-18T12:01:04.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb080607640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07812c340 con 0x7fb0780c96b0 2023-10-18T12:01:04.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb080607640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07812c620 con 0x7fb0780ab500 2023-10-18T12:01:04.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb07e3a3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb07806af00 0x7fb07812bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:04.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb07e3a3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb07806af00 0x7fb07812bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54278/0 (socket says 172.21.15.125:54278) 2023-10-18T12:01:04.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb07e3a3640 1 -- 172.21.15.125:0/143071588 learned_addr learned my addr 172.21.15.125:0/143071588 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:04.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb07e3a3640 1 -- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0780ab500 msgr2=0x7fb0781eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:04.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb07e3a3640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0780ab500 0x7fb0781eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb07e3a3640 1 -- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0780c96b0 msgr2=0x7fb0780aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:04.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb07e3a3640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0780c96b0 0x7fb0780aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.013+0000 7fb07e3a3640 1 -- 172.21.15.125:0/143071588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb070009900 con 0x7fb07806af00 2023-10-18T12:01:04.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.014+0000 7fb07e3a3640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb07806af00 0x7fb07812bd60 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b4a0 tx=0x7fb06800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:04.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.014+0000 7fb06effd640 1 -- 172.21.15.125:0/143071588 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068010610 con 0x7fb07806af00 2023-10-18T12:01:04.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.014+0000 7fb06effd640 1 -- 172.21.15.125:0/143071588 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb068010860 con 0x7fb07806af00 2023-10-18T12:01:04.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.014+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07812c900 con 0x7fb07806af00 2023-10-18T12:01:04.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.014+0000 7fb06effd640 1 -- 172.21.15.125:0/143071588 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068010c70 con 0x7fb07806af00 2023-10-18T12:01:04.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.014+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07812cec0 con 0x7fb07806af00 2023-10-18T12:01:04.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.015+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07812d0e0 con 0x7fb07806af00 2023-10-18T12:01:04.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.015+0000 7fb06effd640 1 -- 172.21.15.125:0/143071588 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb068017360 con 0x7fb07806af00 2023-10-18T12:01:04.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.016+0000 7fb06effd640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb05c06b080 0x7fb05c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:04.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.016+0000 7fb06effd640 1 -- 172.21.15.125:0/143071588 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6222+0+0 (secure 0 0 0) 0x7fb068089af0 con 0x7fb07806af00 2023-10-18T12:01:04.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.016+0000 7fb07dba2640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb05c06b080 0x7fb05c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:04.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.017+0000 7fb07dba2640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb05c06b080 0x7fb05c06d540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb0781f2940 tx=0x7fb074006bf0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:04.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.020+0000 7fb06effd640 1 -- 172.21.15.125:0/143071588 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0680580d0 con 0x7fb07806af00 2023-10-18T12:01:04.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.224+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0780ac470 con 0x7fb07806af00 2023-10-18T12:01:04.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.224+0000 7fb06effd640 1 -- 172.21.15.125:0/143071588 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fb06801ee90 con 0x7fb07806af00 2023-10-18T12:01:04.227 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:04.228 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8951,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/2304539989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":2304539989},{"type":"v1","addr":"172.21.15.125:6835","nonce":2304539989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10165,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/675777912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":675777912},{"type":"v1","addr":"172.21.15.139:6836","nonce":675777912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":12609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1669677937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1669677937},{"type":"v1","addr":"172.21.15.125:6834","nonce":1669677937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":12699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3517995226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3517995226},{"type":"v1","addr":"172.21.15.139:6837","nonce":3517995226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[]} 2023-10-18T12:01:04.228 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 94 2023-10-18T12:01:04.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb05c06b080 msgr2=0x7fb05c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:04.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb05c06b080 0x7fb05c06d540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb0781f2940 tx=0x7fb074006bf0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb07806af00 msgr2=0x7fb07812bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:04.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb07806af00 0x7fb07812bd60 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b4a0 tx=0x7fb06800b960 comp rx=0 tx=0).stop 2023-10-18T12:01:04.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 shutdown_connections 2023-10-18T12:01:04.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb05c06b080 0x7fb05c06d540 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0780c96b0 0x7fb0780aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0780ab500 0x7fb0781eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 --2- 172.21.15.125:0/143071588 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb07806af00 0x7fb07812bd60 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 >> 172.21.15.125:0/143071588 conn(0x7fb078069930 msgr2=0x7fb078069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:04.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 shutdown_connections 2023-10-18T12:01:04.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.226+0000 7fb080607640 1 -- 172.21.15.125:0/143071588 wait complete. 2023-10-18T12:01:04.239 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T12:01:04.240 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T12:01:04.240 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T12:01:04.240 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:01:04.241 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T12:01:04.241 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T12:01:04.241 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T12:01:04.241 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:01:04.242 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T12:01:04.242 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T12:01:04.242 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T12:01:04.243 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:01:04.243 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T12:01:04.243 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T12:01:04.243 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T12:01:04.244 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:01:04.249 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T12:01:04.249 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T12:01:04.250 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T12:01:04.251 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T12:01:04.256 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T12:01:04.256 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T12:01:04.257 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T12:01:04.257 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T12:01:04.292 INFO:tasks.ceph.mds.a:Started 2023-10-18T12:01:04.292 DEBUG:teuthology.parallel:result is None 2023-10-18T12:01:04.293 INFO:tasks.ceph.mds.c:Started 2023-10-18T12:01:04.293 DEBUG:teuthology.parallel:result is None 2023-10-18T12:01:04.298 INFO:tasks.ceph.mds.b:Started 2023-10-18T12:01:04.299 DEBUG:teuthology.parallel:result is None 2023-10-18T12:01:04.299 INFO:tasks.ceph.mds.d:Started 2023-10-18T12:01:04.300 DEBUG:teuthology.parallel:result is None 2023-10-18T12:01:04.300 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T12:01:04.366 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T12:01:04.367 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T12:01:04.377 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T12:01:04.377 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T12:01:04.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.405+0000 7f921f105640 1 -- 172.21.15.125:0/184374898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180cbd20 msgr2=0x7f9218158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:04.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.405+0000 7f921f105640 1 --2- 172.21.15.125:0/184374898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180cbd20 0x7f9218158300 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f92180b10f0 tx=0x7f920c02bd70 comp rx=0 tx=0).stop 2023-10-18T12:01:04.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.405+0000 7f921f105640 1 -- 172.21.15.125:0/184374898 shutdown_connections 2023-10-18T12:01:04.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.405+0000 7f921f105640 1 --2- 172.21.15.125:0/184374898 >> v2:172.21.15.139:3300/0 conn(0x7f92180b0530 0x7f92180b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.405+0000 7f921f105640 1 --2- 172.21.15.125:0/184374898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180cbd20 0x7f9218158300 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.409 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.405+0000 7f921f105640 1 -- 172.21.15.125:0/184374898 >> 172.21.15.125:0/184374898 conn(0x7f9218069930 msgr2=0x7f9218069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:04.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.406+0000 7f921f105640 1 -- 172.21.15.125:0/184374898 shutdown_connections 2023-10-18T12:01:04.410 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.406+0000 7f921f105640 1 -- 172.21.15.125:0/184374898 wait complete. 2023-10-18T12:01:04.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.407+0000 7f921f105640 1 Processor -- start 2023-10-18T12:01:04.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.407+0000 7f921f105640 1 -- start start 2023-10-18T12:01:04.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.407+0000 7f921f105640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92180afa20 0x7f92180ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:04.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.407+0000 7f921f105640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180b0530 0x7f9218156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:04.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.407+0000 7f921f105640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92180cbd20 0x7f9218151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:04.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.407+0000 7f921f105640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180ae1d0 con 0x7f92180b0530 2023-10-18T12:01:04.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.407+0000 7f921f105640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218135070 con 0x7f92180cbd20 2023-10-18T12:01:04.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.407+0000 7f921f105640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92180ad720 con 0x7f92180afa20 2023-10-18T12:01:04.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180b0530 0x7f9218156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:04.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180b0530 0x7f9218156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54358/0 (socket says 172.21.15.125:54358) 2023-10-18T12:01:04.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 -- 172.21.15.125:0/3070772290 learned_addr learned my addr 172.21.15.125:0/3070772290 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:04.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f921d6a2640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92180cbd20 0x7f9218151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:04.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f921cea1640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92180afa20 0x7f92180ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:04.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 -- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92180afa20 msgr2=0x7f92180ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:04.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92180afa20 0x7f92180ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.415 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 -- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92180cbd20 msgr2=0x7f9218151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:04.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92180cbd20 0x7f9218151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:04.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 -- 172.21.15.125:0/3070772290 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c009490 con 0x7f92180b0530 2023-10-18T12:01:04.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f9217fff640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180b0530 0x7f9218156840 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f920400c490 tx=0x7f920400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:04.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f92157fa640 1 -- 172.21.15.125:0/3070772290 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204012830 con 0x7f92180b0530 2023-10-18T12:01:04.416 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f92157fa640 1 -- 172.21.15.125:0/3070772290 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9204012a80 con 0x7f92180b0530 2023-10-18T12:01:04.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f92157fa640 1 -- 172.21.15.125:0/3070772290 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204011380 con 0x7f92180b0530 2023-10-18T12:01:04.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92180ad930 con 0x7f92180b0530 2023-10-18T12:01:04.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.408+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92180adef0 con 0x7f92180b0530 2023-10-18T12:01:04.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.409+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9218134dd0 con 0x7f92180b0530 2023-10-18T12:01:04.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.410+0000 7f92157fa640 1 -- 172.21.15.125:0/3070772290 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f92040114e0 con 0x7f92180b0530 2023-10-18T12:01:04.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.410+0000 7f92157fa640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f91fc06b080 0x7f91fc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:04.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.410+0000 7f92157fa640 1 -- 172.21.15.125:0/3070772290 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6222+0+0 (secure 0 0 0) 0x7f9204089890 con 0x7f92180b0530 2023-10-18T12:01:04.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.410+0000 7f921cea1640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f91fc06b080 0x7f91fc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:04.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.411+0000 7f921cea1640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f91fc06b080 0x7f91fc06d540 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f920c036000 tx=0x7f920c004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:04.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.414+0000 7f92157fa640 1 -- 172.21.15.125:0/3070772290 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9218134dd0 con 0x7f92180b0530 2023-10-18T12:01:04.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:04.598+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f92181351b0 con 0x7f92180b0530 2023-10-18T12:01:05.408 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.405+0000 7f92157fa640 1 -- 172.21.15.125:0/3070772290 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v238) v1 ==== 97+0+0 (secure 0 0 0) 0x7f92181351b0 con 0x7f92180b0530 2023-10-18T12:01:05.408 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T12:01:05.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f91fc06b080 msgr2=0x7f91fc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f91fc06b080 0x7f91fc06d540 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f920c036000 tx=0x7f920c004730 comp rx=0 tx=0).stop 2023-10-18T12:01:05.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180b0530 msgr2=0x7f9218156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.411 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180b0530 0x7f9218156840 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f920400c490 tx=0x7f920400c950 comp rx=0 tx=0).stop 2023-10-18T12:01:05.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 shutdown_connections 2023-10-18T12:01:05.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f91fc06b080 0x7f91fc06d540 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92180cbd20 0x7f9218151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92180b0530 0x7f9218156840 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 --2- 172.21.15.125:0/3070772290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92180afa20 0x7f92180ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 >> 172.21.15.125:0/3070772290 conn(0x7f9218069930 msgr2=0x7f92180ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:05.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 shutdown_connections 2023-10-18T12:01:05.413 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.408+0000 7f921f105640 1 -- 172.21.15.125:0/3070772290 wait complete. 2023-10-18T12:01:05.423 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T12:01:05.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.528+0000 7faf3346f640 1 -- 172.21.15.125:0/686532998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c070350 msgr2=0x7faf2c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.528+0000 7faf3346f640 1 --2- 172.21.15.125:0/686532998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c070350 0x7faf2c0707c0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7faf2c06a7e0 tx=0x7faf2402c260 comp rx=0 tx=0).stop 2023-10-18T12:01:05.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.528+0000 7faf3346f640 1 -- 172.21.15.125:0/686532998 shutdown_connections 2023-10-18T12:01:05.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.528+0000 7faf3346f640 1 --2- 172.21.15.125:0/686532998 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c070350 0x7faf2c0707c0 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.528+0000 7faf3346f640 1 --2- 172.21.15.125:0/686532998 >> v2:172.21.15.139:3300/0 conn(0x7faf2c0cb980 0x7faf2c13db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.528+0000 7faf3346f640 1 -- 172.21.15.125:0/686532998 >> 172.21.15.125:0/686532998 conn(0x7faf2c069930 msgr2=0x7faf2c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:05.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.528+0000 7faf3346f640 1 -- 172.21.15.125:0/686532998 shutdown_connections 2023-10-18T12:01:05.532 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.528+0000 7faf3346f640 1 -- 172.21.15.125:0/686532998 wait complete. 2023-10-18T12:01:05.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.529+0000 7faf3346f640 1 Processor -- start 2023-10-18T12:01:05.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.529+0000 7faf3346f640 1 -- start start 2023-10-18T12:01:05.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.529+0000 7faf3346f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf2c070350 0x7faf2c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:05.535 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.529+0000 7faf3346f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c0cb980 0x7faf2c1593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:05.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf3346f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf2c159920 0x7faf2c1f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:05.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf3346f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0b0250 con 0x7faf2c0cb980 2023-10-18T12:01:05.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf3346f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c159e10 con 0x7faf2c070350 2023-10-18T12:01:05.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf3346f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c15a0f0 con 0x7faf2c159920 2023-10-18T12:01:05.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c0cb980 0x7faf2c1593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:05.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c0cb980 0x7faf2c1593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54376/0 (socket says 172.21.15.125:54376) 2023-10-18T12:01:05.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 -- 172.21.15.125:0/3998775294 learned_addr learned my addr 172.21.15.125:0/3998775294 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:05.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf3120b640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf2c070350 0x7faf2c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:05.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf31a0c640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf2c159920 0x7faf2c1f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:05.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 -- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf2c159920 msgr2=0x7faf2c1f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf2c159920 0x7faf2c1f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 -- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf2c070350 msgr2=0x7faf2c0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf2c070350 0x7faf2c0caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 -- 172.21.15.125:0/3998775294 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24009900 con 0x7faf2c0cb980 2023-10-18T12:01:05.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf30a0a640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c0cb980 0x7faf2c1593e0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00b3f0 tx=0x7faf1c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:05.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf21ffb640 1 -- 172.21.15.125:0/3998775294 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c011830 con 0x7faf2c0cb980 2023-10-18T12:01:05.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf21ffb640 1 -- 172.21.15.125:0/3998775294 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf1c011a80 con 0x7faf2c0cb980 2023-10-18T12:01:05.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf21ffb640 1 -- 172.21.15.125:0/3998775294 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c011e90 con 0x7faf2c0cb980 2023-10-18T12:01:05.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.530+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c15a3d0 con 0x7faf2c0cb980 2023-10-18T12:01:05.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.531+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c0cb020 con 0x7faf2c0cb980 2023-10-18T12:01:05.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.531+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c0b03c0 con 0x7faf2c0cb980 2023-10-18T12:01:05.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.532+0000 7faf21ffb640 1 -- 172.21.15.125:0/3998775294 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7faf1c01a400 con 0x7faf2c0cb980 2023-10-18T12:01:05.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.532+0000 7faf21ffb640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7faf1006b080 0x7faf1006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:05.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.532+0000 7faf21ffb640 1 -- 172.21.15.125:0/3998775294 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 5233+0+0 (secure 0 0 0) 0x7faf1c0897e0 con 0x7faf2c0cb980 2023-10-18T12:01:05.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.532+0000 7faf3120b640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7faf1006b080 0x7faf1006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:05.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.533+0000 7faf3120b640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7faf1006b080 0x7faf1006d540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf28005fd0 tx=0x7faf2800b390 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:05.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.536+0000 7faf21ffb640 1 -- 172.21.15.125:0/3998775294 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf2c0b03c0 con 0x7faf2c0cb980 2023-10-18T12:01:05.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.763+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faf2c0b0390 con 0x7faf2c0cb980 2023-10-18T12:01:05.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.763+0000 7faf21ffb640 1 -- 172.21.15.125:0/3998775294 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v58) v1 ==== 80+0+5304 (secure 0 0 0) 0x7faf1c015090 con 0x7faf2c0cb980 2023-10-18T12:01:05.767 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:05.767 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:01:05.767 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T12:01:05.767 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.767 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T12:01:05.768 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T12:01:05.768 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.768 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:05.768 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:05.768 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:05.768 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.769 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.769 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.769 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.769 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T12:01:05.769 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T12:01:05.770 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.770 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:05.770 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:05.770 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:05.770 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.771 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.771 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.771 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.771 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T12:01:05.771 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T12:01:05.772 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.772 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:05.772 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:05.772 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:05.772 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.773 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.773 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.773 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.773 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T12:01:05.773 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T12:01:05.774 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.774 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:05.774 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:05.774 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:05.774 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.775 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.775 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.775 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.775 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T12:01:05.775 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T12:01:05.776 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.776 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:05.776 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:05.776 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.776 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.777 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.777 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.777 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T12:01:05.777 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T12:01:05.777 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.778 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:05.778 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:05.778 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.778 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.778 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.779 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.779 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T12:01:05.779 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T12:01:05.779 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.779 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:05.780 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:05.780 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.780 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.780 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.780 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T12:01:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T12:01:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T12:01:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T12:01:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T12:01:05.785 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T12:01:05.785 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.785 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:05.785 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:05.785 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.786 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.786 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.786 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.786 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T12:01:05.787 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T12:01:05.787 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.787 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:05.787 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:05.787 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.787 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.788 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.788 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.788 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T12:01:05.788 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T12:01:05.788 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.789 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:05.789 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:05.789 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.789 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.789 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.790 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.790 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T12:01:05.790 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T12:01:05.790 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.790 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:05.791 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T12:01:05.791 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T12:01:05.791 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.791 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.791 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.792 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T12:01:05.792 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T12:01:05.792 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.792 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:05.792 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:01:05.793 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:01:05.793 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:01:05.793 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.793 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.793 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.794 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T12:01:05.794 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T12:01:05.794 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.794 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:05.794 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:01:05.795 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:01:05.795 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:01:05.795 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.795 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.795 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.796 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T12:01:05.796 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T12:01:05.796 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.796 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:05.796 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:01:05.797 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:01:05.797 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:01:05.797 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.797 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.797 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.798 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T12:01:05.798 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T12:01:05.798 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.798 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:01:05.798 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:05.799 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:05.799 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.799 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.799 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.799 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.800 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T12:01:05.800 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T12:01:05.800 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.800 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:01:05.800 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T12:01:05.801 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.801 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.801 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:05.801 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:05.801 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T12:01:05.802 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T12:01:05.802 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:05.802 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:01:05.802 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T12:01:05.802 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:05.803 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.803 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:05.803 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T12:01:05.803 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:01:05.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7faf1006b080 msgr2=0x7faf1006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7faf1006b080 0x7faf1006d540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf28005fd0 tx=0x7faf2800b390 comp rx=0 tx=0).stop 2023-10-18T12:01:05.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c0cb980 msgr2=0x7faf2c1593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c0cb980 0x7faf2c1593e0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00b3f0 tx=0x7faf1c00b8b0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 shutdown_connections 2023-10-18T12:01:05.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7faf1006b080 0x7faf1006d540 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faf2c159920 0x7faf2c1f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faf2c0cb980 0x7faf2c1593e0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 --2- 172.21.15.125:0/3998775294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf2c070350 0x7faf2c0caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 >> 172.21.15.125:0/3998775294 conn(0x7faf2c069930 msgr2=0x7faf2c157840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:05.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 shutdown_connections 2023-10-18T12:01:05.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.765+0000 7faf3346f640 1 -- 172.21.15.125:0/3998775294 wait complete. 2023-10-18T12:01:05.808 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:05.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.908+0000 7fe3885a5640 1 -- 172.21.15.125:0/4131505081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36c0039a0 msgr2=0x7fe36c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.908+0000 7fe3885a5640 1 --2- 172.21.15.125:0/4131505081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe36c0039a0 0x7fe36c003dc0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7fe3800af7e0 tx=0x7fe36c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.908+0000 7fe386b42640 1 -- 172.21.15.125:0/4131505081 reap_dead start 2023-10-18T12:01:05.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.908+0000 7fe3885a5640 1 -- 172.21.15.125:0/4131505081 shutdown_connections 2023-10-18T12:01:05.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.908+0000 7fe3885a5640 1 -- 172.21.15.125:0/4131505081 >> 172.21.15.125:0/4131505081 conn(0x7fe380069930 msgr2=0x7fe380069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:05.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.908+0000 7fe3885a5640 1 -- 172.21.15.125:0/4131505081 shutdown_connections 2023-10-18T12:01:05.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.908+0000 7fe3885a5640 1 -- 172.21.15.125:0/4131505081 wait complete. 2023-10-18T12:01:05.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.909+0000 7fe3885a5640 1 Processor -- start 2023-10-18T12:01:05.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.909+0000 7fe3885a5640 1 -- start start 2023-10-18T12:01:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe3885a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c0039a0 0x7fe3800a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe3885a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe38012ec50 0x7fe38012f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe3885a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe380151b50 0x7fe380151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe3885a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380152c60 con 0x7fe380151b50 2023-10-18T12:01:05.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe3885a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38012f5f0 con 0x7fe36c0039a0 2023-10-18T12:01:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe3885a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe380152980 con 0x7fe38012ec50 2023-10-18T12:01:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe380151b50 0x7fe380151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe380151b50 0x7fe380151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54394/0 (socket says 172.21.15.125:54394) 2023-10-18T12:01:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 -- 172.21.15.125:0/3414895418 learned_addr learned my addr 172.21.15.125:0/3414895418 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:05.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 -- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe38012ec50 msgr2=0x7fe38012f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe38012ec50 0x7fe38012f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 -- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c0039a0 msgr2=0x7fe3800a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386341640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c0039a0 0x7fe3800a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:05.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c0039a0 0x7fe3800a97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:05.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 -- 172.21.15.125:0/3414895418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800a6c70 con 0x7fe380151b50 2023-10-18T12:01:05.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe386b42640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe380151b50 0x7fe380151fd0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fe3800cc430 tx=0x7fe378006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:05.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe376ffd640 1 -- 172.21.15.125:0/3414895418 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378017030 con 0x7fe380151b50 2023-10-18T12:01:05.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe376ffd640 1 -- 172.21.15.125:0/3414895418 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe37800fc30 con 0x7fe380151b50 2023-10-18T12:01:05.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3800a6f50 con 0x7fe380151b50 2023-10-18T12:01:05.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe376ffd640 1 -- 172.21.15.125:0/3414895418 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378006910 con 0x7fe380151b50 2023-10-18T12:01:05.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.910+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3800a9d10 con 0x7fe380151b50 2023-10-18T12:01:05.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.911+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3800a7470 con 0x7fe380151b50 2023-10-18T12:01:05.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.912+0000 7fe376ffd640 1 -- 172.21.15.125:0/3414895418 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe37801a070 con 0x7fe380151b50 2023-10-18T12:01:05.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.912+0000 7fe376ffd640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe36406b080 0x7fe36406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:05.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.912+0000 7fe376ffd640 1 -- 172.21.15.125:0/3414895418 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fe378018070 con 0x7fe380151b50 2023-10-18T12:01:05.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.913+0000 7fe386341640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe36406b080 0x7fe36406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:05.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.913+0000 7fe386341640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe36406b080 0x7fe36406d540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe370036000 tx=0x7fe3700046c0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:05.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:05.916+0000 7fe376ffd640 1 -- 172.21.15.125:0/3414895418 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3800a7470 con 0x7fe380151b50 2023-10-18T12:01:06.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.083+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3800a7470 con 0x7fe380151b50 2023-10-18T12:01:06.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.084+0000 7fe376ffd640 1 -- 172.21.15.125:0/3414895418 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+13210 (secure 0 0 0) 0x7fe3800a7470 con 0x7fe380151b50 2023-10-18T12:01:06.087 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:06.088 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":238,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:05.398280+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:06.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe36406b080 msgr2=0x7fe36406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe36406b080 0x7fe36406d540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe370036000 tx=0x7fe3700046c0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe380151b50 msgr2=0x7fe380151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe380151b50 0x7fe380151fd0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fe3800cc430 tx=0x7fe378006720 comp rx=0 tx=0).stop 2023-10-18T12:01:06.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 shutdown_connections 2023-10-18T12:01:06.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe36406b080 0x7fe36406d540 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe380151b50 0x7fe380151fd0 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe38012ec50 0x7fe38012f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 --2- 172.21.15.125:0/3414895418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c0039a0 0x7fe3800a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 >> 172.21.15.125:0/3414895418 conn(0x7fe380069930 msgr2=0x7fe38015ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:06.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 shutdown_connections 2023-10-18T12:01:06.094 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.086+0000 7fe3885a5640 1 -- 172.21.15.125:0/3414895418 wait complete. 2023-10-18T12:01:06.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:01:06.101 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:01:06.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.204+0000 7f03434d7640 1 -- 172.21.15.125:0/1441039245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c13d350 msgr2=0x7f033c0cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.204+0000 7f03434d7640 1 --2- 172.21.15.125:0/1441039245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c13d350 0x7f033c0cbd60 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f033c0cc560 tx=0x7f032c02bd70 comp rx=0 tx=0).stop 2023-10-18T12:01:06.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.204+0000 7f03434d7640 1 -- 172.21.15.125:0/1441039245 shutdown_connections 2023-10-18T12:01:06.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.204+0000 7f03434d7640 1 --2- 172.21.15.125:0/1441039245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c13d350 0x7f033c0cbd60 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.204+0000 7f03434d7640 1 --2- 172.21.15.125:0/1441039245 >> v2:172.21.15.139:3300/0 conn(0x7f033c0cb980 0x7f033c13db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.204+0000 7f03434d7640 1 -- 172.21.15.125:0/1441039245 >> 172.21.15.125:0/1441039245 conn(0x7f033c069930 msgr2=0x7f033c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:06.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.205+0000 7f03434d7640 1 -- 172.21.15.125:0/1441039245 shutdown_connections 2023-10-18T12:01:06.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.205+0000 7f03434d7640 1 -- 172.21.15.125:0/1441039245 wait complete. 2023-10-18T12:01:06.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f03434d7640 1 Processor -- start 2023-10-18T12:01:06.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f03434d7640 1 -- start start 2023-10-18T12:01:06.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f03434d7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033c0c96b0 0x7f033c135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f03434d7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c0cb980 0x7f033c150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f03434d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f033c134b60 0x7f033c15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f03434d7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c1501e0 con 0x7f033c0cb980 2023-10-18T12:01:06.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f03434d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c1ef100 con 0x7f033c134b60 2023-10-18T12:01:06.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f03434d7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c136280 con 0x7f033c0c96b0 2023-10-18T12:01:06.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f0340a72640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c0cb980 0x7f033c150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f0340a72640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c0cb980 0x7f033c150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54418/0 (socket says 172.21.15.125:54418) 2023-10-18T12:01:06.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.206+0000 7f0340a72640 1 -- 172.21.15.125:0/3227027505 learned_addr learned my addr 172.21.15.125:0/3227027505 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:06.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0341273640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033c0c96b0 0x7f033c135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0341a74640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f033c134b60 0x7f033c15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0340a72640 1 -- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033c0c96b0 msgr2=0x7f033c135a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0340a72640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033c0c96b0 0x7f033c135a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0340a72640 1 -- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f033c134b60 msgr2=0x7f033c15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0340a72640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f033c134b60 0x7f033c15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0340a72640 1 -- 172.21.15.125:0/3227027505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009490 con 0x7f033c0cb980 2023-10-18T12:01:06.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0341a74640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f033c134b60 0x7f033c15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:06.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0340a72640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c0cb980 0x7f033c150f60 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f032c000c00 tx=0x7f032c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:06.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0331ffb640 1 -- 172.21.15.125:0/3227027505 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c034440 con 0x7f033c0cb980 2023-10-18T12:01:06.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0331ffb640 1 -- 172.21.15.125:0/3227027505 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f032c0346b0 con 0x7f033c0cb980 2023-10-18T12:01:06.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f0331ffb640 1 -- 172.21.15.125:0/3227027505 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c034ac0 con 0x7f033c0cb980 2023-10-18T12:01:06.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c150480 con 0x7f033c0cb980 2023-10-18T12:01:06.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.207+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c150a40 con 0x7f033c0cb980 2023-10-18T12:01:06.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.208+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c0690e0 con 0x7f033c0cb980 2023-10-18T12:01:06.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.209+0000 7f0331ffb640 1 -- 172.21.15.125:0/3227027505 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f032c03b040 con 0x7f033c0cb980 2023-10-18T12:01:06.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.209+0000 7f0331ffb640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f032006b0b0 0x7f032006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.209+0000 7f0331ffb640 1 -- 172.21.15.125:0/3227027505 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f032c0abe90 con 0x7f033c0cb980 2023-10-18T12:01:06.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.209+0000 7f0341273640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f032006b0b0 0x7f032006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.210+0000 7f0341273640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f032006b0b0 0x7f032006d570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0338005e10 tx=0x7f0338005da0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:06.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.214+0000 7f0331ffb640 1 -- 172.21.15.125:0/3227027505 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f032c076db0 con 0x7f033c0cb980 2023-10-18T12:01:06.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.378+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f033c06ff80 con 0x7f033c0cb980 2023-10-18T12:01:06.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.378+0000 7f0331ffb640 1 -- 172.21.15.125:0/3227027505 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v238) v1 ==== 93+0+10 (secure 0 0 0) 0x7f032c0410a0 con 0x7f033c0cb980 2023-10-18T12:01:06.382 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:01:06.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f032006b0b0 msgr2=0x7f032006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f032006b0b0 0x7f032006d570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0338005e10 tx=0x7f0338005da0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c0cb980 msgr2=0x7f033c150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c0cb980 0x7f033c150f60 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f032c000c00 tx=0x7f032c0039e0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 shutdown_connections 2023-10-18T12:01:06.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f032006b0b0 0x7f032006d570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f033c134b60 0x7f033c15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f033c0cb980 0x7f033c150f60 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 --2- 172.21.15.125:0/3227027505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f033c0c96b0 0x7f033c135a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 >> 172.21.15.125:0/3227027505 conn(0x7f033c069930 msgr2=0x7f033c0adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:06.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 shutdown_connections 2023-10-18T12:01:06.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.380+0000 7f03434d7640 1 -- 172.21.15.125:0/3227027505 wait complete. 2023-10-18T12:01:06.394 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T12:01:06.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.494+0000 7fb1d1296640 1 -- 172.21.15.125:0/263094499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b80047c0 msgr2=0x7fb1b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.494+0000 7fb1d1296640 1 --2- 172.21.15.125:0/263094499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1b80047c0 0x7fb1b8004c40 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009b50 tx=0x7fb1b80057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.494+0000 7fb1cb7fe640 1 -- 172.21.15.125:0/263094499 reap_dead start 2023-10-18T12:01:06.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.494+0000 7fb1d1296640 1 -- 172.21.15.125:0/263094499 shutdown_connections 2023-10-18T12:01:06.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.494+0000 7fb1d1296640 1 -- 172.21.15.125:0/263094499 >> 172.21.15.125:0/263094499 conn(0x7fb1cc0b3f00 msgr2=0x7fb1cc0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:06.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.494+0000 7fb1d1296640 1 -- 172.21.15.125:0/263094499 shutdown_connections 2023-10-18T12:01:06.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.494+0000 7fb1d1296640 1 -- 172.21.15.125:0/263094499 wait complete. 2023-10-18T12:01:06.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.495+0000 7fb1d1296640 1 Processor -- start 2023-10-18T12:01:06.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1d1296640 1 -- start start 2023-10-18T12:01:06.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1d1296640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b80047c0 0x7fb1cc13e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1d1296640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1cc154a90 0x7fb1cc154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1d1296640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1cc14b2a0 0x7fb1cc14b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1d1296640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc155450 con 0x7fb1cc154a90 2023-10-18T12:01:06.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1d1296640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc14bfd0 con 0x7fb1b80047c0 2023-10-18T12:01:06.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1d1296640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc14c2b0 con 0x7fb1cc14b2a0 2023-10-18T12:01:06.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1ca7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1cc154a90 0x7fb1cc154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1ca7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1cc154a90 0x7fb1cc154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54426/0 (socket says 172.21.15.125:54426) 2023-10-18T12:01:06.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1ca7fc640 1 -- 172.21.15.125:0/2471762097 learned_addr learned my addr 172.21.15.125:0/2471762097 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:06.503 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1cb7fe640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1cc14b2a0 0x7fb1cc14b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.496+0000 7fb1caffd640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b80047c0 0x7fb1cc13e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1ca7fc640 1 -- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1cc14b2a0 msgr2=0x7fb1cc14b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1ca7fc640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1cc14b2a0 0x7fb1cc14b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1ca7fc640 1 -- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b80047c0 msgr2=0x7fb1cc13e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1ca7fc640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b80047c0 0x7fb1cc13e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1ca7fc640 1 -- 172.21.15.125:0/2471762097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc14c590 con 0x7fb1cc154a90 2023-10-18T12:01:06.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1caffd640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b80047c0 0x7fb1cc13e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:06.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1ca7fc640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1cc154a90 0x7fb1cc154f10 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009a70 tx=0x7fb1b4033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:06.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1c37fe640 1 -- 172.21.15.125:0/2471762097 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4004460 con 0x7fb1cc154a90 2023-10-18T12:01:06.505 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1c37fe640 1 -- 172.21.15.125:0/2471762097 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb1b4033e80 con 0x7fb1cc154a90 2023-10-18T12:01:06.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1c37fe640 1 -- 172.21.15.125:0/2471762097 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4003830 con 0x7fb1cc154a90 2023-10-18T12:01:06.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc14c870 con 0x7fb1cc154a90 2023-10-18T12:01:06.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.497+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc14bc60 con 0x7fb1cc154a90 2023-10-18T12:01:06.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.498+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc0690e0 con 0x7fb1cc154a90 2023-10-18T12:01:06.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.498+0000 7fb1c37fe640 1 -- 172.21.15.125:0/2471762097 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb1b4003990 con 0x7fb1cc154a90 2023-10-18T12:01:06.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.499+0000 7fb1c37fe640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb1ac06b0b0 0x7fb1ac06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.499+0000 7fb1c37fe640 1 -- 172.21.15.125:0/2471762097 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fb1b40ac7f0 con 0x7fb1cc154a90 2023-10-18T12:01:06.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.499+0000 7fb1caffd640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb1ac06b0b0 0x7fb1ac06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.499+0000 7fb1caffd640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb1ac06b0b0 0x7fb1ac06d570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4001ce0 tx=0x7fb1c4001b90 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:06.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.503+0000 7fb1c37fe640 1 -- 172.21.15.125:0/2471762097 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1b40776d0 con 0x7fb1cc154a90 2023-10-18T12:01:06.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.671+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb1cc15c040 con 0x7fb1cc154a90 2023-10-18T12:01:06.674 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.672+0000 7fb1c37fe640 1 -- 172.21.15.125:0/2471762097 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v238) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb1b404a070 con 0x7fb1cc154a90 2023-10-18T12:01:06.675 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:06.675 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T12:01:06.675 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:06.675 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T12:01:06.676 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T12:01:06.676 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:06.676 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T12:01:06.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb1ac06b0b0 msgr2=0x7fb1ac06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb1ac06b0b0 0x7fb1ac06d570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4001ce0 tx=0x7fb1c4001b90 comp rx=0 tx=0).stop 2023-10-18T12:01:06.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1cc154a90 msgr2=0x7fb1cc154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1cc154a90 0x7fb1cc154f10 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009a70 tx=0x7fb1b4033c50 comp rx=0 tx=0).stop 2023-10-18T12:01:06.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 shutdown_connections 2023-10-18T12:01:06.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb1ac06b0b0 0x7fb1ac06d570 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1cc14b2a0 0x7fb1cc14b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb1cc154a90 0x7fb1cc154f10 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 --2- 172.21.15.125:0/2471762097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b80047c0 0x7fb1cc13e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.673+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 >> 172.21.15.125:0/2471762097 conn(0x7fb1cc0b3f00 msgr2=0x7fb1cc0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:06.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.674+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 shutdown_connections 2023-10-18T12:01:06.680 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.674+0000 7fb1d1296640 1 -- 172.21.15.125:0/2471762097 wait complete. 2023-10-18T12:01:06.687 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T12:01:06.687 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T12:01:06.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.792+0000 7fa6d814c640 1 -- 172.21.15.125:0/435783018 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 msgr2=0x7fa6d015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.792+0000 7fa6d814c640 1 --2- 172.21.15.125:0/435783018 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 0x7fa6d015a9f0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0140c40 tx=0x7fa6cc02c630 comp rx=0 tx=0).stop 2023-10-18T12:01:06.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.792+0000 7fa6d814c640 1 -- 172.21.15.125:0/435783018 shutdown_connections 2023-10-18T12:01:06.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.792+0000 7fa6d814c640 1 --2- 172.21.15.125:0/435783018 >> v2:172.21.15.139:3300/0 conn(0x7fa6d0140660 0x7fa6d00c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.792+0000 7fa6d814c640 1 --2- 172.21.15.125:0/435783018 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 0x7fa6d015a9f0 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.792+0000 7fa6d814c640 1 -- 172.21.15.125:0/435783018 >> 172.21.15.125:0/435783018 conn(0x7fa6d0069930 msgr2=0x7fa6d0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:06.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.792+0000 7fa6d814c640 1 -- 172.21.15.125:0/435783018 shutdown_connections 2023-10-18T12:01:06.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.792+0000 7fa6d814c640 1 -- 172.21.15.125:0/435783018 wait complete. 2023-10-18T12:01:06.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.793+0000 7fa6d814c640 1 Processor -- start 2023-10-18T12:01:06.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.793+0000 7fa6d814c640 1 -- start start 2023-10-18T12:01:06.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d814c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6d00c9d00 0x7fa6d0135eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d814c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 0x7fa6d013ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d814c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0140660 0x7fa6d01e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d814c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0139cd0 con 0x7fa6d013d360 2023-10-18T12:01:06.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d814c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0139fb0 con 0x7fa6d0140660 2023-10-18T12:01:06.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d814c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d013a290 con 0x7fa6d00c9d00 2023-10-18T12:01:06.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 0x7fa6d013ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 0x7fa6d013ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54446/0 (socket says 172.21.15.125:54446) 2023-10-18T12:01:06.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 -- 172.21.15.125:0/310396115 learned_addr learned my addr 172.21.15.125:0/310396115 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:06.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d5ee8640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6d00c9d00 0x7fa6d0135eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d66e9640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0140660 0x7fa6d01e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 -- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6d00c9d00 msgr2=0x7fa6d0135eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6d00c9d00 0x7fa6d0135eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 -- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0140660 msgr2=0x7fa6d01e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:06.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0140660 0x7fa6d01e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:06.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 -- 172.21.15.125:0/310396115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc009df0 con 0x7fa6d013d360 2023-10-18T12:01:06.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d56e7640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 0x7fa6d013ad70 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000c970 tx=0x7fa6c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:06.806 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6c67fc640 1 -- 172.21.15.125:0/310396115 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0007bb0 con 0x7fa6d013d360 2023-10-18T12:01:06.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6c67fc640 1 -- 172.21.15.125:0/310396115 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa6c0007e20 con 0x7fa6d013d360 2023-10-18T12:01:06.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6c67fc640 1 -- 172.21.15.125:0/310396115 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0005280 con 0x7fa6d013d360 2023-10-18T12:01:06.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d013a570 con 0x7fa6d013d360 2023-10-18T12:01:06.807 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.794+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d01363f0 con 0x7fa6d013d360 2023-10-18T12:01:06.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.795+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d0136610 con 0x7fa6d013d360 2023-10-18T12:01:06.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.796+0000 7fa6c67fc640 1 -- 172.21.15.125:0/310396115 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa6c00053e0 con 0x7fa6d013d360 2023-10-18T12:01:06.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.796+0000 7fa6c67fc640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa6b406b0b0 0x7fa6b406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:06.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.796+0000 7fa6c67fc640 1 -- 172.21.15.125:0/310396115 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fa6c0089cf0 con 0x7fa6d013d360 2023-10-18T12:01:06.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.796+0000 7fa6d5ee8640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa6b406b0b0 0x7fa6b406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:06.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.797+0000 7fa6d5ee8640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa6b406b0b0 0x7fa6b406d570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc035000 tx=0x7fa6cc004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:06.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.800+0000 7fa6c67fc640 1 -- 172.21.15.125:0/310396115 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6c008e070 con 0x7fa6d013d360 2023-10-18T12:01:06.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:06.967+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa6d00690e0 con 0x7fa6d013d360 2023-10-18T12:01:07.095 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.092+0000 7fa6c67fc640 1 -- 172.21.15.125:0/310396115 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v239) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa6c001d070 con 0x7fa6d013d360 2023-10-18T12:01:07.095 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T12:01:07.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa6b406b0b0 msgr2=0x7fa6b406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:07.097 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa6b406b0b0 0x7fa6b406d570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc035000 tx=0x7fa6cc004730 comp rx=0 tx=0).stop 2023-10-18T12:01:07.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 msgr2=0x7fa6d013ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:07.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 0x7fa6d013ad70 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000c970 tx=0x7fa6c000ce30 comp rx=0 tx=0).stop 2023-10-18T12:01:07.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 shutdown_connections 2023-10-18T12:01:07.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fa6b406b0b0 0x7fa6b406d570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:07.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6d0140660 0x7fa6d01e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:07.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa6d013d360 0x7fa6d013ad70 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:07.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 --2- 172.21.15.125:0/310396115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6d00c9d00 0x7fa6d0135eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:07.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.094+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 >> 172.21.15.125:0/310396115 conn(0x7fa6d0069930 msgr2=0x7fa6d0157500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:07.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.095+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 shutdown_connections 2023-10-18T12:01:07.099 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.095+0000 7fa6d814c640 1 -- 172.21.15.125:0/310396115 wait complete. 2023-10-18T12:01:07.111 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T12:01:07.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.217+0000 7f24d7d8e640 1 -- 172.21.15.125:0/1844495576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 msgr2=0x7f24d00b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:07.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.217+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/1844495576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 0x7f24d00b1440 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f24d00b06d0 tx=0x7f24c802be40 comp rx=0 tx=0).stop 2023-10-18T12:01:07.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.217+0000 7f24d7d8e640 1 -- 172.21.15.125:0/1844495576 shutdown_connections 2023-10-18T12:01:07.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.217+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/1844495576 >> v2:172.21.15.139:3300/0 conn(0x7f24d00b1980 0x7f24d0070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:07.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.217+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/1844495576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 0x7f24d00b1440 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:07.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.217+0000 7f24d7d8e640 1 -- 172.21.15.125:0/1844495576 >> 172.21.15.125:0/1844495576 conn(0x7f24d00af730 msgr2=0x7f24d00afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:07.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.218+0000 7f24d7d8e640 1 -- 172.21.15.125:0/1844495576 shutdown_connections 2023-10-18T12:01:07.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.218+0000 7f24d7d8e640 1 -- 172.21.15.125:0/1844495576 wait complete. 2023-10-18T12:01:07.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d7d8e640 1 Processor -- start 2023-10-18T12:01:07.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d7d8e640 1 -- start start 2023-10-18T12:01:07.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d7d8e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24d00705c0 0x7f24d01372a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:07.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d7d8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d00b1980 0x7f24d0069720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:07.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d7d8e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 0x7f24d0069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:07.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d7d8e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d006dfe0 con 0x7f24d01416f0 2023-10-18T12:01:07.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d7d8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d006dd00 con 0x7f24d00b1980 2023-10-18T12:01:07.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d7d8e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d013eea0 con 0x7f24d00705c0 2023-10-18T12:01:07.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.219+0000 7f24d632b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 0x7f24d0069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:07.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d632b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 0x7f24d0069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:54454/0 (socket says 172.21.15.125:54454) 2023-10-18T12:01:07.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d632b640 1 -- 172.21.15.125:0/3081081070 learned_addr learned my addr 172.21.15.125:0/3081081070 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:07.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d5329640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d00b1980 0x7f24d0069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:07.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d5b2a640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24d00705c0 0x7f24d01372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:07.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d632b640 1 -- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24d00705c0 msgr2=0x7f24d01372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:07.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d632b640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24d00705c0 0x7f24d01372a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:07.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d632b640 1 -- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d00b1980 msgr2=0x7f24d0069720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:07.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d632b640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d00b1980 0x7f24d0069720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:07.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d632b640 1 -- 172.21.15.125:0/3081081070 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8009560 con 0x7f24d01416f0 2023-10-18T12:01:07.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d5329640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d00b1980 0x7f24d0069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:07.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d632b640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 0x7f24d0069c60 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f24cc001560 tx=0x7f24cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:07.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24c67fc640 1 -- 172.21.15.125:0/3081081070 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc003da0 con 0x7f24d01416f0 2023-10-18T12:01:07.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24c67fc640 1 -- 172.21.15.125:0/3081081070 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24cc005070 con 0x7f24d01416f0 2023-10-18T12:01:07.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d013f180 con 0x7f24d01416f0 2023-10-18T12:01:07.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24c67fc640 1 -- 172.21.15.125:0/3081081070 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc005460 con 0x7f24d01416f0 2023-10-18T12:01:07.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.220+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d0156c80 con 0x7f24d01416f0 2023-10-18T12:01:07.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.221+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d013f6a0 con 0x7f24d01416f0 2023-10-18T12:01:07.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.222+0000 7f24c67fc640 1 -- 172.21.15.125:0/3081081070 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f24cc0055c0 con 0x7f24d01416f0 2023-10-18T12:01:07.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.222+0000 7f24c67fc640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f24b406b080 0x7f24b406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:07.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.222+0000 7f24c67fc640 1 -- 172.21.15.125:0/3081081070 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f24cc0896b0 con 0x7f24d01416f0 2023-10-18T12:01:07.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.222+0000 7f24d5b2a640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f24b406b080 0x7f24b406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:07.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.223+0000 7f24d5b2a640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f24b406b080 0x7f24b406d540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f24c8035000 tx=0x7f24c8004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:07.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.226+0000 7f24c67fc640 1 -- 172.21.15.125:0/3081081070 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24cc0545d0 con 0x7f24d01416f0 2023-10-18T12:01:07.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:07.394+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f24d013f6a0 con 0x7f24d01416f0 2023-10-18T12:01:08.098 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.095+0000 7f24c67fc640 1 -- 172.21.15.125:0/3081081070 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v240) v1 ==== 170+0+0 (secure 0 0 0) 0x7f24d013f6a0 con 0x7f24d01416f0 2023-10-18T12:01:08.098 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T12:01:08.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f24b406b080 msgr2=0x7f24b406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:08.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f24b406b080 0x7f24b406d540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f24c8035000 tx=0x7f24c8004730 comp rx=0 tx=0).stop 2023-10-18T12:01:08.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 msgr2=0x7f24d0069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:08.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 0x7f24d0069c60 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f24cc001560 tx=0x7f24cc00cc70 comp rx=0 tx=0).stop 2023-10-18T12:01:08.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 shutdown_connections 2023-10-18T12:01:08.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f24b406b080 0x7f24b406d540 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24d00b1980 0x7f24d0069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24d01416f0 0x7f24d0069c60 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 --2- 172.21.15.125:0/3081081070 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24d00705c0 0x7f24d01372a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 >> 172.21.15.125:0/3081081070 conn(0x7f24d00af730 msgr2=0x7f24d015a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:08.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 shutdown_connections 2023-10-18T12:01:08.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.098+0000 7f24d7d8e640 1 -- 172.21.15.125:0/3081081070 wait complete. 2023-10-18T12:01:08.129 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T12:01:08.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.235+0000 7f3def41a640 1 -- 172.21.15.125:0/2468041066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 msgr2=0x7f3de80ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:08.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.235+0000 7f3def41a640 1 --2- 172.21.15.125:0/2468041066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 0x7f3de80ca870 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f3de80af7e0 tx=0x7f3de402c2f0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.235+0000 7f3def41a640 1 -- 172.21.15.125:0/2468041066 shutdown_connections 2023-10-18T12:01:08.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.235+0000 7f3def41a640 1 --2- 172.21.15.125:0/2468041066 >> v2:172.21.15.139:3300/0 conn(0x7f3de80c9730 0x7f3de80afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.235+0000 7f3def41a640 1 --2- 172.21.15.125:0/2468041066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 0x7f3de80ca870 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.235+0000 7f3def41a640 1 -- 172.21.15.125:0/2468041066 >> 172.21.15.125:0/2468041066 conn(0x7f3de8069930 msgr2=0x7f3de8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:08.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.235+0000 7f3def41a640 1 -- 172.21.15.125:0/2468041066 shutdown_connections 2023-10-18T12:01:08.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.235+0000 7f3def41a640 1 -- 172.21.15.125:0/2468041066 wait complete. 2023-10-18T12:01:08.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.236+0000 7f3def41a640 1 Processor -- start 2023-10-18T12:01:08.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.236+0000 7f3def41a640 1 -- start start 2023-10-18T12:01:08.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3def41a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de80c9730 0x7f3de81536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:08.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3def41a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80c9bb0 0x7f3de8145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:08.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3def41a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 0x7f3de81ef670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:08.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3def41a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de814c6d0 con 0x7f3de8157220 2023-10-18T12:01:08.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3def41a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de81efbb0 con 0x7f3de80c9bb0 2023-10-18T12:01:08.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3def41a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8153d30 con 0x7f3de80c9730 2023-10-18T12:01:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 0x7f3de81ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 0x7f3de81ef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35782/0 (socket says 172.21.15.125:35782) 2023-10-18T12:01:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 -- 172.21.15.125:0/2402390978 learned_addr learned my addr 172.21.15.125:0/2402390978 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3dec9b5640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80c9bb0 0x7f3de8145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded1b6640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de80c9730 0x7f3de81536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:08.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 -- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de80c9730 msgr2=0x7f3de81536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:08.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de80c9730 0x7f3de81536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 -- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80c9bb0 msgr2=0x7f3de8145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:08.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80c9bb0 0x7f3de8145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 -- 172.21.15.125:0/2402390978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de4009900 con 0x7f3de8157220 2023-10-18T12:01:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded1b6640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de80c9730 0x7f3de81536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.237+0000 7f3ded9b7640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 0x7f3de81ef670 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c430 tx=0x7f3de000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.238+0000 7f3dddffb640 1 -- 172.21.15.125:0/2402390978 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de00035f0 con 0x7f3de8157220 2023-10-18T12:01:08.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.238+0000 7f3dddffb640 1 -- 172.21.15.125:0/2402390978 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3de0003770 con 0x7f3de8157220 2023-10-18T12:01:08.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.238+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de8154010 con 0x7f3de8157220 2023-10-18T12:01:08.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.238+0000 7f3dddffb640 1 -- 172.21.15.125:0/2402390978 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0003b80 con 0x7f3de8157220 2023-10-18T12:01:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.238+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de81545d0 con 0x7f3de8157220 2023-10-18T12:01:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.239+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de814c880 con 0x7f3de8157220 2023-10-18T12:01:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.239+0000 7f3dddffb640 1 -- 172.21.15.125:0/2402390978 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3de0003ce0 con 0x7f3de8157220 2023-10-18T12:01:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.240+0000 7f3dddffb640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dcc06b080 0x7f3dcc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.240+0000 7f3dddffb640 1 -- 172.21.15.125:0/2402390978 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 5985+0+0 (secure 0 0 0) 0x7f3de0089240 con 0x7f3de8157220 2023-10-18T12:01:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.240+0000 7f3ded1b6640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dcc06b080 0x7f3dcc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.241+0000 7f3ded1b6640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dcc06b080 0x7f3dcc06d540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3de4036000 tx=0x7f3de4004ab0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.245+0000 7f3dddffb640 1 -- 172.21.15.125:0/2402390978 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3de814c880 con 0x7f3de8157220 2023-10-18T12:01:08.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:08.445+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3de814c880 con 0x7f3de8157220 2023-10-18T12:01:09.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.122+0000 7f3dddffb640 1 -- 172.21.15.125:0/2402390978 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v96) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3de814c880 con 0x7f3de8157220 2023-10-18T12:01:09.126 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 31 and data pool 32 2023-10-18T12:01:09.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.129+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dcc06b080 msgr2=0x7f3dcc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.129+0000 7f3def41a640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dcc06b080 0x7f3dcc06d540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3de4036000 tx=0x7f3de4004ab0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.129+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 msgr2=0x7f3de81ef670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.129+0000 7f3def41a640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 0x7f3de81ef670 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c430 tx=0x7f3de000c8f0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.130+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 shutdown_connections 2023-10-18T12:01:09.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.130+0000 7f3def41a640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dcc06b080 0x7f3dcc06d540 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.130+0000 7f3def41a640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80c9bb0 0x7f3de8145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.130+0000 7f3def41a640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3de8157220 0x7f3de81ef670 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.130+0000 7f3def41a640 1 --2- 172.21.15.125:0/2402390978 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3de80c9730 0x7f3de81536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.130+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 >> 172.21.15.125:0/2402390978 conn(0x7f3de8069930 msgr2=0x7f3de80ad760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:09.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.131+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 shutdown_connections 2023-10-18T12:01:09.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.131+0000 7f3def41a640 1 -- 172.21.15.125:0/2402390978 wait complete. 2023-10-18T12:01:09.155 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:09.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.258+0000 7fbed63e9640 1 -- 172.21.15.125:0/2666414089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb80039a0 msgr2=0x7fbeb8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.258+0000 7fbed63e9640 1 --2- 172.21.15.125:0/2666414089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb80039a0 0x7fbeb8003dc0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009b50 tx=0x7fbeb80057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.258+0000 7fbed4986640 1 -- 172.21.15.125:0/2666414089 reap_dead start 2023-10-18T12:01:09.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.258+0000 7fbed63e9640 1 -- 172.21.15.125:0/2666414089 shutdown_connections 2023-10-18T12:01:09.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.258+0000 7fbed63e9640 1 -- 172.21.15.125:0/2666414089 >> 172.21.15.125:0/2666414089 conn(0x7fbed00d0d60 msgr2=0x7fbed00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:09.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.258+0000 7fbed63e9640 1 -- 172.21.15.125:0/2666414089 shutdown_connections 2023-10-18T12:01:09.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.258+0000 7fbed63e9640 1 -- 172.21.15.125:0/2666414089 wait complete. 2023-10-18T12:01:09.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.259+0000 7fbed63e9640 1 Processor -- start 2023-10-18T12:01:09.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.259+0000 7fbed63e9640 1 -- start start 2023-10-18T12:01:09.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed63e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbeb80039a0 0x7fbed0158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed63e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed015df90 0x7fbed015c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed63e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbed015c8c0 0x7fbed00ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed63e9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed015cef0 con 0x7fbed015c8c0 2023-10-18T12:01:09.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed63e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed015d1d0 con 0x7fbed015df90 2023-10-18T12:01:09.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed63e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed015d4b0 con 0x7fbeb80039a0 2023-10-18T12:01:09.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbed015c8c0 0x7fbed00ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbed015c8c0 0x7fbed00ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35808/0 (socket says 172.21.15.125:35808) 2023-10-18T12:01:09.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 -- 172.21.15.125:0/1100719507 learned_addr learned my addr 172.21.15.125:0/1100719507 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:09.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbecf7fe640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed015df90 0x7fbed015c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbecffff640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbeb80039a0 0x7fbed0158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 -- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbeb80039a0 msgr2=0x7fbed0158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbeb80039a0 0x7fbed0158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 -- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed015df90 msgr2=0x7fbed015c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed015df90 0x7fbed015c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 -- 172.21.15.125:0/1100719507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed015d790 con 0x7fbed015c8c0 2023-10-18T12:01:09.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbecf7fe640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed015df90 0x7fbed015c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:09.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.260+0000 7fbed4986640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbed015c8c0 0x7fbed00ca280 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fbec8000f80 tx=0x7fbec80019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:09.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.261+0000 7fbeccff9640 1 -- 172.21.15.125:0/1100719507 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8011870 con 0x7fbed015c8c0 2023-10-18T12:01:09.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.261+0000 7fbeccff9640 1 -- 172.21.15.125:0/1100719507 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbec8011ae0 con 0x7fbed015c8c0 2023-10-18T12:01:09.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.261+0000 7fbeccff9640 1 -- 172.21.15.125:0/1100719507 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80020d0 con 0x7fbed015c8c0 2023-10-18T12:01:09.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.261+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed015da70 con 0x7fbed015c8c0 2023-10-18T12:01:09.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.261+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed015e410 con 0x7fbed015c8c0 2023-10-18T12:01:09.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.262+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed015e600 con 0x7fbed015c8c0 2023-10-18T12:01:09.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.262+0000 7fbeccff9640 1 -- 172.21.15.125:0/1100719507 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbec801c460 con 0x7fbed015c8c0 2023-10-18T12:01:09.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.262+0000 7fbeccff9640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbebc06b080 0x7fbebc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.263+0000 7fbeccff9640 1 -- 172.21.15.125:0/1100719507 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fbec808a710 con 0x7fbed015c8c0 2023-10-18T12:01:09.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.263+0000 7fbecffff640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbebc06b080 0x7fbebc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.263+0000 7fbecffff640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbebc06b080 0x7fbebc06d540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbec0036000 tx=0x7fbec0035040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:09.272 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.267+0000 7fbeccff9640 1 -- 172.21.15.125:0/1100719507 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbec80551d0 con 0x7fbed015c8c0 2023-10-18T12:01:09.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.436+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbed015dc60 con 0x7fbed015c8c0 2023-10-18T12:01:09.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.437+0000 7fbeccff9640 1 -- 172.21.15.125:0/1100719507 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v241) v1 ==== 74+0+16935 (secure 0 0 0) 0x7fbec80587f0 con 0x7fbed015c8c0 2023-10-18T12:01:09.440 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:09.440 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":241,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:08.446970+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:09.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbebc06b080 msgr2=0x7fbebc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbebc06b080 0x7fbebc06d540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbec0036000 tx=0x7fbec0035040 comp rx=0 tx=0).stop 2023-10-18T12:01:09.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbed015c8c0 msgr2=0x7fbed00ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbed015c8c0 0x7fbed00ca280 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fbec8000f80 tx=0x7fbec80019b0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 shutdown_connections 2023-10-18T12:01:09.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fbebc06b080 0x7fbebc06d540 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed015df90 0x7fbed015c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbed015c8c0 0x7fbed00ca280 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 --2- 172.21.15.125:0/1100719507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbeb80039a0 0x7fbed0158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.439+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 >> 172.21.15.125:0/1100719507 conn(0x7fbed00d0d60 msgr2=0x7fbed015a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:09.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.440+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 shutdown_connections 2023-10-18T12:01:09.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.440+0000 7fbed63e9640 1 -- 172.21.15.125:0/1100719507 wait complete. 2023-10-18T12:01:09.454 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:09.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.554+0000 7f534078e640 1 -- 172.21.15.125:0/3677787330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5324005180 msgr2=0x7f532400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.554+0000 7f534078e640 1 --2- 172.21.15.125:0/3677787330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5324005180 0x7f532400b980 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f533806f430 tx=0x7f53240057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.555+0000 7f534078e640 1 -- 172.21.15.125:0/3677787330 shutdown_connections 2023-10-18T12:01:09.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.555+0000 7f534078e640 1 --2- 172.21.15.125:0/3677787330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5324005180 0x7f532400b980 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.555+0000 7f534078e640 1 -- 172.21.15.125:0/3677787330 >> 172.21.15.125:0/3677787330 conn(0x7f5338069930 msgr2=0x7f5338069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:09.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.555+0000 7f534078e640 1 -- 172.21.15.125:0/3677787330 shutdown_connections 2023-10-18T12:01:09.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.555+0000 7f534078e640 1 -- 172.21.15.125:0/3677787330 wait complete. 2023-10-18T12:01:09.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.556+0000 7f534078e640 1 Processor -- start 2023-10-18T12:01:09.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.556+0000 7f534078e640 1 -- start start 2023-10-18T12:01:09.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.556+0000 7f534078e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5324005180 0x7f53381566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.556+0000 7f534078e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f533814df20 0x7f53381ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.556+0000 7f534078e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53381ec280 0x7f53381ec730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.556+0000 7f534078e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533813a740 con 0x7f533814df20 2023-10-18T12:01:09.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f534078e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533814e3b0 con 0x7f5324005180 2023-10-18T12:01:09.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f534078e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533814e690 con 0x7f53381ec280 2023-10-18T12:01:09.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f533814df20 0x7f53381ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f533814df20 0x7f53381ebd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35836/0 (socket says 172.21.15.125:35836) 2023-10-18T12:01:09.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 -- 172.21.15.125:0/2443412152 learned_addr learned my addr 172.21.15.125:0/2443412152 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:09.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533e52a640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5324005180 0x7f53381566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533ed2b640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53381ec280 0x7f53381ec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 -- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53381ec280 msgr2=0x7f53381ec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53381ec280 0x7f53381ec730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 -- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5324005180 msgr2=0x7f53381566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5324005180 0x7f53381566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 -- 172.21.15.125:0/2443412152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533814e970 con 0x7f533814df20 2023-10-18T12:01:09.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533e52a640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5324005180 0x7f53381566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:09.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f533dd29640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f533814df20 0x7f53381ebd40 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f5328004d20 tx=0x7f5328001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:09.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f532effd640 1 -- 172.21.15.125:0/2443412152 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328016030 con 0x7f533814df20 2023-10-18T12:01:09.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f532effd640 1 -- 172.21.15.125:0/2443412152 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f53280108c0 con 0x7f533814df20 2023-10-18T12:01:09.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533814ec50 con 0x7f533814df20 2023-10-18T12:01:09.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f532effd640 1 -- 172.21.15.125:0/2443412152 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328010c90 con 0x7f533814df20 2023-10-18T12:01:09.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.557+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5338156c20 con 0x7f533814df20 2023-10-18T12:01:09.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.558+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53380690e0 con 0x7f533814df20 2023-10-18T12:01:09.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.559+0000 7f532effd640 1 -- 172.21.15.125:0/2443412152 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5328019070 con 0x7f533814df20 2023-10-18T12:01:09.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.559+0000 7f532effd640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f531c06b080 0x7f531c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.559+0000 7f532effd640 1 -- 172.21.15.125:0/2443412152 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f5328017070 con 0x7f533814df20 2023-10-18T12:01:09.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.559+0000 7f533e52a640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f531c06b080 0x7f531c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.560+0000 7f533e52a640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f531c06b080 0x7f531c06d540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5334001050 tx=0x7f5334000fc0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:09.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.563+0000 7f532effd640 1 -- 172.21.15.125:0/2443412152 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53280bcca0 con 0x7f533814df20 2023-10-18T12:01:09.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.730+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f533806f700 con 0x7f533814df20 2023-10-18T12:01:09.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.732+0000 7f532effd640 1 -- 172.21.15.125:0/2443412152 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v241) v1 ==== 74+0+16935 (secure 0 0 0) 0x7f5328054dc0 con 0x7f533814df20 2023-10-18T12:01:09.734 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:09.735 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":241,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:08.446970+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:09.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.733+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f531c06b080 msgr2=0x7f531c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.733+0000 7f534078e640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f531c06b080 0x7f531c06d540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5334001050 tx=0x7f5334000fc0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.733+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f533814df20 msgr2=0x7f53381ebd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.733+0000 7f534078e640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f533814df20 0x7f53381ebd40 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f5328004d20 tx=0x7f5328001630 comp rx=0 tx=0).stop 2023-10-18T12:01:09.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.734+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 shutdown_connections 2023-10-18T12:01:09.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.734+0000 7f534078e640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f531c06b080 0x7f531c06d540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.734+0000 7f534078e640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53381ec280 0x7f53381ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.734+0000 7f534078e640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f533814df20 0x7f53381ebd40 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.734+0000 7f534078e640 1 --2- 172.21.15.125:0/2443412152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5324005180 0x7f53381566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.734+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 >> 172.21.15.125:0/2443412152 conn(0x7f5338069930 msgr2=0x7f53380ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:09.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.734+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 shutdown_connections 2023-10-18T12:01:09.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.734+0000 7f534078e640 1 -- 172.21.15.125:0/2443412152 wait complete. 2023-10-18T12:01:09.747 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T12:01:09.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.850+0000 7f4c5c810640 1 -- 172.21.15.125:0/526260717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c5806af00 msgr2=0x7f4c5806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.850+0000 7f4c5c810640 1 --2- 172.21.15.125:0/526260717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c5806af00 0x7f4c5806b2e0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009900 tx=0x7f4c4802bd70 comp rx=0 tx=0).stop 2023-10-18T12:01:09.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.850+0000 7f4c5c810640 1 -- 172.21.15.125:0/526260717 shutdown_connections 2023-10-18T12:01:09.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.850+0000 7f4c5c810640 1 --2- 172.21.15.125:0/526260717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c580abc10 0x7f4c580c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.850+0000 7f4c5c810640 1 --2- 172.21.15.125:0/526260717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c5806af00 0x7f4c5806b2e0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.850+0000 7f4c5c810640 1 -- 172.21.15.125:0/526260717 >> 172.21.15.125:0/526260717 conn(0x7f4c58069930 msgr2=0x7f4c58069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:09.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.850+0000 7f4c5c810640 1 -- 172.21.15.125:0/526260717 shutdown_connections 2023-10-18T12:01:09.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.850+0000 7f4c5c810640 1 -- 172.21.15.125:0/526260717 wait complete. 2023-10-18T12:01:09.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c5c810640 1 Processor -- start 2023-10-18T12:01:09.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c5c810640 1 -- start start 2023-10-18T12:01:09.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c5c810640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c5806af00 0x7f4c5815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c5c810640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c580abc10 0x7f4c580aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c5c810640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c580c96b0 0x7f4c58154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c5c810640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c581f4da0 con 0x7f4c580abc10 2023-10-18T12:01:09.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c5c810640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c581f5080 con 0x7f4c5806af00 2023-10-18T12:01:09.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c5c810640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c581f5360 con 0x7f4c580c96b0 2023-10-18T12:01:09.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c4dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c580abc10 0x7f4c580aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.852+0000 7f4c4dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c580abc10 0x7f4c580aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35846/0 (socket says 172.21.15.125:35846) 2023-10-18T12:01:09.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4dd9b640 1 -- 172.21.15.125:0/3349499536 learned_addr learned my addr 172.21.15.125:0/3349499536 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:09.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c5659c640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c5806af00 0x7f4c5815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c56d9d640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c580c96b0 0x7f4c58154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4dd9b640 1 -- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c580c96b0 msgr2=0x7f4c58154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4dd9b640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c580c96b0 0x7f4c58154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4dd9b640 1 -- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c5806af00 msgr2=0x7f4c5815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:09.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4dd9b640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c5806af00 0x7f4c5815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:09.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4dd9b640 1 -- 172.21.15.125:0/3349499536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48009560 con 0x7f4c580abc10 2023-10-18T12:01:09.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4dd9b640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c580abc10 0x7f4c580aaaa0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000b3f0 tx=0x7f4c4000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4f7fe640 1 -- 172.21.15.125:0/3349499536 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40011830 con 0x7f4c580abc10 2023-10-18T12:01:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4f7fe640 1 -- 172.21.15.125:0/3349499536 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4c40011a80 con 0x7f4c580abc10 2023-10-18T12:01:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c4f7fe640 1 -- 172.21.15.125:0/3349499536 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40011e90 con 0x7f4c580abc10 2023-10-18T12:01:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c581f5640 con 0x7f4c580abc10 2023-10-18T12:01:09.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.853+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c580aafe0 con 0x7f4c580abc10 2023-10-18T12:01:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.854+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c580ab260 con 0x7f4c580abc10 2023-10-18T12:01:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.855+0000 7f4c4f7fe640 1 -- 172.21.15.125:0/3349499536 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4c4001a400 con 0x7f4c580abc10 2023-10-18T12:01:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.855+0000 7f4c4f7fe640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4c3806b080 0x7f4c3806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.855+0000 7f4c4f7fe640 1 -- 172.21.15.125:0/3349499536 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f4c40089b50 con 0x7f4c580abc10 2023-10-18T12:01:09.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.855+0000 7f4c5659c640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4c3806b080 0x7f4c3806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:09.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.856+0000 7f4c5659c640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4c3806b080 0x7f4c3806d540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4c48035000 tx=0x7f4c48004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:09.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:09.859+0000 7f4c4f7fe640 1 -- 172.21.15.125:0/3349499536 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c400547d0 con 0x7f4c580abc10 2023-10-18T12:01:10.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.059+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c580690e0 con 0x7f4c580abc10 2023-10-18T12:01:10.087 INFO:tasks.ceph.mds.a.smithi125.stderr:2023-10-18T12:01:10.084+0000 7ffaa6c58640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:01:10.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.391+0000 7f4c4f7fe640 1 -- 172.21.15.125:0/3349499536 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4c400150b0 con 0x7f4c580abc10 2023-10-18T12:01:10.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4c3806b080 msgr2=0x7f4c3806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:10.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4c3806b080 0x7f4c3806d540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4c48035000 tx=0x7f4c48004730 comp rx=0 tx=0).stop 2023-10-18T12:01:10.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c580abc10 msgr2=0x7f4c580aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:10.396 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c580abc10 0x7f4c580aaaa0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000b3f0 tx=0x7f4c4000b8b0 comp rx=0 tx=0).stop 2023-10-18T12:01:10.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 shutdown_connections 2023-10-18T12:01:10.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4c3806b080 0x7f4c3806d540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:10.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c580c96b0 0x7f4c58154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:10.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c580abc10 0x7f4c580aaaa0 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:10.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 --2- 172.21.15.125:0/3349499536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c5806af00 0x7f4c5815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:10.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 >> 172.21.15.125:0/3349499536 conn(0x7f4c58069930 msgr2=0x7f4c580ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:10.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 shutdown_connections 2023-10-18T12:01:10.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.393+0000 7f4c5c810640 1 -- 172.21.15.125:0/3349499536 wait complete. 2023-10-18T12:01:10.406 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T12:01:10.407 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T12:01:10.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.511+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1155631929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40afa20 msgr2=0x7f93c4158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:10.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.511+0000 7f93c9d1d640 1 --2- 172.21.15.125:0/1155631929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40afa20 0x7f93c4158300 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f93b8005fd0 tx=0x7f93b802be20 comp rx=0 tx=0).stop 2023-10-18T12:01:10.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.511+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1155631929 shutdown_connections 2023-10-18T12:01:10.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.511+0000 7f93c9d1d640 1 --2- 172.21.15.125:0/1155631929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40afa20 0x7f93c4158300 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:10.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.511+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1155631929 >> 172.21.15.125:0/1155631929 conn(0x7f93c4069930 msgr2=0x7f93c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:10.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.511+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1155631929 shutdown_connections 2023-10-18T12:01:10.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.511+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1155631929 wait complete. 2023-10-18T12:01:10.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.512+0000 7f93c9d1d640 1 Processor -- start 2023-10-18T12:01:10.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c9d1d640 1 -- start start 2023-10-18T12:01:10.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c9d1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93c40afa20 0x7f93c414e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c9d1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40b0530 0x7f93c41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c9d1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93c40cbd20 0x7f93c4159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c9d1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c98e0 con 0x7f93c40b0530 2023-10-18T12:01:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c9d1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c414b610 con 0x7f93c40cbd20 2023-10-18T12:01:10.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c9d1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c9e10 con 0x7f93c40afa20 2023-10-18T12:01:10.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40b0530 0x7f93c41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:10.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40b0530 0x7f93c41465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35862/0 (socket says 172.21.15.125:35862) 2023-10-18T12:01:10.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.513+0000 7f93c2ffd640 1 -- 172.21.15.125:0/1062702778 learned_addr learned my addr 172.21.15.125:0/1062702778 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:10.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93c2ffd640 1 -- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93c40afa20 msgr2=0x7f93c414e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:10.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93c2ffd640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93c40afa20 0x7f93c414e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:10.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93c2ffd640 1 -- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93c40cbd20 msgr2=0x7f93c4159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:10.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93c2ffd640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93c40cbd20 0x7f93c4159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:10.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93c2ffd640 1 -- 172.21.15.125:0/1062702778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b8009560 con 0x7f93c40b0530 2023-10-18T12:01:10.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93c2ffd640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40b0530 0x7f93c41465f0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f93b8036040 tx=0x7f93b802dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:10.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93abfff640 1 -- 172.21.15.125:0/1062702778 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8003800 con 0x7f93c40b0530 2023-10-18T12:01:10.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93abfff640 1 -- 172.21.15.125:0/1062702778 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f93b8003a50 con 0x7f93c40b0530 2023-10-18T12:01:10.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c4148d80 con 0x7f93c40b0530 2023-10-18T12:01:10.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93abfff640 1 -- 172.21.15.125:0/1062702778 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8003e00 con 0x7f93c40b0530 2023-10-18T12:01:10.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.514+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c4149020 con 0x7f93c40b0530 2023-10-18T12:01:10.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.515+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c40690e0 con 0x7f93c40b0530 2023-10-18T12:01:10.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.516+0000 7f93abfff640 1 -- 172.21.15.125:0/1062702778 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f93b803e450 con 0x7f93c40b0530 2023-10-18T12:01:10.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.516+0000 7f93abfff640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f93b006b080 0x7f93b006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:10.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.516+0000 7f93abfff640 1 -- 172.21.15.125:0/1062702778 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f93b80ac6b0 con 0x7f93c40b0530 2023-10-18T12:01:10.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.516+0000 7f93c37fe640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f93b006b080 0x7f93b006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:10.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.517+0000 7f93c37fe640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f93b006b080 0x7f93b006d540 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000f60 tx=0x7f93b40079b0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:10.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.520+0000 7f93abfff640 1 -- 172.21.15.125:0/1062702778 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93b8077310 con 0x7f93c40b0530 2023-10-18T12:01:10.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:10.724+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f93c406f700 con 0x7f93c40b0530 2023-10-18T12:01:11.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.398+0000 7f93abfff640 1 -- 172.21.15.125:0/1062702778 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7f93b807fe60 con 0x7f93c40b0530 2023-10-18T12:01:11.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f93b006b080 msgr2=0x7f93b006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f93b006b080 0x7f93b006d540 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000f60 tx=0x7f93b40079b0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40b0530 msgr2=0x7f93c41465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40b0530 0x7f93c41465f0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f93b8036040 tx=0x7f93b802dba0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 shutdown_connections 2023-10-18T12:01:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f93b006b080 0x7f93b006d540 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93c40cbd20 0x7f93c4159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93c40b0530 0x7f93c41465f0 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 --2- 172.21.15.125:0/1062702778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93c40afa20 0x7f93c414e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 >> 172.21.15.125:0/1062702778 conn(0x7f93c4069930 msgr2=0x7f93c4151100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 shutdown_connections 2023-10-18T12:01:11.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.400+0000 7f93c9d1d640 1 -- 172.21.15.125:0/1062702778 wait complete. 2023-10-18T12:01:11.414 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T12:01:11.414 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:11.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.516+0000 7f6599c26640 1 -- 172.21.15.125:0/4131843787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65940ceb40 msgr2=0x7f65940b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.516+0000 7f6599c26640 1 --2- 172.21.15.125:0/4131843787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65940ceb40 0x7f65940b0650 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f658c005fd0 tx=0x7f658c02c050 comp rx=0 tx=0).stop 2023-10-18T12:01:11.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.516+0000 7f6599c26640 1 -- 172.21.15.125:0/4131843787 shutdown_connections 2023-10-18T12:01:11.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.516+0000 7f6599c26640 1 --2- 172.21.15.125:0/4131843787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65940ceb40 0x7f65940b0650 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.516+0000 7f6599c26640 1 --2- 172.21.15.125:0/4131843787 >> v2:172.21.15.139:3300/0 conn(0x7f659415b2a0 0x7f65940d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.516+0000 7f6599c26640 1 -- 172.21.15.125:0/4131843787 >> 172.21.15.125:0/4131843787 conn(0x7f65940d0d60 msgr2=0x7f65940d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:11.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.516+0000 7f6599c26640 1 -- 172.21.15.125:0/4131843787 shutdown_connections 2023-10-18T12:01:11.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.516+0000 7f6599c26640 1 -- 172.21.15.125:0/4131843787 wait complete. 2023-10-18T12:01:11.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.517+0000 7f6599c26640 1 Processor -- start 2023-10-18T12:01:11.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.517+0000 7f6599c26640 1 -- start start 2023-10-18T12:01:11.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6599c26640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65940ceb40 0x7f6594154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:11.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6599c26640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65940cf340 0x7f65940b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:11.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6599c26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415b2a0 0x7f659406b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:11.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6599c26640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659415bb20 con 0x7f65940ceb40 2023-10-18T12:01:11.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6599c26640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659415be00 con 0x7f659415b2a0 2023-10-18T12:01:11.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6599c26640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659415c0e0 con 0x7f65940cf340 2023-10-18T12:01:11.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65940ceb40 0x7f6594154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:11.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65940ceb40 0x7f6594154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35878/0 (socket says 172.21.15.125:35878) 2023-10-18T12:01:11.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 -- 172.21.15.125:0/4060545115 learned_addr learned my addr 172.21.15.125:0/4060545115 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:11.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6592ffd640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65940cf340 0x7f65940b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:11.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6593fff640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415b2a0 0x7f659406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:11.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 -- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65940cf340 msgr2=0x7f65940b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65940cf340 0x7f65940b5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 -- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415b2a0 msgr2=0x7f659406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415b2a0 0x7f659406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 -- 172.21.15.125:0/4060545115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584009560 con 0x7f65940ceb40 2023-10-18T12:01:11.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f65937fe640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65940ceb40 0x7f6594154d40 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f657c00c400 tx=0x7f657c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:11.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f658bfff640 1 -- 172.21.15.125:0/4060545115 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c0052d0 con 0x7f65940ceb40 2023-10-18T12:01:11.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f658bfff640 1 -- 172.21.15.125:0/4060545115 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f657c007020 con 0x7f65940ceb40 2023-10-18T12:01:11.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f658bfff640 1 -- 172.21.15.125:0/4060545115 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c007470 con 0x7f65940ceb40 2023-10-18T12:01:11.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c009840 con 0x7f65940ceb40 2023-10-18T12:01:11.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.518+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65940b0d50 con 0x7f65940ceb40 2023-10-18T12:01:11.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.520+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659415ae00 con 0x7f65940ceb40 2023-10-18T12:01:11.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.520+0000 7f658bfff640 1 -- 172.21.15.125:0/4060545115 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f657c0067d0 con 0x7f65940ceb40 2023-10-18T12:01:11.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.520+0000 7f658bfff640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f657406b080 0x7f657406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:11.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.520+0000 7f658bfff640 1 -- 172.21.15.125:0/4060545115 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f657c089150 con 0x7f65940ceb40 2023-10-18T12:01:11.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.520+0000 7f6592ffd640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f657406b080 0x7f657406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:11.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.521+0000 7f6592ffd640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f657406b080 0x7f657406d540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f659415e820 tx=0x7f6584009260 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:11.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.525+0000 7f658bfff640 1 -- 172.21.15.125:0/4060545115 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f657c053e20 con 0x7f65940ceb40 2023-10-18T12:01:11.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.726+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65940690e0 con 0x7f65940ceb40 2023-10-18T12:01:11.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.727+0000 7f658bfff640 1 -- 172.21.15.125:0/4060545115 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f657c057440 con 0x7f65940ceb40 2023-10-18T12:01:11.730 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:11.730 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":99,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":12768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:08.447292+0000","modified":"2023-10-18T12:01:10.725386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12771":{"gid":12771,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1851256737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1851256737},{"type":"v1","addr":"172.21.15.125:6835","nonce":1851256737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-18T12:01:11.731 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 99 2023-10-18T12:01:11.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f657406b080 msgr2=0x7f657406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f657406b080 0x7f657406d540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f659415e820 tx=0x7f6584009260 comp rx=0 tx=0).stop 2023-10-18T12:01:11.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65940ceb40 msgr2=0x7f6594154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65940ceb40 0x7f6594154d40 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f657c00c400 tx=0x7f657c00c8c0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 shutdown_connections 2023-10-18T12:01:11.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f657406b080 0x7f657406d540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415b2a0 0x7f659406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65940cf340 0x7f65940b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 --2- 172.21.15.125:0/4060545115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f65940ceb40 0x7f6594154d40 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 >> 172.21.15.125:0/4060545115 conn(0x7f65940d0d60 msgr2=0x7f65940cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:11.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 shutdown_connections 2023-10-18T12:01:11.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.729+0000 7f6599c26640 1 -- 172.21.15.125:0/4060545115 wait complete. 2023-10-18T12:01:11.742 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:11.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.843+0000 7fdd99d41640 1 -- 172.21.15.125:0/891286252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c003fd0 msgr2=0x7fdd7c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.843+0000 7fdd99d41640 1 --2- 172.21.15.125:0/891286252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd7c003fd0 0x7fdd7c0047c0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800bb20 tx=0x7fdd7c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.843+0000 7fdd93fff640 1 -- 172.21.15.125:0/891286252 reap_dead start 2023-10-18T12:01:11.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.843+0000 7fdd99d41640 1 -- 172.21.15.125:0/891286252 shutdown_connections 2023-10-18T12:01:11.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.843+0000 7fdd99d41640 1 -- 172.21.15.125:0/891286252 >> 172.21.15.125:0/891286252 conn(0x7fdd940d0d60 msgr2=0x7fdd940d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:11.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.843+0000 7fdd99d41640 1 -- 172.21.15.125:0/891286252 shutdown_connections 2023-10-18T12:01:11.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.843+0000 7fdd99d41640 1 -- 172.21.15.125:0/891286252 wait complete. 2023-10-18T12:01:11.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.844+0000 7fdd99d41640 1 Processor -- start 2023-10-18T12:01:11.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.844+0000 7fdd99d41640 1 -- start start 2023-10-18T12:01:11.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd99d41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c003fd0 0x7fdd9414f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:11.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd99d41640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd940b3b70 0x7fdd940c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:11.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd99d41640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd940ae700 0x7fdd940aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:11.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd99d41640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9414f940 con 0x7fdd940ae700 2023-10-18T12:01:11.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd99d41640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940af0c0 con 0x7fdd7c003fd0 2023-10-18T12:01:11.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd99d41640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940ca930 con 0x7fdd940b3b70 2023-10-18T12:01:11.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd940ae700 0x7fdd940aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:11.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd940ae700 0x7fdd940aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35906/0 (socket says 172.21.15.125:35906) 2023-10-18T12:01:11.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 -- 172.21.15.125:0/641678937 learned_addr learned my addr 172.21.15.125:0/641678937 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:11.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd92ffd640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd940b3b70 0x7fdd940c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:11.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd937fe640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c003fd0 0x7fdd9414f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:11.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 -- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd940b3b70 msgr2=0x7fdd940c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd940b3b70 0x7fdd940c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 -- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c003fd0 msgr2=0x7fdd9414f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:11.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c003fd0 0x7fdd9414f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:11.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 -- 172.21.15.125:0/641678937 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940c9d20 con 0x7fdd940ae700 2023-10-18T12:01:11.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd92ffd640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd940b3b70 0x7fdd940c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:11.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd937fe640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c003fd0 0x7fdd9414f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:11.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd93fff640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd940ae700 0x7fdd940aeb80 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7fdd940b6e10 tx=0x7fdd8c00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:11.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd7bfff640 1 -- 172.21.15.125:0/641678937 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c00c8f0 con 0x7fdd940ae700 2023-10-18T12:01:11.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd7bfff640 1 -- 172.21.15.125:0/641678937 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd8c00cb80 con 0x7fdd940ae700 2023-10-18T12:01:11.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd940ca000 con 0x7fdd940ae700 2023-10-18T12:01:11.858 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd7bfff640 1 -- 172.21.15.125:0/641678937 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c011460 con 0x7fdd940ae700 2023-10-18T12:01:11.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.845+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd940ca5c0 con 0x7fdd940ae700 2023-10-18T12:01:11.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.846+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd940ca750 con 0x7fdd940ae700 2023-10-18T12:01:11.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.847+0000 7fdd7bfff640 1 -- 172.21.15.125:0/641678937 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fdd8c0115c0 con 0x7fdd940ae700 2023-10-18T12:01:11.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.847+0000 7fdd7bfff640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fdd8006b080 0x7fdd8006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:11.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.847+0000 7fdd7bfff640 1 -- 172.21.15.125:0/641678937 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fdd8c08a380 con 0x7fdd940ae700 2023-10-18T12:01:11.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.847+0000 7fdd937fe640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fdd8006b080 0x7fdd8006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:11.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.848+0000 7fdd937fe640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fdd8006b080 0x7fdd8006d540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd84006660 tx=0x7fdd84006c40 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:11.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:11.851+0000 7fdd7bfff640 1 -- 172.21.15.125:0/641678937 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd8c054db0 con 0x7fdd940ae700 2023-10-18T12:01:12.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.017+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd940690e0 con 0x7fdd940ae700 2023-10-18T12:01:12.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.018+0000 7fdd7bfff640 1 -- 172.21.15.125:0/641678937 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+16922 (secure 0 0 0) 0x7fdd8c0583d0 con 0x7fdd940ae700 2023-10-18T12:01:12.021 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:12.021 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":242,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:10.109552+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:12.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fdd8006b080 msgr2=0x7fdd8006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fdd8006b080 0x7fdd8006d540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd84006660 tx=0x7fdd84006c40 comp rx=0 tx=0).stop 2023-10-18T12:01:12.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd940ae700 msgr2=0x7fdd940aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd940ae700 0x7fdd940aeb80 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7fdd940b6e10 tx=0x7fdd8c00eae0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 shutdown_connections 2023-10-18T12:01:12.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fdd8006b080 0x7fdd8006d540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdd940ae700 0x7fdd940aeb80 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdd940b3b70 0x7fdd940c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 --2- 172.21.15.125:0/641678937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd7c003fd0 0x7fdd9414f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 >> 172.21.15.125:0/641678937 conn(0x7fdd940d0d60 msgr2=0x7fdd940ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:12.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 shutdown_connections 2023-10-18T12:01:12.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.020+0000 7fdd99d41640 1 -- 172.21.15.125:0/641678937 wait complete. 2023-10-18T12:01:12.034 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T12:01:12.034 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:01:12.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.134+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1376063793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 msgr2=0x7f2a5c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.134+0000 7f2a7a70f640 1 --2- 172.21.15.125:0/1376063793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 0x7f2a5c004c40 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c260 tx=0x7f2a5c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.134+0000 7f2a78cac640 1 -- 172.21.15.125:0/1376063793 reap_dead start 2023-10-18T12:01:12.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.135+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1376063793 shutdown_connections 2023-10-18T12:01:12.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.135+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1376063793 >> 172.21.15.125:0/1376063793 conn(0x7f2a740d0d60 msgr2=0x7f2a740d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:12.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.135+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1376063793 shutdown_connections 2023-10-18T12:01:12.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.135+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1376063793 wait complete. 2023-10-18T12:01:12.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.136+0000 7f2a7a70f640 1 Processor -- start 2023-10-18T12:01:12.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.136+0000 7f2a7a70f640 1 -- start start 2023-10-18T12:01:12.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a7a70f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 0x7f2a741f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a7a70f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a74154d50 0x7f2a7415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a7a70f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a741551d0 0x7f2a7415df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a7a70f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a741f5a50 con 0x7f2a5c0047c0 2023-10-18T12:01:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a7a70f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a741f5770 con 0x7f2a741551d0 2023-10-18T12:01:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a7a70f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a74154270 con 0x7f2a74154d50 2023-10-18T12:01:12.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 0x7f2a741f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 0x7f2a741f0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35932/0 (socket says 172.21.15.125:35932) 2023-10-18T12:01:12.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 -- 172.21.15.125:0/1916968248 learned_addr learned my addr 172.21.15.125:0/1916968248 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:12.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a737fe640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a74154d50 0x7f2a7415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a78cac640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a741551d0 0x7f2a7415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 -- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a74154d50 msgr2=0x7f2a7415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a74154d50 0x7f2a7415da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 -- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a741551d0 msgr2=0x7f2a7415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a741551d0 0x7f2a7415df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 -- 172.21.15.125:0/1916968248 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74154550 con 0x7f2a5c0047c0 2023-10-18T12:01:12.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a73fff640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 0x7f2a741f0db0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f2a64011ab0 tx=0x7f2a64011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:12.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a70ff9640 1 -- 172.21.15.125:0/1916968248 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400fe70 con 0x7f2a5c0047c0 2023-10-18T12:01:12.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a70ff9640 1 -- 172.21.15.125:0/1916968248 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a64004050 con 0x7f2a5c0047c0 2023-10-18T12:01:12.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a70ff9640 1 -- 172.21.15.125:0/1916968248 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a64004440 con 0x7f2a5c0047c0 2023-10-18T12:01:12.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a74154830 con 0x7f2a5c0047c0 2023-10-18T12:01:12.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.137+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7415b6d0 con 0x7f2a5c0047c0 2023-10-18T12:01:12.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.138+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a740b15f0 con 0x7f2a5c0047c0 2023-10-18T12:01:12.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.139+0000 7f2a70ff9640 1 -- 172.21.15.125:0/1916968248 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2a640035f0 con 0x7f2a5c0047c0 2023-10-18T12:01:12.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.139+0000 7f2a70ff9640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2a6006b080 0x7f2a6006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.139+0000 7f2a70ff9640 1 -- 172.21.15.125:0/1916968248 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f2a6408a920 con 0x7f2a5c0047c0 2023-10-18T12:01:12.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.140+0000 7f2a737fe640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2a6006b080 0x7f2a6006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.140+0000 7f2a737fe640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2a6006b080 0x7f2a6006d540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a68034000 tx=0x7f2a68001a00 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:12.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.144+0000 7f2a70ff9640 1 -- 172.21.15.125:0/1916968248 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a64055520 con 0x7f2a5c0047c0 2023-10-18T12:01:12.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.307+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a740d1910 con 0x7f2a6006b080 2023-10-18T12:01:12.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.316+0000 7f2a70ff9640 1 -- 172.21.15.125:0/1916968248 <== mgr.9236 v2:172.21.15.139:6832/3422518215 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290152 (secure 0 0 0) 0x7f2a68006050 con 0x7f2a6006b080 2023-10-18T12:01:12.323 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:12.324 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:01:12.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2a6006b080 msgr2=0x7f2a6006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.326 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2a6006b080 0x7f2a6006d540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a68034000 tx=0x7f2a68001a00 comp rx=0 tx=0).stop 2023-10-18T12:01:12.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 msgr2=0x7f2a741f0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 0x7f2a741f0db0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f2a64011ab0 tx=0x7f2a64011f70 comp rx=0 tx=0).stop 2023-10-18T12:01:12.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 shutdown_connections 2023-10-18T12:01:12.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2a6006b080 0x7f2a6006d540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.327 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a741551d0 0x7f2a7415df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a74154d50 0x7f2a7415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 --2- 172.21.15.125:0/1916968248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a5c0047c0 0x7f2a741f0db0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 >> 172.21.15.125:0/1916968248 conn(0x7f2a740d0d60 msgr2=0x7f2a740b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:12.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 shutdown_connections 2023-10-18T12:01:12.328 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.319+0000 7f2a7a70f640 1 -- 172.21.15.125:0/1916968248 wait complete. 2023-10-18T12:01:12.332 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-18T12:01:10.835757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":679,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":417,"ondisk_log_size":417,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420168,"kb_used_data":24968,"kb_used_omap":12,"kb_used_meta":395059,"kb_avail":754554552,"statfs":{"total":773094113280,"available":772663861248,"internally_reserved":0,"allocated":25567232,"data_stored":13706052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":404540984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000863"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.128323+0000","last_change":"2023-10-18T12:01:09.126968+0000","last_active":"2023-10-18T12:01:09.128323+0000","last_peered":"2023-10-18T12:01:09.128323+0000","last_clean":"2023-10-18T12:01:09.128323+0000","last_became_active":"2023-10-18T12:01:09.126705+0000","last_became_peered":"2023-10-18T12:01:09.126705+0000","last_unstale":"2023-10-18T12:01:09.128323+0000","last_undegraded":"2023-10-18T12:01:09.128323+0000","last_fullsized":"2023-10-18T12:01:09.128323+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:14.190713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.130626+0000","last_change":"2023-10-18T12:01:09.125772+0000","last_active":"2023-10-18T12:01:09.130626+0000","last_peered":"2023-10-18T12:01:09.130626+0000","last_clean":"2023-10-18T12:01:09.130626+0000","last_became_active":"2023-10-18T12:01:09.125508+0000","last_became_peered":"2023-10-18T12:01:09.125508+0000","last_unstale":"2023-10-18T12:01:09.130626+0000","last_undegraded":"2023-10-18T12:01:09.130626+0000","last_fullsized":"2023-10-18T12:01:09.130626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:53.920813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.125260+0000","last_change":"2023-10-18T12:01:09.124764+0000","last_active":"2023-10-18T12:01:09.125260+0000","last_peered":"2023-10-18T12:01:09.125260+0000","last_clean":"2023-10-18T12:01:09.125260+0000","last_became_active":"2023-10-18T12:01:09.124472+0000","last_became_peered":"2023-10-18T12:01:09.124472+0000","last_unstale":"2023-10-18T12:01:09.125260+0000","last_undegraded":"2023-10-18T12:01:09.125260+0000","last_fullsized":"2023-10-18T12:01:09.125260+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:47.018047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133236+0000","last_change":"2023-10-18T12:01:09.132202+0000","last_active":"2023-10-18T12:01:09.133236+0000","last_peered":"2023-10-18T12:01:09.133236+0000","last_clean":"2023-10-18T12:01:09.133236+0000","last_became_active":"2023-10-18T12:01:09.131928+0000","last_became_peered":"2023-10-18T12:01:09.131928+0000","last_unstale":"2023-10-18T12:01:09.133236+0000","last_undegraded":"2023-10-18T12:01:09.133236+0000","last_fullsized":"2023-10-18T12:01:09.133236+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:51.005840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136276+0000","last_change":"2023-10-18T12:01:09.133870+0000","last_active":"2023-10-18T12:01:09.136276+0000","last_peered":"2023-10-18T12:01:09.136276+0000","last_clean":"2023-10-18T12:01:09.136276+0000","last_became_active":"2023-10-18T12:01:09.133478+0000","last_became_peered":"2023-10-18T12:01:09.133478+0000","last_unstale":"2023-10-18T12:01:09.136276+0000","last_undegraded":"2023-10-18T12:01:09.136276+0000","last_fullsized":"2023-10-18T12:01:09.136276+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:15.623643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.129938+0000","last_change":"2023-10-18T12:01:09.129113+0000","last_active":"2023-10-18T12:01:09.129938+0000","last_peered":"2023-10-18T12:01:09.129938+0000","last_clean":"2023-10-18T12:01:09.129938+0000","last_became_active":"2023-10-18T12:01:09.128785+0000","last_became_peered":"2023-10-18T12:01:09.128785+0000","last_unstale":"2023-10-18T12:01:09.129938+0000","last_undegraded":"2023-10-18T12:01:09.129938+0000","last_fullsized":"2023-10-18T12:01:09.129938+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:25:20.201191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127495+0000","last_change":"2023-10-18T12:01:09.125901+0000","last_active":"2023-10-18T12:01:09.127495+0000","last_peered":"2023-10-18T12:01:09.127495+0000","last_clean":"2023-10-18T12:01:09.127495+0000","last_became_active":"2023-10-18T12:01:09.125390+0000","last_became_peered":"2023-10-18T12:01:09.125390+0000","last_unstale":"2023-10-18T12:01:09.127495+0000","last_undegraded":"2023-10-18T12:01:09.127495+0000","last_fullsized":"2023-10-18T12:01:09.127495+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:16:51.521674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136838+0000","last_change":"2023-10-18T12:01:09.135081+0000","last_active":"2023-10-18T12:01:09.136838+0000","last_peered":"2023-10-18T12:01:09.136838+0000","last_clean":"2023-10-18T12:01:09.136838+0000","last_became_active":"2023-10-18T12:01:09.134801+0000","last_became_peered":"2023-10-18T12:01:09.134801+0000","last_unstale":"2023-10-18T12:01:09.136838+0000","last_undegraded":"2023-10-18T12:01:09.136838+0000","last_fullsized":"2023-10-18T12:01:09.136838+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:50.011284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136526+0000","last_change":"2023-10-18T12:01:09.135615+0000","last_active":"2023-10-18T12:01:09.136526+0000","last_peered":"2023-10-18T12:01:09.136526+0000","last_clean":"2023-10-18T12:01:09.136526+0000","last_became_active":"2023-10-18T12:01:09.135123+0000","last_became_peered":"2023-10-18T12:01:09.135123+0000","last_unstale":"2023-10-18T12:01:09.136526+0000","last_undegraded":"2023-10-18T12:01:09.136526+0000","last_fullsized":"2023-10-18T12:01:09.136526+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:12.571415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132608+0000","last_change":"2023-10-18T12:01:09.130673+0000","last_active":"2023-10-18T12:01:09.132608+0000","last_peered":"2023-10-18T12:01:09.132608+0000","last_clean":"2023-10-18T12:01:09.132608+0000","last_became_active":"2023-10-18T12:01:09.130094+0000","last_became_peered":"2023-10-18T12:01:09.130094+0000","last_unstale":"2023-10-18T12:01:09.132608+0000","last_undegraded":"2023-10-18T12:01:09.132608+0000","last_fullsized":"2023-10-18T12:01:09.132608+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:48.689497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137102+0000","last_change":"2023-10-18T12:01:09.135828+0000","last_active":"2023-10-18T12:01:09.137102+0000","last_peered":"2023-10-18T12:01:09.137102+0000","last_clean":"2023-10-18T12:01:09.137102+0000","last_became_active":"2023-10-18T12:01:09.135579+0000","last_became_peered":"2023-10-18T12:01:09.135579+0000","last_unstale":"2023-10-18T12:01:09.137102+0000","last_undegraded":"2023-10-18T12:01:09.137102+0000","last_fullsized":"2023-10-18T12:01:09.137102+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:20.398187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.134195+0000","last_change":"2023-10-18T12:01:09.132939+0000","last_active":"2023-10-18T12:01:09.134195+0000","last_peered":"2023-10-18T12:01:09.134195+0000","last_clean":"2023-10-18T12:01:09.134195+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:09.134195+0000","last_undegraded":"2023-10-18T12:01:09.134195+0000","last_fullsized":"2023-10-18T12:01:09.134195+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:04:42.744489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133239+0000","last_change":"2023-10-18T12:01:09.129764+0000","last_active":"2023-10-18T12:01:09.133239+0000","last_peered":"2023-10-18T12:01:09.133239+0000","last_clean":"2023-10-18T12:01:09.133239+0000","last_became_active":"2023-10-18T12:01:09.129483+0000","last_became_peered":"2023-10-18T12:01:09.129483+0000","last_unstale":"2023-10-18T12:01:09.133239+0000","last_undegraded":"2023-10-18T12:01:09.133239+0000","last_fullsized":"2023-10-18T12:01:09.133239+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:52:21.068558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.135525+0000","last_change":"2023-10-18T12:01:09.133024+0000","last_active":"2023-10-18T12:01:09.135525+0000","last_peered":"2023-10-18T12:01:09.135525+0000","last_clean":"2023-10-18T12:01:09.135525+0000","last_became_active":"2023-10-18T12:01:09.132742+0000","last_became_peered":"2023-10-18T12:01:09.132742+0000","last_unstale":"2023-10-18T12:01:09.135525+0000","last_undegraded":"2023-10-18T12:01:09.135525+0000","last_fullsized":"2023-10-18T12:01:09.135525+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:58.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132719+0000","last_change":"2023-10-18T12:01:09.131147+0000","last_active":"2023-10-18T12:01:09.132719+0000","last_peered":"2023-10-18T12:01:09.132719+0000","last_clean":"2023-10-18T12:01:09.132719+0000","last_became_active":"2023-10-18T12:01:09.130682+0000","last_became_peered":"2023-10-18T12:01:09.130682+0000","last_unstale":"2023-10-18T12:01:09.132719+0000","last_undegraded":"2023-10-18T12:01:09.132719+0000","last_fullsized":"2023-10-18T12:01:09.132719+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:56.556729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.124565+0000","last_change":"2023-10-18T12:01:09.122098+0000","last_active":"2023-10-18T12:01:09.124565+0000","last_peered":"2023-10-18T12:01:09.124565+0000","last_clean":"2023-10-18T12:01:09.124565+0000","last_became_active":"2023-10-18T12:01:09.121845+0000","last_became_peered":"2023-10-18T12:01:09.121845+0000","last_unstale":"2023-10-18T12:01:09.124565+0000","last_undegraded":"2023-10-18T12:01:09.124565+0000","last_fullsized":"2023-10-18T12:01:09.124565+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:10.895565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136613+0000","last_change":"2023-10-18T12:01:09.133999+0000","last_active":"2023-10-18T12:01:09.136613+0000","last_peered":"2023-10-18T12:01:09.136613+0000","last_clean":"2023-10-18T12:01:09.136613+0000","last_became_active":"2023-10-18T12:01:09.133725+0000","last_became_peered":"2023-10-18T12:01:09.133725+0000","last_unstale":"2023-10-18T12:01:09.136613+0000","last_undegraded":"2023-10-18T12:01:09.136613+0000","last_fullsized":"2023-10-18T12:01:09.136613+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:33.136459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.125460+0000","last_change":"2023-10-18T12:01:09.124908+0000","last_active":"2023-10-18T12:01:09.125460+0000","last_peered":"2023-10-18T12:01:09.125460+0000","last_clean":"2023-10-18T12:01:09.125460+0000","last_became_active":"2023-10-18T12:01:09.124565+0000","last_became_peered":"2023-10-18T12:01:09.124565+0000","last_unstale":"2023-10-18T12:01:09.125460+0000","last_undegraded":"2023-10-18T12:01:09.125460+0000","last_fullsized":"2023-10-18T12:01:09.125460+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:44.946975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137497+0000","last_change":"2023-10-18T12:01:09.135302+0000","last_active":"2023-10-18T12:01:09.137497+0000","last_peered":"2023-10-18T12:01:09.137497+0000","last_clean":"2023-10-18T12:01:09.137497+0000","last_became_active":"2023-10-18T12:01:09.134978+0000","last_became_peered":"2023-10-18T12:01:09.134978+0000","last_unstale":"2023-10-18T12:01:09.137497+0000","last_undegraded":"2023-10-18T12:01:09.137497+0000","last_fullsized":"2023-10-18T12:01:09.137497+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:47.648103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137927+0000","last_change":"2023-10-18T12:01:09.136717+0000","last_active":"2023-10-18T12:01:09.137927+0000","last_peered":"2023-10-18T12:01:09.137927+0000","last_clean":"2023-10-18T12:01:09.137927+0000","last_became_active":"2023-10-18T12:01:09.136461+0000","last_became_peered":"2023-10-18T12:01:09.136461+0000","last_unstale":"2023-10-18T12:01:09.137927+0000","last_undegraded":"2023-10-18T12:01:09.137927+0000","last_fullsized":"2023-10-18T12:01:09.137927+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:04:30.217797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133477+0000","last_change":"2023-10-18T12:01:09.130016+0000","last_active":"2023-10-18T12:01:09.133477+0000","last_peered":"2023-10-18T12:01:09.133477+0000","last_clean":"2023-10-18T12:01:09.133477+0000","last_became_active":"2023-10-18T12:01:09.129718+0000","last_became_peered":"2023-10-18T12:01:09.129718+0000","last_unstale":"2023-10-18T12:01:09.133477+0000","last_undegraded":"2023-10-18T12:01:09.133477+0000","last_fullsized":"2023-10-18T12:01:09.133477+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:57.709530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133066+0000","last_change":"2023-10-18T12:01:09.129706+0000","last_active":"2023-10-18T12:01:09.133066+0000","last_peered":"2023-10-18T12:01:09.133066+0000","last_clean":"2023-10-18T12:01:09.133066+0000","last_became_active":"2023-10-18T12:01:09.129413+0000","last_became_peered":"2023-10-18T12:01:09.129413+0000","last_unstale":"2023-10-18T12:01:09.133066+0000","last_undegraded":"2023-10-18T12:01:09.133066+0000","last_fullsized":"2023-10-18T12:01:09.133066+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:12.532840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136589+0000","last_change":"2023-10-18T12:01:09.133291+0000","last_active":"2023-10-18T12:01:09.136589+0000","last_peered":"2023-10-18T12:01:09.136589+0000","last_clean":"2023-10-18T12:01:09.136589+0000","last_became_active":"2023-10-18T12:01:09.132968+0000","last_became_peered":"2023-10-18T12:01:09.132968+0000","last_unstale":"2023-10-18T12:01:09.136589+0000","last_undegraded":"2023-10-18T12:01:09.136589+0000","last_fullsized":"2023-10-18T12:01:09.136589+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:10.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133058+0000","last_change":"2023-10-18T12:01:09.128954+0000","last_active":"2023-10-18T12:01:09.133058+0000","last_peered":"2023-10-18T12:01:09.133058+0000","last_clean":"2023-10-18T12:01:09.133058+0000","last_became_active":"2023-10-18T12:01:09.128655+0000","last_became_peered":"2023-10-18T12:01:09.128655+0000","last_unstale":"2023-10-18T12:01:09.133058+0000","last_undegraded":"2023-10-18T12:01:09.133058+0000","last_fullsized":"2023-10-18T12:01:09.133058+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:01.040240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.125729+0000","last_change":"2023-10-18T12:01:09.123355+0000","last_active":"2023-10-18T12:01:09.125729+0000","last_peered":"2023-10-18T12:01:09.125729+0000","last_clean":"2023-10-18T12:01:09.125729+0000","last_became_active":"2023-10-18T12:01:09.123096+0000","last_became_peered":"2023-10-18T12:01:09.123096+0000","last_unstale":"2023-10-18T12:01:09.125729+0000","last_undegraded":"2023-10-18T12:01:09.125729+0000","last_fullsized":"2023-10-18T12:01:09.125729+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:00:59.640033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127384+0000","last_change":"2023-10-18T12:01:09.124947+0000","last_active":"2023-10-18T12:01:09.127384+0000","last_peered":"2023-10-18T12:01:09.127384+0000","last_clean":"2023-10-18T12:01:09.127384+0000","last_became_active":"2023-10-18T12:01:09.124666+0000","last_became_peered":"2023-10-18T12:01:09.124666+0000","last_unstale":"2023-10-18T12:01:09.127384+0000","last_undegraded":"2023-10-18T12:01:09.127384+0000","last_fullsized":"2023-10-18T12:01:09.127384+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:26.664117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132971+0000","last_change":"2023-10-18T12:01:09.131674+0000","last_active":"2023-10-18T12:01:09.132971+0000","last_peered":"2023-10-18T12:01:09.132971+0000","last_clean":"2023-10-18T12:01:09.132971+0000","last_became_active":"2023-10-18T12:01:09.131410+0000","last_became_peered":"2023-10-18T12:01:09.131410+0000","last_unstale":"2023-10-18T12:01:09.132971+0000","last_undegraded":"2023-10-18T12:01:09.132971+0000","last_fullsized":"2023-10-18T12:01:09.132971+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:32.387861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.135855+0000","last_change":"2023-10-18T12:01:09.134865+0000","last_active":"2023-10-18T12:01:09.135855+0000","last_peered":"2023-10-18T12:01:09.135855+0000","last_clean":"2023-10-18T12:01:09.135855+0000","last_became_active":"2023-10-18T12:01:09.134583+0000","last_became_peered":"2023-10-18T12:01:09.134583+0000","last_unstale":"2023-10-18T12:01:09.135855+0000","last_undegraded":"2023-10-18T12:01:09.135855+0000","last_fullsized":"2023-10-18T12:01:09.135855+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:30.258369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.135747+0000","last_change":"2023-10-18T12:01:09.132312+0000","last_active":"2023-10-18T12:01:09.135747+0000","last_peered":"2023-10-18T12:01:09.135747+0000","last_clean":"2023-10-18T12:01:09.135747+0000","last_became_active":"2023-10-18T12:01:09.131781+0000","last_became_peered":"2023-10-18T12:01:09.131781+0000","last_unstale":"2023-10-18T12:01:09.135747+0000","last_undegraded":"2023-10-18T12:01:09.135747+0000","last_fullsized":"2023-10-18T12:01:09.135747+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:13:50.406605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136177+0000","last_change":"2023-10-18T12:01:09.133664+0000","last_active":"2023-10-18T12:01:09.136177+0000","last_peered":"2023-10-18T12:01:09.136177+0000","last_clean":"2023-10-18T12:01:09.136177+0000","last_became_active":"2023-10-18T12:01:09.133406+0000","last_became_peered":"2023-10-18T12:01:09.133406+0000","last_unstale":"2023-10-18T12:01:09.136177+0000","last_undegraded":"2023-10-18T12:01:09.136177+0000","last_fullsized":"2023-10-18T12:01:09.136177+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:03.436140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.131693+0000","last_change":"2023-10-18T12:01:09.130142+0000","last_active":"2023-10-18T12:01:09.131693+0000","last_peered":"2023-10-18T12:01:09.131693+0000","last_clean":"2023-10-18T12:01:09.131693+0000","last_became_active":"2023-10-18T12:01:09.129884+0000","last_became_peered":"2023-10-18T12:01:09.129884+0000","last_unstale":"2023-10-18T12:01:09.131693+0000","last_undegraded":"2023-10-18T12:01:09.131693+0000","last_fullsized":"2023-10-18T12:01:09.131693+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:40.659316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137256+0000","last_change":"2023-10-18T12:01:09.136066+0000","last_active":"2023-10-18T12:01:09.137256+0000","last_peered":"2023-10-18T12:01:09.137256+0000","last_clean":"2023-10-18T12:01:09.137256+0000","last_became_active":"2023-10-18T12:01:09.135802+0000","last_became_peered":"2023-10-18T12:01:09.135802+0000","last_unstale":"2023-10-18T12:01:09.137256+0000","last_undegraded":"2023-10-18T12:01:09.137256+0000","last_fullsized":"2023-10-18T12:01:09.137256+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:48.783775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.121736+0000","last_change":"2023-10-18T12:01:09.120812+0000","last_active":"2023-10-18T12:01:09.121736+0000","last_peered":"2023-10-18T12:01:09.121736+0000","last_clean":"2023-10-18T12:01:09.121736+0000","last_became_active":"2023-10-18T12:01:09.120561+0000","last_became_peered":"2023-10-18T12:01:09.120561+0000","last_unstale":"2023-10-18T12:01:09.121736+0000","last_undegraded":"2023-10-18T12:01:09.121736+0000","last_fullsized":"2023-10-18T12:01:09.121736+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:25.745623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"241'4","reported_seq":15,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.145642+0000","last_change":"2023-10-18T12:01:08.112445+0000","last_active":"2023-10-18T12:01:09.145642+0000","last_peered":"2023-10-18T12:01:09.145642+0000","last_clean":"2023-10-18T12:01:09.145642+0000","last_became_active":"2023-10-18T12:01:08.112017+0000","last_became_peered":"2023-10-18T12:01:08.112017+0000","last_unstale":"2023-10-18T12:01:09.145642+0000","last_undegraded":"2023-10-18T12:01:09.145642+0000","last_fullsized":"2023-10-18T12:01:09.145642+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:11.607481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136639+0000","last_change":"2023-10-18T12:01:09.134916+0000","last_active":"2023-10-18T12:01:09.136639+0000","last_peered":"2023-10-18T12:01:09.136639+0000","last_clean":"2023-10-18T12:01:09.136639+0000","last_became_active":"2023-10-18T12:01:09.134630+0000","last_became_peered":"2023-10-18T12:01:09.134630+0000","last_unstale":"2023-10-18T12:01:09.136639+0000","last_undegraded":"2023-10-18T12:01:09.136639+0000","last_fullsized":"2023-10-18T12:01:09.136639+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:44.846096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146787+0000","last_change":"2023-10-18T12:01:08.119870+0000","last_active":"2023-10-18T12:01:09.146787+0000","last_peered":"2023-10-18T12:01:09.146787+0000","last_clean":"2023-10-18T12:01:09.146787+0000","last_became_active":"2023-10-18T12:01:08.119586+0000","last_became_peered":"2023-10-18T12:01:08.119586+0000","last_unstale":"2023-10-18T12:01:09.146787+0000","last_undegraded":"2023-10-18T12:01:09.146787+0000","last_fullsized":"2023-10-18T12:01:09.146787+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:16.288493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.134216+0000","last_change":"2023-10-18T12:01:09.130894+0000","last_active":"2023-10-18T12:01:09.134216+0000","last_peered":"2023-10-18T12:01:09.134216+0000","last_clean":"2023-10-18T12:01:09.134216+0000","last_became_active":"2023-10-18T12:01:09.130614+0000","last_became_peered":"2023-10-18T12:01:09.130614+0000","last_unstale":"2023-10-18T12:01:09.134216+0000","last_undegraded":"2023-10-18T12:01:09.134216+0000","last_fullsized":"2023-10-18T12:01:09.134216+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:54.509159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.145024+0000","last_change":"2023-10-18T12:01:08.115174+0000","last_active":"2023-10-18T12:01:09.145024+0000","last_peered":"2023-10-18T12:01:09.145024+0000","last_clean":"2023-10-18T12:01:09.145024+0000","last_became_active":"2023-10-18T12:01:08.114546+0000","last_became_peered":"2023-10-18T12:01:08.114546+0000","last_unstale":"2023-10-18T12:01:09.145024+0000","last_undegraded":"2023-10-18T12:01:09.145024+0000","last_fullsized":"2023-10-18T12:01:09.145024+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.123903+0000","last_change":"2023-10-18T12:01:09.122601+0000","last_active":"2023-10-18T12:01:09.123903+0000","last_peered":"2023-10-18T12:01:09.123903+0000","last_clean":"2023-10-18T12:01:09.123903+0000","last_became_active":"2023-10-18T12:01:09.122334+0000","last_became_peered":"2023-10-18T12:01:09.122334+0000","last_unstale":"2023-10-18T12:01:09.123903+0000","last_undegraded":"2023-10-18T12:01:09.123903+0000","last_fullsized":"2023-10-18T12:01:09.123903+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:56.556729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146442+0000","last_change":"2023-10-18T12:01:08.116478+0000","last_active":"2023-10-18T12:01:09.146442+0000","last_peered":"2023-10-18T12:01:09.146442+0000","last_clean":"2023-10-18T12:01:09.146442+0000","last_became_active":"2023-10-18T12:01:08.116181+0000","last_became_peered":"2023-10-18T12:01:08.116181+0000","last_unstale":"2023-10-18T12:01:09.146442+0000","last_undegraded":"2023-10-18T12:01:09.146442+0000","last_fullsized":"2023-10-18T12:01:09.146442+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:10.087064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133703+0000","last_change":"2023-10-18T12:01:09.130166+0000","last_active":"2023-10-18T12:01:09.133703+0000","last_peered":"2023-10-18T12:01:09.133703+0000","last_clean":"2023-10-18T12:01:09.133703+0000","last_became_active":"2023-10-18T12:01:09.129725+0000","last_became_peered":"2023-10-18T12:01:09.129725+0000","last_unstale":"2023-10-18T12:01:09.133703+0000","last_undegraded":"2023-10-18T12:01:09.133703+0000","last_fullsized":"2023-10-18T12:01:09.133703+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:00:18.981967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.141464+0000","last_change":"2023-10-18T12:01:08.107385+0000","last_active":"2023-10-18T12:01:09.141464+0000","last_peered":"2023-10-18T12:01:09.141464+0000","last_clean":"2023-10-18T12:01:09.141464+0000","last_became_active":"2023-10-18T12:01:08.107129+0000","last_became_peered":"2023-10-18T12:01:08.107129+0000","last_unstale":"2023-10-18T12:01:09.141464+0000","last_undegraded":"2023-10-18T12:01:09.141464+0000","last_fullsized":"2023-10-18T12:01:09.141464+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:43.838857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.135346+0000","last_change":"2023-10-18T12:01:09.133737+0000","last_active":"2023-10-18T12:01:09.135346+0000","last_peered":"2023-10-18T12:01:09.135346+0000","last_clean":"2023-10-18T12:01:09.135346+0000","last_became_active":"2023-10-18T12:01:09.133487+0000","last_became_peered":"2023-10-18T12:01:09.133487+0000","last_unstale":"2023-10-18T12:01:09.135346+0000","last_undegraded":"2023-10-18T12:01:09.135346+0000","last_fullsized":"2023-10-18T12:01:09.135346+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:40.465745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.114949+0000","last_change":"2023-10-18T12:01:08.119435+0000","last_active":"2023-10-18T12:01:09.114949+0000","last_peered":"2023-10-18T12:01:09.114949+0000","last_clean":"2023-10-18T12:01:09.114949+0000","last_became_active":"2023-10-18T12:01:08.118740+0000","last_became_peered":"2023-10-18T12:01:08.118740+0000","last_unstale":"2023-10-18T12:01:09.114949+0000","last_undegraded":"2023-10-18T12:01:09.114949+0000","last_fullsized":"2023-10-18T12:01:09.114949+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:54.172871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133665+0000","last_change":"2023-10-18T12:01:09.132487+0000","last_active":"2023-10-18T12:01:09.133665+0000","last_peered":"2023-10-18T12:01:09.133665+0000","last_clean":"2023-10-18T12:01:09.133665+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:09.133665+0000","last_undegraded":"2023-10-18T12:01:09.133665+0000","last_fullsized":"2023-10-18T12:01:09.133665+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:49.358178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.117849+0000","last_change":"2023-10-18T12:01:08.115982+0000","last_active":"2023-10-18T12:01:09.117849+0000","last_peered":"2023-10-18T12:01:09.117849+0000","last_clean":"2023-10-18T12:01:09.117849+0000","last_became_active":"2023-10-18T12:01:08.115682+0000","last_became_peered":"2023-10-18T12:01:08.115682+0000","last_unstale":"2023-10-18T12:01:09.117849+0000","last_undegraded":"2023-10-18T12:01:09.117849+0000","last_fullsized":"2023-10-18T12:01:09.117849+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:36.789486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133512+0000","last_change":"2023-10-18T12:01:09.130010+0000","last_active":"2023-10-18T12:01:09.133512+0000","last_peered":"2023-10-18T12:01:09.133512+0000","last_clean":"2023-10-18T12:01:09.133512+0000","last_became_active":"2023-10-18T12:01:09.129744+0000","last_became_peered":"2023-10-18T12:01:09.129744+0000","last_unstale":"2023-10-18T12:01:09.133512+0000","last_undegraded":"2023-10-18T12:01:09.133512+0000","last_fullsized":"2023-10-18T12:01:09.133512+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:50.179465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.114469+0000","last_change":"2023-10-18T12:01:08.110833+0000","last_active":"2023-10-18T12:01:09.114469+0000","last_peered":"2023-10-18T12:01:09.114469+0000","last_clean":"2023-10-18T12:01:09.114469+0000","last_became_active":"2023-10-18T12:01:08.110005+0000","last_became_peered":"2023-10-18T12:01:08.110005+0000","last_unstale":"2023-10-18T12:01:09.114469+0000","last_undegraded":"2023-10-18T12:01:09.114469+0000","last_fullsized":"2023-10-18T12:01:09.114469+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:32.311648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133039+0000","last_change":"2023-10-18T12:01:09.129659+0000","last_active":"2023-10-18T12:01:09.133039+0000","last_peered":"2023-10-18T12:01:09.133039+0000","last_clean":"2023-10-18T12:01:09.133039+0000","last_became_active":"2023-10-18T12:01:09.129388+0000","last_became_peered":"2023-10-18T12:01:09.129388+0000","last_unstale":"2023-10-18T12:01:09.133039+0000","last_undegraded":"2023-10-18T12:01:09.133039+0000","last_fullsized":"2023-10-18T12:01:09.133039+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:35:25.388252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.115357+0000","last_change":"2023-10-18T12:01:08.118321+0000","last_active":"2023-10-18T12:01:09.115357+0000","last_peered":"2023-10-18T12:01:09.115357+0000","last_clean":"2023-10-18T12:01:09.115357+0000","last_became_active":"2023-10-18T12:01:08.117995+0000","last_became_peered":"2023-10-18T12:01:08.117995+0000","last_unstale":"2023-10-18T12:01:09.115357+0000","last_undegraded":"2023-10-18T12:01:09.115357+0000","last_fullsized":"2023-10-18T12:01:09.115357+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:46.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136242+0000","last_change":"2023-10-18T12:01:09.134396+0000","last_active":"2023-10-18T12:01:09.136242+0000","last_peered":"2023-10-18T12:01:09.136242+0000","last_clean":"2023-10-18T12:01:09.136242+0000","last_became_active":"2023-10-18T12:01:09.134144+0000","last_became_peered":"2023-10-18T12:01:09.134144+0000","last_unstale":"2023-10-18T12:01:09.136242+0000","last_undegraded":"2023-10-18T12:01:09.136242+0000","last_fullsized":"2023-10-18T12:01:09.136242+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:00.179085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"241'3","reported_seq":14,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.141686+0000","last_change":"2023-10-18T12:01:08.118871+0000","last_active":"2023-10-18T12:01:09.141686+0000","last_peered":"2023-10-18T12:01:09.141686+0000","last_clean":"2023-10-18T12:01:09.141686+0000","last_became_active":"2023-10-18T12:01:08.118572+0000","last_became_peered":"2023-10-18T12:01:08.118572+0000","last_unstale":"2023-10-18T12:01:09.141686+0000","last_undegraded":"2023-10-18T12:01:09.141686+0000","last_fullsized":"2023-10-18T12:01:09.141686+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:45.510206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.121494+0000","last_change":"2023-10-18T12:01:09.120444+0000","last_active":"2023-10-18T12:01:09.121494+0000","last_peered":"2023-10-18T12:01:09.121494+0000","last_clean":"2023-10-18T12:01:09.121494+0000","last_became_active":"2023-10-18T12:01:09.120018+0000","last_became_peered":"2023-10-18T12:01:09.120018+0000","last_unstale":"2023-10-18T12:01:09.121494+0000","last_undegraded":"2023-10-18T12:01:09.121494+0000","last_fullsized":"2023-10-18T12:01:09.121494+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:45.901809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"241'6","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.148493+0000","last_change":"2023-10-18T12:01:08.117890+0000","last_active":"2023-10-18T12:01:09.148493+0000","last_peered":"2023-10-18T12:01:09.148493+0000","last_clean":"2023-10-18T12:01:09.148493+0000","last_became_active":"2023-10-18T12:01:08.117606+0000","last_became_peered":"2023-10-18T12:01:08.117606+0000","last_unstale":"2023-10-18T12:01:09.148493+0000","last_undegraded":"2023-10-18T12:01:09.148493+0000","last_fullsized":"2023-10-18T12:01:09.148493+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:49.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.130907+0000","last_change":"2023-10-18T12:01:09.127054+0000","last_active":"2023-10-18T12:01:09.130907+0000","last_peered":"2023-10-18T12:01:09.130907+0000","last_clean":"2023-10-18T12:01:09.130907+0000","last_became_active":"2023-10-18T12:01:09.126804+0000","last_became_peered":"2023-10-18T12:01:09.126804+0000","last_unstale":"2023-10-18T12:01:09.130907+0000","last_undegraded":"2023-10-18T12:01:09.130907+0000","last_fullsized":"2023-10-18T12:01:09.130907+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:42.388425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146319+0000","last_change":"2023-10-18T12:01:08.118544+0000","last_active":"2023-10-18T12:01:09.146319+0000","last_peered":"2023-10-18T12:01:09.146319+0000","last_clean":"2023-10-18T12:01:09.146319+0000","last_became_active":"2023-10-18T12:01:08.118228+0000","last_became_peered":"2023-10-18T12:01:08.118228+0000","last_unstale":"2023-10-18T12:01:09.146319+0000","last_undegraded":"2023-10-18T12:01:09.146319+0000","last_fullsized":"2023-10-18T12:01:09.146319+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.124972+0000","last_change":"2023-10-18T12:01:09.120750+0000","last_active":"2023-10-18T12:01:09.124972+0000","last_peered":"2023-10-18T12:01:09.124972+0000","last_clean":"2023-10-18T12:01:09.124972+0000","last_became_active":"2023-10-18T12:01:09.120490+0000","last_became_peered":"2023-10-18T12:01:09.120490+0000","last_unstale":"2023-10-18T12:01:09.124972+0000","last_undegraded":"2023-10-18T12:01:09.124972+0000","last_fullsized":"2023-10-18T12:01:09.124972+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:50.958401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.123589+0000","last_change":"2023-10-18T12:01:08.118022+0000","last_active":"2023-10-18T12:01:09.123589+0000","last_peered":"2023-10-18T12:01:09.123589+0000","last_clean":"2023-10-18T12:01:09.123589+0000","last_became_active":"2023-10-18T12:01:08.117754+0000","last_became_peered":"2023-10-18T12:01:08.117754+0000","last_unstale":"2023-10-18T12:01:09.123589+0000","last_undegraded":"2023-10-18T12:01:09.123589+0000","last_fullsized":"2023-10-18T12:01:09.123589+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:37.834293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137518+0000","last_change":"2023-10-18T12:01:09.135383+0000","last_active":"2023-10-18T12:01:09.137518+0000","last_peered":"2023-10-18T12:01:09.137518+0000","last_clean":"2023-10-18T12:01:09.137518+0000","last_became_active":"2023-10-18T12:01:09.135114+0000","last_became_peered":"2023-10-18T12:01:09.135114+0000","last_unstale":"2023-10-18T12:01:09.137518+0000","last_undegraded":"2023-10-18T12:01:09.137518+0000","last_fullsized":"2023-10-18T12:01:09.137518+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:58.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.121505+0000","last_change":"2023-10-18T12:01:08.115155+0000","last_active":"2023-10-18T12:01:09.121505+0000","last_peered":"2023-10-18T12:01:09.121505+0000","last_clean":"2023-10-18T12:01:09.121505+0000","last_became_active":"2023-10-18T12:01:08.114843+0000","last_became_peered":"2023-10-18T12:01:08.114843+0000","last_unstale":"2023-10-18T12:01:09.121505+0000","last_undegraded":"2023-10-18T12:01:09.121505+0000","last_fullsized":"2023-10-18T12:01:09.121505+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:28.358756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132946+0000","last_change":"2023-10-18T12:01:09.131820+0000","last_active":"2023-10-18T12:01:09.132946+0000","last_peered":"2023-10-18T12:01:09.132946+0000","last_clean":"2023-10-18T12:01:09.132946+0000","last_became_active":"2023-10-18T12:01:09.131568+0000","last_became_peered":"2023-10-18T12:01:09.131568+0000","last_unstale":"2023-10-18T12:01:09.132946+0000","last_undegraded":"2023-10-18T12:01:09.132946+0000","last_fullsized":"2023-10-18T12:01:09.132946+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:18:19.604112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"241'8","reported_seq":19,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.147954+0000","last_change":"2023-10-18T12:01:08.112711+0000","last_active":"2023-10-18T12:01:09.147954+0000","last_peered":"2023-10-18T12:01:09.147954+0000","last_clean":"2023-10-18T12:01:09.147954+0000","last_became_active":"2023-10-18T12:01:08.112436+0000","last_became_peered":"2023-10-18T12:01:08.112436+0000","last_unstale":"2023-10-18T12:01:09.147954+0000","last_undegraded":"2023-10-18T12:01:09.147954+0000","last_fullsized":"2023-10-18T12:01:09.147954+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:15.568916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136856+0000","last_change":"2023-10-18T12:01:09.135106+0000","last_active":"2023-10-18T12:01:09.136856+0000","last_peered":"2023-10-18T12:01:09.136856+0000","last_clean":"2023-10-18T12:01:09.136856+0000","last_became_active":"2023-10-18T12:01:09.134828+0000","last_became_peered":"2023-10-18T12:01:09.134828+0000","last_unstale":"2023-10-18T12:01:09.136856+0000","last_undegraded":"2023-10-18T12:01:09.136856+0000","last_fullsized":"2023-10-18T12:01:09.136856+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:22.643933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.143890+0000","last_change":"2023-10-18T12:01:08.119248+0000","last_active":"2023-10-18T12:01:09.143890+0000","last_peered":"2023-10-18T12:01:09.143890+0000","last_clean":"2023-10-18T12:01:09.143890+0000","last_became_active":"2023-10-18T12:01:08.118973+0000","last_became_peered":"2023-10-18T12:01:08.118973+0000","last_unstale":"2023-10-18T12:01:09.143890+0000","last_undegraded":"2023-10-18T12:01:09.143890+0000","last_fullsized":"2023-10-18T12:01:09.143890+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:37.079812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133576+0000","last_change":"2023-10-18T12:01:09.132827+0000","last_active":"2023-10-18T12:01:09.133576+0000","last_peered":"2023-10-18T12:01:09.133576+0000","last_clean":"2023-10-18T12:01:09.133576+0000","last_became_active":"2023-10-18T12:01:09.132545+0000","last_became_peered":"2023-10-18T12:01:09.132545+0000","last_unstale":"2023-10-18T12:01:09.133576+0000","last_undegraded":"2023-10-18T12:01:09.133576+0000","last_fullsized":"2023-10-18T12:01:09.133576+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:48.855866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146651+0000","last_change":"2023-10-18T12:01:08.107964+0000","last_active":"2023-10-18T12:01:09.146651+0000","last_peered":"2023-10-18T12:01:09.146651+0000","last_clean":"2023-10-18T12:01:09.146651+0000","last_became_active":"2023-10-18T12:01:08.107696+0000","last_became_peered":"2023-10-18T12:01:08.107696+0000","last_unstale":"2023-10-18T12:01:09.146651+0000","last_undegraded":"2023-10-18T12:01:09.146651+0000","last_fullsized":"2023-10-18T12:01:09.146651+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:18.578919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136873+0000","last_change":"2023-10-18T12:01:09.136078+0000","last_active":"2023-10-18T12:01:09.136873+0000","last_peered":"2023-10-18T12:01:09.136873+0000","last_clean":"2023-10-18T12:01:09.136873+0000","last_became_active":"2023-10-18T12:01:09.134722+0000","last_became_peered":"2023-10-18T12:01:09.134722+0000","last_unstale":"2023-10-18T12:01:09.136873+0000","last_undegraded":"2023-10-18T12:01:09.136873+0000","last_fullsized":"2023-10-18T12:01:09.136873+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:46.059426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146802+0000","last_change":"2023-10-18T12:01:08.120416+0000","last_active":"2023-10-18T12:01:09.146802+0000","last_peered":"2023-10-18T12:01:09.146802+0000","last_clean":"2023-10-18T12:01:09.146802+0000","last_became_active":"2023-10-18T12:01:08.120122+0000","last_became_peered":"2023-10-18T12:01:08.120122+0000","last_unstale":"2023-10-18T12:01:09.146802+0000","last_undegraded":"2023-10-18T12:01:09.146802+0000","last_fullsized":"2023-10-18T12:01:09.146802+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:26.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136217+0000","last_change":"2023-10-18T12:01:09.135042+0000","last_active":"2023-10-18T12:01:09.136217+0000","last_peered":"2023-10-18T12:01:09.136217+0000","last_clean":"2023-10-18T12:01:09.136217+0000","last_became_active":"2023-10-18T12:01:09.134765+0000","last_became_peered":"2023-10-18T12:01:09.134765+0000","last_unstale":"2023-10-18T12:01:09.136217+0000","last_undegraded":"2023-10-18T12:01:09.136217+0000","last_fullsized":"2023-10-18T12:01:09.136217+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:18:15.700428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.122997+0000","last_change":"2023-10-18T12:01:08.117972+0000","last_active":"2023-10-18T12:01:09.122997+0000","last_peered":"2023-10-18T12:01:09.122997+0000","last_clean":"2023-10-18T12:01:09.122997+0000","last_became_active":"2023-10-18T12:01:08.117690+0000","last_became_peered":"2023-10-18T12:01:08.117690+0000","last_unstale":"2023-10-18T12:01:09.122997+0000","last_undegraded":"2023-10-18T12:01:09.122997+0000","last_fullsized":"2023-10-18T12:01:09.122997+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.124607+0000","last_change":"2023-10-18T12:01:09.122673+0000","last_active":"2023-10-18T12:01:09.124607+0000","last_peered":"2023-10-18T12:01:09.124607+0000","last_clean":"2023-10-18T12:01:09.124607+0000","last_became_active":"2023-10-18T12:01:09.122430+0000","last_became_peered":"2023-10-18T12:01:09.122430+0000","last_unstale":"2023-10-18T12:01:09.124607+0000","last_undegraded":"2023-10-18T12:01:09.124607+0000","last_fullsized":"2023-10-18T12:01:09.124607+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:08.858055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.143964+0000","last_change":"2023-10-18T12:01:08.119439+0000","last_active":"2023-10-18T12:01:09.143964+0000","last_peered":"2023-10-18T12:01:09.143964+0000","last_clean":"2023-10-18T12:01:09.143964+0000","last_became_active":"2023-10-18T12:01:08.119158+0000","last_became_peered":"2023-10-18T12:01:08.119158+0000","last_unstale":"2023-10-18T12:01:09.143964+0000","last_undegraded":"2023-10-18T12:01:09.143964+0000","last_fullsized":"2023-10-18T12:01:09.143964+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:56.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127327+0000","last_change":"2023-10-18T12:01:09.124352+0000","last_active":"2023-10-18T12:01:09.127327+0000","last_peered":"2023-10-18T12:01:09.127327+0000","last_clean":"2023-10-18T12:01:09.127327+0000","last_became_active":"2023-10-18T12:01:09.124072+0000","last_became_peered":"2023-10-18T12:01:09.124072+0000","last_unstale":"2023-10-18T12:01:09.127327+0000","last_undegraded":"2023-10-18T12:01:09.127327+0000","last_fullsized":"2023-10-18T12:01:09.127327+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:54:30.629590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.126469+0000","last_change":"2023-10-18T12:01:08.117335+0000","last_active":"2023-10-18T12:01:09.126469+0000","last_peered":"2023-10-18T12:01:09.126469+0000","last_clean":"2023-10-18T12:01:09.126469+0000","last_became_active":"2023-10-18T12:01:08.117032+0000","last_became_peered":"2023-10-18T12:01:08.117032+0000","last_unstale":"2023-10-18T12:01:09.126469+0000","last_undegraded":"2023-10-18T12:01:09.126469+0000","last_fullsized":"2023-10-18T12:01:09.126469+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:33.304569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136754+0000","last_change":"2023-10-18T12:01:09.135148+0000","last_active":"2023-10-18T12:01:09.136754+0000","last_peered":"2023-10-18T12:01:09.136754+0000","last_clean":"2023-10-18T12:01:09.136754+0000","last_became_active":"2023-10-18T12:01:09.134895+0000","last_became_peered":"2023-10-18T12:01:09.134895+0000","last_unstale":"2023-10-18T12:01:09.136754+0000","last_undegraded":"2023-10-18T12:01:09.136754+0000","last_fullsized":"2023-10-18T12:01:09.136754+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:26.884874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.120652+0000","last_change":"2023-10-18T12:01:08.120618+0000","last_active":"2023-10-18T12:01:09.120652+0000","last_peered":"2023-10-18T12:01:09.120652+0000","last_clean":"2023-10-18T12:01:09.120652+0000","last_became_active":"2023-10-18T12:01:08.120315+0000","last_became_peered":"2023-10-18T12:01:08.120315+0000","last_unstale":"2023-10-18T12:01:09.120652+0000","last_undegraded":"2023-10-18T12:01:09.120652+0000","last_fullsized":"2023-10-18T12:01:09.120652+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:36.555722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133217+0000","last_change":"2023-10-18T12:01:09.130881+0000","last_active":"2023-10-18T12:01:09.133217+0000","last_peered":"2023-10-18T12:01:09.133217+0000","last_clean":"2023-10-18T12:01:09.133217+0000","last_became_active":"2023-10-18T12:01:09.130628+0000","last_became_peered":"2023-10-18T12:01:09.130628+0000","last_unstale":"2023-10-18T12:01:09.133217+0000","last_undegraded":"2023-10-18T12:01:09.133217+0000","last_fullsized":"2023-10-18T12:01:09.133217+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:02.473534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.114247+0000","last_change":"2023-10-18T12:01:08.120313+0000","last_active":"2023-10-18T12:01:09.114247+0000","last_peered":"2023-10-18T12:01:09.114247+0000","last_clean":"2023-10-18T12:01:09.114247+0000","last_became_active":"2023-10-18T12:01:08.120059+0000","last_became_peered":"2023-10-18T12:01:08.120059+0000","last_unstale":"2023-10-18T12:01:09.114247+0000","last_undegraded":"2023-10-18T12:01:09.114247+0000","last_fullsized":"2023-10-18T12:01:09.114247+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127524+0000","last_change":"2023-10-18T12:01:09.125685+0000","last_active":"2023-10-18T12:01:09.127524+0000","last_peered":"2023-10-18T12:01:09.127524+0000","last_clean":"2023-10-18T12:01:09.127524+0000","last_became_active":"2023-10-18T12:01:09.125395+0000","last_became_peered":"2023-10-18T12:01:09.125395+0000","last_unstale":"2023-10-18T12:01:09.127524+0000","last_undegraded":"2023-10-18T12:01:09.127524+0000","last_fullsized":"2023-10-18T12:01:09.127524+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:26:35.778496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.142593+0000","last_change":"2023-10-18T12:01:08.117717+0000","last_active":"2023-10-18T12:01:09.142593+0000","last_peered":"2023-10-18T12:01:09.142593+0000","last_clean":"2023-10-18T12:01:09.142593+0000","last_became_active":"2023-10-18T12:01:08.117405+0000","last_became_peered":"2023-10-18T12:01:08.117405+0000","last_unstale":"2023-10-18T12:01:09.142593+0000","last_undegraded":"2023-10-18T12:01:09.142593+0000","last_fullsized":"2023-10-18T12:01:09.142593+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:00.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133286+0000","last_change":"2023-10-18T12:01:09.132599+0000","last_active":"2023-10-18T12:01:09.133286+0000","last_peered":"2023-10-18T12:01:09.133286+0000","last_clean":"2023-10-18T12:01:09.133286+0000","last_became_active":"2023-10-18T12:01:09.131710+0000","last_became_peered":"2023-10-18T12:01:09.131710+0000","last_unstale":"2023-10-18T12:01:09.133286+0000","last_undegraded":"2023-10-18T12:01:09.133286+0000","last_fullsized":"2023-10-18T12:01:09.133286+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:33.318887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.122306+0000","last_change":"2023-10-18T12:01:08.118553+0000","last_active":"2023-10-18T12:01:09.122306+0000","last_peered":"2023-10-18T12:01:09.122306+0000","last_clean":"2023-10-18T12:01:09.122306+0000","last_became_active":"2023-10-18T12:01:08.118286+0000","last_became_peered":"2023-10-18T12:01:08.118286+0000","last_unstale":"2023-10-18T12:01:09.122306+0000","last_undegraded":"2023-10-18T12:01:09.122306+0000","last_fullsized":"2023-10-18T12:01:09.122306+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:05:32.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.134491+0000","last_change":"2023-10-18T12:01:09.133522+0000","last_active":"2023-10-18T12:01:09.134491+0000","last_peered":"2023-10-18T12:01:09.134491+0000","last_clean":"2023-10-18T12:01:09.134491+0000","last_became_active":"2023-10-18T12:01:09.133267+0000","last_became_peered":"2023-10-18T12:01:09.133267+0000","last_unstale":"2023-10-18T12:01:09.134491+0000","last_undegraded":"2023-10-18T12:01:09.134491+0000","last_fullsized":"2023-10-18T12:01:09.134491+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:56.382925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.114721+0000","last_change":"2023-10-18T12:01:08.116830+0000","last_active":"2023-10-18T12:01:09.114721+0000","last_peered":"2023-10-18T12:01:09.114721+0000","last_clean":"2023-10-18T12:01:09.114721+0000","last_became_active":"2023-10-18T12:01:08.116538+0000","last_became_peered":"2023-10-18T12:01:08.116538+0000","last_unstale":"2023-10-18T12:01:09.114721+0000","last_undegraded":"2023-10-18T12:01:09.114721+0000","last_fullsized":"2023-10-18T12:01:09.114721+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:09.165065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132808+0000","last_change":"2023-10-18T12:01:09.128305+0000","last_active":"2023-10-18T12:01:09.132808+0000","last_peered":"2023-10-18T12:01:09.132808+0000","last_clean":"2023-10-18T12:01:09.132808+0000","last_became_active":"2023-10-18T12:01:09.127933+0000","last_became_peered":"2023-10-18T12:01:09.127933+0000","last_unstale":"2023-10-18T12:01:09.132808+0000","last_undegraded":"2023-10-18T12:01:09.132808+0000","last_fullsized":"2023-10-18T12:01:09.132808+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:51:23.836756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.115421+0000","last_change":"2023-10-18T12:01:08.115363+0000","last_active":"2023-10-18T12:01:09.115421+0000","last_peered":"2023-10-18T12:01:09.115421+0000","last_clean":"2023-10-18T12:01:09.115421+0000","last_became_active":"2023-10-18T12:01:08.114903+0000","last_became_peered":"2023-10-18T12:01:08.114903+0000","last_unstale":"2023-10-18T12:01:09.115421+0000","last_undegraded":"2023-10-18T12:01:09.115421+0000","last_fullsized":"2023-10-18T12:01:09.115421+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:46.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.129367+0000","last_change":"2023-10-18T12:01:09.127159+0000","last_active":"2023-10-18T12:01:09.129367+0000","last_peered":"2023-10-18T12:01:09.129367+0000","last_clean":"2023-10-18T12:01:09.129367+0000","last_became_active":"2023-10-18T12:01:09.126874+0000","last_became_peered":"2023-10-18T12:01:09.126874+0000","last_unstale":"2023-10-18T12:01:09.129367+0000","last_undegraded":"2023-10-18T12:01:09.129367+0000","last_fullsized":"2023-10-18T12:01:09.129367+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:11.094303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127585+0000","last_change":"2023-10-18T12:01:08.118539+0000","last_active":"2023-10-18T12:01:09.127585+0000","last_peered":"2023-10-18T12:01:09.127585+0000","last_clean":"2023-10-18T12:01:09.127585+0000","last_became_active":"2023-10-18T12:01:08.116966+0000","last_became_peered":"2023-10-18T12:01:08.116966+0000","last_unstale":"2023-10-18T12:01:09.127585+0000","last_undegraded":"2023-10-18T12:01:09.127585+0000","last_fullsized":"2023-10-18T12:01:09.127585+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:20.828671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.120097+0000","last_change":"2023-10-18T12:01:09.119148+0000","last_active":"2023-10-18T12:01:09.120097+0000","last_peered":"2023-10-18T12:01:09.120097+0000","last_clean":"2023-10-18T12:01:09.120097+0000","last_became_active":"2023-10-18T12:01:09.118894+0000","last_became_peered":"2023-10-18T12:01:09.118894+0000","last_unstale":"2023-10-18T12:01:09.120097+0000","last_undegraded":"2023-10-18T12:01:09.120097+0000","last_fullsized":"2023-10-18T12:01:09.120097+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:32:52.033269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.123725+0000","last_change":"2023-10-18T12:01:08.117526+0000","last_active":"2023-10-18T12:01:09.123725+0000","last_peered":"2023-10-18T12:01:09.123725+0000","last_clean":"2023-10-18T12:01:09.123725+0000","last_became_active":"2023-10-18T12:01:08.117234+0000","last_became_peered":"2023-10-18T12:01:08.117234+0000","last_unstale":"2023-10-18T12:01:09.123725+0000","last_undegraded":"2023-10-18T12:01:09.123725+0000","last_fullsized":"2023-10-18T12:01:09.123725+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:43.295217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.128333+0000","last_change":"2023-10-18T12:01:09.125150+0000","last_active":"2023-10-18T12:01:09.128333+0000","last_peered":"2023-10-18T12:01:09.128333+0000","last_clean":"2023-10-18T12:01:09.128333+0000","last_became_active":"2023-10-18T12:01:09.124900+0000","last_became_peered":"2023-10-18T12:01:09.124900+0000","last_unstale":"2023-10-18T12:01:09.128333+0000","last_undegraded":"2023-10-18T12:01:09.128333+0000","last_fullsized":"2023-10-18T12:01:09.128333+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:37.454945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.115056+0000","last_change":"2023-10-18T12:01:08.119612+0000","last_active":"2023-10-18T12:01:09.115056+0000","last_peered":"2023-10-18T12:01:09.115056+0000","last_clean":"2023-10-18T12:01:09.115056+0000","last_became_active":"2023-10-18T12:01:08.119116+0000","last_became_peered":"2023-10-18T12:01:08.119116+0000","last_unstale":"2023-10-18T12:01:09.115056+0000","last_undegraded":"2023-10-18T12:01:09.115056+0000","last_fullsized":"2023-10-18T12:01:09.115056+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:02.536144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133199+0000","last_change":"2023-10-18T12:01:09.130360+0000","last_active":"2023-10-18T12:01:09.133199+0000","last_peered":"2023-10-18T12:01:09.133199+0000","last_clean":"2023-10-18T12:01:09.133199+0000","last_became_active":"2023-10-18T12:01:09.130086+0000","last_became_peered":"2023-10-18T12:01:09.130086+0000","last_unstale":"2023-10-18T12:01:09.133199+0000","last_undegraded":"2023-10-18T12:01:09.133199+0000","last_fullsized":"2023-10-18T12:01:09.133199+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:15.311291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.120114+0000","last_change":"2023-10-18T12:01:08.119803+0000","last_active":"2023-10-18T12:01:09.120114+0000","last_peered":"2023-10-18T12:01:09.120114+0000","last_clean":"2023-10-18T12:01:09.120114+0000","last_became_active":"2023-10-18T12:01:08.119467+0000","last_became_peered":"2023-10-18T12:01:08.119467+0000","last_unstale":"2023-10-18T12:01:09.120114+0000","last_undegraded":"2023-10-18T12:01:09.120114+0000","last_fullsized":"2023-10-18T12:01:09.120114+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:29.410010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.123340+0000","last_change":"2023-10-18T12:01:09.122670+0000","last_active":"2023-10-18T12:01:09.123340+0000","last_peered":"2023-10-18T12:01:09.123340+0000","last_clean":"2023-10-18T12:01:09.123340+0000","last_became_active":"2023-10-18T12:01:09.122408+0000","last_became_peered":"2023-10-18T12:01:09.122408+0000","last_unstale":"2023-10-18T12:01:09.123340+0000","last_undegraded":"2023-10-18T12:01:09.123340+0000","last_fullsized":"2023-10-18T12:01:09.123340+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:16.307249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.143074+0000","last_change":"2023-10-18T12:01:08.111055+0000","last_active":"2023-10-18T12:01:09.143074+0000","last_peered":"2023-10-18T12:01:09.143074+0000","last_clean":"2023-10-18T12:01:09.143074+0000","last_became_active":"2023-10-18T12:01:08.110728+0000","last_became_peered":"2023-10-18T12:01:08.110728+0000","last_unstale":"2023-10-18T12:01:09.143074+0000","last_undegraded":"2023-10-18T12:01:09.143074+0000","last_fullsized":"2023-10-18T12:01:09.143074+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:15.472803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"227'379","reported_seq":1328,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-18T12:01:08.110987+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:01:08.110987+0000","last_peered":"2023-10-18T12:01:08.110987+0000","last_clean":"2023-10-18T12:01:08.110987+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:01:08.110987+0000","last_undegraded":"2023-10-18T12:01:08.110987+0000","last_fullsized":"2023-10-18T12:01:08.110987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":379,"ondisk_log_size":379,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575998,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52788,"kb_used_data":3556,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319052,"statfs":{"total":96636764160,"available":96582709248,"internally_reserved":0,"allocated":3641344,"data_stored":2155554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.54300000000000004}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.48899999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.627}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.56699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42476,"kb_used_data":2976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329364,"statfs":{"total":96636764160,"available":96593268736,"internally_reserved":0,"allocated":3047424,"data_stored":1565174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.57599999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.436}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.51500000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.501},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48199999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64676,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307164,"statfs":{"total":96636764160,"available":96570535936,"internally_reserved":0,"allocated":3039232,"data_stored":1565062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72199999999999998,"5min":0.92400000000000004,"15min":0.96099999999999997},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.745,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.78700000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.74099999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.755,"5min":0.80500000000000005,"15min":1.0349999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.45000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.71399999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":1},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52100000000000002,"15min":0.53800000000000003},"min":{"1min":0.377,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.01},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.872}]},{"osd":7,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":1.101,"15min":1.101},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81499999999999995,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.80900000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48040,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323800,"statfs":{"total":96636764160,"available":96587571200,"internally_reserved":0,"allocated":3043328,"data_stored":1565132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.22500000000000001},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.755},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":1.123},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69999999999999996}]},{"osd":3,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.63100000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.49299999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":1.018,"15min":1.018},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.88900000000000001,"15min":1.05},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48072,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323768,"statfs":{"total":96636764160,"available":96587538432,"internally_reserved":0,"allocated":3063808,"data_stored":1567312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.23200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55000000000000004,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.309},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.41399999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23400000000000001},"max":{"1min":0.56699999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41599999999999998,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.44500000000000001},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":1.899,"15min":1.899},"last":0.27400000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.29299999999999998,"15min":0.28399999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88400000000000001,"5min":1.034,"15min":1.052},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45300000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.522,"15min":1.522},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.89600000000000002,"5min":0.91100000000000003,"15min":1.026},"last":0.48499999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":1.01,"15min":1.01},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67100000000000004,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81799999999999995,"5min":1.129,"15min":1.129},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.57399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575998,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60524,"kb_used_data":2976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311316,"statfs":{"total":96636764160,"available":96574787584,"internally_reserved":0,"allocated":3047424,"data_stored":1565714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.216},"max":{"1min":0.56999999999999995,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.56399999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.443},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.504}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65300000000000002,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.55300000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.087,"15min":1.087},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.755},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66900000000000004,"15min":0.67400000000000004},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92000000000000004},"last":0.79300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60520,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311320,"statfs":{"total":96636764160,"available":96574791680,"internally_reserved":0,"allocated":3039232,"data_stored":1565062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.307},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.376}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.35999999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.623}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.59599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43072,"kb_used_data":3560,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328768,"statfs":{"total":96636764160,"available":96592658432,"internally_reserved":0,"allocated":3645440,"data_stored":2157042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54700000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61899999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28899999999999998,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.66800000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.312}]},{"osd":3,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.25700000000000001},"max":{"1min":0.62,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.435},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.56899999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96999999999999997,"15min":0.997},"last":0.505},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.876,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.53900000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88200000000000001,"5min":0.95799999999999996,"15min":0.96399999999999997},"last":0.71799999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66900000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":1.0089999999999999,"15min":1.03},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.60599999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65800000000000003,"15min":0.68100000000000005},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.82699999999999996,"5min":0.997,"15min":0.997},"last":0.65400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:01:12.339 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:01:12.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.472+0000 7f3dd3264640 1 -- 172.21.15.125:0/3306591855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 msgr2=0x7f3dcc0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.472+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3306591855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 0x7f3dcc0d1910 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005fd0 tx=0x7f3dbc02c240 comp rx=0 tx=0).stop 2023-10-18T12:01:12.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.472+0000 7f3dd3264640 1 -- 172.21.15.125:0/3306591855 shutdown_connections 2023-10-18T12:01:12.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.472+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3306591855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dcc15b2a0 0x7f3dcc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.472+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3306591855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 0x7f3dcc0d1910 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.472+0000 7f3dd3264640 1 -- 172.21.15.125:0/3306591855 >> 172.21.15.125:0/3306591855 conn(0x7f3dcc0d0d60 msgr2=0x7f3dcc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:12.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.472+0000 7f3dd3264640 1 -- 172.21.15.125:0/3306591855 shutdown_connections 2023-10-18T12:01:12.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.472+0000 7f3dd3264640 1 -- 172.21.15.125:0/3306591855 wait complete. 2023-10-18T12:01:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dd3264640 1 Processor -- start 2023-10-18T12:01:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dd3264640 1 -- start start 2023-10-18T12:01:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dd3264640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dcc0ceb40 0x7f3dcc1f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dd3264640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 0x7f3dcc1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dd3264640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dcc15b2a0 0x7f3dcc0ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dd3264640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1f5b40 con 0x7f3dcc0cf340 2023-10-18T12:01:12.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dd3264640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc158820 con 0x7f3dcc0ceb40 2023-10-18T12:01:12.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dd3264640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc15b7e0 con 0x7f3dcc15b2a0 2023-10-18T12:01:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.474+0000 7f3dcbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 0x7f3dcc1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dcbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 0x7f3dcc1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35956/0 (socket says 172.21.15.125:35956) 2023-10-18T12:01:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dcbfff640 1 -- 172.21.15.125:0/3549981710 learned_addr learned my addr 172.21.15.125:0/3549981710 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dd1801640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dcc15b2a0 0x7f3dcc0ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dd1000640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dcc0ceb40 0x7f3dcc1f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dcbfff640 1 -- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dcc15b2a0 msgr2=0x7f3dcc0ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dcbfff640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dcc15b2a0 0x7f3dcc0ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dcbfff640 1 -- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dcc0ceb40 msgr2=0x7f3dcc1f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dcbfff640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dcc0ceb40 0x7f3dcc1f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dcbfff640 1 -- 172.21.15.125:0/3549981710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009900 con 0x7f3dcc0cf340 2023-10-18T12:01:12.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dd1801640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dcc15b2a0 0x7f3dcc0ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:12.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dcbfff640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 0x7f3dcc1582e0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e790 tx=0x7f3dc400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:12.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dc97fa640 1 -- 172.21.15.125:0/3549981710 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4009580 con 0x7f3dcc0cf340 2023-10-18T12:01:12.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dc97fa640 1 -- 172.21.15.125:0/3549981710 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3dc400cdb0 con 0x7f3dcc0cf340 2023-10-18T12:01:12.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc15bac0 con 0x7f3dcc0cf340 2023-10-18T12:01:12.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dc97fa640 1 -- 172.21.15.125:0/3549981710 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4010690 con 0x7f3dcc0cf340 2023-10-18T12:01:12.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.475+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc15c080 con 0x7f3dcc0cf340 2023-10-18T12:01:12.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.476+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc0690e0 con 0x7f3dcc0cf340 2023-10-18T12:01:12.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.477+0000 7f3dc97fa640 1 -- 172.21.15.125:0/3549981710 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3dc40107f0 con 0x7f3dcc0cf340 2023-10-18T12:01:12.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.477+0000 7f3dc97fa640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dac06b080 0x7f3dac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.477+0000 7f3dc97fa640 1 -- 172.21.15.125:0/3549981710 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f3dc4014070 con 0x7f3dcc0cf340 2023-10-18T12:01:12.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.477+0000 7f3dd1000640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dac06b080 0x7f3dac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.478+0000 7f3dd1000640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dac06b080 0x7f3dac06d540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc036000 tx=0x7f3dbc004ab0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:12.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.481+0000 7f3dc97fa640 1 -- 172.21.15.125:0/3549981710 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dc4051ca0 con 0x7f3dcc0cf340 2023-10-18T12:01:12.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.644+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3dcc0cc170 con 0x7f3dac06b080 2023-10-18T12:01:12.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.652+0000 7f3dc97fa640 1 -- 172.21.15.125:0/3549981710 <== mgr.9236 v2:172.21.15.139:6832/3422518215 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290152 (secure 0 0 0) 0x7f3dbc003b40 con 0x7f3dac06b080 2023-10-18T12:01:12.658 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:12.659 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:01:12.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dac06b080 msgr2=0x7f3dac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dac06b080 0x7f3dac06d540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc036000 tx=0x7f3dbc004ab0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 msgr2=0x7f3dcc1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 0x7f3dcc1582e0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e790 tx=0x7f3dc400ec50 comp rx=0 tx=0).stop 2023-10-18T12:01:12.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 shutdown_connections 2023-10-18T12:01:12.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f3dac06b080 0x7f3dac06d540 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3dcc15b2a0 0x7f3dcc0ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3dcc0cf340 0x7f3dcc1582e0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 --2- 172.21.15.125:0/3549981710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3dcc0ceb40 0x7f3dcc1f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 >> 172.21.15.125:0/3549981710 conn(0x7f3dcc0d0d60 msgr2=0x7f3dcc0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:12.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 shutdown_connections 2023-10-18T12:01:12.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.655+0000 7f3dd3264640 1 -- 172.21.15.125:0/3549981710 wait complete. 2023-10-18T12:01:12.668 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-18T12:01:10.835757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":679,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":417,"ondisk_log_size":417,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420168,"kb_used_data":24968,"kb_used_omap":12,"kb_used_meta":395059,"kb_avail":754554552,"statfs":{"total":773094113280,"available":772663861248,"internally_reserved":0,"allocated":25567232,"data_stored":13706052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":404540984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000863"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.128323+0000","last_change":"2023-10-18T12:01:09.126968+0000","last_active":"2023-10-18T12:01:09.128323+0000","last_peered":"2023-10-18T12:01:09.128323+0000","last_clean":"2023-10-18T12:01:09.128323+0000","last_became_active":"2023-10-18T12:01:09.126705+0000","last_became_peered":"2023-10-18T12:01:09.126705+0000","last_unstale":"2023-10-18T12:01:09.128323+0000","last_undegraded":"2023-10-18T12:01:09.128323+0000","last_fullsized":"2023-10-18T12:01:09.128323+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:14.190713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.130626+0000","last_change":"2023-10-18T12:01:09.125772+0000","last_active":"2023-10-18T12:01:09.130626+0000","last_peered":"2023-10-18T12:01:09.130626+0000","last_clean":"2023-10-18T12:01:09.130626+0000","last_became_active":"2023-10-18T12:01:09.125508+0000","last_became_peered":"2023-10-18T12:01:09.125508+0000","last_unstale":"2023-10-18T12:01:09.130626+0000","last_undegraded":"2023-10-18T12:01:09.130626+0000","last_fullsized":"2023-10-18T12:01:09.130626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:53.920813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.125260+0000","last_change":"2023-10-18T12:01:09.124764+0000","last_active":"2023-10-18T12:01:09.125260+0000","last_peered":"2023-10-18T12:01:09.125260+0000","last_clean":"2023-10-18T12:01:09.125260+0000","last_became_active":"2023-10-18T12:01:09.124472+0000","last_became_peered":"2023-10-18T12:01:09.124472+0000","last_unstale":"2023-10-18T12:01:09.125260+0000","last_undegraded":"2023-10-18T12:01:09.125260+0000","last_fullsized":"2023-10-18T12:01:09.125260+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:47.018047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133236+0000","last_change":"2023-10-18T12:01:09.132202+0000","last_active":"2023-10-18T12:01:09.133236+0000","last_peered":"2023-10-18T12:01:09.133236+0000","last_clean":"2023-10-18T12:01:09.133236+0000","last_became_active":"2023-10-18T12:01:09.131928+0000","last_became_peered":"2023-10-18T12:01:09.131928+0000","last_unstale":"2023-10-18T12:01:09.133236+0000","last_undegraded":"2023-10-18T12:01:09.133236+0000","last_fullsized":"2023-10-18T12:01:09.133236+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:51.005840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136276+0000","last_change":"2023-10-18T12:01:09.133870+0000","last_active":"2023-10-18T12:01:09.136276+0000","last_peered":"2023-10-18T12:01:09.136276+0000","last_clean":"2023-10-18T12:01:09.136276+0000","last_became_active":"2023-10-18T12:01:09.133478+0000","last_became_peered":"2023-10-18T12:01:09.133478+0000","last_unstale":"2023-10-18T12:01:09.136276+0000","last_undegraded":"2023-10-18T12:01:09.136276+0000","last_fullsized":"2023-10-18T12:01:09.136276+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:15.623643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.129938+0000","last_change":"2023-10-18T12:01:09.129113+0000","last_active":"2023-10-18T12:01:09.129938+0000","last_peered":"2023-10-18T12:01:09.129938+0000","last_clean":"2023-10-18T12:01:09.129938+0000","last_became_active":"2023-10-18T12:01:09.128785+0000","last_became_peered":"2023-10-18T12:01:09.128785+0000","last_unstale":"2023-10-18T12:01:09.129938+0000","last_undegraded":"2023-10-18T12:01:09.129938+0000","last_fullsized":"2023-10-18T12:01:09.129938+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:25:20.201191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127495+0000","last_change":"2023-10-18T12:01:09.125901+0000","last_active":"2023-10-18T12:01:09.127495+0000","last_peered":"2023-10-18T12:01:09.127495+0000","last_clean":"2023-10-18T12:01:09.127495+0000","last_became_active":"2023-10-18T12:01:09.125390+0000","last_became_peered":"2023-10-18T12:01:09.125390+0000","last_unstale":"2023-10-18T12:01:09.127495+0000","last_undegraded":"2023-10-18T12:01:09.127495+0000","last_fullsized":"2023-10-18T12:01:09.127495+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:16:51.521674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136838+0000","last_change":"2023-10-18T12:01:09.135081+0000","last_active":"2023-10-18T12:01:09.136838+0000","last_peered":"2023-10-18T12:01:09.136838+0000","last_clean":"2023-10-18T12:01:09.136838+0000","last_became_active":"2023-10-18T12:01:09.134801+0000","last_became_peered":"2023-10-18T12:01:09.134801+0000","last_unstale":"2023-10-18T12:01:09.136838+0000","last_undegraded":"2023-10-18T12:01:09.136838+0000","last_fullsized":"2023-10-18T12:01:09.136838+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:22:50.011284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136526+0000","last_change":"2023-10-18T12:01:09.135615+0000","last_active":"2023-10-18T12:01:09.136526+0000","last_peered":"2023-10-18T12:01:09.136526+0000","last_clean":"2023-10-18T12:01:09.136526+0000","last_became_active":"2023-10-18T12:01:09.135123+0000","last_became_peered":"2023-10-18T12:01:09.135123+0000","last_unstale":"2023-10-18T12:01:09.136526+0000","last_undegraded":"2023-10-18T12:01:09.136526+0000","last_fullsized":"2023-10-18T12:01:09.136526+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:12.571415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132608+0000","last_change":"2023-10-18T12:01:09.130673+0000","last_active":"2023-10-18T12:01:09.132608+0000","last_peered":"2023-10-18T12:01:09.132608+0000","last_clean":"2023-10-18T12:01:09.132608+0000","last_became_active":"2023-10-18T12:01:09.130094+0000","last_became_peered":"2023-10-18T12:01:09.130094+0000","last_unstale":"2023-10-18T12:01:09.132608+0000","last_undegraded":"2023-10-18T12:01:09.132608+0000","last_fullsized":"2023-10-18T12:01:09.132608+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:48.689497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137102+0000","last_change":"2023-10-18T12:01:09.135828+0000","last_active":"2023-10-18T12:01:09.137102+0000","last_peered":"2023-10-18T12:01:09.137102+0000","last_clean":"2023-10-18T12:01:09.137102+0000","last_became_active":"2023-10-18T12:01:09.135579+0000","last_became_peered":"2023-10-18T12:01:09.135579+0000","last_unstale":"2023-10-18T12:01:09.137102+0000","last_undegraded":"2023-10-18T12:01:09.137102+0000","last_fullsized":"2023-10-18T12:01:09.137102+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:20.398187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.134195+0000","last_change":"2023-10-18T12:01:09.132939+0000","last_active":"2023-10-18T12:01:09.134195+0000","last_peered":"2023-10-18T12:01:09.134195+0000","last_clean":"2023-10-18T12:01:09.134195+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:09.134195+0000","last_undegraded":"2023-10-18T12:01:09.134195+0000","last_fullsized":"2023-10-18T12:01:09.134195+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:04:42.744489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133239+0000","last_change":"2023-10-18T12:01:09.129764+0000","last_active":"2023-10-18T12:01:09.133239+0000","last_peered":"2023-10-18T12:01:09.133239+0000","last_clean":"2023-10-18T12:01:09.133239+0000","last_became_active":"2023-10-18T12:01:09.129483+0000","last_became_peered":"2023-10-18T12:01:09.129483+0000","last_unstale":"2023-10-18T12:01:09.133239+0000","last_undegraded":"2023-10-18T12:01:09.133239+0000","last_fullsized":"2023-10-18T12:01:09.133239+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:52:21.068558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.135525+0000","last_change":"2023-10-18T12:01:09.133024+0000","last_active":"2023-10-18T12:01:09.135525+0000","last_peered":"2023-10-18T12:01:09.135525+0000","last_clean":"2023-10-18T12:01:09.135525+0000","last_became_active":"2023-10-18T12:01:09.132742+0000","last_became_peered":"2023-10-18T12:01:09.132742+0000","last_unstale":"2023-10-18T12:01:09.135525+0000","last_undegraded":"2023-10-18T12:01:09.135525+0000","last_fullsized":"2023-10-18T12:01:09.135525+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:58.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132719+0000","last_change":"2023-10-18T12:01:09.131147+0000","last_active":"2023-10-18T12:01:09.132719+0000","last_peered":"2023-10-18T12:01:09.132719+0000","last_clean":"2023-10-18T12:01:09.132719+0000","last_became_active":"2023-10-18T12:01:09.130682+0000","last_became_peered":"2023-10-18T12:01:09.130682+0000","last_unstale":"2023-10-18T12:01:09.132719+0000","last_undegraded":"2023-10-18T12:01:09.132719+0000","last_fullsized":"2023-10-18T12:01:09.132719+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:56.556729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.124565+0000","last_change":"2023-10-18T12:01:09.122098+0000","last_active":"2023-10-18T12:01:09.124565+0000","last_peered":"2023-10-18T12:01:09.124565+0000","last_clean":"2023-10-18T12:01:09.124565+0000","last_became_active":"2023-10-18T12:01:09.121845+0000","last_became_peered":"2023-10-18T12:01:09.121845+0000","last_unstale":"2023-10-18T12:01:09.124565+0000","last_undegraded":"2023-10-18T12:01:09.124565+0000","last_fullsized":"2023-10-18T12:01:09.124565+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:10.895565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136613+0000","last_change":"2023-10-18T12:01:09.133999+0000","last_active":"2023-10-18T12:01:09.136613+0000","last_peered":"2023-10-18T12:01:09.136613+0000","last_clean":"2023-10-18T12:01:09.136613+0000","last_became_active":"2023-10-18T12:01:09.133725+0000","last_became_peered":"2023-10-18T12:01:09.133725+0000","last_unstale":"2023-10-18T12:01:09.136613+0000","last_undegraded":"2023-10-18T12:01:09.136613+0000","last_fullsized":"2023-10-18T12:01:09.136613+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:33.136459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.125460+0000","last_change":"2023-10-18T12:01:09.124908+0000","last_active":"2023-10-18T12:01:09.125460+0000","last_peered":"2023-10-18T12:01:09.125460+0000","last_clean":"2023-10-18T12:01:09.125460+0000","last_became_active":"2023-10-18T12:01:09.124565+0000","last_became_peered":"2023-10-18T12:01:09.124565+0000","last_unstale":"2023-10-18T12:01:09.125460+0000","last_undegraded":"2023-10-18T12:01:09.125460+0000","last_fullsized":"2023-10-18T12:01:09.125460+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:44.946975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137497+0000","last_change":"2023-10-18T12:01:09.135302+0000","last_active":"2023-10-18T12:01:09.137497+0000","last_peered":"2023-10-18T12:01:09.137497+0000","last_clean":"2023-10-18T12:01:09.137497+0000","last_became_active":"2023-10-18T12:01:09.134978+0000","last_became_peered":"2023-10-18T12:01:09.134978+0000","last_unstale":"2023-10-18T12:01:09.137497+0000","last_undegraded":"2023-10-18T12:01:09.137497+0000","last_fullsized":"2023-10-18T12:01:09.137497+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:47.648103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137927+0000","last_change":"2023-10-18T12:01:09.136717+0000","last_active":"2023-10-18T12:01:09.137927+0000","last_peered":"2023-10-18T12:01:09.137927+0000","last_clean":"2023-10-18T12:01:09.137927+0000","last_became_active":"2023-10-18T12:01:09.136461+0000","last_became_peered":"2023-10-18T12:01:09.136461+0000","last_unstale":"2023-10-18T12:01:09.137927+0000","last_undegraded":"2023-10-18T12:01:09.137927+0000","last_fullsized":"2023-10-18T12:01:09.137927+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:04:30.217797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133477+0000","last_change":"2023-10-18T12:01:09.130016+0000","last_active":"2023-10-18T12:01:09.133477+0000","last_peered":"2023-10-18T12:01:09.133477+0000","last_clean":"2023-10-18T12:01:09.133477+0000","last_became_active":"2023-10-18T12:01:09.129718+0000","last_became_peered":"2023-10-18T12:01:09.129718+0000","last_unstale":"2023-10-18T12:01:09.133477+0000","last_undegraded":"2023-10-18T12:01:09.133477+0000","last_fullsized":"2023-10-18T12:01:09.133477+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:57.709530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133066+0000","last_change":"2023-10-18T12:01:09.129706+0000","last_active":"2023-10-18T12:01:09.133066+0000","last_peered":"2023-10-18T12:01:09.133066+0000","last_clean":"2023-10-18T12:01:09.133066+0000","last_became_active":"2023-10-18T12:01:09.129413+0000","last_became_peered":"2023-10-18T12:01:09.129413+0000","last_unstale":"2023-10-18T12:01:09.133066+0000","last_undegraded":"2023-10-18T12:01:09.133066+0000","last_fullsized":"2023-10-18T12:01:09.133066+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:12.532840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136589+0000","last_change":"2023-10-18T12:01:09.133291+0000","last_active":"2023-10-18T12:01:09.136589+0000","last_peered":"2023-10-18T12:01:09.136589+0000","last_clean":"2023-10-18T12:01:09.136589+0000","last_became_active":"2023-10-18T12:01:09.132968+0000","last_became_peered":"2023-10-18T12:01:09.132968+0000","last_unstale":"2023-10-18T12:01:09.136589+0000","last_undegraded":"2023-10-18T12:01:09.136589+0000","last_fullsized":"2023-10-18T12:01:09.136589+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:10.172304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133058+0000","last_change":"2023-10-18T12:01:09.128954+0000","last_active":"2023-10-18T12:01:09.133058+0000","last_peered":"2023-10-18T12:01:09.133058+0000","last_clean":"2023-10-18T12:01:09.133058+0000","last_became_active":"2023-10-18T12:01:09.128655+0000","last_became_peered":"2023-10-18T12:01:09.128655+0000","last_unstale":"2023-10-18T12:01:09.133058+0000","last_undegraded":"2023-10-18T12:01:09.133058+0000","last_fullsized":"2023-10-18T12:01:09.133058+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:01.040240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.125729+0000","last_change":"2023-10-18T12:01:09.123355+0000","last_active":"2023-10-18T12:01:09.125729+0000","last_peered":"2023-10-18T12:01:09.125729+0000","last_clean":"2023-10-18T12:01:09.125729+0000","last_became_active":"2023-10-18T12:01:09.123096+0000","last_became_peered":"2023-10-18T12:01:09.123096+0000","last_unstale":"2023-10-18T12:01:09.125729+0000","last_undegraded":"2023-10-18T12:01:09.125729+0000","last_fullsized":"2023-10-18T12:01:09.125729+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-20T00:00:59.640033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127384+0000","last_change":"2023-10-18T12:01:09.124947+0000","last_active":"2023-10-18T12:01:09.127384+0000","last_peered":"2023-10-18T12:01:09.127384+0000","last_clean":"2023-10-18T12:01:09.127384+0000","last_became_active":"2023-10-18T12:01:09.124666+0000","last_became_peered":"2023-10-18T12:01:09.124666+0000","last_unstale":"2023-10-18T12:01:09.127384+0000","last_undegraded":"2023-10-18T12:01:09.127384+0000","last_fullsized":"2023-10-18T12:01:09.127384+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:26.664117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132971+0000","last_change":"2023-10-18T12:01:09.131674+0000","last_active":"2023-10-18T12:01:09.132971+0000","last_peered":"2023-10-18T12:01:09.132971+0000","last_clean":"2023-10-18T12:01:09.132971+0000","last_became_active":"2023-10-18T12:01:09.131410+0000","last_became_peered":"2023-10-18T12:01:09.131410+0000","last_unstale":"2023-10-18T12:01:09.132971+0000","last_undegraded":"2023-10-18T12:01:09.132971+0000","last_fullsized":"2023-10-18T12:01:09.132971+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:32.387861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.135855+0000","last_change":"2023-10-18T12:01:09.134865+0000","last_active":"2023-10-18T12:01:09.135855+0000","last_peered":"2023-10-18T12:01:09.135855+0000","last_clean":"2023-10-18T12:01:09.135855+0000","last_became_active":"2023-10-18T12:01:09.134583+0000","last_became_peered":"2023-10-18T12:01:09.134583+0000","last_unstale":"2023-10-18T12:01:09.135855+0000","last_undegraded":"2023-10-18T12:01:09.135855+0000","last_fullsized":"2023-10-18T12:01:09.135855+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:30.258369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.135747+0000","last_change":"2023-10-18T12:01:09.132312+0000","last_active":"2023-10-18T12:01:09.135747+0000","last_peered":"2023-10-18T12:01:09.135747+0000","last_clean":"2023-10-18T12:01:09.135747+0000","last_became_active":"2023-10-18T12:01:09.131781+0000","last_became_peered":"2023-10-18T12:01:09.131781+0000","last_unstale":"2023-10-18T12:01:09.135747+0000","last_undegraded":"2023-10-18T12:01:09.135747+0000","last_fullsized":"2023-10-18T12:01:09.135747+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:13:50.406605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136177+0000","last_change":"2023-10-18T12:01:09.133664+0000","last_active":"2023-10-18T12:01:09.136177+0000","last_peered":"2023-10-18T12:01:09.136177+0000","last_clean":"2023-10-18T12:01:09.136177+0000","last_became_active":"2023-10-18T12:01:09.133406+0000","last_became_peered":"2023-10-18T12:01:09.133406+0000","last_unstale":"2023-10-18T12:01:09.136177+0000","last_undegraded":"2023-10-18T12:01:09.136177+0000","last_fullsized":"2023-10-18T12:01:09.136177+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:03.436140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.131693+0000","last_change":"2023-10-18T12:01:09.130142+0000","last_active":"2023-10-18T12:01:09.131693+0000","last_peered":"2023-10-18T12:01:09.131693+0000","last_clean":"2023-10-18T12:01:09.131693+0000","last_became_active":"2023-10-18T12:01:09.129884+0000","last_became_peered":"2023-10-18T12:01:09.129884+0000","last_unstale":"2023-10-18T12:01:09.131693+0000","last_undegraded":"2023-10-18T12:01:09.131693+0000","last_fullsized":"2023-10-18T12:01:09.131693+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:40.659316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137256+0000","last_change":"2023-10-18T12:01:09.136066+0000","last_active":"2023-10-18T12:01:09.137256+0000","last_peered":"2023-10-18T12:01:09.137256+0000","last_clean":"2023-10-18T12:01:09.137256+0000","last_became_active":"2023-10-18T12:01:09.135802+0000","last_became_peered":"2023-10-18T12:01:09.135802+0000","last_unstale":"2023-10-18T12:01:09.137256+0000","last_undegraded":"2023-10-18T12:01:09.137256+0000","last_fullsized":"2023-10-18T12:01:09.137256+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:53:48.783775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.121736+0000","last_change":"2023-10-18T12:01:09.120812+0000","last_active":"2023-10-18T12:01:09.121736+0000","last_peered":"2023-10-18T12:01:09.121736+0000","last_clean":"2023-10-18T12:01:09.121736+0000","last_became_active":"2023-10-18T12:01:09.120561+0000","last_became_peered":"2023-10-18T12:01:09.120561+0000","last_unstale":"2023-10-18T12:01:09.121736+0000","last_undegraded":"2023-10-18T12:01:09.121736+0000","last_fullsized":"2023-10-18T12:01:09.121736+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:25.745623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"241'4","reported_seq":15,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.145642+0000","last_change":"2023-10-18T12:01:08.112445+0000","last_active":"2023-10-18T12:01:09.145642+0000","last_peered":"2023-10-18T12:01:09.145642+0000","last_clean":"2023-10-18T12:01:09.145642+0000","last_became_active":"2023-10-18T12:01:08.112017+0000","last_became_peered":"2023-10-18T12:01:08.112017+0000","last_unstale":"2023-10-18T12:01:09.145642+0000","last_undegraded":"2023-10-18T12:01:09.145642+0000","last_fullsized":"2023-10-18T12:01:09.145642+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:11.607481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136639+0000","last_change":"2023-10-18T12:01:09.134916+0000","last_active":"2023-10-18T12:01:09.136639+0000","last_peered":"2023-10-18T12:01:09.136639+0000","last_clean":"2023-10-18T12:01:09.136639+0000","last_became_active":"2023-10-18T12:01:09.134630+0000","last_became_peered":"2023-10-18T12:01:09.134630+0000","last_unstale":"2023-10-18T12:01:09.136639+0000","last_undegraded":"2023-10-18T12:01:09.136639+0000","last_fullsized":"2023-10-18T12:01:09.136639+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:44.846096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146787+0000","last_change":"2023-10-18T12:01:08.119870+0000","last_active":"2023-10-18T12:01:09.146787+0000","last_peered":"2023-10-18T12:01:09.146787+0000","last_clean":"2023-10-18T12:01:09.146787+0000","last_became_active":"2023-10-18T12:01:08.119586+0000","last_became_peered":"2023-10-18T12:01:08.119586+0000","last_unstale":"2023-10-18T12:01:09.146787+0000","last_undegraded":"2023-10-18T12:01:09.146787+0000","last_fullsized":"2023-10-18T12:01:09.146787+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:16.288493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.134216+0000","last_change":"2023-10-18T12:01:09.130894+0000","last_active":"2023-10-18T12:01:09.134216+0000","last_peered":"2023-10-18T12:01:09.134216+0000","last_clean":"2023-10-18T12:01:09.134216+0000","last_became_active":"2023-10-18T12:01:09.130614+0000","last_became_peered":"2023-10-18T12:01:09.130614+0000","last_unstale":"2023-10-18T12:01:09.134216+0000","last_undegraded":"2023-10-18T12:01:09.134216+0000","last_fullsized":"2023-10-18T12:01:09.134216+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:30:54.509159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.145024+0000","last_change":"2023-10-18T12:01:08.115174+0000","last_active":"2023-10-18T12:01:09.145024+0000","last_peered":"2023-10-18T12:01:09.145024+0000","last_clean":"2023-10-18T12:01:09.145024+0000","last_became_active":"2023-10-18T12:01:08.114546+0000","last_became_peered":"2023-10-18T12:01:08.114546+0000","last_unstale":"2023-10-18T12:01:09.145024+0000","last_undegraded":"2023-10-18T12:01:09.145024+0000","last_fullsized":"2023-10-18T12:01:09.145024+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.123903+0000","last_change":"2023-10-18T12:01:09.122601+0000","last_active":"2023-10-18T12:01:09.123903+0000","last_peered":"2023-10-18T12:01:09.123903+0000","last_clean":"2023-10-18T12:01:09.123903+0000","last_became_active":"2023-10-18T12:01:09.122334+0000","last_became_peered":"2023-10-18T12:01:09.122334+0000","last_unstale":"2023-10-18T12:01:09.123903+0000","last_undegraded":"2023-10-18T12:01:09.123903+0000","last_fullsized":"2023-10-18T12:01:09.123903+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:56.556729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146442+0000","last_change":"2023-10-18T12:01:08.116478+0000","last_active":"2023-10-18T12:01:09.146442+0000","last_peered":"2023-10-18T12:01:09.146442+0000","last_clean":"2023-10-18T12:01:09.146442+0000","last_became_active":"2023-10-18T12:01:08.116181+0000","last_became_peered":"2023-10-18T12:01:08.116181+0000","last_unstale":"2023-10-18T12:01:09.146442+0000","last_undegraded":"2023-10-18T12:01:09.146442+0000","last_fullsized":"2023-10-18T12:01:09.146442+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:10.087064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133703+0000","last_change":"2023-10-18T12:01:09.130166+0000","last_active":"2023-10-18T12:01:09.133703+0000","last_peered":"2023-10-18T12:01:09.133703+0000","last_clean":"2023-10-18T12:01:09.133703+0000","last_became_active":"2023-10-18T12:01:09.129725+0000","last_became_peered":"2023-10-18T12:01:09.129725+0000","last_unstale":"2023-10-18T12:01:09.133703+0000","last_undegraded":"2023-10-18T12:01:09.133703+0000","last_fullsized":"2023-10-18T12:01:09.133703+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:00:18.981967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.141464+0000","last_change":"2023-10-18T12:01:08.107385+0000","last_active":"2023-10-18T12:01:09.141464+0000","last_peered":"2023-10-18T12:01:09.141464+0000","last_clean":"2023-10-18T12:01:09.141464+0000","last_became_active":"2023-10-18T12:01:08.107129+0000","last_became_peered":"2023-10-18T12:01:08.107129+0000","last_unstale":"2023-10-18T12:01:09.141464+0000","last_undegraded":"2023-10-18T12:01:09.141464+0000","last_fullsized":"2023-10-18T12:01:09.141464+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:43.838857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.135346+0000","last_change":"2023-10-18T12:01:09.133737+0000","last_active":"2023-10-18T12:01:09.135346+0000","last_peered":"2023-10-18T12:01:09.135346+0000","last_clean":"2023-10-18T12:01:09.135346+0000","last_became_active":"2023-10-18T12:01:09.133487+0000","last_became_peered":"2023-10-18T12:01:09.133487+0000","last_unstale":"2023-10-18T12:01:09.135346+0000","last_undegraded":"2023-10-18T12:01:09.135346+0000","last_fullsized":"2023-10-18T12:01:09.135346+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:40.465745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.114949+0000","last_change":"2023-10-18T12:01:08.119435+0000","last_active":"2023-10-18T12:01:09.114949+0000","last_peered":"2023-10-18T12:01:09.114949+0000","last_clean":"2023-10-18T12:01:09.114949+0000","last_became_active":"2023-10-18T12:01:08.118740+0000","last_became_peered":"2023-10-18T12:01:08.118740+0000","last_unstale":"2023-10-18T12:01:09.114949+0000","last_undegraded":"2023-10-18T12:01:09.114949+0000","last_fullsized":"2023-10-18T12:01:09.114949+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:54.172871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133665+0000","last_change":"2023-10-18T12:01:09.132487+0000","last_active":"2023-10-18T12:01:09.133665+0000","last_peered":"2023-10-18T12:01:09.133665+0000","last_clean":"2023-10-18T12:01:09.133665+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:09.133665+0000","last_undegraded":"2023-10-18T12:01:09.133665+0000","last_fullsized":"2023-10-18T12:01:09.133665+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:49.358178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.117849+0000","last_change":"2023-10-18T12:01:08.115982+0000","last_active":"2023-10-18T12:01:09.117849+0000","last_peered":"2023-10-18T12:01:09.117849+0000","last_clean":"2023-10-18T12:01:09.117849+0000","last_became_active":"2023-10-18T12:01:08.115682+0000","last_became_peered":"2023-10-18T12:01:08.115682+0000","last_unstale":"2023-10-18T12:01:09.117849+0000","last_undegraded":"2023-10-18T12:01:09.117849+0000","last_fullsized":"2023-10-18T12:01:09.117849+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:36.789486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133512+0000","last_change":"2023-10-18T12:01:09.130010+0000","last_active":"2023-10-18T12:01:09.133512+0000","last_peered":"2023-10-18T12:01:09.133512+0000","last_clean":"2023-10-18T12:01:09.133512+0000","last_became_active":"2023-10-18T12:01:09.129744+0000","last_became_peered":"2023-10-18T12:01:09.129744+0000","last_unstale":"2023-10-18T12:01:09.133512+0000","last_undegraded":"2023-10-18T12:01:09.133512+0000","last_fullsized":"2023-10-18T12:01:09.133512+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:13:50.179465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.114469+0000","last_change":"2023-10-18T12:01:08.110833+0000","last_active":"2023-10-18T12:01:09.114469+0000","last_peered":"2023-10-18T12:01:09.114469+0000","last_clean":"2023-10-18T12:01:09.114469+0000","last_became_active":"2023-10-18T12:01:08.110005+0000","last_became_peered":"2023-10-18T12:01:08.110005+0000","last_unstale":"2023-10-18T12:01:09.114469+0000","last_undegraded":"2023-10-18T12:01:09.114469+0000","last_fullsized":"2023-10-18T12:01:09.114469+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:32.311648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133039+0000","last_change":"2023-10-18T12:01:09.129659+0000","last_active":"2023-10-18T12:01:09.133039+0000","last_peered":"2023-10-18T12:01:09.133039+0000","last_clean":"2023-10-18T12:01:09.133039+0000","last_became_active":"2023-10-18T12:01:09.129388+0000","last_became_peered":"2023-10-18T12:01:09.129388+0000","last_unstale":"2023-10-18T12:01:09.133039+0000","last_undegraded":"2023-10-18T12:01:09.133039+0000","last_fullsized":"2023-10-18T12:01:09.133039+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:35:25.388252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.115357+0000","last_change":"2023-10-18T12:01:08.118321+0000","last_active":"2023-10-18T12:01:09.115357+0000","last_peered":"2023-10-18T12:01:09.115357+0000","last_clean":"2023-10-18T12:01:09.115357+0000","last_became_active":"2023-10-18T12:01:08.117995+0000","last_became_peered":"2023-10-18T12:01:08.117995+0000","last_unstale":"2023-10-18T12:01:09.115357+0000","last_undegraded":"2023-10-18T12:01:09.115357+0000","last_fullsized":"2023-10-18T12:01:09.115357+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:46.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136242+0000","last_change":"2023-10-18T12:01:09.134396+0000","last_active":"2023-10-18T12:01:09.136242+0000","last_peered":"2023-10-18T12:01:09.136242+0000","last_clean":"2023-10-18T12:01:09.136242+0000","last_became_active":"2023-10-18T12:01:09.134144+0000","last_became_peered":"2023-10-18T12:01:09.134144+0000","last_unstale":"2023-10-18T12:01:09.136242+0000","last_undegraded":"2023-10-18T12:01:09.136242+0000","last_fullsized":"2023-10-18T12:01:09.136242+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:00.179085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"241'3","reported_seq":14,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.141686+0000","last_change":"2023-10-18T12:01:08.118871+0000","last_active":"2023-10-18T12:01:09.141686+0000","last_peered":"2023-10-18T12:01:09.141686+0000","last_clean":"2023-10-18T12:01:09.141686+0000","last_became_active":"2023-10-18T12:01:08.118572+0000","last_became_peered":"2023-10-18T12:01:08.118572+0000","last_unstale":"2023-10-18T12:01:09.141686+0000","last_undegraded":"2023-10-18T12:01:09.141686+0000","last_fullsized":"2023-10-18T12:01:09.141686+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:45.510206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.121494+0000","last_change":"2023-10-18T12:01:09.120444+0000","last_active":"2023-10-18T12:01:09.121494+0000","last_peered":"2023-10-18T12:01:09.121494+0000","last_clean":"2023-10-18T12:01:09.121494+0000","last_became_active":"2023-10-18T12:01:09.120018+0000","last_became_peered":"2023-10-18T12:01:09.120018+0000","last_unstale":"2023-10-18T12:01:09.121494+0000","last_undegraded":"2023-10-18T12:01:09.121494+0000","last_fullsized":"2023-10-18T12:01:09.121494+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:45.901809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"241'6","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.148493+0000","last_change":"2023-10-18T12:01:08.117890+0000","last_active":"2023-10-18T12:01:09.148493+0000","last_peered":"2023-10-18T12:01:09.148493+0000","last_clean":"2023-10-18T12:01:09.148493+0000","last_became_active":"2023-10-18T12:01:08.117606+0000","last_became_peered":"2023-10-18T12:01:08.117606+0000","last_unstale":"2023-10-18T12:01:09.148493+0000","last_undegraded":"2023-10-18T12:01:09.148493+0000","last_fullsized":"2023-10-18T12:01:09.148493+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:49.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.130907+0000","last_change":"2023-10-18T12:01:09.127054+0000","last_active":"2023-10-18T12:01:09.130907+0000","last_peered":"2023-10-18T12:01:09.130907+0000","last_clean":"2023-10-18T12:01:09.130907+0000","last_became_active":"2023-10-18T12:01:09.126804+0000","last_became_peered":"2023-10-18T12:01:09.126804+0000","last_unstale":"2023-10-18T12:01:09.130907+0000","last_undegraded":"2023-10-18T12:01:09.130907+0000","last_fullsized":"2023-10-18T12:01:09.130907+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:42.388425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146319+0000","last_change":"2023-10-18T12:01:08.118544+0000","last_active":"2023-10-18T12:01:09.146319+0000","last_peered":"2023-10-18T12:01:09.146319+0000","last_clean":"2023-10-18T12:01:09.146319+0000","last_became_active":"2023-10-18T12:01:08.118228+0000","last_became_peered":"2023-10-18T12:01:08.118228+0000","last_unstale":"2023-10-18T12:01:09.146319+0000","last_undegraded":"2023-10-18T12:01:09.146319+0000","last_fullsized":"2023-10-18T12:01:09.146319+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.124972+0000","last_change":"2023-10-18T12:01:09.120750+0000","last_active":"2023-10-18T12:01:09.124972+0000","last_peered":"2023-10-18T12:01:09.124972+0000","last_clean":"2023-10-18T12:01:09.124972+0000","last_became_active":"2023-10-18T12:01:09.120490+0000","last_became_peered":"2023-10-18T12:01:09.120490+0000","last_unstale":"2023-10-18T12:01:09.124972+0000","last_undegraded":"2023-10-18T12:01:09.124972+0000","last_fullsized":"2023-10-18T12:01:09.124972+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:50.958401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.123589+0000","last_change":"2023-10-18T12:01:08.118022+0000","last_active":"2023-10-18T12:01:09.123589+0000","last_peered":"2023-10-18T12:01:09.123589+0000","last_clean":"2023-10-18T12:01:09.123589+0000","last_became_active":"2023-10-18T12:01:08.117754+0000","last_became_peered":"2023-10-18T12:01:08.117754+0000","last_unstale":"2023-10-18T12:01:09.123589+0000","last_undegraded":"2023-10-18T12:01:09.123589+0000","last_fullsized":"2023-10-18T12:01:09.123589+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:37.834293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.137518+0000","last_change":"2023-10-18T12:01:09.135383+0000","last_active":"2023-10-18T12:01:09.137518+0000","last_peered":"2023-10-18T12:01:09.137518+0000","last_clean":"2023-10-18T12:01:09.137518+0000","last_became_active":"2023-10-18T12:01:09.135114+0000","last_became_peered":"2023-10-18T12:01:09.135114+0000","last_unstale":"2023-10-18T12:01:09.137518+0000","last_undegraded":"2023-10-18T12:01:09.137518+0000","last_fullsized":"2023-10-18T12:01:09.137518+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:58.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.121505+0000","last_change":"2023-10-18T12:01:08.115155+0000","last_active":"2023-10-18T12:01:09.121505+0000","last_peered":"2023-10-18T12:01:09.121505+0000","last_clean":"2023-10-18T12:01:09.121505+0000","last_became_active":"2023-10-18T12:01:08.114843+0000","last_became_peered":"2023-10-18T12:01:08.114843+0000","last_unstale":"2023-10-18T12:01:09.121505+0000","last_undegraded":"2023-10-18T12:01:09.121505+0000","last_fullsized":"2023-10-18T12:01:09.121505+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:28.358756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132946+0000","last_change":"2023-10-18T12:01:09.131820+0000","last_active":"2023-10-18T12:01:09.132946+0000","last_peered":"2023-10-18T12:01:09.132946+0000","last_clean":"2023-10-18T12:01:09.132946+0000","last_became_active":"2023-10-18T12:01:09.131568+0000","last_became_peered":"2023-10-18T12:01:09.131568+0000","last_unstale":"2023-10-18T12:01:09.132946+0000","last_undegraded":"2023-10-18T12:01:09.132946+0000","last_fullsized":"2023-10-18T12:01:09.132946+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:18:19.604112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"241'8","reported_seq":19,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.147954+0000","last_change":"2023-10-18T12:01:08.112711+0000","last_active":"2023-10-18T12:01:09.147954+0000","last_peered":"2023-10-18T12:01:09.147954+0000","last_clean":"2023-10-18T12:01:09.147954+0000","last_became_active":"2023-10-18T12:01:08.112436+0000","last_became_peered":"2023-10-18T12:01:08.112436+0000","last_unstale":"2023-10-18T12:01:09.147954+0000","last_undegraded":"2023-10-18T12:01:09.147954+0000","last_fullsized":"2023-10-18T12:01:09.147954+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:15.568916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136856+0000","last_change":"2023-10-18T12:01:09.135106+0000","last_active":"2023-10-18T12:01:09.136856+0000","last_peered":"2023-10-18T12:01:09.136856+0000","last_clean":"2023-10-18T12:01:09.136856+0000","last_became_active":"2023-10-18T12:01:09.134828+0000","last_became_peered":"2023-10-18T12:01:09.134828+0000","last_unstale":"2023-10-18T12:01:09.136856+0000","last_undegraded":"2023-10-18T12:01:09.136856+0000","last_fullsized":"2023-10-18T12:01:09.136856+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:22.643933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.143890+0000","last_change":"2023-10-18T12:01:08.119248+0000","last_active":"2023-10-18T12:01:09.143890+0000","last_peered":"2023-10-18T12:01:09.143890+0000","last_clean":"2023-10-18T12:01:09.143890+0000","last_became_active":"2023-10-18T12:01:08.118973+0000","last_became_peered":"2023-10-18T12:01:08.118973+0000","last_unstale":"2023-10-18T12:01:09.143890+0000","last_undegraded":"2023-10-18T12:01:09.143890+0000","last_fullsized":"2023-10-18T12:01:09.143890+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:37.079812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133576+0000","last_change":"2023-10-18T12:01:09.132827+0000","last_active":"2023-10-18T12:01:09.133576+0000","last_peered":"2023-10-18T12:01:09.133576+0000","last_clean":"2023-10-18T12:01:09.133576+0000","last_became_active":"2023-10-18T12:01:09.132545+0000","last_became_peered":"2023-10-18T12:01:09.132545+0000","last_unstale":"2023-10-18T12:01:09.133576+0000","last_undegraded":"2023-10-18T12:01:09.133576+0000","last_fullsized":"2023-10-18T12:01:09.133576+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:48.855866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146651+0000","last_change":"2023-10-18T12:01:08.107964+0000","last_active":"2023-10-18T12:01:09.146651+0000","last_peered":"2023-10-18T12:01:09.146651+0000","last_clean":"2023-10-18T12:01:09.146651+0000","last_became_active":"2023-10-18T12:01:08.107696+0000","last_became_peered":"2023-10-18T12:01:08.107696+0000","last_unstale":"2023-10-18T12:01:09.146651+0000","last_undegraded":"2023-10-18T12:01:09.146651+0000","last_fullsized":"2023-10-18T12:01:09.146651+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:18.578919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136873+0000","last_change":"2023-10-18T12:01:09.136078+0000","last_active":"2023-10-18T12:01:09.136873+0000","last_peered":"2023-10-18T12:01:09.136873+0000","last_clean":"2023-10-18T12:01:09.136873+0000","last_became_active":"2023-10-18T12:01:09.134722+0000","last_became_peered":"2023-10-18T12:01:09.134722+0000","last_unstale":"2023-10-18T12:01:09.136873+0000","last_undegraded":"2023-10-18T12:01:09.136873+0000","last_fullsized":"2023-10-18T12:01:09.136873+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:46.059426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.146802+0000","last_change":"2023-10-18T12:01:08.120416+0000","last_active":"2023-10-18T12:01:09.146802+0000","last_peered":"2023-10-18T12:01:09.146802+0000","last_clean":"2023-10-18T12:01:09.146802+0000","last_became_active":"2023-10-18T12:01:08.120122+0000","last_became_peered":"2023-10-18T12:01:08.120122+0000","last_unstale":"2023-10-18T12:01:09.146802+0000","last_undegraded":"2023-10-18T12:01:09.146802+0000","last_fullsized":"2023-10-18T12:01:09.146802+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:26.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136217+0000","last_change":"2023-10-18T12:01:09.135042+0000","last_active":"2023-10-18T12:01:09.136217+0000","last_peered":"2023-10-18T12:01:09.136217+0000","last_clean":"2023-10-18T12:01:09.136217+0000","last_became_active":"2023-10-18T12:01:09.134765+0000","last_became_peered":"2023-10-18T12:01:09.134765+0000","last_unstale":"2023-10-18T12:01:09.136217+0000","last_undegraded":"2023-10-18T12:01:09.136217+0000","last_fullsized":"2023-10-18T12:01:09.136217+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:18:15.700428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.122997+0000","last_change":"2023-10-18T12:01:08.117972+0000","last_active":"2023-10-18T12:01:09.122997+0000","last_peered":"2023-10-18T12:01:09.122997+0000","last_clean":"2023-10-18T12:01:09.122997+0000","last_became_active":"2023-10-18T12:01:08.117690+0000","last_became_peered":"2023-10-18T12:01:08.117690+0000","last_unstale":"2023-10-18T12:01:09.122997+0000","last_undegraded":"2023-10-18T12:01:09.122997+0000","last_fullsized":"2023-10-18T12:01:09.122997+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.124607+0000","last_change":"2023-10-18T12:01:09.122673+0000","last_active":"2023-10-18T12:01:09.124607+0000","last_peered":"2023-10-18T12:01:09.124607+0000","last_clean":"2023-10-18T12:01:09.124607+0000","last_became_active":"2023-10-18T12:01:09.122430+0000","last_became_peered":"2023-10-18T12:01:09.122430+0000","last_unstale":"2023-10-18T12:01:09.124607+0000","last_undegraded":"2023-10-18T12:01:09.124607+0000","last_fullsized":"2023-10-18T12:01:09.124607+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:08.858055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.143964+0000","last_change":"2023-10-18T12:01:08.119439+0000","last_active":"2023-10-18T12:01:09.143964+0000","last_peered":"2023-10-18T12:01:09.143964+0000","last_clean":"2023-10-18T12:01:09.143964+0000","last_became_active":"2023-10-18T12:01:08.119158+0000","last_became_peered":"2023-10-18T12:01:08.119158+0000","last_unstale":"2023-10-18T12:01:09.143964+0000","last_undegraded":"2023-10-18T12:01:09.143964+0000","last_fullsized":"2023-10-18T12:01:09.143964+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:56.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127327+0000","last_change":"2023-10-18T12:01:09.124352+0000","last_active":"2023-10-18T12:01:09.127327+0000","last_peered":"2023-10-18T12:01:09.127327+0000","last_clean":"2023-10-18T12:01:09.127327+0000","last_became_active":"2023-10-18T12:01:09.124072+0000","last_became_peered":"2023-10-18T12:01:09.124072+0000","last_unstale":"2023-10-18T12:01:09.127327+0000","last_undegraded":"2023-10-18T12:01:09.127327+0000","last_fullsized":"2023-10-18T12:01:09.127327+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:54:30.629590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.126469+0000","last_change":"2023-10-18T12:01:08.117335+0000","last_active":"2023-10-18T12:01:09.126469+0000","last_peered":"2023-10-18T12:01:09.126469+0000","last_clean":"2023-10-18T12:01:09.126469+0000","last_became_active":"2023-10-18T12:01:08.117032+0000","last_became_peered":"2023-10-18T12:01:08.117032+0000","last_unstale":"2023-10-18T12:01:09.126469+0000","last_undegraded":"2023-10-18T12:01:09.126469+0000","last_fullsized":"2023-10-18T12:01:09.126469+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:33.304569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.136754+0000","last_change":"2023-10-18T12:01:09.135148+0000","last_active":"2023-10-18T12:01:09.136754+0000","last_peered":"2023-10-18T12:01:09.136754+0000","last_clean":"2023-10-18T12:01:09.136754+0000","last_became_active":"2023-10-18T12:01:09.134895+0000","last_became_peered":"2023-10-18T12:01:09.134895+0000","last_unstale":"2023-10-18T12:01:09.136754+0000","last_undegraded":"2023-10-18T12:01:09.136754+0000","last_fullsized":"2023-10-18T12:01:09.136754+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:26.884874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.120652+0000","last_change":"2023-10-18T12:01:08.120618+0000","last_active":"2023-10-18T12:01:09.120652+0000","last_peered":"2023-10-18T12:01:09.120652+0000","last_clean":"2023-10-18T12:01:09.120652+0000","last_became_active":"2023-10-18T12:01:08.120315+0000","last_became_peered":"2023-10-18T12:01:08.120315+0000","last_unstale":"2023-10-18T12:01:09.120652+0000","last_undegraded":"2023-10-18T12:01:09.120652+0000","last_fullsized":"2023-10-18T12:01:09.120652+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:36.555722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133217+0000","last_change":"2023-10-18T12:01:09.130881+0000","last_active":"2023-10-18T12:01:09.133217+0000","last_peered":"2023-10-18T12:01:09.133217+0000","last_clean":"2023-10-18T12:01:09.133217+0000","last_became_active":"2023-10-18T12:01:09.130628+0000","last_became_peered":"2023-10-18T12:01:09.130628+0000","last_unstale":"2023-10-18T12:01:09.133217+0000","last_undegraded":"2023-10-18T12:01:09.133217+0000","last_fullsized":"2023-10-18T12:01:09.133217+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:02.473534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.114247+0000","last_change":"2023-10-18T12:01:08.120313+0000","last_active":"2023-10-18T12:01:09.114247+0000","last_peered":"2023-10-18T12:01:09.114247+0000","last_clean":"2023-10-18T12:01:09.114247+0000","last_became_active":"2023-10-18T12:01:08.120059+0000","last_became_peered":"2023-10-18T12:01:08.120059+0000","last_unstale":"2023-10-18T12:01:09.114247+0000","last_undegraded":"2023-10-18T12:01:09.114247+0000","last_fullsized":"2023-10-18T12:01:09.114247+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127524+0000","last_change":"2023-10-18T12:01:09.125685+0000","last_active":"2023-10-18T12:01:09.127524+0000","last_peered":"2023-10-18T12:01:09.127524+0000","last_clean":"2023-10-18T12:01:09.127524+0000","last_became_active":"2023-10-18T12:01:09.125395+0000","last_became_peered":"2023-10-18T12:01:09.125395+0000","last_unstale":"2023-10-18T12:01:09.127524+0000","last_undegraded":"2023-10-18T12:01:09.127524+0000","last_fullsized":"2023-10-18T12:01:09.127524+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:26:35.778496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"241'2","reported_seq":13,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.142593+0000","last_change":"2023-10-18T12:01:08.117717+0000","last_active":"2023-10-18T12:01:09.142593+0000","last_peered":"2023-10-18T12:01:09.142593+0000","last_clean":"2023-10-18T12:01:09.142593+0000","last_became_active":"2023-10-18T12:01:08.117405+0000","last_became_peered":"2023-10-18T12:01:08.117405+0000","last_unstale":"2023-10-18T12:01:09.142593+0000","last_undegraded":"2023-10-18T12:01:09.142593+0000","last_fullsized":"2023-10-18T12:01:09.142593+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:00.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133286+0000","last_change":"2023-10-18T12:01:09.132599+0000","last_active":"2023-10-18T12:01:09.133286+0000","last_peered":"2023-10-18T12:01:09.133286+0000","last_clean":"2023-10-18T12:01:09.133286+0000","last_became_active":"2023-10-18T12:01:09.131710+0000","last_became_peered":"2023-10-18T12:01:09.131710+0000","last_unstale":"2023-10-18T12:01:09.133286+0000","last_undegraded":"2023-10-18T12:01:09.133286+0000","last_fullsized":"2023-10-18T12:01:09.133286+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:33.318887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.122306+0000","last_change":"2023-10-18T12:01:08.118553+0000","last_active":"2023-10-18T12:01:09.122306+0000","last_peered":"2023-10-18T12:01:09.122306+0000","last_clean":"2023-10-18T12:01:09.122306+0000","last_became_active":"2023-10-18T12:01:08.118286+0000","last_became_peered":"2023-10-18T12:01:08.118286+0000","last_unstale":"2023-10-18T12:01:09.122306+0000","last_undegraded":"2023-10-18T12:01:09.122306+0000","last_fullsized":"2023-10-18T12:01:09.122306+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:05:32.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.134491+0000","last_change":"2023-10-18T12:01:09.133522+0000","last_active":"2023-10-18T12:01:09.134491+0000","last_peered":"2023-10-18T12:01:09.134491+0000","last_clean":"2023-10-18T12:01:09.134491+0000","last_became_active":"2023-10-18T12:01:09.133267+0000","last_became_peered":"2023-10-18T12:01:09.133267+0000","last_unstale":"2023-10-18T12:01:09.134491+0000","last_undegraded":"2023-10-18T12:01:09.134491+0000","last_fullsized":"2023-10-18T12:01:09.134491+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:15:56.382925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.114721+0000","last_change":"2023-10-18T12:01:08.116830+0000","last_active":"2023-10-18T12:01:09.114721+0000","last_peered":"2023-10-18T12:01:09.114721+0000","last_clean":"2023-10-18T12:01:09.114721+0000","last_became_active":"2023-10-18T12:01:08.116538+0000","last_became_peered":"2023-10-18T12:01:08.116538+0000","last_unstale":"2023-10-18T12:01:09.114721+0000","last_undegraded":"2023-10-18T12:01:09.114721+0000","last_fullsized":"2023-10-18T12:01:09.114721+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:09.165065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.132808+0000","last_change":"2023-10-18T12:01:09.128305+0000","last_active":"2023-10-18T12:01:09.132808+0000","last_peered":"2023-10-18T12:01:09.132808+0000","last_clean":"2023-10-18T12:01:09.132808+0000","last_became_active":"2023-10-18T12:01:09.127933+0000","last_became_peered":"2023-10-18T12:01:09.127933+0000","last_unstale":"2023-10-18T12:01:09.132808+0000","last_undegraded":"2023-10-18T12:01:09.132808+0000","last_fullsized":"2023-10-18T12:01:09.132808+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:51:23.836756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.115421+0000","last_change":"2023-10-18T12:01:08.115363+0000","last_active":"2023-10-18T12:01:09.115421+0000","last_peered":"2023-10-18T12:01:09.115421+0000","last_clean":"2023-10-18T12:01:09.115421+0000","last_became_active":"2023-10-18T12:01:08.114903+0000","last_became_peered":"2023-10-18T12:01:08.114903+0000","last_unstale":"2023-10-18T12:01:09.115421+0000","last_undegraded":"2023-10-18T12:01:09.115421+0000","last_fullsized":"2023-10-18T12:01:09.115421+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:46.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.129367+0000","last_change":"2023-10-18T12:01:09.127159+0000","last_active":"2023-10-18T12:01:09.129367+0000","last_peered":"2023-10-18T12:01:09.129367+0000","last_clean":"2023-10-18T12:01:09.129367+0000","last_became_active":"2023-10-18T12:01:09.126874+0000","last_became_peered":"2023-10-18T12:01:09.126874+0000","last_unstale":"2023-10-18T12:01:09.129367+0000","last_undegraded":"2023-10-18T12:01:09.129367+0000","last_fullsized":"2023-10-18T12:01:09.129367+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:11.094303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.127585+0000","last_change":"2023-10-18T12:01:08.118539+0000","last_active":"2023-10-18T12:01:09.127585+0000","last_peered":"2023-10-18T12:01:09.127585+0000","last_clean":"2023-10-18T12:01:09.127585+0000","last_became_active":"2023-10-18T12:01:08.116966+0000","last_became_peered":"2023-10-18T12:01:08.116966+0000","last_unstale":"2023-10-18T12:01:09.127585+0000","last_undegraded":"2023-10-18T12:01:09.127585+0000","last_fullsized":"2023-10-18T12:01:09.127585+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:20.828671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.120097+0000","last_change":"2023-10-18T12:01:09.119148+0000","last_active":"2023-10-18T12:01:09.120097+0000","last_peered":"2023-10-18T12:01:09.120097+0000","last_clean":"2023-10-18T12:01:09.120097+0000","last_became_active":"2023-10-18T12:01:09.118894+0000","last_became_peered":"2023-10-18T12:01:09.118894+0000","last_unstale":"2023-10-18T12:01:09.120097+0000","last_undegraded":"2023-10-18T12:01:09.120097+0000","last_fullsized":"2023-10-18T12:01:09.120097+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:32:52.033269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.123725+0000","last_change":"2023-10-18T12:01:08.117526+0000","last_active":"2023-10-18T12:01:09.123725+0000","last_peered":"2023-10-18T12:01:09.123725+0000","last_clean":"2023-10-18T12:01:09.123725+0000","last_became_active":"2023-10-18T12:01:08.117234+0000","last_became_peered":"2023-10-18T12:01:08.117234+0000","last_unstale":"2023-10-18T12:01:09.123725+0000","last_undegraded":"2023-10-18T12:01:09.123725+0000","last_fullsized":"2023-10-18T12:01:09.123725+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:43.295217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.128333+0000","last_change":"2023-10-18T12:01:09.125150+0000","last_active":"2023-10-18T12:01:09.128333+0000","last_peered":"2023-10-18T12:01:09.128333+0000","last_clean":"2023-10-18T12:01:09.128333+0000","last_became_active":"2023-10-18T12:01:09.124900+0000","last_became_peered":"2023-10-18T12:01:09.124900+0000","last_unstale":"2023-10-18T12:01:09.128333+0000","last_undegraded":"2023-10-18T12:01:09.128333+0000","last_fullsized":"2023-10-18T12:01:09.128333+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:37.454945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.115056+0000","last_change":"2023-10-18T12:01:08.119612+0000","last_active":"2023-10-18T12:01:09.115056+0000","last_peered":"2023-10-18T12:01:09.115056+0000","last_clean":"2023-10-18T12:01:09.115056+0000","last_became_active":"2023-10-18T12:01:08.119116+0000","last_became_peered":"2023-10-18T12:01:08.119116+0000","last_unstale":"2023-10-18T12:01:09.115056+0000","last_undegraded":"2023-10-18T12:01:09.115056+0000","last_fullsized":"2023-10-18T12:01:09.115056+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:02.536144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.133199+0000","last_change":"2023-10-18T12:01:09.130360+0000","last_active":"2023-10-18T12:01:09.133199+0000","last_peered":"2023-10-18T12:01:09.133199+0000","last_clean":"2023-10-18T12:01:09.133199+0000","last_became_active":"2023-10-18T12:01:09.130086+0000","last_became_peered":"2023-10-18T12:01:09.130086+0000","last_unstale":"2023-10-18T12:01:09.133199+0000","last_undegraded":"2023-10-18T12:01:09.133199+0000","last_fullsized":"2023-10-18T12:01:09.133199+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:15.311291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":11,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.120114+0000","last_change":"2023-10-18T12:01:08.119803+0000","last_active":"2023-10-18T12:01:09.120114+0000","last_peered":"2023-10-18T12:01:09.120114+0000","last_clean":"2023-10-18T12:01:09.120114+0000","last_became_active":"2023-10-18T12:01:08.119467+0000","last_became_peered":"2023-10-18T12:01:08.119467+0000","last_unstale":"2023-10-18T12:01:09.120114+0000","last_undegraded":"2023-10-18T12:01:09.120114+0000","last_fullsized":"2023-10-18T12:01:09.120114+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:29.410010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.123340+0000","last_change":"2023-10-18T12:01:09.122670+0000","last_active":"2023-10-18T12:01:09.123340+0000","last_peered":"2023-10-18T12:01:09.123340+0000","last_clean":"2023-10-18T12:01:09.123340+0000","last_became_active":"2023-10-18T12:01:09.122408+0000","last_became_peered":"2023-10-18T12:01:09.122408+0000","last_unstale":"2023-10-18T12:01:09.123340+0000","last_undegraded":"2023-10-18T12:01:09.123340+0000","last_fullsized":"2023-10-18T12:01:09.123340+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:16.307249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"241'1","reported_seq":12,"reported_epoch":241,"state":"active+clean","last_fresh":"2023-10-18T12:01:09.143074+0000","last_change":"2023-10-18T12:01:08.111055+0000","last_active":"2023-10-18T12:01:09.143074+0000","last_peered":"2023-10-18T12:01:09.143074+0000","last_clean":"2023-10-18T12:01:09.143074+0000","last_became_active":"2023-10-18T12:01:08.110728+0000","last_became_peered":"2023-10-18T12:01:08.110728+0000","last_unstale":"2023-10-18T12:01:09.143074+0000","last_undegraded":"2023-10-18T12:01:09.143074+0000","last_fullsized":"2023-10-18T12:01:09.143074+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:15.472803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"227'379","reported_seq":1328,"reported_epoch":240,"state":"active+clean","last_fresh":"2023-10-18T12:01:08.110987+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:01:08.110987+0000","last_peered":"2023-10-18T12:01:08.110987+0000","last_clean":"2023-10-18T12:01:08.110987+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:01:08.110987+0000","last_undegraded":"2023-10-18T12:01:08.110987+0000","last_fullsized":"2023-10-18T12:01:08.110987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":379,"ondisk_log_size":379,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575998,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52788,"kb_used_data":3556,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319052,"statfs":{"total":96636764160,"available":96582709248,"internally_reserved":0,"allocated":3641344,"data_stored":2155554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.54300000000000004}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.48899999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.627}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.56699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42476,"kb_used_data":2976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329364,"statfs":{"total":96636764160,"available":96593268736,"internally_reserved":0,"allocated":3047424,"data_stored":1565174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.57599999999999996}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.436}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.51500000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.501},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48199999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64676,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307164,"statfs":{"total":96636764160,"available":96570535936,"internally_reserved":0,"allocated":3039232,"data_stored":1565062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72199999999999998,"5min":0.92400000000000004,"15min":0.96099999999999997},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.40100000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.745,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.78700000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.74099999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.755,"5min":0.80500000000000005,"15min":1.0349999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.45000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.71399999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":1},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52100000000000002,"15min":0.53800000000000003},"min":{"1min":0.377,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76900000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.01},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.41299999999999998,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.872}]},{"osd":7,"last update":"Wed Oct 18 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.56399999999999995},"min":{"1min":0.35099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":1.101,"15min":1.101},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81499999999999995,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.80900000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48040,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323800,"statfs":{"total":96636764160,"available":96587571200,"internally_reserved":0,"allocated":3043328,"data_stored":1565132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.22500000000000001},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.755},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":1.123},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69999999999999996}]},{"osd":3,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.63100000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.49299999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":1.018,"15min":1.018},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.88900000000000001,"15min":1.05},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48072,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323768,"statfs":{"total":96636764160,"available":96587538432,"internally_reserved":0,"allocated":3063808,"data_stored":1567312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.23200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55000000000000004,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.309},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40899999999999997,"15min":0.41399999999999998},"min":{"1min":0.27600000000000002,"5min":0.25900000000000001,"15min":0.23400000000000001},"max":{"1min":0.56699999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41599999999999998,"15min":0.41299999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45000000000000001,"15min":0.44500000000000001},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66000000000000003,"5min":1.899,"15min":1.899},"last":0.27400000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.29299999999999998,"15min":0.28399999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88400000000000001,"5min":1.034,"15min":1.052},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":0.84699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45300000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.755,"5min":1.522,"15min":1.522},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40999999999999998},"max":{"1min":0.89600000000000002,"5min":0.91100000000000003,"15min":1.026},"last":0.48499999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85899999999999999,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":1.01,"15min":1.01},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:00:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67100000000000004,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81799999999999995,"5min":1.129,"15min":1.129},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.93899999999999995,"15min":1.0649999999999999},"last":0.57399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575998,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60524,"kb_used_data":2976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311316,"statfs":{"total":96636764160,"available":96574787584,"internally_reserved":0,"allocated":3047424,"data_stored":1565714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.216},"max":{"1min":0.56999999999999995,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.56399999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61499999999999999,"15min":0.70199999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.443},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.504}]},{"osd":3,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.64600000000000002,"15min":0.65300000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65300000000000002,"5min":0.69599999999999995,"15min":0.80300000000000005},"last":0.55300000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.68500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.90600000000000003,"5min":1.087,"15min":1.087},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.755},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94699999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66900000000000004,"15min":0.67400000000000004},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83399999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84999999999999998,"5min":0.89100000000000001,"15min":0.92000000000000004},"last":0.79300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60520,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311320,"statfs":{"total":96636764160,"available":96574791680,"internally_reserved":0,"allocated":3039232,"data_stored":1565062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.307},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.376}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.35999999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.623}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.59599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43072,"kb_used_data":3560,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328768,"statfs":{"total":96636764160,"available":96592658432,"internally_reserved":0,"allocated":3645440,"data_stored":2157042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.54700000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61899999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.437},"min":{"1min":0.28899999999999998,"5min":0.245,"15min":0.24099999999999999},"max":{"1min":0.66800000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.312}]},{"osd":3,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.25700000000000001},"max":{"1min":0.62,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.435},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.56899999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.96299999999999997,"5min":0.96999999999999997,"15min":0.997},"last":0.505},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.876,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.53900000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88200000000000001,"5min":0.95799999999999996,"15min":0.96399999999999997},"last":0.71799999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66900000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":1.0089999999999999,"15min":1.03},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65100000000000002,"15min":0.67000000000000004},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.60599999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65800000000000003,"15min":0.68100000000000005},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.82699999999999996,"5min":0.997,"15min":0.997},"last":0.65400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:01:12.675 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T12:01:12.675 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T12:01:12.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.816+0000 7f960dc15640 1 -- 172.21.15.125:0/1140772533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 msgr2=0x7f960806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.816+0000 7f960dc15640 1 --2- 172.21.15.125:0/1140772533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 0x7f960806b410 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f96080cb9b0 tx=0x7f95f802bd30 comp rx=0 tx=0).stop 2023-10-18T12:01:12.819 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.816+0000 7f960dc15640 1 -- 172.21.15.125:0/1140772533 shutdown_connections 2023-10-18T12:01:12.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.816+0000 7f960dc15640 1 --2- 172.21.15.125:0/1140772533 >> v2:172.21.15.139:3300/0 conn(0x7f9608159bd0 0x7f960815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.816+0000 7f960dc15640 1 --2- 172.21.15.125:0/1140772533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 0x7f960806b410 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.816+0000 7f960dc15640 1 -- 172.21.15.125:0/1140772533 >> 172.21.15.125:0/1140772533 conn(0x7f9608069930 msgr2=0x7f9608069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:12.820 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.816+0000 7f960dc15640 1 -- 172.21.15.125:0/1140772533 shutdown_connections 2023-10-18T12:01:12.821 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.816+0000 7f960dc15640 1 -- 172.21.15.125:0/1140772533 wait complete. 2023-10-18T12:01:12.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.817+0000 7f960dc15640 1 Processor -- start 2023-10-18T12:01:12.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.817+0000 7f960dc15640 1 -- start start 2023-10-18T12:01:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f960dc15640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 0x7f9608153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f960dc15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9608159bd0 0x7f96080aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f960dc15640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f960815ccb0 0x7f9608130650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f960dc15640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960812f890 con 0x7f960806b030 2023-10-18T12:01:12.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f960dc15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960812fb70 con 0x7f9608159bd0 2023-10-18T12:01:12.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f960dc15640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960812fe50 con 0x7f960815ccb0 2023-10-18T12:01:12.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f96077fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 0x7f9608153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f96077fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 0x7f9608153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35970/0 (socket says 172.21.15.125:35970) 2023-10-18T12:01:12.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f96077fe640 1 -- 172.21.15.125:0/2561350961 learned_addr learned my addr 172.21.15.125:0/2561350961 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:12.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f96077fe640 1 -- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f960815ccb0 msgr2=0x7f9608130650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.825 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f96077fe640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f960815ccb0 0x7f9608130650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f96077fe640 1 -- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9608159bd0 msgr2=0x7f96080aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:12.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f96077fe640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9608159bd0 0x7f96080aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:12.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.818+0000 7f96077fe640 1 -- 172.21.15.125:0/2561350961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f8009560 con 0x7f960806b030 2023-10-18T12:01:12.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.819+0000 7f96077fe640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 0x7f9608153840 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f95f8036040 tx=0x7f95f802dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:12.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.819+0000 7f95effff640 1 -- 172.21.15.125:0/2561350961 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8003800 con 0x7f960806b030 2023-10-18T12:01:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.819+0000 7f95effff640 1 -- 172.21.15.125:0/2561350961 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95f8003a50 con 0x7f960806b030 2023-10-18T12:01:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.819+0000 7f95effff640 1 -- 172.21.15.125:0/2561350961 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8003de0 con 0x7f960806b030 2023-10-18T12:01:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.819+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9608130130 con 0x7f960806b030 2023-10-18T12:01:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.819+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9608130b90 con 0x7f960806b030 2023-10-18T12:01:12.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.820+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960812ff90 con 0x7f960806b030 2023-10-18T12:01:12.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.820+0000 7f95effff640 1 -- 172.21.15.125:0/2561350961 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f95f803e450 con 0x7f960806b030 2023-10-18T12:01:12.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.820+0000 7f95effff640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f95f406b080 0x7f95f406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:12.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.821+0000 7f95effff640 1 -- 172.21.15.125:0/2561350961 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f95f80ac5d0 con 0x7f960806b030 2023-10-18T12:01:12.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.821+0000 7f9606ffd640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f95f406b080 0x7f95f406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:12.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.821+0000 7f9606ffd640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f95f406b080 0x7f95f406d540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96080ad470 tx=0x7f95fc005c70 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:12.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:12.825+0000 7f95effff640 1 -- 172.21.15.125:0/2561350961 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f960812ff90 con 0x7f960806b030 2023-10-18T12:01:13.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.052+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9608130370 con 0x7f960806b030 2023-10-18T12:01:13.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.054+0000 7f95effff640 1 -- 172.21.15.125:0/2561350961 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v59) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9608130370 con 0x7f960806b030 2023-10-18T12:01:13.057 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T12:01:13.057 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T12:01:13.058 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T12:01:13.058 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T12:01:13.058 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T12:01:13.059 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T12:01:13.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f95f406b080 msgr2=0x7f95f406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f95f406b080 0x7f95f406d540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96080ad470 tx=0x7f95fc005c70 comp rx=0 tx=0).stop 2023-10-18T12:01:13.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 msgr2=0x7f9608153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 0x7f9608153840 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f95f8036040 tx=0x7f95f802dd00 comp rx=0 tx=0).stop 2023-10-18T12:01:13.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 shutdown_connections 2023-10-18T12:01:13.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f95f406b080 0x7f95f406d540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9608159bd0 0x7f96080aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f960815ccb0 0x7f9608130650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 --2- 172.21.15.125:0/2561350961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f960806b030 0x7f9608153840 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 >> 172.21.15.125:0/2561350961 conn(0x7f9608069930 msgr2=0x7f9608152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 shutdown_connections 2023-10-18T12:01:13.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.056+0000 7f960dc15640 1 -- 172.21.15.125:0/2561350961 wait complete. 2023-10-18T12:01:13.070 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:13.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.171+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/3701553939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d700039a0 msgr2=0x7f1d70003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.171+0000 7f1d8cb7d640 1 --2- 172.21.15.125:0/3701553939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d700039a0 0x7f1d70003dc0 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c001680 tx=0x7f1d700057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.171+0000 7f1d86d9d640 1 -- 172.21.15.125:0/3701553939 reap_dead start 2023-10-18T12:01:13.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.171+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/3701553939 shutdown_connections 2023-10-18T12:01:13.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.171+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/3701553939 >> 172.21.15.125:0/3701553939 conn(0x7f1d88069930 msgr2=0x7f1d88069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.171+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/3701553939 shutdown_connections 2023-10-18T12:01:13.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.171+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/3701553939 wait complete. 2023-10-18T12:01:13.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.172+0000 7f1d8cb7d640 1 Processor -- start 2023-10-18T12:01:13.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.172+0000 7f1d8cb7d640 1 -- start start 2023-10-18T12:01:13.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d8cb7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700039a0 0x7f1d881d9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d8cb7d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d881dc490 0x7f1d881da550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d8cb7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d881dc910 0x7f1d881daa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d8cb7d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881ddb40 con 0x7f1d881dc910 2023-10-18T12:01:13.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d8cb7d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881d9950 con 0x7f1d700039a0 2023-10-18T12:01:13.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d8cb7d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881dcec0 con 0x7f1d881dc490 2023-10-18T12:01:13.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d86d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d881dc910 0x7f1d881daa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d86d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d881dc910 0x7f1d881daa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35992/0 (socket says 172.21.15.125:35992) 2023-10-18T12:01:13.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d86d9d640 1 -- 172.21.15.125:0/4189794814 learned_addr learned my addr 172.21.15.125:0/4189794814 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:13.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d85d9b640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d881dc490 0x7f1d881da550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d8659c640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700039a0 0x7f1d881d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d86d9d640 1 -- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d881dc490 msgr2=0x7f1d881da550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d86d9d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d881dc490 0x7f1d881da550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d86d9d640 1 -- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700039a0 msgr2=0x7f1d881d9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d86d9d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700039a0 0x7f1d881d9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.173+0000 7f1d86d9d640 1 -- 172.21.15.125:0/4189794814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d881dd1a0 con 0x7f1d881dc910 2023-10-18T12:01:13.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.174+0000 7f1d86d9d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d881dc910 0x7f1d881daa90 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f1d8806d570 tx=0x7f1d8000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:13.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.174+0000 7f1d76ffd640 1 -- 172.21.15.125:0/4189794814 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000c9a0 con 0x7f1d881dc910 2023-10-18T12:01:13.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.174+0000 7f1d76ffd640 1 -- 172.21.15.125:0/4189794814 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d8000cc10 con 0x7f1d881dc910 2023-10-18T12:01:13.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.174+0000 7f1d76ffd640 1 -- 172.21.15.125:0/4189794814 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80011460 con 0x7f1d881dc910 2023-10-18T12:01:13.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.174+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d881dd480 con 0x7f1d881dc910 2023-10-18T12:01:13.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.174+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d881e1b90 con 0x7f1d881dc910 2023-10-18T12:01:13.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.175+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d881e1db0 con 0x7f1d881dc910 2023-10-18T12:01:13.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.175+0000 7f1d76ffd640 1 -- 172.21.15.125:0/4189794814 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1d800115e0 con 0x7f1d881dc910 2023-10-18T12:01:13.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.176+0000 7f1d76ffd640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1d6806b080 0x7f1d6806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.176+0000 7f1d76ffd640 1 -- 172.21.15.125:0/4189794814 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f1d8008a380 con 0x7f1d881dc910 2023-10-18T12:01:13.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.176+0000 7f1d8659c640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1d6806b080 0x7f1d6806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.177+0000 7f1d8659c640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1d6806b080 0x7f1d6806d540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c034000 tx=0x7f1d7c001190 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:13.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.180+0000 7f1d76ffd640 1 -- 172.21.15.125:0/4189794814 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d881e1db0 con 0x7f1d881dc910 2023-10-18T12:01:13.384 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.382+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d881ddc80 con 0x7f1d881dc910 2023-10-18T12:01:13.385 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.382+0000 7f1d76ffd640 1 -- 172.21.15.125:0/4189794814 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f1d881ddc80 con 0x7f1d881dc910 2023-10-18T12:01:13.386 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:13.386 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":99,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":12768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:08.447292+0000","modified":"2023-10-18T12:01:10.725386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12771":{"gid":12771,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1851256737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1851256737},{"type":"v1","addr":"172.21.15.125:6835","nonce":1851256737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-18T12:01:13.387 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 99 2023-10-18T12:01:13.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1d6806b080 msgr2=0x7f1d6806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1d6806b080 0x7f1d6806d540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c034000 tx=0x7f1d7c001190 comp rx=0 tx=0).stop 2023-10-18T12:01:13.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d881dc910 msgr2=0x7f1d881daa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d881dc910 0x7f1d881daa90 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f1d8806d570 tx=0x7f1d8000ec50 comp rx=0 tx=0).stop 2023-10-18T12:01:13.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 shutdown_connections 2023-10-18T12:01:13.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f1d6806b080 0x7f1d6806d540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d881dc910 0x7f1d881daa90 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1d881dc490 0x7f1d881da550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 --2- 172.21.15.125:0/4189794814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700039a0 0x7f1d881d9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 >> 172.21.15.125:0/4189794814 conn(0x7f1d88069930 msgr2=0x7f1d881d9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 shutdown_connections 2023-10-18T12:01:13.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.384+0000 7f1d8cb7d640 1 -- 172.21.15.125:0/4189794814 wait complete. 2023-10-18T12:01:13.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T12:01:08.447292+0000', 'modified': '2023-10-18T12:01:10.725386+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12771}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12771': {'gid': 12771, 'name': 'a', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/1851256737', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 1851256737}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 1851256737}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T12:01:13.801 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T12:01:13.801 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T12:01:13.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.902+0000 7fafcf67c640 1 -- 172.21.15.125:0/3225926515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc813e520 msgr2=0x7fafc806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.902+0000 7fafcf67c640 1 --2- 172.21.15.125:0/3225926515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc813e520 0x7fafc806bb60 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005fd0 tx=0x7fafc402c050 comp rx=0 tx=0).stop 2023-10-18T12:01:13.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.902+0000 7fafcf67c640 1 -- 172.21.15.125:0/3225926515 shutdown_connections 2023-10-18T12:01:13.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.902+0000 7fafcf67c640 1 --2- 172.21.15.125:0/3225926515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc813e520 0x7fafc806bb60 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.902+0000 7fafcf67c640 1 -- 172.21.15.125:0/3225926515 >> 172.21.15.125:0/3225926515 conn(0x7fafc8069930 msgr2=0x7fafc8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.903+0000 7fafcf67c640 1 -- 172.21.15.125:0/3225926515 shutdown_connections 2023-10-18T12:01:13.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.903+0000 7fafcf67c640 1 -- 172.21.15.125:0/3225926515 wait complete. 2023-10-18T12:01:13.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcf67c640 1 Processor -- start 2023-10-18T12:01:13.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcf67c640 1 -- start start 2023-10-18T12:01:13.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcf67c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc8070af0 0x7fafc81e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcf67c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafc813e520 0x7fafc80ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcf67c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafc8141690 0x7fafc80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcf67c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81e2f00 con 0x7fafc8070af0 2023-10-18T12:01:13.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcf67c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc813c890 con 0x7fafc813e520 2023-10-18T12:01:13.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcf67c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81e31d0 con 0x7fafc8141690 2023-10-18T12:01:13.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcd418640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc8070af0 0x7fafc81e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcd418640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc8070af0 0x7fafc81e3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36006/0 (socket says 172.21.15.125:36006) 2023-10-18T12:01:13.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.904+0000 7fafcd418640 1 -- 172.21.15.125:0/3027943054 learned_addr learned my addr 172.21.15.125:0/3027943054 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:13.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafcd418640 1 -- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafc8141690 msgr2=0x7fafc80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafcd418640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafc8141690 0x7fafc80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafcd418640 1 -- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafc813e520 msgr2=0x7fafc80ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafcd418640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafc813e520 0x7fafc80ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafcd418640 1 -- 172.21.15.125:0/3027943054 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc4009840 con 0x7fafc8070af0 2023-10-18T12:01:13.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafcd418640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc8070af0 0x7fafc81e3cb0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7fafc4036040 tx=0x7fafc4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:13.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafbdffb640 1 -- 172.21.15.125:0/3027943054 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc403e460 con 0x7fafc8070af0 2023-10-18T12:01:13.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafbdffb640 1 -- 172.21.15.125:0/3027943054 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fafc403e6b0 con 0x7fafc8070af0 2023-10-18T12:01:13.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafcf67c640 1 -- 172.21.15.125:0/3027943054 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc81e34b0 con 0x7fafc8070af0 2023-10-18T12:01:13.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafbdffb640 1 -- 172.21.15.125:0/3027943054 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc403eaa0 con 0x7fafc8070af0 2023-10-18T12:01:13.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.905+0000 7fafcf67c640 1 -- 172.21.15.125:0/3027943054 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc81e41f0 con 0x7fafc8070af0 2023-10-18T12:01:13.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.907+0000 7fafbdffb640 1 -- 172.21.15.125:0/3027943054 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fafc403ec00 con 0x7fafc8070af0 2023-10-18T12:01:13.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.907+0000 7fafbdffb640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fafac06b0b0 0x7fafac06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.908+0000 7fafbdffb640 1 -- 172.21.15.125:0/3027943054 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fafc40ace10 con 0x7fafc8070af0 2023-10-18T12:01:13.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.908+0000 7fafccc17640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fafac06b0b0 0x7fafac06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.908+0000 7fafccc17640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fafac06b0b0 0x7fafac06d570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fafc813dbb0 tx=0x7fafb8006ba0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:13.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.908+0000 7fafd2205740 1 Processor -- start 2023-10-18T12:01:13.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.908+0000 7fafd2205740 1 -- start start 2023-10-18T12:01:13.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafd2205740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017b40 0x5653c1feaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafd2205740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c201a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafd2205740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c1feb020 0x5653c201ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafd2205740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c1efc730 con 0x5653c2017f20 2023-10-18T12:01:13.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafd2205740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c1f5d780 con 0x5653c2017b40 2023-10-18T12:01:13.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafd2205740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c1f5ae70 con 0x5653c1feb020 2023-10-18T12:01:13.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafccc17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c201a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafccc17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c201a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36022/0 (socket says 172.21.15.125:36022) 2023-10-18T12:01:13.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafccc17640 1 -- 172.21.15.125:0/3908857610 learned_addr learned my addr 172.21.15.125:0/3908857610 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:13.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafcd418640 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017b40 0x5653c1feaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafcdc19640 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c1feb020 0x5653c201ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafccc17640 1 -- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c1feb020 msgr2=0x5653c201ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:13.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafccc17640 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c1feb020 0x5653c201ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafccc17640 1 -- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017b40 msgr2=0x5653c1feaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:13.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafccc17640 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017b40 0x5653c1feaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafccc17640 1 -- 172.21.15.125:0/3908857610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653c1f5b3d0 con 0x5653c2017f20 2023-10-18T12:01:13.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafcd418640 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017b40 0x5653c1feaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:13.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.909+0000 7fafcdc19640 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c1feb020 0x5653c201ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:13.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafccc17640 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c201a8d0 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fafb8013850 tx=0x7fafb800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6082a1bebe88e7ce server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:13.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafb2ffd640 1 -- 172.21.15.125:0/3908857610 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8012ca0 con 0x5653c2017f20 2023-10-18T12:01:13.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafb2ffd640 1 -- 172.21.15.125:0/3908857610 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fafb8019040 con 0x5653c2017f20 2023-10-18T12:01:13.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafb2ffd640 1 -- 172.21.15.125:0/3908857610 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb801d650 con 0x5653c2017f20 2023-10-18T12:01:13.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 -- 172.21.15.125:0/3908857610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 msgr2=0x5653c201a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c201a8d0 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fafb8013850 tx=0x7fafb800e000 comp rx=0 tx=0).stop 2023-10-18T12:01:13.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 -- 172.21.15.125:0/3908857610 shutdown_connections 2023-10-18T12:01:13.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c1feb020 0x5653c201ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c201a8d0 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 --2- 172.21.15.125:0/3908857610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017b40 0x5653c1feaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 -- 172.21.15.125:0/3908857610 >> 172.21.15.125:0/3908857610 conn(0x5653c1fecd50 msgr2=0x5653c20358d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 -- 172.21.15.125:0/3908857610 shutdown_connections 2023-10-18T12:01:13.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.910+0000 7fafd2205740 1 -- 172.21.15.125:0/3908857610 wait complete. 2023-10-18T12:01:13.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.911+0000 7fafd2205740 1 -- start start 2023-10-18T12:01:13.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.911+0000 7fafd2205740 1 Processor -- start 2023-10-18T12:01:13.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafd2205740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 0x5653c202ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafd2205740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c2007ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafd2205740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 0x5653c2008880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafd2205740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c1f56760 con 0x5653c2017f20 2023-10-18T12:01:13.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafd2205740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c1f118f0 con 0x5653c2008400 2023-10-18T12:01:13.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafd2205740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c1f11590 con 0x5653c2017b40 2023-10-18T12:01:13.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafd2205740 10 client.0 _ll_register_callbacks cb 0x5653c1f4dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:01:13.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c2007ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c2007ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36024/0 (socket says 172.21.15.125:36024) 2023-10-18T12:01:13.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 -- 172.21.15.125:0/3712725177 learned_addr learned my addr 172.21.15.125:0/3712725177 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:13.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafcd418640 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 0x5653c202ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafcdc19640 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 0x5653c2008880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 -- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 msgr2=0x5653c202ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:13.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 0x5653c202ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 -- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 msgr2=0x5653c2008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 0x5653c2008880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 -- 172.21.15.125:0/3712725177 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb80134d0 con 0x5653c2017f20 2023-10-18T12:01:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafcdc19640 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 0x5653c2008880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafccc17640 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c2007ec0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fafb8012760 tx=0x7fafb801d6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f74773c30afd580 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.912+0000 7fafa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:01:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafa7fff640 1 -- 172.21.15.125:0/3712725177 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb801dcd0 con 0x5653c2017f20 2023-10-18T12:01:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafa7fff640 1 -- 172.21.15.125:0/3712725177 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fafb801de30 con 0x5653c2017f20 2023-10-18T12:01:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 1 -- 172.21.15.125:0/3712725177 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653c1f4f740 con 0x5653c2017f20 2023-10-18T12:01:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafa7fff640 1 -- 172.21.15.125:0/3712725177 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafb8031410 con 0x5653c2017f20 2023-10-18T12:01:13.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 10 client.12921 fetch_fsmap learned FSMap version 99 2023-10-18T12:01:13.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 1 -- 172.21.15.125:0/3712725177 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5653c1f54de0 con 0x5653c2017f20 2023-10-18T12:01:13.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafa7fff640 1 -- 172.21.15.125:0/3712725177 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5653c1f4f740 con 0x5653c2017f20 2023-10-18T12:01:13.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafa7fff640 1 -- 172.21.15.125:0/3712725177 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fafb801c070 con 0x5653c2017f20 2023-10-18T12:01:13.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 10 client.12921 fetch_fsmap finished waiting for FSMap version 99 2023-10-18T12:01:13.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 10 client.12921 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T12:01:13.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 20 client.12921 populate_metadata read hostname 'smithi125' 2023-10-18T12:01:13.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] conn(0x5653c2006e30 0x5653c2007210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 4 client.12921 mds_command: new command op to 12771 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:01:13.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafd2205740 1 -- 172.21.15.125:0/3712725177 --> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653c1f4f460 con 0x5653c2006e30 2023-10-18T12:01:13.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafcd418640 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] conn(0x5653c2006e30 0x5653c2007210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafa7fff640 10 client.12921 ms_handle_connect on v2:172.21.15.125:6833/1851256737 2023-10-18T12:01:13.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.913+0000 7fafcd418640 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] conn(0x5653c2006e30 0x5653c2007210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d9b3730dc367ef server_cookie=df0f3c5f464fc766 in_seq=0 out_seq=0 2023-10-18T12:01:13.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafa7fff640 1 -- 172.21.15.125:0/3712725177 <== mds.0 v2:172.21.15.125:6833/1851256737 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5653c1f4f460 con 0x5653c2006e30 2023-10-18T12:01:13.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafa7fff640 10 client.12921 handle_command_reply: tid=0 2023-10-18T12:01:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 client.12921 shutdown 2023-10-18T12:01:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 -- 172.21.15.125:0/3712725177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 msgr2=0x5653c2007ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c2007ec0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fafb8012760 tx=0x7fafb801d6a0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 -- 172.21.15.125:0/3712725177 shutdown_connections 2023-10-18T12:01:13.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 0x5653c2008880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] conn(0x5653c2006e30 0x5653c2007210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017f20 0x5653c2007ec0 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 --2- 172.21.15.125:0/3712725177 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 0x5653c202ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 -- 172.21.15.125:0/3712725177 >> 172.21.15.125:0/3712725177 conn(0x5653c1feb020 msgr2=0x5653c201a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafa7fff640 0 client.12921 ms_handle_reset on v2:172.21.15.125:6833/1851256737 2023-10-18T12:01:13.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 -- 172.21.15.125:0/3712725177 shutdown_connections 2023-10-18T12:01:13.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 1 -- 172.21.15.125:0/3712725177 wait complete. 2023-10-18T12:01:13.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.914+0000 7fafd2205740 20 client.12921 trim_cache size 0 max 16384 2023-10-18T12:01:13.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.929+0000 7fafd2205740 1 Processor -- start 2023-10-18T12:01:13.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.929+0000 7fafd2205740 1 -- start start 2023-10-18T12:01:13.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafd2205740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 0x5653c2035d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafd2205740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017b40 0x5653c1fee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafd2205740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017f20 0x5653c1feec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafd2205740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c1ed7270 con 0x5653c2017b40 2023-10-18T12:01:13.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafd2205740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c1ff10a0 con 0x5653c2008400 2023-10-18T12:01:13.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafd2205740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c1fec520 con 0x5653c2017f20 2023-10-18T12:01:13.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafccc17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017b40 0x5653c1fee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafccc17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017b40 0x5653c1fee6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36030/0 (socket says 172.21.15.125:36030) 2023-10-18T12:01:13.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.930+0000 7fafccc17640 1 -- 172.21.15.125:0/546785705 learned_addr learned my addr 172.21.15.125:0/546785705 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:13.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafccc17640 1 -- 172.21.15.125:0/546785705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017f20 msgr2=0x5653c1feec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:13.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafccc17640 1 --2- 172.21.15.125:0/546785705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017f20 0x5653c1feec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafccc17640 1 -- 172.21.15.125:0/546785705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 msgr2=0x5653c2035d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:13.946 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafccc17640 1 --2- 172.21.15.125:0/546785705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 0x5653c2035d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafccc17640 1 -- 172.21.15.125:0/546785705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653c1ef3d00 con 0x5653c2017b40 2023-10-18T12:01:13.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafccc17640 1 --2- 172.21.15.125:0/546785705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017b40 0x5653c1fee6e0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006a20 tx=0x7fafb800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e5f8818ca6e2bb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:13.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafb37fe640 1 -- 172.21.15.125:0/546785705 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb801d710 con 0x5653c2017b40 2023-10-18T12:01:13.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafb37fe640 1 -- 172.21.15.125:0/546785705 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fafb8013e90 con 0x5653c2017b40 2023-10-18T12:01:13.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafb37fe640 1 -- 172.21.15.125:0/546785705 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb801dd30 con 0x5653c2017b40 2023-10-18T12:01:13.947 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 -- 172.21.15.125:0/546785705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017b40 msgr2=0x5653c1fee6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 --2- 172.21.15.125:0/546785705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017b40 0x5653c1fee6e0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006a20 tx=0x7fafb800e000 comp rx=0 tx=0).stop 2023-10-18T12:01:13.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 -- 172.21.15.125:0/546785705 shutdown_connections 2023-10-18T12:01:13.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 --2- 172.21.15.125:0/546785705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017f20 0x5653c1feec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 --2- 172.21.15.125:0/546785705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2017b40 0x5653c1fee6e0 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.948 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 --2- 172.21.15.125:0/546785705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2008400 0x5653c2035d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 -- 172.21.15.125:0/546785705 >> 172.21.15.125:0/546785705 conn(0x5653c1feb020 msgr2=0x5653c202ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 -- 172.21.15.125:0/546785705 shutdown_connections 2023-10-18T12:01:13.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.931+0000 7fafd2205740 1 -- 172.21.15.125:0/546785705 wait complete. 2023-10-18T12:01:13.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.932+0000 7fafd2205740 1 -- start start 2023-10-18T12:01:13.949 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.932+0000 7fafd2205740 1 Processor -- start 2023-10-18T12:01:13.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafd2205740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2008400 0x5653c201ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafd2205740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 0x5653c1fef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafd2205740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017f20 0x5653c1fefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafd2205740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c1feefe0 con 0x5653c2008400 2023-10-18T12:01:13.950 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafd2205740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c1ff0440 con 0x5653c2017f20 2023-10-18T12:01:13.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafd2205740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c2004e00 con 0x5653c2017b40 2023-10-18T12:01:13.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafd2205740 10 client.0 _ll_register_callbacks cb 0x7fafb80092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:01:13.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2008400 0x5653c201ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2008400 0x5653c201ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36040/0 (socket says 172.21.15.125:36040) 2023-10-18T12:01:13.951 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 -- 172.21.15.125:0/1141059914 learned_addr learned my addr 172.21.15.125:0/1141059914 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:13.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafccc17640 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 0x5653c1fef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcdc19640 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017f20 0x5653c1fefbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 -- 172.21.15.125:0/1141059914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 msgr2=0x5653c1fef6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:13.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 0x5653c1fef6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 -- 172.21.15.125:0/1141059914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017f20 msgr2=0x5653c1fefbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:13.952 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017f20 0x5653c1fefbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 -- 172.21.15.125:0/1141059914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653c1f54de0 con 0x5653c2008400 2023-10-18T12:01:13.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafcd418640 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2008400 0x5653c201ea30 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fafc40aae60 tx=0x7fafc4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bc489df99838b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:13.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:01:13.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafa7fff640 1 -- 172.21.15.125:0/1141059914 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4083710 con 0x5653c2008400 2023-10-18T12:01:13.953 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.933+0000 7fafa7fff640 1 -- 172.21.15.125:0/1141059914 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fafc4079b00 con 0x5653c2008400 2023-10-18T12:01:13.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 1 -- 172.21.15.125:0/1141059914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafb8019040 con 0x5653c2008400 2023-10-18T12:01:13.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafa7fff640 1 -- 172.21.15.125:0/1141059914 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc40ab090 con 0x5653c2008400 2023-10-18T12:01:13.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 10 client.12927 fetch_fsmap learned FSMap version 99 2023-10-18T12:01:13.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 1 -- 172.21.15.125:0/1141059914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5653c1f194f0 con 0x5653c2008400 2023-10-18T12:01:13.954 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafa7fff640 1 -- 172.21.15.125:0/1141059914 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fafb8019040 con 0x5653c2008400 2023-10-18T12:01:13.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafa7fff640 1 -- 172.21.15.125:0/1141059914 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fafc404e020 con 0x5653c2008400 2023-10-18T12:01:13.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 10 client.12927 fetch_fsmap finished waiting for FSMap version 99 2023-10-18T12:01:13.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 10 client.12927 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T12:01:13.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 20 client.12927 populate_metadata read hostname 'smithi125' 2023-10-18T12:01:13.955 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] conn(0x5653c20478e0 0x5653c2006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 4 client.12927 mds_command: new command op to 12771 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T12:01:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 1 -- 172.21.15.125:0/1141059914 --> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5653c1c586c0 con 0x5653c20478e0 2023-10-18T12:01:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafccc17640 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] conn(0x5653c20478e0 0x5653c2006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafa7fff640 10 client.12927 ms_handle_connect on v2:172.21.15.125:6833/1851256737 2023-10-18T12:01:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafccc17640 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] conn(0x5653c20478e0 0x5653c2006e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6e588ee3387fdb server_cookie=54e0b97987e02748 in_seq=0 out_seq=0 2023-10-18T12:01:13.956 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafa7fff640 1 -- 172.21.15.125:0/1141059914 <== mds.0 v2:172.21.15.125:6833/1851256737 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5653c1c586c0 con 0x5653c20478e0 2023-10-18T12:01:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafa7fff640 10 client.12927 handle_command_reply: tid=0 2023-10-18T12:01:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.934+0000 7fafd2205740 1 client.12927 shutdown 2023-10-18T12:01:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 -- 172.21.15.125:0/1141059914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2008400 msgr2=0x5653c201ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2008400 0x5653c201ea30 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fafc40aae60 tx=0x7fafc4000f40 comp rx=0 tx=0).stop 2023-10-18T12:01:13.957 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 -- 172.21.15.125:0/1141059914 shutdown_connections 2023-10-18T12:01:13.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5653c2017f20 0x5653c1fefbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.125:6833/1851256737,v1:172.21.15.125:6835/1851256737] conn(0x5653c20478e0 0x5653c2006e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x5653c2017b40 0x5653c1fef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 --2- 172.21.15.125:0/1141059914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5653c2008400 0x5653c201ea30 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.958 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafa7fff640 0 client.12927 ms_handle_reset on v2:172.21.15.125:6833/1851256737 2023-10-18T12:01:13.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 -- 172.21.15.125:0/1141059914 >> 172.21.15.125:0/1141059914 conn(0x5653c1feb020 msgr2=0x5653c202eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 -- 172.21.15.125:0/1141059914 shutdown_connections 2023-10-18T12:01:13.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 1 -- 172.21.15.125:0/1141059914 wait complete. 2023-10-18T12:01:13.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafd2205740 20 client.12927 trim_cache size 0 max 16384 2023-10-18T12:01:13.959 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:01:13.960 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T12:01:13.960 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T12:01:13.960 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 12771, 2023-10-18T12:01:13.960 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T12:01:13.960 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T12:01:13.961 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T12:01:13.961 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 4.8113716770000003, 2023-10-18T12:01:13.961 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 99, 2023-10-18T12:01:13.961 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 242, 2023-10-18T12:01:13.961 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 242, 2023-10-18T12:01:13.962 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 9.5690109480000007 2023-10-18T12:01:13.962 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:01:13.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 -- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fafac06b0b0 msgr2=0x7fafac06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fafac06b0b0 0x7fafac06d570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fafc813dbb0 tx=0x7fafb8006ba0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 -- 172.21.15.125:0/3027943054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc8070af0 msgr2=0x7fafc81e3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:13.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc8070af0 0x7fafc81e3cb0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7fafc4036040 tx=0x7fafc4004360 comp rx=0 tx=0).stop 2023-10-18T12:01:13.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 -- 172.21.15.125:0/3027943054 shutdown_connections 2023-10-18T12:01:13.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fafac06b0b0 0x7fafac06d570 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafc813e520 0x7fafc80ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafc8141690 0x7fafc80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 --2- 172.21.15.125:0/3027943054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fafc8070af0 0x7fafc81e3cb0 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:13.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 -- 172.21.15.125:0/3027943054 >> 172.21.15.125:0/3027943054 conn(0x7fafc8069930 msgr2=0x7fafc81403a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:13.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 -- 172.21.15.125:0/3027943054 shutdown_connections 2023-10-18T12:01:13.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:13.935+0000 7fafcf67c640 1 -- 172.21.15.125:0/3027943054 wait complete. 2023-10-18T12:01:13.966 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T12:01:14.068 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.065+0000 7f76fc797640 1 -- 172.21.15.125:0/4115712692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e0003970 msgr2=0x7f76e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:14.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.065+0000 7f76fc797640 1 --2- 172.21.15.125:0/4115712692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e0003970 0x7f76e0003d90 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f76f0001d80 tx=0x7f76e00057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.066+0000 7f76fc797640 1 -- 172.21.15.125:0/4115712692 shutdown_connections 2023-10-18T12:01:14.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.066+0000 7f76fc797640 1 --2- 172.21.15.125:0/4115712692 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76e0003970 0x7f76e0003d90 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.066+0000 7f76fc797640 1 -- 172.21.15.125:0/4115712692 >> 172.21.15.125:0/4115712692 conn(0x7f76f40d0d60 msgr2=0x7f76f40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:14.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.066+0000 7f76fc797640 1 -- 172.21.15.125:0/4115712692 shutdown_connections 2023-10-18T12:01:14.070 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.066+0000 7f76fc797640 1 -- 172.21.15.125:0/4115712692 wait complete. 2023-10-18T12:01:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.067+0000 7f76fc797640 1 Processor -- start 2023-10-18T12:01:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.067+0000 7f76fc797640 1 -- start start 2023-10-18T12:01:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.067+0000 7f76fc797640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e0003970 0x7f76f40c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.067+0000 7f76fc797640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f40caa90 0x7f76f40b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:14.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.067+0000 7f76fc797640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f41510d0 0x7f76f406a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fc797640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40c99f0 con 0x7f76f41510d0 2023-10-18T12:01:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fc797640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40c9cd0 con 0x7f76f40caa90 2023-10-18T12:01:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fc797640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40c9fb0 con 0x7f76e0003970 2023-10-18T12:01:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f41510d0 0x7f76f406a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:14.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f41510d0 0x7f76f406a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36056/0 (socket says 172.21.15.125:36056) 2023-10-18T12:01:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 -- 172.21.15.125:0/82898211 learned_addr learned my addr 172.21.15.125:0/82898211 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76f9d32640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f40caa90 0x7f76f40b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fa533640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e0003970 0x7f76f40c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 -- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e0003970 msgr2=0x7f76f40c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:14.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e0003970 0x7f76f40c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 -- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f40caa90 msgr2=0x7f76f40b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f40caa90 0x7f76f40b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 -- 172.21.15.125:0/82898211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f40ca290 con 0x7f76f41510d0 2023-10-18T12:01:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fad34640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f41510d0 0x7f76f406a540 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f76f40b0080 tx=0x7f76ec011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:14.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76eaffd640 1 -- 172.21.15.125:0/82898211 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004520 con 0x7f76f41510d0 2023-10-18T12:01:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76eaffd640 1 -- 172.21.15.125:0/82898211 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76ec005110 con 0x7f76f41510d0 2023-10-18T12:01:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f40ca570 con 0x7f76f41510d0 2023-10-18T12:01:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76eaffd640 1 -- 172.21.15.125:0/82898211 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec005500 con 0x7f76f41510d0 2023-10-18T12:01:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.068+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f4151550 con 0x7f76f41510d0 2023-10-18T12:01:14.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.069+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f40690e0 con 0x7f76f41510d0 2023-10-18T12:01:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.070+0000 7f76eaffd640 1 -- 172.21.15.125:0/82898211 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f76ec0047a0 con 0x7f76f41510d0 2023-10-18T12:01:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.070+0000 7f76eaffd640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f76d806b080 0x7f76d806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.070+0000 7f76eaffd640 1 -- 172.21.15.125:0/82898211 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f76ec08a6f0 con 0x7f76f41510d0 2023-10-18T12:01:14.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.070+0000 7f76fa533640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f76d806b080 0x7f76d806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:14.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.071+0000 7f76fa533640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f76d806b080 0x7f76d806d540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76f0034000 tx=0x7f76f002e040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:14.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.074+0000 7f76eaffd640 1 -- 172.21.15.125:0/82898211 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76ec055230 con 0x7f76f41510d0 2023-10-18T12:01:14.279 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.276+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f76f40d0b40 con 0x7f76f41510d0 2023-10-18T12:01:14.280 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.277+0000 7f76eaffd640 1 -- 172.21.15.125:0/82898211 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f76ec058850 con 0x7f76f41510d0 2023-10-18T12:01:14.280 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T12:01:14.281 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f76d806b080 msgr2=0x7f76d806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:14.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f76d806b080 0x7f76d806d540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76f0034000 tx=0x7f76f002e040 comp rx=0 tx=0).stop 2023-10-18T12:01:14.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f41510d0 msgr2=0x7f76f406a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:14.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f41510d0 0x7f76f406a540 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f76f40b0080 tx=0x7f76ec011af0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.282 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 shutdown_connections 2023-10-18T12:01:14.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f76d806b080 0x7f76d806d540 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76f40caa90 0x7f76f40b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f41510d0 0x7f76f406a540 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 --2- 172.21.15.125:0/82898211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f76e0003970 0x7f76f40c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:14.283 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 >> 172.21.15.125:0/82898211 conn(0x7f76f40d0d60 msgr2=0x7f76f40ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:14.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 shutdown_connections 2023-10-18T12:01:14.284 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:14.279+0000 7f76fc797640 1 -- 172.21.15.125:0/82898211 wait complete. 2023-10-18T12:01:14.292 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T12:01:14.292 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T12:01:14.293 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T12:01:14.293 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T12:01:14.293 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T12:01:14.294 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T12:01:14.294 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T12:01:14.294 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T12:01:14.295 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:01:14.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:14.295 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T12:01:14.311 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T12:01:14.311 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T12:01:14.311 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T12:01:14.312 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:01:14.312 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T12:01:14.312 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:01:14.312 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:01:14.312 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:01:14.313 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T12:01:14.313 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:01:14.313 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:01:14.313 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T12:01:14.313 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:01:14.313 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:01:14.314 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T12:01:14.314 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:01:14.314 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T12:01:14.314 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 36950sec preferred_lft 36950sec 2023-10-18T12:01:14.314 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T12:01:14.315 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:01:14.315 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T12:01:14.315 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:01:14.315 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T12:01:14.315 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:01:14.315 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T12:01:14.316 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:01:14.316 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T12:01:14.316 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:01:14.316 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T12:01:14.316 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:01:14.316 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T12:01:14.317 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:01:14.317 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T12:01:14.317 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:01:14.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:14.318 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T12:01:14.377 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T12:01:14.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:14.379 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T12:01:14.434 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T12:01:14.434 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T12:01:14.434 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T12:01:14.435 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T12:01:14.435 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T12:01:14.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:14.436 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T12:01:14.436 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T12:01:14.437 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T12:01:14.437 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T12:01:14.438 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T12:01:14.438 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T12:01:14.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:14.530 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T12:01:14.546 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T12:01:14.546 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T12:01:14.546 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:01:14.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:14.548 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T12:01:14.613 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T12:01:14.614 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:01:14.614 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T12:01:14.668 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:01:14.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:01:14.669 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T12:01:14.723 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T12:01:14.789 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:01:14.845 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T12:01:14.845 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T12:01:14.845 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T12:01:14.845 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T12:01:14.846 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T12:01:14.846 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T12:01:14.846 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T12:01:14.846 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T12:01:14.847 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T12:01:14.847 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T12:01:14.847 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T12:01:14.847 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T12:01:14.847 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:01:14.847 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T12:01:14.848 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T12:01:14.848 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T12:01:14.848 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T12:01:14.848 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T12:01:14.848 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T12:01:14.849 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T12:01:14.849 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T12:01:14.849 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T12:01:14.849 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:01:14.849 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T12:01:14.849 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T12:01:14.850 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T12:01:14.850 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T12:01:14.850 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T12:01:14.850 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T12:01:14.850 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T12:01:14.851 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:01:14.851 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:01:14.851 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:01:14.851 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:01:14.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:14.852 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T12:01:14.900 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T12:01:14.900 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T12:01:14.943 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T12:01:14.970 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:01:15.026 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T12:01:15.026 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T12:01:15.026 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T12:01:15.027 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T12:01:15.027 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T12:01:15.027 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T12:01:15.027 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T12:01:15.027 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T12:01:15.028 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T12:01:15.028 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T12:01:15.028 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T12:01:15.028 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T12:01:15.028 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:01:15.029 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T12:01:15.029 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T12:01:15.029 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T12:01:15.029 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T12:01:15.029 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T12:01:15.030 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T12:01:15.030 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T12:01:15.030 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T12:01:15.030 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T12:01:15.030 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:01:15.031 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T12:01:15.031 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T12:01:15.031 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T12:01:15.031 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T12:01:15.031 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T12:01:15.031 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T12:01:15.032 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T12:01:15.032 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:01:15.032 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:01:15.032 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:01:15.032 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:01:15.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:15.033 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T12:01:15.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T12:01:15.025+0000 7f5b10bd7480 -1 init, newargv = 0x55a02149d770 newargc=15 2023-10-18T12:01:15.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[160129]: starting ceph client 2023-10-18T12:01:15.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[160129]: starting fuse 2023-10-18T12:01:15.048 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T12:01:15.049 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T12:01:15.049 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T12:01:15.049 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T12:01:15.050 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T12:01:15.050 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T12:01:15.051 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T12:01:15.051 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:01:15.051 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T12:01:15.052 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T12:01:15.052 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T12:01:15.052 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T12:01:15.053 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:01:15.053 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T12:01:15.054 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T12:01:15.054 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T12:01:15.054 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T12:01:15.055 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:01:15.055 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T12:01:15.056 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T12:01:15.056 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T12:01:15.056 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T12:01:15.057 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T12:01:15.057 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T12:01:15.057 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T12:01:15.058 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T12:01:15.058 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:01:15.059 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T12:01:15.059 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T12:01:15.167 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.160129.asok 2023-10-18T12:01:15.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T12:01:15.177 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.160129.asok 2023-10-18T12:01:15.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:15.178 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.160129.asok status 2023-10-18T12:01:15.313 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:01:15.313 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T12:01:15.314 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T12:01:15.314 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T12:01:15.314 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T12:01:15.314 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T12:01:15.315 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T12:01:15.315 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "160129", 2023-10-18T12:01:15.315 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T12:01:15.315 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:15.315 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T12:01:15.316 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T12:01:15.316 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 12939, 2023-10-18T12:01:15.316 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T12:01:15.316 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T12:01:15.316 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T12:01:15.317 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 12939 2023-10-18T12:01:15.317 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:15.317 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T12:01:15.317 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T12:01:15.317 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T12:01:15.317 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 1054038500 2023-10-18T12:01:15.318 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:15.318 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:15.318 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T12:01:15.318 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T12:01:15.318 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T12:01:15.319 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 1054038500 2023-10-18T12:01:15.319 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:15.319 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.12939 192.168.0.1:0/1054038500", 2023-10-18T12:01:15.319 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/1054038500", 2023-10-18T12:01:15.319 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T12:01:15.320 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 99, 2023-10-18T12:01:15.320 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 242, 2023-10-18T12:01:15.320 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T12:01:15.320 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T12:01:15.320 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T12:01:15.321 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:01:15.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:15.322 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T12:01:15.323 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T12:01:15.337 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T12:01:15.338 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T12:01:15.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:15.339 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T12:01:15.406 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-18T12:01:15.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.550+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/4214384167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0070af0 msgr2=0x7fb9a00affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:15.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.550+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/4214384167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0070af0 0x7fb9a00affe0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00cbd00 tx=0x7fb99802c1d0 comp rx=0 tx=0).stop 2023-10-18T12:01:15.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.550+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/4214384167 shutdown_connections 2023-10-18T12:01:15.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.550+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/4214384167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0070af0 0x7fb9a00affe0 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:15.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.550+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/4214384167 >> v2:172.21.15.139:3300/0 conn(0x7fb9a013e520 0x7fb9a006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:15.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.550+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/4214384167 >> 172.21.15.125:0/4214384167 conn(0x7fb9a0069930 msgr2=0x7fb9a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:15.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.550+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/4214384167 shutdown_connections 2023-10-18T12:01:15.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.550+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/4214384167 wait complete. 2023-10-18T12:01:15.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.551+0000 7fb9a7cbb640 1 Processor -- start 2023-10-18T12:01:15.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.551+0000 7fb9a7cbb640 1 -- start start 2023-10-18T12:01:15.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a7cbb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9a0070af0 0x7fb9a00c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:15.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a7cbb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9a013e520 0x7fb9a00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:15.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a7cbb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0141690 0x7fb9a013b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:15.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a7cbb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00c77a0 con 0x7fb9a0141690 2023-10-18T12:01:15.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a7cbb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00c7a80 con 0x7fb9a0070af0 2023-10-18T12:01:15.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a7cbb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00c7d60 con 0x7fb9a013e520 2023-10-18T12:01:15.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0141690 0x7fb9a013b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:15.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0141690 0x7fb9a013b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36076/0 (socket says 172.21.15.125:36076) 2023-10-18T12:01:15.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 -- 172.21.15.125:0/3109286398 learned_addr learned my addr 172.21.15.125:0/3109286398 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:15.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a5a57640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9a0070af0 0x7fb9a00c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:15.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a5256640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9a013e520 0x7fb9a00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:15.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 -- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9a013e520 msgr2=0x7fb9a00c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:15.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9a013e520 0x7fb9a00c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:15.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 -- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9a0070af0 msgr2=0x7fb9a00c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:15.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9a0070af0 0x7fb9a00c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:15.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 -- 172.21.15.125:0/3109286398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb998009900 con 0x7fb9a0141690 2023-10-18T12:01:15.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a5a57640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9a0070af0 0x7fb9a00c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:15.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a5256640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9a013e520 0x7fb9a00c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:15.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a6258640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0141690 0x7fb9a013b110 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7fb998037040 tx=0x7fb998035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:15.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9967fc640 1 -- 172.21.15.125:0/3109286398 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998034440 con 0x7fb9a0141690 2023-10-18T12:01:15.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9967fc640 1 -- 172.21.15.125:0/3109286398 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb998004450 con 0x7fb9a0141690 2023-10-18T12:01:15.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9967fc640 1 -- 172.21.15.125:0/3109286398 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998004840 con 0x7fb9a0141690 2023-10-18T12:01:15.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a00c8040 con 0x7fb9a0141690 2023-10-18T12:01:15.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.552+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a00c6d60 con 0x7fb9a0141690 2023-10-18T12:01:15.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.553+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a00c8560 con 0x7fb9a0141690 2023-10-18T12:01:15.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.554+0000 7fb9967fc640 1 -- 172.21.15.125:0/3109286398 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb99803b040 con 0x7fb9a0141690 2023-10-18T12:01:15.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.554+0000 7fb9967fc640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb98406b080 0x7fb98406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:15.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.554+0000 7fb9967fc640 1 -- 172.21.15.125:0/3109286398 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fb9980aca80 con 0x7fb9a0141690 2023-10-18T12:01:15.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.554+0000 7fb9a5a57640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb98406b080 0x7fb98406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:15.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.555+0000 7fb9a5a57640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb98406b080 0x7fb98406d540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb99c006fd0 tx=0x7fb99c008040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:15.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.558+0000 7fb9967fc640 1 -- 172.21.15.125:0/3109286398 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9a00c8560 con 0x7fb9a0141690 2023-10-18T12:01:15.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:15.759+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb9a00c8560 con 0x7fb9a0141690 2023-10-18T12:01:16.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.420+0000 7fb9967fc640 1 -- 172.21.15.125:0/3109286398 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v100) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb9a00c8560 con 0x7fb9a0141690 2023-10-18T12:01:16.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb98406b080 msgr2=0x7fb98406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb98406b080 0x7fb98406d540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb99c006fd0 tx=0x7fb99c008040 comp rx=0 tx=0).stop 2023-10-18T12:01:16.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0141690 msgr2=0x7fb9a013b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0141690 0x7fb9a013b110 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7fb998037040 tx=0x7fb998035db0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 shutdown_connections 2023-10-18T12:01:16.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb98406b080 0x7fb98406d540 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9a0141690 0x7fb9a013b110 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9a013e520 0x7fb9a00c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 --2- 172.21.15.125:0/3109286398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9a0070af0 0x7fb9a00c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 >> 172.21.15.125:0/3109286398 conn(0x7fb9a0069930 msgr2=0x7fb9a00af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:16.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 shutdown_connections 2023-10-18T12:01:16.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.422+0000 7fb9a7cbb640 1 -- 172.21.15.125:0/3109286398 wait complete. 2023-10-18T12:01:16.437 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:16.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.540+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/2042082258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9c004d00 msgr2=0x7f9c9c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.540+0000 7f9cb8ba0640 1 --2- 172.21.15.125:0/2042082258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9c9c004d00 0x7f9c9c005180 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c640 tx=0x7f9c9c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.541+0000 7f9cb3fff640 1 -- 172.21.15.125:0/2042082258 reap_dead start 2023-10-18T12:01:16.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.541+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/2042082258 shutdown_connections 2023-10-18T12:01:16.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.541+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/2042082258 >> 172.21.15.125:0/2042082258 conn(0x7f9cb40ae5d0 msgr2=0x7f9cb40ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:16.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.541+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/2042082258 shutdown_connections 2023-10-18T12:01:16.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.541+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/2042082258 wait complete. 2023-10-18T12:01:16.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.542+0000 7f9cb8ba0640 1 Processor -- start 2023-10-18T12:01:16.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.542+0000 7f9cb8ba0640 1 -- start start 2023-10-18T12:01:16.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb8ba0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb41e23e0 0x7f9cb4156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:16.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb8ba0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cb4156ab0 0x7f9cb4142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:16.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb8ba0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cb41e27c0 0x7f9cb4143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:16.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb8ba0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb41516d0 con 0x7f9cb41e23e0 2023-10-18T12:01:16.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb8ba0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb41e0c30 con 0x7f9cb41e27c0 2023-10-18T12:01:16.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb8ba0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb41e0f10 con 0x7f9cb4156ab0 2023-10-18T12:01:16.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb41e23e0 0x7f9cb4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:16.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb41e23e0 0x7f9cb4156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36100/0 (socket says 172.21.15.125:36100) 2023-10-18T12:01:16.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 -- 172.21.15.125:0/1983245311 learned_addr learned my addr 172.21.15.125:0/1983245311 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:16.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb2ffd640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cb4156ab0 0x7f9cb4142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:16.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb3fff640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cb41e27c0 0x7f9cb4143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:16.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 -- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cb4156ab0 msgr2=0x7f9cb4142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cb4156ab0 0x7f9cb4142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 -- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cb41e27c0 msgr2=0x7f9cb4143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cb41e27c0 0x7f9cb4143210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 -- 172.21.15.125:0/1983245311 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb41e11f0 con 0x7f9cb41e23e0 2023-10-18T12:01:16.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb3fff640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cb41e27c0 0x7f9cb4143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:16.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb37fe640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb41e23e0 0x7f9cb4156570 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4003e90 tx=0x7f9ca4001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:16.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9c9bfff640 1 -- 172.21.15.125:0/1983245311 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40129e0 con 0x7f9cb41e23e0 2023-10-18T12:01:16.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9c9bfff640 1 -- 172.21.15.125:0/1983245311 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ca4012b40 con 0x7f9cb41e23e0 2023-10-18T12:01:16.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb41e14d0 con 0x7f9cb41e23e0 2023-10-18T12:01:16.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9c9bfff640 1 -- 172.21.15.125:0/1983245311 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4001d40 con 0x7f9cb41e23e0 2023-10-18T12:01:16.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.543+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb41f1670 con 0x7f9cb41e23e0 2023-10-18T12:01:16.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.544+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb40ca470 con 0x7f9cb41e23e0 2023-10-18T12:01:16.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.545+0000 7f9c9bfff640 1 -- 172.21.15.125:0/1983245311 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9ca4018020 con 0x7f9cb41e23e0 2023-10-18T12:01:16.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.545+0000 7f9c9bfff640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f9ca006b080 0x7f9ca006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:16.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.545+0000 7f9c9bfff640 1 -- 172.21.15.125:0/1983245311 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f9ca408c5f0 con 0x7f9cb41e23e0 2023-10-18T12:01:16.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.545+0000 7f9cb2ffd640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f9ca006b080 0x7f9ca006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:16.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.546+0000 7f9cb2ffd640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f9ca006b080 0x7f9ca006d540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9cb41f15d0 tx=0x7f9ca8005f90 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:16.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.549+0000 7f9c9bfff640 1 -- 172.21.15.125:0/1983245311 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ca4057250 con 0x7f9cb41e23e0 2023-10-18T12:01:16.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.716+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cb40cc100 con 0x7f9cb41e23e0 2023-10-18T12:01:16.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.717+0000 7f9c9bfff640 1 -- 172.21.15.125:0/1983245311 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+16922 (secure 0 0 0) 0x7f9ca4020090 con 0x7f9cb41e23e0 2023-10-18T12:01:16.720 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:16.720 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":242,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:10.109552+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":240,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:16.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f9ca006b080 msgr2=0x7f9ca006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f9ca006b080 0x7f9ca006d540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9cb41f15d0 tx=0x7f9ca8005f90 comp rx=0 tx=0).stop 2023-10-18T12:01:16.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb41e23e0 msgr2=0x7f9cb4156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb41e23e0 0x7f9cb4156570 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4003e90 tx=0x7f9ca4001630 comp rx=0 tx=0).stop 2023-10-18T12:01:16.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 shutdown_connections 2023-10-18T12:01:16.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f9ca006b080 0x7f9ca006d540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cb41e27c0 0x7f9cb4143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cb4156ab0 0x7f9cb4142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 --2- 172.21.15.125:0/1983245311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb41e23e0 0x7f9cb4156570 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 >> 172.21.15.125:0/1983245311 conn(0x7f9cb40ae5d0 msgr2=0x7f9cb4153540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:16.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 shutdown_connections 2023-10-18T12:01:16.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.719+0000 7f9cb8ba0640 1 -- 172.21.15.125:0/1983245311 wait complete. 2023-10-18T12:01:16.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:01:06.989961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '241', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:01:07.416244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '242', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:01:16.733 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:01:16.837 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.834+0000 7fd9b2090640 1 -- 172.21.15.125:0/3357798706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0b0530 msgr2=0x7fd9ac0b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.834+0000 7fd9b2090640 1 --2- 172.21.15.125:0/3357798706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0b0530 0x7fd9ac0b09c0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4005fd0 tx=0x7fd9a402c240 comp rx=0 tx=0).stop 2023-10-18T12:01:16.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.834+0000 7fd9b2090640 1 -- 172.21.15.125:0/3357798706 shutdown_connections 2023-10-18T12:01:16.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.834+0000 7fd9b2090640 1 --2- 172.21.15.125:0/3357798706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0b0530 0x7fd9ac0b09c0 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.834+0000 7fd9b2090640 1 --2- 172.21.15.125:0/3357798706 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9ac0afa20 0x7fd9ac158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.834+0000 7fd9b2090640 1 -- 172.21.15.125:0/3357798706 >> 172.21.15.125:0/3357798706 conn(0x7fd9ac069930 msgr2=0x7fd9ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:16.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.834+0000 7fd9b2090640 1 -- 172.21.15.125:0/3357798706 shutdown_connections 2023-10-18T12:01:16.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.835+0000 7fd9b2090640 1 -- 172.21.15.125:0/3357798706 wait complete. 2023-10-18T12:01:16.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.836+0000 7fd9b2090640 1 Processor -- start 2023-10-18T12:01:16.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.836+0000 7fd9b2090640 1 -- start start 2023-10-18T12:01:16.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.836+0000 7fd9b2090640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0afa20 0x7fd9ac14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:16.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.836+0000 7fd9b2090640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9ac0b0530 0x7fd9ac14f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:16.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.836+0000 7fd9b2090640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9ac0cbd20 0x7fd9ac14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:16.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.836+0000 7fd9b2090640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac150a20 con 0x7fd9ac0afa20 2023-10-18T12:01:16.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.836+0000 7fd9b2090640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac144710 con 0x7fd9ac0b0530 2023-10-18T12:01:16.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.836+0000 7fd9b2090640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac150460 con 0x7fd9ac0cbd20 2023-10-18T12:01:16.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0afa20 0x7fd9ac14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:16.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0afa20 0x7fd9ac14cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36110/0 (socket says 172.21.15.125:36110) 2023-10-18T12:01:16.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 -- 172.21.15.125:0/156575616 learned_addr learned my addr 172.21.15.125:0/156575616 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:16.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9aaffd640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9ac0b0530 0x7fd9ac14f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:16.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9abfff640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9ac0cbd20 0x7fd9ac14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:16.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 -- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9ac0cbd20 msgr2=0x7fd9ac14fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9ac0cbd20 0x7fd9ac14fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 -- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9ac0b0530 msgr2=0x7fd9ac14f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:16.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9ac0b0530 0x7fd9ac14f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:16.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 -- 172.21.15.125:0/156575616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a4009900 con 0x7fd9ac0afa20 2023-10-18T12:01:16.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9ab7fe640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0afa20 0x7fd9ac14cc50 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00b3f0 tx=0x7fd99c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:16.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd993fff640 1 -- 172.21.15.125:0/156575616 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011830 con 0x7fd9ac0afa20 2023-10-18T12:01:16.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd993fff640 1 -- 172.21.15.125:0/156575616 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd99c011a80 con 0x7fd9ac0afa20 2023-10-18T12:01:16.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ac150740 con 0x7fd9ac0afa20 2023-10-18T12:01:16.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd993fff640 1 -- 172.21.15.125:0/156575616 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011e90 con 0x7fd9ac0afa20 2023-10-18T12:01:16.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.837+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ac1497e0 con 0x7fd9ac0afa20 2023-10-18T12:01:16.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.838+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ac149360 con 0x7fd9ac0afa20 2023-10-18T12:01:16.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.839+0000 7fd993fff640 1 -- 172.21.15.125:0/156575616 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd99c01a400 con 0x7fd9ac0afa20 2023-10-18T12:01:16.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.839+0000 7fd993fff640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd99806b0b0 0x7fd99806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:16.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.839+0000 7fd993fff640 1 -- 172.21.15.125:0/156575616 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fd99c089ad0 con 0x7fd9ac0afa20 2023-10-18T12:01:16.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.839+0000 7fd9aaffd640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd99806b0b0 0x7fd99806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:16.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.840+0000 7fd9aaffd640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd99806b0b0 0x7fd99806d570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac06cbb0 tx=0x7fd9a000b390 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:16.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:16.843+0000 7fd993fff640 1 -- 172.21.15.125:0/156575616 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9ac149360 con 0x7fd9ac0afa20 2023-10-18T12:01:17.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.007+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9ac149360 con 0x7fd9ac0afa20 2023-10-18T12:01:17.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.008+0000 7fd993fff640 1 -- 172.21.15.125:0/156575616 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v242) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9ac149360 con 0x7fd9ac0afa20 2023-10-18T12:01:17.011 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:01:17.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd99806b0b0 msgr2=0x7fd99806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd99806b0b0 0x7fd99806d570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac06cbb0 tx=0x7fd9a000b390 comp rx=0 tx=0).stop 2023-10-18T12:01:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0afa20 msgr2=0x7fd9ac14cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0afa20 0x7fd9ac14cc50 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00b3f0 tx=0x7fd99c00b8b0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 shutdown_connections 2023-10-18T12:01:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fd99806b0b0 0x7fd99806d570 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9ac0b0530 0x7fd9ac14f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9ac0cbd20 0x7fd9ac14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.009+0000 7fd9b2090640 1 --2- 172.21.15.125:0/156575616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac0afa20 0x7fd9ac14cc50 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.010+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 >> 172.21.15.125:0/156575616 conn(0x7fd9ac069930 msgr2=0x7fd9ac150d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:17.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.010+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 shutdown_connections 2023-10-18T12:01:17.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.010+0000 7fd9b2090640 1 -- 172.21.15.125:0/156575616 wait complete. 2023-10-18T12:01:17.023 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:01:17.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.127+0000 7fcf844a9640 1 -- 172.21.15.125:0/3838500496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0ceb40 msgr2=0x7fcf7c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.127+0000 7fcf844a9640 1 --2- 172.21.15.125:0/3838500496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0ceb40 0x7fcf7c0b0650 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcf74005fd0 tx=0x7fcf7402bf10 comp rx=0 tx=0).stop 2023-10-18T12:01:17.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.127+0000 7fcf844a9640 1 -- 172.21.15.125:0/3838500496 shutdown_connections 2023-10-18T12:01:17.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.127+0000 7fcf844a9640 1 --2- 172.21.15.125:0/3838500496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0ceb40 0x7fcf7c0b0650 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.127+0000 7fcf844a9640 1 --2- 172.21.15.125:0/3838500496 >> v2:172.21.15.139:3300/0 conn(0x7fcf7c0cf340 0x7fcf7c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.127+0000 7fcf844a9640 1 -- 172.21.15.125:0/3838500496 >> 172.21.15.125:0/3838500496 conn(0x7fcf7c0d0d60 msgr2=0x7fcf7c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:17.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.127+0000 7fcf844a9640 1 -- 172.21.15.125:0/3838500496 shutdown_connections 2023-10-18T12:01:17.132 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.127+0000 7fcf844a9640 1 -- 172.21.15.125:0/3838500496 wait complete. 2023-10-18T12:01:17.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.128+0000 7fcf844a9640 1 Processor -- start 2023-10-18T12:01:17.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.128+0000 7fcf844a9640 1 -- start start 2023-10-18T12:01:17.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf844a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0ceb40 0x7fcf7c0b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf844a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0cf340 0x7fcf7c1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf844a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf7c15b2a0 0x7fcf7c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf844a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1f13c0 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf844a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c0b3d30 con 0x7fcf7c15b2a0 2023-10-18T12:01:17.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf844a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1f1680 con 0x7fcf7c0cf340 2023-10-18T12:01:17.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf82245640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0ceb40 0x7fcf7c0b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf82245640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0ceb40 0x7fcf7c0b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36118/0 (socket says 172.21.15.125:36118) 2023-10-18T12:01:17.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf82245640 1 -- 172.21.15.125:0/1485618491 learned_addr learned my addr 172.21.15.125:0/1485618491 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:17.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf82a46640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf7c15b2a0 0x7fcf7c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.129+0000 7fcf81a44640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0cf340 0x7fcf7c1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf82245640 1 -- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0cf340 msgr2=0x7fcf7c1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf82245640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0cf340 0x7fcf7c1578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf82245640 1 -- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf7c15b2a0 msgr2=0x7fcf7c15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf82245640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf7c15b2a0 0x7fcf7c15dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf82245640 1 -- 172.21.15.125:0/1485618491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf70009490 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf82a46640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf7c15b2a0 0x7fcf7c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf81a44640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0cf340 0x7fcf7c1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf82245640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0ceb40 0x7fcf7c0b6880 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000ea60 tx=0x7fcf7000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf6effd640 1 -- 172.21.15.125:0/1485618491 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000cde0 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf6effd640 1 -- 172.21.15.125:0/1485618491 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf70004020 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf74009840 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf6effd640 1 -- 172.21.15.125:0/1485618491 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70004410 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.130+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c0b0d50 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.131+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c0d07d0 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.132+0000 7fcf6effd640 1 -- 172.21.15.125:0/1485618491 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcf700035f0 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.132+0000 7fcf6effd640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcf6006b0b0 0x7fcf6006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.132+0000 7fcf6effd640 1 -- 172.21.15.125:0/1485618491 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fcf70014070 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.132+0000 7fcf81a44640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcf6006b0b0 0x7fcf6006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.142 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.133+0000 7fcf81a44640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcf6006b0b0 0x7fcf6006d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c15aa20 tx=0x7fcf78009040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:17.143 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.137+0000 7fcf6effd640 1 -- 172.21.15.125:0/1485618491 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf700553f0 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.303+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf7c0690e0 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.303+0000 7fcf6effd640 1 -- 172.21.15.125:0/1485618491 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v242) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf70058a10 con 0x7fcf7c0ceb40 2023-10-18T12:01:17.307 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:01:17.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcf6006b0b0 msgr2=0x7fcf6006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcf6006b0b0 0x7fcf6006d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c15aa20 tx=0x7fcf78009040 comp rx=0 tx=0).stop 2023-10-18T12:01:17.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0ceb40 msgr2=0x7fcf7c0b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0ceb40 0x7fcf7c0b6880 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000ea60 tx=0x7fcf7000ef20 comp rx=0 tx=0).stop 2023-10-18T12:01:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 shutdown_connections 2023-10-18T12:01:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcf6006b0b0 0x7fcf6006d570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf7c15b2a0 0x7fcf7c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf7c0cf340 0x7fcf7c1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 --2- 172.21.15.125:0/1485618491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0ceb40 0x7fcf7c0b6880 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 >> 172.21.15.125:0/1485618491 conn(0x7fcf7c0d0d60 msgr2=0x7fcf7c0ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:17.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 shutdown_connections 2023-10-18T12:01:17.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.305+0000 7fcf844a9640 1 -- 172.21.15.125:0/1485618491 wait complete. 2023-10-18T12:01:17.318 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:01:17.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.424+0000 7fe99e70f640 1 -- 172.21.15.125:0/1939725126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806d2d0 msgr2=0x7fe99806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.424+0000 7fe99e70f640 1 --2- 172.21.15.125:0/1939725126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806d2d0 0x7fe99806d6b0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7fe99806fc50 tx=0x7fe98802be40 comp rx=0 tx=0).stop 2023-10-18T12:01:17.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.424+0000 7fe99e70f640 1 -- 172.21.15.125:0/1939725126 shutdown_connections 2023-10-18T12:01:17.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.424+0000 7fe99e70f640 1 --2- 172.21.15.125:0/1939725126 >> v2:172.21.15.139:3300/0 conn(0x7fe99806f0a0 0x7fe99806dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.424+0000 7fe99e70f640 1 --2- 172.21.15.125:0/1939725126 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806d2d0 0x7fe99806d6b0 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.424+0000 7fe99e70f640 1 -- 172.21.15.125:0/1939725126 >> 172.21.15.125:0/1939725126 conn(0x7fe9980b0670 msgr2=0x7fe9980b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.424+0000 7fe99e70f640 1 -- 172.21.15.125:0/1939725126 shutdown_connections 2023-10-18T12:01:17.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.424+0000 7fe99e70f640 1 -- 172.21.15.125:0/1939725126 wait complete. 2023-10-18T12:01:17.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.425+0000 7fe99e70f640 1 Processor -- start 2023-10-18T12:01:17.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.425+0000 7fe99e70f640 1 -- start start 2023-10-18T12:01:17.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe99e70f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe99806d2d0 0x7fe998156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe99e70f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806f0a0 0x7fe9980abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe99e70f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9980ab950 0x7fe9980ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe99e70f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980ad440 con 0x7fe99806f0a0 2023-10-18T12:01:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe99e70f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980ac830 con 0x7fe99806d2d0 2023-10-18T12:01:17.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe99e70f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980acb10 con 0x7fe9980ab950 2023-10-18T12:01:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806f0a0 0x7fe9980abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806f0a0 0x7fe9980abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36130/0 (socket says 172.21.15.125:36130) 2023-10-18T12:01:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 -- 172.21.15.125:0/3294702607 learned_addr learned my addr 172.21.15.125:0/3294702607 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe997fff640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe99806d2d0 0x7fe998156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe99ccac640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9980ab950 0x7fe9980ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 -- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9980ab950 msgr2=0x7fe9980ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9980ab950 0x7fe9980ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 -- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe99806d2d0 msgr2=0x7fe998156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe99806d2d0 0x7fe998156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 -- 172.21.15.125:0/3294702607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009560 con 0x7fe99806f0a0 2023-10-18T12:01:17.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.426+0000 7fe9977fe640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806f0a0 0x7fe9980abde0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00b4a0 tx=0x7fe98c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.428+0000 7fe994ff9640 1 -- 172.21.15.125:0/3294702607 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c010610 con 0x7fe99806f0a0 2023-10-18T12:01:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.428+0000 7fe994ff9640 1 -- 172.21.15.125:0/3294702607 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe98c010860 con 0x7fe99806f0a0 2023-10-18T12:01:17.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.428+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9980acdf0 con 0x7fe99806f0a0 2023-10-18T12:01:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.428+0000 7fe994ff9640 1 -- 172.21.15.125:0/3294702607 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c010c70 con 0x7fe99806f0a0 2023-10-18T12:01:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.428+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe998155090 con 0x7fe99806f0a0 2023-10-18T12:01:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.429+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9980ce1d0 con 0x7fe99806f0a0 2023-10-18T12:01:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.429+0000 7fe994ff9640 1 -- 172.21.15.125:0/3294702607 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe98c017360 con 0x7fe99806f0a0 2023-10-18T12:01:17.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.430+0000 7fe994ff9640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe98406b080 0x7fe98406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.430+0000 7fe994ff9640 1 -- 172.21.15.125:0/3294702607 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fe98c0899e0 con 0x7fe99806f0a0 2023-10-18T12:01:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.430+0000 7fe997fff640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe98406b080 0x7fe98406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.430+0000 7fe997fff640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe98406b080 0x7fe98406d540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe988035000 tx=0x7fe988004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:17.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.434+0000 7fe994ff9640 1 -- 172.21.15.125:0/3294702607 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe98c0580d0 con 0x7fe99806f0a0 2023-10-18T12:01:17.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.599+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9980690e0 con 0x7fe99806f0a0 2023-10-18T12:01:17.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.599+0000 7fe994ff9640 1 -- 172.21.15.125:0/3294702607 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v242) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe98c0140a0 con 0x7fe99806f0a0 2023-10-18T12:01:17.602 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:01:17.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe98406b080 msgr2=0x7fe98406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe98406b080 0x7fe98406d540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe988035000 tx=0x7fe988004730 comp rx=0 tx=0).stop 2023-10-18T12:01:17.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806f0a0 msgr2=0x7fe9980abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806f0a0 0x7fe9980abde0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00b4a0 tx=0x7fe98c00b960 comp rx=0 tx=0).stop 2023-10-18T12:01:17.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 shutdown_connections 2023-10-18T12:01:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe98406b080 0x7fe98406d540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9980ab950 0x7fe9980ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe99806f0a0 0x7fe9980abde0 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 --2- 172.21.15.125:0/3294702607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe99806d2d0 0x7fe998156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 >> 172.21.15.125:0/3294702607 conn(0x7fe9980b0670 msgr2=0x7fe99815a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:17.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 shutdown_connections 2023-10-18T12:01:17.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.601+0000 7fe99e70f640 1 -- 172.21.15.125:0/3294702607 wait complete. 2023-10-18T12:01:17.614 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T12:01:17.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.713+0000 7f55f6432640 1 -- 172.21.15.125:0/3412710768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 msgr2=0x7f55f00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.716 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.713+0000 7f55f6432640 1 --2- 172.21.15.125:0/3412710768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 0x7f55f00d1910 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f55f008a120 tx=0x7f55d802bcf0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.713+0000 7f55f6432640 1 -- 172.21.15.125:0/3412710768 shutdown_connections 2023-10-18T12:01:17.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.713+0000 7f55f6432640 1 --2- 172.21.15.125:0/3412710768 >> v2:172.21.15.139:3300/0 conn(0x7f55f015b2a0 0x7f55f00cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.713+0000 7f55f6432640 1 --2- 172.21.15.125:0/3412710768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 0x7f55f00d1910 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.713+0000 7f55f6432640 1 -- 172.21.15.125:0/3412710768 >> 172.21.15.125:0/3412710768 conn(0x7f55f00d0d60 msgr2=0x7f55f00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:17.717 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.713+0000 7f55f6432640 1 -- 172.21.15.125:0/3412710768 shutdown_connections 2023-10-18T12:01:17.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.713+0000 7f55f6432640 1 -- 172.21.15.125:0/3412710768 wait complete. 2023-10-18T12:01:17.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.714+0000 7f55f6432640 1 Processor -- start 2023-10-18T12:01:17.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55f6432640 1 -- start start 2023-10-18T12:01:17.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55f6432640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f00ceb40 0x7f55f00c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55f6432640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 0x7f55f009e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55f6432640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f015b2a0 0x7f55f00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55f6432640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00c9130 con 0x7f55f00cf340 2023-10-18T12:01:17.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55f6432640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00cb140 con 0x7f55f00ceb40 2023-10-18T12:01:17.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55f6432640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00ca8a0 con 0x7f55f015b2a0 2023-10-18T12:01:17.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 0x7f55f009e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 0x7f55f009e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:32972/0 (socket says 172.21.15.125:32972) 2023-10-18T12:01:17.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55ef7fe640 1 -- 172.21.15.125:0/2986458270 learned_addr learned my addr 172.21.15.125:0/2986458270 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:17.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55effff640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f00ceb40 0x7f55f00c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.715+0000 7f55f49cf640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f015b2a0 0x7f55f00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ef7fe640 1 -- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f015b2a0 msgr2=0x7f55f00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ef7fe640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f015b2a0 0x7f55f00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ef7fe640 1 -- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f00ceb40 msgr2=0x7f55f00c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ef7fe640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f00ceb40 0x7f55f00c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ef7fe640 1 -- 172.21.15.125:0/2986458270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d8009490 con 0x7f55f00cf340 2023-10-18T12:01:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ef7fe640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 0x7f55f009e390 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f55e000d740 tx=0x7f55e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ecff9640 1 -- 172.21.15.125:0/2986458270 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0009c30 con 0x7f55f00cf340 2023-10-18T12:01:17.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ecff9640 1 -- 172.21.15.125:0/2986458270 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55e0009e80 con 0x7f55f00cf340 2023-10-18T12:01:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55ecff9640 1 -- 172.21.15.125:0/2986458270 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e00106c0 con 0x7f55f00cf340 2023-10-18T12:01:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f00cab80 con 0x7f55f00cf340 2023-10-18T12:01:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.716+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f01f4dc0 con 0x7f55f00cf340 2023-10-18T12:01:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.717+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f01f4f80 con 0x7f55f00cf340 2023-10-18T12:01:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.717+0000 7f55ecff9640 1 -- 172.21.15.125:0/2986458270 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f55e0010820 con 0x7f55f00cf340 2023-10-18T12:01:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.718+0000 7f55ecff9640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f55d006b080 0x7f55d006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.718+0000 7f55ecff9640 1 -- 172.21.15.125:0/2986458270 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f55e008a090 con 0x7f55f00cf340 2023-10-18T12:01:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.718+0000 7f55effff640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f55d006b080 0x7f55d006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.718+0000 7f55effff640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f55d006b080 0x7f55d006d540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55d8036000 tx=0x7f55d8004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:17.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.722+0000 7f55ecff9640 1 -- 172.21.15.125:0/2986458270 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55e0054c90 con 0x7f55f00cf340 2023-10-18T12:01:17.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.889+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f55f00690e0 con 0x7f55f00cf340 2023-10-18T12:01:17.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.890+0000 7f55ecff9640 1 -- 172.21.15.125:0/2986458270 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v242) v1 ==== 84+0+209 (secure 0 0 0) 0x7f55e00210f0 con 0x7f55f00cf340 2023-10-18T12:01:17.893 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:17.893 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T12:01:17.893 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:17.893 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T12:01:17.894 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T12:01:17.894 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:17.894 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:17.894 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 31, 2023-10-18T12:01:17.894 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": "cephfs_metadata" 2023-10-18T12:01:17.895 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:17.895 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:17.895 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 32, 2023-10-18T12:01:17.895 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": "cephfs_data" 2023-10-18T12:01:17.895 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:17.896 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T12:01:17.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f55d006b080 msgr2=0x7f55d006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f55d006b080 0x7f55d006d540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55d8036000 tx=0x7f55d8004730 comp rx=0 tx=0).stop 2023-10-18T12:01:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 msgr2=0x7f55f009e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 0x7f55f009e390 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f55e000d740 tx=0x7f55e000dc00 comp rx=0 tx=0).stop 2023-10-18T12:01:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 shutdown_connections 2023-10-18T12:01:17.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f55d006b080 0x7f55d006d540 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f015b2a0 0x7f55f00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f00cf340 0x7f55f009e390 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 --2- 172.21.15.125:0/2986458270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f00ceb40 0x7f55f00c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:17.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.891+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 >> 172.21.15.125:0/2986458270 conn(0x7f55f00d0d60 msgr2=0x7f55f00b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:17.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.892+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 shutdown_connections 2023-10-18T12:01:17.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:17.892+0000 7f55f6432640 1 -- 172.21.15.125:0/2986458270 wait complete. 2023-10-18T12:01:17.904 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-18T12:01:17.905 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-18T12:01:18.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.008+0000 7fe7d07e9640 1 -- 172.21.15.125:0/2331287702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80cbd20 msgr2=0x7fe7c8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:18.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.008+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/2331287702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80cbd20 0x7fe7c8158300 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4005fd0 tx=0x7fe7c402d400 comp rx=0 tx=0).stop 2023-10-18T12:01:18.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.008+0000 7fe7d07e9640 1 -- 172.21.15.125:0/2331287702 shutdown_connections 2023-10-18T12:01:18.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.008+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/2331287702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7c80afa20 0x7fe7c80705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.008+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/2331287702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80cbd20 0x7fe7c8158300 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.008+0000 7fe7d07e9640 1 -- 172.21.15.125:0/2331287702 >> 172.21.15.125:0/2331287702 conn(0x7fe7c8069930 msgr2=0x7fe7c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:18.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.008+0000 7fe7d07e9640 1 -- 172.21.15.125:0/2331287702 shutdown_connections 2023-10-18T12:01:18.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.008+0000 7fe7d07e9640 1 -- 172.21.15.125:0/2331287702 wait complete. 2023-10-18T12:01:18.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.009+0000 7fe7d07e9640 1 Processor -- start 2023-10-18T12:01:18.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.009+0000 7fe7d07e9640 1 -- start start 2023-10-18T12:01:18.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7d07e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80afa20 0x7fe7c815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:18.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7d07e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7c80b0530 0x7fe7c8156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:18.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7d07e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7c80cbd20 0x7fe7c813de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:18.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7d07e9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c815aa00 con 0x7fe7c80afa20 2023-10-18T12:01:18.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7d07e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c8159cc0 con 0x7fe7c80b0530 2023-10-18T12:01:18.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7d07e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c8158980 con 0x7fe7c80cbd20 2023-10-18T12:01:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7ce585640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80afa20 0x7fe7c815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7ce585640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80afa20 0x7fe7c815a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33004/0 (socket says 172.21.15.125:33004) 2023-10-18T12:01:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7ce585640 1 -- 172.21.15.125:0/4053096364 learned_addr learned my addr 172.21.15.125:0/4053096364 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:18.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7ce585640 1 -- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7c80cbd20 msgr2=0x7fe7c813de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7ce585640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7c80cbd20 0x7fe7c813de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7ce585640 1 -- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7c80b0530 msgr2=0x7fe7c8156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7ce585640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7c80b0530 0x7fe7c8156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.010+0000 7fe7ce585640 1 -- 172.21.15.125:0/4053096364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c400a950 con 0x7fe7c80afa20 2023-10-18T12:01:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.011+0000 7fe7ce585640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80afa20 0x7fe7c815a4c0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4000f80 tx=0x7fe7c4032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.011+0000 7fe7beffd640 1 -- 172.21.15.125:0/4053096364 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4004240 con 0x7fe7c80afa20 2023-10-18T12:01:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.011+0000 7fe7beffd640 1 -- 172.21.15.125:0/4053096364 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe7c4004490 con 0x7fe7c80afa20 2023-10-18T12:01:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.011+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c8158c20 con 0x7fe7c80afa20 2023-10-18T12:01:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.011+0000 7fe7beffd640 1 -- 172.21.15.125:0/4053096364 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40048a0 con 0x7fe7c80afa20 2023-10-18T12:01:18.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.011+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c81591e0 con 0x7fe7c80afa20 2023-10-18T12:01:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.011+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c80690e0 con 0x7fe7c80afa20 2023-10-18T12:01:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.012+0000 7fe7beffd640 1 -- 172.21.15.125:0/4053096364 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe7c403b070 con 0x7fe7c80afa20 2023-10-18T12:01:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.013+0000 7fe7beffd640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe7ac06b080 0x7fe7ac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:18.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.013+0000 7fe7beffd640 1 -- 172.21.15.125:0/4053096364 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fe7c40ad160 con 0x7fe7c80afa20 2023-10-18T12:01:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.013+0000 7fe7cdd84640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe7ac06b080 0x7fe7ac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:18.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.013+0000 7fe7cdd84640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe7ac06b080 0x7fe7ac06d540 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005fd0 tx=0x7fe7b8005850 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:18.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.017+0000 7fe7beffd640 1 -- 172.21.15.125:0/4053096364 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7c4077cf0 con 0x7fe7c80afa20 2023-10-18T12:01:18.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.181+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe7c80675f0 con 0x7fe7c80afa20 2023-10-18T12:01:18.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.434+0000 7fe7beffd640 1 -- 172.21.15.125:0/4053096364 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v243) v1 ==== 144+0+0 (secure 0 0 0) 0x7fe7c407b310 con 0x7fe7c80afa20 2023-10-18T12:01:18.437 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'backup_fs_metadata' created 2023-10-18T12:01:18.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe7ac06b080 msgr2=0x7fe7ac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:18.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe7ac06b080 0x7fe7ac06d540 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005fd0 tx=0x7fe7b8005850 comp rx=0 tx=0).stop 2023-10-18T12:01:18.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80afa20 msgr2=0x7fe7c815a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:18.439 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80afa20 0x7fe7c815a4c0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4000f80 tx=0x7fe7c4032b90 comp rx=0 tx=0).stop 2023-10-18T12:01:18.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 shutdown_connections 2023-10-18T12:01:18.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe7ac06b080 0x7fe7ac06d540 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7c80b0530 0x7fe7c8156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7c80cbd20 0x7fe7c813de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 --2- 172.21.15.125:0/4053096364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7c80afa20 0x7fe7c815a4c0 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 >> 172.21.15.125:0/4053096364 conn(0x7fe7c8069930 msgr2=0x7fe7c8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:18.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 shutdown_connections 2023-10-18T12:01:18.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.436+0000 7fe7d07e9640 1 -- 172.21.15.125:0/4053096364 wait complete. 2023-10-18T12:01:18.453 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T12:01:18.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.560+0000 7f4b64e64640 1 -- 172.21.15.125:0/1811135470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b48003fd0 msgr2=0x7f4b480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:18.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.560+0000 7f4b64e64640 1 --2- 172.21.15.125:0/1811135470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b48003fd0 0x7f4b480047c0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400bb20 tx=0x7f4b480057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.560+0000 7f4b5ed9d640 1 -- 172.21.15.125:0/1811135470 reap_dead start 2023-10-18T12:01:18.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.560+0000 7f4b64e64640 1 -- 172.21.15.125:0/1811135470 shutdown_connections 2023-10-18T12:01:18.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.560+0000 7f4b64e64640 1 -- 172.21.15.125:0/1811135470 >> 172.21.15.125:0/1811135470 conn(0x7f4b60069930 msgr2=0x7f4b60069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:18.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.560+0000 7f4b64e64640 1 -- 172.21.15.125:0/1811135470 shutdown_connections 2023-10-18T12:01:18.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.560+0000 7f4b64e64640 1 -- 172.21.15.125:0/1811135470 wait complete. 2023-10-18T12:01:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b64e64640 1 Processor -- start 2023-10-18T12:01:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b64e64640 1 -- start start 2023-10-18T12:01:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b64e64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b48003fd0 0x7f4b601f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b64e64640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b6013ac00 0x7f4b60150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:18.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b64e64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b600ca870 0x7f4b600cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:18.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b64e64640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600aebf0 con 0x7f4b600ca870 2023-10-18T12:01:18.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b64e64640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600cb260 con 0x7f4b48003fd0 2023-10-18T12:01:18.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b64e64640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600ae3e0 con 0x7f4b6013ac00 2023-10-18T12:01:18.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b5ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b600ca870 0x7f4b600cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:18.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b5ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b600ca870 0x7f4b600cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33020/0 (socket says 172.21.15.125:33020) 2023-10-18T12:01:18.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.562+0000 7f4b5ed9d640 1 -- 172.21.15.125:0/2274111483 learned_addr learned my addr 172.21.15.125:0/2274111483 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:18.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5dd9b640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b6013ac00 0x7f4b60150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:18.569 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5ed9d640 1 -- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b6013ac00 msgr2=0x7f4b60150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:18.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5e59c640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b48003fd0 0x7f4b601f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:18.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5ed9d640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b6013ac00 0x7f4b60150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5ed9d640 1 -- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b48003fd0 msgr2=0x7f4b601f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:18.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5ed9d640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b48003fd0 0x7f4b601f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5ed9d640 1 -- 172.21.15.125:0/2274111483 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600ae630 con 0x7f4b600ca870 2023-10-18T12:01:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5dd9b640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b6013ac00 0x7f4b60150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5e59c640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b48003fd0 0x7f4b601f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b5ed9d640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b600ca870 0x7f4b600cad20 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f4b6006d570 tx=0x7f4b58001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b4effd640 1 -- 172.21.15.125:0/2274111483 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58011660 con 0x7f4b600ca870 2023-10-18T12:01:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b4effd640 1 -- 172.21.15.125:0/2274111483 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b580118b0 con 0x7f4b600ca870 2023-10-18T12:01:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b600ae910 con 0x7f4b600ca870 2023-10-18T12:01:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b4effd640 1 -- 172.21.15.125:0/2274111483 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58011cc0 con 0x7f4b600ca870 2023-10-18T12:01:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.563+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6013b3f0 con 0x7f4b600ca870 2023-10-18T12:01:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.564+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b60154af0 con 0x7f4b600ca870 2023-10-18T12:01:18.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.565+0000 7f4b4effd640 1 -- 172.21.15.125:0/2274111483 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4b5801c460 con 0x7f4b600ca870 2023-10-18T12:01:18.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.565+0000 7f4b4effd640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4b4006b080 0x7f4b4006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.565+0000 7f4b4effd640 1 -- 172.21.15.125:0/2274111483 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f4b5808a540 con 0x7f4b600ca870 2023-10-18T12:01:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.565+0000 7f4b5e59c640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4b4006b080 0x7f4b4006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.566+0000 7f4b5e59c640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4b4006b080 0x7f4b4006d540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b50001b20 tx=0x7f4b50002190 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.569+0000 7f4b4effd640 1 -- 172.21.15.125:0/2274111483 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b60154af0 con 0x7f4b600ca870 2023-10-18T12:01:18.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:18.736+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4b60154ed0 con 0x7f4b600ca870 2023-10-18T12:01:19.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.443+0000 7f4b4effd640 1 -- 172.21.15.125:0/2274111483 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v244) v1 ==== 176+0+0 (secure 0 0 0) 0x7f4b60154ed0 con 0x7f4b600ca870 2023-10-18T12:01:19.446 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'backup_fs_data' created 2023-10-18T12:01:19.448 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.445+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4b4006b080 msgr2=0x7f4b4006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:19.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.445+0000 7f4b64e64640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4b4006b080 0x7f4b4006d540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b50001b20 tx=0x7f4b50002190 comp rx=0 tx=0).stop 2023-10-18T12:01:19.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b600ca870 msgr2=0x7f4b600cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:19.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b600ca870 0x7f4b600cad20 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f4b6006d570 tx=0x7f4b58001630 comp rx=0 tx=0).stop 2023-10-18T12:01:19.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 shutdown_connections 2023-10-18T12:01:19.449 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f4b4006b080 0x7f4b4006d540 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:19.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4b600ca870 0x7f4b600cad20 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:19.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b6013ac00 0x7f4b60150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:19.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 --2- 172.21.15.125:0/2274111483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b48003fd0 0x7f4b601f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:19.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 >> 172.21.15.125:0/2274111483 conn(0x7f4b60069930 msgr2=0x7f4b600affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:19.450 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 shutdown_connections 2023-10-18T12:01:19.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.446+0000 7f4b64e64640 1 -- 172.21.15.125:0/2274111483 wait complete. 2023-10-18T12:01:19.468 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-18T12:01:19.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.572+0000 7ff379f2c640 1 -- 172.21.15.125:0/1730307017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 msgr2=0x7ff3740d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:19.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.572+0000 7ff379f2c640 1 --2- 172.21.15.125:0/1730307017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 0x7ff3740d1910 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7ff368009830 tx=0x7ff36802bd60 comp rx=0 tx=0).stop 2023-10-18T12:01:19.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.572+0000 7ff379f2c640 1 -- 172.21.15.125:0/1730307017 shutdown_connections 2023-10-18T12:01:19.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.572+0000 7ff379f2c640 1 --2- 172.21.15.125:0/1730307017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 0x7ff3740d1910 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:19.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.572+0000 7ff379f2c640 1 -- 172.21.15.125:0/1730307017 >> 172.21.15.125:0/1730307017 conn(0x7ff3740d0d60 msgr2=0x7ff3740d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:19.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.572+0000 7ff379f2c640 1 -- 172.21.15.125:0/1730307017 shutdown_connections 2023-10-18T12:01:19.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.572+0000 7ff379f2c640 1 -- 172.21.15.125:0/1730307017 wait complete. 2023-10-18T12:01:19.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.573+0000 7ff379f2c640 1 Processor -- start 2023-10-18T12:01:19.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.573+0000 7ff379f2c640 1 -- start start 2023-10-18T12:01:19.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff379f2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3740ceb40 0x7ff3740c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:19.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff379f2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740cf340 0x7ff3740c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:19.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff379f2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 0x7ff374158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:19.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff379f2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff374157630 con 0x7ff37415b2a0 2023-10-18T12:01:19.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff379f2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740c9dc0 con 0x7ff3740cf340 2023-10-18T12:01:19.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff379f2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740ca0a0 con 0x7ff3740ceb40 2023-10-18T12:01:19.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff373fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 0x7ff374158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:19.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff373fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 0x7ff374158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33036/0 (socket says 172.21.15.125:33036) 2023-10-18T12:01:19.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff373fff640 1 -- 172.21.15.125:0/3886507049 learned_addr learned my addr 172.21.15.125:0/3886507049 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:19.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff372ffd640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740cf340 0x7ff3740c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:19.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.574+0000 7ff3737fe640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3740ceb40 0x7ff3740c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:19.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff373fff640 1 -- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3740ceb40 msgr2=0x7ff3740c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:19.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff373fff640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3740ceb40 0x7ff3740c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:19.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff373fff640 1 -- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740cf340 msgr2=0x7ff3740c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:19.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff373fff640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740cf340 0x7ff3740c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:19.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff373fff640 1 -- 172.21.15.125:0/3886507049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368009490 con 0x7ff37415b2a0 2023-10-18T12:01:19.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff372ffd640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740cf340 0x7ff3740c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:19.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff3737fe640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3740ceb40 0x7ff3740c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:19.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff373fff640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 0x7ff374158080 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00cb40 tx=0x7ff36c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:19.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff35bfff640 1 -- 172.21.15.125:0/3886507049 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00aca0 con 0x7ff37415b2a0 2023-10-18T12:01:19.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff35bfff640 1 -- 172.21.15.125:0/3886507049 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff36c012420 con 0x7ff37415b2a0 2023-10-18T12:01:19.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff35bfff640 1 -- 172.21.15.125:0/3886507049 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c012850 con 0x7ff37415b2a0 2023-10-18T12:01:19.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff374157880 con 0x7ff37415b2a0 2023-10-18T12:01:19.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.575+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3740c9a60 con 0x7ff37415b2a0 2023-10-18T12:01:19.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.576+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff374157da0 con 0x7ff37415b2a0 2023-10-18T12:01:19.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.576+0000 7ff35bfff640 1 -- 172.21.15.125:0/3886507049 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff36c01a3b0 con 0x7ff37415b2a0 2023-10-18T12:01:19.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.577+0000 7ff35bfff640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff36006b080 0x7ff36006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:19.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.577+0000 7ff35bfff640 1 -- 172.21.15.125:0/3886507049 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6843+0+0 (secure 0 0 0) 0x7ff36c08a5b0 con 0x7ff37415b2a0 2023-10-18T12:01:19.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.577+0000 7ff3737fe640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff36006b080 0x7ff36006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:19.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.577+0000 7ff3737fe640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff36006b080 0x7ff36006d540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff364007fd0 tx=0x7ff364009040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:19.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.581+0000 7ff35bfff640 1 -- 172.21.15.125:0/3886507049 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff374157da0 con 0x7ff37415b2a0 2023-10-18T12:01:19.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:19.782+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7ff374157da0 con 0x7ff37415b2a0 2023-10-18T12:01:20.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.444+0000 7ff35bfff640 1 -- 172.21.15.125:0/3886507049 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 33 and data pool 34 v101) v1 ==== 183+0+0 (secure 0 0 0) 0x7ff374157da0 con 0x7ff37415b2a0 2023-10-18T12:01:20.447 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 33 and data pool 34 2023-10-18T12:01:20.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.448+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff36006b080 msgr2=0x7ff36006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.453 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.448+0000 7ff379f2c640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff36006b080 0x7ff36006d540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff364007fd0 tx=0x7ff364009040 comp rx=0 tx=0).stop 2023-10-18T12:01:20.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.448+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 msgr2=0x7ff374158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.448+0000 7ff379f2c640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 0x7ff374158080 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00cb40 tx=0x7ff36c00a060 comp rx=0 tx=0).stop 2023-10-18T12:01:20.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.451+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 shutdown_connections 2023-10-18T12:01:20.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.451+0000 7ff379f2c640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7ff36006b080 0x7ff36006d540 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.451+0000 7ff379f2c640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3740cf340 0x7ff3740c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.451+0000 7ff379f2c640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff37415b2a0 0x7ff374158080 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.451+0000 7ff379f2c640 1 --2- 172.21.15.125:0/3886507049 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3740ceb40 0x7ff3740c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.451+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 >> 172.21.15.125:0/3886507049 conn(0x7ff3740d0d60 msgr2=0x7ff37415a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:20.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.460+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 shutdown_connections 2023-10-18T12:01:20.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.460+0000 7ff379f2c640 1 -- 172.21.15.125:0/3886507049 wait complete. 2023-10-18T12:01:20.487 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:20.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.588+0000 7fce07525640 1 -- 172.21.15.125:0/2404328892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 msgr2=0x7fce000d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.588+0000 7fce07525640 1 --2- 172.21.15.125:0/2404328892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 0x7fce000d1910 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009830 tx=0x7fcdf802bd60 comp rx=0 tx=0).stop 2023-10-18T12:01:20.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.588+0000 7fce07525640 1 -- 172.21.15.125:0/2404328892 shutdown_connections 2023-10-18T12:01:20.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.588+0000 7fce07525640 1 --2- 172.21.15.125:0/2404328892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 0x7fce000d1910 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.588+0000 7fce07525640 1 -- 172.21.15.125:0/2404328892 >> 172.21.15.125:0/2404328892 conn(0x7fce000d0d60 msgr2=0x7fce000d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:20.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.589+0000 7fce07525640 1 -- 172.21.15.125:0/2404328892 shutdown_connections 2023-10-18T12:01:20.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.589+0000 7fce07525640 1 -- 172.21.15.125:0/2404328892 wait complete. 2023-10-18T12:01:20.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce07525640 1 Processor -- start 2023-10-18T12:01:20.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce07525640 1 -- start start 2023-10-18T12:01:20.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce07525640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce000ceb40 0x7fce0014f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:20.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce07525640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 0x7fce000c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:20.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce07525640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce0015b2a0 0x7fce000c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:20.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce07525640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce000b6090 con 0x7fce0015b2a0 2023-10-18T12:01:20.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce07525640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce000c9230 con 0x7fce000cf340 2023-10-18T12:01:20.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce07525640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce000b6900 con 0x7fce000ceb40 2023-10-18T12:01:20.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.590+0000 7fce05ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce0015b2a0 0x7fce000c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:20.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce05ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce0015b2a0 0x7fce000c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33040/0 (socket says 172.21.15.125:33040) 2023-10-18T12:01:20.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce05ac2640 1 -- 172.21.15.125:0/2223125288 learned_addr learned my addr 172.21.15.125:0/2223125288 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:20.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce04ac0640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 0x7fce000c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:20.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce05ac2640 1 -- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce000ceb40 msgr2=0x7fce0014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce05ac2640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce000ceb40 0x7fce0014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce05ac2640 1 -- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 msgr2=0x7fce000c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce05ac2640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 0x7fce000c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce05ac2640 1 -- 172.21.15.125:0/2223125288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8009490 con 0x7fce0015b2a0 2023-10-18T12:01:20.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce04ac0640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 0x7fce000c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:20.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce05ac2640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce0015b2a0 0x7fce000c8370 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c710 tx=0x7fcdfc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:20.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fcdf5ffb640 1 -- 172.21.15.125:0/2223125288 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc006d50 con 0x7fce0015b2a0 2023-10-18T12:01:20.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fcdf5ffb640 1 -- 172.21.15.125:0/2223125288 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdfc00ce20 con 0x7fce0015b2a0 2023-10-18T12:01:20.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce000c89a0 con 0x7fce0015b2a0 2023-10-18T12:01:20.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fcdf5ffb640 1 -- 172.21.15.125:0/2223125288 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc003bd0 con 0x7fce0015b2a0 2023-10-18T12:01:20.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.591+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce000c9480 con 0x7fce0015b2a0 2023-10-18T12:01:20.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.592+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce000d04f0 con 0x7fce0015b2a0 2023-10-18T12:01:20.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.593+0000 7fcdf5ffb640 1 -- 172.21.15.125:0/2223125288 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcdfc003d30 con 0x7fce0015b2a0 2023-10-18T12:01:20.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.593+0000 7fcdf5ffb640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcde406b080 0x7fcde406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:20.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.593+0000 7fcdf5ffb640 1 -- 172.21.15.125:0/2223125288 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fcdfc089cd0 con 0x7fce0015b2a0 2023-10-18T12:01:20.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.593+0000 7fce052c1640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcde406b080 0x7fcde406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:20.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.594+0000 7fce052c1640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcde406b080 0x7fcde406d540 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8036000 tx=0x7fcdf80046c0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:20.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.597+0000 7fcdf5ffb640 1 -- 172.21.15.125:0/2223125288 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdfc0545d0 con 0x7fce0015b2a0 2023-10-18T12:01:20.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.763+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce000690e0 con 0x7fce0015b2a0 2023-10-18T12:01:20.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.765+0000 7fcdf5ffb640 1 -- 172.21.15.125:0/2223125288 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v245) v1 ==== 74+0+20595 (secure 0 0 0) 0x7fcdfc057bf0 con 0x7fce0015b2a0 2023-10-18T12:01:20.768 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:20.768 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":245,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:19.784020+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:20.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcde406b080 msgr2=0x7fcde406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcde406b080 0x7fcde406d540 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8036000 tx=0x7fcdf80046c0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce0015b2a0 msgr2=0x7fce000c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce0015b2a0 0x7fce000c8370 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c710 tx=0x7fcdfc00cbd0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 shutdown_connections 2023-10-18T12:01:20.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcde406b080 0x7fcde406d540 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce000cf340 0x7fce000c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce0015b2a0 0x7fce000c8370 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 --2- 172.21.15.125:0/2223125288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce000ceb40 0x7fce0014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 >> 172.21.15.125:0/2223125288 conn(0x7fce000d0d60 msgr2=0x7fce000cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:20.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 shutdown_connections 2023-10-18T12:01:20.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.767+0000 7fce07525640 1 -- 172.21.15.125:0/2223125288 wait complete. 2023-10-18T12:01:20.780 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:20.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.880+0000 7f79e714c640 1 -- 172.21.15.125:0/4147953995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b410 msgr2=0x7f79e015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.880+0000 7f79e714c640 1 --2- 172.21.15.125:0/4147953995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b410 0x7f79e015c770 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009830 tx=0x7f79cc02bd60 comp rx=0 tx=0).stop 2023-10-18T12:01:20.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.880+0000 7f79e714c640 1 -- 172.21.15.125:0/4147953995 shutdown_connections 2023-10-18T12:01:20.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.880+0000 7f79e714c640 1 --2- 172.21.15.125:0/4147953995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b410 0x7f79e015c770 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.880+0000 7f79e714c640 1 -- 172.21.15.125:0/4147953995 >> 172.21.15.125:0/4147953995 conn(0x7f79e0069930 msgr2=0x7f79e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:20.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.880+0000 7f79e714c640 1 -- 172.21.15.125:0/4147953995 shutdown_connections 2023-10-18T12:01:20.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.880+0000 7f79e714c640 1 -- 172.21.15.125:0/4147953995 wait complete. 2023-10-18T12:01:20.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.881+0000 7f79e714c640 1 Processor -- start 2023-10-18T12:01:20.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.881+0000 7f79e714c640 1 -- start start 2023-10-18T12:01:20.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e714c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b030 0x7f79e0139290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:20.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e714c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e006b410 0x7f79e00ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:20.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e714c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79e015ccb0 0x7f79e00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:20.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e714c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e01397d0 con 0x7f79e006b030 2023-10-18T12:01:20.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e714c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e013d2a0 con 0x7f79e006b410 2023-10-18T12:01:20.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e714c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e013d580 con 0x7f79e015ccb0 2023-10-18T12:01:20.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e4ee8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b030 0x7f79e0139290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:20.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e4ee8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b030 0x7f79e0139290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33070/0 (socket says 172.21.15.125:33070) 2023-10-18T12:01:20.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e4ee8640 1 -- 172.21.15.125:0/3763390124 learned_addr learned my addr 172.21.15.125:0/3763390124 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:20.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79e56e9640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79e015ccb0 0x7f79e00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:20.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.882+0000 7f79dffff640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e006b410 0x7f79e00ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:20.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e4ee8640 1 -- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79e015ccb0 msgr2=0x7f79e00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e4ee8640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79e015ccb0 0x7f79e00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e4ee8640 1 -- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e006b410 msgr2=0x7f79e00ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:20.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e4ee8640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e006b410 0x7f79e00ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:20.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e4ee8640 1 -- 172.21.15.125:0/3763390124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc009490 con 0x7f79e006b030 2023-10-18T12:01:20.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79dffff640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e006b410 0x7f79e00ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:20.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e56e9640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79e015ccb0 0x7f79e00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:20.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e4ee8640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b030 0x7f79e0139290 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f79d400c450 tx=0x7f79d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:20.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79dd7fa640 1 -- 172.21.15.125:0/3763390124 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40052d0 con 0x7f79e006b030 2023-10-18T12:01:20.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79dd7fa640 1 -- 172.21.15.125:0/3763390124 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79d4005430 con 0x7f79e006b030 2023-10-18T12:01:20.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e013d860 con 0x7f79e006b030 2023-10-18T12:01:20.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79dd7fa640 1 -- 172.21.15.125:0/3763390124 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4012890 con 0x7f79e006b030 2023-10-18T12:01:20.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.883+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e013de20 con 0x7f79e006b030 2023-10-18T12:01:20.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.884+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e00690e0 con 0x7f79e006b030 2023-10-18T12:01:20.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.884+0000 7f79dd7fa640 1 -- 172.21.15.125:0/3763390124 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f79d40035f0 con 0x7f79e006b030 2023-10-18T12:01:20.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.885+0000 7f79dd7fa640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f79c406b080 0x7f79c406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:20.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.885+0000 7f79dd7fa640 1 -- 172.21.15.125:0/3763390124 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f79d4089b40 con 0x7f79e006b030 2023-10-18T12:01:20.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.885+0000 7f79dffff640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f79c406b080 0x7f79c406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:20.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.885+0000 7f79dffff640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f79c406b080 0x7f79c406d540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f79cc036000 tx=0x7f79cc004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:20.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:20.889+0000 7f79dd7fa640 1 -- 172.21.15.125:0/3763390124 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79d4054440 con 0x7f79e006b030 2023-10-18T12:01:21.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.056+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79e013f100 con 0x7f79e006b030 2023-10-18T12:01:21.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.058+0000 7f79dd7fa640 1 -- 172.21.15.125:0/3763390124 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v245) v1 ==== 74+0+20595 (secure 0 0 0) 0x7f79d4057a60 con 0x7f79e006b030 2023-10-18T12:01:21.060 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:21.061 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":245,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:19.784020+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:21.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f79c406b080 msgr2=0x7f79c406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f79c406b080 0x7f79c406d540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f79cc036000 tx=0x7f79cc004730 comp rx=0 tx=0).stop 2023-10-18T12:01:21.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b030 msgr2=0x7f79e0139290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b030 0x7f79e0139290 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f79d400c450 tx=0x7f79d400c910 comp rx=0 tx=0).stop 2023-10-18T12:01:21.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 shutdown_connections 2023-10-18T12:01:21.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f79c406b080 0x7f79c406d540 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79e006b410 0x7f79e00ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79e015ccb0 0x7f79e00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 --2- 172.21.15.125:0/3763390124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79e006b030 0x7f79e0139290 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 >> 172.21.15.125:0/3763390124 conn(0x7f79e0069930 msgr2=0x7f79e015bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:21.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 shutdown_connections 2023-10-18T12:01:21.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.060+0000 7f79e714c640 1 -- 172.21.15.125:0/3763390124 wait complete. 2023-10-18T12:01:21.073 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-18T12:01:21.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.177+0000 7f7cca3fc640 1 -- 172.21.15.125:0/4242909471 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc415b2a0 msgr2=0x7f7cc40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.177+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/4242909471 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc415b2a0 0x7f7cc40cf720 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8005fd0 tx=0x7f7cb802bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:21.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.177+0000 7f7cca3fc640 1 -- 172.21.15.125:0/4242909471 shutdown_connections 2023-10-18T12:01:21.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.177+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/4242909471 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc415b2a0 0x7f7cc40cf720 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.177+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/4242909471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc40cf340 0x7f7cc40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.177+0000 7f7cca3fc640 1 -- 172.21.15.125:0/4242909471 >> 172.21.15.125:0/4242909471 conn(0x7f7cc40d0d60 msgr2=0x7f7cc40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:21.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.177+0000 7f7cca3fc640 1 -- 172.21.15.125:0/4242909471 shutdown_connections 2023-10-18T12:01:21.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.177+0000 7f7cca3fc640 1 -- 172.21.15.125:0/4242909471 wait complete. 2023-10-18T12:01:21.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.178+0000 7f7cca3fc640 1 Processor -- start 2023-10-18T12:01:21.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cca3fc640 1 -- start start 2023-10-18T12:01:21.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cca3fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40ceb40 0x7f7cc409e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:21.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cca3fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc40cf340 0x7f7cc40ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:21.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cca3fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc415b2a0 0x7f7cc40ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:21.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cca3fc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc409dd80 con 0x7f7cc40cf340 2023-10-18T12:01:21.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cca3fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc409ced0 con 0x7f7cc40ceb40 2023-10-18T12:01:21.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cca3fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc409d170 con 0x7f7cc415b2a0 2023-10-18T12:01:21.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cc37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc40cf340 0x7f7cc40ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:21.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cc37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc40cf340 0x7f7cc40ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33086/0 (socket says 172.21.15.125:33086) 2023-10-18T12:01:21.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.179+0000 7f7cc37fe640 1 -- 172.21.15.125:0/3891345865 learned_addr learned my addr 172.21.15.125:0/3891345865 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:21.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc8999640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc415b2a0 0x7f7cc40ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:21.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc3fff640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40ceb40 0x7f7cc409e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:21.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc37fe640 1 -- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc415b2a0 msgr2=0x7f7cc40ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc37fe640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc415b2a0 0x7f7cc40ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc37fe640 1 -- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40ceb40 msgr2=0x7f7cc409e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc37fe640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40ceb40 0x7f7cc409e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc37fe640 1 -- 172.21.15.125:0/3891345865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb8009560 con 0x7f7cc40cf340 2023-10-18T12:01:21.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc8999640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc415b2a0 0x7f7cc40ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:21.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc3fff640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40ceb40 0x7f7cc409e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:21.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc37fe640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc40cf340 0x7f7cc40ca000 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006010 tx=0x7f7cb802db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:21.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc0ff9640 1 -- 172.21.15.125:0/3891345865 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80035f0 con 0x7f7cc40cf340 2023-10-18T12:01:21.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc0ff9640 1 -- 172.21.15.125:0/3891345865 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7cb8003840 con 0x7f7cc40cf340 2023-10-18T12:01:21.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc409d450 con 0x7f7cc40cf340 2023-10-18T12:01:21.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cc0ff9640 1 -- 172.21.15.125:0/3891345865 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8003c50 con 0x7f7cc40cf340 2023-10-18T12:01:21.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.180+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc409da10 con 0x7f7cc40cf340 2023-10-18T12:01:21.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.181+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc40d04f0 con 0x7f7cc40cf340 2023-10-18T12:01:21.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.182+0000 7f7cc0ff9640 1 -- 172.21.15.125:0/3891345865 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7cb803e450 con 0x7f7cc40cf340 2023-10-18T12:01:21.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.182+0000 7f7cc0ff9640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f7cb006b080 0x7f7cb006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:21.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.182+0000 7f7cc0ff9640 1 -- 172.21.15.125:0/3891345865 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f7cb80aca30 con 0x7f7cc40cf340 2023-10-18T12:01:21.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.182+0000 7f7cc3fff640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f7cb006b080 0x7f7cb006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:21.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.183+0000 7f7cc3fff640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f7cb006b080 0x7f7cb006d540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40cac90 tx=0x7f7cb400b2c0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:21.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.186+0000 7f7cc0ff9640 1 -- 172.21.15.125:0/3891345865 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7cb80773e0 con 0x7f7cc40cf340 2023-10-18T12:01:21.388 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.385+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7cc40690e0 con 0x7f7cc40cf340 2023-10-18T12:01:21.460 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.457+0000 7f7cc0ff9640 1 -- 172.21.15.125:0/3891345865 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v102) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7cb8003db0 con 0x7f7cc40cf340 2023-10-18T12:01:21.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f7cb006b080 msgr2=0x7f7cb006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f7cb006b080 0x7f7cb006d540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40cac90 tx=0x7f7cb400b2c0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.463 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc40cf340 msgr2=0x7f7cc40ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc40cf340 0x7f7cc40ca000 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006010 tx=0x7f7cb802db30 comp rx=0 tx=0).stop 2023-10-18T12:01:21.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 shutdown_connections 2023-10-18T12:01:21.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f7cb006b080 0x7f7cb006d540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cc415b2a0 0x7f7cc40ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7cc40cf340 0x7f7cc40ca000 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 --2- 172.21.15.125:0/3891345865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40ceb40 0x7f7cc409e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 >> 172.21.15.125:0/3891345865 conn(0x7f7cc40d0d60 msgr2=0x7f7cc40ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:21.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 shutdown_connections 2023-10-18T12:01:21.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.460+0000 7f7cca3fc640 1 -- 172.21.15.125:0/3891345865 wait complete. 2023-10-18T12:01:21.474 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T12:01:21.474 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-18T12:01:21.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.576+0000 7fb023da9640 1 -- 172.21.15.125:0/1721447661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c13e520 msgr2=0x7fb01c06bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.576+0000 7fb023da9640 1 --2- 172.21.15.125:0/1721447661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c13e520 0x7fb01c06bb60 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7fb018009830 tx=0x7fb01802bcd0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.577+0000 7fb023da9640 1 -- 172.21.15.125:0/1721447661 shutdown_connections 2023-10-18T12:01:21.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.577+0000 7fb023da9640 1 --2- 172.21.15.125:0/1721447661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c13e520 0x7fb01c06bb60 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.577+0000 7fb023da9640 1 -- 172.21.15.125:0/1721447661 >> 172.21.15.125:0/1721447661 conn(0x7fb01c069930 msgr2=0x7fb01c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:21.580 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.577+0000 7fb023da9640 1 -- 172.21.15.125:0/1721447661 shutdown_connections 2023-10-18T12:01:21.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.577+0000 7fb023da9640 1 -- 172.21.15.125:0/1721447661 wait complete. 2023-10-18T12:01:21.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.578+0000 7fb023da9640 1 Processor -- start 2023-10-18T12:01:21.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.578+0000 7fb023da9640 1 -- start start 2023-10-18T12:01:21.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.578+0000 7fb023da9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c070af0 0x7fb01c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:21.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.578+0000 7fb023da9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb01c13e520 0x7fb01c1e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:21.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.578+0000 7fb023da9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c141690 0x7fb01c1e0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:21.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.578+0000 7fb023da9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1e34e0 con 0x7fb01c141690 2023-10-18T12:01:21.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.578+0000 7fb023da9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1e0e80 con 0x7fb01c070af0 2023-10-18T12:01:21.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.578+0000 7fb023da9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1e1160 con 0x7fb01c13e520 2023-10-18T12:01:21.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c141690 0x7fb01c1e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:21.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c141690 0x7fb01c1e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33102/0 (socket says 172.21.15.125:33102) 2023-10-18T12:01:21.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 -- 172.21.15.125:0/3814330432 learned_addr learned my addr 172.21.15.125:0/3814330432 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:21.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb021b45640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c070af0 0x7fb01c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:21.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb021344640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb01c13e520 0x7fb01c1e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:21.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 -- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb01c13e520 msgr2=0x7fb01c1e1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb01c13e520 0x7fb01c1e1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 -- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c070af0 msgr2=0x7fb01c1556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:21.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c070af0 0x7fb01c1556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:21.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 -- 172.21.15.125:0/3814330432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb018009490 con 0x7fb01c141690 2023-10-18T12:01:21.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb022346640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c141690 0x7fb01c1e0510 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7fb01400c780 tx=0x7fb01400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:21.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb0127fc640 1 -- 172.21.15.125:0/3814330432 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014004b60 con 0x7fb01c141690 2023-10-18T12:01:21.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb0127fc640 1 -- 172.21.15.125:0/3814330432 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb01400ce90 con 0x7fb01c141690 2023-10-18T12:01:21.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb0127fc640 1 -- 172.21.15.125:0/3814330432 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400aab0 con 0x7fb01c141690 2023-10-18T12:01:21.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c1e1440 con 0x7fb01c141690 2023-10-18T12:01:21.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.579+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c1e1a00 con 0x7fb01c141690 2023-10-18T12:01:21.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.580+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c0b1bd0 con 0x7fb01c141690 2023-10-18T12:01:21.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.580+0000 7fb0127fc640 1 -- 172.21.15.125:0/3814330432 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb01400ac10 con 0x7fb01c141690 2023-10-18T12:01:21.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.581+0000 7fb0127fc640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb00006b080 0x7fb00006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:21.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.581+0000 7fb0127fc640 1 -- 172.21.15.125:0/3814330432 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fb01408a350 con 0x7fb01c141690 2023-10-18T12:01:21.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.581+0000 7fb021b45640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb00006b080 0x7fb00006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:21.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.582+0000 7fb021b45640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb00006b080 0x7fb00006d540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb018036000 tx=0x7fb018004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:21.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.585+0000 7fb0127fc640 1 -- 172.21.15.125:0/3814330432 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb014054bd0 con 0x7fb01c141690 2023-10-18T12:01:21.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:21.784+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb01c0690e0 con 0x7fb01c141690 2023-10-18T12:01:22.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.462+0000 7fb0127fc640 1 -- 172.21.15.125:0/3814330432 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v103) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb0140581f0 con 0x7fb01c141690 2023-10-18T12:01:22.467 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb00006b080 msgr2=0x7fb00006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb00006b080 0x7fb00006d540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb018036000 tx=0x7fb018004730 comp rx=0 tx=0).stop 2023-10-18T12:01:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c141690 msgr2=0x7fb01c1e0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c141690 0x7fb01c1e0510 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7fb01400c780 tx=0x7fb01400cc40 comp rx=0 tx=0).stop 2023-10-18T12:01:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 shutdown_connections 2023-10-18T12:01:22.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fb00006b080 0x7fb00006d540 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb01c141690 0x7fb01c1e0510 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb01c13e520 0x7fb01c1e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 --2- 172.21.15.125:0/3814330432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c070af0 0x7fb01c1556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 >> 172.21.15.125:0/3814330432 conn(0x7fb01c069930 msgr2=0x7fb01c0c9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 shutdown_connections 2023-10-18T12:01:22.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.465+0000 7fb023da9640 1 -- 172.21.15.125:0/3814330432 wait complete. 2023-10-18T12:01:22.478 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-18T12:01:22.479 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:22.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 -- 172.21.15.125:0/1980142644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c0cf340 msgr2=0x7f484c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 --2- 172.21.15.125:0/1980142644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c0cf340 0x7f484c0d1910 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f484c06c850 tx=0x7f484402c1d0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 -- 172.21.15.125:0/1980142644 shutdown_connections 2023-10-18T12:01:22.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 --2- 172.21.15.125:0/1980142644 >> v2:172.21.15.139:3300/0 conn(0x7f484c0ceb40 0x7f484c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 --2- 172.21.15.125:0/1980142644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f484c15b2a0 0x7f484c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 --2- 172.21.15.125:0/1980142644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c0cf340 0x7f484c0d1910 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.584 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 -- 172.21.15.125:0/1980142644 >> 172.21.15.125:0/1980142644 conn(0x7f484c0d0d60 msgr2=0x7f484c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:22.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 -- 172.21.15.125:0/1980142644 shutdown_connections 2023-10-18T12:01:22.585 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.580+0000 7f48540fc640 1 -- 172.21.15.125:0/1980142644 wait complete. 2023-10-18T12:01:22.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.581+0000 7f48540fc640 1 Processor -- start 2023-10-18T12:01:22.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.581+0000 7f48540fc640 1 -- start start 2023-10-18T12:01:22.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f48540fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f484c0ceb40 0x7f484c09e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:22.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f48540fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f484c0cf340 0x7f484c158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:22.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f48540fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c15b2a0 0x7f484c0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:22.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f48540fc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c0b4400 con 0x7f484c15b2a0 2023-10-18T12:01:22.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f48540fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c0b46e0 con 0x7f484c0ceb40 2023-10-18T12:01:22.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f48540fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484c0b49c0 con 0x7f484c0cf340 2023-10-18T12:01:22.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f4852699640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c15b2a0 0x7f484c0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:22.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f4852699640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c15b2a0 0x7f484c0b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33122/0 (socket says 172.21.15.125:33122) 2023-10-18T12:01:22.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f4852699640 1 -- 172.21.15.125:0/2625315359 learned_addr learned my addr 172.21.15.125:0/2625315359 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:22.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f4851e98640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f484c0ceb40 0x7f484c09e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:22.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f4851697640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f484c0cf340 0x7f484c158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:22.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.582+0000 7f4852699640 1 -- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f484c0cf340 msgr2=0x7f484c158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f4852699640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f484c0cf340 0x7f484c158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f4852699640 1 -- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f484c0ceb40 msgr2=0x7f484c09e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f4852699640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f484c0ceb40 0x7f484c09e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f4852699640 1 -- 172.21.15.125:0/2625315359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4844009900 con 0x7f484c15b2a0 2023-10-18T12:01:22.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f4852699640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c15b2a0 0x7f484c0b59a0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f484800d650 tx=0x7f484800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:22.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f48427fc640 1 -- 172.21.15.125:0/2625315359 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848005220 con 0x7f484c15b2a0 2023-10-18T12:01:22.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f48427fc640 1 -- 172.21.15.125:0/2625315359 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4848005490 con 0x7f484c15b2a0 2023-10-18T12:01:22.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f48427fc640 1 -- 172.21.15.125:0/2625315359 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848009a80 con 0x7f484c15b2a0 2023-10-18T12:01:22.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c0b4ca0 con 0x7f484c15b2a0 2023-10-18T12:01:22.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.583+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c09ea10 con 0x7f484c15b2a0 2023-10-18T12:01:22.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.584+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c0b51c0 con 0x7f484c15b2a0 2023-10-18T12:01:22.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.584+0000 7f48427fc640 1 -- 172.21.15.125:0/2625315359 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4848009be0 con 0x7f484c15b2a0 2023-10-18T12:01:22.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.585+0000 7f48427fc640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f483006b080 0x7f483006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:22.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.585+0000 7f48427fc640 1 -- 172.21.15.125:0/2625315359 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f484808a0d0 con 0x7f484c15b2a0 2023-10-18T12:01:22.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.585+0000 7f4851e98640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f483006b080 0x7f483006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:22.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.585+0000 7f4851e98640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f483006b080 0x7f483006d540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4844036000 tx=0x7f4844004ab0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:22.594 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.589+0000 7f48427fc640 1 -- 172.21.15.125:0/2625315359 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48480548f0 con 0x7f484c15b2a0 2023-10-18T12:01:22.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.787+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f484c0690e0 con 0x7f484c15b2a0 2023-10-18T12:01:22.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.788+0000 7f48427fc640 1 -- 172.21.15.125:0/2625315359 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+6157 (secure 0 0 0) 0x7f4848057f10 con 0x7f484c15b2a0 2023-10-18T12:01:22.791 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:22.791 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":103,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:08.447292+0000","modified":"2023-10-18T12:01:10.725386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12771":{"gid":12771,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1851256737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1851256737},{"type":"v1","addr":"172.21.15.125:6835","nonce":1851256737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:22.792 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 103 2023-10-18T12:01:22.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f483006b080 msgr2=0x7f483006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f483006b080 0x7f483006d540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4844036000 tx=0x7f4844004ab0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c15b2a0 msgr2=0x7f484c0b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c15b2a0 0x7f484c0b59a0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f484800d650 tx=0x7f484800db10 comp rx=0 tx=0).stop 2023-10-18T12:01:22.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 shutdown_connections 2023-10-18T12:01:22.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f483006b080 0x7f483006d540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f484c15b2a0 0x7f484c0b59a0 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f484c0cf340 0x7f484c158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 --2- 172.21.15.125:0/2625315359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f484c0ceb40 0x7f484c09e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 >> 172.21.15.125:0/2625315359 conn(0x7f484c0d0d60 msgr2=0x7f484c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:22.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 shutdown_connections 2023-10-18T12:01:22.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.790+0000 7f48540fc640 1 -- 172.21.15.125:0/2625315359 wait complete. 2023-10-18T12:01:22.803 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:22.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.903+0000 7f0efd621640 1 -- 172.21.15.125:0/3239520200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80b24d0 msgr2=0x7f0ef80b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.903+0000 7f0efd621640 1 --2- 172.21.15.125:0/3239520200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80b24d0 0x7f0ef80b28b0 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8070870 tx=0x7f0ee802c1d0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.904+0000 7f0efd621640 1 -- 172.21.15.125:0/3239520200 shutdown_connections 2023-10-18T12:01:22.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.904+0000 7f0efd621640 1 --2- 172.21.15.125:0/3239520200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ef80acd40 0x7f0ef81595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.904+0000 7f0efd621640 1 --2- 172.21.15.125:0/3239520200 >> v2:172.21.15.139:3300/0 conn(0x7f0ef80b3070 0x7f0ef81590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.904+0000 7f0efd621640 1 --2- 172.21.15.125:0/3239520200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80b24d0 0x7f0ef80b28b0 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.904+0000 7f0efd621640 1 -- 172.21.15.125:0/3239520200 >> 172.21.15.125:0/3239520200 conn(0x7f0ef80b3f00 msgr2=0x7f0ef80b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:22.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.904+0000 7f0efd621640 1 -- 172.21.15.125:0/3239520200 shutdown_connections 2023-10-18T12:01:22.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.904+0000 7f0efd621640 1 -- 172.21.15.125:0/3239520200 wait complete. 2023-10-18T12:01:22.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.905+0000 7f0efd621640 1 Processor -- start 2023-10-18T12:01:22.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.905+0000 7f0efd621640 1 -- start start 2023-10-18T12:01:22.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.905+0000 7f0efd621640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80acd40 0x7f0ef80992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:22.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0efd621640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ef80b24d0 0x7f0ef8099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:22.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0efd621640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef80b3070 0x7f0ef80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:22.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0efd621640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80ce290 con 0x7f0ef80acd40 2023-10-18T12:01:22.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0efd621640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80ce570 con 0x7f0ef80b3070 2023-10-18T12:01:22.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0efd621640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80ce850 con 0x7f0ef80b24d0 2023-10-18T12:01:22.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80acd40 0x7f0ef80992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:22.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80acd40 0x7f0ef80992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33134/0 (socket says 172.21.15.125:33134) 2023-10-18T12:01:22.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 -- 172.21.15.125:0/406048846 learned_addr learned my addr 172.21.15.125:0/406048846 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:22.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef67fc640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ef80b24d0 0x7f0ef8099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:22.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef77fe640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef80b3070 0x7f0ef80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:22.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 -- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ef80b24d0 msgr2=0x7f0ef8099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ef80b24d0 0x7f0ef8099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 -- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef80b3070 msgr2=0x7f0ef80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:22.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef80b3070 0x7f0ef80cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:22.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 -- 172.21.15.125:0/406048846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee8009900 con 0x7f0ef80acd40 2023-10-18T12:01:22.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef67fc640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ef80b24d0 0x7f0ef8099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:22.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0ef6ffd640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80acd40 0x7f0ef80992c0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8000f80 tx=0x7f0ee8035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:22.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0edf7fe640 1 -- 172.21.15.125:0/406048846 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8034440 con 0x7f0ef80acd40 2023-10-18T12:01:22.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0edf7fe640 1 -- 172.21.15.125:0/406048846 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ee8004540 con 0x7f0ef80acd40 2023-10-18T12:01:22.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef80ceb30 con 0x7f0ef80acd40 2023-10-18T12:01:22.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0edf7fe640 1 -- 172.21.15.125:0/406048846 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee803f440 con 0x7f0ef80acd40 2023-10-18T12:01:22.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.906+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef80cf310 con 0x7f0ef80acd40 2023-10-18T12:01:22.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.907+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef80690e0 con 0x7f0ef80acd40 2023-10-18T12:01:22.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.908+0000 7f0edf7fe640 1 -- 172.21.15.125:0/406048846 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0ee803b040 con 0x7f0ef80acd40 2023-10-18T12:01:22.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.908+0000 7f0edf7fe640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0ee406b080 0x7f0ee406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:22.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.908+0000 7f0edf7fe640 1 -- 172.21.15.125:0/406048846 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f0ee80accc0 con 0x7f0ef80acd40 2023-10-18T12:01:22.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.908+0000 7f0ef67fc640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0ee406b080 0x7f0ee406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:22.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.909+0000 7f0ef67fc640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0ee406b080 0x7f0ee406d540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80b34c0 tx=0x7f0eec005db0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:22.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:22.912+0000 7f0edf7fe640 1 -- 172.21.15.125:0/406048846 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ee80775c0 con 0x7f0ef80acd40 2023-10-18T12:01:23.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.078+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ef815be50 con 0x7f0ef80acd40 2023-10-18T12:01:23.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.080+0000 7f0edf7fe640 1 -- 172.21.15.125:0/406048846 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v246) v1 ==== 74+0+20582 (secure 0 0 0) 0x7f0ee8043090 con 0x7f0ef80acd40 2023-10-18T12:01:23.083 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:23.083 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":246,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:21.453468+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:23.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0ee406b080 msgr2=0x7f0ee406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0ee406b080 0x7f0ee406d540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80b34c0 tx=0x7f0eec005db0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80acd40 msgr2=0x7f0ef80992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80acd40 0x7f0ef80992c0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8000f80 tx=0x7f0ee8035e30 comp rx=0 tx=0).stop 2023-10-18T12:01:23.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 shutdown_connections 2023-10-18T12:01:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0ee406b080 0x7f0ee406d540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ef80b3070 0x7f0ef80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ef80b24d0 0x7f0ef8099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 --2- 172.21.15.125:0/406048846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ef80acd40 0x7f0ef80992c0 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 >> 172.21.15.125:0/406048846 conn(0x7f0ef80b3f00 msgr2=0x7f0ef8155820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 shutdown_connections 2023-10-18T12:01:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.082+0000 7f0efd621640 1 -- 172.21.15.125:0/406048846 wait complete. 2023-10-18T12:01:23.095 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T12:01:23.096 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:01:23.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.194+0000 7fe870705640 1 -- 172.21.15.125:0/3343180043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 msgr2=0x7fe86806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.194+0000 7fe870705640 1 --2- 172.21.15.125:0/3343180043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 0x7fe86806bb60 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fe864009830 tx=0x7fe86402bcd0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.195+0000 7fe870705640 1 -- 172.21.15.125:0/3343180043 shutdown_connections 2023-10-18T12:01:23.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.195+0000 7fe870705640 1 --2- 172.21.15.125:0/3343180043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 0x7fe86806bb60 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.195+0000 7fe870705640 1 -- 172.21.15.125:0/3343180043 >> 172.21.15.125:0/3343180043 conn(0x7fe868069930 msgr2=0x7fe868069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:23.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.195+0000 7fe870705640 1 -- 172.21.15.125:0/3343180043 shutdown_connections 2023-10-18T12:01:23.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.195+0000 7fe870705640 1 -- 172.21.15.125:0/3343180043 wait complete. 2023-10-18T12:01:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe870705640 1 Processor -- start 2023-10-18T12:01:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe870705640 1 -- start start 2023-10-18T12:01:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe870705640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe868070af0 0x7fe868140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe870705640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 0x7fe8681ed250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe870705640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe868141690 0x7fe86813afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe870705640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe868156ac0 con 0x7fe86813e520 2023-10-18T12:01:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe870705640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86813f9d0 con 0x7fe868141690 2023-10-18T12:01:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe870705640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86813b4e0 con 0x7fe868070af0 2023-10-18T12:01:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.196+0000 7fe86dca0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 0x7fe8681ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86dca0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 0x7fe8681ed250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33150/0 (socket says 172.21.15.125:33150) 2023-10-18T12:01:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86dca0640 1 -- 172.21.15.125:0/505434963 learned_addr learned my addr 172.21.15.125:0/505434963 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86e4a1640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe868070af0 0x7fe868140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86eca2640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe868141690 0x7fe86813afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86dca0640 1 -- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe868070af0 msgr2=0x7fe868140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86dca0640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe868070af0 0x7fe868140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86dca0640 1 -- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe868141690 msgr2=0x7fe86813afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86dca0640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe868141690 0x7fe86813afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86dca0640 1 -- 172.21.15.125:0/505434963 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe864009490 con 0x7fe86813e520 2023-10-18T12:01:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86e4a1640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe868070af0 0x7fe868140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:23.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86eca2640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe868141690 0x7fe86813afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe86dca0640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 0x7fe8681ed250 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7fe85800b3e0 tx=0x7fe85800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe85effd640 1 -- 172.21.15.125:0/505434963 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858011830 con 0x7fe86813e520 2023-10-18T12:01:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe85effd640 1 -- 172.21.15.125:0/505434963 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe858011aa0 con 0x7fe86813e520 2023-10-18T12:01:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe85effd640 1 -- 172.21.15.125:0/505434963 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858010380 con 0x7fe86813e520 2023-10-18T12:01:23.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8681561d0 con 0x7fe86813e520 2023-10-18T12:01:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.197+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe868156750 con 0x7fe86813e520 2023-10-18T12:01:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.198+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8680b1bd0 con 0x7fe86813e520 2023-10-18T12:01:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.199+0000 7fe85effd640 1 -- 172.21.15.125:0/505434963 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe8580104e0 con 0x7fe86813e520 2023-10-18T12:01:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.199+0000 7fe85effd640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe84c06b080 0x7fe84c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.199+0000 7fe85effd640 1 -- 172.21.15.125:0/505434963 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fe85808a1e0 con 0x7fe86813e520 2023-10-18T12:01:23.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.199+0000 7fe86e4a1640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe84c06b080 0x7fe84c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.200+0000 7fe86e4a1640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe84c06b080 0x7fe84c06d540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe864036000 tx=0x7fe864004730 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:23.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.203+0000 7fe85effd640 1 -- 172.21.15.125:0/505434963 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe858054ae0 con 0x7fe86813e520 2023-10-18T12:01:23.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.365+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8680690e0 con 0x7fe84c06b080 2023-10-18T12:01:23.381 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.379+0000 7fe85effd640 1 -- 172.21.15.125:0/505434963 <== mgr.9236 v2:172.21.15.139:6832/3422518215 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+523685 (secure 0 0 0) 0x7fe8680690e0 con 0x7fe84c06b080 2023-10-18T12:01:23.386 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:23.390 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:01:23.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.382+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe84c06b080 msgr2=0x7fe84c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.382+0000 7fe870705640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe84c06b080 0x7fe84c06d540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe864036000 tx=0x7fe864004730 comp rx=0 tx=0).stop 2023-10-18T12:01:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.382+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 msgr2=0x7fe8681ed250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.382+0000 7fe870705640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 0x7fe8681ed250 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7fe85800b3e0 tx=0x7fe85800b8a0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.383+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 shutdown_connections 2023-10-18T12:01:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.383+0000 7fe870705640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe84c06b080 0x7fe84c06d540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.383+0000 7fe870705640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe868141690 0x7fe86813afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.383+0000 7fe870705640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe86813e520 0x7fe8681ed250 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.383+0000 7fe870705640 1 --2- 172.21.15.125:0/505434963 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe868070af0 0x7fe868140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.383+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 >> 172.21.15.125:0/505434963 conn(0x7fe868069930 msgr2=0x7fe8680c9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:23.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.383+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 shutdown_connections 2023-10-18T12:01:23.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.383+0000 7fe870705640 1 -- 172.21.15.125:0/505434963 wait complete. 2023-10-18T12:01:23.395 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-18T12:01:22.839039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594034,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":682,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":420,"ondisk_log_size":420,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420356,"kb_used_data":25184,"kb_used_omap":12,"kb_used_meta":395059,"kb_avail":754554364,"statfs":{"total":773094113280,"available":772663668736,"internally_reserved":0,"allocated":25788416,"data_stored":13832589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":404540984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1256,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.398513"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454157+0000","last_change":"2023-10-18T12:01:19.454157+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454157+0000","last_undegraded":"2023-10-18T12:01:19.454157+0000","last_fullsized":"2023-10-18T12:01:19.454157+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465723+0000","last_change":"2023-10-18T12:01:19.464169+0000","last_active":"2023-10-18T12:01:19.465723+0000","last_peered":"2023-10-18T12:01:19.465723+0000","last_clean":"2023-10-18T12:01:19.465723+0000","last_became_active":"2023-10-18T12:01:19.463355+0000","last_became_peered":"2023-10-18T12:01:19.463355+0000","last_unstale":"2023-10-18T12:01:19.465723+0000","last_undegraded":"2023-10-18T12:01:19.465723+0000","last_fullsized":"2023-10-18T12:01:19.465723+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:00.513738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444418+0000","last_change":"2023-10-18T12:01:09.126968+0000","last_active":"2023-10-18T12:01:18.444418+0000","last_peered":"2023-10-18T12:01:18.444418+0000","last_clean":"2023-10-18T12:01:18.444418+0000","last_became_active":"2023-10-18T12:01:09.126705+0000","last_became_peered":"2023-10-18T12:01:09.126705+0000","last_unstale":"2023-10-18T12:01:18.444418+0000","last_undegraded":"2023-10-18T12:01:18.444418+0000","last_fullsized":"2023-10-18T12:01:18.444418+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:42:53.088272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453457+0000","last_change":"2023-10-18T12:01:19.453457+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453457+0000","last_undegraded":"2023-10-18T12:01:19.453457+0000","last_fullsized":"2023-10-18T12:01:19.453457+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468820+0000","last_change":"2023-10-18T12:01:19.468191+0000","last_active":"2023-10-18T12:01:19.468820+0000","last_peered":"2023-10-18T12:01:19.468820+0000","last_clean":"2023-10-18T12:01:19.468820+0000","last_became_active":"2023-10-18T12:01:19.467925+0000","last_became_peered":"2023-10-18T12:01:19.467925+0000","last_unstale":"2023-10-18T12:01:19.468820+0000","last_undegraded":"2023-10-18T12:01:19.468820+0000","last_fullsized":"2023-10-18T12:01:19.468820+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:58.124933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439901+0000","last_change":"2023-10-18T12:01:09.125772+0000","last_active":"2023-10-18T12:01:18.439901+0000","last_peered":"2023-10-18T12:01:18.439901+0000","last_clean":"2023-10-18T12:01:18.439901+0000","last_became_active":"2023-10-18T12:01:09.125508+0000","last_became_peered":"2023-10-18T12:01:09.125508+0000","last_unstale":"2023-10-18T12:01:18.439901+0000","last_undegraded":"2023-10-18T12:01:18.439901+0000","last_fullsized":"2023-10-18T12:01:18.439901+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:15.841764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451128+0000","last_change":"2023-10-18T12:01:19.451128+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451128+0000","last_undegraded":"2023-10-18T12:01:19.451128+0000","last_fullsized":"2023-10-18T12:01:19.451128+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.467291+0000","last_change":"2023-10-18T12:01:19.466687+0000","last_active":"2023-10-18T12:01:19.467291+0000","last_peered":"2023-10-18T12:01:19.467291+0000","last_clean":"2023-10-18T12:01:19.467291+0000","last_became_active":"2023-10-18T12:01:19.466395+0000","last_became_peered":"2023-10-18T12:01:19.466395+0000","last_unstale":"2023-10-18T12:01:19.467291+0000","last_undegraded":"2023-10-18T12:01:19.467291+0000","last_fullsized":"2023-10-18T12:01:19.467291+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:02.185827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444714+0000","last_change":"2023-10-18T12:01:09.124764+0000","last_active":"2023-10-18T12:01:18.444714+0000","last_peered":"2023-10-18T12:01:18.444714+0000","last_clean":"2023-10-18T12:01:18.444714+0000","last_became_active":"2023-10-18T12:01:09.124472+0000","last_became_peered":"2023-10-18T12:01:09.124472+0000","last_unstale":"2023-10-18T12:01:18.444714+0000","last_undegraded":"2023-10-18T12:01:18.444714+0000","last_fullsized":"2023-10-18T12:01:18.444714+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:29.891668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454232+0000","last_change":"2023-10-18T12:01:19.454232+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454232+0000","last_undegraded":"2023-10-18T12:01:19.454232+0000","last_fullsized":"2023-10-18T12:01:19.454232+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466648+0000","last_change":"2023-10-18T12:01:19.466181+0000","last_active":"2023-10-18T12:01:19.466648+0000","last_peered":"2023-10-18T12:01:19.466648+0000","last_clean":"2023-10-18T12:01:19.466648+0000","last_became_active":"2023-10-18T12:01:19.465901+0000","last_became_peered":"2023-10-18T12:01:19.465901+0000","last_unstale":"2023-10-18T12:01:19.466648+0000","last_undegraded":"2023-10-18T12:01:19.466648+0000","last_fullsized":"2023-10-18T12:01:19.466648+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:26:57.818590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444737+0000","last_change":"2023-10-18T12:01:09.132202+0000","last_active":"2023-10-18T12:01:18.444737+0000","last_peered":"2023-10-18T12:01:18.444737+0000","last_clean":"2023-10-18T12:01:18.444737+0000","last_became_active":"2023-10-18T12:01:09.131928+0000","last_became_peered":"2023-10-18T12:01:09.131928+0000","last_unstale":"2023-10-18T12:01:18.444737+0000","last_undegraded":"2023-10-18T12:01:18.444737+0000","last_fullsized":"2023-10-18T12:01:18.444737+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:34.826098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450577+0000","last_change":"2023-10-18T12:01:19.450577+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450577+0000","last_undegraded":"2023-10-18T12:01:19.450577+0000","last_fullsized":"2023-10-18T12:01:19.450577+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.461102+0000","last_change":"2023-10-18T12:01:19.459438+0000","last_active":"2023-10-18T12:01:19.461102+0000","last_peered":"2023-10-18T12:01:19.461102+0000","last_clean":"2023-10-18T12:01:19.461102+0000","last_became_active":"2023-10-18T12:01:19.459194+0000","last_became_peered":"2023-10-18T12:01:19.459194+0000","last_unstale":"2023-10-18T12:01:19.461102+0000","last_undegraded":"2023-10-18T12:01:19.461102+0000","last_fullsized":"2023-10-18T12:01:19.461102+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:53.933731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446379+0000","last_change":"2023-10-18T12:01:09.133870+0000","last_active":"2023-10-18T12:01:18.446379+0000","last_peered":"2023-10-18T12:01:18.446379+0000","last_clean":"2023-10-18T12:01:18.446379+0000","last_became_active":"2023-10-18T12:01:09.133478+0000","last_became_peered":"2023-10-18T12:01:09.133478+0000","last_unstale":"2023-10-18T12:01:18.446379+0000","last_undegraded":"2023-10-18T12:01:18.446379+0000","last_fullsized":"2023-10-18T12:01:18.446379+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:11.836101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450431+0000","last_change":"2023-10-18T12:01:19.450431+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450431+0000","last_undegraded":"2023-10-18T12:01:19.450431+0000","last_fullsized":"2023-10-18T12:01:19.450431+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465762+0000","last_change":"2023-10-18T12:01:19.463436+0000","last_active":"2023-10-18T12:01:19.465762+0000","last_peered":"2023-10-18T12:01:19.465762+0000","last_clean":"2023-10-18T12:01:19.465762+0000","last_became_active":"2023-10-18T12:01:19.463190+0000","last_became_peered":"2023-10-18T12:01:19.463190+0000","last_unstale":"2023-10-18T12:01:19.465762+0000","last_undegraded":"2023-10-18T12:01:19.465762+0000","last_fullsized":"2023-10-18T12:01:19.465762+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:55.180369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443988+0000","last_change":"2023-10-18T12:01:09.129113+0000","last_active":"2023-10-18T12:01:18.443988+0000","last_peered":"2023-10-18T12:01:18.443988+0000","last_clean":"2023-10-18T12:01:18.443988+0000","last_became_active":"2023-10-18T12:01:09.128785+0000","last_became_peered":"2023-10-18T12:01:09.128785+0000","last_unstale":"2023-10-18T12:01:18.443988+0000","last_undegraded":"2023-10-18T12:01:18.443988+0000","last_fullsized":"2023-10-18T12:01:18.443988+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:39.875471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458863+0000","last_change":"2023-10-18T12:01:19.458863+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458863+0000","last_undegraded":"2023-10-18T12:01:19.458863+0000","last_fullsized":"2023-10-18T12:01:19.458863+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465879+0000","last_change":"2023-10-18T12:01:19.463023+0000","last_active":"2023-10-18T12:01:19.465879+0000","last_peered":"2023-10-18T12:01:19.465879+0000","last_clean":"2023-10-18T12:01:19.465879+0000","last_became_active":"2023-10-18T12:01:19.462757+0000","last_became_peered":"2023-10-18T12:01:19.462757+0000","last_unstale":"2023-10-18T12:01:19.465879+0000","last_undegraded":"2023-10-18T12:01:19.465879+0000","last_fullsized":"2023-10-18T12:01:19.465879+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:09.123947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445459+0000","last_change":"2023-10-18T12:01:09.125901+0000","last_active":"2023-10-18T12:01:18.445459+0000","last_peered":"2023-10-18T12:01:18.445459+0000","last_clean":"2023-10-18T12:01:18.445459+0000","last_became_active":"2023-10-18T12:01:09.125390+0000","last_became_peered":"2023-10-18T12:01:09.125390+0000","last_unstale":"2023-10-18T12:01:18.445459+0000","last_undegraded":"2023-10-18T12:01:18.445459+0000","last_fullsized":"2023-10-18T12:01:18.445459+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:00.246587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451418+0000","last_change":"2023-10-18T12:01:19.451418+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451418+0000","last_undegraded":"2023-10-18T12:01:19.451418+0000","last_fullsized":"2023-10-18T12:01:19.451418+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465285+0000","last_change":"2023-10-18T12:01:19.463793+0000","last_active":"2023-10-18T12:01:19.465285+0000","last_peered":"2023-10-18T12:01:19.465285+0000","last_clean":"2023-10-18T12:01:19.465285+0000","last_became_active":"2023-10-18T12:01:19.463343+0000","last_became_peered":"2023-10-18T12:01:19.463343+0000","last_unstale":"2023-10-18T12:01:19.465285+0000","last_undegraded":"2023-10-18T12:01:19.465285+0000","last_fullsized":"2023-10-18T12:01:19.465285+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:34.979933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446196+0000","last_change":"2023-10-18T12:01:09.135081+0000","last_active":"2023-10-18T12:01:18.446196+0000","last_peered":"2023-10-18T12:01:18.446196+0000","last_clean":"2023-10-18T12:01:18.446196+0000","last_became_active":"2023-10-18T12:01:09.134801+0000","last_became_peered":"2023-10-18T12:01:09.134801+0000","last_unstale":"2023-10-18T12:01:18.446196+0000","last_undegraded":"2023-10-18T12:01:18.446196+0000","last_fullsized":"2023-10-18T12:01:18.446196+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:03.262316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452320+0000","last_change":"2023-10-18T12:01:19.452320+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452320+0000","last_undegraded":"2023-10-18T12:01:19.452320+0000","last_fullsized":"2023-10-18T12:01:19.452320+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466738+0000","last_change":"2023-10-18T12:01:19.465635+0000","last_active":"2023-10-18T12:01:19.466738+0000","last_peered":"2023-10-18T12:01:19.466738+0000","last_clean":"2023-10-18T12:01:19.466738+0000","last_became_active":"2023-10-18T12:01:19.465363+0000","last_became_peered":"2023-10-18T12:01:19.465363+0000","last_unstale":"2023-10-18T12:01:19.466738+0000","last_undegraded":"2023-10-18T12:01:19.466738+0000","last_fullsized":"2023-10-18T12:01:19.466738+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:56.587842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444490+0000","last_change":"2023-10-18T12:01:09.135615+0000","last_active":"2023-10-18T12:01:18.444490+0000","last_peered":"2023-10-18T12:01:18.444490+0000","last_clean":"2023-10-18T12:01:18.444490+0000","last_became_active":"2023-10-18T12:01:09.135123+0000","last_became_peered":"2023-10-18T12:01:09.135123+0000","last_unstale":"2023-10-18T12:01:18.444490+0000","last_undegraded":"2023-10-18T12:01:18.444490+0000","last_fullsized":"2023-10-18T12:01:18.444490+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:52.311354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458411+0000","last_change":"2023-10-18T12:01:19.458411+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458411+0000","last_undegraded":"2023-10-18T12:01:19.458411+0000","last_fullsized":"2023-10-18T12:01:19.458411+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465526+0000","last_change":"2023-10-18T12:01:19.465001+0000","last_active":"2023-10-18T12:01:19.465526+0000","last_peered":"2023-10-18T12:01:19.465526+0000","last_clean":"2023-10-18T12:01:19.465526+0000","last_became_active":"2023-10-18T12:01:19.464719+0000","last_became_peered":"2023-10-18T12:01:19.464719+0000","last_unstale":"2023-10-18T12:01:19.465526+0000","last_undegraded":"2023-10-18T12:01:19.465526+0000","last_fullsized":"2023-10-18T12:01:19.465526+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:08:12.543232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446023+0000","last_change":"2023-10-18T12:01:09.130673+0000","last_active":"2023-10-18T12:01:18.446023+0000","last_peered":"2023-10-18T12:01:18.446023+0000","last_clean":"2023-10-18T12:01:18.446023+0000","last_became_active":"2023-10-18T12:01:09.130094+0000","last_became_peered":"2023-10-18T12:01:09.130094+0000","last_unstale":"2023-10-18T12:01:18.446023+0000","last_undegraded":"2023-10-18T12:01:18.446023+0000","last_fullsized":"2023-10-18T12:01:18.446023+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:45.652023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456984+0000","last_change":"2023-10-18T12:01:19.456984+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456984+0000","last_undegraded":"2023-10-18T12:01:19.456984+0000","last_fullsized":"2023-10-18T12:01:19.456984+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465010+0000","last_change":"2023-10-18T12:01:19.463067+0000","last_active":"2023-10-18T12:01:19.465010+0000","last_peered":"2023-10-18T12:01:19.465010+0000","last_clean":"2023-10-18T12:01:19.465010+0000","last_became_active":"2023-10-18T12:01:19.462818+0000","last_became_peered":"2023-10-18T12:01:19.462818+0000","last_unstale":"2023-10-18T12:01:19.465010+0000","last_undegraded":"2023-10-18T12:01:19.465010+0000","last_fullsized":"2023-10-18T12:01:19.465010+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:50:23.807977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444802+0000","last_change":"2023-10-18T12:01:09.135828+0000","last_active":"2023-10-18T12:01:18.444802+0000","last_peered":"2023-10-18T12:01:18.444802+0000","last_clean":"2023-10-18T12:01:18.444802+0000","last_became_active":"2023-10-18T12:01:09.135579+0000","last_became_peered":"2023-10-18T12:01:09.135579+0000","last_unstale":"2023-10-18T12:01:18.444802+0000","last_undegraded":"2023-10-18T12:01:18.444802+0000","last_fullsized":"2023-10-18T12:01:18.444802+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:06.517585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454999+0000","last_change":"2023-10-18T12:01:19.454999+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454999+0000","last_undegraded":"2023-10-18T12:01:19.454999+0000","last_fullsized":"2023-10-18T12:01:19.454999+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.464639+0000","last_change":"2023-10-18T12:01:19.462716+0000","last_active":"2023-10-18T12:01:19.464639+0000","last_peered":"2023-10-18T12:01:19.464639+0000","last_clean":"2023-10-18T12:01:19.464639+0000","last_became_active":"2023-10-18T12:01:19.462474+0000","last_became_peered":"2023-10-18T12:01:19.462474+0000","last_unstale":"2023-10-18T12:01:19.464639+0000","last_undegraded":"2023-10-18T12:01:19.464639+0000","last_fullsized":"2023-10-18T12:01:19.464639+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:04.329561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443892+0000","last_change":"2023-10-18T12:01:09.132939+0000","last_active":"2023-10-18T12:01:18.443892+0000","last_peered":"2023-10-18T12:01:18.443892+0000","last_clean":"2023-10-18T12:01:18.443892+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:18.443892+0000","last_undegraded":"2023-10-18T12:01:18.443892+0000","last_fullsized":"2023-10-18T12:01:18.443892+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:22.550156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455867+0000","last_change":"2023-10-18T12:01:19.455867+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455867+0000","last_undegraded":"2023-10-18T12:01:19.455867+0000","last_fullsized":"2023-10-18T12:01:19.455867+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.459783+0000","last_change":"2023-10-18T12:01:19.458472+0000","last_active":"2023-10-18T12:01:19.459783+0000","last_peered":"2023-10-18T12:01:19.459783+0000","last_clean":"2023-10-18T12:01:19.459783+0000","last_became_active":"2023-10-18T12:01:19.458209+0000","last_became_peered":"2023-10-18T12:01:19.458209+0000","last_unstale":"2023-10-18T12:01:19.459783+0000","last_undegraded":"2023-10-18T12:01:19.459783+0000","last_fullsized":"2023-10-18T12:01:19.459783+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:04.778253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444252+0000","last_change":"2023-10-18T12:01:09.129764+0000","last_active":"2023-10-18T12:01:18.444252+0000","last_peered":"2023-10-18T12:01:18.444252+0000","last_clean":"2023-10-18T12:01:18.444252+0000","last_became_active":"2023-10-18T12:01:09.129483+0000","last_became_peered":"2023-10-18T12:01:09.129483+0000","last_unstale":"2023-10-18T12:01:18.444252+0000","last_undegraded":"2023-10-18T12:01:18.444252+0000","last_fullsized":"2023-10-18T12:01:18.444252+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:03.037837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453014+0000","last_change":"2023-10-18T12:01:19.453014+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453014+0000","last_undegraded":"2023-10-18T12:01:19.453014+0000","last_fullsized":"2023-10-18T12:01:19.453014+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468149+0000","last_change":"2023-10-18T12:01:19.466751+0000","last_active":"2023-10-18T12:01:19.468149+0000","last_peered":"2023-10-18T12:01:19.468149+0000","last_clean":"2023-10-18T12:01:19.468149+0000","last_became_active":"2023-10-18T12:01:19.466171+0000","last_became_peered":"2023-10-18T12:01:19.466171+0000","last_unstale":"2023-10-18T12:01:19.468149+0000","last_undegraded":"2023-10-18T12:01:19.468149+0000","last_fullsized":"2023-10-18T12:01:19.468149+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:01.102496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448554+0000","last_change":"2023-10-18T12:01:09.133024+0000","last_active":"2023-10-18T12:01:18.448554+0000","last_peered":"2023-10-18T12:01:18.448554+0000","last_clean":"2023-10-18T12:01:18.448554+0000","last_became_active":"2023-10-18T12:01:09.132742+0000","last_became_peered":"2023-10-18T12:01:09.132742+0000","last_unstale":"2023-10-18T12:01:18.448554+0000","last_undegraded":"2023-10-18T12:01:18.448554+0000","last_fullsized":"2023-10-18T12:01:18.448554+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456037+0000","last_change":"2023-10-18T12:01:19.456037+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456037+0000","last_undegraded":"2023-10-18T12:01:19.456037+0000","last_fullsized":"2023-10-18T12:01:19.456037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465243+0000","last_change":"2023-10-18T12:01:19.464582+0000","last_active":"2023-10-18T12:01:19.465243+0000","last_peered":"2023-10-18T12:01:19.465243+0000","last_clean":"2023-10-18T12:01:19.465243+0000","last_became_active":"2023-10-18T12:01:19.464325+0000","last_became_peered":"2023-10-18T12:01:19.464325+0000","last_unstale":"2023-10-18T12:01:19.465243+0000","last_undegraded":"2023-10-18T12:01:19.465243+0000","last_fullsized":"2023-10-18T12:01:19.465243+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:32:16.943070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445969+0000","last_change":"2023-10-18T12:01:09.131147+0000","last_active":"2023-10-18T12:01:18.445969+0000","last_peered":"2023-10-18T12:01:18.445969+0000","last_clean":"2023-10-18T12:01:18.445969+0000","last_became_active":"2023-10-18T12:01:09.130682+0000","last_became_peered":"2023-10-18T12:01:09.130682+0000","last_unstale":"2023-10-18T12:01:18.445969+0000","last_undegraded":"2023-10-18T12:01:18.445969+0000","last_fullsized":"2023-10-18T12:01:18.445969+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453907+0000","last_change":"2023-10-18T12:01:19.453907+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453907+0000","last_undegraded":"2023-10-18T12:01:19.453907+0000","last_fullsized":"2023-10-18T12:01:19.453907+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463821+0000","last_change":"2023-10-18T12:01:19.459426+0000","last_active":"2023-10-18T12:01:19.463821+0000","last_peered":"2023-10-18T12:01:19.463821+0000","last_clean":"2023-10-18T12:01:19.463821+0000","last_became_active":"2023-10-18T12:01:19.459168+0000","last_became_peered":"2023-10-18T12:01:19.459168+0000","last_unstale":"2023-10-18T12:01:19.463821+0000","last_undegraded":"2023-10-18T12:01:19.463821+0000","last_fullsized":"2023-10-18T12:01:19.463821+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:00.882760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445455+0000","last_change":"2023-10-18T12:01:09.122098+0000","last_active":"2023-10-18T12:01:18.445455+0000","last_peered":"2023-10-18T12:01:18.445455+0000","last_clean":"2023-10-18T12:01:18.445455+0000","last_became_active":"2023-10-18T12:01:09.121845+0000","last_became_peered":"2023-10-18T12:01:09.121845+0000","last_unstale":"2023-10-18T12:01:18.445455+0000","last_undegraded":"2023-10-18T12:01:18.445455+0000","last_fullsized":"2023-10-18T12:01:18.445455+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:19.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453747+0000","last_change":"2023-10-18T12:01:19.453747+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453747+0000","last_undegraded":"2023-10-18T12:01:19.453747+0000","last_fullsized":"2023-10-18T12:01:19.453747+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468474+0000","last_change":"2023-10-18T12:01:19.466123+0000","last_active":"2023-10-18T12:01:19.468474+0000","last_peered":"2023-10-18T12:01:19.468474+0000","last_clean":"2023-10-18T12:01:19.468474+0000","last_became_active":"2023-10-18T12:01:19.465856+0000","last_became_peered":"2023-10-18T12:01:19.465856+0000","last_unstale":"2023-10-18T12:01:19.468474+0000","last_undegraded":"2023-10-18T12:01:19.468474+0000","last_fullsized":"2023-10-18T12:01:19.468474+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:53.379907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447103+0000","last_change":"2023-10-18T12:01:09.133999+0000","last_active":"2023-10-18T12:01:18.447103+0000","last_peered":"2023-10-18T12:01:18.447103+0000","last_clean":"2023-10-18T12:01:18.447103+0000","last_became_active":"2023-10-18T12:01:09.133725+0000","last_became_peered":"2023-10-18T12:01:09.133725+0000","last_unstale":"2023-10-18T12:01:18.447103+0000","last_undegraded":"2023-10-18T12:01:18.447103+0000","last_fullsized":"2023-10-18T12:01:18.447103+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:35.006911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459506+0000","last_change":"2023-10-18T12:01:19.459506+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459506+0000","last_undegraded":"2023-10-18T12:01:19.459506+0000","last_fullsized":"2023-10-18T12:01:19.459506+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468784+0000","last_change":"2023-10-18T12:01:19.468308+0000","last_active":"2023-10-18T12:01:19.468784+0000","last_peered":"2023-10-18T12:01:19.468784+0000","last_clean":"2023-10-18T12:01:19.468784+0000","last_became_active":"2023-10-18T12:01:19.467998+0000","last_became_peered":"2023-10-18T12:01:19.467998+0000","last_unstale":"2023-10-18T12:01:19.468784+0000","last_undegraded":"2023-10-18T12:01:19.468784+0000","last_fullsized":"2023-10-18T12:01:19.468784+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:17.064462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446111+0000","last_change":"2023-10-18T12:01:09.124908+0000","last_active":"2023-10-18T12:01:18.446111+0000","last_peered":"2023-10-18T12:01:18.446111+0000","last_clean":"2023-10-18T12:01:18.446111+0000","last_became_active":"2023-10-18T12:01:09.124565+0000","last_became_peered":"2023-10-18T12:01:09.124565+0000","last_unstale":"2023-10-18T12:01:18.446111+0000","last_undegraded":"2023-10-18T12:01:18.446111+0000","last_fullsized":"2023-10-18T12:01:18.446111+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:22.044677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453595+0000","last_change":"2023-10-18T12:01:19.453595+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453595+0000","last_undegraded":"2023-10-18T12:01:19.453595+0000","last_fullsized":"2023-10-18T12:01:19.453595+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468762+0000","last_change":"2023-10-18T12:01:19.467971+0000","last_active":"2023-10-18T12:01:19.468762+0000","last_peered":"2023-10-18T12:01:19.468762+0000","last_clean":"2023-10-18T12:01:19.468762+0000","last_became_active":"2023-10-18T12:01:19.467699+0000","last_became_peered":"2023-10-18T12:01:19.467699+0000","last_unstale":"2023-10-18T12:01:19.468762+0000","last_undegraded":"2023-10-18T12:01:19.468762+0000","last_fullsized":"2023-10-18T12:01:19.468762+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:01.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447225+0000","last_change":"2023-10-18T12:01:09.135302+0000","last_active":"2023-10-18T12:01:18.447225+0000","last_peered":"2023-10-18T12:01:18.447225+0000","last_clean":"2023-10-18T12:01:18.447225+0000","last_became_active":"2023-10-18T12:01:09.134978+0000","last_became_peered":"2023-10-18T12:01:09.134978+0000","last_unstale":"2023-10-18T12:01:18.447225+0000","last_undegraded":"2023-10-18T12:01:18.447225+0000","last_fullsized":"2023-10-18T12:01:18.447225+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452783+0000","last_change":"2023-10-18T12:01:19.452783+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452783+0000","last_undegraded":"2023-10-18T12:01:19.452783+0000","last_fullsized":"2023-10-18T12:01:19.452783+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463777+0000","last_change":"2023-10-18T12:01:19.461558+0000","last_active":"2023-10-18T12:01:19.463777+0000","last_peered":"2023-10-18T12:01:19.463777+0000","last_clean":"2023-10-18T12:01:19.463777+0000","last_became_active":"2023-10-18T12:01:19.461307+0000","last_became_peered":"2023-10-18T12:01:19.461307+0000","last_unstale":"2023-10-18T12:01:19.463777+0000","last_undegraded":"2023-10-18T12:01:19.463777+0000","last_fullsized":"2023-10-18T12:01:19.463777+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:18:26.034701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440010+0000","last_change":"2023-10-18T12:01:09.136717+0000","last_active":"2023-10-18T12:01:18.440010+0000","last_peered":"2023-10-18T12:01:18.440010+0000","last_clean":"2023-10-18T12:01:18.440010+0000","last_became_active":"2023-10-18T12:01:09.136461+0000","last_became_peered":"2023-10-18T12:01:09.136461+0000","last_unstale":"2023-10-18T12:01:18.440010+0000","last_undegraded":"2023-10-18T12:01:18.440010+0000","last_fullsized":"2023-10-18T12:01:18.440010+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:02.336311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452844+0000","last_change":"2023-10-18T12:01:19.452844+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452844+0000","last_undegraded":"2023-10-18T12:01:19.452844+0000","last_fullsized":"2023-10-18T12:01:19.452844+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463533+0000","last_change":"2023-10-18T12:01:19.462240+0000","last_active":"2023-10-18T12:01:19.463533+0000","last_peered":"2023-10-18T12:01:19.463533+0000","last_clean":"2023-10-18T12:01:19.463533+0000","last_became_active":"2023-10-18T12:01:19.461957+0000","last_became_peered":"2023-10-18T12:01:19.461957+0000","last_unstale":"2023-10-18T12:01:19.463533+0000","last_undegraded":"2023-10-18T12:01:19.463533+0000","last_fullsized":"2023-10-18T12:01:19.463533+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:20.506577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445002+0000","last_change":"2023-10-18T12:01:09.130016+0000","last_active":"2023-10-18T12:01:18.445002+0000","last_peered":"2023-10-18T12:01:18.445002+0000","last_clean":"2023-10-18T12:01:18.445002+0000","last_became_active":"2023-10-18T12:01:09.129718+0000","last_became_peered":"2023-10-18T12:01:09.129718+0000","last_unstale":"2023-10-18T12:01:18.445002+0000","last_undegraded":"2023-10-18T12:01:18.445002+0000","last_fullsized":"2023-10-18T12:01:18.445002+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:36.455465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450045+0000","last_change":"2023-10-18T12:01:19.450045+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450045+0000","last_undegraded":"2023-10-18T12:01:19.450045+0000","last_fullsized":"2023-10-18T12:01:19.450045+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468581+0000","last_change":"2023-10-18T12:01:19.467158+0000","last_active":"2023-10-18T12:01:19.468581+0000","last_peered":"2023-10-18T12:01:19.468581+0000","last_clean":"2023-10-18T12:01:19.468581+0000","last_became_active":"2023-10-18T12:01:19.466854+0000","last_became_peered":"2023-10-18T12:01:19.466854+0000","last_unstale":"2023-10-18T12:01:19.468581+0000","last_undegraded":"2023-10-18T12:01:19.468581+0000","last_fullsized":"2023-10-18T12:01:19.468581+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:21.428576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443731+0000","last_change":"2023-10-18T12:01:09.129706+0000","last_active":"2023-10-18T12:01:18.443731+0000","last_peered":"2023-10-18T12:01:18.443731+0000","last_clean":"2023-10-18T12:01:18.443731+0000","last_became_active":"2023-10-18T12:01:09.129413+0000","last_became_peered":"2023-10-18T12:01:09.129413+0000","last_unstale":"2023-10-18T12:01:18.443731+0000","last_undegraded":"2023-10-18T12:01:18.443731+0000","last_fullsized":"2023-10-18T12:01:18.443731+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:55.309809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.449803+0000","last_change":"2023-10-18T12:01:19.449803+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.449803+0000","last_undegraded":"2023-10-18T12:01:19.449803+0000","last_fullsized":"2023-10-18T12:01:19.449803+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468823+0000","last_change":"2023-10-18T12:01:19.467623+0000","last_active":"2023-10-18T12:01:19.468823+0000","last_peered":"2023-10-18T12:01:19.468823+0000","last_clean":"2023-10-18T12:01:19.468823+0000","last_became_active":"2023-10-18T12:01:19.467358+0000","last_became_peered":"2023-10-18T12:01:19.467358+0000","last_unstale":"2023-10-18T12:01:19.468823+0000","last_undegraded":"2023-10-18T12:01:19.468823+0000","last_fullsized":"2023-10-18T12:01:19.468823+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:58.681098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446063+0000","last_change":"2023-10-18T12:01:09.133291+0000","last_active":"2023-10-18T12:01:18.446063+0000","last_peered":"2023-10-18T12:01:18.446063+0000","last_clean":"2023-10-18T12:01:18.446063+0000","last_became_active":"2023-10-18T12:01:09.132968+0000","last_became_peered":"2023-10-18T12:01:09.132968+0000","last_unstale":"2023-10-18T12:01:18.446063+0000","last_undegraded":"2023-10-18T12:01:18.446063+0000","last_fullsized":"2023-10-18T12:01:18.446063+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454604+0000","last_change":"2023-10-18T12:01:19.454604+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454604+0000","last_undegraded":"2023-10-18T12:01:19.454604+0000","last_fullsized":"2023-10-18T12:01:19.454604+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.460591+0000","last_change":"2023-10-18T12:01:19.458380+0000","last_active":"2023-10-18T12:01:19.460591+0000","last_peered":"2023-10-18T12:01:19.460591+0000","last_clean":"2023-10-18T12:01:19.460591+0000","last_became_active":"2023-10-18T12:01:19.458111+0000","last_became_peered":"2023-10-18T12:01:19.458111+0000","last_unstale":"2023-10-18T12:01:19.460591+0000","last_undegraded":"2023-10-18T12:01:19.460591+0000","last_fullsized":"2023-10-18T12:01:19.460591+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:27.878015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443343+0000","last_change":"2023-10-18T12:01:09.128954+0000","last_active":"2023-10-18T12:01:18.443343+0000","last_peered":"2023-10-18T12:01:18.443343+0000","last_clean":"2023-10-18T12:01:18.443343+0000","last_became_active":"2023-10-18T12:01:09.128655+0000","last_became_peered":"2023-10-18T12:01:09.128655+0000","last_unstale":"2023-10-18T12:01:18.443343+0000","last_undegraded":"2023-10-18T12:01:18.443343+0000","last_fullsized":"2023-10-18T12:01:18.443343+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:51.625573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455208+0000","last_change":"2023-10-18T12:01:19.455208+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455208+0000","last_undegraded":"2023-10-18T12:01:19.455208+0000","last_fullsized":"2023-10-18T12:01:19.455208+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468453+0000","last_change":"2023-10-18T12:01:19.466027+0000","last_active":"2023-10-18T12:01:19.468453+0000","last_peered":"2023-10-18T12:01:19.468453+0000","last_clean":"2023-10-18T12:01:19.468453+0000","last_became_active":"2023-10-18T12:01:19.465745+0000","last_became_peered":"2023-10-18T12:01:19.465745+0000","last_unstale":"2023-10-18T12:01:19.468453+0000","last_undegraded":"2023-10-18T12:01:19.468453+0000","last_fullsized":"2023-10-18T12:01:19.468453+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:15.002306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440030+0000","last_change":"2023-10-18T12:01:09.123355+0000","last_active":"2023-10-18T12:01:18.440030+0000","last_peered":"2023-10-18T12:01:18.440030+0000","last_clean":"2023-10-18T12:01:18.440030+0000","last_became_active":"2023-10-18T12:01:09.123096+0000","last_became_peered":"2023-10-18T12:01:09.123096+0000","last_unstale":"2023-10-18T12:01:18.440030+0000","last_undegraded":"2023-10-18T12:01:18.440030+0000","last_fullsized":"2023-10-18T12:01:18.440030+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:58.130314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454904+0000","last_change":"2023-10-18T12:01:19.454904+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454904+0000","last_undegraded":"2023-10-18T12:01:19.454904+0000","last_fullsized":"2023-10-18T12:01:19.454904+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468820+0000","last_change":"2023-10-18T12:01:19.467893+0000","last_active":"2023-10-18T12:01:19.468820+0000","last_peered":"2023-10-18T12:01:19.468820+0000","last_clean":"2023-10-18T12:01:19.468820+0000","last_became_active":"2023-10-18T12:01:19.467430+0000","last_became_peered":"2023-10-18T12:01:19.467430+0000","last_unstale":"2023-10-18T12:01:19.468820+0000","last_undegraded":"2023-10-18T12:01:19.468820+0000","last_fullsized":"2023-10-18T12:01:19.468820+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:12.990393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447739+0000","last_change":"2023-10-18T12:01:09.124947+0000","last_active":"2023-10-18T12:01:18.447739+0000","last_peered":"2023-10-18T12:01:18.447739+0000","last_clean":"2023-10-18T12:01:18.447739+0000","last_became_active":"2023-10-18T12:01:09.124666+0000","last_became_peered":"2023-10-18T12:01:09.124666+0000","last_unstale":"2023-10-18T12:01:18.447739+0000","last_undegraded":"2023-10-18T12:01:18.447739+0000","last_fullsized":"2023-10-18T12:01:18.447739+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:01.214167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454093+0000","last_change":"2023-10-18T12:01:19.454093+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454093+0000","last_undegraded":"2023-10-18T12:01:19.454093+0000","last_fullsized":"2023-10-18T12:01:19.454093+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463415+0000","last_change":"2023-10-18T12:01:19.462049+0000","last_active":"2023-10-18T12:01:19.463415+0000","last_peered":"2023-10-18T12:01:19.463415+0000","last_clean":"2023-10-18T12:01:19.463415+0000","last_became_active":"2023-10-18T12:01:19.460286+0000","last_became_peered":"2023-10-18T12:01:19.460286+0000","last_unstale":"2023-10-18T12:01:19.463415+0000","last_undegraded":"2023-10-18T12:01:19.463415+0000","last_fullsized":"2023-10-18T12:01:19.463415+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:59.023770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447065+0000","last_change":"2023-10-18T12:01:09.131674+0000","last_active":"2023-10-18T12:01:18.447065+0000","last_peered":"2023-10-18T12:01:18.447065+0000","last_clean":"2023-10-18T12:01:18.447065+0000","last_became_active":"2023-10-18T12:01:09.131410+0000","last_became_peered":"2023-10-18T12:01:09.131410+0000","last_unstale":"2023-10-18T12:01:18.447065+0000","last_undegraded":"2023-10-18T12:01:18.447065+0000","last_fullsized":"2023-10-18T12:01:18.447065+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:22.606198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451954+0000","last_change":"2023-10-18T12:01:19.451954+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451954+0000","last_undegraded":"2023-10-18T12:01:19.451954+0000","last_fullsized":"2023-10-18T12:01:19.451954+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463757+0000","last_change":"2023-10-18T12:01:19.460871+0000","last_active":"2023-10-18T12:01:19.463757+0000","last_peered":"2023-10-18T12:01:19.463757+0000","last_clean":"2023-10-18T12:01:19.463757+0000","last_became_active":"2023-10-18T12:01:19.460570+0000","last_became_peered":"2023-10-18T12:01:19.460570+0000","last_unstale":"2023-10-18T12:01:19.463757+0000","last_undegraded":"2023-10-18T12:01:19.463757+0000","last_fullsized":"2023-10-18T12:01:19.463757+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:21.229838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443961+0000","last_change":"2023-10-18T12:01:09.134865+0000","last_active":"2023-10-18T12:01:18.443961+0000","last_peered":"2023-10-18T12:01:18.443961+0000","last_clean":"2023-10-18T12:01:18.443961+0000","last_became_active":"2023-10-18T12:01:09.134583+0000","last_became_peered":"2023-10-18T12:01:09.134583+0000","last_unstale":"2023-10-18T12:01:18.443961+0000","last_undegraded":"2023-10-18T12:01:18.443961+0000","last_fullsized":"2023-10-18T12:01:18.443961+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:42.480656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.449313+0000","last_change":"2023-10-18T12:01:19.449313+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.449313+0000","last_undegraded":"2023-10-18T12:01:19.449313+0000","last_fullsized":"2023-10-18T12:01:19.449313+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.464534+0000","last_change":"2023-10-18T12:01:19.461910+0000","last_active":"2023-10-18T12:01:19.464534+0000","last_peered":"2023-10-18T12:01:19.464534+0000","last_clean":"2023-10-18T12:01:19.464534+0000","last_became_active":"2023-10-18T12:01:19.461653+0000","last_became_peered":"2023-10-18T12:01:19.461653+0000","last_unstale":"2023-10-18T12:01:19.464534+0000","last_undegraded":"2023-10-18T12:01:19.464534+0000","last_fullsized":"2023-10-18T12:01:19.464534+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:16:33.411957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448538+0000","last_change":"2023-10-18T12:01:09.132312+0000","last_active":"2023-10-18T12:01:18.448538+0000","last_peered":"2023-10-18T12:01:18.448538+0000","last_clean":"2023-10-18T12:01:18.448538+0000","last_became_active":"2023-10-18T12:01:09.131781+0000","last_became_peered":"2023-10-18T12:01:09.131781+0000","last_unstale":"2023-10-18T12:01:18.448538+0000","last_undegraded":"2023-10-18T12:01:18.448538+0000","last_fullsized":"2023-10-18T12:01:18.448538+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457884+0000","last_change":"2023-10-18T12:01:19.457884+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457884+0000","last_undegraded":"2023-10-18T12:01:19.457884+0000","last_fullsized":"2023-10-18T12:01:19.457884+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466498+0000","last_change":"2023-10-18T12:01:19.465117+0000","last_active":"2023-10-18T12:01:19.466498+0000","last_peered":"2023-10-18T12:01:19.466498+0000","last_clean":"2023-10-18T12:01:19.466498+0000","last_became_active":"2023-10-18T12:01:19.464845+0000","last_became_peered":"2023-10-18T12:01:19.464845+0000","last_unstale":"2023-10-18T12:01:19.466498+0000","last_undegraded":"2023-10-18T12:01:19.466498+0000","last_fullsized":"2023-10-18T12:01:19.466498+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:22.112086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448553+0000","last_change":"2023-10-18T12:01:09.133664+0000","last_active":"2023-10-18T12:01:18.448553+0000","last_peered":"2023-10-18T12:01:18.448553+0000","last_clean":"2023-10-18T12:01:18.448553+0000","last_became_active":"2023-10-18T12:01:09.133406+0000","last_became_peered":"2023-10-18T12:01:09.133406+0000","last_unstale":"2023-10-18T12:01:18.448553+0000","last_undegraded":"2023-10-18T12:01:18.448553+0000","last_fullsized":"2023-10-18T12:01:18.448553+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454282+0000","last_change":"2023-10-18T12:01:19.454282+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454282+0000","last_undegraded":"2023-10-18T12:01:19.454282+0000","last_fullsized":"2023-10-18T12:01:19.454282+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.461380+0000","last_change":"2023-10-18T12:01:19.458515+0000","last_active":"2023-10-18T12:01:19.461380+0000","last_peered":"2023-10-18T12:01:19.461380+0000","last_clean":"2023-10-18T12:01:19.461380+0000","last_became_active":"2023-10-18T12:01:19.457749+0000","last_became_peered":"2023-10-18T12:01:19.457749+0000","last_unstale":"2023-10-18T12:01:19.461380+0000","last_undegraded":"2023-10-18T12:01:19.461380+0000","last_fullsized":"2023-10-18T12:01:19.461380+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:24.677682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439756+0000","last_change":"2023-10-18T12:01:09.130142+0000","last_active":"2023-10-18T12:01:18.439756+0000","last_peered":"2023-10-18T12:01:18.439756+0000","last_clean":"2023-10-18T12:01:18.439756+0000","last_became_active":"2023-10-18T12:01:09.129884+0000","last_became_peered":"2023-10-18T12:01:09.129884+0000","last_unstale":"2023-10-18T12:01:18.439756+0000","last_undegraded":"2023-10-18T12:01:18.439756+0000","last_fullsized":"2023-10-18T12:01:18.439756+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:36.037273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450182+0000","last_change":"2023-10-18T12:01:19.450182+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450182+0000","last_undegraded":"2023-10-18T12:01:19.450182+0000","last_fullsized":"2023-10-18T12:01:19.450182+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.459660+0000","last_change":"2023-10-18T12:01:19.458409+0000","last_active":"2023-10-18T12:01:19.459660+0000","last_peered":"2023-10-18T12:01:19.459660+0000","last_clean":"2023-10-18T12:01:19.459660+0000","last_became_active":"2023-10-18T12:01:19.458151+0000","last_became_peered":"2023-10-18T12:01:19.458151+0000","last_unstale":"2023-10-18T12:01:19.459660+0000","last_undegraded":"2023-10-18T12:01:19.459660+0000","last_fullsized":"2023-10-18T12:01:19.459660+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:30.732460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443598+0000","last_change":"2023-10-18T12:01:09.136066+0000","last_active":"2023-10-18T12:01:18.443598+0000","last_peered":"2023-10-18T12:01:18.443598+0000","last_clean":"2023-10-18T12:01:18.443598+0000","last_became_active":"2023-10-18T12:01:09.135802+0000","last_became_peered":"2023-10-18T12:01:09.135802+0000","last_unstale":"2023-10-18T12:01:18.443598+0000","last_undegraded":"2023-10-18T12:01:18.443598+0000","last_fullsized":"2023-10-18T12:01:18.443598+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:26.550223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455962+0000","last_change":"2023-10-18T12:01:19.455962+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455962+0000","last_undegraded":"2023-10-18T12:01:19.455962+0000","last_fullsized":"2023-10-18T12:01:19.455962+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439658+0000","last_change":"2023-10-18T12:01:09.120812+0000","last_active":"2023-10-18T12:01:18.439658+0000","last_peered":"2023-10-18T12:01:18.439658+0000","last_clean":"2023-10-18T12:01:18.439658+0000","last_became_active":"2023-10-18T12:01:09.120561+0000","last_became_peered":"2023-10-18T12:01:09.120561+0000","last_unstale":"2023-10-18T12:01:18.439658+0000","last_undegraded":"2023-10-18T12:01:18.439658+0000","last_fullsized":"2023-10-18T12:01:18.439658+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:02.277242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"241'4","reported_seq":18,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444451+0000","last_change":"2023-10-18T12:01:08.112445+0000","last_active":"2023-10-18T12:01:18.444451+0000","last_peered":"2023-10-18T12:01:18.444451+0000","last_clean":"2023-10-18T12:01:18.444451+0000","last_became_active":"2023-10-18T12:01:08.112017+0000","last_became_peered":"2023-10-18T12:01:08.112017+0000","last_unstale":"2023-10-18T12:01:18.444451+0000","last_undegraded":"2023-10-18T12:01:18.444451+0000","last_fullsized":"2023-10-18T12:01:18.444451+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:11.607481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452634+0000","last_change":"2023-10-18T12:01:19.452634+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452634+0000","last_undegraded":"2023-10-18T12:01:19.452634+0000","last_fullsized":"2023-10-18T12:01:19.452634+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446081+0000","last_change":"2023-10-18T12:01:09.134916+0000","last_active":"2023-10-18T12:01:18.446081+0000","last_peered":"2023-10-18T12:01:18.446081+0000","last_clean":"2023-10-18T12:01:18.446081+0000","last_became_active":"2023-10-18T12:01:09.134630+0000","last_became_peered":"2023-10-18T12:01:09.134630+0000","last_unstale":"2023-10-18T12:01:18.446081+0000","last_undegraded":"2023-10-18T12:01:18.446081+0000","last_fullsized":"2023-10-18T12:01:18.446081+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447293+0000","last_change":"2023-10-18T12:01:08.119870+0000","last_active":"2023-10-18T12:01:18.447293+0000","last_peered":"2023-10-18T12:01:18.447293+0000","last_clean":"2023-10-18T12:01:18.447293+0000","last_became_active":"2023-10-18T12:01:08.119586+0000","last_became_peered":"2023-10-18T12:01:08.119586+0000","last_unstale":"2023-10-18T12:01:18.447293+0000","last_undegraded":"2023-10-18T12:01:18.447293+0000","last_fullsized":"2023-10-18T12:01:18.447293+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:16.288493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453087+0000","last_change":"2023-10-18T12:01:19.453087+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453087+0000","last_undegraded":"2023-10-18T12:01:19.453087+0000","last_fullsized":"2023-10-18T12:01:19.453087+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443755+0000","last_change":"2023-10-18T12:01:09.130894+0000","last_active":"2023-10-18T12:01:18.443755+0000","last_peered":"2023-10-18T12:01:18.443755+0000","last_clean":"2023-10-18T12:01:18.443755+0000","last_became_active":"2023-10-18T12:01:09.130614+0000","last_became_peered":"2023-10-18T12:01:09.130614+0000","last_unstale":"2023-10-18T12:01:18.443755+0000","last_undegraded":"2023-10-18T12:01:18.443755+0000","last_fullsized":"2023-10-18T12:01:18.443755+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:03.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448884+0000","last_change":"2023-10-18T12:01:08.115174+0000","last_active":"2023-10-18T12:01:18.448884+0000","last_peered":"2023-10-18T12:01:18.448884+0000","last_clean":"2023-10-18T12:01:18.448884+0000","last_became_active":"2023-10-18T12:01:08.114546+0000","last_became_peered":"2023-10-18T12:01:08.114546+0000","last_unstale":"2023-10-18T12:01:18.448884+0000","last_undegraded":"2023-10-18T12:01:18.448884+0000","last_fullsized":"2023-10-18T12:01:18.448884+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453540+0000","last_change":"2023-10-18T12:01:19.453540+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453540+0000","last_undegraded":"2023-10-18T12:01:19.453540+0000","last_fullsized":"2023-10-18T12:01:19.453540+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448819+0000","last_change":"2023-10-18T12:01:09.122601+0000","last_active":"2023-10-18T12:01:18.448819+0000","last_peered":"2023-10-18T12:01:18.448819+0000","last_clean":"2023-10-18T12:01:18.448819+0000","last_became_active":"2023-10-18T12:01:09.122334+0000","last_became_peered":"2023-10-18T12:01:09.122334+0000","last_unstale":"2023-10-18T12:01:18.448819+0000","last_undegraded":"2023-10-18T12:01:18.448819+0000","last_fullsized":"2023-10-18T12:01:18.448819+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446549+0000","last_change":"2023-10-18T12:01:08.116478+0000","last_active":"2023-10-18T12:01:18.446549+0000","last_peered":"2023-10-18T12:01:18.446549+0000","last_clean":"2023-10-18T12:01:18.446549+0000","last_became_active":"2023-10-18T12:01:08.116181+0000","last_became_peered":"2023-10-18T12:01:08.116181+0000","last_unstale":"2023-10-18T12:01:18.446549+0000","last_undegraded":"2023-10-18T12:01:18.446549+0000","last_fullsized":"2023-10-18T12:01:18.446549+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:10.087064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451810+0000","last_change":"2023-10-18T12:01:19.451810+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451810+0000","last_undegraded":"2023-10-18T12:01:19.451810+0000","last_fullsized":"2023-10-18T12:01:19.451810+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447548+0000","last_change":"2023-10-18T12:01:09.130166+0000","last_active":"2023-10-18T12:01:18.447548+0000","last_peered":"2023-10-18T12:01:18.447548+0000","last_clean":"2023-10-18T12:01:18.447548+0000","last_became_active":"2023-10-18T12:01:09.129725+0000","last_became_peered":"2023-10-18T12:01:09.129725+0000","last_unstale":"2023-10-18T12:01:18.447548+0000","last_undegraded":"2023-10-18T12:01:18.447548+0000","last_fullsized":"2023-10-18T12:01:18.447548+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:40.892842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448642+0000","last_change":"2023-10-18T12:01:08.107385+0000","last_active":"2023-10-18T12:01:18.448642+0000","last_peered":"2023-10-18T12:01:18.448642+0000","last_clean":"2023-10-18T12:01:18.448642+0000","last_became_active":"2023-10-18T12:01:08.107129+0000","last_became_peered":"2023-10-18T12:01:08.107129+0000","last_unstale":"2023-10-18T12:01:18.448642+0000","last_undegraded":"2023-10-18T12:01:18.448642+0000","last_fullsized":"2023-10-18T12:01:18.448642+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:43.838857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455957+0000","last_change":"2023-10-18T12:01:19.455957+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455957+0000","last_undegraded":"2023-10-18T12:01:19.455957+0000","last_fullsized":"2023-10-18T12:01:19.455957+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445933+0000","last_change":"2023-10-18T12:01:09.133737+0000","last_active":"2023-10-18T12:01:18.445933+0000","last_peered":"2023-10-18T12:01:18.445933+0000","last_clean":"2023-10-18T12:01:18.445933+0000","last_became_active":"2023-10-18T12:01:09.133487+0000","last_became_peered":"2023-10-18T12:01:09.133487+0000","last_unstale":"2023-10-18T12:01:18.445933+0000","last_undegraded":"2023-10-18T12:01:18.445933+0000","last_fullsized":"2023-10-18T12:01:18.445933+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:53:36.261500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445701+0000","last_change":"2023-10-18T12:01:08.119435+0000","last_active":"2023-10-18T12:01:18.445701+0000","last_peered":"2023-10-18T12:01:18.445701+0000","last_clean":"2023-10-18T12:01:18.445701+0000","last_became_active":"2023-10-18T12:01:08.118740+0000","last_became_peered":"2023-10-18T12:01:08.118740+0000","last_unstale":"2023-10-18T12:01:18.445701+0000","last_undegraded":"2023-10-18T12:01:18.445701+0000","last_fullsized":"2023-10-18T12:01:18.445701+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:54.172871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454152+0000","last_change":"2023-10-18T12:01:19.454152+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454152+0000","last_undegraded":"2023-10-18T12:01:19.454152+0000","last_fullsized":"2023-10-18T12:01:19.454152+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446839+0000","last_change":"2023-10-18T12:01:09.132487+0000","last_active":"2023-10-18T12:01:18.446839+0000","last_peered":"2023-10-18T12:01:18.446839+0000","last_clean":"2023-10-18T12:01:18.446839+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:18.446839+0000","last_undegraded":"2023-10-18T12:01:18.446839+0000","last_fullsized":"2023-10-18T12:01:18.446839+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:14.584151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443645+0000","last_change":"2023-10-18T12:01:08.115982+0000","last_active":"2023-10-18T12:01:18.443645+0000","last_peered":"2023-10-18T12:01:18.443645+0000","last_clean":"2023-10-18T12:01:18.443645+0000","last_became_active":"2023-10-18T12:01:08.115682+0000","last_became_peered":"2023-10-18T12:01:08.115682+0000","last_unstale":"2023-10-18T12:01:18.443645+0000","last_undegraded":"2023-10-18T12:01:18.443645+0000","last_fullsized":"2023-10-18T12:01:18.443645+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:36.789486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452968+0000","last_change":"2023-10-18T12:01:19.452968+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452968+0000","last_undegraded":"2023-10-18T12:01:19.452968+0000","last_fullsized":"2023-10-18T12:01:19.452968+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445246+0000","last_change":"2023-10-18T12:01:09.130010+0000","last_active":"2023-10-18T12:01:18.445246+0000","last_peered":"2023-10-18T12:01:18.445246+0000","last_clean":"2023-10-18T12:01:18.445246+0000","last_became_active":"2023-10-18T12:01:09.129744+0000","last_became_peered":"2023-10-18T12:01:09.129744+0000","last_unstale":"2023-10-18T12:01:18.445246+0000","last_undegraded":"2023-10-18T12:01:18.445246+0000","last_fullsized":"2023-10-18T12:01:18.445246+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:48:02.468955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448887+0000","last_change":"2023-10-18T12:01:08.110833+0000","last_active":"2023-10-18T12:01:18.448887+0000","last_peered":"2023-10-18T12:01:18.448887+0000","last_clean":"2023-10-18T12:01:18.448887+0000","last_became_active":"2023-10-18T12:01:08.110005+0000","last_became_peered":"2023-10-18T12:01:08.110005+0000","last_unstale":"2023-10-18T12:01:18.448887+0000","last_undegraded":"2023-10-18T12:01:18.448887+0000","last_fullsized":"2023-10-18T12:01:18.448887+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:32.311648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456437+0000","last_change":"2023-10-18T12:01:19.456437+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456437+0000","last_undegraded":"2023-10-18T12:01:19.456437+0000","last_fullsized":"2023-10-18T12:01:19.456437+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444394+0000","last_change":"2023-10-18T12:01:09.129659+0000","last_active":"2023-10-18T12:01:18.444394+0000","last_peered":"2023-10-18T12:01:18.444394+0000","last_clean":"2023-10-18T12:01:18.444394+0000","last_became_active":"2023-10-18T12:01:09.129388+0000","last_became_peered":"2023-10-18T12:01:09.129388+0000","last_unstale":"2023-10-18T12:01:18.444394+0000","last_undegraded":"2023-10-18T12:01:18.444394+0000","last_fullsized":"2023-10-18T12:01:18.444394+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:17.308708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446188+0000","last_change":"2023-10-18T12:01:08.118321+0000","last_active":"2023-10-18T12:01:18.446188+0000","last_peered":"2023-10-18T12:01:18.446188+0000","last_clean":"2023-10-18T12:01:18.446188+0000","last_became_active":"2023-10-18T12:01:08.117995+0000","last_became_peered":"2023-10-18T12:01:08.117995+0000","last_unstale":"2023-10-18T12:01:18.446188+0000","last_undegraded":"2023-10-18T12:01:18.446188+0000","last_fullsized":"2023-10-18T12:01:18.446188+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:46.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453041+0000","last_change":"2023-10-18T12:01:19.453041+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453041+0000","last_undegraded":"2023-10-18T12:01:19.453041+0000","last_fullsized":"2023-10-18T12:01:19.453041+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445227+0000","last_change":"2023-10-18T12:01:09.134396+0000","last_active":"2023-10-18T12:01:18.445227+0000","last_peered":"2023-10-18T12:01:18.445227+0000","last_clean":"2023-10-18T12:01:18.445227+0000","last_became_active":"2023-10-18T12:01:09.134144+0000","last_became_peered":"2023-10-18T12:01:09.134144+0000","last_unstale":"2023-10-18T12:01:18.445227+0000","last_undegraded":"2023-10-18T12:01:18.445227+0000","last_fullsized":"2023-10-18T12:01:18.445227+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:23:53.396657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"241'3","reported_seq":17,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447167+0000","last_change":"2023-10-18T12:01:08.118871+0000","last_active":"2023-10-18T12:01:18.447167+0000","last_peered":"2023-10-18T12:01:18.447167+0000","last_clean":"2023-10-18T12:01:18.447167+0000","last_became_active":"2023-10-18T12:01:08.118572+0000","last_became_peered":"2023-10-18T12:01:08.118572+0000","last_unstale":"2023-10-18T12:01:18.447167+0000","last_undegraded":"2023-10-18T12:01:18.447167+0000","last_fullsized":"2023-10-18T12:01:18.447167+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:45.510206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453957+0000","last_change":"2023-10-18T12:01:19.453957+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453957+0000","last_undegraded":"2023-10-18T12:01:19.453957+0000","last_fullsized":"2023-10-18T12:01:19.453957+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444883+0000","last_change":"2023-10-18T12:01:09.120444+0000","last_active":"2023-10-18T12:01:18.444883+0000","last_peered":"2023-10-18T12:01:18.444883+0000","last_clean":"2023-10-18T12:01:18.444883+0000","last_became_active":"2023-10-18T12:01:09.120018+0000","last_became_peered":"2023-10-18T12:01:09.120018+0000","last_unstale":"2023-10-18T12:01:18.444883+0000","last_undegraded":"2023-10-18T12:01:18.444883+0000","last_fullsized":"2023-10-18T12:01:18.444883+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:29.145722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.370024+0000","last_change":"2023-10-18T12:01:08.117890+0000","last_active":"2023-10-18T12:01:19.370024+0000","last_peered":"2023-10-18T12:01:19.370024+0000","last_clean":"2023-10-18T12:01:19.370024+0000","last_became_active":"2023-10-18T12:01:08.117606+0000","last_became_peered":"2023-10-18T12:01:08.117606+0000","last_unstale":"2023-10-18T12:01:19.370024+0000","last_undegraded":"2023-10-18T12:01:19.370024+0000","last_fullsized":"2023-10-18T12:01:19.370024+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:49.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454638+0000","last_change":"2023-10-18T12:01:19.454638+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454638+0000","last_undegraded":"2023-10-18T12:01:19.454638+0000","last_fullsized":"2023-10-18T12:01:19.454638+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439867+0000","last_change":"2023-10-18T12:01:09.127054+0000","last_active":"2023-10-18T12:01:18.439867+0000","last_peered":"2023-10-18T12:01:18.439867+0000","last_clean":"2023-10-18T12:01:18.439867+0000","last_became_active":"2023-10-18T12:01:09.126804+0000","last_became_peered":"2023-10-18T12:01:09.126804+0000","last_unstale":"2023-10-18T12:01:18.439867+0000","last_undegraded":"2023-10-18T12:01:18.439867+0000","last_fullsized":"2023-10-18T12:01:18.439867+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:20.948970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446639+0000","last_change":"2023-10-18T12:01:08.118544+0000","last_active":"2023-10-18T12:01:18.446639+0000","last_peered":"2023-10-18T12:01:18.446639+0000","last_clean":"2023-10-18T12:01:18.446639+0000","last_became_active":"2023-10-18T12:01:08.118228+0000","last_became_peered":"2023-10-18T12:01:08.118228+0000","last_unstale":"2023-10-18T12:01:18.446639+0000","last_undegraded":"2023-10-18T12:01:18.446639+0000","last_fullsized":"2023-10-18T12:01:18.446639+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456191+0000","last_change":"2023-10-18T12:01:19.456191+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456191+0000","last_undegraded":"2023-10-18T12:01:19.456191+0000","last_fullsized":"2023-10-18T12:01:19.456191+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447511+0000","last_change":"2023-10-18T12:01:09.120750+0000","last_active":"2023-10-18T12:01:18.447511+0000","last_peered":"2023-10-18T12:01:18.447511+0000","last_clean":"2023-10-18T12:01:18.447511+0000","last_became_active":"2023-10-18T12:01:09.120490+0000","last_became_peered":"2023-10-18T12:01:09.120490+0000","last_unstale":"2023-10-18T12:01:18.447511+0000","last_undegraded":"2023-10-18T12:01:18.447511+0000","last_fullsized":"2023-10-18T12:01:18.447511+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:51.005840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446987+0000","last_change":"2023-10-18T12:01:08.118022+0000","last_active":"2023-10-18T12:01:18.446987+0000","last_peered":"2023-10-18T12:01:18.446987+0000","last_clean":"2023-10-18T12:01:18.446987+0000","last_became_active":"2023-10-18T12:01:08.117754+0000","last_became_peered":"2023-10-18T12:01:08.117754+0000","last_unstale":"2023-10-18T12:01:18.446987+0000","last_undegraded":"2023-10-18T12:01:18.446987+0000","last_fullsized":"2023-10-18T12:01:18.446987+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:37.834293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455355+0000","last_change":"2023-10-18T12:01:19.455355+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455355+0000","last_undegraded":"2023-10-18T12:01:19.455355+0000","last_fullsized":"2023-10-18T12:01:19.455355+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447313+0000","last_change":"2023-10-18T12:01:09.135383+0000","last_active":"2023-10-18T12:01:18.447313+0000","last_peered":"2023-10-18T12:01:18.447313+0000","last_clean":"2023-10-18T12:01:18.447313+0000","last_became_active":"2023-10-18T12:01:09.135114+0000","last_became_peered":"2023-10-18T12:01:09.135114+0000","last_unstale":"2023-10-18T12:01:18.447313+0000","last_undegraded":"2023-10-18T12:01:18.447313+0000","last_fullsized":"2023-10-18T12:01:18.447313+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446147+0000","last_change":"2023-10-18T12:01:08.115155+0000","last_active":"2023-10-18T12:01:18.446147+0000","last_peered":"2023-10-18T12:01:18.446147+0000","last_clean":"2023-10-18T12:01:18.446147+0000","last_became_active":"2023-10-18T12:01:08.114843+0000","last_became_peered":"2023-10-18T12:01:08.114843+0000","last_unstale":"2023-10-18T12:01:18.446147+0000","last_undegraded":"2023-10-18T12:01:18.446147+0000","last_fullsized":"2023-10-18T12:01:18.446147+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:28.358756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455037+0000","last_change":"2023-10-18T12:01:19.455037+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455037+0000","last_undegraded":"2023-10-18T12:01:19.455037+0000","last_fullsized":"2023-10-18T12:01:19.455037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446744+0000","last_change":"2023-10-18T12:01:09.131820+0000","last_active":"2023-10-18T12:01:18.446744+0000","last_peered":"2023-10-18T12:01:18.446744+0000","last_clean":"2023-10-18T12:01:18.446744+0000","last_became_active":"2023-10-18T12:01:09.131568+0000","last_became_peered":"2023-10-18T12:01:09.131568+0000","last_unstale":"2023-10-18T12:01:18.446744+0000","last_undegraded":"2023-10-18T12:01:18.446744+0000","last_fullsized":"2023-10-18T12:01:18.446744+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.395799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"241'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439719+0000","last_change":"2023-10-18T12:01:08.112711+0000","last_active":"2023-10-18T12:01:18.439719+0000","last_peered":"2023-10-18T12:01:18.439719+0000","last_clean":"2023-10-18T12:01:18.439719+0000","last_became_active":"2023-10-18T12:01:08.112436+0000","last_became_peered":"2023-10-18T12:01:08.112436+0000","last_unstale":"2023-10-18T12:01:18.439719+0000","last_undegraded":"2023-10-18T12:01:18.439719+0000","last_fullsized":"2023-10-18T12:01:18.439719+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:15.568916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455648+0000","last_change":"2023-10-18T12:01:19.455648+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455648+0000","last_undegraded":"2023-10-18T12:01:19.455648+0000","last_fullsized":"2023-10-18T12:01:19.455648+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444666+0000","last_change":"2023-10-18T12:01:09.135106+0000","last_active":"2023-10-18T12:01:18.444666+0000","last_peered":"2023-10-18T12:01:18.444666+0000","last_clean":"2023-10-18T12:01:18.444666+0000","last_became_active":"2023-10-18T12:01:09.134828+0000","last_became_peered":"2023-10-18T12:01:09.134828+0000","last_unstale":"2023-10-18T12:01:18.444666+0000","last_undegraded":"2023-10-18T12:01:18.444666+0000","last_fullsized":"2023-10-18T12:01:18.444666+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:39.126921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444574+0000","last_change":"2023-10-18T12:01:08.119248+0000","last_active":"2023-10-18T12:01:18.444574+0000","last_peered":"2023-10-18T12:01:18.444574+0000","last_clean":"2023-10-18T12:01:18.444574+0000","last_became_active":"2023-10-18T12:01:08.118973+0000","last_became_peered":"2023-10-18T12:01:08.118973+0000","last_unstale":"2023-10-18T12:01:18.444574+0000","last_undegraded":"2023-10-18T12:01:18.444574+0000","last_fullsized":"2023-10-18T12:01:18.444574+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:37.079812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459833+0000","last_change":"2023-10-18T12:01:19.459833+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459833+0000","last_undegraded":"2023-10-18T12:01:19.459833+0000","last_fullsized":"2023-10-18T12:01:19.459833+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445946+0000","last_change":"2023-10-18T12:01:09.132827+0000","last_active":"2023-10-18T12:01:18.445946+0000","last_peered":"2023-10-18T12:01:18.445946+0000","last_clean":"2023-10-18T12:01:18.445946+0000","last_became_active":"2023-10-18T12:01:09.132545+0000","last_became_peered":"2023-10-18T12:01:09.132545+0000","last_unstale":"2023-10-18T12:01:18.445946+0000","last_undegraded":"2023-10-18T12:01:18.445946+0000","last_fullsized":"2023-10-18T12:01:18.445946+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:02.034692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439997+0000","last_change":"2023-10-18T12:01:08.107964+0000","last_active":"2023-10-18T12:01:18.439997+0000","last_peered":"2023-10-18T12:01:18.439997+0000","last_clean":"2023-10-18T12:01:18.439997+0000","last_became_active":"2023-10-18T12:01:08.107696+0000","last_became_peered":"2023-10-18T12:01:08.107696+0000","last_unstale":"2023-10-18T12:01:18.439997+0000","last_undegraded":"2023-10-18T12:01:18.439997+0000","last_fullsized":"2023-10-18T12:01:18.439997+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:18.578919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453669+0000","last_change":"2023-10-18T12:01:19.453669+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453669+0000","last_undegraded":"2023-10-18T12:01:19.453669+0000","last_fullsized":"2023-10-18T12:01:19.453669+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439817+0000","last_change":"2023-10-18T12:01:09.136078+0000","last_active":"2023-10-18T12:01:18.439817+0000","last_peered":"2023-10-18T12:01:18.439817+0000","last_clean":"2023-10-18T12:01:18.439817+0000","last_became_active":"2023-10-18T12:01:09.134722+0000","last_became_peered":"2023-10-18T12:01:09.134722+0000","last_unstale":"2023-10-18T12:01:18.439817+0000","last_undegraded":"2023-10-18T12:01:18.439817+0000","last_fullsized":"2023-10-18T12:01:18.439817+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:09.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447390+0000","last_change":"2023-10-18T12:01:08.120416+0000","last_active":"2023-10-18T12:01:18.447390+0000","last_peered":"2023-10-18T12:01:18.447390+0000","last_clean":"2023-10-18T12:01:18.447390+0000","last_became_active":"2023-10-18T12:01:08.120122+0000","last_became_peered":"2023-10-18T12:01:08.120122+0000","last_unstale":"2023-10-18T12:01:18.447390+0000","last_undegraded":"2023-10-18T12:01:18.447390+0000","last_fullsized":"2023-10-18T12:01:18.447390+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:26.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455164+0000","last_change":"2023-10-18T12:01:19.455164+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455164+0000","last_undegraded":"2023-10-18T12:01:19.455164+0000","last_fullsized":"2023-10-18T12:01:19.455164+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446070+0000","last_change":"2023-10-18T12:01:09.135042+0000","last_active":"2023-10-18T12:01:18.446070+0000","last_peered":"2023-10-18T12:01:18.446070+0000","last_clean":"2023-10-18T12:01:18.446070+0000","last_became_active":"2023-10-18T12:01:09.134765+0000","last_became_peered":"2023-10-18T12:01:09.134765+0000","last_unstale":"2023-10-18T12:01:18.446070+0000","last_undegraded":"2023-10-18T12:01:18.446070+0000","last_fullsized":"2023-10-18T12:01:18.446070+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:58.131101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446991+0000","last_change":"2023-10-18T12:01:08.117972+0000","last_active":"2023-10-18T12:01:18.446991+0000","last_peered":"2023-10-18T12:01:18.446991+0000","last_clean":"2023-10-18T12:01:18.446991+0000","last_became_active":"2023-10-18T12:01:08.117690+0000","last_became_peered":"2023-10-18T12:01:08.117690+0000","last_unstale":"2023-10-18T12:01:18.446991+0000","last_undegraded":"2023-10-18T12:01:18.446991+0000","last_fullsized":"2023-10-18T12:01:18.446991+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458648+0000","last_change":"2023-10-18T12:01:19.458648+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458648+0000","last_undegraded":"2023-10-18T12:01:19.458648+0000","last_fullsized":"2023-10-18T12:01:19.458648+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444663+0000","last_change":"2023-10-18T12:01:09.122673+0000","last_active":"2023-10-18T12:01:18.444663+0000","last_peered":"2023-10-18T12:01:18.444663+0000","last_clean":"2023-10-18T12:01:18.444663+0000","last_became_active":"2023-10-18T12:01:09.122430+0000","last_became_peered":"2023-10-18T12:01:09.122430+0000","last_unstale":"2023-10-18T12:01:18.444663+0000","last_undegraded":"2023-10-18T12:01:18.444663+0000","last_fullsized":"2023-10-18T12:01:18.444663+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:29.387267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446157+0000","last_change":"2023-10-18T12:01:08.119439+0000","last_active":"2023-10-18T12:01:18.446157+0000","last_peered":"2023-10-18T12:01:18.446157+0000","last_clean":"2023-10-18T12:01:18.446157+0000","last_became_active":"2023-10-18T12:01:08.119158+0000","last_became_peered":"2023-10-18T12:01:08.119158+0000","last_unstale":"2023-10-18T12:01:18.446157+0000","last_undegraded":"2023-10-18T12:01:18.446157+0000","last_fullsized":"2023-10-18T12:01:18.446157+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:56.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454552+0000","last_change":"2023-10-18T12:01:19.454552+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454552+0000","last_undegraded":"2023-10-18T12:01:19.454552+0000","last_fullsized":"2023-10-18T12:01:19.454552+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440049+0000","last_change":"2023-10-18T12:01:09.124352+0000","last_active":"2023-10-18T12:01:18.440049+0000","last_peered":"2023-10-18T12:01:18.440049+0000","last_clean":"2023-10-18T12:01:18.440049+0000","last_became_active":"2023-10-18T12:01:09.124072+0000","last_became_peered":"2023-10-18T12:01:09.124072+0000","last_unstale":"2023-10-18T12:01:18.440049+0000","last_undegraded":"2023-10-18T12:01:18.440049+0000","last_fullsized":"2023-10-18T12:01:18.440049+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:15.239290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443813+0000","last_change":"2023-10-18T12:01:08.117335+0000","last_active":"2023-10-18T12:01:18.443813+0000","last_peered":"2023-10-18T12:01:18.443813+0000","last_clean":"2023-10-18T12:01:18.443813+0000","last_became_active":"2023-10-18T12:01:08.117032+0000","last_became_peered":"2023-10-18T12:01:08.117032+0000","last_unstale":"2023-10-18T12:01:18.443813+0000","last_undegraded":"2023-10-18T12:01:18.443813+0000","last_fullsized":"2023-10-18T12:01:18.443813+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:33.304569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456597+0000","last_change":"2023-10-18T12:01:19.456597+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456597+0000","last_undegraded":"2023-10-18T12:01:19.456597+0000","last_fullsized":"2023-10-18T12:01:19.456597+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446661+0000","last_change":"2023-10-18T12:01:09.135148+0000","last_active":"2023-10-18T12:01:18.446661+0000","last_peered":"2023-10-18T12:01:18.446661+0000","last_clean":"2023-10-18T12:01:18.446661+0000","last_became_active":"2023-10-18T12:01:09.134895+0000","last_became_peered":"2023-10-18T12:01:09.134895+0000","last_unstale":"2023-10-18T12:01:18.446661+0000","last_undegraded":"2023-10-18T12:01:18.446661+0000","last_fullsized":"2023-10-18T12:01:18.446661+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:19.263310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444467+0000","last_change":"2023-10-18T12:01:08.120618+0000","last_active":"2023-10-18T12:01:18.444467+0000","last_peered":"2023-10-18T12:01:18.444467+0000","last_clean":"2023-10-18T12:01:18.444467+0000","last_became_active":"2023-10-18T12:01:08.120315+0000","last_became_peered":"2023-10-18T12:01:08.120315+0000","last_unstale":"2023-10-18T12:01:18.444467+0000","last_undegraded":"2023-10-18T12:01:18.444467+0000","last_fullsized":"2023-10-18T12:01:18.444467+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:36.555722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457598+0000","last_change":"2023-10-18T12:01:19.457598+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457598+0000","last_undegraded":"2023-10-18T12:01:19.457598+0000","last_fullsized":"2023-10-18T12:01:19.457598+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447164+0000","last_change":"2023-10-18T12:01:09.130881+0000","last_active":"2023-10-18T12:01:18.447164+0000","last_peered":"2023-10-18T12:01:18.447164+0000","last_clean":"2023-10-18T12:01:18.447164+0000","last_became_active":"2023-10-18T12:01:09.130628+0000","last_became_peered":"2023-10-18T12:01:09.130628+0000","last_unstale":"2023-10-18T12:01:18.447164+0000","last_undegraded":"2023-10-18T12:01:18.447164+0000","last_fullsized":"2023-10-18T12:01:18.447164+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:13.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448426+0000","last_change":"2023-10-18T12:01:08.120313+0000","last_active":"2023-10-18T12:01:18.448426+0000","last_peered":"2023-10-18T12:01:18.448426+0000","last_clean":"2023-10-18T12:01:18.448426+0000","last_became_active":"2023-10-18T12:01:08.120059+0000","last_became_peered":"2023-10-18T12:01:08.120059+0000","last_unstale":"2023-10-18T12:01:18.448426+0000","last_undegraded":"2023-10-18T12:01:18.448426+0000","last_fullsized":"2023-10-18T12:01:18.448426+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456394+0000","last_change":"2023-10-18T12:01:19.456394+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456394+0000","last_undegraded":"2023-10-18T12:01:19.456394+0000","last_fullsized":"2023-10-18T12:01:19.456394+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445079+0000","last_change":"2023-10-18T12:01:09.125685+0000","last_active":"2023-10-18T12:01:18.445079+0000","last_peered":"2023-10-18T12:01:18.445079+0000","last_clean":"2023-10-18T12:01:18.445079+0000","last_became_active":"2023-10-18T12:01:09.125395+0000","last_became_peered":"2023-10-18T12:01:09.125395+0000","last_unstale":"2023-10-18T12:01:18.445079+0000","last_undegraded":"2023-10-18T12:01:18.445079+0000","last_fullsized":"2023-10-18T12:01:18.445079+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:47.228016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444968+0000","last_change":"2023-10-18T12:01:08.117717+0000","last_active":"2023-10-18T12:01:18.444968+0000","last_peered":"2023-10-18T12:01:18.444968+0000","last_clean":"2023-10-18T12:01:18.444968+0000","last_became_active":"2023-10-18T12:01:08.117405+0000","last_became_peered":"2023-10-18T12:01:08.117405+0000","last_unstale":"2023-10-18T12:01:18.444968+0000","last_undegraded":"2023-10-18T12:01:18.444968+0000","last_fullsized":"2023-10-18T12:01:18.444968+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:00.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454367+0000","last_change":"2023-10-18T12:01:19.454367+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454367+0000","last_undegraded":"2023-10-18T12:01:19.454367+0000","last_fullsized":"2023-10-18T12:01:19.454367+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445826+0000","last_change":"2023-10-18T12:01:09.132599+0000","last_active":"2023-10-18T12:01:18.445826+0000","last_peered":"2023-10-18T12:01:18.445826+0000","last_clean":"2023-10-18T12:01:18.445826+0000","last_became_active":"2023-10-18T12:01:09.131710+0000","last_became_peered":"2023-10-18T12:01:09.131710+0000","last_unstale":"2023-10-18T12:01:18.445826+0000","last_undegraded":"2023-10-18T12:01:18.445826+0000","last_fullsized":"2023-10-18T12:01:18.445826+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:18.701174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446916+0000","last_change":"2023-10-18T12:01:08.118553+0000","last_active":"2023-10-18T12:01:18.446916+0000","last_peered":"2023-10-18T12:01:18.446916+0000","last_clean":"2023-10-18T12:01:18.446916+0000","last_became_active":"2023-10-18T12:01:08.118286+0000","last_became_peered":"2023-10-18T12:01:08.118286+0000","last_unstale":"2023-10-18T12:01:18.446916+0000","last_undegraded":"2023-10-18T12:01:18.446916+0000","last_fullsized":"2023-10-18T12:01:18.446916+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:05:32.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.460259+0000","last_change":"2023-10-18T12:01:19.460259+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.460259+0000","last_undegraded":"2023-10-18T12:01:19.460259+0000","last_fullsized":"2023-10-18T12:01:19.460259+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445148+0000","last_change":"2023-10-18T12:01:09.133522+0000","last_active":"2023-10-18T12:01:18.445148+0000","last_peered":"2023-10-18T12:01:18.445148+0000","last_clean":"2023-10-18T12:01:18.445148+0000","last_became_active":"2023-10-18T12:01:09.133267+0000","last_became_peered":"2023-10-18T12:01:09.133267+0000","last_unstale":"2023-10-18T12:01:18.445148+0000","last_undegraded":"2023-10-18T12:01:18.445148+0000","last_fullsized":"2023-10-18T12:01:18.445148+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:44:14.702965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448989+0000","last_change":"2023-10-18T12:01:08.116830+0000","last_active":"2023-10-18T12:01:18.448989+0000","last_peered":"2023-10-18T12:01:18.448989+0000","last_clean":"2023-10-18T12:01:18.448989+0000","last_became_active":"2023-10-18T12:01:08.116538+0000","last_became_peered":"2023-10-18T12:01:08.116538+0000","last_unstale":"2023-10-18T12:01:18.448989+0000","last_undegraded":"2023-10-18T12:01:18.448989+0000","last_fullsized":"2023-10-18T12:01:18.448989+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:09.165065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451313+0000","last_change":"2023-10-18T12:01:19.451313+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451313+0000","last_undegraded":"2023-10-18T12:01:19.451313+0000","last_fullsized":"2023-10-18T12:01:19.451313+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446209+0000","last_change":"2023-10-18T12:01:09.128305+0000","last_active":"2023-10-18T12:01:18.446209+0000","last_peered":"2023-10-18T12:01:18.446209+0000","last_clean":"2023-10-18T12:01:18.446209+0000","last_became_active":"2023-10-18T12:01:09.127933+0000","last_became_peered":"2023-10-18T12:01:09.127933+0000","last_unstale":"2023-10-18T12:01:18.446209+0000","last_undegraded":"2023-10-18T12:01:18.446209+0000","last_fullsized":"2023-10-18T12:01:18.446209+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:28.312681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448893+0000","last_change":"2023-10-18T12:01:08.115363+0000","last_active":"2023-10-18T12:01:18.448893+0000","last_peered":"2023-10-18T12:01:18.448893+0000","last_clean":"2023-10-18T12:01:18.448893+0000","last_became_active":"2023-10-18T12:01:08.114903+0000","last_became_peered":"2023-10-18T12:01:08.114903+0000","last_unstale":"2023-10-18T12:01:18.448893+0000","last_undegraded":"2023-10-18T12:01:18.448893+0000","last_fullsized":"2023-10-18T12:01:18.448893+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:46.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456219+0000","last_change":"2023-10-18T12:01:19.456219+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456219+0000","last_undegraded":"2023-10-18T12:01:19.456219+0000","last_fullsized":"2023-10-18T12:01:19.456219+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448801+0000","last_change":"2023-10-18T12:01:09.127159+0000","last_active":"2023-10-18T12:01:18.448801+0000","last_peered":"2023-10-18T12:01:18.448801+0000","last_clean":"2023-10-18T12:01:18.448801+0000","last_became_active":"2023-10-18T12:01:09.126874+0000","last_became_peered":"2023-10-18T12:01:09.126874+0000","last_unstale":"2023-10-18T12:01:18.448801+0000","last_undegraded":"2023-10-18T12:01:18.448801+0000","last_fullsized":"2023-10-18T12:01:18.448801+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444343+0000","last_change":"2023-10-18T12:01:08.118539+0000","last_active":"2023-10-18T12:01:18.444343+0000","last_peered":"2023-10-18T12:01:18.444343+0000","last_clean":"2023-10-18T12:01:18.444343+0000","last_became_active":"2023-10-18T12:01:08.116966+0000","last_became_peered":"2023-10-18T12:01:08.116966+0000","last_unstale":"2023-10-18T12:01:18.444343+0000","last_undegraded":"2023-10-18T12:01:18.444343+0000","last_fullsized":"2023-10-18T12:01:18.444343+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:20.828671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457469+0000","last_change":"2023-10-18T12:01:19.457469+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457469+0000","last_undegraded":"2023-10-18T12:01:19.457469+0000","last_fullsized":"2023-10-18T12:01:19.457469+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444299+0000","last_change":"2023-10-18T12:01:09.119148+0000","last_active":"2023-10-18T12:01:18.444299+0000","last_peered":"2023-10-18T12:01:18.444299+0000","last_clean":"2023-10-18T12:01:18.444299+0000","last_became_active":"2023-10-18T12:01:09.118894+0000","last_became_peered":"2023-10-18T12:01:09.118894+0000","last_unstale":"2023-10-18T12:01:18.444299+0000","last_undegraded":"2023-10-18T12:01:18.444299+0000","last_fullsized":"2023-10-18T12:01:18.444299+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:42.857305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447050+0000","last_change":"2023-10-18T12:01:08.117526+0000","last_active":"2023-10-18T12:01:18.447050+0000","last_peered":"2023-10-18T12:01:18.447050+0000","last_clean":"2023-10-18T12:01:18.447050+0000","last_became_active":"2023-10-18T12:01:08.117234+0000","last_became_peered":"2023-10-18T12:01:08.117234+0000","last_unstale":"2023-10-18T12:01:18.447050+0000","last_undegraded":"2023-10-18T12:01:18.447050+0000","last_fullsized":"2023-10-18T12:01:18.447050+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:43.295217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.460764+0000","last_change":"2023-10-18T12:01:19.460764+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.460764+0000","last_undegraded":"2023-10-18T12:01:19.460764+0000","last_fullsized":"2023-10-18T12:01:19.460764+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439950+0000","last_change":"2023-10-18T12:01:09.125150+0000","last_active":"2023-10-18T12:01:18.439950+0000","last_peered":"2023-10-18T12:01:18.439950+0000","last_clean":"2023-10-18T12:01:18.439950+0000","last_became_active":"2023-10-18T12:01:09.124900+0000","last_became_peered":"2023-10-18T12:01:09.124900+0000","last_unstale":"2023-10-18T12:01:18.439950+0000","last_undegraded":"2023-10-18T12:01:18.439950+0000","last_fullsized":"2023-10-18T12:01:18.439950+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:10.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445789+0000","last_change":"2023-10-18T12:01:08.119612+0000","last_active":"2023-10-18T12:01:18.445789+0000","last_peered":"2023-10-18T12:01:18.445789+0000","last_clean":"2023-10-18T12:01:18.445789+0000","last_became_active":"2023-10-18T12:01:08.119116+0000","last_became_peered":"2023-10-18T12:01:08.119116+0000","last_unstale":"2023-10-18T12:01:18.445789+0000","last_undegraded":"2023-10-18T12:01:18.445789+0000","last_fullsized":"2023-10-18T12:01:18.445789+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:02.536144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451311+0000","last_change":"2023-10-18T12:01:19.451311+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451311+0000","last_undegraded":"2023-10-18T12:01:19.451311+0000","last_fullsized":"2023-10-18T12:01:19.451311+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446463+0000","last_change":"2023-10-18T12:01:09.130360+0000","last_active":"2023-10-18T12:01:18.446463+0000","last_peered":"2023-10-18T12:01:18.446463+0000","last_clean":"2023-10-18T12:01:18.446463+0000","last_became_active":"2023-10-18T12:01:09.130086+0000","last_became_peered":"2023-10-18T12:01:09.130086+0000","last_unstale":"2023-10-18T12:01:18.446463+0000","last_undegraded":"2023-10-18T12:01:18.446463+0000","last_fullsized":"2023-10-18T12:01:18.446463+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:28.085761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443434+0000","last_change":"2023-10-18T12:01:08.119803+0000","last_active":"2023-10-18T12:01:18.443434+0000","last_peered":"2023-10-18T12:01:18.443434+0000","last_clean":"2023-10-18T12:01:18.443434+0000","last_became_active":"2023-10-18T12:01:08.119467+0000","last_became_peered":"2023-10-18T12:01:08.119467+0000","last_unstale":"2023-10-18T12:01:18.443434+0000","last_undegraded":"2023-10-18T12:01:18.443434+0000","last_fullsized":"2023-10-18T12:01:18.443434+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:29.410010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459374+0000","last_change":"2023-10-18T12:01:19.459374+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459374+0000","last_undegraded":"2023-10-18T12:01:19.459374+0000","last_fullsized":"2023-10-18T12:01:19.459374+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443794+0000","last_change":"2023-10-18T12:01:09.122670+0000","last_active":"2023-10-18T12:01:18.443794+0000","last_peered":"2023-10-18T12:01:18.443794+0000","last_clean":"2023-10-18T12:01:18.443794+0000","last_became_active":"2023-10-18T12:01:09.122408+0000","last_became_peered":"2023-10-18T12:01:09.122408+0000","last_unstale":"2023-10-18T12:01:18.443794+0000","last_undegraded":"2023-10-18T12:01:18.443794+0000","last_fullsized":"2023-10-18T12:01:18.443794+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:39.479911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447844+0000","last_change":"2023-10-18T12:01:08.111055+0000","last_active":"2023-10-18T12:01:18.447844+0000","last_peered":"2023-10-18T12:01:18.447844+0000","last_clean":"2023-10-18T12:01:18.447844+0000","last_became_active":"2023-10-18T12:01:08.110728+0000","last_became_peered":"2023-10-18T12:01:08.110728+0000","last_unstale":"2023-10-18T12:01:18.447844+0000","last_undegraded":"2023-10-18T12:01:18.447844+0000","last_fullsized":"2023-10-18T12:01:18.447844+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:15.472803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"227'379","reported_seq":1331,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445541+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:01:18.445541+0000","last_peered":"2023-10-18T12:01:18.445541+0000","last_clean":"2023-10-18T12:01:18.445541+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:01:18.445541+0000","last_undegraded":"2023-10-18T12:01:18.445541+0000","last_fullsized":"2023-10-18T12:01:18.445541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":379,"ondisk_log_size":379,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52808,"kb_used_data":3580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319032,"statfs":{"total":96636764160,"available":96582688768,"internally_reserved":0,"allocated":3665920,"data_stored":2169281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.42399999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.67800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834576000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42500,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329340,"statfs":{"total":96636764160,"available":96593244160,"internally_reserved":0,"allocated":3072000,"data_stored":1578901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.495},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71299999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834576000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64700,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307140,"statfs":{"total":96636764160,"available":96570511360,"internally_reserved":0,"allocated":3063808,"data_stored":1578789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.498},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":1.101,"15min":1.1539999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.71099999999999997,"15min":0.751},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.61699999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72699999999999998,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81399999999999995,"15min":1},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53000000000000003,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.749,"5min":0.82999999999999996,"15min":1.01},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.255,"15min":0.255},"max":{"1min":0.78800000000000003,"5min":0.83699999999999997,"15min":1.101},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48064,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323776,"statfs":{"total":96636764160,"available":96587546624,"internally_reserved":0,"allocated":3067904,"data_stored":1578859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.22500000000000001},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.59599999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":1.123},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.56899999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.66000000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":1.018,"15min":1.018},"last":0.752}]},{"osd":7,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.88900000000000001,"15min":1.05},"last":0.623},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.77900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834576000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48100,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323740,"statfs":{"total":96636764160,"available":96587509760,"internally_reserved":0,"allocated":3100672,"data_stored":1589807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.23999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.378,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.25},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.39300000000000002,"15min":0.40699999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.38400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.39900000000000002,"15min":0.40600000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44900000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.432,"5min":1.899,"15min":1.899},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.42899999999999999,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.434,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.87,"5min":1.034,"15min":1.052},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.629}]},{"osd":5,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":1.522,"15min":1.522},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.91100000000000003,"15min":1.026},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82299999999999995,"5min":1.01,"15min":1.01},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.129,"15min":1.129},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0649999999999999},"last":0.69199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60548,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311292,"statfs":{"total":96636764160,"available":96574763008,"internally_reserved":0,"allocated":3072000,"data_stored":1579441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.52300000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.40500000000000003,"15min":0.41599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.31900000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.61499999999999999,"15min":0.624},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.374,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.63100000000000001,"15min":0.746},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55300000000000005,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86299999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66300000000000003,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.087},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.85199999999999998,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63400000000000001,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.95399999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":1.0609999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60552,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311288,"statfs":{"total":96636764160,"available":96574758912,"internally_reserved":0,"allocated":3076096,"data_stored":1586301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.314}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.32600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.375}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.41099999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575999,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43084,"kb_used_data":3584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328756,"statfs":{"total":96636764160,"available":96592646144,"internally_reserved":0,"allocated":3670016,"data_stored":2171210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52200000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.52300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.26100000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.51100000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.96999999999999997,"15min":0.997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72399999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.95499999999999996,"15min":0.96399999999999997},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64700000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":1.03},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70999999999999996,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.5}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.78000000000000003,"5min":0.997,"15min":0.997},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:01:23.406 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:01:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.535+0000 7f525fba9640 1 -- 172.21.15.125:0/2807493817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5240003f20 msgr2=0x7f52400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.535+0000 7f525fba9640 1 --2- 172.21.15.125:0/2807493817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5240003f20 0x7f52400047c0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f524c001680 tx=0x7f52400057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.535+0000 7f525e146640 1 -- 172.21.15.125:0/2807493817 reap_dead start 2023-10-18T12:01:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.535+0000 7f525fba9640 1 -- 172.21.15.125:0/2807493817 shutdown_connections 2023-10-18T12:01:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.535+0000 7f525fba9640 1 -- 172.21.15.125:0/2807493817 >> 172.21.15.125:0/2807493817 conn(0x7f5258069930 msgr2=0x7f5258069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.535+0000 7f525fba9640 1 -- 172.21.15.125:0/2807493817 shutdown_connections 2023-10-18T12:01:23.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.535+0000 7f525fba9640 1 -- 172.21.15.125:0/2807493817 wait complete. 2023-10-18T12:01:23.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.536+0000 7f525fba9640 1 Processor -- start 2023-10-18T12:01:23.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.536+0000 7f525fba9640 1 -- start start 2023-10-18T12:01:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525fba9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5240003f20 0x7f5258158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525fba9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52580b0810 0x7f52580b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525fba9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52581538b0 0x7f5258153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525fba9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580adbd0 con 0x7f52581538b0 2023-10-18T12:01:23.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525fba9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b11d0 con 0x7f5240003f20 2023-10-18T12:01:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525fba9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580acd40 con 0x7f52580b0810 2023-10-18T12:01:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52581538b0 0x7f5258153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52581538b0 0x7f5258153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33170/0 (socket says 172.21.15.125:33170) 2023-10-18T12:01:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 -- 172.21.15.125:0/3910071274 learned_addr learned my addr 172.21.15.125:0/3910071274 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:23.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525d144640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52580b0810 0x7f52580b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525d945640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5240003f20 0x7f5258158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 -- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52580b0810 msgr2=0x7f52580b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52580b0810 0x7f52580b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 -- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5240003f20 msgr2=0x7f5258158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5240003f20 0x7f5258158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 -- 172.21.15.125:0/3910071274 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580acf50 con 0x7f52581538b0 2023-10-18T12:01:23.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525d144640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52580b0810 0x7f52580b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:23.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525d945640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5240003f20 0x7f5258158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:23.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.537+0000 7f525e146640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52581538b0 0x7f5258153d30 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f52580cc150 tx=0x7f5248011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:23.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.538+0000 7f52567fc640 1 -- 172.21.15.125:0/3910071274 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248004500 con 0x7f52581538b0 2023-10-18T12:01:23.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.538+0000 7f52567fc640 1 -- 172.21.15.125:0/3910071274 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f52480036e0 con 0x7f52581538b0 2023-10-18T12:01:23.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.538+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52580ad230 con 0x7f52581538b0 2023-10-18T12:01:23.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.538+0000 7f52567fc640 1 -- 172.21.15.125:0/3910071274 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248003ab0 con 0x7f52581538b0 2023-10-18T12:01:23.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.538+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52580ad7f0 con 0x7f52581538b0 2023-10-18T12:01:23.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.539+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52580add10 con 0x7f52581538b0 2023-10-18T12:01:23.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.539+0000 7f52567fc640 1 -- 172.21.15.125:0/3910071274 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5248018070 con 0x7f52581538b0 2023-10-18T12:01:23.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.539+0000 7f52567fc640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f524406b0b0 0x7f524406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.540+0000 7f52567fc640 1 -- 172.21.15.125:0/3910071274 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f524808a850 con 0x7f52581538b0 2023-10-18T12:01:23.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.540+0000 7f525d945640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f524406b0b0 0x7f524406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.540+0000 7f525d945640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f524406b0b0 0x7f524406d570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5250001b20 tx=0x7f5250001250 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:23.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.544+0000 7f52567fc640 1 -- 172.21.15.125:0/3910071274 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52580add10 con 0x7f52581538b0 2023-10-18T12:01:23.709 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.706+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5258154270 con 0x7f524406b0b0 2023-10-18T12:01:23.722 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.719+0000 7f52567fc640 1 -- 172.21.15.125:0/3910071274 <== mgr.9236 v2:172.21.15.139:6832/3422518215 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+523685 (secure 0 0 0) 0x7f5258154270 con 0x7f524406b0b0 2023-10-18T12:01:23.727 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:23.731 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:01:23.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f524406b0b0 msgr2=0x7f524406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f524406b0b0 0x7f524406d570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5250001b20 tx=0x7f5250001250 comp rx=0 tx=0).stop 2023-10-18T12:01:23.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52581538b0 msgr2=0x7f5258153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52581538b0 0x7f5258153d30 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f52580cc150 tx=0x7f5248011aa0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 shutdown_connections 2023-10-18T12:01:23.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f524406b0b0 0x7f524406d570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52581538b0 0x7f5258153d30 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52580b0810 0x7f52580b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.723+0000 7f525fba9640 1 --2- 172.21.15.125:0/3910071274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5240003f20 0x7f5258158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.724+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 >> 172.21.15.125:0/3910071274 conn(0x7f5258069930 msgr2=0x7f525813cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:23.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.724+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 shutdown_connections 2023-10-18T12:01:23.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.724+0000 7f525fba9640 1 -- 172.21.15.125:0/3910071274 wait complete. 2023-10-18T12:01:23.737 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-18T12:01:22.839039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594034,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":682,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":420,"ondisk_log_size":420,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420356,"kb_used_data":25184,"kb_used_omap":12,"kb_used_meta":395059,"kb_avail":754554364,"statfs":{"total":773094113280,"available":772663668736,"internally_reserved":0,"allocated":25788416,"data_stored":13832589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":404540984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1256,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.398513"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454157+0000","last_change":"2023-10-18T12:01:19.454157+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454157+0000","last_undegraded":"2023-10-18T12:01:19.454157+0000","last_fullsized":"2023-10-18T12:01:19.454157+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465723+0000","last_change":"2023-10-18T12:01:19.464169+0000","last_active":"2023-10-18T12:01:19.465723+0000","last_peered":"2023-10-18T12:01:19.465723+0000","last_clean":"2023-10-18T12:01:19.465723+0000","last_became_active":"2023-10-18T12:01:19.463355+0000","last_became_peered":"2023-10-18T12:01:19.463355+0000","last_unstale":"2023-10-18T12:01:19.465723+0000","last_undegraded":"2023-10-18T12:01:19.465723+0000","last_fullsized":"2023-10-18T12:01:19.465723+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:00.513738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444418+0000","last_change":"2023-10-18T12:01:09.126968+0000","last_active":"2023-10-18T12:01:18.444418+0000","last_peered":"2023-10-18T12:01:18.444418+0000","last_clean":"2023-10-18T12:01:18.444418+0000","last_became_active":"2023-10-18T12:01:09.126705+0000","last_became_peered":"2023-10-18T12:01:09.126705+0000","last_unstale":"2023-10-18T12:01:18.444418+0000","last_undegraded":"2023-10-18T12:01:18.444418+0000","last_fullsized":"2023-10-18T12:01:18.444418+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:42:53.088272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453457+0000","last_change":"2023-10-18T12:01:19.453457+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453457+0000","last_undegraded":"2023-10-18T12:01:19.453457+0000","last_fullsized":"2023-10-18T12:01:19.453457+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468820+0000","last_change":"2023-10-18T12:01:19.468191+0000","last_active":"2023-10-18T12:01:19.468820+0000","last_peered":"2023-10-18T12:01:19.468820+0000","last_clean":"2023-10-18T12:01:19.468820+0000","last_became_active":"2023-10-18T12:01:19.467925+0000","last_became_peered":"2023-10-18T12:01:19.467925+0000","last_unstale":"2023-10-18T12:01:19.468820+0000","last_undegraded":"2023-10-18T12:01:19.468820+0000","last_fullsized":"2023-10-18T12:01:19.468820+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:58.124933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439901+0000","last_change":"2023-10-18T12:01:09.125772+0000","last_active":"2023-10-18T12:01:18.439901+0000","last_peered":"2023-10-18T12:01:18.439901+0000","last_clean":"2023-10-18T12:01:18.439901+0000","last_became_active":"2023-10-18T12:01:09.125508+0000","last_became_peered":"2023-10-18T12:01:09.125508+0000","last_unstale":"2023-10-18T12:01:18.439901+0000","last_undegraded":"2023-10-18T12:01:18.439901+0000","last_fullsized":"2023-10-18T12:01:18.439901+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:15.841764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451128+0000","last_change":"2023-10-18T12:01:19.451128+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451128+0000","last_undegraded":"2023-10-18T12:01:19.451128+0000","last_fullsized":"2023-10-18T12:01:19.451128+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.467291+0000","last_change":"2023-10-18T12:01:19.466687+0000","last_active":"2023-10-18T12:01:19.467291+0000","last_peered":"2023-10-18T12:01:19.467291+0000","last_clean":"2023-10-18T12:01:19.467291+0000","last_became_active":"2023-10-18T12:01:19.466395+0000","last_became_peered":"2023-10-18T12:01:19.466395+0000","last_unstale":"2023-10-18T12:01:19.467291+0000","last_undegraded":"2023-10-18T12:01:19.467291+0000","last_fullsized":"2023-10-18T12:01:19.467291+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:02.185827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444714+0000","last_change":"2023-10-18T12:01:09.124764+0000","last_active":"2023-10-18T12:01:18.444714+0000","last_peered":"2023-10-18T12:01:18.444714+0000","last_clean":"2023-10-18T12:01:18.444714+0000","last_became_active":"2023-10-18T12:01:09.124472+0000","last_became_peered":"2023-10-18T12:01:09.124472+0000","last_unstale":"2023-10-18T12:01:18.444714+0000","last_undegraded":"2023-10-18T12:01:18.444714+0000","last_fullsized":"2023-10-18T12:01:18.444714+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:29.891668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454232+0000","last_change":"2023-10-18T12:01:19.454232+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454232+0000","last_undegraded":"2023-10-18T12:01:19.454232+0000","last_fullsized":"2023-10-18T12:01:19.454232+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466648+0000","last_change":"2023-10-18T12:01:19.466181+0000","last_active":"2023-10-18T12:01:19.466648+0000","last_peered":"2023-10-18T12:01:19.466648+0000","last_clean":"2023-10-18T12:01:19.466648+0000","last_became_active":"2023-10-18T12:01:19.465901+0000","last_became_peered":"2023-10-18T12:01:19.465901+0000","last_unstale":"2023-10-18T12:01:19.466648+0000","last_undegraded":"2023-10-18T12:01:19.466648+0000","last_fullsized":"2023-10-18T12:01:19.466648+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:26:57.818590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444737+0000","last_change":"2023-10-18T12:01:09.132202+0000","last_active":"2023-10-18T12:01:18.444737+0000","last_peered":"2023-10-18T12:01:18.444737+0000","last_clean":"2023-10-18T12:01:18.444737+0000","last_became_active":"2023-10-18T12:01:09.131928+0000","last_became_peered":"2023-10-18T12:01:09.131928+0000","last_unstale":"2023-10-18T12:01:18.444737+0000","last_undegraded":"2023-10-18T12:01:18.444737+0000","last_fullsized":"2023-10-18T12:01:18.444737+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:34.826098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450577+0000","last_change":"2023-10-18T12:01:19.450577+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450577+0000","last_undegraded":"2023-10-18T12:01:19.450577+0000","last_fullsized":"2023-10-18T12:01:19.450577+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.461102+0000","last_change":"2023-10-18T12:01:19.459438+0000","last_active":"2023-10-18T12:01:19.461102+0000","last_peered":"2023-10-18T12:01:19.461102+0000","last_clean":"2023-10-18T12:01:19.461102+0000","last_became_active":"2023-10-18T12:01:19.459194+0000","last_became_peered":"2023-10-18T12:01:19.459194+0000","last_unstale":"2023-10-18T12:01:19.461102+0000","last_undegraded":"2023-10-18T12:01:19.461102+0000","last_fullsized":"2023-10-18T12:01:19.461102+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:53.933731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446379+0000","last_change":"2023-10-18T12:01:09.133870+0000","last_active":"2023-10-18T12:01:18.446379+0000","last_peered":"2023-10-18T12:01:18.446379+0000","last_clean":"2023-10-18T12:01:18.446379+0000","last_became_active":"2023-10-18T12:01:09.133478+0000","last_became_peered":"2023-10-18T12:01:09.133478+0000","last_unstale":"2023-10-18T12:01:18.446379+0000","last_undegraded":"2023-10-18T12:01:18.446379+0000","last_fullsized":"2023-10-18T12:01:18.446379+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:11.836101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450431+0000","last_change":"2023-10-18T12:01:19.450431+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450431+0000","last_undegraded":"2023-10-18T12:01:19.450431+0000","last_fullsized":"2023-10-18T12:01:19.450431+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465762+0000","last_change":"2023-10-18T12:01:19.463436+0000","last_active":"2023-10-18T12:01:19.465762+0000","last_peered":"2023-10-18T12:01:19.465762+0000","last_clean":"2023-10-18T12:01:19.465762+0000","last_became_active":"2023-10-18T12:01:19.463190+0000","last_became_peered":"2023-10-18T12:01:19.463190+0000","last_unstale":"2023-10-18T12:01:19.465762+0000","last_undegraded":"2023-10-18T12:01:19.465762+0000","last_fullsized":"2023-10-18T12:01:19.465762+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:55.180369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443988+0000","last_change":"2023-10-18T12:01:09.129113+0000","last_active":"2023-10-18T12:01:18.443988+0000","last_peered":"2023-10-18T12:01:18.443988+0000","last_clean":"2023-10-18T12:01:18.443988+0000","last_became_active":"2023-10-18T12:01:09.128785+0000","last_became_peered":"2023-10-18T12:01:09.128785+0000","last_unstale":"2023-10-18T12:01:18.443988+0000","last_undegraded":"2023-10-18T12:01:18.443988+0000","last_fullsized":"2023-10-18T12:01:18.443988+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:39.875471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458863+0000","last_change":"2023-10-18T12:01:19.458863+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458863+0000","last_undegraded":"2023-10-18T12:01:19.458863+0000","last_fullsized":"2023-10-18T12:01:19.458863+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465879+0000","last_change":"2023-10-18T12:01:19.463023+0000","last_active":"2023-10-18T12:01:19.465879+0000","last_peered":"2023-10-18T12:01:19.465879+0000","last_clean":"2023-10-18T12:01:19.465879+0000","last_became_active":"2023-10-18T12:01:19.462757+0000","last_became_peered":"2023-10-18T12:01:19.462757+0000","last_unstale":"2023-10-18T12:01:19.465879+0000","last_undegraded":"2023-10-18T12:01:19.465879+0000","last_fullsized":"2023-10-18T12:01:19.465879+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:09.123947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445459+0000","last_change":"2023-10-18T12:01:09.125901+0000","last_active":"2023-10-18T12:01:18.445459+0000","last_peered":"2023-10-18T12:01:18.445459+0000","last_clean":"2023-10-18T12:01:18.445459+0000","last_became_active":"2023-10-18T12:01:09.125390+0000","last_became_peered":"2023-10-18T12:01:09.125390+0000","last_unstale":"2023-10-18T12:01:18.445459+0000","last_undegraded":"2023-10-18T12:01:18.445459+0000","last_fullsized":"2023-10-18T12:01:18.445459+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:00.246587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451418+0000","last_change":"2023-10-18T12:01:19.451418+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451418+0000","last_undegraded":"2023-10-18T12:01:19.451418+0000","last_fullsized":"2023-10-18T12:01:19.451418+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465285+0000","last_change":"2023-10-18T12:01:19.463793+0000","last_active":"2023-10-18T12:01:19.465285+0000","last_peered":"2023-10-18T12:01:19.465285+0000","last_clean":"2023-10-18T12:01:19.465285+0000","last_became_active":"2023-10-18T12:01:19.463343+0000","last_became_peered":"2023-10-18T12:01:19.463343+0000","last_unstale":"2023-10-18T12:01:19.465285+0000","last_undegraded":"2023-10-18T12:01:19.465285+0000","last_fullsized":"2023-10-18T12:01:19.465285+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:34.979933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446196+0000","last_change":"2023-10-18T12:01:09.135081+0000","last_active":"2023-10-18T12:01:18.446196+0000","last_peered":"2023-10-18T12:01:18.446196+0000","last_clean":"2023-10-18T12:01:18.446196+0000","last_became_active":"2023-10-18T12:01:09.134801+0000","last_became_peered":"2023-10-18T12:01:09.134801+0000","last_unstale":"2023-10-18T12:01:18.446196+0000","last_undegraded":"2023-10-18T12:01:18.446196+0000","last_fullsized":"2023-10-18T12:01:18.446196+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:03.262316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452320+0000","last_change":"2023-10-18T12:01:19.452320+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452320+0000","last_undegraded":"2023-10-18T12:01:19.452320+0000","last_fullsized":"2023-10-18T12:01:19.452320+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466738+0000","last_change":"2023-10-18T12:01:19.465635+0000","last_active":"2023-10-18T12:01:19.466738+0000","last_peered":"2023-10-18T12:01:19.466738+0000","last_clean":"2023-10-18T12:01:19.466738+0000","last_became_active":"2023-10-18T12:01:19.465363+0000","last_became_peered":"2023-10-18T12:01:19.465363+0000","last_unstale":"2023-10-18T12:01:19.466738+0000","last_undegraded":"2023-10-18T12:01:19.466738+0000","last_fullsized":"2023-10-18T12:01:19.466738+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:56.587842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444490+0000","last_change":"2023-10-18T12:01:09.135615+0000","last_active":"2023-10-18T12:01:18.444490+0000","last_peered":"2023-10-18T12:01:18.444490+0000","last_clean":"2023-10-18T12:01:18.444490+0000","last_became_active":"2023-10-18T12:01:09.135123+0000","last_became_peered":"2023-10-18T12:01:09.135123+0000","last_unstale":"2023-10-18T12:01:18.444490+0000","last_undegraded":"2023-10-18T12:01:18.444490+0000","last_fullsized":"2023-10-18T12:01:18.444490+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:52.311354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458411+0000","last_change":"2023-10-18T12:01:19.458411+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458411+0000","last_undegraded":"2023-10-18T12:01:19.458411+0000","last_fullsized":"2023-10-18T12:01:19.458411+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465526+0000","last_change":"2023-10-18T12:01:19.465001+0000","last_active":"2023-10-18T12:01:19.465526+0000","last_peered":"2023-10-18T12:01:19.465526+0000","last_clean":"2023-10-18T12:01:19.465526+0000","last_became_active":"2023-10-18T12:01:19.464719+0000","last_became_peered":"2023-10-18T12:01:19.464719+0000","last_unstale":"2023-10-18T12:01:19.465526+0000","last_undegraded":"2023-10-18T12:01:19.465526+0000","last_fullsized":"2023-10-18T12:01:19.465526+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:08:12.543232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446023+0000","last_change":"2023-10-18T12:01:09.130673+0000","last_active":"2023-10-18T12:01:18.446023+0000","last_peered":"2023-10-18T12:01:18.446023+0000","last_clean":"2023-10-18T12:01:18.446023+0000","last_became_active":"2023-10-18T12:01:09.130094+0000","last_became_peered":"2023-10-18T12:01:09.130094+0000","last_unstale":"2023-10-18T12:01:18.446023+0000","last_undegraded":"2023-10-18T12:01:18.446023+0000","last_fullsized":"2023-10-18T12:01:18.446023+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:45.652023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456984+0000","last_change":"2023-10-18T12:01:19.456984+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456984+0000","last_undegraded":"2023-10-18T12:01:19.456984+0000","last_fullsized":"2023-10-18T12:01:19.456984+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465010+0000","last_change":"2023-10-18T12:01:19.463067+0000","last_active":"2023-10-18T12:01:19.465010+0000","last_peered":"2023-10-18T12:01:19.465010+0000","last_clean":"2023-10-18T12:01:19.465010+0000","last_became_active":"2023-10-18T12:01:19.462818+0000","last_became_peered":"2023-10-18T12:01:19.462818+0000","last_unstale":"2023-10-18T12:01:19.465010+0000","last_undegraded":"2023-10-18T12:01:19.465010+0000","last_fullsized":"2023-10-18T12:01:19.465010+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:50:23.807977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444802+0000","last_change":"2023-10-18T12:01:09.135828+0000","last_active":"2023-10-18T12:01:18.444802+0000","last_peered":"2023-10-18T12:01:18.444802+0000","last_clean":"2023-10-18T12:01:18.444802+0000","last_became_active":"2023-10-18T12:01:09.135579+0000","last_became_peered":"2023-10-18T12:01:09.135579+0000","last_unstale":"2023-10-18T12:01:18.444802+0000","last_undegraded":"2023-10-18T12:01:18.444802+0000","last_fullsized":"2023-10-18T12:01:18.444802+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:06.517585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454999+0000","last_change":"2023-10-18T12:01:19.454999+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454999+0000","last_undegraded":"2023-10-18T12:01:19.454999+0000","last_fullsized":"2023-10-18T12:01:19.454999+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.464639+0000","last_change":"2023-10-18T12:01:19.462716+0000","last_active":"2023-10-18T12:01:19.464639+0000","last_peered":"2023-10-18T12:01:19.464639+0000","last_clean":"2023-10-18T12:01:19.464639+0000","last_became_active":"2023-10-18T12:01:19.462474+0000","last_became_peered":"2023-10-18T12:01:19.462474+0000","last_unstale":"2023-10-18T12:01:19.464639+0000","last_undegraded":"2023-10-18T12:01:19.464639+0000","last_fullsized":"2023-10-18T12:01:19.464639+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:04.329561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443892+0000","last_change":"2023-10-18T12:01:09.132939+0000","last_active":"2023-10-18T12:01:18.443892+0000","last_peered":"2023-10-18T12:01:18.443892+0000","last_clean":"2023-10-18T12:01:18.443892+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:18.443892+0000","last_undegraded":"2023-10-18T12:01:18.443892+0000","last_fullsized":"2023-10-18T12:01:18.443892+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:22.550156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455867+0000","last_change":"2023-10-18T12:01:19.455867+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455867+0000","last_undegraded":"2023-10-18T12:01:19.455867+0000","last_fullsized":"2023-10-18T12:01:19.455867+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.459783+0000","last_change":"2023-10-18T12:01:19.458472+0000","last_active":"2023-10-18T12:01:19.459783+0000","last_peered":"2023-10-18T12:01:19.459783+0000","last_clean":"2023-10-18T12:01:19.459783+0000","last_became_active":"2023-10-18T12:01:19.458209+0000","last_became_peered":"2023-10-18T12:01:19.458209+0000","last_unstale":"2023-10-18T12:01:19.459783+0000","last_undegraded":"2023-10-18T12:01:19.459783+0000","last_fullsized":"2023-10-18T12:01:19.459783+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:04.778253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444252+0000","last_change":"2023-10-18T12:01:09.129764+0000","last_active":"2023-10-18T12:01:18.444252+0000","last_peered":"2023-10-18T12:01:18.444252+0000","last_clean":"2023-10-18T12:01:18.444252+0000","last_became_active":"2023-10-18T12:01:09.129483+0000","last_became_peered":"2023-10-18T12:01:09.129483+0000","last_unstale":"2023-10-18T12:01:18.444252+0000","last_undegraded":"2023-10-18T12:01:18.444252+0000","last_fullsized":"2023-10-18T12:01:18.444252+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:03.037837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453014+0000","last_change":"2023-10-18T12:01:19.453014+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453014+0000","last_undegraded":"2023-10-18T12:01:19.453014+0000","last_fullsized":"2023-10-18T12:01:19.453014+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468149+0000","last_change":"2023-10-18T12:01:19.466751+0000","last_active":"2023-10-18T12:01:19.468149+0000","last_peered":"2023-10-18T12:01:19.468149+0000","last_clean":"2023-10-18T12:01:19.468149+0000","last_became_active":"2023-10-18T12:01:19.466171+0000","last_became_peered":"2023-10-18T12:01:19.466171+0000","last_unstale":"2023-10-18T12:01:19.468149+0000","last_undegraded":"2023-10-18T12:01:19.468149+0000","last_fullsized":"2023-10-18T12:01:19.468149+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:01.102496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448554+0000","last_change":"2023-10-18T12:01:09.133024+0000","last_active":"2023-10-18T12:01:18.448554+0000","last_peered":"2023-10-18T12:01:18.448554+0000","last_clean":"2023-10-18T12:01:18.448554+0000","last_became_active":"2023-10-18T12:01:09.132742+0000","last_became_peered":"2023-10-18T12:01:09.132742+0000","last_unstale":"2023-10-18T12:01:18.448554+0000","last_undegraded":"2023-10-18T12:01:18.448554+0000","last_fullsized":"2023-10-18T12:01:18.448554+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456037+0000","last_change":"2023-10-18T12:01:19.456037+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456037+0000","last_undegraded":"2023-10-18T12:01:19.456037+0000","last_fullsized":"2023-10-18T12:01:19.456037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465243+0000","last_change":"2023-10-18T12:01:19.464582+0000","last_active":"2023-10-18T12:01:19.465243+0000","last_peered":"2023-10-18T12:01:19.465243+0000","last_clean":"2023-10-18T12:01:19.465243+0000","last_became_active":"2023-10-18T12:01:19.464325+0000","last_became_peered":"2023-10-18T12:01:19.464325+0000","last_unstale":"2023-10-18T12:01:19.465243+0000","last_undegraded":"2023-10-18T12:01:19.465243+0000","last_fullsized":"2023-10-18T12:01:19.465243+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:32:16.943070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445969+0000","last_change":"2023-10-18T12:01:09.131147+0000","last_active":"2023-10-18T12:01:18.445969+0000","last_peered":"2023-10-18T12:01:18.445969+0000","last_clean":"2023-10-18T12:01:18.445969+0000","last_became_active":"2023-10-18T12:01:09.130682+0000","last_became_peered":"2023-10-18T12:01:09.130682+0000","last_unstale":"2023-10-18T12:01:18.445969+0000","last_undegraded":"2023-10-18T12:01:18.445969+0000","last_fullsized":"2023-10-18T12:01:18.445969+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453907+0000","last_change":"2023-10-18T12:01:19.453907+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453907+0000","last_undegraded":"2023-10-18T12:01:19.453907+0000","last_fullsized":"2023-10-18T12:01:19.453907+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463821+0000","last_change":"2023-10-18T12:01:19.459426+0000","last_active":"2023-10-18T12:01:19.463821+0000","last_peered":"2023-10-18T12:01:19.463821+0000","last_clean":"2023-10-18T12:01:19.463821+0000","last_became_active":"2023-10-18T12:01:19.459168+0000","last_became_peered":"2023-10-18T12:01:19.459168+0000","last_unstale":"2023-10-18T12:01:19.463821+0000","last_undegraded":"2023-10-18T12:01:19.463821+0000","last_fullsized":"2023-10-18T12:01:19.463821+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:00.882760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445455+0000","last_change":"2023-10-18T12:01:09.122098+0000","last_active":"2023-10-18T12:01:18.445455+0000","last_peered":"2023-10-18T12:01:18.445455+0000","last_clean":"2023-10-18T12:01:18.445455+0000","last_became_active":"2023-10-18T12:01:09.121845+0000","last_became_peered":"2023-10-18T12:01:09.121845+0000","last_unstale":"2023-10-18T12:01:18.445455+0000","last_undegraded":"2023-10-18T12:01:18.445455+0000","last_fullsized":"2023-10-18T12:01:18.445455+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:19.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453747+0000","last_change":"2023-10-18T12:01:19.453747+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453747+0000","last_undegraded":"2023-10-18T12:01:19.453747+0000","last_fullsized":"2023-10-18T12:01:19.453747+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468474+0000","last_change":"2023-10-18T12:01:19.466123+0000","last_active":"2023-10-18T12:01:19.468474+0000","last_peered":"2023-10-18T12:01:19.468474+0000","last_clean":"2023-10-18T12:01:19.468474+0000","last_became_active":"2023-10-18T12:01:19.465856+0000","last_became_peered":"2023-10-18T12:01:19.465856+0000","last_unstale":"2023-10-18T12:01:19.468474+0000","last_undegraded":"2023-10-18T12:01:19.468474+0000","last_fullsized":"2023-10-18T12:01:19.468474+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:53.379907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447103+0000","last_change":"2023-10-18T12:01:09.133999+0000","last_active":"2023-10-18T12:01:18.447103+0000","last_peered":"2023-10-18T12:01:18.447103+0000","last_clean":"2023-10-18T12:01:18.447103+0000","last_became_active":"2023-10-18T12:01:09.133725+0000","last_became_peered":"2023-10-18T12:01:09.133725+0000","last_unstale":"2023-10-18T12:01:18.447103+0000","last_undegraded":"2023-10-18T12:01:18.447103+0000","last_fullsized":"2023-10-18T12:01:18.447103+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:35.006911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459506+0000","last_change":"2023-10-18T12:01:19.459506+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459506+0000","last_undegraded":"2023-10-18T12:01:19.459506+0000","last_fullsized":"2023-10-18T12:01:19.459506+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468784+0000","last_change":"2023-10-18T12:01:19.468308+0000","last_active":"2023-10-18T12:01:19.468784+0000","last_peered":"2023-10-18T12:01:19.468784+0000","last_clean":"2023-10-18T12:01:19.468784+0000","last_became_active":"2023-10-18T12:01:19.467998+0000","last_became_peered":"2023-10-18T12:01:19.467998+0000","last_unstale":"2023-10-18T12:01:19.468784+0000","last_undegraded":"2023-10-18T12:01:19.468784+0000","last_fullsized":"2023-10-18T12:01:19.468784+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:17.064462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446111+0000","last_change":"2023-10-18T12:01:09.124908+0000","last_active":"2023-10-18T12:01:18.446111+0000","last_peered":"2023-10-18T12:01:18.446111+0000","last_clean":"2023-10-18T12:01:18.446111+0000","last_became_active":"2023-10-18T12:01:09.124565+0000","last_became_peered":"2023-10-18T12:01:09.124565+0000","last_unstale":"2023-10-18T12:01:18.446111+0000","last_undegraded":"2023-10-18T12:01:18.446111+0000","last_fullsized":"2023-10-18T12:01:18.446111+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:22.044677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453595+0000","last_change":"2023-10-18T12:01:19.453595+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453595+0000","last_undegraded":"2023-10-18T12:01:19.453595+0000","last_fullsized":"2023-10-18T12:01:19.453595+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468762+0000","last_change":"2023-10-18T12:01:19.467971+0000","last_active":"2023-10-18T12:01:19.468762+0000","last_peered":"2023-10-18T12:01:19.468762+0000","last_clean":"2023-10-18T12:01:19.468762+0000","last_became_active":"2023-10-18T12:01:19.467699+0000","last_became_peered":"2023-10-18T12:01:19.467699+0000","last_unstale":"2023-10-18T12:01:19.468762+0000","last_undegraded":"2023-10-18T12:01:19.468762+0000","last_fullsized":"2023-10-18T12:01:19.468762+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:01.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447225+0000","last_change":"2023-10-18T12:01:09.135302+0000","last_active":"2023-10-18T12:01:18.447225+0000","last_peered":"2023-10-18T12:01:18.447225+0000","last_clean":"2023-10-18T12:01:18.447225+0000","last_became_active":"2023-10-18T12:01:09.134978+0000","last_became_peered":"2023-10-18T12:01:09.134978+0000","last_unstale":"2023-10-18T12:01:18.447225+0000","last_undegraded":"2023-10-18T12:01:18.447225+0000","last_fullsized":"2023-10-18T12:01:18.447225+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452783+0000","last_change":"2023-10-18T12:01:19.452783+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452783+0000","last_undegraded":"2023-10-18T12:01:19.452783+0000","last_fullsized":"2023-10-18T12:01:19.452783+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463777+0000","last_change":"2023-10-18T12:01:19.461558+0000","last_active":"2023-10-18T12:01:19.463777+0000","last_peered":"2023-10-18T12:01:19.463777+0000","last_clean":"2023-10-18T12:01:19.463777+0000","last_became_active":"2023-10-18T12:01:19.461307+0000","last_became_peered":"2023-10-18T12:01:19.461307+0000","last_unstale":"2023-10-18T12:01:19.463777+0000","last_undegraded":"2023-10-18T12:01:19.463777+0000","last_fullsized":"2023-10-18T12:01:19.463777+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:18:26.034701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440010+0000","last_change":"2023-10-18T12:01:09.136717+0000","last_active":"2023-10-18T12:01:18.440010+0000","last_peered":"2023-10-18T12:01:18.440010+0000","last_clean":"2023-10-18T12:01:18.440010+0000","last_became_active":"2023-10-18T12:01:09.136461+0000","last_became_peered":"2023-10-18T12:01:09.136461+0000","last_unstale":"2023-10-18T12:01:18.440010+0000","last_undegraded":"2023-10-18T12:01:18.440010+0000","last_fullsized":"2023-10-18T12:01:18.440010+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:02.336311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452844+0000","last_change":"2023-10-18T12:01:19.452844+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452844+0000","last_undegraded":"2023-10-18T12:01:19.452844+0000","last_fullsized":"2023-10-18T12:01:19.452844+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463533+0000","last_change":"2023-10-18T12:01:19.462240+0000","last_active":"2023-10-18T12:01:19.463533+0000","last_peered":"2023-10-18T12:01:19.463533+0000","last_clean":"2023-10-18T12:01:19.463533+0000","last_became_active":"2023-10-18T12:01:19.461957+0000","last_became_peered":"2023-10-18T12:01:19.461957+0000","last_unstale":"2023-10-18T12:01:19.463533+0000","last_undegraded":"2023-10-18T12:01:19.463533+0000","last_fullsized":"2023-10-18T12:01:19.463533+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:20.506577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445002+0000","last_change":"2023-10-18T12:01:09.130016+0000","last_active":"2023-10-18T12:01:18.445002+0000","last_peered":"2023-10-18T12:01:18.445002+0000","last_clean":"2023-10-18T12:01:18.445002+0000","last_became_active":"2023-10-18T12:01:09.129718+0000","last_became_peered":"2023-10-18T12:01:09.129718+0000","last_unstale":"2023-10-18T12:01:18.445002+0000","last_undegraded":"2023-10-18T12:01:18.445002+0000","last_fullsized":"2023-10-18T12:01:18.445002+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:36.455465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450045+0000","last_change":"2023-10-18T12:01:19.450045+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450045+0000","last_undegraded":"2023-10-18T12:01:19.450045+0000","last_fullsized":"2023-10-18T12:01:19.450045+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468581+0000","last_change":"2023-10-18T12:01:19.467158+0000","last_active":"2023-10-18T12:01:19.468581+0000","last_peered":"2023-10-18T12:01:19.468581+0000","last_clean":"2023-10-18T12:01:19.468581+0000","last_became_active":"2023-10-18T12:01:19.466854+0000","last_became_peered":"2023-10-18T12:01:19.466854+0000","last_unstale":"2023-10-18T12:01:19.468581+0000","last_undegraded":"2023-10-18T12:01:19.468581+0000","last_fullsized":"2023-10-18T12:01:19.468581+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:21.428576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443731+0000","last_change":"2023-10-18T12:01:09.129706+0000","last_active":"2023-10-18T12:01:18.443731+0000","last_peered":"2023-10-18T12:01:18.443731+0000","last_clean":"2023-10-18T12:01:18.443731+0000","last_became_active":"2023-10-18T12:01:09.129413+0000","last_became_peered":"2023-10-18T12:01:09.129413+0000","last_unstale":"2023-10-18T12:01:18.443731+0000","last_undegraded":"2023-10-18T12:01:18.443731+0000","last_fullsized":"2023-10-18T12:01:18.443731+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:55.309809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.449803+0000","last_change":"2023-10-18T12:01:19.449803+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.449803+0000","last_undegraded":"2023-10-18T12:01:19.449803+0000","last_fullsized":"2023-10-18T12:01:19.449803+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468823+0000","last_change":"2023-10-18T12:01:19.467623+0000","last_active":"2023-10-18T12:01:19.468823+0000","last_peered":"2023-10-18T12:01:19.468823+0000","last_clean":"2023-10-18T12:01:19.468823+0000","last_became_active":"2023-10-18T12:01:19.467358+0000","last_became_peered":"2023-10-18T12:01:19.467358+0000","last_unstale":"2023-10-18T12:01:19.468823+0000","last_undegraded":"2023-10-18T12:01:19.468823+0000","last_fullsized":"2023-10-18T12:01:19.468823+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:58.681098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446063+0000","last_change":"2023-10-18T12:01:09.133291+0000","last_active":"2023-10-18T12:01:18.446063+0000","last_peered":"2023-10-18T12:01:18.446063+0000","last_clean":"2023-10-18T12:01:18.446063+0000","last_became_active":"2023-10-18T12:01:09.132968+0000","last_became_peered":"2023-10-18T12:01:09.132968+0000","last_unstale":"2023-10-18T12:01:18.446063+0000","last_undegraded":"2023-10-18T12:01:18.446063+0000","last_fullsized":"2023-10-18T12:01:18.446063+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454604+0000","last_change":"2023-10-18T12:01:19.454604+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454604+0000","last_undegraded":"2023-10-18T12:01:19.454604+0000","last_fullsized":"2023-10-18T12:01:19.454604+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.460591+0000","last_change":"2023-10-18T12:01:19.458380+0000","last_active":"2023-10-18T12:01:19.460591+0000","last_peered":"2023-10-18T12:01:19.460591+0000","last_clean":"2023-10-18T12:01:19.460591+0000","last_became_active":"2023-10-18T12:01:19.458111+0000","last_became_peered":"2023-10-18T12:01:19.458111+0000","last_unstale":"2023-10-18T12:01:19.460591+0000","last_undegraded":"2023-10-18T12:01:19.460591+0000","last_fullsized":"2023-10-18T12:01:19.460591+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:27.878015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443343+0000","last_change":"2023-10-18T12:01:09.128954+0000","last_active":"2023-10-18T12:01:18.443343+0000","last_peered":"2023-10-18T12:01:18.443343+0000","last_clean":"2023-10-18T12:01:18.443343+0000","last_became_active":"2023-10-18T12:01:09.128655+0000","last_became_peered":"2023-10-18T12:01:09.128655+0000","last_unstale":"2023-10-18T12:01:18.443343+0000","last_undegraded":"2023-10-18T12:01:18.443343+0000","last_fullsized":"2023-10-18T12:01:18.443343+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:51.625573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455208+0000","last_change":"2023-10-18T12:01:19.455208+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455208+0000","last_undegraded":"2023-10-18T12:01:19.455208+0000","last_fullsized":"2023-10-18T12:01:19.455208+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468453+0000","last_change":"2023-10-18T12:01:19.466027+0000","last_active":"2023-10-18T12:01:19.468453+0000","last_peered":"2023-10-18T12:01:19.468453+0000","last_clean":"2023-10-18T12:01:19.468453+0000","last_became_active":"2023-10-18T12:01:19.465745+0000","last_became_peered":"2023-10-18T12:01:19.465745+0000","last_unstale":"2023-10-18T12:01:19.468453+0000","last_undegraded":"2023-10-18T12:01:19.468453+0000","last_fullsized":"2023-10-18T12:01:19.468453+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:15.002306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440030+0000","last_change":"2023-10-18T12:01:09.123355+0000","last_active":"2023-10-18T12:01:18.440030+0000","last_peered":"2023-10-18T12:01:18.440030+0000","last_clean":"2023-10-18T12:01:18.440030+0000","last_became_active":"2023-10-18T12:01:09.123096+0000","last_became_peered":"2023-10-18T12:01:09.123096+0000","last_unstale":"2023-10-18T12:01:18.440030+0000","last_undegraded":"2023-10-18T12:01:18.440030+0000","last_fullsized":"2023-10-18T12:01:18.440030+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:58.130314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454904+0000","last_change":"2023-10-18T12:01:19.454904+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454904+0000","last_undegraded":"2023-10-18T12:01:19.454904+0000","last_fullsized":"2023-10-18T12:01:19.454904+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468820+0000","last_change":"2023-10-18T12:01:19.467893+0000","last_active":"2023-10-18T12:01:19.468820+0000","last_peered":"2023-10-18T12:01:19.468820+0000","last_clean":"2023-10-18T12:01:19.468820+0000","last_became_active":"2023-10-18T12:01:19.467430+0000","last_became_peered":"2023-10-18T12:01:19.467430+0000","last_unstale":"2023-10-18T12:01:19.468820+0000","last_undegraded":"2023-10-18T12:01:19.468820+0000","last_fullsized":"2023-10-18T12:01:19.468820+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:12.990393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447739+0000","last_change":"2023-10-18T12:01:09.124947+0000","last_active":"2023-10-18T12:01:18.447739+0000","last_peered":"2023-10-18T12:01:18.447739+0000","last_clean":"2023-10-18T12:01:18.447739+0000","last_became_active":"2023-10-18T12:01:09.124666+0000","last_became_peered":"2023-10-18T12:01:09.124666+0000","last_unstale":"2023-10-18T12:01:18.447739+0000","last_undegraded":"2023-10-18T12:01:18.447739+0000","last_fullsized":"2023-10-18T12:01:18.447739+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:01.214167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454093+0000","last_change":"2023-10-18T12:01:19.454093+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454093+0000","last_undegraded":"2023-10-18T12:01:19.454093+0000","last_fullsized":"2023-10-18T12:01:19.454093+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463415+0000","last_change":"2023-10-18T12:01:19.462049+0000","last_active":"2023-10-18T12:01:19.463415+0000","last_peered":"2023-10-18T12:01:19.463415+0000","last_clean":"2023-10-18T12:01:19.463415+0000","last_became_active":"2023-10-18T12:01:19.460286+0000","last_became_peered":"2023-10-18T12:01:19.460286+0000","last_unstale":"2023-10-18T12:01:19.463415+0000","last_undegraded":"2023-10-18T12:01:19.463415+0000","last_fullsized":"2023-10-18T12:01:19.463415+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:59.023770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447065+0000","last_change":"2023-10-18T12:01:09.131674+0000","last_active":"2023-10-18T12:01:18.447065+0000","last_peered":"2023-10-18T12:01:18.447065+0000","last_clean":"2023-10-18T12:01:18.447065+0000","last_became_active":"2023-10-18T12:01:09.131410+0000","last_became_peered":"2023-10-18T12:01:09.131410+0000","last_unstale":"2023-10-18T12:01:18.447065+0000","last_undegraded":"2023-10-18T12:01:18.447065+0000","last_fullsized":"2023-10-18T12:01:18.447065+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:22.606198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451954+0000","last_change":"2023-10-18T12:01:19.451954+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451954+0000","last_undegraded":"2023-10-18T12:01:19.451954+0000","last_fullsized":"2023-10-18T12:01:19.451954+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463757+0000","last_change":"2023-10-18T12:01:19.460871+0000","last_active":"2023-10-18T12:01:19.463757+0000","last_peered":"2023-10-18T12:01:19.463757+0000","last_clean":"2023-10-18T12:01:19.463757+0000","last_became_active":"2023-10-18T12:01:19.460570+0000","last_became_peered":"2023-10-18T12:01:19.460570+0000","last_unstale":"2023-10-18T12:01:19.463757+0000","last_undegraded":"2023-10-18T12:01:19.463757+0000","last_fullsized":"2023-10-18T12:01:19.463757+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:21.229838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443961+0000","last_change":"2023-10-18T12:01:09.134865+0000","last_active":"2023-10-18T12:01:18.443961+0000","last_peered":"2023-10-18T12:01:18.443961+0000","last_clean":"2023-10-18T12:01:18.443961+0000","last_became_active":"2023-10-18T12:01:09.134583+0000","last_became_peered":"2023-10-18T12:01:09.134583+0000","last_unstale":"2023-10-18T12:01:18.443961+0000","last_undegraded":"2023-10-18T12:01:18.443961+0000","last_fullsized":"2023-10-18T12:01:18.443961+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:42.480656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.449313+0000","last_change":"2023-10-18T12:01:19.449313+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.449313+0000","last_undegraded":"2023-10-18T12:01:19.449313+0000","last_fullsized":"2023-10-18T12:01:19.449313+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.464534+0000","last_change":"2023-10-18T12:01:19.461910+0000","last_active":"2023-10-18T12:01:19.464534+0000","last_peered":"2023-10-18T12:01:19.464534+0000","last_clean":"2023-10-18T12:01:19.464534+0000","last_became_active":"2023-10-18T12:01:19.461653+0000","last_became_peered":"2023-10-18T12:01:19.461653+0000","last_unstale":"2023-10-18T12:01:19.464534+0000","last_undegraded":"2023-10-18T12:01:19.464534+0000","last_fullsized":"2023-10-18T12:01:19.464534+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:16:33.411957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448538+0000","last_change":"2023-10-18T12:01:09.132312+0000","last_active":"2023-10-18T12:01:18.448538+0000","last_peered":"2023-10-18T12:01:18.448538+0000","last_clean":"2023-10-18T12:01:18.448538+0000","last_became_active":"2023-10-18T12:01:09.131781+0000","last_became_peered":"2023-10-18T12:01:09.131781+0000","last_unstale":"2023-10-18T12:01:18.448538+0000","last_undegraded":"2023-10-18T12:01:18.448538+0000","last_fullsized":"2023-10-18T12:01:18.448538+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457884+0000","last_change":"2023-10-18T12:01:19.457884+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457884+0000","last_undegraded":"2023-10-18T12:01:19.457884+0000","last_fullsized":"2023-10-18T12:01:19.457884+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466498+0000","last_change":"2023-10-18T12:01:19.465117+0000","last_active":"2023-10-18T12:01:19.466498+0000","last_peered":"2023-10-18T12:01:19.466498+0000","last_clean":"2023-10-18T12:01:19.466498+0000","last_became_active":"2023-10-18T12:01:19.464845+0000","last_became_peered":"2023-10-18T12:01:19.464845+0000","last_unstale":"2023-10-18T12:01:19.466498+0000","last_undegraded":"2023-10-18T12:01:19.466498+0000","last_fullsized":"2023-10-18T12:01:19.466498+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:22.112086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448553+0000","last_change":"2023-10-18T12:01:09.133664+0000","last_active":"2023-10-18T12:01:18.448553+0000","last_peered":"2023-10-18T12:01:18.448553+0000","last_clean":"2023-10-18T12:01:18.448553+0000","last_became_active":"2023-10-18T12:01:09.133406+0000","last_became_peered":"2023-10-18T12:01:09.133406+0000","last_unstale":"2023-10-18T12:01:18.448553+0000","last_undegraded":"2023-10-18T12:01:18.448553+0000","last_fullsized":"2023-10-18T12:01:18.448553+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454282+0000","last_change":"2023-10-18T12:01:19.454282+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454282+0000","last_undegraded":"2023-10-18T12:01:19.454282+0000","last_fullsized":"2023-10-18T12:01:19.454282+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.461380+0000","last_change":"2023-10-18T12:01:19.458515+0000","last_active":"2023-10-18T12:01:19.461380+0000","last_peered":"2023-10-18T12:01:19.461380+0000","last_clean":"2023-10-18T12:01:19.461380+0000","last_became_active":"2023-10-18T12:01:19.457749+0000","last_became_peered":"2023-10-18T12:01:19.457749+0000","last_unstale":"2023-10-18T12:01:19.461380+0000","last_undegraded":"2023-10-18T12:01:19.461380+0000","last_fullsized":"2023-10-18T12:01:19.461380+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:24.677682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439756+0000","last_change":"2023-10-18T12:01:09.130142+0000","last_active":"2023-10-18T12:01:18.439756+0000","last_peered":"2023-10-18T12:01:18.439756+0000","last_clean":"2023-10-18T12:01:18.439756+0000","last_became_active":"2023-10-18T12:01:09.129884+0000","last_became_peered":"2023-10-18T12:01:09.129884+0000","last_unstale":"2023-10-18T12:01:18.439756+0000","last_undegraded":"2023-10-18T12:01:18.439756+0000","last_fullsized":"2023-10-18T12:01:18.439756+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:36.037273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450182+0000","last_change":"2023-10-18T12:01:19.450182+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450182+0000","last_undegraded":"2023-10-18T12:01:19.450182+0000","last_fullsized":"2023-10-18T12:01:19.450182+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.459660+0000","last_change":"2023-10-18T12:01:19.458409+0000","last_active":"2023-10-18T12:01:19.459660+0000","last_peered":"2023-10-18T12:01:19.459660+0000","last_clean":"2023-10-18T12:01:19.459660+0000","last_became_active":"2023-10-18T12:01:19.458151+0000","last_became_peered":"2023-10-18T12:01:19.458151+0000","last_unstale":"2023-10-18T12:01:19.459660+0000","last_undegraded":"2023-10-18T12:01:19.459660+0000","last_fullsized":"2023-10-18T12:01:19.459660+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:30.732460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443598+0000","last_change":"2023-10-18T12:01:09.136066+0000","last_active":"2023-10-18T12:01:18.443598+0000","last_peered":"2023-10-18T12:01:18.443598+0000","last_clean":"2023-10-18T12:01:18.443598+0000","last_became_active":"2023-10-18T12:01:09.135802+0000","last_became_peered":"2023-10-18T12:01:09.135802+0000","last_unstale":"2023-10-18T12:01:18.443598+0000","last_undegraded":"2023-10-18T12:01:18.443598+0000","last_fullsized":"2023-10-18T12:01:18.443598+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:26.550223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455962+0000","last_change":"2023-10-18T12:01:19.455962+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455962+0000","last_undegraded":"2023-10-18T12:01:19.455962+0000","last_fullsized":"2023-10-18T12:01:19.455962+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439658+0000","last_change":"2023-10-18T12:01:09.120812+0000","last_active":"2023-10-18T12:01:18.439658+0000","last_peered":"2023-10-18T12:01:18.439658+0000","last_clean":"2023-10-18T12:01:18.439658+0000","last_became_active":"2023-10-18T12:01:09.120561+0000","last_became_peered":"2023-10-18T12:01:09.120561+0000","last_unstale":"2023-10-18T12:01:18.439658+0000","last_undegraded":"2023-10-18T12:01:18.439658+0000","last_fullsized":"2023-10-18T12:01:18.439658+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:02.277242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"241'4","reported_seq":18,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444451+0000","last_change":"2023-10-18T12:01:08.112445+0000","last_active":"2023-10-18T12:01:18.444451+0000","last_peered":"2023-10-18T12:01:18.444451+0000","last_clean":"2023-10-18T12:01:18.444451+0000","last_became_active":"2023-10-18T12:01:08.112017+0000","last_became_peered":"2023-10-18T12:01:08.112017+0000","last_unstale":"2023-10-18T12:01:18.444451+0000","last_undegraded":"2023-10-18T12:01:18.444451+0000","last_fullsized":"2023-10-18T12:01:18.444451+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:11.607481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452634+0000","last_change":"2023-10-18T12:01:19.452634+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452634+0000","last_undegraded":"2023-10-18T12:01:19.452634+0000","last_fullsized":"2023-10-18T12:01:19.452634+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446081+0000","last_change":"2023-10-18T12:01:09.134916+0000","last_active":"2023-10-18T12:01:18.446081+0000","last_peered":"2023-10-18T12:01:18.446081+0000","last_clean":"2023-10-18T12:01:18.446081+0000","last_became_active":"2023-10-18T12:01:09.134630+0000","last_became_peered":"2023-10-18T12:01:09.134630+0000","last_unstale":"2023-10-18T12:01:18.446081+0000","last_undegraded":"2023-10-18T12:01:18.446081+0000","last_fullsized":"2023-10-18T12:01:18.446081+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447293+0000","last_change":"2023-10-18T12:01:08.119870+0000","last_active":"2023-10-18T12:01:18.447293+0000","last_peered":"2023-10-18T12:01:18.447293+0000","last_clean":"2023-10-18T12:01:18.447293+0000","last_became_active":"2023-10-18T12:01:08.119586+0000","last_became_peered":"2023-10-18T12:01:08.119586+0000","last_unstale":"2023-10-18T12:01:18.447293+0000","last_undegraded":"2023-10-18T12:01:18.447293+0000","last_fullsized":"2023-10-18T12:01:18.447293+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:16.288493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453087+0000","last_change":"2023-10-18T12:01:19.453087+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453087+0000","last_undegraded":"2023-10-18T12:01:19.453087+0000","last_fullsized":"2023-10-18T12:01:19.453087+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443755+0000","last_change":"2023-10-18T12:01:09.130894+0000","last_active":"2023-10-18T12:01:18.443755+0000","last_peered":"2023-10-18T12:01:18.443755+0000","last_clean":"2023-10-18T12:01:18.443755+0000","last_became_active":"2023-10-18T12:01:09.130614+0000","last_became_peered":"2023-10-18T12:01:09.130614+0000","last_unstale":"2023-10-18T12:01:18.443755+0000","last_undegraded":"2023-10-18T12:01:18.443755+0000","last_fullsized":"2023-10-18T12:01:18.443755+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:03.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448884+0000","last_change":"2023-10-18T12:01:08.115174+0000","last_active":"2023-10-18T12:01:18.448884+0000","last_peered":"2023-10-18T12:01:18.448884+0000","last_clean":"2023-10-18T12:01:18.448884+0000","last_became_active":"2023-10-18T12:01:08.114546+0000","last_became_peered":"2023-10-18T12:01:08.114546+0000","last_unstale":"2023-10-18T12:01:18.448884+0000","last_undegraded":"2023-10-18T12:01:18.448884+0000","last_fullsized":"2023-10-18T12:01:18.448884+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453540+0000","last_change":"2023-10-18T12:01:19.453540+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453540+0000","last_undegraded":"2023-10-18T12:01:19.453540+0000","last_fullsized":"2023-10-18T12:01:19.453540+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448819+0000","last_change":"2023-10-18T12:01:09.122601+0000","last_active":"2023-10-18T12:01:18.448819+0000","last_peered":"2023-10-18T12:01:18.448819+0000","last_clean":"2023-10-18T12:01:18.448819+0000","last_became_active":"2023-10-18T12:01:09.122334+0000","last_became_peered":"2023-10-18T12:01:09.122334+0000","last_unstale":"2023-10-18T12:01:18.448819+0000","last_undegraded":"2023-10-18T12:01:18.448819+0000","last_fullsized":"2023-10-18T12:01:18.448819+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446549+0000","last_change":"2023-10-18T12:01:08.116478+0000","last_active":"2023-10-18T12:01:18.446549+0000","last_peered":"2023-10-18T12:01:18.446549+0000","last_clean":"2023-10-18T12:01:18.446549+0000","last_became_active":"2023-10-18T12:01:08.116181+0000","last_became_peered":"2023-10-18T12:01:08.116181+0000","last_unstale":"2023-10-18T12:01:18.446549+0000","last_undegraded":"2023-10-18T12:01:18.446549+0000","last_fullsized":"2023-10-18T12:01:18.446549+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:10.087064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451810+0000","last_change":"2023-10-18T12:01:19.451810+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451810+0000","last_undegraded":"2023-10-18T12:01:19.451810+0000","last_fullsized":"2023-10-18T12:01:19.451810+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447548+0000","last_change":"2023-10-18T12:01:09.130166+0000","last_active":"2023-10-18T12:01:18.447548+0000","last_peered":"2023-10-18T12:01:18.447548+0000","last_clean":"2023-10-18T12:01:18.447548+0000","last_became_active":"2023-10-18T12:01:09.129725+0000","last_became_peered":"2023-10-18T12:01:09.129725+0000","last_unstale":"2023-10-18T12:01:18.447548+0000","last_undegraded":"2023-10-18T12:01:18.447548+0000","last_fullsized":"2023-10-18T12:01:18.447548+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:40.892842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448642+0000","last_change":"2023-10-18T12:01:08.107385+0000","last_active":"2023-10-18T12:01:18.448642+0000","last_peered":"2023-10-18T12:01:18.448642+0000","last_clean":"2023-10-18T12:01:18.448642+0000","last_became_active":"2023-10-18T12:01:08.107129+0000","last_became_peered":"2023-10-18T12:01:08.107129+0000","last_unstale":"2023-10-18T12:01:18.448642+0000","last_undegraded":"2023-10-18T12:01:18.448642+0000","last_fullsized":"2023-10-18T12:01:18.448642+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:43.838857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455957+0000","last_change":"2023-10-18T12:01:19.455957+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455957+0000","last_undegraded":"2023-10-18T12:01:19.455957+0000","last_fullsized":"2023-10-18T12:01:19.455957+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445933+0000","last_change":"2023-10-18T12:01:09.133737+0000","last_active":"2023-10-18T12:01:18.445933+0000","last_peered":"2023-10-18T12:01:18.445933+0000","last_clean":"2023-10-18T12:01:18.445933+0000","last_became_active":"2023-10-18T12:01:09.133487+0000","last_became_peered":"2023-10-18T12:01:09.133487+0000","last_unstale":"2023-10-18T12:01:18.445933+0000","last_undegraded":"2023-10-18T12:01:18.445933+0000","last_fullsized":"2023-10-18T12:01:18.445933+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:53:36.261500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445701+0000","last_change":"2023-10-18T12:01:08.119435+0000","last_active":"2023-10-18T12:01:18.445701+0000","last_peered":"2023-10-18T12:01:18.445701+0000","last_clean":"2023-10-18T12:01:18.445701+0000","last_became_active":"2023-10-18T12:01:08.118740+0000","last_became_peered":"2023-10-18T12:01:08.118740+0000","last_unstale":"2023-10-18T12:01:18.445701+0000","last_undegraded":"2023-10-18T12:01:18.445701+0000","last_fullsized":"2023-10-18T12:01:18.445701+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:54.172871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454152+0000","last_change":"2023-10-18T12:01:19.454152+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454152+0000","last_undegraded":"2023-10-18T12:01:19.454152+0000","last_fullsized":"2023-10-18T12:01:19.454152+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446839+0000","last_change":"2023-10-18T12:01:09.132487+0000","last_active":"2023-10-18T12:01:18.446839+0000","last_peered":"2023-10-18T12:01:18.446839+0000","last_clean":"2023-10-18T12:01:18.446839+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:18.446839+0000","last_undegraded":"2023-10-18T12:01:18.446839+0000","last_fullsized":"2023-10-18T12:01:18.446839+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:14.584151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443645+0000","last_change":"2023-10-18T12:01:08.115982+0000","last_active":"2023-10-18T12:01:18.443645+0000","last_peered":"2023-10-18T12:01:18.443645+0000","last_clean":"2023-10-18T12:01:18.443645+0000","last_became_active":"2023-10-18T12:01:08.115682+0000","last_became_peered":"2023-10-18T12:01:08.115682+0000","last_unstale":"2023-10-18T12:01:18.443645+0000","last_undegraded":"2023-10-18T12:01:18.443645+0000","last_fullsized":"2023-10-18T12:01:18.443645+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:36.789486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452968+0000","last_change":"2023-10-18T12:01:19.452968+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452968+0000","last_undegraded":"2023-10-18T12:01:19.452968+0000","last_fullsized":"2023-10-18T12:01:19.452968+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445246+0000","last_change":"2023-10-18T12:01:09.130010+0000","last_active":"2023-10-18T12:01:18.445246+0000","last_peered":"2023-10-18T12:01:18.445246+0000","last_clean":"2023-10-18T12:01:18.445246+0000","last_became_active":"2023-10-18T12:01:09.129744+0000","last_became_peered":"2023-10-18T12:01:09.129744+0000","last_unstale":"2023-10-18T12:01:18.445246+0000","last_undegraded":"2023-10-18T12:01:18.445246+0000","last_fullsized":"2023-10-18T12:01:18.445246+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:48:02.468955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448887+0000","last_change":"2023-10-18T12:01:08.110833+0000","last_active":"2023-10-18T12:01:18.448887+0000","last_peered":"2023-10-18T12:01:18.448887+0000","last_clean":"2023-10-18T12:01:18.448887+0000","last_became_active":"2023-10-18T12:01:08.110005+0000","last_became_peered":"2023-10-18T12:01:08.110005+0000","last_unstale":"2023-10-18T12:01:18.448887+0000","last_undegraded":"2023-10-18T12:01:18.448887+0000","last_fullsized":"2023-10-18T12:01:18.448887+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:32.311648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456437+0000","last_change":"2023-10-18T12:01:19.456437+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456437+0000","last_undegraded":"2023-10-18T12:01:19.456437+0000","last_fullsized":"2023-10-18T12:01:19.456437+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444394+0000","last_change":"2023-10-18T12:01:09.129659+0000","last_active":"2023-10-18T12:01:18.444394+0000","last_peered":"2023-10-18T12:01:18.444394+0000","last_clean":"2023-10-18T12:01:18.444394+0000","last_became_active":"2023-10-18T12:01:09.129388+0000","last_became_peered":"2023-10-18T12:01:09.129388+0000","last_unstale":"2023-10-18T12:01:18.444394+0000","last_undegraded":"2023-10-18T12:01:18.444394+0000","last_fullsized":"2023-10-18T12:01:18.444394+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:17.308708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446188+0000","last_change":"2023-10-18T12:01:08.118321+0000","last_active":"2023-10-18T12:01:18.446188+0000","last_peered":"2023-10-18T12:01:18.446188+0000","last_clean":"2023-10-18T12:01:18.446188+0000","last_became_active":"2023-10-18T12:01:08.117995+0000","last_became_peered":"2023-10-18T12:01:08.117995+0000","last_unstale":"2023-10-18T12:01:18.446188+0000","last_undegraded":"2023-10-18T12:01:18.446188+0000","last_fullsized":"2023-10-18T12:01:18.446188+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:46.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453041+0000","last_change":"2023-10-18T12:01:19.453041+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453041+0000","last_undegraded":"2023-10-18T12:01:19.453041+0000","last_fullsized":"2023-10-18T12:01:19.453041+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445227+0000","last_change":"2023-10-18T12:01:09.134396+0000","last_active":"2023-10-18T12:01:18.445227+0000","last_peered":"2023-10-18T12:01:18.445227+0000","last_clean":"2023-10-18T12:01:18.445227+0000","last_became_active":"2023-10-18T12:01:09.134144+0000","last_became_peered":"2023-10-18T12:01:09.134144+0000","last_unstale":"2023-10-18T12:01:18.445227+0000","last_undegraded":"2023-10-18T12:01:18.445227+0000","last_fullsized":"2023-10-18T12:01:18.445227+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:23:53.396657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"241'3","reported_seq":17,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447167+0000","last_change":"2023-10-18T12:01:08.118871+0000","last_active":"2023-10-18T12:01:18.447167+0000","last_peered":"2023-10-18T12:01:18.447167+0000","last_clean":"2023-10-18T12:01:18.447167+0000","last_became_active":"2023-10-18T12:01:08.118572+0000","last_became_peered":"2023-10-18T12:01:08.118572+0000","last_unstale":"2023-10-18T12:01:18.447167+0000","last_undegraded":"2023-10-18T12:01:18.447167+0000","last_fullsized":"2023-10-18T12:01:18.447167+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:45.510206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453957+0000","last_change":"2023-10-18T12:01:19.453957+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453957+0000","last_undegraded":"2023-10-18T12:01:19.453957+0000","last_fullsized":"2023-10-18T12:01:19.453957+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444883+0000","last_change":"2023-10-18T12:01:09.120444+0000","last_active":"2023-10-18T12:01:18.444883+0000","last_peered":"2023-10-18T12:01:18.444883+0000","last_clean":"2023-10-18T12:01:18.444883+0000","last_became_active":"2023-10-18T12:01:09.120018+0000","last_became_peered":"2023-10-18T12:01:09.120018+0000","last_unstale":"2023-10-18T12:01:18.444883+0000","last_undegraded":"2023-10-18T12:01:18.444883+0000","last_fullsized":"2023-10-18T12:01:18.444883+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:29.145722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.370024+0000","last_change":"2023-10-18T12:01:08.117890+0000","last_active":"2023-10-18T12:01:19.370024+0000","last_peered":"2023-10-18T12:01:19.370024+0000","last_clean":"2023-10-18T12:01:19.370024+0000","last_became_active":"2023-10-18T12:01:08.117606+0000","last_became_peered":"2023-10-18T12:01:08.117606+0000","last_unstale":"2023-10-18T12:01:19.370024+0000","last_undegraded":"2023-10-18T12:01:19.370024+0000","last_fullsized":"2023-10-18T12:01:19.370024+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:49.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454638+0000","last_change":"2023-10-18T12:01:19.454638+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454638+0000","last_undegraded":"2023-10-18T12:01:19.454638+0000","last_fullsized":"2023-10-18T12:01:19.454638+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439867+0000","last_change":"2023-10-18T12:01:09.127054+0000","last_active":"2023-10-18T12:01:18.439867+0000","last_peered":"2023-10-18T12:01:18.439867+0000","last_clean":"2023-10-18T12:01:18.439867+0000","last_became_active":"2023-10-18T12:01:09.126804+0000","last_became_peered":"2023-10-18T12:01:09.126804+0000","last_unstale":"2023-10-18T12:01:18.439867+0000","last_undegraded":"2023-10-18T12:01:18.439867+0000","last_fullsized":"2023-10-18T12:01:18.439867+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:20.948970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446639+0000","last_change":"2023-10-18T12:01:08.118544+0000","last_active":"2023-10-18T12:01:18.446639+0000","last_peered":"2023-10-18T12:01:18.446639+0000","last_clean":"2023-10-18T12:01:18.446639+0000","last_became_active":"2023-10-18T12:01:08.118228+0000","last_became_peered":"2023-10-18T12:01:08.118228+0000","last_unstale":"2023-10-18T12:01:18.446639+0000","last_undegraded":"2023-10-18T12:01:18.446639+0000","last_fullsized":"2023-10-18T12:01:18.446639+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456191+0000","last_change":"2023-10-18T12:01:19.456191+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456191+0000","last_undegraded":"2023-10-18T12:01:19.456191+0000","last_fullsized":"2023-10-18T12:01:19.456191+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447511+0000","last_change":"2023-10-18T12:01:09.120750+0000","last_active":"2023-10-18T12:01:18.447511+0000","last_peered":"2023-10-18T12:01:18.447511+0000","last_clean":"2023-10-18T12:01:18.447511+0000","last_became_active":"2023-10-18T12:01:09.120490+0000","last_became_peered":"2023-10-18T12:01:09.120490+0000","last_unstale":"2023-10-18T12:01:18.447511+0000","last_undegraded":"2023-10-18T12:01:18.447511+0000","last_fullsized":"2023-10-18T12:01:18.447511+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:51.005840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446987+0000","last_change":"2023-10-18T12:01:08.118022+0000","last_active":"2023-10-18T12:01:18.446987+0000","last_peered":"2023-10-18T12:01:18.446987+0000","last_clean":"2023-10-18T12:01:18.446987+0000","last_became_active":"2023-10-18T12:01:08.117754+0000","last_became_peered":"2023-10-18T12:01:08.117754+0000","last_unstale":"2023-10-18T12:01:18.446987+0000","last_undegraded":"2023-10-18T12:01:18.446987+0000","last_fullsized":"2023-10-18T12:01:18.446987+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:37.834293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455355+0000","last_change":"2023-10-18T12:01:19.455355+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455355+0000","last_undegraded":"2023-10-18T12:01:19.455355+0000","last_fullsized":"2023-10-18T12:01:19.455355+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447313+0000","last_change":"2023-10-18T12:01:09.135383+0000","last_active":"2023-10-18T12:01:18.447313+0000","last_peered":"2023-10-18T12:01:18.447313+0000","last_clean":"2023-10-18T12:01:18.447313+0000","last_became_active":"2023-10-18T12:01:09.135114+0000","last_became_peered":"2023-10-18T12:01:09.135114+0000","last_unstale":"2023-10-18T12:01:18.447313+0000","last_undegraded":"2023-10-18T12:01:18.447313+0000","last_fullsized":"2023-10-18T12:01:18.447313+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446147+0000","last_change":"2023-10-18T12:01:08.115155+0000","last_active":"2023-10-18T12:01:18.446147+0000","last_peered":"2023-10-18T12:01:18.446147+0000","last_clean":"2023-10-18T12:01:18.446147+0000","last_became_active":"2023-10-18T12:01:08.114843+0000","last_became_peered":"2023-10-18T12:01:08.114843+0000","last_unstale":"2023-10-18T12:01:18.446147+0000","last_undegraded":"2023-10-18T12:01:18.446147+0000","last_fullsized":"2023-10-18T12:01:18.446147+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:28.358756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455037+0000","last_change":"2023-10-18T12:01:19.455037+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455037+0000","last_undegraded":"2023-10-18T12:01:19.455037+0000","last_fullsized":"2023-10-18T12:01:19.455037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446744+0000","last_change":"2023-10-18T12:01:09.131820+0000","last_active":"2023-10-18T12:01:18.446744+0000","last_peered":"2023-10-18T12:01:18.446744+0000","last_clean":"2023-10-18T12:01:18.446744+0000","last_became_active":"2023-10-18T12:01:09.131568+0000","last_became_peered":"2023-10-18T12:01:09.131568+0000","last_unstale":"2023-10-18T12:01:18.446744+0000","last_undegraded":"2023-10-18T12:01:18.446744+0000","last_fullsized":"2023-10-18T12:01:18.446744+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.395799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"241'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439719+0000","last_change":"2023-10-18T12:01:08.112711+0000","last_active":"2023-10-18T12:01:18.439719+0000","last_peered":"2023-10-18T12:01:18.439719+0000","last_clean":"2023-10-18T12:01:18.439719+0000","last_became_active":"2023-10-18T12:01:08.112436+0000","last_became_peered":"2023-10-18T12:01:08.112436+0000","last_unstale":"2023-10-18T12:01:18.439719+0000","last_undegraded":"2023-10-18T12:01:18.439719+0000","last_fullsized":"2023-10-18T12:01:18.439719+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:15.568916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455648+0000","last_change":"2023-10-18T12:01:19.455648+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455648+0000","last_undegraded":"2023-10-18T12:01:19.455648+0000","last_fullsized":"2023-10-18T12:01:19.455648+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444666+0000","last_change":"2023-10-18T12:01:09.135106+0000","last_active":"2023-10-18T12:01:18.444666+0000","last_peered":"2023-10-18T12:01:18.444666+0000","last_clean":"2023-10-18T12:01:18.444666+0000","last_became_active":"2023-10-18T12:01:09.134828+0000","last_became_peered":"2023-10-18T12:01:09.134828+0000","last_unstale":"2023-10-18T12:01:18.444666+0000","last_undegraded":"2023-10-18T12:01:18.444666+0000","last_fullsized":"2023-10-18T12:01:18.444666+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:39.126921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444574+0000","last_change":"2023-10-18T12:01:08.119248+0000","last_active":"2023-10-18T12:01:18.444574+0000","last_peered":"2023-10-18T12:01:18.444574+0000","last_clean":"2023-10-18T12:01:18.444574+0000","last_became_active":"2023-10-18T12:01:08.118973+0000","last_became_peered":"2023-10-18T12:01:08.118973+0000","last_unstale":"2023-10-18T12:01:18.444574+0000","last_undegraded":"2023-10-18T12:01:18.444574+0000","last_fullsized":"2023-10-18T12:01:18.444574+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:37.079812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459833+0000","last_change":"2023-10-18T12:01:19.459833+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459833+0000","last_undegraded":"2023-10-18T12:01:19.459833+0000","last_fullsized":"2023-10-18T12:01:19.459833+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445946+0000","last_change":"2023-10-18T12:01:09.132827+0000","last_active":"2023-10-18T12:01:18.445946+0000","last_peered":"2023-10-18T12:01:18.445946+0000","last_clean":"2023-10-18T12:01:18.445946+0000","last_became_active":"2023-10-18T12:01:09.132545+0000","last_became_peered":"2023-10-18T12:01:09.132545+0000","last_unstale":"2023-10-18T12:01:18.445946+0000","last_undegraded":"2023-10-18T12:01:18.445946+0000","last_fullsized":"2023-10-18T12:01:18.445946+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:02.034692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439997+0000","last_change":"2023-10-18T12:01:08.107964+0000","last_active":"2023-10-18T12:01:18.439997+0000","last_peered":"2023-10-18T12:01:18.439997+0000","last_clean":"2023-10-18T12:01:18.439997+0000","last_became_active":"2023-10-18T12:01:08.107696+0000","last_became_peered":"2023-10-18T12:01:08.107696+0000","last_unstale":"2023-10-18T12:01:18.439997+0000","last_undegraded":"2023-10-18T12:01:18.439997+0000","last_fullsized":"2023-10-18T12:01:18.439997+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:18.578919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453669+0000","last_change":"2023-10-18T12:01:19.453669+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453669+0000","last_undegraded":"2023-10-18T12:01:19.453669+0000","last_fullsized":"2023-10-18T12:01:19.453669+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439817+0000","last_change":"2023-10-18T12:01:09.136078+0000","last_active":"2023-10-18T12:01:18.439817+0000","last_peered":"2023-10-18T12:01:18.439817+0000","last_clean":"2023-10-18T12:01:18.439817+0000","last_became_active":"2023-10-18T12:01:09.134722+0000","last_became_peered":"2023-10-18T12:01:09.134722+0000","last_unstale":"2023-10-18T12:01:18.439817+0000","last_undegraded":"2023-10-18T12:01:18.439817+0000","last_fullsized":"2023-10-18T12:01:18.439817+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:09.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447390+0000","last_change":"2023-10-18T12:01:08.120416+0000","last_active":"2023-10-18T12:01:18.447390+0000","last_peered":"2023-10-18T12:01:18.447390+0000","last_clean":"2023-10-18T12:01:18.447390+0000","last_became_active":"2023-10-18T12:01:08.120122+0000","last_became_peered":"2023-10-18T12:01:08.120122+0000","last_unstale":"2023-10-18T12:01:18.447390+0000","last_undegraded":"2023-10-18T12:01:18.447390+0000","last_fullsized":"2023-10-18T12:01:18.447390+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:26.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455164+0000","last_change":"2023-10-18T12:01:19.455164+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455164+0000","last_undegraded":"2023-10-18T12:01:19.455164+0000","last_fullsized":"2023-10-18T12:01:19.455164+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446070+0000","last_change":"2023-10-18T12:01:09.135042+0000","last_active":"2023-10-18T12:01:18.446070+0000","last_peered":"2023-10-18T12:01:18.446070+0000","last_clean":"2023-10-18T12:01:18.446070+0000","last_became_active":"2023-10-18T12:01:09.134765+0000","last_became_peered":"2023-10-18T12:01:09.134765+0000","last_unstale":"2023-10-18T12:01:18.446070+0000","last_undegraded":"2023-10-18T12:01:18.446070+0000","last_fullsized":"2023-10-18T12:01:18.446070+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:58.131101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446991+0000","last_change":"2023-10-18T12:01:08.117972+0000","last_active":"2023-10-18T12:01:18.446991+0000","last_peered":"2023-10-18T12:01:18.446991+0000","last_clean":"2023-10-18T12:01:18.446991+0000","last_became_active":"2023-10-18T12:01:08.117690+0000","last_became_peered":"2023-10-18T12:01:08.117690+0000","last_unstale":"2023-10-18T12:01:18.446991+0000","last_undegraded":"2023-10-18T12:01:18.446991+0000","last_fullsized":"2023-10-18T12:01:18.446991+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458648+0000","last_change":"2023-10-18T12:01:19.458648+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458648+0000","last_undegraded":"2023-10-18T12:01:19.458648+0000","last_fullsized":"2023-10-18T12:01:19.458648+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444663+0000","last_change":"2023-10-18T12:01:09.122673+0000","last_active":"2023-10-18T12:01:18.444663+0000","last_peered":"2023-10-18T12:01:18.444663+0000","last_clean":"2023-10-18T12:01:18.444663+0000","last_became_active":"2023-10-18T12:01:09.122430+0000","last_became_peered":"2023-10-18T12:01:09.122430+0000","last_unstale":"2023-10-18T12:01:18.444663+0000","last_undegraded":"2023-10-18T12:01:18.444663+0000","last_fullsized":"2023-10-18T12:01:18.444663+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:29.387267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446157+0000","last_change":"2023-10-18T12:01:08.119439+0000","last_active":"2023-10-18T12:01:18.446157+0000","last_peered":"2023-10-18T12:01:18.446157+0000","last_clean":"2023-10-18T12:01:18.446157+0000","last_became_active":"2023-10-18T12:01:08.119158+0000","last_became_peered":"2023-10-18T12:01:08.119158+0000","last_unstale":"2023-10-18T12:01:18.446157+0000","last_undegraded":"2023-10-18T12:01:18.446157+0000","last_fullsized":"2023-10-18T12:01:18.446157+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:56.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454552+0000","last_change":"2023-10-18T12:01:19.454552+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454552+0000","last_undegraded":"2023-10-18T12:01:19.454552+0000","last_fullsized":"2023-10-18T12:01:19.454552+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440049+0000","last_change":"2023-10-18T12:01:09.124352+0000","last_active":"2023-10-18T12:01:18.440049+0000","last_peered":"2023-10-18T12:01:18.440049+0000","last_clean":"2023-10-18T12:01:18.440049+0000","last_became_active":"2023-10-18T12:01:09.124072+0000","last_became_peered":"2023-10-18T12:01:09.124072+0000","last_unstale":"2023-10-18T12:01:18.440049+0000","last_undegraded":"2023-10-18T12:01:18.440049+0000","last_fullsized":"2023-10-18T12:01:18.440049+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:15.239290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443813+0000","last_change":"2023-10-18T12:01:08.117335+0000","last_active":"2023-10-18T12:01:18.443813+0000","last_peered":"2023-10-18T12:01:18.443813+0000","last_clean":"2023-10-18T12:01:18.443813+0000","last_became_active":"2023-10-18T12:01:08.117032+0000","last_became_peered":"2023-10-18T12:01:08.117032+0000","last_unstale":"2023-10-18T12:01:18.443813+0000","last_undegraded":"2023-10-18T12:01:18.443813+0000","last_fullsized":"2023-10-18T12:01:18.443813+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:33.304569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456597+0000","last_change":"2023-10-18T12:01:19.456597+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456597+0000","last_undegraded":"2023-10-18T12:01:19.456597+0000","last_fullsized":"2023-10-18T12:01:19.456597+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446661+0000","last_change":"2023-10-18T12:01:09.135148+0000","last_active":"2023-10-18T12:01:18.446661+0000","last_peered":"2023-10-18T12:01:18.446661+0000","last_clean":"2023-10-18T12:01:18.446661+0000","last_became_active":"2023-10-18T12:01:09.134895+0000","last_became_peered":"2023-10-18T12:01:09.134895+0000","last_unstale":"2023-10-18T12:01:18.446661+0000","last_undegraded":"2023-10-18T12:01:18.446661+0000","last_fullsized":"2023-10-18T12:01:18.446661+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:19.263310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444467+0000","last_change":"2023-10-18T12:01:08.120618+0000","last_active":"2023-10-18T12:01:18.444467+0000","last_peered":"2023-10-18T12:01:18.444467+0000","last_clean":"2023-10-18T12:01:18.444467+0000","last_became_active":"2023-10-18T12:01:08.120315+0000","last_became_peered":"2023-10-18T12:01:08.120315+0000","last_unstale":"2023-10-18T12:01:18.444467+0000","last_undegraded":"2023-10-18T12:01:18.444467+0000","last_fullsized":"2023-10-18T12:01:18.444467+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:36.555722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457598+0000","last_change":"2023-10-18T12:01:19.457598+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457598+0000","last_undegraded":"2023-10-18T12:01:19.457598+0000","last_fullsized":"2023-10-18T12:01:19.457598+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447164+0000","last_change":"2023-10-18T12:01:09.130881+0000","last_active":"2023-10-18T12:01:18.447164+0000","last_peered":"2023-10-18T12:01:18.447164+0000","last_clean":"2023-10-18T12:01:18.447164+0000","last_became_active":"2023-10-18T12:01:09.130628+0000","last_became_peered":"2023-10-18T12:01:09.130628+0000","last_unstale":"2023-10-18T12:01:18.447164+0000","last_undegraded":"2023-10-18T12:01:18.447164+0000","last_fullsized":"2023-10-18T12:01:18.447164+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:13.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448426+0000","last_change":"2023-10-18T12:01:08.120313+0000","last_active":"2023-10-18T12:01:18.448426+0000","last_peered":"2023-10-18T12:01:18.448426+0000","last_clean":"2023-10-18T12:01:18.448426+0000","last_became_active":"2023-10-18T12:01:08.120059+0000","last_became_peered":"2023-10-18T12:01:08.120059+0000","last_unstale":"2023-10-18T12:01:18.448426+0000","last_undegraded":"2023-10-18T12:01:18.448426+0000","last_fullsized":"2023-10-18T12:01:18.448426+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456394+0000","last_change":"2023-10-18T12:01:19.456394+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456394+0000","last_undegraded":"2023-10-18T12:01:19.456394+0000","last_fullsized":"2023-10-18T12:01:19.456394+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445079+0000","last_change":"2023-10-18T12:01:09.125685+0000","last_active":"2023-10-18T12:01:18.445079+0000","last_peered":"2023-10-18T12:01:18.445079+0000","last_clean":"2023-10-18T12:01:18.445079+0000","last_became_active":"2023-10-18T12:01:09.125395+0000","last_became_peered":"2023-10-18T12:01:09.125395+0000","last_unstale":"2023-10-18T12:01:18.445079+0000","last_undegraded":"2023-10-18T12:01:18.445079+0000","last_fullsized":"2023-10-18T12:01:18.445079+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:47.228016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444968+0000","last_change":"2023-10-18T12:01:08.117717+0000","last_active":"2023-10-18T12:01:18.444968+0000","last_peered":"2023-10-18T12:01:18.444968+0000","last_clean":"2023-10-18T12:01:18.444968+0000","last_became_active":"2023-10-18T12:01:08.117405+0000","last_became_peered":"2023-10-18T12:01:08.117405+0000","last_unstale":"2023-10-18T12:01:18.444968+0000","last_undegraded":"2023-10-18T12:01:18.444968+0000","last_fullsized":"2023-10-18T12:01:18.444968+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:00.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454367+0000","last_change":"2023-10-18T12:01:19.454367+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454367+0000","last_undegraded":"2023-10-18T12:01:19.454367+0000","last_fullsized":"2023-10-18T12:01:19.454367+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445826+0000","last_change":"2023-10-18T12:01:09.132599+0000","last_active":"2023-10-18T12:01:18.445826+0000","last_peered":"2023-10-18T12:01:18.445826+0000","last_clean":"2023-10-18T12:01:18.445826+0000","last_became_active":"2023-10-18T12:01:09.131710+0000","last_became_peered":"2023-10-18T12:01:09.131710+0000","last_unstale":"2023-10-18T12:01:18.445826+0000","last_undegraded":"2023-10-18T12:01:18.445826+0000","last_fullsized":"2023-10-18T12:01:18.445826+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:18.701174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446916+0000","last_change":"2023-10-18T12:01:08.118553+0000","last_active":"2023-10-18T12:01:18.446916+0000","last_peered":"2023-10-18T12:01:18.446916+0000","last_clean":"2023-10-18T12:01:18.446916+0000","last_became_active":"2023-10-18T12:01:08.118286+0000","last_became_peered":"2023-10-18T12:01:08.118286+0000","last_unstale":"2023-10-18T12:01:18.446916+0000","last_undegraded":"2023-10-18T12:01:18.446916+0000","last_fullsized":"2023-10-18T12:01:18.446916+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:05:32.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.460259+0000","last_change":"2023-10-18T12:01:19.460259+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.460259+0000","last_undegraded":"2023-10-18T12:01:19.460259+0000","last_fullsized":"2023-10-18T12:01:19.460259+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445148+0000","last_change":"2023-10-18T12:01:09.133522+0000","last_active":"2023-10-18T12:01:18.445148+0000","last_peered":"2023-10-18T12:01:18.445148+0000","last_clean":"2023-10-18T12:01:18.445148+0000","last_became_active":"2023-10-18T12:01:09.133267+0000","last_became_peered":"2023-10-18T12:01:09.133267+0000","last_unstale":"2023-10-18T12:01:18.445148+0000","last_undegraded":"2023-10-18T12:01:18.445148+0000","last_fullsized":"2023-10-18T12:01:18.445148+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:44:14.702965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448989+0000","last_change":"2023-10-18T12:01:08.116830+0000","last_active":"2023-10-18T12:01:18.448989+0000","last_peered":"2023-10-18T12:01:18.448989+0000","last_clean":"2023-10-18T12:01:18.448989+0000","last_became_active":"2023-10-18T12:01:08.116538+0000","last_became_peered":"2023-10-18T12:01:08.116538+0000","last_unstale":"2023-10-18T12:01:18.448989+0000","last_undegraded":"2023-10-18T12:01:18.448989+0000","last_fullsized":"2023-10-18T12:01:18.448989+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:09.165065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451313+0000","last_change":"2023-10-18T12:01:19.451313+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451313+0000","last_undegraded":"2023-10-18T12:01:19.451313+0000","last_fullsized":"2023-10-18T12:01:19.451313+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446209+0000","last_change":"2023-10-18T12:01:09.128305+0000","last_active":"2023-10-18T12:01:18.446209+0000","last_peered":"2023-10-18T12:01:18.446209+0000","last_clean":"2023-10-18T12:01:18.446209+0000","last_became_active":"2023-10-18T12:01:09.127933+0000","last_became_peered":"2023-10-18T12:01:09.127933+0000","last_unstale":"2023-10-18T12:01:18.446209+0000","last_undegraded":"2023-10-18T12:01:18.446209+0000","last_fullsized":"2023-10-18T12:01:18.446209+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:28.312681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448893+0000","last_change":"2023-10-18T12:01:08.115363+0000","last_active":"2023-10-18T12:01:18.448893+0000","last_peered":"2023-10-18T12:01:18.448893+0000","last_clean":"2023-10-18T12:01:18.448893+0000","last_became_active":"2023-10-18T12:01:08.114903+0000","last_became_peered":"2023-10-18T12:01:08.114903+0000","last_unstale":"2023-10-18T12:01:18.448893+0000","last_undegraded":"2023-10-18T12:01:18.448893+0000","last_fullsized":"2023-10-18T12:01:18.448893+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:46.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456219+0000","last_change":"2023-10-18T12:01:19.456219+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456219+0000","last_undegraded":"2023-10-18T12:01:19.456219+0000","last_fullsized":"2023-10-18T12:01:19.456219+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448801+0000","last_change":"2023-10-18T12:01:09.127159+0000","last_active":"2023-10-18T12:01:18.448801+0000","last_peered":"2023-10-18T12:01:18.448801+0000","last_clean":"2023-10-18T12:01:18.448801+0000","last_became_active":"2023-10-18T12:01:09.126874+0000","last_became_peered":"2023-10-18T12:01:09.126874+0000","last_unstale":"2023-10-18T12:01:18.448801+0000","last_undegraded":"2023-10-18T12:01:18.448801+0000","last_fullsized":"2023-10-18T12:01:18.448801+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444343+0000","last_change":"2023-10-18T12:01:08.118539+0000","last_active":"2023-10-18T12:01:18.444343+0000","last_peered":"2023-10-18T12:01:18.444343+0000","last_clean":"2023-10-18T12:01:18.444343+0000","last_became_active":"2023-10-18T12:01:08.116966+0000","last_became_peered":"2023-10-18T12:01:08.116966+0000","last_unstale":"2023-10-18T12:01:18.444343+0000","last_undegraded":"2023-10-18T12:01:18.444343+0000","last_fullsized":"2023-10-18T12:01:18.444343+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:20.828671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457469+0000","last_change":"2023-10-18T12:01:19.457469+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457469+0000","last_undegraded":"2023-10-18T12:01:19.457469+0000","last_fullsized":"2023-10-18T12:01:19.457469+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444299+0000","last_change":"2023-10-18T12:01:09.119148+0000","last_active":"2023-10-18T12:01:18.444299+0000","last_peered":"2023-10-18T12:01:18.444299+0000","last_clean":"2023-10-18T12:01:18.444299+0000","last_became_active":"2023-10-18T12:01:09.118894+0000","last_became_peered":"2023-10-18T12:01:09.118894+0000","last_unstale":"2023-10-18T12:01:18.444299+0000","last_undegraded":"2023-10-18T12:01:18.444299+0000","last_fullsized":"2023-10-18T12:01:18.444299+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:42.857305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447050+0000","last_change":"2023-10-18T12:01:08.117526+0000","last_active":"2023-10-18T12:01:18.447050+0000","last_peered":"2023-10-18T12:01:18.447050+0000","last_clean":"2023-10-18T12:01:18.447050+0000","last_became_active":"2023-10-18T12:01:08.117234+0000","last_became_peered":"2023-10-18T12:01:08.117234+0000","last_unstale":"2023-10-18T12:01:18.447050+0000","last_undegraded":"2023-10-18T12:01:18.447050+0000","last_fullsized":"2023-10-18T12:01:18.447050+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:43.295217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.460764+0000","last_change":"2023-10-18T12:01:19.460764+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.460764+0000","last_undegraded":"2023-10-18T12:01:19.460764+0000","last_fullsized":"2023-10-18T12:01:19.460764+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439950+0000","last_change":"2023-10-18T12:01:09.125150+0000","last_active":"2023-10-18T12:01:18.439950+0000","last_peered":"2023-10-18T12:01:18.439950+0000","last_clean":"2023-10-18T12:01:18.439950+0000","last_became_active":"2023-10-18T12:01:09.124900+0000","last_became_peered":"2023-10-18T12:01:09.124900+0000","last_unstale":"2023-10-18T12:01:18.439950+0000","last_undegraded":"2023-10-18T12:01:18.439950+0000","last_fullsized":"2023-10-18T12:01:18.439950+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:10.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445789+0000","last_change":"2023-10-18T12:01:08.119612+0000","last_active":"2023-10-18T12:01:18.445789+0000","last_peered":"2023-10-18T12:01:18.445789+0000","last_clean":"2023-10-18T12:01:18.445789+0000","last_became_active":"2023-10-18T12:01:08.119116+0000","last_became_peered":"2023-10-18T12:01:08.119116+0000","last_unstale":"2023-10-18T12:01:18.445789+0000","last_undegraded":"2023-10-18T12:01:18.445789+0000","last_fullsized":"2023-10-18T12:01:18.445789+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:02.536144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451311+0000","last_change":"2023-10-18T12:01:19.451311+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451311+0000","last_undegraded":"2023-10-18T12:01:19.451311+0000","last_fullsized":"2023-10-18T12:01:19.451311+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446463+0000","last_change":"2023-10-18T12:01:09.130360+0000","last_active":"2023-10-18T12:01:18.446463+0000","last_peered":"2023-10-18T12:01:18.446463+0000","last_clean":"2023-10-18T12:01:18.446463+0000","last_became_active":"2023-10-18T12:01:09.130086+0000","last_became_peered":"2023-10-18T12:01:09.130086+0000","last_unstale":"2023-10-18T12:01:18.446463+0000","last_undegraded":"2023-10-18T12:01:18.446463+0000","last_fullsized":"2023-10-18T12:01:18.446463+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:28.085761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443434+0000","last_change":"2023-10-18T12:01:08.119803+0000","last_active":"2023-10-18T12:01:18.443434+0000","last_peered":"2023-10-18T12:01:18.443434+0000","last_clean":"2023-10-18T12:01:18.443434+0000","last_became_active":"2023-10-18T12:01:08.119467+0000","last_became_peered":"2023-10-18T12:01:08.119467+0000","last_unstale":"2023-10-18T12:01:18.443434+0000","last_undegraded":"2023-10-18T12:01:18.443434+0000","last_fullsized":"2023-10-18T12:01:18.443434+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:29.410010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459374+0000","last_change":"2023-10-18T12:01:19.459374+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459374+0000","last_undegraded":"2023-10-18T12:01:19.459374+0000","last_fullsized":"2023-10-18T12:01:19.459374+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443794+0000","last_change":"2023-10-18T12:01:09.122670+0000","last_active":"2023-10-18T12:01:18.443794+0000","last_peered":"2023-10-18T12:01:18.443794+0000","last_clean":"2023-10-18T12:01:18.443794+0000","last_became_active":"2023-10-18T12:01:09.122408+0000","last_became_peered":"2023-10-18T12:01:09.122408+0000","last_unstale":"2023-10-18T12:01:18.443794+0000","last_undegraded":"2023-10-18T12:01:18.443794+0000","last_fullsized":"2023-10-18T12:01:18.443794+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:39.479911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447844+0000","last_change":"2023-10-18T12:01:08.111055+0000","last_active":"2023-10-18T12:01:18.447844+0000","last_peered":"2023-10-18T12:01:18.447844+0000","last_clean":"2023-10-18T12:01:18.447844+0000","last_became_active":"2023-10-18T12:01:08.110728+0000","last_became_peered":"2023-10-18T12:01:08.110728+0000","last_unstale":"2023-10-18T12:01:18.447844+0000","last_undegraded":"2023-10-18T12:01:18.447844+0000","last_fullsized":"2023-10-18T12:01:18.447844+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:15.472803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"227'379","reported_seq":1331,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445541+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:01:18.445541+0000","last_peered":"2023-10-18T12:01:18.445541+0000","last_clean":"2023-10-18T12:01:18.445541+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:01:18.445541+0000","last_undegraded":"2023-10-18T12:01:18.445541+0000","last_fullsized":"2023-10-18T12:01:18.445541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":379,"ondisk_log_size":379,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52808,"kb_used_data":3580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319032,"statfs":{"total":96636764160,"available":96582688768,"internally_reserved":0,"allocated":3665920,"data_stored":2169281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.42399999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.67800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834576000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42500,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329340,"statfs":{"total":96636764160,"available":96593244160,"internally_reserved":0,"allocated":3072000,"data_stored":1578901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.495},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71299999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834576000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64700,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307140,"statfs":{"total":96636764160,"available":96570511360,"internally_reserved":0,"allocated":3063808,"data_stored":1578789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.498},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":1.101,"15min":1.1539999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.71099999999999997,"15min":0.751},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.61699999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72699999999999998,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81399999999999995,"15min":1},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53000000000000003,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.749,"5min":0.82999999999999996,"15min":1.01},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.255,"15min":0.255},"max":{"1min":0.78800000000000003,"5min":0.83699999999999997,"15min":1.101},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48064,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323776,"statfs":{"total":96636764160,"available":96587546624,"internally_reserved":0,"allocated":3067904,"data_stored":1578859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.22500000000000001},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.59599999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":1.123},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.56899999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.66000000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":1.018,"15min":1.018},"last":0.752}]},{"osd":7,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.88900000000000001,"15min":1.05},"last":0.623},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.77900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834576000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48100,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323740,"statfs":{"total":96636764160,"available":96587509760,"internally_reserved":0,"allocated":3100672,"data_stored":1589807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.23999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.378,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.25},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.39300000000000002,"15min":0.40699999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.38400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.39900000000000002,"15min":0.40600000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44900000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.432,"5min":1.899,"15min":1.899},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.42899999999999999,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.434,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.87,"5min":1.034,"15min":1.052},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.629}]},{"osd":5,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":1.522,"15min":1.522},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.91100000000000003,"15min":1.026},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82299999999999995,"5min":1.01,"15min":1.01},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.129,"15min":1.129},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0649999999999999},"last":0.69199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60548,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311292,"statfs":{"total":96636764160,"available":96574763008,"internally_reserved":0,"allocated":3072000,"data_stored":1579441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.52300000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.40500000000000003,"15min":0.41599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.31900000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.61499999999999999,"15min":0.624},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.374,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.63100000000000001,"15min":0.746},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55300000000000005,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86299999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66300000000000003,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.087},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.85199999999999998,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63400000000000001,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.95399999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":1.0609999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60552,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311288,"statfs":{"total":96636764160,"available":96574758912,"internally_reserved":0,"allocated":3076096,"data_stored":1586301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.314}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.32600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.375}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.41099999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575999,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43084,"kb_used_data":3584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328756,"statfs":{"total":96636764160,"available":96592646144,"internally_reserved":0,"allocated":3670016,"data_stored":2171210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52200000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.52300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.26100000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.51100000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.96999999999999997,"15min":0.997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72399999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.95499999999999996,"15min":0.96399999999999997},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64700000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":1.03},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70999999999999996,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.5}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.78000000000000003,"5min":0.997,"15min":0.997},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:01:23.747 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:01:23.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.876+0000 7f83d5aaf640 1 -- 172.21.15.125:0/2325228022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00c96b0 msgr2=0x7f83d00c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.876+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/2325228022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00c96b0 0x7f83d00c9b60 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f83c4005fd0 tx=0x7f83c402bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:23.879 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.876+0000 7f83d5aaf640 1 -- 172.21.15.125:0/2325228022 shutdown_connections 2023-10-18T12:01:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.876+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/2325228022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00c96b0 0x7f83d00c9b60 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.876+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/2325228022 >> v2:172.21.15.139:3300/0 conn(0x7f83d006af00 0x7f83d006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.876+0000 7f83d5aaf640 1 -- 172.21.15.125:0/2325228022 >> 172.21.15.125:0/2325228022 conn(0x7f83d0069930 msgr2=0x7f83d0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:23.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.876+0000 7f83d5aaf640 1 -- 172.21.15.125:0/2325228022 shutdown_connections 2023-10-18T12:01:23.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.876+0000 7f83d5aaf640 1 -- 172.21.15.125:0/2325228022 wait complete. 2023-10-18T12:01:23.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.877+0000 7f83d5aaf640 1 Processor -- start 2023-10-18T12:01:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.877+0000 7f83d5aaf640 1 -- start start 2023-10-18T12:01:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83d5aaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83d006af00 0x7f83d00a8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83d5aaf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00ab500 0x7f83d0151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83d5aaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00c96b0 0x7f83d01527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83d5aaf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00a7660 con 0x7f83d00ab500 2023-10-18T12:01:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83d5aaf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00a7940 con 0x7f83d00c96b0 2023-10-18T12:01:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83d5aaf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00a7c20 con 0x7f83d006af00 2023-10-18T12:01:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83ceffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00ab500 0x7f83d0151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83ceffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00ab500 0x7f83d0151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33200/0 (socket says 172.21.15.125:33200) 2023-10-18T12:01:23.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83ceffd640 1 -- 172.21.15.125:0/77311404 learned_addr learned my addr 172.21.15.125:0/77311404 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83cf7fe640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83d006af00 0x7f83d00a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.878+0000 7f83cffff640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00c96b0 0x7f83d01527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83ceffd640 1 -- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83d006af00 msgr2=0x7f83d00a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83ceffd640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83d006af00 0x7f83d00a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83ceffd640 1 -- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00c96b0 msgr2=0x7f83d01527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83ceffd640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00c96b0 0x7f83d01527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83ceffd640 1 -- 172.21.15.125:0/77311404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4009560 con 0x7f83d00ab500 2023-10-18T12:01:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83ceffd640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00ab500 0x7f83d0151ec0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009850 tx=0x7f83c402da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83b7fff640 1 -- 172.21.15.125:0/77311404 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c402de60 con 0x7f83d00ab500 2023-10-18T12:01:23.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83b7fff640 1 -- 172.21.15.125:0/77311404 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83c40036e0 con 0x7f83d00ab500 2023-10-18T12:01:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d00a7f00 con 0x7f83d00ab500 2023-10-18T12:01:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83b7fff640 1 -- 172.21.15.125:0/77311404 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4003a90 con 0x7f83d00ab500 2023-10-18T12:01:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.879+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d0152400 con 0x7f83d00ab500 2023-10-18T12:01:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.880+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d00a8420 con 0x7f83d00ab500 2023-10-18T12:01:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.880+0000 7f83b7fff640 1 -- 172.21.15.125:0/77311404 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f83c4003bf0 con 0x7f83d00ab500 2023-10-18T12:01:23.888 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.881+0000 7f83b7fff640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f83bc06b080 0x7f83bc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.881+0000 7f83b7fff640 1 -- 172.21.15.125:0/77311404 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f83c40aca00 con 0x7f83d00ab500 2023-10-18T12:01:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.881+0000 7f83cf7fe640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f83bc06b080 0x7f83bc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.881+0000 7f83cf7fe640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f83bc06b080 0x7f83bc06d540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83c0005d00 tx=0x7f83c000b410 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:23.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:23.885+0000 7f83b7fff640 1 -- 172.21.15.125:0/77311404 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83d00a8420 con 0x7f83d00ab500 2023-10-18T12:01:24.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.046+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83d0159c50 con 0x7f83bc06b080 2023-10-18T12:01:24.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.059+0000 7f83b7fff640 1 -- 172.21.15.125:0/77311404 <== mgr.9236 v2:172.21.15.139:6832/3422518215 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+523685 (secure 0 0 0) 0x7f83c0015070 con 0x7f83bc06b080 2023-10-18T12:01:24.067 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:24.070 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:01:24.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f83bc06b080 msgr2=0x7f83bc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f83bc06b080 0x7f83bc06d540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83c0005d00 tx=0x7f83c000b410 comp rx=0 tx=0).stop 2023-10-18T12:01:24.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00ab500 msgr2=0x7f83d0151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00ab500 0x7f83d0151ec0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009850 tx=0x7f83c402da60 comp rx=0 tx=0).stop 2023-10-18T12:01:24.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 shutdown_connections 2023-10-18T12:01:24.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f83bc06b080 0x7f83bc06d540 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00c96b0 0x7f83d01527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f83d00ab500 0x7f83d0151ec0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 --2- 172.21.15.125:0/77311404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f83d006af00 0x7f83d00a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 >> 172.21.15.125:0/77311404 conn(0x7f83d0069930 msgr2=0x7f83d013b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:24.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 shutdown_connections 2023-10-18T12:01:24.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.063+0000 7f83d5aaf640 1 -- 172.21.15.125:0/77311404 wait complete. 2023-10-18T12:01:24.076 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-18T12:01:22.839039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594034,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":682,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":420,"ondisk_log_size":420,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420356,"kb_used_data":25184,"kb_used_omap":12,"kb_used_meta":395059,"kb_avail":754554364,"statfs":{"total":773094113280,"available":772663668736,"internally_reserved":0,"allocated":25788416,"data_stored":13832589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":404540984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1256,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.398513"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454157+0000","last_change":"2023-10-18T12:01:19.454157+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454157+0000","last_undegraded":"2023-10-18T12:01:19.454157+0000","last_fullsized":"2023-10-18T12:01:19.454157+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465723+0000","last_change":"2023-10-18T12:01:19.464169+0000","last_active":"2023-10-18T12:01:19.465723+0000","last_peered":"2023-10-18T12:01:19.465723+0000","last_clean":"2023-10-18T12:01:19.465723+0000","last_became_active":"2023-10-18T12:01:19.463355+0000","last_became_peered":"2023-10-18T12:01:19.463355+0000","last_unstale":"2023-10-18T12:01:19.465723+0000","last_undegraded":"2023-10-18T12:01:19.465723+0000","last_fullsized":"2023-10-18T12:01:19.465723+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:00.513738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444418+0000","last_change":"2023-10-18T12:01:09.126968+0000","last_active":"2023-10-18T12:01:18.444418+0000","last_peered":"2023-10-18T12:01:18.444418+0000","last_clean":"2023-10-18T12:01:18.444418+0000","last_became_active":"2023-10-18T12:01:09.126705+0000","last_became_peered":"2023-10-18T12:01:09.126705+0000","last_unstale":"2023-10-18T12:01:18.444418+0000","last_undegraded":"2023-10-18T12:01:18.444418+0000","last_fullsized":"2023-10-18T12:01:18.444418+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:42:53.088272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453457+0000","last_change":"2023-10-18T12:01:19.453457+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453457+0000","last_undegraded":"2023-10-18T12:01:19.453457+0000","last_fullsized":"2023-10-18T12:01:19.453457+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468820+0000","last_change":"2023-10-18T12:01:19.468191+0000","last_active":"2023-10-18T12:01:19.468820+0000","last_peered":"2023-10-18T12:01:19.468820+0000","last_clean":"2023-10-18T12:01:19.468820+0000","last_became_active":"2023-10-18T12:01:19.467925+0000","last_became_peered":"2023-10-18T12:01:19.467925+0000","last_unstale":"2023-10-18T12:01:19.468820+0000","last_undegraded":"2023-10-18T12:01:19.468820+0000","last_fullsized":"2023-10-18T12:01:19.468820+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:58.124933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439901+0000","last_change":"2023-10-18T12:01:09.125772+0000","last_active":"2023-10-18T12:01:18.439901+0000","last_peered":"2023-10-18T12:01:18.439901+0000","last_clean":"2023-10-18T12:01:18.439901+0000","last_became_active":"2023-10-18T12:01:09.125508+0000","last_became_peered":"2023-10-18T12:01:09.125508+0000","last_unstale":"2023-10-18T12:01:18.439901+0000","last_undegraded":"2023-10-18T12:01:18.439901+0000","last_fullsized":"2023-10-18T12:01:18.439901+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:15.841764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451128+0000","last_change":"2023-10-18T12:01:19.451128+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451128+0000","last_undegraded":"2023-10-18T12:01:19.451128+0000","last_fullsized":"2023-10-18T12:01:19.451128+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.467291+0000","last_change":"2023-10-18T12:01:19.466687+0000","last_active":"2023-10-18T12:01:19.467291+0000","last_peered":"2023-10-18T12:01:19.467291+0000","last_clean":"2023-10-18T12:01:19.467291+0000","last_became_active":"2023-10-18T12:01:19.466395+0000","last_became_peered":"2023-10-18T12:01:19.466395+0000","last_unstale":"2023-10-18T12:01:19.467291+0000","last_undegraded":"2023-10-18T12:01:19.467291+0000","last_fullsized":"2023-10-18T12:01:19.467291+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:02.185827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444714+0000","last_change":"2023-10-18T12:01:09.124764+0000","last_active":"2023-10-18T12:01:18.444714+0000","last_peered":"2023-10-18T12:01:18.444714+0000","last_clean":"2023-10-18T12:01:18.444714+0000","last_became_active":"2023-10-18T12:01:09.124472+0000","last_became_peered":"2023-10-18T12:01:09.124472+0000","last_unstale":"2023-10-18T12:01:18.444714+0000","last_undegraded":"2023-10-18T12:01:18.444714+0000","last_fullsized":"2023-10-18T12:01:18.444714+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:29.891668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454232+0000","last_change":"2023-10-18T12:01:19.454232+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454232+0000","last_undegraded":"2023-10-18T12:01:19.454232+0000","last_fullsized":"2023-10-18T12:01:19.454232+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466648+0000","last_change":"2023-10-18T12:01:19.466181+0000","last_active":"2023-10-18T12:01:19.466648+0000","last_peered":"2023-10-18T12:01:19.466648+0000","last_clean":"2023-10-18T12:01:19.466648+0000","last_became_active":"2023-10-18T12:01:19.465901+0000","last_became_peered":"2023-10-18T12:01:19.465901+0000","last_unstale":"2023-10-18T12:01:19.466648+0000","last_undegraded":"2023-10-18T12:01:19.466648+0000","last_fullsized":"2023-10-18T12:01:19.466648+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:26:57.818590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444737+0000","last_change":"2023-10-18T12:01:09.132202+0000","last_active":"2023-10-18T12:01:18.444737+0000","last_peered":"2023-10-18T12:01:18.444737+0000","last_clean":"2023-10-18T12:01:18.444737+0000","last_became_active":"2023-10-18T12:01:09.131928+0000","last_became_peered":"2023-10-18T12:01:09.131928+0000","last_unstale":"2023-10-18T12:01:18.444737+0000","last_undegraded":"2023-10-18T12:01:18.444737+0000","last_fullsized":"2023-10-18T12:01:18.444737+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:34.826098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450577+0000","last_change":"2023-10-18T12:01:19.450577+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450577+0000","last_undegraded":"2023-10-18T12:01:19.450577+0000","last_fullsized":"2023-10-18T12:01:19.450577+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.461102+0000","last_change":"2023-10-18T12:01:19.459438+0000","last_active":"2023-10-18T12:01:19.461102+0000","last_peered":"2023-10-18T12:01:19.461102+0000","last_clean":"2023-10-18T12:01:19.461102+0000","last_became_active":"2023-10-18T12:01:19.459194+0000","last_became_peered":"2023-10-18T12:01:19.459194+0000","last_unstale":"2023-10-18T12:01:19.461102+0000","last_undegraded":"2023-10-18T12:01:19.461102+0000","last_fullsized":"2023-10-18T12:01:19.461102+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:53.933731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446379+0000","last_change":"2023-10-18T12:01:09.133870+0000","last_active":"2023-10-18T12:01:18.446379+0000","last_peered":"2023-10-18T12:01:18.446379+0000","last_clean":"2023-10-18T12:01:18.446379+0000","last_became_active":"2023-10-18T12:01:09.133478+0000","last_became_peered":"2023-10-18T12:01:09.133478+0000","last_unstale":"2023-10-18T12:01:18.446379+0000","last_undegraded":"2023-10-18T12:01:18.446379+0000","last_fullsized":"2023-10-18T12:01:18.446379+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:11.836101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450431+0000","last_change":"2023-10-18T12:01:19.450431+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450431+0000","last_undegraded":"2023-10-18T12:01:19.450431+0000","last_fullsized":"2023-10-18T12:01:19.450431+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465762+0000","last_change":"2023-10-18T12:01:19.463436+0000","last_active":"2023-10-18T12:01:19.465762+0000","last_peered":"2023-10-18T12:01:19.465762+0000","last_clean":"2023-10-18T12:01:19.465762+0000","last_became_active":"2023-10-18T12:01:19.463190+0000","last_became_peered":"2023-10-18T12:01:19.463190+0000","last_unstale":"2023-10-18T12:01:19.465762+0000","last_undegraded":"2023-10-18T12:01:19.465762+0000","last_fullsized":"2023-10-18T12:01:19.465762+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:55.180369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443988+0000","last_change":"2023-10-18T12:01:09.129113+0000","last_active":"2023-10-18T12:01:18.443988+0000","last_peered":"2023-10-18T12:01:18.443988+0000","last_clean":"2023-10-18T12:01:18.443988+0000","last_became_active":"2023-10-18T12:01:09.128785+0000","last_became_peered":"2023-10-18T12:01:09.128785+0000","last_unstale":"2023-10-18T12:01:18.443988+0000","last_undegraded":"2023-10-18T12:01:18.443988+0000","last_fullsized":"2023-10-18T12:01:18.443988+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:39.875471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458863+0000","last_change":"2023-10-18T12:01:19.458863+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458863+0000","last_undegraded":"2023-10-18T12:01:19.458863+0000","last_fullsized":"2023-10-18T12:01:19.458863+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465879+0000","last_change":"2023-10-18T12:01:19.463023+0000","last_active":"2023-10-18T12:01:19.465879+0000","last_peered":"2023-10-18T12:01:19.465879+0000","last_clean":"2023-10-18T12:01:19.465879+0000","last_became_active":"2023-10-18T12:01:19.462757+0000","last_became_peered":"2023-10-18T12:01:19.462757+0000","last_unstale":"2023-10-18T12:01:19.465879+0000","last_undegraded":"2023-10-18T12:01:19.465879+0000","last_fullsized":"2023-10-18T12:01:19.465879+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:09.123947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445459+0000","last_change":"2023-10-18T12:01:09.125901+0000","last_active":"2023-10-18T12:01:18.445459+0000","last_peered":"2023-10-18T12:01:18.445459+0000","last_clean":"2023-10-18T12:01:18.445459+0000","last_became_active":"2023-10-18T12:01:09.125390+0000","last_became_peered":"2023-10-18T12:01:09.125390+0000","last_unstale":"2023-10-18T12:01:18.445459+0000","last_undegraded":"2023-10-18T12:01:18.445459+0000","last_fullsized":"2023-10-18T12:01:18.445459+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:00.246587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451418+0000","last_change":"2023-10-18T12:01:19.451418+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451418+0000","last_undegraded":"2023-10-18T12:01:19.451418+0000","last_fullsized":"2023-10-18T12:01:19.451418+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465285+0000","last_change":"2023-10-18T12:01:19.463793+0000","last_active":"2023-10-18T12:01:19.465285+0000","last_peered":"2023-10-18T12:01:19.465285+0000","last_clean":"2023-10-18T12:01:19.465285+0000","last_became_active":"2023-10-18T12:01:19.463343+0000","last_became_peered":"2023-10-18T12:01:19.463343+0000","last_unstale":"2023-10-18T12:01:19.465285+0000","last_undegraded":"2023-10-18T12:01:19.465285+0000","last_fullsized":"2023-10-18T12:01:19.465285+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:34.979933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446196+0000","last_change":"2023-10-18T12:01:09.135081+0000","last_active":"2023-10-18T12:01:18.446196+0000","last_peered":"2023-10-18T12:01:18.446196+0000","last_clean":"2023-10-18T12:01:18.446196+0000","last_became_active":"2023-10-18T12:01:09.134801+0000","last_became_peered":"2023-10-18T12:01:09.134801+0000","last_unstale":"2023-10-18T12:01:18.446196+0000","last_undegraded":"2023-10-18T12:01:18.446196+0000","last_fullsized":"2023-10-18T12:01:18.446196+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:03.262316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452320+0000","last_change":"2023-10-18T12:01:19.452320+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452320+0000","last_undegraded":"2023-10-18T12:01:19.452320+0000","last_fullsized":"2023-10-18T12:01:19.452320+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466738+0000","last_change":"2023-10-18T12:01:19.465635+0000","last_active":"2023-10-18T12:01:19.466738+0000","last_peered":"2023-10-18T12:01:19.466738+0000","last_clean":"2023-10-18T12:01:19.466738+0000","last_became_active":"2023-10-18T12:01:19.465363+0000","last_became_peered":"2023-10-18T12:01:19.465363+0000","last_unstale":"2023-10-18T12:01:19.466738+0000","last_undegraded":"2023-10-18T12:01:19.466738+0000","last_fullsized":"2023-10-18T12:01:19.466738+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:56.587842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444490+0000","last_change":"2023-10-18T12:01:09.135615+0000","last_active":"2023-10-18T12:01:18.444490+0000","last_peered":"2023-10-18T12:01:18.444490+0000","last_clean":"2023-10-18T12:01:18.444490+0000","last_became_active":"2023-10-18T12:01:09.135123+0000","last_became_peered":"2023-10-18T12:01:09.135123+0000","last_unstale":"2023-10-18T12:01:18.444490+0000","last_undegraded":"2023-10-18T12:01:18.444490+0000","last_fullsized":"2023-10-18T12:01:18.444490+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:52.311354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458411+0000","last_change":"2023-10-18T12:01:19.458411+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458411+0000","last_undegraded":"2023-10-18T12:01:19.458411+0000","last_fullsized":"2023-10-18T12:01:19.458411+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465526+0000","last_change":"2023-10-18T12:01:19.465001+0000","last_active":"2023-10-18T12:01:19.465526+0000","last_peered":"2023-10-18T12:01:19.465526+0000","last_clean":"2023-10-18T12:01:19.465526+0000","last_became_active":"2023-10-18T12:01:19.464719+0000","last_became_peered":"2023-10-18T12:01:19.464719+0000","last_unstale":"2023-10-18T12:01:19.465526+0000","last_undegraded":"2023-10-18T12:01:19.465526+0000","last_fullsized":"2023-10-18T12:01:19.465526+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:08:12.543232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446023+0000","last_change":"2023-10-18T12:01:09.130673+0000","last_active":"2023-10-18T12:01:18.446023+0000","last_peered":"2023-10-18T12:01:18.446023+0000","last_clean":"2023-10-18T12:01:18.446023+0000","last_became_active":"2023-10-18T12:01:09.130094+0000","last_became_peered":"2023-10-18T12:01:09.130094+0000","last_unstale":"2023-10-18T12:01:18.446023+0000","last_undegraded":"2023-10-18T12:01:18.446023+0000","last_fullsized":"2023-10-18T12:01:18.446023+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:45.652023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456984+0000","last_change":"2023-10-18T12:01:19.456984+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456984+0000","last_undegraded":"2023-10-18T12:01:19.456984+0000","last_fullsized":"2023-10-18T12:01:19.456984+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465010+0000","last_change":"2023-10-18T12:01:19.463067+0000","last_active":"2023-10-18T12:01:19.465010+0000","last_peered":"2023-10-18T12:01:19.465010+0000","last_clean":"2023-10-18T12:01:19.465010+0000","last_became_active":"2023-10-18T12:01:19.462818+0000","last_became_peered":"2023-10-18T12:01:19.462818+0000","last_unstale":"2023-10-18T12:01:19.465010+0000","last_undegraded":"2023-10-18T12:01:19.465010+0000","last_fullsized":"2023-10-18T12:01:19.465010+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:50:23.807977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444802+0000","last_change":"2023-10-18T12:01:09.135828+0000","last_active":"2023-10-18T12:01:18.444802+0000","last_peered":"2023-10-18T12:01:18.444802+0000","last_clean":"2023-10-18T12:01:18.444802+0000","last_became_active":"2023-10-18T12:01:09.135579+0000","last_became_peered":"2023-10-18T12:01:09.135579+0000","last_unstale":"2023-10-18T12:01:18.444802+0000","last_undegraded":"2023-10-18T12:01:18.444802+0000","last_fullsized":"2023-10-18T12:01:18.444802+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:06.517585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454999+0000","last_change":"2023-10-18T12:01:19.454999+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454999+0000","last_undegraded":"2023-10-18T12:01:19.454999+0000","last_fullsized":"2023-10-18T12:01:19.454999+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.464639+0000","last_change":"2023-10-18T12:01:19.462716+0000","last_active":"2023-10-18T12:01:19.464639+0000","last_peered":"2023-10-18T12:01:19.464639+0000","last_clean":"2023-10-18T12:01:19.464639+0000","last_became_active":"2023-10-18T12:01:19.462474+0000","last_became_peered":"2023-10-18T12:01:19.462474+0000","last_unstale":"2023-10-18T12:01:19.464639+0000","last_undegraded":"2023-10-18T12:01:19.464639+0000","last_fullsized":"2023-10-18T12:01:19.464639+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:04.329561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443892+0000","last_change":"2023-10-18T12:01:09.132939+0000","last_active":"2023-10-18T12:01:18.443892+0000","last_peered":"2023-10-18T12:01:18.443892+0000","last_clean":"2023-10-18T12:01:18.443892+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:18.443892+0000","last_undegraded":"2023-10-18T12:01:18.443892+0000","last_fullsized":"2023-10-18T12:01:18.443892+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:22.550156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455867+0000","last_change":"2023-10-18T12:01:19.455867+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455867+0000","last_undegraded":"2023-10-18T12:01:19.455867+0000","last_fullsized":"2023-10-18T12:01:19.455867+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.459783+0000","last_change":"2023-10-18T12:01:19.458472+0000","last_active":"2023-10-18T12:01:19.459783+0000","last_peered":"2023-10-18T12:01:19.459783+0000","last_clean":"2023-10-18T12:01:19.459783+0000","last_became_active":"2023-10-18T12:01:19.458209+0000","last_became_peered":"2023-10-18T12:01:19.458209+0000","last_unstale":"2023-10-18T12:01:19.459783+0000","last_undegraded":"2023-10-18T12:01:19.459783+0000","last_fullsized":"2023-10-18T12:01:19.459783+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:04.778253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444252+0000","last_change":"2023-10-18T12:01:09.129764+0000","last_active":"2023-10-18T12:01:18.444252+0000","last_peered":"2023-10-18T12:01:18.444252+0000","last_clean":"2023-10-18T12:01:18.444252+0000","last_became_active":"2023-10-18T12:01:09.129483+0000","last_became_peered":"2023-10-18T12:01:09.129483+0000","last_unstale":"2023-10-18T12:01:18.444252+0000","last_undegraded":"2023-10-18T12:01:18.444252+0000","last_fullsized":"2023-10-18T12:01:18.444252+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:03.037837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453014+0000","last_change":"2023-10-18T12:01:19.453014+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453014+0000","last_undegraded":"2023-10-18T12:01:19.453014+0000","last_fullsized":"2023-10-18T12:01:19.453014+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468149+0000","last_change":"2023-10-18T12:01:19.466751+0000","last_active":"2023-10-18T12:01:19.468149+0000","last_peered":"2023-10-18T12:01:19.468149+0000","last_clean":"2023-10-18T12:01:19.468149+0000","last_became_active":"2023-10-18T12:01:19.466171+0000","last_became_peered":"2023-10-18T12:01:19.466171+0000","last_unstale":"2023-10-18T12:01:19.468149+0000","last_undegraded":"2023-10-18T12:01:19.468149+0000","last_fullsized":"2023-10-18T12:01:19.468149+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:01.102496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448554+0000","last_change":"2023-10-18T12:01:09.133024+0000","last_active":"2023-10-18T12:01:18.448554+0000","last_peered":"2023-10-18T12:01:18.448554+0000","last_clean":"2023-10-18T12:01:18.448554+0000","last_became_active":"2023-10-18T12:01:09.132742+0000","last_became_peered":"2023-10-18T12:01:09.132742+0000","last_unstale":"2023-10-18T12:01:18.448554+0000","last_undegraded":"2023-10-18T12:01:18.448554+0000","last_fullsized":"2023-10-18T12:01:18.448554+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456037+0000","last_change":"2023-10-18T12:01:19.456037+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456037+0000","last_undegraded":"2023-10-18T12:01:19.456037+0000","last_fullsized":"2023-10-18T12:01:19.456037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465243+0000","last_change":"2023-10-18T12:01:19.464582+0000","last_active":"2023-10-18T12:01:19.465243+0000","last_peered":"2023-10-18T12:01:19.465243+0000","last_clean":"2023-10-18T12:01:19.465243+0000","last_became_active":"2023-10-18T12:01:19.464325+0000","last_became_peered":"2023-10-18T12:01:19.464325+0000","last_unstale":"2023-10-18T12:01:19.465243+0000","last_undegraded":"2023-10-18T12:01:19.465243+0000","last_fullsized":"2023-10-18T12:01:19.465243+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:32:16.943070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445969+0000","last_change":"2023-10-18T12:01:09.131147+0000","last_active":"2023-10-18T12:01:18.445969+0000","last_peered":"2023-10-18T12:01:18.445969+0000","last_clean":"2023-10-18T12:01:18.445969+0000","last_became_active":"2023-10-18T12:01:09.130682+0000","last_became_peered":"2023-10-18T12:01:09.130682+0000","last_unstale":"2023-10-18T12:01:18.445969+0000","last_undegraded":"2023-10-18T12:01:18.445969+0000","last_fullsized":"2023-10-18T12:01:18.445969+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453907+0000","last_change":"2023-10-18T12:01:19.453907+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453907+0000","last_undegraded":"2023-10-18T12:01:19.453907+0000","last_fullsized":"2023-10-18T12:01:19.453907+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463821+0000","last_change":"2023-10-18T12:01:19.459426+0000","last_active":"2023-10-18T12:01:19.463821+0000","last_peered":"2023-10-18T12:01:19.463821+0000","last_clean":"2023-10-18T12:01:19.463821+0000","last_became_active":"2023-10-18T12:01:19.459168+0000","last_became_peered":"2023-10-18T12:01:19.459168+0000","last_unstale":"2023-10-18T12:01:19.463821+0000","last_undegraded":"2023-10-18T12:01:19.463821+0000","last_fullsized":"2023-10-18T12:01:19.463821+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:00.882760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445455+0000","last_change":"2023-10-18T12:01:09.122098+0000","last_active":"2023-10-18T12:01:18.445455+0000","last_peered":"2023-10-18T12:01:18.445455+0000","last_clean":"2023-10-18T12:01:18.445455+0000","last_became_active":"2023-10-18T12:01:09.121845+0000","last_became_peered":"2023-10-18T12:01:09.121845+0000","last_unstale":"2023-10-18T12:01:18.445455+0000","last_undegraded":"2023-10-18T12:01:18.445455+0000","last_fullsized":"2023-10-18T12:01:18.445455+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:19.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453747+0000","last_change":"2023-10-18T12:01:19.453747+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453747+0000","last_undegraded":"2023-10-18T12:01:19.453747+0000","last_fullsized":"2023-10-18T12:01:19.453747+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468474+0000","last_change":"2023-10-18T12:01:19.466123+0000","last_active":"2023-10-18T12:01:19.468474+0000","last_peered":"2023-10-18T12:01:19.468474+0000","last_clean":"2023-10-18T12:01:19.468474+0000","last_became_active":"2023-10-18T12:01:19.465856+0000","last_became_peered":"2023-10-18T12:01:19.465856+0000","last_unstale":"2023-10-18T12:01:19.468474+0000","last_undegraded":"2023-10-18T12:01:19.468474+0000","last_fullsized":"2023-10-18T12:01:19.468474+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:53.379907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447103+0000","last_change":"2023-10-18T12:01:09.133999+0000","last_active":"2023-10-18T12:01:18.447103+0000","last_peered":"2023-10-18T12:01:18.447103+0000","last_clean":"2023-10-18T12:01:18.447103+0000","last_became_active":"2023-10-18T12:01:09.133725+0000","last_became_peered":"2023-10-18T12:01:09.133725+0000","last_unstale":"2023-10-18T12:01:18.447103+0000","last_undegraded":"2023-10-18T12:01:18.447103+0000","last_fullsized":"2023-10-18T12:01:18.447103+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:35.006911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459506+0000","last_change":"2023-10-18T12:01:19.459506+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459506+0000","last_undegraded":"2023-10-18T12:01:19.459506+0000","last_fullsized":"2023-10-18T12:01:19.459506+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468784+0000","last_change":"2023-10-18T12:01:19.468308+0000","last_active":"2023-10-18T12:01:19.468784+0000","last_peered":"2023-10-18T12:01:19.468784+0000","last_clean":"2023-10-18T12:01:19.468784+0000","last_became_active":"2023-10-18T12:01:19.467998+0000","last_became_peered":"2023-10-18T12:01:19.467998+0000","last_unstale":"2023-10-18T12:01:19.468784+0000","last_undegraded":"2023-10-18T12:01:19.468784+0000","last_fullsized":"2023-10-18T12:01:19.468784+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:17.064462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446111+0000","last_change":"2023-10-18T12:01:09.124908+0000","last_active":"2023-10-18T12:01:18.446111+0000","last_peered":"2023-10-18T12:01:18.446111+0000","last_clean":"2023-10-18T12:01:18.446111+0000","last_became_active":"2023-10-18T12:01:09.124565+0000","last_became_peered":"2023-10-18T12:01:09.124565+0000","last_unstale":"2023-10-18T12:01:18.446111+0000","last_undegraded":"2023-10-18T12:01:18.446111+0000","last_fullsized":"2023-10-18T12:01:18.446111+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:22.044677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453595+0000","last_change":"2023-10-18T12:01:19.453595+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453595+0000","last_undegraded":"2023-10-18T12:01:19.453595+0000","last_fullsized":"2023-10-18T12:01:19.453595+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468762+0000","last_change":"2023-10-18T12:01:19.467971+0000","last_active":"2023-10-18T12:01:19.468762+0000","last_peered":"2023-10-18T12:01:19.468762+0000","last_clean":"2023-10-18T12:01:19.468762+0000","last_became_active":"2023-10-18T12:01:19.467699+0000","last_became_peered":"2023-10-18T12:01:19.467699+0000","last_unstale":"2023-10-18T12:01:19.468762+0000","last_undegraded":"2023-10-18T12:01:19.468762+0000","last_fullsized":"2023-10-18T12:01:19.468762+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:01.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447225+0000","last_change":"2023-10-18T12:01:09.135302+0000","last_active":"2023-10-18T12:01:18.447225+0000","last_peered":"2023-10-18T12:01:18.447225+0000","last_clean":"2023-10-18T12:01:18.447225+0000","last_became_active":"2023-10-18T12:01:09.134978+0000","last_became_peered":"2023-10-18T12:01:09.134978+0000","last_unstale":"2023-10-18T12:01:18.447225+0000","last_undegraded":"2023-10-18T12:01:18.447225+0000","last_fullsized":"2023-10-18T12:01:18.447225+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452783+0000","last_change":"2023-10-18T12:01:19.452783+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452783+0000","last_undegraded":"2023-10-18T12:01:19.452783+0000","last_fullsized":"2023-10-18T12:01:19.452783+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463777+0000","last_change":"2023-10-18T12:01:19.461558+0000","last_active":"2023-10-18T12:01:19.463777+0000","last_peered":"2023-10-18T12:01:19.463777+0000","last_clean":"2023-10-18T12:01:19.463777+0000","last_became_active":"2023-10-18T12:01:19.461307+0000","last_became_peered":"2023-10-18T12:01:19.461307+0000","last_unstale":"2023-10-18T12:01:19.463777+0000","last_undegraded":"2023-10-18T12:01:19.463777+0000","last_fullsized":"2023-10-18T12:01:19.463777+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:18:26.034701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440010+0000","last_change":"2023-10-18T12:01:09.136717+0000","last_active":"2023-10-18T12:01:18.440010+0000","last_peered":"2023-10-18T12:01:18.440010+0000","last_clean":"2023-10-18T12:01:18.440010+0000","last_became_active":"2023-10-18T12:01:09.136461+0000","last_became_peered":"2023-10-18T12:01:09.136461+0000","last_unstale":"2023-10-18T12:01:18.440010+0000","last_undegraded":"2023-10-18T12:01:18.440010+0000","last_fullsized":"2023-10-18T12:01:18.440010+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:02.336311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452844+0000","last_change":"2023-10-18T12:01:19.452844+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452844+0000","last_undegraded":"2023-10-18T12:01:19.452844+0000","last_fullsized":"2023-10-18T12:01:19.452844+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463533+0000","last_change":"2023-10-18T12:01:19.462240+0000","last_active":"2023-10-18T12:01:19.463533+0000","last_peered":"2023-10-18T12:01:19.463533+0000","last_clean":"2023-10-18T12:01:19.463533+0000","last_became_active":"2023-10-18T12:01:19.461957+0000","last_became_peered":"2023-10-18T12:01:19.461957+0000","last_unstale":"2023-10-18T12:01:19.463533+0000","last_undegraded":"2023-10-18T12:01:19.463533+0000","last_fullsized":"2023-10-18T12:01:19.463533+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:20.506577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445002+0000","last_change":"2023-10-18T12:01:09.130016+0000","last_active":"2023-10-18T12:01:18.445002+0000","last_peered":"2023-10-18T12:01:18.445002+0000","last_clean":"2023-10-18T12:01:18.445002+0000","last_became_active":"2023-10-18T12:01:09.129718+0000","last_became_peered":"2023-10-18T12:01:09.129718+0000","last_unstale":"2023-10-18T12:01:18.445002+0000","last_undegraded":"2023-10-18T12:01:18.445002+0000","last_fullsized":"2023-10-18T12:01:18.445002+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:36.455465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450045+0000","last_change":"2023-10-18T12:01:19.450045+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450045+0000","last_undegraded":"2023-10-18T12:01:19.450045+0000","last_fullsized":"2023-10-18T12:01:19.450045+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468581+0000","last_change":"2023-10-18T12:01:19.467158+0000","last_active":"2023-10-18T12:01:19.468581+0000","last_peered":"2023-10-18T12:01:19.468581+0000","last_clean":"2023-10-18T12:01:19.468581+0000","last_became_active":"2023-10-18T12:01:19.466854+0000","last_became_peered":"2023-10-18T12:01:19.466854+0000","last_unstale":"2023-10-18T12:01:19.468581+0000","last_undegraded":"2023-10-18T12:01:19.468581+0000","last_fullsized":"2023-10-18T12:01:19.468581+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:21.428576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443731+0000","last_change":"2023-10-18T12:01:09.129706+0000","last_active":"2023-10-18T12:01:18.443731+0000","last_peered":"2023-10-18T12:01:18.443731+0000","last_clean":"2023-10-18T12:01:18.443731+0000","last_became_active":"2023-10-18T12:01:09.129413+0000","last_became_peered":"2023-10-18T12:01:09.129413+0000","last_unstale":"2023-10-18T12:01:18.443731+0000","last_undegraded":"2023-10-18T12:01:18.443731+0000","last_fullsized":"2023-10-18T12:01:18.443731+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:55.309809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.449803+0000","last_change":"2023-10-18T12:01:19.449803+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.449803+0000","last_undegraded":"2023-10-18T12:01:19.449803+0000","last_fullsized":"2023-10-18T12:01:19.449803+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468823+0000","last_change":"2023-10-18T12:01:19.467623+0000","last_active":"2023-10-18T12:01:19.468823+0000","last_peered":"2023-10-18T12:01:19.468823+0000","last_clean":"2023-10-18T12:01:19.468823+0000","last_became_active":"2023-10-18T12:01:19.467358+0000","last_became_peered":"2023-10-18T12:01:19.467358+0000","last_unstale":"2023-10-18T12:01:19.468823+0000","last_undegraded":"2023-10-18T12:01:19.468823+0000","last_fullsized":"2023-10-18T12:01:19.468823+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:58.681098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446063+0000","last_change":"2023-10-18T12:01:09.133291+0000","last_active":"2023-10-18T12:01:18.446063+0000","last_peered":"2023-10-18T12:01:18.446063+0000","last_clean":"2023-10-18T12:01:18.446063+0000","last_became_active":"2023-10-18T12:01:09.132968+0000","last_became_peered":"2023-10-18T12:01:09.132968+0000","last_unstale":"2023-10-18T12:01:18.446063+0000","last_undegraded":"2023-10-18T12:01:18.446063+0000","last_fullsized":"2023-10-18T12:01:18.446063+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454604+0000","last_change":"2023-10-18T12:01:19.454604+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454604+0000","last_undegraded":"2023-10-18T12:01:19.454604+0000","last_fullsized":"2023-10-18T12:01:19.454604+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.460591+0000","last_change":"2023-10-18T12:01:19.458380+0000","last_active":"2023-10-18T12:01:19.460591+0000","last_peered":"2023-10-18T12:01:19.460591+0000","last_clean":"2023-10-18T12:01:19.460591+0000","last_became_active":"2023-10-18T12:01:19.458111+0000","last_became_peered":"2023-10-18T12:01:19.458111+0000","last_unstale":"2023-10-18T12:01:19.460591+0000","last_undegraded":"2023-10-18T12:01:19.460591+0000","last_fullsized":"2023-10-18T12:01:19.460591+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:27.878015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443343+0000","last_change":"2023-10-18T12:01:09.128954+0000","last_active":"2023-10-18T12:01:18.443343+0000","last_peered":"2023-10-18T12:01:18.443343+0000","last_clean":"2023-10-18T12:01:18.443343+0000","last_became_active":"2023-10-18T12:01:09.128655+0000","last_became_peered":"2023-10-18T12:01:09.128655+0000","last_unstale":"2023-10-18T12:01:18.443343+0000","last_undegraded":"2023-10-18T12:01:18.443343+0000","last_fullsized":"2023-10-18T12:01:18.443343+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:51.625573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455208+0000","last_change":"2023-10-18T12:01:19.455208+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455208+0000","last_undegraded":"2023-10-18T12:01:19.455208+0000","last_fullsized":"2023-10-18T12:01:19.455208+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468453+0000","last_change":"2023-10-18T12:01:19.466027+0000","last_active":"2023-10-18T12:01:19.468453+0000","last_peered":"2023-10-18T12:01:19.468453+0000","last_clean":"2023-10-18T12:01:19.468453+0000","last_became_active":"2023-10-18T12:01:19.465745+0000","last_became_peered":"2023-10-18T12:01:19.465745+0000","last_unstale":"2023-10-18T12:01:19.468453+0000","last_undegraded":"2023-10-18T12:01:19.468453+0000","last_fullsized":"2023-10-18T12:01:19.468453+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:15.002306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440030+0000","last_change":"2023-10-18T12:01:09.123355+0000","last_active":"2023-10-18T12:01:18.440030+0000","last_peered":"2023-10-18T12:01:18.440030+0000","last_clean":"2023-10-18T12:01:18.440030+0000","last_became_active":"2023-10-18T12:01:09.123096+0000","last_became_peered":"2023-10-18T12:01:09.123096+0000","last_unstale":"2023-10-18T12:01:18.440030+0000","last_undegraded":"2023-10-18T12:01:18.440030+0000","last_fullsized":"2023-10-18T12:01:18.440030+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:58.130314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454904+0000","last_change":"2023-10-18T12:01:19.454904+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454904+0000","last_undegraded":"2023-10-18T12:01:19.454904+0000","last_fullsized":"2023-10-18T12:01:19.454904+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468820+0000","last_change":"2023-10-18T12:01:19.467893+0000","last_active":"2023-10-18T12:01:19.468820+0000","last_peered":"2023-10-18T12:01:19.468820+0000","last_clean":"2023-10-18T12:01:19.468820+0000","last_became_active":"2023-10-18T12:01:19.467430+0000","last_became_peered":"2023-10-18T12:01:19.467430+0000","last_unstale":"2023-10-18T12:01:19.468820+0000","last_undegraded":"2023-10-18T12:01:19.468820+0000","last_fullsized":"2023-10-18T12:01:19.468820+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:12.990393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447739+0000","last_change":"2023-10-18T12:01:09.124947+0000","last_active":"2023-10-18T12:01:18.447739+0000","last_peered":"2023-10-18T12:01:18.447739+0000","last_clean":"2023-10-18T12:01:18.447739+0000","last_became_active":"2023-10-18T12:01:09.124666+0000","last_became_peered":"2023-10-18T12:01:09.124666+0000","last_unstale":"2023-10-18T12:01:18.447739+0000","last_undegraded":"2023-10-18T12:01:18.447739+0000","last_fullsized":"2023-10-18T12:01:18.447739+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:01.214167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454093+0000","last_change":"2023-10-18T12:01:19.454093+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454093+0000","last_undegraded":"2023-10-18T12:01:19.454093+0000","last_fullsized":"2023-10-18T12:01:19.454093+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463415+0000","last_change":"2023-10-18T12:01:19.462049+0000","last_active":"2023-10-18T12:01:19.463415+0000","last_peered":"2023-10-18T12:01:19.463415+0000","last_clean":"2023-10-18T12:01:19.463415+0000","last_became_active":"2023-10-18T12:01:19.460286+0000","last_became_peered":"2023-10-18T12:01:19.460286+0000","last_unstale":"2023-10-18T12:01:19.463415+0000","last_undegraded":"2023-10-18T12:01:19.463415+0000","last_fullsized":"2023-10-18T12:01:19.463415+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:59.023770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447065+0000","last_change":"2023-10-18T12:01:09.131674+0000","last_active":"2023-10-18T12:01:18.447065+0000","last_peered":"2023-10-18T12:01:18.447065+0000","last_clean":"2023-10-18T12:01:18.447065+0000","last_became_active":"2023-10-18T12:01:09.131410+0000","last_became_peered":"2023-10-18T12:01:09.131410+0000","last_unstale":"2023-10-18T12:01:18.447065+0000","last_undegraded":"2023-10-18T12:01:18.447065+0000","last_fullsized":"2023-10-18T12:01:18.447065+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:22.606198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451954+0000","last_change":"2023-10-18T12:01:19.451954+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451954+0000","last_undegraded":"2023-10-18T12:01:19.451954+0000","last_fullsized":"2023-10-18T12:01:19.451954+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463757+0000","last_change":"2023-10-18T12:01:19.460871+0000","last_active":"2023-10-18T12:01:19.463757+0000","last_peered":"2023-10-18T12:01:19.463757+0000","last_clean":"2023-10-18T12:01:19.463757+0000","last_became_active":"2023-10-18T12:01:19.460570+0000","last_became_peered":"2023-10-18T12:01:19.460570+0000","last_unstale":"2023-10-18T12:01:19.463757+0000","last_undegraded":"2023-10-18T12:01:19.463757+0000","last_fullsized":"2023-10-18T12:01:19.463757+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:21.229838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443961+0000","last_change":"2023-10-18T12:01:09.134865+0000","last_active":"2023-10-18T12:01:18.443961+0000","last_peered":"2023-10-18T12:01:18.443961+0000","last_clean":"2023-10-18T12:01:18.443961+0000","last_became_active":"2023-10-18T12:01:09.134583+0000","last_became_peered":"2023-10-18T12:01:09.134583+0000","last_unstale":"2023-10-18T12:01:18.443961+0000","last_undegraded":"2023-10-18T12:01:18.443961+0000","last_fullsized":"2023-10-18T12:01:18.443961+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:42.480656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.449313+0000","last_change":"2023-10-18T12:01:19.449313+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.449313+0000","last_undegraded":"2023-10-18T12:01:19.449313+0000","last_fullsized":"2023-10-18T12:01:19.449313+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.464534+0000","last_change":"2023-10-18T12:01:19.461910+0000","last_active":"2023-10-18T12:01:19.464534+0000","last_peered":"2023-10-18T12:01:19.464534+0000","last_clean":"2023-10-18T12:01:19.464534+0000","last_became_active":"2023-10-18T12:01:19.461653+0000","last_became_peered":"2023-10-18T12:01:19.461653+0000","last_unstale":"2023-10-18T12:01:19.464534+0000","last_undegraded":"2023-10-18T12:01:19.464534+0000","last_fullsized":"2023-10-18T12:01:19.464534+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:16:33.411957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448538+0000","last_change":"2023-10-18T12:01:09.132312+0000","last_active":"2023-10-18T12:01:18.448538+0000","last_peered":"2023-10-18T12:01:18.448538+0000","last_clean":"2023-10-18T12:01:18.448538+0000","last_became_active":"2023-10-18T12:01:09.131781+0000","last_became_peered":"2023-10-18T12:01:09.131781+0000","last_unstale":"2023-10-18T12:01:18.448538+0000","last_undegraded":"2023-10-18T12:01:18.448538+0000","last_fullsized":"2023-10-18T12:01:18.448538+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457884+0000","last_change":"2023-10-18T12:01:19.457884+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457884+0000","last_undegraded":"2023-10-18T12:01:19.457884+0000","last_fullsized":"2023-10-18T12:01:19.457884+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466498+0000","last_change":"2023-10-18T12:01:19.465117+0000","last_active":"2023-10-18T12:01:19.466498+0000","last_peered":"2023-10-18T12:01:19.466498+0000","last_clean":"2023-10-18T12:01:19.466498+0000","last_became_active":"2023-10-18T12:01:19.464845+0000","last_became_peered":"2023-10-18T12:01:19.464845+0000","last_unstale":"2023-10-18T12:01:19.466498+0000","last_undegraded":"2023-10-18T12:01:19.466498+0000","last_fullsized":"2023-10-18T12:01:19.466498+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:22.112086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448553+0000","last_change":"2023-10-18T12:01:09.133664+0000","last_active":"2023-10-18T12:01:18.448553+0000","last_peered":"2023-10-18T12:01:18.448553+0000","last_clean":"2023-10-18T12:01:18.448553+0000","last_became_active":"2023-10-18T12:01:09.133406+0000","last_became_peered":"2023-10-18T12:01:09.133406+0000","last_unstale":"2023-10-18T12:01:18.448553+0000","last_undegraded":"2023-10-18T12:01:18.448553+0000","last_fullsized":"2023-10-18T12:01:18.448553+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454282+0000","last_change":"2023-10-18T12:01:19.454282+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454282+0000","last_undegraded":"2023-10-18T12:01:19.454282+0000","last_fullsized":"2023-10-18T12:01:19.454282+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.461380+0000","last_change":"2023-10-18T12:01:19.458515+0000","last_active":"2023-10-18T12:01:19.461380+0000","last_peered":"2023-10-18T12:01:19.461380+0000","last_clean":"2023-10-18T12:01:19.461380+0000","last_became_active":"2023-10-18T12:01:19.457749+0000","last_became_peered":"2023-10-18T12:01:19.457749+0000","last_unstale":"2023-10-18T12:01:19.461380+0000","last_undegraded":"2023-10-18T12:01:19.461380+0000","last_fullsized":"2023-10-18T12:01:19.461380+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:24.677682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439756+0000","last_change":"2023-10-18T12:01:09.130142+0000","last_active":"2023-10-18T12:01:18.439756+0000","last_peered":"2023-10-18T12:01:18.439756+0000","last_clean":"2023-10-18T12:01:18.439756+0000","last_became_active":"2023-10-18T12:01:09.129884+0000","last_became_peered":"2023-10-18T12:01:09.129884+0000","last_unstale":"2023-10-18T12:01:18.439756+0000","last_undegraded":"2023-10-18T12:01:18.439756+0000","last_fullsized":"2023-10-18T12:01:18.439756+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:36.037273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450182+0000","last_change":"2023-10-18T12:01:19.450182+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450182+0000","last_undegraded":"2023-10-18T12:01:19.450182+0000","last_fullsized":"2023-10-18T12:01:19.450182+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.459660+0000","last_change":"2023-10-18T12:01:19.458409+0000","last_active":"2023-10-18T12:01:19.459660+0000","last_peered":"2023-10-18T12:01:19.459660+0000","last_clean":"2023-10-18T12:01:19.459660+0000","last_became_active":"2023-10-18T12:01:19.458151+0000","last_became_peered":"2023-10-18T12:01:19.458151+0000","last_unstale":"2023-10-18T12:01:19.459660+0000","last_undegraded":"2023-10-18T12:01:19.459660+0000","last_fullsized":"2023-10-18T12:01:19.459660+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:30.732460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443598+0000","last_change":"2023-10-18T12:01:09.136066+0000","last_active":"2023-10-18T12:01:18.443598+0000","last_peered":"2023-10-18T12:01:18.443598+0000","last_clean":"2023-10-18T12:01:18.443598+0000","last_became_active":"2023-10-18T12:01:09.135802+0000","last_became_peered":"2023-10-18T12:01:09.135802+0000","last_unstale":"2023-10-18T12:01:18.443598+0000","last_undegraded":"2023-10-18T12:01:18.443598+0000","last_fullsized":"2023-10-18T12:01:18.443598+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:26.550223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455962+0000","last_change":"2023-10-18T12:01:19.455962+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455962+0000","last_undegraded":"2023-10-18T12:01:19.455962+0000","last_fullsized":"2023-10-18T12:01:19.455962+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439658+0000","last_change":"2023-10-18T12:01:09.120812+0000","last_active":"2023-10-18T12:01:18.439658+0000","last_peered":"2023-10-18T12:01:18.439658+0000","last_clean":"2023-10-18T12:01:18.439658+0000","last_became_active":"2023-10-18T12:01:09.120561+0000","last_became_peered":"2023-10-18T12:01:09.120561+0000","last_unstale":"2023-10-18T12:01:18.439658+0000","last_undegraded":"2023-10-18T12:01:18.439658+0000","last_fullsized":"2023-10-18T12:01:18.439658+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:02.277242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"241'4","reported_seq":18,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444451+0000","last_change":"2023-10-18T12:01:08.112445+0000","last_active":"2023-10-18T12:01:18.444451+0000","last_peered":"2023-10-18T12:01:18.444451+0000","last_clean":"2023-10-18T12:01:18.444451+0000","last_became_active":"2023-10-18T12:01:08.112017+0000","last_became_peered":"2023-10-18T12:01:08.112017+0000","last_unstale":"2023-10-18T12:01:18.444451+0000","last_undegraded":"2023-10-18T12:01:18.444451+0000","last_fullsized":"2023-10-18T12:01:18.444451+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:11.607481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452634+0000","last_change":"2023-10-18T12:01:19.452634+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452634+0000","last_undegraded":"2023-10-18T12:01:19.452634+0000","last_fullsized":"2023-10-18T12:01:19.452634+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446081+0000","last_change":"2023-10-18T12:01:09.134916+0000","last_active":"2023-10-18T12:01:18.446081+0000","last_peered":"2023-10-18T12:01:18.446081+0000","last_clean":"2023-10-18T12:01:18.446081+0000","last_became_active":"2023-10-18T12:01:09.134630+0000","last_became_peered":"2023-10-18T12:01:09.134630+0000","last_unstale":"2023-10-18T12:01:18.446081+0000","last_undegraded":"2023-10-18T12:01:18.446081+0000","last_fullsized":"2023-10-18T12:01:18.446081+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447293+0000","last_change":"2023-10-18T12:01:08.119870+0000","last_active":"2023-10-18T12:01:18.447293+0000","last_peered":"2023-10-18T12:01:18.447293+0000","last_clean":"2023-10-18T12:01:18.447293+0000","last_became_active":"2023-10-18T12:01:08.119586+0000","last_became_peered":"2023-10-18T12:01:08.119586+0000","last_unstale":"2023-10-18T12:01:18.447293+0000","last_undegraded":"2023-10-18T12:01:18.447293+0000","last_fullsized":"2023-10-18T12:01:18.447293+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:16.288493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453087+0000","last_change":"2023-10-18T12:01:19.453087+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453087+0000","last_undegraded":"2023-10-18T12:01:19.453087+0000","last_fullsized":"2023-10-18T12:01:19.453087+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443755+0000","last_change":"2023-10-18T12:01:09.130894+0000","last_active":"2023-10-18T12:01:18.443755+0000","last_peered":"2023-10-18T12:01:18.443755+0000","last_clean":"2023-10-18T12:01:18.443755+0000","last_became_active":"2023-10-18T12:01:09.130614+0000","last_became_peered":"2023-10-18T12:01:09.130614+0000","last_unstale":"2023-10-18T12:01:18.443755+0000","last_undegraded":"2023-10-18T12:01:18.443755+0000","last_fullsized":"2023-10-18T12:01:18.443755+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:03.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448884+0000","last_change":"2023-10-18T12:01:08.115174+0000","last_active":"2023-10-18T12:01:18.448884+0000","last_peered":"2023-10-18T12:01:18.448884+0000","last_clean":"2023-10-18T12:01:18.448884+0000","last_became_active":"2023-10-18T12:01:08.114546+0000","last_became_peered":"2023-10-18T12:01:08.114546+0000","last_unstale":"2023-10-18T12:01:18.448884+0000","last_undegraded":"2023-10-18T12:01:18.448884+0000","last_fullsized":"2023-10-18T12:01:18.448884+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453540+0000","last_change":"2023-10-18T12:01:19.453540+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453540+0000","last_undegraded":"2023-10-18T12:01:19.453540+0000","last_fullsized":"2023-10-18T12:01:19.453540+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448819+0000","last_change":"2023-10-18T12:01:09.122601+0000","last_active":"2023-10-18T12:01:18.448819+0000","last_peered":"2023-10-18T12:01:18.448819+0000","last_clean":"2023-10-18T12:01:18.448819+0000","last_became_active":"2023-10-18T12:01:09.122334+0000","last_became_peered":"2023-10-18T12:01:09.122334+0000","last_unstale":"2023-10-18T12:01:18.448819+0000","last_undegraded":"2023-10-18T12:01:18.448819+0000","last_fullsized":"2023-10-18T12:01:18.448819+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446549+0000","last_change":"2023-10-18T12:01:08.116478+0000","last_active":"2023-10-18T12:01:18.446549+0000","last_peered":"2023-10-18T12:01:18.446549+0000","last_clean":"2023-10-18T12:01:18.446549+0000","last_became_active":"2023-10-18T12:01:08.116181+0000","last_became_peered":"2023-10-18T12:01:08.116181+0000","last_unstale":"2023-10-18T12:01:18.446549+0000","last_undegraded":"2023-10-18T12:01:18.446549+0000","last_fullsized":"2023-10-18T12:01:18.446549+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:10.087064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451810+0000","last_change":"2023-10-18T12:01:19.451810+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451810+0000","last_undegraded":"2023-10-18T12:01:19.451810+0000","last_fullsized":"2023-10-18T12:01:19.451810+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447548+0000","last_change":"2023-10-18T12:01:09.130166+0000","last_active":"2023-10-18T12:01:18.447548+0000","last_peered":"2023-10-18T12:01:18.447548+0000","last_clean":"2023-10-18T12:01:18.447548+0000","last_became_active":"2023-10-18T12:01:09.129725+0000","last_became_peered":"2023-10-18T12:01:09.129725+0000","last_unstale":"2023-10-18T12:01:18.447548+0000","last_undegraded":"2023-10-18T12:01:18.447548+0000","last_fullsized":"2023-10-18T12:01:18.447548+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:40.892842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448642+0000","last_change":"2023-10-18T12:01:08.107385+0000","last_active":"2023-10-18T12:01:18.448642+0000","last_peered":"2023-10-18T12:01:18.448642+0000","last_clean":"2023-10-18T12:01:18.448642+0000","last_became_active":"2023-10-18T12:01:08.107129+0000","last_became_peered":"2023-10-18T12:01:08.107129+0000","last_unstale":"2023-10-18T12:01:18.448642+0000","last_undegraded":"2023-10-18T12:01:18.448642+0000","last_fullsized":"2023-10-18T12:01:18.448642+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:43.838857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455957+0000","last_change":"2023-10-18T12:01:19.455957+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455957+0000","last_undegraded":"2023-10-18T12:01:19.455957+0000","last_fullsized":"2023-10-18T12:01:19.455957+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445933+0000","last_change":"2023-10-18T12:01:09.133737+0000","last_active":"2023-10-18T12:01:18.445933+0000","last_peered":"2023-10-18T12:01:18.445933+0000","last_clean":"2023-10-18T12:01:18.445933+0000","last_became_active":"2023-10-18T12:01:09.133487+0000","last_became_peered":"2023-10-18T12:01:09.133487+0000","last_unstale":"2023-10-18T12:01:18.445933+0000","last_undegraded":"2023-10-18T12:01:18.445933+0000","last_fullsized":"2023-10-18T12:01:18.445933+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:53:36.261500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445701+0000","last_change":"2023-10-18T12:01:08.119435+0000","last_active":"2023-10-18T12:01:18.445701+0000","last_peered":"2023-10-18T12:01:18.445701+0000","last_clean":"2023-10-18T12:01:18.445701+0000","last_became_active":"2023-10-18T12:01:08.118740+0000","last_became_peered":"2023-10-18T12:01:08.118740+0000","last_unstale":"2023-10-18T12:01:18.445701+0000","last_undegraded":"2023-10-18T12:01:18.445701+0000","last_fullsized":"2023-10-18T12:01:18.445701+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:54.172871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454152+0000","last_change":"2023-10-18T12:01:19.454152+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454152+0000","last_undegraded":"2023-10-18T12:01:19.454152+0000","last_fullsized":"2023-10-18T12:01:19.454152+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446839+0000","last_change":"2023-10-18T12:01:09.132487+0000","last_active":"2023-10-18T12:01:18.446839+0000","last_peered":"2023-10-18T12:01:18.446839+0000","last_clean":"2023-10-18T12:01:18.446839+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:18.446839+0000","last_undegraded":"2023-10-18T12:01:18.446839+0000","last_fullsized":"2023-10-18T12:01:18.446839+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:14.584151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443645+0000","last_change":"2023-10-18T12:01:08.115982+0000","last_active":"2023-10-18T12:01:18.443645+0000","last_peered":"2023-10-18T12:01:18.443645+0000","last_clean":"2023-10-18T12:01:18.443645+0000","last_became_active":"2023-10-18T12:01:08.115682+0000","last_became_peered":"2023-10-18T12:01:08.115682+0000","last_unstale":"2023-10-18T12:01:18.443645+0000","last_undegraded":"2023-10-18T12:01:18.443645+0000","last_fullsized":"2023-10-18T12:01:18.443645+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:36.789486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452968+0000","last_change":"2023-10-18T12:01:19.452968+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452968+0000","last_undegraded":"2023-10-18T12:01:19.452968+0000","last_fullsized":"2023-10-18T12:01:19.452968+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445246+0000","last_change":"2023-10-18T12:01:09.130010+0000","last_active":"2023-10-18T12:01:18.445246+0000","last_peered":"2023-10-18T12:01:18.445246+0000","last_clean":"2023-10-18T12:01:18.445246+0000","last_became_active":"2023-10-18T12:01:09.129744+0000","last_became_peered":"2023-10-18T12:01:09.129744+0000","last_unstale":"2023-10-18T12:01:18.445246+0000","last_undegraded":"2023-10-18T12:01:18.445246+0000","last_fullsized":"2023-10-18T12:01:18.445246+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:48:02.468955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448887+0000","last_change":"2023-10-18T12:01:08.110833+0000","last_active":"2023-10-18T12:01:18.448887+0000","last_peered":"2023-10-18T12:01:18.448887+0000","last_clean":"2023-10-18T12:01:18.448887+0000","last_became_active":"2023-10-18T12:01:08.110005+0000","last_became_peered":"2023-10-18T12:01:08.110005+0000","last_unstale":"2023-10-18T12:01:18.448887+0000","last_undegraded":"2023-10-18T12:01:18.448887+0000","last_fullsized":"2023-10-18T12:01:18.448887+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:32.311648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456437+0000","last_change":"2023-10-18T12:01:19.456437+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456437+0000","last_undegraded":"2023-10-18T12:01:19.456437+0000","last_fullsized":"2023-10-18T12:01:19.456437+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444394+0000","last_change":"2023-10-18T12:01:09.129659+0000","last_active":"2023-10-18T12:01:18.444394+0000","last_peered":"2023-10-18T12:01:18.444394+0000","last_clean":"2023-10-18T12:01:18.444394+0000","last_became_active":"2023-10-18T12:01:09.129388+0000","last_became_peered":"2023-10-18T12:01:09.129388+0000","last_unstale":"2023-10-18T12:01:18.444394+0000","last_undegraded":"2023-10-18T12:01:18.444394+0000","last_fullsized":"2023-10-18T12:01:18.444394+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:17.308708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446188+0000","last_change":"2023-10-18T12:01:08.118321+0000","last_active":"2023-10-18T12:01:18.446188+0000","last_peered":"2023-10-18T12:01:18.446188+0000","last_clean":"2023-10-18T12:01:18.446188+0000","last_became_active":"2023-10-18T12:01:08.117995+0000","last_became_peered":"2023-10-18T12:01:08.117995+0000","last_unstale":"2023-10-18T12:01:18.446188+0000","last_undegraded":"2023-10-18T12:01:18.446188+0000","last_fullsized":"2023-10-18T12:01:18.446188+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:46.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453041+0000","last_change":"2023-10-18T12:01:19.453041+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453041+0000","last_undegraded":"2023-10-18T12:01:19.453041+0000","last_fullsized":"2023-10-18T12:01:19.453041+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445227+0000","last_change":"2023-10-18T12:01:09.134396+0000","last_active":"2023-10-18T12:01:18.445227+0000","last_peered":"2023-10-18T12:01:18.445227+0000","last_clean":"2023-10-18T12:01:18.445227+0000","last_became_active":"2023-10-18T12:01:09.134144+0000","last_became_peered":"2023-10-18T12:01:09.134144+0000","last_unstale":"2023-10-18T12:01:18.445227+0000","last_undegraded":"2023-10-18T12:01:18.445227+0000","last_fullsized":"2023-10-18T12:01:18.445227+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:23:53.396657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"241'3","reported_seq":17,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447167+0000","last_change":"2023-10-18T12:01:08.118871+0000","last_active":"2023-10-18T12:01:18.447167+0000","last_peered":"2023-10-18T12:01:18.447167+0000","last_clean":"2023-10-18T12:01:18.447167+0000","last_became_active":"2023-10-18T12:01:08.118572+0000","last_became_peered":"2023-10-18T12:01:08.118572+0000","last_unstale":"2023-10-18T12:01:18.447167+0000","last_undegraded":"2023-10-18T12:01:18.447167+0000","last_fullsized":"2023-10-18T12:01:18.447167+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:45.510206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453957+0000","last_change":"2023-10-18T12:01:19.453957+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453957+0000","last_undegraded":"2023-10-18T12:01:19.453957+0000","last_fullsized":"2023-10-18T12:01:19.453957+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444883+0000","last_change":"2023-10-18T12:01:09.120444+0000","last_active":"2023-10-18T12:01:18.444883+0000","last_peered":"2023-10-18T12:01:18.444883+0000","last_clean":"2023-10-18T12:01:18.444883+0000","last_became_active":"2023-10-18T12:01:09.120018+0000","last_became_peered":"2023-10-18T12:01:09.120018+0000","last_unstale":"2023-10-18T12:01:18.444883+0000","last_undegraded":"2023-10-18T12:01:18.444883+0000","last_fullsized":"2023-10-18T12:01:18.444883+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:29.145722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.370024+0000","last_change":"2023-10-18T12:01:08.117890+0000","last_active":"2023-10-18T12:01:19.370024+0000","last_peered":"2023-10-18T12:01:19.370024+0000","last_clean":"2023-10-18T12:01:19.370024+0000","last_became_active":"2023-10-18T12:01:08.117606+0000","last_became_peered":"2023-10-18T12:01:08.117606+0000","last_unstale":"2023-10-18T12:01:19.370024+0000","last_undegraded":"2023-10-18T12:01:19.370024+0000","last_fullsized":"2023-10-18T12:01:19.370024+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:49.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454638+0000","last_change":"2023-10-18T12:01:19.454638+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454638+0000","last_undegraded":"2023-10-18T12:01:19.454638+0000","last_fullsized":"2023-10-18T12:01:19.454638+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439867+0000","last_change":"2023-10-18T12:01:09.127054+0000","last_active":"2023-10-18T12:01:18.439867+0000","last_peered":"2023-10-18T12:01:18.439867+0000","last_clean":"2023-10-18T12:01:18.439867+0000","last_became_active":"2023-10-18T12:01:09.126804+0000","last_became_peered":"2023-10-18T12:01:09.126804+0000","last_unstale":"2023-10-18T12:01:18.439867+0000","last_undegraded":"2023-10-18T12:01:18.439867+0000","last_fullsized":"2023-10-18T12:01:18.439867+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:20.948970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446639+0000","last_change":"2023-10-18T12:01:08.118544+0000","last_active":"2023-10-18T12:01:18.446639+0000","last_peered":"2023-10-18T12:01:18.446639+0000","last_clean":"2023-10-18T12:01:18.446639+0000","last_became_active":"2023-10-18T12:01:08.118228+0000","last_became_peered":"2023-10-18T12:01:08.118228+0000","last_unstale":"2023-10-18T12:01:18.446639+0000","last_undegraded":"2023-10-18T12:01:18.446639+0000","last_fullsized":"2023-10-18T12:01:18.446639+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456191+0000","last_change":"2023-10-18T12:01:19.456191+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456191+0000","last_undegraded":"2023-10-18T12:01:19.456191+0000","last_fullsized":"2023-10-18T12:01:19.456191+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447511+0000","last_change":"2023-10-18T12:01:09.120750+0000","last_active":"2023-10-18T12:01:18.447511+0000","last_peered":"2023-10-18T12:01:18.447511+0000","last_clean":"2023-10-18T12:01:18.447511+0000","last_became_active":"2023-10-18T12:01:09.120490+0000","last_became_peered":"2023-10-18T12:01:09.120490+0000","last_unstale":"2023-10-18T12:01:18.447511+0000","last_undegraded":"2023-10-18T12:01:18.447511+0000","last_fullsized":"2023-10-18T12:01:18.447511+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:51.005840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446987+0000","last_change":"2023-10-18T12:01:08.118022+0000","last_active":"2023-10-18T12:01:18.446987+0000","last_peered":"2023-10-18T12:01:18.446987+0000","last_clean":"2023-10-18T12:01:18.446987+0000","last_became_active":"2023-10-18T12:01:08.117754+0000","last_became_peered":"2023-10-18T12:01:08.117754+0000","last_unstale":"2023-10-18T12:01:18.446987+0000","last_undegraded":"2023-10-18T12:01:18.446987+0000","last_fullsized":"2023-10-18T12:01:18.446987+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:37.834293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455355+0000","last_change":"2023-10-18T12:01:19.455355+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455355+0000","last_undegraded":"2023-10-18T12:01:19.455355+0000","last_fullsized":"2023-10-18T12:01:19.455355+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447313+0000","last_change":"2023-10-18T12:01:09.135383+0000","last_active":"2023-10-18T12:01:18.447313+0000","last_peered":"2023-10-18T12:01:18.447313+0000","last_clean":"2023-10-18T12:01:18.447313+0000","last_became_active":"2023-10-18T12:01:09.135114+0000","last_became_peered":"2023-10-18T12:01:09.135114+0000","last_unstale":"2023-10-18T12:01:18.447313+0000","last_undegraded":"2023-10-18T12:01:18.447313+0000","last_fullsized":"2023-10-18T12:01:18.447313+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446147+0000","last_change":"2023-10-18T12:01:08.115155+0000","last_active":"2023-10-18T12:01:18.446147+0000","last_peered":"2023-10-18T12:01:18.446147+0000","last_clean":"2023-10-18T12:01:18.446147+0000","last_became_active":"2023-10-18T12:01:08.114843+0000","last_became_peered":"2023-10-18T12:01:08.114843+0000","last_unstale":"2023-10-18T12:01:18.446147+0000","last_undegraded":"2023-10-18T12:01:18.446147+0000","last_fullsized":"2023-10-18T12:01:18.446147+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:28.358756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455037+0000","last_change":"2023-10-18T12:01:19.455037+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455037+0000","last_undegraded":"2023-10-18T12:01:19.455037+0000","last_fullsized":"2023-10-18T12:01:19.455037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446744+0000","last_change":"2023-10-18T12:01:09.131820+0000","last_active":"2023-10-18T12:01:18.446744+0000","last_peered":"2023-10-18T12:01:18.446744+0000","last_clean":"2023-10-18T12:01:18.446744+0000","last_became_active":"2023-10-18T12:01:09.131568+0000","last_became_peered":"2023-10-18T12:01:09.131568+0000","last_unstale":"2023-10-18T12:01:18.446744+0000","last_undegraded":"2023-10-18T12:01:18.446744+0000","last_fullsized":"2023-10-18T12:01:18.446744+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.395799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"241'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439719+0000","last_change":"2023-10-18T12:01:08.112711+0000","last_active":"2023-10-18T12:01:18.439719+0000","last_peered":"2023-10-18T12:01:18.439719+0000","last_clean":"2023-10-18T12:01:18.439719+0000","last_became_active":"2023-10-18T12:01:08.112436+0000","last_became_peered":"2023-10-18T12:01:08.112436+0000","last_unstale":"2023-10-18T12:01:18.439719+0000","last_undegraded":"2023-10-18T12:01:18.439719+0000","last_fullsized":"2023-10-18T12:01:18.439719+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:15.568916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455648+0000","last_change":"2023-10-18T12:01:19.455648+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455648+0000","last_undegraded":"2023-10-18T12:01:19.455648+0000","last_fullsized":"2023-10-18T12:01:19.455648+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444666+0000","last_change":"2023-10-18T12:01:09.135106+0000","last_active":"2023-10-18T12:01:18.444666+0000","last_peered":"2023-10-18T12:01:18.444666+0000","last_clean":"2023-10-18T12:01:18.444666+0000","last_became_active":"2023-10-18T12:01:09.134828+0000","last_became_peered":"2023-10-18T12:01:09.134828+0000","last_unstale":"2023-10-18T12:01:18.444666+0000","last_undegraded":"2023-10-18T12:01:18.444666+0000","last_fullsized":"2023-10-18T12:01:18.444666+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:39.126921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444574+0000","last_change":"2023-10-18T12:01:08.119248+0000","last_active":"2023-10-18T12:01:18.444574+0000","last_peered":"2023-10-18T12:01:18.444574+0000","last_clean":"2023-10-18T12:01:18.444574+0000","last_became_active":"2023-10-18T12:01:08.118973+0000","last_became_peered":"2023-10-18T12:01:08.118973+0000","last_unstale":"2023-10-18T12:01:18.444574+0000","last_undegraded":"2023-10-18T12:01:18.444574+0000","last_fullsized":"2023-10-18T12:01:18.444574+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:37.079812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459833+0000","last_change":"2023-10-18T12:01:19.459833+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459833+0000","last_undegraded":"2023-10-18T12:01:19.459833+0000","last_fullsized":"2023-10-18T12:01:19.459833+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445946+0000","last_change":"2023-10-18T12:01:09.132827+0000","last_active":"2023-10-18T12:01:18.445946+0000","last_peered":"2023-10-18T12:01:18.445946+0000","last_clean":"2023-10-18T12:01:18.445946+0000","last_became_active":"2023-10-18T12:01:09.132545+0000","last_became_peered":"2023-10-18T12:01:09.132545+0000","last_unstale":"2023-10-18T12:01:18.445946+0000","last_undegraded":"2023-10-18T12:01:18.445946+0000","last_fullsized":"2023-10-18T12:01:18.445946+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:02.034692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439997+0000","last_change":"2023-10-18T12:01:08.107964+0000","last_active":"2023-10-18T12:01:18.439997+0000","last_peered":"2023-10-18T12:01:18.439997+0000","last_clean":"2023-10-18T12:01:18.439997+0000","last_became_active":"2023-10-18T12:01:08.107696+0000","last_became_peered":"2023-10-18T12:01:08.107696+0000","last_unstale":"2023-10-18T12:01:18.439997+0000","last_undegraded":"2023-10-18T12:01:18.439997+0000","last_fullsized":"2023-10-18T12:01:18.439997+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:18.578919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453669+0000","last_change":"2023-10-18T12:01:19.453669+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453669+0000","last_undegraded":"2023-10-18T12:01:19.453669+0000","last_fullsized":"2023-10-18T12:01:19.453669+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439817+0000","last_change":"2023-10-18T12:01:09.136078+0000","last_active":"2023-10-18T12:01:18.439817+0000","last_peered":"2023-10-18T12:01:18.439817+0000","last_clean":"2023-10-18T12:01:18.439817+0000","last_became_active":"2023-10-18T12:01:09.134722+0000","last_became_peered":"2023-10-18T12:01:09.134722+0000","last_unstale":"2023-10-18T12:01:18.439817+0000","last_undegraded":"2023-10-18T12:01:18.439817+0000","last_fullsized":"2023-10-18T12:01:18.439817+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:09.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447390+0000","last_change":"2023-10-18T12:01:08.120416+0000","last_active":"2023-10-18T12:01:18.447390+0000","last_peered":"2023-10-18T12:01:18.447390+0000","last_clean":"2023-10-18T12:01:18.447390+0000","last_became_active":"2023-10-18T12:01:08.120122+0000","last_became_peered":"2023-10-18T12:01:08.120122+0000","last_unstale":"2023-10-18T12:01:18.447390+0000","last_undegraded":"2023-10-18T12:01:18.447390+0000","last_fullsized":"2023-10-18T12:01:18.447390+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:26.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455164+0000","last_change":"2023-10-18T12:01:19.455164+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455164+0000","last_undegraded":"2023-10-18T12:01:19.455164+0000","last_fullsized":"2023-10-18T12:01:19.455164+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446070+0000","last_change":"2023-10-18T12:01:09.135042+0000","last_active":"2023-10-18T12:01:18.446070+0000","last_peered":"2023-10-18T12:01:18.446070+0000","last_clean":"2023-10-18T12:01:18.446070+0000","last_became_active":"2023-10-18T12:01:09.134765+0000","last_became_peered":"2023-10-18T12:01:09.134765+0000","last_unstale":"2023-10-18T12:01:18.446070+0000","last_undegraded":"2023-10-18T12:01:18.446070+0000","last_fullsized":"2023-10-18T12:01:18.446070+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:58.131101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446991+0000","last_change":"2023-10-18T12:01:08.117972+0000","last_active":"2023-10-18T12:01:18.446991+0000","last_peered":"2023-10-18T12:01:18.446991+0000","last_clean":"2023-10-18T12:01:18.446991+0000","last_became_active":"2023-10-18T12:01:08.117690+0000","last_became_peered":"2023-10-18T12:01:08.117690+0000","last_unstale":"2023-10-18T12:01:18.446991+0000","last_undegraded":"2023-10-18T12:01:18.446991+0000","last_fullsized":"2023-10-18T12:01:18.446991+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458648+0000","last_change":"2023-10-18T12:01:19.458648+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458648+0000","last_undegraded":"2023-10-18T12:01:19.458648+0000","last_fullsized":"2023-10-18T12:01:19.458648+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444663+0000","last_change":"2023-10-18T12:01:09.122673+0000","last_active":"2023-10-18T12:01:18.444663+0000","last_peered":"2023-10-18T12:01:18.444663+0000","last_clean":"2023-10-18T12:01:18.444663+0000","last_became_active":"2023-10-18T12:01:09.122430+0000","last_became_peered":"2023-10-18T12:01:09.122430+0000","last_unstale":"2023-10-18T12:01:18.444663+0000","last_undegraded":"2023-10-18T12:01:18.444663+0000","last_fullsized":"2023-10-18T12:01:18.444663+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:29.387267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446157+0000","last_change":"2023-10-18T12:01:08.119439+0000","last_active":"2023-10-18T12:01:18.446157+0000","last_peered":"2023-10-18T12:01:18.446157+0000","last_clean":"2023-10-18T12:01:18.446157+0000","last_became_active":"2023-10-18T12:01:08.119158+0000","last_became_peered":"2023-10-18T12:01:08.119158+0000","last_unstale":"2023-10-18T12:01:18.446157+0000","last_undegraded":"2023-10-18T12:01:18.446157+0000","last_fullsized":"2023-10-18T12:01:18.446157+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:56.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454552+0000","last_change":"2023-10-18T12:01:19.454552+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454552+0000","last_undegraded":"2023-10-18T12:01:19.454552+0000","last_fullsized":"2023-10-18T12:01:19.454552+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440049+0000","last_change":"2023-10-18T12:01:09.124352+0000","last_active":"2023-10-18T12:01:18.440049+0000","last_peered":"2023-10-18T12:01:18.440049+0000","last_clean":"2023-10-18T12:01:18.440049+0000","last_became_active":"2023-10-18T12:01:09.124072+0000","last_became_peered":"2023-10-18T12:01:09.124072+0000","last_unstale":"2023-10-18T12:01:18.440049+0000","last_undegraded":"2023-10-18T12:01:18.440049+0000","last_fullsized":"2023-10-18T12:01:18.440049+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:15.239290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443813+0000","last_change":"2023-10-18T12:01:08.117335+0000","last_active":"2023-10-18T12:01:18.443813+0000","last_peered":"2023-10-18T12:01:18.443813+0000","last_clean":"2023-10-18T12:01:18.443813+0000","last_became_active":"2023-10-18T12:01:08.117032+0000","last_became_peered":"2023-10-18T12:01:08.117032+0000","last_unstale":"2023-10-18T12:01:18.443813+0000","last_undegraded":"2023-10-18T12:01:18.443813+0000","last_fullsized":"2023-10-18T12:01:18.443813+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:33.304569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456597+0000","last_change":"2023-10-18T12:01:19.456597+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456597+0000","last_undegraded":"2023-10-18T12:01:19.456597+0000","last_fullsized":"2023-10-18T12:01:19.456597+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446661+0000","last_change":"2023-10-18T12:01:09.135148+0000","last_active":"2023-10-18T12:01:18.446661+0000","last_peered":"2023-10-18T12:01:18.446661+0000","last_clean":"2023-10-18T12:01:18.446661+0000","last_became_active":"2023-10-18T12:01:09.134895+0000","last_became_peered":"2023-10-18T12:01:09.134895+0000","last_unstale":"2023-10-18T12:01:18.446661+0000","last_undegraded":"2023-10-18T12:01:18.446661+0000","last_fullsized":"2023-10-18T12:01:18.446661+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:19.263310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444467+0000","last_change":"2023-10-18T12:01:08.120618+0000","last_active":"2023-10-18T12:01:18.444467+0000","last_peered":"2023-10-18T12:01:18.444467+0000","last_clean":"2023-10-18T12:01:18.444467+0000","last_became_active":"2023-10-18T12:01:08.120315+0000","last_became_peered":"2023-10-18T12:01:08.120315+0000","last_unstale":"2023-10-18T12:01:18.444467+0000","last_undegraded":"2023-10-18T12:01:18.444467+0000","last_fullsized":"2023-10-18T12:01:18.444467+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:36.555722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457598+0000","last_change":"2023-10-18T12:01:19.457598+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457598+0000","last_undegraded":"2023-10-18T12:01:19.457598+0000","last_fullsized":"2023-10-18T12:01:19.457598+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447164+0000","last_change":"2023-10-18T12:01:09.130881+0000","last_active":"2023-10-18T12:01:18.447164+0000","last_peered":"2023-10-18T12:01:18.447164+0000","last_clean":"2023-10-18T12:01:18.447164+0000","last_became_active":"2023-10-18T12:01:09.130628+0000","last_became_peered":"2023-10-18T12:01:09.130628+0000","last_unstale":"2023-10-18T12:01:18.447164+0000","last_undegraded":"2023-10-18T12:01:18.447164+0000","last_fullsized":"2023-10-18T12:01:18.447164+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:13.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448426+0000","last_change":"2023-10-18T12:01:08.120313+0000","last_active":"2023-10-18T12:01:18.448426+0000","last_peered":"2023-10-18T12:01:18.448426+0000","last_clean":"2023-10-18T12:01:18.448426+0000","last_became_active":"2023-10-18T12:01:08.120059+0000","last_became_peered":"2023-10-18T12:01:08.120059+0000","last_unstale":"2023-10-18T12:01:18.448426+0000","last_undegraded":"2023-10-18T12:01:18.448426+0000","last_fullsized":"2023-10-18T12:01:18.448426+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456394+0000","last_change":"2023-10-18T12:01:19.456394+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456394+0000","last_undegraded":"2023-10-18T12:01:19.456394+0000","last_fullsized":"2023-10-18T12:01:19.456394+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445079+0000","last_change":"2023-10-18T12:01:09.125685+0000","last_active":"2023-10-18T12:01:18.445079+0000","last_peered":"2023-10-18T12:01:18.445079+0000","last_clean":"2023-10-18T12:01:18.445079+0000","last_became_active":"2023-10-18T12:01:09.125395+0000","last_became_peered":"2023-10-18T12:01:09.125395+0000","last_unstale":"2023-10-18T12:01:18.445079+0000","last_undegraded":"2023-10-18T12:01:18.445079+0000","last_fullsized":"2023-10-18T12:01:18.445079+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:47.228016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444968+0000","last_change":"2023-10-18T12:01:08.117717+0000","last_active":"2023-10-18T12:01:18.444968+0000","last_peered":"2023-10-18T12:01:18.444968+0000","last_clean":"2023-10-18T12:01:18.444968+0000","last_became_active":"2023-10-18T12:01:08.117405+0000","last_became_peered":"2023-10-18T12:01:08.117405+0000","last_unstale":"2023-10-18T12:01:18.444968+0000","last_undegraded":"2023-10-18T12:01:18.444968+0000","last_fullsized":"2023-10-18T12:01:18.444968+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:00.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454367+0000","last_change":"2023-10-18T12:01:19.454367+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454367+0000","last_undegraded":"2023-10-18T12:01:19.454367+0000","last_fullsized":"2023-10-18T12:01:19.454367+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445826+0000","last_change":"2023-10-18T12:01:09.132599+0000","last_active":"2023-10-18T12:01:18.445826+0000","last_peered":"2023-10-18T12:01:18.445826+0000","last_clean":"2023-10-18T12:01:18.445826+0000","last_became_active":"2023-10-18T12:01:09.131710+0000","last_became_peered":"2023-10-18T12:01:09.131710+0000","last_unstale":"2023-10-18T12:01:18.445826+0000","last_undegraded":"2023-10-18T12:01:18.445826+0000","last_fullsized":"2023-10-18T12:01:18.445826+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:18.701174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446916+0000","last_change":"2023-10-18T12:01:08.118553+0000","last_active":"2023-10-18T12:01:18.446916+0000","last_peered":"2023-10-18T12:01:18.446916+0000","last_clean":"2023-10-18T12:01:18.446916+0000","last_became_active":"2023-10-18T12:01:08.118286+0000","last_became_peered":"2023-10-18T12:01:08.118286+0000","last_unstale":"2023-10-18T12:01:18.446916+0000","last_undegraded":"2023-10-18T12:01:18.446916+0000","last_fullsized":"2023-10-18T12:01:18.446916+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:05:32.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.460259+0000","last_change":"2023-10-18T12:01:19.460259+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.460259+0000","last_undegraded":"2023-10-18T12:01:19.460259+0000","last_fullsized":"2023-10-18T12:01:19.460259+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445148+0000","last_change":"2023-10-18T12:01:09.133522+0000","last_active":"2023-10-18T12:01:18.445148+0000","last_peered":"2023-10-18T12:01:18.445148+0000","last_clean":"2023-10-18T12:01:18.445148+0000","last_became_active":"2023-10-18T12:01:09.133267+0000","last_became_peered":"2023-10-18T12:01:09.133267+0000","last_unstale":"2023-10-18T12:01:18.445148+0000","last_undegraded":"2023-10-18T12:01:18.445148+0000","last_fullsized":"2023-10-18T12:01:18.445148+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:44:14.702965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448989+0000","last_change":"2023-10-18T12:01:08.116830+0000","last_active":"2023-10-18T12:01:18.448989+0000","last_peered":"2023-10-18T12:01:18.448989+0000","last_clean":"2023-10-18T12:01:18.448989+0000","last_became_active":"2023-10-18T12:01:08.116538+0000","last_became_peered":"2023-10-18T12:01:08.116538+0000","last_unstale":"2023-10-18T12:01:18.448989+0000","last_undegraded":"2023-10-18T12:01:18.448989+0000","last_fullsized":"2023-10-18T12:01:18.448989+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:09.165065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451313+0000","last_change":"2023-10-18T12:01:19.451313+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451313+0000","last_undegraded":"2023-10-18T12:01:19.451313+0000","last_fullsized":"2023-10-18T12:01:19.451313+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446209+0000","last_change":"2023-10-18T12:01:09.128305+0000","last_active":"2023-10-18T12:01:18.446209+0000","last_peered":"2023-10-18T12:01:18.446209+0000","last_clean":"2023-10-18T12:01:18.446209+0000","last_became_active":"2023-10-18T12:01:09.127933+0000","last_became_peered":"2023-10-18T12:01:09.127933+0000","last_unstale":"2023-10-18T12:01:18.446209+0000","last_undegraded":"2023-10-18T12:01:18.446209+0000","last_fullsized":"2023-10-18T12:01:18.446209+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:28.312681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448893+0000","last_change":"2023-10-18T12:01:08.115363+0000","last_active":"2023-10-18T12:01:18.448893+0000","last_peered":"2023-10-18T12:01:18.448893+0000","last_clean":"2023-10-18T12:01:18.448893+0000","last_became_active":"2023-10-18T12:01:08.114903+0000","last_became_peered":"2023-10-18T12:01:08.114903+0000","last_unstale":"2023-10-18T12:01:18.448893+0000","last_undegraded":"2023-10-18T12:01:18.448893+0000","last_fullsized":"2023-10-18T12:01:18.448893+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:46.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456219+0000","last_change":"2023-10-18T12:01:19.456219+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456219+0000","last_undegraded":"2023-10-18T12:01:19.456219+0000","last_fullsized":"2023-10-18T12:01:19.456219+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448801+0000","last_change":"2023-10-18T12:01:09.127159+0000","last_active":"2023-10-18T12:01:18.448801+0000","last_peered":"2023-10-18T12:01:18.448801+0000","last_clean":"2023-10-18T12:01:18.448801+0000","last_became_active":"2023-10-18T12:01:09.126874+0000","last_became_peered":"2023-10-18T12:01:09.126874+0000","last_unstale":"2023-10-18T12:01:18.448801+0000","last_undegraded":"2023-10-18T12:01:18.448801+0000","last_fullsized":"2023-10-18T12:01:18.448801+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444343+0000","last_change":"2023-10-18T12:01:08.118539+0000","last_active":"2023-10-18T12:01:18.444343+0000","last_peered":"2023-10-18T12:01:18.444343+0000","last_clean":"2023-10-18T12:01:18.444343+0000","last_became_active":"2023-10-18T12:01:08.116966+0000","last_became_peered":"2023-10-18T12:01:08.116966+0000","last_unstale":"2023-10-18T12:01:18.444343+0000","last_undegraded":"2023-10-18T12:01:18.444343+0000","last_fullsized":"2023-10-18T12:01:18.444343+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:20.828671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457469+0000","last_change":"2023-10-18T12:01:19.457469+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457469+0000","last_undegraded":"2023-10-18T12:01:19.457469+0000","last_fullsized":"2023-10-18T12:01:19.457469+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444299+0000","last_change":"2023-10-18T12:01:09.119148+0000","last_active":"2023-10-18T12:01:18.444299+0000","last_peered":"2023-10-18T12:01:18.444299+0000","last_clean":"2023-10-18T12:01:18.444299+0000","last_became_active":"2023-10-18T12:01:09.118894+0000","last_became_peered":"2023-10-18T12:01:09.118894+0000","last_unstale":"2023-10-18T12:01:18.444299+0000","last_undegraded":"2023-10-18T12:01:18.444299+0000","last_fullsized":"2023-10-18T12:01:18.444299+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:42.857305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447050+0000","last_change":"2023-10-18T12:01:08.117526+0000","last_active":"2023-10-18T12:01:18.447050+0000","last_peered":"2023-10-18T12:01:18.447050+0000","last_clean":"2023-10-18T12:01:18.447050+0000","last_became_active":"2023-10-18T12:01:08.117234+0000","last_became_peered":"2023-10-18T12:01:08.117234+0000","last_unstale":"2023-10-18T12:01:18.447050+0000","last_undegraded":"2023-10-18T12:01:18.447050+0000","last_fullsized":"2023-10-18T12:01:18.447050+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:43.295217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.460764+0000","last_change":"2023-10-18T12:01:19.460764+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.460764+0000","last_undegraded":"2023-10-18T12:01:19.460764+0000","last_fullsized":"2023-10-18T12:01:19.460764+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439950+0000","last_change":"2023-10-18T12:01:09.125150+0000","last_active":"2023-10-18T12:01:18.439950+0000","last_peered":"2023-10-18T12:01:18.439950+0000","last_clean":"2023-10-18T12:01:18.439950+0000","last_became_active":"2023-10-18T12:01:09.124900+0000","last_became_peered":"2023-10-18T12:01:09.124900+0000","last_unstale":"2023-10-18T12:01:18.439950+0000","last_undegraded":"2023-10-18T12:01:18.439950+0000","last_fullsized":"2023-10-18T12:01:18.439950+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:10.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445789+0000","last_change":"2023-10-18T12:01:08.119612+0000","last_active":"2023-10-18T12:01:18.445789+0000","last_peered":"2023-10-18T12:01:18.445789+0000","last_clean":"2023-10-18T12:01:18.445789+0000","last_became_active":"2023-10-18T12:01:08.119116+0000","last_became_peered":"2023-10-18T12:01:08.119116+0000","last_unstale":"2023-10-18T12:01:18.445789+0000","last_undegraded":"2023-10-18T12:01:18.445789+0000","last_fullsized":"2023-10-18T12:01:18.445789+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:02.536144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451311+0000","last_change":"2023-10-18T12:01:19.451311+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451311+0000","last_undegraded":"2023-10-18T12:01:19.451311+0000","last_fullsized":"2023-10-18T12:01:19.451311+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446463+0000","last_change":"2023-10-18T12:01:09.130360+0000","last_active":"2023-10-18T12:01:18.446463+0000","last_peered":"2023-10-18T12:01:18.446463+0000","last_clean":"2023-10-18T12:01:18.446463+0000","last_became_active":"2023-10-18T12:01:09.130086+0000","last_became_peered":"2023-10-18T12:01:09.130086+0000","last_unstale":"2023-10-18T12:01:18.446463+0000","last_undegraded":"2023-10-18T12:01:18.446463+0000","last_fullsized":"2023-10-18T12:01:18.446463+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:28.085761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443434+0000","last_change":"2023-10-18T12:01:08.119803+0000","last_active":"2023-10-18T12:01:18.443434+0000","last_peered":"2023-10-18T12:01:18.443434+0000","last_clean":"2023-10-18T12:01:18.443434+0000","last_became_active":"2023-10-18T12:01:08.119467+0000","last_became_peered":"2023-10-18T12:01:08.119467+0000","last_unstale":"2023-10-18T12:01:18.443434+0000","last_undegraded":"2023-10-18T12:01:18.443434+0000","last_fullsized":"2023-10-18T12:01:18.443434+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:29.410010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459374+0000","last_change":"2023-10-18T12:01:19.459374+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459374+0000","last_undegraded":"2023-10-18T12:01:19.459374+0000","last_fullsized":"2023-10-18T12:01:19.459374+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443794+0000","last_change":"2023-10-18T12:01:09.122670+0000","last_active":"2023-10-18T12:01:18.443794+0000","last_peered":"2023-10-18T12:01:18.443794+0000","last_clean":"2023-10-18T12:01:18.443794+0000","last_became_active":"2023-10-18T12:01:09.122408+0000","last_became_peered":"2023-10-18T12:01:09.122408+0000","last_unstale":"2023-10-18T12:01:18.443794+0000","last_undegraded":"2023-10-18T12:01:18.443794+0000","last_fullsized":"2023-10-18T12:01:18.443794+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:39.479911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447844+0000","last_change":"2023-10-18T12:01:08.111055+0000","last_active":"2023-10-18T12:01:18.447844+0000","last_peered":"2023-10-18T12:01:18.447844+0000","last_clean":"2023-10-18T12:01:18.447844+0000","last_became_active":"2023-10-18T12:01:08.110728+0000","last_became_peered":"2023-10-18T12:01:08.110728+0000","last_unstale":"2023-10-18T12:01:18.447844+0000","last_undegraded":"2023-10-18T12:01:18.447844+0000","last_fullsized":"2023-10-18T12:01:18.447844+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:15.472803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"227'379","reported_seq":1331,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445541+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:01:18.445541+0000","last_peered":"2023-10-18T12:01:18.445541+0000","last_clean":"2023-10-18T12:01:18.445541+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:01:18.445541+0000","last_undegraded":"2023-10-18T12:01:18.445541+0000","last_fullsized":"2023-10-18T12:01:18.445541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":379,"ondisk_log_size":379,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52808,"kb_used_data":3580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319032,"statfs":{"total":96636764160,"available":96582688768,"internally_reserved":0,"allocated":3665920,"data_stored":2169281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.42399999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.67800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834576000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42500,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329340,"statfs":{"total":96636764160,"available":96593244160,"internally_reserved":0,"allocated":3072000,"data_stored":1578901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.495},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71299999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834576000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64700,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307140,"statfs":{"total":96636764160,"available":96570511360,"internally_reserved":0,"allocated":3063808,"data_stored":1578789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.498},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":1.101,"15min":1.1539999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.71099999999999997,"15min":0.751},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.61699999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72699999999999998,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81399999999999995,"15min":1},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53000000000000003,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.749,"5min":0.82999999999999996,"15min":1.01},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.255,"15min":0.255},"max":{"1min":0.78800000000000003,"5min":0.83699999999999997,"15min":1.101},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48064,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323776,"statfs":{"total":96636764160,"available":96587546624,"internally_reserved":0,"allocated":3067904,"data_stored":1578859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.22500000000000001},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.59599999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":1.123},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.56899999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.66000000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":1.018,"15min":1.018},"last":0.752}]},{"osd":7,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.88900000000000001,"15min":1.05},"last":0.623},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.77900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834576000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48100,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323740,"statfs":{"total":96636764160,"available":96587509760,"internally_reserved":0,"allocated":3100672,"data_stored":1589807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.23999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.378,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.25},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.39300000000000002,"15min":0.40699999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.38400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.39900000000000002,"15min":0.40600000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44900000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.432,"5min":1.899,"15min":1.899},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.42899999999999999,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.434,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.87,"5min":1.034,"15min":1.052},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.629}]},{"osd":5,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":1.522,"15min":1.522},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.91100000000000003,"15min":1.026},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82299999999999995,"5min":1.01,"15min":1.01},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.129,"15min":1.129},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0649999999999999},"last":0.69199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60548,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311292,"statfs":{"total":96636764160,"available":96574763008,"internally_reserved":0,"allocated":3072000,"data_stored":1579441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.52300000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.40500000000000003,"15min":0.41599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.31900000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.61499999999999999,"15min":0.624},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.374,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.63100000000000001,"15min":0.746},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55300000000000005,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86299999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66300000000000003,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.087},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.85199999999999998,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63400000000000001,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.95399999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":1.0609999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60552,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311288,"statfs":{"total":96636764160,"available":96574758912,"internally_reserved":0,"allocated":3076096,"data_stored":1586301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.314}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.32600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.375}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.41099999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575999,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43084,"kb_used_data":3584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328756,"statfs":{"total":96636764160,"available":96592646144,"internally_reserved":0,"allocated":3670016,"data_stored":2171210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52200000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.52300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.26100000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.51100000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.96999999999999997,"15min":0.997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72399999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.95499999999999996,"15min":0.96399999999999997},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64700000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":1.03},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70999999999999996,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.5}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.78000000000000003,"5min":0.997,"15min":0.997},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:01:24.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.1d is not active+clean 2023-10-18T12:01:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454157+0000', 'last_change': '2023-10-18T12:01:19.454157+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454157+0000', 'last_undegraded': '2023-10-18T12:01:19.454157+0000', 'last_fullsized': '2023-10-18T12:01:19.454157+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T12:01:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.1c is not active+clean 2023-10-18T12:01:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453457+0000', 'last_change': '2023-10-18T12:01:19.453457+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453457+0000', 'last_undegraded': '2023-10-18T12:01:19.453457+0000', 'last_fullsized': '2023-10-18T12:01:19.453457+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T12:01:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.1f is not active+clean 2023-10-18T12:01:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.451128+0000', 'last_change': '2023-10-18T12:01:19.451128+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.451128+0000', 'last_undegraded': '2023-10-18T12:01:19.451128+0000', 'last_fullsized': '2023-10-18T12:01:19.451128+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.1e is not active+clean 2023-10-18T12:01:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454232+0000', 'last_change': '2023-10-18T12:01:19.454232+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454232+0000', 'last_undegraded': '2023-10-18T12:01:19.454232+0000', 'last_fullsized': '2023-10-18T12:01:19.454232+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T12:01:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.19 is not active+clean 2023-10-18T12:01:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.450577+0000', 'last_change': '2023-10-18T12:01:19.450577+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.450577+0000', 'last_undegraded': '2023-10-18T12:01:19.450577+0000', 'last_fullsized': '2023-10-18T12:01:19.450577+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.18 is not active+clean 2023-10-18T12:01:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.450431+0000', 'last_change': '2023-10-18T12:01:19.450431+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.450431+0000', 'last_undegraded': '2023-10-18T12:01:19.450431+0000', 'last_fullsized': '2023-10-18T12:01:19.450431+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.1b is not active+clean 2023-10-18T12:01:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.458863+0000', 'last_change': '2023-10-18T12:01:19.458863+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.458863+0000', 'last_undegraded': '2023-10-18T12:01:19.458863+0000', 'last_fullsized': '2023-10-18T12:01:19.458863+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.1a is not active+clean 2023-10-18T12:01:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.451418+0000', 'last_change': '2023-10-18T12:01:19.451418+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.451418+0000', 'last_undegraded': '2023-10-18T12:01:19.451418+0000', 'last_fullsized': '2023-10-18T12:01:19.451418+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T12:01:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.15 is not active+clean 2023-10-18T12:01:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.452320+0000', 'last_change': '2023-10-18T12:01:19.452320+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.452320+0000', 'last_undegraded': '2023-10-18T12:01:19.452320+0000', 'last_fullsized': '2023-10-18T12:01:19.452320+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T12:01:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.14 is not active+clean 2023-10-18T12:01:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.458411+0000', 'last_change': '2023-10-18T12:01:19.458411+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.458411+0000', 'last_undegraded': '2023-10-18T12:01:19.458411+0000', 'last_fullsized': '2023-10-18T12:01:19.458411+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.17 is not active+clean 2023-10-18T12:01:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.456984+0000', 'last_change': '2023-10-18T12:01:19.456984+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.456984+0000', 'last_undegraded': '2023-10-18T12:01:19.456984+0000', 'last_fullsized': '2023-10-18T12:01:19.456984+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.16 is not active+clean 2023-10-18T12:01:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454999+0000', 'last_change': '2023-10-18T12:01:19.454999+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454999+0000', 'last_undegraded': '2023-10-18T12:01:19.454999+0000', 'last_fullsized': '2023-10-18T12:01:19.454999+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.11 is not active+clean 2023-10-18T12:01:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.455867+0000', 'last_change': '2023-10-18T12:01:19.455867+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.455867+0000', 'last_undegraded': '2023-10-18T12:01:19.455867+0000', 'last_fullsized': '2023-10-18T12:01:19.455867+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.10 is not active+clean 2023-10-18T12:01:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453014+0000', 'last_change': '2023-10-18T12:01:19.453014+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453014+0000', 'last_undegraded': '2023-10-18T12:01:19.453014+0000', 'last_fullsized': '2023-10-18T12:01:19.453014+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T12:01:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.13 is not active+clean 2023-10-18T12:01:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.456037+0000', 'last_change': '2023-10-18T12:01:19.456037+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.456037+0000', 'last_undegraded': '2023-10-18T12:01:19.456037+0000', 'last_fullsized': '2023-10-18T12:01:19.456037+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T12:01:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.12 is not active+clean 2023-10-18T12:01:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453907+0000', 'last_change': '2023-10-18T12:01:19.453907+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453907+0000', 'last_undegraded': '2023-10-18T12:01:19.453907+0000', 'last_fullsized': '2023-10-18T12:01:19.453907+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T12:01:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.d is not active+clean 2023-10-18T12:01:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453747+0000', 'last_change': '2023-10-18T12:01:19.453747+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453747+0000', 'last_undegraded': '2023-10-18T12:01:19.453747+0000', 'last_fullsized': '2023-10-18T12:01:19.453747+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.c is not active+clean 2023-10-18T12:01:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.459506+0000', 'last_change': '2023-10-18T12:01:19.459506+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.459506+0000', 'last_undegraded': '2023-10-18T12:01:19.459506+0000', 'last_fullsized': '2023-10-18T12:01:19.459506+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.f is not active+clean 2023-10-18T12:01:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453595+0000', 'last_change': '2023-10-18T12:01:19.453595+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453595+0000', 'last_undegraded': '2023-10-18T12:01:19.453595+0000', 'last_fullsized': '2023-10-18T12:01:19.453595+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.e is not active+clean 2023-10-18T12:01:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.452783+0000', 'last_change': '2023-10-18T12:01:19.452783+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.452783+0000', 'last_undegraded': '2023-10-18T12:01:19.452783+0000', 'last_fullsized': '2023-10-18T12:01:19.452783+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.9 is not active+clean 2023-10-18T12:01:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.452844+0000', 'last_change': '2023-10-18T12:01:19.452844+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.452844+0000', 'last_undegraded': '2023-10-18T12:01:19.452844+0000', 'last_fullsized': '2023-10-18T12:01:19.452844+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.8 is not active+clean 2023-10-18T12:01:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.450045+0000', 'last_change': '2023-10-18T12:01:19.450045+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.450045+0000', 'last_undegraded': '2023-10-18T12:01:19.450045+0000', 'last_fullsized': '2023-10-18T12:01:19.450045+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.b is not active+clean 2023-10-18T12:01:24.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.449803+0000', 'last_change': '2023-10-18T12:01:19.449803+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.449803+0000', 'last_undegraded': '2023-10-18T12:01:19.449803+0000', 'last_fullsized': '2023-10-18T12:01:19.449803+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.a is not active+clean 2023-10-18T12:01:24.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454604+0000', 'last_change': '2023-10-18T12:01:19.454604+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454604+0000', 'last_undegraded': '2023-10-18T12:01:19.454604+0000', 'last_fullsized': '2023-10-18T12:01:19.454604+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.5 is not active+clean 2023-10-18T12:01:24.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.455208+0000', 'last_change': '2023-10-18T12:01:19.455208+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.455208+0000', 'last_undegraded': '2023-10-18T12:01:19.455208+0000', 'last_fullsized': '2023-10-18T12:01:19.455208+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.4 is not active+clean 2023-10-18T12:01:24.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454904+0000', 'last_change': '2023-10-18T12:01:19.454904+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454904+0000', 'last_undegraded': '2023-10-18T12:01:19.454904+0000', 'last_fullsized': '2023-10-18T12:01:19.454904+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.7 is not active+clean 2023-10-18T12:01:24.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454093+0000', 'last_change': '2023-10-18T12:01:19.454093+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454093+0000', 'last_undegraded': '2023-10-18T12:01:19.454093+0000', 'last_fullsized': '2023-10-18T12:01:19.454093+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.6 is not active+clean 2023-10-18T12:01:24.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.451954+0000', 'last_change': '2023-10-18T12:01:19.451954+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.451954+0000', 'last_undegraded': '2023-10-18T12:01:19.451954+0000', 'last_fullsized': '2023-10-18T12:01:19.451954+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T12:01:24.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.1 is not active+clean 2023-10-18T12:01:24.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.449313+0000', 'last_change': '2023-10-18T12:01:19.449313+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.449313+0000', 'last_undegraded': '2023-10-18T12:01:19.449313+0000', 'last_fullsized': '2023-10-18T12:01:19.449313+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.0 is not active+clean 2023-10-18T12:01:24.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.457884+0000', 'last_change': '2023-10-18T12:01:19.457884+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.457884+0000', 'last_undegraded': '2023-10-18T12:01:19.457884+0000', 'last_fullsized': '2023-10-18T12:01:19.457884+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.3 is not active+clean 2023-10-18T12:01:24.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454282+0000', 'last_change': '2023-10-18T12:01:19.454282+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454282+0000', 'last_undegraded': '2023-10-18T12:01:19.454282+0000', 'last_fullsized': '2023-10-18T12:01:19.454282+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T12:01:24.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.2 is not active+clean 2023-10-18T12:01:24.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.450182+0000', 'last_change': '2023-10-18T12:01:19.450182+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.450182+0000', 'last_undegraded': '2023-10-18T12:01:19.450182+0000', 'last_fullsized': '2023-10-18T12:01:19.450182+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.22 is not active+clean 2023-10-18T12:01:24.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.455962+0000', 'last_change': '2023-10-18T12:01:19.455962+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.455962+0000', 'last_undegraded': '2023-10-18T12:01:19.455962+0000', 'last_fullsized': '2023-10-18T12:01:19.455962+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.20 is not active+clean 2023-10-18T12:01:24.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.452634+0000', 'last_change': '2023-10-18T12:01:19.452634+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.452634+0000', 'last_undegraded': '2023-10-18T12:01:19.452634+0000', 'last_fullsized': '2023-10-18T12:01:19.452634+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.21 is not active+clean 2023-10-18T12:01:24.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453087+0000', 'last_change': '2023-10-18T12:01:19.453087+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453087+0000', 'last_undegraded': '2023-10-18T12:01:19.453087+0000', 'last_fullsized': '2023-10-18T12:01:19.453087+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.26 is not active+clean 2023-10-18T12:01:24.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453540+0000', 'last_change': '2023-10-18T12:01:19.453540+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453540+0000', 'last_undegraded': '2023-10-18T12:01:19.453540+0000', 'last_fullsized': '2023-10-18T12:01:19.453540+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.27 is not active+clean 2023-10-18T12:01:24.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.451810+0000', 'last_change': '2023-10-18T12:01:19.451810+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.451810+0000', 'last_undegraded': '2023-10-18T12:01:19.451810+0000', 'last_fullsized': '2023-10-18T12:01:19.451810+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T12:01:24.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.24 is not active+clean 2023-10-18T12:01:24.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.455957+0000', 'last_change': '2023-10-18T12:01:19.455957+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.455957+0000', 'last_undegraded': '2023-10-18T12:01:19.455957+0000', 'last_fullsized': '2023-10-18T12:01:19.455957+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.25 is not active+clean 2023-10-18T12:01:24.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454152+0000', 'last_change': '2023-10-18T12:01:19.454152+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454152+0000', 'last_undegraded': '2023-10-18T12:01:19.454152+0000', 'last_fullsized': '2023-10-18T12:01:19.454152+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.2a is not active+clean 2023-10-18T12:01:24.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.452968+0000', 'last_change': '2023-10-18T12:01:19.452968+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.452968+0000', 'last_undegraded': '2023-10-18T12:01:19.452968+0000', 'last_fullsized': '2023-10-18T12:01:19.452968+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.2b is not active+clean 2023-10-18T12:01:24.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.456437+0000', 'last_change': '2023-10-18T12:01:19.456437+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.456437+0000', 'last_undegraded': '2023-10-18T12:01:19.456437+0000', 'last_fullsized': '2023-10-18T12:01:19.456437+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T12:01:24.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.28 is not active+clean 2023-10-18T12:01:24.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453041+0000', 'last_change': '2023-10-18T12:01:19.453041+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453041+0000', 'last_undegraded': '2023-10-18T12:01:19.453041+0000', 'last_fullsized': '2023-10-18T12:01:19.453041+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.29 is not active+clean 2023-10-18T12:01:24.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453957+0000', 'last_change': '2023-10-18T12:01:19.453957+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453957+0000', 'last_undegraded': '2023-10-18T12:01:19.453957+0000', 'last_fullsized': '2023-10-18T12:01:19.453957+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.2e is not active+clean 2023-10-18T12:01:24.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454638+0000', 'last_change': '2023-10-18T12:01:19.454638+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454638+0000', 'last_undegraded': '2023-10-18T12:01:19.454638+0000', 'last_fullsized': '2023-10-18T12:01:19.454638+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T12:01:24.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.2f is not active+clean 2023-10-18T12:01:24.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.456191+0000', 'last_change': '2023-10-18T12:01:19.456191+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.456191+0000', 'last_undegraded': '2023-10-18T12:01:19.456191+0000', 'last_fullsized': '2023-10-18T12:01:19.456191+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T12:01:24.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.2c is not active+clean 2023-10-18T12:01:24.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.455355+0000', 'last_change': '2023-10-18T12:01:19.455355+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.455355+0000', 'last_undegraded': '2023-10-18T12:01:19.455355+0000', 'last_fullsized': '2023-10-18T12:01:19.455355+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.2d is not active+clean 2023-10-18T12:01:24.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.455037+0000', 'last_change': '2023-10-18T12:01:19.455037+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.455037+0000', 'last_undegraded': '2023-10-18T12:01:19.455037+0000', 'last_fullsized': '2023-10-18T12:01:19.455037+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T12:01:24.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.32 is not active+clean 2023-10-18T12:01:24.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.455648+0000', 'last_change': '2023-10-18T12:01:19.455648+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.455648+0000', 'last_undegraded': '2023-10-18T12:01:19.455648+0000', 'last_fullsized': '2023-10-18T12:01:19.455648+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.33 is not active+clean 2023-10-18T12:01:24.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.459833+0000', 'last_change': '2023-10-18T12:01:19.459833+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.459833+0000', 'last_undegraded': '2023-10-18T12:01:19.459833+0000', 'last_fullsized': '2023-10-18T12:01:19.459833+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.30 is not active+clean 2023-10-18T12:01:24.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.453669+0000', 'last_change': '2023-10-18T12:01:19.453669+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.453669+0000', 'last_undegraded': '2023-10-18T12:01:19.453669+0000', 'last_fullsized': '2023-10-18T12:01:19.453669+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T12:01:24.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.31 is not active+clean 2023-10-18T12:01:24.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.455164+0000', 'last_change': '2023-10-18T12:01:19.455164+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.455164+0000', 'last_undegraded': '2023-10-18T12:01:19.455164+0000', 'last_fullsized': '2023-10-18T12:01:19.455164+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.36 is not active+clean 2023-10-18T12:01:24.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.458648+0000', 'last_change': '2023-10-18T12:01:19.458648+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.458648+0000', 'last_undegraded': '2023-10-18T12:01:19.458648+0000', 'last_fullsized': '2023-10-18T12:01:19.458648+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T12:01:24.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.37 is not active+clean 2023-10-18T12:01:24.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454552+0000', 'last_change': '2023-10-18T12:01:19.454552+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454552+0000', 'last_undegraded': '2023-10-18T12:01:19.454552+0000', 'last_fullsized': '2023-10-18T12:01:19.454552+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.34 is not active+clean 2023-10-18T12:01:24.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.456597+0000', 'last_change': '2023-10-18T12:01:19.456597+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.456597+0000', 'last_undegraded': '2023-10-18T12:01:19.456597+0000', 'last_fullsized': '2023-10-18T12:01:19.456597+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.35 is not active+clean 2023-10-18T12:01:24.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.457598+0000', 'last_change': '2023-10-18T12:01:19.457598+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.457598+0000', 'last_undegraded': '2023-10-18T12:01:19.457598+0000', 'last_fullsized': '2023-10-18T12:01:19.457598+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.3a is not active+clean 2023-10-18T12:01:24.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.456394+0000', 'last_change': '2023-10-18T12:01:19.456394+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.456394+0000', 'last_undegraded': '2023-10-18T12:01:19.456394+0000', 'last_fullsized': '2023-10-18T12:01:19.456394+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T12:01:24.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.3b is not active+clean 2023-10-18T12:01:24.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.454367+0000', 'last_change': '2023-10-18T12:01:19.454367+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.454367+0000', 'last_undegraded': '2023-10-18T12:01:19.454367+0000', 'last_fullsized': '2023-10-18T12:01:19.454367+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T12:01:24.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.38 is not active+clean 2023-10-18T12:01:24.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.460259+0000', 'last_change': '2023-10-18T12:01:19.460259+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.460259+0000', 'last_undegraded': '2023-10-18T12:01:19.460259+0000', 'last_fullsized': '2023-10-18T12:01:19.460259+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.39 is not active+clean 2023-10-18T12:01:24.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.451313+0000', 'last_change': '2023-10-18T12:01:19.451313+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.451313+0000', 'last_undegraded': '2023-10-18T12:01:19.451313+0000', 'last_fullsized': '2023-10-18T12:01:19.451313+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T12:01:24.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.3e is not active+clean 2023-10-18T12:01:24.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.456219+0000', 'last_change': '2023-10-18T12:01:19.456219+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.456219+0000', 'last_undegraded': '2023-10-18T12:01:19.456219+0000', 'last_fullsized': '2023-10-18T12:01:19.456219+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T12:01:24.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.3f is not active+clean 2023-10-18T12:01:24.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.457469+0000', 'last_change': '2023-10-18T12:01:19.457469+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.457469+0000', 'last_undegraded': '2023-10-18T12:01:19.457469+0000', 'last_fullsized': '2023-10-18T12:01:19.457469+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T12:01:24.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.3c is not active+clean 2023-10-18T12:01:24.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.460764+0000', 'last_change': '2023-10-18T12:01:19.460764+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.460764+0000', 'last_undegraded': '2023-10-18T12:01:19.460764+0000', 'last_fullsized': '2023-10-18T12:01:19.460764+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.3d is not active+clean 2023-10-18T12:01:24.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.451311+0000', 'last_change': '2023-10-18T12:01:19.451311+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.451311+0000', 'last_undegraded': '2023-10-18T12:01:19.451311+0000', 'last_fullsized': '2023-10-18T12:01:19.451311+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T12:01:24.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 34.23 is not active+clean 2023-10-18T12:01:24.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '34.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 244, 'state': 'creating+peering', 'last_fresh': '2023-10-18T12:01:19.459374+0000', 'last_change': '2023-10-18T12:01:19.459374+0000', 'last_active': '2023-10-18T12:01:19.435707+0000', 'last_peered': '2023-10-18T12:01:19.435707+0000', 'last_clean': '2023-10-18T12:01:19.435707+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T12:01:19.459374+0000', 'last_undegraded': '2023-10-18T12:01:19.459374+0000', 'last_fullsized': '2023-10-18T12:01:19.459374+0000', 'mapping_epoch': 244, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 244, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'last_clean_scrub_stamp': '2023-10-18T12:01:19.435707+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T12:01:24.138 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T12:01:24.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 -- 172.21.15.125:0/225189172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff813e520 msgr2=0x7fcff806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 --2- 172.21.15.125:0/225189172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff813e520 0x7fcff806bb60 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fcff80cbdc0 tx=0x7fcff402bd10 comp rx=0 tx=0).stop 2023-10-18T12:01:24.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 -- 172.21.15.125:0/225189172 shutdown_connections 2023-10-18T12:01:24.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 --2- 172.21.15.125:0/225189172 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff806c0a0 0x7fcff8070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 --2- 172.21.15.125:0/225189172 >> v2:172.21.15.139:3300/0 conn(0x7fcff8141690 0x7fcff80703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 --2- 172.21.15.125:0/225189172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff813e520 0x7fcff806bb60 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 -- 172.21.15.125:0/225189172 >> 172.21.15.125:0/225189172 conn(0x7fcff8069930 msgr2=0x7fcff8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:24.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 -- 172.21.15.125:0/225189172 shutdown_connections 2023-10-18T12:01:24.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.237+0000 7fcffff4c640 1 -- 172.21.15.125:0/225189172 wait complete. 2023-10-18T12:01:24.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.238+0000 7fcffff4c640 1 Processor -- start 2023-10-18T12:01:24.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.238+0000 7fcffff4c640 1 -- start start 2023-10-18T12:01:24.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffff4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff806c0a0 0x7fcff81efd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:24.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffff4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff813e520 0x7fcff813b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:24.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffff4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff8141690 0x7fcff81f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:24.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffff4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81e2a70 con 0x7fcff8141690 2023-10-18T12:01:24.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffff4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81f0290 con 0x7fcff813e520 2023-10-18T12:01:24.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffff4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80a6930 con 0x7fcff806c0a0 2023-10-18T12:01:24.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff8141690 0x7fcff81f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:24.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff8141690 0x7fcff81f04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33210/0 (socket says 172.21.15.125:33210) 2023-10-18T12:01:24.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 -- 172.21.15.125:0/2283730439 learned_addr learned my addr 172.21.15.125:0/2283730439 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:24.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffd4e7640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff813e520 0x7fcff813b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:24.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 -- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff806c0a0 msgr2=0x7fcff81efd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff806c0a0 0x7fcff81efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 -- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff813e520 msgr2=0x7fcff813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff813e520 0x7fcff813b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 -- 172.21.15.125:0/2283730439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff4009560 con 0x7fcff8141690 2023-10-18T12:01:24.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.239+0000 7fcffe4e9640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff8141690 0x7fcff81f04e0 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7fcff000b970 tx=0x7fcff000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:24.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.240+0000 7fcfee7fc640 1 -- 172.21.15.125:0/2283730439 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000cb40 con 0x7fcff8141690 2023-10-18T12:01:24.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.240+0000 7fcfee7fc640 1 -- 172.21.15.125:0/2283730439 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcff000cdb0 con 0x7fcff8141690 2023-10-18T12:01:24.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.240+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff80a6c10 con 0x7fcff8141690 2023-10-18T12:01:24.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.240+0000 7fcfee7fc640 1 -- 172.21.15.125:0/2283730439 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff00146a0 con 0x7fcff8141690 2023-10-18T12:01:24.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.240+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff80a71d0 con 0x7fcff8141690 2023-10-18T12:01:24.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.241+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff80690e0 con 0x7fcff8141690 2023-10-18T12:01:24.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.241+0000 7fcfee7fc640 1 -- 172.21.15.125:0/2283730439 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcff001d3b0 con 0x7fcff8141690 2023-10-18T12:01:24.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.241+0000 7fcfee7fc640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcfdc06b080 0x7fcfdc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:24.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.241+0000 7fcfee7fc640 1 -- 172.21.15.125:0/2283730439 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fcff008a0d0 con 0x7fcff8141690 2023-10-18T12:01:24.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.242+0000 7fcffdce8640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcfdc06b080 0x7fcfdc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:24.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.242+0000 7fcffdce8640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcfdc06b080 0x7fcfdc06d540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcff4035000 tx=0x7fcff40046c0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:24.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.245+0000 7fcfee7fc640 1 -- 172.21.15.125:0/2283730439 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcff0054940 con 0x7fcff8141690 2023-10-18T12:01:24.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.476+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcff80ad490 con 0x7fcff8141690 2023-10-18T12:01:24.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.477+0000 7fcfee7fc640 1 -- 172.21.15.125:0/2283730439 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3405 (secure 0 0 0) 0x7fcff0019d60 con 0x7fcff8141690 2023-10-18T12:01:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:24.480 INFO:teuthology.orchestra.run.smithi125.stdout:{"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5806,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":246,"num_osds":8,"num_up_osds":8,"osd_up_since":1697624690,"num_in_osds":8,"osd_in_since":1697624681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129},{"state_name":"creating+peering","count":64}],"num_pgs":193,"num_pools":5,"num_objects":24,"data_bytes":594034,"bytes_used":430444544,"bytes_avail":772663668736,"bytes_total":773094113280,"inactive_pgs_ratio":0.33160620927810669,"write_bytes_sec":465,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":103,"id":15,"up":1,"in":1,"max":1,"id":16,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":16,"rank":0,"name":"c","status":"up:active","gid":12768},{"filesystem_id":15,"rank":0,"name":"a","status":"up:active","gid":12771}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2023-10-18T12:00:58.833289+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-18T12:01:24.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcfdc06b080 msgr2=0x7fcfdc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcfdc06b080 0x7fcfdc06d540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcff4035000 tx=0x7fcff40046c0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff8141690 msgr2=0x7fcff81f04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff8141690 0x7fcff81f04e0 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7fcff000b970 tx=0x7fcff000be30 comp rx=0 tx=0).stop 2023-10-18T12:01:24.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 shutdown_connections 2023-10-18T12:01:24.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fcfdc06b080 0x7fcfdc06d540 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff813e520 0x7fcff813b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcff8141690 0x7fcff81f04e0 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 --2- 172.21.15.125:0/2283730439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff806c0a0 0x7fcff81efd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 >> 172.21.15.125:0/2283730439 conn(0x7fcff8069930 msgr2=0x7fcff8158710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:24.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 shutdown_connections 2023-10-18T12:01:24.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.479+0000 7fcffff4c640 1 -- 172.21.15.125:0/2283730439 wait complete. 2023-10-18T12:01:24.492 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T12:01:24.492 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:01:24.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.593+0000 7f8a20207640 1 -- 172.21.15.125:0/2395905937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a040047c0 msgr2=0x7f8a04004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.593+0000 7f8a20207640 1 --2- 172.21.15.125:0/2395905937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a040047c0 0x7f8a04004c40 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f8a1815b6a0 tx=0x7f8a040057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.593+0000 7f8a1e7a4640 1 -- 172.21.15.125:0/2395905937 reap_dead start 2023-10-18T12:01:24.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.593+0000 7f8a20207640 1 -- 172.21.15.125:0/2395905937 shutdown_connections 2023-10-18T12:01:24.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.593+0000 7f8a20207640 1 -- 172.21.15.125:0/2395905937 >> 172.21.15.125:0/2395905937 conn(0x7f8a180d0d60 msgr2=0x7f8a180d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:24.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.593+0000 7f8a20207640 1 -- 172.21.15.125:0/2395905937 shutdown_connections 2023-10-18T12:01:24.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.593+0000 7f8a20207640 1 -- 172.21.15.125:0/2395905937 wait complete. 2023-10-18T12:01:24.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.594+0000 7f8a20207640 1 Processor -- start 2023-10-18T12:01:24.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.594+0000 7f8a20207640 1 -- start start 2023-10-18T12:01:24.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a20207640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a040047c0 0x7f8a1815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:24.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a20207640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a181e4620 0x7f8a180cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:24.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a20207640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a180ccb00 0x7f8a181f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a20207640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180ccfb0 con 0x7f8a180ccb00 2023-10-18T12:01:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a20207640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181f2890 con 0x7f8a181e4620 2023-10-18T12:01:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a20207640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181e6550 con 0x7f8a040047c0 2023-10-18T12:01:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a180ccb00 0x7f8a181f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a180ccb00 0x7f8a181f4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33226/0 (socket says 172.21.15.125:33226) 2023-10-18T12:01:24.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 -- 172.21.15.125:0/2071607345 learned_addr learned my addr 172.21.15.125:0/2071607345 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:24.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1d7a2640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a181e4620 0x7f8a180cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:24.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1dfa3640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a040047c0 0x7f8a1815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:24.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 -- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a040047c0 msgr2=0x7f8a1815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a040047c0 0x7f8a1815dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 -- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a181e4620 msgr2=0x7f8a180cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a181e4620 0x7f8a180cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 -- 172.21.15.125:0/2071607345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1815e130 con 0x7f8a180ccb00 2023-10-18T12:01:24.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a1e7a4640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a180ccb00 0x7f8a181f4de0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005fd0 tx=0x7f8a1000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:24.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a0e7fc640 1 -- 172.21.15.125:0/2071607345 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10004230 con 0x7f8a180ccb00 2023-10-18T12:01:24.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a0e7fc640 1 -- 172.21.15.125:0/2071607345 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a10004480 con 0x7f8a180ccb00 2023-10-18T12:01:24.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a181f1c80 con 0x7f8a180ccb00 2023-10-18T12:01:24.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a0e7fc640 1 -- 172.21.15.125:0/2071607345 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10005280 con 0x7f8a180ccb00 2023-10-18T12:01:24.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.595+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a181f2240 con 0x7f8a180ccb00 2023-10-18T12:01:24.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.597+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a180ae760 con 0x7f8a180ccb00 2023-10-18T12:01:24.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.597+0000 7f8a0e7fc640 1 -- 172.21.15.125:0/2071607345 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8a100053e0 con 0x7f8a180ccb00 2023-10-18T12:01:24.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.597+0000 7f8a0e7fc640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f89fc06b080 0x7f89fc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:24.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.597+0000 7f8a0e7fc640 1 -- 172.21.15.125:0/2071607345 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f8a1008a730 con 0x7f8a180ccb00 2023-10-18T12:01:24.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.598+0000 7f8a1dfa3640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f89fc06b080 0x7f89fc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:24.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.598+0000 7f8a1dfa3640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f89fc06b080 0x7f89fc06d540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005fd0 tx=0x7f8a14009530 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:24.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.602+0000 7f8a0e7fc640 1 -- 172.21.15.125:0/2071607345 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a10054fd0 con 0x7f8a180ccb00 2023-10-18T12:01:24.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.763+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a180d1910 con 0x7f89fc06b080 2023-10-18T12:01:24.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.776+0000 7f8a0e7fc640 1 -- 172.21.15.125:0/2071607345 <== mgr.9236 v2:172.21.15.139:6832/3422518215 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+523685 (secure 0 0 0) 0x7f8a1401d3f0 con 0x7f89fc06b080 2023-10-18T12:01:24.784 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:24.788 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:01:24.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f89fc06b080 msgr2=0x7f89fc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f89fc06b080 0x7f89fc06d540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005fd0 tx=0x7f8a14009530 comp rx=0 tx=0).stop 2023-10-18T12:01:24.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a180ccb00 msgr2=0x7f8a181f4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:24.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a180ccb00 0x7f8a181f4de0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005fd0 tx=0x7f8a1000fab0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 shutdown_connections 2023-10-18T12:01:24.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f89fc06b080 0x7f89fc06d540 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a181e4620 0x7f8a180cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a180ccb00 0x7f8a181f4de0 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 --2- 172.21.15.125:0/2071607345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a040047c0 0x7f8a1815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:24.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 >> 172.21.15.125:0/2071607345 conn(0x7f8a180d0d60 msgr2=0x7f8a180ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:24.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 shutdown_connections 2023-10-18T12:01:24.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:24.780+0000 7f8a20207640 1 -- 172.21.15.125:0/2071607345 wait complete. 2023-10-18T12:01:24.793 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-18T12:01:22.839039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594034,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":682,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":420,"ondisk_log_size":420,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420356,"kb_used_data":25184,"kb_used_omap":12,"kb_used_meta":395059,"kb_avail":754554364,"statfs":{"total":773094113280,"available":772663668736,"internally_reserved":0,"allocated":25788416,"data_stored":13832589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":404540984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1256,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.398513"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454157+0000","last_change":"2023-10-18T12:01:19.454157+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454157+0000","last_undegraded":"2023-10-18T12:01:19.454157+0000","last_fullsized":"2023-10-18T12:01:19.454157+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465723+0000","last_change":"2023-10-18T12:01:19.464169+0000","last_active":"2023-10-18T12:01:19.465723+0000","last_peered":"2023-10-18T12:01:19.465723+0000","last_clean":"2023-10-18T12:01:19.465723+0000","last_became_active":"2023-10-18T12:01:19.463355+0000","last_became_peered":"2023-10-18T12:01:19.463355+0000","last_unstale":"2023-10-18T12:01:19.465723+0000","last_undegraded":"2023-10-18T12:01:19.465723+0000","last_fullsized":"2023-10-18T12:01:19.465723+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:00.513738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444418+0000","last_change":"2023-10-18T12:01:09.126968+0000","last_active":"2023-10-18T12:01:18.444418+0000","last_peered":"2023-10-18T12:01:18.444418+0000","last_clean":"2023-10-18T12:01:18.444418+0000","last_became_active":"2023-10-18T12:01:09.126705+0000","last_became_peered":"2023-10-18T12:01:09.126705+0000","last_unstale":"2023-10-18T12:01:18.444418+0000","last_undegraded":"2023-10-18T12:01:18.444418+0000","last_fullsized":"2023-10-18T12:01:18.444418+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:42:53.088272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453457+0000","last_change":"2023-10-18T12:01:19.453457+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453457+0000","last_undegraded":"2023-10-18T12:01:19.453457+0000","last_fullsized":"2023-10-18T12:01:19.453457+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468820+0000","last_change":"2023-10-18T12:01:19.468191+0000","last_active":"2023-10-18T12:01:19.468820+0000","last_peered":"2023-10-18T12:01:19.468820+0000","last_clean":"2023-10-18T12:01:19.468820+0000","last_became_active":"2023-10-18T12:01:19.467925+0000","last_became_peered":"2023-10-18T12:01:19.467925+0000","last_unstale":"2023-10-18T12:01:19.468820+0000","last_undegraded":"2023-10-18T12:01:19.468820+0000","last_fullsized":"2023-10-18T12:01:19.468820+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:58.124933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439901+0000","last_change":"2023-10-18T12:01:09.125772+0000","last_active":"2023-10-18T12:01:18.439901+0000","last_peered":"2023-10-18T12:01:18.439901+0000","last_clean":"2023-10-18T12:01:18.439901+0000","last_became_active":"2023-10-18T12:01:09.125508+0000","last_became_peered":"2023-10-18T12:01:09.125508+0000","last_unstale":"2023-10-18T12:01:18.439901+0000","last_undegraded":"2023-10-18T12:01:18.439901+0000","last_fullsized":"2023-10-18T12:01:18.439901+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:15.841764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451128+0000","last_change":"2023-10-18T12:01:19.451128+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451128+0000","last_undegraded":"2023-10-18T12:01:19.451128+0000","last_fullsized":"2023-10-18T12:01:19.451128+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.467291+0000","last_change":"2023-10-18T12:01:19.466687+0000","last_active":"2023-10-18T12:01:19.467291+0000","last_peered":"2023-10-18T12:01:19.467291+0000","last_clean":"2023-10-18T12:01:19.467291+0000","last_became_active":"2023-10-18T12:01:19.466395+0000","last_became_peered":"2023-10-18T12:01:19.466395+0000","last_unstale":"2023-10-18T12:01:19.467291+0000","last_undegraded":"2023-10-18T12:01:19.467291+0000","last_fullsized":"2023-10-18T12:01:19.467291+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:02.185827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444714+0000","last_change":"2023-10-18T12:01:09.124764+0000","last_active":"2023-10-18T12:01:18.444714+0000","last_peered":"2023-10-18T12:01:18.444714+0000","last_clean":"2023-10-18T12:01:18.444714+0000","last_became_active":"2023-10-18T12:01:09.124472+0000","last_became_peered":"2023-10-18T12:01:09.124472+0000","last_unstale":"2023-10-18T12:01:18.444714+0000","last_undegraded":"2023-10-18T12:01:18.444714+0000","last_fullsized":"2023-10-18T12:01:18.444714+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:29.891668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454232+0000","last_change":"2023-10-18T12:01:19.454232+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454232+0000","last_undegraded":"2023-10-18T12:01:19.454232+0000","last_fullsized":"2023-10-18T12:01:19.454232+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466648+0000","last_change":"2023-10-18T12:01:19.466181+0000","last_active":"2023-10-18T12:01:19.466648+0000","last_peered":"2023-10-18T12:01:19.466648+0000","last_clean":"2023-10-18T12:01:19.466648+0000","last_became_active":"2023-10-18T12:01:19.465901+0000","last_became_peered":"2023-10-18T12:01:19.465901+0000","last_unstale":"2023-10-18T12:01:19.466648+0000","last_undegraded":"2023-10-18T12:01:19.466648+0000","last_fullsized":"2023-10-18T12:01:19.466648+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:26:57.818590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444737+0000","last_change":"2023-10-18T12:01:09.132202+0000","last_active":"2023-10-18T12:01:18.444737+0000","last_peered":"2023-10-18T12:01:18.444737+0000","last_clean":"2023-10-18T12:01:18.444737+0000","last_became_active":"2023-10-18T12:01:09.131928+0000","last_became_peered":"2023-10-18T12:01:09.131928+0000","last_unstale":"2023-10-18T12:01:18.444737+0000","last_undegraded":"2023-10-18T12:01:18.444737+0000","last_fullsized":"2023-10-18T12:01:18.444737+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:34.826098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450577+0000","last_change":"2023-10-18T12:01:19.450577+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450577+0000","last_undegraded":"2023-10-18T12:01:19.450577+0000","last_fullsized":"2023-10-18T12:01:19.450577+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.461102+0000","last_change":"2023-10-18T12:01:19.459438+0000","last_active":"2023-10-18T12:01:19.461102+0000","last_peered":"2023-10-18T12:01:19.461102+0000","last_clean":"2023-10-18T12:01:19.461102+0000","last_became_active":"2023-10-18T12:01:19.459194+0000","last_became_peered":"2023-10-18T12:01:19.459194+0000","last_unstale":"2023-10-18T12:01:19.461102+0000","last_undegraded":"2023-10-18T12:01:19.461102+0000","last_fullsized":"2023-10-18T12:01:19.461102+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:53.933731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446379+0000","last_change":"2023-10-18T12:01:09.133870+0000","last_active":"2023-10-18T12:01:18.446379+0000","last_peered":"2023-10-18T12:01:18.446379+0000","last_clean":"2023-10-18T12:01:18.446379+0000","last_became_active":"2023-10-18T12:01:09.133478+0000","last_became_peered":"2023-10-18T12:01:09.133478+0000","last_unstale":"2023-10-18T12:01:18.446379+0000","last_undegraded":"2023-10-18T12:01:18.446379+0000","last_fullsized":"2023-10-18T12:01:18.446379+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:11.836101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450431+0000","last_change":"2023-10-18T12:01:19.450431+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450431+0000","last_undegraded":"2023-10-18T12:01:19.450431+0000","last_fullsized":"2023-10-18T12:01:19.450431+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465762+0000","last_change":"2023-10-18T12:01:19.463436+0000","last_active":"2023-10-18T12:01:19.465762+0000","last_peered":"2023-10-18T12:01:19.465762+0000","last_clean":"2023-10-18T12:01:19.465762+0000","last_became_active":"2023-10-18T12:01:19.463190+0000","last_became_peered":"2023-10-18T12:01:19.463190+0000","last_unstale":"2023-10-18T12:01:19.465762+0000","last_undegraded":"2023-10-18T12:01:19.465762+0000","last_fullsized":"2023-10-18T12:01:19.465762+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:55.180369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443988+0000","last_change":"2023-10-18T12:01:09.129113+0000","last_active":"2023-10-18T12:01:18.443988+0000","last_peered":"2023-10-18T12:01:18.443988+0000","last_clean":"2023-10-18T12:01:18.443988+0000","last_became_active":"2023-10-18T12:01:09.128785+0000","last_became_peered":"2023-10-18T12:01:09.128785+0000","last_unstale":"2023-10-18T12:01:18.443988+0000","last_undegraded":"2023-10-18T12:01:18.443988+0000","last_fullsized":"2023-10-18T12:01:18.443988+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:39.875471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458863+0000","last_change":"2023-10-18T12:01:19.458863+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458863+0000","last_undegraded":"2023-10-18T12:01:19.458863+0000","last_fullsized":"2023-10-18T12:01:19.458863+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465879+0000","last_change":"2023-10-18T12:01:19.463023+0000","last_active":"2023-10-18T12:01:19.465879+0000","last_peered":"2023-10-18T12:01:19.465879+0000","last_clean":"2023-10-18T12:01:19.465879+0000","last_became_active":"2023-10-18T12:01:19.462757+0000","last_became_peered":"2023-10-18T12:01:19.462757+0000","last_unstale":"2023-10-18T12:01:19.465879+0000","last_undegraded":"2023-10-18T12:01:19.465879+0000","last_fullsized":"2023-10-18T12:01:19.465879+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:09.123947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445459+0000","last_change":"2023-10-18T12:01:09.125901+0000","last_active":"2023-10-18T12:01:18.445459+0000","last_peered":"2023-10-18T12:01:18.445459+0000","last_clean":"2023-10-18T12:01:18.445459+0000","last_became_active":"2023-10-18T12:01:09.125390+0000","last_became_peered":"2023-10-18T12:01:09.125390+0000","last_unstale":"2023-10-18T12:01:18.445459+0000","last_undegraded":"2023-10-18T12:01:18.445459+0000","last_fullsized":"2023-10-18T12:01:18.445459+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:00.246587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451418+0000","last_change":"2023-10-18T12:01:19.451418+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451418+0000","last_undegraded":"2023-10-18T12:01:19.451418+0000","last_fullsized":"2023-10-18T12:01:19.451418+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465285+0000","last_change":"2023-10-18T12:01:19.463793+0000","last_active":"2023-10-18T12:01:19.465285+0000","last_peered":"2023-10-18T12:01:19.465285+0000","last_clean":"2023-10-18T12:01:19.465285+0000","last_became_active":"2023-10-18T12:01:19.463343+0000","last_became_peered":"2023-10-18T12:01:19.463343+0000","last_unstale":"2023-10-18T12:01:19.465285+0000","last_undegraded":"2023-10-18T12:01:19.465285+0000","last_fullsized":"2023-10-18T12:01:19.465285+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:34.979933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446196+0000","last_change":"2023-10-18T12:01:09.135081+0000","last_active":"2023-10-18T12:01:18.446196+0000","last_peered":"2023-10-18T12:01:18.446196+0000","last_clean":"2023-10-18T12:01:18.446196+0000","last_became_active":"2023-10-18T12:01:09.134801+0000","last_became_peered":"2023-10-18T12:01:09.134801+0000","last_unstale":"2023-10-18T12:01:18.446196+0000","last_undegraded":"2023-10-18T12:01:18.446196+0000","last_fullsized":"2023-10-18T12:01:18.446196+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:03.262316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452320+0000","last_change":"2023-10-18T12:01:19.452320+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452320+0000","last_undegraded":"2023-10-18T12:01:19.452320+0000","last_fullsized":"2023-10-18T12:01:19.452320+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466738+0000","last_change":"2023-10-18T12:01:19.465635+0000","last_active":"2023-10-18T12:01:19.466738+0000","last_peered":"2023-10-18T12:01:19.466738+0000","last_clean":"2023-10-18T12:01:19.466738+0000","last_became_active":"2023-10-18T12:01:19.465363+0000","last_became_peered":"2023-10-18T12:01:19.465363+0000","last_unstale":"2023-10-18T12:01:19.466738+0000","last_undegraded":"2023-10-18T12:01:19.466738+0000","last_fullsized":"2023-10-18T12:01:19.466738+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:56.587842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444490+0000","last_change":"2023-10-18T12:01:09.135615+0000","last_active":"2023-10-18T12:01:18.444490+0000","last_peered":"2023-10-18T12:01:18.444490+0000","last_clean":"2023-10-18T12:01:18.444490+0000","last_became_active":"2023-10-18T12:01:09.135123+0000","last_became_peered":"2023-10-18T12:01:09.135123+0000","last_unstale":"2023-10-18T12:01:18.444490+0000","last_undegraded":"2023-10-18T12:01:18.444490+0000","last_fullsized":"2023-10-18T12:01:18.444490+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:52.311354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458411+0000","last_change":"2023-10-18T12:01:19.458411+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458411+0000","last_undegraded":"2023-10-18T12:01:19.458411+0000","last_fullsized":"2023-10-18T12:01:19.458411+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465526+0000","last_change":"2023-10-18T12:01:19.465001+0000","last_active":"2023-10-18T12:01:19.465526+0000","last_peered":"2023-10-18T12:01:19.465526+0000","last_clean":"2023-10-18T12:01:19.465526+0000","last_became_active":"2023-10-18T12:01:19.464719+0000","last_became_peered":"2023-10-18T12:01:19.464719+0000","last_unstale":"2023-10-18T12:01:19.465526+0000","last_undegraded":"2023-10-18T12:01:19.465526+0000","last_fullsized":"2023-10-18T12:01:19.465526+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:08:12.543232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446023+0000","last_change":"2023-10-18T12:01:09.130673+0000","last_active":"2023-10-18T12:01:18.446023+0000","last_peered":"2023-10-18T12:01:18.446023+0000","last_clean":"2023-10-18T12:01:18.446023+0000","last_became_active":"2023-10-18T12:01:09.130094+0000","last_became_peered":"2023-10-18T12:01:09.130094+0000","last_unstale":"2023-10-18T12:01:18.446023+0000","last_undegraded":"2023-10-18T12:01:18.446023+0000","last_fullsized":"2023-10-18T12:01:18.446023+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:45.652023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456984+0000","last_change":"2023-10-18T12:01:19.456984+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456984+0000","last_undegraded":"2023-10-18T12:01:19.456984+0000","last_fullsized":"2023-10-18T12:01:19.456984+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465010+0000","last_change":"2023-10-18T12:01:19.463067+0000","last_active":"2023-10-18T12:01:19.465010+0000","last_peered":"2023-10-18T12:01:19.465010+0000","last_clean":"2023-10-18T12:01:19.465010+0000","last_became_active":"2023-10-18T12:01:19.462818+0000","last_became_peered":"2023-10-18T12:01:19.462818+0000","last_unstale":"2023-10-18T12:01:19.465010+0000","last_undegraded":"2023-10-18T12:01:19.465010+0000","last_fullsized":"2023-10-18T12:01:19.465010+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:50:23.807977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444802+0000","last_change":"2023-10-18T12:01:09.135828+0000","last_active":"2023-10-18T12:01:18.444802+0000","last_peered":"2023-10-18T12:01:18.444802+0000","last_clean":"2023-10-18T12:01:18.444802+0000","last_became_active":"2023-10-18T12:01:09.135579+0000","last_became_peered":"2023-10-18T12:01:09.135579+0000","last_unstale":"2023-10-18T12:01:18.444802+0000","last_undegraded":"2023-10-18T12:01:18.444802+0000","last_fullsized":"2023-10-18T12:01:18.444802+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:06.517585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454999+0000","last_change":"2023-10-18T12:01:19.454999+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454999+0000","last_undegraded":"2023-10-18T12:01:19.454999+0000","last_fullsized":"2023-10-18T12:01:19.454999+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.464639+0000","last_change":"2023-10-18T12:01:19.462716+0000","last_active":"2023-10-18T12:01:19.464639+0000","last_peered":"2023-10-18T12:01:19.464639+0000","last_clean":"2023-10-18T12:01:19.464639+0000","last_became_active":"2023-10-18T12:01:19.462474+0000","last_became_peered":"2023-10-18T12:01:19.462474+0000","last_unstale":"2023-10-18T12:01:19.464639+0000","last_undegraded":"2023-10-18T12:01:19.464639+0000","last_fullsized":"2023-10-18T12:01:19.464639+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:04.329561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443892+0000","last_change":"2023-10-18T12:01:09.132939+0000","last_active":"2023-10-18T12:01:18.443892+0000","last_peered":"2023-10-18T12:01:18.443892+0000","last_clean":"2023-10-18T12:01:18.443892+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:18.443892+0000","last_undegraded":"2023-10-18T12:01:18.443892+0000","last_fullsized":"2023-10-18T12:01:18.443892+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:22.550156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455867+0000","last_change":"2023-10-18T12:01:19.455867+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455867+0000","last_undegraded":"2023-10-18T12:01:19.455867+0000","last_fullsized":"2023-10-18T12:01:19.455867+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.459783+0000","last_change":"2023-10-18T12:01:19.458472+0000","last_active":"2023-10-18T12:01:19.459783+0000","last_peered":"2023-10-18T12:01:19.459783+0000","last_clean":"2023-10-18T12:01:19.459783+0000","last_became_active":"2023-10-18T12:01:19.458209+0000","last_became_peered":"2023-10-18T12:01:19.458209+0000","last_unstale":"2023-10-18T12:01:19.459783+0000","last_undegraded":"2023-10-18T12:01:19.459783+0000","last_fullsized":"2023-10-18T12:01:19.459783+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:04.778253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444252+0000","last_change":"2023-10-18T12:01:09.129764+0000","last_active":"2023-10-18T12:01:18.444252+0000","last_peered":"2023-10-18T12:01:18.444252+0000","last_clean":"2023-10-18T12:01:18.444252+0000","last_became_active":"2023-10-18T12:01:09.129483+0000","last_became_peered":"2023-10-18T12:01:09.129483+0000","last_unstale":"2023-10-18T12:01:18.444252+0000","last_undegraded":"2023-10-18T12:01:18.444252+0000","last_fullsized":"2023-10-18T12:01:18.444252+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:03.037837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453014+0000","last_change":"2023-10-18T12:01:19.453014+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453014+0000","last_undegraded":"2023-10-18T12:01:19.453014+0000","last_fullsized":"2023-10-18T12:01:19.453014+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468149+0000","last_change":"2023-10-18T12:01:19.466751+0000","last_active":"2023-10-18T12:01:19.468149+0000","last_peered":"2023-10-18T12:01:19.468149+0000","last_clean":"2023-10-18T12:01:19.468149+0000","last_became_active":"2023-10-18T12:01:19.466171+0000","last_became_peered":"2023-10-18T12:01:19.466171+0000","last_unstale":"2023-10-18T12:01:19.468149+0000","last_undegraded":"2023-10-18T12:01:19.468149+0000","last_fullsized":"2023-10-18T12:01:19.468149+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:01.102496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448554+0000","last_change":"2023-10-18T12:01:09.133024+0000","last_active":"2023-10-18T12:01:18.448554+0000","last_peered":"2023-10-18T12:01:18.448554+0000","last_clean":"2023-10-18T12:01:18.448554+0000","last_became_active":"2023-10-18T12:01:09.132742+0000","last_became_peered":"2023-10-18T12:01:09.132742+0000","last_unstale":"2023-10-18T12:01:18.448554+0000","last_undegraded":"2023-10-18T12:01:18.448554+0000","last_fullsized":"2023-10-18T12:01:18.448554+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456037+0000","last_change":"2023-10-18T12:01:19.456037+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456037+0000","last_undegraded":"2023-10-18T12:01:19.456037+0000","last_fullsized":"2023-10-18T12:01:19.456037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.465243+0000","last_change":"2023-10-18T12:01:19.464582+0000","last_active":"2023-10-18T12:01:19.465243+0000","last_peered":"2023-10-18T12:01:19.465243+0000","last_clean":"2023-10-18T12:01:19.465243+0000","last_became_active":"2023-10-18T12:01:19.464325+0000","last_became_peered":"2023-10-18T12:01:19.464325+0000","last_unstale":"2023-10-18T12:01:19.465243+0000","last_undegraded":"2023-10-18T12:01:19.465243+0000","last_fullsized":"2023-10-18T12:01:19.465243+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:32:16.943070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445969+0000","last_change":"2023-10-18T12:01:09.131147+0000","last_active":"2023-10-18T12:01:18.445969+0000","last_peered":"2023-10-18T12:01:18.445969+0000","last_clean":"2023-10-18T12:01:18.445969+0000","last_became_active":"2023-10-18T12:01:09.130682+0000","last_became_peered":"2023-10-18T12:01:09.130682+0000","last_unstale":"2023-10-18T12:01:18.445969+0000","last_undegraded":"2023-10-18T12:01:18.445969+0000","last_fullsized":"2023-10-18T12:01:18.445969+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453907+0000","last_change":"2023-10-18T12:01:19.453907+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453907+0000","last_undegraded":"2023-10-18T12:01:19.453907+0000","last_fullsized":"2023-10-18T12:01:19.453907+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463821+0000","last_change":"2023-10-18T12:01:19.459426+0000","last_active":"2023-10-18T12:01:19.463821+0000","last_peered":"2023-10-18T12:01:19.463821+0000","last_clean":"2023-10-18T12:01:19.463821+0000","last_became_active":"2023-10-18T12:01:19.459168+0000","last_became_peered":"2023-10-18T12:01:19.459168+0000","last_unstale":"2023-10-18T12:01:19.463821+0000","last_undegraded":"2023-10-18T12:01:19.463821+0000","last_fullsized":"2023-10-18T12:01:19.463821+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:00.882760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445455+0000","last_change":"2023-10-18T12:01:09.122098+0000","last_active":"2023-10-18T12:01:18.445455+0000","last_peered":"2023-10-18T12:01:18.445455+0000","last_clean":"2023-10-18T12:01:18.445455+0000","last_became_active":"2023-10-18T12:01:09.121845+0000","last_became_peered":"2023-10-18T12:01:09.121845+0000","last_unstale":"2023-10-18T12:01:18.445455+0000","last_undegraded":"2023-10-18T12:01:18.445455+0000","last_fullsized":"2023-10-18T12:01:18.445455+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:19.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453747+0000","last_change":"2023-10-18T12:01:19.453747+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453747+0000","last_undegraded":"2023-10-18T12:01:19.453747+0000","last_fullsized":"2023-10-18T12:01:19.453747+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468474+0000","last_change":"2023-10-18T12:01:19.466123+0000","last_active":"2023-10-18T12:01:19.468474+0000","last_peered":"2023-10-18T12:01:19.468474+0000","last_clean":"2023-10-18T12:01:19.468474+0000","last_became_active":"2023-10-18T12:01:19.465856+0000","last_became_peered":"2023-10-18T12:01:19.465856+0000","last_unstale":"2023-10-18T12:01:19.468474+0000","last_undegraded":"2023-10-18T12:01:19.468474+0000","last_fullsized":"2023-10-18T12:01:19.468474+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:53.379907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447103+0000","last_change":"2023-10-18T12:01:09.133999+0000","last_active":"2023-10-18T12:01:18.447103+0000","last_peered":"2023-10-18T12:01:18.447103+0000","last_clean":"2023-10-18T12:01:18.447103+0000","last_became_active":"2023-10-18T12:01:09.133725+0000","last_became_peered":"2023-10-18T12:01:09.133725+0000","last_unstale":"2023-10-18T12:01:18.447103+0000","last_undegraded":"2023-10-18T12:01:18.447103+0000","last_fullsized":"2023-10-18T12:01:18.447103+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:35.006911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459506+0000","last_change":"2023-10-18T12:01:19.459506+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459506+0000","last_undegraded":"2023-10-18T12:01:19.459506+0000","last_fullsized":"2023-10-18T12:01:19.459506+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468784+0000","last_change":"2023-10-18T12:01:19.468308+0000","last_active":"2023-10-18T12:01:19.468784+0000","last_peered":"2023-10-18T12:01:19.468784+0000","last_clean":"2023-10-18T12:01:19.468784+0000","last_became_active":"2023-10-18T12:01:19.467998+0000","last_became_peered":"2023-10-18T12:01:19.467998+0000","last_unstale":"2023-10-18T12:01:19.468784+0000","last_undegraded":"2023-10-18T12:01:19.468784+0000","last_fullsized":"2023-10-18T12:01:19.468784+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:17.064462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446111+0000","last_change":"2023-10-18T12:01:09.124908+0000","last_active":"2023-10-18T12:01:18.446111+0000","last_peered":"2023-10-18T12:01:18.446111+0000","last_clean":"2023-10-18T12:01:18.446111+0000","last_became_active":"2023-10-18T12:01:09.124565+0000","last_became_peered":"2023-10-18T12:01:09.124565+0000","last_unstale":"2023-10-18T12:01:18.446111+0000","last_undegraded":"2023-10-18T12:01:18.446111+0000","last_fullsized":"2023-10-18T12:01:18.446111+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:22.044677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453595+0000","last_change":"2023-10-18T12:01:19.453595+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453595+0000","last_undegraded":"2023-10-18T12:01:19.453595+0000","last_fullsized":"2023-10-18T12:01:19.453595+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468762+0000","last_change":"2023-10-18T12:01:19.467971+0000","last_active":"2023-10-18T12:01:19.468762+0000","last_peered":"2023-10-18T12:01:19.468762+0000","last_clean":"2023-10-18T12:01:19.468762+0000","last_became_active":"2023-10-18T12:01:19.467699+0000","last_became_peered":"2023-10-18T12:01:19.467699+0000","last_unstale":"2023-10-18T12:01:19.468762+0000","last_undegraded":"2023-10-18T12:01:19.468762+0000","last_fullsized":"2023-10-18T12:01:19.468762+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:12:01.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447225+0000","last_change":"2023-10-18T12:01:09.135302+0000","last_active":"2023-10-18T12:01:18.447225+0000","last_peered":"2023-10-18T12:01:18.447225+0000","last_clean":"2023-10-18T12:01:18.447225+0000","last_became_active":"2023-10-18T12:01:09.134978+0000","last_became_peered":"2023-10-18T12:01:09.134978+0000","last_unstale":"2023-10-18T12:01:18.447225+0000","last_undegraded":"2023-10-18T12:01:18.447225+0000","last_fullsized":"2023-10-18T12:01:18.447225+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452783+0000","last_change":"2023-10-18T12:01:19.452783+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452783+0000","last_undegraded":"2023-10-18T12:01:19.452783+0000","last_fullsized":"2023-10-18T12:01:19.452783+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463777+0000","last_change":"2023-10-18T12:01:19.461558+0000","last_active":"2023-10-18T12:01:19.463777+0000","last_peered":"2023-10-18T12:01:19.463777+0000","last_clean":"2023-10-18T12:01:19.463777+0000","last_became_active":"2023-10-18T12:01:19.461307+0000","last_became_peered":"2023-10-18T12:01:19.461307+0000","last_unstale":"2023-10-18T12:01:19.463777+0000","last_undegraded":"2023-10-18T12:01:19.463777+0000","last_fullsized":"2023-10-18T12:01:19.463777+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:18:26.034701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440010+0000","last_change":"2023-10-18T12:01:09.136717+0000","last_active":"2023-10-18T12:01:18.440010+0000","last_peered":"2023-10-18T12:01:18.440010+0000","last_clean":"2023-10-18T12:01:18.440010+0000","last_became_active":"2023-10-18T12:01:09.136461+0000","last_became_peered":"2023-10-18T12:01:09.136461+0000","last_unstale":"2023-10-18T12:01:18.440010+0000","last_undegraded":"2023-10-18T12:01:18.440010+0000","last_fullsized":"2023-10-18T12:01:18.440010+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:02.336311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452844+0000","last_change":"2023-10-18T12:01:19.452844+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452844+0000","last_undegraded":"2023-10-18T12:01:19.452844+0000","last_fullsized":"2023-10-18T12:01:19.452844+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463533+0000","last_change":"2023-10-18T12:01:19.462240+0000","last_active":"2023-10-18T12:01:19.463533+0000","last_peered":"2023-10-18T12:01:19.463533+0000","last_clean":"2023-10-18T12:01:19.463533+0000","last_became_active":"2023-10-18T12:01:19.461957+0000","last_became_peered":"2023-10-18T12:01:19.461957+0000","last_unstale":"2023-10-18T12:01:19.463533+0000","last_undegraded":"2023-10-18T12:01:19.463533+0000","last_fullsized":"2023-10-18T12:01:19.463533+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:20.506577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445002+0000","last_change":"2023-10-18T12:01:09.130016+0000","last_active":"2023-10-18T12:01:18.445002+0000","last_peered":"2023-10-18T12:01:18.445002+0000","last_clean":"2023-10-18T12:01:18.445002+0000","last_became_active":"2023-10-18T12:01:09.129718+0000","last_became_peered":"2023-10-18T12:01:09.129718+0000","last_unstale":"2023-10-18T12:01:18.445002+0000","last_undegraded":"2023-10-18T12:01:18.445002+0000","last_fullsized":"2023-10-18T12:01:18.445002+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:36.455465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450045+0000","last_change":"2023-10-18T12:01:19.450045+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450045+0000","last_undegraded":"2023-10-18T12:01:19.450045+0000","last_fullsized":"2023-10-18T12:01:19.450045+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468581+0000","last_change":"2023-10-18T12:01:19.467158+0000","last_active":"2023-10-18T12:01:19.468581+0000","last_peered":"2023-10-18T12:01:19.468581+0000","last_clean":"2023-10-18T12:01:19.468581+0000","last_became_active":"2023-10-18T12:01:19.466854+0000","last_became_peered":"2023-10-18T12:01:19.466854+0000","last_unstale":"2023-10-18T12:01:19.468581+0000","last_undegraded":"2023-10-18T12:01:19.468581+0000","last_fullsized":"2023-10-18T12:01:19.468581+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:21.428576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443731+0000","last_change":"2023-10-18T12:01:09.129706+0000","last_active":"2023-10-18T12:01:18.443731+0000","last_peered":"2023-10-18T12:01:18.443731+0000","last_clean":"2023-10-18T12:01:18.443731+0000","last_became_active":"2023-10-18T12:01:09.129413+0000","last_became_peered":"2023-10-18T12:01:09.129413+0000","last_unstale":"2023-10-18T12:01:18.443731+0000","last_undegraded":"2023-10-18T12:01:18.443731+0000","last_fullsized":"2023-10-18T12:01:18.443731+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:55.309809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.449803+0000","last_change":"2023-10-18T12:01:19.449803+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.449803+0000","last_undegraded":"2023-10-18T12:01:19.449803+0000","last_fullsized":"2023-10-18T12:01:19.449803+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468823+0000","last_change":"2023-10-18T12:01:19.467623+0000","last_active":"2023-10-18T12:01:19.468823+0000","last_peered":"2023-10-18T12:01:19.468823+0000","last_clean":"2023-10-18T12:01:19.468823+0000","last_became_active":"2023-10-18T12:01:19.467358+0000","last_became_peered":"2023-10-18T12:01:19.467358+0000","last_unstale":"2023-10-18T12:01:19.468823+0000","last_undegraded":"2023-10-18T12:01:19.468823+0000","last_fullsized":"2023-10-18T12:01:19.468823+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:58.681098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446063+0000","last_change":"2023-10-18T12:01:09.133291+0000","last_active":"2023-10-18T12:01:18.446063+0000","last_peered":"2023-10-18T12:01:18.446063+0000","last_clean":"2023-10-18T12:01:18.446063+0000","last_became_active":"2023-10-18T12:01:09.132968+0000","last_became_peered":"2023-10-18T12:01:09.132968+0000","last_unstale":"2023-10-18T12:01:18.446063+0000","last_undegraded":"2023-10-18T12:01:18.446063+0000","last_fullsized":"2023-10-18T12:01:18.446063+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454604+0000","last_change":"2023-10-18T12:01:19.454604+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454604+0000","last_undegraded":"2023-10-18T12:01:19.454604+0000","last_fullsized":"2023-10-18T12:01:19.454604+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.460591+0000","last_change":"2023-10-18T12:01:19.458380+0000","last_active":"2023-10-18T12:01:19.460591+0000","last_peered":"2023-10-18T12:01:19.460591+0000","last_clean":"2023-10-18T12:01:19.460591+0000","last_became_active":"2023-10-18T12:01:19.458111+0000","last_became_peered":"2023-10-18T12:01:19.458111+0000","last_unstale":"2023-10-18T12:01:19.460591+0000","last_undegraded":"2023-10-18T12:01:19.460591+0000","last_fullsized":"2023-10-18T12:01:19.460591+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:27.878015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443343+0000","last_change":"2023-10-18T12:01:09.128954+0000","last_active":"2023-10-18T12:01:18.443343+0000","last_peered":"2023-10-18T12:01:18.443343+0000","last_clean":"2023-10-18T12:01:18.443343+0000","last_became_active":"2023-10-18T12:01:09.128655+0000","last_became_peered":"2023-10-18T12:01:09.128655+0000","last_unstale":"2023-10-18T12:01:18.443343+0000","last_undegraded":"2023-10-18T12:01:18.443343+0000","last_fullsized":"2023-10-18T12:01:18.443343+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:51.625573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455208+0000","last_change":"2023-10-18T12:01:19.455208+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455208+0000","last_undegraded":"2023-10-18T12:01:19.455208+0000","last_fullsized":"2023-10-18T12:01:19.455208+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468453+0000","last_change":"2023-10-18T12:01:19.466027+0000","last_active":"2023-10-18T12:01:19.468453+0000","last_peered":"2023-10-18T12:01:19.468453+0000","last_clean":"2023-10-18T12:01:19.468453+0000","last_became_active":"2023-10-18T12:01:19.465745+0000","last_became_peered":"2023-10-18T12:01:19.465745+0000","last_unstale":"2023-10-18T12:01:19.468453+0000","last_undegraded":"2023-10-18T12:01:19.468453+0000","last_fullsized":"2023-10-18T12:01:19.468453+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:15.002306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440030+0000","last_change":"2023-10-18T12:01:09.123355+0000","last_active":"2023-10-18T12:01:18.440030+0000","last_peered":"2023-10-18T12:01:18.440030+0000","last_clean":"2023-10-18T12:01:18.440030+0000","last_became_active":"2023-10-18T12:01:09.123096+0000","last_became_peered":"2023-10-18T12:01:09.123096+0000","last_unstale":"2023-10-18T12:01:18.440030+0000","last_undegraded":"2023-10-18T12:01:18.440030+0000","last_fullsized":"2023-10-18T12:01:18.440030+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:58.130314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454904+0000","last_change":"2023-10-18T12:01:19.454904+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454904+0000","last_undegraded":"2023-10-18T12:01:19.454904+0000","last_fullsized":"2023-10-18T12:01:19.454904+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.468820+0000","last_change":"2023-10-18T12:01:19.467893+0000","last_active":"2023-10-18T12:01:19.468820+0000","last_peered":"2023-10-18T12:01:19.468820+0000","last_clean":"2023-10-18T12:01:19.468820+0000","last_became_active":"2023-10-18T12:01:19.467430+0000","last_became_peered":"2023-10-18T12:01:19.467430+0000","last_unstale":"2023-10-18T12:01:19.468820+0000","last_undegraded":"2023-10-18T12:01:19.468820+0000","last_fullsized":"2023-10-18T12:01:19.468820+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:12.990393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447739+0000","last_change":"2023-10-18T12:01:09.124947+0000","last_active":"2023-10-18T12:01:18.447739+0000","last_peered":"2023-10-18T12:01:18.447739+0000","last_clean":"2023-10-18T12:01:18.447739+0000","last_became_active":"2023-10-18T12:01:09.124666+0000","last_became_peered":"2023-10-18T12:01:09.124666+0000","last_unstale":"2023-10-18T12:01:18.447739+0000","last_undegraded":"2023-10-18T12:01:18.447739+0000","last_fullsized":"2023-10-18T12:01:18.447739+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:01.214167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454093+0000","last_change":"2023-10-18T12:01:19.454093+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454093+0000","last_undegraded":"2023-10-18T12:01:19.454093+0000","last_fullsized":"2023-10-18T12:01:19.454093+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463415+0000","last_change":"2023-10-18T12:01:19.462049+0000","last_active":"2023-10-18T12:01:19.463415+0000","last_peered":"2023-10-18T12:01:19.463415+0000","last_clean":"2023-10-18T12:01:19.463415+0000","last_became_active":"2023-10-18T12:01:19.460286+0000","last_became_peered":"2023-10-18T12:01:19.460286+0000","last_unstale":"2023-10-18T12:01:19.463415+0000","last_undegraded":"2023-10-18T12:01:19.463415+0000","last_fullsized":"2023-10-18T12:01:19.463415+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:59.023770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447065+0000","last_change":"2023-10-18T12:01:09.131674+0000","last_active":"2023-10-18T12:01:18.447065+0000","last_peered":"2023-10-18T12:01:18.447065+0000","last_clean":"2023-10-18T12:01:18.447065+0000","last_became_active":"2023-10-18T12:01:09.131410+0000","last_became_peered":"2023-10-18T12:01:09.131410+0000","last_unstale":"2023-10-18T12:01:18.447065+0000","last_undegraded":"2023-10-18T12:01:18.447065+0000","last_fullsized":"2023-10-18T12:01:18.447065+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:22.606198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451954+0000","last_change":"2023-10-18T12:01:19.451954+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451954+0000","last_undegraded":"2023-10-18T12:01:19.451954+0000","last_fullsized":"2023-10-18T12:01:19.451954+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.463757+0000","last_change":"2023-10-18T12:01:19.460871+0000","last_active":"2023-10-18T12:01:19.463757+0000","last_peered":"2023-10-18T12:01:19.463757+0000","last_clean":"2023-10-18T12:01:19.463757+0000","last_became_active":"2023-10-18T12:01:19.460570+0000","last_became_peered":"2023-10-18T12:01:19.460570+0000","last_unstale":"2023-10-18T12:01:19.463757+0000","last_undegraded":"2023-10-18T12:01:19.463757+0000","last_fullsized":"2023-10-18T12:01:19.463757+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:16:21.229838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443961+0000","last_change":"2023-10-18T12:01:09.134865+0000","last_active":"2023-10-18T12:01:18.443961+0000","last_peered":"2023-10-18T12:01:18.443961+0000","last_clean":"2023-10-18T12:01:18.443961+0000","last_became_active":"2023-10-18T12:01:09.134583+0000","last_became_peered":"2023-10-18T12:01:09.134583+0000","last_unstale":"2023-10-18T12:01:18.443961+0000","last_undegraded":"2023-10-18T12:01:18.443961+0000","last_fullsized":"2023-10-18T12:01:18.443961+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:42.480656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.449313+0000","last_change":"2023-10-18T12:01:19.449313+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.449313+0000","last_undegraded":"2023-10-18T12:01:19.449313+0000","last_fullsized":"2023-10-18T12:01:19.449313+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.464534+0000","last_change":"2023-10-18T12:01:19.461910+0000","last_active":"2023-10-18T12:01:19.464534+0000","last_peered":"2023-10-18T12:01:19.464534+0000","last_clean":"2023-10-18T12:01:19.464534+0000","last_became_active":"2023-10-18T12:01:19.461653+0000","last_became_peered":"2023-10-18T12:01:19.461653+0000","last_unstale":"2023-10-18T12:01:19.464534+0000","last_undegraded":"2023-10-18T12:01:19.464534+0000","last_fullsized":"2023-10-18T12:01:19.464534+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:16:33.411957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448538+0000","last_change":"2023-10-18T12:01:09.132312+0000","last_active":"2023-10-18T12:01:18.448538+0000","last_peered":"2023-10-18T12:01:18.448538+0000","last_clean":"2023-10-18T12:01:18.448538+0000","last_became_active":"2023-10-18T12:01:09.131781+0000","last_became_peered":"2023-10-18T12:01:09.131781+0000","last_unstale":"2023-10-18T12:01:18.448538+0000","last_undegraded":"2023-10-18T12:01:18.448538+0000","last_fullsized":"2023-10-18T12:01:18.448538+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457884+0000","last_change":"2023-10-18T12:01:19.457884+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457884+0000","last_undegraded":"2023-10-18T12:01:19.457884+0000","last_fullsized":"2023-10-18T12:01:19.457884+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.466498+0000","last_change":"2023-10-18T12:01:19.465117+0000","last_active":"2023-10-18T12:01:19.466498+0000","last_peered":"2023-10-18T12:01:19.466498+0000","last_clean":"2023-10-18T12:01:19.466498+0000","last_became_active":"2023-10-18T12:01:19.464845+0000","last_became_peered":"2023-10-18T12:01:19.464845+0000","last_unstale":"2023-10-18T12:01:19.466498+0000","last_undegraded":"2023-10-18T12:01:19.466498+0000","last_fullsized":"2023-10-18T12:01:19.466498+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:02:22.112086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448553+0000","last_change":"2023-10-18T12:01:09.133664+0000","last_active":"2023-10-18T12:01:18.448553+0000","last_peered":"2023-10-18T12:01:18.448553+0000","last_clean":"2023-10-18T12:01:18.448553+0000","last_became_active":"2023-10-18T12:01:09.133406+0000","last_became_peered":"2023-10-18T12:01:09.133406+0000","last_unstale":"2023-10-18T12:01:18.448553+0000","last_undegraded":"2023-10-18T12:01:18.448553+0000","last_fullsized":"2023-10-18T12:01:18.448553+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454282+0000","last_change":"2023-10-18T12:01:19.454282+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454282+0000","last_undegraded":"2023-10-18T12:01:19.454282+0000","last_fullsized":"2023-10-18T12:01:19.454282+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.461380+0000","last_change":"2023-10-18T12:01:19.458515+0000","last_active":"2023-10-18T12:01:19.461380+0000","last_peered":"2023-10-18T12:01:19.461380+0000","last_clean":"2023-10-18T12:01:19.461380+0000","last_became_active":"2023-10-18T12:01:19.457749+0000","last_became_peered":"2023-10-18T12:01:19.457749+0000","last_unstale":"2023-10-18T12:01:19.461380+0000","last_undegraded":"2023-10-18T12:01:19.461380+0000","last_fullsized":"2023-10-18T12:01:19.461380+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:24.677682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439756+0000","last_change":"2023-10-18T12:01:09.130142+0000","last_active":"2023-10-18T12:01:18.439756+0000","last_peered":"2023-10-18T12:01:18.439756+0000","last_clean":"2023-10-18T12:01:18.439756+0000","last_became_active":"2023-10-18T12:01:09.129884+0000","last_became_peered":"2023-10-18T12:01:09.129884+0000","last_unstale":"2023-10-18T12:01:18.439756+0000","last_undegraded":"2023-10-18T12:01:18.439756+0000","last_fullsized":"2023-10-18T12:01:18.439756+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:36.037273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.450182+0000","last_change":"2023-10-18T12:01:19.450182+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.450182+0000","last_undegraded":"2023-10-18T12:01:19.450182+0000","last_fullsized":"2023-10-18T12:01:19.450182+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":244,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.459660+0000","last_change":"2023-10-18T12:01:19.458409+0000","last_active":"2023-10-18T12:01:19.459660+0000","last_peered":"2023-10-18T12:01:19.459660+0000","last_clean":"2023-10-18T12:01:19.459660+0000","last_became_active":"2023-10-18T12:01:19.458151+0000","last_became_peered":"2023-10-18T12:01:19.458151+0000","last_unstale":"2023-10-18T12:01:19.459660+0000","last_undegraded":"2023-10-18T12:01:19.459660+0000","last_fullsized":"2023-10-18T12:01:19.459660+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:30.732460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443598+0000","last_change":"2023-10-18T12:01:09.136066+0000","last_active":"2023-10-18T12:01:18.443598+0000","last_peered":"2023-10-18T12:01:18.443598+0000","last_clean":"2023-10-18T12:01:18.443598+0000","last_became_active":"2023-10-18T12:01:09.135802+0000","last_became_peered":"2023-10-18T12:01:09.135802+0000","last_unstale":"2023-10-18T12:01:18.443598+0000","last_undegraded":"2023-10-18T12:01:18.443598+0000","last_fullsized":"2023-10-18T12:01:18.443598+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:26.550223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455962+0000","last_change":"2023-10-18T12:01:19.455962+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455962+0000","last_undegraded":"2023-10-18T12:01:19.455962+0000","last_fullsized":"2023-10-18T12:01:19.455962+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439658+0000","last_change":"2023-10-18T12:01:09.120812+0000","last_active":"2023-10-18T12:01:18.439658+0000","last_peered":"2023-10-18T12:01:18.439658+0000","last_clean":"2023-10-18T12:01:18.439658+0000","last_became_active":"2023-10-18T12:01:09.120561+0000","last_became_peered":"2023-10-18T12:01:09.120561+0000","last_unstale":"2023-10-18T12:01:18.439658+0000","last_undegraded":"2023-10-18T12:01:18.439658+0000","last_fullsized":"2023-10-18T12:01:18.439658+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:02.277242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"241'4","reported_seq":18,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444451+0000","last_change":"2023-10-18T12:01:08.112445+0000","last_active":"2023-10-18T12:01:18.444451+0000","last_peered":"2023-10-18T12:01:18.444451+0000","last_clean":"2023-10-18T12:01:18.444451+0000","last_became_active":"2023-10-18T12:01:08.112017+0000","last_became_peered":"2023-10-18T12:01:08.112017+0000","last_unstale":"2023-10-18T12:01:18.444451+0000","last_undegraded":"2023-10-18T12:01:18.444451+0000","last_fullsized":"2023-10-18T12:01:18.444451+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:11.607481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452634+0000","last_change":"2023-10-18T12:01:19.452634+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452634+0000","last_undegraded":"2023-10-18T12:01:19.452634+0000","last_fullsized":"2023-10-18T12:01:19.452634+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446081+0000","last_change":"2023-10-18T12:01:09.134916+0000","last_active":"2023-10-18T12:01:18.446081+0000","last_peered":"2023-10-18T12:01:18.446081+0000","last_clean":"2023-10-18T12:01:18.446081+0000","last_became_active":"2023-10-18T12:01:09.134630+0000","last_became_peered":"2023-10-18T12:01:09.134630+0000","last_unstale":"2023-10-18T12:01:18.446081+0000","last_undegraded":"2023-10-18T12:01:18.446081+0000","last_fullsized":"2023-10-18T12:01:18.446081+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447293+0000","last_change":"2023-10-18T12:01:08.119870+0000","last_active":"2023-10-18T12:01:18.447293+0000","last_peered":"2023-10-18T12:01:18.447293+0000","last_clean":"2023-10-18T12:01:18.447293+0000","last_became_active":"2023-10-18T12:01:08.119586+0000","last_became_peered":"2023-10-18T12:01:08.119586+0000","last_unstale":"2023-10-18T12:01:18.447293+0000","last_undegraded":"2023-10-18T12:01:18.447293+0000","last_fullsized":"2023-10-18T12:01:18.447293+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:16.288493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453087+0000","last_change":"2023-10-18T12:01:19.453087+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453087+0000","last_undegraded":"2023-10-18T12:01:19.453087+0000","last_fullsized":"2023-10-18T12:01:19.453087+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443755+0000","last_change":"2023-10-18T12:01:09.130894+0000","last_active":"2023-10-18T12:01:18.443755+0000","last_peered":"2023-10-18T12:01:18.443755+0000","last_clean":"2023-10-18T12:01:18.443755+0000","last_became_active":"2023-10-18T12:01:09.130614+0000","last_became_peered":"2023-10-18T12:01:09.130614+0000","last_unstale":"2023-10-18T12:01:18.443755+0000","last_undegraded":"2023-10-18T12:01:18.443755+0000","last_fullsized":"2023-10-18T12:01:18.443755+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:03.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448884+0000","last_change":"2023-10-18T12:01:08.115174+0000","last_active":"2023-10-18T12:01:18.448884+0000","last_peered":"2023-10-18T12:01:18.448884+0000","last_clean":"2023-10-18T12:01:18.448884+0000","last_became_active":"2023-10-18T12:01:08.114546+0000","last_became_peered":"2023-10-18T12:01:08.114546+0000","last_unstale":"2023-10-18T12:01:18.448884+0000","last_undegraded":"2023-10-18T12:01:18.448884+0000","last_fullsized":"2023-10-18T12:01:18.448884+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453540+0000","last_change":"2023-10-18T12:01:19.453540+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453540+0000","last_undegraded":"2023-10-18T12:01:19.453540+0000","last_fullsized":"2023-10-18T12:01:19.453540+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448819+0000","last_change":"2023-10-18T12:01:09.122601+0000","last_active":"2023-10-18T12:01:18.448819+0000","last_peered":"2023-10-18T12:01:18.448819+0000","last_clean":"2023-10-18T12:01:18.448819+0000","last_became_active":"2023-10-18T12:01:09.122334+0000","last_became_peered":"2023-10-18T12:01:09.122334+0000","last_unstale":"2023-10-18T12:01:18.448819+0000","last_undegraded":"2023-10-18T12:01:18.448819+0000","last_fullsized":"2023-10-18T12:01:18.448819+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446549+0000","last_change":"2023-10-18T12:01:08.116478+0000","last_active":"2023-10-18T12:01:18.446549+0000","last_peered":"2023-10-18T12:01:18.446549+0000","last_clean":"2023-10-18T12:01:18.446549+0000","last_became_active":"2023-10-18T12:01:08.116181+0000","last_became_peered":"2023-10-18T12:01:08.116181+0000","last_unstale":"2023-10-18T12:01:18.446549+0000","last_undegraded":"2023-10-18T12:01:18.446549+0000","last_fullsized":"2023-10-18T12:01:18.446549+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:10.087064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451810+0000","last_change":"2023-10-18T12:01:19.451810+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451810+0000","last_undegraded":"2023-10-18T12:01:19.451810+0000","last_fullsized":"2023-10-18T12:01:19.451810+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447548+0000","last_change":"2023-10-18T12:01:09.130166+0000","last_active":"2023-10-18T12:01:18.447548+0000","last_peered":"2023-10-18T12:01:18.447548+0000","last_clean":"2023-10-18T12:01:18.447548+0000","last_became_active":"2023-10-18T12:01:09.129725+0000","last_became_peered":"2023-10-18T12:01:09.129725+0000","last_unstale":"2023-10-18T12:01:18.447548+0000","last_undegraded":"2023-10-18T12:01:18.447548+0000","last_fullsized":"2023-10-18T12:01:18.447548+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:40.892842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448642+0000","last_change":"2023-10-18T12:01:08.107385+0000","last_active":"2023-10-18T12:01:18.448642+0000","last_peered":"2023-10-18T12:01:18.448642+0000","last_clean":"2023-10-18T12:01:18.448642+0000","last_became_active":"2023-10-18T12:01:08.107129+0000","last_became_peered":"2023-10-18T12:01:08.107129+0000","last_unstale":"2023-10-18T12:01:18.448642+0000","last_undegraded":"2023-10-18T12:01:18.448642+0000","last_fullsized":"2023-10-18T12:01:18.448642+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:43.838857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455957+0000","last_change":"2023-10-18T12:01:19.455957+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455957+0000","last_undegraded":"2023-10-18T12:01:19.455957+0000","last_fullsized":"2023-10-18T12:01:19.455957+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445933+0000","last_change":"2023-10-18T12:01:09.133737+0000","last_active":"2023-10-18T12:01:18.445933+0000","last_peered":"2023-10-18T12:01:18.445933+0000","last_clean":"2023-10-18T12:01:18.445933+0000","last_became_active":"2023-10-18T12:01:09.133487+0000","last_became_peered":"2023-10-18T12:01:09.133487+0000","last_unstale":"2023-10-18T12:01:18.445933+0000","last_undegraded":"2023-10-18T12:01:18.445933+0000","last_fullsized":"2023-10-18T12:01:18.445933+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:53:36.261500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445701+0000","last_change":"2023-10-18T12:01:08.119435+0000","last_active":"2023-10-18T12:01:18.445701+0000","last_peered":"2023-10-18T12:01:18.445701+0000","last_clean":"2023-10-18T12:01:18.445701+0000","last_became_active":"2023-10-18T12:01:08.118740+0000","last_became_peered":"2023-10-18T12:01:08.118740+0000","last_unstale":"2023-10-18T12:01:18.445701+0000","last_undegraded":"2023-10-18T12:01:18.445701+0000","last_fullsized":"2023-10-18T12:01:18.445701+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:54.172871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454152+0000","last_change":"2023-10-18T12:01:19.454152+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454152+0000","last_undegraded":"2023-10-18T12:01:19.454152+0000","last_fullsized":"2023-10-18T12:01:19.454152+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446839+0000","last_change":"2023-10-18T12:01:09.132487+0000","last_active":"2023-10-18T12:01:18.446839+0000","last_peered":"2023-10-18T12:01:18.446839+0000","last_clean":"2023-10-18T12:01:18.446839+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:18.446839+0000","last_undegraded":"2023-10-18T12:01:18.446839+0000","last_fullsized":"2023-10-18T12:01:18.446839+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:14.584151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443645+0000","last_change":"2023-10-18T12:01:08.115982+0000","last_active":"2023-10-18T12:01:18.443645+0000","last_peered":"2023-10-18T12:01:18.443645+0000","last_clean":"2023-10-18T12:01:18.443645+0000","last_became_active":"2023-10-18T12:01:08.115682+0000","last_became_peered":"2023-10-18T12:01:08.115682+0000","last_unstale":"2023-10-18T12:01:18.443645+0000","last_undegraded":"2023-10-18T12:01:18.443645+0000","last_fullsized":"2023-10-18T12:01:18.443645+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:36.789486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.452968+0000","last_change":"2023-10-18T12:01:19.452968+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.452968+0000","last_undegraded":"2023-10-18T12:01:19.452968+0000","last_fullsized":"2023-10-18T12:01:19.452968+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445246+0000","last_change":"2023-10-18T12:01:09.130010+0000","last_active":"2023-10-18T12:01:18.445246+0000","last_peered":"2023-10-18T12:01:18.445246+0000","last_clean":"2023-10-18T12:01:18.445246+0000","last_became_active":"2023-10-18T12:01:09.129744+0000","last_became_peered":"2023-10-18T12:01:09.129744+0000","last_unstale":"2023-10-18T12:01:18.445246+0000","last_undegraded":"2023-10-18T12:01:18.445246+0000","last_fullsized":"2023-10-18T12:01:18.445246+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:48:02.468955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448887+0000","last_change":"2023-10-18T12:01:08.110833+0000","last_active":"2023-10-18T12:01:18.448887+0000","last_peered":"2023-10-18T12:01:18.448887+0000","last_clean":"2023-10-18T12:01:18.448887+0000","last_became_active":"2023-10-18T12:01:08.110005+0000","last_became_peered":"2023-10-18T12:01:08.110005+0000","last_unstale":"2023-10-18T12:01:18.448887+0000","last_undegraded":"2023-10-18T12:01:18.448887+0000","last_fullsized":"2023-10-18T12:01:18.448887+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:32.311648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456437+0000","last_change":"2023-10-18T12:01:19.456437+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456437+0000","last_undegraded":"2023-10-18T12:01:19.456437+0000","last_fullsized":"2023-10-18T12:01:19.456437+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444394+0000","last_change":"2023-10-18T12:01:09.129659+0000","last_active":"2023-10-18T12:01:18.444394+0000","last_peered":"2023-10-18T12:01:18.444394+0000","last_clean":"2023-10-18T12:01:18.444394+0000","last_became_active":"2023-10-18T12:01:09.129388+0000","last_became_peered":"2023-10-18T12:01:09.129388+0000","last_unstale":"2023-10-18T12:01:18.444394+0000","last_undegraded":"2023-10-18T12:01:18.444394+0000","last_fullsized":"2023-10-18T12:01:18.444394+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:17.308708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446188+0000","last_change":"2023-10-18T12:01:08.118321+0000","last_active":"2023-10-18T12:01:18.446188+0000","last_peered":"2023-10-18T12:01:18.446188+0000","last_clean":"2023-10-18T12:01:18.446188+0000","last_became_active":"2023-10-18T12:01:08.117995+0000","last_became_peered":"2023-10-18T12:01:08.117995+0000","last_unstale":"2023-10-18T12:01:18.446188+0000","last_undegraded":"2023-10-18T12:01:18.446188+0000","last_fullsized":"2023-10-18T12:01:18.446188+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:46.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453041+0000","last_change":"2023-10-18T12:01:19.453041+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453041+0000","last_undegraded":"2023-10-18T12:01:19.453041+0000","last_fullsized":"2023-10-18T12:01:19.453041+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445227+0000","last_change":"2023-10-18T12:01:09.134396+0000","last_active":"2023-10-18T12:01:18.445227+0000","last_peered":"2023-10-18T12:01:18.445227+0000","last_clean":"2023-10-18T12:01:18.445227+0000","last_became_active":"2023-10-18T12:01:09.134144+0000","last_became_peered":"2023-10-18T12:01:09.134144+0000","last_unstale":"2023-10-18T12:01:18.445227+0000","last_undegraded":"2023-10-18T12:01:18.445227+0000","last_fullsized":"2023-10-18T12:01:18.445227+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:23:53.396657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"241'3","reported_seq":17,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447167+0000","last_change":"2023-10-18T12:01:08.118871+0000","last_active":"2023-10-18T12:01:18.447167+0000","last_peered":"2023-10-18T12:01:18.447167+0000","last_clean":"2023-10-18T12:01:18.447167+0000","last_became_active":"2023-10-18T12:01:08.118572+0000","last_became_peered":"2023-10-18T12:01:08.118572+0000","last_unstale":"2023-10-18T12:01:18.447167+0000","last_undegraded":"2023-10-18T12:01:18.447167+0000","last_fullsized":"2023-10-18T12:01:18.447167+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:45.510206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453957+0000","last_change":"2023-10-18T12:01:19.453957+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453957+0000","last_undegraded":"2023-10-18T12:01:19.453957+0000","last_fullsized":"2023-10-18T12:01:19.453957+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444883+0000","last_change":"2023-10-18T12:01:09.120444+0000","last_active":"2023-10-18T12:01:18.444883+0000","last_peered":"2023-10-18T12:01:18.444883+0000","last_clean":"2023-10-18T12:01:18.444883+0000","last_became_active":"2023-10-18T12:01:09.120018+0000","last_became_peered":"2023-10-18T12:01:09.120018+0000","last_unstale":"2023-10-18T12:01:18.444883+0000","last_undegraded":"2023-10-18T12:01:18.444883+0000","last_fullsized":"2023-10-18T12:01:18.444883+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:29.145722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:19.370024+0000","last_change":"2023-10-18T12:01:08.117890+0000","last_active":"2023-10-18T12:01:19.370024+0000","last_peered":"2023-10-18T12:01:19.370024+0000","last_clean":"2023-10-18T12:01:19.370024+0000","last_became_active":"2023-10-18T12:01:08.117606+0000","last_became_peered":"2023-10-18T12:01:08.117606+0000","last_unstale":"2023-10-18T12:01:19.370024+0000","last_undegraded":"2023-10-18T12:01:19.370024+0000","last_fullsized":"2023-10-18T12:01:19.370024+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:49.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454638+0000","last_change":"2023-10-18T12:01:19.454638+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454638+0000","last_undegraded":"2023-10-18T12:01:19.454638+0000","last_fullsized":"2023-10-18T12:01:19.454638+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439867+0000","last_change":"2023-10-18T12:01:09.127054+0000","last_active":"2023-10-18T12:01:18.439867+0000","last_peered":"2023-10-18T12:01:18.439867+0000","last_clean":"2023-10-18T12:01:18.439867+0000","last_became_active":"2023-10-18T12:01:09.126804+0000","last_became_peered":"2023-10-18T12:01:09.126804+0000","last_unstale":"2023-10-18T12:01:18.439867+0000","last_undegraded":"2023-10-18T12:01:18.439867+0000","last_fullsized":"2023-10-18T12:01:18.439867+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:20.948970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446639+0000","last_change":"2023-10-18T12:01:08.118544+0000","last_active":"2023-10-18T12:01:18.446639+0000","last_peered":"2023-10-18T12:01:18.446639+0000","last_clean":"2023-10-18T12:01:18.446639+0000","last_became_active":"2023-10-18T12:01:08.118228+0000","last_became_peered":"2023-10-18T12:01:08.118228+0000","last_unstale":"2023-10-18T12:01:18.446639+0000","last_undegraded":"2023-10-18T12:01:18.446639+0000","last_fullsized":"2023-10-18T12:01:18.446639+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456191+0000","last_change":"2023-10-18T12:01:19.456191+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456191+0000","last_undegraded":"2023-10-18T12:01:19.456191+0000","last_fullsized":"2023-10-18T12:01:19.456191+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447511+0000","last_change":"2023-10-18T12:01:09.120750+0000","last_active":"2023-10-18T12:01:18.447511+0000","last_peered":"2023-10-18T12:01:18.447511+0000","last_clean":"2023-10-18T12:01:18.447511+0000","last_became_active":"2023-10-18T12:01:09.120490+0000","last_became_peered":"2023-10-18T12:01:09.120490+0000","last_unstale":"2023-10-18T12:01:18.447511+0000","last_undegraded":"2023-10-18T12:01:18.447511+0000","last_fullsized":"2023-10-18T12:01:18.447511+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:51.005840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446987+0000","last_change":"2023-10-18T12:01:08.118022+0000","last_active":"2023-10-18T12:01:18.446987+0000","last_peered":"2023-10-18T12:01:18.446987+0000","last_clean":"2023-10-18T12:01:18.446987+0000","last_became_active":"2023-10-18T12:01:08.117754+0000","last_became_peered":"2023-10-18T12:01:08.117754+0000","last_unstale":"2023-10-18T12:01:18.446987+0000","last_undegraded":"2023-10-18T12:01:18.446987+0000","last_fullsized":"2023-10-18T12:01:18.446987+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:37.834293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455355+0000","last_change":"2023-10-18T12:01:19.455355+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455355+0000","last_undegraded":"2023-10-18T12:01:19.455355+0000","last_fullsized":"2023-10-18T12:01:19.455355+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447313+0000","last_change":"2023-10-18T12:01:09.135383+0000","last_active":"2023-10-18T12:01:18.447313+0000","last_peered":"2023-10-18T12:01:18.447313+0000","last_clean":"2023-10-18T12:01:18.447313+0000","last_became_active":"2023-10-18T12:01:09.135114+0000","last_became_peered":"2023-10-18T12:01:09.135114+0000","last_unstale":"2023-10-18T12:01:18.447313+0000","last_undegraded":"2023-10-18T12:01:18.447313+0000","last_fullsized":"2023-10-18T12:01:18.447313+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446147+0000","last_change":"2023-10-18T12:01:08.115155+0000","last_active":"2023-10-18T12:01:18.446147+0000","last_peered":"2023-10-18T12:01:18.446147+0000","last_clean":"2023-10-18T12:01:18.446147+0000","last_became_active":"2023-10-18T12:01:08.114843+0000","last_became_peered":"2023-10-18T12:01:08.114843+0000","last_unstale":"2023-10-18T12:01:18.446147+0000","last_undegraded":"2023-10-18T12:01:18.446147+0000","last_fullsized":"2023-10-18T12:01:18.446147+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:28.358756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455037+0000","last_change":"2023-10-18T12:01:19.455037+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455037+0000","last_undegraded":"2023-10-18T12:01:19.455037+0000","last_fullsized":"2023-10-18T12:01:19.455037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446744+0000","last_change":"2023-10-18T12:01:09.131820+0000","last_active":"2023-10-18T12:01:18.446744+0000","last_peered":"2023-10-18T12:01:18.446744+0000","last_clean":"2023-10-18T12:01:18.446744+0000","last_became_active":"2023-10-18T12:01:09.131568+0000","last_became_peered":"2023-10-18T12:01:09.131568+0000","last_unstale":"2023-10-18T12:01:18.446744+0000","last_undegraded":"2023-10-18T12:01:18.446744+0000","last_fullsized":"2023-10-18T12:01:18.446744+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.395799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"241'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439719+0000","last_change":"2023-10-18T12:01:08.112711+0000","last_active":"2023-10-18T12:01:18.439719+0000","last_peered":"2023-10-18T12:01:18.439719+0000","last_clean":"2023-10-18T12:01:18.439719+0000","last_became_active":"2023-10-18T12:01:08.112436+0000","last_became_peered":"2023-10-18T12:01:08.112436+0000","last_unstale":"2023-10-18T12:01:18.439719+0000","last_undegraded":"2023-10-18T12:01:18.439719+0000","last_fullsized":"2023-10-18T12:01:18.439719+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:15.568916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455648+0000","last_change":"2023-10-18T12:01:19.455648+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455648+0000","last_undegraded":"2023-10-18T12:01:19.455648+0000","last_fullsized":"2023-10-18T12:01:19.455648+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444666+0000","last_change":"2023-10-18T12:01:09.135106+0000","last_active":"2023-10-18T12:01:18.444666+0000","last_peered":"2023-10-18T12:01:18.444666+0000","last_clean":"2023-10-18T12:01:18.444666+0000","last_became_active":"2023-10-18T12:01:09.134828+0000","last_became_peered":"2023-10-18T12:01:09.134828+0000","last_unstale":"2023-10-18T12:01:18.444666+0000","last_undegraded":"2023-10-18T12:01:18.444666+0000","last_fullsized":"2023-10-18T12:01:18.444666+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:39.126921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444574+0000","last_change":"2023-10-18T12:01:08.119248+0000","last_active":"2023-10-18T12:01:18.444574+0000","last_peered":"2023-10-18T12:01:18.444574+0000","last_clean":"2023-10-18T12:01:18.444574+0000","last_became_active":"2023-10-18T12:01:08.118973+0000","last_became_peered":"2023-10-18T12:01:08.118973+0000","last_unstale":"2023-10-18T12:01:18.444574+0000","last_undegraded":"2023-10-18T12:01:18.444574+0000","last_fullsized":"2023-10-18T12:01:18.444574+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:37.079812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459833+0000","last_change":"2023-10-18T12:01:19.459833+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459833+0000","last_undegraded":"2023-10-18T12:01:19.459833+0000","last_fullsized":"2023-10-18T12:01:19.459833+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445946+0000","last_change":"2023-10-18T12:01:09.132827+0000","last_active":"2023-10-18T12:01:18.445946+0000","last_peered":"2023-10-18T12:01:18.445946+0000","last_clean":"2023-10-18T12:01:18.445946+0000","last_became_active":"2023-10-18T12:01:09.132545+0000","last_became_peered":"2023-10-18T12:01:09.132545+0000","last_unstale":"2023-10-18T12:01:18.445946+0000","last_undegraded":"2023-10-18T12:01:18.445946+0000","last_fullsized":"2023-10-18T12:01:18.445946+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:02.034692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439997+0000","last_change":"2023-10-18T12:01:08.107964+0000","last_active":"2023-10-18T12:01:18.439997+0000","last_peered":"2023-10-18T12:01:18.439997+0000","last_clean":"2023-10-18T12:01:18.439997+0000","last_became_active":"2023-10-18T12:01:08.107696+0000","last_became_peered":"2023-10-18T12:01:08.107696+0000","last_unstale":"2023-10-18T12:01:18.439997+0000","last_undegraded":"2023-10-18T12:01:18.439997+0000","last_fullsized":"2023-10-18T12:01:18.439997+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:18.578919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.453669+0000","last_change":"2023-10-18T12:01:19.453669+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.453669+0000","last_undegraded":"2023-10-18T12:01:19.453669+0000","last_fullsized":"2023-10-18T12:01:19.453669+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439817+0000","last_change":"2023-10-18T12:01:09.136078+0000","last_active":"2023-10-18T12:01:18.439817+0000","last_peered":"2023-10-18T12:01:18.439817+0000","last_clean":"2023-10-18T12:01:18.439817+0000","last_became_active":"2023-10-18T12:01:09.134722+0000","last_became_peered":"2023-10-18T12:01:09.134722+0000","last_unstale":"2023-10-18T12:01:18.439817+0000","last_undegraded":"2023-10-18T12:01:18.439817+0000","last_fullsized":"2023-10-18T12:01:18.439817+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:09.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447390+0000","last_change":"2023-10-18T12:01:08.120416+0000","last_active":"2023-10-18T12:01:18.447390+0000","last_peered":"2023-10-18T12:01:18.447390+0000","last_clean":"2023-10-18T12:01:18.447390+0000","last_became_active":"2023-10-18T12:01:08.120122+0000","last_became_peered":"2023-10-18T12:01:08.120122+0000","last_unstale":"2023-10-18T12:01:18.447390+0000","last_undegraded":"2023-10-18T12:01:18.447390+0000","last_fullsized":"2023-10-18T12:01:18.447390+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:26.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.455164+0000","last_change":"2023-10-18T12:01:19.455164+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.455164+0000","last_undegraded":"2023-10-18T12:01:19.455164+0000","last_fullsized":"2023-10-18T12:01:19.455164+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446070+0000","last_change":"2023-10-18T12:01:09.135042+0000","last_active":"2023-10-18T12:01:18.446070+0000","last_peered":"2023-10-18T12:01:18.446070+0000","last_clean":"2023-10-18T12:01:18.446070+0000","last_became_active":"2023-10-18T12:01:09.134765+0000","last_became_peered":"2023-10-18T12:01:09.134765+0000","last_unstale":"2023-10-18T12:01:18.446070+0000","last_undegraded":"2023-10-18T12:01:18.446070+0000","last_fullsized":"2023-10-18T12:01:18.446070+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:58.131101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446991+0000","last_change":"2023-10-18T12:01:08.117972+0000","last_active":"2023-10-18T12:01:18.446991+0000","last_peered":"2023-10-18T12:01:18.446991+0000","last_clean":"2023-10-18T12:01:18.446991+0000","last_became_active":"2023-10-18T12:01:08.117690+0000","last_became_peered":"2023-10-18T12:01:08.117690+0000","last_unstale":"2023-10-18T12:01:18.446991+0000","last_undegraded":"2023-10-18T12:01:18.446991+0000","last_fullsized":"2023-10-18T12:01:18.446991+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.458648+0000","last_change":"2023-10-18T12:01:19.458648+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.458648+0000","last_undegraded":"2023-10-18T12:01:19.458648+0000","last_fullsized":"2023-10-18T12:01:19.458648+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444663+0000","last_change":"2023-10-18T12:01:09.122673+0000","last_active":"2023-10-18T12:01:18.444663+0000","last_peered":"2023-10-18T12:01:18.444663+0000","last_clean":"2023-10-18T12:01:18.444663+0000","last_became_active":"2023-10-18T12:01:09.122430+0000","last_became_peered":"2023-10-18T12:01:09.122430+0000","last_unstale":"2023-10-18T12:01:18.444663+0000","last_undegraded":"2023-10-18T12:01:18.444663+0000","last_fullsized":"2023-10-18T12:01:18.444663+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:29.387267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446157+0000","last_change":"2023-10-18T12:01:08.119439+0000","last_active":"2023-10-18T12:01:18.446157+0000","last_peered":"2023-10-18T12:01:18.446157+0000","last_clean":"2023-10-18T12:01:18.446157+0000","last_became_active":"2023-10-18T12:01:08.119158+0000","last_became_peered":"2023-10-18T12:01:08.119158+0000","last_unstale":"2023-10-18T12:01:18.446157+0000","last_undegraded":"2023-10-18T12:01:18.446157+0000","last_fullsized":"2023-10-18T12:01:18.446157+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:56.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454552+0000","last_change":"2023-10-18T12:01:19.454552+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454552+0000","last_undegraded":"2023-10-18T12:01:19.454552+0000","last_fullsized":"2023-10-18T12:01:19.454552+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.440049+0000","last_change":"2023-10-18T12:01:09.124352+0000","last_active":"2023-10-18T12:01:18.440049+0000","last_peered":"2023-10-18T12:01:18.440049+0000","last_clean":"2023-10-18T12:01:18.440049+0000","last_became_active":"2023-10-18T12:01:09.124072+0000","last_became_peered":"2023-10-18T12:01:09.124072+0000","last_unstale":"2023-10-18T12:01:18.440049+0000","last_undegraded":"2023-10-18T12:01:18.440049+0000","last_fullsized":"2023-10-18T12:01:18.440049+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:15.239290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443813+0000","last_change":"2023-10-18T12:01:08.117335+0000","last_active":"2023-10-18T12:01:18.443813+0000","last_peered":"2023-10-18T12:01:18.443813+0000","last_clean":"2023-10-18T12:01:18.443813+0000","last_became_active":"2023-10-18T12:01:08.117032+0000","last_became_peered":"2023-10-18T12:01:08.117032+0000","last_unstale":"2023-10-18T12:01:18.443813+0000","last_undegraded":"2023-10-18T12:01:18.443813+0000","last_fullsized":"2023-10-18T12:01:18.443813+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:33.304569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456597+0000","last_change":"2023-10-18T12:01:19.456597+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456597+0000","last_undegraded":"2023-10-18T12:01:19.456597+0000","last_fullsized":"2023-10-18T12:01:19.456597+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446661+0000","last_change":"2023-10-18T12:01:09.135148+0000","last_active":"2023-10-18T12:01:18.446661+0000","last_peered":"2023-10-18T12:01:18.446661+0000","last_clean":"2023-10-18T12:01:18.446661+0000","last_became_active":"2023-10-18T12:01:09.134895+0000","last_became_peered":"2023-10-18T12:01:09.134895+0000","last_unstale":"2023-10-18T12:01:18.446661+0000","last_undegraded":"2023-10-18T12:01:18.446661+0000","last_fullsized":"2023-10-18T12:01:18.446661+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:19.263310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444467+0000","last_change":"2023-10-18T12:01:08.120618+0000","last_active":"2023-10-18T12:01:18.444467+0000","last_peered":"2023-10-18T12:01:18.444467+0000","last_clean":"2023-10-18T12:01:18.444467+0000","last_became_active":"2023-10-18T12:01:08.120315+0000","last_became_peered":"2023-10-18T12:01:08.120315+0000","last_unstale":"2023-10-18T12:01:18.444467+0000","last_undegraded":"2023-10-18T12:01:18.444467+0000","last_fullsized":"2023-10-18T12:01:18.444467+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:36.555722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457598+0000","last_change":"2023-10-18T12:01:19.457598+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457598+0000","last_undegraded":"2023-10-18T12:01:19.457598+0000","last_fullsized":"2023-10-18T12:01:19.457598+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447164+0000","last_change":"2023-10-18T12:01:09.130881+0000","last_active":"2023-10-18T12:01:18.447164+0000","last_peered":"2023-10-18T12:01:18.447164+0000","last_clean":"2023-10-18T12:01:18.447164+0000","last_became_active":"2023-10-18T12:01:09.130628+0000","last_became_peered":"2023-10-18T12:01:09.130628+0000","last_unstale":"2023-10-18T12:01:18.447164+0000","last_undegraded":"2023-10-18T12:01:18.447164+0000","last_fullsized":"2023-10-18T12:01:18.447164+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:13.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448426+0000","last_change":"2023-10-18T12:01:08.120313+0000","last_active":"2023-10-18T12:01:18.448426+0000","last_peered":"2023-10-18T12:01:18.448426+0000","last_clean":"2023-10-18T12:01:18.448426+0000","last_became_active":"2023-10-18T12:01:08.120059+0000","last_became_peered":"2023-10-18T12:01:08.120059+0000","last_unstale":"2023-10-18T12:01:18.448426+0000","last_undegraded":"2023-10-18T12:01:18.448426+0000","last_fullsized":"2023-10-18T12:01:18.448426+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456394+0000","last_change":"2023-10-18T12:01:19.456394+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456394+0000","last_undegraded":"2023-10-18T12:01:19.456394+0000","last_fullsized":"2023-10-18T12:01:19.456394+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445079+0000","last_change":"2023-10-18T12:01:09.125685+0000","last_active":"2023-10-18T12:01:18.445079+0000","last_peered":"2023-10-18T12:01:18.445079+0000","last_clean":"2023-10-18T12:01:18.445079+0000","last_became_active":"2023-10-18T12:01:09.125395+0000","last_became_peered":"2023-10-18T12:01:09.125395+0000","last_unstale":"2023-10-18T12:01:18.445079+0000","last_undegraded":"2023-10-18T12:01:18.445079+0000","last_fullsized":"2023-10-18T12:01:18.445079+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:47.228016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"241'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444968+0000","last_change":"2023-10-18T12:01:08.117717+0000","last_active":"2023-10-18T12:01:18.444968+0000","last_peered":"2023-10-18T12:01:18.444968+0000","last_clean":"2023-10-18T12:01:18.444968+0000","last_became_active":"2023-10-18T12:01:08.117405+0000","last_became_peered":"2023-10-18T12:01:08.117405+0000","last_unstale":"2023-10-18T12:01:18.444968+0000","last_undegraded":"2023-10-18T12:01:18.444968+0000","last_fullsized":"2023-10-18T12:01:18.444968+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:00.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.454367+0000","last_change":"2023-10-18T12:01:19.454367+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.454367+0000","last_undegraded":"2023-10-18T12:01:19.454367+0000","last_fullsized":"2023-10-18T12:01:19.454367+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445826+0000","last_change":"2023-10-18T12:01:09.132599+0000","last_active":"2023-10-18T12:01:18.445826+0000","last_peered":"2023-10-18T12:01:18.445826+0000","last_clean":"2023-10-18T12:01:18.445826+0000","last_became_active":"2023-10-18T12:01:09.131710+0000","last_became_peered":"2023-10-18T12:01:09.131710+0000","last_unstale":"2023-10-18T12:01:18.445826+0000","last_undegraded":"2023-10-18T12:01:18.445826+0000","last_fullsized":"2023-10-18T12:01:18.445826+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:18.701174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446916+0000","last_change":"2023-10-18T12:01:08.118553+0000","last_active":"2023-10-18T12:01:18.446916+0000","last_peered":"2023-10-18T12:01:18.446916+0000","last_clean":"2023-10-18T12:01:18.446916+0000","last_became_active":"2023-10-18T12:01:08.118286+0000","last_became_peered":"2023-10-18T12:01:08.118286+0000","last_unstale":"2023-10-18T12:01:18.446916+0000","last_undegraded":"2023-10-18T12:01:18.446916+0000","last_fullsized":"2023-10-18T12:01:18.446916+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:05:32.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.460259+0000","last_change":"2023-10-18T12:01:19.460259+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.460259+0000","last_undegraded":"2023-10-18T12:01:19.460259+0000","last_fullsized":"2023-10-18T12:01:19.460259+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445148+0000","last_change":"2023-10-18T12:01:09.133522+0000","last_active":"2023-10-18T12:01:18.445148+0000","last_peered":"2023-10-18T12:01:18.445148+0000","last_clean":"2023-10-18T12:01:18.445148+0000","last_became_active":"2023-10-18T12:01:09.133267+0000","last_became_peered":"2023-10-18T12:01:09.133267+0000","last_unstale":"2023-10-18T12:01:18.445148+0000","last_undegraded":"2023-10-18T12:01:18.445148+0000","last_fullsized":"2023-10-18T12:01:18.445148+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:44:14.702965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448989+0000","last_change":"2023-10-18T12:01:08.116830+0000","last_active":"2023-10-18T12:01:18.448989+0000","last_peered":"2023-10-18T12:01:18.448989+0000","last_clean":"2023-10-18T12:01:18.448989+0000","last_became_active":"2023-10-18T12:01:08.116538+0000","last_became_peered":"2023-10-18T12:01:08.116538+0000","last_unstale":"2023-10-18T12:01:18.448989+0000","last_undegraded":"2023-10-18T12:01:18.448989+0000","last_fullsized":"2023-10-18T12:01:18.448989+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:09.165065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451313+0000","last_change":"2023-10-18T12:01:19.451313+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451313+0000","last_undegraded":"2023-10-18T12:01:19.451313+0000","last_fullsized":"2023-10-18T12:01:19.451313+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446209+0000","last_change":"2023-10-18T12:01:09.128305+0000","last_active":"2023-10-18T12:01:18.446209+0000","last_peered":"2023-10-18T12:01:18.446209+0000","last_clean":"2023-10-18T12:01:18.446209+0000","last_became_active":"2023-10-18T12:01:09.127933+0000","last_became_peered":"2023-10-18T12:01:09.127933+0000","last_unstale":"2023-10-18T12:01:18.446209+0000","last_undegraded":"2023-10-18T12:01:18.446209+0000","last_fullsized":"2023-10-18T12:01:18.446209+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:28.312681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448893+0000","last_change":"2023-10-18T12:01:08.115363+0000","last_active":"2023-10-18T12:01:18.448893+0000","last_peered":"2023-10-18T12:01:18.448893+0000","last_clean":"2023-10-18T12:01:18.448893+0000","last_became_active":"2023-10-18T12:01:08.114903+0000","last_became_peered":"2023-10-18T12:01:08.114903+0000","last_unstale":"2023-10-18T12:01:18.448893+0000","last_undegraded":"2023-10-18T12:01:18.448893+0000","last_fullsized":"2023-10-18T12:01:18.448893+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:46.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.456219+0000","last_change":"2023-10-18T12:01:19.456219+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.456219+0000","last_undegraded":"2023-10-18T12:01:19.456219+0000","last_fullsized":"2023-10-18T12:01:19.456219+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.448801+0000","last_change":"2023-10-18T12:01:09.127159+0000","last_active":"2023-10-18T12:01:18.448801+0000","last_peered":"2023-10-18T12:01:18.448801+0000","last_clean":"2023-10-18T12:01:18.448801+0000","last_became_active":"2023-10-18T12:01:09.126874+0000","last_became_peered":"2023-10-18T12:01:09.126874+0000","last_unstale":"2023-10-18T12:01:18.448801+0000","last_undegraded":"2023-10-18T12:01:18.448801+0000","last_fullsized":"2023-10-18T12:01:18.448801+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444343+0000","last_change":"2023-10-18T12:01:08.118539+0000","last_active":"2023-10-18T12:01:18.444343+0000","last_peered":"2023-10-18T12:01:18.444343+0000","last_clean":"2023-10-18T12:01:18.444343+0000","last_became_active":"2023-10-18T12:01:08.116966+0000","last_became_peered":"2023-10-18T12:01:08.116966+0000","last_unstale":"2023-10-18T12:01:18.444343+0000","last_undegraded":"2023-10-18T12:01:18.444343+0000","last_fullsized":"2023-10-18T12:01:18.444343+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:20.828671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.457469+0000","last_change":"2023-10-18T12:01:19.457469+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.457469+0000","last_undegraded":"2023-10-18T12:01:19.457469+0000","last_fullsized":"2023-10-18T12:01:19.457469+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.444299+0000","last_change":"2023-10-18T12:01:09.119148+0000","last_active":"2023-10-18T12:01:18.444299+0000","last_peered":"2023-10-18T12:01:18.444299+0000","last_clean":"2023-10-18T12:01:18.444299+0000","last_became_active":"2023-10-18T12:01:09.118894+0000","last_became_peered":"2023-10-18T12:01:09.118894+0000","last_unstale":"2023-10-18T12:01:18.444299+0000","last_undegraded":"2023-10-18T12:01:18.444299+0000","last_fullsized":"2023-10-18T12:01:18.444299+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:42.857305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447050+0000","last_change":"2023-10-18T12:01:08.117526+0000","last_active":"2023-10-18T12:01:18.447050+0000","last_peered":"2023-10-18T12:01:18.447050+0000","last_clean":"2023-10-18T12:01:18.447050+0000","last_became_active":"2023-10-18T12:01:08.117234+0000","last_became_peered":"2023-10-18T12:01:08.117234+0000","last_unstale":"2023-10-18T12:01:18.447050+0000","last_undegraded":"2023-10-18T12:01:18.447050+0000","last_fullsized":"2023-10-18T12:01:18.447050+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:43.295217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.460764+0000","last_change":"2023-10-18T12:01:19.460764+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.460764+0000","last_undegraded":"2023-10-18T12:01:19.460764+0000","last_fullsized":"2023-10-18T12:01:19.460764+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.439950+0000","last_change":"2023-10-18T12:01:09.125150+0000","last_active":"2023-10-18T12:01:18.439950+0000","last_peered":"2023-10-18T12:01:18.439950+0000","last_clean":"2023-10-18T12:01:18.439950+0000","last_became_active":"2023-10-18T12:01:09.124900+0000","last_became_peered":"2023-10-18T12:01:09.124900+0000","last_unstale":"2023-10-18T12:01:18.439950+0000","last_undegraded":"2023-10-18T12:01:18.439950+0000","last_fullsized":"2023-10-18T12:01:18.439950+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:10.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445789+0000","last_change":"2023-10-18T12:01:08.119612+0000","last_active":"2023-10-18T12:01:18.445789+0000","last_peered":"2023-10-18T12:01:18.445789+0000","last_clean":"2023-10-18T12:01:18.445789+0000","last_became_active":"2023-10-18T12:01:08.119116+0000","last_became_peered":"2023-10-18T12:01:08.119116+0000","last_unstale":"2023-10-18T12:01:18.445789+0000","last_undegraded":"2023-10-18T12:01:18.445789+0000","last_fullsized":"2023-10-18T12:01:18.445789+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:02.536144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.451311+0000","last_change":"2023-10-18T12:01:19.451311+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.451311+0000","last_undegraded":"2023-10-18T12:01:19.451311+0000","last_fullsized":"2023-10-18T12:01:19.451311+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.446463+0000","last_change":"2023-10-18T12:01:09.130360+0000","last_active":"2023-10-18T12:01:18.446463+0000","last_peered":"2023-10-18T12:01:18.446463+0000","last_clean":"2023-10-18T12:01:18.446463+0000","last_became_active":"2023-10-18T12:01:09.130086+0000","last_became_peered":"2023-10-18T12:01:09.130086+0000","last_unstale":"2023-10-18T12:01:18.446463+0000","last_undegraded":"2023-10-18T12:01:18.446463+0000","last_fullsized":"2023-10-18T12:01:18.446463+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:28.085761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443434+0000","last_change":"2023-10-18T12:01:08.119803+0000","last_active":"2023-10-18T12:01:18.443434+0000","last_peered":"2023-10-18T12:01:18.443434+0000","last_clean":"2023-10-18T12:01:18.443434+0000","last_became_active":"2023-10-18T12:01:08.119467+0000","last_became_peered":"2023-10-18T12:01:08.119467+0000","last_unstale":"2023-10-18T12:01:18.443434+0000","last_undegraded":"2023-10-18T12:01:18.443434+0000","last_fullsized":"2023-10-18T12:01:18.443434+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:29.410010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":1,"reported_epoch":244,"state":"creating+peering","last_fresh":"2023-10-18T12:01:19.459374+0000","last_change":"2023-10-18T12:01:19.459374+0000","last_active":"2023-10-18T12:01:19.435707+0000","last_peered":"2023-10-18T12:01:19.435707+0000","last_clean":"2023-10-18T12:01:19.435707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T12:01:19.459374+0000","last_undegraded":"2023-10-18T12:01:19.459374+0000","last_fullsized":"2023-10-18T12:01:19.459374+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":13,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.443794+0000","last_change":"2023-10-18T12:01:09.122670+0000","last_active":"2023-10-18T12:01:18.443794+0000","last_peered":"2023-10-18T12:01:18.443794+0000","last_clean":"2023-10-18T12:01:18.443794+0000","last_became_active":"2023-10-18T12:01:09.122408+0000","last_became_peered":"2023-10-18T12:01:09.122408+0000","last_unstale":"2023-10-18T12:01:18.443794+0000","last_undegraded":"2023-10-18T12:01:18.443794+0000","last_fullsized":"2023-10-18T12:01:18.443794+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:39.479911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"241'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.447844+0000","last_change":"2023-10-18T12:01:08.111055+0000","last_active":"2023-10-18T12:01:18.447844+0000","last_peered":"2023-10-18T12:01:18.447844+0000","last_clean":"2023-10-18T12:01:18.447844+0000","last_became_active":"2023-10-18T12:01:08.110728+0000","last_became_peered":"2023-10-18T12:01:08.110728+0000","last_unstale":"2023-10-18T12:01:18.447844+0000","last_undegraded":"2023-10-18T12:01:18.447844+0000","last_fullsized":"2023-10-18T12:01:18.447844+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:15.472803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"227'379","reported_seq":1331,"reported_epoch":243,"state":"active+clean","last_fresh":"2023-10-18T12:01:18.445541+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:01:18.445541+0000","last_peered":"2023-10-18T12:01:18.445541+0000","last_clean":"2023-10-18T12:01:18.445541+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:01:18.445541+0000","last_undegraded":"2023-10-18T12:01:18.445541+0000","last_fullsized":"2023-10-18T12:01:18.445541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":379,"ondisk_log_size":379,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52808,"kb_used_data":3580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319032,"statfs":{"total":96636764160,"available":96582688768,"internally_reserved":0,"allocated":3665920,"data_stored":2169281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.363},"last":0.42399999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.749,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73999999999999999,"5min":0.82199999999999995,"15min":0.86399999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.298},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.82499999999999996,"15min":0.82799999999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.754,"15min":1.5249999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71299999999999997,"5min":0.75800000000000001,"15min":0.81299999999999994},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.5},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":1.478},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.874},"last":0.67800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834576000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42500,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329340,"statfs":{"total":96636764160,"available":96593244160,"internally_reserved":0,"allocated":3072000,"data_stored":1578901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.495},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71299999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834576000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64700,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307140,"statfs":{"total":96636764160,"available":96570511360,"internally_reserved":0,"allocated":3063808,"data_stored":1578789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.498},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":1.101,"15min":1.1539999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.71099999999999997,"15min":0.751},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.61699999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72699999999999998,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81399999999999995,"15min":1},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53000000000000003,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.749,"5min":0.82999999999999996,"15min":1.01},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.255,"15min":0.255},"max":{"1min":0.78800000000000003,"5min":0.83699999999999997,"15min":1.101},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48064,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323776,"statfs":{"total":96636764160,"available":96587546624,"internally_reserved":0,"allocated":3067904,"data_stored":1578859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.22500000000000001},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.59599999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":1.123},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.56899999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.66000000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":1.018,"15min":1.018},"last":0.752}]},{"osd":7,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.88900000000000001,"15min":1.05},"last":0.623},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.77900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834576000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48100,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323740,"statfs":{"total":96636764160,"available":96587509760,"internally_reserved":0,"allocated":3100672,"data_stored":1589807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.23999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.378,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.25},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.39300000000000002,"15min":0.40699999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.38400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.39900000000000002,"15min":0.40600000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44900000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.432,"5min":1.899,"15min":1.899},"last":0.28799999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.42899999999999999,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.434,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.87,"5min":1.034,"15min":1.052},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.629}]},{"osd":5,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":1.522,"15min":1.522},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.91100000000000003,"15min":1.026},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82299999999999995,"5min":1.01,"15min":1.01},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.129,"15min":1.129},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0649999999999999},"last":0.69199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60548,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311292,"statfs":{"total":96636764160,"available":96574763008,"internally_reserved":0,"allocated":3072000,"data_stored":1579441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.52300000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.40500000000000003,"15min":0.41599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.31900000000000001}]},{"osd":1,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.61499999999999999,"15min":0.624},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.374,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.63100000000000001,"15min":0.746},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55300000000000005,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86299999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66300000000000003,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.087},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.85199999999999998,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63400000000000001,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.95399999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":1.0609999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60552,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311288,"statfs":{"total":96636764160,"available":96574758912,"internally_reserved":0,"allocated":3076096,"data_stored":1586301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.314}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.32600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.375}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.41099999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.38500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575999,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43084,"kb_used_data":3584,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328756,"statfs":{"total":96636764160,"available":96592646144,"internally_reserved":0,"allocated":3670016,"data_stored":2171210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52200000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.52300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.26100000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.51100000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.96999999999999997,"15min":0.997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72399999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.95499999999999996,"15min":0.96399999999999997},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64700000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":1.03},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70999999999999996,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.5}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.78000000000000003,"5min":0.997,"15min":0.997},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:01:25.088 INFO:tasks.ceph.mds.c.smithi125.stderr:2023-10-18T12:01:25.086+0000 7ff006230640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:01:27.068 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:01:27.071 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:01:27.803 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:01:27.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/3382197466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 msgr2=0x7fe7b006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:27.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/3382197466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 0x7fe7b006b2e0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fe7b006d590 tx=0x7fe7a402bda0 comp rx=0 tx=0).stop 2023-10-18T12:01:27.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/3382197466 shutdown_connections 2023-10-18T12:01:27.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/3382197466 >> v2:172.21.15.139:3300/0 conn(0x7fe7b00ab500 0x7fe7b00ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:27.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/3382197466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7b00c96b0 0x7fe7b00c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:27.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/3382197466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 0x7fe7b006b2e0 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:27.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/3382197466 >> 172.21.15.125:0/3382197466 conn(0x7fe7b0069930 msgr2=0x7fe7b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:27.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/3382197466 shutdown_connections 2023-10-18T12:01:27.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.907+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/3382197466 wait complete. 2023-10-18T12:01:27.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.909+0000 7fe7b4ba9640 1 Processor -- start 2023-10-18T12:01:27.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.909+0000 7fe7b4ba9640 1 -- start start 2023-10-18T12:01:27.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.909+0000 7fe7b4ba9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 0x7fe7b015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:27.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.909+0000 7fe7b4ba9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7b00ab500 0x7fe7b01f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:27.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.909+0000 7fe7b4ba9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b00c96b0 0x7fe7b00b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:27.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.909+0000 7fe7b4ba9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b01f18f0 con 0x7fe7b006af00 2023-10-18T12:01:27.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.909+0000 7fe7b4ba9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b01f0d50 con 0x7fe7b00c96b0 2023-10-18T12:01:27.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.909+0000 7fe7b4ba9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b01f0fc0 con 0x7fe7b00ab500 2023-10-18T12:01:27.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 0x7fe7b015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:27.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 0x7fe7b015bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39540/0 (socket says 172.21.15.125:39540) 2023-10-18T12:01:27.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 -- 172.21.15.125:0/1318315459 learned_addr learned my addr 172.21.15.125:0/1318315459 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:27.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7aed9d640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b00c96b0 0x7fe7b00b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:27.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7add9b640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7b00ab500 0x7fe7b01f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:27.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 -- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7b00ab500 msgr2=0x7fe7b01f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:27.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7b00ab500 0x7fe7b01f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:27.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 -- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b00c96b0 msgr2=0x7fe7b00b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:27.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b00c96b0 0x7fe7b00b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:27.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 -- 172.21.15.125:0/1318315459 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a4009560 con 0x7fe7b006af00 2023-10-18T12:01:27.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7ae59c640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 0x7fe7b015bb00 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4036040 tx=0x7fe7a402db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:27.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe79effd640 1 -- 172.21.15.125:0/1318315459 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4003800 con 0x7fe7b006af00 2023-10-18T12:01:27.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b01f12a0 con 0x7fe7b006af00 2023-10-18T12:01:27.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe79effd640 1 -- 172.21.15.125:0/1318315459 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe7a4003a50 con 0x7fe7b006af00 2023-10-18T12:01:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b00b1dd0 con 0x7fe7b006af00 2023-10-18T12:01:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.910+0000 7fe79effd640 1 -- 172.21.15.125:0/1318315459 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4003de0 con 0x7fe7b006af00 2023-10-18T12:01:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.911+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b01f1100 con 0x7fe7b006af00 2023-10-18T12:01:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.912+0000 7fe79effd640 1 -- 172.21.15.125:0/1318315459 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe7a403e450 con 0x7fe7b006af00 2023-10-18T12:01:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.912+0000 7fe79effd640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe79006b080 0x7fe79006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.912+0000 7fe79effd640 1 -- 172.21.15.125:0/1318315459 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fe7a40ac980 con 0x7fe7b006af00 2023-10-18T12:01:27.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.912+0000 7fe7add9b640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe79006b080 0x7fe79006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:27.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.913+0000 7fe7add9b640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe79006b080 0x7fe79006d540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7b015c270 tx=0x7fe7a000b2e0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:27.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:27.916+0000 7fe79effd640 1 -- 172.21.15.125:0/1318315459 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7b01f1100 con 0x7fe7b006af00 2023-10-18T12:01:28.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.086+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7b00abed0 con 0x7fe79006b080 2023-10-18T12:01:28.102 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.099+0000 7fe79effd640 1 -- 172.21.15.125:0/1318315459 <== mgr.9236 v2:172.21.15.139:6832/3422518215 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529004 (secure 0 0 0) 0x7fe7a0015070 con 0x7fe79006b080 2023-10-18T12:01:28.108 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:28.111 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:01:28.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe79006b080 msgr2=0x7fe79006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:28.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe79006b080 0x7fe79006d540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7b015c270 tx=0x7fe7a000b2e0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.113 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 msgr2=0x7fe7b015bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:28.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 0x7fe7b015bb00 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4036040 tx=0x7fe7a402db20 comp rx=0 tx=0).stop 2023-10-18T12:01:28.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 shutdown_connections 2023-10-18T12:01:28.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fe79006b080 0x7fe79006d540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7b00c96b0 0x7fe7b00b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7b00ab500 0x7fe7b01f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 --2- 172.21.15.125:0/1318315459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7b006af00 0x7fe7b015bb00 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 >> 172.21.15.125:0/1318315459 conn(0x7fe7b0069930 msgr2=0x7fe7b00cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:28.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 shutdown_connections 2023-10-18T12:01:28.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.104+0000 7fe7b4ba9640 1 -- 172.21.15.125:0/1318315459 wait complete. 2023-10-18T12:01:28.117 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-10-18T12:01:26.840300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596444,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":727,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":459,"ondisk_log_size":459,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408224,"kb_used_data":25512,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754566496,"statfs":{"total":773094113280,"available":772676091904,"internally_reserved":0,"allocated":26124288,"data_stored":14007808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":391761464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.391821"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.877380+0000","last_change":"2023-10-18T12:01:20.463211+0000","last_active":"2023-10-18T12:01:21.877380+0000","last_peered":"2023-10-18T12:01:21.877380+0000","last_clean":"2023-10-18T12:01:21.877380+0000","last_became_active":"2023-10-18T12:01:20.462965+0000","last_became_peered":"2023-10-18T12:01:20.462965+0000","last_unstale":"2023-10-18T12:01:21.877380+0000","last_undegraded":"2023-10-18T12:01:21.877380+0000","last_fullsized":"2023-10-18T12:01:21.877380+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:35:36.730402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"245'1","reported_seq":13,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.473873+0000","last_change":"2023-10-18T12:01:19.464169+0000","last_active":"2023-10-18T12:01:20.473873+0000","last_peered":"2023-10-18T12:01:20.473873+0000","last_clean":"2023-10-18T12:01:20.473873+0000","last_became_active":"2023-10-18T12:01:19.463355+0000","last_became_peered":"2023-10-18T12:01:19.463355+0000","last_unstale":"2023-10-18T12:01:20.473873+0000","last_undegraded":"2023-10-18T12:01:20.473873+0000","last_fullsized":"2023-10-18T12:01:20.473873+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:10.580860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453819+0000","last_change":"2023-10-18T12:01:09.126968+0000","last_active":"2023-10-18T12:01:20.453819+0000","last_peered":"2023-10-18T12:01:20.453819+0000","last_clean":"2023-10-18T12:01:20.453819+0000","last_became_active":"2023-10-18T12:01:09.126705+0000","last_became_peered":"2023-10-18T12:01:09.126705+0000","last_unstale":"2023-10-18T12:01:20.453819+0000","last_undegraded":"2023-10-18T12:01:20.453819+0000","last_fullsized":"2023-10-18T12:01:20.453819+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:42:53.088272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.462232+0000","last_change":"2023-10-18T12:01:20.469676+0000","last_active":"2023-10-18T12:01:21.462232+0000","last_peered":"2023-10-18T12:01:21.462232+0000","last_clean":"2023-10-18T12:01:21.462232+0000","last_became_active":"2023-10-18T12:01:20.469199+0000","last_became_peered":"2023-10-18T12:01:20.469199+0000","last_unstale":"2023-10-18T12:01:21.462232+0000","last_undegraded":"2023-10-18T12:01:21.462232+0000","last_fullsized":"2023-10-18T12:01:21.462232+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:11.521234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"245'4","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.474415+0000","last_change":"2023-10-18T12:01:19.468191+0000","last_active":"2023-10-18T12:01:20.474415+0000","last_peered":"2023-10-18T12:01:20.474415+0000","last_clean":"2023-10-18T12:01:20.474415+0000","last_became_active":"2023-10-18T12:01:19.467925+0000","last_became_peered":"2023-10-18T12:01:19.467925+0000","last_unstale":"2023-10-18T12:01:20.474415+0000","last_undegraded":"2023-10-18T12:01:20.474415+0000","last_fullsized":"2023-10-18T12:01:20.474415+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:29:11.374513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.449937+0000","last_change":"2023-10-18T12:01:09.125772+0000","last_active":"2023-10-18T12:01:20.449937+0000","last_peered":"2023-10-18T12:01:20.449937+0000","last_clean":"2023-10-18T12:01:20.449937+0000","last_became_active":"2023-10-18T12:01:09.125508+0000","last_became_peered":"2023-10-18T12:01:09.125508+0000","last_unstale":"2023-10-18T12:01:20.449937+0000","last_undegraded":"2023-10-18T12:01:20.449937+0000","last_fullsized":"2023-10-18T12:01:20.449937+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:15.841764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.877943+0000","last_change":"2023-10-18T12:01:20.465549+0000","last_active":"2023-10-18T12:01:21.877943+0000","last_peered":"2023-10-18T12:01:21.877943+0000","last_clean":"2023-10-18T12:01:21.877943+0000","last_became_active":"2023-10-18T12:01:20.465299+0000","last_became_peered":"2023-10-18T12:01:20.465299+0000","last_unstale":"2023-10-18T12:01:21.877943+0000","last_undegraded":"2023-10-18T12:01:21.877943+0000","last_fullsized":"2023-10-18T12:01:21.877943+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:18.921856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"245'2","reported_seq":14,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.470731+0000","last_change":"2023-10-18T12:01:19.466687+0000","last_active":"2023-10-18T12:01:20.470731+0000","last_peered":"2023-10-18T12:01:20.470731+0000","last_clean":"2023-10-18T12:01:20.470731+0000","last_became_active":"2023-10-18T12:01:19.466395+0000","last_became_peered":"2023-10-18T12:01:19.466395+0000","last_unstale":"2023-10-18T12:01:20.470731+0000","last_undegraded":"2023-10-18T12:01:20.470731+0000","last_fullsized":"2023-10-18T12:01:20.470731+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:36.998390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453517+0000","last_change":"2023-10-18T12:01:09.124764+0000","last_active":"2023-10-18T12:01:20.453517+0000","last_peered":"2023-10-18T12:01:20.453517+0000","last_clean":"2023-10-18T12:01:20.453517+0000","last_became_active":"2023-10-18T12:01:09.124472+0000","last_became_peered":"2023-10-18T12:01:09.124472+0000","last_unstale":"2023-10-18T12:01:20.453517+0000","last_undegraded":"2023-10-18T12:01:20.453517+0000","last_fullsized":"2023-10-18T12:01:20.453517+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:29.891668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.877881+0000","last_change":"2023-10-18T12:01:20.476354+0000","last_active":"2023-10-18T12:01:21.877881+0000","last_peered":"2023-10-18T12:01:21.877881+0000","last_clean":"2023-10-18T12:01:21.877881+0000","last_became_active":"2023-10-18T12:01:20.476098+0000","last_became_peered":"2023-10-18T12:01:20.476098+0000","last_unstale":"2023-10-18T12:01:21.877881+0000","last_undegraded":"2023-10-18T12:01:21.877881+0000","last_fullsized":"2023-10-18T12:01:21.877881+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:48.905110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"245'2","reported_seq":14,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.471865+0000","last_change":"2023-10-18T12:01:19.466181+0000","last_active":"2023-10-18T12:01:20.471865+0000","last_peered":"2023-10-18T12:01:20.471865+0000","last_clean":"2023-10-18T12:01:20.471865+0000","last_became_active":"2023-10-18T12:01:19.465901+0000","last_became_peered":"2023-10-18T12:01:19.465901+0000","last_unstale":"2023-10-18T12:01:20.471865+0000","last_undegraded":"2023-10-18T12:01:20.471865+0000","last_fullsized":"2023-10-18T12:01:20.471865+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:10.214597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453226+0000","last_change":"2023-10-18T12:01:09.132202+0000","last_active":"2023-10-18T12:01:20.453226+0000","last_peered":"2023-10-18T12:01:20.453226+0000","last_clean":"2023-10-18T12:01:20.453226+0000","last_became_active":"2023-10-18T12:01:09.131928+0000","last_became_peered":"2023-10-18T12:01:09.131928+0000","last_unstale":"2023-10-18T12:01:20.453226+0000","last_undegraded":"2023-10-18T12:01:20.453226+0000","last_fullsized":"2023-10-18T12:01:20.453226+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:34.826098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876353+0000","last_change":"2023-10-18T12:01:20.468231+0000","last_active":"2023-10-18T12:01:21.876353+0000","last_peered":"2023-10-18T12:01:21.876353+0000","last_clean":"2023-10-18T12:01:21.876353+0000","last_became_active":"2023-10-18T12:01:20.467788+0000","last_became_peered":"2023-10-18T12:01:20.467788+0000","last_unstale":"2023-10-18T12:01:21.876353+0000","last_undegraded":"2023-10-18T12:01:21.876353+0000","last_fullsized":"2023-10-18T12:01:21.876353+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:11:03.653504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"245'1","reported_seq":13,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.478357+0000","last_change":"2023-10-18T12:01:19.459438+0000","last_active":"2023-10-18T12:01:20.478357+0000","last_peered":"2023-10-18T12:01:20.478357+0000","last_clean":"2023-10-18T12:01:20.478357+0000","last_became_active":"2023-10-18T12:01:19.459194+0000","last_became_peered":"2023-10-18T12:01:19.459194+0000","last_unstale":"2023-10-18T12:01:20.478357+0000","last_undegraded":"2023-10-18T12:01:20.478357+0000","last_fullsized":"2023-10-18T12:01:20.478357+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:48.130998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454788+0000","last_change":"2023-10-18T12:01:09.133870+0000","last_active":"2023-10-18T12:01:20.454788+0000","last_peered":"2023-10-18T12:01:20.454788+0000","last_clean":"2023-10-18T12:01:20.454788+0000","last_became_active":"2023-10-18T12:01:09.133478+0000","last_became_peered":"2023-10-18T12:01:09.133478+0000","last_unstale":"2023-10-18T12:01:20.454788+0000","last_undegraded":"2023-10-18T12:01:20.454788+0000","last_fullsized":"2023-10-18T12:01:20.454788+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:03:11.836101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876007+0000","last_change":"2023-10-18T12:01:20.465890+0000","last_active":"2023-10-18T12:01:21.876007+0000","last_peered":"2023-10-18T12:01:21.876007+0000","last_clean":"2023-10-18T12:01:21.876007+0000","last_became_active":"2023-10-18T12:01:20.465642+0000","last_became_peered":"2023-10-18T12:01:20.465642+0000","last_unstale":"2023-10-18T12:01:21.876007+0000","last_undegraded":"2023-10-18T12:01:21.876007+0000","last_fullsized":"2023-10-18T12:01:21.876007+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:53.822805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"245'1","reported_seq":13,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.468073+0000","last_change":"2023-10-18T12:01:19.463436+0000","last_active":"2023-10-18T12:01:20.468073+0000","last_peered":"2023-10-18T12:01:20.468073+0000","last_clean":"2023-10-18T12:01:20.468073+0000","last_became_active":"2023-10-18T12:01:19.463190+0000","last_became_peered":"2023-10-18T12:01:19.463190+0000","last_unstale":"2023-10-18T12:01:20.468073+0000","last_undegraded":"2023-10-18T12:01:20.468073+0000","last_fullsized":"2023-10-18T12:01:20.468073+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:53.933731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452297+0000","last_change":"2023-10-18T12:01:09.129113+0000","last_active":"2023-10-18T12:01:20.452297+0000","last_peered":"2023-10-18T12:01:20.452297+0000","last_clean":"2023-10-18T12:01:20.452297+0000","last_became_active":"2023-10-18T12:01:09.128785+0000","last_became_peered":"2023-10-18T12:01:09.128785+0000","last_unstale":"2023-10-18T12:01:20.452297+0000","last_undegraded":"2023-10-18T12:01:20.452297+0000","last_fullsized":"2023-10-18T12:01:20.452297+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:39.875471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876917+0000","last_change":"2023-10-18T12:01:20.476222+0000","last_active":"2023-10-18T12:01:21.876917+0000","last_peered":"2023-10-18T12:01:21.876917+0000","last_clean":"2023-10-18T12:01:21.876917+0000","last_became_active":"2023-10-18T12:01:20.475954+0000","last_became_peered":"2023-10-18T12:01:20.475954+0000","last_unstale":"2023-10-18T12:01:21.876917+0000","last_undegraded":"2023-10-18T12:01:21.876917+0000","last_fullsized":"2023-10-18T12:01:21.876917+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:25:50.213583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452592+0000","last_change":"2023-10-18T12:01:19.463023+0000","last_active":"2023-10-18T12:01:20.452592+0000","last_peered":"2023-10-18T12:01:20.452592+0000","last_clean":"2023-10-18T12:01:20.452592+0000","last_became_active":"2023-10-18T12:01:19.462757+0000","last_became_peered":"2023-10-18T12:01:19.462757+0000","last_unstale":"2023-10-18T12:01:20.452592+0000","last_undegraded":"2023-10-18T12:01:20.452592+0000","last_fullsized":"2023-10-18T12:01:20.452592+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:29.035447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.456765+0000","last_change":"2023-10-18T12:01:09.125901+0000","last_active":"2023-10-18T12:01:20.456765+0000","last_peered":"2023-10-18T12:01:20.456765+0000","last_clean":"2023-10-18T12:01:20.456765+0000","last_became_active":"2023-10-18T12:01:09.125390+0000","last_became_peered":"2023-10-18T12:01:09.125390+0000","last_unstale":"2023-10-18T12:01:20.456765+0000","last_undegraded":"2023-10-18T12:01:20.456765+0000","last_fullsized":"2023-10-18T12:01:20.456765+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:00.246587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.778420+0000","last_change":"2023-10-18T12:01:20.469264+0000","last_active":"2023-10-18T12:01:21.778420+0000","last_peered":"2023-10-18T12:01:21.778420+0000","last_clean":"2023-10-18T12:01:21.778420+0000","last_became_active":"2023-10-18T12:01:20.469011+0000","last_became_peered":"2023-10-18T12:01:20.469011+0000","last_unstale":"2023-10-18T12:01:21.778420+0000","last_undegraded":"2023-10-18T12:01:21.778420+0000","last_fullsized":"2023-10-18T12:01:21.778420+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:04.738807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453105+0000","last_change":"2023-10-18T12:01:19.463793+0000","last_active":"2023-10-18T12:01:20.453105+0000","last_peered":"2023-10-18T12:01:20.453105+0000","last_clean":"2023-10-18T12:01:20.453105+0000","last_became_active":"2023-10-18T12:01:19.463343+0000","last_became_peered":"2023-10-18T12:01:19.463343+0000","last_unstale":"2023-10-18T12:01:20.453105+0000","last_undegraded":"2023-10-18T12:01:20.453105+0000","last_fullsized":"2023-10-18T12:01:20.453105+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:24:56.236082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.455343+0000","last_change":"2023-10-18T12:01:09.135081+0000","last_active":"2023-10-18T12:01:20.455343+0000","last_peered":"2023-10-18T12:01:20.455343+0000","last_clean":"2023-10-18T12:01:20.455343+0000","last_became_active":"2023-10-18T12:01:09.134801+0000","last_became_peered":"2023-10-18T12:01:09.134801+0000","last_unstale":"2023-10-18T12:01:20.455343+0000","last_undegraded":"2023-10-18T12:01:20.455343+0000","last_fullsized":"2023-10-18T12:01:20.455343+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:03.262316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.877501+0000","last_change":"2023-10-18T12:01:20.466024+0000","last_active":"2023-10-18T12:01:21.877501+0000","last_peered":"2023-10-18T12:01:21.877501+0000","last_clean":"2023-10-18T12:01:21.877501+0000","last_became_active":"2023-10-18T12:01:20.465419+0000","last_became_peered":"2023-10-18T12:01:20.465419+0000","last_unstale":"2023-10-18T12:01:21.877501+0000","last_undegraded":"2023-10-18T12:01:21.877501+0000","last_fullsized":"2023-10-18T12:01:21.877501+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:26.653440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.458751+0000","last_change":"2023-10-18T12:01:19.465635+0000","last_active":"2023-10-18T12:01:20.458751+0000","last_peered":"2023-10-18T12:01:20.458751+0000","last_clean":"2023-10-18T12:01:20.458751+0000","last_became_active":"2023-10-18T12:01:19.465363+0000","last_became_peered":"2023-10-18T12:01:19.465363+0000","last_unstale":"2023-10-18T12:01:20.458751+0000","last_undegraded":"2023-10-18T12:01:20.458751+0000","last_fullsized":"2023-10-18T12:01:20.458751+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:48:12.803228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454010+0000","last_change":"2023-10-18T12:01:09.135615+0000","last_active":"2023-10-18T12:01:20.454010+0000","last_peered":"2023-10-18T12:01:20.454010+0000","last_clean":"2023-10-18T12:01:20.454010+0000","last_became_active":"2023-10-18T12:01:09.135123+0000","last_became_peered":"2023-10-18T12:01:09.135123+0000","last_unstale":"2023-10-18T12:01:20.454010+0000","last_undegraded":"2023-10-18T12:01:20.454010+0000","last_fullsized":"2023-10-18T12:01:20.454010+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:52.311354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876119+0000","last_change":"2023-10-18T12:01:20.465093+0000","last_active":"2023-10-18T12:01:21.876119+0000","last_peered":"2023-10-18T12:01:21.876119+0000","last_clean":"2023-10-18T12:01:21.876119+0000","last_became_active":"2023-10-18T12:01:20.464852+0000","last_became_peered":"2023-10-18T12:01:20.464852+0000","last_unstale":"2023-10-18T12:01:21.876119+0000","last_undegraded":"2023-10-18T12:01:21.876119+0000","last_fullsized":"2023-10-18T12:01:21.876119+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:06:55.096723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453113+0000","last_change":"2023-10-18T12:01:19.465001+0000","last_active":"2023-10-18T12:01:20.453113+0000","last_peered":"2023-10-18T12:01:20.453113+0000","last_clean":"2023-10-18T12:01:20.453113+0000","last_became_active":"2023-10-18T12:01:19.464719+0000","last_became_peered":"2023-10-18T12:01:19.464719+0000","last_unstale":"2023-10-18T12:01:20.453113+0000","last_undegraded":"2023-10-18T12:01:20.453113+0000","last_fullsized":"2023-10-18T12:01:20.453113+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:17.634838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452313+0000","last_change":"2023-10-18T12:01:09.130673+0000","last_active":"2023-10-18T12:01:20.452313+0000","last_peered":"2023-10-18T12:01:20.452313+0000","last_clean":"2023-10-18T12:01:20.452313+0000","last_became_active":"2023-10-18T12:01:09.130094+0000","last_became_peered":"2023-10-18T12:01:09.130094+0000","last_unstale":"2023-10-18T12:01:20.452313+0000","last_undegraded":"2023-10-18T12:01:20.452313+0000","last_fullsized":"2023-10-18T12:01:20.452313+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:58:45.652023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.628178+0000","last_change":"2023-10-18T12:01:20.457496+0000","last_active":"2023-10-18T12:01:21.628178+0000","last_peered":"2023-10-18T12:01:21.628178+0000","last_clean":"2023-10-18T12:01:21.628178+0000","last_became_active":"2023-10-18T12:01:20.457213+0000","last_became_peered":"2023-10-18T12:01:20.457213+0000","last_unstale":"2023-10-18T12:01:21.628178+0000","last_undegraded":"2023-10-18T12:01:21.628178+0000","last_fullsized":"2023-10-18T12:01:21.628178+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:55.644606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"245'6","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.475505+0000","last_change":"2023-10-18T12:01:19.463067+0000","last_active":"2023-10-18T12:01:20.475505+0000","last_peered":"2023-10-18T12:01:20.475505+0000","last_clean":"2023-10-18T12:01:20.475505+0000","last_became_active":"2023-10-18T12:01:19.462818+0000","last_became_peered":"2023-10-18T12:01:19.462818+0000","last_unstale":"2023-10-18T12:01:20.475505+0000","last_undegraded":"2023-10-18T12:01:20.475505+0000","last_fullsized":"2023-10-18T12:01:20.475505+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:35:48.838501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453590+0000","last_change":"2023-10-18T12:01:09.135828+0000","last_active":"2023-10-18T12:01:20.453590+0000","last_peered":"2023-10-18T12:01:20.453590+0000","last_clean":"2023-10-18T12:01:20.453590+0000","last_became_active":"2023-10-18T12:01:09.135579+0000","last_became_peered":"2023-10-18T12:01:09.135579+0000","last_unstale":"2023-10-18T12:01:20.453590+0000","last_undegraded":"2023-10-18T12:01:20.453590+0000","last_fullsized":"2023-10-18T12:01:20.453590+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:06.517585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876773+0000","last_change":"2023-10-18T12:01:20.472932+0000","last_active":"2023-10-18T12:01:21.876773+0000","last_peered":"2023-10-18T12:01:21.876773+0000","last_clean":"2023-10-18T12:01:21.876773+0000","last_became_active":"2023-10-18T12:01:20.472669+0000","last_became_peered":"2023-10-18T12:01:20.472669+0000","last_unstale":"2023-10-18T12:01:21.876773+0000","last_undegraded":"2023-10-18T12:01:21.876773+0000","last_fullsized":"2023-10-18T12:01:21.876773+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:50.822060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"245'3","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.476716+0000","last_change":"2023-10-18T12:01:19.462716+0000","last_active":"2023-10-18T12:01:20.476716+0000","last_peered":"2023-10-18T12:01:20.476716+0000","last_clean":"2023-10-18T12:01:20.476716+0000","last_became_active":"2023-10-18T12:01:19.462474+0000","last_became_peered":"2023-10-18T12:01:19.462474+0000","last_unstale":"2023-10-18T12:01:20.476716+0000","last_undegraded":"2023-10-18T12:01:20.476716+0000","last_fullsized":"2023-10-18T12:01:20.476716+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:51.379003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453868+0000","last_change":"2023-10-18T12:01:09.132939+0000","last_active":"2023-10-18T12:01:20.453868+0000","last_peered":"2023-10-18T12:01:20.453868+0000","last_clean":"2023-10-18T12:01:20.453868+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:20.453868+0000","last_undegraded":"2023-10-18T12:01:20.453868+0000","last_fullsized":"2023-10-18T12:01:20.453868+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:22.550156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.627290+0000","last_change":"2023-10-18T12:01:20.472297+0000","last_active":"2023-10-18T12:01:21.627290+0000","last_peered":"2023-10-18T12:01:21.627290+0000","last_clean":"2023-10-18T12:01:21.627290+0000","last_became_active":"2023-10-18T12:01:20.471977+0000","last_became_peered":"2023-10-18T12:01:20.471977+0000","last_unstale":"2023-10-18T12:01:21.627290+0000","last_undegraded":"2023-10-18T12:01:21.627290+0000","last_fullsized":"2023-10-18T12:01:21.627290+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:26.965792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452099+0000","last_change":"2023-10-18T12:01:19.458472+0000","last_active":"2023-10-18T12:01:20.452099+0000","last_peered":"2023-10-18T12:01:20.452099+0000","last_clean":"2023-10-18T12:01:20.452099+0000","last_became_active":"2023-10-18T12:01:19.458209+0000","last_became_peered":"2023-10-18T12:01:19.458209+0000","last_unstale":"2023-10-18T12:01:20.452099+0000","last_undegraded":"2023-10-18T12:01:20.452099+0000","last_fullsized":"2023-10-18T12:01:20.452099+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:29:34.708043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.449645+0000","last_change":"2023-10-18T12:01:09.129764+0000","last_active":"2023-10-18T12:01:20.449645+0000","last_peered":"2023-10-18T12:01:20.449645+0000","last_clean":"2023-10-18T12:01:20.449645+0000","last_became_active":"2023-10-18T12:01:09.129483+0000","last_became_peered":"2023-10-18T12:01:09.129483+0000","last_unstale":"2023-10-18T12:01:20.449645+0000","last_undegraded":"2023-10-18T12:01:20.449645+0000","last_fullsized":"2023-10-18T12:01:20.449645+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:20:03.037837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.460677+0000","last_change":"2023-10-18T12:01:20.468056+0000","last_active":"2023-10-18T12:01:21.460677+0000","last_peered":"2023-10-18T12:01:21.460677+0000","last_clean":"2023-10-18T12:01:21.460677+0000","last_became_active":"2023-10-18T12:01:20.467797+0000","last_became_peered":"2023-10-18T12:01:20.467797+0000","last_unstale":"2023-10-18T12:01:21.460677+0000","last_undegraded":"2023-10-18T12:01:21.460677+0000","last_fullsized":"2023-10-18T12:01:21.460677+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:18.072339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"245'2","reported_seq":14,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.466878+0000","last_change":"2023-10-18T12:01:19.466751+0000","last_active":"2023-10-18T12:01:20.466878+0000","last_peered":"2023-10-18T12:01:20.466878+0000","last_clean":"2023-10-18T12:01:20.466878+0000","last_became_active":"2023-10-18T12:01:19.466171+0000","last_became_peered":"2023-10-18T12:01:19.466171+0000","last_unstale":"2023-10-18T12:01:20.466878+0000","last_undegraded":"2023-10-18T12:01:20.466878+0000","last_fullsized":"2023-10-18T12:01:20.466878+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:29:50.331922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.460229+0000","last_change":"2023-10-18T12:01:09.133024+0000","last_active":"2023-10-18T12:01:20.460229+0000","last_peered":"2023-10-18T12:01:20.460229+0000","last_clean":"2023-10-18T12:01:20.460229+0000","last_became_active":"2023-10-18T12:01:09.132742+0000","last_became_peered":"2023-10-18T12:01:09.132742+0000","last_unstale":"2023-10-18T12:01:20.460229+0000","last_undegraded":"2023-10-18T12:01:20.460229+0000","last_fullsized":"2023-10-18T12:01:20.460229+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.460918+0000","last_change":"2023-10-18T12:01:20.469254+0000","last_active":"2023-10-18T12:01:21.460918+0000","last_peered":"2023-10-18T12:01:21.460918+0000","last_clean":"2023-10-18T12:01:21.460918+0000","last_became_active":"2023-10-18T12:01:20.468985+0000","last_became_peered":"2023-10-18T12:01:20.468985+0000","last_unstale":"2023-10-18T12:01:21.460918+0000","last_undegraded":"2023-10-18T12:01:21.460918+0000","last_fullsized":"2023-10-18T12:01:21.460918+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:03.375418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"245'8","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.479695+0000","last_change":"2023-10-18T12:01:19.464582+0000","last_active":"2023-10-18T12:01:20.479695+0000","last_peered":"2023-10-18T12:01:20.479695+0000","last_clean":"2023-10-18T12:01:20.479695+0000","last_became_active":"2023-10-18T12:01:19.464325+0000","last_became_peered":"2023-10-18T12:01:19.464325+0000","last_unstale":"2023-10-18T12:01:20.479695+0000","last_undegraded":"2023-10-18T12:01:20.479695+0000","last_fullsized":"2023-10-18T12:01:20.479695+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:55.127558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.458142+0000","last_change":"2023-10-18T12:01:09.131147+0000","last_active":"2023-10-18T12:01:20.458142+0000","last_peered":"2023-10-18T12:01:20.458142+0000","last_clean":"2023-10-18T12:01:20.458142+0000","last_became_active":"2023-10-18T12:01:09.130682+0000","last_became_peered":"2023-10-18T12:01:09.130682+0000","last_unstale":"2023-10-18T12:01:20.458142+0000","last_undegraded":"2023-10-18T12:01:20.458142+0000","last_fullsized":"2023-10-18T12:01:20.458142+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.460821+0000","last_change":"2023-10-18T12:01:20.457232+0000","last_active":"2023-10-18T12:01:21.460821+0000","last_peered":"2023-10-18T12:01:21.460821+0000","last_clean":"2023-10-18T12:01:21.460821+0000","last_became_active":"2023-10-18T12:01:20.456980+0000","last_became_peered":"2023-10-18T12:01:20.456980+0000","last_unstale":"2023-10-18T12:01:21.460821+0000","last_undegraded":"2023-10-18T12:01:21.460821+0000","last_fullsized":"2023-10-18T12:01:21.460821+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:35.495319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452433+0000","last_change":"2023-10-18T12:01:19.459426+0000","last_active":"2023-10-18T12:01:20.452433+0000","last_peered":"2023-10-18T12:01:20.452433+0000","last_clean":"2023-10-18T12:01:20.452433+0000","last_became_active":"2023-10-18T12:01:19.459168+0000","last_became_peered":"2023-10-18T12:01:19.459168+0000","last_unstale":"2023-10-18T12:01:20.452433+0000","last_undegraded":"2023-10-18T12:01:20.452433+0000","last_fullsized":"2023-10-18T12:01:20.452433+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:31:47.641324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452667+0000","last_change":"2023-10-18T12:01:09.122098+0000","last_active":"2023-10-18T12:01:20.452667+0000","last_peered":"2023-10-18T12:01:20.452667+0000","last_clean":"2023-10-18T12:01:20.452667+0000","last_became_active":"2023-10-18T12:01:09.121845+0000","last_became_peered":"2023-10-18T12:01:09.121845+0000","last_unstale":"2023-10-18T12:01:20.452667+0000","last_undegraded":"2023-10-18T12:01:20.452667+0000","last_fullsized":"2023-10-18T12:01:20.452667+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:19.003259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.628032+0000","last_change":"2023-10-18T12:01:20.461956+0000","last_active":"2023-10-18T12:01:21.628032+0000","last_peered":"2023-10-18T12:01:21.628032+0000","last_clean":"2023-10-18T12:01:21.628032+0000","last_became_active":"2023-10-18T12:01:20.461282+0000","last_became_peered":"2023-10-18T12:01:20.461282+0000","last_unstale":"2023-10-18T12:01:21.628032+0000","last_undegraded":"2023-10-18T12:01:21.628032+0000","last_fullsized":"2023-10-18T12:01:21.628032+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:18:30.946261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"245'2","reported_seq":14,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.471839+0000","last_change":"2023-10-18T12:01:19.466123+0000","last_active":"2023-10-18T12:01:20.471839+0000","last_peered":"2023-10-18T12:01:20.471839+0000","last_clean":"2023-10-18T12:01:20.471839+0000","last_became_active":"2023-10-18T12:01:19.465856+0000","last_became_peered":"2023-10-18T12:01:19.465856+0000","last_unstale":"2023-10-18T12:01:20.471839+0000","last_undegraded":"2023-10-18T12:01:20.471839+0000","last_fullsized":"2023-10-18T12:01:20.471839+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:37.219147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452256+0000","last_change":"2023-10-18T12:01:09.133999+0000","last_active":"2023-10-18T12:01:20.452256+0000","last_peered":"2023-10-18T12:01:20.452256+0000","last_clean":"2023-10-18T12:01:20.452256+0000","last_became_active":"2023-10-18T12:01:09.133725+0000","last_became_peered":"2023-10-18T12:01:09.133725+0000","last_unstale":"2023-10-18T12:01:20.452256+0000","last_undegraded":"2023-10-18T12:01:20.452256+0000","last_fullsized":"2023-10-18T12:01:20.452256+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:35.006911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.627613+0000","last_change":"2023-10-18T12:01:20.461649+0000","last_active":"2023-10-18T12:01:21.627613+0000","last_peered":"2023-10-18T12:01:21.627613+0000","last_clean":"2023-10-18T12:01:21.627613+0000","last_became_active":"2023-10-18T12:01:20.461369+0000","last_became_peered":"2023-10-18T12:01:20.461369+0000","last_unstale":"2023-10-18T12:01:21.627613+0000","last_undegraded":"2023-10-18T12:01:21.627613+0000","last_fullsized":"2023-10-18T12:01:21.627613+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:40.708144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"245'1","reported_seq":13,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.472475+0000","last_change":"2023-10-18T12:01:19.468308+0000","last_active":"2023-10-18T12:01:20.472475+0000","last_peered":"2023-10-18T12:01:20.472475+0000","last_clean":"2023-10-18T12:01:20.472475+0000","last_became_active":"2023-10-18T12:01:19.467998+0000","last_became_peered":"2023-10-18T12:01:19.467998+0000","last_unstale":"2023-10-18T12:01:20.472475+0000","last_undegraded":"2023-10-18T12:01:20.472475+0000","last_fullsized":"2023-10-18T12:01:20.472475+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:00.507459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452334+0000","last_change":"2023-10-18T12:01:09.124908+0000","last_active":"2023-10-18T12:01:20.452334+0000","last_peered":"2023-10-18T12:01:20.452334+0000","last_clean":"2023-10-18T12:01:20.452334+0000","last_became_active":"2023-10-18T12:01:09.124565+0000","last_became_peered":"2023-10-18T12:01:09.124565+0000","last_unstale":"2023-10-18T12:01:20.452334+0000","last_undegraded":"2023-10-18T12:01:20.452334+0000","last_fullsized":"2023-10-18T12:01:20.452334+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:20:22.044677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.628228+0000","last_change":"2023-10-18T12:01:20.470567+0000","last_active":"2023-10-18T12:01:21.628228+0000","last_peered":"2023-10-18T12:01:21.628228+0000","last_clean":"2023-10-18T12:01:21.628228+0000","last_became_active":"2023-10-18T12:01:20.470314+0000","last_became_peered":"2023-10-18T12:01:20.470314+0000","last_unstale":"2023-10-18T12:01:21.628228+0000","last_undegraded":"2023-10-18T12:01:21.628228+0000","last_fullsized":"2023-10-18T12:01:21.628228+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:57.859594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.448216+0000","last_change":"2023-10-18T12:01:19.467971+0000","last_active":"2023-10-18T12:01:20.448216+0000","last_peered":"2023-10-18T12:01:20.448216+0000","last_clean":"2023-10-18T12:01:20.448216+0000","last_became_active":"2023-10-18T12:01:19.467699+0000","last_became_peered":"2023-10-18T12:01:19.467699+0000","last_unstale":"2023-10-18T12:01:20.448216+0000","last_undegraded":"2023-10-18T12:01:20.448216+0000","last_fullsized":"2023-10-18T12:01:20.448216+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:02.924280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.459006+0000","last_change":"2023-10-18T12:01:09.135302+0000","last_active":"2023-10-18T12:01:20.459006+0000","last_peered":"2023-10-18T12:01:20.459006+0000","last_clean":"2023-10-18T12:01:20.459006+0000","last_became_active":"2023-10-18T12:01:09.134978+0000","last_became_peered":"2023-10-18T12:01:09.134978+0000","last_unstale":"2023-10-18T12:01:20.459006+0000","last_undegraded":"2023-10-18T12:01:20.459006+0000","last_fullsized":"2023-10-18T12:01:20.459006+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876538+0000","last_change":"2023-10-18T12:01:20.458031+0000","last_active":"2023-10-18T12:01:21.876538+0000","last_peered":"2023-10-18T12:01:21.876538+0000","last_clean":"2023-10-18T12:01:21.876538+0000","last_became_active":"2023-10-18T12:01:20.457764+0000","last_became_peered":"2023-10-18T12:01:20.457764+0000","last_unstale":"2023-10-18T12:01:21.876538+0000","last_undegraded":"2023-10-18T12:01:21.876538+0000","last_fullsized":"2023-10-18T12:01:21.876538+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:54:51.698365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"245'1","reported_seq":13,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.471259+0000","last_change":"2023-10-18T12:01:19.461558+0000","last_active":"2023-10-18T12:01:20.471259+0000","last_peered":"2023-10-18T12:01:20.471259+0000","last_clean":"2023-10-18T12:01:20.471259+0000","last_became_active":"2023-10-18T12:01:19.461307+0000","last_became_peered":"2023-10-18T12:01:19.461307+0000","last_unstale":"2023-10-18T12:01:20.471259+0000","last_undegraded":"2023-10-18T12:01:20.471259+0000","last_fullsized":"2023-10-18T12:01:20.471259+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:41:45.160371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.448482+0000","last_change":"2023-10-18T12:01:09.136717+0000","last_active":"2023-10-18T12:01:20.448482+0000","last_peered":"2023-10-18T12:01:20.448482+0000","last_clean":"2023-10-18T12:01:20.448482+0000","last_became_active":"2023-10-18T12:01:09.136461+0000","last_became_peered":"2023-10-18T12:01:09.136461+0000","last_unstale":"2023-10-18T12:01:20.448482+0000","last_undegraded":"2023-10-18T12:01:20.448482+0000","last_fullsized":"2023-10-18T12:01:20.448482+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:02.336311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.627426+0000","last_change":"2023-10-18T12:01:20.472919+0000","last_active":"2023-10-18T12:01:21.627426+0000","last_peered":"2023-10-18T12:01:21.627426+0000","last_clean":"2023-10-18T12:01:21.627426+0000","last_became_active":"2023-10-18T12:01:20.472672+0000","last_became_peered":"2023-10-18T12:01:20.472672+0000","last_unstale":"2023-10-18T12:01:21.627426+0000","last_undegraded":"2023-10-18T12:01:21.627426+0000","last_fullsized":"2023-10-18T12:01:21.627426+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:20:00.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453060+0000","last_change":"2023-10-18T12:01:19.462240+0000","last_active":"2023-10-18T12:01:20.453060+0000","last_peered":"2023-10-18T12:01:20.453060+0000","last_clean":"2023-10-18T12:01:20.453060+0000","last_became_active":"2023-10-18T12:01:19.461957+0000","last_became_peered":"2023-10-18T12:01:19.461957+0000","last_unstale":"2023-10-18T12:01:20.453060+0000","last_undegraded":"2023-10-18T12:01:20.453060+0000","last_fullsized":"2023-10-18T12:01:20.453060+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:53.432181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451531+0000","last_change":"2023-10-18T12:01:09.130016+0000","last_active":"2023-10-18T12:01:20.451531+0000","last_peered":"2023-10-18T12:01:20.451531+0000","last_clean":"2023-10-18T12:01:20.451531+0000","last_became_active":"2023-10-18T12:01:09.129718+0000","last_became_peered":"2023-10-18T12:01:09.129718+0000","last_unstale":"2023-10-18T12:01:20.451531+0000","last_undegraded":"2023-10-18T12:01:20.451531+0000","last_fullsized":"2023-10-18T12:01:20.451531+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:36.455465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876091+0000","last_change":"2023-10-18T12:01:20.465414+0000","last_active":"2023-10-18T12:01:21.876091+0000","last_peered":"2023-10-18T12:01:21.876091+0000","last_clean":"2023-10-18T12:01:21.876091+0000","last_became_active":"2023-10-18T12:01:20.465172+0000","last_became_peered":"2023-10-18T12:01:20.465172+0000","last_unstale":"2023-10-18T12:01:21.876091+0000","last_undegraded":"2023-10-18T12:01:21.876091+0000","last_fullsized":"2023-10-18T12:01:21.876091+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:19:33.892306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"245'2","reported_seq":14,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.463122+0000","last_change":"2023-10-18T12:01:19.467158+0000","last_active":"2023-10-18T12:01:20.463122+0000","last_peered":"2023-10-18T12:01:20.463122+0000","last_clean":"2023-10-18T12:01:20.463122+0000","last_became_active":"2023-10-18T12:01:19.466854+0000","last_became_peered":"2023-10-18T12:01:19.466854+0000","last_unstale":"2023-10-18T12:01:20.463122+0000","last_undegraded":"2023-10-18T12:01:20.463122+0000","last_fullsized":"2023-10-18T12:01:20.463122+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:27.878015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.449448+0000","last_change":"2023-10-18T12:01:09.129706+0000","last_active":"2023-10-18T12:01:20.449448+0000","last_peered":"2023-10-18T12:01:20.449448+0000","last_clean":"2023-10-18T12:01:20.449448+0000","last_became_active":"2023-10-18T12:01:09.129413+0000","last_became_peered":"2023-10-18T12:01:09.129413+0000","last_unstale":"2023-10-18T12:01:20.449448+0000","last_undegraded":"2023-10-18T12:01:20.449448+0000","last_fullsized":"2023-10-18T12:01:20.449448+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:55.309809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.877139+0000","last_change":"2023-10-18T12:01:20.468957+0000","last_active":"2023-10-18T12:01:21.877139+0000","last_peered":"2023-10-18T12:01:21.877139+0000","last_clean":"2023-10-18T12:01:21.877139+0000","last_became_active":"2023-10-18T12:01:20.468718+0000","last_became_peered":"2023-10-18T12:01:20.468718+0000","last_unstale":"2023-10-18T12:01:21.877139+0000","last_undegraded":"2023-10-18T12:01:21.877139+0000","last_fullsized":"2023-10-18T12:01:21.877139+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:43.415976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454659+0000","last_change":"2023-10-18T12:01:19.467623+0000","last_active":"2023-10-18T12:01:20.454659+0000","last_peered":"2023-10-18T12:01:20.454659+0000","last_clean":"2023-10-18T12:01:20.454659+0000","last_became_active":"2023-10-18T12:01:19.467358+0000","last_became_peered":"2023-10-18T12:01:19.467358+0000","last_unstale":"2023-10-18T12:01:20.454659+0000","last_undegraded":"2023-10-18T12:01:20.454659+0000","last_fullsized":"2023-10-18T12:01:20.454659+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:06.064432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.458016+0000","last_change":"2023-10-18T12:01:09.133291+0000","last_active":"2023-10-18T12:01:20.458016+0000","last_peered":"2023-10-18T12:01:20.458016+0000","last_clean":"2023-10-18T12:01:20.458016+0000","last_became_active":"2023-10-18T12:01:09.132968+0000","last_became_peered":"2023-10-18T12:01:09.132968+0000","last_unstale":"2023-10-18T12:01:20.458016+0000","last_undegraded":"2023-10-18T12:01:20.458016+0000","last_fullsized":"2023-10-18T12:01:20.458016+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876079+0000","last_change":"2023-10-18T12:01:20.466262+0000","last_active":"2023-10-18T12:01:21.876079+0000","last_peered":"2023-10-18T12:01:21.876079+0000","last_clean":"2023-10-18T12:01:21.876079+0000","last_became_active":"2023-10-18T12:01:20.465825+0000","last_became_peered":"2023-10-18T12:01:20.465825+0000","last_unstale":"2023-10-18T12:01:21.876079+0000","last_undegraded":"2023-10-18T12:01:21.876079+0000","last_fullsized":"2023-10-18T12:01:21.876079+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:32:51.819777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.460218+0000","last_change":"2023-10-18T12:01:19.458380+0000","last_active":"2023-10-18T12:01:20.460218+0000","last_peered":"2023-10-18T12:01:20.460218+0000","last_clean":"2023-10-18T12:01:20.460218+0000","last_became_active":"2023-10-18T12:01:19.458111+0000","last_became_peered":"2023-10-18T12:01:19.458111+0000","last_unstale":"2023-10-18T12:01:20.460218+0000","last_undegraded":"2023-10-18T12:01:20.460218+0000","last_fullsized":"2023-10-18T12:01:20.460218+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:36.884496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451251+0000","last_change":"2023-10-18T12:01:09.128954+0000","last_active":"2023-10-18T12:01:20.451251+0000","last_peered":"2023-10-18T12:01:20.451251+0000","last_clean":"2023-10-18T12:01:20.451251+0000","last_became_active":"2023-10-18T12:01:09.128655+0000","last_became_peered":"2023-10-18T12:01:09.128655+0000","last_unstale":"2023-10-18T12:01:20.451251+0000","last_undegraded":"2023-10-18T12:01:20.451251+0000","last_fullsized":"2023-10-18T12:01:20.451251+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:29:51.625573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876589+0000","last_change":"2023-10-18T12:01:20.474717+0000","last_active":"2023-10-18T12:01:21.876589+0000","last_peered":"2023-10-18T12:01:21.876589+0000","last_clean":"2023-10-18T12:01:21.876589+0000","last_became_active":"2023-10-18T12:01:20.474475+0000","last_became_peered":"2023-10-18T12:01:20.474475+0000","last_unstale":"2023-10-18T12:01:21.876589+0000","last_undegraded":"2023-10-18T12:01:21.876589+0000","last_fullsized":"2023-10-18T12:01:21.876589+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:49.138874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"246'1","reported_seq":13,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.878046+0000","last_change":"2023-10-18T12:01:19.466027+0000","last_active":"2023-10-18T12:01:21.878046+0000","last_peered":"2023-10-18T12:01:21.878046+0000","last_clean":"2023-10-18T12:01:21.878046+0000","last_became_active":"2023-10-18T12:01:19.465745+0000","last_became_peered":"2023-10-18T12:01:19.465745+0000","last_unstale":"2023-10-18T12:01:21.878046+0000","last_undegraded":"2023-10-18T12:01:21.878046+0000","last_fullsized":"2023-10-18T12:01:21.878046+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:12.807807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.449366+0000","last_change":"2023-10-18T12:01:09.123355+0000","last_active":"2023-10-18T12:01:20.449366+0000","last_peered":"2023-10-18T12:01:20.449366+0000","last_clean":"2023-10-18T12:01:20.449366+0000","last_became_active":"2023-10-18T12:01:09.123096+0000","last_became_peered":"2023-10-18T12:01:09.123096+0000","last_unstale":"2023-10-18T12:01:20.449366+0000","last_undegraded":"2023-10-18T12:01:20.449366+0000","last_fullsized":"2023-10-18T12:01:20.449366+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:13:58.130314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876875+0000","last_change":"2023-10-18T12:01:20.467273+0000","last_active":"2023-10-18T12:01:21.876875+0000","last_peered":"2023-10-18T12:01:21.876875+0000","last_clean":"2023-10-18T12:01:21.876875+0000","last_became_active":"2023-10-18T12:01:20.467019+0000","last_became_peered":"2023-10-18T12:01:20.467019+0000","last_unstale":"2023-10-18T12:01:21.876875+0000","last_undegraded":"2023-10-18T12:01:21.876875+0000","last_fullsized":"2023-10-18T12:01:21.876875+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:14.644930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"245'2","reported_seq":14,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.474160+0000","last_change":"2023-10-18T12:01:19.467893+0000","last_active":"2023-10-18T12:01:20.474160+0000","last_peered":"2023-10-18T12:01:20.474160+0000","last_clean":"2023-10-18T12:01:20.474160+0000","last_became_active":"2023-10-18T12:01:19.467430+0000","last_became_peered":"2023-10-18T12:01:19.467430+0000","last_unstale":"2023-10-18T12:01:20.474160+0000","last_undegraded":"2023-10-18T12:01:20.474160+0000","last_fullsized":"2023-10-18T12:01:20.474160+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:50:40.670980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452386+0000","last_change":"2023-10-18T12:01:09.124947+0000","last_active":"2023-10-18T12:01:20.452386+0000","last_peered":"2023-10-18T12:01:20.452386+0000","last_clean":"2023-10-18T12:01:20.452386+0000","last_became_active":"2023-10-18T12:01:09.124666+0000","last_became_peered":"2023-10-18T12:01:09.124666+0000","last_unstale":"2023-10-18T12:01:20.452386+0000","last_undegraded":"2023-10-18T12:01:20.452386+0000","last_fullsized":"2023-10-18T12:01:20.452386+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:01.214167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876944+0000","last_change":"2023-10-18T12:01:20.469294+0000","last_active":"2023-10-18T12:01:21.876944+0000","last_peered":"2023-10-18T12:01:21.876944+0000","last_clean":"2023-10-18T12:01:21.876944+0000","last_became_active":"2023-10-18T12:01:20.469019+0000","last_became_peered":"2023-10-18T12:01:20.469019+0000","last_unstale":"2023-10-18T12:01:21.876944+0000","last_undegraded":"2023-10-18T12:01:21.876944+0000","last_fullsized":"2023-10-18T12:01:21.876944+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:53.822805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.456055+0000","last_change":"2023-10-18T12:01:19.462049+0000","last_active":"2023-10-18T12:01:20.456055+0000","last_peered":"2023-10-18T12:01:20.456055+0000","last_clean":"2023-10-18T12:01:20.456055+0000","last_became_active":"2023-10-18T12:01:19.460286+0000","last_became_peered":"2023-10-18T12:01:19.460286+0000","last_unstale":"2023-10-18T12:01:20.456055+0000","last_undegraded":"2023-10-18T12:01:20.456055+0000","last_fullsized":"2023-10-18T12:01:20.456055+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:18:07.352120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454718+0000","last_change":"2023-10-18T12:01:09.131674+0000","last_active":"2023-10-18T12:01:20.454718+0000","last_peered":"2023-10-18T12:01:20.454718+0000","last_clean":"2023-10-18T12:01:20.454718+0000","last_became_active":"2023-10-18T12:01:09.131410+0000","last_became_peered":"2023-10-18T12:01:09.131410+0000","last_unstale":"2023-10-18T12:01:20.454718+0000","last_undegraded":"2023-10-18T12:01:20.454718+0000","last_fullsized":"2023-10-18T12:01:20.454718+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:01:22.606198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.778724+0000","last_change":"2023-10-18T12:01:20.465448+0000","last_active":"2023-10-18T12:01:21.778724+0000","last_peered":"2023-10-18T12:01:21.778724+0000","last_clean":"2023-10-18T12:01:21.778724+0000","last_became_active":"2023-10-18T12:01:20.465209+0000","last_became_peered":"2023-10-18T12:01:20.465209+0000","last_unstale":"2023-10-18T12:01:21.778724+0000","last_undegraded":"2023-10-18T12:01:21.778724+0000","last_fullsized":"2023-10-18T12:01:21.778724+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:09.473251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452313+0000","last_change":"2023-10-18T12:01:19.460871+0000","last_active":"2023-10-18T12:01:20.452313+0000","last_peered":"2023-10-18T12:01:20.452313+0000","last_clean":"2023-10-18T12:01:20.452313+0000","last_became_active":"2023-10-18T12:01:19.460570+0000","last_became_peered":"2023-10-18T12:01:19.460570+0000","last_unstale":"2023-10-18T12:01:20.452313+0000","last_undegraded":"2023-10-18T12:01:20.452313+0000","last_fullsized":"2023-10-18T12:01:20.452313+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:19.192328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453954+0000","last_change":"2023-10-18T12:01:09.134865+0000","last_active":"2023-10-18T12:01:20.453954+0000","last_peered":"2023-10-18T12:01:20.453954+0000","last_clean":"2023-10-18T12:01:20.453954+0000","last_became_active":"2023-10-18T12:01:09.134583+0000","last_became_peered":"2023-10-18T12:01:09.134583+0000","last_unstale":"2023-10-18T12:01:20.453954+0000","last_undegraded":"2023-10-18T12:01:20.453954+0000","last_fullsized":"2023-10-18T12:01:20.453954+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:42.480656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876465+0000","last_change":"2023-10-18T12:01:20.478140+0000","last_active":"2023-10-18T12:01:21.876465+0000","last_peered":"2023-10-18T12:01:21.876465+0000","last_clean":"2023-10-18T12:01:21.876465+0000","last_became_active":"2023-10-18T12:01:20.477787+0000","last_became_peered":"2023-10-18T12:01:20.477787+0000","last_unstale":"2023-10-18T12:01:21.876465+0000","last_undegraded":"2023-10-18T12:01:21.876465+0000","last_fullsized":"2023-10-18T12:01:21.876465+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:43:29.203805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.447790+0000","last_change":"2023-10-18T12:01:19.461910+0000","last_active":"2023-10-18T12:01:20.447790+0000","last_peered":"2023-10-18T12:01:20.447790+0000","last_clean":"2023-10-18T12:01:20.447790+0000","last_became_active":"2023-10-18T12:01:19.461653+0000","last_became_peered":"2023-10-18T12:01:19.461653+0000","last_unstale":"2023-10-18T12:01:20.447790+0000","last_undegraded":"2023-10-18T12:01:20.447790+0000","last_fullsized":"2023-10-18T12:01:20.447790+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:11.377805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.460522+0000","last_change":"2023-10-18T12:01:09.132312+0000","last_active":"2023-10-18T12:01:20.460522+0000","last_peered":"2023-10-18T12:01:20.460522+0000","last_clean":"2023-10-18T12:01:20.460522+0000","last_became_active":"2023-10-18T12:01:09.131781+0000","last_became_peered":"2023-10-18T12:01:09.131781+0000","last_unstale":"2023-10-18T12:01:20.460522+0000","last_undegraded":"2023-10-18T12:01:20.460522+0000","last_fullsized":"2023-10-18T12:01:20.460522+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.875750+0000","last_change":"2023-10-18T12:01:20.467342+0000","last_active":"2023-10-18T12:01:21.875750+0000","last_peered":"2023-10-18T12:01:21.875750+0000","last_clean":"2023-10-18T12:01:21.875750+0000","last_became_active":"2023-10-18T12:01:20.462061+0000","last_became_peered":"2023-10-18T12:01:20.462061+0000","last_unstale":"2023-10-18T12:01:21.875750+0000","last_undegraded":"2023-10-18T12:01:21.875750+0000","last_fullsized":"2023-10-18T12:01:21.875750+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:18:40.353231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.448022+0000","last_change":"2023-10-18T12:01:19.465117+0000","last_active":"2023-10-18T12:01:20.448022+0000","last_peered":"2023-10-18T12:01:20.448022+0000","last_clean":"2023-10-18T12:01:20.448022+0000","last_became_active":"2023-10-18T12:01:19.464845+0000","last_became_peered":"2023-10-18T12:01:19.464845+0000","last_unstale":"2023-10-18T12:01:20.448022+0000","last_undegraded":"2023-10-18T12:01:20.448022+0000","last_fullsized":"2023-10-18T12:01:20.448022+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:18.497954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.460336+0000","last_change":"2023-10-18T12:01:09.133664+0000","last_active":"2023-10-18T12:01:20.460336+0000","last_peered":"2023-10-18T12:01:20.460336+0000","last_clean":"2023-10-18T12:01:20.460336+0000","last_became_active":"2023-10-18T12:01:09.133406+0000","last_became_peered":"2023-10-18T12:01:09.133406+0000","last_unstale":"2023-10-18T12:01:20.460336+0000","last_undegraded":"2023-10-18T12:01:20.460336+0000","last_fullsized":"2023-10-18T12:01:20.460336+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:57:22.305729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.462037+0000","last_change":"2023-10-18T12:01:20.468350+0000","last_active":"2023-10-18T12:01:21.462037+0000","last_peered":"2023-10-18T12:01:21.462037+0000","last_clean":"2023-10-18T12:01:21.462037+0000","last_became_active":"2023-10-18T12:01:20.468053+0000","last_became_peered":"2023-10-18T12:01:20.468053+0000","last_unstale":"2023-10-18T12:01:21.462037+0000","last_undegraded":"2023-10-18T12:01:21.462037+0000","last_fullsized":"2023-10-18T12:01:21.462037+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:52.234991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.447595+0000","last_change":"2023-10-18T12:01:19.458515+0000","last_active":"2023-10-18T12:01:20.447595+0000","last_peered":"2023-10-18T12:01:20.447595+0000","last_clean":"2023-10-18T12:01:20.447595+0000","last_became_active":"2023-10-18T12:01:19.457749+0000","last_became_peered":"2023-10-18T12:01:19.457749+0000","last_unstale":"2023-10-18T12:01:20.447595+0000","last_undegraded":"2023-10-18T12:01:20.447595+0000","last_fullsized":"2023-10-18T12:01:20.447595+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:57:19.477093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.448094+0000","last_change":"2023-10-18T12:01:09.130142+0000","last_active":"2023-10-18T12:01:20.448094+0000","last_peered":"2023-10-18T12:01:20.448094+0000","last_clean":"2023-10-18T12:01:20.448094+0000","last_became_active":"2023-10-18T12:01:09.129884+0000","last_became_peered":"2023-10-18T12:01:09.129884+0000","last_unstale":"2023-10-18T12:01:20.448094+0000","last_undegraded":"2023-10-18T12:01:20.448094+0000","last_fullsized":"2023-10-18T12:01:20.448094+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:36.037273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876750+0000","last_change":"2023-10-18T12:01:20.476864+0000","last_active":"2023-10-18T12:01:21.876750+0000","last_peered":"2023-10-18T12:01:21.876750+0000","last_clean":"2023-10-18T12:01:21.876750+0000","last_became_active":"2023-10-18T12:01:20.476575+0000","last_became_peered":"2023-10-18T12:01:20.476575+0000","last_unstale":"2023-10-18T12:01:21.876750+0000","last_undegraded":"2023-10-18T12:01:21.876750+0000","last_fullsized":"2023-10-18T12:01:21.876750+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:34.179199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.456676+0000","last_change":"2023-10-18T12:01:19.458409+0000","last_active":"2023-10-18T12:01:20.456676+0000","last_peered":"2023-10-18T12:01:20.456676+0000","last_clean":"2023-10-18T12:01:20.456676+0000","last_became_active":"2023-10-18T12:01:19.458151+0000","last_became_peered":"2023-10-18T12:01:19.458151+0000","last_unstale":"2023-10-18T12:01:20.456676+0000","last_undegraded":"2023-10-18T12:01:20.456676+0000","last_fullsized":"2023-10-18T12:01:20.456676+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:18.427831+0000","last_clean_scrub_stamp":"2023-10-18T12:01:18.427831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:57.352320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454842+0000","last_change":"2023-10-18T12:01:09.136066+0000","last_active":"2023-10-18T12:01:20.454842+0000","last_peered":"2023-10-18T12:01:20.454842+0000","last_clean":"2023-10-18T12:01:20.454842+0000","last_became_active":"2023-10-18T12:01:09.135802+0000","last_became_peered":"2023-10-18T12:01:09.135802+0000","last_unstale":"2023-10-18T12:01:20.454842+0000","last_undegraded":"2023-10-18T12:01:20.454842+0000","last_fullsized":"2023-10-18T12:01:20.454842+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:26.550223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876223+0000","last_change":"2023-10-18T12:01:20.472296+0000","last_active":"2023-10-18T12:01:21.876223+0000","last_peered":"2023-10-18T12:01:21.876223+0000","last_clean":"2023-10-18T12:01:21.876223+0000","last_became_active":"2023-10-18T12:01:20.472043+0000","last_became_peered":"2023-10-18T12:01:20.472043+0000","last_unstale":"2023-10-18T12:01:21.876223+0000","last_undegraded":"2023-10-18T12:01:21.876223+0000","last_fullsized":"2023-10-18T12:01:21.876223+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:49.429201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.447781+0000","last_change":"2023-10-18T12:01:09.120812+0000","last_active":"2023-10-18T12:01:20.447781+0000","last_peered":"2023-10-18T12:01:20.447781+0000","last_clean":"2023-10-18T12:01:20.447781+0000","last_became_active":"2023-10-18T12:01:09.120561+0000","last_became_peered":"2023-10-18T12:01:09.120561+0000","last_unstale":"2023-10-18T12:01:20.447781+0000","last_undegraded":"2023-10-18T12:01:20.447781+0000","last_fullsized":"2023-10-18T12:01:20.447781+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:02.277242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"241'4","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453914+0000","last_change":"2023-10-18T12:01:08.112445+0000","last_active":"2023-10-18T12:01:20.453914+0000","last_peered":"2023-10-18T12:01:20.453914+0000","last_clean":"2023-10-18T12:01:20.453914+0000","last_became_active":"2023-10-18T12:01:08.112017+0000","last_became_peered":"2023-10-18T12:01:08.112017+0000","last_unstale":"2023-10-18T12:01:20.453914+0000","last_undegraded":"2023-10-18T12:01:20.453914+0000","last_fullsized":"2023-10-18T12:01:20.453914+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:11.607481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.875955+0000","last_change":"2023-10-18T12:01:20.469600+0000","last_active":"2023-10-18T12:01:21.875955+0000","last_peered":"2023-10-18T12:01:21.875955+0000","last_clean":"2023-10-18T12:01:21.875955+0000","last_became_active":"2023-10-18T12:01:20.469334+0000","last_became_peered":"2023-10-18T12:01:20.469334+0000","last_unstale":"2023-10-18T12:01:21.875955+0000","last_undegraded":"2023-10-18T12:01:21.875955+0000","last_fullsized":"2023-10-18T12:01:21.875955+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:51.217620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.462484+0000","last_change":"2023-10-18T12:01:09.134916+0000","last_active":"2023-10-18T12:01:20.462484+0000","last_peered":"2023-10-18T12:01:20.462484+0000","last_clean":"2023-10-18T12:01:20.462484+0000","last_became_active":"2023-10-18T12:01:09.134630+0000","last_became_peered":"2023-10-18T12:01:09.134630+0000","last_unstale":"2023-10-18T12:01:20.462484+0000","last_undegraded":"2023-10-18T12:01:20.462484+0000","last_fullsized":"2023-10-18T12:01:20.462484+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:38.619253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"241'2","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451110+0000","last_change":"2023-10-18T12:01:08.119870+0000","last_active":"2023-10-18T12:01:20.451110+0000","last_peered":"2023-10-18T12:01:20.451110+0000","last_clean":"2023-10-18T12:01:20.451110+0000","last_became_active":"2023-10-18T12:01:08.119586+0000","last_became_peered":"2023-10-18T12:01:08.119586+0000","last_unstale":"2023-10-18T12:01:20.451110+0000","last_undegraded":"2023-10-18T12:01:20.451110+0000","last_fullsized":"2023-10-18T12:01:20.451110+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:16.288493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876240+0000","last_change":"2023-10-18T12:01:20.478139+0000","last_active":"2023-10-18T12:01:21.876240+0000","last_peered":"2023-10-18T12:01:21.876240+0000","last_clean":"2023-10-18T12:01:21.876240+0000","last_became_active":"2023-10-18T12:01:20.477802+0000","last_became_peered":"2023-10-18T12:01:20.477802+0000","last_unstale":"2023-10-18T12:01:21.876240+0000","last_undegraded":"2023-10-18T12:01:21.876240+0000","last_fullsized":"2023-10-18T12:01:21.876240+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:43:04.430421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.455771+0000","last_change":"2023-10-18T12:01:09.130894+0000","last_active":"2023-10-18T12:01:20.455771+0000","last_peered":"2023-10-18T12:01:20.455771+0000","last_clean":"2023-10-18T12:01:20.455771+0000","last_became_active":"2023-10-18T12:01:09.130614+0000","last_became_peered":"2023-10-18T12:01:09.130614+0000","last_unstale":"2023-10-18T12:01:20.455771+0000","last_undegraded":"2023-10-18T12:01:20.455771+0000","last_fullsized":"2023-10-18T12:01:20.455771+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:03.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"241'2","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.463291+0000","last_change":"2023-10-18T12:01:08.115174+0000","last_active":"2023-10-18T12:01:20.463291+0000","last_peered":"2023-10-18T12:01:20.463291+0000","last_clean":"2023-10-18T12:01:20.463291+0000","last_became_active":"2023-10-18T12:01:08.114546+0000","last_became_peered":"2023-10-18T12:01:08.114546+0000","last_unstale":"2023-10-18T12:01:20.463291+0000","last_undegraded":"2023-10-18T12:01:20.463291+0000","last_fullsized":"2023-10-18T12:01:20.463291+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.877805+0000","last_change":"2023-10-18T12:01:20.464627+0000","last_active":"2023-10-18T12:01:21.877805+0000","last_peered":"2023-10-18T12:01:21.877805+0000","last_clean":"2023-10-18T12:01:21.877805+0000","last_became_active":"2023-10-18T12:01:20.463912+0000","last_became_peered":"2023-10-18T12:01:20.463912+0000","last_unstale":"2023-10-18T12:01:21.877805+0000","last_undegraded":"2023-10-18T12:01:21.877805+0000","last_fullsized":"2023-10-18T12:01:21.877805+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:45:43.629383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.463207+0000","last_change":"2023-10-18T12:01:09.122601+0000","last_active":"2023-10-18T12:01:20.463207+0000","last_peered":"2023-10-18T12:01:20.463207+0000","last_clean":"2023-10-18T12:01:20.463207+0000","last_became_active":"2023-10-18T12:01:09.122334+0000","last_became_peered":"2023-10-18T12:01:09.122334+0000","last_unstale":"2023-10-18T12:01:20.463207+0000","last_undegraded":"2023-10-18T12:01:20.463207+0000","last_fullsized":"2023-10-18T12:01:20.463207+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:51.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"241'1","reported_seq":17,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.458740+0000","last_change":"2023-10-18T12:01:08.116478+0000","last_active":"2023-10-18T12:01:20.458740+0000","last_peered":"2023-10-18T12:01:20.458740+0000","last_clean":"2023-10-18T12:01:20.458740+0000","last_became_active":"2023-10-18T12:01:08.116181+0000","last_became_peered":"2023-10-18T12:01:08.116181+0000","last_unstale":"2023-10-18T12:01:20.458740+0000","last_undegraded":"2023-10-18T12:01:20.458740+0000","last_fullsized":"2023-10-18T12:01:20.458740+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:10.087064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.778823+0000","last_change":"2023-10-18T12:01:20.473644+0000","last_active":"2023-10-18T12:01:21.778823+0000","last_peered":"2023-10-18T12:01:21.778823+0000","last_clean":"2023-10-18T12:01:21.778823+0000","last_became_active":"2023-10-18T12:01:20.473384+0000","last_became_peered":"2023-10-18T12:01:20.473384+0000","last_unstale":"2023-10-18T12:01:21.778823+0000","last_undegraded":"2023-10-18T12:01:21.778823+0000","last_fullsized":"2023-10-18T12:01:21.778823+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:33:41.233817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.457992+0000","last_change":"2023-10-18T12:01:09.130166+0000","last_active":"2023-10-18T12:01:20.457992+0000","last_peered":"2023-10-18T12:01:20.457992+0000","last_clean":"2023-10-18T12:01:20.457992+0000","last_became_active":"2023-10-18T12:01:09.129725+0000","last_became_peered":"2023-10-18T12:01:09.129725+0000","last_unstale":"2023-10-18T12:01:20.457992+0000","last_undegraded":"2023-10-18T12:01:20.457992+0000","last_fullsized":"2023-10-18T12:01:20.457992+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:37:40.892842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"241'1","reported_seq":17,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.460446+0000","last_change":"2023-10-18T12:01:08.107385+0000","last_active":"2023-10-18T12:01:20.460446+0000","last_peered":"2023-10-18T12:01:20.460446+0000","last_clean":"2023-10-18T12:01:20.460446+0000","last_became_active":"2023-10-18T12:01:08.107129+0000","last_became_peered":"2023-10-18T12:01:08.107129+0000","last_unstale":"2023-10-18T12:01:20.460446+0000","last_undegraded":"2023-10-18T12:01:20.460446+0000","last_fullsized":"2023-10-18T12:01:20.460446+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:45:43.838857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.627858+0000","last_change":"2023-10-18T12:01:20.468854+0000","last_active":"2023-10-18T12:01:21.627858+0000","last_peered":"2023-10-18T12:01:21.627858+0000","last_clean":"2023-10-18T12:01:21.627858+0000","last_became_active":"2023-10-18T12:01:20.468540+0000","last_became_peered":"2023-10-18T12:01:20.468540+0000","last_unstale":"2023-10-18T12:01:21.627858+0000","last_undegraded":"2023-10-18T12:01:21.627858+0000","last_fullsized":"2023-10-18T12:01:21.627858+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:43.730011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452258+0000","last_change":"2023-10-18T12:01:09.133737+0000","last_active":"2023-10-18T12:01:20.452258+0000","last_peered":"2023-10-18T12:01:20.452258+0000","last_clean":"2023-10-18T12:01:20.452258+0000","last_became_active":"2023-10-18T12:01:09.133487+0000","last_became_peered":"2023-10-18T12:01:09.133487+0000","last_unstale":"2023-10-18T12:01:20.452258+0000","last_undegraded":"2023-10-18T12:01:20.452258+0000","last_fullsized":"2023-10-18T12:01:20.452258+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:53:36.261500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452872+0000","last_change":"2023-10-18T12:01:08.119435+0000","last_active":"2023-10-18T12:01:20.452872+0000","last_peered":"2023-10-18T12:01:20.452872+0000","last_clean":"2023-10-18T12:01:20.452872+0000","last_became_active":"2023-10-18T12:01:08.118740+0000","last_became_peered":"2023-10-18T12:01:08.118740+0000","last_unstale":"2023-10-18T12:01:20.452872+0000","last_undegraded":"2023-10-18T12:01:20.452872+0000","last_fullsized":"2023-10-18T12:01:20.452872+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:04:54.172871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.627962+0000","last_change":"2023-10-18T12:01:20.469816+0000","last_active":"2023-10-18T12:01:21.627962+0000","last_peered":"2023-10-18T12:01:21.627962+0000","last_clean":"2023-10-18T12:01:21.627962+0000","last_became_active":"2023-10-18T12:01:20.469560+0000","last_became_peered":"2023-10-18T12:01:20.469560+0000","last_unstale":"2023-10-18T12:01:21.627962+0000","last_undegraded":"2023-10-18T12:01:21.627962+0000","last_fullsized":"2023-10-18T12:01:21.627962+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:49:00.700327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454419+0000","last_change":"2023-10-18T12:01:09.132487+0000","last_active":"2023-10-18T12:01:20.454419+0000","last_peered":"2023-10-18T12:01:20.454419+0000","last_clean":"2023-10-18T12:01:20.454419+0000","last_became_active":"2023-10-18T12:01:09.131789+0000","last_became_peered":"2023-10-18T12:01:09.131789+0000","last_unstale":"2023-10-18T12:01:20.454419+0000","last_undegraded":"2023-10-18T12:01:20.454419+0000","last_fullsized":"2023-10-18T12:01:20.454419+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:14.584151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454911+0000","last_change":"2023-10-18T12:01:08.115982+0000","last_active":"2023-10-18T12:01:20.454911+0000","last_peered":"2023-10-18T12:01:20.454911+0000","last_clean":"2023-10-18T12:01:20.454911+0000","last_became_active":"2023-10-18T12:01:08.115682+0000","last_became_peered":"2023-10-18T12:01:08.115682+0000","last_unstale":"2023-10-18T12:01:20.454911+0000","last_undegraded":"2023-10-18T12:01:20.454911+0000","last_fullsized":"2023-10-18T12:01:20.454911+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:36.789486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.627577+0000","last_change":"2023-10-18T12:01:20.471648+0000","last_active":"2023-10-18T12:01:21.627577+0000","last_peered":"2023-10-18T12:01:21.627577+0000","last_clean":"2023-10-18T12:01:21.627577+0000","last_became_active":"2023-10-18T12:01:20.471399+0000","last_became_peered":"2023-10-18T12:01:20.471399+0000","last_unstale":"2023-10-18T12:01:21.627577+0000","last_undegraded":"2023-10-18T12:01:21.627577+0000","last_fullsized":"2023-10-18T12:01:21.627577+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:51:35.178905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451639+0000","last_change":"2023-10-18T12:01:09.130010+0000","last_active":"2023-10-18T12:01:20.451639+0000","last_peered":"2023-10-18T12:01:20.451639+0000","last_clean":"2023-10-18T12:01:20.451639+0000","last_became_active":"2023-10-18T12:01:09.129744+0000","last_became_peered":"2023-10-18T12:01:09.129744+0000","last_unstale":"2023-10-18T12:01:20.451639+0000","last_undegraded":"2023-10-18T12:01:20.451639+0000","last_fullsized":"2023-10-18T12:01:20.451639+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:48:02.468955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.463128+0000","last_change":"2023-10-18T12:01:08.110833+0000","last_active":"2023-10-18T12:01:20.463128+0000","last_peered":"2023-10-18T12:01:20.463128+0000","last_clean":"2023-10-18T12:01:20.463128+0000","last_became_active":"2023-10-18T12:01:08.110005+0000","last_became_peered":"2023-10-18T12:01:08.110005+0000","last_unstale":"2023-10-18T12:01:20.463128+0000","last_undegraded":"2023-10-18T12:01:20.463128+0000","last_fullsized":"2023-10-18T12:01:20.463128+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:32.311648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.460907+0000","last_change":"2023-10-18T12:01:20.463685+0000","last_active":"2023-10-18T12:01:21.460907+0000","last_peered":"2023-10-18T12:01:21.460907+0000","last_clean":"2023-10-18T12:01:21.460907+0000","last_became_active":"2023-10-18T12:01:20.463447+0000","last_became_peered":"2023-10-18T12:01:20.463447+0000","last_unstale":"2023-10-18T12:01:21.460907+0000","last_undegraded":"2023-10-18T12:01:21.460907+0000","last_fullsized":"2023-10-18T12:01:21.460907+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:05.851308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451615+0000","last_change":"2023-10-18T12:01:09.129659+0000","last_active":"2023-10-18T12:01:20.451615+0000","last_peered":"2023-10-18T12:01:20.451615+0000","last_clean":"2023-10-18T12:01:20.451615+0000","last_became_active":"2023-10-18T12:01:09.129388+0000","last_became_peered":"2023-10-18T12:01:09.129388+0000","last_unstale":"2023-10-18T12:01:20.451615+0000","last_undegraded":"2023-10-18T12:01:20.451615+0000","last_fullsized":"2023-10-18T12:01:20.451615+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:17.308708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452393+0000","last_change":"2023-10-18T12:01:08.118321+0000","last_active":"2023-10-18T12:01:20.452393+0000","last_peered":"2023-10-18T12:01:20.452393+0000","last_clean":"2023-10-18T12:01:20.452393+0000","last_became_active":"2023-10-18T12:01:08.117995+0000","last_became_peered":"2023-10-18T12:01:08.117995+0000","last_unstale":"2023-10-18T12:01:20.452393+0000","last_undegraded":"2023-10-18T12:01:20.452393+0000","last_fullsized":"2023-10-18T12:01:20.452393+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:46.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.627239+0000","last_change":"2023-10-18T12:01:20.461525+0000","last_active":"2023-10-18T12:01:21.627239+0000","last_peered":"2023-10-18T12:01:21.627239+0000","last_clean":"2023-10-18T12:01:21.627239+0000","last_became_active":"2023-10-18T12:01:20.461230+0000","last_became_peered":"2023-10-18T12:01:20.461230+0000","last_unstale":"2023-10-18T12:01:21.627239+0000","last_undegraded":"2023-10-18T12:01:21.627239+0000","last_fullsized":"2023-10-18T12:01:21.627239+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:08.960634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453205+0000","last_change":"2023-10-18T12:01:09.134396+0000","last_active":"2023-10-18T12:01:20.453205+0000","last_peered":"2023-10-18T12:01:20.453205+0000","last_clean":"2023-10-18T12:01:20.453205+0000","last_became_active":"2023-10-18T12:01:09.134144+0000","last_became_peered":"2023-10-18T12:01:09.134144+0000","last_unstale":"2023-10-18T12:01:20.453205+0000","last_undegraded":"2023-10-18T12:01:20.453205+0000","last_fullsized":"2023-10-18T12:01:20.453205+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:23:53.396657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"241'3","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454817+0000","last_change":"2023-10-18T12:01:08.118871+0000","last_active":"2023-10-18T12:01:20.454817+0000","last_peered":"2023-10-18T12:01:20.454817+0000","last_clean":"2023-10-18T12:01:20.454817+0000","last_became_active":"2023-10-18T12:01:08.118572+0000","last_became_peered":"2023-10-18T12:01:08.118572+0000","last_unstale":"2023-10-18T12:01:20.454817+0000","last_undegraded":"2023-10-18T12:01:20.454817+0000","last_fullsized":"2023-10-18T12:01:20.454817+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:45.510206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876117+0000","last_change":"2023-10-18T12:01:20.468061+0000","last_active":"2023-10-18T12:01:21.876117+0000","last_peered":"2023-10-18T12:01:21.876117+0000","last_clean":"2023-10-18T12:01:21.876117+0000","last_became_active":"2023-10-18T12:01:20.467443+0000","last_became_peered":"2023-10-18T12:01:20.467443+0000","last_unstale":"2023-10-18T12:01:21.876117+0000","last_undegraded":"2023-10-18T12:01:21.876117+0000","last_fullsized":"2023-10-18T12:01:21.876117+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:54.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451440+0000","last_change":"2023-10-18T12:01:09.120444+0000","last_active":"2023-10-18T12:01:20.451440+0000","last_peered":"2023-10-18T12:01:20.451440+0000","last_clean":"2023-10-18T12:01:20.451440+0000","last_became_active":"2023-10-18T12:01:09.120018+0000","last_became_peered":"2023-10-18T12:01:09.120018+0000","last_unstale":"2023-10-18T12:01:20.451440+0000","last_undegraded":"2023-10-18T12:01:20.451440+0000","last_fullsized":"2023-10-18T12:01:20.451440+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:29.145722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":24,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454877+0000","last_change":"2023-10-18T12:01:08.117890+0000","last_active":"2023-10-18T12:01:20.454877+0000","last_peered":"2023-10-18T12:01:20.454877+0000","last_clean":"2023-10-18T12:01:20.454877+0000","last_became_active":"2023-10-18T12:01:08.117606+0000","last_became_peered":"2023-10-18T12:01:08.117606+0000","last_unstale":"2023-10-18T12:01:20.454877+0000","last_undegraded":"2023-10-18T12:01:20.454877+0000","last_fullsized":"2023-10-18T12:01:20.454877+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:55:49.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.878076+0000","last_change":"2023-10-18T12:01:20.463917+0000","last_active":"2023-10-18T12:01:21.878076+0000","last_peered":"2023-10-18T12:01:21.878076+0000","last_clean":"2023-10-18T12:01:21.878076+0000","last_became_active":"2023-10-18T12:01:20.463685+0000","last_became_peered":"2023-10-18T12:01:20.463685+0000","last_unstale":"2023-10-18T12:01:21.878076+0000","last_undegraded":"2023-10-18T12:01:21.878076+0000","last_fullsized":"2023-10-18T12:01:21.878076+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:08:44.478608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.448414+0000","last_change":"2023-10-18T12:01:09.127054+0000","last_active":"2023-10-18T12:01:20.448414+0000","last_peered":"2023-10-18T12:01:20.448414+0000","last_clean":"2023-10-18T12:01:20.448414+0000","last_became_active":"2023-10-18T12:01:09.126804+0000","last_became_peered":"2023-10-18T12:01:09.126804+0000","last_unstale":"2023-10-18T12:01:20.448414+0000","last_undegraded":"2023-10-18T12:01:20.448414+0000","last_fullsized":"2023-10-18T12:01:20.448414+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:20.948970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"241'2","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.458833+0000","last_change":"2023-10-18T12:01:08.118544+0000","last_active":"2023-10-18T12:01:20.458833+0000","last_peered":"2023-10-18T12:01:20.458833+0000","last_clean":"2023-10-18T12:01:20.458833+0000","last_became_active":"2023-10-18T12:01:08.118228+0000","last_became_peered":"2023-10-18T12:01:08.118228+0000","last_unstale":"2023-10-18T12:01:20.458833+0000","last_undegraded":"2023-10-18T12:01:20.458833+0000","last_fullsized":"2023-10-18T12:01:20.458833+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:49.760984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.462220+0000","last_change":"2023-10-18T12:01:20.469009+0000","last_active":"2023-10-18T12:01:21.462220+0000","last_peered":"2023-10-18T12:01:21.462220+0000","last_clean":"2023-10-18T12:01:21.462220+0000","last_became_active":"2023-10-18T12:01:20.468545+0000","last_became_peered":"2023-10-18T12:01:20.468545+0000","last_unstale":"2023-10-18T12:01:21.462220+0000","last_undegraded":"2023-10-18T12:01:21.462220+0000","last_fullsized":"2023-10-18T12:01:21.462220+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:12.556316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453295+0000","last_change":"2023-10-18T12:01:09.120750+0000","last_active":"2023-10-18T12:01:20.453295+0000","last_peered":"2023-10-18T12:01:20.453295+0000","last_clean":"2023-10-18T12:01:20.453295+0000","last_became_active":"2023-10-18T12:01:09.120490+0000","last_became_peered":"2023-10-18T12:01:09.120490+0000","last_unstale":"2023-10-18T12:01:20.453295+0000","last_undegraded":"2023-10-18T12:01:20.453295+0000","last_fullsized":"2023-10-18T12:01:20.453295+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:51.005840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454060+0000","last_change":"2023-10-18T12:01:08.118022+0000","last_active":"2023-10-18T12:01:20.454060+0000","last_peered":"2023-10-18T12:01:20.454060+0000","last_clean":"2023-10-18T12:01:20.454060+0000","last_became_active":"2023-10-18T12:01:08.117754+0000","last_became_peered":"2023-10-18T12:01:08.117754+0000","last_unstale":"2023-10-18T12:01:20.454060+0000","last_undegraded":"2023-10-18T12:01:20.454060+0000","last_fullsized":"2023-10-18T12:01:20.454060+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:37.834293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876594+0000","last_change":"2023-10-18T12:01:20.467060+0000","last_active":"2023-10-18T12:01:21.876594+0000","last_peered":"2023-10-18T12:01:21.876594+0000","last_clean":"2023-10-18T12:01:21.876594+0000","last_became_active":"2023-10-18T12:01:20.466780+0000","last_became_peered":"2023-10-18T12:01:20.466780+0000","last_unstale":"2023-10-18T12:01:21.876594+0000","last_undegraded":"2023-10-18T12:01:21.876594+0000","last_fullsized":"2023-10-18T12:01:21.876594+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:37.892372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.459411+0000","last_change":"2023-10-18T12:01:09.135383+0000","last_active":"2023-10-18T12:01:20.459411+0000","last_peered":"2023-10-18T12:01:20.459411+0000","last_clean":"2023-10-18T12:01:20.459411+0000","last_became_active":"2023-10-18T12:01:09.135114+0000","last_became_peered":"2023-10-18T12:01:09.135114+0000","last_unstale":"2023-10-18T12:01:20.459411+0000","last_undegraded":"2023-10-18T12:01:20.459411+0000","last_fullsized":"2023-10-18T12:01:20.459411+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:52:37.854776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453841+0000","last_change":"2023-10-18T12:01:08.115155+0000","last_active":"2023-10-18T12:01:20.453841+0000","last_peered":"2023-10-18T12:01:20.453841+0000","last_clean":"2023-10-18T12:01:20.453841+0000","last_became_active":"2023-10-18T12:01:08.114843+0000","last_became_peered":"2023-10-18T12:01:08.114843+0000","last_unstale":"2023-10-18T12:01:20.453841+0000","last_undegraded":"2023-10-18T12:01:20.453841+0000","last_fullsized":"2023-10-18T12:01:20.453841+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:28.358756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.462049+0000","last_change":"2023-10-18T12:01:20.463257+0000","last_active":"2023-10-18T12:01:21.462049+0000","last_peered":"2023-10-18T12:01:21.462049+0000","last_clean":"2023-10-18T12:01:21.462049+0000","last_became_active":"2023-10-18T12:01:20.462987+0000","last_became_peered":"2023-10-18T12:01:20.462987+0000","last_unstale":"2023-10-18T12:01:21.462049+0000","last_undegraded":"2023-10-18T12:01:21.462049+0000","last_fullsized":"2023-10-18T12:01:21.462049+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:40.487871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454336+0000","last_change":"2023-10-18T12:01:09.131820+0000","last_active":"2023-10-18T12:01:20.454336+0000","last_peered":"2023-10-18T12:01:20.454336+0000","last_clean":"2023-10-18T12:01:20.454336+0000","last_became_active":"2023-10-18T12:01:09.131568+0000","last_became_peered":"2023-10-18T12:01:09.131568+0000","last_unstale":"2023-10-18T12:01:20.454336+0000","last_undegraded":"2023-10-18T12:01:20.454336+0000","last_fullsized":"2023-10-18T12:01:20.454336+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.395799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"241'8","reported_seq":24,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.447810+0000","last_change":"2023-10-18T12:01:08.112711+0000","last_active":"2023-10-18T12:01:20.447810+0000","last_peered":"2023-10-18T12:01:20.447810+0000","last_clean":"2023-10-18T12:01:20.447810+0000","last_became_active":"2023-10-18T12:01:08.112436+0000","last_became_peered":"2023-10-18T12:01:08.112436+0000","last_unstale":"2023-10-18T12:01:20.447810+0000","last_undegraded":"2023-10-18T12:01:20.447810+0000","last_fullsized":"2023-10-18T12:01:20.447810+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:15.568916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876735+0000","last_change":"2023-10-18T12:01:20.477308+0000","last_active":"2023-10-18T12:01:21.876735+0000","last_peered":"2023-10-18T12:01:21.876735+0000","last_clean":"2023-10-18T12:01:21.876735+0000","last_became_active":"2023-10-18T12:01:20.477031+0000","last_became_peered":"2023-10-18T12:01:20.477031+0000","last_unstale":"2023-10-18T12:01:21.876735+0000","last_undegraded":"2023-10-18T12:01:21.876735+0000","last_fullsized":"2023-10-18T12:01:21.876735+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:54:47.410223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.455256+0000","last_change":"2023-10-18T12:01:09.135106+0000","last_active":"2023-10-18T12:01:20.455256+0000","last_peered":"2023-10-18T12:01:20.455256+0000","last_clean":"2023-10-18T12:01:20.455256+0000","last_became_active":"2023-10-18T12:01:09.134828+0000","last_became_peered":"2023-10-18T12:01:09.134828+0000","last_unstale":"2023-10-18T12:01:20.455256+0000","last_undegraded":"2023-10-18T12:01:20.455256+0000","last_fullsized":"2023-10-18T12:01:20.455256+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:28:39.126921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"241'1","reported_seq":17,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454775+0000","last_change":"2023-10-18T12:01:08.119248+0000","last_active":"2023-10-18T12:01:20.454775+0000","last_peered":"2023-10-18T12:01:20.454775+0000","last_clean":"2023-10-18T12:01:20.454775+0000","last_became_active":"2023-10-18T12:01:08.118973+0000","last_became_peered":"2023-10-18T12:01:08.118973+0000","last_unstale":"2023-10-18T12:01:20.454775+0000","last_undegraded":"2023-10-18T12:01:20.454775+0000","last_fullsized":"2023-10-18T12:01:20.454775+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:37.079812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876670+0000","last_change":"2023-10-18T12:01:20.477369+0000","last_active":"2023-10-18T12:01:21.876670+0000","last_peered":"2023-10-18T12:01:21.876670+0000","last_clean":"2023-10-18T12:01:21.876670+0000","last_became_active":"2023-10-18T12:01:20.477129+0000","last_became_peered":"2023-10-18T12:01:20.477129+0000","last_unstale":"2023-10-18T12:01:21.876670+0000","last_undegraded":"2023-10-18T12:01:21.876670+0000","last_fullsized":"2023-10-18T12:01:21.876670+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:51:19.290379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453740+0000","last_change":"2023-10-18T12:01:09.132827+0000","last_active":"2023-10-18T12:01:20.453740+0000","last_peered":"2023-10-18T12:01:20.453740+0000","last_clean":"2023-10-18T12:01:20.453740+0000","last_became_active":"2023-10-18T12:01:09.132545+0000","last_became_peered":"2023-10-18T12:01:09.132545+0000","last_unstale":"2023-10-18T12:01:20.453740+0000","last_undegraded":"2023-10-18T12:01:20.453740+0000","last_fullsized":"2023-10-18T12:01:20.453740+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:02.034692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"241'2","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.450111+0000","last_change":"2023-10-18T12:01:08.107964+0000","last_active":"2023-10-18T12:01:20.450111+0000","last_peered":"2023-10-18T12:01:20.450111+0000","last_clean":"2023-10-18T12:01:20.450111+0000","last_became_active":"2023-10-18T12:01:08.107696+0000","last_became_peered":"2023-10-18T12:01:08.107696+0000","last_unstale":"2023-10-18T12:01:20.450111+0000","last_undegraded":"2023-10-18T12:01:20.450111+0000","last_fullsized":"2023-10-18T12:01:20.450111+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:18.578919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.461567+0000","last_change":"2023-10-18T12:01:20.471356+0000","last_active":"2023-10-18T12:01:21.461567+0000","last_peered":"2023-10-18T12:01:21.461567+0000","last_clean":"2023-10-18T12:01:21.461567+0000","last_became_active":"2023-10-18T12:01:20.471102+0000","last_became_peered":"2023-10-18T12:01:20.471102+0000","last_unstale":"2023-10-18T12:01:21.461567+0000","last_undegraded":"2023-10-18T12:01:21.461567+0000","last_fullsized":"2023-10-18T12:01:21.461567+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:47.797615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.448004+0000","last_change":"2023-10-18T12:01:09.136078+0000","last_active":"2023-10-18T12:01:20.448004+0000","last_peered":"2023-10-18T12:01:20.448004+0000","last_clean":"2023-10-18T12:01:20.448004+0000","last_became_active":"2023-10-18T12:01:09.134722+0000","last_became_peered":"2023-10-18T12:01:09.134722+0000","last_unstale":"2023-10-18T12:01:20.448004+0000","last_undegraded":"2023-10-18T12:01:20.448004+0000","last_fullsized":"2023-10-18T12:01:20.448004+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:09.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"241'1","reported_seq":17,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451203+0000","last_change":"2023-10-18T12:01:08.120416+0000","last_active":"2023-10-18T12:01:20.451203+0000","last_peered":"2023-10-18T12:01:20.451203+0000","last_clean":"2023-10-18T12:01:20.451203+0000","last_became_active":"2023-10-18T12:01:08.120122+0000","last_became_peered":"2023-10-18T12:01:08.120122+0000","last_unstale":"2023-10-18T12:01:20.451203+0000","last_undegraded":"2023-10-18T12:01:20.451203+0000","last_fullsized":"2023-10-18T12:01:20.451203+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:26.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.875982+0000","last_change":"2023-10-18T12:01:20.468486+0000","last_active":"2023-10-18T12:01:21.875982+0000","last_peered":"2023-10-18T12:01:21.875982+0000","last_clean":"2023-10-18T12:01:21.875982+0000","last_became_active":"2023-10-18T12:01:20.468164+0000","last_became_peered":"2023-10-18T12:01:20.468164+0000","last_unstale":"2023-10-18T12:01:21.875982+0000","last_undegraded":"2023-10-18T12:01:21.875982+0000","last_fullsized":"2023-10-18T12:01:21.875982+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:14.644930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454986+0000","last_change":"2023-10-18T12:01:09.135042+0000","last_active":"2023-10-18T12:01:20.454986+0000","last_peered":"2023-10-18T12:01:20.454986+0000","last_clean":"2023-10-18T12:01:20.454986+0000","last_became_active":"2023-10-18T12:01:09.134765+0000","last_became_peered":"2023-10-18T12:01:09.134765+0000","last_unstale":"2023-10-18T12:01:20.454986+0000","last_undegraded":"2023-10-18T12:01:20.454986+0000","last_fullsized":"2023-10-18T12:01:20.454986+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:58.131101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.460261+0000","last_change":"2023-10-18T12:01:08.117972+0000","last_active":"2023-10-18T12:01:20.460261+0000","last_peered":"2023-10-18T12:01:20.460261+0000","last_clean":"2023-10-18T12:01:20.460261+0000","last_became_active":"2023-10-18T12:01:08.117690+0000","last_became_peered":"2023-10-18T12:01:08.117690+0000","last_unstale":"2023-10-18T12:01:20.460261+0000","last_undegraded":"2023-10-18T12:01:20.460261+0000","last_fullsized":"2023-10-18T12:01:20.460261+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.462153+0000","last_change":"2023-10-18T12:01:20.464761+0000","last_active":"2023-10-18T12:01:21.462153+0000","last_peered":"2023-10-18T12:01:21.462153+0000","last_clean":"2023-10-18T12:01:21.462153+0000","last_became_active":"2023-10-18T12:01:20.464517+0000","last_became_peered":"2023-10-18T12:01:20.464517+0000","last_unstale":"2023-10-18T12:01:21.462153+0000","last_undegraded":"2023-10-18T12:01:21.462153+0000","last_fullsized":"2023-10-18T12:01:21.462153+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:02.347990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453158+0000","last_change":"2023-10-18T12:01:09.122673+0000","last_active":"2023-10-18T12:01:20.453158+0000","last_peered":"2023-10-18T12:01:20.453158+0000","last_clean":"2023-10-18T12:01:20.453158+0000","last_became_active":"2023-10-18T12:01:09.122430+0000","last_became_peered":"2023-10-18T12:01:09.122430+0000","last_unstale":"2023-10-18T12:01:20.453158+0000","last_undegraded":"2023-10-18T12:01:20.453158+0000","last_fullsized":"2023-10-18T12:01:20.453158+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:29.387267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"241'2","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.455068+0000","last_change":"2023-10-18T12:01:08.119439+0000","last_active":"2023-10-18T12:01:20.455068+0000","last_peered":"2023-10-18T12:01:20.455068+0000","last_clean":"2023-10-18T12:01:20.455068+0000","last_became_active":"2023-10-18T12:01:08.119158+0000","last_became_peered":"2023-10-18T12:01:08.119158+0000","last_unstale":"2023-10-18T12:01:20.455068+0000","last_undegraded":"2023-10-18T12:01:20.455068+0000","last_fullsized":"2023-10-18T12:01:20.455068+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:17:56.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876790+0000","last_change":"2023-10-18T12:01:20.469101+0000","last_active":"2023-10-18T12:01:21.876790+0000","last_peered":"2023-10-18T12:01:21.876790+0000","last_clean":"2023-10-18T12:01:21.876790+0000","last_became_active":"2023-10-18T12:01:20.468827+0000","last_became_peered":"2023-10-18T12:01:20.468827+0000","last_unstale":"2023-10-18T12:01:21.876790+0000","last_undegraded":"2023-10-18T12:01:21.876790+0000","last_fullsized":"2023-10-18T12:01:21.876790+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:31:51.217620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.449837+0000","last_change":"2023-10-18T12:01:09.124352+0000","last_active":"2023-10-18T12:01:20.449837+0000","last_peered":"2023-10-18T12:01:20.449837+0000","last_clean":"2023-10-18T12:01:20.449837+0000","last_became_active":"2023-10-18T12:01:09.124072+0000","last_became_peered":"2023-10-18T12:01:09.124072+0000","last_unstale":"2023-10-18T12:01:20.449837+0000","last_undegraded":"2023-10-18T12:01:20.449837+0000","last_fullsized":"2023-10-18T12:01:20.449837+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:15.239290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.450998+0000","last_change":"2023-10-18T12:01:08.117335+0000","last_active":"2023-10-18T12:01:20.450998+0000","last_peered":"2023-10-18T12:01:20.450998+0000","last_clean":"2023-10-18T12:01:20.450998+0000","last_became_active":"2023-10-18T12:01:08.117032+0000","last_became_peered":"2023-10-18T12:01:08.117032+0000","last_unstale":"2023-10-18T12:01:20.450998+0000","last_undegraded":"2023-10-18T12:01:20.450998+0000","last_fullsized":"2023-10-18T12:01:20.450998+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:33.304569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876581+0000","last_change":"2023-10-18T12:01:20.466442+0000","last_active":"2023-10-18T12:01:21.876581+0000","last_peered":"2023-10-18T12:01:21.876581+0000","last_clean":"2023-10-18T12:01:21.876581+0000","last_became_active":"2023-10-18T12:01:20.466173+0000","last_became_peered":"2023-10-18T12:01:20.466173+0000","last_unstale":"2023-10-18T12:01:21.876581+0000","last_undegraded":"2023-10-18T12:01:21.876581+0000","last_fullsized":"2023-10-18T12:01:21.876581+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:00.125924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451983+0000","last_change":"2023-10-18T12:01:09.135148+0000","last_active":"2023-10-18T12:01:20.451983+0000","last_peered":"2023-10-18T12:01:20.451983+0000","last_clean":"2023-10-18T12:01:20.451983+0000","last_became_active":"2023-10-18T12:01:09.134895+0000","last_became_peered":"2023-10-18T12:01:09.134895+0000","last_unstale":"2023-10-18T12:01:20.451983+0000","last_undegraded":"2023-10-18T12:01:20.451983+0000","last_fullsized":"2023-10-18T12:01:20.451983+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:04:19.263310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451692+0000","last_change":"2023-10-18T12:01:08.120618+0000","last_active":"2023-10-18T12:01:20.451692+0000","last_peered":"2023-10-18T12:01:20.451692+0000","last_clean":"2023-10-18T12:01:20.451692+0000","last_became_active":"2023-10-18T12:01:08.120315+0000","last_became_peered":"2023-10-18T12:01:08.120315+0000","last_unstale":"2023-10-18T12:01:20.451692+0000","last_undegraded":"2023-10-18T12:01:20.451692+0000","last_fullsized":"2023-10-18T12:01:20.451692+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:36.555722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.627837+0000","last_change":"2023-10-18T12:01:20.462133+0000","last_active":"2023-10-18T12:01:21.627837+0000","last_peered":"2023-10-18T12:01:21.627837+0000","last_clean":"2023-10-18T12:01:21.627837+0000","last_became_active":"2023-10-18T12:01:20.461671+0000","last_became_peered":"2023-10-18T12:01:20.461671+0000","last_unstale":"2023-10-18T12:01:21.627837+0000","last_undegraded":"2023-10-18T12:01:21.627837+0000","last_fullsized":"2023-10-18T12:01:21.627837+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:05:44.394055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452177+0000","last_change":"2023-10-18T12:01:09.130881+0000","last_active":"2023-10-18T12:01:20.452177+0000","last_peered":"2023-10-18T12:01:20.452177+0000","last_clean":"2023-10-18T12:01:20.452177+0000","last_became_active":"2023-10-18T12:01:09.130628+0000","last_became_peered":"2023-10-18T12:01:09.130628+0000","last_unstale":"2023-10-18T12:01:20.452177+0000","last_undegraded":"2023-10-18T12:01:20.452177+0000","last_fullsized":"2023-10-18T12:01:20.452177+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:13.883184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.456177+0000","last_change":"2023-10-18T12:01:08.120313+0000","last_active":"2023-10-18T12:01:20.456177+0000","last_peered":"2023-10-18T12:01:20.456177+0000","last_clean":"2023-10-18T12:01:20.456177+0000","last_became_active":"2023-10-18T12:01:08.120059+0000","last_became_peered":"2023-10-18T12:01:08.120059+0000","last_unstale":"2023-10-18T12:01:20.456177+0000","last_undegraded":"2023-10-18T12:01:20.456177+0000","last_fullsized":"2023-10-18T12:01:20.456177+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:05.696928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876188+0000","last_change":"2023-10-18T12:01:20.464125+0000","last_active":"2023-10-18T12:01:21.876188+0000","last_peered":"2023-10-18T12:01:21.876188+0000","last_clean":"2023-10-18T12:01:21.876188+0000","last_became_active":"2023-10-18T12:01:20.463846+0000","last_became_peered":"2023-10-18T12:01:20.463846+0000","last_unstale":"2023-10-18T12:01:21.876188+0000","last_undegraded":"2023-10-18T12:01:21.876188+0000","last_fullsized":"2023-10-18T12:01:21.876188+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:23.913565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451556+0000","last_change":"2023-10-18T12:01:09.125685+0000","last_active":"2023-10-18T12:01:20.451556+0000","last_peered":"2023-10-18T12:01:20.451556+0000","last_clean":"2023-10-18T12:01:20.451556+0000","last_became_active":"2023-10-18T12:01:09.125395+0000","last_became_peered":"2023-10-18T12:01:09.125395+0000","last_unstale":"2023-10-18T12:01:20.451556+0000","last_undegraded":"2023-10-18T12:01:20.451556+0000","last_fullsized":"2023-10-18T12:01:20.451556+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:39:47.228016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"241'2","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452065+0000","last_change":"2023-10-18T12:01:08.117717+0000","last_active":"2023-10-18T12:01:20.452065+0000","last_peered":"2023-10-18T12:01:20.452065+0000","last_clean":"2023-10-18T12:01:20.452065+0000","last_became_active":"2023-10-18T12:01:08.117405+0000","last_became_peered":"2023-10-18T12:01:08.117405+0000","last_unstale":"2023-10-18T12:01:20.452065+0000","last_undegraded":"2023-10-18T12:01:20.452065+0000","last_fullsized":"2023-10-18T12:01:20.452065+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:00.206928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.461300+0000","last_change":"2023-10-18T12:01:20.462717+0000","last_active":"2023-10-18T12:01:21.461300+0000","last_peered":"2023-10-18T12:01:21.461300+0000","last_clean":"2023-10-18T12:01:21.461300+0000","last_became_active":"2023-10-18T12:01:20.462454+0000","last_became_peered":"2023-10-18T12:01:20.462454+0000","last_unstale":"2023-10-18T12:01:21.461300+0000","last_undegraded":"2023-10-18T12:01:21.461300+0000","last_fullsized":"2023-10-18T12:01:21.461300+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:39:25.532862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454849+0000","last_change":"2023-10-18T12:01:09.132599+0000","last_active":"2023-10-18T12:01:20.454849+0000","last_peered":"2023-10-18T12:01:20.454849+0000","last_clean":"2023-10-18T12:01:20.454849+0000","last_became_active":"2023-10-18T12:01:09.131710+0000","last_became_peered":"2023-10-18T12:01:09.131710+0000","last_unstale":"2023-10-18T12:01:20.454849+0000","last_undegraded":"2023-10-18T12:01:20.454849+0000","last_fullsized":"2023-10-18T12:01:20.454849+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:51:18.701174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"242'1","reported_seq":17,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.458978+0000","last_change":"2023-10-18T12:01:08.118553+0000","last_active":"2023-10-18T12:01:20.458978+0000","last_peered":"2023-10-18T12:01:20.458978+0000","last_clean":"2023-10-18T12:01:20.458978+0000","last_became_active":"2023-10-18T12:01:08.118286+0000","last_became_peered":"2023-10-18T12:01:08.118286+0000","last_unstale":"2023-10-18T12:01:20.458978+0000","last_undegraded":"2023-10-18T12:01:20.458978+0000","last_fullsized":"2023-10-18T12:01:20.458978+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:05:32.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876125+0000","last_change":"2023-10-18T12:01:20.477993+0000","last_active":"2023-10-18T12:01:21.876125+0000","last_peered":"2023-10-18T12:01:21.876125+0000","last_clean":"2023-10-18T12:01:21.876125+0000","last_became_active":"2023-10-18T12:01:20.477745+0000","last_became_peered":"2023-10-18T12:01:20.477745+0000","last_unstale":"2023-10-18T12:01:21.876125+0000","last_undegraded":"2023-10-18T12:01:21.876125+0000","last_fullsized":"2023-10-18T12:01:21.876125+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:08:40.820675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.453134+0000","last_change":"2023-10-18T12:01:09.133522+0000","last_active":"2023-10-18T12:01:20.453134+0000","last_peered":"2023-10-18T12:01:20.453134+0000","last_clean":"2023-10-18T12:01:20.453134+0000","last_became_active":"2023-10-18T12:01:09.133267+0000","last_became_peered":"2023-10-18T12:01:09.133267+0000","last_unstale":"2023-10-18T12:01:20.453134+0000","last_undegraded":"2023-10-18T12:01:20.453134+0000","last_fullsized":"2023-10-18T12:01:20.453134+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:44:14.702965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.463214+0000","last_change":"2023-10-18T12:01:08.116830+0000","last_active":"2023-10-18T12:01:20.463214+0000","last_peered":"2023-10-18T12:01:20.463214+0000","last_clean":"2023-10-18T12:01:20.463214+0000","last_became_active":"2023-10-18T12:01:08.116538+0000","last_became_peered":"2023-10-18T12:01:08.116538+0000","last_unstale":"2023-10-18T12:01:20.463214+0000","last_undegraded":"2023-10-18T12:01:20.463214+0000","last_fullsized":"2023-10-18T12:01:20.463214+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:09.165065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.778270+0000","last_change":"2023-10-18T12:01:20.465856+0000","last_active":"2023-10-18T12:01:21.778270+0000","last_peered":"2023-10-18T12:01:21.778270+0000","last_clean":"2023-10-18T12:01:21.778270+0000","last_became_active":"2023-10-18T12:01:20.465583+0000","last_became_peered":"2023-10-18T12:01:20.465583+0000","last_unstale":"2023-10-18T12:01:21.778270+0000","last_undegraded":"2023-10-18T12:01:21.778270+0000","last_fullsized":"2023-10-18T12:01:21.778270+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:30.345408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454852+0000","last_change":"2023-10-18T12:01:09.128305+0000","last_active":"2023-10-18T12:01:20.454852+0000","last_peered":"2023-10-18T12:01:20.454852+0000","last_clean":"2023-10-18T12:01:20.454852+0000","last_became_active":"2023-10-18T12:01:09.127933+0000","last_became_peered":"2023-10-18T12:01:09.127933+0000","last_unstale":"2023-10-18T12:01:20.454852+0000","last_undegraded":"2023-10-18T12:01:20.454852+0000","last_fullsized":"2023-10-18T12:01:20.454852+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:28.312681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.463388+0000","last_change":"2023-10-18T12:01:08.115363+0000","last_active":"2023-10-18T12:01:20.463388+0000","last_peered":"2023-10-18T12:01:20.463388+0000","last_clean":"2023-10-18T12:01:20.463388+0000","last_became_active":"2023-10-18T12:01:08.114903+0000","last_became_peered":"2023-10-18T12:01:08.114903+0000","last_unstale":"2023-10-18T12:01:20.463388+0000","last_undegraded":"2023-10-18T12:01:20.463388+0000","last_fullsized":"2023-10-18T12:01:20.463388+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:46.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.877686+0000","last_change":"2023-10-18T12:01:20.470320+0000","last_active":"2023-10-18T12:01:21.877686+0000","last_peered":"2023-10-18T12:01:21.877686+0000","last_clean":"2023-10-18T12:01:21.877686+0000","last_became_active":"2023-10-18T12:01:20.470061+0000","last_became_peered":"2023-10-18T12:01:20.470061+0000","last_unstale":"2023-10-18T12:01:21.877686+0000","last_undegraded":"2023-10-18T12:01:21.877686+0000","last_fullsized":"2023-10-18T12:01:21.877686+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:19.314294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.463129+0000","last_change":"2023-10-18T12:01:09.127159+0000","last_active":"2023-10-18T12:01:20.463129+0000","last_peered":"2023-10-18T12:01:20.463129+0000","last_clean":"2023-10-18T12:01:20.463129+0000","last_became_active":"2023-10-18T12:01:09.126874+0000","last_became_peered":"2023-10-18T12:01:09.126874+0000","last_unstale":"2023-10-18T12:01:20.463129+0000","last_undegraded":"2023-10-18T12:01:20.463129+0000","last_fullsized":"2023-10-18T12:01:20.463129+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:09:43.097908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.449729+0000","last_change":"2023-10-18T12:01:08.118539+0000","last_active":"2023-10-18T12:01:20.449729+0000","last_peered":"2023-10-18T12:01:20.449729+0000","last_clean":"2023-10-18T12:01:20.449729+0000","last_became_active":"2023-10-18T12:01:08.116966+0000","last_became_peered":"2023-10-18T12:01:08.116966+0000","last_unstale":"2023-10-18T12:01:20.449729+0000","last_undegraded":"2023-10-18T12:01:20.449729+0000","last_fullsized":"2023-10-18T12:01:20.449729+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:07:20.828671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.628116+0000","last_change":"2023-10-18T12:01:20.472044+0000","last_active":"2023-10-18T12:01:21.628116+0000","last_peered":"2023-10-18T12:01:21.628116+0000","last_clean":"2023-10-18T12:01:21.628116+0000","last_became_active":"2023-10-18T12:01:20.470948+0000","last_became_peered":"2023-10-18T12:01:20.470948+0000","last_unstale":"2023-10-18T12:01:21.628116+0000","last_undegraded":"2023-10-18T12:01:21.628116+0000","last_fullsized":"2023-10-18T12:01:21.628116+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:50.183681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452598+0000","last_change":"2023-10-18T12:01:09.119148+0000","last_active":"2023-10-18T12:01:20.452598+0000","last_peered":"2023-10-18T12:01:20.452598+0000","last_clean":"2023-10-18T12:01:20.452598+0000","last_became_active":"2023-10-18T12:01:09.118894+0000","last_became_peered":"2023-10-18T12:01:09.118894+0000","last_unstale":"2023-10-18T12:01:20.452598+0000","last_undegraded":"2023-10-18T12:01:20.452598+0000","last_fullsized":"2023-10-18T12:01:20.452598+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:42.857305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.454148+0000","last_change":"2023-10-18T12:01:08.117526+0000","last_active":"2023-10-18T12:01:20.454148+0000","last_peered":"2023-10-18T12:01:20.454148+0000","last_clean":"2023-10-18T12:01:20.454148+0000","last_became_active":"2023-10-18T12:01:08.117234+0000","last_became_peered":"2023-10-18T12:01:08.117234+0000","last_unstale":"2023-10-18T12:01:20.454148+0000","last_undegraded":"2023-10-18T12:01:20.454148+0000","last_fullsized":"2023-10-18T12:01:20.454148+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:07:43.295217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.875990+0000","last_change":"2023-10-18T12:01:20.461808+0000","last_active":"2023-10-18T12:01:21.875990+0000","last_peered":"2023-10-18T12:01:21.875990+0000","last_clean":"2023-10-18T12:01:21.875990+0000","last_became_active":"2023-10-18T12:01:20.461546+0000","last_became_peered":"2023-10-18T12:01:20.461546+0000","last_unstale":"2023-10-18T12:01:21.875990+0000","last_undegraded":"2023-10-18T12:01:21.875990+0000","last_fullsized":"2023-10-18T12:01:21.875990+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:16.840406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.450025+0000","last_change":"2023-10-18T12:01:09.125150+0000","last_active":"2023-10-18T12:01:20.450025+0000","last_peered":"2023-10-18T12:01:20.450025+0000","last_clean":"2023-10-18T12:01:20.450025+0000","last_became_active":"2023-10-18T12:01:09.124900+0000","last_became_peered":"2023-10-18T12:01:09.124900+0000","last_unstale":"2023-10-18T12:01:20.450025+0000","last_undegraded":"2023-10-18T12:01:20.450025+0000","last_fullsized":"2023-10-18T12:01:20.450025+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:58:10.077999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452948+0000","last_change":"2023-10-18T12:01:08.119612+0000","last_active":"2023-10-18T12:01:20.452948+0000","last_peered":"2023-10-18T12:01:20.452948+0000","last_clean":"2023-10-18T12:01:20.452948+0000","last_became_active":"2023-10-18T12:01:08.119116+0000","last_became_peered":"2023-10-18T12:01:08.119116+0000","last_unstale":"2023-10-18T12:01:20.452948+0000","last_undegraded":"2023-10-18T12:01:20.452948+0000","last_fullsized":"2023-10-18T12:01:20.452948+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:10:02.536144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.778825+0000","last_change":"2023-10-18T12:01:20.469904+0000","last_active":"2023-10-18T12:01:21.778825+0000","last_peered":"2023-10-18T12:01:21.778825+0000","last_clean":"2023-10-18T12:01:21.778825+0000","last_became_active":"2023-10-18T12:01:20.469648+0000","last_became_peered":"2023-10-18T12:01:20.469648+0000","last_unstale":"2023-10-18T12:01:21.778825+0000","last_undegraded":"2023-10-18T12:01:21.778825+0000","last_fullsized":"2023-10-18T12:01:21.778825+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:53:47.603649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452091+0000","last_change":"2023-10-18T12:01:09.130360+0000","last_active":"2023-10-18T12:01:20.452091+0000","last_peered":"2023-10-18T12:01:20.452091+0000","last_clean":"2023-10-18T12:01:20.452091+0000","last_became_active":"2023-10-18T12:01:09.130086+0000","last_became_peered":"2023-10-18T12:01:09.130086+0000","last_unstale":"2023-10-18T12:01:20.452091+0000","last_undegraded":"2023-10-18T12:01:20.452091+0000","last_fullsized":"2023-10-18T12:01:20.452091+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:22:28.085761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.451332+0000","last_change":"2023-10-18T12:01:08.119803+0000","last_active":"2023-10-18T12:01:20.451332+0000","last_peered":"2023-10-18T12:01:20.451332+0000","last_clean":"2023-10-18T12:01:20.451332+0000","last_became_active":"2023-10-18T12:01:08.119467+0000","last_became_peered":"2023-10-18T12:01:08.119467+0000","last_unstale":"2023-10-18T12:01:20.451332+0000","last_undegraded":"2023-10-18T12:01:20.451332+0000","last_fullsized":"2023-10-18T12:01:20.451332+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:29.410010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":11,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876796+0000","last_change":"2023-10-18T12:01:20.476773+0000","last_active":"2023-10-18T12:01:21.876796+0000","last_peered":"2023-10-18T12:01:21.876796+0000","last_clean":"2023-10-18T12:01:21.876796+0000","last_became_active":"2023-10-18T12:01:20.476526+0000","last_became_peered":"2023-10-18T12:01:20.476526+0000","last_unstale":"2023-10-18T12:01:21.876796+0000","last_undegraded":"2023-10-18T12:01:21.876796+0000","last_fullsized":"2023-10-18T12:01:21.876796+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:19.435707+0000","last_clean_scrub_stamp":"2023-10-18T12:01:19.435707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:40.306804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":15,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452626+0000","last_change":"2023-10-18T12:01:09.122670+0000","last_active":"2023-10-18T12:01:20.452626+0000","last_peered":"2023-10-18T12:01:20.452626+0000","last_clean":"2023-10-18T12:01:20.452626+0000","last_became_active":"2023-10-18T12:01:09.122408+0000","last_became_peered":"2023-10-18T12:01:09.122408+0000","last_unstale":"2023-10-18T12:01:20.452626+0000","last_undegraded":"2023-10-18T12:01:20.452626+0000","last_fullsized":"2023-10-18T12:01:20.452626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:08.093558+0000","last_clean_scrub_stamp":"2023-10-18T12:01:08.093558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:19:39.479911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"241'1","reported_seq":17,"reported_epoch":245,"state":"active+clean","last_fresh":"2023-10-18T12:01:20.452738+0000","last_change":"2023-10-18T12:01:08.111055+0000","last_active":"2023-10-18T12:01:20.452738+0000","last_peered":"2023-10-18T12:01:20.452738+0000","last_clean":"2023-10-18T12:01:20.452738+0000","last_became_active":"2023-10-18T12:01:08.110728+0000","last_became_peered":"2023-10-18T12:01:08.110728+0000","last_unstale":"2023-10-18T12:01:20.452738+0000","last_undegraded":"2023-10-18T12:01:20.452738+0000","last_fullsized":"2023-10-18T12:01:20.452738+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":239,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:07.086319+0000","last_clean_scrub_stamp":"2023-10-18T12:01:07.086319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:15.472803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"227'379","reported_seq":1334,"reported_epoch":246,"state":"active+clean","last_fresh":"2023-10-18T12:01:21.876120+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:01:21.876120+0000","last_peered":"2023-10-18T12:01:21.876120+0000","last_clean":"2023-10-18T12:01:21.876120+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:01:21.876120+0000","last_undegraded":"2023-10-18T12:01:21.876120+0000","last_fullsized":"2023-10-18T12:01:21.876120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2402,"num_read_kb":5029,"num_write":635,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":379,"ondisk_log_size":379,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576001,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52856,"kb_used_data":3624,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318984,"statfs":{"total":96636764160,"available":96582639616,"internally_reserved":0,"allocated":3710976,"data_stored":2192961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.80800000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.56200000000000006,"15min":0.60099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":1.363},"last":0.59999999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.86399999999999999},"last":0.54600000000000004}]},{"osd":2,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.627},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56100000000000005,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.90000000000000002,"15min":1.298},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71899999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":1.5249999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.81299999999999994},"last":0.39700000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.503,"15min":0.5},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.314},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.41899999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70699999999999996,"5min":0.86599999999999999,"15min":1.478},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.83599999999999997,"15min":0.874},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42544,"kb_used_data":3040,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329296,"statfs":{"total":96636764160,"available":96593199104,"internally_reserved":0,"allocated":3112960,"data_stored":1602011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.439}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66600000000000004}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.62}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834576001,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48104,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323736,"statfs":{"total":96636764160,"available":96587505664,"internally_reserved":0,"allocated":3104768,"data_stored":1601935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.54600000000000004}]},{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":1.101,"15min":1.1539999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.71099999999999997,"15min":0.751},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.64200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.628},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.67400000000000004}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72699999999999998,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.71799999999999997}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81399999999999995,"15min":1},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53000000000000003,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.45800000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.749,"5min":0.82999999999999996,"15min":1.01},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.42799999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.255,"15min":0.255},"max":{"1min":0.78800000000000003,"5min":0.83699999999999997,"15min":1.101},"last":0.376},{"interface":"front","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.60499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834576001,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48112,"kb_used_data":3044,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323728,"statfs":{"total":96636764160,"available":96587497472,"internally_reserved":0,"allocated":3117056,"data_stored":1603547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.22500000000000001},"max":{"1min":0.81499999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.61499999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":1.123},"last":0.64800000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58899999999999997,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.503,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":1.018,"15min":1.018},"last":0.754}]},{"osd":7,"last update":"Wed Oct 18 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.88900000000000001,"15min":1.05},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.79100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834576001,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48124,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323716,"statfs":{"total":96636764160,"available":96587485184,"internally_reserved":0,"allocated":3129344,"data_stored":1605461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.23999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.378,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.39300000000000002,"15min":0.40699999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.38400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.40100000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.39900000000000002,"15min":0.40600000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44900000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.432,"5min":1.899,"15min":1.899},"last":0.33200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.42899999999999999,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.434,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.318},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.87,"5min":1.034,"15min":1.052},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.50800000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":1.522,"15min":1.522},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.91100000000000003,"15min":1.026},"last":0.52700000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82299999999999995,"5min":1.01,"15min":1.01},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.129,"15min":1.129},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0649999999999999},"last":0.56000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834576001,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60608,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311232,"statfs":{"total":96636764160,"available":96574701568,"internally_reserved":0,"allocated":3129344,"data_stored":1604733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.52300000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.40500000000000003,"15min":0.41599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.29599999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.61499999999999999,"15min":0.624},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.63100000000000001,"15min":0.746},"last":0.36399999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55300000000000005,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.41799999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86299999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66300000000000003,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.087},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.85199999999999998,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63400000000000001,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.95399999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":1.0609999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834576001,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64752,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307088,"statfs":{"total":96636764160,"available":96570458112,"internally_reserved":0,"allocated":3108864,"data_stored":1601989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.254},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.315},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.36399999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66000000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.73299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834576000,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43124,"kb_used_data":3624,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328716,"statfs":{"total":96636764160,"available":96592605184,"internally_reserved":0,"allocated":3710976,"data_stored":2195171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52200000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.371},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.52300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.26500000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.51100000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.434}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.96999999999999997,"15min":0.997},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.81699999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72399999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.95499999999999996,"15min":0.96399999999999997},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64700000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":1.03},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70999999999999996,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.78000000000000003,"5min":0.997,"15min":0.997},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:01:28.127 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T12:01:28.128 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:28.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.258+0000 7f474e444640 1 -- 172.21.15.125:0/3905927422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 msgr2=0x7f4730003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.258+0000 7f474e444640 1 --2- 172.21.15.125:0/3905927422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 0x7f4730003d70 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c6e0 tx=0x7f47300057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.258+0000 7f474e444640 1 -- 172.21.15.125:0/3905927422 shutdown_connections 2023-10-18T12:01:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.258+0000 7f474e444640 1 --2- 172.21.15.125:0/3905927422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 0x7f4730003d70 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.258+0000 7f474e444640 1 -- 172.21.15.125:0/3905927422 >> 172.21.15.125:0/3905927422 conn(0x7f47480d0d60 msgr2=0x7f47480d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:28.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.258+0000 7f474e444640 1 -- 172.21.15.125:0/3905927422 shutdown_connections 2023-10-18T12:01:28.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.258+0000 7f474e444640 1 -- 172.21.15.125:0/3905927422 wait complete. 2023-10-18T12:01:28.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.259+0000 7f474e444640 1 Processor -- start 2023-10-18T12:01:28.265 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f474e444640 1 -- start start 2023-10-18T12:01:28.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f474e444640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 0x7f47480c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:28.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f474e444640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474815e150 0x7f47480b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:28.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f474e444640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47480c9ea0 0x7f47480ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:28.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f474e444640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47480b3f80 con 0x7f4730003970 2023-10-18T12:01:28.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f474e444640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47480ca860 con 0x7f47480c9ea0 2023-10-18T12:01:28.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f474e444640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474814fa90 con 0x7f474815e150 2023-10-18T12:01:28.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f4747fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 0x7f47480c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:28.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.260+0000 7f4747fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 0x7f47480c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39558/0 (socket says 172.21.15.125:39558) 2023-10-18T12:01:28.267 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4747fff640 1 -- 172.21.15.125:0/3944166570 learned_addr learned my addr 172.21.15.125:0/3944166570 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:28.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f474c9e1640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47480c9ea0 0x7f47480ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:28.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f47477fe640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474815e150 0x7f47480b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:28.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4747fff640 1 -- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474815e150 msgr2=0x7f47480b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:28.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4747fff640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474815e150 0x7f47480b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.268 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4747fff640 1 -- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47480c9ea0 msgr2=0x7f47480ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:28.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4747fff640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47480c9ea0 0x7f47480ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4747fff640 1 -- 172.21.15.125:0/3944166570 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47480b3950 con 0x7f4730003970 2023-10-18T12:01:28.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4747fff640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 0x7f47480c8840 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c800 tx=0x7f473c004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:28.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4744ff9640 1 -- 172.21.15.125:0/3944166570 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c02f670 con 0x7f4730003970 2023-10-18T12:01:28.269 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4744ff9640 1 -- 172.21.15.125:0/3944166570 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f473c02f8c0 con 0x7f4730003970 2023-10-18T12:01:28.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47480b6740 con 0x7f4730003970 2023-10-18T12:01:28.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f4744ff9640 1 -- 172.21.15.125:0/3944166570 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c02fcd0 con 0x7f4730003970 2023-10-18T12:01:28.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.261+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47480c8e20 con 0x7f4730003970 2023-10-18T12:01:28.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.262+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47480690e0 con 0x7f4730003970 2023-10-18T12:01:28.270 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.262+0000 7f4744ff9640 1 -- 172.21.15.125:0/3944166570 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f473c039430 con 0x7f4730003970 2023-10-18T12:01:28.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.263+0000 7f4744ff9640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f473406b080 0x7f473406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:28.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.263+0000 7f4744ff9640 1 -- 172.21.15.125:0/3944166570 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f473c0ac180 con 0x7f4730003970 2023-10-18T12:01:28.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.263+0000 7f47477fe640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f473406b080 0x7f473406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:28.271 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.264+0000 7f47477fe640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f473406b080 0x7f473406d540 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4738001d20 tx=0x7f4738001bd0 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:28.273 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.271+0000 7f4744ff9640 1 -- 172.21.15.125:0/3944166570 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f473c076970 con 0x7f4730003970 2023-10-18T12:01:28.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.482+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47480d0b40 con 0x7f4730003970 2023-10-18T12:01:28.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.483+0000 7f4744ff9640 1 -- 172.21.15.125:0/3944166570 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+6157 (secure 0 0 0) 0x7f473c079f90 con 0x7f4730003970 2023-10-18T12:01:28.486 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:28.486 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":103,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:08.447292+0000","modified":"2023-10-18T12:01:10.725386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12771":{"gid":12771,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1851256737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1851256737},{"type":"v1","addr":"172.21.15.125:6835","nonce":1851256737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:28.487 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 103 2023-10-18T12:01:28.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f473406b080 msgr2=0x7f473406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:28.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f473406b080 0x7f473406d540 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4738001d20 tx=0x7f4738001bd0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 msgr2=0x7f47480c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:28.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 0x7f47480c8840 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c800 tx=0x7f473c004e90 comp rx=0 tx=0).stop 2023-10-18T12:01:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 shutdown_connections 2023-10-18T12:01:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f473406b080 0x7f473406d540 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47480c9ea0 0x7f47480ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474815e150 0x7f47480b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.484+0000 7f474e444640 1 --2- 172.21.15.125:0/3944166570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4730003970 0x7f47480c8840 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:28.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.485+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 >> 172.21.15.125:0/3944166570 conn(0x7f47480d0d60 msgr2=0x7f47480b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:28.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.485+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 shutdown_connections 2023-10-18T12:01:28.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:28.485+0000 7f474e444640 1 -- 172.21.15.125:0/3944166570 wait complete. 2023-10-18T12:01:28.900 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T12:01:19.784343+0000', 'modified': '2023-10-18T12:01:21.785616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12768}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12768': {'gid': 12768, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.125:6834/3879855806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6832', 'nonce': 3879855806}, {'type': 'v1', 'addr': '172.21.15.125:6834', 'nonce': 3879855806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T12:01:28.901 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T12:01:28.901 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T12:01:29.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.002+0000 7f55239d0640 1 -- 172.21.15.125:0/1477688321 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c0c9d00 msgr2=0x7f551c06b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.002+0000 7f55239d0640 1 --2- 172.21.15.125:0/1477688321 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c0c9d00 0x7f551c06b910 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f551c140c40 tx=0x7f551402bdb0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.002+0000 7f55239d0640 1 -- 172.21.15.125:0/1477688321 shutdown_connections 2023-10-18T12:01:29.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.002+0000 7f55239d0640 1 --2- 172.21.15.125:0/1477688321 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c0c9d00 0x7f551c06b910 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.002+0000 7f55239d0640 1 --2- 172.21.15.125:0/1477688321 >> v2:172.21.15.139:3300/0 conn(0x7f551c140660 0x7f551c0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.002+0000 7f55239d0640 1 -- 172.21.15.125:0/1477688321 >> 172.21.15.125:0/1477688321 conn(0x7f551c069930 msgr2=0x7f551c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.002+0000 7f55239d0640 1 -- 172.21.15.125:0/1477688321 shutdown_connections 2023-10-18T12:01:29.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.002+0000 7f55239d0640 1 -- 172.21.15.125:0/1477688321 wait complete. 2023-10-18T12:01:29.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.003+0000 7f55239d0640 1 Processor -- start 2023-10-18T12:01:29.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.003+0000 7f55239d0640 1 -- start start 2023-10-18T12:01:29.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f55239d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f551c0c9d00 0x7f551c13f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f55239d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f551c13d360 0x7f551c0a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f55239d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c140660 0x7f551c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f55239d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c0adfb0 con 0x7f551c140660 2023-10-18T12:01:29.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f55239d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c0ae850 con 0x7f551c0c9d00 2023-10-18T12:01:29.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f55239d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c0b0d60 con 0x7f551c13d360 2023-10-18T12:01:29.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5521f6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c140660 0x7f551c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5521f6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c140660 0x7f551c0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39578/0 (socket says 172.21.15.125:39578) 2023-10-18T12:01:29.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5521f6d640 1 -- 172.21.15.125:0/1344811649 learned_addr learned my addr 172.21.15.125:0/1344811649 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:29.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f552176c640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f551c0c9d00 0x7f551c13f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f551c13d360 0x7f551c0a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5521f6d640 1 -- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f551c13d360 msgr2=0x7f551c0a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5521f6d640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f551c13d360 0x7f551c0a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5521f6d640 1 -- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f551c0c9d00 msgr2=0x7f551c13f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5521f6d640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f551c0c9d00 0x7f551c13f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f5521f6d640 1 -- 172.21.15.125:0/1344811649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514009560 con 0x7f551c140660 2023-10-18T12:01:29.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.004+0000 7f552176c640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f551c0c9d00 0x7f551c13f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:29.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.005+0000 7f5521f6d640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c140660 0x7f551c0aece0 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f5514009230 tx=0x7f551402dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.005+0000 7f5511ffb640 1 -- 172.21.15.125:0/1344811649 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514005690 con 0x7f551c140660 2023-10-18T12:01:29.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.005+0000 7f5511ffb640 1 -- 172.21.15.125:0/1344811649 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55140038f0 con 0x7f551c140660 2023-10-18T12:01:29.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.005+0000 7f5511ffb640 1 -- 172.21.15.125:0/1344811649 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514003d00 con 0x7f551c140660 2023-10-18T12:01:29.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.005+0000 7f55239d0640 1 -- 172.21.15.125:0/1344811649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c0ae200 con 0x7f551c140660 2023-10-18T12:01:29.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.005+0000 7f55239d0640 1 -- 172.21.15.125:0/1344811649 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c1563c0 con 0x7f551c140660 2023-10-18T12:01:29.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.006+0000 7f5511ffb640 1 -- 172.21.15.125:0/1344811649 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f551403e450 con 0x7f551c140660 2023-10-18T12:01:29.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.007+0000 7f5511ffb640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f550006b0b0 0x7f550006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.007+0000 7f5511ffb640 1 -- 172.21.15.125:0/1344811649 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f55140ac820 con 0x7f551c140660 2023-10-18T12:01:29.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.007+0000 7f552176c640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f550006b0b0 0x7f550006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.008+0000 7f552176c640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f550006b0b0 0x7f550006d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f551c13c620 tx=0x7f5518009040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.008+0000 7f5526567740 1 Processor -- start 2023-10-18T12:01:29.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.008+0000 7f5526567740 1 -- start start 2023-10-18T12:01:29.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5526567740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42b40 0x56194ea15ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5526567740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42f20 0x56194ea458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5526567740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea16020 0x56194ea45e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5526567740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194e927730 con 0x56194ea42f20 2023-10-18T12:01:29.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5526567740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194e988780 con 0x56194ea42b40 2023-10-18T12:01:29.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5526567740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56194e985e70 con 0x56194ea16020 2023-10-18T12:01:29.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42f20 0x56194ea458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42f20 0x56194ea458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39580/0 (socket says 172.21.15.125:39580) 2023-10-18T12:01:29.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 -- 172.21.15.125:0/1004253423 learned_addr learned my addr 172.21.15.125:0/1004253423 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:29.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5521f6d640 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea16020 0x56194ea45e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f552176c640 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42b40 0x56194ea15ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 -- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea16020 msgr2=0x56194ea45e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:29.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea16020 0x56194ea45e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 -- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42b40 msgr2=0x56194ea15ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:29.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42b40 0x56194ea15ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 -- 172.21.15.125:0/1004253423 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56194e9863d0 con 0x56194ea42f20 2023-10-18T12:01:29.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5521f6d640 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea16020 0x56194ea45e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:29.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f552176c640 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42b40 0x56194ea15ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:29.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42f20 0x56194ea458d0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b780 tx=0x7f550c00bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505c45adf2558753 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f54feffd640 1 -- 172.21.15.125:0/1004253423 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00be90 con 0x56194ea42f20 2023-10-18T12:01:29.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f54feffd640 1 -- 172.21.15.125:0/1004253423 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f550c00da40 con 0x56194ea42f20 2023-10-18T12:01:29.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.009+0000 7f54feffd640 1 -- 172.21.15.125:0/1004253423 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00de50 con 0x56194ea42f20 2023-10-18T12:01:29.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 -- 172.21.15.125:0/1004253423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42f20 msgr2=0x56194ea458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42f20 0x56194ea458d0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b780 tx=0x7f550c00bc40 comp rx=0 tx=0).stop 2023-10-18T12:01:29.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 -- 172.21.15.125:0/1004253423 shutdown_connections 2023-10-18T12:01:29.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea16020 0x56194ea45e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42f20 0x56194ea458d0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 --2- 172.21.15.125:0/1004253423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42b40 0x56194ea15ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 -- 172.21.15.125:0/1004253423 >> 172.21.15.125:0/1004253423 conn(0x56194ea17d50 msgr2=0x56194ea608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 -- 172.21.15.125:0/1004253423 shutdown_connections 2023-10-18T12:01:29.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.010+0000 7f5526567740 1 -- 172.21.15.125:0/1004253423 wait complete. 2023-10-18T12:01:29.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.011+0000 7f5526567740 1 -- start start 2023-10-18T12:01:29.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.011+0000 7f5526567740 1 Processor -- start 2023-10-18T12:01:29.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5526567740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea70680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5526567740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194ea48420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5526567740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 0x56194ea5af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5526567740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194e981760 con 0x56194ea42b40 2023-10-18T12:01:29.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5526567740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194e93c8f0 con 0x56194ea42f20 2023-10-18T12:01:29.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5526567740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56194e93c590 con 0x56194ea48960 2023-10-18T12:01:29.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5526567740 10 client.0 _ll_register_callbacks cb 0x56194e978ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:01:29.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea70680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea70680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39590/0 (socket says 172.21.15.125:39590) 2023-10-18T12:01:29.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 -- 172.21.15.125:0/1541352221 learned_addr learned my addr 172.21.15.125:0/1541352221 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:29.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5521f6d640 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 0x56194ea5af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194ea48420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 -- 172.21.15.125:0/1541352221 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 msgr2=0x56194ea5af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 0x56194ea5af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 -- 172.21.15.125:0/1541352221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 msgr2=0x56194ea48420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194ea48420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 -- 172.21.15.125:0/1541352221 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550c00e040 con 0x56194ea42b40 2023-10-18T12:01:29.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f552176c640 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea70680 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f551800ecf0 tx=0x7f5518001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c35402d273c96b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f54f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:01:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f54f7fff640 1 -- 172.21.15.125:0/1541352221 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518006b10 con 0x56194ea42b40 2023-10-18T12:01:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f54f7fff640 1 -- 172.21.15.125:0/1541352221 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f5518006c70 con 0x56194ea42b40 2023-10-18T12:01:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.012+0000 7f5526567740 1 -- 172.21.15.125:0/1541352221 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56194e97a740 con 0x56194ea42b40 2023-10-18T12:01:29.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f54f7fff640 1 -- 172.21.15.125:0/1541352221 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5518006e30 con 0x56194ea42b40 2023-10-18T12:01:29.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5526567740 10 client.13101 fetch_fsmap learned FSMap version 103 2023-10-18T12:01:29.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5526567740 1 -- 172.21.15.125:0/1541352221 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x56194e97fde0 con 0x56194ea42b40 2023-10-18T12:01:29.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f54f7fff640 1 -- 172.21.15.125:0/1541352221 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x56194e97a740 con 0x56194ea42b40 2023-10-18T12:01:29.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f54f7fff640 1 -- 172.21.15.125:0/1541352221 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 103) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f5518007140 con 0x56194ea42b40 2023-10-18T12:01:29.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5526567740 10 client.13101 fetch_fsmap finished waiting for FSMap version 103 2023-10-18T12:01:29.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5526567740 10 client.13101 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T12:01:29.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5526567740 20 client.13101 populate_metadata read hostname 'smithi125' 2023-10-18T12:01:29.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5526567740 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] conn(0x56194ea5a150 0x56194ea33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5526567740 4 client.13101 mds_command: new command op to 12768 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:01:29.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5526567740 1 -- 172.21.15.125:0/1541352221 --> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56194e97a460 con 0x56194ea5a150 2023-10-18T12:01:29.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] conn(0x56194ea5a150 0x56194ea33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f54f7fff640 10 client.13101 ms_handle_connect on v2:172.21.15.125:6832/3879855806 2023-10-18T12:01:29.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.013+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] conn(0x56194ea5a150 0x56194ea33610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26ce6c3cd94069 server_cookie=61d04d75dddb310a in_seq=0 out_seq=0 2023-10-18T12:01:29.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f54f7fff640 1 -- 172.21.15.125:0/1541352221 <== mds.0 v2:172.21.15.125:6832/3879855806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56194e97a460 con 0x56194ea5a150 2023-10-18T12:01:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f54f7fff640 10 client.13101 handle_command_reply: tid=0 2023-10-18T12:01:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 client.13101 shutdown 2023-10-18T12:01:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 -- 172.21.15.125:0/1541352221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 msgr2=0x56194ea70680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea70680 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f551800ecf0 tx=0x7f5518001330 comp rx=0 tx=0).stop 2023-10-18T12:01:29.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 -- 172.21.15.125:0/1541352221 shutdown_connections 2023-10-18T12:01:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] conn(0x56194ea5a150 0x56194ea33610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194ea48420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 0x56194ea5af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 --2- 172.21.15.125:0/1541352221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea70680 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f54f7fff640 0 client.13101 ms_handle_reset on v2:172.21.15.125:6832/3879855806 2023-10-18T12:01:29.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 -- 172.21.15.125:0/1541352221 >> 172.21.15.125:0/1541352221 conn(0x56194ea16020 msgr2=0x56194ea199c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 -- 172.21.15.125:0/1541352221 shutdown_connections 2023-10-18T12:01:29.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 1 -- 172.21.15.125:0/1541352221 wait complete. 2023-10-18T12:01:29.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.014+0000 7f5526567740 20 client.13101 trim_cache size 0 max 16384 2023-10-18T12:01:29.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f5526567740 1 Processor -- start 2023-10-18T12:01:29.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f5526567740 1 -- start start 2023-10-18T12:01:29.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f5526567740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea45ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f5526567740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194eac4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f5526567740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 0x56194eac4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f5526567740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194e902270 con 0x56194ea42b40 2023-10-18T12:01:29.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f5526567740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194ea630e0 con 0x56194ea42f20 2023-10-18T12:01:29.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f5526567740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56194ea5b870 con 0x56194ea48960 2023-10-18T12:01:29.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f552176c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea45ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f552176c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea45ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39598/0 (socket says 172.21.15.125:39598) 2023-10-18T12:01:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.030+0000 7f552176c640 1 -- 172.21.15.125:0/114942809 learned_addr learned my addr 172.21.15.125:0/114942809 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f552176c640 1 -- 172.21.15.125:0/114942809 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 msgr2=0x56194eac4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f552176c640 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 0x56194eac4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f5520f6b640 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194eac4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f552176c640 1 -- 172.21.15.125:0/114942809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 msgr2=0x56194eac4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:29.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f552176c640 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194eac4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f552176c640 1 -- 172.21.15.125:0/114942809 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56194e91ed00 con 0x56194ea42b40 2023-10-18T12:01:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f5520f6b640 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194eac4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f552176c640 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea45ab0 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f55180058a0 tx=0x7f55180013a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6879cbfc86ca9fc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f54ff7fe640 1 -- 172.21.15.125:0/114942809 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518001640 con 0x56194ea42b40 2023-10-18T12:01:29.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f54ff7fe640 1 -- 172.21.15.125:0/114942809 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55180079c0 con 0x56194ea42b40 2023-10-18T12:01:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.031+0000 7f54ff7fe640 1 -- 172.21.15.125:0/114942809 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518003d70 con 0x56194ea42b40 2023-10-18T12:01:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 -- 172.21.15.125:0/114942809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 msgr2=0x56194ea45ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea45ab0 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f55180058a0 tx=0x7f55180013a0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 -- 172.21.15.125:0/114942809 shutdown_connections 2023-10-18T12:01:29.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea42f20 0x56194eac4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea48960 0x56194eac4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 --2- 172.21.15.125:0/114942809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea45ab0 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 -- 172.21.15.125:0/114942809 >> 172.21.15.125:0/114942809 conn(0x56194ea16020 msgr2=0x56194ea1cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 -- 172.21.15.125:0/114942809 shutdown_connections 2023-10-18T12:01:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 -- 172.21.15.125:0/114942809 wait complete. 2023-10-18T12:01:29.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 -- start start 2023-10-18T12:01:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.032+0000 7f5526567740 1 Processor -- start 2023-10-18T12:01:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5526567740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5526567740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea42f20 0x56194ea601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5526567740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea48960 0x56194ea606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5526567740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194eac9ee0 con 0x56194ea42b40 2023-10-18T12:01:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5526567740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194ea63f70 con 0x56194ea48960 2023-10-18T12:01:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5526567740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x56194ea45ab0 con 0x56194ea42f20 2023-10-18T12:01:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5526567740 10 client.0 _ll_register_callbacks cb 0x7f550c00b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:01:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f552176c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea6fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f552176c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea6fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39602/0 (socket says 172.21.15.125:39602) 2023-10-18T12:01:29.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f552176c640 1 -- 172.21.15.125:0/1540253705 learned_addr learned my addr 172.21.15.125:0/1540253705 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:29.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5521f6d640 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea48960 0x56194ea606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea42f20 0x56194ea601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f552176c640 1 -- 172.21.15.125:0/1540253705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea42f20 msgr2=0x56194ea601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:29.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f552176c640 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea42f20 0x56194ea601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f552176c640 1 -- 172.21.15.125:0/1540253705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea48960 msgr2=0x56194ea606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:01:29.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f552176c640 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea48960 0x56194ea606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.033+0000 7f552176c640 1 -- 172.21.15.125:0/1540253705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56194e97fde0 con 0x56194ea42b40 2023-10-18T12:01:29.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f552176c640 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea6fcf0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f5518033040 tx=0x7f5518007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab851a7ab1091409 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f54f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:01:29.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f54f7fff640 1 -- 172.21.15.125:0/1540253705 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518006d20 con 0x56194ea42b40 2023-10-18T12:01:29.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f54f7fff640 1 -- 172.21.15.125:0/1540253705 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f5518006e80 con 0x56194ea42b40 2023-10-18T12:01:29.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 1 -- 172.21.15.125:0/1540253705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f550c00da40 con 0x56194ea42b40 2023-10-18T12:01:29.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f54f7fff640 1 -- 172.21.15.125:0/1540253705 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5518007040 con 0x56194ea42b40 2023-10-18T12:01:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 10 client.13107 fetch_fsmap learned FSMap version 103 2023-10-18T12:01:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 1 -- 172.21.15.125:0/1540253705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x56194e9444f0 con 0x56194ea42b40 2023-10-18T12:01:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f54f7fff640 1 -- 172.21.15.125:0/1540253705 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f550c00da40 con 0x56194ea42b40 2023-10-18T12:01:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f54f7fff640 1 -- 172.21.15.125:0/1540253705 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 103) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f551803e250 con 0x56194ea42b40 2023-10-18T12:01:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 10 client.13107 fetch_fsmap finished waiting for FSMap version 103 2023-10-18T12:01:29.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 10 client.13107 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T12:01:29.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 20 client.13107 populate_metadata read hostname 'smithi125' 2023-10-18T12:01:29.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] conn(0x56194ea72da0 0x56194ea73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 4 client.13107 mds_command: new command op to 12768 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T12:01:29.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5526567740 1 -- 172.21.15.125:0/1540253705 --> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56194e6836c0 con 0x56194ea72da0 2023-10-18T12:01:29.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.034+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] conn(0x56194ea72da0 0x56194ea73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f54f7fff640 10 client.13107 ms_handle_connect on v2:172.21.15.125:6832/3879855806 2023-10-18T12:01:29.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5520f6b640 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] conn(0x56194ea72da0 0x56194ea73180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f59838b363080c71 server_cookie=484ee0b4c3caa271 in_seq=0 out_seq=0 2023-10-18T12:01:29.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f54f7fff640 1 -- 172.21.15.125:0/1540253705 <== mds.0 v2:172.21.15.125:6832/3879855806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+338 (crc 0 0 0) 0x56194e6836c0 con 0x56194ea72da0 2023-10-18T12:01:29.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f54f7fff640 10 client.13107 handle_command_reply: tid=0 2023-10-18T12:01:29.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 client.13107 shutdown 2023-10-18T12:01:29.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 -- 172.21.15.125:0/1540253705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 msgr2=0x56194ea6fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea6fcf0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f5518033040 tx=0x7f5518007600 comp rx=0 tx=0).stop 2023-10-18T12:01:29.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 -- 172.21.15.125:0/1540253705 shutdown_connections 2023-10-18T12:01:29.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.125:6832/3879855806,v1:172.21.15.125:6834/3879855806] conn(0x56194ea72da0 0x56194ea73180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56194ea48960 0x56194ea606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x56194ea42f20 0x56194ea601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f54f7fff640 0 client.13107 ms_handle_reset on v2:172.21.15.125:6832/3879855806 2023-10-18T12:01:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 --2- 172.21.15.125:0/1540253705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56194ea42b40 0x56194ea6fcf0 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 -- 172.21.15.125:0/1540253705 >> 172.21.15.125:0/1540253705 conn(0x56194ea16020 msgr2=0x56194ea115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 -- 172.21.15.125:0/1540253705 shutdown_connections 2023-10-18T12:01:29.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 1 -- 172.21.15.125:0/1540253705 wait complete. 2023-10-18T12:01:29.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.035+0000 7f5526567740 20 client.13107 trim_cache size 0 max 16384 2023-10-18T12:01:29.058 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:01:29.058 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T12:01:29.058 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T12:01:29.059 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 12768, 2023-10-18T12:01:29.059 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T12:01:29.061 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T12:01:29.061 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "backup_fs", 2023-10-18T12:01:29.061 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 8.5905728680000006, 2023-10-18T12:01:29.062 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 103, 2023-10-18T12:01:29.062 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 246, 2023-10-18T12:01:29.062 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 246, 2023-10-18T12:01:29.062 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 24.670189992000001 2023-10-18T12:01:29.062 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:01:29.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 -- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f550006b0b0 msgr2=0x7f550006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f550006b0b0 0x7f550006d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f551c13c620 tx=0x7f5518009040 comp rx=0 tx=0).stop 2023-10-18T12:01:29.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 -- 172.21.15.125:0/1344811649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c140660 msgr2=0x7f551c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c140660 0x7f551c0aece0 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f5514009230 tx=0x7f551402dd00 comp rx=0 tx=0).stop 2023-10-18T12:01:29.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 -- 172.21.15.125:0/1344811649 shutdown_connections 2023-10-18T12:01:29.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f550006b0b0 0x7f550006d570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f551c140660 0x7f551c0aece0 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f551c13d360 0x7f551c0a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 --2- 172.21.15.125:0/1344811649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f551c0c9d00 0x7f551c13f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 -- 172.21.15.125:0/1344811649 >> 172.21.15.125:0/1344811649 conn(0x7f551c069930 msgr2=0x7f551c158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 -- 172.21.15.125:0/1344811649 shutdown_connections 2023-10-18T12:01:29.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.036+0000 7f55239d0640 1 -- 172.21.15.125:0/1344811649 wait complete. 2023-10-18T12:01:29.066 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T12:01:29.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.168+0000 7f281679e640 1 -- 172.21.15.125:0/1730316174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28100b1b40 msgr2=0x7f28100b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.168+0000 7f281679e640 1 --2- 172.21.15.125:0/1730316174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28100b1b40 0x7f28100b1fd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2808005fd0 tx=0x7f280802bcf0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.168+0000 7f281679e640 1 -- 172.21.15.125:0/1730316174 shutdown_connections 2023-10-18T12:01:29.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.168+0000 7f281679e640 1 --2- 172.21.15.125:0/1730316174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28100b1b40 0x7f28100b1fd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.168+0000 7f281679e640 1 --2- 172.21.15.125:0/1730316174 >> v2:172.21.15.139:3300/0 conn(0x7f281015e120 0x7f2810069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.168+0000 7f281679e640 1 -- 172.21.15.125:0/1730316174 >> 172.21.15.125:0/1730316174 conn(0x7f2810070080 msgr2=0x7f2810070490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.168+0000 7f281679e640 1 -- 172.21.15.125:0/1730316174 shutdown_connections 2023-10-18T12:01:29.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.168+0000 7f281679e640 1 -- 172.21.15.125:0/1730316174 wait complete. 2023-10-18T12:01:29.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.169+0000 7f281679e640 1 Processor -- start 2023-10-18T12:01:29.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.169+0000 7f281679e640 1 -- start start 2023-10-18T12:01:29.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f281679e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2810070dd0 0x7f28100aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f281679e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28100b1b40 0x7f281013fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f281679e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f281015e120 0x7f28100b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f281679e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28101537d0 con 0x7f281015e120 2023-10-18T12:01:29.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f281679e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28100af000 con 0x7f28100b1b40 2023-10-18T12:01:29.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f281679e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28101539c0 con 0x7f2810070dd0 2023-10-18T12:01:29.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2815f9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f281015e120 0x7f28100b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2815f9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f281015e120 0x7f28100b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39604/0 (socket says 172.21.15.125:39604) 2023-10-18T12:01:29.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2815f9d640 1 -- 172.21.15.125:0/3909423864 learned_addr learned my addr 172.21.15.125:0/3909423864 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:29.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2814f9b640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28100b1b40 0x7f281013fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f281579c640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2810070dd0 0x7f28100aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2815f9d640 1 -- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2810070dd0 msgr2=0x7f28100aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2815f9d640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2810070dd0 0x7f28100aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2815f9d640 1 -- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28100b1b40 msgr2=0x7f281013fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2815f9d640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28100b1b40 0x7f281013fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.170+0000 7f2815f9d640 1 -- 172.21.15.125:0/3909423864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c009840 con 0x7f281015e120 2023-10-18T12:01:29.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.171+0000 7f2815f9d640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f281015e120 0x7f28100b0b10 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f2808006010 tx=0x7f280802d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.171+0000 7f2805ffb640 1 -- 172.21.15.125:0/3909423864 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280802ddc0 con 0x7f281015e120 2023-10-18T12:01:29.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.171+0000 7f2805ffb640 1 -- 172.21.15.125:0/3909423864 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2808003700 con 0x7f281015e120 2023-10-18T12:01:29.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.171+0000 7f2805ffb640 1 -- 172.21.15.125:0/3909423864 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808003b10 con 0x7f281015e120 2023-10-18T12:01:29.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.171+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2808009560 con 0x7f281015e120 2023-10-18T12:01:29.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.171+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2810153f80 con 0x7f281015e120 2023-10-18T12:01:29.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.172+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2810066d10 con 0x7f281015e120 2023-10-18T12:01:29.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.172+0000 7f2805ffb640 1 -- 172.21.15.125:0/3909423864 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2808003c70 con 0x7f281015e120 2023-10-18T12:01:29.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.173+0000 7f2805ffb640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f27f406b080 0x7f27f406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.173+0000 7f2805ffb640 1 -- 172.21.15.125:0/3909423864 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f28080aca10 con 0x7f281015e120 2023-10-18T12:01:29.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.173+0000 7f281579c640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f27f406b080 0x7f27f406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.173+0000 7f281579c640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f27f406b080 0x7f27f406d540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28100b1760 tx=0x7f280c009500 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.177+0000 7f2805ffb640 1 -- 172.21.15.125:0/3909423864 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28080774e0 con 0x7f281015e120 2023-10-18T12:01:29.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.364+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f28100ac930 con 0x7f27f406b080 2023-10-18T12:01:29.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.366+0000 7f2805ffb640 1 -- 172.21.15.125:0/3909423864 <== mgr.9236 v2:172.21.15.139:6832/3422518215 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f28100ac930 con 0x7f27f406b080 2023-10-18T12:01:29.368 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T12:01:29.369 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:29.369 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T12:01:29.369 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:29.369 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:29.369 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "backup_fs" 2023-10-18T12:01:29.369 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:29.370 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T12:01:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f27f406b080 msgr2=0x7f27f406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f27f406b080 0x7f27f406d540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28100b1760 tx=0x7f280c009500 comp rx=0 tx=0).stop 2023-10-18T12:01:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f281015e120 msgr2=0x7f28100b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f281015e120 0x7f28100b0b10 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f2808006010 tx=0x7f280802d9c0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 shutdown_connections 2023-10-18T12:01:29.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f27f406b080 0x7f27f406d540 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28100b1b40 0x7f281013fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f281015e120 0x7f28100b0b10 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 --2- 172.21.15.125:0/3909423864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2810070dd0 0x7f28100aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 >> 172.21.15.125:0/3909423864 conn(0x7f2810070080 msgr2=0x7f28100abf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 shutdown_connections 2023-10-18T12:01:29.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.368+0000 7f281679e640 1 -- 172.21.15.125:0/3909423864 wait complete. 2023-10-18T12:01:29.381 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T12:01:29.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.483+0000 7f2ac3275640 1 -- 172.21.15.125:0/2895009094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc0c9ed0 msgr2=0x7f2abc161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.483+0000 7f2ac3275640 1 --2- 172.21.15.125:0/2895009094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc0c9ed0 0x7f2abc161b70 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400a940 tx=0x7f2ab402d090 comp rx=0 tx=0).stop 2023-10-18T12:01:29.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.483+0000 7f2ac3275640 1 -- 172.21.15.125:0/2895009094 shutdown_connections 2023-10-18T12:01:29.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.483+0000 7f2ac3275640 1 --2- 172.21.15.125:0/2895009094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc0c9ed0 0x7f2abc161b70 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.483+0000 7f2ac3275640 1 -- 172.21.15.125:0/2895009094 >> 172.21.15.125:0/2895009094 conn(0x7f2abc069930 msgr2=0x7f2abc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:29.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.483+0000 7f2ac3275640 1 -- 172.21.15.125:0/2895009094 shutdown_connections 2023-10-18T12:01:29.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.483+0000 7f2ac3275640 1 -- 172.21.15.125:0/2895009094 wait complete. 2023-10-18T12:01:29.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.484+0000 7f2ac3275640 1 Processor -- start 2023-10-18T12:01:29.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.484+0000 7f2ac3275640 1 -- start start 2023-10-18T12:01:29.490 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac3275640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2abc0c9ed0 0x7f2abc0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac3275640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc13d360 0x7f2abc151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac3275640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2abc140660 0x7f2abc139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac3275640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0b14a0 con 0x7f2abc13d360 2023-10-18T12:01:29.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac3275640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0b08d0 con 0x7f2abc140660 2023-10-18T12:01:29.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac3275640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc13a730 con 0x7f2abc0c9ed0 2023-10-18T12:01:29.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc13d360 0x7f2abc151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc13d360 0x7f2abc151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39610/0 (socket says 172.21.15.125:39610) 2023-10-18T12:01:29.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 -- 172.21.15.125:0/1922731024 learned_addr learned my addr 172.21.15.125:0/1922731024 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:29.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac1011640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2abc0c9ed0 0x7f2abc0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac1812640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2abc140660 0x7f2abc139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 -- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2abc0c9ed0 msgr2=0x7f2abc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2abc0c9ed0 0x7f2abc0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 -- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2abc140660 msgr2=0x7f2abc139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:29.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2abc140660 0x7f2abc139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:29.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 -- 172.21.15.125:0/1922731024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab400a5a0 con 0x7f2abc13d360 2023-10-18T12:01:29.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac0810640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc13d360 0x7f2abc151610 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00b3f0 tx=0x7f2aac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ab1ffb640 1 -- 172.21.15.125:0/1922731024 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011830 con 0x7f2abc13d360 2023-10-18T12:01:29.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ab1ffb640 1 -- 172.21.15.125:0/1922731024 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2aac011a80 con 0x7f2abc13d360 2023-10-18T12:01:29.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc151b50 con 0x7f2abc13d360 2023-10-18T12:01:29.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.485+0000 7f2ab1ffb640 1 -- 172.21.15.125:0/1922731024 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011e90 con 0x7f2abc13d360 2023-10-18T12:01:29.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.486+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc0b0b00 con 0x7f2abc13d360 2023-10-18T12:01:29.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.487+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc0b15e0 con 0x7f2abc13d360 2023-10-18T12:01:29.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.487+0000 7f2ab1ffb640 1 -- 172.21.15.125:0/1922731024 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 139) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2aac01a400 con 0x7f2abc13d360 2023-10-18T12:01:29.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.487+0000 7f2ab1ffb640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2aa006b080 0x7f2aa006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:29.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.487+0000 7f2ab1ffb640 1 -- 172.21.15.125:0/1922731024 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f2aac089fd0 con 0x7f2abc13d360 2023-10-18T12:01:29.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.487+0000 7f2ac1011640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2aa006b080 0x7f2aa006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:29.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.488+0000 7f2ac1011640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2aa006b080 0x7f2aa006d540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8002110 tx=0x7f2ab8009040 comp rx=0 tx=0).ready entity=mgr.9236 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:29.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.491+0000 7f2ab1ffb640 1 -- 172.21.15.125:0/1922731024 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2abc0b15e0 con 0x7f2abc13d360 2023-10-18T12:01:29.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:29.701+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2abc0b15e0 con 0x7f2abc13d360 2023-10-18T12:01:30.414 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.411+0000 7f2ab1ffb640 1 -- 172.21.15.125:0/1922731024 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v140) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2abc0b15e0 con 0x7f2abc13d360 2023-10-18T12:01:30.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.414+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2aa006b080 msgr2=0x7f2aa006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.414+0000 7f2ac3275640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2aa006b080 0x7f2aa006d540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8002110 tx=0x7f2ab8009040 comp rx=0 tx=0).stop 2023-10-18T12:01:30.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.414+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc13d360 msgr2=0x7f2abc151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.417 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.414+0000 7f2ac3275640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc13d360 0x7f2abc151610 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00b3f0 tx=0x7f2aac00b8b0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.415+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 shutdown_connections 2023-10-18T12:01:30.418 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.415+0000 7f2ac3275640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f2aa006b080 0x7f2aa006d540 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.415+0000 7f2ac3275640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2abc140660 0x7f2abc139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.415+0000 7f2ac3275640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2abc13d360 0x7f2abc151610 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.415+0000 7f2ac3275640 1 --2- 172.21.15.125:0/1922731024 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2abc0c9ed0 0x7f2abc0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.415+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 >> 172.21.15.125:0/1922731024 conn(0x7f2abc069930 msgr2=0x7f2abc13ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:30.419 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.415+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 shutdown_connections 2023-10-18T12:01:30.420 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.415+0000 7f2ac3275640 1 -- 172.21.15.125:0/1922731024 wait complete. 2023-10-18T12:01:30.432 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T12:01:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.544+0000 7f0c6334c640 1 -- 172.21.15.125:0/1988052567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 msgr2=0x7f0c5c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.544+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1988052567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 0x7f0c5c0cdfb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009830 tx=0x7f0c4c02bcd0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.544+0000 7f0c6334c640 1 -- 172.21.15.125:0/1988052567 shutdown_connections 2023-10-18T12:01:30.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.544+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1988052567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 0x7f0c5c0cdfb0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.544+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1988052567 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c5c159060 0x7f0c5c06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.544+0000 7f0c6334c640 1 -- 172.21.15.125:0/1988052567 >> 172.21.15.125:0/1988052567 conn(0x7f0c5c0b0670 msgr2=0x7f0c5c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.544+0000 7f0c6334c640 1 -- 172.21.15.125:0/1988052567 shutdown_connections 2023-10-18T12:01:30.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.544+0000 7f0c6334c640 1 -- 172.21.15.125:0/1988052567 wait complete. 2023-10-18T12:01:30.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.545+0000 7f0c6334c640 1 Processor -- start 2023-10-18T12:01:30.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.545+0000 7f0c6334c640 1 -- start start 2023-10-18T12:01:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.545+0000 7f0c6334c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 0x7f0c5c0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.545+0000 7f0c6334c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c5c159060 0x7f0c5c0acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.545+0000 7f0c6334c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c5c15ccf0 0x7f0c5c157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c6334c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0c3d40 con 0x7f0c5c159060 2023-10-18T12:01:30.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c6334c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0c3130 con 0x7f0c5c0cdb30 2023-10-18T12:01:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c6334c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0c3410 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c608e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c5c159060 0x7f0c5c0acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c610e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 0x7f0c5c0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c618e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c5c15ccf0 0x7f0c5c157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c608e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c5c159060 0x7f0c5c0acc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39626/0 (socket says 172.21.15.125:39626) 2023-10-18T12:01:30.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c608e7640 1 -- 172.21.15.125:0/1198924667 learned_addr learned my addr 172.21.15.125:0/1198924667 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:30.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c610e8640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 0x7f0c5c0c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:33260/0 (socket says 172.21.15.125:33260) 2023-10-18T12:01:30.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c618e9640 1 -- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 msgr2=0x7f0c5c0c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c618e9640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 0x7f0c5c0c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c618e9640 1 -- 172.21.15.125:0/1198924667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c5c159060 msgr2=0x7f0c5c0acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c618e9640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c5c159060 0x7f0c5c0acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c618e9640 1 -- 172.21.15.125:0/1198924667 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c58009490 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c610e8640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 0x7f0c5c0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:30.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c608e7640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c5c159060 0x7f0c5c0acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:30.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c618e9640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c5c15ccf0 0x7f0c5c157c40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400caa0 tx=0x7f0c5400cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:30.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c51ffb640 1 -- 172.21.15.125:0/1198924667 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400ab40 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c51ffb640 1 -- 172.21.15.125:0/1198924667 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0c5400adb0 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c009490 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.546+0000 7f0c51ffb640 1 -- 172.21.15.125:0/1198924667 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54012680 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.547+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c0c39d0 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.548+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c0c3eb0 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.548+0000 7f0c51ffb640 1 -- 172.21.15.125:0/1198924667 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 140) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f0c5401a3b0 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.548+0000 7f0c51ffb640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0c4006b100 0x7f0c4006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:30.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.548+0000 7f0c610e8640 1 -- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0c4006b100 msgr2=0x7f0c4006d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3422518215 2023-10-18T12:01:30.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.549+0000 7f0c51ffb640 1 -- 172.21.15.125:0/1198924667 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f0c5408a500 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.549+0000 7f0c610e8640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0c4006b100 0x7f0c4006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:30.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.553+0000 7f0c51ffb640 1 -- 172.21.15.125:0/1198924667 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c54054da0 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.741+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f0c5c0675f0 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.750+0000 7f0c610e8640 1 -- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0c4006b100 msgr2=0x7f0c4006d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3422518215 2023-10-18T12:01:30.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.750+0000 7f0c610e8640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0c4006b100 0x7f0c4006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:01:30.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.750+0000 7f0c51ffb640 1 -- 172.21.15.125:0/1198924667 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v80)=0 v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0c540583c0 con 0x7f0c5c15ccf0 2023-10-18T12:01:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0c4006b100 msgr2=0x7f0c4006d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:30.755 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0c4006b100 0x7f0c4006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c5c15ccf0 msgr2=0x7f0c5c157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c5c15ccf0 0x7f0c5c157c40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400caa0 tx=0x7f0c5400cea0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 shutdown_connections 2023-10-18T12:01:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f0c4006b100 0x7f0c4006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.756 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c5c15ccf0 0x7f0c5c157c40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c5c159060 0x7f0c5c0acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 --2- 172.21.15.125:0/1198924667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c5c0cdb30 0x7f0c5c0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 >> 172.21.15.125:0/1198924667 conn(0x7f0c5c0b0670 msgr2=0x7f0c5c0ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:30.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.752+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 shutdown_connections 2023-10-18T12:01:30.757 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.753+0000 7f0c6334c640 1 -- 172.21.15.125:0/1198924667 wait complete. 2023-10-18T12:01:30.766 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T12:01:30.801 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:30.798+0000 7f266a043040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:01:30.828 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:30.825+0000 7fd9f5811040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:01:30.846 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:30.843+0000 7f266a043040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:01:30.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.870+0000 7f117601a640 1 -- 172.21.15.125:0/1530043995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 msgr2=0x7f1158003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.870+0000 7f117601a640 1 --2- 172.21.15.125:0/1530043995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 0x7f1158003dc0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f1160006fd0 tx=0x7f11580057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.870+0000 7f116ffff640 1 -- 172.21.15.125:0/1530043995 reap_dead start 2023-10-18T12:01:30.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.870+0000 7f117601a640 1 -- 172.21.15.125:0/1530043995 shutdown_connections 2023-10-18T12:01:30.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.870+0000 7f117601a640 1 -- 172.21.15.125:0/1530043995 >> 172.21.15.125:0/1530043995 conn(0x7f11700d0d60 msgr2=0x7f11700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:30.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.870+0000 7f117601a640 1 -- 172.21.15.125:0/1530043995 shutdown_connections 2023-10-18T12:01:30.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.870+0000 7f117601a640 1 -- 172.21.15.125:0/1530043995 wait complete. 2023-10-18T12:01:30.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.872+0000 7f117601a640 1 Processor -- start 2023-10-18T12:01:30.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.872+0000 7f117601a640 1 -- start start 2023-10-18T12:01:30.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.872+0000 7f117601a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 0x7f11700b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:30.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.872+0000 7f117601a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f117015e050 0x7f117015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:30.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.872+0000 7f117601a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b0ea0 0x7f11700b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:30.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.872+0000 7f117601a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11700b1890 con 0x7f11580039a0 2023-10-18T12:01:30.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.872+0000 7f117601a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117015cd40 con 0x7f11700b0ea0 2023-10-18T12:01:30.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.872+0000 7f117601a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11700b5dd0 con 0x7f117015e050 2023-10-18T12:01:30.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 0x7f11700b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:30.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 0x7f11700b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39648/0 (socket says 172.21.15.125:39648) 2023-10-18T12:01:30.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 -- 172.21.15.125:0/16227435 learned_addr learned my addr 172.21.15.125:0/16227435 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:30.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116effd640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f117015e050 0x7f117015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:30.878 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116ffff640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b0ea0 0x7f11700b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:30.879 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:30.873+0000 7fd9f5811040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:01:30.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 -- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f117015e050 msgr2=0x7f117015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.880 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f117015e050 0x7f117015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 -- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b0ea0 msgr2=0x7f11700b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:30.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b0ea0 0x7f11700b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:30.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 -- 172.21.15.125:0/16227435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700b60b0 con 0x7f11580039a0 2023-10-18T12:01:30.881 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f116f7fe640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 0x7f11700b57f0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f1160007010 tx=0x7f11600015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:30.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f1157fff640 1 -- 172.21.15.125:0/16227435 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160006230 con 0x7f11580039a0 2023-10-18T12:01:30.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f1157fff640 1 -- 172.21.15.125:0/16227435 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f11600063b0 con 0x7f11580039a0 2023-10-18T12:01:30.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11700b6390 con 0x7f11580039a0 2023-10-18T12:01:30.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f1157fff640 1 -- 172.21.15.125:0/16227435 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160003850 con 0x7f11580039a0 2023-10-18T12:01:30.882 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.873+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117015e4d0 con 0x7f11580039a0 2023-10-18T12:01:30.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.874+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117015b6d0 con 0x7f11580039a0 2023-10-18T12:01:30.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.875+0000 7f1157fff640 1 -- 172.21.15.125:0/16227435 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 140) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f1160031030 con 0x7f11580039a0 2023-10-18T12:01:30.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.875+0000 7f1157fff640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f115c06b0d0 0x7f115c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:30.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.875+0000 7f1157fff640 1 -- 172.21.15.125:0/16227435 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7f11600aabc0 con 0x7f11580039a0 2023-10-18T12:01:30.883 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.875+0000 7f116effd640 1 -- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f115c06b0d0 msgr2=0x7f115c06d590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3422518215 2023-10-18T12:01:30.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.875+0000 7f116effd640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f115c06b0d0 0x7f115c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:30.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:30.879+0000 7f1157fff640 1 -- 172.21.15.125:0/16227435 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11600754a0 con 0x7f11580039a0 2023-10-18T12:01:31.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.048+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f11700b6850 con 0x7f11580039a0 2023-10-18T12:01:31.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.051+0000 7f1157fff640 1 -- 172.21.15.125:0/16227435 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v81)=0 v81) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1160044020 con 0x7f11580039a0 2023-10-18T12:01:31.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f115c06b0d0 msgr2=0x7f115c06d590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:31.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f115c06b0d0 0x7f115c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 msgr2=0x7f11700b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:31.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 0x7f11700b57f0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f1160007010 tx=0x7f11600015f0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 shutdown_connections 2023-10-18T12:01:31.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7f115c06b0d0 0x7f115c06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b0ea0 0x7f11700b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f117015e050 0x7f117015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 --2- 172.21.15.125:0/16227435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11580039a0 0x7f11700b57f0 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.053+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 >> 172.21.15.125:0/16227435 conn(0x7f11700d0d60 msgr2=0x7f11700ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:31.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.054+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 shutdown_connections 2023-10-18T12:01:31.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.054+0000 7f117601a640 1 -- 172.21.15.125:0/16227435 wait complete. 2023-10-18T12:01:31.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:31.068 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-18T12:01:31.086 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2023-10-18T12:01:31.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.230+0000 7fde8066b640 1 -- 172.21.15.125:0/949390722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde7815b2a0 msgr2=0x7fde780d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:31.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.230+0000 7fde8066b640 1 --2- 172.21.15.125:0/949390722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde7815b2a0 0x7fde780d1910 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7fde7815b680 tx=0x7fde6802bda0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.230+0000 7fde8066b640 1 -- 172.21.15.125:0/949390722 shutdown_connections 2023-10-18T12:01:31.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.230+0000 7fde8066b640 1 --2- 172.21.15.125:0/949390722 >> v2:172.21.15.139:3300/0 conn(0x7fde780ceb40 0x7fde780b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.230+0000 7fde8066b640 1 --2- 172.21.15.125:0/949390722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde7815b2a0 0x7fde780d1910 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.230+0000 7fde8066b640 1 -- 172.21.15.125:0/949390722 >> 172.21.15.125:0/949390722 conn(0x7fde780d0d60 msgr2=0x7fde780d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:31.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.230+0000 7fde8066b640 1 -- 172.21.15.125:0/949390722 shutdown_connections 2023-10-18T12:01:31.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.230+0000 7fde8066b640 1 -- 172.21.15.125:0/949390722 wait complete. 2023-10-18T12:01:31.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde8066b640 1 Processor -- start 2023-10-18T12:01:31.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde8066b640 1 -- start start 2023-10-18T12:01:31.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde8066b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde780ceb40 0x7fde7814f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:31.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde8066b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde780cf340 0x7fde7815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:31.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde8066b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde7815b2a0 0x7fde7815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:31.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde8066b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde781503e0 con 0x7fde780cf340 2023-10-18T12:01:31.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde8066b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7814fe40 con 0x7fde780ceb40 2023-10-18T12:01:31.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde8066b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7815bfd0 con 0x7fde7815b2a0 2023-10-18T12:01:31.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde7dc06640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde780cf340 0x7fde7815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:31.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde7dc06640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde780cf340 0x7fde7815cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39664/0 (socket says 172.21.15.125:39664) 2023-10-18T12:01:31.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.232+0000 7fde7dc06640 1 -- 172.21.15.125:0/1305151681 learned_addr learned my addr 172.21.15.125:0/1305151681 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:31.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde7e407640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde780ceb40 0x7fde7814f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:31.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde7ec08640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde7815b2a0 0x7fde7815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:31.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde7dc06640 1 -- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde7815b2a0 msgr2=0x7fde7815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:31.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde7dc06640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde7815b2a0 0x7fde7815b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde7dc06640 1 -- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde780ceb40 msgr2=0x7fde7814f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:31.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde7dc06640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde780ceb40 0x7fde7814f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:31.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde7dc06640 1 -- 172.21.15.125:0/1305151681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde68009560 con 0x7fde780cf340 2023-10-18T12:01:31.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde7dc06640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde780cf340 0x7fde7815cd40 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fde68009850 tx=0x7fde6802dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:31.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68003800 con 0x7fde780cf340 2023-10-18T12:01:31.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde68003a50 con 0x7fde780cf340 2023-10-18T12:01:31.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde8066b640 1 -- 172.21.15.125:0/1305151681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7815c2b0 con 0x7fde780cf340 2023-10-18T12:01:31.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68003e20 con 0x7fde780cf340 2023-10-18T12:01:31.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.233+0000 7fde8066b640 1 -- 172.21.15.125:0/1305151681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde78150000 con 0x7fde780cf340 2023-10-18T12:01:31.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.234+0000 7fde8066b640 1 -- 172.21.15.125:0/1305151681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde78150520 con 0x7fde780cf340 2023-10-18T12:01:31.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.235+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 140) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fde6803e450 con 0x7fde780cf340 2023-10-18T12:01:31.243 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.236+0000 7f266a043040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:01:31.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.235+0000 7fde6effd640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 0x7fde5c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:31.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.235+0000 7fde7e407640 1 -- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 msgr2=0x7fde5c06d590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3422518215 2023-10-18T12:01:31.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.235+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fde680ac880 con 0x7fde780cf340 2023-10-18T12:01:31.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.235+0000 7fde7e407640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 0x7fde5c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:31.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.240+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde78150520 con 0x7fde780cf340 2023-10-18T12:01:31.280 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.277+0000 7fd9f5811040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:01:31.282 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.280+0000 7f266a043040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:01:31.326 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.322+0000 7f266a043040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:01:31.327 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.321+0000 7fd9f5811040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:01:31.366 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.363+0000 7fd9f5811040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:01:31.375 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.373+0000 7f266a043040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:01:31.417 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.414+0000 7f266a043040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:01:31.418 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.415+0000 7fd9f5811040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:01:31.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.428+0000 7fde8066b640 1 -- 172.21.15.125:0/1305151681 --> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fde78150520 con 0x7fde5c06b0d0 2023-10-18T12:01:31.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.436+0000 7fde7e407640 1 -- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 msgr2=0x7fde5c06d590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3422518215 2023-10-18T12:01:31.438 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.436+0000 7fde7e407640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 0x7fde5c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:01:31.461 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.458+0000 7fd9f5811040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:01:31.517 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.514+0000 7f266a043040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:01:31.557 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.555+0000 7fd9f5811040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:01:31.569 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.567+0000 7f266a043040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:01:31.609 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.607+0000 7fd9f5811040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:01:31.610 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.608+0000 7f266a043040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:01:31.650 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.648+0000 7fd9f5811040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:01:31.658 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.655+0000 7f266a043040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:01:31.697 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.695+0000 7fd9f5811040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:01:31.699 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:31.697+0000 7f266a043040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:01:31.739 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:31.736+0000 7fd9f5811040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:01:31.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.837+0000 7fde7e407640 1 -- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 msgr2=0x7fde5c06d590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3422518215 2023-10-18T12:01:31.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:31.837+0000 7fde7e407640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 0x7fde5c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T12:01:32.013 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.011+0000 7f266a043040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:01:32.049 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.046+0000 7fd9f5811040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:01:32.145 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.142+0000 7f266a043040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:01:32.180 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.177+0000 7fd9f5811040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:01:32.236 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.233+0000 7f266a043040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:01:32.270 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.267+0000 7fd9f5811040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:01:32.277 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.275+0000 7f266a043040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:01:32.312 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.310+0000 7fd9f5811040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:01:32.371 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.369+0000 7f266a043040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:01:32.406 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.404+0000 7fd9f5811040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:01:32.462 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.459+0000 7f266a043040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:01:32.495 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.493+0000 7fd9f5811040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:01:32.598 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.596+0000 7f266a043040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:01:32.630 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.628+0000 7fd9f5811040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:01:32.639 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.637+0000 7f266a043040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:01:32.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:32.638+0000 7fde7e407640 1 -- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 msgr2=0x7fde5c06d590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/3422518215 2023-10-18T12:01:32.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:32.638+0000 7fde7e407640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 0x7fde5c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T12:01:32.672 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.669+0000 7fd9f5811040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:01:32.744 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:32.741+0000 7f266a043040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:01:32.775 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:32.772+0000 7fd9f5811040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:01:33.363 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.361+0000 7f266a043040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:01:33.391 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:33.389+0000 7fd9f5811040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:01:33.624 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.622+0000 7f266a043040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:01:33.650 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:33.648+0000 7fd9f5811040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:01:33.672 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.670+0000 7f266a043040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:01:33.698 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:33.696+0000 7fd9f5811040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:01:33.771 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.769+0000 7f266a043040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:01:33.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:33.791+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 141) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fde68077720 con 0x7fde780cf340 2023-10-18T12:01:33.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:33.791+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 msgr2=0x7fde5c06d590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:33.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:33.791+0000 7fde6effd640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/3422518215,v1:172.21.15.139:6833/3422518215] conn(0x7fde5c06b0d0 0x7fde5c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:33.802 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:33.800+0000 7fd9f5811040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:01:33.928 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.928 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.928 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.929 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.929 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.929 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.929 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.929 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.929 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.930 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.925+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.936 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.934+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.936 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.934+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.937 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.934+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.937 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.934+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.937 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.934+0000 7f25e63a5640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.938 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.935+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.938 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.935+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.938 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.935+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.938 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.935+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:33.939 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:33.935+0000 7f25e93ab640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:34.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.796+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 142) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fde68076e00 con 0x7fde780cf340 2023-10-18T12:01:34.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.796+0000 7fde6effd640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fde5c0357d0 0x7fde5c077c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:34.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.796+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 --> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fde78150520 con 0x7fde5c0357d0 2023-10-18T12:01:34.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.797+0000 7fde7e407640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fde5c0357d0 0x7fde5c077c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:34.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.800+0000 7fde7e407640 1 --2- 172.21.15.125:0/1305151681 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fde5c0357d0 0x7fde5c077c30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fde74000f60 tx=0x7fde74007e30 comp rx=0 tx=0).ready entity=mgr.9659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:34.805 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:34.802+0000 7f25f03f9640 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2023-10-18T12:01:34.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.803+0000 7fde6effd640 1 -- 172.21.15.125:0/1305151681 <== mgr.9659 v2:172.21.15.139:6832/2483056868 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) v1 ==== 79+0+0 (secure 0 0 0) 0x7fde78150520 con 0x7fde5c0357d0 2023-10-18T12:01:34.806 INFO:teuthology.orchestra.run.smithi125.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2023-10-18T12:01:34.811 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-18T12:01:34.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:34.812 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-18T12:01:34.831 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T12:01:34.978 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.975+0000 7f5efe18e640 1 -- 172.21.15.125:0/2625605591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ee00047c0 msgr2=0x7f5ee0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:34.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.975+0000 7f5efe18e640 1 --2- 172.21.15.125:0/2625605591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ee00047c0 0x7f5ee0004c40 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00baf0 tx=0x7f5ee00057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:34.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.975+0000 7f5efe18e640 1 -- 172.21.15.125:0/2625605591 shutdown_connections 2023-10-18T12:01:34.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.975+0000 7f5efe18e640 1 --2- 172.21.15.125:0/2625605591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ee00047c0 0x7f5ee0004c40 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:34.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.975+0000 7f5efe18e640 1 -- 172.21.15.125:0/2625605591 >> 172.21.15.125:0/2625605591 conn(0x7f5ef8069930 msgr2=0x7f5ef8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:34.979 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.975+0000 7f5efe18e640 1 -- 172.21.15.125:0/2625605591 shutdown_connections 2023-10-18T12:01:34.980 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.975+0000 7f5efe18e640 1 -- 172.21.15.125:0/2625605591 wait complete. 2023-10-18T12:01:34.982 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.976+0000 7f5efe18e640 1 Processor -- start 2023-10-18T12:01:34.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.977+0000 7f5efe18e640 1 -- start start 2023-10-18T12:01:34.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.977+0000 7f5efe18e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ee00047c0 0x7f5ef8154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:34.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.977+0000 7f5efe18e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ef80a6540 0x7f5ef8138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:34.983 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.977+0000 7f5efe18e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ef8138e70 0x7f5ef81f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:34.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.977+0000 7f5efe18e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef81f03e0 con 0x7f5ef80a6540 2023-10-18T12:01:34.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.977+0000 7f5efe18e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8154830 con 0x7f5ee00047c0 2023-10-18T12:01:34.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.977+0000 7f5efe18e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8154b10 con 0x7f5ef8138e70 2023-10-18T12:01:34.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.977+0000 7f5ef6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ef80a6540 0x7f5ef8138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:34.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ef80a6540 0x7f5ef8138930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39692/0 (socket says 172.21.15.125:39692) 2023-10-18T12:01:34.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef6ffd640 1 -- 172.21.15.125:0/23630276 learned_addr learned my addr 172.21.15.125:0/23630276 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:34.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef77fe640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ee00047c0 0x7f5ef8154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:34.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef6ffd640 1 -- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ef8138e70 msgr2=0x7f5ef81f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:34.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef6ffd640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ef8138e70 0x7f5ef81f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:34.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef6ffd640 1 -- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ee00047c0 msgr2=0x7f5ef8154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:34.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef6ffd640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ee00047c0 0x7f5ef8154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:34.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef6ffd640 1 -- 172.21.15.125:0/23630276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef8154df0 con 0x7f5ef80a6540 2023-10-18T12:01:34.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef77fe640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ee00047c0 0x7f5ef8154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:34.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5ef6ffd640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ef80a6540 0x7f5ef8138930 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f5eec038040 tx=0x7f5eec036700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:34.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5edffff640 1 -- 172.21.15.125:0/23630276 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec036c80 con 0x7f5ef80a6540 2023-10-18T12:01:34.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5edffff640 1 -- 172.21.15.125:0/23630276 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5eec004030 con 0x7f5ef80a6540 2023-10-18T12:01:34.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef81550d0 con 0x7f5ef80a6540 2023-10-18T12:01:34.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5edffff640 1 -- 172.21.15.125:0/23630276 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec004420 con 0x7f5ef80a6540 2023-10-18T12:01:34.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.978+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef80a6ba0 con 0x7f5ef80a6540 2023-10-18T12:01:34.987 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.979+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef80690e0 con 0x7f5ef80a6540 2023-10-18T12:01:34.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.979+0000 7f5edffff640 1 -- 172.21.15.125:0/23630276 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 142) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f5eec0035f0 con 0x7f5ef80a6540 2023-10-18T12:01:34.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.980+0000 7f5edffff640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5ee406ad10 0x7f5ee406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:34.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.980+0000 7f5edffff640 1 -- 172.21.15.125:0/23630276 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7f5eec0ab980 con 0x7f5ef80a6540 2023-10-18T12:01:34.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.980+0000 7f5ef77fe640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5ee406ad10 0x7f5ee406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:34.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.980+0000 7f5ef77fe640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5ee406ad10 0x7f5ee406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006530 tx=0x7f5ee8006b30 comp rx=0 tx=0).ready entity=mgr.9659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:34.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:34.984+0000 7f5edffff640 1 -- 172.21.15.125:0/23630276 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5eec0766e0 con 0x7f5ef80a6540 2023-10-18T12:01:35.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.196+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f5ef80cc460 con 0x7f5ef80a6540 2023-10-18T12:01:35.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.842+0000 7f5edffff640 1 -- 172.21.15.125:0/23630276 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v143) v1 ==== 93+0+0 (secure 0 0 0) 0x7f5eec079d00 con 0x7f5ef80a6540 2023-10-18T12:01:35.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5ee406ad10 msgr2=0x7f5ee406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:35.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5ee406ad10 0x7f5ee406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006530 tx=0x7f5ee8006b30 comp rx=0 tx=0).stop 2023-10-18T12:01:35.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ef80a6540 msgr2=0x7f5ef8138930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:35.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ef80a6540 0x7f5ef8138930 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f5eec038040 tx=0x7f5eec036700 comp rx=0 tx=0).stop 2023-10-18T12:01:35.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 shutdown_connections 2023-10-18T12:01:35.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5ee406ad10 0x7f5ee406d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:35.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ef8138e70 0x7f5ef81f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:35.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ef80a6540 0x7f5ef8138930 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:35.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 --2- 172.21.15.125:0/23630276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ee00047c0 0x7f5ef8154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:35.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 >> 172.21.15.125:0/23630276 conn(0x7f5ef8069930 msgr2=0x7f5ef80ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:35.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 shutdown_connections 2023-10-18T12:01:35.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:35.846+0000 7f5efe18e640 1 -- 172.21.15.125:0/23630276 wait complete. 2023-10-18T12:01:35.864 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T12:01:35.864 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:01:35.913 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:01:35.954 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T12:01:35.954 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:01:35.954 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:35.970 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:36.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.113+0000 7f288fb1d640 1 -- 172.21.15.125:0/498237985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 msgr2=0x7f28880b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.113+0000 7f288fb1d640 1 --2- 172.21.15.125:0/498237985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 0x7f28880b0650 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f288808a810 tx=0x7f288002c260 comp rx=0 tx=0).stop 2023-10-18T12:01:36.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.113+0000 7f288fb1d640 1 -- 172.21.15.125:0/498237985 shutdown_connections 2023-10-18T12:01:36.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.113+0000 7f288fb1d640 1 --2- 172.21.15.125:0/498237985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 0x7f28880b0650 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.113+0000 7f288fb1d640 1 --2- 172.21.15.125:0/498237985 >> v2:172.21.15.139:3300/0 conn(0x7f28880cf340 0x7f28880d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.113+0000 7f288fb1d640 1 -- 172.21.15.125:0/498237985 >> 172.21.15.125:0/498237985 conn(0x7f28880d0d60 msgr2=0x7f28880d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:36.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.113+0000 7f288fb1d640 1 -- 172.21.15.125:0/498237985 shutdown_connections 2023-10-18T12:01:36.117 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.113+0000 7f288fb1d640 1 -- 172.21.15.125:0/498237985 wait complete. 2023-10-18T12:01:36.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.114+0000 7f288fb1d640 1 Processor -- start 2023-10-18T12:01:36.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.114+0000 7f288fb1d640 1 -- start start 2023-10-18T12:01:36.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288fb1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 0x7f288815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288fb1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28880cf340 0x7f2888158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288fb1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f288815b2a0 0x7f28880c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288fb1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288809ab70 con 0x7f28880ceb40 2023-10-18T12:01:36.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288fb1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288815d620 con 0x7f288815b2a0 2023-10-18T12:01:36.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288fb1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28880af2d0 con 0x7f28880cf340 2023-10-18T12:01:36.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 0x7f288815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 0x7f288815d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39718/0 (socket says 172.21.15.125:39718) 2023-10-18T12:01:36.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 -- 172.21.15.125:0/2974868799 learned_addr learned my addr 172.21.15.125:0/2974868799 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:36.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288e0ba640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f288815b2a0 0x7f28880c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d0b8640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28880cf340 0x7f2888158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 -- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28880cf340 msgr2=0x7f2888158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28880cf340 0x7f2888158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 -- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f288815b2a0 msgr2=0x7f28880c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.123 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f288815b2a0 0x7f28880c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 -- 172.21.15.125:0/2974868799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880009900 con 0x7f28880ceb40 2023-10-18T12:01:36.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.115+0000 7f288d8b9640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 0x7f288815d0e0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f287800e9f0 tx=0x7f287800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:36.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.116+0000 7f287e7fc640 1 -- 172.21.15.125:0/2974868799 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2878009700 con 0x7f28880ceb40 2023-10-18T12:01:36.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.116+0000 7f287e7fc640 1 -- 172.21.15.125:0/2974868799 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f287800ce70 con 0x7f28880ceb40 2023-10-18T12:01:36.124 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.116+0000 7f287e7fc640 1 -- 172.21.15.125:0/2974868799 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2878010670 con 0x7f28880ceb40 2023-10-18T12:01:36.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.116+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2888158680 con 0x7f28880ceb40 2023-10-18T12:01:36.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.116+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28880af800 con 0x7f28880ceb40 2023-10-18T12:01:36.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.117+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28880690e0 con 0x7f28880ceb40 2023-10-18T12:01:36.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.117+0000 7f287e7fc640 1 -- 172.21.15.125:0/2974868799 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f28780107d0 con 0x7f28880ceb40 2023-10-18T12:01:36.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.118+0000 7f287e7fc640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f286c06b080 0x7f286c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.118+0000 7f288d0b8640 1 -- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f286c06b080 msgr2=0x7f286c06d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:36.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.118+0000 7f287e7fc640 1 -- 172.21.15.125:0/2974868799 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7f2878014070 con 0x7f28880ceb40 2023-10-18T12:01:36.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.118+0000 7f288d0b8640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f286c06b080 0x7f286c06d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:36.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.126+0000 7f287e7fc640 1 -- 172.21.15.125:0/2974868799 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2878051ca0 con 0x7f28880ceb40 2023-10-18T12:01:36.236 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:36.233+0000 7f4940c15040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:01:36.248 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:36.246+0000 7f5b64e2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:01:36.281 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:36.278+0000 7f4940c15040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:01:36.295 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:36.292+0000 7f5b64e2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:01:36.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.299+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28880aedd0 con 0x7f28880ceb40 2023-10-18T12:01:36.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.302+0000 7f287e7fc640 1 -- 172.21.15.125:0/2974868799 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v247) v1 ==== 74+0+21215 (secure 0 0 0) 0x7f2878055480 con 0x7f28880ceb40 2023-10-18T12:01:36.304 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:36.305 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":247,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:33.783850+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6833/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6832/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3037918695":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3823693717":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3119295615":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3910260091":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/558106977":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2896936218":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/695463051":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2469983511":"2023-10-19T12:01:33.783814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:36.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f286c06b080 msgr2=0x7f286c06d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:36.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f286c06b080 0x7f286c06d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 msgr2=0x7f288815d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 0x7f288815d0e0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f287800e9f0 tx=0x7f287800eeb0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 shutdown_connections 2023-10-18T12:01:36.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f286c06b080 0x7f286c06d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f288815b2a0 0x7f28880c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f28880cf340 0x7f2888158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 --2- 172.21.15.125:0/2974868799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28880ceb40 0x7f288815d0e0 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 >> 172.21.15.125:0/2974868799 conn(0x7f28880d0d60 msgr2=0x7f288815a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:36.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 shutdown_connections 2023-10-18T12:01:36.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.304+0000 7f288fb1d640 1 -- 172.21.15.125:0/2974868799 wait complete. 2023-10-18T12:01:36.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:01:06.989961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '241', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:01:07.416244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '242', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-18T12:01:18.203802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '245', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-18T12:01:18.758795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '246', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:01:36.319 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:01:36.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.424+0000 7fdf7866f640 1 -- 172.21.15.125:0/3877185396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 msgr2=0x7fdf7006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.424+0000 7fdf7866f640 1 --2- 172.21.15.125:0/3877185396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 0x7fdf7006d260 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fdf60005fd0 tx=0x7fdf6002be20 comp rx=0 tx=0).stop 2023-10-18T12:01:36.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.424+0000 7fdf7866f640 1 -- 172.21.15.125:0/3877185396 shutdown_connections 2023-10-18T12:01:36.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.424+0000 7fdf7866f640 1 --2- 172.21.15.125:0/3877185396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 0x7fdf7006d260 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.424+0000 7fdf7866f640 1 -- 172.21.15.125:0/3877185396 >> 172.21.15.125:0/3877185396 conn(0x7fdf700b0670 msgr2=0x7fdf700b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:36.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.424+0000 7fdf7866f640 1 -- 172.21.15.125:0/3877185396 shutdown_connections 2023-10-18T12:01:36.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.424+0000 7fdf7866f640 1 -- 172.21.15.125:0/3877185396 wait complete. 2023-10-18T12:01:36.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.425+0000 7fdf7866f640 1 Processor -- start 2023-10-18T12:01:36.430 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.425+0000 7fdf7866f640 1 -- start start 2023-10-18T12:01:36.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7866f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 0x7fdf700b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7866f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf70159060 0x7fdf701f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7866f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf7015ccf0 0x7fdf700b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7866f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700c8ae0 con 0x7fdf700ce240 2023-10-18T12:01:36.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7866f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b3ab0 con 0x7fdf70159060 2023-10-18T12:01:36.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7866f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b3d90 con 0x7fdf7015ccf0 2023-10-18T12:01:36.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 0x7fdf700b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 0x7fdf700b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39734/0 (socket says 172.21.15.125:39734) 2023-10-18T12:01:36.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 -- 172.21.15.125:0/2716077921 learned_addr learned my addr 172.21.15.125:0/2716077921 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:36.432 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf75c0a640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf70159060 0x7fdf701f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf76c0c640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf7015ccf0 0x7fdf700b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 -- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf7015ccf0 msgr2=0x7fdf700b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf7015ccf0 0x7fdf700b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 -- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf70159060 msgr2=0x7fdf701f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf70159060 0x7fdf701f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 -- 172.21.15.125:0/2716077921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60009560 con 0x7fdf700ce240 2023-10-18T12:01:36.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7640b640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 0x7fdf700b33d0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c450 tx=0x7fdf6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:36.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf66ffd640 1 -- 172.21.15.125:0/2716077921 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0052d0 con 0x7fdf700ce240 2023-10-18T12:01:36.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf66ffd640 1 -- 172.21.15.125:0/2716077921 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf6c005430 con 0x7fdf700ce240 2023-10-18T12:01:36.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf66ffd640 1 -- 172.21.15.125:0/2716077921 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c012890 con 0x7fdf700ce240 2023-10-18T12:01:36.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf700b4070 con 0x7fdf700ce240 2023-10-18T12:01:36.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.426+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf701f15b0 con 0x7fdf700ce240 2023-10-18T12:01:36.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.427+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf700675f0 con 0x7fdf700ce240 2023-10-18T12:01:36.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.428+0000 7fdf66ffd640 1 -- 172.21.15.125:0/2716077921 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fdf6c0035f0 con 0x7fdf700ce240 2023-10-18T12:01:36.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.428+0000 7fdf66ffd640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fdf5406b080 0x7fdf5406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.429+0000 7fdf75c0a640 1 -- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fdf5406b080 msgr2=0x7fdf5406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:36.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.429+0000 7fdf66ffd640 1 -- 172.21.15.125:0/2716077921 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7fdf6c089c30 con 0x7fdf700ce240 2023-10-18T12:01:36.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.429+0000 7fdf75c0a640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fdf5406b080 0x7fdf5406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:36.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.433+0000 7fdf66ffd640 1 -- 172.21.15.125:0/2716077921 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf6c0543a0 con 0x7fdf700ce240 2023-10-18T12:01:36.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.609+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf7006a830 con 0x7fdf700ce240 2023-10-18T12:01:36.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.609+0000 7fdf66ffd640 1 -- 172.21.15.125:0/2716077921 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v247) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf6c0579c0 con 0x7fdf700ce240 2023-10-18T12:01:36.612 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:01:36.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fdf5406b080 msgr2=0x7fdf5406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:36.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fdf5406b080 0x7fdf5406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 msgr2=0x7fdf700b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 0x7fdf700b33d0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c450 tx=0x7fdf6c00c910 comp rx=0 tx=0).stop 2023-10-18T12:01:36.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 shutdown_connections 2023-10-18T12:01:36.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fdf5406b080 0x7fdf5406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf70159060 0x7fdf701f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf7015ccf0 0x7fdf700b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 --2- 172.21.15.125:0/2716077921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf700ce240 0x7fdf700b33d0 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 >> 172.21.15.125:0/2716077921 conn(0x7fdf700b0670 msgr2=0x7fdf7006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:36.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 shutdown_connections 2023-10-18T12:01:36.616 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.611+0000 7fdf7866f640 1 -- 172.21.15.125:0/2716077921 wait complete. 2023-10-18T12:01:36.625 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:01:36.674 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:36.671+0000 7f4940c15040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:01:36.709 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:36.707+0000 7f5b64e2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:01:36.717 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:36.715+0000 7f4940c15040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:01:36.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.730+0000 7fb173ec9640 1 -- 172.21.15.125:0/2563376820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c0cf340 msgr2=0x7fb16c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.730+0000 7fb173ec9640 1 --2- 172.21.15.125:0/2563376820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c0cf340 0x7fb16c0d1910 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7fb15c005fd0 tx=0x7fb15c02bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:36.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.730+0000 7fb173ec9640 1 -- 172.21.15.125:0/2563376820 shutdown_connections 2023-10-18T12:01:36.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.730+0000 7fb173ec9640 1 --2- 172.21.15.125:0/2563376820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16c15b2a0 0x7fb16c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.730+0000 7fb173ec9640 1 --2- 172.21.15.125:0/2563376820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c0cf340 0x7fb16c0d1910 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.730+0000 7fb173ec9640 1 -- 172.21.15.125:0/2563376820 >> 172.21.15.125:0/2563376820 conn(0x7fb16c0d0d60 msgr2=0x7fb16c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:36.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.730+0000 7fb173ec9640 1 -- 172.21.15.125:0/2563376820 shutdown_connections 2023-10-18T12:01:36.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.730+0000 7fb173ec9640 1 -- 172.21.15.125:0/2563376820 wait complete. 2023-10-18T12:01:36.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb173ec9640 1 Processor -- start 2023-10-18T12:01:36.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb173ec9640 1 -- start start 2023-10-18T12:01:36.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb173ec9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16c0ceb40 0x7fb16c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb173ec9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16c0cf340 0x7fb16c15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb173ec9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c15b2a0 0x7fb16c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb173ec9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c0c8b00 con 0x7fb16c15b2a0 2023-10-18T12:01:36.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb173ec9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c0c8da0 con 0x7fb16c0cf340 2023-10-18T12:01:36.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb173ec9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c15c180 con 0x7fb16c0ceb40 2023-10-18T12:01:36.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb172466640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c15b2a0 0x7fb16c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb172466640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c15b2a0 0x7fb16c0c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39754/0 (socket says 172.21.15.125:39754) 2023-10-18T12:01:36.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb172466640 1 -- 172.21.15.125:0/3638143189 learned_addr learned my addr 172.21.15.125:0/3638143189 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:36.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb171c65640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16c0ceb40 0x7fb16c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.732+0000 7fb171464640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16c0cf340 0x7fb16c15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:36.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb172466640 1 -- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16c0ceb40 msgr2=0x7fb16c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb172466640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16c0ceb40 0x7fb16c156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb172466640 1 -- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16c0cf340 msgr2=0x7fb16c15c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb172466640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16c0cf340 0x7fb16c15c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb172466640 1 -- 172.21.15.125:0/3638143189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c009560 con 0x7fb16c15b2a0 2023-10-18T12:01:36.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb172466640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c15b2a0 0x7fb16c0c7e30 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fb16400c790 tx=0x7fb16400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:36.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb1627fc640 1 -- 172.21.15.125:0/3638143189 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400cea0 con 0x7fb16c15b2a0 2023-10-18T12:01:36.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb1627fc640 1 -- 172.21.15.125:0/3638143189 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb164003b70 con 0x7fb16c15b2a0 2023-10-18T12:01:36.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c15c460 con 0x7fb16c15b2a0 2023-10-18T12:01:36.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb1627fc640 1 -- 172.21.15.125:0/3638143189 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400a850 con 0x7fb16c15b2a0 2023-10-18T12:01:36.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.733+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c0c9080 con 0x7fb16c15b2a0 2023-10-18T12:01:36.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.734+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c15c680 con 0x7fb16c15b2a0 2023-10-18T12:01:36.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.735+0000 7fb1627fc640 1 -- 172.21.15.125:0/3638143189 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fb16400a9b0 con 0x7fb16c15b2a0 2023-10-18T12:01:36.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.735+0000 7fb1627fc640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fb15006b080 0x7fb15006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:36.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.735+0000 7fb171c65640 1 -- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fb15006b080 msgr2=0x7fb15006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:36.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.735+0000 7fb1627fc640 1 -- 172.21.15.125:0/3638143189 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7fb164089ff0 con 0x7fb16c15b2a0 2023-10-18T12:01:36.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.735+0000 7fb171c65640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fb15006b080 0x7fb15006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:36.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.739+0000 7fb1627fc640 1 -- 172.21.15.125:0/3638143189 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb164054760 con 0x7fb16c15b2a0 2023-10-18T12:01:36.756 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:36.754+0000 7f5b64e2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:01:36.760 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:36.758+0000 7f4940c15040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:01:36.801 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:36.798+0000 7f5b64e2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:01:36.812 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:36.810+0000 7f4940c15040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:01:36.854 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:36.852+0000 7f5b64e2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:01:36.855 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:36.852+0000 7f4940c15040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:01:36.897 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:36.895+0000 7f5b64e2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:01:36.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.914+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb16c0690e0 con 0x7fb16c15b2a0 2023-10-18T12:01:36.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.914+0000 7fb1627fc640 1 -- 172.21.15.125:0/3638143189 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v247) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb164057d80 con 0x7fb16c15b2a0 2023-10-18T12:01:36.917 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:01:36.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fb15006b080 msgr2=0x7fb15006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:36.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fb15006b080 0x7fb15006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c15b2a0 msgr2=0x7fb16c0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:36.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c15b2a0 0x7fb16c0c7e30 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fb16400c790 tx=0x7fb16400cc50 comp rx=0 tx=0).stop 2023-10-18T12:01:36.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 shutdown_connections 2023-10-18T12:01:36.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fb15006b080 0x7fb15006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16c0cf340 0x7fb16c15c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb16c15b2a0 0x7fb16c0c7e30 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 --2- 172.21.15.125:0/3638143189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16c0ceb40 0x7fb16c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:36.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 >> 172.21.15.125:0/3638143189 conn(0x7fb16c0d0d60 msgr2=0x7fb16c15a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:36.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 shutdown_connections 2023-10-18T12:01:36.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:36.917+0000 7fb173ec9640 1 -- 172.21.15.125:0/3638143189 wait complete. 2023-10-18T12:01:36.930 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:01:36.953 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:36.950+0000 7f4940c15040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:01:37.001 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:36.998+0000 7f5b64e2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:01:37.006 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.003+0000 7f4940c15040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:01:37.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 -- 172.21.15.125:0/3076732114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0cf340 msgr2=0x7f271c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 --2- 172.21.15.125:0/3076732114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0cf340 0x7f271c0d1910 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f271c08a120 tx=0x7f270c02bd10 comp rx=0 tx=0).stop 2023-10-18T12:01:37.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 -- 172.21.15.125:0/3076732114 shutdown_connections 2023-10-18T12:01:37.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 --2- 172.21.15.125:0/3076732114 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f271c0ceb40 0x7f271c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 --2- 172.21.15.125:0/3076732114 >> v2:172.21.15.139:3300/0 conn(0x7f271c15b2a0 0x7f271c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 --2- 172.21.15.125:0/3076732114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0cf340 0x7f271c0d1910 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 -- 172.21.15.125:0/3076732114 >> 172.21.15.125:0/3076732114 conn(0x7f271c0d0d60 msgr2=0x7f271c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:37.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 -- 172.21.15.125:0/3076732114 shutdown_connections 2023-10-18T12:01:37.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.034+0000 7f2723439640 1 -- 172.21.15.125:0/3076732114 wait complete. 2023-10-18T12:01:37.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.035+0000 7f2723439640 1 Processor -- start 2023-10-18T12:01:37.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.035+0000 7f2723439640 1 -- start start 2023-10-18T12:01:37.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f2723439640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0ceb40 0x7f271c1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f2723439640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c0cf340 0x7f271c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f2723439640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f271c15b2a0 0x7f271c0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f2723439640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c15d090 con 0x7f271c0ceb40 2023-10-18T12:01:37.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f2723439640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c156b10 con 0x7f271c0cf340 2023-10-18T12:01:37.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f2723439640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c15c510 con 0x7f271c15b2a0 2023-10-18T12:01:37.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27211d5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0ceb40 0x7f271c1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27211d5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0ceb40 0x7f271c1526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39762/0 (socket says 172.21.15.125:39762) 2023-10-18T12:01:37.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27211d5640 1 -- 172.21.15.125:0/2613682052 learned_addr learned my addr 172.21.15.125:0/2613682052 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:37.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27209d4640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c0cf340 0x7f271c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27219d6640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f271c15b2a0 0x7f271c0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27211d5640 1 -- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f271c15b2a0 msgr2=0x7f271c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27211d5640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f271c15b2a0 0x7f271c0b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27211d5640 1 -- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c0cf340 msgr2=0x7f271c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27211d5640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c0cf340 0x7f271c15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27211d5640 1 -- 172.21.15.125:0/2613682052 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c009560 con 0x7f271c0ceb40 2023-10-18T12:01:37.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.036+0000 7f27219d6640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f271c15b2a0 0x7f271c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:37.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.037+0000 7f27211d5640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0ceb40 0x7f271c1526d0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f270c009850 tx=0x7f270c02d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:37.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.037+0000 7f2711ffb640 1 -- 172.21.15.125:0/2613682052 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c02dde0 con 0x7f271c0ceb40 2023-10-18T12:01:37.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.037+0000 7f2711ffb640 1 -- 172.21.15.125:0/2613682052 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f270c0036e0 con 0x7f271c0ceb40 2023-10-18T12:01:37.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.037+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c15c7f0 con 0x7f271c0ceb40 2023-10-18T12:01:37.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.037+0000 7f2711ffb640 1 -- 172.21.15.125:0/2613682052 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c003ab0 con 0x7f271c0ceb40 2023-10-18T12:01:37.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.037+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c15cdb0 con 0x7f271c0ceb40 2023-10-18T12:01:37.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.038+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c0690e0 con 0x7f271c0ceb40 2023-10-18T12:01:37.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.038+0000 7f2711ffb640 1 -- 172.21.15.125:0/2613682052 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f270c003c10 con 0x7f271c0ceb40 2023-10-18T12:01:37.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.039+0000 7f2711ffb640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f270006b080 0x7f270006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.039+0000 7f27209d4640 1 -- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f270006b080 msgr2=0x7f270006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:37.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.039+0000 7f2711ffb640 1 -- 172.21.15.125:0/2613682052 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7f270c0ac820 con 0x7f271c0ceb40 2023-10-18T12:01:37.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.039+0000 7f27209d4640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f270006b080 0x7f270006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:37.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.043+0000 7f2711ffb640 1 -- 172.21.15.125:0/2613682052 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f270c077000 con 0x7f271c0ceb40 2023-10-18T12:01:37.048 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.045+0000 7f4940c15040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:01:37.066 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.063+0000 7f5b64e2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:01:37.096 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.094+0000 7f4940c15040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:01:37.115 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.113+0000 7f5b64e2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:01:37.139 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.136+0000 7f4940c15040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:01:37.164 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.162+0000 7f5b64e2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:01:37.208 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.205+0000 7f5b64e2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:01:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.232+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f271c0d0b40 con 0x7f271c0ceb40 2023-10-18T12:01:37.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.233+0000 7f2711ffb640 1 -- 172.21.15.125:0/2613682052 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v247) v1 ==== 100+0+11 (secure 0 0 0) 0x7f270c07a620 con 0x7f271c0ceb40 2023-10-18T12:01:37.236 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:01:37.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f270006b080 msgr2=0x7f270006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:37.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f270006b080 0x7f270006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0ceb40 msgr2=0x7f271c1526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0ceb40 0x7f271c1526d0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f270c009850 tx=0x7f270c02d9e0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 shutdown_connections 2023-10-18T12:01:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f270006b080 0x7f270006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c0cf340 0x7f271c15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f271c15b2a0 0x7f271c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 --2- 172.21.15.125:0/2613682052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f271c0ceb40 0x7f271c1526d0 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 >> 172.21.15.125:0/2613682052 conn(0x7f271c0d0d60 msgr2=0x7f271c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 shutdown_connections 2023-10-18T12:01:37.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.235+0000 7f2723439640 1 -- 172.21.15.125:0/2613682052 wait complete. 2023-10-18T12:01:37.248 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-18T12:01:37.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.353+0000 7f3d3ce10640 1 -- 172.21.15.125:0/1233783853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3815b2a0 msgr2=0x7f3d380d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.353+0000 7f3d3ce10640 1 --2- 172.21.15.125:0/1233783853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3815b2a0 0x7f3d380d1910 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009830 tx=0x7f3d2002bd60 comp rx=0 tx=0).stop 2023-10-18T12:01:37.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.353+0000 7f3d3ce10640 1 -- 172.21.15.125:0/1233783853 shutdown_connections 2023-10-18T12:01:37.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.353+0000 7f3d3ce10640 1 --2- 172.21.15.125:0/1233783853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3815b2a0 0x7f3d380d1910 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.354+0000 7f3d3ce10640 1 -- 172.21.15.125:0/1233783853 >> 172.21.15.125:0/1233783853 conn(0x7f3d380d0d60 msgr2=0x7f3d380d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:37.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.354+0000 7f3d3ce10640 1 -- 172.21.15.125:0/1233783853 shutdown_connections 2023-10-18T12:01:37.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.354+0000 7f3d3ce10640 1 -- 172.21.15.125:0/1233783853 wait complete. 2023-10-18T12:01:37.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.355+0000 7f3d3ce10640 1 Processor -- start 2023-10-18T12:01:37.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.355+0000 7f3d3ce10640 1 -- start start 2023-10-18T12:01:37.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3ce10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d380ceb40 0x7f3d3814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3ce10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d380cf340 0x7f3d381568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3ce10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d3815b2a0 0x7f3d380b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3ce10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380c87c0 con 0x7f3d380cf340 2023-10-18T12:01:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3ce10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380c7ed0 con 0x7f3d380ceb40 2023-10-18T12:01:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3ce10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380c8170 con 0x7f3d3815b2a0 2023-10-18T12:01:37.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d2ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d380cf340 0x7f3d381568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d380ceb40 0x7f3d3814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d2ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d380cf340 0x7f3d381568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39778/0 (socket says 172.21.15.125:39778) 2023-10-18T12:01:37.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2644545754 learned_addr learned my addr 172.21.15.125:0/2644545754 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:37.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d36d9d640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d3815b2a0 0x7f3d380b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3659c640 1 -- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d3815b2a0 msgr2=0x7f3d380b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3659c640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d3815b2a0 0x7f3d380b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3659c640 1 -- 172.21.15.125:0/2644545754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d380cf340 msgr2=0x7f3d381568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3659c640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d380cf340 0x7f3d381568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3659c640 1 -- 172.21.15.125:0/2644545754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d20009490 con 0x7f3d380ceb40 2023-10-18T12:01:37.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d2ffff640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d380cf340 0x7f3d381568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:37.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3659c640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d380ceb40 0x7f3d3814d2c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c390 tx=0x7f3d2800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:37.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d2effd640 1 -- 172.21.15.125:0/2644545754 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280052d0 con 0x7f3d380ceb40 2023-10-18T12:01:37.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d2effd640 1 -- 172.21.15.125:0/2644545754 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d280035f0 con 0x7f3d380ceb40 2023-10-18T12:01:37.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.356+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d380c8450 con 0x7f3d380ceb40 2023-10-18T12:01:37.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.357+0000 7f3d2effd640 1 -- 172.21.15.125:0/2644545754 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28003a00 con 0x7f3d380ceb40 2023-10-18T12:01:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.357+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d380caaa0 con 0x7f3d380ceb40 2023-10-18T12:01:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.357+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d380690e0 con 0x7f3d380ceb40 2023-10-18T12:01:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.358+0000 7f3d2effd640 1 -- 172.21.15.125:0/2644545754 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f3d28003b60 con 0x7f3d380ceb40 2023-10-18T12:01:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.358+0000 7f3d2effd640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f3d1806b080 0x7f3d1806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.358+0000 7f3d2ffff640 1 -- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f3d1806b080 msgr2=0x7f3d1806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:37.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.358+0000 7f3d2effd640 1 -- 172.21.15.125:0/2644545754 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7f3d28089690 con 0x7f3d380ceb40 2023-10-18T12:01:37.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.358+0000 7f3d2ffff640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f3d1806b080 0x7f3d1806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:37.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.363+0000 7f3d2effd640 1 -- 172.21.15.125:0/2644545754 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d28053f60 con 0x7f3d380ceb40 2023-10-18T12:01:37.456 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.454+0000 7f4940c15040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:01:37.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.536+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d380d0b40 con 0x7f3d380ceb40 2023-10-18T12:01:37.539 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.536+0000 7f5b64e2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:01:37.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.537+0000 7f3d2effd640 1 -- 172.21.15.125:0/2644545754 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v247) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3d28057580 con 0x7f3d380ceb40 2023-10-18T12:01:37.540 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:01:37.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f3d1806b080 msgr2=0x7f3d1806d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:37.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f3d1806b080 0x7f3d1806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d380ceb40 msgr2=0x7f3d3814d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d380ceb40 0x7f3d3814d2c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c390 tx=0x7f3d2800c850 comp rx=0 tx=0).stop 2023-10-18T12:01:37.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 shutdown_connections 2023-10-18T12:01:37.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f3d1806b080 0x7f3d1806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d3815b2a0 0x7f3d380b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d380cf340 0x7f3d381568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 --2- 172.21.15.125:0/2644545754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d380ceb40 0x7f3d3814d2c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 >> 172.21.15.125:0/2644545754 conn(0x7f3d380d0d60 msgr2=0x7f3d380ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:37.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 shutdown_connections 2023-10-18T12:01:37.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.539+0000 7f3d3ce10640 1 -- 172.21.15.125:0/2644545754 wait complete. 2023-10-18T12:01:37.552 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-18T12:01:37.588 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.586+0000 7f4940c15040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:01:37.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.656+0000 7f10fc905640 1 -- 172.21.15.125:0/535725977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80ceb40 msgr2=0x7f10f80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.656+0000 7f10fc905640 1 --2- 172.21.15.125:0/535725977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80ceb40 0x7f10f80b0650 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7f10f808a810 tx=0x7f10f002c260 comp rx=0 tx=0).stop 2023-10-18T12:01:37.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.656+0000 7f10fc905640 1 -- 172.21.15.125:0/535725977 shutdown_connections 2023-10-18T12:01:37.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.656+0000 7f10fc905640 1 --2- 172.21.15.125:0/535725977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80ceb40 0x7f10f80b0650 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.656+0000 7f10fc905640 1 --2- 172.21.15.125:0/535725977 >> v2:172.21.15.139:3300/0 conn(0x7f10f80cf340 0x7f10f80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.656+0000 7f10fc905640 1 -- 172.21.15.125:0/535725977 >> 172.21.15.125:0/535725977 conn(0x7f10f80d0d60 msgr2=0x7f10f80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:37.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.656+0000 7f10fc905640 1 -- 172.21.15.125:0/535725977 shutdown_connections 2023-10-18T12:01:37.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.656+0000 7f10fc905640 1 -- 172.21.15.125:0/535725977 wait complete. 2023-10-18T12:01:37.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.657+0000 7f10fc905640 1 Processor -- start 2023-10-18T12:01:37.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.657+0000 7f10fc905640 1 -- start start 2023-10-18T12:01:37.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10fc905640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10f80ceb40 0x7f10f809e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10fc905640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80cf340 0x7f10f8159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10fc905640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10f815b2a0 0x7f10f81f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10fc905640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8159010 con 0x7f10f80cf340 2023-10-18T12:01:37.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10fc905640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f809e950 con 0x7f10f815b2a0 2023-10-18T12:01:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10fc905640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81f5ad0 con 0x7f10f80ceb40 2023-10-18T12:01:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10f5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80cf340 0x7f10f8159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10f5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80cf340 0x7f10f8159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59346/0 (socket says 172.21.15.125:59346) 2023-10-18T12:01:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10f5d9b640 1 -- 172.21.15.125:0/3832650030 learned_addr learned my addr 172.21.15.125:0/3832650030 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:37.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10f6d9d640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10f815b2a0 0x7f10f81f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.658+0000 7f10f659c640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10f80ceb40 0x7f10f809e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10f5d9b640 1 -- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10f80ceb40 msgr2=0x7f10f809e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10f5d9b640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10f80ceb40 0x7f10f809e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10f5d9b640 1 -- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10f815b2a0 msgr2=0x7f10f81f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10f5d9b640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10f815b2a0 0x7f10f81f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10f5d9b640 1 -- 172.21.15.125:0/3832650030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0009900 con 0x7f10f80cf340 2023-10-18T12:01:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10f6d9d640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10f815b2a0 0x7f10f81f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10f5d9b640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80cf340 0x7f10f8159430 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c460 tx=0x7f10e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10e6ffd640 1 -- 172.21.15.125:0/3832650030 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80052d0 con 0x7f10f80cf340 2023-10-18T12:01:37.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10e6ffd640 1 -- 172.21.15.125:0/3832650030 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f10e8005430 con 0x7f10f80cf340 2023-10-18T12:01:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f8159970 con 0x7f10f80cf340 2023-10-18T12:01:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10e6ffd640 1 -- 172.21.15.125:0/3832650030 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8012890 con 0x7f10f80cf340 2023-10-18T12:01:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.659+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f81f57f0 con 0x7f10f80cf340 2023-10-18T12:01:37.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.660+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f8159150 con 0x7f10f80cf340 2023-10-18T12:01:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.661+0000 7f10e6ffd640 1 -- 172.21.15.125:0/3832650030 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f10e80035f0 con 0x7f10f80cf340 2023-10-18T12:01:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.661+0000 7f10e6ffd640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f10d806b080 0x7f10d806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.661+0000 7f10f659c640 1 -- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f10d806b080 msgr2=0x7f10d806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:37.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.661+0000 7f10e6ffd640 1 -- 172.21.15.125:0/3832650030 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7f10e8089c80 con 0x7f10f80cf340 2023-10-18T12:01:37.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.661+0000 7f10f659c640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f10d806b080 0x7f10d806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:37.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.669+0000 7f10e6ffd640 1 -- 172.21.15.125:0/3832650030 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10e80542d0 con 0x7f10f80cf340 2023-10-18T12:01:37.679 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.677+0000 7f5b64e2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:01:37.681 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.678+0000 7f4940c15040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:01:37.723 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.721+0000 7f4940c15040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:01:37.772 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.769+0000 7f5b64e2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:01:37.815 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.812+0000 7f5b64e2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:01:37.818 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.816+0000 7f4940c15040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:01:37.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.837+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f10f80b0d00 con 0x7f10f80cf340 2023-10-18T12:01:37.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.838+0000 7f10e6ffd640 1 -- 172.21.15.125:0/3832650030 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v247) v1 ==== 103+0+11 (secure 0 0 0) 0x7f10e80578f0 con 0x7f10f80cf340 2023-10-18T12:01:37.840 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:01:37.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f10d806b080 msgr2=0x7f10d806d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:37.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f10d806b080 0x7f10d806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80cf340 msgr2=0x7f10f8159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80cf340 0x7f10f8159430 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c460 tx=0x7f10e800c920 comp rx=0 tx=0).stop 2023-10-18T12:01:37.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 shutdown_connections 2023-10-18T12:01:37.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f10d806b080 0x7f10d806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10f815b2a0 0x7f10f81f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10f80cf340 0x7f10f8159430 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 --2- 172.21.15.125:0/3832650030 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10f80ceb40 0x7f10f809e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 >> 172.21.15.125:0/3832650030 conn(0x7f10f80d0d60 msgr2=0x7f10f815a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:37.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 shutdown_connections 2023-10-18T12:01:37.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.840+0000 7f10fc905640 1 -- 172.21.15.125:0/3832650030 wait complete. 2023-10-18T12:01:37.854 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T12:01:37.910 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:37.908+0000 7f4940c15040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:01:37.915 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:37.913+0000 7f5b64e2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:01:37.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.956+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1253693733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd0415b2a0 msgr2=0x7fbd040d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.956+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1253693733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd0415b2a0 0x7fbd040d1910 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7fbd0415b680 tx=0x7fbcec02bd30 comp rx=0 tx=0).stop 2023-10-18T12:01:37.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.956+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1253693733 shutdown_connections 2023-10-18T12:01:37.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.956+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1253693733 >> v2:172.21.15.139:3300/0 conn(0x7fbd040ceb40 0x7fbd040b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.956+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1253693733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd0415b2a0 0x7fbd040d1910 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.956+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1253693733 >> 172.21.15.125:0/1253693733 conn(0x7fbd040d0d60 msgr2=0x7fbd040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:37.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.956+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1253693733 shutdown_connections 2023-10-18T12:01:37.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.956+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1253693733 wait complete. 2023-10-18T12:01:37.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.958+0000 7fbd0a18e640 1 Processor -- start 2023-10-18T12:01:37.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.958+0000 7fbd0a18e640 1 -- start start 2023-10-18T12:01:37.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd0a18e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd040ceb40 0x7fbd041f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd0a18e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd040cf340 0x7fbd041f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd0a18e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0415b2a0 0x7fbd04151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd0a18e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040c8360 con 0x7fbd040ceb40 2023-10-18T12:01:37.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd0a18e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd041f1eb0 con 0x7fbd040cf340 2023-10-18T12:01:37.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd0a18e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd041f2190 con 0x7fbd0415b2a0 2023-10-18T12:01:37.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd040ceb40 0x7fbd041f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd040ceb40 0x7fbd041f57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59370/0 (socket says 172.21.15.125:59370) 2023-10-18T12:01:37.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.959+0000 7fbd037fe640 1 -- 172.21.15.125:0/1124695238 learned_addr learned my addr 172.21.15.125:0/1124695238 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:37.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbcfaffd640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd040cf340 0x7fbd041f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd03fff640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0415b2a0 0x7fbd04151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:37.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd037fe640 1 -- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0415b2a0 msgr2=0x7fbd04151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd037fe640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0415b2a0 0x7fbd04151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd037fe640 1 -- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd040cf340 msgr2=0x7fbd041f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:37.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd037fe640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd040cf340 0x7fbd041f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:37.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd037fe640 1 -- 172.21.15.125:0/1124695238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec009560 con 0x7fbd040ceb40 2023-10-18T12:01:37.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd037fe640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd040ceb40 0x7fbd041f57c0 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400c430 tx=0x7fbcf400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:37.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd00ff9640 1 -- 172.21.15.125:0/1124695238 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4011790 con 0x7fbd040ceb40 2023-10-18T12:01:37.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd00ff9640 1 -- 172.21.15.125:0/1124695238 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbcf40119e0 con 0x7fbd040ceb40 2023-10-18T12:01:37.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd00ff9640 1 -- 172.21.15.125:0/1124695238 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4011e30 con 0x7fbd040ceb40 2023-10-18T12:01:37.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd041f2470 con 0x7fbd040ceb40 2023-10-18T12:01:37.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.960+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd041f1b70 con 0x7fbd040ceb40 2023-10-18T12:01:37.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.961+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd040c87f0 con 0x7fbd040ceb40 2023-10-18T12:01:37.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.962+0000 7fbd00ff9640 1 -- 172.21.15.125:0/1124695238 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fbcf401c460 con 0x7fbd040ceb40 2023-10-18T12:01:37.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.962+0000 7fbd00ff9640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fbce406b0b0 0x7fbce406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:37.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.962+0000 7fbcfaffd640 1 -- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fbce406b0b0 msgr2=0x7fbce406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:37.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.962+0000 7fbd00ff9640 1 -- 172.21.15.125:0/1124695238 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7fbcf408a0c0 con 0x7fbd040ceb40 2023-10-18T12:01:37.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.962+0000 7fbcfaffd640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fbce406b0b0 0x7fbce406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:37.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:37.966+0000 7fbd00ff9640 1 -- 172.21.15.125:0/1124695238 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd040c87f0 con 0x7fbd040ceb40 2023-10-18T12:01:38.012 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:38.009+0000 7f5b64e2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:01:38.048 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:38.045+0000 7f4940c15040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:01:38.089 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:38.087+0000 7f4940c15040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:01:38.151 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:38.148+0000 7f5b64e2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:01:38.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.149+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd040c87f0 con 0x7fbd040ceb40 2023-10-18T12:01:38.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.150+0000 7fbd00ff9640 1 -- 172.21.15.125:0/1124695238 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v247) v1 ==== 81+0+594 (secure 0 0 0) 0x7fbd040c87f0 con 0x7fbd040ceb40 2023-10-18T12:01:38.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/3422518215 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/3422518215 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3037918695 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3823693717 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.153 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3119295615 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.154 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/3910260091 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.154 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/558106977 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.154 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2896936218 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.154 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/695463051 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.154 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2469983511 2023-10-19T12:01:33.783814+0000 2023-10-18T12:01:38.155 INFO:teuthology.orchestra.run.smithi125.stderr:listed 10 entries 2023-10-18T12:01:38.155 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.151+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fbce406b0b0 msgr2=0x7fbce406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:38.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.151+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fbce406b0b0 0x7fbce406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.151+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd040ceb40 msgr2=0x7fbd041f57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:38.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.151+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd040ceb40 0x7fbd041f57c0 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400c430 tx=0x7fbcf400c8f0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.152+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 shutdown_connections 2023-10-18T12:01:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.152+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fbce406b0b0 0x7fbce406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.152+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd040cf340 0x7fbd041f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.152+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd0415b2a0 0x7fbd04151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.152+0000 7fbd0a18e640 1 --2- 172.21.15.125:0/1124695238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd040ceb40 0x7fbd041f57c0 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.152+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 >> 172.21.15.125:0/1124695238 conn(0x7fbd040d0d60 msgr2=0x7fbd0406b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:38.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.152+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 shutdown_connections 2023-10-18T12:01:38.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.152+0000 7fbd0a18e640 1 -- 172.21.15.125:0/1124695238 wait complete. 2023-10-18T12:01:38.166 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T12:01:38.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:38.167 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T12:01:38.193 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[160129]: fuse finished with error 0 and tester_r 0 2023-10-18T12:01:38.194 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:01:38.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:38.195 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T12:01:38.196 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:38.191+0000 7f4940c15040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:01:38.199 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:38.196+0000 7f5b64e2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:01:38.245 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:01:38.246 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:38.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:01:38.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:01:38.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:38.262 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T12:01:38.306 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:38.304+0000 7f5b64e2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:01:38.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:01:38.316 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T12:01:38.317 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:01:38.317 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:38.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:01:38.333 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:01:38.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:38.334 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T12:01:38.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:01:38.389 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T12:01:38.389 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:01:38.390 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:38.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:01:38.406 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:01:38.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:01:38.406 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T12:01:38.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:01:38.460 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T12:01:38.461 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:38.565 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.563+0000 7fc99dc83640 1 -- 172.21.15.125:0/1964904768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc99815b2a0 msgr2=0x7fc9980cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:38.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.563+0000 7fc99dc83640 1 --2- 172.21.15.125:0/1964904768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc99815b2a0 0x7fc9980cf720 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fc99808a810 tx=0x7fc98002bdb0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.563+0000 7fc99dc83640 1 -- 172.21.15.125:0/1964904768 shutdown_connections 2023-10-18T12:01:38.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.563+0000 7fc99dc83640 1 --2- 172.21.15.125:0/1964904768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc99815b2a0 0x7fc9980cf720 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.563+0000 7fc99dc83640 1 --2- 172.21.15.125:0/1964904768 >> v2:172.21.15.139:3300/0 conn(0x7fc9980cf340 0x7fc9980d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.563+0000 7fc99dc83640 1 -- 172.21.15.125:0/1964904768 >> 172.21.15.125:0/1964904768 conn(0x7fc9980d0d60 msgr2=0x7fc9980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:38.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.563+0000 7fc99dc83640 1 -- 172.21.15.125:0/1964904768 shutdown_connections 2023-10-18T12:01:38.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.563+0000 7fc99dc83640 1 -- 172.21.15.125:0/1964904768 wait complete. 2023-10-18T12:01:38.570 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.564+0000 7fc99dc83640 1 Processor -- start 2023-10-18T12:01:38.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.564+0000 7fc99dc83640 1 -- start start 2023-10-18T12:01:38.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc99dc83640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9980ceb40 0x7fc998159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:38.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc99dc83640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9980cf340 0x7fc99815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:38.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc99dc83640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99815b2a0 0x7fc9980b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:38.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc99dc83640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980b4c00 con 0x7fc9980cf340 2023-10-18T12:01:38.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc99dc83640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99815c040 con 0x7fc99815b2a0 2023-10-18T12:01:38.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc99dc83640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99815c320 con 0x7fc9980ceb40 2023-10-18T12:01:38.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9980cf340 0x7fc99815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:38.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9980cf340 0x7fc99815b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59392/0 (socket says 172.21.15.125:59392) 2023-10-18T12:01:38.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 -- 172.21.15.125:0/2501645946 learned_addr learned my addr 172.21.15.125:0/2501645946 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:38.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc9977fe640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9980ceb40 0x7fc998159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:38.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc997fff640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99815b2a0 0x7fc9980b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:38.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 -- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9980ceb40 msgr2=0x7fc998159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:38.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9980ceb40 0x7fc998159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 -- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99815b2a0 msgr2=0x7fc9980b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:38.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99815b2a0 0x7fc9980b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 -- 172.21.15.125:0/2501645946 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980009560 con 0x7fc9980cf340 2023-10-18T12:01:38.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc997fff640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99815b2a0 0x7fc9980b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:38.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc9977fe640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9980ceb40 0x7fc998159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:38.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc98effd640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9980cf340 0x7fc99815b6d0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc98002db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:38.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc994ff9640 1 -- 172.21.15.125:0/2501645946 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980003800 con 0x7fc9980cf340 2023-10-18T12:01:38.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc994ff9640 1 -- 172.21.15.125:0/2501645946 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc980003a50 con 0x7fc9980cf340 2023-10-18T12:01:38.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99815c600 con 0x7fc9980cf340 2023-10-18T12:01:38.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc994ff9640 1 -- 172.21.15.125:0/2501645946 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980003e60 con 0x7fc9980cf340 2023-10-18T12:01:38.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.565+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc998159ac0 con 0x7fc9980cf340 2023-10-18T12:01:38.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.567+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9980690e0 con 0x7fc9980cf340 2023-10-18T12:01:38.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.567+0000 7fc994ff9640 1 -- 172.21.15.125:0/2501645946 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fc98003e450 con 0x7fc9980cf340 2023-10-18T12:01:38.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.567+0000 7fc994ff9640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fc97806b080 0x7fc97806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:38.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.567+0000 7fc9977fe640 1 -- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fc97806b080 msgr2=0x7fc97806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:38.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.567+0000 7fc994ff9640 1 -- 172.21.15.125:0/2501645946 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7fc9800acbd0 con 0x7fc9980cf340 2023-10-18T12:01:38.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.567+0000 7fc9977fe640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fc97806b080 0x7fc97806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:38.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.572+0000 7fc994ff9640 1 -- 172.21.15.125:0/2501645946 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9800773c0 con 0x7fc9980cf340 2023-10-18T12:01:38.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.768+0000 7fc9977fe640 1 -- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fc97806b080 msgr2=0x7fc97806d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:38.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.768+0000 7fc9977fe640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fc97806b080 0x7fc97806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:01:38.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.778+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9980cb9d0 con 0x7fc9980cf340 2023-10-18T12:01:38.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.779+0000 7fc994ff9640 1 -- 172.21.15.125:0/2501645946 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+6157 (secure 0 0 0) 0x7fc98007a9e0 con 0x7fc9980cf340 2023-10-18T12:01:38.782 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:38.782 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":103,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:08.447292+0000","modified":"2023-10-18T12:01:10.725386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12771":{"gid":12771,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1851256737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1851256737},{"type":"v1","addr":"172.21.15.125:6835","nonce":1851256737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:38.784 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 103 2023-10-18T12:01:38.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fc97806b080 msgr2=0x7fc97806d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:38.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fc97806b080 0x7fc97806d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9980cf340 msgr2=0x7fc99815b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:38.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9980cf340 0x7fc99815b6d0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc98002db50 comp rx=0 tx=0).stop 2023-10-18T12:01:38.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 shutdown_connections 2023-10-18T12:01:38.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fc97806b080 0x7fc97806d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc99815b2a0 0x7fc9980b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc9980cf340 0x7fc99815b6d0 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 --2- 172.21.15.125:0/2501645946 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc9980ceb40 0x7fc998159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 >> 172.21.15.125:0/2501645946 conn(0x7fc9980d0d60 msgr2=0x7fc9980ca3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:38.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 shutdown_connections 2023-10-18T12:01:38.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.781+0000 7fc99dc83640 1 -- 172.21.15.125:0/2501645946 wait complete. 2023-10-18T12:01:38.794 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:38.822 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:38.820+0000 7f4940c15040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:01:38.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.896+0000 7f5a36d24640 1 -- 172.21.15.125:0/3768314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a300ce240 msgr2=0x7f5a3006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:38.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.896+0000 7f5a36d24640 1 --2- 172.21.15.125:0/3768314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a300ce240 0x7f5a3006d260 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c005fd0 tx=0x7f5a1c02bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:38.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.896+0000 7f5a36d24640 1 -- 172.21.15.125:0/3768314 shutdown_connections 2023-10-18T12:01:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.896+0000 7f5a36d24640 1 --2- 172.21.15.125:0/3768314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a3015ccf0 0x7f5a3015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.896+0000 7f5a36d24640 1 --2- 172.21.15.125:0/3768314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a300ce240 0x7f5a3006d260 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.896+0000 7f5a36d24640 1 -- 172.21.15.125:0/3768314 >> 172.21.15.125:0/3768314 conn(0x7f5a300b0670 msgr2=0x7f5a300b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:38.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.896+0000 7f5a36d24640 1 -- 172.21.15.125:0/3768314 shutdown_connections 2023-10-18T12:01:38.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.896+0000 7f5a36d24640 1 -- 172.21.15.125:0/3768314 wait complete. 2023-10-18T12:01:38.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.897+0000 7f5a36d24640 1 Processor -- start 2023-10-18T12:01:38.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.897+0000 7f5a36d24640 1 -- start start 2023-10-18T12:01:38.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a36d24640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a300ce240 0x7f5a3014de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:38.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a36d24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a30159060 0x7f5a300c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:38.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a36d24640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3015ccf0 0x7f5a300c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:38.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a36d24640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300c3f10 con 0x7f5a3015ccf0 2023-10-18T12:01:38.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a36d24640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300c3300 con 0x7f5a30159060 2023-10-18T12:01:38.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a36d24640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300c35e0 con 0x7f5a300ce240 2023-10-18T12:01:38.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3015ccf0 0x7f5a300c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:38.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3015ccf0 0x7f5a300c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59408/0 (socket says 172.21.15.125:59408) 2023-10-18T12:01:38.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a34ac0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a300ce240 0x7f5a3014de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:38.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 -- 172.21.15.125:0/2821050821 learned_addr learned my addr 172.21.15.125:0/2821050821 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:38.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a2ffff640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a30159060 0x7f5a300c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:38.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 -- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a300ce240 msgr2=0x7f5a3014de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:38.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a300ce240 0x7f5a3014de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 -- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a30159060 msgr2=0x7f5a300c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:38.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a30159060 0x7f5a300c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:38.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 -- 172.21.15.125:0/2821050821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c009560 con 0x7f5a3015ccf0 2023-10-18T12:01:38.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.898+0000 7f5a352c1640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3015ccf0 0x7f5a300c50f0 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800e960 tx=0x7f5a2800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:38.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.899+0000 7f5a2d7fa640 1 -- 172.21.15.125:0/2821050821 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28004d20 con 0x7f5a3015ccf0 2023-10-18T12:01:38.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.899+0000 7f5a2d7fa640 1 -- 172.21.15.125:0/2821050821 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a2800cb20 con 0x7f5a3015ccf0 2023-10-18T12:01:38.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.899+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a300c38c0 con 0x7f5a3015ccf0 2023-10-18T12:01:38.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.899+0000 7f5a2d7fa640 1 -- 172.21.15.125:0/2821050821 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28005020 con 0x7f5a3015ccf0 2023-10-18T12:01:38.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.899+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3014e3c0 con 0x7f5a3015ccf0 2023-10-18T12:01:38.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.900+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a30158830 con 0x7f5a3015ccf0 2023-10-18T12:01:38.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.901+0000 7f5a2d7fa640 1 -- 172.21.15.125:0/2821050821 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f5a28005180 con 0x7f5a3015ccf0 2023-10-18T12:01:38.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.901+0000 7f5a2d7fa640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5a1406b080 0x7f5a1406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:38.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.901+0000 7f5a34ac0640 1 -- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5a1406b080 msgr2=0x7f5a1406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:38.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.901+0000 7f5a2d7fa640 1 -- 172.21.15.125:0/2821050821 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7f5a2808aaa0 con 0x7f5a3015ccf0 2023-10-18T12:01:38.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.901+0000 7f5a34ac0640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5a1406b080 0x7f5a1406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:38.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:38.905+0000 7f5a2d7fa640 1 -- 172.21.15.125:0/2821050821 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a28055100 con 0x7f5a3015ccf0 2023-10-18T12:01:38.973 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:38.971+0000 7f5b64e2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:01:39.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.075+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a300c4050 con 0x7f5a3015ccf0 2023-10-18T12:01:39.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.076+0000 7f5a2d7fa640 1 -- 172.21.15.125:0/2821050821 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v247) v1 ==== 74+0+21215 (secure 0 0 0) 0x7f5a300c4050 con 0x7f5a3015ccf0 2023-10-18T12:01:39.079 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:39.080 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":247,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:33.783850+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6833/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6832/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3037918695":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3823693717":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3119295615":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3910260091":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/558106977":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2896936218":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/695463051":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2469983511":"2023-10-19T12:01:33.783814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:39.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.078+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5a1406b080 msgr2=0x7f5a1406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:39.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.078+0000 7f5a36d24640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5a1406b080 0x7f5a1406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.078+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3015ccf0 msgr2=0x7f5a300c50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.078+0000 7f5a36d24640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3015ccf0 0x7f5a300c50f0 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800e960 tx=0x7f5a2800ee20 comp rx=0 tx=0).stop 2023-10-18T12:01:39.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.079+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 shutdown_connections 2023-10-18T12:01:39.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.079+0000 7f5a36d24640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7f5a1406b080 0x7f5a1406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.079+0000 7f5a36d24640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a30159060 0x7f5a300c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.079+0000 7f5a36d24640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5a3015ccf0 0x7f5a300c50f0 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.079+0000 7f5a36d24640 1 --2- 172.21.15.125:0/2821050821 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a300ce240 0x7f5a3014de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.079+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 >> 172.21.15.125:0/2821050821 conn(0x7f5a300b0670 msgr2=0x7f5a300b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:39.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.079+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 shutdown_connections 2023-10-18T12:01:39.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.079+0000 7f5a36d24640 1 -- 172.21.15.125:0/2821050821 wait complete. 2023-10-18T12:01:39.084 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.082+0000 7f4940c15040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:01:39.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:01:06.989961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '241', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:01:07.416244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '242', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-18T12:01:18.203802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '245', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-18T12:01:18.758795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '246', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:01:39.093 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:01:39.132 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.130+0000 7f4940c15040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:01:39.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.197+0000 7fe623997640 1 -- 172.21.15.125:0/2979374864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 msgr2=0x7fe61c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.197+0000 7fe623997640 1 --2- 172.21.15.125:0/2979374864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 0x7fe61c0cf720 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fe618009830 tx=0x7fe61802bcd0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.197+0000 7fe623997640 1 -- 172.21.15.125:0/2979374864 shutdown_connections 2023-10-18T12:01:39.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.197+0000 7fe623997640 1 --2- 172.21.15.125:0/2979374864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 0x7fe61c0cf720 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.197+0000 7fe623997640 1 --2- 172.21.15.125:0/2979374864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0cf340 0x7fe61c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.197+0000 7fe623997640 1 -- 172.21.15.125:0/2979374864 >> 172.21.15.125:0/2979374864 conn(0x7fe61c0d0d60 msgr2=0x7fe61c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:39.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.197+0000 7fe623997640 1 -- 172.21.15.125:0/2979374864 shutdown_connections 2023-10-18T12:01:39.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.197+0000 7fe623997640 1 -- 172.21.15.125:0/2979374864 wait complete. 2023-10-18T12:01:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.198+0000 7fe623997640 1 Processor -- start 2023-10-18T12:01:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.198+0000 7fe623997640 1 -- start start 2023-10-18T12:01:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe623997640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0ceb40 0x7fe61c159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe623997640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c0cf340 0x7fe61c1f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe623997640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 0x7fe61c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe623997640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0b6530 con 0x7fe61c15b2a0 2023-10-18T12:01:39.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe623997640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0b6810 con 0x7fe61c0cf340 2023-10-18T12:01:39.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe623997640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c15cd80 con 0x7fe61c0ceb40 2023-10-18T12:01:39.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe621f34640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 0x7fe61c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe621f34640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 0x7fe61c158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59436/0 (socket says 172.21.15.125:59436) 2023-10-18T12:01:39.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe621f34640 1 -- 172.21.15.125:0/2115785580 learned_addr learned my addr 172.21.15.125:0/2115785580 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:39.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe621733640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0ceb40 0x7fe61c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.199+0000 7fe620f32640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c0cf340 0x7fe61c1f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe621f34640 1 -- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0ceb40 msgr2=0x7fe61c159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe621f34640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0ceb40 0x7fe61c159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe621f34640 1 -- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c0cf340 msgr2=0x7fe61c1f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe621f34640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c0cf340 0x7fe61c1f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe621f34640 1 -- 172.21.15.125:0/2115785580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe618009490 con 0x7fe61c15b2a0 2023-10-18T12:01:39.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe621f34640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 0x7fe61c158b30 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c760 tx=0x7fe61400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:39.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614003da0 con 0x7fe61c15b2a0 2023-10-18T12:01:39.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe61400ce70 con 0x7fe61c15b2a0 2023-10-18T12:01:39.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c15d060 con 0x7fe61c15b2a0 2023-10-18T12:01:39.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140052b0 con 0x7fe61c15b2a0 2023-10-18T12:01:39.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.200+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c15c9c0 con 0x7fe61c15b2a0 2023-10-18T12:01:39.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.201+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c15d250 con 0x7fe61c15b2a0 2023-10-18T12:01:39.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.202+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 143) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fe614005410 con 0x7fe61c15b2a0 2023-10-18T12:01:39.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.202+0000 7fe611ffb640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fe60006b080 0x7fe60006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.202+0000 7fe621733640 1 -- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fe60006b080 msgr2=0x7fe60006d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/2483056868 2023-10-18T12:01:39.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.202+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 7379+0+0 (secure 0 0 0) 0x7fe614089f90 con 0x7fe61c15b2a0 2023-10-18T12:01:39.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.202+0000 7fe621733640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fe60006b080 0x7fe60006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:01:39.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.206+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6140bcca0 con 0x7fe61c15b2a0 2023-10-18T12:01:39.235 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.232+0000 7f4940c15040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:01:39.246 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:39.243+0000 7f5b64e2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:01:39.262 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.260+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 144) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe6140056c0 con 0x7fe61c15b2a0 2023-10-18T12:01:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.260+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fe60006b080 msgr2=0x7fe60006d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:01:39.263 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.260+0000 7fe611ffb640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fe60006b080 0x7fe60006d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.305 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:39.300+0000 7f5b64e2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:01:39.317 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.317 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.317 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.318 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.318 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.318 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.318 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.319 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.319 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.319 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.314+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.326 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.326 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.326 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.326 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c5ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.327 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:39.323+0000 7f48c2acc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:01:39.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.388+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe61c0690e0 con 0x7fe61c15b2a0 2023-10-18T12:01:39.391 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.388+0000 7fe611ffb640 1 -- 172.21.15.125:0/2115785580 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v248) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe614071910 con 0x7fe61c15b2a0 2023-10-18T12:01:39.391 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:01:39.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.390+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 msgr2=0x7fe61c158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.390+0000 7fe623997640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 0x7fe61c158b30 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c760 tx=0x7fe61400cc20 comp rx=0 tx=0).stop 2023-10-18T12:01:39.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.390+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 shutdown_connections 2023-10-18T12:01:39.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.390+0000 7fe623997640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:6832/2483056868,v1:172.21.15.139:6833/2483056868] conn(0x7fe60006b080 0x7fe60006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.390+0000 7fe623997640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c0cf340 0x7fe61c1f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.390+0000 7fe623997640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe61c15b2a0 0x7fe61c158b30 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.390+0000 7fe623997640 1 --2- 172.21.15.125:0/2115785580 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0ceb40 0x7fe61c159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.390+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 >> 172.21.15.125:0/2115785580 conn(0x7fe61c0d0d60 msgr2=0x7fe61c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:39.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.391+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 shutdown_connections 2023-10-18T12:01:39.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.391+0000 7fe623997640 1 -- 172.21.15.125:0/2115785580 wait complete. 2023-10-18T12:01:39.404 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:01:39.409 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:01:39.407+0000 7f5b64e2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:01:39.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.507+0000 7f66cf373640 1 -- 172.21.15.125:0/4223385327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 msgr2=0x7f66c80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.507+0000 7f66cf373640 1 --2- 172.21.15.125:0/4223385327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 0x7f66c80b0650 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f66c4005fd0 tx=0x7f66c402bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:39.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.507+0000 7f66cf373640 1 -- 172.21.15.125:0/4223385327 shutdown_connections 2023-10-18T12:01:39.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.507+0000 7f66cf373640 1 --2- 172.21.15.125:0/4223385327 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 0x7f66c80b0650 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.507+0000 7f66cf373640 1 --2- 172.21.15.125:0/4223385327 >> v2:172.21.15.139:3300/0 conn(0x7f66c80cf340 0x7f66c80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.507+0000 7f66cf373640 1 -- 172.21.15.125:0/4223385327 >> 172.21.15.125:0/4223385327 conn(0x7f66c80d0d60 msgr2=0x7f66c80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:39.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.508+0000 7f66cf373640 1 -- 172.21.15.125:0/4223385327 shutdown_connections 2023-10-18T12:01:39.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.508+0000 7f66cf373640 1 -- 172.21.15.125:0/4223385327 wait complete. 2023-10-18T12:01:39.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.509+0000 7f66cf373640 1 Processor -- start 2023-10-18T12:01:39.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.509+0000 7f66cf373640 1 -- start start 2023-10-18T12:01:39.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.509+0000 7f66cf373640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 0x7f66c80c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.509+0000 7f66cf373640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66c80cf340 0x7f66c8153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.509+0000 7f66cf373640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66c815b2a0 0x7f66c806ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.509+0000 7f66cf373640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80b5380 con 0x7f66c80cf340 2023-10-18T12:01:39.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.509+0000 7f66cf373640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80c9710 con 0x7f66c815b2a0 2023-10-18T12:01:39.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.509+0000 7f66cf373640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80b4800 con 0x7f66c80ceb40 2023-10-18T12:01:39.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66c80cf340 0x7f66c8153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66c80cf340 0x7f66c8153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59442/0 (socket says 172.21.15.125:59442) 2023-10-18T12:01:39.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 -- 172.21.15.125:0/1717287034 learned_addr learned my addr 172.21.15.125:0/1717287034 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:39.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cd10f640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 0x7f66c80c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cd910640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66c815b2a0 0x7f66c806ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 -- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 msgr2=0x7f66c80c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 0x7f66c80c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 -- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66c815b2a0 msgr2=0x7f66c806ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66c815b2a0 0x7f66c806ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 -- 172.21.15.125:0/1717287034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4009560 con 0x7f66c80cf340 2023-10-18T12:01:39.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cd10f640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 0x7f66c80c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:39.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cd910640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66c815b2a0 0x7f66c806ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:39.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cc90e640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66c80cf340 0x7f66c8153de0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c430 tx=0x7f66b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:39.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66bdffb640 1 -- 172.21.15.125:0/1717287034 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b80052d0 con 0x7f66c80cf340 2023-10-18T12:01:39.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66bdffb640 1 -- 172.21.15.125:0/1717287034 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66b80035f0 con 0x7f66c80cf340 2023-10-18T12:01:39.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c80b0d50 con 0x7f66c80cf340 2023-10-18T12:01:39.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.510+0000 7f66bdffb640 1 -- 172.21.15.125:0/1717287034 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8003a00 con 0x7f66c80cf340 2023-10-18T12:01:39.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.511+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c80b4dc0 con 0x7f66c80cf340 2023-10-18T12:01:39.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.512+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c80690e0 con 0x7f66c80cf340 2023-10-18T12:01:39.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.512+0000 7f66bdffb640 1 -- 172.21.15.125:0/1717287034 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 144) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f66b8003b60 con 0x7f66c80cf340 2023-10-18T12:01:39.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.512+0000 7f66bdffb640 1 -- 172.21.15.125:0/1717287034 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7f66b80891a0 con 0x7f66c80cf340 2023-10-18T12:01:39.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.516+0000 7f66bdffb640 1 -- 172.21.15.125:0/1717287034 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66b8053de0 con 0x7f66c80cf340 2023-10-18T12:01:39.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.683+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66c80c9850 con 0x7f66c80cf340 2023-10-18T12:01:39.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.683+0000 7f66bdffb640 1 -- 172.21.15.125:0/1717287034 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v248) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66b8057400 con 0x7f66c80cf340 2023-10-18T12:01:39.687 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:01:39.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.685+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66c80cf340 msgr2=0x7f66c8153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.685+0000 7f66cf373640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66c80cf340 0x7f66c8153de0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c430 tx=0x7f66b800c8f0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.686+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 shutdown_connections 2023-10-18T12:01:39.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.686+0000 7f66cf373640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66c815b2a0 0x7f66c806ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.686+0000 7f66cf373640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66c80cf340 0x7f66c8153de0 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.686+0000 7f66cf373640 1 --2- 172.21.15.125:0/1717287034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66c80ceb40 0x7f66c80c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.686+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 >> 172.21.15.125:0/1717287034 conn(0x7f66c80d0d60 msgr2=0x7f66c8159000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:39.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.686+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 shutdown_connections 2023-10-18T12:01:39.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.686+0000 7f66cf373640 1 -- 172.21.15.125:0/1717287034 wait complete. 2023-10-18T12:01:39.699 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:01:39.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.841+0000 7fbe4d58e640 1 -- 172.21.15.125:0/1099647566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480acd40 msgr2=0x7fbe4806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.841+0000 7fbe4d58e640 1 --2- 172.21.15.125:0/1099647566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480acd40 0x7fbe4806e130 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009ca0 tx=0x7fbe4002c2e0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.841+0000 7fbe4d58e640 1 -- 172.21.15.125:0/1099647566 shutdown_connections 2023-10-18T12:01:39.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.841+0000 7fbe4d58e640 1 --2- 172.21.15.125:0/1099647566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480acd40 0x7fbe4806e130 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.841+0000 7fbe4d58e640 1 -- 172.21.15.125:0/1099647566 >> 172.21.15.125:0/1099647566 conn(0x7fbe480b3f00 msgr2=0x7fbe480b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:39.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.841+0000 7fbe4d58e640 1 -- 172.21.15.125:0/1099647566 shutdown_connections 2023-10-18T12:01:39.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.841+0000 7fbe4d58e640 1 -- 172.21.15.125:0/1099647566 wait complete. 2023-10-18T12:01:39.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe4d58e640 1 Processor -- start 2023-10-18T12:01:39.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe4d58e640 1 -- start start 2023-10-18T12:01:39.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe4d58e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe480acd40 0x7fbe481f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe4d58e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe480b24d0 0x7fbe480b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe4d58e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480b3070 0x7fbe481543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:39.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe4d58e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe481e3450 con 0x7fbe480b3070 2023-10-18T12:01:39.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe4d58e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe481e2e90 con 0x7fbe480acd40 2023-10-18T12:01:39.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe4d58e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe481e3170 con 0x7fbe480b24d0 2023-10-18T12:01:39.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480b3070 0x7fbe481543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480b3070 0x7fbe481543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59462/0 (socket says 172.21.15.125:59462) 2023-10-18T12:01:39.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.843+0000 7fbe477fe640 1 -- 172.21.15.125:0/255930252 learned_addr learned my addr 172.21.15.125:0/255930252 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe46ffd640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe480acd40 0x7fbe481f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe3ffff640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe480b24d0 0x7fbe480b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe477fe640 1 -- 172.21.15.125:0/255930252 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe480b24d0 msgr2=0x7fbe480b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe477fe640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe480b24d0 0x7fbe480b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe477fe640 1 -- 172.21.15.125:0/255930252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe480acd40 msgr2=0x7fbe481f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe477fe640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe480acd40 0x7fbe481f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe477fe640 1 -- 172.21.15.125:0/255930252 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe40009900 con 0x7fbe480b3070 2023-10-18T12:01:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe477fe640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480b3070 0x7fbe481543a0 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009bf0 tx=0x7fbe40035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:39.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe3f7fe640 1 -- 172.21.15.125:0/255930252 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40004240 con 0x7fbe480b3070 2023-10-18T12:01:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe3f7fe640 1 -- 172.21.15.125:0/255930252 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe40004490 con 0x7fbe480b3070 2023-10-18T12:01:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe3f7fe640 1 -- 172.21.15.125:0/255930252 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe400048a0 con 0x7fbe480b3070 2023-10-18T12:01:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe480ae6f0 con 0x7fbe480b3070 2023-10-18T12:01:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.844+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe480b06f0 con 0x7fbe480b3070 2023-10-18T12:01:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.845+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe480aec10 con 0x7fbe480b3070 2023-10-18T12:01:39.853 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.846+0000 7fbe3f7fe640 1 -- 172.21.15.125:0/255930252 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 144) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fbe4003b050 con 0x7fbe480b3070 2023-10-18T12:01:39.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.846+0000 7fbe3f7fe640 1 -- 172.21.15.125:0/255930252 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7fbe400ac720 con 0x7fbe480b3070 2023-10-18T12:01:39.854 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:39.850+0000 7fbe3f7fe640 1 -- 172.21.15.125:0/255930252 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe40077410 con 0x7fbe480b3070 2023-10-18T12:01:40.018 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.015+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe480690e0 con 0x7fbe480b3070 2023-10-18T12:01:40.019 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.016+0000 7fbe3f7fe640 1 -- 172.21.15.125:0/255930252 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v248) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe4007aa30 con 0x7fbe480b3070 2023-10-18T12:01:40.019 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:01:40.020 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480b3070 msgr2=0x7fbe481543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480b3070 0x7fbe481543a0 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009bf0 tx=0x7fbe40035b50 comp rx=0 tx=0).stop 2023-10-18T12:01:40.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 shutdown_connections 2023-10-18T12:01:40.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe480b3070 0x7fbe481543a0 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.021 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe480b24d0 0x7fbe480b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 --2- 172.21.15.125:0/255930252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe480acd40 0x7fbe481f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 >> 172.21.15.125:0/255930252 conn(0x7fbe480b3f00 msgr2=0x7fbe48154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:40.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 shutdown_connections 2023-10-18T12:01:40.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.018+0000 7fbe4d58e640 1 -- 172.21.15.125:0/255930252 wait complete. 2023-10-18T12:01:40.031 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-18T12:01:40.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.135+0000 7f18b1741640 1 -- 172.21.15.125:0/3084815769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0afa20 msgr2=0x7f18ac158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.135+0000 7f18b1741640 1 --2- 172.21.15.125:0/3084815769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0afa20 0x7f18ac158300 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0b10f0 tx=0x7f18a002bd70 comp rx=0 tx=0).stop 2023-10-18T12:01:40.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.135+0000 7f18b1741640 1 -- 172.21.15.125:0/3084815769 shutdown_connections 2023-10-18T12:01:40.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.135+0000 7f18b1741640 1 --2- 172.21.15.125:0/3084815769 >> v2:172.21.15.139:3300/0 conn(0x7f18ac0b0530 0x7f18ac0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.135+0000 7f18b1741640 1 --2- 172.21.15.125:0/3084815769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0afa20 0x7f18ac158300 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.135+0000 7f18b1741640 1 -- 172.21.15.125:0/3084815769 >> 172.21.15.125:0/3084815769 conn(0x7f18ac069930 msgr2=0x7f18ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:40.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.135+0000 7f18b1741640 1 -- 172.21.15.125:0/3084815769 shutdown_connections 2023-10-18T12:01:40.141 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.135+0000 7f18b1741640 1 -- 172.21.15.125:0/3084815769 wait complete. 2023-10-18T12:01:40.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18b1741640 1 Processor -- start 2023-10-18T12:01:40.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18b1741640 1 -- start start 2023-10-18T12:01:40.144 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18b1741640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18ac0afa20 0x7f18ac0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18b1741640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18ac0b0530 0x7f18ac143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18b1741640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0cbd20 0x7f18ac156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18b1741640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac0ca5d0 con 0x7f18ac0cbd20 2023-10-18T12:01:40.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18b1741640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac156690 con 0x7f18ac0afa20 2023-10-18T12:01:40.145 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18b1741640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac155aa0 con 0x7f18ac0b0530 2023-10-18T12:01:40.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0cbd20 0x7f18ac156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0cbd20 0x7f18ac156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59476/0 (socket says 172.21.15.125:59476) 2023-10-18T12:01:40.146 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.137+0000 7f18ab7fe640 1 -- 172.21.15.125:0/3775978355 learned_addr learned my addr 172.21.15.125:0/3775978355 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:40.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18aaffd640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18ac0afa20 0x7f18ac0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18ab7fe640 1 -- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18ac0b0530 msgr2=0x7f18ac143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.147 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18aa7fc640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18ac0b0530 0x7f18ac143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18ab7fe640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18ac0b0530 0x7f18ac143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18ab7fe640 1 -- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18ac0afa20 msgr2=0x7f18ac0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18ab7fe640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18ac0afa20 0x7f18ac0ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18ab7fe640 1 -- 172.21.15.125:0/3775978355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a0009490 con 0x7f18ac0cbd20 2023-10-18T12:01:40.148 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18aaffd640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18ac0afa20 0x7f18ac0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:40.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18ab7fe640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0cbd20 0x7f18ac156b20 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f189c00cb00 tx=0x7f189c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:40.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18937fe640 1 -- 172.21.15.125:0/3775978355 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00ae90 con 0x7f18ac0cbd20 2023-10-18T12:01:40.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18937fe640 1 -- 172.21.15.125:0/3775978355 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f189c004b20 con 0x7f18ac0cbd20 2023-10-18T12:01:40.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac155d60 con 0x7f18ac0cbd20 2023-10-18T12:01:40.149 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18937fe640 1 -- 172.21.15.125:0/3775978355 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c005020 con 0x7f18ac0cbd20 2023-10-18T12:01:40.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.138+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac156320 con 0x7f18ac0cbd20 2023-10-18T12:01:40.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.139+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac0aff00 con 0x7f18ac0cbd20 2023-10-18T12:01:40.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.139+0000 7f18937fe640 1 -- 172.21.15.125:0/3775978355 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 144) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f189c005180 con 0x7f18ac0cbd20 2023-10-18T12:01:40.150 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.140+0000 7f18937fe640 1 -- 172.21.15.125:0/3775978355 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7f189c08a180 con 0x7f18ac0cbd20 2023-10-18T12:01:40.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.144+0000 7f18937fe640 1 -- 172.21.15.125:0/3775978355 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f189c054e40 con 0x7f18ac0cbd20 2023-10-18T12:01:40.260 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.258+0000 7f18937fe640 1 -- 172.21.15.125:0/3775978355 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 145) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f189c0548f0 con 0x7f18ac0cbd20 2023-10-18T12:01:40.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.258+0000 7f18937fe640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f18980733e0 0x7f18980757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.261 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.258+0000 7f18aaffd640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f18980733e0 0x7f18980757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.264 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.262+0000 7f18aaffd640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f18980733e0 0x7f18980757d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f18a4000f60 tx=0x7f18a4009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:40.316 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.314+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18ac0690e0 con 0x7f18ac0cbd20 2023-10-18T12:01:40.317 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.314+0000 7f18937fe640 1 -- 172.21.15.125:0/3775978355 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v248) v1 ==== 107+0+11 (secure 0 0 0) 0x7f189c01d020 con 0x7f18ac0cbd20 2023-10-18T12:01:40.317 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:01:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f18980733e0 msgr2=0x7f18980757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f18980733e0 0x7f18980757d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f18a4000f60 tx=0x7f18a4009040 comp rx=0 tx=0).stop 2023-10-18T12:01:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0cbd20 msgr2=0x7f18ac156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0cbd20 0x7f18ac156b20 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f189c00cb00 tx=0x7f189c00a250 comp rx=0 tx=0).stop 2023-10-18T12:01:40.319 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 shutdown_connections 2023-10-18T12:01:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18ac0cbd20 0x7f18ac156b20 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18ac0b0530 0x7f18ac143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f18980733e0 0x7f18980757d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 --2- 172.21.15.125:0/3775978355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18ac0afa20 0x7f18ac0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.320 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 >> 172.21.15.125:0/3775978355 conn(0x7f18ac069930 msgr2=0x7f18ac0ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:40.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 shutdown_connections 2023-10-18T12:01:40.321 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.316+0000 7f18b1741640 1 -- 172.21.15.125:0/3775978355 wait complete. 2023-10-18T12:01:40.329 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-18T12:01:40.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.432+0000 7f8ac51e9640 1 -- 172.21.15.125:0/2441476310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac006b410 msgr2=0x7f8ac015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.432+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/2441476310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac006b410 0x7f8ac015c770 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8005fd0 tx=0x7f8aa802bcf0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.432+0000 7f8ac51e9640 1 -- 172.21.15.125:0/2441476310 shutdown_connections 2023-10-18T12:01:40.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.432+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/2441476310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac015ccb0 0x7f8ac00c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.432+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/2441476310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac006b410 0x7f8ac015c770 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.432+0000 7f8ac51e9640 1 -- 172.21.15.125:0/2441476310 >> 172.21.15.125:0/2441476310 conn(0x7f8ac0069930 msgr2=0x7f8ac0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:40.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.432+0000 7f8ac51e9640 1 -- 172.21.15.125:0/2441476310 shutdown_connections 2023-10-18T12:01:40.437 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.432+0000 7f8ac51e9640 1 -- 172.21.15.125:0/2441476310 wait complete. 2023-10-18T12:01:40.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8ac51e9640 1 Processor -- start 2023-10-18T12:01:40.440 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8ac51e9640 1 -- start start 2023-10-18T12:01:40.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8ac51e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac006b030 0x7f8ac01ec620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8ac51e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac006b410 0x7f8ac01f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8ac51e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac015ccb0 0x7f8ac01f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.441 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8ac51e9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01519b0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8ac51e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01465b0 con 0x7f8ac006b030 2023-10-18T12:01:40.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8ac51e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01f0050 con 0x7f8ac006b410 2023-10-18T12:01:40.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.434+0000 7f8abf59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac015ccb0 0x7f8ac01f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.442 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abf59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac015ccb0 0x7f8ac01f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59498/0 (socket says 172.21.15.125:59498) 2023-10-18T12:01:40.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abf59e640 1 -- 172.21.15.125:0/629293460 learned_addr learned my addr 172.21.15.125:0/629293460 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:40.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8ab7fff640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac006b410 0x7f8ac01f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abed9d640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac006b030 0x7f8ac01ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abf59e640 1 -- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac006b410 msgr2=0x7f8ac01f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.443 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abf59e640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac006b410 0x7f8ac01f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abf59e640 1 -- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac006b030 msgr2=0x7f8ac01ec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abf59e640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac006b030 0x7f8ac01ec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abf59e640 1 -- 172.21.15.125:0/629293460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa8009560 con 0x7f8ac015ccb0 2023-10-18T12:01:40.444 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abed9d640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac006b030 0x7f8ac01ec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:40.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8abf59e640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac015ccb0 0x7f8ac01f2fa0 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800ca80 tx=0x7f8ab800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:40.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8ab77fe640 1 -- 172.21.15.125:0/629293460 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8007df0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8ab77fe640 1 -- 172.21.15.125:0/629293460 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ab8003aa0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac01f0330 con 0x7f8ac015ccb0 2023-10-18T12:01:40.445 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8ab77fe640 1 -- 172.21.15.125:0/629293460 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8004030 con 0x7f8ac015ccb0 2023-10-18T12:01:40.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.435+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac01515d0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.436+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac0151af0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.436+0000 7f8ab77fe640 1 -- 172.21.15.125:0/629293460 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 145) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f8ab8004190 con 0x7f8ac015ccb0 2023-10-18T12:01:40.446 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.437+0000 7f8ab77fe640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f8aa006ac90 0x7f8aa006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.437+0000 7f8ab77fe640 1 -- 172.21.15.125:0/629293460 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7f8ab808a5a0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.437+0000 7f8abed9d640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f8aa006ac90 0x7f8aa006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.438+0000 7f8abed9d640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f8aa006ac90 0x7f8aa006d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0007fd0 tx=0x7f8ab0009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:40.447 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.441+0000 7f8ab77fe640 1 -- 172.21.15.125:0/629293460 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ac0151af0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.608+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ac0151af0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.608+0000 7f8ab77fe640 1 -- 172.21.15.125:0/629293460 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v248) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8ac0151af0 con 0x7f8ac015ccb0 2023-10-18T12:01:40.611 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:01:40.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f8aa006ac90 msgr2=0x7f8aa006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f8aa006ac90 0x7f8aa006d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0007fd0 tx=0x7f8ab0009040 comp rx=0 tx=0).stop 2023-10-18T12:01:40.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac015ccb0 msgr2=0x7f8ac01f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.613 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac015ccb0 0x7f8ac01f2fa0 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800ca80 tx=0x7f8ab800cf40 comp rx=0 tx=0).stop 2023-10-18T12:01:40.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 shutdown_connections 2023-10-18T12:01:40.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ac015ccb0 0x7f8ac01f2fa0 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ac006b410 0x7f8ac01f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.614 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f8aa006ac90 0x7f8aa006d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 --2- 172.21.15.125:0/629293460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ac006b030 0x7f8ac01ec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 >> 172.21.15.125:0/629293460 conn(0x7f8ac0069930 msgr2=0x7f8ac00afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:40.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 shutdown_connections 2023-10-18T12:01:40.615 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.610+0000 7f8ac51e9640 1 -- 172.21.15.125:0/629293460 wait complete. 2023-10-18T12:01:40.623 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:40.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.723+0000 7f23dbbe2640 1 -- 172.21.15.125:0/1422444520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d40cf340 msgr2=0x7f23d40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.723+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/1422444520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d40cf340 0x7f23d40d1910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f23d0005fd0 tx=0x7f23d002c050 comp rx=0 tx=0).stop 2023-10-18T12:01:40.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.723+0000 7f23dbbe2640 1 -- 172.21.15.125:0/1422444520 shutdown_connections 2023-10-18T12:01:40.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.723+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/1422444520 >> v2:172.21.15.139:3300/0 conn(0x7f23d40ceb40 0x7f23d40b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.723+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/1422444520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d40cf340 0x7f23d40d1910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.723+0000 7f23dbbe2640 1 -- 172.21.15.125:0/1422444520 >> 172.21.15.125:0/1422444520 conn(0x7f23d40d0d60 msgr2=0x7f23d40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:40.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.723+0000 7f23dbbe2640 1 -- 172.21.15.125:0/1422444520 shutdown_connections 2023-10-18T12:01:40.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.723+0000 7f23dbbe2640 1 -- 172.21.15.125:0/1422444520 wait complete. 2023-10-18T12:01:40.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23dbbe2640 1 Processor -- start 2023-10-18T12:01:40.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23dbbe2640 1 -- start start 2023-10-18T12:01:40.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23dbbe2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23d40ceb40 0x7f23d41592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23dbbe2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d40cf340 0x7f23d41f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23dbbe2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d415b2a0 0x7f23d41f1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.732 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23dbbe2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41f5880 con 0x7f23d40ceb40 2023-10-18T12:01:40.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23dbbe2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41f52c0 con 0x7f23d40cf340 2023-10-18T12:01:40.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23dbbe2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41f55a0 con 0x7f23d415b2a0 2023-10-18T12:01:40.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23d997e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23d40ceb40 0x7f23d41592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.733 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23d997e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23d40ceb40 0x7f23d41592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59504/0 (socket says 172.21.15.125:59504) 2023-10-18T12:01:40.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23d997e640 1 -- 172.21.15.125:0/2803598864 learned_addr learned my addr 172.21.15.125:0/2803598864 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:40.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.725+0000 7f23da17f640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d415b2a0 0x7f23d41f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23d917d640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d40cf340 0x7f23d41f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23d997e640 1 -- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d415b2a0 msgr2=0x7f23d41f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.734 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23d997e640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d415b2a0 0x7f23d41f1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23d997e640 1 -- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d40cf340 msgr2=0x7f23d41f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23d997e640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d40cf340 0x7f23d41f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23d997e640 1 -- 172.21.15.125:0/2803598864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0009840 con 0x7f23d40ceb40 2023-10-18T12:01:40.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23d997e640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23d40ceb40 0x7f23d41592f0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f23d0036040 tx=0x7f23d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:40.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23ca7fc640 1 -- 172.21.15.125:0/2803598864 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0004870 con 0x7f23d40ceb40 2023-10-18T12:01:40.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23ca7fc640 1 -- 172.21.15.125:0/2803598864 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f23d003e460 con 0x7f23d40ceb40 2023-10-18T12:01:40.736 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d40b5060 con 0x7f23d40ceb40 2023-10-18T12:01:40.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23ca7fc640 1 -- 172.21.15.125:0/2803598864 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d003e890 con 0x7f23d40ceb40 2023-10-18T12:01:40.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.726+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d41f4f20 con 0x7f23d40ceb40 2023-10-18T12:01:40.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.727+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d40690e0 con 0x7f23d40ceb40 2023-10-18T12:01:40.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.727+0000 7f23ca7fc640 1 -- 172.21.15.125:0/2803598864 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 145) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f23d003e9f0 con 0x7f23d40ceb40 2023-10-18T12:01:40.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.728+0000 7f23ca7fc640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f23b806ac90 0x7f23b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:40.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.728+0000 7f23ca7fc640 1 -- 172.21.15.125:0/2803598864 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7f23d00ac9a0 con 0x7f23d40ceb40 2023-10-18T12:01:40.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.728+0000 7f23d917d640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f23b806ac90 0x7f23b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:40.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.729+0000 7f23d917d640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f23b806ac90 0x7f23b806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f23d4159830 tx=0x7f23c4009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:40.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.732+0000 7f23ca7fc640 1 -- 172.21.15.125:0/2803598864 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23d0077550 con 0x7f23d40ceb40 2023-10-18T12:01:40.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.932+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23d40cb9b0 con 0x7f23d40ceb40 2023-10-18T12:01:40.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.933+0000 7f23ca7fc640 1 -- 172.21.15.125:0/2803598864 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+6157 (secure 0 0 0) 0x7f23d0042070 con 0x7f23d40ceb40 2023-10-18T12:01:40.938 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:40.938 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":103,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:08.447292+0000","modified":"2023-10-18T12:01:10.725386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12771":{"gid":12771,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1851256737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1851256737},{"type":"v1","addr":"172.21.15.125:6835","nonce":1851256737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:40.940 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 103 2023-10-18T12:01:40.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f23b806ac90 msgr2=0x7f23b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f23b806ac90 0x7f23b806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f23d4159830 tx=0x7f23c4009040 comp rx=0 tx=0).stop 2023-10-18T12:01:40.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23d40ceb40 msgr2=0x7f23d41592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:40.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23d40ceb40 0x7f23d41592f0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f23d0036040 tx=0x7f23d0004270 comp rx=0 tx=0).stop 2023-10-18T12:01:40.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 shutdown_connections 2023-10-18T12:01:40.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f23b806ac90 0x7f23b806d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d40cf340 0x7f23d41f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d415b2a0 0x7f23d41f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 --2- 172.21.15.125:0/2803598864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23d40ceb40 0x7f23d41592f0 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:40.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 >> 172.21.15.125:0/2803598864 conn(0x7f23d40d0d60 msgr2=0x7f23d406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:40.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 shutdown_connections 2023-10-18T12:01:40.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:40.935+0000 7f23dbbe2640 1 -- 172.21.15.125:0/2803598864 wait complete. 2023-10-18T12:01:40.948 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:41.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.048+0000 7f903bd33640 1 -- 172.21.15.125:0/2771543394 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90340ceb40 msgr2=0x7f90340b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.048+0000 7f903bd33640 1 --2- 172.21.15.125:0/2771543394 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90340ceb40 0x7f90340b0650 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f9030009830 tx=0x7f903002bcd0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.048+0000 7f903bd33640 1 -- 172.21.15.125:0/2771543394 shutdown_connections 2023-10-18T12:01:41.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.048+0000 7f903bd33640 1 --2- 172.21.15.125:0/2771543394 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f90340ceb40 0x7f90340b0650 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.048+0000 7f903bd33640 1 --2- 172.21.15.125:0/2771543394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90340cf340 0x7f90340d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.048+0000 7f903bd33640 1 -- 172.21.15.125:0/2771543394 >> 172.21.15.125:0/2771543394 conn(0x7f90340d0d60 msgr2=0x7f90340d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:41.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.048+0000 7f903bd33640 1 -- 172.21.15.125:0/2771543394 shutdown_connections 2023-10-18T12:01:41.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.048+0000 7f903bd33640 1 -- 172.21.15.125:0/2771543394 wait complete. 2023-10-18T12:01:41.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.049+0000 7f903bd33640 1 Processor -- start 2023-10-18T12:01:41.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.049+0000 7f903bd33640 1 -- start start 2023-10-18T12:01:41.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903bd33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90340ceb40 0x7f90341f1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903bd33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90340cf340 0x7f903415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903bd33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f903415b2a0 0x7f90340b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903bd33640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90341f1860 con 0x7f903415b2a0 2023-10-18T12:01:41.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903bd33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903415bb60 con 0x7f90340ceb40 2023-10-18T12:01:41.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903bd33640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903415be40 con 0x7f90340cf340 2023-10-18T12:01:41.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903a2d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f903415b2a0 0x7f90340b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903a2d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f903415b2a0 0x7f90340b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59512/0 (socket says 172.21.15.125:59512) 2023-10-18T12:01:41.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.050+0000 7f903a2d0640 1 -- 172.21.15.125:0/2005899966 learned_addr learned my addr 172.21.15.125:0/2005899966 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:41.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f903a2d0640 1 -- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90340cf340 msgr2=0x7f903415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f903a2d0640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90340cf340 0x7f903415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f903a2d0640 1 -- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90340ceb40 msgr2=0x7f90341f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f903a2d0640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90340ceb40 0x7f90341f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f903a2d0640 1 -- 172.21.15.125:0/2005899966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9030009490 con 0x7f903415b2a0 2023-10-18T12:01:41.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f903a2d0640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f903415b2a0 0x7f90340b1560 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f903002c2e0 tx=0x7f90300038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:41.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f902a7fc640 1 -- 172.21.15.125:0/2005899966 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030003ea0 con 0x7f903415b2a0 2023-10-18T12:01:41.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f902a7fc640 1 -- 172.21.15.125:0/2005899966 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9030005020 con 0x7f903415b2a0 2023-10-18T12:01:41.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903415c120 con 0x7f903415b2a0 2023-10-18T12:01:41.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f902a7fc640 1 -- 172.21.15.125:0/2005899966 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90300053b0 con 0x7f903415b2a0 2023-10-18T12:01:41.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.051+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903415b6d0 con 0x7f903415b2a0 2023-10-18T12:01:41.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.052+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903415c640 con 0x7f903415b2a0 2023-10-18T12:01:41.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.052+0000 7f902a7fc640 1 -- 172.21.15.125:0/2005899966 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 145) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f9030005510 con 0x7f903415b2a0 2023-10-18T12:01:41.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.053+0000 7f902a7fc640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f901806ac90 0x7f901806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.053+0000 7f902a7fc640 1 -- 172.21.15.125:0/2005899966 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7f90300ac330 con 0x7f903415b2a0 2023-10-18T12:01:41.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.053+0000 7f9039acf640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f901806ac90 0x7f901806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.053+0000 7f9039acf640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f901806ac90 0x7f901806d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f903406b050 tx=0x7f902c009710 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:41.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.057+0000 7f902a7fc640 1 -- 172.21.15.125:0/2005899966 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9030076ee0 con 0x7f903415b2a0 2023-10-18T12:01:41.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.227+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90340afdc0 con 0x7f903415b2a0 2023-10-18T12:01:41.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.229+0000 7f902a7fc640 1 -- 172.21.15.125:0/2005899966 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v248) v1 ==== 74+0+21848 (secure 0 0 0) 0x7f903004b090 con 0x7f903415b2a0 2023-10-18T12:01:41.232 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:41.233 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":248,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:39.247545+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3172661499":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1929890550":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6832/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1706867588":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/206803400":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1890059388":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/662473594":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/2446560793":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6833/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6832/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3037918695":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3823693717":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3119295615":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/668821395":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/3910260091":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/558106977":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2896936218":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6833/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/695463051":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2469983511":"2023-10-19T12:01:33.783814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:41.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f901806ac90 msgr2=0x7f901806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f901806ac90 0x7f901806d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f903406b050 tx=0x7f902c009710 comp rx=0 tx=0).stop 2023-10-18T12:01:41.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f903415b2a0 msgr2=0x7f90340b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f903415b2a0 0x7f90340b1560 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f903002c2e0 tx=0x7f90300038a0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 shutdown_connections 2023-10-18T12:01:41.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f903415b2a0 0x7f90340b1560 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90340cf340 0x7f903415d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f901806ac90 0x7f901806d150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 --2- 172.21.15.125:0/2005899966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90340ceb40 0x7f90341f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 >> 172.21.15.125:0/2005899966 conn(0x7f90340d0d60 msgr2=0x7f903415a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:41.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 shutdown_connections 2023-10-18T12:01:41.239 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.231+0000 7f903bd33640 1 -- 172.21.15.125:0/2005899966 wait complete. 2023-10-18T12:01:41.245 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T12:01:41.246 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:41.296 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:41.293+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:41.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.349+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2282582840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 msgr2=0x7fa0dc0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.349+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2282582840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 0x7fa0dc0cf720 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0005fd0 tx=0x7fa0d002bcf0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.349+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2282582840 shutdown_connections 2023-10-18T12:01:41.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.349+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2282582840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 0x7fa0dc0cf720 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.349+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2282582840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0cf340 0x7fa0dc0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.349+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2282582840 >> 172.21.15.125:0/2282582840 conn(0x7fa0dc0d0d60 msgr2=0x7fa0dc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:41.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.349+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2282582840 shutdown_connections 2023-10-18T12:01:41.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.349+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2282582840 wait complete. 2023-10-18T12:01:41.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.350+0000 7fa0e10d4640 1 Processor -- start 2023-10-18T12:01:41.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0e10d4640 1 -- start start 2023-10-18T12:01:41.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0e10d4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0ceb40 0x7fa0dc157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0e10d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0cf340 0x7fa0dc0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0e10d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 0x7fa0dc0ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0e10d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0b6900 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0e10d4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0c8cf0 con 0x7fa0dc0cf340 2023-10-18T12:01:41.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0e10d4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0b3c00 con 0x7fa0dc0ceb40 2023-10-18T12:01:41.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0db59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 0x7fa0dc0ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0db59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 0x7fa0dc0ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59530/0 (socket says 172.21.15.125:59530) 2023-10-18T12:01:41.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0db59e640 1 -- 172.21.15.125:0/2912545060 learned_addr learned my addr 172.21.15.125:0/2912545060 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:41.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0da59c640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0cf340 0x7fa0dc0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.351+0000 7fa0dad9d640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0ceb40 0x7fa0dc157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0db59e640 1 -- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0ceb40 msgr2=0x7fa0dc157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0db59e640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0ceb40 0x7fa0dc157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0db59e640 1 -- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0cf340 msgr2=0x7fa0dc0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0db59e640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0cf340 0x7fa0dc0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0db59e640 1 -- 172.21.15.125:0/2912545060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d0009560 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0db59e640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 0x7fa0dc0ae590 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c410 tx=0x7fa0d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:41.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0c37fe640 1 -- 172.21.15.125:0/2912545060 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4011790 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0c37fe640 1 -- 172.21.15.125:0/2912545060 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa0d40119e0 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc0b3ee0 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0c37fe640 1 -- 172.21.15.125:0/2912545060 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4011e30 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.352+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc0b44a0 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.353+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc0d04f0 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.353+0000 7fa0c37fe640 1 -- 172.21.15.125:0/2912545060 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 145) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa0d401c460 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.354+0000 7fa0c37fe640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa0c806ac90 0x7fa0c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.354+0000 7fa0c37fe640 1 -- 172.21.15.125:0/2912545060 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7fa0d40898e0 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.354+0000 7fa0dad9d640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa0c806ac90 0x7fa0c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.355+0000 7fa0dad9d640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa0c806ac90 0x7fa0c806d150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005fd0 tx=0x7fa0cc009680 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:41.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.358+0000 7fa0c37fe640 1 -- 172.21.15.125:0/2912545060 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0d4054540 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.571+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0dc0b4690 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.572+0000 7fa0c37fe640 1 -- 172.21.15.125:0/2912545060 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+6157 (secure 0 0 0) 0x7fa0d4057b60 con 0x7fa0dc15b2a0 2023-10-18T12:01:41.575 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:41.576 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":103,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:08.447292+0000","modified":"2023-10-18T12:01:10.725386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12771":{"gid":12771,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1851256737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1851256737},{"type":"v1","addr":"172.21.15.125:6835","nonce":1851256737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:41.577 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 103 2023-10-18T12:01:41.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa0c806ac90 msgr2=0x7fa0c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa0c806ac90 0x7fa0c806d150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005fd0 tx=0x7fa0cc009680 comp rx=0 tx=0).stop 2023-10-18T12:01:41.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 msgr2=0x7fa0dc0ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.577 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 0x7fa0dc0ae590 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c410 tx=0x7fa0d400c8d0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 shutdown_connections 2023-10-18T12:01:41.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa0c806ac90 0x7fa0c806d150 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0dc0cf340 0x7fa0dc0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0dc15b2a0 0x7fa0dc0ae590 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 --2- 172.21.15.125:0/2912545060 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0dc0ceb40 0x7fa0dc157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 >> 172.21.15.125:0/2912545060 conn(0x7fa0dc0d0d60 msgr2=0x7fa0dc0b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:41.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 shutdown_connections 2023-10-18T12:01:41.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.574+0000 7fa0e10d4640 1 -- 172.21.15.125:0/2912545060 wait complete. 2023-10-18T12:01:41.587 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:41.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.689+0000 7fba93b4c640 1 -- 172.21.15.125:0/2520671492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0afa20 msgr2=0x7fba8c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.689+0000 7fba93b4c640 1 --2- 172.21.15.125:0/2520671492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0afa20 0x7fba8c0705a0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0cc7e0 tx=0x7fba7c02bd70 comp rx=0 tx=0).stop 2023-10-18T12:01:41.692 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.689+0000 7fba93b4c640 1 -- 172.21.15.125:0/2520671492 shutdown_connections 2023-10-18T12:01:41.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.689+0000 7fba93b4c640 1 --2- 172.21.15.125:0/2520671492 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0afa20 0x7fba8c0705a0 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.689+0000 7fba93b4c640 1 --2- 172.21.15.125:0/2520671492 >> v2:172.21.15.139:3300/0 conn(0x7fba8c0cbd20 0x7fba8c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.689+0000 7fba93b4c640 1 -- 172.21.15.125:0/2520671492 >> 172.21.15.125:0/2520671492 conn(0x7fba8c069930 msgr2=0x7fba8c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:41.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.689+0000 7fba93b4c640 1 -- 172.21.15.125:0/2520671492 shutdown_connections 2023-10-18T12:01:41.693 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.689+0000 7fba93b4c640 1 -- 172.21.15.125:0/2520671492 wait complete. 2023-10-18T12:01:41.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.690+0000 7fba93b4c640 1 Processor -- start 2023-10-18T12:01:41.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.690+0000 7fba93b4c640 1 -- start start 2023-10-18T12:01:41.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba93b4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba8c0afa20 0x7fba8c143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba93b4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0b0530 0x7fba8c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba93b4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba8c0cbd20 0x7fba8c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba93b4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c15a370 con 0x7fba8c0b0530 2023-10-18T12:01:41.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba93b4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c1534a0 con 0x7fba8c0cbd20 2023-10-18T12:01:41.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba93b4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c143870 con 0x7fba8c0afa20 2023-10-18T12:01:41.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0b0530 0x7fba8c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0b0530 0x7fba8c151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59550/0 (socket says 172.21.15.125:59550) 2023-10-18T12:01:41.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 -- 172.21.15.125:0/3374250583 learned_addr learned my addr 172.21.15.125:0/3374250583 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:41.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba918e8640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba8c0afa20 0x7fba8c143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba920e9640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba8c0cbd20 0x7fba8c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 -- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba8c0afa20 msgr2=0x7fba8c143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba8c0afa20 0x7fba8c143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 -- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba8c0cbd20 msgr2=0x7fba8c0ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba8c0cbd20 0x7fba8c0ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.699 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 -- 172.21.15.125:0/3374250583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c009490 con 0x7fba8c0b0530 2023-10-18T12:01:41.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba920e9640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba8c0cbd20 0x7fba8c0ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:41.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba910e7640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0b0530 0x7fba8c151eb0 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7fba7c02c380 tx=0x7fba7c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:41.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba827fc640 1 -- 172.21.15.125:0/3374250583 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c003ea0 con 0x7fba8c0b0530 2023-10-18T12:01:41.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba827fc640 1 -- 172.21.15.125:0/3374250583 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fba7c005c90 con 0x7fba8c0b0530 2023-10-18T12:01:41.700 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.691+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c159db0 con 0x7fba8c0b0530 2023-10-18T12:01:41.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.692+0000 7fba827fc640 1 -- 172.21.15.125:0/3374250583 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c034640 con 0x7fba8c0b0530 2023-10-18T12:01:41.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.692+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c154650 con 0x7fba8c0b0530 2023-10-18T12:01:41.701 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.692+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c1544b0 con 0x7fba8c0b0530 2023-10-18T12:01:41.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.693+0000 7fba827fc640 1 -- 172.21.15.125:0/3374250583 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 145) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fba7c0347a0 con 0x7fba8c0b0530 2023-10-18T12:01:41.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.693+0000 7fba827fc640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fba7006ac90 0x7fba7006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:41.702 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.693+0000 7fba827fc640 1 -- 172.21.15.125:0/3374250583 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7fba7c0ac910 con 0x7fba8c0b0530 2023-10-18T12:01:41.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.693+0000 7fba918e8640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fba7006ac90 0x7fba7006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:41.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.694+0000 7fba918e8640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fba7006ac90 0x7fba7006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fba88005fd0 tx=0x7fba88009500 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:41.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.697+0000 7fba827fc640 1 -- 172.21.15.125:0/3374250583 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba8c1544b0 con 0x7fba8c0b0530 2023-10-18T12:01:41.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.905+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba8c1544b0 con 0x7fba8c0b0530 2023-10-18T12:01:41.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.906+0000 7fba827fc640 1 -- 172.21.15.125:0/3374250583 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+6157 (secure 0 0 0) 0x7fba8c1544b0 con 0x7fba8c0b0530 2023-10-18T12:01:41.909 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:41.909 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":103,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:08.447292+0000","modified":"2023-10-18T12:01:10.725386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12771":{"gid":12771,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1851256737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1851256737},{"type":"v1","addr":"172.21.15.125:6835","nonce":1851256737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:41.909 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 103 2023-10-18T12:01:41.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fba7006ac90 msgr2=0x7fba7006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fba7006ac90 0x7fba7006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fba88005fd0 tx=0x7fba88009500 comp rx=0 tx=0).stop 2023-10-18T12:01:41.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0b0530 msgr2=0x7fba8c151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:41.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0b0530 0x7fba8c151eb0 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7fba7c02c380 tx=0x7fba7c0038a0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 shutdown_connections 2023-10-18T12:01:41.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fba7006ac90 0x7fba7006d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba8c0cbd20 0x7fba8c0ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba8c0b0530 0x7fba8c151eb0 unknown :-1 s=CLOSED pgs=2921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 --2- 172.21.15.125:0/3374250583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fba8c0afa20 0x7fba8c143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:41.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.907+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 >> 172.21.15.125:0/3374250583 conn(0x7fba8c069930 msgr2=0x7fba8c0ac590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:41.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.908+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 shutdown_connections 2023-10-18T12:01:41.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:41.908+0000 7fba93b4c640 1 -- 172.21.15.125:0/3374250583 wait complete. 2023-10-18T12:01:41.921 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:41.974 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:41.971+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:42.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.022+0000 7f3d29da5640 1 -- 172.21.15.125:0/3915682347 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 msgr2=0x7f3d240d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.022+0000 7f3d29da5640 1 --2- 172.21.15.125:0/3915682347 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 0x7f3d240d1910 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009830 tx=0x7f3d1402bd60 comp rx=0 tx=0).stop 2023-10-18T12:01:42.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.022+0000 7f3d29da5640 1 -- 172.21.15.125:0/3915682347 shutdown_connections 2023-10-18T12:01:42.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.022+0000 7f3d29da5640 1 --2- 172.21.15.125:0/3915682347 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 0x7f3d240d1910 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.022+0000 7f3d29da5640 1 -- 172.21.15.125:0/3915682347 >> 172.21.15.125:0/3915682347 conn(0x7f3d240d0d60 msgr2=0x7f3d240d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:42.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.022+0000 7f3d29da5640 1 -- 172.21.15.125:0/3915682347 shutdown_connections 2023-10-18T12:01:42.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.022+0000 7f3d29da5640 1 -- 172.21.15.125:0/3915682347 wait complete. 2023-10-18T12:01:42.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.023+0000 7f3d29da5640 1 Processor -- start 2023-10-18T12:01:42.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.023+0000 7f3d29da5640 1 -- start start 2023-10-18T12:01:42.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d29da5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d240ceb40 0x7f3d240b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d29da5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 0x7f3d2414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d29da5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d2415b2a0 0x7f3d240b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d29da5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2414e6a0 con 0x7f3d240cf340 2023-10-18T12:01:42.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d29da5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2414e980 con 0x7f3d240ceb40 2023-10-18T12:01:42.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d29da5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2414ec60 con 0x7f3d2415b2a0 2023-10-18T12:01:42.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d1affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 0x7f3d2414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d1affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 0x7f3d2414dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59570/0 (socket says 172.21.15.125:59570) 2023-10-18T12:01:42.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d1affd640 1 -- 172.21.15.125:0/802077243 learned_addr learned my addr 172.21.15.125:0/802077243 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:42.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d237fe640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d240ceb40 0x7f3d240b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.024+0000 7f3d23fff640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d2415b2a0 0x7f3d240b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d1affd640 1 -- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d2415b2a0 msgr2=0x7f3d240b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d1affd640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d2415b2a0 0x7f3d240b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d1affd640 1 -- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d240ceb40 msgr2=0x7f3d240b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d1affd640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d240ceb40 0x7f3d240b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d1affd640 1 -- 172.21.15.125:0/802077243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14009490 con 0x7f3d240cf340 2023-10-18T12:01:42.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d1affd640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 0x7f3d2414dda0 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c430 tx=0x7f3d0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:42.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d20ff9640 1 -- 172.21.15.125:0/802077243 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c011790 con 0x7f3d240cf340 2023-10-18T12:01:42.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d20ff9640 1 -- 172.21.15.125:0/802077243 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d0c011a00 con 0x7f3d240cf340 2023-10-18T12:01:42.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2414ef40 con 0x7f3d240cf340 2023-10-18T12:01:42.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d20ff9640 1 -- 172.21.15.125:0/802077243 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c011e10 con 0x7f3d240cf340 2023-10-18T12:01:42.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.025+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d240b5e80 con 0x7f3d240cf340 2023-10-18T12:01:42.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.026+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2414f460 con 0x7f3d240cf340 2023-10-18T12:01:42.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.026+0000 7f3d20ff9640 1 -- 172.21.15.125:0/802077243 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 145) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3d0c01c460 con 0x7f3d240cf340 2023-10-18T12:01:42.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.027+0000 7f3d20ff9640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f3d0406acc0 0x7f3d0406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.027+0000 7f3d20ff9640 1 -- 172.21.15.125:0/802077243 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7f3d0c089b00 con 0x7f3d240cf340 2023-10-18T12:01:42.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.027+0000 7f3d237fe640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f3d0406acc0 0x7f3d0406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.027+0000 7f3d237fe640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f3d0406acc0 0x7f3d0406d180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d14036000 tx=0x7f3d14004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:42.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.031+0000 7f3d20ff9640 1 -- 172.21.15.125:0/802077243 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d2414f460 con 0x7f3d240cf340 2023-10-18T12:01:42.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.207+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d2414f460 con 0x7f3d240cf340 2023-10-18T12:01:42.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.209+0000 7f3d20ff9640 1 -- 172.21.15.125:0/802077243 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v248) v1 ==== 74+0+21848 (secure 0 0 0) 0x7f3d2414f460 con 0x7f3d240cf340 2023-10-18T12:01:42.212 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:42.212 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":248,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:39.247545+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:06.989961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"241","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:07.416244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3172661499":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1929890550":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6832/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1706867588":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/206803400":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1890059388":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/662473594":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/2446560793":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6833/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6832/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3037918695":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3823693717":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3119295615":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/668821395":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/3910260091":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/558106977":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2896936218":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6833/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/695463051":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2469983511":"2023-10-19T12:01:33.783814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:42.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f3d0406acc0 msgr2=0x7f3d0406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f3d0406acc0 0x7f3d0406d180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d14036000 tx=0x7f3d14004730 comp rx=0 tx=0).stop 2023-10-18T12:01:42.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 msgr2=0x7f3d2414dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 0x7f3d2414dda0 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c430 tx=0x7f3d0c00c8f0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 shutdown_connections 2023-10-18T12:01:42.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d2415b2a0 0x7f3d240b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d240cf340 0x7f3d2414dda0 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f3d0406acc0 0x7f3d0406d180 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 --2- 172.21.15.125:0/802077243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d240ceb40 0x7f3d240b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 >> 172.21.15.125:0/802077243 conn(0x7f3d240d0d60 msgr2=0x7f3d2406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:42.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 shutdown_connections 2023-10-18T12:01:42.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.211+0000 7f3d29da5640 1 -- 172.21.15.125:0/802077243 wait complete. 2023-10-18T12:01:42.225 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T12:01:42.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.330+0000 7fac00333640 1 -- 172.21.15.125:0/2427721438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf80ceb40 msgr2=0x7fabf80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.330+0000 7fac00333640 1 --2- 172.21.15.125:0/2427721438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf80ceb40 0x7fabf80b0650 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009830 tx=0x7fabf002bcd0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.330+0000 7fac00333640 1 -- 172.21.15.125:0/2427721438 shutdown_connections 2023-10-18T12:01:42.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.330+0000 7fac00333640 1 --2- 172.21.15.125:0/2427721438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf80ceb40 0x7fabf80b0650 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.330+0000 7fac00333640 1 --2- 172.21.15.125:0/2427721438 >> v2:172.21.15.139:3300/0 conn(0x7fabf80cf340 0x7fabf80d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.330+0000 7fac00333640 1 -- 172.21.15.125:0/2427721438 >> 172.21.15.125:0/2427721438 conn(0x7fabf80d0d60 msgr2=0x7fabf80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:42.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.330+0000 7fac00333640 1 -- 172.21.15.125:0/2427721438 shutdown_connections 2023-10-18T12:01:42.335 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.330+0000 7fac00333640 1 -- 172.21.15.125:0/2427721438 wait complete. 2023-10-18T12:01:42.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fac00333640 1 Processor -- start 2023-10-18T12:01:42.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fac00333640 1 -- start start 2023-10-18T12:01:42.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fac00333640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabf80ceb40 0x7fabf815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fac00333640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf80cf340 0x7fabf8158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fac00333640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf815b2a0 0x7fabf80b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fac00333640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf815c550 con 0x7fabf80ceb40 2023-10-18T12:01:42.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fac00333640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf815c830 con 0x7fabf80cf340 2023-10-18T12:01:42.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fac00333640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf815cb10 con 0x7fabf815b2a0 2023-10-18T12:01:42.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.332+0000 7fabfe0cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabf80ceb40 0x7fabf815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe0cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabf80ceb40 0x7fabf815d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59580/0 (socket says 172.21.15.125:59580) 2023-10-18T12:01:42.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe0cf640 1 -- 172.21.15.125:0/1725988798 learned_addr learned my addr 172.21.15.125:0/1725988798 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:42.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfd8ce640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf80cf340 0x7fabf8158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe8d0640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf815b2a0 0x7fabf80b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe0cf640 1 -- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf815b2a0 msgr2=0x7fabf80b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe0cf640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf815b2a0 0x7fabf80b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe0cf640 1 -- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf80cf340 msgr2=0x7fabf8158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe0cf640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf80cf340 0x7fabf8158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe0cf640 1 -- 172.21.15.125:0/1725988798 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf0009490 con 0x7fabf80ceb40 2023-10-18T12:01:42.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfd8ce640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf80cf340 0x7fabf8158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:42.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabfe0cf640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabf80ceb40 0x7fabf815d5f0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fabf400efb0 tx=0x7fabf400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:42.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabeeffd640 1 -- 172.21.15.125:0/1725988798 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf400b570 con 0x7fabf80ceb40 2023-10-18T12:01:42.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabeeffd640 1 -- 172.21.15.125:0/1725988798 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fabf400b7c0 con 0x7fabf80ceb40 2023-10-18T12:01:42.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf815cdf0 con 0x7fabf80ceb40 2023-10-18T12:01:42.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fabeeffd640 1 -- 172.21.15.125:0/1725988798 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf400bbd0 con 0x7fabf80ceb40 2023-10-18T12:01:42.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.333+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf8158740 con 0x7fabf80ceb40 2023-10-18T12:01:42.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.334+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf80690e0 con 0x7fabf80ceb40 2023-10-18T12:01:42.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.335+0000 7fabeeffd640 1 -- 172.21.15.125:0/1725988798 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fabf400a040 con 0x7fabf80ceb40 2023-10-18T12:01:42.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.335+0000 7fabeeffd640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fabdc06aff0 0x7fabdc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.335+0000 7fabeeffd640 1 -- 172.21.15.125:0/1725988798 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7809+0+0 (secure 0 0 0) 0x7fabf408a9c0 con 0x7fabf80ceb40 2023-10-18T12:01:42.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.335+0000 7fabfd8ce640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fabdc06aff0 0x7fabdc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.336+0000 7fabfd8ce640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fabdc06aff0 0x7fabdc06d4b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fabf80b6220 tx=0x7fabe8009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:42.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.339+0000 7fabeeffd640 1 -- 172.21.15.125:0/1725988798 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabf4054f70 con 0x7fabf80ceb40 2023-10-18T12:01:42.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.551+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fabf80b0d30 con 0x7fabf80ceb40 2023-10-18T12:01:42.554 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T12:01:42.552+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:01:42.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.553+0000 7fabeeffd640 1 -- 172.21.15.125:0/1725988798 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 161+0+0 (secure 0 0 0) 0x7fabf401b030 con 0x7fabf80ceb40 2023-10-18T12:01:42.556 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T12:01:42.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fabdc06aff0 msgr2=0x7fabdc06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fabdc06aff0 0x7fabdc06d4b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fabf80b6220 tx=0x7fabe8009040 comp rx=0 tx=0).stop 2023-10-18T12:01:42.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabf80ceb40 msgr2=0x7fabf815d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabf80ceb40 0x7fabf815d5f0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fabf400efb0 tx=0x7fabf400c800 comp rx=0 tx=0).stop 2023-10-18T12:01:42.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 shutdown_connections 2023-10-18T12:01:42.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fabdc06aff0 0x7fabdc06d4b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf80cf340 0x7fabf8158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf815b2a0 0x7fabf80b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 --2- 172.21.15.125:0/1725988798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fabf80ceb40 0x7fabf815d5f0 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.555+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 >> 172.21.15.125:0/1725988798 conn(0x7fabf80d0d60 msgr2=0x7fabf815a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:42.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.556+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 shutdown_connections 2023-10-18T12:01:42.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.556+0000 7fac00333640 1 -- 172.21.15.125:0/1725988798 wait complete. 2023-10-18T12:01:42.561 INFO:tasks.ceph.mds.a.smithi125.stderr: -3320> 2023-10-18T12:01:10.084+0000 7ffaa6c58640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:01:42.569 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T12:01:42.586 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T12:01:42.641 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:42.638+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:42.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 -- 172.21.15.125:0/1913731405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780afa20 msgr2=0x7f16780705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 --2- 172.21.15.125:0/1913731405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780afa20 0x7f16780705a0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f16780b1450 tx=0x7f166802bda0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 -- 172.21.15.125:0/1913731405 shutdown_connections 2023-10-18T12:01:42.677 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 --2- 172.21.15.125:0/1913731405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780afa20 0x7f16780705a0 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 --2- 172.21.15.125:0/1913731405 >> v2:172.21.15.139:3300/0 conn(0x7f16780b0530 0x7f16780b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 --2- 172.21.15.125:0/1913731405 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16780cbd20 0x7f1678158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 -- 172.21.15.125:0/1913731405 >> 172.21.15.125:0/1913731405 conn(0x7f1678069930 msgr2=0x7f1678069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:42.678 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 -- 172.21.15.125:0/1913731405 shutdown_connections 2023-10-18T12:01:42.679 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.674+0000 7f16802f2640 1 -- 172.21.15.125:0/1913731405 wait complete. 2023-10-18T12:01:42.681 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.675+0000 7f16802f2640 1 Processor -- start 2023-10-18T12:01:42.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.675+0000 7f16802f2640 1 -- start start 2023-10-18T12:01:42.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f16802f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16780afa20 0x7f1678141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f16802f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780b0530 0x7f1678142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.682 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f16802f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16780cbd20 0x7f1678145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f16802f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678130060 con 0x7f16780b0530 2023-10-18T12:01:42.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f16802f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167812d140 con 0x7f16780afa20 2023-10-18T12:01:42.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f16802f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167812d420 con 0x7f16780cbd20 2023-10-18T12:01:42.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780b0530 0x7f1678142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780b0530 0x7f1678142dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59628/0 (socket says 172.21.15.125:59628) 2023-10-18T12:01:42.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 -- 172.21.15.125:0/2917502522 learned_addr learned my addr 172.21.15.125:0/2917502522 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:42.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167e88f640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16780cbd20 0x7f1678145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167e08e640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16780afa20 0x7f1678141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 -- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16780cbd20 msgr2=0x7f1678145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.684 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16780cbd20 0x7f1678145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 -- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16780afa20 msgr2=0x7f1678141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:42.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16780afa20 0x7f1678141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:42.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 -- 172.21.15.125:0/2917502522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668009560 con 0x7f16780b0530 2023-10-18T12:01:42.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f167d88d640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780b0530 0x7f1678142dc0 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f1668000c00 tx=0x7f166802da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:42.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f166effd640 1 -- 172.21.15.125:0/2917502522 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668003800 con 0x7f16780b0530 2023-10-18T12:01:42.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f166effd640 1 -- 172.21.15.125:0/2917502522 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1668003a70 con 0x7f16780b0530 2023-10-18T12:01:42.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167812d700 con 0x7f16780b0530 2023-10-18T12:01:42.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.676+0000 7f166effd640 1 -- 172.21.15.125:0/2917502522 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668003e80 con 0x7f16780b0530 2023-10-18T12:01:42.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.677+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167812dcc0 con 0x7f16780b0530 2023-10-18T12:01:42.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.677+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16780690e0 con 0x7f16780b0530 2023-10-18T12:01:42.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.678+0000 7f166effd640 1 -- 172.21.15.125:0/2917502522 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f166803e450 con 0x7f16780b0530 2023-10-18T12:01:42.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.678+0000 7f166effd640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f165c06aff0 0x7f165c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:42.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.678+0000 7f166effd640 1 -- 172.21.15.125:0/2917502522 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 7895+0+0 (secure 0 0 0) 0x7f16680acad0 con 0x7f16780b0530 2023-10-18T12:01:42.687 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.678+0000 7f167e08e640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f165c06aff0 0x7f165c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:42.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.679+0000 7f167e08e640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f165c06aff0 0x7f165c06d4b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1674009b30 tx=0x7f1674005f90 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:42.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.683+0000 7f166effd640 1 -- 172.21.15.125:0/2917502522 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1668077220 con 0x7f16780b0530 2023-10-18T12:01:42.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:42.894+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f167806f700 con 0x7f16780b0530 2023-10-18T12:01:43.312 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:43.309+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:43.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.554+0000 7f166effd640 1 -- 172.21.15.125:0/2917502522 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 104+0+0 (secure 0 0 0) 0x7f166807a840 con 0x7f16780b0530 2023-10-18T12:01:43.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.556+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f165c06aff0 msgr2=0x7f165c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:43.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.556+0000 7f16802f2640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f165c06aff0 0x7f165c06d4b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1674009b30 tx=0x7f1674005f90 comp rx=0 tx=0).stop 2023-10-18T12:01:43.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.556+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780b0530 msgr2=0x7f1678142dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:43.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.556+0000 7f16802f2640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780b0530 0x7f1678142dc0 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f1668000c00 tx=0x7f166802da00 comp rx=0 tx=0).stop 2023-10-18T12:01:43.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.557+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 shutdown_connections 2023-10-18T12:01:43.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.557+0000 7f16802f2640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f16780cbd20 0x7f1678145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:43.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.557+0000 7f16802f2640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f16780b0530 0x7f1678142dc0 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:43.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.557+0000 7f16802f2640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f165c06aff0 0x7f165c06d4b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:43.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.557+0000 7f16802f2640 1 --2- 172.21.15.125:0/2917502522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16780afa20 0x7f1678141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:43.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.557+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 >> 172.21.15.125:0/2917502522 conn(0x7f1678069930 msgr2=0x7f1678157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:43.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.557+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 shutdown_connections 2023-10-18T12:01:43.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.557+0000 7f16802f2640 1 -- 172.21.15.125:0/2917502522 wait complete. 2023-10-18T12:01:43.570 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T12:01:43.718 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.715+0000 7f41fb5f5640 1 -- 172.21.15.125:0/364015172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40afa20 msgr2=0x7f41f4158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:43.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.715+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/364015172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40afa20 0x7f41f4158300 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f41e4005fd0 tx=0x7f41e402bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:43.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.715+0000 7f41fb5f5640 1 -- 172.21.15.125:0/364015172 shutdown_connections 2023-10-18T12:01:43.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.715+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/364015172 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41f40b0530 0x7f41f40b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:43.719 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.715+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/364015172 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40afa20 0x7f41f4158300 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:43.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.715+0000 7f41fb5f5640 1 -- 172.21.15.125:0/364015172 >> 172.21.15.125:0/364015172 conn(0x7f41f4069930 msgr2=0x7f41f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:43.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.715+0000 7f41fb5f5640 1 -- 172.21.15.125:0/364015172 shutdown_connections 2023-10-18T12:01:43.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.715+0000 7f41fb5f5640 1 -- 172.21.15.125:0/364015172 wait complete. 2023-10-18T12:01:43.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41fb5f5640 1 Processor -- start 2023-10-18T12:01:43.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41fb5f5640 1 -- start start 2023-10-18T12:01:43.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41fb5f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41f40afa20 0x7f41f40ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:43.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41fb5f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40b0530 0x7f41f4154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:43.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41fb5f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41f40cbd20 0x7f41f4155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:43.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41fb5f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40a6870 con 0x7f41f40b0530 2023-10-18T12:01:43.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41fb5f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4155750 con 0x7f41f40cbd20 2023-10-18T12:01:43.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41fb5f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4156560 con 0x7f41f40afa20 2023-10-18T12:01:43.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41f8b90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40b0530 0x7f41f4154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:43.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41f9391640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41f40afa20 0x7f41f40ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:43.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41f8b90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40b0530 0x7f41f4154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59646/0 (socket says 172.21.15.125:59646) 2023-10-18T12:01:43.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41f8b90640 1 -- 172.21.15.125:0/2370423152 learned_addr learned my addr 172.21.15.125:0/2370423152 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:43.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.717+0000 7f41f9b92640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41f40cbd20 0x7f41f4155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:43.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41f8b90640 1 -- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41f40afa20 msgr2=0x7f41f40ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:43.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41f8b90640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41f40afa20 0x7f41f40ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:43.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41f8b90640 1 -- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41f40cbd20 msgr2=0x7f41f4155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41f8b90640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41f40cbd20 0x7f41f4155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41f8b90640 1 -- 172.21.15.125:0/2370423152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4009560 con 0x7f41f40b0530 2023-10-18T12:01:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41f8b90640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40b0530 0x7f41f4154cd0 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006010 tx=0x7f41e402db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41e9ffb640 1 -- 172.21.15.125:0/2370423152 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40035f0 con 0x7f41f40b0530 2023-10-18T12:01:43.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41e9ffb640 1 -- 172.21.15.125:0/2370423152 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f41e4003840 con 0x7f41f40b0530 2023-10-18T12:01:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f4155950 con 0x7f41f40b0530 2023-10-18T12:01:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41e9ffb640 1 -- 172.21.15.125:0/2370423152 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4003c50 con 0x7f41f40b0530 2023-10-18T12:01:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.718+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f4155ce0 con 0x7f41f40b0530 2023-10-18T12:01:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.719+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f40690e0 con 0x7f41f40b0530 2023-10-18T12:01:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.720+0000 7f41e9ffb640 1 -- 172.21.15.125:0/2370423152 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f41e403e450 con 0x7f41f40b0530 2023-10-18T12:01:43.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.720+0000 7f41e9ffb640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f41d806aff0 0x7f41d806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:43.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.720+0000 7f41e9ffb640 1 -- 172.21.15.125:0/2370423152 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 7895+0+0 (secure 0 0 0) 0x7f41e40acc10 con 0x7f41f40b0530 2023-10-18T12:01:43.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.720+0000 7f41f9391640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f41d806aff0 0x7f41d806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:43.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.721+0000 7f41f9391640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f41d806aff0 0x7f41d806d4b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006fd0 tx=0x7f41f0009060 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:43.730 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.724+0000 7f41e9ffb640 1 -- 172.21.15.125:0/2370423152 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41e4077380 con 0x7f41f40b0530 2023-10-18T12:01:43.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:43.897+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f41f41566a0 con 0x7f41f40b0530 2023-10-18T12:01:43.979 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:43.976+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:44.660 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:44.657+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:44.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.865+0000 7f41e9ffb640 1 -- 172.21.15.125:0/2370423152 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v250) v1 ==== 181+0+0 (secure 0 0 0) 0x7f41e4043020 con 0x7f41f40b0530 2023-10-18T12:01:44.868 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T12:01:44.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f41d806aff0 msgr2=0x7f41d806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:44.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f41d806aff0 0x7f41d806d4b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006fd0 tx=0x7f41f0009060 comp rx=0 tx=0).stop 2023-10-18T12:01:44.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40b0530 msgr2=0x7f41f4154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:44.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40b0530 0x7f41f4154cd0 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006010 tx=0x7f41e402db30 comp rx=0 tx=0).stop 2023-10-18T12:01:44.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 shutdown_connections 2023-10-18T12:01:44.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f41d806aff0 0x7f41d806d4b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41f40cbd20 0x7f41f4155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41f40b0530 0x7f41f4154cd0 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 --2- 172.21.15.125:0/2370423152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41f40afa20 0x7f41f40ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.868+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 >> 172.21.15.125:0/2370423152 conn(0x7f41f4069930 msgr2=0x7f41f4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:44.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.869+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 shutdown_connections 2023-10-18T12:01:44.873 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.869+0000 7f41fb5f5640 1 -- 172.21.15.125:0/2370423152 wait complete. 2023-10-18T12:01:44.883 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T12:01:44.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.988+0000 7f179076c640 1 -- 172.21.15.125:0/896510677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f178815b2a0 msgr2=0x7f17880cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:44.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.988+0000 7f179076c640 1 --2- 172.21.15.125:0/896510677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f178815b2a0 0x7f17880cf720 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f178808a810 tx=0x7f177802bcf0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.988+0000 7f179076c640 1 -- 172.21.15.125:0/896510677 shutdown_connections 2023-10-18T12:01:44.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.988+0000 7f179076c640 1 --2- 172.21.15.125:0/896510677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f178815b2a0 0x7f17880cf720 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.988+0000 7f179076c640 1 --2- 172.21.15.125:0/896510677 >> v2:172.21.15.139:3300/0 conn(0x7f17880cf340 0x7f17880d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.988+0000 7f179076c640 1 -- 172.21.15.125:0/896510677 >> 172.21.15.125:0/896510677 conn(0x7f17880d0d60 msgr2=0x7f17880d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:44.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.988+0000 7f179076c640 1 -- 172.21.15.125:0/896510677 shutdown_connections 2023-10-18T12:01:44.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.988+0000 7f179076c640 1 -- 172.21.15.125:0/896510677 wait complete. 2023-10-18T12:01:44.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.989+0000 7f179076c640 1 Processor -- start 2023-10-18T12:01:44.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.989+0000 7f179076c640 1 -- start start 2023-10-18T12:01:44.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f179076c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17880ceb40 0x7f17880c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:44.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f179076c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17880cf340 0x7f1788156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:44.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f179076c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f178815b2a0 0x7f17880c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:44.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f179076c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1788159250 con 0x7f17880cf340 2023-10-18T12:01:44.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f179076c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17880c89e0 con 0x7f17880ceb40 2023-10-18T12:01:44.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f179076c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1788158c90 con 0x7f178815b2a0 2023-10-18T12:01:44.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f178dd07640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17880cf340 0x7f1788156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:44.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f178dd07640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17880cf340 0x7f1788156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59666/0 (socket says 172.21.15.125:59666) 2023-10-18T12:01:44.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f178dd07640 1 -- 172.21.15.125:0/2984613888 learned_addr learned my addr 172.21.15.125:0/2984613888 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:44.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f178dd07640 1 -- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f178815b2a0 msgr2=0x7f17880c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:44.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f178dd07640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f178815b2a0 0x7f17880c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.990+0000 7f178dd07640 1 -- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17880ceb40 msgr2=0x7f17880c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:44.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.991+0000 7f178dd07640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17880ceb40 0x7f17880c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:44.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.991+0000 7f178dd07640 1 -- 172.21.15.125:0/2984613888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778009490 con 0x7f17880cf340 2023-10-18T12:01:44.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.991+0000 7f178dd07640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17880cf340 0x7f1788156970 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f1778000c00 tx=0x7f177802c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:44.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.991+0000 7f177effd640 1 -- 172.21.15.125:0/2984613888 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778003a80 con 0x7f17880cf340 2023-10-18T12:01:44.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.991+0000 7f177effd640 1 -- 172.21.15.125:0/2984613888 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1778003cf0 con 0x7f17880cf340 2023-10-18T12:01:45.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.991+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1788158f70 con 0x7f17880cf340 2023-10-18T12:01:45.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.991+0000 7f177effd640 1 -- 172.21.15.125:0/2984613888 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780346a0 con 0x7f17880cf340 2023-10-18T12:01:45.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.991+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1788156eb0 con 0x7f17880cf340 2023-10-18T12:01:45.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.992+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17880c8ec0 con 0x7f17880cf340 2023-10-18T12:01:45.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.992+0000 7f177effd640 1 -- 172.21.15.125:0/2984613888 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f177803b040 con 0x7f17880cf340 2023-10-18T12:01:45.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.993+0000 7f177effd640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f176c06aff0 0x7f176c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:45.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.993+0000 7f177effd640 1 -- 172.21.15.125:0/2984613888 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 7457+0+0 (secure 0 0 0) 0x7f17780ac190 con 0x7f17880cf340 2023-10-18T12:01:45.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.993+0000 7f178e508640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f176c06aff0 0x7f176c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:45.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.993+0000 7f178e508640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f176c06aff0 0x7f176c06d4b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1784005fd0 tx=0x7f1784009710 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:45.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:44.997+0000 7f177effd640 1 -- 172.21.15.125:0/2984613888 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17880c8ec0 con 0x7f17880cf340 2023-10-18T12:01:45.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.171+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f17880c8ec0 con 0x7f17880cf340 2023-10-18T12:01:45.339 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:45.336+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:45.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.869+0000 7f177effd640 1 -- 172.21.15.125:0/2984613888 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v251) v1 ==== 169+0+0 (secure 0 0 0) 0x7f17880c8ec0 con 0x7f17880cf340 2023-10-18T12:01:45.872 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T12:01:45.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f176c06aff0 msgr2=0x7f176c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:45.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f176c06aff0 0x7f176c06d4b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1784005fd0 tx=0x7f1784009710 comp rx=0 tx=0).stop 2023-10-18T12:01:45.874 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17880cf340 msgr2=0x7f1788156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:45.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17880cf340 0x7f1788156970 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f1778000c00 tx=0x7f177802c880 comp rx=0 tx=0).stop 2023-10-18T12:01:45.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 shutdown_connections 2023-10-18T12:01:45.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f178815b2a0 0x7f17880c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:45.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17880cf340 0x7f1788156970 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:45.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f176c06aff0 0x7f176c06d4b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:45.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 --2- 172.21.15.125:0/2984613888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17880ceb40 0x7f17880c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:45.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 >> 172.21.15.125:0/2984613888 conn(0x7f17880d0d60 msgr2=0x7f1788069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:45.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 shutdown_connections 2023-10-18T12:01:45.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.871+0000 7f179076c640 1 -- 172.21.15.125:0/2984613888 wait complete. 2023-10-18T12:01:45.885 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:45.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.986+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/2099696676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 msgr2=0x7f7bcc0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:45.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.986+0000 7f7bd0fb6640 1 --2- 172.21.15.125:0/2099696676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 0x7f7bcc0d1910 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009830 tx=0x7f7bbc02bd60 comp rx=0 tx=0).stop 2023-10-18T12:01:45.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.986+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/2099696676 shutdown_connections 2023-10-18T12:01:45.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.986+0000 7f7bd0fb6640 1 --2- 172.21.15.125:0/2099696676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 0x7f7bcc0d1910 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:45.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.986+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/2099696676 >> 172.21.15.125:0/2099696676 conn(0x7f7bcc0d0d60 msgr2=0x7f7bcc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:45.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.986+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/2099696676 shutdown_connections 2023-10-18T12:01:45.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.986+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/2099696676 wait complete. 2023-10-18T12:01:45.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bd0fb6640 1 Processor -- start 2023-10-18T12:01:45.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bd0fb6640 1 -- start start 2023-10-18T12:01:45.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bd0fb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bcc0ceb40 0x7f7bcc158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:45.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bd0fb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 0x7f7bcc1f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:45.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bd0fb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bcc15b2a0 0x7f7bcc09df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:45.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bd0fb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b6780 con 0x7f7bcc0cf340 2023-10-18T12:01:45.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bd0fb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b38e0 con 0x7f7bcc15b2a0 2023-10-18T12:01:45.994 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bd0fb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b3ea0 con 0x7f7bcc0ceb40 2023-10-18T12:01:45.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bc3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 0x7f7bcc1f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:45.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bc3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 0x7f7bcc1f1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59690/0 (socket says 172.21.15.125:59690) 2023-10-18T12:01:45.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.988+0000 7f7bc3fff640 1 -- 172.21.15.125:0/4208938913 learned_addr learned my addr 172.21.15.125:0/4208938913 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:45.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bca59c640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bcc0ceb40 0x7f7bcc158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:45.995 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bcad9d640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bcc15b2a0 0x7f7bcc09df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:45.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc3fff640 1 -- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bcc0ceb40 msgr2=0x7f7bcc158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:45.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc3fff640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bcc0ceb40 0x7f7bcc158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:45.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc3fff640 1 -- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bcc15b2a0 msgr2=0x7f7bcc09df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:45.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc3fff640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bcc15b2a0 0x7f7bcc09df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:45.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc3fff640 1 -- 172.21.15.125:0/4208938913 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc009490 con 0x7f7bcc0cf340 2023-10-18T12:01:45.996 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bca59c640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bcc0ceb40 0x7f7bcc158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:45.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc3fff640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 0x7f7bcc1f1970 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400c3f0 tx=0x7f7bb400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:45.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc2ffd640 1 -- 172.21.15.125:0/4208938913 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4011790 con 0x7f7bcc0cf340 2023-10-18T12:01:45.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc2ffd640 1 -- 172.21.15.125:0/4208938913 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7bb40119e0 con 0x7f7bcc0cf340 2023-10-18T12:01:45.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bc2ffd640 1 -- 172.21.15.125:0/4208938913 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4011dd0 con 0x7f7bcc0cf340 2023-10-18T12:01:45.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc158ed0 con 0x7f7bcc0cf340 2023-10-18T12:01:45.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.989+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1f2740 con 0x7f7bcc0cf340 2023-10-18T12:01:45.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.990+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc1f28d0 con 0x7f7bcc0cf340 2023-10-18T12:01:45.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.991+0000 7f7bc2ffd640 1 -- 172.21.15.125:0/4208938913 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f7bb401c460 con 0x7f7bcc0cf340 2023-10-18T12:01:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.991+0000 7f7bc2ffd640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7bac06aff0 0x7f7bac06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.991+0000 7f7bc2ffd640 1 -- 172.21.15.125:0/4208938913 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f7bb4089fb0 con 0x7f7bcc0cf340 2023-10-18T12:01:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.991+0000 7f7bca59c640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7bac06aff0 0x7f7bac06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:45.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.992+0000 7f7bca59c640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7bac06aff0 0x7f7bac06d4b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc036000 tx=0x7f7bbc004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:46.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:45.995+0000 7f7bc2ffd640 1 -- 172.21.15.125:0/4208938913 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bb40bcca0 con 0x7f7bcc0cf340 2023-10-18T12:01:46.034 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:46.031+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:46.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.165+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bcc0690e0 con 0x7f7bcc0cf340 2023-10-18T12:01:46.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.166+0000 7f7bc2ffd640 1 -- 172.21.15.125:0/4208938913 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v251) v1 ==== 74+0+18268 (secure 0 0 0) 0x7f7bb40548a0 con 0x7f7bcc0cf340 2023-10-18T12:01:46.169 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:46.169 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":251,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:45.866554+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3172661499":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1929890550":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6832/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.125:6835/1851256737":"2023-10-19T12:01:42.552858+0000","172.21.15.139:0/1706867588":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/206803400":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1890059388":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/662473594":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/2446560793":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6833/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6832/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3037918695":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3823693717":"2023-10-19T12:01:33.783814+0000","172.21.15.125:6833/1851256737":"2023-10-19T12:01:42.552858+0000","172.21.15.139:0/3119295615":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/668821395":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/3910260091":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/558106977":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2896936218":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6833/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/695463051":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2469983511":"2023-10-19T12:01:33.783814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:46.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7bac06aff0 msgr2=0x7f7bac06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7bac06aff0 0x7f7bac06d4b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc036000 tx=0x7f7bbc004730 comp rx=0 tx=0).stop 2023-10-18T12:01:46.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 msgr2=0x7f7bcc1f1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 0x7f7bcc1f1970 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400c3f0 tx=0x7f7bb400c8b0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 shutdown_connections 2023-10-18T12:01:46.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7bac06aff0 0x7f7bac06d4b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bcc15b2a0 0x7f7bcc09df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7bcc0cf340 0x7f7bcc1f1970 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 --2- 172.21.15.125:0/4208938913 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bcc0ceb40 0x7f7bcc158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 >> 172.21.15.125:0/4208938913 conn(0x7f7bcc0d0d60 msgr2=0x7f7bcc0b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:46.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 shutdown_connections 2023-10-18T12:01:46.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.168+0000 7f7bd0fb6640 1 -- 172.21.15.125:0/4208938913 wait complete. 2023-10-18T12:01:46.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-18T12:01:18.203802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '245', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-18T12:01:18.758795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '246', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:01:46.183 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:01:46.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.287+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/3564973181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc0003970 msgr2=0x7f4cc0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.290 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.287+0000 7f4cdeb3c640 1 --2- 172.21.15.125:0/3564973181 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc0003970 0x7f4cc0003d90 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400bb50 tx=0x7f4cc00057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.287+0000 7f4cdd0d9640 1 -- 172.21.15.125:0/3564973181 reap_dead start 2023-10-18T12:01:46.291 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.287+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/3564973181 shutdown_connections 2023-10-18T12:01:46.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.287+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/3564973181 >> 172.21.15.125:0/3564973181 conn(0x7f4cd80d0d60 msgr2=0x7f4cd80d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:46.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.287+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/3564973181 shutdown_connections 2023-10-18T12:01:46.292 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.287+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/3564973181 wait complete. 2023-10-18T12:01:46.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.288+0000 7f4cdeb3c640 1 Processor -- start 2023-10-18T12:01:46.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.288+0000 7f4cdeb3c640 1 -- start start 2023-10-18T12:01:46.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdeb3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc0003970 0x7f4cd80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdeb3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cd8144b30 0x7f4cd815d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdeb3c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cd815ded0 0x7f4cd80b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdeb3c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd806a9a0 con 0x7f4cd815ded0 2023-10-18T12:01:46.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdeb3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd806b520 con 0x7f4cc0003970 2023-10-18T12:01:46.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdeb3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd806abf0 con 0x7f4cd8144b30 2023-10-18T12:01:46.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdd0d9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cd815ded0 0x7f4cd80b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdd0d9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cd815ded0 0x7f4cd80b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59696/0 (socket says 172.21.15.125:59696) 2023-10-18T12:01:46.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdd0d9640 1 -- 172.21.15.125:0/1148789651 learned_addr learned my addr 172.21.15.125:0/1148789651 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:46.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdc8d8640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc0003970 0x7f4cd80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4ccffff640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cd8144b30 0x7f4cd815d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdd0d9640 1 -- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cd8144b30 msgr2=0x7f4cd815d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdd0d9640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cd8144b30 0x7f4cd815d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdd0d9640 1 -- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc0003970 msgr2=0x7f4cd80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdd0d9640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc0003970 0x7f4cd80b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.289+0000 7f4cdd0d9640 1 -- 172.21.15.125:0/1148789651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd806aed0 con 0x7f4cd815ded0 2023-10-18T12:01:46.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.290+0000 7f4cdd0d9640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cd815ded0 0x7f4cd80b5550 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80cef20 tx=0x7f4cd0004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:46.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.290+0000 7f4ccd7fa640 1 -- 172.21.15.125:0/1148789651 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0010d10 con 0x7f4cd815ded0 2023-10-18T12:01:46.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.290+0000 7f4ccd7fa640 1 -- 172.21.15.125:0/1148789651 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4cd0010e90 con 0x7f4cd815ded0 2023-10-18T12:01:46.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.290+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd806b1b0 con 0x7f4cd815ded0 2023-10-18T12:01:46.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.290+0000 7f4ccd7fa640 1 -- 172.21.15.125:0/1148789651 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd001a6d0 con 0x7f4cd815ded0 2023-10-18T12:01:46.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.290+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd815e350 con 0x7f4cd815ded0 2023-10-18T12:01:46.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.291+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd806b340 con 0x7f4cd815ded0 2023-10-18T12:01:46.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.291+0000 7f4ccd7fa640 1 -- 172.21.15.125:0/1148789651 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f4cd0017070 con 0x7f4cd815ded0 2023-10-18T12:01:46.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.291+0000 7f4ccd7fa640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f4cc406aff0 0x7f4cc406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.292+0000 7f4ccd7fa640 1 -- 172.21.15.125:0/1148789651 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f4cd008a9a0 con 0x7f4cd815ded0 2023-10-18T12:01:46.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.292+0000 7f4cdc8d8640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f4cc406aff0 0x7f4cc406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.292+0000 7f4cdc8d8640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f4cc406aff0 0x7f4cc406d4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400bb50 tx=0x7f4cd402e040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:46.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.296+0000 7f4ccd7fa640 1 -- 172.21.15.125:0/1148789651 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4cd00552b0 con 0x7f4cd815ded0 2023-10-18T12:01:46.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.468+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cd815e630 con 0x7f4cd815ded0 2023-10-18T12:01:46.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.469+0000 7f4ccd7fa640 1 -- 172.21.15.125:0/1148789651 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v251) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cd00588d0 con 0x7f4cd815ded0 2023-10-18T12:01:46.472 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:01:46.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f4cc406aff0 msgr2=0x7f4cc406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f4cc406aff0 0x7f4cc406d4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400bb50 tx=0x7f4cd402e040 comp rx=0 tx=0).stop 2023-10-18T12:01:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cd815ded0 msgr2=0x7f4cd80b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cd815ded0 0x7f4cd80b5550 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80cef20 tx=0x7f4cd0004700 comp rx=0 tx=0).stop 2023-10-18T12:01:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 shutdown_connections 2023-10-18T12:01:46.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cd815ded0 0x7f4cd80b5550 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cd8144b30 0x7f4cd815d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f4cc406aff0 0x7f4cc406d4b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 --2- 172.21.15.125:0/1148789651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc0003970 0x7f4cd80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 >> 172.21.15.125:0/1148789651 conn(0x7f4cd80d0d60 msgr2=0x7f4cd815a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:46.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 shutdown_connections 2023-10-18T12:01:46.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.471+0000 7f4cdeb3c640 1 -- 172.21.15.125:0/1148789651 wait complete. 2023-10-18T12:01:46.484 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-18T12:01:46.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.598+0000 7f809a16c640 1 -- 172.21.15.125:0/3384373723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f807c005180 msgr2=0x7f807c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.598+0000 7f809a16c640 1 --2- 172.21.15.125:0/3384373723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f807c005180 0x7f807c00b980 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7f808c009ed0 tx=0x7f807c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.598+0000 7f809a16c640 1 -- 172.21.15.125:0/3384373723 shutdown_connections 2023-10-18T12:01:46.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.598+0000 7f809a16c640 1 --2- 172.21.15.125:0/3384373723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f807c005180 0x7f807c00b980 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.598+0000 7f809a16c640 1 -- 172.21.15.125:0/3384373723 >> 172.21.15.125:0/3384373723 conn(0x7f8094069930 msgr2=0x7f8094069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:46.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.598+0000 7f809a16c640 1 -- 172.21.15.125:0/3384373723 shutdown_connections 2023-10-18T12:01:46.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.598+0000 7f809a16c640 1 -- 172.21.15.125:0/3384373723 wait complete. 2023-10-18T12:01:46.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.599+0000 7f809a16c640 1 Processor -- start 2023-10-18T12:01:46.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.599+0000 7f809a16c640 1 -- start start 2023-10-18T12:01:46.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f809a16c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c005180 0x7f80941e2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f809a16c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80940ab3a0 0x7f80941381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f809a16c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f809413a800 0x7f80940ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f809a16c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80940aaf10 con 0x7f809413a800 2023-10-18T12:01:46.605 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f809a16c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941386e0 con 0x7f807c005180 2023-10-18T12:01:46.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f809a16c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80941e2be0 con 0x7f80940ab3a0 2023-10-18T12:01:46.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8093fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f809413a800 0x7f80940ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8093fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f809413a800 0x7f80940ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59710/0 (socket says 172.21.15.125:59710) 2023-10-18T12:01:46.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8093fff640 1 -- 172.21.15.125:0/2424515849 learned_addr learned my addr 172.21.15.125:0/2424515849 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:46.606 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f80937fe640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c005180 0x7f80941e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8092ffd640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80940ab3a0 0x7f80941381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8093fff640 1 -- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80940ab3a0 msgr2=0x7f80941381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8093fff640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80940ab3a0 0x7f80941381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8093fff640 1 -- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c005180 msgr2=0x7f80941e2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.607 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8093fff640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c005180 0x7f80941e2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8093fff640 1 -- 172.21.15.125:0/2424515849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80941e2ec0 con 0x7f809413a800 2023-10-18T12:01:46.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f8092ffd640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80940ab3a0 0x7f80941381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:46.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.600+0000 7f80937fe640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c005180 0x7f80941e2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:46.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.601+0000 7f8093fff640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f809413a800 0x7f80940ca870 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f808c009e20 tx=0x7f808c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:46.608 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.601+0000 7f807bfff640 1 -- 172.21.15.125:0/2424515849 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c03d070 con 0x7f809413a800 2023-10-18T12:01:46.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.601+0000 7f807bfff640 1 -- 172.21.15.125:0/2424515849 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f808c004370 con 0x7f809413a800 2023-10-18T12:01:46.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.601+0000 7f807bfff640 1 -- 172.21.15.125:0/2424515849 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c004700 con 0x7f809413a800 2023-10-18T12:01:46.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.601+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80940aa8c0 con 0x7f809413a800 2023-10-18T12:01:46.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.601+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809413ac90 con 0x7f809413a800 2023-10-18T12:01:46.609 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.602+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809413ae20 con 0x7f809413a800 2023-10-18T12:01:46.610 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.602+0000 7f807bfff640 1 -- 172.21.15.125:0/2424515849 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f808c0304c0 con 0x7f809413a800 2023-10-18T12:01:46.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.603+0000 7f807bfff640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f808006aff0 0x7f808006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.603+0000 7f807bfff640 1 -- 172.21.15.125:0/2424515849 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f808c0abf40 con 0x7f809413a800 2023-10-18T12:01:46.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.603+0000 7f80937fe640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f808006aff0 0x7f808006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.611 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.603+0000 7f80937fe640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f808006aff0 0x7f808006d4b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8084001ce0 tx=0x7f8084001b90 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:46.612 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.607+0000 7f807bfff640 1 -- 172.21.15.125:0/2424515849 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f808c076610 con 0x7f809413a800 2023-10-18T12:01:46.718 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:46.715+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:46.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.784+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80940690e0 con 0x7f809413a800 2023-10-18T12:01:46.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.784+0000 7f807bfff640 1 -- 172.21.15.125:0/2424515849 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v251) v1 ==== 107+0+11 (secure 0 0 0) 0x7f808c004860 con 0x7f809413a800 2023-10-18T12:01:46.787 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:01:46.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f808006aff0 msgr2=0x7f808006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f808006aff0 0x7f808006d4b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8084001ce0 tx=0x7f8084001b90 comp rx=0 tx=0).stop 2023-10-18T12:01:46.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f809413a800 msgr2=0x7f80940ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f809413a800 0x7f80940ca870 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f808c009e20 tx=0x7f808c003ba0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 shutdown_connections 2023-10-18T12:01:46.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f809413a800 0x7f80940ca870 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80940ab3a0 0x7f80941381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f808006aff0 0x7f808006d4b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 --2- 172.21.15.125:0/2424515849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f807c005180 0x7f80941e2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 >> 172.21.15.125:0/2424515849 conn(0x7f8094069930 msgr2=0x7f809413b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:46.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 shutdown_connections 2023-10-18T12:01:46.791 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.786+0000 7f809a16c640 1 -- 172.21.15.125:0/2424515849 wait complete. 2023-10-18T12:01:46.800 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-18T12:01:46.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.904+0000 7f306c2f2640 1 -- 172.21.15.125:0/2327302095 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 msgr2=0x7f30640b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.904+0000 7f306c2f2640 1 --2- 172.21.15.125:0/2327302095 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 0x7f30640b0650 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f305c005fd0 tx=0x7f305c02bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:46.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.904+0000 7f306c2f2640 1 -- 172.21.15.125:0/2327302095 shutdown_connections 2023-10-18T12:01:46.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.904+0000 7f306c2f2640 1 --2- 172.21.15.125:0/2327302095 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 0x7f30640b0650 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.904+0000 7f306c2f2640 1 --2- 172.21.15.125:0/2327302095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30640cf340 0x7f30640d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.904+0000 7f306c2f2640 1 -- 172.21.15.125:0/2327302095 >> 172.21.15.125:0/2327302095 conn(0x7f30640d0d60 msgr2=0x7f30640d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:46.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.904+0000 7f306c2f2640 1 -- 172.21.15.125:0/2327302095 shutdown_connections 2023-10-18T12:01:46.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.904+0000 7f306c2f2640 1 -- 172.21.15.125:0/2327302095 wait complete. 2023-10-18T12:01:46.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.905+0000 7f306c2f2640 1 Processor -- start 2023-10-18T12:01:46.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.905+0000 7f306c2f2640 1 -- start start 2023-10-18T12:01:46.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306c2f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 0x7f30641e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306c2f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30640cf340 0x7f30641e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306c2f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306415b2a0 0x7f30640c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306c2f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30640b6510 con 0x7f30640ceb40 2023-10-18T12:01:46.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306c2f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3064151b40 con 0x7f306415b2a0 2023-10-18T12:01:46.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306c2f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30640c9170 con 0x7f30640cf340 2023-10-18T12:01:46.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 0x7f30641e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 0x7f30641e62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59738/0 (socket says 172.21.15.125:59738) 2023-10-18T12:01:46.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 -- 172.21.15.125:0/3874101091 learned_addr learned my addr 172.21.15.125:0/3874101091 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:46.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a88f640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306415b2a0 0x7f30640c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306988d640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30640cf340 0x7f30641e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 -- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30640cf340 msgr2=0x7f30641e52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30640cf340 0x7f30641e52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 -- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306415b2a0 msgr2=0x7f30640c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:46.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306415b2a0 0x7f30640c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:46.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 -- 172.21.15.125:0/3874101091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c009560 con 0x7f30640ceb40 2023-10-18T12:01:46.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.906+0000 7f306a08e640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 0x7f30641e62d0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f306000ebc0 tx=0x7f306000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:46.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.907+0000 7f305affd640 1 -- 172.21.15.125:0/3874101091 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060010630 con 0x7f30640ceb40 2023-10-18T12:01:46.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.907+0000 7f305affd640 1 -- 172.21.15.125:0/3874101091 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3060010880 con 0x7f30640ceb40 2023-10-18T12:01:46.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.907+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30641e5930 con 0x7f30640ceb40 2023-10-18T12:01:46.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.907+0000 7f305affd640 1 -- 172.21.15.125:0/3874101091 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060010c90 con 0x7f30640ceb40 2023-10-18T12:01:46.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.907+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30640c8b20 con 0x7f30640ceb40 2023-10-18T12:01:46.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.908+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30640690e0 con 0x7f30640ceb40 2023-10-18T12:01:46.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.908+0000 7f305affd640 1 -- 172.21.15.125:0/3874101091 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f3060018420 con 0x7f30640ceb40 2023-10-18T12:01:46.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.909+0000 7f305affd640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f304806b020 0x7f304806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:46.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.909+0000 7f305affd640 1 -- 172.21.15.125:0/3874101091 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f3060014070 con 0x7f30640ceb40 2023-10-18T12:01:46.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.909+0000 7f306988d640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f304806b020 0x7f304806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:46.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.909+0000 7f306988d640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f304806b020 0x7f304806d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f30641e4dc0 tx=0x7f3054009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:46.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:46.913+0000 7f305affd640 1 -- 172.21.15.125:0/3874101091 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3060051ca0 con 0x7f30640ceb40 2023-10-18T12:01:47.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.084+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f30640cb9b0 con 0x7f30640ceb40 2023-10-18T12:01:47.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.085+0000 7f305affd640 1 -- 172.21.15.125:0/3874101091 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v251) v1 ==== 103+0+11 (secure 0 0 0) 0x7f30600555d0 con 0x7f30640ceb40 2023-10-18T12:01:47.089 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:01:47.090 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f304806b020 msgr2=0x7f304806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f304806b020 0x7f304806d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f30641e4dc0 tx=0x7f3054009040 comp rx=0 tx=0).stop 2023-10-18T12:01:47.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 msgr2=0x7f30641e62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 0x7f30641e62d0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f306000ebc0 tx=0x7f306000c6a0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 shutdown_connections 2023-10-18T12:01:47.091 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f304806b020 0x7f304806d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f306415b2a0 0x7f30640c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f30640cf340 0x7f30641e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 --2- 172.21.15.125:0/3874101091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30640ceb40 0x7f30641e62d0 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.086+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 >> 172.21.15.125:0/3874101091 conn(0x7f30640d0d60 msgr2=0x7f30640adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:47.092 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.087+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 shutdown_connections 2023-10-18T12:01:47.093 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.087+0000 7f306c2f2640 1 -- 172.21.15.125:0/3874101091 wait complete. 2023-10-18T12:01:47.100 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:47.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.202+0000 7fddbf010640 1 -- 172.21.15.125:0/3736992330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdda8003970 msgr2=0x7fdda8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.202+0000 7fddbf010640 1 --2- 172.21.15.125:0/3736992330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdda8003970 0x7fdda8003d90 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7fddac006fd0 tx=0x7fdda80057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.202+0000 7fddbf010640 1 -- 172.21.15.125:0/3736992330 shutdown_connections 2023-10-18T12:01:47.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.202+0000 7fddbf010640 1 --2- 172.21.15.125:0/3736992330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdda8003970 0x7fdda8003d90 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.202+0000 7fddbf010640 1 -- 172.21.15.125:0/3736992330 >> 172.21.15.125:0/3736992330 conn(0x7fddb8069930 msgr2=0x7fddb8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:47.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.202+0000 7fddbf010640 1 -- 172.21.15.125:0/3736992330 shutdown_connections 2023-10-18T12:01:47.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.202+0000 7fddbf010640 1 -- 172.21.15.125:0/3736992330 wait complete. 2023-10-18T12:01:47.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.203+0000 7fddbf010640 1 Processor -- start 2023-10-18T12:01:47.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.203+0000 7fddbf010640 1 -- start start 2023-10-18T12:01:47.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddbf010640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdda8003970 0x7fddb8150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddbf010640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb8152980 0x7fddb81f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddbf010640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb814f5b0 0x7fddb814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddbf010640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb814ff70 con 0x7fddb8152980 2023-10-18T12:01:47.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddbf010640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb8151500 con 0x7fddb814f5b0 2023-10-18T12:01:47.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddbf010640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb81508f0 con 0x7fdda8003970 2023-10-18T12:01:47.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb8152980 0x7fddb81f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb8152980 0x7fddb81f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:59756/0 (socket says 172.21.15.125:59756) 2023-10-18T12:01:47.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddb7fff640 1 -- 172.21.15.125:0/3280488826 learned_addr learned my addr 172.21.15.125:0/3280488826 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:47.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddbcdac640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdda8003970 0x7fddb8150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.204+0000 7fddbd5ad640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb814f5b0 0x7fddb814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb7fff640 1 -- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdda8003970 msgr2=0x7fddb8150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb7fff640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdda8003970 0x7fddb8150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb7fff640 1 -- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb814f5b0 msgr2=0x7fddb814fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb7fff640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb814f5b0 0x7fddb814fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb7fff640 1 -- 172.21.15.125:0/3280488826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8150bd0 con 0x7fddb8152980 2023-10-18T12:01:47.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddbd5ad640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb814f5b0 0x7fddb814fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:47.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb7fff640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb8152980 0x7fddb81f1fe0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7fdda4004cb0 tx=0x7fdda4001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:47.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb57fa640 1 -- 172.21.15.125:0/3280488826 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4011660 con 0x7fddb8152980 2023-10-18T12:01:47.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb57fa640 1 -- 172.21.15.125:0/3280488826 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdda40118b0 con 0x7fddb8152980 2023-10-18T12:01:47.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb8150eb0 con 0x7fddb8152980 2023-10-18T12:01:47.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddb57fa640 1 -- 172.21.15.125:0/3280488826 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4011cc0 con 0x7fddb8152980 2023-10-18T12:01:47.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.205+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb8152e00 con 0x7fddb8152980 2023-10-18T12:01:47.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.206+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb80690e0 con 0x7fddb8152980 2023-10-18T12:01:47.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.206+0000 7fddb57fa640 1 -- 172.21.15.125:0/3280488826 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fdda401a400 con 0x7fddb8152980 2023-10-18T12:01:47.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.207+0000 7fddb57fa640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdd9c06aff0 0x7fdd9c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.207+0000 7fddb57fa640 1 -- 172.21.15.125:0/3280488826 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fdda408aa70 con 0x7fddb8152980 2023-10-18T12:01:47.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.207+0000 7fddbcdac640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdd9c06aff0 0x7fdd9c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.208+0000 7fddbcdac640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdd9c06aff0 0x7fdd9c06d4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fddac034000 tx=0x7fddac039040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:47.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.211+0000 7fddb57fa640 1 -- 172.21.15.125:0/3280488826 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdda40552e0 con 0x7fddb8152980 2023-10-18T12:01:47.404 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:47.401+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:47.421 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.419+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddb80cc420 con 0x7fddb8152980 2023-10-18T12:01:47.422 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.419+0000 7fddb57fa640 1 -- 172.21.15.125:0/3280488826 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4937 (secure 0 0 0) 0x7fdda4058900 con 0x7fddb8152980 2023-10-18T12:01:47.422 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:47.423 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":13389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1493462072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1493462072},{"type":"v1","addr":"172.21.15.125:6835","nonce":1493462072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:47.423 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 105 2023-10-18T12:01:47.424 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdd9c06aff0 msgr2=0x7fdd9c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdd9c06aff0 0x7fdd9c06d4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fddac034000 tx=0x7fddac039040 comp rx=0 tx=0).stop 2023-10-18T12:01:47.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb8152980 msgr2=0x7fddb81f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.425 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb8152980 0x7fddb81f1fe0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7fdda4004cb0 tx=0x7fdda4001630 comp rx=0 tx=0).stop 2023-10-18T12:01:47.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 shutdown_connections 2023-10-18T12:01:47.426 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdd9c06aff0 0x7fdd9c06d4b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fddb814f5b0 0x7fddb814fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fddb8152980 0x7fddb81f1fe0 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 --2- 172.21.15.125:0/3280488826 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdda8003970 0x7fddb8150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 >> 172.21.15.125:0/3280488826 conn(0x7fddb8069930 msgr2=0x7fddb815adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:47.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 shutdown_connections 2023-10-18T12:01:47.428 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.421+0000 7fddbf010640 1 -- 172.21.15.125:0/3280488826 wait complete. 2023-10-18T12:01:47.435 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:47.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.536+0000 7f14f5e87640 1 -- 172.21.15.125:0/1330384003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 msgr2=0x7f14f00b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.536+0000 7f14f5e87640 1 --2- 172.21.15.125:0/1330384003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 0x7f14f00b0650 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f14e8005fd0 tx=0x7f14e802c050 comp rx=0 tx=0).stop 2023-10-18T12:01:47.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.537+0000 7f14f5e87640 1 -- 172.21.15.125:0/1330384003 shutdown_connections 2023-10-18T12:01:47.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.537+0000 7f14f5e87640 1 --2- 172.21.15.125:0/1330384003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 0x7f14f00b0650 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.537+0000 7f14f5e87640 1 --2- 172.21.15.125:0/1330384003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14f015b2a0 0x7f14f00d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.537+0000 7f14f5e87640 1 -- 172.21.15.125:0/1330384003 >> 172.21.15.125:0/1330384003 conn(0x7f14f00d0d60 msgr2=0x7f14f00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:47.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.537+0000 7f14f5e87640 1 -- 172.21.15.125:0/1330384003 shutdown_connections 2023-10-18T12:01:47.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.537+0000 7f14f5e87640 1 -- 172.21.15.125:0/1330384003 wait complete. 2023-10-18T12:01:47.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.538+0000 7f14f5e87640 1 Processor -- start 2023-10-18T12:01:47.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.538+0000 7f14f5e87640 1 -- start start 2023-10-18T12:01:47.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14f5e87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 0x7f14f0154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14f5e87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14f00cf340 0x7f14f01e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14f5e87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14f015b2a0 0x7f14f01e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14f5e87640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00ca9c0 con 0x7f14f00ceb40 2023-10-18T12:01:47.545 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14f5e87640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00c9db0 con 0x7f14f015b2a0 2023-10-18T12:01:47.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14f5e87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00ca090 con 0x7f14f00cf340 2023-10-18T12:01:47.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 0x7f14f0154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 0x7f14f0154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57504/0 (socket says 172.21.15.125:57504) 2023-10-18T12:01:47.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 -- 172.21.15.125:0/4071313800 learned_addr learned my addr 172.21.15.125:0/4071313800 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:47.546 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14effff640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14f015b2a0 0x7f14f01e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14eeffd640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14f00cf340 0x7f14f01e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 -- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14f00cf340 msgr2=0x7f14f01e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14f00cf340 0x7f14f01e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 -- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14f015b2a0 msgr2=0x7f14f01e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.547 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14f015b2a0 0x7f14f01e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 -- 172.21.15.125:0/4071313800 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e8009840 con 0x7f14f00ceb40 2023-10-18T12:01:47.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14ef7fe640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 0x7f14f0154700 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c500 tx=0x7f14e000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:47.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14d7fff640 1 -- 172.21.15.125:0/4071313800 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0012830 con 0x7f14f00ceb40 2023-10-18T12:01:47.548 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14d7fff640 1 -- 172.21.15.125:0/4071313800 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f14e0012a80 con 0x7f14f00ceb40 2023-10-18T12:01:47.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f00b0d50 con 0x7f14f00ceb40 2023-10-18T12:01:47.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14d7fff640 1 -- 172.21.15.125:0/4071313800 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0012e90 con 0x7f14f00ceb40 2023-10-18T12:01:47.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.539+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f00ca650 con 0x7f14f00ceb40 2023-10-18T12:01:47.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.540+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f00690e0 con 0x7f14f00ceb40 2023-10-18T12:01:47.549 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.541+0000 7f14d7fff640 1 -- 172.21.15.125:0/4071313800 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f14e001b400 con 0x7f14f00ceb40 2023-10-18T12:01:47.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.541+0000 7f14d7fff640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f14dc06b020 0x7f14dc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.542+0000 7f14d7fff640 1 -- 172.21.15.125:0/4071313800 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f14e008a060 con 0x7f14f00ceb40 2023-10-18T12:01:47.550 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.542+0000 7f14eeffd640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f14dc06b020 0x7f14dc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.542+0000 7f14eeffd640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f14dc06b020 0x7f14dc06d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005fd0 tx=0x7f14e4005850 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:47.551 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.546+0000 7f14d7fff640 1 -- 172.21.15.125:0/4071313800 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14e0054960 con 0x7f14f00ceb40 2023-10-18T12:01:47.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.721+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14f00d0b40 con 0x7f14f00ceb40 2023-10-18T12:01:47.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.722+0000 7f14d7fff640 1 -- 172.21.15.125:0/4071313800 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v251) v1 ==== 74+0+18268 (secure 0 0 0) 0x7f14e00011a0 con 0x7f14f00ceb40 2023-10-18T12:01:47.725 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:47.725 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":251,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:45.866554+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3172661499":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1929890550":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6832/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.125:6835/1851256737":"2023-10-19T12:01:42.552858+0000","172.21.15.139:0/1706867588":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/206803400":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1890059388":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/662473594":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/2446560793":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6833/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6832/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3037918695":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3823693717":"2023-10-19T12:01:33.783814+0000","172.21.15.125:6833/1851256737":"2023-10-19T12:01:42.552858+0000","172.21.15.139:0/3119295615":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/668821395":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/3910260091":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/558106977":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2896936218":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6833/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/695463051":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2469983511":"2023-10-19T12:01:33.783814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:47.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f14dc06b020 msgr2=0x7f14dc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f14dc06b020 0x7f14dc06d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005fd0 tx=0x7f14e4005850 comp rx=0 tx=0).stop 2023-10-18T12:01:47.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 msgr2=0x7f14f0154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.727 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 0x7f14f0154700 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c500 tx=0x7f14e000c9c0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 shutdown_connections 2023-10-18T12:01:47.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f14dc06b020 0x7f14dc06d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14f015b2a0 0x7f14f01e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14f00cf340 0x7f14f01e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.728 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 --2- 172.21.15.125:0/4071313800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14f00ceb40 0x7f14f0154700 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 >> 172.21.15.125:0/4071313800 conn(0x7f14f00d0d60 msgr2=0x7f14f015a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:47.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 shutdown_connections 2023-10-18T12:01:47.729 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.724+0000 7f14f5e87640 1 -- 172.21.15.125:0/4071313800 wait complete. 2023-10-18T12:01:47.738 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-18T12:01:47.738 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:47.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 -- 172.21.15.125:0/2894663727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700ceb40 msgr2=0x7fa0700b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 --2- 172.21.15.125:0/2894663727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700ceb40 0x7fa0700b0650 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7fa07008a810 tx=0x7fa06802c1d0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 -- 172.21.15.125:0/2894663727 shutdown_connections 2023-10-18T12:01:47.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 --2- 172.21.15.125:0/2894663727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700ceb40 0x7fa0700b0650 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 --2- 172.21.15.125:0/2894663727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa07015b2a0 0x7fa0700cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 --2- 172.21.15.125:0/2894663727 >> v2:172.21.15.139:3300/0 conn(0x7fa0700cf340 0x7fa0700d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 -- 172.21.15.125:0/2894663727 >> 172.21.15.125:0/2894663727 conn(0x7fa0700d0d60 msgr2=0x7fa0700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:47.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 -- 172.21.15.125:0/2894663727 shutdown_connections 2023-10-18T12:01:47.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.838+0000 7fa07839e640 1 -- 172.21.15.125:0/2894663727 wait complete. 2023-10-18T12:01:47.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.840+0000 7fa07839e640 1 Processor -- start 2023-10-18T12:01:47.846 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.840+0000 7fa07839e640 1 -- start start 2023-10-18T12:01:47.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.840+0000 7fa07839e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0700ceb40 0x7fa0700c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.840+0000 7fa07839e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700cf340 0x7fa0701584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.840+0000 7fa07839e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa07015b2a0 0x7fa0700b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.840+0000 7fa07839e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700c9b30 con 0x7fa0700cf340 2023-10-18T12:01:47.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.840+0000 7fa07839e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700c8f20 con 0x7fa07015b2a0 2023-10-18T12:01:47.847 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.840+0000 7fa07839e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700c9200 con 0x7fa0700ceb40 2023-10-18T12:01:47.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700cf340 0x7fa0701584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa07613a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0700ceb40 0x7fa0700c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700cf340 0x7fa0701584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57514/0 (socket says 172.21.15.125:57514) 2023-10-18T12:01:47.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa07693b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa07015b2a0 0x7fa0700b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.848 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 -- 172.21.15.125:0/1663123359 learned_addr learned my addr 172.21.15.125:0/1663123359 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:47.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 -- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0700ceb40 msgr2=0x7fa0700c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0700ceb40 0x7fa0700c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 -- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa07015b2a0 msgr2=0x7fa0700b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:47.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa07015b2a0 0x7fa0700b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:47.849 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 -- 172.21.15.125:0/1663123359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068009900 con 0x7fa0700cf340 2023-10-18T12:01:47.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa075939640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700cf340 0x7fa0701584e0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7fa06000e7c0 tx=0x7fa06000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:47.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa066ffd640 1 -- 172.21.15.125:0/1663123359 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0600095a0 con 0x7fa0700cf340 2023-10-18T12:01:47.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa066ffd640 1 -- 172.21.15.125:0/1663123359 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa06000eed0 con 0x7fa0700cf340 2023-10-18T12:01:47.850 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0700c94e0 con 0x7fa0700cf340 2023-10-18T12:01:47.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa066ffd640 1 -- 172.21.15.125:0/1663123359 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060005020 con 0x7fa0700cf340 2023-10-18T12:01:47.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.841+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07006aab0 con 0x7fa0700cf340 2023-10-18T12:01:47.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.842+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0700c9ca0 con 0x7fa0700cf340 2023-10-18T12:01:47.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.843+0000 7fa066ffd640 1 -- 172.21.15.125:0/1663123359 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fa060005180 con 0x7fa0700cf340 2023-10-18T12:01:47.851 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.843+0000 7fa066ffd640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa05406aff0 0x7fa05406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:47.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.843+0000 7fa066ffd640 1 -- 172.21.15.125:0/1663123359 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fa060014070 con 0x7fa0700cf340 2023-10-18T12:01:47.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.843+0000 7fa07613a640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa05406aff0 0x7fa05406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:47.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.844+0000 7fa07613a640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa05406aff0 0x7fa05406d4b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009b50 tx=0x7fa06c005f90 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:47.852 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:47.847+0000 7fa066ffd640 1 -- 172.21.15.125:0/1663123359 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa060055110 con 0x7fa0700cf340 2023-10-18T12:01:48.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.057+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0700690e0 con 0x7fa0700cf340 2023-10-18T12:01:48.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.058+0000 7fa066ffd640 1 -- 172.21.15.125:0/1663123359 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4937 (secure 0 0 0) 0x7fa060058730 con 0x7fa0700cf340 2023-10-18T12:01:48.061 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:48.061 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":13389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1493462072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1493462072},{"type":"v1","addr":"172.21.15.125:6835","nonce":1493462072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:48.061 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 105 2023-10-18T12:01:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa05406aff0 msgr2=0x7fa05406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa05406aff0 0x7fa05406d4b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009b50 tx=0x7fa06c005f90 comp rx=0 tx=0).stop 2023-10-18T12:01:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700cf340 msgr2=0x7fa0701584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700cf340 0x7fa0701584e0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7fa06000e7c0 tx=0x7fa06000ec80 comp rx=0 tx=0).stop 2023-10-18T12:01:48.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 shutdown_connections 2023-10-18T12:01:48.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa05406aff0 0x7fa05406d4b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa07015b2a0 0x7fa0700b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0700cf340 0x7fa0701584e0 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 --2- 172.21.15.125:0/1663123359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0700ceb40 0x7fa0700c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 >> 172.21.15.125:0/1663123359 conn(0x7fa0700d0d60 msgr2=0x7fa07015a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:48.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 shutdown_connections 2023-10-18T12:01:48.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.060+0000 7fa07839e640 1 -- 172.21.15.125:0/1663123359 wait complete. 2023-10-18T12:01:48.073 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:48.090 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:48.087+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:48.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.174+0000 7f75739a5640 1 -- 172.21.15.125:0/2663104643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c13d350 msgr2=0x7f756c0cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.174+0000 7f75739a5640 1 --2- 172.21.15.125:0/2663104643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c13d350 0x7f756c0cbd60 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f755c009830 tx=0x7f755c02bd60 comp rx=0 tx=0).stop 2023-10-18T12:01:48.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.174+0000 7f75739a5640 1 -- 172.21.15.125:0/2663104643 shutdown_connections 2023-10-18T12:01:48.178 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.174+0000 7f75739a5640 1 --2- 172.21.15.125:0/2663104643 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f756c070350 0x7f756c0707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.174+0000 7f75739a5640 1 --2- 172.21.15.125:0/2663104643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c13d350 0x7f756c0cbd60 unknown :-1 s=CLOSED pgs=2955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.174+0000 7f75739a5640 1 -- 172.21.15.125:0/2663104643 >> 172.21.15.125:0/2663104643 conn(0x7f756c069930 msgr2=0x7f756c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:48.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.174+0000 7f75739a5640 1 -- 172.21.15.125:0/2663104643 shutdown_connections 2023-10-18T12:01:48.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.174+0000 7f75739a5640 1 -- 172.21.15.125:0/2663104643 wait complete. 2023-10-18T12:01:48.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.175+0000 7f75739a5640 1 Processor -- start 2023-10-18T12:01:48.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.175+0000 7f75739a5640 1 -- start start 2023-10-18T12:01:48.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f75739a5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c070350 0x7f756c1f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f75739a5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f756c0cb980 0x7f756c0b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f75739a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f756c1538b0 0x7f756c135900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f75739a5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c152030 con 0x7f756c070350 2023-10-18T12:01:48.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f75739a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c1f2260 con 0x7f756c1538b0 2023-10-18T12:01:48.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f75739a5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c1513b0 con 0x7f756c0cb980 2023-10-18T12:01:48.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c070350 0x7f756c1f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c070350 0x7f756c1f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57532/0 (socket says 172.21.15.125:57532) 2023-10-18T12:01:48.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 -- 172.21.15.125:0/3224329439 learned_addr learned my addr 172.21.15.125:0/3224329439 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:48.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7570f40640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f756c0cb980 0x7f756c0b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571f42640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f756c1538b0 0x7f756c135900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 -- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f756c0cb980 msgr2=0x7f756c0b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f756c0cb980 0x7f756c0b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 -- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f756c1538b0 msgr2=0x7f756c135900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f756c1538b0 0x7f756c135900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 -- 172.21.15.125:0/3224329439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009490 con 0x7f756c070350 2023-10-18T12:01:48.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7571741640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c070350 0x7f756c1f1d20 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f756800c400 tx=0x7f756800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:48.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7561ffb640 1 -- 172.21.15.125:0/3224329439 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568004230 con 0x7f756c070350 2023-10-18T12:01:48.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7561ffb640 1 -- 172.21.15.125:0/3224329439 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75680044a0 con 0x7f756c070350 2023-10-18T12:01:48.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c151690 con 0x7f756c070350 2023-10-18T12:01:48.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.176+0000 7f7561ffb640 1 -- 172.21.15.125:0/3224329439 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568003850 con 0x7f756c070350 2023-10-18T12:01:48.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.177+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c151c50 con 0x7f756c070350 2023-10-18T12:01:48.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.178+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756c0690e0 con 0x7f756c070350 2023-10-18T12:01:48.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.178+0000 7f7561ffb640 1 -- 172.21.15.125:0/3224329439 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f75680039b0 con 0x7f756c070350 2023-10-18T12:01:48.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.178+0000 7f7561ffb640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f755006aff0 0x7f755006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.178+0000 7f7561ffb640 1 -- 172.21.15.125:0/3224329439 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f7568089530 con 0x7f756c070350 2023-10-18T12:01:48.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.178+0000 7f7570f40640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f755006aff0 0x7f755006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.179+0000 7f7570f40640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f755006aff0 0x7f755006d4b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f755c036000 tx=0x7f755c004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:48.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.182+0000 7f7561ffb640 1 -- 172.21.15.125:0/3224329439 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7568053f80 con 0x7f756c070350 2023-10-18T12:01:48.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.386+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f756c06ff80 con 0x7f756c070350 2023-10-18T12:01:48.389 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.387+0000 7f7561ffb640 1 -- 172.21.15.125:0/3224329439 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f7568017090 con 0x7f756c070350 2023-10-18T12:01:48.390 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:48.390 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":13389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1493462072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1493462072},{"type":"v1","addr":"172.21.15.125:6835","nonce":1493462072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:19.784343+0000","modified":"2023-10-18T12:01:21.785616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12768},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12768":{"gid":12768,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":6,"addr":"172.21.15.125:6834/3879855806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":3879855806},{"type":"v1","addr":"172.21.15.125:6834","nonce":3879855806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-18T12:01:48.392 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 105 2023-10-18T12:01:48.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f755006aff0 msgr2=0x7f755006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f755006aff0 0x7f755006d4b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f755c036000 tx=0x7f755c004730 comp rx=0 tx=0).stop 2023-10-18T12:01:48.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c070350 msgr2=0x7f756c1f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c070350 0x7f756c1f1d20 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f756800c400 tx=0x7f756800c8c0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 shutdown_connections 2023-10-18T12:01:48.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f755006aff0 0x7f755006d4b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f756c1538b0 0x7f756c135900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f756c0cb980 0x7f756c0b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 --2- 172.21.15.125:0/3224329439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f756c070350 0x7f756c1f1d20 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 >> 172.21.15.125:0/3224329439 conn(0x7f756c069930 msgr2=0x7f756c157600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:48.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 shutdown_connections 2023-10-18T12:01:48.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.389+0000 7f75739a5640 1 -- 172.21.15.125:0/3224329439 wait complete. 2023-10-18T12:01:48.402 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:48.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.503+0000 7f03b04b4640 1 -- 172.21.15.125:0/3644759179 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 msgr2=0x7f03a80affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.503+0000 7f03b04b4640 1 --2- 172.21.15.125:0/3644759179 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 0x7f03a80affe0 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005fd0 tx=0x7f03a002c240 comp rx=0 tx=0).stop 2023-10-18T12:01:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.503+0000 7f03b04b4640 1 -- 172.21.15.125:0/3644759179 shutdown_connections 2023-10-18T12:01:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.503+0000 7f03b04b4640 1 --2- 172.21.15.125:0/3644759179 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 0x7f03a80affe0 unknown :-1 s=CLOSED pgs=2957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.503+0000 7f03b04b4640 1 --2- 172.21.15.125:0/3644759179 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03a813e520 0x7f03a806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.503+0000 7f03b04b4640 1 -- 172.21.15.125:0/3644759179 >> 172.21.15.125:0/3644759179 conn(0x7f03a8069930 msgr2=0x7f03a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:48.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.503+0000 7f03b04b4640 1 -- 172.21.15.125:0/3644759179 shutdown_connections 2023-10-18T12:01:48.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.503+0000 7f03b04b4640 1 -- 172.21.15.125:0/3644759179 wait complete. 2023-10-18T12:01:48.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.504+0000 7f03b04b4640 1 Processor -- start 2023-10-18T12:01:48.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.504+0000 7f03b04b4640 1 -- start start 2023-10-18T12:01:48.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03b04b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 0x7f03a81ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03b04b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03a813e520 0x7f03a80ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03b04b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a8141690 0x7f03a80acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03b04b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81eee90 con 0x7f03a8070af0 2023-10-18T12:01:48.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03b04b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81cd440 con 0x7f03a8141690 2023-10-18T12:01:48.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03b04b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81cd720 con 0x7f03a813e520 2023-10-18T12:01:48.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 0x7f03a81ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 0x7f03a81ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57544/0 (socket says 172.21.15.125:57544) 2023-10-18T12:01:48.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 -- 172.21.15.125:0/1464400525 learned_addr learned my addr 172.21.15.125:0/1464400525 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:48.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03aea51640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a8141690 0x7f03a80acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ada4f640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03a813e520 0x7f03a80ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 -- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03a813e520 msgr2=0x7f03a80ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03a813e520 0x7f03a80ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 -- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a8141690 msgr2=0x7f03a80acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a8141690 0x7f03a80acce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 -- 172.21.15.125:0/1464400525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0009900 con 0x7f03a8070af0 2023-10-18T12:01:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03ae250640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 0x7f03a81ce4e0 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7f03a400e9f0 tx=0x7f03a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f039effd640 1 -- 172.21.15.125:0/1464400525 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4009700 con 0x7f03a8070af0 2023-10-18T12:01:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f039effd640 1 -- 172.21.15.125:0/1464400525 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03a400ce70 con 0x7f03a8070af0 2023-10-18T12:01:48.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a81cda00 con 0x7f03a8070af0 2023-10-18T12:01:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f039effd640 1 -- 172.21.15.125:0/1464400525 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4010670 con 0x7f03a8070af0 2023-10-18T12:01:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.505+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a81cdfc0 con 0x7f03a8070af0 2023-10-18T12:01:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.506+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a81ce150 con 0x7f03a8070af0 2023-10-18T12:01:48.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.507+0000 7f039effd640 1 -- 172.21.15.125:0/1464400525 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f03a40107d0 con 0x7f03a8070af0 2023-10-18T12:01:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.507+0000 7f039effd640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f038c06aff0 0x7f038c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.507+0000 7f039effd640 1 -- 172.21.15.125:0/1464400525 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f03a4014070 con 0x7f03a8070af0 2023-10-18T12:01:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.507+0000 7f03ada4f640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f038c06aff0 0x7f038c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.508+0000 7f03ada4f640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f038c06aff0 0x7f038c06d4b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03a8140b50 tx=0x7f0398009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:48.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.511+0000 7f039effd640 1 -- 172.21.15.125:0/1464400525 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03a4051ca0 con 0x7f03a8070af0 2023-10-18T12:01:48.685 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.681+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03a813f360 con 0x7f03a8070af0 2023-10-18T12:01:48.686 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.683+0000 7f039effd640 1 -- 172.21.15.125:0/1464400525 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v251) v1 ==== 74+0+18268 (secure 0 0 0) 0x7f03a4088e60 con 0x7f03a8070af0 2023-10-18T12:01:48.687 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:48.687 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":251,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:45.866554+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-18T12:01:18.203802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"245","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-18T12:01:18.758795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/3172661499":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1929890550":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6832/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.125:6835/1851256737":"2023-10-19T12:01:42.552858+0000","172.21.15.139:0/1706867588":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/206803400":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1890059388":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/662473594":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/2446560793":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6833/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6832/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3037918695":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3823693717":"2023-10-19T12:01:33.783814+0000","172.21.15.125:6833/1851256737":"2023-10-19T12:01:42.552858+0000","172.21.15.139:0/3119295615":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/668821395":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/3910260091":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/558106977":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2896936218":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6833/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/695463051":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2469983511":"2023-10-19T12:01:33.783814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:48.688 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f038c06aff0 msgr2=0x7f038c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f038c06aff0 0x7f038c06d4b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03a8140b50 tx=0x7f0398009040 comp rx=0 tx=0).stop 2023-10-18T12:01:48.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 msgr2=0x7f03a81ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 0x7f03a81ce4e0 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7f03a400e9f0 tx=0x7f03a400eeb0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 shutdown_connections 2023-10-18T12:01:48.689 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f038c06aff0 0x7f038c06d4b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03a8141690 0x7f03a80acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03a813e520 0x7f03a80ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 --2- 172.21.15.125:0/1464400525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f03a8070af0 0x7f03a81ce4e0 unknown :-1 s=CLOSED pgs=2958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 >> 172.21.15.125:0/1464400525 conn(0x7f03a8069930 msgr2=0x7f03a81e0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:48.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 shutdown_connections 2023-10-18T12:01:48.691 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.685+0000 7f03b04b4640 1 -- 172.21.15.125:0/1464400525 wait complete. 2023-10-18T12:01:48.699 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-18T12:01:48.782 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:48.779+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:48.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.800+0000 7fa30b5a4640 1 -- 172.21.15.125:0/2814315109 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3040cf340 msgr2=0x7fa3040d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.800+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/2814315109 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3040cf340 0x7fa3040d1910 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7fa30415b680 tx=0x7fa2fc02c260 comp rx=0 tx=0).stop 2023-10-18T12:01:48.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.800+0000 7fa30b5a4640 1 -- 172.21.15.125:0/2814315109 shutdown_connections 2023-10-18T12:01:48.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.800+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/2814315109 >> v2:172.21.15.139:3300/0 conn(0x7fa3040ceb40 0x7fa3040b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.800+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/2814315109 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3040cf340 0x7fa3040d1910 unknown :-1 s=CLOSED pgs=2959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.800+0000 7fa30b5a4640 1 -- 172.21.15.125:0/2814315109 >> 172.21.15.125:0/2814315109 conn(0x7fa3040d0d60 msgr2=0x7fa3040d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:48.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.800+0000 7fa30b5a4640 1 -- 172.21.15.125:0/2814315109 shutdown_connections 2023-10-18T12:01:48.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.800+0000 7fa30b5a4640 1 -- 172.21.15.125:0/2814315109 wait complete. 2023-10-18T12:01:48.808 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.801+0000 7fa30b5a4640 1 Processor -- start 2023-10-18T12:01:48.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.801+0000 7fa30b5a4640 1 -- start start 2023-10-18T12:01:48.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa30b5a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3040ceb40 0x7fa3041f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa30b5a4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3040cf340 0x7fa30406b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa30b5a4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa30415b2a0 0x7fa3040b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.809 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa30b5a4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3041f5ba0 con 0x7fa30415b2a0 2023-10-18T12:01:48.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa30b5a4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3041f4c10 con 0x7fa3040ceb40 2023-10-18T12:01:48.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa30b5a4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040b62d0 con 0x7fa3040cf340 2023-10-18T12:01:48.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309b41640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa30415b2a0 0x7fa3040b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.810 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309b41640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa30415b2a0 0x7fa3040b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57564/0 (socket says 172.21.15.125:57564) 2023-10-18T12:01:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309b41640 1 -- 172.21.15.125:0/3065488012 learned_addr learned my addr 172.21.15.125:0/3065488012 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309340640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3040ceb40 0x7fa3041f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa308b3f640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3040cf340 0x7fa30406b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309b41640 1 -- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3040cf340 msgr2=0x7fa30406b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.811 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309b41640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3040cf340 0x7fa30406b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309b41640 1 -- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3040ceb40 msgr2=0x7fa3041f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:48.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309b41640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3040ceb40 0x7fa3041f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:48.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.802+0000 7fa309b41640 1 -- 172.21.15.125:0/3065488012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc009900 con 0x7fa30415b2a0 2023-10-18T12:01:48.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.803+0000 7fa309340640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3040ceb40 0x7fa3041f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:01:48.812 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.803+0000 7fa309b41640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa30415b2a0 0x7fa3040b5880 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7fa30000e970 tx=0x7fa30000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:48.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.803+0000 7fa2f9ffb640 1 -- 172.21.15.125:0/3065488012 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300009720 con 0x7fa30415b2a0 2023-10-18T12:01:48.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.803+0000 7fa2f9ffb640 1 -- 172.21.15.125:0/3065488012 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa30000ccc0 con 0x7fa30415b2a0 2023-10-18T12:01:48.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.803+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3040b65b0 con 0x7fa30415b2a0 2023-10-18T12:01:48.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.803+0000 7fa2f9ffb640 1 -- 172.21.15.125:0/3065488012 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300005280 con 0x7fa30415b2a0 2023-10-18T12:01:48.813 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.803+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3041f5140 con 0x7fa30415b2a0 2023-10-18T12:01:48.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.804+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3040b67f0 con 0x7fa30415b2a0 2023-10-18T12:01:48.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.804+0000 7fa2f9ffb640 1 -- 172.21.15.125:0/3065488012 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fa3000053e0 con 0x7fa30415b2a0 2023-10-18T12:01:48.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.805+0000 7fa2f9ffb640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa2e806aff0 0x7fa2e806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:48.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.805+0000 7fa2f9ffb640 1 -- 172.21.15.125:0/3065488012 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fa300014070 con 0x7fa30415b2a0 2023-10-18T12:01:48.814 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.805+0000 7fa309340640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa2e806aff0 0x7fa2e806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:48.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.805+0000 7fa309340640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa2e806aff0 0x7fa2e806d4b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc036000 tx=0x7fa2fc004ab0 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:48.815 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:48.809+0000 7fa2f9ffb640 1 -- 172.21.15.125:0/3065488012 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa300055210 con 0x7fa30415b2a0 2023-10-18T12:01:49.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.023+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7fa3040690e0 con 0x7fa30415b2a0 2023-10-18T12:01:49.027 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T12:01:49.024+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:01:49.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.025+0000 7fa2f9ffb640 1 -- 172.21.15.125:0/3065488012 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 167+0+0 (secure 0 0 0) 0x7fa300058830 con 0x7fa30415b2a0 2023-10-18T12:01:49.028 INFO:teuthology.orchestra.run.smithi125.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T12:01:49.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.027+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa2e806aff0 msgr2=0x7fa2e806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:49.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.027+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa2e806aff0 0x7fa2e806d4b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc036000 tx=0x7fa2fc004ab0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.027+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa30415b2a0 msgr2=0x7fa3040b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:49.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.027+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa30415b2a0 0x7fa3040b5880 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7fa30000e970 tx=0x7fa30000ed70 comp rx=0 tx=0).stop 2023-10-18T12:01:49.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.028+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 shutdown_connections 2023-10-18T12:01:49.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.028+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa30415b2a0 0x7fa3040b5880 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.028+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa3040cf340 0x7fa30406b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.028+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa2e806aff0 0x7fa2e806d4b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.028+0000 7fa30b5a4640 1 --2- 172.21.15.125:0/3065488012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3040ceb40 0x7fa3041f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.028+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 >> 172.21.15.125:0/3065488012 conn(0x7fa3040d0d60 msgr2=0x7fa30415d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:49.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.028+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 shutdown_connections 2023-10-18T12:01:49.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.028+0000 7fa30b5a4640 1 -- 172.21.15.125:0/3065488012 wait complete. 2023-10-18T12:01:49.034 INFO:tasks.ceph.mds.c.smithi125.stderr: -1514> 2023-10-18T12:01:25.086+0000 7ff006230640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:01:49.043 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-18T12:01:49.057 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T12:01:49.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.148+0000 7ff4bb26f640 1 -- 172.21.15.125:0/4035767200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b4140660 msgr2=0x7ff4b40c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:49.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.148+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/4035767200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b4140660 0x7ff4b40c97c0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4005fd0 tx=0x7ff4a402be10 comp rx=0 tx=0).stop 2023-10-18T12:01:49.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.148+0000 7ff4bb26f640 1 -- 172.21.15.125:0/4035767200 shutdown_connections 2023-10-18T12:01:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.148+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/4035767200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b4140660 0x7ff4b40c97c0 unknown :-1 s=CLOSED pgs=2963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.148+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/4035767200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4b413d360 0x7ff4b415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.148+0000 7ff4bb26f640 1 -- 172.21.15.125:0/4035767200 >> 172.21.15.125:0/4035767200 conn(0x7ff4b4069930 msgr2=0x7ff4b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.148+0000 7ff4bb26f640 1 -- 172.21.15.125:0/4035767200 shutdown_connections 2023-10-18T12:01:49.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.148+0000 7ff4bb26f640 1 -- 172.21.15.125:0/4035767200 wait complete. 2023-10-18T12:01:49.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.149+0000 7ff4bb26f640 1 Processor -- start 2023-10-18T12:01:49.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.149+0000 7ff4bb26f640 1 -- start start 2023-10-18T12:01:49.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4bb26f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b406d170 0x7ff4b4136af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:49.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4bb26f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b413d360 0x7ff4b4132ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:49.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4bb26f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4b4140660 0x7ff4b4155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:49.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4bb26f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b414c9f0 con 0x7ff4b413d360 2023-10-18T12:01:49.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4bb26f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b414d570 con 0x7ff4b406d170 2023-10-18T12:01:49.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4bb26f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b414cc40 con 0x7ff4b4140660 2023-10-18T12:01:49.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b413d360 0x7ff4b4132ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:49.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b413d360 0x7ff4b4132ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57592/0 (socket says 172.21.15.125:57592) 2023-10-18T12:01:49.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 -- 172.21.15.125:0/415175460 learned_addr learned my addr 172.21.15.125:0/415175460 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:49.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b980c640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4b4140660 0x7ff4b4155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:49.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b900b640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b406d170 0x7ff4b4136af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:49.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 -- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4b4140660 msgr2=0x7ff4b4155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:49.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4b4140660 0x7ff4b4155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 -- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b406d170 msgr2=0x7ff4b4136af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:49.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b406d170 0x7ff4b4136af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:49.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 -- 172.21.15.125:0/415175460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009560 con 0x7ff4b413d360 2023-10-18T12:01:49.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b980c640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4b4140660 0x7ff4b4155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:49.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b900b640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b406d170 0x7ff4b4136af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:01:49.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.150+0000 7ff4b880a640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b413d360 0x7ff4b4132ce0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40099f0 tx=0x7ff4a402da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:49.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.151+0000 7ff4a9ffb640 1 -- 172.21.15.125:0/415175460 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4003800 con 0x7ff4b413d360 2023-10-18T12:01:49.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.151+0000 7ff4a9ffb640 1 -- 172.21.15.125:0/415175460 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4a4003a50 con 0x7ff4b413d360 2023-10-18T12:01:49.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.151+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b414cf20 con 0x7ff4b413d360 2023-10-18T12:01:49.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.151+0000 7ff4a9ffb640 1 -- 172.21.15.125:0/415175460 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4003e60 con 0x7ff4b413d360 2023-10-18T12:01:49.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.151+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b4133220 con 0x7ff4b413d360 2023-10-18T12:01:49.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.152+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b415ad50 con 0x7ff4b413d360 2023-10-18T12:01:49.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.152+0000 7ff4a9ffb640 1 -- 172.21.15.125:0/415175460 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7ff4a403e450 con 0x7ff4b413d360 2023-10-18T12:01:49.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.153+0000 7ff4a9ffb640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff49806aff0 0x7ff49806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:49.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.153+0000 7ff4a9ffb640 1 -- 172.21.15.125:0/415175460 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 7129+0+0 (secure 0 0 0) 0x7ff4a40aca10 con 0x7ff4b413d360 2023-10-18T12:01:49.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.153+0000 7ff4b900b640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff49806aff0 0x7ff49806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:49.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.153+0000 7ff4b900b640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff49806aff0 0x7ff49806d4b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40b1720 tx=0x7ff4b0009500 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:49.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.157+0000 7ff4a9ffb640 1 -- 172.21.15.125:0/415175460 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4a40773d0 con 0x7ff4b413d360 2023-10-18T12:01:49.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:49.368+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff4b40c9c80 con 0x7ff4b413d360 2023-10-18T12:01:49.474 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:49.471+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:50.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.032+0000 7ff4a9ffb640 1 -- 172.21.15.125:0/415175460 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v107) v1 ==== 107+0+0 (secure 0 0 0) 0x7ff4a407a9f0 con 0x7ff4b413d360 2023-10-18T12:01:50.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff49806aff0 msgr2=0x7ff49806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:50.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff49806aff0 0x7ff49806d4b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40b1720 tx=0x7ff4b0009500 comp rx=0 tx=0).stop 2023-10-18T12:01:50.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b413d360 msgr2=0x7ff4b4132ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b413d360 0x7ff4b4132ce0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40099f0 tx=0x7ff4a402da70 comp rx=0 tx=0).stop 2023-10-18T12:01:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 shutdown_connections 2023-10-18T12:01:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4b4140660 0x7ff4b4155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4b413d360 0x7ff4b4132ce0 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:50.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff49806aff0 0x7ff49806d4b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:50.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 --2- 172.21.15.125:0/415175460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4b406d170 0x7ff4b4136af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:50.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 >> 172.21.15.125:0/415175460 conn(0x7ff4b4069930 msgr2=0x7ff4b41523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:50.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 shutdown_connections 2023-10-18T12:01:50.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.034+0000 7ff4bb26f640 1 -- 172.21.15.125:0/415175460 wait complete. 2023-10-18T12:01:50.048 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-18T12:01:50.178 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:50.175+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:50.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.193+0000 7f27da525640 1 -- 172.21.15.125:0/2679947503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 msgr2=0x7f27bc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:50.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.193+0000 7f27da525640 1 --2- 172.21.15.125:0/2679947503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 0x7f27bc003d90 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009b50 tx=0x7f27bc0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:50.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.193+0000 7f27d8ac2640 1 -- 172.21.15.125:0/2679947503 reap_dead start 2023-10-18T12:01:50.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.194+0000 7f27da525640 1 -- 172.21.15.125:0/2679947503 shutdown_connections 2023-10-18T12:01:50.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.194+0000 7f27da525640 1 -- 172.21.15.125:0/2679947503 >> 172.21.15.125:0/2679947503 conn(0x7f27d40d0d60 msgr2=0x7f27d40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:50.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.194+0000 7f27da525640 1 -- 172.21.15.125:0/2679947503 shutdown_connections 2023-10-18T12:01:50.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.194+0000 7f27da525640 1 -- 172.21.15.125:0/2679947503 wait complete. 2023-10-18T12:01:50.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.195+0000 7f27da525640 1 Processor -- start 2023-10-18T12:01:50.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.195+0000 7f27da525640 1 -- start start 2023-10-18T12:01:50.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27da525640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 0x7f27d415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:50.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27da525640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27d415c860 0x7f27d415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:50.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27da525640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d41f0e40 0x7f27d41f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:50.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27da525640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d4155a80 con 0x7f27bc003970 2023-10-18T12:01:50.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27da525640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d415d200 con 0x7f27d41f0e40 2023-10-18T12:01:50.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27da525640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d41f1ba0 con 0x7f27d415c860 2023-10-18T12:01:50.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 0x7f27d415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:50.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 0x7f27d415c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57620/0 (socket says 172.21.15.125:57620) 2023-10-18T12:01:50.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 -- 172.21.15.125:0/340228494 learned_addr learned my addr 172.21.15.125:0/340228494 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:50.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d8ac2640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d41f0e40 0x7f27d41f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:50.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d37fe640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27d415c860 0x7f27d415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:50.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 -- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27d415c860 msgr2=0x7f27d415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:50.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27d415c860 0x7f27d415ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:50.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 -- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d41f0e40 msgr2=0x7f27d41f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:50.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d41f0e40 0x7f27d41f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:50.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 -- 172.21.15.125:0/340228494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d41f1e80 con 0x7f27bc003970 2023-10-18T12:01:50.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d3fff640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 0x7f27d415c1e0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009b50 tx=0x7f27c4033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:50.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d0ff9640 1 -- 172.21.15.125:0/340228494 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4033ea0 con 0x7f27bc003970 2023-10-18T12:01:50.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d0ff9640 1 -- 172.21.15.125:0/340228494 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f27c4004460 con 0x7f27bc003970 2023-10-18T12:01:50.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d41f2160 con 0x7f27bc003970 2023-10-18T12:01:50.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.196+0000 7f27d0ff9640 1 -- 172.21.15.125:0/340228494 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4003850 con 0x7f27bc003970 2023-10-18T12:01:50.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.197+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d41f2850 con 0x7f27bc003970 2023-10-18T12:01:50.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.197+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d41f29e0 con 0x7f27bc003970 2023-10-18T12:01:50.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.198+0000 7f27d0ff9640 1 -- 172.21.15.125:0/340228494 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f27c40039b0 con 0x7f27bc003970 2023-10-18T12:01:50.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.198+0000 7f27d0ff9640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f27c006aff0 0x7f27c006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:50.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.199+0000 7f27d0ff9640 1 -- 172.21.15.125:0/340228494 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f27c40acae0 con 0x7f27bc003970 2023-10-18T12:01:50.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.199+0000 7f27d37fe640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f27c006aff0 0x7f27c006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:50.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.199+0000 7f27d37fe640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f27c006aff0 0x7f27c006d4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f27c8000c00 tx=0x7f27c8006040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:50.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.203+0000 7f27d0ff9640 1 -- 172.21.15.125:0/340228494 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27c40773d0 con 0x7f27bc003970 2023-10-18T12:01:50.383 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:50.380+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f27d40690e0 con 0x7f27bc003970 2023-10-18T12:01:50.888 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:50.886+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:51.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.033+0000 7f27d0ff9640 1 -- 172.21.15.125:0/340228494 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v253) v1 ==== 190+0+0 (secure 0 0 0) 0x7f27c40410e0 con 0x7f27bc003970 2023-10-18T12:01:51.036 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'backup_fs_metadata' removed 2023-10-18T12:01:51.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f27c006aff0 msgr2=0x7f27c006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:51.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f27c006aff0 0x7f27c006d4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f27c8000c00 tx=0x7f27c8006040 comp rx=0 tx=0).stop 2023-10-18T12:01:51.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 msgr2=0x7f27d415c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:51.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 0x7f27d415c1e0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009b50 tx=0x7f27c4033c70 comp rx=0 tx=0).stop 2023-10-18T12:01:51.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 shutdown_connections 2023-10-18T12:01:51.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f27c006aff0 0x7f27c006d4b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:51.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27d41f0e40 0x7f27d41f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:51.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27d415c860 0x7f27d415ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:51.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 --2- 172.21.15.125:0/340228494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f27bc003970 0x7f27d415c1e0 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:51.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 >> 172.21.15.125:0/340228494 conn(0x7f27d40d0d60 msgr2=0x7f27d406b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:51.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 shutdown_connections 2023-10-18T12:01:51.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.035+0000 7f27da525640 1 -- 172.21.15.125:0/340228494 wait complete. 2023-10-18T12:01:51.050 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-18T12:01:51.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.160+0000 7fb943bce640 1 -- 172.21.15.125:0/3181768547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c156790 msgr2=0x7fb93c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:51.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.160+0000 7fb943bce640 1 --2- 172.21.15.125:0/3181768547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c156790 0x7fb93c0cc990 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7fb93c065ed0 tx=0x7fb92c02be20 comp rx=0 tx=0).stop 2023-10-18T12:01:51.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.160+0000 7fb943bce640 1 -- 172.21.15.125:0/3181768547 shutdown_connections 2023-10-18T12:01:51.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.160+0000 7fb943bce640 1 --2- 172.21.15.125:0/3181768547 >> v2:172.21.15.139:3300/0 conn(0x7fb93c06a730 0x7fb93c06abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:51.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.160+0000 7fb943bce640 1 --2- 172.21.15.125:0/3181768547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c156790 0x7fb93c0cc990 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:51.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.160+0000 7fb943bce640 1 -- 172.21.15.125:0/3181768547 >> 172.21.15.125:0/3181768547 conn(0x7fb93c06fae0 msgr2=0x7fb93c06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:51.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.160+0000 7fb943bce640 1 -- 172.21.15.125:0/3181768547 shutdown_connections 2023-10-18T12:01:51.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.160+0000 7fb943bce640 1 -- 172.21.15.125:0/3181768547 wait complete. 2023-10-18T12:01:51.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.161+0000 7fb943bce640 1 Processor -- start 2023-10-18T12:01:51.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.161+0000 7fb943bce640 1 -- start start 2023-10-18T12:01:51.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb943bce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c06a730 0x7fb93c145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:51.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb943bce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c0ac870 0x7fb93c15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:51.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb943bce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c156790 0x7fb93c15c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:51.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb943bce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c15bc60 con 0x7fb93c0ac870 2023-10-18T12:01:51.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb943bce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0a6b00 con 0x7fb93c156790 2023-10-18T12:01:51.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb943bce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c15ac60 con 0x7fb93c06a730 2023-10-18T12:01:51.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c0ac870 0x7fb93c15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:51.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c0ac870 0x7fb93c15a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57648/0 (socket says 172.21.15.125:57648) 2023-10-18T12:01:51.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 -- 172.21.15.125:0/1802043208 learned_addr learned my addr 172.21.15.125:0/1802043208 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:51.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb94196a640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c06a730 0x7fb93c145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:51.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb94216b640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c156790 0x7fb93c15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:51.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 -- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c06a730 msgr2=0x7fb93c145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:51.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c06a730 0x7fb93c145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:51.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 -- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c156790 msgr2=0x7fb93c15c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:51.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c156790 0x7fb93c15c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:51.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 -- 172.21.15.125:0/1802043208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c009560 con 0x7fb93c0ac870 2023-10-18T12:01:51.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.162+0000 7fb941169640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c0ac870 0x7fb93c15a720 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c450 tx=0x7fb93800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:51.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.163+0000 7fb9327fc640 1 -- 172.21.15.125:0/1802043208 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9380052d0 con 0x7fb93c0ac870 2023-10-18T12:01:51.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.163+0000 7fb9327fc640 1 -- 172.21.15.125:0/1802043208 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb938005430 con 0x7fb93c0ac870 2023-10-18T12:01:51.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.163+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c0a6820 con 0x7fb93c0ac870 2023-10-18T12:01:51.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.163+0000 7fb9327fc640 1 -- 172.21.15.125:0/1802043208 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938012890 con 0x7fb93c0ac870 2023-10-18T12:01:51.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.163+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c15c1e0 con 0x7fb93c0ac870 2023-10-18T12:01:51.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.163+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93c15bac0 con 0x7fb93c0ac870 2023-10-18T12:01:51.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.164+0000 7fb9327fc640 1 -- 172.21.15.125:0/1802043208 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb9380035f0 con 0x7fb93c0ac870 2023-10-18T12:01:51.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.164+0000 7fb9327fc640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb92006b020 0x7fb92006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:51.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.164+0000 7fb9327fc640 1 -- 172.21.15.125:0/1802043208 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fb938089820 con 0x7fb93c0ac870 2023-10-18T12:01:51.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.165+0000 7fb94196a640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb92006b020 0x7fb92006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:51.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.165+0000 7fb94196a640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb92006b020 0x7fb92006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb92c035000 tx=0x7fb92c004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:51.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.169+0000 7fb9327fc640 1 -- 172.21.15.125:0/1802043208 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb93c15bac0 con 0x7fb93c0ac870 2023-10-18T12:01:51.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:51.341+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb93c15bac0 con 0x7fb93c0ac870 2023-10-18T12:01:51.602 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:51.599+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:52.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.037+0000 7fb9327fc640 1 -- 172.21.15.125:0/1802043208 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v254) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb93c15bac0 con 0x7fb93c0ac870 2023-10-18T12:01:52.040 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'backup_fs_data' removed 2023-10-18T12:01:52.041 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb92006b020 msgr2=0x7fb92006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb92006b020 0x7fb92006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb92c035000 tx=0x7fb92c004730 comp rx=0 tx=0).stop 2023-10-18T12:01:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c0ac870 msgr2=0x7fb93c15a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c0ac870 0x7fb93c15a720 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c450 tx=0x7fb93800c910 comp rx=0 tx=0).stop 2023-10-18T12:01:52.042 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 shutdown_connections 2023-10-18T12:01:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb92006b020 0x7fb92006d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb93c156790 0x7fb93c15c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb93c0ac870 0x7fb93c15a720 unknown :-1 s=CLOSED pgs=2970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 --2- 172.21.15.125:0/1802043208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb93c06a730 0x7fb93c145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 >> 172.21.15.125:0/1802043208 conn(0x7fb93c06fae0 msgr2=0x7fb93c1e4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:52.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 shutdown_connections 2023-10-18T12:01:52.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.039+0000 7fb943bce640 1 -- 172.21.15.125:0/1802043208 wait complete. 2023-10-18T12:01:52.054 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T12:01:52.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.159+0000 7fdccc0b4640 1 -- 172.21.15.125:0/2267595875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40ab500 msgr2=0x7fdcc40ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.159+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/2267595875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40ab500 0x7fdcc40ab990 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40ca260 tx=0x7fdcbc02c2f0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.159+0000 7fdccc0b4640 1 -- 172.21.15.125:0/2267595875 shutdown_connections 2023-10-18T12:01:52.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.159+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/2267595875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40ab500 0x7fdcc40ab990 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.159+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/2267595875 >> v2:172.21.15.139:3300/0 conn(0x7fdcc40c9dc0 0x7fdcc406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.159+0000 7fdccc0b4640 1 -- 172.21.15.125:0/2267595875 >> 172.21.15.125:0/2267595875 conn(0x7fdcc4069930 msgr2=0x7fdcc4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:52.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.159+0000 7fdccc0b4640 1 -- 172.21.15.125:0/2267595875 shutdown_connections 2023-10-18T12:01:52.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.159+0000 7fdccc0b4640 1 -- 172.21.15.125:0/2267595875 wait complete. 2023-10-18T12:01:52.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.160+0000 7fdccc0b4640 1 Processor -- start 2023-10-18T12:01:52.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.160+0000 7fdccc0b4640 1 -- start start 2023-10-18T12:01:52.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdccc0b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc406af00 0x7fdcc4138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:52.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdccc0b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc40ab500 0x7fdcc40b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:52.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdccc0b4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40c9dc0 0x7fdcc413a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:52.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdccc0b4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40afed0 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdccc0b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40b01b0 con 0x7fdcc406af00 2023-10-18T12:01:52.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdccc0b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40b0490 con 0x7fdcc40ab500 2023-10-18T12:01:52.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40c9dc0 0x7fdcc413a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:52.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40c9dc0 0x7fdcc413a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57664/0 (socket says 172.21.15.125:57664) 2023-10-18T12:01:52.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 -- 172.21.15.125:0/389661071 learned_addr learned my addr 172.21.15.125:0/389661071 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:52.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcc964f640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc40ab500 0x7fdcc40b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:52.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 -- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc40ab500 msgr2=0x7fdcc40b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcc9e50640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc406af00 0x7fdcc4138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:52.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc40ab500 0x7fdcc40b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 -- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc406af00 msgr2=0x7fdcc4138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc406af00 0x7fdcc4138e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 -- 172.21.15.125:0/389661071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc009900 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcca651640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40c9dc0 0x7fdcc413a190 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc037040 tx=0x7fdcbc035e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:52.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcba7fc640 1 -- 172.21.15.125:0/389661071 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc02cea0 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcba7fc640 1 -- 172.21.15.125:0/389661071 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdcbc004250 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc40b0770 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.161+0000 7fdcba7fc640 1 -- 172.21.15.125:0/389661071 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc004640 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.162+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc4139680 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.162+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc406c500 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.163+0000 7fdcba7fc640 1 -- 172.21.15.125:0/389661071 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fdcbc03b040 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.163+0000 7fdcba7fc640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdca806aff0 0x7fdca806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:52.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.163+0000 7fdcba7fc640 1 -- 172.21.15.125:0/389661071 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fdcbc0ac830 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.163+0000 7fdcc9e50640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdca806aff0 0x7fdca806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:52.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.164+0000 7fdcc9e50640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdca806aff0 0x7fdca806d4b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0000f60 tx=0x7fdcc0009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:52.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.167+0000 7fdcba7fc640 1 -- 172.21.15.125:0/389661071 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcbc0774c0 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.313 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:52.311+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:52.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.344+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7fdcc413baf0 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.347+0000 7fdcba7fc640 1 -- 172.21.15.125:0/389661071 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v82) v1 ==== 71+0+0 (secure 0 0 0) 0x7fdcbc07aae0 con 0x7fdcc40c9dc0 2023-10-18T12:01:52.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdca806aff0 msgr2=0x7fdca806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdca806aff0 0x7fdca806d4b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0000f60 tx=0x7fdcc0009040 comp rx=0 tx=0).stop 2023-10-18T12:01:52.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40c9dc0 msgr2=0x7fdcc413a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40c9dc0 0x7fdcc413a190 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc037040 tx=0x7fdcbc035e50 comp rx=0 tx=0).stop 2023-10-18T12:01:52.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 shutdown_connections 2023-10-18T12:01:52.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdcc40c9dc0 0x7fdcc413a190 unknown :-1 s=CLOSED pgs=2973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdcc40ab500 0x7fdcc40b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdca806aff0 0x7fdca806d4b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 --2- 172.21.15.125:0/389661071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdcc406af00 0x7fdcc4138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 >> 172.21.15.125:0/389661071 conn(0x7fdcc4069930 msgr2=0x7fdcc4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:52.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 shutdown_connections 2023-10-18T12:01:52.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.349+0000 7fdccc0b4640 1 -- 172.21.15.125:0/389661071 wait complete. 2023-10-18T12:01:52.363 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-18T12:01:52.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.468+0000 7f997d41a640 1 -- 172.21.15.125:0/3797875568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99780cf9e0 msgr2=0x7f99780d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.468+0000 7f997d41a640 1 --2- 172.21.15.125:0/3797875568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99780cf9e0 0x7f99780d1810 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9960009860 tx=0x7f996002bde0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.468+0000 7f997d41a640 1 -- 172.21.15.125:0/3797875568 shutdown_connections 2023-10-18T12:01:52.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.468+0000 7f997d41a640 1 --2- 172.21.15.125:0/3797875568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99780cf9e0 0x7f99780d1810 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.468+0000 7f997d41a640 1 -- 172.21.15.125:0/3797875568 >> 172.21.15.125:0/3797875568 conn(0x7f99780d0cf0 msgr2=0x7f99780d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:52.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.468+0000 7f997d41a640 1 -- 172.21.15.125:0/3797875568 shutdown_connections 2023-10-18T12:01:52.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.468+0000 7f997d41a640 1 -- 172.21.15.125:0/3797875568 wait complete. 2023-10-18T12:01:52.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.469+0000 7f997d41a640 1 Processor -- start 2023-10-18T12:01:52.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.469+0000 7f997d41a640 1 -- start start 2023-10-18T12:01:52.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f997d41a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99780cf9e0 0x7f9978151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:52.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f997d41a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f997815b000 0x7f99780c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:52.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f997d41a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f997815b3e0 0x7f99780c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:52.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f997d41a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781f1200 con 0x7f997815b000 2023-10-18T12:01:52.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f997d41a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c4910 con 0x7f997815b3e0 2023-10-18T12:01:52.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f997d41a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c4bf0 con 0x7f99780cf9e0 2023-10-18T12:01:52.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f997815b000 0x7f99780c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:52.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f997815b000 0x7f99780c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57672/0 (socket says 172.21.15.125:57672) 2023-10-18T12:01:52.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 -- 172.21.15.125:0/3303030464 learned_addr learned my addr 172.21.15.125:0/3303030464 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:52.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f9976ffd640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99780cf9e0 0x7f9978151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:52.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f99777fe640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f997815b3e0 0x7f99780c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:52.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 -- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99780cf9e0 msgr2=0x7f9978151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99780cf9e0 0x7f9978151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 -- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f997815b3e0 msgr2=0x7f99780c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:52.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f997815b3e0 0x7f99780c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 -- 172.21.15.125:0/3303030464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9960009490 con 0x7f997815b000 2023-10-18T12:01:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.470+0000 7f996ffff640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f997815b000 0x7f99780c87d0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f9960009980 tx=0x7f99600038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.471+0000 7f996f7fe640 1 -- 172.21.15.125:0/3303030464 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960003ea0 con 0x7f997815b000 2023-10-18T12:01:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.471+0000 7f996f7fe640 1 -- 172.21.15.125:0/3303030464 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9960005020 con 0x7f997815b000 2023-10-18T12:01:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.471+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99780c4ed0 con 0x7f997815b000 2023-10-18T12:01:52.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.471+0000 7f996f7fe640 1 -- 172.21.15.125:0/3303030464 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960005430 con 0x7f997815b000 2023-10-18T12:01:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.471+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99780c5490 con 0x7f997815b000 2023-10-18T12:01:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.472+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99780690e0 con 0x7f997815b000 2023-10-18T12:01:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.472+0000 7f996f7fe640 1 -- 172.21.15.125:0/3303030464 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f996002dd50 con 0x7f997815b000 2023-10-18T12:01:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.472+0000 7f996f7fe640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f995806aff0 0x7f995806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:52.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.472+0000 7f996f7fe640 1 -- 172.21.15.125:0/3303030464 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f99600abbb0 con 0x7f997815b000 2023-10-18T12:01:52.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.472+0000 7f9976ffd640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f995806aff0 0x7f995806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:52.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.473+0000 7f9976ffd640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f995806aff0 0x7f995806d4b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9968001650 tx=0x7f9968009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:52.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.477+0000 7f996f7fe640 1 -- 172.21.15.125:0/3303030464 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9960076840 con 0x7f997815b000 2023-10-18T12:01:52.715 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:52.713+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f99780cd3f0 con 0x7f997815b000 2023-10-18T12:01:53.032 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:53.029+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:53.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.038+0000 7f996f7fe640 1 -- 172.21.15.125:0/3303030464 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3176) v1 ==== 187+0+0 (secure 0 0 0) 0x7f9960039100 con 0x7f997815b000 2023-10-18T12:01:53.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f995806aff0 msgr2=0x7f995806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f995806aff0 0x7f995806d4b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9968001650 tx=0x7f9968009040 comp rx=0 tx=0).stop 2023-10-18T12:01:53.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f997815b000 msgr2=0x7f99780c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.043 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f997815b000 0x7f99780c87d0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f9960009980 tx=0x7f99600038a0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 shutdown_connections 2023-10-18T12:01:53.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f995806aff0 0x7f995806d4b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f997815b3e0 0x7f99780c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f997815b000 0x7f99780c87d0 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 --2- 172.21.15.125:0/3303030464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99780cf9e0 0x7f9978151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 >> 172.21.15.125:0/3303030464 conn(0x7f99780d0cf0 msgr2=0x7f9978154040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:53.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 shutdown_connections 2023-10-18T12:01:53.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.040+0000 7f997d41a640 1 -- 172.21.15.125:0/3303030464 wait complete. 2023-10-18T12:01:53.053 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2023-10-18T12:01:53.054 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-18T12:01:53.054 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-18T12:01:53.055 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.155+0000 7f7d22e97640 1 -- 172.21.15.125:0/1566526215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d1c15b2a0 msgr2=0x7f7d1c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.155+0000 7f7d22e97640 1 --2- 172.21.15.125:0/1566526215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d1c15b2a0 0x7f7d1c0d1910 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009830 tx=0x7f7d0802bd60 comp rx=0 tx=0).stop 2023-10-18T12:01:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.155+0000 7f7d22e97640 1 -- 172.21.15.125:0/1566526215 shutdown_connections 2023-10-18T12:01:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.155+0000 7f7d22e97640 1 --2- 172.21.15.125:0/1566526215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d1c15b2a0 0x7f7d1c0d1910 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.155+0000 7f7d22e97640 1 -- 172.21.15.125:0/1566526215 >> 172.21.15.125:0/1566526215 conn(0x7f7d1c0d0d60 msgr2=0x7f7d1c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.155+0000 7f7d22e97640 1 -- 172.21.15.125:0/1566526215 shutdown_connections 2023-10-18T12:01:53.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.155+0000 7f7d22e97640 1 -- 172.21.15.125:0/1566526215 wait complete. 2023-10-18T12:01:53.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.156+0000 7f7d22e97640 1 Processor -- start 2023-10-18T12:01:53.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.156+0000 7f7d22e97640 1 -- start start 2023-10-18T12:01:53.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d22e97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d1c0ceb40 0x7f7d1c1f1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d22e97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d1c0cf340 0x7f7d1c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d22e97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d1c15b2a0 0x7f7d1c0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d22e97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c153d90 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d22e97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0b6140 con 0x7f7d1c0cf340 2023-10-18T12:01:53.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d22e97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0b3c80 con 0x7f7d1c0ceb40 2023-10-18T12:01:53.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d21434640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d1c15b2a0 0x7f7d1c0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d21434640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d1c15b2a0 0x7f7d1c0c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57684/0 (socket says 172.21.15.125:57684) 2023-10-18T12:01:53.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d21434640 1 -- 172.21.15.125:0/2363053106 learned_addr learned my addr 172.21.15.125:0/2363053106 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:53.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.157+0000 7f7d1bfff640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d1c0cf340 0x7f7d1c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d20c33640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d1c0ceb40 0x7f7d1c1f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d21434640 1 -- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d1c0ceb40 msgr2=0x7f7d1c1f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d21434640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d1c0ceb40 0x7f7d1c1f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d21434640 1 -- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d1c0cf340 msgr2=0x7f7d1c15de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d21434640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d1c0cf340 0x7f7d1c15de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d21434640 1 -- 172.21.15.125:0/2363053106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08009490 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d21434640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d1c15b2a0 0x7f7d1c0c9e40 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c790 tx=0x7f7d1400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:53.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d197fa640 1 -- 172.21.15.125:0/2363053106 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400cea0 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c0b3f60 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d197fa640 1 -- 172.21.15.125:0/2363053106 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7d14003b70 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d197fa640 1 -- 172.21.15.125:0/2363053106 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400a850 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.158+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c0b4520 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.159+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c0690e0 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.160+0000 7f7d197fa640 1 -- 172.21.15.125:0/2363053106 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f7d1400a9b0 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.160+0000 7f7d197fa640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7d0006aff0 0x7f7d0006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.160+0000 7f7d197fa640 1 -- 172.21.15.125:0/2363053106 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f7d14089a50 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.160+0000 7f7d20c33640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7d0006aff0 0x7f7d0006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.161+0000 7f7d20c33640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7d0006aff0 0x7f7d0006d4b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d100011e0 tx=0x7f7d10007d20 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:53.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.164+0000 7f7d197fa640 1 -- 172.21.15.125:0/2363053106 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d14054710 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.344+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d1c1589e0 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.345+0000 7f7d197fa640 1 -- 172.21.15.125:0/2363053106 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+14740 (secure 0 0 0) 0x7f7d14089070 con 0x7f7d1c15b2a0 2023-10-18T12:01:53.348 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:53.348 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":254,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:52.034512+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6834/3879855806":"2023-10-19T12:01:49.025704+0000","172.21.15.125:6832/3879855806":"2023-10-19T12:01:49.025704+0000","172.21.15.139:0/3172661499":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1929890550":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6832/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.125:6835/1851256737":"2023-10-19T12:01:42.552858+0000","172.21.15.139:0/1706867588":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/206803400":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/1890059388":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/662473594":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/2446560793":"2023-10-19T12:01:39.247485+0000","172.21.15.139:6833/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6832/3422518215":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3037918695":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/3823693717":"2023-10-19T12:01:33.783814+0000","172.21.15.125:6833/1851256737":"2023-10-19T12:01:42.552858+0000","172.21.15.139:0/3119295615":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/668821395":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/3910260091":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/558106977":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2896936218":"2023-10-19T12:01:33.783814+0000","172.21.15.139:6833/2483056868":"2023-10-19T12:01:39.247485+0000","172.21.15.139:0/695463051":"2023-10-19T12:01:33.783814+0000","172.21.15.139:0/2469983511":"2023-10-19T12:01:33.783814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:53.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7d0006aff0 msgr2=0x7f7d0006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7d0006aff0 0x7f7d0006d4b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d100011e0 tx=0x7f7d10007d20 comp rx=0 tx=0).stop 2023-10-18T12:01:53.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d1c15b2a0 msgr2=0x7f7d1c0c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d1c15b2a0 0x7f7d1c0c9e40 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c790 tx=0x7f7d1400cc50 comp rx=0 tx=0).stop 2023-10-18T12:01:53.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 shutdown_connections 2023-10-18T12:01:53.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f7d0006aff0 0x7f7d0006d4b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d1c0cf340 0x7f7d1c15de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7d1c15b2a0 0x7f7d1c0c9e40 unknown :-1 s=CLOSED pgs=2977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 --2- 172.21.15.125:0/2363053106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d1c0ceb40 0x7f7d1c1f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 >> 172.21.15.125:0/2363053106 conn(0x7f7d1c0d0d60 msgr2=0x7f7d1c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:53.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 shutdown_connections 2023-10-18T12:01:53.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.347+0000 7f7d22e97640 1 -- 172.21.15.125:0/2363053106 wait complete. 2023-10-18T12:01:53.360 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:01:53.361 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:01:53.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.465+0000 7f062244f640 1 -- 172.21.15.125:0/1642961623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 msgr2=0x7f061c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.465+0000 7f062244f640 1 --2- 172.21.15.125:0/1642961623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c0d1910 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7f0610005fd0 tx=0x7f061002bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:53.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.465+0000 7f062244f640 1 -- 172.21.15.125:0/1642961623 shutdown_connections 2023-10-18T12:01:53.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.465+0000 7f062244f640 1 --2- 172.21.15.125:0/1642961623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c0ceb40 0x7f061c0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.465+0000 7f062244f640 1 --2- 172.21.15.125:0/1642961623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c0d1910 unknown :-1 s=CLOSED pgs=2979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.465+0000 7f062244f640 1 -- 172.21.15.125:0/1642961623 >> 172.21.15.125:0/1642961623 conn(0x7f061c0d0d60 msgr2=0x7f061c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:53.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.465+0000 7f062244f640 1 -- 172.21.15.125:0/1642961623 shutdown_connections 2023-10-18T12:01:53.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.465+0000 7f062244f640 1 -- 172.21.15.125:0/1642961623 wait complete. 2023-10-18T12:01:53.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.466+0000 7f062244f640 1 Processor -- start 2023-10-18T12:01:53.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.466+0000 7f062244f640 1 -- start start 2023-10-18T12:01:53.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f062244f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f062244f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f062244f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c15b2a0 0x7f061c06a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f062244f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c156ce0 con 0x7f061c0cf340 2023-10-18T12:01:53.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f062244f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c155b80 con 0x7f061c0ceb40 2023-10-18T12:01:53.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f062244f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c156ef0 con 0x7f061c15b2a0 2023-10-18T12:01:53.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f061b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f061b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57706/0 (socket says 172.21.15.125:57706) 2023-10-18T12:01:53.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f061b7fe640 1 -- 172.21.15.125:0/2575872816 learned_addr learned my addr 172.21.15.125:0/2575872816 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:53.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f061bfff640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.467+0000 7f06209ec640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c15b2a0 0x7f061c06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f061b7fe640 1 -- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c15b2a0 msgr2=0x7f061c06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f061b7fe640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c15b2a0 0x7f061c06a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f061b7fe640 1 -- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 msgr2=0x7f061c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f061b7fe640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f061b7fe640 1 -- 172.21.15.125:0/2575872816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610009560 con 0x7f061c0cf340 2023-10-18T12:01:53.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f061b7fe640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c158880 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f060c00c400 tx=0x7f060c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:53.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f0618ff9640 1 -- 172.21.15.125:0/2575872816 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c0052d0 con 0x7f061c0cf340 2023-10-18T12:01:53.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f0618ff9640 1 -- 172.21.15.125:0/2575872816 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f060c0035f0 con 0x7f061c0cf340 2023-10-18T12:01:53.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c0b0d50 con 0x7f061c0cf340 2023-10-18T12:01:53.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f0618ff9640 1 -- 172.21.15.125:0/2575872816 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c003a00 con 0x7f061c0cf340 2023-10-18T12:01:53.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.468+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c157790 con 0x7f061c0cf340 2023-10-18T12:01:53.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.469+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c1572e0 con 0x7f061c0cf340 2023-10-18T12:01:53.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.469+0000 7f0618ff9640 1 -- 172.21.15.125:0/2575872816 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f060c003b60 con 0x7f061c0cf340 2023-10-18T12:01:53.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.470+0000 7f0618ff9640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f060806aff0 0x7f060806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.470+0000 7f0618ff9640 1 -- 172.21.15.125:0/2575872816 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f060c089130 con 0x7f061c0cf340 2023-10-18T12:01:53.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.470+0000 7f061bfff640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f060806aff0 0x7f060806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.470+0000 7f061bfff640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f060806aff0 0x7f060806d4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0610035000 tx=0x7f0610004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:53.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.474+0000 7f0618ff9640 1 -- 172.21.15.125:0/2575872816 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f060c053f20 con 0x7f061c0cf340 2023-10-18T12:01:53.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.644+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f061c0690e0 con 0x7f061c0cf340 2023-10-18T12:01:53.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.644+0000 7f0618ff9640 1 -- 172.21.15.125:0/2575872816 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v254) v1 ==== 93+0+10 (secure 0 0 0) 0x7f060c057540 con 0x7f061c0cf340 2023-10-18T12:01:53.647 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:01:53.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f060806aff0 msgr2=0x7f060806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f060806aff0 0x7f060806d4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0610035000 tx=0x7f0610004730 comp rx=0 tx=0).stop 2023-10-18T12:01:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 msgr2=0x7f061c158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c158880 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f060c00c400 tx=0x7f060c00c8c0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 shutdown_connections 2023-10-18T12:01:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f061c15b2a0 0x7f061c06a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f061c0cf340 0x7f061c158880 unknown :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f060806aff0 0x7f060806d4b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 --2- 172.21.15.125:0/2575872816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f061c0ceb40 0x7f061c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 >> 172.21.15.125:0/2575872816 conn(0x7f061c0d0d60 msgr2=0x7f061c06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:53.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 shutdown_connections 2023-10-18T12:01:53.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.646+0000 7f062244f640 1 -- 172.21.15.125:0/2575872816 wait complete. 2023-10-18T12:01:53.659 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-18T12:01:53.750 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:53.748+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:53.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.760+0000 7f50273f5640 1 -- 172.21.15.125:0/244766418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f502015b2a0 msgr2=0x7f50200cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.760+0000 7f50273f5640 1 --2- 172.21.15.125:0/244766418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f502015b2a0 0x7f50200cf720 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5010005fd0 tx=0x7f501002bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.761+0000 7f50273f5640 1 -- 172.21.15.125:0/244766418 shutdown_connections 2023-10-18T12:01:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.761+0000 7f50273f5640 1 --2- 172.21.15.125:0/244766418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f502015b2a0 0x7f50200cf720 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.761+0000 7f50273f5640 1 --2- 172.21.15.125:0/244766418 >> v2:172.21.15.139:3300/0 conn(0x7f50200cf340 0x7f50200d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.761+0000 7f50273f5640 1 -- 172.21.15.125:0/244766418 >> 172.21.15.125:0/244766418 conn(0x7f50200d0d60 msgr2=0x7f50200d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:53.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.761+0000 7f50273f5640 1 -- 172.21.15.125:0/244766418 shutdown_connections 2023-10-18T12:01:53.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.761+0000 7f50273f5640 1 -- 172.21.15.125:0/244766418 wait complete. 2023-10-18T12:01:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.762+0000 7f50273f5640 1 Processor -- start 2023-10-18T12:01:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.762+0000 7f50273f5640 1 -- start start 2023-10-18T12:01:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f50273f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50200ceb40 0x7f502014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f50273f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50200cf340 0x7f502014cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f50273f5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f502015b2a0 0x7f502015df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f50273f5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200c8e90 con 0x7f502015b2a0 2023-10-18T12:01:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f50273f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502014c810 con 0x7f50200ceb40 2023-10-18T12:01:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f50273f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200c8210 con 0x7f50200cf340 2023-10-18T12:01:53.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025992640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f502015b2a0 0x7f502015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50200ceb40 0x7f502014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50200ceb40 0x7f502014c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:40906/0 (socket says 172.21.15.125:40906) 2023-10-18T12:01:53.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025992640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f502015b2a0 0x7f502015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57712/0 (socket says 172.21.15.125:57712) 2023-10-18T12:01:53.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 -- 172.21.15.125:0/3303645610 learned_addr learned my addr 172.21.15.125:0/3303645610 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5024990640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50200cf340 0x7f502014cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 -- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50200cf340 msgr2=0x7f502014cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50200cf340 0x7f502014cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 -- 172.21.15.125:0/3303645610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f502015b2a0 msgr2=0x7f502015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:53.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f502015b2a0 0x7f502015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:53.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 -- 172.21.15.125:0/3303645610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c009490 con 0x7f50200ceb40 2023-10-18T12:01:53.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5024990640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50200cf340 0x7f502014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:53.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.763+0000 7f5025191640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50200ceb40 0x7f502014c2d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f501c00ea90 tx=0x7f501c00ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:53.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.764+0000 7f5015ffb640 1 -- 172.21.15.125:0/3303645610 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00ce40 con 0x7f50200ceb40 2023-10-18T12:01:53.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.764+0000 7f5015ffb640 1 -- 172.21.15.125:0/3303645610 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f501c004020 con 0x7f50200ceb40 2023-10-18T12:01:53.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.764+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5010009560 con 0x7f50200ceb40 2023-10-18T12:01:53.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.764+0000 7f5015ffb640 1 -- 172.21.15.125:0/3303645610 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c004430 con 0x7f50200ceb40 2023-10-18T12:01:53.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.764+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50200c87d0 con 0x7f50200ceb40 2023-10-18T12:01:53.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.765+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50200c9100 con 0x7f50200ceb40 2023-10-18T12:01:53.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.765+0000 7f5015ffb640 1 -- 172.21.15.125:0/3303645610 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f501c0035f0 con 0x7f50200ceb40 2023-10-18T12:01:53.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.765+0000 7f5015ffb640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f500406aff0 0x7f500406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:53.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.766+0000 7f5015ffb640 1 -- 172.21.15.125:0/3303645610 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f501c014070 con 0x7f50200ceb40 2023-10-18T12:01:53.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.766+0000 7f5024990640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f500406aff0 0x7f500406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:53.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.766+0000 7f5024990640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f500406aff0 0x7f500406d4b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5010035000 tx=0x7f5010004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:53.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:53.770+0000 7f5015ffb640 1 -- 172.21.15.125:0/3303645610 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f501c055310 con 0x7f50200ceb40 2023-10-18T12:01:54.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.013+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f50200690e0 con 0x7f50200ceb40 2023-10-18T12:01:54.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.044+0000 7f5015ffb640 1 -- 172.21.15.125:0/3303645610 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3177) v1 ==== 158+0+0 (secure 0 0 0) 0x7f501c058930 con 0x7f50200ceb40 2023-10-18T12:01:54.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f500406aff0 msgr2=0x7f500406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f500406aff0 0x7f500406d4b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5010035000 tx=0x7f5010004730 comp rx=0 tx=0).stop 2023-10-18T12:01:54.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50200ceb40 msgr2=0x7f502014c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50200ceb40 0x7f502014c2d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f501c00ea90 tx=0x7f501c00ef50 comp rx=0 tx=0).stop 2023-10-18T12:01:54.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 shutdown_connections 2023-10-18T12:01:54.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f502015b2a0 0x7f502015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50200cf340 0x7f502014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f500406aff0 0x7f500406d4b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 --2- 172.21.15.125:0/3303645610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50200ceb40 0x7f502014c2d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 >> 172.21.15.125:0/3303645610 conn(0x7f50200d0d60 msgr2=0x7f502015a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:54.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 shutdown_connections 2023-10-18T12:01:54.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.046+0000 7f50273f5640 1 -- 172.21.15.125:0/3303645610 wait complete. 2023-10-18T12:01:54.059 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T12:01:54.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.161+0000 7f22e8f8e640 1 -- 172.21.15.125:0/1081852906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22e406af00 msgr2=0x7f22e406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.161+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/1081852906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22e406af00 0x7f22e406b2e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f22d4005fd0 tx=0x7f22d402c0e0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.161+0000 7f22e8f8e640 1 -- 172.21.15.125:0/1081852906 shutdown_connections 2023-10-18T12:01:54.164 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.161+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/1081852906 >> v2:172.21.15.139:3300/0 conn(0x7f22e40abc10 0x7f22e40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.161+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/1081852906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22e406af00 0x7f22e406b2e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.161+0000 7f22e8f8e640 1 -- 172.21.15.125:0/1081852906 >> 172.21.15.125:0/1081852906 conn(0x7f22e4069930 msgr2=0x7f22e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:54.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.161+0000 7f22e8f8e640 1 -- 172.21.15.125:0/1081852906 shutdown_connections 2023-10-18T12:01:54.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.161+0000 7f22e8f8e640 1 -- 172.21.15.125:0/1081852906 wait complete. 2023-10-18T12:01:54.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.162+0000 7f22e8f8e640 1 Processor -- start 2023-10-18T12:01:54.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.162+0000 7f22e8f8e640 1 -- start start 2023-10-18T12:01:54.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e8f8e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22e406af00 0x7f22e40b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e8f8e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40abc10 0x7f22e40afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e8f8e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22e40c96b0 0x7f22e412eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e8f8e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40b0340 con 0x7f22e40c96b0 2023-10-18T12:01:54.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e8f8e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e412f070 con 0x7f22e40abc10 2023-10-18T12:01:54.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e8f8e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41f1200 con 0x7f22e406af00 2023-10-18T12:01:54.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22e40c96b0 0x7f22e412eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e2d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22e40c96b0 0x7f22e412eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57726/0 (socket says 172.21.15.125:57726) 2023-10-18T12:01:54.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e2d9d640 1 -- 172.21.15.125:0/2206085237 learned_addr learned my addr 172.21.15.125:0/2206085237 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:54.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e259c640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22e406af00 0x7f22e40b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.169 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.163+0000 7f22e1d9b640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40abc10 0x7f22e40afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22e2d9d640 1 -- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22e406af00 msgr2=0x7f22e40b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22e2d9d640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22e406af00 0x7f22e40b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22e2d9d640 1 -- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40abc10 msgr2=0x7f22e40afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22e2d9d640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40abc10 0x7f22e40afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22e2d9d640 1 -- 172.21.15.125:0/2206085237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4009840 con 0x7f22e40c96b0 2023-10-18T12:01:54.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22e2d9d640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22e40c96b0 0x7f22e412eb30 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00e970 tx=0x7f22dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:54.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22d2ffd640 1 -- 172.21.15.125:0/2206085237 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc009720 con 0x7f22e40c96b0 2023-10-18T12:01:54.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22d2ffd640 1 -- 172.21.15.125:0/2206085237 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f22dc00ccc0 con 0x7f22e40c96b0 2023-10-18T12:01:54.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e412f310 con 0x7f22e40c96b0 2023-10-18T12:01:54.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22d2ffd640 1 -- 172.21.15.125:0/2206085237 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc005280 con 0x7f22e40c96b0 2023-10-18T12:01:54.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.164+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e412f8d0 con 0x7f22e40c96b0 2023-10-18T12:01:54.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.165+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40690e0 con 0x7f22e40c96b0 2023-10-18T12:01:54.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.165+0000 7f22d2ffd640 1 -- 172.21.15.125:0/2206085237 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f22dc0053e0 con 0x7f22e40c96b0 2023-10-18T12:01:54.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.166+0000 7f22d2ffd640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f22c406aff0 0x7f22c406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.166+0000 7f22d2ffd640 1 -- 172.21.15.125:0/2206085237 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f22dc014070 con 0x7f22e40c96b0 2023-10-18T12:01:54.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.166+0000 7f22e259c640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f22c406aff0 0x7f22c406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.166+0000 7f22e259c640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f22c406aff0 0x7f22c406d4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22d4005fd0 tx=0x7f22d4004ab0 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:54.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.170+0000 7f22d2ffd640 1 -- 172.21.15.125:0/2206085237 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22dc0551e0 con 0x7f22e40c96b0 2023-10-18T12:01:54.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.340+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f22e40cc420 con 0x7f22e40c96b0 2023-10-18T12:01:54.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.340+0000 7f22d2ffd640 1 -- 172.21.15.125:0/2206085237 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v82) v1 ==== 59+0+2220 (secure 0 0 0) 0x7f22dc058800 con 0x7f22e40c96b0 2023-10-18T12:01:54.343 INFO:teuthology.orchestra.run.smithi125.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T12:01:54.344 INFO:teuthology.orchestra.run.smithi125.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T12:01:54.344 INFO:teuthology.orchestra.run.smithi125.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T12:01:54.344 INFO:teuthology.orchestra.run.smithi125.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5134.943464 2023-10-18T12:01:54.344 INFO:teuthology.orchestra.run.smithi125.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5500.423174 2023-10-18T12:01:54.345 INFO:teuthology.orchestra.run.smithi125.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4899.127456 2023-10-18T12:01:54.345 INFO:teuthology.orchestra.run.smithi125.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5000.675208 2023-10-18T12:01:54.345 INFO:teuthology.orchestra.run.smithi125.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4928.112854 2023-10-18T12:01:54.345 INFO:teuthology.orchestra.run.smithi125.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4674.690523 2023-10-18T12:01:54.345 INFO:teuthology.orchestra.run.smithi125.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5061.899360 2023-10-18T12:01:54.346 INFO:teuthology.orchestra.run.smithi125.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5056.983856 2023-10-18T12:01:54.346 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds 20/20 2023-10-18T12:01:54.346 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T12:01:54.346 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced debug_ms 1/1 2023-10-18T12:01:54.346 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T12:01:54.347 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T12:01:54.347 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T12:01:54.347 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_bal_split_size 100 2023-10-18T12:01:54.347 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_frag true 2023-10-18T12:01:54.347 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_debug_scatterstat true 2023-10-18T12:01:54.348 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T12:01:54.348 INFO:teuthology.orchestra.run.smithi125.stdout:mds dev mds_verify_scatter true 2023-10-18T12:01:54.348 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T12:01:54.348 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T12:01:54.348 INFO:teuthology.orchestra.run.smithi125.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T12:01:54.349 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced client_mount_timeout 600 2023-10-18T12:01:54.349 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_client 20/20 2023-10-18T12:01:54.349 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced debug_ms 1/1 2023-10-18T12:01:54.349 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T12:01:54.349 INFO:teuthology.orchestra.run.smithi125.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T12:01:54.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f22c406aff0 msgr2=0x7f22c406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f22c406aff0 0x7f22c406d4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22d4005fd0 tx=0x7f22d4004ab0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22e40c96b0 msgr2=0x7f22e412eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22e40c96b0 0x7f22e412eb30 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00e970 tx=0x7f22dc00ed70 comp rx=0 tx=0).stop 2023-10-18T12:01:54.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 shutdown_connections 2023-10-18T12:01:54.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f22c406aff0 0x7f22c406d4b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40abc10 0x7f22e40afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f22e40c96b0 0x7f22e412eb30 unknown :-1 s=CLOSED pgs=2981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 --2- 172.21.15.125:0/2206085237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22e406af00 0x7f22e40b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 >> 172.21.15.125:0/2206085237 conn(0x7f22e4069930 msgr2=0x7f22e415adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:54.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 shutdown_connections 2023-10-18T12:01:54.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.343+0000 7f22e8f8e640 1 -- 172.21.15.125:0/2206085237 wait complete. 2023-10-18T12:01:54.356 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T12:01:54.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.461+0000 7f5613173640 1 -- 172.21.15.125:0/647557777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f40052a0 msgr2=0x7f55f400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.461+0000 7f5613173640 1 --2- 172.21.15.125:0/647557777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f40052a0 0x7f55f400b980 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00bbb0 tx=0x7f55f40057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.462+0000 7f5613173640 1 -- 172.21.15.125:0/647557777 shutdown_connections 2023-10-18T12:01:54.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.462+0000 7f5613173640 1 --2- 172.21.15.125:0/647557777 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f55f40052a0 0x7f55f400b980 unknown :-1 s=CLOSED pgs=2983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.462+0000 7f5613173640 1 -- 172.21.15.125:0/647557777 >> 172.21.15.125:0/647557777 conn(0x7f560c0d0d60 msgr2=0x7f560c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:54.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.462+0000 7f5613173640 1 -- 172.21.15.125:0/647557777 shutdown_connections 2023-10-18T12:01:54.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.462+0000 7f5613173640 1 -- 172.21.15.125:0/647557777 wait complete. 2023-10-18T12:01:54.468 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.463+0000 7f5613173640 1 Processor -- start 2023-10-18T12:01:54.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.463+0000 7f5613173640 1 -- start start 2023-10-18T12:01:54.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5613173640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f40052a0 0x7f560c1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5613173640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f560c1f4b90 0x7f560c1f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5613173640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f560c0c7e30 0x7f560c0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5613173640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0c87f0 con 0x7f560c1f4b90 2023-10-18T12:01:54.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5613173640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1f2580 con 0x7f560c0c7e30 2023-10-18T12:01:54.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5613173640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1f1410 con 0x7f55f40052a0 2023-10-18T12:01:54.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f560c1f4b90 0x7f560c1f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.470 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f560c1f4b90 0x7f560c1f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57750/0 (socket says 172.21.15.125:57750) 2023-10-18T12:01:54.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 -- 172.21.15.125:0/421663883 learned_addr learned my addr 172.21.15.125:0/421663883 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:54.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5610f0f640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f40052a0 0x7f560c1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5611710640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f560c0c7e30 0x7f560c0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 -- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f40052a0 msgr2=0x7f560c1f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f40052a0 0x7f560c1f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 -- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f560c0c7e30 msgr2=0x7f560c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f560c0c7e30 0x7f560c0c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 -- 172.21.15.125:0/421663883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c1f5150 con 0x7f560c1f4b90 2023-10-18T12:01:54.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5611710640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f560c0c7e30 0x7f560c0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:54.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f560bfff640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f560c1f4b90 0x7f560c1f0ed0 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f5600001200 tx=0x7f56000039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:54.472 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f56097fa640 1 -- 172.21.15.125:0/421663883 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600011900 con 0x7f560c1f4b90 2023-10-18T12:01:54.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f56097fa640 1 -- 172.21.15.125:0/421663883 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5600011a60 con 0x7f560c1f4b90 2023-10-18T12:01:54.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c1f5430 con 0x7f560c1f4b90 2023-10-18T12:01:54.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f56097fa640 1 -- 172.21.15.125:0/421663883 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600011e70 con 0x7f560c1f4b90 2023-10-18T12:01:54.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.464+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c1f59f0 con 0x7f560c1f4b90 2023-10-18T12:01:54.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.465+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c0b03d0 con 0x7f560c1f4b90 2023-10-18T12:01:54.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.466+0000 7f56097fa640 1 -- 172.21.15.125:0/421663883 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f56000103c0 con 0x7f560c1f4b90 2023-10-18T12:01:54.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.466+0000 7f56097fa640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f55f806aff0 0x7f55f806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.466+0000 7f56097fa640 1 -- 172.21.15.125:0/421663883 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f560008a870 con 0x7f560c1f4b90 2023-10-18T12:01:54.475 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:54.472+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:54.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.466+0000 7f5610f0f640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f55f806aff0 0x7f55f806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.467+0000 7f5610f0f640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f55f806aff0 0x7f55f806d4b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5604000f80 tx=0x7f5604006eb0 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:54.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.471+0000 7f56097fa640 1 -- 172.21.15.125:0/421663883 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5600055550 con 0x7f560c1f4b90 2023-10-18T12:01:54.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.647+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f560c0690e0 con 0x7f560c1f4b90 2023-10-18T12:01:54.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.650+0000 7f56097fa640 1 -- 172.21.15.125:0/421663883 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v83)=0 v83) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5600020020 con 0x7f560c1f4b90 2023-10-18T12:01:54.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f55f806aff0 msgr2=0x7f55f806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f55f806aff0 0x7f55f806d4b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5604000f80 tx=0x7f5604006eb0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f560c1f4b90 msgr2=0x7f560c1f0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f560c1f4b90 0x7f560c1f0ed0 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f5600001200 tx=0x7f56000039e0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 shutdown_connections 2023-10-18T12:01:54.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f55f806aff0 0x7f55f806d4b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f560c0c7e30 0x7f560c0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f560c1f4b90 0x7f560c1f0ed0 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 --2- 172.21.15.125:0/421663883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f40052a0 0x7f560c1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 >> 172.21.15.125:0/421663883 conn(0x7f560c0d0d60 msgr2=0x7f560c15a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:54.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 shutdown_connections 2023-10-18T12:01:54.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.652+0000 7f5613173640 1 -- 172.21.15.125:0/421663883 wait complete. 2023-10-18T12:01:54.665 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T12:01:54.666 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:01:54.703 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:01:54.744 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T12:01:54.745 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:01:54.745 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:54.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:01:54.761 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:01:54.762 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:54.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:01:54.800 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:01:54.801 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:54.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:01:54.816 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:01:54.817 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:01:54.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:01:54.856 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T12:01:54.856 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:01:54.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.958+0000 7fbffe4a9640 1 -- 172.21.15.125:0/1332592605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 msgr2=0x7fbff806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.958+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/1332592605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 0x7fbff806bb60 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005fd0 tx=0x7fbfe802bcf0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.958+0000 7fbffe4a9640 1 -- 172.21.15.125:0/1332592605 shutdown_connections 2023-10-18T12:01:54.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.958+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/1332592605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff8070af0 0x7fbff80affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.958+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/1332592605 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 0x7fbff806bb60 unknown :-1 s=CLOSED pgs=2985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.958+0000 7fbffe4a9640 1 -- 172.21.15.125:0/1332592605 >> 172.21.15.125:0/1332592605 conn(0x7fbff8069930 msgr2=0x7fbff8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:54.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.958+0000 7fbffe4a9640 1 -- 172.21.15.125:0/1332592605 shutdown_connections 2023-10-18T12:01:54.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.958+0000 7fbffe4a9640 1 -- 172.21.15.125:0/1332592605 wait complete. 2023-10-18T12:01:54.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.960+0000 7fbffe4a9640 1 Processor -- start 2023-10-18T12:01:54.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.960+0000 7fbffe4a9640 1 -- start start 2023-10-18T12:01:54.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.960+0000 7fbffe4a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff8070af0 0x7fbff8131770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.960+0000 7fbffe4a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 0x7fbff8156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.960+0000 7fbffe4a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff8141690 0x7fbff80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.960+0000 7fbffe4a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff81572f0 con 0x7fbff813e520 2023-10-18T12:01:54.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.960+0000 7fbffe4a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff81575d0 con 0x7fbff8141690 2023-10-18T12:01:54.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.960+0000 7fbffe4a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff81578b0 con 0x7fbff8070af0 2023-10-18T12:01:54.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 0x7fbff8156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 0x7fbff8156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57782/0 (socket says 172.21.15.125:57782) 2023-10-18T12:01:54.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 -- 172.21.15.125:0/165934206 learned_addr learned my addr 172.21.15.125:0/165934206 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:54.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff7fff640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff8070af0 0x7fbff8131770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbffca46640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff8141690 0x7fbff80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 -- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff8070af0 msgr2=0x7fbff8131770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff8070af0 0x7fbff8131770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 -- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff8141690 msgr2=0x7fbff80a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:54.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff8141690 0x7fbff80a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:54.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 -- 172.21.15.125:0/165934206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe8009560 con 0x7fbff813e520 2023-10-18T12:01:54.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff77fe640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 0x7fbff8156920 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00b3f0 tx=0x7fbfec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:54.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff4ff9640 1 -- 172.21.15.125:0/165934206 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011830 con 0x7fbff813e520 2023-10-18T12:01:54.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff4ff9640 1 -- 172.21.15.125:0/165934206 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfec011a80 con 0x7fbff813e520 2023-10-18T12:01:54.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff8157b90 con 0x7fbff813e520 2023-10-18T12:01:54.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbff4ff9640 1 -- 172.21.15.125:0/165934206 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011e90 con 0x7fbff813e520 2023-10-18T12:01:54.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.961+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff8156e60 con 0x7fbff813e520 2023-10-18T12:01:54.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.962+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff81580b0 con 0x7fbff813e520 2023-10-18T12:01:54.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.963+0000 7fbff4ff9640 1 -- 172.21.15.125:0/165934206 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fbfec01a400 con 0x7fbff813e520 2023-10-18T12:01:54.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.963+0000 7fbff4ff9640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fbfe406aff0 0x7fbfe406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:54.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.963+0000 7fbff4ff9640 1 -- 172.21.15.125:0/165934206 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fbfec089c20 con 0x7fbff813e520 2023-10-18T12:01:54.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.963+0000 7fbff7fff640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fbfe406aff0 0x7fbfe406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:54.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.964+0000 7fbff7fff640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fbfe406aff0 0x7fbfe406d4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8035000 tx=0x7fbfe8004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:54.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:54.967+0000 7fbff4ff9640 1 -- 172.21.15.125:0/165934206 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff81580b0 con 0x7fbff813e520 2023-10-18T12:01:55.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.178+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbff81580b0 con 0x7fbff813e520 2023-10-18T12:01:55.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.178+0000 7fbff4ff9640 1 -- 172.21.15.125:0/165934206 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+3713 (secure 0 0 0) 0x7fbff81580b0 con 0x7fbff813e520 2023-10-18T12:01:55.182 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:55.182 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":107,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1951849392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1951849392},{"type":"v1","addr":"172.21.15.139:6836","nonce":1951849392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":9470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/290676319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":290676319},{"type":"v1","addr":"172.21.15.139:6837","nonce":290676319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":11320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1767177929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1767177929},{"type":"v1","addr":"172.21.15.125:6834","nonce":1767177929}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":13389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1493462072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1493462072},{"type":"v1","addr":"172.21.15.125:6835","nonce":1493462072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[]} 2023-10-18T12:01:55.182 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 107 2023-10-18T12:01:55.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fbfe406aff0 msgr2=0x7fbfe406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:55.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fbfe406aff0 0x7fbfe406d4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8035000 tx=0x7fbfe8004730 comp rx=0 tx=0).stop 2023-10-18T12:01:55.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 msgr2=0x7fbff8156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:55.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 0x7fbff8156920 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00b3f0 tx=0x7fbfec00b8b0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 shutdown_connections 2023-10-18T12:01:55.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fbfe406aff0 0x7fbfe406d4b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff8141690 0x7fbff80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff813e520 0x7fbff8156920 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 --2- 172.21.15.125:0/165934206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff8070af0 0x7fbff8131770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.180+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 >> 172.21.15.125:0/165934206 conn(0x7fbff8069930 msgr2=0x7fbff8140440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:55.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.181+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 shutdown_connections 2023-10-18T12:01:55.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.181+0000 7fbffe4a9640 1 -- 172.21.15.125:0/165934206 wait complete. 2023-10-18T12:01:55.194 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T12:01:55.194 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T12:01:55.195 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T12:01:55.195 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:01:55.195 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T12:01:55.196 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T12:01:55.196 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T12:01:55.196 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:01:55.197 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T12:01:55.197 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T12:01:55.197 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T12:01:55.198 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:01:55.198 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T12:01:55.198 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T12:01:55.199 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T12:01:55.199 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:01:55.201 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:55.199+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:55.204 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T12:01:55.205 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T12:01:55.206 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T12:01:55.206 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T12:01:55.207 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T12:01:55.208 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T12:01:55.208 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T12:01:55.209 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T12:01:55.247 INFO:tasks.ceph.mds.a:Started 2023-10-18T12:01:55.247 DEBUG:teuthology.parallel:result is None 2023-10-18T12:01:55.248 INFO:tasks.ceph.mds.c:Started 2023-10-18T12:01:55.249 DEBUG:teuthology.parallel:result is None 2023-10-18T12:01:55.250 INFO:tasks.ceph.mds.b:Started 2023-10-18T12:01:55.250 DEBUG:teuthology.parallel:result is None 2023-10-18T12:01:55.251 INFO:tasks.ceph.mds.d:Started 2023-10-18T12:01:55.251 DEBUG:teuthology.parallel:result is None 2023-10-18T12:01:55.252 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T12:01:55.323 INFO:tasks.ceph.mds.a.smithi125.stdout:starting mds.a at 2023-10-18T12:01:55.323 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T12:01:55.325 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2023-10-18T12:01:55.325 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2023-10-18T12:01:55.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.359+0000 7fdc84c26640 1 -- 172.21.15.125:0/1599578336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 msgr2=0x7fdc6c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.359+0000 7fdc84c26640 1 --2- 172.21.15.125:0/1599578336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 0x7fdc6c004c40 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009b50 tx=0x7fdc6c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.359+0000 7fdc84c26640 1 -- 172.21.15.125:0/1599578336 shutdown_connections 2023-10-18T12:01:55.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.359+0000 7fdc84c26640 1 --2- 172.21.15.125:0/1599578336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 0x7fdc6c004c40 unknown :-1 s=CLOSED pgs=2993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.359+0000 7fdc84c26640 1 -- 172.21.15.125:0/1599578336 >> 172.21.15.125:0/1599578336 conn(0x7fdc800d0d60 msgr2=0x7fdc800d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:55.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.359+0000 7fdc84c26640 1 -- 172.21.15.125:0/1599578336 shutdown_connections 2023-10-18T12:01:55.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.359+0000 7fdc84c26640 1 -- 172.21.15.125:0/1599578336 wait complete. 2023-10-18T12:01:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.361+0000 7fdc84c26640 1 Processor -- start 2023-10-18T12:01:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.361+0000 7fdc84c26640 1 -- start start 2023-10-18T12:01:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.361+0000 7fdc84c26640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 0x7fdc801f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.361+0000 7fdc84c26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc8015b680 0x7fdc8015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:55.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.361+0000 7fdc84c26640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc800ca2b0 0x7fdc800ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:55.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.361+0000 7fdc84c26640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc801f1410 con 0x7fdc6c0047c0 2023-10-18T12:01:55.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.361+0000 7fdc84c26640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc800cac50 con 0x7fdc8015b680 2023-10-18T12:01:55.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.361+0000 7fdc84c26640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8015cd10 con 0x7fdc800ca2b0 2023-10-18T12:01:55.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 0x7fdc801f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc77fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc8015b680 0x7fdc8015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 0x7fdc801f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57860/0 (socket says 172.21.15.125:57860) 2023-10-18T12:01:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 -- 172.21.15.125:0/3040500087 learned_addr learned my addr 172.21.15.125:0/3040500087 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7ed9d640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc800ca2b0 0x7fdc800ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 -- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc800ca2b0 msgr2=0x7fdc800ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:55.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc800ca2b0 0x7fdc800ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 -- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc8015b680 msgr2=0x7fdc8015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:55.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc8015b680 0x7fdc8015bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:55.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 -- 172.21.15.125:0/3040500087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8015c100 con 0x7fdc6c0047c0 2023-10-18T12:01:55.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc7e59c640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 0x7fdc801f0ed0 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7fdc70006630 tx=0x7fdc700039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:55.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc76ffd640 1 -- 172.21.15.125:0/3040500087 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70011900 con 0x7fdc6c0047c0 2023-10-18T12:01:55.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc76ffd640 1 -- 172.21.15.125:0/3040500087 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdc70011a60 con 0x7fdc6c0047c0 2023-10-18T12:01:55.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8015c3e0 con 0x7fdc6c0047c0 2023-10-18T12:01:55.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc76ffd640 1 -- 172.21.15.125:0/3040500087 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70011e50 con 0x7fdc6c0047c0 2023-10-18T12:01:55.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.362+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8015c9a0 con 0x7fdc6c0047c0 2023-10-18T12:01:55.371 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.363+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8015cb30 con 0x7fdc6c0047c0 2023-10-18T12:01:55.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.364+0000 7fdc76ffd640 1 -- 172.21.15.125:0/3040500087 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fdc70010420 con 0x7fdc6c0047c0 2023-10-18T12:01:55.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.364+0000 7fdc76ffd640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdc6006b020 0x7fdc6006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:55.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.364+0000 7fdc76ffd640 1 -- 172.21.15.125:0/3040500087 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fdc7008a880 con 0x7fdc6c0047c0 2023-10-18T12:01:55.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.364+0000 7fdc77fff640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdc6006b020 0x7fdc6006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:55.372 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.364+0000 7fdc77fff640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdc6006b020 0x7fdc6006d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc68036000 tx=0x7fdc680046c0 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:55.373 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.368+0000 7fdc76ffd640 1 -- 172.21.15.125:0/3040500087 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc700554e0 con 0x7fdc6c0047c0 2023-10-18T12:01:55.555 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:55.552+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdc800690e0 con 0x7fdc6c0047c0 2023-10-18T12:01:55.934 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:55.932+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:56.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.052+0000 7fdc76ffd640 1 -- 172.21.15.125:0/3040500087 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v256) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdc70058b00 con 0x7fdc6c0047c0 2023-10-18T12:01:56.055 INFO:teuthology.orchestra.run.smithi125.stderr: removed all blocklist entries 2023-10-18T12:01:56.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdc6006b020 msgr2=0x7fdc6006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdc6006b020 0x7fdc6006d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc68036000 tx=0x7fdc680046c0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 msgr2=0x7fdc801f0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 0x7fdc801f0ed0 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7fdc70006630 tx=0x7fdc700039c0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 shutdown_connections 2023-10-18T12:01:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fdc6006b020 0x7fdc6006d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc8015b680 0x7fdc8015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc800ca2b0 0x7fdc800ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 --2- 172.21.15.125:0/3040500087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc6c0047c0 0x7fdc801f0ed0 unknown :-1 s=CLOSED pgs=2994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 >> 172.21.15.125:0/3040500087 conn(0x7fdc800d0d60 msgr2=0x7fdc8006a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:56.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 shutdown_connections 2023-10-18T12:01:56.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.054+0000 7fdc84c26640 1 -- 172.21.15.125:0/3040500087 wait complete. 2023-10-18T12:01:56.069 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T12:01:56.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.173+0000 7f2550100640 1 -- 172.21.15.125:0/1827858183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 msgr2=0x7f25480c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.173+0000 7f2550100640 1 --2- 172.21.15.125:0/1827858183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 0x7f25480c97c0 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f2538007350 tx=0x7f253802ca80 comp rx=0 tx=0).stop 2023-10-18T12:01:56.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.173+0000 7f2550100640 1 -- 172.21.15.125:0/1827858183 shutdown_connections 2023-10-18T12:01:56.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.173+0000 7f2550100640 1 --2- 172.21.15.125:0/1827858183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 0x7f25480c97c0 unknown :-1 s=CLOSED pgs=2996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.173+0000 7f2550100640 1 -- 172.21.15.125:0/1827858183 >> 172.21.15.125:0/1827858183 conn(0x7f2548069930 msgr2=0x7f2548069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:56.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.173+0000 7f2550100640 1 -- 172.21.15.125:0/1827858183 shutdown_connections 2023-10-18T12:01:56.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.173+0000 7f2550100640 1 -- 172.21.15.125:0/1827858183 wait complete. 2023-10-18T12:01:56.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.174+0000 7f2550100640 1 Processor -- start 2023-10-18T12:01:56.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.174+0000 7f2550100640 1 -- start start 2023-10-18T12:01:56.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f2550100640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480c9d00 0x7f25480b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f2550100640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f254813d360 0x7f25480a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f2550100640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 0x7f25480af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f2550100640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254812a950 con 0x7f2548140660 2023-10-18T12:01:56.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f2550100640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254812abf0 con 0x7f25480c9d00 2023-10-18T12:01:56.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f2550100640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254812aed0 con 0x7f254813d360 2023-10-18T12:01:56.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 0x7f25480af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 0x7f25480af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57872/0 (socket says 172.21.15.125:57872) 2023-10-18T12:01:56.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 -- 172.21.15.125:0/3777232770 learned_addr learned my addr 172.21.15.125:0/3777232770 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:56.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254d69b640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f254813d360 0x7f25480a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254de9c640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480c9d00 0x7f25480b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 -- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f254813d360 msgr2=0x7f25480a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f254813d360 0x7f25480a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 -- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480c9d00 msgr2=0x7f25480b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480c9d00 0x7f25480b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 -- 172.21.15.125:0/3777232770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253800ae30 con 0x7f2548140660 2023-10-18T12:01:56.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.175+0000 7f254e69d640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 0x7f25480af660 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f254000cbb0 tx=0x7f254000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:56.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.176+0000 7f253e7fc640 1 -- 172.21.15.125:0/3777232770 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000ad40 con 0x7f2548140660 2023-10-18T12:01:56.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.176+0000 7f253e7fc640 1 -- 172.21.15.125:0/3777232770 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2540012420 con 0x7f2548140660 2023-10-18T12:01:56.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.176+0000 7f253e7fc640 1 -- 172.21.15.125:0/3777232770 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540012870 con 0x7f2548140660 2023-10-18T12:01:56.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.176+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254812b1b0 con 0x7f2548140660 2023-10-18T12:01:56.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.176+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254812b770 con 0x7f2548140660 2023-10-18T12:01:56.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.177+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2548070d50 con 0x7f2548140660 2023-10-18T12:01:56.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.177+0000 7f253e7fc640 1 -- 172.21.15.125:0/3777232770 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f254001a3b0 con 0x7f2548140660 2023-10-18T12:01:56.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.177+0000 7f253e7fc640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252c06aff0 0x7f252c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.177+0000 7f253e7fc640 1 -- 172.21.15.125:0/3777232770 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f2540089ea0 con 0x7f2548140660 2023-10-18T12:01:56.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.178+0000 7f254de9c640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252c06aff0 0x7f252c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.178+0000 7f254de9c640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252c06aff0 0x7f252c06d4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2544001640 tx=0x7f2544009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:56.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.182+0000 7f253e7fc640 1 -- 172.21.15.125:0/3777232770 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2540054e10 con 0x7f2548140660 2023-10-18T12:01:56.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.433+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f25480ca150 con 0x7f2548140660 2023-10-18T12:01:56.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.433+0000 7f253e7fc640 1 -- 172.21.15.125:0/3777232770 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v59) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f25480ca150 con 0x7f2548140660 2023-10-18T12:01:56.437 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:56.437 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:01:56.437 INFO:teuthology.orchestra.run.smithi125.stdout: "auth_dump": [ 2023-10-18T12:01:56.437 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.437 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.a", 2023-10-18T12:01:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lDZS6HhAACmTqdSteSWGKKtNHXmzT+g==", 2023-10-18T12:01:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:56.438 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:56.439 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.439 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.439 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.439 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.439 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.b", 2023-10-18T12:01:56.440 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lixocKxAA0pwGRs0MRsAH5t0zMbqucQ==", 2023-10-18T12:01:56.440 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.440 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:56.440 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:56.440 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:56.440 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.441 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.441 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.441 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.441 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.c", 2023-10-18T12:01:56.441 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lpX0VJhAAkPGfl8hsp1i9z/HtdqwW/g==", 2023-10-18T12:01:56.442 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.442 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:56.442 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:56.442 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:56.443 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.443 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.443 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.443 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.443 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mds.d", 2023-10-18T12:01:56.443 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lTrJuMhAAq7npEVPD+ikUKAE+pxagsw==", 2023-10-18T12:01:56.444 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.444 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:56.444 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:56.444 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:56.444 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.445 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.445 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.445 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.445 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.0", 2023-10-18T12:01:56.445 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBIsi9lxMqMHRAAcxMgm1WSdMt7sRr13Ai36A==", 2023-10-18T12:01:56.446 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.446 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:56.446 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:56.446 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.446 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.447 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.447 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.447 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.1", 2023-10-18T12:01:56.447 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBLsi9lYS22HBAAmIaHoZNrqpSyg8iB4+PQIg==", 2023-10-18T12:01:56.447 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.447 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:56.448 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:56.448 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.448 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.448 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.448 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.449 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.2", 2023-10-18T12:01:56.449 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBOsi9l32XuGhAAu5w4jA4JVfNHdgIbDpFfUA==", 2023-10-18T12:01:56.449 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.449 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:56.449 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:56.449 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.450 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.450 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.450 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.450 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.3", 2023-10-18T12:01:56.450 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBRsi9lG0zXGRAAuhdSCGM2anU4KmxBOUKPyw==", 2023-10-18T12:01:56.451 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.451 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:56.451 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:56.451 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.451 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.452 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.452 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.452 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.4", 2023-10-18T12:01:56.452 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBXsi9l0vgqIRAAxqXYMDyW3DiMgrBwgxa5ww==", 2023-10-18T12:01:56.452 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.453 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:56.453 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:56.453 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.453 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.453 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.453 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.454 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.5", 2023-10-18T12:01:56.454 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBasi9lszDsHxAAK1j3GXP60g8talvytaV87Q==", 2023-10-18T12:01:56.454 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.454 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:56.454 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:56.455 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.455 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.455 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.455 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.455 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.6", 2023-10-18T12:01:56.455 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBdsi9lDfYiHhAAGETNM4PVgVomfXJ+23tTQw==", 2023-10-18T12:01:56.456 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.456 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:56.456 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:56.456 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.456 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.456 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.457 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.457 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "osd.7", 2023-10-18T12:01:56.457 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBgsi9luZ80HBAAUnt2TiW9uuWUoXzg6k45gw==", 2023-10-18T12:01:56.457 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.457 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow profile osd", 2023-10-18T12:01:56.458 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile osd", 2023-10-18T12:01:56.458 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.458 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.458 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.458 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.0", 2023-10-18T12:01:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA==", 2023-10-18T12:01:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:56.459 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow r", 2023-10-18T12:01:56.460 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T12:01:56.460 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.460 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.460 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.460 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.1", 2023-10-18T12:01:56.460 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCYUrBBAAdIDLipCpHmWSZ1x5/OBJfg==", 2023-10-18T12:01:56.461 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.461 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:56.461 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:01:56.461 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:01:56.461 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:01:56.462 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.462 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.462 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.462 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.2", 2023-10-18T12:01:56.462 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lEWTQABAADgA/KKdlHAAYBVU/2rjBAg==", 2023-10-18T12:01:56.463 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.463 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:56.463 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:01:56.463 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:01:56.463 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:01:56.463 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.464 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.464 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.464 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.3", 2023-10-18T12:01:56.464 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBFsi9lCHb0CRAAZhv+P3OGGim704HfX/1V6g==", 2023-10-18T12:01:56.464 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.465 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow", 2023-10-18T12:01:56.465 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow r", 2023-10-18T12:01:56.465 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow rw", 2023-10-18T12:01:56.465 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow rwx" 2023-10-18T12:01:56.465 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.466 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.466 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.466 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "client.admin", 2023-10-18T12:01:56.466 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9l5cUCARAA60VcyGKiNgtnwOdKZPzrBg==", 2023-10-18T12:01:56.466 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.467 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:01:56.467 INFO:teuthology.orchestra.run.smithi125.stdout: "mgr": "allow *", 2023-10-18T12:01:56.467 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow *", 2023-10-18T12:01:56.467 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.467 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.467 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.468 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.468 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.x", 2023-10-18T12:01:56.468 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lr71OGxAApsxzYAFPH9QcYEgCGfhAYw==", 2023-10-18T12:01:56.468 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.468 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:01:56.469 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T12:01:56.469 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.469 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.469 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:01:56.469 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:56.470 INFO:teuthology.orchestra.run.smithi125.stdout: "entity": "mgr.y", 2023-10-18T12:01:56.470 INFO:teuthology.orchestra.run.smithi125.stdout: "key": "AQBEsi9lvw55FhAAvyDsW6jVDTcaJiDhfX7hYw==", 2023-10-18T12:01:56.470 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": { 2023-10-18T12:01:56.470 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": "allow *", 2023-10-18T12:01:56.470 INFO:teuthology.orchestra.run.smithi125.stdout: "mon": "allow profile mgr", 2023-10-18T12:01:56.471 INFO:teuthology.orchestra.run.smithi125.stdout: "osd": "allow *" 2023-10-18T12:01:56.471 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.471 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:56.471 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2023-10-18T12:01:56.471 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:01:56.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252c06aff0 msgr2=0x7f252c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.473 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252c06aff0 0x7f252c06d4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2544001640 tx=0x7f2544009040 comp rx=0 tx=0).stop 2023-10-18T12:01:56.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 msgr2=0x7f25480af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 0x7f25480af660 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f254000cbb0 tx=0x7f254000a430 comp rx=0 tx=0).stop 2023-10-18T12:01:56.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 shutdown_connections 2023-10-18T12:01:56.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2548140660 0x7f25480af660 unknown :-1 s=CLOSED pgs=2997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f254813d360 0x7f25480a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252c06aff0 0x7f252c06d4b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 --2- 172.21.15.125:0/3777232770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480c9d00 0x7f25480b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 >> 172.21.15.125:0/3777232770 conn(0x7f2548069930 msgr2=0x7f2548065490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:56.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 shutdown_connections 2023-10-18T12:01:56.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.435+0000 7f2550100640 1 -- 172.21.15.125:0/3777232770 wait complete. 2023-10-18T12:01:56.476 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:01:56.581 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 -- 172.21.15.125:0/3683127142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e400cf6e0 msgr2=0x7f2e400b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 --2- 172.21.15.125:0/3683127142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e400cf6e0 0x7f2e400b26d0 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7f2e400899f0 tx=0x7f2e3802c250 comp rx=0 tx=0).stop 2023-10-18T12:01:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 -- 172.21.15.125:0/3683127142 shutdown_connections 2023-10-18T12:01:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 --2- 172.21.15.125:0/3683127142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e400cf6e0 0x7f2e400b26d0 unknown :-1 s=CLOSED pgs=2998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 --2- 172.21.15.125:0/3683127142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e400b1ca0 0x7f2e400cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 --2- 172.21.15.125:0/3683127142 >> v2:172.21.15.139:3300/0 conn(0x7f2e4015e5e0 0x7f2e400ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 -- 172.21.15.125:0/3683127142 >> 172.21.15.125:0/3683127142 conn(0x7f2e400d0c50 msgr2=0x7f2e400d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:56.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 -- 172.21.15.125:0/3683127142 shutdown_connections 2023-10-18T12:01:56.583 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.578+0000 7f2e456cf640 1 -- 172.21.15.125:0/3683127142 wait complete. 2023-10-18T12:01:56.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.580+0000 7f2e456cf640 1 Processor -- start 2023-10-18T12:01:56.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.580+0000 7f2e456cf640 1 -- start start 2023-10-18T12:01:56.586 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.580+0000 7f2e456cf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e400b1ca0 0x7f2e40158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e456cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e400cf6e0 0x7f2e401e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e456cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e4015e5e0 0x7f2e4015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e456cf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401e4160 con 0x7f2e4015e5e0 2023-10-18T12:01:56.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e456cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401e5800 con 0x7f2e400cf6e0 2023-10-18T12:01:56.587 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e456cf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401e38e0 con 0x7f2e400b1ca0 2023-10-18T12:01:56.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e4015e5e0 0x7f2e4015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e4015e5e0 0x7f2e4015bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57900/0 (socket says 172.21.15.125:57900) 2023-10-18T12:01:56.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 -- 172.21.15.125:0/188788404 learned_addr learned my addr 172.21.15.125:0/188788404 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:56.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3effd640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e400b1ca0 0x7f2e40158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.588 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3e7fc640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e400cf6e0 0x7f2e401e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 -- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e400b1ca0 msgr2=0x7f2e40158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e400b1ca0 0x7f2e40158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 -- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e400cf6e0 msgr2=0x7f2e401e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e400cf6e0 0x7f2e401e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 -- 172.21.15.125:0/188788404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38009900 con 0x7f2e4015e5e0 2023-10-18T12:01:56.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e3f7fe640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e4015e5e0 0x7f2e4015bd20 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009d60 tx=0x7f2e38035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:56.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e277fe640 1 -- 172.21.15.125:0/188788404 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38034440 con 0x7f2e4015e5e0 2023-10-18T12:01:56.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e277fe640 1 -- 172.21.15.125:0/188788404 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2e38004450 con 0x7f2e4015e5e0 2023-10-18T12:01:56.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e401e3ba0 con 0x7f2e4015e5e0 2023-10-18T12:01:56.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e277fe640 1 -- 172.21.15.125:0/188788404 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004860 con 0x7f2e4015e5e0 2023-10-18T12:01:56.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.581+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e401e5aa0 con 0x7f2e4015e5e0 2023-10-18T12:01:56.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.583+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e400675f0 con 0x7f2e4015e5e0 2023-10-18T12:01:56.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.583+0000 7f2e277fe640 1 -- 172.21.15.125:0/188788404 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f2e3800a040 con 0x7f2e4015e5e0 2023-10-18T12:01:56.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.583+0000 7f2e277fe640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2e2c06aff0 0x7f2e2c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.583+0000 7f2e277fe640 1 -- 172.21.15.125:0/188788404 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f2e380acad0 con 0x7f2e4015e5e0 2023-10-18T12:01:56.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.584+0000 7f2e3effd640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2e2c06aff0 0x7f2e2c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.584+0000 7f2e3effd640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2e2c06aff0 0x7f2e2c06d4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000a490 tx=0x7f2e3000a420 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:56.593 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.588+0000 7f2e277fe640 1 -- 172.21.15.125:0/188788404 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e38077bf0 con 0x7f2e4015e5e0 2023-10-18T12:01:56.688 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:56.686+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:56.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.769+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e400690e0 con 0x7f2e4015e5e0 2023-10-18T12:01:56.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.770+0000 7f2e277fe640 1 -- 172.21.15.125:0/188788404 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v258) v1 ==== 74+0+13210 (secure 0 0 0) 0x7f2e38039090 con 0x7f2e4015e5e0 2023-10-18T12:01:56.773 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:56.773 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":258,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:56.066025+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":244,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:01:56.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2e2c06aff0 msgr2=0x7f2e2c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2e2c06aff0 0x7f2e2c06d4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000a490 tx=0x7f2e3000a420 comp rx=0 tx=0).stop 2023-10-18T12:01:56.775 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e4015e5e0 msgr2=0x7f2e4015bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e4015e5e0 0x7f2e4015bd20 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009d60 tx=0x7f2e38035e30 comp rx=0 tx=0).stop 2023-10-18T12:01:56.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 shutdown_connections 2023-10-18T12:01:56.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2e2c06aff0 0x7f2e2c06d4b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e400cf6e0 0x7f2e401e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.776 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2e4015e5e0 0x7f2e4015bd20 unknown :-1 s=CLOSED pgs=2999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 --2- 172.21.15.125:0/188788404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e400b1ca0 0x7f2e40158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 >> 172.21.15.125:0/188788404 conn(0x7f2e400d0c50 msgr2=0x7f2e4006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:56.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 shutdown_connections 2023-10-18T12:01:56.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.772+0000 7f2e456cf640 1 -- 172.21.15.125:0/188788404 wait complete. 2023-10-18T12:01:56.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:01:56.787 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:01:56.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.891+0000 7fb76292c640 1 -- 172.21.15.125:0/2530842338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 msgr2=0x7fb75c0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.891+0000 7fb76292c640 1 --2- 172.21.15.125:0/2530842338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 0x7fb75c0d1910 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005fd0 tx=0x7fb74c02bd90 comp rx=0 tx=0).stop 2023-10-18T12:01:56.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.891+0000 7fb76292c640 1 -- 172.21.15.125:0/2530842338 shutdown_connections 2023-10-18T12:01:56.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.891+0000 7fb76292c640 1 --2- 172.21.15.125:0/2530842338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 0x7fb75c0d1910 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.891+0000 7fb76292c640 1 -- 172.21.15.125:0/2530842338 >> 172.21.15.125:0/2530842338 conn(0x7fb75c0d0d60 msgr2=0x7fb75c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:56.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.891+0000 7fb76292c640 1 -- 172.21.15.125:0/2530842338 shutdown_connections 2023-10-18T12:01:56.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.891+0000 7fb76292c640 1 -- 172.21.15.125:0/2530842338 wait complete. 2023-10-18T12:01:56.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.892+0000 7fb76292c640 1 Processor -- start 2023-10-18T12:01:56.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.892+0000 7fb76292c640 1 -- start start 2023-10-18T12:01:56.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb76292c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb75c0ceb40 0x7fb75c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb76292c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb75c0cf340 0x7fb75c0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.900 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb76292c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 0x7fb75c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb76292c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c15d230 con 0x7fb75c15b2a0 2023-10-18T12:01:56.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb76292c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0b4990 con 0x7fb75c0cf340 2023-10-18T12:01:56.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb76292c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0b4c70 con 0x7fb75c0ceb40 2023-10-18T12:01:56.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 0x7fb75c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.901 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 0x7fb75c14f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57910/0 (socket says 172.21.15.125:57910) 2023-10-18T12:01:56.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 -- 172.21.15.125:0/4069999814 learned_addr learned my addr 172.21.15.125:0/4069999814 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:56.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb75b7fe640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb75c0cf340 0x7fb75c0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb75bfff640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb75c0ceb40 0x7fb75c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 -- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb75c0ceb40 msgr2=0x7fb75c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.902 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb75c0ceb40 0x7fb75c15cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 -- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb75c0cf340 msgr2=0x7fb75c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:56.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb75c0cf340 0x7fb75c0b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:56.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 -- 172.21.15.125:0/4069999814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c009560 con 0x7fb75c15b2a0 2023-10-18T12:01:56.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.893+0000 7fb760ec9640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 0x7fb75c14f150 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c780 tx=0x7fb75400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:56.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.894+0000 7fb758ff9640 1 -- 172.21.15.125:0/4069999814 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754004b60 con 0x7fb75c15b2a0 2023-10-18T12:01:56.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.894+0000 7fb758ff9640 1 -- 172.21.15.125:0/4069999814 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb75400ce90 con 0x7fb75c15b2a0 2023-10-18T12:01:56.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.894+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c15d480 con 0x7fb75c15b2a0 2023-10-18T12:01:56.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.894+0000 7fb758ff9640 1 -- 172.21.15.125:0/4069999814 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75400aab0 con 0x7fb75c15b2a0 2023-10-18T12:01:56.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.894+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c0b46a0 con 0x7fb75c15b2a0 2023-10-18T12:01:56.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.895+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c15d9a0 con 0x7fb75c15b2a0 2023-10-18T12:01:56.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.895+0000 7fb758ff9640 1 -- 172.21.15.125:0/4069999814 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb75400ac10 con 0x7fb75c15b2a0 2023-10-18T12:01:56.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.896+0000 7fb758ff9640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb74806aff0 0x7fb74806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:56.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.896+0000 7fb758ff9640 1 -- 172.21.15.125:0/4069999814 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 5233+0+0 (secure 0 0 0) 0x7fb754089b20 con 0x7fb75c15b2a0 2023-10-18T12:01:56.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.896+0000 7fb75bfff640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb74806aff0 0x7fb74806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:56.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.896+0000 7fb75bfff640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb74806aff0 0x7fb74806d4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb750000f60 tx=0x7fb750009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:56.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:56.900+0000 7fb758ff9640 1 -- 172.21.15.125:0/4069999814 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb754054b50 con 0x7fb75c15b2a0 2023-10-18T12:01:57.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.079+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb75c0d04f0 con 0x7fb75c15b2a0 2023-10-18T12:01:57.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.079+0000 7fb758ff9640 1 -- 172.21.15.125:0/4069999814 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v258) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb754058170 con 0x7fb75c15b2a0 2023-10-18T12:01:57.082 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:01:57.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb74806aff0 msgr2=0x7fb74806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb74806aff0 0x7fb74806d4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb750000f60 tx=0x7fb750009040 comp rx=0 tx=0).stop 2023-10-18T12:01:57.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 msgr2=0x7fb75c14f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 0x7fb75c14f150 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c780 tx=0x7fb75400cc40 comp rx=0 tx=0).stop 2023-10-18T12:01:57.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 shutdown_connections 2023-10-18T12:01:57.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb74806aff0 0x7fb74806d4b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb75c0cf340 0x7fb75c0b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb75c15b2a0 0x7fb75c14f150 unknown :-1 s=CLOSED pgs=3002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 --2- 172.21.15.125:0/4069999814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb75c0ceb40 0x7fb75c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 >> 172.21.15.125:0/4069999814 conn(0x7fb75c0d0d60 msgr2=0x7fb75c15a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:57.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 shutdown_connections 2023-10-18T12:01:57.086 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.081+0000 7fb76292c640 1 -- 172.21.15.125:0/4069999814 wait complete. 2023-10-18T12:01:57.095 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T12:01:57.195 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:01:57.198 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:01:57.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.199+0000 7ff5bead7640 1 -- 172.21.15.125:0/323127342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 msgr2=0x7ff5b806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.199+0000 7ff5bead7640 1 --2- 172.21.15.125:0/323127342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 0x7ff5b806bb60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009830 tx=0x7ff5ac02bcd0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.199+0000 7ff5bead7640 1 -- 172.21.15.125:0/323127342 shutdown_connections 2023-10-18T12:01:57.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.199+0000 7ff5bead7640 1 --2- 172.21.15.125:0/323127342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8070af0 0x7ff5b80affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.199+0000 7ff5bead7640 1 --2- 172.21.15.125:0/323127342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 0x7ff5b806bb60 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.199+0000 7ff5bead7640 1 -- 172.21.15.125:0/323127342 >> 172.21.15.125:0/323127342 conn(0x7ff5b8069930 msgr2=0x7ff5b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:57.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.199+0000 7ff5bead7640 1 -- 172.21.15.125:0/323127342 shutdown_connections 2023-10-18T12:01:57.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.199+0000 7ff5bead7640 1 -- 172.21.15.125:0/323127342 wait complete. 2023-10-18T12:01:57.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.200+0000 7ff5bead7640 1 Processor -- start 2023-10-18T12:01:57.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.200+0000 7ff5bead7640 1 -- start start 2023-10-18T12:01:57.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bead7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8070af0 0x7ff5b8152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:57.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bead7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 0x7ff5b813ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:57.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bead7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8141690 0x7ff5b81e2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:57.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bead7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b8154610 con 0x7ff5b8141690 2023-10-18T12:01:57.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bead7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b8154180 con 0x7ff5b813e520 2023-10-18T12:01:57.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bead7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b813b1a0 con 0x7ff5b8070af0 2023-10-18T12:01:57.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8141690 0x7ff5b81e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:57.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bc873640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8070af0 0x7ff5b8152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:57.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8141690 0x7ff5b81e2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57920/0 (socket says 172.21.15.125:57920) 2023-10-18T12:01:57.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bc873640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8070af0 0x7ff5b8152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:53506/0 (socket says 172.21.15.125:53506) 2023-10-18T12:01:57.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 0x7ff5b813ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:57.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 -- 172.21.15.125:0/2303600206 learned_addr learned my addr 172.21.15.125:0/2303600206 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:57.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 -- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8070af0 msgr2=0x7ff5b8152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8070af0 0x7ff5b8152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 -- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 msgr2=0x7ff5b813ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 0x7ff5b813ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 -- 172.21.15.125:0/2303600206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0009840 con 0x7ff5b8141690 2023-10-18T12:01:57.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bc873640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8070af0 0x7ff5b8152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:57.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5b7fff640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 0x7ff5b813ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:01:57.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.201+0000 7ff5bd074640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8141690 0x7ff5b81e2420 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009f10 tx=0x7ff5b000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:57.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.202+0000 7ff5b57fa640 1 -- 172.21.15.125:0/2303600206 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0013070 con 0x7ff5b8141690 2023-10-18T12:01:57.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.202+0000 7ff5b57fa640 1 -- 172.21.15.125:0/2303600206 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5b0017450 con 0x7ff5b8141690 2023-10-18T12:01:57.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.202+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac009490 con 0x7ff5b8141690 2023-10-18T12:01:57.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.202+0000 7ff5b57fa640 1 -- 172.21.15.125:0/2303600206 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0017800 con 0x7ff5b8141690 2023-10-18T12:01:57.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.202+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b8153850 con 0x7ff5b8141690 2023-10-18T12:01:57.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.203+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b8154330 con 0x7ff5b8141690 2023-10-18T12:01:57.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.203+0000 7ff5b57fa640 1 -- 172.21.15.125:0/2303600206 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7ff5b0021440 con 0x7ff5b8141690 2023-10-18T12:01:57.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.204+0000 7ff5b57fa640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff59c06aff0 0x7ff59c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:57.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.204+0000 7ff5b57fa640 1 -- 172.21.15.125:0/2303600206 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 5233+0+0 (secure 0 0 0) 0x7ff5b008bf20 con 0x7ff5b8141690 2023-10-18T12:01:57.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.204+0000 7ff5bc873640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff59c06aff0 0x7ff59c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:57.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.204+0000 7ff5bc873640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff59c06aff0 0x7ff59c06d4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac036000 tx=0x7ff5ac004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:57.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.208+0000 7ff5b57fa640 1 -- 172.21.15.125:0/2303600206 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5b8154330 con 0x7ff5b8141690 2023-10-18T12:01:57.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.391+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff5b81542f0 con 0x7ff5b8141690 2023-10-18T12:01:57.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.392+0000 7ff5b57fa640 1 -- 172.21.15.125:0/2303600206 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v258) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff5b001c020 con 0x7ff5b8141690 2023-10-18T12:01:57.395 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:01:57.395 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T12:01:57.395 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:01:57.396 INFO:teuthology.orchestra.run.smithi125.stdout: "poolnum": 1, 2023-10-18T12:01:57.396 INFO:teuthology.orchestra.run.smithi125.stdout: "poolname": ".mgr" 2023-10-18T12:01:57.396 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:01:57.396 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T12:01:57.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff59c06aff0 msgr2=0x7ff59c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff59c06aff0 0x7ff59c06d4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac036000 tx=0x7ff5ac004730 comp rx=0 tx=0).stop 2023-10-18T12:01:57.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8141690 msgr2=0x7ff5b81e2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8141690 0x7ff5b81e2420 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009f10 tx=0x7ff5b000b690 comp rx=0 tx=0).stop 2023-10-18T12:01:57.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 shutdown_connections 2023-10-18T12:01:57.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7ff59c06aff0 0x7ff59c06d4b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5b813e520 0x7ff5b813ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff5b8141690 0x7ff5b81e2420 unknown :-1 s=CLOSED pgs=3003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 --2- 172.21.15.125:0/2303600206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5b8070af0 0x7ff5b8152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.393+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 >> 172.21.15.125:0/2303600206 conn(0x7ff5b8069930 msgr2=0x7ff5b80af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:57.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.394+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 shutdown_connections 2023-10-18T12:01:57.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.394+0000 7ff5bead7640 1 -- 172.21.15.125:0/2303600206 wait complete. 2023-10-18T12:01:57.408 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T12:01:57.408 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T12:01:57.437 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:57.434+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:57.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.516+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/656522385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f4c003970 msgr2=0x7f2f4c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.516+0000 7f2f6a3a9640 1 --2- 172.21.15.125:0/656522385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f4c003970 0x7f2f4c003d90 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400ccf0 tx=0x7f2f4c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.516+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/656522385 shutdown_connections 2023-10-18T12:01:57.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.516+0000 7f2f6a3a9640 1 --2- 172.21.15.125:0/656522385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f4c003970 0x7f2f4c003d90 unknown :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.516+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/656522385 >> 172.21.15.125:0/656522385 conn(0x7f2f64069930 msgr2=0x7f2f64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:57.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.516+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/656522385 shutdown_connections 2023-10-18T12:01:57.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.516+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/656522385 wait complete. 2023-10-18T12:01:57.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.517+0000 7f2f6a3a9640 1 Processor -- start 2023-10-18T12:01:57.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.517+0000 7f2f6a3a9640 1 -- start start 2023-10-18T12:01:57.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f6a3a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f4c003970 0x7f2f641483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:57.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f6a3a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f64138b60 0x7f2f6413b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:57.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f6a3a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f6413ba50 0x7f2f6413a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:57.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f6a3a9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f641478d0 con 0x7f2f64138b60 2023-10-18T12:01:57.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f6a3a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6413bed0 con 0x7f2f4c003970 2023-10-18T12:01:57.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f6a3a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f641395b0 con 0x7f2f6413ba50 2023-10-18T12:01:57.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f64138b60 0x7f2f6413b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:57.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f64138b60 0x7f2f6413b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55944/0 (socket says 172.21.15.125:55944) 2023-10-18T12:01:57.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 -- 172.21.15.125:0/1958713013 learned_addr learned my addr 172.21.15.125:0/1958713013 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:57.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f63fff640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f4c003970 0x7f2f641483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:57.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f68946640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f6413ba50 0x7f2f6413a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:57.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 -- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f6413ba50 msgr2=0x7f2f6413a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f6413ba50 0x7f2f6413a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 -- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f4c003970 msgr2=0x7f2f641483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:57.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f4c003970 0x7f2f641483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:57.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 -- 172.21.15.125:0/1958713013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64139890 con 0x7f2f64138b60 2023-10-18T12:01:57.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f637fe640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f64138b60 0x7f2f6413b510 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e570 tx=0x7f2f5800ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:57.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.518+0000 7f2f60ff9640 1 -- 172.21.15.125:0/1958713013 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58010890 con 0x7f2f64138b60 2023-10-18T12:01:57.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.519+0000 7f2f60ff9640 1 -- 172.21.15.125:0/1958713013 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2f58010ae0 con 0x7f2f64138b60 2023-10-18T12:01:57.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.519+0000 7f2f60ff9640 1 -- 172.21.15.125:0/1958713013 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58010e50 con 0x7f2f64138b60 2023-10-18T12:01:57.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.519+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f64139b70 con 0x7f2f64138b60 2023-10-18T12:01:57.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.519+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6413a130 con 0x7f2f64138b60 2023-10-18T12:01:57.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.519+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6413a320 con 0x7f2f64138b60 2023-10-18T12:01:57.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.520+0000 7f2f60ff9640 1 -- 172.21.15.125:0/1958713013 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f2f58011460 con 0x7f2f64138b60 2023-10-18T12:01:57.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.520+0000 7f2f60ff9640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2f5006b020 0x7f2f5006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:57.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.520+0000 7f2f60ff9640 1 -- 172.21.15.125:0/1958713013 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 5233+0+0 (secure 0 0 0) 0x7f2f58089ec0 con 0x7f2f64138b60 2023-10-18T12:01:57.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.521+0000 7f2f63fff640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2f5006b020 0x7f2f5006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:57.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.521+0000 7f2f63fff640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2f5006b020 0x7f2f5006d4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f54037000 tx=0x7f2f540046c0 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:57.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.524+0000 7f2f60ff9640 1 -- 172.21.15.125:0/1958713013 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f58054e40 con 0x7f2f64138b60 2023-10-18T12:01:57.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:57.694+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2f640690e0 con 0x7f2f64138b60 2023-10-18T12:01:58.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.054+0000 7f2f60ff9640 1 -- 172.21.15.125:0/1958713013 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v259) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2f5801f020 con 0x7f2f64138b60 2023-10-18T12:01:58.058 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' created 2023-10-18T12:01:58.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2f5006b020 msgr2=0x7f2f5006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:58.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2f5006b020 0x7f2f5006d4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f54037000 tx=0x7f2f540046c0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f64138b60 msgr2=0x7f2f6413b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:58.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f64138b60 0x7f2f6413b510 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e570 tx=0x7f2f5800ea30 comp rx=0 tx=0).stop 2023-10-18T12:01:58.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 shutdown_connections 2023-10-18T12:01:58.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f6413ba50 0x7f2f6413a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f64138b60 0x7f2f6413b510 unknown :-1 s=CLOSED pgs=3006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f2f5006b020 0x7f2f5006d4e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 --2- 172.21.15.125:0/1958713013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f4c003970 0x7f2f641483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 >> 172.21.15.125:0/1958713013 conn(0x7f2f64069930 msgr2=0x7f2f6406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:58.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 shutdown_connections 2023-10-18T12:01:58.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.057+0000 7f2f6a3a9640 1 -- 172.21.15.125:0/1958713013 wait complete. 2023-10-18T12:01:58.073 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T12:01:58.172 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:58.169+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:58.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.181+0000 7f319900e640 1 -- 172.21.15.125:0/3705767118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31940ceb40 msgr2=0x7f31940b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:58.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.181+0000 7f319900e640 1 --2- 172.21.15.125:0/3705767118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31940ceb40 0x7f31940b0650 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7f319408a120 tx=0x7f318c02bdb0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.181+0000 7f319900e640 1 -- 172.21.15.125:0/3705767118 shutdown_connections 2023-10-18T12:01:58.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.181+0000 7f319900e640 1 --2- 172.21.15.125:0/3705767118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31940ceb40 0x7f31940b0650 unknown :-1 s=CLOSED pgs=3007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.181+0000 7f319900e640 1 --2- 172.21.15.125:0/3705767118 >> v2:172.21.15.139:3300/0 conn(0x7f319415b2a0 0x7f31940d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.181+0000 7f319900e640 1 -- 172.21.15.125:0/3705767118 >> 172.21.15.125:0/3705767118 conn(0x7f31940d0d60 msgr2=0x7f31940d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:58.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.181+0000 7f319900e640 1 -- 172.21.15.125:0/3705767118 shutdown_connections 2023-10-18T12:01:58.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.181+0000 7f319900e640 1 -- 172.21.15.125:0/3705767118 wait complete. 2023-10-18T12:01:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.182+0000 7f319900e640 1 Processor -- start 2023-10-18T12:01:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319900e640 1 -- start start 2023-10-18T12:01:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319900e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31940ceb40 0x7f3194151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319900e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31940cf340 0x7f31940c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:58.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319900e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f319415b2a0 0x7f31941e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319900e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c7ed0 con 0x7f319415b2a0 2023-10-18T12:01:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319900e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31941516d0 con 0x7f31940cf340 2023-10-18T12:01:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319900e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c8140 con 0x7f31940ceb40 2023-10-18T12:01:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f319415b2a0 0x7f31941e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:58.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f319415b2a0 0x7f31941e5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55958/0 (socket says 172.21.15.125:55958) 2023-10-18T12:01:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31940cf340 0x7f31940c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f319359e640 1 -- 172.21.15.125:0/3624046627 learned_addr learned my addr 172.21.15.125:0/3624046627 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.183+0000 7f3192d9d640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31940ceb40 0x7f3194151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f319359e640 1 -- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31940ceb40 msgr2=0x7f3194151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:58.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f319359e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31940ceb40 0x7f3194151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f319359e640 1 -- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31940cf340 msgr2=0x7f31940c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:58.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f319359e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31940cf340 0x7f31940c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:58.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f319359e640 1 -- 172.21.15.125:0/3624046627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c009560 con 0x7f319415b2a0 2023-10-18T12:01:58.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f319359e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f319415b2a0 0x7f31941e5880 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7f318c036040 tx=0x7f318c02da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:58.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f317b7fe640 1 -- 172.21.15.125:0/3624046627 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c003800 con 0x7f319415b2a0 2023-10-18T12:01:58.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f317b7fe640 1 -- 172.21.15.125:0/3624046627 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f318c003a90 con 0x7f319415b2a0 2023-10-18T12:01:58.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31940c8420 con 0x7f319415b2a0 2023-10-18T12:01:58.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f317b7fe640 1 -- 172.21.15.125:0/3624046627 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c003ea0 con 0x7f319415b2a0 2023-10-18T12:01:58.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.184+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31940c89e0 con 0x7f319415b2a0 2023-10-18T12:01:58.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.185+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31940690e0 con 0x7f319415b2a0 2023-10-18T12:01:58.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.186+0000 7f317b7fe640 1 -- 172.21.15.125:0/3624046627 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f318c03e450 con 0x7f319415b2a0 2023-10-18T12:01:58.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.186+0000 7f317b7fe640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f318006b020 0x7f318006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:58.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.186+0000 7f317b7fe640 1 -- 172.21.15.125:0/3624046627 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f318c0ac6b0 con 0x7f319415b2a0 2023-10-18T12:01:58.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.186+0000 7f3192d9d640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f318006b020 0x7f318006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:58.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.187+0000 7f3192d9d640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f318006b020 0x7f318006d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f31940c96b0 tx=0x7f3184009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:58.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.190+0000 7f317b7fe640 1 -- 172.21.15.125:0/3624046627 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f318c077680 con 0x7f319415b2a0 2023-10-18T12:01:58.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:58.362+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f31940b0d00 con 0x7f319415b2a0 2023-10-18T12:01:58.907 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:58.904+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:01:59.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.056+0000 7f317b7fe640 1 -- 172.21.15.125:0/3624046627 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v260) v1 ==== 170+0+0 (secure 0 0 0) 0x7f318c07aca0 con 0x7f319415b2a0 2023-10-18T12:01:59.059 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' created 2023-10-18T12:01:59.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f318006b020 msgr2=0x7f318006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:59.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f318006b020 0x7f318006d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f31940c96b0 tx=0x7f3184009040 comp rx=0 tx=0).stop 2023-10-18T12:01:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f319415b2a0 msgr2=0x7f31941e5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f319415b2a0 0x7f31941e5880 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7f318c036040 tx=0x7f318c02da10 comp rx=0 tx=0).stop 2023-10-18T12:01:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 shutdown_connections 2023-10-18T12:01:59.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f318006b020 0x7f318006d4e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31940cf340 0x7f31940c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f319415b2a0 0x7f31941e5880 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 --2- 172.21.15.125:0/3624046627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31940ceb40 0x7f3194151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 >> 172.21.15.125:0/3624046627 conn(0x7f31940d0d60 msgr2=0x7f319415a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:59.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 shutdown_connections 2023-10-18T12:01:59.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.058+0000 7f319900e640 1 -- 172.21.15.125:0/3624046627 wait complete. 2023-10-18T12:01:59.080 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T12:01:59.187 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.184+0000 7fc89bc0e640 1 -- 172.21.15.125:0/3489775365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc89413d450 msgr2=0x7fc8941585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:59.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.184+0000 7fc89bc0e640 1 --2- 172.21.15.125:0/3489775365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc89413d450 0x7fc8941585b0 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7fc890005fd0 tx=0x7fc89002c970 comp rx=0 tx=0).stop 2023-10-18T12:01:59.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.184+0000 7fc89bc0e640 1 -- 172.21.15.125:0/3489775365 shutdown_connections 2023-10-18T12:01:59.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.184+0000 7fc89bc0e640 1 --2- 172.21.15.125:0/3489775365 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc89413d450 0x7fc8941585b0 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:59.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.184+0000 7fc89bc0e640 1 -- 172.21.15.125:0/3489775365 >> 172.21.15.125:0/3489775365 conn(0x7fc894069930 msgr2=0x7fc894069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:01:59.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.184+0000 7fc89bc0e640 1 -- 172.21.15.125:0/3489775365 shutdown_connections 2023-10-18T12:01:59.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.184+0000 7fc89bc0e640 1 -- 172.21.15.125:0/3489775365 wait complete. 2023-10-18T12:01:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.185+0000 7fc89bc0e640 1 Processor -- start 2023-10-18T12:01:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.185+0000 7fc89bc0e640 1 -- start start 2023-10-18T12:01:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc89bc0e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8940c9770 0x7fc8941f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:59.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc89bc0e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc89413d450 0x7fc8940abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc89bc0e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc894158af0 0x7fc89413e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc89bc0e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc894139510 con 0x7fc8940c9770 2023-10-18T12:01:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc89bc0e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc894138900 con 0x7fc89413d450 2023-10-18T12:01:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc89bc0e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc894138be0 con 0x7fc894158af0 2023-10-18T12:01:59.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8999aa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8940c9770 0x7fc8941f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:59.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8999aa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8940c9770 0x7fc8941f0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55968/0 (socket says 172.21.15.125:55968) 2023-10-18T12:01:59.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8999aa640 1 -- 172.21.15.125:0/1557000496 learned_addr learned my addr 172.21.15.125:0/1557000496 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:01:59.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8991a9640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc89413d450 0x7fc8940abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:59.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc89a1ab640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc894158af0 0x7fc89413e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8999aa640 1 -- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc894158af0 msgr2=0x7fc89413e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8999aa640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc894158af0 0x7fc89413e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8999aa640 1 -- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc89413d450 msgr2=0x7fc8940abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:01:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8999aa640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc89413d450 0x7fc8940abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:01:59.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.186+0000 7fc8999aa640 1 -- 172.21.15.125:0/1557000496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc890009840 con 0x7fc8940c9770 2023-10-18T12:01:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.187+0000 7fc8999aa640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8940c9770 0x7fc8941f0e10 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7fc890009b30 tx=0x7fc890031f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.187+0000 7fc88a7fc640 1 -- 172.21.15.125:0/1557000496 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89003e070 con 0x7fc8940c9770 2023-10-18T12:01:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.187+0000 7fc88a7fc640 1 -- 172.21.15.125:0/1557000496 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc890003c40 con 0x7fc8940c9770 2023-10-18T12:01:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.187+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc894138ec0 con 0x7fc8940c9770 2023-10-18T12:01:59.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.187+0000 7fc88a7fc640 1 -- 172.21.15.125:0/1557000496 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89000b360 con 0x7fc8940c9770 2023-10-18T12:01:59.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.187+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8941f1350 con 0x7fc8940c9770 2023-10-18T12:01:59.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.188+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8940690e0 con 0x7fc8940c9770 2023-10-18T12:01:59.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.188+0000 7fc88a7fc640 1 -- 172.21.15.125:0/1557000496 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fc89000b4c0 con 0x7fc8940c9770 2023-10-18T12:01:59.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.189+0000 7fc88a7fc640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fc87806aff0 0x7fc87806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:01:59.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.189+0000 7fc88a7fc640 1 -- 172.21.15.125:0/1557000496 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 5985+0+0 (secure 0 0 0) 0x7fc8900ac100 con 0x7fc8940c9770 2023-10-18T12:01:59.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.189+0000 7fc8991a9640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fc87806aff0 0x7fc87806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:01:59.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.189+0000 7fc8991a9640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fc87806aff0 0x7fc87806d4b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc894065500 tx=0x7fc884009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:01:59.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.193+0000 7fc88a7fc640 1 -- 172.21.15.125:0/1557000496 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc890076de0 con 0x7fc8940c9770 2023-10-18T12:01:59.407 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:01:59.404+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc894139650 con 0x7fc8940c9770 2023-10-18T12:01:59.659 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:01:59.656+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:00.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.395+0000 7fc88a7fc640 1 -- 172.21.15.125:0/1557000496 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 35 and data pool 36 v110) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc894139650 con 0x7fc8940c9770 2023-10-18T12:02:00.398 INFO:teuthology.orchestra.run.smithi125.stderr:new fs with metadata pool 35 and data pool 36 2023-10-18T12:02:00.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fc87806aff0 msgr2=0x7fc87806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fc87806aff0 0x7fc87806d4b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc894065500 tx=0x7fc884009040 comp rx=0 tx=0).stop 2023-10-18T12:02:00.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8940c9770 msgr2=0x7fc8941f0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8940c9770 0x7fc8941f0e10 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7fc890009b30 tx=0x7fc890031f80 comp rx=0 tx=0).stop 2023-10-18T12:02:00.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 shutdown_connections 2023-10-18T12:02:00.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fc87806aff0 0x7fc87806d4b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc89413d450 0x7fc8940abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc894158af0 0x7fc89413e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 --2- 172.21.15.125:0/1557000496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8940c9770 0x7fc8941f0e10 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 >> 172.21.15.125:0/1557000496 conn(0x7fc894069930 msgr2=0x7fc89406f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:00.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 shutdown_connections 2023-10-18T12:02:00.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.398+0000 7fc89bc0e640 1 -- 172.21.15.125:0/1557000496 wait complete. 2023-10-18T12:02:00.415 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:02:00.468 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:00.466+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:00.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.517+0000 7fec47c10640 1 -- 172.21.15.125:0/3612525641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec4015b2a0 msgr2=0x7fec400d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.517+0000 7fec47c10640 1 --2- 172.21.15.125:0/3612525641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec4015b2a0 0x7fec400d1910 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fec30009830 tx=0x7fec3002bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.517+0000 7fec47c10640 1 -- 172.21.15.125:0/3612525641 shutdown_connections 2023-10-18T12:02:00.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.517+0000 7fec47c10640 1 --2- 172.21.15.125:0/3612525641 >> v2:172.21.15.139:3300/0 conn(0x7fec400ceb40 0x7fec400b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.517+0000 7fec47c10640 1 --2- 172.21.15.125:0/3612525641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec4015b2a0 0x7fec400d1910 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.517+0000 7fec47c10640 1 -- 172.21.15.125:0/3612525641 >> 172.21.15.125:0/3612525641 conn(0x7fec400d0d60 msgr2=0x7fec400d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:00.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.517+0000 7fec47c10640 1 -- 172.21.15.125:0/3612525641 shutdown_connections 2023-10-18T12:02:00.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.517+0000 7fec47c10640 1 -- 172.21.15.125:0/3612525641 wait complete. 2023-10-18T12:02:00.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.518+0000 7fec47c10640 1 Processor -- start 2023-10-18T12:02:00.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.518+0000 7fec47c10640 1 -- start start 2023-10-18T12:02:00.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec47c10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec400ceb40 0x7fec4015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:00.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec47c10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec400cf340 0x7fec400c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:00.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec47c10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec4015b2a0 0x7fec400c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:00.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec47c10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400c9420 con 0x7fec4015b2a0 2023-10-18T12:02:00.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec47c10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400c8810 con 0x7fec400ceb40 2023-10-18T12:02:00.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec47c10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec400c8af0 con 0x7fec400cf340 2023-10-18T12:02:00.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec461ad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec4015b2a0 0x7fec400c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:00.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec451ab640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec400cf340 0x7fec400c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:00.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec459ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec400ceb40 0x7fec4015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:00.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec451ab640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec400cf340 0x7fec400c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:43258/0 (socket says 172.21.15.125:43258) 2023-10-18T12:02:00.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec451ab640 1 -- 172.21.15.125:0/1323680320 learned_addr learned my addr 172.21.15.125:0/1323680320 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:00.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.519+0000 7fec461ad640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec4015b2a0 0x7fec400c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55978/0 (socket says 172.21.15.125:55978) 2023-10-18T12:02:00.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec459ac640 1 -- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec400cf340 msgr2=0x7fec400c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec459ac640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec400cf340 0x7fec400c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec459ac640 1 -- 172.21.15.125:0/1323680320 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec4015b2a0 msgr2=0x7fec400c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec459ac640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec4015b2a0 0x7fec400c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec459ac640 1 -- 172.21.15.125:0/1323680320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec38009840 con 0x7fec400ceb40 2023-10-18T12:02:00.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec451ab640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec400cf340 0x7fec400c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:02:00.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec461ad640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec4015b2a0 0x7fec400c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:02:00.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec459ac640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec400ceb40 0x7fec4015cf30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c3c0 tx=0x7fec3c00c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:00.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec367fc640 1 -- 172.21.15.125:0/1323680320 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c004440 con 0x7fec400ceb40 2023-10-18T12:02:00.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec367fc640 1 -- 172.21.15.125:0/1323680320 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec3c0035f0 con 0x7fec400ceb40 2023-10-18T12:02:00.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec30009490 con 0x7fec400ceb40 2023-10-18T12:02:00.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec367fc640 1 -- 172.21.15.125:0/1323680320 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c003a80 con 0x7fec400ceb40 2023-10-18T12:02:00.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.520+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec400c90b0 con 0x7fec400ceb40 2023-10-18T12:02:00.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.521+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec400690e0 con 0x7fec400ceb40 2023-10-18T12:02:00.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.521+0000 7fec367fc640 1 -- 172.21.15.125:0/1323680320 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fec3c003be0 con 0x7fec400ceb40 2023-10-18T12:02:00.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.522+0000 7fec367fc640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fec2406b020 0x7fec2406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:00.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.522+0000 7fec367fc640 1 -- 172.21.15.125:0/1323680320 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fec3c0895a0 con 0x7fec400ceb40 2023-10-18T12:02:00.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.522+0000 7fec451ab640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fec2406b020 0x7fec2406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:00.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.522+0000 7fec451ab640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fec2406b020 0x7fec2406d4e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec400c7590 tx=0x7fec30004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:00.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.526+0000 7fec367fc640 1 -- 172.21.15.125:0/1323680320 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec3c054280 con 0x7fec400ceb40 2023-10-18T12:02:00.703 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.700+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec400d0b40 con 0x7fec400ceb40 2023-10-18T12:02:00.704 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.701+0000 7fec367fc640 1 -- 172.21.15.125:0/1323680320 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v261) v1 ==== 74+0+16871 (secure 0 0 0) 0x7fec3c0578a0 con 0x7fec400ceb40 2023-10-18T12:02:00.704 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:00.705 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":261,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:59.405931+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:57.717906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:58.385232+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:02:00.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fec2406b020 msgr2=0x7fec2406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.706 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fec2406b020 0x7fec2406d4e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec400c7590 tx=0x7fec30004730 comp rx=0 tx=0).stop 2023-10-18T12:02:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec400ceb40 msgr2=0x7fec4015cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec400ceb40 0x7fec4015cf30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c3c0 tx=0x7fec3c00c7c0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 shutdown_connections 2023-10-18T12:02:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec4015b2a0 0x7fec400c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.707 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec400cf340 0x7fec400c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fec2406b020 0x7fec2406d4e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 --2- 172.21.15.125:0/1323680320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec400ceb40 0x7fec4015cf30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 >> 172.21.15.125:0/1323680320 conn(0x7fec400d0d60 msgr2=0x7fec4006b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 shutdown_connections 2023-10-18T12:02:00.708 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.703+0000 7fec47c10640 1 -- 172.21.15.125:0/1323680320 wait complete. 2023-10-18T12:02:00.717 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:02:00.822 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.819+0000 7f38dfbfc640 1 -- 172.21.15.125:0/1801013301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 msgr2=0x7f38d806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.819+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/1801013301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 0x7f38d806bb60 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005fd0 tx=0x7f38c802bd90 comp rx=0 tx=0).stop 2023-10-18T12:02:00.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.819+0000 7f38dfbfc640 1 -- 172.21.15.125:0/1801013301 shutdown_connections 2023-10-18T12:02:00.823 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.819+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/1801013301 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38d8070af0 0x7f38d80affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.819+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/1801013301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 0x7f38d806bb60 unknown :-1 s=CLOSED pgs=3012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.819+0000 7f38dfbfc640 1 -- 172.21.15.125:0/1801013301 >> 172.21.15.125:0/1801013301 conn(0x7f38d8069930 msgr2=0x7f38d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:00.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.819+0000 7f38dfbfc640 1 -- 172.21.15.125:0/1801013301 shutdown_connections 2023-10-18T12:02:00.824 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.819+0000 7f38dfbfc640 1 -- 172.21.15.125:0/1801013301 wait complete. 2023-10-18T12:02:00.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.820+0000 7f38dfbfc640 1 Processor -- start 2023-10-18T12:02:00.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dfbfc640 1 -- start start 2023-10-18T12:02:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dfbfc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38d8070af0 0x7f38d80a7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dfbfc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 0x7f38d80c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dfbfc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38d8141690 0x7f38d80ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dfbfc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d81d1190 con 0x7f38d813e520 2023-10-18T12:02:00.828 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dfbfc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80a7670 con 0x7f38d8070af0 2023-10-18T12:02:00.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dfbfc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d81d0eb0 con 0x7f38d8141690 2023-10-18T12:02:00.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dd197640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 0x7f38d80c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:00.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dd197640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 0x7f38d80c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:55996/0 (socket says 172.21.15.125:55996) 2023-10-18T12:02:00.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dd197640 1 -- 172.21.15.125:0/2294446087 learned_addr learned my addr 172.21.15.125:0/2294446087 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:00.829 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38dd998640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38d8070af0 0x7f38d80a7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:00.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.821+0000 7f38de199640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38d8141690 0x7f38d80ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:00.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38dd197640 1 -- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38d8141690 msgr2=0x7f38d80ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38dd197640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38d8141690 0x7f38d80ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.830 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38dd197640 1 -- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38d8070af0 msgr2=0x7f38d80a7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38dd197640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38d8070af0 0x7f38d80a7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38dd197640 1 -- 172.21.15.125:0/2294446087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8009560 con 0x7f38d813e520 2023-10-18T12:02:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38dd197640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 0x7f38d80c8220 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c430 tx=0x7f38d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38ce7fc640 1 -- 172.21.15.125:0/2294446087 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40052d0 con 0x7f38d813e520 2023-10-18T12:02:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38ce7fc640 1 -- 172.21.15.125:0/2294446087 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f38d40035f0 con 0x7f38d813e520 2023-10-18T12:02:00.831 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d80a6650 con 0x7f38d813e520 2023-10-18T12:02:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38ce7fc640 1 -- 172.21.15.125:0/2294446087 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4003a00 con 0x7f38d813e520 2023-10-18T12:02:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.822+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d80a6c10 con 0x7f38d813e520 2023-10-18T12:02:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.823+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d80a64b0 con 0x7f38d813e520 2023-10-18T12:02:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.824+0000 7f38ce7fc640 1 -- 172.21.15.125:0/2294446087 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f38d4003b60 con 0x7f38d813e520 2023-10-18T12:02:00.832 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.824+0000 7f38ce7fc640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f38bc06aff0 0x7f38bc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:00.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.824+0000 7f38ce7fc640 1 -- 172.21.15.125:0/2294446087 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f38d4089140 con 0x7f38d813e520 2023-10-18T12:02:00.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.824+0000 7f38dd998640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f38bc06aff0 0x7f38bc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:00.833 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.824+0000 7f38dd998640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f38bc06aff0 0x7f38bc06d4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38c8035000 tx=0x7f38c8004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:00.834 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:00.828+0000 7f38ce7fc640 1 -- 172.21.15.125:0/2294446087 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38d80a64b0 con 0x7f38d813e520 2023-10-18T12:02:01.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.002+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38d80a64b0 con 0x7f38d813e520 2023-10-18T12:02:01.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.003+0000 7f38ce7fc640 1 -- 172.21.15.125:0/2294446087 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v261) v1 ==== 74+0+16871 (secure 0 0 0) 0x7f38d80a64b0 con 0x7f38d813e520 2023-10-18T12:02:01.006 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:01.006 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":261,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:01:59.405931+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:57.717906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:58.385232+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:02:01.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f38bc06aff0 msgr2=0x7f38bc06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f38bc06aff0 0x7f38bc06d4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38c8035000 tx=0x7f38c8004730 comp rx=0 tx=0).stop 2023-10-18T12:02:01.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 msgr2=0x7f38d80c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 0x7f38d80c8220 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c430 tx=0x7f38d400c8f0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 shutdown_connections 2023-10-18T12:02:01.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f38d8141690 0x7f38d80ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38d813e520 0x7f38d80c8220 unknown :-1 s=CLOSED pgs=3013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f38bc06aff0 0x7f38bc06d4b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 --2- 172.21.15.125:0/2294446087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38d8070af0 0x7f38d80a7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 >> 172.21.15.125:0/2294446087 conn(0x7f38d8069930 msgr2=0x7f38d80b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:01.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 shutdown_connections 2023-10-18T12:02:01.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.005+0000 7f38dfbfc640 1 -- 172.21.15.125:0/2294446087 wait complete. 2023-10-18T12:02:01.019 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T12:02:01.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.127+0000 7fef5400e640 1 -- 172.21.15.125:0/1278717627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef4c141690 msgr2=0x7fef4c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.127+0000 7fef5400e640 1 --2- 172.21.15.125:0/1278717627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef4c141690 0x7fef4c0703e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009860 tx=0x7fef3c02bd60 comp rx=0 tx=0).stop 2023-10-18T12:02:01.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.127+0000 7fef5400e640 1 -- 172.21.15.125:0/1278717627 shutdown_connections 2023-10-18T12:02:01.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.127+0000 7fef5400e640 1 --2- 172.21.15.125:0/1278717627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef4c141690 0x7fef4c0703e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.127+0000 7fef5400e640 1 -- 172.21.15.125:0/1278717627 >> 172.21.15.125:0/1278717627 conn(0x7fef4c069930 msgr2=0x7fef4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:01.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.127+0000 7fef5400e640 1 -- 172.21.15.125:0/1278717627 shutdown_connections 2023-10-18T12:02:01.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.127+0000 7fef5400e640 1 -- 172.21.15.125:0/1278717627 wait complete. 2023-10-18T12:02:01.133 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.128+0000 7fef5400e640 1 Processor -- start 2023-10-18T12:02:01.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.128+0000 7fef5400e640 1 -- start start 2023-10-18T12:02:01.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.128+0000 7fef5400e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c070920 0x7fef4c13b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:01.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.128+0000 7fef5400e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef4c13e520 0x7fef4c1f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:01.134 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.128+0000 7fef5400e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef4c141690 0x7fef4c1df6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:01.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.128+0000 7fef5400e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1e2af0 con 0x7fef4c13e520 2023-10-18T12:02:01.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.128+0000 7fef5400e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1f2270 con 0x7fef4c070920 2023-10-18T12:02:01.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.128+0000 7fef5400e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1e1ee0 con 0x7fef4c141690 2023-10-18T12:02:01.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef4c13e520 0x7fef4c1f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:01.135 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef4c13e520 0x7fef4c1f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56006/0 (socket says 172.21.15.125:56006) 2023-10-18T12:02:01.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 -- 172.21.15.125:0/2746604440 learned_addr learned my addr 172.21.15.125:0/2746604440 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:01.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef51daa640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c070920 0x7fef4c13b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:01.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef525ab640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef4c141690 0x7fef4c1df6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:01.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 -- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef4c141690 msgr2=0x7fef4c1df6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.136 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef4c141690 0x7fef4c1df6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 -- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c070920 msgr2=0x7fef4c13b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c070920 0x7fef4c13b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 -- 172.21.15.125:0/2746604440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c009490 con 0x7fef4c13e520 2023-10-18T12:02:01.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef515a9640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef4c13e520 0x7fef4c1f1d30 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009980 tx=0x7fef3c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:01.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef427fc640 1 -- 172.21.15.125:0/2746604440 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c003ea0 con 0x7fef4c13e520 2023-10-18T12:02:01.137 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef427fc640 1 -- 172.21.15.125:0/2746604440 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef3c034440 con 0x7fef4c13e520 2023-10-18T12:02:01.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c1e21c0 con 0x7fef4c13e520 2023-10-18T12:02:01.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef427fc640 1 -- 172.21.15.125:0/2746604440 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c034850 con 0x7fef4c13e520 2023-10-18T12:02:01.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.129+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c1e2780 con 0x7fef4c13e520 2023-10-18T12:02:01.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.130+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c1e2ca0 con 0x7fef4c13e520 2023-10-18T12:02:01.138 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.131+0000 7fef427fc640 1 -- 172.21.15.125:0/2746604440 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fef3c03b040 con 0x7fef4c13e520 2023-10-18T12:02:01.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.131+0000 7fef427fc640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fef3006b020 0x7fef3006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:01.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.131+0000 7fef427fc640 1 -- 172.21.15.125:0/2746604440 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fef3c0abdb0 con 0x7fef4c13e520 2023-10-18T12:02:01.139 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.131+0000 7fef51daa640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fef3006b020 0x7fef3006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:01.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.132+0000 7fef51daa640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fef3006b020 0x7fef3006d4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef480011e0 tx=0x7fef48009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:01.140 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.136+0000 7fef427fc640 1 -- 172.21.15.125:0/2746604440 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef4c1e2ca0 con 0x7fef4c13e520 2023-10-18T12:02:01.225 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:01.223+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:01.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.347+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fef4c067930 con 0x7fef4c13e520 2023-10-18T12:02:01.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.396+0000 7fef427fc640 1 -- 172.21.15.125:0/2746604440 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v111) v1 ==== 118+0+0 (secure 0 0 0) 0x7fef3c076a90 con 0x7fef4c13e520 2023-10-18T12:02:01.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fef3006b020 msgr2=0x7fef3006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fef3006b020 0x7fef3006d4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef480011e0 tx=0x7fef48009040 comp rx=0 tx=0).stop 2023-10-18T12:02:01.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef4c13e520 msgr2=0x7fef4c1f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef4c13e520 0x7fef4c1f1d30 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009980 tx=0x7fef3c0038a0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 shutdown_connections 2023-10-18T12:02:01.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef4c141690 0x7fef4c1df6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef4c13e520 0x7fef4c1f1d30 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.402 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fef3006b020 0x7fef3006d4e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 --2- 172.21.15.125:0/2746604440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c070920 0x7fef4c13b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 >> 172.21.15.125:0/2746604440 conn(0x7fef4c069930 msgr2=0x7fef4c065570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:01.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 shutdown_connections 2023-10-18T12:02:01.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.398+0000 7fef5400e640 1 -- 172.21.15.125:0/2746604440 wait complete. 2023-10-18T12:02:01.411 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T12:02:01.412 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T12:02:01.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.518+0000 7f2a01539640 1 -- 172.21.15.125:0/1947972252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 msgr2=0x7f29e40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.518+0000 7f2a01539640 1 --2- 172.21.15.125:0/1947972252 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 0x7f29e40047c0 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f29f000bb20 tx=0x7f29e40057f0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.518+0000 7f29fb7fe640 1 -- 172.21.15.125:0/1947972252 reap_dead start 2023-10-18T12:02:01.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.518+0000 7f2a01539640 1 -- 172.21.15.125:0/1947972252 shutdown_connections 2023-10-18T12:02:01.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.518+0000 7f2a01539640 1 -- 172.21.15.125:0/1947972252 >> 172.21.15.125:0/1947972252 conn(0x7f29fc0b0670 msgr2=0x7f29fc0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:01.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.518+0000 7f2a01539640 1 -- 172.21.15.125:0/1947972252 shutdown_connections 2023-10-18T12:02:01.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.518+0000 7f2a01539640 1 -- 172.21.15.125:0/1947972252 wait complete. 2023-10-18T12:02:01.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.519+0000 7f2a01539640 1 Processor -- start 2023-10-18T12:02:01.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.519+0000 7f2a01539640 1 -- start start 2023-10-18T12:02:01.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f2a01539640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 0x7f29fc150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:01.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f2a01539640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29fc15c2b0 0x7f29fc0ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:01.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f2a01539640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29fc0ad8c0 0x7f29fc1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:01.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f2a01539640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0c7fb0 con 0x7f29e4003fd0 2023-10-18T12:02:01.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f2a01539640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0add40 con 0x7f29fc0ad8c0 2023-10-18T12:02:01.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f2a01539640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc15b580 con 0x7f29fc15c2b0 2023-10-18T12:02:01.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 0x7f29fc150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:01.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 0x7f29fc150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56036/0 (socket says 172.21.15.125:56036) 2023-10-18T12:02:01.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 -- 172.21.15.125:0/2710324874 learned_addr learned my addr 172.21.15.125:0/2710324874 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:01.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29fa7fc640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29fc15c2b0 0x7f29fc0ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:01.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29fb7fe640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29fc0ad8c0 0x7f29fc1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:01.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 -- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29fc15c2b0 msgr2=0x7f29fc0ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29fc15c2b0 0x7f29fc0ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 -- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29fc0ad8c0 msgr2=0x7f29fc1597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:01.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29fc0ad8c0 0x7f29fc1597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:01.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 -- 172.21.15.125:0/2710324874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc15a490 con 0x7f29e4003fd0 2023-10-18T12:02:01.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29fb7fe640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29fc0ad8c0 0x7f29fc1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:01.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.520+0000 7f29faffd640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 0x7f29fc150b90 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7f29ec006990 tx=0x7f29ec001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:01.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.521+0000 7f29e37fe640 1 -- 172.21.15.125:0/2710324874 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00f9a0 con 0x7f29e4003fd0 2023-10-18T12:02:01.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.521+0000 7f29e37fe640 1 -- 172.21.15.125:0/2710324874 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29ec00fb00 con 0x7f29e4003fd0 2023-10-18T12:02:01.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.521+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc15b7d0 con 0x7f29e4003fd0 2023-10-18T12:02:01.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.521+0000 7f29e37fe640 1 -- 172.21.15.125:0/2710324874 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec011430 con 0x7f29e4003fd0 2023-10-18T12:02:01.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.521+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc15bd90 con 0x7f29e4003fd0 2023-10-18T12:02:01.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.522+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29fc0675f0 con 0x7f29e4003fd0 2023-10-18T12:02:01.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.522+0000 7f29e37fe640 1 -- 172.21.15.125:0/2710324874 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f29ec011590 con 0x7f29e4003fd0 2023-10-18T12:02:01.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.522+0000 7f29e37fe640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f29e806aff0 0x7f29e806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:01.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.523+0000 7f29e37fe640 1 -- 172.21.15.125:0/2710324874 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f29ec08a880 con 0x7f29e4003fd0 2023-10-18T12:02:01.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.523+0000 7f29fa7fc640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f29e806aff0 0x7f29e806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:01.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.523+0000 7f29fa7fc640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f29e806aff0 0x7f29e806d4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29f0037000 tx=0x7f29f002e040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:01.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.527+0000 7f29e37fe640 1 -- 172.21.15.125:0/2710324874 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29ec0554d0 con 0x7f29e4003fd0 2023-10-18T12:02:01.737 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:01.735+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f29fc0b4aa0 con 0x7f29e4003fd0 2023-10-18T12:02:01.976 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:01.974+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:02.057 INFO:tasks.ceph.mds.c.smithi125.stderr:2023-10-18T12:02:02.054+0000 7f41aba33640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:02:02.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.397+0000 7f29e37fe640 1 -- 172.21.15.125:0/2710324874 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v112) v1 ==== 122+0+0 (secure 0 0 0) 0x7f29ec020020 con 0x7f29e4003fd0 2023-10-18T12:02:02.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f29e806aff0 msgr2=0x7f29e806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f29e806aff0 0x7f29e806d4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29f0037000 tx=0x7f29f002e040 comp rx=0 tx=0).stop 2023-10-18T12:02:02.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 msgr2=0x7f29fc150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 0x7f29fc150b90 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7f29ec006990 tx=0x7f29ec001cf0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 shutdown_connections 2023-10-18T12:02:02.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f29e806aff0 0x7f29e806d4b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29fc0ad8c0 0x7f29fc1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.404 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29fc15c2b0 0x7f29fc0ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 --2- 172.21.15.125:0/2710324874 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29e4003fd0 0x7f29fc150b90 unknown :-1 s=CLOSED pgs=3018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 >> 172.21.15.125:0/2710324874 conn(0x7f29fc0b0670 msgr2=0x7f29fc0cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:02.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 shutdown_connections 2023-10-18T12:02:02.405 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.400+0000 7f2a01539640 1 -- 172.21.15.125:0/2710324874 wait complete. 2023-10-18T12:02:02.413 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T12:02:02.414 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:02:02.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.517+0000 7fa79fb84640 1 -- 172.21.15.125:0/193184668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7980ceb40 msgr2=0x7fa7980b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.517+0000 7fa79fb84640 1 --2- 172.21.15.125:0/193184668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7980ceb40 0x7fa7980b0650 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7fa79808a810 tx=0x7fa79002c260 comp rx=0 tx=0).stop 2023-10-18T12:02:02.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.517+0000 7fa79fb84640 1 -- 172.21.15.125:0/193184668 shutdown_connections 2023-10-18T12:02:02.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.517+0000 7fa79fb84640 1 --2- 172.21.15.125:0/193184668 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7980ceb40 0x7fa7980b0650 unknown :-1 s=CLOSED pgs=3019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.517+0000 7fa79fb84640 1 --2- 172.21.15.125:0/193184668 >> v2:172.21.15.139:3300/0 conn(0x7fa7980cf340 0x7fa7980d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.517+0000 7fa79fb84640 1 -- 172.21.15.125:0/193184668 >> 172.21.15.125:0/193184668 conn(0x7fa7980d0d60 msgr2=0x7fa7980d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:02.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.517+0000 7fa79fb84640 1 -- 172.21.15.125:0/193184668 shutdown_connections 2023-10-18T12:02:02.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.517+0000 7fa79fb84640 1 -- 172.21.15.125:0/193184668 wait complete. 2023-10-18T12:02:02.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.518+0000 7fa79fb84640 1 Processor -- start 2023-10-18T12:02:02.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.518+0000 7fa79fb84640 1 -- start start 2023-10-18T12:02:02.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79fb84640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7980ceb40 0x7fa79815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:02.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79fb84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7980cf340 0x7fa798150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:02.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79fb84640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa79815b2a0 0x7fa7981e5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:02.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79fb84640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980b4690 con 0x7fa79815b2a0 2023-10-18T12:02:02.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79fb84640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79815bfe0 con 0x7fa7980cf340 2023-10-18T12:02:02.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79fb84640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980b3a80 con 0x7fa7980ceb40 2023-10-18T12:02:02.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79e121640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa79815b2a0 0x7fa7981e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:02.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79d920640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7980ceb40 0x7fa79815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:02.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79d920640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7980ceb40 0x7fa79815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:43334/0 (socket says 172.21.15.125:43334) 2023-10-18T12:02:02.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79d920640 1 -- 172.21.15.125:0/1709547054 learned_addr learned my addr 172.21.15.125:0/1709547054 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:02.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79d11f640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7980cf340 0x7fa798150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:02.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79e121640 1 -- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7980ceb40 msgr2=0x7fa79815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79e121640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7980ceb40 0x7fa79815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79e121640 1 -- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7980cf340 msgr2=0x7fa798150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.527 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79e121640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7980cf340 0x7fa798150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.519+0000 7fa79e121640 1 -- 172.21.15.125:0/1709547054 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa790009900 con 0x7fa79815b2a0 2023-10-18T12:02:02.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.520+0000 7fa79e121640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa79815b2a0 0x7fa7981e5400 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7fa790033000 tx=0x7fa790003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:02.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.520+0000 7fa78e7fc640 1 -- 172.21.15.125:0/1709547054 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790004240 con 0x7fa79815b2a0 2023-10-18T12:02:02.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.520+0000 7fa78e7fc640 1 -- 172.21.15.125:0/1709547054 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa7900044b0 con 0x7fa79815b2a0 2023-10-18T12:02:02.528 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.520+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7980b3d60 con 0x7fa79815b2a0 2023-10-18T12:02:02.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.520+0000 7fa78e7fc640 1 -- 172.21.15.125:0/1709547054 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79003f440 con 0x7fa79815b2a0 2023-10-18T12:02:02.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.520+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7980b4320 con 0x7fa79815b2a0 2023-10-18T12:02:02.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.521+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7980b4800 con 0x7fa79815b2a0 2023-10-18T12:02:02.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.521+0000 7fa78e7fc640 1 -- 172.21.15.125:0/1709547054 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fa79003b040 con 0x7fa79815b2a0 2023-10-18T12:02:02.529 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.522+0000 7fa78e7fc640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa77c06aff0 0x7fa77c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:02.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.522+0000 7fa78e7fc640 1 -- 172.21.15.125:0/1709547054 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fa7900ac620 con 0x7fa79815b2a0 2023-10-18T12:02:02.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.522+0000 7fa79d920640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa77c06aff0 0x7fa77c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:02.530 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.522+0000 7fa79d920640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa77c06aff0 0x7fa77c06d4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa788009c90 tx=0x7fa788005f90 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:02.531 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.526+0000 7fa78e7fc640 1 -- 172.21.15.125:0/1709547054 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa790077380 con 0x7fa79815b2a0 2023-10-18T12:02:02.735 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:02.733+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:02.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.738+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7980b0d00 con 0x7fa79815b2a0 2023-10-18T12:02:02.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.738+0000 7fa78e7fc640 1 -- 172.21.15.125:0/1709547054 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7fa79007a9a0 con 0x7fa79815b2a0 2023-10-18T12:02:02.741 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:02.741 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":112,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3962342896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3962342896},{"type":"v1","addr":"172.21.15.139:6837","nonce":3962342896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11413,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1953593032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1953593032},{"type":"v1","addr":"172.21.15.139:6836","nonce":1953593032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":13566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1230654530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1230654530},{"type":"v1","addr":"172.21.15.125:6834","nonce":1230654530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:59.406238+0000","modified":"2023-10-18T12:02:01.736375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13569":{"gid":13569,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1922177652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1922177652},{"type":"v1","addr":"172.21.15.125:6835","nonce":1922177652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-18T12:02:02.742 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 112 2023-10-18T12:02:02.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa77c06aff0 msgr2=0x7fa77c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa77c06aff0 0x7fa77c06d4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa788009c90 tx=0x7fa788005f90 comp rx=0 tx=0).stop 2023-10-18T12:02:02.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa79815b2a0 msgr2=0x7fa7981e5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa79815b2a0 0x7fa7981e5400 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7fa790033000 tx=0x7fa790003ba0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 shutdown_connections 2023-10-18T12:02:02.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fa77c06aff0 0x7fa77c06d4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7980cf340 0x7fa798150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa79815b2a0 0x7fa7981e5400 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 --2- 172.21.15.125:0/1709547054 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7980ceb40 0x7fa79815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 >> 172.21.15.125:0/1709547054 conn(0x7fa7980d0d60 msgr2=0x7fa7980ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:02.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 shutdown_connections 2023-10-18T12:02:02.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.740+0000 7fa79fb84640 1 -- 172.21.15.125:0/1709547054 wait complete. 2023-10-18T12:02:02.753 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:02:02.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.856+0000 7f2545490640 1 -- 172.21.15.125:0/2339101091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 msgr2=0x7f254015b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.856+0000 7f2545490640 1 --2- 172.21.15.125:0/2339101091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 0x7f254015b760 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7f2540066820 tx=0x7f253002bda0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.856+0000 7f2545490640 1 -- 172.21.15.125:0/2339101091 shutdown_connections 2023-10-18T12:02:02.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.856+0000 7f2545490640 1 --2- 172.21.15.125:0/2339101091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f254006cea0 0x7f254006f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.856+0000 7f2545490640 1 --2- 172.21.15.125:0/2339101091 >> v2:172.21.15.139:3300/0 conn(0x7f25401582d0 0x7f254006c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.856+0000 7f2545490640 1 --2- 172.21.15.125:0/2339101091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 0x7f254015b760 unknown :-1 s=CLOSED pgs=3021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.856+0000 7f2545490640 1 -- 172.21.15.125:0/2339101091 >> 172.21.15.125:0/2339101091 conn(0x7f25400cc970 msgr2=0x7f25400ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:02.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.857+0000 7f2545490640 1 -- 172.21.15.125:0/2339101091 shutdown_connections 2023-10-18T12:02:02.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.857+0000 7f2545490640 1 -- 172.21.15.125:0/2339101091 wait complete. 2023-10-18T12:02:02.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.858+0000 7f2545490640 1 Processor -- start 2023-10-18T12:02:02.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.858+0000 7f2545490640 1 -- start start 2023-10-18T12:02:02.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.858+0000 7f2545490640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f254006cea0 0x7f25400ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.858+0000 7f2545490640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 0x7f25401387e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.858+0000 7f2545490640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25401582d0 0x7f2540138d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.858+0000 7f2545490640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25400ad080 con 0x7f25401413a0 2023-10-18T12:02:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.858+0000 7f2545490640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25400ad260 con 0x7f254006cea0 2023-10-18T12:02:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2545490640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25400ad540 con 0x7f25401582d0 2023-10-18T12:02:02.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 0x7f25401387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 0x7f25401387e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56070/0 (socket says 172.21.15.125:56070) 2023-10-18T12:02:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 -- 172.21.15.125:0/2577302927 learned_addr learned my addr 172.21.15.125:0/2577302927 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f253effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f254006cea0 0x7f25400ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:02.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f253f7fe640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25401582d0 0x7f2540138d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 -- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25401582d0 msgr2=0x7f2540138d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25401582d0 0x7f2540138d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 -- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f254006cea0 msgr2=0x7f25400ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f254006cea0 0x7f25400ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:02.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 -- 172.21.15.125:0/2577302927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530009560 con 0x7f25401413a0 2023-10-18T12:02:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2537fff640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 0x7f25401387e0 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x7f252800d6d0 tx=0x7f252800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f25377fe640 1 -- 172.21.15.125:0/2577302927 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528009a20 con 0x7f25401413a0 2023-10-18T12:02:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f25377fe640 1 -- 172.21.15.125:0/2577302927 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2528009c70 con 0x7f25401413a0 2023-10-18T12:02:02.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25400ad820 con 0x7f25401413a0 2023-10-18T12:02:02.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f25377fe640 1 -- 172.21.15.125:0/2577302927 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528010460 con 0x7f25401413a0 2023-10-18T12:02:02.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.859+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25400adde0 con 0x7f25401413a0 2023-10-18T12:02:02.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.860+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2540072100 con 0x7f25401413a0 2023-10-18T12:02:02.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.861+0000 7f25377fe640 1 -- 172.21.15.125:0/2577302927 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f25280107d0 con 0x7f25401413a0 2023-10-18T12:02:02.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.861+0000 7f25377fe640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252006aff0 0x7f252006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:02.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.861+0000 7f25377fe640 1 -- 172.21.15.125:0/2577302927 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f252808a150 con 0x7f25401413a0 2023-10-18T12:02:02.871 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.861+0000 7f253effd640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252006aff0 0x7f252006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:02.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.862+0000 7f253effd640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252006aff0 0x7f252006d4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2530035000 tx=0x7f2530004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:02.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:02.866+0000 7f25377fe640 1 -- 172.21.15.125:0/2577302927 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2540072100 con 0x7f25401413a0 2023-10-18T12:02:03.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.044+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2540072100 con 0x7f25401413a0 2023-10-18T12:02:03.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.045+0000 7f25377fe640 1 -- 172.21.15.125:0/2577302927 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v262) v1 ==== 74+0+16858 (secure 0 0 0) 0x7f2540072100 con 0x7f25401413a0 2023-10-18T12:02:03.048 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:03.048 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":262,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:02:01.073982+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:57.717906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:58.385232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:02:03.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252006aff0 msgr2=0x7f252006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252006aff0 0x7f252006d4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2530035000 tx=0x7f2530004730 comp rx=0 tx=0).stop 2023-10-18T12:02:03.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 msgr2=0x7f25401387e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 0x7f25401387e0 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x7f252800d6d0 tx=0x7f252800db90 comp rx=0 tx=0).stop 2023-10-18T12:02:03.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 shutdown_connections 2023-10-18T12:02:03.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f25401582d0 0x7f2540138d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f25401413a0 0x7f25401387e0 unknown :-1 s=CLOSED pgs=3022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f252006aff0 0x7f252006d4b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 --2- 172.21.15.125:0/2577302927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f254006cea0 0x7f25400ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 >> 172.21.15.125:0/2577302927 conn(0x7f25400cc970 msgr2=0x7f25401573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:03.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 shutdown_connections 2023-10-18T12:02:03.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.047+0000 7f2545490640 1 -- 172.21.15.125:0/2577302927 wait complete. 2023-10-18T12:02:03.061 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T12:02:03.061 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:02:03.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.163+0000 7fd3b1100640 1 -- 172.21.15.125:0/2014797867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3ac15b2a0 msgr2=0x7fd3ac0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.163+0000 7fd3b1100640 1 --2- 172.21.15.125:0/2014797867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3ac15b2a0 0x7fd3ac0cf720 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd394009830 tx=0x7fd39402bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.166 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.163+0000 7fd3b1100640 1 -- 172.21.15.125:0/2014797867 shutdown_connections 2023-10-18T12:02:03.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.163+0000 7fd3b1100640 1 --2- 172.21.15.125:0/2014797867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3ac15b2a0 0x7fd3ac0cf720 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.163+0000 7fd3b1100640 1 --2- 172.21.15.125:0/2014797867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3ac0cf340 0x7fd3ac0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.163+0000 7fd3b1100640 1 -- 172.21.15.125:0/2014797867 >> 172.21.15.125:0/2014797867 conn(0x7fd3ac0d0d60 msgr2=0x7fd3ac0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:03.167 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.163+0000 7fd3b1100640 1 -- 172.21.15.125:0/2014797867 shutdown_connections 2023-10-18T12:02:03.168 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.163+0000 7fd3b1100640 1 -- 172.21.15.125:0/2014797867 wait complete. 2023-10-18T12:02:03.170 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.164+0000 7fd3b1100640 1 Processor -- start 2023-10-18T12:02:03.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.164+0000 7fd3b1100640 1 -- start start 2023-10-18T12:02:03.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3b1100640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3ac0ceb40 0x7fd3ac0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3b1100640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3ac0cf340 0x7fd3ac0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3b1100640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3ac15b2a0 0x7fd3ac1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.171 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3b1100640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac1f1550 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3b1100640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac1f2500 con 0x7fd3ac0cf340 2023-10-18T12:02:03.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3b1100640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c75b0 con 0x7fd3ac0ceb40 2023-10-18T12:02:03.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3ac15b2a0 0x7fd3ac1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.172 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3ac15b2a0 0x7fd3ac1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56072/0 (socket says 172.21.15.125:56072) 2023-10-18T12:02:03.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 -- 172.21.15.125:0/3337216623 learned_addr learned my addr 172.21.15.125:0/3337216623 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:03.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3a3fff640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3ac0cf340 0x7fd3ac0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3aad9d640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3ac0ceb40 0x7fd3ac0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 -- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3ac0ceb40 msgr2=0x7fd3ac0b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.173 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3ac0ceb40 0x7fd3ac0b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 -- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3ac0cf340 msgr2=0x7fd3ac0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3ac0cf340 0x7fd3ac0b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 -- 172.21.15.125:0/3337216623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c009490 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.165+0000 7fd3ab59e640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3ac15b2a0 0x7fd3ac1582e0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400cbf0 tx=0x7fd3a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:03.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.166+0000 7fd3a37fe640 1 -- 172.21.15.125:0/3337216623 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400aca0 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.166+0000 7fd3a37fe640 1 -- 172.21.15.125:0/3337216623 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd3a4003970 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.166+0000 7fd3a37fe640 1 -- 172.21.15.125:0/3337216623 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4003d80 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.166+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd394009490 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.175 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.166+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac0c7e30 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.167+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ac158820 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.167+0000 7fd3a37fe640 1 -- 172.21.15.125:0/3337216623 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fd3a4004030 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.168+0000 7fd3a37fe640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fd38c06aff0 0x7fd38c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.168+0000 7fd3a37fe640 1 -- 172.21.15.125:0/3337216623 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fd3a408a4b0 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.176 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.168+0000 7fd3aad9d640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fd38c06aff0 0x7fd38c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.169+0000 7fd3aad9d640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fd38c06aff0 0x7fd38c06d4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009cf0 tx=0x7fd39c005f90 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:03.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.172+0000 7fd3a37fe640 1 -- 172.21.15.125:0/3337216623 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3a40551f0 con 0x7fd3ac15b2a0 2023-10-18T12:02:03.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.339+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 --> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3ac0cc170 con 0x7fd38c06aff0 2023-10-18T12:02:03.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.348+0000 7fd3a37fe640 1 -- 172.21.15.125:0/3337216623 <== mgr.9734 v2:172.21.15.139:6832/666436799 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289775 (secure 0 0 0) 0x7fd39c016070 con 0x7fd38c06aff0 2023-10-18T12:02:03.353 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:03.355 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:02:03.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fd38c06aff0 msgr2=0x7fd38c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fd38c06aff0 0x7fd38c06d4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009cf0 tx=0x7fd39c005f90 comp rx=0 tx=0).stop 2023-10-18T12:02:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3ac15b2a0 msgr2=0x7fd3ac1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3ac15b2a0 0x7fd3ac1582e0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400cbf0 tx=0x7fd3a400a060 comp rx=0 tx=0).stop 2023-10-18T12:02:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 shutdown_connections 2023-10-18T12:02:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fd38c06aff0 0x7fd38c06d4b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3ac0cf340 0x7fd3ac0b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3ac15b2a0 0x7fd3ac1582e0 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 --2- 172.21.15.125:0/3337216623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3ac0ceb40 0x7fd3ac0b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 >> 172.21.15.125:0/3337216623 conn(0x7fd3ac0d0d60 msgr2=0x7fd3ac15a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:03.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 shutdown_connections 2023-10-18T12:02:03.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.351+0000 7fd3b1100640 1 -- 172.21.15.125:0/3337216623 wait complete. 2023-10-18T12:02:03.364 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-18T12:02:03.267970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2500,"num_read_kb":5242,"num_write":649,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3062,"ondisk_log_size":3062,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423476,"kb_used_data":26868,"kb_used_omap":12,"kb_used_meta":396467,"kb_avail":754551244,"statfs":{"total":773094113280,"available":772660473856,"internally_reserved":0,"allocated":27512832,"data_stored":14790727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":405982776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.204752"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111113+0000","last_change":"2023-10-18T12:02:00.091567+0000","last_active":"2023-10-18T12:02:00.111113+0000","last_peered":"2023-10-18T12:02:00.111113+0000","last_clean":"2023-10-18T12:02:00.111113+0000","last_became_active":"2023-10-18T12:02:00.091295+0000","last_became_peered":"2023-10-18T12:02:00.091295+0000","last_unstale":"2023-10-18T12:02:00.111113+0000","last_undegraded":"2023-10-18T12:02:00.111113+0000","last_fullsized":"2023-10-18T12:02:00.111113+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:21.598944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111306+0000","last_change":"2023-10-18T12:02:00.091973+0000","last_active":"2023-10-18T12:02:00.111306+0000","last_peered":"2023-10-18T12:02:00.111306+0000","last_clean":"2023-10-18T12:02:00.111306+0000","last_became_active":"2023-10-18T12:02:00.090926+0000","last_became_peered":"2023-10-18T12:02:00.090926+0000","last_unstale":"2023-10-18T12:02:00.111306+0000","last_undegraded":"2023-10-18T12:02:00.111306+0000","last_fullsized":"2023-10-18T12:02:00.111306+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:34:20.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096051+0000","last_change":"2023-10-18T12:02:00.093710+0000","last_active":"2023-10-18T12:02:00.096051+0000","last_peered":"2023-10-18T12:02:00.096051+0000","last_clean":"2023-10-18T12:02:00.096051+0000","last_became_active":"2023-10-18T12:02:00.093469+0000","last_became_peered":"2023-10-18T12:02:00.093469+0000","last_unstale":"2023-10-18T12:02:00.096051+0000","last_undegraded":"2023-10-18T12:02:00.096051+0000","last_fullsized":"2023-10-18T12:02:00.096051+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:22:06.585710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096077+0000","last_change":"2023-10-18T12:02:00.091806+0000","last_active":"2023-10-18T12:02:00.096077+0000","last_peered":"2023-10-18T12:02:00.096077+0000","last_clean":"2023-10-18T12:02:00.096077+0000","last_became_active":"2023-10-18T12:02:00.090635+0000","last_became_peered":"2023-10-18T12:02:00.090635+0000","last_unstale":"2023-10-18T12:02:00.096077+0000","last_undegraded":"2023-10-18T12:02:00.096077+0000","last_fullsized":"2023-10-18T12:02:00.096077+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:20.311774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084531+0000","last_change":"2023-10-18T12:02:00.083558+0000","last_active":"2023-10-18T12:02:00.084531+0000","last_peered":"2023-10-18T12:02:00.084531+0000","last_clean":"2023-10-18T12:02:00.084531+0000","last_became_active":"2023-10-18T12:02:00.083274+0000","last_became_peered":"2023-10-18T12:02:00.083274+0000","last_unstale":"2023-10-18T12:02:00.084531+0000","last_undegraded":"2023-10-18T12:02:00.084531+0000","last_fullsized":"2023-10-18T12:02:00.084531+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:43:44.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097118+0000","last_change":"2023-10-18T12:02:00.096585+0000","last_active":"2023-10-18T12:02:00.097118+0000","last_peered":"2023-10-18T12:02:00.097118+0000","last_clean":"2023-10-18T12:02:00.097118+0000","last_became_active":"2023-10-18T12:02:00.096334+0000","last_became_peered":"2023-10-18T12:02:00.096334+0000","last_unstale":"2023-10-18T12:02:00.097118+0000","last_undegraded":"2023-10-18T12:02:00.097118+0000","last_fullsized":"2023-10-18T12:02:00.097118+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:44:08.823723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.093043+0000","last_change":"2023-10-18T12:02:00.090486+0000","last_active":"2023-10-18T12:02:00.093043+0000","last_peered":"2023-10-18T12:02:00.093043+0000","last_clean":"2023-10-18T12:02:00.093043+0000","last_became_active":"2023-10-18T12:02:00.090236+0000","last_became_peered":"2023-10-18T12:02:00.090236+0000","last_unstale":"2023-10-18T12:02:00.093043+0000","last_undegraded":"2023-10-18T12:02:00.093043+0000","last_fullsized":"2023-10-18T12:02:00.093043+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:28.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.090045+0000","last_change":"2023-10-18T12:02:00.088034+0000","last_active":"2023-10-18T12:02:00.090045+0000","last_peered":"2023-10-18T12:02:00.090045+0000","last_clean":"2023-10-18T12:02:00.090045+0000","last_became_active":"2023-10-18T12:02:00.087782+0000","last_became_peered":"2023-10-18T12:02:00.087782+0000","last_unstale":"2023-10-18T12:02:00.090045+0000","last_undegraded":"2023-10-18T12:02:00.090045+0000","last_fullsized":"2023-10-18T12:02:00.090045+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:23.537671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.098768+0000","last_change":"2023-10-18T12:02:00.098175+0000","last_active":"2023-10-18T12:02:00.098768+0000","last_peered":"2023-10-18T12:02:00.098768+0000","last_clean":"2023-10-18T12:02:00.098768+0000","last_became_active":"2023-10-18T12:02:00.097852+0000","last_became_peered":"2023-10-18T12:02:00.097852+0000","last_unstale":"2023-10-18T12:02:00.098768+0000","last_undegraded":"2023-10-18T12:02:00.098768+0000","last_fullsized":"2023-10-18T12:02:00.098768+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:21.298774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112296+0000","last_change":"2023-10-18T12:02:00.111845+0000","last_active":"2023-10-18T12:02:00.112296+0000","last_peered":"2023-10-18T12:02:00.112296+0000","last_clean":"2023-10-18T12:02:00.112296+0000","last_became_active":"2023-10-18T12:02:00.111570+0000","last_became_peered":"2023-10-18T12:02:00.111570+0000","last_unstale":"2023-10-18T12:02:00.112296+0000","last_undegraded":"2023-10-18T12:02:00.112296+0000","last_fullsized":"2023-10-18T12:02:00.112296+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:18.470466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.091058+0000","last_change":"2023-10-18T12:02:00.090291+0000","last_active":"2023-10-18T12:02:00.091058+0000","last_peered":"2023-10-18T12:02:00.091058+0000","last_clean":"2023-10-18T12:02:00.091058+0000","last_became_active":"2023-10-18T12:02:00.090044+0000","last_became_peered":"2023-10-18T12:02:00.090044+0000","last_unstale":"2023-10-18T12:02:00.091058+0000","last_undegraded":"2023-10-18T12:02:00.091058+0000","last_fullsized":"2023-10-18T12:02:00.091058+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:07.420755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111379+0000","last_change":"2023-10-18T12:02:00.092495+0000","last_active":"2023-10-18T12:02:00.111379+0000","last_peered":"2023-10-18T12:02:00.111379+0000","last_clean":"2023-10-18T12:02:00.111379+0000","last_became_active":"2023-10-18T12:02:00.092249+0000","last_became_peered":"2023-10-18T12:02:00.092249+0000","last_unstale":"2023-10-18T12:02:00.111379+0000","last_undegraded":"2023-10-18T12:02:00.111379+0000","last_fullsized":"2023-10-18T12:02:00.111379+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:57.479653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089991+0000","last_change":"2023-10-18T12:02:00.088647+0000","last_active":"2023-10-18T12:02:00.089991+0000","last_peered":"2023-10-18T12:02:00.089991+0000","last_clean":"2023-10-18T12:02:00.089991+0000","last_became_active":"2023-10-18T12:02:00.088410+0000","last_became_peered":"2023-10-18T12:02:00.088410+0000","last_unstale":"2023-10-18T12:02:00.089991+0000","last_undegraded":"2023-10-18T12:02:00.089991+0000","last_fullsized":"2023-10-18T12:02:00.089991+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:50.165803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097324+0000","last_change":"2023-10-18T12:02:00.093226+0000","last_active":"2023-10-18T12:02:00.097324+0000","last_peered":"2023-10-18T12:02:00.097324+0000","last_clean":"2023-10-18T12:02:00.097324+0000","last_became_active":"2023-10-18T12:02:00.092569+0000","last_became_peered":"2023-10-18T12:02:00.092569+0000","last_unstale":"2023-10-18T12:02:00.097324+0000","last_undegraded":"2023-10-18T12:02:00.097324+0000","last_fullsized":"2023-10-18T12:02:00.097324+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:12.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.083508+0000","last_change":"2023-10-18T12:02:00.082189+0000","last_active":"2023-10-18T12:02:00.083508+0000","last_peered":"2023-10-18T12:02:00.083508+0000","last_clean":"2023-10-18T12:02:00.083508+0000","last_became_active":"2023-10-18T12:02:00.081931+0000","last_became_peered":"2023-10-18T12:02:00.081931+0000","last_unstale":"2023-10-18T12:02:00.083508+0000","last_undegraded":"2023-10-18T12:02:00.083508+0000","last_fullsized":"2023-10-18T12:02:00.083508+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:49:40.953084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087335+0000","last_change":"2023-10-18T12:02:00.084985+0000","last_active":"2023-10-18T12:02:00.087335+0000","last_peered":"2023-10-18T12:02:00.087335+0000","last_clean":"2023-10-18T12:02:00.087335+0000","last_became_active":"2023-10-18T12:02:00.084719+0000","last_became_peered":"2023-10-18T12:02:00.084719+0000","last_unstale":"2023-10-18T12:02:00.087335+0000","last_undegraded":"2023-10-18T12:02:00.087335+0000","last_fullsized":"2023-10-18T12:02:00.087335+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:42.813621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087376+0000","last_change":"2023-10-18T12:02:00.085482+0000","last_active":"2023-10-18T12:02:00.087376+0000","last_peered":"2023-10-18T12:02:00.087376+0000","last_clean":"2023-10-18T12:02:00.087376+0000","last_became_active":"2023-10-18T12:02:00.085247+0000","last_became_peered":"2023-10-18T12:02:00.085247+0000","last_unstale":"2023-10-18T12:02:00.087376+0000","last_undegraded":"2023-10-18T12:02:00.087376+0000","last_fullsized":"2023-10-18T12:02:00.087376+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:42.483741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096483+0000","last_change":"2023-10-18T12:02:00.093997+0000","last_active":"2023-10-18T12:02:00.096483+0000","last_peered":"2023-10-18T12:02:00.096483+0000","last_clean":"2023-10-18T12:02:00.096483+0000","last_became_active":"2023-10-18T12:02:00.093749+0000","last_became_peered":"2023-10-18T12:02:00.093749+0000","last_unstale":"2023-10-18T12:02:00.096483+0000","last_undegraded":"2023-10-18T12:02:00.096483+0000","last_fullsized":"2023-10-18T12:02:00.096483+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:28.518720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087307+0000","last_change":"2023-10-18T12:02:00.084763+0000","last_active":"2023-10-18T12:02:00.087307+0000","last_peered":"2023-10-18T12:02:00.087307+0000","last_clean":"2023-10-18T12:02:00.087307+0000","last_became_active":"2023-10-18T12:02:00.084521+0000","last_became_peered":"2023-10-18T12:02:00.084521+0000","last_unstale":"2023-10-18T12:02:00.087307+0000","last_undegraded":"2023-10-18T12:02:00.087307+0000","last_fullsized":"2023-10-18T12:02:00.087307+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:57:15.607727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.091582+0000","last_change":"2023-10-18T12:02:00.086170+0000","last_active":"2023-10-18T12:02:00.091582+0000","last_peered":"2023-10-18T12:02:00.091582+0000","last_clean":"2023-10-18T12:02:00.091582+0000","last_became_active":"2023-10-18T12:02:00.085928+0000","last_became_peered":"2023-10-18T12:02:00.085928+0000","last_unstale":"2023-10-18T12:02:00.091582+0000","last_undegraded":"2023-10-18T12:02:00.091582+0000","last_fullsized":"2023-10-18T12:02:00.091582+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:41.141532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.090083+0000","last_change":"2023-10-18T12:02:00.085548+0000","last_active":"2023-10-18T12:02:00.090083+0000","last_peered":"2023-10-18T12:02:00.090083+0000","last_clean":"2023-10-18T12:02:00.090083+0000","last_became_active":"2023-10-18T12:02:00.084282+0000","last_became_peered":"2023-10-18T12:02:00.084282+0000","last_unstale":"2023-10-18T12:02:00.090083+0000","last_undegraded":"2023-10-18T12:02:00.090083+0000","last_fullsized":"2023-10-18T12:02:00.090083+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:48.005895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097197+0000","last_change":"2023-10-18T12:02:00.095767+0000","last_active":"2023-10-18T12:02:00.097197+0000","last_peered":"2023-10-18T12:02:00.097197+0000","last_clean":"2023-10-18T12:02:00.097197+0000","last_became_active":"2023-10-18T12:02:00.095485+0000","last_became_peered":"2023-10-18T12:02:00.095485+0000","last_unstale":"2023-10-18T12:02:00.097197+0000","last_undegraded":"2023-10-18T12:02:00.097197+0000","last_fullsized":"2023-10-18T12:02:00.097197+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:11.928499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.095163+0000","last_change":"2023-10-18T12:02:00.092602+0000","last_active":"2023-10-18T12:02:00.095163+0000","last_peered":"2023-10-18T12:02:00.095163+0000","last_clean":"2023-10-18T12:02:00.095163+0000","last_became_active":"2023-10-18T12:02:00.092317+0000","last_became_peered":"2023-10-18T12:02:00.092317+0000","last_unstale":"2023-10-18T12:02:00.095163+0000","last_undegraded":"2023-10-18T12:02:00.095163+0000","last_fullsized":"2023-10-18T12:02:00.095163+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:32.006797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112350+0000","last_change":"2023-10-18T12:02:00.111846+0000","last_active":"2023-10-18T12:02:00.112350+0000","last_peered":"2023-10-18T12:02:00.112350+0000","last_clean":"2023-10-18T12:02:00.112350+0000","last_became_active":"2023-10-18T12:02:00.111450+0000","last_became_peered":"2023-10-18T12:02:00.111450+0000","last_unstale":"2023-10-18T12:02:00.112350+0000","last_undegraded":"2023-10-18T12:02:00.112350+0000","last_fullsized":"2023-10-18T12:02:00.112350+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:21.298774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111473+0000","last_change":"2023-10-18T12:02:00.094688+0000","last_active":"2023-10-18T12:02:00.111473+0000","last_peered":"2023-10-18T12:02:00.111473+0000","last_clean":"2023-10-18T12:02:00.111473+0000","last_became_active":"2023-10-18T12:02:00.094438+0000","last_became_peered":"2023-10-18T12:02:00.094438+0000","last_unstale":"2023-10-18T12:02:00.111473+0000","last_undegraded":"2023-10-18T12:02:00.111473+0000","last_fullsized":"2023-10-18T12:02:00.111473+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:13.006744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.091961+0000","last_change":"2023-10-18T12:02:00.089600+0000","last_active":"2023-10-18T12:02:00.091961+0000","last_peered":"2023-10-18T12:02:00.091961+0000","last_clean":"2023-10-18T12:02:00.091961+0000","last_became_active":"2023-10-18T12:02:00.089364+0000","last_became_peered":"2023-10-18T12:02:00.089364+0000","last_unstale":"2023-10-18T12:02:00.091961+0000","last_undegraded":"2023-10-18T12:02:00.091961+0000","last_fullsized":"2023-10-18T12:02:00.091961+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:55:27.030141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.088927+0000","last_change":"2023-10-18T12:02:00.088526+0000","last_active":"2023-10-18T12:02:00.088927+0000","last_peered":"2023-10-18T12:02:00.088927+0000","last_clean":"2023-10-18T12:02:00.088927+0000","last_became_active":"2023-10-18T12:02:00.088287+0000","last_became_peered":"2023-10-18T12:02:00.088287+0000","last_unstale":"2023-10-18T12:02:00.088927+0000","last_undegraded":"2023-10-18T12:02:00.088927+0000","last_fullsized":"2023-10-18T12:02:00.088927+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:20:13.512224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.095998+0000","last_change":"2023-10-18T12:02:00.093606+0000","last_active":"2023-10-18T12:02:00.095998+0000","last_peered":"2023-10-18T12:02:00.095998+0000","last_clean":"2023-10-18T12:02:00.095998+0000","last_became_active":"2023-10-18T12:02:00.093359+0000","last_became_peered":"2023-10-18T12:02:00.093359+0000","last_unstale":"2023-10-18T12:02:00.095998+0000","last_undegraded":"2023-10-18T12:02:00.095998+0000","last_fullsized":"2023-10-18T12:02:00.095998+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:35.755353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.082374+0000","last_change":"2023-10-18T12:02:00.081885+0000","last_active":"2023-10-18T12:02:00.082374+0000","last_peered":"2023-10-18T12:02:00.082374+0000","last_clean":"2023-10-18T12:02:00.082374+0000","last_became_active":"2023-10-18T12:02:00.081634+0000","last_became_peered":"2023-10-18T12:02:00.081634+0000","last_unstale":"2023-10-18T12:02:00.082374+0000","last_undegraded":"2023-10-18T12:02:00.082374+0000","last_fullsized":"2023-10-18T12:02:00.082374+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:45.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096897+0000","last_change":"2023-10-18T12:02:00.094737+0000","last_active":"2023-10-18T12:02:00.096897+0000","last_peered":"2023-10-18T12:02:00.096897+0000","last_clean":"2023-10-18T12:02:00.096897+0000","last_became_active":"2023-10-18T12:02:00.094483+0000","last_became_peered":"2023-10-18T12:02:00.094483+0000","last_unstale":"2023-10-18T12:02:00.096897+0000","last_undegraded":"2023-10-18T12:02:00.096897+0000","last_fullsized":"2023-10-18T12:02:00.096897+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:44.083388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085627+0000","last_change":"2023-10-18T12:02:00.084636+0000","last_active":"2023-10-18T12:02:00.085627+0000","last_peered":"2023-10-18T12:02:00.085627+0000","last_clean":"2023-10-18T12:02:00.085627+0000","last_became_active":"2023-10-18T12:02:00.084394+0000","last_became_peered":"2023-10-18T12:02:00.084394+0000","last_unstale":"2023-10-18T12:02:00.085627+0000","last_undegraded":"2023-10-18T12:02:00.085627+0000","last_fullsized":"2023-10-18T12:02:00.085627+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:50.842391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112368+0000","last_change":"2023-10-18T12:02:00.111868+0000","last_active":"2023-10-18T12:02:00.112368+0000","last_peered":"2023-10-18T12:02:00.112368+0000","last_clean":"2023-10-18T12:02:00.112368+0000","last_became_active":"2023-10-18T12:02:00.111563+0000","last_became_peered":"2023-10-18T12:02:00.111563+0000","last_unstale":"2023-10-18T12:02:00.112368+0000","last_undegraded":"2023-10-18T12:02:00.112368+0000","last_fullsized":"2023-10-18T12:02:00.112368+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:18.470466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089249+0000","last_change":"2023-10-18T12:01:59.077726+0000","last_active":"2023-10-18T12:02:00.089249+0000","last_peered":"2023-10-18T12:02:00.089249+0000","last_clean":"2023-10-18T12:02:00.089249+0000","last_became_active":"2023-10-18T12:01:59.077387+0000","last_became_peered":"2023-10-18T12:01:59.077387+0000","last_unstale":"2023-10-18T12:02:00.089249+0000","last_undegraded":"2023-10-18T12:02:00.089249+0000","last_fullsized":"2023-10-18T12:02:00.089249+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:34.750918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.093062+0000","last_change":"2023-10-18T12:02:00.091016+0000","last_active":"2023-10-18T12:02:00.093062+0000","last_peered":"2023-10-18T12:02:00.093062+0000","last_clean":"2023-10-18T12:02:00.093062+0000","last_became_active":"2023-10-18T12:02:00.090761+0000","last_became_peered":"2023-10-18T12:02:00.090761+0000","last_unstale":"2023-10-18T12:02:00.093062+0000","last_undegraded":"2023-10-18T12:02:00.093062+0000","last_fullsized":"2023-10-18T12:02:00.093062+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:58.262632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.091573+0000","last_change":"2023-10-18T12:01:59.078336+0000","last_active":"2023-10-18T12:02:00.091573+0000","last_peered":"2023-10-18T12:02:00.091573+0000","last_clean":"2023-10-18T12:02:00.091573+0000","last_became_active":"2023-10-18T12:01:59.078049+0000","last_became_peered":"2023-10-18T12:01:59.078049+0000","last_unstale":"2023-10-18T12:02:00.091573+0000","last_undegraded":"2023-10-18T12:02:00.091573+0000","last_fullsized":"2023-10-18T12:02:00.091573+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:30.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.092069+0000","last_change":"2023-10-18T12:02:00.088499+0000","last_active":"2023-10-18T12:02:00.092069+0000","last_peered":"2023-10-18T12:02:00.092069+0000","last_clean":"2023-10-18T12:02:00.092069+0000","last_became_active":"2023-10-18T12:02:00.088259+0000","last_became_peered":"2023-10-18T12:02:00.088259+0000","last_unstale":"2023-10-18T12:02:00.092069+0000","last_undegraded":"2023-10-18T12:02:00.092069+0000","last_fullsized":"2023-10-18T12:02:00.092069+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:44.358725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.078907+0000","last_change":"2023-10-18T12:01:59.076700+0000","last_active":"2023-10-18T12:02:00.078907+0000","last_peered":"2023-10-18T12:02:00.078907+0000","last_clean":"2023-10-18T12:02:00.078907+0000","last_became_active":"2023-10-18T12:01:59.076469+0000","last_became_peered":"2023-10-18T12:01:59.076469+0000","last_unstale":"2023-10-18T12:02:00.078907+0000","last_undegraded":"2023-10-18T12:02:00.078907+0000","last_fullsized":"2023-10-18T12:02:00.078907+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:42:25.381773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.098794+0000","last_change":"2023-10-18T12:02:00.098160+0000","last_active":"2023-10-18T12:02:00.098794+0000","last_peered":"2023-10-18T12:02:00.098794+0000","last_clean":"2023-10-18T12:02:00.098794+0000","last_became_active":"2023-10-18T12:02:00.097849+0000","last_became_peered":"2023-10-18T12:02:00.097849+0000","last_unstale":"2023-10-18T12:02:00.098794+0000","last_undegraded":"2023-10-18T12:02:00.098794+0000","last_fullsized":"2023-10-18T12:02:00.098794+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:44.083388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.078456+0000","last_change":"2023-10-18T12:01:59.075883+0000","last_active":"2023-10-18T12:02:00.078456+0000","last_peered":"2023-10-18T12:02:00.078456+0000","last_clean":"2023-10-18T12:02:00.078456+0000","last_became_active":"2023-10-18T12:01:59.075650+0000","last_became_peered":"2023-10-18T12:01:59.075650+0000","last_unstale":"2023-10-18T12:02:00.078456+0000","last_undegraded":"2023-10-18T12:02:00.078456+0000","last_fullsized":"2023-10-18T12:02:00.078456+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:48.682066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096056+0000","last_change":"2023-10-18T12:02:00.090267+0000","last_active":"2023-10-18T12:02:00.096056+0000","last_peered":"2023-10-18T12:02:00.096056+0000","last_clean":"2023-10-18T12:02:00.096056+0000","last_became_active":"2023-10-18T12:02:00.089992+0000","last_became_peered":"2023-10-18T12:02:00.089992+0000","last_unstale":"2023-10-18T12:02:00.096056+0000","last_undegraded":"2023-10-18T12:02:00.096056+0000","last_fullsized":"2023-10-18T12:02:00.096056+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:02:08.857034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085759+0000","last_change":"2023-10-18T12:01:59.074639+0000","last_active":"2023-10-18T12:02:00.085759+0000","last_peered":"2023-10-18T12:02:00.085759+0000","last_clean":"2023-10-18T12:02:00.085759+0000","last_became_active":"2023-10-18T12:01:59.074384+0000","last_became_peered":"2023-10-18T12:01:59.074384+0000","last_unstale":"2023-10-18T12:02:00.085759+0000","last_undegraded":"2023-10-18T12:02:00.085759+0000","last_fullsized":"2023-10-18T12:02:00.085759+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:40.364238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112266+0000","last_change":"2023-10-18T12:02:00.111703+0000","last_active":"2023-10-18T12:02:00.112266+0000","last_peered":"2023-10-18T12:02:00.112266+0000","last_clean":"2023-10-18T12:02:00.112266+0000","last_became_active":"2023-10-18T12:02:00.111355+0000","last_became_peered":"2023-10-18T12:02:00.111355+0000","last_unstale":"2023-10-18T12:02:00.112266+0000","last_undegraded":"2023-10-18T12:02:00.112266+0000","last_fullsized":"2023-10-18T12:02:00.112266+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:40:03.465179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079098+0000","last_change":"2023-10-18T12:01:59.069553+0000","last_active":"2023-10-18T12:02:00.079098+0000","last_peered":"2023-10-18T12:02:00.079098+0000","last_clean":"2023-10-18T12:02:00.079098+0000","last_became_active":"2023-10-18T12:01:59.069166+0000","last_became_peered":"2023-10-18T12:01:59.069166+0000","last_unstale":"2023-10-18T12:02:00.079098+0000","last_undegraded":"2023-10-18T12:02:00.079098+0000","last_fullsized":"2023-10-18T12:02:00.079098+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:12.794683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.098414+0000","last_change":"2023-10-18T12:02:00.097948+0000","last_active":"2023-10-18T12:02:00.098414+0000","last_peered":"2023-10-18T12:02:00.098414+0000","last_clean":"2023-10-18T12:02:00.098414+0000","last_became_active":"2023-10-18T12:02:00.097604+0000","last_became_peered":"2023-10-18T12:02:00.097604+0000","last_unstale":"2023-10-18T12:02:00.098414+0000","last_undegraded":"2023-10-18T12:02:00.098414+0000","last_fullsized":"2023-10-18T12:02:00.098414+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:46.692227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077824+0000","last_change":"2023-10-18T12:01:59.065006+0000","last_active":"2023-10-18T12:02:00.077824+0000","last_peered":"2023-10-18T12:02:00.077824+0000","last_clean":"2023-10-18T12:02:00.077824+0000","last_became_active":"2023-10-18T12:01:59.064756+0000","last_became_peered":"2023-10-18T12:01:59.064756+0000","last_unstale":"2023-10-18T12:02:00.077824+0000","last_undegraded":"2023-10-18T12:02:00.077824+0000","last_fullsized":"2023-10-18T12:02:00.077824+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:10.158824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087361+0000","last_change":"2023-10-18T12:02:00.084599+0000","last_active":"2023-10-18T12:02:00.087361+0000","last_peered":"2023-10-18T12:02:00.087361+0000","last_clean":"2023-10-18T12:02:00.087361+0000","last_became_active":"2023-10-18T12:02:00.083594+0000","last_became_peered":"2023-10-18T12:02:00.083594+0000","last_unstale":"2023-10-18T12:02:00.087361+0000","last_undegraded":"2023-10-18T12:02:00.087361+0000","last_fullsized":"2023-10-18T12:02:00.087361+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:48.671597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.081208+0000","last_change":"2023-10-18T12:01:59.071925+0000","last_active":"2023-10-18T12:02:00.081208+0000","last_peered":"2023-10-18T12:02:00.081208+0000","last_clean":"2023-10-18T12:02:00.081208+0000","last_became_active":"2023-10-18T12:01:59.071637+0000","last_became_peered":"2023-10-18T12:01:59.071637+0000","last_unstale":"2023-10-18T12:02:00.081208+0000","last_undegraded":"2023-10-18T12:02:00.081208+0000","last_fullsized":"2023-10-18T12:02:00.081208+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:47.576118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096386+0000","last_change":"2023-10-18T12:02:00.091291+0000","last_active":"2023-10-18T12:02:00.096386+0000","last_peered":"2023-10-18T12:02:00.096386+0000","last_clean":"2023-10-18T12:02:00.096386+0000","last_became_active":"2023-10-18T12:02:00.091009+0000","last_became_peered":"2023-10-18T12:02:00.091009+0000","last_unstale":"2023-10-18T12:02:00.096386+0000","last_undegraded":"2023-10-18T12:02:00.096386+0000","last_fullsized":"2023-10-18T12:02:00.096386+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:18.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079818+0000","last_change":"2023-10-18T12:01:59.073955+0000","last_active":"2023-10-18T12:02:00.079818+0000","last_peered":"2023-10-18T12:02:00.079818+0000","last_clean":"2023-10-18T12:02:00.079818+0000","last_became_active":"2023-10-18T12:01:59.073704+0000","last_became_peered":"2023-10-18T12:01:59.073704+0000","last_unstale":"2023-10-18T12:02:00.079818+0000","last_undegraded":"2023-10-18T12:02:00.079818+0000","last_fullsized":"2023-10-18T12:02:00.079818+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:22:06.437675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111027+0000","last_change":"2023-10-18T12:02:00.090490+0000","last_active":"2023-10-18T12:02:00.111027+0000","last_peered":"2023-10-18T12:02:00.111027+0000","last_clean":"2023-10-18T12:02:00.111027+0000","last_became_active":"2023-10-18T12:02:00.090221+0000","last_became_peered":"2023-10-18T12:02:00.090221+0000","last_unstale":"2023-10-18T12:02:00.111027+0000","last_undegraded":"2023-10-18T12:02:00.111027+0000","last_fullsized":"2023-10-18T12:02:00.111027+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:49.093168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.078064+0000","last_change":"2023-10-18T12:01:59.065597+0000","last_active":"2023-10-18T12:02:00.078064+0000","last_peered":"2023-10-18T12:02:00.078064+0000","last_clean":"2023-10-18T12:02:00.078064+0000","last_became_active":"2023-10-18T12:01:59.065362+0000","last_became_peered":"2023-10-18T12:01:59.065362+0000","last_unstale":"2023-10-18T12:02:00.078064+0000","last_undegraded":"2023-10-18T12:02:00.078064+0000","last_fullsized":"2023-10-18T12:02:00.078064+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:02.529048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.083403+0000","last_change":"2023-10-18T12:02:00.082234+0000","last_active":"2023-10-18T12:02:00.083403+0000","last_peered":"2023-10-18T12:02:00.083403+0000","last_clean":"2023-10-18T12:02:00.083403+0000","last_became_active":"2023-10-18T12:02:00.081951+0000","last_became_peered":"2023-10-18T12:02:00.081951+0000","last_unstale":"2023-10-18T12:02:00.083403+0000","last_undegraded":"2023-10-18T12:02:00.083403+0000","last_fullsized":"2023-10-18T12:02:00.083403+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:48.171262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079727+0000","last_change":"2023-10-18T12:01:59.077426+0000","last_active":"2023-10-18T12:02:00.079727+0000","last_peered":"2023-10-18T12:02:00.079727+0000","last_clean":"2023-10-18T12:02:00.079727+0000","last_became_active":"2023-10-18T12:01:59.077066+0000","last_became_peered":"2023-10-18T12:01:59.077066+0000","last_unstale":"2023-10-18T12:02:00.079727+0000","last_undegraded":"2023-10-18T12:02:00.079727+0000","last_fullsized":"2023-10-18T12:02:00.079727+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:31:40.916034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084755+0000","last_change":"2023-10-18T12:02:00.084146+0000","last_active":"2023-10-18T12:02:00.084755+0000","last_peered":"2023-10-18T12:02:00.084755+0000","last_clean":"2023-10-18T12:02:00.084755+0000","last_became_active":"2023-10-18T12:02:00.083888+0000","last_became_peered":"2023-10-18T12:02:00.083888+0000","last_unstale":"2023-10-18T12:02:00.084755+0000","last_undegraded":"2023-10-18T12:02:00.084755+0000","last_fullsized":"2023-10-18T12:02:00.084755+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:58.934212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087173+0000","last_change":"2023-10-18T12:01:59.076212+0000","last_active":"2023-10-18T12:02:00.087173+0000","last_peered":"2023-10-18T12:02:00.087173+0000","last_clean":"2023-10-18T12:02:00.087173+0000","last_became_active":"2023-10-18T12:01:59.075966+0000","last_became_peered":"2023-10-18T12:01:59.075966+0000","last_unstale":"2023-10-18T12:02:00.087173+0000","last_undegraded":"2023-10-18T12:02:00.087173+0000","last_fullsized":"2023-10-18T12:02:00.087173+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:20:48.512427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111516+0000","last_change":"2023-10-18T12:02:00.092997+0000","last_active":"2023-10-18T12:02:00.111516+0000","last_peered":"2023-10-18T12:02:00.111516+0000","last_clean":"2023-10-18T12:02:00.111516+0000","last_became_active":"2023-10-18T12:02:00.092749+0000","last_became_peered":"2023-10-18T12:02:00.092749+0000","last_unstale":"2023-10-18T12:02:00.111516+0000","last_undegraded":"2023-10-18T12:02:00.111516+0000","last_fullsized":"2023-10-18T12:02:00.111516+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:54:27.223567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077492+0000","last_change":"2023-10-18T12:01:59.074924+0000","last_active":"2023-10-18T12:02:00.077492+0000","last_peered":"2023-10-18T12:02:00.077492+0000","last_clean":"2023-10-18T12:02:00.077492+0000","last_became_active":"2023-10-18T12:01:59.074699+0000","last_became_peered":"2023-10-18T12:01:59.074699+0000","last_unstale":"2023-10-18T12:02:00.077492+0000","last_undegraded":"2023-10-18T12:02:00.077492+0000","last_fullsized":"2023-10-18T12:02:00.077492+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:14.110802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.083910+0000","last_change":"2023-10-18T12:02:00.082860+0000","last_active":"2023-10-18T12:02:00.083910+0000","last_peered":"2023-10-18T12:02:00.083910+0000","last_clean":"2023-10-18T12:02:00.083910+0000","last_became_active":"2023-10-18T12:02:00.082624+0000","last_became_peered":"2023-10-18T12:02:00.082624+0000","last_unstale":"2023-10-18T12:02:00.083910+0000","last_undegraded":"2023-10-18T12:02:00.083910+0000","last_fullsized":"2023-10-18T12:02:00.083910+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:38.446384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085713+0000","last_change":"2023-10-18T12:01:59.076827+0000","last_active":"2023-10-18T12:02:00.085713+0000","last_peered":"2023-10-18T12:02:00.085713+0000","last_clean":"2023-10-18T12:02:00.085713+0000","last_became_active":"2023-10-18T12:01:59.076580+0000","last_became_peered":"2023-10-18T12:01:59.076580+0000","last_unstale":"2023-10-18T12:02:00.085713+0000","last_undegraded":"2023-10-18T12:02:00.085713+0000","last_fullsized":"2023-10-18T12:02:00.085713+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:35.609656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087253+0000","last_change":"2023-10-18T12:02:00.084183+0000","last_active":"2023-10-18T12:02:00.087253+0000","last_peered":"2023-10-18T12:02:00.087253+0000","last_clean":"2023-10-18T12:02:00.087253+0000","last_became_active":"2023-10-18T12:02:00.083767+0000","last_became_peered":"2023-10-18T12:02:00.083767+0000","last_unstale":"2023-10-18T12:02:00.087253+0000","last_undegraded":"2023-10-18T12:02:00.087253+0000","last_fullsized":"2023-10-18T12:02:00.087253+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:37.215637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085974+0000","last_change":"2023-10-18T12:01:59.072059+0000","last_active":"2023-10-18T12:02:00.085974+0000","last_peered":"2023-10-18T12:02:00.085974+0000","last_clean":"2023-10-18T12:02:00.085974+0000","last_became_active":"2023-10-18T12:01:59.071801+0000","last_became_peered":"2023-10-18T12:01:59.071801+0000","last_unstale":"2023-10-18T12:02:00.085974+0000","last_undegraded":"2023-10-18T12:02:00.085974+0000","last_fullsized":"2023-10-18T12:02:00.085974+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:53.393773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087391+0000","last_change":"2023-10-18T12:02:00.085585+0000","last_active":"2023-10-18T12:02:00.087391+0000","last_peered":"2023-10-18T12:02:00.087391+0000","last_clean":"2023-10-18T12:02:00.087391+0000","last_became_active":"2023-10-18T12:02:00.083910+0000","last_became_peered":"2023-10-18T12:02:00.083910+0000","last_unstale":"2023-10-18T12:02:00.087391+0000","last_undegraded":"2023-10-18T12:02:00.087391+0000","last_fullsized":"2023-10-18T12:02:00.087391+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:44:17.626184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084808+0000","last_change":"2023-10-18T12:01:59.075302+0000","last_active":"2023-10-18T12:02:00.084808+0000","last_peered":"2023-10-18T12:02:00.084808+0000","last_clean":"2023-10-18T12:02:00.084808+0000","last_became_active":"2023-10-18T12:01:59.075052+0000","last_became_peered":"2023-10-18T12:01:59.075052+0000","last_unstale":"2023-10-18T12:02:00.084808+0000","last_undegraded":"2023-10-18T12:02:00.084808+0000","last_fullsized":"2023-10-18T12:02:00.084808+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:04:18.809260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084461+0000","last_change":"2023-10-18T12:02:00.082223+0000","last_active":"2023-10-18T12:02:00.084461+0000","last_peered":"2023-10-18T12:02:00.084461+0000","last_clean":"2023-10-18T12:02:00.084461+0000","last_became_active":"2023-10-18T12:02:00.081967+0000","last_became_peered":"2023-10-18T12:02:00.081967+0000","last_unstale":"2023-10-18T12:02:00.084461+0000","last_undegraded":"2023-10-18T12:02:00.084461+0000","last_fullsized":"2023-10-18T12:02:00.084461+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:33.442723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.078644+0000","last_change":"2023-10-18T12:01:59.066746+0000","last_active":"2023-10-18T12:02:00.078644+0000","last_peered":"2023-10-18T12:02:00.078644+0000","last_clean":"2023-10-18T12:02:00.078644+0000","last_became_active":"2023-10-18T12:01:59.066487+0000","last_became_peered":"2023-10-18T12:01:59.066487+0000","last_unstale":"2023-10-18T12:02:00.078644+0000","last_undegraded":"2023-10-18T12:02:00.078644+0000","last_fullsized":"2023-10-18T12:02:00.078644+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:09:04.803564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112269+0000","last_change":"2023-10-18T12:02:00.111686+0000","last_active":"2023-10-18T12:02:00.112269+0000","last_peered":"2023-10-18T12:02:00.112269+0000","last_clean":"2023-10-18T12:02:00.112269+0000","last_became_active":"2023-10-18T12:02:00.111422+0000","last_became_peered":"2023-10-18T12:02:00.111422+0000","last_unstale":"2023-10-18T12:02:00.112269+0000","last_undegraded":"2023-10-18T12:02:00.112269+0000","last_fullsized":"2023-10-18T12:02:00.112269+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:19.973149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079200+0000","last_change":"2023-10-18T12:01:59.074367+0000","last_active":"2023-10-18T12:02:00.079200+0000","last_peered":"2023-10-18T12:02:00.079200+0000","last_clean":"2023-10-18T12:02:00.079200+0000","last_became_active":"2023-10-18T12:01:59.073893+0000","last_became_peered":"2023-10-18T12:01:59.073893+0000","last_unstale":"2023-10-18T12:02:00.079200+0000","last_undegraded":"2023-10-18T12:02:00.079200+0000","last_fullsized":"2023-10-18T12:02:00.079200+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:22.031460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084530+0000","last_change":"2023-10-18T12:02:00.082876+0000","last_active":"2023-10-18T12:02:00.084530+0000","last_peered":"2023-10-18T12:02:00.084530+0000","last_clean":"2023-10-18T12:02:00.084530+0000","last_became_active":"2023-10-18T12:02:00.082612+0000","last_became_peered":"2023-10-18T12:02:00.082612+0000","last_unstale":"2023-10-18T12:02:00.084530+0000","last_undegraded":"2023-10-18T12:02:00.084530+0000","last_fullsized":"2023-10-18T12:02:00.084530+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:33.819373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079021+0000","last_change":"2023-10-18T12:01:59.069156+0000","last_active":"2023-10-18T12:02:00.079021+0000","last_peered":"2023-10-18T12:02:00.079021+0000","last_clean":"2023-10-18T12:02:00.079021+0000","last_became_active":"2023-10-18T12:01:59.068896+0000","last_became_peered":"2023-10-18T12:01:59.068896+0000","last_unstale":"2023-10-18T12:02:00.079021+0000","last_undegraded":"2023-10-18T12:02:00.079021+0000","last_fullsized":"2023-10-18T12:02:00.079021+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:52.440185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.083851+0000","last_change":"2023-10-18T12:02:00.082802+0000","last_active":"2023-10-18T12:02:00.083851+0000","last_peered":"2023-10-18T12:02:00.083851+0000","last_clean":"2023-10-18T12:02:00.083851+0000","last_became_active":"2023-10-18T12:02:00.082546+0000","last_became_peered":"2023-10-18T12:02:00.082546+0000","last_unstale":"2023-10-18T12:02:00.083851+0000","last_undegraded":"2023-10-18T12:02:00.083851+0000","last_fullsized":"2023-10-18T12:02:00.083851+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:06.242977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077825+0000","last_change":"2023-10-18T12:01:59.072818+0000","last_active":"2023-10-18T12:02:00.077825+0000","last_peered":"2023-10-18T12:02:00.077825+0000","last_clean":"2023-10-18T12:02:00.077825+0000","last_became_active":"2023-10-18T12:01:59.072567+0000","last_became_peered":"2023-10-18T12:01:59.072567+0000","last_unstale":"2023-10-18T12:02:00.077825+0000","last_undegraded":"2023-10-18T12:02:00.077825+0000","last_fullsized":"2023-10-18T12:02:00.077825+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:02.572353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112293+0000","last_change":"2023-10-18T12:02:00.111828+0000","last_active":"2023-10-18T12:02:00.112293+0000","last_peered":"2023-10-18T12:02:00.112293+0000","last_clean":"2023-10-18T12:02:00.112293+0000","last_became_active":"2023-10-18T12:02:00.111348+0000","last_became_peered":"2023-10-18T12:02:00.111348+0000","last_unstale":"2023-10-18T12:02:00.112293+0000","last_undegraded":"2023-10-18T12:02:00.112293+0000","last_fullsized":"2023-10-18T12:02:00.112293+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:20:49.516861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077919+0000","last_change":"2023-10-18T12:01:59.070180+0000","last_active":"2023-10-18T12:02:00.077919+0000","last_peered":"2023-10-18T12:02:00.077919+0000","last_clean":"2023-10-18T12:02:00.077919+0000","last_became_active":"2023-10-18T12:01:59.069908+0000","last_became_peered":"2023-10-18T12:01:59.069908+0000","last_unstale":"2023-10-18T12:02:00.077919+0000","last_undegraded":"2023-10-18T12:02:00.077919+0000","last_fullsized":"2023-10-18T12:02:00.077919+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:20.216002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097259+0000","last_change":"2023-10-18T12:02:00.095196+0000","last_active":"2023-10-18T12:02:00.097259+0000","last_peered":"2023-10-18T12:02:00.097259+0000","last_clean":"2023-10-18T12:02:00.097259+0000","last_became_active":"2023-10-18T12:02:00.094955+0000","last_became_peered":"2023-10-18T12:02:00.094955+0000","last_unstale":"2023-10-18T12:02:00.097259+0000","last_undegraded":"2023-10-18T12:02:00.097259+0000","last_fullsized":"2023-10-18T12:02:00.097259+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:16.719297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079551+0000","last_change":"2023-10-18T12:01:59.072866+0000","last_active":"2023-10-18T12:02:00.079551+0000","last_peered":"2023-10-18T12:02:00.079551+0000","last_clean":"2023-10-18T12:02:00.079551+0000","last_became_active":"2023-10-18T12:01:59.072623+0000","last_became_peered":"2023-10-18T12:01:59.072623+0000","last_unstale":"2023-10-18T12:02:00.079551+0000","last_undegraded":"2023-10-18T12:02:00.079551+0000","last_fullsized":"2023-10-18T12:02:00.079551+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:32:33.134330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087231+0000","last_change":"2023-10-18T12:02:00.084012+0000","last_active":"2023-10-18T12:02:00.087231+0000","last_peered":"2023-10-18T12:02:00.087231+0000","last_clean":"2023-10-18T12:02:00.087231+0000","last_became_active":"2023-10-18T12:02:00.083435+0000","last_became_peered":"2023-10-18T12:02:00.083435+0000","last_unstale":"2023-10-18T12:02:00.087231+0000","last_undegraded":"2023-10-18T12:02:00.087231+0000","last_fullsized":"2023-10-18T12:02:00.087231+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:36.863876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089545+0000","last_change":"2023-10-18T12:01:59.074836+0000","last_active":"2023-10-18T12:02:00.089545+0000","last_peered":"2023-10-18T12:02:00.089545+0000","last_clean":"2023-10-18T12:02:00.089545+0000","last_became_active":"2023-10-18T12:01:59.074579+0000","last_became_peered":"2023-10-18T12:01:59.074579+0000","last_unstale":"2023-10-18T12:02:00.089545+0000","last_undegraded":"2023-10-18T12:02:00.089545+0000","last_fullsized":"2023-10-18T12:02:00.089545+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:55.455890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.093952+0000","last_change":"2023-10-18T12:02:00.092675+0000","last_active":"2023-10-18T12:02:00.093952+0000","last_peered":"2023-10-18T12:02:00.093952+0000","last_clean":"2023-10-18T12:02:00.093952+0000","last_became_active":"2023-10-18T12:02:00.092430+0000","last_became_peered":"2023-10-18T12:02:00.092430+0000","last_unstale":"2023-10-18T12:02:00.093952+0000","last_undegraded":"2023-10-18T12:02:00.093952+0000","last_fullsized":"2023-10-18T12:02:00.093952+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:54.264848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077604+0000","last_change":"2023-10-18T12:01:59.073960+0000","last_active":"2023-10-18T12:02:00.077604+0000","last_peered":"2023-10-18T12:02:00.077604+0000","last_clean":"2023-10-18T12:02:00.077604+0000","last_became_active":"2023-10-18T12:01:59.073561+0000","last_became_peered":"2023-10-18T12:01:59.073561+0000","last_unstale":"2023-10-18T12:02:00.077604+0000","last_undegraded":"2023-10-18T12:02:00.077604+0000","last_fullsized":"2023-10-18T12:02:00.077604+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:41.990902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096404+0000","last_change":"2023-10-18T12:02:00.091497+0000","last_active":"2023-10-18T12:02:00.096404+0000","last_peered":"2023-10-18T12:02:00.096404+0000","last_clean":"2023-10-18T12:02:00.096404+0000","last_became_active":"2023-10-18T12:02:00.090667+0000","last_became_peered":"2023-10-18T12:02:00.090667+0000","last_unstale":"2023-10-18T12:02:00.096404+0000","last_undegraded":"2023-10-18T12:02:00.096404+0000","last_fullsized":"2023-10-18T12:02:00.096404+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:57.383536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077776+0000","last_change":"2023-10-18T12:01:59.073576+0000","last_active":"2023-10-18T12:02:00.077776+0000","last_peered":"2023-10-18T12:02:00.077776+0000","last_clean":"2023-10-18T12:02:00.077776+0000","last_became_active":"2023-10-18T12:01:59.073353+0000","last_became_peered":"2023-10-18T12:01:59.073353+0000","last_unstale":"2023-10-18T12:02:00.077776+0000","last_undegraded":"2023-10-18T12:02:00.077776+0000","last_fullsized":"2023-10-18T12:02:00.077776+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:06.264882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089016+0000","last_change":"2023-10-18T12:02:00.081823+0000","last_active":"2023-10-18T12:02:00.089016+0000","last_peered":"2023-10-18T12:02:00.089016+0000","last_clean":"2023-10-18T12:02:00.089016+0000","last_became_active":"2023-10-18T12:02:00.081546+0000","last_became_peered":"2023-10-18T12:02:00.081546+0000","last_unstale":"2023-10-18T12:02:00.089016+0000","last_undegraded":"2023-10-18T12:02:00.089016+0000","last_fullsized":"2023-10-18T12:02:00.089016+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:26.740564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.086124+0000","last_change":"2023-10-18T12:01:59.076858+0000","last_active":"2023-10-18T12:02:00.086124+0000","last_peered":"2023-10-18T12:02:00.086124+0000","last_clean":"2023-10-18T12:02:00.086124+0000","last_became_active":"2023-10-18T12:01:59.076595+0000","last_became_peered":"2023-10-18T12:01:59.076595+0000","last_unstale":"2023-10-18T12:02:00.086124+0000","last_undegraded":"2023-10-18T12:02:00.086124+0000","last_fullsized":"2023-10-18T12:02:00.086124+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:23.276520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.090014+0000","last_change":"2023-10-18T12:02:00.088655+0000","last_active":"2023-10-18T12:02:00.090014+0000","last_peered":"2023-10-18T12:02:00.090014+0000","last_clean":"2023-10-18T12:02:00.090014+0000","last_became_active":"2023-10-18T12:02:00.088403+0000","last_became_peered":"2023-10-18T12:02:00.088403+0000","last_unstale":"2023-10-18T12:02:00.090014+0000","last_undegraded":"2023-10-18T12:02:00.090014+0000","last_fullsized":"2023-10-18T12:02:00.090014+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:33.903058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079127+0000","last_change":"2023-10-18T12:01:59.076067+0000","last_active":"2023-10-18T12:02:00.079127+0000","last_peered":"2023-10-18T12:02:00.079127+0000","last_clean":"2023-10-18T12:02:00.079127+0000","last_became_active":"2023-10-18T12:01:59.075817+0000","last_became_peered":"2023-10-18T12:01:59.075817+0000","last_unstale":"2023-10-18T12:02:00.079127+0000","last_undegraded":"2023-10-18T12:02:00.079127+0000","last_fullsized":"2023-10-18T12:02:00.079127+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:10.825855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097320+0000","last_change":"2023-10-18T12:02:00.093215+0000","last_active":"2023-10-18T12:02:00.097320+0000","last_peered":"2023-10-18T12:02:00.097320+0000","last_clean":"2023-10-18T12:02:00.097320+0000","last_became_active":"2023-10-18T12:02:00.092886+0000","last_became_peered":"2023-10-18T12:02:00.092886+0000","last_unstale":"2023-10-18T12:02:00.097320+0000","last_undegraded":"2023-10-18T12:02:00.097320+0000","last_fullsized":"2023-10-18T12:02:00.097320+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:11.928499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085999+0000","last_change":"2023-10-18T12:01:59.077679+0000","last_active":"2023-10-18T12:02:00.085999+0000","last_peered":"2023-10-18T12:02:00.085999+0000","last_clean":"2023-10-18T12:02:00.085999+0000","last_became_active":"2023-10-18T12:01:59.077429+0000","last_became_peered":"2023-10-18T12:01:59.077429+0000","last_unstale":"2023-10-18T12:02:00.085999+0000","last_undegraded":"2023-10-18T12:02:00.085999+0000","last_fullsized":"2023-10-18T12:02:00.085999+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:56.661800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.095189+0000","last_change":"2023-10-18T12:02:00.092818+0000","last_active":"2023-10-18T12:02:00.095189+0000","last_peered":"2023-10-18T12:02:00.095189+0000","last_clean":"2023-10-18T12:02:00.095189+0000","last_became_active":"2023-10-18T12:02:00.092501+0000","last_became_peered":"2023-10-18T12:02:00.092501+0000","last_unstale":"2023-10-18T12:02:00.095189+0000","last_undegraded":"2023-10-18T12:02:00.095189+0000","last_fullsized":"2023-10-18T12:02:00.095189+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:16.719297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077971+0000","last_change":"2023-10-18T12:01:59.074836+0000","last_active":"2023-10-18T12:02:00.077971+0000","last_peered":"2023-10-18T12:02:00.077971+0000","last_clean":"2023-10-18T12:02:00.077971+0000","last_became_active":"2023-10-18T12:01:59.074396+0000","last_became_peered":"2023-10-18T12:01:59.074396+0000","last_unstale":"2023-10-18T12:02:00.077971+0000","last_undegraded":"2023-10-18T12:02:00.077971+0000","last_fullsized":"2023-10-18T12:02:00.077971+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:32.702122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.095402+0000","last_change":"2023-10-18T12:02:00.090143+0000","last_active":"2023-10-18T12:02:00.095402+0000","last_peered":"2023-10-18T12:02:00.095402+0000","last_clean":"2023-10-18T12:02:00.095402+0000","last_became_active":"2023-10-18T12:02:00.089862+0000","last_became_peered":"2023-10-18T12:02:00.089862+0000","last_unstale":"2023-10-18T12:02:00.095402+0000","last_undegraded":"2023-10-18T12:02:00.095402+0000","last_fullsized":"2023-10-18T12:02:00.095402+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:48.241385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079661+0000","last_change":"2023-10-18T12:01:59.072791+0000","last_active":"2023-10-18T12:02:00.079661+0000","last_peered":"2023-10-18T12:02:00.079661+0000","last_clean":"2023-10-18T12:02:00.079661+0000","last_became_active":"2023-10-18T12:01:59.072560+0000","last_became_peered":"2023-10-18T12:01:59.072560+0000","last_unstale":"2023-10-18T12:02:00.079661+0000","last_undegraded":"2023-10-18T12:02:00.079661+0000","last_fullsized":"2023-10-18T12:02:00.079661+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:00.799261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.092219+0000","last_change":"2023-10-18T12:02:00.090879+0000","last_active":"2023-10-18T12:02:00.092219+0000","last_peered":"2023-10-18T12:02:00.092219+0000","last_clean":"2023-10-18T12:02:00.092219+0000","last_became_active":"2023-10-18T12:02:00.090644+0000","last_became_peered":"2023-10-18T12:02:00.090644+0000","last_unstale":"2023-10-18T12:02:00.092219+0000","last_undegraded":"2023-10-18T12:02:00.092219+0000","last_fullsized":"2023-10-18T12:02:00.092219+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:11:43.273421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079709+0000","last_change":"2023-10-18T12:01:59.072905+0000","last_active":"2023-10-18T12:02:00.079709+0000","last_peered":"2023-10-18T12:02:00.079709+0000","last_clean":"2023-10-18T12:02:00.079709+0000","last_became_active":"2023-10-18T12:01:59.072671+0000","last_became_peered":"2023-10-18T12:01:59.072671+0000","last_unstale":"2023-10-18T12:02:00.079709+0000","last_undegraded":"2023-10-18T12:02:00.079709+0000","last_fullsized":"2023-10-18T12:02:00.079709+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:01:08.939600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096579+0000","last_change":"2023-10-18T12:02:00.093396+0000","last_active":"2023-10-18T12:02:00.096579+0000","last_peered":"2023-10-18T12:02:00.096579+0000","last_clean":"2023-10-18T12:02:00.096579+0000","last_became_active":"2023-10-18T12:02:00.093155+0000","last_became_peered":"2023-10-18T12:02:00.093155+0000","last_unstale":"2023-10-18T12:02:00.096579+0000","last_undegraded":"2023-10-18T12:02:00.096579+0000","last_fullsized":"2023-10-18T12:02:00.096579+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:37.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077686+0000","last_change":"2023-10-18T12:01:59.073803+0000","last_active":"2023-10-18T12:02:00.077686+0000","last_peered":"2023-10-18T12:02:00.077686+0000","last_clean":"2023-10-18T12:02:00.077686+0000","last_became_active":"2023-10-18T12:01:59.073240+0000","last_became_peered":"2023-10-18T12:01:59.073240+0000","last_unstale":"2023-10-18T12:02:00.077686+0000","last_undegraded":"2023-10-18T12:02:00.077686+0000","last_fullsized":"2023-10-18T12:02:00.077686+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:04.148346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089794+0000","last_change":"2023-10-18T12:02:00.087413+0000","last_active":"2023-10-18T12:02:00.089794+0000","last_peered":"2023-10-18T12:02:00.089794+0000","last_clean":"2023-10-18T12:02:00.089794+0000","last_became_active":"2023-10-18T12:02:00.087086+0000","last_became_peered":"2023-10-18T12:02:00.087086+0000","last_unstale":"2023-10-18T12:02:00.089794+0000","last_undegraded":"2023-10-18T12:02:00.089794+0000","last_fullsized":"2023-10-18T12:02:00.089794+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:41.135253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"261'3062","reported_seq":4079,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.253515+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:02:00.253515+0000","last_peered":"2023-10-18T12:02:00.253515+0000","last_clean":"2023-10-18T12:02:00.253515+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:02:00.253515+0000","last_undegraded":"2023-10-18T12:02:00.253515+0000","last_fullsized":"2023-10-18T12:02:00.253515+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":3062,"log_dups_size":0,"ondisk_log_size":3062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2500,"num_read_kb":5242,"num_write":649,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2500,"num_read_kb":5242,"num_write":649,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3062,"ondisk_log_size":3062,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576008,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48860,"kb_used_data":3792,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322980,"statfs":{"total":96636764160,"available":96586731520,"internally_reserved":0,"allocated":3883008,"data_stored":2290723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.80800000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.56200000000000006,"15min":0.60099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":1.363},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.86399999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56100000000000005,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.90000000000000002,"15min":1.298},"last":0.375}]},{"osd":3,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71899999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":1.5249999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.81299999999999994},"last":0.29899999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.503,"15min":0.5},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.27000000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70699999999999996,"5min":0.86599999999999999,"15min":1.478},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.83599999999999997,"15min":0.874},"last":0.34899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42720,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329120,"statfs":{"total":96636764160,"available":96593018880,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.30499999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46100000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.38600000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.373},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48284,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323556,"statfs":{"total":96636764160,"available":96587321344,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.33800000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":1.101,"15min":1.1539999999999999},"last":0.308},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.71099999999999997,"15min":0.751},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.29299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72699999999999998,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81399999999999995,"15min":1},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53000000000000003,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.438}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.749,"5min":0.82999999999999996,"15min":1.01},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.32500000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.255,"15min":0.255},"max":{"1min":0.78800000000000003,"5min":0.83699999999999997,"15min":1.101},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.45100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576008,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48284,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323556,"statfs":{"total":96636764160,"available":96587321344,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.60999999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.81499999999999995,"5min":0.872,"15min":1.123},"last":0.50700000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73299999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.435}]},{"osd":3,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.44900000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.755,"5min":1.018,"15min":1.018},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.88900000000000001,"15min":1.05},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.496}]}]},{"osd":3,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48284,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323556,"statfs":{"total":96636764160,"available":96587321344,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.23999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.378,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.39300000000000002,"15min":0.40699999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.38400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.39900000000000002,"15min":0.40600000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44900000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.313},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.432,"5min":1.899,"15min":1.899},"last":0.41299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.42899999999999999,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.434,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.87,"5min":1.034,"15min":1.052},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.434}]},{"osd":5,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":1.522,"15min":1.522},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.91100000000000003,"15min":1.026},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82299999999999995,"5min":1.01,"15min":1.01},"last":0.59899999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.129,"15min":1.129},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0649999999999999},"last":0.625}]}]},{"osd":2,"up_from":13,"seq":55834576008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64924,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306916,"statfs":{"total":96636764160,"available":96570281984,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.52300000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.40500000000000003,"15min":0.41599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.35099999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.61499999999999999,"15min":0.624},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.374,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.63100000000000001,"15min":0.746},"last":0.29399999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55300000000000005,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.311}]},{"osd":4,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86299999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66300000000000003,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.087},"last":0.36199999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.85199999999999998,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63400000000000001,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.95399999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":1.0609999999999999},"last":0.502},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.434}]}]},{"osd":1,"up_from":13,"seq":55834576008,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64920,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306920,"statfs":{"total":96636764160,"available":96570286080,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.218},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.29399999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.307},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.24099999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.27400000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47499999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.439}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.498},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.42999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57200,"kb_used_data":3804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314640,"statfs":{"total":96636764160,"available":96578191360,"internally_reserved":0,"allocated":3895296,"data_stored":2297874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.245},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.27800000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52200000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.52300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.313}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.51100000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.379}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.96999999999999997,"15min":0.997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.39000000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72399999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.95499999999999996,"15min":0.96399999999999997},"last":0.35399999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64700000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":1.03},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70999999999999996,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.36699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.78000000000000003,"5min":0.997,"15min":0.997},"last":0.43099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:02:03.371 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:02:03.506 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:03.503+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:03.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.505+0000 7f1859a10640 1 -- 172.21.15.125:0/2582584812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18540d1910 msgr2=0x7f18540b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.505+0000 7f1859a10640 1 --2- 172.21.15.125:0/2582584812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18540d1910 0x7f18540b0650 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f184c009be0 tx=0x7f184c02c0c0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.505+0000 7f1859a10640 1 -- 172.21.15.125:0/2582584812 shutdown_connections 2023-10-18T12:02:03.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.505+0000 7f1859a10640 1 --2- 172.21.15.125:0/2582584812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18540d1910 0x7f18540b0650 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.505+0000 7f1859a10640 1 -- 172.21.15.125:0/2582584812 >> 172.21.15.125:0/2582584812 conn(0x7f18540d0d60 msgr2=0x7f18540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:03.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.505+0000 7f1859a10640 1 -- 172.21.15.125:0/2582584812 shutdown_connections 2023-10-18T12:02:03.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.505+0000 7f1859a10640 1 -- 172.21.15.125:0/2582584812 wait complete. 2023-10-18T12:02:03.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.506+0000 7f1859a10640 1 Processor -- start 2023-10-18T12:02:03.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.506+0000 7f1859a10640 1 -- start start 2023-10-18T12:02:03.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f1859a10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18540cf340 0x7f18540caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f1859a10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18540d1910 0x7f18540c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f1859a10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f185415b2a0 0x7f18541f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f1859a10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18541f2320 con 0x7f18540d1910 2023-10-18T12:02:03.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f1859a10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18541f1d60 con 0x7f18540cf340 2023-10-18T12:02:03.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f1859a10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18541f2040 con 0x7f185415b2a0 2023-10-18T12:02:03.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18527fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18540d1910 0x7f18540c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18527fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18540d1910 0x7f18540c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56084/0 (socket says 172.21.15.125:56084) 2023-10-18T12:02:03.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18527fc640 1 -- 172.21.15.125:0/489221638 learned_addr learned my addr 172.21.15.125:0/489221638 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:03.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f1852ffd640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18540cf340 0x7f18540caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18537fe640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f185415b2a0 0x7f18541f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18527fc640 1 -- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f185415b2a0 msgr2=0x7f18541f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18527fc640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f185415b2a0 0x7f18541f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18527fc640 1 -- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18540cf340 msgr2=0x7f18540caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18527fc640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18540cf340 0x7f18540caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.507+0000 7f18527fc640 1 -- 172.21.15.125:0/489221638 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c009840 con 0x7f18540d1910 2023-10-18T12:02:03.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.508+0000 7f18527fc640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18540d1910 0x7f18540c8760 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f184800c430 tx=0x7f184800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:03.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.508+0000 7f183bfff640 1 -- 172.21.15.125:0/489221638 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18480052d0 con 0x7f18540d1910 2023-10-18T12:02:03.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.508+0000 7f183bfff640 1 -- 172.21.15.125:0/489221638 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18480035f0 con 0x7f18540d1910 2023-10-18T12:02:03.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.508+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18540c7f60 con 0x7f18540d1910 2023-10-18T12:02:03.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.508+0000 7f183bfff640 1 -- 172.21.15.125:0/489221638 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848003a00 con 0x7f18540d1910 2023-10-18T12:02:03.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.508+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18540caf50 con 0x7f18540d1910 2023-10-18T12:02:03.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.509+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18540c8480 con 0x7f18540d1910 2023-10-18T12:02:03.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.509+0000 7f183bfff640 1 -- 172.21.15.125:0/489221638 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f1848003b60 con 0x7f18540d1910 2023-10-18T12:02:03.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.510+0000 7f183bfff640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f184006b020 0x7f184006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.510+0000 7f183bfff640 1 -- 172.21.15.125:0/489221638 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f1848089200 con 0x7f18540d1910 2023-10-18T12:02:03.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.510+0000 7f1852ffd640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f184006b020 0x7f184006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.510+0000 7f1852ffd640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f184006b020 0x7f184006d4e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18440011e0 tx=0x7f1844009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:03.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.514+0000 7f183bfff640 1 -- 172.21.15.125:0/489221638 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18540c8480 con 0x7f18540d1910 2023-10-18T12:02:03.683 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.680+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 --> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18540c8480 con 0x7f184006b020 2023-10-18T12:02:03.690 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.688+0000 7f183bfff640 1 -- 172.21.15.125:0/489221638 <== mgr.9734 v2:172.21.15.139:6832/666436799 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289775 (secure 0 0 0) 0x7f18540c8480 con 0x7f184006b020 2023-10-18T12:02:03.693 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:03.695 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:02:03.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f184006b020 msgr2=0x7f184006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.696 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f184006b020 0x7f184006d4e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18440011e0 tx=0x7f1844009040 comp rx=0 tx=0).stop 2023-10-18T12:02:03.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18540d1910 msgr2=0x7f18540c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18540d1910 0x7f18540c8760 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f184800c430 tx=0x7f184800c8f0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 shutdown_connections 2023-10-18T12:02:03.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f185415b2a0 0x7f18541f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.697 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f18540d1910 0x7f18540c8760 unknown :-1 s=CLOSED pgs=3026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f184006b020 0x7f184006d4e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 --2- 172.21.15.125:0/489221638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18540cf340 0x7f18540caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 >> 172.21.15.125:0/489221638 conn(0x7f18540d0d60 msgr2=0x7f18540ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:03.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 shutdown_connections 2023-10-18T12:02:03.698 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.691+0000 7f1859a10640 1 -- 172.21.15.125:0/489221638 wait complete. 2023-10-18T12:02:03.705 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-18T12:02:03.267970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2500,"num_read_kb":5242,"num_write":649,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3062,"ondisk_log_size":3062,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423476,"kb_used_data":26868,"kb_used_omap":12,"kb_used_meta":396467,"kb_avail":754551244,"statfs":{"total":773094113280,"available":772660473856,"internally_reserved":0,"allocated":27512832,"data_stored":14790727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":405982776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.204752"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111113+0000","last_change":"2023-10-18T12:02:00.091567+0000","last_active":"2023-10-18T12:02:00.111113+0000","last_peered":"2023-10-18T12:02:00.111113+0000","last_clean":"2023-10-18T12:02:00.111113+0000","last_became_active":"2023-10-18T12:02:00.091295+0000","last_became_peered":"2023-10-18T12:02:00.091295+0000","last_unstale":"2023-10-18T12:02:00.111113+0000","last_undegraded":"2023-10-18T12:02:00.111113+0000","last_fullsized":"2023-10-18T12:02:00.111113+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:21.598944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111306+0000","last_change":"2023-10-18T12:02:00.091973+0000","last_active":"2023-10-18T12:02:00.111306+0000","last_peered":"2023-10-18T12:02:00.111306+0000","last_clean":"2023-10-18T12:02:00.111306+0000","last_became_active":"2023-10-18T12:02:00.090926+0000","last_became_peered":"2023-10-18T12:02:00.090926+0000","last_unstale":"2023-10-18T12:02:00.111306+0000","last_undegraded":"2023-10-18T12:02:00.111306+0000","last_fullsized":"2023-10-18T12:02:00.111306+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:34:20.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096051+0000","last_change":"2023-10-18T12:02:00.093710+0000","last_active":"2023-10-18T12:02:00.096051+0000","last_peered":"2023-10-18T12:02:00.096051+0000","last_clean":"2023-10-18T12:02:00.096051+0000","last_became_active":"2023-10-18T12:02:00.093469+0000","last_became_peered":"2023-10-18T12:02:00.093469+0000","last_unstale":"2023-10-18T12:02:00.096051+0000","last_undegraded":"2023-10-18T12:02:00.096051+0000","last_fullsized":"2023-10-18T12:02:00.096051+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:22:06.585710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096077+0000","last_change":"2023-10-18T12:02:00.091806+0000","last_active":"2023-10-18T12:02:00.096077+0000","last_peered":"2023-10-18T12:02:00.096077+0000","last_clean":"2023-10-18T12:02:00.096077+0000","last_became_active":"2023-10-18T12:02:00.090635+0000","last_became_peered":"2023-10-18T12:02:00.090635+0000","last_unstale":"2023-10-18T12:02:00.096077+0000","last_undegraded":"2023-10-18T12:02:00.096077+0000","last_fullsized":"2023-10-18T12:02:00.096077+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:30:20.311774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084531+0000","last_change":"2023-10-18T12:02:00.083558+0000","last_active":"2023-10-18T12:02:00.084531+0000","last_peered":"2023-10-18T12:02:00.084531+0000","last_clean":"2023-10-18T12:02:00.084531+0000","last_became_active":"2023-10-18T12:02:00.083274+0000","last_became_peered":"2023-10-18T12:02:00.083274+0000","last_unstale":"2023-10-18T12:02:00.084531+0000","last_undegraded":"2023-10-18T12:02:00.084531+0000","last_fullsized":"2023-10-18T12:02:00.084531+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:43:44.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097118+0000","last_change":"2023-10-18T12:02:00.096585+0000","last_active":"2023-10-18T12:02:00.097118+0000","last_peered":"2023-10-18T12:02:00.097118+0000","last_clean":"2023-10-18T12:02:00.097118+0000","last_became_active":"2023-10-18T12:02:00.096334+0000","last_became_peered":"2023-10-18T12:02:00.096334+0000","last_unstale":"2023-10-18T12:02:00.097118+0000","last_undegraded":"2023-10-18T12:02:00.097118+0000","last_fullsized":"2023-10-18T12:02:00.097118+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:44:08.823723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.093043+0000","last_change":"2023-10-18T12:02:00.090486+0000","last_active":"2023-10-18T12:02:00.093043+0000","last_peered":"2023-10-18T12:02:00.093043+0000","last_clean":"2023-10-18T12:02:00.093043+0000","last_became_active":"2023-10-18T12:02:00.090236+0000","last_became_peered":"2023-10-18T12:02:00.090236+0000","last_unstale":"2023-10-18T12:02:00.093043+0000","last_undegraded":"2023-10-18T12:02:00.093043+0000","last_fullsized":"2023-10-18T12:02:00.093043+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:28.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.090045+0000","last_change":"2023-10-18T12:02:00.088034+0000","last_active":"2023-10-18T12:02:00.090045+0000","last_peered":"2023-10-18T12:02:00.090045+0000","last_clean":"2023-10-18T12:02:00.090045+0000","last_became_active":"2023-10-18T12:02:00.087782+0000","last_became_peered":"2023-10-18T12:02:00.087782+0000","last_unstale":"2023-10-18T12:02:00.090045+0000","last_undegraded":"2023-10-18T12:02:00.090045+0000","last_fullsized":"2023-10-18T12:02:00.090045+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:23.537671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.098768+0000","last_change":"2023-10-18T12:02:00.098175+0000","last_active":"2023-10-18T12:02:00.098768+0000","last_peered":"2023-10-18T12:02:00.098768+0000","last_clean":"2023-10-18T12:02:00.098768+0000","last_became_active":"2023-10-18T12:02:00.097852+0000","last_became_peered":"2023-10-18T12:02:00.097852+0000","last_unstale":"2023-10-18T12:02:00.098768+0000","last_undegraded":"2023-10-18T12:02:00.098768+0000","last_fullsized":"2023-10-18T12:02:00.098768+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:21.298774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112296+0000","last_change":"2023-10-18T12:02:00.111845+0000","last_active":"2023-10-18T12:02:00.112296+0000","last_peered":"2023-10-18T12:02:00.112296+0000","last_clean":"2023-10-18T12:02:00.112296+0000","last_became_active":"2023-10-18T12:02:00.111570+0000","last_became_peered":"2023-10-18T12:02:00.111570+0000","last_unstale":"2023-10-18T12:02:00.112296+0000","last_undegraded":"2023-10-18T12:02:00.112296+0000","last_fullsized":"2023-10-18T12:02:00.112296+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:18.470466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.091058+0000","last_change":"2023-10-18T12:02:00.090291+0000","last_active":"2023-10-18T12:02:00.091058+0000","last_peered":"2023-10-18T12:02:00.091058+0000","last_clean":"2023-10-18T12:02:00.091058+0000","last_became_active":"2023-10-18T12:02:00.090044+0000","last_became_peered":"2023-10-18T12:02:00.090044+0000","last_unstale":"2023-10-18T12:02:00.091058+0000","last_undegraded":"2023-10-18T12:02:00.091058+0000","last_fullsized":"2023-10-18T12:02:00.091058+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:57:07.420755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111379+0000","last_change":"2023-10-18T12:02:00.092495+0000","last_active":"2023-10-18T12:02:00.111379+0000","last_peered":"2023-10-18T12:02:00.111379+0000","last_clean":"2023-10-18T12:02:00.111379+0000","last_became_active":"2023-10-18T12:02:00.092249+0000","last_became_peered":"2023-10-18T12:02:00.092249+0000","last_unstale":"2023-10-18T12:02:00.111379+0000","last_undegraded":"2023-10-18T12:02:00.111379+0000","last_fullsized":"2023-10-18T12:02:00.111379+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:57.479653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089991+0000","last_change":"2023-10-18T12:02:00.088647+0000","last_active":"2023-10-18T12:02:00.089991+0000","last_peered":"2023-10-18T12:02:00.089991+0000","last_clean":"2023-10-18T12:02:00.089991+0000","last_became_active":"2023-10-18T12:02:00.088410+0000","last_became_peered":"2023-10-18T12:02:00.088410+0000","last_unstale":"2023-10-18T12:02:00.089991+0000","last_undegraded":"2023-10-18T12:02:00.089991+0000","last_fullsized":"2023-10-18T12:02:00.089991+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:46:50.165803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097324+0000","last_change":"2023-10-18T12:02:00.093226+0000","last_active":"2023-10-18T12:02:00.097324+0000","last_peered":"2023-10-18T12:02:00.097324+0000","last_clean":"2023-10-18T12:02:00.097324+0000","last_became_active":"2023-10-18T12:02:00.092569+0000","last_became_peered":"2023-10-18T12:02:00.092569+0000","last_unstale":"2023-10-18T12:02:00.097324+0000","last_undegraded":"2023-10-18T12:02:00.097324+0000","last_fullsized":"2023-10-18T12:02:00.097324+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:48:12.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.083508+0000","last_change":"2023-10-18T12:02:00.082189+0000","last_active":"2023-10-18T12:02:00.083508+0000","last_peered":"2023-10-18T12:02:00.083508+0000","last_clean":"2023-10-18T12:02:00.083508+0000","last_became_active":"2023-10-18T12:02:00.081931+0000","last_became_peered":"2023-10-18T12:02:00.081931+0000","last_unstale":"2023-10-18T12:02:00.083508+0000","last_undegraded":"2023-10-18T12:02:00.083508+0000","last_fullsized":"2023-10-18T12:02:00.083508+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:49:40.953084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087335+0000","last_change":"2023-10-18T12:02:00.084985+0000","last_active":"2023-10-18T12:02:00.087335+0000","last_peered":"2023-10-18T12:02:00.087335+0000","last_clean":"2023-10-18T12:02:00.087335+0000","last_became_active":"2023-10-18T12:02:00.084719+0000","last_became_peered":"2023-10-18T12:02:00.084719+0000","last_unstale":"2023-10-18T12:02:00.087335+0000","last_undegraded":"2023-10-18T12:02:00.087335+0000","last_fullsized":"2023-10-18T12:02:00.087335+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:42.813621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087376+0000","last_change":"2023-10-18T12:02:00.085482+0000","last_active":"2023-10-18T12:02:00.087376+0000","last_peered":"2023-10-18T12:02:00.087376+0000","last_clean":"2023-10-18T12:02:00.087376+0000","last_became_active":"2023-10-18T12:02:00.085247+0000","last_became_peered":"2023-10-18T12:02:00.085247+0000","last_unstale":"2023-10-18T12:02:00.087376+0000","last_undegraded":"2023-10-18T12:02:00.087376+0000","last_fullsized":"2023-10-18T12:02:00.087376+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:17:42.483741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096483+0000","last_change":"2023-10-18T12:02:00.093997+0000","last_active":"2023-10-18T12:02:00.096483+0000","last_peered":"2023-10-18T12:02:00.096483+0000","last_clean":"2023-10-18T12:02:00.096483+0000","last_became_active":"2023-10-18T12:02:00.093749+0000","last_became_peered":"2023-10-18T12:02:00.093749+0000","last_unstale":"2023-10-18T12:02:00.096483+0000","last_undegraded":"2023-10-18T12:02:00.096483+0000","last_fullsized":"2023-10-18T12:02:00.096483+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:28.518720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087307+0000","last_change":"2023-10-18T12:02:00.084763+0000","last_active":"2023-10-18T12:02:00.087307+0000","last_peered":"2023-10-18T12:02:00.087307+0000","last_clean":"2023-10-18T12:02:00.087307+0000","last_became_active":"2023-10-18T12:02:00.084521+0000","last_became_peered":"2023-10-18T12:02:00.084521+0000","last_unstale":"2023-10-18T12:02:00.087307+0000","last_undegraded":"2023-10-18T12:02:00.087307+0000","last_fullsized":"2023-10-18T12:02:00.087307+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:57:15.607727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.091582+0000","last_change":"2023-10-18T12:02:00.086170+0000","last_active":"2023-10-18T12:02:00.091582+0000","last_peered":"2023-10-18T12:02:00.091582+0000","last_clean":"2023-10-18T12:02:00.091582+0000","last_became_active":"2023-10-18T12:02:00.085928+0000","last_became_peered":"2023-10-18T12:02:00.085928+0000","last_unstale":"2023-10-18T12:02:00.091582+0000","last_undegraded":"2023-10-18T12:02:00.091582+0000","last_fullsized":"2023-10-18T12:02:00.091582+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:14:41.141532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.090083+0000","last_change":"2023-10-18T12:02:00.085548+0000","last_active":"2023-10-18T12:02:00.090083+0000","last_peered":"2023-10-18T12:02:00.090083+0000","last_clean":"2023-10-18T12:02:00.090083+0000","last_became_active":"2023-10-18T12:02:00.084282+0000","last_became_peered":"2023-10-18T12:02:00.084282+0000","last_unstale":"2023-10-18T12:02:00.090083+0000","last_undegraded":"2023-10-18T12:02:00.090083+0000","last_fullsized":"2023-10-18T12:02:00.090083+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:48.005895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097197+0000","last_change":"2023-10-18T12:02:00.095767+0000","last_active":"2023-10-18T12:02:00.097197+0000","last_peered":"2023-10-18T12:02:00.097197+0000","last_clean":"2023-10-18T12:02:00.097197+0000","last_became_active":"2023-10-18T12:02:00.095485+0000","last_became_peered":"2023-10-18T12:02:00.095485+0000","last_unstale":"2023-10-18T12:02:00.097197+0000","last_undegraded":"2023-10-18T12:02:00.097197+0000","last_fullsized":"2023-10-18T12:02:00.097197+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:11.928499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.095163+0000","last_change":"2023-10-18T12:02:00.092602+0000","last_active":"2023-10-18T12:02:00.095163+0000","last_peered":"2023-10-18T12:02:00.095163+0000","last_clean":"2023-10-18T12:02:00.095163+0000","last_became_active":"2023-10-18T12:02:00.092317+0000","last_became_peered":"2023-10-18T12:02:00.092317+0000","last_unstale":"2023-10-18T12:02:00.095163+0000","last_undegraded":"2023-10-18T12:02:00.095163+0000","last_fullsized":"2023-10-18T12:02:00.095163+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:32.006797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112350+0000","last_change":"2023-10-18T12:02:00.111846+0000","last_active":"2023-10-18T12:02:00.112350+0000","last_peered":"2023-10-18T12:02:00.112350+0000","last_clean":"2023-10-18T12:02:00.112350+0000","last_became_active":"2023-10-18T12:02:00.111450+0000","last_became_peered":"2023-10-18T12:02:00.111450+0000","last_unstale":"2023-10-18T12:02:00.112350+0000","last_undegraded":"2023-10-18T12:02:00.112350+0000","last_fullsized":"2023-10-18T12:02:00.112350+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:51:21.298774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111473+0000","last_change":"2023-10-18T12:02:00.094688+0000","last_active":"2023-10-18T12:02:00.111473+0000","last_peered":"2023-10-18T12:02:00.111473+0000","last_clean":"2023-10-18T12:02:00.111473+0000","last_became_active":"2023-10-18T12:02:00.094438+0000","last_became_peered":"2023-10-18T12:02:00.094438+0000","last_unstale":"2023-10-18T12:02:00.111473+0000","last_undegraded":"2023-10-18T12:02:00.111473+0000","last_fullsized":"2023-10-18T12:02:00.111473+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:13.006744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.091961+0000","last_change":"2023-10-18T12:02:00.089600+0000","last_active":"2023-10-18T12:02:00.091961+0000","last_peered":"2023-10-18T12:02:00.091961+0000","last_clean":"2023-10-18T12:02:00.091961+0000","last_became_active":"2023-10-18T12:02:00.089364+0000","last_became_peered":"2023-10-18T12:02:00.089364+0000","last_unstale":"2023-10-18T12:02:00.091961+0000","last_undegraded":"2023-10-18T12:02:00.091961+0000","last_fullsized":"2023-10-18T12:02:00.091961+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:55:27.030141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.088927+0000","last_change":"2023-10-18T12:02:00.088526+0000","last_active":"2023-10-18T12:02:00.088927+0000","last_peered":"2023-10-18T12:02:00.088927+0000","last_clean":"2023-10-18T12:02:00.088927+0000","last_became_active":"2023-10-18T12:02:00.088287+0000","last_became_peered":"2023-10-18T12:02:00.088287+0000","last_unstale":"2023-10-18T12:02:00.088927+0000","last_undegraded":"2023-10-18T12:02:00.088927+0000","last_fullsized":"2023-10-18T12:02:00.088927+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:20:13.512224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.095998+0000","last_change":"2023-10-18T12:02:00.093606+0000","last_active":"2023-10-18T12:02:00.095998+0000","last_peered":"2023-10-18T12:02:00.095998+0000","last_clean":"2023-10-18T12:02:00.095998+0000","last_became_active":"2023-10-18T12:02:00.093359+0000","last_became_peered":"2023-10-18T12:02:00.093359+0000","last_unstale":"2023-10-18T12:02:00.095998+0000","last_undegraded":"2023-10-18T12:02:00.095998+0000","last_fullsized":"2023-10-18T12:02:00.095998+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:35.755353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.082374+0000","last_change":"2023-10-18T12:02:00.081885+0000","last_active":"2023-10-18T12:02:00.082374+0000","last_peered":"2023-10-18T12:02:00.082374+0000","last_clean":"2023-10-18T12:02:00.082374+0000","last_became_active":"2023-10-18T12:02:00.081634+0000","last_became_peered":"2023-10-18T12:02:00.081634+0000","last_unstale":"2023-10-18T12:02:00.082374+0000","last_undegraded":"2023-10-18T12:02:00.082374+0000","last_fullsized":"2023-10-18T12:02:00.082374+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:31:45.471226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096897+0000","last_change":"2023-10-18T12:02:00.094737+0000","last_active":"2023-10-18T12:02:00.096897+0000","last_peered":"2023-10-18T12:02:00.096897+0000","last_clean":"2023-10-18T12:02:00.096897+0000","last_became_active":"2023-10-18T12:02:00.094483+0000","last_became_peered":"2023-10-18T12:02:00.094483+0000","last_unstale":"2023-10-18T12:02:00.096897+0000","last_undegraded":"2023-10-18T12:02:00.096897+0000","last_fullsized":"2023-10-18T12:02:00.096897+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:44.083388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085627+0000","last_change":"2023-10-18T12:02:00.084636+0000","last_active":"2023-10-18T12:02:00.085627+0000","last_peered":"2023-10-18T12:02:00.085627+0000","last_clean":"2023-10-18T12:02:00.085627+0000","last_became_active":"2023-10-18T12:02:00.084394+0000","last_became_peered":"2023-10-18T12:02:00.084394+0000","last_unstale":"2023-10-18T12:02:00.085627+0000","last_undegraded":"2023-10-18T12:02:00.085627+0000","last_fullsized":"2023-10-18T12:02:00.085627+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:52:50.842391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112368+0000","last_change":"2023-10-18T12:02:00.111868+0000","last_active":"2023-10-18T12:02:00.112368+0000","last_peered":"2023-10-18T12:02:00.112368+0000","last_clean":"2023-10-18T12:02:00.112368+0000","last_became_active":"2023-10-18T12:02:00.111563+0000","last_became_peered":"2023-10-18T12:02:00.111563+0000","last_unstale":"2023-10-18T12:02:00.112368+0000","last_undegraded":"2023-10-18T12:02:00.112368+0000","last_fullsized":"2023-10-18T12:02:00.112368+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:18.470466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089249+0000","last_change":"2023-10-18T12:01:59.077726+0000","last_active":"2023-10-18T12:02:00.089249+0000","last_peered":"2023-10-18T12:02:00.089249+0000","last_clean":"2023-10-18T12:02:00.089249+0000","last_became_active":"2023-10-18T12:01:59.077387+0000","last_became_peered":"2023-10-18T12:01:59.077387+0000","last_unstale":"2023-10-18T12:02:00.089249+0000","last_undegraded":"2023-10-18T12:02:00.089249+0000","last_fullsized":"2023-10-18T12:02:00.089249+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:09:34.750918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.093062+0000","last_change":"2023-10-18T12:02:00.091016+0000","last_active":"2023-10-18T12:02:00.093062+0000","last_peered":"2023-10-18T12:02:00.093062+0000","last_clean":"2023-10-18T12:02:00.093062+0000","last_became_active":"2023-10-18T12:02:00.090761+0000","last_became_peered":"2023-10-18T12:02:00.090761+0000","last_unstale":"2023-10-18T12:02:00.093062+0000","last_undegraded":"2023-10-18T12:02:00.093062+0000","last_fullsized":"2023-10-18T12:02:00.093062+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:51:58.262632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.091573+0000","last_change":"2023-10-18T12:01:59.078336+0000","last_active":"2023-10-18T12:02:00.091573+0000","last_peered":"2023-10-18T12:02:00.091573+0000","last_clean":"2023-10-18T12:02:00.091573+0000","last_became_active":"2023-10-18T12:01:59.078049+0000","last_became_peered":"2023-10-18T12:01:59.078049+0000","last_unstale":"2023-10-18T12:02:00.091573+0000","last_undegraded":"2023-10-18T12:02:00.091573+0000","last_fullsized":"2023-10-18T12:02:00.091573+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:30.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.092069+0000","last_change":"2023-10-18T12:02:00.088499+0000","last_active":"2023-10-18T12:02:00.092069+0000","last_peered":"2023-10-18T12:02:00.092069+0000","last_clean":"2023-10-18T12:02:00.092069+0000","last_became_active":"2023-10-18T12:02:00.088259+0000","last_became_peered":"2023-10-18T12:02:00.088259+0000","last_unstale":"2023-10-18T12:02:00.092069+0000","last_undegraded":"2023-10-18T12:02:00.092069+0000","last_fullsized":"2023-10-18T12:02:00.092069+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:24:44.358725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.078907+0000","last_change":"2023-10-18T12:01:59.076700+0000","last_active":"2023-10-18T12:02:00.078907+0000","last_peered":"2023-10-18T12:02:00.078907+0000","last_clean":"2023-10-18T12:02:00.078907+0000","last_became_active":"2023-10-18T12:01:59.076469+0000","last_became_peered":"2023-10-18T12:01:59.076469+0000","last_unstale":"2023-10-18T12:02:00.078907+0000","last_undegraded":"2023-10-18T12:02:00.078907+0000","last_fullsized":"2023-10-18T12:02:00.078907+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:42:25.381773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.098794+0000","last_change":"2023-10-18T12:02:00.098160+0000","last_active":"2023-10-18T12:02:00.098794+0000","last_peered":"2023-10-18T12:02:00.098794+0000","last_clean":"2023-10-18T12:02:00.098794+0000","last_became_active":"2023-10-18T12:02:00.097849+0000","last_became_peered":"2023-10-18T12:02:00.097849+0000","last_unstale":"2023-10-18T12:02:00.098794+0000","last_undegraded":"2023-10-18T12:02:00.098794+0000","last_fullsized":"2023-10-18T12:02:00.098794+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:44.083388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.078456+0000","last_change":"2023-10-18T12:01:59.075883+0000","last_active":"2023-10-18T12:02:00.078456+0000","last_peered":"2023-10-18T12:02:00.078456+0000","last_clean":"2023-10-18T12:02:00.078456+0000","last_became_active":"2023-10-18T12:01:59.075650+0000","last_became_peered":"2023-10-18T12:01:59.075650+0000","last_unstale":"2023-10-18T12:02:00.078456+0000","last_undegraded":"2023-10-18T12:02:00.078456+0000","last_fullsized":"2023-10-18T12:02:00.078456+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:34:48.682066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096056+0000","last_change":"2023-10-18T12:02:00.090267+0000","last_active":"2023-10-18T12:02:00.096056+0000","last_peered":"2023-10-18T12:02:00.096056+0000","last_clean":"2023-10-18T12:02:00.096056+0000","last_became_active":"2023-10-18T12:02:00.089992+0000","last_became_peered":"2023-10-18T12:02:00.089992+0000","last_unstale":"2023-10-18T12:02:00.096056+0000","last_undegraded":"2023-10-18T12:02:00.096056+0000","last_fullsized":"2023-10-18T12:02:00.096056+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:02:08.857034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085759+0000","last_change":"2023-10-18T12:01:59.074639+0000","last_active":"2023-10-18T12:02:00.085759+0000","last_peered":"2023-10-18T12:02:00.085759+0000","last_clean":"2023-10-18T12:02:00.085759+0000","last_became_active":"2023-10-18T12:01:59.074384+0000","last_became_peered":"2023-10-18T12:01:59.074384+0000","last_unstale":"2023-10-18T12:02:00.085759+0000","last_undegraded":"2023-10-18T12:02:00.085759+0000","last_fullsized":"2023-10-18T12:02:00.085759+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:14:40.364238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112266+0000","last_change":"2023-10-18T12:02:00.111703+0000","last_active":"2023-10-18T12:02:00.112266+0000","last_peered":"2023-10-18T12:02:00.112266+0000","last_clean":"2023-10-18T12:02:00.112266+0000","last_became_active":"2023-10-18T12:02:00.111355+0000","last_became_peered":"2023-10-18T12:02:00.111355+0000","last_unstale":"2023-10-18T12:02:00.112266+0000","last_undegraded":"2023-10-18T12:02:00.112266+0000","last_fullsized":"2023-10-18T12:02:00.112266+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:40:03.465179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079098+0000","last_change":"2023-10-18T12:01:59.069553+0000","last_active":"2023-10-18T12:02:00.079098+0000","last_peered":"2023-10-18T12:02:00.079098+0000","last_clean":"2023-10-18T12:02:00.079098+0000","last_became_active":"2023-10-18T12:01:59.069166+0000","last_became_peered":"2023-10-18T12:01:59.069166+0000","last_unstale":"2023-10-18T12:02:00.079098+0000","last_undegraded":"2023-10-18T12:02:00.079098+0000","last_fullsized":"2023-10-18T12:02:00.079098+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:50:12.794683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.098414+0000","last_change":"2023-10-18T12:02:00.097948+0000","last_active":"2023-10-18T12:02:00.098414+0000","last_peered":"2023-10-18T12:02:00.098414+0000","last_clean":"2023-10-18T12:02:00.098414+0000","last_became_active":"2023-10-18T12:02:00.097604+0000","last_became_peered":"2023-10-18T12:02:00.097604+0000","last_unstale":"2023-10-18T12:02:00.098414+0000","last_undegraded":"2023-10-18T12:02:00.098414+0000","last_fullsized":"2023-10-18T12:02:00.098414+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:46.692227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077824+0000","last_change":"2023-10-18T12:01:59.065006+0000","last_active":"2023-10-18T12:02:00.077824+0000","last_peered":"2023-10-18T12:02:00.077824+0000","last_clean":"2023-10-18T12:02:00.077824+0000","last_became_active":"2023-10-18T12:01:59.064756+0000","last_became_peered":"2023-10-18T12:01:59.064756+0000","last_unstale":"2023-10-18T12:02:00.077824+0000","last_undegraded":"2023-10-18T12:02:00.077824+0000","last_fullsized":"2023-10-18T12:02:00.077824+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:10.158824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087361+0000","last_change":"2023-10-18T12:02:00.084599+0000","last_active":"2023-10-18T12:02:00.087361+0000","last_peered":"2023-10-18T12:02:00.087361+0000","last_clean":"2023-10-18T12:02:00.087361+0000","last_became_active":"2023-10-18T12:02:00.083594+0000","last_became_peered":"2023-10-18T12:02:00.083594+0000","last_unstale":"2023-10-18T12:02:00.087361+0000","last_undegraded":"2023-10-18T12:02:00.087361+0000","last_fullsized":"2023-10-18T12:02:00.087361+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:48.671597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.081208+0000","last_change":"2023-10-18T12:01:59.071925+0000","last_active":"2023-10-18T12:02:00.081208+0000","last_peered":"2023-10-18T12:02:00.081208+0000","last_clean":"2023-10-18T12:02:00.081208+0000","last_became_active":"2023-10-18T12:01:59.071637+0000","last_became_peered":"2023-10-18T12:01:59.071637+0000","last_unstale":"2023-10-18T12:02:00.081208+0000","last_undegraded":"2023-10-18T12:02:00.081208+0000","last_fullsized":"2023-10-18T12:02:00.081208+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:47.576118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096386+0000","last_change":"2023-10-18T12:02:00.091291+0000","last_active":"2023-10-18T12:02:00.096386+0000","last_peered":"2023-10-18T12:02:00.096386+0000","last_clean":"2023-10-18T12:02:00.096386+0000","last_became_active":"2023-10-18T12:02:00.091009+0000","last_became_peered":"2023-10-18T12:02:00.091009+0000","last_unstale":"2023-10-18T12:02:00.096386+0000","last_undegraded":"2023-10-18T12:02:00.096386+0000","last_fullsized":"2023-10-18T12:02:00.096386+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:18.399487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079818+0000","last_change":"2023-10-18T12:01:59.073955+0000","last_active":"2023-10-18T12:02:00.079818+0000","last_peered":"2023-10-18T12:02:00.079818+0000","last_clean":"2023-10-18T12:02:00.079818+0000","last_became_active":"2023-10-18T12:01:59.073704+0000","last_became_peered":"2023-10-18T12:01:59.073704+0000","last_unstale":"2023-10-18T12:02:00.079818+0000","last_undegraded":"2023-10-18T12:02:00.079818+0000","last_fullsized":"2023-10-18T12:02:00.079818+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:22:06.437675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111027+0000","last_change":"2023-10-18T12:02:00.090490+0000","last_active":"2023-10-18T12:02:00.111027+0000","last_peered":"2023-10-18T12:02:00.111027+0000","last_clean":"2023-10-18T12:02:00.111027+0000","last_became_active":"2023-10-18T12:02:00.090221+0000","last_became_peered":"2023-10-18T12:02:00.090221+0000","last_unstale":"2023-10-18T12:02:00.111027+0000","last_undegraded":"2023-10-18T12:02:00.111027+0000","last_fullsized":"2023-10-18T12:02:00.111027+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:49.093168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.078064+0000","last_change":"2023-10-18T12:01:59.065597+0000","last_active":"2023-10-18T12:02:00.078064+0000","last_peered":"2023-10-18T12:02:00.078064+0000","last_clean":"2023-10-18T12:02:00.078064+0000","last_became_active":"2023-10-18T12:01:59.065362+0000","last_became_peered":"2023-10-18T12:01:59.065362+0000","last_unstale":"2023-10-18T12:02:00.078064+0000","last_undegraded":"2023-10-18T12:02:00.078064+0000","last_fullsized":"2023-10-18T12:02:00.078064+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:02.529048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.083403+0000","last_change":"2023-10-18T12:02:00.082234+0000","last_active":"2023-10-18T12:02:00.083403+0000","last_peered":"2023-10-18T12:02:00.083403+0000","last_clean":"2023-10-18T12:02:00.083403+0000","last_became_active":"2023-10-18T12:02:00.081951+0000","last_became_peered":"2023-10-18T12:02:00.081951+0000","last_unstale":"2023-10-18T12:02:00.083403+0000","last_undegraded":"2023-10-18T12:02:00.083403+0000","last_fullsized":"2023-10-18T12:02:00.083403+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:48.171262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079727+0000","last_change":"2023-10-18T12:01:59.077426+0000","last_active":"2023-10-18T12:02:00.079727+0000","last_peered":"2023-10-18T12:02:00.079727+0000","last_clean":"2023-10-18T12:02:00.079727+0000","last_became_active":"2023-10-18T12:01:59.077066+0000","last_became_peered":"2023-10-18T12:01:59.077066+0000","last_unstale":"2023-10-18T12:02:00.079727+0000","last_undegraded":"2023-10-18T12:02:00.079727+0000","last_fullsized":"2023-10-18T12:02:00.079727+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:31:40.916034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084755+0000","last_change":"2023-10-18T12:02:00.084146+0000","last_active":"2023-10-18T12:02:00.084755+0000","last_peered":"2023-10-18T12:02:00.084755+0000","last_clean":"2023-10-18T12:02:00.084755+0000","last_became_active":"2023-10-18T12:02:00.083888+0000","last_became_peered":"2023-10-18T12:02:00.083888+0000","last_unstale":"2023-10-18T12:02:00.084755+0000","last_undegraded":"2023-10-18T12:02:00.084755+0000","last_fullsized":"2023-10-18T12:02:00.084755+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:58.934212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087173+0000","last_change":"2023-10-18T12:01:59.076212+0000","last_active":"2023-10-18T12:02:00.087173+0000","last_peered":"2023-10-18T12:02:00.087173+0000","last_clean":"2023-10-18T12:02:00.087173+0000","last_became_active":"2023-10-18T12:01:59.075966+0000","last_became_peered":"2023-10-18T12:01:59.075966+0000","last_unstale":"2023-10-18T12:02:00.087173+0000","last_undegraded":"2023-10-18T12:02:00.087173+0000","last_fullsized":"2023-10-18T12:02:00.087173+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:20:48.512427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.111516+0000","last_change":"2023-10-18T12:02:00.092997+0000","last_active":"2023-10-18T12:02:00.111516+0000","last_peered":"2023-10-18T12:02:00.111516+0000","last_clean":"2023-10-18T12:02:00.111516+0000","last_became_active":"2023-10-18T12:02:00.092749+0000","last_became_peered":"2023-10-18T12:02:00.092749+0000","last_unstale":"2023-10-18T12:02:00.111516+0000","last_undegraded":"2023-10-18T12:02:00.111516+0000","last_fullsized":"2023-10-18T12:02:00.111516+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:54:27.223567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077492+0000","last_change":"2023-10-18T12:01:59.074924+0000","last_active":"2023-10-18T12:02:00.077492+0000","last_peered":"2023-10-18T12:02:00.077492+0000","last_clean":"2023-10-18T12:02:00.077492+0000","last_became_active":"2023-10-18T12:01:59.074699+0000","last_became_peered":"2023-10-18T12:01:59.074699+0000","last_unstale":"2023-10-18T12:02:00.077492+0000","last_undegraded":"2023-10-18T12:02:00.077492+0000","last_fullsized":"2023-10-18T12:02:00.077492+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:14.110802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.083910+0000","last_change":"2023-10-18T12:02:00.082860+0000","last_active":"2023-10-18T12:02:00.083910+0000","last_peered":"2023-10-18T12:02:00.083910+0000","last_clean":"2023-10-18T12:02:00.083910+0000","last_became_active":"2023-10-18T12:02:00.082624+0000","last_became_peered":"2023-10-18T12:02:00.082624+0000","last_unstale":"2023-10-18T12:02:00.083910+0000","last_undegraded":"2023-10-18T12:02:00.083910+0000","last_fullsized":"2023-10-18T12:02:00.083910+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:38.446384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085713+0000","last_change":"2023-10-18T12:01:59.076827+0000","last_active":"2023-10-18T12:02:00.085713+0000","last_peered":"2023-10-18T12:02:00.085713+0000","last_clean":"2023-10-18T12:02:00.085713+0000","last_became_active":"2023-10-18T12:01:59.076580+0000","last_became_peered":"2023-10-18T12:01:59.076580+0000","last_unstale":"2023-10-18T12:02:00.085713+0000","last_undegraded":"2023-10-18T12:02:00.085713+0000","last_fullsized":"2023-10-18T12:02:00.085713+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:35.609656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087253+0000","last_change":"2023-10-18T12:02:00.084183+0000","last_active":"2023-10-18T12:02:00.087253+0000","last_peered":"2023-10-18T12:02:00.087253+0000","last_clean":"2023-10-18T12:02:00.087253+0000","last_became_active":"2023-10-18T12:02:00.083767+0000","last_became_peered":"2023-10-18T12:02:00.083767+0000","last_unstale":"2023-10-18T12:02:00.087253+0000","last_undegraded":"2023-10-18T12:02:00.087253+0000","last_fullsized":"2023-10-18T12:02:00.087253+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:32:37.215637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085974+0000","last_change":"2023-10-18T12:01:59.072059+0000","last_active":"2023-10-18T12:02:00.085974+0000","last_peered":"2023-10-18T12:02:00.085974+0000","last_clean":"2023-10-18T12:02:00.085974+0000","last_became_active":"2023-10-18T12:01:59.071801+0000","last_became_peered":"2023-10-18T12:01:59.071801+0000","last_unstale":"2023-10-18T12:02:00.085974+0000","last_undegraded":"2023-10-18T12:02:00.085974+0000","last_fullsized":"2023-10-18T12:02:00.085974+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:53.393773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087391+0000","last_change":"2023-10-18T12:02:00.085585+0000","last_active":"2023-10-18T12:02:00.087391+0000","last_peered":"2023-10-18T12:02:00.087391+0000","last_clean":"2023-10-18T12:02:00.087391+0000","last_became_active":"2023-10-18T12:02:00.083910+0000","last_became_peered":"2023-10-18T12:02:00.083910+0000","last_unstale":"2023-10-18T12:02:00.087391+0000","last_undegraded":"2023-10-18T12:02:00.087391+0000","last_fullsized":"2023-10-18T12:02:00.087391+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:44:17.626184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084808+0000","last_change":"2023-10-18T12:01:59.075302+0000","last_active":"2023-10-18T12:02:00.084808+0000","last_peered":"2023-10-18T12:02:00.084808+0000","last_clean":"2023-10-18T12:02:00.084808+0000","last_became_active":"2023-10-18T12:01:59.075052+0000","last_became_peered":"2023-10-18T12:01:59.075052+0000","last_unstale":"2023-10-18T12:02:00.084808+0000","last_undegraded":"2023-10-18T12:02:00.084808+0000","last_fullsized":"2023-10-18T12:02:00.084808+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:04:18.809260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084461+0000","last_change":"2023-10-18T12:02:00.082223+0000","last_active":"2023-10-18T12:02:00.084461+0000","last_peered":"2023-10-18T12:02:00.084461+0000","last_clean":"2023-10-18T12:02:00.084461+0000","last_became_active":"2023-10-18T12:02:00.081967+0000","last_became_peered":"2023-10-18T12:02:00.081967+0000","last_unstale":"2023-10-18T12:02:00.084461+0000","last_undegraded":"2023-10-18T12:02:00.084461+0000","last_fullsized":"2023-10-18T12:02:00.084461+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:33.442723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.078644+0000","last_change":"2023-10-18T12:01:59.066746+0000","last_active":"2023-10-18T12:02:00.078644+0000","last_peered":"2023-10-18T12:02:00.078644+0000","last_clean":"2023-10-18T12:02:00.078644+0000","last_became_active":"2023-10-18T12:01:59.066487+0000","last_became_peered":"2023-10-18T12:01:59.066487+0000","last_unstale":"2023-10-18T12:02:00.078644+0000","last_undegraded":"2023-10-18T12:02:00.078644+0000","last_fullsized":"2023-10-18T12:02:00.078644+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:09:04.803564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112269+0000","last_change":"2023-10-18T12:02:00.111686+0000","last_active":"2023-10-18T12:02:00.112269+0000","last_peered":"2023-10-18T12:02:00.112269+0000","last_clean":"2023-10-18T12:02:00.112269+0000","last_became_active":"2023-10-18T12:02:00.111422+0000","last_became_peered":"2023-10-18T12:02:00.111422+0000","last_unstale":"2023-10-18T12:02:00.112269+0000","last_undegraded":"2023-10-18T12:02:00.112269+0000","last_fullsized":"2023-10-18T12:02:00.112269+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:19:19.973149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079200+0000","last_change":"2023-10-18T12:01:59.074367+0000","last_active":"2023-10-18T12:02:00.079200+0000","last_peered":"2023-10-18T12:02:00.079200+0000","last_clean":"2023-10-18T12:02:00.079200+0000","last_became_active":"2023-10-18T12:01:59.073893+0000","last_became_peered":"2023-10-18T12:01:59.073893+0000","last_unstale":"2023-10-18T12:02:00.079200+0000","last_undegraded":"2023-10-18T12:02:00.079200+0000","last_fullsized":"2023-10-18T12:02:00.079200+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:22.031460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.084530+0000","last_change":"2023-10-18T12:02:00.082876+0000","last_active":"2023-10-18T12:02:00.084530+0000","last_peered":"2023-10-18T12:02:00.084530+0000","last_clean":"2023-10-18T12:02:00.084530+0000","last_became_active":"2023-10-18T12:02:00.082612+0000","last_became_peered":"2023-10-18T12:02:00.082612+0000","last_unstale":"2023-10-18T12:02:00.084530+0000","last_undegraded":"2023-10-18T12:02:00.084530+0000","last_fullsized":"2023-10-18T12:02:00.084530+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:33.819373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079021+0000","last_change":"2023-10-18T12:01:59.069156+0000","last_active":"2023-10-18T12:02:00.079021+0000","last_peered":"2023-10-18T12:02:00.079021+0000","last_clean":"2023-10-18T12:02:00.079021+0000","last_became_active":"2023-10-18T12:01:59.068896+0000","last_became_peered":"2023-10-18T12:01:59.068896+0000","last_unstale":"2023-10-18T12:02:00.079021+0000","last_undegraded":"2023-10-18T12:02:00.079021+0000","last_fullsized":"2023-10-18T12:02:00.079021+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:52.440185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.083851+0000","last_change":"2023-10-18T12:02:00.082802+0000","last_active":"2023-10-18T12:02:00.083851+0000","last_peered":"2023-10-18T12:02:00.083851+0000","last_clean":"2023-10-18T12:02:00.083851+0000","last_became_active":"2023-10-18T12:02:00.082546+0000","last_became_peered":"2023-10-18T12:02:00.082546+0000","last_unstale":"2023-10-18T12:02:00.083851+0000","last_undegraded":"2023-10-18T12:02:00.083851+0000","last_fullsized":"2023-10-18T12:02:00.083851+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:06.242977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077825+0000","last_change":"2023-10-18T12:01:59.072818+0000","last_active":"2023-10-18T12:02:00.077825+0000","last_peered":"2023-10-18T12:02:00.077825+0000","last_clean":"2023-10-18T12:02:00.077825+0000","last_became_active":"2023-10-18T12:01:59.072567+0000","last_became_peered":"2023-10-18T12:01:59.072567+0000","last_unstale":"2023-10-18T12:02:00.077825+0000","last_undegraded":"2023-10-18T12:02:00.077825+0000","last_fullsized":"2023-10-18T12:02:00.077825+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:02.572353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.112293+0000","last_change":"2023-10-18T12:02:00.111828+0000","last_active":"2023-10-18T12:02:00.112293+0000","last_peered":"2023-10-18T12:02:00.112293+0000","last_clean":"2023-10-18T12:02:00.112293+0000","last_became_active":"2023-10-18T12:02:00.111348+0000","last_became_peered":"2023-10-18T12:02:00.111348+0000","last_unstale":"2023-10-18T12:02:00.112293+0000","last_undegraded":"2023-10-18T12:02:00.112293+0000","last_fullsized":"2023-10-18T12:02:00.112293+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:20:49.516861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077919+0000","last_change":"2023-10-18T12:01:59.070180+0000","last_active":"2023-10-18T12:02:00.077919+0000","last_peered":"2023-10-18T12:02:00.077919+0000","last_clean":"2023-10-18T12:02:00.077919+0000","last_became_active":"2023-10-18T12:01:59.069908+0000","last_became_peered":"2023-10-18T12:01:59.069908+0000","last_unstale":"2023-10-18T12:02:00.077919+0000","last_undegraded":"2023-10-18T12:02:00.077919+0000","last_fullsized":"2023-10-18T12:02:00.077919+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:23:20.216002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097259+0000","last_change":"2023-10-18T12:02:00.095196+0000","last_active":"2023-10-18T12:02:00.097259+0000","last_peered":"2023-10-18T12:02:00.097259+0000","last_clean":"2023-10-18T12:02:00.097259+0000","last_became_active":"2023-10-18T12:02:00.094955+0000","last_became_peered":"2023-10-18T12:02:00.094955+0000","last_unstale":"2023-10-18T12:02:00.097259+0000","last_undegraded":"2023-10-18T12:02:00.097259+0000","last_fullsized":"2023-10-18T12:02:00.097259+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:16.719297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079551+0000","last_change":"2023-10-18T12:01:59.072866+0000","last_active":"2023-10-18T12:02:00.079551+0000","last_peered":"2023-10-18T12:02:00.079551+0000","last_clean":"2023-10-18T12:02:00.079551+0000","last_became_active":"2023-10-18T12:01:59.072623+0000","last_became_peered":"2023-10-18T12:01:59.072623+0000","last_unstale":"2023-10-18T12:02:00.079551+0000","last_undegraded":"2023-10-18T12:02:00.079551+0000","last_fullsized":"2023-10-18T12:02:00.079551+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:32:33.134330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.087231+0000","last_change":"2023-10-18T12:02:00.084012+0000","last_active":"2023-10-18T12:02:00.087231+0000","last_peered":"2023-10-18T12:02:00.087231+0000","last_clean":"2023-10-18T12:02:00.087231+0000","last_became_active":"2023-10-18T12:02:00.083435+0000","last_became_peered":"2023-10-18T12:02:00.083435+0000","last_unstale":"2023-10-18T12:02:00.087231+0000","last_undegraded":"2023-10-18T12:02:00.087231+0000","last_fullsized":"2023-10-18T12:02:00.087231+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:36.863876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089545+0000","last_change":"2023-10-18T12:01:59.074836+0000","last_active":"2023-10-18T12:02:00.089545+0000","last_peered":"2023-10-18T12:02:00.089545+0000","last_clean":"2023-10-18T12:02:00.089545+0000","last_became_active":"2023-10-18T12:01:59.074579+0000","last_became_peered":"2023-10-18T12:01:59.074579+0000","last_unstale":"2023-10-18T12:02:00.089545+0000","last_undegraded":"2023-10-18T12:02:00.089545+0000","last_fullsized":"2023-10-18T12:02:00.089545+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:55.455890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.093952+0000","last_change":"2023-10-18T12:02:00.092675+0000","last_active":"2023-10-18T12:02:00.093952+0000","last_peered":"2023-10-18T12:02:00.093952+0000","last_clean":"2023-10-18T12:02:00.093952+0000","last_became_active":"2023-10-18T12:02:00.092430+0000","last_became_peered":"2023-10-18T12:02:00.092430+0000","last_unstale":"2023-10-18T12:02:00.093952+0000","last_undegraded":"2023-10-18T12:02:00.093952+0000","last_fullsized":"2023-10-18T12:02:00.093952+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:35:54.264848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077604+0000","last_change":"2023-10-18T12:01:59.073960+0000","last_active":"2023-10-18T12:02:00.077604+0000","last_peered":"2023-10-18T12:02:00.077604+0000","last_clean":"2023-10-18T12:02:00.077604+0000","last_became_active":"2023-10-18T12:01:59.073561+0000","last_became_peered":"2023-10-18T12:01:59.073561+0000","last_unstale":"2023-10-18T12:02:00.077604+0000","last_undegraded":"2023-10-18T12:02:00.077604+0000","last_fullsized":"2023-10-18T12:02:00.077604+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:33:41.990902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096404+0000","last_change":"2023-10-18T12:02:00.091497+0000","last_active":"2023-10-18T12:02:00.096404+0000","last_peered":"2023-10-18T12:02:00.096404+0000","last_clean":"2023-10-18T12:02:00.096404+0000","last_became_active":"2023-10-18T12:02:00.090667+0000","last_became_peered":"2023-10-18T12:02:00.090667+0000","last_unstale":"2023-10-18T12:02:00.096404+0000","last_undegraded":"2023-10-18T12:02:00.096404+0000","last_fullsized":"2023-10-18T12:02:00.096404+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:57.383536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077776+0000","last_change":"2023-10-18T12:01:59.073576+0000","last_active":"2023-10-18T12:02:00.077776+0000","last_peered":"2023-10-18T12:02:00.077776+0000","last_clean":"2023-10-18T12:02:00.077776+0000","last_became_active":"2023-10-18T12:01:59.073353+0000","last_became_peered":"2023-10-18T12:01:59.073353+0000","last_unstale":"2023-10-18T12:02:00.077776+0000","last_undegraded":"2023-10-18T12:02:00.077776+0000","last_fullsized":"2023-10-18T12:02:00.077776+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:04:06.264882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089016+0000","last_change":"2023-10-18T12:02:00.081823+0000","last_active":"2023-10-18T12:02:00.089016+0000","last_peered":"2023-10-18T12:02:00.089016+0000","last_clean":"2023-10-18T12:02:00.089016+0000","last_became_active":"2023-10-18T12:02:00.081546+0000","last_became_peered":"2023-10-18T12:02:00.081546+0000","last_unstale":"2023-10-18T12:02:00.089016+0000","last_undegraded":"2023-10-18T12:02:00.089016+0000","last_fullsized":"2023-10-18T12:02:00.089016+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:27:26.740564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.086124+0000","last_change":"2023-10-18T12:01:59.076858+0000","last_active":"2023-10-18T12:02:00.086124+0000","last_peered":"2023-10-18T12:02:00.086124+0000","last_clean":"2023-10-18T12:02:00.086124+0000","last_became_active":"2023-10-18T12:01:59.076595+0000","last_became_peered":"2023-10-18T12:01:59.076595+0000","last_unstale":"2023-10-18T12:02:00.086124+0000","last_undegraded":"2023-10-18T12:02:00.086124+0000","last_fullsized":"2023-10-18T12:02:00.086124+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:23.276520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.090014+0000","last_change":"2023-10-18T12:02:00.088655+0000","last_active":"2023-10-18T12:02:00.090014+0000","last_peered":"2023-10-18T12:02:00.090014+0000","last_clean":"2023-10-18T12:02:00.090014+0000","last_became_active":"2023-10-18T12:02:00.088403+0000","last_became_peered":"2023-10-18T12:02:00.088403+0000","last_unstale":"2023-10-18T12:02:00.090014+0000","last_undegraded":"2023-10-18T12:02:00.090014+0000","last_fullsized":"2023-10-18T12:02:00.090014+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:33.903058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079127+0000","last_change":"2023-10-18T12:01:59.076067+0000","last_active":"2023-10-18T12:02:00.079127+0000","last_peered":"2023-10-18T12:02:00.079127+0000","last_clean":"2023-10-18T12:02:00.079127+0000","last_became_active":"2023-10-18T12:01:59.075817+0000","last_became_peered":"2023-10-18T12:01:59.075817+0000","last_unstale":"2023-10-18T12:02:00.079127+0000","last_undegraded":"2023-10-18T12:02:00.079127+0000","last_fullsized":"2023-10-18T12:02:00.079127+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:10.825855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.097320+0000","last_change":"2023-10-18T12:02:00.093215+0000","last_active":"2023-10-18T12:02:00.097320+0000","last_peered":"2023-10-18T12:02:00.097320+0000","last_clean":"2023-10-18T12:02:00.097320+0000","last_became_active":"2023-10-18T12:02:00.092886+0000","last_became_peered":"2023-10-18T12:02:00.092886+0000","last_unstale":"2023-10-18T12:02:00.097320+0000","last_undegraded":"2023-10-18T12:02:00.097320+0000","last_fullsized":"2023-10-18T12:02:00.097320+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:11.928499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.085999+0000","last_change":"2023-10-18T12:01:59.077679+0000","last_active":"2023-10-18T12:02:00.085999+0000","last_peered":"2023-10-18T12:02:00.085999+0000","last_clean":"2023-10-18T12:02:00.085999+0000","last_became_active":"2023-10-18T12:01:59.077429+0000","last_became_peered":"2023-10-18T12:01:59.077429+0000","last_unstale":"2023-10-18T12:02:00.085999+0000","last_undegraded":"2023-10-18T12:02:00.085999+0000","last_fullsized":"2023-10-18T12:02:00.085999+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:56.661800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.095189+0000","last_change":"2023-10-18T12:02:00.092818+0000","last_active":"2023-10-18T12:02:00.095189+0000","last_peered":"2023-10-18T12:02:00.095189+0000","last_clean":"2023-10-18T12:02:00.095189+0000","last_became_active":"2023-10-18T12:02:00.092501+0000","last_became_peered":"2023-10-18T12:02:00.092501+0000","last_unstale":"2023-10-18T12:02:00.095189+0000","last_undegraded":"2023-10-18T12:02:00.095189+0000","last_fullsized":"2023-10-18T12:02:00.095189+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:41:16.719297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077971+0000","last_change":"2023-10-18T12:01:59.074836+0000","last_active":"2023-10-18T12:02:00.077971+0000","last_peered":"2023-10-18T12:02:00.077971+0000","last_clean":"2023-10-18T12:02:00.077971+0000","last_became_active":"2023-10-18T12:01:59.074396+0000","last_became_peered":"2023-10-18T12:01:59.074396+0000","last_unstale":"2023-10-18T12:02:00.077971+0000","last_undegraded":"2023-10-18T12:02:00.077971+0000","last_fullsized":"2023-10-18T12:02:00.077971+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:32.702122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.095402+0000","last_change":"2023-10-18T12:02:00.090143+0000","last_active":"2023-10-18T12:02:00.095402+0000","last_peered":"2023-10-18T12:02:00.095402+0000","last_clean":"2023-10-18T12:02:00.095402+0000","last_became_active":"2023-10-18T12:02:00.089862+0000","last_became_peered":"2023-10-18T12:02:00.089862+0000","last_unstale":"2023-10-18T12:02:00.095402+0000","last_undegraded":"2023-10-18T12:02:00.095402+0000","last_fullsized":"2023-10-18T12:02:00.095402+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:48.241385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079661+0000","last_change":"2023-10-18T12:01:59.072791+0000","last_active":"2023-10-18T12:02:00.079661+0000","last_peered":"2023-10-18T12:02:00.079661+0000","last_clean":"2023-10-18T12:02:00.079661+0000","last_became_active":"2023-10-18T12:01:59.072560+0000","last_became_peered":"2023-10-18T12:01:59.072560+0000","last_unstale":"2023-10-18T12:02:00.079661+0000","last_undegraded":"2023-10-18T12:02:00.079661+0000","last_fullsized":"2023-10-18T12:02:00.079661+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:00.799261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.092219+0000","last_change":"2023-10-18T12:02:00.090879+0000","last_active":"2023-10-18T12:02:00.092219+0000","last_peered":"2023-10-18T12:02:00.092219+0000","last_clean":"2023-10-18T12:02:00.092219+0000","last_became_active":"2023-10-18T12:02:00.090644+0000","last_became_peered":"2023-10-18T12:02:00.090644+0000","last_unstale":"2023-10-18T12:02:00.092219+0000","last_undegraded":"2023-10-18T12:02:00.092219+0000","last_fullsized":"2023-10-18T12:02:00.092219+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:11:43.273421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.079709+0000","last_change":"2023-10-18T12:01:59.072905+0000","last_active":"2023-10-18T12:02:00.079709+0000","last_peered":"2023-10-18T12:02:00.079709+0000","last_clean":"2023-10-18T12:02:00.079709+0000","last_became_active":"2023-10-18T12:01:59.072671+0000","last_became_peered":"2023-10-18T12:01:59.072671+0000","last_unstale":"2023-10-18T12:02:00.079709+0000","last_undegraded":"2023-10-18T12:02:00.079709+0000","last_fullsized":"2023-10-18T12:02:00.079709+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T23:01:08.939600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.096579+0000","last_change":"2023-10-18T12:02:00.093396+0000","last_active":"2023-10-18T12:02:00.096579+0000","last_peered":"2023-10-18T12:02:00.096579+0000","last_clean":"2023-10-18T12:02:00.096579+0000","last_became_active":"2023-10-18T12:02:00.093155+0000","last_became_peered":"2023-10-18T12:02:00.093155+0000","last_unstale":"2023-10-18T12:02:00.096579+0000","last_undegraded":"2023-10-18T12:02:00.096579+0000","last_fullsized":"2023-10-18T12:02:00.096579+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:37.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":11,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.077686+0000","last_change":"2023-10-18T12:01:59.073803+0000","last_active":"2023-10-18T12:02:00.077686+0000","last_peered":"2023-10-18T12:02:00.077686+0000","last_clean":"2023-10-18T12:02:00.077686+0000","last_became_active":"2023-10-18T12:01:59.073240+0000","last_became_peered":"2023-10-18T12:01:59.073240+0000","last_unstale":"2023-10-18T12:02:00.077686+0000","last_undegraded":"2023-10-18T12:02:00.077686+0000","last_fullsized":"2023-10-18T12:02:00.077686+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:58.051191+0000","last_clean_scrub_stamp":"2023-10-18T12:01:58.051191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:04.148346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":9,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.089794+0000","last_change":"2023-10-18T12:02:00.087413+0000","last_active":"2023-10-18T12:02:00.089794+0000","last_peered":"2023-10-18T12:02:00.089794+0000","last_clean":"2023-10-18T12:02:00.089794+0000","last_became_active":"2023-10-18T12:02:00.087086+0000","last_became_peered":"2023-10-18T12:02:00.087086+0000","last_unstale":"2023-10-18T12:02:00.089794+0000","last_undegraded":"2023-10-18T12:02:00.089794+0000","last_fullsized":"2023-10-18T12:02:00.089794+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T12:01:59.055625+0000","last_clean_scrub_stamp":"2023-10-18T12:01:59.055625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T22:56:41.135253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"261'3062","reported_seq":4079,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-10-18T12:02:00.253515+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:02:00.253515+0000","last_peered":"2023-10-18T12:02:00.253515+0000","last_clean":"2023-10-18T12:02:00.253515+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:02:00.253515+0000","last_undegraded":"2023-10-18T12:02:00.253515+0000","last_fullsized":"2023-10-18T12:02:00.253515+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":3062,"log_dups_size":0,"ondisk_log_size":3062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2500,"num_read_kb":5242,"num_write":649,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2500,"num_read_kb":5242,"num_write":649,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3062,"ondisk_log_size":3062,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576008,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48860,"kb_used_data":3792,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322980,"statfs":{"total":96636764160,"available":96586731520,"internally_reserved":0,"allocated":3883008,"data_stored":2290723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.80800000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.56200000000000006,"15min":0.60099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":1.363},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.86399999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56100000000000005,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.90000000000000002,"15min":1.298},"last":0.375}]},{"osd":3,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71899999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":1.5249999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.81299999999999994},"last":0.29899999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.503,"15min":0.5},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.27000000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70699999999999996,"5min":0.86599999999999999,"15min":1.478},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.83599999999999997,"15min":0.874},"last":0.34899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42720,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329120,"statfs":{"total":96636764160,"available":96593018880,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.29899999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.30499999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.59499999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74199999999999999,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.501}]},{"osd":2,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.83899999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28000000000000003},"max":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46100000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.38600000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:01:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.373},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.38300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48284,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323556,"statfs":{"total":96636764160,"available":96587321344,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.33800000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":1.101,"15min":1.1539999999999999},"last":0.308},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.71099999999999997,"15min":0.751},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.29299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.34000000000000002},"max":{"1min":0.73399999999999999,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72699999999999998,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81000000000000005,"5min":0.81399999999999995,"15min":1},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53000000000000003,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.30299999999999999},"max":{"1min":0.77500000000000002,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.438}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.749,"5min":0.82999999999999996,"15min":1.01},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.872,"5min":0.88700000000000001,"15min":0.97999999999999998},"last":0.32500000000000001}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.255,"15min":0.255},"max":{"1min":0.78800000000000003,"5min":0.83699999999999997,"15min":1.101},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0309999999999999},"last":0.45100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576008,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48284,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323556,"statfs":{"total":96636764160,"available":96587321344,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.60999999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.81499999999999995,"5min":0.872,"15min":1.123},"last":0.50700000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73299999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.435}]},{"osd":3,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.44900000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.755,"5min":1.018,"15min":1.018},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.88900000000000001,"15min":1.05},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.496}]}]},{"osd":3,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48284,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323556,"statfs":{"total":96636764160,"available":96587321344,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.23999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.378,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.39300000000000002,"15min":0.40699999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.38400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.39900000000000002,"15min":0.40600000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44900000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.313},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.432,"5min":1.899,"15min":1.899},"last":0.41299999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.42899999999999999,"15min":0.44600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.434,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54900000000000004,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.87,"5min":1.034,"15min":1.052},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.434}]},{"osd":5,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":0.752,"5min":1.522,"15min":1.522},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81699999999999995,"5min":0.91100000000000003,"15min":1.026},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.94499999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.67700000000000005},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82299999999999995,"5min":1.01,"15min":1.01},"last":0.59899999999999998}]},{"osd":7,"last update":"Wed Oct 18 12:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65800000000000003,"15min":0.67300000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.129,"15min":1.129},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0649999999999999},"last":0.625}]}]},{"osd":2,"up_from":13,"seq":55834576008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64924,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306916,"statfs":{"total":96636764160,"available":96570281984,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.52300000000000002,"5min":0.67200000000000004,"15min":0.77200000000000002},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.40500000000000003,"15min":0.41599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.35099999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.41599999999999998,"15min":0.41699999999999998},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.61499999999999999,"15min":0.624},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.374,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.63100000000000001,"15min":0.746},"last":0.29399999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.27800000000000002},"max":{"1min":0.53900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45200000000000001,"15min":0.46400000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55300000000000005,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.311}]},{"osd":4,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86299999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66300000000000003,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":1.087},"last":0.36199999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.36699999999999999},"max":{"1min":0.85199999999999998,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63400000000000001,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.95399999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":1.0609999999999999},"last":0.502},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.434}]}]},{"osd":1,"up_from":13,"seq":55834576008,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64920,"kb_used_data":3212,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306920,"statfs":{"total":96636764160,"available":96570286080,"internally_reserved":0,"allocated":3289088,"data_stored":1700355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.191},"max":{"1min":0.41199999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.218},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.45800000000000002,"5min":0.623,"15min":0.82599999999999996},"last":0.29399999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.42699999999999999,"15min":0.44600000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":0.50800000000000001,"5min":0.71899999999999997,"15min":0.81000000000000005},"last":0.307},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.40400000000000003},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.23000000000000001},"max":{"1min":0.40600000000000003,"5min":0.68999999999999995,"15min":0.877},"last":0.24099999999999999}]},{"osd":3,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.47899999999999998,"15min":0.503},"min":{"1min":0.309,"5min":0.28899999999999998,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.47399999999999998,"15min":0.502},"min":{"1min":0.26400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47499999999999998,"5min":0.77400000000000002,"15min":0.91000000000000003},"last":0.27400000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.51400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.311},"max":{"1min":0.84099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47499999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.81000000000000005,"5min":0.91500000000000004,"15min":1.05},"last":0.439}]},{"osd":6,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.376},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82699999999999996,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.498},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.42999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57200,"kb_used_data":3804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314640,"statfs":{"total":96636764160,"available":96578191360,"internally_reserved":0,"allocated":3895296,"data_stored":2297874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.245},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58099999999999996,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.27800000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.41099999999999998,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52200000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.52300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.313}]},{"osd":3,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25700000000000001},"max":{"1min":0.51100000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.55200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.379}]},{"osd":4,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.96999999999999997,"15min":0.997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.39000000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.65000000000000002},"min":{"1min":0.47099999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72399999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.95499999999999996,"15min":0.96399999999999997},"last":0.35399999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64700000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76600000000000001,"5min":0.89600000000000002,"15min":1.03},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70999999999999996,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.36699999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.42999999999999999},"max":{"1min":0.78000000000000003,"5min":0.997,"15min":0.997},"last":0.43099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:02:03.711 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T12:02:03.711 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T12:02:03.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.852+0000 7f95e9407640 1 -- 172.21.15.125:0/1502870840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 msgr2=0x7f95e40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.852+0000 7f95e9407640 1 --2- 172.21.15.125:0/1502870840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 0x7f95e40d1910 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009830 tx=0x7f95d802bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.852+0000 7f95e9407640 1 -- 172.21.15.125:0/1502870840 shutdown_connections 2023-10-18T12:02:03.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.852+0000 7f95e9407640 1 --2- 172.21.15.125:0/1502870840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95e40ceb40 0x7f95e40b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.853+0000 7f95e9407640 1 --2- 172.21.15.125:0/1502870840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 0x7f95e40d1910 unknown :-1 s=CLOSED pgs=3028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.856 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.853+0000 7f95e9407640 1 -- 172.21.15.125:0/1502870840 >> 172.21.15.125:0/1502870840 conn(0x7f95e40d0d60 msgr2=0x7f95e40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:03.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.853+0000 7f95e9407640 1 -- 172.21.15.125:0/1502870840 shutdown_connections 2023-10-18T12:02:03.857 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.853+0000 7f95e9407640 1 -- 172.21.15.125:0/1502870840 wait complete. 2023-10-18T12:02:03.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.854+0000 7f95e9407640 1 Processor -- start 2023-10-18T12:02:03.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.854+0000 7f95e9407640 1 -- start start 2023-10-18T12:02:03.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e9407640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95e40ceb40 0x7f95e409bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e9407640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95e40cf340 0x7f95e41f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e9407640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 0x7f95e415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e9407640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e414e1f0 con 0x7f95e415b2a0 2023-10-18T12:02:03.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e9407640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e414e4d0 con 0x7f95e40cf340 2023-10-18T12:02:03.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e9407640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e414e7b0 con 0x7f95e40ceb40 2023-10-18T12:02:03.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 0x7f95e415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 0x7f95e415b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56096/0 (socket says 172.21.15.125:56096) 2023-10-18T12:02:03.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 -- 172.21.15.125:0/2070306708 learned_addr learned my addr 172.21.15.125:0/2070306708 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:03.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e2ffd640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95e40ceb40 0x7f95e409bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e27fc640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95e40cf340 0x7f95e41f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 -- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95e40ceb40 msgr2=0x7f95e409bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95e40ceb40 0x7f95e409bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 -- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95e40cf340 msgr2=0x7f95e41f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:03.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95e40cf340 0x7f95e41f0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:03.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 -- 172.21.15.125:0/2070306708 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d8009490 con 0x7f95e415b2a0 2023-10-18T12:02:03.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e27fc640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95e40cf340 0x7f95e41f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:03.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e2ffd640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95e40ceb40 0x7f95e409bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:03.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e37fe640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 0x7f95e415b930 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00cce0 tx=0x7f95dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:03.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95cb7fe640 1 -- 172.21.15.125:0/2070306708 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00f450 con 0x7f95e415b2a0 2023-10-18T12:02:03.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95cb7fe640 1 -- 172.21.15.125:0/2070306708 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95dc00f6c0 con 0x7f95e415b2a0 2023-10-18T12:02:03.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e414ea90 con 0x7f95e415b2a0 2023-10-18T12:02:03.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95cb7fe640 1 -- 172.21.15.125:0/2070306708 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00fad0 con 0x7f95e415b2a0 2023-10-18T12:02:03.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.855+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e4150b00 con 0x7f95e415b2a0 2023-10-18T12:02:03.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.856+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e40690e0 con 0x7f95e415b2a0 2023-10-18T12:02:03.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.857+0000 7f95cb7fe640 1 -- 172.21.15.125:0/2070306708 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f95dc00fc30 con 0x7f95e415b2a0 2023-10-18T12:02:03.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.857+0000 7f95cb7fe640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f95d006b020 0x7f95d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:03.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.857+0000 7f95cb7fe640 1 -- 172.21.15.125:0/2070306708 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f95dc08a7d0 con 0x7f95e415b2a0 2023-10-18T12:02:03.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.857+0000 7f95e2ffd640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f95d006b020 0x7f95d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:03.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.858+0000 7f95e2ffd640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f95d006b020 0x7f95d006d4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95d4000f60 tx=0x7f95d4009040 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:03.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:03.861+0000 7f95cb7fe640 1 -- 172.21.15.125:0/2070306708 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95dc055370 con 0x7f95e415b2a0 2023-10-18T12:02:04.115 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.112+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f95e40b0d30 con 0x7f95e415b2a0 2023-10-18T12:02:04.116 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.114+0000 7f95cb7fe640 1 -- 172.21.15.125:0/2070306708 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v60) v1 ==== 190+0+147 (secure 0 0 0) 0x7f95dc058990 con 0x7f95e415b2a0 2023-10-18T12:02:04.117 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2023-10-18T12:02:04.117 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQBEsi9lFrCxNhAA+xCIAJrCoxB6PlxkGK4mgA== 2023-10-18T12:02:04.117 INFO:teuthology.orchestra.run.smithi125.stdout: caps mds = "allow" 2023-10-18T12:02:04.118 INFO:teuthology.orchestra.run.smithi125.stdout: caps mon = "allow r" 2023-10-18T12:02:04.118 INFO:teuthology.orchestra.run.smithi125.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T12:02:04.118 INFO:teuthology.orchestra.run.smithi125.stderr:updated caps for client.0 2023-10-18T12:02:04.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f95d006b020 msgr2=0x7f95d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f95d006b020 0x7f95d006d4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95d4000f60 tx=0x7f95d4009040 comp rx=0 tx=0).stop 2023-10-18T12:02:04.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 msgr2=0x7f95e415b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 0x7f95e415b930 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00cce0 tx=0x7f95dc00a620 comp rx=0 tx=0).stop 2023-10-18T12:02:04.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 shutdown_connections 2023-10-18T12:02:04.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f95d006b020 0x7f95d006d4e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95e40cf340 0x7f95e41f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f95e415b2a0 0x7f95e415b930 unknown :-1 s=CLOSED pgs=3029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 --2- 172.21.15.125:0/2070306708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95e40ceb40 0x7f95e409bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 >> 172.21.15.125:0/2070306708 conn(0x7f95e40d0d60 msgr2=0x7f95e4069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:04.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 shutdown_connections 2023-10-18T12:02:04.122 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.116+0000 7f95e9407640 1 -- 172.21.15.125:0/2070306708 wait complete. 2023-10-18T12:02:04.130 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:02:04.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.233+0000 7f42c32e5640 1 -- 172.21.15.125:0/715422100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 msgr2=0x7f42bc0cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.233+0000 7f42c32e5640 1 --2- 172.21.15.125:0/715422100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 0x7f42bc0cdb30 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f42bc071e10 tx=0x7f42ac02bcf0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.233+0000 7f42c32e5640 1 -- 172.21.15.125:0/715422100 shutdown_connections 2023-10-18T12:02:04.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.233+0000 7f42c32e5640 1 --2- 172.21.15.125:0/715422100 >> v2:172.21.15.139:3300/0 conn(0x7f42bc069720 0x7f42bc069bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.233+0000 7f42c32e5640 1 --2- 172.21.15.125:0/715422100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 0x7f42bc0cdb30 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.237 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.233+0000 7f42c32e5640 1 -- 172.21.15.125:0/715422100 >> 172.21.15.125:0/715422100 conn(0x7f42bc0af7a0 msgr2=0x7f42bc0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:04.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.233+0000 7f42c32e5640 1 -- 172.21.15.125:0/715422100 shutdown_connections 2023-10-18T12:02:04.238 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.233+0000 7f42c32e5640 1 -- 172.21.15.125:0/715422100 wait complete. 2023-10-18T12:02:04.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.234+0000 7f42c32e5640 1 Processor -- start 2023-10-18T12:02:04.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.234+0000 7f42c32e5640 1 -- start start 2023-10-18T12:02:04.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c32e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42bc069720 0x7f42bc13b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:04.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c32e5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 0x7f42bc1388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:04.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c32e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42bc142850 0x7f42bc1f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:04.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c32e5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc13c250 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c32e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc1351c0 con 0x7f42bc069720 2023-10-18T12:02:04.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c32e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc13b920 con 0x7f42bc142850 2023-10-18T12:02:04.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c0880640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 0x7f42bc1388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:04.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c0880640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 0x7f42bc1388b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56112/0 (socket says 172.21.15.125:56112) 2023-10-18T12:02:04.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c0880640 1 -- 172.21.15.125:0/3956342094 learned_addr learned my addr 172.21.15.125:0/3956342094 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:04.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c0880640 1 -- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42bc142850 msgr2=0x7f42bc1f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c0880640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42bc142850 0x7f42bc1f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c0880640 1 -- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42bc069720 msgr2=0x7f42bc13b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.235+0000 7f42c0880640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42bc069720 0x7f42bc13b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.236+0000 7f42c0880640 1 -- 172.21.15.125:0/3956342094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac009490 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.236+0000 7f42c0880640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 0x7f42bc1388b0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f42ac02c300 tx=0x7f42ac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:04.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.236+0000 7f42b1ffb640 1 -- 172.21.15.125:0/3956342094 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac003ea0 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.236+0000 7f42b1ffb640 1 -- 172.21.15.125:0/3956342094 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f42ac034440 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.236+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc13bc00 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.236+0000 7f42b1ffb640 1 -- 172.21.15.125:0/3956342094 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac034810 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.236+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc138df0 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.237+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc0690e0 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.237+0000 7f42b1ffb640 1 -- 172.21.15.125:0/3956342094 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f42ac034970 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.238+0000 7f42b1ffb640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f42a006aff0 0x7f42a006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:04.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.238+0000 7f42b1ffb640 1 -- 172.21.15.125:0/3956342094 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7f42ac0acb50 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.238+0000 7f42c1081640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f42a006aff0 0x7f42a006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:04.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.238+0000 7f42c1081640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f42a006aff0 0x7f42a006d4b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42b8001560 tx=0x7f42b8007930 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:04.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.242+0000 7f42b1ffb640 1 -- 172.21.15.125:0/3956342094 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42ac077800 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.289 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:04.286+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:04.461 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.458+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42bc0713e0 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.462 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.459+0000 7f42b1ffb640 1 -- 172.21.15.125:0/3956342094 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f42ac04c090 con 0x7f42bc0cd6d0 2023-10-18T12:02:04.462 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:04.463 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":112,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3962342896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3962342896},{"type":"v1","addr":"172.21.15.139:6837","nonce":3962342896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11413,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1953593032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1953593032},{"type":"v1","addr":"172.21.15.139:6836","nonce":1953593032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":13566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1230654530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1230654530},{"type":"v1","addr":"172.21.15.125:6834","nonce":1230654530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:59.406238+0000","modified":"2023-10-18T12:02:01.736375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13569":{"gid":13569,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1922177652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1922177652},{"type":"v1","addr":"172.21.15.125:6835","nonce":1922177652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-18T12:02:04.464 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 112 2023-10-18T12:02:04.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f42a006aff0 msgr2=0x7f42a006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f42a006aff0 0x7f42a006d4b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42b8001560 tx=0x7f42b8007930 comp rx=0 tx=0).stop 2023-10-18T12:02:04.464 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 msgr2=0x7f42bc1388b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 0x7f42bc1388b0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f42ac02c300 tx=0x7f42ac0038a0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 shutdown_connections 2023-10-18T12:02:04.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42bc142850 0x7f42bc1f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.465 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f42bc0cd6d0 0x7f42bc1388b0 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f42a006aff0 0x7f42a006d4b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 --2- 172.21.15.125:0/3956342094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42bc069720 0x7f42bc13b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 >> 172.21.15.125:0/3956342094 conn(0x7f42bc0af7a0 msgr2=0x7f42bc0709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:04.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 shutdown_connections 2023-10-18T12:02:04.466 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.461+0000 7f42c32e5640 1 -- 172.21.15.125:0/3956342094 wait complete. 2023-10-18T12:02:04.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T12:01:59.406238+0000', 'modified': '2023-10-18T12:02:01.736375+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13569': {'gid': 13569, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6835/1922177652', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6833', 'nonce': 1922177652}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 1922177652}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [36], 'metadata_pool': 35, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T12:02:04.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T12:02:04.878 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T12:02:04.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.981+0000 7fe33ee4f640 1 -- 172.21.15.125:0/1458709064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe338159bd0 msgr2=0x7fe33815c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.984 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.981+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/1458709064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe338159bd0 0x7fe33815c770 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cbd30 tx=0x7fe32002bcf0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.981+0000 7fe33ee4f640 1 -- 172.21.15.125:0/1458709064 shutdown_connections 2023-10-18T12:02:04.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.981+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/1458709064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe338159bd0 0x7fe33815c770 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.981+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/1458709064 >> v2:172.21.15.139:3300/0 conn(0x7fe33806b030 0x7fe33806b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.981+0000 7fe33ee4f640 1 -- 172.21.15.125:0/1458709064 >> 172.21.15.125:0/1458709064 conn(0x7fe338069930 msgr2=0x7fe338069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:04.985 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.981+0000 7fe33ee4f640 1 -- 172.21.15.125:0/1458709064 shutdown_connections 2023-10-18T12:02:04.986 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.981+0000 7fe33ee4f640 1 -- 172.21.15.125:0/1458709064 wait complete. 2023-10-18T12:02:04.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.982+0000 7fe33ee4f640 1 Processor -- start 2023-10-18T12:02:04.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33ee4f640 1 -- start start 2023-10-18T12:02:04.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33ee4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe33806b030 0x7fe33813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:04.988 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33ee4f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe338159bd0 0x7fe338154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:04.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33ee4f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe33815ccb0 0x7fe3381581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:04.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33ee4f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe338154580 con 0x7fe33806b030 2023-10-18T12:02:04.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33ee4f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe338154f50 con 0x7fe33815ccb0 2023-10-18T12:02:04.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33ee4f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe338153970 con 0x7fe338159bd0 2023-10-18T12:02:04.989 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33cbeb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe33806b030 0x7fe33813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:04.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33cbeb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe33806b030 0x7fe33813a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56128/0 (socket says 172.21.15.125:56128) 2023-10-18T12:02:04.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.983+0000 7fe33cbeb640 1 -- 172.21.15.125:0/2202715601 learned_addr learned my addr 172.21.15.125:0/2202715601 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:04.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe337fff640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe338159bd0 0x7fe338154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:04.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe33815ccb0 0x7fe3381581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:04.990 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33cbeb640 1 -- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe338159bd0 msgr2=0x7fe338154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe338159bd0 0x7fe338154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33cbeb640 1 -- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe33815ccb0 msgr2=0x7fe3381581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:04.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe33815ccb0 0x7fe3381581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:04.991 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33cbeb640 1 -- 172.21.15.125:0/2202715601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe320009490 con 0x7fe33806b030 2023-10-18T12:02:04.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe33806b030 0x7fe33813a3c0 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7fe33000ebc0 tx=0x7fe33000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:04.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe3357fa640 1 -- 172.21.15.125:0/2202715601 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300099b0 con 0x7fe33806b030 2023-10-18T12:02:04.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe3357fa640 1 -- 172.21.15.125:0/2202715601 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe330010720 con 0x7fe33806b030 2023-10-18T12:02:04.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33ee4f640 1 -- 172.21.15.125:0/2202715601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe338153c50 con 0x7fe33806b030 2023-10-18T12:02:04.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe3357fa640 1 -- 172.21.15.125:0/2202715601 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330010b30 con 0x7fe33806b030 2023-10-18T12:02:04.992 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.984+0000 7fe33ee4f640 1 -- 172.21.15.125:0/2202715601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe338154210 con 0x7fe33806b030 2023-10-18T12:02:04.993 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.986+0000 7fe3357fa640 1 -- 172.21.15.125:0/2202715601 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fe330010c90 con 0x7fe33806b030 2023-10-18T12:02:04.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.986+0000 7fe3357fa640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fe31806aff0 0x7fe31806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.986+0000 7fe3357fa640 1 -- 172.21.15.125:0/2202715601 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fe330014070 con 0x7fe33806b030 2023-10-18T12:02:05.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.986+0000 7fe337fff640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fe31806aff0 0x7fe31806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.987+0000 7fe337fff640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fe31806aff0 0x7fe31806d4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe320036000 tx=0x7fe320004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:05.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.987+0000 7fe3419e7740 1 Processor -- start 2023-10-18T12:02:05.000 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.987+0000 7fe3419e7740 1 -- start start 2023-10-18T12:02:05.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c2aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c5a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c2b020 0x55a079c5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe3419e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a079b3c730 con 0x55a079c57f20 2023-10-18T12:02:05.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe3419e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a079b9d780 con 0x55a079c2b020 2023-10-18T12:02:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe3419e7740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a079b9ae70 con 0x55a079c57b40 2023-10-18T12:02:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c5a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c5a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56134/0 (socket says 172.21.15.125:56134) 2023-10-18T12:02:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 -- 172.21.15.125:0/165518857 learned_addr learned my addr 172.21.15.125:0/165518857 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:05.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c2aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c2b020 0x55a079c5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 -- 172.21.15.125:0/165518857 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 msgr2=0x55a079c2aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:02:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c2aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 -- 172.21.15.125:0/165518857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c2b020 msgr2=0x55a079c5ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:02:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c2b020 0x55a079c5ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 -- 172.21.15.125:0/165518857 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a079b9b3d0 con 0x55a079c57f20 2023-10-18T12:02:05.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe337fff640 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c5a8d0 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7fe3200098f0 tx=0x7fe32003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1817c62921825d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:05.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe32e7fc640 1 -- 172.21.15.125:0/165518857 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320003730 con 0x55a079c57f20 2023-10-18T12:02:05.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe32e7fc640 1 -- 172.21.15.125:0/165518857 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe320003980 con 0x55a079c57f20 2023-10-18T12:02:05.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.988+0000 7fe32e7fc640 1 -- 172.21.15.125:0/165518857 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320003d30 con 0x55a079c57f20 2023-10-18T12:02:05.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 -- 172.21.15.125:0/165518857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 msgr2=0x55a079c5a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c5a8d0 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7fe3200098f0 tx=0x7fe32003f000 comp rx=0 tx=0).stop 2023-10-18T12:02:05.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 -- 172.21.15.125:0/165518857 shutdown_connections 2023-10-18T12:02:05.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c2b020 0x55a079c5ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c5a8d0 unknown :-1 s=CLOSED pgs=3036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 --2- 172.21.15.125:0/165518857 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c2aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 -- 172.21.15.125:0/165518857 >> 172.21.15.125:0/165518857 conn(0x55a079c2cd50 msgr2=0x55a079c758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:05.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 -- 172.21.15.125:0/165518857 shutdown_connections 2023-10-18T12:02:05.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.989+0000 7fe3419e7740 1 -- 172.21.15.125:0/165518857 wait complete. 2023-10-18T12:02:05.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.990+0000 7fe3419e7740 1 -- start start 2023-10-18T12:02:05.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.990+0000 7fe3419e7740 1 Processor -- start 2023-10-18T12:02:05.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c6cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079c53830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3419e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a079b96760 con 0x55a079c53d70 2023-10-18T12:02:05.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3419e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a079b518f0 con 0x55a079c57f20 2023-10-18T12:02:05.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3419e7740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a079b51590 con 0x55a079c57b40 2023-10-18T12:02:05.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3419e7740 10 client.0 _ll_register_callbacks cb 0x55a079b8dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:02:05.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33cbeb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c6cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56150/0 (socket says 172.21.15.125:56150) 2023-10-18T12:02:05.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 -- 172.21.15.125:0/1587434310 learned_addr learned my addr 172.21.15.125:0/1587434310 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:05.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe337fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079c53830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 -- 172.21.15.125:0/1587434310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 msgr2=0x55a079c6cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:02:05.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c6cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 -- 172.21.15.125:0/1587434310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 msgr2=0x55a079c53830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:02:05.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079c53830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 -- 172.21.15.125:0/1587434310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33815d490 con 0x55a079c53d70 2023-10-18T12:02:05.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe337fff640 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079c53830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:02:05.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c6cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:02:05.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c541f0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7fe328012580 tx=0x7fe328012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cddf2366b1b4d1c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:05.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3177fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:02:05.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3177fe640 1 -- 172.21.15.125:0/1587434310 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328006890 con 0x55a079c53d70 2023-10-18T12:02:05.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3177fe640 1 -- 172.21.15.125:0/1587434310 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fe328019030 con 0x55a079c53d70 2023-10-18T12:02:05.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3419e7740 1 -- 172.21.15.125:0/1587434310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a079b8f740 con 0x55a079c53d70 2023-10-18T12:02:05.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.991+0000 7fe3177fe640 1 -- 172.21.15.125:0/1587434310 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3280069f0 con 0x55a079c53d70 2023-10-18T12:02:05.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3419e7740 10 client.13707 fetch_fsmap learned FSMap version 112 2023-10-18T12:02:05.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3419e7740 1 -- 172.21.15.125:0/1587434310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55a079b94de0 con 0x55a079c53d70 2023-10-18T12:02:05.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3177fe640 1 -- 172.21.15.125:0/1587434310 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55a079b8f740 con 0x55a079c53d70 2023-10-18T12:02:05.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3177fe640 1 -- 172.21.15.125:0/1587434310 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe328006c40 con 0x55a079c53d70 2023-10-18T12:02:05.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3419e7740 10 client.13707 fetch_fsmap finished waiting for FSMap version 112 2023-10-18T12:02:05.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3419e7740 10 client.13707 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T12:02:05.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3419e7740 20 client.13707 populate_metadata read hostname 'smithi125' 2023-10-18T12:02:05.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] conn(0x55a079c47510 0x55a079c93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3419e7740 4 client.13707 mds_command: new command op to 13569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T12:02:05.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3419e7740 1 -- 172.21.15.125:0/1587434310 --> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a079b8f460 con 0x55a079c47510 2023-10-18T12:02:05.013 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] conn(0x55a079c47510 0x55a079c93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe3177fe640 10 client.13707 ms_handle_connect on v2:172.21.15.125:6833/1922177652 2023-10-18T12:02:05.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.992+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] conn(0x55a079c47510 0x55a079c93350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c77e41ab4623012 server_cookie=cc626108ec1b09bd in_seq=0 out_seq=0 2023-10-18T12:02:05.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3177fe640 1 -- 172.21.15.125:0/1587434310 <== mds.0 v2:172.21.15.125:6833/1922177652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a079b8f460 con 0x55a079c47510 2023-10-18T12:02:05.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3177fe640 10 client.13707 handle_command_reply: tid=0 2023-10-18T12:02:05.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 client.13707 shutdown 2023-10-18T12:02:05.014 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 -- 172.21.15.125:0/1587434310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 msgr2=0x55a079c541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c541f0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7fe328012580 tx=0x7fe328012a40 comp rx=0 tx=0).stop 2023-10-18T12:02:05.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 -- 172.21.15.125:0/1587434310 shutdown_connections 2023-10-18T12:02:05.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] conn(0x55a079c47510 0x55a079c93350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079c53830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.015 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c541f0 unknown :-1 s=CLOSED pgs=3037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3177fe640 0 client.13707 ms_handle_reset on v2:172.21.15.125:6833/1922177652 2023-10-18T12:02:05.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1587434310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079c6cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 -- 172.21.15.125:0/1587434310 >> 172.21.15.125:0/1587434310 conn(0x55a079c2b020 msgr2=0x55a079c2e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:05.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 -- 172.21.15.125:0/1587434310 shutdown_connections 2023-10-18T12:02:05.016 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 1 -- 172.21.15.125:0/1587434310 wait complete. 2023-10-18T12:02:05.017 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:04.993+0000 7fe3419e7740 20 client.13707 trim_cache size 0 max 16384 2023-10-18T12:02:05.022 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.009+0000 7fe3419e7740 1 Processor -- start 2023-10-18T12:02:05.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.009+0000 7fe3419e7740 1 -- start start 2023-10-18T12:02:05.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c53d70 0x55a079c26440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57b40 0x55a079c793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe3419e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a079b17270 con 0x55a079c57f20 2023-10-18T12:02:05.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe3419e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a079b1c970 con 0x55a079c57b40 2023-10-18T12:02:05.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe3419e7740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a079b95900 con 0x55a079c53d70 2023-10-18T12:02:05.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56152/0 (socket says 172.21.15.125:56152) 2023-10-18T12:02:05.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 -- 172.21.15.125:0/4159504942 learned_addr learned my addr 172.21.15.125:0/4159504942 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:05.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe337fff640 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57b40 0x55a079c793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c53d70 0x55a079c26440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 -- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c53d70 msgr2=0x55a079c26440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:02:05.025 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c53d70 0x55a079c26440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 -- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57b40 msgr2=0x55a079c793b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:02:05.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57b40 0x55a079c793b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 -- 172.21.15.125:0/4159504942 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a079b33d00 con 0x55a079c57f20 2023-10-18T12:02:05.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c53d70 0x55a079c26440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:05.026 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe337fff640 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57b40 0x55a079c793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:05.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c552f0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7fe328001eb0 tx=0x7fe328000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd51df3d1ee9cfaa server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:05.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe32effd640 1 -- 172.21.15.125:0/4159504942 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328007470 con 0x55a079c57f20 2023-10-18T12:02:05.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe32effd640 1 -- 172.21.15.125:0/4159504942 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe32800b040 con 0x55a079c57f20 2023-10-18T12:02:05.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.010+0000 7fe32effd640 1 -- 172.21.15.125:0/4159504942 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328006890 con 0x55a079c57f20 2023-10-18T12:02:05.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 -- 172.21.15.125:0/4159504942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 msgr2=0x55a079c552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c552f0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7fe328001eb0 tx=0x7fe328000ed0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 -- 172.21.15.125:0/4159504942 shutdown_connections 2023-10-18T12:02:05.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57b40 0x55a079c793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c57f20 0x55a079c552f0 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 --2- 172.21.15.125:0/4159504942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c53d70 0x55a079c26440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 -- 172.21.15.125:0/4159504942 >> 172.21.15.125:0/4159504942 conn(0x55a079c2b020 msgr2=0x55a079c8f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:05.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 -- 172.21.15.125:0/4159504942 shutdown_connections 2023-10-18T12:02:05.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.011+0000 7fe3419e7740 1 -- 172.21.15.125:0/4159504942 wait complete. 2023-10-18T12:02:05.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.012+0000 7fe3419e7740 1 -- start start 2023-10-18T12:02:05.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.012+0000 7fe3419e7740 1 Processor -- start 2023-10-18T12:02:05.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c54810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079ce2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079ce59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a079ce65a0 con 0x55a079c53d70 2023-10-18T12:02:05.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a079c6bf00 con 0x55a079c57f20 2023-10-18T12:02:05.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a079c55730 con 0x55a079c57b40 2023-10-18T12:02:05.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 10 client.0 _ll_register_callbacks cb 0x7fe3300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T12:02:05.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c54810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c54810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56156/0 (socket says 172.21.15.125:56156) 2023-10-18T12:02:05.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 -- 172.21.15.125:0/1250921559 learned_addr learned my addr 172.21.15.125:0/1250921559 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:05.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33d3ec640 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079ce59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe337fff640 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079ce2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 -- 172.21.15.125:0/1250921559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 msgr2=0x55a079ce2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:02:05.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079ce2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 -- 172.21.15.125:0/1250921559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 msgr2=0x55a079ce59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T12:02:05.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079ce59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 -- 172.21.15.125:0/1250921559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a079b94de0 con 0x55a079c53d70 2023-10-18T12:02:05.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe33cbeb640 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c54810 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7fe330088aa0 tx=0x7fe330058db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc2e9fc9816e9f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:05.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3177fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2023-10-18T12:02:05.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3177fe640 1 -- 172.21.15.125:0/1250921559 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330088de0 con 0x55a079c53d70 2023-10-18T12:02:05.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3177fe640 1 -- 172.21.15.125:0/1250921559 <== mon.0 v2:172.21.15.125:3300/0 2 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fe330057780 con 0x55a079c53d70 2023-10-18T12:02:05.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 1 -- 172.21.15.125:0/1250921559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe320003980 con 0x55a079c53d70 2023-10-18T12:02:05.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3177fe640 1 -- 172.21.15.125:0/1250921559 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3300610e0 con 0x55a079c53d70 2023-10-18T12:02:05.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 10 client.13713 fetch_fsmap learned FSMap version 112 2023-10-18T12:02:05.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.013+0000 7fe3419e7740 1 -- 172.21.15.125:0/1250921559 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55a079b594f0 con 0x55a079c53d70 2023-10-18T12:02:05.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3177fe640 1 -- 172.21.15.125:0/1250921559 <== mon.0 v2:172.21.15.125:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe320003980 con 0x55a079c53d70 2023-10-18T12:02:05.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3177fe640 1 -- 172.21.15.125:0/1250921559 <== mon.0 v2:172.21.15.125:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe33005f170 con 0x55a079c53d70 2023-10-18T12:02:05.035 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 10 client.13713 fetch_fsmap finished waiting for FSMap version 112 2023-10-18T12:02:05.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 10 client.13713 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T12:02:05.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 20 client.13713 populate_metadata read hostname 'smithi125' 2023-10-18T12:02:05.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] conn(0x55a079c43120 0x55a079c41b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 4 client.13713 mds_command: new command op to 13569 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T12:02:05.036 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 1 -- 172.21.15.125:0/1250921559 --> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a0798986c0 con 0x55a079c43120 2023-10-18T12:02:05.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe337fff640 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] conn(0x55a079c43120 0x55a079c41b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3177fe640 10 client.13713 ms_handle_connect on v2:172.21.15.125:6833/1922177652 2023-10-18T12:02:05.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe337fff640 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] conn(0x55a079c43120 0x55a079c41b80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25807bcc167dd6bc server_cookie=540abe4c4a7d846b in_seq=0 out_seq=0 2023-10-18T12:02:05.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3177fe640 1 -- 172.21.15.125:0/1250921559 <== mds.0 v2:172.21.15.125:6833/1922177652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55a0798986c0 con 0x55a079c43120 2023-10-18T12:02:05.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3177fe640 10 client.13713 handle_command_reply: tid=0 2023-10-18T12:02:05.037 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 1 client.13713 shutdown 2023-10-18T12:02:05.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 1 -- 172.21.15.125:0/1250921559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 msgr2=0x55a079c54810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.014+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c54810 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7fe330088aa0 tx=0x7fe330058db0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 1 -- 172.21.15.125:0/1250921559 shutdown_connections 2023-10-18T12:02:05.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.125:6833/1922177652,v1:172.21.15.125:6835/1922177652] conn(0x55a079c43120 0x55a079c41b80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.038 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a079c57f20 0x55a079ce59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a079c57b40 0x55a079ce2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3177fe640 0 client.13713 ms_handle_reset on v2:172.21.15.125:6833/1922177652 2023-10-18T12:02:05.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 1 --2- 172.21.15.125:0/1250921559 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a079c53d70 0x55a079c54810 unknown :-1 s=CLOSED pgs=3039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 1 -- 172.21.15.125:0/1250921559 >> 172.21.15.125:0/1250921559 conn(0x55a079c2b020 msgr2=0x55a079c26440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:05.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 1 -- 172.21.15.125:0/1250921559 shutdown_connections 2023-10-18T12:02:05.039 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 1 -- 172.21.15.125:0/1250921559 wait complete. 2023-10-18T12:02:05.040 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe3419e7740 20 client.13713 trim_cache size 0 max 16384 2023-10-18T12:02:05.040 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:05.040 INFO:teuthology.orchestra.run.smithi125.stdout: "cluster_fsid": "9708ef69-1ff8-449e-8a32-79d14c57ee5d", 2023-10-18T12:02:05.041 INFO:teuthology.orchestra.run.smithi125.stdout: "whoami": 0, 2023-10-18T12:02:05.041 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 13569, 2023-10-18T12:02:05.041 INFO:teuthology.orchestra.run.smithi125.stdout: "want_state": "up:active", 2023-10-18T12:02:05.041 INFO:teuthology.orchestra.run.smithi125.stdout: "state": "up:active", 2023-10-18T12:02:05.041 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs", 2023-10-18T12:02:05.042 INFO:teuthology.orchestra.run.smithi125.stdout: "rank_uptime": 4.6186923310000001, 2023-10-18T12:02:05.042 INFO:teuthology.orchestra.run.smithi125.stdout: "mdsmap_epoch": 112, 2023-10-18T12:02:05.042 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch": 262, 2023-10-18T12:02:05.042 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_epoch_barrier": 262, 2023-10-18T12:02:05.042 INFO:teuthology.orchestra.run.smithi125.stdout: "uptime": 9.6923121779999999 2023-10-18T12:02:05.043 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:05.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 -- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fe31806aff0 msgr2=0x7fe31806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fe31806aff0 0x7fe31806d4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe320036000 tx=0x7fe320004730 comp rx=0 tx=0).stop 2023-10-18T12:02:05.044 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 -- 172.21.15.125:0/2202715601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe33806b030 msgr2=0x7fe33813a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe33806b030 0x7fe33813a3c0 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7fe33000ebc0 tx=0x7fe33000c6a0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 -- 172.21.15.125:0/2202715601 shutdown_connections 2023-10-18T12:02:05.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fe31806aff0 0x7fe31806d4b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe33815ccb0 0x7fe3381581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe338159bd0 0x7fe338154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 --2- 172.21.15.125:0/2202715601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe33806b030 0x7fe33813a3c0 unknown :-1 s=CLOSED pgs=3035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 -- 172.21.15.125:0/2202715601 >> 172.21.15.125:0/2202715601 conn(0x7fe338069930 msgr2=0x7fe33815bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:05.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 -- 172.21.15.125:0/2202715601 shutdown_connections 2023-10-18T12:02:05.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.015+0000 7fe33ee4f640 1 -- 172.21.15.125:0/2202715601 wait complete. 2023-10-18T12:02:05.046 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T12:02:05.063 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:05.059+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:05.151 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 -- 172.21.15.125:0/896789831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980c96b0 msgr2=0x7fb6980c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/896789831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980c96b0 0x7fb6980c9b60 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7fb684005fd0 tx=0x7fb68402bd90 comp rx=0 tx=0).stop 2023-10-18T12:02:05.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 -- 172.21.15.125:0/896789831 shutdown_connections 2023-10-18T12:02:05.152 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/896789831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980c96b0 0x7fb6980c9b60 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/896789831 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6980ab500 0x7fb6980ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/896789831 >> v2:172.21.15.139:3300/0 conn(0x7fb69806af00 0x7fb69806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 -- 172.21.15.125:0/896789831 >> 172.21.15.125:0/896789831 conn(0x7fb698069930 msgr2=0x7fb698069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:05.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 -- 172.21.15.125:0/896789831 shutdown_connections 2023-10-18T12:02:05.153 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.149+0000 7fb69ee7c640 1 -- 172.21.15.125:0/896789831 wait complete. 2023-10-18T12:02:05.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.150+0000 7fb69ee7c640 1 Processor -- start 2023-10-18T12:02:05.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.150+0000 7fb69ee7c640 1 -- start start 2023-10-18T12:02:05.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb69ee7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb69806af00 0x7fb698133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb69ee7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980ab500 0x7fb6981f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb69ee7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6980c96b0 0x7fb69813d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb69ee7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69812ffb0 con 0x7fb6980ab500 2023-10-18T12:02:05.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb69ee7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698130290 con 0x7fb6980c96b0 2023-10-18T12:02:05.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb69ee7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb698130570 con 0x7fb69806af00 2023-10-18T12:02:05.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb697fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980ab500 0x7fb6981f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb697fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980ab500 0x7fb6981f0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56170/0 (socket says 172.21.15.125:56170) 2023-10-18T12:02:05.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb697fff640 1 -- 172.21.15.125:0/3613729131 learned_addr learned my addr 172.21.15.125:0/3613729131 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:05.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb69cc18640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb69806af00 0x7fb698133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.158 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb69d419640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6980c96b0 0x7fb69813d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb697fff640 1 -- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb69806af00 msgr2=0x7fb698133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb697fff640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb69806af00 0x7fb698133790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb697fff640 1 -- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6980c96b0 msgr2=0x7fb69813d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb697fff640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6980c96b0 0x7fb69813d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.151+0000 7fb697fff640 1 -- 172.21.15.125:0/3613729131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb684009560 con 0x7fb6980ab500 2023-10-18T12:02:05.159 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.152+0000 7fb697fff640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980ab500 0x7fb6981f0b50 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7fb684009850 tx=0x7fb68402d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:05.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.152+0000 7fb6957fa640 1 -- 172.21.15.125:0/3613729131 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68402ddf0 con 0x7fb6980ab500 2023-10-18T12:02:05.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.152+0000 7fb6957fa640 1 -- 172.21.15.125:0/3613729131 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb6840035f0 con 0x7fb6980ab500 2023-10-18T12:02:05.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.152+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb698130850 con 0x7fb6980ab500 2023-10-18T12:02:05.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.152+0000 7fb6957fa640 1 -- 172.21.15.125:0/3613729131 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840039a0 con 0x7fb6980ab500 2023-10-18T12:02:05.160 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.152+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb698133cd0 con 0x7fb6980ab500 2023-10-18T12:02:05.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.153+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6980690e0 con 0x7fb6980ab500 2023-10-18T12:02:05.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.153+0000 7fb6957fa640 1 -- 172.21.15.125:0/3613729131 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb684003b00 con 0x7fb6980ab500 2023-10-18T12:02:05.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.153+0000 7fb6957fa640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb67c06aff0 0x7fb67c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:05.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.154+0000 7fb6957fa640 1 -- 172.21.15.125:0/3613729131 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 6085+0+0 (secure 0 0 0) 0x7fb6840ac410 con 0x7fb6980ab500 2023-10-18T12:02:05.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.154+0000 7fb69cc18640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb67c06aff0 0x7fb67c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:05.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.154+0000 7fb69cc18640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb67c06aff0 0x7fb67c06d4b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009bd0 tx=0x7fb68c005f90 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:05.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.159+0000 7fb6957fa640 1 -- 172.21.15.125:0/3613729131 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb684077170 con 0x7fb6980ab500 2023-10-18T12:02:05.375 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.373+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb69806b600 con 0x7fb6980ab500 2023-10-18T12:02:05.376 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.373+0000 7fb6957fa640 1 -- 172.21.15.125:0/3613729131 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb68407a790 con 0x7fb6980ab500 2023-10-18T12:02:05.376 INFO:teuthology.orchestra.run.smithi125.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T12:02:05.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb67c06aff0 msgr2=0x7fb67c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb67c06aff0 0x7fb67c06d4b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009bd0 tx=0x7fb68c005f90 comp rx=0 tx=0).stop 2023-10-18T12:02:05.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980ab500 msgr2=0x7fb6981f0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:05.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980ab500 0x7fb6981f0b50 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7fb684009850 tx=0x7fb68402d9f0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 shutdown_connections 2023-10-18T12:02:05.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7fb67c06aff0 0x7fb67c06d4b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6980c96b0 0x7fb69813d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.379 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6980ab500 0x7fb6981f0b50 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 --2- 172.21.15.125:0/3613729131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb69806af00 0x7fb698133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:05.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 >> 172.21.15.125:0/3613729131 conn(0x7fb698069930 msgr2=0x7fb6980ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:05.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 shutdown_connections 2023-10-18T12:02:05.380 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:05.375+0000 7fb69ee7c640 1 -- 172.21.15.125:0/3613729131 wait complete. 2023-10-18T12:02:05.389 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T12:02:05.389 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T12:02:05.390 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi125.front.sepia.ceph.com 2023-10-18T12:02:05.390 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T12:02:05.390 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T12:02:05.391 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T12:02:05.391 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T12:02:05.392 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T12:02:05.392 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:02:05.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:05.393 DEBUG:teuthology.orchestra.run.smithi125:> ip addr 2023-10-18T12:02:05.408 INFO:teuthology.orchestra.run.smithi125.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T12:02:05.408 INFO:teuthology.orchestra.run.smithi125.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T12:02:05.409 INFO:teuthology.orchestra.run.smithi125.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T12:02:05.409 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:02:05.409 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 ::1/128 scope host 2023-10-18T12:02:05.409 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:02:05.409 INFO:teuthology.orchestra.run.smithi125.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:02:05.410 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f8 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:02:05.410 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f0 2023-10-18T12:02:05.410 INFO:teuthology.orchestra.run.smithi125.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:02:05.410 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:d8:da:f9 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:02:05.411 INFO:teuthology.orchestra.run.smithi125.stdout: altname enp7s0f1 2023-10-18T12:02:05.411 INFO:teuthology.orchestra.run.smithi125.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T12:02:05.411 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:38 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:02:05.411 INFO:teuthology.orchestra.run.smithi125.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T12:02:05.411 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 0c:c4:7a:8f:d7:39 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:02:05.412 INFO:teuthology.orchestra.run.smithi125.stdout: inet 172.21.15.125/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T12:02:05.412 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft 36899sec preferred_lft 36899sec 2023-10-18T12:02:05.412 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::ec4:7aff:fe8f:d739/64 scope link 2023-10-18T12:02:05.412 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:02:05.412 INFO:teuthology.orchestra.run.smithi125.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T12:02:05.413 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff 2023-10-18T12:02:05.413 INFO:teuthology.orchestra.run.smithi125.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T12:02:05.413 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:02:05.413 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::38a2:d2ff:fe89:d74b/64 scope link 2023-10-18T12:02:05.413 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:02:05.413 INFO:teuthology.orchestra.run.smithi125.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T12:02:05.414 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 16:b7:39:2f:ae:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:02:05.414 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::14b7:39ff:fe2f:ae83/64 scope link 2023-10-18T12:02:05.414 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:02:05.414 INFO:teuthology.orchestra.run.smithi125.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T12:02:05.414 INFO:teuthology.orchestra.run.smithi125.stdout: link/ether 32:cc:24:d4:53:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:02:05.415 INFO:teuthology.orchestra.run.smithi125.stdout: inet6 fe80::30cc:24ff:fed4:5391/64 scope link 2023-10-18T12:02:05.415 INFO:teuthology.orchestra.run.smithi125.stdout: valid_lft forever preferred_lft forever 2023-10-18T12:02:05.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:05.416 DEBUG:teuthology.orchestra.run.smithi125:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T12:02:05.475 INFO:teuthology.orchestra.run.smithi125.stdout:1 2023-10-18T12:02:05.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:05.477 DEBUG:teuthology.orchestra.run.smithi125:> route 2023-10-18T12:02:05.532 INFO:teuthology.orchestra.run.smithi125.stdout:Kernel IP routing table 2023-10-18T12:02:05.532 INFO:teuthology.orchestra.run.smithi125.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T12:02:05.532 INFO:teuthology.orchestra.run.smithi125.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T12:02:05.533 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T12:02:05.533 INFO:teuthology.orchestra.run.smithi125.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T12:02:05.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:05.534 DEBUG:teuthology.orchestra.run.smithi125:> (cd / && exec bash -c ' 2023-10-18T12:02:05.534 DEBUG:teuthology.orchestra.run.smithi125:> set -e 2023-10-18T12:02:05.535 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T12:02:05.535 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T12:02:05.536 DEBUG:teuthology.orchestra.run.smithi125:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T12:02:05.536 DEBUG:teuthology.orchestra.run.smithi125:> ') 2023-10-18T12:02:05.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:05.629 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T12:02:05.644 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T12:02:05.644 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T12:02:05.644 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:02:05.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:05.645 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link set brx.0 up 2023-10-18T12:02:05.711 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-18T12:02:05.712 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:02:05.712 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T12:02:05.766 INFO:teuthology.orchestra.run.smithi125.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T12:02:05.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T12:02:05.767 DEBUG:teuthology.orchestra.run.smithi125:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T12:02:05.820 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T12:02:05.834 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:05.831+0000 7f48da3bb640 -1 client.11164: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9680:b28c8a4d-d6f8-4948-821e-c6970d71264f:v1:172.21.15.139:0/2446560793] 2023-10-18T12:02:05.888 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:02:05.943 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T12:02:05.944 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T12:02:05.944 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T12:02:05.944 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T12:02:05.944 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T12:02:05.945 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T12:02:05.945 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T12:02:05.945 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T12:02:05.945 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T12:02:05.945 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T12:02:05.946 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T12:02:05.946 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T12:02:05.946 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:02:05.946 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T12:02:05.946 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T12:02:05.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T12:02:05.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T12:02:05.947 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T12:02:05.947 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T12:02:05.947 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T12:02:05.948 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T12:02:05.948 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T12:02:05.948 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:02:05.948 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T12:02:05.948 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T12:02:05.949 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T12:02:05.949 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T12:02:05.949 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T12:02:05.949 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T12:02:05.949 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T12:02:05.950 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:02:05.950 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:02:05.950 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:02:05.950 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:02:05.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:05.951 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T12:02:05.998 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T12:02:05.999 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-18T12:02:06.041 DEBUG:teuthology.orchestra.run.smithi125:> sudo modprobe fuse 2023-10-18T12:02:06.067 DEBUG:teuthology.orchestra.run.smithi125:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T12:02:06.123 INFO:teuthology.orchestra.run.smithi125.stdout:/proc 2023-10-18T12:02:06.123 INFO:teuthology.orchestra.run.smithi125.stdout:/sys 2023-10-18T12:02:06.123 INFO:teuthology.orchestra.run.smithi125.stdout:/dev 2023-10-18T12:02:06.123 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/security 2023-10-18T12:02:06.124 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/shm 2023-10-18T12:02:06.124 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/pts 2023-10-18T12:02:06.124 INFO:teuthology.orchestra.run.smithi125.stdout:/run 2023-10-18T12:02:06.124 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/cgroup 2023-10-18T12:02:06.124 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/pstore 2023-10-18T12:02:06.125 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/bpf 2023-10-18T12:02:06.125 INFO:teuthology.orchestra.run.smithi125.stdout:/ 2023-10-18T12:02:06.125 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/selinux 2023-10-18T12:02:06.125 INFO:teuthology.orchestra.run.smithi125.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T12:02:06.125 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/mqueue 2023-10-18T12:02:06.126 INFO:teuthology.orchestra.run.smithi125.stdout:/dev/hugepages 2023-10-18T12:02:06.126 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/debug 2023-10-18T12:02:06.126 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/tracing 2023-10-18T12:02:06.126 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/fs/fuse/connections 2023-10-18T12:02:06.126 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-sysctl.service 2023-10-18T12:02:06.126 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-18T12:02:06.127 INFO:teuthology.orchestra.run.smithi125.stdout:/sys/kernel/config 2023-10-18T12:02:06.127 INFO:teuthology.orchestra.run.smithi125.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-18T12:02:06.127 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T12:02:06.127 INFO:teuthology.orchestra.run.smithi125.stdout:/run/user/1000 2023-10-18T12:02:06.127 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph 2023-10-18T12:02:06.128 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-0 2023-10-18T12:02:06.128 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-1 2023-10-18T12:02:06.128 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-2 2023-10-18T12:02:06.128 INFO:teuthology.orchestra.run.smithi125.stdout:/var/lib/ceph/osd/ceph-3 2023-10-18T12:02:06.128 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns 2023-10-18T12:02:06.129 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:02:06.129 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:02:06.129 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:02:06.129 INFO:teuthology.orchestra.run.smithi125.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:02:06.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:06.130 DEBUG:teuthology.orchestra.run.smithi125:> ls /sys/fs/fuse/connections 2023-10-18T12:02:06.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:2023-10-18T12:02:06.125+0000 7f8c9ea3c480 -1 init, newargv = 0x5595bc789a60 newargc=15 2023-10-18T12:02:06.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[164915]: starting ceph client 2023-10-18T12:02:06.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[164915]: starting fuse 2023-10-18T12:02:06.145 INFO:teuthology.orchestra.run.smithi125.stdout:41 2023-10-18T12:02:06.145 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-10-18T12:02:06.146 DEBUG:teuthology.orchestra.run.smithi125:> sudo stdin-killer -- python3 -c ' 2023-10-18T12:02:06.146 DEBUG:teuthology.orchestra.run.smithi125:> import glob 2023-10-18T12:02:06.147 DEBUG:teuthology.orchestra.run.smithi125:> import re 2023-10-18T12:02:06.147 DEBUG:teuthology.orchestra.run.smithi125:> import os 2023-10-18T12:02:06.148 DEBUG:teuthology.orchestra.run.smithi125:> import subprocess 2023-10-18T12:02:06.148 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:02:06.149 DEBUG:teuthology.orchestra.run.smithi125:> def _find_admin_socket(client_name): 2023-10-18T12:02:06.150 DEBUG:teuthology.orchestra.run.smithi125:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-18T12:02:06.150 DEBUG:teuthology.orchestra.run.smithi125:> files = glob.glob(asok_path) 2023-10-18T12:02:06.151 DEBUG:teuthology.orchestra.run.smithi125:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-18T12:02:06.151 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:02:06.152 DEBUG:teuthology.orchestra.run.smithi125:> # Given a non-glob path, it better be there 2023-10-18T12:02:06.152 DEBUG:teuthology.orchestra.run.smithi125:> if "*" not in asok_path: 2023-10-18T12:02:06.153 DEBUG:teuthology.orchestra.run.smithi125:> assert(len(files) == 1) 2023-10-18T12:02:06.153 DEBUG:teuthology.orchestra.run.smithi125:> return files[0] 2023-10-18T12:02:06.154 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:02:06.154 DEBUG:teuthology.orchestra.run.smithi125:> for f in files: 2023-10-18T12:02:06.155 DEBUG:teuthology.orchestra.run.smithi125:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T12:02:06.155 DEBUG:teuthology.orchestra.run.smithi125:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T12:02:06.156 DEBUG:teuthology.orchestra.run.smithi125:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T12:02:06.156 DEBUG:teuthology.orchestra.run.smithi125:> contents = proc_f.read() 2023-10-18T12:02:06.157 DEBUG:teuthology.orchestra.run.smithi125:> if mountpoint in contents: 2023-10-18T12:02:06.158 DEBUG:teuthology.orchestra.run.smithi125:> return f 2023-10-18T12:02:06.158 DEBUG:teuthology.orchestra.run.smithi125:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T12:02:06.159 DEBUG:teuthology.orchestra.run.smithi125:> 2023-10-18T12:02:06.159 DEBUG:teuthology.orchestra.run.smithi125:> print(_find_admin_socket("client.0")) 2023-10-18T12:02:06.160 DEBUG:teuthology.orchestra.run.smithi125:> ' 2023-10-18T12:02:06.263 INFO:teuthology.orchestra.run.smithi125.stdout:/var/run/ceph/ceph-client.0.164915.asok 2023-10-18T12:02:06.266 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T12:02:06.274 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.164915.asok 2023-10-18T12:02:06.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:06.275 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.164915.asok status 2023-10-18T12:02:06.408 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:06.408 INFO:teuthology.orchestra.run.smithi125.stdout: "metadata": { 2023-10-18T12:02:06.408 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T12:02:06.408 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T12:02:06.409 INFO:teuthology.orchestra.run.smithi125.stdout: "entity_id": "0", 2023-10-18T12:02:06.409 INFO:teuthology.orchestra.run.smithi125.stdout: "hostname": "smithi125", 2023-10-18T12:02:06.409 INFO:teuthology.orchestra.run.smithi125.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-18T12:02:06.409 INFO:teuthology.orchestra.run.smithi125.stdout: "pid": "164915", 2023-10-18T12:02:06.409 INFO:teuthology.orchestra.run.smithi125.stdout: "root": "/" 2023-10-18T12:02:06.410 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:02:06.410 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_count": 0, 2023-10-18T12:02:06.410 INFO:teuthology.orchestra.run.smithi125.stdout: "dentry_pinned_count": 0, 2023-10-18T12:02:06.410 INFO:teuthology.orchestra.run.smithi125.stdout: "id": 13725, 2023-10-18T12:02:06.410 INFO:teuthology.orchestra.run.smithi125.stdout: "inst": { 2023-10-18T12:02:06.411 INFO:teuthology.orchestra.run.smithi125.stdout: "name": { 2023-10-18T12:02:06.411 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "client", 2023-10-18T12:02:06.411 INFO:teuthology.orchestra.run.smithi125.stdout: "num": 13725 2023-10-18T12:02:06.411 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:02:06.411 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T12:02:06.412 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T12:02:06.412 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T12:02:06.412 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 2431493498 2023-10-18T12:02:06.412 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:02:06.412 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:02:06.412 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": { 2023-10-18T12:02:06.413 INFO:teuthology.orchestra.run.smithi125.stdout: "type": "v1", 2023-10-18T12:02:06.413 INFO:teuthology.orchestra.run.smithi125.stdout: "addr": "192.168.0.1:0", 2023-10-18T12:02:06.413 INFO:teuthology.orchestra.run.smithi125.stdout: "nonce": 2431493498 2023-10-18T12:02:06.413 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2023-10-18T12:02:06.413 INFO:teuthology.orchestra.run.smithi125.stdout: "inst_str": "client.13725 192.168.0.1:0/2431493498", 2023-10-18T12:02:06.414 INFO:teuthology.orchestra.run.smithi125.stdout: "addr_str": "192.168.0.1:0/2431493498", 2023-10-18T12:02:06.414 INFO:teuthology.orchestra.run.smithi125.stdout: "inode_count": 1, 2023-10-18T12:02:06.414 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_epoch": 112, 2023-10-18T12:02:06.414 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch": 263, 2023-10-18T12:02:06.414 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_epoch_barrier": 0, 2023-10-18T12:02:06.415 INFO:teuthology.orchestra.run.smithi125.stdout: "blocklisted": false, 2023-10-18T12:02:06.415 INFO:teuthology.orchestra.run.smithi125.stdout: "fs_name": "cephfs" 2023-10-18T12:02:06.415 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:06.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:06.416 DEBUG:teuthology.orchestra.run.smithi125:> stat --file-system '--printf=%T 2023-10-18T12:02:06.417 DEBUG:teuthology.orchestra.run.smithi125:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-18T12:02:06.432 INFO:teuthology.orchestra.run.smithi125.stdout:fuseblk 2023-10-18T12:02:06.433 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-18T12:02:06.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:06.434 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T12:02:06.499 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-18T12:02:06.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.640+0000 7f0728056640 1 -- 172.21.15.125:0/998069560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f070c005140 msgr2=0x7f070c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:06.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.640+0000 7f0728056640 1 --2- 172.21.15.125:0/998069560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f070c005140 0x7f070c00b980 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f071800c640 tx=0x7f070c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.640+0000 7f07265f3640 1 -- 172.21.15.125:0/998069560 reap_dead start 2023-10-18T12:02:06.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.640+0000 7f0728056640 1 -- 172.21.15.125:0/998069560 shutdown_connections 2023-10-18T12:02:06.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.640+0000 7f0728056640 1 -- 172.21.15.125:0/998069560 >> 172.21.15.125:0/998069560 conn(0x7f0720069930 msgr2=0x7f0720069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:06.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.640+0000 7f0728056640 1 -- 172.21.15.125:0/998069560 shutdown_connections 2023-10-18T12:02:06.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.640+0000 7f0728056640 1 -- 172.21.15.125:0/998069560 wait complete. 2023-10-18T12:02:06.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.641+0000 7f0728056640 1 Processor -- start 2023-10-18T12:02:06.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.641+0000 7f0728056640 1 -- start start 2023-10-18T12:02:06.646 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f0728056640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0720157040 0x7f0720150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:06.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f0728056640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0720157420 0x7f07201514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:06.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f0728056640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0720135d50 0x7f07201361d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:06.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f0728056640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720137eb0 con 0x7f0720157420 2023-10-18T12:02:06.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f0728056640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720136710 con 0x7f0720135d50 2023-10-18T12:02:06.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f0728056640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07201372a0 con 0x7f0720157040 2023-10-18T12:02:06.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0720157420 0x7f07201514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:06.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0720157420 0x7f07201514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56178/0 (socket says 172.21.15.125:56178) 2023-10-18T12:02:06.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 -- 172.21.15.125:0/4178239549 learned_addr learned my addr 172.21.15.125:0/4178239549 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:06.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07265f3640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0720135d50 0x7f07201361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:06.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f0725df2640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0720157040 0x7f0720150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:06.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 -- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0720157040 msgr2=0x7f0720150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:06.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0720157040 0x7f0720150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 -- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0720135d50 msgr2=0x7f07201361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:06.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0720135d50 0x7f07201361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 -- 172.21.15.125:0/4178239549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720137580 con 0x7f0720157420 2023-10-18T12:02:06.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07255f1640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0720157420 0x7f07201514a0 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7f0710004650 tx=0x7f071000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:06.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.642+0000 7f07167fc640 1 -- 172.21.15.125:0/4178239549 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07100041d0 con 0x7f0720157420 2023-10-18T12:02:06.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.643+0000 7f07167fc640 1 -- 172.21.15.125:0/4178239549 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0710015070 con 0x7f0720157420 2023-10-18T12:02:06.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.643+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0720137860 con 0x7f0720157420 2023-10-18T12:02:06.650 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.643+0000 7f07167fc640 1 -- 172.21.15.125:0/4178239549 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710010ad0 con 0x7f0720157420 2023-10-18T12:02:06.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.643+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0720140e00 con 0x7f0720157420 2023-10-18T12:02:06.651 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.644+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0720138060 con 0x7f0720157420 2023-10-18T12:02:06.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.644+0000 7f07167fc640 1 -- 172.21.15.125:0/4178239549 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f0710017030 con 0x7f0720157420 2023-10-18T12:02:06.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.644+0000 7f07167fc640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f070406b020 0x7f070406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:06.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.645+0000 7f07167fc640 1 -- 172.21.15.125:0/4178239549 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 6128+0+0 (secure 0 0 0) 0x7f071008a450 con 0x7f0720157420 2023-10-18T12:02:06.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.645+0000 7f0725df2640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f070406b020 0x7f070406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:06.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.645+0000 7f0725df2640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f070406b020 0x7f070406d4e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f071c003700 tx=0x7f071c00b260 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:06.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.649+0000 7f07167fc640 1 -- 172.21.15.125:0/4178239549 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0720138060 con 0x7f0720157420 2023-10-18T12:02:06.838 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.836+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 --> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f07200ca230 con 0x7f070406b020 2023-10-18T12:02:06.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.838+0000 7f07167fc640 1 -- 172.21.15.125:0/4178239549 <== mgr.9734 v2:172.21.15.139:6832/666436799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f071c00b8e0 con 0x7f070406b020 2023-10-18T12:02:06.840 INFO:teuthology.orchestra.run.smithi125.stdout:[ 2023-10-18T12:02:06.841 INFO:teuthology.orchestra.run.smithi125.stdout: { 2023-10-18T12:02:06.841 INFO:teuthology.orchestra.run.smithi125.stdout: "name": "cephfs" 2023-10-18T12:02:06.841 INFO:teuthology.orchestra.run.smithi125.stdout: } 2023-10-18T12:02:06.841 INFO:teuthology.orchestra.run.smithi125.stdout:] 2023-10-18T12:02:06.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.839+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f070406b020 msgr2=0x7f070406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:06.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.839+0000 7f0728056640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f070406b020 0x7f070406d4e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f071c003700 tx=0x7f071c00b260 comp rx=0 tx=0).stop 2023-10-18T12:02:06.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0720157420 msgr2=0x7f07201514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:06.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0720157420 0x7f07201514a0 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7f0710004650 tx=0x7f071000ba40 comp rx=0 tx=0).stop 2023-10-18T12:02:06.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 shutdown_connections 2023-10-18T12:02:06.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f070406b020 0x7f070406d4e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0720135d50 0x7f07201361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0720157420 0x7f07201514a0 unknown :-1 s=CLOSED pgs=3046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 --2- 172.21.15.125:0/4178239549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0720157040 0x7f0720150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 >> 172.21.15.125:0/4178239549 conn(0x7f0720069930 msgr2=0x7f072015aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:06.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 shutdown_connections 2023-10-18T12:02:06.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.840+0000 7f0728056640 1 -- 172.21.15.125:0/4178239549 wait complete. 2023-10-18T12:02:06.852 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-18T12:02:06.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.956+0000 7f72bc300640 1 -- 172.21.15.125:0/1618806173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b415b2a0 msgr2=0x7f72b40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:06.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.956+0000 7f72bc300640 1 --2- 172.21.15.125:0/1618806173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b415b2a0 0x7f72b40d1910 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009830 tx=0x7f72a402bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.959 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.956+0000 7f72bc300640 1 -- 172.21.15.125:0/1618806173 shutdown_connections 2023-10-18T12:02:06.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.956+0000 7f72bc300640 1 --2- 172.21.15.125:0/1618806173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72b40ceb40 0x7f72b40b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.956+0000 7f72bc300640 1 --2- 172.21.15.125:0/1618806173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b415b2a0 0x7f72b40d1910 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.956+0000 7f72bc300640 1 -- 172.21.15.125:0/1618806173 >> 172.21.15.125:0/1618806173 conn(0x7f72b40d0d60 msgr2=0x7f72b40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:06.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.956+0000 7f72bc300640 1 -- 172.21.15.125:0/1618806173 shutdown_connections 2023-10-18T12:02:06.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.956+0000 7f72bc300640 1 -- 172.21.15.125:0/1618806173 wait complete. 2023-10-18T12:02:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.957+0000 7f72bc300640 1 Processor -- start 2023-10-18T12:02:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.957+0000 7f72bc300640 1 -- start start 2023-10-18T12:02:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72bc300640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72b40ceb40 0x7f72b41517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:06.963 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72bc300640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72b40cf340 0x7f72b415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:06.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72bc300640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b415b2a0 0x7f72b40d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:06.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72bc300640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b415dad0 con 0x7f72b40ceb40 2023-10-18T12:02:06.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72bc300640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b415cec0 con 0x7f72b415b2a0 2023-10-18T12:02:06.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72bc300640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b415d1a0 con 0x7f72b40cf340 2023-10-18T12:02:06.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72b40ceb40 0x7f72b41517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:06.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72b40ceb40 0x7f72b41517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:56188/0 (socket says 172.21.15.125:56188) 2023-10-18T12:02:06.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 -- 172.21.15.125:0/258827183 learned_addr learned my addr 172.21.15.125:0/258827183 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:06.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba89d640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b415b2a0 0x7f72b40d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:06.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72b989b640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72b40cf340 0x7f72b415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:06.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 -- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72b40cf340 msgr2=0x7f72b415df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:06.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72b40cf340 0x7f72b415df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 -- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b415b2a0 msgr2=0x7f72b40d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:06.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b415b2a0 0x7f72b40d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:06.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 -- 172.21.15.125:0/258827183 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a4009490 con 0x7f72b40ceb40 2023-10-18T12:02:06.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.958+0000 7f72ba09c640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72b40ceb40 0x7f72b41517c0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c400 tx=0x7f72b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.959+0000 7f72aaffd640 1 -- 172.21.15.125:0/258827183 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00052d0 con 0x7f72b40ceb40 2023-10-18T12:02:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.959+0000 7f72aaffd640 1 -- 172.21.15.125:0/258827183 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f72b0007020 con 0x7f72b40ceb40 2023-10-18T12:02:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.959+0000 7f72aaffd640 1 -- 172.21.15.125:0/258827183 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0007470 con 0x7f72b40ceb40 2023-10-18T12:02:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.959+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b415d480 con 0x7f72b40ceb40 2023-10-18T12:02:06.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.959+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b415e4a0 con 0x7f72b40ceb40 2023-10-18T12:02:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.960+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b40690e0 con 0x7f72b40ceb40 2023-10-18T12:02:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.960+0000 7f72aaffd640 1 -- 172.21.15.125:0/258827183 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 146) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f72b00067d0 con 0x7f72b40ceb40 2023-10-18T12:02:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.960+0000 7f72aaffd640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f729806aff0 0x7f729806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.960+0000 7f72aaffd640 1 -- 172.21.15.125:0/258827183 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 6128+0+0 (secure 0 0 0) 0x7f72b0089000 con 0x7f72b40ceb40 2023-10-18T12:02:06.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.961+0000 7f72b989b640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f729806aff0 0x7f729806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:06.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.961+0000 7f72b989b640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f729806aff0 0x7f729806d4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72a4036000 tx=0x7f72a4004730 comp rx=0 tx=0).ready entity=mgr.9734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:06.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:06.965+0000 7f72aaffd640 1 -- 172.21.15.125:0/258827183 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72b0053df0 con 0x7f72b40ceb40 2023-10-18T12:02:07.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:07.182+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f72b40b0cd0 con 0x7f72b40ceb40 2023-10-18T12:02:08.114 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.111+0000 7f72aaffd640 1 -- 172.21.15.125:0/258827183 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v147) v1 ==== 92+0+0 (secure 0 0 0) 0x7f72b0057410 con 0x7f72b40ceb40 2023-10-18T12:02:08.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f729806aff0 msgr2=0x7f729806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f729806aff0 0x7f729806d4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72a4036000 tx=0x7f72a4004730 comp rx=0 tx=0).stop 2023-10-18T12:02:08.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72b40ceb40 msgr2=0x7f72b41517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.118 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72b40ceb40 0x7f72b41517c0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c400 tx=0x7f72b000c8c0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 shutdown_connections 2023-10-18T12:02:08.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f729806aff0 0x7f729806d4b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b415b2a0 0x7f72b40d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.119 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f72b40cf340 0x7f72b415df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 --2- 172.21.15.125:0/258827183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72b40ceb40 0x7f72b41517c0 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 >> 172.21.15.125:0/258827183 conn(0x7f72b40d0d60 msgr2=0x7f72b406adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:08.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.115+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 shutdown_connections 2023-10-18T12:02:08.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.116+0000 7f72bc300640 1 -- 172.21.15.125:0/258827183 wait complete. 2023-10-18T12:02:08.132 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-18T12:02:08.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.243+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/3517711992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 msgr2=0x7f6bbc0d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.243+0000 7f6bc1ed4640 1 --2- 172.21.15.125:0/3517711992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 0x7f6bbc0d1910 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4005fd0 tx=0x7f6ba402be20 comp rx=0 tx=0).stop 2023-10-18T12:02:08.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.243+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/3517711992 shutdown_connections 2023-10-18T12:02:08.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.243+0000 7f6bc1ed4640 1 --2- 172.21.15.125:0/3517711992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 0x7f6bbc0d1910 unknown :-1 s=CLOSED pgs=3051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.243+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/3517711992 >> 172.21.15.125:0/3517711992 conn(0x7f6bbc0d0d60 msgr2=0x7f6bbc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.243+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/3517711992 shutdown_connections 2023-10-18T12:02:08.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.243+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/3517711992 wait complete. 2023-10-18T12:02:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.244+0000 7f6bc1ed4640 1 Processor -- start 2023-10-18T12:02:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.244+0000 7f6bc1ed4640 1 -- start start 2023-10-18T12:02:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bc1ed4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc0ceb40 0x7f6bbc157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bc1ed4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bbc0cf340 0x7f6bbc0c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bc1ed4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 0x7f6bbc06b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bc1ed4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1e5060 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bc1ed4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1e4420 con 0x7f6bbc0ceb40 2023-10-18T12:02:08.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bc1ed4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1e46c0 con 0x7f6bbc0cf340 2023-10-18T12:02:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 0x7f6bbc06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 0x7f6bbc06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48626/0 (socket says 172.21.15.125:48626) 2023-10-18T12:02:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbbfff640 1 -- 172.21.15.125:0/1177763631 learned_addr learned my addr 172.21.15.125:0/1177763631 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbb7fe640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc0ceb40 0x7f6bbc157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbaffd640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bbc0cf340 0x7f6bbc0c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbbfff640 1 -- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bbc0cf340 msgr2=0x7f6bbc0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbbfff640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bbc0cf340 0x7f6bbc0c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbbfff640 1 -- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc0ceb40 msgr2=0x7f6bbc157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbbfff640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc0ceb40 0x7f6bbc157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.245+0000 7f6bbbfff640 1 -- 172.21.15.125:0/1177763631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba4009560 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.246+0000 7f6bbbfff640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 0x7f6bbc06b110 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c7a0 tx=0x7f6bb400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.246+0000 7f6bb3fff640 1 -- 172.21.15.125:0/1177763631 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4003b70 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.246+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc1e49a0 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.246+0000 7f6bb3fff640 1 -- 172.21.15.125:0/1177763631 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6bb4003cf0 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.246+0000 7f6bb3fff640 1 -- 172.21.15.125:0/1177763631 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400aab0 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.246+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc0ca7e0 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.254 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.247+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc0690e0 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.247+0000 7f6bb3fff640 1 -- 172.21.15.125:0/1177763631 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 147) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f6bb400ac10 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.247+0000 7f6bb3fff640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f6b9c06b0d0 0x7f6b9c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.248+0000 7f6bbb7fe640 1 -- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f6b9c06b0d0 msgr2=0x7f6b9c06d590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/666436799 2023-10-18T12:02:08.255 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.248+0000 7f6bb3fff640 1 -- 172.21.15.125:0/1177763631 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 6128+0+0 (secure 0 0 0) 0x7f6bb4089af0 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.248+0000 7f6bbb7fe640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f6b9c06b0d0 0x7f6b9c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:08.256 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.252+0000 7f6bb3fff640 1 -- 172.21.15.125:0/1177763631 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bb40546d0 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.427 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.424+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f6bbc0d0b40 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.431 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.428+0000 7f6bb3fff640 1 -- 172.21.15.125:0/1177763631 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v84)=0 v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6bb405e020 con 0x7f6bbc15b2a0 2023-10-18T12:02:08.433 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f6b9c06b0d0 msgr2=0x7f6b9c06d590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:08.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f6b9c06b0d0 0x7f6b9c06d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 msgr2=0x7f6bbc06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 0x7f6bbc06b110 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c7a0 tx=0x7f6bb400cc60 comp rx=0 tx=0).stop 2023-10-18T12:02:08.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 shutdown_connections 2023-10-18T12:02:08.434 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bbc15b2a0 0x7f6bbc06b110 unknown :-1 s=CLOSED pgs=3052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bbc0cf340 0x7f6bbc0c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f6b9c06b0d0 0x7f6b9c06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 --2- 172.21.15.125:0/1177763631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc0ceb40 0x7f6bbc157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 >> 172.21.15.125:0/1177763631 conn(0x7f6bbc0d0d60 msgr2=0x7f6bbc0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:08.435 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 shutdown_connections 2023-10-18T12:02:08.436 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.431+0000 7f6bc1ed4640 1 -- 172.21.15.125:0/1177763631 wait complete. 2023-10-18T12:02:08.444 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-18T12:02:08.500 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:08.498+0000 7fd167e13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:02:08.513 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:08.510+0000 7f7a8083a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:02:08.545 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:08.543+0000 7fd167e13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:02:08.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 -- 172.21.15.125:0/3909146808 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15b2a0 msgr2=0x7f57fc0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.552 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 --2- 172.21.15.125:0/3909146808 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15b2a0 0x7f57fc0cf720 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7f57fc08a810 tx=0x7f57e402bd10 comp rx=0 tx=0).stop 2023-10-18T12:02:08.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 -- 172.21.15.125:0/3909146808 shutdown_connections 2023-10-18T12:02:08.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 --2- 172.21.15.125:0/3909146808 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15b2a0 0x7f57fc0cf720 unknown :-1 s=CLOSED pgs=3053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 --2- 172.21.15.125:0/3909146808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc0ceb40 0x7f57fc0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 --2- 172.21.15.125:0/3909146808 >> v2:172.21.15.139:3300/0 conn(0x7f57fc0cf340 0x7f57fc0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.553 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 -- 172.21.15.125:0/3909146808 >> 172.21.15.125:0/3909146808 conn(0x7f57fc0d0d60 msgr2=0x7f57fc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:08.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 -- 172.21.15.125:0/3909146808 shutdown_connections 2023-10-18T12:02:08.554 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.550+0000 7f5801c10640 1 -- 172.21.15.125:0/3909146808 wait complete. 2023-10-18T12:02:08.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.551+0000 7f5801c10640 1 Processor -- start 2023-10-18T12:02:08.556 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.551+0000 7f5801c10640 1 -- start start 2023-10-18T12:02:08.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f5801c10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc0ceb40 0x7f57fc14a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f5801c10640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc0cf340 0x7f57fc0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f5801c10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc15b2a0 0x7f57fc0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.557 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f5801c10640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0c9570 con 0x7f57fc0cf340 2023-10-18T12:02:08.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f5801c10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1597d0 con 0x7f57fc15b2a0 2023-10-18T12:02:08.558 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f5801c10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0c8960 con 0x7f57fc0ceb40 2023-10-18T12:02:08.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc0cf340 0x7f57fc0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc0cf340 0x7f57fc0c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48652/0 (socket says 172.21.15.125:48652) 2023-10-18T12:02:08.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 -- 172.21.15.125:0/2265309968 learned_addr learned my addr 172.21.15.125:0/2265309968 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:08.559 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57fb7fe640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc0ceb40 0x7f57fc14a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57fbfff640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc15b2a0 0x7f57fc0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 -- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc0ceb40 msgr2=0x7f57fc14a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc0ceb40 0x7f57fc14a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 -- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc15b2a0 msgr2=0x7f57fc0c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.560 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc15b2a0 0x7f57fc0c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 -- 172.21.15.125:0/2265309968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4009560 con 0x7f57fc0cf340 2023-10-18T12:02:08.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.552+0000 7f57f2ffd640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc0cf340 0x7f57fc0c7e30 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009850 tx=0x7f57e402d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:08.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.553+0000 7f57f8ff9640 1 -- 172.21.15.125:0/2265309968 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e402dde0 con 0x7f57fc0cf340 2023-10-18T12:02:08.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.553+0000 7f57f8ff9640 1 -- 172.21.15.125:0/2265309968 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57e40036e0 con 0x7f57fc0cf340 2023-10-18T12:02:08.561 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.553+0000 7f57f8ff9640 1 -- 172.21.15.125:0/2265309968 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4003ab0 con 0x7f57fc0cf340 2023-10-18T12:02:08.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.553+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc0c8c40 con 0x7f57fc0cf340 2023-10-18T12:02:08.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.553+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc0c9200 con 0x7f57fc0cf340 2023-10-18T12:02:08.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.554+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc0c93c0 con 0x7f57fc0cf340 2023-10-18T12:02:08.562 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.554+0000 7f57f8ff9640 1 -- 172.21.15.125:0/2265309968 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 147) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f57e4003c10 con 0x7f57fc0cf340 2023-10-18T12:02:08.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.554+0000 7f57f8ff9640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f57e006b0d0 0x7f57e006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.554+0000 7f57f8ff9640 1 -- 172.21.15.125:0/2265309968 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 6128+0+0 (secure 0 0 0) 0x7f57e40ac400 con 0x7f57fc0cf340 2023-10-18T12:02:08.563 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.554+0000 7f57fb7fe640 1 -- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f57e006b0d0 msgr2=0x7f57e006d590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/666436799 2023-10-18T12:02:08.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.554+0000 7f57fb7fe640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f57e006b0d0 0x7f57e006d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:08.564 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:08.558+0000 7f7a8083a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:02:08.564 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.559+0000 7f57f8ff9640 1 -- 172.21.15.125:0/2265309968 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57e40770d0 con 0x7f57fc0cf340 2023-10-18T12:02:08.731 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.729+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f57fc0690e0 con 0x7f57fc0cf340 2023-10-18T12:02:08.735 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.733+0000 7f57f8ff9640 1 -- 172.21.15.125:0/2265309968 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v85)=0 v85) v1 ==== 133+0+0 (secure 0 0 0) 0x7f57e407a6f0 con 0x7f57fc0cf340 2023-10-18T12:02:08.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f57e006b0d0 msgr2=0x7f57e006d590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:08.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f57e006b0d0 0x7f57e006d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc0cf340 msgr2=0x7f57fc0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc0cf340 0x7f57fc0c7e30 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009850 tx=0x7f57e402d9e0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.738 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 shutdown_connections 2023-10-18T12:02:08.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f57e006b0d0 0x7f57e006d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc15b2a0 0x7f57fc0c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc0cf340 0x7f57fc0c7e30 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 --2- 172.21.15.125:0/2265309968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc0ceb40 0x7f57fc14a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.739 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 >> 172.21.15.125:0/2265309968 conn(0x7f57fc0d0d60 msgr2=0x7f57fc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:08.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 shutdown_connections 2023-10-18T12:02:08.740 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.735+0000 7f5801c10640 1 -- 172.21.15.125:0/2265309968 wait complete. 2023-10-18T12:02:08.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:08.749 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-18T12:02:08.768 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-18T12:02:08.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 -- 172.21.15.125:0/3339739540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a2815b180 msgr2=0x7f9a280cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 --2- 172.21.15.125:0/3339739540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a2815b180 0x7f9a280cf350 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009900 tx=0x7f9a1c02be00 comp rx=0 tx=0).stop 2023-10-18T12:02:08.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 -- 172.21.15.125:0/3339739540 shutdown_connections 2023-10-18T12:02:08.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 --2- 172.21.15.125:0/3339739540 >> v2:172.21.15.139:3300/0 conn(0x7f9a280cdbf0 0x7f9a280cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 --2- 172.21.15.125:0/3339739540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280ce420 0x7f9a280cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 --2- 172.21.15.125:0/3339739540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a2815b180 0x7f9a280cf350 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 -- 172.21.15.125:0/3339739540 >> 172.21.15.125:0/3339739540 conn(0x7f9a28099ff0 msgr2=0x7f9a280d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:08.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 -- 172.21.15.125:0/3339739540 shutdown_connections 2023-10-18T12:02:08.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.913+0000 7f9a2f933640 1 -- 172.21.15.125:0/3339739540 wait complete. 2023-10-18T12:02:08.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.914+0000 7f9a2f933640 1 Processor -- start 2023-10-18T12:02:08.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.914+0000 7f9a2f933640 1 -- start start 2023-10-18T12:02:08.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2f933640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a280cdbf0 0x7f9a280c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2f933640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280ce420 0x7f9a280c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2f933640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815b180 0x7f9a280c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2f933640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a281598e0 con 0x7f9a280cdbf0 2023-10-18T12:02:08.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2f933640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280ca810 con 0x7f9a2815b180 2023-10-18T12:02:08.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2f933640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280caaf0 con 0x7f9a280ce420 2023-10-18T12:02:08.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a280cdbf0 0x7f9a280c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a280cdbf0 0x7f9a280c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48664/0 (socket says 172.21.15.125:48664) 2023-10-18T12:02:08.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 -- 172.21.15.125:0/2414522138 learned_addr learned my addr 172.21.15.125:0/2414522138 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:08.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2cece640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280ce420 0x7f9a280c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2ded0640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815b180 0x7f9a280c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:08.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280ce420 msgr2=0x7f9a280c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a280ce420 0x7f9a280c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815b180 msgr2=0x7f9a280c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:08.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a2815b180 0x7f9a280c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:08.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 -- 172.21.15.125:0/2414522138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c009560 con 0x7f9a280cdbf0 2023-10-18T12:02:08.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2d6cf640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a280cdbf0 0x7f9a280c9ec0 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c036040 tx=0x7f9a1c02dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:08.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0035f0 con 0x7f9a280cdbf0 2023-10-18T12:02:08.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9a1c003840 con 0x7f9a280cdbf0 2023-10-18T12:02:08.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.915+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a280cadd0 con 0x7f9a280cdbf0 2023-10-18T12:02:08.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.916+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c003bf0 con 0x7f9a280cdbf0 2023-10-18T12:02:08.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.916+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a281f2960 con 0x7f9a280cdbf0 2023-10-18T12:02:08.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.917+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a28066960 con 0x7f9a280cdbf0 2023-10-18T12:02:08.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.917+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 147) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f9a1c003d50 con 0x7f9a280cdbf0 2023-10-18T12:02:08.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.918+0000 7f9a19ffb640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 0x7f9a0c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:08.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.918+0000 7f9a2cece640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 msgr2=0x7f9a0c06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/666436799 2023-10-18T12:02:08.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.918+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 6128+0+0 (secure 0 0 0) 0x7f9a1c0ac7d0 con 0x7f9a280cdbf0 2023-10-18T12:02:08.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.918+0000 7f9a2cece640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 0x7f9a0c06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:08.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:08.922+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a1c077550 con 0x7f9a280cdbf0 2023-10-18T12:02:08.934 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:08.932+0000 7fd167e13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:02:08.967 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:08.965+0000 7f7a8083a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:02:08.978 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:08.975+0000 7fd167e13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:02:09.013 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.010+0000 7f7a8083a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:02:09.019 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.017+0000 7fd167e13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:02:09.056 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.053+0000 7f7a8083a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:02:09.070 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.067+0000 7fd167e13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:02:09.109 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.106+0000 7f7a8083a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:02:09.110 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.108+0000 7fd167e13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:02:09.120 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:09.118+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 --> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a280cac30 con 0x7f9a0c06b100 2023-10-18T12:02:09.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:09.119+0000 7f9a2cece640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 msgr2=0x7f9a0c06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/666436799 2023-10-18T12:02:09.121 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:09.119+0000 7f9a2cece640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 0x7f9a0c06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:02:09.150 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.148+0000 7f7a8083a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:02:09.209 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.207+0000 7fd167e13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:02:09.247 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.245+0000 7f7a8083a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:02:09.262 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.259+0000 7fd167e13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:02:09.299 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.296+0000 7f7a8083a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:02:09.303 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.300+0000 7fd167e13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:02:09.340 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.338+0000 7f7a8083a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:02:09.350 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.348+0000 7fd167e13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:02:09.387 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.385+0000 7f7a8083a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:02:09.392 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.390+0000 7fd167e13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:02:09.430 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.427+0000 7f7a8083a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:02:09.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:09.520+0000 7f9a2cece640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 msgr2=0x7f9a0c06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/666436799 2023-10-18T12:02:09.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:09.520+0000 7f9a2cece640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 0x7f9a0c06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-18T12:02:09.710 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.707+0000 7fd167e13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:02:09.744 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.742+0000 7f7a8083a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:02:09.839 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.837+0000 7fd167e13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:02:09.874 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.872+0000 7f7a8083a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:02:09.929 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.926+0000 7fd167e13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:02:09.964 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:09.962+0000 7f7a8083a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:02:09.970 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:09.968+0000 7fd167e13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:02:10.005 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:10.003+0000 7f7a8083a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:02:10.064 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:10.061+0000 7fd167e13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:02:10.099 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:10.097+0000 7f7a8083a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:02:10.153 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:10.150+0000 7fd167e13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:02:10.188 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:10.186+0000 7f7a8083a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:02:10.287 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:10.285+0000 7fd167e13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:02:10.322 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:10.320+0000 7f7a8083a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:02:10.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:10.321+0000 7f9a2cece640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 msgr2=0x7f9a0c06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/666436799 2023-10-18T12:02:10.323 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:10.321+0000 7f9a2cece640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 0x7f9a0c06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-18T12:02:10.328 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:10.326+0000 7fd167e13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:02:10.364 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:10.362+0000 7f7a8083a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:02:10.432 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:10.429+0000 7fd167e13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:02:10.468 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:10.466+0000 7f7a8083a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:02:11.046 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.044+0000 7fd167e13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:02:11.090 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:11.087+0000 7f7a8083a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:02:11.306 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.303+0000 7fd167e13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:02:11.351 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:11.348+0000 7f7a8083a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:02:11.354 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.352+0000 7fd167e13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:02:11.399 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:11.396+0000 7f7a8083a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:02:11.454 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.452+0000 7fd167e13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:02:11.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:11.472+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 148) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f9a1c077920 con 0x7f9a280cdbf0 2023-10-18T12:02:11.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:11.472+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 msgr2=0x7f9a0c06d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:11.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:11.472+0000 7f9a19ffb640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/666436799,v1:172.21.15.139:6833/666436799] conn(0x7f9a0c06b100 0x7f9a0c06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:11.503 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:11.501+0000 7f7a8083a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:02:11.569 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e45fe640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.569 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e45fe640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.569 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e45fe640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.570 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e45fe640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.570 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e45fe640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.570 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e7644640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.570 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e7644640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.571 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e7644640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.571 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e7644640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:11.571 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:11.566+0000 7fd0e7644640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:12.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.476+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mgrmap(e 149) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f9a1c03e450 con 0x7f9a280cdbf0 2023-10-18T12:02:12.479 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.477+0000 7f9a19ffb640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f9a0c035860 0x7f9a0c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:12.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.477+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 --> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a280cac30 con 0x7f9a0c035860 2023-10-18T12:02:12.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.479+0000 7f9a2cece640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f9a0c035860 0x7f9a0c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:12.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.484+0000 7f9a2cece640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f9a0c035860 0x7f9a0c075670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9a280b65f0 tx=0x7f9a2400b390 comp rx=0 tx=0).ready entity=mgr.11581 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:12.537 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.535+0000 7f9a19ffb640 1 -- 172.21.15.125:0/2414522138 <== mgr.11581 v2:172.21.15.139:6832/1808248507 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9a280cac30 con 0x7f9a0c035860 2023-10-18T12:02:12.538 INFO:teuthology.orchestra.run.smithi125.stdout:Schedule set for path snap_test_dir1 2023-10-18T12:02:12.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f9a0c035860 msgr2=0x7f9a0c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:12.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2f933640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f9a0c035860 0x7f9a0c075670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9a280b65f0 tx=0x7f9a2400b390 comp rx=0 tx=0).stop 2023-10-18T12:02:12.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a280cdbf0 msgr2=0x7f9a280c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:12.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2f933640 1 --2- 172.21.15.125:0/2414522138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9a280cdbf0 0x7f9a280c9ec0 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c036040 tx=0x7f9a1c02dad0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2ded0640 1 -- 172.21.15.125:0/2414522138 reap_dead start 2023-10-18T12:02:12.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 shutdown_connections 2023-10-18T12:02:12.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 >> 172.21.15.125:0/2414522138 conn(0x7f9a28099ff0 msgr2=0x7f9a280b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:12.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 shutdown_connections 2023-10-18T12:02:12.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.537+0000 7f9a2f933640 1 -- 172.21.15.125:0/2414522138 wait complete. 2023-10-18T12:02:12.552 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-18T12:02:12.552 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-18T12:02:12.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.659+0000 7f8605b4c640 1 -- 172.21.15.125:0/1446945822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85e80047c0 msgr2=0x7f85e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:12.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.659+0000 7f8605b4c640 1 --2- 172.21.15.125:0/1446945822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f85e80047c0 0x7f85e8004c40 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7f85f40017f0 tx=0x7f85e80057f0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.659+0000 7f85fffff640 1 -- 172.21.15.125:0/1446945822 reap_dead start 2023-10-18T12:02:12.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.659+0000 7f8605b4c640 1 -- 172.21.15.125:0/1446945822 shutdown_connections 2023-10-18T12:02:12.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.659+0000 7f8605b4c640 1 -- 172.21.15.125:0/1446945822 >> 172.21.15.125:0/1446945822 conn(0x7f8600069930 msgr2=0x7f8600069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:12.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.659+0000 7f8605b4c640 1 -- 172.21.15.125:0/1446945822 shutdown_connections 2023-10-18T12:02:12.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.659+0000 7f8605b4c640 1 -- 172.21.15.125:0/1446945822 wait complete. 2023-10-18T12:02:12.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.660+0000 7f8605b4c640 1 Processor -- start 2023-10-18T12:02:12.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.660+0000 7f8605b4c640 1 -- start start 2023-10-18T12:02:12.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f8605b4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85e80047c0 0x7f86001eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:12.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f8605b4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86000a64b0 0x7f86000a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:12.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f8605b4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86001f1d30 0x7f86000ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:12.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f8605b4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86000a69b0 con 0x7f86000a64b0 2023-10-18T12:02:12.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f8605b4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86000a6c90 con 0x7f86001f1d30 2023-10-18T12:02:12.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f8605b4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86000a6f70 con 0x7f85e80047c0 2023-10-18T12:02:12.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f85feffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86000a64b0 0x7f86000a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:12.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f85feffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86000a64b0 0x7f86000a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48670/0 (socket says 172.21.15.125:48670) 2023-10-18T12:02:12.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f85feffd640 1 -- 172.21.15.125:0/2890612345 learned_addr learned my addr 172.21.15.125:0/2890612345 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:12.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f85fffff640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86001f1d30 0x7f86000ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:12.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f85ff7fe640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85e80047c0 0x7f86001eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:12.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.661+0000 7f85feffd640 1 -- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85e80047c0 msgr2=0x7f86001eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:12.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85feffd640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85e80047c0 0x7f86001eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85feffd640 1 -- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86001f1d30 msgr2=0x7f86000ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:12.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85feffd640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86001f1d30 0x7f86000ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85feffd640 1 -- 172.21.15.125:0/2890612345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000a7250 con 0x7f86000a64b0 2023-10-18T12:02:12.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85fffff640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86001f1d30 0x7f86000ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:12.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85ff7fe640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85e80047c0 0x7f86001eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:12.670 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85feffd640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86000a64b0 0x7f86000a9e30 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7f85f4001910 tx=0x7f85f40062e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:12.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85e7fff640 1 -- 172.21.15.125:0/2890612345 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f403a070 con 0x7f86000a64b0 2023-10-18T12:02:12.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85e7fff640 1 -- 172.21.15.125:0/2890612345 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f85f402c9c0 con 0x7f86000a64b0 2023-10-18T12:02:12.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86000a7530 con 0x7f86000a64b0 2023-10-18T12:02:12.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f85e7fff640 1 -- 172.21.15.125:0/2890612345 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f402cd90 con 0x7f86000a64b0 2023-10-18T12:02:12.671 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.662+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86000a9130 con 0x7f86000a64b0 2023-10-18T12:02:12.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.663+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86000690e0 con 0x7f86000a64b0 2023-10-18T12:02:12.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.663+0000 7f85e7fff640 1 -- 172.21.15.125:0/2890612345 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 149) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f85f40035f0 con 0x7f86000a64b0 2023-10-18T12:02:12.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.664+0000 7f85e7fff640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f85ec06ad10 0x7f85ec06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:12.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.664+0000 7f85e7fff640 1 -- 172.21.15.125:0/2890612345 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7f85f40aa3a0 con 0x7f86000a64b0 2023-10-18T12:02:12.672 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.664+0000 7f85ff7fe640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f85ec06ad10 0x7f85ec06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:12.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.664+0000 7f85ff7fe640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f85ec06ad10 0x7f85ec06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8600158cd0 tx=0x7f85f0005f90 comp rx=0 tx=0).ready entity=mgr.11581 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:12.673 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.668+0000 7f85e7fff640 1 -- 172.21.15.125:0/2890612345 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85f40753c0 con 0x7f86000a64b0 2023-10-18T12:02:12.855 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.853+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 --> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8600158f90 con 0x7f85ec06ad10 2023-10-18T12:02:12.903 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.901+0000 7f85e7fff640 1 -- 172.21.15.125:0/2890612345 <== mgr.11581 v2:172.21.15.139:6832/1808248507 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f8600158f90 con 0x7f85ec06ad10 2023-10-18T12:02:12.904 INFO:teuthology.orchestra.run.smithi125.stdout:Retention added to path snap_test_dir1 2023-10-18T12:02:12.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f85ec06ad10 msgr2=0x7f85ec06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:12.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f85ec06ad10 0x7f85ec06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8600158cd0 tx=0x7f85f0005f90 comp rx=0 tx=0).stop 2023-10-18T12:02:12.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86000a64b0 msgr2=0x7f86000a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:12.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86000a64b0 0x7f86000a9e30 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7f85f4001910 tx=0x7f85f40062e0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 shutdown_connections 2023-10-18T12:02:12.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f85ec06ad10 0x7f85ec06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86001f1d30 0x7f86000ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86000a64b0 0x7f86000a9e30 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 --2- 172.21.15.125:0/2890612345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85e80047c0 0x7f86001eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:12.907 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 >> 172.21.15.125:0/2890612345 conn(0x7f8600069930 msgr2=0x7f860013b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:12.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 shutdown_connections 2023-10-18T12:02:12.908 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:12.903+0000 7f8605b4c640 1 -- 172.21.15.125:0/2890612345 wait complete. 2023-10-18T12:02:12.916 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-18T12:02:12.917 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-18T12:02:13.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.020+0000 7f29f395a640 1 -- 172.21.15.125:0/2040594984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 msgr2=0x7f29ec06b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.020+0000 7f29f395a640 1 --2- 172.21.15.125:0/2040594984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 0x7f29ec06b410 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f29e8005fd0 tx=0x7f29e802c0e0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.020+0000 7f29f395a640 1 -- 172.21.15.125:0/2040594984 shutdown_connections 2023-10-18T12:02:13.023 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.020+0000 7f29f395a640 1 --2- 172.21.15.125:0/2040594984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 0x7f29ec06b410 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.020+0000 7f29f395a640 1 -- 172.21.15.125:0/2040594984 >> 172.21.15.125:0/2040594984 conn(0x7f29ec069930 msgr2=0x7f29ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:13.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.020+0000 7f29f395a640 1 -- 172.21.15.125:0/2040594984 shutdown_connections 2023-10-18T12:02:13.024 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.020+0000 7f29f395a640 1 -- 172.21.15.125:0/2040594984 wait complete. 2023-10-18T12:02:13.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.021+0000 7f29f395a640 1 Processor -- start 2023-10-18T12:02:13.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.021+0000 7f29f395a640 1 -- start start 2023-10-18T12:02:13.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f395a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 0x7f29ec14f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:13.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f395a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0c9750 0x7f29ec13b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:13.027 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f395a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec159bd0 0x7f29ec151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:13.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f395a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec13b870 con 0x7f29ec06b030 2023-10-18T12:02:13.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f395a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec14f990 con 0x7f29ec0c9750 2023-10-18T12:02:13.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f395a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec14fc70 con 0x7f29ec159bd0 2023-10-18T12:02:13.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 0x7f29ec14f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:13.028 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 0x7f29ec14f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48688/0 (socket says 172.21.15.125:48688) 2023-10-18T12:02:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 -- 172.21.15.125:0/1667336731 learned_addr learned my addr 172.21.15.125:0/1667336731 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f0ef5640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0c9750 0x7f29ec13b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f1ef7640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec159bd0 0x7f29ec151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 -- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec159bd0 msgr2=0x7f29ec151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.029 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec159bd0 0x7f29ec151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 -- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0c9750 msgr2=0x7f29ec13b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0c9750 0x7f29ec13b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 -- 172.21.15.125:0/1667336731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e8009840 con 0x7f29ec06b030 2023-10-18T12:02:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f1ef7640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec159bd0 0x7f29ec151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:13.030 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f0ef5640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0c9750 0x7f29ec13b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:13.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.022+0000 7f29f16f6640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 0x7f29ec14f3b0 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f29e8036040 tx=0x7f29e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:13.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.023+0000 7f29e1ffb640 1 -- 172.21.15.125:0/1667336731 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e80046f0 con 0x7f29ec06b030 2023-10-18T12:02:13.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.023+0000 7f29e1ffb640 1 -- 172.21.15.125:0/1667336731 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29e80345a0 con 0x7f29ec06b030 2023-10-18T12:02:13.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.023+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec14ff50 con 0x7f29ec06b030 2023-10-18T12:02:13.031 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.023+0000 7f29e1ffb640 1 -- 172.21.15.125:0/1667336731 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e80349b0 con 0x7f29ec06b030 2023-10-18T12:02:13.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.023+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec150510 con 0x7f29ec06b030 2023-10-18T12:02:13.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.024+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec153640 con 0x7f29ec06b030 2023-10-18T12:02:13.032 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.024+0000 7f29e1ffb640 1 -- 172.21.15.125:0/1667336731 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 149) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f29e8034b10 con 0x7f29ec06b030 2023-10-18T12:02:13.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.025+0000 7f29e1ffb640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f29d006ad10 0x7f29d006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:13.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.025+0000 7f29e1ffb640 1 -- 172.21.15.125:0/1667336731 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7f29e80ac7b0 con 0x7f29ec06b030 2023-10-18T12:02:13.033 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.025+0000 7f29f0ef5640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f29d006ad10 0x7f29d006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:13.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.025+0000 7f29f0ef5640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f29d006ad10 0x7f29d006d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f29dc007fd0 tx=0x7f29dc009040 comp rx=0 tx=0).ready entity=mgr.11581 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:13.034 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.029+0000 7f29e1ffb640 1 -- 172.21.15.125:0/1667336731 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29e8077850 con 0x7f29ec06b030 2023-10-18T12:02:13.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.239+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f29ec0690e0 con 0x7f29ec06b030 2023-10-18T12:02:13.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.480+0000 7f29e1ffb640 1 -- 172.21.15.125:0/1667336731 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v150) v1 ==== 93+0+0 (secure 0 0 0) 0x7f29e807ae70 con 0x7f29ec06b030 2023-10-18T12:02:13.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f29d006ad10 msgr2=0x7f29d006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f29d006ad10 0x7f29d006d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f29dc007fd0 tx=0x7f29dc009040 comp rx=0 tx=0).stop 2023-10-18T12:02:13.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 msgr2=0x7f29ec14f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 0x7f29ec14f3b0 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f29e8036040 tx=0x7f29e80042f0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 shutdown_connections 2023-10-18T12:02:13.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f29d006ad10 0x7f29d006d1d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ec0c9750 0x7f29ec13b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ec159bd0 0x7f29ec151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 --2- 172.21.15.125:0/1667336731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ec06b030 0x7f29ec14f3b0 unknown :-1 s=CLOSED pgs=3063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 >> 172.21.15.125:0/1667336731 conn(0x7f29ec069930 msgr2=0x7f29ec069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:13.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 shutdown_connections 2023-10-18T12:02:13.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.482+0000 7f29f395a640 1 -- 172.21.15.125:0/1667336731 wait complete. 2023-10-18T12:02:13.497 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T12:02:13.498 DEBUG:teuthology.orchestra.run.smithi125:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:02:13.536 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T12:02:13.582 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T12:02:13.582 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:02:13.583 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:02:13.598 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:02:13.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.739+0000 7f847bce5640 1 -- 172.21.15.125:0/1701238408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f847415e5e0 msgr2=0x7f84740ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.739+0000 7f847bce5640 1 --2- 172.21.15.125:0/1701238408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f847415e5e0 0x7f84740ce9a0 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7f84740b1c50 tx=0x7f847002be40 comp rx=0 tx=0).stop 2023-10-18T12:02:13.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.739+0000 7f847bce5640 1 -- 172.21.15.125:0/1701238408 shutdown_connections 2023-10-18T12:02:13.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.739+0000 7f847bce5640 1 --2- 172.21.15.125:0/1701238408 >> v2:172.21.15.139:3300/0 conn(0x7f84740cf8b0 0x7f84740b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.739+0000 7f847bce5640 1 --2- 172.21.15.125:0/1701238408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f847415e5e0 0x7f84740ce9a0 unknown :-1 s=CLOSED pgs=3068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.739+0000 7f847bce5640 1 -- 172.21.15.125:0/1701238408 >> 172.21.15.125:0/1701238408 conn(0x7f84740d0c50 msgr2=0x7f84740d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:13.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.739+0000 7f847bce5640 1 -- 172.21.15.125:0/1701238408 shutdown_connections 2023-10-18T12:02:13.743 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.739+0000 7f847bce5640 1 -- 172.21.15.125:0/1701238408 wait complete. 2023-10-18T12:02:13.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.740+0000 7f847bce5640 1 Processor -- start 2023-10-18T12:02:13.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.740+0000 7f847bce5640 1 -- start start 2023-10-18T12:02:13.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f847bce5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84740b1ca0 0x7f847406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:13.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f847bce5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84740cf8b0 0x7f84740b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:13.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f847bce5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f847415e5e0 0x7f84740b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:13.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f847bce5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84740b1120 con 0x7f84740b1ca0 2023-10-18T12:02:13.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f847bce5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84740adb60 con 0x7f84740cf8b0 2023-10-18T12:02:13.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f847bce5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84741f5a00 con 0x7f847415e5e0 2023-10-18T12:02:13.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f8479a81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84740b1ca0 0x7f847406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:13.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f8479a81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84740b1ca0 0x7f847406b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48708/0 (socket says 172.21.15.125:48708) 2023-10-18T12:02:13.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f8479a81640 1 -- 172.21.15.125:0/157465450 learned_addr learned my addr 172.21.15.125:0/157465450 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:13.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f8479280640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84740cf8b0 0x7f84740b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:13.748 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.741+0000 7f847a282640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f847415e5e0 0x7f84740b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:13.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f8479a81640 1 -- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f847415e5e0 msgr2=0x7f84740b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f8479a81640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f847415e5e0 0x7f84740b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f8479a81640 1 -- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84740cf8b0 msgr2=0x7f84740b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f8479a81640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84740cf8b0 0x7f84740b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f8479a81640 1 -- 172.21.15.125:0/157465450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009560 con 0x7f84740b1ca0 2023-10-18T12:02:13.749 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f8479a81640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84740b1ca0 0x7f847406b1f0 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 crypto rx=0x7f8470009850 tx=0x7f847002db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:13.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f846a7fc640 1 -- 172.21.15.125:0/157465450 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470003800 con 0x7f84740b1ca0 2023-10-18T12:02:13.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f846a7fc640 1 -- 172.21.15.125:0/157465450 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8470003a50 con 0x7f84740b1ca0 2023-10-18T12:02:13.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84740ade00 con 0x7f84740b1ca0 2023-10-18T12:02:13.750 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f846a7fc640 1 -- 172.21.15.125:0/157465450 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470003e20 con 0x7f84740b1ca0 2023-10-18T12:02:13.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.742+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847406b730 con 0x7f84740b1ca0 2023-10-18T12:02:13.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.743+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84740d0440 con 0x7f84740b1ca0 2023-10-18T12:02:13.751 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.744+0000 7f846a7fc640 1 -- 172.21.15.125:0/157465450 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 150) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f847003e450 con 0x7f84740b1ca0 2023-10-18T12:02:13.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.744+0000 7f846a7fc640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f845806acc0 0x7f845806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:13.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.744+0000 7f8479280640 1 -- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f845806acc0 msgr2=0x7f845806d180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:13.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.744+0000 7f846a7fc640 1 -- 172.21.15.125:0/157465450 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7f84700abf10 con 0x7f84740b1ca0 2023-10-18T12:02:13.752 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.744+0000 7f8479280640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f845806acc0 0x7f845806d180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:13.753 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.748+0000 7f846a7fc640 1 -- 172.21.15.125:0/157465450 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8470077180 con 0x7f84740b1ca0 2023-10-18T12:02:13.871 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:13.868+0000 7f592c411040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:02:13.874 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:13.872+0000 7f076480d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T12:02:13.915 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:13.913+0000 7f592c411040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:02:13.921 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:13.918+0000 7f076480d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T12:02:13.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.924+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f847406b8c0 con 0x7f84740b1ca0 2023-10-18T12:02:13.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.925+0000 7f846a7fc640 1 -- 172.21.15.125:0/157465450 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v264) v1 ==== 74+0+17554 (secure 0 0 0) 0x7f8470042090 con 0x7f84740b1ca0 2023-10-18T12:02:13.928 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:13.928 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":264,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:02:11.466373+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:57.717906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:58.385232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2488548050":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2767408321":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6832/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1739931468":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2511136316":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6833/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2813907753":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1094096408":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1205310989":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1383817777":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2446560793":"2023-10-18T13:02:05.851547+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:02:13.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f845806acc0 msgr2=0x7f845806d180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:13.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f845806acc0 0x7f845806d180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84740b1ca0 msgr2=0x7f847406b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84740b1ca0 0x7f847406b1f0 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 crypto rx=0x7f8470009850 tx=0x7f847002db10 comp rx=0 tx=0).stop 2023-10-18T12:02:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 shutdown_connections 2023-10-18T12:02:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f845806acc0 0x7f845806d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84740cf8b0 0x7f84740b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f847415e5e0 0x7f84740b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 --2- 172.21.15.125:0/157465450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84740b1ca0 0x7f847406b1f0 unknown :-1 s=CLOSED pgs=3069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 >> 172.21.15.125:0/157465450 conn(0x7f84740d0c50 msgr2=0x7f847415bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 shutdown_connections 2023-10-18T12:02:13.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:13.927+0000 7f847bce5640 1 -- 172.21.15.125:0/157465450 wait complete. 2023-10-18T12:02:13.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:01:57.717906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '261', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:01:58.385232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '262', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:02:13.942 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:02:14.049 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.047+0000 7fee2fba5640 1 -- 172.21.15.125:0/3411733619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee280ceb40 msgr2=0x7fee280b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.047+0000 7fee2fba5640 1 --2- 172.21.15.125:0/3411733619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee280ceb40 0x7fee280b0650 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 crypto rx=0x7fee20005fd0 tx=0x7fee2002c240 comp rx=0 tx=0).stop 2023-10-18T12:02:14.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.047+0000 7fee2fba5640 1 -- 172.21.15.125:0/3411733619 shutdown_connections 2023-10-18T12:02:14.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.047+0000 7fee2fba5640 1 --2- 172.21.15.125:0/3411733619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee280ceb40 0x7fee280b0650 unknown :-1 s=CLOSED pgs=3070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.050 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.047+0000 7fee2fba5640 1 --2- 172.21.15.125:0/3411733619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee2815b2a0 0x7fee280d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.047+0000 7fee2fba5640 1 -- 172.21.15.125:0/3411733619 >> 172.21.15.125:0/3411733619 conn(0x7fee280d0d60 msgr2=0x7fee280d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:14.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.047+0000 7fee2fba5640 1 -- 172.21.15.125:0/3411733619 shutdown_connections 2023-10-18T12:02:14.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.047+0000 7fee2fba5640 1 -- 172.21.15.125:0/3411733619 wait complete. 2023-10-18T12:02:14.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.048+0000 7fee2fba5640 1 Processor -- start 2023-10-18T12:02:14.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.048+0000 7fee2fba5640 1 -- start start 2023-10-18T12:02:14.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2fba5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee280ceb40 0x7fee28150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2fba5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee280cf340 0x7fee280c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2fba5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee2815b2a0 0x7fee280c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2fba5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee280c9d10 con 0x7fee2815b2a0 2023-10-18T12:02:14.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2fba5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281e4a20 con 0x7fee280cf340 2023-10-18T12:02:14.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2fba5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee28150e50 con 0x7fee280ceb40 2023-10-18T12:02:14.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee2815b2a0 0x7fee280c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee2815b2a0 0x7fee280c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48726/0 (socket says 172.21.15.125:48726) 2023-10-18T12:02:14.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 -- 172.21.15.125:0/1872752879 learned_addr learned my addr 172.21.15.125:0/1872752879 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:14.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2d941640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee280ceb40 0x7fee28150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2d140640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee280cf340 0x7fee280c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 -- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee280ceb40 msgr2=0x7fee28150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee280ceb40 0x7fee28150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 -- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee280cf340 msgr2=0x7fee280c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee280cf340 0x7fee280c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 -- 172.21.15.125:0/1872752879 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20009900 con 0x7fee2815b2a0 2023-10-18T12:02:14.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2d941640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee280ceb40 0x7fee28150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:02:14.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.049+0000 7fee2e142640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee2815b2a0 0x7fee280c8830 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7fee20037040 tx=0x7fee20035a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:14.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.050+0000 7fee1e7fc640 1 -- 172.21.15.125:0/1872752879 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20034440 con 0x7fee2815b2a0 2023-10-18T12:02:14.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.050+0000 7fee1e7fc640 1 -- 172.21.15.125:0/1872752879 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee20004330 con 0x7fee2815b2a0 2023-10-18T12:02:14.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.050+0000 7fee1e7fc640 1 -- 172.21.15.125:0/1872752879 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200046c0 con 0x7fee2815b2a0 2023-10-18T12:02:14.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.050+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee280b5a20 con 0x7fee2815b2a0 2023-10-18T12:02:14.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.050+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee280b5fe0 con 0x7fee2815b2a0 2023-10-18T12:02:14.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.051+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee281e4bb0 con 0x7fee2815b2a0 2023-10-18T12:02:14.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.051+0000 7fee1e7fc640 1 -- 172.21.15.125:0/1872752879 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 150) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fee2003b040 con 0x7fee2815b2a0 2023-10-18T12:02:14.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.052+0000 7fee1e7fc640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fee0c06ac90 0x7fee0c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.052+0000 7fee2d941640 1 -- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fee0c06ac90 msgr2=0x7fee0c06d150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:14.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.052+0000 7fee1e7fc640 1 -- 172.21.15.125:0/1872752879 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7fee200abf10 con 0x7fee2815b2a0 2023-10-18T12:02:14.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.052+0000 7fee2d941640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fee0c06ac90 0x7fee0c06d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:14.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.056+0000 7fee1e7fc640 1 -- 172.21.15.125:0/1872752879 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee20077060 con 0x7fee2815b2a0 2023-10-18T12:02:14.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.228+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee280cb980 con 0x7fee2815b2a0 2023-10-18T12:02:14.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.229+0000 7fee1e7fc640 1 -- 172.21.15.125:0/1872752879 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v264) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee200390a0 con 0x7fee2815b2a0 2023-10-18T12:02:14.232 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:02:14.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fee0c06ac90 msgr2=0x7fee0c06d150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:14.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fee0c06ac90 0x7fee0c06d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee2815b2a0 msgr2=0x7fee280c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee2815b2a0 0x7fee280c8830 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7fee20037040 tx=0x7fee20035a20 comp rx=0 tx=0).stop 2023-10-18T12:02:14.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 shutdown_connections 2023-10-18T12:02:14.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fee0c06ac90 0x7fee0c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee280cf340 0x7fee280c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fee2815b2a0 0x7fee280c8830 unknown :-1 s=CLOSED pgs=3071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 --2- 172.21.15.125:0/1872752879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee280ceb40 0x7fee28150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 >> 172.21.15.125:0/1872752879 conn(0x7fee280d0d60 msgr2=0x7fee281512b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:14.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 shutdown_connections 2023-10-18T12:02:14.236 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.231+0000 7fee2fba5640 1 -- 172.21.15.125:0/1872752879 wait complete. 2023-10-18T12:02:14.244 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:02:14.303 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.301+0000 7f592c411040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:02:14.331 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.329+0000 7f076480d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T12:02:14.348 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.345+0000 7f592c411040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:02:14.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.351+0000 7f2f7befc640 1 -- 172.21.15.125:0/3981126464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740b1980 msgr2=0x7f2f74070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.351+0000 7f2f7befc640 1 --2- 172.21.15.125:0/3981126464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740b1980 0x7f2f74070080 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009830 tx=0x7f2f7002bd60 comp rx=0 tx=0).stop 2023-10-18T12:02:14.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.351+0000 7f2f7befc640 1 -- 172.21.15.125:0/3981126464 shutdown_connections 2023-10-18T12:02:14.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.351+0000 7f2f7befc640 1 --2- 172.21.15.125:0/3981126464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740b1980 0x7f2f74070080 unknown :-1 s=CLOSED pgs=3073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.351+0000 7f2f7befc640 1 --2- 172.21.15.125:0/3981126464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f741416f0 0x7f2f740b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.351+0000 7f2f7befc640 1 -- 172.21.15.125:0/3981126464 >> 172.21.15.125:0/3981126464 conn(0x7f2f740af730 msgr2=0x7f2f740afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:14.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.351+0000 7f2f7befc640 1 -- 172.21.15.125:0/3981126464 shutdown_connections 2023-10-18T12:02:14.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.351+0000 7f2f7befc640 1 -- 172.21.15.125:0/3981126464 wait complete. 2023-10-18T12:02:14.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.353+0000 7f2f7befc640 1 Processor -- start 2023-10-18T12:02:14.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.353+0000 7f2f7befc640 1 -- start start 2023-10-18T12:02:14.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f7befc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740705c0 0x7f2f74134520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f7befc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f740b1980 0x7f2f74158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f7befc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f741416f0 0x7f2f7413ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f7befc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74134a60 con 0x7f2f740705c0 2023-10-18T12:02:14.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f7befc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740a6da0 con 0x7f2f741416f0 2023-10-18T12:02:14.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f7befc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740a7080 con 0x7f2f740b1980 2023-10-18T12:02:14.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79c98640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740705c0 0x7f2f74134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79c98640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740705c0 0x7f2f74134520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48746/0 (socket says 172.21.15.125:48746) 2023-10-18T12:02:14.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79c98640 1 -- 172.21.15.125:0/2758200300 learned_addr learned my addr 172.21.15.125:0/2758200300 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:14.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79497640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f740b1980 0x7f2f74158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f7a499640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f741416f0 0x7f2f7413ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79c98640 1 -- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f740b1980 msgr2=0x7f2f74158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79c98640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f740b1980 0x7f2f74158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.360 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79c98640 1 -- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f741416f0 msgr2=0x7f2f7413ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79c98640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f741416f0 0x7f2f7413ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f79c98640 1 -- 172.21.15.125:0/2758200300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f70009490 con 0x7f2f740705c0 2023-10-18T12:02:14.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.354+0000 7f2f7a499640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f741416f0 0x7f2f7413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:14.361 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.355+0000 7f2f79c98640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740705c0 0x7f2f74134520 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400e790 tx=0x7f2f6400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:14.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.355+0000 7f2f6a7fc640 1 -- 172.21.15.125:0/2758200300 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64009580 con 0x7f2f740705c0 2023-10-18T12:02:14.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.355+0000 7f2f6a7fc640 1 -- 172.21.15.125:0/2758200300 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2f6400cdb0 con 0x7f2f740705c0 2023-10-18T12:02:14.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.355+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f740a7360 con 0x7f2f740705c0 2023-10-18T12:02:14.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.355+0000 7f2f6a7fc640 1 -- 172.21.15.125:0/2758200300 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64010690 con 0x7f2f740705c0 2023-10-18T12:02:14.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.355+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f740a7920 con 0x7f2f740705c0 2023-10-18T12:02:14.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.356+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f740ca800 con 0x7f2f740705c0 2023-10-18T12:02:14.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.356+0000 7f2f6a7fc640 1 -- 172.21.15.125:0/2758200300 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 150) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f2f640107f0 con 0x7f2f740705c0 2023-10-18T12:02:14.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.357+0000 7f2f6a7fc640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f2f5806ac90 0x7f2f5806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.357+0000 7f2f79497640 1 -- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f2f5806ac90 msgr2=0x7f2f5806d150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:14.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.357+0000 7f2f6a7fc640 1 -- 172.21.15.125:0/2758200300 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7f2f64014070 con 0x7f2f740705c0 2023-10-18T12:02:14.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.357+0000 7f2f79497640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f2f5806ac90 0x7f2f5806d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:14.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.361+0000 7f2f6a7fc640 1 -- 172.21.15.125:0/2758200300 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f64051ca0 con 0x7f2f740705c0 2023-10-18T12:02:14.380 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.378+0000 7f076480d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T12:02:14.390 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.388+0000 7f592c411040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:02:14.425 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.423+0000 7f076480d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T12:02:14.444 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.442+0000 7f592c411040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:02:14.479 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.476+0000 7f076480d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T12:02:14.486 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.484+0000 7f592c411040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:02:14.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.494+0000 7f2f6a7fc640 1 -- 172.21.15.125:0/2758200300 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 151) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2f640554a0 con 0x7f2f740705c0 2023-10-18T12:02:14.525 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.522+0000 7f076480d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T12:02:14.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.536+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f740ad7d0 con 0x7f2f740705c0 2023-10-18T12:02:14.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.537+0000 7f2f6a7fc640 1 -- 172.21.15.125:0/2758200300 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v264) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f640880f0 con 0x7f2f740705c0 2023-10-18T12:02:14.539 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:02:14.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.538+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f2f5806ac90 msgr2=0x7f2f5806d150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:14.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.538+0000 7f2f7befc640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f2f5806ac90 0x7f2f5806d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.538+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740705c0 msgr2=0x7f2f74134520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.538+0000 7f2f7befc640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740705c0 0x7f2f74134520 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400e790 tx=0x7f2f6400ec50 comp rx=0 tx=0).stop 2023-10-18T12:02:14.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.539+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 shutdown_connections 2023-10-18T12:02:14.542 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.539+0000 7f2f7befc640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f2f5806ac90 0x7f2f5806d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.539+0000 7f2f7befc640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f741416f0 0x7f2f7413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.539+0000 7f2f7befc640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f740b1980 0x7f2f74158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.539+0000 7f2f7befc640 1 --2- 172.21.15.125:0/2758200300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2f740705c0 0x7f2f74134520 unknown :-1 s=CLOSED pgs=3074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.539+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 >> 172.21.15.125:0/2758200300 conn(0x7f2f740af730 msgr2=0x7f2f7406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:14.543 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.539+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 shutdown_connections 2023-10-18T12:02:14.544 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.539+0000 7f2f7befc640 1 -- 172.21.15.125:0/2758200300 wait complete. 2023-10-18T12:02:14.552 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:02:14.588 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.585+0000 7f592c411040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:02:14.628 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.626+0000 7f076480d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T12:02:14.640 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.638+0000 7f592c411040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:02:14.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.656+0000 7ff675b16640 1 -- 172.21.15.125:0/1718173672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 msgr2=0x7ff6700b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.659 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.656+0000 7ff675b16640 1 --2- 172.21.15.125:0/1718173672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 0x7ff6700b0650 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff668005fd0 tx=0x7ff66802c050 comp rx=0 tx=0).stop 2023-10-18T12:02:14.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.656+0000 7ff675b16640 1 -- 172.21.15.125:0/1718173672 shutdown_connections 2023-10-18T12:02:14.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.656+0000 7ff675b16640 1 --2- 172.21.15.125:0/1718173672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 0x7ff6700b0650 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.656+0000 7ff675b16640 1 --2- 172.21.15.125:0/1718173672 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700cf340 0x7ff6700d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.657+0000 7ff675b16640 1 -- 172.21.15.125:0/1718173672 >> 172.21.15.125:0/1718173672 conn(0x7ff6700d0d60 msgr2=0x7ff6700d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:14.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.657+0000 7ff675b16640 1 -- 172.21.15.125:0/1718173672 shutdown_connections 2023-10-18T12:02:14.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.657+0000 7ff675b16640 1 -- 172.21.15.125:0/1718173672 wait complete. 2023-10-18T12:02:14.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.658+0000 7ff675b16640 1 Processor -- start 2023-10-18T12:02:14.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.658+0000 7ff675b16640 1 -- start start 2023-10-18T12:02:14.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff675b16640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 0x7ff6701592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff675b16640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6700cf340 0x7ff67015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff675b16640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff67015b2a0 0x7ff6700b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff675b16640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670159830 con 0x7ff6700cf340 2023-10-18T12:02:14.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff675b16640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67015bf00 con 0x7ff6700ceb40 2023-10-18T12:02:14.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff675b16640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67015c1e0 con 0x7ff67015b2a0 2023-10-18T12:02:14.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 0x7ff6701592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6700cf340 0x7ff67015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 0x7ff6701592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.125:40452/0 (socket says 172.21.15.125:40452) 2023-10-18T12:02:14.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 -- 172.21.15.125:0/3713742451 learned_addr learned my addr 172.21.15.125:0/3713742451 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:14.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66ffff640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff67015b2a0 0x7ff6700b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 -- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff67015b2a0 msgr2=0x7ff6700b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff67015b2a0 0x7ff6700b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 -- 172.21.15.125:0/3713742451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6700cf340 msgr2=0x7ff67015b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6700cf340 0x7ff67015b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 -- 172.21.15.125:0/3713742451 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660009490 con 0x7ff6700ceb40 2023-10-18T12:02:14.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff66f7fe640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 0x7ff6701592f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff66000ea60 tx=0x7ff66000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:14.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff657fff640 1 -- 172.21.15.125:0/3713742451 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000cde0 con 0x7ff6700ceb40 2023-10-18T12:02:14.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff657fff640 1 -- 172.21.15.125:0/3713742451 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff660004020 con 0x7ff6700ceb40 2023-10-18T12:02:14.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff657fff640 1 -- 172.21.15.125:0/3713742451 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660004410 con 0x7ff6700ceb40 2023-10-18T12:02:14.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.659+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff668009840 con 0x7ff6700ceb40 2023-10-18T12:02:14.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.660+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67015c7a0 con 0x7ff6700ceb40 2023-10-18T12:02:14.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.660+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6700690e0 con 0x7ff6700ceb40 2023-10-18T12:02:14.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.661+0000 7ff657fff640 1 -- 172.21.15.125:0/3713742451 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 151) v1 ==== 89332+0+0 (secure 0 0 0) 0x7ff6600035f0 con 0x7ff6700ceb40 2023-10-18T12:02:14.668 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.661+0000 7ff657fff640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7ff65c06af60 0x7ff65c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.661+0000 7ff66effd640 1 -- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7ff65c06af60 msgr2=0x7ff65c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:14.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.661+0000 7ff657fff640 1 -- 172.21.15.125:0/3713742451 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7ff660014070 con 0x7ff6700ceb40 2023-10-18T12:02:14.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.661+0000 7ff66effd640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7ff65c06af60 0x7ff65c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:14.669 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.666+0000 7ff657fff640 1 -- 172.21.15.125:0/3713742451 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff660055300 con 0x7ff6700ceb40 2023-10-18T12:02:14.683 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.681+0000 7f592c411040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:02:14.684 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.682+0000 7f076480d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T12:02:14.728 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.725+0000 7f076480d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T12:02:14.731 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.729+0000 7f592c411040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:02:14.773 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:14.770+0000 7f592c411040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:02:14.777 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.775+0000 7f076480d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T12:02:14.821 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:14.819+0000 7f076480d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T12:02:14.840 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.838+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6700cb9b0 con 0x7ff6700ceb40 2023-10-18T12:02:14.841 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.838+0000 7ff657fff640 1 -- 172.21.15.125:0/3713742451 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v264) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff660058920 con 0x7ff6700ceb40 2023-10-18T12:02:14.841 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:02:14.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7ff65c06af60 msgr2=0x7ff65c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:14.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7ff65c06af60 0x7ff65c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 msgr2=0x7ff6701592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 0x7ff6701592f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff66000ea60 tx=0x7ff66000ef20 comp rx=0 tx=0).stop 2023-10-18T12:02:14.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 shutdown_connections 2023-10-18T12:02:14.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7ff65c06af60 0x7ff65c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff67015b2a0 0x7ff6700b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6700cf340 0x7ff67015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 --2- 172.21.15.125:0/3713742451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6700ceb40 0x7ff6701592f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 >> 172.21.15.125:0/3713742451 conn(0x7ff6700d0d60 msgr2=0x7ff67006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:14.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 shutdown_connections 2023-10-18T12:02:14.845 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.840+0000 7ff675b16640 1 -- 172.21.15.125:0/3713742451 wait complete. 2023-10-18T12:02:14.854 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T12:02:14.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.957+0000 7fd05a239640 1 -- 172.21.15.125:0/1029877296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0540cf340 msgr2=0x7fd0540d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.957+0000 7fd05a239640 1 --2- 172.21.15.125:0/1029877296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0540cf340 0x7fd0540d1910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd044009830 tx=0x7fd04402bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.957+0000 7fd05a239640 1 -- 172.21.15.125:0/1029877296 shutdown_connections 2023-10-18T12:02:14.960 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.957+0000 7fd05a239640 1 --2- 172.21.15.125:0/1029877296 >> v2:172.21.15.139:3300/0 conn(0x7fd0540ceb40 0x7fd0540b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.957+0000 7fd05a239640 1 --2- 172.21.15.125:0/1029877296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0540cf340 0x7fd0540d1910 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.957+0000 7fd05a239640 1 -- 172.21.15.125:0/1029877296 >> 172.21.15.125:0/1029877296 conn(0x7fd0540d0d60 msgr2=0x7fd0540d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:14.961 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.957+0000 7fd05a239640 1 -- 172.21.15.125:0/1029877296 shutdown_connections 2023-10-18T12:02:14.962 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.957+0000 7fd05a239640 1 -- 172.21.15.125:0/1029877296 wait complete. 2023-10-18T12:02:14.964 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.958+0000 7fd05a239640 1 Processor -- start 2023-10-18T12:02:14.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.958+0000 7fd05a239640 1 -- start start 2023-10-18T12:02:14.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd05a239640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0540ceb40 0x7fd0540c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd05a239640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0540cf340 0x7fd05415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd05a239640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd05415b2a0 0x7fd054142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.965 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd05a239640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05415c750 con 0x7fd05415b2a0 2023-10-18T12:02:14.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd05a239640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05415c470 con 0x7fd0540cf340 2023-10-18T12:02:14.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd05a239640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05415d3a0 con 0x7fd0540ceb40 2023-10-18T12:02:14.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd05415b2a0 0x7fd054142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd05415b2a0 0x7fd054142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48772/0 (socket says 172.21.15.125:48772) 2023-10-18T12:02:14.966 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 -- 172.21.15.125:0/3718045333 learned_addr learned my addr 172.21.15.125:0/3718045333 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:14.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd0537fe640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0540ceb40 0x7fd0540c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd052ffd640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0540cf340 0x7fd05415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:14.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 -- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0540ceb40 msgr2=0x7fd0540c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.967 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0540ceb40 0x7fd0540c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 -- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0540cf340 msgr2=0x7fd05415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:14.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0540cf340 0x7fd05415de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:14.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 -- 172.21.15.125:0/3718045333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c009840 con 0x7fd05415b2a0 2023-10-18T12:02:14.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.959+0000 7fd053fff640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd05415b2a0 0x7fd054142c80 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00efd0 tx=0x7fd04c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:14.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.960+0000 7fd04bfff640 1 -- 172.21.15.125:0/3718045333 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c009b40 con 0x7fd05415b2a0 2023-10-18T12:02:14.968 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.960+0000 7fd04bfff640 1 -- 172.21.15.125:0/3718045333 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd04c003c60 con 0x7fd05415b2a0 2023-10-18T12:02:14.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.960+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd044009490 con 0x7fd05415b2a0 2023-10-18T12:02:14.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.960+0000 7fd04bfff640 1 -- 172.21.15.125:0/3718045333 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c004020 con 0x7fd05415b2a0 2023-10-18T12:02:14.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.960+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05415d960 con 0x7fd05415b2a0 2023-10-18T12:02:14.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.961+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05415db50 con 0x7fd05415b2a0 2023-10-18T12:02:14.969 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.961+0000 7fd04bfff640 1 -- 172.21.15.125:0/3718045333 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 151) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd04c0041a0 con 0x7fd05415b2a0 2023-10-18T12:02:14.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.962+0000 7fd04bfff640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fd03406af60 0x7fd03406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:14.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.962+0000 7fd0537fe640 1 -- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fd03406af60 msgr2=0x7fd03406d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:14.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.962+0000 7fd04bfff640 1 -- 172.21.15.125:0/3718045333 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7fd04c014070 con 0x7fd05415b2a0 2023-10-18T12:02:14.970 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.962+0000 7fd0537fe640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fd03406af60 0x7fd03406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:14.971 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:14.966+0000 7fd04bfff640 1 -- 172.21.15.125:0/3718045333 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd04c055830 con 0x7fd05415b2a0 2023-10-18T12:02:15.093 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.090+0000 7f592c411040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:02:15.153 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.151+0000 7f076480d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T12:02:15.156 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.154+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd0540690e0 con 0x7fd05415b2a0 2023-10-18T12:02:15.157 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.154+0000 7fd04bfff640 1 -- 172.21.15.125:0/3718045333 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v264) v1 ==== 81+0+653 (secure 0 0 0) 0x7fd04c058e50 con 0x7fd05415b2a0 2023-10-18T12:02:15.157 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2488548050 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.157 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2767408321 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.157 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6832/666436799 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.158 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1739931468 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.158 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2511136316 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.158 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:6833/666436799 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.159 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2813907753 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.159 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1094096408 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.159 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1205310989 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.159 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/1383817777 2023-10-19T12:02:11.466337+0000 2023-10-18T12:02:15.159 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.139:0/2446560793 2023-10-18T13:02:05.851547+0000 2023-10-18T12:02:15.160 INFO:teuthology.orchestra.run.smithi125.stderr:listed 11 entries 2023-10-18T12:02:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fd03406af60 msgr2=0x7fd03406d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fd03406af60 0x7fd03406d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd05415b2a0 msgr2=0x7fd054142c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:15.161 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd05415b2a0 0x7fd054142c80 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00efd0 tx=0x7fd04c00c6a0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 shutdown_connections 2023-10-18T12:02:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fd03406af60 0x7fd03406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0540cf340 0x7fd05415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd05415b2a0 0x7fd054142c80 unknown :-1 s=CLOSED pgs=3076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.162 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 --2- 172.21.15.125:0/3718045333 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0540ceb40 0x7fd0540c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.163 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.156+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 >> 172.21.15.125:0/3718045333 conn(0x7fd0540d0d60 msgr2=0x7fd0540ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:15.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.157+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 shutdown_connections 2023-10-18T12:02:15.165 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.157+0000 7fd05a239640 1 -- 172.21.15.125:0/3718045333 wait complete. 2023-10-18T12:02:15.170 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi125.front.sepia.ceph.com... 2023-10-18T12:02:15.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:15.171 DEBUG:teuthology.orchestra.run.smithi125:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-18T12:02:15.196 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi125.stderr:ceph-fuse[164915]: fuse finished with error 0 and tester_r 0 2023-10-18T12:02:15.198 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:02:15.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:15.199 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T12:02:15.223 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.221+0000 7f592c411040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:02:15.290 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.288+0000 7f076480d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T12:02:15.315 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.312+0000 7f592c411040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:02:15.325 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:02:15.326 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:02:15.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:02:15.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:02:15.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:15.342 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T12:02:15.359 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.357+0000 7f592c411040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:02:15.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:02:15.373 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-18T12:02:15.374 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:02:15.374 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:02:15.386 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.383+0000 7f076480d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T12:02:15.389 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:02:15.390 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:02:15.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:15.391 DEBUG:teuthology.orchestra.run.smithi125:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T12:02:15.429 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.427+0000 7f076480d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T12:02:15.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:02:15.444 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-18T12:02:15.445 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:02:15.445 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:02:15.457 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.455+0000 7f592c411040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:02:15.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:02:15.461 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:02:15.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:15.462 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T12:02:15.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:02:15.515 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-18T12:02:15.516 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:02:15.528 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.526+0000 7f076480d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:02:15.553 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.550+0000 7f592c411040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:02:15.621 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.618+0000 7f6b6a261640 1 -- 172.21.15.125:0/550182440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413d350 msgr2=0x7f6b640cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:15.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.618+0000 7f6b6a261640 1 --2- 172.21.15.125:0/550182440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413d350 0x7f6b640cbd60 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 crypto rx=0x7f6b6406a7e0 tx=0x7f6b5802be40 comp rx=0 tx=0).stop 2023-10-18T12:02:15.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.619+0000 7f6b6a261640 1 -- 172.21.15.125:0/550182440 shutdown_connections 2023-10-18T12:02:15.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.619+0000 7f6b6a261640 1 --2- 172.21.15.125:0/550182440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413d350 0x7f6b640cbd60 unknown :-1 s=CLOSED pgs=3077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.619+0000 7f6b6a261640 1 --2- 172.21.15.125:0/550182440 >> v2:172.21.15.139:3300/0 conn(0x7f6b640cb980 0x7f6b6413db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.619+0000 7f6b6a261640 1 -- 172.21.15.125:0/550182440 >> 172.21.15.125:0/550182440 conn(0x7f6b64069930 msgr2=0x7f6b64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:15.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.619+0000 7f6b6a261640 1 -- 172.21.15.125:0/550182440 shutdown_connections 2023-10-18T12:02:15.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.619+0000 7f6b6a261640 1 -- 172.21.15.125:0/550182440 wait complete. 2023-10-18T12:02:15.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.620+0000 7f6b6a261640 1 Processor -- start 2023-10-18T12:02:15.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.620+0000 7f6b6a261640 1 -- start start 2023-10-18T12:02:15.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.620+0000 7f6b6a261640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b640c9750 0x7f6b640b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:15.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.620+0000 7f6b6a261640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b640cb980 0x7f6b641f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:15.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b6a261640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413bab0 0x7f6b64134db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:15.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b6a261640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640aee50 con 0x7f6b6413bab0 2023-10-18T12:02:15.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b6a261640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640afe00 con 0x7f6b640cb980 2023-10-18T12:02:15.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b6a261640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640b0300 con 0x7f6b640c9750 2023-10-18T12:02:15.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413bab0 0x7f6b64134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:15.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413bab0 0x7f6b64134db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48780/0 (socket says 172.21.15.125:48780) 2023-10-18T12:02:15.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 -- 172.21.15.125:0/3383217831 learned_addr learned my addr 172.21.15.125:0/3383217831 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:15.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b62ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b640cb980 0x7f6b641f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:15.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b637fe640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b640c9750 0x7f6b640b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:15.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 -- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b640c9750 msgr2=0x7f6b640b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:15.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b640c9750 0x7f6b640b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 -- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b640cb980 msgr2=0x7f6b641f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b640cb980 0x7f6b641f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 -- 172.21.15.125:0/3383217831 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58009560 con 0x7f6b6413bab0 2023-10-18T12:02:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b63fff640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413bab0 0x7f6b64134db0 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c7a0 tx=0x7f6b5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b4bfff640 1 -- 172.21.15.125:0/3383217831 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c003b70 con 0x7f6b6413bab0 2023-10-18T12:02:15.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b4bfff640 1 -- 172.21.15.125:0/3383217831 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b5c003cf0 con 0x7f6b6413bab0 2023-10-18T12:02:15.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b640aeb70 con 0x7f6b6413bab0 2023-10-18T12:02:15.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b4bfff640 1 -- 172.21.15.125:0/3383217831 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00aab0 con 0x7f6b6413bab0 2023-10-18T12:02:15.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.621+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b640b0880 con 0x7f6b6413bab0 2023-10-18T12:02:15.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.622+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b640690e0 con 0x7f6b6413bab0 2023-10-18T12:02:15.632 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.623+0000 7f076480d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T12:02:15.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.623+0000 7f6b4bfff640 1 -- 172.21.15.125:0/3383217831 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 151) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f6b5c00ac10 con 0x7f6b6413bab0 2023-10-18T12:02:15.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.623+0000 7f6b4bfff640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f6b5006af90 0x7f6b5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:15.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.623+0000 7f6b637fe640 1 -- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f6b5006af90 msgr2=0x7f6b5006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:15.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.623+0000 7f6b4bfff640 1 -- 172.21.15.125:0/3383217831 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7f6b5c0899c0 con 0x7f6b6413bab0 2023-10-18T12:02:15.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.623+0000 7f6b637fe640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f6b5006af90 0x7f6b5006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:15.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.628+0000 7f6b4bfff640 1 -- 172.21.15.125:0/3383217831 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b5c054640 con 0x7f6b6413bab0 2023-10-18T12:02:15.690 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.688+0000 7f592c411040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:02:15.732 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.729+0000 7f592c411040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:02:15.766 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.764+0000 7f076480d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T12:02:15.809 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.807+0000 7f076480d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T12:02:15.826 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.824+0000 7f6b637fe640 1 -- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f6b5006af90 msgr2=0x7f6b5006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:15.827 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.824+0000 7f6b637fe640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f6b5006af90 0x7f6b5006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:02:15.835 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:15.833+0000 7f592c411040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:02:15.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.836+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b640c8f30 con 0x7f6b6413bab0 2023-10-18T12:02:15.839 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.837+0000 7f6b4bfff640 1 -- 172.21.15.125:0/3383217831 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f6b5c057c60 con 0x7f6b6413bab0 2023-10-18T12:02:15.840 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:15.840 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":112,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3962342896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3962342896},{"type":"v1","addr":"172.21.15.139:6837","nonce":3962342896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11413,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1953593032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1953593032},{"type":"v1","addr":"172.21.15.139:6836","nonce":1953593032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":13566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1230654530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1230654530},{"type":"v1","addr":"172.21.15.125:6834","nonce":1230654530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:59.406238+0000","modified":"2023-10-18T12:02:01.736375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13569":{"gid":13569,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1922177652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1922177652},{"type":"v1","addr":"172.21.15.125:6835","nonce":1922177652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-18T12:02:15.840 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 112 2023-10-18T12:02:15.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f6b5006af90 msgr2=0x7f6b5006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:15.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f6b5006af90 0x7f6b5006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.842 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413bab0 msgr2=0x7f6b64134db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:15.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413bab0 0x7f6b64134db0 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c7a0 tx=0x7f6b5c00cc60 comp rx=0 tx=0).stop 2023-10-18T12:02:15.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 shutdown_connections 2023-10-18T12:02:15.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f6b5006af90 0x7f6b5006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b640cb980 0x7f6b641f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.843 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b6413bab0 0x7f6b64134db0 unknown :-1 s=CLOSED pgs=3078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 --2- 172.21.15.125:0/3383217831 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b640c9750 0x7f6b640b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 >> 172.21.15.125:0/3383217831 conn(0x7f6b64069930 msgr2=0x7f6b6413d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:15.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 shutdown_connections 2023-10-18T12:02:15.844 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.839+0000 7f6b6a261640 1 -- 172.21.15.125:0/3383217831 wait complete. 2023-10-18T12:02:15.853 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:02:15.919 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:15.917+0000 7f076480d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T12:02:15.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.994+0000 7fc0022f2640 1 -- 172.21.15.125:0/1541405234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbffc0ceb40 msgr2=0x7fbffc0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:15.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.994+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1541405234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbffc0ceb40 0x7fbffc0b0650 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005fd0 tx=0x7fbff402c240 comp rx=0 tx=0).stop 2023-10-18T12:02:15.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.994+0000 7fc0022f2640 1 -- 172.21.15.125:0/1541405234 shutdown_connections 2023-10-18T12:02:15.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.994+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1541405234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbffc0ceb40 0x7fbffc0b0650 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.994+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1541405234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbffc15b2a0 0x7fbffc0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:15.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.994+0000 7fc0022f2640 1 -- 172.21.15.125:0/1541405234 >> 172.21.15.125:0/1541405234 conn(0x7fbffc0d0d60 msgr2=0x7fbffc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:15.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.994+0000 7fc0022f2640 1 -- 172.21.15.125:0/1541405234 shutdown_connections 2023-10-18T12:02:15.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.994+0000 7fc0022f2640 1 -- 172.21.15.125:0/1541405234 wait complete. 2023-10-18T12:02:16.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.995+0000 7fc0022f2640 1 Processor -- start 2023-10-18T12:02:16.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.995+0000 7fc0022f2640 1 -- start start 2023-10-18T12:02:16.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fc0022f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbffc0ceb40 0x7fbffc1f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.001 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fc0022f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbffc0cf340 0x7fbffc15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fc0022f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbffc15b2a0 0x7fbffc15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fc0022f2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1f24d0 con 0x7fbffc0ceb40 2023-10-18T12:02:16.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fc0022f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1f2720 con 0x7fbffc15b2a0 2023-10-18T12:02:16.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fc0022f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1f2a00 con 0x7fbffc0cf340 2023-10-18T12:02:16.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fbffbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbffc0ceb40 0x7fbffc1f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fbffbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbffc0ceb40 0x7fbffc1f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48788/0 (socket says 172.21.15.125:48788) 2023-10-18T12:02:16.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fbffbfff640 1 -- 172.21.15.125:0/1485315606 learned_addr learned my addr 172.21.15.125:0/1485315606 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:16.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fc00088f640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbffc15b2a0 0x7fbffc15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.996+0000 7fbffb7fe640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbffc0cf340 0x7fbffc15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbffbfff640 1 -- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbffc0cf340 msgr2=0x7fbffc15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbffbfff640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbffc0cf340 0x7fbffc15d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbffbfff640 1 -- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbffc15b2a0 msgr2=0x7fbffc15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbffbfff640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbffc15b2a0 0x7fbffc15dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbffbfff640 1 -- 172.21.15.125:0/1485315606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec009490 con 0x7fbffc0ceb40 2023-10-18T12:02:16.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbffbfff640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbffc0ceb40 0x7fbffc1f1b80 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7fbff000c4b0 tx=0x7fbff000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:16.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbff8ff9640 1 -- 172.21.15.125:0/1485315606 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0005480 con 0x7fbffc0ceb40 2023-10-18T12:02:16.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbff8ff9640 1 -- 172.21.15.125:0/1485315606 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbff00036e0 con 0x7fbffc0ceb40 2023-10-18T12:02:16.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff4009900 con 0x7fbffc0ceb40 2023-10-18T12:02:16.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fbff8ff9640 1 -- 172.21.15.125:0/1485315606 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0003b30 con 0x7fbffc0ceb40 2023-10-18T12:02:16.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.997+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc1f2fc0 con 0x7fbffc0ceb40 2023-10-18T12:02:16.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.998+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc0690e0 con 0x7fbffc0ceb40 2023-10-18T12:02:16.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.999+0000 7fbff8ff9640 1 -- 172.21.15.125:0/1485315606 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 151) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fbff0003c90 con 0x7fbffc0ceb40 2023-10-18T12:02:16.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.999+0000 7fbff8ff9640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fbfe806af60 0x7fbfe806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:15.999+0000 7fbffb7fe640 1 -- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fbfe806af60 msgr2=0x7fbfe806d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:16.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.000+0000 7fbff8ff9640 1 -- 172.21.15.125:0/1485315606 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7fbff0089230 con 0x7fbffc0ceb40 2023-10-18T12:02:16.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.000+0000 7fbffb7fe640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fbfe806af60 0x7fbfe806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:16.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.004+0000 7fbff8ff9640 1 -- 172.21.15.125:0/1485315606 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff0053f40 con 0x7fbffc0ceb40 2023-10-18T12:02:16.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.180+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbffc0cb980 con 0x7fbffc0ceb40 2023-10-18T12:02:16.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.181+0000 7fbff8ff9640 1 -- 172.21.15.125:0/1485315606 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v264) v1 ==== 74+0+17554 (secure 0 0 0) 0x7fbff0057560 con 0x7fbffc0ceb40 2023-10-18T12:02:16.187 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:16.187 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":264,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:02:11.466373+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:57.717906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:58.385232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2488548050":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2767408321":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6832/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1739931468":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2511136316":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6833/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2813907753":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1094096408":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1205310989":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1383817777":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2446560793":"2023-10-18T13:02:05.851547+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:02:16.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fbfe806af60 msgr2=0x7fbfe806d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:16.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fbfe806af60 0x7fbfe806d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.188 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbffc0ceb40 msgr2=0x7fbffc1f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbffc0ceb40 0x7fbffc1f1b80 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7fbff000c4b0 tx=0x7fbff000c970 comp rx=0 tx=0).stop 2023-10-18T12:02:16.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 shutdown_connections 2023-10-18T12:02:16.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7fbfe806af60 0x7fbfe806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbffc15b2a0 0x7fbffc15dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbffc0cf340 0x7fbffc15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 --2- 172.21.15.125:0/1485315606 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbffc0ceb40 0x7fbffc1f1b80 unknown :-1 s=CLOSED pgs=3079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 >> 172.21.15.125:0/1485315606 conn(0x7fbffc0d0d60 msgr2=0x7fbffc06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:16.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 shutdown_connections 2023-10-18T12:02:16.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.183+0000 7fc0022f2640 1 -- 172.21.15.125:0/1485315606 wait complete. 2023-10-18T12:02:16.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T10:24:52.607581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T12:01:57.717906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '261', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T12:01:58.385232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '262', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T12:02:16.198 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T12:02:16.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.344+0000 7f21b0f16640 1 -- 172.21.15.125:0/1291556494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21ac0ce240 msgr2=0x7f21ac06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.344+0000 7f21b0f16640 1 --2- 172.21.15.125:0/1291556494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21ac0ce240 0x7f21ac06d260 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2194009830 tx=0x7f219402bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.344+0000 7f21b0f16640 1 -- 172.21.15.125:0/1291556494 shutdown_connections 2023-10-18T12:02:16.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.344+0000 7f21b0f16640 1 --2- 172.21.15.125:0/1291556494 >> v2:172.21.15.139:3300/0 conn(0x7f21ac15ccf0 0x7f21ac15d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.344+0000 7f21b0f16640 1 --2- 172.21.15.125:0/1291556494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21ac0ce240 0x7f21ac06d260 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.344+0000 7f21b0f16640 1 -- 172.21.15.125:0/1291556494 >> 172.21.15.125:0/1291556494 conn(0x7f21ac0b0670 msgr2=0x7f21ac0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:16.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.344+0000 7f21b0f16640 1 -- 172.21.15.125:0/1291556494 shutdown_connections 2023-10-18T12:02:16.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.344+0000 7f21b0f16640 1 -- 172.21.15.125:0/1291556494 wait complete. 2023-10-18T12:02:16.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.345+0000 7f21b0f16640 1 Processor -- start 2023-10-18T12:02:16.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.345+0000 7f21b0f16640 1 -- start start 2023-10-18T12:02:16.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21b0f16640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21ac0ce240 0x7f21ac15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21b0f16640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21ac159060 0x7f21ac1f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21b0f16640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21ac15ccf0 0x7f21ac0b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21b0f16640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac15c6d0 con 0x7f21ac159060 2023-10-18T12:02:16.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21b0f16640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac15a220 con 0x7f21ac15ccf0 2023-10-18T12:02:16.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21b0f16640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac1f15b0 con 0x7f21ac0ce240 2023-10-18T12:02:16.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21ac159060 0x7f21ac1f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21aa59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21ac0ce240 0x7f21ac15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21ac159060 0x7f21ac1f1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48794/0 (socket says 172.21.15.125:48794) 2023-10-18T12:02:16.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 -- 172.21.15.125:0/581424445 learned_addr learned my addr 172.21.15.125:0/581424445 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:16.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21aad9d640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21ac15ccf0 0x7f21ac0b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 -- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21ac0ce240 msgr2=0x7f21ac15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21ac0ce240 0x7f21ac15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 -- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21ac15ccf0 msgr2=0x7f21ac0b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21ac15ccf0 0x7f21ac0b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 -- 172.21.15.125:0/581424445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4009840 con 0x7f21ac159060 2023-10-18T12:02:16.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.346+0000 7f21a3fff640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21ac159060 0x7f21ac1f1070 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 crypto rx=0x7f2194009950 tx=0x7f2194003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:16.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.347+0000 7f21a2ffd640 1 -- 172.21.15.125:0/581424445 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194003e20 con 0x7f21ac159060 2023-10-18T12:02:16.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.347+0000 7f21a2ffd640 1 -- 172.21.15.125:0/581424445 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2194034440 con 0x7f21ac159060 2023-10-18T12:02:16.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.347+0000 7f21a2ffd640 1 -- 172.21.15.125:0/581424445 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194034850 con 0x7f21ac159060 2023-10-18T12:02:16.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.347+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2194009490 con 0x7f21ac159060 2023-10-18T12:02:16.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.347+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac06bfb0 con 0x7f21ac159060 2023-10-18T12:02:16.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.348+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac0675f0 con 0x7f21ac159060 2023-10-18T12:02:16.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.348+0000 7f21a2ffd640 1 -- 172.21.15.125:0/581424445 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 151) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f21940349b0 con 0x7f21ac159060 2023-10-18T12:02:16.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.348+0000 7f21a2ffd640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f218c06af60 0x7f218c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.349+0000 7f21a2ffd640 1 -- 172.21.15.125:0/581424445 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7f21940ac0e0 con 0x7f21ac159060 2023-10-18T12:02:16.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.349+0000 7f21aa59c640 1 -- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f218c06af60 msgr2=0x7f218c06d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:16.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.349+0000 7f21aa59c640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f218c06af60 0x7f218c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:16.358 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.353+0000 7f21a2ffd640 1 -- 172.21.15.125:0/581424445 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2194076d30 con 0x7f21ac159060 2023-10-18T12:02:16.459 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.457+0000 7f592c411040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:02:16.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.533+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21ac0b4aa0 con 0x7f21ac159060 2023-10-18T12:02:16.536 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.533+0000 7f21a2ffd640 1 -- 172.21.15.125:0/581424445 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v264) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21940390a0 con 0x7f21ac159060 2023-10-18T12:02:16.537 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 1 2023-10-18T12:02:16.538 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f218c06af60 msgr2=0x7f218c06d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:16.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f218c06af60 0x7f218c06d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21ac159060 msgr2=0x7f21ac1f1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21ac159060 0x7f21ac1f1070 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 crypto rx=0x7f2194009950 tx=0x7f2194003820 comp rx=0 tx=0).stop 2023-10-18T12:02:16.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 shutdown_connections 2023-10-18T12:02:16.539 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f218c06af60 0x7f218c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21ac15ccf0 0x7f21ac0b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21ac159060 0x7f21ac1f1070 unknown :-1 s=CLOSED pgs=3080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 --2- 172.21.15.125:0/581424445 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21ac0ce240 0x7f21ac15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 >> 172.21.15.125:0/581424445 conn(0x7f21ac0b0670 msgr2=0x7f21ac0ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:16.540 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 shutdown_connections 2023-10-18T12:02:16.541 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.535+0000 7f21b0f16640 1 -- 172.21.15.125:0/581424445 wait complete. 2023-10-18T12:02:16.549 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T12:02:16.585 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:16.583+0000 7f076480d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T12:02:16.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.653+0000 7f3654a7c640 1 -- 172.21.15.125:0/1366503507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36500cdb30 msgr2=0x7f36500cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.653+0000 7f3654a7c640 1 --2- 172.21.15.125:0/1366503507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36500cdb30 0x7f36500cdfb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f3638009830 tx=0x7f363802bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.653+0000 7f3654a7c640 1 -- 172.21.15.125:0/1366503507 shutdown_connections 2023-10-18T12:02:16.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.653+0000 7f3654a7c640 1 --2- 172.21.15.125:0/1366503507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36500cdb30 0x7f36500cdfb0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.653+0000 7f3654a7c640 1 --2- 172.21.15.125:0/1366503507 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3650159060 0x7f365006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.653+0000 7f3654a7c640 1 -- 172.21.15.125:0/1366503507 >> 172.21.15.125:0/1366503507 conn(0x7f36500b0670 msgr2=0x7f36500b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:16.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.653+0000 7f3654a7c640 1 -- 172.21.15.125:0/1366503507 shutdown_connections 2023-10-18T12:02:16.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.653+0000 7f3654a7c640 1 -- 172.21.15.125:0/1366503507 wait complete. 2023-10-18T12:02:16.660 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.655+0000 7f3654a7c640 1 Processor -- start 2023-10-18T12:02:16.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.655+0000 7f3654a7c640 1 -- start start 2023-10-18T12:02:16.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.655+0000 7f3654a7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36500cdb30 0x7f3650154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3654a7c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3650159060 0x7f36500c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.661 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3654a7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f365015ccf0 0x7f3650156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3654a7c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36500c80a0 con 0x7f3650159060 2023-10-18T12:02:16.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3654a7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36500c8380 con 0x7f365015ccf0 2023-10-18T12:02:16.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3654a7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36500c8660 con 0x7f36500cdb30 2023-10-18T12:02:16.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3647fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3650159060 0x7f36500c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.662 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364e59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36500cdb30 0x7f3650154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3647fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3650159060 0x7f36500c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48806/0 (socket says 172.21.15.125:48806) 2023-10-18T12:02:16.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364e59c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36500cdb30 0x7f3650154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.125:47860/0 (socket says 172.21.15.125:47860) 2023-10-18T12:02:16.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3647fff640 1 -- 172.21.15.125:0/855000404 learned_addr learned my addr 172.21.15.125:0/855000404 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:16.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364ed9d640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f365015ccf0 0x7f3650156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:16.663 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364e59c640 1 -- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f365015ccf0 msgr2=0x7f3650156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364e59c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f365015ccf0 0x7f3650156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364e59c640 1 -- 172.21.15.125:0/855000404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3650159060 msgr2=0x7f36500c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364e59c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3650159060 0x7f36500c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364e59c640 1 -- 172.21.15.125:0/855000404 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009490 con 0x7f36500cdb30 2023-10-18T12:02:16.664 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364ed9d640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f365015ccf0 0x7f3650156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T12:02:16.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f364e59c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36500cdb30 0x7f3650154c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f364000e9f0 tx=0x7f364000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:16.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3646ffd640 1 -- 172.21.15.125:0/855000404 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000ccc0 con 0x7f36500cdb30 2023-10-18T12:02:16.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3646ffd640 1 -- 172.21.15.125:0/855000404 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3640010430 con 0x7f36500cdb30 2023-10-18T12:02:16.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36500c8940 con 0x7f36500cdb30 2023-10-18T12:02:16.665 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3646ffd640 1 -- 172.21.15.125:0/855000404 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640010820 con 0x7f36500cdb30 2023-10-18T12:02:16.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.656+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3650155180 con 0x7f36500cdb30 2023-10-18T12:02:16.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.658+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36500690e0 con 0x7f36500cdb30 2023-10-18T12:02:16.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.658+0000 7f3646ffd640 1 -- 172.21.15.125:0/855000404 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 151) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3640010980 con 0x7f36500cdb30 2023-10-18T12:02:16.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.658+0000 7f3646ffd640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f363006af60 0x7f363006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:16.666 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.658+0000 7f3647fff640 1 -- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f363006af60 msgr2=0x7f363006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:16.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.658+0000 7f3646ffd640 1 -- 172.21.15.125:0/855000404 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 6558+0+0 (secure 0 0 0) 0x7f3640014070 con 0x7f36500cdb30 2023-10-18T12:02:16.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.658+0000 7f3647fff640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f363006af60 0x7f363006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-18T12:02:16.667 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.663+0000 7f3646ffd640 1 -- 172.21.15.125:0/855000404 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3640051ca0 con 0x7f36500cdb30 2023-10-18T12:02:16.723 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.721+0000 7f592c411040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:02:16.771 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.769+0000 7f592c411040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:02:16.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.859+0000 7f3647fff640 1 -- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f363006af60 msgr2=0x7f363006d420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.139:6832/1808248507 2023-10-18T12:02:16.862 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.859+0000 7f3647fff640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f363006af60 0x7f363006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-18T12:02:16.870 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.868+0000 7f592c411040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:02:16.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.869+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36500675f0 con 0x7f36500cdb30 2023-10-18T12:02:16.872 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.870+0000 7f3646ffd640 1 -- 172.21.15.125:0/855000404 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v264) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3640055250 con 0x7f36500cdb30 2023-10-18T12:02:16.873 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 32 2023-10-18T12:02:16.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f363006af60 msgr2=0x7f363006d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T12:02:16.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f363006af60 0x7f363006d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36500cdb30 msgr2=0x7f3650154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:16.875 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36500cdb30 0x7f3650154c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f364000e9f0 tx=0x7f364000eeb0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 shutdown_connections 2023-10-18T12:02:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:6832/1808248507,v1:172.21.15.139:6833/1808248507] conn(0x7f363006af60 0x7f363006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f365015ccf0 0x7f3650156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3650159060 0x7f36500c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.876 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 --2- 172.21.15.125:0/855000404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36500cdb30 0x7f3650154c40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:16.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 >> 172.21.15.125:0/855000404 conn(0x7f36500b0670 msgr2=0x7f365015d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:16.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 shutdown_connections 2023-10-18T12:02:16.877 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.872+0000 7f3654a7c640 1 -- 172.21.15.125:0/855000404 wait complete. 2023-10-18T12:02:16.885 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T12:02:16.890 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:16.887+0000 7f076480d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T12:02:16.945 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:16.943+0000 7f076480d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T12:02:16.949 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.947+0000 7f58b1e9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.950 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.947+0000 7f58b1e9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.950 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.947+0000 7f58b1e9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.950 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.947+0000 7f58b1e9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.950 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.947+0000 7f58b1e9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.951 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.948+0000 7f58aee99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.951 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.948+0000 7f58aee99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.951 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.948+0000 7f58aee99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.952 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.948+0000 7f58aee99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:16.952 INFO:tasks.ceph.mgr.x.smithi139.stderr:2023-10-18T12:02:16.948+0000 7f58aee99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T12:02:17.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.998+0000 7f418a5d0640 1 -- 172.21.15.125:0/3614871014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f416c0050e0 msgr2=0x7f416c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.998+0000 7f418a5d0640 1 --2- 172.21.15.125:0/3614871014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f416c0050e0 0x7f416c00b980 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 crypto rx=0x7f417c009ed0 tx=0x7f416c0057f0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.998+0000 7f418a5d0640 1 -- 172.21.15.125:0/3614871014 shutdown_connections 2023-10-18T12:02:17.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.998+0000 7f418a5d0640 1 --2- 172.21.15.125:0/3614871014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f416c0050e0 0x7f416c00b980 unknown :-1 s=CLOSED pgs=3082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.998+0000 7f418a5d0640 1 -- 172.21.15.125:0/3614871014 >> 172.21.15.125:0/3614871014 conn(0x7f41840d0d60 msgr2=0x7f41840d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:17.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.998+0000 7f418a5d0640 1 -- 172.21.15.125:0/3614871014 shutdown_connections 2023-10-18T12:02:17.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:16.998+0000 7f418a5d0640 1 -- 172.21.15.125:0/3614871014 wait complete. 2023-10-18T12:02:17.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.000+0000 7f418a5d0640 1 Processor -- start 2023-10-18T12:02:17.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.000+0000 7f418a5d0640 1 -- start start 2023-10-18T12:02:17.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f418a5d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416c0050e0 0x7f41841f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f418a5d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840c98a0 0x7f41841e5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f418a5d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41840c9d20 0x7f41841e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f418a5d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418415d090 con 0x7f41840c9d20 2023-10-18T12:02:17.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f418a5d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418415c480 con 0x7f416c0050e0 2023-10-18T12:02:17.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f418a5d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418415c760 con 0x7f41840c98a0 2023-10-18T12:02:17.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f4188b6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41840c9d20 0x7f41841e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f4188b6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41840c9d20 0x7f41841e5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48824/0 (socket says 172.21.15.125:48824) 2023-10-18T12:02:17.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f4188b6d640 1 -- 172.21.15.125:0/4161220336 learned_addr learned my addr 172.21.15.125:0/4161220336 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:17.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f4183fff640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416c0050e0 0x7f41841f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.001+0000 7f41837fe640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840c98a0 0x7f41841e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4188b6d640 1 -- 172.21.15.125:0/4161220336 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840c98a0 msgr2=0x7f41841e5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4188b6d640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840c98a0 0x7f41841e5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4188b6d640 1 -- 172.21.15.125:0/4161220336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416c0050e0 msgr2=0x7f41841f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4188b6d640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416c0050e0 0x7f41841f0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4188b6d640 1 -- 172.21.15.125:0/4161220336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418415ca40 con 0x7f41840c9d20 2023-10-18T12:02:17.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4188b6d640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41840c9d20 0x7f41841e5a60 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f417c009e20 tx=0x7f417c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:17.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4180ff9640 1 -- 172.21.15.125:0/4161220336 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c03d070 con 0x7f41840c9d20 2023-10-18T12:02:17.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4180ff9640 1 -- 172.21.15.125:0/4161220336 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f417c004230 con 0x7f41840c9d20 2023-10-18T12:02:17.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418415cd20 con 0x7f41840c9d20 2023-10-18T12:02:17.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f4180ff9640 1 -- 172.21.15.125:0/4161220336 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0045c0 con 0x7f41840c9d20 2023-10-18T12:02:17.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.002+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418415c0c0 con 0x7f41840c9d20 2023-10-18T12:02:17.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.003+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418415c8a0 con 0x7f41840c9d20 2023-10-18T12:02:17.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.004+0000 7f4180ff9640 1 -- 172.21.15.125:0/4161220336 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 152) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f417c004720 con 0x7f41840c9d20 2023-10-18T12:02:17.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.004+0000 7f4180ff9640 1 -- 172.21.15.125:0/4161220336 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6902+0+0 (secure 0 0 0) 0x7f417c0ab640 con 0x7f41840c9d20 2023-10-18T12:02:17.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.008+0000 7f4180ff9640 1 -- 172.21.15.125:0/4161220336 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f418415c8a0 con 0x7f41840c9d20 2023-10-18T12:02:17.054 INFO:tasks.ceph.mgr.y.smithi125.stderr:2023-10-18T12:02:17.051+0000 7f076480d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T12:02:17.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.176+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f418415c8a0 con 0x7f41840c9d20 2023-10-18T12:02:17.179 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.177+0000 7f4180ff9640 1 -- 172.21.15.125:0/4161220336 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v265) v1 ==== 100+0+11 (secure 0 0 0) 0x7f418415c8a0 con 0x7f41840c9d20 2023-10-18T12:02:17.180 INFO:teuthology.orchestra.run.smithi125.stdout:pg_num: 64 2023-10-18T12:02:17.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.178+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41840c9d20 msgr2=0x7f41841e5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.179+0000 7f418a5d0640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41840c9d20 0x7f41841e5a60 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f417c009e20 tx=0x7f417c003ba0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.179+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 shutdown_connections 2023-10-18T12:02:17.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.179+0000 7f418a5d0640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41840c9d20 0x7f41841e5a60 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.179+0000 7f418a5d0640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840c98a0 0x7f41841e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.179+0000 7f418a5d0640 1 --2- 172.21.15.125:0/4161220336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416c0050e0 0x7f41841f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.179+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 >> 172.21.15.125:0/4161220336 conn(0x7f41840d0d60 msgr2=0x7f41840cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:17.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.179+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 shutdown_connections 2023-10-18T12:02:17.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.179+0000 7f418a5d0640 1 -- 172.21.15.125:0/4161220336 wait complete. 2023-10-18T12:02:17.192 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:02:17.294 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.291+0000 7ff96514c640 1 -- 172.21.15.125:0/3720596003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 msgr2=0x7ff960158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.291+0000 7ff96514c640 1 --2- 172.21.15.125:0/3720596003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 0x7ff960158300 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 crypto rx=0x7ff950005fd0 tx=0x7ff95002be10 comp rx=0 tx=0).stop 2023-10-18T12:02:17.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.291+0000 7ff96514c640 1 -- 172.21.15.125:0/3720596003 shutdown_connections 2023-10-18T12:02:17.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.291+0000 7ff96514c640 1 --2- 172.21.15.125:0/3720596003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9600b0530 0x7ff9600b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.295 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.291+0000 7ff96514c640 1 --2- 172.21.15.125:0/3720596003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 0x7ff960158300 unknown :-1 s=CLOSED pgs=3084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.291+0000 7ff96514c640 1 -- 172.21.15.125:0/3720596003 >> 172.21.15.125:0/3720596003 conn(0x7ff960069930 msgr2=0x7ff960069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:17.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.292+0000 7ff96514c640 1 -- 172.21.15.125:0/3720596003 shutdown_connections 2023-10-18T12:02:17.296 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.292+0000 7ff96514c640 1 -- 172.21.15.125:0/3720596003 wait complete. 2023-10-18T12:02:17.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.293+0000 7ff96514c640 1 Processor -- start 2023-10-18T12:02:17.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.293+0000 7ff96514c640 1 -- start start 2023-10-18T12:02:17.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.293+0000 7ff96514c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9600afa20 0x7ff9600abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.293+0000 7ff96514c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9600b0530 0x7ff9600adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.293+0000 7ff96514c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 0x7ff9600ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.293+0000 7ff96514c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96013e990 con 0x7ff9600cbd20 2023-10-18T12:02:17.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.293+0000 7ff96514c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff960152da0 con 0x7ff9600b0530 2023-10-18T12:02:17.300 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.293+0000 7ff96514c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96013dff0 con 0x7ff9600afa20 2023-10-18T12:02:17.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 0x7ff9600ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 0x7ff9600ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:48840/0 (socket says 172.21.15.125:48840) 2023-10-18T12:02:17.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 -- 172.21.15.125:0/1105272598 learned_addr learned my addr 172.21.15.125:0/1105272598 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:17.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95ed9d640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9600afa20 0x7ff9600abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff957fff640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9600b0530 0x7ff9600adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 -- 172.21.15.125:0/1105272598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9600afa20 msgr2=0x7ff9600abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9600afa20 0x7ff9600abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 -- 172.21.15.125:0/1105272598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9600b0530 msgr2=0x7ff9600adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9600b0530 0x7ff9600adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 -- 172.21.15.125:0/1105272598 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff950009560 con 0x7ff9600cbd20 2023-10-18T12:02:17.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff95f59e640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 0x7ff9600ae060 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 crypto rx=0x7ff95800eaa0 tx=0x7ff95800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:17.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff9577fe640 1 -- 172.21.15.125:0/1105272598 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95800cdb0 con 0x7ff9600cbd20 2023-10-18T12:02:17.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff9577fe640 1 -- 172.21.15.125:0/1105272598 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff958004b20 con 0x7ff9600cbd20 2023-10-18T12:02:17.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff9577fe640 1 -- 172.21.15.125:0/1105272598 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958005020 con 0x7ff9600cbd20 2023-10-18T12:02:17.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96013e2d0 con 0x7ff9600cbd20 2023-10-18T12:02:17.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.294+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff960153790 con 0x7ff9600cbd20 2023-10-18T12:02:17.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.295+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96013ead0 con 0x7ff9600cbd20 2023-10-18T12:02:17.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.296+0000 7ff9577fe640 1 -- 172.21.15.125:0/1105272598 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 152) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff958005180 con 0x7ff9600cbd20 2023-10-18T12:02:17.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.296+0000 7ff9577fe640 1 -- 172.21.15.125:0/1105272598 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6902+0+0 (secure 0 0 0) 0x7ff958089f40 con 0x7ff9600cbd20 2023-10-18T12:02:17.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.301+0000 7ff9577fe640 1 -- 172.21.15.125:0/1105272598 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9580bcca0 con 0x7ff9600cbd20 2023-10-18T12:02:17.506 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.504+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9600690e0 con 0x7ff9600cbd20 2023-10-18T12:02:17.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.504+0000 7ff9577fe640 1 -- 172.21.15.125:0/1105272598 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7ff95801f070 con 0x7ff9600cbd20 2023-10-18T12:02:17.507 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:17.507 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":112,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3962342896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3962342896},{"type":"v1","addr":"172.21.15.139:6837","nonce":3962342896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11413,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1953593032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1953593032},{"type":"v1","addr":"172.21.15.139:6836","nonce":1953593032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":13566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1230654530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1230654530},{"type":"v1","addr":"172.21.15.125:6834","nonce":1230654530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:59.406238+0000","modified":"2023-10-18T12:02:01.736375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13569":{"gid":13569,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1922177652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1922177652},{"type":"v1","addr":"172.21.15.125:6835","nonce":1922177652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-18T12:02:17.508 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 112 2023-10-18T12:02:17.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 msgr2=0x7ff9600ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 0x7ff9600ae060 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 crypto rx=0x7ff95800eaa0 tx=0x7ff95800ef60 comp rx=0 tx=0).stop 2023-10-18T12:02:17.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 shutdown_connections 2023-10-18T12:02:17.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9600b0530 0x7ff9600adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9600cbd20 0x7ff9600ae060 unknown :-1 s=CLOSED pgs=3085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 --2- 172.21.15.125:0/1105272598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9600afa20 0x7ff9600abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 >> 172.21.15.125:0/1105272598 conn(0x7ff960069930 msgr2=0x7ff9600ac740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:17.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 shutdown_connections 2023-10-18T12:02:17.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.506+0000 7ff96514c640 1 -- 172.21.15.125:0/1105272598 wait complete. 2023-10-18T12:02:17.519 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:02:17.622 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.619+0000 7ff394a51640 1 -- 172.21.15.125:0/2187921408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 msgr2=0x7ff3900705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.619+0000 7ff394a51640 1 --2- 172.21.15.125:0/2187921408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 0x7ff3900705a0 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7ff3900cc7e0 tx=0x7ff38402d420 comp rx=0 tx=0).stop 2023-10-18T12:02:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.619+0000 7ff394a51640 1 -- 172.21.15.125:0/2187921408 shutdown_connections 2023-10-18T12:02:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.619+0000 7ff394a51640 1 --2- 172.21.15.125:0/2187921408 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 0x7ff3900705a0 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.619+0000 7ff394a51640 1 --2- 172.21.15.125:0/2187921408 >> v2:172.21.15.139:3300/0 conn(0x7ff3900cbd20 0x7ff390158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.619+0000 7ff394a51640 1 -- 172.21.15.125:0/2187921408 >> 172.21.15.125:0/2187921408 conn(0x7ff390069930 msgr2=0x7ff390069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:17.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.619+0000 7ff394a51640 1 -- 172.21.15.125:0/2187921408 shutdown_connections 2023-10-18T12:02:17.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.619+0000 7ff394a51640 1 -- 172.21.15.125:0/2187921408 wait complete. 2023-10-18T12:02:17.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff394a51640 1 Processor -- start 2023-10-18T12:02:17.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff394a51640 1 -- start start 2023-10-18T12:02:17.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff394a51640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 0x7ff39014b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff394a51640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3900b0530 0x7ff3900aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff394a51640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3900cbd20 0x7ff3901e3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff394a51640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39014b010 con 0x7ff3900afa20 2023-10-18T12:02:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff394a51640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39014a400 con 0x7ff3900b0530 2023-10-18T12:02:17.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff394a51640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39014a6e0 con 0x7ff3900cbd20 2023-10-18T12:02:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff38e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 0x7ff39014b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff38e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 0x7ff39014b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40834/0 (socket says 172.21.15.125:40834) 2023-10-18T12:02:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.621+0000 7ff38e59c640 1 -- 172.21.15.125:0/2351861639 learned_addr learned my addr 172.21.15.125:0/2351861639 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:17.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff38dd9b640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3900b0530 0x7ff3900aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff38ed9d640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3900cbd20 0x7ff3901e3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff38e59c640 1 -- 172.21.15.125:0/2351861639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3900cbd20 msgr2=0x7ff3901e3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff38e59c640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3900cbd20 0x7ff3901e3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff38e59c640 1 -- 172.21.15.125:0/2351861639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3900b0530 msgr2=0x7ff3900aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff38e59c640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3900b0530 0x7ff3900aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff38e59c640 1 -- 172.21.15.125:0/2351861639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38400a950 con 0x7ff3900afa20 2023-10-18T12:02:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff38e59c640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 0x7ff39014b4a0 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7ff38800d740 tx=0x7ff38800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff37effd640 1 -- 172.21.15.125:0/2351861639 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388009c30 con 0x7ff3900afa20 2023-10-18T12:02:17.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff37effd640 1 -- 172.21.15.125:0/2351861639 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff388009e80 con 0x7ff3900afa20 2023-10-18T12:02:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39014a9c0 con 0x7ff3900afa20 2023-10-18T12:02:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff37effd640 1 -- 172.21.15.125:0/2351861639 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880106c0 con 0x7ff3900afa20 2023-10-18T12:02:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.622+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39014b9e0 con 0x7ff3900afa20 2023-10-18T12:02:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.623+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3900690e0 con 0x7ff3900afa20 2023-10-18T12:02:17.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.624+0000 7ff37effd640 1 -- 172.21.15.125:0/2351861639 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 152) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff388010820 con 0x7ff3900afa20 2023-10-18T12:02:17.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.624+0000 7ff37effd640 1 -- 172.21.15.125:0/2351861639 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6902+0+0 (secure 0 0 0) 0x7ff388089b10 con 0x7ff3900afa20 2023-10-18T12:02:17.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.628+0000 7ff37effd640 1 -- 172.21.15.125:0/2351861639 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff388054ad0 con 0x7ff3900afa20 2023-10-18T12:02:17.796 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.793+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3900675f0 con 0x7ff3900afa20 2023-10-18T12:02:17.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.795+0000 7ff37effd640 1 -- 172.21.15.125:0/2351861639 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) v1 ==== 74+0+18063 (secure 0 0 0) 0x7ff388021090 con 0x7ff3900afa20 2023-10-18T12:02:17.798 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:17.798 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":265,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:02:16.882561+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:57.717906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:58.385232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2140935837":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/246588314":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/1154764603":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/3832700950":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/2488548050":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2767408321":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6832/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1739931468":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/4281362145":"2023-10-19T12:02:16.882516+0000","172.21.15.139:6833/1808248507":"2023-10-19T12:02:16.882516+0000","172.21.15.139:6832/1808248507":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/2511136316":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6833/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2813907753":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1094096408":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1599845747":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/1205310989":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1383817777":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2446560793":"2023-10-18T13:02:05.851547+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:02:17.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.796+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 msgr2=0x7ff39014b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.797+0000 7ff394a51640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 0x7ff39014b4a0 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7ff38800d740 tx=0x7ff38800dc00 comp rx=0 tx=0).stop 2023-10-18T12:02:17.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.797+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 shutdown_connections 2023-10-18T12:02:17.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.797+0000 7ff394a51640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3900b0530 0x7ff3900aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.797+0000 7ff394a51640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3900cbd20 0x7ff3901e3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.797+0000 7ff394a51640 1 --2- 172.21.15.125:0/2351861639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3900afa20 0x7ff39014b4a0 unknown :-1 s=CLOSED pgs=3088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.797+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 >> 172.21.15.125:0/2351861639 conn(0x7ff390069930 msgr2=0x7ff390069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:17.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.797+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 shutdown_connections 2023-10-18T12:02:17.801 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.797+0000 7ff394a51640 1 -- 172.21.15.125:0/2351861639 wait complete. 2023-10-18T12:02:17.810 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T12:02:17.811 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:02:17.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.914+0000 7f113406b640 1 -- 172.21.15.125:0/3033949483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 msgr2=0x7f112c0c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.914+0000 7f113406b640 1 --2- 172.21.15.125:0/3033949483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 0x7f112c0c96b0 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7f112c0ca460 tx=0x7f111c02bd70 comp rx=0 tx=0).stop 2023-10-18T12:02:17.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.914+0000 7f113406b640 1 -- 172.21.15.125:0/3033949483 shutdown_connections 2023-10-18T12:02:17.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.914+0000 7f113406b640 1 --2- 172.21.15.125:0/3033949483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 0x7f112c0c96b0 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.914+0000 7f113406b640 1 --2- 172.21.15.125:0/3033949483 >> v2:172.21.15.139:3300/0 conn(0x7f112c0b1920 0x7f112c06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.914+0000 7f113406b640 1 -- 172.21.15.125:0/3033949483 >> 172.21.15.125:0/3033949483 conn(0x7f112c069930 msgr2=0x7f112c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:17.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.914+0000 7f113406b640 1 -- 172.21.15.125:0/3033949483 shutdown_connections 2023-10-18T12:02:17.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.914+0000 7f113406b640 1 -- 172.21.15.125:0/3033949483 wait complete. 2023-10-18T12:02:17.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f113406b640 1 Processor -- start 2023-10-18T12:02:17.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f113406b640 1 -- start start 2023-10-18T12:02:17.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f113406b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 0x7f112c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f113406b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112c0b1920 0x7f112c152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f113406b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112c0c9bf0 0x7f112c153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f113406b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c15ca50 con 0x7f112c06ffb0 2023-10-18T12:02:17.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f113406b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c15cd30 con 0x7f112c0b1920 2023-10-18T12:02:17.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f113406b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c15d010 con 0x7f112c0c9bf0 2023-10-18T12:02:17.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f1131e07640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 0x7f112c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f1131e07640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 0x7f112c15df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40856/0 (socket says 172.21.15.125:40856) 2023-10-18T12:02:17.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.916+0000 7f1131e07640 1 -- 172.21.15.125:0/2773667671 learned_addr learned my addr 172.21.15.125:0/2773667671 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:17.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f1131606640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112c0b1920 0x7f112c152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f1132608640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112c0c9bf0 0x7f112c153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f1131e07640 1 -- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112c0c9bf0 msgr2=0x7f112c153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f1131e07640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112c0c9bf0 0x7f112c153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f1131e07640 1 -- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112c0b1920 msgr2=0x7f112c152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:17.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f1131e07640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112c0b1920 0x7f112c152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:17.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f1131e07640 1 -- 172.21.15.125:0/2773667671 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c009490 con 0x7f112c06ffb0 2023-10-18T12:02:17.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f1131e07640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 0x7f112c15df60 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f112800d950 tx=0x7f112800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:17.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f11227fc640 1 -- 172.21.15.125:0/2773667671 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128009c30 con 0x7f112c06ffb0 2023-10-18T12:02:17.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f11227fc640 1 -- 172.21.15.125:0/2773667671 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1128009e80 con 0x7f112c06ffb0 2023-10-18T12:02:17.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c15d2f0 con 0x7f112c06ffb0 2023-10-18T12:02:17.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f11227fc640 1 -- 172.21.15.125:0/2773667671 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11280106c0 con 0x7f112c06ffb0 2023-10-18T12:02:17.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.917+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c15e4a0 con 0x7f112c06ffb0 2023-10-18T12:02:17.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.918+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c0690e0 con 0x7f112c06ffb0 2023-10-18T12:02:17.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.918+0000 7f11227fc640 1 -- 172.21.15.125:0/2773667671 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 153) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f1128010820 con 0x7f112c06ffb0 2023-10-18T12:02:17.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.919+0000 7f11227fc640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f111006ac90 0x7f111006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:17.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.919+0000 7f11227fc640 1 -- 172.21.15.125:0/2773667671 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6902+0+0 (secure 0 0 0) 0x7f1128089be0 con 0x7f112c06ffb0 2023-10-18T12:02:17.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.919+0000 7f1131606640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f111006ac90 0x7f111006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:17.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.919+0000 7f1131606640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f111006ac90 0x7f111006d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f111c036000 tx=0x7f111c004730 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:17.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:17.923+0000 7f11227fc640 1 -- 172.21.15.125:0/2773667671 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1128054b20 con 0x7f112c06ffb0 2023-10-18T12:02:18.125 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.122+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f112c14a8a0 con 0x7f112c06ffb0 2023-10-18T12:02:18.126 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.123+0000 7f11227fc640 1 -- 172.21.15.125:0/2773667671 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f1128017090 con 0x7f112c06ffb0 2023-10-18T12:02:18.126 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:18.126 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":112,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3962342896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3962342896},{"type":"v1","addr":"172.21.15.139:6837","nonce":3962342896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11413,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1953593032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1953593032},{"type":"v1","addr":"172.21.15.139:6836","nonce":1953593032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":13566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1230654530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1230654530},{"type":"v1","addr":"172.21.15.125:6834","nonce":1230654530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:59.406238+0000","modified":"2023-10-18T12:02:01.736375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13569":{"gid":13569,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1922177652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1922177652},{"type":"v1","addr":"172.21.15.125:6835","nonce":1922177652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-18T12:02:18.127 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 112 2023-10-18T12:02:18.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f111006ac90 msgr2=0x7f111006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f111006ac90 0x7f111006d150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f111c036000 tx=0x7f111c004730 comp rx=0 tx=0).stop 2023-10-18T12:02:18.128 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 msgr2=0x7f112c15df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 0x7f112c15df60 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f112800d950 tx=0x7f112800de10 comp rx=0 tx=0).stop 2023-10-18T12:02:18.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 shutdown_connections 2023-10-18T12:02:18.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f111006ac90 0x7f111006d150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.129 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112c0b1920 0x7f112c152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f112c0c9bf0 0x7f112c153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 --2- 172.21.15.125:0/2773667671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f112c06ffb0 0x7f112c15df60 unknown :-1 s=CLOSED pgs=3091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 >> 172.21.15.125:0/2773667671 conn(0x7f112c069930 msgr2=0x7f112c159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:18.130 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 shutdown_connections 2023-10-18T12:02:18.131 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.125+0000 7f113406b640 1 -- 172.21.15.125:0/2773667671 wait complete. 2023-10-18T12:02:18.138 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:02:18.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.238+0000 7ff4cc75e640 1 -- 172.21.15.125:0/2102114995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c40ceb40 msgr2=0x7ff4c40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.238+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/2102114995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c40ceb40 0x7ff4c40b0650 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc005fd0 tx=0x7ff4bc02bcf0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.238+0000 7ff4cc75e640 1 -- 172.21.15.125:0/2102114995 shutdown_connections 2023-10-18T12:02:18.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.238+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/2102114995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c40ceb40 0x7ff4c40b0650 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.242 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.238+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/2102114995 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c415b2a0 0x7ff4c40d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.238+0000 7ff4cc75e640 1 -- 172.21.15.125:0/2102114995 >> 172.21.15.125:0/2102114995 conn(0x7ff4c40d0d60 msgr2=0x7ff4c40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:18.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.238+0000 7ff4cc75e640 1 -- 172.21.15.125:0/2102114995 shutdown_connections 2023-10-18T12:02:18.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.238+0000 7ff4cc75e640 1 -- 172.21.15.125:0/2102114995 wait complete. 2023-10-18T12:02:18.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.239+0000 7ff4cc75e640 1 Processor -- start 2023-10-18T12:02:18.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.239+0000 7ff4cc75e640 1 -- start start 2023-10-18T12:02:18.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cc75e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c40ceb40 0x7ff4c40ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cc75e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40cf340 0x7ff4c41504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cc75e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415b2a0 0x7ff4c4069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cc75e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41f2e30 con 0x7ff4c415b2a0 2023-10-18T12:02:18.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cc75e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41f1d90 con 0x7ff4c40ceb40 2023-10-18T12:02:18.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cc75e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41f2070 con 0x7ff4c40cf340 2023-10-18T12:02:18.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cacfb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415b2a0 0x7ff4c4069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cacfb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415b2a0 0x7ff4c4069f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40876/0 (socket says 172.21.15.125:40876) 2023-10-18T12:02:18.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cacfb640 1 -- 172.21.15.125:0/3910932145 learned_addr learned my addr 172.21.15.125:0/3910932145 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:18.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4c9cf9640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40cf340 0x7ff4c41504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4ca4fa640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c40ceb40 0x7ff4c40ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cacfb640 1 -- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40cf340 msgr2=0x7ff4c41504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cacfb640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40cf340 0x7ff4c41504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cacfb640 1 -- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c40ceb40 msgr2=0x7ff4c40ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cacfb640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c40ceb40 0x7ff4c40ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.240+0000 7ff4cacfb640 1 -- 172.21.15.125:0/3910932145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc009560 con 0x7ff4c415b2a0 2023-10-18T12:02:18.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.241+0000 7ff4cacfb640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415b2a0 0x7ff4c4069f90 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc036040 tx=0x7ff4bc02d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:18.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.241+0000 7ff4baffd640 1 -- 172.21.15.125:0/3910932145 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc02dda0 con 0x7ff4c415b2a0 2023-10-18T12:02:18.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.241+0000 7ff4baffd640 1 -- 172.21.15.125:0/3910932145 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4bc0035f0 con 0x7ff4c415b2a0 2023-10-18T12:02:18.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.241+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c40b0d50 con 0x7ff4c415b2a0 2023-10-18T12:02:18.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.241+0000 7ff4baffd640 1 -- 172.21.15.125:0/3910932145 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0039e0 con 0x7ff4c415b2a0 2023-10-18T12:02:18.251 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.241+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c41f2630 con 0x7ff4c415b2a0 2023-10-18T12:02:18.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.242+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c40690e0 con 0x7ff4c415b2a0 2023-10-18T12:02:18.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.242+0000 7ff4baffd640 1 -- 172.21.15.125:0/3910932145 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 153) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff4bc003b40 con 0x7ff4c415b2a0 2023-10-18T12:02:18.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.243+0000 7ff4baffd640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff4a806ac90 0x7ff4a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.243+0000 7ff4baffd640 1 -- 172.21.15.125:0/3910932145 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6902+0+0 (secure 0 0 0) 0x7ff4bc0ac280 con 0x7ff4c415b2a0 2023-10-18T12:02:18.252 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.243+0000 7ff4ca4fa640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff4a806ac90 0x7ff4a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.243+0000 7ff4ca4fa640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff4a806ac90 0x7ff4a806d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005520 tx=0x7ff4c0009040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:18.253 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.247+0000 7ff4baffd640 1 -- 172.21.15.125:0/3910932145 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4bc077130 con 0x7ff4c415b2a0 2023-10-18T12:02:18.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.448+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4c40d0b40 con 0x7ff4c415b2a0 2023-10-18T12:02:18.451 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.449+0000 7ff4baffd640 1 -- 172.21.15.125:0/3910932145 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7ff4bc043020 con 0x7ff4c415b2a0 2023-10-18T12:02:18.452 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:18.452 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":112,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3962342896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3962342896},{"type":"v1","addr":"172.21.15.139:6837","nonce":3962342896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11413,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1953593032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1953593032},{"type":"v1","addr":"172.21.15.139:6836","nonce":1953593032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":13566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1230654530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1230654530},{"type":"v1","addr":"172.21.15.125:6834","nonce":1230654530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T12:01:59.406238+0000","modified":"2023-10-18T12:02:01.736375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13569":{"gid":13569,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6835/1922177652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1922177652},{"type":"v1","addr":"172.21.15.125:6835","nonce":1922177652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-18T12:02:18.453 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 112 2023-10-18T12:02:18.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff4a806ac90 msgr2=0x7ff4a806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff4a806ac90 0x7ff4a806d150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005520 tx=0x7ff4c0009040 comp rx=0 tx=0).stop 2023-10-18T12:02:18.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415b2a0 msgr2=0x7ff4c4069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415b2a0 0x7ff4c4069f90 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc036040 tx=0x7ff4bc02d9a0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.454 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 shutdown_connections 2023-10-18T12:02:18.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff4a806ac90 0x7ff4a806d150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4c415b2a0 0x7ff4c4069f90 unknown :-1 s=CLOSED pgs=3093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4c40cf340 0x7ff4c41504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.455 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 --2- 172.21.15.125:0/3910932145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4c40ceb40 0x7ff4c40ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 >> 172.21.15.125:0/3910932145 conn(0x7ff4c40d0d60 msgr2=0x7ff4c415a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:18.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.450+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 shutdown_connections 2023-10-18T12:02:18.456 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.451+0000 7ff4cc75e640 1 -- 172.21.15.125:0/3910932145 wait complete. 2023-10-18T12:02:18.464 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:02:18.566 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.563+0000 7ff77c1e2640 1 -- 172.21.15.125:0/4010216948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7740cf340 msgr2=0x7ff7740d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.563+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/4010216948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7740cf340 0x7ff7740d1910 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 crypto rx=0x7ff77408a120 tx=0x7ff77002c260 comp rx=0 tx=0).stop 2023-10-18T12:02:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.563+0000 7ff77c1e2640 1 -- 172.21.15.125:0/4010216948 shutdown_connections 2023-10-18T12:02:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.563+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/4010216948 >> v2:172.21.15.139:3300/0 conn(0x7ff77415b2a0 0x7ff7740cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.563+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/4010216948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7740cf340 0x7ff7740d1910 unknown :-1 s=CLOSED pgs=3095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.567 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.563+0000 7ff77c1e2640 1 -- 172.21.15.125:0/4010216948 >> 172.21.15.125:0/4010216948 conn(0x7ff7740d0d60 msgr2=0x7ff7740d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:18.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.563+0000 7ff77c1e2640 1 -- 172.21.15.125:0/4010216948 shutdown_connections 2023-10-18T12:02:18.568 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.563+0000 7ff77c1e2640 1 -- 172.21.15.125:0/4010216948 wait complete. 2023-10-18T12:02:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.564+0000 7ff77c1e2640 1 Processor -- start 2023-10-18T12:02:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.564+0000 7ff77c1e2640 1 -- start start 2023-10-18T12:02:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77c1e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7740ceb40 0x7ff774156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77c1e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7740cf340 0x7ff77415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.571 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77c1e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff77415b2a0 0x7ff7740cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77c1e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff774157560 con 0x7ff77415b2a0 2023-10-18T12:02:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77c1e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff774157840 con 0x7ff7740cf340 2023-10-18T12:02:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77c1e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff774157b20 con 0x7ff7740ceb40 2023-10-18T12:02:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77a77f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff77415b2a0 0x7ff7740cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.572 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77a77f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff77415b2a0 0x7ff7740cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40888/0 (socket says 172.21.15.125:40888) 2023-10-18T12:02:18.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77a77f640 1 -- 172.21.15.125:0/1618911192 learned_addr learned my addr 172.21.15.125:0/1618911192 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:18.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff779f7e640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7740ceb40 0x7ff774156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77977d640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7740cf340 0x7ff77415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77a77f640 1 -- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7740ceb40 msgr2=0x7ff774156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.573 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77a77f640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7740ceb40 0x7ff774156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77a77f640 1 -- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7740cf340 msgr2=0x7ff77415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77a77f640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7740cf340 0x7ff77415e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.565+0000 7ff77a77f640 1 -- 172.21.15.125:0/1618911192 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770009900 con 0x7ff77415b2a0 2023-10-18T12:02:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.566+0000 7ff77a77f640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff77415b2a0 0x7ff7740cc8b0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00c3c0 tx=0x7ff76c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:18.574 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.566+0000 7ff76a7fc640 1 -- 172.21.15.125:0/1618911192 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0052d0 con 0x7ff77415b2a0 2023-10-18T12:02:18.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.566+0000 7ff76a7fc640 1 -- 172.21.15.125:0/1618911192 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff76c003610 con 0x7ff77415b2a0 2023-10-18T12:02:18.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.566+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff774157e00 con 0x7ff77415b2a0 2023-10-18T12:02:18.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.566+0000 7ff76a7fc640 1 -- 172.21.15.125:0/1618911192 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c003a20 con 0x7ff77415b2a0 2023-10-18T12:02:18.575 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.566+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7741571e0 con 0x7ff77415b2a0 2023-10-18T12:02:18.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.567+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff774158320 con 0x7ff77415b2a0 2023-10-18T12:02:18.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.567+0000 7ff76a7fc640 1 -- 172.21.15.125:0/1618911192 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 153) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff76c003b80 con 0x7ff77415b2a0 2023-10-18T12:02:18.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.567+0000 7ff76a7fc640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff75806ac90 0x7ff75806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.576 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.567+0000 7ff76a7fc640 1 -- 172.21.15.125:0/1618911192 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6902+0+0 (secure 0 0 0) 0x7ff76c088e70 con 0x7ff77415b2a0 2023-10-18T12:02:18.578 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.568+0000 7ff779f7e640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff75806ac90 0x7ff75806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.568+0000 7ff779f7e640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff75806ac90 0x7ff75806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff770036000 tx=0x7ff770004ab0 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:18.579 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.572+0000 7ff76a7fc640 1 -- 172.21.15.125:0/1618911192 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff774158320 con 0x7ff77415b2a0 2023-10-18T12:02:18.741 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.739+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff774158320 con 0x7ff77415b2a0 2023-10-18T12:02:18.742 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.740+0000 7ff76a7fc640 1 -- 172.21.15.125:0/1618911192 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) v1 ==== 74+0+18063 (secure 0 0 0) 0x7ff774158320 con 0x7ff77415b2a0 2023-10-18T12:02:18.743 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:18.743 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":265,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:02:16.882561+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-18T12:01:57.717906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-18T12:01:58.385232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:0/2140935837":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/246588314":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/1154764603":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/3832700950":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/2488548050":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2767408321":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6832/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1739931468":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/4281362145":"2023-10-19T12:02:16.882516+0000","172.21.15.139:6833/1808248507":"2023-10-19T12:02:16.882516+0000","172.21.15.139:6832/1808248507":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/2511136316":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6833/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2813907753":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1094096408":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1599845747":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/1205310989":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1383817777":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2446560793":"2023-10-18T13:02:05.851547+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:02:18.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff75806ac90 msgr2=0x7ff75806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff75806ac90 0x7ff75806d150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff770036000 tx=0x7ff770004ab0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff77415b2a0 msgr2=0x7ff7740cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff77415b2a0 0x7ff7740cc8b0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00c3c0 tx=0x7ff76c00c880 comp rx=0 tx=0).stop 2023-10-18T12:02:18.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 shutdown_connections 2023-10-18T12:02:18.745 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7ff75806ac90 0x7ff75806d150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7740cf340 0x7ff77415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff77415b2a0 0x7ff7740cc8b0 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 --2- 172.21.15.125:0/1618911192 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7740ceb40 0x7ff774156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.746 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 >> 172.21.15.125:0/1618911192 conn(0x7ff7740d0d60 msgr2=0x7ff7740b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:18.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 shutdown_connections 2023-10-18T12:02:18.747 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.742+0000 7ff77c1e2640 1 -- 172.21.15.125:0/1618911192 wait complete. 2023-10-18T12:02:18.755 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T12:02:18.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.856+0000 7f97e41ce640 1 -- 172.21.15.125:0/3961810722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 msgr2=0x7f97dc0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.859 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.856+0000 7f97e41ce640 1 --2- 172.21.15.125:0/3961810722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 0x7f97dc0b0650 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 crypto rx=0x7f97dc08a810 tx=0x7f97d402bcf0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.856+0000 7f97e41ce640 1 -- 172.21.15.125:0/3961810722 shutdown_connections 2023-10-18T12:02:18.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.856+0000 7f97e41ce640 1 --2- 172.21.15.125:0/3961810722 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 0x7f97dc0b0650 unknown :-1 s=CLOSED pgs=3098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.856+0000 7f97e41ce640 1 --2- 172.21.15.125:0/3961810722 >> v2:172.21.15.139:3300/0 conn(0x7f97dc0cf340 0x7f97dc0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.856+0000 7f97e41ce640 1 -- 172.21.15.125:0/3961810722 >> 172.21.15.125:0/3961810722 conn(0x7f97dc0d0d60 msgr2=0x7f97dc0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:18.860 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.856+0000 7f97e41ce640 1 -- 172.21.15.125:0/3961810722 shutdown_connections 2023-10-18T12:02:18.861 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.856+0000 7f97e41ce640 1 -- 172.21.15.125:0/3961810722 wait complete. 2023-10-18T12:02:18.863 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.857+0000 7f97e41ce640 1 Processor -- start 2023-10-18T12:02:18.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.857+0000 7f97e41ce640 1 -- start start 2023-10-18T12:02:18.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e41ce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 0x7f97dc157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e41ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc0cf340 0x7f97dc09e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e41ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97dc15b2a0 0x7f97dc06a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.864 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e41ce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc15ddd0 con 0x7f97dc0ceb40 2023-10-18T12:02:18.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e41ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc15d200 con 0x7f97dc0cf340 2023-10-18T12:02:18.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e41ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc09aa60 con 0x7f97dc15b2a0 2023-10-18T12:02:18.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 0x7f97dc157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 0x7f97dc157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40906/0 (socket says 172.21.15.125:40906) 2023-10-18T12:02:18.865 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 -- 172.21.15.125:0/1730294681 learned_addr learned my addr 172.21.15.125:0/1730294681 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:18.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e276b640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97dc15b2a0 0x7f97dc06a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1769640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc0cf340 0x7f97dc09e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 -- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97dc15b2a0 msgr2=0x7f97dc06a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97dc15b2a0 0x7f97dc06a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.866 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 -- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc0cf340 msgr2=0x7f97dc09e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:18.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc0cf340 0x7f97dc09e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:18.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 -- 172.21.15.125:0/1730294681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d4009490 con 0x7f97dc0ceb40 2023-10-18T12:02:18.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e276b640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97dc15b2a0 0x7f97dc06a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:18.867 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1769640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc0cf340 0x7f97dc09e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:18.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.858+0000 7f97e1f6a640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 0x7f97dc157b40 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7f97d800eab0 tx=0x7f97d800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:18.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.859+0000 7f97d27fc640 1 -- 172.21.15.125:0/1730294681 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d800ce40 con 0x7f97dc0ceb40 2023-10-18T12:02:18.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.859+0000 7f97d27fc640 1 -- 172.21.15.125:0/1730294681 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97d8011560 con 0x7f97dc0ceb40 2023-10-18T12:02:18.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.859+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc15d4a0 con 0x7f97dc0ceb40 2023-10-18T12:02:18.868 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.859+0000 7f97d27fc640 1 -- 172.21.15.125:0/1730294681 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8011910 con 0x7f97dc0ceb40 2023-10-18T12:02:18.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.859+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc15da60 con 0x7f97dc0ceb40 2023-10-18T12:02:18.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.860+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc0690e0 con 0x7f97dc0ceb40 2023-10-18T12:02:18.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.860+0000 7f97d27fc640 1 -- 172.21.15.125:0/1730294681 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 153) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f97d8011a70 con 0x7f97dc0ceb40 2023-10-18T12:02:18.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.860+0000 7f97d27fc640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f97c006ac90 0x7f97c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:18.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.860+0000 7f97d27fc640 1 -- 172.21.15.125:0/1730294681 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6902+0+0 (secure 0 0 0) 0x7f97d808a400 con 0x7f97dc0ceb40 2023-10-18T12:02:18.869 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.861+0000 7f97e1769640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f97c006ac90 0x7f97c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:18.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.861+0000 7f97e1769640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f97c006ac90 0x7f97c006d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0ae8c0 tx=0x7f97cc009040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:18.870 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:18.865+0000 7f97d27fc640 1 -- 172.21.15.125:0/1730294681 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97d8058dd0 con 0x7f97dc0ceb40 2023-10-18T12:02:19.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.069+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f97dc0d0b40 con 0x7f97dc0ceb40 2023-10-18T12:02:19.072 INFO:tasks.ceph.mon.a.smithi125.stderr:2023-10-18T12:02:19.070+0000 7fc00148c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T12:02:19.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.071+0000 7f97d27fc640 1 -- 172.21.15.125:0/1730294681 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) v1 ==== 161+0+0 (secure 0 0 0) 0x7f97d8058960 con 0x7f97dc0ceb40 2023-10-18T12:02:19.073 INFO:teuthology.orchestra.run.smithi125.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T12:02:19.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f97c006ac90 msgr2=0x7f97c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:19.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f97c006ac90 0x7f97c006d150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0ae8c0 tx=0x7f97cc009040 comp rx=0 tx=0).stop 2023-10-18T12:02:19.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 msgr2=0x7f97dc157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:19.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 0x7f97dc157b40 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7f97d800eab0 tx=0x7f97d800ef70 comp rx=0 tx=0).stop 2023-10-18T12:02:19.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 shutdown_connections 2023-10-18T12:02:19.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f97c006ac90 0x7f97c006d150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:19.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97dc0cf340 0x7f97dc09e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:19.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97dc15b2a0 0x7f97dc06a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:19.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 --2- 172.21.15.125:0/1730294681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f97dc0ceb40 0x7f97dc157b40 unknown :-1 s=CLOSED pgs=3099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:19.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 >> 172.21.15.125:0/1730294681 conn(0x7f97dc0d0d60 msgr2=0x7f97dc0af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:19.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 shutdown_connections 2023-10-18T12:02:19.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.074+0000 7f97e41ce640 1 -- 172.21.15.125:0/1730294681 wait complete. 2023-10-18T12:02:19.079 INFO:tasks.ceph.mds.c.smithi125.stderr: -2151> 2023-10-18T12:02:02.054+0000 7f41aba33640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T12:02:19.088 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T12:02:19.102 INFO:tasks.ceph.mds.c.smithi125.stdout:starting mds.c at 2023-10-18T12:02:19.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.190+0000 7f4ef9064640 1 -- 172.21.15.125:0/2714346637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 msgr2=0x7f4ef40d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:19.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.190+0000 7f4ef9064640 1 --2- 172.21.15.125:0/2714346637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 0x7f4ef40d1910 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009830 tx=0x7f4ee402bd60 comp rx=0 tx=0).stop 2023-10-18T12:02:19.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.190+0000 7f4ef9064640 1 -- 172.21.15.125:0/2714346637 shutdown_connections 2023-10-18T12:02:19.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.190+0000 7f4ef9064640 1 --2- 172.21.15.125:0/2714346637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 0x7f4ef40d1910 unknown :-1 s=CLOSED pgs=3105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:19.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.190+0000 7f4ef9064640 1 -- 172.21.15.125:0/2714346637 >> 172.21.15.125:0/2714346637 conn(0x7f4ef40d0d60 msgr2=0x7f4ef40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:19.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.190+0000 7f4ef9064640 1 -- 172.21.15.125:0/2714346637 shutdown_connections 2023-10-18T12:02:19.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.190+0000 7f4ef9064640 1 -- 172.21.15.125:0/2714346637 wait complete. 2023-10-18T12:02:19.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.191+0000 7f4ef9064640 1 Processor -- start 2023-10-18T12:02:19.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef9064640 1 -- start start 2023-10-18T12:02:19.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef9064640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ef40ceb40 0x7f4ef41501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:19.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef9064640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef40cf340 0x7f4ef40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:19.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef9064640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 0x7f4ef40b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:19.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef9064640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4150710 con 0x7f4ef415b2a0 2023-10-18T12:02:19.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef9064640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40b3e90 con 0x7f4ef40cf340 2023-10-18T12:02:19.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef9064640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40b4130 con 0x7f4ef40ceb40 2023-10-18T12:02:19.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 0x7f4ef40b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:19.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 0x7f4ef40b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40932/0 (socket says 172.21.15.125:40932) 2023-10-18T12:02:19.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.192+0000 7f4ef359e640 1 -- 172.21.15.125:0/953765025 learned_addr learned my addr 172.21.15.125:0/953765025 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:19.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef2d9d640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ef40ceb40 0x7f4ef41501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:19.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef259c640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef40cf340 0x7f4ef40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:19.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef359e640 1 -- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ef40ceb40 msgr2=0x7f4ef41501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:19.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef359e640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ef40ceb40 0x7f4ef41501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:19.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef359e640 1 -- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef40cf340 msgr2=0x7f4ef40b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:19.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef359e640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef40cf340 0x7f4ef40b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:19.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef359e640 1 -- 172.21.15.125:0/953765025 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4009490 con 0x7f4ef415b2a0 2023-10-18T12:02:19.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef359e640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 0x7f4ef40b51d0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00c760 tx=0x7f4eec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:19.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4edb7fe640 1 -- 172.21.15.125:0/953765025 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec003da0 con 0x7f4ef415b2a0 2023-10-18T12:02:19.201 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4edb7fe640 1 -- 172.21.15.125:0/953765025 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4eec00ce70 con 0x7f4ef415b2a0 2023-10-18T12:02:19.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef40b4410 con 0x7f4ef415b2a0 2023-10-18T12:02:19.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4edb7fe640 1 -- 172.21.15.125:0/953765025 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0052b0 con 0x7f4ef415b2a0 2023-10-18T12:02:19.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.193+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef40b49d0 con 0x7f4ef415b2a0 2023-10-18T12:02:19.202 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.194+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef40690e0 con 0x7f4ef415b2a0 2023-10-18T12:02:19.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.194+0000 7f4edb7fe640 1 -- 172.21.15.125:0/953765025 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 153) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4eec005410 con 0x7f4ef415b2a0 2023-10-18T12:02:19.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.195+0000 7f4edb7fe640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f4ee006ac90 0x7f4ee006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:19.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.195+0000 7f4edb7fe640 1 -- 172.21.15.125:0/953765025 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6988+0+0 (secure 0 0 0) 0x7f4eec089750 con 0x7f4ef415b2a0 2023-10-18T12:02:19.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.195+0000 7f4ef2d9d640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f4ee006ac90 0x7f4ee006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:19.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.195+0000 7f4ef2d9d640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f4ee006ac90 0x7f4ee006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8000f60 tx=0x7f4ee8009040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:19.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.199+0000 7f4edb7fe640 1 -- 172.21.15.125:0/953765025 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4eec054640 con 0x7f4ef415b2a0 2023-10-18T12:02:19.403 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.401+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4ef40d0b40 con 0x7f4ef415b2a0 2023-10-18T12:02:19.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:19.897+0000 7f4edb7fe640 1 -- 172.21.15.125:0/953765025 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 154) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4eec0056c0 con 0x7f4ef415b2a0 2023-10-18T12:02:20.075 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.072+0000 7f4edb7fe640 1 -- 172.21.15.125:0/953765025 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4eec072160 con 0x7f4ef415b2a0 2023-10-18T12:02:20.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f4ee006ac90 msgr2=0x7f4ee006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:20.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f4ee006ac90 0x7f4ee006d150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8000f60 tx=0x7f4ee8009040 comp rx=0 tx=0).stop 2023-10-18T12:02:20.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 msgr2=0x7f4ef40b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:20.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 0x7f4ef40b51d0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00c760 tx=0x7f4eec00cc20 comp rx=0 tx=0).stop 2023-10-18T12:02:20.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 shutdown_connections 2023-10-18T12:02:20.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f4ee006ac90 0x7f4ee006d150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:20.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef40cf340 0x7f4ef40b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:20.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4ef415b2a0 0x7f4ef40b51d0 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:20.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 --2- 172.21.15.125:0/953765025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ef40ceb40 0x7f4ef41501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:20.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.074+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 >> 172.21.15.125:0/953765025 conn(0x7f4ef40d0d60 msgr2=0x7f4ef406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:20.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.075+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 shutdown_connections 2023-10-18T12:02:20.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.075+0000 7f4ef9064640 1 -- 172.21.15.125:0/953765025 wait complete. 2023-10-18T12:02:20.089 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T12:02:20.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.237+0000 7f433b141640 1 -- 172.21.15.125:0/1534826499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4324003970 msgr2=0x7f4324003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:20.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.237+0000 7f433b141640 1 --2- 172.21.15.125:0/1534826499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4324003970 0x7f4324003d90 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 crypto rx=0x7f4328001680 tx=0x7f43240057f0 comp rx=0 tx=0).stop 2023-10-18T12:02:20.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.237+0000 7f433b141640 1 -- 172.21.15.125:0/1534826499 shutdown_connections 2023-10-18T12:02:20.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.237+0000 7f433b141640 1 --2- 172.21.15.125:0/1534826499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4324003970 0x7f4324003d90 unknown :-1 s=CLOSED pgs=3108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:20.240 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.237+0000 7f433b141640 1 -- 172.21.15.125:0/1534826499 >> 172.21.15.125:0/1534826499 conn(0x7f4334069930 msgr2=0x7f4334069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:20.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.237+0000 7f433b141640 1 -- 172.21.15.125:0/1534826499 shutdown_connections 2023-10-18T12:02:20.241 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.237+0000 7f433b141640 1 -- 172.21.15.125:0/1534826499 wait complete. 2023-10-18T12:02:20.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.238+0000 7f433b141640 1 Processor -- start 2023-10-18T12:02:20.243 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.238+0000 7f433b141640 1 -- start start 2023-10-18T12:02:20.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f433b141640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324003970 0x7f433412e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:20.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f433b141640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43341df000 0x7f43341df480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:20.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f433b141640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43341e1f10 0x7f43341e2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:20.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f433b141640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43341d9f30 con 0x7f43341e1f10 2023-10-18T12:02:20.244 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f433b141640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43341df9c0 con 0x7f4324003970 2023-10-18T12:02:20.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f433b141640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433413c780 con 0x7f43341df000 2023-10-18T12:02:20.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43341e1f10 0x7f43341e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:20.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43341e1f10 0x7f43341e2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40950/0 (socket says 172.21.15.125:40950) 2023-10-18T12:02:20.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 -- 172.21.15.125:0/3676267243 learned_addr learned my addr 172.21.15.125:0/3676267243 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:20.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f4338edd640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324003970 0x7f433412e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:20.245 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f4333fff640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43341df000 0x7f43341df480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:20.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 -- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43341df000 msgr2=0x7f43341df480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:20.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43341df000 0x7f43341df480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:20.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 -- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324003970 msgr2=0x7f433412e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:20.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324003970 0x7f433412e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:20.246 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 -- 172.21.15.125:0/3676267243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433412e620 con 0x7f43341e1f10 2023-10-18T12:02:20.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f4338edd640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324003970 0x7f433412e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:20.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f4333fff640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43341df000 0x7f43341df480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:20.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.239+0000 7f43396de640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43341e1f10 0x7f43341e2390 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f433406d570 tx=0x7f432c00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:20.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.240+0000 7f43317fa640 1 -- 172.21.15.125:0/3676267243 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c00c970 con 0x7f43341e1f10 2023-10-18T12:02:20.247 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.240+0000 7f43317fa640 1 -- 172.21.15.125:0/3676267243 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f432c00cc00 con 0x7f43341e1f10 2023-10-18T12:02:20.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.240+0000 7f43317fa640 1 -- 172.21.15.125:0/3676267243 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c011460 con 0x7f43341e1f10 2023-10-18T12:02:20.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.240+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433413bb70 con 0x7f43341e1f10 2023-10-18T12:02:20.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.240+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433413c130 con 0x7f43341e1f10 2023-10-18T12:02:20.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.240+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43340690e0 con 0x7f43341e1f10 2023-10-18T12:02:20.248 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.241+0000 7f43317fa640 1 -- 172.21.15.125:0/3676267243 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 154) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f432c0115c0 con 0x7f43341e1f10 2023-10-18T12:02:20.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.241+0000 7f43317fa640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f431806aed0 0x7f431806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:20.249 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.241+0000 7f43317fa640 1 -- 172.21.15.125:0/3676267243 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6988+0+0 (secure 0 0 0) 0x7f432c08a390 con 0x7f43341e1f10 2023-10-18T12:02:20.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.242+0000 7f4338edd640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f431806aed0 0x7f431806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:20.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.242+0000 7f4338edd640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f431806aed0 0x7f431806d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4328034000 tx=0x7f432802e040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:20.250 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.246+0000 7f43317fa640 1 -- 172.21.15.125:0/3676267243 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f432c054e40 con 0x7f43341e1f10 2023-10-18T12:02:20.412 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:20.410+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f433413c8c0 con 0x7f43341e1f10 2023-10-18T12:02:21.076 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.074+0000 7f43317fa640 1 -- 172.21.15.125:0/3676267243 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v267) v1 ==== 181+0+0 (secure 0 0 0) 0x7f432c058460 con 0x7f43341e1f10 2023-10-18T12:02:21.077 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_metadata' removed 2023-10-18T12:02:21.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f431806aed0 msgr2=0x7f431806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:21.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f431806aed0 0x7f431806d390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4328034000 tx=0x7f432802e040 comp rx=0 tx=0).stop 2023-10-18T12:02:21.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43341e1f10 msgr2=0x7f43341e2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:21.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43341e1f10 0x7f43341e2390 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f433406d570 tx=0x7f432c00eae0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 shutdown_connections 2023-10-18T12:02:21.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f431806aed0 0x7f431806d390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43341e1f10 0x7f43341e2390 unknown :-1 s=CLOSED pgs=3109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43341df000 0x7f43341df480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 --2- 172.21.15.125:0/3676267243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4324003970 0x7f433412e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 >> 172.21.15.125:0/3676267243 conn(0x7f4334069930 msgr2=0x7f43340ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:21.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.076+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 shutdown_connections 2023-10-18T12:02:21.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.077+0000 7f433b141640 1 -- 172.21.15.125:0/3676267243 wait complete. 2023-10-18T12:02:21.092 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T12:02:21.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.200+0000 7fa9a6d05640 1 -- 172.21.15.125:0/523585568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a015b2a0 msgr2=0x7fa9a00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:21.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.200+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/523585568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a015b2a0 0x7fa9a00d1910 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7fa9a015b680 tx=0x7fa98c02bcf0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.200+0000 7fa9a6d05640 1 -- 172.21.15.125:0/523585568 shutdown_connections 2023-10-18T12:02:21.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.200+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/523585568 >> v2:172.21.15.139:3300/0 conn(0x7fa9a00ceb40 0x7fa9a00b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.200+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/523585568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a015b2a0 0x7fa9a00d1910 unknown :-1 s=CLOSED pgs=3111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.200+0000 7fa9a6d05640 1 -- 172.21.15.125:0/523585568 >> 172.21.15.125:0/523585568 conn(0x7fa9a00d0d60 msgr2=0x7fa9a00d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:21.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.200+0000 7fa9a6d05640 1 -- 172.21.15.125:0/523585568 shutdown_connections 2023-10-18T12:02:21.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.200+0000 7fa9a6d05640 1 -- 172.21.15.125:0/523585568 wait complete. 2023-10-18T12:02:21.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.201+0000 7fa9a6d05640 1 Processor -- start 2023-10-18T12:02:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.201+0000 7fa9a6d05640 1 -- start start 2023-10-18T12:02:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a6d05640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a00ceb40 0x7fa9a01e6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a6d05640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9a00cf340 0x7fa9a0158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:21.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a6d05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9a015b2a0 0x7fa9a00b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a6d05640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00c9be0 con 0x7fa9a00ceb40 2023-10-18T12:02:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a6d05640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00ca3f0 con 0x7fa9a015b2a0 2023-10-18T12:02:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a6d05640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a01e67d0 con 0x7fa9a00cf340 2023-10-18T12:02:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a4aa1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a00ceb40 0x7fa9a01e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:21.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a4aa1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a00ceb40 0x7fa9a01e6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:40982/0 (socket says 172.21.15.125:40982) 2023-10-18T12:02:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.202+0000 7fa9a4aa1640 1 -- 172.21.15.125:0/631572448 learned_addr learned my addr 172.21.15.125:0/631572448 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a4aa1640 1 -- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9a00cf340 msgr2=0x7fa9a0158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a4aa1640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9a00cf340 0x7fa9a0158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a4aa1640 1 -- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9a015b2a0 msgr2=0x7fa9a00b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:21.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a52a2640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9a015b2a0 0x7fa9a00b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a4aa1640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9a015b2a0 0x7fa9a00b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a4aa1640 1 -- 172.21.15.125:0/631572448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009490 con 0x7fa9a00ceb40 2023-10-18T12:02:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a52a2640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9a015b2a0 0x7fa9a00b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a4aa1640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a00ceb40 0x7fa9a01e6290 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7fa99400c530 tx=0x7fa99400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:21.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa99d7fa640 1 -- 172.21.15.125:0/631572448 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994012830 con 0x7fa9a00ceb40 2023-10-18T12:02:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa99d7fa640 1 -- 172.21.15.125:0/631572448 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa994012a80 con 0x7fa9a00ceb40 2023-10-18T12:02:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a00c9e30 con 0x7fa9a00ceb40 2023-10-18T12:02:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa99d7fa640 1 -- 172.21.15.125:0/631572448 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994012e50 con 0x7fa9a00ceb40 2023-10-18T12:02:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.203+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a015b800 con 0x7fa9a00ceb40 2023-10-18T12:02:21.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.204+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a00690e0 con 0x7fa9a00ceb40 2023-10-18T12:02:21.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.204+0000 7fa99d7fa640 1 -- 172.21.15.125:0/631572448 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 154) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa99401a3b0 con 0x7fa9a00ceb40 2023-10-18T12:02:21.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.205+0000 7fa99d7fa640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fa98406aed0 0x7fa98406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:21.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.205+0000 7fa99d7fa640 1 -- 172.21.15.125:0/631572448 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(267..267 src has 1..267) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fa9940899f0 con 0x7fa9a00ceb40 2023-10-18T12:02:21.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.205+0000 7fa99ffff640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fa98406aed0 0x7fa98406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:21.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.206+0000 7fa99ffff640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fa98406aed0 0x7fa98406d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa98c036000 tx=0x7fa98c0046c0 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:21.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.209+0000 7fa99d7fa640 1 -- 172.21.15.125:0/631572448 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa99408be60 con 0x7fa9a00ceb40 2023-10-18T12:02:21.378 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.375+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa9a00b0d00 con 0x7fa9a00ceb40 2023-10-18T12:02:21.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:21.506+0000 7fa99d7fa640 1 -- 172.21.15.125:0/631572448 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa994054bc0 con 0x7fa9a00ceb40 2023-10-18T12:02:22.078 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.075+0000 7fa99d7fa640 1 -- 172.21.15.125:0/631572448 <== mon.0 v2:172.21.15.125:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v268) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa994071ab0 con 0x7fa9a00ceb40 2023-10-18T12:02:22.078 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'cephfs_data' removed 2023-10-18T12:02:22.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fa98406aed0 msgr2=0x7fa98406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fa98406aed0 0x7fa98406d390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa98c036000 tx=0x7fa98c0046c0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a00ceb40 msgr2=0x7fa9a01e6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a00ceb40 0x7fa9a01e6290 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7fa99400c530 tx=0x7fa99400c9f0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 shutdown_connections 2023-10-18T12:02:22.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fa98406aed0 0x7fa98406d390 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa98c036000 tx=0x7fa98c0046c0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9a015b2a0 0x7fa9a00b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa9a00cf340 0x7fa9a0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 --2- 172.21.15.125:0/631572448 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9a00ceb40 0x7fa9a01e6290 unknown :-1 s=CLOSED pgs=3112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.079+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 >> 172.21.15.125:0/631572448 conn(0x7fa9a00d0d60 msgr2=0x7fa9a015a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:22.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.080+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 shutdown_connections 2023-10-18T12:02:22.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.080+0000 7fa9a6d05640 1 -- 172.21.15.125:0/631572448 wait complete. 2023-10-18T12:02:22.098 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 30 2023-10-18T12:02:22.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.205+0000 7f436cd6e640 1 -- 172.21.15.125:0/1425844171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43500052a0 msgr2=0x7f435000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.205+0000 7f436cd6e640 1 --2- 172.21.15.125:0/1425844171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43500052a0 0x7f435000b980 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7f436000baf0 tx=0x7f43500057f0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.205+0000 7f436cd6e640 1 -- 172.21.15.125:0/1425844171 shutdown_connections 2023-10-18T12:02:22.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.205+0000 7f436cd6e640 1 --2- 172.21.15.125:0/1425844171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43500052a0 0x7f435000b980 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.205+0000 7f436cd6e640 1 -- 172.21.15.125:0/1425844171 >> 172.21.15.125:0/1425844171 conn(0x7f43680d0d60 msgr2=0x7f43680d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:22.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.205+0000 7f436cd6e640 1 -- 172.21.15.125:0/1425844171 shutdown_connections 2023-10-18T12:02:22.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.205+0000 7f436cd6e640 1 -- 172.21.15.125:0/1425844171 wait complete. 2023-10-18T12:02:22.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.206+0000 7f436cd6e640 1 Processor -- start 2023-10-18T12:02:22.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.206+0000 7f436cd6e640 1 -- start start 2023-10-18T12:02:22.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436cd6e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43500052a0 0x7f4368158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:22.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436cd6e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680c9290 0x7f43680c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:22.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436cd6e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43681f1620 0x7f43681f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:22.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436cd6e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436815c050 con 0x7f43681f1620 2023-10-18T12:02:22.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436cd6e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680c9c50 con 0x7f43500052a0 2023-10-18T12:02:22.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436cd6e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436815c4e0 con 0x7f43680c9290 2023-10-18T12:02:22.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43681f1620 0x7f43681f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:22.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43681f1620 0x7f43681f1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41004/0 (socket says 172.21.15.125:41004) 2023-10-18T12:02:22.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 -- 172.21.15.125:0/967094451 learned_addr learned my addr 172.21.15.125:0/967094451 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:22.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436659c640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43500052a0 0x7f4368158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:22.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4365d9b640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680c9290 0x7f43680c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:22.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 -- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680c9290 msgr2=0x7f43680c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680c9290 0x7f43680c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 -- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43500052a0 msgr2=0x7f4368158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43500052a0 0x7f4368158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 -- 172.21.15.125:0/967094451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436815b720 con 0x7f43681f1620 2023-10-18T12:02:22.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4365d9b640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680c9290 0x7f43680c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:22.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4366d9d640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43681f1620 0x7f43681f1a80 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7f436000bc10 tx=0x7f4360004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:22.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4356ffd640 1 -- 172.21.15.125:0/967094451 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360003800 con 0x7f43681f1620 2023-10-18T12:02:22.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436815ba00 con 0x7f43681f1620 2023-10-18T12:02:22.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4356ffd640 1 -- 172.21.15.125:0/967094451 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4360003a50 con 0x7f43681f1620 2023-10-18T12:02:22.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f4356ffd640 1 -- 172.21.15.125:0/967094451 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360003e40 con 0x7f43681f1620 2023-10-18T12:02:22.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.207+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43681f2390 con 0x7f43681f1620 2023-10-18T12:02:22.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.208+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43680d04f0 con 0x7f43681f1620 2023-10-18T12:02:22.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.209+0000 7f4356ffd640 1 -- 172.21.15.125:0/967094451 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4360036450 con 0x7f43681f1620 2023-10-18T12:02:22.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.209+0000 7f4356ffd640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f434806af90 0x7f434806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:22.217 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.209+0000 7f4356ffd640 1 -- 172.21.15.125:0/967094451 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f43600aa880 con 0x7f43681f1620 2023-10-18T12:02:22.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.210+0000 7f436659c640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f434806af90 0x7f434806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:22.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.210+0000 7f436659c640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f434806af90 0x7f434806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4358006fd0 tx=0x7f4358006f20 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:22.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.214+0000 7f4356ffd640 1 -- 172.21.15.125:0/967094451 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43600756a0 con 0x7f43681f1620 2023-10-18T12:02:22.387 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.384+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "config reset", "num": 30} v 0) v1 -- 0x7f43680690e0 con 0x7f43681f1620 2023-10-18T12:02:22.390 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.388+0000 7f4356ffd640 1 -- 172.21.15.125:0/967094451 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 30}]=0 v86) v1 ==== 71+0+0 (secure 0 0 0) 0x7f4360044080 con 0x7f43681f1620 2023-10-18T12:02:22.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f434806af90 msgr2=0x7f434806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f434806af90 0x7f434806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4358006fd0 tx=0x7f4358006f20 comp rx=0 tx=0).stop 2023-10-18T12:02:22.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43681f1620 msgr2=0x7f43681f1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.393 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43681f1620 0x7f43681f1a80 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7f436000bc10 tx=0x7f4360004380 comp rx=0 tx=0).stop 2023-10-18T12:02:22.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 shutdown_connections 2023-10-18T12:02:22.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f434806af90 0x7f434806d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43681f1620 0x7f43681f1a80 unknown :-1 s=CLOSED pgs=3115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.394 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680c9290 0x7f43680c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 --2- 172.21.15.125:0/967094451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43500052a0 0x7f4368158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 >> 172.21.15.125:0/967094451 conn(0x7f43680d0d60 msgr2=0x7f436806b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:22.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 shutdown_connections 2023-10-18T12:02:22.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.390+0000 7f436cd6e640 1 -- 172.21.15.125:0/967094451 wait complete. 2023-10-18T12:02:22.405 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-18T12:02:22.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.505+0000 7fdb61bce640 1 -- 172.21.15.125:0/4135792158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 msgr2=0x7fdb5c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.505+0000 7fdb61bce640 1 --2- 172.21.15.125:0/4135792158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 0x7fdb5c0c9b30 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009900 tx=0x7fdb5002be00 comp rx=0 tx=0).stop 2023-10-18T12:02:22.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.505+0000 7fdb61bce640 1 -- 172.21.15.125:0/4135792158 shutdown_connections 2023-10-18T12:02:22.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.505+0000 7fdb61bce640 1 --2- 172.21.15.125:0/4135792158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 0x7fdb5c0c9b30 unknown :-1 s=CLOSED pgs=3116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.505+0000 7fdb61bce640 1 --2- 172.21.15.125:0/4135792158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c06af00 0x7fdb5c06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.505+0000 7fdb61bce640 1 -- 172.21.15.125:0/4135792158 >> 172.21.15.125:0/4135792158 conn(0x7fdb5c069930 msgr2=0x7fdb5c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:22.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.505+0000 7fdb61bce640 1 -- 172.21.15.125:0/4135792158 shutdown_connections 2023-10-18T12:02:22.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.505+0000 7fdb61bce640 1 -- 172.21.15.125:0/4135792158 wait complete. 2023-10-18T12:02:22.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.506+0000 7fdb61bce640 1 Processor -- start 2023-10-18T12:02:22.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.506+0000 7fdb61bce640 1 -- start start 2023-10-18T12:02:22.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb61bce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c06af00 0x7fdb5c0a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:22.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb61bce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0ab500 0x7fdb5c130f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:22.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb61bce640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 0x7fdb5c12f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:22.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb61bce640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c12ee60 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb61bce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0a69f0 con 0x7fdb5c06af00 2023-10-18T12:02:22.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb61bce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0aab50 con 0x7fdb5c0ab500 2023-10-18T12:02:22.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 0x7fdb5c12f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:22.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 0x7fdb5c12f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41034/0 (socket says 172.21.15.125:41034) 2023-10-18T12:02:22.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5bfff640 1 -- 172.21.15.125:0/3887945704 learned_addr learned my addr 172.21.15.125:0/3887945704 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:22.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5affd640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0ab500 0x7fdb5c130f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:22.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5b7fe640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c06af00 0x7fdb5c0a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:22.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5bfff640 1 -- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0ab500 msgr2=0x7fdb5c130f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5bfff640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0ab500 0x7fdb5c130f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5bfff640 1 -- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c06af00 msgr2=0x7fdb5c0a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:22.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5bfff640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c06af00 0x7fdb5c0a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:22.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5bfff640 1 -- 172.21.15.125:0/3887945704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb50009560 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.507+0000 7fdb5b7fe640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c06af00 0x7fdb5c0a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:22.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.508+0000 7fdb5bfff640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 0x7fdb5c12f280 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400c740 tx=0x7fdb5400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.508+0000 7fdb43fff640 1 -- 172.21.15.125:0/3887945704 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54006d50 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.508+0000 7fdb43fff640 1 -- 172.21.15.125:0/3887945704 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb5400ce50 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.508+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c0aae30 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.508+0000 7fdb43fff640 1 -- 172.21.15.125:0/3887945704 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54003bd0 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.508+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c131450 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.516 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.509+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c12ebc0 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.509+0000 7fdb43fff640 1 -- 172.21.15.125:0/3887945704 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdb54003d30 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.510+0000 7fdb43fff640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fdb4806af90 0x7fdb4806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.510+0000 7fdb43fff640 1 -- 172.21.15.125:0/3887945704 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fdb54089680 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.510+0000 7fdb5b7fe640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fdb4806af90 0x7fdb4806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.510+0000 7fdb5b7fe640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fdb4806af90 0x7fdb4806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0058c0 tx=0x7fdb4c005850 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:22.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.514+0000 7fdb43fff640 1 -- 172.21.15.125:0/3887945704 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb5c12ebc0 con 0x7fdb5c0c96b0 2023-10-18T12:02:22.744 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:22.742+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fdb5c12efa0 con 0x7fdb5c0c96b0 2023-10-18T12:02:23.084 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.081+0000 7fdb43fff640 1 -- 172.21.15.125:0/3887945704 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3203) v1 ==== 155+0+0 (secure 0 0 0) 0x7fdb5c12efa0 con 0x7fdb5c0c96b0 2023-10-18T12:02:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fdb4806af90 msgr2=0x7fdb4806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fdb4806af90 0x7fdb4806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0058c0 tx=0x7fdb4c005850 comp rx=0 tx=0).stop 2023-10-18T12:02:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 msgr2=0x7fdb5c12f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 0x7fdb5c12f280 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400c740 tx=0x7fdb5400cc00 comp rx=0 tx=0).stop 2023-10-18T12:02:23.087 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 shutdown_connections 2023-10-18T12:02:23.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fdb4806af90 0x7fdb4806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:23.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb5c0c96b0 0x7fdb5c12f280 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:23.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb5c0ab500 0x7fdb5c130f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:23.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 --2- 172.21.15.125:0/3887945704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb5c06af00 0x7fdb5c0a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:23.088 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 >> 172.21.15.125:0/3887945704 conn(0x7fdb5c069930 msgr2=0x7fdb5c15c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:23.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 shutdown_connections 2023-10-18T12:02:23.089 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.084+0000 7fdb61bce640 1 -- 172.21.15.125:0/3887945704 wait complete. 2023-10-18T12:02:23.098 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ERROR 2023-10-18T12:02:23.099 INFO:tasks.cephfs_test_runner: 2023-10-18T12:02:23.099 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-18T12:02:23.100 INFO:tasks.cephfs_test_runner:ERROR: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-18T12:02:23.100 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name 2023-10-18T12:02:23.100 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-18T12:02:23.101 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-18T12:02:23.101 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/cephfs/test_snap_schedules.py", line 612, in test_snap_dir_name 2023-10-18T12:02:23.101 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/cephfs/test_snap_schedules.py", line 592, in get_snap_dir_name 2023-10-18T12:02:23.102 INFO:tasks.cephfs_test_runner:ModuleNotFoundError: No module named 'tasks.cephfs.kernel_mount' 2023-10-18T12:02:23.102 INFO:tasks.cephfs_test_runner: 2023-10-18T12:02:23.105 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-18T12:02:23.105 INFO:tasks.cephfs_test_runner:Ran 14 tests in 5797.916s 2023-10-18T12:02:23.106 INFO:tasks.cephfs_test_runner: 2023-10-18T12:02:23.106 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-18T12:02:23.106 INFO:tasks.cephfs_test_runner: 2023-10-18T12:02:23.107 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-18T12:02:23.107 INFO:tasks.cephfs_test_runner:ERROR: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-18T12:02:23.107 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name 2023-10-18T12:02:23.108 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-18T12:02:23.108 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-18T12:02:23.108 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/cephfs/test_snap_schedules.py", line 612, in test_snap_dir_name 2023-10-18T12:02:23.109 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/cephfs/test_snap_schedules.py", line 592, in get_snap_dir_name 2023-10-18T12:02:23.109 INFO:tasks.cephfs_test_runner:ModuleNotFoundError: No module named 'tasks.cephfs.kernel_mount' 2023-10-18T12:02:23.109 INFO:tasks.cephfs_test_runner: 2023-10-18T12:02:23.110 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/cephfs_test_runner.py", line 211, in task RuntimeError: Test failure: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-18T12:02:23.183 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ac04b0600b0344af80b64f54ca23fe9d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/cephfs_test_runner.py", line 211, in task RuntimeError: Test failure: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-18T12:02:23.185 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-18T12:02:23.196 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-18T12:02:23.204 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-18T12:02:23.204 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:02:23.205 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:02:23.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:02:23.225 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2023-10-18T12:02:23.225 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:02:23.280 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054828,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=174 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247628k,nr_inodes=811907,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495264k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T12:02:23.281 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:02:23.281 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:02:23.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:02:23.300 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2023-10-18T12:02:23.300 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T12:02:23.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054833,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16941 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3247632k,nr_inodes=811908,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6495272k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T12:02:23.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:23.359 DEBUG:teuthology.orchestra.run.smithi125:> ip netns list 2023-10-18T12:02:23.374 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T12:02:23.375 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T12:02:23.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:23.376 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T12:02:23.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:23.444 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T12:02:23.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:23.512 DEBUG:teuthology.orchestra.run.smithi125:> sudo ip link delete ceph-brx 2023-10-18T12:02:23.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:23.617 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2023-10-18T12:02:23.636 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T12:02:23.637 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T12:02:23.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:23.638 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T12:02:23.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:23.709 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T12:02:23.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T12:02:23.780 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2023-10-18T12:02:23.882 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-18T12:02:23.892 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T12:02:23.893 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:02:23.997 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.994+0000 7f63dd35e640 1 -- 172.21.15.125:0/3143522928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 msgr2=0x7f63d8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:23.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.994+0000 7f63dd35e640 1 --2- 172.21.15.125:0/3143522928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 0x7f63d8158300 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 crypto rx=0x7f63d80b10f0 tx=0x7f63cc02c260 comp rx=0 tx=0).stop 2023-10-18T12:02:23.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.994+0000 7f63dd35e640 1 -- 172.21.15.125:0/3143522928 shutdown_connections 2023-10-18T12:02:23.998 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.994+0000 7f63dd35e640 1 --2- 172.21.15.125:0/3143522928 >> v2:172.21.15.139:3300/0 conn(0x7f63d80b0530 0x7f63d80b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.994+0000 7f63dd35e640 1 --2- 172.21.15.125:0/3143522928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 0x7f63d8158300 unknown :-1 s=CLOSED pgs=3118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.994+0000 7f63dd35e640 1 -- 172.21.15.125:0/3143522928 >> 172.21.15.125:0/3143522928 conn(0x7f63d8069930 msgr2=0x7f63d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.994+0000 7f63dd35e640 1 -- 172.21.15.125:0/3143522928 shutdown_connections 2023-10-18T12:02:23.999 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.994+0000 7f63dd35e640 1 -- 172.21.15.125:0/3143522928 wait complete. 2023-10-18T12:02:24.002 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.995+0000 7f63dd35e640 1 Processor -- start 2023-10-18T12:02:24.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63dd35e640 1 -- start start 2023-10-18T12:02:24.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63dd35e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 0x7f63d8126fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63dd35e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63d80b0530 0x7f63d813f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.003 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63dd35e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63d80cbd20 0x7f63d813fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63dd35e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d81467d0 con 0x7f63d80afa20 2023-10-18T12:02:24.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63dd35e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8141140 con 0x7f63d80b0530 2023-10-18T12:02:24.004 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63dd35e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8140530 con 0x7f63d80cbd20 2023-10-18T12:02:24.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63d6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 0x7f63d8126fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63d6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 0x7f63d8126fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41052/0 (socket says 172.21.15.125:41052) 2023-10-18T12:02:24.005 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.996+0000 7f63d6ffd640 1 -- 172.21.15.125:0/2364089105 learned_addr learned my addr 172.21.15.125:0/2364089105 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:24.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d67fc640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63d80b0530 0x7f63d813f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d6ffd640 1 -- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63d80cbd20 msgr2=0x7f63d813fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d77fe640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63d80cbd20 0x7f63d813fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.006 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d6ffd640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63d80cbd20 0x7f63d813fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d6ffd640 1 -- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63d80b0530 msgr2=0x7f63d813f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d6ffd640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63d80b0530 0x7f63d813f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d6ffd640 1 -- 172.21.15.125:0/2364089105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc009900 con 0x7f63d80afa20 2023-10-18T12:02:24.007 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d67fc640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63d80b0530 0x7f63d813f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:24.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d77fe640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63d80cbd20 0x7f63d813fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:24.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63d6ffd640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 0x7f63d8126fe0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b3f0 tx=0x7f63d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:24.008 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63bf7fe640 1 -- 172.21.15.125:0/2364089105 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0011830 con 0x7f63d80afa20 2023-10-18T12:02:24.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63bf7fe640 1 -- 172.21.15.125:0/2364089105 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f63d0011a80 con 0x7f63d80afa20 2023-10-18T12:02:24.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63bf7fe640 1 -- 172.21.15.125:0/2364089105 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0011e90 con 0x7f63d80afa20 2023-10-18T12:02:24.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d8140810 con 0x7f63d80afa20 2023-10-18T12:02:24.009 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.997+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d8140dd0 con 0x7f63d80afa20 2023-10-18T12:02:24.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.998+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d80690e0 con 0x7f63d80afa20 2023-10-18T12:02:24.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.998+0000 7f63bf7fe640 1 -- 172.21.15.125:0/2364089105 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f63d001a400 con 0x7f63d80afa20 2023-10-18T12:02:24.010 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.999+0000 7f63bf7fe640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f63c406af60 0x7f63c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.999+0000 7f63bf7fe640 1 -- 172.21.15.125:0/2364089105 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f63d0089a00 con 0x7f63d80afa20 2023-10-18T12:02:24.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:23.999+0000 7f63d67fc640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f63c406af60 0x7f63c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.011 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.000+0000 7f63d67fc640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f63c406af60 0x7f63c406d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f63cc036000 tx=0x7f63cc004ab0 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:24.012 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.003+0000 7f63bf7fe640 1 -- 172.21.15.125:0/2364089105 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63d00547f0 con 0x7f63d80afa20 2023-10-18T12:02:24.174 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.171+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63d80cc100 con 0x7f63c406af60 2023-10-18T12:02:24.177 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.174+0000 7f63bf7fe640 1 -- 172.21.15.125:0/2364089105 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50982 (secure 0 0 0) 0x7f63cc003970 con 0x7f63c406af60 2023-10-18T12:02:24.178 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:24.179 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:02:24.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f63c406af60 msgr2=0x7f63c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.180 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f63c406af60 0x7f63c406d420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f63cc036000 tx=0x7f63cc004ab0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 msgr2=0x7f63d8126fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 0x7f63d8126fe0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b3f0 tx=0x7f63d000b8b0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 shutdown_connections 2023-10-18T12:02:24.181 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f63c406af60 0x7f63c406d420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63d80b0530 0x7f63d813f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63d80cbd20 0x7f63d813fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.182 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 --2- 172.21.15.125:0/2364089105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f63d80afa20 0x7f63d8126fe0 unknown :-1 s=CLOSED pgs=3119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 >> 172.21.15.125:0/2364089105 conn(0x7f63d8069930 msgr2=0x7f63d806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:24.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 shutdown_connections 2023-10-18T12:02:24.183 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.176+0000 7f63dd35e640 1 -- 172.21.15.125:0/2364089105 wait complete. 2023-10-18T12:02:24.190 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2023-10-18T12:02:22.901251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3813,"ondisk_log_size":3813,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":455036,"kb_used_data":27916,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754519684,"statfs":{"total":773094113280,"available":772628156416,"internally_reserved":0,"allocated":28585984,"data_stored":15518402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":437243448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-663619,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-120,"num_read_kb":16,"num_write":-196,"num_write_kb":-1395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.999651"},"pg_stats":[{"pgid":"1.0","version":"265'3813","reported_seq":4900,"reported_epoch":267,"state":"active+clean","last_fresh":"2023-10-18T12:02:21.401052+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:02:21.401052+0000","last_peered":"2023-10-18T12:02:21.401052+0000","last_clean":"2023-10-18T12:02:21.401052+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:02:21.401052+0000","last_undegraded":"2023-10-18T12:02:21.401052+0000","last_fullsized":"2023-10-18T12:02:21.401052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3813,"ondisk_log_size":3813,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576012,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71160,"kb_used_data":3880,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300680,"statfs":{"total":96636764160,"available":96563896320,"internally_reserved":0,"allocated":3973120,"data_stored":2340025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.80800000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.56200000000000006,"15min":0.60099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":1.363},"last":0.621}]},{"osd":1,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.86399999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56100000000000005,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.90000000000000002,"15min":1.298},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71899999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":1.5249999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.81299999999999994},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.503,"15min":0.5},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70699999999999996,"5min":0.86599999999999999,"15min":1.478},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.83599999999999997,"15min":0.874},"last":0.76900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43104,"kb_used_data":3604,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328736,"statfs":{"total":96636764160,"available":96592625664,"internally_reserved":0,"allocated":3690496,"data_stored":2070141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54100000000000004,"15min":0.57799999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56999999999999995,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65700000000000003,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.54100000000000004,"15min":0.58499999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66900000000000004,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.626},{"interface":"front","average":{"1min":0.505,"5min":0.58199999999999996,"15min":0.61399999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73399999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72899999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.55900000000000005,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.55900000000000005,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.59499999999999997,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.58899999999999997,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.60599999999999998,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.70199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.56799999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.872,"15min":0.872},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.75,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834576012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48368,"kb_used_data":3300,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323472,"statfs":{"total":96636764160,"available":96587235328,"internally_reserved":0,"allocated":3379200,"data_stored":1749657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.56499999999999995,"15min":0.59299999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60999999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.58899999999999997,"15min":0.72099999999999997},"min":{"1min":0.308,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67400000000000004,"5min":1.081,"15min":1.1539999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.626,"15min":0.73299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67800000000000005,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56000000000000005,"15min":0.59199999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.57099999999999995,"15min":0.60299999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.628}]},{"osd":3,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.60599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65100000000000002,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65000000000000002,"5min":0.81399999999999995,"15min":1},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.46999999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.59899999999999998,"5min":0.82999999999999996,"15min":1.01},"last":0.499},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.314,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.72399999999999998,"5min":0.872,"15min":0.97999999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.26900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.101},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.58099999999999996,"15min":0.59299999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70999999999999996,"5min":0.83399999999999996,"15min":1.0309999999999999},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834576012,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52548,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319292,"statfs":{"total":96636764160,"available":96582955008,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.60999999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.81499999999999995,"5min":0.872,"15min":1.123},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73299999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.42199999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.755,"5min":1.018,"15min":1.018},"last":0.441}]},{"osd":7,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.88900000000000001,"15min":1.05},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.45300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48388,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323452,"statfs":{"total":96636764160,"available":96587214848,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.34899999999999998,"15min":0.376},"min":{"1min":0.219,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.434,"5min":0.57099999999999995,"15min":0.61599999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.312,"5min":0.371,"15min":0.40000000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41499999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.28199999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.40000000000000002},"min":{"1min":0.26000000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45600000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41099999999999998,"15min":0.43099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.441,"5min":1.899,"15min":1.899},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40500000000000003,"15min":0.44},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.42399999999999999,"15min":0.45400000000000001},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.29899999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.65700000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.755}]},{"osd":5,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61699999999999999,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.522,"15min":1.522},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.65600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.621,"15min":0.65200000000000002},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.66800000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65028,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306812,"statfs":{"total":96636764160,"available":96570175488,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36699999999999999,"15min":0.38400000000000001},"min":{"1min":0.20799999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.54800000000000004,"5min":0.64500000000000002,"15min":0.67200000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.38,"15min":0.40999999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.41199999999999998},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.624},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.44500000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.39400000000000002,"15min":0.41599999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48299999999999998,"5min":0.57499999999999996,"15min":0.64600000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42999999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.50600000000000001,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.54700000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63900000000000001,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92100000000000004,"15min":0.93100000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.67500000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.91300000000000003,"15min":1.087},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60999999999999999,"15min":0.64300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.65000000000000002},"min":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.60199999999999998,"15min":0.64200000000000002},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67400000000000004,"5min":0.879,"15min":0.95399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64100000000000001,"15min":0.66800000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78200000000000003,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.628,"15min":0.66400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79900000000000004,"5min":0.86899999999999999,"15min":1.0609999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.627,"15min":0.66300000000000003},"min":{"1min":0.434,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64996,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306844,"statfs":{"total":96636764160,"available":96570208256,"internally_reserved":0,"allocated":3366912,"data_stored":1742551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.375,"15min":0.41899999999999998},"min":{"1min":0.218,"5min":0.217,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.73899999999999999,"15min":0.85599999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.35699999999999998,"15min":0.36899999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.42099999999999999,"5min":0.622,"15min":0.82599999999999996},"last":0.30499999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38700000000000001,"15min":0.436},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.65800000000000003,"15min":0.81000000000000005},"last":0.25},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.48699999999999999,"5min":0.68999999999999995,"15min":0.877},"last":0.35299999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.435,"15min":0.48899999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47899999999999998,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":0.74399999999999999,"15min":0.91000000000000003},"last":0.29299999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.73399999999999999,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":0.89900000000000002,"15min":1.004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.91500000000000004,"15min":1.05},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.621,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.67400000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.65200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834576011,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61444,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310396,"statfs":{"total":96636764160,"available":96573845504,"internally_reserved":0,"allocated":3989504,"data_stored":2346783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36799999999999999,"15min":0.38800000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.65600000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38400000000000001,"15min":0.41199999999999998},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.27100000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.252,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.42899999999999999},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.377}]},{"osd":3,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.53600000000000003,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.495}]},{"osd":4,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60199999999999998,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90900000000000003,"5min":0.96299999999999997,"15min":0.997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.64100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60699999999999998,"15min":0.64600000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96399999999999997},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.03},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61099999999999999,"15min":0.66000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79200000000000004,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.67100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.63200000000000001,"15min":0.66900000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.997,"15min":0.997},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:02:24.194 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:02:24.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.294+0000 7fd81b505640 1 -- 172.21.15.125:0/3013116873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd800003970 msgr2=0x7fd800003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.294+0000 7fd81b505640 1 --2- 172.21.15.125:0/3013116873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd800003970 0x7fd800003d90 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7fd810001d50 tx=0x7fd8000057f0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.297 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.294+0000 7fd81b505640 1 -- 172.21.15.125:0/3013116873 shutdown_connections 2023-10-18T12:02:24.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.294+0000 7fd81b505640 1 --2- 172.21.15.125:0/3013116873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd800003970 0x7fd800003d90 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.294+0000 7fd81b505640 1 -- 172.21.15.125:0/3013116873 >> 172.21.15.125:0/3013116873 conn(0x7fd8140d0d60 msgr2=0x7fd8140d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:24.298 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.294+0000 7fd81b505640 1 -- 172.21.15.125:0/3013116873 shutdown_connections 2023-10-18T12:02:24.299 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.294+0000 7fd81b505640 1 -- 172.21.15.125:0/3013116873 wait complete. 2023-10-18T12:02:24.301 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.295+0000 7fd81b505640 1 Processor -- start 2023-10-18T12:02:24.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.295+0000 7fd81b505640 1 -- start start 2023-10-18T12:02:24.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd81b505640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd800003970 0x7fd814140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.302 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd81b505640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8140c7f30 0x7fd8140c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd81b505640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8140b49a0 0x7fd8140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd81b505640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8140c88f0 con 0x7fd8140c7f30 2023-10-18T12:02:24.303 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd81b505640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8140b56d0 con 0x7fd800003970 2023-10-18T12:02:24.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd81b505640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8140b59b0 con 0x7fd8140b49a0 2023-10-18T12:02:24.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8140c7f30 0x7fd8140c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8140c7f30 0x7fd8140c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41066/0 (socket says 172.21.15.125:41066) 2023-10-18T12:02:24.304 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 -- 172.21.15.125:0/2840685950 learned_addr learned my addr 172.21.15.125:0/2840685950 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:24.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd8192a1640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd800003970 0x7fd814140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd819aa2640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8140b49a0 0x7fd8140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.305 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 -- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8140b49a0 msgr2=0x7fd8140b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8140b49a0 0x7fd8140b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 -- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd800003970 msgr2=0x7fd814140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.306 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd800003970 0x7fd814140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 -- 172.21.15.125:0/2840685950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8140b5c90 con 0x7fd8140c7f30 2023-10-18T12:02:24.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd819aa2640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8140b49a0 0x7fd8140b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:24.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd818aa0640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8140c7f30 0x7fd8140c83b0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7fd804001c40 tx=0x7fd80400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:24.307 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd809ffb640 1 -- 172.21.15.125:0/2840685950 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804010890 con 0x7fd8140c7f30 2023-10-18T12:02:24.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd809ffb640 1 -- 172.21.15.125:0/2840685950 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd804010ae0 con 0x7fd8140c7f30 2023-10-18T12:02:24.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd809ffb640 1 -- 172.21.15.125:0/2840685950 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804011460 con 0x7fd8140c7f30 2023-10-18T12:02:24.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.296+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8140b5f70 con 0x7fd8140c7f30 2023-10-18T12:02:24.308 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.297+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8140b5390 con 0x7fd8140c7f30 2023-10-18T12:02:24.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.298+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8140690e0 con 0x7fd8140c7f30 2023-10-18T12:02:24.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.298+0000 7fd809ffb640 1 -- 172.21.15.125:0/2840685950 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd8040115c0 con 0x7fd8140c7f30 2023-10-18T12:02:24.309 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.298+0000 7fd809ffb640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd7f806af90 0x7fd7f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.298+0000 7fd809ffb640 1 -- 172.21.15.125:0/2840685950 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fd80408a070 con 0x7fd8140c7f30 2023-10-18T12:02:24.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.299+0000 7fd8192a1640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd7f806af90 0x7fd7f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.310 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.299+0000 7fd8192a1640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd7f806af90 0x7fd7f806d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd810034000 tx=0x7fd810001510 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:24.311 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.303+0000 7fd809ffb640 1 -- 172.21.15.125:0/2840685950 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd804054e00 con 0x7fd8140c7f30 2023-10-18T12:02:24.469 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.466+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8140d0b40 con 0x7fd7f806af90 2023-10-18T12:02:24.471 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.469+0000 7fd809ffb640 1 -- 172.21.15.125:0/2840685950 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50982 (secure 0 0 0) 0x7fd8140d0b40 con 0x7fd7f806af90 2023-10-18T12:02:24.472 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:24.472 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:02:24.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd7f806af90 msgr2=0x7fd7f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd7f806af90 0x7fd7f806d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd810034000 tx=0x7fd810001510 comp rx=0 tx=0).stop 2023-10-18T12:02:24.474 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8140c7f30 msgr2=0x7fd8140c83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8140c7f30 0x7fd8140c83b0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7fd804001c40 tx=0x7fd80400e960 comp rx=0 tx=0).stop 2023-10-18T12:02:24.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 shutdown_connections 2023-10-18T12:02:24.475 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd7f806af90 0x7fd7f806d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8140b49a0 0x7fd8140b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd8140c7f30 0x7fd8140c83b0 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 --2- 172.21.15.125:0/2840685950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd800003970 0x7fd814140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 >> 172.21.15.125:0/2840685950 conn(0x7fd8140d0d60 msgr2=0x7fd81415a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:24.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 shutdown_connections 2023-10-18T12:02:24.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.471+0000 7fd81b505640 1 -- 172.21.15.125:0/2840685950 wait complete. 2023-10-18T12:02:24.484 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2023-10-18T12:02:22.901251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3813,"ondisk_log_size":3813,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":455036,"kb_used_data":27916,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754519684,"statfs":{"total":773094113280,"available":772628156416,"internally_reserved":0,"allocated":28585984,"data_stored":15518402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":437243448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-663619,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-120,"num_read_kb":16,"num_write":-196,"num_write_kb":-1395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.999651"},"pg_stats":[{"pgid":"1.0","version":"265'3813","reported_seq":4900,"reported_epoch":267,"state":"active+clean","last_fresh":"2023-10-18T12:02:21.401052+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:02:21.401052+0000","last_peered":"2023-10-18T12:02:21.401052+0000","last_clean":"2023-10-18T12:02:21.401052+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:02:21.401052+0000","last_undegraded":"2023-10-18T12:02:21.401052+0000","last_fullsized":"2023-10-18T12:02:21.401052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3813,"ondisk_log_size":3813,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576012,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71160,"kb_used_data":3880,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300680,"statfs":{"total":96636764160,"available":96563896320,"internally_reserved":0,"allocated":3973120,"data_stored":2340025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.80800000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.56200000000000006,"15min":0.60099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":1.363},"last":0.621}]},{"osd":1,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.86399999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56100000000000005,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.90000000000000002,"15min":1.298},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71899999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":1.5249999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.81299999999999994},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.503,"15min":0.5},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70699999999999996,"5min":0.86599999999999999,"15min":1.478},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.83599999999999997,"15min":0.874},"last":0.76900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43104,"kb_used_data":3604,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328736,"statfs":{"total":96636764160,"available":96592625664,"internally_reserved":0,"allocated":3690496,"data_stored":2070141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54100000000000004,"15min":0.57799999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56999999999999995,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65700000000000003,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.54100000000000004,"15min":0.58499999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66900000000000004,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.626},{"interface":"front","average":{"1min":0.505,"5min":0.58199999999999996,"15min":0.61399999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73399999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72899999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.55900000000000005,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.55900000000000005,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.59499999999999997,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.58899999999999997,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.60599999999999998,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.70199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.56799999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.872,"15min":0.872},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.75,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834576012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48368,"kb_used_data":3300,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323472,"statfs":{"total":96636764160,"available":96587235328,"internally_reserved":0,"allocated":3379200,"data_stored":1749657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.56499999999999995,"15min":0.59299999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60999999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.58899999999999997,"15min":0.72099999999999997},"min":{"1min":0.308,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67400000000000004,"5min":1.081,"15min":1.1539999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.626,"15min":0.73299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67800000000000005,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56000000000000005,"15min":0.59199999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.57099999999999995,"15min":0.60299999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.628}]},{"osd":3,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.60599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65100000000000002,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65000000000000002,"5min":0.81399999999999995,"15min":1},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.46999999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.59899999999999998,"5min":0.82999999999999996,"15min":1.01},"last":0.499},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.314,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.72399999999999998,"5min":0.872,"15min":0.97999999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.26900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.101},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.58099999999999996,"15min":0.59299999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70999999999999996,"5min":0.83399999999999996,"15min":1.0309999999999999},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834576012,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52548,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319292,"statfs":{"total":96636764160,"available":96582955008,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.60999999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.81499999999999995,"5min":0.872,"15min":1.123},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73299999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.42199999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.755,"5min":1.018,"15min":1.018},"last":0.441}]},{"osd":7,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.88900000000000001,"15min":1.05},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.45300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48388,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323452,"statfs":{"total":96636764160,"available":96587214848,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.34899999999999998,"15min":0.376},"min":{"1min":0.219,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.434,"5min":0.57099999999999995,"15min":0.61599999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.312,"5min":0.371,"15min":0.40000000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41499999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.28199999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.40000000000000002},"min":{"1min":0.26000000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45600000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41099999999999998,"15min":0.43099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.441,"5min":1.899,"15min":1.899},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40500000000000003,"15min":0.44},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.42399999999999999,"15min":0.45400000000000001},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.29899999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.65700000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.755}]},{"osd":5,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61699999999999999,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.522,"15min":1.522},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.65600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.621,"15min":0.65200000000000002},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.66800000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65028,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306812,"statfs":{"total":96636764160,"available":96570175488,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36699999999999999,"15min":0.38400000000000001},"min":{"1min":0.20799999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.54800000000000004,"5min":0.64500000000000002,"15min":0.67200000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.38,"15min":0.40999999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.41199999999999998},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.624},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.44500000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.39400000000000002,"15min":0.41599999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48299999999999998,"5min":0.57499999999999996,"15min":0.64600000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42999999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.50600000000000001,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.54700000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63900000000000001,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92100000000000004,"15min":0.93100000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.67500000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.91300000000000003,"15min":1.087},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60999999999999999,"15min":0.64300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.65000000000000002},"min":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.60199999999999998,"15min":0.64200000000000002},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67400000000000004,"5min":0.879,"15min":0.95399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64100000000000001,"15min":0.66800000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78200000000000003,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.628,"15min":0.66400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79900000000000004,"5min":0.86899999999999999,"15min":1.0609999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.627,"15min":0.66300000000000003},"min":{"1min":0.434,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64996,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306844,"statfs":{"total":96636764160,"available":96570208256,"internally_reserved":0,"allocated":3366912,"data_stored":1742551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.375,"15min":0.41899999999999998},"min":{"1min":0.218,"5min":0.217,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.73899999999999999,"15min":0.85599999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.35699999999999998,"15min":0.36899999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.42099999999999999,"5min":0.622,"15min":0.82599999999999996},"last":0.30499999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38700000000000001,"15min":0.436},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.65800000000000003,"15min":0.81000000000000005},"last":0.25},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.48699999999999999,"5min":0.68999999999999995,"15min":0.877},"last":0.35299999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.435,"15min":0.48899999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47899999999999998,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":0.74399999999999999,"15min":0.91000000000000003},"last":0.29299999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.73399999999999999,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":0.89900000000000002,"15min":1.004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.91500000000000004,"15min":1.05},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.621,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.67400000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.65200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834576011,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61444,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310396,"statfs":{"total":96636764160,"available":96573845504,"internally_reserved":0,"allocated":3989504,"data_stored":2346783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36799999999999999,"15min":0.38800000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.65600000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38400000000000001,"15min":0.41199999999999998},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.27100000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.252,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.42899999999999999},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.377}]},{"osd":3,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.53600000000000003,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.495}]},{"osd":4,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60199999999999998,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90900000000000003,"5min":0.96299999999999997,"15min":0.997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.64100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60699999999999998,"15min":0.64600000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96399999999999997},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.03},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61099999999999999,"15min":0.66000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79200000000000004,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.67100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.63200000000000001,"15min":0.66900000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.997,"15min":0.997},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:02:24.486 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T12:02:24.486 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:02:24.589 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.586+0000 7f96c6d4c640 1 -- 172.21.15.125:0/572314731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c0070920 msgr2=0x7f96c00affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.586+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/572314731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c0070920 0x7f96c00affe0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005fd0 tx=0x7f96b802c240 comp rx=0 tx=0).stop 2023-10-18T12:02:24.590 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.586+0000 7f96c6d4c640 1 -- 172.21.15.125:0/572314731 shutdown_connections 2023-10-18T12:02:24.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.586+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/572314731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c0070920 0x7f96c00affe0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.586+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/572314731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c0141690 0x7f96c00703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.591 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.586+0000 7f96c6d4c640 1 -- 172.21.15.125:0/572314731 >> 172.21.15.125:0/572314731 conn(0x7f96c0069930 msgr2=0x7f96c0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:24.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.586+0000 7f96c6d4c640 1 -- 172.21.15.125:0/572314731 shutdown_connections 2023-10-18T12:02:24.592 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.586+0000 7f96c6d4c640 1 -- 172.21.15.125:0/572314731 wait complete. 2023-10-18T12:02:24.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.587+0000 7f96c6d4c640 1 Processor -- start 2023-10-18T12:02:24.595 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.587+0000 7f96c6d4c640 1 -- start start 2023-10-18T12:02:24.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c6d4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c0070920 0x7f96c013fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c6d4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c013e520 0x7f96c01f0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.596 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c6d4c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96c0141690 0x7f96c01f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c6d4c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01daa60 con 0x7f96c0141690 2023-10-18T12:02:24.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c6d4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01dad40 con 0x7f96c0070920 2023-10-18T12:02:24.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c6d4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01db020 con 0x7f96c013e520 2023-10-18T12:02:24.597 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96c0141690 0x7f96c01f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96c0141690 0x7f96c01f0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41078/0 (socket says 172.21.15.125:41078) 2023-10-18T12:02:24.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 -- 172.21.15.125:0/2942597267 learned_addr learned my addr 172.21.15.125:0/2942597267 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:24.598 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c4ae8640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c0070920 0x7f96c013fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96bffff640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c013e520 0x7f96c01f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 -- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c013e520 msgr2=0x7f96c01f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c013e520 0x7f96c01f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.599 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 -- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c0070920 msgr2=0x7f96c013fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c0070920 0x7f96c013fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 -- 172.21.15.125:0/2942597267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8009900 con 0x7f96c0141690 2023-10-18T12:02:24.600 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96bffff640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c013e520 0x7f96c01f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.588+0000 7f96c52e9640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96c0141690 0x7f96c01f0da0 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009bf0 tx=0x7f96b8035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.589+0000 7f96bd7fa640 1 -- 172.21.15.125:0/2942597267 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8034440 con 0x7f96c0141690 2023-10-18T12:02:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.589+0000 7f96bd7fa640 1 -- 172.21.15.125:0/2942597267 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96b8003e60 con 0x7f96c0141690 2023-10-18T12:02:24.601 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.589+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c01db300 con 0x7f96c0141690 2023-10-18T12:02:24.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.589+0000 7f96bd7fa640 1 -- 172.21.15.125:0/2942597267 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b80044b0 con 0x7f96c0141690 2023-10-18T12:02:24.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.589+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c01404a0 con 0x7f96c0141690 2023-10-18T12:02:24.602 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.590+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c00b1900 con 0x7f96c0141690 2023-10-18T12:02:24.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.590+0000 7f96bd7fa640 1 -- 172.21.15.125:0/2942597267 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f96b800a040 con 0x7f96c0141690 2023-10-18T12:02:24.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.590+0000 7f96bd7fa640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f96a406af60 0x7f96a406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.590+0000 7f96bd7fa640 1 -- 172.21.15.125:0/2942597267 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f96b80ac5a0 con 0x7f96c0141690 2023-10-18T12:02:24.603 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.591+0000 7f96c4ae8640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f96a406af60 0x7f96a406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.591+0000 7f96c4ae8640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f96a406af60 0x7f96a406d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f96b4001640 tx=0x7f96b4009040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:24.604 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.595+0000 7f96bd7fa640 1 -- 172.21.15.125:0/2942597267 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96b8077420 con 0x7f96c0141690 2023-10-18T12:02:24.762 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.760+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96c01daeb0 con 0x7f96a406af60 2023-10-18T12:02:24.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.762+0000 7f96bd7fa640 1 -- 172.21.15.125:0/2942597267 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50982 (secure 0 0 0) 0x7f96c01daeb0 con 0x7f96a406af60 2023-10-18T12:02:24.765 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:24.766 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:02:24.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f96a406af60 msgr2=0x7f96a406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f96a406af60 0x7f96a406d420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f96b4001640 tx=0x7f96b4009040 comp rx=0 tx=0).stop 2023-10-18T12:02:24.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96c0141690 msgr2=0x7f96c01f0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96c0141690 0x7f96c01f0da0 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009bf0 tx=0x7f96b8035e30 comp rx=0 tx=0).stop 2023-10-18T12:02:24.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 shutdown_connections 2023-10-18T12:02:24.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f96a406af60 0x7f96a406d420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96c0141690 0x7f96c01f0da0 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c013e520 0x7f96c01f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 --2- 172.21.15.125:0/2942597267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c0070920 0x7f96c013fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 >> 172.21.15.125:0/2942597267 conn(0x7f96c0069930 msgr2=0x7f96c01df620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:24.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 shutdown_connections 2023-10-18T12:02:24.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.764+0000 7f96c6d4c640 1 -- 172.21.15.125:0/2942597267 wait complete. 2023-10-18T12:02:24.777 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2023-10-18T12:02:22.901251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3813,"ondisk_log_size":3813,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":455036,"kb_used_data":27916,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754519684,"statfs":{"total":773094113280,"available":772628156416,"internally_reserved":0,"allocated":28585984,"data_stored":15518402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":437243448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-663619,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-120,"num_read_kb":16,"num_write":-196,"num_write_kb":-1395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.999651"},"pg_stats":[{"pgid":"1.0","version":"265'3813","reported_seq":4900,"reported_epoch":267,"state":"active+clean","last_fresh":"2023-10-18T12:02:21.401052+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:02:21.401052+0000","last_peered":"2023-10-18T12:02:21.401052+0000","last_clean":"2023-10-18T12:02:21.401052+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:02:21.401052+0000","last_undegraded":"2023-10-18T12:02:21.401052+0000","last_fullsized":"2023-10-18T12:02:21.401052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3813,"ondisk_log_size":3813,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576012,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71160,"kb_used_data":3880,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300680,"statfs":{"total":96636764160,"available":96563896320,"internally_reserved":0,"allocated":3973120,"data_stored":2340025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.59199999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.80800000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.56200000000000006,"15min":0.60099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":1.363},"last":0.621}]},{"osd":1,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55800000000000005,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73799999999999999,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.86399999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.82799999999999996,"15min":0.83999999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56100000000000005,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.90000000000000002,"15min":1.298},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71899999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.29099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66800000000000004,"5min":0.69899999999999995,"15min":1.5249999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.81299999999999994},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.503,"15min":0.5},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57899999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.85299999999999998,"15min":1.4179999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70699999999999996,"5min":0.86599999999999999,"15min":1.478},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.83599999999999997,"15min":0.874},"last":0.76900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43104,"kb_used_data":3604,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328736,"statfs":{"total":96636764160,"available":96592625664,"internally_reserved":0,"allocated":3690496,"data_stored":2070141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54100000000000004,"15min":0.57799999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56999999999999995,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65700000000000003,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.54100000000000004,"15min":0.58499999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66900000000000004,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.626},{"interface":"front","average":{"1min":0.505,"5min":0.58199999999999996,"15min":0.61399999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73399999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72899999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.55900000000000005,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.55900000000000005,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.59499999999999997,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.58899999999999997,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.60599999999999998,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.70199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.56799999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.872,"15min":0.872},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.75,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834576012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48368,"kb_used_data":3300,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323472,"statfs":{"total":96636764160,"available":96587235328,"internally_reserved":0,"allocated":3379200,"data_stored":1749657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.56499999999999995,"15min":0.59299999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60999999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.58899999999999997,"15min":0.72099999999999997},"min":{"1min":0.308,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67400000000000004,"5min":1.081,"15min":1.1539999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.626,"15min":0.73299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67800000000000005,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56000000000000005,"15min":0.59199999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.57099999999999995,"15min":0.60299999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.628}]},{"osd":3,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.60599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65100000000000002,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65000000000000002,"5min":0.81399999999999995,"15min":1},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.46999999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.59899999999999998,"5min":0.82999999999999996,"15min":1.01},"last":0.499},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.314,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.72399999999999998,"5min":0.872,"15min":0.97999999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.26900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.101},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.58099999999999996,"15min":0.59299999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70999999999999996,"5min":0.83399999999999996,"15min":1.0309999999999999},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834576012,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52548,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319292,"statfs":{"total":96636764160,"available":96582955008,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.60999999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.81499999999999995,"5min":0.872,"15min":1.123},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73299999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.628}]},{"osd":5,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.42199999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.755,"5min":1.018,"15min":1.018},"last":0.441}]},{"osd":7,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.88900000000000001,"15min":1.05},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.45300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48388,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323452,"statfs":{"total":96636764160,"available":96587214848,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.34899999999999998,"15min":0.376},"min":{"1min":0.219,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.434,"5min":0.57099999999999995,"15min":0.61599999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.312,"5min":0.371,"15min":0.40000000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41499999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.28199999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.40000000000000002},"min":{"1min":0.26000000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45600000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41099999999999998,"15min":0.43099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.441,"5min":1.899,"15min":1.899},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40500000000000003,"15min":0.44},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.42399999999999999,"15min":0.45400000000000001},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.29899999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.65700000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.755}]},{"osd":5,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61699999999999999,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.522,"15min":1.522},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.65600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.621,"15min":0.65200000000000002},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.66800000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834576012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65028,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306812,"statfs":{"total":96636764160,"available":96570175488,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36699999999999999,"15min":0.38400000000000001},"min":{"1min":0.20799999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.54800000000000004,"5min":0.64500000000000002,"15min":0.67200000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.38,"15min":0.40999999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40200000000000002}]},{"osd":1,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.41199999999999998},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.624},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.44500000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.39400000000000002,"15min":0.41599999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48299999999999998,"5min":0.57499999999999996,"15min":0.64600000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42999999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.50600000000000001,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.54700000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63900000000000001,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92100000000000004,"15min":0.93100000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.67500000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.91300000000000003,"15min":1.087},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60999999999999999,"15min":0.64300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.65000000000000002},"min":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.60199999999999998,"15min":0.64200000000000002},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67400000000000004,"5min":0.879,"15min":0.95399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64100000000000001,"15min":0.66800000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78200000000000003,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.628,"15min":0.66400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79900000000000004,"5min":0.86899999999999999,"15min":1.0609999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.627,"15min":0.66300000000000003},"min":{"1min":0.434,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64996,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306844,"statfs":{"total":96636764160,"available":96570208256,"internally_reserved":0,"allocated":3366912,"data_stored":1742551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.375,"15min":0.41899999999999998},"min":{"1min":0.218,"5min":0.217,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.73899999999999999,"15min":0.85599999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.35699999999999998,"15min":0.36899999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.42099999999999999,"5min":0.622,"15min":0.82599999999999996},"last":0.30499999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38700000000000001,"15min":0.436},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.65800000000000003,"15min":0.81000000000000005},"last":0.25},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.48699999999999999,"5min":0.68999999999999995,"15min":0.877},"last":0.35299999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.435,"15min":0.48899999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47899999999999998,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":0.74399999999999999,"15min":0.91000000000000003},"last":0.29299999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.73399999999999999,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":0.89900000000000002,"15min":1.004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.91500000000000004,"15min":1.05},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.621,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.67400000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.65200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834576011,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61444,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310396,"statfs":{"total":96636764160,"available":96573845504,"internally_reserved":0,"allocated":3989504,"data_stored":2346783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36799999999999999,"15min":0.38800000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.65600000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38400000000000001,"15min":0.41199999999999998},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.27100000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.252,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.42899999999999999},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.377}]},{"osd":3,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.53600000000000003,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.495}]},{"osd":4,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60199999999999998,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90900000000000003,"5min":0.96299999999999997,"15min":0.997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.64100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60699999999999998,"15min":0.64600000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96399999999999997},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.03},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61099999999999999,"15min":0.66000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79200000000000004,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.67100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.63200000000000001,"15min":0.66900000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.997,"15min":0.997},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:02:24.782 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T12:02:24.884 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.882+0000 7f446edb6640 1 -- 172.21.15.125:0/1029185140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f446806b030 msgr2=0x7f446806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.882+0000 7f446edb6640 1 --2- 172.21.15.125:0/1029185140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f446806b030 0x7f446806b410 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 crypto rx=0x7f445c005fd0 tx=0x7f445c02bd90 comp rx=0 tx=0).stop 2023-10-18T12:02:24.885 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.882+0000 7f446edb6640 1 -- 172.21.15.125:0/1029185140 shutdown_connections 2023-10-18T12:02:24.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.882+0000 7f446edb6640 1 --2- 172.21.15.125:0/1029185140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44680c9750 0x7f44680c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.882+0000 7f446edb6640 1 --2- 172.21.15.125:0/1029185140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f446806b030 0x7f446806b410 unknown :-1 s=CLOSED pgs=3124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.882+0000 7f446edb6640 1 -- 172.21.15.125:0/1029185140 >> 172.21.15.125:0/1029185140 conn(0x7f4468069930 msgr2=0x7f4468069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:24.886 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.882+0000 7f446edb6640 1 -- 172.21.15.125:0/1029185140 shutdown_connections 2023-10-18T12:02:24.887 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.882+0000 7f446edb6640 1 -- 172.21.15.125:0/1029185140 wait complete. 2023-10-18T12:02:24.889 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.883+0000 7f446edb6640 1 Processor -- start 2023-10-18T12:02:24.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.883+0000 7f446edb6640 1 -- start start 2023-10-18T12:02:24.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446edb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f446806b030 0x7f44680a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.890 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446edb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44680c9750 0x7f44680a7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446edb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4468159bd0 0x7f44681390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446edb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44680a88d0 con 0x7f4468159bd0 2023-10-18T12:02:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446edb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44680a7a60 con 0x7f446806b030 2023-10-18T12:02:24.891 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446edb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4468139630 con 0x7f44680c9750 2023-10-18T12:02:24.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4468159bd0 0x7f44681390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4468159bd0 0x7f44681390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41108/0 (socket says 172.21.15.125:41108) 2023-10-18T12:02:24.892 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 -- 172.21.15.125:0/1346156990 learned_addr learned my addr 172.21.15.125:0/1346156990 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:24.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f4467fff640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44680c9750 0x7f44680a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446cb52640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f446806b030 0x7f44680a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 -- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44680c9750 msgr2=0x7f44680a7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.893 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44680c9750 0x7f44680a7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 -- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f446806b030 msgr2=0x7f44680a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:24.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f446806b030 0x7f44680a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:24.894 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 -- 172.21.15.125:0/1346156990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c009560 con 0x7f4468159bd0 2023-10-18T12:02:24.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446cb52640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f446806b030 0x7f44680a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:24.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446d353640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4468159bd0 0x7f44681390f0 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f446000eaa0 tx=0x7f446000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:24.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f44657fa640 1 -- 172.21.15.125:0/1346156990 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446000cdb0 con 0x7f4468159bd0 2023-10-18T12:02:24.895 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f44657fa640 1 -- 172.21.15.125:0/1346156990 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4460004b20 con 0x7f4468159bd0 2023-10-18T12:02:24.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44680a7cc0 con 0x7f4468159bd0 2023-10-18T12:02:24.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.884+0000 7f44657fa640 1 -- 172.21.15.125:0/1346156990 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460005020 con 0x7f4468159bd0 2023-10-18T12:02:24.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.885+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44680a8280 con 0x7f4468159bd0 2023-10-18T12:02:24.896 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.886+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44680a8a80 con 0x7f4468159bd0 2023-10-18T12:02:24.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.886+0000 7f44657fa640 1 -- 172.21.15.125:0/1346156990 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4460005180 con 0x7f4468159bd0 2023-10-18T12:02:24.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.886+0000 7f44657fa640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f444c06af90 0x7f444c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:24.897 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.887+0000 7f44657fa640 1 -- 172.21.15.125:0/1346156990 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f446008a340 con 0x7f4468159bd0 2023-10-18T12:02:24.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.887+0000 7f446cb52640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f444c06af90 0x7f444c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:24.898 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.887+0000 7f446cb52640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f444c06af90 0x7f444c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f445c035000 tx=0x7f445c004730 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:24.899 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:24.891+0000 7f44657fa640 1 -- 172.21.15.125:0/1346156990 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44680a8a80 con 0x7f4468159bd0 2023-10-18T12:02:25.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.057+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f446806d410 con 0x7f4468159bd0 2023-10-18T12:02:25.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.058+0000 7f44657fa640 1 -- 172.21.15.125:0/1346156990 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v268) v1 ==== 74+0+14547 (secure 0 0 0) 0x7f44600550d0 con 0x7f4468159bd0 2023-10-18T12:02:25.061 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:25.062 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":268,"fsid":"9708ef69-1ff8-449e-8a32-79d14c57ee5d","created":"2023-10-18T10:24:38.389071+0000","modified":"2023-10-18T12:02:22.075110+0000","last_up_change":"2023-10-18T10:24:50.608437+0000","last_in_change":"2023-10-18T10:24:41.522717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T10:24:52.607581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96f5dda2-779b-493a-a915-3c2243c90fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6801","nonce":3876811193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6803","nonce":3876811193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6807","nonce":3876811193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3876811193},{"type":"v1","addr":"172.21.15.125:6805","nonce":3876811193}]},"public_addr":"172.21.15.125:6801/3876811193","cluster_addr":"172.21.15.125:6803/3876811193","heartbeat_back_addr":"172.21.15.125:6807/3876811193","heartbeat_front_addr":"172.21.15.125:6805/3876811193","state":["exists","up"]},{"osd":1,"uuid":"e2cb000b-290a-481f-a916-397d23e8561b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6817","nonce":817623589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6819","nonce":817623589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6823","nonce":817623589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":817623589},{"type":"v1","addr":"172.21.15.125:6821","nonce":817623589}]},"public_addr":"172.21.15.125:6817/817623589","cluster_addr":"172.21.15.125:6819/817623589","heartbeat_back_addr":"172.21.15.125:6823/817623589","heartbeat_front_addr":"172.21.15.125:6821/817623589","state":["exists","up"]},{"osd":2,"uuid":"df7960de-984d-4b38-95ec-b0ee55f0e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6825","nonce":3658119855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6827","nonce":3658119855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6831","nonce":3658119855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":3658119855},{"type":"v1","addr":"172.21.15.125:6829","nonce":3658119855}]},"public_addr":"172.21.15.125:6825/3658119855","cluster_addr":"172.21.15.125:6827/3658119855","heartbeat_back_addr":"172.21.15.125:6831/3658119855","heartbeat_front_addr":"172.21.15.125:6829/3658119855","state":["exists","up"]},{"osd":3,"uuid":"e37e6670-26f5-4525-9bfe-488052e330f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6809","nonce":3581645100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6811","nonce":3581645100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6815","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6816","nonce":3581645100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":3581645100},{"type":"v1","addr":"172.21.15.125:6813","nonce":3581645100}]},"public_addr":"172.21.15.125:6809/3581645100","cluster_addr":"172.21.15.125:6811/3581645100","heartbeat_back_addr":"172.21.15.125:6816/3581645100","heartbeat_front_addr":"172.21.15.125:6813/3581645100","state":["exists","up"]},{"osd":4,"uuid":"ff2a9e1a-8966-4124-a0ea-d32a9c494f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6801","nonce":3552682439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6803","nonce":3552682439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6807","nonce":3552682439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3552682439},{"type":"v1","addr":"172.21.15.139:6805","nonce":3552682439}]},"public_addr":"172.21.15.139:6801/3552682439","cluster_addr":"172.21.15.139:6803/3552682439","heartbeat_back_addr":"172.21.15.139:6807/3552682439","heartbeat_front_addr":"172.21.15.139:6805/3552682439","state":["exists","up"]},{"osd":5,"uuid":"7edb5f0f-6e32-4285-8345-e719307b0955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6817","nonce":1786373778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6819","nonce":1786373778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6823","nonce":1786373778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1786373778},{"type":"v1","addr":"172.21.15.139:6821","nonce":1786373778}]},"public_addr":"172.21.15.139:6817/1786373778","cluster_addr":"172.21.15.139:6819/1786373778","heartbeat_back_addr":"172.21.15.139:6823/1786373778","heartbeat_front_addr":"172.21.15.139:6821/1786373778","state":["exists","up"]},{"osd":6,"uuid":"54361b74-6765-46c0-b15b-37fdb24f950b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6809","nonce":1564178764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6811","nonce":1564178764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6815","nonce":1564178764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1564178764},{"type":"v1","addr":"172.21.15.139:6813","nonce":1564178764}]},"public_addr":"172.21.15.139:6809/1564178764","cluster_addr":"172.21.15.139:6811/1564178764","heartbeat_back_addr":"172.21.15.139:6815/1564178764","heartbeat_front_addr":"172.21.15.139:6813/1564178764","state":["exists","up"]},{"osd":7,"uuid":"ac3ffb65-f724-44ce-a9cf-d06ce7966b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":260,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6825","nonce":2679488483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6827","nonce":2679488483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6831","nonce":2679488483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2679488483},{"type":"v1","addr":"172.21.15.139:6829","nonce":2679488483}]},"public_addr":"172.21.15.139:6825/2679488483","cluster_addr":"172.21.15.139:6827/2679488483","heartbeat_back_addr":"172.21.15.139:6831/2679488483","heartbeat_front_addr":"172.21.15.139:6829/2679488483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.587181+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.691313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.650502+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.635245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.615217+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.690256+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.684919+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T10:24:46.712577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.125:6835/1922177652":"2023-10-19T12:02:19.070696+0000","172.21.15.139:0/2140935837":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/246588314":"2023-10-19T12:02:16.882516+0000","172.21.15.125:6833/1922177652":"2023-10-19T12:02:19.070696+0000","172.21.15.139:0/1154764603":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/3832700950":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/2488548050":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2767408321":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6832/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1739931468":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/4281362145":"2023-10-19T12:02:16.882516+0000","172.21.15.139:6833/1808248507":"2023-10-19T12:02:16.882516+0000","172.21.15.139:6832/1808248507":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/2511136316":"2023-10-19T12:02:11.466337+0000","172.21.15.139:6833/666436799":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2813907753":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1094096408":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1599845747":"2023-10-19T12:02:16.882516+0000","172.21.15.139:0/1205310989":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/1383817777":"2023-10-19T12:02:11.466337+0000","172.21.15.139:0/2446560793":"2023-10-18T13:02:05.851547+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T12:02:25.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f444c06af90 msgr2=0x7f444c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:25.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f444c06af90 0x7f444c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f445c035000 tx=0x7f445c004730 comp rx=0 tx=0).stop 2023-10-18T12:02:25.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4468159bd0 msgr2=0x7f44681390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:25.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4468159bd0 0x7f44681390f0 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f446000eaa0 tx=0x7f446000ef60 comp rx=0 tx=0).stop 2023-10-18T12:02:25.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 shutdown_connections 2023-10-18T12:02:25.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f444c06af90 0x7f444c06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:25.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4468159bd0 0x7f44681390f0 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:25.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44680c9750 0x7f44680a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:25.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 --2- 172.21.15.125:0/1346156990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f446806b030 0x7f44680a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:25.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 >> 172.21.15.125:0/1346156990 conn(0x7f4468069930 msgr2=0x7f446815bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:25.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 shutdown_connections 2023-10-18T12:02:25.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:25.060+0000 7f446edb6640 1 -- 172.21.15.125:0/1346156990 wait complete. 2023-10-18T12:02:26.075 INFO:tasks.ceph:Scrubbing osd.0 2023-10-18T12:02:26.075 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-18T12:02:26.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.194+0000 7f750eb8a640 1 -- 172.21.15.125:0/231296708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f750806b410 msgr2=0x7f750815c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.194+0000 7f750eb8a640 1 --2- 172.21.15.125:0/231296708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f750806b410 0x7f750815c770 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f74f4005fd0 tx=0x7f74f402bd90 comp rx=0 tx=0).stop 2023-10-18T12:02:26.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.194+0000 7f750eb8a640 1 -- 172.21.15.125:0/231296708 shutdown_connections 2023-10-18T12:02:26.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.194+0000 7f750eb8a640 1 --2- 172.21.15.125:0/231296708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f750815ccb0 0x7f75080c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.194+0000 7f750eb8a640 1 --2- 172.21.15.125:0/231296708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f750806b410 0x7f750815c770 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.194+0000 7f750eb8a640 1 -- 172.21.15.125:0/231296708 >> 172.21.15.125:0/231296708 conn(0x7f7508069930 msgr2=0x7f7508069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:26.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.194+0000 7f750eb8a640 1 -- 172.21.15.125:0/231296708 shutdown_connections 2023-10-18T12:02:26.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.194+0000 7f750eb8a640 1 -- 172.21.15.125:0/231296708 wait complete. 2023-10-18T12:02:26.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.195+0000 7f750eb8a640 1 Processor -- start 2023-10-18T12:02:26.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.195+0000 7f750eb8a640 1 -- start start 2023-10-18T12:02:26.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750eb8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f750806b030 0x7f75081502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750eb8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f750806b410 0x7f7508137b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750eb8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f750815ccb0 0x7f7508138060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750eb8a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75081507e0 con 0x7f750806b030 2023-10-18T12:02:26.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750eb8a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75081509f0 con 0x7f750815ccb0 2023-10-18T12:02:26.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750eb8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7508150cd0 con 0x7f750806b410 2023-10-18T12:02:26.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750c926640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f750806b030 0x7f75081502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750d127640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f750815ccb0 0x7f7508138060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f7507fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f750806b410 0x7f7508137b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750c926640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f750806b030 0x7f75081502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41112/0 (socket says 172.21.15.125:41112) 2023-10-18T12:02:26.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.196+0000 7f750c926640 1 -- 172.21.15.125:0/2033851186 learned_addr learned my addr 172.21.15.125:0/2033851186 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:26.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f750d127640 1 -- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f750806b410 msgr2=0x7f7508137b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f750d127640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f750806b410 0x7f7508137b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f750d127640 1 -- 172.21.15.125:0/2033851186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f750806b030 msgr2=0x7f75081502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f750d127640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f750806b030 0x7f75081502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f750d127640 1 -- 172.21.15.125:0/2033851186 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500009840 con 0x7f750815ccb0 2023-10-18T12:02:26.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f750d127640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f750815ccb0 0x7f7508138060 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f7500005fd0 tx=0x7f750000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500010630 con 0x7f750815ccb0 2023-10-18T12:02:26.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7500010790 con 0x7f750815ccb0 2023-10-18T12:02:26.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500010b80 con 0x7f750815ccb0 2023-10-18T12:02:26.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f4009560 con 0x7f750815ccb0 2023-10-18T12:02:26.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.197+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7508151290 con 0x7f750815ccb0 2023-10-18T12:02:26.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.198+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75080ab500 con 0x7f750815ccb0 2023-10-18T12:02:26.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.199+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7500010ce0 con 0x7f750815ccb0 2023-10-18T12:02:26.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.199+0000 7f75057fa640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f74ec06af90 0x7f74ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.199+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f7500014070 con 0x7f750815ccb0 2023-10-18T12:02:26.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.199+0000 7f75057fa640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7f74ec071970 0x7f74ec073df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.199+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 --> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74ec0744c0 con 0x7f74ec071970 2023-10-18T12:02:26.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.199+0000 7f750c926640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f74ec06af90 0x7f74ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.199+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=268) v2 ==== 24+0+0 (secure 0 0 0) 0x7f750008af20 con 0x7f750815ccb0 2023-10-18T12:02:26.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.199+0000 7f7507fff640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7f74ec071970 0x7f74ec073df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.200+0000 7f7507fff640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7f74ec071970 0x7f74ec073df0 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.214 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.200+0000 7f750c926640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f74ec06af90 0x7f74ec06d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74fc001840 tx=0x7f74fc009040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.215 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.200+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 <== osd.0 v2:172.21.15.125:6800/3876811193 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f74ec0744c0 con 0x7f74ec071970 2023-10-18T12:02:26.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 --> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f750806c850 con 0x7f74ec071970 2023-10-18T12:02:26.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f75057fa640 1 -- 172.21.15.125:0/2033851186 <== osd.0 v2:172.21.15.125:6800/3876811193 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f750806c850 con 0x7f74ec071970 2023-10-18T12:02:26.219 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-10-18T12:02:26.220 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:26.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7f74ec071970 msgr2=0x7f74ec073df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f750eb8a640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.125:6800/3876811193,v1:172.21.15.125:6801/3876811193] conn(0x7f74ec071970 0x7f74ec073df0 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f74ec06af90 msgr2=0x7f74ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f750eb8a640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f74ec06af90 0x7f74ec06d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74fc001840 tx=0x7f74fc009040 comp rx=0 tx=0).stop 2023-10-18T12:02:26.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f750815ccb0 msgr2=0x7f7508138060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f750eb8a640 1 --2- 172.21.15.125:0/2033851186 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f750815ccb0 0x7f7508138060 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f7500005fd0 tx=0x7f750000c6a0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.213+0000 7f750d127640 1 -- 172.21.15.125:0/2033851186 reap_dead start 2023-10-18T12:02:26.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.214+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 shutdown_connections 2023-10-18T12:02:26.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.214+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 >> 172.21.15.125:0/2033851186 conn(0x7f7508069930 msgr2=0x7f750813b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:26.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.214+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 shutdown_connections 2023-10-18T12:02:26.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.214+0000 7f750eb8a640 1 -- 172.21.15.125:0/2033851186 wait complete. 2023-10-18T12:02:26.227 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-18T12:02:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.328+0000 7f5c44856640 1 -- 172.21.15.125:0/906489004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c40141690 msgr2=0x7f5c400703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.331 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.328+0000 7f5c44856640 1 --2- 172.21.15.125:0/906489004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c40141690 0x7f5c400703e0 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7f5c400cbd00 tx=0x7f5c3402bdb0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.328+0000 7f5c44856640 1 -- 172.21.15.125:0/906489004 shutdown_connections 2023-10-18T12:02:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.328+0000 7f5c44856640 1 --2- 172.21.15.125:0/906489004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c40141690 0x7f5c400703e0 unknown :-1 s=CLOSED pgs=3126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.328+0000 7f5c44856640 1 --2- 172.21.15.125:0/906489004 >> v2:172.21.15.139:3300/0 conn(0x7f5c4013e520 0x7f5c4006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.328+0000 7f5c44856640 1 -- 172.21.15.125:0/906489004 >> 172.21.15.125:0/906489004 conn(0x7f5c40069930 msgr2=0x7f5c40069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.328+0000 7f5c44856640 1 -- 172.21.15.125:0/906489004 shutdown_connections 2023-10-18T12:02:26.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.328+0000 7f5c44856640 1 -- 172.21.15.125:0/906489004 wait complete. 2023-10-18T12:02:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.329+0000 7f5c44856640 1 Processor -- start 2023-10-18T12:02:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.329+0000 7f5c44856640 1 -- start start 2023-10-18T12:02:26.336 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c44856640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c4006c0a0 0x7f5c401d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c44856640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c4013e520 0x7f5c401da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c44856640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141690 0x7f5c400ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c44856640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401d7060 con 0x7f5c4013e520 2023-10-18T12:02:26.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c44856640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401d7340 con 0x7f5c4006c0a0 2023-10-18T12:02:26.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c44856640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401d7980 con 0x7f5c40141690 2023-10-18T12:02:26.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c4013e520 0x7f5c401da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c4013e520 0x7f5c401da2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41126/0 (socket says 172.21.15.125:41126) 2023-10-18T12:02:26.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3dd9b640 1 -- 172.21.15.125:0/237953171 learned_addr learned my addr 172.21.15.125:0/237953171 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:26.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3e59c640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c4006c0a0 0x7f5c401d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3ed9d640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141690 0x7f5c400ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3dd9b640 1 -- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141690 msgr2=0x7f5c400ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3dd9b640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141690 0x7f5c400ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3dd9b640 1 -- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c4006c0a0 msgr2=0x7f5c401d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3dd9b640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c4006c0a0 0x7f5c401d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.330+0000 7f5c3dd9b640 1 -- 172.21.15.125:0/237953171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34009560 con 0x7f5c4013e520 2023-10-18T12:02:26.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.331+0000 7f5c3dd9b640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c4013e520 0x7f5c401da2d0 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f5c34000c00 tx=0x7f5c3402db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.331+0000 7f5c2effd640 1 -- 172.21.15.125:0/237953171 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34003800 con 0x7f5c4013e520 2023-10-18T12:02:26.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.331+0000 7f5c2effd640 1 -- 172.21.15.125:0/237953171 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c34003a50 con 0x7f5c4013e520 2023-10-18T12:02:26.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.331+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c401d7c60 con 0x7f5c4013e520 2023-10-18T12:02:26.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.331+0000 7f5c2effd640 1 -- 172.21.15.125:0/237953171 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34003e60 con 0x7f5c4013e520 2023-10-18T12:02:26.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.331+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c400cadb0 con 0x7f5c4013e520 2023-10-18T12:02:26.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.332+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c401d8180 con 0x7f5c4013e520 2023-10-18T12:02:26.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.332+0000 7f5c2effd640 1 -- 172.21.15.125:0/237953171 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c3403e450 con 0x7f5c4013e520 2023-10-18T12:02:26.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.332+0000 7f5c2effd640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5c2006af90 0x7f5c2006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.333+0000 7f5c2effd640 1 -- 172.21.15.125:0/237953171 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f5c340ac3f0 con 0x7f5c4013e520 2023-10-18T12:02:26.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.333+0000 7f5c3e59c640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5c2006af90 0x7f5c2006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.333+0000 7f5c3e59c640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5c2006af90 0x7f5c2006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c38005fd0 tx=0x7f5c38009700 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.337+0000 7f5c2effd640 1 -- 172.21.15.125:0/237953171 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c401d8180 con 0x7f5c4013e520 2023-10-18T12:02:26.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.504+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c401d8180 con 0x7f5c2006af90 2023-10-18T12:02:26.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.505+0000 7f5c2effd640 1 -- 172.21.15.125:0/237953171 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5c401d8180 con 0x7f5c2006af90 2023-10-18T12:02:26.508 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 0 to deep-scrub 2023-10-18T12:02:26.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.506+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5c2006af90 msgr2=0x7f5c2006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.506+0000 7f5c44856640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5c2006af90 0x7f5c2006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c38005fd0 tx=0x7f5c38009700 comp rx=0 tx=0).stop 2023-10-18T12:02:26.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.506+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c4013e520 msgr2=0x7f5c401da2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.506+0000 7f5c44856640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c4013e520 0x7f5c401da2d0 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f5c34000c00 tx=0x7f5c3402db50 comp rx=0 tx=0).stop 2023-10-18T12:02:26.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.506+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 shutdown_connections 2023-10-18T12:02:26.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.506+0000 7f5c44856640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5c2006af90 0x7f5c2006d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.506+0000 7f5c44856640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141690 0x7f5c400ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.506+0000 7f5c44856640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5c4013e520 0x7f5c401da2d0 unknown :-1 s=CLOSED pgs=3127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.507+0000 7f5c44856640 1 --2- 172.21.15.125:0/237953171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c4006c0a0 0x7f5c401d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.507+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 >> 172.21.15.125:0/237953171 conn(0x7f5c40069930 msgr2=0x7f5c400c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:26.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.507+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 shutdown_connections 2023-10-18T12:02:26.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.507+0000 7f5c44856640 1 -- 172.21.15.125:0/237953171 wait complete. 2023-10-18T12:02:26.519 INFO:tasks.ceph:Scrubbing osd.1 2023-10-18T12:02:26.520 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-18T12:02:26.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 -- 172.21.15.125:0/3614876263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f813e520 msgr2=0x7f82f806bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 --2- 172.21.15.125:0/3614876263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f813e520 0x7f82f806bb60 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7f82f80ae6b0 tx=0x7f82f002c140 comp rx=0 tx=0).stop 2023-10-18T12:02:26.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 -- 172.21.15.125:0/3614876263 shutdown_connections 2023-10-18T12:02:26.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 --2- 172.21.15.125:0/3614876263 >> v2:172.21.15.139:3300/0 conn(0x7f82f806c0a0 0x7f82f8070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 --2- 172.21.15.125:0/3614876263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82f8141690 0x7f82f80703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 --2- 172.21.15.125:0/3614876263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f813e520 0x7f82f806bb60 unknown :-1 s=CLOSED pgs=3128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 -- 172.21.15.125:0/3614876263 >> 172.21.15.125:0/3614876263 conn(0x7f82f8069930 msgr2=0x7f82f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:26.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 -- 172.21.15.125:0/3614876263 shutdown_connections 2023-10-18T12:02:26.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.622+0000 7f82fefb6640 1 -- 172.21.15.125:0/3614876263 wait complete. 2023-10-18T12:02:26.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.624+0000 7f82fefb6640 1 Processor -- start 2023-10-18T12:02:26.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.624+0000 7f82fefb6640 1 -- start start 2023-10-18T12:02:26.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.624+0000 7f82fefb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82f806c0a0 0x7f82f80aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.624+0000 7f82fefb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f813e520 0x7f82f80ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.624+0000 7f82fefb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f8141690 0x7f82f813af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.624+0000 7f82fefb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81e3160 con 0x7f82f8141690 2023-10-18T12:02:26.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.624+0000 7f82fefb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80ab830 con 0x7f82f813e520 2023-10-18T12:02:26.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.624+0000 7f82fefb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80af4c0 con 0x7f82f806c0a0 2023-10-18T12:02:26.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f8141690 0x7f82f813af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f8141690 0x7f82f813af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41140/0 (socket says 172.21.15.125:41140) 2023-10-18T12:02:26.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 -- 172.21.15.125:0/783609171 learned_addr learned my addr 172.21.15.125:0/783609171 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:26.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82f7fff640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f813e520 0x7f82f80ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fcd52640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82f806c0a0 0x7f82f80aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 -- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82f806c0a0 msgr2=0x7f82f80aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f82f806c0a0 0x7f82f80aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 -- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f813e520 msgr2=0x7f82f80ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82f813e520 0x7f82f80ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 -- 172.21.15.125:0/783609171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0009900 con 0x7f82f8141690 2023-10-18T12:02:26.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fd553640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f8141690 0x7f82f813af90 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f82e800e970 tx=0x7f82e800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e80096a0 con 0x7f82f8141690 2023-10-18T12:02:26.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82e800ccc0 con 0x7f82f8141690 2023-10-18T12:02:26.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8005280 con 0x7f82f8141690 2023-10-18T12:02:26.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f81e2b10 con 0x7f82f8141690 2023-10-18T12:02:26.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.625+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f813b4d0 con 0x7f82f8141690 2023-10-18T12:02:26.639 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.626+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82f80690e0 con 0x7f82f8141690 2023-10-18T12:02:26.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.627+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f82e80053e0 con 0x7f82f8141690 2023-10-18T12:02:26.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.627+0000 7f82f57fa640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f82e406af60 0x7f82e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.640 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.627+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f82e8014070 con 0x7f82f8141690 2023-10-18T12:02:26.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.627+0000 7f82fcd52640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f82e406af60 0x7f82e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.627+0000 7f82f57fa640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7f82e4071940 0x7f82e4073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.641 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.627+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 --> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82e4074490 con 0x7f82e4071940 2023-10-18T12:02:26.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.627+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=268) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82e808c7e0 con 0x7f82f8141690 2023-10-18T12:02:26.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.627+0000 7f82f7fff640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7f82e4071940 0x7f82e4073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.642 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.628+0000 7f82f7fff640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7f82e4071940 0x7f82e4073dc0 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.628+0000 7f82fcd52640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f82e406af60 0x7f82e406d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f82f0036000 tx=0x7f82f0004ab0 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.643 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.628+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 <== osd.1 v2:172.21.15.125:6814/817623589 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f82e4074490 con 0x7f82e4071940 2023-10-18T12:02:26.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 --> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f82f8070c40 con 0x7f82e4071940 2023-10-18T12:02:26.644 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82f57fa640 1 -- 172.21.15.125:0/783609171 <== osd.1 v2:172.21.15.125:6814/817623589 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f82f8070c40 con 0x7f82e4071940 2023-10-18T12:02:26.645 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:26.645 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-10-18T12:02:26.646 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:26.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7f82e4071940 msgr2=0x7f82e4073dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.125:6814/817623589,v1:172.21.15.125:6817/817623589] conn(0x7f82e4071940 0x7f82e4073dc0 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f82e406af60 msgr2=0x7f82e406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.647 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f82e406af60 0x7f82e406d420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f82f0036000 tx=0x7f82f0004ab0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f8141690 msgr2=0x7f82f813af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 --2- 172.21.15.125:0/783609171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f82f8141690 0x7f82f813af90 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f82e800e970 tx=0x7f82e800ee30 comp rx=0 tx=0).stop 2023-10-18T12:02:26.648 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fd553640 1 -- 172.21.15.125:0/783609171 reap_dead start 2023-10-18T12:02:26.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 shutdown_connections 2023-10-18T12:02:26.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 >> 172.21.15.125:0/783609171 conn(0x7f82f8069930 msgr2=0x7f82f81e0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:26.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 shutdown_connections 2023-10-18T12:02:26.649 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.641+0000 7f82fefb6640 1 -- 172.21.15.125:0/783609171 wait complete. 2023-10-18T12:02:26.654 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-18T12:02:26.758 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.756+0000 7f073bb6c640 1 -- 172.21.15.125:0/867730548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 msgr2=0x7f073406b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.756+0000 7f073bb6c640 1 --2- 172.21.15.125:0/867730548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 0x7f073406b410 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7f0730005fd0 tx=0x7f073002bd90 comp rx=0 tx=0).stop 2023-10-18T12:02:26.759 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.756+0000 7f073bb6c640 1 -- 172.21.15.125:0/867730548 shutdown_connections 2023-10-18T12:02:26.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.756+0000 7f073bb6c640 1 --2- 172.21.15.125:0/867730548 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07340c9750 0x7f07340c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.756+0000 7f073bb6c640 1 --2- 172.21.15.125:0/867730548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 0x7f073406b410 unknown :-1 s=CLOSED pgs=3130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.760 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.756+0000 7f073bb6c640 1 -- 172.21.15.125:0/867730548 >> 172.21.15.125:0/867730548 conn(0x7f0734069930 msgr2=0x7f0734069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:26.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.756+0000 7f073bb6c640 1 -- 172.21.15.125:0/867730548 shutdown_connections 2023-10-18T12:02:26.761 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.756+0000 7f073bb6c640 1 -- 172.21.15.125:0/867730548 wait complete. 2023-10-18T12:02:26.763 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.757+0000 7f073bb6c640 1 Processor -- start 2023-10-18T12:02:26.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.757+0000 7f073bb6c640 1 -- start start 2023-10-18T12:02:26.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.757+0000 7f073bb6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 0x7f073412d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.764 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f073bb6c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07340c9750 0x7f07340b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f073bb6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0734159bd0 0x7f07341eec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f073bb6c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073412a920 con 0x7f073406b030 2023-10-18T12:02:26.765 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f073bb6c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0734129d10 con 0x7f07340c9750 2023-10-18T12:02:26.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f073bb6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0734129ff0 con 0x7f0734159bd0 2023-10-18T12:02:26.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 0x7f073412d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.766 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 0x7f073412d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41154/0 (socket says 172.21.15.125:41154) 2023-10-18T12:02:26.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 -- 172.21.15.125:0/1459224332 learned_addr learned my addr 172.21.15.125:0/1459224332 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:26.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739107640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07340c9750 0x7f07340b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f073a109640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0734159bd0 0x7f07341eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.767 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 -- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0734159bd0 msgr2=0x7f07341eec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0734159bd0 0x7f07341eec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 -- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07340c9750 msgr2=0x7f07340b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.768 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07340c9750 0x7f07340b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 -- 172.21.15.125:0/1459224332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0730009560 con 0x7f073406b030 2023-10-18T12:02:26.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f0739908640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 0x7f073412d0b0 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7f0730036040 tx=0x7f073002da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f072a7fc640 1 -- 172.21.15.125:0/1459224332 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073002de60 con 0x7f073406b030 2023-10-18T12:02:26.769 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f072a7fc640 1 -- 172.21.15.125:0/1459224332 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07300036e0 con 0x7f073406b030 2023-10-18T12:02:26.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073412a2d0 con 0x7f073406b030 2023-10-18T12:02:26.770 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f072a7fc640 1 -- 172.21.15.125:0/1459224332 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730003af0 con 0x7f073406b030 2023-10-18T12:02:26.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.758+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07340b1430 con 0x7f073406b030 2023-10-18T12:02:26.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.759+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073406c850 con 0x7f073406b030 2023-10-18T12:02:26.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.760+0000 7f072a7fc640 1 -- 172.21.15.125:0/1459224332 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0730003c50 con 0x7f073406b030 2023-10-18T12:02:26.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.760+0000 7f072a7fc640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f071806af60 0x7f071806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:26.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.760+0000 7f072a7fc640 1 -- 172.21.15.125:0/1459224332 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f07300ac270 con 0x7f073406b030 2023-10-18T12:02:26.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.760+0000 7f0739107640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f071806af60 0x7f071806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:26.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.761+0000 7f0739107640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f071806af60 0x7f071806d420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f07341302b0 tx=0x7f0724009040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:26.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.764+0000 7f072a7fc640 1 -- 172.21.15.125:0/1459224332 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07300771a0 con 0x7f073406b030 2023-10-18T12:02:26.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.929+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f073415a060 con 0x7f071806af60 2023-10-18T12:02:26.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.929+0000 7f072a7fc640 1 -- 172.21.15.125:0/1459224332 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0724001c90 con 0x7f071806af60 2023-10-18T12:02:26.933 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 1 to deep-scrub 2023-10-18T12:02:26.934 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f071806af60 msgr2=0x7f071806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f071806af60 0x7f071806d420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f07341302b0 tx=0x7f0724009040 comp rx=0 tx=0).stop 2023-10-18T12:02:26.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 msgr2=0x7f073412d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:26.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 0x7f073412d0b0 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7f0730036040 tx=0x7f073002da60 comp rx=0 tx=0).stop 2023-10-18T12:02:26.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 shutdown_connections 2023-10-18T12:02:26.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f071806af60 0x7f071806d420 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07340c9750 0x7f07340b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0734159bd0 0x7f07341eec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 --2- 172.21.15.125:0/1459224332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f073406b030 0x7f073412d0b0 unknown :-1 s=CLOSED pgs=3131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:26.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 >> 172.21.15.125:0/1459224332 conn(0x7f0734069930 msgr2=0x7f073414fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:26.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 shutdown_connections 2023-10-18T12:02:26.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:26.931+0000 7f073bb6c640 1 -- 172.21.15.125:0/1459224332 wait complete. 2023-10-18T12:02:26.944 INFO:tasks.ceph:Scrubbing osd.2 2023-10-18T12:02:26.945 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-18T12:02:27.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.048+0000 7f8a478af640 1 -- 172.21.15.125:0/1654747228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a4015b2a0 msgr2=0x7f8a400d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.048+0000 7f8a478af640 1 --2- 172.21.15.125:0/1654747228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a4015b2a0 0x7f8a400d1910 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005fd0 tx=0x7f8a3002bd90 comp rx=0 tx=0).stop 2023-10-18T12:02:27.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.048+0000 7f8a478af640 1 -- 172.21.15.125:0/1654747228 shutdown_connections 2023-10-18T12:02:27.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.048+0000 7f8a478af640 1 --2- 172.21.15.125:0/1654747228 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a400ceb40 0x7f8a400b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.048+0000 7f8a478af640 1 --2- 172.21.15.125:0/1654747228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a4015b2a0 0x7f8a400d1910 unknown :-1 s=CLOSED pgs=3132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.048+0000 7f8a478af640 1 -- 172.21.15.125:0/1654747228 >> 172.21.15.125:0/1654747228 conn(0x7f8a400d0d60 msgr2=0x7f8a400d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.048+0000 7f8a478af640 1 -- 172.21.15.125:0/1654747228 shutdown_connections 2023-10-18T12:02:27.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.048+0000 7f8a478af640 1 -- 172.21.15.125:0/1654747228 wait complete. 2023-10-18T12:02:27.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.049+0000 7f8a478af640 1 Processor -- start 2023-10-18T12:02:27.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.049+0000 7f8a478af640 1 -- start start 2023-10-18T12:02:27.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a478af640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a400ceb40 0x7f8a4014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a478af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a400cf340 0x7f8a400c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a478af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4015b2a0 0x7f8a400b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a478af640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a401f1580 con 0x7f8a400ceb40 2023-10-18T12:02:27.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a478af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a400c8d30 con 0x7f8a4015b2a0 2023-10-18T12:02:27.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a478af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a401f50a0 con 0x7f8a400cf340 2023-10-18T12:02:27.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a4564b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a400ceb40 0x7f8a4014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a4564b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a400ceb40 0x7f8a4014fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41166/0 (socket says 172.21.15.125:41166) 2023-10-18T12:02:27.059 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a4564b640 1 -- 172.21.15.125:0/112999666 learned_addr learned my addr 172.21.15.125:0/112999666 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:27.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a44e4a640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a400cf340 0x7f8a400c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a45e4c640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4015b2a0 0x7f8a400b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.060 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a4564b640 1 -- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a400cf340 msgr2=0x7f8a400c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a4564b640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8a400cf340 0x7f8a400c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a4564b640 1 -- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4015b2a0 msgr2=0x7f8a400b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a4564b640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a4015b2a0 0x7f8a400b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.050+0000 7f8a4564b640 1 -- 172.21.15.125:0/112999666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a30009560 con 0x7f8a400ceb40 2023-10-18T12:02:27.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.051+0000 7f8a4564b640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a400ceb40 0x7f8a4014fb70 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00c400 tx=0x7f8a3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.051+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0052d0 con 0x7f8a400ceb40 2023-10-18T12:02:27.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.051+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a3c007020 con 0x7f8a400ceb40 2023-10-18T12:02:27.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.051+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a400b0d50 con 0x7f8a400ceb40 2023-10-18T12:02:27.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.051+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c007470 con 0x7f8a400ceb40 2023-10-18T12:02:27.063 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.051+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a401f5660 con 0x7f8a400ceb40 2023-10-18T12:02:27.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.052+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a400690e0 con 0x7f8a400ceb40 2023-10-18T12:02:27.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.052+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8a3c0067d0 con 0x7f8a400ceb40 2023-10-18T12:02:27.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a35ffb640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f8a2406af60 0x7f8a2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.064 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f8a3c088e90 con 0x7f8a400ceb40 2023-10-18T12:02:27.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a44e4a640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f8a2406af60 0x7f8a2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a35ffb640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f8a24071940 0x7f8a24073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.065 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 --> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a24074490 con 0x7f8a24071940 2023-10-18T12:02:27.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=268) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a3c08b550 con 0x7f8a400ceb40 2023-10-18T12:02:27.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a45e4c640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f8a24071940 0x7f8a24073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.066 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a45e4c640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f8a24071940 0x7f8a24073dc0 crc :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.053+0000 7f8a44e4a640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f8a2406af60 0x7f8a2406d420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a30035000 tx=0x7f8a30004730 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.067 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.054+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 <== osd.2 v2:172.21.15.125:6824/3658119855 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8a24074490 con 0x7f8a24071940 2023-10-18T12:02:27.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.066+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 --> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8a40155240 con 0x7f8a24071940 2023-10-18T12:02:27.069 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.066+0000 7f8a35ffb640 1 -- 172.21.15.125:0/112999666 <== osd.2 v2:172.21.15.125:6824/3658119855 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8a40155240 con 0x7f8a24071940 2023-10-18T12:02:27.070 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:27.070 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-10-18T12:02:27.070 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:27.071 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f8a24071940 msgr2=0x7f8a24073dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.125:6824/3658119855,v1:172.21.15.125:6825/3658119855] conn(0x7f8a24071940 0x7f8a24073dc0 crc :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f8a2406af60 msgr2=0x7f8a2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.072 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f8a2406af60 0x7f8a2406d420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a30035000 tx=0x7f8a30004730 comp rx=0 tx=0).stop 2023-10-18T12:02:27.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a400ceb40 msgr2=0x7f8a4014fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 --2- 172.21.15.125:0/112999666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8a400ceb40 0x7f8a4014fb70 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00c400 tx=0x7f8a3c00c8c0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.073 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a45e4c640 1 -- 172.21.15.125:0/112999666 reap_dead start 2023-10-18T12:02:27.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 shutdown_connections 2023-10-18T12:02:27.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 >> 172.21.15.125:0/112999666 conn(0x7f8a400d0d60 msgr2=0x7f8a400ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 shutdown_connections 2023-10-18T12:02:27.074 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.067+0000 7f8a478af640 1 -- 172.21.15.125:0/112999666 wait complete. 2023-10-18T12:02:27.080 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-18T12:02:27.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.180+0000 7fcbf0807640 1 -- 172.21.15.125:0/4141715084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec06b030 msgr2=0x7fcbec06b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.184 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.180+0000 7fcbf0807640 1 --2- 172.21.15.125:0/4141715084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec06b030 0x7fcbec06b410 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0c9410 tx=0x7fcbdc02bce0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.181+0000 7fcbf0807640 1 -- 172.21.15.125:0/4141715084 shutdown_connections 2023-10-18T12:02:27.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.181+0000 7fcbf0807640 1 --2- 172.21.15.125:0/4141715084 >> v2:172.21.15.139:3300/0 conn(0x7fcbec0c9750 0x7fcbec0c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.181+0000 7fcbf0807640 1 --2- 172.21.15.125:0/4141715084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec06b030 0x7fcbec06b410 unknown :-1 s=CLOSED pgs=3135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.185 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.181+0000 7fcbf0807640 1 -- 172.21.15.125:0/4141715084 >> 172.21.15.125:0/4141715084 conn(0x7fcbec069930 msgr2=0x7fcbec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.181+0000 7fcbf0807640 1 -- 172.21.15.125:0/4141715084 shutdown_connections 2023-10-18T12:02:27.186 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.181+0000 7fcbf0807640 1 -- 172.21.15.125:0/4141715084 wait complete. 2023-10-18T12:02:27.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.182+0000 7fcbf0807640 1 Processor -- start 2023-10-18T12:02:27.189 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.182+0000 7fcbf0807640 1 -- start start 2023-10-18T12:02:27.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbf0807640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbec06b030 0x7fcbec0a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.190 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbf0807640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbec0c9750 0x7fcbec0aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbf0807640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec159bd0 0x7fcbec12d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbf0807640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec14e420 con 0x7fcbec159bd0 2023-10-18T12:02:27.191 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbf0807640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec1501d0 con 0x7fcbec0c9750 2023-10-18T12:02:27.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbf0807640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec1504b0 con 0x7fcbec06b030 2023-10-18T12:02:27.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec159bd0 0x7fcbec12d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec159bd0 0x7fcbec12d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41192/0 (socket says 172.21.15.125:41192) 2023-10-18T12:02:27.192 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 -- 172.21.15.125:0/898904965 learned_addr learned my addr 172.21.15.125:0/898904965 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:27.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbe3fff640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbec0c9750 0x7fcbec0aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbea59c640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbec06b030 0x7fcbec0a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.193 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 -- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbec06b030 msgr2=0x7fcbec0a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbec06b030 0x7fcbec0a8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 -- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbec0c9750 msgr2=0x7fcbec0aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbec0c9750 0x7fcbec0aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.194 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 -- 172.21.15.125:0/898904965 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc009490 con 0x7fcbec159bd0 2023-10-18T12:02:27.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbea59c640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbec06b030 0x7fcbec0a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:27.195 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbe3fff640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbec0c9750 0x7fcbec0aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:27.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbead9d640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec159bd0 0x7fcbec12d070 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400b970 tx=0x7fcbe400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbe2ffd640 1 -- 172.21.15.125:0/898904965 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400cb40 con 0x7fcbec159bd0 2023-10-18T12:02:27.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbe2ffd640 1 -- 172.21.15.125:0/898904965 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcbe400cdb0 con 0x7fcbec159bd0 2023-10-18T12:02:27.196 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbe2ffd640 1 -- 172.21.15.125:0/898904965 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40146a0 con 0x7fcbec159bd0 2023-10-18T12:02:27.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec150790 con 0x7fcbec159bd0 2023-10-18T12:02:27.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.183+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec150d50 con 0x7fcbec159bd0 2023-10-18T12:02:27.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.184+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbec067930 con 0x7fcbec159bd0 2023-10-18T12:02:27.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.185+0000 7fcbe2ffd640 1 -- 172.21.15.125:0/898904965 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcbe401d3b0 con 0x7fcbec159bd0 2023-10-18T12:02:27.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.185+0000 7fcbe2ffd640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fcbcc06af60 0x7fcbcc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.185+0000 7fcbe2ffd640 1 -- 172.21.15.125:0/898904965 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fcbe4089b90 con 0x7fcbec159bd0 2023-10-18T12:02:27.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.185+0000 7fcbea59c640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fcbcc06af60 0x7fcbcc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.186+0000 7fcbea59c640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fcbcc06af60 0x7fcbcc06d420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc036000 tx=0x7fcbdc004730 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.189+0000 7fcbe2ffd640 1 -- 172.21.15.125:0/898904965 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbe40548f0 con 0x7fcbec159bd0 2023-10-18T12:02:27.259 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:02:27.262 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:02:27.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.361+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbec15ce80 con 0x7fcbcc06af60 2023-10-18T12:02:27.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.362+0000 7fcbe2ffd640 1 -- 172.21.15.125:0/898904965 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcbdc02c840 con 0x7fcbcc06af60 2023-10-18T12:02:27.365 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 2 to deep-scrub 2023-10-18T12:02:27.366 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fcbcc06af60 msgr2=0x7fcbcc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fcbcc06af60 0x7fcbcc06d420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc036000 tx=0x7fcbdc004730 comp rx=0 tx=0).stop 2023-10-18T12:02:27.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec159bd0 msgr2=0x7fcbec12d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.367 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec159bd0 0x7fcbec12d070 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400b970 tx=0x7fcbe400be30 comp rx=0 tx=0).stop 2023-10-18T12:02:27.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 shutdown_connections 2023-10-18T12:02:27.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fcbcc06af60 0x7fcbcc06d420 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.368 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbec0c9750 0x7fcbec0aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcbec159bd0 0x7fcbec12d070 unknown :-1 s=CLOSED pgs=3136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 --2- 172.21.15.125:0/898904965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbec06b030 0x7fcbec0a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.369 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 >> 172.21.15.125:0/898904965 conn(0x7fcbec069930 msgr2=0x7fcbec0ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 shutdown_connections 2023-10-18T12:02:27.370 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.364+0000 7fcbf0807640 1 -- 172.21.15.125:0/898904965 wait complete. 2023-10-18T12:02:27.377 INFO:tasks.ceph:Scrubbing osd.3 2023-10-18T12:02:27.378 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-18T12:02:27.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.483+0000 7f85709b6640 1 -- 172.21.15.125:0/693461988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f856c15b2a0 msgr2=0x7f856c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.483+0000 7f85709b6640 1 --2- 172.21.15.125:0/693461988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f856c15b2a0 0x7f856c0cf720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8560009830 tx=0x7f856002bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.483+0000 7f85709b6640 1 -- 172.21.15.125:0/693461988 shutdown_connections 2023-10-18T12:02:27.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.483+0000 7f85709b6640 1 --2- 172.21.15.125:0/693461988 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f856c15b2a0 0x7f856c0cf720 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.483+0000 7f85709b6640 1 --2- 172.21.15.125:0/693461988 >> v2:172.21.15.139:3300/0 conn(0x7f856c0cf340 0x7f856c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.483+0000 7f85709b6640 1 -- 172.21.15.125:0/693461988 >> 172.21.15.125:0/693461988 conn(0x7f856c0d0d60 msgr2=0x7f856c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.483+0000 7f85709b6640 1 -- 172.21.15.125:0/693461988 shutdown_connections 2023-10-18T12:02:27.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.483+0000 7f85709b6640 1 -- 172.21.15.125:0/693461988 wait complete. 2023-10-18T12:02:27.491 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f85709b6640 1 Processor -- start 2023-10-18T12:02:27.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f85709b6640 1 -- start start 2023-10-18T12:02:27.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f85709b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f856c0ceb40 0x7f856c159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.492 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f85709b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f856c0cf340 0x7f856c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f85709b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f856c15b2a0 0x7f856c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f85709b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c159950 con 0x7f856c0cf340 2023-10-18T12:02:27.493 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f85709b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c09e6d0 con 0x7f856c0ceb40 2023-10-18T12:02:27.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f85709b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856c0b3980 con 0x7f856c15b2a0 2023-10-18T12:02:27.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f8569d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f856c0cf340 0x7f856c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.494 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f8569d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f856c0cf340 0x7f856c15df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37572/0 (socket says 172.21.15.125:37572) 2023-10-18T12:02:27.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.485+0000 7f8569d9b640 1 -- 172.21.15.125:0/776396142 learned_addr learned my addr 172.21.15.125:0/776396142 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:27.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f856a59c640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f856c0ceb40 0x7f856c159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f856ad9d640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f856c15b2a0 0x7f856c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.495 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f8569d9b640 1 -- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f856c15b2a0 msgr2=0x7f856c0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f8569d9b640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f856c15b2a0 0x7f856c0b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f8569d9b640 1 -- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f856c0ceb40 msgr2=0x7f856c159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f8569d9b640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f856c0ceb40 0x7f856c159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.496 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f8569d9b640 1 -- 172.21.15.125:0/776396142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560009490 con 0x7f856c0cf340 2023-10-18T12:02:27.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f8569d9b640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f856c0cf340 0x7f856c15df20 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7f8560000c00 tx=0x7f8560003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560003e20 con 0x7f856c0cf340 2023-10-18T12:02:27.497 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8560034440 con 0x7f856c0cf340 2023-10-18T12:02:27.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560034850 con 0x7f856c0cf340 2023-10-18T12:02:27.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856c0b3c60 con 0x7f856c0cf340 2023-10-18T12:02:27.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.486+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856c0b4220 con 0x7f856c0cf340 2023-10-18T12:02:27.498 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.487+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f856c0cc170 con 0x7f856c0cf340 2023-10-18T12:02:27.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.488+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f85600349b0 con 0x7f856c0cf340 2023-10-18T12:02:27.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.488+0000 7f855affd640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f854c06af60 0x7f854c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.488+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f85600abfe0 con 0x7f856c0cf340 2023-10-18T12:02:27.499 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.488+0000 7f855affd640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f854c071940 0x7f854c073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.488+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 --> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f854c074490 con 0x7f854c071940 2023-10-18T12:02:27.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.488+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=268) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85600ae6a0 con 0x7f856c0cf340 2023-10-18T12:02:27.500 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.488+0000 7f856a59c640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f854c06af60 0x7f854c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.488+0000 7f856ad9d640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f854c071940 0x7f854c073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.489+0000 7f856a59c640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f854c06af60 0x7f854c06d420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f856c09dc40 tx=0x7f855c009500 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.501 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.489+0000 7f856ad9d640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f854c071940 0x7f854c073dc0 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.502 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.489+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 <== osd.3 v2:172.21.15.125:6808/3581645100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f854c074490 con 0x7f854c071940 2023-10-18T12:02:27.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 --> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f856c0690e0 con 0x7f854c071940 2023-10-18T12:02:27.504 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f855affd640 1 -- 172.21.15.125:0/776396142 <== osd.3 v2:172.21.15.125:6808/3581645100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f856c0690e0 con 0x7f854c071940 2023-10-18T12:02:27.505 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:27.505 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-10-18T12:02:27.506 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:27.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f854c071940 msgr2=0x7f854c073dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.125:6808/3581645100,v1:172.21.15.125:6809/3581645100] conn(0x7f854c071940 0x7f854c073dc0 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.507 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f854c06af60 msgr2=0x7f854c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f854c06af60 0x7f854c06d420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f856c09dc40 tx=0x7f855c009500 comp rx=0 tx=0).stop 2023-10-18T12:02:27.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f856c0cf340 msgr2=0x7f856c15df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 --2- 172.21.15.125:0/776396142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f856c0cf340 0x7f856c15df20 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7f8560000c00 tx=0x7f8560003820 comp rx=0 tx=0).stop 2023-10-18T12:02:27.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f856ad9d640 1 -- 172.21.15.125:0/776396142 reap_dead start 2023-10-18T12:02:27.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 shutdown_connections 2023-10-18T12:02:27.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 >> 172.21.15.125:0/776396142 conn(0x7f856c0d0d60 msgr2=0x7f856c0aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 shutdown_connections 2023-10-18T12:02:27.510 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.502+0000 7f85709b6640 1 -- 172.21.15.125:0/776396142 wait complete. 2023-10-18T12:02:27.515 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-18T12:02:27.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 -- 172.21.15.125:0/2751416920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40cb980 msgr2=0x7fbff413db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.618 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 --2- 172.21.15.125:0/2751416920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40cb980 0x7fbff413db80 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7fbff406fa40 tx=0x7fbff002c250 comp rx=0 tx=0).stop 2023-10-18T12:02:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 -- 172.21.15.125:0/2751416920 shutdown_connections 2023-10-18T12:02:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 --2- 172.21.15.125:0/2751416920 >> v2:172.21.15.139:3300/0 conn(0x7fbff40c96b0 0x7fbff40c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.619 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 --2- 172.21.15.125:0/2751416920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff413d350 0x7fbff40cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 --2- 172.21.15.125:0/2751416920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40cb980 0x7fbff413db80 unknown :-1 s=CLOSED pgs=3139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 -- 172.21.15.125:0/2751416920 >> 172.21.15.125:0/2751416920 conn(0x7fbff4069930 msgr2=0x7fbff4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 -- 172.21.15.125:0/2751416920 shutdown_connections 2023-10-18T12:02:27.620 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.615+0000 7fbffc365640 1 -- 172.21.15.125:0/2751416920 wait complete. 2023-10-18T12:02:27.623 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.616+0000 7fbffc365640 1 Processor -- start 2023-10-18T12:02:27.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.616+0000 7fbffc365640 1 -- start start 2023-10-18T12:02:27.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffc365640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40c96b0 0x7fbff4150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffc365640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff40cb980 0x7fbff4154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffc365640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff41f1610 0x7fbff41f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffc365640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4152180 con 0x7fbff40c96b0 2023-10-18T12:02:27.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffc365640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4151500 con 0x7fbff41f1610 2023-10-18T12:02:27.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffc365640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff41517e0 con 0x7fbff40cb980 2023-10-18T12:02:27.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40c96b0 0x7fbff4150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40c96b0 0x7fbff4150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37602/0 (socket says 172.21.15.125:37602) 2023-10-18T12:02:27.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 -- 172.21.15.125:0/3386710130 learned_addr learned my addr 172.21.15.125:0/3386710130 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:27.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 -- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff40cb980 msgr2=0x7fbff4154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff40cb980 0x7fbff4154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 -- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff41f1610 msgr2=0x7fbff41f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa902640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff41f1610 0x7fbff41f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff41f1610 0x7fbff41f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 -- 172.21.15.125:0/3386710130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0009900 con 0x7fbff40c96b0 2023-10-18T12:02:27.628 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.617+0000 7fbffa101640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40c96b0 0x7fbff4150e00 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000f80 tx=0x7fbff0035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.618+0000 7fbfeaffd640 1 -- 172.21.15.125:0/3386710130 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0034440 con 0x7fbff40c96b0 2023-10-18T12:02:27.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.618+0000 7fbfeaffd640 1 -- 172.21.15.125:0/3386710130 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbff0003e60 con 0x7fbff40c96b0 2023-10-18T12:02:27.629 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.618+0000 7fbfeaffd640 1 -- 172.21.15.125:0/3386710130 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff00044b0 con 0x7fbff40c96b0 2023-10-18T12:02:27.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.618+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff4151ac0 con 0x7fbff40c96b0 2023-10-18T12:02:27.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.618+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff4154f80 con 0x7fbff40c96b0 2023-10-18T12:02:27.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.619+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff4155110 con 0x7fbff40c96b0 2023-10-18T12:02:27.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.619+0000 7fbfeaffd640 1 -- 172.21.15.125:0/3386710130 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbff000a040 con 0x7fbff40c96b0 2023-10-18T12:02:27.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.619+0000 7fbfeaffd640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fbfd806af60 0x7fbfd806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.620+0000 7fbfeaffd640 1 -- 172.21.15.125:0/3386710130 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fbff00ad630 con 0x7fbff40c96b0 2023-10-18T12:02:27.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.620+0000 7fbff9900640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fbfd806af60 0x7fbfd806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.620+0000 7fbff9900640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fbfd806af60 0x7fbfd806d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40058c0 tx=0x7fbfe4005850 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.624+0000 7fbfeaffd640 1 -- 172.21.15.125:0/3386710130 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff00784b0 con 0x7fbff40c96b0 2023-10-18T12:02:27.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.791+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbff40ca060 con 0x7fbfd806af60 2023-10-18T12:02:27.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.792+0000 7fbfeaffd640 1 -- 172.21.15.125:0/3386710130 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbfe40096b0 con 0x7fbfd806af60 2023-10-18T12:02:27.795 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 3 to deep-scrub 2023-10-18T12:02:27.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fbfd806af60 msgr2=0x7fbfd806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fbfd806af60 0x7fbfd806d420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40058c0 tx=0x7fbfe4005850 comp rx=0 tx=0).stop 2023-10-18T12:02:27.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40c96b0 msgr2=0x7fbff4150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40c96b0 0x7fbff4150e00 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000f80 tx=0x7fbff0035d80 comp rx=0 tx=0).stop 2023-10-18T12:02:27.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 shutdown_connections 2023-10-18T12:02:27.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fbfd806af60 0x7fbfd806d420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.798 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbff41f1610 0x7fbff41f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbff40cb980 0x7fbff4154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 --2- 172.21.15.125:0/3386710130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbff40c96b0 0x7fbff4150e00 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 >> 172.21.15.125:0/3386710130 conn(0x7fbff4069930 msgr2=0x7fbff415a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 shutdown_connections 2023-10-18T12:02:27.800 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.794+0000 7fbffc365640 1 -- 172.21.15.125:0/3386710130 wait complete. 2023-10-18T12:02:27.807 INFO:tasks.ceph:Scrubbing osd.4 2023-10-18T12:02:27.807 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-18T12:02:27.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.910+0000 7fd577e75640 1 -- 172.21.15.125:0/3065085166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd570070920 msgr2=0x7fd5700affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.910+0000 7fd577e75640 1 --2- 172.21.15.125:0/3065085166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd570070920 0x7fd5700affe0 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 crypto rx=0x7fd568005fd0 tx=0x7fd56802c050 comp rx=0 tx=0).stop 2023-10-18T12:02:27.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.911+0000 7fd577e75640 1 -- 172.21.15.125:0/3065085166 shutdown_connections 2023-10-18T12:02:27.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.911+0000 7fd577e75640 1 --2- 172.21.15.125:0/3065085166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd570070920 0x7fd5700affe0 unknown :-1 s=CLOSED pgs=3142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.911+0000 7fd577e75640 1 --2- 172.21.15.125:0/3065085166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd570141690 0x7fd5700703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.911+0000 7fd577e75640 1 -- 172.21.15.125:0/3065085166 >> 172.21.15.125:0/3065085166 conn(0x7fd570069930 msgr2=0x7fd570069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.911+0000 7fd577e75640 1 -- 172.21.15.125:0/3065085166 shutdown_connections 2023-10-18T12:02:27.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.911+0000 7fd577e75640 1 -- 172.21.15.125:0/3065085166 wait complete. 2023-10-18T12:02:27.919 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.912+0000 7fd577e75640 1 Processor -- start 2023-10-18T12:02:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.912+0000 7fd577e75640 1 -- start start 2023-10-18T12:02:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.912+0000 7fd577e75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd570070920 0x7fd5701f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.920 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.912+0000 7fd577e75640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57013e520 0x7fd57013d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.912+0000 7fd577e75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd570141690 0x7fd570130210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.912+0000 7fd577e75640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700adbb0 con 0x7fd57013e520 2023-10-18T12:02:27.921 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.912+0000 7fd577e75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701e2fe0 con 0x7fd570070920 2023-10-18T12:02:27.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.912+0000 7fd577e75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701f2cd0 con 0x7fd570141690 2023-10-18T12:02:27.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57013e520 0x7fd57013d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.922 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57013e520 0x7fd57013d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37606/0 (socket says 172.21.15.125:37606) 2023-10-18T12:02:27.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 -- 172.21.15.125:0/4030858965 learned_addr learned my addr 172.21.15.125:0/4030858965 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:27.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd576412640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd570141690 0x7fd570130210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.923 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575c11640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd570070920 0x7fd5701f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 -- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd570141690 msgr2=0x7fd570130210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd570141690 0x7fd570130210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 -- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd570070920 msgr2=0x7fd5701f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.924 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd570070920 0x7fd5701f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 -- 172.21.15.125:0/4030858965 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568009840 con 0x7fd57013e520 2023-10-18T12:02:27.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd576412640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd570141690 0x7fd570130210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:27.925 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd575410640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57013e520 0x7fd57013d570 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7fd56000c9a0 tx=0x7fd56000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560007bb0 con 0x7fd57013e520 2023-10-18T12:02:27.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd560007e00 con 0x7fd57013e520 2023-10-18T12:02:27.926 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560005280 con 0x7fd57013e520 2023-10-18T12:02:27.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5701f2fb0 con 0x7fd57013e520 2023-10-18T12:02:27.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.913+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57012dbb0 con 0x7fd57013e520 2023-10-18T12:02:27.927 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.914+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5700ad910 con 0x7fd57013e520 2023-10-18T12:02:27.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.915+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd5600053e0 con 0x7fd57013e520 2023-10-18T12:02:27.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.915+0000 7fd5667fc640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd55406af60 0x7fd55406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.928 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.915+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fd560089e60 con 0x7fd57013e520 2023-10-18T12:02:27.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.915+0000 7fd5667fc640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fd554071940 0x7fd554073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:27.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.915+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 --> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd554074490 con 0x7fd554071940 2023-10-18T12:02:27.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.915+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=268) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd56008c520 con 0x7fd57013e520 2023-10-18T12:02:27.929 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.915+0000 7fd575c11640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd55406af60 0x7fd55406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.915+0000 7fd576412640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fd554071940 0x7fd554073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:27.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.916+0000 7fd575c11640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd55406af60 0x7fd55406d420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0046b0 tx=0x7fd56c007eb0 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.916+0000 7fd576412640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fd554071940 0x7fd554073dc0 crc :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:27.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.916+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 <== osd.4 v2:172.21.15.139:6800/3552682439 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fd554074490 con 0x7fd554071940 2023-10-18T12:02:27.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.929+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 --> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd5700adcf0 con 0x7fd554071940 2023-10-18T12:02:27.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.929+0000 7fd5667fc640 1 -- 172.21.15.125:0/4030858965 <== osd.4 v2:172.21.15.139:6800/3552682439 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd5700adcf0 con 0x7fd554071940 2023-10-18T12:02:27.933 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:27.933 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-10-18T12:02:27.933 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:27.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fd554071940 msgr2=0x7fd554073dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6800/3552682439,v1:172.21.15.139:6801/3552682439] conn(0x7fd554071940 0x7fd554073dc0 crc :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.935 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd55406af60 msgr2=0x7fd55406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd55406af60 0x7fd55406d420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0046b0 tx=0x7fd56c007eb0 comp rx=0 tx=0).stop 2023-10-18T12:02:27.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57013e520 msgr2=0x7fd57013d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:27.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 --2- 172.21.15.125:0/4030858965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd57013e520 0x7fd57013d570 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7fd56000c9a0 tx=0x7fd56000ce60 comp rx=0 tx=0).stop 2023-10-18T12:02:27.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd576412640 1 -- 172.21.15.125:0/4030858965 reap_dead start 2023-10-18T12:02:27.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 shutdown_connections 2023-10-18T12:02:27.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 >> 172.21.15.125:0/4030858965 conn(0x7fd570069930 msgr2=0x7fd5701df5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:27.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 shutdown_connections 2023-10-18T12:02:27.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:27.930+0000 7fd577e75640 1 -- 172.21.15.125:0/4030858965 wait complete. 2023-10-18T12:02:27.942 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-18T12:02:28.045 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.042+0000 7f36f7518640 1 -- 172.21.15.125:0/2098371325 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00c96b0 msgr2=0x7f36f00c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.042+0000 7f36f7518640 1 --2- 172.21.15.125:0/2098371325 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00c96b0 0x7f36f00c9b30 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009900 tx=0x7f36e002be00 comp rx=0 tx=0).stop 2023-10-18T12:02:28.046 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.042+0000 7f36f7518640 1 -- 172.21.15.125:0/2098371325 shutdown_connections 2023-10-18T12:02:28.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.042+0000 7f36f7518640 1 --2- 172.21.15.125:0/2098371325 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00c96b0 0x7f36f00c9b30 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.042+0000 7f36f7518640 1 --2- 172.21.15.125:0/2098371325 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f006af00 0x7f36f006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.042+0000 7f36f7518640 1 -- 172.21.15.125:0/2098371325 >> 172.21.15.125:0/2098371325 conn(0x7f36f0069930 msgr2=0x7f36f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.047 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.043+0000 7f36f7518640 1 -- 172.21.15.125:0/2098371325 shutdown_connections 2023-10-18T12:02:28.048 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.043+0000 7f36f7518640 1 -- 172.21.15.125:0/2098371325 wait complete. 2023-10-18T12:02:28.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f7518640 1 Processor -- start 2023-10-18T12:02:28.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f7518640 1 -- start start 2023-10-18T12:02:28.051 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f7518640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f006af00 0x7f36f014ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f7518640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00ab500 0x7f36f00a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f7518640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f00c96b0 0x7f36f015b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f7518640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00a69f0 con 0x7f36f00ab500 2023-10-18T12:02:28.052 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f7518640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00a9e10 con 0x7f36f006af00 2023-10-18T12:02:28.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f7518640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00a9200 con 0x7f36f00c96b0 2023-10-18T12:02:28.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.044+0000 7f36f4ab3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00ab500 0x7f36f00a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f4ab3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00ab500 0x7f36f00a64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37620/0 (socket says 172.21.15.125:37620) 2023-10-18T12:02:28.053 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f4ab3640 1 -- 172.21.15.125:0/3913158731 learned_addr learned my addr 172.21.15.125:0/3913158731 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:28.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f52b4640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f006af00 0x7f36f014ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f5ab5640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f00c96b0 0x7f36f015b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.054 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f4ab3640 1 -- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f00c96b0 msgr2=0x7f36f015b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f4ab3640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f00c96b0 0x7f36f015b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f4ab3640 1 -- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f006af00 msgr2=0x7f36f014ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f4ab3640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f006af00 0x7f36f014ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.055 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f4ab3640 1 -- 172.21.15.125:0/3913158731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e0009560 con 0x7f36f00ab500 2023-10-18T12:02:28.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f4ab3640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00ab500 0x7f36f00a64b0 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7f36e0006010 tx=0x7f36e002da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36e5ffb640 1 -- 172.21.15.125:0/3913158731 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e002de60 con 0x7f36f00ab500 2023-10-18T12:02:28.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36e5ffb640 1 -- 172.21.15.125:0/3913158731 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f36e00036e0 con 0x7f36f00ab500 2023-10-18T12:02:28.056 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f00a94e0 con 0x7f36f00ab500 2023-10-18T12:02:28.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36e5ffb640 1 -- 172.21.15.125:0/3913158731 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0003ad0 con 0x7f36f00ab500 2023-10-18T12:02:28.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.045+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f00a9aa0 con 0x7f36f00ab500 2023-10-18T12:02:28.057 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.046+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f00690e0 con 0x7f36f00ab500 2023-10-18T12:02:28.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.047+0000 7f36e5ffb640 1 -- 172.21.15.125:0/3913158731 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f36e0003c30 con 0x7f36f00ab500 2023-10-18T12:02:28.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.047+0000 7f36e5ffb640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f36d406af60 0x7f36d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.058 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.047+0000 7f36e5ffb640 1 -- 172.21.15.125:0/3913158731 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f36e00ac4a0 con 0x7f36f00ab500 2023-10-18T12:02:28.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.047+0000 7f36f52b4640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f36d406af60 0x7f36d406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.061 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.048+0000 7f36f52b4640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f36d406af60 0x7f36d406d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009bd0 tx=0x7f36ec005f90 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.062 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.051+0000 7f36e5ffb640 1 -- 172.21.15.125:0/3913158731 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36e00773e0 con 0x7f36f00ab500 2023-10-18T12:02:28.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.218+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f36f0159c50 con 0x7f36d406af60 2023-10-18T12:02:28.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.219+0000 7f36e5ffb640 1 -- 172.21.15.125:0/3913158731 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f36ec01b3f0 con 0x7f36d406af60 2023-10-18T12:02:28.222 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 4 to deep-scrub 2023-10-18T12:02:28.224 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f36d406af60 msgr2=0x7f36d406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f36d406af60 0x7f36d406d420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009bd0 tx=0x7f36ec005f90 comp rx=0 tx=0).stop 2023-10-18T12:02:28.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00ab500 msgr2=0x7f36f00a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00ab500 0x7f36f00a64b0 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7f36e0006010 tx=0x7f36e002da60 comp rx=0 tx=0).stop 2023-10-18T12:02:28.225 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 shutdown_connections 2023-10-18T12:02:28.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f36d406af60 0x7f36d406d420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f00c96b0 0x7f36f015b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36f00ab500 0x7f36f00a64b0 unknown :-1 s=CLOSED pgs=3145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 --2- 172.21.15.125:0/3913158731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f006af00 0x7f36f014ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 >> 172.21.15.125:0/3913158731 conn(0x7f36f0069930 msgr2=0x7f36f013cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 shutdown_connections 2023-10-18T12:02:28.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.221+0000 7f36f7518640 1 -- 172.21.15.125:0/3913158731 wait complete. 2023-10-18T12:02:28.234 INFO:tasks.ceph:Scrubbing osd.5 2023-10-18T12:02:28.235 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-18T12:02:28.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.338+0000 7fe54be4f640 1 -- 172.21.15.125:0/1225576624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 msgr2=0x7fe5440b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.338+0000 7fe54be4f640 1 --2- 172.21.15.125:0/1225576624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 0x7fe5440b0650 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7fe54408a120 tx=0x7fe53c02bdb0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.338+0000 7fe54be4f640 1 -- 172.21.15.125:0/1225576624 shutdown_connections 2023-10-18T12:02:28.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.338+0000 7fe54be4f640 1 --2- 172.21.15.125:0/1225576624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 0x7fe5440b0650 unknown :-1 s=CLOSED pgs=3146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.338+0000 7fe54be4f640 1 --2- 172.21.15.125:0/1225576624 >> v2:172.21.15.139:3300/0 conn(0x7fe54415b2a0 0x7fe5440d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.338+0000 7fe54be4f640 1 -- 172.21.15.125:0/1225576624 >> 172.21.15.125:0/1225576624 conn(0x7fe5440d0d60 msgr2=0x7fe5440d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.338+0000 7fe54be4f640 1 -- 172.21.15.125:0/1225576624 shutdown_connections 2023-10-18T12:02:28.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.338+0000 7fe54be4f640 1 -- 172.21.15.125:0/1225576624 wait complete. 2023-10-18T12:02:28.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.339+0000 7fe54be4f640 1 Processor -- start 2023-10-18T12:02:28.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.339+0000 7fe54be4f640 1 -- start start 2023-10-18T12:02:28.347 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe54be4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 0x7fe544151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe54be4f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5440cf340 0x7fe5440c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe54be4f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe54415b2a0 0x7fe5441e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.348 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe54be4f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440c7ed0 con 0x7fe5440ceb40 2023-10-18T12:02:28.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe54be4f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5441516d0 con 0x7fe54415b2a0 2023-10-18T12:02:28.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe54be4f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440c8140 con 0x7fe5440cf340 2023-10-18T12:02:28.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe549beb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 0x7fe544151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.349 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe549beb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 0x7fe544151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37646/0 (socket says 172.21.15.125:37646) 2023-10-18T12:02:28.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe549beb640 1 -- 172.21.15.125:0/4225399549 learned_addr learned my addr 172.21.15.125:0/4225399549 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:28.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe54a3ec640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe54415b2a0 0x7fe5441e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.350 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe5493ea640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5440cf340 0x7fe5440c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe549beb640 1 -- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5440cf340 msgr2=0x7fe5440c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe549beb640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5440cf340 0x7fe5440c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe549beb640 1 -- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe54415b2a0 msgr2=0x7fe5441e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.351 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe549beb640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe54415b2a0 0x7fe5441e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe549beb640 1 -- 172.21.15.125:0/4225399549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c009560 con 0x7fe5440ceb40 2023-10-18T12:02:28.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.340+0000 7fe54a3ec640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe54415b2a0 0x7fe5441e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:28.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.341+0000 7fe549beb640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 0x7fe544151190 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7fe54000c790 tx=0x7fe54000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.352 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.341+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54000cea0 con 0x7fe5440ceb40 2023-10-18T12:02:28.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.341+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe540006e40 con 0x7fe5440ceb40 2023-10-18T12:02:28.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.341+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5440c8420 con 0x7fe5440ceb40 2023-10-18T12:02:28.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.341+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540003bd0 con 0x7fe5440ceb40 2023-10-18T12:02:28.353 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.341+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5440c89e0 con 0x7fe5440ceb40 2023-10-18T12:02:28.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.342+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5440b0d30 con 0x7fe5440ceb40 2023-10-18T12:02:28.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.342+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe540003d30 con 0x7fe5440ceb40 2023-10-18T12:02:28.354 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe53a7fc640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fe52806af90 0x7fe52806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fe540089790 con 0x7fe5440ceb40 2023-10-18T12:02:28.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe5493ea640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fe52806af90 0x7fe52806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe53a7fc640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7fe528071970 0x7fe528073df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.355 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 --> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5280744c0 con 0x7fe528071970 2023-10-18T12:02:28.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=268) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe54008be50 con 0x7fe5440ceb40 2023-10-18T12:02:28.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe54a3ec640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7fe528071970 0x7fe528073df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.356 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe5493ea640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fe52806af90 0x7fe52806d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe544159ae0 tx=0x7fe534005f90 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.343+0000 7fe54a3ec640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7fe528071970 0x7fe528073df0 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.357 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.344+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 <== osd.5 v2:172.21.15.139:6816/1786373778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fe5280744c0 con 0x7fe528071970 2023-10-18T12:02:28.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 --> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe5440690e0 con 0x7fe528071970 2023-10-18T12:02:28.359 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe53a7fc640 1 -- 172.21.15.125:0/4225399549 <== osd.5 v2:172.21.15.139:6816/1786373778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe5440690e0 con 0x7fe528071970 2023-10-18T12:02:28.360 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:28.360 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-10-18T12:02:28.361 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:28.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7fe528071970 msgr2=0x7fe528073df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6816/1786373778,v1:172.21.15.139:6817/1786373778] conn(0x7fe528071970 0x7fe528073df0 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.362 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fe52806af90 msgr2=0x7fe52806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fe52806af90 0x7fe52806d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe544159ae0 tx=0x7fe534005f90 comp rx=0 tx=0).stop 2023-10-18T12:02:28.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 msgr2=0x7fe544151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.363 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 --2- 172.21.15.125:0/4225399549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe5440ceb40 0x7fe544151190 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7fe54000c790 tx=0x7fe54000cc50 comp rx=0 tx=0).stop 2023-10-18T12:02:28.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54a3ec640 1 -- 172.21.15.125:0/4225399549 reap_dead start 2023-10-18T12:02:28.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 shutdown_connections 2023-10-18T12:02:28.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 >> 172.21.15.125:0/4225399549 conn(0x7fe5440d0d60 msgr2=0x7fe54415a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.364 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 shutdown_connections 2023-10-18T12:02:28.365 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.357+0000 7fe54be4f640 1 -- 172.21.15.125:0/4225399549 wait complete. 2023-10-18T12:02:28.370 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-18T12:02:28.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.473+0000 7fb779a56640 1 -- 172.21.15.125:0/3477412144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 msgr2=0x7fb7740c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.473+0000 7fb779a56640 1 --2- 172.21.15.125:0/3477412144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 0x7fb7740c9730 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005fd0 tx=0x7fb75c02bd90 comp rx=0 tx=0).stop 2023-10-18T12:02:28.476 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.473+0000 7fb779a56640 1 -- 172.21.15.125:0/3477412144 shutdown_connections 2023-10-18T12:02:28.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.473+0000 7fb779a56640 1 --2- 172.21.15.125:0/3477412144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 0x7fb7740c9730 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.473+0000 7fb779a56640 1 --2- 172.21.15.125:0/3477412144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7740afa20 0x7fb7740cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.473+0000 7fb779a56640 1 -- 172.21.15.125:0/3477412144 >> 172.21.15.125:0/3477412144 conn(0x7fb774069930 msgr2=0x7fb774069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.477 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.473+0000 7fb779a56640 1 -- 172.21.15.125:0/3477412144 shutdown_connections 2023-10-18T12:02:28.478 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.473+0000 7fb779a56640 1 -- 172.21.15.125:0/3477412144 wait complete. 2023-10-18T12:02:28.480 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.474+0000 7fb779a56640 1 Processor -- start 2023-10-18T12:02:28.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.474+0000 7fb779a56640 1 -- start start 2023-10-18T12:02:28.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb779a56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7740705a0 0x7fb77414f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.481 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb779a56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7740afa20 0x7fb7741e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb779a56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 0x7fb7741e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb779a56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7741e59f0 con 0x7fb774158300 2023-10-18T12:02:28.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb779a56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7741e5430 con 0x7fb7740705a0 2023-10-18T12:02:28.482 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb779a56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7741e5710 con 0x7fb7740afa20 2023-10-18T12:02:28.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb7737fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 0x7fb7741e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb7737fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 0x7fb7741e4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37664/0 (socket says 172.21.15.125:37664) 2023-10-18T12:02:28.483 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb7737fe640 1 -- 172.21.15.125:0/2491902957 learned_addr learned my addr 172.21.15.125:0/2491902957 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:28.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb76a7fc640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7740afa20 0x7fb7741e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb772ffd640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7740705a0 0x7fb77414f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb7737fe640 1 -- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7740afa20 msgr2=0x7fb7741e47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.484 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb7737fe640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7740afa20 0x7fb7741e47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb7737fe640 1 -- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7740705a0 msgr2=0x7fb77414f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb7737fe640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7740705a0 0x7fb77414f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.485 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.475+0000 7fb7737fe640 1 -- 172.21.15.125:0/2491902957 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c009560 con 0x7fb774158300 2023-10-18T12:02:28.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.476+0000 7fb7737fe640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 0x7fb7741e4d30 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c780 tx=0x7fb76c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.476+0000 7fb770ff9640 1 -- 172.21.15.125:0/2491902957 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c003d80 con 0x7fb774158300 2023-10-18T12:02:28.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.476+0000 7fb770ff9640 1 -- 172.21.15.125:0/2491902957 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb76c00ce90 con 0x7fb774158300 2023-10-18T12:02:28.486 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.476+0000 7fb770ff9640 1 -- 172.21.15.125:0/2491902957 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0052d0 con 0x7fb774158300 2023-10-18T12:02:28.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.476+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb774154720 con 0x7fb774158300 2023-10-18T12:02:28.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.476+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77414f640 con 0x7fb774158300 2023-10-18T12:02:28.487 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.477+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb774154580 con 0x7fb774158300 2023-10-18T12:02:28.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.477+0000 7fb770ff9640 1 -- 172.21.15.125:0/2491902957 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb76c005430 con 0x7fb774158300 2023-10-18T12:02:28.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.477+0000 7fb770ff9640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb75406af60 0x7fb75406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.478+0000 7fb770ff9640 1 -- 172.21.15.125:0/2491902957 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fb76c0890f0 con 0x7fb774158300 2023-10-18T12:02:28.488 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.478+0000 7fb772ffd640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb75406af60 0x7fb75406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.478+0000 7fb772ffd640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb75406af60 0x7fb75406d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb764009890 tx=0x7fb764008040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.489 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.482+0000 7fb770ff9640 1 -- 172.21.15.125:0/2491902957 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb774154580 con 0x7fb774158300 2023-10-18T12:02:28.652 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.650+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb774154c40 con 0x7fb75406af60 2023-10-18T12:02:28.653 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.650+0000 7fb770ff9640 1 -- 172.21.15.125:0/2491902957 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb774154c40 con 0x7fb75406af60 2023-10-18T12:02:28.654 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 5 to deep-scrub 2023-10-18T12:02:28.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb75406af60 msgr2=0x7fb75406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.655 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb75406af60 0x7fb75406d420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb764009890 tx=0x7fb764008040 comp rx=0 tx=0).stop 2023-10-18T12:02:28.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 msgr2=0x7fb7741e4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 0x7fb7741e4d30 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c780 tx=0x7fb76c00cc40 comp rx=0 tx=0).stop 2023-10-18T12:02:28.656 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 shutdown_connections 2023-10-18T12:02:28.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb75406af60 0x7fb75406d420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774158300 0x7fb7741e4d30 unknown :-1 s=CLOSED pgs=3149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb7740afa20 0x7fb7741e47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.657 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 --2- 172.21.15.125:0/2491902957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7740705a0 0x7fb77414f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 >> 172.21.15.125:0/2491902957 conn(0x7fb774069930 msgr2=0x7fb774069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 shutdown_connections 2023-10-18T12:02:28.658 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.652+0000 7fb779a56640 1 -- 172.21.15.125:0/2491902957 wait complete. 2023-10-18T12:02:28.665 INFO:tasks.ceph:Scrubbing osd.6 2023-10-18T12:02:28.666 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-18T12:02:28.771 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.768+0000 7fc512df5640 1 -- 172.21.15.125:0/1244170020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c06af00 msgr2=0x7fc50c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.768+0000 7fc512df5640 1 --2- 172.21.15.125:0/1244170020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c06af00 0x7fc50c06b2e0 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc005fd0 tx=0x7fc4fc02bd10 comp rx=0 tx=0).stop 2023-10-18T12:02:28.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.768+0000 7fc512df5640 1 -- 172.21.15.125:0/1244170020 shutdown_connections 2023-10-18T12:02:28.772 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.768+0000 7fc512df5640 1 --2- 172.21.15.125:0/1244170020 >> v2:172.21.15.139:3300/0 conn(0x7fc50c0abc10 0x7fc50c0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.768+0000 7fc512df5640 1 --2- 172.21.15.125:0/1244170020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c06af00 0x7fc50c06b2e0 unknown :-1 s=CLOSED pgs=3150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.768+0000 7fc512df5640 1 -- 172.21.15.125:0/1244170020 >> 172.21.15.125:0/1244170020 conn(0x7fc50c069930 msgr2=0x7fc50c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.773 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.768+0000 7fc512df5640 1 -- 172.21.15.125:0/1244170020 shutdown_connections 2023-10-18T12:02:28.774 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.768+0000 7fc512df5640 1 -- 172.21.15.125:0/1244170020 wait complete. 2023-10-18T12:02:28.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.769+0000 7fc512df5640 1 Processor -- start 2023-10-18T12:02:28.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc512df5640 1 -- start start 2023-10-18T12:02:28.777 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc512df5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc50c06af00 0x7fc50c1eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc512df5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c0abc10 0x7fc50c12f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc512df5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc50c0c96b0 0x7fc50c129b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.778 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc512df5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c12f2b0 con 0x7fc50c0abc10 2023-10-18T12:02:28.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc512df5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c12e6a0 con 0x7fc50c06af00 2023-10-18T12:02:28.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc512df5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c12e980 con 0x7fc50c0c96b0 2023-10-18T12:02:28.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc50bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c0abc10 0x7fc50c12f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.779 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc50bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c0abc10 0x7fc50c12f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37682/0 (socket says 172.21.15.125:37682) 2023-10-18T12:02:28.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.770+0000 7fc50bfff640 1 -- 172.21.15.125:0/154394665 learned_addr learned my addr 172.21.15.125:0/154394665 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:28.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc511392640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc50c0c96b0 0x7fc50c129b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.780 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc510b91640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc50c06af00 0x7fc50c1eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc50bfff640 1 -- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc50c0c96b0 msgr2=0x7fc50c129b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc50bfff640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc50c0c96b0 0x7fc50c129b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.781 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc50bfff640 1 -- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc50c06af00 msgr2=0x7fc50c1eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc50bfff640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc50c06af00 0x7fc50c1eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc50bfff640 1 -- 172.21.15.125:0/154394665 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc009560 con 0x7fc50c0abc10 2023-10-18T12:02:28.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc511392640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc50c0c96b0 0x7fc50c129b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:28.782 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc510b91640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc50c06af00 0x7fc50c1eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T12:02:28.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc50bfff640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c0abc10 0x7fc50c12f740 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400b3f0 tx=0x7fc4f400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4011830 con 0x7fc50c0abc10 2023-10-18T12:02:28.783 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4f4011a80 con 0x7fc50c0abc10 2023-10-18T12:02:28.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4011e90 con 0x7fc50c0abc10 2023-10-18T12:02:28.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c12ec60 con 0x7fc50c0abc10 2023-10-18T12:02:28.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.771+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c1ef190 con 0x7fc50c0abc10 2023-10-18T12:02:28.784 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.772+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc50c1ef3b0 con 0x7fc50c0abc10 2023-10-18T12:02:28.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.772+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc4f401a400 con 0x7fc50c0abc10 2023-10-18T12:02:28.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.773+0000 7fc5097fa640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fc4ec06af60 0x7fc4ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.785 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.773+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fc4f40899f0 con 0x7fc50c0abc10 2023-10-18T12:02:28.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.773+0000 7fc5097fa640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7fc4ec0717d0 0x7fc4ec073c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.773+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 --> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4ec074320 con 0x7fc4ec0717d0 2023-10-18T12:02:28.786 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.773+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=268) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4f408e2c0 con 0x7fc50c0abc10 2023-10-18T12:02:28.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.773+0000 7fc510b91640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fc4ec06af60 0x7fc4ec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.773+0000 7fc511392640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7fc4ec0717d0 0x7fc4ec073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.774+0000 7fc510b91640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fc4ec06af60 0x7fc4ec06d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc035000 tx=0x7fc4fc004730 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.787 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.774+0000 7fc511392640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7fc4ec0717d0 0x7fc4ec073c50 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.788 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.774+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 <== osd.6 v2:172.21.15.139:6808/1564178764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fc4ec074320 con 0x7fc4ec0717d0 2023-10-18T12:02:28.789 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.787+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 --> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc50c159c50 con 0x7fc4ec0717d0 2023-10-18T12:02:28.790 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.787+0000 7fc5097fa640 1 -- 172.21.15.125:0/154394665 <== osd.6 v2:172.21.15.139:6808/1564178764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc504010430 con 0x7fc4ec0717d0 2023-10-18T12:02:28.790 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:28.791 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-10-18T12:02:28.791 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:28.792 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7fc4ec0717d0 msgr2=0x7fc4ec073c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6808/1564178764,v1:172.21.15.139:6809/1564178764] conn(0x7fc4ec0717d0 0x7fc4ec073c50 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fc4ec06af60 msgr2=0x7fc4ec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fc4ec06af60 0x7fc4ec06d420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc035000 tx=0x7fc4fc004730 comp rx=0 tx=0).stop 2023-10-18T12:02:28.793 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c0abc10 msgr2=0x7fc50c12f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 --2- 172.21.15.125:0/154394665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc50c0abc10 0x7fc50c12f740 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400b3f0 tx=0x7fc4f400b8b0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc511392640 1 -- 172.21.15.125:0/154394665 reap_dead start 2023-10-18T12:02:28.794 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 shutdown_connections 2023-10-18T12:02:28.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 >> 172.21.15.125:0/154394665 conn(0x7fc50c069930 msgr2=0x7fc50c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 shutdown_connections 2023-10-18T12:02:28.795 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.788+0000 7fc512df5640 1 -- 172.21.15.125:0/154394665 wait complete. 2023-10-18T12:02:28.800 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-18T12:02:28.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.901+0000 7f80f2155640 1 -- 172.21.15.125:0/2378735759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0afa20 msgr2=0x7f80ec158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.901+0000 7f80f2155640 1 --2- 172.21.15.125:0/2378735759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0afa20 0x7f80ec158300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f80dc005fd0 tx=0x7f80dc02be10 comp rx=0 tx=0).stop 2023-10-18T12:02:28.904 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.901+0000 7f80f2155640 1 -- 172.21.15.125:0/2378735759 shutdown_connections 2023-10-18T12:02:28.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.901+0000 7f80f2155640 1 --2- 172.21.15.125:0/2378735759 >> v2:172.21.15.139:3300/0 conn(0x7f80ec0b0530 0x7f80ec0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.901+0000 7f80f2155640 1 --2- 172.21.15.125:0/2378735759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0afa20 0x7f80ec158300 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.905 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.901+0000 7f80f2155640 1 -- 172.21.15.125:0/2378735759 >> 172.21.15.125:0/2378735759 conn(0x7f80ec069930 msgr2=0x7f80ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:28.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.901+0000 7f80f2155640 1 -- 172.21.15.125:0/2378735759 shutdown_connections 2023-10-18T12:02:28.906 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.901+0000 7f80f2155640 1 -- 172.21.15.125:0/2378735759 wait complete. 2023-10-18T12:02:28.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.902+0000 7f80f2155640 1 Processor -- start 2023-10-18T12:02:28.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.902+0000 7f80f2155640 1 -- start start 2023-10-18T12:02:28.909 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80f2155640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80ec0afa20 0x7f80ec13cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80f2155640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0b0530 0x7f80ec156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80f2155640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80ec0cbd20 0x7f80ec156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.910 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80f2155640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1403f0 con 0x7f80ec0afa20 2023-10-18T12:02:28.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80f2155640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1e4090 con 0x7f80ec0cbd20 2023-10-18T12:02:28.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80f2155640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec13d120 con 0x7f80ec0b0530 2023-10-18T12:02:28.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80ec0afa20 0x7f80ec13cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.911 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80ec0afa20 0x7f80ec13cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37696/0 (socket says 172.21.15.125:37696) 2023-10-18T12:02:28.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eb7fe640 1 -- 172.21.15.125:0/2329657412 learned_addr learned my addr 172.21.15.125:0/2329657412 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:28.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eaffd640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0b0530 0x7f80ec156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.912 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80ebfff640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80ec0cbd20 0x7f80ec156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eb7fe640 1 -- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0b0530 msgr2=0x7f80ec156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eb7fe640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0b0530 0x7f80ec156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eb7fe640 1 -- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80ec0cbd20 msgr2=0x7f80ec156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:28.913 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eb7fe640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80ec0cbd20 0x7f80ec156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eb7fe640 1 -- 172.21.15.125:0/2329657412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc009560 con 0x7f80ec0afa20 2023-10-18T12:02:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80eaffd640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0b0530 0x7f80ec156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.903+0000 7f80ebfff640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80ec0cbd20 0x7f80ec156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.904+0000 7f80eb7fe640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80ec0afa20 0x7f80ec13cbe0 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7f80e000c3c0 tx=0x7f80e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.904+0000 7f80d3fff640 1 -- 172.21.15.125:0/2329657412 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e00052d0 con 0x7f80ec0afa20 2023-10-18T12:02:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.904+0000 7f80d3fff640 1 -- 172.21.15.125:0/2329657412 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f80e0007020 con 0x7f80ec0afa20 2023-10-18T12:02:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.904+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec13d560 con 0x7f80ec0afa20 2023-10-18T12:02:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.904+0000 7f80d3fff640 1 -- 172.21.15.125:0/2329657412 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0007470 con 0x7f80ec0afa20 2023-10-18T12:02:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.904+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec13db20 con 0x7f80ec0afa20 2023-10-18T12:02:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.905+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec0690e0 con 0x7f80ec0afa20 2023-10-18T12:02:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.905+0000 7f80d3fff640 1 -- 172.21.15.125:0/2329657412 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f80e00067d0 con 0x7f80ec0afa20 2023-10-18T12:02:28.916 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.905+0000 7f80d3fff640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f80d806af60 0x7f80d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:28.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.906+0000 7f80d3fff640 1 -- 172.21.15.125:0/2329657412 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f80e0088f90 con 0x7f80ec0afa20 2023-10-18T12:02:28.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.906+0000 7f80eaffd640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f80d806af60 0x7f80d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:28.917 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.906+0000 7f80eaffd640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f80d806af60 0x7f80d806d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f80dc035000 tx=0x7f80dc004730 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:28.918 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:28.910+0000 7f80d3fff640 1 -- 172.21.15.125:0/2329657412 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80e00bbca0 con 0x7f80ec0afa20 2023-10-18T12:02:29.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.074+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f80ec0cc100 con 0x7f80d806af60 2023-10-18T12:02:29.077 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.075+0000 7f80d3fff640 1 -- 172.21.15.125:0/2329657412 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f80dc004250 con 0x7f80d806af60 2023-10-18T12:02:29.078 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 6 to deep-scrub 2023-10-18T12:02:29.079 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.076+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f80d806af60 msgr2=0x7f80d806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.076+0000 7f80f2155640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f80d806af60 0x7f80d806d420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f80dc035000 tx=0x7f80dc004730 comp rx=0 tx=0).stop 2023-10-18T12:02:29.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80ec0afa20 msgr2=0x7f80ec13cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.080 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80ec0afa20 0x7f80ec13cbe0 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7f80e000c3c0 tx=0x7f80e000c880 comp rx=0 tx=0).stop 2023-10-18T12:02:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 shutdown_connections 2023-10-18T12:02:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f80d806af60 0x7f80d806d420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80ec0cbd20 0x7f80ec156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80ec0b0530 0x7f80ec156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 --2- 172.21.15.125:0/2329657412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80ec0afa20 0x7f80ec13cbe0 unknown :-1 s=CLOSED pgs=3152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.082 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 >> 172.21.15.125:0/2329657412 conn(0x7f80ec069930 msgr2=0x7f80ec069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:29.083 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 shutdown_connections 2023-10-18T12:02:29.085 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.077+0000 7f80f2155640 1 -- 172.21.15.125:0/2329657412 wait complete. 2023-10-18T12:02:29.090 INFO:tasks.ceph:Scrubbing osd.7 2023-10-18T12:02:29.090 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-18T12:02:29.197 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 -- 172.21.15.125:0/2137936685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb94406c0a0 msgr2=0x7fb944070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 --2- 172.21.15.125:0/2137936685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb94406c0a0 0x7fb944070920 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 crypto rx=0x7fb9440cbd00 tx=0x7fb93c02c140 comp rx=0 tx=0).stop 2023-10-18T12:02:29.198 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 -- 172.21.15.125:0/2137936685 shutdown_connections 2023-10-18T12:02:29.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 --2- 172.21.15.125:0/2137936685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb94406c0a0 0x7fb944070920 unknown :-1 s=CLOSED pgs=3153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 --2- 172.21.15.125:0/2137936685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb944141690 0x7fb9440703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 --2- 172.21.15.125:0/2137936685 >> v2:172.21.15.139:3300/0 conn(0x7fb94413e520 0x7fb94406bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.199 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 -- 172.21.15.125:0/2137936685 >> 172.21.15.125:0/2137936685 conn(0x7fb944069930 msgr2=0x7fb944069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:29.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 -- 172.21.15.125:0/2137936685 shutdown_connections 2023-10-18T12:02:29.200 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.194+0000 7fb948fb0640 1 -- 172.21.15.125:0/2137936685 wait complete. 2023-10-18T12:02:29.203 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb948fb0640 1 Processor -- start 2023-10-18T12:02:29.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb948fb0640 1 -- start start 2023-10-18T12:02:29.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb948fb0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb94406c0a0 0x7fb944140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb948fb0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb94413e520 0x7fb9441e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.204 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb948fb0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb944141690 0x7fb9440b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb948fb0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440c6f50 con 0x7fb944141690 2023-10-18T12:02:29.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb948fb0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440c8e50 con 0x7fb94413e520 2023-10-18T12:02:29.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb948fb0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440c7370 con 0x7fb94406c0a0 2023-10-18T12:02:29.205 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb942d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb944141690 0x7fb9440b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb942d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb944141690 0x7fb9440b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37710/0 (socket says 172.21.15.125:37710) 2023-10-18T12:02:29.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.196+0000 7fb942d9d640 1 -- 172.21.15.125:0/838265353 learned_addr learned my addr 172.21.15.125:0/838265353 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:29.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb94259c640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb94406c0a0 0x7fb944140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.206 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb942d9d640 1 -- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb94406c0a0 msgr2=0x7fb944140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb942d9d640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb94406c0a0 0x7fb944140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb942d9d640 1 -- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb94413e520 msgr2=0x7fb9441e15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb941d9b640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb94413e520 0x7fb9441e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.207 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb942d9d640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb94413e520 0x7fb9441e15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb942d9d640 1 -- 172.21.15.125:0/838265353 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c009900 con 0x7fb944141690 2023-10-18T12:02:29.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb94259c640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb94406c0a0 0x7fb944140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T12:02:29.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb942d9d640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb944141690 0x7fb9440b0b40 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009d60 tx=0x7fb93c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:29.208 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c004240 con 0x7fb944141690 2023-10-18T12:02:29.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb93c004490 con 0x7fb944141690 2023-10-18T12:02:29.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9440c62d0 con 0x7fb944141690 2023-10-18T12:02:29.209 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c004880 con 0x7fb944141690 2023-10-18T12:02:29.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.197+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9440c6890 con 0x7fb944141690 2023-10-18T12:02:29.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.198+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9440690e0 con 0x7fb944141690 2023-10-18T12:02:29.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.198+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb93c03b040 con 0x7fb944141690 2023-10-18T12:02:29.210 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.199+0000 7fb932ffd640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb92406af60 0x7fb92406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.199+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fb93c0ac5b0 con 0x7fb944141690 2023-10-18T12:02:29.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.199+0000 7fb932ffd640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7fb924071940 0x7fb924073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.211 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.199+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 --> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb924074490 con 0x7fb924071940 2023-10-18T12:02:29.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.199+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=268) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb93c0aeb90 con 0x7fb944141690 2023-10-18T12:02:29.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.199+0000 7fb94259c640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb92406af60 0x7fb92406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.199+0000 7fb941d9b640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7fb924071940 0x7fb924073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.212 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.200+0000 7fb94259c640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb92406af60 0x7fb92406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb93800baa0 tx=0x7fb938005d20 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:29.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.200+0000 7fb941d9b640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7fb924071940 0x7fb924073dc0 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:29.213 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.201+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 <== osd.7 v2:172.21.15.139:6824/2679488483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fb924074490 con 0x7fb924071940 2023-10-18T12:02:29.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.213+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 --> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb9440c6c30 con 0x7fb924071940 2023-10-18T12:02:29.216 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb932ffd640 1 -- 172.21.15.125:0/838265353 <== osd.7 v2:172.21.15.139:6824/2679488483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb9440c6c30 con 0x7fb924071940 2023-10-18T12:02:29.217 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2023-10-18T12:02:29.217 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-10-18T12:02:29.217 INFO:teuthology.orchestra.run.smithi125.stdout:} 2023-10-18T12:02:29.218 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7fb924071940 msgr2=0x7fb924073dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb948fb0640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6824/2679488483,v1:172.21.15.139:6825/2679488483] conn(0x7fb924071940 0x7fb924073dc0 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb92406af60 msgr2=0x7fb92406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb948fb0640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fb92406af60 0x7fb92406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb93800baa0 tx=0x7fb938005d20 comp rx=0 tx=0).stop 2023-10-18T12:02:29.219 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb944141690 msgr2=0x7fb9440b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb948fb0640 1 --2- 172.21.15.125:0/838265353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb944141690 0x7fb9440b0b40 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009d60 tx=0x7fb93c003ba0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb942d9d640 1 -- 172.21.15.125:0/838265353 reap_dead start 2023-10-18T12:02:29.220 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 shutdown_connections 2023-10-18T12:02:29.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.214+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 >> 172.21.15.125:0/838265353 conn(0x7fb944069930 msgr2=0x7fb944069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:29.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.215+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 shutdown_connections 2023-10-18T12:02:29.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.215+0000 7fb948fb0640 1 -- 172.21.15.125:0/838265353 wait complete. 2023-10-18T12:02:29.227 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-18T12:02:29.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.329+0000 7fad5d8af640 1 -- 172.21.15.125:0/2416818115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad440039a0 msgr2=0x7fad44003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.332 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.329+0000 7fad5d8af640 1 --2- 172.21.15.125:0/2416818115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad440039a0 0x7fad44003dc0 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7fad48001eb0 tx=0x7fad440057f0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.329+0000 7fad577fe640 1 -- 172.21.15.125:0/2416818115 reap_dead start 2023-10-18T12:02:29.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.329+0000 7fad5d8af640 1 -- 172.21.15.125:0/2416818115 shutdown_connections 2023-10-18T12:02:29.333 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.329+0000 7fad5d8af640 1 -- 172.21.15.125:0/2416818115 >> 172.21.15.125:0/2416818115 conn(0x7fad580d0cf0 msgr2=0x7fad580d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:29.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.329+0000 7fad5d8af640 1 -- 172.21.15.125:0/2416818115 shutdown_connections 2023-10-18T12:02:29.334 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.329+0000 7fad5d8af640 1 -- 172.21.15.125:0/2416818115 wait complete. 2023-10-18T12:02:29.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.330+0000 7fad5d8af640 1 Processor -- start 2023-10-18T12:02:29.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.330+0000 7fad5d8af640 1 -- start start 2023-10-18T12:02:29.337 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad5d8af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad440039a0 0x7fad58158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad5d8af640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad580b6310 0x7fad580b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad5d8af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580c72a0 0x7fad5809df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.338 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad5d8af640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58158a60 con 0x7fad580b6310 2023-10-18T12:02:29.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad5d8af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580b5270 con 0x7fad440039a0 2023-10-18T12:02:29.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad5d8af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580b5550 con 0x7fad580c72a0 2023-10-18T12:02:29.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad580b6310 0x7fad580b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.339 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad580b6310 0x7fad580b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37722/0 (socket says 172.21.15.125:37722) 2023-10-18T12:02:29.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 -- 172.21.15.125:0/2442307461 learned_addr learned my addr 172.21.15.125:0/2442307461 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:29.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad56ffd640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad440039a0 0x7fad58158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.340 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad577fe640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580c72a0 0x7fad5809df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 -- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580c72a0 msgr2=0x7fad5809df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580c72a0 0x7fad5809df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 -- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad440039a0 msgr2=0x7fad58158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.341 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad440039a0 0x7fad58158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 -- 172.21.15.125:0/2442307461 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580b5830 con 0x7fad580b6310 2023-10-18T12:02:29.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.331+0000 7fad4ffff640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad580b6310 0x7fad580b4a20 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 crypto rx=0x7fad40003ed0 tx=0x7fad40001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:29.342 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.332+0000 7fad54ff9640 1 -- 172.21.15.125:0/2442307461 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40006110 con 0x7fad580b6310 2023-10-18T12:02:29.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.332+0000 7fad54ff9640 1 -- 172.21.15.125:0/2442307461 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad40011840 con 0x7fad580b6310 2023-10-18T12:02:29.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.332+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad580b5b10 con 0x7fad580b6310 2023-10-18T12:02:29.343 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.332+0000 7fad54ff9640 1 -- 172.21.15.125:0/2442307461 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40011c30 con 0x7fad580b6310 2023-10-18T12:02:29.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.332+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad580b4f60 con 0x7fad580b6310 2023-10-18T12:02:29.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.333+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad580690e0 con 0x7fad580b6310 2023-10-18T12:02:29.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.333+0000 7fad54ff9640 1 -- 172.21.15.125:0/2442307461 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fad400103c0 con 0x7fad580b6310 2023-10-18T12:02:29.344 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.333+0000 7fad54ff9640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fad3806af90 0x7fad3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.333+0000 7fad54ff9640 1 -- 172.21.15.125:0/2442307461 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fad4008a8c0 con 0x7fad580b6310 2023-10-18T12:02:29.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.333+0000 7fad56ffd640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fad3806af90 0x7fad3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.345 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.334+0000 7fad56ffd640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fad3806af90 0x7fad3806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad48034000 tx=0x7fad48039040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:29.346 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.337+0000 7fad54ff9640 1 -- 172.21.15.125:0/2442307461 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad400556e0 con 0x7fad580b6310 2023-10-18T12:02:29.508 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.506+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fad580b0ef0 con 0x7fad3806af90 2023-10-18T12:02:29.509 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.506+0000 7fad54ff9640 1 -- 172.21.15.125:0/2442307461 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fad580b0ef0 con 0x7fad3806af90 2023-10-18T12:02:29.509 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 7 to deep-scrub 2023-10-18T12:02:29.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fad3806af90 msgr2=0x7fad3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.511 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fad3806af90 0x7fad3806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad48034000 tx=0x7fad48039040 comp rx=0 tx=0).stop 2023-10-18T12:02:29.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad580b6310 msgr2=0x7fad580b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad580b6310 0x7fad580b4a20 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 crypto rx=0x7fad40003ed0 tx=0x7fad40001630 comp rx=0 tx=0).stop 2023-10-18T12:02:29.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 shutdown_connections 2023-10-18T12:02:29.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fad3806af90 0x7fad3806d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad580c72a0 0x7fad5809df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad580b6310 0x7fad580b4a20 unknown :-1 s=CLOSED pgs=3157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 --2- 172.21.15.125:0/2442307461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad440039a0 0x7fad58158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 >> 172.21.15.125:0/2442307461 conn(0x7fad580d0cf0 msgr2=0x7fad580ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:29.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 shutdown_connections 2023-10-18T12:02:29.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.508+0000 7fad5d8af640 1 -- 172.21.15.125:0/2442307461 wait complete. 2023-10-18T12:02:29.522 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:02:29.624 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 -- 172.21.15.125:0/2588821398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea00cb980 msgr2=0x7f5ea00cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 --2- 172.21.15.125:0/2588821398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea00cb980 0x7f5ea00cbdf0 secure :-1 s=READY pgs=3158 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00891a0 tx=0x7f5e9802c1d0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.625 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 -- 172.21.15.125:0/2588821398 shutdown_connections 2023-10-18T12:02:29.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 --2- 172.21.15.125:0/2588821398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea00cb980 0x7f5ea00cbdf0 unknown :-1 s=CLOSED pgs=3158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 --2- 172.21.15.125:0/2588821398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea00cd740 0x7f5ea00cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.626 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 --2- 172.21.15.125:0/2588821398 >> v2:172.21.15.139:3300/0 conn(0x7f5ea015aff0 0x7f5ea00d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 -- 172.21.15.125:0/2588821398 >> 172.21.15.125:0/2588821398 conn(0x7f5ea009a010 msgr2=0x7f5ea00d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:29.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 -- 172.21.15.125:0/2588821398 shutdown_connections 2023-10-18T12:02:29.627 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.622+0000 7f5ea5968640 1 -- 172.21.15.125:0/2588821398 wait complete. 2023-10-18T12:02:29.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.623+0000 7f5ea5968640 1 Processor -- start 2023-10-18T12:02:29.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.623+0000 7f5ea5968640 1 -- start start 2023-10-18T12:02:29.630 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.623+0000 7f5ea5968640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea00cb980 0x7f5ea00c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.623+0000 7f5ea5968640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea00cd740 0x7f5ea00c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.623+0000 7f5ea5968640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea015aff0 0x7f5ea00c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.623+0000 7f5ea5968640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea01f15e0 con 0x7f5ea015aff0 2023-10-18T12:02:29.631 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.623+0000 7f5ea5968640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00ca850 con 0x7f5ea00cd740 2023-10-18T12:02:29.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.623+0000 7f5ea5968640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00cace0 con 0x7f5ea00cb980 2023-10-18T12:02:29.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea015aff0 0x7f5ea00c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.632 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea015aff0 0x7f5ea00c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37734/0 (socket says 172.21.15.125:37734) 2023-10-18T12:02:29.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 -- 172.21.15.125:0/492907211 learned_addr learned my addr 172.21.15.125:0/492907211 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:29.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9effd640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea00cb980 0x7f5ea00c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.633 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e97fff640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea00cd740 0x7f5ea00c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 -- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea00cb980 msgr2=0x7f5ea00c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea00cb980 0x7f5ea00c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 -- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea00cd740 msgr2=0x7f5ea00c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.634 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea00cd740 0x7f5ea00c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 -- 172.21.15.125:0/492907211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009900 con 0x7f5ea015aff0 2023-10-18T12:02:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9f7fe640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea015aff0 0x7f5ea00c48f0 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 crypto rx=0x7f5e98037040 tx=0x7f5e98035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:29.635 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9cff9640 1 -- 172.21.15.125:0/492907211 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98034440 con 0x7f5ea015aff0 2023-10-18T12:02:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9cff9640 1 -- 172.21.15.125:0/492907211 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e98004540 con 0x7f5ea015aff0 2023-10-18T12:02:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea00d1d50 con 0x7f5ea015aff0 2023-10-18T12:02:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5e9cff9640 1 -- 172.21.15.125:0/492907211 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9803f440 con 0x7f5ea015aff0 2023-10-18T12:02:29.636 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.624+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea00c9f20 con 0x7f5ea015aff0 2023-10-18T12:02:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.625+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea00690e0 con 0x7f5ea015aff0 2023-10-18T12:02:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.626+0000 7f5e9cff9640 1 -- 172.21.15.125:0/492907211 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5e9803b040 con 0x7f5ea015aff0 2023-10-18T12:02:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.626+0000 7f5e9cff9640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5e8006af90 0x7f5e8006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:29.637 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.626+0000 7f5e9cff9640 1 -- 172.21.15.125:0/492907211 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f5e980ac550 con 0x7f5ea015aff0 2023-10-18T12:02:29.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.626+0000 7f5e9effd640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5e8006af90 0x7f5e8006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:29.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.627+0000 7f5e9effd640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5e8006af90 0x7f5e8006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000bb50 tx=0x7f5e9000b2e0 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:29.638 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.630+0000 7f5e9cff9640 1 -- 172.21.15.125:0/492907211 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e980773f0 con 0x7f5ea015aff0 2023-10-18T12:02:29.797 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.794+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ea00cc330 con 0x7f5e8006af90 2023-10-18T12:02:29.799 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.797+0000 7f5e9cff9640 1 -- 172.21.15.125:0/492907211 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50788 (secure 0 0 0) 0x7f5e90015070 con 0x7f5e8006af90 2023-10-18T12:02:29.800 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:29.801 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:02:29.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5e8006af90 msgr2=0x7f5e8006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.802 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5e8006af90 0x7f5e8006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000bb50 tx=0x7f5e9000b2e0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea015aff0 msgr2=0x7f5ea00c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:29.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea015aff0 0x7f5ea00c48f0 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 crypto rx=0x7f5e98037040 tx=0x7f5e98035db0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.803 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 shutdown_connections 2023-10-18T12:02:29.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f5e8006af90 0x7f5e8006d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea00cd740 0x7f5ea00c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5ea015aff0 0x7f5ea00c48f0 unknown :-1 s=CLOSED pgs=3159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.804 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 --2- 172.21.15.125:0/492907211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ea00cb980 0x7f5ea00c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:29.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 >> 172.21.15.125:0/492907211 conn(0x7f5ea009a010 msgr2=0x7f5ea00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:29.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 shutdown_connections 2023-10-18T12:02:29.805 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:29.799+0000 7f5ea5968640 1 -- 172.21.15.125:0/492907211 wait complete. 2023-10-18T12:02:29.812 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2023-10-18T12:02:28.902191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3813,"ondisk_log_size":3813,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":454808,"kb_used_data":27688,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754519912,"statfs":{"total":773094113280,"available":772628389888,"internally_reserved":0,"allocated":28352512,"data_stored":15232056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":437243448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-663619,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-120,"num_read_kb":16,"num_write":-196,"num_write_kb":-1395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001380"},"pg_stats":[{"pgid":"1.0","version":"265'3813","reported_seq":4900,"reported_epoch":267,"state":"active+clean","last_fresh":"2023-10-18T12:02:21.401052+0000","last_change":"2023-10-18T10:24:53.706145+0000","last_active":"2023-10-18T12:02:21.401052+0000","last_peered":"2023-10-18T12:02:21.401052+0000","last_clean":"2023-10-18T12:02:21.401052+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:02:21.401052+0000","last_undegraded":"2023-10-18T12:02:21.401052+0000","last_fullsized":"2023-10-18T12:02:21.401052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T10:24:52.613712+0000","last_clean_scrub_stamp":"2023-10-18T10:24:52.613712+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:14.880194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3813,"ondisk_log_size":3813,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576013,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71172,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300668,"statfs":{"total":96636764160,"available":96563884032,"internally_reserved":0,"allocated":3989504,"data_stored":2346783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.53100000000000003,"15min":0.57799999999999996},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.71799999999999997,"5min":0.80800000000000005,"15min":0.85999999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53500000000000003,"15min":0.59199999999999997},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.363},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.52700000000000002,"15min":0.57499999999999996},"min":{"1min":0.32300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.55600000000000005,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.54400000000000004,"15min":0.59099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59599999999999997,"5min":0.76900000000000002,"15min":0.86399999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.54700000000000004,"15min":0.59599999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.79600000000000004,"15min":0.83999999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.54400000000000004,"15min":0.58499999999999996},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.294,"5min":1.294,"15min":1.298},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.53100000000000003,"15min":0.58199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.64400000000000002,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.57499999999999996,"15min":0.61199999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.442,"15min":0.45300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.5249999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":0.81299999999999994},"last":0.33700000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.495,"15min":0.496},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56899999999999995,"15min":0.57899999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.4179999999999999},"last":0.434}]},{"osd":6,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55100000000000005,"15min":0.56200000000000006},"min":{"1min":0.33400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.363,"5min":1.363,"15min":1.478},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":0.83599999999999997,"15min":0.874},"last":0.372}]}]},{"osd":6,"up_from":13,"seq":55834576013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42816,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329024,"statfs":{"total":96636764160,"available":96592920576,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54100000000000004,"15min":0.57799999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56999999999999995,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65700000000000003,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.30299999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.54100000000000004,"15min":0.58499999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66900000000000004,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.58199999999999996,"15min":0.61399999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73399999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.433},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.55900000000000005,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.55900000000000005,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.34899999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.59499999999999997,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.58899999999999997,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.60599999999999998,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.47499999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.70199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.56799999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51800000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.872,"15min":0.872},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.75,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.503}]}]},{"osd":5,"up_from":13,"seq":55834576013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48388,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323452,"statfs":{"total":96636764160,"available":96587214848,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.56499999999999995,"15min":0.59299999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60999999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.58899999999999997,"15min":0.72099999999999997},"min":{"1min":0.308,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67400000000000004,"5min":1.081,"15min":1.1539999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.626,"15min":0.73299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67800000000000005,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.745}]},{"osd":2,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56000000000000005,"15min":0.59199999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.57099999999999995,"15min":0.60299999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.60599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65100000000000002,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.65100000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65000000000000002,"5min":0.81399999999999995,"15min":1},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.59899999999999998,"5min":0.82999999999999996,"15min":1.01},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.314,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.72399999999999998,"5min":0.872,"15min":0.97999999999999998},"last":0.71399999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.26900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.101},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.58099999999999996,"15min":0.59299999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70999999999999996,"5min":0.83399999999999996,"15min":1.0309999999999999},"last":0.72599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834576013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52548,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319292,"statfs":{"total":96636764160,"available":96582955008,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.22500000000000001},"max":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.81599999999999995},"last":0.499}]},{"osd":1,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.60999999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.81499999999999995,"5min":0.872,"15min":1.123},"last":0.61499999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73299999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.66500000000000004}]},{"osd":3,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.71999999999999997,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.27800000000000002},"max":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.626}]},{"osd":6,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34000000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.755,"5min":1.018,"15min":1.018},"last":0.54200000000000004}]},{"osd":7,"last update":"Wed Oct 18 12:01:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.74099999999999999,"5min":0.88900000000000001,"15min":1.05},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.64800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834576013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48388,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323452,"statfs":{"total":96636764160,"available":96587214848,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.34899999999999998,"15min":0.376},"min":{"1min":0.219,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.434,"5min":0.57099999999999995,"15min":0.61599999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.312,"5min":0.371,"15min":0.40000000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41499999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.40000000000000002},"min":{"1min":0.26000000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45600000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41099999999999998,"15min":0.43099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.441,"5min":1.899,"15min":1.899},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40500000000000003,"15min":0.44},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.42399999999999999,"15min":0.45400000000000001},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.65700000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61699999999999999,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.522,"15min":1.522},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.65600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.621,"15min":0.65200000000000002},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.42299999999999999}]},{"osd":7,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.66800000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.51800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834576013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65028,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306812,"statfs":{"total":96636764160,"available":96570175488,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36699999999999999,"15min":0.38400000000000001},"min":{"1min":0.20799999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.54800000000000004,"5min":0.64500000000000002,"15min":0.67200000000000004},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.38,"15min":0.40999999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.314}]},{"osd":1,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.41199999999999998},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.624},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.44500000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.38800000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.39400000000000002,"15min":0.41599999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48299999999999998,"5min":0.57499999999999996,"15min":0.64600000000000002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42999999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.50600000000000001,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63900000000000001,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92100000000000004,"15min":0.93100000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.67500000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.91300000000000003,"15min":1.087},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60999999999999999,"15min":0.64300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.65000000000000002},"min":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.60199999999999998,"15min":0.64200000000000002},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67400000000000004,"5min":0.879,"15min":0.95399999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64100000000000001,"15min":0.66800000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78200000000000003,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.628,"15min":0.66400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79900000000000004,"5min":0.86899999999999999,"15min":1.0609999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.627,"15min":0.66300000000000003},"min":{"1min":0.434,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834576013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65024,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306816,"statfs":{"total":96636764160,"available":96570179584,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.375,"15min":0.41899999999999998},"min":{"1min":0.218,"5min":0.217,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.73899999999999999,"15min":0.85599999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.35699999999999998,"15min":0.36899999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.42099999999999999,"5min":0.622,"15min":0.82599999999999996},"last":0.30499999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38700000000000001,"15min":0.436},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.65800000000000003,"15min":0.81000000000000005},"last":0.25},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.48699999999999999,"5min":0.68999999999999995,"15min":0.877},"last":0.35299999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.435,"15min":0.48899999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47899999999999998,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":0.74399999999999999,"15min":0.91000000000000003},"last":0.29299999999999998}]},{"osd":4,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.73399999999999999,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":0.89900000000000002,"15min":1.004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.91500000000000004,"15min":1.05},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.621,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.67400000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.65200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834576012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61444,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310396,"statfs":{"total":96636764160,"available":96573845504,"internally_reserved":0,"allocated":3989504,"data_stored":2346783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36799999999999999,"15min":0.38800000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.65600000000000003},"last":0.375},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38400000000000001,"15min":0.41199999999999998},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.27200000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.252,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.42899999999999999},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.35499999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.53600000000000003,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.33800000000000002}]},{"osd":4,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60199999999999998,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90900000000000003,"5min":0.96299999999999997,"15min":0.997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.64100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60699999999999998,"15min":0.64600000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96399999999999997},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.03},"last":0.439},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61099999999999999,"15min":0.66000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79200000000000004,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.67100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.63200000000000001,"15min":0.66900000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.997,"15min":0.997},"last":0.52700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:02:29.817 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T10:24:52.613712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=10, tm_min=24, tm_sec=52, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=12, tm_min=2, tm_sec=25, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T12:02:29.817 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T12:02:49.819 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T12:02:49.930 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.926+0000 7fa010b97640 1 -- 172.21.15.125:0/3078259649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 msgr2=0x7fa00c0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:49.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.926+0000 7fa010b97640 1 --2- 172.21.15.125:0/3078259649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 0x7fa00c0b0650 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 crypto rx=0x7fa00c08a810 tx=0x7f9ffc02bd30 comp rx=0 tx=0).stop 2023-10-18T12:02:49.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.926+0000 7fa010b97640 1 -- 172.21.15.125:0/3078259649 shutdown_connections 2023-10-18T12:02:49.931 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.926+0000 7fa010b97640 1 --2- 172.21.15.125:0/3078259649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 0x7fa00c0b0650 unknown :-1 s=CLOSED pgs=3160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:49.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.926+0000 7fa010b97640 1 --2- 172.21.15.125:0/3078259649 >> v2:172.21.15.139:3300/0 conn(0x7fa00c0cf340 0x7fa00c0d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:49.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.926+0000 7fa010b97640 1 -- 172.21.15.125:0/3078259649 >> 172.21.15.125:0/3078259649 conn(0x7fa00c0d0d60 msgr2=0x7fa00c0d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:49.932 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.926+0000 7fa010b97640 1 -- 172.21.15.125:0/3078259649 shutdown_connections 2023-10-18T12:02:49.933 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.926+0000 7fa010b97640 1 -- 172.21.15.125:0/3078259649 wait complete. 2023-10-18T12:02:49.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa010b97640 1 Processor -- start 2023-10-18T12:02:49.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa010b97640 1 -- start start 2023-10-18T12:02:49.936 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa010b97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 0x7fa00c153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:49.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa010b97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa00c0cf340 0x7fa00c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:49.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa010b97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa00c15b2a0 0x7fa00c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:49.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa010b97640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c0b4a60 con 0x7fa00c0ceb40 2023-10-18T12:02:49.937 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa010b97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c0b3e50 con 0x7fa00c0cf340 2023-10-18T12:02:49.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa010b97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c0b4130 con 0x7fa00c15b2a0 2023-10-18T12:02:49.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.928+0000 7fa00a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 0x7fa00c153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:49.938 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 0x7fa00c153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57372/0 (socket says 172.21.15.125:57372) 2023-10-18T12:02:49.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00a59c640 1 -- 172.21.15.125:0/1177970071 learned_addr learned my addr 172.21.15.125:0/1177970071 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:49.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00ad9d640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa00c15b2a0 0x7fa00c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:49.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa009d9b640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa00c0cf340 0x7fa00c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:49.939 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00a59c640 1 -- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa00c15b2a0 msgr2=0x7fa00c0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:49.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00a59c640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa00c15b2a0 0x7fa00c0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:49.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00a59c640 1 -- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa00c0cf340 msgr2=0x7fa00c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:49.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00a59c640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa00c0cf340 0x7fa00c15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:49.940 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00a59c640 1 -- 172.21.15.125:0/1177970071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc009560 con 0x7fa00c0ceb40 2023-10-18T12:02:49.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa00a59c640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 0x7fa00c153bb0 secure :-1 s=READY pgs=3161 cs=0 l=1 rev1=1 crypto rx=0x7fa00400ba50 tx=0x7fa00400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:49.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa002ffd640 1 -- 172.21.15.125:0/1177970071 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00400cbe0 con 0x7fa00c0ceb40 2023-10-18T12:02:49.941 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa002ffd640 1 -- 172.21.15.125:0/1177970071 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa00400ce50 con 0x7fa00c0ceb40 2023-10-18T12:02:49.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa002ffd640 1 -- 172.21.15.125:0/1177970071 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004014680 con 0x7fa00c0ceb40 2023-10-18T12:02:49.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c0b4410 con 0x7fa00c0ceb40 2023-10-18T12:02:49.942 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.929+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c1f5a50 con 0x7fa00c0ceb40 2023-10-18T12:02:49.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.930+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c1f5c70 con 0x7fa00c0ceb40 2023-10-18T12:02:49.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.931+0000 7fa002ffd640 1 -- 172.21.15.125:0/1177970071 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa00401d3b0 con 0x7fa00c0ceb40 2023-10-18T12:02:49.943 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.931+0000 7fa002ffd640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f9fec06af60 0x7f9fec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:49.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.931+0000 7fa002ffd640 1 -- 172.21.15.125:0/1177970071 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fa004089b10 con 0x7fa00c0ceb40 2023-10-18T12:02:49.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.931+0000 7fa009d9b640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f9fec06af60 0x7f9fec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:49.944 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.932+0000 7fa009d9b640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f9fec06af60 0x7f9fec06d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4006fd0 tx=0x7f9ff4008040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:49.945 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:49.935+0000 7fa002ffd640 1 -- 172.21.15.125:0/1177970071 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa004054990 con 0x7fa00c0ceb40 2023-10-18T12:02:50.101 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.097+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 --> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa00c0b4ba0 con 0x7f9fec06af60 2023-10-18T12:02:50.103 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.100+0000 7fa002ffd640 1 -- 172.21.15.125:0/1177970071 <== mgr.10178 v2:172.21.15.139:6832/1734365842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50739 (secure 0 0 0) 0x7f9ff400be30 con 0x7f9fec06af60 2023-10-18T12:02:50.104 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:50.105 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2023-10-18T12:02:50.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f9fec06af60 msgr2=0x7f9fec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.106 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f9fec06af60 0x7f9fec06d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4006fd0 tx=0x7f9ff4008040 comp rx=0 tx=0).stop 2023-10-18T12:02:50.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 msgr2=0x7fa00c153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 0x7fa00c153bb0 secure :-1 s=READY pgs=3161 cs=0 l=1 rev1=1 crypto rx=0x7fa00400ba50 tx=0x7fa00400bf10 comp rx=0 tx=0).stop 2023-10-18T12:02:50.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 shutdown_connections 2023-10-18T12:02:50.107 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f9fec06af60 0x7f9fec06d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa00c0cf340 0x7fa00c15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa00c15b2a0 0x7fa00c0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 --2- 172.21.15.125:0/1177970071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa00c0ceb40 0x7fa00c153bb0 unknown :-1 s=CLOSED pgs=3161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.108 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 >> 172.21.15.125:0/1177970071 conn(0x7fa00c0d0d60 msgr2=0x7fa00c15a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:50.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 shutdown_connections 2023-10-18T12:02:50.109 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.102+0000 7fa010b97640 1 -- 172.21.15.125:0/1177970071 wait complete. 2023-10-18T12:02:50.116 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T12:02:48.906740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3814,"ondisk_log_size":3814,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":454800,"kb_used_data":27688,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754519920,"statfs":{"total":773094113280,"available":772628398080,"internally_reserved":0,"allocated":28352512,"data_stored":15232056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":437243448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002857"},"pg_stats":[{"pgid":"1.0","version":"268'3814","reported_seq":4910,"reported_epoch":268,"state":"active+clean","last_fresh":"2023-10-18T12:02:31.232355+0000","last_change":"2023-10-18T12:02:31.232415+0000","last_active":"2023-10-18T12:02:31.232355+0000","last_peered":"2023-10-18T12:02:31.232355+0000","last_clean":"2023-10-18T12:02:31.232355+0000","last_became_active":"2023-10-18T10:24:53.705329+0000","last_became_peered":"2023-10-18T10:24:53.705329+0000","last_unstale":"2023-10-18T12:02:31.232355+0000","last_undegraded":"2023-10-18T12:02:31.232355+0000","last_fullsized":"2023-10-18T12:02:31.232355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"268'3814","last_scrub_stamp":"2023-10-18T12:02:31.232264+0000","last_deep_scrub":"268'3814","last_deep_scrub_stamp":"2023-10-18T12:02:31.232264+0000","last_clean_scrub_stamp":"2023-10-18T12:02:31.232264+0000","objects_scrubbed":2,"log_size":3814,"log_dups_size":0,"ondisk_log_size":3814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:48.400704+0000","scrub_duration":0.0074457669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2695,"num_read_kb":5659,"num_write":689,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3814,"ondisk_log_size":3814,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71168,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300672,"statfs":{"total":96636764160,"available":96563888128,"internally_reserved":0,"allocated":3989504,"data_stored":2346783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.53100000000000003,"15min":0.57799999999999996},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.71799999999999997,"5min":0.80800000000000005,"15min":0.85999999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53500000000000003,"15min":0.59199999999999997},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.363},"last":0.67800000000000005}]},{"osd":1,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.52700000000000002,"15min":0.57499999999999996},"min":{"1min":0.32300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.55600000000000005,"5min":0.78500000000000003,"15min":0.84099999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.54400000000000004,"15min":0.59099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59599999999999997,"5min":0.76900000000000002,"15min":0.86399999999999999},"last":0.70599999999999996}]},{"osd":2,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.54700000000000004,"15min":0.59599999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.79600000000000004,"15min":0.83999999999999997},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.54400000000000004,"15min":0.58499999999999996},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.294,"5min":1.294,"15min":1.298},"last":0.626}]},{"osd":3,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.53100000000000003,"15min":0.58199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.64400000000000002,"5min":0.81899999999999995,"15min":0.82699999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.57499999999999996,"15min":0.61199999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.80600000000000005,"15min":0.82499999999999996},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.442,"15min":0.45300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.5249999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":0.81299999999999994},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.495,"15min":0.496},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.90800000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56899999999999995,"15min":0.57899999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.4179999999999999},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Oct 18 12:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55100000000000005,"15min":0.56200000000000006},"min":{"1min":0.33400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.363,"5min":1.363,"15min":1.478},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":0.83599999999999997,"15min":0.874},"last":0.44400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42816,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329024,"statfs":{"total":96636764160,"available":96592920576,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54100000000000004,"15min":0.57799999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58699999999999997,"5min":0.82599999999999996,"15min":0.83699999999999997},"last":0.628},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56999999999999995,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65700000000000003,"5min":0.80800000000000005,"15min":0.82599999999999996},"last":0.53500000000000003}]},{"osd":1,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.54100000000000004,"15min":0.58499999999999996},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66900000000000004,"5min":0.78800000000000003,"15min":0.85699999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.58199999999999996,"15min":0.61399999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73399999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.55900000000000005,"15min":0.59999999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.55900000000000005,"5min":0.79600000000000004,"15min":0.81899999999999995},"last":0.73199999999999998}]},{"osd":3,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.59499999999999997,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.58899999999999997,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.60599999999999998,"5min":0.82999999999999996,"15min":0.84699999999999998},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.70199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.56799999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78200000000000003,"5min":0.872,"15min":0.872},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.75,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67900000000000005}]},{"osd":7,"last update":"Wed Oct 18 12:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.748},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.86799999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834576017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48388,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323452,"statfs":{"total":96636764160,"available":96587214848,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.56499999999999995,"15min":0.59299999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71799999999999997,"5min":0.84899999999999998,"15min":0.90800000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60999999999999999,"5min":0.83799999999999997,"15min":0.96099999999999997},"last":0.504}]},{"osd":1,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.58899999999999997,"15min":0.72099999999999997},"min":{"1min":0.308,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67400000000000004,"5min":1.081,"15min":1.1539999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.626,"15min":0.73299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67800000000000005,"5min":1.1879999999999999,"15min":1.2310000000000001},"last":0.54600000000000004}]},{"osd":2,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56000000000000005,"15min":0.59199999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.92700000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.57099999999999995,"15min":0.60299999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.81699999999999995,"15min":0.98399999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.57399999999999995,"15min":0.60599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65100000000000002,"5min":0.82499999999999996,"15min":1.0349999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.58099999999999996,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71799999999999997,"5min":0.81100000000000005,"15min":0.95199999999999996},"last":0.58399999999999996}]},{"osd":4,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48899999999999999,"15min":0.495},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65000000000000002,"5min":0.81399999999999995,"15min":1},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.96499999999999997,"15min":0.99099999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.59899999999999998,"5min":0.82999999999999996,"15min":1.01},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.314,"5min":0.307,"15min":0.29199999999999998},"max":{"1min":0.72399999999999998,"5min":0.872,"15min":0.97999999999999998},"last":0.69599999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.26900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.101},"last":0.63},{"interface":"front","average":{"1min":0.5,"5min":0.58099999999999996,"15min":0.59299999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70999999999999996,"5min":0.83399999999999996,"15min":1.0309999999999999},"last":0.52400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834576017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52548,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319292,"statfs":{"total":96636764160,"available":96582955008,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58499999999999996,"15min":0.60599999999999998},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.22500000000000001},"max":{"1min":0.68700000000000006,"5min":0.81499999999999995,"15min":0.89900000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56000000000000005,"15min":0.59199999999999997},"min":{"1min":0.376,"5min":0.35299999999999998,"15min":0.29399999999999998},"max":{"1min":0.72599999999999998,"5min":0.76300000000000001,"15min":0.81599999999999995},"last":0.56399999999999995}]},{"osd":1,"last update":"Wed Oct 18 12:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55500000000000005,"15min":0.60299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70799999999999996,"5min":0.86899999999999999,"15min":0.91900000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58499999999999996,"15min":0.60799999999999998},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.872,"15min":1.123},"last":0.52700000000000002}]},{"osd":2,"last update":"Wed Oct 18 12:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58199999999999996,"15min":0.60799999999999998},"min":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73999999999999999,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.57899999999999996,"15min":0.61499999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69499999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Oct 18 12:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":0.83199999999999996,"15min":0.91600000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.377,"5min":0.373,"15min":0.373},"max":{"1min":0.75600000000000001,"5min":0.84399999999999997,"15min":0.93700000000000006},"last":0.497}]},{"osd":5,"last update":"Wed Oct 18 12:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54900000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67500000000000004,"5min":0.78200000000000003,"15min":0.96199999999999997},"last":0.63}]},{"osd":6,"last update":"Wed Oct 18 12:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55800000000000005,"15min":0.55900000000000005},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68000000000000005,"5min":1.018,"15min":1.018},"last":0.68700000000000006}]},{"osd":7,"last update":"Wed Oct 18 12:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.55400000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.88900000000000001,"15min":1.05},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72499999999999998,"5min":0.88100000000000001,"15min":0.98499999999999999},"last":0.75}]}]},{"osd":3,"up_from":13,"seq":55834576017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48388,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323452,"statfs":{"total":96636764160,"available":96587214848,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.34899999999999998,"15min":0.376},"min":{"1min":0.219,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.434,"5min":0.57099999999999995,"15min":0.61599999999999999},"last":0.435},{"interface":"front","average":{"1min":0.312,"5min":0.371,"15min":0.40000000000000002},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.41499999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.375,"15min":0.40000000000000002},"min":{"1min":0.26000000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45600000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.41099999999999998,"15min":0.43099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.441,"5min":1.899,"15min":1.899},"last":0.504}]},{"osd":2,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40500000000000003,"15min":0.44},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.63400000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.42399999999999999,"15min":0.45400000000000001},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.61699999999999999,"15min":0.73399999999999999},"last":0.54500000000000004}]},{"osd":4,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.65700000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.88}]},{"osd":5,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61699999999999999,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.224,"5min":1.522,"15min":1.522},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63200000000000001,"15min":0.65600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.621,"15min":0.65200000000000002},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.94099999999999995}]},{"osd":7,"last update":"Wed Oct 18 12:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.66800000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65500000000000003,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.78000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834576017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65028,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306812,"statfs":{"total":96636764160,"available":96570175488,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36699999999999999,"15min":0.38400000000000001},"min":{"1min":0.20799999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.54800000000000004,"5min":0.64500000000000002,"15min":0.67200000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.38,"15min":0.40999999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.43099999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.39700000000000002,"15min":0.41199999999999998},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.57499999999999996,"5min":0.61199999999999999,"15min":0.624},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.44500000000000001,"5min":0.63100000000000001,"15min":0.746},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.39400000000000002,"15min":0.41599999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48299999999999998,"5min":0.57499999999999996,"15min":0.64600000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42999999999999999,"15min":0.45900000000000002},"min":{"1min":0.311,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.50600000000000001,"5min":0.65300000000000002,"15min":0.70899999999999996},"last":0.503}]},{"osd":4,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63900000000000001,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92100000000000004,"15min":0.93100000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.67500000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76500000000000001,"5min":0.91300000000000003,"15min":1.087},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60999999999999999,"15min":0.64300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":0.90400000000000003,"15min":0.95199999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61899999999999999,"15min":0.65000000000000002},"min":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73399999999999999,"5min":0.89100000000000001,"15min":0.98199999999999998},"last":0.81000000000000005}]},{"osd":6,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.60199999999999998,"15min":0.64200000000000002},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67400000000000004,"5min":0.879,"15min":0.95399999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.64100000000000001,"15min":0.66800000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78200000000000003,"5min":0.94699999999999995,"15min":1.0169999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Oct 18 12:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.628,"15min":0.66400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79900000000000004,"5min":0.86899999999999999,"15min":1.0609999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.627,"15min":0.66300000000000003},"min":{"1min":0.434,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.79200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834576017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65024,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306816,"statfs":{"total":96636764160,"available":96570179584,"internally_reserved":0,"allocated":3395584,"data_stored":1756415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.375,"15min":0.41899999999999998},"min":{"1min":0.218,"5min":0.217,"15min":0.191},"max":{"1min":0.41799999999999998,"5min":0.73899999999999999,"15min":0.85599999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.35699999999999998,"15min":0.36899999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.42099999999999999,"5min":0.622,"15min":0.82599999999999996},"last":0.49099999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38700000000000001,"15min":0.436},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.65800000000000003,"15min":0.81000000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.48699999999999999,"5min":0.68999999999999995,"15min":0.877},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.435,"15min":0.48899999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.47899999999999998,"5min":0.76700000000000002,"15min":0.92000000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.433,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57199999999999995,"5min":0.74399999999999999,"15min":0.91000000000000003},"last":0.56100000000000005}]},{"osd":4,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.317,"15min":0.29999999999999999},"max":{"1min":0.73399999999999999,"5min":0.85399999999999998,"15min":0.98599999999999999},"last":0.624},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":0.89900000000000002,"15min":1.004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.91500000000000004,"15min":1.05},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.89900000000000002,"15min":1.0209999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.621,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.67400000000000004,"5min":0.93300000000000005,"15min":1.0389999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed Oct 18 12:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":0.95399999999999996,"15min":0.97499999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77000000000000002,"5min":0.97099999999999997,"15min":1.0620000000000001},"last":0.74399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61440,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310400,"statfs":{"total":96636764160,"available":96573849600,"internally_reserved":0,"allocated":3989504,"data_stored":2346783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36799999999999999,"15min":0.38800000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.65600000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38400000000000001,"15min":0.41199999999999998},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.64200000000000002,"15min":0.66200000000000003},"last":0.49199999999999999}]},{"osd":2,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.252,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.42899999999999999},"min":{"1min":0.26500000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.43099999999999999,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.53600000000000003,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60199999999999998,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90900000000000003,"5min":0.96299999999999997,"15min":0.997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59099999999999997,"15min":0.64100000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":0.89600000000000002,"15min":0.95599999999999996},"last":0.752}]},{"osd":5,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60699999999999998,"15min":0.64600000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.623},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.96399999999999997},"last":0.71599999999999997}]},{"osd":6,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.624,"15min":0.66000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.03},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61099999999999999,"15min":0.66000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79200000000000004,"5min":0.93000000000000005,"15min":0.95099999999999996},"last":0.78200000000000003}]},{"osd":7,"last update":"Wed Oct 18 12:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.67100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97299999999999998,"15min":1.0109999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.63200000000000001,"15min":0.66900000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.997,"15min":0.997},"last":0.66000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T12:02:50.118 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-18T12:02:50.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.217+0000 7f5801501640 1 -- 172.21.15.125:0/3738562129 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc159bd0 msgr2=0x7f57fc15c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.221 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.217+0000 7f5801501640 1 --2- 172.21.15.125:0/3738562129 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc159bd0 0x7f57fc15c770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005fd0 tx=0x7f57f002bd90 comp rx=0 tx=0).stop 2023-10-18T12:02:50.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.217+0000 7f5801501640 1 -- 172.21.15.125:0/3738562129 shutdown_connections 2023-10-18T12:02:50.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.217+0000 7f5801501640 1 --2- 172.21.15.125:0/3738562129 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc159bd0 0x7f57fc15c770 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.222 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.217+0000 7f5801501640 1 --2- 172.21.15.125:0/3738562129 >> v2:172.21.15.139:3300/0 conn(0x7f57fc06b030 0x7f57fc06b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.217+0000 7f5801501640 1 -- 172.21.15.125:0/3738562129 >> 172.21.15.125:0/3738562129 conn(0x7f57fc069930 msgr2=0x7f57fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:50.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.217+0000 7f5801501640 1 -- 172.21.15.125:0/3738562129 shutdown_connections 2023-10-18T12:02:50.223 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.217+0000 7f5801501640 1 -- 172.21.15.125:0/3738562129 wait complete. 2023-10-18T12:02:50.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.218+0000 7f5801501640 1 Processor -- start 2023-10-18T12:02:50.226 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.218+0000 7f5801501640 1 -- start start 2023-10-18T12:02:50.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f5801501640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc06b030 0x7f57fc14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:50.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f5801501640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc159bd0 0x7f57fc15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:50.227 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f5801501640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15ccb0 0x7f57fc15abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:50.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f5801501640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc12ba90 con 0x7f57fc15ccb0 2023-10-18T12:02:50.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f5801501640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc12ae80 con 0x7f57fc159bd0 2023-10-18T12:02:50.228 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f5801501640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc12b160 con 0x7f57fc06b030 2023-10-18T12:02:50.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15ccb0 0x7f57fc15abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:50.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15ccb0 0x7f57fc15abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57376/0 (socket says 172.21.15.125:57376) 2023-10-18T12:02:50.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 -- 172.21.15.125:0/1016062890 learned_addr learned my addr 172.21.15.125:0/1016062890 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:50.229 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57faffd640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc06b030 0x7f57fc14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:50.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fa7fc640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc159bd0 0x7f57fc15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:50.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 -- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc06b030 msgr2=0x7f57fc14ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.230 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc06b030 0x7f57fc14ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 -- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc159bd0 msgr2=0x7f57fc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc159bd0 0x7f57fc15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 -- 172.21.15.125:0/1016062890 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009490 con 0x7f57fc15ccb0 2023-10-18T12:02:50.231 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57fb7fe640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15ccb0 0x7f57fc15abc0 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c680 tx=0x7f57f400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:50.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57e37fe640 1 -- 172.21.15.125:0/1016062890 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4003b70 con 0x7f57fc15ccb0 2023-10-18T12:02:50.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57e37fe640 1 -- 172.21.15.125:0/1016062890 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57f4003cd0 con 0x7f57fc15ccb0 2023-10-18T12:02:50.232 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f0009560 con 0x7f57fc15ccb0 2023-10-18T12:02:50.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f57e37fe640 1 -- 172.21.15.125:0/1016062890 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4005330 con 0x7f57fc15ccb0 2023-10-18T12:02:50.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.219+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc12b720 con 0x7f57fc15ccb0 2023-10-18T12:02:50.233 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.220+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc12bc00 con 0x7f57fc15ccb0 2023-10-18T12:02:50.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.221+0000 7f57e37fe640 1 -- 172.21.15.125:0/1016062890 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f57f4005490 con 0x7f57fc15ccb0 2023-10-18T12:02:50.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.221+0000 7f57e37fe640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f57e806af90 0x7f57e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:50.234 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.221+0000 7f57e37fe640 1 -- 172.21.15.125:0/1016062890 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f57f4089870 con 0x7f57fc15ccb0 2023-10-18T12:02:50.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.221+0000 7f57faffd640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f57e806af90 0x7f57e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:50.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.222+0000 7f57faffd640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f57e806af90 0x7f57e806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009cf0 tx=0x7f57ec005f90 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:50.235 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.225+0000 7f57e37fe640 1 -- 172.21.15.125:0/1016062890 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57f40545e0 con 0x7f57fc15ccb0 2023-10-18T12:02:50.392 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.389+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f57fc0690e0 con 0x7f57fc15ccb0 2023-10-18T12:02:50.395 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.392+0000 7f57e37fe640 1 -- 172.21.15.125:0/1016062890 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v87)=0 v87) v1 ==== 123+0+0 (secure 0 0 0) 0x7f57f4057c00 con 0x7f57fc15ccb0 2023-10-18T12:02:50.397 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.393+0000 7f57e37fe640 1 -- 172.21.15.125:0/1016062890 <== mon.0 v2:172.21.15.125:3300/0 8 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7f57f4001760 con 0x7f57fc15ccb0 2023-10-18T12:02:50.398 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.394+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f57e806af90 msgr2=0x7f57e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.394+0000 7f5801501640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f57e806af90 0x7f57e806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009cf0 tx=0x7f57ec005f90 comp rx=0 tx=0).stop 2023-10-18T12:02:50.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.394+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15ccb0 msgr2=0x7f57fc15abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.399 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.394+0000 7f5801501640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15ccb0 0x7f57fc15abc0 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c680 tx=0x7f57f400cb40 comp rx=0 tx=0).stop 2023-10-18T12:02:50.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.394+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 shutdown_connections 2023-10-18T12:02:50.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.394+0000 7f5801501640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7f57e806af90 0x7f57e806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.394+0000 7f5801501640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57fc159bd0 0x7f57fc15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.400 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.394+0000 7f5801501640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57fc15ccb0 0x7f57fc15abc0 unknown :-1 s=CLOSED pgs=3162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.395+0000 7f5801501640 1 --2- 172.21.15.125:0/1016062890 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f57fc06b030 0x7f57fc14ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.395+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 >> 172.21.15.125:0/1016062890 conn(0x7f57fc069930 msgr2=0x7f57fc15bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:50.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.395+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 shutdown_connections 2023-10-18T12:02:50.401 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.395+0000 7f5801501640 1 -- 172.21.15.125:0/1016062890 wait complete. 2023-10-18T12:02:50.409 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T12:02:50.409 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T12:02:50.512 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.508+0000 7fd4aa17d640 1 -- 172.21.15.125:0/3442367673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a406b410 msgr2=0x7fd4a415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.508+0000 7fd4aa17d640 1 --2- 172.21.15.125:0/3442367673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a406b410 0x7fd4a415c770 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 crypto rx=0x7fd498009830 tx=0x7fd49802bcd0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.513 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.509+0000 7fd4aa17d640 1 -- 172.21.15.125:0/3442367673 shutdown_connections 2023-10-18T12:02:50.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.509+0000 7fd4aa17d640 1 --2- 172.21.15.125:0/3442367673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a406b410 0x7fd4a415c770 unknown :-1 s=CLOSED pgs=3164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.509+0000 7fd4aa17d640 1 -- 172.21.15.125:0/3442367673 >> 172.21.15.125:0/3442367673 conn(0x7fd4a4069930 msgr2=0x7fd4a4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:50.514 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.509+0000 7fd4aa17d640 1 -- 172.21.15.125:0/3442367673 shutdown_connections 2023-10-18T12:02:50.515 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.509+0000 7fd4aa17d640 1 -- 172.21.15.125:0/3442367673 wait complete. 2023-10-18T12:02:50.517 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.510+0000 7fd4aa17d640 1 Processor -- start 2023-10-18T12:02:50.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.510+0000 7fd4aa17d640 1 -- start start 2023-10-18T12:02:50.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.510+0000 7fd4aa17d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4a406b030 0x7fd4a40a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:50.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4aa17d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4a406b410 0x7fd4a414ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:50.518 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4aa17d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a415ccb0 0x7fd4a40aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:50.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4aa17d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a413f1e0 con 0x7fd4a415ccb0 2023-10-18T12:02:50.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4aa17d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40a90e0 con 0x7fd4a406b410 2023-10-18T12:02:50.519 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4aa17d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a413e9d0 con 0x7fd4a406b030 2023-10-18T12:02:50.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a415ccb0 0x7fd4a40aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:50.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a415ccb0 0x7fd4a40aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:57400/0 (socket says 172.21.15.125:57400) 2023-10-18T12:02:50.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 -- 172.21.15.125:0/506018077 learned_addr learned my addr 172.21.15.125:0/506018077 (peer_addr_for_me v2:172.21.15.125:0/0) 2023-10-18T12:02:50.520 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a2ffd640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4a406b410 0x7fd4a414ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:50.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a37fe640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4a406b030 0x7fd4a40a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:50.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 -- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4a406b030 msgr2=0x7fd4a40a8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4a406b030 0x7fd4a40a8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.521 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 -- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4a406b410 msgr2=0x7fd4a414ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4a406b410 0x7fd4a414ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 -- 172.21.15.125:0/506018077 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd498009490 con 0x7fd4a415ccb0 2023-10-18T12:02:50.522 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4a3fff640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a415ccb0 0x7fd4a40aa4d0 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00b9e0 tx=0x7fd49c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:50.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd48bfff640 1 -- 172.21.15.125:0/506018077 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c003b70 con 0x7fd4a415ccb0 2023-10-18T12:02:50.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd48bfff640 1 -- 172.21.15.125:0/506018077 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd49c003cd0 con 0x7fd4a415ccb0 2023-10-18T12:02:50.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a40ca0f0 con 0x7fd4a415ccb0 2023-10-18T12:02:50.523 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd48bfff640 1 -- 172.21.15.125:0/506018077 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00cb80 con 0x7fd4a415ccb0 2023-10-18T12:02:50.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.511+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a413ef00 con 0x7fd4a415ccb0 2023-10-18T12:02:50.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.512+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a406c570 con 0x7fd4a415ccb0 2023-10-18T12:02:50.524 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.513+0000 7fd48bfff640 1 -- 172.21.15.125:0/506018077 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd49c00cce0 con 0x7fd4a415ccb0 2023-10-18T12:02:50.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.513+0000 7fd48bfff640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd49006af60 0x7fd49006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T12:02:50.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.513+0000 7fd48bfff640 1 -- 172.21.15.125:0/506018077 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fd49c089910 con 0x7fd4a415ccb0 2023-10-18T12:02:50.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.513+0000 7fd4a37fe640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd49006af60 0x7fd49006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T12:02:50.525 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.514+0000 7fd4a37fe640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd49006af60 0x7fd49006d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd494000f60 tx=0x7fd494009040 comp rx=0 tx=0).ready entity=mgr.10178 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T12:02:50.526 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.517+0000 7fd48bfff640 1 -- 172.21.15.125:0/506018077 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd49c054620 con 0x7fd4a415ccb0 2023-10-18T12:02:50.720 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.716+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4a40cb9e0 con 0x7fd4a415ccb0 2023-10-18T12:02:50.721 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.717+0000 7fd48bfff640 1 -- 172.21.15.125:0/506018077 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+3719 (secure 0 0 0) 0x7fd49c057c40 con 0x7fd4a415ccb0 2023-10-18T12:02:50.722 INFO:teuthology.orchestra.run.smithi125.stdout: 2023-10-18T12:02:50.722 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9986,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3962342896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3962342896},{"type":"v1","addr":"172.21.15.139:6837","nonce":3962342896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11413,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1953593032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1953593032},{"type":"v1","addr":"172.21.15.139:6836","nonce":1953593032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":13566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6834/1230654530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1230654530},{"type":"v1","addr":"172.21.15.125:6834","nonce":1230654530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":13971,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1353617818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6833","nonce":1353617818},{"type":"v1","addr":"172.21.15.125:6835","nonce":1353617818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[]} 2023-10-18T12:02:50.723 INFO:teuthology.orchestra.run.smithi125.stderr:dumped fsmap epoch 114 2023-10-18T12:02:50.723 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd49006af60 msgr2=0x7fd49006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd49006af60 0x7fd49006d420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd494000f60 tx=0x7fd494009040 comp rx=0 tx=0).stop 2023-10-18T12:02:50.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a415ccb0 msgr2=0x7fd4a40aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T12:02:50.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a415ccb0 0x7fd4a40aa4d0 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00b9e0 tx=0x7fd49c00bea0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.724 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 shutdown_connections 2023-10-18T12:02:50.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:6832/1734365842,v1:172.21.15.139:6833/1734365842] conn(0x7fd49006af60 0x7fd49006d420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4a406b410 0x7fd4a414ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.725 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4a415ccb0 0x7fd4a40aa4d0 unknown :-1 s=CLOSED pgs=3165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 --2- 172.21.15.125:0/506018077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4a406b030 0x7fd4a40a8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T12:02:50.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 >> 172.21.15.125:0/506018077 conn(0x7fd4a4069930 msgr2=0x7fd4a40b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T12:02:50.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 shutdown_connections 2023-10-18T12:02:50.726 INFO:teuthology.orchestra.run.smithi125.stderr:2023-10-18T12:02:50.719+0000 7fd4aa17d640 1 -- 172.21.15.125:0/506018077 wait complete. 2023-10-18T12:02:50.733 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-18T12:02:50.734 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-18T12:02:50.734 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T12:02:50.735 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:50.743 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T12:02:50.743 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T12:02:50.744 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:50.792 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T12:02:50.793 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T12:02:50.793 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:50.804 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T12:02:50.805 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T12:02:50.806 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:50.856 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T12:02:50.857 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-18T12:02:50.858 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-18T12:02:50.858 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:50.884 INFO:tasks.ceph.osd.0:Stopped 2023-10-18T12:02:50.885 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-18T12:02:50.886 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:50.954 INFO:tasks.ceph.osd.1:Stopped 2023-10-18T12:02:50.955 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-18T12:02:50.956 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.025 INFO:tasks.ceph.osd.2:Stopped 2023-10-18T12:02:51.025 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-18T12:02:51.026 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.094 INFO:tasks.ceph.osd.3:Stopped 2023-10-18T12:02:51.094 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-18T12:02:51.095 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.128 INFO:tasks.ceph.osd.4:Stopped 2023-10-18T12:02:51.129 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-18T12:02:51.129 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.198 INFO:tasks.ceph.osd.5:Stopped 2023-10-18T12:02:51.199 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-18T12:02:51.199 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.267 INFO:tasks.ceph.osd.6:Stopped 2023-10-18T12:02:51.268 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-18T12:02:51.268 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.336 INFO:tasks.ceph.osd.7:Stopped 2023-10-18T12:02:51.337 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-18T12:02:51.338 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-18T12:02:51.338 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.364 INFO:tasks.ceph.mgr.y:Stopped 2023-10-18T12:02:51.365 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-18T12:02:51.365 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.409 INFO:tasks.ceph.mgr.x:Stopped 2023-10-18T12:02:51.410 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-18T12:02:51.411 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-18T12:02:51.411 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.426 INFO:tasks.ceph.mon.a:Stopped 2023-10-18T12:02:51.427 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-18T12:02:51.427 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.464 INFO:tasks.ceph.mon.b:Stopped 2023-10-18T12:02:51.465 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-18T12:02:51.465 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T12:02:51.518 INFO:tasks.ceph.mon.c:Stopped 2023-10-18T12:02:51.519 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-18T12:02:51.520 DEBUG:teuthology.orchestra.run.smithi125:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-18T12:02:51.551 INFO:teuthology.orchestra.run.smithi125.stdout:2023-10-18T10:39:11.392624+0000 mon.a (mon.0) 605 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-10-18T12:02:51.552 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-18T12:02:51.552 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:02:51.553 DEBUG:teuthology.orchestra.run.smithi125:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-18T12:02:51.781 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:02:51.782 DEBUG:teuthology.orchestra.run.smithi125:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-18T12:02:51.865 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:02:51.865 DEBUG:teuthology.orchestra.run.smithi125:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-18T12:02:51.923 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:02:51.923 DEBUG:teuthology.orchestra.run.smithi125:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-18T12:02:51.980 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:02:51.981 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-18T12:02:52.127 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:02:52.128 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-18T12:02:52.205 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:02:52.206 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-18T12:02:52.254 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:02:52.254 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-18T12:02:52.305 INFO:tasks.ceph:Archiving mon data... 2023-10-18T12:02:52.307 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/data/mon.a.tgz 2023-10-18T12:02:52.308 DEBUG:teuthology.orchestra.run.smithi125:> mktemp 2023-10-18T12:02:52.323 INFO:teuthology.orchestra.run.smithi125.stdout:/tmp/tmp.W4EWnxy00r 2023-10-18T12:02:52.323 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.W4EWnxy00r 2023-10-18T12:02:53.007 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0666 /tmp/tmp.W4EWnxy00r 2023-10-18T12:02:53.092 DEBUG:teuthology.orchestra.remote:smithi125:/tmp/tmp.W4EWnxy00r is 4MB 2023-10-18T12:02:53.229 DEBUG:teuthology.orchestra.run.smithi125:> rm -fr /tmp/tmp.W4EWnxy00r 2023-10-18T12:02:53.244 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/data/mon.b.tgz 2023-10-18T12:02:53.245 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2023-10-18T12:02:53.263 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.PhcXY4lP4k 2023-10-18T12:02:53.264 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.PhcXY4lP4k 2023-10-18T12:02:53.929 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.PhcXY4lP4k 2023-10-18T12:02:54.014 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.PhcXY4lP4k is 4MB 2023-10-18T12:02:54.169 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.PhcXY4lP4k 2023-10-18T12:02:54.184 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/data/mon.c.tgz 2023-10-18T12:02:54.185 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2023-10-18T12:02:54.238 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.VE2xyb2lEp 2023-10-18T12:02:54.239 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.VE2xyb2lEp 2023-10-18T12:02:54.906 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.VE2xyb2lEp 2023-10-18T12:02:54.991 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.VE2xyb2lEp is 4MB 2023-10-18T12:02:55.154 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.VE2xyb2lEp 2023-10-18T12:02:55.170 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-18T12:02:55.171 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T12:02:55.174 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T12:02:55.235 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-18T12:02:55.236 DEBUG:teuthology.orchestra.run.smithi125:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T12:02:55.245 DEBUG:teuthology.orchestra.run.smithi139:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T12:02:55.274 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T12:02:55.303 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T12:02:55.307 INFO:tasks.ceph:Archiving crash dumps... 2023-10-18T12:02:55.310 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/remote/smithi125/crash 2023-10-18T12:02:55.311 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T12:02:55.347 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/remote/smithi139/crash 2023-10-18T12:02:55.349 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T12:02:55.375 INFO:tasks.ceph:Shutting down logrotate 2023-10-18T12:02:55.376 DEBUG:teuthology.orchestra.run.smithi125:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:02:55.393 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T12:02:55.691 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-18T12:02:55.720 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-18T12:02:55.748 INFO:tasks.ceph:Compressing logs... 2023-10-18T12:02:55.748 DEBUG:teuthology.orchestra.run.smithi125:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T12:02:55.763 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T12:02:55.789 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101176.log 2023-10-18T12:02:55.790 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161056.log 2023-10-18T12:02:55.790 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158598.log 2023-10-18T12:02:55.792 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96774.log 2023-10-18T12:02:55.792 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161056.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101176.log.gz 2023-10-18T12:02:55.792 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135600.log 2023-10-18T12:02:55.792 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.161056.log.gz 2023-10-18T12:02:55.793 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.158598.log.gz 2023-10-18T12:02:55.833 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-18T12:02:55.833 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87878.log 2023-10-18T12:02:55.834 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.94882.log 2023-10-18T12:02:55.834 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.87878.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T12:02:55.834 INFO:teuthology.orchestra.run.smithi139.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.87878.log.gz 2023-10-18T12:02:55.835 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.3.94882.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-18T12:02:55.835 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87798.log 2023-10-18T12:02:55.835 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-18T12:02:55.835 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.87798.log: 91.2% -- replaced with /var/log/ceph/ceph-client.3.94882.log.gz 2023-10-18T12:02:55.836 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87598.log 2023-10-18T12:02:55.836 INFO:teuthology.orchestra.run.smithi139.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.87798.log.gz 2023-10-18T12:02:55.836 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87758.log 2023-10-18T12:02:55.836 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-18T12:02:55.837 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.87598.log: /var/log/ceph/ceph-client.admin.87758.log: gzip 86.6% -- replaced with /var/log/ceph/ceph-client.admin.87598.log.gz 2023-10-18T12:02:55.837 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-18T12:02:55.837 INFO:teuthology.orchestra.run.smithi139.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.87758.log.gz 2023-10-18T12:02:55.838 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87838.log 2023-10-18T12:02:55.838 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5/var/log/ceph/ceph-osd.5.log: --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-18T12:02:55.838 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.87838.log: gzip -5 --verbose 86.5% -- replaced with /var/log/ceph/ceph-client.admin.87838.log.gz 2023-10-18T12:02:55.838 INFO:teuthology.orchestra.run.smithi139.stderr: -- /var/log/ceph/ceph-client.admin.87718.log 2023-10-18T12:02:55.839 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87638.log 2023-10-18T12:02:55.839 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.87718.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.87718.log.gz 2023-10-18T12:02:55.839 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T12:02:55.839 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.87638.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.87638.log.gz 2023-10-18T12:02:55.840 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-18T12:02:55.840 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T12:02:55.840 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94437.log 2023-10-18T12:02:55.844 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mon.b.log: 94.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T12:02:55.844 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87678.log 2023-10-18T12:02:55.845 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.94437.log: 90.0% -- replaced with /var/log/ceph/ceph-client.1.94437.log.gz 2023-10-18T12:02:55.845 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-18T12:02:55.845 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.87678.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.87678.log.gz 2023-10-18T12:02:55.846 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-18T12:02:55.846 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mds.d.log: 93.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-18T12:02:56.206 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95371.log 2023-10-18T12:02:56.207 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135600.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.96774.log.gz --verbose 2023-10-18T12:02:56.207 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.113303.log 2023-10-18T12:02:56.207 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135600.log.gz 2023-10-18T12:02:56.207 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154452.log 2023-10-18T12:02:56.208 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95371.log.gz 2023-10-18T12:02:56.208 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167347.log 2023-10-18T12:02:56.208 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113303.log.gz 2023-10-18T12:02:56.209 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103999.log 2023-10-18T12:02:56.209 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.154452.log.gz 2023-10-18T12:02:56.209 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145695.log 2023-10-18T12:02:56.209 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4%/var/log/ceph/ceph-client.admin.103999.log: -- replaced with /var/log/ceph/ceph-client.admin.167347.log.gz 2023-10-18T12:02:56.210 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142603.log 2023-10-18T12:02:56.210 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103999.log.gz 2023-10-18T12:02:56.210 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142519.log 2023-10-18T12:02:56.210 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6%/var/log/ceph/ceph-client.admin.142603.log: -- replaced with /var/log/ceph/ceph-client.admin.145695.log.gz 2023-10-18T12:02:56.211 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97830.log 2023-10-18T12:02:56.211 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142603.log.gz 2023-10-18T12:02:56.211 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.136921.log 2023-10-18T12:02:56.212 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.142519.log.gz 2023-10-18T12:02:56.212 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143614.log 2023-10-18T12:02:56.212 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97830.log.gz 2023-10-18T12:02:56.212 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.136921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122148.log 2023-10-18T12:02:56.213 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88151.log 2023-10-18T12:02:56.213 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.143614.log.gz 2023-10-18T12:02:56.213 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159202.log 2023-10-18T12:02:56.213 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.122148.log.gz 2023-10-18T12:02:56.214 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.88151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157011.log 2023-10-18T12:02:56.214 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159202.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.88151.log.gz 2023-10-18T12:02:56.214 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135808.log 2023-10-18T12:02:56.215 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.159202.log.gz 2023-10-18T12:02:56.215 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152809.log 2023-10-18T12:02:56.215 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.157011.log.gz 2023-10-18T12:02:56.215 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141785.log 2023-10-18T12:02:56.216 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.152809.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135808.log.gz 2023-10-18T12:02:56.216 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132183.log 2023-10-18T12:02:56.216 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.152809.log.gz 2023-10-18T12:02:56.217 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113173.log 2023-10-18T12:02:56.217 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.141785.log.gz 2023-10-18T12:02:56.217 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142252.log 2023-10-18T12:02:56.217 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113173.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.132183.log.gz 2023-10-18T12:02:56.218 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97495.log 2023-10-18T12:02:56.218 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113173.log.gz 2023-10-18T12:02:56.218 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105020.log 2023-10-18T12:02:56.218 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.142252.log.gz 2023-10-18T12:02:56.219 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107227.log 2023-10-18T12:02:56.219 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105020.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97495.log.gz 2023-10-18T12:02:56.219 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105748.log 2023-10-18T12:02:56.220 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.105020.log.gz 2023-10-18T12:02:56.220 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107227.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-18T12:02:56.220 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.107227.log.gz 2023-10-18T12:02:56.220 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123358.log 2023-10-18T12:02:56.221 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130750.log 2023-10-18T12:02:56.221 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123358.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.105748.log.gz 2023-10-18T12:02:56.221 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95241.log 2023-10-18T12:02:56.221 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123358.log.gz 2023-10-18T12:02:56.222 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128145.log 2023-10-18T12:02:56.222 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130750.log: /var/log/ceph/ceph-client.admin.95241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149192.log 2023-10-18T12:02:56.222 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.130750.log.gz 2023-10-18T12:02:56.223 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95241.log.gz 2023-10-18T12:02:56.223 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157763.log 2023-10-18T12:02:56.223 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.128145.log.gz 2023-10-18T12:02:56.223 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145481.log 2023-10-18T12:02:56.224 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149192.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149192.log.gz 2023-10-18T12:02:56.224 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.143307.log 2023-10-18T12:02:56.224 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113395.log 2023-10-18T12:02:56.224 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145481.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157763.log.gz 2023-10-18T12:02:56.225 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.143307.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.145481.log.gz 2023-10-18T12:02:56.225 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162286.log 2023-10-18T12:02:56.225 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130579.log 2023-10-18T12:02:56.225 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113395.log: /var/log/ceph/ceph-client.admin.162286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131199.log 2023-10-18T12:02:56.226 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113395.log.gz 2023-10-18T12:02:56.226 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.162286.log.gz 2023-10-18T12:02:56.226 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164464.log 2023-10-18T12:02:56.227 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.130579.log.gz 2023-10-18T12:02:56.227 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150168.log 2023-10-18T12:02:56.227 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164464.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131199.log.gz 2023-10-18T12:02:56.227 INFO:teuthology.orchestra.run.smithi125.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.143307.log.gz 2023-10-18T12:02:56.228 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154280.log 2023-10-18T12:02:56.228 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.164464.log.gz 2023-10-18T12:02:56.228 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125357.log 2023-10-18T12:02:56.228 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.150168.log.gz 2023-10-18T12:02:56.229 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145439.log 2023-10-18T12:02:56.229 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154280.log: /var/log/ceph/ceph-client.admin.125357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131326.log 2023-10-18T12:02:56.229 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154280.log.gz 2023-10-18T12:02:56.230 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125357.log.gz 2023-10-18T12:02:56.230 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159244.log 2023-10-18T12:02:56.230 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130307.log 2023-10-18T12:02:56.230 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145439.log.gz 2023-10-18T12:02:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131326.log: /var/log/ceph/ceph-client.admin.159244.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131326.log.gz 2023-10-18T12:02:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96978.log 2023-10-18T12:02:56.231 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.159244.log.gz 2023-10-18T12:02:56.231 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134692.log 2023-10-18T12:02:56.232 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130307.log: /var/log/ceph/ceph-client.admin.96978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153519.log 2023-10-18T12:02:56.232 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130307.log.gz 2023-10-18T12:02:56.232 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134692.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96978.log.gz 2023-10-18T12:02:56.233 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162725.log 2023-10-18T12:02:56.233 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.134692.log.gz 2023-10-18T12:02:56.233 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106958.log 2023-10-18T12:02:56.233 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.153519.log.gz 2023-10-18T12:02:56.234 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123892.log 2023-10-18T12:02:56.234 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162725.log: /var/log/ceph/ceph-client.admin.106958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165979.log 2023-10-18T12:02:56.234 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.162725.log.gz 2023-10-18T12:02:56.235 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106958.log.gz 2023-10-18T12:02:56.235 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148922.log 2023-10-18T12:02:56.235 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.123892.log.gz -5 2023-10-18T12:02:56.235 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-client.admin.123400.log 2023-10-18T12:02:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165979.log: /var/log/ceph/ceph-client.admin.148922.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.165979.log.gz 2023-10-18T12:02:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149914.log 2023-10-18T12:02:56.236 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148922.log.gz 2023-10-18T12:02:56.236 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120640.log 2023-10-18T12:02:56.237 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123400.log: /var/log/ceph/ceph-client.admin.149914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133213.log 2023-10-18T12:02:56.237 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123400.log.gz 2023-10-18T12:02:56.237 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.149914.log.gz 2023-10-18T12:02:56.238 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108374.log 2023-10-18T12:02:56.238 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120640.log.gz 2023-10-18T12:02:56.238 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126246.log 2023-10-18T12:02:56.238 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.133213.log.gz 2023-10-18T12:02:56.239 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108172.log 2023-10-18T12:02:56.239 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108374.log: /var/log/ceph/ceph-client.admin.126246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167513.log 2023-10-18T12:02:56.239 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.108374.log.gz 2023-10-18T12:02:56.239 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.126246.log.gz 2023-10-18T12:02:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94655.log 2023-10-18T12:02:56.240 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- 86.9% /var/log/ceph/ceph-client.admin.133255.log 2023-10-18T12:02:56.240 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108172.log.gz 2023-10-18T12:02:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167513.log: /var/log/ceph/ceph-client.admin.94655.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.167513.log.gz 2023-10-18T12:02:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165800.log 2023-10-18T12:02:56.241 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94655.log.gz 2023-10-18T12:02:56.241 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113607.log 2023-10-18T12:02:56.242 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133255.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.165800.log: /var/log/ceph/ceph-client.admin.142210.log 2023-10-18T12:02:56.242 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133255.log.gz 2023-10-18T12:02:56.242 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113607.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.165800.log.gz 2023-10-18T12:02:56.243 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159468.log 2023-10-18T12:02:56.243 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113607.log.gz 2023-10-18T12:02:56.243 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95689.log 2023-10-18T12:02:56.243 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142210.log: /var/log/ceph/ceph-client.admin.159468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157053.log 2023-10-18T12:02:56.244 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142210.log.gz 2023-10-18T12:02:56.244 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95689.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.159468.log.gz 2023-10-18T12:02:56.244 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156100.log 2023-10-18T12:02:56.244 INFO:teuthology.orchestra.run.smithi125.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95689.log.gz 2023-10-18T12:02:56.245 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150546.log 2023-10-18T12:02:56.245 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.157053.log.gz 2023-10-18T12:02:56.245 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131284.log 2023-10-18T12:02:56.246 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156100.log: /var/log/ceph/ceph-client.admin.150546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130214.log 2023-10-18T12:02:56.246 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.156100.log.gz 2023-10-18T12:02:56.246 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.150546.log.gz 2023-10-18T12:02:56.246 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150654.log 2023-10-18T12:02:56.247 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.131284.log.gz 2023-10-18T12:02:56.247 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114538.log 2023-10-18T12:02:56.247 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130214.log: /var/log/ceph/ceph-client.admin.150654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142340.log 2023-10-18T12:02:56.247 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.130214.log.gz 2023-10-18T12:02:56.248 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6%/var/log/ceph/ceph-client.admin.114538.log: -- replaced with /var/log/ceph/ceph-client.admin.150654.log.gz 2023-10-18T12:02:56.248 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107962.log 2023-10-18T12:02:56.248 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.114538.log.gz 2023-10-18T12:02:56.248 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119827.log 2023-10-18T12:02:56.249 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142340.log: /var/log/ceph/ceph-client.admin.107962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123705.log 2023-10-18T12:02:56.249 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142340.log.gz 2023-10-18T12:02:56.249 INFO:teuthology.orchestra.run.smithi125.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.107962.log.gz 2023-10-18T12:02:56.250 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88588.log 2023-10-18T12:02:56.250 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119827.log.gz 2023-10-18T12:02:56.250 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154924.log 2023-10-18T12:02:56.250 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123705.log: /var/log/ceph/ceph-client.admin.88588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103105.log 2023-10-18T12:02:56.251 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.123705.log.gz 2023-10-18T12:02:56.251 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.88588.log.gz 2023-10-18T12:02:56.251 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98658.log 2023-10-18T12:02:56.251 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135046.log 2023-10-18T12:02:56.252 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103105.log: /var/log/ceph/ceph-client.admin.98658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156512.log 2023-10-18T12:02:56.252 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103105.log.gz 2023-10-18T12:02:56.252 INFO:teuthology.orchestra.run.smithi125.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.154924.log.gz 2023-10-18T12:02:56.253 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98658.log.gz 2023-10-18T12:02:56.253 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158202.log 2023-10-18T12:02:56.253 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103021.log 2023-10-18T12:02:56.253 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.135046.log.gz 2023-10-18T12:02:56.254 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158202.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.156512.log.gz 2023-10-18T12:02:56.254 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98364.log 2023-10-18T12:02:56.254 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158202.log.gz 2023-10-18T12:02:56.254 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120014.log 2023-10-18T12:02:56.255 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose 87.7% -- /var/log/ceph/ceph-client.admin.99732.log -- replaced with /var/log/ceph/ceph-client.admin.103021.log.gz 2023-10-18T12:02:56.255 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.255 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98364.log: /var/log/ceph/ceph-client.admin.120014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.164915.log 2023-10-18T12:02:56.256 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98364.log.gz 2023-10-18T12:02:56.256 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120014.log.gz 2023-10-18T12:02:56.256 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128711.log 2023-10-18T12:02:56.256 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.99732.log: /var/log/ceph/ceph-client.0.164915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105706.log 2023-10-18T12:02:56.257 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99732.log.gz 2023-10-18T12:02:56.257 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120743.log 2023-10-18T12:02:56.257 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.100104.log 2023-10-18T12:02:56.257 INFO:teuthology.orchestra.run.smithi125.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.128711.log.gz 2023-10-18T12:02:56.258 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105706.log.gz 2023-10-18T12:02:56.258 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122298.log 2023-10-18T12:02:56.258 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120743.log.gz 2023-10-18T12:02:56.259 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.2.100104.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.125171.log 2023-10-18T12:02:56.259 INFO:teuthology.orchestra.run.smithi125.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.164915.log.gz 2023-10-18T12:02:56.259 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105328.log 2023-10-18T12:02:56.259 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122298.log.gz 2023-10-18T12:02:56.260 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106545.log 2023-10-18T12:02:56.260 INFO:teuthology.orchestra.run.smithi125.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.2.100104.log.gz 2023-10-18T12:02:56.260 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.125171.log.gz 2023-10-18T12:02:56.260 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166189.log 2023-10-18T12:02:56.261 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105328.log: /var/log/ceph/ceph-client.admin.106545.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105328.log.gz 2023-10-18T12:02:56.261 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145193.log 2023-10-18T12:02:56.261 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161412.log 2023-10-18T12:02:56.262 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166189.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.106545.log.gz 2023-10-18T12:02:56.262 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145193.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.166189.log.gz 2023-10-18T12:02:56.262 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134318.log 2023-10-18T12:02:56.262 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146979.log 2023-10-18T12:02:56.263 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145193.log.gz 2023-10-18T12:02:56.263 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161412.log: /var/log/ceph/ceph-client.admin.134318.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161412.log.gz 2023-10-18T12:02:56.263 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144533.log 2023-10-18T12:02:56.263 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.134318.log.gz 2023-10-18T12:02:56.264 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102001.log 2023-10-18T12:02:56.264 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146979.log: /var/log/ceph/ceph-client.admin.144533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162202.log 2023-10-18T12:02:56.264 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146979.log.gz 2023-10-18T12:02:56.265 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.144533.log.gz 2023-10-18T12:02:56.265 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162244.log 2023-10-18T12:02:56.265 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102001.log.gz 2023-10-18T12:02:56.265 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107269.log 2023-10-18T12:02:56.266 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162202.log: /var/log/ceph/ceph-client.admin.162244.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.162202.log.gz 2023-10-18T12:02:56.266 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145905.log 2023-10-18T12:02:56.266 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.162244.log.gz 2023-10-18T12:02:56.266 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117769.log 2023-10-18T12:02:56.267 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107269.log: /var/log/ceph/ceph-client.admin.145905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123789.log 2023-10-18T12:02:56.267 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.107269.log.gz 2023-10-18T12:02:56.267 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.145905.log.gz 2023-10-18T12:02:56.268 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105454.log 2023-10-18T12:02:56.268 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117769.log.gz 2023-10-18T12:02:56.268 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150588.log 2023-10-18T12:02:56.268 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.123789.log.gz 2023-10-18T12:02:56.269 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133847.log 2023-10-18T12:02:56.269 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105454.log: /var/log/ceph/ceph-client.admin.150588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98165.log 2023-10-18T12:02:56.269 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.105454.log.gz 2023-10-18T12:02:56.269 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95329.log 2023-10-18T12:02:56.270 INFO:teuthology.orchestra.run.smithi125.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.150588.log.gz 2023-10-18T12:02:56.270 INFO:teuthology.orchestra.run.smithi125.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.133847.log.gz 2023-10-18T12:02:56.270 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88003.log 2023-10-18T12:02:56.271 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150294.log 2023-10-18T12:02:56.271 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98165.log.gz 2023-10-18T12:02:56.271 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95329.log.gz 2023-10-18T12:02:56.271 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.88003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134818.log 2023-10-18T12:02:56.272 INFO:teuthology.orchestra.run.smithi125.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.88003.log.gz 2023-10-18T12:02:56.272 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100840.log 2023-10-18T12:02:56.272 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.150294.log.gz 2023-10-18T12:02:56.272 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121642.log 2023-10-18T12:02:56.273 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134818.log: /var/log/ceph/ceph-client.admin.100840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144405.log 2023-10-18T12:02:56.273 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.134818.log.gz 2023-10-18T12:02:56.273 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100840.log.gz 2023-10-18T12:02:56.274 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112495.log 2023-10-18T12:02:56.274 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121642.log.gz 2023-10-18T12:02:56.274 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113841.log 2023-10-18T12:02:56.274 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144405.log: /var/log/ceph/ceph-client.admin.112495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136662.log 2023-10-18T12:02:56.275 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.144405.log.gz 2023-10-18T12:02:56.275 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112495.log.gz 2023-10-18T12:02:56.275 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163968.log 2023-10-18T12:02:56.275 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113841.log.gz 2023-10-18T12:02:56.276 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153687.log 2023-10-18T12:02:56.276 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136662.log: /var/log/ceph/ceph-client.admin.163968.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136662.log.gz 2023-10-18T12:02:56.276 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136116.log 2023-10-18T12:02:56.277 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.163968.log.gz 2023-10-18T12:02:56.277 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150042.log 2023-10-18T12:02:56.277 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153687.log: /var/log/ceph/ceph-client.admin.136116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126204.log 2023-10-18T12:02:56.277 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153687.log.gz 2023-10-18T12:02:56.278 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.136116.log.gz 2023-10-18T12:02:56.278 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157342.log 2023-10-18T12:02:56.278 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167555.log 2023-10-18T12:02:56.278 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.150042.log.gz 2023-10-18T12:02:56.279 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126204.log: /var/log/ceph/ceph-client.admin.157342.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.126204.log.gz 2023-10-18T12:02:56.279 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145653.log 2023-10-18T12:02:56.279 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.157342.log.gz 2023-10-18T12:02:56.279 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101959.log 2023-10-18T12:02:56.280 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167555.log: /var/log/ceph/ceph-client.admin.145653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124018.log 2023-10-18T12:02:56.280 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.167555.log.gz 2023-10-18T12:02:56.280 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145653.log.gz 2023-10-18T12:02:56.281 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124961.log 2023-10-18T12:02:56.281 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.101959.log.gz 2023-10-18T12:02:56.281 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97823.log 2023-10-18T12:02:56.281 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124018.log: /var/log/ceph/ceph-client.admin.124961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97819.log 2023-10-18T12:02:56.282 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.124018.log.gz 2023-10-18T12:02:56.282 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124961.log.gz 2023-10-18T12:02:56.282 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123621.log 2023-10-18T12:02:56.282 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97823.log.gz 2023-10-18T12:02:56.283 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156764.log 2023-10-18T12:02:56.283 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97819.log.gz 2023-10-18T12:02:56.283 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117372.log 2023-10-18T12:02:56.284 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.123621.log.gz 2023-10-18T12:02:56.284 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153088.log 2023-10-18T12:02:56.284 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156764.log: /var/log/ceph/ceph-client.admin.117372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105108.log 2023-10-18T12:02:56.284 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.156764.log.gz 2023-10-18T12:02:56.285 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117372.log.gz 2023-10-18T12:02:56.285 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142687.log 2023-10-18T12:02:56.285 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.153088.log.gz 2023-10-18T12:02:56.285 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96648.log 2023-10-18T12:02:56.286 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.105108.log.gz --verbose 2023-10-18T12:02:56.286 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.104125.log 2023-10-18T12:02:56.286 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142687.log: /var/log/ceph/ceph-client.admin.96648.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.142687.log.gz 2023-10-18T12:02:56.287 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100924.log 2023-10-18T12:02:56.287 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.96648.log.gz 2023-10-18T12:02:56.287 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162936.log 2023-10-18T12:02:56.287 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104125.log: /var/log/ceph/ceph-client.admin.100924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149234.log 2023-10-18T12:02:56.288 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104125.log.gz 2023-10-18T12:02:56.288 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100924.log.gz 2023-10-18T12:02:56.288 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120388.log 2023-10-18T12:02:56.288 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.162936.log.gz 2023-10-18T12:02:56.289 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118880.log 2023-10-18T12:02:56.289 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.149234.log.gz 2023-10-18T12:02:56.289 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166573.log 2023-10-18T12:02:56.290 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120388.log: /var/log/ceph/ceph-client.admin.118880.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.120388.log.gz 2023-10-18T12:02:56.290 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118586.log 2023-10-18T12:02:56.290 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118880.log.gz 2023-10-18T12:02:56.290 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139546.log 2023-10-18T12:02:56.291 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166573.log: /var/log/ceph/ceph-client.admin.118586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125273.log 2023-10-18T12:02:56.291 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.166573.log.gz 2023-10-18T12:02:56.291 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.139546.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118586.log.gz 2023-10-18T12:02:56.291 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104894.log 2023-10-18T12:02:56.292 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.139546.log.gz 2023-10-18T12:02:56.292 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104936.log 2023-10-18T12:02:56.292 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164380.log 2023-10-18T12:02:56.293 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104894.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.125273.log.gz 2023-10-18T12:02:56.293 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104936.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104894.log.gz 2023-10-18T12:02:56.293 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135894.log 2023-10-18T12:02:56.293 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104936.log.gz 2023-10-18T12:02:56.294 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165281.log 2023-10-18T12:02:56.294 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164380.log: /var/log/ceph/ceph-client.admin.135894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99078.log 2023-10-18T12:02:56.294 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164380.log.gz 2023-10-18T12:02:56.294 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135894.log.gz 2023-10-18T12:02:56.295 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141178.log 2023-10-18T12:02:56.295 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.99078.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165281.log.gz 2023-10-18T12:02:56.295 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100882.log 2023-10-18T12:02:56.296 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137120.log 2023-10-18T12:02:56.296 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141178.log: /var/log/ceph/ceph-client.admin.100882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154544.log 2023-10-18T12:02:56.296 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.141178.log.gz 2023-10-18T12:02:56.296 INFO:teuthology.orchestra.run.smithi125.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.99078.log.gz 2023-10-18T12:02:56.297 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100882.log.gz 2023-10-18T12:02:56.297 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.137120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150336.log 2023-10-18T12:02:56.297 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150210.log 2023-10-18T12:02:56.297 INFO:teuthology.orchestra.run.smithi125.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137120.log.gz 2023-10-18T12:02:56.298 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.154544.log.gz 2023-10-18T12:02:56.298 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144449.log 2023-10-18T12:02:56.298 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.150336.log.gz 2023-10-18T12:02:56.299 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134444.log 2023-10-18T12:02:56.299 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.150210.log.gz 2023-10-18T12:02:56.299 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113047.log 2023-10-18T12:02:56.299 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.144449.log.gz 2023-10-18T12:02:56.300 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96422.log 2023-10-18T12:02:56.300 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.134444.log.gz 2023-10-18T12:02:56.300 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164506.log 2023-10-18T12:02:56.300 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113047.log.gz 2023-10-18T12:02:56.301 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122976.log 2023-10-18T12:02:56.301 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96422.log.gz 2023-10-18T12:02:56.301 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156250.log 2023-10-18T12:02:56.302 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164506.log.gz 2023-10-18T12:02:56.302 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124833.log 2023-10-18T12:02:56.302 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122976.log.gz 2023-10-18T12:02:56.302 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141220.log 2023-10-18T12:02:56.303 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156250.log: /var/log/ceph/ceph-client.admin.124833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118754.log 2023-10-18T12:02:56.303 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.156250.log.gz 2023-10-18T12:02:56.303 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141220.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124833.log.gz 2023-10-18T12:02:56.304 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117144.log 2023-10-18T12:02:56.304 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.141220.log.gz 2023-10-18T12:02:56.304 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124611.log 2023-10-18T12:02:56.304 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117144.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118754.log.gz 2023-10-18T12:02:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167431.log 2023-10-18T12:02:56.305 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117144.log.gz 2023-10-18T12:02:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122882.log 2023-10-18T12:02:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124611.log: /var/log/ceph/ceph-client.admin.167431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147742.log 2023-10-18T12:02:56.306 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124611.log.gz 2023-10-18T12:02:56.306 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.167431.log.gz 2023-10-18T12:02:56.306 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145863.log 2023-10-18T12:02:56.307 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.122882.log.gz 2023-10-18T12:02:56.307 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97832.log 2023-10-18T12:02:56.307 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.147742.log: /var/log/ceph/ceph-client.admin.145863.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.147742.log.gz 2023-10-18T12:02:56.307 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149743.log 2023-10-18T12:02:56.308 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145863.log.gz 2023-10-18T12:02:56.308 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109414.log 2023-10-18T12:02:56.308 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97832.log: /var/log/ceph/ceph-client.admin.149743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87616.log 2023-10-18T12:02:56.308 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97832.log.gz 2023-10-18T12:02:56.309 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149743.log.gz 2023-10-18T12:02:56.309 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.109414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126849.log 2023-10-18T12:02:56.309 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134360.log 2023-10-18T12:02:56.310 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.109414.log.gz 2023-10-18T12:02:56.310 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87616.log: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.87616.log.gz 2023-10-18T12:02:56.310 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122829.log 2023-10-18T12:02:56.310 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.126849.log.gz 2023-10-18T12:02:56.311 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149828.log 2023-10-18T12:02:56.311 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134360.log: /var/log/ceph/ceph-client.admin.122829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144804.log 2023-10-18T12:02:56.311 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.134360.log.gz 2023-10-18T12:02:56.311 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.122829.log.gz 2023-10-18T12:02:56.312 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160582.log 2023-10-18T12:02:56.312 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.149828.log.gz 2023-10-18T12:02:56.312 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143464.log 2023-10-18T12:02:56.312 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144804.log: /var/log/ceph/ceph-client.admin.160582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103957.log 2023-10-18T12:02:56.313 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.144804.log.gz 2023-10-18T12:02:56.313 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.160582.log.gz 2023-10-18T12:02:56.313 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104768.log 2023-10-18T12:02:56.314 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.143464.log.gz 2023-10-18T12:02:56.314 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118922.log 2023-10-18T12:02:56.314 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103957.log: /var/log/ceph/ceph-client.admin.104768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123579.log 2023-10-18T12:02:56.314 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103957.log.gz 2023-10-18T12:02:56.315 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104768.log.gz/var/log/ceph/ceph-client.admin.118922.log: 2023-10-18T12:02:56.315 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96852.log 2023-10-18T12:02:56.315 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.118922.log.gz 2023-10-18T12:02:56.315 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101817.log 2023-10-18T12:02:56.316 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.123579.log.gz 2023-10-18T12:02:56.316 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126126.log 2023-10-18T12:02:56.316 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96852.log: /var/log/ceph/ceph-client.admin.101817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161743.log 2023-10-18T12:02:56.317 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96852.log.gz 2023-10-18T12:02:56.317 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101817.log.gz 2023-10-18T12:02:56.317 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156016.log 2023-10-18T12:02:56.317 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104251.log 2023-10-18T12:02:56.318 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161743.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.126126.log.gz 2023-10-18T12:02:56.318 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156016.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.161743.log.gzgzip 2023-10-18T12:02:56.318 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.152893.log 2023-10-18T12:02:56.318 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.156016.log.gz 2023-10-18T12:02:56.319 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148296.log 2023-10-18T12:02:56.319 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104251.log: /var/log/ceph/ceph-client.admin.152893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160538.log 2023-10-18T12:02:56.319 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104251.log.gz 2023-10-18T12:02:56.319 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.152893.log.gz 2023-10-18T12:02:56.320 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112266.log 2023-10-18T12:02:56.320 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.148296.log.gz 2023-10-18T12:02:56.320 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133511.log 2023-10-18T12:02:56.321 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.160538.log.gz 2023-10-18T12:02:56.321 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95001.log 2023-10-18T12:02:56.321 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112266.log: /var/log/ceph/ceph-client.admin.133511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165895.log 2023-10-18T12:02:56.321 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112266.log.gz 2023-10-18T12:02:56.322 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95001.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.133511.log.gz 2023-10-18T12:02:56.322 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.146596.log 2023-10-18T12:02:56.322 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95001.log.gz 2023-10-18T12:02:56.322 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113733.log 2023-10-18T12:02:56.323 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165895.log: /var/log/ceph/ceph-client.0.146596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120098.log 2023-10-18T12:02:56.323 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165895.log.gz 2023-10-18T12:02:56.323 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104810.log 2023-10-18T12:02:56.324 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113733.log.gz 2023-10-18T12:02:56.324 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161869.log 2023-10-18T12:02:56.324 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120098.log.gz 2023-10-18T12:02:56.324 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151332.log 2023-10-18T12:02:56.325 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144226.log 2023-10-18T12:02:56.325 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104810.log.gz 2023-10-18T12:02:56.325 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0%/var/log/ceph/ceph-client.admin.151332.log: -- replaced with /var/log/ceph/ceph-client.admin.161869.log.gz 2023-10-18T12:02:56.325 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149958.log 2023-10-18T12:02:56.326 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.151332.log.gz 2023-10-18T12:02:56.326 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153939.log 2023-10-18T12:02:56.326 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118964.log 2023-10-18T12:02:56.327 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144226.log.gz 2023-10-18T12:02:56.327 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149958.log: /var/log/ceph/ceph-client.admin.153939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165164.log 2023-10-18T12:02:56.327 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.149958.log.gz 2023-10-18T12:02:56.327 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153939.log.gz 2023-10-18T12:02:56.328 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156554.log 2023-10-18T12:02:56.328 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117288.log 2023-10-18T12:02:56.328 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118964.log.gz 2023-10-18T12:02:56.328 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.165164.log.gz 2023-10-18T12:02:56.329 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159288.log 2023-10-18T12:02:56.329 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117288.log: gzip 86.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.156554.log.gz -- 2023-10-18T12:02:56.329 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.98826.log 2023-10-18T12:02:56.329 INFO:teuthology.orchestra.run.smithi125.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.117288.log.gz 2023-10-18T12:02:56.330 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119530.log 2023-10-18T12:02:56.330 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.159288.log.gz 2023-10-18T12:02:56.330 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159594.log 2023-10-18T12:02:56.331 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119530.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98826.log.gz 2023-10-18T12:02:56.331 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165334.log 2023-10-18T12:02:56.331 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119530.log.gz 2023-10-18T12:02:56.331 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97146.log 2023-10-18T12:02:56.332 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165334.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159594.log.gz 2023-10-18T12:02:56.332 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155720.log 2023-10-18T12:02:56.332 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.165334.log.gz 2023-10-18T12:02:56.332 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124653.log 2023-10-18T12:02:56.333 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97146.log.gz 2023-10-18T12:02:56.333 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97828.log 2023-10-18T12:02:56.333 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124653.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.155720.log.gz 2023-10-18T12:02:56.334 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102753.log 2023-10-18T12:02:56.334 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124653.log.gz 2023-10-18T12:02:56.334 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130537.log 2023-10-18T12:02:56.334 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136030.log 2023-10-18T12:02:56.335 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97828.log.gz 2023-10-18T12:02:56.335 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102753.log: /var/log/ceph/ceph-client.admin.130537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165563.log 2023-10-18T12:02:56.335 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102753.log.gz 2023-10-18T12:02:56.335 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.130537.log.gz 2023-10-18T12:02:56.336 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98952.log 2023-10-18T12:02:56.336 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113775.log 2023-10-18T12:02:56.336 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.136030.log.gz 2023-10-18T12:02:56.336 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165563.log: /var/log/ceph/ceph-client.admin.98952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118320.log 2023-10-18T12:02:56.337 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.165563.log.gz 2023-10-18T12:02:56.337 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98952.log.gz 2023-10-18T12:02:56.337 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131578.log 2023-10-18T12:02:56.338 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144930.log 2023-10-18T12:02:56.338 INFO:teuthology.orchestra.run.smithi125.stderr: 91.1% 87.0%/var/log/ceph/ceph-client.admin.131578.log: -- replaced with /var/log/ceph/ceph-client.admin.118320.log.gz 2023-10-18T12:02:56.338 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97020.log 2023-10-18T12:02:56.338 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113775.log.gz 2023-10-18T12:02:56.339 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2%/var/log/ceph/ceph-client.admin.144930.log: -- replaced with /var/log/ceph/ceph-client.admin.131578.log.gz 2023-10-18T12:02:56.339 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135299.log 2023-10-18T12:02:56.339 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.144930.log.gz 2023-10-18T12:02:56.339 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100226.log 2023-10-18T12:02:56.340 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135299.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97020.log.gz 2023-10-18T12:02:56.340 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124186.log 2023-10-18T12:02:56.340 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135299.log.gz 2023-10-18T12:02:56.341 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158556.log 2023-10-18T12:02:56.341 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100226.log.gz 2023-10-18T12:02:56.341 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113259.log 2023-10-18T12:02:56.341 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.124186.log.gz 2023-10-18T12:02:56.342 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151220.log 2023-10-18T12:02:56.342 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113259.log: gzip 87.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.158556.log.gz -- 2023-10-18T12:02:56.342 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.149444.log 2023-10-18T12:02:56.342 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.151220.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.113259.log.gz 2023-10-18T12:02:56.343 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113215.log 2023-10-18T12:02:56.343 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.151220.log.gz 2023-10-18T12:02:56.343 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145523.log 2023-10-18T12:02:56.344 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113215.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.149444.log.gz 2023-10-18T12:02:56.344 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141827.log 2023-10-18T12:02:56.344 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.113215.log.gz 2023-10-18T12:02:56.344 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166334.log 2023-10-18T12:02:56.345 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141827.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.145523.log.gz 2023-10-18T12:02:56.345 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87784.log 2023-10-18T12:02:56.345 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.141827.log.gz 2023-10-18T12:02:56.345 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129457.log 2023-10-18T12:02:56.346 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87784.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.166334.log.gz 2023-10-18T12:02:56.346 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163459.log 2023-10-18T12:02:56.346 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.87784.log.gz/var/log/ceph/ceph-client.admin.129457.log: 2023-10-18T12:02:56.346 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118364.log 2023-10-18T12:02:56.347 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163459.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.129457.log.gz 2023-10-18T12:02:56.347 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153603.log 2023-10-18T12:02:56.347 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163459.log.gz 2023-10-18T12:02:56.348 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104683.log 2023-10-18T12:02:56.348 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153603.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.118364.log.gz 2023-10-18T12:02:56.348 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108728.log 2023-10-18T12:02:56.348 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.153603.log.gz/var/log/ceph/ceph-client.admin.104683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159332.log 2023-10-18T12:02:56.349 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.349 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108728.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104683.log.gz 2023-10-18T12:02:56.349 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139504.log 2023-10-18T12:02:56.349 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.108728.log.gz 2023-10-18T12:02:56.350 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146186.log 2023-10-18T12:02:56.350 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.139504.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.159332.log.gz 2023-10-18T12:02:56.350 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100588.log 2023-10-18T12:02:56.351 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.139504.log.gz 2023-10-18T12:02:56.351 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98868.log 2023-10-18T12:02:56.351 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100588.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.146186.log.gz 2023-10-18T12:02:56.351 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100462.log 2023-10-18T12:02:56.352 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100588.log.gz 2023-10-18T12:02:56.352 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118152.log 2023-10-18T12:02:56.352 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 95.6% -5 -- replaced with /var/log/ceph/ceph-client.0.136921.log.gz --verbose 2023-10-18T12:02:56.352 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.167473.log 2023-10-18T12:02:56.353 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98868.log.gz 2023-10-18T12:02:56.353 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100462.log: /var/log/ceph/ceph-client.admin.118152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154798.log 2023-10-18T12:02:56.353 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100462.log.gz 2023-10-18T12:02:56.353 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118152.log.gz 2023-10-18T12:02:56.354 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150420.log 2023-10-18T12:02:56.354 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.167473.log.gz 2023-10-18T12:02:56.354 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118278.log 2023-10-18T12:02:56.355 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154798.log: /var/log/ceph/ceph-client.admin.150420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128417.log 2023-10-18T12:02:56.355 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.154798.log.gz 2023-10-18T12:02:56.355 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.150420.log.gz 2023-10-18T12:02:56.355 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103477.log 2023-10-18T12:02:56.356 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118278.log.gz 2023-10-18T12:02:56.356 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134597.log 2023-10-18T12:02:56.356 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128417.log: /var/log/ceph/ceph-client.admin.103477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142391.log 2023-10-18T12:02:56.356 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.128417.log.gz 2023-10-18T12:02:56.357 INFO:teuthology.orchestra.run.smithi125.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103477.log.gz 2023-10-18T12:02:56.357 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.134597.log: /var/log/ceph/ceph-client.admin.162160.log 2023-10-18T12:02:56.357 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101691.log 2023-10-18T12:02:56.358 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142391.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134597.log.gz 2023-10-18T12:02:56.358 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119680.log 2023-10-18T12:02:56.358 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.142391.log.gz 2023-10-18T12:02:56.358 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.162160.log.gz 2023-10-18T12:02:56.359 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111766.log 2023-10-18T12:02:56.361 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101691.log: /var/log/ceph/ceph-client.admin.119680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105242.log 2023-10-18T12:02:56.361 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101691.log.gz 2023-10-18T12:02:56.362 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119680.log.gz 2023-10-18T12:02:56.362 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162767.log 2023-10-18T12:02:56.362 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111766.log.gz 2023-10-18T12:02:56.362 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101134.log 2023-10-18T12:02:56.363 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105242.log.gz 2023-10-18T12:02:56.363 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163333.log 2023-10-18T12:02:56.363 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%/var/log/ceph/ceph-client.admin.101134.log: -- replaced with /var/log/ceph/ceph-client.admin.162767.log.gz 2023-10-18T12:02:56.363 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149048.log 2023-10-18T12:02:56.364 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.101134.log.gz 2023-10-18T12:02:56.364 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144617.log 2023-10-18T12:02:56.364 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163333.log: /var/log/ceph/ceph-client.admin.149048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132225.log 2023-10-18T12:02:56.365 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163333.log.gz 2023-10-18T12:02:56.365 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149048.log.gz 2023-10-18T12:02:56.365 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88000.log 2023-10-18T12:02:56.365 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154714.log 2023-10-18T12:02:56.366 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144617.log.gz 2023-10-18T12:02:56.366 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132225.log: /var/log/ceph/ceph-client.admin.88000.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.132225.log.gz 2023-10-18T12:02:56.366 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132269.log 2023-10-18T12:02:56.366 INFO:teuthology.orchestra.run.smithi125.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.88000.log.gz 2023-10-18T12:02:56.367 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87938.log 2023-10-18T12:02:56.367 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154714.log: /var/log/ceph/ceph-client.admin.132269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150084.log 2023-10-18T12:02:56.367 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.154714.log.gz 2023-10-18T12:02:56.368 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87938.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.132269.log.gz 2023-10-18T12:02:56.368 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111919.log 2023-10-18T12:02:56.368 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.87938.log.gz 2023-10-18T12:02:56.368 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148380.log 2023-10-18T12:02:56.369 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156292.log 2023-10-18T12:02:56.369 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.150084.log.gz 2023-10-18T12:02:56.369 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111919.log: /var/log/ceph/ceph-client.admin.148380.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111919.log.gz 2023-10-18T12:02:56.369 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163104.log 2023-10-18T12:02:56.370 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148380.log.gz 2023-10-18T12:02:56.370 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97062.log 2023-10-18T12:02:56.370 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156292.log: /var/log/ceph/ceph-client.admin.163104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125973.log 2023-10-18T12:02:56.371 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156292.log.gz 2023-10-18T12:02:56.371 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.163104.log.gz 2023-10-18T12:02:56.371 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120911.log 2023-10-18T12:02:56.371 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97062.log.gz 2023-10-18T12:02:56.372 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155932.log 2023-10-18T12:02:56.372 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126891.log 2023-10-18T12:02:56.372 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.125973.log.gz 2023-10-18T12:02:56.372 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120911.log: /var/log/ceph/ceph-client.admin.155932.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120911.log.gz 2023-10-18T12:02:56.373 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153645.log 2023-10-18T12:02:56.373 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.155932.log.gz 2023-10-18T12:02:56.373 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101092.log 2023-10-18T12:02:56.374 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126891.log: /var/log/ceph/ceph-client.admin.153645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101733.log 2023-10-18T12:02:56.374 INFO:teuthology.orchestra.run.smithi125.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.126891.log.gz 2023-10-18T12:02:56.374 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153645.log.gz 2023-10-18T12:02:56.374 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137382.log 2023-10-18T12:02:56.375 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101092.log.gz 2023-10-18T12:02:56.375 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146271.log 2023-10-18T12:02:56.375 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101733.log: /var/log/ceph/ceph-client.admin.137382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106899.log 2023-10-18T12:02:56.375 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101733.log.gz 2023-10-18T12:02:56.376 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.137382.log.gz 2023-10-18T12:02:56.376 INFO:teuthology.orchestra.run.smithi125.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.146596.log.gz 2023-10-18T12:02:56.376 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110747.log 2023-10-18T12:02:56.376 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121174.log 2023-10-18T12:02:56.377 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.106899.log.gz 2023-10-18T12:02:56.377 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.110747.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.146271.log.gz 2023-10-18T12:02:56.377 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159870.log 2023-10-18T12:02:56.378 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110747.log.gz 2023-10-18T12:02:56.378 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122064.log 2023-10-18T12:02:56.378 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121174.log.gz 2023-10-18T12:02:56.378 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116896.log 2023-10-18T12:02:56.379 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.159870.log.gz 2023-10-18T12:02:56.379 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167179.log 2023-10-18T12:02:56.379 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.116896.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122064.log.gz 2023-10-18T12:02:56.380 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112003.log 2023-10-18T12:02:56.380 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116896.log.gz 2023-10-18T12:02:56.380 INFO:teuthology.orchestra.run.smithi125.stderr:gzip/var/log/ceph/ceph-client.admin.167179.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.98784.log 2023-10-18T12:02:56.380 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117895.log 2023-10-18T12:02:56.381 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.167179.log.gz 2023-10-18T12:02:56.381 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98784.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112003.log.gz 2023-10-18T12:02:56.381 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112098.log 2023-10-18T12:02:56.381 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98784.log.gz 2023-10-18T12:02:56.382 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163882.log 2023-10-18T12:02:56.382 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117895.log.gz 2023-10-18T12:02:56.382 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154408.log 2023-10-18T12:02:56.383 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1%/var/log/ceph/ceph-client.admin.163882.log: -- replaced with /var/log/ceph/ceph-client.admin.112098.log.gz 2023-10-18T12:02:56.383 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124144.log 2023-10-18T12:02:56.383 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163882.log.gz 2023-10-18T12:02:56.383 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132731.log 2023-10-18T12:02:56.384 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124144.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.154408.log.gz -- 2023-10-18T12:02:56.384 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.128501.log 2023-10-18T12:02:56.384 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124144.log.gz 2023-10-18T12:02:56.384 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102247.log 2023-10-18T12:02:56.385 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165072.log 2023-10-18T12:02:56.385 INFO:teuthology.orchestra.run.smithi125.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.132731.log.gz 2023-10-18T12:02:56.385 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102247.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128501.log.gz 2023-10-18T12:02:56.385 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166741.log 2023-10-18T12:02:56.386 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102247.log.gz 2023-10-18T12:02:56.386 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103663.log 2023-10-18T12:02:56.386 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.165072.log.gz 2023-10-18T12:02:56.387 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96732.log 2023-10-18T12:02:56.387 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103663.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.166741.log.gz 2023-10-18T12:02:56.387 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103063.log 2023-10-18T12:02:56.387 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103663.log.gz 2023-10-18T12:02:56.388 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106272.log 2023-10-18T12:02:56.388 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96732.log.gz 2023-10-18T12:02:56.388 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103147.log 2023-10-18T12:02:56.388 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106272.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103063.log.gz 2023-10-18T12:02:56.389 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166063.log 2023-10-18T12:02:56.389 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106272.log.gz 2023-10-18T12:02:56.389 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158651.log 2023-10-18T12:02:56.390 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103147.log.gz 2023-10-18T12:02:56.390 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96506.log 2023-10-18T12:02:56.390 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158651.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.166063.log.gz 2023-10-18T12:02:56.390 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158777.log 2023-10-18T12:02:56.391 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.158651.log.gz 2023-10-18T12:02:56.391 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167095.log 2023-10-18T12:02:56.391 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96506.log.gz 2023-10-18T12:02:56.391 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127059.log 2023-10-18T12:02:56.392 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167095.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158777.log.gz 2023-10-18T12:02:56.392 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146142.log 2023-10-18T12:02:56.392 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.167095.log.gz 2023-10-18T12:02:56.393 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118796.log 2023-10-18T12:02:56.393 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127059.log.gz 2023-10-18T12:02:56.393 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98616.log 2023-10-18T12:02:56.393 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118796.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.146142.log.gz 2023-10-18T12:02:56.394 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103189.log 2023-10-18T12:02:56.394 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118796.log.gz 2023-10-18T12:02:56.394 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123934.log 2023-10-18T12:02:56.395 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103189.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98616.log.gz 2023-10-18T12:02:56.395 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120556.log 2023-10-18T12:02:56.395 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103189.log.gz 2023-10-18T12:02:56.395 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153897.log 2023-10-18T12:02:56.396 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123934.log.gz 2023-10-18T12:02:56.396 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145737.log 2023-10-18T12:02:56.396 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120556.log.gz 2023-10-18T12:02:56.396 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107017.log 2023-10-18T12:02:56.397 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145737.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.153897.log.gz 2023-10-18T12:02:56.397 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100672.log 2023-10-18T12:02:56.397 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107017.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145737.log.gz 2023-10-18T12:02:56.397 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149620.log 2023-10-18T12:02:56.398 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.107017.log.gz 2023-10-18T12:02:56.398 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141659.log 2023-10-18T12:02:56.398 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100672.log.gz 2023-10-18T12:02:56.399 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100420.log 2023-10-18T12:02:56.399 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141659.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149620.log.gz 2023-10-18T12:02:56.399 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127965.log 2023-10-18T12:02:56.399 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100420.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.141659.log.gz 2023-10-18T12:02:56.400 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163585.log 2023-10-18T12:02:56.400 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100420.log.gz 2023-10-18T12:02:56.400 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157595.log 2023-10-18T12:02:56.400 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163680.log 2023-10-18T12:02:56.401 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127965.log.gz 2023-10-18T12:02:56.401 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157595.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.163585.log.gz 2023-10-18T12:02:56.401 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142729.log 2023-10-18T12:02:56.402 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.157595.log.gz 2023-10-18T12:02:56.402 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166657.log 2023-10-18T12:02:56.402 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142729.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163680.log.gz 2023-10-18T12:02:56.402 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141094.log 2023-10-18T12:02:56.403 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166657.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.142729.log.gz 2023-10-18T12:02:56.403 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162580.log 2023-10-18T12:02:56.403 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.166657.log.gz 2023-10-18T12:02:56.403 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112885.log 2023-10-18T12:02:56.404 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.141094.log.gz 2023-10-18T12:02:56.404 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166021.log 2023-10-18T12:02:56.404 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.162580.log.gz 2023-10-18T12:02:56.404 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137078.log 2023-10-18T12:02:56.405 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166021.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112885.log.gz 2023-10-18T12:02:56.405 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133747.log 2023-10-18T12:02:56.405 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.166021.log.gz 2023-10-18T12:02:56.406 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.137078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104599.log 2023-10-18T12:02:56.406 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.137078.log.gz 2023-10-18T12:02:56.406 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97490.log 2023-10-18T12:02:56.406 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104599.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.133747.log.gz 2023-10-18T12:02:56.407 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144363.log 2023-10-18T12:02:56.407 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97490.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104599.log.gz 2023-10-18T12:02:56.407 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164338.log 2023-10-18T12:02:56.408 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.97490.log.gz 2023-10-18T12:02:56.408 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144846.log 2023-10-18T12:02:56.408 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.144363.log.gz 2023-10-18T12:02:56.408 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130495.log 2023-10-18T12:02:56.409 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.164338.log.gz/var/log/ceph/ceph-client.admin.144846.log: 2023-10-18T12:02:56.409 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119972.log 2023-10-18T12:02:56.409 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144846.log.gz 2023-10-18T12:02:56.410 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130971.log 2023-10-18T12:02:56.410 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119972.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.130495.log.gz 2023-10-18T12:02:56.410 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105412.log 2023-10-18T12:02:56.410 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119972.log.gz 2023-10-18T12:02:56.411 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125315.log 2023-10-18T12:02:56.411 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.130971.log.gz 2023-10-18T12:02:56.411 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157805.log 2023-10-18T12:02:56.412 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105412.log: /var/log/ceph/ceph-client.admin.125315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128103.log 2023-10-18T12:02:56.412 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105412.log.gz 2023-10-18T12:02:56.412 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157805.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.125315.log.gz 2023-10-18T12:02:56.412 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112537.log 2023-10-18T12:02:56.413 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.157805.log.gz 2023-10-18T12:02:56.413 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141575.log 2023-10-18T12:02:56.413 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112537.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128103.log.gz 2023-10-18T12:02:56.414 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157553.log 2023-10-18T12:02:56.414 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141575.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112537.log.gz 2023-10-18T12:02:56.414 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133056.log 2023-10-18T12:02:56.414 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141575.log.gz 2023-10-18T12:02:56.415 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118236.log 2023-10-18T12:02:56.415 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.133056.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.157553.log.gz 2023-10-18T12:02:56.415 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160972.log 2023-10-18T12:02:56.415 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107143.log 2023-10-18T12:02:56.416 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.118236.log.gz 2023-10-18T12:02:56.416 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150462.log 2023-10-18T12:02:56.416 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.160972.log.gz 2023-10-18T12:02:56.417 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153435.log 2023-10-18T12:02:56.417 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107143.log: /var/log/ceph/ceph-client.admin.150462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108787.log 2023-10-18T12:02:56.417 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.107143.log.gz 2023-10-18T12:02:56.417 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153435.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.150462.log.gz 2023-10-18T12:02:56.418 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107185.log 2023-10-18T12:02:56.418 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.153435.log.gz 2023-10-18T12:02:56.418 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94832.log 2023-10-18T12:02:56.419 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108787.log.gz 2023-10-18T12:02:56.419 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117091.log 2023-10-18T12:02:56.419 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107185.log: /var/log/ceph/ceph-client.admin.94832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101260.log 2023-10-18T12:02:56.419 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.107185.log.gz 2023-10-18T12:02:56.420 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117091.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94832.log.gz 2023-10-18T12:02:56.420 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104354.log 2023-10-18T12:02:56.420 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159552.log 2023-10-18T12:02:56.421 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117091.log.gz 2023-10-18T12:02:56.421 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101260.log: /var/log/ceph/ceph-client.admin.104354.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101260.log.gz 2023-10-18T12:02:56.421 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135425.log 2023-10-18T12:02:56.421 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104354.log.gz 2023-10-18T12:02:56.422 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T12:02:56.422 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159552.log: /var/log/ceph/ceph-client.admin.135425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148712.log 2023-10-18T12:02:56.422 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.159552.log.gz 2023-10-18T12:02:56.422 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.135425.log.gz 2023-10-18T12:02:56.423 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112308.log 2023-10-18T12:02:56.423 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104978.log 2023-10-18T12:02:56.423 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.148712.log.gz 2023-10-18T12:02:56.424 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135383.log 2023-10-18T12:02:56.424 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112308.log.gz 2023-10-18T12:02:56.424 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96606.log 2023-10-18T12:02:56.424 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135383.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104978.log.gz 2023-10-18T12:02:56.425 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130793.log 2023-10-18T12:02:56.425 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135383.log.gz 2023-10-18T12:02:56.425 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148338.log 2023-10-18T12:02:56.425 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96606.log.gz 2023-10-18T12:02:56.426 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126288.log 2023-10-18T12:02:56.426 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144073.log 2023-10-18T12:02:56.426 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148338.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.130793.log.gz 2023-10-18T12:02:56.427 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126288.log: 93.7% -- replaced with /var/log/ceph/ceph-client.0.133056.log.gz 2023-10-18T12:02:56.427 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.148338.log.gz 2023-10-18T12:02:56.427 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110466.log 2023-10-18T12:02:56.427 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126288.log.gz 2023-10-18T12:02:56.428 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97492.log 2023-10-18T12:02:56.428 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144073.log: /var/log/ceph/ceph-client.admin.110466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102085.log 2023-10-18T12:02:56.428 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.144073.log.gz 2023-10-18T12:02:56.428 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.110466.log.gz 2023-10-18T12:02:56.429 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102879.log 2023-10-18T12:02:56.429 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102085.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.97492.log.gz 2023-10-18T12:02:56.429 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164028.log 2023-10-18T12:02:56.429 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102085.log.gz 2023-10-18T12:02:56.430 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156638.log 2023-10-18T12:02:56.430 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102879.log.gz 2023-10-18T12:02:56.430 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157095.log 2023-10-18T12:02:56.431 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.164028.log.gz 2023-10-18T12:02:56.431 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97489.log 2023-10-18T12:02:56.431 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156638.log: /var/log/ceph/ceph-client.admin.157095.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.156638.log.gz 2023-10-18T12:02:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150252.log 2023-10-18T12:02:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.157095.log.gz --verbose 2023-10-18T12:02:56.432 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.158430.log 2023-10-18T12:02:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97489.log: /var/log/ceph/ceph-client.admin.150252.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97489.log.gz 2023-10-18T12:02:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140957.log 2023-10-18T12:02:56.433 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.150252.log.gz 2023-10-18T12:02:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134944.log 2023-10-18T12:02:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158430.log: /var/log/ceph/ceph-client.admin.140957.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.158430.log.gz 2023-10-18T12:02:56.434 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109677.log 2023-10-18T12:02:56.434 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140957.log.gz 2023-10-18T12:02:56.434 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101428.log 2023-10-18T12:02:56.435 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134944.log: /var/log/ceph/ceph-client.admin.109677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162328.log 2023-10-18T12:02:56.435 INFO:teuthology.orchestra.run.smithi125.stderr: 90.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T12:02:56.435 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134944.log.gz 2023-10-18T12:02:56.435 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.109677.log.gz 2023-10-18T12:02:56.436 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148796.log 2023-10-18T12:02:56.436 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101428.log.gz 2023-10-18T12:02:56.436 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142002.log 2023-10-18T12:02:56.437 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162328.log.gz 2023-10-18T12:02:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131536.log 2023-10-18T12:02:56.437 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.148796.log.gz 2023-10-18T12:02:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132605.log 2023-10-18T12:02:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131536.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.142002.log.gz 2023-10-18T12:02:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154586.log 2023-10-18T12:02:56.438 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131536.log.gz 2023-10-18T12:02:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98490.log 2023-10-18T12:02:56.439 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.132605.log.gz 2023-10-18T12:02:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158914.log 2023-10-18T12:02:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98490.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154586.log.gz 2023-10-18T12:02:56.440 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103393.log 2023-10-18T12:02:56.440 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98490.log.gz 2023-10-18T12:02:56.440 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116854.log 2023-10-18T12:02:56.440 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.158914.log.gz 2023-10-18T12:02:56.441 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132479.log 2023-10-18T12:02:56.441 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.116854.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103393.log.gz 2023-10-18T12:02:56.441 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124791.log 2023-10-18T12:02:56.441 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.116854.log.gz 2023-10-18T12:02:56.442 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122756.log 2023-10-18T12:02:56.442 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.132479.log.gz 2023-10-18T12:02:56.442 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157950.log 2023-10-18T12:02:56.443 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122756.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124791.log.gz 2023-10-18T12:02:56.443 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95623.log 2023-10-18T12:02:56.443 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122756.log.gz 2023-10-18T12:02:56.443 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147032.log 2023-10-18T12:02:56.444 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.157950.log.gz 2023-10-18T12:02:56.444 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164422.log 2023-10-18T12:02:56.444 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.147032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105538.log 2023-10-18T12:02:56.445 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.147032.log.gz 2023-10-18T12:02:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164422.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.95623.log.gz 2023-10-18T12:02:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120430.log 2023-10-18T12:02:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.164422.log.gz -- 2023-10-18T12:02:56.446 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.136072.log 2023-10-18T12:02:56.446 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105538.log: /var/log/ceph/ceph-client.admin.120430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128333.log 2023-10-18T12:02:56.446 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105538.log.gz 2023-10-18T12:02:56.446 INFO:teuthology.orchestra.run.smithi125.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.120430.log.gz 2023-10-18T12:02:56.447 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106781.log 2023-10-18T12:02:56.447 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136072.log.gz 2023-10-18T12:02:56.447 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146337.log 2023-10-18T12:02:56.448 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.128333.log.gz 2023-10-18T12:02:56.448 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158244.log 2023-10-18T12:02:56.448 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106781.log.gz 2023-10-18T12:02:56.448 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120293.log 2023-10-18T12:02:56.449 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158244.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146337.log.gz 2023-10-18T12:02:56.449 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155676.log 2023-10-18T12:02:56.449 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.158244.log.gz 2023-10-18T12:02:56.449 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154502.log 2023-10-18T12:02:56.450 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120293.log.gz 2023-10-18T12:02:56.450 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.155676.log: /var/log/ceph/ceph-client.admin.122232.log 2023-10-18T12:02:56.450 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154502.log: gzip 87.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.155676.log.gz -- 2023-10-18T12:02:56.451 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.120346.log 2023-10-18T12:02:56.451 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122232.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154502.log.gz 2023-10-18T12:02:56.451 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165222.log 2023-10-18T12:02:56.451 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87700.log 2023-10-18T12:02:56.452 INFO:teuthology.orchestra.run.smithi125.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.122232.log.gz 2023-10-18T12:02:56.452 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.120346.log.gz 2023-10-18T12:02:56.452 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160718.log 2023-10-18T12:02:56.452 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162538.log 2023-10-18T12:02:56.453 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87700.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.165222.log.gz 2023-10-18T12:02:56.453 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145611.log 2023-10-18T12:02:56.453 INFO:teuthology.orchestra.run.smithi125.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.87700.log.gz 2023-10-18T12:02:56.453 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160718.log.gz 2023-10-18T12:02:56.454 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102205.log 2023-10-18T12:02:56.454 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145611.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162538.log.gz 2023-10-18T12:02:56.454 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102979.log 2023-10-18T12:02:56.455 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145611.log.gz 2023-10-18T12:02:56.455 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121336.log 2023-10-18T12:02:56.455 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102205.log.gz 2023-10-18T12:02:56.455 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151172.log 2023-10-18T12:02:56.456 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121336.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102979.log.gz 2023-10-18T12:02:56.456 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104726.log 2023-10-18T12:02:56.456 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121336.log.gz 2023-10-18T12:02:56.456 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.151172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98168.log 2023-10-18T12:02:56.457 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.151172.log.gz 2023-10-18T12:02:56.457 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145273.log 2023-10-18T12:02:56.457 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98168.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104726.log.gz 2023-10-18T12:02:56.458 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156722.log 2023-10-18T12:02:56.458 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145273.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98168.log.gz 2023-10-18T12:02:56.458 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98700.log 2023-10-18T12:02:56.458 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.145273.log.gz 2023-10-18T12:02:56.459 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111724.log 2023-10-18T12:02:56.459 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98700.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.156722.log.gz 2023-10-18T12:02:56.459 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148575.log 2023-10-18T12:02:56.459 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111724.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98700.log.gz 2023-10-18T12:02:56.460 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108088.log 2023-10-18T12:02:56.460 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111724.log.gz 2023-10-18T12:02:56.460 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141617.log 2023-10-18T12:02:56.461 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108088.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148575.log.gz 2023-10-18T12:02:56.461 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132647.log 2023-10-18T12:02:56.461 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108088.log.gz 2023-10-18T12:02:56.461 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131835.log 2023-10-18T12:02:56.462 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.141617.log.gz 2023-10-18T12:02:56.462 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157637.log 2023-10-18T12:02:56.462 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131835.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.132647.log.gz 2023-10-18T12:02:56.462 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117853.log 2023-10-18T12:02:56.463 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131835.log.gz 2023-10-18T12:02:56.463 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153729.log 2023-10-18T12:02:56.463 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157637.log.gz 2023-10-18T12:02:56.464 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166909.log 2023-10-18T12:02:56.464 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117853.log.gz 2023-10-18T12:02:56.464 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98028.log 2023-10-18T12:02:56.464 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.153729.log.gz 2023-10-18T12:02:56.465 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118070.log 2023-10-18T12:02:56.465 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98028.log: gzip -5 87.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.166909.log.gz 2023-10-18T12:02:56.465 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.123442.log 2023-10-18T12:02:56.466 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118070.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.98028.log.gz 2023-10-18T12:02:56.466 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102163.log 2023-10-18T12:02:56.466 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118070.log.gz 2023-10-18T12:02:56.466 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120598.log 2023-10-18T12:02:56.467 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123442.log.gz 2023-10-18T12:02:56.467 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101649.log 2023-10-18T12:02:56.467 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120598.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.102163.log.gz 2023-10-18T12:02:56.467 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.155466.log 2023-10-18T12:02:56.468 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120598.log.gz 2023-10-18T12:02:56.468 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123663.log 2023-10-18T12:02:56.468 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101649.log.gz 2023-10-18T12:02:56.469 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127414.log 2023-10-18T12:02:56.469 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123663.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.155466.log.gz 2023-10-18T12:02:56.469 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160328.log 2023-10-18T12:02:56.469 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123663.log.gz 2023-10-18T12:02:56.470 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136596.log 2023-10-18T12:02:56.470 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127414.log.gz 2023-10-18T12:02:56.470 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149090.log 2023-10-18T12:02:56.470 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160328.log: /var/log/ceph/ceph-client.admin.136596.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.160328.log.gz /var/log/ceph/ceph-client.admin.117990.log 2023-10-18T12:02:56.471 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.471 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95455.log 2023-10-18T12:02:56.471 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.149090.log.gz 2023-10-18T12:02:56.471 INFO:teuthology.orchestra.run.smithi125.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.136596.log.gz 2023-10-18T12:02:56.472 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114257.log 2023-10-18T12:02:56.472 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117990.log.gz 2023-10-18T12:02:56.472 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125087.log 2023-10-18T12:02:56.473 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95455.log.gz 2023-10-18T12:02:56.473 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.114257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101218.log 2023-10-18T12:02:56.473 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114257.log.gz 2023-10-18T12:02:56.473 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159426.log 2023-10-18T12:02:56.474 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101218.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.125087.log.gz 2023-10-18T12:02:56.474 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122106.log 2023-10-18T12:02:56.474 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101218.log.gz 2023-10-18T12:02:56.474 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145779.log 2023-10-18T12:02:56.475 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.159426.log.gz 2023-10-18T12:02:56.475 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146845.log 2023-10-18T12:02:56.475 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.122106.log.gz 2023-10-18T12:02:56.476 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87887.log 2023-10-18T12:02:56.476 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146845.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.145779.log.gz 2023-10-18T12:02:56.476 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106604.log 2023-10-18T12:02:56.476 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.146845.log.gz 2023-10-18T12:02:56.477 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100268.log 2023-10-18T12:02:56.477 INFO:teuthology.orchestra.run.smithi125.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.87887.log.gz 2023-10-18T12:02:56.477 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157511.log 2023-10-18T12:02:56.477 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106604.log.gz 2023-10-18T12:02:56.478 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165842.log 2023-10-18T12:02:56.478 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100268.log.gz 2023-10-18T12:02:56.478 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155550.log 2023-10-18T12:02:56.479 INFO:teuthology.orchestra.run.smithi125.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.157511.log.gz 2023-10-18T12:02:56.479 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123495.log 2023-10-18T12:02:56.479 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155550.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.165842.log.gz 2023-10-18T12:02:56.479 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127246.log 2023-10-18T12:02:56.480 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155550.log.gz 2023-10-18T12:02:56.480 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153813.log 2023-10-18T12:02:56.480 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.123495.log.gz 2023-10-18T12:02:56.481 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118628.log 2023-10-18T12:02:56.481 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153813.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127246.log.gz 2023-10-18T12:02:56.481 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112579.log 2023-10-18T12:02:56.481 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.153813.log.gz 2023-10-18T12:02:56.482 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166376.log 2023-10-18T12:02:56.482 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118628.log.gz 2023-10-18T12:02:56.482 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117643.log 2023-10-18T12:02:56.483 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166376.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112579.log.gz 2023-10-18T12:02:56.483 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145947.log 2023-10-18T12:02:56.483 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166376.log.gz 2023-10-18T12:02:56.483 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102457.log 2023-10-18T12:02:56.484 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117643.log.gz 2023-10-18T12:02:56.484 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130690.log 2023-10-18T12:02:56.484 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.145947.log.gz 2023-10-18T12:02:56.484 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157137.log 2023-10-18T12:02:56.485 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130690.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102457.log.gz 2023-10-18T12:02:56.485 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167053.log 2023-10-18T12:02:56.485 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130690.log.gz 2023-10-18T12:02:56.486 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166147.log 2023-10-18T12:02:56.486 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.157137.log.gz 2023-10-18T12:02:56.486 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87498.log 2023-10-18T12:02:56.486 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166147.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.167053.log.gz --verbose 2023-10-18T12:02:56.487 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.98532.log 2023-10-18T12:02:56.487 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166147.log.gz 2023-10-18T12:02:56.487 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87498.log.gz 2023-10-18T12:02:56.488 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127797.log 2023-10-18T12:02:56.488 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131242.log 2023-10-18T12:02:56.488 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127797.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98532.log.gz 2023-10-18T12:02:56.488 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116728.log 2023-10-18T12:02:56.489 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131242.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127797.log.gz 2023-10-18T12:02:56.489 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162978.log 2023-10-18T12:02:56.489 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.131242.log.gz 2023-10-18T12:02:56.489 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.116728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160454.log 2023-10-18T12:02:56.490 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162978.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116728.log.gz 2023-10-18T12:02:56.490 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161529.log 2023-10-18T12:02:56.490 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160454.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162978.log.gz 2023-10-18T12:02:56.491 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166783.log 2023-10-18T12:02:56.491 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.160454.log.gz 2023-10-18T12:02:56.491 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165521.log 2023-10-18T12:02:56.491 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114425.log 2023-10-18T12:02:56.492 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.161529.log.gz 2023-10-18T12:02:56.492 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.166783.log.gz/var/log/ceph/ceph-client.admin.165521.log: 2023-10-18T12:02:56.492 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.165521.log.gz 2023-10-18T12:02:56.493 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124403.log 2023-10-18T12:02:56.493 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.114425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104311.log 2023-10-18T12:02:56.493 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124403.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114425.log.gz 2023-10-18T12:02:56.493 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123849.log 2023-10-18T12:02:56.494 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.124403.log.gz 2023-10-18T12:02:56.494 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107101.log 2023-10-18T12:02:56.494 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123849.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104311.log.gz 2023-10-18T12:02:56.495 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164166.log 2023-10-18T12:02:56.495 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123849.log.gz 2023-10-18T12:02:56.495 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133970.log 2023-10-18T12:02:56.495 INFO:teuthology.orchestra.run.smithi125.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.107101.log.gz 2023-10-18T12:02:56.496 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142813.log 2023-10-18T12:02:56.496 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133970.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164166.log.gzgzip 2023-10-18T12:02:56.496 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.154034.log 2023-10-18T12:02:56.497 INFO:teuthology.orchestra.run.smithi125.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.133970.log.gz 2023-10-18T12:02:56.497 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113353.log 2023-10-18T12:02:56.497 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.142813.log.gz 2023-10-18T12:02:56.497 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144659.log 2023-10-18T12:02:56.498 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113353.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.154034.log.gz 2023-10-18T12:02:56.498 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121592.log 2023-10-18T12:02:56.498 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113353.log.gz 2023-10-18T12:02:56.498 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103267.log 2023-10-18T12:02:56.499 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.144659.log.gz 2023-10-18T12:02:56.499 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102592.log 2023-10-18T12:02:56.499 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103267.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121592.log.gz 2023-10-18T12:02:56.500 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135173.log 2023-10-18T12:02:56.500 INFO:teuthology.orchestra.run.smithi125.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103267.log.gz 2023-10-18T12:02:56.500 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161254.log 2023-10-18T12:02:56.500 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102592.log.gz 2023-10-18T12:02:56.501 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160412.log 2023-10-18T12:02:56.501 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161254.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135173.log.gz 2023-10-18T12:02:56.501 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160286.log 2023-10-18T12:02:56.501 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102373.log 2023-10-18T12:02:56.502 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.160412.log.gz -- replaced with /var/log/ceph/ceph-client.admin.161254.log.gz 2023-10-18T12:02:56.502 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.502 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164072.log 2023-10-18T12:02:56.503 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160286.log.gz 2023-10-18T12:02:56.503 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121420.log 2023-10-18T12:02:56.503 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9%/var/log/ceph/ceph-client.admin.164072.log: -- replaced with /var/log/ceph/ceph-client.admin.102373.log.gz 2023-10-18T12:02:56.503 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105496.log 2023-10-18T12:02:56.504 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.164072.log.gz 2023-10-18T12:02:56.504 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113089.log 2023-10-18T12:02:56.504 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105496.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121420.log.gz 2023-10-18T12:02:56.505 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123247.log 2023-10-18T12:02:56.505 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105496.log.gz 2023-10-18T12:02:56.505 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136242.log 2023-10-18T12:02:56.505 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113089.log.gz 2023-10-18T12:02:56.506 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88154.log 2023-10-18T12:02:56.506 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136242.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.123247.log.gz 2023-10-18T12:02:56.506 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132797.log 2023-10-18T12:02:56.506 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.136242.log.gz 2023-10-18T12:02:56.507 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.88154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162412.log 2023-10-18T12:02:56.507 INFO:teuthology.orchestra.run.smithi125.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.88154.log.gz 2023-10-18T12:02:56.507 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162064.log 2023-10-18T12:02:56.508 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162412.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.132797.log.gz 2023-10-18T12:02:56.508 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153172.log 2023-10-18T12:02:56.508 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162064.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.162412.log.gz 2023-10-18T12:02:56.508 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143556.log 2023-10-18T12:02:56.509 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162064.log.gz 2023-10-18T12:02:56.509 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125129.log 2023-10-18T12:02:56.509 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.153172.log.gz 2023-10-18T12:02:56.509 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121812.log 2023-10-18T12:02:56.510 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.143556.log.gz 2023-10-18T12:02:56.510 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121079.log 2023-10-18T12:02:56.510 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.125129.log.gz 2023-10-18T12:02:56.511 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163062.log 2023-10-18T12:02:56.511 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121079.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121812.log.gz 2023-10-18T12:02:56.511 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134986.log 2023-10-18T12:02:56.511 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163062.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121079.log.gz 2023-10-18T12:02:56.512 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-18T12:02:56.512 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.163062.log.gz 2023-10-18T12:02:56.512 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133305.log 2023-10-18T12:02:56.512 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-osd.1.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134986.log.gz 2023-10-18T12:02:56.513 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131673.log 2023-10-18T12:02:56.513 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96464.log 2023-10-18T12:02:56.513 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.133305.log.gz 2023-10-18T12:02:56.514 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96338.log 2023-10-18T12:02:56.514 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.131673.log.gz 2023-10-18T12:02:56.514 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107920.log 2023-10-18T12:02:56.514 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96464.log: /var/log/ceph/ceph-client.admin.96338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165937.log 2023-10-18T12:02:56.515 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96464.log.gz 2023-10-18T12:02:56.515 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96338.log.gz 2023-10-18T12:02:56.515 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102795.log 2023-10-18T12:02:56.516 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.107920.log.gz 2023-10-18T12:02:56.516 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96894.log 2023-10-18T12:02:56.516 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.165937.log.gz 2023-10-18T12:02:56.516 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106457.log 2023-10-18T12:02:56.517 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102795.log: /var/log/ceph/ceph-client.admin.96894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87998.log 2023-10-18T12:02:56.517 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102795.log.gz 2023-10-18T12:02:56.517 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96894.log.gz 2023-10-18T12:02:56.517 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144268.log 2023-10-18T12:02:56.518 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105200.log 2023-10-18T12:02:56.518 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87998.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106457.log.gz 2023-10-18T12:02:56.518 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144268.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.87998.log.gz 2023-10-18T12:02:56.519 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149402.log 2023-10-18T12:02:56.519 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.144268.log.gz 2023-10-18T12:02:56.519 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167305.log 2023-10-18T12:02:56.519 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105200.log: /var/log/ceph/ceph-client.admin.149402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95043.log 2023-10-18T12:02:56.520 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105200.log.gz 2023-10-18T12:02:56.520 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167305.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149402.log.gz 2023-10-18T12:02:56.520 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103519.log 2023-10-18T12:02:56.521 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.167305.log.gz 2023-10-18T12:02:56.521 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121462.log 2023-10-18T12:02:56.521 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94751.log 2023-10-18T12:02:56.521 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95043.log.gz 2023-10-18T12:02:56.522 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103519.log: /var/log/ceph/ceph-client.admin.121462.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103519.log.gz 2023-10-18T12:02:56.522 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94959.log 2023-10-18T12:02:56.522 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121462.log.gz 2023-10-18T12:02:56.522 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121378.log 2023-10-18T12:02:56.523 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.94751.log: /var/log/ceph/ceph-client.admin.94959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157721.log 2023-10-18T12:02:56.523 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94751.log.gz 2023-10-18T12:02:56.523 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121378.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94959.log.gz 2023-10-18T12:02:56.524 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164656.log 2023-10-18T12:02:56.524 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121378.log.gz 2023-10-18T12:02:56.524 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124527.log 2023-10-18T12:02:56.524 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96564.log 2023-10-18T12:02:56.525 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157721.log.gz 2023-10-18T12:02:56.525 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164656.log: /var/log/ceph/ceph-client.admin.124527.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.164656.log.gz 2023-10-18T12:02:56.525 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.160129.log 2023-10-18T12:02:56.525 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124527.log.gz 2023-10-18T12:02:56.526 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167137.log 2023-10-18T12:02:56.526 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96564.log: /var/log/ceph/ceph-client.0.160129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131055.log 2023-10-18T12:02:56.526 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96564.log.gz 2023-10-18T12:02:56.527 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141304.log 2023-10-18T12:02:56.527 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.167137.log.gz 2023-10-18T12:02:56.527 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122557.log 2023-10-18T12:02:56.527 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100378.log 2023-10-18T12:02:56.528 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.131055.log.gz 2023-10-18T12:02:56.528 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141304.log: /var/log/ceph/ceph-client.0.122557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158735.log 2023-10-18T12:02:56.528 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141304.log.gz 2023-10-18T12:02:56.529 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132311.log 2023-10-18T12:02:56.529 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100378.log.gz 2023-10-18T12:02:56.529 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158735.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.0.122557.log.gz 2023-10-18T12:02:56.529 INFO:teuthology.orchestra.run.smithi125.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.0.160129.log.gz 2023-10-18T12:02:56.530 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.133363.log 2023-10-18T12:02:56.530 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.158735.log.gz 2023-10-18T12:02:56.530 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117811.log 2023-10-18T12:02:56.531 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157290.log 2023-10-18T12:02:56.531 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.132311.log.gz 2023-10-18T12:02:56.531 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117811.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.133363.log.gz 2023-10-18T12:02:56.531 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159636.log 2023-10-18T12:02:56.532 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.117811.log.gz 2023-10-18T12:02:56.532 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87814.log 2023-10-18T12:02:56.532 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159636.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.157290.log.gz 2023-10-18T12:02:56.532 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135938.log 2023-10-18T12:02:56.533 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.159636.log.gz 2023-10-18T12:02:56.533 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127288.log 2023-10-18T12:02:56.533 INFO:teuthology.orchestra.run.smithi125.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.87814.log.gz 2023-10-18T12:02:56.534 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154990.log 2023-10-18T12:02:56.534 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135938.log: /var/log/ceph/ceph-client.admin.127288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153855.log 2023-10-18T12:02:56.534 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.135938.log.gz 2023-10-18T12:02:56.534 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127288.log.gz 2023-10-18T12:02:56.535 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142561.log 2023-10-18T12:02:56.535 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154990.log.gz 2023-10-18T12:02:56.535 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121770.log 2023-10-18T12:02:56.535 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.153855.log.gz 2023-10-18T12:02:56.536 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144310.log 2023-10-18T12:02:56.536 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142561.log: /var/log/ceph/ceph-client.admin.121770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127330.log 2023-10-18T12:02:56.536 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.142561.log.gz 2023-10-18T12:02:56.537 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121770.log.gz 2023-10-18T12:02:56.537 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131753.log 2023-10-18T12:02:56.537 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161014.log 2023-10-18T12:02:56.537 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.144310.log.gz 2023-10-18T12:02:56.538 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127330.log: /var/log/ceph/ceph-client.admin.131753.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.127330.log.gz 2023-10-18T12:02:56.538 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121938.log 2023-10-18T12:02:56.538 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.131753.log.gz 2023-10-18T12:02:56.539 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128585.log 2023-10-18T12:02:56.539 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133423.log 2023-10-18T12:02:56.539 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121938.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.161014.log.gz 2023-10-18T12:02:56.539 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128585.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.121938.log.gz 2023-10-18T12:02:56.540 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160888.log 2023-10-18T12:02:56.540 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128585.log.gz 2023-10-18T12:02:56.540 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161827.log 2023-10-18T12:02:56.540 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106722.log 2023-10-18T12:02:56.541 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160888.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133423.log.gz 2023-10-18T12:02:56.541 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117007.log 2023-10-18T12:02:56.541 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160888.log.gz 2023-10-18T12:02:56.542 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.161827.log.gz 2023-10-18T12:02:56.542 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151374.log 2023-10-18T12:02:56.542 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106722.log: /var/log/ceph/ceph-client.admin.117007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117330.log 2023-10-18T12:02:56.542 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.106722.log.gz 2023-10-18T12:02:56.543 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117007.log.gz 2023-10-18T12:02:56.543 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.151374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151279.log 2023-10-18T12:02:56.543 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.151374.log.gz 2023-10-18T12:02:56.543 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142435.log 2023-10-18T12:02:56.544 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose 87.1% -- /var/log/ceph/ceph-client.admin.122924.log -- replaced with /var/log/ceph/ceph-client.admin.117330.log.gz 2023-10-18T12:02:56.544 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.544 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.151279.log: /var/log/ceph/ceph-client.admin.142435.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.151279.log.gz 2023-10-18T12:02:56.544 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-client.admin.95085.log 2023-10-18T12:02:56.545 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.142435.log.gz 2023-10-18T12:02:56.545 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134171.log 2023-10-18T12:02:56.545 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122924.log: /var/log/ceph/ceph-client.admin.95085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136554.log 2023-10-18T12:02:56.546 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122924.log.gz 2023-10-18T12:02:56.546 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134171.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95085.log.gz 2023-10-18T12:02:56.546 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97488.log 2023-10-18T12:02:56.546 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.134171.log.gz 2023-10-18T12:02:56.547 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125724.log 2023-10-18T12:02:56.547 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98574.log 2023-10-18T12:02:56.547 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136554.log.gz/var/log/ceph/ceph-client.admin.97488.log: 2023-10-18T12:02:56.548 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.125724.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.97488.log.gz /var/log/ceph/ceph-client.admin.163760.log 2023-10-18T12:02:56.548 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.548 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149276.log 2023-10-18T12:02:56.548 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98574.log: /var/log/ceph/ceph-client.admin.163760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142645.log 2023-10-18T12:02:56.549 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98574.log.gz 2023-10-18T12:02:56.549 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%/var/log/ceph/ceph-client.admin.149276.log: -- replaced with /var/log/ceph/ceph-client.admin.163760.log.gz 2023-10-18T12:02:56.549 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137287.log 2023-10-18T12:02:56.549 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149276.log.gz 2023-10-18T12:02:56.550 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116625.log 2023-10-18T12:02:56.550 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95191.log 2023-10-18T12:02:56.550 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% 94.1% -- replaced with /var/log/ceph/ceph-client.admin.142645.log.gz -- replaced with /var/log/ceph/ceph-client.0.125724.log.gz 2023-10-18T12:02:56.551 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.551 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.137287.log: /var/log/ceph/ceph-client.admin.116625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129404.log 2023-10-18T12:02:56.551 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137287.log.gz 2023-10-18T12:02:56.551 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116625.log.gz 2023-10-18T12:02:56.552 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144719.log 2023-10-18T12:02:56.552 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117727.log 2023-10-18T12:02:56.552 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.95191.log.gz 2023-10-18T12:02:56.552 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.129404.log: /var/log/ceph/ceph-client.admin.144719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108214.log 2023-10-18T12:02:56.553 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129404.log.gz 2023-10-18T12:02:56.553 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144719.log.gz 2023-10-18T12:02:56.553 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152977.log 2023-10-18T12:02:56.554 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117727.log: /var/log/ceph/ceph-client.admin.108214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157992.log 2023-10-18T12:02:56.554 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117727.log.gz 2023-10-18T12:02:56.554 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108214.log.gz 2023-10-18T12:02:56.554 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.152977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107311.log 2023-10-18T12:02:56.555 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.152977.log.gz 2023-10-18T12:02:56.555 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149318.log 2023-10-18T12:02:56.555 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135850.log 2023-10-18T12:02:56.555 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157992.log.gz 2023-10-18T12:02:56.556 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107311.log: /var/log/ceph/ceph-client.admin.149318.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.107311.log.gz 2023-10-18T12:02:56.556 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131097.log 2023-10-18T12:02:56.556 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149318.log.gz 2023-10-18T12:02:56.556 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108669.log 2023-10-18T12:02:56.557 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135850.log: /var/log/ceph/ceph-client.admin.131097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128273.log 2023-10-18T12:02:56.557 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135850.log.gz 2023-10-18T12:02:56.557 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108669.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.131097.log.gz 2023-10-18T12:02:56.558 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129036.log 2023-10-18T12:02:56.558 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.108669.log.gz 2023-10-18T12:02:56.558 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120472.log 2023-10-18T12:02:56.558 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.128273.log.gz 2023-10-18T12:02:56.559 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118458.log 2023-10-18T12:02:56.559 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.129036.log: /var/log/ceph/ceph-client.admin.120472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126433.log 2023-10-18T12:02:56.559 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.120472.log.gz 2023-10-18T12:02:56.559 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103351.log 2023-10-18T12:02:56.560 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.118458.log.gz 2023-10-18T12:02:56.560 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.133806.log 2023-10-18T12:02:56.560 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126433.log: /var/log/ceph/ceph-client.admin.103351.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.126433.log.gzgzip 2023-10-18T12:02:56.560 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.143506.log 2023-10-18T12:02:56.561 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146031.log 2023-10-18T12:02:56.561 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133806.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103351.log.gz 2023-10-18T12:02:56.561 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143506.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.133806.log.gz 2023-10-18T12:02:56.562 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164208.log 2023-10-18T12:02:56.562 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.143506.log.gz 2023-10-18T12:02:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132395.log 2023-10-18T12:02:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146031.log: /var/log/ceph/ceph-client.admin.164208.log: gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146031.log.gz 2023-10-18T12:02:56.565 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-client.admin.157385.log 2023-10-18T12:02:56.566 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164208.log.gz 2023-10-18T12:02:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97272.log 2023-10-18T12:02:56.566 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.132395.log.gz 2023-10-18T12:02:56.567 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132353.log 2023-10-18T12:02:56.567 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.157385.log.gz 2023-10-18T12:02:56.567 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161162.log 2023-10-18T12:02:56.567 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97272.log: /var/log/ceph/ceph-client.admin.132353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98994.log 2023-10-18T12:02:56.568 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97272.log.gz 2023-10-18T12:02:56.568 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.132353.log.gz 2023-10-18T12:02:56.568 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163375.log 2023-10-18T12:02:56.568 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161911.log 2023-10-18T12:02:56.569 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.161162.log.gz 2023-10-18T12:02:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98994.log: /var/log/ceph/ceph-client.admin.163375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95285.log 87.5% 2023-10-18T12:02:56.569 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98994.log.gz 2023-10-18T12:02:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104438.log 2023-10-18T12:02:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161911.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.163375.log.gz 2023-10-18T12:02:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95285.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.161911.log.gz 2023-10-18T12:02:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142168.log 2023-10-18T12:02:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100630.log 2023-10-18T12:02:56.571 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3%/var/log/ceph/ceph-client.admin.104438.log: -- replaced with /var/log/ceph/ceph-client.admin.95285.log.gz 2023-10-18T12:02:56.571 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104438.log.gz 2023-10-18T12:02:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111273.log 2023-10-18T12:02:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100630.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.142168.log.gz 2023-10-18T12:02:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153267.log 2023-10-18T12:02:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111273.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100630.log.gz 2023-10-18T12:02:56.572 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133929.log 2023-10-18T12:02:56.573 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.111273.log.gz 2023-10-18T12:02:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131410.log 2023-10-18T12:02:56.573 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.150913.log 2023-10-18T12:02:56.574 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133929.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.153267.log.gz 2023-10-18T12:02:56.574 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131410.log: gzip -5 84.2% -- replaced with /var/log/ceph/ceph-client.admin.133929.log.gz 2023-10-18T12:02:56.574 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103915.log 2023-10-18T12:02:56.574 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.131410.log.gz 2023-10-18T12:02:56.575 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.150913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131139.log 2023-10-18T12:02:56.575 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140744.log 2023-10-18T12:02:56.575 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103915.log.gz 2023-10-18T12:02:56.575 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134860.log 2023-10-18T12:02:56.576 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131139.log.gz 2023-10-18T12:02:56.576 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.140744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126559.log 2023-10-18T12:02:56.576 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.140744.log.gz 2023-10-18T12:02:56.577 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157469.log 2023-10-18T12:02:56.577 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.127017.log/var/log/ceph/ceph-client.admin.126559.log: 2023-10-18T12:02:56.577 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.134860.log.gz 2023-10-18T12:02:56.577 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101008.log 2023-10-18T12:02:56.578 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126559.log.gz 2023-10-18T12:02:56.578 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159510.log 2023-10-18T12:02:56.578 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.157469.log.gz 2023-10-18T12:02:56.578 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157908.log 2023-10-18T12:02:56.579 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127017.log.gz 2023-10-18T12:02:56.579 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101008.log.gz 2023-10-18T12:02:56.579 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162640.log 2023-10-18T12:02:56.579 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.159510.log.gz/var/log/ceph/ceph-client.admin.157908.log: 2023-10-18T12:02:56.580 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152706.log 2023-10-18T12:02:56.580 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121896.log 2023-10-18T12:02:56.580 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.157908.log.gz 2023-10-18T12:02:56.581 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.152706.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.157679.log -- replaced with /var/log/ceph/ceph-client.admin.162640.log.gz 2023-10-18T12:02:56.581 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.581 INFO:teuthology.orchestra.run.smithi125.stderr:gzip/var/log/ceph/ceph-client.admin.121896.log: -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152706.log.gz 2023-10-18T12:02:56.581 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-client.admin.165758.log 2023-10-18T12:02:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157679.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121896.log.gz 2023-10-18T12:02:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127161.log 2023-10-18T12:02:56.582 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157679.log.gz 2023-10-18T12:02:56.582 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111010.log 2023-10-18T12:02:56.583 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127161.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148192.log 2023-10-18T12:02:56.583 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.165758.log.gz 2023-10-18T12:02:56.583 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.127161.log.gz 2023-10-18T12:02:56.584 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156969.log 2023-10-18T12:02:56.584 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148192.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.111010.log.gz 2023-10-18T12:02:56.584 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156142.log 2023-10-18T12:02:56.584 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%/var/log/ceph/ceph-client.admin.156969.log: -- replaced with /var/log/ceph/ceph-client.admin.148192.log.gz 2023-10-18T12:02:56.585 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166292.log 2023-10-18T12:02:56.585 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.156969.log.gz 2023-10-18T12:02:56.585 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126475.log 2023-10-18T12:02:56.586 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156142.log: /var/log/ceph/ceph-client.admin.166292.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.156142.log.gz 2023-10-18T12:02:56.586 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.161701.log 2023-10-18T12:02:56.586 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166292.log.gz 2023-10-18T12:02:56.586 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112663.log 2023-10-18T12:02:56.587 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126475.log.gz 2023-10-18T12:02:56.587 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145989.log 2023-10-18T12:02:56.587 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.161701.log.gz 2023-10-18T12:02:56.587 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87554.log 2023-10-18T12:02:56.588 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112663.log.gz 2023-10-18T12:02:56.588 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165114.log 2023-10-18T12:02:56.588 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145989.log: /var/log/ceph/ceph-client.admin.87554.log: gzip -5 87.3% -- replaced with /var/log/ceph/ceph-client.admin.145989.log.gz 2023-10-18T12:02:56.589 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- 77.0% -- replaced with /var/log/ceph/ceph-client.admin.87554.log.gz 2023-10-18T12:02:56.589 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.135089.log 2023-10-18T12:02:56.589 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126031.log 2023-10-18T12:02:56.589 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.165114.log.gz 2023-10-18T12:02:56.590 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158160.log 2023-10-18T12:02:56.590 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135089.log.gz 2023-10-18T12:02:56.590 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109151.log 2023-10-18T12:02:56.590 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158160.log: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156680.log 2023-10-18T12:02:56.591 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.126031.log.gz 2023-10-18T12:02:56.591 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.158160.log.gz 2023-10-18T12:02:56.591 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.109151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135341.log 2023-10-18T12:02:56.592 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156680.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.109151.log.gz 2023-10-18T12:02:56.592 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155508.log 2023-10-18T12:02:56.592 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.156680.log.gz 2023-10-18T12:02:56.592 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145397.log 2023-10-18T12:02:56.593 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.135341.log.gz 2023-10-18T12:02:56.593 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131368.log 2023-10-18T12:02:56.593 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.155508.log.gz 2023-10-18T12:02:56.593 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147519.log 2023-10-18T12:02:56.594 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145397.log.gz 2023-10-18T12:02:56.594 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131368.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.156596.log 2023-10-18T12:02:56.594 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.147519.log: gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.131368.log.gz 2023-10-18T12:02:56.595 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-18T12:02:56.595 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.147519.log.gz 2023-10-18T12:02:56.595 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155634.log 2023-10-18T12:02:56.595 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.156596.log.gz 2023-10-18T12:02:56.596 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122022.log 2023-10-18T12:02:56.596 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166418.log 2023-10-18T12:02:56.596 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155634.log.gz 2023-10-18T12:02:56.596 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122022.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122022.log.gz 2023-10-18T12:02:56.597 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.97494.log 2023-10-18T12:02:56.597 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101344.log 2023-10-18T12:02:56.597 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.166418.log.gz 2023-10-18T12:02:56.598 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156184.log 2023-10-18T12:02:56.598 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1%/var/log/ceph/ceph-client.admin.101344.log: -- replaced with /var/log/ceph/ceph-client.admin.97494.log.gz 2023-10-18T12:02:56.598 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127101.log 2023-10-18T12:02:56.598 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101344.log.gz 2023-10-18T12:02:56.599 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132563.log 2023-10-18T12:02:56.599 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102921.log 2023-10-18T12:02:56.599 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127101.log: /var/log/ceph/ceph-client.admin.132563.log: 87.1% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.127101.log.gz 2023-10-18T12:02:56.599 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.156184.log.gz 2023-10-18T12:02:56.600 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.132563.log.gz 2023-10-18T12:02:56.600 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128627.log 2023-10-18T12:02:56.600 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128459.log 2023-10-18T12:02:56.601 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.102921.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.108888.log 2023-10-18T12:02:56.601 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.601 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128627.log: /var/log/ceph/ceph-client.admin.128459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118670.log 2023-10-18T12:02:56.601 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6%/var/log/ceph/ceph-client.admin.108888.log: -- replaced with /var/log/ceph/ceph-client.admin.128627.log.gz 87.2% 2023-10-18T12:02:56.602 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.128459.log.gz 2023-10-18T12:02:56.602 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108888.log.gz 2023-10-18T12:02:56.602 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98160.log 2023-10-18T12:02:56.602 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140846.log 2023-10-18T12:02:56.603 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118670.log.gz 2023-10-18T12:02:56.603 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133688.log 2023-10-18T12:02:56.603 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.98160.log.gz 2023-10-18T12:02:56.604 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161470.log 2023-10-18T12:02:56.604 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.140846.log: /var/log/ceph/ceph-client.admin.133688.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.140846.log.gz 2023-10-18T12:02:56.604 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.133688.log.gz 2023-10-18T12:02:56.604 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167263.log 2023-10-18T12:02:56.605 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136470.log 2023-10-18T12:02:56.605 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161470.log: /var/log/ceph/ceph-client.admin.167263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163924.log 2023-10-18T12:02:56.605 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136470.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.161470.log.gz 2023-10-18T12:02:56.605 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.167263.log.gz 2023-10-18T12:02:56.606 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148422.log 2023-10-18T12:02:56.606 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.136470.log.gz 2023-10-18T12:02:56.606 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136284.log 2023-10-18T12:02:56.607 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163924.log: gzip/var/log/ceph/ceph-client.admin.148422.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.153393.log 2023-10-18T12:02:56.607 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.163924.log.gz 2023-10-18T12:02:56.607 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6%/var/log/ceph/ceph-client.admin.136284.log: -- replaced with /var/log/ceph/ceph-client.admin.148422.log.gz 2023-10-18T12:02:56.607 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 86.7% --verbose -- /var/log/ceph/ceph-client.admin.143048.log 2023-10-18T12:02:56.608 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.136284.log.gz 2023-10-18T12:02:56.608 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163291.log 2023-10-18T12:02:56.608 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.153393.log.gz/var/log/ceph/ceph-client.admin.143048.log: 2023-10-18T12:02:56.608 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154114.log 2023-10-18T12:02:56.609 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143048.log.gz 2023-10-18T12:02:56.609 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127839.log 2023-10-18T12:02:56.609 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88153.log 2023-10-18T12:02:56.610 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154114.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.163291.log.gz 2023-10-18T12:02:56.610 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127839.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154114.log.gz 2023-10-18T12:02:56.610 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127839.log.gz 2023-10-18T12:02:56.610 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.154630.log 2023-10-18T12:02:56.611 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.88153.log: gzip -5 --verbose -- 86.1% /var/log/ceph/ceph-client.admin.123163.log 2023-10-18T12:02:56.611 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88153.log.gz 2023-10-18T12:02:56.611 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127540.log 2023-10-18T12:02:56.612 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154630.log.gz 2023-10-18T12:02:56.612 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126712.log 2023-10-18T12:02:56.612 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104083.log 2023-10-18T12:02:56.612 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123163.log.gz/var/log/ceph/ceph-client.admin.127540.log: 2023-10-18T12:02:56.613 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126712.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.127540.log.gz 2023-10-18T12:02:56.613 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.126712.log.gz 2023-10-18T12:02:56.613 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142940.log 2023-10-18T12:02:56.614 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163501.log 2023-10-18T12:02:56.614 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8%/var/log/ceph/ceph-client.admin.142940.log: -- replaced with /var/log/ceph/ceph-client.admin.104083.log.gz 2023-10-18T12:02:56.614 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124323.log 2023-10-18T12:02:56.614 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142940.log.gz 2023-10-18T12:02:56.615 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155806.log 2023-10-18T12:02:56.615 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119624.log 2023-10-18T12:02:56.615 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124323.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.163501.log.gz 2023-10-18T12:02:56.616 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149785.log 2023-10-18T12:02:56.616 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155806.log.gz 2023-10-18T12:02:56.616 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124323.log.gz 2023-10-18T12:02:56.616 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119624.log: gzip -5 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119624.log.gz 2023-10-18T12:02:56.617 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106230.log 2023-10-18T12:02:56.617 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148659.log 94.5% -- replaced with /var/log/ceph/ceph-client.0.129036.log.gz 2023-10-18T12:02:56.617 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149785.log.gz 2023-10-18T12:02:56.617 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.618 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137228.log 2023-10-18T12:02:56.618 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.106230.log.gz 2023-10-18T12:02:56.618 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123537.log 2023-10-18T12:02:56.619 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1%/var/log/ceph/ceph-client.admin.137228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141533.log 2023-10-18T12:02:56.619 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.148659.log.gz 2023-10-18T12:02:56.619 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123537.log: gzip -5 --verbose -- 87.0% /var/log/ceph/ceph-client.admin.134011.log 2023-10-18T12:02:56.619 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.137228.log.gz 2023-10-18T12:02:56.620 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102289.log 2023-10-18T12:02:56.620 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.123537.log.gz 2023-10-18T12:02:56.620 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141533.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.134011.log: -- /var/log/ceph/ceph-client.admin.94917.log 2023-10-18T12:02:56.620 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.141533.log.gz 2023-10-18T12:02:56.621 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102289.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.134011.log.gz 2023-10-18T12:02:56.621 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102289.log.gz 2023-10-18T12:02:56.621 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.134734.log 2023-10-18T12:02:56.622 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.94917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155764.log 2023-10-18T12:02:56.622 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94917.log.gz 2023-10-18T12:02:56.622 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120995.log 2023-10-18T12:02:56.622 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.134734.log.gz 2023-10-18T12:02:56.623 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134402.log 2023-10-18T12:02:56.623 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.155764.log.gz 2023-10-18T12:02:56.623 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110203.log 2023-10-18T12:02:56.624 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.120995.log.gz 2023-10-18T12:02:56.624 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144888.log 2023-10-18T12:02:56.624 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.134402.log.gz 2023-10-18T12:02:56.624 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.110203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165479.log 2023-10-18T12:02:56.625 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.110203.log.gz 2023-10-18T12:02:56.625 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158693.log 2023-10-18T12:02:56.625 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.144888.log.gz 2023-10-18T12:02:56.625 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142296.log 2023-10-18T12:02:56.626 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.165479.log.gz 2023-10-18T12:02:56.626 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96296.log 2023-10-18T12:02:56.626 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.158693.log.gz 2023-10-18T12:02:56.626 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161211.log 2023-10-18T12:02:56.627 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112056.log 2023-10-18T12:02:56.627 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.142296.log.gz 2023-10-18T12:02:56.627 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96296.log.gz 2023-10-18T12:02:56.628 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148880.log 2023-10-18T12:02:56.628 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161211.log: /var/log/ceph/ceph-client.admin.112056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136158.log 2023-10-18T12:02:56.628 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1%/var/log/ceph/ceph-client.admin.148880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97104.log 2023-10-18T12:02:56.628 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112056.log.gz 2023-10-18T12:02:56.629 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.161211.log.gz 2023-10-18T12:02:56.629 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136158.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148880.log.gz 2023-10-18T12:02:56.629 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104518.log 2023-10-18T12:02:56.630 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136158.log.gz 2023-10-18T12:02:56.630 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108315.log 2023-10-18T12:02:56.630 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.97104.log.gz 2023-10-18T12:02:56.630 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131877.log 2023-10-18T12:02:56.631 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104518.log.gz 2023-10-18T12:02:56.631 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98406.log 2023-10-18T12:02:56.631 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6%/var/log/ceph/ceph-client.admin.131877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159382.log 2023-10-18T12:02:56.631 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108315.log.gz 2023-10-18T12:02:56.632 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6%/var/log/ceph/ceph-client.admin.98406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104041.log 2023-10-18T12:02:56.632 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.131877.log.gz 2023-10-18T12:02:56.632 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98406.log.gz 2023-10-18T12:02:56.633 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123747.log 2023-10-18T12:02:56.633 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127923.log 2023-10-18T12:02:56.633 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.159382.log.gz/var/log/ceph/ceph-client.admin.104041.log: 2023-10-18T12:02:56.633 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112410.log 2023-10-18T12:02:56.634 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104041.log.gz 2023-10-18T12:02:56.634 INFO:teuthology.orchestra.run.smithi125.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123747.log.gz 2023-10-18T12:02:56.634 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100756.log 2023-10-18T12:02:56.634 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127923.log: /var/log/ceph/ceph-client.admin.112410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132689.log 2023-10-18T12:02:56.635 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%/var/log/ceph/ceph-client.admin.100756.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112410.log.gz 2023-10-18T12:02:56.635 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98167.log 2023-10-18T12:02:56.635 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.127923.log.gz 2023-10-18T12:02:56.636 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7%/var/log/ceph/ceph-client.admin.132689.log: -- replaced with /var/log/ceph/ceph-client.admin.100756.log.gz 2023-10-18T12:02:56.636 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120827.log 2023-10-18T12:02:56.636 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.132689.log.gz 2023-10-18T12:02:56.636 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97831.log 2023-10-18T12:02:56.637 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98167.log.gz 2023-10-18T12:02:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103561.log 2023-10-18T12:02:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97831.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120827.log.gz 2023-10-18T12:02:56.637 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117204.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97831.log.gz 2023-10-18T12:02:56.638 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.638 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143744.log 2023-10-18T12:02:56.638 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103561.log.gz 2023-10-18T12:02:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.155249.log 2023-10-18T12:02:56.639 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141346.log 2023-10-18T12:02:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143744.log: -- replaced with /var/log/ceph/ceph-client.admin.117204.log.gz 2023-10-18T12:02:56.639 INFO:teuthology.orchestra.run.smithi125.stderr:gzip/var/log/ceph/ceph-client.0.155249.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.165605.log 2023-10-18T12:02:56.640 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.143744.log.gz 2023-10-18T12:02:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.141346.log: -- /var/log/ceph/ceph-client.admin.128053.log 2023-10-18T12:02:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165605.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141346.log.gz 2023-10-18T12:02:56.640 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104641.log 2023-10-18T12:02:56.641 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.165605.log.gz 2023-10-18T12:02:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112965.log 2023-10-18T12:02:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156459.log 2023-10-18T12:02:56.641 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104641.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128053.log.gz 2023-10-18T12:02:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112965.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104641.log.gz 2023-10-18T12:02:56.642 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165647.log 2023-10-18T12:02:56.642 INFO:teuthology.orchestra.run.smithi125.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.0.155249.log.gz 2023-10-18T12:02:56.643 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112965.log.gz 2023-10-18T12:02:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118712.log 2023-10-18T12:02:56.643 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156459.log.gz 2023-10-18T12:02:56.643 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149701.log 2023-10-18T12:02:56.644 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.165647.log.gz 2023-10-18T12:02:56.644 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95581.log 2023-10-18T12:02:56.644 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.118712.log.gz/var/log/ceph/ceph-client.admin.149701.log: 2023-10-18T12:02:56.645 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147801.log 2023-10-18T12:02:56.645 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149701.log.gz 2023-10-18T12:02:56.645 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156400.log 2023-10-18T12:02:56.645 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95581.log.gz 2023-10-18T12:02:56.646 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.147801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95147.log 2023-10-18T12:02:56.646 INFO:teuthology.orchestra.run.smithi125.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.147801.log.gz 2023-10-18T12:02:56.646 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103603.log 2023-10-18T12:02:56.647 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.156400.log.gz 2023-10-18T12:02:56.647 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151070.log 2023-10-18T12:02:56.647 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%/var/log/ceph/ceph-client.admin.103603.log: -- replaced with /var/log/ceph/ceph-client.admin.95147.log.gz 2023-10-18T12:02:56.647 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153225.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103603.log.gz 2023-10-18T12:02:56.648 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.648 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.151070.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.151070.log.gz 2023-10-18T12:02:56.648 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158472.log 2023-10-18T12:02:56.648 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128009.log 2023-10-18T12:02:56.649 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.153225.log.gz 2023-10-18T12:02:56.649 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163020.log 2023-10-18T12:02:56.649 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.158472.log.gz 2023-10-18T12:02:56.649 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107059.log 2023-10-18T12:02:56.650 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98166.log 2023-10-18T12:02:56.650 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128009.log.gz 2023-10-18T12:02:56.650 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101875.log 2023-10-18T12:02:56.651 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163020.log.gz 2023-10-18T12:02:56.651 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.107059.log.gz 2023-10-18T12:02:56.651 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136368.log 2023-10-18T12:02:56.651 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98166.log: /var/log/ceph/ceph-client.admin.101875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116770.log 2023-10-18T12:02:56.652 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98166.log.gz 2023-10-18T12:02:56.652 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101875.log.gz 2023-10-18T12:02:56.652 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113439.log 2023-10-18T12:02:56.652 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136368.log: /var/log/ceph/ceph-client.admin.116770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143947.log 2023-10-18T12:02:56.653 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0%/var/log/ceph/ceph-client.admin.113439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108046.log 2023-10-18T12:02:56.653 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116770.log.gz 2023-10-18T12:02:56.653 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136368.log.gz 2023-10-18T12:02:56.654 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113439.log.gz 2023-10-18T12:02:56.654 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104209.log 2023-10-18T12:02:56.654 INFO:teuthology.orchestra.run.smithi125.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.0.150913.log.gz 2023-10-18T12:02:56.654 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.143947.log.gz 2023-10-18T12:02:56.655 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135257.log 2023-10-18T12:02:56.655 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121037.log 2023-10-18T12:02:56.655 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108046.log.gz 2023-10-18T12:02:56.655 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104209.log: /var/log/ceph/ceph-client.admin.135257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146795.log 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104209.log.gz 2023-10-18T12:02:56.656 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.656 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135257.log.gz 2023-10-18T12:02:56.656 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129286.log 2023-10-18T12:02:56.656 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121037.log.gz 2023-10-18T12:02:56.657 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121254.log 2023-10-18T12:02:56.657 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146795.log: /var/log/ceph/ceph-client.admin.129286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159804.log 2023-10-18T12:02:56.657 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.146795.log.gz 2023-10-18T12:02:56.658 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.129286.log.gz 2023-10-18T12:02:56.658 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158819.log 2023-10-18T12:02:56.658 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121254.log.gz 2023-10-18T12:02:56.659 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159804.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.130411.log 2023-10-18T12:02:56.659 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158819.log: gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.158819.log.gz 2023-10-18T12:02:56.659 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.100714.log 2023-10-18T12:02:56.659 INFO:teuthology.orchestra.run.smithi125.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.159804.log.gz 2023-10-18T12:02:56.660 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132437.log 2023-10-18T12:02:56.660 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.130411.log.gz 2023-10-18T12:02:56.660 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158514.log 2023-10-18T12:02:56.660 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100714.log.gz 2023-10-18T12:02:56.661 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152935.log 2023-10-18T12:02:56.661 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5/var/log/ceph/ceph-client.admin.158514.log: --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.132437.log.gz 2023-10-18T12:02:56.661 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.103705.log 2023-10-18T12:02:56.662 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.152935.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.158514.log.gz 2023-10-18T12:02:56.662 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124102.log 2023-10-18T12:02:56.662 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.152935.log.gz 2023-10-18T12:02:56.663 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125465.log 2023-10-18T12:02:56.663 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124102.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103705.log.gz 2023-10-18T12:02:56.663 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101775.log 2023-10-18T12:02:56.663 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124102.log.gz 2023-10-18T12:02:56.664 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155424.log 2023-10-18T12:02:56.664 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108610.log 2023-10-18T12:02:56.664 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8%/var/log/ceph/ceph-client.admin.101775.log: -- replaced with /var/log/ceph/ceph-client.admin.125465.log.gz 2023-10-18T12:02:56.664 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163417.log 2023-10-18T12:02:56.665 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101775.log.gz 2023-10-18T12:02:56.665 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108610.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.155424.log.gz 2023-10-18T12:02:56.665 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132003.log 2023-10-18T12:02:56.666 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.108610.log.gz 2023-10-18T12:02:56.666 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123121.log 2023-10-18T12:02:56.666 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132003.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163417.log.gz 2023-10-18T12:02:56.666 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.87848.log 2023-10-18T12:02:56.667 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.132003.log.gz 2023-10-18T12:02:56.667 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103789.log 2023-10-18T12:02:56.667 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123121.log: /var/log/ceph/ceph-client.admin.87848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98162.log 2023-10-18T12:02:56.667 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123121.log.gz 2023-10-18T12:02:56.668 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112350.log 2023-10-18T12:02:56.668 INFO:teuthology.orchestra.run.smithi125.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.87848.log.gz 2023-10-18T12:02:56.668 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103789.log.gz 2023-10-18T12:02:56.669 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117559.log 2023-10-18T12:02:56.669 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.98162.log.gz 2023-10-18T12:02:56.669 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.152611.log 2023-10-18T12:02:56.669 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112350.log: /var/log/ceph/ceph-client.admin.117559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165376.log 2023-10-18T12:02:56.670 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.112350.log.gz 2023-10-18T12:02:56.670 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.152611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154364.log 2023-10-18T12:02:56.670 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117559.log.gz 2023-10-18T12:02:56.671 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.152611.log.gz 2023-10-18T12:02:56.671 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.165376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114485.log 2023-10-18T12:02:56.671 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127635.log 2023-10-18T12:02:56.671 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165376.log.gz 2023-10-18T12:02:56.672 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154364.log: /var/log/ceph/ceph-client.admin.114485.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154364.log.gz 2023-10-18T12:02:56.672 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.141491.log 2023-10-18T12:02:56.672 INFO:teuthology.orchestra.run.smithi125.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114485.log.gz 2023-10-18T12:02:56.673 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113649.log 2023-10-18T12:02:56.673 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.127635.log.gz 2023-10-18T12:02:56.673 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103309.log 2023-10-18T12:02:56.673 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141491.log.gz 2023-10-18T12:02:56.674 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140557.log 2023-10-18T12:02:56.674 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.113649.log.gz 2023-10-18T12:02:56.674 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128777.log 2023-10-18T12:02:56.674 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103309.log.gz 2023-10-18T12:02:56.675 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.140557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101470.log 2023-10-18T12:02:56.675 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128777.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.140557.log.gz 2023-10-18T12:02:56.675 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105622.log 2023-10-18T12:02:56.676 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128777.log.gz 2023-10-18T12:02:56.676 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128543.log 2023-10-18T12:02:56.676 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105622.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101470.log.gz 2023-10-18T12:02:56.676 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141448.log 2023-10-18T12:02:56.677 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105622.log.gz 2023-10-18T12:02:56.677 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112705.log 2023-10-18T12:02:56.677 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128543.log.gz 2023-10-18T12:02:56.677 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119572.log 2023-10-18T12:02:56.678 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112705.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.141448.log.gz 2023-10-18T12:02:56.678 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127204.log 2023-10-18T12:02:56.678 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112705.log.gz 2023-10-18T12:02:56.679 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102837.log 2023-10-18T12:02:56.679 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119572.log.gz 2023-10-18T12:02:56.679 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96380.log 2023-10-18T12:02:56.679 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102837.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127204.log.gz 2023-10-18T12:02:56.680 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116514.log 2023-10-18T12:02:56.680 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%/var/log/ceph/ceph-client.admin.96380.log: -- replaced with /var/log/ceph/ceph-client.admin.102837.log.gz 2023-10-18T12:02:56.680 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-18T12:02:56.681 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96380.log.gz 2023-10-18T12:02:56.681 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.116514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105814.log 2023-10-18T12:02:56.681 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106840.log 2023-10-18T12:02:56.681 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116514.log.gz 2023-10-18T12:02:56.682 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105814.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105814.log.gz 2023-10-18T12:02:56.682 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105158.log 2023-10-18T12:02:56.682 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126073.log 2023-10-18T12:02:56.683 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8%/var/log/ceph/ceph-client.admin.105158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140660.log 2023-10-18T12:02:56.683 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106840.log.gz 2023-10-18T12:02:56.683 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105158.log.gz 2023-10-18T12:02:56.683 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5/var/log/ceph/ceph-client.admin.126073.log: --verbose -- /var/log/ceph/ceph-client.admin.150126.log 2023-10-18T12:02:56.684 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111640.log 2023-10-18T12:02:56.684 INFO:teuthology.orchestra.run.smithi125.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126073.log.gz 2023-10-18T12:02:56.684 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.140660.log: /var/log/ceph/ceph-client.admin.150126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130929.log 2023-10-18T12:02:56.684 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.140660.log.gz 2023-10-18T12:02:56.685 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153309.log 2023-10-18T12:02:56.685 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.150126.log.gz 2023-10-18T12:02:56.685 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111640.log: /var/log/ceph/ceph-client.admin.130929.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111640.log.gz 2023-10-18T12:02:56.686 INFO:teuthology.orchestra.run.smithi125.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.130929.log.gz 2023-10-18T12:02:56.686 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154672.log 2023-10-18T12:02:56.686 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141701.log 2023-10-18T12:02:56.686 INFO:teuthology.orchestra.run.smithi125.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.153309.log.gz 2023-10-18T12:02:56.687 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154840.log 2023-10-18T12:02:56.687 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%/var/log/ceph/ceph-client.admin.141701.log: -- replaced with /var/log/ceph/ceph-client.admin.154672.log.gz 2023-10-18T12:02:56.687 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141701.log.gz 2023-10-18T12:02:56.687 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-client.admin.124485.log 2023-10-18T12:02:56.688 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118194.log 2023-10-18T12:02:56.688 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.154840.log.gz 2023-10-18T12:02:56.688 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142856.log 2023-10-18T12:02:56.689 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124485.log.gz 2023-10-18T12:02:56.689 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160846.log 2023-10-18T12:02:56.689 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118194.log.gz 2023-10-18T12:02:56.689 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145014.log 2023-10-18T12:02:56.690 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.142856.log.gz 2023-10-18T12:02:56.690 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166867.log 2023-10-18T12:02:56.690 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160846.log.gz 2023-10-18T12:02:56.690 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141922.log 2023-10-18T12:02:56.691 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.145014.log.gz 2023-10-18T12:02:56.691 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162118.log 2023-10-18T12:02:56.691 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125003.log 2023-10-18T12:02:56.692 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.166867.log.gz 2023-10-18T12:02:56.692 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108433.log 2023-10-18T12:02:56.692 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.141922.log.gz 2023-10-18T12:02:56.692 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154238.log 2023-10-18T12:02:56.693 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.162118.log.gz 2023-10-18T12:02:56.693 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125003.log: /var/log/ceph/ceph-client.admin.108433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149006.log 2023-10-18T12:02:56.693 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108433.log.gz 2023-10-18T12:02:56.693 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.125003.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112140.log 2023-10-18T12:02:56.694 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.694 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154238.log: /var/log/ceph/ceph-client.admin.149006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146753.log 2023-10-18T12:02:56.694 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154238.log.gz 2023-10-18T12:02:56.695 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125213.log 2023-10-18T12:02:56.695 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.149006.log.gz 2023-10-18T12:02:56.695 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.112140.log.gz 2023-10-18T12:02:56.695 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148754.log 2023-10-18T12:02:56.696 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.146753.log.gz 2023-10-18T12:02:56.696 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113523.log 2023-10-18T12:02:56.696 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4%/var/log/ceph/ceph-client.admin.148754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137340.log 2023-10-18T12:02:56.697 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.125213.log.gz 2023-10-18T12:02:56.697 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118838.log 2023-10-18T12:02:56.697 INFO:teuthology.orchestra.run.smithi125.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.148754.log.gz 2023-10-18T12:02:56.697 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126933.log 2023-10-18T12:02:56.698 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113523.log.gz 2023-10-18T12:02:56.698 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.137340.log: /var/log/ceph/ceph-client.admin.118838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134555.log 2023-10-18T12:02:56.698 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126933.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.137340.log.gz 2023-10-18T12:02:56.699 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118838.log.gz 2023-10-18T12:02:56.699 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114580.log 2023-10-18T12:02:56.699 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97826.log 2023-10-18T12:02:56.699 INFO:teuthology.orchestra.run.smithi125.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.126933.log.gz 2023-10-18T12:02:56.700 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.114580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142477.log 2023-10-18T12:02:56.700 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134555.log.gz 2023-10-18T12:02:56.700 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114580.log.gz 2023-10-18T12:02:56.700 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101917.log 2023-10-18T12:02:56.701 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97826.log: /var/log/ceph/ceph-client.admin.142477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141388.log 2023-10-18T12:02:56.701 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4%/var/log/ceph/ceph-client.admin.101917.log: gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.142477.log.gz 2023-10-18T12:02:56.701 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.87672.log 2023-10-18T12:02:56.702 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97826.log.gz 2023-10-18T12:02:56.702 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101917.log.gz 2023-10-18T12:02:56.702 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141388.log: gzip -5 --verbose 87.1% -- replaced with /var/log/ceph/ceph-client.admin.141388.log.gz 2023-10-18T12:02:56.702 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.153771.log 2023-10-18T12:02:56.703 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87672.log: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.87672.log.gz 2023-10-18T12:02:56.703 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142771.log 2023-10-18T12:02:56.703 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118408.log 2023-10-18T12:02:56.703 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153771.log.gz/var/log/ceph/ceph-client.admin.142771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167221.log 2023-10-18T12:02:56.704 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.704 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9%/var/log/ceph/ceph-client.admin.118408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144762.log 2023-10-18T12:02:56.704 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.142771.log.gz 2023-10-18T12:02:56.705 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167221.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.118408.log.gz 2023-10-18T12:02:56.705 INFO:teuthology.orchestra.run.smithi125.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.135988.log 2023-10-18T12:02:56.705 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.167221.log.gz 2023-10-18T12:02:56.705 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.144762.log: -- /var/log/ceph/ceph-client.admin.129193.log 2023-10-18T12:02:56.706 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135988.log: gzip -5 87.1% -- replaced with /var/log/ceph/ceph-client.admin.144762.log.gz 2023-10-18T12:02:56.706 INFO:teuthology.orchestra.run.smithi125.stderr: --verbose -- /var/log/ceph/ceph-client.admin.125399.log 2023-10-18T12:02:56.706 INFO:teuthology.orchestra.run.smithi125.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135988.log.gz 2023-10-18T12:02:56.706 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.129193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124697.log 2023-10-18T12:02:56.707 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.129193.log.gz 2023-10-18T12:02:56.707 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144031.log 2023-10-18T12:02:56.707 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124697.log: gzip -5 --verbose 91.1% -- replaced with /var/log/ceph/ceph-client.admin.125399.log.gz 2023-10-18T12:02:56.708 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.122190.log 2023-10-18T12:02:56.708 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124697.log.gz 2023-10-18T12:02:56.708 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162370.log 87.0% -- replaced with /var/log/ceph/ceph-client.admin.144031.log.gz 2023-10-18T12:02:56.708 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.709 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122190.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122190.log.gz 2023-10-18T12:02:56.709 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.124060.log 2023-10-18T12:02:56.709 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142126.log 2023-10-18T12:02:56.709 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.162370.log.gz 2023-10-18T12:02:56.710 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125045.log 2023-10-18T12:02:56.710 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124060.log.gz/var/log/ceph/ceph-client.admin.142126.log: 2023-10-18T12:02:56.710 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164250.log 87.4% 2023-10-18T12:02:56.711 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.142126.log.gz 2023-10-18T12:02:56.711 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108492.log 2023-10-18T12:02:56.711 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.125045.log.gz 2023-10-18T12:02:56.711 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132521.log 2023-10-18T12:02:56.712 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108492.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164250.log.gz 2023-10-18T12:02:56.712 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121504.log 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108492.log.gz 2023-10-18T12:02:56.712 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.712 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132047.log 87.7% 2023-10-18T12:02:56.713 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.132521.log.gz 2023-10-18T12:02:56.713 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162454.log 2023-10-18T12:02:56.713 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121504.log.gz 2023-10-18T12:02:56.714 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103435.log 2023-10-18T12:02:56.714 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4%/var/log/ceph/ceph-client.admin.162454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128375.log 2023-10-18T12:02:56.714 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.132047.log.gz 2023-10-18T12:02:56.714 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.162454.log.gz 2023-10-18T12:02:56.715 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103873.log 2023-10-18T12:02:56.715 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99036.log 2023-10-18T12:02:56.715 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103435.log.gz 2023-10-18T12:02:56.715 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128375.log: /var/log/ceph/ceph-client.admin.103873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160760.log 2023-10-18T12:02:56.716 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9%/var/log/ceph/ceph-client.admin.99036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163249.log 2023-10-18T12:02:56.716 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103873.log.gz 2023-10-18T12:02:56.716 INFO:teuthology.orchestra.run.smithi125.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.128375.log.gz 2023-10-18T12:02:56.717 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7%/var/log/ceph/ceph-client.admin.160760.log: -- replaced with /var/log/ceph/ceph-client.admin.99036.log.gz 2023-10-18T12:02:56.717 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108846.log 2023-10-18T12:02:56.717 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160760.log.gz 2023-10-18T12:02:56.717 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166105.log 2023-10-18T12:02:56.718 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163249.log: gzip/var/log/ceph/ceph-client.admin.108846.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.112789.log 2023-10-18T12:02:56.718 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166105.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.108846.log.gz 2023-10-18T12:02:56.718 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145567.log 2023-10-18T12:02:56.719 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.163249.log.gz 2023-10-18T12:02:56.719 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.166105.log.gz 2023-10-18T12:02:56.719 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.112789.log: /var/log/ceph/ceph-client.admin.122714.log 2023-10-18T12:02:56.719 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145567.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112789.log.gz 2023-10-18T12:02:56.720 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose 87.0% -- replaced with /var/log/ceph/ceph-client.admin.145567.log.gz 2023-10-18T12:02:56.720 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.126670.log 2023-10-18T12:02:56.720 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.122714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106338.log 2023-10-18T12:02:56.720 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122714.log.gz 2023-10-18T12:02:56.721 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123976.log 2023-10-18T12:02:56.721 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.126670.log.gz 2023-10-18T12:02:56.721 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106338.log: gzip -5 --verbose 87.0% -- /var/log/ceph/ceph-client.admin.158076.log 2023-10-18T12:02:56.722 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106338.log.gz 2023-10-18T12:02:56.722 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114377.log 2023-10-18T12:02:56.722 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123976.log.gz 2023-10-18T12:02:56.722 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149870.log 2023-10-18T12:02:56.723 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.114377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121684.log 2023-10-18T12:02:56.723 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.158076.log.gz 2023-10-18T12:02:56.723 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114377.log.gz 2023-10-18T12:02:56.723 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132091.log 2023-10-18T12:02:56.724 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149870.log: /var/log/ceph/ceph-client.admin.121684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102043.log 2023-10-18T12:02:56.724 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.149870.log.gz 2023-10-18T12:02:56.724 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132091.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121684.log.gz 2023-10-18T12:02:56.725 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100504.log 2023-10-18T12:02:56.725 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.132091.log.gz 2023-10-18T12:02:56.725 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134776.log 2023-10-18T12:02:56.725 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102043.log.gz 2023-10-18T12:02:56.726 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137170.log 2023-10-18T12:02:56.726 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100504.log.gz 2023-10-18T12:02:56.726 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146229.log 2023-10-18T12:02:56.726 INFO:teuthology.orchestra.run.smithi125.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.134776.log.gz 2023-10-18T12:02:56.727 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.137170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130453.log 2023-10-18T12:02:56.727 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.137170.log.gz 2023-10-18T12:02:56.727 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113691.log 2023-10-18T12:02:56.728 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.146229.log.gz 2023-10-18T12:02:56.728 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107878.log 2023-10-18T12:02:56.728 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.130453.log.gz 2023-10-18T12:02:56.728 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107353.log 2023-10-18T12:02:56.729 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113691.log.gz 2023-10-18T12:02:56.729 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147843.log 2023-10-18T12:02:56.729 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.107878.log.gz 2023-10-18T12:02:56.729 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.107353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88150.log 2023-10-18T12:02:56.729 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.107353.log.gz 2023-10-18T12:02:56.730 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.147843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153351.log 2023-10-18T12:02:56.730 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.147843.log.gz 2023-10-18T12:02:56.730 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.88150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98742.log 87.0% -- replaced with /var/log/ceph/ceph-client.admin.88150.log.gz 2023-10-18T12:02:56.731 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.731 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126517.log 85.8% 2023-10-18T12:02:56.731 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.153351.log.gz 2023-10-18T12:02:56.731 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116812.log 2023-10-18T12:02:56.732 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98742.log.gz 2023-10-18T12:02:56.732 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117456.log 2023-10-18T12:02:56.732 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.126517.log.gz 2023-10-18T12:02:56.733 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.116812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136410.log 2023-10-18T12:02:56.733 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1%/var/log/ceph/ceph-client.admin.117456.log: -- replaced with /var/log/ceph/ceph-client.admin.116812.log.gz 2023-10-18T12:02:56.733 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155592.log 2023-10-18T12:02:56.733 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.117456.log.gz 2023-10-18T12:02:56.734 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157865.log 2023-10-18T12:02:56.734 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.136410.log.gz 2023-10-18T12:02:56.734 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99206.log 2023-10-18T12:02:56.734 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.155592.log.gz 2023-10-18T12:02:56.735 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130845.log 2023-10-18T12:02:56.735 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157865.log.gz 2023-10-18T12:02:56.735 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.99206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105286.log 2023-10-18T12:02:56.735 INFO:teuthology.orchestra.run.smithi125.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99206.log.gz 2023-10-18T12:02:56.736 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130845.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-18T12:02:56.736 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.130845.log.gz 2023-10-18T12:02:56.736 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162894.log 2023-10-18T12:02:56.737 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105286.log.gz 2023-10-18T12:02:56.737 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103831.log 2023-10-18T12:02:56.737 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150504.log 2023-10-18T12:02:56.737 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.162894.log.gz 2023-10-18T12:02:56.738 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107395.log 2023-10-18T12:02:56.738 INFO:teuthology.orchestra.run.smithi125.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103831.log.gz 2023-10-18T12:02:56.738 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87984.log 2023-10-18T12:02:56.739 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141041.log 2023-10-18T12:02:56.739 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5%/var/log/ceph/ceph-client.admin.107395.log: -- replaced with /var/log/ceph/ceph-client.admin.150504.log.gz 2023-10-18T12:02:56.739 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135215.log 2023-10-18T12:02:56.739 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107395.log.gz 85.6% -- replaced with /var/log/ceph/ceph-client.admin.87984.log.gz 2023-10-18T12:02:56.740 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.740 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141041.log: gzip -5 --verbose -- 87.1% -- replaced with /var/log/ceph/ceph-client.admin.141041.log.gz 2023-10-18T12:02:56.740 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.131961.log 2023-10-18T12:02:56.740 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135215.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135215.log.gz 2023-10-18T12:02:56.741 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.134640.log 2023-10-18T12:02:56.741 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160804.log 2023-10-18T12:02:56.741 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.131961.log.gz 2023-10-18T12:02:56.741 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105370.log 2023-10-18T12:02:56.742 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134640.log.gz 2023-10-18T12:02:56.742 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160804.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.160804.log.gz 2023-10-18T12:02:56.742 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.167011.log 2023-10-18T12:02:56.743 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125881.log 2023-10-18T12:02:56.743 INFO:teuthology.orchestra.run.smithi125.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105370.log.gz 2023-10-18T12:02:56.743 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128187.log 2023-10-18T12:02:56.743 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.167011.log.gz/var/log/ceph/ceph-client.admin.125881.log: 2023-10-18T12:02:56.744 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109940.log 2023-10-18T12:02:56.744 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.125881.log.gz 2023-10-18T12:02:56.744 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113481.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128187.log.gz 2023-10-18T12:02:56.744 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.745 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.109940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96936.log 2023-10-18T12:02:56.745 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2%/var/log/ceph/ceph-client.admin.113481.log: -- replaced with /var/log/ceph/ceph-client.admin.109940.log.gz 2023-10-18T12:02:56.745 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140786.log 2023-10-18T12:02:56.746 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.113481.log.gz 2023-10-18T12:02:56.746 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167389.log 2023-10-18T12:02:56.746 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96936.log: /var/log/ceph/ceph-client.admin.140786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144972.log 2023-10-18T12:02:56.746 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.167389.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96936.log.gz 2023-10-18T12:02:56.747 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140786.log.gz 2023-10-18T12:02:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157179.log 2023-10-18T12:02:56.747 INFO:teuthology.orchestra.run.smithi125.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.167389.log.gz 2023-10-18T12:02:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144115.log 2023-10-18T12:02:56.748 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144972.log: /var/log/ceph/ceph-client.admin.157179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108004.log 2023-10-18T12:02:56.748 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.144972.log.gz 2023-10-18T12:02:56.748 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144115.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.157179.log.gz 2023-10-18T12:02:56.749 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.144115.log.gz 2023-10-18T12:02:56.749 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158304.log 2023-10-18T12:02:56.749 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102543.log 2023-10-18T12:02:56.749 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108004.log: /var/log/ceph/ceph-client.admin.158304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129344.log 2023-10-18T12:02:56.750 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102543.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108004.log.gz 2023-10-18T12:02:56.750 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158304.log.gz 2023-10-18T12:02:56.750 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131494.log 2023-10-18T12:02:56.750 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102543.log.gz 2023-10-18T12:02:56.751 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133570.log 2023-10-18T12:02:56.751 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.129344.log: /var/log/ceph/ceph-client.admin.131494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142084.log 2023-10-18T12:02:56.751 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133570.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.129344.log.gz 2023-10-18T12:02:56.751 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131494.log.gz 2023-10-18T12:02:56.752 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.133570.log.gz 2023-10-18T12:02:56.752 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127456.log 2023-10-18T12:02:56.752 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135723.log 2023-10-18T12:02:56.753 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142084.log.gz 2023-10-18T12:02:56.753 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108256.log 2023-10-18T12:02:56.753 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127456.log.gz 2023-10-18T12:02:56.753 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121854.log 2023-10-18T12:02:56.754 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136512.log 2023-10-18T12:02:56.754 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108256.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135723.log.gz 2023-10-18T12:02:56.754 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121854.log: 86.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111682.log 2023-10-18T12:02:56.754 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108256.log.gz 2023-10-18T12:02:56.755 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136512.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125923.log 2023-10-18T12:02:56.755 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.136512.log.gz 2023-10-18T12:02:56.755 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.121854.log.gz 2023-10-18T12:02:56.755 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111682.log: gzip -5 87.6% --verbose -- /var/log/ceph/ceph-client.admin.97822.log 2023-10-18T12:02:56.756 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.111682.log.gz 2023-10-18T12:02:56.756 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.125923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121726.log 2023-10-18T12:02:56.756 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125923.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143786.log 2023-10-18T12:02:56.757 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.757 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97822.log: /var/log/ceph/ceph-client.admin.121726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160626.log 2023-10-18T12:02:56.757 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7%/var/log/ceph/ceph-client.admin.143786.log: -- replaced with /var/log/ceph/ceph-client.admin.97822.log.gz 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121726.log.gz 2023-10-18T12:02:56.757 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.758 INFO:teuthology.orchestra.run.smithi125.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.143786.log.gz 2023-10-18T12:02:56.758 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159160.log 2023-10-18T12:02:56.758 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101386.log 2023-10-18T12:02:56.758 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161785.log 2023-10-18T12:02:56.759 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160626.log.gz/var/log/ceph/ceph-client.admin.159160.log: 2023-10-18T12:02:56.759 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101386.log: 87.5% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101386.log.gz 2023-10-18T12:02:56.759 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161098.log 2023-10-18T12:02:56.759 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.159160.log.gz 2023-10-18T12:02:56.760 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98161.log 2023-10-18T12:02:56.760 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.161785.log.gz 2023-10-18T12:02:56.760 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95539.log 86.8% -- replaced with /var/log/ceph/ceph-client.admin.161098.log.gz 2023-10-18T12:02:56.760 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.761 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117246.log 87.6% 2023-10-18T12:02:56.761 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98161.log.gz 2023-10-18T12:02:56.761 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127715.log 2023-10-18T12:02:56.762 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95539.log.gz 2023-10-18T12:02:56.762 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100798.log 2023-10-18T12:02:56.762 INFO:teuthology.orchestra.run.smithi125.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.117246.log.gz 2023-10-18T12:02:56.762 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120785.log 2023-10-18T12:02:56.763 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127715.log.gz 2023-10-18T12:02:56.763 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154882.log 2023-10-18T12:02:56.763 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164548.log 2023-10-18T12:02:56.763 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100798.log.gz/var/log/ceph/ceph-client.admin.120785.log: 2023-10-18T12:02:56.764 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154882.log: 87.4%gzip -5 --verbose -- 86.8% -- replaced with /var/log/ceph/ceph-client.admin.154882.log.gz 2023-10-18T12:02:56.764 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.135131.log 2023-10-18T12:02:56.765 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120785.log.gz 2023-10-18T12:02:56.765 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164548.log: gzip -5 --verbose -- 86.9% -- replaced with /var/log/ceph/ceph-client.admin.164548.log.gz 2023-10-18T12:02:56.765 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.148617.log 2023-10-18T12:02:56.765 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158388.log 87.1% -- replaced with /var/log/ceph/ceph-client.admin.135131.log.gz 2023-10-18T12:02:56.766 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.766 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148617.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T12:02:56.766 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.148617.log.gz 2023-10-18T12:02:56.767 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159720.log 86.6% -- replaced with /var/log/ceph/ceph-client.admin.158388.log.gz 2023-10-18T12:02:56.767 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.767 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160496.log 2023-10-18T12:02:56.767 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120869.log 2023-10-18T12:02:56.769 INFO:teuthology.orchestra.run.smithi125.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.159720.log.gz 2023-10-18T12:02:56.769 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160496.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.160496.log.gz 2023-10-18T12:02:56.770 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166951.log 2023-10-18T12:02:56.770 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124919.log 2023-10-18T12:02:56.770 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120869.log: /var/log/ceph/ceph-client.admin.166951.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.120869.log.gz 2023-10-18T12:02:56.771 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117601.log 2023-10-18T12:02:56.771 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.166951.log.gz 2023-10-18T12:02:56.771 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162852.log 2023-10-18T12:02:56.771 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117601.log: 87.2% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117601.log.gz 2023-10-18T12:02:56.772 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124919.log.gz 2023-10-18T12:02:56.772 INFO:teuthology.orchestra.run.smithi125.stderr: 89.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T12:02:56.772 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158346.log 2023-10-18T12:02:56.772 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154322.log 2023-10-18T12:02:56.773 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162852.log.gz 2023-10-18T12:02:56.773 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140702.log 2023-10-18T12:02:56.773 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158346.log.gz 2023-10-18T12:02:56.774 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150000.log 2023-10-18T12:02:56.774 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124228.log 2023-10-18T12:02:56.774 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8%/var/log/ceph/ceph-client.admin.140702.log: -- replaced with /var/log/ceph/ceph-client.admin.154322.log.gz 2023-10-18T12:02:56.774 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150000.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.140702.log.gzgzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.150000.log.gz 2023-10-18T12:02:56.775 INFO:teuthology.orchestra.run.smithi125.stderr: /var/log/ceph/ceph-client.admin.126391.log 2023-10-18T12:02:56.775 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.775 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124228.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124228.log.gz 2023-10-18T12:02:56.775 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.126754.log 2023-10-18T12:02:56.776 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126391.log: gzip -5 --verbose 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126391.log.gz 2023-10-18T12:02:56.776 INFO:teuthology.orchestra.run.smithi125.stderr: -- /var/log/ceph/ceph-client.admin.111961.log 2023-10-18T12:02:56.776 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104167.log 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126754.log.gz 2023-10-18T12:02:56.777 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.777 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111961.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-18T12:02:56.778 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111961.log.gz 2023-10-18T12:02:56.778 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108551.log 2023-10-18T12:02:56.778 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104167.log.gz 2023-10-18T12:02:56.779 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120953.log 2023-10-18T12:02:56.779 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162496.log 2023-10-18T12:02:56.779 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108551.log.gz 2023-10-18T12:02:56.780 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143989.log 2023-10-18T12:02:56.780 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162496.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120953.log.gz 2023-10-18T12:02:56.780 INFO:teuthology.orchestra.run.smithi125.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162496.log.gz 2023-10-18T12:02:56.781 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97188.log 2023-10-18T12:02:56.781 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148464.log 2023-10-18T12:02:56.781 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143989.log: /var/log/ceph/ceph-client.admin.97188.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.143989.log.gz 2023-10-18T12:02:56.781 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97188.log.gz 2023-10-18T12:02:56.782 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158118.log 2023-10-18T12:02:56.782 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148464.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.148464.log.gz 2023-10-18T12:02:56.782 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131919.log 2023-10-18T12:02:56.782 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-18T12:02:56.783 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158118.log: /var/log/ceph/ceph-client.admin.131919.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158118.log.gz 2023-10-18T12:02:56.783 INFO:teuthology.orchestra.run.smithi125.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131919.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163146.log 2023-10-18T12:02:56.783 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.783 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101607.log 2023-10-18T12:02:56.784 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.163146.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.163146.log.gz 2023-10-18T12:02:56.784 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120514.log 2023-10-18T12:02:56.784 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101607.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101607.log.gz 2023-10-18T12:02:56.785 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104852.log 2023-10-18T12:02:56.785 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120514.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120514.log.gz 2023-10-18T12:02:56.785 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134276.log 2023-10-18T12:02:56.785 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104852.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104852.log.gz 2023-10-18T12:02:56.786 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132141.log 2023-10-18T12:02:56.786 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134276.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.134276.log.gz 2023-10-18T12:02:56.786 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114299.log 2023-10-18T12:02:56.787 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.132141.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.132141.log.gz 2023-10-18T12:02:56.787 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147477.log 2023-10-18T12:02:56.787 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.114299.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114299.log.gz 2023-10-18T12:02:56.788 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106073.log 2023-10-18T12:02:56.788 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.147477.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.147477.log.gz 2023-10-18T12:02:56.788 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149540.log 2023-10-18T12:02:56.788 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.106073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159762.log 2023-10-18T12:02:56.789 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149540.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.149540.log.gz 2023-10-18T12:02:56.789 INFO:teuthology.orchestra.run.smithi125.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-18T12:02:56.789 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88139.log 2023-10-18T12:02:56.790 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159762.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.159762.log.gz 2023-10-18T12:02:56.790 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119114.log 2023-10-18T12:02:56.790 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.88139.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.88139.log.gz 2023-10-18T12:02:56.790 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96110.log 2023-10-18T12:02:56.791 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119114.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119114.log.gz 2023-10-18T12:02:56.791 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156927.log 2023-10-18T12:02:56.791 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96110.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.96110.log.gz 2023-10-18T12:02:56.791 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156342.log 2023-10-18T12:02:56.792 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156927.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.156927.log.gz 2023-10-18T12:02:56.792 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149150.log 2023-10-18T12:02:56.792 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156342.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.156342.log.gz 2023-10-18T12:02:56.793 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160370.log 2023-10-18T12:02:56.793 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149150.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149150.log.gz 2023-10-18T12:02:56.793 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88398.log 2023-10-18T12:02:56.793 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160370.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160370.log.gz 2023-10-18T12:02:56.794 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156823.log 2023-10-18T12:02:56.794 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.88398.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.88398.log.gz 2023-10-18T12:02:56.794 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117685.log 2023-10-18T12:02:56.795 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156823.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156823.log.gz 2023-10-18T12:02:56.795 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134902.log 2023-10-18T12:02:56.795 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117685.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117685.log.gz 2023-10-18T12:02:56.795 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161320.log 2023-10-18T12:02:56.796 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.134902.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134902.log.gz 2023-10-18T12:02:56.796 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97230.log 2023-10-18T12:02:56.796 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161320.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.161320.log.gz 2023-10-18T12:02:56.797 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112182.log 2023-10-18T12:02:56.797 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97230.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97230.log.gz 2023-10-18T12:02:56.797 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141743.log 2023-10-18T12:02:56.797 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112182.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.112182.log.gz 2023-10-18T12:02:56.798 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150378.log 2023-10-18T12:02:56.798 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141743.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.141743.log.gz 2023-10-18T12:02:56.798 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159076.log 2023-10-18T12:02:56.799 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.150378.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150378.log.gz 2023-10-18T12:02:56.799 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161953.log 2023-10-18T12:02:56.799 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159076.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159076.log.gz 2023-10-18T12:02:56.799 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103747.log 2023-10-18T12:02:56.800 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161953.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.161953.log.gz 2023-10-18T12:02:56.800 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106380.log 2023-10-18T12:02:56.800 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.103747.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103747.log.gz 2023-10-18T12:02:56.801 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145821.log 2023-10-18T12:02:56.801 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106380.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.106380.log.gz 2023-10-18T12:02:56.801 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163206.log 2023-10-18T12:02:56.801 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145821.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145821.log.gz 2023-10-18T12:02:56.802 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163543.log 2023-10-18T12:02:56.802 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163206.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163206.log.gz 2023-10-18T12:02:56.802 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112224.log 2023-10-18T12:02:56.803 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163543.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163543.log.gz 2023-10-18T12:02:56.803 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128231.log 2023-10-18T12:02:56.803 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112224.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112224.log.gz 2023-10-18T12:02:56.803 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155890.log 2023-10-18T12:02:56.804 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128231.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128231.log.gz 2023-10-18T12:02:56.804 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161598.log 2023-10-18T12:02:56.804 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155890.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155890.log.gz 2023-10-18T12:02:56.805 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161362.log 2023-10-18T12:02:56.805 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161598.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.161598.log.gz 2023-10-18T12:02:56.805 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111808.log 2023-10-18T12:02:56.805 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.161362.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.161362.log.gz 2023-10-18T12:02:56.806 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121980.log 2023-10-18T12:02:56.806 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111808.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111808.log.gz 2023-10-18T12:02:56.806 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130887.log 2023-10-18T12:02:56.806 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121980.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121980.log.gz 2023-10-18T12:02:56.807 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120056.log 2023-10-18T12:02:56.807 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.130887.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.130887.log.gz 2023-10-18T12:02:56.807 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105580.log 2023-10-18T12:02:56.808 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120056.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.120056.log.gz 2023-10-18T12:02:56.808 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104396.log 2023-10-18T12:02:56.808 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105580.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.105580.log.gz 2023-10-18T12:02:56.808 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94874.log 2023-10-18T12:02:56.809 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.104396.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104396.log.gz 2023-10-18T12:02:56.809 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123079.log 2023-10-18T12:02:56.809 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.94874.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.94874.log.gz 2023-10-18T12:02:56.810 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119006.log 2023-10-18T12:02:56.810 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123079.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123079.log.gz 2023-10-18T12:02:56.810 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102331.log 2023-10-18T12:02:56.810 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119006.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119006.log.gz 2023-10-18T12:02:56.811 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100546.log 2023-10-18T12:02:56.811 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102331.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102331.log.gz 2023-10-18T12:02:56.811 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153561.log 2023-10-18T12:02:56.811 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100546.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100546.log.gz 2023-10-18T12:02:56.812 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144491.log 2023-10-18T12:02:56.812 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153561.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.153561.log.gz 2023-10-18T12:02:56.812 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146903.log 2023-10-18T12:02:56.812 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144491.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.144491.log.gz 2023-10-18T12:02:56.813 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135681.log 2023-10-18T12:02:56.814 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mon.c.log: 93.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-18T12:02:56.868 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146903.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.146903.log.gz 2023-10-18T12:02:56.869 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87588.log 2023-10-18T12:02:56.869 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135681.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135681.log.gz 2023-10-18T12:02:56.869 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158034.log 2023-10-18T12:02:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87588.log: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.87588.log.gz 2023-10-18T12:02:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108130.log 2023-10-18T12:02:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.158034.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.158034.log.gz 2023-10-18T12:02:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111536.log 2023-10-18T12:02:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.108130.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.108130.log.gz 2023-10-18T12:02:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151112.log 2023-10-18T12:02:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.111536.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111536.log.gz 2023-10-18T12:02:56.872 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136200.log 2023-10-18T12:02:56.872 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.151112.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151112.log.gz 2023-10-18T12:02:56.872 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.119373.log 2023-10-18T12:02:56.872 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136200.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.136200.log.gz 2023-10-18T12:02:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120251.log 2023-10-18T12:02:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.119373.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.119373.log.gz 2023-10-18T12:02:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118542.log 2023-10-18T12:02:56.874 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120251.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120251.log.gz 2023-10-18T12:02:56.874 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105064.log 2023-10-18T12:02:56.874 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118542.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118542.log.gz 2023-10-18T12:02:56.874 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162683.log 2023-10-18T12:02:56.875 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.105064.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105064.log.gz 2023-10-18T12:02:56.875 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153477.log 2023-10-18T12:02:56.875 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.162683.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.162683.log.gz 2023-10-18T12:02:56.876 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112621.log 2023-10-18T12:02:56.876 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153477.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153477.log.gz 2023-10-18T12:02:56.876 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102711.log 2023-10-18T12:02:56.876 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112621.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112621.log.gz 2023-10-18T12:02:56.877 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145098.log 2023-10-18T12:02:56.877 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102711.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102711.log.gz 2023-10-18T12:02:56.877 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95497.log 2023-10-18T12:02:56.877 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145098.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145098.log.gz 2023-10-18T12:02:56.878 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96690.log 2023-10-18T12:02:56.878 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95497.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95497.log.gz 2023-10-18T12:02:56.878 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133629.log 2023-10-18T12:02:56.879 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.96690.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96690.log.gz 2023-10-18T12:02:56.879 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117414.log 2023-10-18T12:02:56.879 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133629.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.133629.log.gz 2023-10-18T12:02:56.879 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160930.log 2023-10-18T12:02:56.880 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117414.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117414.log.gz 2023-10-18T12:02:56.880 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155974.log 2023-10-18T12:02:56.880 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160930.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.160930.log.gz 2023-10-18T12:02:56.881 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119048.log 2023-10-18T12:02:56.881 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155974.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.155974.log.gz 2023-10-18T12:02:56.881 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126807.log 2023-10-18T12:02:56.881 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119048.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.119048.log.gz 2023-10-18T12:02:56.882 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149360.log 2023-10-18T12:02:56.882 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126807.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126807.log.gz 2023-10-18T12:02:56.882 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94573.log 2023-10-18T12:02:56.883 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.149360.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.149360.log.gz 2023-10-18T12:02:56.883 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117049.log 2023-10-18T12:02:56.883 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.94573.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94573.log.gz 2023-10-18T12:02:56.883 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146073.log 2023-10-18T12:02:56.884 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117049.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117049.log.gz 2023-10-18T12:02:56.884 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152851.log 2023-10-18T12:02:56.884 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.146073.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.146073.log.gz 2023-10-18T12:02:56.884 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142982.log 2023-10-18T12:02:56.885 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.152851.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.152851.log.gz 2023-10-18T12:02:56.885 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95413.log 2023-10-18T12:02:56.885 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142982.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.142982.log.gz 2023-10-18T12:02:56.885 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113565.log 2023-10-18T12:02:56.886 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.95413.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95413.log.gz 2023-10-18T12:02:56.886 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135520.log 2023-10-18T12:02:56.886 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113565.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113565.log.gz 2023-10-18T12:02:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166615.log 2023-10-18T12:02:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135520.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135520.log.gz 2023-10-18T12:02:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101512.log 2023-10-18T12:02:56.887 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166615.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.166615.log.gz 2023-10-18T12:02:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102636.log 2023-10-18T12:02:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101512.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101512.log.gz 2023-10-18T12:02:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164590.log 2023-10-18T12:02:56.888 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102636.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102636.log.gz 2023-10-18T12:02:56.889 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118500.log 2023-10-18T12:02:56.889 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164590.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.164590.log.gz 2023-10-18T12:02:56.889 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155848.log 2023-10-18T12:02:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.118500.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118500.log.gz 2023-10-18T12:02:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164294.log 2023-10-18T12:02:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.155848.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155848.log.gz 2023-10-18T12:02:56.890 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102415.log 2023-10-18T12:02:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164294.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.164294.log.gz 2023-10-18T12:02:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135765.log 2023-10-18T12:02:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102415.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102415.log.gz 2023-10-18T12:02:56.891 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136326.log 2023-10-18T12:02:56.892 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.135765.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135765.log.gz 2023-10-18T12:02:56.892 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119740.log 2023-10-18T12:02:56.892 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.136326.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.136326.log.gz 2023-10-18T12:02:56.893 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144575.log 2023-10-18T12:02:56.893 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119740.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119740.log.gz 2023-10-18T12:02:56.893 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99610.log 2023-10-18T12:02:56.893 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.144575.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144575.log.gz 2023-10-18T12:02:56.894 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119930.log 2023-10-18T12:02:56.894 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.0.99610.log: 90.1% -- replaced with /var/log/ceph/ceph-client.0.99610.log.gz 2023-10-18T12:02:56.894 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157427.log 2023-10-18T12:02:56.895 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.119930.log: 93.9% -- replaced with /var/log/ceph/ceph-client.0.106073.log.gz 2023-10-18T12:02:56.895 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119930.log.gz 2023-10-18T12:02:56.895 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163840.log 2023-10-18T12:02:56.896 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.157427.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.157427.log.gz 2023-10-18T12:02:56.896 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127372.log 2023-10-18T12:02:56.896 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128669.log 2023-10-18T12:02:56.896 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.163840.log: /var/log/ceph/ceph-client.admin.127372.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163840.log.gz 2023-10-18T12:02:56.897 INFO:teuthology.orchestra.run.smithi125.stderr:gzip 87.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.141262.log 2023-10-18T12:02:56.897 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.127372.log.gz 2023-10-18T12:02:56.897 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.128669.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128669.log.gz 2023-10-18T12:02:56.897 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162809.log 2023-10-18T12:02:56.898 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129236.log 2023-10-18T12:02:56.898 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141262.log: /var/log/ceph/ceph-client.admin.162809.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.141262.log.gz 2023-10-18T12:02:56.898 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.162809.log.gz 2023-10-18T12:02:56.899 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97314.log 2023-10-18T12:02:56.899 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.129236.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.129236.log.gz 2023-10-18T12:02:56.899 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124569.log 2023-10-18T12:02:56.899 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154756.log 2023-10-18T12:02:56.900 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97314.log: /var/log/ceph/ceph-client.admin.124569.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.97314.log.gz 2023-10-18T12:02:56.900 INFO:teuthology.orchestra.run.smithi125.stderr: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.124569.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.98910.log 2023-10-18T12:02:56.900 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.900 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154756.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154756.log.gz 2023-10-18T12:02:56.901 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105664.log 2023-10-18T12:02:56.901 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112453.log 2023-10-18T12:02:56.901 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98910.log: /var/log/ceph/ceph-client.admin.105664.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98910.log.gz 2023-10-18T12:02:56.902 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.105664.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.148964.log 2023-10-18T12:02:56.902 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.902 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112453.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112453.log.gz 2023-10-18T12:02:56.902 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158994.log 2023-10-18T12:02:56.903 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141136.log 2023-10-18T12:02:56.903 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148964.log: /var/log/ceph/ceph-client.admin.158994.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148964.log.gz 2023-10-18T12:02:56.903 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.158994.log.gz 2023-10-18T12:02:56.903 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164116.log 2023-10-18T12:02:56.904 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.141136.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.141136.log.gz 2023-10-18T12:02:56.904 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97487.log 2023-10-18T12:02:56.904 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88702.log 2023-10-18T12:02:56.905 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.164116.log: /var/log/ceph/ceph-client.admin.97487.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.164116.log.gz 2023-10-18T12:02:56.905 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97487.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81468.log 2023-10-18T12:02:56.905 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.905 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123205.log 2023-10-18T12:02:56.906 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.88702.log: /var/log/ceph/ceph.tmp-client.admin.81468.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81468.log.gz 2023-10-18T12:02:56.906 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.88702.log.gz 2023-10-18T12:02:56.906 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160676.log 2023-10-18T12:02:56.906 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.123205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124741.log 2023-10-18T12:02:56.907 INFO:teuthology.orchestra.run.smithi125.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123205.log.gz 2023-10-18T12:02:56.907 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.160676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120700.log 2023-10-18T12:02:56.907 INFO:teuthology.orchestra.run.smithi125.stderr: 87.2%/var/log/ceph/ceph-client.admin.124741.log: -- replaced with /var/log/ceph/ceph-client.admin.160676.log.gz 2023-10-18T12:02:56.908 INFO:teuthology.orchestra.run.smithi125.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124741.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145355.log 2023-10-18T12:02:56.908 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:02:56.908 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140999.log 2023-10-18T12:02:56.908 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120700.log: /var/log/ceph/ceph-client.admin.145355.log: 87.6% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120700.log.gz 2023-10-18T12:02:56.909 INFO:teuthology.orchestra.run.smithi125.stderr: -- replaced with /var/log/ceph/ceph-client.admin.145355.log.gz 2023-10-18T12:02:56.909 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107437.log 2023-10-18T12:02:56.909 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.140999.log: -- /var/log/ceph/ceph-client.0.114100.log 2023-10-18T12:02:56.909 INFO:teuthology.orchestra.run.smithi125.stderr: 87.9%/var/log/ceph/ceph-client.admin.107437.log: -- replaced with /var/log/ceph/ceph-client.admin.140999.log.gz 2023-10-18T12:02:56.910 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120209.log 2023-10-18T12:02:56.910 INFO:teuthology.orchestra.run.smithi125.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107437.log.gz/var/log/ceph/ceph-client.0.114100.log: 2023-10-18T12:02:56.910 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87644.log 2023-10-18T12:02:56.911 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.120209.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120209.log.gz 2023-10-18T12:02:56.911 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148838.log 2023-10-18T12:02:56.911 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.87644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87644.log.gz 2023-10-18T12:02:56.911 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154196.log 2023-10-18T12:02:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.148838.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.148838.log.gz 2023-10-18T12:02:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101050.log 2023-10-18T12:02:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.154196.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154196.log.gz 2023-10-18T12:02:56.912 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-18T12:02:56.913 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101050.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101050.log.gz 2023-10-18T12:02:56.913 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117516.log 2023-10-18T12:02:56.913 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145056.log 2023-10-18T12:02:56.914 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.117516.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117516.log.gz 2023-10-18T12:02:56.914 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112747.log 2023-10-18T12:02:56.914 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.145056.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.145056.log.gz 2023-10-18T12:02:56.914 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143844.log 2023-10-18T12:02:56.915 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.112747.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.112747.log.gz 2023-10-18T12:02:56.915 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127498.log 2023-10-18T12:02:56.915 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143844.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143844.log.gz 2023-10-18T12:02:56.916 INFO:teuthology.orchestra.run.smithi125.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-18T12:02:56.916 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124875.log 2023-10-18T12:02:56.916 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127498.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127498.log.gz 2023-10-18T12:02:56.916 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106663.log 2023-10-18T12:02:56.917 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.124875.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124875.log.gz 2023-10-18T12:02:56.917 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159118.log 2023-10-18T12:02:56.917 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.106663.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.106663.log.gz 2023-10-18T12:02:56.918 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133888.log 2023-10-18T12:02:56.918 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159118.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.159118.log.gz 2023-10-18T12:02:56.918 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143691.log 2023-10-18T12:02:56.918 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.133888.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.133888.log.gz 2023-10-18T12:02:56.919 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126975.log 2023-10-18T12:02:56.919 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.143691.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143691.log.gz 2023-10-18T12:02:56.919 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127881.log 2023-10-18T12:02:56.919 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.126975.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.126975.log.gz 2023-10-18T12:02:56.920 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166699.log 2023-10-18T12:02:56.920 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.127881.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127881.log.gz 2023-10-18T12:02:56.920 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153130.log 2023-10-18T12:02:56.921 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166699.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166699.log.gz 2023-10-18T12:02:56.921 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113131.log 2023-10-18T12:02:56.921 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.153130.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.153130.log.gz 2023-10-18T12:02:56.921 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156058.log 2023-10-18T12:02:56.922 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.113131.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113131.log.gz 2023-10-18T12:02:56.922 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131013.log 2023-10-18T12:02:56.922 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.156058.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.156058.log.gz 2023-10-18T12:02:56.922 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159678.log 2023-10-18T12:02:56.923 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131013.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.131013.log.gz 2023-10-18T12:02:56.923 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98448.log 2023-10-18T12:02:56.923 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.159678.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.159678.log.gz 2023-10-18T12:02:56.923 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101302.log 2023-10-18T12:02:56.924 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.98448.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98448.log.gz 2023-10-18T12:02:56.924 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102500.log 2023-10-18T12:02:56.924 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.101302.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101302.log.gz 2023-10-18T12:02:56.925 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142898.log 2023-10-18T12:02:56.925 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.102500.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102500.log.gz 2023-10-18T12:02:56.925 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166825.log 2023-10-18T12:02:56.925 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.142898.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.142898.log.gz 2023-10-18T12:02:56.926 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121548.log 2023-10-18T12:02:56.926 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166825.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.166825.log.gz 2023-10-18T12:02:56.926 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166249.log 2023-10-18T12:02:56.926 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.121548.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.121548.log.gz 2023-10-18T12:02:56.927 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131452.log 2023-10-18T12:02:56.927 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.166249.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.166249.log.gz 2023-10-18T12:02:56.927 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100966.log 2023-10-18T12:02:56.928 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.131452.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.131452.log.gz 2023-10-18T12:02:56.928 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97496.log 2023-10-18T12:02:56.928 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.100966.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100966.log.gz 2023-10-18T12:02:56.928 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.97496.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97496.log.gz 2023-10-18T12:02:56.929 INFO:teuthology.orchestra.run.smithi125.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.114100.log.gz 2023-10-18T12:02:57.399 INFO:teuthology.orchestra.run.smithi139.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-18T12:02:57.562 INFO:teuthology.orchestra.run.smithi139.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-18T12:02:58.091 INFO:teuthology.orchestra.run.smithi139.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-18T12:02:58.285 INFO:teuthology.orchestra.run.smithi125.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-18T12:02:59.120 INFO:teuthology.orchestra.run.smithi125.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-18T12:03:00.263 INFO:teuthology.orchestra.run.smithi139.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-18T12:03:00.588 INFO:teuthology.orchestra.run.smithi139.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-18T12:03:00.904 INFO:teuthology.orchestra.run.smithi139.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-18T12:03:01.271 INFO:teuthology.orchestra.run.smithi125.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-18T12:03:01.419 INFO:teuthology.orchestra.run.smithi125.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-18T12:03:01.448 INFO:teuthology.orchestra.run.smithi125.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-18T12:03:01.950 INFO:teuthology.orchestra.run.smithi125.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-18T12:03:01.952 INFO:teuthology.orchestra.run.smithi125.stderr: 2023-10-18T12:03:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:real 0m6.176s 2023-10-18T12:03:01.952 INFO:teuthology.orchestra.run.smithi125.stderr:user 0m27.544s 2023-10-18T12:03:01.953 INFO:teuthology.orchestra.run.smithi125.stderr:sys 0m1.558s 2023-10-18T12:03:02.061 INFO:teuthology.orchestra.run.smithi139.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-18T12:03:02.063 INFO:teuthology.orchestra.run.smithi139.stderr: 2023-10-18T12:03:02.064 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m6.259s 2023-10-18T12:03:02.064 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m25.680s 2023-10-18T12:03:02.064 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m0.884s 2023-10-18T12:03:02.065 INFO:tasks.ceph:Archiving logs... 2023-10-18T12:03:02.065 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/remote/smithi125/log 2023-10-18T12:03:02.067 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T12:03:06.612 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/remote/smithi139/log 2023-10-18T12:03:06.613 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T12:03:10.120 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-18T12:03:10.129 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-18T12:03:10.129 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T12:03:10.132 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T12:03:10.230 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-18T12:03:10.239 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-18T12:03:10.240 DEBUG:teuthology.orchestra.run.smithi125:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T12:03:10.242 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T12:03:10.256 INFO:teuthology.orchestra.run.smithi125.stderr:bash: line 1: ntpq: command not found 2023-10-18T12:03:10.259 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2023-10-18T12:03:10.260 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T12:03:10.260 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2023-10-18T12:03:10.260 INFO:teuthology.orchestra.run.smithi125.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 +3044us[+3015us] +/- 67ms 2023-10-18T12:03:10.261 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 18 -1516us[-1544us] +/- 53ms 2023-10-18T12:03:10.261 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 -1372us[-1401us] +/- 66ms 2023-10-18T12:03:10.261 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:03:10.263 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T12:03:10.263 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2023-10-18T12:03:10.263 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 24 +3069us[+3074us] +/- 67ms 2023-10-18T12:03:10.264 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv02.front.sepia.ceph.com 3 8 377 25 -1482us[-1477us] +/- 53ms 2023-10-18T12:03:10.264 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -1344us[-1344us] +/- 66ms 2023-10-18T12:03:10.264 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T12:03:10.265 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-18T12:03:10.274 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-18T12:03:10.275 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-18T12:03:10.283 DEBUG:teuthology.orchestra.run.smithi125:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T12:03:10.366 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T12:03:10.471 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T12:03:10.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:03:10.502 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T12:03:10.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:03:10.539 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-18T12:03:10.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-18T12:03:10.558 INFO:teuthology.task.internal:Duration was 6362.810170 seconds 2023-10-18T12:03:10.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-18T12:03:10.568 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-18T12:03:10.568 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T12:03:10.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T12:03:10.614 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T12:03:10.622 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T12:03:11.019 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-18T12:03:11.020 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi125.front.sepia.ceph.com 2023-10-18T12:03:11.020 DEBUG:teuthology.orchestra.run.smithi125:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T12:03:11.042 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2023-10-18T12:03:11.042 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T12:03:11.064 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-18T12:03:11.065 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T12:03:11.085 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T12:03:11.134 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-18T12:03:11.134 DEBUG:teuthology.orchestra.run.smithi125:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T12:03:11.155 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T12:03:11.340 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-18T12:03:11.351 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-18T12:03:11.352 DEBUG:teuthology.orchestra.run.smithi125:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T12:03:11.380 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T12:03:11.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-18T12:03:11.638 DEBUG:teuthology.orchestra.run.smithi125:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:03:11.641 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:03:11.664 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = core 2023-10-18T12:03:11.667 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2023-10-18T12:03:11.682 DEBUG:teuthology.orchestra.run.smithi125:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:03:11.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:03:11.735 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T12:03:11.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T12:03:11.749 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-18T12:03:11.762 INFO:teuthology.task.internal:Transferring archived files... 2023-10-18T12:03:11.763 DEBUG:teuthology.misc:Transferring archived files from smithi125:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/remote/smithi125 2023-10-18T12:03:11.764 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T12:03:11.811 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772/remote/smithi139 2023-10-18T12:03:11.812 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T12:03:11.848 INFO:teuthology.task.internal:Removing archive directory... 2023-10-18T12:03:11.849 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T12:03:11.852 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T12:03:11.898 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-18T12:03:11.915 INFO:teuthology.task.internal:Not uploading archives. 2023-10-18T12:03:11.916 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-18T12:03:11.926 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-18T12:03:11.926 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T12:03:11.929 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T12:03:11.941 INFO:teuthology.orchestra.run.smithi125.stdout: 265480 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 12:03 /home/ubuntu/cephtest 2023-10-18T12:03:11.952 INFO:teuthology.orchestra.run.smithi139.stdout: 265531 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 12:03 /home/ubuntu/cephtest 2023-10-18T12:03:11.954 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-18T12:03:11.964 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-18T12:03:11.986 INFO:teuthology.nuke:Checking targets against current locks 2023-10-18T12:03:12.013 DEBUG:teuthology.nuke:shortname: smithi125 2023-10-18T12:03:12.014 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T12:03:12.037 DEBUG:teuthology.nuke:shortname: smithi139 2023-10-18T12:03:12.037 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T12:03:12.041 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 10:09:09.842624', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgPjGOGMEwwnQSgDNGZ8Aebsniz3u1sRO9Cd/9KcNxkMp6n9mP36ADoPQ1fAXDMOtBc/tTwQAGL4sR1kzdS10w='} 2023-10-18T12:03:12.060 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430772', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 10:09:09.841463', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbVFAJYQJyVQjPaD1xZQAnsbNlp7gdhA4zpfnjzKe/rNxup6kcxf7nwkxc83xyMAEE+KyWlyHHVeaQnT9zM0VA='} 2023-10-18T12:03:12.093 INFO:teuthology.orchestra.console.smithi125:Power off 2023-10-18T12:03:12.094 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T12:03:12.117 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2023-10-18T12:03:12.118 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:03:12.133 INFO:teuthology.orchestra.console.smithi139:Power off 2023-10-18T12:03:12.134 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T12:03:12.144 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-18T12:03:12.160 DEBUG:teuthology.orchestra.console.smithi139:power off output: Chassis Power Control: Down/Off 2023-10-18T12:03:12.161 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:03:12.184 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2023-10-18T12:03:16.146 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:03:16.186 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:03:16.257 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-18T12:03:16.296 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2023-10-18T12:03:20.262 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:03:20.297 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T12:03:20.374 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-18T12:03:20.408 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is off 2023-10-18T12:03:20.476 INFO:teuthology.orchestra.console.smithi125:Power off completed 2023-10-18T12:03:20.509 INFO:teuthology.orchestra.console.smithi139:Power off completed 2023-10-18T12:03:20.615 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6362.810170173645 failure_reason: 'Test failure: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ac04b0600b0344af80b64f54ca23fe9d status: fail success: false 2023-10-18T12:03:20.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T12:03:20.711 INFO:teuthology.run:FAIL